2024-04-25T00:16:10.663 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T00:16:10.664 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260 --verbose --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260/orig.config.yaml 2024-04-25T00:16:10.709 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:16:10.795 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260 branch: squid description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672260' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi029.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM7bw2OTJadZ4T9ESUbGrYLKBFA6yciK7RQICbMNGNp5tmDdwVix8ruHnJfXtgCqqjQKREAy3w0M94gZlG6d3e0= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbTiAvhgqE7jG79pNqQqg9j60hLL1U0CV4mIIEny+V73yHLRLiZZcJv9SbEVhDHEzTWoYyZa9unNBGk+KyPR8c= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: - /home/teuthology/src/git.ceph.com_ceph_squid/qa/suites/fs/volumes/overrides/pg_health.yaml meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T00:16:10.795 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-25T00:16:10.796 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-25T00:16:10.796 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T00:16:10.798 INFO:teuthology.task.internal:Checking packages... 2024-04-25T00:16:10.824 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-25T00:16:10.824 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T00:16:10.824 INFO:teuthology.packaging:ref: None 2024-04-25T00:16:10.824 INFO:teuthology.packaging:tag: None 2024-04-25T00:16:10.824 INFO:teuthology.packaging:branch: squid 2024-04-25T00:16:10.824 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:16:10.824 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-04-25T00:16:11.118 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:16:11.120 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T00:16:11.127 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T00:16:11.127 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T00:16:11.135 INFO:teuthology.task.internal:Saving configuration 2024-04-25T00:16:11.152 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T00:16:11.158 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T00:16:11.183 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:09:22.642648', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEJJL/QxT6H3NsKjlmmjEBE52hUSJHL2vTL2jf442Sqe'} 2024-04-25T00:16:11.205 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:09:22.644159', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJCDuTz80QzQ9tmJ0WpVMfSR8Dlo71HdNFsoDpVr++x+'} 2024-04-25T00:16:11.205 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T00:16:11.212 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T00:16:11.212 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T00:16:11.212 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T00:16:11.343 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2d1edbcf70>, signals=[15]) 2024-04-25T00:16:11.344 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T00:16:11.349 INFO:teuthology.task.internal:Opening connections... 2024-04-25T00:16:11.350 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:16:11.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:16:11.441 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:16:11.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:16:11.520 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T00:16:11.526 DEBUG:teuthology.orchestra.run.smithi029:> uname -m 2024-04-25T00:16:11.532 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2024-04-25T00:16:11.532 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2024-04-25T00:16:11.581 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T00:16:11.581 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="Ubuntu" 2024-04-25T00:16:11.581 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="22.04" 2024-04-25T00:16:11.581 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T00:16:11.581 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_CODENAME=jammy 2024-04-25T00:16:11.581 INFO:teuthology.orchestra.run.smithi029.stdout:ID=ubuntu 2024-04-25T00:16:11.581 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE=debian 2024-04-25T00:16:11.581 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T00:16:11.581 INFO:teuthology.orchestra.run.smithi029.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T00:16:11.581 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T00:16:11.581 INFO:teuthology.orchestra.run.smithi029.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T00:16:11.581 INFO:teuthology.orchestra.run.smithi029.stdout:UBUNTU_CODENAME=jammy 2024-04-25T00:16:11.582 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2024-04-25T00:16:11.600 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-04-25T00:16:11.604 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-04-25T00:16:11.604 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-04-25T00:16:11.653 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T00:16:11.653 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Ubuntu" 2024-04-25T00:16:11.653 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="22.04" 2024-04-25T00:16:11.653 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T00:16:11.653 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_CODENAME=jammy 2024-04-25T00:16:11.654 INFO:teuthology.orchestra.run.smithi157.stdout:ID=ubuntu 2024-04-25T00:16:11.654 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE=debian 2024-04-25T00:16:11.654 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T00:16:11.654 INFO:teuthology.orchestra.run.smithi157.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T00:16:11.654 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T00:16:11.654 INFO:teuthology.orchestra.run.smithi157.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T00:16:11.654 INFO:teuthology.orchestra.run.smithi157.stdout:UBUNTU_CODENAME=jammy 2024-04-25T00:16:11.654 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-04-25T00:16:11.675 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T00:16:11.684 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T00:16:11.749 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T00:16:11.750 DEBUG:teuthology.orchestra.run.smithi029:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:16:11.753 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:16:11.758 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T00:16:11.805 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T00:16:11.805 DEBUG:teuthology.orchestra.run.smithi029:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:16:11.807 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:16:11.905 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T00:16:11.960 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T00:16:11.974 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-25T00:16:11.974 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-25T00:16:11.975 DEBUG:teuthology.orchestra.run.smithi029:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:16:11.975 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:16:11.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:16:11.980 DEBUG:teuthology.orchestra.run.smithi029:> uname -r 2024-04-25T00:16:11.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:16:11.981 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-04-25T00:16:12.025 INFO:teuthology.orchestra.run.smithi029.stdout:5.15.0-56-generic 2024-04-25T00:16:12.025 INFO:teuthology.task.kernel:Running kernel on smithi029: 5.15.0-56-generic 2024-04-25T00:16:12.025 DEBUG:teuthology.orchestra.run.smithi029:> sudo apt-get clean 2024-04-25T00:16:12.027 INFO:teuthology.orchestra.run.smithi157.stdout:5.15.0-56-generic 2024-04-25T00:16:12.027 INFO:teuthology.task.kernel:Running kernel on smithi157: 5.15.0-56-generic 2024-04-25T00:16:12.027 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get clean 2024-04-25T00:16:12.171 DEBUG:teuthology.orchestra.run.smithi029:> sudo apt-get update 2024-04-25T00:16:12.171 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get update 2024-04-25T00:16:12.306 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:16:12.330 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T00:16:12.375 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T00:16:12.398 INFO:teuthology.orchestra.run.smithi029.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T00:16:12.405 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T00:16:12.463 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:16:12.524 INFO:teuthology.orchestra.run.smithi029.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T00:16:12.553 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T00:16:12.663 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-25T00:16:12.670 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T00:16:12.673 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T00:16:12.673 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T00:16:12.689 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T00:16:12.701 INFO:teuthology.orchestra.run.smithi157.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T00:16:12.701 INFO:teuthology.orchestra.run.smithi157.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T00:16:12.705 INFO:teuthology.orchestra.run.smithi157.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:16:12.705 INFO:teuthology.orchestra.run.smithi157.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T00:16:12.711 INFO:teuthology.orchestra.run.smithi157.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T00:16:12.718 INFO:teuthology.orchestra.run.smithi029.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T00:16:12.722 INFO:teuthology.orchestra.run.smithi157.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T00:16:12.724 INFO:teuthology.orchestra.run.smithi157.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T00:16:12.726 INFO:teuthology.orchestra.run.smithi157.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T00:16:12.759 INFO:teuthology.orchestra.run.smithi157.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T00:16:12.759 INFO:teuthology.orchestra.run.smithi157.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T00:16:12.759 INFO:teuthology.orchestra.run.smithi157.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T00:16:12.760 INFO:teuthology.orchestra.run.smithi157.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T00:16:12.760 INFO:teuthology.orchestra.run.smithi157.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T00:16:12.761 INFO:teuthology.orchestra.run.smithi157.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T00:16:12.762 INFO:teuthology.orchestra.run.smithi157.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T00:16:12.762 INFO:teuthology.orchestra.run.smithi157.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T00:16:12.762 INFO:teuthology.orchestra.run.smithi157.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T00:16:12.780 INFO:teuthology.orchestra.run.smithi157.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T00:16:12.804 INFO:teuthology.orchestra.run.smithi157.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T00:16:12.810 INFO:teuthology.orchestra.run.smithi029.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T00:16:12.813 INFO:teuthology.orchestra.run.smithi157.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T00:16:12.822 INFO:teuthology.orchestra.run.smithi157.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T00:16:12.828 INFO:teuthology.orchestra.run.smithi157.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T00:16:12.828 INFO:teuthology.orchestra.run.smithi157.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T00:16:12.828 INFO:teuthology.orchestra.run.smithi157.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T00:16:12.864 INFO:teuthology.orchestra.run.smithi157.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T00:16:12.872 INFO:teuthology.orchestra.run.smithi157.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:16:12.873 INFO:teuthology.orchestra.run.smithi157.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T00:16:12.877 INFO:teuthology.orchestra.run.smithi029.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T00:16:12.884 INFO:teuthology.orchestra.run.smithi029.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T00:16:12.890 INFO:teuthology.orchestra.run.smithi157.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T00:16:12.898 INFO:teuthology.orchestra.run.smithi029.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T00:16:12.898 INFO:teuthology.orchestra.run.smithi029.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T00:16:12.899 INFO:teuthology.orchestra.run.smithi157.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T00:16:12.901 INFO:teuthology.orchestra.run.smithi157.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T00:16:12.906 INFO:teuthology.orchestra.run.smithi157.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T00:16:12.907 INFO:teuthology.orchestra.run.smithi157.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T00:16:12.907 INFO:teuthology.orchestra.run.smithi157.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T00:16:12.948 INFO:teuthology.orchestra.run.smithi029.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T00:16:12.948 INFO:teuthology.orchestra.run.smithi029.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T00:16:12.949 INFO:teuthology.orchestra.run.smithi157.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T00:16:12.954 INFO:teuthology.orchestra.run.smithi029.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:16:12.954 INFO:teuthology.orchestra.run.smithi029.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T00:16:12.962 INFO:teuthology.orchestra.run.smithi029.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T00:16:13.020 INFO:teuthology.orchestra.run.smithi029.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T00:16:13.020 INFO:teuthology.orchestra.run.smithi029.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-25T00:16:13.021 INFO:teuthology.orchestra.run.smithi029.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T00:16:13.022 INFO:teuthology.orchestra.run.smithi029.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T00:16:13.022 INFO:teuthology.orchestra.run.smithi029.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T00:16:13.022 INFO:teuthology.orchestra.run.smithi029.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T00:16:13.022 INFO:teuthology.orchestra.run.smithi029.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T00:16:13.217 INFO:teuthology.orchestra.run.smithi029.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T00:16:13.367 INFO:teuthology.orchestra.run.smithi029.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T00:16:13.381 INFO:teuthology.orchestra.run.smithi029.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T00:16:13.382 INFO:teuthology.orchestra.run.smithi029.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T00:16:13.384 INFO:teuthology.orchestra.run.smithi029.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T00:16:13.446 INFO:teuthology.orchestra.run.smithi029.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T00:16:13.452 INFO:teuthology.orchestra.run.smithi029.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:16:13.617 INFO:teuthology.orchestra.run.smithi029.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T00:16:14.244 INFO:teuthology.orchestra.run.smithi029.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T00:16:14.281 INFO:teuthology.orchestra.run.smithi029.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T00:16:14.294 INFO:teuthology.orchestra.run.smithi029.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T00:16:14.295 INFO:teuthology.orchestra.run.smithi029.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T00:16:14.296 INFO:teuthology.orchestra.run.smithi029.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T00:16:14.298 INFO:teuthology.orchestra.run.smithi029.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T00:16:14.299 INFO:teuthology.orchestra.run.smithi029.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T00:16:14.299 INFO:teuthology.orchestra.run.smithi029.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T00:16:14.326 INFO:teuthology.orchestra.run.smithi029.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T00:16:14.331 INFO:teuthology.orchestra.run.smithi029.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T00:16:14.496 INFO:teuthology.orchestra.run.smithi029.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T00:16:14.579 INFO:teuthology.orchestra.run.smithi029.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T00:16:14.692 INFO:teuthology.orchestra.run.smithi029.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T00:16:14.750 INFO:teuthology.orchestra.run.smithi029.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T00:16:14.785 INFO:teuthology.orchestra.run.smithi029.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T00:16:18.424 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 13.3 MB in 2s (7,208 kB/s) 2024-04-25T00:16:19.471 INFO:teuthology.orchestra.run.smithi029.stdout:Fetched 13.3 MB in 3s (5,241 kB/s) 2024-04-25T00:16:19.759 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2024-04-25T00:16:19.776 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:16:19.835 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2024-04-25T00:16:20.036 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2024-04-25T00:16:20.036 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2024-04-25T00:16:20.212 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:16:20.212 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:16:20.213 INFO:teuthology.orchestra.run.smithi157.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:16:20.213 INFO:teuthology.orchestra.run.smithi157.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:16:20.213 INFO:teuthology.orchestra.run.smithi157.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T00:16:20.213 INFO:teuthology.orchestra.run.smithi157.stdout: python2.7-minimal 2024-04-25T00:16:20.213 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:16:20.214 INFO:teuthology.orchestra.run.smithi157.stdout:The following additional packages will be installed: 2024-04-25T00:16:20.215 INFO:teuthology.orchestra.run.smithi157.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:16:20.215 INFO:teuthology.orchestra.run.smithi157.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T00:16:20.215 INFO:teuthology.orchestra.run.smithi157.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T00:16:20.216 INFO:teuthology.orchestra.run.smithi157.stdout:Suggested packages: 2024-04-25T00:16:20.216 INFO:teuthology.orchestra.run.smithi157.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T00:16:20.216 INFO:teuthology.orchestra.run.smithi157.stdout:Recommended packages: 2024-04-25T00:16:20.216 INFO:teuthology.orchestra.run.smithi157.stdout: thermald 2024-04-25T00:16:20.270 INFO:teuthology.orchestra.run.smithi157.stdout:The following NEW packages will be installed: 2024-04-25T00:16:20.270 INFO:teuthology.orchestra.run.smithi157.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:16:20.270 INFO:teuthology.orchestra.run.smithi157.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T00:16:20.270 INFO:teuthology.orchestra.run.smithi157.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T00:16:20.271 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages will be upgraded: 2024-04-25T00:16:20.272 INFO:teuthology.orchestra.run.smithi157.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T00:16:20.336 INFO:teuthology.orchestra.run.smithi157.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:16:20.336 INFO:teuthology.orchestra.run.smithi157.stdout:Need to get 113 MB of archives. 2024-04-25T00:16:20.336 INFO:teuthology.orchestra.run.smithi157.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T00:16:20.336 INFO:teuthology.orchestra.run.smithi157.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T00:16:20.860 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T00:16:20.970 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T00:16:21.722 INFO:teuthology.orchestra.run.smithi029.stdout:Reading package lists... 2024-04-25T00:16:21.739 DEBUG:teuthology.orchestra.run.smithi029:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:16:21.791 INFO:teuthology.orchestra.run.smithi029.stdout:Reading package lists... 2024-04-25T00:16:21.973 INFO:teuthology.orchestra.run.smithi029.stdout:Building dependency tree... 2024-04-25T00:16:21.974 INFO:teuthology.orchestra.run.smithi029.stdout:Reading state information... 2024-04-25T00:16:22.043 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T00:16:22.043 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T00:16:22.044 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T00:16:22.138 INFO:teuthology.orchestra.run.smithi029.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:16:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:16:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:16:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:16:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T00:16:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: python2.7-minimal 2024-04-25T00:16:22.139 INFO:teuthology.orchestra.run.smithi029.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:16:22.139 INFO:teuthology.orchestra.run.smithi029.stdout:The following additional packages will be installed: 2024-04-25T00:16:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:16:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T00:16:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T00:16:22.141 INFO:teuthology.orchestra.run.smithi029.stdout:Suggested packages: 2024-04-25T00:16:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T00:16:22.141 INFO:teuthology.orchestra.run.smithi029.stdout:Recommended packages: 2024-04-25T00:16:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: thermald 2024-04-25T00:16:22.235 INFO:teuthology.orchestra.run.smithi029.stdout:The following NEW packages will be installed: 2024-04-25T00:16:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:16:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T00:16:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T00:16:22.237 INFO:teuthology.orchestra.run.smithi029.stdout:The following packages will be upgraded: 2024-04-25T00:16:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T00:16:22.500 INFO:teuthology.orchestra.run.smithi029.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:16:22.500 INFO:teuthology.orchestra.run.smithi029.stdout:Need to get 113 MB of archives. 2024-04-25T00:16:22.500 INFO:teuthology.orchestra.run.smithi029.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T00:16:22.500 INFO:teuthology.orchestra.run.smithi029.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T00:16:22.500 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T00:16:22.574 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T00:16:22.975 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 113 MB in 2s (49.4 MB/s) 2024-04-25T00:16:23.139 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T00:16:23.923 INFO:teuthology.orchestra.run.smithi029.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T00:16:24.261 INFO:teuthology.orchestra.run.smithi029.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T00:16:25.252 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T00:16:25.257 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:16:25.403 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:26.717 INFO:teuthology.orchestra.run.smithi029.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T00:16:26.717 INFO:teuthology.orchestra.run.smithi029.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T00:16:26.717 INFO:teuthology.orchestra.run.smithi029.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T00:16:27.187 INFO:teuthology.orchestra.run.smithi029.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T00:16:27.276 INFO:teuthology.orchestra.run.smithi029.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T00:16:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T00:16:27.509 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:16:27.586 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:27.695 INFO:teuthology.orchestra.run.smithi029.stdout:Fetched 113 MB in 5s (22.5 MB/s) 2024-04-25T00:16:27.911 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T00:16:28.074 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T00:16:28.095 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:16:28.132 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:30.917 INFO:teuthology.orchestra.run.smithi029.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T00:16:30.921 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:16:30.994 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:33.734 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T00:16:33.755 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:16:33.866 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:34.423 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T00:16:34.444 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:16:34.481 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:35.085 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:16:35.218 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:16:35.577 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:16:35.689 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:16:35.975 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T00:16:35.997 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T00:16:36.041 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:16:41.754 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T00:16:41.777 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:16:41.821 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:43.132 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:16:43.259 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:16:43.594 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:16:43.724 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:16:44.054 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T00:16:44.079 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T00:16:44.121 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:16:44.514 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:16:44.639 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:16:44.972 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:16:45.094 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:45.228 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T00:16:45.356 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:46.678 INFO:teuthology.orchestra.run.smithi157.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:16:46.678 INFO:teuthology.orchestra.run.smithi157.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T00:16:46.678 INFO:teuthology.orchestra.run.smithi157.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:16:46.678 INFO:teuthology.orchestra.run.smithi157.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T00:16:46.833 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:48.611 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:49.994 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T00:16:50.128 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T00:16:50.256 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:50.328 INFO:teuthology.orchestra.run.smithi157.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T00:16:50.328 INFO:teuthology.orchestra.run.smithi157.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:16:52.329 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T00:16:52.366 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:16:52.613 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:56.526 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:16:56.652 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:16:57.058 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:16:57.179 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:57.299 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T00:16:57.411 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:16:58.777 INFO:teuthology.orchestra.run.smithi029.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:16:58.777 INFO:teuthology.orchestra.run.smithi029.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T00:16:58.777 INFO:teuthology.orchestra.run.smithi029.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:16:58.777 INFO:teuthology.orchestra.run.smithi029.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T00:16:58.945 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:17:00.357 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:17:01.751 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T00:17:01.879 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T00:17:02.006 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:17:02.057 INFO:teuthology.orchestra.run.smithi029.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T00:17:02.058 INFO:teuthology.orchestra.run.smithi029.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:17:04.888 INFO:teuthology.orchestra.run.smithi157.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T00:17:04.888 INFO:teuthology.orchestra.run.smithi157.stdout:Sourcing file `/etc/default/grub' 2024-04-25T00:17:04.905 INFO:teuthology.orchestra.run.smithi157.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:17:04.929 INFO:teuthology.orchestra.run.smithi157.stdout:Generating grub configuration file ... 2024-04-25T00:17:05.215 INFO:teuthology.orchestra.run.smithi157.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:17:05.239 INFO:teuthology.orchestra.run.smithi157.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:17:05.517 INFO:teuthology.orchestra.run.smithi157.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:17:05.523 INFO:teuthology.orchestra.run.smithi157.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:17:05.590 INFO:teuthology.orchestra.run.smithi157.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:17:05.592 INFO:teuthology.orchestra.run.smithi157.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:17:05.759 INFO:teuthology.orchestra.run.smithi157.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:17:05.759 INFO:teuthology.orchestra.run.smithi157.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:17:05.759 INFO:teuthology.orchestra.run.smithi157.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:17:05.794 INFO:teuthology.orchestra.run.smithi157.stdout:done 2024-04-25T00:17:06.245 DEBUG:teuthology.orchestra.run.smithi157:> dpkg -s linux-image-generic 2024-04-25T00:17:06.265 INFO:teuthology.orchestra.run.smithi157.stdout:Package: linux-image-generic 2024-04-25T00:17:06.265 INFO:teuthology.orchestra.run.smithi157.stdout:Status: install ok installed 2024-04-25T00:17:06.265 INFO:teuthology.orchestra.run.smithi157.stdout:Priority: optional 2024-04-25T00:17:06.265 INFO:teuthology.orchestra.run.smithi157.stdout:Section: kernel 2024-04-25T00:17:06.265 INFO:teuthology.orchestra.run.smithi157.stdout:Installed-Size: 21 2024-04-25T00:17:06.266 INFO:teuthology.orchestra.run.smithi157.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:17:06.266 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: amd64 2024-04-25T00:17:06.266 INFO:teuthology.orchestra.run.smithi157.stdout:Source: linux-meta 2024-04-25T00:17:06.266 INFO:teuthology.orchestra.run.smithi157.stdout:Version: 5.15.0.105.102 2024-04-25T00:17:06.266 INFO:teuthology.orchestra.run.smithi157.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:17:06.266 INFO:teuthology.orchestra.run.smithi157.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:17:06.266 INFO:teuthology.orchestra.run.smithi157.stdout:Recommends: thermald 2024-04-25T00:17:06.266 INFO:teuthology.orchestra.run.smithi157.stdout:Description: Generic Linux kernel image 2024-04-25T00:17:06.266 INFO:teuthology.orchestra.run.smithi157.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:17:06.266 INFO:teuthology.orchestra.run.smithi157.stdout: available. 2024-04-25T00:17:06.266 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T00:17:06.267 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T00:17:06.267 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T00:17:06.267 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-25T00:17:06.267 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T00:17:06.267 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get clean 2024-04-25T00:17:06.453 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get update 2024-04-25T00:17:06.575 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:17:06.685 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:17:06.772 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:17:06.849 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:17:07.981 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2024-04-25T00:17:08.000 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:17:08.053 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2024-04-25T00:17:08.253 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2024-04-25T00:17:08.257 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2024-04-25T00:17:08.415 INFO:teuthology.orchestra.run.smithi157.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T00:17:08.415 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:17:08.415 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:17:08.415 INFO:teuthology.orchestra.run.smithi157.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:17:08.415 INFO:teuthology.orchestra.run.smithi157.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:17:08.415 INFO:teuthology.orchestra.run.smithi157.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:17:08.416 INFO:teuthology.orchestra.run.smithi157.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:17:08.416 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:17:08.453 INFO:teuthology.orchestra.run.smithi157.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:17:08.458 DEBUG:teuthology.orchestra.run.smithi157:> dpkg -s linux-image-generic 2024-04-25T00:17:08.512 INFO:teuthology.orchestra.run.smithi157.stdout:Package: linux-image-generic 2024-04-25T00:17:08.512 INFO:teuthology.orchestra.run.smithi157.stdout:Status: install ok installed 2024-04-25T00:17:08.513 INFO:teuthology.orchestra.run.smithi157.stdout:Priority: optional 2024-04-25T00:17:08.513 INFO:teuthology.orchestra.run.smithi157.stdout:Section: kernel 2024-04-25T00:17:08.513 INFO:teuthology.orchestra.run.smithi157.stdout:Installed-Size: 21 2024-04-25T00:17:08.513 INFO:teuthology.orchestra.run.smithi157.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:17:08.513 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: amd64 2024-04-25T00:17:08.513 INFO:teuthology.orchestra.run.smithi157.stdout:Source: linux-meta 2024-04-25T00:17:08.513 INFO:teuthology.orchestra.run.smithi157.stdout:Version: 5.15.0.105.102 2024-04-25T00:17:08.513 INFO:teuthology.orchestra.run.smithi157.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:17:08.513 INFO:teuthology.orchestra.run.smithi157.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:17:08.513 INFO:teuthology.orchestra.run.smithi157.stdout:Recommends: thermald 2024-04-25T00:17:08.513 INFO:teuthology.orchestra.run.smithi157.stdout:Description: Generic Linux kernel image 2024-04-25T00:17:08.513 INFO:teuthology.orchestra.run.smithi157.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:17:08.513 INFO:teuthology.orchestra.run.smithi157.stdout: available. 2024-04-25T00:17:08.514 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-04-25T00:17:08.557 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.xQ93dIGIjZ 2024-04-25T00:17:08.558 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub/grub.cfg /tmp/tmp.xQ93dIGIjZ 2024-04-25T00:17:08.612 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.xQ93dIGIjZ 2024-04-25T00:17:08.741 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.xQ93dIGIjZ is 10KB 2024-04-25T00:17:08.754 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.xQ93dIGIjZ 2024-04-25T00:17:08.760 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T00:17:08.813 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:17:08.813 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T00:17:08.813 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T00:17:08.881 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T00:17:08.881 DEBUG:teuthology.orchestra.run.smithi157:> sudo update-grub 2024-04-25T00:17:10.033 INFO:teuthology.orchestra.run.smithi157.stderr:Sourcing file `/etc/default/grub' 2024-04-25T00:17:10.034 INFO:teuthology.orchestra.run.smithi157.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:17:10.036 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2024-04-25T00:17:10.211 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:17:10.216 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:17:10.442 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:17:10.447 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:17:10.491 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:17:10.498 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:17:10.616 INFO:teuthology.orchestra.run.smithi157.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:17:10.616 INFO:teuthology.orchestra.run.smithi157.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:17:10.616 INFO:teuthology.orchestra.run.smithi157.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:17:10.634 INFO:teuthology.orchestra.run.smithi157.stderr:done 2024-04-25T00:17:10.652 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2024-04-25T00:17:19.588 INFO:teuthology.orchestra.run.smithi029.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T00:17:19.588 INFO:teuthology.orchestra.run.smithi029.stdout:Sourcing file `/etc/default/grub' 2024-04-25T00:17:19.620 INFO:teuthology.orchestra.run.smithi029.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:17:19.642 INFO:teuthology.orchestra.run.smithi029.stdout:Generating grub configuration file ... 2024-04-25T00:17:19.929 INFO:teuthology.orchestra.run.smithi029.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:17:19.958 INFO:teuthology.orchestra.run.smithi029.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:17:20.238 INFO:teuthology.orchestra.run.smithi029.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:17:20.243 INFO:teuthology.orchestra.run.smithi029.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:17:20.328 INFO:teuthology.orchestra.run.smithi029.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:17:20.333 INFO:teuthology.orchestra.run.smithi029.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:17:20.690 INFO:teuthology.orchestra.run.smithi029.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:17:20.690 INFO:teuthology.orchestra.run.smithi029.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:17:20.690 INFO:teuthology.orchestra.run.smithi029.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:17:20.779 INFO:teuthology.orchestra.run.smithi029.stdout:done 2024-04-25T00:17:21.368 DEBUG:teuthology.orchestra.run.smithi029:> dpkg -s linux-image-generic 2024-04-25T00:17:21.395 INFO:teuthology.orchestra.run.smithi029.stdout:Package: linux-image-generic 2024-04-25T00:17:21.395 INFO:teuthology.orchestra.run.smithi029.stdout:Status: install ok installed 2024-04-25T00:17:21.395 INFO:teuthology.orchestra.run.smithi029.stdout:Priority: optional 2024-04-25T00:17:21.395 INFO:teuthology.orchestra.run.smithi029.stdout:Section: kernel 2024-04-25T00:17:21.395 INFO:teuthology.orchestra.run.smithi029.stdout:Installed-Size: 21 2024-04-25T00:17:21.396 INFO:teuthology.orchestra.run.smithi029.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:17:21.396 INFO:teuthology.orchestra.run.smithi029.stdout:Architecture: amd64 2024-04-25T00:17:21.396 INFO:teuthology.orchestra.run.smithi029.stdout:Source: linux-meta 2024-04-25T00:17:21.396 INFO:teuthology.orchestra.run.smithi029.stdout:Version: 5.15.0.105.102 2024-04-25T00:17:21.396 INFO:teuthology.orchestra.run.smithi029.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:17:21.396 INFO:teuthology.orchestra.run.smithi029.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:17:21.396 INFO:teuthology.orchestra.run.smithi029.stdout:Recommends: thermald 2024-04-25T00:17:21.396 INFO:teuthology.orchestra.run.smithi029.stdout:Description: Generic Linux kernel image 2024-04-25T00:17:21.396 INFO:teuthology.orchestra.run.smithi029.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:17:21.396 INFO:teuthology.orchestra.run.smithi029.stdout: available. 2024-04-25T00:17:21.397 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T00:17:21.397 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T00:17:21.397 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T00:17:21.397 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T00:17:21.397 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi029.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T00:17:21.397 DEBUG:teuthology.orchestra.run.smithi029:> sudo apt-get clean 2024-04-25T00:17:21.550 DEBUG:teuthology.orchestra.run.smithi029:> sudo apt-get update 2024-04-25T00:17:21.672 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:17:21.678 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:17:21.681 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:17:21.699 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:17:23.126 INFO:teuthology.orchestra.run.smithi029.stdout:Reading package lists... 2024-04-25T00:17:23.143 DEBUG:teuthology.orchestra.run.smithi029:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:17:23.194 INFO:teuthology.orchestra.run.smithi029.stdout:Reading package lists... 2024-04-25T00:17:23.376 INFO:teuthology.orchestra.run.smithi029.stdout:Building dependency tree... 2024-04-25T00:17:23.377 INFO:teuthology.orchestra.run.smithi029.stdout:Reading state information... 2024-04-25T00:17:23.541 INFO:teuthology.orchestra.run.smithi029.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T00:17:23.541 INFO:teuthology.orchestra.run.smithi029.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:17:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:17:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:17:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:17:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:17:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:17:23.542 INFO:teuthology.orchestra.run.smithi029.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:17:23.580 INFO:teuthology.orchestra.run.smithi029.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:17:23.582 DEBUG:teuthology.orchestra.run.smithi029:> dpkg -s linux-image-generic 2024-04-25T00:17:23.636 INFO:teuthology.orchestra.run.smithi029.stdout:Package: linux-image-generic 2024-04-25T00:17:23.636 INFO:teuthology.orchestra.run.smithi029.stdout:Status: install ok installed 2024-04-25T00:17:23.637 INFO:teuthology.orchestra.run.smithi029.stdout:Priority: optional 2024-04-25T00:17:23.637 INFO:teuthology.orchestra.run.smithi029.stdout:Section: kernel 2024-04-25T00:17:23.637 INFO:teuthology.orchestra.run.smithi029.stdout:Installed-Size: 21 2024-04-25T00:17:23.637 INFO:teuthology.orchestra.run.smithi029.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:17:23.637 INFO:teuthology.orchestra.run.smithi029.stdout:Architecture: amd64 2024-04-25T00:17:23.637 INFO:teuthology.orchestra.run.smithi029.stdout:Source: linux-meta 2024-04-25T00:17:23.637 INFO:teuthology.orchestra.run.smithi029.stdout:Version: 5.15.0.105.102 2024-04-25T00:17:23.637 INFO:teuthology.orchestra.run.smithi029.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:17:23.637 INFO:teuthology.orchestra.run.smithi029.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:17:23.637 INFO:teuthology.orchestra.run.smithi029.stdout:Recommends: thermald 2024-04-25T00:17:23.638 INFO:teuthology.orchestra.run.smithi029.stdout:Description: Generic Linux kernel image 2024-04-25T00:17:23.638 INFO:teuthology.orchestra.run.smithi029.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:17:23.638 INFO:teuthology.orchestra.run.smithi029.stdout: available. 2024-04-25T00:17:23.638 DEBUG:teuthology.orchestra.run.smithi029:> mktemp 2024-04-25T00:17:23.684 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/tmp.9iume5dMsm 2024-04-25T00:17:23.684 DEBUG:teuthology.orchestra.run.smithi029:> sudo cp /boot/grub/grub.cfg /tmp/tmp.9iume5dMsm 2024-04-25T00:17:23.739 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0666 /tmp/tmp.9iume5dMsm 2024-04-25T00:17:23.861 DEBUG:teuthology.orchestra.remote:smithi029:/tmp/tmp.9iume5dMsm is 10KB 2024-04-25T00:17:23.874 DEBUG:teuthology.orchestra.run.smithi029:> rm -fr /tmp/tmp.9iume5dMsm 2024-04-25T00:17:23.879 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T00:17:23.932 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:17:23.932 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T00:17:23.932 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T00:17:24.026 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T00:17:24.027 DEBUG:teuthology.orchestra.run.smithi029:> sudo update-grub 2024-04-25T00:17:26.366 INFO:teuthology.orchestra.run.smithi029.stderr:Sourcing file `/etc/default/grub' 2024-04-25T00:17:26.377 INFO:teuthology.orchestra.run.smithi029.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:17:26.391 INFO:teuthology.orchestra.run.smithi029.stderr:Generating grub configuration file ... 2024-04-25T00:17:26.611 INFO:teuthology.orchestra.run.smithi029.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:17:26.618 INFO:teuthology.orchestra.run.smithi029.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:17:26.855 INFO:teuthology.orchestra.run.smithi029.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:17:26.861 INFO:teuthology.orchestra.run.smithi029.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:17:26.904 INFO:teuthology.orchestra.run.smithi029.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:17:26.910 INFO:teuthology.orchestra.run.smithi029.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:17:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:17:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:17:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:17:27.058 INFO:teuthology.orchestra.run.smithi029.stderr:done 2024-04-25T00:17:27.068 DEBUG:teuthology.orchestra.run.smithi029:> sudo shutdown -r now 2024-04-25T00:17:40.666 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-25T00:17:40.667 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-25T00:17:40.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:17:57.086 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T00:17:57.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-04-25T00:17:57.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:18:15.199 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-04-25T00:18:24.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-25T00:18:24.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:18:27.266 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-04-25T00:18:39.278 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-25T00:18:39.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:18:57.091 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T00:19:06.101 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-04-25T00:19:06.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:19:22.853 DEBUG:teuthology.orchestra.run.smithi029:> true 2024-04-25T00:19:24.605 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-04-25T00:19:24.605 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-25T00:19:24.606 DEBUG:teuthology.orchestra.run.smithi029:> uname -r 2024-04-25T00:19:24.611 INFO:teuthology.orchestra.run.smithi029.stdout:5.15.0-105-generic 2024-04-25T00:19:24.612 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T00:19:24.612 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T00:19:24.612 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T00:19:25.612 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T00:19:25.613 DEBUG:teuthology.orchestra.run.smithi029:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T00:19:25.976 INFO:teuthology.orchestra.run.smithi029.stdout:ttyS1 2024-04-25T00:19:26.006 DEBUG:teuthology.parallel:result is None 2024-04-25T00:19:39.281 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T00:19:54.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-25T00:19:54.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:19:54.793 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-04-25T00:19:55.333 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-25T00:19:55.333 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-25T00:19:55.333 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-04-25T00:19:55.378 INFO:teuthology.orchestra.run.smithi157.stdout:5.15.0-105-generic 2024-04-25T00:19:55.378 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T00:19:55.378 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T00:19:55.378 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T00:19:56.379 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T00:19:56.379 DEBUG:teuthology.orchestra.run.smithi157:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T00:19:56.719 INFO:teuthology.orchestra.run.smithi157.stdout:ttyS1 2024-04-25T00:19:56.760 DEBUG:teuthology.parallel:result is None 2024-04-25T00:19:56.760 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T00:19:56.800 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T00:19:56.800 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:19:56.803 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:19:56.809 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T00:19:56.816 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T00:19:56.837 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T00:19:56.837 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:19:56.852 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:19:56.882 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T00:19:56.889 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T00:19:56.890 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:19:56.916 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:19:56.933 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:19:56.940 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:19:56.942 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:19:56.948 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:19:56.949 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T00:19:56.957 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T00:19:56.957 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:19:56.992 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:19:57.006 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T00:19:57.015 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T00:19:57.015 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:19:57.052 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:19:57.057 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:19:57.103 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:19:57.151 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:19:57.152 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:19:57.225 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:19:57.231 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:19:57.278 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:19:57.278 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:19:57.349 DEBUG:teuthology.orchestra.run.smithi029:> sudo service rsyslog restart 2024-04-25T00:19:57.352 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-04-25T00:19:57.425 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T00:19:57.433 INFO:teuthology.task.internal:Starting timer... 2024-04-25T00:19:57.433 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T00:19:57.495 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T00:19:57.557 DEBUG:teuthology.task.selinux:Excluding smithi029: OS 'ubuntu' does not support SELinux 2024-04-25T00:19:57.557 DEBUG:teuthology.task.selinux:Excluding smithi157: OS 'ubuntu' does not support SELinux 2024-04-25T00:19:57.558 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T00:19:57.558 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T00:19:57.558 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T00:19:57.558 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T00:19:57.572 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T00:19:57.580 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T00:19:57.695 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T00:19:57.712 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T00:19:57.714 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi029.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T00:27:21.225 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi029.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-04-25T00:27:21.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-04-25T00:27:21.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:27:21.314 DEBUG:teuthology.orchestra.run.smithi029:> true 2024-04-25T00:27:21.394 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-04-25T00:27:21.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-25T00:27:21.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:27:21.474 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-04-25T00:27:21.550 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-25T00:27:21.551 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T00:27:21.562 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T00:27:21.562 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:27:21.563 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:27:21.565 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:27:21.565 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:27:21.595 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T00:27:21.595 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: Command line: ntpd -gq 2024-04-25T00:27:21.595 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: ---------------------------------------------------- 2024-04-25T00:27:21.595 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T00:27:21.595 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T00:27:21.595 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: corporation. Support and training for ntp-4 are 2024-04-25T00:27:21.595 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: available at https://www.nwtime.org/support 2024-04-25T00:27:21.596 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: ---------------------------------------------------- 2024-04-25T00:27:21.596 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: proto: precision = 0.069 usec (-24) 2024-04-25T00:27:21.596 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: basedate set to 2022-02-04 2024-04-25T00:27:21.596 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: gps base set to 2022-02-06 (week 2196) 2024-04-25T00:27:21.596 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:27:21.597 INFO:teuthology.orchestra.run.smithi029.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:27:21.597 INFO:teuthology.orchestra.run.smithi029.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:27:21.598 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:27:21.598 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T00:27:21.598 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T00:27:21.598 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T00:27:21.598 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: Listen normally on 3 ens1f0 172.21.15.29:123 2024-04-25T00:27:21.598 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: Listen normally on 4 lo [::1]:123 2024-04-25T00:27:21.598 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:125a%4]:123 2024-04-25T00:27:21.598 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:21 ntpd[17196]: Listening on routing socket on fd #22 for interface updates 2024-04-25T00:27:21.616 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T00:27:21.617 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: Command line: ntpd -gq 2024-04-25T00:27:21.617 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: ---------------------------------------------------- 2024-04-25T00:27:21.617 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T00:27:21.617 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T00:27:21.617 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: corporation. Support and training for ntp-4 are 2024-04-25T00:27:21.617 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: available at https://www.nwtime.org/support 2024-04-25T00:27:21.617 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: ---------------------------------------------------- 2024-04-25T00:27:21.617 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: proto: precision = 0.045 usec (-24) 2024-04-25T00:27:21.618 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: basedate set to 2022-02-04 2024-04-25T00:27:21.618 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: gps base set to 2022-02-06 (week 2196) 2024-04-25T00:27:21.618 INFO:teuthology.orchestra.run.smithi157.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:27:21.618 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:27:21.619 INFO:teuthology.orchestra.run.smithi157.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:27:21.620 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:27:21.620 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T00:27:21.620 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T00:27:21.620 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T00:27:21.620 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: Listen normally on 3 enp3s0f1 172.21.15.157:123 2024-04-25T00:27:21.620 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: Listen normally on 4 lo [::1]:123 2024-04-25T00:27:21.620 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7ea1%5]:123 2024-04-25T00:27:21.620 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:21 ntpd[17231]: Listening on routing socket on fd #22 for interface updates 2024-04-25T00:27:22.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:22 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:22.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:22 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:22.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:22 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:22.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:22 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:22.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:22 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:22.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:22 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:22.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:22 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:22.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:22 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:22.619 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:22.619 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:22.619 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:22 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:23.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:23 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:23.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:23 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:23.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:23 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:23.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:23 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:23.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:23 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:23.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:23 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:24 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:24 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:24 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:24 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:24 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:24 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:24 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:24 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:24 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:24 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:24 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:24 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:24.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:24 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:24.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:24 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:24.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:24 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:24.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:24 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:24.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:24 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:24.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:24 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:24.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:24 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:24.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:24 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:24.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:24 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:24.619 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:24 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:24.619 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:24 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:24.619 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:24 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:25.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:25 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:25.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:25 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:25.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:25 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:25.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:25 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:25.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:25 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:25.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:25 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:26 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:26.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:26 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:26.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:26 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:26.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:26 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:26.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:26 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:26.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:26 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:26.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:26 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:26.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:26 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:26.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:26 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:26.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:26 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:26.619 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:26 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:26.619 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:26 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:26.619 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:26 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:27 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:27 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:27 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:27 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:27.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:27 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:27.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:27 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:28.600 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:28.600 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:28.600 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:28.600 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:28.600 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:28.600 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:28.600 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:28.600 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:28.600 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:28 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:28.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:28 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:28.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:28 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:28.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:28 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:28.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:28 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:28.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:28 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:28.617 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:28 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:28.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:28 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:28.618 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:28 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:28.618 INFO:teuthology.orchestra.run.smithi157.stdout:25 Apr 00:27:28 ntpd[17231]: ntpd: time slew -0.000372 s 2024-04-25T00:27:28.618 INFO:teuthology.orchestra.run.smithi157.stdout:ntpd: time slew -0.000372s 2024-04-25T00:27:28.619 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:28 ntpd[17231]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T00:27:28.619 INFO:teuthology.orchestra.run.smithi157.stderr:25 Apr 00:27:28 ntpd[17231]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T00:27:28.667 INFO:teuthology.orchestra.run.smithi157.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:27:28.667 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================== 2024-04-25T00:27:28.667 INFO:teuthology.orchestra.run.smithi157.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:27:28.667 INFO:teuthology.orchestra.run.smithi157.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:27:28.667 INFO:teuthology.orchestra.run.smithi157.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:27:28.667 INFO:teuthology.orchestra.run.smithi157.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:27:30.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:30 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:30.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:30 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:30.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:30 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:30.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:30 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:30.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:30 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:30.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:30 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:30.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:30 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:27:30.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:30 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:27:30.596 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:30 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:27:30.597 INFO:teuthology.orchestra.run.smithi029.stdout:25 Apr 00:27:30 ntpd[17196]: ntpd: time slew -0.000637 s 2024-04-25T00:27:30.597 INFO:teuthology.orchestra.run.smithi029.stdout:ntpd: time slew -0.000637s 2024-04-25T00:27:30.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:30 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:27:30.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:30 ntpd[17196]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T00:27:30.597 INFO:teuthology.orchestra.run.smithi029.stderr:25 Apr 00:27:30 ntpd[17196]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T00:27:30.656 INFO:teuthology.orchestra.run.smithi029.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:27:30.656 INFO:teuthology.orchestra.run.smithi029.stdout:============================================================================== 2024-04-25T00:27:30.656 INFO:teuthology.orchestra.run.smithi029.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:27:30.657 INFO:teuthology.orchestra.run.smithi029.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:27:30.657 INFO:teuthology.orchestra.run.smithi029.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:27:30.657 INFO:teuthology.orchestra.run.smithi029.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:27:30.658 INFO:teuthology.run_tasks:Running task install... 2024-04-25T00:27:30.667 DEBUG:teuthology.task.install:project ceph 2024-04-25T00:27:30.667 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T00:27:30.667 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T00:27:30.667 INFO:teuthology.task.install:Using flavor: default 2024-04-25T00:27:30.673 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T00:27:30.673 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T00:27:30.674 DEBUG:teuthology.orchestra.run.smithi029:> sudo apt-key list | grep Ceph 2024-04-25T00:27:30.674 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-key list | grep Ceph 2024-04-25T00:27:30.736 INFO:teuthology.orchestra.run.smithi157.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T00:27:30.763 INFO:teuthology.orchestra.run.smithi029.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T00:27:30.765 INFO:teuthology.orchestra.run.smithi157.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T00:27:30.765 INFO:teuthology.orchestra.run.smithi157.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T00:27:30.766 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T00:27:30.766 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T00:27:30.766 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:27:30.793 INFO:teuthology.orchestra.run.smithi029.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T00:27:30.794 INFO:teuthology.orchestra.run.smithi029.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T00:27:30.795 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T00:27:30.795 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T00:27:30.795 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:27:30.987 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default/ 2024-04-25T00:27:30.987 INFO:teuthology.task.install.deb:Package version is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:27:31.010 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default/ 2024-04-25T00:27:31.010 INFO:teuthology.task.install.deb:Package version is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:27:31.155 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:27:31.155 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T00:27:31.169 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get update 2024-04-25T00:27:31.174 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:27:31.174 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T00:27:31.188 DEBUG:teuthology.orchestra.run.smithi029:> sudo apt-get update 2024-04-25T00:27:31.342 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:27:31.356 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:27:31.358 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:27:31.364 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:27:31.364 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:27:31.375 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:27:31.376 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:27:31.387 INFO:teuthology.orchestra.run.smithi157.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T00:27:31.396 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:27:31.409 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T00:27:31.430 INFO:teuthology.orchestra.run.smithi157.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T00:27:31.451 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T00:27:31.475 INFO:teuthology.orchestra.run.smithi029.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T00:27:31.478 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-04-25T00:27:31.512 INFO:teuthology.orchestra.run.smithi029.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T00:27:31.552 INFO:teuthology.orchestra.run.smithi029.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T00:27:31.588 INFO:teuthology.orchestra.run.smithi029.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-04-25T00:27:31.632 INFO:teuthology.orchestra.run.smithi029.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T00:27:31.679 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 28.9 kB in 0s (86.6 kB/s) 2024-04-25T00:27:31.745 INFO:teuthology.orchestra.run.smithi029.stdout:Fetched 28.9 kB in 0s (76.7 kB/s) 2024-04-25T00:27:32.662 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2024-04-25T00:27:32.679 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2445-gd2c45d97-1jammy cephadm=19.0.0-2445-gd2c45d97-1jammy ceph-mds=19.0.0-2445-gd2c45d97-1jammy ceph-mgr=19.0.0-2445-gd2c45d97-1jammy ceph-common=19.0.0-2445-gd2c45d97-1jammy ceph-fuse=19.0.0-2445-gd2c45d97-1jammy ceph-test=19.0.0-2445-gd2c45d97-1jammy ceph-volume=19.0.0-2445-gd2c45d97-1jammy radosgw=19.0.0-2445-gd2c45d97-1jammy python3-rados=19.0.0-2445-gd2c45d97-1jammy python3-rgw=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy python3-rbd=19.0.0-2445-gd2c45d97-1jammy libcephfs2=19.0.0-2445-gd2c45d97-1jammy libcephfs-dev=19.0.0-2445-gd2c45d97-1jammy librados2=19.0.0-2445-gd2c45d97-1jammy librbd1=19.0.0-2445-gd2c45d97-1jammy rbd-fuse=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy cephfs-shell=19.0.0-2445-gd2c45d97-1jammy cephfs-top=19.0.0-2445-gd2c45d97-1jammy cephfs-mirror=19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:27:32.734 INFO:teuthology.orchestra.run.smithi029.stdout:Reading package lists... 2024-04-25T00:27:32.739 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2024-04-25T00:27:32.751 DEBUG:teuthology.orchestra.run.smithi029:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2445-gd2c45d97-1jammy cephadm=19.0.0-2445-gd2c45d97-1jammy ceph-mds=19.0.0-2445-gd2c45d97-1jammy ceph-mgr=19.0.0-2445-gd2c45d97-1jammy ceph-common=19.0.0-2445-gd2c45d97-1jammy ceph-fuse=19.0.0-2445-gd2c45d97-1jammy ceph-test=19.0.0-2445-gd2c45d97-1jammy ceph-volume=19.0.0-2445-gd2c45d97-1jammy radosgw=19.0.0-2445-gd2c45d97-1jammy python3-rados=19.0.0-2445-gd2c45d97-1jammy python3-rgw=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy python3-rbd=19.0.0-2445-gd2c45d97-1jammy libcephfs2=19.0.0-2445-gd2c45d97-1jammy libcephfs-dev=19.0.0-2445-gd2c45d97-1jammy librados2=19.0.0-2445-gd2c45d97-1jammy librbd1=19.0.0-2445-gd2c45d97-1jammy rbd-fuse=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy cephfs-shell=19.0.0-2445-gd2c45d97-1jammy cephfs-top=19.0.0-2445-gd2c45d97-1jammy cephfs-mirror=19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:27:32.814 INFO:teuthology.orchestra.run.smithi029.stdout:Reading package lists... 2024-04-25T00:27:32.939 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2024-04-25T00:27:32.940 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2024-04-25T00:27:33.017 INFO:teuthology.orchestra.run.smithi029.stdout:Building dependency tree... 2024-04-25T00:27:33.017 INFO:teuthology.orchestra.run.smithi029.stdout:Reading state information... 2024-04-25T00:27:33.133 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:27:33.134 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:27:33.134 INFO:teuthology.orchestra.run.smithi157.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:27:33.134 INFO:teuthology.orchestra.run.smithi157.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:27:33.134 INFO:teuthology.orchestra.run.smithi157.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:27:33.135 INFO:teuthology.orchestra.run.smithi157.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:27:33.135 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:27:33.135 INFO:teuthology.orchestra.run.smithi157.stdout:The following additional packages will be installed: 2024-04-25T00:27:33.135 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T00:27:33.135 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T00:27:33.135 INFO:teuthology.orchestra.run.smithi157.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:27:33.136 INFO:teuthology.orchestra.run.smithi157.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T00:27:33.137 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T00:27:33.137 INFO:teuthology.orchestra.run.smithi157.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T00:27:33.137 INFO:teuthology.orchestra.run.smithi157.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T00:27:33.137 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T00:27:33.137 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T00:27:33.137 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T00:27:33.137 INFO:teuthology.orchestra.run.smithi157.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T00:27:33.137 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T00:27:33.137 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T00:27:33.137 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T00:27:33.137 INFO:teuthology.orchestra.run.smithi157.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T00:27:33.138 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T00:27:33.138 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T00:27:33.138 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T00:27:33.138 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T00:27:33.138 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T00:27:33.138 INFO:teuthology.orchestra.run.smithi157.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T00:27:33.138 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T00:27:33.138 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T00:27:33.138 INFO:teuthology.orchestra.run.smithi157.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T00:27:33.138 INFO:teuthology.orchestra.run.smithi157.stdout:Suggested packages: 2024-04-25T00:27:33.139 INFO:teuthology.orchestra.run.smithi157.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T00:27:33.139 INFO:teuthology.orchestra.run.smithi157.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T00:27:33.139 INFO:teuthology.orchestra.run.smithi157.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T00:27:33.139 INFO:teuthology.orchestra.run.smithi157.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T00:27:33.139 INFO:teuthology.orchestra.run.smithi157.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T00:27:33.139 INFO:teuthology.orchestra.run.smithi157.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T00:27:33.139 INFO:teuthology.orchestra.run.smithi157.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T00:27:33.139 INFO:teuthology.orchestra.run.smithi157.stdout:Recommended packages: 2024-04-25T00:27:33.139 INFO:teuthology.orchestra.run.smithi157.stdout: btrfs-tools 2024-04-25T00:27:33.218 INFO:teuthology.orchestra.run.smithi029.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:27:33.218 INFO:teuthology.orchestra.run.smithi029.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:27:33.218 INFO:teuthology.orchestra.run.smithi029.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:27:33.218 INFO:teuthology.orchestra.run.smithi029.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:27:33.219 INFO:teuthology.orchestra.run.smithi029.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:27:33.219 INFO:teuthology.orchestra.run.smithi029.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:27:33.219 INFO:teuthology.orchestra.run.smithi029.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:27:33.219 INFO:teuthology.orchestra.run.smithi029.stdout:The following additional packages will be installed: 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi029.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi029.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi029.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi029.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi029.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi029.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi029.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi029.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi029.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T00:27:33.223 INFO:teuthology.orchestra.run.smithi029.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T00:27:33.223 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T00:27:33.223 INFO:teuthology.orchestra.run.smithi029.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T00:27:33.223 INFO:teuthology.orchestra.run.smithi029.stdout:Suggested packages: 2024-04-25T00:27:33.223 INFO:teuthology.orchestra.run.smithi029.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T00:27:33.223 INFO:teuthology.orchestra.run.smithi029.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T00:27:33.223 INFO:teuthology.orchestra.run.smithi029.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T00:27:33.223 INFO:teuthology.orchestra.run.smithi029.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T00:27:33.224 INFO:teuthology.orchestra.run.smithi029.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T00:27:33.224 INFO:teuthology.orchestra.run.smithi029.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T00:27:33.224 INFO:teuthology.orchestra.run.smithi029.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T00:27:33.224 INFO:teuthology.orchestra.run.smithi029.stdout:Recommended packages: 2024-04-25T00:27:33.224 INFO:teuthology.orchestra.run.smithi029.stdout: btrfs-tools 2024-04-25T00:27:33.246 INFO:teuthology.orchestra.run.smithi157.stdout:The following NEW packages will be installed: 2024-04-25T00:27:33.246 INFO:teuthology.orchestra.run.smithi157.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T00:27:33.246 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T00:27:33.246 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T00:27:33.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T00:27:33.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T00:27:33.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T00:27:33.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T00:27:33.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T00:27:33.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T00:27:33.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T00:27:33.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T00:27:33.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T00:27:33.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T00:27:33.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T00:27:33.248 INFO:teuthology.orchestra.run.smithi157.stdout: socat unzip xmlstarlet zip 2024-04-25T00:27:33.312 INFO:teuthology.orchestra.run.smithi157.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T00:27:33.312 INFO:teuthology.orchestra.run.smithi157.stdout:Need to get 180 MB of archives. 2024-04-25T00:27:33.312 INFO:teuthology.orchestra.run.smithi157.stdout:After this operation, 823 MB of additional disk space will be used. 2024-04-25T00:27:33.312 INFO:teuthology.orchestra.run.smithi157.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T00:27:33.333 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T00:27:33.335 INFO:teuthology.orchestra.run.smithi029.stdout:The following NEW packages will be installed: 2024-04-25T00:27:33.335 INFO:teuthology.orchestra.run.smithi029.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T00:27:33.336 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T00:27:33.336 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T00:27:33.338 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T00:27:33.338 INFO:teuthology.orchestra.run.smithi029.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:27:33.338 INFO:teuthology.orchestra.run.smithi029.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T00:27:33.338 INFO:teuthology.orchestra.run.smithi029.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T00:27:33.338 INFO:teuthology.orchestra.run.smithi029.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T00:27:33.338 INFO:teuthology.orchestra.run.smithi029.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T00:27:33.338 INFO:teuthology.orchestra.run.smithi029.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T00:27:33.338 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T00:27:33.338 INFO:teuthology.orchestra.run.smithi029.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T00:27:33.339 INFO:teuthology.orchestra.run.smithi029.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T00:27:33.340 INFO:teuthology.orchestra.run.smithi029.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T00:27:33.340 INFO:teuthology.orchestra.run.smithi029.stdout: socat unzip xmlstarlet zip 2024-04-25T00:27:33.344 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T00:27:33.388 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2445-gd2c45d97-1jammy [3,540 kB] 2024-04-25T00:27:33.393 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T00:27:33.406 INFO:teuthology.orchestra.run.smithi029.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T00:27:33.406 INFO:teuthology.orchestra.run.smithi029.stdout:Need to get 180 MB of archives. 2024-04-25T00:27:33.406 INFO:teuthology.orchestra.run.smithi029.stdout:After this operation, 823 MB of additional disk space will be used. 2024-04-25T00:27:33.407 INFO:teuthology.orchestra.run.smithi029.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T00:27:33.419 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T00:27:33.422 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T00:27:33.428 INFO:teuthology.orchestra.run.smithi029.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T00:27:33.436 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T00:27:33.439 INFO:teuthology.orchestra.run.smithi029.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T00:27:33.489 INFO:teuthology.orchestra.run.smithi029.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T00:27:33.493 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T00:27:33.498 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T00:27:33.509 INFO:teuthology.orchestra.run.smithi157.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T00:27:33.513 INFO:teuthology.orchestra.run.smithi157.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T00:27:33.513 INFO:teuthology.orchestra.run.smithi157.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T00:27:33.514 INFO:teuthology.orchestra.run.smithi029.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T00:27:33.518 INFO:teuthology.orchestra.run.smithi029.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T00:27:33.521 INFO:teuthology.orchestra.run.smithi157.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T00:27:33.523 INFO:teuthology.orchestra.run.smithi157.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T00:27:33.525 INFO:teuthology.orchestra.run.smithi157.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T00:27:33.526 INFO:teuthology.orchestra.run.smithi157.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T00:27:33.526 INFO:teuthology.orchestra.run.smithi157.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T00:27:33.533 INFO:teuthology.orchestra.run.smithi029.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T00:27:33.538 INFO:teuthology.orchestra.run.smithi157.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T00:27:33.541 INFO:teuthology.orchestra.run.smithi157.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T00:27:33.543 INFO:teuthology.orchestra.run.smithi157.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T00:27:33.543 INFO:teuthology.orchestra.run.smithi157.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T00:27:33.559 INFO:teuthology.orchestra.run.smithi157.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T00:27:33.560 INFO:teuthology.orchestra.run.smithi157.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T00:27:33.561 INFO:teuthology.orchestra.run.smithi157.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T00:27:33.561 INFO:teuthology.orchestra.run.smithi157.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T00:27:33.561 INFO:teuthology.orchestra.run.smithi029.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2445-gd2c45d97-1jammy [3,540 kB] 2024-04-25T00:27:33.561 INFO:teuthology.orchestra.run.smithi157.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T00:27:33.562 INFO:teuthology.orchestra.run.smithi157.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T00:27:33.580 INFO:teuthology.orchestra.run.smithi157.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T00:27:33.580 INFO:teuthology.orchestra.run.smithi157.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T00:27:33.583 INFO:teuthology.orchestra.run.smithi157.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T00:27:33.583 INFO:teuthology.orchestra.run.smithi157.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T00:27:33.588 INFO:teuthology.orchestra.run.smithi029.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T00:27:33.593 INFO:teuthology.orchestra.run.smithi029.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T00:27:33.605 INFO:teuthology.orchestra.run.smithi157.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T00:27:33.605 INFO:teuthology.orchestra.run.smithi157.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T00:27:33.605 INFO:teuthology.orchestra.run.smithi029.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T00:27:33.606 INFO:teuthology.orchestra.run.smithi157.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T00:27:33.607 INFO:teuthology.orchestra.run.smithi157.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T00:27:33.607 INFO:teuthology.orchestra.run.smithi157.stdout:Get:37 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2445-gd2c45d97-1jammy [3,218 kB] 2024-04-25T00:27:33.607 INFO:teuthology.orchestra.run.smithi157.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T00:27:33.608 INFO:teuthology.orchestra.run.smithi157.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T00:27:33.610 INFO:teuthology.orchestra.run.smithi029.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T00:27:33.610 INFO:teuthology.orchestra.run.smithi029.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T00:27:33.619 INFO:teuthology.orchestra.run.smithi029.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T00:27:33.621 INFO:teuthology.orchestra.run.smithi029.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T00:27:33.623 INFO:teuthology.orchestra.run.smithi029.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T00:27:33.625 INFO:teuthology.orchestra.run.smithi029.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T00:27:33.625 INFO:teuthology.orchestra.run.smithi029.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T00:27:33.625 INFO:teuthology.orchestra.run.smithi157.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T00:27:33.626 INFO:teuthology.orchestra.run.smithi157.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T00:27:33.631 INFO:teuthology.orchestra.run.smithi157.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T00:27:33.631 INFO:teuthology.orchestra.run.smithi157.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T00:27:33.634 INFO:teuthology.orchestra.run.smithi029.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T00:27:33.636 INFO:teuthology.orchestra.run.smithi029.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T00:27:33.638 INFO:teuthology.orchestra.run.smithi029.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T00:27:33.646 INFO:teuthology.orchestra.run.smithi157.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T00:27:33.646 INFO:teuthology.orchestra.run.smithi157.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T00:27:33.647 INFO:teuthology.orchestra.run.smithi157.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T00:27:33.650 INFO:teuthology.orchestra.run.smithi157.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T00:27:33.650 INFO:teuthology.orchestra.run.smithi157.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T00:27:33.651 INFO:teuthology.orchestra.run.smithi157.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T00:27:33.655 INFO:teuthology.orchestra.run.smithi029.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T00:27:33.655 INFO:teuthology.orchestra.run.smithi029.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T00:27:33.656 INFO:teuthology.orchestra.run.smithi029.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T00:27:33.657 INFO:teuthology.orchestra.run.smithi029.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T00:27:33.657 INFO:teuthology.orchestra.run.smithi029.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T00:27:33.658 INFO:teuthology.orchestra.run.smithi029.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T00:27:33.658 INFO:teuthology.orchestra.run.smithi029.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T00:27:33.666 INFO:teuthology.orchestra.run.smithi157.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T00:27:33.668 INFO:teuthology.orchestra.run.smithi157.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T00:27:33.669 INFO:teuthology.orchestra.run.smithi157.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T00:27:33.670 INFO:teuthology.orchestra.run.smithi157.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T00:27:33.677 INFO:teuthology.orchestra.run.smithi029.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T00:27:33.677 INFO:teuthology.orchestra.run.smithi029.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T00:27:33.680 INFO:teuthology.orchestra.run.smithi029.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T00:27:33.687 INFO:teuthology.orchestra.run.smithi157.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T00:27:33.713 INFO:teuthology.orchestra.run.smithi029.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T00:27:33.714 INFO:teuthology.orchestra.run.smithi029.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T00:27:33.715 INFO:teuthology.orchestra.run.smithi029.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T00:27:33.717 INFO:teuthology.orchestra.run.smithi029.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T00:27:33.718 INFO:teuthology.orchestra.run.smithi029.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T00:27:33.718 INFO:teuthology.orchestra.run.smithi029.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T00:27:33.721 INFO:teuthology.orchestra.run.smithi029.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T00:27:33.729 INFO:teuthology.orchestra.run.smithi157.stdout:Get:55 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2445-gd2c45d97-1jammy [711 kB] 2024-04-25T00:27:33.738 INFO:teuthology.orchestra.run.smithi157.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T00:27:33.739 INFO:teuthology.orchestra.run.smithi157.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T00:27:33.740 INFO:teuthology.orchestra.run.smithi157.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T00:27:33.740 INFO:teuthology.orchestra.run.smithi157.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T00:27:33.742 INFO:teuthology.orchestra.run.smithi157.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T00:27:33.745 INFO:teuthology.orchestra.run.smithi029.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T00:27:33.746 INFO:teuthology.orchestra.run.smithi029.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T00:27:33.752 INFO:teuthology.orchestra.run.smithi157.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2445-gd2c45d97-1jammy [356 kB] 2024-04-25T00:27:33.756 INFO:teuthology.orchestra.run.smithi029.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T00:27:33.765 INFO:teuthology.orchestra.run.smithi157.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T00:27:33.767 INFO:teuthology.orchestra.run.smithi157.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T00:27:33.767 INFO:teuthology.orchestra.run.smithi157.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T00:27:33.768 INFO:teuthology.orchestra.run.smithi157.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2445-gd2c45d97-1jammy [32.8 kB] 2024-04-25T00:27:33.770 INFO:teuthology.orchestra.run.smithi157.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2445-gd2c45d97-1jammy [180 kB] 2024-04-25T00:27:33.779 INFO:teuthology.orchestra.run.smithi029.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T00:27:33.779 INFO:teuthology.orchestra.run.smithi157.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2445-gd2c45d97-1jammy [65.7 kB] 2024-04-25T00:27:33.780 INFO:teuthology.orchestra.run.smithi029.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T00:27:33.781 INFO:teuthology.orchestra.run.smithi029.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T00:27:33.782 INFO:teuthology.orchestra.run.smithi157.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2445-gd2c45d97-1jammy [330 kB] 2024-04-25T00:27:33.784 INFO:teuthology.orchestra.run.smithi029.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T00:27:33.788 INFO:teuthology.orchestra.run.smithi029.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T00:27:33.789 INFO:teuthology.orchestra.run.smithi029.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T00:27:33.790 INFO:teuthology.orchestra.run.smithi157.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T00:27:33.790 INFO:teuthology.orchestra.run.smithi157.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T00:27:33.792 INFO:teuthology.orchestra.run.smithi029.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T00:27:33.794 INFO:teuthology.orchestra.run.smithi157.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T00:27:33.795 INFO:teuthology.orchestra.run.smithi157.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T00:27:33.796 INFO:teuthology.orchestra.run.smithi157.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T00:27:33.796 INFO:teuthology.orchestra.run.smithi157.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T00:27:33.797 INFO:teuthology.orchestra.run.smithi157.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T00:27:33.797 INFO:teuthology.orchestra.run.smithi157.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2445-gd2c45d97-1jammy [6,775 kB] 2024-04-25T00:27:33.797 INFO:teuthology.orchestra.run.smithi157.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T00:27:33.800 INFO:teuthology.orchestra.run.smithi157.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T00:27:33.802 INFO:teuthology.orchestra.run.smithi029.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T00:27:33.806 INFO:teuthology.orchestra.run.smithi029.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T00:27:33.807 INFO:teuthology.orchestra.run.smithi029.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T00:27:33.815 INFO:teuthology.orchestra.run.smithi157.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T00:27:33.817 INFO:teuthology.orchestra.run.smithi157.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T00:27:33.817 INFO:teuthology.orchestra.run.smithi157.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T00:27:33.823 INFO:teuthology.orchestra.run.smithi029.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T00:27:33.827 INFO:teuthology.orchestra.run.smithi029.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T00:27:33.850 INFO:teuthology.orchestra.run.smithi157.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T00:27:33.853 INFO:teuthology.orchestra.run.smithi157.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T00:27:33.853 INFO:teuthology.orchestra.run.smithi157.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T00:27:33.854 INFO:teuthology.orchestra.run.smithi157.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T00:27:33.855 INFO:teuthology.orchestra.run.smithi157.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T00:27:33.860 INFO:teuthology.orchestra.run.smithi157.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T00:27:33.861 INFO:teuthology.orchestra.run.smithi157.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T00:27:33.882 INFO:teuthology.orchestra.run.smithi157.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T00:27:33.883 INFO:teuthology.orchestra.run.smithi157.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T00:27:33.883 INFO:teuthology.orchestra.run.smithi157.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T00:27:33.903 INFO:teuthology.orchestra.run.smithi157.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T00:27:33.905 INFO:teuthology.orchestra.run.smithi157.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T00:27:33.906 INFO:teuthology.orchestra.run.smithi157.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T00:27:33.907 INFO:teuthology.orchestra.run.smithi029.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T00:27:33.908 INFO:teuthology.orchestra.run.smithi029.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T00:27:33.909 INFO:teuthology.orchestra.run.smithi029.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T00:27:33.910 INFO:teuthology.orchestra.run.smithi029.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T00:27:33.911 INFO:teuthology.orchestra.run.smithi029.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T00:27:33.918 INFO:teuthology.orchestra.run.smithi157.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T00:27:33.919 INFO:teuthology.orchestra.run.smithi157.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T00:27:33.919 INFO:teuthology.orchestra.run.smithi157.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T00:27:33.919 INFO:teuthology.orchestra.run.smithi157.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T00:27:33.930 INFO:teuthology.orchestra.run.smithi029.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T00:27:33.931 INFO:teuthology.orchestra.run.smithi029.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T00:27:33.932 INFO:teuthology.orchestra.run.smithi029.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T00:27:33.933 INFO:teuthology.orchestra.run.smithi157.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T00:27:33.961 INFO:teuthology.orchestra.run.smithi029.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T00:27:33.962 INFO:teuthology.orchestra.run.smithi029.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T00:27:33.971 INFO:teuthology.orchestra.run.smithi029.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T00:27:33.971 INFO:teuthology.orchestra.run.smithi029.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T00:27:33.972 INFO:teuthology.orchestra.run.smithi029.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T00:27:33.972 INFO:teuthology.orchestra.run.smithi029.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T00:27:33.973 INFO:teuthology.orchestra.run.smithi029.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T00:27:33.973 INFO:teuthology.orchestra.run.smithi029.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T00:27:33.976 INFO:teuthology.orchestra.run.smithi029.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T00:27:33.991 INFO:teuthology.orchestra.run.smithi029.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T00:27:33.993 INFO:teuthology.orchestra.run.smithi029.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T00:27:33.993 INFO:teuthology.orchestra.run.smithi029.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T00:27:34.012 INFO:teuthology.orchestra.run.smithi029.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T00:27:34.015 INFO:teuthology.orchestra.run.smithi029.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T00:27:34.015 INFO:teuthology.orchestra.run.smithi029.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T00:27:34.017 INFO:teuthology.orchestra.run.smithi029.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T00:27:34.017 INFO:teuthology.orchestra.run.smithi029.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T00:27:34.023 INFO:teuthology.orchestra.run.smithi029.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T00:27:34.024 INFO:teuthology.orchestra.run.smithi029.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T00:27:34.032 INFO:teuthology.orchestra.run.smithi029.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T00:27:34.033 INFO:teuthology.orchestra.run.smithi029.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T00:27:34.033 INFO:teuthology.orchestra.run.smithi029.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T00:27:34.053 INFO:teuthology.orchestra.run.smithi029.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T00:27:34.055 INFO:teuthology.orchestra.run.smithi029.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T00:27:34.056 INFO:teuthology.orchestra.run.smithi029.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T00:27:34.066 INFO:teuthology.orchestra.run.smithi029.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T00:27:34.066 INFO:teuthology.orchestra.run.smithi029.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T00:27:34.067 INFO:teuthology.orchestra.run.smithi029.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T00:27:34.069 INFO:teuthology.orchestra.run.smithi029.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T00:27:34.073 INFO:teuthology.orchestra.run.smithi029.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T00:27:34.095 INFO:teuthology.orchestra.run.smithi157.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2445-gd2c45d97-1jammy [112 kB] 2024-04-25T00:27:34.098 INFO:teuthology.orchestra.run.smithi157.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2445-gd2c45d97-1jammy [466 kB] 2024-04-25T00:27:34.112 INFO:teuthology.orchestra.run.smithi157.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2445-gd2c45d97-1jammy [25.6 MB] 2024-04-25T00:27:34.158 INFO:teuthology.orchestra.run.smithi029.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2445-gd2c45d97-1jammy [3,218 kB] 2024-04-25T00:27:34.498 INFO:teuthology.orchestra.run.smithi029.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2445-gd2c45d97-1jammy [711 kB] 2024-04-25T00:27:34.581 INFO:teuthology.orchestra.run.smithi029.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2445-gd2c45d97-1jammy [356 kB] 2024-04-25T00:27:34.624 INFO:teuthology.orchestra.run.smithi029.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2445-gd2c45d97-1jammy [32.8 kB] 2024-04-25T00:27:34.632 INFO:teuthology.orchestra.run.smithi029.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2445-gd2c45d97-1jammy [180 kB] 2024-04-25T00:27:34.644 INFO:teuthology.orchestra.run.smithi029.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2445-gd2c45d97-1jammy [65.7 kB] 2024-04-25T00:27:34.648 INFO:teuthology.orchestra.run.smithi029.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2445-gd2c45d97-1jammy [330 kB] 2024-04-25T00:27:34.672 INFO:teuthology.orchestra.run.smithi029.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2445-gd2c45d97-1jammy [6,775 kB] 2024-04-25T00:27:35.239 INFO:teuthology.orchestra.run.smithi029.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2445-gd2c45d97-1jammy [112 kB] 2024-04-25T00:27:35.247 INFO:teuthology.orchestra.run.smithi029.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2445-gd2c45d97-1jammy [466 kB] 2024-04-25T00:27:35.280 INFO:teuthology.orchestra.run.smithi029.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2445-gd2c45d97-1jammy [25.6 MB] 2024-04-25T00:27:35.596 INFO:teuthology.orchestra.run.smithi157.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2445-gd2c45d97-1jammy [5,084 kB] 2024-04-25T00:27:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2445-gd2c45d97-1jammy [244 kB] 2024-04-25T00:27:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2445-gd2c45d97-1jammy [124 kB] 2024-04-25T00:27:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2445-gd2c45d97-1jammy [1,525 kB] 2024-04-25T00:27:35.863 INFO:teuthology.orchestra.run.smithi157.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2445-gd2c45d97-1jammy [6,245 kB] 2024-04-25T00:27:36.297 INFO:teuthology.orchestra.run.smithi157.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2445-gd2c45d97-1jammy [22.7 MB] 2024-04-25T00:27:37.546 INFO:teuthology.orchestra.run.smithi029.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2445-gd2c45d97-1jammy [5,084 kB] 2024-04-25T00:27:38.078 INFO:teuthology.orchestra.run.smithi029.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2445-gd2c45d97-1jammy [244 kB] 2024-04-25T00:27:38.121 INFO:teuthology.orchestra.run.smithi029.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2445-gd2c45d97-1jammy [124 kB] 2024-04-25T00:27:38.142 INFO:teuthology.orchestra.run.smithi029.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2445-gd2c45d97-1jammy [1,525 kB] 2024-04-25T00:27:38.333 INFO:teuthology.orchestra.run.smithi029.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2445-gd2c45d97-1jammy [6,245 kB] 2024-04-25T00:27:38.765 INFO:teuthology.orchestra.run.smithi157.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2445-gd2c45d97-1jammy [14.0 kB] 2024-04-25T00:27:38.766 INFO:teuthology.orchestra.run.smithi157.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [869 kB] 2024-04-25T00:27:38.810 INFO:teuthology.orchestra.run.smithi157.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2445-gd2c45d97-1jammy [2,437 kB] 2024-04-25T00:27:39.161 INFO:teuthology.orchestra.run.smithi157.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2445-gd2c45d97-1jammy [790 kB] 2024-04-25T00:27:39.266 INFO:teuthology.orchestra.run.smithi029.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2445-gd2c45d97-1jammy [22.7 MB] 2024-04-25T00:27:39.278 INFO:teuthology.orchestra.run.smithi157.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2445-gd2c45d97-1jammy [147 kB] 2024-04-25T00:27:39.283 INFO:teuthology.orchestra.run.smithi157.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2445-gd2c45d97-1jammy [2,293 kB] 2024-04-25T00:27:39.659 INFO:teuthology.orchestra.run.smithi157.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2445-gd2c45d97-1jammy [8,625 kB] 2024-04-25T00:27:41.567 INFO:teuthology.orchestra.run.smithi157.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2445-gd2c45d97-1jammy [14.2 kB] 2024-04-25T00:27:41.577 INFO:teuthology.orchestra.run.smithi157.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2445-gd2c45d97-1jammy [50.7 MB] 2024-04-25T00:27:44.912 INFO:teuthology.orchestra.run.smithi029.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2445-gd2c45d97-1jammy [14.0 kB] 2024-04-25T00:27:44.913 INFO:teuthology.orchestra.run.smithi029.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [869 kB] 2024-04-25T00:27:45.067 INFO:teuthology.orchestra.run.smithi029.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2445-gd2c45d97-1jammy [2,437 kB] 2024-04-25T00:27:45.740 INFO:teuthology.orchestra.run.smithi029.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2445-gd2c45d97-1jammy [790 kB] 2024-04-25T00:27:45.944 INFO:teuthology.orchestra.run.smithi029.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2445-gd2c45d97-1jammy [147 kB] 2024-04-25T00:27:46.006 INFO:teuthology.orchestra.run.smithi029.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2445-gd2c45d97-1jammy [2,293 kB] 2024-04-25T00:27:46.412 INFO:teuthology.orchestra.run.smithi029.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2445-gd2c45d97-1jammy [8,625 kB] 2024-04-25T00:27:48.356 INFO:teuthology.orchestra.run.smithi029.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2445-gd2c45d97-1jammy [14.2 kB] 2024-04-25T00:27:48.356 INFO:teuthology.orchestra.run.smithi029.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2445-gd2c45d97-1jammy [50.7 MB] 2024-04-25T00:27:51.852 INFO:teuthology.orchestra.run.smithi157.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2445-gd2c45d97-1jammy [121 kB] 2024-04-25T00:27:51.855 INFO:teuthology.orchestra.run.smithi157.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2445-gd2c45d97-1jammy [229 kB] 2024-04-25T00:27:51.859 INFO:teuthology.orchestra.run.smithi157.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2445-gd2c45d97-1jammy [34.0 kB] 2024-04-25T00:27:51.860 INFO:teuthology.orchestra.run.smithi157.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2445-gd2c45d97-1jammy [23.4 kB] 2024-04-25T00:27:51.861 INFO:teuthology.orchestra.run.smithi157.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2445-gd2c45d97-1jammy [38.9 kB] 2024-04-25T00:27:51.862 INFO:teuthology.orchestra.run.smithi157.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2445-gd2c45d97-1jammy [13.4 MB] 2024-04-25T00:27:54.108 INFO:teuthology.orchestra.run.smithi157.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [91.2 kB] 2024-04-25T00:27:54.735 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 180 MB in 21s (8,625 kB/s) 2024-04-25T00:27:54.877 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T00:27:54.976 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T00:27:54.982 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T00:27:55.033 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T00:27:55.344 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T00:27:55.370 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T00:27:55.419 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:27:55.805 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T00:27:55.831 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T00:27:55.880 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T00:27:56.308 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T00:27:56.334 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T00:27:56.433 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:27:56.795 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T00:27:56.821 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T00:27:56.862 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:27:57.381 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T00:27:57.408 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T00:27:57.448 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:27:57.739 INFO:teuthology.orchestra.run.smithi029.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2445-gd2c45d97-1jammy [121 kB] 2024-04-25T00:27:57.741 INFO:teuthology.orchestra.run.smithi029.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2445-gd2c45d97-1jammy [229 kB] 2024-04-25T00:27:57.746 INFO:teuthology.orchestra.run.smithi029.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2445-gd2c45d97-1jammy [34.0 kB] 2024-04-25T00:27:57.747 INFO:teuthology.orchestra.run.smithi029.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2445-gd2c45d97-1jammy [23.4 kB] 2024-04-25T00:27:57.747 INFO:teuthology.orchestra.run.smithi029.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2445-gd2c45d97-1jammy [38.9 kB] 2024-04-25T00:27:57.748 INFO:teuthology.orchestra.run.smithi029.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2445-gd2c45d97-1jammy [13.4 MB] 2024-04-25T00:27:57.994 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T00:27:58.020 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:27:58.076 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:27:58.472 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T00:27:58.498 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:27:58.538 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:27:58.883 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T00:27:58.909 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:27:58.949 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:27:59.004 INFO:teuthology.orchestra.run.smithi029.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [91.2 kB] 2024-04-25T00:27:59.278 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T00:27:59.304 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T00:27:59.344 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:27:59.642 INFO:teuthology.orchestra.run.smithi029.stdout:Fetched 180 MB in 26s (7,012 kB/s) 2024-04-25T00:27:59.672 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librados2. 2024-04-25T00:27:59.698 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../010-librados2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:27:59.738 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:27:59.782 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi029.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T00:27:59.888 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T00:27:59.922 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T00:28:00.166 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librbd1. 2024-04-25T00:28:00.192 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../011-librbd1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:00.219 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T00:28:00.233 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:00.246 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T00:28:00.470 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:28:00.652 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T00:28:00.679 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:00.719 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T00:28:00.844 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T00:28:00.892 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T00:28:01.013 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rados. 2024-04-25T00:28:01.028 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:01.071 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:01.249 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T00:28:01.276 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T00:28:01.375 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:28:01.383 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T00:28:01.409 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:01.449 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:01.701 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T00:28:01.716 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:01.756 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T00:28:01.760 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:01.783 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T00:28:01.823 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:28:02.046 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T00:28:02.072 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:02.112 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:02.180 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T00:28:02.207 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:28:02.524 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T00:28:02.551 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T00:28:02.591 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T00:28:02.738 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T00:28:02.765 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:28:02.821 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:28:02.851 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T00:28:02.878 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:02.918 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:03.269 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T00:28:03.271 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T00:28:03.295 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:28:03.298 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T00:28:03.337 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:28:03.337 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:28:03.607 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T00:28:03.622 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T00:28:03.657 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:28:03.710 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T00:28:03.737 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:28:03.776 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:28:03.951 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T00:28:03.967 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T00:28:04.010 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:28:04.125 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T00:28:04.152 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T00:28:04.192 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:28:04.396 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package lua5.1. 2024-04-25T00:28:04.423 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T00:28:04.530 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package librados2. 2024-04-25T00:28:04.557 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../010-librados2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:04.598 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:04.947 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:28:05.145 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package librbd1. 2024-04-25T00:28:05.172 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../011-librbd1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:05.212 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:05.517 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package lua-any. 2024-04-25T00:28:05.543 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T00:28:05.592 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T00:28:05.844 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package zip. 2024-04-25T00:28:05.859 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T00:28:05.902 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T00:28:05.902 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T00:28:05.929 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:05.969 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:06.214 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package unzip. 2024-04-25T00:28:06.240 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T00:28:06.280 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:28:06.315 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-rados. 2024-04-25T00:28:06.331 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:06.373 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:06.726 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package luarocks. 2024-04-25T00:28:06.752 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T00:28:06.801 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:28:07.209 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librgw2. 2024-04-25T00:28:07.236 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../027-librgw2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:07.476 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:08.045 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T00:28:08.071 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:08.111 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:08.164 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T00:28:08.191 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:08.230 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:08.406 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T00:28:08.432 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T00:28:08.473 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:28:08.478 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T00:28:08.494 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:08.537 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:08.792 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T00:28:08.819 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:08.859 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:08.866 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T00:28:08.893 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:08.933 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:09.274 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T00:28:09.279 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-common. 2024-04-25T00:28:09.300 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T00:28:09.305 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:09.340 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T00:28:09.345 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:09.603 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T00:28:09.630 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:09.672 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:10.035 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T00:28:10.062 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T00:28:10.110 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:28:10.416 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T00:28:10.432 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T00:28:10.475 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:28:10.634 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-base. 2024-04-25T00:28:10.660 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:10.710 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:10.814 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T00:28:10.840 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T00:28:10.880 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:28:11.213 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package lua5.1. 2024-04-25T00:28:11.240 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T00:28:11.272 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T00:28:11.279 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:28:11.298 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T00:28:11.338 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:28:11.603 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package lua-any. 2024-04-25T00:28:11.630 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T00:28:11.666 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T00:28:11.670 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T00:28:11.692 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T00:28:11.733 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:28:11.916 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package zip. 2024-04-25T00:28:11.943 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T00:28:11.991 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T00:28:12.061 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T00:28:12.087 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T00:28:12.127 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:28:12.324 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package unzip. 2024-04-25T00:28:12.350 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T00:28:12.390 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:28:12.405 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T00:28:12.432 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T00:28:12.463 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:28:12.758 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T00:28:12.785 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T00:28:12.790 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package luarocks. 2024-04-25T00:28:12.817 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T00:28:12.824 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:28:12.857 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:28:13.127 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T00:28:13.154 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T00:28:13.194 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T00:28:13.239 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package librgw2. 2024-04-25T00:28:13.266 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../027-librgw2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:13.308 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:13.472 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-portend. 2024-04-25T00:28:13.487 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T00:28:13.522 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T00:28:13.800 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T00:28:13.826 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T00:28:13.866 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T00:28:14.029 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T00:28:14.045 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:14.087 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:14.169 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T00:28:14.185 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T00:28:14.228 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:28:14.419 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T00:28:14.446 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T00:28:14.494 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:28:14.564 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T00:28:14.591 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T00:28:14.631 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T00:28:14.816 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T00:28:14.843 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:14.883 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:14.936 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T00:28:14.963 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T00:28:15.003 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T00:28:15.306 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T00:28:15.332 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-common. 2024-04-25T00:28:15.332 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T00:28:15.359 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:15.372 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:28:15.398 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:15.642 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-mako. 2024-04-25T00:28:15.668 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T00:28:15.708 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:28:16.045 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T00:28:16.071 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T00:28:16.111 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:28:16.414 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T00:28:16.441 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T00:28:16.481 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:28:16.825 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-webob. 2024-04-25T00:28:16.852 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T00:28:16.900 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:28:17.320 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T00:28:17.346 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T00:28:17.397 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:28:17.460 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-base. 2024-04-25T00:28:17.487 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:17.536 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:17.928 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T00:28:17.954 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T00:28:18.014 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:28:18.261 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T00:28:18.288 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T00:28:18.326 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-paste. 2024-04-25T00:28:18.336 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:28:18.352 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T00:28:18.392 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:28:18.643 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T00:28:18.670 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T00:28:18.709 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:28:18.754 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T00:28:18.769 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T00:28:18.837 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:28:19.040 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T00:28:19.067 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T00:28:19.116 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:28:19.140 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T00:28:19.166 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T00:28:19.206 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:28:19.413 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T00:28:19.441 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T00:28:19.480 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:28:19.493 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T00:28:19.519 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T00:28:19.559 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T00:28:19.770 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T00:28:19.786 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T00:28:19.828 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:28:19.829 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T00:28:19.855 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T00:28:19.896 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:28:20.109 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T00:28:20.136 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T00:28:20.175 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T00:28:20.225 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T00:28:20.252 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T00:28:20.292 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T00:28:20.474 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-portend. 2024-04-25T00:28:20.501 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T00:28:20.540 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T00:28:20.620 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-idna. 2024-04-25T00:28:20.647 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T00:28:20.686 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T00:28:20.830 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T00:28:20.858 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T00:28:20.905 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T00:28:21.006 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T00:28:21.033 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T00:28:21.073 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:28:21.203 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T00:28:21.230 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T00:28:21.284 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:28:21.342 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-requests. 2024-04-25T00:28:21.358 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T00:28:21.392 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:28:21.619 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T00:28:21.646 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T00:28:21.694 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T00:28:21.695 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T00:28:21.722 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T00:28:21.762 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:28:21.976 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T00:28:22.003 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T00:28:22.043 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T00:28:22.048 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T00:28:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:22.115 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:22.339 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T00:28:22.367 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T00:28:22.414 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:28:22.495 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T00:28:22.511 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:22.545 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:22.721 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-mako. 2024-04-25T00:28:22.748 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T00:28:22.788 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:28:22.864 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T00:28:22.891 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T00:28:22.931 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:28:23.129 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T00:28:23.156 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T00:28:23.204 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:28:23.350 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T00:28:23.377 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T00:28:23.417 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T00:28:23.492 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T00:28:23.519 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T00:28:23.559 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:28:23.745 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T00:28:23.772 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:23.811 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:23.857 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-webob. 2024-04-25T00:28:23.884 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T00:28:23.923 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:28:24.215 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T00:28:24.242 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:24.281 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:24.314 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T00:28:24.341 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T00:28:24.384 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:28:24.686 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T00:28:24.715 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T00:28:24.761 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:28:24.835 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T00:28:24.862 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:24.901 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:25.068 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-paste. 2024-04-25T00:28:25.095 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T00:28:25.134 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:28:25.466 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T00:28:25.483 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T00:28:25.525 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:28:25.838 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T00:28:25.866 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T00:28:26.106 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:28:26.146 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph. 2024-04-25T00:28:26.173 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../068-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:26.213 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:26.524 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T00:28:26.550 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:26.590 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:26.670 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T00:28:26.698 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T00:28:26.745 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T00:28:26.910 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T00:28:26.937 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:26.968 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:27.051 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T00:28:27.078 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T00:28:27.119 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:28:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephadm. 2024-04-25T00:28:27.390 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../071-cephadm_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:27.496 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T00:28:27.523 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T00:28:27.562 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T00:28:27.732 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T00:28:27.758 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T00:28:27.798 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T00:28:27.884 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-idna. 2024-04-25T00:28:27.912 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T00:28:27.951 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T00:28:28.126 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T00:28:28.153 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:28:28.192 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:28:28.275 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T00:28:28.302 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T00:28:28.350 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:28:28.655 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-requests. 2024-04-25T00:28:28.683 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T00:28:28.724 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:28:28.995 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T00:28:29.023 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T00:28:29.062 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:28:29.248 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-babel. 2024-04-25T00:28:29.275 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:28:29.314 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:28:29.361 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T00:28:29.388 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:29.427 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:29.650 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T00:28:29.678 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T00:28:29.717 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:28:29.838 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T00:28:29.865 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:29.913 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:30.062 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T00:28:30.089 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:30.120 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:30.268 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T00:28:30.295 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T00:28:30.336 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:28:30.465 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T00:28:30.492 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T00:28:30.531 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T00:28:30.658 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T00:28:30.685 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T00:28:30.725 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T00:28:30.834 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-routes. 2024-04-25T00:28:30.861 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T00:28:30.900 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:28:31.073 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T00:28:31.100 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:31.140 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:31.246 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T00:28:31.272 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:31.312 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:31.579 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T00:28:31.606 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:31.646 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:32.404 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T00:28:32.431 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:32.470 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:32.575 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T00:28:32.591 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T00:28:32.633 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:28:33.179 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T00:28:33.245 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:28:33.590 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T00:28:33.618 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T00:28:33.836 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph. 2024-04-25T00:28:33.852 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../068-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:33.886 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:33.892 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:28:34.204 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T00:28:34.207 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T00:28:34.231 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T00:28:34.234 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:34.270 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:28:34.275 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T00:28:34.700 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:34.739 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:34.913 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T00:28:34.937 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:34.971 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:35.205 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package cephadm. 2024-04-25T00:28:35.232 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../071-cephadm_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:35.280 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:35.618 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T00:28:35.645 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T00:28:35.684 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T00:28:35.966 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T00:28:35.993 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T00:28:36.016 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T00:28:36.032 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:28:36.032 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T00:28:36.074 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:28:36.318 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T00:28:36.346 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T00:28:36.385 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:28:36.747 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T00:28:36.774 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T00:28:36.813 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:28:37.125 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T00:28:37.152 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T00:28:37.191 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T00:28:37.335 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-babel. 2024-04-25T00:28:37.362 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:28:37.401 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:28:37.478 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T00:28:37.505 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T00:28:37.545 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T00:28:37.741 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T00:28:37.769 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T00:28:37.808 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:28:37.865 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T00:28:37.892 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T00:28:37.931 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:28:38.172 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T00:28:38.199 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:38.238 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:38.242 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T00:28:38.270 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T00:28:38.309 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T00:28:38.596 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T00:28:38.623 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T00:28:38.629 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T00:28:38.657 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T00:28:38.662 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T00:28:38.695 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:28:38.944 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-routes. 2024-04-25T00:28:38.971 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T00:28:39.010 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:28:39.343 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T00:28:39.350 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T00:28:39.359 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:39.377 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:39.392 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:39.416 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:39.687 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T00:28:39.715 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T00:28:39.762 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:28:40.115 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T00:28:40.143 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:28:40.181 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:28:40.467 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package jq. 2024-04-25T00:28:40.483 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:28:40.517 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T00:28:40.745 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package socat. 2024-04-25T00:28:40.761 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T00:28:40.804 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:28:41.107 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T00:28:41.134 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T00:28:41.144 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T00:28:41.172 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T00:28:41.173 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:28:41.212 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:28:41.527 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-test. 2024-04-25T00:28:41.543 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:41.585 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:41.718 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T00:28:41.746 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T00:28:41.785 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:28:42.151 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T00:28:42.178 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T00:28:42.407 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:28:42.714 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T00:28:42.739 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T00:28:42.779 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:28:43.424 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T00:28:43.452 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:43.490 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:43.890 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T00:28:43.917 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:43.956 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:44.585 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T00:28:44.613 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:44.652 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:44.939 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T00:28:44.947 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T00:28:44.968 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T00:28:44.974 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T00:28:45.006 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T00:28:45.013 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T00:28:45.232 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T00:28:45.248 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:45.290 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:45.302 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T00:28:45.331 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T00:28:45.371 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:28:45.551 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T00:28:45.567 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:45.609 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:45.710 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T00:28:45.738 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T00:28:45.777 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:28:45.853 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T00:28:45.870 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:45.912 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:46.084 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T00:28:46.112 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T00:28:46.150 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T00:28:46.231 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T00:28:46.259 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T00:28:46.298 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:28:46.467 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T00:28:46.495 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T00:28:46.542 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T00:28:46.619 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T00:28:46.646 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T00:28:46.677 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:28:46.848 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T00:28:46.875 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T00:28:46.923 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:28:47.056 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T00:28:47.084 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T00:28:47.122 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:28:47.230 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T00:28:47.258 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T00:28:47.296 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T00:28:47.578 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package pkg-config. 2024-04-25T00:28:47.605 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T00:28:47.644 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:28:47.854 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T00:28:47.881 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T00:28:47.906 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T00:28:47.920 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:28:47.933 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T00:28:47.972 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:28:48.417 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T00:28:48.445 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T00:28:48.483 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T00:28:48.549 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T00:28:48.566 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:48.607 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:48.761 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T00:28:48.789 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T00:28:48.827 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T00:28:48.930 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T00:28:48.959 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T00:28:49.005 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:28:49.090 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T00:28:49.117 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T00:28:49.147 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:28:49.361 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T00:28:49.389 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:28:49.428 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:28:49.442 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T00:28:49.470 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T00:28:49.533 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T00:28:49.709 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package jq. 2024-04-25T00:28:49.737 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:28:49.775 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T00:28:49.837 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-py. 2024-04-25T00:28:49.866 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T00:28:49.903 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T00:28:50.014 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package socat. 2024-04-25T00:28:50.042 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T00:28:50.097 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:28:50.232 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T00:28:50.259 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T00:28:50.298 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:28:50.497 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T00:28:50.525 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T00:28:50.579 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:28:50.744 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T00:28:50.771 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T00:28:50.810 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:28:51.014 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-test. 2024-04-25T00:28:51.042 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:51.097 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:51.138 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-toml. 2024-04-25T00:28:51.165 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T00:28:51.204 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T00:28:51.457 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T00:28:51.484 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T00:28:51.515 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:28:51.885 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T00:28:51.913 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T00:28:51.951 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:28:52.246 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T00:28:52.274 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T00:28:52.313 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:28:53.050 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package radosgw. 2024-04-25T00:28:53.078 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../122-radosgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:53.108 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:53.911 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T00:28:53.927 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:53.969 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:54.276 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T00:28:54.534 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:28:54.606 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T00:28:54.635 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:54.660 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:28:54.689 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:54.927 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T00:28:55.082 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T00:28:55.110 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:55.148 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:55.246 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T00:28:55.454 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T00:28:55.482 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T00:28:55.522 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T00:28:55.724 INFO:teuthology.orchestra.run.smithi157.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T00:28:55.768 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T00:28:55.795 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:55.796 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:55.835 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:56.099 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T00:28:56.127 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:28:56.166 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:56.254 INFO:teuthology.orchestra.run.smithi157.stdout:Adding system user cephadm....done 2024-04-25T00:28:56.377 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:28:56.412 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T00:28:56.429 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:28:56.470 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:28:56.653 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:28:56.802 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T00:28:56.830 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T00:28:56.868 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:28:56.913 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:28:57.022 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:28:57.220 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T00:28:57.248 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T00:28:57.258 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T00:28:57.286 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:28:57.491 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:28:57.609 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T00:28:57.695 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T00:28:57.723 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T00:28:57.762 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:28:57.919 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:28:58.206 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package pkg-config. 2024-04-25T00:28:58.234 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T00:28:58.281 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:28:58.295 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T00:28:58.545 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T00:28:58.564 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T00:28:58.573 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T00:28:58.611 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:28:58.798 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T00:28:59.027 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T00:28:59.049 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:28:59.055 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T00:28:59.093 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T00:28:59.309 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:28:59.391 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T00:28:59.420 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T00:28:59.466 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T00:28:59.569 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:28:59.789 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T00:28:59.817 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T00:28:59.828 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:28:59.864 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:29:00.113 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:00.179 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T00:29:00.208 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T00:29:00.245 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T00:29:00.364 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:29:00.465 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:29:00.574 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T00:29:00.602 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-py. 2024-04-25T00:29:00.631 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T00:29:00.669 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T00:29:00.825 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:29:00.951 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:29:00.992 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T00:29:01.020 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T00:29:01.058 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:29:01.219 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:29:01.517 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T00:29:01.545 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T00:29:01.584 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:29:01.594 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T00:29:01.625 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T00:29:01.689 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:29:01.815 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T00:29:01.898 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-toml. 2024-04-25T00:29:01.927 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T00:29:01.965 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T00:29:02.174 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:29:02.270 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T00:29:02.292 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T00:29:02.298 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T00:29:02.336 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:29:02.568 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:29:02.686 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T00:29:02.714 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T00:29:02.761 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:29:02.827 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:29:03.129 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T00:29:03.257 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T00:29:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:29:03.285 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T00:29:03.443 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:29:03.940 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:29:04.150 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package radosgw. 2024-04-25T00:29:04.178 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../122-radosgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:29:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:29:04.218 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:04.309 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:29:04.435 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:29:04.736 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T00:29:05.162 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T00:29:05.206 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T00:29:05.223 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:29:05.265 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:05.421 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:29:05.608 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T00:29:05.815 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:29:05.868 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:29:05.996 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:29:06.091 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T00:29:06.266 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T00:29:06.400 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:29:06.662 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T00:29:06.743 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:29:06.994 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:29:07.109 INFO:teuthology.orchestra.run.smithi029.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T00:29:07.120 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:07.183 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:07.414 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T00:29:07.706 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:29:07.831 INFO:teuthology.orchestra.run.smithi029.stdout:Adding system user cephadm....done 2024-04-25T00:29:07.979 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:29:08.258 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:29:08.527 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:29:08.654 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:29:08.795 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:29:08.915 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T00:29:09.039 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:29:09.148 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T00:29:09.185 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:29:09.312 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T00:29:09.416 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:29:09.542 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:29:09.632 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:29:09.668 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:29:09.961 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:29:09.995 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T00:29:10.212 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T00:29:10.256 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T00:29:10.455 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:29:10.517 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T00:29:10.556 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:29:10.803 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:29:10.824 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:29:11.057 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:29:11.067 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T00:29:11.317 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:29:11.326 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T00:29:11.585 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:29:11.586 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:29:11.836 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T00:29:11.848 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:12.104 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:29:12.116 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:29:12.205 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T00:29:12.252 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:29:12.379 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T00:29:12.448 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:29:12.582 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:29:12.640 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:29:12.768 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:29:12.858 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:29:13.078 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:29:13.151 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:29:13.402 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:29:13.504 INFO:teuthology.orchestra.run.smithi029.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T00:29:13.528 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:29:13.538 INFO:teuthology.orchestra.run.smithi029.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T00:29:13.610 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:29:13.654 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:29:13.734 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T00:29:13.737 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T00:29:13.964 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:29:14.065 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:29:14.106 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:29:14.235 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T00:29:14.433 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:29:14.504 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:29:14.693 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T00:29:14.781 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:29:14.819 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T00:29:15.078 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:29:15.083 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T00:29:15.204 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:29:15.211 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:29:15.539 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:29:15.840 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:29:15.907 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:29:16.141 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T00:29:16.167 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:29:16.296 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:29:16.352 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T00:29:16.423 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:29:16.620 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T00:29:16.725 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T00:29:16.888 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:29:17.119 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T00:29:17.123 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:29:17.398 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:29:17.483 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T00:29:17.760 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:29:17.784 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:29:18.086 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T00:29:18.095 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:29:18.197 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:18.298 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:18.424 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:29:18.432 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:29:18.777 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:18.801 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:29:18.902 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:19.029 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T00:29:19.070 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:29:19.197 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:19.289 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:19.390 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:19.516 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:19.517 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T00:29:19.726 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:29:19.727 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:29:19.844 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:29:20.234 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:20.360 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:20.486 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:20.612 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:20.730 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:29:21.032 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:29:21.312 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:29:21.447 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T00:29:21.716 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:29:21.843 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:29:21.946 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:21.971 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:29:22.068 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:22.194 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:22.274 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:29:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:22.446 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:22.543 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T00:29:22.572 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:22.820 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:29:22.940 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:29:23.056 INFO:teuthology.orchestra.run.smithi157.stdout:Adding group ceph....done 2024-04-25T00:29:23.259 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:29:23.482 INFO:teuthology.orchestra.run.smithi157.stdout:Adding system user ceph....done 2024-04-25T00:29:23.519 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T00:29:23.618 INFO:teuthology.orchestra.run.smithi157.stdout:Setting system user ceph properties....done 2024-04-25T00:29:23.634 INFO:teuthology.orchestra.run.smithi157.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T00:29:23.765 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T00:29:23.781 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T00:29:24.050 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:29:24.087 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T00:29:24.311 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T00:29:24.564 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:24.571 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:29:24.699 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T00:29:24.769 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:29:24.769 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:29:24.985 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:29:25.112 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:29:25.249 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:25.375 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:25.414 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:29:25.727 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:29:25.844 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:29:25.844 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:29:25.995 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:29:26.122 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:29:26.249 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:29:26.318 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:26.339 INFO:teuthology.orchestra.run.smithi029.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T00:29:26.604 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:29:26.668 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T00:29:26.742 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:29:27.137 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:27.142 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:29:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:29:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:29:27.412 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T00:29:27.539 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T00:29:27.800 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:29:27.813 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:27.927 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:29:28.019 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:29:28.019 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:29:28.255 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:29:28.507 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:28.574 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:29:28.777 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:29:28.851 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T00:29:29.084 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T00:29:29.233 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:29.359 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:29.361 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T00:29:29.502 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:29.622 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:29:29.712 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:29:29.713 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:29:29.884 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:29:30.178 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:30.313 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T00:29:30.321 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:30.430 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:30.581 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:30.632 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:29:30.910 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:29:30.996 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:29:31.018 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:29:31.123 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:31.171 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:29:31.250 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:31.378 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:29:31.725 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:31.852 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:31.988 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T00:29:32.264 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:29:32.266 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:32.393 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:32.520 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:32.742 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:29:32.743 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:29:32.907 INFO:teuthology.orchestra.run.smithi157.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:29:32.910 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T00:29:33.008 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2024-04-25T00:29:33.208 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2024-04-25T00:29:33.209 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2024-04-25T00:29:33.249 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:33.376 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:33.397 INFO:teuthology.orchestra.run.smithi157.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T00:29:33.397 INFO:teuthology.orchestra.run.smithi157.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T00:29:33.397 INFO:teuthology.orchestra.run.smithi157.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T00:29:33.397 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:29:33.397 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:29:33.398 INFO:teuthology.orchestra.run.smithi157.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:29:33.398 INFO:teuthology.orchestra.run.smithi157.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:29:33.399 INFO:teuthology.orchestra.run.smithi157.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:29:33.399 INFO:teuthology.orchestra.run.smithi157.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:29:33.399 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:29:33.399 INFO:teuthology.orchestra.run.smithi157.stdout:The following additional packages will be installed: 2024-04-25T00:29:33.399 INFO:teuthology.orchestra.run.smithi157.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T00:29:33.399 INFO:teuthology.orchestra.run.smithi157.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T00:29:33.400 INFO:teuthology.orchestra.run.smithi157.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T00:29:33.400 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T00:29:33.400 INFO:teuthology.orchestra.run.smithi157.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:29:33.401 INFO:teuthology.orchestra.run.smithi157.stdout:Suggested packages: 2024-04-25T00:29:33.401 INFO:teuthology.orchestra.run.smithi157.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T00:29:33.401 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T00:29:33.401 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T00:29:33.439 INFO:teuthology.orchestra.run.smithi157.stdout:The following NEW packages will be installed: 2024-04-25T00:29:33.439 INFO:teuthology.orchestra.run.smithi157.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T00:29:33.440 INFO:teuthology.orchestra.run.smithi157.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T00:29:33.440 INFO:teuthology.orchestra.run.smithi157.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T00:29:33.440 INFO:teuthology.orchestra.run.smithi157.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T00:29:33.440 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T00:29:33.441 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T00:29:33.441 INFO:teuthology.orchestra.run.smithi157.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:29:33.442 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages will be upgraded: 2024-04-25T00:29:33.442 INFO:teuthology.orchestra.run.smithi157.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T00:29:33.501 INFO:teuthology.orchestra.run.smithi157.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T00:29:33.501 INFO:teuthology.orchestra.run.smithi157.stdout:Need to get 55.7 MB of archives. 2024-04-25T00:29:33.501 INFO:teuthology.orchestra.run.smithi157.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T00:29:33.501 INFO:teuthology.orchestra.run.smithi157.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T00:29:33.503 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:33.630 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:33.661 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T00:29:33.666 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T00:29:33.708 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T00:29:33.709 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T00:29:33.710 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T00:29:33.719 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T00:29:33.719 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T00:29:33.721 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T00:29:33.723 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T00:29:33.723 INFO:teuthology.orchestra.run.smithi157.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T00:29:33.724 INFO:teuthology.orchestra.run.smithi157.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T00:29:33.730 INFO:teuthology.orchestra.run.smithi157.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T00:29:33.746 INFO:teuthology.orchestra.run.smithi157.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T00:29:33.747 INFO:teuthology.orchestra.run.smithi157.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T00:29:33.748 INFO:teuthology.orchestra.run.smithi157.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T00:29:33.759 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:29:34.094 INFO:teuthology.orchestra.run.smithi157.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T00:29:34.096 INFO:teuthology.orchestra.run.smithi157.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T00:29:34.097 INFO:teuthology.orchestra.run.smithi157.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T00:29:34.100 INFO:teuthology.orchestra.run.smithi157.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T00:29:34.133 INFO:teuthology.orchestra.run.smithi157.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T00:29:34.140 INFO:teuthology.orchestra.run.smithi157.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T00:29:34.142 INFO:teuthology.orchestra.run.smithi157.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T00:29:34.153 INFO:teuthology.orchestra.run.smithi157.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T00:29:34.153 INFO:teuthology.orchestra.run.smithi157.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T00:29:34.184 INFO:teuthology.orchestra.run.smithi157.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T00:29:34.185 INFO:teuthology.orchestra.run.smithi157.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T00:29:34.197 INFO:teuthology.orchestra.run.smithi157.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T00:29:34.232 INFO:teuthology.orchestra.run.smithi157.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T00:29:34.236 INFO:teuthology.orchestra.run.smithi157.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T00:29:34.237 INFO:teuthology.orchestra.run.smithi157.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T00:29:34.238 INFO:teuthology.orchestra.run.smithi157.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T00:29:34.238 INFO:teuthology.orchestra.run.smithi157.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T00:29:34.260 INFO:teuthology.orchestra.run.smithi157.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T00:29:34.262 INFO:teuthology.orchestra.run.smithi157.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T00:29:34.545 INFO:teuthology.orchestra.run.smithi157.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T00:29:34.546 INFO:teuthology.orchestra.run.smithi157.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T00:29:34.546 INFO:teuthology.orchestra.run.smithi157.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T00:29:34.546 INFO:teuthology.orchestra.run.smithi157.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T00:29:34.546 INFO:teuthology.orchestra.run.smithi157.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T00:29:34.546 INFO:teuthology.orchestra.run.smithi157.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T00:29:34.558 INFO:teuthology.orchestra.run.smithi157.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T00:29:34.558 INFO:teuthology.orchestra.run.smithi157.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T00:29:34.915 INFO:teuthology.orchestra.run.smithi157.stdout:Preconfiguring packages ... 2024-04-25T00:29:35.017 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:35.063 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 55.7 MB in 1s (50.3 MB/s) 2024-04-25T00:29:35.144 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:35.271 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:35.384 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T00:29:35.390 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:29:35.398 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:35.525 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:35.561 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:29:35.653 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:Adding group ceph....done 2024-04-25T00:29:36.606 INFO:teuthology.orchestra.run.smithi029.stdout:Adding system user ceph....done 2024-04-25T00:29:36.778 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T00:29:36.786 INFO:teuthology.orchestra.run.smithi029.stdout:Setting system user ceph properties....done 2024-04-25T00:29:36.800 INFO:teuthology.orchestra.run.smithi029.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T00:29:36.942 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T00:29:37.271 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T00:29:37.750 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:37.788 INFO:teuthology.orchestra.run.smithi157.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package flex. 2024-04-25T00:29:38.049 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:29:38.049 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:29:38.108 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T00:29:38.113 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T00:29:38.157 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T00:29:38.560 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:29:38.687 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:38.771 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T00:29:39.175 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:29:39.176 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:29:39.181 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:29:39.643 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:39.669 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T00:29:39.674 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:29:39.996 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T00:29:40.163 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:29:40.277 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:29:40.465 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:40.612 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:29:40.719 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:29:40.719 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:29:40.986 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package bison. 2024-04-25T00:29:41.087 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T00:29:41.092 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T00:29:41.146 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:29:41.201 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:41.426 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:29:41.426 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:29:41.674 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T00:29:41.702 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T00:29:41.740 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:29:41.937 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:42.069 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package dump. 2024-04-25T00:29:42.096 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T00:29:42.135 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T00:29:42.223 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:29:42.223 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:29:42.430 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package indent. 2024-04-25T00:29:42.446 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:29:42.480 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:29:42.708 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T00:29:42.735 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T00:29:42.748 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:42.774 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:29:42.877 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:43.029 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:43.112 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T00:29:43.139 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T00:29:43.178 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:29:43.248 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:29:43.248 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:29:43.484 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T00:29:43.511 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T00:29:43.558 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:29:43.715 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:43.869 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:43.920 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T00:29:43.948 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T00:29:43.987 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:29:43.995 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:44.156 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:29:44.239 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T00:29:44.257 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T00:29:44.306 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:29:44.563 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:29:44.601 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T00:29:44.628 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T00:29:44.676 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T00:29:44.681 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:29:44.836 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:29:45.079 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T00:29:45.106 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T00:29:45.145 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:29:46.006 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:29:46.761 INFO:teuthology.orchestra.run.smithi029.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:29:46.763 DEBUG:teuthology.orchestra.run.smithi029:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T00:29:46.819 INFO:teuthology.orchestra.run.smithi029.stdout:Reading package lists... 2024-04-25T00:29:46.858 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T00:29:46.885 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:29:46.924 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:29:47.024 INFO:teuthology.orchestra.run.smithi029.stdout:Building dependency tree... 2024-04-25T00:29:47.025 INFO:teuthology.orchestra.run.smithi029.stdout:Reading state information... 2024-04-25T00:29:47.194 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T00:29:47.209 INFO:teuthology.orchestra.run.smithi029.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T00:29:47.209 INFO:teuthology.orchestra.run.smithi029.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T00:29:47.209 INFO:teuthology.orchestra.run.smithi029.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T00:29:47.209 INFO:teuthology.orchestra.run.smithi029.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:29:47.209 INFO:teuthology.orchestra.run.smithi029.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:29:47.209 INFO:teuthology.orchestra.run.smithi029.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:29:47.209 INFO:teuthology.orchestra.run.smithi029.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:29:47.210 INFO:teuthology.orchestra.run.smithi029.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:29:47.210 INFO:teuthology.orchestra.run.smithi029.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:29:47.210 INFO:teuthology.orchestra.run.smithi029.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:29:47.211 INFO:teuthology.orchestra.run.smithi029.stdout:The following additional packages will be installed: 2024-04-25T00:29:47.212 INFO:teuthology.orchestra.run.smithi029.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T00:29:47.212 INFO:teuthology.orchestra.run.smithi029.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T00:29:47.212 INFO:teuthology.orchestra.run.smithi029.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T00:29:47.212 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T00:29:47.212 INFO:teuthology.orchestra.run.smithi029.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:29:47.213 INFO:teuthology.orchestra.run.smithi029.stdout:Suggested packages: 2024-04-25T00:29:47.213 INFO:teuthology.orchestra.run.smithi029.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T00:29:47.213 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T00:29:47.213 INFO:teuthology.orchestra.run.smithi029.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T00:29:47.222 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:29:47.254 INFO:teuthology.orchestra.run.smithi029.stdout:The following NEW packages will be installed: 2024-04-25T00:29:47.254 INFO:teuthology.orchestra.run.smithi029.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T00:29:47.254 INFO:teuthology.orchestra.run.smithi029.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T00:29:47.254 INFO:teuthology.orchestra.run.smithi029.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T00:29:47.254 INFO:teuthology.orchestra.run.smithi029.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T00:29:47.254 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T00:29:47.255 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T00:29:47.255 INFO:teuthology.orchestra.run.smithi029.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:29:47.255 INFO:teuthology.orchestra.run.smithi029.stdout:The following packages will be upgraded: 2024-04-25T00:29:47.256 INFO:teuthology.orchestra.run.smithi029.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T00:29:47.263 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:29:47.312 INFO:teuthology.orchestra.run.smithi029.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T00:29:47.312 INFO:teuthology.orchestra.run.smithi029.stdout:Need to get 55.7 MB of archives. 2024-04-25T00:29:47.312 INFO:teuthology.orchestra.run.smithi029.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T00:29:47.312 INFO:teuthology.orchestra.run.smithi029.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T00:29:47.482 INFO:teuthology.orchestra.run.smithi029.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T00:29:47.489 INFO:teuthology.orchestra.run.smithi029.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T00:29:47.522 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T00:29:47.531 INFO:teuthology.orchestra.run.smithi029.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T00:29:47.532 INFO:teuthology.orchestra.run.smithi029.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T00:29:47.532 INFO:teuthology.orchestra.run.smithi029.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T00:29:47.544 INFO:teuthology.orchestra.run.smithi029.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T00:29:47.545 INFO:teuthology.orchestra.run.smithi029.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T00:29:47.547 INFO:teuthology.orchestra.run.smithi029.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T00:29:47.549 INFO:teuthology.orchestra.run.smithi029.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T00:29:47.549 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:29:47.550 INFO:teuthology.orchestra.run.smithi029.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T00:29:47.551 INFO:teuthology.orchestra.run.smithi029.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T00:29:47.552 INFO:teuthology.orchestra.run.smithi029.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T00:29:47.572 INFO:teuthology.orchestra.run.smithi029.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T00:29:47.572 INFO:teuthology.orchestra.run.smithi029.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T00:29:47.573 INFO:teuthology.orchestra.run.smithi029.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T00:29:47.579 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:29:47.933 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T00:29:47.946 INFO:teuthology.orchestra.run.smithi029.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T00:29:47.948 INFO:teuthology.orchestra.run.smithi029.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T00:29:47.948 INFO:teuthology.orchestra.run.smithi029.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T00:29:47.951 INFO:teuthology.orchestra.run.smithi029.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T00:29:47.951 INFO:teuthology.orchestra.run.smithi029.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T00:29:47.956 INFO:teuthology.orchestra.run.smithi029.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T00:29:47.958 INFO:teuthology.orchestra.run.smithi029.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T00:29:47.960 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T00:29:47.966 INFO:teuthology.orchestra.run.smithi029.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T00:29:47.966 INFO:teuthology.orchestra.run.smithi029.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T00:29:47.990 INFO:teuthology.orchestra.run.smithi029.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T00:29:47.990 INFO:teuthology.orchestra.run.smithi029.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T00:29:47.999 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:29:48.000 INFO:teuthology.orchestra.run.smithi029.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T00:29:48.023 INFO:teuthology.orchestra.run.smithi029.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T00:29:48.027 INFO:teuthology.orchestra.run.smithi029.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T00:29:48.027 INFO:teuthology.orchestra.run.smithi029.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T00:29:48.028 INFO:teuthology.orchestra.run.smithi029.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T00:29:48.028 INFO:teuthology.orchestra.run.smithi029.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T00:29:48.042 INFO:teuthology.orchestra.run.smithi029.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T00:29:48.042 INFO:teuthology.orchestra.run.smithi029.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T00:29:48.327 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T00:29:48.355 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:29:48.369 INFO:teuthology.orchestra.run.smithi029.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T00:29:48.379 INFO:teuthology.orchestra.run.smithi029.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T00:29:48.382 INFO:teuthology.orchestra.run.smithi029.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T00:29:48.384 INFO:teuthology.orchestra.run.smithi029.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T00:29:48.387 INFO:teuthology.orchestra.run.smithi029.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T00:29:48.390 INFO:teuthology.orchestra.run.smithi029.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T00:29:48.419 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:29:48.470 INFO:teuthology.orchestra.run.smithi029.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T00:29:48.479 INFO:teuthology.orchestra.run.smithi029.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T00:29:48.714 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T00:29:48.741 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:29:48.772 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:29:48.914 INFO:teuthology.orchestra.run.smithi029.stdout:Preconfiguring packages ... 2024-04-25T00:29:49.109 INFO:teuthology.orchestra.run.smithi029.stdout:Fetched 55.7 MB in 1s (45.4 MB/s) 2024-04-25T00:29:49.133 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T00:29:49.161 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:29:49.200 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:29:49.430 INFO:teuthology.orchestra.run.smithi029.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T00:29:49.436 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:29:49.503 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T00:29:49.530 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:29:49.569 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:29:49.589 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:29:49.788 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T00:29:49.816 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:29:49.846 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:29:50.359 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T00:29:50.371 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T00:29:50.386 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T00:29:50.425 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:29:50.720 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package modemmanager. 2024-04-25T00:29:50.736 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:29:50.770 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:29:51.359 INFO:teuthology.orchestra.run.smithi029.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T00:29:51.486 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package network-manager. 2024-04-25T00:29:51.513 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:29:51.561 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:29:51.645 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package flex. 2024-04-25T00:29:51.742 INFO:teuthology.orchestra.run.smithi029.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T00:29:51.748 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T00:29:51.793 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T00:29:52.365 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:29:52.371 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ppp. 2024-04-25T00:29:52.398 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T00:29:52.539 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T00:29:52.682 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:29:53.001 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T00:29:53.029 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T00:29:53.039 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:29:53.067 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T00:29:53.320 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T00:29:53.348 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T00:29:53.386 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T00:29:53.639 INFO:teuthology.orchestra.run.smithi029.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T00:29:53.645 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:29:53.689 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T00:29:53.717 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T00:29:53.756 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T00:29:53.840 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:29:54.035 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T00:29:54.063 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:29:54.101 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:29:54.264 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:29:54.404 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:29:54.706 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T00:29:54.734 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T00:29:54.792 INFO:teuthology.orchestra.run.smithi157.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T00:29:54.812 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-common (238) ... 2024-04-25T00:29:54.957 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:29:55.278 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T00:29:55.306 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:29:55.359 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:29:55.375 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package bison. 2024-04-25T00:29:55.482 INFO:teuthology.orchestra.run.smithi029.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T00:29:55.488 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T00:29:55.527 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:29:55.942 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T00:29:55.970 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T00:29:56.017 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:29:56.374 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package dump. 2024-04-25T00:29:56.402 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T00:29:56.450 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T00:29:56.700 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql. 2024-04-25T00:29:56.729 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T00:29:56.741 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package indent. 2024-04-25T00:29:56.767 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql (14+238) ... 2024-04-25T00:29:56.769 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:29:56.808 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:29:57.011 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T00:29:57.027 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T00:29:57.069 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T00:29:57.104 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T00:29:57.133 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T00:29:57.179 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:29:57.313 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T00:29:57.329 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T00:29:57.371 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T00:29:57.521 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T00:29:57.550 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T00:29:57.597 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:29:57.640 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T00:29:57.657 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T00:29:57.699 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:29:57.886 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T00:29:57.903 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T00:29:57.945 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:29:57.968 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package tcl. 2024-04-25T00:29:57.996 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T00:29:58.040 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T00:29:58.319 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T00:29:58.348 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T00:29:58.371 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T00:29:58.386 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:29:58.399 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T00:29:58.437 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:29:58.641 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T00:29:58.658 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T00:29:58.699 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:29:58.792 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T00:29:58.809 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T00:29:58.842 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T00:29:59.064 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T00:29:59.081 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T00:29:59.123 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T00:29:59.187 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T00:29:59.215 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T00:29:59.253 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:29:59.446 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T00:29:59.475 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T00:29:59.521 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:29:59.549 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T00:29:59.759 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T00:29:59.885 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:29:59.986 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:30:00.087 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:30:00.214 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:30:00.342 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:30:00.808 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T00:30:00.904 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:30:01.263 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:30:01.263 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:30:01.773 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:30:01.883 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:30:01.894 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T00:30:01.923 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:30:01.961 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:30:02.026 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:30:02.168 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T00:30:02.251 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T00:30:02.279 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:30:02.294 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:30:02.318 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:30:02.420 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:30:02.546 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:30:02.658 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T00:30:02.672 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:30:02.686 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:30:02.733 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:30:03.105 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T00:30:03.134 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T00:30:03.180 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:30:03.294 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:30:03.437 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T00:30:03.520 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T00:30:03.530 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T00:30:03.549 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:30:03.587 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:30:03.597 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:30:03.723 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:30:03.832 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T00:30:03.933 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T00:30:04.043 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:30:04.043 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T00:30:04.072 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:30:04.127 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:30:04.133 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T00:30:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:30:04.329 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:30:04.505 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:30:04.582 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T00:30:04.611 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:30:04.631 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:30:04.666 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:30:04.757 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T00:30:04.908 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T00:30:05.009 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:30:05.336 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T00:30:05.365 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:30:05.419 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:30:05.484 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:05.484 INFO:teuthology.orchestra.run.smithi157.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T00:30:05.485 INFO:teuthology.orchestra.run.smithi157.stdout:which means they are currently configured by ifupdown: 2024-04-25T00:30:05.485 INFO:teuthology.orchestra.run.smithi157.stdout:- enp3s0f1 2024-04-25T00:30:05.485 INFO:teuthology.orchestra.run.smithi157.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T00:30:05.485 INFO:teuthology.orchestra.run.smithi157.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T00:30:05.485 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:05.602 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T00:30:05.734 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T00:30:05.762 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:30:05.817 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:30:05.945 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T00:30:06.265 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T00:30:06.534 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T00:30:06.562 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T00:30:06.617 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:30:06.770 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:30:06.921 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:30:07.022 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T00:30:07.141 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package modemmanager. 2024-04-25T00:30:07.170 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:30:07.224 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:30:07.790 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-common (238) ... 2024-04-25T00:30:07.930 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package network-manager. 2024-04-25T00:30:07.959 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:30:08.014 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:30:08.781 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ppp. 2024-04-25T00:30:08.802 INFO:teuthology.orchestra.run.smithi157.stdout:Adding user postgres to group ssl-cert 2024-04-25T00:30:08.809 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T00:30:09.032 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:09.032 INFO:teuthology.orchestra.run.smithi157.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T00:30:09.080 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:30:09.225 INFO:teuthology.orchestra.run.smithi157.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T00:30:09.226 INFO:teuthology.orchestra.run.smithi157.stdout:Removing obsolete dictionary files: 2024-04-25T00:30:09.433 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T00:30:09.461 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T00:30:09.500 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T00:30:09.747 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T00:30:09.775 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T00:30:09.801 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T00:30:09.874 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T00:30:10.159 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:30:10.492 INFO:teuthology.orchestra.run.smithi157.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T00:30:10.506 INFO:teuthology.orchestra.run.smithi157.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T00:30:10.556 INFO:teuthology.orchestra.run.smithi157.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T00:30:10.556 INFO:teuthology.orchestra.run.smithi157.stdout:This user must also own the server process. 2024-04-25T00:30:10.556 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:10.556 INFO:teuthology.orchestra.run.smithi157.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T00:30:10.556 INFO:teuthology.orchestra.run.smithi157.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T00:30:10.557 INFO:teuthology.orchestra.run.smithi157.stdout:The default text search configuration will be set to "english". 2024-04-25T00:30:10.557 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:10.557 INFO:teuthology.orchestra.run.smithi157.stdout:Data page checksums are disabled. 2024-04-25T00:30:10.557 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:10.557 INFO:teuthology.orchestra.run.smithi157.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T00:30:10.557 INFO:teuthology.orchestra.run.smithi157.stdout:creating subdirectories ... ok 2024-04-25T00:30:10.557 INFO:teuthology.orchestra.run.smithi157.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T00:30:10.571 INFO:teuthology.orchestra.run.smithi157.stdout:selecting default max_connections ... 100 2024-04-25T00:30:10.591 INFO:teuthology.orchestra.run.smithi157.stdout:selecting default shared_buffers ... 128MB 2024-04-25T00:30:10.602 INFO:teuthology.orchestra.run.smithi157.stdout:selecting default time zone ... Etc/UTC 2024-04-25T00:30:10.602 INFO:teuthology.orchestra.run.smithi157.stdout:creating configuration files ... ok 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi157.stdout:running bootstrap script ... ok 2024-04-25T00:30:10.746 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T00:30:10.775 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T00:30:10.813 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T00:30:11.130 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T00:30:11.158 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:30:11.204 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:30:11.386 INFO:teuthology.orchestra.run.smithi157.stdout:performing post-bootstrap initialization ... ok 2024-04-25T00:30:11.815 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T00:30:11.843 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T00:30:11.909 INFO:teuthology.orchestra.run.smithi029.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T00:30:11.947 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking postgresql-common (238) ... 2024-04-25T00:30:12.062 INFO:teuthology.orchestra.run.smithi157.stdout:syncing data to disk ... ok 2024-04-25T00:30:12.444 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T00:30:12.581 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T00:30:12.610 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:30:12.660 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:30:13.955 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package postgresql. 2024-04-25T00:30:13.985 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T00:30:14.030 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking postgresql (14+238) ... 2024-04-25T00:30:14.302 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T00:30:14.331 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T00:30:14.377 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T00:30:14.640 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T00:30:14.668 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T00:30:14.715 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T00:30:15.063 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T00:30:15.092 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T00:30:15.129 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:30:15.157 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:30:15.291 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T00:30:15.417 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql (14+238) ... 2024-04-25T00:30:15.418 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package tcl. 2024-04-25T00:30:15.446 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T00:30:15.498 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T00:30:15.535 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:30:15.644 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:30:15.858 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T00:30:15.880 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:30:15.880 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:30:15.886 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T00:30:15.924 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:30:16.300 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T00:30:16.329 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T00:30:16.375 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T00:30:16.422 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:30:16.514 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T00:30:16.758 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T00:30:16.786 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T00:30:16.817 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:30:16.841 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:30:16.968 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:30:17.177 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T00:30:17.387 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T00:30:17.514 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:30:17.643 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:30:17.770 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:30:17.896 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:30:18.040 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:30:18.288 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T00:30:18.380 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:30:18.558 INFO:teuthology.orchestra.run.smithi029.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T00:30:18.654 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:30:18.862 INFO:teuthology.orchestra.run.smithi157.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:30:18.868 DEBUG:teuthology.parallel:result is None 2024-04-25T00:30:19.042 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:30:19.043 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:30:19.534 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:30:19.661 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:30:19.781 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:30:19.917 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T00:30:20.043 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:30:20.178 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:30:20.307 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:30:20.434 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:30:22.021 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:30:22.148 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T00:30:22.242 INFO:teuthology.orchestra.run.smithi029.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T00:30:22.310 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:30:22.446 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:30:22.572 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T00:30:22.699 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T00:30:22.826 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:30:22.919 INFO:teuthology.orchestra.run.smithi029.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T00:30:22.989 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:30:23.116 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:30:23.293 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:30:23.422 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:30:23.549 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T00:30:23.718 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T00:30:23.845 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:30:24.314 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:24.314 INFO:teuthology.orchestra.run.smithi029.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T00:30:24.315 INFO:teuthology.orchestra.run.smithi029.stdout:which means they are currently configured by ifupdown: 2024-04-25T00:30:24.315 INFO:teuthology.orchestra.run.smithi029.stdout:- ens1f0 2024-04-25T00:30:24.315 INFO:teuthology.orchestra.run.smithi029.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T00:30:24.315 INFO:teuthology.orchestra.run.smithi029.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T00:30:24.315 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:24.446 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T00:30:24.776 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T00:30:25.099 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T00:30:25.627 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:30:25.779 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:30:25.908 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T00:30:26.035 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up postgresql-common (238) ... 2024-04-25T00:30:26.966 INFO:teuthology.orchestra.run.smithi029.stdout:Adding user postgres to group ssl-cert 2024-04-25T00:30:27.216 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:27.216 INFO:teuthology.orchestra.run.smithi029.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T00:30:27.440 INFO:teuthology.orchestra.run.smithi029.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T00:30:27.440 INFO:teuthology.orchestra.run.smithi029.stdout:Removing obsolete dictionary files: 2024-04-25T00:30:28.003 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T00:30:28.448 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:30:28.790 INFO:teuthology.orchestra.run.smithi029.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T00:30:28.805 INFO:teuthology.orchestra.run.smithi029.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T00:30:28.859 INFO:teuthology.orchestra.run.smithi029.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T00:30:28.859 INFO:teuthology.orchestra.run.smithi029.stdout:This user must also own the server process. 2024-04-25T00:30:28.859 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:28.859 INFO:teuthology.orchestra.run.smithi029.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T00:30:28.860 INFO:teuthology.orchestra.run.smithi029.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T00:30:28.860 INFO:teuthology.orchestra.run.smithi029.stdout:The default text search configuration will be set to "english". 2024-04-25T00:30:28.860 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:28.860 INFO:teuthology.orchestra.run.smithi029.stdout:Data page checksums are disabled. 2024-04-25T00:30:28.860 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:28.860 INFO:teuthology.orchestra.run.smithi029.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T00:30:28.860 INFO:teuthology.orchestra.run.smithi029.stdout:creating subdirectories ... ok 2024-04-25T00:30:28.860 INFO:teuthology.orchestra.run.smithi029.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T00:30:28.872 INFO:teuthology.orchestra.run.smithi029.stdout:selecting default max_connections ... 100 2024-04-25T00:30:28.894 INFO:teuthology.orchestra.run.smithi029.stdout:selecting default shared_buffers ... 128MB 2024-04-25T00:30:28.905 INFO:teuthology.orchestra.run.smithi029.stdout:selecting default time zone ... Etc/UTC 2024-04-25T00:30:28.905 INFO:teuthology.orchestra.run.smithi029.stdout:creating configuration files ... ok 2024-04-25T00:30:29.018 INFO:teuthology.orchestra.run.smithi029.stdout:running bootstrap script ... ok 2024-04-25T00:30:29.920 INFO:teuthology.orchestra.run.smithi029.stdout:performing post-bootstrap initialization ... ok 2024-04-25T00:30:30.791 INFO:teuthology.orchestra.run.smithi029.stdout:syncing data to disk ... ok 2024-04-25T00:30:31.212 INFO:teuthology.orchestra.run.smithi029.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T00:30:34.055 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:30:34.182 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T00:30:34.308 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up postgresql (14+238) ... 2024-04-25T00:30:34.444 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:30:34.570 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:30:34.834 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:30:34.834 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:30:35.435 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:30:35.536 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T00:30:35.900 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:30:36.062 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:30:37.827 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T00:30:37.946 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:30:38.475 INFO:teuthology.orchestra.run.smithi029.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:30:38.480 DEBUG:teuthology.parallel:result is None 2024-04-25T00:30:38.481 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:30:38.621 DEBUG:teuthology.orchestra.run.smithi029:> dpkg-query -W -f '${Version}' ceph 2024-04-25T00:30:38.640 INFO:teuthology.orchestra.run.smithi029.stdout:19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:30:38.641 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:30:38.641 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445-gd2c45d97-1jammy is installed. 2024-04-25T00:30:38.643 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:30:38.783 DEBUG:teuthology.orchestra.run.smithi157:> dpkg-query -W -f '${Version}' ceph 2024-04-25T00:30:38.802 INFO:teuthology.orchestra.run.smithi157.stdout:19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:30:38.803 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:30:38.803 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445-gd2c45d97-1jammy is installed. 2024-04-25T00:30:38.804 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T00:30:38.805 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:30:38.805 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:30:38.820 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:30:38.820 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:30:38.860 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T00:30:38.860 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:30:38.860 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:30:38.877 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:30:38.932 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:30:38.933 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:30:38.946 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:30:39.002 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T00:30:39.003 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:30:39.003 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:30:39.016 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:30:39.071 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:30:39.072 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:30:39.084 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:30:39.139 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T00:30:39.139 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:30:39.139 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:30:39.153 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:30:39.208 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:30:39.208 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:30:39.220 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:30:39.274 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T00:30:39.360 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T00:30:39.360 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 777 /var/log/ceph 2024-04-25T00:30:39.362 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /var/log/ceph 2024-04-25T00:30:39.374 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T00:30:39.375 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T00:30:39.417 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T00:30:39.429 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T00:30:39.429 DEBUG:teuthology.orchestra.run.smithi029:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T00:30:39.477 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T00:30:39.496 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T00:30:39.496 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T00:30:39.496 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T00:30:39.497 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:30:39.497 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T00:30:39.541 DEBUG:teuthology.orchestra.run.smithi029:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T00:30:39.616 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:30:39.617 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T00:30:39.622 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T00:30:39.704 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T00:30:39.704 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T00:30:39.704 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260', 'branch': 'squid', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672260', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8528, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '122/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi029.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM7bw2OTJadZ4T9ESUbGrYLKBFA6yciK7RQICbMNGNp5tmDdwVix8ruHnJfXtgCqqjQKREAy3w0M94gZlG6d3e0=', 'smithi157.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbTiAvhgqE7jG79pNqQqg9j60hLL1U0CV4mIIEny+V73yHLRLiZZcJv9SbEVhDHEzTWoYyZa9unNBGk+KyPR8c='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': ['/home/teuthology/src/git.ceph.com_ceph_squid/qa/suites/fs/volumes/overrides/pg_health.yaml'], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T00:30:39.705 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T00:30:39.707 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T00:30:39.747 DEBUG:teuthology.orchestra.run.smithi029:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T00:30:39.753 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T00:30:39.803 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:30:39.804 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:30:39.812 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:30:39.813 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_1 2024-04-25T00:30:39.861 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:30:39.861 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:30:39.861 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 873 Links: 1 2024-04-25T00:30:39.861 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:30:39.861 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-04-25 00:23:55.480036606 +0000 2024-04-25T00:30:39.861 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-04-25 00:23:55.240038750 +0000 2024-04-25T00:30:39.861 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-04-25 00:23:55.240038750 +0000 2024-04-25T00:30:39.861 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2024-04-25T00:30:39.862 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:30:39.915 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-25T00:30:39.915 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-25T00:30:39.915 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000287359 s, 1.8 MB/s 2024-04-25T00:30:39.916 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:30:39.966 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_2 2024-04-25T00:30:40.016 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:30:40.016 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:30:40.017 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 897 Links: 1 2024-04-25T00:30:40.017 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:30:40.017 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-04-25 00:23:55.984032104 +0000 2024-04-25T00:30:40.017 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-04-25 00:23:55.708034569 +0000 2024-04-25T00:30:40.017 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-04-25 00:23:55.708034569 +0000 2024-04-25T00:30:40.017 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2024-04-25T00:30:40.017 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:30:40.075 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-25T00:30:40.075 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-25T00:30:40.075 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000306679 s, 1.7 MB/s 2024-04-25T00:30:40.077 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:30:40.126 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_3 2024-04-25T00:30:40.177 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:30:40.177 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:30:40.177 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 908 Links: 1 2024-04-25T00:30:40.177 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:30:40.177 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-04-25 00:23:56.448027960 +0000 2024-04-25T00:30:40.177 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-04-25 00:23:56.200030175 +0000 2024-04-25T00:30:40.177 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-04-25 00:23:56.200030175 +0000 2024-04-25T00:30:40.177 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2024-04-25T00:30:40.177 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:30:40.231 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-25T00:30:40.231 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-25T00:30:40.231 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000284865 s, 1.8 MB/s 2024-04-25T00:30:40.232 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:30:40.282 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_4 2024-04-25T00:30:40.329 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:30:40.329 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:30:40.329 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 925 Links: 1 2024-04-25T00:30:40.329 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:30:40.329 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-04-25 00:23:56.980023207 +0000 2024-04-25T00:30:40.329 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-04-25 00:23:56.680025888 +0000 2024-04-25T00:30:40.329 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-04-25 00:23:56.680025888 +0000 2024-04-25T00:30:40.329 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2024-04-25T00:30:40.329 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:30:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-25T00:30:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-25T00:30:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000245981 s, 2.1 MB/s 2024-04-25T00:30:40.388 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:30:40.442 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:30:40.442 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:30:40.442 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:30:40.448 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:30:40.448 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-04-25T00:30:40.496 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:30:40.496 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:30:40.496 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 891 Links: 1 2024-04-25T00:30:40.496 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:30:40.496 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-25 00:23:43.689292958 +0000 2024-04-25T00:30:40.496 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-25 00:23:43.441295172 +0000 2024-04-25T00:30:40.497 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-25 00:23:43.441295172 +0000 2024-04-25T00:30:40.497 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-04-25T00:30:40.497 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:30:40.550 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-25T00:30:40.550 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-25T00:30:40.550 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000316394 s, 1.6 MB/s 2024-04-25T00:30:40.551 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:30:40.600 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-04-25T00:30:40.647 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:30:40.648 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:30:40.648 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 910 Links: 1 2024-04-25T00:30:40.648 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:30:40.648 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-25 00:23:44.153288815 +0000 2024-04-25T00:30:40.648 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-25 00:23:43.905291029 +0000 2024-04-25T00:30:40.648 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-25 00:23:43.905291029 +0000 2024-04-25T00:30:40.648 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-04-25T00:30:40.648 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:30:40.702 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-25T00:30:40.702 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-25T00:30:40.702 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000319618 s, 1.6 MB/s 2024-04-25T00:30:40.703 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:30:40.752 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-04-25T00:30:40.799 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:30:40.799 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:30:40.799 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-04-25T00:30:40.799 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:30:40.799 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-25 00:23:44.649284387 +0000 2024-04-25T00:30:40.799 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-25 00:23:44.389286709 +0000 2024-04-25T00:30:40.800 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-25 00:23:44.389286709 +0000 2024-04-25T00:30:40.800 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-04-25T00:30:40.800 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:30:40.854 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-25T00:30:40.854 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-25T00:30:40.854 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000306061 s, 1.7 MB/s 2024-04-25T00:30:40.855 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:30:40.904 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-04-25T00:30:40.951 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:30:40.951 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:30:40.951 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 939 Links: 1 2024-04-25T00:30:40.951 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:30:40.951 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-25 00:23:45.129280102 +0000 2024-04-25T00:30:40.951 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-25 00:23:44.873282387 +0000 2024-04-25T00:30:40.952 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-25 00:23:44.873282387 +0000 2024-04-25T00:30:40.952 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-04-25T00:30:40.952 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:30:41.005 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-25T00:30:41.005 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-25T00:30:41.005 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000301618 s, 1.7 MB/s 2024-04-25T00:30:41.007 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:30:41.056 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:30:41.056 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi029.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T00:30:41.057 INFO:tasks.ceph:Generating config... 2024-04-25T00:30:41.058 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T00:30:41.058 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T00:30:41.058 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T00:30:41.058 INFO:tasks.ceph:[client] pid file = 2024-04-25T00:30:41.058 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T00:30:41.058 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T00:30:41.058 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T00:30:41.058 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T00:30:41.058 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T00:30:41.059 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T00:30:41.060 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T00:30:41.060 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T00:30:41.061 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T00:30:41.061 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T00:30:41.061 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T00:30:41.097 INFO:teuthology.orchestra.run.smithi029.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T00:30:41.118 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T00:30:41.195 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T00:30:41.252 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.29'), ('mon.b', '172.21.15.157'), ('mon.c', '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]')] 2024-04-25T00:30:41.253 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.29,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T00:30:41.253 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:30:41.254 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T00:30:41.301 DEBUG:teuthology.orchestra.run.smithi029:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.29 --add b 172.21.15.157 --addv c '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:30:41.391 INFO:teuthology.orchestra.run.smithi029.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:30:41.391 INFO:teuthology.orchestra.run.smithi029.stdout:monmaptool: generated fsid 59afbe7d-aaa8-45e2-b468-69ee150cacea 2024-04-25T00:30:41.391 INFO:teuthology.orchestra.run.smithi029.stdout:setting min_mon_release = quincy 2024-04-25T00:30:41.392 INFO:teuthology.orchestra.run.smithi029.stdout:epoch 0 2024-04-25T00:30:41.392 INFO:teuthology.orchestra.run.smithi029.stdout:fsid 59afbe7d-aaa8-45e2-b468-69ee150cacea 2024-04-25T00:30:41.392 INFO:teuthology.orchestra.run.smithi029.stdout:last_changed 2024-04-25T00:30:41.393494+0000 2024-04-25T00:30:41.392 INFO:teuthology.orchestra.run.smithi029.stdout:created 2024-04-25T00:30:41.393494+0000 2024-04-25T00:30:41.392 INFO:teuthology.orchestra.run.smithi029.stdout:min_mon_release 17 (quincy) 2024-04-25T00:30:41.392 INFO:teuthology.orchestra.run.smithi029.stdout:election_strategy: 1 2024-04-25T00:30:41.392 INFO:teuthology.orchestra.run.smithi029.stdout:0: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.a 2024-04-25T00:30:41.392 INFO:teuthology.orchestra.run.smithi029.stdout:1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.b 2024-04-25T00:30:41.392 INFO:teuthology.orchestra.run.smithi029.stdout:2: [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] mon.c 2024-04-25T00:30:41.392 INFO:teuthology.orchestra.run.smithi029.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T00:30:41.393 DEBUG:teuthology.orchestra.run.smithi029:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T00:30:41.441 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 59afbe7d-aaa8-45e2-b468-69ee150cacea... 2024-04-25T00:30:41.442 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:30:41.485 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:30:41.516 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2024-04-25T00:30:41.517 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2024-04-25T00:30:41.517 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:30:41.517 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2024-04-25T00:30:41.517 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.517 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2024-04-25T00:30:41.517 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.517 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2024-04-25T00:30:41.517 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.517 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2024-04-25T00:30:41.517 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2024-04-25T00:30:41.517 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.517 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2024-04-25T00:30:41.517 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:30:41.518 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2024-04-25T00:30:41.519 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.29,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 59afbe7d-aaa8-45e2-b468-69ee150cacea 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2024-04-25T00:30:41.520 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug trim objects = true 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.521 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: bdev async discard = True 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: bdev enable discard = True 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore allocator = bitmap 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 1/20 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 1/20 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 4/10 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:30:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: debug finisher = 20 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2024-04-25T00:30:41.523 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.524 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: mon down mkfs grace = 300 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:30:41.525 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-04-25T00:30:41.526 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2024-04-25T00:30:41.527 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2024-04-25T00:30:41.527 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2024-04-25T00:30:41.527 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-25T00:30:41.527 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2024-04-25T00:30:41.527 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2024-04-25T00:30:41.527 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2024-04-25T00:30:41.527 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2024-04-25T00:30:41.527 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-04-25T00:30:41.527 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-04-25T00:30:41.527 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-04-25T00:30:41.529 INFO:teuthology.orchestra.run.smithi029.stdout:[global] 2024-04-25T00:30:41.529 INFO:teuthology.orchestra.run.smithi029.stdout: chdir = "" 2024-04-25T00:30:41.529 INFO:teuthology.orchestra.run.smithi029.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:30:41.529 INFO:teuthology.orchestra.run.smithi029.stdout: auth supported = cephx 2024-04-25T00:30:41.529 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.529 INFO:teuthology.orchestra.run.smithi029.stdout: filestore xattr use omap = true 2024-04-25T00:30:41.529 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.529 INFO:teuthology.orchestra.run.smithi029.stdout: mon clock drift allowed = 1.000 2024-04-25T00:30:41.529 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: osd crush chooseleaf type = 0 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: auth debug = true 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: ms die on old message = true 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: ms die on bug = true 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: mon pg warn max object skew = 0 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: osd pool default size = 2 2024-04-25T00:30:41.530 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd allow primary affinity = true 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd allow pg remap = true 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on no sortbitwise = false 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on too few osds = false 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: mon_allow_pool_size_one = true 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: osd default data pool replay window = 5 2024-04-25T00:30:41.531 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: mon allow pool delete = true 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: mon cluster log file level = debug 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: debug asserts on shutdown = true 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: mon health detail to clog = false 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: mon host = "172.21.15.29,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: fsid = 59afbe7d-aaa8-45e2-b468-69ee150cacea 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout:[osd] 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: osd journal size = 100 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: osd scrub load threshold = 5.0 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: osd scrub max interval = 600 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:30:41.532 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.533 INFO:teuthology.orchestra.run.smithi029.stdout: osd recover clone overlap = true 2024-04-25T00:30:41.533 INFO:teuthology.orchestra.run.smithi029.stdout: osd recovery max chunk = 1048576 2024-04-25T00:30:41.533 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.533 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug shutdown = true 2024-04-25T00:30:41.533 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug op order = true 2024-04-25T00:30:41.533 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug verify stray on activate = true 2024-04-25T00:30:41.533 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug trim objects = true 2024-04-25T00:30:41.533 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.533 INFO:teuthology.orchestra.run.smithi029.stdout: osd open classes on start = true 2024-04-25T00:30:41.533 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug pg log writeout = true 2024-04-25T00:30:41.533 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.533 INFO:teuthology.orchestra.run.smithi029.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:30:41.533 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: osd map max advance = 10 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: journal zero on create = true 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: filestore apply finisher threads = 3 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: bdev debug aio = true 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug misdirected ops = true 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: bdev async discard = True 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: bdev enable discard = True 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore allocator = bitmap 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore block size = 96636764160 2024-04-25T00:30:41.534 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore fsck on mount = True 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: debug bluefs = 1/20 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: debug bluestore = 1/20 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: debug osd = 20 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: debug rocksdb = 4/10 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd full ratio = 0.9 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: osd objectstore = bluestore 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: osd op complaint time = 180 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr] 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: debug mgr = 20 2024-04-25T00:30:41.535 INFO:teuthology.orchestra.run.smithi029.stdout: debug mon = 20 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: debug auth = 20 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: mgr/telemetry/nag = false 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: debug client = 20 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: debug finisher = 20 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout:[mon] 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: debug mon = 20 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: debug paxos = 20 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: debug auth = 20 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: mon data avail warn = 5 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: mon mgr mkfs grace = 240 2024-04-25T00:30:41.536 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd reporter subtree level = osd 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd prime pg temp = true 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: # 1m isn't quite enough 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:30:41.537 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: mon down mkfs grace = 300 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: mon op complaint time = 120 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout:[client] 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: rgw cache enabled = true 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: rgw enable ops log = true 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: rgw enable usage log = true 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: client mount timeout = 600 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: debug client = 20 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: pid file = "" 2024-04-25T00:30:41.538 INFO:teuthology.orchestra.run.smithi029.stdout: rados mon op timeout = 900 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout: rados osd op timeout = 900 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.a] 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.b] 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.c] 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.a] 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.c] 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.b] 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.d] 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout:[mds] 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout: debug mds = 20 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout: debug mds balancer = 20 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout: mds debug frag = True 2024-04-25T00:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout: mds debug scatterstat = True 2024-04-25T00:30:41.540 INFO:teuthology.orchestra.run.smithi029.stdout: mds op complaint time = 180 2024-04-25T00:30:41.540 INFO:teuthology.orchestra.run.smithi029.stdout: mds verify scatter = True 2024-04-25T00:30:41.540 INFO:teuthology.orchestra.run.smithi029.stdout: osd op complaint time = 180 2024-04-25T00:30:41.540 INFO:teuthology.orchestra.run.smithi029.stdout: rados mon op timeout = 900 2024-04-25T00:30:41.540 INFO:teuthology.orchestra.run.smithi029.stdout: rados osd op timeout = 900 2024-04-25T00:30:41.540 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T00:30:41.540 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T00:30:41.615 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T00:30:41.615 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:30:41.615 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T00:30:41.661 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:30:41.661 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T00:30:41.709 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:30:41.709 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:30:41.709 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T00:30:41.710 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T00:30:41.773 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:30:41.773 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:30:41.821 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:30:41.821 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:30:41.821 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T00:30:41.821 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T00:30:41.844 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:30:41.844 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:30:41.892 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T00:30:41.892 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T00:30:41.892 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T00:30:41.939 INFO:teuthology.orchestra.run.smithi029.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T00:30:41.941 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T00:30:41.985 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T00:30:42.005 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T00:30:42.005 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T00:30:42.049 INFO:teuthology.orchestra.run.smithi029.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T00:30:42.052 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T00:30:42.109 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T00:30:42.190 INFO:teuthology.orchestra.run.smithi029.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T00:30:42.193 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T00:30:42.250 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T00:30:42.296 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T00:30:42.299 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T00:30:42.355 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T00:30:42.438 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T00:30:42.441 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T00:30:42.498 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T00:30:42.498 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T00:30:42.537 INFO:teuthology.orchestra.run.smithi029.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T00:30:42.547 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T00:30:42.619 INFO:teuthology.orchestra.run.smithi029.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T00:30:42.629 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T00:30:42.665 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T00:30:42.676 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T00:30:42.755 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T00:30:42.768 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T00:30:42.769 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi029.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T00:30:42.769 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T00:30:42.784 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:30:42.785 INFO:tasks.ceph:role: osd.0 2024-04-25T00:30:42.785 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:30:42.785 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T00:30:42.840 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:30:42.840 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:30:42.840 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:30:42.841 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:30:42.841 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:30:42.841 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:30:42.841 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:30:42.841 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:30:42.841 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:30:42.841 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:30:42.864 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-04-25T00:30:42.866 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-04-25T00:30:42.866 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T00:30:42.924 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T00:30:42.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:30:42.977 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:30:42.978 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T00:30:43.030 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:30:43.030 INFO:tasks.ceph:role: osd.1 2024-04-25T00:30:43.031 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:30:43.031 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T00:30:43.084 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:30:43.085 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:30:43.085 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:30:43.085 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:30:43.085 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:30:43.085 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:30:43.085 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:30:43.085 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:30:43.085 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:30:43.085 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:30:43.111 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-04-25T00:30:43.113 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-04-25T00:30:43.113 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T00:30:43.174 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T00:30:43.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:30:43.226 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:30:43.227 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T00:30:43.280 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:30:43.280 INFO:tasks.ceph:role: osd.2 2024-04-25T00:30:43.280 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:30:43.280 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T00:30:43.337 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:30:43.338 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:30:43.338 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:30:43.338 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:30:43.338 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:30:43.338 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:30:43.338 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:30:43.338 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:30:43.339 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:30:43.339 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:30:43.360 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-04-25T00:30:43.362 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-04-25T00:30:43.362 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T00:30:43.421 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T00:30:43.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:30:43.474 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:30:43.475 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T00:30:43.532 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:30:43.532 INFO:tasks.ceph:role: osd.3 2024-04-25T00:30:43.533 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:30:43.533 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T00:30:43.589 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:30:43.589 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:30:43.589 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:30:43.589 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:30:43.589 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:30:43.589 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:30:43.590 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:30:43.590 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:30:43.590 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:30:43.590 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:30:43.613 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-04-25T00:30:43.615 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-04-25T00:30:43.615 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T00:30:43.672 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T00:30:43.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:30:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:30:43.725 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:30:43.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:43.838+0000 7fad1d8b77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T00:30:43.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:43.838+0000 7fad1d8b77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T00:30:43.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:43.838+0000 7fad1d8b77c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T00:30:43.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:43.838+0000 7fad1d8b77c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T00:30:43.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:43.838+0000 7fad1d8b77c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T00:30:43.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:43.838+0000 7fad1d8b77c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T00:30:46.804 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T00:30:46.818 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:30:46.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:46.894+0000 7f333ae327c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T00:30:46.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:46.894+0000 7f333ae327c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T00:30:46.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:46.894+0000 7f333ae327c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T00:30:46.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:46.894+0000 7f333ae327c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T00:30:46.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:46.894+0000 7f333ae327c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T00:30:46.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:46.894+0000 7f333ae327c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T00:30:49.861 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T00:30:49.917 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:30:49.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:49.998+0000 7f1b65d557c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T00:30:49.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:49.998+0000 7f1b65d557c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T00:30:49.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:49.998+0000 7f1b65d557c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T00:30:49.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:49.998+0000 7f1b65d557c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T00:30:49.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:49.998+0000 7f1b65d557c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T00:30:49.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:49.998+0000 7f1b65d557c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T00:30:52.970 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T00:30:53.026 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:30:53.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:53.106+0000 7f0af89577c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T00:30:53.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:53.110+0000 7f0af89577c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T00:30:53.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:53.110+0000 7f0af89577c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T00:30:53.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:53.110+0000 7f0af89577c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T00:30:53.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:53.110+0000 7f0af89577c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T00:30:53.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:30:53.110+0000 7f0af89577c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T00:30:56.083 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T00:30:56.137 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T00:30:56.152 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:30:56.152 INFO:tasks.ceph:role: osd.4 2024-04-25T00:30:56.152 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:30:56.152 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T00:30:56.207 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:30:56.208 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:30:56.208 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:30:56.208 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:30:56.208 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:30:56.208 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:30:56.209 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:30:56.209 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:30:56.209 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:30:56.209 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:30:56.231 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-25T00:30:56.234 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-25T00:30:56.234 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T00:30:56.290 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T00:30:56.339 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:30:56.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:30:56.340 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T00:30:56.393 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:30:56.393 INFO:tasks.ceph:role: osd.5 2024-04-25T00:30:56.393 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:30:56.393 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T00:30:56.447 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:30:56.447 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:30:56.447 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:30:56.448 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:30:56.448 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:30:56.448 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:30:56.448 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:30:56.448 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:30:56.448 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:30:56.448 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:30:56.471 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-25T00:30:56.473 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-25T00:30:56.473 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T00:30:56.529 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T00:30:56.579 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:30:56.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:30:56.580 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T00:30:56.636 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:30:56.636 INFO:tasks.ceph:role: osd.6 2024-04-25T00:30:56.636 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:30:56.637 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T00:30:56.690 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:30:56.690 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:30:56.690 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:30:56.690 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:30:56.690 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:30:56.690 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:30:56.690 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:30:56.691 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:30:56.691 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:30:56.691 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:30:56.713 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-25T00:30:56.715 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-25T00:30:56.715 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T00:30:56.776 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T00:30:56.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:30:56.828 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:30:56.828 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T00:30:56.883 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:30:56.883 INFO:tasks.ceph:role: osd.7 2024-04-25T00:30:56.883 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:30:56.883 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T00:30:56.939 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:30:56.939 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:30:56.939 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:30:56.939 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:30:56.939 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:30:56.939 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:30:56.940 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:30:56.940 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:30:56.940 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:30:56.940 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:30:56.963 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-25T00:30:56.965 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-25T00:30:56.965 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T00:30:57.027 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T00:30:57.084 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:30:57.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:30:57.085 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:30:57.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:30:57.198+0000 7fe57aafc7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T00:30:57.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:30:57.198+0000 7fe57aafc7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T00:30:57.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:30:57.198+0000 7fe57aafc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T00:30:57.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:30:57.198+0000 7fe57aafc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T00:30:57.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:30:57.198+0000 7fe57aafc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T00:30:57.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:30:57.198+0000 7fe57aafc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T00:31:00.182 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T00:31:00.196 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:31:00.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:00.274+0000 7f8081cfd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T00:31:00.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:00.274+0000 7f8081cfd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T00:31:00.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:00.274+0000 7f8081cfd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T00:31:00.277 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:00.274+0000 7f8081cfd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T00:31:00.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:00.274+0000 7f8081cfd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T00:31:00.278 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:00.274+0000 7f8081cfd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T00:31:03.243 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T00:31:03.301 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:31:03.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:03.377+0000 7fd52e38e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T00:31:03.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:03.377+0000 7fd52e38e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T00:31:03.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:03.377+0000 7fd52e38e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T00:31:03.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:03.377+0000 7fd52e38e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T00:31:03.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:03.377+0000 7fd52e38e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T00:31:03.383 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:03.377+0000 7fd52e38e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T00:31:06.345 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T00:31:06.400 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:31:06.483 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:06.477+0000 7ff4704377c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T00:31:06.484 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:06.481+0000 7ff4704377c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T00:31:06.484 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:06.481+0000 7ff4704377c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T00:31:06.484 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:06.481+0000 7ff4704377c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T00:31:06.484 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:06.481+0000 7ff4704377c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T00:31:06.484 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:06.481+0000 7ff4704377c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T00:31:09.444 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T00:31:09.500 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T00:31:09.501 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:31:09.501 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T00:31:09.516 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:31:09.516 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T00:31:09.574 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:31:09.574 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T00:31:09.629 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:31:09.629 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T00:31:09.688 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:31:09.688 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T00:31:09.706 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:31:09.735 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:31:09.745 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:31:09.746 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T00:31:09.802 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:31:09.802 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T00:31:09.856 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:31:09.857 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T00:31:09.870 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:31:09.870 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T00:31:09.923 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:31:09.923 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T00:31:09.982 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:31:09.983 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T00:31:10.035 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:31:10.036 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T00:31:10.091 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:31:10.091 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T00:31:10.147 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:31:10.148 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T00:31:10.202 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:31:10.202 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T00:31:10.208 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:31:10.208 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T00:31:10.257 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:31:10.257 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T00:31:10.263 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:31:10.263 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T00:31:10.308 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T00:31:10.308 DEBUG:teuthology.orchestra.run.smithi029:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T00:31:10.310 DEBUG:teuthology.orchestra.run.smithi157:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T00:31:10.354 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr.y] 2024-04-25T00:31:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAxpClmeR4UOBAANs9SRolOm6E8+/4ciupk5w== 2024-04-25T00:31:10.354 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.a] 2024-04-25T00:31:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClmyzgKAxAAU+nrUJgQhDX6xlARNFI/ng== 2024-04-25T00:31:10.354 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.c] 2024-04-25T00:31:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClmYUl8CxAAj6pV1+s4a0rpWLq2EgZ+8w== 2024-04-25T00:31:10.354 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.0] 2024-04-25T00:31:10.354 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAzpClmudk6MhAA6YFyd57jwtB2pdOaad7bxQ== 2024-04-25T00:31:10.354 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.1] 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQA2pClm76q0NRAAxWhEP2ZGoFnZcuiQyGjglw== 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.2] 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQA6pClmcgULABAA2F5WBR9Df0v4Sn875AzF7g== 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.3] 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQA9pClm9QuRBhAALFLruv4JU/uNZYMX0yNuXw== 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr.x] 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAxpClm6WrdOhAAsY9FLCxG7PGpq2NCY4hGhQ== 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.b] 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClmtyzPERAAxNUDANw1H/ZzOo+dhgzxlg== 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.d] 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClmuAhKGhAAOLIcWUodW5TqtVgHWZl12A== 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.4] 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBBpClmNj4VDBAAxaxMvyaCvjynPY3f81udnw== 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.5] 2024-04-25T00:31:10.355 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBEpClmzBybEBAArry1lM4RIsM/txKfgBqDpw== 2024-04-25T00:31:10.356 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.6] 2024-04-25T00:31:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBHpClm5HfjFhAAbFwJCpbhegLWDu0mbS8FoA== 2024-04-25T00:31:10.356 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.7] 2024-04-25T00:31:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBKpClme/7oHBAAOR2iQ6CweexR7sC1PgesZw== 2024-04-25T00:31:10.356 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-04-25T00:31:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A== 2024-04-25T00:31:10.356 INFO:teuthology.orchestra.run.smithi029.stdout:[client.2] 2024-04-25T00:31:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClmGuQJJRAAhVd4D0jc6Whgyf+B6xgFMw== 2024-04-25T00:31:10.356 INFO:teuthology.orchestra.run.smithi029.stdout:[client.1] 2024-04-25T00:31:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClm8W/EJxAA0HZW24Hhcwf+soSZn6jHJg== 2024-04-25T00:31:10.356 INFO:teuthology.orchestra.run.smithi029.stdout:[client.3] 2024-04-25T00:31:10.356 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClmn+ciLRAAg4tQuZ13WP0RnWJaJq5fUQ== 2024-04-25T00:31:10.362 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.y] 2024-04-25T00:31:10.362 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAxpClmeR4UOBAANs9SRolOm6E8+/4ciupk5w== 2024-04-25T00:31:10.362 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAypClmyzgKAxAAU+nrUJgQhDX6xlARNFI/ng== 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAypClmYUl8CxAAj6pV1+s4a0rpWLq2EgZ+8w== 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.0] 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAzpClmudk6MhAA6YFyd57jwtB2pdOaad7bxQ== 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.1] 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQA2pClm76q0NRAAxWhEP2ZGoFnZcuiQyGjglw== 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.2] 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQA6pClmcgULABAA2F5WBR9Df0v4Sn875AzF7g== 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.3] 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQA9pClm9QuRBhAALFLruv4JU/uNZYMX0yNuXw== 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.x] 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAxpClm6WrdOhAAsY9FLCxG7PGpq2NCY4hGhQ== 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-04-25T00:31:10.363 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAypClmtyzPERAAxNUDANw1H/ZzOo+dhgzxlg== 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAypClmuAhKGhAAOLIcWUodW5TqtVgHWZl12A== 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.4] 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBBpClmNj4VDBAAxaxMvyaCvjynPY3f81udnw== 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.5] 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBEpClmzBybEBAArry1lM4RIsM/txKfgBqDpw== 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.6] 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBHpClm5HfjFhAAbFwJCpbhegLWDu0mbS8FoA== 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.7] 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBKpClme/7oHBAAOR2iQ6CweexR7sC1PgesZw== 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout:[client.0] 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A== 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout:[client.2] 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAypClmGuQJJRAAhVd4D0jc6Whgyf+B6xgFMw== 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout:[client.1] 2024-04-25T00:31:10.364 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAypClm8W/EJxAA0HZW24Hhcwf+soSZn6jHJg== 2024-04-25T00:31:10.365 INFO:teuthology.orchestra.run.smithi157.stdout:[client.3] 2024-04-25T00:31:10.365 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQAypClmn+ciLRAAg4tQuZ13WP0RnWJaJq5fUQ== 2024-04-25T00:31:10.365 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:31:10.402 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:31:10.451 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:31:10.453 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:31:10.544 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:31:10.546 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:31:10.590 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:10.592 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:10.670 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:10.678 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:10.752 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:10.762 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:10.834 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:10.850 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:10.918 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:31:10.920 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:31:11.000 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:31:11.006 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:31:11.086 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:31:11.094 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:31:11.169 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:11.171 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:11.250 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:11.254 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:11.331 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:11.333 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:11.413 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:11.422 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:31:11.493 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:31:11.509 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:31:11.578 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:31:11.580 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:31:11.667 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:31:11.670 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:31:11.717 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:31:11.719 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:31:11.798 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T00:31:11.798 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T00:31:11.815 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T00:31:11.906 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T00:31:11.916 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T00:31:11.929 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T00:31:12.021 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T00:31:12.032 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T00:31:12.085 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T00:31:12.172 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T00:31:12.227 DEBUG:teuthology.orchestra.run.smithi029:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:31:12.229 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:31:12.275 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T00:31:12.276 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T00:31:12.276 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T00:31:12.278 INFO:tasks.ceph.mon.a:Started 2024-04-25T00:31:12.278 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T00:31:12.278 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T00:31:12.321 INFO:tasks.ceph.mon.b:Started 2024-04-25T00:31:12.321 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T00:31:12.321 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T00:31:12.323 INFO:tasks.ceph.mon.c:Started 2024-04-25T00:31:12.323 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T00:31:12.323 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T00:31:12.323 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T00:31:12.325 INFO:tasks.ceph.mgr.y:Started 2024-04-25T00:31:12.326 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T00:31:12.326 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T00:31:12.327 INFO:tasks.ceph.mgr.x:Started 2024-04-25T00:31:12.327 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T00:31:12.328 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T00:31:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.445+0000 7fce0b943640 1 Processor -- start 2024-04-25T00:31:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.445+0000 7fce0b943640 1 -- start start 2024-04-25T00:31:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.445+0000 7fce0b943640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 0x7fce040721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.445+0000 7fce0b943640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce040726e0 con 0x7fce0415ad20 2024-04-25T00:31:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.445+0000 7fce0b943640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce0415bc30 con 0x7fce040d0390 2024-04-25T00:31:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.445+0000 7fce0b943640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fce040cb6a0 con 0x7fce04071d80 2024-04-25T00:31:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.445+0000 7fce09ee0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 0x7fce040721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.445+0000 7fce09ee0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 0x7fce040721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:50850/0 (socket says 172.21.15.29:50850) 2024-04-25T00:31:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.445+0000 7fce09ee0640 1 -- 172.21.15.29:0/3270592137 learned_addr learned my addr 172.21.15.29:0/3270592137 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.445+0000 7fce09ee0640 1 -- 172.21.15.29:0/3270592137 >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 msgr2=0x7fce040721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T00:31:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.445+0000 7fce09ee0640 1 -- 172.21.15.29:0/3270592137 >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 msgr2=0x7fce040721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T00:31:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.445+0000 7fce09ee0640 1 --2- 172.21.15.29:0/3270592137 >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 0x7fce040721a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:31:12.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.445+0000 7fce09ee0640 1 --2- 172.21.15.29:0/3270592137 >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 0x7fce040721a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:31:12.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.649+0000 7fce09ee0640 1 --2- 172.21.15.29:0/3270592137 >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 0x7fce040721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:12.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.649+0000 7fce09ee0640 1 -- 172.21.15.29:0/3270592137 >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 msgr2=0x7fce040721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T00:31:12.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.649+0000 7fce09ee0640 1 -- 172.21.15.29:0/3270592137 >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 msgr2=0x7fce040721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T00:31:12.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.649+0000 7fce09ee0640 1 --2- 172.21.15.29:0/3270592137 >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 0x7fce040721a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:31:12.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.649+0000 7fce09ee0640 1 --2- 172.21.15.29:0/3270592137 >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 0x7fce040721a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:31:12.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.677+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 650340152 0 0) 0x7fce040726e0 con 0x7fce0415ad20 2024-04-25T00:31:12.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.677+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdf0003530 con 0x7fce0415ad20 2024-04-25T00:31:12.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.677+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 582691944 0 0) 0x7fce0415bc30 con 0x7fce040d0390 2024-04-25T00:31:12.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.677+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce040726e0 con 0x7fce040d0390 2024-04-25T00:31:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4270739272 0 0) 0x7fce040726e0 con 0x7fce040d0390 2024-04-25T00:31:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 msgr2=0x7fce040721a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:31:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fcdfbfff640 1 --2- 172.21.15.29:0/3270592137 >> v2:172.21.15.157:3300/0 conn(0x7fce04071d80 0x7fce040721a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 >> v1:172.21.15.29:6789/0 conn(0x7fce0415ad20 legacy=0x7fce040cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0415bf10 con 0x7fce040d0390 2024-04-25T00:31:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fcdf4001680 con 0x7fce040d0390 2024-04-25T00:31:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 >> v1:172.21.15.157:6789/0 conn(0x7fce040d0390 legacy=0x7fce0415b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fcdfbfff640 1 --2- 172.21.15.29:0/3270592137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdf0003ba0 0x7fcdf0003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:12.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fcdfbfff640 1 --2- 172.21.15.29:0/3270592137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0004810 0x7fcdf0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:12.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fcdfbfff640 1 --2- 172.21.15.29:0/3270592137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdf0005220 0x7fcdf000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:12.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fce040cb6a0 con 0x7fcdf0003ba0 2024-04-25T00:31:12.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf000c150 con 0x7fcdf0004810 2024-04-25T00:31:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf0010760 con 0x7fcdf0005220 2024-04-25T00:31:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fce096df640 1 --2- 172.21.15.29:0/3270592137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdf0003ba0 0x7fcdf0003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fce08ede640 1 --2- 172.21.15.29:0/3270592137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0004810 0x7fcdf0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fce09ee0640 1 --2- 172.21.15.29:0/3270592137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdf0005220 0x7fcdf000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fce096df640 1 -- 172.21.15.29:0/3270592137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdf0005220 msgr2=0x7fcdf000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fce096df640 1 --2- 172.21.15.29:0/3270592137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdf0005220 0x7fcdf000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fce096df640 1 -- 172.21.15.29:0/3270592137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0004810 msgr2=0x7fcdf0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fce096df640 1 --2- 172.21.15.29:0/3270592137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0004810 0x7fcdf0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fce096df640 1 -- 172.21.15.29:0/3270592137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce040726e0 con 0x7fcdf0003ba0 2024-04-25T00:31:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fce09ee0640 1 --2- 172.21.15.29:0/3270592137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdf0005220 0x7fcdf000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fce09ee0640 1 -- 172.21.15.29:0/3270592137 reap_dead start 2024-04-25T00:31:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.681+0000 7fce08ede640 1 --2- 172.21.15.29:0/3270592137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0004810 0x7fcdf0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:12.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.685+0000 7fce096df640 1 --2- 172.21.15.29:0/3270592137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdf0003ba0 0x7fcdf0003fa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fce00001900 tx=0x7fce0001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72162e3e5450120 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:12.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.685+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce000b0070 con 0x7fcdf0003ba0 2024-04-25T00:31:12.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.685+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce000052d0 con 0x7fcdf0003ba0 2024-04-25T00:31:12.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.685+0000 7fcdfbfff640 1 -- 172.21.15.29:0/3270592137 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00001550 con 0x7fcdf0003ba0 2024-04-25T00:31:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.689+0000 7fce0b943640 1 -- 172.21.15.29:0/3270592137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdf0003ba0 msgr2=0x7fcdf0003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.689+0000 7fce0b943640 1 --2- 172.21.15.29:0/3270592137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdf0003ba0 0x7fcdf0003fa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fce00001900 tx=0x7fce0001c540 comp rx=0 tx=0).stop 2024-04-25T00:31:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.689+0000 7fce0b943640 1 -- 172.21.15.29:0/3270592137 shutdown_connections 2024-04-25T00:31:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.689+0000 7fce0b943640 1 --2- 172.21.15.29:0/3270592137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdf0003ba0 0x7fcdf0003fa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.689+0000 7fce0b943640 1 -- 172.21.15.29:0/3270592137 >> 172.21.15.29:0/3270592137 conn(0x7fce040d0c40 msgr2=0x7fce040d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.689+0000 7fce0b943640 1 -- 172.21.15.29:0/3270592137 shutdown_connections 2024-04-25T00:31:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.689+0000 7fce0b943640 1 -- 172.21.15.29:0/3270592137 wait complete. 2024-04-25T00:31:12.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.689+0000 7fce0b943640 1 Processor -- start 2024-04-25T00:31:12.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.689+0000 7fce0b943640 1 -- start start 2024-04-25T00:31:12.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce0b943640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0003ba0 0x7fce040c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:12.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce0b943640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0415cc40 0x7fce040b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:12.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce0b943640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce040b5e60 0x7fce040b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:12.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce0b943640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fce0406f5e0 con 0x7fce040b5e60 2024-04-25T00:31:12.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce0b943640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fce0406f460 con 0x7fcdf0003ba0 2024-04-25T00:31:12.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce0b943640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fce0406f760 con 0x7fce0415cc40 2024-04-25T00:31:12.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce096df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0003ba0 0x7fce040c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:12.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce096df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0003ba0 0x7fce040c4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:50910/0 (socket says 172.21.15.29:50910) 2024-04-25T00:31:12.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce08ede640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0415cc40 0x7fce040b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:12.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce096df640 1 -- 172.21.15.29:0/4059547322 learned_addr learned my addr 172.21.15.29:0/4059547322 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:12.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce09ee0640 1 --2- 172.21.15.29:0/4059547322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce040b5e60 0x7fce040b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce08ede640 1 -- 172.21.15.29:0/4059547322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0003ba0 msgr2=0x7fce040c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce08ede640 1 --2- 172.21.15.29:0/4059547322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0003ba0 0x7fce040c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce08ede640 1 -- 172.21.15.29:0/4059547322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce040b5e60 msgr2=0x7fce040b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce08ede640 1 --2- 172.21.15.29:0/4059547322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce040b5e60 0x7fce040b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce08ede640 1 -- 172.21.15.29:0/4059547322 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0415d1e0 con 0x7fce0415cc40 2024-04-25T00:31:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce09ee0640 1 --2- 172.21.15.29:0/4059547322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce040b5e60 0x7fce040b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce08ede640 1 --2- 172.21.15.29:0/4059547322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0415cc40 0x7fce040b5920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fce04068930 tx=0x7fcdf400ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fcdf9ffb640 1 -- 172.21.15.29:0/4059547322 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40050d0 con 0x7fce0415cc40 2024-04-25T00:31:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fcdf9ffb640 1 -- 172.21.15.29:0/4059547322 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf4018070 con 0x7fce0415cc40 2024-04-25T00:31:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce0b943640 1 -- 172.21.15.29:0/4059547322 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0415d4c0 con 0x7fce0415cc40 2024-04-25T00:31:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fcdf9ffb640 1 -- 172.21.15.29:0/4059547322 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4005470 con 0x7fce0415cc40 2024-04-25T00:31:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce0b943640 1 -- 172.21.15.29:0/4059547322 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0415da80 con 0x7fce0415cc40 2024-04-25T00:31:12.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fcdf9ffb640 1 -- 172.21.15.29:0/4059547322 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcdf400c660 con 0x7fce0415cc40 2024-04-25T00:31:12.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fce0b943640 1 -- 172.21.15.29:0/4059547322 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce04069270 con 0x7fce0415cc40 2024-04-25T00:31:12.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.693+0000 7fcdf9ffb640 1 -- 172.21.15.29:0/4059547322 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcdf4003d60 con 0x7fce0415cc40 2024-04-25T00:31:12.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.705+0000 7fcdf9ffb640 1 -- 172.21.15.29:0/4059547322 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcdf4003600 con 0x7fce0415cc40 2024-04-25T00:31:12.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.769+0000 7fce0b943640 1 -- 172.21.15.29:0/4059547322 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fce040b6ac0 con 0x7fce0415cc40 2024-04-25T00:31:12.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.777+0000 7fcdf9ffb640 1 -- 172.21.15.29:0/4059547322 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fcdf400c950 con 0x7fce0415cc40 2024-04-25T00:31:12.778 INFO:teuthology.orchestra.run.smithi029.stderr:adjusted tunables profile to default 2024-04-25T00:31:12.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.777+0000 7fce0b943640 1 -- 172.21.15.29:0/4059547322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0415cc40 msgr2=0x7fce040b5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:12.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.777+0000 7fce0b943640 1 --2- 172.21.15.29:0/4059547322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0415cc40 0x7fce040b5920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fce04068930 tx=0x7fcdf400ec10 comp rx=0 tx=0).stop 2024-04-25T00:31:12.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.777+0000 7fce0b943640 1 -- 172.21.15.29:0/4059547322 shutdown_connections 2024-04-25T00:31:12.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.777+0000 7fce0b943640 1 --2- 172.21.15.29:0/4059547322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce040b5e60 0x7fce040b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:12.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.777+0000 7fce0b943640 1 --2- 172.21.15.29:0/4059547322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0415cc40 0x7fce040b5920 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:12.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.777+0000 7fce0b943640 1 --2- 172.21.15.29:0/4059547322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdf0003ba0 0x7fce040c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:12.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.777+0000 7fce0b943640 1 -- 172.21.15.29:0/4059547322 >> 172.21.15.29:0/4059547322 conn(0x7fce040d0c40 msgr2=0x7fce0415b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:12.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.777+0000 7fce0b943640 1 -- 172.21.15.29:0/4059547322 shutdown_connections 2024-04-25T00:31:12.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:12.777+0000 7fce0b943640 1 -- 172.21.15.29:0/4059547322 wait complete. 2024-04-25T00:31:12.800 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T00:31:12.800 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T00:31:12.800 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:31:12.800 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T00:31:12.812 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:31:12.812 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T00:31:12.866 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:31:12.867 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T00:31:12.906 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:12.901+0000 7f487255b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:31:12.908 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:12.905+0000 7f98f227a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:31:12.917 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:31:12.917 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T00:31:12.974 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:31:12.974 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T00:31:12.987 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:31:12.987 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T00:31:13.043 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:31:13.043 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T00:31:13.099 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:31:13.099 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T00:31:13.154 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 93e92106-8ca5-4141-9322-34a10ee2457b 0 2024-04-25T00:31:13.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.313+0000 7f4e38d8e640 1 Processor -- start 2024-04-25T00:31:13.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.313+0000 7f4e38d8e640 1 -- start start 2024-04-25T00:31:13.321 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e38d8e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4e3415ac90 0x7f4e34071560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.321 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e38d8e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4e340b3d00 0x7f4e34071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.321 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e38d8e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e3415f7b0 con 0x7f4e340cd890 2024-04-25T00:31:13.321 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e38d8e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4e3406f3f0 con 0x7f4e340b3d00 2024-04-25T00:31:13.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e38d8e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4e3406d620 con 0x7f4e3415ac90 2024-04-25T00:31:13.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e3259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4e3415ac90 0x7f4e34071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e3259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4e3415ac90 0x7f4e34071560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36898/0 (socket says 172.21.15.157:36898) 2024-04-25T00:31:13.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e3259c640 1 -- 172.21.15.157:0/4020513609 learned_addr learned my addr 172.21.15.157:0/4020513609 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:13.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e32d9d640 1 --2- 172.21.15.157:0/4020513609 >> v2:172.21.15.29:3300/0 conn(0x7f4e340b3d00 0x7f4e34071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e3259c640 1 -- 172.21.15.157:0/4020513609 >> v2:172.21.15.29:3300/0 conn(0x7f4e340b3d00 msgr2=0x7f4e34071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:13.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e3259c640 1 --2- 172.21.15.157:0/4020513609 >> v2:172.21.15.29:3300/0 conn(0x7f4e340b3d00 0x7f4e34071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e3259c640 1 -- 172.21.15.157:0/4020513609 >> v1:172.21.15.29:6789/0 conn(0x7f4e340cd890 legacy=0x7f4e340b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e3259c640 1 -- 172.21.15.157:0/4020513609 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3406d900 con 0x7f4e3415ac90 2024-04-25T00:31:13.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e3259c640 1 --2- 172.21.15.157:0/4020513609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e3415ac90 0x7f4e34071560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e34074960 tx=0x7f4e2401c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dfd6923180f583e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:13.323 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e31d9b640 1 -- 172.21.15.157:0/4020513609 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e240aa650 con 0x7f4e3415ac90 2024-04-25T00:31:13.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e31d9b640 1 -- 172.21.15.157:0/4020513609 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e240af070 con 0x7f4e3415ac90 2024-04-25T00:31:13.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.317+0000 7f4e31d9b640 1 -- 172.21.15.157:0/4020513609 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e240aaa10 con 0x7f4e3415ac90 2024-04-25T00:31:13.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 -- 172.21.15.157:0/4020513609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e3415ac90 msgr2=0x7f4e34071560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.329 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 --2- 172.21.15.157:0/4020513609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e3415ac90 0x7f4e34071560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e34074960 tx=0x7f4e2401c4f0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 -- 172.21.15.157:0/4020513609 shutdown_connections 2024-04-25T00:31:13.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 --2- 172.21.15.157:0/4020513609 >> v2:172.21.15.29:3300/0 conn(0x7f4e340b3d00 0x7f4e34071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 --2- 172.21.15.157:0/4020513609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e3415ac90 0x7f4e34071560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 -- 172.21.15.157:0/4020513609 >> 172.21.15.157:0/4020513609 conn(0x7f4e34074c00 msgr2=0x7f4e3406b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:13.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 -- 172.21.15.157:0/4020513609 shutdown_connections 2024-04-25T00:31:13.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 -- 172.21.15.157:0/4020513609 wait complete. 2024-04-25T00:31:13.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 Processor -- start 2024-04-25T00:31:13.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 -- start start 2024-04-25T00:31:13.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e340b3d00 0x7f4e34144df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e340cd890 0x7f4e3415b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e3415ac90 0x7f4e34146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4e340d0510 con 0x7f4e3415ac90 2024-04-25T00:31:13.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e340d0390 con 0x7f4e340b3d00 2024-04-25T00:31:13.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4e340d0690 con 0x7f4e340cd890 2024-04-25T00:31:13.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e3259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e340b3d00 0x7f4e34144df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e3259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e340b3d00 0x7f4e34144df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36906/0 (socket says 172.21.15.157:36906) 2024-04-25T00:31:13.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e3259c640 1 -- 172.21.15.157:0/3974341498 learned_addr learned my addr 172.21.15.157:0/3974341498 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:13.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e2bfff640 1 --2- 172.21.15.157:0/3974341498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e340cd890 0x7f4e3415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e32d9d640 1 --2- 172.21.15.157:0/3974341498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e3415ac90 0x7f4e34146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e3259c640 1 -- 172.21.15.157:0/3974341498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e340cd890 msgr2=0x7f4e3415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e3259c640 1 --2- 172.21.15.157:0/3974341498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e340cd890 0x7f4e3415b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e3259c640 1 -- 172.21.15.157:0/3974341498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e3415ac90 msgr2=0x7f4e34146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e3259c640 1 --2- 172.21.15.157:0/3974341498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e3415ac90 0x7f4e34146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e3259c640 1 -- 172.21.15.157:0/3974341498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3414e6c0 con 0x7f4e340b3d00 2024-04-25T00:31:13.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e3259c640 1 --2- 172.21.15.157:0/3974341498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e340b3d00 0x7f4e34144df0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e240ac040 tx=0x7f4e2401ad30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:13.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e2affd640 1 -- 172.21.15.157:0/3974341498 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e240bd5f0 con 0x7f4e340b3d00 2024-04-25T00:31:13.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e2affd640 1 -- 172.21.15.157:0/3974341498 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e240a6040 con 0x7f4e340b3d00 2024-04-25T00:31:13.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e2affd640 1 -- 172.21.15.157:0/3974341498 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e240bd9f0 con 0x7f4e340b3d00 2024-04-25T00:31:13.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 -- 172.21.15.157:0/3974341498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3414e9a0 con 0x7f4e340b3d00 2024-04-25T00:31:13.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e38d8e640 1 -- 172.21.15.157:0/3974341498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e34146120 con 0x7f4e340b3d00 2024-04-25T00:31:13.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e2affd640 1 -- 172.21.15.157:0/3974341498 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4e240c6440 con 0x7f4e340b3d00 2024-04-25T00:31:13.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.321+0000 7f4e2affd640 1 -- 172.21.15.157:0/3974341498 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f4e240b3580 con 0x7f4e340b3d00 2024-04-25T00:31:13.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.325+0000 7f4e38d8e640 1 -- 172.21.15.157:0/3974341498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e34069270 con 0x7f4e340b3d00 2024-04-25T00:31:13.336 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.325+0000 7f4e2affd640 1 -- 172.21.15.157:0/3974341498 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4e240dcdd0 con 0x7f4e340b3d00 2024-04-25T00:31:13.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.393+0000 7f4e38d8e640 1 -- 172.21.15.157:0/3974341498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "93e92106-8ca5-4141-9322-34a10ee2457b", "id": 0} v 0) -- 0x7f4e3414a950 con 0x7f4e340b3d00 2024-04-25T00:31:13.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.401+0000 7f4e2affd640 1 -- 172.21.15.157:0/3974341498 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "93e92106-8ca5-4141-9322-34a10ee2457b", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f4e240cb020 con 0x7f4e340b3d00 2024-04-25T00:31:13.404 INFO:teuthology.orchestra.run.smithi157.stdout:0 2024-04-25T00:31:13.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.401+0000 7f4e38d8e640 1 -- 172.21.15.157:0/3974341498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e340b3d00 msgr2=0x7f4e34144df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.401+0000 7f4e38d8e640 1 --2- 172.21.15.157:0/3974341498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e340b3d00 0x7f4e34144df0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e240ac040 tx=0x7f4e2401ad30 comp rx=0 tx=0).stop 2024-04-25T00:31:13.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.401+0000 7f4e38d8e640 1 -- 172.21.15.157:0/3974341498 shutdown_connections 2024-04-25T00:31:13.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.401+0000 7f4e38d8e640 1 --2- 172.21.15.157:0/3974341498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e3415ac90 0x7f4e34146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.401+0000 7f4e38d8e640 1 --2- 172.21.15.157:0/3974341498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e340cd890 0x7f4e3415b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.401+0000 7f4e38d8e640 1 --2- 172.21.15.157:0/3974341498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e340b3d00 0x7f4e34144df0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.401+0000 7f4e38d8e640 1 -- 172.21.15.157:0/3974341498 >> 172.21.15.157:0/3974341498 conn(0x7f4e34074c00 msgr2=0x7f4e3406b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:13.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.401+0000 7f4e38d8e640 1 -- 172.21.15.157:0/3974341498 shutdown_connections 2024-04-25T00:31:13.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.401+0000 7f4e38d8e640 1 -- 172.21.15.157:0/3974341498 wait complete. 2024-04-25T00:31:13.415 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:13.409+0000 7f487255b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:31:13.422 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 8cee5ae1-c34c-4c4d-9402-eb383be68cac 1 2024-04-25T00:31:13.459 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:13.457+0000 7f98f227a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:31:13.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f61c98640 1 Processor -- start 2024-04-25T00:31:13.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f61c98640 1 -- start start 2024-04-25T00:31:13.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f61c98640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f5c15a490 0x7f0f5c06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f61c98640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c0b3860 0x7f0f5c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f61c98640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f5c06e930 con 0x7f0f5c15a870 2024-04-25T00:31:13.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f61c98640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0f5c15ad60 con 0x7f0f5c15a490 2024-04-25T00:31:13.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f61c98640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c15b040 con 0x7f0f5c0b3860 2024-04-25T00:31:13.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f5b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f5c15a490 0x7f0f5c06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f5affd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c0b3860 0x7f0f5c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f5b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f5c15a490 0x7f0f5c06df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36916/0 (socket says 172.21.15.157:36916) 2024-04-25T00:31:13.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f5b7fe640 1 -- 172.21.15.157:0/2708362945 learned_addr learned my addr 172.21.15.157:0/2708362945 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:13.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f5affd640 1 -- 172.21.15.157:0/2708362945 >> v2:172.21.15.157:3300/0 conn(0x7f0f5c15a490 msgr2=0x7f0f5c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:13.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f5affd640 1 --2- 172.21.15.157:0/2708362945 >> v2:172.21.15.157:3300/0 conn(0x7f0f5c15a490 0x7f0f5c06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f5affd640 1 -- 172.21.15.157:0/2708362945 >> v1:172.21.15.29:6789/0 conn(0x7f0f5c15a870 legacy=0x7f0f5c071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:13.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f5affd640 1 -- 172.21.15.157:0/2708362945 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c15b320 con 0x7f0f5c0b3860 2024-04-25T00:31:13.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.537+0000 7f0f5b7fe640 1 --2- 172.21.15.157:0/2708362945 >> v2:172.21.15.157:3300/0 conn(0x7f0f5c15a490 0x7f0f5c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:13.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5affd640 1 --2- 172.21.15.157:0/2708362945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c0b3860 0x7f0f5c0b3e00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0097c0 tx=0x7f0f4c01c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=354b6bea0006a99b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:13.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5a7fc640 1 -- 172.21.15.157:0/2708362945 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0b0070 con 0x7f0f5c0b3860 2024-04-25T00:31:13.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5a7fc640 1 -- 172.21.15.157:0/2708362945 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5c06e6e0 con 0x7f0f5c0b3860 2024-04-25T00:31:13.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5a7fc640 1 -- 172.21.15.157:0/2708362945 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0ab820 con 0x7f0f5c0b3860 2024-04-25T00:31:13.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 -- 172.21.15.157:0/2708362945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c0b3860 msgr2=0x7f0f5c0b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 --2- 172.21.15.157:0/2708362945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c0b3860 0x7f0f5c0b3e00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0097c0 tx=0x7f0f4c01c3d0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 -- 172.21.15.157:0/2708362945 shutdown_connections 2024-04-25T00:31:13.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 --2- 172.21.15.157:0/2708362945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c0b3860 0x7f0f5c0b3e00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 --2- 172.21.15.157:0/2708362945 >> v2:172.21.15.157:3300/0 conn(0x7f0f5c15a490 0x7f0f5c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 -- 172.21.15.157:0/2708362945 >> 172.21.15.157:0/2708362945 conn(0x7f0f5c074c00 msgr2=0x7f0f5c06b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:13.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 -- 172.21.15.157:0/2708362945 shutdown_connections 2024-04-25T00:31:13.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 -- 172.21.15.157:0/2708362945 wait complete. 2024-04-25T00:31:13.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 Processor -- start 2024-04-25T00:31:13.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 -- start start 2024-04-25T00:31:13.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f5c0b3860 0x7f0f5c1470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c15a490 0x7f0f5c1548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f5c15a870 0x7f0f5c1f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c072000 con 0x7f0f5c0b3860 2024-04-25T00:31:13.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c071e80 con 0x7f0f5c15a870 2024-04-25T00:31:13.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f61c98640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c072180 con 0x7f0f5c15a490 2024-04-25T00:31:13.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5affd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c15a490 0x7f0f5c1548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5affd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c15a490 0x7f0f5c1548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:55824/0 (socket says 172.21.15.157:55824) 2024-04-25T00:31:13.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5affd640 1 -- 172.21.15.157:0/261479400 learned_addr learned my addr 172.21.15.157:0/261479400 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:13.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5bfff640 1 --2- 172.21.15.157:0/261479400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f5c15a870 0x7f0f5c1f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5b7fe640 1 --2- 172.21.15.157:0/261479400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f5c0b3860 0x7f0f5c1470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5affd640 1 -- 172.21.15.157:0/261479400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f5c15a870 msgr2=0x7f0f5c1f91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5affd640 1 --2- 172.21.15.157:0/261479400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f5c15a870 0x7f0f5c1f91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5affd640 1 -- 172.21.15.157:0/261479400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f5c0b3860 msgr2=0x7f0f5c1470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5affd640 1 --2- 172.21.15.157:0/261479400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f5c0b3860 0x7f0f5c1470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5affd640 1 -- 172.21.15.157:0/261479400 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c146df0 con 0x7f0f5c15a490 2024-04-25T00:31:13.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.541+0000 7f0f5b7fe640 1 --2- 172.21.15.157:0/261479400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f5c0b3860 0x7f0f5c1470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:13.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.545+0000 7f0f5affd640 1 --2- 172.21.15.157:0/261479400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c15a490 0x7f0f5c1548a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c01aa80 tx=0x7f0f4c019170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:13.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.545+0000 7f0f43fff640 1 -- 172.21.15.157:0/261479400 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0b8020 con 0x7f0f5c15a490 2024-04-25T00:31:13.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.545+0000 7f0f43fff640 1 -- 172.21.15.157:0/261479400 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f4c0abdf0 con 0x7f0f5c15a490 2024-04-25T00:31:13.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.545+0000 7f0f61c98640 1 -- 172.21.15.157:0/261479400 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5c14de70 con 0x7f0f5c15a490 2024-04-25T00:31:13.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.545+0000 7f0f43fff640 1 -- 172.21.15.157:0/261479400 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0b0040 con 0x7f0f5c15a490 2024-04-25T00:31:13.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.545+0000 7f0f61c98640 1 -- 172.21.15.157:0/261479400 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5c154de0 con 0x7f0f5c15a490 2024-04-25T00:31:13.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.545+0000 7f0f43fff640 1 -- 172.21.15.157:0/261479400 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0f4c0bb020 con 0x7f0f5c15a490 2024-04-25T00:31:13.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.545+0000 7f0f43fff640 1 -- 172.21.15.157:0/261479400 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f0f4c0bdb20 con 0x7f0f5c15a490 2024-04-25T00:31:13.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.545+0000 7f0f61c98640 1 -- 172.21.15.157:0/261479400 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5c069270 con 0x7f0f5c15a490 2024-04-25T00:31:13.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.545+0000 7f0f43fff640 1 -- 172.21.15.157:0/261479400 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0f4c0c2030 con 0x7f0f5c15a490 2024-04-25T00:31:13.619 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.613+0000 7f0f61c98640 1 -- 172.21.15.157:0/261479400 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8cee5ae1-c34c-4c4d-9402-eb383be68cac", "id": 1} v 0) -- 0x7f0f5c15c460 con 0x7f0f5c15a490 2024-04-25T00:31:13.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.621+0000 7f0f43fff640 1 -- 172.21.15.157:0/261479400 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8cee5ae1-c34c-4c4d-9402-eb383be68cac", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f0f4c016050 con 0x7f0f5c15a490 2024-04-25T00:31:13.624 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:31:13.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.621+0000 7f0f61c98640 1 -- 172.21.15.157:0/261479400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c15a490 msgr2=0x7f0f5c1548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.621+0000 7f0f61c98640 1 --2- 172.21.15.157:0/261479400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c15a490 0x7f0f5c1548a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c01aa80 tx=0x7f0f4c019170 comp rx=0 tx=0).stop 2024-04-25T00:31:13.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.621+0000 7f0f61c98640 1 -- 172.21.15.157:0/261479400 shutdown_connections 2024-04-25T00:31:13.626 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.621+0000 7f0f61c98640 1 --2- 172.21.15.157:0/261479400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f5c15a870 0x7f0f5c1f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.626 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.621+0000 7f0f61c98640 1 --2- 172.21.15.157:0/261479400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f5c15a490 0x7f0f5c1548a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.626 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.621+0000 7f0f61c98640 1 --2- 172.21.15.157:0/261479400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f5c0b3860 0x7f0f5c1470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.626 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.621+0000 7f0f61c98640 1 -- 172.21.15.157:0/261479400 >> 172.21.15.157:0/261479400 conn(0x7f0f5c074c00 msgr2=0x7f0f5c15d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:13.626 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.621+0000 7f0f61c98640 1 -- 172.21.15.157:0/261479400 shutdown_connections 2024-04-25T00:31:13.626 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.621+0000 7f0f61c98640 1 -- 172.21.15.157:0/261479400 wait complete. 2024-04-25T00:31:13.626 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:13.621+0000 7f487255b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:31:13.642 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0 2 2024-04-25T00:31:13.690 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:13.689+0000 7f98f227a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:31:13.762 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.757+0000 7f6020102640 1 Processor -- start 2024-04-25T00:31:13.762 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.757+0000 7f6020102640 1 -- start start 2024-04-25T00:31:13.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.757+0000 7f6020102640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f601815e060 0x7f601815c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f6020102640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f601815e850 con 0x7f6018072490 2024-04-25T00:31:13.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f6020102640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f601815eaf0 con 0x7f601815bce0 2024-04-25T00:31:13.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f6020102640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f601815f700 con 0x7f601815e060 2024-04-25T00:31:13.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f601de9e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f601815e060 0x7f601815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f601de9e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f601815e060 0x7f601815c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36942/0 (socket says 172.21.15.157:36942) 2024-04-25T00:31:13.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f601de9e640 1 -- 172.21.15.157:0/157121331 learned_addr learned my addr 172.21.15.157:0/157121331 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:13.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f601de9e640 1 -- 172.21.15.157:0/157121331 >> v1:172.21.15.157:6789/0 conn(0x7f601815bce0 legacy=0x7f6018071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f601de9e640 1 -- 172.21.15.157:0/157121331 >> v1:172.21.15.29:6789/0 conn(0x7f6018072490 legacy=0x7f60180cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:13.764 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f601de9e640 1 -- 172.21.15.157:0/157121331 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601815edd0 con 0x7f601815e060 2024-04-25T00:31:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f601de9e640 1 --2- 172.21.15.157:0/157121331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601815e060 0x7f601815c420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6014009870 tx=0x7f601401c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34b7e2b4562a9f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f601ce9c640 1 -- 172.21.15.157:0/157121331 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60140ab650 con 0x7f601815e060 2024-04-25T00:31:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f601ce9c640 1 -- 172.21.15.157:0/157121331 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60140b0070 con 0x7f601815e060 2024-04-25T00:31:13.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f601ce9c640 1 -- 172.21.15.157:0/157121331 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60140aba10 con 0x7f601815e060 2024-04-25T00:31:13.767 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f6020102640 1 -- 172.21.15.157:0/157121331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601815e060 msgr2=0x7f601815c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.767 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f6020102640 1 --2- 172.21.15.157:0/157121331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601815e060 0x7f601815c420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6014009870 tx=0x7f601401c4c0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.767 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f6020102640 1 -- 172.21.15.157:0/157121331 shutdown_connections 2024-04-25T00:31:13.767 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f6020102640 1 --2- 172.21.15.157:0/157121331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601815e060 0x7f601815c420 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.767 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f6020102640 1 -- 172.21.15.157:0/157121331 >> 172.21.15.157:0/157121331 conn(0x7f601806c320 msgr2=0x7f601806c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:13.767 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f6020102640 1 -- 172.21.15.157:0/157121331 shutdown_connections 2024-04-25T00:31:13.767 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.761+0000 7f6020102640 1 -- 172.21.15.157:0/157121331 wait complete. 2024-04-25T00:31:13.768 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f6020102640 1 Processor -- start 2024-04-25T00:31:13.768 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f6020102640 1 -- start start 2024-04-25T00:31:13.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f6020102640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6018072490 0x7f60180c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f6020102640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601815bce0 0x7f601814d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f6020102640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f601815e060 0x7f60180c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f6020102640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f601815e5c0 con 0x7f601815e060 2024-04-25T00:31:13.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f6020102640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f601815e440 con 0x7f601815bce0 2024-04-25T00:31:13.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f6020102640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f601806f610 con 0x7f6018072490 2024-04-25T00:31:13.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f601de9e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6018072490 0x7f60180c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.771 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f601de9e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6018072490 0x7f60180c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:55826/0 (socket says 172.21.15.157:55826) 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f601de9e640 1 -- 172.21.15.157:0/274278011 learned_addr learned my addr 172.21.15.157:0/274278011 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f601d69d640 1 --2- 172.21.15.157:0/274278011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601815bce0 0x7f601814d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f601e69f640 1 --2- 172.21.15.157:0/274278011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f601815e060 0x7f60180c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f601de9e640 1 -- 172.21.15.157:0/274278011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601815bce0 msgr2=0x7f601814d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f601de9e640 1 --2- 172.21.15.157:0/274278011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601815bce0 0x7f601814d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f601de9e640 1 -- 172.21.15.157:0/274278011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f601815e060 msgr2=0x7f60180c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f601de9e640 1 --2- 172.21.15.157:0/274278011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f601815e060 0x7f60180c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f601de9e640 1 -- 172.21.15.157:0/274278011 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601814d700 con 0x7f6018072490 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f601d69d640 1 --2- 172.21.15.157:0/274278011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601815bce0 0x7f601814d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f601e69f640 1 --2- 172.21.15.157:0/274278011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f601815e060 0x7f60180c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f601de9e640 1 --2- 172.21.15.157:0/274278011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6018072490 0x7f60180c9890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f60140190d0 tx=0x7f601401ad00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f600e7fc640 1 -- 172.21.15.157:0/274278011 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60140abe60 con 0x7f6018072490 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f600e7fc640 1 -- 172.21.15.157:0/274278011 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60140b4820 con 0x7f6018072490 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f600e7fc640 1 -- 172.21.15.157:0/274278011 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60140b4c10 con 0x7f6018072490 2024-04-25T00:31:13.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f6020102640 1 -- 172.21.15.157:0/274278011 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60180c87f0 con 0x7f6018072490 2024-04-25T00:31:13.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f6020102640 1 -- 172.21.15.157:0/274278011 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f60180c8db0 con 0x7f6018072490 2024-04-25T00:31:13.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f600e7fc640 1 -- 172.21.15.157:0/274278011 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f60140be780 con 0x7f6018072490 2024-04-25T00:31:13.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f600e7fc640 1 -- 172.21.15.157:0/274278011 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f60140b4db0 con 0x7f6018072490 2024-04-25T00:31:13.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.765+0000 7f6020102640 1 -- 172.21.15.157:0/274278011 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6018069270 con 0x7f6018072490 2024-04-25T00:31:13.773 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.769+0000 7f600e7fc640 1 -- 172.21.15.157:0/274278011 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f60140c4060 con 0x7f6018072490 2024-04-25T00:31:13.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.837+0000 7f6020102640 1 -- 172.21.15.157:0/274278011 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0", "id": 2} v 0) -- 0x7f601806e090 con 0x7f6018072490 2024-04-25T00:31:13.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.841+0000 7f600e7fc640 1 -- 172.21.15.157:0/274278011 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f6014016110 con 0x7f6018072490 2024-04-25T00:31:13.848 INFO:teuthology.orchestra.run.smithi157.stdout:2 2024-04-25T00:31:13.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.845+0000 7f6020102640 1 -- 172.21.15.157:0/274278011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6018072490 msgr2=0x7f60180c9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.845+0000 7f6020102640 1 --2- 172.21.15.157:0/274278011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6018072490 0x7f60180c9890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f60140190d0 tx=0x7f601401ad00 comp rx=0 tx=0).stop 2024-04-25T00:31:13.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.845+0000 7f6020102640 1 -- 172.21.15.157:0/274278011 shutdown_connections 2024-04-25T00:31:13.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.845+0000 7f6020102640 1 --2- 172.21.15.157:0/274278011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f601815e060 0x7f60180c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.845+0000 7f6020102640 1 --2- 172.21.15.157:0/274278011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f601815bce0 0x7f601814d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.845+0000 7f6020102640 1 --2- 172.21.15.157:0/274278011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6018072490 0x7f60180c9890 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.845+0000 7f6020102640 1 -- 172.21.15.157:0/274278011 >> 172.21.15.157:0/274278011 conn(0x7f601806c320 msgr2=0x7f60180cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:13.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.845+0000 7f6020102640 1 -- 172.21.15.157:0/274278011 shutdown_connections 2024-04-25T00:31:13.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.845+0000 7f6020102640 1 -- 172.21.15.157:0/274278011 wait complete. 2024-04-25T00:31:13.863 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 03f003b6-1663-4b90-b6d0-d470e68a52c1 3 2024-04-25T00:31:13.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.977+0000 7fb055029640 1 Processor -- start 2024-04-25T00:31:13.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.977+0000 7fb055029640 1 -- start start 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.977+0000 7fb055029640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb05015e600 0x7fb0500ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.977+0000 7fb055029640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0500cfcb0 0x7fb0500adac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.977+0000 7fb055029640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0500af620 con 0x7fb0500cf8d0 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.977+0000 7fb055029640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb050074320 con 0x7fb05015e600 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.977+0000 7fb055029640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb050074600 con 0x7fb0500cfcb0 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.977+0000 7fb04f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0500cfcb0 0x7fb0500adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.977+0000 7fb04e59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb05015e600 0x7fb0500ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.977+0000 7fb04e59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb05015e600 0x7fb0500ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36970/0 (socket says 172.21.15.157:36970) 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.977+0000 7fb04ed9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb0500cf8d0 0x7fb05015def0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52378/0 (socket says 172.21.15.157:52378) 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.977+0000 7fb04e59c640 1 -- 172.21.15.157:0/2328506982 learned_addr learned my addr 172.21.15.157:0/2328506982 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.977+0000 7fb04f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0500cfcb0 0x7fb0500adac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:55842/0 (socket says 172.21.15.157:55842) 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb04e59c640 1 -- 172.21.15.157:0/2328506982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0500cfcb0 msgr2=0x7fb0500adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb04e59c640 1 --2- 172.21.15.157:0/2328506982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0500cfcb0 0x7fb0500adac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb04e59c640 1 -- 172.21.15.157:0/2328506982 >> v1:172.21.15.157:6789/0 conn(0x7fb0500cf8d0 legacy=0x7fb05015def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb04e59c640 1 -- 172.21.15.157:0/2328506982 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05006bf80 con 0x7fb05015e600 2024-04-25T00:31:13.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb04e59c640 1 --2- 172.21.15.157:0/2328506982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb05015e600 0x7fb0500ad580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb038009870 tx=0x7fb03801c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d26e3ba33c650e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:13.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb04dd9b640 1 -- 172.21.15.157:0/2328506982 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380b0070 con 0x7fb05015e600 2024-04-25T00:31:13.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb04dd9b640 1 -- 172.21.15.157:0/2328506982 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb038004230 con 0x7fb05015e600 2024-04-25T00:31:13.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb04dd9b640 1 -- 172.21.15.157:0/2328506982 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038003620 con 0x7fb05015e600 2024-04-25T00:31:13.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb055029640 1 -- 172.21.15.157:0/2328506982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb05015e600 msgr2=0x7fb0500ad580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb055029640 1 --2- 172.21.15.157:0/2328506982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb05015e600 0x7fb0500ad580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb038009870 tx=0x7fb03801c440 comp rx=0 tx=0).stop 2024-04-25T00:31:13.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb055029640 1 -- 172.21.15.157:0/2328506982 shutdown_connections 2024-04-25T00:31:13.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb055029640 1 --2- 172.21.15.157:0/2328506982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0500cfcb0 0x7fb0500adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb055029640 1 --2- 172.21.15.157:0/2328506982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb05015e600 0x7fb0500ad580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb055029640 1 -- 172.21.15.157:0/2328506982 >> 172.21.15.157:0/2328506982 conn(0x7fb0500b1ec0 msgr2=0x7fb0500b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:13.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb055029640 1 -- 172.21.15.157:0/2328506982 shutdown_connections 2024-04-25T00:31:13.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.981+0000 7fb055029640 1 -- 172.21.15.157:0/2328506982 wait complete. 2024-04-25T00:31:13.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb055029640 1 Processor -- start 2024-04-25T00:31:13.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb055029640 1 -- start start 2024-04-25T00:31:13.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb055029640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0500cf8d0 0x7fb0500b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb055029640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0500cfcb0 0x7fb05015a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb055029640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb05015e600 0x7fb0500c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:13.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb055029640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb05016b450 con 0x7fb0500cf8d0 2024-04-25T00:31:13.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb055029640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb05016b2d0 con 0x7fb05015e600 2024-04-25T00:31:13.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb055029640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb05016b5d0 con 0x7fb0500cfcb0 2024-04-25T00:31:13.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb04e59c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0500cfcb0 0x7fb05015a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb04f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb05015e600 0x7fb0500c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:13.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb04e59c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0500cfcb0 0x7fb05015a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:55846/0 (socket says 172.21.15.157:55846) 2024-04-25T00:31:13.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb04f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb05015e600 0x7fb0500c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36986/0 (socket says 172.21.15.157:36986) 2024-04-25T00:31:13.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb04e59c640 1 -- 172.21.15.157:0/2315539676 learned_addr learned my addr 172.21.15.157:0/2315539676 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:13.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb04e59c640 1 -- 172.21.15.157:0/2315539676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb05015e600 msgr2=0x7fb0500c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb04e59c640 1 --2- 172.21.15.157:0/2315539676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb05015e600 0x7fb0500c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb04e59c640 1 -- 172.21.15.157:0/2315539676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0500cf8d0 msgr2=0x7fb0500b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:13.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb04e59c640 1 --2- 172.21.15.157:0/2315539676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0500cf8d0 0x7fb0500b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:13.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb04e59c640 1 -- 172.21.15.157:0/2315539676 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0500b56f0 con 0x7fb0500cfcb0 2024-04-25T00:31:13.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb04f59e640 1 --2- 172.21.15.157:0/2315539676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb05015e600 0x7fb0500c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:13.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb04e59c640 1 --2- 172.21.15.157:0/2315539676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0500cfcb0 0x7fb05015a2b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb03801a070 tx=0x7fb0380191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:13.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb0477fe640 1 -- 172.21.15.157:0/2315539676 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380b0030 con 0x7fb0500cfcb0 2024-04-25T00:31:13.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb0477fe640 1 -- 172.21.15.157:0/2315539676 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0380ba070 con 0x7fb0500cfcb0 2024-04-25T00:31:13.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb055029640 1 -- 172.21.15.157:0/2315539676 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0500b59d0 con 0x7fb0500cfcb0 2024-04-25T00:31:13.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb0477fe640 1 -- 172.21.15.157:0/2315539676 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380b5440 con 0x7fb0500cfcb0 2024-04-25T00:31:13.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb055029640 1 -- 172.21.15.157:0/2315539676 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0500b5f90 con 0x7fb0500cfcb0 2024-04-25T00:31:13.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb055029640 1 -- 172.21.15.157:0/2315539676 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb050069270 con 0x7fb0500cfcb0 2024-04-25T00:31:13.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb0477fe640 1 -- 172.21.15.157:0/2315539676 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb0380b55e0 con 0x7fb0500cfcb0 2024-04-25T00:31:13.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.985+0000 7fb0477fe640 1 -- 172.21.15.157:0/2315539676 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fb0380bfa30 con 0x7fb0500cfcb0 2024-04-25T00:31:13.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:13.989+0000 7fb0477fe640 1 -- 172.21.15.157:0/2315539676 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb038016150 con 0x7fb0500cfcb0 2024-04-25T00:31:14.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.057+0000 7fb055029640 1 -- 172.21.15.157:0/2315539676 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "03f003b6-1663-4b90-b6d0-d470e68a52c1", "id": 3} v 0) -- 0x7fb05015aa60 con 0x7fb0500cfcb0 2024-04-25T00:31:14.066 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.061+0000 7fb0477fe640 1 -- 172.21.15.157:0/2315539676 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "03f003b6-1663-4b90-b6d0-d470e68a52c1", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fb0380bf5a0 con 0x7fb0500cfcb0 2024-04-25T00:31:14.066 INFO:teuthology.orchestra.run.smithi157.stdout:3 2024-04-25T00:31:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.061+0000 7fb055029640 1 -- 172.21.15.157:0/2315539676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0500cfcb0 msgr2=0x7fb05015a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.061+0000 7fb055029640 1 --2- 172.21.15.157:0/2315539676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0500cfcb0 0x7fb05015a2b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb03801a070 tx=0x7fb0380191e0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.061+0000 7fb055029640 1 -- 172.21.15.157:0/2315539676 shutdown_connections 2024-04-25T00:31:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.061+0000 7fb055029640 1 --2- 172.21.15.157:0/2315539676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb05015e600 0x7fb0500c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.061+0000 7fb055029640 1 --2- 172.21.15.157:0/2315539676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0500cfcb0 0x7fb05015a2b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.061+0000 7fb055029640 1 --2- 172.21.15.157:0/2315539676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0500cf8d0 0x7fb0500b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.061+0000 7fb055029640 1 -- 172.21.15.157:0/2315539676 >> 172.21.15.157:0/2315539676 conn(0x7fb0500b1ec0 msgr2=0x7fb05006d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.061+0000 7fb055029640 1 -- 172.21.15.157:0/2315539676 shutdown_connections 2024-04-25T00:31:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.061+0000 7fb055029640 1 -- 172.21.15.157:0/2315539676 wait complete. 2024-04-25T00:31:14.083 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 80ae2f99-2daf-47c9-b454-978268265b24 4 2024-04-25T00:31:14.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.189+0000 7f3f20373640 1 Processor -- start 2024-04-25T00:31:14.195 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.189+0000 7f3f20373640 1 -- start start 2024-04-25T00:31:14.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f20373640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f180701a0 0x7f3f1815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f20373640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1815d2b0 con 0x7f3f18070580 2024-04-25T00:31:14.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f20373640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1815fef0 con 0x7f3f1815f930 2024-04-25T00:31:14.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f20373640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3f181601d0 con 0x7f3f180701a0 2024-04-25T00:31:14.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f1e10f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f180701a0 0x7f3f1815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f1e10f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f180701a0 0x7f3f1815cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:55856/0 (socket says 172.21.15.157:55856) 2024-04-25T00:31:14.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f1e910640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3f1815f930 0x7f3f1806b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52386/0 (socket says 172.21.15.157:52386) 2024-04-25T00:31:14.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f1e10f640 1 -- 172.21.15.157:0/3649300889 learned_addr learned my addr 172.21.15.157:0/3649300889 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:14.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f1d10d640 1 -- 172.21.15.157:0/3649300889 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3512653212 0 0) 0x7f3f1815fef0 con 0x7f3f1815f930 2024-04-25T00:31:14.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f1d10d640 1 -- 172.21.15.157:0/3649300889 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f04003530 con 0x7f3f1815f930 2024-04-25T00:31:14.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f1e10f640 1 -- 172.21.15.157:0/3649300889 >> v1:172.21.15.157:6789/0 conn(0x7f3f1815f930 legacy=0x7f3f1806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f1e10f640 1 -- 172.21.15.157:0/3649300889 >> v1:172.21.15.29:6789/0 conn(0x7f3f18070580 legacy=0x7f3f1813f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f1e10f640 1 -- 172.21.15.157:0/3649300889 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f181604b0 con 0x7f3f180701a0 2024-04-25T00:31:14.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f1e10f640 1 --2- 172.21.15.157:0/3649300889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f180701a0 0x7f3f1815cd70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3f140098a0 tx=0x7f3f1401c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe5a76e9d103f4ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:14.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f1d10d640 1 -- 172.21.15.157:0/3649300889 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f140ab660 con 0x7f3f180701a0 2024-04-25T00:31:14.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f1d10d640 1 -- 172.21.15.157:0/3649300889 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f140b0070 con 0x7f3f180701a0 2024-04-25T00:31:14.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f1d10d640 1 -- 172.21.15.157:0/3649300889 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f140aba20 con 0x7f3f180701a0 2024-04-25T00:31:14.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f20373640 1 -- 172.21.15.157:0/3649300889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f180701a0 msgr2=0x7f3f1815cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f20373640 1 --2- 172.21.15.157:0/3649300889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f180701a0 0x7f3f1815cd70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3f140098a0 tx=0x7f3f1401c550 comp rx=0 tx=0).stop 2024-04-25T00:31:14.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f20373640 1 -- 172.21.15.157:0/3649300889 shutdown_connections 2024-04-25T00:31:14.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f20373640 1 --2- 172.21.15.157:0/3649300889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f180701a0 0x7f3f1815cd70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f20373640 1 -- 172.21.15.157:0/3649300889 >> 172.21.15.157:0/3649300889 conn(0x7f3f180d0c30 msgr2=0x7f3f180d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:14.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f20373640 1 -- 172.21.15.157:0/3649300889 shutdown_connections 2024-04-25T00:31:14.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.193+0000 7f3f20373640 1 -- 172.21.15.157:0/3649300889 wait complete. 2024-04-25T00:31:14.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f20373640 1 Processor -- start 2024-04-25T00:31:14.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f20373640 1 -- start start 2024-04-25T00:31:14.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f20373640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180701a0 0x7f3f180c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f20373640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f18070580 0x7f3f1806ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f20373640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f1815f930 0x7f3f18150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f20373640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1806baa0 con 0x7f3f1815f930 2024-04-25T00:31:14.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f20373640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1806b920 con 0x7f3f180701a0 2024-04-25T00:31:14.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f20373640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3f18072310 con 0x7f3f18070580 2024-04-25T00:31:14.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f1e10f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180701a0 0x7f3f180c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f1e10f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180701a0 0x7f3f180c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36988/0 (socket says 172.21.15.157:36988) 2024-04-25T00:31:14.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f1e10f640 1 -- 172.21.15.157:0/1568154705 learned_addr learned my addr 172.21.15.157:0/1568154705 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:14.203 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f1d90e640 1 --2- 172.21.15.157:0/1568154705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f18070580 0x7f3f1806ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f1e910640 1 --2- 172.21.15.157:0/1568154705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f1815f930 0x7f3f18150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f1e10f640 1 -- 172.21.15.157:0/1568154705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f18070580 msgr2=0x7f3f1806ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f1e10f640 1 --2- 172.21.15.157:0/1568154705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f18070580 0x7f3f1806ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f1e10f640 1 -- 172.21.15.157:0/1568154705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f1815f930 msgr2=0x7f3f18150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f1e10f640 1 --2- 172.21.15.157:0/1568154705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f1815f930 0x7f3f18150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f1e10f640 1 -- 172.21.15.157:0/1568154705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f18152b90 con 0x7f3f180701a0 2024-04-25T00:31:14.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f1e10f640 1 --2- 172.21.15.157:0/1568154705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180701a0 0x7f3f180c7550 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3f140ad040 tx=0x7f3f1401ad90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:14.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f0effd640 1 -- 172.21.15.157:0/1568154705 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f140be5f0 con 0x7f3f180701a0 2024-04-25T00:31:14.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f0effd640 1 -- 172.21.15.157:0/1568154705 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f140be7b0 con 0x7f3f180701a0 2024-04-25T00:31:14.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f0effd640 1 -- 172.21.15.157:0/1568154705 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f140beba0 con 0x7f3f180701a0 2024-04-25T00:31:14.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f20373640 1 -- 172.21.15.157:0/1568154705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f18152e50 con 0x7f3f180701a0 2024-04-25T00:31:14.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f20373640 1 -- 172.21.15.157:0/1568154705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f18140540 con 0x7f3f180701a0 2024-04-25T00:31:14.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f0effd640 1 -- 172.21.15.157:0/1568154705 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3f140b4780 con 0x7f3f180701a0 2024-04-25T00:31:14.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.197+0000 7f3f0effd640 1 -- 172.21.15.157:0/1568154705 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f3f140c8c70 con 0x7f3f180701a0 2024-04-25T00:31:14.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.201+0000 7f3f20373640 1 -- 172.21.15.157:0/1568154705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f18069270 con 0x7f3f180701a0 2024-04-25T00:31:14.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.201+0000 7f3f0effd640 1 -- 172.21.15.157:0/1568154705 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3f14016170 con 0x7f3f180701a0 2024-04-25T00:31:14.249 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:14.245+0000 7f487255b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:31:14.274 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.269+0000 7f3f20373640 1 -- 172.21.15.157:0/1568154705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "80ae2f99-2daf-47c9-b454-978268265b24", "id": 4} v 0) -- 0x7f3f1813eba0 con 0x7f3f180701a0 2024-04-25T00:31:14.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.273+0000 7f3f0effd640 1 -- 172.21.15.157:0/1568154705 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "80ae2f99-2daf-47c9-b454-978268265b24", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f3f140b4bb0 con 0x7f3f180701a0 2024-04-25T00:31:14.279 INFO:teuthology.orchestra.run.smithi157.stdout:4 2024-04-25T00:31:14.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.277+0000 7f3f20373640 1 -- 172.21.15.157:0/1568154705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180701a0 msgr2=0x7f3f180c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.277+0000 7f3f20373640 1 --2- 172.21.15.157:0/1568154705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180701a0 0x7f3f180c7550 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3f140ad040 tx=0x7f3f1401ad90 comp rx=0 tx=0).stop 2024-04-25T00:31:14.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.277+0000 7f3f20373640 1 -- 172.21.15.157:0/1568154705 shutdown_connections 2024-04-25T00:31:14.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.277+0000 7f3f20373640 1 --2- 172.21.15.157:0/1568154705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f1815f930 0x7f3f18150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.277+0000 7f3f20373640 1 --2- 172.21.15.157:0/1568154705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f18070580 0x7f3f1806ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.277+0000 7f3f20373640 1 --2- 172.21.15.157:0/1568154705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180701a0 0x7f3f180c7550 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.277+0000 7f3f20373640 1 -- 172.21.15.157:0/1568154705 >> 172.21.15.157:0/1568154705 conn(0x7f3f180d0c30 msgr2=0x7f3f180b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:14.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.277+0000 7f3f20373640 1 -- 172.21.15.157:0/1568154705 shutdown_connections 2024-04-25T00:31:14.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.277+0000 7f3f20373640 1 -- 172.21.15.157:0/1568154705 wait complete. 2024-04-25T00:31:14.281 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:14.281+0000 7f98f227a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:31:14.295 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new ba24b2d8-51c4-4a54-abd1-cafb823ae33e 5 2024-04-25T00:31:14.306 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:14.301+0000 7f487255b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:31:14.338 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:14.337+0000 7f98f227a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:31:14.361 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:14.357+0000 7f487255b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:31:14.392 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:14.393+0000 7f98f227a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:31:14.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.405+0000 7fd77a9e8640 1 Processor -- start 2024-04-25T00:31:14.411 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.405+0000 7fd77a9e8640 1 -- start start 2024-04-25T00:31:14.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd77a9e8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd77415a490 0x7fd77406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd77a9e8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd7740b3860 0x7fd7740b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd77a9e8640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd77406e930 con 0x7fd77415a870 2024-04-25T00:31:14.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd77a9e8640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd77415ad60 con 0x7fd7740b3860 2024-04-25T00:31:14.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd77a9e8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd77415b040 con 0x7fd77415a490 2024-04-25T00:31:14.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd773fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd77415a490 0x7fd77406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd773fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd77415a490 0x7fd77406df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36996/0 (socket says 172.21.15.157:36996) 2024-04-25T00:31:14.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd773fff640 1 -- 172.21.15.157:0/2747631026 learned_addr learned my addr 172.21.15.157:0/2747631026 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:14.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd76b7fe640 1 --2- 172.21.15.157:0/2747631026 >> v2:172.21.15.29:3300/0 conn(0x7fd7740b3860 0x7fd7740b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd773fff640 1 -- 172.21.15.157:0/2747631026 >> v2:172.21.15.29:3300/0 conn(0x7fd7740b3860 msgr2=0x7fd7740b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:14.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd773fff640 1 --2- 172.21.15.157:0/2747631026 >> v2:172.21.15.29:3300/0 conn(0x7fd7740b3860 0x7fd7740b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd773fff640 1 -- 172.21.15.157:0/2747631026 >> v1:172.21.15.29:6789/0 conn(0x7fd77415a870 legacy=0x7fd774071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:14.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd773fff640 1 -- 172.21.15.157:0/2747631026 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77415b320 con 0x7fd77415a490 2024-04-25T00:31:14.414 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd773fff640 1 --2- 172.21.15.157:0/2747631026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd77415a490 0x7fd77406df90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd774072600 tx=0x7fd76401eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbd1f27df9ca0464 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:14.415 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd7737fe640 1 -- 172.21.15.157:0/2747631026 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76400b2d0 con 0x7fd77415a490 2024-04-25T00:31:14.415 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd7737fe640 1 -- 172.21.15.157:0/2747631026 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd77406e6e0 con 0x7fd77415a490 2024-04-25T00:31:14.415 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd7737fe640 1 -- 172.21.15.157:0/2747631026 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7640a7bb0 con 0x7fd77415a490 2024-04-25T00:31:14.415 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd77a9e8640 1 -- 172.21.15.157:0/2747631026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd77415a490 msgr2=0x7fd77406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.416 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.409+0000 7fd77a9e8640 1 --2- 172.21.15.157:0/2747631026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd77415a490 0x7fd77406df90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd774072600 tx=0x7fd76401eb20 comp rx=0 tx=0).stop 2024-04-25T00:31:14.416 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 -- 172.21.15.157:0/2747631026 shutdown_connections 2024-04-25T00:31:14.416 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 --2- 172.21.15.157:0/2747631026 >> v2:172.21.15.29:3300/0 conn(0x7fd7740b3860 0x7fd7740b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.416 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 --2- 172.21.15.157:0/2747631026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd77415a490 0x7fd77406df90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.416 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 -- 172.21.15.157:0/2747631026 >> 172.21.15.157:0/2747631026 conn(0x7fd774074c00 msgr2=0x7fd77406b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:14.416 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 -- 172.21.15.157:0/2747631026 shutdown_connections 2024-04-25T00:31:14.416 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 -- 172.21.15.157:0/2747631026 wait complete. 2024-04-25T00:31:14.417 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 Processor -- start 2024-04-25T00:31:14.417 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 -- start start 2024-04-25T00:31:14.419 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7740b3860 0x7fd774146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.419 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd77415a490 0x7fd7740b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.419 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd77415a870 0x7fd7740b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.419 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd774072000 con 0x7fd77415a490 2024-04-25T00:31:14.419 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd774071e80 con 0x7fd7740b3860 2024-04-25T00:31:14.419 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd774072180 con 0x7fd77415a870 2024-04-25T00:31:14.419 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd773fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7740b3860 0x7fd774146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd773fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7740b3860 0x7fd774146a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37012/0 (socket says 172.21.15.157:37012) 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd773fff640 1 -- 172.21.15.157:0/777692256 learned_addr learned my addr 172.21.15.157:0/777692256 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd778f85640 1 --2- 172.21.15.157:0/777692256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd77415a870 0x7fd7740b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd76b7fe640 1 --2- 172.21.15.157:0/777692256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd77415a490 0x7fd7740b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd773fff640 1 -- 172.21.15.157:0/777692256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd77415a870 msgr2=0x7fd7740b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd773fff640 1 --2- 172.21.15.157:0/777692256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd77415a870 0x7fd7740b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd773fff640 1 -- 172.21.15.157:0/777692256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd77415a490 msgr2=0x7fd7740b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd773fff640 1 --2- 172.21.15.157:0/777692256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd77415a490 0x7fd7740b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd773fff640 1 -- 172.21.15.157:0/777692256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7740cf270 con 0x7fd7740b3860 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd76b7fe640 1 --2- 172.21.15.157:0/777692256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd77415a490 0x7fd7740b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd773fff640 1 --2- 172.21.15.157:0/777692256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7740b3860 0x7fd774146a50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7640b1040 tx=0x7fd76401b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd7717fa640 1 -- 172.21.15.157:0/777692256 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7640be020 con 0x7fd7740b3860 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd7717fa640 1 -- 172.21.15.157:0/777692256 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd764009600 con 0x7fd7740b3860 2024-04-25T00:31:14.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 -- 172.21.15.157:0/777692256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7740cf550 con 0x7fd7740b3860 2024-04-25T00:31:14.421 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd7717fa640 1 -- 172.21.15.157:0/777692256 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764009a10 con 0x7fd7740b3860 2024-04-25T00:31:14.421 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd77a9e8640 1 -- 172.21.15.157:0/777692256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7741556e0 con 0x7fd7740b3860 2024-04-25T00:31:14.421 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd7717fa640 1 -- 172.21.15.157:0/777692256 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd7640c1020 con 0x7fd7740b3860 2024-04-25T00:31:14.421 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.413+0000 7fd7717fa640 1 -- 172.21.15.157:0/777692256 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fd7640c3980 con 0x7fd7740b3860 2024-04-25T00:31:14.421 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.417+0000 7fd77a9e8640 1 -- 172.21.15.157:0/777692256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd774155220 con 0x7fd7740b3860 2024-04-25T00:31:14.421 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.417+0000 7fd7717fa640 1 -- 172.21.15.157:0/777692256 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd774155220 con 0x7fd7740b3860 2024-04-25T00:31:14.442 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:14.437+0000 7f487255b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:31:14.473 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:14.473+0000 7f98f227a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:31:14.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.481+0000 7fd77a9e8640 1 -- 172.21.15.157:0/777692256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ba24b2d8-51c4-4a54-abd1-cafb823ae33e", "id": 5} v 0) -- 0x7fd774069270 con 0x7fd7740b3860 2024-04-25T00:31:14.490 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.485+0000 7fd7717fa640 1 -- 172.21.15.157:0/777692256 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ba24b2d8-51c4-4a54-abd1-cafb823ae33e", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fd7640c8030 con 0x7fd7740b3860 2024-04-25T00:31:14.491 INFO:teuthology.orchestra.run.smithi157.stdout:5 2024-04-25T00:31:14.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.489+0000 7fd77a9e8640 1 -- 172.21.15.157:0/777692256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7740b3860 msgr2=0x7fd774146a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.489+0000 7fd77a9e8640 1 --2- 172.21.15.157:0/777692256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7740b3860 0x7fd774146a50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7640b1040 tx=0x7fd76401b8c0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.489+0000 7fd77a9e8640 1 -- 172.21.15.157:0/777692256 shutdown_connections 2024-04-25T00:31:14.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.489+0000 7fd77a9e8640 1 --2- 172.21.15.157:0/777692256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd77415a870 0x7fd7740b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.489+0000 7fd77a9e8640 1 --2- 172.21.15.157:0/777692256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd77415a490 0x7fd7740b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.489+0000 7fd77a9e8640 1 --2- 172.21.15.157:0/777692256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7740b3860 0x7fd774146a50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.489+0000 7fd77a9e8640 1 -- 172.21.15.157:0/777692256 >> 172.21.15.157:0/777692256 conn(0x7fd774074c00 msgr2=0x7fd7740b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:14.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.489+0000 7fd77a9e8640 1 -- 172.21.15.157:0/777692256 shutdown_connections 2024-04-25T00:31:14.492 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.489+0000 7fd77a9e8640 1 -- 172.21.15.157:0/777692256 wait complete. 2024-04-25T00:31:14.495 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:14.493+0000 7f487255b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:31:14.507 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 99523015-c5be-4adc-bc1c-6b44b0cbe928 6 2024-04-25T00:31:14.527 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:14.525+0000 7f98f227a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:31:14.575 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:14.573+0000 7f487255b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:31:14.608 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:14.605+0000 7f98f227a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:31:14.631 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:14.625+0000 7f487255b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:31:14.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.657+0000 7fa5e7b37640 1 Processor -- start 2024-04-25T00:31:14.662 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.657+0000 7fa5e7b37640 1 -- start start 2024-04-25T00:31:14.663 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.657+0000 7fa5e7b37640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa5e006ea50 0x7fa5e00d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.657+0000 7fa5e7b37640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5e006ef10 0x7fa5e00d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.657+0000 7fa5e7b37640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5e00722e0 con 0x7fa5e015be30 2024-04-25T00:31:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.657+0000 7fa5e7b37640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e015fa90 con 0x7fa5e006ea50 2024-04-25T00:31:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.657+0000 7fa5e7b37640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e00ccd80 con 0x7fa5e006ef10 2024-04-25T00:31:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.657+0000 7fa5e60d4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5e006ef10 0x7fa5e00d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.657+0000 7fa5e60d4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5e006ef10 0x7fa5e00d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37018/0 (socket says 172.21.15.157:37018) 2024-04-25T00:31:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.657+0000 7fa5e60d4640 1 -- 172.21.15.157:0/3826125028 learned_addr learned my addr 172.21.15.157:0/3826125028 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e50d2640 1 --2- 172.21.15.157:0/3826125028 >> v2:172.21.15.29:3300/0 conn(0x7fa5e006ea50 0x7fa5e00d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e60d4640 1 -- 172.21.15.157:0/3826125028 >> v2:172.21.15.29:3300/0 conn(0x7fa5e006ea50 msgr2=0x7fa5e00d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e60d4640 1 --2- 172.21.15.157:0/3826125028 >> v2:172.21.15.29:3300/0 conn(0x7fa5e006ea50 0x7fa5e00d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e60d4640 1 -- 172.21.15.157:0/3826125028 >> v1:172.21.15.29:6789/0 conn(0x7fa5e015be30 legacy=0x7fa5e00ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e60d4640 1 -- 172.21.15.157:0/3826125028 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e00cd060 con 0x7fa5e006ef10 2024-04-25T00:31:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e50d2640 1 --2- 172.21.15.157:0/3826125028 >> v2:172.21.15.29:3300/0 conn(0x7fa5e006ea50 0x7fa5e00d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:14.665 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:14.661+0000 7f98f227a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:31:14.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e60d4640 1 --2- 172.21.15.157:0/3826125028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e006ef10 0x7fa5e00d0690 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00b59d0 tx=0x7fa5dc01c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2fb55e4599d8655 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:14.665 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e48d1640 1 -- 172.21.15.157:0/3826125028 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0a7070 con 0x7fa5e006ef10 2024-04-25T00:31:14.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e48d1640 1 -- 172.21.15.157:0/3826125028 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5dc0b0070 con 0x7fa5e006ef10 2024-04-25T00:31:14.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e48d1640 1 -- 172.21.15.157:0/3826125028 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0ab890 con 0x7fa5e006ef10 2024-04-25T00:31:14.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3826125028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e006ef10 msgr2=0x7fa5e00d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.667 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e7b37640 1 --2- 172.21.15.157:0/3826125028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e006ef10 0x7fa5e00d0690 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00b59d0 tx=0x7fa5dc01c4b0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.667 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3826125028 shutdown_connections 2024-04-25T00:31:14.667 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e7b37640 1 --2- 172.21.15.157:0/3826125028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e006ef10 0x7fa5e00d0690 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.667 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e7b37640 1 --2- 172.21.15.157:0/3826125028 >> v2:172.21.15.29:3300/0 conn(0x7fa5e006ea50 0x7fa5e00d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.667 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3826125028 >> 172.21.15.157:0/3826125028 conn(0x7fa5e0074c00 msgr2=0x7fa5e006b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:14.667 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3826125028 shutdown_connections 2024-04-25T00:31:14.667 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.661+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3826125028 wait complete. 2024-04-25T00:31:14.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e7b37640 1 Processor -- start 2024-04-25T00:31:14.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e7b37640 1 -- start start 2024-04-25T00:31:14.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e7b37640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e006ea50 0x7fa5e0144df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e7b37640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e006ef10 0x7fa5e015a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e7b37640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5e015be30 0x7fa5e015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e7b37640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0168da0 con 0x7fa5e006ef10 2024-04-25T00:31:14.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e7b37640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0071f70 con 0x7fa5e006ea50 2024-04-25T00:31:14.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e7b37640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e0168f20 con 0x7fa5e015be30 2024-04-25T00:31:14.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e58d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e006ea50 0x7fa5e0144df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e60d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5e015be30 0x7fa5e015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e58d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e006ea50 0x7fa5e0144df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37026/0 (socket says 172.21.15.157:37026) 2024-04-25T00:31:14.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e58d3640 1 -- 172.21.15.157:0/3796561605 learned_addr learned my addr 172.21.15.157:0/3796561605 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:14.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e50d2640 1 --2- 172.21.15.157:0/3796561605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e006ef10 0x7fa5e015a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e60d4640 1 -- 172.21.15.157:0/3796561605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e006ea50 msgr2=0x7fa5e0144df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e60d4640 1 --2- 172.21.15.157:0/3796561605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e006ea50 0x7fa5e0144df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e60d4640 1 -- 172.21.15.157:0/3796561605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e006ef10 msgr2=0x7fa5e015a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e60d4640 1 --2- 172.21.15.157:0/3796561605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e006ef10 0x7fa5e015a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e60d4640 1 -- 172.21.15.157:0/3796561605 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e01595a0 con 0x7fa5e015be30 2024-04-25T00:31:14.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e58d3640 1 --2- 172.21.15.157:0/3796561605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e006ea50 0x7fa5e0144df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:14.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e60d4640 1 --2- 172.21.15.157:0/3796561605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5e015be30 0x7fa5e015aa60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0ac000 tx=0x7fa5dc01acf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:14.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5d67fc640 1 -- 172.21.15.157:0/3796561605 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0abe60 con 0x7fa5e015be30 2024-04-25T00:31:14.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5d67fc640 1 -- 172.21.15.157:0/3796561605 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5dc0b4820 con 0x7fa5e015be30 2024-04-25T00:31:14.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3796561605 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e0159880 con 0x7fa5e015be30 2024-04-25T00:31:14.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5d67fc640 1 -- 172.21.15.157:0/3796561605 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0b4c10 con 0x7fa5e015be30 2024-04-25T00:31:14.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3796561605 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e00a99c0 con 0x7fa5e015be30 2024-04-25T00:31:14.672 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5d67fc640 1 -- 172.21.15.157:0/3796561605 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa5dc0be780 con 0x7fa5e015be30 2024-04-25T00:31:14.672 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5d67fc640 1 -- 172.21.15.157:0/3796561605 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fa5dc0b4db0 con 0x7fa5e015be30 2024-04-25T00:31:14.672 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.665+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3796561605 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e0069270 con 0x7fa5e015be30 2024-04-25T00:31:14.672 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.669+0000 7fa5d67fc640 1 -- 172.21.15.157:0/3796561605 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa5dc0c4060 con 0x7fa5e015be30 2024-04-25T00:31:14.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.733+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3796561605 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "99523015-c5be-4adc-bc1c-6b44b0cbe928", "id": 6} v 0) -- 0x7fa5e0075660 con 0x7fa5e015be30 2024-04-25T00:31:14.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.737+0000 7fa5d67fc640 1 -- 172.21.15.157:0/3796561605 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "99523015-c5be-4adc-bc1c-6b44b0cbe928", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fa5dc0160d0 con 0x7fa5e015be30 2024-04-25T00:31:14.743 INFO:teuthology.orchestra.run.smithi157.stdout:6 2024-04-25T00:31:14.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.741+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3796561605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5e015be30 msgr2=0x7fa5e015aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.741+0000 7fa5e7b37640 1 --2- 172.21.15.157:0/3796561605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5e015be30 0x7fa5e015aa60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0ac000 tx=0x7fa5dc01acf0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.741+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3796561605 shutdown_connections 2024-04-25T00:31:14.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.741+0000 7fa5e7b37640 1 --2- 172.21.15.157:0/3796561605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5e015be30 0x7fa5e015aa60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.741+0000 7fa5e7b37640 1 --2- 172.21.15.157:0/3796561605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e006ef10 0x7fa5e015a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.741+0000 7fa5e7b37640 1 --2- 172.21.15.157:0/3796561605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e006ea50 0x7fa5e0144df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.741+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3796561605 >> 172.21.15.157:0/3796561605 conn(0x7fa5e0074c00 msgr2=0x7fa5e00cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:14.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.741+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3796561605 shutdown_connections 2024-04-25T00:31:14.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.741+0000 7fa5e7b37640 1 -- 172.21.15.157:0/3796561605 wait complete. 2024-04-25T00:31:14.759 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 898aac88-8174-4b35-8e59-cd06cef7d993 7 2024-04-25T00:31:14.787 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:14.781+0000 7f487255b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:31:14.821 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:14.821+0000 7f98f227a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:31:14.851 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:14.849+0000 7f487255b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:31:14.885 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:14.885+0000 7f98f227a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:31:14.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c9d88640 1 Processor -- start 2024-04-25T00:31:14.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c9d88640 1 -- start start 2024-04-25T00:31:14.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c9d88640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c40cddc0 0x7fb3c40cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c9d88640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3c4070060 0x7fb3c415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c9d88640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3c40ce1a0 con 0x7fb3c40d11f0 2024-04-25T00:31:14.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c9d88640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb3c40d0d60 con 0x7fb3c4070060 2024-04-25T00:31:14.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c9d88640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c40cf540 con 0x7fb3c40cddc0 2024-04-25T00:31:14.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c37fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c40cddc0 0x7fb3c40cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3c4070060 0x7fb3c415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c37fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c40cddc0 0x7fb3c40cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:55898/0 (socket says 172.21.15.157:55898) 2024-04-25T00:31:14.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c37fe640 1 -- 172.21.15.157:0/3996489386 learned_addr learned my addr 172.21.15.157:0/3996489386 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:14.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c37fe640 1 -- 172.21.15.157:0/3996489386 >> v2:172.21.15.157:3300/0 conn(0x7fb3c4070060 msgr2=0x7fb3c415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:14.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c37fe640 1 --2- 172.21.15.157:0/3996489386 >> v2:172.21.15.157:3300/0 conn(0x7fb3c4070060 0x7fb3c415d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.889+0000 7fb3c37fe640 1 -- 172.21.15.157:0/3996489386 >> v1:172.21.15.29:6789/0 conn(0x7fb3c40d11f0 legacy=0x7fb3c406f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:14.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c37fe640 1 -- 172.21.15.157:0/3996489386 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c40d0150 con 0x7fb3c40cddc0 2024-04-25T00:31:14.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c3fff640 1 --2- 172.21.15.157:0/3996489386 >> v2:172.21.15.157:3300/0 conn(0x7fb3c4070060 0x7fb3c415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:14.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c37fe640 1 --2- 172.21.15.157:0/3996489386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c40cddc0 0x7fb3c40cf000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009870 tx=0x7fb3b401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51ad34713de897b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:14.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c2ffd640 1 -- 172.21.15.157:0/3996489386 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40b0070 con 0x7fb3c40cddc0 2024-04-25T00:31:14.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c2ffd640 1 -- 172.21.15.157:0/3996489386 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b40ab400 con 0x7fb3c40cddc0 2024-04-25T00:31:14.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c2ffd640 1 -- 172.21.15.157:0/3996489386 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40ab7d0 con 0x7fb3c40cddc0 2024-04-25T00:31:14.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c9d88640 1 -- 172.21.15.157:0/3996489386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c40cddc0 msgr2=0x7fb3c40cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c9d88640 1 --2- 172.21.15.157:0/3996489386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c40cddc0 0x7fb3c40cf000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009870 tx=0x7fb3b401c490 comp rx=0 tx=0).stop 2024-04-25T00:31:14.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c9d88640 1 -- 172.21.15.157:0/3996489386 shutdown_connections 2024-04-25T00:31:14.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c9d88640 1 --2- 172.21.15.157:0/3996489386 >> v2:172.21.15.157:3300/0 conn(0x7fb3c4070060 0x7fb3c415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c9d88640 1 --2- 172.21.15.157:0/3996489386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c40cddc0 0x7fb3c40cf000 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c9d88640 1 -- 172.21.15.157:0/3996489386 >> 172.21.15.157:0/3996489386 conn(0x7fb3c40b3920 msgr2=0x7fb3c40b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:14.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c9d88640 1 -- 172.21.15.157:0/3996489386 shutdown_connections 2024-04-25T00:31:14.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.893+0000 7fb3c9d88640 1 -- 172.21.15.157:0/3996489386 wait complete. 2024-04-25T00:31:14.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c9d88640 1 Processor -- start 2024-04-25T00:31:14.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c9d88640 1 -- start start 2024-04-25T00:31:14.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c9d88640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c4070060 0x7fb3c415a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c9d88640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3c40cddc0 0x7fb3c4153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c9d88640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c40d11f0 0x7fb3c40cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:14.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c9d88640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c40cc540 con 0x7fb3c40cddc0 2024-04-25T00:31:14.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c9d88640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c40cc3c0 con 0x7fb3c40d11f0 2024-04-25T00:31:14.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c9d88640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c40cc6c0 con 0x7fb3c4070060 2024-04-25T00:31:14.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c37fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c4070060 0x7fb3c415a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c40d11f0 0x7fb3c40cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c37fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c4070060 0x7fb3c415a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:55902/0 (socket says 172.21.15.157:55902) 2024-04-25T00:31:14.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c37fe640 1 -- 172.21.15.157:0/1864710014 learned_addr learned my addr 172.21.15.157:0/1864710014 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-25T00:31:14.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3baffd640 1 --2- 172.21.15.157:0/1864710014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3c40cddc0 0x7fb3c4153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:14.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c37fe640 1 -- 172.21.15.157:0/1864710014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c40d11f0 msgr2=0x7fb3c40cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c37fe640 1 --2- 172.21.15.157:0/1864710014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c40d11f0 0x7fb3c40cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c37fe640 1 -- 172.21.15.157:0/1864710014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3c40cddc0 msgr2=0x7fb3c4153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c37fe640 1 --2- 172.21.15.157:0/1864710014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3c40cddc0 0x7fb3c4153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c37fe640 1 -- 172.21.15.157:0/1864710014 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c40af100 con 0x7fb3c4070060 2024-04-25T00:31:14.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c3fff640 1 --2- 172.21.15.157:0/1864710014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c40d11f0 0x7fb3c40cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:14.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c37fe640 1 --2- 172.21.15.157:0/1864710014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c4070060 0x7fb3c415a4d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb3b401cd30 tx=0x7fb3b4019230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:14.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c0ff9640 1 -- 172.21.15.157:0/1864710014 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40b9070 con 0x7fb3c4070060 2024-04-25T00:31:14.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c0ff9640 1 -- 172.21.15.157:0/1864710014 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b40b4620 con 0x7fb3c4070060 2024-04-25T00:31:14.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c9d88640 1 -- 172.21.15.157:0/1864710014 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c40af3e0 con 0x7fb3c4070060 2024-04-25T00:31:14.903 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c0ff9640 1 -- 172.21.15.157:0/1864710014 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40b0040 con 0x7fb3c4070060 2024-04-25T00:31:14.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c9d88640 1 -- 172.21.15.157:0/1864710014 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c40af9a0 con 0x7fb3c4070060 2024-04-25T00:31:14.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c0ff9640 1 -- 172.21.15.157:0/1864710014 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb3b40bc070 con 0x7fb3c4070060 2024-04-25T00:31:14.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c0ff9640 1 -- 172.21.15.157:0/1864710014 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fb3b40bda10 con 0x7fb3c4070060 2024-04-25T00:31:14.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.897+0000 7fb3c9d88640 1 -- 172.21.15.157:0/1864710014 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c4069270 con 0x7fb3c4070060 2024-04-25T00:31:14.906 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.901+0000 7fb3c0ff9640 1 -- 172.21.15.157:0/1864710014 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb3b4016110 con 0x7fb3c4070060 2024-04-25T00:31:14.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.965+0000 7fb3c9d88640 1 -- 172.21.15.157:0/1864710014 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "898aac88-8174-4b35-8e59-cd06cef7d993", "id": 7} v 0) -- 0x7fb3c415a1a0 con 0x7fb3c4070060 2024-04-25T00:31:14.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.969+0000 7fb3c0ff9640 1 -- 172.21.15.157:0/1864710014 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "898aac88-8174-4b35-8e59-cd06cef7d993", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fb3b40be440 con 0x7fb3c4070060 2024-04-25T00:31:14.974 INFO:teuthology.orchestra.run.smithi157.stdout:7 2024-04-25T00:31:14.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.973+0000 7fb3c9d88640 1 -- 172.21.15.157:0/1864710014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c4070060 msgr2=0x7fb3c415a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:14.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.973+0000 7fb3c9d88640 1 --2- 172.21.15.157:0/1864710014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c4070060 0x7fb3c415a4d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb3b401cd30 tx=0x7fb3b4019230 comp rx=0 tx=0).stop 2024-04-25T00:31:14.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.973+0000 7fb3c9d88640 1 -- 172.21.15.157:0/1864710014 shutdown_connections 2024-04-25T00:31:14.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.973+0000 7fb3c9d88640 1 --2- 172.21.15.157:0/1864710014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c40d11f0 0x7fb3c40cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.976 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.973+0000 7fb3c9d88640 1 --2- 172.21.15.157:0/1864710014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3c40cddc0 0x7fb3c4153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.973+0000 7fb3c9d88640 1 --2- 172.21.15.157:0/1864710014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3c4070060 0x7fb3c415a4d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:14.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.973+0000 7fb3c9d88640 1 -- 172.21.15.157:0/1864710014 >> 172.21.15.157:0/1864710014 conn(0x7fb3c40b3920 msgr2=0x7fb3c40b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:14.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.973+0000 7fb3c9d88640 1 -- 172.21.15.157:0/1864710014 shutdown_connections 2024-04-25T00:31:14.977 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:14.973+0000 7fb3c9d88640 1 -- 172.21.15.157:0/1864710014 wait complete. 2024-04-25T00:31:14.990 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T00:31:14.991 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T00:31:14.993 INFO:tasks.ceph.osd.0:Started 2024-04-25T00:31:14.993 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T00:31:14.993 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T00:31:14.998 INFO:tasks.ceph.osd.1:Started 2024-04-25T00:31:14.998 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T00:31:14.998 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T00:31:15.000 INFO:tasks.ceph.osd.2:Started 2024-04-25T00:31:15.001 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T00:31:15.001 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T00:31:15.003 INFO:tasks.ceph.osd.3:Started 2024-04-25T00:31:15.003 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T00:31:15.003 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T00:31:15.005 INFO:tasks.ceph.osd.4:Started 2024-04-25T00:31:15.005 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T00:31:15.006 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T00:31:15.007 INFO:tasks.ceph.osd.5:Started 2024-04-25T00:31:15.007 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T00:31:15.008 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T00:31:15.009 INFO:tasks.ceph.osd.6:Started 2024-04-25T00:31:15.009 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T00:31:15.009 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T00:31:15.011 INFO:tasks.ceph.osd.7:Started 2024-04-25T00:31:15.012 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:15.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.133+0000 7f4c30d27640 1 Processor -- start 2024-04-25T00:31:15.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.133+0000 7f4c30d27640 1 -- start start 2024-04-25T00:31:15.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c30d27640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4c2c15af40 0x7f4c2c0741a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c30d27640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2c0ccbd0 0x7f4c2c0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c30d27640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c2c15fa70 con 0x7f4c2c15f570 2024-04-25T00:31:15.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c30d27640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4c2c0cf8c0 con 0x7f4c2c15af40 2024-04-25T00:31:15.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c30d27640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2c0746e0 con 0x7f4c2c0ccbd0 2024-04-25T00:31:15.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c2a59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4c2c15af40 0x7f4c2c0741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c2a59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4c2c15af40 0x7f4c2c0741a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45372/0 (socket says 172.21.15.29:45372) 2024-04-25T00:31:15.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c2a59c640 1 -- 172.21.15.29:0/3843292290 learned_addr learned my addr 172.21.15.29:0/3843292290 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:15.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c2ad9d640 1 --2- 172.21.15.29:0/3843292290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2c0ccbd0 0x7f4c2c0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c2a59c640 1 -- 172.21.15.29:0/3843292290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2c0ccbd0 msgr2=0x7f4c2c0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c2a59c640 1 --2- 172.21.15.29:0/3843292290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2c0ccbd0 0x7f4c2c0cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c2a59c640 1 -- 172.21.15.29:0/3843292290 >> v1:172.21.15.157:6789/0 conn(0x7f4c2c15f570 legacy=0x7f4c2c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c2a59c640 1 -- 172.21.15.29:0/3843292290 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2c0d0230 con 0x7f4c2c15af40 2024-04-25T00:31:15.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c2a59c640 1 --2- 172.21.15.29:0/3843292290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15af40 0x7f4c2c0741a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009e50 tx=0x7f4c1c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50213d59ac01e129 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c2959a640 1 -- 172.21.15.29:0/3843292290 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0a5700 con 0x7f4c2c15af40 2024-04-25T00:31:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c2959a640 1 -- 172.21.15.29:0/3843292290 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c1c0a58a0 con 0x7f4c2c15af40 2024-04-25T00:31:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c2959a640 1 -- 172.21.15.29:0/3843292290 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0a5c70 con 0x7f4c2c15af40 2024-04-25T00:31:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c30d27640 1 -- 172.21.15.29:0/3843292290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15af40 msgr2=0x7f4c2c0741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c30d27640 1 --2- 172.21.15.29:0/3843292290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15af40 0x7f4c2c0741a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009e50 tx=0x7f4c1c01ca30 comp rx=0 tx=0).stop 2024-04-25T00:31:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c30d27640 1 -- 172.21.15.29:0/3843292290 shutdown_connections 2024-04-25T00:31:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c30d27640 1 --2- 172.21.15.29:0/3843292290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2c0ccbd0 0x7f4c2c0cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c30d27640 1 --2- 172.21.15.29:0/3843292290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15af40 0x7f4c2c0741a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c30d27640 1 -- 172.21.15.29:0/3843292290 >> 172.21.15.29:0/3843292290 conn(0x7f4c2c0b5820 msgr2=0x7f4c2c0b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c30d27640 1 -- 172.21.15.29:0/3843292290 shutdown_connections 2024-04-25T00:31:15.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.137+0000 7f4c30d27640 1 -- 172.21.15.29:0/3843292290 wait complete. 2024-04-25T00:31:15.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c30d27640 1 Processor -- start 2024-04-25T00:31:15.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c30d27640 1 -- start start 2024-04-25T00:31:15.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c30d27640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c2c0ccbd0 0x7f4c2c143ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c30d27640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2c15af40 0x7f4c2c158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c30d27640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15f570 0x7f4c2c09c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c30d27640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c15b4a0 con 0x7f4c2c15f570 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c30d27640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c15b320 con 0x7f4c2c0ccbd0 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c30d27640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2c167cf0 con 0x7f4c2c15af40 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c2ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15f570 0x7f4c2c09c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c2ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15f570 0x7f4c2c09c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45378/0 (socket says 172.21.15.29:45378) 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c2ad9d640 1 -- 172.21.15.29:0/2164391244 learned_addr learned my addr 172.21.15.29:0/2164391244 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c29d9b640 1 --2- 172.21.15.29:0/2164391244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2c15af40 0x7f4c2c158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c2a59c640 1 --2- 172.21.15.29:0/2164391244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c2c0ccbd0 0x7f4c2c143ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c2ad9d640 1 -- 172.21.15.29:0/2164391244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2c15af40 msgr2=0x7f4c2c158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c2ad9d640 1 --2- 172.21.15.29:0/2164391244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2c15af40 0x7f4c2c158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c2ad9d640 1 -- 172.21.15.29:0/2164391244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c2c0ccbd0 msgr2=0x7f4c2c143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c2ad9d640 1 --2- 172.21.15.29:0/2164391244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c2c0ccbd0 0x7f4c2c143ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c2ad9d640 1 -- 172.21.15.29:0/2164391244 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2c144220 con 0x7f4c2c15f570 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c2ad9d640 1 --2- 172.21.15.29:0/2164391244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15f570 0x7f4c2c09c020 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400d800 tx=0x7f4c2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c1affd640 1 -- 172.21.15.29:0/2164391244 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400b6c0 con 0x7f4c2c15f570 2024-04-25T00:31:15.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c1affd640 1 -- 172.21.15.29:0/2164391244 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c24019070 con 0x7f4c2c15f570 2024-04-25T00:31:15.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c30d27640 1 -- 172.21.15.29:0/2164391244 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2c1592a0 con 0x7f4c2c15f570 2024-04-25T00:31:15.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c1affd640 1 -- 172.21.15.29:0/2164391244 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400baa0 con 0x7f4c2c15f570 2024-04-25T00:31:15.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c30d27640 1 -- 172.21.15.29:0/2164391244 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c2c141c50 con 0x7f4c2c15f570 2024-04-25T00:31:15.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c1affd640 1 -- 172.21.15.29:0/2164391244 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4c2400bc40 con 0x7f4c2c15f570 2024-04-25T00:31:15.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c1affd640 1 -- 172.21.15.29:0/2164391244 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f4c2401eb30 con 0x7f4c2c15f570 2024-04-25T00:31:15.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.141+0000 7f4c30d27640 1 -- 172.21.15.29:0/2164391244 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c2c069270 con 0x7f4c2c15f570 2024-04-25T00:31:15.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.177+0000 7f4c1affd640 1 -- 172.21.15.29:0/2164391244 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4c2c144220 con 0x7f4c2c15f570 2024-04-25T00:31:15.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.249+0000 7f4c30d27640 1 -- 172.21.15.29:0/2164391244 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c2c0b5380 con 0x7f4c2c15f570 2024-04-25T00:31:15.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.249+0000 7f4c1affd640 1 -- 172.21.15.29:0/2164391244 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f4c2401e460 con 0x7f4c2c15f570 2024-04-25T00:31:15.252 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:15.252 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":10,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:14.972478+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.253+0000 7f4c18ff9640 1 -- 172.21.15.29:0/2164391244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15f570 msgr2=0x7f4c2c09c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.253+0000 7f4c18ff9640 1 --2- 172.21.15.29:0/2164391244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15f570 0x7f4c2c09c020 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400d800 tx=0x7f4c2400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.253+0000 7f4c18ff9640 1 -- 172.21.15.29:0/2164391244 shutdown_connections 2024-04-25T00:31:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.253+0000 7f4c18ff9640 1 --2- 172.21.15.29:0/2164391244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15f570 0x7f4c2c09c020 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.253+0000 7f4c18ff9640 1 --2- 172.21.15.29:0/2164391244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2c15af40 0x7f4c2c158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.253+0000 7f4c18ff9640 1 --2- 172.21.15.29:0/2164391244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c2c0ccbd0 0x7f4c2c143ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.253+0000 7f4c18ff9640 1 -- 172.21.15.29:0/2164391244 >> 172.21.15.29:0/2164391244 conn(0x7f4c2c0b5820 msgr2=0x7f4c2c0b5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.253+0000 7f4c18ff9640 1 -- 172.21.15.29:0/2164391244 shutdown_connections 2024-04-25T00:31:15.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:15.253+0000 7f4c18ff9640 1 -- 172.21.15.29:0/2164391244 wait complete. 2024-04-25T00:31:15.281 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T00:31:15.281 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T00:31:15.494 INFO:tasks.ceph.mgr.x.smithi157.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T00:31:15.494 INFO:tasks.ceph.mgr.x.smithi157.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T00:31:15.494 INFO:tasks.ceph.mgr.x.smithi157.stderr: from numpy import show_config as show_numpy_config 2024-04-25T00:31:15.575 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:15.569+0000 7f487255b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:31:15.606 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-04-25T00:31:15.601+0000 7f744da687c0 -1 Falling back to public interface 2024-04-25T00:31:15.606 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-04-25T00:31:15.601+0000 7fc043d297c0 -1 Falling back to public interface 2024-04-25T00:31:15.607 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-04-25T00:31:15.601+0000 7f744ca28640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-25T00:31:15.618 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-04-25T00:31:15.617+0000 7fc893a5e7c0 -1 Falling back to public interface 2024-04-25T00:31:15.620 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-04-25T00:31:15.617+0000 7f8bd2be77c0 -1 Falling back to public interface 2024-04-25T00:31:15.635 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-04-25T00:31:15.633+0000 7efeef26e7c0 -1 Falling back to public interface 2024-04-25T00:31:15.636 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-04-25T00:31:15.633+0000 7fe849dd87c0 -1 Falling back to public interface 2024-04-25T00:31:15.653 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-04-25T00:31:15.653+0000 7f74bb55e7c0 -1 Falling back to public interface 2024-04-25T00:31:15.654 INFO:tasks.ceph.osd.3.smithi029.stderr:2024-04-25T00:31:15.653+0000 7f552c1127c0 -1 Falling back to public interface 2024-04-25T00:31:15.658 INFO:tasks.ceph.mgr.y.smithi029.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T00:31:15.658 INFO:tasks.ceph.mgr.y.smithi029.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T00:31:15.658 INFO:tasks.ceph.mgr.y.smithi029.stderr: from numpy import show_config as show_numpy_config 2024-04-25T00:31:15.745 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:15.745+0000 7f98f227a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:31:15.756 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:15.753+0000 7f487255b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:31:15.921 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:15.917+0000 7f487255b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:31:15.923 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:15.921+0000 7f98f227a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:31:15.985 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:15.981+0000 7f487255b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:31:16.088 DEBUG:teuthology.orchestra.run.smithi029:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:16.089 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:16.089+0000 7f98f227a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:31:16.117 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:16.113+0000 7f487255b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:31:16.173 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:16.173+0000 7f98f227a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:31:16.209 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7ae1f69640 1 Processor -- start 2024-04-25T00:31:16.209 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7ae1f69640 1 -- start start 2024-04-25T00:31:16.211 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7ae1f69640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7adc06f8d0 0x7f7adc06fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.211 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7ae1f69640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7adc06ef40 0x7f7adc15da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.211 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7ae1f69640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7adc0cffd0 con 0x7f7adc06f3a0 2024-04-25T00:31:16.211 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7ae1f69640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7adc0d0e80 con 0x7f7adc06f8d0 2024-04-25T00:31:16.211 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7ae1f69640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7adc15e650 con 0x7f7adc06ef40 2024-04-25T00:31:16.211 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7adb7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7adc06f8d0 0x7f7adc06fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.211 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7adb7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7adc06f8d0 0x7f7adc06fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45394/0 (socket says 172.21.15.29:45394) 2024-04-25T00:31:16.211 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7adb7fe640 1 -- 172.21.15.29:0/3738831379 learned_addr learned my addr 172.21.15.29:0/3738831379 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:16.211 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7adaffd640 1 --2- 172.21.15.29:0/3738831379 >> v2:172.21.15.157:3300/0 conn(0x7f7adc06ef40 0x7f7adc15da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.211 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7adb7fe640 1 -- 172.21.15.29:0/3738831379 >> v2:172.21.15.157:3300/0 conn(0x7f7adc06ef40 msgr2=0x7f7adc15da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.212 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7adb7fe640 1 --2- 172.21.15.29:0/3738831379 >> v2:172.21.15.157:3300/0 conn(0x7f7adc06ef40 0x7f7adc15da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.212 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7adb7fe640 1 -- 172.21.15.29:0/3738831379 >> v1:172.21.15.157:6789/0 conn(0x7f7adc06f3a0 legacy=0x7f7adc15df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.212 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.209+0000 7f7adb7fe640 1 -- 172.21.15.29:0/3738831379 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc0d0270 con 0x7f7adc06f8d0 2024-04-25T00:31:16.213 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7adb7fe640 1 --2- 172.21.15.29:0/3738831379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7adc06f8d0 0x7f7adc06fcb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7adc06c1f0 tx=0x7f7acc01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565839cd1b15fb6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.213 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7ada7fc640 1 -- 172.21.15.29:0/3738831379 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0b2070 con 0x7f7adc06f8d0 2024-04-25T00:31:16.213 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7ada7fc640 1 -- 172.21.15.29:0/3738831379 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7acc0ad640 con 0x7f7adc06f8d0 2024-04-25T00:31:16.213 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7ada7fc640 1 -- 172.21.15.29:0/3738831379 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0ada10 con 0x7f7adc06f8d0 2024-04-25T00:31:16.214 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7ae1f69640 1 -- 172.21.15.29:0/3738831379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7adc06f8d0 msgr2=0x7f7adc06fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.214 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7ae1f69640 1 --2- 172.21.15.29:0/3738831379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7adc06f8d0 0x7f7adc06fcb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7adc06c1f0 tx=0x7f7acc01c390 comp rx=0 tx=0).stop 2024-04-25T00:31:16.214 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7ae1f69640 1 -- 172.21.15.29:0/3738831379 shutdown_connections 2024-04-25T00:31:16.214 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7ae1f69640 1 --2- 172.21.15.29:0/3738831379 >> v2:172.21.15.157:3300/0 conn(0x7f7adc06ef40 0x7f7adc15da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.214 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7ae1f69640 1 --2- 172.21.15.29:0/3738831379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7adc06f8d0 0x7f7adc06fcb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.214 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7ae1f69640 1 -- 172.21.15.29:0/3738831379 >> 172.21.15.29:0/3738831379 conn(0x7f7adc0b38c0 msgr2=0x7f7adc0b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.214 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7ae1f69640 1 -- 172.21.15.29:0/3738831379 shutdown_connections 2024-04-25T00:31:16.215 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7ae1f69640 1 -- 172.21.15.29:0/3738831379 wait complete. 2024-04-25T00:31:16.215 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7ae1f69640 1 Processor -- start 2024-04-25T00:31:16.216 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.213+0000 7f7ae1f69640 1 -- start start 2024-04-25T00:31:16.219 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ae1f69640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc06ef40 0x7f7adc14c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.219 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ae1f69640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc06f3a0 0x7f7adc14e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.219 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ae1f69640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7adc06f8d0 0x7f7adc155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.219 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ae1f69640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc0d1310 con 0x7f7adc06f8d0 2024-04-25T00:31:16.219 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ae1f69640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc0701f0 con 0x7f7adc06f3a0 2024-04-25T00:31:16.219 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ae1f69640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7adc15f6d0 con 0x7f7adc06ef40 2024-04-25T00:31:16.219 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7adc06f8d0 0x7f7adc155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.219 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7adc06f8d0 0x7f7adc155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45408/0 (socket says 172.21.15.29:45408) 2024-04-25T00:31:16.219 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7adbfff640 1 -- 172.21.15.29:0/2686469290 learned_addr learned my addr 172.21.15.29:0/2686469290 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:16.219 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7adb7fe640 1 --2- 172.21.15.29:0/2686469290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc06ef40 0x7f7adc14c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.219 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7adaffd640 1 --2- 172.21.15.29:0/2686469290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc06f3a0 0x7f7adc14e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.219 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7adbfff640 1 -- 172.21.15.29:0/2686469290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc06ef40 msgr2=0x7f7adc14c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.220 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7adbfff640 1 --2- 172.21.15.29:0/2686469290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc06ef40 0x7f7adc14c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.220 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7adbfff640 1 -- 172.21.15.29:0/2686469290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc06f3a0 msgr2=0x7f7adc14e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.220 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7adbfff640 1 --2- 172.21.15.29:0/2686469290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc06f3a0 0x7f7adc14e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.220 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7adbfff640 1 -- 172.21.15.29:0/2686469290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc15c2a0 con 0x7f7adc06f8d0 2024-04-25T00:31:16.220 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7adbfff640 1 --2- 172.21.15.29:0/2686469290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7adc06f8d0 0x7f7adc155b60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400b970 tx=0x7f7ad400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.220 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ad3fff640 1 -- 172.21.15.29:0/2686469290 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4019070 con 0x7f7adc06f8d0 2024-04-25T00:31:16.220 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ad3fff640 1 -- 172.21.15.29:0/2686469290 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad400cae0 con 0x7f7adc06f8d0 2024-04-25T00:31:16.220 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ae1f69640 1 -- 172.21.15.29:0/2686469290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7adc14ebd0 con 0x7f7adc06f8d0 2024-04-25T00:31:16.220 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ad3fff640 1 -- 172.21.15.29:0/2686469290 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4014440 con 0x7f7adc06f8d0 2024-04-25T00:31:16.220 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ae1f69640 1 -- 172.21.15.29:0/2686469290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7adc15c540 con 0x7f7adc06f8d0 2024-04-25T00:31:16.220 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ad3fff640 1 -- 172.21.15.29:0/2686469290 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ad401c070 con 0x7f7adc06f8d0 2024-04-25T00:31:16.221 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ad3fff640 1 -- 172.21.15.29:0/2686469290 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7ad401da80 con 0x7f7adc06f8d0 2024-04-25T00:31:16.221 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.217+0000 7f7ae1f69640 1 -- 172.21.15.29:0/2686469290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7adc09ff20 con 0x7f7adc06f8d0 2024-04-25T00:31:16.221 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.221+0000 7f7ad3fff640 1 -- 172.21.15.29:0/2686469290 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7ad4023070 con 0x7f7adc06f8d0 2024-04-25T00:31:16.277 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.277+0000 7f7ae1f69640 1 -- 172.21.15.29:0/2686469290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7adc04f5f0 con 0x7f7adc06f8d0 2024-04-25T00:31:16.278 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.277+0000 7f7ad3fff640 1 -- 172.21.15.29:0/2686469290 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7ad401dd50 con 0x7f7adc06f8d0 2024-04-25T00:31:16.278 INFO:teuthology.misc.health.smithi029.stdout: 2024-04-25T00:31:16.278 INFO:teuthology.misc.health.smithi029.stdout:{"epoch":10,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:14.972478+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:16.279 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.277+0000 7f7ae1f69640 1 -- 172.21.15.29:0/2686469290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7adc06f8d0 msgr2=0x7f7adc155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.279 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.277+0000 7f7ae1f69640 1 --2- 172.21.15.29:0/2686469290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7adc06f8d0 0x7f7adc155b60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400b970 tx=0x7f7ad400be30 comp rx=0 tx=0).stop 2024-04-25T00:31:16.279 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.277+0000 7f7ae1f69640 1 -- 172.21.15.29:0/2686469290 shutdown_connections 2024-04-25T00:31:16.280 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.277+0000 7f7ae1f69640 1 --2- 172.21.15.29:0/2686469290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7adc06f8d0 0x7f7adc155b60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.280 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.277+0000 7f7ae1f69640 1 --2- 172.21.15.29:0/2686469290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc06f3a0 0x7f7adc14e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.280 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.277+0000 7f7ae1f69640 1 --2- 172.21.15.29:0/2686469290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7adc06ef40 0x7f7adc14c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.280 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.277+0000 7f7ae1f69640 1 -- 172.21.15.29:0/2686469290 >> 172.21.15.29:0/2686469290 conn(0x7f7adc0b38c0 msgr2=0x7f7adc156e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.280 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.277+0000 7f7ae1f69640 1 -- 172.21.15.29:0/2686469290 shutdown_connections 2024-04-25T00:31:16.280 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:16.277+0000 7f7ae1f69640 1 -- 172.21.15.29:0/2686469290 wait complete. 2024-04-25T00:31:16.293 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T00:31:16.326 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:16.325+0000 7f98f227a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:31:16.732 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:16.729+0000 7f487255b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:31:16.807 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:16.801+0000 7f487255b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:31:16.960 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:16.957+0000 7f98f227a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:31:16.988 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:16.985+0000 7f487255b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:31:17.039 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:17.037+0000 7f98f227a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:31:17.061 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:17.057+0000 7f487255b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:31:17.228 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:17.225+0000 7f98f227a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:31:17.305 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:17.305+0000 7f98f227a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:31:17.350 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:17.345+0000 7f487255b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:31:17.411 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:17.405+0000 7f487255b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:31:17.612 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:17.609+0000 7f98f227a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:31:17.676 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:31:17.673+0000 7f98f227a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:31:18.447 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:18.441+0000 7f48198a1640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T00:31:18.447 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:31:18.441+0000 7f48198a1640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T00:31:19.408 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-04-25T00:31:19.405+0000 7f744da687c0 -1 osd.4 0 log_to_monitors true 2024-04-25T00:31:19.503 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-04-25T00:31:19.501+0000 7fc893a5e7c0 -1 osd.1 0 log_to_monitors true 2024-04-25T00:31:19.509 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-04-25T00:31:19.509+0000 7efeef26e7c0 -1 osd.0 0 log_to_monitors true 2024-04-25T00:31:19.520 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-04-25T00:31:19.521+0000 7f74bb55e7c0 -1 osd.2 0 log_to_monitors true 2024-04-25T00:31:19.526 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-04-25T00:31:19.521+0000 7fe849dd87c0 -1 osd.7 0 log_to_monitors true 2024-04-25T00:31:19.527 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-04-25T00:31:19.525+0000 7f8bd2be77c0 -1 osd.6 0 log_to_monitors true 2024-04-25T00:31:19.540 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-04-25T00:31:19.537+0000 7fc043d297c0 -1 osd.5 0 log_to_monitors true 2024-04-25T00:31:19.606 INFO:tasks.ceph.osd.3.smithi029.stderr:2024-04-25T00:31:19.605+0000 7f552c1127c0 -1 osd.3 0 log_to_monitors true 2024-04-25T00:31:22.177 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-04-25T00:31:22.177+0000 7fe84555e640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T00:31:22.204 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-04-25T00:31:22.205+0000 7fe83dba5640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:31:22.208 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-04-25T00:31:22.209+0000 7fc03f4af640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T00:31:22.225 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-04-25T00:31:22.225+0000 7f744aa24640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T00:31:22.244 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-04-25T00:31:22.245+0000 7fc037af6640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:31:22.248 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-04-25T00:31:22.249+0000 7f7441835640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:31:22.257 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-04-25T00:31:22.256+0000 7fc88f1e4640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T00:31:22.259 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-04-25T00:31:22.261+0000 7f8bcfba3640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T00:31:22.284 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-04-25T00:31:22.285+0000 7f8bc69b4640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:31:22.285 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-04-25T00:31:22.284+0000 7fc88782b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:31:22.298 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-04-25T00:31:22.296+0000 7f74b6ce4640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T00:31:22.325 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-04-25T00:31:22.324+0000 7f74af32b640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:31:22.347 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-04-25T00:31:22.344+0000 7efeea9f4640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T00:31:22.369 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-04-25T00:31:22.368+0000 7efee303b640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:31:22.382 INFO:tasks.ceph.osd.3.smithi029.stderr:2024-04-25T00:31:22.380+0000 7f5527898640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T00:31:22.418 INFO:tasks.ceph.osd.3.smithi029.stderr:2024-04-25T00:31:22.416+0000 7f551fedf640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:31:23.100 DEBUG:teuthology.orchestra.run.smithi029:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:23.228 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.224+0000 7f3aabfff640 1 Processor -- start 2024-04-25T00:31:23.229 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.224+0000 7f3aabfff640 1 -- start start 2024-04-25T00:31:23.230 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aabfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3aac0ae820 0x7f3aac0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.230 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aabfff640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aac15a730 con 0x7f3aac157c20 2024-04-25T00:31:23.230 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aabfff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aac072350 con 0x7f3aac06cd10 2024-04-25T00:31:23.230 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aabfff640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f3aac0b28c0 con 0x7f3aac0ae820 2024-04-25T00:31:23.230 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aaaffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3aac157c20 0x7f3aac0717e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52366/0 (socket says 172.21.15.29:52366) 2024-04-25T00:31:23.230 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aaaffd640 1 -- 172.21.15.29:0/1522831729 learned_addr learned my addr 172.21.15.29:0/1522831729 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:23.230 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aaa7fc640 1 --2- 172.21.15.29:0/1522831729 >> v2:172.21.15.29:3300/0 conn(0x7f3aac0ae820 0x7f3aac0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.230 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aa9ffb640 1 -- 172.21.15.29:0/1522831729 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2086944060 0 0) 0x7f3aac15a730 con 0x7f3aac157c20 2024-04-25T00:31:23.230 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aa9ffb640 1 -- 172.21.15.29:0/1522831729 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a94003450 con 0x7f3aac157c20 2024-04-25T00:31:23.231 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aa9ffb640 1 -- 172.21.15.29:0/1522831729 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1837853923 0 0) 0x7f3a94003450 con 0x7f3aac157c20 2024-04-25T00:31:23.231 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aaa7fc640 1 -- 172.21.15.29:0/1522831729 >> v1:172.21.15.157:6789/0 conn(0x7f3aac06cd10 legacy=0x7f3aac06d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.231 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aaa7fc640 1 -- 172.21.15.29:0/1522831729 >> v1:172.21.15.29:6789/0 conn(0x7f3aac157c20 legacy=0x7f3aac0717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.231 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aaa7fc640 1 -- 172.21.15.29:0/1522831729 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aac072630 con 0x7f3aac0ae820 2024-04-25T00:31:23.232 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.228+0000 7f3aaa7fc640 1 --2- 172.21.15.29:0/1522831729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3aac0ae820 0x7f3aac0aecd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c007140 tx=0x7f3a9c01c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bee0986a1e52f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:23.233 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.232+0000 7f3aa9ffb640 1 -- 172.21.15.29:0/1522831729 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0b0070 con 0x7f3aac0ae820 2024-04-25T00:31:23.233 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.232+0000 7f3aa9ffb640 1 -- 172.21.15.29:0/1522831729 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a9c0078d0 con 0x7f3aac0ae820 2024-04-25T00:31:23.233 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.232+0000 7f3aa9ffb640 1 -- 172.21.15.29:0/1522831729 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0001680 con 0x7f3aac0ae820 2024-04-25T00:31:23.234 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.232+0000 7f3aabfff640 1 -- 172.21.15.29:0/1522831729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3aac0ae820 msgr2=0x7f3aac0aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.234 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.232+0000 7f3aabfff640 1 --2- 172.21.15.29:0/1522831729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3aac0ae820 0x7f3aac0aecd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c007140 tx=0x7f3a9c01c2d0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.234 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.232+0000 7f3aabfff640 1 -- 172.21.15.29:0/1522831729 shutdown_connections 2024-04-25T00:31:23.234 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.232+0000 7f3aabfff640 1 --2- 172.21.15.29:0/1522831729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3aac0ae820 0x7f3aac0aecd0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.234 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.232+0000 7f3aabfff640 1 -- 172.21.15.29:0/1522831729 >> 172.21.15.29:0/1522831729 conn(0x7f3aac0b15c0 msgr2=0x7f3aac0b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:23.234 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.232+0000 7f3aabfff640 1 -- 172.21.15.29:0/1522831729 shutdown_connections 2024-04-25T00:31:23.234 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.232+0000 7f3aabfff640 1 -- 172.21.15.29:0/1522831729 wait complete. 2024-04-25T00:31:23.236 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.232+0000 7f3aabfff640 1 Processor -- start 2024-04-25T00:31:23.236 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.232+0000 7f3aabfff640 1 -- start start 2024-04-25T00:31:23.238 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aabfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3aac06cd10 0x7f3aac14c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.238 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aabfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aac0ae820 0x7f3aac0ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.238 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aabfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aac157c20 0x7f3aac0aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.239 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aabfff640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac167b70 con 0x7f3aac06cd10 2024-04-25T00:31:23.239 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aabfff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac072100 con 0x7f3aac0ae820 2024-04-25T00:31:23.239 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aabfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3aac167cf0 con 0x7f3aac157c20 2024-04-25T00:31:23.239 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aaaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3aac06cd10 0x7f3aac14c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.239 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aaaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3aac06cd10 0x7f3aac14c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52792/0 (socket says 172.21.15.29:52792) 2024-04-25T00:31:23.239 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aaaffd640 1 -- 172.21.15.29:0/668766307 learned_addr learned my addr 172.21.15.29:0/668766307 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:23.239 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aaa7fc640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aac0ae820 0x7f3aac0ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.239 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aab7fe640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aac157c20 0x7f3aac0aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.239 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aaaffd640 1 -- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aac157c20 msgr2=0x7f3aac0aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.239 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aaaffd640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aac157c20 0x7f3aac0aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.239 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aaaffd640 1 -- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aac0ae820 msgr2=0x7f3aac0ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.239 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aaaffd640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aac0ae820 0x7f3aac0ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.239 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aaaffd640 1 -- 172.21.15.29:0/668766307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aac14f030 con 0x7f3aac06cd10 2024-04-25T00:31:23.240 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aaaffd640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3aac06cd10 0x7f3aac14c4b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000c520 tx=0x7f3aa000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:23.240 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3a93fff640 1 -- 172.21.15.29:0/668766307 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0017070 con 0x7f3aac06cd10 2024-04-25T00:31:23.240 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3a93fff640 1 -- 172.21.15.29:0/668766307 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aa0005d10 con 0x7f3aac06cd10 2024-04-25T00:31:23.240 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aabfff640 1 -- 172.21.15.29:0/668766307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aac14df90 con 0x7f3aac06cd10 2024-04-25T00:31:23.240 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3a93fff640 1 -- 172.21.15.29:0/668766307 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa00072b0 con 0x7f3aac06cd10 2024-04-25T00:31:23.240 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aabfff640 1 -- 172.21.15.29:0/668766307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aac14e550 con 0x7f3aac06cd10 2024-04-25T00:31:23.241 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aabfff640 1 -- 172.21.15.29:0/668766307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aac04f580 con 0x7f3aac06cd10 2024-04-25T00:31:23.241 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3a93fff640 1 -- 172.21.15.29:0/668766307 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f3aa001a070 con 0x7f3aac06cd10 2024-04-25T00:31:23.241 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3a93fff640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a98067830 0x7f3a98069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.241 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3a93fff640 1 -- 172.21.15.29:0/668766307 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f3aa0084bb0 con 0x7f3aac06cd10 2024-04-25T00:31:23.241 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.236+0000 7f3aaa7fc640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a98067830 0x7f3a98069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.241 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.240+0000 7f3aaa7fc640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a98067830 0x7f3a98069cf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0ad000 tx=0x7f3a9c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:23.244 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.244+0000 7f3a93fff640 1 -- 172.21.15.29:0/668766307 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3aa0055ca0 con 0x7f3aac06cd10 2024-04-25T00:31:23.399 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.396+0000 7f3aabfff640 1 -- 172.21.15.29:0/668766307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3aac14f2d0 con 0x7f3aac06cd10 2024-04-25T00:31:23.400 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.396+0000 7f3a93fff640 1 -- 172.21.15.29:0/668766307 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10861 (secure 0 0 0) 0x7f3aa00555a0 con 0x7f3aac06cd10 2024-04-25T00:31:23.401 INFO:teuthology.misc.health.smithi029.stdout: 2024-04-25T00:31:23.401 INFO:teuthology.misc.health.smithi029.stdout:{"epoch":13,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:23.211682+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:23.403 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.400+0000 7f3aabfff640 1 -- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a98067830 msgr2=0x7f3a98069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.403 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.400+0000 7f3aabfff640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a98067830 0x7f3a98069cf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0ad000 tx=0x7f3a9c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:31:23.403 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.400+0000 7f3aabfff640 1 -- 172.21.15.29:0/668766307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3aac06cd10 msgr2=0x7f3aac14c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.403 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.400+0000 7f3aabfff640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3aac06cd10 0x7f3aac14c4b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000c520 tx=0x7f3aa000c9e0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.403 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.400+0000 7f3aabfff640 1 -- 172.21.15.29:0/668766307 shutdown_connections 2024-04-25T00:31:23.403 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.400+0000 7f3aabfff640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3aac157c20 0x7f3aac0aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.403 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.400+0000 7f3aabfff640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a98067830 0x7f3a98069cf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.403 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.400+0000 7f3aabfff640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aac0ae820 0x7f3aac0ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.403 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.400+0000 7f3aabfff640 1 --2- 172.21.15.29:0/668766307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3aac06cd10 0x7f3aac14c4b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.403 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.400+0000 7f3aabfff640 1 -- 172.21.15.29:0/668766307 >> 172.21.15.29:0/668766307 conn(0x7f3aac0b15c0 msgr2=0x7f3aac14fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:23.403 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.400+0000 7f3aabfff640 1 -- 172.21.15.29:0/668766307 shutdown_connections 2024-04-25T00:31:23.403 INFO:teuthology.misc.health.smithi029.stderr:2024-04-25T00:31:23.400+0000 7f3aabfff640 1 -- 172.21.15.29:0/668766307 wait complete. 2024-04-25T00:31:23.417 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T00:31:23.417 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T00:31:23.417 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:31:23.417 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:31:23.419 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:31:23.419 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:31:23.419 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:31:23.421 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:31:23.421 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:31:23.421 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:31:23.424 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:31:23.424 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:31:23.424 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:31:23.427 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:31:23.427 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T00:31:23.428 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:23.495 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:31:23.497 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:31:23.515 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-25T00:31:23.518 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-25T00:31:23.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f3372435640 1 Processor -- start 2024-04-25T00:31:23.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f3372435640 1 -- start start 2024-04-25T00:31:23.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f3372435640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f336c15c190 0x7f336c0d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f3372435640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f336c0ccf30 0x7f336c0d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f3372435640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f336c0cd3e0 con 0x7f336c15bdb0 2024-04-25T00:31:23.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f3372435640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f336c0cd6c0 con 0x7f336c15c190 2024-04-25T00:31:23.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f3372435640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f336c0cd9a0 con 0x7f336c0ccf30 2024-04-25T00:31:23.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f3363fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f336c15c190 0x7f336c0d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f336bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f336c15bdb0 0x7f336c0cc820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:50760/0 (socket says 172.21.15.29:50760) 2024-04-25T00:31:23.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f33709d2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f336c0ccf30 0x7f336c0d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f336bfff640 1 -- 172.21.15.29:0/3302815480 learned_addr learned my addr 172.21.15.29:0/3302815480 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:23.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f3363fff640 1 -- 172.21.15.29:0/3302815480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f336c0ccf30 msgr2=0x7f336c0d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:23.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f3363fff640 1 --2- 172.21.15.29:0/3302815480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f336c0ccf30 0x7f336c0d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f3363fff640 1 -- 172.21.15.29:0/3302815480 >> v1:172.21.15.157:6789/0 conn(0x7f336c15bdb0 legacy=0x7f336c0cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.548+0000 7f3363fff640 1 -- 172.21.15.29:0/3302815480 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c0cdc80 con 0x7f336c15c190 2024-04-25T00:31:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f3363fff640 1 --2- 172.21.15.29:0/3302815480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15c190 0x7f336c0d02d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33540097c0 tx=0x7f335401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cff831e01f59640 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f336b7fe640 1 -- 172.21.15.29:0/3302815480 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33540b1070 con 0x7f336c15c190 2024-04-25T00:31:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f336b7fe640 1 -- 172.21.15.29:0/3302815480 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33540ac600 con 0x7f336c15c190 2024-04-25T00:31:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f336b7fe640 1 -- 172.21.15.29:0/3302815480 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33540ac9d0 con 0x7f336c15c190 2024-04-25T00:31:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f3372435640 1 -- 172.21.15.29:0/3302815480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15c190 msgr2=0x7f336c0d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f3372435640 1 --2- 172.21.15.29:0/3302815480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15c190 0x7f336c0d02d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33540097c0 tx=0x7f335401c3e0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f3372435640 1 -- 172.21.15.29:0/3302815480 shutdown_connections 2024-04-25T00:31:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f3372435640 1 --2- 172.21.15.29:0/3302815480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f336c0ccf30 0x7f336c0d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f3372435640 1 --2- 172.21.15.29:0/3302815480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15c190 0x7f336c0d02d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f3372435640 1 -- 172.21.15.29:0/3302815480 >> 172.21.15.29:0/3302815480 conn(0x7f336c09aa30 msgr2=0x7f336c09ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:23.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f3372435640 1 -- 172.21.15.29:0/3302815480 shutdown_connections 2024-04-25T00:31:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f3372435640 1 -- 172.21.15.29:0/3302815480 wait complete. 2024-04-25T00:31:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f3372435640 1 Processor -- start 2024-04-25T00:31:23.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.552+0000 7f3372435640 1 -- start start 2024-04-25T00:31:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f3372435640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336c0ccf30 0x7f336c15c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f3372435640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15bdb0 0x7f336c0cead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f3372435640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f336c14b860 0x7f336c14bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f3372435640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f336c0c6400 con 0x7f336c0ccf30 2024-04-25T00:31:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f3372435640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f336c0c6280 con 0x7f336c15bdb0 2024-04-25T00:31:23.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f3372435640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f336c0c6580 con 0x7f336c14b860 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f336bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336c0ccf30 0x7f336c15c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f336bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336c0ccf30 0x7f336c15c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52838/0 (socket says 172.21.15.29:52838) 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f336bfff640 1 -- 172.21.15.29:0/4066737102 learned_addr learned my addr 172.21.15.29:0/4066737102 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f3363fff640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15bdb0 0x7f336c0cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f33709d2640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f336c14b860 0x7f336c14bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f336bfff640 1 -- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f336c14b860 msgr2=0x7f336c14bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f336bfff640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f336c14b860 0x7f336c14bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f336bfff640 1 -- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15bdb0 msgr2=0x7f336c0cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f336bfff640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15bdb0 0x7f336c0cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f336bfff640 1 -- 172.21.15.29:0/4066737102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c14c230 con 0x7f336c0ccf30 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f336bfff640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336c0ccf30 0x7f336c15c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f335c00c420 tx=0x7f335c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f33697fa640 1 -- 172.21.15.29:0/4066737102 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c017070 con 0x7f336c0ccf30 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f33697fa640 1 -- 172.21.15.29:0/4066737102 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f335c005cf0 con 0x7f336c0ccf30 2024-04-25T00:31:23.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f3372435640 1 -- 172.21.15.29:0/4066737102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c15cda0 con 0x7f336c0ccf30 2024-04-25T00:31:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f33697fa640 1 -- 172.21.15.29:0/4066737102 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0072f0 con 0x7f336c0ccf30 2024-04-25T00:31:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f3372435640 1 -- 172.21.15.29:0/4066737102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c0c9260 con 0x7f336c0ccf30 2024-04-25T00:31:23.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f3372435640 1 -- 172.21.15.29:0/4066737102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f336c069270 con 0x7f336c0ccf30 2024-04-25T00:31:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f33697fa640 1 -- 172.21.15.29:0/4066737102 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f335c01a070 con 0x7f336c0ccf30 2024-04-25T00:31:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f33697fa640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f334c067830 0x7f334c069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f33697fa640 1 -- 172.21.15.29:0/4066737102 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f335c084940 con 0x7f336c0ccf30 2024-04-25T00:31:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f3363fff640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f334c067830 0x7f334c069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.556+0000 7f3363fff640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f334c067830 0x7f334c069cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f33540ae000 tx=0x7f33540a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:23.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.560+0000 7f33697fa640 1 -- 172.21.15.29:0/4066737102 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f335c055470 con 0x7f336c0ccf30 2024-04-25T00:31:23.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.720+0000 7f3372435640 1 -- 172.21.15.29:0/4066737102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f336c0c5470 con 0x7f336c0ccf30 2024-04-25T00:31:23.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.724+0000 7f33697fa640 1 -- 172.21.15.29:0/4066737102 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10861 (secure 0 0 0) 0x7f335c059770 con 0x7f336c0ccf30 2024-04-25T00:31:23.728 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:23.728 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":13,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:23.211682+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:23.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.728+0000 7f3372435640 1 -- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f334c067830 msgr2=0x7f334c069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.728+0000 7f3372435640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f334c067830 0x7f334c069cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f33540ae000 tx=0x7f33540a7000 comp rx=0 tx=0).stop 2024-04-25T00:31:23.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.728+0000 7f3372435640 1 -- 172.21.15.29:0/4066737102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336c0ccf30 msgr2=0x7f336c15c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.728+0000 7f3372435640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336c0ccf30 0x7f336c15c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f335c00c420 tx=0x7f335c00c8e0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.728+0000 7f3372435640 1 -- 172.21.15.29:0/4066737102 shutdown_connections 2024-04-25T00:31:23.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.728+0000 7f3372435640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f336c14b860 0x7f336c14bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.728+0000 7f3372435640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f334c067830 0x7f334c069cf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.728+0000 7f3372435640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15bdb0 0x7f336c0cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.728+0000 7f3372435640 1 --2- 172.21.15.29:0/4066737102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336c0ccf30 0x7f336c15c630 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.728+0000 7f3372435640 1 -- 172.21.15.29:0/4066737102 >> 172.21.15.29:0/4066737102 conn(0x7f336c09aa30 msgr2=0x7f336c06efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:23.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.728+0000 7f3372435640 1 -- 172.21.15.29:0/4066737102 shutdown_connections 2024-04-25T00:31:23.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.728+0000 7f3372435640 1 -- 172.21.15.29:0/4066737102 wait complete. 2024-04-25T00:31:23.747 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-25T00:31:23.747 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:31:23.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.852+0000 7f155279d640 1 Processor -- start 2024-04-25T00:31:23.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.852+0000 7f155279d640 1 -- start start 2024-04-25T00:31:23.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f155279d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f154c15e3c0 0x7f154c0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f155279d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f154c06c530 0x7f154c06cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f155279d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f154c15e7a0 con 0x7f154c06d150 2024-04-25T00:31:23.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f155279d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f154c06d5c0 con 0x7f154c06c530 2024-04-25T00:31:23.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f155279d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f154c0b4bc0 con 0x7f154c15e3c0 2024-04-25T00:31:23.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f1550d3a640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f154c06d150 0x7f154c0ce5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52376/0 (socket says 172.21.15.29:52376) 2024-04-25T00:31:23.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f1550d3a640 1 -- 172.21.15.29:0/3486408103 learned_addr learned my addr 172.21.15.29:0/3486408103 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:23.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3134348563 0 0) 0x7f154c15e7a0 con 0x7f154c06d150 2024-04-25T00:31:23.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f1543fff640 1 --2- 172.21.15.29:0/3486408103 >> v2:172.21.15.157:3300/0 conn(0x7f154c06c530 0x7f154c06cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154bfff640 1 --2- 172.21.15.29:0/3486408103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f154c15e3c0 0x7f154c0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f152c003530 con 0x7f154c06d150 2024-04-25T00:31:23.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 168972394 0 0) 0x7f152c003530 con 0x7f154c06d150 2024-04-25T00:31:23.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f154c15e3c0 msgr2=0x7f154c0d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:23.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 --2- 172.21.15.29:0/3486408103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f154c15e3c0 0x7f154c0d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 >> v2:172.21.15.157:3300/0 conn(0x7f154c06c530 msgr2=0x7f154c06cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:23.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 --2- 172.21.15.29:0/3486408103 >> v2:172.21.15.157:3300/0 conn(0x7f154c06c530 0x7f154c06cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154c0d10f0 con 0x7f154c06d150 2024-04-25T00:31:23.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f1544001a00 con 0x7f154c06d150 2024-04-25T00:31:23.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 >> v1:172.21.15.29:6789/0 conn(0x7f154c06d150 legacy=0x7f154c0ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 --2- 172.21.15.29:0/3486408103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f152c0039c0 0x7f152c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 --2- 172.21.15.29:0/3486408103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c004810 0x7f152c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 --2- 172.21.15.29:0/3486408103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f154c15e3c0 0x7f152c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f154c06d5c0 con 0x7f152c0039c0 2024-04-25T00:31:23.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f152c00ba80 con 0x7f152c004810 2024-04-25T00:31:23.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f152c00bc50 con 0x7f154c15e3c0 2024-04-25T00:31:23.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154bfff640 1 --2- 172.21.15.29:0/3486408103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f152c0039c0 0x7f152c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f1543fff640 1 --2- 172.21.15.29:0/3486408103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c004810 0x7f152c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f1550d3a640 1 --2- 172.21.15.29:0/3486408103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f154c15e3c0 0x7f152c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154bfff640 1 -- 172.21.15.29:0/3486408103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f154c15e3c0 msgr2=0x7f152c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:23.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154bfff640 1 --2- 172.21.15.29:0/3486408103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f154c15e3c0 0x7f152c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154bfff640 1 -- 172.21.15.29:0/3486408103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c004810 msgr2=0x7f152c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:23.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154bfff640 1 --2- 172.21.15.29:0/3486408103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c004810 0x7f152c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.856+0000 7f154bfff640 1 -- 172.21.15.29:0/3486408103 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152c003530 con 0x7f152c0039c0 2024-04-25T00:31:23.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.860+0000 7f154bfff640 1 --2- 172.21.15.29:0/3486408103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f152c0039c0 0x7f152c003de0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f153c00ba00 tx=0x7f153c01c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178593f627544289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:23.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.860+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c00b600 con 0x7f152c0039c0 2024-04-25T00:31:23.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.860+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153c0af620 con 0x7f152c0039c0 2024-04-25T00:31:23.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.860+0000 7f154b7fe640 1 -- 172.21.15.29:0/3486408103 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c0afa10 con 0x7f152c0039c0 2024-04-25T00:31:23.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.860+0000 7f155279d640 1 -- 172.21.15.29:0/3486408103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f152c0039c0 msgr2=0x7f152c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.860+0000 7f155279d640 1 --2- 172.21.15.29:0/3486408103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f152c0039c0 0x7f152c003de0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f153c00ba00 tx=0x7f153c01c640 comp rx=0 tx=0).stop 2024-04-25T00:31:23.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.860+0000 7f1550d3a640 1 -- 172.21.15.29:0/3486408103 reap_dead start 2024-04-25T00:31:23.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.860+0000 7f155279d640 1 -- 172.21.15.29:0/3486408103 shutdown_connections 2024-04-25T00:31:23.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.860+0000 7f155279d640 1 -- 172.21.15.29:0/3486408103 >> 172.21.15.29:0/3486408103 conn(0x7f154c0d1d70 msgr2=0x7f154c0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:23.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.860+0000 7f155279d640 1 -- 172.21.15.29:0/3486408103 shutdown_connections 2024-04-25T00:31:23.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.860+0000 7f155279d640 1 -- 172.21.15.29:0/3486408103 wait complete. 2024-04-25T00:31:23.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.860+0000 7f155279d640 1 Processor -- start 2024-04-25T00:31:23.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.860+0000 7f155279d640 1 -- start start 2024-04-25T00:31:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f155279d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c0039c0 0x7f154c0b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f155279d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f154c0c5d40 0x7f154c0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f155279d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f154c151180 0x7f154c0d2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f155279d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f154c0ce050 con 0x7f154c0c5d40 2024-04-25T00:31:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f155279d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f154c06c910 con 0x7f152c0039c0 2024-04-25T00:31:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f155279d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f154c0ce1d0 con 0x7f154c151180 2024-04-25T00:31:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f1543fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f154c0c5d40 0x7f154c0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f1543fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f154c0c5d40 0x7f154c0b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52848/0 (socket says 172.21.15.29:52848) 2024-04-25T00:31:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f1543fff640 1 -- 172.21.15.29:0/2997119611 learned_addr learned my addr 172.21.15.29:0/2997119611 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f154bfff640 1 --2- 172.21.15.29:0/2997119611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c0039c0 0x7f154c0b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f1543fff640 1 -- 172.21.15.29:0/2997119611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f154c151180 msgr2=0x7f154c0d2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f1543fff640 1 --2- 172.21.15.29:0/2997119611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f154c151180 0x7f154c0d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f1543fff640 1 -- 172.21.15.29:0/2997119611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c0039c0 msgr2=0x7f154c0b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f1543fff640 1 --2- 172.21.15.29:0/2997119611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c0039c0 0x7f154c0b6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f1543fff640 1 -- 172.21.15.29:0/2997119611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154c0c8e40 con 0x7f154c0c5d40 2024-04-25T00:31:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f1543fff640 1 --2- 172.21.15.29:0/2997119611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f154c0c5d40 0x7f154c0b55d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f153400b7f0 tx=0x7f153400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f15497fa640 1 -- 172.21.15.29:0/2997119611 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534018070 con 0x7f154c0c5d40 2024-04-25T00:31:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f15497fa640 1 -- 172.21.15.29:0/2997119611 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1534010a40 con 0x7f154c0c5d40 2024-04-25T00:31:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f155279d640 1 -- 172.21.15.29:0/2997119611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154c0c9120 con 0x7f154c0c5d40 2024-04-25T00:31:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f15497fa640 1 -- 172.21.15.29:0/2997119611 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534010e10 con 0x7f154c0c5d40 2024-04-25T00:31:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f155279d640 1 -- 172.21.15.29:0/2997119611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f154c0c96e0 con 0x7f154c0c5d40 2024-04-25T00:31:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f155279d640 1 -- 172.21.15.29:0/2997119611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f154c0c9bc0 con 0x7f154c0c5d40 2024-04-25T00:31:23.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f15497fa640 1 -- 172.21.15.29:0/2997119611 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f153401b070 con 0x7f154c0c5d40 2024-04-25T00:31:23.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f15497fa640 1 --2- 172.21.15.29:0/2997119611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1530067830 0x7f1530069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:23.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.864+0000 7f15497fa640 1 -- 172.21.15.29:0/2997119611 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f1534085f90 con 0x7f154c0c5d40 2024-04-25T00:31:23.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.868+0000 7f154bfff640 1 --2- 172.21.15.29:0/2997119611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1530067830 0x7f1530069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:23.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.868+0000 7f154bfff640 1 --2- 172.21.15.29:0/2997119611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1530067830 0x7f1530069cf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f153c0b0000 tx=0x7f153c0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:23.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:23.868+0000 7f15497fa640 1 -- 172.21.15.29:0/2997119611 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f154c0c9bc0 con 0x7f154c0c5d40 2024-04-25T00:31:24.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.060+0000 7f155279d640 1 -- 172.21.15.29:0/2997119611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f154c069270 con 0x7f154c0c5d40 2024-04-25T00:31:24.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.060+0000 7f15497fa640 1 -- 172.21.15.29:0/2997119611 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7f1534057060 con 0x7f154c0c5d40 2024-04-25T00:31:24.065 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:24.065 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1617208213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1617208213},{"type":"v1","addr":"172.21.15.157:6837","nonce":1617208213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:31:24.065 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 2 2024-04-25T00:31:24.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.064+0000 7f155279d640 1 -- 172.21.15.29:0/2997119611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1530067830 msgr2=0x7f1530069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.064+0000 7f155279d640 1 --2- 172.21.15.29:0/2997119611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1530067830 0x7f1530069cf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f153c0b0000 tx=0x7f153c0046d0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.064+0000 7f155279d640 1 -- 172.21.15.29:0/2997119611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f154c0c5d40 msgr2=0x7f154c0b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.064+0000 7f155279d640 1 --2- 172.21.15.29:0/2997119611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f154c0c5d40 0x7f154c0b55d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f153400b7f0 tx=0x7f153400bcb0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.064+0000 7f155279d640 1 -- 172.21.15.29:0/2997119611 shutdown_connections 2024-04-25T00:31:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.064+0000 7f155279d640 1 --2- 172.21.15.29:0/2997119611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f154c151180 0x7f154c0d2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.064+0000 7f155279d640 1 --2- 172.21.15.29:0/2997119611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f154c0c5d40 0x7f154c0b55d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.064+0000 7f155279d640 1 --2- 172.21.15.29:0/2997119611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1530067830 0x7f1530069cf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.064+0000 7f155279d640 1 --2- 172.21.15.29:0/2997119611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c0039c0 0x7f154c0b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.064+0000 7f155279d640 1 -- 172.21.15.29:0/2997119611 >> 172.21.15.29:0/2997119611 conn(0x7f154c0d1d70 msgr2=0x7f154c0b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.064+0000 7f155279d640 1 -- 172.21.15.29:0/2997119611 shutdown_connections 2024-04-25T00:31:24.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:24.064+0000 7f155279d640 1 -- 172.21.15.29:0/2997119611 wait complete. 2024-04-25T00:31:26.082 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:31:26.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c6dc2b640 1 Processor -- start 2024-04-25T00:31:26.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c6dc2b640 1 -- start start 2024-04-25T00:31:26.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c6dc2b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8c680cfa10 0x7f8c680cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c6dc2b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8c680d04a0 0x7f8c680b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c6dc2b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c680cdbd0 con 0x7f8c6815c220 2024-04-25T00:31:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c6dc2b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8c680d0910 con 0x7f8c680d04a0 2024-04-25T00:31:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c6dc2b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8c6815c600 con 0x7f8c680cfa10 2024-04-25T00:31:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c67fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8c680d04a0 0x7f8c680b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c67fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8c680d04a0 0x7f8c680b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52860/0 (socket says 172.21.15.29:52860) 2024-04-25T00:31:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c67fff640 1 -- 172.21.15.29:0/152678825 learned_addr learned my addr 172.21.15.29:0/152678825 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c66ffd640 1 --2- 172.21.15.29:0/152678825 >> v2:172.21.15.157:3300/0 conn(0x7f8c680cfa10 0x7f8c680cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c67fff640 1 -- 172.21.15.29:0/152678825 >> v2:172.21.15.157:3300/0 conn(0x7f8c680cfa10 msgr2=0x7f8c680cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c67fff640 1 --2- 172.21.15.29:0/152678825 >> v2:172.21.15.157:3300/0 conn(0x7f8c680cfa10 0x7f8c680cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c67fff640 1 -- 172.21.15.29:0/152678825 >> v1:172.21.15.29:6789/0 conn(0x7f8c6815c220 legacy=0x7f8c680cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c67fff640 1 -- 172.21.15.29:0/152678825 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c680b13e0 con 0x7f8c680d04a0 2024-04-25T00:31:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c667fc640 1 -- 172.21.15.29:0/152678825 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1080274999 0 0) 0x7f8c680cdbd0 con 0x7f8c6815c220 2024-04-25T00:31:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.208+0000 7f8c67fff640 1 --2- 172.21.15.29:0/152678825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c680d04a0 0x7f8c680b1ec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8c6806bcb0 tx=0x7f8c6001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff23f65327aba0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c667fc640 1 -- 172.21.15.29:0/152678825 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c600a56c0 con 0x7f8c680d04a0 2024-04-25T00:31:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c667fc640 1 -- 172.21.15.29:0/152678825 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c600a5860 con 0x7f8c680d04a0 2024-04-25T00:31:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c667fc640 1 -- 172.21.15.29:0/152678825 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c600a5c30 con 0x7f8c680d04a0 2024-04-25T00:31:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/152678825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c680d04a0 msgr2=0x7f8c680b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 --2- 172.21.15.29:0/152678825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c680d04a0 0x7f8c680b1ec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8c6806bcb0 tx=0x7f8c6001c9f0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/152678825 shutdown_connections 2024-04-25T00:31:26.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 --2- 172.21.15.29:0/152678825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c680d04a0 0x7f8c680b1ec0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 --2- 172.21.15.29:0/152678825 >> v2:172.21.15.157:3300/0 conn(0x7f8c680cfa10 0x7f8c680cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/152678825 >> 172.21.15.29:0/152678825 conn(0x7f8c680d1d60 msgr2=0x7f8c680d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:26.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/152678825 shutdown_connections 2024-04-25T00:31:26.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/152678825 wait complete. 2024-04-25T00:31:26.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 Processor -- start 2024-04-25T00:31:26.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 -- start start 2024-04-25T00:31:26.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c680cfa10 0x7f8c681545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c680d04a0 0x7f8c68164b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c6815c220 0x7f8c6806d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8c680b2fa0 con 0x7f8c680cfa10 2024-04-25T00:31:26.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c680b2e20 con 0x7f8c680d04a0 2024-04-25T00:31:26.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c6dc2b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8c680b3120 con 0x7f8c6815c220 2024-04-25T00:31:26.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c680cfa10 0x7f8c681545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c680cfa10 0x7f8c681545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52862/0 (socket says 172.21.15.29:52862) 2024-04-25T00:31:26.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c677fe640 1 -- 172.21.15.29:0/4200555755 learned_addr learned my addr 172.21.15.29:0/4200555755 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:26.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c66ffd640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c680d04a0 0x7f8c68164b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c67fff640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c6815c220 0x7f8c6806d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c677fe640 1 -- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c6815c220 msgr2=0x7f8c6806d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c677fe640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c6815c220 0x7f8c6806d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c677fe640 1 -- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c680d04a0 msgr2=0x7f8c68164b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c677fe640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c680d04a0 0x7f8c68164b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c677fe640 1 -- 172.21.15.29:0/4200555755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6806c920 con 0x7f8c680cfa10 2024-04-25T00:31:26.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c66ffd640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c680d04a0 0x7f8c68164b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:26.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c677fe640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c680cfa10 0x7f8c681545d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8c5800c500 tx=0x7f8c5800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:26.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.212+0000 7f8c4ffff640 1 -- 172.21.15.29:0/4200555755 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58017070 con 0x7f8c680cfa10 2024-04-25T00:31:26.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.216+0000 7f8c4ffff640 1 -- 172.21.15.29:0/4200555755 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c58003600 con 0x7f8c680cfa10 2024-04-25T00:31:26.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.216+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/4200555755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6806cc00 con 0x7f8c680cfa10 2024-04-25T00:31:26.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.216+0000 7f8c4ffff640 1 -- 172.21.15.29:0/4200555755 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c580039f0 con 0x7f8c680cfa10 2024-04-25T00:31:26.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.216+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/4200555755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c6806d1c0 con 0x7f8c680cfa10 2024-04-25T00:31:26.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.216+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/4200555755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c68069270 con 0x7f8c680cfa10 2024-04-25T00:31:26.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.216+0000 7f8c4ffff640 1 -- 172.21.15.29:0/4200555755 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f8c5801a070 con 0x7f8c680cfa10 2024-04-25T00:31:26.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.216+0000 7f8c4ffff640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8c54067860 0x7f8c54069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.216+0000 7f8c4ffff640 1 -- 172.21.15.29:0/4200555755 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4857+0+0 (secure 0 0 0) 0x7f8c58084c00 con 0x7f8c680cfa10 2024-04-25T00:31:26.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.216+0000 7f8c66ffd640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8c54067860 0x7f8c54069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.216+0000 7f8c66ffd640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8c54067860 0x7f8c54069d20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8c680d2e80 tx=0x7f8c5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:26.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.220+0000 7f8c4ffff640 1 -- 172.21.15.29:0/4200555755 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8c58055cc0 con 0x7f8c680cfa10 2024-04-25T00:31:26.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.424+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/4200555755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c680af0d0 con 0x7f8c680cfa10 2024-04-25T00:31:26.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.424+0000 7f8c4ffff640 1 -- 172.21.15.29:0/4200555755 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7f8c580555c0 con 0x7f8c680cfa10 2024-04-25T00:31:26.427 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:26.427 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1532858457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532858457},{"type":"v1","addr":"172.21.15.29:6833","nonce":1532858457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/718997473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":718997473},{"type":"v1","addr":"172.21.15.157:6835","nonce":718997473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1617208213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1617208213},{"type":"v1","addr":"172.21.15.157:6837","nonce":1617208213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4252,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/1924706890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1924706890},{"type":"v1","addr":"172.21.15.29:6835","nonce":1924706890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T00:31:26.427 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 3 2024-04-25T00:31:26.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.428+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8c54067860 msgr2=0x7f8c54069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.428+0000 7f8c6dc2b640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8c54067860 0x7f8c54069d20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8c680d2e80 tx=0x7f8c5c008040 comp rx=0 tx=0).stop 2024-04-25T00:31:26.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.428+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/4200555755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c680cfa10 msgr2=0x7f8c681545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.428+0000 7f8c6dc2b640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c680cfa10 0x7f8c681545d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8c5800c500 tx=0x7f8c5800c9c0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.428+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/4200555755 shutdown_connections 2024-04-25T00:31:26.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.428+0000 7f8c6dc2b640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c6815c220 0x7f8c6806d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.428+0000 7f8c6dc2b640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8c54067860 0x7f8c54069d20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.428+0000 7f8c6dc2b640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c680d04a0 0x7f8c68164b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.428+0000 7f8c6dc2b640 1 --2- 172.21.15.29:0/4200555755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c680cfa10 0x7f8c681545d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.428+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/4200555755 >> 172.21.15.29:0/4200555755 conn(0x7f8c680d1d60 msgr2=0x7f8c680af880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:26.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.428+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/4200555755 shutdown_connections 2024-04-25T00:31:26.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.428+0000 7f8c6dc2b640 1 -- 172.21.15.29:0/4200555755 wait complete. 2024-04-25T00:31:26.448 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:26.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f341bf0a640 1 Processor -- start 2024-04-25T00:31:26.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f341bf0a640 1 -- start start 2024-04-25T00:31:26.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f341bf0a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34140cfa10 0x7f34140cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f341bf0a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34140cdbd0 con 0x7f34140d04a0 2024-04-25T00:31:26.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f341bf0a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34140b13e0 con 0x7f341415c220 2024-04-25T00:31:26.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f341bf0a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f34140d0900 con 0x7f34140cfa10 2024-04-25T00:31:26.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f341a4a7640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f34140d04a0 0x7f34140b1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52394/0 (socket says 172.21.15.29:52394) 2024-04-25T00:31:26.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f341a4a7640 1 -- 172.21.15.29:0/4165336536 learned_addr learned my addr 172.21.15.29:0/4165336536 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:26.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f34194a5640 1 --2- 172.21.15.29:0/4165336536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34140cfa10 0x7f34140cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f3418ca4640 1 -- 172.21.15.29:0/4165336536 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3812447241 0 0) 0x7f34140cdbd0 con 0x7f34140d04a0 2024-04-25T00:31:26.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f3418ca4640 1 -- 172.21.15.29:0/4165336536 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33fc003530 con 0x7f34140d04a0 2024-04-25T00:31:26.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f34194a5640 1 -- 172.21.15.29:0/4165336536 >> v1:172.21.15.157:6789/0 conn(0x7f341415c220 legacy=0x7f34140cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f34194a5640 1 -- 172.21.15.29:0/4165336536 >> v1:172.21.15.29:6789/0 conn(0x7f34140d04a0 legacy=0x7f34140b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.564+0000 7f34194a5640 1 -- 172.21.15.29:0/4165336536 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34140b16c0 con 0x7f34140cfa10 2024-04-25T00:31:26.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f34194a5640 1 --2- 172.21.15.29:0/4165336536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34140cfa10 0x7f34140cff60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3408009dc0 tx=0x7f340801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6837444e67ef9d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:26.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f3418ca4640 1 -- 172.21.15.29:0/4165336536 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34080b2070 con 0x7f34140cfa10 2024-04-25T00:31:26.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f3418ca4640 1 -- 172.21.15.29:0/4165336536 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34080a5850 con 0x7f34140cfa10 2024-04-25T00:31:26.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f3418ca4640 1 -- 172.21.15.29:0/4165336536 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34080a5c00 con 0x7f34140cfa10 2024-04-25T00:31:26.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 -- 172.21.15.29:0/4165336536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34140cfa10 msgr2=0x7f34140cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 --2- 172.21.15.29:0/4165336536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34140cfa10 0x7f34140cff60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3408009dc0 tx=0x7f340801c990 comp rx=0 tx=0).stop 2024-04-25T00:31:26.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 -- 172.21.15.29:0/4165336536 shutdown_connections 2024-04-25T00:31:26.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 --2- 172.21.15.29:0/4165336536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34140cfa10 0x7f34140cff60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 -- 172.21.15.29:0/4165336536 >> 172.21.15.29:0/4165336536 conn(0x7f34140d1d60 msgr2=0x7f34140d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:26.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 -- 172.21.15.29:0/4165336536 shutdown_connections 2024-04-25T00:31:26.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 -- 172.21.15.29:0/4165336536 wait complete. 2024-04-25T00:31:26.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 Processor -- start 2024-04-25T00:31:26.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 -- start start 2024-04-25T00:31:26.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34140cfa10 0x7f34141536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34140d04a0 0x7f34140caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f341415c220 0x7f34140cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f34140b1a30 con 0x7f341415c220 2024-04-25T00:31:26.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34140b30b0 con 0x7f34140cfa10 2024-04-25T00:31:26.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.568+0000 7f341bf0a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3414169d10 con 0x7f34140d04a0 2024-04-25T00:31:26.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f341a4a7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f341415c220 0x7f34140cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f341a4a7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f341415c220 0x7f34140cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52876/0 (socket says 172.21.15.29:52876) 2024-04-25T00:31:26.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f341a4a7640 1 -- 172.21.15.29:0/1672826195 learned_addr learned my addr 172.21.15.29:0/1672826195 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:26.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f34194a5640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34140d04a0 0x7f34140caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f3419ca6640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34140cfa10 0x7f34141536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f341a4a7640 1 -- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34140d04a0 msgr2=0x7f34140caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f341a4a7640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34140d04a0 0x7f34140caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f341a4a7640 1 -- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34140cfa10 msgr2=0x7f34141536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f341a4a7640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34140cfa10 0x7f34141536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f341a4a7640 1 -- 172.21.15.29:0/1672826195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3414153be0 con 0x7f341415c220 2024-04-25T00:31:26.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f341a4a7640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f341415c220 0x7f34140cb430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f340400c3c0 tx=0x7f340400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:26.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f34127fc640 1 -- 172.21.15.29:0/1672826195 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404017070 con 0x7f341415c220 2024-04-25T00:31:26.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f34127fc640 1 -- 172.21.15.29:0/1672826195 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3404005cf0 con 0x7f341415c220 2024-04-25T00:31:26.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f341bf0a640 1 -- 172.21.15.29:0/1672826195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341415d5a0 con 0x7f341415c220 2024-04-25T00:31:26.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f34127fc640 1 -- 172.21.15.29:0/1672826195 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404007310 con 0x7f341415c220 2024-04-25T00:31:26.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f341bf0a640 1 -- 172.21.15.29:0/1672826195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f341415db60 con 0x7f341415c220 2024-04-25T00:31:26.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f341bf0a640 1 -- 172.21.15.29:0/1672826195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34140c66d0 con 0x7f341415c220 2024-04-25T00:31:26.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f34127fc640 1 -- 172.21.15.29:0/1672826195 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f340401a070 con 0x7f341415c220 2024-04-25T00:31:26.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f34127fc640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3400067830 0x7f3400069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f34127fc640 1 -- 172.21.15.29:0/1672826195 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3404084960 con 0x7f341415c220 2024-04-25T00:31:26.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f3419ca6640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3400067830 0x7f3400069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.572+0000 7f3419ca6640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3400067830 0x7f3400069cf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f340c001770 tx=0x7f340c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:26.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.576+0000 7f34127fc640 1 -- 172.21.15.29:0/1672826195 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3404055310 con 0x7f341415c220 2024-04-25T00:31:26.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.740+0000 7f341bf0a640 1 -- 172.21.15.29:0/1672826195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3414069270 con 0x7f341415c220 2024-04-25T00:31:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.764+0000 7f34127fc640 1 -- 172.21.15.29:0/1672826195 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12836 (secure 0 0 0) 0x7f3404059610 con 0x7f341415c220 2024-04-25T00:31:26.765 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:26.765 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":16,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:26.219231+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:26.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.764+0000 7f341bf0a640 1 -- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3400067830 msgr2=0x7f3400069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.764+0000 7f341bf0a640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3400067830 0x7f3400069cf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f340c001770 tx=0x7f340c009040 comp rx=0 tx=0).stop 2024-04-25T00:31:26.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.764+0000 7f341bf0a640 1 -- 172.21.15.29:0/1672826195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f341415c220 msgr2=0x7f34140cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.764+0000 7f341bf0a640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f341415c220 0x7f34140cb430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f340400c3c0 tx=0x7f340400c880 comp rx=0 tx=0).stop 2024-04-25T00:31:26.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.764+0000 7f341bf0a640 1 -- 172.21.15.29:0/1672826195 shutdown_connections 2024-04-25T00:31:26.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.764+0000 7f341bf0a640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f341415c220 0x7f34140cb430 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.764+0000 7f341bf0a640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34140d04a0 0x7f34140caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.764+0000 7f341bf0a640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3400067830 0x7f3400069cf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.764+0000 7f341bf0a640 1 --2- 172.21.15.29:0/1672826195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34140cfa10 0x7f34141536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.764+0000 7f341bf0a640 1 -- 172.21.15.29:0/1672826195 >> 172.21.15.29:0/1672826195 conn(0x7f34140d1d60 msgr2=0x7f3414154f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:26.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.764+0000 7f341bf0a640 1 -- 172.21.15.29:0/1672826195 shutdown_connections 2024-04-25T00:31:26.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.764+0000 7f341bf0a640 1 -- 172.21.15.29:0/1672826195 wait complete. 2024-04-25T00:31:26.785 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:31:26.785 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:31:26.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f931e82b640 1 Processor -- start 2024-04-25T00:31:26.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f931e82b640 1 -- start start 2024-04-25T00:31:26.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f931e82b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93180d06c0 0x7f9318072040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f931e82b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93180cf930 0x7f931806b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f931e82b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93180cd740 con 0x7f931806b210 2024-04-25T00:31:26.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f931e82b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f93180d0aa0 con 0x7f93180d06c0 2024-04-25T00:31:26.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f931e82b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f93180ce350 con 0x7f93180cf930 2024-04-25T00:31:26.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f9317fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93180d06c0 0x7f9318072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f930ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f931806b210 0x7f931815f9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:50772/0 (socket says 172.21.15.29:50772) 2024-04-25T00:31:26.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f930ffff640 1 -- 172.21.15.29:0/314924410 learned_addr learned my addr 172.21.15.29:0/314924410 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:26.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f931cdc8640 1 --2- 172.21.15.29:0/314924410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93180cf930 0x7f931806b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f93177fe640 1 -- 172.21.15.29:0/314924410 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2927599937 0 0) 0x7f93180cd740 con 0x7f931806b210 2024-04-25T00:31:26.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f93177fe640 1 -- 172.21.15.29:0/314924410 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9304003530 con 0x7f931806b210 2024-04-25T00:31:26.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f9317fff640 1 -- 172.21.15.29:0/314924410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93180cf930 msgr2=0x7f931806b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:26.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f9317fff640 1 --2- 172.21.15.29:0/314924410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93180cf930 0x7f931806b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f9317fff640 1 -- 172.21.15.29:0/314924410 >> v1:172.21.15.157:6789/0 conn(0x7f931806b210 legacy=0x7f931815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.904+0000 7f9317fff640 1 -- 172.21.15.29:0/314924410 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93180cda20 con 0x7f93180d06c0 2024-04-25T00:31:26.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f9317fff640 1 --2- 172.21.15.29:0/314924410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93180d06c0 0x7f9318072040 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f93080097c0 tx=0x7f930801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67b672ca577c3f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:26.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f93177fe640 1 -- 172.21.15.29:0/314924410 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93080b0070 con 0x7f93180d06c0 2024-04-25T00:31:26.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f93177fe640 1 -- 172.21.15.29:0/314924410 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93080ab600 con 0x7f93180d06c0 2024-04-25T00:31:26.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f93177fe640 1 -- 172.21.15.29:0/314924410 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93080ab9b0 con 0x7f93180d06c0 2024-04-25T00:31:26.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f931e82b640 1 -- 172.21.15.29:0/314924410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93180d06c0 msgr2=0x7f9318072040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f931e82b640 1 --2- 172.21.15.29:0/314924410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93180d06c0 0x7f9318072040 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f93080097c0 tx=0x7f930801c410 comp rx=0 tx=0).stop 2024-04-25T00:31:26.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f931e82b640 1 -- 172.21.15.29:0/314924410 shutdown_connections 2024-04-25T00:31:26.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f931e82b640 1 --2- 172.21.15.29:0/314924410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93180cf930 0x7f931806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f931e82b640 1 --2- 172.21.15.29:0/314924410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93180d06c0 0x7f9318072040 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f931e82b640 1 -- 172.21.15.29:0/314924410 >> 172.21.15.29:0/314924410 conn(0x7f93180b2920 msgr2=0x7f93180b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:26.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f931e82b640 1 -- 172.21.15.29:0/314924410 shutdown_connections 2024-04-25T00:31:26.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f931e82b640 1 -- 172.21.15.29:0/314924410 wait complete. 2024-04-25T00:31:26.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f931e82b640 1 Processor -- start 2024-04-25T00:31:26.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.908+0000 7f931e82b640 1 -- start start 2024-04-25T00:31:26.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931e82b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f931806b210 0x7f93181436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931e82b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93180cf930 0x7f93180b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931e82b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93180d06c0 0x7f93180b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931e82b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f93180cd450 con 0x7f93180d06c0 2024-04-25T00:31:26.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931e82b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f93180cd2d0 con 0x7f931806b210 2024-04-25T00:31:26.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931e82b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9318168b10 con 0x7f93180cf930 2024-04-25T00:31:26.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931cdc8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93180d06c0 0x7f93180b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931cdc8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93180d06c0 0x7f93180b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52882/0 (socket says 172.21.15.29:52882) 2024-04-25T00:31:26.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931cdc8640 1 -- 172.21.15.29:0/1249158919 learned_addr learned my addr 172.21.15.29:0/1249158919 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:26.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f930ffff640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93180cf930 0x7f93180b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f9317fff640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f931806b210 0x7f93181436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931cdc8640 1 -- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93180cf930 msgr2=0x7f93180b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931cdc8640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93180cf930 0x7f93180b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931cdc8640 1 -- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f931806b210 msgr2=0x7f93181436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:26.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931cdc8640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f931806b210 0x7f93181436b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:26.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931cdc8640 1 -- 172.21.15.29:0/1249158919 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93180b4870 con 0x7f93180d06c0 2024-04-25T00:31:26.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f9317fff640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f931806b210 0x7f93181436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:26.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931cdc8640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93180d06c0 0x7f93180b41f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f931000efc0 tx=0x7f931000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:26.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f93157fa640 1 -- 172.21.15.29:0/1249158919 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9310018020 con 0x7f93180d06c0 2024-04-25T00:31:26.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f93157fa640 1 -- 172.21.15.29:0/1249158919 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9310014480 con 0x7f93180d06c0 2024-04-25T00:31:26.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931e82b640 1 -- 172.21.15.29:0/1249158919 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931815c680 con 0x7f93180d06c0 2024-04-25T00:31:26.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f93157fa640 1 -- 172.21.15.29:0/1249158919 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9310014870 con 0x7f93180d06c0 2024-04-25T00:31:26.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931e82b640 1 -- 172.21.15.29:0/1249158919 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9318143bf0 con 0x7f93180d06c0 2024-04-25T00:31:26.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f931e82b640 1 -- 172.21.15.29:0/1249158919 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f931815c4a0 con 0x7f93180d06c0 2024-04-25T00:31:26.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.912+0000 7f93157fa640 1 -- 172.21.15.29:0/1249158919 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f931001b020 con 0x7f93180d06c0 2024-04-25T00:31:26.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.916+0000 7f93157fa640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f92f8067830 0x7f92f8069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:26.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.916+0000 7f93157fa640 1 -- 172.21.15.29:0/1249158919 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f9310086b20 con 0x7f93180d06c0 2024-04-25T00:31:26.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.916+0000 7f9317fff640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f92f8067830 0x7f92f8069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:26.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.916+0000 7f9317fff640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f92f8067830 0x7f92f8069cf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f93080ad000 tx=0x7f93080a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:26.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:26.920+0000 7f93157fa640 1 -- 172.21.15.29:0/1249158919 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f931815c4a0 con 0x7f93180d06c0 2024-04-25T00:31:27.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f931e82b640 1 -- 172.21.15.29:0/1249158919 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9318069270 con 0x7f93180d06c0 2024-04-25T00:31:27.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f93157fa640 1 -- 172.21.15.29:0/1249158919 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f9310022070 con 0x7f93180d06c0 2024-04-25T00:31:27.085 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:31:27.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f931e82b640 1 -- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f92f8067830 msgr2=0x7f92f8069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:27.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f931e82b640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f92f8067830 0x7f92f8069cf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f93080ad000 tx=0x7f93080a6000 comp rx=0 tx=0).stop 2024-04-25T00:31:27.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f931e82b640 1 -- 172.21.15.29:0/1249158919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93180d06c0 msgr2=0x7f93180b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:27.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f931e82b640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93180d06c0 0x7f93180b41f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f931000efc0 tx=0x7f931000c490 comp rx=0 tx=0).stop 2024-04-25T00:31:27.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f931e82b640 1 -- 172.21.15.29:0/1249158919 shutdown_connections 2024-04-25T00:31:27.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f931e82b640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93180d06c0 0x7f93180b41f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f931e82b640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93180cf930 0x7f93180b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f931e82b640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f92f8067830 0x7f92f8069cf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f931e82b640 1 --2- 172.21.15.29:0/1249158919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f931806b210 0x7f93181436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f931e82b640 1 -- 172.21.15.29:0/1249158919 >> 172.21.15.29:0/1249158919 conn(0x7f93180b2920 msgr2=0x7f93180adfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:27.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f931e82b640 1 -- 172.21.15.29:0/1249158919 shutdown_connections 2024-04-25T00:31:27.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.084+0000 7f931e82b640 1 -- 172.21.15.29:0/1249158919 wait complete. 2024-04-25T00:31:27.105 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:31:27.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a72212640 1 Processor -- start 2024-04-25T00:31:27.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a72212640 1 -- start start 2024-04-25T00:31:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a72212640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1a6c156420 0x7f1a6c15ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a72212640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1a6c15fa80 0x7f1a6c06b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a72212640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a6c15ff10 0x7f1a6c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a72212640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1a6c15d3a0 con 0x7f1a6c156420 2024-04-25T00:31:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a72212640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1a6c15c680 con 0x7f1a6c15fa80 2024-04-25T00:31:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a72212640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6c160440 con 0x7f1a6c15ff10 2024-04-25T00:31:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a6b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1a6c156420 0x7f1a6c15ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a6b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1a6c156420 0x7f1a6c15ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52898/0 (socket says 172.21.15.29:52898) 2024-04-25T00:31:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a6b7fe640 1 -- 172.21.15.29:0/3256756068 learned_addr learned my addr 172.21.15.29:0/3256756068 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a6affd640 1 --2- 172.21.15.29:0/3256756068 >> v2:172.21.15.157:3300/0 conn(0x7f1a6c15fa80 0x7f1a6c06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a6bfff640 1 --2- 172.21.15.29:0/3256756068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a6c15ff10 0x7f1a6c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a6b7fe640 1 -- 172.21.15.29:0/3256756068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a6c15ff10 msgr2=0x7f1a6c06b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a6b7fe640 1 --2- 172.21.15.29:0/3256756068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a6c15ff10 0x7f1a6c06b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a6b7fe640 1 -- 172.21.15.29:0/3256756068 >> v2:172.21.15.157:3300/0 conn(0x7f1a6c15fa80 msgr2=0x7f1a6c06b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a6b7fe640 1 --2- 172.21.15.29:0/3256756068 >> v2:172.21.15.157:3300/0 conn(0x7f1a6c15fa80 0x7f1a6c06b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a6b7fe640 1 -- 172.21.15.29:0/3256756068 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6c160720 con 0x7f1a6c156420 2024-04-25T00:31:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a6bfff640 1 --2- 172.21.15.29:0/3256756068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a6c15ff10 0x7f1a6c06b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.228+0000 7f1a6affd640 1 --2- 172.21.15.29:0/3256756068 >> v2:172.21.15.157:3300/0 conn(0x7f1a6c15fa80 0x7f1a6c06b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:27.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.232+0000 7f1a6b7fe640 1 --2- 172.21.15.29:0/3256756068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c156420 0x7f1a6c15ce60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c159fd0 tx=0x7f1a6001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fc6f9b2a15ef86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:27.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.232+0000 7f1a6a7fc640 1 -- 172.21.15.29:0/3256756068 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a600ab5f0 con 0x7f1a6c156420 2024-04-25T00:31:27.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.232+0000 7f1a6a7fc640 1 -- 172.21.15.29:0/3256756068 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a600b0070 con 0x7f1a6c156420 2024-04-25T00:31:27.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.232+0000 7f1a6a7fc640 1 -- 172.21.15.29:0/3256756068 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a600ab9b0 con 0x7f1a6c156420 2024-04-25T00:31:27.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.232+0000 7f1a72212640 1 -- 172.21.15.29:0/3256756068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c156420 msgr2=0x7f1a6c15ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:27.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.232+0000 7f1a72212640 1 --2- 172.21.15.29:0/3256756068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c156420 0x7f1a6c15ce60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c159fd0 tx=0x7f1a6001c520 comp rx=0 tx=0).stop 2024-04-25T00:31:27.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.232+0000 7f1a72212640 1 -- 172.21.15.29:0/3256756068 shutdown_connections 2024-04-25T00:31:27.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.232+0000 7f1a72212640 1 --2- 172.21.15.29:0/3256756068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a6c15ff10 0x7f1a6c06b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.232+0000 7f1a72212640 1 --2- 172.21.15.29:0/3256756068 >> v2:172.21.15.157:3300/0 conn(0x7f1a6c15fa80 0x7f1a6c06b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.232+0000 7f1a72212640 1 --2- 172.21.15.29:0/3256756068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c156420 0x7f1a6c15ce60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.232+0000 7f1a72212640 1 -- 172.21.15.29:0/3256756068 >> 172.21.15.29:0/3256756068 conn(0x7f1a6c0d0c40 msgr2=0x7f1a6c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:27.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.232+0000 7f1a72212640 1 -- 172.21.15.29:0/3256756068 shutdown_connections 2024-04-25T00:31:27.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.232+0000 7f1a72212640 1 -- 172.21.15.29:0/3256756068 wait complete. 2024-04-25T00:31:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a72212640 1 Processor -- start 2024-04-25T00:31:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a72212640 1 -- start start 2024-04-25T00:31:27.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a72212640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c156420 0x7f1a6c157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:27.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a72212640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a6c15fa80 0x7f1a6c154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:27.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a72212640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a6c15ff10 0x7f1a6c1550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a72212640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c0cbc20 con 0x7f1a6c156420 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a72212640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c0cbaa0 con 0x7f1a6c15ff10 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a72212640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6c0cbda0 con 0x7f1a6c15fa80 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a6b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c156420 0x7f1a6c157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a6b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c156420 0x7f1a6c157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52906/0 (socket says 172.21.15.29:52906) 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a6b7fe640 1 -- 172.21.15.29:0/1728624642 learned_addr learned my addr 172.21.15.29:0/1728624642 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a6bfff640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a6c15ff10 0x7f1a6c1550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a6affd640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a6c15fa80 0x7f1a6c154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a6b7fe640 1 -- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a6c15fa80 msgr2=0x7f1a6c154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a6b7fe640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a6c15fa80 0x7f1a6c154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a6b7fe640 1 -- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a6c15ff10 msgr2=0x7f1a6c1550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a6b7fe640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a6c15ff10 0x7f1a6c1550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a6b7fe640 1 -- 172.21.15.29:0/1728624642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6c158590 con 0x7f1a6c156420 2024-04-25T00:31:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a6bfff640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a6c15ff10 0x7f1a6c1550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:27.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a6b7fe640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c156420 0x7f1a6c157bd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a600ad040 tx=0x7f1a6001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:27.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a53fff640 1 -- 172.21.15.29:0/1728624642 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a600be5f0 con 0x7f1a6c156420 2024-04-25T00:31:27.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a53fff640 1 -- 172.21.15.29:0/1728624642 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a600be790 con 0x7f1a6c156420 2024-04-25T00:31:27.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a72212640 1 -- 172.21.15.29:0/1728624642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6c158870 con 0x7f1a6c156420 2024-04-25T00:31:27.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a53fff640 1 -- 172.21.15.29:0/1728624642 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a600beb80 con 0x7f1a6c156420 2024-04-25T00:31:27.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a72212640 1 -- 172.21.15.29:0/1728624642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6c158e30 con 0x7f1a6c156420 2024-04-25T00:31:27.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.236+0000 7f1a72212640 1 -- 172.21.15.29:0/1728624642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a6c069270 con 0x7f1a6c156420 2024-04-25T00:31:27.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.240+0000 7f1a53fff640 1 -- 172.21.15.29:0/1728624642 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f1a600b43f0 con 0x7f1a6c156420 2024-04-25T00:31:27.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.240+0000 7f1a53fff640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1a58067830 0x7f1a58069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:27.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.240+0000 7f1a53fff640 1 -- 172.21.15.29:0/1728624642 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f1a60120230 con 0x7f1a6c156420 2024-04-25T00:31:27.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.240+0000 7f1a6affd640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1a58067830 0x7f1a58069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:27.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.240+0000 7f1a6affd640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1a58067830 0x7f1a58069cf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0c7050 tx=0x7f1a5c0039a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:27.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.244+0000 7f1a53fff640 1 -- 172.21.15.29:0/1728624642 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1a600c4060 con 0x7f1a6c156420 2024-04-25T00:31:27.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.404+0000 7f1a72212640 1 -- 172.21.15.29:0/1728624642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1a6c166bf0 con 0x7f1a6c156420 2024-04-25T00:31:27.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.404+0000 7f1a53fff640 1 -- 172.21.15.29:0/1728624642 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f1a60016110 con 0x7f1a6c156420 2024-04-25T00:31:27.408 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:27.408 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:31:27.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:31:27.408 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-04-25T00:31:27.408 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": ".mgr" 2024-04-25T00:31:27.408 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:31:27.408 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:31:27.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.408+0000 7f1a72212640 1 -- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1a58067830 msgr2=0x7f1a58069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:27.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.408+0000 7f1a72212640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1a58067830 0x7f1a58069cf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0c7050 tx=0x7f1a5c0039a0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.408+0000 7f1a72212640 1 -- 172.21.15.29:0/1728624642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c156420 msgr2=0x7f1a6c157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:27.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.408+0000 7f1a72212640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c156420 0x7f1a6c157bd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a600ad040 tx=0x7f1a6001ad60 comp rx=0 tx=0).stop 2024-04-25T00:31:27.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.408+0000 7f1a72212640 1 -- 172.21.15.29:0/1728624642 shutdown_connections 2024-04-25T00:31:27.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.408+0000 7f1a72212640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1a58067830 0x7f1a58069cf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.408+0000 7f1a72212640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a6c15ff10 0x7f1a6c1550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.408+0000 7f1a72212640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a6c15fa80 0x7f1a6c154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.408+0000 7f1a72212640 1 --2- 172.21.15.29:0/1728624642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c156420 0x7f1a6c157bd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.408+0000 7f1a72212640 1 -- 172.21.15.29:0/1728624642 >> 172.21.15.29:0/1728624642 conn(0x7f1a6c0d0c40 msgr2=0x7f1a6c1532b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:27.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.408+0000 7f1a72212640 1 -- 172.21.15.29:0/1728624642 shutdown_connections 2024-04-25T00:31:27.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.408+0000 7f1a72212640 1 -- 172.21.15.29:0/1728624642 wait complete. 2024-04-25T00:31:27.427 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:31:27.428 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:31:27.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff24e582640 1 Processor -- start 2024-04-25T00:31:27.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff24e582640 1 -- start start 2024-04-25T00:31:27.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff24e582640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff24815e9a0 0x7ff248070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:27.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff24e582640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff2480b4590 0x7ff24815ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:27.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff24e582640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff24815f2c0 con 0x7ff2480cd1c0 2024-04-25T00:31:27.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff24e582640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff248071ed0 con 0x7ff24815e9a0 2024-04-25T00:31:27.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff24e582640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff2480712c0 con 0x7ff2480b4590 2024-04-25T00:31:27.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff247fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff24815e9a0 0x7ff248070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:27.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff24cb1f640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff2480cd1c0 0x7ff248072360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52404/0 (socket says 172.21.15.29:52404) 2024-04-25T00:31:27.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff24cb1f640 1 -- 172.21.15.29:0/794712378 learned_addr learned my addr 172.21.15.29:0/794712378 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:27.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff2477fe640 1 --2- 172.21.15.29:0/794712378 >> v2:172.21.15.157:3300/0 conn(0x7ff2480b4590 0x7ff24815ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:27.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff247fff640 1 -- 172.21.15.29:0/794712378 >> v2:172.21.15.157:3300/0 conn(0x7ff2480b4590 msgr2=0x7ff24815ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:27.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff247fff640 1 --2- 172.21.15.29:0/794712378 >> v2:172.21.15.157:3300/0 conn(0x7ff2480b4590 0x7ff24815ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff247fff640 1 -- 172.21.15.29:0/794712378 >> v1:172.21.15.29:6789/0 conn(0x7ff2480cd1c0 legacy=0x7ff248072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:27.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff247fff640 1 -- 172.21.15.29:0/794712378 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2480715a0 con 0x7ff24815e9a0 2024-04-25T00:31:27.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.556+0000 7ff246ffd640 1 -- 172.21.15.29:0/794712378 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1342035685 0 0) 0x7ff24815f2c0 con 0x7ff2480cd1c0 2024-04-25T00:31:27.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff247fff640 1 --2- 172.21.15.29:0/794712378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff24815e9a0 0x7ff248070ce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff248067fa0 tx=0x7ff23c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bc744295fa0aef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:27.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff246ffd640 1 -- 172.21.15.29:0/794712378 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0ab650 con 0x7ff24815e9a0 2024-04-25T00:31:27.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff246ffd640 1 -- 172.21.15.29:0/794712378 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff23c0b0070 con 0x7ff24815e9a0 2024-04-25T00:31:27.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff246ffd640 1 -- 172.21.15.29:0/794712378 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0aba10 con 0x7ff24815e9a0 2024-04-25T00:31:27.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff24e582640 1 -- 172.21.15.29:0/794712378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff24815e9a0 msgr2=0x7ff248070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:27.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff24e582640 1 --2- 172.21.15.29:0/794712378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff24815e9a0 0x7ff248070ce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff248067fa0 tx=0x7ff23c01c450 comp rx=0 tx=0).stop 2024-04-25T00:31:27.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff24e582640 1 -- 172.21.15.29:0/794712378 shutdown_connections 2024-04-25T00:31:27.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff24e582640 1 --2- 172.21.15.29:0/794712378 >> v2:172.21.15.157:3300/0 conn(0x7ff2480b4590 0x7ff24815ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff24e582640 1 --2- 172.21.15.29:0/794712378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff24815e9a0 0x7ff248070ce0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff24e582640 1 -- 172.21.15.29:0/794712378 >> 172.21.15.29:0/794712378 conn(0x7ff2480b1870 msgr2=0x7ff2480b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff24e582640 1 -- 172.21.15.29:0/794712378 shutdown_connections 2024-04-25T00:31:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff24e582640 1 -- 172.21.15.29:0/794712378 wait complete. 2024-04-25T00:31:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff24e582640 1 Processor -- start 2024-04-25T00:31:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.560+0000 7ff24e582640 1 -- start start 2024-04-25T00:31:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24e582640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2480b4590 0x7ff2481f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24e582640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2480cd1c0 0x7ff248159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24e582640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff24815e9a0 0x7ff24815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24e582640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff248144870 con 0x7ff24815e9a0 2024-04-25T00:31:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24e582640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff248072e80 con 0x7ff2480cd1c0 2024-04-25T00:31:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24e582640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff2481449f0 con 0x7ff2480b4590 2024-04-25T00:31:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24cb1f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff24815e9a0 0x7ff24815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24cb1f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff24815e9a0 0x7ff24815dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52918/0 (socket says 172.21.15.29:52918) 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24cb1f640 1 -- 172.21.15.29:0/212130790 learned_addr learned my addr 172.21.15.29:0/212130790 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff2477fe640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2480cd1c0 0x7ff248159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff247fff640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2480b4590 0x7ff2481f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24cb1f640 1 -- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2480b4590 msgr2=0x7ff2481f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24cb1f640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2480b4590 0x7ff2481f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24cb1f640 1 -- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2480cd1c0 msgr2=0x7ff248159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24cb1f640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2480cd1c0 0x7ff248159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24cb1f640 1 -- 172.21.15.29:0/212130790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2480ceb70 con 0x7ff24815e9a0 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24cb1f640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff24815e9a0 0x7ff24815dec0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff240001570 tx=0x7ff24000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff244ff9640 1 -- 172.21.15.29:0/212130790 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240018070 con 0x7ff24815e9a0 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff244ff9640 1 -- 172.21.15.29:0/212130790 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff24000a640 con 0x7ff24815e9a0 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24e582640 1 -- 172.21.15.29:0/212130790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2480cee70 con 0x7ff24815e9a0 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff244ff9640 1 -- 172.21.15.29:0/212130790 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24000aa30 con 0x7ff24815e9a0 2024-04-25T00:31:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24e582640 1 -- 172.21.15.29:0/212130790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2480cf6b0 con 0x7ff24815e9a0 2024-04-25T00:31:27.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff24e582640 1 -- 172.21.15.29:0/212130790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2480692e0 con 0x7ff24815e9a0 2024-04-25T00:31:27.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff244ff9640 1 -- 172.21.15.29:0/212130790 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7ff24001b070 con 0x7ff24815e9a0 2024-04-25T00:31:27.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff244ff9640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff234067830 0x7ff234069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:27.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff244ff9640 1 -- 172.21.15.29:0/212130790 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff240084ea0 con 0x7ff24815e9a0 2024-04-25T00:31:27.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.564+0000 7ff247fff640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff234067830 0x7ff234069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:27.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.568+0000 7ff247fff640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff234067830 0x7ff234069cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0af000 tx=0x7ff23c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:27.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.568+0000 7ff244ff9640 1 -- 172.21.15.29:0/212130790 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff2480cee70 con 0x7ff24815e9a0 2024-04-25T00:31:27.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:27.736+0000 7ff24e582640 1 -- 172.21.15.29:0/212130790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff24806edc0 con 0x7ff24815e9a0 2024-04-25T00:31:28.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.240+0000 7ff244ff9640 1 -- 172.21.15.29:0/212130790 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7ff240055e00 con 0x7ff24815e9a0 2024-04-25T00:31:28.244 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-04-25T00:31:28.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.244+0000 7ff24e582640 1 -- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff234067830 msgr2=0x7ff234069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:28.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.244+0000 7ff24e582640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff234067830 0x7ff234069cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0af000 tx=0x7ff23c0a7000 comp rx=0 tx=0).stop 2024-04-25T00:31:28.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.244+0000 7ff24e582640 1 -- 172.21.15.29:0/212130790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff24815e9a0 msgr2=0x7ff24815dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:28.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.244+0000 7ff24e582640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff24815e9a0 0x7ff24815dec0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff240001570 tx=0x7ff24000cca0 comp rx=0 tx=0).stop 2024-04-25T00:31:28.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.244+0000 7ff24e582640 1 -- 172.21.15.29:0/212130790 shutdown_connections 2024-04-25T00:31:28.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.244+0000 7ff24e582640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff24815e9a0 0x7ff24815dec0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:28.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.244+0000 7ff24e582640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff234067830 0x7ff234069cf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:28.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.244+0000 7ff24e582640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2480cd1c0 0x7ff248159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:28.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.244+0000 7ff24e582640 1 --2- 172.21.15.29:0/212130790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2480b4590 0x7ff2481f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:28.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.244+0000 7ff24e582640 1 -- 172.21.15.29:0/212130790 >> 172.21.15.29:0/212130790 conn(0x7ff2480b1870 msgr2=0x7ff24815ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:28.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.244+0000 7ff24e582640 1 -- 172.21.15.29:0/212130790 shutdown_connections 2024-04-25T00:31:28.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.244+0000 7ff24e582640 1 -- 172.21.15.29:0/212130790 wait complete. 2024-04-25T00:31:28.275 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:31:28.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.404+0000 7fb7dfa51640 1 Processor -- start 2024-04-25T00:31:28.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.404+0000 7fb7dfa51640 1 -- start start 2024-04-25T00:31:28.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.404+0000 7fb7dfa51640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb7d815f3c0 0x7fb7d80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:28.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.404+0000 7fb7dfa51640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7d80b3720 con 0x7fb7d815e8d0 2024-04-25T00:31:28.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.404+0000 7fb7dfa51640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7d80b39c0 con 0x7fb7d80b4780 2024-04-25T00:31:28.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.404+0000 7fb7dfa51640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb7d80b42f0 con 0x7fb7d815f3c0 2024-04-25T00:31:28.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.404+0000 7fb7ddfee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb7d815f3c0 0x7fb7d80cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:28.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.404+0000 7fb7ddfee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb7d815f3c0 0x7fb7d80cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51554/0 (socket says 172.21.15.29:51554) 2024-04-25T00:31:28.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.404+0000 7fb7ddfee640 1 -- 172.21.15.29:0/1924770485 learned_addr learned my addr 172.21.15.29:0/1924770485 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:28.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.404+0000 7fb7ddfee640 1 -- 172.21.15.29:0/1924770485 >> v1:172.21.15.157:6789/0 conn(0x7fb7d80b4780 legacy=0x7fb7d815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:28.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.404+0000 7fb7ddfee640 1 -- 172.21.15.29:0/1924770485 >> v1:172.21.15.29:6789/0 conn(0x7fb7d815e8d0 legacy=0x7fb7d815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:28.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.404+0000 7fb7ddfee640 1 -- 172.21.15.29:0/1924770485 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d80b3ca0 con 0x7fb7d815f3c0 2024-04-25T00:31:28.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.408+0000 7fb7ddfee640 1 --2- 172.21.15.29:0/1924770485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7d815f3c0 0x7fb7d80cc240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009e50 tx=0x7fb7d001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e542faf18bc9db9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:28.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.408+0000 7fb7cffff640 1 -- 172.21.15.29:0/1924770485 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00a55d0 con 0x7fb7d815f3c0 2024-04-25T00:31:28.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.408+0000 7fb7cffff640 1 -- 172.21.15.29:0/1924770485 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d00a5770 con 0x7fb7d815f3c0 2024-04-25T00:31:28.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.408+0000 7fb7cffff640 1 -- 172.21.15.29:0/1924770485 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00a5b40 con 0x7fb7d815f3c0 2024-04-25T00:31:28.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.408+0000 7fb7dfa51640 1 -- 172.21.15.29:0/1924770485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7d815f3c0 msgr2=0x7fb7d80cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:28.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.408+0000 7fb7dfa51640 1 --2- 172.21.15.29:0/1924770485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7d815f3c0 0x7fb7d80cc240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009e50 tx=0x7fb7d001c880 comp rx=0 tx=0).stop 2024-04-25T00:31:28.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.408+0000 7fb7dfa51640 1 -- 172.21.15.29:0/1924770485 shutdown_connections 2024-04-25T00:31:28.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.408+0000 7fb7dfa51640 1 --2- 172.21.15.29:0/1924770485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7d815f3c0 0x7fb7d80cc240 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:28.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.408+0000 7fb7dfa51640 1 -- 172.21.15.29:0/1924770485 >> 172.21.15.29:0/1924770485 conn(0x7fb7d80b1870 msgr2=0x7fb7d80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:28.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.408+0000 7fb7dfa51640 1 -- 172.21.15.29:0/1924770485 shutdown_connections 2024-04-25T00:31:28.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.408+0000 7fb7dfa51640 1 -- 172.21.15.29:0/1924770485 wait complete. 2024-04-25T00:31:28.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7dfa51640 1 Processor -- start 2024-04-25T00:31:28.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7dfa51640 1 -- start start 2024-04-25T00:31:28.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7dfa51640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7d80b4780 0x7fb7d81394c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:28.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7dfa51640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d815e8d0 0x7fb7d80cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:28.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7dfa51640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7d815f3c0 0x7fb7d8158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:28.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7dfa51640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d8168e00 con 0x7fb7d815f3c0 2024-04-25T00:31:28.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7dfa51640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d8168c80 con 0x7fb7d80b4780 2024-04-25T00:31:28.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7dfa51640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb7d8168f80 con 0x7fb7d815e8d0 2024-04-25T00:31:28.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7ddfee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7d815f3c0 0x7fb7d8158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:28.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7ddfee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7d815f3c0 0x7fb7d8158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51558/0 (socket says 172.21.15.29:51558) 2024-04-25T00:31:28.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7ddfee640 1 -- 172.21.15.29:0/2487337598 learned_addr learned my addr 172.21.15.29:0/2487337598 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:28.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7ddfee640 1 -- 172.21.15.29:0/2487337598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d815e8d0 msgr2=0x7fb7d80cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:28.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7ddfee640 1 --2- 172.21.15.29:0/2487337598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d815e8d0 0x7fb7d80cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:28.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7ddfee640 1 -- 172.21.15.29:0/2487337598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7d80b4780 msgr2=0x7fb7d81394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:28.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7dd7ed640 1 --2- 172.21.15.29:0/2487337598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7d80b4780 0x7fb7d81394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:28.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7ddfee640 1 --2- 172.21.15.29:0/2487337598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7d80b4780 0x7fb7d81394c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:28.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7ddfee640 1 -- 172.21.15.29:0/2487337598 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d813e770 con 0x7fb7d815f3c0 2024-04-25T00:31:28.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7ddfee640 1 --2- 172.21.15.29:0/2487337598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7d815f3c0 0x7fb7d8158a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00af040 tx=0x7fb7d0019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:28.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7cdffb640 1 -- 172.21.15.29:0/2487337598 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00bc020 con 0x7fb7d815f3c0 2024-04-25T00:31:28.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7cdffb640 1 -- 172.21.15.29:0/2487337598 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d00b8670 con 0x7fb7d815f3c0 2024-04-25T00:31:28.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7dfa51640 1 -- 172.21.15.29:0/2487337598 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d813ea50 con 0x7fb7d815f3c0 2024-04-25T00:31:28.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7cdffb640 1 -- 172.21.15.29:0/2487337598 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0009510 con 0x7fb7d815f3c0 2024-04-25T00:31:28.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7dfa51640 1 -- 172.21.15.29:0/2487337598 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d813f010 con 0x7fb7d815f3c0 2024-04-25T00:31:28.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7dfa51640 1 -- 172.21.15.29:0/2487337598 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d8069270 con 0x7fb7d815f3c0 2024-04-25T00:31:28.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.412+0000 7fb7cdffb640 1 -- 172.21.15.29:0/2487337598 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb7d00bf020 con 0x7fb7d815f3c0 2024-04-25T00:31:28.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.416+0000 7fb7cdffb640 1 --2- 172.21.15.29:0/2487337598 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7bc0678c0 0x7fb7bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:28.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.416+0000 7fb7cdffb640 1 -- 172.21.15.29:0/2487337598 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fb7d01209c0 con 0x7fb7d815f3c0 2024-04-25T00:31:28.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.416+0000 7fb7dd7ed640 1 --2- 172.21.15.29:0/2487337598 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7bc0678c0 0x7fb7bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:28.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.416+0000 7fb7dd7ed640 1 --2- 172.21.15.29:0/2487337598 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7bc0678c0 0x7fb7bc069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb7d81f8890 tx=0x7fb7d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:28.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.420+0000 7fb7cdffb640 1 -- 172.21.15.29:0/2487337598 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb7d0122e20 con 0x7fb7d815f3c0 2024-04-25T00:31:28.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:28.584+0000 7fb7dfa51640 1 -- 172.21.15.29:0/2487337598 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb7d8069270 con 0x7fb7d815f3c0 2024-04-25T00:31:29.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.252+0000 7fb7cdffb640 1 -- 172.21.15.29:0/2487337598 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7fb7d00c6030 con 0x7fb7d815f3c0 2024-04-25T00:31:29.255 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-04-25T00:31:29.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.256+0000 7fb7dfa51640 1 -- 172.21.15.29:0/2487337598 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7bc0678c0 msgr2=0x7fb7bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:29.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.256+0000 7fb7dfa51640 1 --2- 172.21.15.29:0/2487337598 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7bc0678c0 0x7fb7bc069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb7d81f8890 tx=0x7fb7d4009040 comp rx=0 tx=0).stop 2024-04-25T00:31:29.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.256+0000 7fb7dfa51640 1 -- 172.21.15.29:0/2487337598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7d815f3c0 msgr2=0x7fb7d8158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:29.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.256+0000 7fb7dfa51640 1 --2- 172.21.15.29:0/2487337598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7d815f3c0 0x7fb7d8158a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00af040 tx=0x7fb7d0019620 comp rx=0 tx=0).stop 2024-04-25T00:31:29.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.256+0000 7fb7dfa51640 1 -- 172.21.15.29:0/2487337598 shutdown_connections 2024-04-25T00:31:29.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.256+0000 7fb7dfa51640 1 --2- 172.21.15.29:0/2487337598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7d815f3c0 0x7fb7d8158a60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:29.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.256+0000 7fb7dfa51640 1 --2- 172.21.15.29:0/2487337598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7d815e8d0 0x7fb7d80cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:29.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.256+0000 7fb7dfa51640 1 --2- 172.21.15.29:0/2487337598 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7bc0678c0 0x7fb7bc069d80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:29.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.256+0000 7fb7dfa51640 1 --2- 172.21.15.29:0/2487337598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7d80b4780 0x7fb7d81394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:29.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.256+0000 7fb7dfa51640 1 -- 172.21.15.29:0/2487337598 >> 172.21.15.29:0/2487337598 conn(0x7fb7d80b1870 msgr2=0x7fb7d813de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:29.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.256+0000 7fb7dfa51640 1 -- 172.21.15.29:0/2487337598 shutdown_connections 2024-04-25T00:31:29.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.256+0000 7fb7dfa51640 1 -- 172.21.15.29:0/2487337598 wait complete. 2024-04-25T00:31:29.294 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:31:29.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.420+0000 7ffb21675640 1 Processor -- start 2024-04-25T00:31:29.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.420+0000 7ffb21675640 1 -- start start 2024-04-25T00:31:29.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb21675640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb1c15e200 0x7ffb1c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:29.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb21675640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffb1c0cf690 0x7ffb1c0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:29.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb21675640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb1c0cfc10 con 0x7ffb1c0d2a90 2024-04-25T00:31:29.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb21675640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ffb1c0d2ee0 con 0x7ffb1c0cf690 2024-04-25T00:31:29.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb21675640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffb1c0b38d0 con 0x7ffb1c15e200 2024-04-25T00:31:29.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb1affd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb1c15e200 0x7ffb1c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:29.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb1affd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb1c15e200 0x7ffb1c0d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:52796/0 (socket says 172.21.15.29:52796) 2024-04-25T00:31:29.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb1affd640 1 -- 172.21.15.29:0/3210473274 learned_addr learned my addr 172.21.15.29:0/3210473274 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:29.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb13fff640 1 --2- 172.21.15.29:0/3210473274 >> v2:172.21.15.157:3300/0 conn(0x7ffb1c0cf690 0x7ffb1c0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:29.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb13fff640 1 -- 172.21.15.29:0/3210473274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb1c15e200 msgr2=0x7ffb1c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:29.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb13fff640 1 --2- 172.21.15.29:0/3210473274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb1c15e200 0x7ffb1c0d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:29.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb13fff640 1 -- 172.21.15.29:0/3210473274 >> v1:172.21.15.157:6789/0 conn(0x7ffb1c0d2a90 legacy=0x7ffb1c0d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:29.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb13fff640 1 -- 172.21.15.29:0/3210473274 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c0cfef0 con 0x7ffb1c0cf690 2024-04-25T00:31:29.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb13fff640 1 --2- 172.21.15.29:0/3210473274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb1c0cf690 0x7ffb1c0d2550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffb040097c0 tx=0x7ffb0401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e3481e379c8686 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:29.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb1a7fc640 1 -- 172.21.15.29:0/3210473274 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb040a7070 con 0x7ffb1c0cf690 2024-04-25T00:31:29.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb1a7fc640 1 -- 172.21.15.29:0/3210473274 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb040b0070 con 0x7ffb1c0cf690 2024-04-25T00:31:29.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.424+0000 7ffb1a7fc640 1 -- 172.21.15.29:0/3210473274 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb040ab890 con 0x7ffb1c0cf690 2024-04-25T00:31:29.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 -- 172.21.15.29:0/3210473274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb1c0cf690 msgr2=0x7ffb1c0d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:29.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 --2- 172.21.15.29:0/3210473274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb1c0cf690 0x7ffb1c0d2550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffb040097c0 tx=0x7ffb0401c470 comp rx=0 tx=0).stop 2024-04-25T00:31:29.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 -- 172.21.15.29:0/3210473274 shutdown_connections 2024-04-25T00:31:29.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 --2- 172.21.15.29:0/3210473274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb1c0cf690 0x7ffb1c0d2550 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:29.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 --2- 172.21.15.29:0/3210473274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb1c15e200 0x7ffb1c0d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:29.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 -- 172.21.15.29:0/3210473274 >> 172.21.15.29:0/3210473274 conn(0x7ffb1c09d280 msgr2=0x7ffb1c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:29.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 -- 172.21.15.29:0/3210473274 shutdown_connections 2024-04-25T00:31:29.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 -- 172.21.15.29:0/3210473274 wait complete. 2024-04-25T00:31:29.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 Processor -- start 2024-04-25T00:31:29.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 -- start start 2024-04-25T00:31:29.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb1c0cf690 0x7ffb1c0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:29.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb1c0d2a90 0x7ffb1c0cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:29.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb1c15e200 0x7ffb1c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:29.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c15e760 con 0x7ffb1c15e200 2024-04-25T00:31:29.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c15e5e0 con 0x7ffb1c0cf690 2024-04-25T00:31:29.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb21675640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffb1c16ae30 con 0x7ffb1c0d2a90 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb1b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb1c15e200 0x7ffb1c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb1b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb1c15e200 0x7ffb1c0b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51566/0 (socket says 172.21.15.29:51566) 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb1b7fe640 1 -- 172.21.15.29:0/617733652 learned_addr learned my addr 172.21.15.29:0/617733652 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb13fff640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb1c0d2a90 0x7ffb1c0cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb1affd640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb1c0cf690 0x7ffb1c0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb1b7fe640 1 -- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb1c0d2a90 msgr2=0x7ffb1c0cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb1b7fe640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb1c0d2a90 0x7ffb1c0cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb1b7fe640 1 -- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb1c0cf690 msgr2=0x7ffb1c0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb1b7fe640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb1c0cf690 0x7ffb1c0cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.428+0000 7ffb1b7fe640 1 -- 172.21.15.29:0/617733652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c155930 con 0x7ffb1c15e200 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.432+0000 7ffb1b7fe640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb1c15e200 0x7ffb1c0b2170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400c730 tx=0x7ffb1400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.432+0000 7ffb137fe640 1 -- 172.21.15.29:0/617733652 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14016020 con 0x7ffb1c15e200 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.432+0000 7ffb137fe640 1 -- 172.21.15.29:0/617733652 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb14006d60 con 0x7ffb1c15e200 2024-04-25T00:31:29.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.432+0000 7ffb21675640 1 -- 172.21.15.29:0/617733652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1c155c10 con 0x7ffb1c15e200 2024-04-25T00:31:29.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.432+0000 7ffb137fe640 1 -- 172.21.15.29:0/617733652 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14003bc0 con 0x7ffb1c15e200 2024-04-25T00:31:29.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.432+0000 7ffb21675640 1 -- 172.21.15.29:0/617733652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1c1561d0 con 0x7ffb1c15e200 2024-04-25T00:31:29.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.432+0000 7ffb21675640 1 -- 172.21.15.29:0/617733652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1c069270 con 0x7ffb1c15e200 2024-04-25T00:31:29.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.432+0000 7ffb137fe640 1 -- 172.21.15.29:0/617733652 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffb140050d0 con 0x7ffb1c15e200 2024-04-25T00:31:29.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.432+0000 7ffb137fe640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffafc0678c0 0x7ffafc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:29.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.432+0000 7ffb137fe640 1 -- 172.21.15.29:0/617733652 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7ffb14085840 con 0x7ffb1c15e200 2024-04-25T00:31:29.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.432+0000 7ffb1affd640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffafc0678c0 0x7ffafc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:29.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.432+0000 7ffb1affd640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffafc0678c0 0x7ffafc069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0ca7d0 tx=0x7ffb0c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:29.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.436+0000 7ffb137fe640 1 -- 172.21.15.29:0/617733652 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffb14020020 con 0x7ffb1c15e200 2024-04-25T00:31:29.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:29.636+0000 7ffb21675640 1 -- 172.21.15.29:0/617733652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ffb1c0b1100 con 0x7ffb1c15e200 2024-04-25T00:31:30.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.256+0000 7ffb137fe640 1 -- 172.21.15.29:0/617733652 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7ffb140563d0 con 0x7ffb1c15e200 2024-04-25T00:31:30.258 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T00:31:30.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.260+0000 7ffb21675640 1 -- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffafc0678c0 msgr2=0x7ffafc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.260+0000 7ffb21675640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffafc0678c0 0x7ffafc069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0ca7d0 tx=0x7ffb0c008040 comp rx=0 tx=0).stop 2024-04-25T00:31:30.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.260+0000 7ffb21675640 1 -- 172.21.15.29:0/617733652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb1c15e200 msgr2=0x7ffb1c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.260+0000 7ffb21675640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb1c15e200 0x7ffb1c0b2170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400c730 tx=0x7ffb1400cb30 comp rx=0 tx=0).stop 2024-04-25T00:31:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.260+0000 7ffb21675640 1 -- 172.21.15.29:0/617733652 shutdown_connections 2024-04-25T00:31:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.260+0000 7ffb21675640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb1c15e200 0x7ffb1c0b2170 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.260+0000 7ffb21675640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb1c0d2a90 0x7ffb1c0cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.260+0000 7ffb21675640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffafc0678c0 0x7ffafc069d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.260+0000 7ffb21675640 1 --2- 172.21.15.29:0/617733652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb1c0cf690 0x7ffb1c0cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.260+0000 7ffb21675640 1 -- 172.21.15.29:0/617733652 >> 172.21.15.29:0/617733652 conn(0x7ffb1c09d280 msgr2=0x7ffb1c157380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.260+0000 7ffb21675640 1 -- 172.21.15.29:0/617733652 shutdown_connections 2024-04-25T00:31:30.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.260+0000 7ffb21675640 1 -- 172.21.15.29:0/617733652 wait complete. 2024-04-25T00:31:30.302 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:30.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.424+0000 7f59c3843640 1 Processor -- start 2024-04-25T00:31:30.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.424+0000 7f59c3843640 1 -- start start 2024-04-25T00:31:30.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c3843640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f59bc15e200 0x7f59bc0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:30.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c3843640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59bc0d2c60 0x7f59bc0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:30.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c3843640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59bc0cfc10 con 0x7f59bc0cf690 2024-04-25T00:31:30.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c3843640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f59bc0b38d0 con 0x7f59bc15e200 2024-04-25T00:31:30.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c3843640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f59bc0cfef0 con 0x7f59bc0d2c60 2024-04-25T00:31:30.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c15df640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f59bc15e200 0x7f59bc0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:30.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c15df640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f59bc15e200 0x7f59bc0d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51570/0 (socket says 172.21.15.29:51570) 2024-04-25T00:31:30.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c15df640 1 -- 172.21.15.29:0/2260577246 learned_addr learned my addr 172.21.15.29:0/2260577246 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:30.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c0dde640 1 --1- 172.21.15.29:0/2260577246 >> v1:172.21.15.29:6789/0 conn(0x7f59bc0cf690 0x7f59bc0d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43196/0 (socket says 172.21.15.29:43196) 2024-04-25T00:31:30.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c15df640 1 -- 172.21.15.29:0/2260577246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59bc0d2c60 msgr2=0x7f59bc0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:30.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c1de0640 1 --2- 172.21.15.29:0/2260577246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59bc0d2c60 0x7f59bc0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:30.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c15df640 1 --2- 172.21.15.29:0/2260577246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59bc0d2c60 0x7f59bc0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c15df640 1 -- 172.21.15.29:0/2260577246 >> v1:172.21.15.29:6789/0 conn(0x7f59bc0cf690 legacy=0x7f59bc0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59b3fff640 1 -- 172.21.15.29:0/2260577246 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 306014847 0 0) 0x7f59bc0cfc10 con 0x7f59bc0cf690 2024-04-25T00:31:30.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c15df640 1 -- 172.21.15.29:0/2260577246 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc0b3b20 con 0x7f59bc15e200 2024-04-25T00:31:30.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59c15df640 1 --2- 172.21.15.29:0/2260577246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59bc15e200 0x7f59bc0d47f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009e50 tx=0x7f59b401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac1c16641fb735b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:30.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59b3fff640 1 -- 172.21.15.29:0/2260577246 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40a58f0 con 0x7f59bc15e200 2024-04-25T00:31:30.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59b3fff640 1 -- 172.21.15.29:0/2260577246 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b40a5a90 con 0x7f59bc15e200 2024-04-25T00:31:30.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.428+0000 7f59b3fff640 1 -- 172.21.15.29:0/2260577246 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40a5e60 con 0x7f59bc15e200 2024-04-25T00:31:30.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 -- 172.21.15.29:0/2260577246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59bc15e200 msgr2=0x7f59bc0d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 --2- 172.21.15.29:0/2260577246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59bc15e200 0x7f59bc0d47f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009e50 tx=0x7f59b401ca00 comp rx=0 tx=0).stop 2024-04-25T00:31:30.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 -- 172.21.15.29:0/2260577246 shutdown_connections 2024-04-25T00:31:30.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 --2- 172.21.15.29:0/2260577246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59bc0d2c60 0x7f59bc0b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 --2- 172.21.15.29:0/2260577246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59bc15e200 0x7f59bc0d47f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 -- 172.21.15.29:0/2260577246 >> 172.21.15.29:0/2260577246 conn(0x7f59bc09d280 msgr2=0x7f59bc0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:30.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 -- 172.21.15.29:0/2260577246 shutdown_connections 2024-04-25T00:31:30.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 -- 172.21.15.29:0/2260577246 wait complete. 2024-04-25T00:31:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 Processor -- start 2024-04-25T00:31:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 -- start start 2024-04-25T00:31:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59bc0cf690 0x7f59bc0c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59bc0d2c60 0x7f59bc151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59bc15e200 0x7f59bc15e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc0b51f0 con 0x7f59bc0cf690 2024-04-25T00:31:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc0b5070 con 0x7f59bc0d2c60 2024-04-25T00:31:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f59bc0b5370 con 0x7f59bc15e200 2024-04-25T00:31:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c15df640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59bc0cf690 0x7f59bc0c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c15df640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59bc0cf690 0x7f59bc0c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51580/0 (socket says 172.21.15.29:51580) 2024-04-25T00:31:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c15df640 1 -- 172.21.15.29:0/2193340237 learned_addr learned my addr 172.21.15.29:0/2193340237 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c15df640 1 -- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59bc15e200 msgr2=0x7f59bc15e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c15df640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59bc15e200 0x7f59bc15e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c15df640 1 -- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59bc0d2c60 msgr2=0x7f59bc151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c0dde640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59bc0d2c60 0x7f59bc151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:30.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c15df640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59bc0d2c60 0x7f59bc151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c15df640 1 -- 172.21.15.29:0/2193340237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc0c84a0 con 0x7f59bc0cf690 2024-04-25T00:31:30.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c0dde640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59bc0d2c60 0x7f59bc151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:30.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c15df640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59bc0cf690 0x7f59bc0c7f60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f59b40af040 tx=0x7f59b40197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:30.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59b1ffb640 1 -- 172.21.15.29:0/2193340237 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40a58f0 con 0x7f59bc0cf690 2024-04-25T00:31:30.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59b1ffb640 1 -- 172.21.15.29:0/2193340237 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b40be070 con 0x7f59bc0cf690 2024-04-25T00:31:30.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 -- 172.21.15.29:0/2193340237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59bc160fa0 con 0x7f59bc0cf690 2024-04-25T00:31:30.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59b1ffb640 1 -- 172.21.15.29:0/2193340237 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4009510 con 0x7f59bc0cf690 2024-04-25T00:31:30.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.432+0000 7f59c3843640 1 -- 172.21.15.29:0/2193340237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59bc161560 con 0x7f59bc0cf690 2024-04-25T00:31:30.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.436+0000 7f59c3843640 1 -- 172.21.15.29:0/2193340237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59bc069270 con 0x7f59bc0cf690 2024-04-25T00:31:30.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.436+0000 7f59b1ffb640 1 -- 172.21.15.29:0/2193340237 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f59b40b86b0 con 0x7f59bc0cf690 2024-04-25T00:31:30.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.436+0000 7f59b1ffb640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f59a00678f0 0x7f59a0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:30.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.436+0000 7f59b1ffb640 1 -- 172.21.15.29:0/2193340237 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f59b4120d60 con 0x7f59bc0cf690 2024-04-25T00:31:30.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.436+0000 7f59c0dde640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f59a00678f0 0x7f59a0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.436+0000 7f59c0dde640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f59a00678f0 0x7f59a0069db0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f59bc1fa010 tx=0x7f59ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.440+0000 7f59b1ffb640 1 -- 172.21.15.29:0/2193340237 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f59b40c7020 con 0x7f59bc0cf690 2024-04-25T00:31:30.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.600+0000 7f59c3843640 1 -- 172.21.15.29:0/2193340237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59bc0d1d90 con 0x7f59bc0cf690 2024-04-25T00:31:30.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.604+0000 7f59b1ffb640 1 -- 172.21.15.29:0/2193340237 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16570 (secure 0 0 0) 0x7f59b4016620 con 0x7f59bc0cf690 2024-04-25T00:31:30.606 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:30.606 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":19,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:29.640247+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:30.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.604+0000 7f59c3843640 1 -- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f59a00678f0 msgr2=0x7f59a0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.604+0000 7f59c3843640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f59a00678f0 0x7f59a0069db0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f59bc1fa010 tx=0x7f59ac009040 comp rx=0 tx=0).stop 2024-04-25T00:31:30.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.604+0000 7f59c3843640 1 -- 172.21.15.29:0/2193340237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59bc0cf690 msgr2=0x7f59bc0c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.604+0000 7f59c3843640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59bc0cf690 0x7f59bc0c7f60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f59b40af040 tx=0x7f59b40197a0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.604+0000 7f59c3843640 1 -- 172.21.15.29:0/2193340237 shutdown_connections 2024-04-25T00:31:30.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.604+0000 7f59c3843640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59bc15e200 0x7f59bc15e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.604+0000 7f59c3843640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f59a00678f0 0x7f59a0069db0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.604+0000 7f59c3843640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59bc0d2c60 0x7f59bc151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.604+0000 7f59c3843640 1 --2- 172.21.15.29:0/2193340237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59bc0cf690 0x7f59bc0c7f60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.604+0000 7f59c3843640 1 -- 172.21.15.29:0/2193340237 >> 172.21.15.29:0/2193340237 conn(0x7f59bc09d280 msgr2=0x7f59bc0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:30.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.604+0000 7f59c3843640 1 -- 172.21.15.29:0/2193340237 shutdown_connections 2024-04-25T00:31:30.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.604+0000 7f59c3843640 1 -- 172.21.15.29:0/2193340237 wait complete. 2024-04-25T00:31:30.626 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f507a48e640 1 Processor -- start 2024-04-25T00:31:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f507a48e640 1 -- start start 2024-04-25T00:31:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f507a48e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50740b3140 0x7f507406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f507a48e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50740b4b10 0x7f50740b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f507a48e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50740b41e0 con 0x7f50740b3a60 2024-04-25T00:31:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f507a48e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f507406d960 con 0x7f50740b3140 2024-04-25T00:31:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f507a48e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f507406dc40 con 0x7f50740b4b10 2024-04-25T00:31:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f50737fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50740b4b10 0x7f50740b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f5073fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50740b3140 0x7f507406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f50737fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50740b4b10 0x7f50740b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:52834/0 (socket says 172.21.15.29:52834) 2024-04-25T00:31:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f50737fe640 1 -- 172.21.15.29:0/3525780142 learned_addr learned my addr 172.21.15.29:0/3525780142 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f5078a2b640 1 --1- 172.21.15.29:0/3525780142 >> v1:172.21.15.157:6789/0 conn(0x7f50740b3a60 0x7f507415c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:39848/0 (socket says 172.21.15.29:39848) 2024-04-25T00:31:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f5073fff640 1 -- 172.21.15.29:0/3525780142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50740b4b10 msgr2=0x7f50740b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f5073fff640 1 --2- 172.21.15.29:0/3525780142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50740b4b10 0x7f50740b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f5073fff640 1 -- 172.21.15.29:0/3525780142 >> v1:172.21.15.157:6789/0 conn(0x7f50740b3a60 legacy=0x7f507415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f5073fff640 1 -- 172.21.15.29:0/3525780142 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507406df20 con 0x7f50740b3140 2024-04-25T00:31:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.744+0000 7f50737fe640 1 --2- 172.21.15.29:0/3525780142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50740b4b10 0x7f50740b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:30.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f5073fff640 1 --2- 172.21.15.29:0/3525780142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740b3140 0x7f507406e440 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50640097c0 tx=0x7f506401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4a023c1a89920e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:30.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f5072ffd640 1 -- 172.21.15.29:0/3525780142 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640b0070 con 0x7f50740b3140 2024-04-25T00:31:30.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f5072ffd640 1 -- 172.21.15.29:0/3525780142 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50640ab600 con 0x7f50740b3140 2024-04-25T00:31:30.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f5072ffd640 1 -- 172.21.15.29:0/3525780142 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640ab9d0 con 0x7f50740b3140 2024-04-25T00:31:30.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f507a48e640 1 -- 172.21.15.29:0/3525780142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740b3140 msgr2=0x7f507406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f507a48e640 1 --2- 172.21.15.29:0/3525780142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740b3140 0x7f507406e440 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50640097c0 tx=0x7f506401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f507a48e640 1 -- 172.21.15.29:0/3525780142 shutdown_connections 2024-04-25T00:31:30.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f507a48e640 1 --2- 172.21.15.29:0/3525780142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50740b4b10 0x7f50740b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f507a48e640 1 --2- 172.21.15.29:0/3525780142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740b3140 0x7f507406e440 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f507a48e640 1 -- 172.21.15.29:0/3525780142 >> 172.21.15.29:0/3525780142 conn(0x7f50740b1870 msgr2=0x7f50740b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f507a48e640 1 -- 172.21.15.29:0/3525780142 shutdown_connections 2024-04-25T00:31:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f507a48e640 1 -- 172.21.15.29:0/3525780142 wait complete. 2024-04-25T00:31:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f507a48e640 1 Processor -- start 2024-04-25T00:31:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.748+0000 7f507a48e640 1 -- start start 2024-04-25T00:31:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f507a48e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740b3140 0x7f507413c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f507a48e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50740b3a60 0x7f50741404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f507a48e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b4b10 0x7f50741409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f507a48e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5074168b70 con 0x7f50740b4b10 2024-04-25T00:31:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f507a48e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50741689f0 con 0x7f50740b3140 2024-04-25T00:31:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f507a48e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5074168cf0 con 0x7f50740b3a60 2024-04-25T00:31:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5078a2b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b4b10 0x7f50741409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5078a2b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b4b10 0x7f50741409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51586/0 (socket says 172.21.15.29:51586) 2024-04-25T00:31:30.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5078a2b640 1 -- 172.21.15.29:0/4022454756 learned_addr learned my addr 172.21.15.29:0/4022454756 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f50737fe640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50740b3a60 0x7f50741404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5073fff640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740b3140 0x7f507413c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5078a2b640 1 -- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50740b3a60 msgr2=0x7f50741404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5078a2b640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50740b3a60 0x7f50741404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5078a2b640 1 -- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740b3140 msgr2=0x7f507413c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5078a2b640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740b3140 0x7f507413c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5078a2b640 1 -- 172.21.15.29:0/4022454756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507413c830 con 0x7f50740b4b10 2024-04-25T00:31:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5078a2b640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b4b10 0x7f50741409e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f506c00c7a0 tx=0x7f506c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5070ff9640 1 -- 172.21.15.29:0/4022454756 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c017070 con 0x7f50740b4b10 2024-04-25T00:31:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5070ff9640 1 -- 172.21.15.29:0/4022454756 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506c00a5e0 con 0x7f50740b4b10 2024-04-25T00:31:30.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f507a48e640 1 -- 172.21.15.29:0/4022454756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507415b050 con 0x7f50740b4b10 2024-04-25T00:31:30.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5070ff9640 1 -- 172.21.15.29:0/4022454756 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c00a990 con 0x7f50740b4b10 2024-04-25T00:31:30.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f507a48e640 1 -- 172.21.15.29:0/4022454756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507415b610 con 0x7f50740b4b10 2024-04-25T00:31:30.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f507a48e640 1 -- 172.21.15.29:0/4022454756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5074069270 con 0x7f50740b4b10 2024-04-25T00:31:30.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5070ff9640 1 -- 172.21.15.29:0/4022454756 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f506c01a070 con 0x7f50740b4b10 2024-04-25T00:31:30.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5070ff9640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f50600678c0 0x7f5060069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:30.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.752+0000 7f5070ff9640 1 -- 172.21.15.29:0/4022454756 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f506c0851c0 con 0x7f50740b4b10 2024-04-25T00:31:30.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.756+0000 7f5073fff640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f50600678c0 0x7f5060069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:30.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.756+0000 7f5073fff640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f50600678c0 0x7f5060069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f50640ad000 tx=0x7f50640a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:30.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.760+0000 7f5070ff9640 1 -- 172.21.15.29:0/4022454756 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f506c0556b0 con 0x7f50740b4b10 2024-04-25T00:31:30.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.920+0000 7f507a48e640 1 -- 172.21.15.29:0/4022454756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f507413d1e0 con 0x7f50740b4b10 2024-04-25T00:31:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.920+0000 7f5070ff9640 1 -- 172.21.15.29:0/4022454756 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16570 (secure 0 0 0) 0x7f506c0599b0 con 0x7f50740b4b10 2024-04-25T00:31:30.923 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:30.923 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":19,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:29.640247+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.920+0000 7f507a48e640 1 -- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f50600678c0 msgr2=0x7f5060069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.920+0000 7f507a48e640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f50600678c0 0x7f5060069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f50640ad000 tx=0x7f50640a6000 comp rx=0 tx=0).stop 2024-04-25T00:31:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.920+0000 7f507a48e640 1 -- 172.21.15.29:0/4022454756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b4b10 msgr2=0x7f50741409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.920+0000 7f507a48e640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b4b10 0x7f50741409e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f506c00c7a0 tx=0x7f506c00cc60 comp rx=0 tx=0).stop 2024-04-25T00:31:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.924+0000 7f507a48e640 1 -- 172.21.15.29:0/4022454756 shutdown_connections 2024-04-25T00:31:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.924+0000 7f507a48e640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b4b10 0x7f50741409e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.924+0000 7f507a48e640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50740b3a60 0x7f50741404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.924+0000 7f507a48e640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f50600678c0 0x7f5060069d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.924+0000 7f507a48e640 1 --2- 172.21.15.29:0/4022454756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740b3140 0x7f507413c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:30.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.924+0000 7f507a48e640 1 -- 172.21.15.29:0/4022454756 >> 172.21.15.29:0/4022454756 conn(0x7f50740b1870 msgr2=0x7f507415e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:30.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.924+0000 7f507a48e640 1 -- 172.21.15.29:0/4022454756 shutdown_connections 2024-04-25T00:31:30.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:30.924+0000 7f507a48e640 1 -- 172.21.15.29:0/4022454756 wait complete. 2024-04-25T00:31:30.941 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:31:31.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f4565c9e640 1 Processor -- start 2024-04-25T00:31:31.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f4565c9e640 1 -- start start 2024-04-25T00:31:31.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f4565c9e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f45600b3140 0x7f456006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:31.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f4565c9e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45600b4b10 0x7f45600b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:31.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f4565c9e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45600b41e0 con 0x7f45600b3a60 2024-04-25T00:31:31.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f4565c9e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f456006d960 con 0x7f45600b4b10 2024-04-25T00:31:31.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f4565c9e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f456006dc40 con 0x7f45600b3140 2024-04-25T00:31:31.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f455effd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45600b4b10 0x7f45600b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:31.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f455ffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f45600b3a60 0x7f456015c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43204/0 (socket says 172.21.15.29:43204) 2024-04-25T00:31:31.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f455ffff640 1 -- 172.21.15.29:0/6766534 learned_addr learned my addr 172.21.15.29:0/6766534 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:31.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f455f7fe640 1 --2- 172.21.15.29:0/6766534 >> v2:172.21.15.157:3300/0 conn(0x7f45600b3140 0x7f456006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:31.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f455e7fc640 1 -- 172.21.15.29:0/6766534 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 897804999 0 0) 0x7f45600b41e0 con 0x7f45600b3a60 2024-04-25T00:31:31.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f455effd640 1 -- 172.21.15.29:0/6766534 >> v2:172.21.15.157:3300/0 conn(0x7f45600b3140 msgr2=0x7f456006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:31.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f455effd640 1 --2- 172.21.15.29:0/6766534 >> v2:172.21.15.157:3300/0 conn(0x7f45600b3140 0x7f456006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f455effd640 1 -- 172.21.15.29:0/6766534 >> v1:172.21.15.29:6789/0 conn(0x7f45600b3a60 legacy=0x7f456015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:31.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.056+0000 7f455effd640 1 -- 172.21.15.29:0/6766534 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456006df20 con 0x7f45600b4b10 2024-04-25T00:31:31.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455effd640 1 --2- 172.21.15.29:0/6766534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45600b4b10 0x7f45600b3520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f45540097c0 tx=0x7f455401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27079363a89d3098 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:31.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455e7fc640 1 -- 172.21.15.29:0/6766534 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45540b0070 con 0x7f45600b4b10 2024-04-25T00:31:31.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455e7fc640 1 -- 172.21.15.29:0/6766534 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45540ab600 con 0x7f45600b4b10 2024-04-25T00:31:31.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455e7fc640 1 -- 172.21.15.29:0/6766534 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45540ab9d0 con 0x7f45600b4b10 2024-04-25T00:31:31.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 -- 172.21.15.29:0/6766534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45600b4b10 msgr2=0x7f45600b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:31.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 --2- 172.21.15.29:0/6766534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45600b4b10 0x7f45600b3520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f45540097c0 tx=0x7f455401c460 comp rx=0 tx=0).stop 2024-04-25T00:31:31.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 -- 172.21.15.29:0/6766534 shutdown_connections 2024-04-25T00:31:31.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 --2- 172.21.15.29:0/6766534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45600b4b10 0x7f45600b3520 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 --2- 172.21.15.29:0/6766534 >> v2:172.21.15.157:3300/0 conn(0x7f45600b3140 0x7f456006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 -- 172.21.15.29:0/6766534 >> 172.21.15.29:0/6766534 conn(0x7f45600b1870 msgr2=0x7f45600b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:31.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 -- 172.21.15.29:0/6766534 shutdown_connections 2024-04-25T00:31:31.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 -- 172.21.15.29:0/6766534 wait complete. 2024-04-25T00:31:31.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 Processor -- start 2024-04-25T00:31:31.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 -- start start 2024-04-25T00:31:31.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45600b3140 0x7f456013a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:31.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45600b3a60 0x7f4560159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45600b4b10 0x7f4560141190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4560168b70 con 0x7f45600b3140 2024-04-25T00:31:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f45601689f0 con 0x7f45600b4b10 2024-04-25T00:31:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f4565c9e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4560168cf0 con 0x7f45600b3a60 2024-04-25T00:31:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45600b3140 0x7f456013a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45600b3140 0x7f456013a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51604/0 (socket says 172.21.15.29:51604) 2024-04-25T00:31:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455f7fe640 1 -- 172.21.15.29:0/2893966856 learned_addr learned my addr 172.21.15.29:0/2893966856 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455ffff640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45600b4b10 0x7f4560141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455effd640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45600b3a60 0x7f4560159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455f7fe640 1 -- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45600b3a60 msgr2=0x7f4560159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455f7fe640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45600b3a60 0x7f4560159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455f7fe640 1 -- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45600b4b10 msgr2=0x7f4560141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455f7fe640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45600b4b10 0x7f4560141190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455f7fe640 1 -- 172.21.15.29:0/2893966856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456015a690 con 0x7f45600b3140 2024-04-25T00:31:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455ffff640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45600b4b10 0x7f4560141190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.060+0000 7f455f7fe640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45600b3140 0x7f456013a380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f455000e870 tx=0x7f455000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.064+0000 7f4547fff640 1 -- 172.21.15.29:0/2893966856 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550017070 con 0x7f45600b3140 2024-04-25T00:31:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.064+0000 7f4547fff640 1 -- 172.21.15.29:0/2893966856 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455000c940 con 0x7f45600b3140 2024-04-25T00:31:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.064+0000 7f4565c9e640 1 -- 172.21.15.29:0/2893966856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456015a930 con 0x7f45600b3140 2024-04-25T00:31:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.064+0000 7f4547fff640 1 -- 172.21.15.29:0/2893966856 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455000ccf0 con 0x7f45600b3140 2024-04-25T00:31:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.064+0000 7f4565c9e640 1 -- 172.21.15.29:0/2893966856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4560159450 con 0x7f45600b3140 2024-04-25T00:31:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.064+0000 7f4565c9e640 1 -- 172.21.15.29:0/2893966856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456013a920 con 0x7f45600b3140 2024-04-25T00:31:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.064+0000 7f4547fff640 1 -- 172.21.15.29:0/2893966856 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f455001a070 con 0x7f45600b3140 2024-04-25T00:31:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.064+0000 7f4547fff640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f454c0678c0 0x7f454c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.064+0000 7f4547fff640 1 -- 172.21.15.29:0/2893966856 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f4550085bb0 con 0x7f45600b3140 2024-04-25T00:31:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.064+0000 7f455effd640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f454c0678c0 0x7f454c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.064+0000 7f455effd640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f454c0678c0 0x7f454c069d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f45540ad000 tx=0x7f45540a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.068+0000 7f4547fff640 1 -- 172.21.15.29:0/2893966856 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f45500566f0 con 0x7f45600b3140 2024-04-25T00:31:31.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.260+0000 7f4565c9e640 1 -- 172.21.15.29:0/2893966856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4560069270 con 0x7f45600b3140 2024-04-25T00:31:31.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.276+0000 7f4547fff640 1 -- 172.21.15.29:0/2893966856 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7f4550055ff0 con 0x7f45600b3140 2024-04-25T00:31:31.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.280+0000 7f4565c9e640 1 -- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f454c0678c0 msgr2=0x7f454c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:31.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.280+0000 7f4565c9e640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f454c0678c0 0x7f454c069d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f45540ad000 tx=0x7f45540a6000 comp rx=0 tx=0).stop 2024-04-25T00:31:31.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.280+0000 7f4565c9e640 1 -- 172.21.15.29:0/2893966856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45600b3140 msgr2=0x7f456013a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:31.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.280+0000 7f4565c9e640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45600b3140 0x7f456013a380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f455000e870 tx=0x7f455000ec70 comp rx=0 tx=0).stop 2024-04-25T00:31:31.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.280+0000 7f4565c9e640 1 -- 172.21.15.29:0/2893966856 shutdown_connections 2024-04-25T00:31:31.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.280+0000 7f4565c9e640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f454c0678c0 0x7f454c069d80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.280+0000 7f4565c9e640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45600b4b10 0x7f4560141190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.280+0000 7f4565c9e640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45600b3a60 0x7f4560159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.280+0000 7f4565c9e640 1 --2- 172.21.15.29:0/2893966856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45600b3140 0x7f456013a380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.280+0000 7f4565c9e640 1 -- 172.21.15.29:0/2893966856 >> 172.21.15.29:0/2893966856 conn(0x7f45600b1870 msgr2=0x7f4560144aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:31.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.280+0000 7f4565c9e640 1 -- 172.21.15.29:0/2893966856 shutdown_connections 2024-04-25T00:31:31.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.280+0000 7f4565c9e640 1 -- 172.21.15.29:0/2893966856 wait complete. 2024-04-25T00:31:31.300 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:31:31.301 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:31:31.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81bde20640 1 Processor -- start 2024-04-25T00:31:31.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81bde20640 1 -- start start 2024-04-25T00:31:31.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81bde20640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f81b80b3c30 0x7f81b806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:31.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81bde20640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f81b80b4b10 0x7f81b80ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:31.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81bde20640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b80b41e0 con 0x7f81b80b3140 2024-04-25T00:31:31.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81bde20640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f81b806d960 con 0x7f81b80b3c30 2024-04-25T00:31:31.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81bde20640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f81b806dc40 con 0x7f81b80b4b10 2024-04-25T00:31:31.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81b6ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f81b80b3c30 0x7f81b806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:31.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81b6ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f81b80b3c30 0x7f81b806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51618/0 (socket says 172.21.15.29:51618) 2024-04-25T00:31:31.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81b6ffd640 1 -- 172.21.15.29:0/1309239006 learned_addr learned my addr 172.21.15.29:0/1309239006 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:31.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81b7fff640 1 --2- 172.21.15.29:0/1309239006 >> v2:172.21.15.157:3300/0 conn(0x7f81b80b4b10 0x7f81b80ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:31.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81b6ffd640 1 -- 172.21.15.29:0/1309239006 >> v2:172.21.15.157:3300/0 conn(0x7f81b80b4b10 msgr2=0x7f81b80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:31.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81b6ffd640 1 --2- 172.21.15.29:0/1309239006 >> v2:172.21.15.157:3300/0 conn(0x7f81b80b4b10 0x7f81b80ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81b6ffd640 1 -- 172.21.15.29:0/1309239006 >> v1:172.21.15.157:6789/0 conn(0x7f81b80b3140 legacy=0x7f81b80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:31.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.420+0000 7f81b6ffd640 1 -- 172.21.15.29:0/1309239006 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b806df20 con 0x7f81b80b3c30 2024-04-25T00:31:31.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81b6ffd640 1 --2- 172.21.15.29:0/1309239006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81b80b3c30 0x7f81b806e440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009870 tx=0x7f81ac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b820c9640096758 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:31.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81b67fc640 1 -- 172.21.15.29:0/1309239006 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0b0070 con 0x7f81b80b3c30 2024-04-25T00:31:31.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81b67fc640 1 -- 172.21.15.29:0/1309239006 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ac0ab600 con 0x7f81b80b3c30 2024-04-25T00:31:31.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81b67fc640 1 -- 172.21.15.29:0/1309239006 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0ab9d0 con 0x7f81b80b3c30 2024-04-25T00:31:31.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 -- 172.21.15.29:0/1309239006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81b80b3c30 msgr2=0x7f81b806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:31.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 --2- 172.21.15.29:0/1309239006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81b80b3c30 0x7f81b806e440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009870 tx=0x7f81ac01c520 comp rx=0 tx=0).stop 2024-04-25T00:31:31.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 -- 172.21.15.29:0/1309239006 shutdown_connections 2024-04-25T00:31:31.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 --2- 172.21.15.29:0/1309239006 >> v2:172.21.15.157:3300/0 conn(0x7f81b80b4b10 0x7f81b80ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 --2- 172.21.15.29:0/1309239006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81b80b3c30 0x7f81b806e440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 -- 172.21.15.29:0/1309239006 >> 172.21.15.29:0/1309239006 conn(0x7f81b80b1870 msgr2=0x7f81b80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:31.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 -- 172.21.15.29:0/1309239006 shutdown_connections 2024-04-25T00:31:31.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 -- 172.21.15.29:0/1309239006 wait complete. 2024-04-25T00:31:31.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 Processor -- start 2024-04-25T00:31:31.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 -- start start 2024-04-25T00:31:31.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81b80b3140 0x7f81b8154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:31.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b80b3c30 0x7f81b8155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:31.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b80b4b10 0x7f81b80cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:31.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81b8168d80 con 0x7f81b80b3140 2024-04-25T00:31:31.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81b8168c00 con 0x7f81b80b4b10 2024-04-25T00:31:31.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.424+0000 7f81bde20640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f81b8168f00 con 0x7f81b80b3c30 2024-04-25T00:31:31.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81b77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81b80b3140 0x7f81b8154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:31.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81b77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81b80b3140 0x7f81b8154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51622/0 (socket says 172.21.15.29:51622) 2024-04-25T00:31:31.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81b77fe640 1 -- 172.21.15.29:0/3012665766 learned_addr learned my addr 172.21.15.29:0/3012665766 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81b7fff640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b80b4b10 0x7f81b80cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81b6ffd640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b80b3c30 0x7f81b8155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81b77fe640 1 -- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b80b3c30 msgr2=0x7f81b8155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81b77fe640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b80b3c30 0x7f81b8155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81b77fe640 1 -- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b80b4b10 msgr2=0x7f81b80cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81b77fe640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b80b4b10 0x7f81b80cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81b77fe640 1 -- 172.21.15.29:0/3012665766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b81374c0 con 0x7f81b80b3140 2024-04-25T00:31:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81b77fe640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81b80b3140 0x7f81b8154920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c370 tx=0x7f81a800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f819ffff640 1 -- 172.21.15.29:0/3012665766 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a80052d0 con 0x7f81b80b3140 2024-04-25T00:31:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f819ffff640 1 -- 172.21.15.29:0/3012665766 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a8005470 con 0x7f81b80b3140 2024-04-25T00:31:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81bde20640 1 -- 172.21.15.29:0/3012665766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b81377a0 con 0x7f81b80b3140 2024-04-25T00:31:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f819ffff640 1 -- 172.21.15.29:0/3012665766 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8007260 con 0x7f81b80b3140 2024-04-25T00:31:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81bde20640 1 -- 172.21.15.29:0/3012665766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b8137d60 con 0x7f81b80b3140 2024-04-25T00:31:31.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81bde20640 1 -- 172.21.15.29:0/3012665766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b8069270 con 0x7f81b80b3140 2024-04-25T00:31:31.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f819ffff640 1 -- 172.21.15.29:0/3012665766 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f81a8007400 con 0x7f81b80b3140 2024-04-25T00:31:31.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f819ffff640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81a40678f0 0x7f81a4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:31.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f819ffff640 1 -- 172.21.15.29:0/3012665766 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f81a8085050 con 0x7f81b80b3140 2024-04-25T00:31:31.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81b6ffd640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81a40678f0 0x7f81a4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:31.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.428+0000 7f81b6ffd640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81a40678f0 0x7f81a4069db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0ad000 tx=0x7f81ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:31.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.432+0000 7f819ffff640 1 -- 172.21.15.29:0/3012665766 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f81a8055c20 con 0x7f81b80b3140 2024-04-25T00:31:31.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:31.628+0000 7f81bde20640 1 -- 172.21.15.29:0/3012665766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f81b80d07a0 con 0x7f81b80b3140 2024-04-25T00:31:32.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.292+0000 7f819ffff640 1 -- 172.21.15.29:0/3012665766 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7f81a8055520 con 0x7f81b80b3140 2024-04-25T00:31:32.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.296+0000 7f81bde20640 1 -- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81a40678f0 msgr2=0x7f81a4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.296+0000 7f81bde20640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81a40678f0 0x7f81a4069db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0ad000 tx=0x7f81ac008040 comp rx=0 tx=0).stop 2024-04-25T00:31:32.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.296+0000 7f81bde20640 1 -- 172.21.15.29:0/3012665766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81b80b3140 msgr2=0x7f81b8154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.296+0000 7f81bde20640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81b80b3140 0x7f81b8154920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c370 tx=0x7f81a800c830 comp rx=0 tx=0).stop 2024-04-25T00:31:32.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.296+0000 7f81bde20640 1 -- 172.21.15.29:0/3012665766 shutdown_connections 2024-04-25T00:31:32.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.296+0000 7f81bde20640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81a40678f0 0x7f81a4069db0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.296+0000 7f81bde20640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b80b4b10 0x7f81b80cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.296+0000 7f81bde20640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81b80b3c30 0x7f81b8155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.296+0000 7f81bde20640 1 --2- 172.21.15.29:0/3012665766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81b80b3140 0x7f81b8154920 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.296+0000 7f81bde20640 1 -- 172.21.15.29:0/3012665766 >> 172.21.15.29:0/3012665766 conn(0x7f81b80b1870 msgr2=0x7f81b80b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:32.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.296+0000 7f81bde20640 1 -- 172.21.15.29:0/3012665766 shutdown_connections 2024-04-25T00:31:32.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.296+0000 7f81bde20640 1 -- 172.21.15.29:0/3012665766 wait complete. 2024-04-25T00:31:32.320 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:31:32.320 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:31:32.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.444+0000 7f995981d640 1 Processor -- start 2024-04-25T00:31:32.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f995981d640 1 -- start start 2024-04-25T00:31:32.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f995981d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9954071cc0 0x7f99540b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f995981d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f995415a950 con 0x7f995415a570 2024-04-25T00:31:32.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f995981d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99540b4130 con 0x7f99540b53a0 2024-04-25T00:31:32.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f995981d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f99540b3ee0 con 0x7f9954071cc0 2024-04-25T00:31:32.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9952ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f995415a570 0x7f99540715b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43212/0 (socket says 172.21.15.29:43212) 2024-04-25T00:31:32.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9952ffd640 1 -- 172.21.15.29:0/2450188908 learned_addr learned my addr 172.21.15.29:0/2450188908 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:32.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f99527fc640 1 --2- 172.21.15.29:0/2450188908 >> v2:172.21.15.157:3300/0 conn(0x7f9954071cc0 0x7f99540b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3967102752 0 0) 0x7f995415a950 con 0x7f995415a570 2024-04-25T00:31:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f993c003530 con 0x7f995415a570 2024-04-25T00:31:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 526949864 0 0) 0x7f993c003530 con 0x7f995415a570 2024-04-25T00:31:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 >> v2:172.21.15.157:3300/0 conn(0x7f9954071cc0 msgr2=0x7f99540b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 --2- 172.21.15.29:0/2450188908 >> v2:172.21.15.157:3300/0 conn(0x7f9954071cc0 0x7f99540b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 >> v1:172.21.15.157:6789/0 conn(0x7f99540b53a0 legacy=0x7f99540ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99540b4410 con 0x7f995415a570 2024-04-25T00:31:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f994c001a00 con 0x7f995415a570 2024-04-25T00:31:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f99527fc640 1 --2- 172.21.15.29:0/2450188908 >> v2:172.21.15.157:3300/0 conn(0x7f9954071cc0 0x7f99540b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 >> v1:172.21.15.29:6789/0 conn(0x7f995415a570 legacy=0x7f99540715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 --2- 172.21.15.29:0/2450188908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f993c0039c0 0x7f993c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 --2- 172.21.15.29:0/2450188908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f993c004810 0x7f993c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 --2- 172.21.15.29:0/2450188908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0051d0 0x7f993c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f99540b3ee0 con 0x7f993c0039c0 2024-04-25T00:31:32.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f993c00c070 con 0x7f993c0051d0 2024-04-25T00:31:32.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f993c00c270 con 0x7f993c004810 2024-04-25T00:31:32.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9952ffd640 1 --2- 172.21.15.29:0/2450188908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f993c0039c0 0x7f993c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f99527fc640 1 --2- 172.21.15.29:0/2450188908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f993c004810 0x7f993c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9952ffd640 1 -- 172.21.15.29:0/2450188908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f993c004810 msgr2=0x7f993c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:32.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9952ffd640 1 --2- 172.21.15.29:0/2450188908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f993c004810 0x7f993c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f99537fe640 1 --2- 172.21.15.29:0/2450188908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0051d0 0x7f993c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9952ffd640 1 -- 172.21.15.29:0/2450188908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0051d0 msgr2=0x7f993c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:32.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9952ffd640 1 --2- 172.21.15.29:0/2450188908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0051d0 0x7f993c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f9952ffd640 1 -- 172.21.15.29:0/2450188908 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c003530 con 0x7f993c0039c0 2024-04-25T00:31:32.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.448+0000 7f99537fe640 1 -- 172.21.15.29:0/2450188908 reap_dead start 2024-04-25T00:31:32.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f9952ffd640 1 --2- 172.21.15.29:0/2450188908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f993c0039c0 0x7f993c003de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f994c00a840 tx=0x7f994c01d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04ea8a4b70baf16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c00a460 con 0x7f993c0039c0 2024-04-25T00:31:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f994c0ad650 con 0x7f993c0039c0 2024-04-25T00:31:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f9951ffb640 1 -- 172.21.15.29:0/2450188908 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c0ada80 con 0x7f993c0039c0 2024-04-25T00:31:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 -- 172.21.15.29:0/2450188908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f993c0039c0 msgr2=0x7f993c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 --2- 172.21.15.29:0/2450188908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f993c0039c0 0x7f993c003de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f994c00a840 tx=0x7f994c01d360 comp rx=0 tx=0).stop 2024-04-25T00:31:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 -- 172.21.15.29:0/2450188908 shutdown_connections 2024-04-25T00:31:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 --2- 172.21.15.29:0/2450188908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f993c0039c0 0x7f993c003de0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 -- 172.21.15.29:0/2450188908 >> 172.21.15.29:0/2450188908 conn(0x7f9954075030 msgr2=0x7f99540b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 -- 172.21.15.29:0/2450188908 shutdown_connections 2024-04-25T00:31:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 -- 172.21.15.29:0/2450188908 wait complete. 2024-04-25T00:31:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 Processor -- start 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 -- start start 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0039c0 0x7f995414a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f995414ac00 0x7f9954139fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f995413a510 0x7f9954137f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f995406d4b0 con 0x7f995414ac00 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f995406d330 con 0x7f993c0039c0 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f995981d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f995406d630 con 0x7f995413a510 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f99527fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f995414ac00 0x7f9954139fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f99527fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f995414ac00 0x7f9954139fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51648/0 (socket says 172.21.15.29:51648) 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.452+0000 7f99527fc640 1 -- 172.21.15.29:0/2053806298 learned_addr learned my addr 172.21.15.29:0/2053806298 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f9952ffd640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0039c0 0x7f995414a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f99537fe640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f995413a510 0x7f9954137f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f99527fc640 1 -- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f995413a510 msgr2=0x7f9954137f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f99527fc640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f995413a510 0x7f9954137f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f99527fc640 1 -- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0039c0 msgr2=0x7f995414a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f99527fc640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0039c0 0x7f995414a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f99527fc640 1 -- 172.21.15.29:0/2053806298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995413aac0 con 0x7f995414ac00 2024-04-25T00:31:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f99537fe640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f995413a510 0x7f9954137f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f99527fc640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f995414ac00 0x7f9954139fd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9948011930 tx=0x7f9948011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f993bfff640 1 -- 172.21.15.29:0/2053806298 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994800fbd0 con 0x7f995414ac00 2024-04-25T00:31:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f993bfff640 1 -- 172.21.15.29:0/2053806298 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f994800fd70 con 0x7f995414ac00 2024-04-25T00:31:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f995981d640 1 -- 172.21.15.29:0/2053806298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995413ada0 con 0x7f995414ac00 2024-04-25T00:31:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f993bfff640 1 -- 172.21.15.29:0/2053806298 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948015660 con 0x7f995414ac00 2024-04-25T00:31:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f995981d640 1 -- 172.21.15.29:0/2053806298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f995414a1a0 con 0x7f995414ac00 2024-04-25T00:31:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f995981d640 1 -- 172.21.15.29:0/2053806298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9954069270 con 0x7f995414ac00 2024-04-25T00:31:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f993bfff640 1 -- 172.21.15.29:0/2053806298 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9948015800 con 0x7f995414ac00 2024-04-25T00:31:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f993bfff640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f99400678c0 0x7f9940069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f993bfff640 1 -- 172.21.15.29:0/2053806298 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9948086740 con 0x7f995414ac00 2024-04-25T00:31:32.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f9952ffd640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f99400678c0 0x7f9940069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.456+0000 7f9952ffd640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f99400678c0 0x7f9940069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f994c0ad000 tx=0x7f994c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:32.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.460+0000 7f993bfff640 1 -- 172.21.15.29:0/2053806298 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9948019030 con 0x7f995414ac00 2024-04-25T00:31:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.656+0000 7f995981d640 1 -- 172.21.15.29:0/2053806298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99540cf150 con 0x7f995414ac00 2024-04-25T00:31:32.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.660+0000 7f993bfff640 1 -- 172.21.15.29:0/2053806298 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f9948057310 con 0x7f995414ac00 2024-04-25T00:31:32.661 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:32.661 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1532858457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532858457},{"type":"v1","addr":"172.21.15.29:6833","nonce":1532858457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/718997473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":718997473},{"type":"v1","addr":"172.21.15.157:6835","nonce":718997473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1617208213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1617208213},{"type":"v1","addr":"172.21.15.157:6837","nonce":1617208213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:31:29.640568+0000","modified":"2024-04-25T00:31:31.633585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4252":{"gid":4252,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1924706890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1924706890},{"type":"v1","addr":"172.21.15.29:6835","nonce":1924706890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4252,"qdb_cluster":[4252]},"id":1}]} 2024-04-25T00:31:32.661 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 6 2024-04-25T00:31:32.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.660+0000 7f995981d640 1 -- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f99400678c0 msgr2=0x7f9940069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.660+0000 7f995981d640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f99400678c0 0x7f9940069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f994c0ad000 tx=0x7f994c009040 comp rx=0 tx=0).stop 2024-04-25T00:31:32.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.660+0000 7f995981d640 1 -- 172.21.15.29:0/2053806298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f995414ac00 msgr2=0x7f9954139fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.660+0000 7f995981d640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f995414ac00 0x7f9954139fd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9948011930 tx=0x7f9948011df0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.660+0000 7f995981d640 1 -- 172.21.15.29:0/2053806298 shutdown_connections 2024-04-25T00:31:32.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.660+0000 7f995981d640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f995413a510 0x7f9954137f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.660+0000 7f995981d640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f995414ac00 0x7f9954139fd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.660+0000 7f995981d640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f99400678c0 0x7f9940069d80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.660+0000 7f995981d640 1 --2- 172.21.15.29:0/2053806298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0039c0 0x7f995414a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.660+0000 7f995981d640 1 -- 172.21.15.29:0/2053806298 >> 172.21.15.29:0/2053806298 conn(0x7f9954075030 msgr2=0x7f99540b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:32.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.660+0000 7f995981d640 1 -- 172.21.15.29:0/2053806298 shutdown_connections 2024-04-25T00:31:32.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.660+0000 7f995981d640 1 -- 172.21.15.29:0/2053806298 wait complete. 2024-04-25T00:31:32.680 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:32.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58f757640 1 Processor -- start 2024-04-25T00:31:32.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58f757640 1 -- start start 2024-04-25T00:31:32.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58f757640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa58815e200 0x7fa5880d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58f757640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5880d2c60 0x7fa5880b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58f757640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5880cfc10 con 0x7fa5880cf690 2024-04-25T00:31:32.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58f757640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa5880b38d0 con 0x7fa58815e200 2024-04-25T00:31:32.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58f757640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa5880cfef0 con 0x7fa5880d2c60 2024-04-25T00:31:32.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58ccf2640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa5880cf690 0x7fa5880d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43218/0 (socket says 172.21.15.29:43218) 2024-04-25T00:31:32.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58ccf2640 1 -- 172.21.15.29:0/1045359074 learned_addr learned my addr 172.21.15.29:0/1045359074 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:32.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58d4f3640 1 --2- 172.21.15.29:0/1045359074 >> v2:172.21.15.157:3300/0 conn(0x7fa58815e200 0x7fa5880d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58dcf4640 1 --2- 172.21.15.29:0/1045359074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5880d2c60 0x7fa5880b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3157905437 0 0) 0x7fa5880cfc10 con 0x7fa5880cf690 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa570003530 con 0x7fa5880cf690 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2538314648 0 0) 0x7fa570003530 con 0x7fa5880cf690 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5880d2c60 msgr2=0x7fa5880b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 --2- 172.21.15.29:0/1045359074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5880d2c60 0x7fa5880b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 >> v2:172.21.15.157:3300/0 conn(0x7fa58815e200 msgr2=0x7fa5880d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 --2- 172.21.15.29:0/1045359074 >> v2:172.21.15.157:3300/0 conn(0x7fa58815e200 0x7fa5880d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5880b3b20 con 0x7fa5880cf690 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fa57c001a20 con 0x7fa5880cf690 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 >> v1:172.21.15.29:6789/0 conn(0x7fa5880cf690 legacy=0x7fa5880d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 --2- 172.21.15.29:0/1045359074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5700039c0 0x7fa570003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 --2- 172.21.15.29:0/1045359074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5880d2c60 0x7fa570004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 --2- 172.21.15.29:0/1045359074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa570004d50 0x7fa5700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa5880b38d0 con 0x7fa570004d50 2024-04-25T00:31:32.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5880cfef0 con 0x7fa5700039c0 2024-04-25T00:31:32.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa57000b9d0 con 0x7fa5880d2c60 2024-04-25T00:31:32.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58dcf4640 1 --2- 172.21.15.29:0/1045359074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa570004d50 0x7fa5700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58ccf2640 1 --2- 172.21.15.29:0/1045359074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5880d2c60 0x7fa570004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58d4f3640 1 --2- 172.21.15.29:0/1045359074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5700039c0 0x7fa570003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58dcf4640 1 -- 172.21.15.29:0/1045359074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5880d2c60 msgr2=0x7fa570004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:32.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58dcf4640 1 --2- 172.21.15.29:0/1045359074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5880d2c60 0x7fa570004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58dcf4640 1 -- 172.21.15.29:0/1045359074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5700039c0 msgr2=0x7fa570003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:32.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58dcf4640 1 --2- 172.21.15.29:0/1045359074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5700039c0 0x7fa570003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.804+0000 7fa58dcf4640 1 -- 172.21.15.29:0/1045359074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa570003530 con 0x7fa570004d50 2024-04-25T00:31:32.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58dcf4640 1 --2- 172.21.15.29:0/1045359074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa570004d50 0x7fa5700051d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa578009ae0 tx=0x7fa57801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cac593b1b7cb3c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5780b2070 con 0x7fa570004d50 2024-04-25T00:31:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5780a5620 con 0x7fa570004d50 2024-04-25T00:31:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa587fff640 1 -- 172.21.15.29:0/1045359074 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5780096b0 con 0x7fa570004d50 2024-04-25T00:31:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 -- 172.21.15.29:0/1045359074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa570004d50 msgr2=0x7fa5700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 --2- 172.21.15.29:0/1045359074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa570004d50 0x7fa5700051d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa578009ae0 tx=0x7fa57801ca30 comp rx=0 tx=0).stop 2024-04-25T00:31:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58dcf4640 1 -- 172.21.15.29:0/1045359074 reap_dead start 2024-04-25T00:31:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 -- 172.21.15.29:0/1045359074 shutdown_connections 2024-04-25T00:31:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 -- 172.21.15.29:0/1045359074 >> 172.21.15.29:0/1045359074 conn(0x7fa58809d280 msgr2=0x7fa5880d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 -- 172.21.15.29:0/1045359074 shutdown_connections 2024-04-25T00:31:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 -- 172.21.15.29:0/1045359074 wait complete. 2024-04-25T00:31:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 Processor -- start 2024-04-25T00:31:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 -- start start 2024-04-25T00:31:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa58814aa90 0x7fa58814de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa58814ae70 0x7fa5880d0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5880d0670 0x7fa5881608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa58815e5e0 con 0x7fa5880d0670 2024-04-25T00:31:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5880d4d30 con 0x7fa58814ae70 2024-04-25T00:31:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58f757640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa58815e760 con 0x7fa58814aa90 2024-04-25T00:31:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58dcf4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5880d0670 0x7fa5881608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58dcf4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5880d0670 0x7fa5881608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51666/0 (socket says 172.21.15.29:51666) 2024-04-25T00:31:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58dcf4640 1 -- 172.21.15.29:0/2927489142 learned_addr learned my addr 172.21.15.29:0/2927489142 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58d4f3640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa58814aa90 0x7fa58814de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.808+0000 7fa58ccf2640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa58814ae70 0x7fa5880d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa58dcf4640 1 -- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa58814aa90 msgr2=0x7fa58814de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa58dcf4640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa58814aa90 0x7fa58814de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa58dcf4640 1 -- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa58814ae70 msgr2=0x7fa5880d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa58dcf4640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa58814ae70 0x7fa5880d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa58dcf4640 1 -- 172.21.15.29:0/2927489142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58814e380 con 0x7fa5880d0670 2024-04-25T00:31:32.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa58d4f3640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa58814aa90 0x7fa58814de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:32.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa58dcf4640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5880d0670 0x7fa5881608f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa578009ae0 tx=0x7fa570005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:32.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa585ffb640 1 -- 172.21.15.29:0/2927489142 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5780b2040 con 0x7fa5880d0670 2024-04-25T00:31:32.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa585ffb640 1 -- 172.21.15.29:0/2927489142 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa578000e50 con 0x7fa5880d0670 2024-04-25T00:31:32.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa58f757640 1 -- 172.21.15.29:0/2927489142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5880c9de0 con 0x7fa5880d0670 2024-04-25T00:31:32.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa585ffb640 1 -- 172.21.15.29:0/2927489142 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5780096b0 con 0x7fa5880d0670 2024-04-25T00:31:32.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa58f757640 1 -- 172.21.15.29:0/2927489142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5880ca3a0 con 0x7fa5880d0670 2024-04-25T00:31:32.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa58f757640 1 -- 172.21.15.29:0/2927489142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa588069270 con 0x7fa5880d0670 2024-04-25T00:31:32.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa585ffb640 1 -- 172.21.15.29:0/2927489142 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa5780bd030 con 0x7fa5880d0670 2024-04-25T00:31:32.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa585ffb640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa5740678c0 0x7fa574069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:32.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa585ffb640 1 -- 172.21.15.29:0/2927489142 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa57811ecd0 con 0x7fa5880d0670 2024-04-25T00:31:32.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa58d4f3640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa5740678c0 0x7fa574069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:32.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.812+0000 7fa58d4f3640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa5740678c0 0x7fa574069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa580009b70 tx=0x7fa580008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:32.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.816+0000 7fa585ffb640 1 -- 172.21.15.29:0/2927489142 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa58814e380 con 0x7fa5880d0670 2024-04-25T00:31:32.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.984+0000 7fa58f757640 1 -- 172.21.15.29:0/2927489142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5880b4750 con 0x7fa5880d0670 2024-04-25T00:31:32.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.988+0000 7fa585ffb640 1 -- 172.21.15.29:0/2927489142 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fa578016780 con 0x7fa5880d0670 2024-04-25T00:31:32.990 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:32.990 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:32.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.988+0000 7fa58f757640 1 -- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa5740678c0 msgr2=0x7fa574069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.988+0000 7fa58f757640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa5740678c0 0x7fa574069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa580009b70 tx=0x7fa580008040 comp rx=0 tx=0).stop 2024-04-25T00:31:32.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.988+0000 7fa58f757640 1 -- 172.21.15.29:0/2927489142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5880d0670 msgr2=0x7fa5881608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:32.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.988+0000 7fa58f757640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5880d0670 0x7fa5881608f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa578009ae0 tx=0x7fa570005840 comp rx=0 tx=0).stop 2024-04-25T00:31:32.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.988+0000 7fa58f757640 1 -- 172.21.15.29:0/2927489142 shutdown_connections 2024-04-25T00:31:32.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.988+0000 7fa58f757640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5880d0670 0x7fa5881608f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.988+0000 7fa58f757640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa5740678c0 0x7fa574069d80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.988+0000 7fa58f757640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa58814ae70 0x7fa5880d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.988+0000 7fa58f757640 1 --2- 172.21.15.29:0/2927489142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa58814aa90 0x7fa58814de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:32.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.988+0000 7fa58f757640 1 -- 172.21.15.29:0/2927489142 >> 172.21.15.29:0/2927489142 conn(0x7fa58809d280 msgr2=0x7fa5880d2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:32.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.988+0000 7fa58f757640 1 -- 172.21.15.29:0/2927489142 shutdown_connections 2024-04-25T00:31:32.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:32.992+0000 7fa58f757640 1 -- 172.21.15.29:0/2927489142 wait complete. 2024-04-25T00:31:33.011 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:31:33.011 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:31:33.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2dfbf65640 1 Processor -- start 2024-04-25T00:31:33.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2dfbf65640 1 -- start start 2024-04-25T00:31:33.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2dfbf65640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2df415e4e0 0x7f2df40d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2dfbf65640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df415e8c0 0x7f2df40b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2dfbf65640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df40cfe30 con 0x7f2df40d4bf0 2024-04-25T00:31:33.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2dfbf65640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2df40b6f80 con 0x7f2df415e4e0 2024-04-25T00:31:33.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2dfbf65640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2df40b7260 con 0x7f2df415e8c0 2024-04-25T00:31:33.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df9500640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2df40d4bf0 0x7f2df40d23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43220/0 (socket says 172.21.15.29:43220) 2024-04-25T00:31:33.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df9500640 1 -- 172.21.15.29:0/2372434204 learned_addr learned my addr 172.21.15.29:0/2372434204 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:33.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df9d01640 1 --2- 172.21.15.29:0/2372434204 >> v2:172.21.15.157:3300/0 conn(0x7f2df415e4e0 0x7f2df40d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2dfa502640 1 --2- 172.21.15.29:0/2372434204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df415e8c0 0x7f2df40b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2068386472 0 0) 0x7f2df40cfe30 con 0x7f2df40d4bf0 2024-04-25T00:31:33.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2de0003530 con 0x7f2df40d4bf0 2024-04-25T00:31:33.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1561806357 0 0) 0x7f2de0003530 con 0x7f2df40d4bf0 2024-04-25T00:31:33.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df415e8c0 msgr2=0x7f2df40b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:33.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 --2- 172.21.15.29:0/2372434204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df415e8c0 0x7f2df40b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 >> v2:172.21.15.157:3300/0 conn(0x7f2df415e4e0 msgr2=0x7f2df40d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:33.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 --2- 172.21.15.29:0/2372434204 >> v2:172.21.15.157:3300/0 conn(0x7f2df415e4e0 0x7f2df40d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df40ced50 con 0x7f2df40d4bf0 2024-04-25T00:31:33.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2de4001680 con 0x7f2df40d4bf0 2024-04-25T00:31:33.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 >> v1:172.21.15.29:6789/0 conn(0x7f2df40d4bf0 legacy=0x7f2df40d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 --2- 172.21.15.29:0/2372434204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2de00039c0 0x7f2de0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 --2- 172.21.15.29:0/2372434204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df415e8c0 0x7f2de0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 --2- 172.21.15.29:0/2372434204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de0004d50 0x7f2de00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2df40b6f80 con 0x7f2de00039c0 2024-04-25T00:31:33.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2df40b7260 con 0x7f2de0004d50 2024-04-25T00:31:33.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2de000b9d0 con 0x7f2df415e8c0 2024-04-25T00:31:33.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df9d01640 1 --2- 172.21.15.29:0/2372434204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2de00039c0 0x7f2de0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2dfa502640 1 --2- 172.21.15.29:0/2372434204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de0004d50 0x7f2de00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df9500640 1 --2- 172.21.15.29:0/2372434204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df415e8c0 0x7f2de0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df9d01640 1 -- 172.21.15.29:0/2372434204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df415e8c0 msgr2=0x7f2de0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:33.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df9d01640 1 --2- 172.21.15.29:0/2372434204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df415e8c0 0x7f2de0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df9d01640 1 -- 172.21.15.29:0/2372434204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de0004d50 msgr2=0x7f2de00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:33.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df9d01640 1 --2- 172.21.15.29:0/2372434204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de0004d50 0x7f2de00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.132+0000 7f2df9d01640 1 -- 172.21.15.29:0/2372434204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de0003530 con 0x7f2de00039c0 2024-04-25T00:31:33.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.136+0000 7f2df9d01640 1 --2- 172.21.15.29:0/2372434204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2de00039c0 0x7f2de0003da0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009ae0 tx=0x7f2df001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ecf0d175bb25fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:33.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.136+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df00b2070 con 0x7f2de00039c0 2024-04-25T00:31:33.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.136+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df00a56a0 con 0x7f2de00039c0 2024-04-25T00:31:33.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.136+0000 7f2df8cff640 1 -- 172.21.15.29:0/2372434204 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df00096b0 con 0x7f2de00039c0 2024-04-25T00:31:33.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.136+0000 7f2dfbf65640 1 -- 172.21.15.29:0/2372434204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2de00039c0 msgr2=0x7f2de0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.136+0000 7f2dfbf65640 1 --2- 172.21.15.29:0/2372434204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2de00039c0 0x7f2de0003da0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009ae0 tx=0x7f2df001cab0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.136+0000 7f2dfa502640 1 -- 172.21.15.29:0/2372434204 reap_dead start 2024-04-25T00:31:33.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.136+0000 7f2dfbf65640 1 -- 172.21.15.29:0/2372434204 shutdown_connections 2024-04-25T00:31:33.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.136+0000 7f2dfbf65640 1 -- 172.21.15.29:0/2372434204 >> 172.21.15.29:0/2372434204 conn(0x7f2df40d3fb0 msgr2=0x7f2df40d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:33.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.136+0000 7f2dfbf65640 1 -- 172.21.15.29:0/2372434204 shutdown_connections 2024-04-25T00:31:33.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.136+0000 7f2dfbf65640 1 -- 172.21.15.29:0/2372434204 wait complete. 2024-04-25T00:31:33.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.136+0000 7f2dfbf65640 1 Processor -- start 2024-04-25T00:31:33.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.136+0000 7f2dfbf65640 1 -- start start 2024-04-25T00:31:33.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfbf65640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de00039c0 0x7f2df40cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfbf65640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40c9420 0x7f2df41f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfbf65640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2df41f4b70 0x7f2df40b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfbf65640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2df40b7900 con 0x7f2df41f4b70 2024-04-25T00:31:33.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfbf65640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2df40b7780 con 0x7f2de00039c0 2024-04-25T00:31:33.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfbf65640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2df40b7a80 con 0x7f2df40c9420 2024-04-25T00:31:33.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfa502640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2df41f4b70 0x7f2df40b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfa502640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2df41f4b70 0x7f2df40b8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51682/0 (socket says 172.21.15.29:51682) 2024-04-25T00:31:33.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfa502640 1 -- 172.21.15.29:0/3840299145 learned_addr learned my addr 172.21.15.29:0/3840299145 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2df9500640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40c9420 0x7f2df41f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2df9d01640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de00039c0 0x7f2df40cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfa502640 1 -- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40c9420 msgr2=0x7f2df41f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfa502640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40c9420 0x7f2df41f4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfa502640 1 -- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de00039c0 msgr2=0x7f2df40cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfa502640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de00039c0 0x7f2df40cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfa502640 1 -- 172.21.15.29:0/3840299145 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df41f5060 con 0x7f2df41f4b70 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfa502640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2df41f4b70 0x7f2df40b8f90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2df41621e0 tx=0x7f2dec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dea7fc640 1 -- 172.21.15.29:0/3840299145 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec011e10 con 0x7f2df41f4b70 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dea7fc640 1 -- 172.21.15.29:0/3840299145 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dec01b070 con 0x7f2df41f4b70 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dea7fc640 1 -- 172.21.15.29:0/3840299145 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec0039a0 con 0x7f2df41f4b70 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfbf65640 1 -- 172.21.15.29:0/3840299145 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df41f5340 con 0x7f2df41f4b70 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfbf65640 1 -- 172.21.15.29:0/3840299145 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df40c8f00 con 0x7f2df41f4b70 2024-04-25T00:31:33.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dfbf65640 1 -- 172.21.15.29:0/3840299145 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df4069270 con 0x7f2df41f4b70 2024-04-25T00:31:33.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dea7fc640 1 -- 172.21.15.29:0/3840299145 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2dec003b60 con 0x7f2df41f4b70 2024-04-25T00:31:33.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dea7fc640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2dd80678c0 0x7f2dd8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2dea7fc640 1 -- 172.21.15.29:0/3840299145 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2dec086560 con 0x7f2df41f4b70 2024-04-25T00:31:33.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.140+0000 7f2df9d01640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2dd80678c0 0x7f2dd8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.144+0000 7f2df9d01640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2dd80678c0 0x7f2dd8069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2df00af000 tx=0x7f2df00046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:33.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.144+0000 7f2dea7fc640 1 -- 172.21.15.29:0/3840299145 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2dec025020 con 0x7f2df41f4b70 2024-04-25T00:31:33.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.304+0000 7f2dfbf65640 1 -- 172.21.15.29:0/3840299145 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2df40c90d0 con 0x7f2dd80678c0 2024-04-25T00:31:33.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.320+0000 7f2dea7fc640 1 -- 172.21.15.29:0/3840299145 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240068 (secure 0 0 0) 0x7f2df00f79e0 con 0x7f2dd80678c0 2024-04-25T00:31:33.327 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:33.329 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:31:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.324+0000 7f2dfbf65640 1 -- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2dd80678c0 msgr2=0x7f2dd8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.324+0000 7f2dfbf65640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2dd80678c0 0x7f2dd8069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2df00af000 tx=0x7f2df00046d0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.324+0000 7f2dfbf65640 1 -- 172.21.15.29:0/3840299145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2df41f4b70 msgr2=0x7f2df40b8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.324+0000 7f2dfbf65640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2df41f4b70 0x7f2df40b8f90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2df41621e0 tx=0x7f2dec0115a0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.328+0000 7f2dfbf65640 1 -- 172.21.15.29:0/3840299145 shutdown_connections 2024-04-25T00:31:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.328+0000 7f2dfbf65640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2df41f4b70 0x7f2df40b8f90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.328+0000 7f2dfbf65640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40c9420 0x7f2df41f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.328+0000 7f2dfbf65640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2dd80678c0 0x7f2dd8069d80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.328+0000 7f2dfbf65640 1 --2- 172.21.15.29:0/3840299145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de00039c0 0x7f2df40cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.328+0000 7f2dfbf65640 1 -- 172.21.15.29:0/3840299145 >> 172.21.15.29:0/3840299145 conn(0x7f2df40d3fb0 msgr2=0x7f2df4161a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.328+0000 7f2dfbf65640 1 -- 172.21.15.29:0/3840299145 shutdown_connections 2024-04-25T00:31:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.328+0000 7f2dfbf65640 1 -- 172.21.15.29:0/3840299145 wait complete. 2024-04-25T00:31:33.346 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-04-25T00:31:31.443960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251920,"kb_used_data":2472,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722800,"statfs":{"total":773094113280,"available":772836147200,"internally_reserved":0,"allocated":2531328,"data_stored":1593408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.185692"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.258680+0000","last_change":"2024-04-25T00:31:28.258680+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.258680+0000","last_undegraded":"2024-04-25T00:31:28.258680+0000","last_fullsized":"2024-04-25T00:31:28.258680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260346+0000","last_change":"2024-04-25T00:31:28.260346+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260346+0000","last_undegraded":"2024-04-25T00:31:28.260346+0000","last_fullsized":"2024-04-25T00:31:28.260346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261252+0000","last_change":"2024-04-25T00:31:28.261252+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261252+0000","last_undegraded":"2024-04-25T00:31:28.261252+0000","last_fullsized":"2024-04-25T00:31:28.261252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252572+0000","last_change":"2024-04-25T00:31:28.252572+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252572+0000","last_undegraded":"2024-04-25T00:31:28.252572+0000","last_fullsized":"2024-04-25T00:31:28.252572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.264117+0000","last_change":"2024-04-25T00:31:28.264117+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.264117+0000","last_undegraded":"2024-04-25T00:31:28.264117+0000","last_fullsized":"2024-04-25T00:31:28.264117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252610+0000","last_change":"2024-04-25T00:31:28.252610+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252610+0000","last_undegraded":"2024-04-25T00:31:28.252610+0000","last_fullsized":"2024-04-25T00:31:28.252610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.259359+0000","last_change":"2024-04-25T00:31:28.259359+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.259359+0000","last_undegraded":"2024-04-25T00:31:28.259359+0000","last_fullsized":"2024-04-25T00:31:28.259359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.264603+0000","last_change":"2024-04-25T00:31:28.264603+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.264603+0000","last_undegraded":"2024-04-25T00:31:28.264603+0000","last_fullsized":"2024-04-25T00:31:28.264603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.255699+0000","last_change":"2024-04-25T00:31:28.255699+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.255699+0000","last_undegraded":"2024-04-25T00:31:28.255699+0000","last_fullsized":"2024-04-25T00:31:28.255699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.254358+0000","last_change":"2024-04-25T00:31:28.254358+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.254358+0000","last_undegraded":"2024-04-25T00:31:28.254358+0000","last_fullsized":"2024-04-25T00:31:28.254358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260583+0000","last_change":"2024-04-25T00:31:28.260583+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260583+0000","last_undegraded":"2024-04-25T00:31:28.260583+0000","last_fullsized":"2024-04-25T00:31:28.260583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261140+0000","last_change":"2024-04-25T00:31:28.261140+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261140+0000","last_undegraded":"2024-04-25T00:31:28.261140+0000","last_fullsized":"2024-04-25T00:31:28.261140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252595+0000","last_change":"2024-04-25T00:31:28.252595+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252595+0000","last_undegraded":"2024-04-25T00:31:28.252595+0000","last_fullsized":"2024-04-25T00:31:28.252595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260605+0000","last_change":"2024-04-25T00:31:28.260605+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260605+0000","last_undegraded":"2024-04-25T00:31:28.260605+0000","last_fullsized":"2024-04-25T00:31:28.260605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.258823+0000","last_change":"2024-04-25T00:31:28.258823+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.258823+0000","last_undegraded":"2024-04-25T00:31:28.258823+0000","last_fullsized":"2024-04-25T00:31:28.258823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261543+0000","last_change":"2024-04-25T00:31:28.261543+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261543+0000","last_undegraded":"2024-04-25T00:31:28.261543+0000","last_fullsized":"2024-04-25T00:31:28.261543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T00:31:28.252054+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:31:28.252054+0000","last_peered":"2024-04-25T00:31:28.252054+0000","last_clean":"2024-04-25T00:31:28.252054+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:31:28.252054+0000","last_undegraded":"2024-04-25T00:31:28.252054+0000","last_fullsized":"2024-04-25T00:31:28.252054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.253465+0000","last_change":"2024-04-25T00:31:28.253465+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.253465+0000","last_undegraded":"2024-04-25T00:31:28.253465+0000","last_fullsized":"2024-04-25T00:31:28.253465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257931+0000","last_change":"2024-04-25T00:31:28.257931+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257931+0000","last_undegraded":"2024-04-25T00:31:28.257931+0000","last_fullsized":"2024-04-25T00:31:28.257931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.259930+0000","last_change":"2024-04-25T00:31:28.259930+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.259930+0000","last_undegraded":"2024-04-25T00:31:28.259930+0000","last_fullsized":"2024-04-25T00:31:28.259930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252566+0000","last_change":"2024-04-25T00:31:28.252566+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252566+0000","last_undegraded":"2024-04-25T00:31:28.252566+0000","last_fullsized":"2024-04-25T00:31:28.252566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257713+0000","last_change":"2024-04-25T00:31:28.257713+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257713+0000","last_undegraded":"2024-04-25T00:31:28.257713+0000","last_fullsized":"2024-04-25T00:31:28.257713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261729+0000","last_change":"2024-04-25T00:31:28.261729+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261729+0000","last_undegraded":"2024-04-25T00:31:28.261729+0000","last_fullsized":"2024-04-25T00:31:28.261729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257418+0000","last_change":"2024-04-25T00:31:28.257418+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257418+0000","last_undegraded":"2024-04-25T00:31:28.257418+0000","last_fullsized":"2024-04-25T00:31:28.257418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261709+0000","last_change":"2024-04-25T00:31:28.261709+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261709+0000","last_undegraded":"2024-04-25T00:31:28.261709+0000","last_fullsized":"2024-04-25T00:31:28.261709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.267496+0000","last_change":"2024-04-25T00:31:28.267496+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.267496+0000","last_undegraded":"2024-04-25T00:31:28.267496+0000","last_fullsized":"2024-04-25T00:31:28.267496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257881+0000","last_change":"2024-04-25T00:31:28.257881+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257881+0000","last_undegraded":"2024-04-25T00:31:28.257881+0000","last_fullsized":"2024-04-25T00:31:28.257881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.258806+0000","last_change":"2024-04-25T00:31:28.258806+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.258806+0000","last_undegraded":"2024-04-25T00:31:28.258806+0000","last_fullsized":"2024-04-25T00:31:28.258806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.266347+0000","last_change":"2024-04-25T00:31:28.266347+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.266347+0000","last_undegraded":"2024-04-25T00:31:28.266347+0000","last_fullsized":"2024-04-25T00:31:28.266347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.269229+0000","last_change":"2024-04-25T00:31:28.269229+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.269229+0000","last_undegraded":"2024-04-25T00:31:28.269229+0000","last_fullsized":"2024-04-25T00:31:28.269229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.268083+0000","last_change":"2024-04-25T00:31:28.268083+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.268083+0000","last_undegraded":"2024-04-25T00:31:28.268083+0000","last_fullsized":"2024-04-25T00:31:28.268083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260490+0000","last_change":"2024-04-25T00:31:28.260490+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260490+0000","last_undegraded":"2024-04-25T00:31:28.260490+0000","last_fullsized":"2024-04-25T00:31:28.260490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.256066+0000","last_change":"2024-04-25T00:31:28.256066+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.256066+0000","last_undegraded":"2024-04-25T00:31:28.256066+0000","last_fullsized":"2024-04-25T00:31:28.256066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:31:33.350 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:31:33.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f2735d8e640 1 Processor -- start 2024-04-25T00:31:33.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f2735d8e640 1 -- start start 2024-04-25T00:31:33.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f2735d8e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27300b4d30 0x7f27300b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f2735d8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273006d210 0x7f27300ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f2735d8e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f273006d6c0 con 0x7f273015df70 2024-04-25T00:31:33.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f2735d8e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f273006d980 con 0x7f27300b4d30 2024-04-25T00:31:33.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f2735d8e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f273006dc60 con 0x7f273006d210 2024-04-25T00:31:33.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f272f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f273015df70 0x7f27300b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43230/0 (socket says 172.21.15.29:43230) 2024-04-25T00:31:33.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f272f7fe640 1 -- 172.21.15.29:0/1138389414 learned_addr learned my addr 172.21.15.29:0/1138389414 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:33.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f272effd640 1 --2- 172.21.15.29:0/1138389414 >> v2:172.21.15.157:3300/0 conn(0x7f27300b4d30 0x7f27300b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f272ffff640 1 --2- 172.21.15.29:0/1138389414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273006d210 0x7f27300ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3993145114 0 0) 0x7f273006d6c0 con 0x7f273015df70 2024-04-25T00:31:33.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2718003530 con 0x7f273015df70 2024-04-25T00:31:33.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1978295890 0 0) 0x7f2718003530 con 0x7f273015df70 2024-04-25T00:31:33.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273006d210 msgr2=0x7f27300ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:33.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.508+0000 7f272e7fc640 1 --2- 172.21.15.29:0/1138389414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273006d210 0x7f27300ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 >> v2:172.21.15.157:3300/0 conn(0x7f27300b4d30 msgr2=0x7f27300b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:33.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 --2- 172.21.15.29:0/1138389414 >> v2:172.21.15.157:3300/0 conn(0x7f27300b4d30 0x7f27300b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273006df40 con 0x7f273015df70 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2728001680 con 0x7f273015df70 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 >> v1:172.21.15.29:6789/0 conn(0x7f273015df70 legacy=0x7f27300b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 --2- 172.21.15.29:0/1138389414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27180039c0 0x7f2718003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 --2- 172.21.15.29:0/1138389414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273006d210 0x7f2718004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 --2- 172.21.15.29:0/1138389414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2718004d50 0x7f27180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f273006d980 con 0x7f27180039c0 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f273006dc60 con 0x7f2718004d50 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f271800b9d0 con 0x7f273006d210 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272f7fe640 1 --2- 172.21.15.29:0/1138389414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27180039c0 0x7f2718003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272effd640 1 --2- 172.21.15.29:0/1138389414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273006d210 0x7f2718004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272f7fe640 1 -- 172.21.15.29:0/1138389414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273006d210 msgr2=0x7f2718004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272f7fe640 1 --2- 172.21.15.29:0/1138389414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f273006d210 0x7f2718004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272f7fe640 1 -- 172.21.15.29:0/1138389414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2718004d50 msgr2=0x7f27180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:33.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272ffff640 1 --2- 172.21.15.29:0/1138389414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2718004d50 0x7f27180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272f7fe640 1 --2- 172.21.15.29:0/1138389414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2718004d50 0x7f27180051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272f7fe640 1 -- 172.21.15.29:0/1138389414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2718003530 con 0x7f27180039c0 2024-04-25T00:31:33.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272f7fe640 1 --2- 172.21.15.29:0/1138389414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27180039c0 0x7f2718003de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f272800c2c0 tx=0x7f272801cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcc2a1cd694d9da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:33.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728007d60 con 0x7f27180039c0 2024-04-25T00:31:33.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27280a9020 con 0x7f27180039c0 2024-04-25T00:31:33.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272e7fc640 1 -- 172.21.15.29:0/1138389414 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27280a5b60 con 0x7f27180039c0 2024-04-25T00:31:33.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f2735d8e640 1 -- 172.21.15.29:0/1138389414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27180039c0 msgr2=0x7f2718003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f2735d8e640 1 --2- 172.21.15.29:0/1138389414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27180039c0 0x7f2718003de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f272800c2c0 tx=0x7f272801cd10 comp rx=0 tx=0).stop 2024-04-25T00:31:33.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f272ffff640 1 -- 172.21.15.29:0/1138389414 reap_dead start 2024-04-25T00:31:33.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f2735d8e640 1 -- 172.21.15.29:0/1138389414 shutdown_connections 2024-04-25T00:31:33.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f2735d8e640 1 -- 172.21.15.29:0/1138389414 >> 172.21.15.29:0/1138389414 conn(0x7f27300b1870 msgr2=0x7f27300b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:33.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f2735d8e640 1 -- 172.21.15.29:0/1138389414 shutdown_connections 2024-04-25T00:31:33.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.512+0000 7f2735d8e640 1 -- 172.21.15.29:0/1138389414 wait complete. 2024-04-25T00:31:33.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2735d8e640 1 Processor -- start 2024-04-25T00:31:33.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2735d8e640 1 -- start start 2024-04-25T00:31:33.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2735d8e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2730146590 0x7f27300cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2735d8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2730146970 0x7f27300cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2735d8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27300cdf40 0x7f27300ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2735d8e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f273006eba0 con 0x7f2730146590 2024-04-25T00:31:33.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2735d8e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f273006ea20 con 0x7f27300cdf40 2024-04-25T00:31:33.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2735d8e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f27300ae0b0 con 0x7f2730146970 2024-04-25T00:31:33.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f272f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2730146590 0x7f27300cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f272f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2730146590 0x7f27300cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51706/0 (socket says 172.21.15.29:51706) 2024-04-25T00:31:33.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f272f7fe640 1 -- 172.21.15.29:0/1142895115 learned_addr learned my addr 172.21.15.29:0/1142895115 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:33.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f272ffff640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27300cdf40 0x7f27300ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f272f7fe640 1 -- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2730146970 msgr2=0x7f27300cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f272effd640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2730146970 0x7f27300cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f272f7fe640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2730146970 0x7f27300cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f272f7fe640 1 -- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27300cdf40 msgr2=0x7f27300ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f272f7fe640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27300cdf40 0x7f27300ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f272f7fe640 1 -- 172.21.15.29:0/1142895115 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27300cf740 con 0x7f2730146590 2024-04-25T00:31:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f272ffff640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27300cdf40 0x7f27300ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f272effd640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2730146970 0x7f27300cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f272f7fe640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2730146590 0x7f27300cc330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f27280ab040 tx=0x7f2718005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2717fff640 1 -- 172.21.15.29:0/1142895115 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27280a5b60 con 0x7f2730146590 2024-04-25T00:31:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2717fff640 1 -- 172.21.15.29:0/1142895115 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27280b2030 con 0x7f2730146590 2024-04-25T00:31:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2717fff640 1 -- 172.21.15.29:0/1142895115 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27280048b0 con 0x7f2730146590 2024-04-25T00:31:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2735d8e640 1 -- 172.21.15.29:0/1142895115 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27300ccea0 con 0x7f2730146590 2024-04-25T00:31:33.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2735d8e640 1 -- 172.21.15.29:0/1142895115 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27300cd460 con 0x7f2730146590 2024-04-25T00:31:33.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2735d8e640 1 -- 172.21.15.29:0/1142895115 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2730069270 con 0x7f2730146590 2024-04-25T00:31:33.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.516+0000 7f2717fff640 1 -- 172.21.15.29:0/1142895115 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2728004a50 con 0x7f2730146590 2024-04-25T00:31:33.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.520+0000 7f2717fff640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f271c0678c0 0x7f271c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.520+0000 7f2717fff640 1 -- 172.21.15.29:0/1142895115 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f272811eff0 con 0x7f2730146590 2024-04-25T00:31:33.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.520+0000 7f272effd640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f271c0678c0 0x7f271c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.520+0000 7f272effd640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f271c0678c0 0x7f271c069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2724006cf0 tx=0x7f2724008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:33.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.524+0000 7f2717fff640 1 -- 172.21.15.29:0/1142895115 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2728016a50 con 0x7f2730146590 2024-04-25T00:31:33.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.680+0000 7f2735d8e640 1 -- 172.21.15.29:0/1142895115 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f273015a350 con 0x7f271c0678c0 2024-04-25T00:31:33.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.696+0000 7f2717fff640 1 -- 172.21.15.29:0/1142895115 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240068 (secure 0 0 0) 0x7f273015a350 con 0x7f271c0678c0 2024-04-25T00:31:33.772 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:33.773 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:31:33.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.700+0000 7f2735d8e640 1 -- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f271c0678c0 msgr2=0x7f271c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.700+0000 7f2735d8e640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f271c0678c0 0x7f271c069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2724006cf0 tx=0x7f2724008040 comp rx=0 tx=0).stop 2024-04-25T00:31:33.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.700+0000 7f2735d8e640 1 -- 172.21.15.29:0/1142895115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2730146590 msgr2=0x7f27300cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.700+0000 7f2735d8e640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2730146590 0x7f27300cc330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f27280ab040 tx=0x7f2718005840 comp rx=0 tx=0).stop 2024-04-25T00:31:33.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.700+0000 7f2735d8e640 1 -- 172.21.15.29:0/1142895115 shutdown_connections 2024-04-25T00:31:33.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.700+0000 7f2735d8e640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f271c0678c0 0x7f271c069d80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.700+0000 7f2735d8e640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27300cdf40 0x7f27300ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.700+0000 7f2735d8e640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2730146970 0x7f27300cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.700+0000 7f2735d8e640 1 --2- 172.21.15.29:0/1142895115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2730146590 0x7f27300cc330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.700+0000 7f2735d8e640 1 -- 172.21.15.29:0/1142895115 >> 172.21.15.29:0/1142895115 conn(0x7f27300b1870 msgr2=0x7f27300cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:33.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.700+0000 7f2735d8e640 1 -- 172.21.15.29:0/1142895115 shutdown_connections 2024-04-25T00:31:33.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.700+0000 7f2735d8e640 1 -- 172.21.15.29:0/1142895115 wait complete. 2024-04-25T00:31:33.776 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-25T00:31:33.444554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251920,"kb_used_data":2472,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722800,"statfs":{"total":773094113280,"available":772836147200,"internally_reserved":0,"allocated":2531328,"data_stored":1593408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.186286"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.258680+0000","last_change":"2024-04-25T00:31:28.258680+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.258680+0000","last_undegraded":"2024-04-25T00:31:28.258680+0000","last_fullsized":"2024-04-25T00:31:28.258680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260346+0000","last_change":"2024-04-25T00:31:28.260346+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260346+0000","last_undegraded":"2024-04-25T00:31:28.260346+0000","last_fullsized":"2024-04-25T00:31:28.260346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261252+0000","last_change":"2024-04-25T00:31:28.261252+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261252+0000","last_undegraded":"2024-04-25T00:31:28.261252+0000","last_fullsized":"2024-04-25T00:31:28.261252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252572+0000","last_change":"2024-04-25T00:31:28.252572+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252572+0000","last_undegraded":"2024-04-25T00:31:28.252572+0000","last_fullsized":"2024-04-25T00:31:28.252572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.264117+0000","last_change":"2024-04-25T00:31:28.264117+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.264117+0000","last_undegraded":"2024-04-25T00:31:28.264117+0000","last_fullsized":"2024-04-25T00:31:28.264117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252610+0000","last_change":"2024-04-25T00:31:28.252610+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252610+0000","last_undegraded":"2024-04-25T00:31:28.252610+0000","last_fullsized":"2024-04-25T00:31:28.252610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.259359+0000","last_change":"2024-04-25T00:31:28.259359+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.259359+0000","last_undegraded":"2024-04-25T00:31:28.259359+0000","last_fullsized":"2024-04-25T00:31:28.259359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.264603+0000","last_change":"2024-04-25T00:31:28.264603+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.264603+0000","last_undegraded":"2024-04-25T00:31:28.264603+0000","last_fullsized":"2024-04-25T00:31:28.264603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.255699+0000","last_change":"2024-04-25T00:31:28.255699+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.255699+0000","last_undegraded":"2024-04-25T00:31:28.255699+0000","last_fullsized":"2024-04-25T00:31:28.255699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.254358+0000","last_change":"2024-04-25T00:31:28.254358+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.254358+0000","last_undegraded":"2024-04-25T00:31:28.254358+0000","last_fullsized":"2024-04-25T00:31:28.254358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260583+0000","last_change":"2024-04-25T00:31:28.260583+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260583+0000","last_undegraded":"2024-04-25T00:31:28.260583+0000","last_fullsized":"2024-04-25T00:31:28.260583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261140+0000","last_change":"2024-04-25T00:31:28.261140+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261140+0000","last_undegraded":"2024-04-25T00:31:28.261140+0000","last_fullsized":"2024-04-25T00:31:28.261140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252595+0000","last_change":"2024-04-25T00:31:28.252595+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252595+0000","last_undegraded":"2024-04-25T00:31:28.252595+0000","last_fullsized":"2024-04-25T00:31:28.252595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260605+0000","last_change":"2024-04-25T00:31:28.260605+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260605+0000","last_undegraded":"2024-04-25T00:31:28.260605+0000","last_fullsized":"2024-04-25T00:31:28.260605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.258823+0000","last_change":"2024-04-25T00:31:28.258823+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.258823+0000","last_undegraded":"2024-04-25T00:31:28.258823+0000","last_fullsized":"2024-04-25T00:31:28.258823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261543+0000","last_change":"2024-04-25T00:31:28.261543+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261543+0000","last_undegraded":"2024-04-25T00:31:28.261543+0000","last_fullsized":"2024-04-25T00:31:28.261543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T00:31:28.252054+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:31:28.252054+0000","last_peered":"2024-04-25T00:31:28.252054+0000","last_clean":"2024-04-25T00:31:28.252054+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:31:28.252054+0000","last_undegraded":"2024-04-25T00:31:28.252054+0000","last_fullsized":"2024-04-25T00:31:28.252054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.253465+0000","last_change":"2024-04-25T00:31:28.253465+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.253465+0000","last_undegraded":"2024-04-25T00:31:28.253465+0000","last_fullsized":"2024-04-25T00:31:28.253465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257931+0000","last_change":"2024-04-25T00:31:28.257931+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257931+0000","last_undegraded":"2024-04-25T00:31:28.257931+0000","last_fullsized":"2024-04-25T00:31:28.257931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.259930+0000","last_change":"2024-04-25T00:31:28.259930+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.259930+0000","last_undegraded":"2024-04-25T00:31:28.259930+0000","last_fullsized":"2024-04-25T00:31:28.259930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252566+0000","last_change":"2024-04-25T00:31:28.252566+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252566+0000","last_undegraded":"2024-04-25T00:31:28.252566+0000","last_fullsized":"2024-04-25T00:31:28.252566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257713+0000","last_change":"2024-04-25T00:31:28.257713+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257713+0000","last_undegraded":"2024-04-25T00:31:28.257713+0000","last_fullsized":"2024-04-25T00:31:28.257713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261729+0000","last_change":"2024-04-25T00:31:28.261729+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261729+0000","last_undegraded":"2024-04-25T00:31:28.261729+0000","last_fullsized":"2024-04-25T00:31:28.261729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257418+0000","last_change":"2024-04-25T00:31:28.257418+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257418+0000","last_undegraded":"2024-04-25T00:31:28.257418+0000","last_fullsized":"2024-04-25T00:31:28.257418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261709+0000","last_change":"2024-04-25T00:31:28.261709+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261709+0000","last_undegraded":"2024-04-25T00:31:28.261709+0000","last_fullsized":"2024-04-25T00:31:28.261709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.267496+0000","last_change":"2024-04-25T00:31:28.267496+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.267496+0000","last_undegraded":"2024-04-25T00:31:28.267496+0000","last_fullsized":"2024-04-25T00:31:28.267496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257881+0000","last_change":"2024-04-25T00:31:28.257881+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257881+0000","last_undegraded":"2024-04-25T00:31:28.257881+0000","last_fullsized":"2024-04-25T00:31:28.257881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.258806+0000","last_change":"2024-04-25T00:31:28.258806+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.258806+0000","last_undegraded":"2024-04-25T00:31:28.258806+0000","last_fullsized":"2024-04-25T00:31:28.258806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.266347+0000","last_change":"2024-04-25T00:31:28.266347+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.266347+0000","last_undegraded":"2024-04-25T00:31:28.266347+0000","last_fullsized":"2024-04-25T00:31:28.266347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.269229+0000","last_change":"2024-04-25T00:31:28.269229+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.269229+0000","last_undegraded":"2024-04-25T00:31:28.269229+0000","last_fullsized":"2024-04-25T00:31:28.269229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.268083+0000","last_change":"2024-04-25T00:31:28.268083+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.268083+0000","last_undegraded":"2024-04-25T00:31:28.268083+0000","last_fullsized":"2024-04-25T00:31:28.268083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260490+0000","last_change":"2024-04-25T00:31:28.260490+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260490+0000","last_undegraded":"2024-04-25T00:31:28.260490+0000","last_fullsized":"2024-04-25T00:31:28.260490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.256066+0000","last_change":"2024-04-25T00:31:28.256066+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.256066+0000","last_undegraded":"2024-04-25T00:31:28.256066+0000","last_fullsized":"2024-04-25T00:31:28.256066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:31:33.779 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:31:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f4590677640 1 Processor -- start 2024-04-25T00:31:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f4590677640 1 -- start start 2024-04-25T00:31:33.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f4590677640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458815df70 0x7f45880b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f4590677640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f45880b4d30 0x7f45880ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f4590677640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458806d980 con 0x7f458806d210 2024-04-25T00:31:33.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f4590677640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f458806dc60 con 0x7f45880b4d30 2024-04-25T00:31:33.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f4590677640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f458806df40 con 0x7f458815df70 2024-04-25T00:31:33.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f458dc12640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f458806d210 0x7f45880b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43242/0 (socket says 172.21.15.29:43242) 2024-04-25T00:31:33.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f458dc12640 1 -- 172.21.15.29:0/2349863497 learned_addr learned my addr 172.21.15.29:0/2349863497 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:33.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f458e413640 1 --2- 172.21.15.29:0/2349863497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458815df70 0x7f45880b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f458ec14640 1 --2- 172.21.15.29:0/2349863497 >> v2:172.21.15.157:3300/0 conn(0x7f45880b4d30 0x7f45880ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f458d411640 1 -- 172.21.15.29:0/2349863497 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3963853614 0 0) 0x7f458806d980 con 0x7f458806d210 2024-04-25T00:31:33.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f458d411640 1 -- 172.21.15.29:0/2349863497 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4574003530 con 0x7f458806d210 2024-04-25T00:31:33.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f458e413640 1 -- 172.21.15.29:0/2349863497 >> v2:172.21.15.157:3300/0 conn(0x7f45880b4d30 msgr2=0x7f45880ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:33.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f458e413640 1 --2- 172.21.15.29:0/2349863497 >> v2:172.21.15.157:3300/0 conn(0x7f45880b4d30 0x7f45880ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f458e413640 1 -- 172.21.15.29:0/2349863497 >> v1:172.21.15.29:6789/0 conn(0x7f458806d210 legacy=0x7f45880b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.908+0000 7f458e413640 1 -- 172.21.15.29:0/2349863497 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458806e220 con 0x7f458815df70 2024-04-25T00:31:33.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f458e413640 1 --2- 172.21.15.29:0/2349863497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458815df70 0x7f45880b3e40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f45840097c0 tx=0x7f458401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=858e81bd07978c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:33.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f458d411640 1 -- 172.21.15.29:0/2349863497 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45840ab610 con 0x7f458815df70 2024-04-25T00:31:33.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f458d411640 1 -- 172.21.15.29:0/2349863497 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45840b0070 con 0x7f458815df70 2024-04-25T00:31:33.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f458d411640 1 -- 172.21.15.29:0/2349863497 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45840ab9d0 con 0x7f458815df70 2024-04-25T00:31:33.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f4590677640 1 -- 172.21.15.29:0/2349863497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458815df70 msgr2=0x7f45880b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f4590677640 1 --2- 172.21.15.29:0/2349863497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458815df70 0x7f45880b3e40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f45840097c0 tx=0x7f458401c470 comp rx=0 tx=0).stop 2024-04-25T00:31:33.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f4590677640 1 -- 172.21.15.29:0/2349863497 shutdown_connections 2024-04-25T00:31:33.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f4590677640 1 --2- 172.21.15.29:0/2349863497 >> v2:172.21.15.157:3300/0 conn(0x7f45880b4d30 0x7f45880ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f4590677640 1 --2- 172.21.15.29:0/2349863497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458815df70 0x7f45880b3e40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f4590677640 1 -- 172.21.15.29:0/2349863497 >> 172.21.15.29:0/2349863497 conn(0x7f45880b1870 msgr2=0x7f45880b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:33.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f4590677640 1 -- 172.21.15.29:0/2349863497 shutdown_connections 2024-04-25T00:31:33.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f4590677640 1 -- 172.21.15.29:0/2349863497 wait complete. 2024-04-25T00:31:33.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f4590677640 1 Processor -- start 2024-04-25T00:31:33.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.912+0000 7f4590677640 1 -- start start 2024-04-25T00:31:33.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f4590677640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458806d210 0x7f45880ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f4590677640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45880b4d30 0x7f4588155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f4590677640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f458815df70 0x7f45880cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f4590677640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f458806eba0 con 0x7f45880b4d30 2024-04-25T00:31:33.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f4590677640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f458806ea20 con 0x7f458815df70 2024-04-25T00:31:33.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f4590677640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f45880ae5f0 con 0x7f458806d210 2024-04-25T00:31:33.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f458dc12640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45880b4d30 0x7f4588155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f458dc12640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45880b4d30 0x7f4588155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51718/0 (socket says 172.21.15.29:51718) 2024-04-25T00:31:33.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f458dc12640 1 -- 172.21.15.29:0/3479098348 learned_addr learned my addr 172.21.15.29:0/3479098348 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:33.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f458e413640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458806d210 0x7f45880ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f458ec14640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f458815df70 0x7f45880cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f458dc12640 1 -- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458806d210 msgr2=0x7f45880ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f458dc12640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458806d210 0x7f45880ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f458dc12640 1 -- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f458815df70 msgr2=0x7f45880cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:33.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f458dc12640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f458815df70 0x7f45880cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:33.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f458dc12640 1 -- 172.21.15.29:0/3479098348 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458815cad0 con 0x7f45880b4d30 2024-04-25T00:31:33.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f458ec14640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f458815df70 0x7f45880cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:33.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f458dc12640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45880b4d30 0x7f4588155470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f457800c500 tx=0x7f457800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:33.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f457effd640 1 -- 172.21.15.29:0/3479098348 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578017070 con 0x7f45880b4d30 2024-04-25T00:31:33.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f457effd640 1 -- 172.21.15.29:0/3479098348 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4578005cf0 con 0x7f45880b4d30 2024-04-25T00:31:33.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f4590677640 1 -- 172.21.15.29:0/3479098348 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45880cd870 con 0x7f45880b4d30 2024-04-25T00:31:33.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f457effd640 1 -- 172.21.15.29:0/3479098348 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45780072f0 con 0x7f45880b4d30 2024-04-25T00:31:33.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f4590677640 1 -- 172.21.15.29:0/3479098348 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45880cde30 con 0x7f45880b4d30 2024-04-25T00:31:33.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f4590677640 1 -- 172.21.15.29:0/3479098348 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4588069270 con 0x7f45880b4d30 2024-04-25T00:31:33.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f457effd640 1 -- 172.21.15.29:0/3479098348 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f457801a070 con 0x7f45880b4d30 2024-04-25T00:31:33.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f457effd640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f456c0678c0 0x7f456c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:33.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.916+0000 7f457effd640 1 -- 172.21.15.29:0/3479098348 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4578084fb0 con 0x7f45880b4d30 2024-04-25T00:31:33.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.920+0000 7f458e413640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f456c0678c0 0x7f456c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:33.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.920+0000 7f458e413640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f456c0678c0 0x7f456c069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f45840af000 tx=0x7f45840b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:33.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:33.924+0000 7f457effd640 1 -- 172.21.15.29:0/3479098348 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4578055530 con 0x7f45880b4d30 2024-04-25T00:31:34.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.076+0000 7f4590677640 1 -- 172.21.15.29:0/3479098348 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f458815ccd0 con 0x7f456c0678c0 2024-04-25T00:31:34.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.096+0000 7f457effd640 1 -- 172.21.15.29:0/3479098348 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240068 (secure 0 0 0) 0x7f458815ccd0 con 0x7f456c0678c0 2024-04-25T00:31:34.102 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:34.106 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:31:34.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.096+0000 7f4590677640 1 -- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f456c0678c0 msgr2=0x7f456c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.096+0000 7f4590677640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f456c0678c0 0x7f456c069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f45840af000 tx=0x7f45840b6040 comp rx=0 tx=0).stop 2024-04-25T00:31:34.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.096+0000 7f4590677640 1 -- 172.21.15.29:0/3479098348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45880b4d30 msgr2=0x7f4588155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.096+0000 7f4590677640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45880b4d30 0x7f4588155470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f457800c500 tx=0x7f457800c9c0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.100+0000 7f4590677640 1 -- 172.21.15.29:0/3479098348 shutdown_connections 2024-04-25T00:31:34.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.100+0000 7f4590677640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f456c0678c0 0x7f456c069d80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.100+0000 7f4590677640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f458815df70 0x7f45880cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.100+0000 7f4590677640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45880b4d30 0x7f4588155470 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.100+0000 7f4590677640 1 --2- 172.21.15.29:0/3479098348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458806d210 0x7f45880ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.100+0000 7f4590677640 1 -- 172.21.15.29:0/3479098348 >> 172.21.15.29:0/3479098348 conn(0x7f45880b1870 msgr2=0x7f4588156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:34.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.100+0000 7f4590677640 1 -- 172.21.15.29:0/3479098348 shutdown_connections 2024-04-25T00:31:34.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.100+0000 7f4590677640 1 -- 172.21.15.29:0/3479098348 wait complete. 2024-04-25T00:31:34.117 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-25T00:31:33.444554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251920,"kb_used_data":2472,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722800,"statfs":{"total":773094113280,"available":772836147200,"internally_reserved":0,"allocated":2531328,"data_stored":1593408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.186286"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.258680+0000","last_change":"2024-04-25T00:31:28.258680+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.258680+0000","last_undegraded":"2024-04-25T00:31:28.258680+0000","last_fullsized":"2024-04-25T00:31:28.258680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260346+0000","last_change":"2024-04-25T00:31:28.260346+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260346+0000","last_undegraded":"2024-04-25T00:31:28.260346+0000","last_fullsized":"2024-04-25T00:31:28.260346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261252+0000","last_change":"2024-04-25T00:31:28.261252+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261252+0000","last_undegraded":"2024-04-25T00:31:28.261252+0000","last_fullsized":"2024-04-25T00:31:28.261252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252572+0000","last_change":"2024-04-25T00:31:28.252572+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252572+0000","last_undegraded":"2024-04-25T00:31:28.252572+0000","last_fullsized":"2024-04-25T00:31:28.252572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.264117+0000","last_change":"2024-04-25T00:31:28.264117+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.264117+0000","last_undegraded":"2024-04-25T00:31:28.264117+0000","last_fullsized":"2024-04-25T00:31:28.264117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252610+0000","last_change":"2024-04-25T00:31:28.252610+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252610+0000","last_undegraded":"2024-04-25T00:31:28.252610+0000","last_fullsized":"2024-04-25T00:31:28.252610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.259359+0000","last_change":"2024-04-25T00:31:28.259359+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.259359+0000","last_undegraded":"2024-04-25T00:31:28.259359+0000","last_fullsized":"2024-04-25T00:31:28.259359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.264603+0000","last_change":"2024-04-25T00:31:28.264603+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.264603+0000","last_undegraded":"2024-04-25T00:31:28.264603+0000","last_fullsized":"2024-04-25T00:31:28.264603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.255699+0000","last_change":"2024-04-25T00:31:28.255699+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.255699+0000","last_undegraded":"2024-04-25T00:31:28.255699+0000","last_fullsized":"2024-04-25T00:31:28.255699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.254358+0000","last_change":"2024-04-25T00:31:28.254358+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.254358+0000","last_undegraded":"2024-04-25T00:31:28.254358+0000","last_fullsized":"2024-04-25T00:31:28.254358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260583+0000","last_change":"2024-04-25T00:31:28.260583+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260583+0000","last_undegraded":"2024-04-25T00:31:28.260583+0000","last_fullsized":"2024-04-25T00:31:28.260583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261140+0000","last_change":"2024-04-25T00:31:28.261140+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261140+0000","last_undegraded":"2024-04-25T00:31:28.261140+0000","last_fullsized":"2024-04-25T00:31:28.261140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252595+0000","last_change":"2024-04-25T00:31:28.252595+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252595+0000","last_undegraded":"2024-04-25T00:31:28.252595+0000","last_fullsized":"2024-04-25T00:31:28.252595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260605+0000","last_change":"2024-04-25T00:31:28.260605+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260605+0000","last_undegraded":"2024-04-25T00:31:28.260605+0000","last_fullsized":"2024-04-25T00:31:28.260605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.258823+0000","last_change":"2024-04-25T00:31:28.258823+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.258823+0000","last_undegraded":"2024-04-25T00:31:28.258823+0000","last_fullsized":"2024-04-25T00:31:28.258823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261543+0000","last_change":"2024-04-25T00:31:28.261543+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261543+0000","last_undegraded":"2024-04-25T00:31:28.261543+0000","last_fullsized":"2024-04-25T00:31:28.261543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T00:31:28.252054+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:31:28.252054+0000","last_peered":"2024-04-25T00:31:28.252054+0000","last_clean":"2024-04-25T00:31:28.252054+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:31:28.252054+0000","last_undegraded":"2024-04-25T00:31:28.252054+0000","last_fullsized":"2024-04-25T00:31:28.252054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.253465+0000","last_change":"2024-04-25T00:31:28.253465+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.253465+0000","last_undegraded":"2024-04-25T00:31:28.253465+0000","last_fullsized":"2024-04-25T00:31:28.253465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257931+0000","last_change":"2024-04-25T00:31:28.257931+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257931+0000","last_undegraded":"2024-04-25T00:31:28.257931+0000","last_fullsized":"2024-04-25T00:31:28.257931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.259930+0000","last_change":"2024-04-25T00:31:28.259930+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.259930+0000","last_undegraded":"2024-04-25T00:31:28.259930+0000","last_fullsized":"2024-04-25T00:31:28.259930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252566+0000","last_change":"2024-04-25T00:31:28.252566+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252566+0000","last_undegraded":"2024-04-25T00:31:28.252566+0000","last_fullsized":"2024-04-25T00:31:28.252566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257713+0000","last_change":"2024-04-25T00:31:28.257713+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257713+0000","last_undegraded":"2024-04-25T00:31:28.257713+0000","last_fullsized":"2024-04-25T00:31:28.257713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261729+0000","last_change":"2024-04-25T00:31:28.261729+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261729+0000","last_undegraded":"2024-04-25T00:31:28.261729+0000","last_fullsized":"2024-04-25T00:31:28.261729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257418+0000","last_change":"2024-04-25T00:31:28.257418+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257418+0000","last_undegraded":"2024-04-25T00:31:28.257418+0000","last_fullsized":"2024-04-25T00:31:28.257418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261709+0000","last_change":"2024-04-25T00:31:28.261709+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261709+0000","last_undegraded":"2024-04-25T00:31:28.261709+0000","last_fullsized":"2024-04-25T00:31:28.261709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.267496+0000","last_change":"2024-04-25T00:31:28.267496+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.267496+0000","last_undegraded":"2024-04-25T00:31:28.267496+0000","last_fullsized":"2024-04-25T00:31:28.267496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257881+0000","last_change":"2024-04-25T00:31:28.257881+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257881+0000","last_undegraded":"2024-04-25T00:31:28.257881+0000","last_fullsized":"2024-04-25T00:31:28.257881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.258806+0000","last_change":"2024-04-25T00:31:28.258806+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.258806+0000","last_undegraded":"2024-04-25T00:31:28.258806+0000","last_fullsized":"2024-04-25T00:31:28.258806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.266347+0000","last_change":"2024-04-25T00:31:28.266347+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.266347+0000","last_undegraded":"2024-04-25T00:31:28.266347+0000","last_fullsized":"2024-04-25T00:31:28.266347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.269229+0000","last_change":"2024-04-25T00:31:28.269229+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.269229+0000","last_undegraded":"2024-04-25T00:31:28.269229+0000","last_fullsized":"2024-04-25T00:31:28.269229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.268083+0000","last_change":"2024-04-25T00:31:28.268083+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.268083+0000","last_undegraded":"2024-04-25T00:31:28.268083+0000","last_fullsized":"2024-04-25T00:31:28.268083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260490+0000","last_change":"2024-04-25T00:31:28.260490+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260490+0000","last_undegraded":"2024-04-25T00:31:28.260490+0000","last_fullsized":"2024-04-25T00:31:28.260490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.256066+0000","last_change":"2024-04-25T00:31:28.256066+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.256066+0000","last_undegraded":"2024-04-25T00:31:28.256066+0000","last_fullsized":"2024-04-25T00:31:28.256066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:31:34.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2024-04-25T00:31:34.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-04-25T00:31:34.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-04-25T00:31:34.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-04-25T00:31:34.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2024-04-25T00:31:34.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-04-25T00:31:34.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2024-04-25T00:31:34.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-04-25T00:31:34.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2024-04-25T00:31:34.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2024-04-25T00:31:34.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2024-04-25T00:31:34.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-04-25T00:31:34.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-04-25T00:31:34.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-04-25T00:31:34.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-04-25T00:31:34.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-04-25T00:31:34.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-04-25T00:31:34.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2024-04-25T00:31:34.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2024-04-25T00:31:34.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2024-04-25T00:31:34.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-04-25T00:31:34.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-04-25T00:31:34.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-04-25T00:31:34.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-04-25T00:31:34.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-04-25T00:31:34.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2024-04-25T00:31:34.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-04-25T00:31:34.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-04-25T00:31:34.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-04-25T00:31:34.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-04-25T00:31:34.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-04-25T00:31:34.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2024-04-25T00:31:34.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.128 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-04-25T00:31:34.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.128 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2024-04-25T00:31:34.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.258680+0000', 'last_change': '2024-04-25T00:31:28.258680+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.258680+0000', 'last_undegraded': '2024-04-25T00:31:28.258680+0000', 'last_fullsized': '2024-04-25T00:31:28.258680+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:31:34.128 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-04-25T00:31:34.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.128 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2024-04-25T00:31:34.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.260346+0000', 'last_change': '2024-04-25T00:31:28.260346+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.260346+0000', 'last_undegraded': '2024-04-25T00:31:28.260346+0000', 'last_fullsized': '2024-04-25T00:31:28.260346+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:31:34.128 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2024-04-25T00:31:34.129 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.261252+0000', 'last_change': '2024-04-25T00:31:28.261252+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.261252+0000', 'last_undegraded': '2024-04-25T00:31:28.261252+0000', 'last_fullsized': '2024-04-25T00:31:28.261252+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:31:34.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-04-25T00:31:34.129 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2024-04-25T00:31:34.129 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.252572+0000', 'last_change': '2024-04-25T00:31:28.252572+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.252572+0000', 'last_undegraded': '2024-04-25T00:31:28.252572+0000', 'last_fullsized': '2024-04-25T00:31:28.252572+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:31:34.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-04-25T00:31:34.129 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2024-04-25T00:31:34.129 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.264117+0000', 'last_change': '2024-04-25T00:31:28.264117+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.264117+0000', 'last_undegraded': '2024-04-25T00:31:28.264117+0000', 'last_fullsized': '2024-04-25T00:31:28.264117+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:31:34.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-04-25T00:31:34.129 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2024-04-25T00:31:34.130 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.252610+0000', 'last_change': '2024-04-25T00:31:28.252610+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.252610+0000', 'last_undegraded': '2024-04-25T00:31:28.252610+0000', 'last_fullsized': '2024-04-25T00:31:28.252610+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:31:34.130 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-04-25T00:31:34.130 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.130 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2024-04-25T00:31:34.130 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.259359+0000', 'last_change': '2024-04-25T00:31:28.259359+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.259359+0000', 'last_undegraded': '2024-04-25T00:31:28.259359+0000', 'last_fullsized': '2024-04-25T00:31:28.259359+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:31:34.130 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-04-25T00:31:34.130 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.130 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2024-04-25T00:31:34.130 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.264603+0000', 'last_change': '2024-04-25T00:31:28.264603+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.264603+0000', 'last_undegraded': '2024-04-25T00:31:28.264603+0000', 'last_fullsized': '2024-04-25T00:31:28.264603+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:31:34.130 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-04-25T00:31:34.130 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.131 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2024-04-25T00:31:34.131 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.255699+0000', 'last_change': '2024-04-25T00:31:28.255699+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.255699+0000', 'last_undegraded': '2024-04-25T00:31:28.255699+0000', 'last_fullsized': '2024-04-25T00:31:28.255699+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:31:34.131 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-04-25T00:31:34.131 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.131 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2024-04-25T00:31:34.131 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.254358+0000', 'last_change': '2024-04-25T00:31:28.254358+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.254358+0000', 'last_undegraded': '2024-04-25T00:31:28.254358+0000', 'last_fullsized': '2024-04-25T00:31:28.254358+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:31:34.131 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-04-25T00:31:34.131 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.131 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2024-04-25T00:31:34.131 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.260583+0000', 'last_change': '2024-04-25T00:31:28.260583+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.260583+0000', 'last_undegraded': '2024-04-25T00:31:28.260583+0000', 'last_fullsized': '2024-04-25T00:31:28.260583+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:31:34.131 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-04-25T00:31:34.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2024-04-25T00:31:34.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.261140+0000', 'last_change': '2024-04-25T00:31:28.261140+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.261140+0000', 'last_undegraded': '2024-04-25T00:31:28.261140+0000', 'last_fullsized': '2024-04-25T00:31:28.261140+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:31:34.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-04-25T00:31:34.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2024-04-25T00:31:34.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.252595+0000', 'last_change': '2024-04-25T00:31:28.252595+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.252595+0000', 'last_undegraded': '2024-04-25T00:31:28.252595+0000', 'last_fullsized': '2024-04-25T00:31:28.252595+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:31:34.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-04-25T00:31:34.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2024-04-25T00:31:34.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.260605+0000', 'last_change': '2024-04-25T00:31:28.260605+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.260605+0000', 'last_undegraded': '2024-04-25T00:31:28.260605+0000', 'last_fullsized': '2024-04-25T00:31:28.260605+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:31:34.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-04-25T00:31:34.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-04-25T00:31:34.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.258823+0000', 'last_change': '2024-04-25T00:31:28.258823+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.258823+0000', 'last_undegraded': '2024-04-25T00:31:28.258823+0000', 'last_fullsized': '2024-04-25T00:31:28.258823+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:31:34.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-04-25T00:31:34.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2024-04-25T00:31:34.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.261543+0000', 'last_change': '2024-04-25T00:31:28.261543+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.261543+0000', 'last_undegraded': '2024-04-25T00:31:28.261543+0000', 'last_fullsized': '2024-04-25T00:31:28.261543+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:31:34.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-04-25T00:31:34.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2024-04-25T00:31:34.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.253465+0000', 'last_change': '2024-04-25T00:31:28.253465+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.253465+0000', 'last_undegraded': '2024-04-25T00:31:28.253465+0000', 'last_fullsized': '2024-04-25T00:31:28.253465+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:31:34.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-04-25T00:31:34.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2024-04-25T00:31:34.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.257931+0000', 'last_change': '2024-04-25T00:31:28.257931+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.257931+0000', 'last_undegraded': '2024-04-25T00:31:28.257931+0000', 'last_fullsized': '2024-04-25T00:31:28.257931+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:31:34.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-04-25T00:31:34.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2024-04-25T00:31:34.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.259930+0000', 'last_change': '2024-04-25T00:31:28.259930+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.259930+0000', 'last_undegraded': '2024-04-25T00:31:28.259930+0000', 'last_fullsized': '2024-04-25T00:31:28.259930+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:31:34.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-04-25T00:31:34.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2024-04-25T00:31:34.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.252566+0000', 'last_change': '2024-04-25T00:31:28.252566+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.252566+0000', 'last_undegraded': '2024-04-25T00:31:28.252566+0000', 'last_fullsized': '2024-04-25T00:31:28.252566+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:31:34.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-04-25T00:31:34.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-04-25T00:31:34.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.257713+0000', 'last_change': '2024-04-25T00:31:28.257713+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.257713+0000', 'last_undegraded': '2024-04-25T00:31:28.257713+0000', 'last_fullsized': '2024-04-25T00:31:28.257713+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:31:34.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-04-25T00:31:34.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2024-04-25T00:31:34.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.261729+0000', 'last_change': '2024-04-25T00:31:28.261729+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.261729+0000', 'last_undegraded': '2024-04-25T00:31:28.261729+0000', 'last_fullsized': '2024-04-25T00:31:28.261729+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:31:34.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-04-25T00:31:34.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2024-04-25T00:31:34.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.257418+0000', 'last_change': '2024-04-25T00:31:28.257418+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.257418+0000', 'last_undegraded': '2024-04-25T00:31:28.257418+0000', 'last_fullsized': '2024-04-25T00:31:28.257418+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:31:34.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-04-25T00:31:34.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2024-04-25T00:31:34.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.261709+0000', 'last_change': '2024-04-25T00:31:28.261709+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.261709+0000', 'last_undegraded': '2024-04-25T00:31:28.261709+0000', 'last_fullsized': '2024-04-25T00:31:28.261709+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:31:34.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-04-25T00:31:34.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2024-04-25T00:31:34.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.267496+0000', 'last_change': '2024-04-25T00:31:28.267496+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.267496+0000', 'last_undegraded': '2024-04-25T00:31:28.267496+0000', 'last_fullsized': '2024-04-25T00:31:28.267496+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:31:34.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-04-25T00:31:34.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2024-04-25T00:31:34.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.257881+0000', 'last_change': '2024-04-25T00:31:28.257881+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.257881+0000', 'last_undegraded': '2024-04-25T00:31:28.257881+0000', 'last_fullsized': '2024-04-25T00:31:28.257881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:31:34.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-04-25T00:31:34.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2024-04-25T00:31:34.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.258806+0000', 'last_change': '2024-04-25T00:31:28.258806+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.258806+0000', 'last_undegraded': '2024-04-25T00:31:28.258806+0000', 'last_fullsized': '2024-04-25T00:31:28.258806+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:31:34.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-04-25T00:31:34.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2024-04-25T00:31:34.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.266347+0000', 'last_change': '2024-04-25T00:31:28.266347+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.266347+0000', 'last_undegraded': '2024-04-25T00:31:28.266347+0000', 'last_fullsized': '2024-04-25T00:31:28.266347+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:31:34.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-04-25T00:31:34.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-04-25T00:31:34.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2024-04-25T00:31:34.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.269229+0000', 'last_change': '2024-04-25T00:31:28.269229+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.269229+0000', 'last_undegraded': '2024-04-25T00:31:28.269229+0000', 'last_fullsized': '2024-04-25T00:31:28.269229+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:31:34.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-04-25T00:31:34.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2024-04-25T00:31:34.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.268083+0000', 'last_change': '2024-04-25T00:31:28.268083+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.268083+0000', 'last_undegraded': '2024-04-25T00:31:28.268083+0000', 'last_fullsized': '2024-04-25T00:31:28.268083+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:31:34.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-04-25T00:31:34.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2024-04-25T00:31:34.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.260490+0000', 'last_change': '2024-04-25T00:31:28.260490+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.260490+0000', 'last_undegraded': '2024-04-25T00:31:28.260490+0000', 'last_fullsized': '2024-04-25T00:31:28.260490+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:31:34.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-04-25T00:31:34.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:31:29.245289+0000', 'last_change': '2024-04-25T00:31:29.245289+0000', 'last_active': '2024-04-25T00:31:29.245289+0000', 'last_peered': '2024-04-25T00:31:29.245289+0000', 'last_clean': '2024-04-25T00:31:29.245289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:29.245289+0000', 'last_undegraded': '2024-04-25T00:31:29.245289+0000', 'last_fullsized': '2024-04-25T00:31:29.245289+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:29.245289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:31:34.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2024-04-25T00:31:34.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:31:28.256066+0000', 'last_change': '2024-04-25T00:31:28.256066+0000', 'last_active': '2024-04-25T00:31:28.233437+0000', 'last_peered': '2024-04-25T00:31:28.233437+0000', 'last_clean': '2024-04-25T00:31:28.233437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:31:28.256066+0000', 'last_undegraded': '2024-04-25T00:31:28.256066+0000', 'last_fullsized': '2024-04-25T00:31:28.256066+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'last_clean_scrub_stamp': '2024-04-25T00:31:28.233437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:31:34.141 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:31:34.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.276+0000 7ffb1799c640 1 Processor -- start 2024-04-25T00:31:34.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.276+0000 7ffb1799c640 1 -- start start 2024-04-25T00:31:34.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.276+0000 7ffb1799c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffb100b10d0 0x7ffb100b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:34.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.276+0000 7ffb1799c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb1006c320 con 0x7ffb100c5400 2024-04-25T00:31:34.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.276+0000 7ffb1799c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb10166eb0 con 0x7ffb100cf8f0 2024-04-25T00:31:34.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.276+0000 7ffb1799c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ffb1006c5e0 con 0x7ffb100b10d0 2024-04-25T00:31:34.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.276+0000 7ffb15738640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffb100b10d0 0x7ffb100b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:34.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.276+0000 7ffb15f39640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ffb100c5400 0x7ffb100b1a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43248/0 (socket says 172.21.15.29:43248) 2024-04-25T00:31:34.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.276+0000 7ffb15f39640 1 -- 172.21.15.29:0/4072800630 learned_addr learned my addr 172.21.15.29:0/4072800630 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:34.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.276+0000 7ffb15738640 1 -- 172.21.15.29:0/4072800630 >> v1:172.21.15.157:6789/0 conn(0x7ffb100cf8f0 legacy=0x7ffb100c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:34.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.276+0000 7ffb15738640 1 -- 172.21.15.29:0/4072800630 >> v1:172.21.15.29:6789/0 conn(0x7ffb100c5400 legacy=0x7ffb100b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.276+0000 7ffb15738640 1 -- 172.21.15.29:0/4072800630 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1006c8c0 con 0x7ffb100b10d0 2024-04-25T00:31:34.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.280+0000 7ffb15738640 1 --2- 172.21.15.29:0/4072800630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb100b10d0 0x7ffb100b2c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0097c0 tx=0x7ffb0c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19116ec6aaba35e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:34.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.280+0000 7ffb07fff640 1 -- 172.21.15.29:0/4072800630 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0b0070 con 0x7ffb100b10d0 2024-04-25T00:31:34.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.280+0000 7ffb07fff640 1 -- 172.21.15.29:0/4072800630 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb0c0ab600 con 0x7ffb100b10d0 2024-04-25T00:31:34.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.280+0000 7ffb07fff640 1 -- 172.21.15.29:0/4072800630 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0ab9d0 con 0x7ffb100b10d0 2024-04-25T00:31:34.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.280+0000 7ffb1799c640 1 -- 172.21.15.29:0/4072800630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb100b10d0 msgr2=0x7ffb100b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.280+0000 7ffb1799c640 1 --2- 172.21.15.29:0/4072800630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb100b10d0 0x7ffb100b2c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0097c0 tx=0x7ffb0c01c460 comp rx=0 tx=0).stop 2024-04-25T00:31:34.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.280+0000 7ffb1799c640 1 -- 172.21.15.29:0/4072800630 shutdown_connections 2024-04-25T00:31:34.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.280+0000 7ffb1799c640 1 --2- 172.21.15.29:0/4072800630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb100b10d0 0x7ffb100b2c50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.280+0000 7ffb1799c640 1 -- 172.21.15.29:0/4072800630 >> 172.21.15.29:0/4072800630 conn(0x7ffb1009aa30 msgr2=0x7ffb1009ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:34.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.280+0000 7ffb1799c640 1 -- 172.21.15.29:0/4072800630 shutdown_connections 2024-04-25T00:31:34.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.280+0000 7ffb1799c640 1 -- 172.21.15.29:0/4072800630 wait complete. 2024-04-25T00:31:34.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb1799c640 1 Processor -- start 2024-04-25T00:31:34.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb1799c640 1 -- start start 2024-04-25T00:31:34.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb1799c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb100b10d0 0x7ffb100b6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:34.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb1799c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb100c5400 0x7ffb10162a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:34.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb1799c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb100cf8f0 0x7ffb100b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:34.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb1799c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffb100cfed0 con 0x7ffb100c5400 2024-04-25T00:31:34.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb1799c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb100cfd50 con 0x7ffb100cf8f0 2024-04-25T00:31:34.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb1799c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffb10166ad0 con 0x7ffb100b10d0 2024-04-25T00:31:34.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb14f37640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb100c5400 0x7ffb10162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:34.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb14f37640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb100c5400 0x7ffb10162a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51736/0 (socket says 172.21.15.29:51736) 2024-04-25T00:31:34.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb14f37640 1 -- 172.21.15.29:0/2004041365 learned_addr learned my addr 172.21.15.29:0/2004041365 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb14f37640 1 -- 172.21.15.29:0/2004041365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb100b10d0 msgr2=0x7ffb100b6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb14f37640 1 --2- 172.21.15.29:0/2004041365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb100b10d0 0x7ffb100b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb14f37640 1 -- 172.21.15.29:0/2004041365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb100cf8f0 msgr2=0x7ffb100b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb14f37640 1 --2- 172.21.15.29:0/2004041365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb100cf8f0 0x7ffb100b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb14f37640 1 -- 172.21.15.29:0/2004041365 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb100b6fc0 con 0x7ffb100c5400 2024-04-25T00:31:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb14f37640 1 --2- 172.21.15.29:0/2004041365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb100c5400 0x7ffb10162a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000c480 tx=0x7ffb0000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb05ffb640 1 -- 172.21.15.29:0/2004041365 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00012830 con 0x7ffb100c5400 2024-04-25T00:31:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb05ffb640 1 -- 172.21.15.29:0/2004041365 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb00017070 con 0x7ffb100c5400 2024-04-25T00:31:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb1799c640 1 -- 172.21.15.29:0/2004041365 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb100b72a0 con 0x7ffb100c5400 2024-04-25T00:31:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb05ffb640 1 -- 172.21.15.29:0/2004041365 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00012c30 con 0x7ffb100c5400 2024-04-25T00:31:34.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb1799c640 1 -- 172.21.15.29:0/2004041365 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1015d920 con 0x7ffb100c5400 2024-04-25T00:31:34.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb1799c640 1 -- 172.21.15.29:0/2004041365 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb100b77c0 con 0x7ffb100c5400 2024-04-25T00:31:34.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb05ffb640 1 -- 172.21.15.29:0/2004041365 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffb0001c460 con 0x7ffb100c5400 2024-04-25T00:31:34.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb05ffb640 1 --2- 172.21.15.29:0/2004041365 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffaf40678c0 0x7ffaf4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:34.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.284+0000 7ffb05ffb640 1 -- 172.21.15.29:0/2004041365 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffb00085ab0 con 0x7ffb100c5400 2024-04-25T00:31:34.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.288+0000 7ffb15738640 1 --2- 172.21.15.29:0/2004041365 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffaf40678c0 0x7ffaf4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:34.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.288+0000 7ffb15738640 1 --2- 172.21.15.29:0/2004041365 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffaf40678c0 0x7ffaf4069d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0ad000 tx=0x7ffb0c004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:34.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.288+0000 7ffb05ffb640 1 -- 172.21.15.29:0/2004041365 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffb00056680 con 0x7ffb100c5400 2024-04-25T00:31:34.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.508+0000 7ffb1799c640 1 -- 172.21.15.29:0/2004041365 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ffb1015d740 con 0x7ffb100c5400 2024-04-25T00:31:34.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.512+0000 7ffb05ffb640 1 -- 172.21.15.29:0/2004041365 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1107 (secure 0 0 0) 0x7ffb1015d740 con 0x7ffb100c5400 2024-04-25T00:31:34.513 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:34.513 INFO:teuthology.orchestra.run.smithi029.stdout:{"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":21,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1714005083,"num_in_osds":8,"osd_in_since":1714005074,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":257966080,"bytes_avail":772836147200,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"c","status":"up:active","gid":4252}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-25T00:31:12.661264+0000","services":{}},"progress_events":{}} 2024-04-25T00:31:34.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.512+0000 7ffb1799c640 1 -- 172.21.15.29:0/2004041365 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffaf40678c0 msgr2=0x7ffaf4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.512+0000 7ffb1799c640 1 --2- 172.21.15.29:0/2004041365 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffaf40678c0 0x7ffaf4069d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0ad000 tx=0x7ffb0c004740 comp rx=0 tx=0).stop 2024-04-25T00:31:34.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.512+0000 7ffb1799c640 1 -- 172.21.15.29:0/2004041365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb100c5400 msgr2=0x7ffb10162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.512+0000 7ffb1799c640 1 --2- 172.21.15.29:0/2004041365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb100c5400 0x7ffb10162a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000c480 tx=0x7ffb0000c940 comp rx=0 tx=0).stop 2024-04-25T00:31:34.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.512+0000 7ffb1799c640 1 -- 172.21.15.29:0/2004041365 shutdown_connections 2024-04-25T00:31:34.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.512+0000 7ffb1799c640 1 --2- 172.21.15.29:0/2004041365 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffaf40678c0 0x7ffaf4069d80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.512+0000 7ffb1799c640 1 --2- 172.21.15.29:0/2004041365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb100cf8f0 0x7ffb100b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.512+0000 7ffb1799c640 1 --2- 172.21.15.29:0/2004041365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb100c5400 0x7ffb10162a50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.512+0000 7ffb1799c640 1 --2- 172.21.15.29:0/2004041365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb100b10d0 0x7ffb100b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.512+0000 7ffb1799c640 1 -- 172.21.15.29:0/2004041365 >> 172.21.15.29:0/2004041365 conn(0x7ffb1009aa30 msgr2=0x7ffb100aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:34.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.512+0000 7ffb1799c640 1 -- 172.21.15.29:0/2004041365 shutdown_connections 2024-04-25T00:31:34.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.512+0000 7ffb1799c640 1 -- 172.21.15.29:0/2004041365 wait complete. 2024-04-25T00:31:34.531 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T00:31:34.532 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:31:34.539 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-04-25T00:31:34.536+0000 7f469ae65640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:31:34.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.652+0000 7fcd8f193640 1 Processor -- start 2024-04-25T00:31:34.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8f193640 1 -- start start 2024-04-25T00:31:34.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8f193640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8815ce60 0x7fcd8806b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8f193640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd88160190 con 0x7fcd8806b750 2024-04-25T00:31:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8f193640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd8815c680 con 0x7fcd88156420 2024-04-25T00:31:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8f193640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8806bb80 con 0x7fcd8815ce60 2024-04-25T00:31:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8d730640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcd8806b750 0x7fcd880cb290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43258/0 (socket says 172.21.15.29:43258) 2024-04-25T00:31:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8d730640 1 -- 172.21.15.29:0/4059015231 learned_addr learned my addr 172.21.15.29:0/4059015231 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd87fff640 1 --2- 172.21.15.29:0/4059015231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8815ce60 0x7fcd8806b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1184347522 0 0) 0x7fcd88160190 con 0x7fcd8806b750 2024-04-25T00:31:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd70003530 con 0x7fcd8806b750 2024-04-25T00:31:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1628613296 0 0) 0x7fcd70003530 con 0x7fcd8806b750 2024-04-25T00:31:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8815ce60 msgr2=0x7fcd8806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 --2- 172.21.15.29:0/4059015231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8815ce60 0x7fcd8806b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 >> v1:172.21.15.157:6789/0 conn(0x7fcd88156420 legacy=0x7fcd8815fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8815d2f0 con 0x7fcd8806b750 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fcd80001680 con 0x7fcd8806b750 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 >> v1:172.21.15.29:6789/0 conn(0x7fcd8806b750 legacy=0x7fcd880cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 --2- 172.21.15.29:0/4059015231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd700039f0 0x7fcd70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 --2- 172.21.15.29:0/4059015231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8815ce60 0x7fcd70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 --2- 172.21.15.29:0/4059015231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd70004d50 0x7fcd700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8806bb80 con 0x7fcd70004d50 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7000ba00 con 0x7fcd700039f0 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7000bc00 con 0x7fcd8815ce60 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8d730640 1 --2- 172.21.15.29:0/4059015231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd70004d50 0x7fcd700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8cf2f640 1 --2- 172.21.15.29:0/4059015231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd700039f0 0x7fcd70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd87fff640 1 --2- 172.21.15.29:0/4059015231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8815ce60 0x7fcd70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8d730640 1 -- 172.21.15.29:0/4059015231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8815ce60 msgr2=0x7fcd70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8d730640 1 --2- 172.21.15.29:0/4059015231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8815ce60 0x7fcd70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8d730640 1 -- 172.21.15.29:0/4059015231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd700039f0 msgr2=0x7fcd70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8d730640 1 --2- 172.21.15.29:0/4059015231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd700039f0 0x7fcd70003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.656+0000 7fcd8d730640 1 -- 172.21.15.29:0/4059015231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd70003530 con 0x7fcd70004d50 2024-04-25T00:31:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.660+0000 7fcd8d730640 1 --2- 172.21.15.29:0/4059015231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd70004d50 0x7fcd700051d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000c2c0 tx=0x7fcd8001f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745e1ffbcd94f274 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.660+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80007d60 con 0x7fcd70004d50 2024-04-25T00:31:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.660+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd80005b00 con 0x7fcd70004d50 2024-04-25T00:31:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.660+0000 7fcd877fe640 1 -- 172.21.15.29:0/4059015231 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80007070 con 0x7fcd70004d50 2024-04-25T00:31:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.660+0000 7fcd8f193640 1 -- 172.21.15.29:0/4059015231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd70004d50 msgr2=0x7fcd700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.660+0000 7fcd8f193640 1 --2- 172.21.15.29:0/4059015231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd70004d50 0x7fcd700051d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000c2c0 tx=0x7fcd8001f490 comp rx=0 tx=0).stop 2024-04-25T00:31:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.660+0000 7fcd8d730640 1 -- 172.21.15.29:0/4059015231 reap_dead start 2024-04-25T00:31:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.660+0000 7fcd8f193640 1 -- 172.21.15.29:0/4059015231 shutdown_connections 2024-04-25T00:31:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.660+0000 7fcd8f193640 1 -- 172.21.15.29:0/4059015231 >> 172.21.15.29:0/4059015231 conn(0x7fcd880d0c40 msgr2=0x7fcd880d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.660+0000 7fcd8f193640 1 -- 172.21.15.29:0/4059015231 shutdown_connections 2024-04-25T00:31:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.660+0000 7fcd8f193640 1 -- 172.21.15.29:0/4059015231 wait complete. 2024-04-25T00:31:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.660+0000 7fcd8f193640 1 Processor -- start 2024-04-25T00:31:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.660+0000 7fcd8f193640 1 -- start start 2024-04-25T00:31:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8f193640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd88155180 0x7fcd881f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8f193640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd88155560 0x7fcd88158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8f193640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd881593f0 0x7fcd880b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8f193640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8815c940 con 0x7fcd88155560 2024-04-25T00:31:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8f193640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd88160720 con 0x7fcd881593f0 2024-04-25T00:31:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8f193640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd88156800 con 0x7fcd88155180 2024-04-25T00:31:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd87fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd88155560 0x7fcd88158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd87fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd88155560 0x7fcd88158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51746/0 (socket says 172.21.15.29:51746) 2024-04-25T00:31:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd87fff640 1 -- 172.21.15.29:0/2940892194 learned_addr learned my addr 172.21.15.29:0/2940892194 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8cf2f640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd88155180 0x7fcd881f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8d730640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd881593f0 0x7fcd880b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd87fff640 1 -- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd88155180 msgr2=0x7fcd881f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd87fff640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd88155180 0x7fcd881f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd87fff640 1 -- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd881593f0 msgr2=0x7fcd880b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd87fff640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd881593f0 0x7fcd880b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd87fff640 1 -- 172.21.15.29:0/2940892194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd88156cf0 con 0x7fcd88155560 2024-04-25T00:31:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8d730640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd881593f0 0x7fcd880b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8cf2f640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd88155180 0x7fcd881f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd87fff640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd88155560 0x7fcd88158eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c011a00 tx=0x7fcd7c011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd857fa640 1 -- 172.21.15.29:0/2940892194 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00fb50 con 0x7fcd88155560 2024-04-25T00:31:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd857fa640 1 -- 172.21.15.29:0/2940892194 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd7c00fcf0 con 0x7fcd88155560 2024-04-25T00:31:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd857fa640 1 -- 172.21.15.29:0/2940892194 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00fea0 con 0x7fcd88155560 2024-04-25T00:31:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8f193640 1 -- 172.21.15.29:0/2940892194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd88156fd0 con 0x7fcd88155560 2024-04-25T00:31:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8f193640 1 -- 172.21.15.29:0/2940892194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd88157590 con 0x7fcd88155560 2024-04-25T00:31:34.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8f193640 1 -- 172.21.15.29:0/2940892194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd88159800 con 0x7fcd88155560 2024-04-25T00:31:34.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd857fa640 1 -- 172.21.15.29:0/2940892194 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcd7c012040 con 0x7fcd88155560 2024-04-25T00:31:34.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd857fa640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcd740678f0 0x7fcd74069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:34.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd857fa640 1 -- 172.21.15.29:0/2940892194 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcd7c086380 con 0x7fcd88155560 2024-04-25T00:31:34.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.664+0000 7fcd8cf2f640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcd740678f0 0x7fcd74069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:34.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.668+0000 7fcd8cf2f640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcd740678f0 0x7fcd74069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcd78000f90 tx=0x7fcd78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.668+0000 7fcd857fa640 1 -- 172.21.15.29:0/2940892194 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcd7c019030 con 0x7fcd88155560 2024-04-25T00:31:34.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.828+0000 7fcd8f193640 1 -- 172.21.15.29:0/2940892194 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd88069270 con 0x7fcd740678f0 2024-04-25T00:31:34.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.844+0000 7fcd857fa640 1 -- 172.21.15.29:0/2940892194 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240068 (secure 0 0 0) 0x7fcd88069270 con 0x7fcd740678f0 2024-04-25T00:31:34.850 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:34.851 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:31:34.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.848+0000 7fcd8f193640 1 -- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcd740678f0 msgr2=0x7fcd74069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.848+0000 7fcd8f193640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcd740678f0 0x7fcd74069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcd78000f90 tx=0x7fcd78009040 comp rx=0 tx=0).stop 2024-04-25T00:31:34.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.848+0000 7fcd8f193640 1 -- 172.21.15.29:0/2940892194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd88155560 msgr2=0x7fcd88158eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:34.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.848+0000 7fcd8f193640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd88155560 0x7fcd88158eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c011a00 tx=0x7fcd7c011ec0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.848+0000 7fcd8f193640 1 -- 172.21.15.29:0/2940892194 shutdown_connections 2024-04-25T00:31:34.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.848+0000 7fcd8f193640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcd740678f0 0x7fcd74069db0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.848+0000 7fcd8f193640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd881593f0 0x7fcd880b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.848+0000 7fcd8f193640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd88155560 0x7fcd88158eb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.848+0000 7fcd8f193640 1 --2- 172.21.15.29:0/2940892194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd88155180 0x7fcd881f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:34.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.848+0000 7fcd8f193640 1 -- 172.21.15.29:0/2940892194 >> 172.21.15.29:0/2940892194 conn(0x7fcd880d0c40 msgr2=0x7fcd880b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:34.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.848+0000 7fcd8f193640 1 -- 172.21.15.29:0/2940892194 shutdown_connections 2024-04-25T00:31:34.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:34.848+0000 7fcd8f193640 1 -- 172.21.15.29:0/2940892194 wait complete. 2024-04-25T00:31:34.868 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-25T00:31:33.444554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251920,"kb_used_data":2472,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722800,"statfs":{"total":773094113280,"available":772836147200,"internally_reserved":0,"allocated":2531328,"data_stored":1593408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.186286"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.258680+0000","last_change":"2024-04-25T00:31:28.258680+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.258680+0000","last_undegraded":"2024-04-25T00:31:28.258680+0000","last_fullsized":"2024-04-25T00:31:28.258680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260346+0000","last_change":"2024-04-25T00:31:28.260346+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260346+0000","last_undegraded":"2024-04-25T00:31:28.260346+0000","last_fullsized":"2024-04-25T00:31:28.260346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261252+0000","last_change":"2024-04-25T00:31:28.261252+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261252+0000","last_undegraded":"2024-04-25T00:31:28.261252+0000","last_fullsized":"2024-04-25T00:31:28.261252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252572+0000","last_change":"2024-04-25T00:31:28.252572+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252572+0000","last_undegraded":"2024-04-25T00:31:28.252572+0000","last_fullsized":"2024-04-25T00:31:28.252572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.264117+0000","last_change":"2024-04-25T00:31:28.264117+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.264117+0000","last_undegraded":"2024-04-25T00:31:28.264117+0000","last_fullsized":"2024-04-25T00:31:28.264117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252610+0000","last_change":"2024-04-25T00:31:28.252610+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252610+0000","last_undegraded":"2024-04-25T00:31:28.252610+0000","last_fullsized":"2024-04-25T00:31:28.252610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.259359+0000","last_change":"2024-04-25T00:31:28.259359+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.259359+0000","last_undegraded":"2024-04-25T00:31:28.259359+0000","last_fullsized":"2024-04-25T00:31:28.259359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.264603+0000","last_change":"2024-04-25T00:31:28.264603+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.264603+0000","last_undegraded":"2024-04-25T00:31:28.264603+0000","last_fullsized":"2024-04-25T00:31:28.264603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.255699+0000","last_change":"2024-04-25T00:31:28.255699+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.255699+0000","last_undegraded":"2024-04-25T00:31:28.255699+0000","last_fullsized":"2024-04-25T00:31:28.255699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.254358+0000","last_change":"2024-04-25T00:31:28.254358+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.254358+0000","last_undegraded":"2024-04-25T00:31:28.254358+0000","last_fullsized":"2024-04-25T00:31:28.254358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260583+0000","last_change":"2024-04-25T00:31:28.260583+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260583+0000","last_undegraded":"2024-04-25T00:31:28.260583+0000","last_fullsized":"2024-04-25T00:31:28.260583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261140+0000","last_change":"2024-04-25T00:31:28.261140+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261140+0000","last_undegraded":"2024-04-25T00:31:28.261140+0000","last_fullsized":"2024-04-25T00:31:28.261140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252595+0000","last_change":"2024-04-25T00:31:28.252595+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252595+0000","last_undegraded":"2024-04-25T00:31:28.252595+0000","last_fullsized":"2024-04-25T00:31:28.252595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260605+0000","last_change":"2024-04-25T00:31:28.260605+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260605+0000","last_undegraded":"2024-04-25T00:31:28.260605+0000","last_fullsized":"2024-04-25T00:31:28.260605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.258823+0000","last_change":"2024-04-25T00:31:28.258823+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.258823+0000","last_undegraded":"2024-04-25T00:31:28.258823+0000","last_fullsized":"2024-04-25T00:31:28.258823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261543+0000","last_change":"2024-04-25T00:31:28.261543+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261543+0000","last_undegraded":"2024-04-25T00:31:28.261543+0000","last_fullsized":"2024-04-25T00:31:28.261543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T00:31:28.252054+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:31:28.252054+0000","last_peered":"2024-04-25T00:31:28.252054+0000","last_clean":"2024-04-25T00:31:28.252054+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:31:28.252054+0000","last_undegraded":"2024-04-25T00:31:28.252054+0000","last_fullsized":"2024-04-25T00:31:28.252054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.253465+0000","last_change":"2024-04-25T00:31:28.253465+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.253465+0000","last_undegraded":"2024-04-25T00:31:28.253465+0000","last_fullsized":"2024-04-25T00:31:28.253465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257931+0000","last_change":"2024-04-25T00:31:28.257931+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257931+0000","last_undegraded":"2024-04-25T00:31:28.257931+0000","last_fullsized":"2024-04-25T00:31:28.257931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.259930+0000","last_change":"2024-04-25T00:31:28.259930+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.259930+0000","last_undegraded":"2024-04-25T00:31:28.259930+0000","last_fullsized":"2024-04-25T00:31:28.259930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.252566+0000","last_change":"2024-04-25T00:31:28.252566+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.252566+0000","last_undegraded":"2024-04-25T00:31:28.252566+0000","last_fullsized":"2024-04-25T00:31:28.252566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257713+0000","last_change":"2024-04-25T00:31:28.257713+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257713+0000","last_undegraded":"2024-04-25T00:31:28.257713+0000","last_fullsized":"2024-04-25T00:31:28.257713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261729+0000","last_change":"2024-04-25T00:31:28.261729+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261729+0000","last_undegraded":"2024-04-25T00:31:28.261729+0000","last_fullsized":"2024-04-25T00:31:28.261729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257418+0000","last_change":"2024-04-25T00:31:28.257418+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257418+0000","last_undegraded":"2024-04-25T00:31:28.257418+0000","last_fullsized":"2024-04-25T00:31:28.257418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.261709+0000","last_change":"2024-04-25T00:31:28.261709+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.261709+0000","last_undegraded":"2024-04-25T00:31:28.261709+0000","last_fullsized":"2024-04-25T00:31:28.261709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.267496+0000","last_change":"2024-04-25T00:31:28.267496+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.267496+0000","last_undegraded":"2024-04-25T00:31:28.267496+0000","last_fullsized":"2024-04-25T00:31:28.267496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.257881+0000","last_change":"2024-04-25T00:31:28.257881+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.257881+0000","last_undegraded":"2024-04-25T00:31:28.257881+0000","last_fullsized":"2024-04-25T00:31:28.257881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.258806+0000","last_change":"2024-04-25T00:31:28.258806+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.258806+0000","last_undegraded":"2024-04-25T00:31:28.258806+0000","last_fullsized":"2024-04-25T00:31:28.258806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.266347+0000","last_change":"2024-04-25T00:31:28.266347+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.266347+0000","last_undegraded":"2024-04-25T00:31:28.266347+0000","last_fullsized":"2024-04-25T00:31:28.266347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.269229+0000","last_change":"2024-04-25T00:31:28.269229+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.269229+0000","last_undegraded":"2024-04-25T00:31:28.269229+0000","last_fullsized":"2024-04-25T00:31:28.269229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.268083+0000","last_change":"2024-04-25T00:31:28.268083+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.268083+0000","last_undegraded":"2024-04-25T00:31:28.268083+0000","last_fullsized":"2024-04-25T00:31:28.268083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.260490+0000","last_change":"2024-04-25T00:31:28.260490+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.260490+0000","last_undegraded":"2024-04-25T00:31:28.260490+0000","last_fullsized":"2024-04-25T00:31:28.260490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:31:29.245289+0000","last_change":"2024-04-25T00:31:29.245289+0000","last_active":"2024-04-25T00:31:29.245289+0000","last_peered":"2024-04-25T00:31:29.245289+0000","last_clean":"2024-04-25T00:31:29.245289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:29.245289+0000","last_undegraded":"2024-04-25T00:31:29.245289+0000","last_fullsized":"2024-04-25T00:31:29.245289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T00:31:28.256066+0000","last_change":"2024-04-25T00:31:28.256066+0000","last_active":"2024-04-25T00:31:28.233437+0000","last_peered":"2024-04-25T00:31:28.233437+0000","last_clean":"2024-04-25T00:31:28.233437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:31:28.256066+0000","last_undegraded":"2024-04-25T00:31:28.256066+0000","last_fullsized":"2024-04-25T00:31:28.256066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:31:37.873 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:31:38.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f99834a7640 1 Processor -- start 2024-04-25T00:31:38.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f99834a7640 1 -- start start 2024-04-25T00:31:38.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f99834a7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f997c06b210 0x7f997c06db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f99834a7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f997c06b5f0 0x7f997c071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f99834a7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f997c0d0410 con 0x7f997c0d18f0 2024-04-25T00:31:38.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f99834a7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f997c0ce880 con 0x7f997c06b5f0 2024-04-25T00:31:38.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f99834a7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f997c0d06b0 con 0x7f997c06b210 2024-04-25T00:31:38.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f9980a42640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f997c0d18f0 0x7f997c15f1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:49950/0 (socket says 172.21.15.29:49950) 2024-04-25T00:31:38.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f9980a42640 1 -- 172.21.15.29:0/1683273318 learned_addr learned my addr 172.21.15.29:0/1683273318 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:38.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f9981a44640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f997c06b5f0 0x7f997c071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f9981243640 1 --2- 172.21.15.29:0/1683273318 >> v2:172.21.15.157:3300/0 conn(0x7f997c06b210 0x7f997c06db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f9981a44640 1 -- 172.21.15.29:0/1683273318 >> v2:172.21.15.157:3300/0 conn(0x7f997c06b210 msgr2=0x7f997c06db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:38.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f9981a44640 1 --2- 172.21.15.29:0/1683273318 >> v2:172.21.15.157:3300/0 conn(0x7f997c06b210 0x7f997c06db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f9981a44640 1 -- 172.21.15.29:0/1683273318 >> v1:172.21.15.29:6789/0 conn(0x7f997c0d18f0 legacy=0x7f997c15f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f997bfff640 1 -- 172.21.15.29:0/1683273318 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3880085944 0 0) 0x7f997c0d0410 con 0x7f997c0d18f0 2024-04-25T00:31:38.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.008+0000 7f9981a44640 1 -- 172.21.15.29:0/1683273318 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c0d0990 con 0x7f997c06b5f0 2024-04-25T00:31:38.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f9981a44640 1 --2- 172.21.15.29:0/1683273318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c06b5f0 0x7f997c071fc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f997c075730 tx=0x7f996c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d459581419e91c16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:38.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f997bfff640 1 -- 172.21.15.29:0/1683273318 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c0b0070 con 0x7f997c06b5f0 2024-04-25T00:31:38.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f997bfff640 1 -- 172.21.15.29:0/1683273318 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996c0ab600 con 0x7f997c06b5f0 2024-04-25T00:31:38.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f997bfff640 1 -- 172.21.15.29:0/1683273318 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c0ab9d0 con 0x7f997c06b5f0 2024-04-25T00:31:38.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 -- 172.21.15.29:0/1683273318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c06b5f0 msgr2=0x7f997c071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 --2- 172.21.15.29:0/1683273318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c06b5f0 0x7f997c071fc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f997c075730 tx=0x7f996c01c480 comp rx=0 tx=0).stop 2024-04-25T00:31:38.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 -- 172.21.15.29:0/1683273318 shutdown_connections 2024-04-25T00:31:38.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 --2- 172.21.15.29:0/1683273318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c06b5f0 0x7f997c071fc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 --2- 172.21.15.29:0/1683273318 >> v2:172.21.15.157:3300/0 conn(0x7f997c06b210 0x7f997c06db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 -- 172.21.15.29:0/1683273318 >> 172.21.15.29:0/1683273318 conn(0x7f997c0d0cb0 msgr2=0x7f997c0d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:38.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 -- 172.21.15.29:0/1683273318 shutdown_connections 2024-04-25T00:31:38.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 -- 172.21.15.29:0/1683273318 wait complete. 2024-04-25T00:31:38.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 Processor -- start 2024-04-25T00:31:38.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 -- start start 2024-04-25T00:31:38.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c06b210 0x7f997c160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c06b5f0 0x7f997c0c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c0d18f0 0x7f997c151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f997c16adb0 con 0x7f997c06b5f0 2024-04-25T00:31:38.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f997c158b40 con 0x7f997c0d18f0 2024-04-25T00:31:38.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f99834a7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f997c16af30 con 0x7f997c06b210 2024-04-25T00:31:38.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f9980a42640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c06b5f0 0x7f997c0c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f9980a42640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c06b5f0 0x7f997c0c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49532/0 (socket says 172.21.15.29:49532) 2024-04-25T00:31:38.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f9980a42640 1 -- 172.21.15.29:0/549886151 learned_addr learned my addr 172.21.15.29:0/549886151 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:38.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f9981a44640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c0d18f0 0x7f997c151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f9981243640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c06b210 0x7f997c160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f9980a42640 1 -- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c06b210 msgr2=0x7f997c160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f9980a42640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c06b210 0x7f997c160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f9980a42640 1 -- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c0d18f0 msgr2=0x7f997c151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f9980a42640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c0d18f0 0x7f997c151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f9980a42640 1 -- 172.21.15.29:0/549886151 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c15eff0 con 0x7f997c06b5f0 2024-04-25T00:31:38.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f9981243640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c06b210 0x7f997c160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:38.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.012+0000 7f9980a42640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c06b5f0 0x7f997c0c43e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f997000c4b0 tx=0x7f997000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:38.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.016+0000 7f9979ffb640 1 -- 172.21.15.29:0/549886151 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970018070 con 0x7f997c06b5f0 2024-04-25T00:31:38.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.016+0000 7f9979ffb640 1 -- 172.21.15.29:0/549886151 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9970007500 con 0x7f997c06b5f0 2024-04-25T00:31:38.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.016+0000 7f9979ffb640 1 -- 172.21.15.29:0/549886151 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970004a30 con 0x7f997c06b5f0 2024-04-25T00:31:38.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.016+0000 7f99834a7640 1 -- 172.21.15.29:0/549886151 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997c15f2d0 con 0x7f997c06b5f0 2024-04-25T00:31:38.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.016+0000 7f99834a7640 1 -- 172.21.15.29:0/549886151 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997c151cd0 con 0x7f997c06b5f0 2024-04-25T00:31:38.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.016+0000 7f99834a7640 1 -- 172.21.15.29:0/549886151 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f997c069270 con 0x7f997c06b5f0 2024-04-25T00:31:38.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.016+0000 7f9979ffb640 1 -- 172.21.15.29:0/549886151 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9970004bd0 con 0x7f997c06b5f0 2024-04-25T00:31:38.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.016+0000 7f9979ffb640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f99680678c0 0x7f9968069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.016+0000 7f9979ffb640 1 -- 172.21.15.29:0/549886151 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9970084f80 con 0x7f997c06b5f0 2024-04-25T00:31:38.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.016+0000 7f9981243640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f99680678c0 0x7f9968069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.016+0000 7f9981243640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f99680678c0 0x7f9968069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f997c158850 tx=0x7f997400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:38.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.020+0000 7f9979ffb640 1 -- 172.21.15.29:0/549886151 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9970055b70 con 0x7f997c06b5f0 2024-04-25T00:31:38.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.180+0000 7f99834a7640 1 -- 172.21.15.29:0/549886151 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f997c161230 con 0x7f99680678c0 2024-04-25T00:31:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.196+0000 7f9979ffb640 1 -- 172.21.15.29:0/549886151 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251960 (secure 0 0 0) 0x7f997c161230 con 0x7f99680678c0 2024-04-25T00:31:38.203 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:38.205 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:31:38.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.200+0000 7f99834a7640 1 -- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f99680678c0 msgr2=0x7f9968069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.200+0000 7f99834a7640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f99680678c0 0x7f9968069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f997c158850 tx=0x7f997400b040 comp rx=0 tx=0).stop 2024-04-25T00:31:38.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.200+0000 7f99834a7640 1 -- 172.21.15.29:0/549886151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c06b5f0 msgr2=0x7f997c0c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.200+0000 7f99834a7640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c06b5f0 0x7f997c0c43e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f997000c4b0 tx=0x7f997000c970 comp rx=0 tx=0).stop 2024-04-25T00:31:38.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.200+0000 7f99834a7640 1 -- 172.21.15.29:0/549886151 shutdown_connections 2024-04-25T00:31:38.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.200+0000 7f99834a7640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f99680678c0 0x7f9968069d80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.200+0000 7f99834a7640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c0d18f0 0x7f997c151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.200+0000 7f99834a7640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c06b5f0 0x7f997c0c43e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.200+0000 7f99834a7640 1 --2- 172.21.15.29:0/549886151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c06b210 0x7f997c160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.200+0000 7f99834a7640 1 -- 172.21.15.29:0/549886151 >> 172.21.15.29:0/549886151 conn(0x7f997c0d0cb0 msgr2=0x7f997c0ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:38.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.200+0000 7f99834a7640 1 -- 172.21.15.29:0/549886151 shutdown_connections 2024-04-25T00:31:38.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.200+0000 7f99834a7640 1 -- 172.21.15.29:0/549886151 wait complete. 2024-04-25T00:31:38.220 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-25T00:31:37.446073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252264,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722456,"statfs":{"total":773094113280,"available":772835794944,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":60,"apply_latency_ms":60,"commit_latency_ns":60000000,"apply_latency_ns":60000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002765"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339973+0000","last_change":"2024-04-25T00:31:30.311935+0000","last_active":"2024-04-25T00:31:32.339973+0000","last_peered":"2024-04-25T00:31:32.339973+0000","last_clean":"2024-04-25T00:31:32.339973+0000","last_became_active":"2024-04-25T00:31:30.311276+0000","last_became_peered":"2024-04-25T00:31:30.311276+0000","last_unstale":"2024-04-25T00:31:32.339973+0000","last_undegraded":"2024-04-25T00:31:32.339973+0000","last_fullsized":"2024-04-25T00:31:32.339973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:00.435231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289979+0000","last_change":"2024-04-25T00:31:30.286809+0000","last_active":"2024-04-25T00:31:31.289979+0000","last_peered":"2024-04-25T00:31:31.289979+0000","last_clean":"2024-04-25T00:31:31.289979+0000","last_became_active":"2024-04-25T00:31:30.286285+0000","last_became_peered":"2024-04-25T00:31:30.286285+0000","last_unstale":"2024-04-25T00:31:31.289979+0000","last_undegraded":"2024-04-25T00:31:31.289979+0000","last_fullsized":"2024-04-25T00:31:31.289979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339468+0000","last_change":"2024-04-25T00:31:30.295646+0000","last_active":"2024-04-25T00:31:32.339468+0000","last_peered":"2024-04-25T00:31:32.339468+0000","last_clean":"2024-04-25T00:31:32.339468+0000","last_became_active":"2024-04-25T00:31:30.295302+0000","last_became_peered":"2024-04-25T00:31:30.295302+0000","last_unstale":"2024-04-25T00:31:32.339468+0000","last_undegraded":"2024-04-25T00:31:32.339468+0000","last_fullsized":"2024-04-25T00:31:32.339468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293552+0000","last_change":"2024-04-25T00:31:30.317738+0000","last_active":"2024-04-25T00:31:31.293552+0000","last_peered":"2024-04-25T00:31:31.293552+0000","last_clean":"2024-04-25T00:31:31.293552+0000","last_became_active":"2024-04-25T00:31:30.315216+0000","last_became_peered":"2024-04-25T00:31:30.315216+0000","last_unstale":"2024-04-25T00:31:31.293552+0000","last_undegraded":"2024-04-25T00:31:31.293552+0000","last_fullsized":"2024-04-25T00:31:31.293552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.340299+0000","last_change":"2024-04-25T00:31:30.308169+0000","last_active":"2024-04-25T00:31:32.340299+0000","last_peered":"2024-04-25T00:31:32.340299+0000","last_clean":"2024-04-25T00:31:32.340299+0000","last_became_active":"2024-04-25T00:31:30.307068+0000","last_became_peered":"2024-04-25T00:31:30.307068+0000","last_unstale":"2024-04-25T00:31:32.340299+0000","last_undegraded":"2024-04-25T00:31:32.340299+0000","last_fullsized":"2024-04-25T00:31:32.340299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:44.328314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293940+0000","last_change":"2024-04-25T00:31:30.318086+0000","last_active":"2024-04-25T00:31:31.293940+0000","last_peered":"2024-04-25T00:31:31.293940+0000","last_clean":"2024-04-25T00:31:31.293940+0000","last_became_active":"2024-04-25T00:31:30.314553+0000","last_became_peered":"2024-04-25T00:31:30.314553+0000","last_unstale":"2024-04-25T00:31:31.293940+0000","last_undegraded":"2024-04-25T00:31:31.293940+0000","last_fullsized":"2024-04-25T00:31:31.293940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:12.015188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338903+0000","last_change":"2024-04-25T00:31:30.323525+0000","last_active":"2024-04-25T00:31:32.338903+0000","last_peered":"2024-04-25T00:31:32.338903+0000","last_clean":"2024-04-25T00:31:32.338903+0000","last_became_active":"2024-04-25T00:31:30.322985+0000","last_became_peered":"2024-04-25T00:31:30.322985+0000","last_unstale":"2024-04-25T00:31:32.338903+0000","last_undegraded":"2024-04-25T00:31:32.338903+0000","last_fullsized":"2024-04-25T00:31:32.338903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:37.829118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337996+0000","last_change":"2024-04-25T00:31:30.302700+0000","last_active":"2024-04-25T00:31:32.337996+0000","last_peered":"2024-04-25T00:31:32.337996+0000","last_clean":"2024-04-25T00:31:32.337996+0000","last_became_active":"2024-04-25T00:31:30.302417+0000","last_became_peered":"2024-04-25T00:31:30.302417+0000","last_unstale":"2024-04-25T00:31:32.337996+0000","last_undegraded":"2024-04-25T00:31:32.337996+0000","last_fullsized":"2024-04-25T00:31:32.337996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:26.475549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.340068+0000","last_change":"2024-04-25T00:31:30.322870+0000","last_active":"2024-04-25T00:31:32.340068+0000","last_peered":"2024-04-25T00:31:32.340068+0000","last_clean":"2024-04-25T00:31:32.340068+0000","last_became_active":"2024-04-25T00:31:30.322271+0000","last_became_peered":"2024-04-25T00:31:30.322271+0000","last_unstale":"2024-04-25T00:31:32.340068+0000","last_undegraded":"2024-04-25T00:31:32.340068+0000","last_fullsized":"2024-04-25T00:31:32.340068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:27.228158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273989+0000","last_change":"2024-04-25T00:31:30.299652+0000","last_active":"2024-04-25T00:31:32.273989+0000","last_peered":"2024-04-25T00:31:32.273989+0000","last_clean":"2024-04-25T00:31:32.273989+0000","last_became_active":"2024-04-25T00:31:30.298705+0000","last_became_peered":"2024-04-25T00:31:30.298705+0000","last_unstale":"2024-04-25T00:31:32.273989+0000","last_undegraded":"2024-04-25T00:31:32.273989+0000","last_fullsized":"2024-04-25T00:31:32.273989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273870+0000","last_change":"2024-04-25T00:31:30.305174+0000","last_active":"2024-04-25T00:31:32.273870+0000","last_peered":"2024-04-25T00:31:32.273870+0000","last_clean":"2024-04-25T00:31:32.273870+0000","last_became_active":"2024-04-25T00:31:30.299316+0000","last_became_peered":"2024-04-25T00:31:30.299316+0000","last_unstale":"2024-04-25T00:31:32.273870+0000","last_undegraded":"2024-04-25T00:31:32.273870+0000","last_fullsized":"2024-04-25T00:31:32.273870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295466+0000","last_change":"2024-04-25T00:31:30.313575+0000","last_active":"2024-04-25T00:31:31.295466+0000","last_peered":"2024-04-25T00:31:31.295466+0000","last_clean":"2024-04-25T00:31:31.295466+0000","last_became_active":"2024-04-25T00:31:30.312375+0000","last_became_peered":"2024-04-25T00:31:30.312375+0000","last_unstale":"2024-04-25T00:31:31.295466+0000","last_undegraded":"2024-04-25T00:31:31.295466+0000","last_fullsized":"2024-04-25T00:31:31.295466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:07.140320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338585+0000","last_change":"2024-04-25T00:31:30.301894+0000","last_active":"2024-04-25T00:31:32.338585+0000","last_peered":"2024-04-25T00:31:32.338585+0000","last_clean":"2024-04-25T00:31:32.338585+0000","last_became_active":"2024-04-25T00:31:30.301566+0000","last_became_peered":"2024-04-25T00:31:30.301566+0000","last_unstale":"2024-04-25T00:31:32.338585+0000","last_undegraded":"2024-04-25T00:31:32.338585+0000","last_fullsized":"2024-04-25T00:31:32.338585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.057143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339233+0000","last_change":"2024-04-25T00:31:30.295471+0000","last_active":"2024-04-25T00:31:32.339233+0000","last_peered":"2024-04-25T00:31:32.339233+0000","last_clean":"2024-04-25T00:31:32.339233+0000","last_became_active":"2024-04-25T00:31:30.295041+0000","last_became_peered":"2024-04-25T00:31:30.295041+0000","last_unstale":"2024-04-25T00:31:32.339233+0000","last_undegraded":"2024-04-25T00:31:32.339233+0000","last_fullsized":"2024-04-25T00:31:32.339233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:39.536187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289654+0000","last_change":"2024-04-25T00:31:30.329846+0000","last_active":"2024-04-25T00:31:31.289654+0000","last_peered":"2024-04-25T00:31:31.289654+0000","last_clean":"2024-04-25T00:31:31.289654+0000","last_became_active":"2024-04-25T00:31:30.329327+0000","last_became_peered":"2024-04-25T00:31:30.329327+0000","last_unstale":"2024-04-25T00:31:31.289654+0000","last_undegraded":"2024-04-25T00:31:31.289654+0000","last_fullsized":"2024-04-25T00:31:31.289654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293587+0000","last_change":"2024-04-25T00:31:30.317089+0000","last_active":"2024-04-25T00:31:31.293587+0000","last_peered":"2024-04-25T00:31:31.293587+0000","last_clean":"2024-04-25T00:31:31.293587+0000","last_became_active":"2024-04-25T00:31:30.314593+0000","last_became_peered":"2024-04-25T00:31:30.314593+0000","last_unstale":"2024-04-25T00:31:31.293587+0000","last_undegraded":"2024-04-25T00:31:31.293587+0000","last_fullsized":"2024-04-25T00:31:31.293587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.057143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295132+0000","last_change":"2024-04-25T00:31:30.330739+0000","last_active":"2024-04-25T00:31:31.295132+0000","last_peered":"2024-04-25T00:31:31.295132+0000","last_clean":"2024-04-25T00:31:31.295132+0000","last_became_active":"2024-04-25T00:31:30.330173+0000","last_became_peered":"2024-04-25T00:31:30.330173+0000","last_unstale":"2024-04-25T00:31:31.295132+0000","last_undegraded":"2024-04-25T00:31:31.295132+0000","last_fullsized":"2024-04-25T00:31:31.295132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:51.764905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337280+0000","last_change":"2024-04-25T00:31:30.310551+0000","last_active":"2024-04-25T00:31:32.337280+0000","last_peered":"2024-04-25T00:31:32.337280+0000","last_clean":"2024-04-25T00:31:32.337280+0000","last_became_active":"2024-04-25T00:31:30.310236+0000","last_became_peered":"2024-04-25T00:31:30.310236+0000","last_unstale":"2024-04-25T00:31:32.337280+0000","last_undegraded":"2024-04-25T00:31:32.337280+0000","last_fullsized":"2024-04-25T00:31:32.337280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:50.429652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339723+0000","last_change":"2024-04-25T00:31:30.318248+0000","last_active":"2024-04-25T00:31:32.339723+0000","last_peered":"2024-04-25T00:31:32.339723+0000","last_clean":"2024-04-25T00:31:32.339723+0000","last_became_active":"2024-04-25T00:31:30.317643+0000","last_became_peered":"2024-04-25T00:31:30.317643+0000","last_unstale":"2024-04-25T00:31:32.339723+0000","last_undegraded":"2024-04-25T00:31:32.339723+0000","last_fullsized":"2024-04-25T00:31:32.339723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:02.546759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273823+0000","last_change":"2024-04-25T00:31:30.316790+0000","last_active":"2024-04-25T00:31:32.273823+0000","last_peered":"2024-04-25T00:31:32.273823+0000","last_clean":"2024-04-25T00:31:32.273823+0000","last_became_active":"2024-04-25T00:31:30.315664+0000","last_became_peered":"2024-04-25T00:31:30.315664+0000","last_unstale":"2024-04-25T00:31:32.273823+0000","last_undegraded":"2024-04-25T00:31:32.273823+0000","last_fullsized":"2024-04-25T00:31:32.273823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339315+0000","last_change":"2024-04-25T00:31:30.322906+0000","last_active":"2024-04-25T00:31:32.339315+0000","last_peered":"2024-04-25T00:31:32.339315+0000","last_clean":"2024-04-25T00:31:32.339315+0000","last_became_active":"2024-04-25T00:31:30.322626+0000","last_became_peered":"2024-04-25T00:31:30.322626+0000","last_unstale":"2024-04-25T00:31:32.339315+0000","last_undegraded":"2024-04-25T00:31:32.339315+0000","last_fullsized":"2024-04-25T00:31:32.339315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:26.475549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293998+0000","last_change":"2024-04-25T00:31:30.327130+0000","last_active":"2024-04-25T00:31:31.293998+0000","last_peered":"2024-04-25T00:31:31.293998+0000","last_clean":"2024-04-25T00:31:31.293998+0000","last_became_active":"2024-04-25T00:31:30.326309+0000","last_became_peered":"2024-04-25T00:31:30.326309+0000","last_unstale":"2024-04-25T00:31:31.293998+0000","last_undegraded":"2024-04-25T00:31:31.293998+0000","last_fullsized":"2024-04-25T00:31:31.293998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:06.663363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338079+0000","last_change":"2024-04-25T00:31:30.329677+0000","last_active":"2024-04-25T00:31:32.338079+0000","last_peered":"2024-04-25T00:31:32.338079+0000","last_clean":"2024-04-25T00:31:32.338079+0000","last_became_active":"2024-04-25T00:31:30.329278+0000","last_became_peered":"2024-04-25T00:31:30.329278+0000","last_unstale":"2024-04-25T00:31:32.338079+0000","last_undegraded":"2024-04-25T00:31:32.338079+0000","last_fullsized":"2024-04-25T00:31:32.338079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:24.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.292924+0000","last_change":"2024-04-25T00:31:30.317304+0000","last_active":"2024-04-25T00:31:31.292924+0000","last_peered":"2024-04-25T00:31:31.292924+0000","last_clean":"2024-04-25T00:31:31.292924+0000","last_became_active":"2024-04-25T00:31:30.314963+0000","last_became_peered":"2024-04-25T00:31:30.314963+0000","last_unstale":"2024-04-25T00:31:31.292924+0000","last_undegraded":"2024-04-25T00:31:31.292924+0000","last_fullsized":"2024-04-25T00:31:31.292924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338403+0000","last_change":"2024-04-25T00:31:30.329416+0000","last_active":"2024-04-25T00:31:32.338403+0000","last_peered":"2024-04-25T00:31:32.338403+0000","last_clean":"2024-04-25T00:31:32.338403+0000","last_became_active":"2024-04-25T00:31:30.329055+0000","last_became_peered":"2024-04-25T00:31:30.329055+0000","last_unstale":"2024-04-25T00:31:32.338403+0000","last_undegraded":"2024-04-25T00:31:32.338403+0000","last_fullsized":"2024-04-25T00:31:32.338403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.274281+0000","last_change":"2024-04-25T00:31:30.325387+0000","last_active":"2024-04-25T00:31:32.274281+0000","last_peered":"2024-04-25T00:31:32.274281+0000","last_clean":"2024-04-25T00:31:32.274281+0000","last_became_active":"2024-04-25T00:31:30.324786+0000","last_became_peered":"2024-04-25T00:31:30.324786+0000","last_unstale":"2024-04-25T00:31:32.274281+0000","last_undegraded":"2024-04-25T00:31:32.274281+0000","last_fullsized":"2024-04-25T00:31:32.274281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.057143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289883+0000","last_change":"2024-04-25T00:31:30.288677+0000","last_active":"2024-04-25T00:31:31.289883+0000","last_peered":"2024-04-25T00:31:31.289883+0000","last_clean":"2024-04-25T00:31:31.289883+0000","last_became_active":"2024-04-25T00:31:30.288327+0000","last_became_peered":"2024-04-25T00:31:30.288327+0000","last_unstale":"2024-04-25T00:31:31.289883+0000","last_undegraded":"2024-04-25T00:31:31.289883+0000","last_fullsized":"2024-04-25T00:31:31.289883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338740+0000","last_change":"2024-04-25T00:31:30.287147+0000","last_active":"2024-04-25T00:31:32.338740+0000","last_peered":"2024-04-25T00:31:32.338740+0000","last_clean":"2024-04-25T00:31:32.338740+0000","last_became_active":"2024-04-25T00:31:30.286375+0000","last_became_peered":"2024-04-25T00:31:30.286375+0000","last_unstale":"2024-04-25T00:31:32.338740+0000","last_undegraded":"2024-04-25T00:31:32.338740+0000","last_fullsized":"2024-04-25T00:31:32.338740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:06.663363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339232+0000","last_change":"2024-04-25T00:31:30.306816+0000","last_active":"2024-04-25T00:31:32.339232+0000","last_peered":"2024-04-25T00:31:32.339232+0000","last_clean":"2024-04-25T00:31:32.339232+0000","last_became_active":"2024-04-25T00:31:30.306476+0000","last_became_peered":"2024-04-25T00:31:30.306476+0000","last_unstale":"2024-04-25T00:31:32.339232+0000","last_undegraded":"2024-04-25T00:31:32.339232+0000","last_fullsized":"2024-04-25T00:31:32.339232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:30.262696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294366+0000","last_change":"2024-04-25T00:31:30.317416+0000","last_active":"2024-04-25T00:31:31.294366+0000","last_peered":"2024-04-25T00:31:31.294366+0000","last_clean":"2024-04-25T00:31:31.294366+0000","last_became_active":"2024-04-25T00:31:30.315307+0000","last_became_peered":"2024-04-25T00:31:30.315307+0000","last_unstale":"2024-04-25T00:31:31.294366+0000","last_undegraded":"2024-04-25T00:31:31.294366+0000","last_fullsized":"2024-04-25T00:31:31.294366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:56.905979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337140+0000","last_change":"2024-04-25T00:31:30.299856+0000","last_active":"2024-04-25T00:31:32.337140+0000","last_peered":"2024-04-25T00:31:32.337140+0000","last_clean":"2024-04-25T00:31:32.337140+0000","last_became_active":"2024-04-25T00:31:30.299616+0000","last_became_peered":"2024-04-25T00:31:30.299616+0000","last_unstale":"2024-04-25T00:31:32.337140+0000","last_undegraded":"2024-04-25T00:31:32.337140+0000","last_fullsized":"2024-04-25T00:31:32.337140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:08.442361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273483+0000","last_change":"2024-04-25T00:31:30.304725+0000","last_active":"2024-04-25T00:31:32.273483+0000","last_peered":"2024-04-25T00:31:32.273483+0000","last_clean":"2024-04-25T00:31:32.273483+0000","last_became_active":"2024-04-25T00:31:30.298142+0000","last_became_peered":"2024-04-25T00:31:30.298142+0000","last_unstale":"2024-04-25T00:31:32.273483+0000","last_undegraded":"2024-04-25T00:31:32.273483+0000","last_fullsized":"2024-04-25T00:31:32.273483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293069+0000","last_change":"2024-04-25T00:31:30.304440+0000","last_active":"2024-04-25T00:31:31.293069+0000","last_peered":"2024-04-25T00:31:31.293069+0000","last_clean":"2024-04-25T00:31:31.293069+0000","last_became_active":"2024-04-25T00:31:30.303603+0000","last_became_peered":"2024-04-25T00:31:30.303603+0000","last_unstale":"2024-04-25T00:31:31.293069+0000","last_undegraded":"2024-04-25T00:31:31.293069+0000","last_fullsized":"2024-04-25T00:31:31.293069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:13.138372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338669+0000","last_change":"2024-04-25T00:31:29.306305+0000","last_active":"2024-04-25T00:31:32.338669+0000","last_peered":"2024-04-25T00:31:32.338669+0000","last_clean":"2024-04-25T00:31:32.338669+0000","last_became_active":"2024-04-25T00:31:29.305825+0000","last_became_peered":"2024-04-25T00:31:29.305825+0000","last_unstale":"2024-04-25T00:31:32.338669+0000","last_undegraded":"2024-04-25T00:31:32.338669+0000","last_fullsized":"2024-04-25T00:31:32.338669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:12.847032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339405+0000","last_change":"2024-04-25T00:31:30.280578+0000","last_active":"2024-04-25T00:31:32.339405+0000","last_peered":"2024-04-25T00:31:32.339405+0000","last_clean":"2024-04-25T00:31:32.339405+0000","last_became_active":"2024-04-25T00:31:30.278616+0000","last_became_peered":"2024-04-25T00:31:30.278616+0000","last_unstale":"2024-04-25T00:31:32.339405+0000","last_undegraded":"2024-04-25T00:31:32.339405+0000","last_fullsized":"2024-04-25T00:31:32.339405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:24.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273089+0000","last_change":"2024-04-25T00:31:29.295550+0000","last_active":"2024-04-25T00:31:32.273089+0000","last_peered":"2024-04-25T00:31:32.273089+0000","last_clean":"2024-04-25T00:31:32.273089+0000","last_became_active":"2024-04-25T00:31:29.294791+0000","last_became_peered":"2024-04-25T00:31:29.294791+0000","last_unstale":"2024-04-25T00:31:32.273089+0000","last_undegraded":"2024-04-25T00:31:32.273089+0000","last_fullsized":"2024-04-25T00:31:32.273089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:28.847603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338214+0000","last_change":"2024-04-25T00:31:29.295660+0000","last_active":"2024-04-25T00:31:32.338214+0000","last_peered":"2024-04-25T00:31:32.338214+0000","last_clean":"2024-04-25T00:31:32.338214+0000","last_became_active":"2024-04-25T00:31:29.295125+0000","last_became_peered":"2024-04-25T00:31:29.295125+0000","last_unstale":"2024-04-25T00:31:32.338214+0000","last_undegraded":"2024-04-25T00:31:32.338214+0000","last_fullsized":"2024-04-25T00:31:32.338214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:43.126464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338169+0000","last_change":"2024-04-25T00:31:30.301078+0000","last_active":"2024-04-25T00:31:32.338169+0000","last_peered":"2024-04-25T00:31:32.338169+0000","last_clean":"2024-04-25T00:31:32.338169+0000","last_became_active":"2024-04-25T00:31:30.298382+0000","last_became_peered":"2024-04-25T00:31:30.298382+0000","last_unstale":"2024-04-25T00:31:32.338169+0000","last_undegraded":"2024-04-25T00:31:32.338169+0000","last_fullsized":"2024-04-25T00:31:32.338169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293054+0000","last_change":"2024-04-25T00:31:29.273641+0000","last_active":"2024-04-25T00:31:31.293054+0000","last_peered":"2024-04-25T00:31:31.293054+0000","last_clean":"2024-04-25T00:31:31.293054+0000","last_became_active":"2024-04-25T00:31:29.272664+0000","last_became_peered":"2024-04-25T00:31:29.272664+0000","last_unstale":"2024-04-25T00:31:31.293054+0000","last_undegraded":"2024-04-25T00:31:31.293054+0000","last_fullsized":"2024-04-25T00:31:31.293054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:12.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289635+0000","last_change":"2024-04-25T00:31:30.287860+0000","last_active":"2024-04-25T00:31:31.289635+0000","last_peered":"2024-04-25T00:31:31.289635+0000","last_clean":"2024-04-25T00:31:31.289635+0000","last_became_active":"2024-04-25T00:31:30.287518+0000","last_became_peered":"2024-04-25T00:31:30.287518+0000","last_unstale":"2024-04-25T00:31:31.289635+0000","last_undegraded":"2024-04-25T00:31:31.289635+0000","last_fullsized":"2024-04-25T00:31:31.289635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:24.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.274623+0000","last_change":"2024-04-25T00:31:29.300290+0000","last_active":"2024-04-25T00:31:32.274623+0000","last_peered":"2024-04-25T00:31:32.274623+0000","last_clean":"2024-04-25T00:31:32.274623+0000","last_became_active":"2024-04-25T00:31:29.297912+0000","last_became_peered":"2024-04-25T00:31:29.297912+0000","last_unstale":"2024-04-25T00:31:32.274623+0000","last_undegraded":"2024-04-25T00:31:32.274623+0000","last_fullsized":"2024-04-25T00:31:32.274623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:06.128468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294033+0000","last_change":"2024-04-25T00:31:30.323107+0000","last_active":"2024-04-25T00:31:31.294033+0000","last_peered":"2024-04-25T00:31:31.294033+0000","last_clean":"2024-04-25T00:31:31.294033+0000","last_became_active":"2024-04-25T00:31:30.322670+0000","last_became_peered":"2024-04-25T00:31:30.322670+0000","last_unstale":"2024-04-25T00:31:31.294033+0000","last_undegraded":"2024-04-25T00:31:31.294033+0000","last_fullsized":"2024-04-25T00:31:31.294033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:37.829118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294169+0000","last_change":"2024-04-25T00:31:29.297610+0000","last_active":"2024-04-25T00:31:31.294169+0000","last_peered":"2024-04-25T00:31:31.294169+0000","last_clean":"2024-04-25T00:31:31.294169+0000","last_became_active":"2024-04-25T00:31:29.296820+0000","last_became_peered":"2024-04-25T00:31:29.296820+0000","last_unstale":"2024-04-25T00:31:31.294169+0000","last_undegraded":"2024-04-25T00:31:31.294169+0000","last_fullsized":"2024-04-25T00:31:31.294169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:28.847603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339641+0000","last_change":"2024-04-25T00:31:30.306807+0000","last_active":"2024-04-25T00:31:32.339641+0000","last_peered":"2024-04-25T00:31:32.339641+0000","last_clean":"2024-04-25T00:31:32.339641+0000","last_became_active":"2024-04-25T00:31:30.306523+0000","last_became_peered":"2024-04-25T00:31:30.306523+0000","last_unstale":"2024-04-25T00:31:32.339641+0000","last_undegraded":"2024-04-25T00:31:32.339641+0000","last_fullsized":"2024-04-25T00:31:32.339641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289930+0000","last_change":"2024-04-25T00:31:29.296029+0000","last_active":"2024-04-25T00:31:31.289930+0000","last_peered":"2024-04-25T00:31:31.289930+0000","last_clean":"2024-04-25T00:31:31.289930+0000","last_became_active":"2024-04-25T00:31:29.295557+0000","last_became_peered":"2024-04-25T00:31:29.295557+0000","last_unstale":"2024-04-25T00:31:31.289930+0000","last_undegraded":"2024-04-25T00:31:31.289930+0000","last_fullsized":"2024-04-25T00:31:31.289930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:12.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295669+0000","last_change":"2024-04-25T00:31:30.311562+0000","last_active":"2024-04-25T00:31:31.295669+0000","last_peered":"2024-04-25T00:31:31.295669+0000","last_clean":"2024-04-25T00:31:31.295669+0000","last_became_active":"2024-04-25T00:31:30.309517+0000","last_became_peered":"2024-04-25T00:31:30.309517+0000","last_unstale":"2024-04-25T00:31:31.295669+0000","last_undegraded":"2024-04-25T00:31:31.295669+0000","last_fullsized":"2024-04-25T00:31:31.295669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:07.434483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273705+0000","last_change":"2024-04-25T00:31:29.295877+0000","last_active":"2024-04-25T00:31:32.273705+0000","last_peered":"2024-04-25T00:31:32.273705+0000","last_clean":"2024-04-25T00:31:32.273705+0000","last_became_active":"2024-04-25T00:31:29.295208+0000","last_became_peered":"2024-04-25T00:31:29.295208+0000","last_unstale":"2024-04-25T00:31:32.273705+0000","last_undegraded":"2024-04-25T00:31:32.273705+0000","last_fullsized":"2024-04-25T00:31:32.273705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:24.278463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339130+0000","last_change":"2024-04-25T00:31:30.307864+0000","last_active":"2024-04-25T00:31:32.339130+0000","last_peered":"2024-04-25T00:31:32.339130+0000","last_clean":"2024-04-25T00:31:32.339130+0000","last_became_active":"2024-04-25T00:31:30.306945+0000","last_became_peered":"2024-04-25T00:31:30.306945+0000","last_unstale":"2024-04-25T00:31:32.339130+0000","last_undegraded":"2024-04-25T00:31:32.339130+0000","last_fullsized":"2024-04-25T00:31:32.339130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:16.301733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339404+0000","last_change":"2024-04-25T00:31:29.291150+0000","last_active":"2024-04-25T00:31:32.339404+0000","last_peered":"2024-04-25T00:31:32.339404+0000","last_clean":"2024-04-25T00:31:32.339404+0000","last_became_active":"2024-04-25T00:31:29.290597+0000","last_became_peered":"2024-04-25T00:31:29.290597+0000","last_unstale":"2024-04-25T00:31:32.339404+0000","last_undegraded":"2024-04-25T00:31:32.339404+0000","last_fullsized":"2024-04-25T00:31:32.339404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:26.121929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339478+0000","last_change":"2024-04-25T00:31:30.322864+0000","last_active":"2024-04-25T00:31:32.339478+0000","last_peered":"2024-04-25T00:31:32.339478+0000","last_clean":"2024-04-25T00:31:32.339478+0000","last_became_active":"2024-04-25T00:31:30.322561+0000","last_became_peered":"2024-04-25T00:31:30.322561+0000","last_unstale":"2024-04-25T00:31:32.339478+0000","last_undegraded":"2024-04-25T00:31:32.339478+0000","last_fullsized":"2024-04-25T00:31:32.339478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294164+0000","last_change":"2024-04-25T00:31:29.289693+0000","last_active":"2024-04-25T00:31:31.294164+0000","last_peered":"2024-04-25T00:31:31.294164+0000","last_clean":"2024-04-25T00:31:31.294164+0000","last_became_active":"2024-04-25T00:31:29.286958+0000","last_became_peered":"2024-04-25T00:31:29.286958+0000","last_unstale":"2024-04-25T00:31:31.294164+0000","last_undegraded":"2024-04-25T00:31:31.294164+0000","last_fullsized":"2024-04-25T00:31:31.294164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:06.128468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339729+0000","last_change":"2024-04-25T00:31:30.327490+0000","last_active":"2024-04-25T00:31:32.339729+0000","last_peered":"2024-04-25T00:31:32.339729+0000","last_clean":"2024-04-25T00:31:32.339729+0000","last_became_active":"2024-04-25T00:31:30.324807+0000","last_became_peered":"2024-04-25T00:31:30.324807+0000","last_unstale":"2024-04-25T00:31:32.339729+0000","last_undegraded":"2024-04-25T00:31:32.339729+0000","last_fullsized":"2024-04-25T00:31:32.339729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:12.335111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273055+0000","last_change":"2024-04-25T00:31:29.292889+0000","last_active":"2024-04-25T00:31:32.273055+0000","last_peered":"2024-04-25T00:31:32.273055+0000","last_clean":"2024-04-25T00:31:32.273055+0000","last_became_active":"2024-04-25T00:31:29.292160+0000","last_became_peered":"2024-04-25T00:31:29.292160+0000","last_unstale":"2024-04-25T00:31:32.273055+0000","last_undegraded":"2024-04-25T00:31:32.273055+0000","last_fullsized":"2024-04-25T00:31:32.273055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:43.126464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273426+0000","last_change":"2024-04-25T00:31:30.306169+0000","last_active":"2024-04-25T00:31:32.273426+0000","last_peered":"2024-04-25T00:31:32.273426+0000","last_clean":"2024-04-25T00:31:32.273426+0000","last_became_active":"2024-04-25T00:31:30.299130+0000","last_became_peered":"2024-04-25T00:31:30.299130+0000","last_unstale":"2024-04-25T00:31:32.273426+0000","last_undegraded":"2024-04-25T00:31:32.273426+0000","last_fullsized":"2024-04-25T00:31:32.273426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:24.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.336849+0000","last_change":"2024-04-25T00:31:29.273896+0000","last_active":"2024-04-25T00:31:32.336849+0000","last_peered":"2024-04-25T00:31:32.336849+0000","last_clean":"2024-04-25T00:31:32.336849+0000","last_became_active":"2024-04-25T00:31:29.273531+0000","last_became_peered":"2024-04-25T00:31:29.273531+0000","last_unstale":"2024-04-25T00:31:32.336849+0000","last_undegraded":"2024-04-25T00:31:32.336849+0000","last_fullsized":"2024-04-25T00:31:32.336849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:43.126464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338915+0000","last_change":"2024-04-25T00:31:30.314038+0000","last_active":"2024-04-25T00:31:32.338915+0000","last_peered":"2024-04-25T00:31:32.338915+0000","last_clean":"2024-04-25T00:31:32.338915+0000","last_became_active":"2024-04-25T00:31:30.309018+0000","last_became_peered":"2024-04-25T00:31:30.309018+0000","last_unstale":"2024-04-25T00:31:32.338915+0000","last_undegraded":"2024-04-25T00:31:32.338915+0000","last_fullsized":"2024-04-25T00:31:32.338915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:56.905979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295834+0000","last_change":"2024-04-25T00:31:29.286381+0000","last_active":"2024-04-25T00:31:31.295834+0000","last_peered":"2024-04-25T00:31:31.295834+0000","last_clean":"2024-04-25T00:31:31.295834+0000","last_became_active":"2024-04-25T00:31:29.285385+0000","last_became_peered":"2024-04-25T00:31:29.285385+0000","last_unstale":"2024-04-25T00:31:31.295834+0000","last_undegraded":"2024-04-25T00:31:31.295834+0000","last_fullsized":"2024-04-25T00:31:31.295834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:33.874197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339927+0000","last_change":"2024-04-25T00:31:30.313477+0000","last_active":"2024-04-25T00:31:32.339927+0000","last_peered":"2024-04-25T00:31:32.339927+0000","last_clean":"2024-04-25T00:31:32.339927+0000","last_became_active":"2024-04-25T00:31:30.312831+0000","last_became_peered":"2024-04-25T00:31:30.312831+0000","last_unstale":"2024-04-25T00:31:32.339927+0000","last_undegraded":"2024-04-25T00:31:32.339927+0000","last_fullsized":"2024-04-25T00:31:32.339927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:36.378500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337220+0000","last_change":"2024-04-25T00:31:29.272826+0000","last_active":"2024-04-25T00:31:32.337220+0000","last_peered":"2024-04-25T00:31:32.337220+0000","last_clean":"2024-04-25T00:31:32.337220+0000","last_became_active":"2024-04-25T00:31:29.272432+0000","last_became_peered":"2024-04-25T00:31:29.272432+0000","last_unstale":"2024-04-25T00:31:32.337220+0000","last_undegraded":"2024-04-25T00:31:32.337220+0000","last_fullsized":"2024-04-25T00:31:32.337220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:24.278463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.340202+0000","last_change":"2024-04-25T00:31:30.312201+0000","last_active":"2024-04-25T00:31:32.340202+0000","last_peered":"2024-04-25T00:31:32.340202+0000","last_clean":"2024-04-25T00:31:32.340202+0000","last_became_active":"2024-04-25T00:31:30.311412+0000","last_became_peered":"2024-04-25T00:31:30.311412+0000","last_unstale":"2024-04-25T00:31:32.340202+0000","last_undegraded":"2024-04-25T00:31:32.340202+0000","last_fullsized":"2024-04-25T00:31:32.340202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:36.992645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.290023+0000","last_change":"2024-04-25T00:31:29.289893+0000","last_active":"2024-04-25T00:31:31.290023+0000","last_peered":"2024-04-25T00:31:31.290023+0000","last_clean":"2024-04-25T00:31:31.290023+0000","last_became_active":"2024-04-25T00:31:29.289495+0000","last_became_peered":"2024-04-25T00:31:29.289495+0000","last_unstale":"2024-04-25T00:31:31.290023+0000","last_undegraded":"2024-04-25T00:31:31.290023+0000","last_fullsized":"2024-04-25T00:31:31.290023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:06.128468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294386+0000","last_change":"2024-04-25T00:31:30.314359+0000","last_active":"2024-04-25T00:31:31.294386+0000","last_peered":"2024-04-25T00:31:31.294386+0000","last_clean":"2024-04-25T00:31:31.294386+0000","last_became_active":"2024-04-25T00:31:30.313625+0000","last_became_peered":"2024-04-25T00:31:30.313625+0000","last_unstale":"2024-04-25T00:31:31.294386+0000","last_undegraded":"2024-04-25T00:31:31.294386+0000","last_fullsized":"2024-04-25T00:31:31.294386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:16.301733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337138+0000","last_change":"2024-04-25T00:31:29.297325+0000","last_active":"2024-04-25T00:31:32.337138+0000","last_peered":"2024-04-25T00:31:32.337138+0000","last_clean":"2024-04-25T00:31:32.337138+0000","last_became_active":"2024-04-25T00:31:29.296822+0000","last_became_peered":"2024-04-25T00:31:29.296822+0000","last_unstale":"2024-04-25T00:31:32.337138+0000","last_undegraded":"2024-04-25T00:31:32.337138+0000","last_fullsized":"2024-04-25T00:31:32.337138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:28.847603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339313+0000","last_change":"2024-04-25T00:31:30.323788+0000","last_active":"2024-04-25T00:31:32.339313+0000","last_peered":"2024-04-25T00:31:32.339313+0000","last_clean":"2024-04-25T00:31:32.339313+0000","last_became_active":"2024-04-25T00:31:30.323115+0000","last_became_peered":"2024-04-25T00:31:30.323115+0000","last_unstale":"2024-04-25T00:31:32.339313+0000","last_undegraded":"2024-04-25T00:31:32.339313+0000","last_fullsized":"2024-04-25T00:31:32.339313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:51.764905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338672+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:31:32.338672+0000","last_peered":"2024-04-25T00:31:32.338672+0000","last_clean":"2024-04-25T00:31:32.338672+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:31:32.338672+0000","last_undegraded":"2024-04-25T00:31:32.338672+0000","last_fullsized":"2024-04-25T00:31:32.338672+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.292686+0000","last_change":"2024-04-25T00:31:29.287393+0000","last_active":"2024-04-25T00:31:31.292686+0000","last_peered":"2024-04-25T00:31:31.292686+0000","last_clean":"2024-04-25T00:31:31.292686+0000","last_became_active":"2024-04-25T00:31:29.286842+0000","last_became_peered":"2024-04-25T00:31:29.286842+0000","last_unstale":"2024-04-25T00:31:31.292686+0000","last_undegraded":"2024-04-25T00:31:31.292686+0000","last_fullsized":"2024-04-25T00:31:31.292686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:43.126464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338536+0000","last_change":"2024-04-25T00:31:30.302310+0000","last_active":"2024-04-25T00:31:32.338536+0000","last_peered":"2024-04-25T00:31:32.338536+0000","last_clean":"2024-04-25T00:31:32.338536+0000","last_became_active":"2024-04-25T00:31:30.302000+0000","last_became_peered":"2024-04-25T00:31:30.302000+0000","last_unstale":"2024-04-25T00:31:32.338536+0000","last_undegraded":"2024-04-25T00:31:32.338536+0000","last_fullsized":"2024-04-25T00:31:32.338536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338493+0000","last_change":"2024-04-25T00:31:29.308692+0000","last_active":"2024-04-25T00:31:32.338493+0000","last_peered":"2024-04-25T00:31:32.338493+0000","last_clean":"2024-04-25T00:31:32.338493+0000","last_became_active":"2024-04-25T00:31:29.306907+0000","last_became_peered":"2024-04-25T00:31:29.306907+0000","last_unstale":"2024-04-25T00:31:32.338493+0000","last_undegraded":"2024-04-25T00:31:32.338493+0000","last_fullsized":"2024-04-25T00:31:32.338493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:07.430509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339054+0000","last_change":"2024-04-25T00:31:30.278855+0000","last_active":"2024-04-25T00:31:32.339054+0000","last_peered":"2024-04-25T00:31:32.339054+0000","last_clean":"2024-04-25T00:31:32.339054+0000","last_became_active":"2024-04-25T00:31:30.278557+0000","last_became_peered":"2024-04-25T00:31:30.278557+0000","last_unstale":"2024-04-25T00:31:32.339054+0000","last_undegraded":"2024-04-25T00:31:32.339054+0000","last_fullsized":"2024-04-25T00:31:32.339054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:00.643971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337108+0000","last_change":"2024-04-25T00:31:29.271815+0000","last_active":"2024-04-25T00:31:32.337108+0000","last_peered":"2024-04-25T00:31:32.337108+0000","last_clean":"2024-04-25T00:31:32.337108+0000","last_became_active":"2024-04-25T00:31:29.271154+0000","last_became_peered":"2024-04-25T00:31:29.271154+0000","last_unstale":"2024-04-25T00:31:32.337108+0000","last_undegraded":"2024-04-25T00:31:32.337108+0000","last_fullsized":"2024-04-25T00:31:32.337108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:12.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289615+0000","last_change":"2024-04-25T00:31:30.287295+0000","last_active":"2024-04-25T00:31:31.289615+0000","last_peered":"2024-04-25T00:31:31.289615+0000","last_clean":"2024-04-25T00:31:31.289615+0000","last_became_active":"2024-04-25T00:31:30.285530+0000","last_became_peered":"2024-04-25T00:31:30.285530+0000","last_unstale":"2024-04-25T00:31:31.289615+0000","last_undegraded":"2024-04-25T00:31:31.289615+0000","last_fullsized":"2024-04-25T00:31:31.289615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:26.475549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294485+0000","last_change":"2024-04-25T00:31:29.288097+0000","last_active":"2024-04-25T00:31:31.294485+0000","last_peered":"2024-04-25T00:31:31.294485+0000","last_clean":"2024-04-25T00:31:31.294485+0000","last_became_active":"2024-04-25T00:31:29.286580+0000","last_became_peered":"2024-04-25T00:31:29.286580+0000","last_unstale":"2024-04-25T00:31:31.294485+0000","last_undegraded":"2024-04-25T00:31:31.294485+0000","last_fullsized":"2024-04-25T00:31:31.294485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:24.278463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294183+0000","last_change":"2024-04-25T00:31:30.323523+0000","last_active":"2024-04-25T00:31:31.294183+0000","last_peered":"2024-04-25T00:31:31.294183+0000","last_clean":"2024-04-25T00:31:31.294183+0000","last_became_active":"2024-04-25T00:31:30.310509+0000","last_became_peered":"2024-04-25T00:31:30.310509+0000","last_unstale":"2024-04-25T00:31:31.294183+0000","last_undegraded":"2024-04-25T00:31:31.294183+0000","last_fullsized":"2024-04-25T00:31:31.294183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:25.290315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289773+0000","last_change":"2024-04-25T00:31:29.283436+0000","last_active":"2024-04-25T00:31:31.289773+0000","last_peered":"2024-04-25T00:31:31.289773+0000","last_clean":"2024-04-25T00:31:31.289773+0000","last_became_active":"2024-04-25T00:31:29.283038+0000","last_became_peered":"2024-04-25T00:31:29.283038+0000","last_unstale":"2024-04-25T00:31:31.289773+0000","last_undegraded":"2024-04-25T00:31:31.289773+0000","last_fullsized":"2024-04-25T00:31:31.289773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:28.847603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295871+0000","last_change":"2024-04-25T00:31:30.311973+0000","last_active":"2024-04-25T00:31:31.295871+0000","last_peered":"2024-04-25T00:31:31.295871+0000","last_clean":"2024-04-25T00:31:31.295871+0000","last_became_active":"2024-04-25T00:31:30.311141+0000","last_became_peered":"2024-04-25T00:31:30.311141+0000","last_unstale":"2024-04-25T00:31:31.295871+0000","last_undegraded":"2024-04-25T00:31:31.295871+0000","last_fullsized":"2024-04-25T00:31:31.295871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:12.335111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339611+0000","last_change":"2024-04-25T00:31:29.309713+0000","last_active":"2024-04-25T00:31:32.339611+0000","last_peered":"2024-04-25T00:31:32.339611+0000","last_clean":"2024-04-25T00:31:32.339611+0000","last_became_active":"2024-04-25T00:31:29.309033+0000","last_became_peered":"2024-04-25T00:31:29.309033+0000","last_unstale":"2024-04-25T00:31:32.339611+0000","last_undegraded":"2024-04-25T00:31:32.339611+0000","last_fullsized":"2024-04-25T00:31:32.339611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:59.632119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289794+0000","last_change":"2024-04-25T00:31:30.285762+0000","last_active":"2024-04-25T00:31:31.289794+0000","last_peered":"2024-04-25T00:31:31.289794+0000","last_clean":"2024-04-25T00:31:31.289794+0000","last_became_active":"2024-04-25T00:31:30.284685+0000","last_became_peered":"2024-04-25T00:31:30.284685+0000","last_unstale":"2024-04-25T00:31:31.289794+0000","last_undegraded":"2024-04-25T00:31:31.289794+0000","last_fullsized":"2024-04-25T00:31:31.289794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.290064+0000","last_change":"2024-04-25T00:31:29.296589+0000","last_active":"2024-04-25T00:31:31.290064+0000","last_peered":"2024-04-25T00:31:31.290064+0000","last_clean":"2024-04-25T00:31:31.290064+0000","last_became_active":"2024-04-25T00:31:29.296082+0000","last_became_peered":"2024-04-25T00:31:29.296082+0000","last_unstale":"2024-04-25T00:31:31.290064+0000","last_undegraded":"2024-04-25T00:31:31.290064+0000","last_fullsized":"2024-04-25T00:31:31.290064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:06.422631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338590+0000","last_change":"2024-04-25T00:31:30.318568+0000","last_active":"2024-04-25T00:31:32.338590+0000","last_peered":"2024-04-25T00:31:32.338590+0000","last_clean":"2024-04-25T00:31:32.338590+0000","last_became_active":"2024-04-25T00:31:30.318203+0000","last_became_peered":"2024-04-25T00:31:30.318203+0000","last_unstale":"2024-04-25T00:31:32.338590+0000","last_undegraded":"2024-04-25T00:31:32.338590+0000","last_fullsized":"2024-04-25T00:31:32.338590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:12.015188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.274864+0000","last_change":"2024-04-25T00:31:29.310321+0000","last_active":"2024-04-25T00:31:32.274864+0000","last_peered":"2024-04-25T00:31:32.274864+0000","last_clean":"2024-04-25T00:31:32.274864+0000","last_became_active":"2024-04-25T00:31:29.309796+0000","last_became_peered":"2024-04-25T00:31:29.309796+0000","last_unstale":"2024-04-25T00:31:32.274864+0000","last_undegraded":"2024-04-25T00:31:32.274864+0000","last_fullsized":"2024-04-25T00:31:32.274864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:12.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339672+0000","last_change":"2024-04-25T00:31:30.306140+0000","last_active":"2024-04-25T00:31:32.339672+0000","last_peered":"2024-04-25T00:31:32.339672+0000","last_clean":"2024-04-25T00:31:32.339672+0000","last_became_active":"2024-04-25T00:31:30.294939+0000","last_became_peered":"2024-04-25T00:31:30.294939+0000","last_unstale":"2024-04-25T00:31:32.339672+0000","last_undegraded":"2024-04-25T00:31:32.339672+0000","last_fullsized":"2024-04-25T00:31:32.339672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338396+0000","last_change":"2024-04-25T00:31:29.313166+0000","last_active":"2024-04-25T00:31:32.338396+0000","last_peered":"2024-04-25T00:31:32.338396+0000","last_clean":"2024-04-25T00:31:32.338396+0000","last_became_active":"2024-04-25T00:31:29.308641+0000","last_became_peered":"2024-04-25T00:31:29.308641+0000","last_unstale":"2024-04-25T00:31:32.338396+0000","last_undegraded":"2024-04-25T00:31:32.338396+0000","last_fullsized":"2024-04-25T00:31:32.338396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:06.422631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.296415+0000","last_change":"2024-04-25T00:31:30.303363+0000","last_active":"2024-04-25T00:31:31.296415+0000","last_peered":"2024-04-25T00:31:31.296415+0000","last_clean":"2024-04-25T00:31:31.296415+0000","last_became_active":"2024-04-25T00:31:30.302602+0000","last_became_peered":"2024-04-25T00:31:30.302602+0000","last_unstale":"2024-04-25T00:31:31.296415+0000","last_undegraded":"2024-04-25T00:31:31.296415+0000","last_fullsized":"2024-04-25T00:31:31.296415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:08.442361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289690+0000","last_change":"2024-04-25T00:31:29.306640+0000","last_active":"2024-04-25T00:31:31.289690+0000","last_peered":"2024-04-25T00:31:31.289690+0000","last_clean":"2024-04-25T00:31:31.289690+0000","last_became_active":"2024-04-25T00:31:29.306117+0000","last_became_peered":"2024-04-25T00:31:29.306117+0000","last_unstale":"2024-04-25T00:31:31.289690+0000","last_undegraded":"2024-04-25T00:31:31.289690+0000","last_fullsized":"2024-04-25T00:31:31.289690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:07.430509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338904+0000","last_change":"2024-04-25T00:31:30.287149+0000","last_active":"2024-04-25T00:31:32.338904+0000","last_peered":"2024-04-25T00:31:32.338904+0000","last_clean":"2024-04-25T00:31:32.338904+0000","last_became_active":"2024-04-25T00:31:30.286140+0000","last_became_peered":"2024-04-25T00:31:30.286140+0000","last_unstale":"2024-04-25T00:31:32.338904+0000","last_undegraded":"2024-04-25T00:31:32.338904+0000","last_fullsized":"2024-04-25T00:31:32.338904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:37.829118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.290109+0000","last_change":"2024-04-25T00:31:29.278390+0000","last_active":"2024-04-25T00:31:31.290109+0000","last_peered":"2024-04-25T00:31:31.290109+0000","last_clean":"2024-04-25T00:31:31.290109+0000","last_became_active":"2024-04-25T00:31:29.277901+0000","last_became_peered":"2024-04-25T00:31:29.277901+0000","last_unstale":"2024-04-25T00:31:31.290109+0000","last_undegraded":"2024-04-25T00:31:31.290109+0000","last_fullsized":"2024-04-25T00:31:31.290109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:24.278463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338701+0000","last_change":"2024-04-25T00:31:30.298582+0000","last_active":"2024-04-25T00:31:32.338701+0000","last_peered":"2024-04-25T00:31:32.338701+0000","last_clean":"2024-04-25T00:31:32.338701+0000","last_became_active":"2024-04-25T00:31:30.298289+0000","last_became_peered":"2024-04-25T00:31:30.298289+0000","last_unstale":"2024-04-25T00:31:32.338701+0000","last_undegraded":"2024-04-25T00:31:32.338701+0000","last_fullsized":"2024-04-25T00:31:32.338701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:50.437582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339895+0000","last_change":"2024-04-25T00:31:29.309564+0000","last_active":"2024-04-25T00:31:32.339895+0000","last_peered":"2024-04-25T00:31:32.339895+0000","last_clean":"2024-04-25T00:31:32.339895+0000","last_became_active":"2024-04-25T00:31:29.308824+0000","last_became_peered":"2024-04-25T00:31:29.308824+0000","last_unstale":"2024-04-25T00:31:32.339895+0000","last_undegraded":"2024-04-25T00:31:32.339895+0000","last_fullsized":"2024-04-25T00:31:32.339895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:49.417800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273411+0000","last_change":"2024-04-25T00:31:30.316035+0000","last_active":"2024-04-25T00:31:32.273411+0000","last_peered":"2024-04-25T00:31:32.273411+0000","last_clean":"2024-04-25T00:31:32.273411+0000","last_became_active":"2024-04-25T00:31:30.315263+0000","last_became_peered":"2024-04-25T00:31:30.315263+0000","last_unstale":"2024-04-25T00:31:32.273411+0000","last_undegraded":"2024-04-25T00:31:32.273411+0000","last_fullsized":"2024-04-25T00:31:32.273411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:26.475549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295648+0000","last_change":"2024-04-25T00:31:30.329587+0000","last_active":"2024-04-25T00:31:31.295648+0000","last_peered":"2024-04-25T00:31:31.295648+0000","last_clean":"2024-04-25T00:31:31.295648+0000","last_became_active":"2024-04-25T00:31:30.329014+0000","last_became_peered":"2024-04-25T00:31:30.329014+0000","last_unstale":"2024-04-25T00:31:31.295648+0000","last_undegraded":"2024-04-25T00:31:31.295648+0000","last_fullsized":"2024-04-25T00:31:31.295648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:02.546759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338944+0000","last_change":"2024-04-25T00:31:29.307154+0000","last_active":"2024-04-25T00:31:32.338944+0000","last_peered":"2024-04-25T00:31:32.338944+0000","last_clean":"2024-04-25T00:31:32.338944+0000","last_became_active":"2024-04-25T00:31:29.306693+0000","last_became_peered":"2024-04-25T00:31:29.306693+0000","last_unstale":"2024-04-25T00:31:32.338944+0000","last_undegraded":"2024-04-25T00:31:32.338944+0000","last_fullsized":"2024-04-25T00:31:32.338944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:20.941312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293968+0000","last_change":"2024-04-25T00:31:30.329462+0000","last_active":"2024-04-25T00:31:31.293968+0000","last_peered":"2024-04-25T00:31:31.293968+0000","last_clean":"2024-04-25T00:31:31.293968+0000","last_became_active":"2024-04-25T00:31:30.328941+0000","last_became_peered":"2024-04-25T00:31:30.328941+0000","last_unstale":"2024-04-25T00:31:31.293968+0000","last_undegraded":"2024-04-25T00:31:31.293968+0000","last_fullsized":"2024-04-25T00:31:31.293968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:50.437582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339296+0000","last_change":"2024-04-25T00:31:29.311168+0000","last_active":"2024-04-25T00:31:32.339296+0000","last_peered":"2024-04-25T00:31:32.339296+0000","last_clean":"2024-04-25T00:31:32.339296+0000","last_became_active":"2024-04-25T00:31:29.310402+0000","last_became_peered":"2024-04-25T00:31:29.310402+0000","last_unstale":"2024-04-25T00:31:32.339296+0000","last_undegraded":"2024-04-25T00:31:32.339296+0000","last_fullsized":"2024-04-25T00:31:32.339296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:56.307057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339924+0000","last_change":"2024-04-25T00:31:30.307691+0000","last_active":"2024-04-25T00:31:32.339924+0000","last_peered":"2024-04-25T00:31:32.339924+0000","last_clean":"2024-04-25T00:31:32.339924+0000","last_became_active":"2024-04-25T00:31:30.307158+0000","last_became_peered":"2024-04-25T00:31:30.307158+0000","last_unstale":"2024-04-25T00:31:32.339924+0000","last_undegraded":"2024-04-25T00:31:32.339924+0000","last_fullsized":"2024-04-25T00:31:32.339924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.057143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338533+0000","last_change":"2024-04-25T00:31:29.282318+0000","last_active":"2024-04-25T00:31:32.338533+0000","last_peered":"2024-04-25T00:31:32.338533+0000","last_clean":"2024-04-25T00:31:32.338533+0000","last_became_active":"2024-04-25T00:31:29.281826+0000","last_became_peered":"2024-04-25T00:31:29.281826+0000","last_unstale":"2024-04-25T00:31:32.338533+0000","last_undegraded":"2024-04-25T00:31:32.338533+0000","last_fullsized":"2024-04-25T00:31:32.338533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:26.121929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338333+0000","last_change":"2024-04-25T00:31:30.302619+0000","last_active":"2024-04-25T00:31:32.338333+0000","last_peered":"2024-04-25T00:31:32.338333+0000","last_clean":"2024-04-25T00:31:32.338333+0000","last_became_active":"2024-04-25T00:31:30.301637+0000","last_became_peered":"2024-04-25T00:31:30.301637+0000","last_unstale":"2024-04-25T00:31:32.338333+0000","last_undegraded":"2024-04-25T00:31:32.338333+0000","last_fullsized":"2024-04-25T00:31:32.338333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339800+0000","last_change":"2024-04-25T00:31:29.311460+0000","last_active":"2024-04-25T00:31:32.339800+0000","last_peered":"2024-04-25T00:31:32.339800+0000","last_clean":"2024-04-25T00:31:32.339800+0000","last_became_active":"2024-04-25T00:31:29.311052+0000","last_became_peered":"2024-04-25T00:31:29.311052+0000","last_unstale":"2024-04-25T00:31:32.339800+0000","last_undegraded":"2024-04-25T00:31:32.339800+0000","last_fullsized":"2024-04-25T00:31:32.339800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:12.847032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:31:38.224 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:31:38.224 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T00:31:38.224 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T00:31:38.225 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T00:31:38.225 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T00:31:38.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.388+0000 7f8726d29640 1 Processor -- start 2024-04-25T00:31:38.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f8726d29640 1 -- start start 2024-04-25T00:31:38.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f8726d29640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8720071e10 0x7f872015ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f8726d29640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f872006e580 con 0x7f87200721f0 2024-04-25T00:31:38.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f8726d29640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f872006e820 con 0x7f87200ce820 2024-04-25T00:31:38.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f8726d29640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f872006f5e0 con 0x7f8720071e10 2024-04-25T00:31:38.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871ffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f87200721f0 0x7f8720072650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:49964/0 (socket says 172.21.15.29:49964) 2024-04-25T00:31:38.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871ffff640 1 -- 172.21.15.29:0/2320012205 learned_addr learned my addr 172.21.15.29:0/2320012205 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:38.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f8724ac5640 1 --2- 172.21.15.29:0/2320012205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8720071e10 0x7f872015ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3573595174 0 0) 0x7f872006e580 con 0x7f87200721f0 2024-04-25T00:31:38.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8710003530 con 0x7f87200721f0 2024-04-25T00:31:38.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2268428368 0 0) 0x7f8710003530 con 0x7f87200721f0 2024-04-25T00:31:38.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8720071e10 msgr2=0x7f872015ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 --2- 172.21.15.29:0/2320012205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8720071e10 0x7f872015ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 >> v1:172.21.15.157:6789/0 conn(0x7f87200ce820 legacy=0x7f87200cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872006eb00 con 0x7f87200721f0 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f870c001680 con 0x7f87200721f0 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 >> v1:172.21.15.29:6789/0 conn(0x7f87200721f0 legacy=0x7f8720072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 --2- 172.21.15.29:0/2320012205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87100039f0 0x7f8710003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 --2- 172.21.15.29:0/2320012205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8720071e10 0x7f8710004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 --2- 172.21.15.29:0/2320012205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8710004d50 0x7f87100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f872006f5e0 con 0x7f8710004d50 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f871000ba80 con 0x7f87100039f0 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f871000bc80 con 0x7f8720071e10 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f87252c6640 1 --2- 172.21.15.29:0/2320012205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8710004d50 0x7f87100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871ffff640 1 --2- 172.21.15.29:0/2320012205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8720071e10 0x7f8710004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f8724ac5640 1 --2- 172.21.15.29:0/2320012205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87100039f0 0x7f8710003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f87252c6640 1 -- 172.21.15.29:0/2320012205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8720071e10 msgr2=0x7f8710004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:38.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f87252c6640 1 --2- 172.21.15.29:0/2320012205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8720071e10 0x7f8710004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f87252c6640 1 -- 172.21.15.29:0/2320012205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87100039f0 msgr2=0x7f8710003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:38.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f87252c6640 1 --2- 172.21.15.29:0/2320012205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87100039f0 0x7f8710003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f87252c6640 1 -- 172.21.15.29:0/2320012205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8710003530 con 0x7f8710004d50 2024-04-25T00:31:38.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.392+0000 7f871ffff640 1 --2- 172.21.15.29:0/2320012205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8720071e10 0x7f8710004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:38.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.396+0000 7f87252c6640 1 --2- 172.21.15.29:0/2320012205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8710004d50 0x7f87100051d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8718001e40 tx=0x7f871801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad9254160d7b3d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:38.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.396+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8718001a90 con 0x7f8710004d50 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.396+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8718003b80 con 0x7f8710004d50 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.396+0000 7f871f7fe640 1 -- 172.21.15.29:0/2320012205 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8718005080 con 0x7f8710004d50 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.396+0000 7f8726d29640 1 -- 172.21.15.29:0/2320012205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8710004d50 msgr2=0x7f87100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.396+0000 7f8726d29640 1 --2- 172.21.15.29:0/2320012205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8710004d50 0x7f87100051d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8718001e40 tx=0x7f871801c760 comp rx=0 tx=0).stop 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.396+0000 7f87252c6640 1 -- 172.21.15.29:0/2320012205 reap_dead start 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.396+0000 7f8726d29640 1 -- 172.21.15.29:0/2320012205 shutdown_connections 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.396+0000 7f8726d29640 1 -- 172.21.15.29:0/2320012205 >> 172.21.15.29:0/2320012205 conn(0x7f87200b2920 msgr2=0x7f87200b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.396+0000 7f8726d29640 1 -- 172.21.15.29:0/2320012205 shutdown_connections 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.396+0000 7f8726d29640 1 -- 172.21.15.29:0/2320012205 wait complete. 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.396+0000 7f8726d29640 1 Processor -- start 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f8726d29640 1 -- start start 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f8726d29640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8710004d50 0x7f87200cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f8726d29640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87200cb760 0x7f872014b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f8726d29640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87200c9fa0 0x7f87200ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f8726d29640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f872006f0c0 con 0x7f87200cb760 2024-04-25T00:31:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f8726d29640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87200b0370 con 0x7f8710004d50 2024-04-25T00:31:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f8726d29640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f87200cf3a0 con 0x7f87200c9fa0 2024-04-25T00:31:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87200cb760 0x7f872014b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87200cb760 0x7f872014b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49558/0 (socket says 172.21.15.29:49558) 2024-04-25T00:31:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871ffff640 1 -- 172.21.15.29:0/2360903274 learned_addr learned my addr 172.21.15.29:0/2360903274 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f8724ac5640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8710004d50 0x7f87200cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f87252c6640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87200c9fa0 0x7f87200ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871ffff640 1 -- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87200c9fa0 msgr2=0x7f87200ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871ffff640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87200c9fa0 0x7f87200ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871ffff640 1 -- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8710004d50 msgr2=0x7f87200cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871ffff640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8710004d50 0x7f87200cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871ffff640 1 -- 172.21.15.29:0/2360903274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87200cbc40 con 0x7f87200cb760 2024-04-25T00:31:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871ffff640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87200cb760 0x7f872014b690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f870c00eb20 tx=0x7f870c005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871d7fa640 1 -- 172.21.15.29:0/2360903274 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c019070 con 0x7f87200cb760 2024-04-25T00:31:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871d7fa640 1 -- 172.21.15.29:0/2360903274 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870c004ac0 con 0x7f87200cb760 2024-04-25T00:31:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f8726d29640 1 -- 172.21.15.29:0/2360903274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87200cbf20 con 0x7f87200cb760 2024-04-25T00:31:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871d7fa640 1 -- 172.21.15.29:0/2360903274 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c005050 con 0x7f87200cb760 2024-04-25T00:31:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f8726d29640 1 -- 172.21.15.29:0/2360903274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87200cc4e0 con 0x7f87200cb760 2024-04-25T00:31:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f8726d29640 1 -- 172.21.15.29:0/2360903274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8720069270 con 0x7f87200cb760 2024-04-25T00:31:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871d7fa640 1 -- 172.21.15.29:0/2360903274 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f870c01c070 con 0x7f87200cb760 2024-04-25T00:31:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871d7fa640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f87040678f0 0x7f8704069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f871d7fa640 1 -- 172.21.15.29:0/2360903274 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f870c0855d0 con 0x7f87200cb760 2024-04-25T00:31:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.400+0000 7f8724ac5640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f87040678f0 0x7f8704069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.404+0000 7f8724ac5640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f87040678f0 0x7f8704069db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f87140099b0 tx=0x7f8714008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:38.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.408+0000 7f871d7fa640 1 -- 172.21.15.29:0/2360903274 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f870c055ac0 con 0x7f87200cb760 2024-04-25T00:31:38.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.612+0000 7f8726d29640 1 -- 172.21.15.29:0/2360903274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f872006e050 con 0x7f87200cb760 2024-04-25T00:31:38.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.620+0000 7f871d7fa640 1 -- 172.21.15.29:0/2360903274 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+166894 (secure 0 0 0) 0x7f870c059dc0 con 0x7f87200cb760 2024-04-25T00:31:38.624 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:38.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.620+0000 7f8726d29640 1 -- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f87040678f0 msgr2=0x7f8704069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.620+0000 7f8726d29640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f87040678f0 0x7f8704069db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f87140099b0 tx=0x7f8714008040 comp rx=0 tx=0).stop 2024-04-25T00:31:38.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.620+0000 7f8726d29640 1 -- 172.21.15.29:0/2360903274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87200cb760 msgr2=0x7f872014b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.620+0000 7f8726d29640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87200cb760 0x7f872014b690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f870c00eb20 tx=0x7f870c005e00 comp rx=0 tx=0).stop 2024-04-25T00:31:38.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.620+0000 7f8726d29640 1 -- 172.21.15.29:0/2360903274 shutdown_connections 2024-04-25T00:31:38.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.620+0000 7f8726d29640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87200c9fa0 0x7f87200ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.620+0000 7f8726d29640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87200cb760 0x7f872014b690 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.620+0000 7f8726d29640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f87040678f0 0x7f8704069db0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.620+0000 7f8726d29640 1 --2- 172.21.15.29:0/2360903274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8710004d50 0x7f87200cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.620+0000 7f8726d29640 1 -- 172.21.15.29:0/2360903274 >> 172.21.15.29:0/2360903274 conn(0x7f87200b2920 msgr2=0x7f87200aec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:38.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.620+0000 7f8726d29640 1 -- 172.21.15.29:0/2360903274 shutdown_connections 2024-04-25T00:31:38.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.620+0000 7f8726d29640 1 -- 172.21.15.29:0/2360903274 wait complete. 2024-04-25T00:31:38.642 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3474584311},{"type":"v1","addr":"172.21.15.157:6833","nonce":3474584311}]},"active_addr":"172.21.15.157:6833/3474584311","active_change":"2024-04-25T00:31:17.431955+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":1408011519}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":2500505584}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":3796173968}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":3565775874}]}]} 2024-04-25T00:31:38.646 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T00:31:38.646 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T00:31:38.647 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:38.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.772+0000 7f242731d640 1 Processor -- start 2024-04-25T00:31:38.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.772+0000 7f242731d640 1 -- start start 2024-04-25T00:31:38.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.772+0000 7f242731d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f242006b210 0x7f2420072530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.772+0000 7f242731d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24200cb2a0 0x7f24200ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.772+0000 7f242731d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f242015e2c0 0x7f242006b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.772+0000 7f242731d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2420072140 con 0x7f242006b210 2024-04-25T00:31:38.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.772+0000 7f242731d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f24200cfbd0 con 0x7f242015e2c0 2024-04-25T00:31:38.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.772+0000 7f242731d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f24200cd190 con 0x7f24200cb2a0 2024-04-25T00:31:38.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.772+0000 7f24250b9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f242006b210 0x7f2420072530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.772+0000 7f24250b9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f242006b210 0x7f2420072530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49568/0 (socket says 172.21.15.29:49568) 2024-04-25T00:31:38.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.772+0000 7f24250b9640 1 -- 172.21.15.29:0/417260156 learned_addr learned my addr 172.21.15.29:0/417260156 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:38.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f24258ba640 1 --2- 172.21.15.29:0/417260156 >> v2:172.21.15.157:3300/0 conn(0x7f242015e2c0 0x7f242006b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f24248b8640 1 --2- 172.21.15.29:0/417260156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24200cb2a0 0x7f24200ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f24250b9640 1 -- 172.21.15.29:0/417260156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24200cb2a0 msgr2=0x7f24200ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:38.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f24250b9640 1 --2- 172.21.15.29:0/417260156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24200cb2a0 0x7f24200ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f24250b9640 1 -- 172.21.15.29:0/417260156 >> v2:172.21.15.157:3300/0 conn(0x7f242015e2c0 msgr2=0x7f242006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:38.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f24250b9640 1 --2- 172.21.15.29:0/417260156 >> v2:172.21.15.157:3300/0 conn(0x7f242015e2c0 0x7f242006b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f24250b9640 1 -- 172.21.15.29:0/417260156 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242006bb30 con 0x7f242006b210 2024-04-25T00:31:38.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f24248b8640 1 --2- 172.21.15.29:0/417260156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24200cb2a0 0x7f24200ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:38.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f24250b9640 1 --2- 172.21.15.29:0/417260156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f242006b210 0x7f2420072530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f24200ce710 tx=0x7f241c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25508f230288f2b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:38.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f2417fff640 1 -- 172.21.15.29:0/417260156 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c0a5640 con 0x7f242006b210 2024-04-25T00:31:38.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f2417fff640 1 -- 172.21.15.29:0/417260156 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241c0a57e0 con 0x7f242006b210 2024-04-25T00:31:38.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f2417fff640 1 -- 172.21.15.29:0/417260156 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c0a5bb0 con 0x7f242006b210 2024-04-25T00:31:38.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f242731d640 1 -- 172.21.15.29:0/417260156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f242006b210 msgr2=0x7f2420072530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f242731d640 1 --2- 172.21.15.29:0/417260156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f242006b210 0x7f2420072530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f24200ce710 tx=0x7f241c01c8d0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f242731d640 1 -- 172.21.15.29:0/417260156 shutdown_connections 2024-04-25T00:31:38.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f242731d640 1 --2- 172.21.15.29:0/417260156 >> v2:172.21.15.157:3300/0 conn(0x7f242015e2c0 0x7f242006b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f242731d640 1 --2- 172.21.15.29:0/417260156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24200cb2a0 0x7f24200ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f242731d640 1 --2- 172.21.15.29:0/417260156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f242006b210 0x7f2420072530 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f242731d640 1 -- 172.21.15.29:0/417260156 >> 172.21.15.29:0/417260156 conn(0x7f24200b2920 msgr2=0x7f24200b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:38.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f242731d640 1 -- 172.21.15.29:0/417260156 shutdown_connections 2024-04-25T00:31:38.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.776+0000 7f242731d640 1 -- 172.21.15.29:0/417260156 wait complete. 2024-04-25T00:31:38.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f242731d640 1 Processor -- start 2024-04-25T00:31:38.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f242731d640 1 -- start start 2024-04-25T00:31:38.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f242731d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f242006b210 0x7f242015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f242731d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24200cb2a0 0x7f242015bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f242731d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f242015e2c0 0x7f2420156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f242731d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f242006e520 con 0x7f242006b210 2024-04-25T00:31:38.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f242731d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f242006e3a0 con 0x7f242015e2c0 2024-04-25T00:31:38.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f242731d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f242006e6a0 con 0x7f24200cb2a0 2024-04-25T00:31:38.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f24250b9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f242006b210 0x7f242015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f24250b9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f242006b210 0x7f242015ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49580/0 (socket says 172.21.15.29:49580) 2024-04-25T00:31:38.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f24250b9640 1 -- 172.21.15.29:0/4286353855 learned_addr learned my addr 172.21.15.29:0/4286353855 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:38.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f24250b9640 1 -- 172.21.15.29:0/4286353855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24200cb2a0 msgr2=0x7f242015bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f24250b9640 1 --2- 172.21.15.29:0/4286353855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24200cb2a0 0x7f242015bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f24258ba640 1 --2- 172.21.15.29:0/4286353855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f242015e2c0 0x7f2420156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f24250b9640 1 -- 172.21.15.29:0/4286353855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f242015e2c0 msgr2=0x7f2420156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f24250b9640 1 --2- 172.21.15.29:0/4286353855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f242015e2c0 0x7f2420156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f24250b9640 1 -- 172.21.15.29:0/4286353855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2420153740 con 0x7f242006b210 2024-04-25T00:31:38.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f24250b9640 1 --2- 172.21.15.29:0/4286353855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f242006b210 0x7f242015ce00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f241c01af80 tx=0x7f241c019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:38.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f2415ffb640 1 -- 172.21.15.29:0/4286353855 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c0bc020 con 0x7f242006b210 2024-04-25T00:31:38.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f2415ffb640 1 -- 172.21.15.29:0/4286353855 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241c0b8670 con 0x7f242006b210 2024-04-25T00:31:38.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f242731d640 1 -- 172.21.15.29:0/4286353855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2420152ea0 con 0x7f242006b210 2024-04-25T00:31:38.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f2415ffb640 1 -- 172.21.15.29:0/4286353855 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c0b8a40 con 0x7f242006b210 2024-04-25T00:31:38.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f242731d640 1 -- 172.21.15.29:0/4286353855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2420153460 con 0x7f242006b210 2024-04-25T00:31:38.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.780+0000 7f242731d640 1 -- 172.21.15.29:0/4286353855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2420069270 con 0x7f242006b210 2024-04-25T00:31:38.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.784+0000 7f2415ffb640 1 -- 172.21.15.29:0/4286353855 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f241c0bf020 con 0x7f242006b210 2024-04-25T00:31:38.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.784+0000 7f2415ffb640 1 --2- 172.21.15.29:0/4286353855 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f24040678f0 0x7f2404069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:38.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.784+0000 7f2415ffb640 1 -- 172.21.15.29:0/4286353855 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f241c120d10 con 0x7f242006b210 2024-04-25T00:31:38.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.784+0000 7f24248b8640 1 --2- 172.21.15.29:0/4286353855 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f24040678f0 0x7f2404069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:38.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.784+0000 7f24248b8640 1 --2- 172.21.15.29:0/4286353855 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f24040678f0 0x7f2404069db0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2410006fd0 tx=0x7f2410008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:38.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.788+0000 7f2415ffb640 1 -- 172.21.15.29:0/4286353855 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f241c126020 con 0x7f242006b210 2024-04-25T00:31:38.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.952+0000 7f242731d640 1 -- 172.21.15.29:0/4286353855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2420067180 con 0x7f242006b210 2024-04-25T00:31:38.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.952+0000 7f2415ffb640 1 -- 172.21.15.29:0/4286353855 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f241c0c6030 con 0x7f242006b210 2024-04-25T00:31:38.957 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:38.957 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:38.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.956+0000 7f242731d640 1 -- 172.21.15.29:0/4286353855 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f24040678f0 msgr2=0x7f2404069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.956+0000 7f242731d640 1 --2- 172.21.15.29:0/4286353855 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f24040678f0 0x7f2404069db0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2410006fd0 tx=0x7f2410008040 comp rx=0 tx=0).stop 2024-04-25T00:31:38.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.956+0000 7f242731d640 1 -- 172.21.15.29:0/4286353855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f242006b210 msgr2=0x7f242015ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:38.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.956+0000 7f242731d640 1 --2- 172.21.15.29:0/4286353855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f242006b210 0x7f242015ce00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f241c01af80 tx=0x7f241c019670 comp rx=0 tx=0).stop 2024-04-25T00:31:38.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.956+0000 7f242731d640 1 -- 172.21.15.29:0/4286353855 shutdown_connections 2024-04-25T00:31:38.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.956+0000 7f242731d640 1 --2- 172.21.15.29:0/4286353855 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f24040678f0 0x7f2404069db0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.956+0000 7f242731d640 1 --2- 172.21.15.29:0/4286353855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f242015e2c0 0x7f2420156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.956+0000 7f242731d640 1 --2- 172.21.15.29:0/4286353855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24200cb2a0 0x7f242015bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.956+0000 7f242731d640 1 --2- 172.21.15.29:0/4286353855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f242006b210 0x7f242015ce00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:38.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.956+0000 7f242731d640 1 -- 172.21.15.29:0/4286353855 >> 172.21.15.29:0/4286353855 conn(0x7f24200b2920 msgr2=0x7f24200b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:38.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.956+0000 7f242731d640 1 -- 172.21.15.29:0/4286353855 shutdown_connections 2024-04-25T00:31:38.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:38.956+0000 7f242731d640 1 -- 172.21.15.29:0/4286353855 wait complete. 2024-04-25T00:31:38.975 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T00:31:38.975 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:39.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.088+0000 7fb623b8f640 1 Processor -- start 2024-04-25T00:31:39.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb623b8f640 1 -- start start 2024-04-25T00:31:39.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb623b8f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb61c0d2c60 0x7fb61c168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb623b8f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb61c0cfc10 con 0x7fb61c0cf690 2024-04-25T00:31:39.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb623b8f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb61c0cfef0 con 0x7fb61c15e200 2024-04-25T00:31:39.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb623b8f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb61c0b38d0 con 0x7fb61c0d2c60 2024-04-25T00:31:39.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb62212c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb61c0d2c60 0x7fb61c168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb62212c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb61c0d2c60 0x7fb61c168e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49584/0 (socket says 172.21.15.29:49584) 2024-04-25T00:31:39.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb62112a640 1 --1- 172.21.15.29:0/4211459100 >> v1:172.21.15.29:6789/0 conn(0x7fb61c0cf690 0x7fb61c0d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:49976/0 (socket says 172.21.15.29:49976) 2024-04-25T00:31:39.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb62212c640 1 -- 172.21.15.29:0/4211459100 learned_addr learned my addr 172.21.15.29:0/4211459100 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb620929640 1 -- 172.21.15.29:0/4211459100 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099485183 0 0) 0x7fb61c0cfc10 con 0x7fb61c0cf690 2024-04-25T00:31:39.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb620929640 1 -- 172.21.15.29:0/4211459100 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb608003530 con 0x7fb61c0cf690 2024-04-25T00:31:39.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb62212c640 1 -- 172.21.15.29:0/4211459100 >> v1:172.21.15.157:6789/0 conn(0x7fb61c15e200 legacy=0x7fb61c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb62212c640 1 -- 172.21.15.29:0/4211459100 >> v1:172.21.15.29:6789/0 conn(0x7fb61c0cf690 legacy=0x7fb61c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.092+0000 7fb62212c640 1 -- 172.21.15.29:0/4211459100 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61c0b3b20 con 0x7fb61c0d2c60 2024-04-25T00:31:39.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb62212c640 1 --2- 172.21.15.29:0/4211459100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c0d2c60 0x7fb61c168e20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb614009e50 tx=0x7fb61401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4901df5d11b2e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb620929640 1 -- 172.21.15.29:0/4211459100 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6140a5720 con 0x7fb61c0d2c60 2024-04-25T00:31:39.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb620929640 1 -- 172.21.15.29:0/4211459100 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6140a58c0 con 0x7fb61c0d2c60 2024-04-25T00:31:39.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb620929640 1 -- 172.21.15.29:0/4211459100 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6140a5c90 con 0x7fb61c0d2c60 2024-04-25T00:31:39.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 -- 172.21.15.29:0/4211459100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c0d2c60 msgr2=0x7fb61c168e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 --2- 172.21.15.29:0/4211459100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c0d2c60 0x7fb61c168e20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb614009e50 tx=0x7fb61401ca30 comp rx=0 tx=0).stop 2024-04-25T00:31:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 -- 172.21.15.29:0/4211459100 shutdown_connections 2024-04-25T00:31:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 --2- 172.21.15.29:0/4211459100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c0d2c60 0x7fb61c168e20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 -- 172.21.15.29:0/4211459100 >> 172.21.15.29:0/4211459100 conn(0x7fb61c09d280 msgr2=0x7fb61c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 -- 172.21.15.29:0/4211459100 shutdown_connections 2024-04-25T00:31:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 -- 172.21.15.29:0/4211459100 wait complete. 2024-04-25T00:31:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 Processor -- start 2024-04-25T00:31:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 -- start start 2024-04-25T00:31:39.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61c0cf690 0x7fb61c154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb61c0d2c60 0x7fb61c0c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c15e200 0x7fb61c15e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c16bf10 con 0x7fb61c15e200 2024-04-25T00:31:39.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c16bd90 con 0x7fb61c0cf690 2024-04-25T00:31:39.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb623b8f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb61c16c090 con 0x7fb61c0d2c60 2024-04-25T00:31:39.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb62212c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c15e200 0x7fb61c15e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb62212c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c15e200 0x7fb61c15e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49596/0 (socket says 172.21.15.29:49596) 2024-04-25T00:31:39.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.096+0000 7fb62212c640 1 -- 172.21.15.29:0/2555144670 learned_addr learned my addr 172.21.15.29:0/2555144670 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb62192b640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61c0cf690 0x7fb61c154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb62112a640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb61c0d2c60 0x7fb61c0c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb62212c640 1 -- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb61c0d2c60 msgr2=0x7fb61c0c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb62212c640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb61c0d2c60 0x7fb61c0c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb62212c640 1 -- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61c0cf690 msgr2=0x7fb61c154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb62212c640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61c0cf690 0x7fb61c154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb62212c640 1 -- 172.21.15.29:0/2555144670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61c15f3b0 con 0x7fb61c15e200 2024-04-25T00:31:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb62212c640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c15e200 0x7fb61c15e8e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb614019640 tx=0x7fb6140197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb60e7fc640 1 -- 172.21.15.29:0/2555144670 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6140bc020 con 0x7fb61c15e200 2024-04-25T00:31:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb60e7fc640 1 -- 172.21.15.29:0/2555144670 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6140b8670 con 0x7fb61c15e200 2024-04-25T00:31:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb623b8f640 1 -- 172.21.15.29:0/2555144670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61c1544f0 con 0x7fb61c15e200 2024-04-25T00:31:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb60e7fc640 1 -- 172.21.15.29:0/2555144670 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6140b8a40 con 0x7fb61c15e200 2024-04-25T00:31:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb623b8f640 1 -- 172.21.15.29:0/2555144670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb61c0c8cf0 con 0x7fb61c15e200 2024-04-25T00:31:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb623b8f640 1 -- 172.21.15.29:0/2555144670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb61c069270 con 0x7fb61c15e200 2024-04-25T00:31:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb60e7fc640 1 -- 172.21.15.29:0/2555144670 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb6140bf020 con 0x7fb61c15e200 2024-04-25T00:31:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb60e7fc640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb6000678f0 0x7fb600069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb60e7fc640 1 -- 172.21.15.29:0/2555144670 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb614120e30 con 0x7fb61c15e200 2024-04-25T00:31:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb62192b640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb6000678f0 0x7fb600069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.100+0000 7fb62192b640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb6000678f0 0x7fb600069db0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb610000f70 tx=0x7fb610009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.104+0000 7fb60e7fc640 1 -- 172.21.15.29:0/2555144670 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb614016620 con 0x7fb61c15e200 2024-04-25T00:31:39.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.268+0000 7fb623b8f640 1 -- 172.21.15.29:0/2555144670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb61c0b35b0 con 0x7fb61c15e200 2024-04-25T00:31:39.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.272+0000 7fb60e7fc640 1 -- 172.21.15.29:0/2555144670 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fb6140f1b20 con 0x7fb61c15e200 2024-04-25T00:31:39.273 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:39.273 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:39.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.272+0000 7fb623b8f640 1 -- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb6000678f0 msgr2=0x7fb600069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.272+0000 7fb623b8f640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb6000678f0 0x7fb600069db0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb610000f70 tx=0x7fb610009040 comp rx=0 tx=0).stop 2024-04-25T00:31:39.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.272+0000 7fb623b8f640 1 -- 172.21.15.29:0/2555144670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c15e200 msgr2=0x7fb61c15e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.272+0000 7fb623b8f640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c15e200 0x7fb61c15e8e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb614019640 tx=0x7fb6140197d0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.272+0000 7fb623b8f640 1 -- 172.21.15.29:0/2555144670 shutdown_connections 2024-04-25T00:31:39.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.272+0000 7fb623b8f640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c15e200 0x7fb61c15e8e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.272+0000 7fb623b8f640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb61c0d2c60 0x7fb61c0c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.272+0000 7fb623b8f640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb6000678f0 0x7fb600069db0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.272+0000 7fb623b8f640 1 --2- 172.21.15.29:0/2555144670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61c0cf690 0x7fb61c154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.272+0000 7fb623b8f640 1 -- 172.21.15.29:0/2555144670 >> 172.21.15.29:0/2555144670 conn(0x7fb61c09d280 msgr2=0x7fb61c0d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.272+0000 7fb623b8f640 1 -- 172.21.15.29:0/2555144670 shutdown_connections 2024-04-25T00:31:39.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.272+0000 7fb623b8f640 1 -- 172.21.15.29:0/2555144670 wait complete. 2024-04-25T00:31:39.293 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T00:31:39.293 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T00:31:39.294 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T00:31:39.294 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T00:31:39.295 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T00:31:39.295 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T00:31:39.295 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T00:31:39.296 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T00:31:39.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.472+0000 7fb936f84640 1 Processor -- start 2024-04-25T00:31:39.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.472+0000 7fb936f84640 1 -- start start 2024-04-25T00:31:39.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.472+0000 7fa3aa996640 1 Processor -- start 2024-04-25T00:31:39.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.472+0000 7fa3aa996640 1 -- start start 2024-04-25T00:31:39.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.472+0000 7f2612968640 1 Processor -- start 2024-04-25T00:31:39.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.472+0000 7f915429f640 1 Processor -- start 2024-04-25T00:31:39.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.472+0000 7f915429f640 1 -- start start 2024-04-25T00:31:39.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.472+0000 7fb936f84640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9300c5170 0x7fb9300ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fb934d20640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb9300b2200 0x7fb93006c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:49980/0 (socket says 172.21.15.29:49980) 2024-04-25T00:31:39.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fb934d20640 1 -- 172.21.15.29:0/2645122672 learned_addr learned my addr 172.21.15.29:0/2645122672 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:39.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fb936f84640 1 -- 172.21.15.29:0/2645122672 --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9300c5600 con 0x7fb9300b2200 2024-04-25T00:31:39.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fb936f84640 1 -- 172.21.15.29:0/2645122672 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9301663e0 con 0x7fb9300c4ce0 2024-04-25T00:31:39.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f915429f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f914c0aec60 0x7f914c0b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f915429f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f914c072da0 con 0x7f914c071750 2024-04-25T00:31:39.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f915429f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f914c0cf950 con 0x7f914c0b2fc0 2024-04-25T00:31:39.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f915429f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f914c172710 con 0x7f914c0aec60 2024-04-25T00:31:39.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fa3aa996640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa39c0e30d0 0x7fa39c03a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fb936f84640 1 -- 172.21.15.29:0/2645122672 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb9300b25e0 con 0x7fb9300c5170 2024-04-25T00:31:39.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fb935521640 1 --2- 172.21.15.29:0/2645122672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9300c5170 0x7fb9300ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.472+0000 7f2612968640 1 -- start start 2024-04-25T00:31:39.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f2612968640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2604033e40 0x7f26040eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f2612968640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2604047c20 0x7f26040eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f2612968640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f26040ebc00 0x7f2604052f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f2612968640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2604050210 con 0x7f2604033e40 2024-04-25T00:31:39.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f2612968640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2604037bc0 con 0x7f26040ebc00 2024-04-25T00:31:39.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f2612968640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f260404f090 con 0x7f2604047c20 2024-04-25T00:31:39.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fa3aa996640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa39c04ff20 0x7fa39c006380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f915183a640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f914c071750 0x7f914c0b28b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:49990/0 (socket says 172.21.15.29:49990) 2024-04-25T00:31:39.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f915183a640 1 -- 172.21.15.29:0/1914123826 learned_addr learned my addr 172.21.15.29:0/1914123826 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:39.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fa3aa996640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa39c053e70 con 0x7fa39c053970 2024-04-25T00:31:39.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f915203b640 1 --2- 172.21.15.29:0/1914123826 >> v2:172.21.15.157:3300/0 conn(0x7f914c0aec60 0x7f914c0b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fb92f7fe640 1 -- 172.21.15.29:0/2645122672 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3332523885 0 0) 0x7fb9301663e0 con 0x7fb9300c4ce0 2024-04-25T00:31:39.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fb92f7fe640 1 -- 172.21.15.29:0/2645122672 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb920003530 con 0x7fb9300c4ce0 2024-04-25T00:31:39.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f260bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2604033e40 0x7f26040eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f260bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2604033e40 0x7f26040eb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49602/0 (socket says 172.21.15.29:49602) 2024-04-25T00:31:39.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f260bfff640 1 -- 172.21.15.29:0/1195562651 learned_addr learned my addr 172.21.15.29:0/1195562651 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f2610f05640 1 --2- 172.21.15.29:0/1195562651 >> v2:172.21.15.157:3300/0 conn(0x7f26040ebc00 0x7f2604052f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fa3a37fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa39c053970 0x7fa39c04f810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:43958/0 (socket says 172.21.15.29:43958) 2024-04-25T00:31:39.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fa3a37fe640 1 -- 172.21.15.29:0/2085750280 learned_addr learned my addr 172.21.15.29:0/2085750280 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:39.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/2085750280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa39c0e30d0 0x7fa39c03a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fa3aa996640 1 -- 172.21.15.29:0/2085750280 --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa39c037bc0 con 0x7fa39c04ff20 2024-04-25T00:31:39.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fa3aa996640 1 -- 172.21.15.29:0/2085750280 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa39c054150 con 0x7fa39c0e30d0 2024-04-25T00:31:39.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f260b7fe640 1 --2- 172.21.15.29:0/1195562651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2604047c20 0x7f26040eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f2610f05640 1 -- 172.21.15.29:0/1195562651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2604047c20 msgr2=0x7f26040eb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f2610f05640 1 --2- 172.21.15.29:0/1195562651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2604047c20 0x7f26040eb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f2610f05640 1 -- 172.21.15.29:0/1195562651 >> v2:172.21.15.29:3300/0 conn(0x7f2604033e40 msgr2=0x7f26040eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f2610f05640 1 --2- 172.21.15.29:0/1195562651 >> v2:172.21.15.29:3300/0 conn(0x7f2604033e40 0x7f26040eb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f2610f05640 1 -- 172.21.15.29:0/1195562651 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2604034220 con 0x7f26040ebc00 2024-04-25T00:31:39.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f9151039640 1 -- 172.21.15.29:0/1914123826 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3009545274 0 0) 0x7f914c072da0 con 0x7f914c071750 2024-04-25T00:31:39.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f9151039640 1 -- 172.21.15.29:0/1914123826 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9138003530 con 0x7f914c071750 2024-04-25T00:31:39.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f915203b640 1 -- 172.21.15.29:0/1914123826 >> v1:172.21.15.157:6789/0 conn(0x7f914c0b2fc0 legacy=0x7f914c08b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f915203b640 1 -- 172.21.15.29:0/1914123826 >> v1:172.21.15.29:6789/0 conn(0x7f914c071750 legacy=0x7f914c0b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f915203b640 1 -- 172.21.15.29:0/1914123826 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c0cfc30 con 0x7f914c0aec60 2024-04-25T00:31:39.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fb935521640 1 -- 172.21.15.29:0/2645122672 >> v1:172.21.15.157:6789/0 conn(0x7fb9300c4ce0 legacy=0x7fb9300cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fb935521640 1 -- 172.21.15.29:0/2645122672 >> v1:172.21.15.29:6789/0 conn(0x7fb9300b2200 legacy=0x7fb93006c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fb935521640 1 -- 172.21.15.29:0/2645122672 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9301666c0 con 0x7fb9300c5170 2024-04-25T00:31:39.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eaf59e640 1 Processor -- start 2024-04-25T00:31:39.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eaf59e640 1 -- start start 2024-04-25T00:31:39.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eaf59e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7eb015fce0 0x7f7eb01633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eaf59e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb0166570 con 0x7f7eb00748b0 2024-04-25T00:31:39.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eaf59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb0166850 con 0x7f7eb01600c0 2024-04-25T00:31:39.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eaf59e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb0166e10 con 0x7f7eb015fce0 2024-04-25T00:31:39.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eaed9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7eb00748b0 0x7f7eb0164090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50000/0 (socket says 172.21.15.29:50000) 2024-04-25T00:31:39.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eaed9d640 1 -- 172.21.15.29:0/2754171303 learned_addr learned my addr 172.21.15.29:0/2754171303 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:39.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eae59c640 1 --2- 172.21.15.29:0/2754171303 >> v2:172.21.15.29:3300/0 conn(0x7f7eb015fce0 0x7f7eb01633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fa3a3fff640 1 -- 172.21.15.29:0/2085750280 >> v2:172.21.15.29:3300/0 conn(0x7fa39c04ff20 msgr2=0x7fa39c006380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/2085750280 >> v2:172.21.15.29:3300/0 conn(0x7fa39c04ff20 0x7fa39c006380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fa3a3fff640 1 -- 172.21.15.29:0/2085750280 >> v1:172.21.15.157:6789/0 conn(0x7fa39c053970 legacy=0x7fa39c04f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7fa3a3fff640 1 -- 172.21.15.29:0/2085750280 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39c054430 con 0x7fa39c0e30d0 2024-04-25T00:31:39.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fb935521640 1 --2- 172.21.15.29:0/2645122672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9300c5170 0x7fb9300ccd50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb928009dc0 tx=0x7fb92801c9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10e18d11cc63502c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eadd9b640 1 -- 172.21.15.29:0/2754171303 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 618688752 0 0) 0x7f7eb0166570 con 0x7f7eb00748b0 2024-04-25T00:31:39.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eadd9b640 1 -- 172.21.15.29:0/2754171303 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e9c003530 con 0x7f7eb00748b0 2024-04-25T00:31:39.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eae59c640 1 -- 172.21.15.29:0/2754171303 >> v1:172.21.15.157:6789/0 conn(0x7f7eb01600c0 legacy=0x7f7eb00741a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:31:39.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eae59c640 1 -- 172.21.15.29:0/2754171303 >> v1:172.21.15.29:6789/0 conn(0x7f7eb00748b0 legacy=0x7f7eb0164090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.476+0000 7f7eae59c640 1 -- 172.21.15.29:0/2754171303 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb0166b30 con 0x7f7eb015fce0 2024-04-25T00:31:39.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f2610f05640 1 --2- 172.21.15.29:0/1195562651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26040ebc00 0x7f2604052f60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f260400df50 tx=0x7f25f801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3e8371c55575010 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/2085750280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa39c0e30d0 0x7fa39c03a440 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa3900097c0 tx=0x7fa39001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d693118b48a172d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fb92f7fe640 1 -- 172.21.15.29:0/2645122672 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9280b2070 con 0x7fb9300c5170 2024-04-25T00:31:39.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fb92f7fe640 1 -- 172.21.15.29:0/2645122672 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9280a5a90 con 0x7fb9300c5170 2024-04-25T00:31:39.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fb92f7fe640 1 -- 172.21.15.29:0/2645122672 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9280a5e40 con 0x7fb9300c5170 2024-04-25T00:31:39.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f260affd640 1 -- 172.21.15.29:0/1195562651 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f80b5070 con 0x7f26040ebc00 2024-04-25T00:31:39.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f260affd640 1 -- 172.21.15.29:0/1195562651 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f80a8830 con 0x7f26040ebc00 2024-04-25T00:31:39.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f3725d8e640 1 Processor -- start 2024-04-25T00:31:39.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f3725d8e640 1 -- start start 2024-04-25T00:31:39.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fdf564e6640 1 Processor -- start 2024-04-25T00:31:39.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fa3a2ffd640 1 -- 172.21.15.29:0/2085750280 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3900ab5f0 con 0x7fa39c0e30d0 2024-04-25T00:31:39.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f915203b640 1 --2- 172.21.15.29:0/1914123826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914c0aec60 0x7f914c0b4720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f913c0097c0 tx=0x7f913c01ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c437c13340036c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fa3a2ffd640 1 -- 172.21.15.29:0/2085750280 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3900b0070 con 0x7fa39c0e30d0 2024-04-25T00:31:39.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fa3a2ffd640 1 -- 172.21.15.29:0/2085750280 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3900ab9f0 con 0x7fa39c0e30d0 2024-04-25T00:31:39.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f260affd640 1 -- 172.21.15.29:0/1195562651 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f80a8c00 con 0x7f26040ebc00 2024-04-25T00:31:39.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f2612968640 1 -- 172.21.15.29:0/1195562651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26040ebc00 msgr2=0x7f2604052f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f2612968640 1 --2- 172.21.15.29:0/1195562651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26040ebc00 0x7f2604052f60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f260400df50 tx=0x7f25f801ec90 comp rx=0 tx=0).stop 2024-04-25T00:31:39.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fb936f84640 1 -- 172.21.15.29:0/2645122672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9300c5170 msgr2=0x7fb9300ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fb936f84640 1 --2- 172.21.15.29:0/2645122672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9300c5170 0x7fb9300ccd50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb928009dc0 tx=0x7fb92801c9d0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fb936f84640 1 -- 172.21.15.29:0/2645122672 shutdown_connections 2024-04-25T00:31:39.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fb936f84640 1 --2- 172.21.15.29:0/2645122672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9300c5170 0x7fb9300ccd50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fb936f84640 1 -- 172.21.15.29:0/2645122672 >> 172.21.15.29:0/2645122672 conn(0x7fb93009aa30 msgr2=0x7fb93009ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f2612968640 1 -- 172.21.15.29:0/1195562651 shutdown_connections 2024-04-25T00:31:39.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f2612968640 1 --2- 172.21.15.29:0/1195562651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26040ebc00 0x7f2604052f60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f2612968640 1 --2- 172.21.15.29:0/1195562651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2604047c20 0x7f26040eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f2612968640 1 --2- 172.21.15.29:0/1195562651 >> v2:172.21.15.29:3300/0 conn(0x7f2604033e40 0x7f26040eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f2612968640 1 -- 172.21.15.29:0/1195562651 >> 172.21.15.29:0/1195562651 conn(0x7f2604009fc0 msgr2=0x7f260400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f9151039640 1 -- 172.21.15.29:0/1914123826 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c0b5070 con 0x7f914c0aec60 2024-04-25T00:31:39.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f9151039640 1 -- 172.21.15.29:0/1914123826 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f913c0a8990 con 0x7f914c0aec60 2024-04-25T00:31:39.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fa3aa996640 1 -- 172.21.15.29:0/2085750280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa39c0e30d0 msgr2=0x7fa39c03a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fa3aa996640 1 --2- 172.21.15.29:0/2085750280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa39c0e30d0 0x7fa39c03a440 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa3900097c0 tx=0x7fa39001c4e0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fb936f84640 1 -- 172.21.15.29:0/2645122672 shutdown_connections 2024-04-25T00:31:39.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f9151039640 1 -- 172.21.15.29:0/1914123826 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c0a8d60 con 0x7f914c0aec60 2024-04-25T00:31:39.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fb936f84640 1 -- 172.21.15.29:0/2645122672 wait complete. 2024-04-25T00:31:39.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f2612968640 1 -- 172.21.15.29:0/1195562651 shutdown_connections 2024-04-25T00:31:39.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fdf564e6640 1 -- start start 2024-04-25T00:31:39.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fa3aa996640 1 -- 172.21.15.29:0/2085750280 shutdown_connections 2024-04-25T00:31:39.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fa3aa996640 1 --2- 172.21.15.29:0/2085750280 >> v2:172.21.15.29:3300/0 conn(0x7fa39c04ff20 0x7fa39c006380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fa3aa996640 1 --2- 172.21.15.29:0/2085750280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa39c0e30d0 0x7fa39c03a440 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fa3aa996640 1 -- 172.21.15.29:0/2085750280 >> 172.21.15.29:0/2085750280 conn(0x7fa39c009fc0 msgr2=0x7fa39c00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f3725d8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37180e00d0 0x7f3718038e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f3725d8e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f371803a610 0x7f3718039370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f3725d8e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3718039b00 con 0x7f3718033a10 2024-04-25T00:31:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f3725d8e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f37180076f0 con 0x7f371803a610 2024-04-25T00:31:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f3725d8e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f371803a110 con 0x7f37180e00d0 2024-04-25T00:31:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fdf564e6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf5015c220 0x7fdf500cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fdf564e6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdf500cfa10 0x7fdf500cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fdf564e6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf500cdbd0 con 0x7fdf500d04a0 2024-04-25T00:31:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fdf564e6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdf500d0900 con 0x7fdf500cfa10 2024-04-25T00:31:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fdf564e6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5015c600 con 0x7fdf5015c220 2024-04-25T00:31:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f371ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f371803a610 0x7f3718039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f371f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37180e00d0 0x7f3718038e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f371ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f371803a610 0x7f3718039370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49622/0 (socket says 172.21.15.29:49622) 2024-04-25T00:31:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f371ffff640 1 -- 172.21.15.29:0/2632340693 learned_addr learned my addr 172.21.15.29:0/2632340693 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fdf54a83640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdf500d04a0 0x7fdf500ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50014/0 (socket says 172.21.15.29:50014) 2024-04-25T00:31:39.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fdf54a83640 1 -- 172.21.15.29:0/736199854 learned_addr learned my addr 172.21.15.29:0/736199854 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:39.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f915429f640 1 -- 172.21.15.29:0/1914123826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914c0aec60 msgr2=0x7f914c0b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f915429f640 1 --2- 172.21.15.29:0/1914123826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914c0aec60 0x7f914c0b4720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f913c0097c0 tx=0x7f913c01ecb0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fdf4ffff640 1 --2- 172.21.15.29:0/736199854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf5015c220 0x7fdf500cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fa3aa996640 1 -- 172.21.15.29:0/2085750280 shutdown_connections 2024-04-25T00:31:39.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fa3aa996640 1 -- 172.21.15.29:0/2085750280 wait complete. 2024-04-25T00:31:39.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7fdf4f7fe640 1 --2- 172.21.15.29:0/736199854 >> v2:172.21.15.157:3300/0 conn(0x7fdf500cfa10 0x7fdf500cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f371f7fe640 1 -- 172.21.15.29:0/2632340693 >> v2:172.21.15.29:3300/0 conn(0x7f371803a610 msgr2=0x7f3718039370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f371f7fe640 1 --2- 172.21.15.29:0/2632340693 >> v2:172.21.15.29:3300/0 conn(0x7f371803a610 0x7f3718039370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f371f7fe640 1 -- 172.21.15.29:0/2632340693 >> v1:172.21.15.157:6789/0 conn(0x7f3718033a10 legacy=0x7f37180526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f371f7fe640 1 -- 172.21.15.29:0/2632340693 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3718034000 con 0x7f37180e00d0 2024-04-25T00:31:39.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fdf4ffff640 1 -- 172.21.15.29:0/736199854 >> v2:172.21.15.157:3300/0 conn(0x7fdf500cfa10 msgr2=0x7fdf500cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.480+0000 7f2612968640 1 -- 172.21.15.29:0/1195562651 wait complete. 2024-04-25T00:31:39.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f2612968640 1 Processor -- start 2024-04-25T00:31:39.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb936f84640 1 Processor -- start 2024-04-25T00:31:39.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb936f84640 1 -- start start 2024-04-25T00:31:39.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3aa996640 1 Processor -- start 2024-04-25T00:31:39.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3aa996640 1 -- start start 2024-04-25T00:31:39.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f7eae59c640 1 --2- 172.21.15.29:0/2754171303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb015fce0 0x7f7eb01633b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0166f90 tx=0x7f7ea001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f531467b18a143 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fdf4effd640 1 -- 172.21.15.29:0/736199854 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2791160404 0 0) 0x7fdf500cdbd0 con 0x7fdf500d04a0 2024-04-25T00:31:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f2612968640 1 -- start start 2024-04-25T00:31:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f7eadd9b640 1 -- 172.21.15.29:0/2754171303 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00b2070 con 0x7f7eb015fce0 2024-04-25T00:31:39.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3aa996640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa39c04ff20 0x7fa39c0cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3aa996640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa39c053970 0x7fa39c04e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f7eadd9b640 1 -- 172.21.15.29:0/2754171303 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea00ad680 con 0x7f7eb015fce0 2024-04-25T00:31:39.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3aa996640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39c0e30d0 0x7fa39c04c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3aa996640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c007250 con 0x7fa39c04ff20 2024-04-25T00:31:39.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3aa996640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c0070d0 con 0x7fa39c0e30d0 2024-04-25T00:31:39.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3aa996640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa39c0073d0 con 0x7fa39c053970 2024-04-25T00:31:39.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f915429f640 1 -- 172.21.15.29:0/1914123826 shutdown_connections 2024-04-25T00:31:39.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f915429f640 1 --2- 172.21.15.29:0/1914123826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914c0aec60 0x7f914c0b4720 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f915429f640 1 -- 172.21.15.29:0/1914123826 >> 172.21.15.29:0/1914123826 conn(0x7f914c074880 msgr2=0x7f914c06b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f915429f640 1 -- 172.21.15.29:0/1914123826 shutdown_connections 2024-04-25T00:31:39.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f915429f640 1 -- 172.21.15.29:0/1914123826 wait complete. 2024-04-25T00:31:39.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3a3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa39c04ff20 0x7fa39c0cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f7eadd9b640 1 -- 172.21.15.29:0/2754171303 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00b2070 con 0x7f7eb015fce0 2024-04-25T00:31:39.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f2612968640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2604033e40 0x7f26040daf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3a3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa39c04ff20 0x7fa39c0cbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49636/0 (socket says 172.21.15.29:49636) 2024-04-25T00:31:39.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3a3fff640 1 -- 172.21.15.29:0/1726896486 learned_addr learned my addr 172.21.15.29:0/1726896486 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3a37fe640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa39c053970 0x7fa39c04e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3a8f33640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39c0e30d0 0x7fa39c04c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb936f84640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9300b2200 0x7fb9300d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb936f84640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9300c4ce0 0x7fb930158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb936f84640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9300c5170 0x7fb9300cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb936f84640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb93006ca30 con 0x7fb9300c4ce0 2024-04-25T00:31:39.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb936f84640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb9301660b0 con 0x7fb9300b2200 2024-04-25T00:31:39.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f2612968640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2604047c20 0x7f2604045aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb936f84640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb93006cbb0 con 0x7fb9300c5170 2024-04-25T00:31:39.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fdf4ffff640 1 --2- 172.21.15.29:0/736199854 >> v2:172.21.15.157:3300/0 conn(0x7fdf500cfa10 0x7fdf500cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fdf4ffff640 1 -- 172.21.15.29:0/736199854 >> v1:172.21.15.29:6789/0 conn(0x7fdf500d04a0 legacy=0x7fdf500ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fdf4ffff640 1 -- 172.21.15.29:0/736199854 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf500b13e0 con 0x7fdf5015c220 2024-04-25T00:31:39.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3a3fff640 1 -- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa39c053970 msgr2=0x7fa39c04e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa39c053970 0x7fa39c04e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3a3fff640 1 -- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39c0e30d0 msgr2=0x7fa39c04c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39c0e30d0 0x7fa39c04c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fa3a3fff640 1 -- 172.21.15.29:0/1726896486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39c0346f0 con 0x7fa39c04ff20 2024-04-25T00:31:39.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb934d20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9300b2200 0x7fb9300d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb92ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9300c4ce0 0x7fb930158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb934d20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9300b2200 0x7fb9300d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:60670/0 (socket says 172.21.15.29:60670) 2024-04-25T00:31:39.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb934d20640 1 -- 172.21.15.29:0/3291469533 learned_addr learned my addr 172.21.15.29:0/3291469533 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb935521640 1 --2- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9300c5170 0x7fb9300cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f260b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2604047c20 0x7f2604045aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f260b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2604047c20 0x7f2604045aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:60674/0 (socket says 172.21.15.29:60674) 2024-04-25T00:31:39.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f260bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2604033e40 0x7f26040daf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f260bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2604033e40 0x7f26040daf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49652/0 (socket says 172.21.15.29:49652) 2024-04-25T00:31:39.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f2612968640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26040ebc00 0x7f2604045fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f915429f640 1 Processor -- start 2024-04-25T00:31:39.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb92ffff640 1 -- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9300c5170 msgr2=0x7fb9300cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb92ffff640 1 --2- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9300c5170 0x7fb9300cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb92ffff640 1 -- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9300b2200 msgr2=0x7fb9300d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb92ffff640 1 --2- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9300b2200 0x7fb9300d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb92ffff640 1 -- 172.21.15.29:0/3291469533 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9300ae980 con 0x7fb9300c4ce0 2024-04-25T00:31:39.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f2612968640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f260404f2e0 con 0x7f2604033e40 2024-04-25T00:31:39.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f2612968640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f260404fd80 con 0x7f2604047c20 2024-04-25T00:31:39.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f2612968640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f260404f5c0 con 0x7f26040ebc00 2024-04-25T00:31:39.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f260bfff640 1 -- 172.21.15.29:0/3965086109 learned_addr learned my addr 172.21.15.29:0/3965086109 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f2610f05640 1 --2- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26040ebc00 0x7f2604045fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7fb92ffff640 1 --2- 172.21.15.29:0/3291469533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9300c4ce0 0x7fb930158590 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb92400c430 tx=0x7fb92400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f260b7fe640 1 -- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26040ebc00 msgr2=0x7f2604045fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f260b7fe640 1 --2- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26040ebc00 0x7f2604045fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f260b7fe640 1 -- 172.21.15.29:0/3965086109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2604033e40 msgr2=0x7f26040daf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f260b7fe640 1 --2- 172.21.15.29:0/3965086109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2604033e40 0x7f26040daf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f260b7fe640 1 -- 172.21.15.29:0/3965086109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2604034880 con 0x7f2604047c20 2024-04-25T00:31:39.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f915429f640 1 -- start start 2024-04-25T00:31:39.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f915429f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914c071750 0x7f914c172080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f915429f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914c0aec60 0x7f914c15ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f915429f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f914c0b2fc0 0x7f914c15f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f915429f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f914c16c760 con 0x7f914c0b2fc0 2024-04-25T00:31:39.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f915429f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f914c0d01f0 con 0x7f914c0aec60 2024-04-25T00:31:39.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.484+0000 7f915429f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f914c16c8e0 con 0x7f914c071750 2024-04-25T00:31:39.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fa3a3fff640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa39c04ff20 0x7fa39c0cbd10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa39001a110 tx=0x7fa39001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fdf4f7fe640 1 --2- 172.21.15.29:0/736199854 >> v2:172.21.15.157:3300/0 conn(0x7fdf500cfa10 0x7fdf500cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:39.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fb92d7fa640 1 -- 172.21.15.29:0/3291469533 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924017070 con 0x7fb9300c4ce0 2024-04-25T00:31:39.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fb92d7fa640 1 -- 172.21.15.29:0/3291469533 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9240052d0 con 0x7fb9300c4ce0 2024-04-25T00:31:39.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fb92d7fa640 1 -- 172.21.15.29:0/3291469533 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240054c0 con 0x7fb9300c4ce0 2024-04-25T00:31:39.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f7eaf59e640 1 -- 172.21.15.29:0/2754171303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb015fce0 msgr2=0x7f7eb01633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f7eaf59e640 1 --2- 172.21.15.29:0/2754171303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb015fce0 0x7f7eb01633b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0166f90 tx=0x7f7ea001c3a0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f7eaf59e640 1 -- 172.21.15.29:0/2754171303 shutdown_connections 2024-04-25T00:31:39.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f7eaf59e640 1 --2- 172.21.15.29:0/2754171303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb015fce0 0x7f7eb01633b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f7eaf59e640 1 -- 172.21.15.29:0/2754171303 >> 172.21.15.29:0/2754171303 conn(0x7f7eb006b210 msgr2=0x7f7eb006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f7eaf59e640 1 -- 172.21.15.29:0/2754171303 shutdown_connections 2024-04-25T00:31:39.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f7eaf59e640 1 -- 172.21.15.29:0/2754171303 wait complete. 2024-04-25T00:31:39.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f915283c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f914c0b2fc0 0x7f914c15f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f915203b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914c071750 0x7f914c172080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f915283c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f914c0b2fc0 0x7f914c15f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49670/0 (socket says 172.21.15.29:49670) 2024-04-25T00:31:39.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f915283c640 1 -- 172.21.15.29:0/4244893273 learned_addr learned my addr 172.21.15.29:0/4244893273 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f915283c640 1 -- 172.21.15.29:0/4244893273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914c071750 msgr2=0x7f914c172080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f915283c640 1 --2- 172.21.15.29:0/4244893273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914c071750 0x7f914c172080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f915283c640 1 -- 172.21.15.29:0/4244893273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914c0aec60 msgr2=0x7f914c15ec30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:31:39.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f915283c640 1 --2- 172.21.15.29:0/4244893273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914c0aec60 0x7f914c15ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f915283c640 1 -- 172.21.15.29:0/4244893273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c0ce0e0 con 0x7f914c0b2fc0 2024-04-25T00:31:39.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f371f7fe640 1 --2- 172.21.15.29:0/2632340693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37180e00d0 0x7f3718038e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f37140097c0 tx=0x7f371401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d845d561c06442a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f371e7fc640 1 -- 172.21.15.29:0/2632340693 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37140b0070 con 0x7f37180e00d0 2024-04-25T00:31:39.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f371e7fc640 1 -- 172.21.15.29:0/2632340693 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37140ab600 con 0x7f37180e00d0 2024-04-25T00:31:39.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fdf4ffff640 1 --2- 172.21.15.29:0/736199854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf5015c220 0x7fdf500cd350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdf400097c0 tx=0x7fdf4001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=756e1b36c9c2c399 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fdf4effd640 1 -- 172.21.15.29:0/736199854 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf400b0070 con 0x7fdf5015c220 2024-04-25T00:31:39.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fdf4effd640 1 -- 172.21.15.29:0/736199854 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf400ab600 con 0x7fdf5015c220 2024-04-25T00:31:39.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fb936f84640 1 -- 172.21.15.29:0/3291469533 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9300aec60 con 0x7fb9300c4ce0 2024-04-25T00:31:39.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fb936f84640 1 -- 172.21.15.29:0/3291469533 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9300af220 con 0x7fb9300c4ce0 2024-04-25T00:31:39.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fb936f84640 1 -- 172.21.15.29:0/3291469533 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb930069270 con 0x7fb9300c4ce0 2024-04-25T00:31:39.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f260b7fe640 1 --2- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2604047c20 0x7f2604045aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f260000d5f0 tx=0x7f260000dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f2608ff9640 1 -- 172.21.15.29:0/3965086109 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26000125f0 con 0x7f2604047c20 2024-04-25T00:31:39.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f2608ff9640 1 -- 172.21.15.29:0/3965086109 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2600019070 con 0x7f2604047c20 2024-04-25T00:31:39.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f2608ff9640 1 -- 172.21.15.29:0/3965086109 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26000129b0 con 0x7f2604047c20 2024-04-25T00:31:39.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f2612968640 1 -- 172.21.15.29:0/3965086109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2604037080 con 0x7f2604047c20 2024-04-25T00:31:39.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f2612968640 1 -- 172.21.15.29:0/3965086109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26040e73d0 con 0x7f2604047c20 2024-04-25T00:31:39.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f2612968640 1 -- 172.21.15.29:0/3965086109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f26040c94d0 con 0x7f2604047c20 2024-04-25T00:31:39.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f7eaf59e640 1 Processor -- start 2024-04-25T00:31:39.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f7eaf59e640 1 -- start start 2024-04-25T00:31:39.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f915283c640 1 --2- 172.21.15.29:0/4244893273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f914c0b2fc0 0x7f914c15f170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f914400c760 tx=0x7f914400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f371e7fc640 1 -- 172.21.15.29:0/2632340693 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37140ab9d0 con 0x7f37180e00d0 2024-04-25T00:31:39.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f3725d8e640 1 -- 172.21.15.29:0/2632340693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37180e00d0 msgr2=0x7f3718038e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f3725d8e640 1 --2- 172.21.15.29:0/2632340693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37180e00d0 0x7f3718038e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f37140097c0 tx=0x7f371401c4c0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f3725d8e640 1 -- 172.21.15.29:0/2632340693 shutdown_connections 2024-04-25T00:31:39.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f3725d8e640 1 --2- 172.21.15.29:0/2632340693 >> v2:172.21.15.29:3300/0 conn(0x7f371803a610 0x7f3718039370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f3725d8e640 1 --2- 172.21.15.29:0/2632340693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37180e00d0 0x7f3718038e30 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f37140097c0 tx=0x7f371401c4c0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f3725d8e640 1 -- 172.21.15.29:0/2632340693 >> 172.21.15.29:0/2632340693 conn(0x7f3718054b00 msgr2=0x7f3718054f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fdf564e6640 1 -- 172.21.15.29:0/736199854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf5015c220 msgr2=0x7fdf500cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fdf564e6640 1 --2- 172.21.15.29:0/736199854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf5015c220 0x7fdf500cd350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdf400097c0 tx=0x7fdf4001c4c0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f9142ffd640 1 -- 172.21.15.29:0/4244893273 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144016020 con 0x7f914c0b2fc0 2024-04-25T00:31:39.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f915429f640 1 -- 172.21.15.29:0/4244893273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914c08b660 con 0x7f914c0b2fc0 2024-04-25T00:31:39.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f915429f640 1 -- 172.21.15.29:0/4244893273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914c0ce660 con 0x7f914c0b2fc0 2024-04-25T00:31:39.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f9142ffd640 1 -- 172.21.15.29:0/4244893273 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9144006d80 con 0x7f914c0b2fc0 2024-04-25T00:31:39.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f9142ffd640 1 -- 172.21.15.29:0/4244893273 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144003bc0 con 0x7f914c0b2fc0 2024-04-25T00:31:39.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f3725d8e640 1 -- 172.21.15.29:0/2632340693 shutdown_connections 2024-04-25T00:31:39.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fdf564e6640 1 -- 172.21.15.29:0/736199854 shutdown_connections 2024-04-25T00:31:39.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fdf564e6640 1 --2- 172.21.15.29:0/736199854 >> v2:172.21.15.157:3300/0 conn(0x7fdf500cfa10 0x7fdf500cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fdf564e6640 1 --2- 172.21.15.29:0/736199854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf5015c220 0x7fdf500cd350 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fdf564e6640 1 -- 172.21.15.29:0/736199854 >> 172.21.15.29:0/736199854 conn(0x7fdf500d1d60 msgr2=0x7fdf500d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fdf564e6640 1 -- 172.21.15.29:0/736199854 shutdown_connections 2024-04-25T00:31:39.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fa3a0ff9640 1 -- 172.21.15.29:0/1726896486 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3900bf610 con 0x7fa39c04ff20 2024-04-25T00:31:39.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fa3a0ff9640 1 -- 172.21.15.29:0/1726896486 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3900bf7d0 con 0x7fa39c04ff20 2024-04-25T00:31:39.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fa3a0ff9640 1 -- 172.21.15.29:0/1726896486 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3900bfbc0 con 0x7fa39c04ff20 2024-04-25T00:31:39.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eaf59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb00748b0 0x7f7eb00cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eaf59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7eb015fce0 0x7f7eb0161a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eaf59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eb01600c0 0x7f7eb00aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eaf59e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0165340 con 0x7f7eb00748b0 2024-04-25T00:31:39.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eaf59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01651c0 con 0x7f7eb01600c0 2024-04-25T00:31:39.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eaf59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb01289a0 con 0x7f7eb015fce0 2024-04-25T00:31:39.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eae59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb00748b0 0x7f7eb00cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eae59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb00748b0 0x7f7eb00cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49676/0 (socket says 172.21.15.29:49676) 2024-04-25T00:31:39.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eae59c640 1 -- 172.21.15.29:0/3257332226 learned_addr learned my addr 172.21.15.29:0/3257332226 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eae59c640 1 -- 172.21.15.29:0/3257332226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7eb015fce0 msgr2=0x7f7eb0161a40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:31:39.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eae59c640 1 --2- 172.21.15.29:0/3257332226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7eb015fce0 0x7f7eb0161a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eae59c640 1 -- 172.21.15.29:0/3257332226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eb01600c0 msgr2=0x7f7eb00aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eae59c640 1 --2- 172.21.15.29:0/3257332226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eb01600c0 0x7f7eb00aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eae59c640 1 -- 172.21.15.29:0/3257332226 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb00ced20 con 0x7f7eb00748b0 2024-04-25T00:31:39.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eae59c640 1 --2- 172.21.15.29:0/3257332226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb00748b0 0x7f7eb00cfdc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0018fb0 tx=0x7f7ea001abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7ea6ffd640 1 -- 172.21.15.29:0/3257332226 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00bd070 con 0x7f7eb00748b0 2024-04-25T00:31:39.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7ea6ffd640 1 -- 172.21.15.29:0/3257332226 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea00adc40 con 0x7f7eb00748b0 2024-04-25T00:31:39.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eaf59e640 1 -- 172.21.15.29:0/3257332226 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb00cf000 con 0x7f7eb00748b0 2024-04-25T00:31:39.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7ea6ffd640 1 -- 172.21.15.29:0/3257332226 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00b2070 con 0x7f7eb00748b0 2024-04-25T00:31:39.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eaf59e640 1 -- 172.21.15.29:0/3257332226 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb00cf5c0 con 0x7f7eb00748b0 2024-04-25T00:31:39.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f7eaf59e640 1 -- 172.21.15.29:0/3257332226 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7eb0069200 con 0x7f7eb00748b0 2024-04-25T00:31:39.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f915429f640 1 -- 172.21.15.29:0/4244893273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f914c069200 con 0x7f914c0b2fc0 2024-04-25T00:31:39.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f3725d8e640 1 -- 172.21.15.29:0/2632340693 wait complete. 2024-04-25T00:31:39.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f3725d8e640 1 Processor -- start 2024-04-25T00:31:39.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fdf564e6640 1 -- 172.21.15.29:0/736199854 wait complete. 2024-04-25T00:31:39.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf564e6640 1 Processor -- start 2024-04-25T00:31:39.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf564e6640 1 -- start start 2024-04-25T00:31:39.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf564e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf500cfa10 0x7fdf5015e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf564e6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf500d04a0 0x7fdf500c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf564e6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf500c9590 0x7fdf500c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf564e6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdf501670c0 con 0x7fdf500d04a0 2024-04-25T00:31:39.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf564e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50166f40 con 0x7fdf500cfa10 2024-04-25T00:31:39.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf564e6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdf50167240 con 0x7fdf500c9590 2024-04-25T00:31:39.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf4ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf500cfa10 0x7fdf5015e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf4ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf500cfa10 0x7fdf5015e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:60710/0 (socket says 172.21.15.29:60710) 2024-04-25T00:31:39.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf4ffff640 1 -- 172.21.15.29:0/222435954 learned_addr learned my addr 172.21.15.29:0/222435954 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf54a83640 1 --2- 172.21.15.29:0/222435954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf500c9590 0x7fdf500c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf4ffff640 1 -- 172.21.15.29:0/222435954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf500c9590 msgr2=0x7fdf500c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf4ffff640 1 --2- 172.21.15.29:0/222435954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf500c9590 0x7fdf500c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf4ffff640 1 -- 172.21.15.29:0/222435954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf500d04a0 msgr2=0x7fdf500c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf4ffff640 1 --2- 172.21.15.29:0/222435954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf500d04a0 0x7fdf500c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fb92d7fa640 1 -- 172.21.15.29:0/3291469533 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb92401a070 con 0x7fb9300c4ce0 2024-04-25T00:31:39.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fb92d7fa640 1 --2- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb9140678c0 0x7fb914069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fb92d7fa640 1 -- 172.21.15.29:0/3291469533 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb9240857f0 con 0x7fb9300c4ce0 2024-04-25T00:31:39.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fb92d7fa640 1 --2- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] conn(0x7fb91406fb40 0x7fb914071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fb92d7fa640 1 -- 172.21.15.29:0/3291469533 --> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb914072610 con 0x7fb91406fb40 2024-04-25T00:31:39.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fb92d7fa640 1 -- 172.21.15.29:0/3291469533 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb924085be0 con 0x7fb9300c4ce0 2024-04-25T00:31:39.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fb934d20640 1 --2- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb9140678c0 0x7fb914069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fb935521640 1 --2- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] conn(0x7fb91406fb40 0x7fb914071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fb934d20640 1 --2- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb9140678c0 0x7fb914069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb91c007fd0 tx=0x7fb91c007f20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fb935521640 1 --2- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] conn(0x7fb91406fb40 0x7fb914071f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7f2608ff9640 1 -- 172.21.15.29:0/3965086109 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2600012b50 con 0x7f2604047c20 2024-04-25T00:31:39.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f2608ff9640 1 --2- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f25f0067910 0x7f25f0069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f2608ff9640 1 -- 172.21.15.29:0/3965086109 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2600086760 con 0x7f2604047c20 2024-04-25T00:31:39.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f2608ff9640 1 --2- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] conn(0x7f25f006fb90 0x7f25f0071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f2608ff9640 1 -- 172.21.15.29:0/3965086109 --> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25f0072660 con 0x7f25f006fb90 2024-04-25T00:31:39.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f2608ff9640 1 -- 172.21.15.29:0/3965086109 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2600086c40 con 0x7f2604047c20 2024-04-25T00:31:39.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f260bfff640 1 --2- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f25f0067910 0x7f25f0069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f2610f05640 1 --2- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] conn(0x7f25f006fb90 0x7f25f0071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f260bfff640 1 --2- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f25f0067910 0x7f25f0069dd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f260c06a070 tx=0x7f260c08e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7f2610f05640 1 --2- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] conn(0x7f25f006fb90 0x7f25f0071fb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fa3aa996640 1 -- 172.21.15.29:0/1726896486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39c0349d0 con 0x7fa39c04ff20 2024-04-25T00:31:39.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.488+0000 7fa3aa996640 1 -- 172.21.15.29:0/1726896486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39c034f90 con 0x7fa39c04ff20 2024-04-25T00:31:39.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fa3aa996640 1 -- 172.21.15.29:0/1726896486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa39c0053a0 con 0x7fa39c04ff20 2024-04-25T00:31:39.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fa3a0ff9640 1 -- 172.21.15.29:0/1726896486 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa3900bd020 con 0x7fa39c04ff20 2024-04-25T00:31:39.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fdf4ffff640 1 -- 172.21.15.29:0/222435954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf500b2c30 con 0x7fdf500cfa10 2024-04-25T00:31:39.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fdf4ffff640 1 --2- 172.21.15.29:0/222435954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf500cfa10 0x7fdf5015e500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdf4001a0f0 tx=0x7fdf40019260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fa3a0ff9640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa388067910 0x7fa388069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fa3a0ff9640 1 -- 172.21.15.29:0/1726896486 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa390120800 con 0x7fa39c04ff20 2024-04-25T00:31:39.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fa3a0ff9640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] conn(0x7fa38806fb90 0x7fa388071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fa3a0ff9640 1 -- 172.21.15.29:0/1726896486 --> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa388072660 con 0x7fa38806fb90 2024-04-25T00:31:39.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.492+0000 7fa3a0ff9640 1 -- 172.21.15.29:0/1726896486 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa390120bb0 con 0x7fa39c04ff20 2024-04-25T00:31:39.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fa3a37fe640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa388067910 0x7fa388069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fa3a8f33640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] conn(0x7fa38806fb90 0x7fa388071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fb92d7fa640 1 -- 172.21.15.29:0/3291469533 <== osd.4 v2:172.21.15.157:6801/674816998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb914072610 con 0x7fb91406fb40 2024-04-25T00:31:39.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f3725d8e640 1 -- start start 2024-04-25T00:31:39.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fa3a37fe640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa388067910 0x7fa388069dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4062db0 tx=0x7fa3a406b010 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fdf4cff9640 1 -- 172.21.15.29:0/222435954 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf400b0040 con 0x7fdf500cfa10 2024-04-25T00:31:39.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fdf4cff9640 1 -- 172.21.15.29:0/222435954 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf400bb070 con 0x7fdf500cfa10 2024-04-25T00:31:39.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fdf564e6640 1 -- 172.21.15.29:0/222435954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5015ea40 con 0x7fdf500cfa10 2024-04-25T00:31:39.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fdf564e6640 1 -- 172.21.15.29:0/222435954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5015d460 con 0x7fdf500cfa10 2024-04-25T00:31:39.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fdf4cff9640 1 -- 172.21.15.29:0/222435954 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf400aa9c0 con 0x7fdf500cfa10 2024-04-25T00:31:39.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fdf564e6640 1 -- 172.21.15.29:0/222435954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdf50069270 con 0x7fdf500cfa10 2024-04-25T00:31:39.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f9142ffd640 1 -- 172.21.15.29:0/4244893273 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f914400a850 con 0x7f914c0b2fc0 2024-04-25T00:31:39.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f9142ffd640 1 --2- 172.21.15.29:0/4244893273 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f91300678c0 0x7f9130069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f9142ffd640 1 -- 172.21.15.29:0/4244893273 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9144086660 con 0x7f914c0b2fc0 2024-04-25T00:31:39.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f9142ffd640 1 --2- 172.21.15.29:0/4244893273 >> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] conn(0x7f913006fa60 0x7f9130071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f9142ffd640 1 -- 172.21.15.29:0/4244893273 --> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9144024e50 con 0x7f913006fa60 2024-04-25T00:31:39.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f9142ffd640 1 -- 172.21.15.29:0/4244893273 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9144086a10 con 0x7f914c0b2fc0 2024-04-25T00:31:39.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f915183a640 1 --2- 172.21.15.29:0/4244893273 >> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] conn(0x7f913006fa60 0x7f9130071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f915203b640 1 --2- 172.21.15.29:0/4244893273 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f91300678c0 0x7f9130069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f3725d8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3718033a10 0x7f37181886a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f3725d8e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371803a610 0x7f37180344f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f3725d8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f371804a4c0 0x7f3718034a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f3725d8e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f371804f300 con 0x7f371803a610 2024-04-25T00:31:39.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f3725d8e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f37180e8a90 con 0x7f3718033a10 2024-04-25T00:31:39.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f3725d8e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f371804f480 con 0x7f371804a4c0 2024-04-25T00:31:39.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fa3a8f33640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] conn(0x7fa38806fb90 0x7fa388071fb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f915183a640 1 --2- 172.21.15.29:0/4244893273 >> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] conn(0x7f913006fa60 0x7f9130071e80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f371ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f371804a4c0 0x7f3718034a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f371ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f371804a4c0 0x7f3718034a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:58598/0 (socket says 172.21.15.29:58598) 2024-04-25T00:31:39.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f371ffff640 1 -- 172.21.15.29:0/446406869 learned_addr learned my addr 172.21.15.29:0/446406869 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f371f7fe640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3718033a10 0x7f37181886a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f7ea6ffd640 1 -- 172.21.15.29:0/3257332226 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7ea0004220 con 0x7f7eb00748b0 2024-04-25T00:31:39.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f371effd640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371803a610 0x7f37180344f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f7ea6ffd640 1 --2- 172.21.15.29:0/3257332226 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7e90067910 0x7f7e90069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f371effd640 1 -- 172.21.15.29:0/446406869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f371804a4c0 msgr2=0x7f3718034a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f371effd640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f371804a4c0 0x7f3718034a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f371effd640 1 -- 172.21.15.29:0/446406869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3718033a10 msgr2=0x7f37181886a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f371effd640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3718033a10 0x7f37181886a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f371effd640 1 -- 172.21.15.29:0/446406869 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3718047750 con 0x7f371803a610 2024-04-25T00:31:39.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f7ea6ffd640 1 -- 172.21.15.29:0/3257332226 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7ea0120ab0 con 0x7f7eb00748b0 2024-04-25T00:31:39.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f7ea6ffd640 1 --2- 172.21.15.29:0/3257332226 >> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] conn(0x7f7e9006fb30 0x7f7e90071f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f7ea6ffd640 1 -- 172.21.15.29:0/3257332226 --> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ea0007e50 con 0x7f7e9006fb30 2024-04-25T00:31:39.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f7ea6ffd640 1 -- 172.21.15.29:0/3257332226 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7ea0120e60 con 0x7f7eb00748b0 2024-04-25T00:31:39.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f9142ffd640 1 -- 172.21.15.29:0/4244893273 <== osd.1 v2:172.21.15.29:6800/738833181 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9144024e50 con 0x7f913006fa60 2024-04-25T00:31:39.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f371effd640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371803a610 0x7f37180344f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f370c000f70 tx=0x7f370c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f7ea7fff640 1 --2- 172.21.15.29:0/3257332226 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7e90067910 0x7f7e90069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fdf4cff9640 1 -- 172.21.15.29:0/222435954 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdf400b3070 con 0x7fdf500cfa10 2024-04-25T00:31:39.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fdf4cff9640 1 --2- 172.21.15.29:0/222435954 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdf3c0678f0 0x7fdf3c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fdf4cff9640 1 -- 172.21.15.29:0/222435954 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdf401207d0 con 0x7fdf500cfa10 2024-04-25T00:31:39.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fdf4cff9640 1 --2- 172.21.15.29:0/222435954 >> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] conn(0x7fdf3c06fa90 0x7fdf3c071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fdf4cff9640 1 -- 172.21.15.29:0/222435954 --> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdf400c9ec0 con 0x7fdf3c06fa90 2024-04-25T00:31:39.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7fdf4cff9640 1 -- 172.21.15.29:0/222435954 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdf40120c60 con 0x7fdf500cfa10 2024-04-25T00:31:39.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f370bfff640 1 -- 172.21.15.29:0/446406869 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c017070 con 0x7f371803a610 2024-04-25T00:31:39.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f370bfff640 1 -- 172.21.15.29:0/446406869 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f370c0052d0 con 0x7f371803a610 2024-04-25T00:31:39.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.500+0000 7fdf54a83640 1 --2- 172.21.15.29:0/222435954 >> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] conn(0x7fdf3c06fa90 0x7fdf3c071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.500+0000 7fdf4f7fe640 1 --2- 172.21.15.29:0/222435954 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdf3c0678f0 0x7fdf3c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.500+0000 7fa3a0ff9640 1 -- 172.21.15.29:0/1726896486 <== osd.7 v2:172.21.15.157:6824/739472638 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa388072660 con 0x7fa38806fb90 2024-04-25T00:31:39.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.500+0000 7fdf54a83640 1 --2- 172.21.15.29:0/222435954 >> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] conn(0x7fdf3c06fa90 0x7fdf3c071eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.500+0000 7fdf4f7fe640 1 --2- 172.21.15.29:0/222435954 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdf3c0678f0 0x7fdf3c069db0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf44006b20 tx=0x7fdf44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.500+0000 7f7ea7fff640 1 --2- 172.21.15.29:0/3257332226 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7e90067910 0x7f7e90069dd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e98006fd0 tx=0x7f7e98006b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.500+0000 7fdf4cff9640 1 -- 172.21.15.29:0/222435954 <== osd.3 v2:172.21.15.29:6817/1979009155 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdf400c9ec0 con 0x7fdf3c06fa90 2024-04-25T00:31:39.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.504+0000 7f7eaed9d640 1 --2- 172.21.15.29:0/3257332226 >> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] conn(0x7f7e9006fb30 0x7f7e90071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.504+0000 7f7eaed9d640 1 --2- 172.21.15.29:0/3257332226 >> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] conn(0x7f7e9006fb30 0x7f7e90071f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.504+0000 7f7ea6ffd640 1 -- 172.21.15.29:0/3257332226 <== osd.0 v2:172.21.15.29:6808/2412845288 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ea0007e50 con 0x7f7e9006fb30 2024-04-25T00:31:39.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f915203b640 1 --2- 172.21.15.29:0/4244893273 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f91300678c0 0x7f9130069d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f913c0b2000 tx=0x7f913c0aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.504+0000 7f370bfff640 1 -- 172.21.15.29:0/446406869 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c003840 con 0x7f371803a610 2024-04-25T00:31:39.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.504+0000 7f3725d8e640 1 -- 172.21.15.29:0/446406869 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3718047470 con 0x7f371803a610 2024-04-25T00:31:39.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.504+0000 7f3725d8e640 1 -- 172.21.15.29:0/446406869 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37180499e0 con 0x7f371803a610 2024-04-25T00:31:39.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.504+0000 7f3725d8e640 1 -- 172.21.15.29:0/446406869 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37180053d0 con 0x7f371803a610 2024-04-25T00:31:39.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.496+0000 7f2608ff9640 1 -- 172.21.15.29:0/3965086109 <== osd.5 v2:172.21.15.157:6800/486133226 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25f0072660 con 0x7f25f006fb90 2024-04-25T00:31:39.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.504+0000 7f370bfff640 1 -- 172.21.15.29:0/446406869 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f370c01a070 con 0x7f371803a610 2024-04-25T00:31:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.508+0000 7f370bfff640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37040678c0 0x7f3704069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.508+0000 7f370bfff640 1 -- 172.21.15.29:0/446406869 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f370c085650 con 0x7f371803a610 2024-04-25T00:31:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.508+0000 7f370bfff640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] conn(0x7f370406fa60 0x7f3704071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.508+0000 7f370bfff640 1 -- 172.21.15.29:0/446406869 --> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f370c0052d0 con 0x7f370406fa60 2024-04-25T00:31:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.508+0000 7f370bfff640 1 -- 172.21.15.29:0/446406869 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f370c085a00 con 0x7f371803a610 2024-04-25T00:31:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.508+0000 7f371f7fe640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37040678c0 0x7f3704069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.508+0000 7f371f7fe640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37040678c0 0x7f3704069d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f37140ad000 tx=0x7f37140a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.512+0000 7f371ffff640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] conn(0x7f370406fa60 0x7f3704071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.512+0000 7f371ffff640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] conn(0x7f370406fa60 0x7f3704071e80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.512+0000 7f370bfff640 1 -- 172.21.15.29:0/446406869 <== osd.2 v2:172.21.15.29:6816/802770835 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f370c0052d0 con 0x7f370406fa60 2024-04-25T00:31:39.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.516+0000 7fb936f84640 1 -- 172.21.15.29:0/3291469533 --> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb930069270 con 0x7fb91406fb40 2024-04-25T00:31:39.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fb92d7fa640 1 -- 172.21.15.29:0/3291469533 <== osd.4 v2:172.21.15.157:6801/674816998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb930069270 con 0x7fb91406fb40 2024-04-25T00:31:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fb936f84640 1 -- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] conn(0x7fb91406fb40 msgr2=0x7fb914071f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fb936f84640 1 --2- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] conn(0x7fb91406fb40 0x7fb914071f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fb936f84640 1 -- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb9140678c0 msgr2=0x7fb914069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fb936f84640 1 --2- 172.21.15.29:0/3291469533 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb9140678c0 0x7fb914069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb91c007fd0 tx=0x7fb91c007f20 comp rx=0 tx=0).stop 2024-04-25T00:31:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fb936f84640 1 -- 172.21.15.29:0/3291469533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9300c4ce0 msgr2=0x7fb930158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fb936f84640 1 --2- 172.21.15.29:0/3291469533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9300c4ce0 0x7fb930158590 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb92400c430 tx=0x7fb92400c8f0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fb935521640 1 -- 172.21.15.29:0/3291469533 reap_dead start 2024-04-25T00:31:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fb936f84640 1 -- 172.21.15.29:0/3291469533 shutdown_connections 2024-04-25T00:31:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fb936f84640 1 -- 172.21.15.29:0/3291469533 >> 172.21.15.29:0/3291469533 conn(0x7fb93009aa30 msgr2=0x7fb9300cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fb936f84640 1 -- 172.21.15.29:0/3291469533 shutdown_connections 2024-04-25T00:31:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fb936f84640 1 -- 172.21.15.29:0/3291469533 wait complete. 2024-04-25T00:31:39.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbeee04f640 1 Processor -- start 2024-04-25T00:31:39.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbeee04f640 1 -- start start 2024-04-25T00:31:39.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbeee04f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbee816ada0 0x7fbee80b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbeee04f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee80b3280 0x7fbee806d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbeee04f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee8071be0 con 0x7fbee816b180 2024-04-25T00:31:39.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbeee04f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbee8071ec0 con 0x7fbee816ada0 2024-04-25T00:31:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbeee04f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbee806cd40 con 0x7fbee80b3280 2024-04-25T00:31:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbee77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbee816ada0 0x7fbee80b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbee77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbee816ada0 0x7fbee80b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49686/0 (socket says 172.21.15.29:49686) 2024-04-25T00:31:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbee77fe640 1 -- 172.21.15.29:0/11599200 learned_addr learned my addr 172.21.15.29:0/11599200 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbee7fff640 1 --2- 172.21.15.29:0/11599200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee80b3280 0x7fbee806d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbee77fe640 1 -- 172.21.15.29:0/11599200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee80b3280 msgr2=0x7fbee806d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbee77fe640 1 --2- 172.21.15.29:0/11599200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee80b3280 0x7fbee806d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbee77fe640 1 -- 172.21.15.29:0/11599200 >> v1:172.21.15.157:6789/0 conn(0x7fbee816b180 legacy=0x7fbee80b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fbee77fe640 1 -- 172.21.15.29:0/11599200 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee806d020 con 0x7fbee816ada0 2024-04-25T00:31:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fdf564e6640 1 -- 172.21.15.29:0/222435954 --> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdf500d2980 con 0x7fdf3c06fa90 2024-04-25T00:31:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.520+0000 7fdf4cff9640 1 -- 172.21.15.29:0/222435954 <== osd.3 v2:172.21.15.29:6817/1979009155 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdf500d2980 con 0x7fdf3c06fa90 2024-04-25T00:31:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fbee77fe640 1 --2- 172.21.15.29:0/11599200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee816ada0 0x7fbee80b4c20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbee0008ec0 tx=0x7fbee001f140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1cd6f7c2581eb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fbee67fc640 1 -- 172.21.15.29:0/11599200 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee00b4070 con 0x7fbee816ada0 2024-04-25T00:31:39.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fbee67fc640 1 -- 172.21.15.29:0/11599200 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee00a7c40 con 0x7fbee816ada0 2024-04-25T00:31:39.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fbee67fc640 1 -- 172.21.15.29:0/11599200 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee00af440 con 0x7fbee816ada0 2024-04-25T00:31:39.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fbeee04f640 1 -- 172.21.15.29:0/11599200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee816ada0 msgr2=0x7fbee80b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fbeee04f640 1 --2- 172.21.15.29:0/11599200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee816ada0 0x7fbee80b4c20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbee0008ec0 tx=0x7fbee001f140 comp rx=0 tx=0).stop 2024-04-25T00:31:39.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fbeee04f640 1 -- 172.21.15.29:0/11599200 shutdown_connections 2024-04-25T00:31:39.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fbeee04f640 1 --2- 172.21.15.29:0/11599200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee80b3280 0x7fbee806d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fbeee04f640 1 --2- 172.21.15.29:0/11599200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee816ada0 0x7fbee80b4c20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fbeee04f640 1 -- 172.21.15.29:0/11599200 >> 172.21.15.29:0/11599200 conn(0x7fbee80b1870 msgr2=0x7fbee80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fdf564e6640 1 -- 172.21.15.29:0/222435954 >> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] conn(0x7fdf3c06fa90 msgr2=0x7fdf3c071eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fdf564e6640 1 --2- 172.21.15.29:0/222435954 >> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] conn(0x7fdf3c06fa90 0x7fdf3c071eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fbeee04f640 1 -- 172.21.15.29:0/11599200 shutdown_connections 2024-04-25T00:31:39.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fdf564e6640 1 -- 172.21.15.29:0/222435954 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdf3c0678f0 msgr2=0x7fdf3c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fdf564e6640 1 --2- 172.21.15.29:0/222435954 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdf3c0678f0 0x7fdf3c069db0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf44006b20 tx=0x7fdf44008040 comp rx=0 tx=0).stop 2024-04-25T00:31:39.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fdf564e6640 1 -- 172.21.15.29:0/222435954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf500cfa10 msgr2=0x7fdf5015e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fdf564e6640 1 --2- 172.21.15.29:0/222435954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf500cfa10 0x7fdf5015e500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdf4001a0f0 tx=0x7fdf40019260 comp rx=0 tx=0).stop 2024-04-25T00:31:39.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fdf54a83640 1 -- 172.21.15.29:0/222435954 reap_dead start 2024-04-25T00:31:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fdf564e6640 1 -- 172.21.15.29:0/222435954 shutdown_connections 2024-04-25T00:31:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fdf564e6640 1 -- 172.21.15.29:0/222435954 >> 172.21.15.29:0/222435954 conn(0x7fdf500d1d60 msgr2=0x7fdf500aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fdf564e6640 1 -- 172.21.15.29:0/222435954 shutdown_connections 2024-04-25T00:31:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.524+0000 7fdf564e6640 1 -- 172.21.15.29:0/222435954 wait complete. 2024-04-25T00:31:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.528+0000 7fbeee04f640 1 -- 172.21.15.29:0/11599200 wait complete. 2024-04-25T00:31:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.528+0000 7fbeee04f640 1 Processor -- start 2024-04-25T00:31:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.528+0000 7fa3aa996640 1 -- 172.21.15.29:0/1726896486 --> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa39c0053a0 con 0x7fa38806fb90 2024-04-25T00:31:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.528+0000 7f915429f640 1 -- 172.21.15.29:0/4244893273 --> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f914c15d9a0 con 0x7f913006fa60 2024-04-25T00:31:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.528+0000 7fa3a0ff9640 1 -- 172.21.15.29:0/1726896486 <== osd.7 v2:172.21.15.157:6824/739472638 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa39c0053a0 con 0x7fa38806fb90 2024-04-25T00:31:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.528+0000 7fbeee04f640 1 -- start start 2024-04-25T00:31:39.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.528+0000 7fbeee04f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee80b3280 0x7fbee816e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.528+0000 7f9142ffd640 1 -- 172.21.15.29:0/4244893273 <== osd.1 v2:172.21.15.29:6800/738833181 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f914c15d9a0 con 0x7f913006fa60 2024-04-25T00:31:39.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.528+0000 7fbeee04f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee816ada0 0x7fbee820bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.528+0000 7fbeee04f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee816b180 0x7fbee816b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.528+0000 7fa3aa996640 1 -- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] conn(0x7fa38806fb90 msgr2=0x7fa388071fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.528+0000 7fa3aa996640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] conn(0x7fa38806fb90 0x7fa388071fb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fa3aa996640 1 -- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa388067910 msgr2=0x7fa388069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fa3aa996640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa388067910 0x7fa388069dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4062db0 tx=0x7fa3a406b010 comp rx=0 tx=0).stop 2024-04-25T00:31:39.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fa3aa996640 1 -- 172.21.15.29:0/1726896486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa39c04ff20 msgr2=0x7fa39c0cbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fa3aa996640 1 --2- 172.21.15.29:0/1726896486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa39c04ff20 0x7fa39c0cbd10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa39001a110 tx=0x7fa39001ad20 comp rx=0 tx=0).stop 2024-04-25T00:31:39.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fa3a8f33640 1 -- 172.21.15.29:0/1726896486 reap_dead start 2024-04-25T00:31:39.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.528+0000 7fbee77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee80b3280 0x7fbee816e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbee77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee80b3280 0x7fbee816e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:60734/0 (socket says 172.21.15.29:60734) 2024-04-25T00:31:39.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbee6ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee816ada0 0x7fbee820bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbee6ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee816ada0 0x7fbee820bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:58614/0 (socket says 172.21.15.29:58614) 2024-04-25T00:31:39.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbee7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee816b180 0x7fbee816b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbee7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee816b180 0x7fbee816b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49696/0 (socket says 172.21.15.29:49696) 2024-04-25T00:31:39.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fa3aa996640 1 -- 172.21.15.29:0/1726896486 shutdown_connections 2024-04-25T00:31:39.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fa3aa996640 1 -- 172.21.15.29:0/1726896486 >> 172.21.15.29:0/1726896486 conn(0x7fa39c009fc0 msgr2=0x7fa39c04ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbee77fe640 1 -- 172.21.15.29:0/1068586378 learned_addr learned my addr 172.21.15.29:0/1068586378 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fa3aa996640 1 -- 172.21.15.29:0/1726896486 shutdown_connections 2024-04-25T00:31:39.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbee8177bd0 con 0x7fbee816b180 2024-04-25T00:31:39.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbee806e780 con 0x7fbee80b3280 2024-04-25T00:31:39.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbee8177d50 con 0x7fbee816ada0 2024-04-25T00:31:39.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fa3aa996640 1 -- 172.21.15.29:0/1726896486 wait complete. 2024-04-25T00:31:39.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbee7fff640 1 -- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee816ada0 msgr2=0x7fbee820bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbee7fff640 1 --2- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee816ada0 0x7fbee820bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbee7fff640 1 -- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee80b3280 msgr2=0x7fbee816e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbee7fff640 1 --2- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee80b3280 0x7fbee816e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbee7fff640 1 -- 172.21.15.29:0/1068586378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee815bf70 con 0x7fbee816b180 2024-04-25T00:31:39.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbee7fff640 1 --2- 172.21.15.29:0/1068586378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee816b180 0x7fbee816b590 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbee806ec10 tx=0x7fbed800d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7f7eaf59e640 1 -- 172.21.15.29:0/3257332226 --> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7eb00cd390 con 0x7f7e9006fb30 2024-04-25T00:31:39.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7f7ea6ffd640 1 -- 172.21.15.29:0/3257332226 <== osd.0 v2:172.21.15.29:6808/2412845288 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7eb00cd390 con 0x7f7e9006fb30 2024-04-25T00:31:39.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7f2612968640 1 -- 172.21.15.29:0/3965086109 --> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f26040e7660 con 0x7f25f006fb90 2024-04-25T00:31:39.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbecffff640 1 -- 172.21.15.29:0/1068586378 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8004020 con 0x7fbee816b180 2024-04-25T00:31:39.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbecffff640 1 -- 172.21.15.29:0/1068586378 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed80041e0 con 0x7fbee816b180 2024-04-25T00:31:39.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee815c250 con 0x7fbee816b180 2024-04-25T00:31:39.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee815c810 con 0x7fbee816b180 2024-04-25T00:31:39.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7fbecffff640 1 -- 172.21.15.29:0/1068586378 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed80043d0 con 0x7fbee816b180 2024-04-25T00:31:39.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.532+0000 7f2608ff9640 1 -- 172.21.15.29:0/3965086109 <== osd.5 v2:172.21.15.157:6800/486133226 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f25f80a86b0 con 0x7f25f006fb90 2024-04-25T00:31:39.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f7eaf59e640 1 -- 172.21.15.29:0/3257332226 >> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] conn(0x7f7e9006fb30 msgr2=0x7f7e90071f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f7eaf59e640 1 --2- 172.21.15.29:0/3257332226 >> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] conn(0x7f7e9006fb30 0x7f7e90071f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f7eaf59e640 1 -- 172.21.15.29:0/3257332226 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7e90067910 msgr2=0x7f7e90069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f7eaf59e640 1 --2- 172.21.15.29:0/3257332226 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7e90067910 0x7f7e90069dd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e98006fd0 tx=0x7f7e98006b40 comp rx=0 tx=0).stop 2024-04-25T00:31:39.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbec8001040 con 0x7fbee816b180 2024-04-25T00:31:39.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f7eaf59e640 1 -- 172.21.15.29:0/3257332226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb00748b0 msgr2=0x7f7eb00cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f7eaf59e640 1 --2- 172.21.15.29:0/3257332226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb00748b0 0x7f7eb00cfdc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0018fb0 tx=0x7f7ea001abe0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f7eaed9d640 1 -- 172.21.15.29:0/3257332226 reap_dead start 2024-04-25T00:31:39.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f7eaf59e640 1 -- 172.21.15.29:0/3257332226 shutdown_connections 2024-04-25T00:31:39.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f7eaf59e640 1 -- 172.21.15.29:0/3257332226 >> 172.21.15.29:0/3257332226 conn(0x7f7eb006b210 msgr2=0x7f7eb01604a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f7eaf59e640 1 -- 172.21.15.29:0/3257332226 shutdown_connections 2024-04-25T00:31:39.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f7eaf59e640 1 -- 172.21.15.29:0/3257332226 wait complete. 2024-04-25T00:31:39.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f2612968640 1 -- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] conn(0x7f25f006fb90 msgr2=0x7f25f0071fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f2612968640 1 --2- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] conn(0x7f25f006fb90 0x7f25f0071fb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7fbecffff640 1 -- 172.21.15.29:0/1068586378 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbed801b020 con 0x7fbee816b180 2024-04-25T00:31:39.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7fbecffff640 1 --2- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbed4067910 0x7fbed4069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7fbecffff640 1 -- 172.21.15.29:0/1068586378 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbed80860f0 con 0x7fbee816b180 2024-04-25T00:31:39.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f2612968640 1 -- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f25f0067910 msgr2=0x7f25f0069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f2612968640 1 --2- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f25f0067910 0x7f25f0069dd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f260c06a070 tx=0x7f260c08e040 comp rx=0 tx=0).stop 2024-04-25T00:31:39.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f2612968640 1 -- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2604047c20 msgr2=0x7f2604045aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f2612968640 1 --2- 172.21.15.29:0/3965086109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2604047c20 0x7f2604045aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f260000d5f0 tx=0x7f260000dab0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f2610f05640 1 -- 172.21.15.29:0/3965086109 reap_dead start 2024-04-25T00:31:39.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f2612968640 1 -- 172.21.15.29:0/3965086109 shutdown_connections 2024-04-25T00:31:39.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7f2612968640 1 -- 172.21.15.29:0/3965086109 >> 172.21.15.29:0/3965086109 conn(0x7f2604009fc0 msgr2=0x7f26040eaf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7fbecffff640 1 --2- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] conn(0x7fbed406fb90 0x7fbed4071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7fbecffff640 1 -- 172.21.15.29:0/1068586378 --> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbed8001d40 con 0x7fbed406fb90 2024-04-25T00:31:39.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7fbecffff640 1 -- 172.21.15.29:0/1068586378 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbed80865d0 con 0x7fbee816b180 2024-04-25T00:31:39.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7fbee77fe640 1 --2- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbed4067910 0x7fbed4069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7fbee6ffd640 1 --2- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] conn(0x7fbed406fb90 0x7fbed4071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7fbee77fe640 1 --2- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbed4067910 0x7fbed4069dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbee0000f90 tx=0x7fbee00a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.536+0000 7fbee6ffd640 1 --2- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] conn(0x7fbed406fb90 0x7fbed4071fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.540+0000 7f2612968640 1 -- 172.21.15.29:0/3965086109 shutdown_connections 2024-04-25T00:31:39.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.540+0000 7f2612968640 1 -- 172.21.15.29:0/3965086109 wait complete. 2024-04-25T00:31:39.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.540+0000 7f9140ff9640 1 -- 172.21.15.29:0/4244893273 >> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] conn(0x7f913006fa60 msgr2=0x7f9130071e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.540+0000 7f9140ff9640 1 --2- 172.21.15.29:0/4244893273 >> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] conn(0x7f913006fa60 0x7f9130071e80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.540+0000 7f9140ff9640 1 -- 172.21.15.29:0/4244893273 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f91300678c0 msgr2=0x7f9130069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.540+0000 7f9140ff9640 1 --2- 172.21.15.29:0/4244893273 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f91300678c0 0x7f9130069d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f913c0b2000 tx=0x7f913c0aa070 comp rx=0 tx=0).stop 2024-04-25T00:31:39.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.540+0000 7f9140ff9640 1 -- 172.21.15.29:0/4244893273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f914c0b2fc0 msgr2=0x7f914c15f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.540+0000 7f9140ff9640 1 --2- 172.21.15.29:0/4244893273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f914c0b2fc0 0x7f914c15f170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f914400c760 tx=0x7f914400cc20 comp rx=0 tx=0).stop 2024-04-25T00:31:39.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.540+0000 7f915283c640 1 -- 172.21.15.29:0/4244893273 reap_dead start 2024-04-25T00:31:39.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.540+0000 7f9140ff9640 1 -- 172.21.15.29:0/4244893273 shutdown_connections 2024-04-25T00:31:39.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.540+0000 7f9140ff9640 1 -- 172.21.15.29:0/4244893273 >> 172.21.15.29:0/4244893273 conn(0x7f914c074880 msgr2=0x7f914c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.540+0000 7f9140ff9640 1 -- 172.21.15.29:0/4244893273 shutdown_connections 2024-04-25T00:31:39.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.540+0000 7f9140ff9640 1 -- 172.21.15.29:0/4244893273 wait complete. 2024-04-25T00:31:39.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.544+0000 7f3725d8e640 1 -- 172.21.15.29:0/446406869 --> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3718049bb0 con 0x7f370406fa60 2024-04-25T00:31:39.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.544+0000 7fbecffff640 1 -- 172.21.15.29:0/1068586378 <== osd.6 v2:172.21.15.157:6816/2845420196 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbed8001d40 con 0x7fbed406fb90 2024-04-25T00:31:39.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.544+0000 7f370bfff640 1 -- 172.21.15.29:0/446406869 <== osd.2 v2:172.21.15.29:6816/802770835 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3718049bb0 con 0x7f370406fa60 2024-04-25T00:31:39.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.544+0000 7f3725d8e640 1 -- 172.21.15.29:0/446406869 >> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] conn(0x7f370406fa60 msgr2=0x7f3704071e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.544+0000 7f3725d8e640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] conn(0x7f370406fa60 0x7f3704071e80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.544+0000 7f3725d8e640 1 -- 172.21.15.29:0/446406869 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37040678c0 msgr2=0x7f3704069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.544+0000 7f3725d8e640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37040678c0 0x7f3704069d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f37140ad000 tx=0x7f37140a6000 comp rx=0 tx=0).stop 2024-04-25T00:31:39.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.544+0000 7f3725d8e640 1 -- 172.21.15.29:0/446406869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371803a610 msgr2=0x7f37180344f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.544+0000 7f3725d8e640 1 --2- 172.21.15.29:0/446406869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371803a610 0x7f37180344f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f370c000f70 tx=0x7f370c00c7d0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.544+0000 7f371ffff640 1 -- 172.21.15.29:0/446406869 reap_dead start 2024-04-25T00:31:39.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.544+0000 7f3725d8e640 1 -- 172.21.15.29:0/446406869 shutdown_connections 2024-04-25T00:31:39.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.548+0000 7f3725d8e640 1 -- 172.21.15.29:0/446406869 >> 172.21.15.29:0/446406869 conn(0x7f3718054b00 msgr2=0x7f3718006fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.548+0000 7f3725d8e640 1 -- 172.21.15.29:0/446406869 shutdown_connections 2024-04-25T00:31:39.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.548+0000 7f3725d8e640 1 -- 172.21.15.29:0/446406869 wait complete. 2024-04-25T00:31:39.573 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-04-25T00:31:39.573 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T00:31:39.574 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-04-25T00:31:39.574 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T00:31:39.575 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-04-25T00:31:39.575 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T00:31:39.576 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-04-25T00:31:39.577 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T00:31:39.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.572+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 --> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbec8002de0 con 0x7fbed406fb90 2024-04-25T00:31:39.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.572+0000 7fbecffff640 1 -- 172.21.15.29:0/1068586378 <== osd.6 v2:172.21.15.157:6816/2845420196 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbec8002de0 con 0x7fbed406fb90 2024-04-25T00:31:39.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.572+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] conn(0x7fbed406fb90 msgr2=0x7fbed4071fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.572+0000 7fbeee04f640 1 --2- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] conn(0x7fbed406fb90 0x7fbed4071fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.572+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbed4067910 msgr2=0x7fbed4069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.572+0000 7fbeee04f640 1 --2- 172.21.15.29:0/1068586378 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbed4067910 0x7fbed4069dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbee0000f90 tx=0x7fbee00a9070 comp rx=0 tx=0).stop 2024-04-25T00:31:39.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.572+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee816b180 msgr2=0x7fbee816b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.572+0000 7fbeee04f640 1 --2- 172.21.15.29:0/1068586378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee816b180 0x7fbee816b590 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbee806ec10 tx=0x7fbed800d910 comp rx=0 tx=0).stop 2024-04-25T00:31:39.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.572+0000 7fbee7fff640 1 -- 172.21.15.29:0/1068586378 reap_dead start 2024-04-25T00:31:39.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.572+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 shutdown_connections 2024-04-25T00:31:39.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.572+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 >> 172.21.15.29:0/1068586378 conn(0x7fbee80b1870 msgr2=0x7fbee8070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.572+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 shutdown_connections 2024-04-25T00:31:39.579 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-04-25T00:31:39.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.572+0000 7fbeee04f640 1 -- 172.21.15.29:0/1068586378 wait complete. 2024-04-25T00:31:39.580 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T00:31:39.582 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-04-25T00:31:39.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T00:31:39.583 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-04-25T00:31:39.583 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T00:31:39.589 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-04-25T00:31:39.590 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T00:31:39.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.732+0000 7effa6d54640 1 Processor -- start 2024-04-25T00:31:39.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.732+0000 7effa6d54640 1 -- start start 2024-04-25T00:31:39.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f821c8a6640 1 Processor -- start 2024-04-25T00:31:39.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa6d54640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7effa0162ef0 0x7effa0070e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa6d54640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effa00b3180 0x7effa006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa6d54640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa00713f0 con 0x7effa01632d0 2024-04-25T00:31:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa6d54640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7effa00716d0 con 0x7effa0162ef0 2024-04-25T00:31:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa6d54640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7effa00719b0 con 0x7effa00b3180 2024-04-25T00:31:39.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f821c8a6640 1 -- start start 2024-04-25T00:31:39.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa5551640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7effa01632d0 0x7effa00b2a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:43980/0 (socket says 172.21.15.29:43980) 2024-04-25T00:31:39.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa5551640 1 -- 172.21.15.29:0/10141058 learned_addr learned my addr 172.21.15.29:0/10141058 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:39.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa6553640 1 --2- 172.21.15.29:0/10141058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effa00b3180 0x7effa006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa5d52640 1 --2- 172.21.15.29:0/10141058 >> v2:172.21.15.157:3300/0 conn(0x7effa0162ef0 0x7effa0070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f821c8a6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821815bcf0 0x7f821806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa4d50640 1 -- 172.21.15.29:0/10141058 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1532681 0 0) 0x7effa00713f0 con 0x7effa01632d0 2024-04-25T00:31:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa4d50640 1 -- 172.21.15.29:0/10141058 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff8c003530 con 0x7effa01632d0 2024-04-25T00:31:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f821c8a6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f82180d1fd0 0x7f82180af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f821c8a6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f821815c0d0 con 0x7f8218070260 2024-04-25T00:31:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f821c8a6640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f82180d0520 con 0x7f82180d1fd0 2024-04-25T00:31:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f821c8a6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f82180aee50 con 0x7f821815bcf0 2024-04-25T00:31:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa6553640 1 -- 172.21.15.29:0/10141058 >> v2:172.21.15.157:3300/0 conn(0x7effa0162ef0 msgr2=0x7effa0070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa6553640 1 --2- 172.21.15.29:0/10141058 >> v2:172.21.15.157:3300/0 conn(0x7effa0162ef0 0x7effa0070e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa6553640 1 -- 172.21.15.29:0/10141058 >> v1:172.21.15.157:6789/0 conn(0x7effa01632d0 legacy=0x7effa00b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa6553640 1 -- 172.21.15.29:0/10141058 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa0071c90 con 0x7effa00b3180 2024-04-25T00:31:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f8216ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8218070260 0x7f82180d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:43990/0 (socket says 172.21.15.29:43990) 2024-04-25T00:31:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f8216ffd640 1 -- 172.21.15.29:0/2687427793 learned_addr learned my addr 172.21.15.29:0/2687427793 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7effa5d52640 1 --2- 172.21.15.29:0/10141058 >> v2:172.21.15.157:3300/0 conn(0x7effa0162ef0 0x7effa0070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f82177fe640 1 --2- 172.21.15.29:0/2687427793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821815bcf0 0x7f821806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f82167fc640 1 -- 172.21.15.29:0/2687427793 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 552170047 0 0) 0x7f821815c0d0 con 0x7f8218070260 2024-04-25T00:31:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f82167fc640 1 -- 172.21.15.29:0/2687427793 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8200003530 con 0x7f8218070260 2024-04-25T00:31:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f8217fff640 1 --2- 172.21.15.29:0/2687427793 >> v2:172.21.15.29:3300/0 conn(0x7f82180d1fd0 0x7f82180af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f82167fc640 1 -- 172.21.15.29:0/2687427793 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3259780635 0 0) 0x7f8200003530 con 0x7f8218070260 2024-04-25T00:31:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f82177fe640 1 -- 172.21.15.29:0/2687427793 >> v2:172.21.15.29:3300/0 conn(0x7f82180d1fd0 msgr2=0x7f82180af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f82177fe640 1 --2- 172.21.15.29:0/2687427793 >> v2:172.21.15.29:3300/0 conn(0x7f82180d1fd0 0x7f82180af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f82177fe640 1 -- 172.21.15.29:0/2687427793 >> v1:172.21.15.157:6789/0 conn(0x7f8218070260 legacy=0x7f82180d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.736+0000 7f82177fe640 1 -- 172.21.15.29:0/2687427793 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82180708f0 con 0x7f821815bcf0 2024-04-25T00:31:39.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7fcb1a93d640 1 Processor -- start 2024-04-25T00:31:39.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7fcb1a93d640 1 -- start start 2024-04-25T00:31:39.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7effa6553640 1 --2- 172.21.15.29:0/10141058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effa00b3180 0x7effa006cc60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7eff900097c0 tx=0x7eff9001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25bde8fd971fbbca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7effa4d50640 1 -- 172.21.15.29:0/10141058 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff900ab430 con 0x7effa00b3180 2024-04-25T00:31:39.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7effa4d50640 1 -- 172.21.15.29:0/10141058 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff900b0070 con 0x7effa00b3180 2024-04-25T00:31:39.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7effa4d50640 1 -- 172.21.15.29:0/10141058 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff900ab7d0 con 0x7effa00b3180 2024-04-25T00:31:39.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7effa6d54640 1 -- 172.21.15.29:0/10141058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effa00b3180 msgr2=0x7effa006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7effa6d54640 1 --2- 172.21.15.29:0/10141058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effa00b3180 0x7effa006cc60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7eff900097c0 tx=0x7eff9001c490 comp rx=0 tx=0).stop 2024-04-25T00:31:39.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7f82177fe640 1 --2- 172.21.15.29:0/2687427793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821815bcf0 0x7f821806fd20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f82080097c0 tx=0x7f820801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=681a580592a9f51e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7f82167fc640 1 -- 172.21.15.29:0/2687427793 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82080b0070 con 0x7f821815bcf0 2024-04-25T00:31:39.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7f82167fc640 1 -- 172.21.15.29:0/2687427793 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82080ab400 con 0x7f821815bcf0 2024-04-25T00:31:39.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7f82167fc640 1 -- 172.21.15.29:0/2687427793 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82080b0070 con 0x7f821815bcf0 2024-04-25T00:31:39.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7fcb1a93d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcb141579b0 0x7fcb14072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7effa6d54640 1 -- 172.21.15.29:0/10141058 shutdown_connections 2024-04-25T00:31:39.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7effa6d54640 1 --2- 172.21.15.29:0/10141058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effa00b3180 0x7effa006cc60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7effa6d54640 1 --2- 172.21.15.29:0/10141058 >> v2:172.21.15.157:3300/0 conn(0x7effa0162ef0 0x7effa0070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7effa6d54640 1 -- 172.21.15.29:0/10141058 >> 172.21.15.29:0/10141058 conn(0x7effa00b1770 msgr2=0x7effa00b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7fcb13fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcb141579b0 0x7fcb14072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7f821c8a6640 1 -- 172.21.15.29:0/2687427793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821815bcf0 msgr2=0x7f821806fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.740+0000 7fcb13fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcb141579b0 0x7fcb14072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49724/0 (socket says 172.21.15.29:49724) 2024-04-25T00:31:39.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7effa6d54640 1 -- 172.21.15.29:0/10141058 shutdown_connections 2024-04-25T00:31:39.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7effa6d54640 1 -- 172.21.15.29:0/10141058 wait complete. 2024-04-25T00:31:39.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb137fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcb140acd60 0x7fcb140b2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50018/0 (socket says 172.21.15.29:50018) 2024-04-25T00:31:39.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 --2- 172.21.15.29:0/2687427793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821815bcf0 0x7f821806fd20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f82080097c0 tx=0x7f820801c3c0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 -- 172.21.15.29:0/2687427793 shutdown_connections 2024-04-25T00:31:39.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 --2- 172.21.15.29:0/2687427793 >> v2:172.21.15.29:3300/0 conn(0x7f82180d1fd0 0x7f82180af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 --2- 172.21.15.29:0/2687427793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821815bcf0 0x7f821806fd20 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f82080097c0 tx=0x7f820801c3c0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 -- 172.21.15.29:0/2687427793 >> 172.21.15.29:0/2687427793 conn(0x7f821809aa30 msgr2=0x7f821809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb13fff640 1 -- 172.21.15.29:0/738219261 learned_addr learned my addr 172.21.15.29:0/738219261 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 -- 172.21.15.29:0/2687427793 shutdown_connections 2024-04-25T00:31:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 -- 172.21.15.29:0/2687427793 wait complete. 2024-04-25T00:31:39.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb1a93d640 1 -- 172.21.15.29:0/738219261 --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb140ad6d0 con 0x7fcb140acd60 2024-04-25T00:31:39.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb1a93d640 1 -- 172.21.15.29:0/738219261 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb140ad990 con 0x7fcb14072ea0 2024-04-25T00:31:39.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb1a93d640 1 -- 172.21.15.29:0/738219261 --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fcb140adf50 con 0x7fcb141579b0 2024-04-25T00:31:39.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595171534 0 0) 0x7fcb140ad6d0 con 0x7fcb140acd60 2024-04-25T00:31:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcafc003530 con 0x7fcb140acd60 2024-04-25T00:31:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4094715036 0 0) 0x7fcb140ad990 con 0x7fcb14072ea0 2024-04-25T00:31:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb140ad6d0 con 0x7fcb14072ea0 2024-04-25T00:31:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2421610500 0 0) 0x7fcafc003530 con 0x7fcb140acd60 2024-04-25T00:31:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 >> v2:172.21.15.29:3300/0 conn(0x7fcb141579b0 msgr2=0x7fcb14072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 --2- 172.21.15.29:0/738219261 >> v2:172.21.15.29:3300/0 conn(0x7fcb141579b0 0x7fcb14072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 >> v1:172.21.15.157:6789/0 conn(0x7fcb14072ea0 legacy=0x7fcb1415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb140adc70 con 0x7fcb140acd60 2024-04-25T00:31:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fcb04001680 con 0x7fcb140acd60 2024-04-25T00:31:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 >> v1:172.21.15.29:6789/0 conn(0x7fcb140acd60 legacy=0x7fcb140b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 --2- 172.21.15.29:0/738219261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcafc003ba0 0x7fcafc003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 --2- 172.21.15.29:0/738219261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcafc004810 0x7fcafc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 --2- 172.21.15.29:0/738219261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcafc005220 0x7fcafc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcb140adf50 con 0x7fcafc003ba0 2024-04-25T00:31:39.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc00c150 con 0x7fcafc004810 2024-04-25T00:31:39.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcafc00c350 con 0x7fcafc005220 2024-04-25T00:31:39.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb18eda640 1 --2- 172.21.15.29:0/738219261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcafc005220 0x7fcafc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb13fff640 1 --2- 172.21.15.29:0/738219261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcafc003ba0 0x7fcafc003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb137fe640 1 --2- 172.21.15.29:0/738219261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcafc004810 0x7fcafc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7effa6d54640 1 Processor -- start 2024-04-25T00:31:39.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 Processor -- start 2024-04-25T00:31:39.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb18eda640 1 -- 172.21.15.29:0/738219261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcafc004810 msgr2=0x7fcafc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb18eda640 1 --2- 172.21.15.29:0/738219261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcafc004810 0x7fcafc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb18eda640 1 -- 172.21.15.29:0/738219261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcafc003ba0 msgr2=0x7fcafc003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb18eda640 1 --2- 172.21.15.29:0/738219261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcafc003ba0 0x7fcafc003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb18eda640 1 -- 172.21.15.29:0/738219261 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc003530 con 0x7fcafc005220 2024-04-25T00:31:39.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7fcb13fff640 1 --2- 172.21.15.29:0/738219261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcafc003ba0 0x7fcafc003fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:39.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7effa6d54640 1 -- start start 2024-04-25T00:31:39.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7effa6d54640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effa00b3180 0x7effa014df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 -- start start 2024-04-25T00:31:39.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8218070260 0x7f82180c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82180d1fd0 0x7f8218150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82180b6cc0 0x7f8218149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f82180b01b0 con 0x7f82180b6cc0 2024-04-25T00:31:39.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f82180b0030 con 0x7f8218070260 2024-04-25T00:31:39.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f821c8a6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f82180b0330 con 0x7f82180d1fd0 2024-04-25T00:31:39.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7effa6d54640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effa0162ef0 0x7effa01599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f8217fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82180b6cc0 0x7f8218149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f8217fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82180b6cc0 0x7f8218149ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49742/0 (socket says 172.21.15.29:49742) 2024-04-25T00:31:39.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f8217fff640 1 -- 172.21.15.29:0/2052475724 learned_addr learned my addr 172.21.15.29:0/2052475724 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7effa5d52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effa00b3180 0x7effa014df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7f8216ffd640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82180d1fd0 0x7f8218150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7effa5d52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effa00b3180 0x7effa014df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:60746/0 (socket says 172.21.15.29:60746) 2024-04-25T00:31:39.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7effa5d52640 1 -- 172.21.15.29:0/1224149074 learned_addr learned my addr 172.21.15.29:0/1224149074 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.744+0000 7effa6d54640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa01632d0 0x7effa00cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7effa6d54640 1 -- 172.21.15.29:0/1224149074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7effa006db30 con 0x7effa01632d0 2024-04-25T00:31:39.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7effa6d54640 1 -- 172.21.15.29:0/1224149074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7effa006d9b0 con 0x7effa00b3180 2024-04-25T00:31:39.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7f82177fe640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8218070260 0x7f82180c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7f8217fff640 1 -- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82180d1fd0 msgr2=0x7f8218150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7f8217fff640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82180d1fd0 0x7f8218150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7f8217fff640 1 -- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8218070260 msgr2=0x7f82180c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7effa6d54640 1 -- 172.21.15.29:0/1224149074 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7effa006dcb0 con 0x7effa0162ef0 2024-04-25T00:31:39.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7effa5551640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effa0162ef0 0x7effa01599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7effa6553640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa01632d0 0x7effa00cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7f8217fff640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8218070260 0x7f82180c4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7f8217fff640 1 -- 172.21.15.29:0/2052475724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82180b71f0 con 0x7f82180b6cc0 2024-04-25T00:31:39.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7f82177fe640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8218070260 0x7f82180c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:39.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7f8217fff640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82180b6cc0 0x7f8218149ec0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f820c006e90 tx=0x7f820c00d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7effa6553640 1 -- 172.21.15.29:0/1224149074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effa0162ef0 msgr2=0x7effa01599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7effa6553640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effa0162ef0 0x7effa01599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7effa6553640 1 -- 172.21.15.29:0/1224149074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effa00b3180 msgr2=0x7effa014df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7effa6553640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effa00b3180 0x7effa014df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7effa6553640 1 -- 172.21.15.29:0/1224149074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa0159200 con 0x7effa01632d0 2024-04-25T00:31:39.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7effa6553640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa01632d0 0x7effa00cc490 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7eff900ac000 tx=0x7eff90019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7fcb18eda640 1 -- 172.21.15.29:0/738219261 reap_dead start 2024-04-25T00:31:39.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7f81fffff640 1 -- 172.21.15.29:0/2052475724 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c0125f0 con 0x7f82180b6cc0 2024-04-25T00:31:39.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7fcb18eda640 1 --2- 172.21.15.29:0/738219261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcafc005220 0x7fcafc00b9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c001e70 tx=0x7fcb0c01cd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb195c36afa1d29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7f821c8a6640 1 -- 172.21.15.29:0/2052475724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82180b74d0 con 0x7f82180b6cc0 2024-04-25T00:31:39.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7f821c8a6640 1 -- 172.21.15.29:0/2052475724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82180c9160 con 0x7f82180b6cc0 2024-04-25T00:31:39.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7f81fffff640 1 -- 172.21.15.29:0/2052475724 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f820c019070 con 0x7f82180b6cc0 2024-04-25T00:31:39.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb080090e0 con 0x7fcafc005220 2024-04-25T00:31:39.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb0c0a5da0 con 0x7fcafc005220 2024-04-25T00:31:39.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7fcb12ffd640 1 -- 172.21.15.29:0/738219261 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c001ac0 con 0x7fcafc005220 2024-04-25T00:31:39.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7fcb1a93d640 1 -- 172.21.15.29:0/738219261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcafc005220 msgr2=0x7fcafc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.748+0000 7fcb1a93d640 1 --2- 172.21.15.29:0/738219261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcafc005220 0x7fcafc00b9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c001e70 tx=0x7fcb0c01cd90 comp rx=0 tx=0).stop 2024-04-25T00:31:39.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7eff967fc640 1 -- 172.21.15.29:0/1224149074 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90003970 con 0x7effa01632d0 2024-04-25T00:31:39.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7fcb1a93d640 1 -- 172.21.15.29:0/738219261 shutdown_connections 2024-04-25T00:31:39.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7fcb1a93d640 1 --2- 172.21.15.29:0/738219261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcafc005220 0x7fcafc00b9d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7fcb1a93d640 1 -- 172.21.15.29:0/738219261 >> 172.21.15.29:0/738219261 conn(0x7fcb140b1910 msgr2=0x7fcb140b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7fcb1a93d640 1 -- 172.21.15.29:0/738219261 shutdown_connections 2024-04-25T00:31:39.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7effa6d54640 1 -- 172.21.15.29:0/1224149074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa01594e0 con 0x7effa01632d0 2024-04-25T00:31:39.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7effa6d54640 1 -- 172.21.15.29:0/1224149074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa014fb70 con 0x7effa01632d0 2024-04-25T00:31:39.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7f81fffff640 1 -- 172.21.15.29:0/2052475724 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c0129f0 con 0x7f82180b6cc0 2024-04-25T00:31:39.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7f81fffff640 1 -- 172.21.15.29:0/2052475724 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f820c012b90 con 0x7f82180b6cc0 2024-04-25T00:31:39.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7fcb1a93d640 1 -- 172.21.15.29:0/738219261 wait complete. 2024-04-25T00:31:39.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7fcb1a93d640 1 Processor -- start 2024-04-25T00:31:39.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7f81fffff640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8200067c80 0x7f820006a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7f821c8a6640 1 -- 172.21.15.29:0/2052475724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82040052a0 con 0x7f82180b6cc0 2024-04-25T00:31:39.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7eff967fc640 1 -- 172.21.15.29:0/1224149074 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff90003b10 con 0x7effa01632d0 2024-04-25T00:31:39.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7eff967fc640 1 -- 172.21.15.29:0/1224149074 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90003cc0 con 0x7effa01632d0 2024-04-25T00:31:39.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7fcb1a93d640 1 -- start start 2024-04-25T00:31:39.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7fcb1a93d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcafc005220 0x7fcb14145230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7fcb1a93d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb140b3cd0 0x7fcb1409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7fcb1a93d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb1406e720 0x7fcb1406ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7fcb1a93d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcb14072ae0 con 0x7fcb140b3cd0 2024-04-25T00:31:39.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7fcb1a93d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb14072960 con 0x7fcafc005220 2024-04-25T00:31:39.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.752+0000 7fcb1a93d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcb14072c60 con 0x7fcb1406e720 2024-04-25T00:31:39.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7eff967fc640 1 -- 172.21.15.29:0/1224149074 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7eff900be030 con 0x7effa01632d0 2024-04-25T00:31:39.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb140b3cd0 0x7fcb1409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb140b3cd0 0x7fcb1409aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49760/0 (socket says 172.21.15.29:49760) 2024-04-25T00:31:39.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb137fe640 1 -- 172.21.15.29:0/4252038713 learned_addr learned my addr 172.21.15.29:0/4252038713 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb13fff640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcafc005220 0x7fcb14145230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb18eda640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb1406e720 0x7fcb1406ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7eff967fc640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff840679f0 0x7eff84069eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7eff967fc640 1 -- 172.21.15.29:0/1224149074 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7eff9011f2e0 con 0x7effa01632d0 2024-04-25T00:31:39.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb13fff640 1 -- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb1406e720 msgr2=0x7fcb1406ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb13fff640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb1406e720 0x7fcb1406ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb13fff640 1 -- 172.21.15.29:0/4252038713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb140b3cd0 msgr2=0x7fcb1409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb13fff640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb140b3cd0 0x7fcb1409aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb13fff640 1 -- 172.21.15.29:0/4252038713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1409af70 con 0x7fcafc005220 2024-04-25T00:31:39.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7effa5d52640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff840679f0 0x7eff84069eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb13fff640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcafc005220 0x7fcb14145230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009fd0 tx=0x7fcb0800fd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb10ff9640 1 -- 172.21.15.29:0/4252038713 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800dc50 con 0x7fcafc005220 2024-04-25T00:31:39.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7effa5d52640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff840679f0 0x7eff84069eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7eff98009bd0 tx=0x7eff9800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb1a93d640 1 -- 172.21.15.29:0/4252038713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb140b4150 con 0x7fcafc005220 2024-04-25T00:31:39.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb1a93d640 1 -- 172.21.15.29:0/4252038713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb14151590 con 0x7fcafc005220 2024-04-25T00:31:39.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb1a93d640 1 -- 172.21.15.29:0/4252038713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb14069200 con 0x7fcafc005220 2024-04-25T00:31:39.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb10ff9640 1 -- 172.21.15.29:0/4252038713 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb0800ddf0 con 0x7fcafc005220 2024-04-25T00:31:39.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.756+0000 7fcb10ff9640 1 -- 172.21.15.29:0/4252038713 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08004260 con 0x7fcafc005220 2024-04-25T00:31:39.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.760+0000 7fcb10ff9640 1 -- 172.21.15.29:0/4252038713 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcb08003620 con 0x7fcafc005220 2024-04-25T00:31:39.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.760+0000 7fcb10ff9640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcafc069580 0x7fcafc06ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.760+0000 7fcb137fe640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcafc069580 0x7fcafc06ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.760+0000 7fcb10ff9640 1 -- 172.21.15.29:0/4252038713 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcb08086a30 con 0x7fcafc005220 2024-04-25T00:31:39.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.764+0000 7f81fffff640 1 -- 172.21.15.29:0/2052475724 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f820c0860c0 con 0x7f82180b6cc0 2024-04-25T00:31:39.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.764+0000 7f82177fe640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8200067c80 0x7f820006a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.764+0000 7f81fffff640 1 -- 172.21.15.29:0/2052475724 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f820c088500 con 0x7f82180b6cc0 2024-04-25T00:31:39.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.764+0000 7effa6d54640 1 -- 172.21.15.29:0/1224149074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff880052a0 con 0x7effa01632d0 2024-04-25T00:31:39.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.764+0000 7fcb137fe640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcafc069580 0x7fcafc06ba40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcb040011f0 tx=0x7fcb040010a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.764+0000 7f82177fe640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8200067c80 0x7f820006a140 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f82080ad000 tx=0x7f8208008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.772+0000 7eff967fc640 1 -- 172.21.15.29:0/1224149074 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7eff900c5020 con 0x7effa01632d0 2024-04-25T00:31:39.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.772+0000 7fcb10ff9640 1 -- 172.21.15.29:0/4252038713 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcb08029020 con 0x7fcafc005220 2024-04-25T00:31:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4aa097640 1 Processor -- start 2024-04-25T00:31:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4aa097640 1 -- start start 2024-04-25T00:31:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4aa097640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4a40b28b0 0x7fd4a4166a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4aa097640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4a40b4a30 0x7fd4a40cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4aa097640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4a40725d0 con 0x7fd4a40b2c90 2024-04-25T00:31:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4aa097640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd4a4165f20 con 0x7fd4a40b4a30 2024-04-25T00:31:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4aa097640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a4166200 con 0x7fd4a40b28b0 2024-04-25T00:31:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4a2ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4a40b4a30 0x7fd4a40cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4a2ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4a40b4a30 0x7fd4a40cfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49784/0 (socket says 172.21.15.29:49784) 2024-04-25T00:31:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4a2ffd640 1 -- 172.21.15.29:0/2760566979 learned_addr learned my addr 172.21.15.29:0/2760566979 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4a37fe640 1 --2- 172.21.15.29:0/2760566979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4a40b28b0 0x7fd4a4166a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7f7b91835640 1 Processor -- start 2024-04-25T00:31:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7f7b91835640 1 -- start start 2024-04-25T00:31:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7f7b91835640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7b8c0b2fc0 0x7f7b8c0b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4a37fe640 1 -- 172.21.15.29:0/2760566979 >> v2:172.21.15.29:3300/0 conn(0x7fd4a40b4a30 msgr2=0x7fd4a40cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4a37fe640 1 --2- 172.21.15.29:0/2760566979 >> v2:172.21.15.29:3300/0 conn(0x7fd4a40b4a30 0x7fd4a40cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4a37fe640 1 -- 172.21.15.29:0/2760566979 >> v1:172.21.15.29:6789/0 conn(0x7fd4a40b2c90 legacy=0x7fd4a4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4a37fe640 1 -- 172.21.15.29:0/2760566979 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a41664e0 con 0x7fd4a40b28b0 2024-04-25T00:31:39.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7fd4a2ffd640 1 --2- 172.21.15.29:0/2760566979 >> v2:172.21.15.29:3300/0 conn(0x7fd4a40b4a30 0x7fd4a40cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:39.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7f7b91835640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b8c072d90 con 0x7f7b8c070d40 2024-04-25T00:31:39.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7f7b91835640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b8c16e430 con 0x7f7b8c16e050 2024-04-25T00:31:39.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7f7b91835640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7b8c071300 con 0x7f7b8c0b2fc0 2024-04-25T00:31:39.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7f7b8a7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7b8c0b2fc0 0x7f7b8c0b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7f7b8a7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7b8c0b2fc0 0x7f7b8c0b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49776/0 (socket says 172.21.15.29:49776) 2024-04-25T00:31:39.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7f7b8a7fc640 1 -- 172.21.15.29:0/2469711169 learned_addr learned my addr 172.21.15.29:0/2469711169 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7f7b8a7fc640 1 -- 172.21.15.29:0/2469711169 >> v1:172.21.15.157:6789/0 conn(0x7f7b8c16e050 legacy=0x7f7b8c0b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7f7b8a7fc640 1 -- 172.21.15.29:0/2469711169 >> v1:172.21.15.29:6789/0 conn(0x7f7b8c070d40 legacy=0x7f7b8c08b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.796+0000 7f7b8a7fc640 1 -- 172.21.15.29:0/2469711169 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8c0715e0 con 0x7f7b8c0b2fc0 2024-04-25T00:31:39.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.800+0000 7f7b8a7fc640 1 --2- 172.21.15.29:0/2469711169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b8c0b2fc0 0x7f7b8c0b4720 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c006de0 tx=0x7f7b7c020250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb90326ba1bb3a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.800+0000 7fd4a37fe640 1 --2- 172.21.15.29:0/2760566979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4a40b28b0 0x7fd4a4166a00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd4980097f0 tx=0x7fd49801c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6ef898654a00b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.800+0000 7f7b89ffb640 1 -- 172.21.15.29:0/2469711169 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0092d0 con 0x7f7b8c0b2fc0 2024-04-25T00:31:39.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.800+0000 7f7b89ffb640 1 -- 172.21.15.29:0/2469711169 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b7c0a8970 con 0x7f7b8c0b2fc0 2024-04-25T00:31:39.806 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:31:39.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.800+0000 7fd4a27fc640 1 -- 172.21.15.29:0/2760566979 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4980b0070 con 0x7fd4a40b28b0 2024-04-25T00:31:39.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.800+0000 7fd4a27fc640 1 -- 172.21.15.29:0/2760566979 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4980ab600 con 0x7fd4a40b28b0 2024-04-25T00:31:39.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.800+0000 7fd4a27fc640 1 -- 172.21.15.29:0/2760566979 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4980ab9d0 con 0x7fd4a40b28b0 2024-04-25T00:31:39.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 -- 172.21.15.29:0/2469711169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b8c0b2fc0 msgr2=0x7f7b8c0b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 --2- 172.21.15.29:0/2469711169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b8c0b2fc0 0x7f7b8c0b4720 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c006de0 tx=0x7f7b7c020250 comp rx=0 tx=0).stop 2024-04-25T00:31:39.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 -- 172.21.15.29:0/2469711169 shutdown_connections 2024-04-25T00:31:39.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 --2- 172.21.15.29:0/2469711169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b8c0b2fc0 0x7f7b8c0b4720 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 -- 172.21.15.29:0/2469711169 >> 172.21.15.29:0/2469711169 conn(0x7f7b8c074880 msgr2=0x7f7b8c06b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7fd4aa097640 1 -- 172.21.15.29:0/2760566979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4a40b28b0 msgr2=0x7fd4a4166a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7fd4aa097640 1 --2- 172.21.15.29:0/2760566979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4a40b28b0 0x7fd4a4166a00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd4980097f0 tx=0x7fd49801c340 comp rx=0 tx=0).stop 2024-04-25T00:31:39.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7fd4aa097640 1 -- 172.21.15.29:0/2760566979 shutdown_connections 2024-04-25T00:31:39.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7fd4aa097640 1 --2- 172.21.15.29:0/2760566979 >> v2:172.21.15.29:3300/0 conn(0x7fd4a40b4a30 0x7fd4a40cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7fd4aa097640 1 --2- 172.21.15.29:0/2760566979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4a40b28b0 0x7fd4a4166a00 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd4980097f0 tx=0x7fd49801c340 comp rx=0 tx=0).stop 2024-04-25T00:31:39.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7fd4aa097640 1 -- 172.21.15.29:0/2760566979 >> 172.21.15.29:0/2760566979 conn(0x7fd4a4074880 msgr2=0x7fd4a406b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7fd4aa097640 1 -- 172.21.15.29:0/2760566979 shutdown_connections 2024-04-25T00:31:39.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7fd4aa097640 1 -- 172.21.15.29:0/2760566979 wait complete. 2024-04-25T00:31:39.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 -- 172.21.15.29:0/2469711169 shutdown_connections 2024-04-25T00:31:39.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7fd4aa097640 1 Processor -- start 2024-04-25T00:31:39.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7fd4aa097640 1 -- start start 2024-04-25T00:31:39.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 -- 172.21.15.29:0/2469711169 wait complete. 2024-04-25T00:31:39.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 Processor -- start 2024-04-25T00:31:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 -- start start 2024-04-25T00:31:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b8c070d40 0x7f7b8c15c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b8c16e050 0x7f7b8c1694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b8c169a10 0x7f7b8c16b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8c08bd50 con 0x7f7b8c070d40 2024-04-25T00:31:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8c08bbd0 con 0x7f7b8c16e050 2024-04-25T00:31:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b91835640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7b8c176720 con 0x7f7b8c169a10 2024-04-25T00:31:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b8affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b8c070d40 0x7f7b8c15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b8affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b8c070d40 0x7f7b8c15c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49798/0 (socket says 172.21.15.29:49798) 2024-04-25T00:31:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7fd4aa097640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4a40b2c90 0x7fd4a41457d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7fd4aa097640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a40b4a30 0x7fd4a40ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b8affd640 1 -- 172.21.15.29:0/149693611 learned_addr learned my addr 172.21.15.29:0/149693611 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b8b7fe640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b8c169a10 0x7f7b8c16b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b8a7fc640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b8c16e050 0x7f7b8c1694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b8affd640 1 -- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b8c169a10 msgr2=0x7f7b8c16b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b8affd640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b8c169a10 0x7f7b8c16b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b8affd640 1 -- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b8c16e050 msgr2=0x7f7b8c1694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b8affd640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b8c16e050 0x7f7b8c1694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b8affd640 1 -- 172.21.15.29:0/149693611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8c169f40 con 0x7f7b8c070d40 2024-04-25T00:31:39.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7f7b8affd640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b8c070d40 0x7f7b8c15c310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000c530 tx=0x7f7b8000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.804+0000 7fd4aa097640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4a4146c70 0x7fd4a4147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7f7b73fff640 1 -- 172.21.15.29:0/149693611 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80017070 con 0x7f7b8c070d40 2024-04-25T00:31:39.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7f7b73fff640 1 -- 172.21.15.29:0/149693611 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b80005480 con 0x7f7b8c070d40 2024-04-25T00:31:39.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7f7b91835640 1 -- 172.21.15.29:0/149693611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b8c15cff0 con 0x7f7b8c070d40 2024-04-25T00:31:39.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4aa097640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a4162760 con 0x7fd4a40b2c90 2024-04-25T00:31:39.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4aa097640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a41625e0 con 0x7fd4a40b4a30 2024-04-25T00:31:39.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4aa097640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a4071650 con 0x7fd4a4146c70 2024-04-25T00:31:39.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4a2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a40b4a30 0x7fd4a40ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4a2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a40b4a30 0x7fd4a40ab210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:60776/0 (socket says 172.21.15.29:60776) 2024-04-25T00:31:39.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4a2ffd640 1 -- 172.21.15.29:0/733255197 learned_addr learned my addr 172.21.15.29:0/733255197 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4a3fff640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4a4146c70 0x7fd4a4147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7f7b73fff640 1 -- 172.21.15.29:0/149693611 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80003840 con 0x7f7b8c070d40 2024-04-25T00:31:39.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7f7b91835640 1 -- 172.21.15.29:0/149693611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b8c15c850 con 0x7f7b8c070d40 2024-04-25T00:31:39.817 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:31:39.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4a2ffd640 1 -- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4a4146c70 msgr2=0x7fd4a4147370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4a2ffd640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4a4146c70 0x7fd4a4147370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4a2ffd640 1 -- 172.21.15.29:0/733255197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4a40b2c90 msgr2=0x7fd4a41457d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:31:39.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4a37fe640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4a40b2c90 0x7fd4a41457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4a2ffd640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4a40b2c90 0x7fd4a41457d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4a2ffd640 1 -- 172.21.15.29:0/733255197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a40aa450 con 0x7fd4a40b4a30 2024-04-25T00:31:39.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4a37fe640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4a40b2c90 0x7fd4a41457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:39.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4a2ffd640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a40b4a30 0x7fd4a40ab210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00efc0 tx=0x7fd49c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd48bfff640 1 -- 172.21.15.29:0/733255197 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00ce60 con 0x7fd4a40b4a30 2024-04-25T00:31:39.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7f7b73fff640 1 -- 172.21.15.29:0/149693611 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7b8001a070 con 0x7f7b8c070d40 2024-04-25T00:31:39.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4aa097640 1 -- 172.21.15.29:0/733255197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a40aa730 con 0x7fd4a40b4a30 2024-04-25T00:31:39.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd4aa097640 1 -- 172.21.15.29:0/733255197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a40aacf0 con 0x7fd4a40b4a30 2024-04-25T00:31:39.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd48bfff640 1 -- 172.21.15.29:0/733255197 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd49c019070 con 0x7fd4a40b4a30 2024-04-25T00:31:39.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.808+0000 7fd48bfff640 1 -- 172.21.15.29:0/733255197 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0146c0 con 0x7fd4a40b4a30 2024-04-25T00:31:39.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f7b73fff640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7b78067910 0x7f7b78069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f7b91835640 1 -- 172.21.15.29:0/149693611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b8c069200 con 0x7f7b8c070d40 2024-04-25T00:31:39.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f7b73fff640 1 -- 172.21.15.29:0/149693611 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7b80084ed0 con 0x7f7b8c070d40 2024-04-25T00:31:39.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f7b8a7fc640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7b78067910 0x7f7b78069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7fd4aa097640 1 -- 172.21.15.29:0/733255197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4840052a0 con 0x7fd4a40b4a30 2024-04-25T00:31:39.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f7b8a7fc640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7b78067910 0x7f7b78069dd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0b1000 tx=0x7f7b7c0aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f41441e2640 1 Processor -- start 2024-04-25T00:31:39.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f41441e2640 1 -- start start 2024-04-25T00:31:39.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f1db359e640 1 Processor -- start 2024-04-25T00:31:39.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f41441e2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f413c1406f0 0x7f413c0b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f41441e2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f413c15aa30 con 0x7f413c071840 2024-04-25T00:31:39.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f41441e2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f413c071d10 con 0x7f413c0b33c0 2024-04-25T00:31:39.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f41441e2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f413c072920 con 0x7f413c1406f0 2024-04-25T00:31:39.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f1db359e640 1 -- start start 2024-04-25T00:31:39.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7fd48bfff640 1 -- 172.21.15.29:0/733255197 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd49c012050 con 0x7fd4a40b4a30 2024-04-25T00:31:39.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7fd48bfff640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd4900678c0 0x7fd490069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7fd48bfff640 1 -- 172.21.15.29:0/733255197 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd49c0860b0 con 0x7fd4a40b4a30 2024-04-25T00:31:39.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f1db359e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1db4162ef0 0x7f1db4070e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f1db359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1db41632d0 0x7f1db40b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f414177d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f413c0b33c0 0x7f413c071130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:44020/0 (socket says 172.21.15.29:44020) 2024-04-25T00:31:39.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f414177d640 1 -- 172.21.15.29:0/931814118 learned_addr learned my addr 172.21.15.29:0/931814118 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:39.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f4141f7e640 1 --2- 172.21.15.29:0/931814118 >> v2:172.21.15.29:3300/0 conn(0x7f413c1406f0 0x7f413c0b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f1db359e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1db40b2fb0 0x7f1db4165ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f1db359e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1db40b3400 con 0x7f1db40b2fb0 2024-04-25T00:31:39.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f1db359e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1db40713f0 con 0x7f1db4162ef0 2024-04-25T00:31:39.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f1db359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1db40716d0 con 0x7f1db41632d0 2024-04-25T00:31:39.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f1db259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1db4162ef0 0x7f1db4070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f1db259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1db4162ef0 0x7f1db4070e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:60782/0 (socket says 172.21.15.29:60782) 2024-04-25T00:31:39.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f1db259c640 1 -- 172.21.15.29:0/3280622952 learned_addr learned my addr 172.21.15.29:0/3280622952 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f1db1d9b640 1 --2- 172.21.15.29:0/3280622952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1db41632d0 0x7f1db40b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.812+0000 7f1db2d9d640 1 --2- 172.21.15.29:0/3280622952 >> v2:172.21.15.29:3300/0 conn(0x7f1db40b2fb0 0x7f1db4165ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.816+0000 7f1db259c640 1 -- 172.21.15.29:0/3280622952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1db41632d0 msgr2=0x7f1db40b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.816+0000 7f1db259c640 1 --2- 172.21.15.29:0/3280622952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1db41632d0 0x7f1db40b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.816+0000 7f1db259c640 1 -- 172.21.15.29:0/3280622952 >> v2:172.21.15.29:3300/0 conn(0x7f1db40b2fb0 msgr2=0x7f1db4165ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.816+0000 7f1db259c640 1 --2- 172.21.15.29:0/3280622952 >> v2:172.21.15.29:3300/0 conn(0x7f1db40b2fb0 0x7f1db4165ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.816+0000 7f1db259c640 1 -- 172.21.15.29:0/3280622952 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db40719b0 con 0x7f1db4162ef0 2024-04-25T00:31:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.816+0000 7f4141f7e640 1 -- 172.21.15.29:0/931814118 >> v1:172.21.15.157:6789/0 conn(0x7f413c0b33c0 legacy=0x7f413c071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.816+0000 7f4141f7e640 1 -- 172.21.15.29:0/931814118 >> v1:172.21.15.29:6789/0 conn(0x7f413c071840 legacy=0x7f413c06ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.816+0000 7f4141f7e640 1 -- 172.21.15.29:0/931814118 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c071ff0 con 0x7f413c1406f0 2024-04-25T00:31:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.816+0000 7f1db259c640 1 --2- 172.21.15.29:0/3280622952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db4162ef0 0x7f1db4070e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1db406c590 tx=0x7f1dac01c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39818ecaeb85e5f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.816+0000 7f4141f7e640 1 --2- 172.21.15.29:0/931814118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f413c1406f0 0x7f413c0b2e80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f413c072aa0 tx=0x7f412c020a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babc8409f21215a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.816+0000 7fd4a37fe640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd4900678c0 0x7fd490069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7f7b73fff640 1 -- 172.21.15.29:0/149693611 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7b80055a00 con 0x7f7b8c070d40 2024-04-25T00:31:39.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7f4140f7c640 1 -- 172.21.15.29:0/931814118 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c0af070 con 0x7f413c1406f0 2024-04-25T00:31:39.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7fdc9759e640 1 Processor -- start 2024-04-25T00:31:39.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7fd4a37fe640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd4900678c0 0x7fd490069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd4980ad000 tx=0x7fd498008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7fdc9759e640 1 -- start start 2024-04-25T00:31:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7f4140f7c640 1 -- 172.21.15.29:0/931814118 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f412c006c90 con 0x7f413c1406f0 2024-04-25T00:31:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7f41441e2640 1 -- 172.21.15.29:0/931814118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f413c1406f0 msgr2=0x7f413c0b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7f41441e2640 1 --2- 172.21.15.29:0/931814118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f413c1406f0 0x7f413c0b2e80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f413c072aa0 tx=0x7f412c020a50 comp rx=0 tx=0).stop 2024-04-25T00:31:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7f41441e2640 1 -- 172.21.15.29:0/931814118 shutdown_connections 2024-04-25T00:31:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7f41441e2640 1 --2- 172.21.15.29:0/931814118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f413c1406f0 0x7f413c0b2e80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7f41441e2640 1 -- 172.21.15.29:0/931814118 >> 172.21.15.29:0/931814118 conn(0x7f413c0b1870 msgr2=0x7f413c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7f41441e2640 1 -- 172.21.15.29:0/931814118 shutdown_connections 2024-04-25T00:31:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7f41441e2640 1 -- 172.21.15.29:0/931814118 wait complete. 2024-04-25T00:31:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7fdc9659c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdc9815b380 0x7fdc980d0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50046/0 (socket says 172.21.15.29:50046) 2024-04-25T00:31:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7fdc9659c640 1 -- 172.21.15.29:0/26547786 learned_addr learned my addr 172.21.15.29:0/26547786 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7fdc9759e640 1 --2- 172.21.15.29:0/26547786 >> v2:172.21.15.29:3300/0 conn(0x7fdc9815b760 0x7fdc9815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7fdc9759e640 1 --2- 172.21.15.29:0/26547786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc9806ddd0 0x7fdc9806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7f41441e2640 1 Processor -- start 2024-04-25T00:31:39.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7fdc9759e640 1 -- 172.21.15.29:0/26547786 --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc980ceff0 con 0x7fdc9815b380 2024-04-25T00:31:39.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7fdc95d9b640 1 --2- 172.21.15.29:0/26547786 >> v2:172.21.15.29:3300/0 conn(0x7fdc9815b760 0x7fdc9815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7fdc96d9d640 1 --2- 172.21.15.29:0/26547786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc9806ddd0 0x7fdc9806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7fdc9759e640 1 -- 172.21.15.29:0/26547786 --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdc980b6ab0 con 0x7fdc9815b760 2024-04-25T00:31:39.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.820+0000 7f41441e2640 1 -- start start 2024-04-25T00:31:39.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7fdc9759e640 1 -- 172.21.15.29:0/26547786 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc980b5be0 con 0x7fdc9806ddd0 2024-04-25T00:31:39.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7fdc95d9b640 1 -- 172.21.15.29:0/26547786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc9806ddd0 msgr2=0x7fdc9806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:39.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7fdc95d9b640 1 --2- 172.21.15.29:0/26547786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc9806ddd0 0x7fdc9806e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7fdc95d9b640 1 -- 172.21.15.29:0/26547786 >> v1:172.21.15.29:6789/0 conn(0x7fdc9815b380 legacy=0x7fdc980d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f41441e2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f413c071840 0x7f413c1580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f41441e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f413c0b33c0 0x7f413c0b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f41441e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413c1406f0 0x7f413c154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f41441e2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f413c06e170 con 0x7f413c071840 2024-04-25T00:31:39.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f41441e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f413c06dff0 con 0x7f413c1406f0 2024-04-25T00:31:39.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f41441e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f413c168020 con 0x7f413c0b33c0 2024-04-25T00:31:39.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7fdc95d9b640 1 -- 172.21.15.29:0/26547786 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc980d0760 con 0x7fdc9815b760 2024-04-25T00:31:39.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7fdc96d9d640 1 --2- 172.21.15.29:0/26547786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc9806ddd0 0x7fdc9806e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:39.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f4141f7e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f413c071840 0x7f413c1580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f4141f7e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f413c071840 0x7f413c1580f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49854/0 (socket says 172.21.15.29:49854) 2024-04-25T00:31:39.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f4141f7e640 1 -- 172.21.15.29:0/2890641986 learned_addr learned my addr 172.21.15.29:0/2890641986 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f414177d640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f413c0b33c0 0x7f413c0b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f414277f640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413c1406f0 0x7f413c154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f4141f7e640 1 -- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f413c0b33c0 msgr2=0x7f413c0b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f4141f7e640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f413c0b33c0 0x7f413c0b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f4141f7e640 1 -- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413c1406f0 msgr2=0x7f413c154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f4141f7e640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413c1406f0 0x7f413c154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f4141f7e640 1 -- 172.21.15.29:0/2890641986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c1fee80 con 0x7f413c071840 2024-04-25T00:31:39.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f4141f7e640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f413c071840 0x7f413c1580f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f412c027060 tx=0x7f412c01d7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f41327fc640 1 -- 172.21.15.29:0/2890641986 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c0af070 con 0x7f413c071840 2024-04-25T00:31:39.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f41327fc640 1 -- 172.21.15.29:0/2890641986 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f412c010460 con 0x7f413c071840 2024-04-25T00:31:39.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f41327fc640 1 -- 172.21.15.29:0/2890641986 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c0a9960 con 0x7f413c071840 2024-04-25T00:31:39.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f41441e2640 1 -- 172.21.15.29:0/2890641986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413c158630 con 0x7f413c071840 2024-04-25T00:31:39.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.824+0000 7f41441e2640 1 -- 172.21.15.29:0/2890641986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413c156900 con 0x7f413c071840 2024-04-25T00:31:39.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7f41441e2640 1 -- 172.21.15.29:0/2890641986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f413c1ff050 con 0x7f413c071840 2024-04-25T00:31:39.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7fdc95d9b640 1 --2- 172.21.15.29:0/26547786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc9815b760 0x7fdc9815bbf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0097c0 tx=0x7fdc8c01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d8aae2a816002d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7fdc9559a640 1 -- 172.21.15.29:0/26547786 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c0b0070 con 0x7fdc9815b760 2024-04-25T00:31:39.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7f1db159a640 1 -- 172.21.15.29:0/3280622952 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0a5580 con 0x7f1db4162ef0 2024-04-25T00:31:39.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7f1db159a640 1 -- 172.21.15.29:0/3280622952 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dac0a5720 con 0x7f1db4162ef0 2024-04-25T00:31:39.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7f1db159a640 1 -- 172.21.15.29:0/3280622952 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0a5af0 con 0x7f1db4162ef0 2024-04-25T00:31:39.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7fdc9559a640 1 -- 172.21.15.29:0/26547786 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc8c0ab600 con 0x7fdc9815b760 2024-04-25T00:31:39.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7fd48bfff640 1 -- 172.21.15.29:0/733255197 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd49c010040 con 0x7fd4a40b4a30 2024-04-25T00:31:39.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7f1db359e640 1 -- 172.21.15.29:0/3280622952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db4162ef0 msgr2=0x7f1db4070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7f1db359e640 1 --2- 172.21.15.29:0/3280622952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db4162ef0 0x7f1db4070e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1db406c590 tx=0x7f1dac01c720 comp rx=0 tx=0).stop 2024-04-25T00:31:39.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7fdc9759e640 1 -- 172.21.15.29:0/26547786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc9815b760 msgr2=0x7fdc9815bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7fdc9559a640 1 -- 172.21.15.29:0/26547786 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c0b0070 con 0x7fdc9815b760 2024-04-25T00:31:39.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7fdc9759e640 1 --2- 172.21.15.29:0/26547786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc9815b760 0x7fdc9815bbf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0097c0 tx=0x7fdc8c01c330 comp rx=0 tx=0).stop 2024-04-25T00:31:39.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7f1db359e640 1 -- 172.21.15.29:0/3280622952 shutdown_connections 2024-04-25T00:31:39.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7f1db359e640 1 --2- 172.21.15.29:0/3280622952 >> v2:172.21.15.29:3300/0 conn(0x7f1db40b2fb0 0x7f1db4165ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7f1db359e640 1 --2- 172.21.15.29:0/3280622952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1db41632d0 0x7f1db40b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7f1db359e640 1 --2- 172.21.15.29:0/3280622952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db4162ef0 0x7f1db4070e10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7f1db359e640 1 -- 172.21.15.29:0/3280622952 >> 172.21.15.29:0/3280622952 conn(0x7f1db40b1770 msgr2=0x7f1db40b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7f1db359e640 1 -- 172.21.15.29:0/3280622952 shutdown_connections 2024-04-25T00:31:39.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f41327fc640 1 -- 172.21.15.29:0/2890641986 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f412c0ba070 con 0x7f413c071840 2024-04-25T00:31:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.828+0000 7f1db359e640 1 -- 172.21.15.29:0/3280622952 wait complete. 2024-04-25T00:31:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db359e640 1 Processor -- start 2024-04-25T00:31:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db359e640 1 -- start start 2024-04-25T00:31:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db40b2fb0 0x7f1db414d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1db4162ef0 0x7f1db4173290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db359e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db41632d0 0x7f1db413d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db359e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1db4163850 con 0x7f1db41632d0 2024-04-25T00:31:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41636d0 con 0x7f1db40b2fb0 2024-04-25T00:31:39.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1db406cc60 con 0x7f1db4162ef0 2024-04-25T00:31:39.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db41632d0 0x7f1db413d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db41632d0 0x7f1db413d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49860/0 (socket says 172.21.15.29:49860) 2024-04-25T00:31:39.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db2d9d640 1 -- 172.21.15.29:0/3018787793 learned_addr learned my addr 172.21.15.29:0/3018787793 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db259c640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db40b2fb0 0x7f1db414d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db1d9b640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1db4162ef0 0x7f1db4173290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f41327fc640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f41200678c0 0x7f4120069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f41327fc640 1 -- 172.21.15.29:0/2890641986 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f412c11fa50 con 0x7f413c071840 2024-04-25T00:31:39.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f414177d640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f41200678c0 0x7f4120069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db2d9d640 1 -- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1db4162ef0 msgr2=0x7f1db4173290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db2d9d640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1db4162ef0 0x7f1db4173290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db2d9d640 1 -- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db40b2fb0 msgr2=0x7f1db414d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db2d9d640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db40b2fb0 0x7f1db414d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db2d9d640 1 -- 172.21.15.29:0/3018787793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db40b3cf0 con 0x7f1db41632d0 2024-04-25T00:31:39.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f1db2d9d640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db41632d0 0x7f1db413d600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1da800eab0 tx=0x7f1da800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7fdc9759e640 1 -- 172.21.15.29:0/26547786 shutdown_connections 2024-04-25T00:31:39.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7fdc9759e640 1 --2- 172.21.15.29:0/26547786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc9806ddd0 0x7fdc9806e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7fdc9759e640 1 --2- 172.21.15.29:0/26547786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc9815b760 0x7fdc9815bbf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7fdc9759e640 1 -- 172.21.15.29:0/26547786 >> 172.21.15.29:0/26547786 conn(0x7fdc9806b210 msgr2=0x7fdc9806b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:39.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7fdc9759e640 1 -- 172.21.15.29:0/26547786 shutdown_connections 2024-04-25T00:31:39.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7fdc9759e640 1 -- 172.21.15.29:0/26547786 wait complete. 2024-04-25T00:31:39.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.832+0000 7f414177d640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f41200678c0 0x7f4120069d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f413c15a2b0 tx=0x7f413800b220 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9759e640 1 Processor -- start 2024-04-25T00:31:39.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9759e640 1 -- start start 2024-04-25T00:31:39.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9759e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc9806ddd0 0x7fdc981433c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc9806ddd0 0x7fdc981433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc9806ddd0 0x7fdc981433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49876/0 (socket says 172.21.15.29:49876) 2024-04-25T00:31:39.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9659c640 1 -- 172.21.15.29:0/3470479998 learned_addr learned my addr 172.21.15.29:0/3470479998 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:39.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9759e640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc9815b380 0x7fdc981461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9759e640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc98138f30 0x7fdc98150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9759e640 1 -- 172.21.15.29:0/3470479998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98167740 con 0x7fdc9806ddd0 2024-04-25T00:31:39.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9759e640 1 -- 172.21.15.29:0/3470479998 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc980cfb30 con 0x7fdc9815b380 2024-04-25T00:31:39.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9759e640 1 -- 172.21.15.29:0/3470479998 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc981678c0 con 0x7fdc98138f30 2024-04-25T00:31:39.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc96d9d640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc98138f30 0x7fdc98150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7f41327fc640 1 -- 172.21.15.29:0/2890641986 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f412c01a640 con 0x7f413c071840 2024-04-25T00:31:39.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9659c640 1 -- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc98138f30 msgr2=0x7fdc98150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:39.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9659c640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc98138f30 0x7fdc98150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9659c640 1 -- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc9815b380 msgr2=0x7fdc981461d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:31:39.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9659c640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc9815b380 0x7fdc981461d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.836+0000 7fdc9659c640 1 -- 172.21.15.29:0/3470479998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc98150970 con 0x7fdc9806ddd0 2024-04-25T00:31:39.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.840+0000 7fdc9659c640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc9806ddd0 0x7fdc981433c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800c7b0 tx=0x7fdc8800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.840+0000 7f1da2ffd640 1 -- 172.21.15.29:0/3018787793 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8017070 con 0x7f1db41632d0 2024-04-25T00:31:39.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.840+0000 7f1da2ffd640 1 -- 172.21.15.29:0/3018787793 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da800c910 con 0x7f1db41632d0 2024-04-25T00:31:39.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.840+0000 7f1da2ffd640 1 -- 172.21.15.29:0/3018787793 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da800ccc0 con 0x7f1db41632d0 2024-04-25T00:31:39.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.840+0000 7f1db359e640 1 -- 172.21.15.29:0/3018787793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db41737d0 con 0x7f1db41632d0 2024-04-25T00:31:39.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.840+0000 7f1db359e640 1 -- 172.21.15.29:0/3018787793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db40ce000 con 0x7f1db41632d0 2024-04-25T00:31:39.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.840+0000 7f1db359e640 1 -- 172.21.15.29:0/3018787793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db4069200 con 0x7f1db41632d0 2024-04-25T00:31:39.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.840+0000 7f1da2ffd640 1 -- 172.21.15.29:0/3018787793 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1da801a070 con 0x7f1db41632d0 2024-04-25T00:31:39.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.844+0000 7f1da2ffd640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d940678f0 0x7f1d94069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.844+0000 7f1db259c640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d940678f0 0x7f1d94069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.844+0000 7f1da2ffd640 1 -- 172.21.15.29:0/3018787793 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1da8054960 con 0x7f1db41632d0 2024-04-25T00:31:39.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.844+0000 7f1db259c640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d940678f0 0x7f1d94069db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0ae000 tx=0x7f1dac0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.852+0000 7fdc86ffd640 1 -- 172.21.15.29:0/3470479998 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88018070 con 0x7fdc9806ddd0 2024-04-25T00:31:39.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.852+0000 7fdc86ffd640 1 -- 172.21.15.29:0/3470479998 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc880072d0 con 0x7fdc9806ddd0 2024-04-25T00:31:39.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.852+0000 7fdc9759e640 1 -- 172.21.15.29:0/3470479998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc98146950 con 0x7fdc9806ddd0 2024-04-25T00:31:39.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.852+0000 7fdc9759e640 1 -- 172.21.15.29:0/3470479998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9814f950 con 0x7fdc9806ddd0 2024-04-25T00:31:39.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.852+0000 7fdc86ffd640 1 -- 172.21.15.29:0/3470479998 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc880047f0 con 0x7fdc9806ddd0 2024-04-25T00:31:39.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.852+0000 7fdc9759e640 1 -- 172.21.15.29:0/3470479998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7c0052a0 con 0x7fdc9806ddd0 2024-04-25T00:31:39.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.852+0000 7f1da2ffd640 1 -- 172.21.15.29:0/3018787793 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1da800ce60 con 0x7f1db41632d0 2024-04-25T00:31:39.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.856+0000 7fdc86ffd640 1 -- 172.21.15.29:0/3470479998 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdc8801b070 con 0x7fdc9806ddd0 2024-04-25T00:31:39.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.856+0000 7fdc86ffd640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc78067940 0x7fdc78069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:39.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.856+0000 7fdc86ffd640 1 -- 172.21.15.29:0/3470479998 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdc88053a00 con 0x7fdc9806ddd0 2024-04-25T00:31:39.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.856+0000 7fdc95d9b640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc78067940 0x7fdc78069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:39.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.856+0000 7fdc95d9b640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc78067940 0x7fdc78069e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0ad000 tx=0x7fdc8c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:39.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:39.864+0000 7fdc86ffd640 1 -- 172.21.15.29:0/3470479998 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdc88022020 con 0x7fdc9806ddd0 2024-04-25T00:31:40.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.100+0000 7fcb1a93d640 1 -- 172.21.15.29:0/4252038713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fcb140d0900 con 0x7fcafc005220 2024-04-25T00:31:40.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.100+0000 7fcb10ff9640 1 -- 172.21.15.29:0/4252038713 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcb08056f90 con 0x7fcafc005220 2024-04-25T00:31:40.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.100+0000 7effa6d54640 1 -- 172.21.15.29:0/1224149074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7eff88005590 con 0x7effa01632d0 2024-04-25T00:31:40.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.100+0000 7f821c8a6640 1 -- 172.21.15.29:0/2052475724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f8204005590 con 0x7f82180b6cc0 2024-04-25T00:31:40.105 INFO:teuthology.orchestra.run.smithi029.stdout:55834574853 2024-04-25T00:31:40.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.104+0000 7eff967fc640 1 -- 172.21.15.29:0/1224149074 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7eff900160b0 con 0x7effa01632d0 2024-04-25T00:31:40.107 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-04-25T00:31:40.108 INFO:teuthology.orchestra.run.smithi029.stdout:55834574853 2024-04-25T00:31:40.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.104+0000 7f81fffff640 1 -- 172.21.15.29:0/2052475724 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f820c056c70 con 0x7f82180b6cc0 2024-04-25T00:31:40.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.104+0000 7eff83fff640 1 -- 172.21.15.29:0/1224149074 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff840679f0 msgr2=0x7eff84069eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.104+0000 7eff83fff640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff840679f0 0x7eff84069eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7eff98009bd0 tx=0x7eff9800b040 comp rx=0 tx=0).stop 2024-04-25T00:31:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.104+0000 7eff83fff640 1 -- 172.21.15.29:0/1224149074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa01632d0 msgr2=0x7effa00cc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.104+0000 7eff83fff640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa01632d0 0x7effa00cc490 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7eff900ac000 tx=0x7eff90019230 comp rx=0 tx=0).stop 2024-04-25T00:31:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.104+0000 7eff83fff640 1 -- 172.21.15.29:0/1224149074 shutdown_connections 2024-04-25T00:31:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.104+0000 7eff83fff640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa01632d0 0x7effa00cc490 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.104+0000 7eff83fff640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effa0162ef0 0x7effa01599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.104+0000 7eff83fff640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff840679f0 0x7eff84069eb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.104+0000 7eff83fff640 1 --2- 172.21.15.29:0/1224149074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effa00b3180 0x7effa014df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.104+0000 7eff83fff640 1 -- 172.21.15.29:0/1224149074 >> 172.21.15.29:0/1224149074 conn(0x7effa00b1770 msgr2=0x7effa0164640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7eff83fff640 1 -- 172.21.15.29:0/1224149074 shutdown_connections 2024-04-25T00:31:40.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7fcb1a93d640 1 -- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcafc069580 msgr2=0x7fcafc06ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7fcb1a93d640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcafc069580 0x7fcafc06ba40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcb040011f0 tx=0x7fcb040010a0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7eff83fff640 1 -- 172.21.15.29:0/1224149074 wait complete. 2024-04-25T00:31:40.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7f81fdffb640 1 -- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8200067c80 msgr2=0x7f820006a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7f81fdffb640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8200067c80 0x7f820006a140 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f82080ad000 tx=0x7f8208008040 comp rx=0 tx=0).stop 2024-04-25T00:31:40.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7fcb1a93d640 1 -- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcafc005220 msgr2=0x7fcb14145230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7fcb1a93d640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcafc005220 0x7fcb14145230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009fd0 tx=0x7fcb0800fd60 comp rx=0 tx=0).stop 2024-04-25T00:31:40.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7f81fdffb640 1 -- 172.21.15.29:0/2052475724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82180b6cc0 msgr2=0x7f8218149ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7f81fdffb640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82180b6cc0 0x7f8218149ec0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f820c006e90 tx=0x7f820c00d9e0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7fcb1a93d640 1 -- 172.21.15.29:0/4252038713 shutdown_connections 2024-04-25T00:31:40.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7fcb1a93d640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb1406e720 0x7fcb1406ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7fcb1a93d640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb140b3cd0 0x7fcb1409aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7fcb1a93d640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcafc069580 0x7fcafc06ba40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7fcb1a93d640 1 --2- 172.21.15.29:0/4252038713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcafc005220 0x7fcb14145230 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7fcb1a93d640 1 -- 172.21.15.29:0/4252038713 >> 172.21.15.29:0/4252038713 conn(0x7fcb140b1910 msgr2=0x7fcb1415c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:40.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7fcb1a93d640 1 -- 172.21.15.29:0/4252038713 shutdown_connections 2024-04-25T00:31:40.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7fcb1a93d640 1 -- 172.21.15.29:0/4252038713 wait complete. 2024-04-25T00:31:40.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7f81fdffb640 1 -- 172.21.15.29:0/2052475724 shutdown_connections 2024-04-25T00:31:40.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7f81fdffb640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82180b6cc0 0x7f8218149ec0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7f81fdffb640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82180d1fd0 0x7f8218150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7f81fdffb640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8200067c80 0x7f820006a140 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7f81fdffb640 1 --2- 172.21.15.29:0/2052475724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8218070260 0x7f82180c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7f81fdffb640 1 -- 172.21.15.29:0/2052475724 >> 172.21.15.29:0/2052475724 conn(0x7f821809aa30 msgr2=0x7f82180cdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:40.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7f81fdffb640 1 -- 172.21.15.29:0/2052475724 shutdown_connections 2024-04-25T00:31:40.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.108+0000 7f81fdffb640 1 -- 172.21.15.29:0/2052475724 wait complete. 2024-04-25T00:31:40.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-04-25T00:31:40.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-04-25T00:31:40.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-04-25T00:31:40.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.144+0000 7f41441e2640 1 -- 172.21.15.29:0/2890641986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f413c0692e0 con 0x7f413c071840 2024-04-25T00:31:40.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.144+0000 7f41327fc640 1 -- 172.21.15.29:0/2890641986 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f412c0f0620 con 0x7f413c071840 2024-04-25T00:31:40.147 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-04-25T00:31:40.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.148+0000 7f41441e2640 1 -- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f41200678c0 msgr2=0x7f4120069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.148+0000 7f41441e2640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f41200678c0 0x7f4120069d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f413c15a2b0 tx=0x7f413800b220 comp rx=0 tx=0).stop 2024-04-25T00:31:40.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.148+0000 7f41441e2640 1 -- 172.21.15.29:0/2890641986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f413c071840 msgr2=0x7f413c1580f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.148+0000 7f41441e2640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f413c071840 0x7f413c1580f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f412c027060 tx=0x7f412c01d7f0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.148+0000 7f41441e2640 1 -- 172.21.15.29:0/2890641986 shutdown_connections 2024-04-25T00:31:40.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.148+0000 7f41441e2640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f41200678c0 0x7f4120069d80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.148+0000 7f41441e2640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413c1406f0 0x7f413c154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.148+0000 7f41441e2640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f413c0b33c0 0x7f413c0b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.148+0000 7f41441e2640 1 --2- 172.21.15.29:0/2890641986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f413c071840 0x7f413c1580f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.148+0000 7f41441e2640 1 -- 172.21.15.29:0/2890641986 >> 172.21.15.29:0/2890641986 conn(0x7f413c0b1870 msgr2=0x7f413c15ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:40.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.148+0000 7f41441e2640 1 -- 172.21.15.29:0/2890641986 shutdown_connections 2024-04-25T00:31:40.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.148+0000 7f41441e2640 1 -- 172.21.15.29:0/2890641986 wait complete. 2024-04-25T00:31:40.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.152+0000 7f7b91835640 1 -- 172.21.15.29:0/149693611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7b8c15cda0 con 0x7f7b8c070d40 2024-04-25T00:31:40.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.152+0000 7f7b73fff640 1 -- 172.21.15.29:0/149693611 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7b80055300 con 0x7f7b8c070d40 2024-04-25T00:31:40.154 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-04-25T00:31:40.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.156+0000 7f7b91835640 1 -- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7b78067910 msgr2=0x7f7b78069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.156+0000 7f7b91835640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7b78067910 0x7f7b78069dd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0b1000 tx=0x7f7b7c0aa070 comp rx=0 tx=0).stop 2024-04-25T00:31:40.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.156+0000 7f7b91835640 1 -- 172.21.15.29:0/149693611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b8c070d40 msgr2=0x7f7b8c15c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.156+0000 7f7b91835640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b8c070d40 0x7f7b8c15c310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000c530 tx=0x7f7b8000c9f0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.156+0000 7f7b91835640 1 -- 172.21.15.29:0/149693611 shutdown_connections 2024-04-25T00:31:40.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.156+0000 7f7b91835640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b8c169a10 0x7f7b8c16b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.156+0000 7f7b91835640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7b78067910 0x7f7b78069dd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.156+0000 7f7b91835640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b8c16e050 0x7f7b8c1694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.156+0000 7f7b91835640 1 --2- 172.21.15.29:0/149693611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b8c070d40 0x7f7b8c15c310 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.156+0000 7f7b91835640 1 -- 172.21.15.29:0/149693611 >> 172.21.15.29:0/149693611 conn(0x7f7b8c074880 msgr2=0x7f7b8c072650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:40.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.156+0000 7f7b91835640 1 -- 172.21.15.29:0/149693611 shutdown_connections 2024-04-25T00:31:40.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.156+0000 7f7b91835640 1 -- 172.21.15.29:0/149693611 wait complete. 2024-04-25T00:31:40.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.160+0000 7f1db359e640 1 -- 172.21.15.29:0/3018787793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1db40cc270 con 0x7f1db41632d0 2024-04-25T00:31:40.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.160+0000 7f1da2ffd640 1 -- 172.21.15.29:0/3018787793 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1da80568c0 con 0x7f1db41632d0 2024-04-25T00:31:40.164 INFO:teuthology.orchestra.run.smithi029.stdout:55834574853 2024-04-25T00:31:40.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-04-25T00:31:40.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc9759e640 1 -- 172.21.15.29:0/3470479998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdc7c005860 con 0x7fdc9806ddd0 2024-04-25T00:31:40.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc86ffd640 1 -- 172.21.15.29:0/3470479998 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdc88056060 con 0x7fdc9806ddd0 2024-04-25T00:31:40.166 INFO:teuthology.orchestra.run.smithi029.stdout:55834574853 2024-04-25T00:31:40.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7f1db359e640 1 -- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d940678f0 msgr2=0x7f1d94069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7f1db359e640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d940678f0 0x7f1d94069db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0ae000 tx=0x7f1dac0a7070 comp rx=0 tx=0).stop 2024-04-25T00:31:40.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7f1db359e640 1 -- 172.21.15.29:0/3018787793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db41632d0 msgr2=0x7f1db413d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7f1db359e640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db41632d0 0x7f1db413d600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1da800eab0 tx=0x7f1da800ef70 comp rx=0 tx=0).stop 2024-04-25T00:31:40.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7f1db359e640 1 -- 172.21.15.29:0/3018787793 shutdown_connections 2024-04-25T00:31:40.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7f1db359e640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1db41632d0 0x7f1db413d600 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7f1db359e640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1db4162ef0 0x7f1db4173290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7f1db359e640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d940678f0 0x7f1d94069db0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7f1db359e640 1 --2- 172.21.15.29:0/3018787793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db40b2fb0 0x7f1db414d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7f1db359e640 1 -- 172.21.15.29:0/3018787793 >> 172.21.15.29:0/3018787793 conn(0x7f1db40b1770 msgr2=0x7f1db4165ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:40.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7f1db359e640 1 -- 172.21.15.29:0/3018787793 shutdown_connections 2024-04-25T00:31:40.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7f1db359e640 1 -- 172.21.15.29:0/3018787793 wait complete. 2024-04-25T00:31:40.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc9759e640 1 -- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc78067940 msgr2=0x7fdc78069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc9759e640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc78067940 0x7fdc78069e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0ad000 tx=0x7fdc8c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:31:40.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc9759e640 1 -- 172.21.15.29:0/3470479998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc9806ddd0 msgr2=0x7fdc981433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc9759e640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc9806ddd0 0x7fdc981433c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800c7b0 tx=0x7fdc8800cbb0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc9759e640 1 -- 172.21.15.29:0/3470479998 shutdown_connections 2024-04-25T00:31:40.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc9759e640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc98138f30 0x7fdc98150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc9759e640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc78067940 0x7fdc78069e00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc9759e640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc9815b380 0x7fdc981461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc9759e640 1 --2- 172.21.15.29:0/3470479998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc9806ddd0 0x7fdc981433c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc9759e640 1 -- 172.21.15.29:0/3470479998 >> 172.21.15.29:0/3470479998 conn(0x7fdc9806b210 msgr2=0x7fdc9806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:40.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc9759e640 1 -- 172.21.15.29:0/3470479998 shutdown_connections 2024-04-25T00:31:40.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.164+0000 7fdc9759e640 1 -- 172.21.15.29:0/3470479998 wait complete. 2024-04-25T00:31:40.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.172+0000 7fd4aa097640 1 -- 172.21.15.29:0/733255197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd484005590 con 0x7fd4a40b4a30 2024-04-25T00:31:40.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.172+0000 7fd48bfff640 1 -- 172.21.15.29:0/733255197 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd49c056be0 con 0x7fd4a40b4a30 2024-04-25T00:31:40.174 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-04-25T00:31:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.176+0000 7fd4aa097640 1 -- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd4900678c0 msgr2=0x7fd490069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.176+0000 7fd4aa097640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd4900678c0 0x7fd490069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd4980ad000 tx=0x7fd498008040 comp rx=0 tx=0).stop 2024-04-25T00:31:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.176+0000 7fd4aa097640 1 -- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a40b4a30 msgr2=0x7fd4a40ab210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.176+0000 7fd4aa097640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a40b4a30 0x7fd4a40ab210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00efc0 tx=0x7fd49c00c490 comp rx=0 tx=0).stop 2024-04-25T00:31:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.176+0000 7fd4aa097640 1 -- 172.21.15.29:0/733255197 shutdown_connections 2024-04-25T00:31:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.176+0000 7fd4aa097640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4a4146c70 0x7fd4a4147370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.176+0000 7fd4aa097640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd4900678c0 0x7fd490069d80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.176+0000 7fd4aa097640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a40b4a30 0x7fd4a40ab210 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.176+0000 7fd4aa097640 1 --2- 172.21.15.29:0/733255197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4a40b2c90 0x7fd4a41457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.176+0000 7fd4aa097640 1 -- 172.21.15.29:0/733255197 >> 172.21.15.29:0/733255197 conn(0x7fd4a4074880 msgr2=0x7fd4a40b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.176+0000 7fd4aa097640 1 -- 172.21.15.29:0/733255197 shutdown_connections 2024-04-25T00:31:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:40.176+0000 7fd4aa097640 1 -- 172.21.15.29:0/733255197 wait complete. 2024-04-25T00:31:40.183 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-04-25T00:31:40.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-04-25T00:31:40.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-04-25T00:31:40.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-04-25T00:31:41.140 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T00:31:41.141 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T00:31:41.141 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T00:31:41.167 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T00:31:41.184 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T00:31:41.188 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T00:31:41.194 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T00:31:41.200 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T00:31:41.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6f0a6640 1 Processor -- start 2024-04-25T00:31:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6f0a6640 1 -- start start 2024-04-25T00:31:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6f0a6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e68151020 0x7f2e680cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6f0a6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e680af020 0x7f2e6815a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6f0a6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e6815de10 con 0x7f2e680cfeb0 2024-04-25T00:31:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6f0a6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2e6815e740 con 0x7f2e680af020 2024-04-25T00:31:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6f0a6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6815e0f0 con 0x7f2e68151020 2024-04-25T00:31:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6ce42640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e68151020 0x7f2e680cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6ce42640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e68151020 0x7f2e680cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:58720/0 (socket says 172.21.15.29:58720) 2024-04-25T00:31:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6ce42640 1 -- 172.21.15.29:0/2712290419 learned_addr learned my addr 172.21.15.29:0/2712290419 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6d643640 1 --2- 172.21.15.29:0/2712290419 >> v2:172.21.15.157:3300/0 conn(0x7f2e680af020 0x7f2e6815a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6ce42640 1 -- 172.21.15.29:0/2712290419 >> v2:172.21.15.157:3300/0 conn(0x7f2e680af020 msgr2=0x7f2e6815a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6ce42640 1 --2- 172.21.15.29:0/2712290419 >> v2:172.21.15.157:3300/0 conn(0x7f2e680af020 0x7f2e6815a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6ce42640 1 -- 172.21.15.29:0/2712290419 >> v1:172.21.15.157:6789/0 conn(0x7f2e680cfeb0 legacy=0x7f2e680ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6ce42640 1 -- 172.21.15.29:0/2712290419 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6815e3d0 con 0x7f2e68151020 2024-04-25T00:31:41.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.300+0000 7f2e6d643640 1 --2- 172.21.15.29:0/2712290419 >> v2:172.21.15.157:3300/0 conn(0x7f2e680af020 0x7f2e6815a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:41.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ff5c640 1 Processor -- start 2024-04-25T00:31:41.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ff5c640 1 -- start start 2024-04-25T00:31:41.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ff5c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66580b47b0 0x7f66580cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ff5c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6658166770 con 0x7f665814bdc0 2024-04-25T00:31:41.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ff5c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6658166a50 con 0x7f66580d00b0 2024-04-25T00:31:41.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ff5c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f66580b4c10 con 0x7f66580b47b0 2024-04-25T00:31:41.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665dcf8640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f665814bdc0 0x7f6658165f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50060/0 (socket says 172.21.15.29:50060) 2024-04-25T00:31:41.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665dcf8640 1 -- 172.21.15.29:0/2256911517 learned_addr learned my addr 172.21.15.29:0/2256911517 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:41.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665d4f7640 1 --2- 172.21.15.29:0/2256911517 >> v2:172.21.15.157:3300/0 conn(0x7f66580b47b0 0x7f66580cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1908505551 0 0) 0x7f6658166770 con 0x7f665814bdc0 2024-04-25T00:31:41.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6644003530 con 0x7f665814bdc0 2024-04-25T00:31:41.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f2e6ce42640 1 --2- 172.21.15.29:0/2712290419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e68151020 0x7f2e680cf970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2e58008870 tx=0x7f2e5801ea70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7b93696427c492f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1470780719 0 0) 0x7f6658166a50 con 0x7f66580d00b0 2024-04-25T00:31:41.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6658166770 con 0x7f66580d00b0 2024-04-25T00:31:41.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3172640888 0 0) 0x7f6644003530 con 0x7f665814bdc0 2024-04-25T00:31:41.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 >> v2:172.21.15.157:3300/0 conn(0x7f66580b47b0 msgr2=0x7f66580cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 --2- 172.21.15.29:0/2256911517 >> v2:172.21.15.157:3300/0 conn(0x7f66580b47b0 0x7f66580cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 >> v1:172.21.15.157:6789/0 conn(0x7f66580d00b0 legacy=0x7f6658071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6658166d30 con 0x7f665814bdc0 2024-04-25T00:31:41.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f2e677fe640 1 -- 172.21.15.29:0/2712290419 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e580ad070 con 0x7f2e68151020 2024-04-25T00:31:41.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f2e677fe640 1 -- 172.21.15.29:0/2712290419 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e580a7940 con 0x7f2e68151020 2024-04-25T00:31:41.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f2e677fe640 1 -- 172.21.15.29:0/2712290419 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e580a7d10 con 0x7f2e68151020 2024-04-25T00:31:41.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f6650001980 con 0x7f665814bdc0 2024-04-25T00:31:41.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 >> v1:172.21.15.29:6789/0 conn(0x7f665814bdc0 legacy=0x7f6658165f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 --2- 172.21.15.29:0/2256911517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6644003ba0 0x7f6644003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 --2- 172.21.15.29:0/2256911517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6644004810 0x7f6644004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 --2- 172.21.15.29:0/2256911517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66440051d0 0x7f664400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f66580b4c10 con 0x7f66440051d0 2024-04-25T00:31:41.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f664400c020 con 0x7f6644004810 2024-04-25T00:31:41.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f664400c220 con 0x7f6644003ba0 2024-04-25T00:31:41.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.304+0000 7f665dcf8640 1 --2- 172.21.15.29:0/2256911517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6644003ba0 0x7f6644003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/2712290419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e68151020 msgr2=0x7f2e680cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f2e6f0a6640 1 --2- 172.21.15.29:0/2712290419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e68151020 0x7f2e680cf970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2e58008870 tx=0x7f2e5801ea70 comp rx=0 tx=0).stop 2024-04-25T00:31:41.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f665e4f9640 1 --2- 172.21.15.29:0/2256911517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66440051d0 0x7f664400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f665d4f7640 1 --2- 172.21.15.29:0/2256911517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6644004810 0x7f6644004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/2712290419 shutdown_connections 2024-04-25T00:31:41.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f2e6f0a6640 1 --2- 172.21.15.29:0/2712290419 >> v2:172.21.15.157:3300/0 conn(0x7f2e680af020 0x7f2e6815a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f2e6f0a6640 1 --2- 172.21.15.29:0/2712290419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e68151020 0x7f2e680cf970 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/2712290419 >> 172.21.15.29:0/2712290419 conn(0x7f2e68074880 msgr2=0x7f2e6806b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/2712290419 shutdown_connections 2024-04-25T00:31:41.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/2712290419 wait complete. 2024-04-25T00:31:41.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f665dcf8640 1 -- 172.21.15.29:0/2256911517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6644004810 msgr2=0x7f6644004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f665dcf8640 1 --2- 172.21.15.29:0/2256911517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6644004810 0x7f6644004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f665dcf8640 1 -- 172.21.15.29:0/2256911517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66440051d0 msgr2=0x7f664400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f665dcf8640 1 --2- 172.21.15.29:0/2256911517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66440051d0 0x7f664400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f665dcf8640 1 -- 172.21.15.29:0/2256911517 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6644003530 con 0x7f6644003ba0 2024-04-25T00:31:41.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f665e4f9640 1 --2- 172.21.15.29:0/2256911517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66440051d0 0x7f664400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:41.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f665e4f9640 1 -- 172.21.15.29:0/2256911517 reap_dead start 2024-04-25T00:31:41.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f2e6f0a6640 1 Processor -- start 2024-04-25T00:31:41.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f2e6f0a6640 1 -- start start 2024-04-25T00:31:41.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f2e6f0a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e680af020 0x7f2e681474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f2e6f0a6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e680cfeb0 0x7f2e681516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.308+0000 7f2e6f0a6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e68151020 0x7f2e6815cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6f0a6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2e68166d90 con 0x7f2e68151020 2024-04-25T00:31:41.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6f0a6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e68166f10 con 0x7f2e680af020 2024-04-25T00:31:41.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6ce42640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e680af020 0x7f2e681474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6ce42640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e680af020 0x7f2e681474d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:60866/0 (socket says 172.21.15.29:60866) 2024-04-25T00:31:41.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6ce42640 1 -- 172.21.15.29:0/1386757799 learned_addr learned my addr 172.21.15.29:0/1386757799 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e67fff640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e680cfeb0 0x7f2e681516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6d643640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e68151020 0x7f2e6815cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f665dcf8640 1 --2- 172.21.15.29:0/2256911517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6644003ba0 0x7f6644003f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f66500058b0 tx=0x7f665001ff50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb54f2b5fec50e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/1386757799 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2e68167090 con 0x7f2e680cfeb0 2024-04-25T00:31:41.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7ff125bbf640 1 Processor -- start 2024-04-25T00:31:41.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500054d0 con 0x7f6644003ba0 2024-04-25T00:31:41.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7ff125bbf640 1 -- start start 2024-04-25T00:31:41.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66500ad030 con 0x7f6644003ba0 2024-04-25T00:31:41.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f665ccf6640 1 -- 172.21.15.29:0/2256911517 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500a9dd0 con 0x7f6644003ba0 2024-04-25T00:31:41.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6d643640 1 -- 172.21.15.29:0/1386757799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e680cfeb0 msgr2=0x7f2e681516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6d643640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e680cfeb0 0x7f2e681516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6d643640 1 -- 172.21.15.29:0/1386757799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e680af020 msgr2=0x7f2e681474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6d643640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e680af020 0x7f2e681474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6d643640 1 -- 172.21.15.29:0/1386757799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e680b3380 con 0x7f2e68151020 2024-04-25T00:31:41.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6d643640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e68151020 0x7f2e6815cdb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2e680af4f0 tx=0x7f2e5c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f665ff5c640 1 -- 172.21.15.29:0/2256911517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6644003ba0 msgr2=0x7f6644003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f665ff5c640 1 --2- 172.21.15.29:0/2256911517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6644003ba0 0x7f6644003f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f66500058b0 tx=0x7f665001ff50 comp rx=0 tx=0).stop 2024-04-25T00:31:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f665ff5c640 1 -- 172.21.15.29:0/2256911517 shutdown_connections 2024-04-25T00:31:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f665ff5c640 1 --2- 172.21.15.29:0/2256911517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6644003ba0 0x7f6644003f80 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f66500058b0 tx=0x7f665001ff50 comp rx=0 tx=0).stop 2024-04-25T00:31:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f665ff5c640 1 -- 172.21.15.29:0/2256911517 >> 172.21.15.29:0/2256911517 conn(0x7f665806b3e0 msgr2=0x7f665806b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f665ff5c640 1 -- 172.21.15.29:0/2256911517 shutdown_connections 2024-04-25T00:31:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f665ff5c640 1 -- 172.21.15.29:0/2256911517 wait complete. 2024-04-25T00:31:41.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e657fa640 1 -- 172.21.15.29:0/1386757799 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c017070 con 0x7f2e68151020 2024-04-25T00:31:41.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e657fa640 1 -- 172.21.15.29:0/1386757799 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5c00c940 con 0x7f2e68151020 2024-04-25T00:31:41.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e657fa640 1 -- 172.21.15.29:0/1386757799 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c00eea0 con 0x7f2e68151020 2024-04-25T00:31:41.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/1386757799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6814b5e0 con 0x7f2e68151020 2024-04-25T00:31:41.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/1386757799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6815c5b0 con 0x7f2e68151020 2024-04-25T00:31:41.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.312+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/1386757799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e68069200 con 0x7f2e68151020 2024-04-25T00:31:41.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665ff5c640 1 Processor -- start 2024-04-25T00:31:41.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665ff5c640 1 -- start start 2024-04-25T00:31:41.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f2e657fa640 1 -- 172.21.15.29:0/1386757799 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2e5c01a070 con 0x7f2e68151020 2024-04-25T00:31:41.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665ff5c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66581438a0 0x7f6658163ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665ff5c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6658143c80 0x7f6658164530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665ff5c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66581441a0 0x7f6658144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665dcf8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66581438a0 0x7f6658163ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665ff5c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f66580cfb70 con 0x7f66581438a0 2024-04-25T00:31:41.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665dcf8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66581438a0 0x7f6658163ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49914/0 (socket says 172.21.15.29:49914) 2024-04-25T00:31:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665dcf8640 1 -- 172.21.15.29:0/3081283154 learned_addr learned my addr 172.21.15.29:0/3081283154 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665ff5c640 1 -- 172.21.15.29:0/3081283154 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6658070e10 con 0x7f6658143c80 2024-04-25T00:31:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665ff5c640 1 -- 172.21.15.29:0/3081283154 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f66580cfcf0 con 0x7f66581441a0 2024-04-25T00:31:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665d4f7640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6658143c80 0x7f6658164530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665e4f9640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66581441a0 0x7f6658144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f2e657fa640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2e54067990 0x7f2e54069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665dcf8640 1 -- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66581441a0 msgr2=0x7f6658144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665dcf8640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66581441a0 0x7f6658144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665dcf8640 1 -- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6658143c80 msgr2=0x7f6658164530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665dcf8640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6658143c80 0x7f6658164530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f665dcf8640 1 -- 172.21.15.29:0/3081283154 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66580a9490 con 0x7f66581438a0 2024-04-25T00:31:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f2e657fa640 1 -- 172.21.15.29:0/1386757799 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2e5c086160 con 0x7f2e68151020 2024-04-25T00:31:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7ff125bbf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff120145840 0x7ff12015c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.316+0000 7f2e6ce42640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2e54067990 0x7f2e54069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7f665dcf8640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66581438a0 0x7f6658163ff0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f66500058b0 tx=0x7f6644005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff125bbf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff12006e510 0x7ff12015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff125bbf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff12015cfb0 0x7ff1200ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff11f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff120145840 0x7ff12015c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff125bbf640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff1200b2f30 con 0x7ff12015cfb0 2024-04-25T00:31:41.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff125bbf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff12015d410 con 0x7ff12006e510 2024-04-25T00:31:41.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff125bbf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff1200cf4b0 con 0x7ff120145840 2024-04-25T00:31:41.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff11effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff12006e510 0x7ff12015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff11f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff120145840 0x7ff12015c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:58746/0 (socket says 172.21.15.29:58746) 2024-04-25T00:31:41.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff11f7fe640 1 -- 172.21.15.29:0/1531474828 learned_addr learned my addr 172.21.15.29:0/1531474828 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff11ffff640 1 --2- 172.21.15.29:0/1531474828 >> v2:172.21.15.29:3300/0 conn(0x7ff12015cfb0 0x7ff1200ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7f2e6ce42640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2e54067990 0x7f2e54069e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e58000c10 tx=0x7f2e5800a040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff11effd640 1 -- 172.21.15.29:0/1531474828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff120145840 msgr2=0x7ff12015c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff11effd640 1 --2- 172.21.15.29:0/1531474828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff120145840 0x7ff12015c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff11effd640 1 -- 172.21.15.29:0/1531474828 >> v2:172.21.15.29:3300/0 conn(0x7ff12015cfb0 msgr2=0x7ff1200ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff11effd640 1 --2- 172.21.15.29:0/1531474828 >> v2:172.21.15.29:3300/0 conn(0x7ff12015cfb0 0x7ff1200ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.320+0000 7ff11effd640 1 -- 172.21.15.29:0/1531474828 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12006d750 con 0x7ff12006e510 2024-04-25T00:31:41.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7ff11effd640 1 --2- 172.21.15.29:0/1531474828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff12006e510 0x7ff12015ca70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff1200b5e30 tx=0x7ff10c01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8528b510526ddac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7ff11e7fc640 1 -- 172.21.15.29:0/1531474828 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0b5070 con 0x7ff12006e510 2024-04-25T00:31:41.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7ff11e7fc640 1 -- 172.21.15.29:0/1531474828 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff10c0a8930 con 0x7ff12006e510 2024-04-25T00:31:41.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7ff11e7fc640 1 -- 172.21.15.29:0/1531474828 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0a8d00 con 0x7ff12006e510 2024-04-25T00:31:41.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7f2e657fa640 1 -- 172.21.15.29:0/1386757799 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2e5c0566c0 con 0x7f2e68151020 2024-04-25T00:31:41.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7ff125bbf640 1 -- 172.21.15.29:0/1531474828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff12006e510 msgr2=0x7ff12015ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7ff125bbf640 1 --2- 172.21.15.29:0/1531474828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff12006e510 0x7ff12015ca70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff1200b5e30 tx=0x7ff10c01ec70 comp rx=0 tx=0).stop 2024-04-25T00:31:41.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7ff125bbf640 1 -- 172.21.15.29:0/1531474828 shutdown_connections 2024-04-25T00:31:41.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7ff125bbf640 1 --2- 172.21.15.29:0/1531474828 >> v2:172.21.15.29:3300/0 conn(0x7ff12015cfb0 0x7ff1200ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7ff125bbf640 1 --2- 172.21.15.29:0/1531474828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff12006e510 0x7ff12015ca70 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff1200b5e30 tx=0x7ff10c01ec70 comp rx=0 tx=0).stop 2024-04-25T00:31:41.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7ff125bbf640 1 --2- 172.21.15.29:0/1531474828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff120145840 0x7ff12015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7ff125bbf640 1 -- 172.21.15.29:0/1531474828 >> 172.21.15.29:0/1531474828 conn(0x7ff1200b1870 msgr2=0x7ff1200b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7f664e7fc640 1 -- 172.21.15.29:0/3081283154 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650001da0 con 0x7f66581438a0 2024-04-25T00:31:41.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7f664e7fc640 1 -- 172.21.15.29:0/3081283154 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6650001f40 con 0x7f66581438a0 2024-04-25T00:31:41.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7f665ff5c640 1 -- 172.21.15.29:0/3081283154 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6658141640 con 0x7f66581438a0 2024-04-25T00:31:41.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7f665ff5c640 1 -- 172.21.15.29:0/3081283154 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6658141c00 con 0x7f66581438a0 2024-04-25T00:31:41.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.324+0000 7f665ff5c640 1 -- 172.21.15.29:0/3081283154 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6658069200 con 0x7f66581438a0 2024-04-25T00:31:41.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.328+0000 7f664e7fc640 1 -- 172.21.15.29:0/3081283154 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500a9b90 con 0x7f66581438a0 2024-04-25T00:31:41.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.328+0000 7f664e7fc640 1 -- 172.21.15.29:0/3081283154 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f66500be450 con 0x7f66581438a0 2024-04-25T00:31:41.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.328+0000 7f664e7fc640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f663c0679c0 0x7f663c069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.328+0000 7f664e7fc640 1 -- 172.21.15.29:0/3081283154 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6650123f70 con 0x7f66581438a0 2024-04-25T00:31:41.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.328+0000 7f665d4f7640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f663c0679c0 0x7f663c069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.328+0000 7f665d4f7640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f663c0679c0 0x7f663c069e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f66580add60 tx=0x7f6654009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.332+0000 7ff125bbf640 1 -- 172.21.15.29:0/1531474828 shutdown_connections 2024-04-25T00:31:41.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.332+0000 7ff125bbf640 1 -- 172.21.15.29:0/1531474828 wait complete. 2024-04-25T00:31:41.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.332+0000 7ff125bbf640 1 Processor -- start 2024-04-25T00:31:41.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.332+0000 7f664e7fc640 1 -- 172.21.15.29:0/3081283154 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f66500c3060 con 0x7f66581438a0 2024-04-25T00:31:41.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.332+0000 7ff125bbf640 1 -- start start 2024-04-25T00:31:41.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.332+0000 7ff125bbf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff120145840 0x7ff1201e2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.332+0000 7ff125bbf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff12015cfb0 0x7ff1201395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff11f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff120145840 0x7ff1201e2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff11f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff120145840 0x7ff1201e2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:58758/0 (socket says 172.21.15.29:58758) 2024-04-25T00:31:41.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff11f7fe640 1 -- 172.21.15.29:0/1182117518 learned_addr learned my addr 172.21.15.29:0/1182117518 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff11effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff12015cfb0 0x7ff1201395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff125bbf640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff120157ae0 0x7ff12015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff125bbf640 1 -- 172.21.15.29:0/1182117518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff12006dff0 con 0x7ff12015cfb0 2024-04-25T00:31:41.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff125bbf640 1 -- 172.21.15.29:0/1182117518 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff12006dd10 con 0x7ff120157ae0 2024-04-25T00:31:41.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff125bbf640 1 -- 172.21.15.29:0/1182117518 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff1200ce730 con 0x7ff120145840 2024-04-25T00:31:41.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff11ffff640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff120157ae0 0x7ff12015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff11effd640 1 -- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff120145840 msgr2=0x7ff1201e2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff11effd640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff120145840 0x7ff1201e2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff11effd640 1 -- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff120157ae0 msgr2=0x7ff12015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff11effd640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff120157ae0 0x7ff12015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff11effd640 1 -- 172.21.15.29:0/1182117518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1200a9490 con 0x7ff12015cfb0 2024-04-25T00:31:41.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff11ffff640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff120157ae0 0x7ff12015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:41.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff11effd640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff12015cfb0 0x7ff1201395f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff12015d590 tx=0x7ff10c01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff10bfff640 1 -- 172.21.15.29:0/1182117518 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0bf070 con 0x7ff12015cfb0 2024-04-25T00:31:41.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff125bbf640 1 -- 172.21.15.29:0/1182117518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1201e2b80 con 0x7ff12015cfb0 2024-04-25T00:31:41.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff125bbf640 1 -- 172.21.15.29:0/1182117518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1201e3140 con 0x7ff12015cfb0 2024-04-25T00:31:41.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.336+0000 7ff10bfff640 1 -- 172.21.15.29:0/1182117518 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff10c0a8830 con 0x7ff12015cfb0 2024-04-25T00:31:41.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.340+0000 7ff10bfff640 1 -- 172.21.15.29:0/1182117518 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0b5040 con 0x7ff12015cfb0 2024-04-25T00:31:41.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.340+0000 7ff10bfff640 1 -- 172.21.15.29:0/1182117518 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff10c0c2070 con 0x7ff12015cfb0 2024-04-25T00:31:41.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.340+0000 7ff10bfff640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1040678c0 0x7ff104069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.344+0000 7ff11f7fe640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1040678c0 0x7ff104069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.344+0000 7ff10bfff640 1 -- 172.21.15.29:0/1182117518 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff10c0f08f0 con 0x7ff12015cfb0 2024-04-25T00:31:41.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.344+0000 7ff125bbf640 1 -- 172.21.15.29:0/1182117518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1000052a0 con 0x7ff12015cfb0 2024-04-25T00:31:41.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.344+0000 7ff11f7fe640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1040678c0 0x7ff104069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff114009bd0 tx=0x7ff114009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.352+0000 7ff10bfff640 1 -- 172.21.15.29:0/1182117518 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff10c0187d0 con 0x7ff12015cfb0 2024-04-25T00:31:41.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.352+0000 7f43eac3a640 1 Processor -- start 2024-04-25T00:31:41.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.352+0000 7f43eac3a640 1 -- start start 2024-04-25T00:31:41.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43eac3a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43e41438f0 0x7f43e415db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43eac3a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43e406de90 0x7f43e40710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43eac3a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43e40b3510 con 0x7f43e415e0b0 2024-04-25T00:31:41.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43eac3a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f43e4143cd0 con 0x7f43e41438f0 2024-04-25T00:31:41.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43eac3a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f43e406f5b0 con 0x7f43e406de90 2024-04-25T00:31:41.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e9437640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f43e415e0b0 0x7f43e406d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:44052/0 (socket says 172.21.15.29:44052) 2024-04-25T00:31:41.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e9437640 1 -- 172.21.15.29:0/2102591656 learned_addr learned my addr 172.21.15.29:0/2102591656 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:41.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1518698451 0 0) 0x7f43e40b3510 con 0x7f43e415e0b0 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43d0003530 con 0x7f43e415e0b0 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2291113558 0 0) 0x7f43d0003530 con 0x7f43e415e0b0 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43e406de90 msgr2=0x7f43e40710d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 --2- 172.21.15.29:0/2102591656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43e406de90 0x7f43e40710d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 >> v2:172.21.15.157:3300/0 conn(0x7f43e41438f0 msgr2=0x7f43e415db70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 --2- 172.21.15.29:0/2102591656 >> v2:172.21.15.157:3300/0 conn(0x7f43e41438f0 0x7f43e415db70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e40b37f0 con 0x7f43e415e0b0 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f43dc001a00 con 0x7f43e415e0b0 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 >> v1:172.21.15.157:6789/0 conn(0x7f43e415e0b0 legacy=0x7f43e406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 --2- 172.21.15.29:0/2102591656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43d00039c0 0x7f43d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 --2- 172.21.15.29:0/2102591656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43d0004810 0x7f43d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 --2- 172.21.15.29:0/2102591656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d00052f0 0x7f43d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4143cd0 con 0x7f43d00039c0 2024-04-25T00:31:41.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f43e406f5b0 con 0x7f43d00052f0 2024-04-25T00:31:41.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.356+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f43d000c220 con 0x7f43d0004810 2024-04-25T00:31:41.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.360+0000 7f43e9437640 1 --2- 172.21.15.29:0/2102591656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43d0004810 0x7f43d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.360+0000 7f43ea439640 1 --2- 172.21.15.29:0/2102591656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d00052f0 0x7f43d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.360+0000 7f43e9c38640 1 --2- 172.21.15.29:0/2102591656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43d00039c0 0x7f43d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.360+0000 7f43e9437640 1 -- 172.21.15.29:0/2102591656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d00052f0 msgr2=0x7f43d000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.360+0000 7f43e9437640 1 --2- 172.21.15.29:0/2102591656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d00052f0 0x7f43d000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.360+0000 7f43e9437640 1 -- 172.21.15.29:0/2102591656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43d00039c0 msgr2=0x7f43d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.360+0000 7f43e9437640 1 --2- 172.21.15.29:0/2102591656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43d00039c0 0x7f43d0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.360+0000 7f43e9437640 1 -- 172.21.15.29:0/2102591656 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d0003530 con 0x7f43d0004810 2024-04-25T00:31:41.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.360+0000 7f43e9c38640 1 --2- 172.21.15.29:0/2102591656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43d00039c0 0x7f43d0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:41.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.364+0000 7f43e9437640 1 --2- 172.21.15.29:0/2102591656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43d0004810 0x7f43d0004c90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f43dc005040 tx=0x7f43dc021510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c24d7da3d1cfc3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.364+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00d040 con 0x7f43d0004810 2024-04-25T00:31:41.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.364+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43dc008270 con 0x7f43d0004810 2024-04-25T00:31:41.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.364+0000 7f43e8c36640 1 -- 172.21.15.29:0/2102591656 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc006730 con 0x7f43d0004810 2024-04-25T00:31:41.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.364+0000 7f43eac3a640 1 -- 172.21.15.29:0/2102591656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43d0004810 msgr2=0x7f43d0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.364+0000 7f43eac3a640 1 --2- 172.21.15.29:0/2102591656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43d0004810 0x7f43d0004c90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f43dc005040 tx=0x7f43dc021510 comp rx=0 tx=0).stop 2024-04-25T00:31:41.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.364+0000 7f43ea439640 1 -- 172.21.15.29:0/2102591656 reap_dead start 2024-04-25T00:31:41.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.364+0000 7f43eac3a640 1 -- 172.21.15.29:0/2102591656 shutdown_connections 2024-04-25T00:31:41.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.364+0000 7f43eac3a640 1 -- 172.21.15.29:0/2102591656 >> 172.21.15.29:0/2102591656 conn(0x7f43e40b1790 msgr2=0x7f43e40b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.364+0000 7f43eac3a640 1 -- 172.21.15.29:0/2102591656 shutdown_connections 2024-04-25T00:31:41.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.364+0000 7f43eac3a640 1 -- 172.21.15.29:0/2102591656 wait complete. 2024-04-25T00:31:41.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43eac3a640 1 Processor -- start 2024-04-25T00:31:41.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43eac3a640 1 -- start start 2024-04-25T00:31:41.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43eac3a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d0004810 0x7f43e4141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43eac3a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43e42072a0 0x7f43e413b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43eac3a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43e41f6700 0x7f43e41f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f0890f80640 1 Processor -- start 2024-04-25T00:31:41.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43ea439640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43e41f6700 0x7f43e41f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43ea439640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43e41f6700 0x7f43e41f6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49948/0 (socket says 172.21.15.29:49948) 2024-04-25T00:31:41.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43e9c38640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d0004810 0x7f43e4141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43e9c38640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d0004810 0x7f43e4141f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:60916/0 (socket says 172.21.15.29:60916) 2024-04-25T00:31:41.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43e9437640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43e42072a0 0x7f43e413b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43e9437640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43e42072a0 0x7f43e413b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:58790/0 (socket says 172.21.15.29:58790) 2024-04-25T00:31:41.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43ea439640 1 -- 172.21.15.29:0/1948763161 learned_addr learned my addr 172.21.15.29:0/1948763161 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43eac3a640 1 -- 172.21.15.29:0/1948763161 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f43e40718e0 con 0x7f43e41f6700 2024-04-25T00:31:41.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43eac3a640 1 -- 172.21.15.29:0/1948763161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f43e40b45b0 con 0x7f43d0004810 2024-04-25T00:31:41.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43eac3a640 1 -- 172.21.15.29:0/1948763161 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f43e4071a60 con 0x7f43e42072a0 2024-04-25T00:31:41.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43ea439640 1 -- 172.21.15.29:0/1948763161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43e42072a0 msgr2=0x7f43e413b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43ea439640 1 --2- 172.21.15.29:0/1948763161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43e42072a0 0x7f43e413b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43ea439640 1 -- 172.21.15.29:0/1948763161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d0004810 msgr2=0x7f43e4141f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43ea439640 1 --2- 172.21.15.29:0/1948763161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d0004810 0x7f43e4141f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f43ea439640 1 -- 172.21.15.29:0/1948763161 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e4207a40 con 0x7f43e41f6700 2024-04-25T00:31:41.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f0890f80640 1 -- start start 2024-04-25T00:31:41.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f0890f80640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f088c1624b0 0x7f088c0c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f0890f80640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f088c0cee60 0x7f088c0c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f0890f80640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f088c166de0 con 0x7f088c06ce40 2024-04-25T00:31:41.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f0890f80640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f088c1668b0 con 0x7f088c0cee60 2024-04-25T00:31:41.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f0890f80640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f088c06ef50 con 0x7f088c1624b0 2024-04-25T00:31:41.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f088affd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f088c06ce40 0x7f088c0d1b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50064/0 (socket says 172.21.15.29:50064) 2024-04-25T00:31:41.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f088affd640 1 -- 172.21.15.29:0/3059723354 learned_addr learned my addr 172.21.15.29:0/3059723354 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:41.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f088b7fe640 1 --2- 172.21.15.29:0/3059723354 >> v2:172.21.15.157:3300/0 conn(0x7f088c1624b0 0x7f088c0c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.368+0000 7f088bfff640 1 --2- 172.21.15.29:0/3059723354 >> v2:172.21.15.29:3300/0 conn(0x7f088c0cee60 0x7f088c0c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.372+0000 7f088b7fe640 1 -- 172.21.15.29:0/3059723354 >> v2:172.21.15.29:3300/0 conn(0x7f088c0cee60 msgr2=0x7f088c0c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.372+0000 7f088b7fe640 1 --2- 172.21.15.29:0/3059723354 >> v2:172.21.15.29:3300/0 conn(0x7f088c0cee60 0x7f088c0c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.372+0000 7f43ea439640 1 --2- 172.21.15.29:0/1948763161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43e41f6700 0x7f43e41f6b80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f43e406e270 tx=0x7f43d400e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.372+0000 7f088b7fe640 1 -- 172.21.15.29:0/3059723354 >> v1:172.21.15.29:6789/0 conn(0x7f088c06ce40 legacy=0x7f088c0d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.372+0000 7f088b7fe640 1 -- 172.21.15.29:0/3059723354 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c070cd0 con 0x7f088c1624b0 2024-04-25T00:31:41.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.372+0000 7f43da7fc640 1 -- 172.21.15.29:0/1948763161 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e00093d0 con 0x7f43e41f6700 2024-04-25T00:31:41.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.372+0000 7f43da7fc640 1 -- 172.21.15.29:0/1948763161 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43d401a070 con 0x7f43e41f6700 2024-04-25T00:31:41.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.372+0000 7f43da7fc640 1 -- 172.21.15.29:0/1948763161 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4013880 con 0x7f43e41f6700 2024-04-25T00:31:41.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.372+0000 7f43eac3a640 1 -- 172.21.15.29:0/1948763161 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e4207d20 con 0x7f43e41f6700 2024-04-25T00:31:41.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.372+0000 7f43eac3a640 1 -- 172.21.15.29:0/1948763161 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e40cc0f0 con 0x7f43e41f6700 2024-04-25T00:31:41.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.372+0000 7f088bfff640 1 --2- 172.21.15.29:0/3059723354 >> v2:172.21.15.29:3300/0 conn(0x7f088c0cee60 0x7f088c0c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:41.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.372+0000 7f088b7fe640 1 --2- 172.21.15.29:0/3059723354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c1624b0 0x7f088c0c4bd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f088c0b47f0 tx=0x7f087c01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9533eb99b2944d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f43da7fc640 1 -- 172.21.15.29:0/1948763161 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f43d4013a20 con 0x7f43e41f6700 2024-04-25T00:31:41.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f43da7fc640 1 --2- 172.21.15.29:0/1948763161 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f43c80679c0 0x7f43c8069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f43da7fc640 1 -- 172.21.15.29:0/1948763161 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f43d4085440 con 0x7f43e41f6700 2024-04-25T00:31:41.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f43eac3a640 1 -- 172.21.15.29:0/1948763161 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e40cc2f0 con 0x7f43e41f6700 2024-04-25T00:31:41.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f088a7fc640 1 -- 172.21.15.29:0/3059723354 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c0b0070 con 0x7f088c1624b0 2024-04-25T00:31:41.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f43e9c38640 1 --2- 172.21.15.29:0/1948763161 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f43c80679c0 0x7f43c8069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f088a7fc640 1 -- 172.21.15.29:0/3059723354 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f087c0a4e50 con 0x7f088c1624b0 2024-04-25T00:31:41.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f088a7fc640 1 -- 172.21.15.29:0/3059723354 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c0ab860 con 0x7f088c1624b0 2024-04-25T00:31:41.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f43e9c38640 1 --2- 172.21.15.29:0/1948763161 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f43c80679c0 0x7f43c8069e80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f43e00093a0 tx=0x7f43e0006ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f0890f80640 1 -- 172.21.15.29:0/3059723354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c1624b0 msgr2=0x7f088c0c4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f0890f80640 1 --2- 172.21.15.29:0/3059723354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c1624b0 0x7f088c0c4bd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f088c0b47f0 tx=0x7f087c01c330 comp rx=0 tx=0).stop 2024-04-25T00:31:41.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f0890f80640 1 -- 172.21.15.29:0/3059723354 shutdown_connections 2024-04-25T00:31:41.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f0890f80640 1 --2- 172.21.15.29:0/3059723354 >> v2:172.21.15.29:3300/0 conn(0x7f088c0cee60 0x7f088c0c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f0890f80640 1 --2- 172.21.15.29:0/3059723354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c1624b0 0x7f088c0c4bd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.376+0000 7f0890f80640 1 -- 172.21.15.29:0/3059723354 >> 172.21.15.29:0/3059723354 conn(0x7f088c09aa30 msgr2=0x7f088c09ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.384+0000 7f0890f80640 1 -- 172.21.15.29:0/3059723354 shutdown_connections 2024-04-25T00:31:41.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.384+0000 7f43da7fc640 1 -- 172.21.15.29:0/1948763161 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f43e40cc2f0 con 0x7f43e41f6700 2024-04-25T00:31:41.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.384+0000 7f0890f80640 1 -- 172.21.15.29:0/3059723354 wait complete. 2024-04-25T00:31:41.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.384+0000 7ff17da9e640 1 Processor -- start 2024-04-25T00:31:41.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.384+0000 7f0890f80640 1 Processor -- start 2024-04-25T00:31:41.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.384+0000 7ff17da9e640 1 -- start start 2024-04-25T00:31:41.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7ff17da9e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1781438d0 0x7ff17806dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7ff17da9e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1780b3490 0x7ff178072500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7ff17da9e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff17806cc60 con 0x7ff178072070 2024-04-25T00:31:41.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7ff17da9e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff17806cf40 con 0x7ff1780b3490 2024-04-25T00:31:41.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7ff17da9e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff17806d220 con 0x7ff1781438d0 2024-04-25T00:31:41.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7ff17d29d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1780b3490 0x7ff178072500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7ff17d29d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1780b3490 0x7ff178072500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:60924/0 (socket says 172.21.15.29:60924) 2024-04-25T00:31:41.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7ff17d29d640 1 -- 172.21.15.29:0/814832290 learned_addr learned my addr 172.21.15.29:0/814832290 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7ff17ca9c640 1 --2- 172.21.15.29:0/814832290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1781438d0 0x7ff17806dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f0890f80640 1 -- start start 2024-04-25T00:31:41.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f0890f80640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c06ce40 0x7f088c0c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f0890f80640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f088c0cee60 0x7f088c0c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f0890f80640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f088c1624b0 0x7f088c0b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f0890f80640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f088c070980 con 0x7f088c1624b0 2024-04-25T00:31:41.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f0890f80640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f088c06c220 con 0x7f088c06ce40 2024-04-25T00:31:41.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f0890f80640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f088c06f430 con 0x7f088c0cee60 2024-04-25T00:31:41.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f088b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c06ce40 0x7f088c0c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f088bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f088c1624b0 0x7f088c0b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f088bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f088c1624b0 0x7f088c0b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49966/0 (socket says 172.21.15.29:49966) 2024-04-25T00:31:41.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f088bfff640 1 -- 172.21.15.29:0/2990924767 learned_addr learned my addr 172.21.15.29:0/2990924767 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f088affd640 1 --2- 172.21.15.29:0/2990924767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f088c0cee60 0x7f088c0c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f088bfff640 1 -- 172.21.15.29:0/2990924767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f088c0cee60 msgr2=0x7f088c0c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f088bfff640 1 --2- 172.21.15.29:0/2990924767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f088c0cee60 0x7f088c0c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f088bfff640 1 -- 172.21.15.29:0/2990924767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c06ce40 msgr2=0x7f088c0c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f088bfff640 1 --2- 172.21.15.29:0/2990924767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c06ce40 0x7f088c0c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f088bfff640 1 -- 172.21.15.29:0/2990924767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c14e6d0 con 0x7f088c1624b0 2024-04-25T00:31:41.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f088bfff640 1 --2- 172.21.15.29:0/2990924767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f088c1624b0 0x7f088c0b1560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f088000e810 tx=0x7f088000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f0873fff640 1 -- 172.21.15.29:0/2990924767 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880017070 con 0x7f088c1624b0 2024-04-25T00:31:41.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7f0890f80640 1 -- 172.21.15.29:0/2990924767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088c14e9b0 con 0x7f088c1624b0 2024-04-25T00:31:41.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.392+0000 7f0890f80640 1 -- 172.21.15.29:0/2990924767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088c14ef70 con 0x7f088c1624b0 2024-04-25T00:31:41.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7ff17ca9c640 1 -- 172.21.15.29:0/814832290 >> v2:172.21.15.157:3300/0 conn(0x7ff1780b3490 msgr2=0x7ff178072500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7ff17ca9c640 1 --2- 172.21.15.29:0/814832290 >> v2:172.21.15.157:3300/0 conn(0x7ff1780b3490 0x7ff178072500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7ff17ca9c640 1 -- 172.21.15.29:0/814832290 >> v1:172.21.15.157:6789/0 conn(0x7ff178072070 legacy=0x7ff1780b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.388+0000 7ff17ca9c640 1 -- 172.21.15.29:0/814832290 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17806d500 con 0x7ff1781438d0 2024-04-25T00:31:41.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.392+0000 7f0873fff640 1 -- 172.21.15.29:0/2990924767 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f088000c6d0 con 0x7f088c1624b0 2024-04-25T00:31:41.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.392+0000 7f0873fff640 1 -- 172.21.15.29:0/2990924767 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088000f040 con 0x7f088c1624b0 2024-04-25T00:31:41.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.392+0000 7f0873fff640 1 -- 172.21.15.29:0/2990924767 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f088001a070 con 0x7f088c1624b0 2024-04-25T00:31:41.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.392+0000 7ff17ca9c640 1 --2- 172.21.15.29:0/814832290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1781438d0 0x7ff17806dfe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff16800dbe0 tx=0x7ff168020660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=851332f74aa646a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.392+0000 7ff1777fe640 1 -- 172.21.15.29:0/814832290 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1680b0020 con 0x7ff1781438d0 2024-04-25T00:31:41.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.392+0000 7ff1777fe640 1 -- 172.21.15.29:0/814832290 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1680aa4a0 con 0x7ff1781438d0 2024-04-25T00:31:41.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.392+0000 7f0873fff640 1 --2- 172.21.15.29:0/2990924767 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f08780679c0 0x7f0878069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.392+0000 7f0873fff640 1 -- 172.21.15.29:0/2990924767 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f08800858a0 con 0x7f088c1624b0 2024-04-25T00:31:41.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7f088b7fe640 1 --2- 172.21.15.29:0/2990924767 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f08780679c0 0x7f0878069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7ff1777fe640 1 -- 172.21.15.29:0/814832290 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1680aa8b0 con 0x7ff1781438d0 2024-04-25T00:31:41.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7ff17da9e640 1 -- 172.21.15.29:0/814832290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1781438d0 msgr2=0x7ff17806dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7ff17da9e640 1 --2- 172.21.15.29:0/814832290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1781438d0 0x7ff17806dfe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff16800dbe0 tx=0x7ff168020660 comp rx=0 tx=0).stop 2024-04-25T00:31:41.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7ff17da9e640 1 -- 172.21.15.29:0/814832290 shutdown_connections 2024-04-25T00:31:41.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7ff17da9e640 1 --2- 172.21.15.29:0/814832290 >> v2:172.21.15.157:3300/0 conn(0x7ff1780b3490 0x7ff178072500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7ff17da9e640 1 --2- 172.21.15.29:0/814832290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1781438d0 0x7ff17806dfe0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7ff17da9e640 1 -- 172.21.15.29:0/814832290 >> 172.21.15.29:0/814832290 conn(0x7ff1780b1770 msgr2=0x7ff1780b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7f0890f80640 1 -- 172.21.15.29:0/2990924767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f086c0052a0 con 0x7f088c1624b0 2024-04-25T00:31:41.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7ff17da9e640 1 -- 172.21.15.29:0/814832290 shutdown_connections 2024-04-25T00:31:41.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7ff17da9e640 1 -- 172.21.15.29:0/814832290 wait complete. 2024-04-25T00:31:41.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7ff17da9e640 1 Processor -- start 2024-04-25T00:31:41.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7ff17da9e640 1 -- start start 2024-04-25T00:31:41.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7f088b7fe640 1 --2- 172.21.15.29:0/2990924767 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f08780679c0 0x7f0878069e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f087c0ad000 tx=0x7f087c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7ff17da9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff178072070 0x7ff1781563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.396+0000 7ff17da9e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1780b3490 0x7ff17813b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff17da9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1781438d0 0x7ff1780b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff17da9e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff17806e520 con 0x7ff178072070 2024-04-25T00:31:41.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff17da9e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff17806dac0 con 0x7ff1781438d0 2024-04-25T00:31:41.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff177fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1780b3490 0x7ff17813b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff177fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1780b3490 0x7ff17813b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:58820/0 (socket says 172.21.15.29:58820) 2024-04-25T00:31:41.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff177fff640 1 -- 172.21.15.29:0/1570088643 learned_addr learned my addr 172.21.15.29:0/1570088643 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff17d29d640 1 --2- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1781438d0 0x7ff1780b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff17da9e640 1 -- 172.21.15.29:0/1570088643 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff17806e6a0 con 0x7ff1780b3490 2024-04-25T00:31:41.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff17d29d640 1 -- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1780b3490 msgr2=0x7ff17813b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff17d29d640 1 --2- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1780b3490 0x7ff17813b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff17d29d640 1 -- 172.21.15.29:0/1570088643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff178072070 msgr2=0x7ff1781563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff17d29d640 1 --2- 172.21.15.29:0/1570088643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff178072070 0x7ff1781563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff17d29d640 1 -- 172.21.15.29:0/1570088643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17813c970 con 0x7ff1781438d0 2024-04-25T00:31:41.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff17d29d640 1 --2- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1781438d0 0x7ff1780b0950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff178075870 tx=0x7ff1700049a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff1757fa640 1 -- 172.21.15.29:0/1570088643 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17000be60 con 0x7ff1781438d0 2024-04-25T00:31:41.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff17da9e640 1 -- 172.21.15.29:0/1570088643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17813c3b0 con 0x7ff1781438d0 2024-04-25T00:31:41.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff17da9e640 1 -- 172.21.15.29:0/1570088643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff178153da0 con 0x7ff1781438d0 2024-04-25T00:31:41.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff1757fa640 1 -- 172.21.15.29:0/1570088643 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff17000ed70 con 0x7ff1781438d0 2024-04-25T00:31:41.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.400+0000 7ff1757fa640 1 -- 172.21.15.29:0/1570088643 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170007a50 con 0x7ff1781438d0 2024-04-25T00:31:41.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.404+0000 7ff17da9e640 1 -- 172.21.15.29:0/1570088643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1580052a0 con 0x7ff1781438d0 2024-04-25T00:31:41.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.404+0000 7ff1757fa640 1 -- 172.21.15.29:0/1570088643 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff17001b030 con 0x7ff1781438d0 2024-04-25T00:31:41.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.404+0000 7f0873fff640 1 -- 172.21.15.29:0/2990924767 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0880055d70 con 0x7f088c1624b0 2024-04-25T00:31:41.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.404+0000 7ff1757fa640 1 --2- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1640678c0 0x7ff164069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.404+0000 7ff1757fa640 1 -- 172.21.15.29:0/1570088643 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff170086430 con 0x7ff1781438d0 2024-04-25T00:31:41.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.404+0000 7ff17ca9c640 1 --2- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1640678c0 0x7ff164069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.404+0000 7ff17ca9c640 1 --2- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1640678c0 0x7ff164069d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff1680b1000 tx=0x7ff168004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.412+0000 7f6c9b833640 1 Processor -- start 2024-04-25T00:31:41.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.412+0000 7f6c9b833640 1 -- start start 2024-04-25T00:31:41.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c9b833640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6c8c055260 0x7f6c8c0355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c9b833640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6c8c052ca0 0x7f6c8c008ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c9b833640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c8c055640 con 0x7f6c8c036fa0 2024-04-25T00:31:41.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c9b833640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6c8c03b670 con 0x7f6c8c052ca0 2024-04-25T00:31:41.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c9b833640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f6c8c03aa60 con 0x7f6c8c055260 2024-04-25T00:31:41.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c99dd0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6c8c052ca0 0x7f6c8c008ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c995cf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6c8c055260 0x7f6c8c0355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c99dd0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6c8c052ca0 0x7f6c8c008ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49988/0 (socket says 172.21.15.29:49988) 2024-04-25T00:31:41.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c995cf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6c8c055260 0x7f6c8c0355b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:60954/0 (socket says 172.21.15.29:60954) 2024-04-25T00:31:41.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c99dd0640 1 -- 172.21.15.29:0/3768610424 learned_addr learned my addr 172.21.15.29:0/3768610424 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7ff1757fa640 1 -- 172.21.15.29:0/1570088643 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff170022020 con 0x7ff1781438d0 2024-04-25T00:31:41.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c99dd0640 1 -- 172.21.15.29:0/3768610424 >> v2:172.21.15.157:3300/0 conn(0x7f6c8c055260 msgr2=0x7f6c8c0355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c99dd0640 1 --2- 172.21.15.29:0/3768610424 >> v2:172.21.15.157:3300/0 conn(0x7f6c8c055260 0x7f6c8c0355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c99dd0640 1 -- 172.21.15.29:0/3768610424 >> v1:172.21.15.157:6789/0 conn(0x7f6c8c036fa0 legacy=0x7f6c8c052590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c99dd0640 1 -- 172.21.15.29:0/3768610424 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c03ad40 con 0x7f6c8c052ca0 2024-04-25T00:31:41.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.416+0000 7f6c99dd0640 1 --2- 172.21.15.29:0/3768610424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c8c052ca0 0x7f6c8c008ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6c9406a4c0 tx=0x7f6c94098070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9e23986fefb271 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac2ca2640 1 Processor -- start 2024-04-25T00:31:41.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac2ca2640 1 -- start start 2024-04-25T00:31:41.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac2ca2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ab40dfee0 0x7f5ab4055580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac2ca2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ab4050920 0x7f5ab4050d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac2ca2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab40e02c0 con 0x7f5ab403b570 2024-04-25T00:31:41.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac1ca0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ab40dfee0 0x7f5ab4055580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac1ca0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ab40dfee0 0x7f5ab4055580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:60970/0 (socket says 172.21.15.29:60970) 2024-04-25T00:31:41.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac1ca0640 1 -- 172.21.15.29:0/2076093692 learned_addr learned my addr 172.21.15.29:0/2076093692 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/2076093692 --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab40512b0 con 0x7f5ab4050920 2024-04-25T00:31:41.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/2076093692 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab40550f0 con 0x7f5ab40dfee0 2024-04-25T00:31:41.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac24a1640 1 --2- 172.21.15.29:0/2076093692 >> v2:172.21.15.29:3300/0 conn(0x7f5ab4050920 0x7f5ab4050d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac1ca0640 1 -- 172.21.15.29:0/2076093692 >> v2:172.21.15.29:3300/0 conn(0x7f5ab4050920 msgr2=0x7f5ab4050d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac1ca0640 1 --2- 172.21.15.29:0/2076093692 >> v2:172.21.15.29:3300/0 conn(0x7f5ab4050920 0x7f5ab4050d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac1ca0640 1 -- 172.21.15.29:0/2076093692 >> v1:172.21.15.29:6789/0 conn(0x7f5ab403b570 legacy=0x7f5ab4053c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.420+0000 7f5ac1ca0640 1 -- 172.21.15.29:0/2076093692 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab40544e0 con 0x7f5ab40dfee0 2024-04-25T00:31:41.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f5ac24a1640 1 --2- 172.21.15.29:0/2076093692 >> v2:172.21.15.29:3300/0 conn(0x7f5ab4050920 0x7f5ab4050d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:41.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f5ac1ca0640 1 --2- 172.21.15.29:0/2076093692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab40dfee0 0x7f5ab4055580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5abc07a8e0 tx=0x7f5abc0980f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9940b4d5693feef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f6c8bfff640 1 -- 172.21.15.29:0/3768610424 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9412d070 con 0x7f6c8c052ca0 2024-04-25T00:31:41.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f6c8bfff640 1 -- 172.21.15.29:0/3768610424 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c94128440 con 0x7f6c8c052ca0 2024-04-25T00:31:41.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f6c8bfff640 1 -- 172.21.15.29:0/3768610424 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94120e60 con 0x7f6c8c052ca0 2024-04-25T00:31:41.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f5ac0c9e640 1 -- 172.21.15.29:0/2076093692 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc12c070 con 0x7f5ab40dfee0 2024-04-25T00:31:41.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f5ac0c9e640 1 -- 172.21.15.29:0/2076093692 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5abc127600 con 0x7f5ab40dfee0 2024-04-25T00:31:41.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f5ac0c9e640 1 -- 172.21.15.29:0/2076093692 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc127ab0 con 0x7f5ab40dfee0 2024-04-25T00:31:41.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f6c9b833640 1 -- 172.21.15.29:0/3768610424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c8c052ca0 msgr2=0x7f6c8c008ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f6c9b833640 1 --2- 172.21.15.29:0/3768610424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c8c052ca0 0x7f6c8c008ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6c9406a4c0 tx=0x7f6c94098070 comp rx=0 tx=0).stop 2024-04-25T00:31:41.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f6c9b833640 1 -- 172.21.15.29:0/3768610424 shutdown_connections 2024-04-25T00:31:41.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f6c9b833640 1 --2- 172.21.15.29:0/3768610424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c8c052ca0 0x7f6c8c008ad0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f6c9b833640 1 --2- 172.21.15.29:0/3768610424 >> v2:172.21.15.157:3300/0 conn(0x7f6c8c055260 0x7f6c8c0355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f6c9b833640 1 -- 172.21.15.29:0/3768610424 >> 172.21.15.29:0/3768610424 conn(0x7f6c8c01ef50 msgr2=0x7f6c8c055c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f6c9b833640 1 -- 172.21.15.29:0/3768610424 shutdown_connections 2024-04-25T00:31:41.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f6c9b833640 1 -- 172.21.15.29:0/3768610424 wait complete. 2024-04-25T00:31:41.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/2076093692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab40dfee0 msgr2=0x7f5ab4055580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.424+0000 7f5ac2ca2640 1 --2- 172.21.15.29:0/2076093692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab40dfee0 0x7f5ab4055580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5abc07a8e0 tx=0x7f5abc0980f0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/2076093692 shutdown_connections 2024-04-25T00:31:41.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f5ac2ca2640 1 --2- 172.21.15.29:0/2076093692 >> v2:172.21.15.29:3300/0 conn(0x7f5ab4050920 0x7f5ab4050d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f5ac2ca2640 1 --2- 172.21.15.29:0/2076093692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab40dfee0 0x7f5ab4055580 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/2076093692 >> 172.21.15.29:0/2076093692 conn(0x7f5ab4055be0 msgr2=0x7f5ab4055ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c9b833640 1 Processor -- start 2024-04-25T00:31:41.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c9b833640 1 -- start start 2024-04-25T00:31:41.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c9b833640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c8c036fa0 0x7f6c8c0dbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c9b833640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c8c052ca0 0x7f6c8c0d6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c9b833640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c8c055260 0x7f6c8c0df170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c98dce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c8c052ca0 0x7f6c8c0d6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c98dce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c8c052ca0 0x7f6c8c0d6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50002/0 (socket says 172.21.15.29:50002) 2024-04-25T00:31:41.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c9b833640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c0e0260 con 0x7f6c8c052ca0 2024-04-25T00:31:41.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c9b833640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c0e00e0 con 0x7f6c8c036fa0 2024-04-25T00:31:41.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c9b833640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6c8c0e03e0 con 0x7f6c8c055260 2024-04-25T00:31:41.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c98dce640 1 -- 172.21.15.29:0/3471931521 learned_addr learned my addr 172.21.15.29:0/3471931521 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c995cf640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c8c036fa0 0x7f6c8c0dbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c99dd0640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c8c055260 0x7f6c8c0df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c98dce640 1 -- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c8c055260 msgr2=0x7f6c8c0df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c98dce640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c8c055260 0x7f6c8c0df170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c98dce640 1 -- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c8c036fa0 msgr2=0x7f6c8c0dbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c98dce640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c8c036fa0 0x7f6c8c0dbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c98dce640 1 -- 172.21.15.29:0/3471931521 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c0df6b0 con 0x7f6c8c052ca0 2024-04-25T00:31:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f6c98dce640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c8c052ca0 0x7f6c8c0d6fb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000c480 tx=0x7f6c9000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/2076093692 shutdown_connections 2024-04-25T00:31:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.428+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/2076093692 wait complete. 2024-04-25T00:31:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.432+0000 7f6c89ffb640 1 -- 172.21.15.29:0/3471931521 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90012830 con 0x7f6c8c052ca0 2024-04-25T00:31:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.432+0000 7f6c9b833640 1 -- 172.21.15.29:0/3471931521 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8c0dc170 con 0x7f6c8c052ca0 2024-04-25T00:31:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.432+0000 7f6c9b833640 1 -- 172.21.15.29:0/3471931521 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c8c0da5d0 con 0x7f6c8c052ca0 2024-04-25T00:31:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.432+0000 7f6c89ffb640 1 -- 172.21.15.29:0/3471931521 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c90017070 con 0x7f6c8c052ca0 2024-04-25T00:31:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.432+0000 7f6c89ffb640 1 -- 172.21.15.29:0/3471931521 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90012c30 con 0x7f6c8c052ca0 2024-04-25T00:31:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.432+0000 7f5ac2ca2640 1 Processor -- start 2024-04-25T00:31:41.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.432+0000 7f5ac2ca2640 1 -- start start 2024-04-25T00:31:41.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.432+0000 7f6c89ffb640 1 -- 172.21.15.29:0/3471931521 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6c9001c460 con 0x7f6c8c052ca0 2024-04-25T00:31:41.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.432+0000 7f6c89ffb640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6c80067a50 0x7f6c80069f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.432+0000 7f6c89ffb640 1 -- 172.21.15.29:0/3471931521 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6c90085a50 con 0x7f6c8c052ca0 2024-04-25T00:31:41.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.432+0000 7f6c995cf640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6c80067a50 0x7f6c80069f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac2ca2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ab403b570 0x7f5ab40e0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac2ca2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab4050920 0x7f5ab40d7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac2ca2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ab40dfee0 0x7f5ab40d8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac2ca2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4039710 con 0x7f5ab403b570 2024-04-25T00:31:41.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac2ca2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4039590 con 0x7f5ab4050920 2024-04-25T00:31:41.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac2ca2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab4039890 con 0x7f5ab40dfee0 2024-04-25T00:31:41.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac149f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab4050920 0x7f5ab40d7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac24a1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ab40dfee0 0x7f5ab40d8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac149f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab4050920 0x7f5ab40d7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:60984/0 (socket says 172.21.15.29:60984) 2024-04-25T00:31:41.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac149f640 1 -- 172.21.15.29:0/638595523 learned_addr learned my addr 172.21.15.29:0/638595523 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac24a1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ab40dfee0 0x7f5ab40d8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:58848/0 (socket says 172.21.15.29:58848) 2024-04-25T00:31:41.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac1ca0640 1 --2- 172.21.15.29:0/638595523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ab403b570 0x7f5ab40e0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac149f640 1 -- 172.21.15.29:0/638595523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ab40dfee0 msgr2=0x7f5ab40d8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac149f640 1 --2- 172.21.15.29:0/638595523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ab40dfee0 0x7f5ab40d8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac149f640 1 -- 172.21.15.29:0/638595523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ab403b570 msgr2=0x7f5ab40e0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac149f640 1 --2- 172.21.15.29:0/638595523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ab403b570 0x7f5ab40e0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac149f640 1 -- 172.21.15.29:0/638595523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab40db4e0 con 0x7f5ab4050920 2024-04-25T00:31:41.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f6c995cf640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6c80067a50 0x7f6c80069f10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009a50 tx=0x7f6c84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac149f640 1 --2- 172.21.15.29:0/638595523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab4050920 0x7f5ab40d7ce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800c480 tx=0x7f5ab800c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ab27fc640 1 -- 172.21.15.29:0/638595523 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8016020 con 0x7f5ab4050920 2024-04-25T00:31:41.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/638595523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab40db7c0 con 0x7f5ab4050920 2024-04-25T00:31:41.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/638595523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab40dbd80 con 0x7f5ab4050920 2024-04-25T00:31:41.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ab27fc640 1 -- 172.21.15.29:0/638595523 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab8012830 con 0x7f5ab4050920 2024-04-25T00:31:41.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ab27fc640 1 -- 172.21.15.29:0/638595523 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8012c20 con 0x7f5ab4050920 2024-04-25T00:31:41.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.436+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/638595523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab4009af0 con 0x7f5ab4050920 2024-04-25T00:31:41.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.440+0000 7f5ab27fc640 1 -- 172.21.15.29:0/638595523 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5ab8019020 con 0x7f5ab4050920 2024-04-25T00:31:41.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.440+0000 7f5ab27fc640 1 --2- 172.21.15.29:0/638595523 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5aa8067a20 0x7f5aa8069ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.440+0000 7f5ab27fc640 1 -- 172.21.15.29:0/638595523 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5ab8085ff0 con 0x7f5ab4050920 2024-04-25T00:31:41.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.440+0000 7f5ac1ca0640 1 --2- 172.21.15.29:0/638595523 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5aa8067a20 0x7f5aa8069ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.440+0000 7f5ac1ca0640 1 --2- 172.21.15.29:0/638595523 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5aa8067a20 0x7f5aa8069ee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40e0490 tx=0x7f5abc129040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.448+0000 7f6c9b833640 1 -- 172.21.15.29:0/3471931521 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c740052a0 con 0x7f6c8c052ca0 2024-04-25T00:31:41.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.452+0000 7f5ab27fc640 1 -- 172.21.15.29:0/638595523 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5ab8020020 con 0x7f5ab4050920 2024-04-25T00:31:41.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.456+0000 7f6c89ffb640 1 -- 172.21.15.29:0/3471931521 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6c90056570 con 0x7f6c8c052ca0 2024-04-25T00:31:41.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.660+0000 7ff125bbf640 1 -- 172.21.15.29:0/1182117518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff100005590 con 0x7ff12015cfb0 2024-04-25T00:31:41.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.660+0000 7ff10bfff640 1 -- 172.21.15.29:0/1182117518 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff10c0f2850 con 0x7ff12015cfb0 2024-04-25T00:31:41.669 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-04-25T00:31:41.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7f665ff5c640 1 -- 172.21.15.29:0/3081283154 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f665806e5d0 con 0x7f66581438a0 2024-04-25T00:31:41.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7f664e7fc640 1 -- 172.21.15.29:0/3081283154 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f66500b9070 con 0x7f66581438a0 2024-04-25T00:31:41.670 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-04-25T00:31:41.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7ff125bbf640 1 -- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1040678c0 msgr2=0x7ff104069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7ff125bbf640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1040678c0 0x7ff104069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff114009bd0 tx=0x7ff114009240 comp rx=0 tx=0).stop 2024-04-25T00:31:41.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7ff125bbf640 1 -- 172.21.15.29:0/1182117518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff12015cfb0 msgr2=0x7ff1201395f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7ff125bbf640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff12015cfb0 0x7ff1201395f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff12015d590 tx=0x7ff10c01ba10 comp rx=0 tx=0).stop 2024-04-25T00:31:41.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/1386757799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2e680af7c0 con 0x7f2e68151020 2024-04-25T00:31:41.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7ff125bbf640 1 -- 172.21.15.29:0/1182117518 shutdown_connections 2024-04-25T00:31:41.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7ff125bbf640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1040678c0 0x7ff104069d80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7ff125bbf640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff120157ae0 0x7ff12015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7ff125bbf640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff12015cfb0 0x7ff1201395f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7ff125bbf640 1 --2- 172.21.15.29:0/1182117518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff120145840 0x7ff1201e2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7ff125bbf640 1 -- 172.21.15.29:0/1182117518 >> 172.21.15.29:0/1182117518 conn(0x7ff1200b1870 msgr2=0x7ff120144960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.664+0000 7f2e657fa640 1 -- 172.21.15.29:0/1386757799 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2e5c05a9c0 con 0x7f2e68151020 2024-04-25T00:31:41.672 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-04-25T00:31:41.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7ff125bbf640 1 -- 172.21.15.29:0/1182117518 shutdown_connections 2024-04-25T00:31:41.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7ff125bbf640 1 -- 172.21.15.29:0/1182117518 wait complete. 2024-04-25T00:31:41.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f665ff5c640 1 -- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f663c0679c0 msgr2=0x7f663c069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f665ff5c640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f663c0679c0 0x7f663c069e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f66580add60 tx=0x7f6654009190 comp rx=0 tx=0).stop 2024-04-25T00:31:41.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f665ff5c640 1 -- 172.21.15.29:0/3081283154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66581438a0 msgr2=0x7f6658163ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f665ff5c640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66581438a0 0x7f6658163ff0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f66500058b0 tx=0x7f6644005840 comp rx=0 tx=0).stop 2024-04-25T00:31:41.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/1386757799 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2e54067990 msgr2=0x7f2e54069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f2e6f0a6640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2e54067990 0x7f2e54069e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e58000c10 tx=0x7f2e5800a040 comp rx=0 tx=0).stop 2024-04-25T00:31:41.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/1386757799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e68151020 msgr2=0x7f2e6815cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f2e6f0a6640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e68151020 0x7f2e6815cdb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2e680af4f0 tx=0x7f2e5c00ec50 comp rx=0 tx=0).stop 2024-04-25T00:31:41.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f665ff5c640 1 -- 172.21.15.29:0/3081283154 shutdown_connections 2024-04-25T00:31:41.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f665ff5c640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66581441a0 0x7f6658144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f665ff5c640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f663c0679c0 0x7f663c069e80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/1386757799 shutdown_connections 2024-04-25T00:31:41.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f2e6f0a6640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e68151020 0x7f2e6815cdb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f2e6f0a6640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e680cfeb0 0x7f2e681516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f2e6f0a6640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2e54067990 0x7f2e54069e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f2e6f0a6640 1 --2- 172.21.15.29:0/1386757799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e680af020 0x7f2e681474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f665ff5c640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6658143c80 0x7f6658164530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f665ff5c640 1 --2- 172.21.15.29:0/3081283154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66581438a0 0x7f6658163ff0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f665ff5c640 1 -- 172.21.15.29:0/3081283154 >> 172.21.15.29:0/3081283154 conn(0x7f665806b3e0 msgr2=0x7f66580af520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/1386757799 >> 172.21.15.29:0/1386757799 conn(0x7f2e68074880 msgr2=0x7f2e680b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f665ff5c640 1 -- 172.21.15.29:0/3081283154 shutdown_connections 2024-04-25T00:31:41.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/1386757799 shutdown_connections 2024-04-25T00:31:41.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f665ff5c640 1 -- 172.21.15.29:0/3081283154 wait complete. 2024-04-25T00:31:41.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.668+0000 7f2e6f0a6640 1 -- 172.21.15.29:0/1386757799 wait complete. 2024-04-25T00:31:41.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-04-25T00:31:41.694 DEBUG:teuthology.parallel:result is None 2024-04-25T00:31:41.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-04-25T00:31:41.695 DEBUG:teuthology.parallel:result is None 2024-04-25T00:31:41.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-04-25T00:31:41.709 DEBUG:teuthology.parallel:result is None 2024-04-25T00:31:41.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.728+0000 7f43eac3a640 1 -- 172.21.15.29:0/1948763161 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f43e40cbee0 con 0x7f43e41f6700 2024-04-25T00:31:41.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.728+0000 7f43da7fc640 1 -- 172.21.15.29:0/1948763161 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f43d4024020 con 0x7f43e41f6700 2024-04-25T00:31:41.730 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-04-25T00:31:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.728+0000 7f43eac3a640 1 -- 172.21.15.29:0/1948763161 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f43c80679c0 msgr2=0x7f43c8069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.728+0000 7f43eac3a640 1 --2- 172.21.15.29:0/1948763161 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f43c80679c0 0x7f43c8069e80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f43e00093a0 tx=0x7f43e0006ef0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.728+0000 7f43eac3a640 1 -- 172.21.15.29:0/1948763161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43e41f6700 msgr2=0x7f43e41f6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.728+0000 7f43eac3a640 1 --2- 172.21.15.29:0/1948763161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43e41f6700 0x7f43e41f6b80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f43e406e270 tx=0x7f43d400e8e0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.732+0000 7f43eac3a640 1 -- 172.21.15.29:0/1948763161 shutdown_connections 2024-04-25T00:31:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.732+0000 7f43eac3a640 1 --2- 172.21.15.29:0/1948763161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43e41f6700 0x7f43e41f6b80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.732+0000 7f43eac3a640 1 --2- 172.21.15.29:0/1948763161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43e42072a0 0x7f43e413b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.732+0000 7f43eac3a640 1 --2- 172.21.15.29:0/1948763161 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f43c80679c0 0x7f43c8069e80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.732+0000 7f43eac3a640 1 --2- 172.21.15.29:0/1948763161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d0004810 0x7f43e4141f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.732+0000 7f43eac3a640 1 -- 172.21.15.29:0/1948763161 >> 172.21.15.29:0/1948763161 conn(0x7f43e40b1790 msgr2=0x7f43e406e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.732+0000 7f43eac3a640 1 -- 172.21.15.29:0/1948763161 shutdown_connections 2024-04-25T00:31:41.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.732+0000 7f43eac3a640 1 -- 172.21.15.29:0/1948763161 wait complete. 2024-04-25T00:31:41.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.732+0000 7f0890f80640 1 -- 172.21.15.29:0/2990924767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f086c005590 con 0x7f088c1624b0 2024-04-25T00:31:41.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.732+0000 7f0873fff640 1 -- 172.21.15.29:0/2990924767 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f088005a070 con 0x7f088c1624b0 2024-04-25T00:31:41.736 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-04-25T00:31:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7f0890f80640 1 -- 172.21.15.29:0/2990924767 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f08780679c0 msgr2=0x7f0878069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7f0890f80640 1 --2- 172.21.15.29:0/2990924767 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f08780679c0 0x7f0878069e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f087c0ad000 tx=0x7f087c008040 comp rx=0 tx=0).stop 2024-04-25T00:31:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7f0890f80640 1 -- 172.21.15.29:0/2990924767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f088c1624b0 msgr2=0x7f088c0b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7f0890f80640 1 --2- 172.21.15.29:0/2990924767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f088c1624b0 0x7f088c0b1560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f088000e810 tx=0x7f088000ecd0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7f0890f80640 1 -- 172.21.15.29:0/2990924767 shutdown_connections 2024-04-25T00:31:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7f0890f80640 1 --2- 172.21.15.29:0/2990924767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f088c1624b0 0x7f088c0b1560 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7f0890f80640 1 --2- 172.21.15.29:0/2990924767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f088c0cee60 0x7f088c0c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7f0890f80640 1 --2- 172.21.15.29:0/2990924767 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f08780679c0 0x7f0878069e80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7f0890f80640 1 --2- 172.21.15.29:0/2990924767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c06ce40 0x7f088c0c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7f0890f80640 1 -- 172.21.15.29:0/2990924767 >> 172.21.15.29:0/2990924767 conn(0x7f088c09aa30 msgr2=0x7f088c06c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7f0890f80640 1 -- 172.21.15.29:0/2990924767 shutdown_connections 2024-04-25T00:31:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7f0890f80640 1 -- 172.21.15.29:0/2990924767 wait complete. 2024-04-25T00:31:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7ff17da9e640 1 -- 172.21.15.29:0/1570088643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff158005860 con 0x7ff1781438d0 2024-04-25T00:31:41.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.736+0000 7ff1757fa640 1 -- 172.21.15.29:0/1570088643 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff17005be20 con 0x7ff1781438d0 2024-04-25T00:31:41.741 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-04-25T00:31:41.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7f6c9b833640 1 -- 172.21.15.29:0/3471931521 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6c74005590 con 0x7f6c8c052ca0 2024-04-25T00:31:41.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7f6c89ffb640 1 -- 172.21.15.29:0/3471931521 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6c90055e70 con 0x7f6c8c052ca0 2024-04-25T00:31:41.742 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-04-25T00:31:41.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7ff17da9e640 1 -- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1640678c0 msgr2=0x7ff164069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7ff17da9e640 1 --2- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1640678c0 0x7ff164069d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff1680b1000 tx=0x7ff168004740 comp rx=0 tx=0).stop 2024-04-25T00:31:41.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7ff17da9e640 1 -- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1781438d0 msgr2=0x7ff1780b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7ff17da9e640 1 --2- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1781438d0 0x7ff1780b0950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff178075870 tx=0x7ff1700049a0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7ff17da9e640 1 -- 172.21.15.29:0/1570088643 shutdown_connections 2024-04-25T00:31:41.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7ff17da9e640 1 --2- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1640678c0 0x7ff164069d80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7ff17da9e640 1 --2- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1781438d0 0x7ff1780b0950 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7ff17da9e640 1 --2- 172.21.15.29:0/1570088643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1780b3490 0x7ff17813b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7ff17da9e640 1 --2- 172.21.15.29:0/1570088643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff178072070 0x7ff1781563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7ff17da9e640 1 -- 172.21.15.29:0/1570088643 >> 172.21.15.29:0/1570088643 conn(0x7ff1780b1770 msgr2=0x7ff17815c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7ff17da9e640 1 -- 172.21.15.29:0/1570088643 shutdown_connections 2024-04-25T00:31:41.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.740+0000 7ff17da9e640 1 -- 172.21.15.29:0/1570088643 wait complete. 2024-04-25T00:31:41.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.744+0000 7f6c9b833640 1 -- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6c80067a50 msgr2=0x7f6c80069f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.744+0000 7f6c9b833640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6c80067a50 0x7f6c80069f10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009a50 tx=0x7f6c84008040 comp rx=0 tx=0).stop 2024-04-25T00:31:41.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.744+0000 7f6c9b833640 1 -- 172.21.15.29:0/3471931521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c8c052ca0 msgr2=0x7f6c8c0d6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.744+0000 7f6c9b833640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c8c052ca0 0x7f6c8c0d6fb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000c480 tx=0x7f6c9000c940 comp rx=0 tx=0).stop 2024-04-25T00:31:41.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.744+0000 7f6c9b833640 1 -- 172.21.15.29:0/3471931521 shutdown_connections 2024-04-25T00:31:41.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.744+0000 7f6c9b833640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c8c055260 0x7f6c8c0df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.744+0000 7f6c9b833640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6c8c052ca0 0x7f6c8c0d6fb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.744+0000 7f6c9b833640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6c80067a50 0x7f6c80069f10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.744+0000 7f6c9b833640 1 --2- 172.21.15.29:0/3471931521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c8c036fa0 0x7f6c8c0dbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.744+0000 7f6c9b833640 1 -- 172.21.15.29:0/3471931521 >> 172.21.15.29:0/3471931521 conn(0x7f6c8c01ef50 msgr2=0x7f6c8c00c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.744+0000 7f6c9b833640 1 -- 172.21.15.29:0/3471931521 shutdown_connections 2024-04-25T00:31:41.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.744+0000 7f6c9b833640 1 -- 172.21.15.29:0/3471931521 wait complete. 2024-04-25T00:31:41.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-04-25T00:31:41.754 DEBUG:teuthology.parallel:result is None 2024-04-25T00:31:41.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-04-25T00:31:41.761 DEBUG:teuthology.parallel:result is None 2024-04-25T00:31:41.765 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-25T00:31:41.765 DEBUG:teuthology.parallel:result is None 2024-04-25T00:31:41.772 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-04-25T00:31:41.773 DEBUG:teuthology.parallel:result is None 2024-04-25T00:31:41.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/638595523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5ab40057f0 con 0x7f5ab4050920 2024-04-25T00:31:41.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ab27fc640 1 -- 172.21.15.29:0/638595523 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5ab8056550 con 0x7f5ab4050920 2024-04-25T00:31:41.778 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-04-25T00:31:41.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/638595523 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5aa8067a20 msgr2=0x7f5aa8069ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ac2ca2640 1 --2- 172.21.15.29:0/638595523 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5aa8067a20 0x7f5aa8069ee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40e0490 tx=0x7f5abc129040 comp rx=0 tx=0).stop 2024-04-25T00:31:41.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/638595523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab4050920 msgr2=0x7f5ab40d7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ac2ca2640 1 --2- 172.21.15.29:0/638595523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab4050920 0x7f5ab40d7ce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800c480 tx=0x7f5ab800c940 comp rx=0 tx=0).stop 2024-04-25T00:31:41.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/638595523 shutdown_connections 2024-04-25T00:31:41.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ac2ca2640 1 --2- 172.21.15.29:0/638595523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ab40dfee0 0x7f5ab40d8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ac2ca2640 1 --2- 172.21.15.29:0/638595523 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5aa8067a20 0x7f5aa8069ee0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ac2ca2640 1 --2- 172.21.15.29:0/638595523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab4050920 0x7f5ab40d7ce0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ac2ca2640 1 --2- 172.21.15.29:0/638595523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ab403b570 0x7f5ab40e0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/638595523 >> 172.21.15.29:0/638595523 conn(0x7f5ab4055be0 msgr2=0x7f5ab40ede70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/638595523 shutdown_connections 2024-04-25T00:31:41.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.776+0000 7f5ac2ca2640 1 -- 172.21.15.29:0/638595523 wait complete. 2024-04-25T00:31:41.795 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-04-25T00:31:41.795 DEBUG:teuthology.parallel:result is None 2024-04-25T00:31:41.795 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T00:31:41.795 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:31:41.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd61a2e9640 1 Processor -- start 2024-04-25T00:31:41.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd61a2e9640 1 -- start start 2024-04-25T00:31:41.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd61a2e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd61406e440 0x7fd6140b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd61a2e9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6140b41e0 con 0x7fd6140b3140 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd61a2e9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd61406d960 con 0x7fd6140b3c30 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd61a2e9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd6140b5050 con 0x7fd61406e440 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd613fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd6140b3140 0x7fd6140b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50074/0 (socket says 172.21.15.29:50074) 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd613fff640 1 -- 172.21.15.29:0/2995277577 learned_addr learned my addr 172.21.15.29:0/2995277577 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd618886640 1 --2- 172.21.15.29:0/2995277577 >> v2:172.21.15.157:3300/0 conn(0x7fd61406e440 0x7fd6140b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3697980325 0 0) 0x7fd6140b41e0 con 0x7fd6140b3140 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5fc003530 con 0x7fd6140b3140 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 301638363 0 0) 0x7fd5fc003530 con 0x7fd6140b3140 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 >> v2:172.21.15.157:3300/0 conn(0x7fd61406e440 msgr2=0x7fd6140b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 --2- 172.21.15.29:0/2995277577 >> v2:172.21.15.157:3300/0 conn(0x7fd61406e440 0x7fd6140b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 >> v1:172.21.15.157:6789/0 conn(0x7fd6140b3c30 legacy=0x7fd6140ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61406dc40 con 0x7fd6140b3140 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd604001680 con 0x7fd6140b3140 2024-04-25T00:31:41.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 >> v1:172.21.15.29:6789/0 conn(0x7fd6140b3140 legacy=0x7fd6140b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 --2- 172.21.15.29:0/2995277577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc003a10 0x7fd5fc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 --2- 172.21.15.29:0/2995277577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc004810 0x7fd5fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 --2- 172.21.15.29:0/2995277577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc0051d0 0x7fd5fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd6140b5050 con 0x7fd5fc003a10 2024-04-25T00:31:41.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc00c0d0 con 0x7fd5fc0051d0 2024-04-25T00:31:41.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd5fc00c2d0 con 0x7fd5fc004810 2024-04-25T00:31:41.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd613fff640 1 --2- 172.21.15.29:0/2995277577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc003a10 0x7fd5fc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd618886640 1 --2- 172.21.15.29:0/2995277577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc0051d0 0x7fd5fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd6137fe640 1 --2- 172.21.15.29:0/2995277577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc004810 0x7fd5fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd613fff640 1 -- 172.21.15.29:0/2995277577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc004810 msgr2=0x7fd5fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd613fff640 1 --2- 172.21.15.29:0/2995277577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc004810 0x7fd5fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd613fff640 1 -- 172.21.15.29:0/2995277577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc0051d0 msgr2=0x7fd5fc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:41.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd613fff640 1 --2- 172.21.15.29:0/2995277577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc0051d0 0x7fd5fc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd618886640 1 -- 172.21.15.29:0/2995277577 reap_dead start 2024-04-25T00:31:41.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd613fff640 1 -- 172.21.15.29:0/2995277577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc003530 con 0x7fd5fc003a10 2024-04-25T00:31:41.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.944+0000 7fd6137fe640 1 --2- 172.21.15.29:0/2995277577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc004810 0x7fd5fc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.948+0000 7fd613fff640 1 --2- 172.21.15.29:0/2995277577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc003a10 0x7fd5fc003e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd60400c110 tx=0x7fd60401cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a6a9cc15123aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.948+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604007c50 con 0x7fd5fc003a10 2024-04-25T00:31:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.948+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6040a9070 con 0x7fd5fc003a10 2024-04-25T00:31:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.948+0000 7fd612ffd640 1 -- 172.21.15.29:0/2995277577 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604005d80 con 0x7fd5fc003a10 2024-04-25T00:31:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.948+0000 7fd61a2e9640 1 -- 172.21.15.29:0/2995277577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc003a10 msgr2=0x7fd5fc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.948+0000 7fd61a2e9640 1 --2- 172.21.15.29:0/2995277577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc003a10 0x7fd5fc003e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd60400c110 tx=0x7fd60401cb20 comp rx=0 tx=0).stop 2024-04-25T00:31:41.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.948+0000 7fd61a2e9640 1 -- 172.21.15.29:0/2995277577 shutdown_connections 2024-04-25T00:31:41.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.948+0000 7fd61a2e9640 1 --2- 172.21.15.29:0/2995277577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc003a10 0x7fd5fc003e30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.948+0000 7fd61a2e9640 1 -- 172.21.15.29:0/2995277577 >> 172.21.15.29:0/2995277577 conn(0x7fd6140b1870 msgr2=0x7fd6140b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:41.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.948+0000 7fd61a2e9640 1 -- 172.21.15.29:0/2995277577 shutdown_connections 2024-04-25T00:31:41.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.948+0000 7fd61a2e9640 1 -- 172.21.15.29:0/2995277577 wait complete. 2024-04-25T00:31:41.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.948+0000 7fd61a2e9640 1 Processor -- start 2024-04-25T00:31:41.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.948+0000 7fd61a2e9640 1 -- start start 2024-04-25T00:31:41.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd61a2e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc003a10 0x7fd614158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd61a2e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6141e5ae0 0x7fd6140b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd61a2e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6141e5f60 0x7fd6140b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd61a2e9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd614072cf0 con 0x7fd6141e5ae0 2024-04-25T00:31:41.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd61a2e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd61406e820 con 0x7fd5fc003a10 2024-04-25T00:31:41.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd61a2e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd6140b3520 con 0x7fd6141e5f60 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd6137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6141e5ae0 0x7fd6140b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd6137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6141e5ae0 0x7fd6140b01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50038/0 (socket says 172.21.15.29:50038) 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd6137fe640 1 -- 172.21.15.29:0/3457925763 learned_addr learned my addr 172.21.15.29:0/3457925763 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd618886640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6141e5f60 0x7fd6140b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd613fff640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc003a10 0x7fd614158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd6137fe640 1 -- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6141e5f60 msgr2=0x7fd6140b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd6137fe640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6141e5f60 0x7fd6140b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd6137fe640 1 -- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc003a10 msgr2=0x7fd614158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd6137fe640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc003a10 0x7fd614158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd6137fe640 1 -- 172.21.15.29:0/3457925763 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6141e64a0 con 0x7fd6141e5ae0 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd6137fe640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6141e5ae0 0x7fd6140b01f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd60800e480 tx=0x7fd60800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd610ff9640 1 -- 172.21.15.29:0/3457925763 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6080052e0 con 0x7fd6141e5ae0 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd610ff9640 1 -- 172.21.15.29:0/3457925763 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd608018070 con 0x7fd6141e5ae0 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd61a2e9640 1 -- 172.21.15.29:0/3457925763 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6141e6780 con 0x7fd6141e5ae0 2024-04-25T00:31:41.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd610ff9640 1 -- 172.21.15.29:0/3457925763 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608005480 con 0x7fd6141e5ae0 2024-04-25T00:31:41.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd61a2e9640 1 -- 172.21.15.29:0/3457925763 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6141e6d40 con 0x7fd6141e5ae0 2024-04-25T00:31:41.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd61a2e9640 1 -- 172.21.15.29:0/3457925763 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd614069270 con 0x7fd6141e5ae0 2024-04-25T00:31:41.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd610ff9640 1 -- 172.21.15.29:0/3457925763 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd608003840 con 0x7fd6141e5ae0 2024-04-25T00:31:41.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd610ff9640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd6000678c0 0x7fd600069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:41.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd610ff9640 1 -- 172.21.15.29:0/3457925763 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd6080858f0 con 0x7fd6141e5ae0 2024-04-25T00:31:41.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.952+0000 7fd613fff640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd6000678c0 0x7fd600069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:41.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.956+0000 7fd613fff640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd6000678c0 0x7fd600069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd6040ab000 tx=0x7fd6040a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:41.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:41.956+0000 7fd610ff9640 1 -- 172.21.15.29:0/3457925763 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd6080564c0 con 0x7fd6141e5ae0 2024-04-25T00:31:42.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.120+0000 7fd61a2e9640 1 -- 172.21.15.29:0/3457925763 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd614142680 con 0x7fd6000678c0 2024-04-25T00:31:42.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.136+0000 7fd610ff9640 1 -- 172.21.15.29:0/3457925763 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7fd614142680 con 0x7fd6000678c0 2024-04-25T00:31:42.143 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:42.144 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:31:42.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.140+0000 7fd61a2e9640 1 -- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd6000678c0 msgr2=0x7fd600069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.140+0000 7fd61a2e9640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd6000678c0 0x7fd600069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd6040ab000 tx=0x7fd6040a7070 comp rx=0 tx=0).stop 2024-04-25T00:31:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.140+0000 7fd61a2e9640 1 -- 172.21.15.29:0/3457925763 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6141e5ae0 msgr2=0x7fd6140b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.140+0000 7fd61a2e9640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6141e5ae0 0x7fd6140b01f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd60800e480 tx=0x7fd60800e940 comp rx=0 tx=0).stop 2024-04-25T00:31:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.140+0000 7fd61a2e9640 1 -- 172.21.15.29:0/3457925763 shutdown_connections 2024-04-25T00:31:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.140+0000 7fd61a2e9640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6141e5f60 0x7fd6140b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.140+0000 7fd61a2e9640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6141e5ae0 0x7fd6140b01f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.140+0000 7fd61a2e9640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd6000678c0 0x7fd600069d80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.140+0000 7fd61a2e9640 1 --2- 172.21.15.29:0/3457925763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc003a10 0x7fd614158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.140+0000 7fd61a2e9640 1 -- 172.21.15.29:0/3457925763 >> 172.21.15.29:0/3457925763 conn(0x7fd6140b1870 msgr2=0x7fd61415b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.144+0000 7fd61a2e9640 1 -- 172.21.15.29:0/3457925763 shutdown_connections 2024-04-25T00:31:42.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.144+0000 7fd61a2e9640 1 -- 172.21.15.29:0/3457925763 wait complete. 2024-04-25T00:31:42.161 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-25T00:31:41.447533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252264,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722456,"statfs":{"total":773094113280,"available":772835794944,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.158100"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339973+0000","last_change":"2024-04-25T00:31:30.311935+0000","last_active":"2024-04-25T00:31:32.339973+0000","last_peered":"2024-04-25T00:31:32.339973+0000","last_clean":"2024-04-25T00:31:32.339973+0000","last_became_active":"2024-04-25T00:31:30.311276+0000","last_became_peered":"2024-04-25T00:31:30.311276+0000","last_unstale":"2024-04-25T00:31:32.339973+0000","last_undegraded":"2024-04-25T00:31:32.339973+0000","last_fullsized":"2024-04-25T00:31:32.339973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:00.435231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289979+0000","last_change":"2024-04-25T00:31:30.286809+0000","last_active":"2024-04-25T00:31:31.289979+0000","last_peered":"2024-04-25T00:31:31.289979+0000","last_clean":"2024-04-25T00:31:31.289979+0000","last_became_active":"2024-04-25T00:31:30.286285+0000","last_became_peered":"2024-04-25T00:31:30.286285+0000","last_unstale":"2024-04-25T00:31:31.289979+0000","last_undegraded":"2024-04-25T00:31:31.289979+0000","last_fullsized":"2024-04-25T00:31:31.289979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339468+0000","last_change":"2024-04-25T00:31:30.295646+0000","last_active":"2024-04-25T00:31:32.339468+0000","last_peered":"2024-04-25T00:31:32.339468+0000","last_clean":"2024-04-25T00:31:32.339468+0000","last_became_active":"2024-04-25T00:31:30.295302+0000","last_became_peered":"2024-04-25T00:31:30.295302+0000","last_unstale":"2024-04-25T00:31:32.339468+0000","last_undegraded":"2024-04-25T00:31:32.339468+0000","last_fullsized":"2024-04-25T00:31:32.339468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293552+0000","last_change":"2024-04-25T00:31:30.317738+0000","last_active":"2024-04-25T00:31:31.293552+0000","last_peered":"2024-04-25T00:31:31.293552+0000","last_clean":"2024-04-25T00:31:31.293552+0000","last_became_active":"2024-04-25T00:31:30.315216+0000","last_became_peered":"2024-04-25T00:31:30.315216+0000","last_unstale":"2024-04-25T00:31:31.293552+0000","last_undegraded":"2024-04-25T00:31:31.293552+0000","last_fullsized":"2024-04-25T00:31:31.293552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.340299+0000","last_change":"2024-04-25T00:31:30.308169+0000","last_active":"2024-04-25T00:31:32.340299+0000","last_peered":"2024-04-25T00:31:32.340299+0000","last_clean":"2024-04-25T00:31:32.340299+0000","last_became_active":"2024-04-25T00:31:30.307068+0000","last_became_peered":"2024-04-25T00:31:30.307068+0000","last_unstale":"2024-04-25T00:31:32.340299+0000","last_undegraded":"2024-04-25T00:31:32.340299+0000","last_fullsized":"2024-04-25T00:31:32.340299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:44.328314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293940+0000","last_change":"2024-04-25T00:31:30.318086+0000","last_active":"2024-04-25T00:31:31.293940+0000","last_peered":"2024-04-25T00:31:31.293940+0000","last_clean":"2024-04-25T00:31:31.293940+0000","last_became_active":"2024-04-25T00:31:30.314553+0000","last_became_peered":"2024-04-25T00:31:30.314553+0000","last_unstale":"2024-04-25T00:31:31.293940+0000","last_undegraded":"2024-04-25T00:31:31.293940+0000","last_fullsized":"2024-04-25T00:31:31.293940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:12.015188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338903+0000","last_change":"2024-04-25T00:31:30.323525+0000","last_active":"2024-04-25T00:31:32.338903+0000","last_peered":"2024-04-25T00:31:32.338903+0000","last_clean":"2024-04-25T00:31:32.338903+0000","last_became_active":"2024-04-25T00:31:30.322985+0000","last_became_peered":"2024-04-25T00:31:30.322985+0000","last_unstale":"2024-04-25T00:31:32.338903+0000","last_undegraded":"2024-04-25T00:31:32.338903+0000","last_fullsized":"2024-04-25T00:31:32.338903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:37.829118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337996+0000","last_change":"2024-04-25T00:31:30.302700+0000","last_active":"2024-04-25T00:31:32.337996+0000","last_peered":"2024-04-25T00:31:32.337996+0000","last_clean":"2024-04-25T00:31:32.337996+0000","last_became_active":"2024-04-25T00:31:30.302417+0000","last_became_peered":"2024-04-25T00:31:30.302417+0000","last_unstale":"2024-04-25T00:31:32.337996+0000","last_undegraded":"2024-04-25T00:31:32.337996+0000","last_fullsized":"2024-04-25T00:31:32.337996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:26.475549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.340068+0000","last_change":"2024-04-25T00:31:30.322870+0000","last_active":"2024-04-25T00:31:32.340068+0000","last_peered":"2024-04-25T00:31:32.340068+0000","last_clean":"2024-04-25T00:31:32.340068+0000","last_became_active":"2024-04-25T00:31:30.322271+0000","last_became_peered":"2024-04-25T00:31:30.322271+0000","last_unstale":"2024-04-25T00:31:32.340068+0000","last_undegraded":"2024-04-25T00:31:32.340068+0000","last_fullsized":"2024-04-25T00:31:32.340068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:27.228158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273989+0000","last_change":"2024-04-25T00:31:30.299652+0000","last_active":"2024-04-25T00:31:32.273989+0000","last_peered":"2024-04-25T00:31:32.273989+0000","last_clean":"2024-04-25T00:31:32.273989+0000","last_became_active":"2024-04-25T00:31:30.298705+0000","last_became_peered":"2024-04-25T00:31:30.298705+0000","last_unstale":"2024-04-25T00:31:32.273989+0000","last_undegraded":"2024-04-25T00:31:32.273989+0000","last_fullsized":"2024-04-25T00:31:32.273989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273870+0000","last_change":"2024-04-25T00:31:30.305174+0000","last_active":"2024-04-25T00:31:32.273870+0000","last_peered":"2024-04-25T00:31:32.273870+0000","last_clean":"2024-04-25T00:31:32.273870+0000","last_became_active":"2024-04-25T00:31:30.299316+0000","last_became_peered":"2024-04-25T00:31:30.299316+0000","last_unstale":"2024-04-25T00:31:32.273870+0000","last_undegraded":"2024-04-25T00:31:32.273870+0000","last_fullsized":"2024-04-25T00:31:32.273870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295466+0000","last_change":"2024-04-25T00:31:30.313575+0000","last_active":"2024-04-25T00:31:31.295466+0000","last_peered":"2024-04-25T00:31:31.295466+0000","last_clean":"2024-04-25T00:31:31.295466+0000","last_became_active":"2024-04-25T00:31:30.312375+0000","last_became_peered":"2024-04-25T00:31:30.312375+0000","last_unstale":"2024-04-25T00:31:31.295466+0000","last_undegraded":"2024-04-25T00:31:31.295466+0000","last_fullsized":"2024-04-25T00:31:31.295466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:07.140320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338585+0000","last_change":"2024-04-25T00:31:30.301894+0000","last_active":"2024-04-25T00:31:32.338585+0000","last_peered":"2024-04-25T00:31:32.338585+0000","last_clean":"2024-04-25T00:31:32.338585+0000","last_became_active":"2024-04-25T00:31:30.301566+0000","last_became_peered":"2024-04-25T00:31:30.301566+0000","last_unstale":"2024-04-25T00:31:32.338585+0000","last_undegraded":"2024-04-25T00:31:32.338585+0000","last_fullsized":"2024-04-25T00:31:32.338585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.057143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339233+0000","last_change":"2024-04-25T00:31:30.295471+0000","last_active":"2024-04-25T00:31:32.339233+0000","last_peered":"2024-04-25T00:31:32.339233+0000","last_clean":"2024-04-25T00:31:32.339233+0000","last_became_active":"2024-04-25T00:31:30.295041+0000","last_became_peered":"2024-04-25T00:31:30.295041+0000","last_unstale":"2024-04-25T00:31:32.339233+0000","last_undegraded":"2024-04-25T00:31:32.339233+0000","last_fullsized":"2024-04-25T00:31:32.339233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:39.536187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289654+0000","last_change":"2024-04-25T00:31:30.329846+0000","last_active":"2024-04-25T00:31:31.289654+0000","last_peered":"2024-04-25T00:31:31.289654+0000","last_clean":"2024-04-25T00:31:31.289654+0000","last_became_active":"2024-04-25T00:31:30.329327+0000","last_became_peered":"2024-04-25T00:31:30.329327+0000","last_unstale":"2024-04-25T00:31:31.289654+0000","last_undegraded":"2024-04-25T00:31:31.289654+0000","last_fullsized":"2024-04-25T00:31:31.289654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293587+0000","last_change":"2024-04-25T00:31:30.317089+0000","last_active":"2024-04-25T00:31:31.293587+0000","last_peered":"2024-04-25T00:31:31.293587+0000","last_clean":"2024-04-25T00:31:31.293587+0000","last_became_active":"2024-04-25T00:31:30.314593+0000","last_became_peered":"2024-04-25T00:31:30.314593+0000","last_unstale":"2024-04-25T00:31:31.293587+0000","last_undegraded":"2024-04-25T00:31:31.293587+0000","last_fullsized":"2024-04-25T00:31:31.293587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.057143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295132+0000","last_change":"2024-04-25T00:31:30.330739+0000","last_active":"2024-04-25T00:31:31.295132+0000","last_peered":"2024-04-25T00:31:31.295132+0000","last_clean":"2024-04-25T00:31:31.295132+0000","last_became_active":"2024-04-25T00:31:30.330173+0000","last_became_peered":"2024-04-25T00:31:30.330173+0000","last_unstale":"2024-04-25T00:31:31.295132+0000","last_undegraded":"2024-04-25T00:31:31.295132+0000","last_fullsized":"2024-04-25T00:31:31.295132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:51.764905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337280+0000","last_change":"2024-04-25T00:31:30.310551+0000","last_active":"2024-04-25T00:31:32.337280+0000","last_peered":"2024-04-25T00:31:32.337280+0000","last_clean":"2024-04-25T00:31:32.337280+0000","last_became_active":"2024-04-25T00:31:30.310236+0000","last_became_peered":"2024-04-25T00:31:30.310236+0000","last_unstale":"2024-04-25T00:31:32.337280+0000","last_undegraded":"2024-04-25T00:31:32.337280+0000","last_fullsized":"2024-04-25T00:31:32.337280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:50.429652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339723+0000","last_change":"2024-04-25T00:31:30.318248+0000","last_active":"2024-04-25T00:31:32.339723+0000","last_peered":"2024-04-25T00:31:32.339723+0000","last_clean":"2024-04-25T00:31:32.339723+0000","last_became_active":"2024-04-25T00:31:30.317643+0000","last_became_peered":"2024-04-25T00:31:30.317643+0000","last_unstale":"2024-04-25T00:31:32.339723+0000","last_undegraded":"2024-04-25T00:31:32.339723+0000","last_fullsized":"2024-04-25T00:31:32.339723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:02.546759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273823+0000","last_change":"2024-04-25T00:31:30.316790+0000","last_active":"2024-04-25T00:31:32.273823+0000","last_peered":"2024-04-25T00:31:32.273823+0000","last_clean":"2024-04-25T00:31:32.273823+0000","last_became_active":"2024-04-25T00:31:30.315664+0000","last_became_peered":"2024-04-25T00:31:30.315664+0000","last_unstale":"2024-04-25T00:31:32.273823+0000","last_undegraded":"2024-04-25T00:31:32.273823+0000","last_fullsized":"2024-04-25T00:31:32.273823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339315+0000","last_change":"2024-04-25T00:31:30.322906+0000","last_active":"2024-04-25T00:31:32.339315+0000","last_peered":"2024-04-25T00:31:32.339315+0000","last_clean":"2024-04-25T00:31:32.339315+0000","last_became_active":"2024-04-25T00:31:30.322626+0000","last_became_peered":"2024-04-25T00:31:30.322626+0000","last_unstale":"2024-04-25T00:31:32.339315+0000","last_undegraded":"2024-04-25T00:31:32.339315+0000","last_fullsized":"2024-04-25T00:31:32.339315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:26.475549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293998+0000","last_change":"2024-04-25T00:31:30.327130+0000","last_active":"2024-04-25T00:31:31.293998+0000","last_peered":"2024-04-25T00:31:31.293998+0000","last_clean":"2024-04-25T00:31:31.293998+0000","last_became_active":"2024-04-25T00:31:30.326309+0000","last_became_peered":"2024-04-25T00:31:30.326309+0000","last_unstale":"2024-04-25T00:31:31.293998+0000","last_undegraded":"2024-04-25T00:31:31.293998+0000","last_fullsized":"2024-04-25T00:31:31.293998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:06.663363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338079+0000","last_change":"2024-04-25T00:31:30.329677+0000","last_active":"2024-04-25T00:31:32.338079+0000","last_peered":"2024-04-25T00:31:32.338079+0000","last_clean":"2024-04-25T00:31:32.338079+0000","last_became_active":"2024-04-25T00:31:30.329278+0000","last_became_peered":"2024-04-25T00:31:30.329278+0000","last_unstale":"2024-04-25T00:31:32.338079+0000","last_undegraded":"2024-04-25T00:31:32.338079+0000","last_fullsized":"2024-04-25T00:31:32.338079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:24.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.292924+0000","last_change":"2024-04-25T00:31:30.317304+0000","last_active":"2024-04-25T00:31:31.292924+0000","last_peered":"2024-04-25T00:31:31.292924+0000","last_clean":"2024-04-25T00:31:31.292924+0000","last_became_active":"2024-04-25T00:31:30.314963+0000","last_became_peered":"2024-04-25T00:31:30.314963+0000","last_unstale":"2024-04-25T00:31:31.292924+0000","last_undegraded":"2024-04-25T00:31:31.292924+0000","last_fullsized":"2024-04-25T00:31:31.292924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338403+0000","last_change":"2024-04-25T00:31:30.329416+0000","last_active":"2024-04-25T00:31:32.338403+0000","last_peered":"2024-04-25T00:31:32.338403+0000","last_clean":"2024-04-25T00:31:32.338403+0000","last_became_active":"2024-04-25T00:31:30.329055+0000","last_became_peered":"2024-04-25T00:31:30.329055+0000","last_unstale":"2024-04-25T00:31:32.338403+0000","last_undegraded":"2024-04-25T00:31:32.338403+0000","last_fullsized":"2024-04-25T00:31:32.338403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.274281+0000","last_change":"2024-04-25T00:31:30.325387+0000","last_active":"2024-04-25T00:31:32.274281+0000","last_peered":"2024-04-25T00:31:32.274281+0000","last_clean":"2024-04-25T00:31:32.274281+0000","last_became_active":"2024-04-25T00:31:30.324786+0000","last_became_peered":"2024-04-25T00:31:30.324786+0000","last_unstale":"2024-04-25T00:31:32.274281+0000","last_undegraded":"2024-04-25T00:31:32.274281+0000","last_fullsized":"2024-04-25T00:31:32.274281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.057143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289883+0000","last_change":"2024-04-25T00:31:30.288677+0000","last_active":"2024-04-25T00:31:31.289883+0000","last_peered":"2024-04-25T00:31:31.289883+0000","last_clean":"2024-04-25T00:31:31.289883+0000","last_became_active":"2024-04-25T00:31:30.288327+0000","last_became_peered":"2024-04-25T00:31:30.288327+0000","last_unstale":"2024-04-25T00:31:31.289883+0000","last_undegraded":"2024-04-25T00:31:31.289883+0000","last_fullsized":"2024-04-25T00:31:31.289883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338740+0000","last_change":"2024-04-25T00:31:30.287147+0000","last_active":"2024-04-25T00:31:32.338740+0000","last_peered":"2024-04-25T00:31:32.338740+0000","last_clean":"2024-04-25T00:31:32.338740+0000","last_became_active":"2024-04-25T00:31:30.286375+0000","last_became_peered":"2024-04-25T00:31:30.286375+0000","last_unstale":"2024-04-25T00:31:32.338740+0000","last_undegraded":"2024-04-25T00:31:32.338740+0000","last_fullsized":"2024-04-25T00:31:32.338740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:06.663363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339232+0000","last_change":"2024-04-25T00:31:30.306816+0000","last_active":"2024-04-25T00:31:32.339232+0000","last_peered":"2024-04-25T00:31:32.339232+0000","last_clean":"2024-04-25T00:31:32.339232+0000","last_became_active":"2024-04-25T00:31:30.306476+0000","last_became_peered":"2024-04-25T00:31:30.306476+0000","last_unstale":"2024-04-25T00:31:32.339232+0000","last_undegraded":"2024-04-25T00:31:32.339232+0000","last_fullsized":"2024-04-25T00:31:32.339232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:30.262696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294366+0000","last_change":"2024-04-25T00:31:30.317416+0000","last_active":"2024-04-25T00:31:31.294366+0000","last_peered":"2024-04-25T00:31:31.294366+0000","last_clean":"2024-04-25T00:31:31.294366+0000","last_became_active":"2024-04-25T00:31:30.315307+0000","last_became_peered":"2024-04-25T00:31:30.315307+0000","last_unstale":"2024-04-25T00:31:31.294366+0000","last_undegraded":"2024-04-25T00:31:31.294366+0000","last_fullsized":"2024-04-25T00:31:31.294366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:56.905979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337140+0000","last_change":"2024-04-25T00:31:30.299856+0000","last_active":"2024-04-25T00:31:32.337140+0000","last_peered":"2024-04-25T00:31:32.337140+0000","last_clean":"2024-04-25T00:31:32.337140+0000","last_became_active":"2024-04-25T00:31:30.299616+0000","last_became_peered":"2024-04-25T00:31:30.299616+0000","last_unstale":"2024-04-25T00:31:32.337140+0000","last_undegraded":"2024-04-25T00:31:32.337140+0000","last_fullsized":"2024-04-25T00:31:32.337140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:08.442361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273483+0000","last_change":"2024-04-25T00:31:30.304725+0000","last_active":"2024-04-25T00:31:32.273483+0000","last_peered":"2024-04-25T00:31:32.273483+0000","last_clean":"2024-04-25T00:31:32.273483+0000","last_became_active":"2024-04-25T00:31:30.298142+0000","last_became_peered":"2024-04-25T00:31:30.298142+0000","last_unstale":"2024-04-25T00:31:32.273483+0000","last_undegraded":"2024-04-25T00:31:32.273483+0000","last_fullsized":"2024-04-25T00:31:32.273483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293069+0000","last_change":"2024-04-25T00:31:30.304440+0000","last_active":"2024-04-25T00:31:31.293069+0000","last_peered":"2024-04-25T00:31:31.293069+0000","last_clean":"2024-04-25T00:31:31.293069+0000","last_became_active":"2024-04-25T00:31:30.303603+0000","last_became_peered":"2024-04-25T00:31:30.303603+0000","last_unstale":"2024-04-25T00:31:31.293069+0000","last_undegraded":"2024-04-25T00:31:31.293069+0000","last_fullsized":"2024-04-25T00:31:31.293069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:13.138372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338669+0000","last_change":"2024-04-25T00:31:29.306305+0000","last_active":"2024-04-25T00:31:32.338669+0000","last_peered":"2024-04-25T00:31:32.338669+0000","last_clean":"2024-04-25T00:31:32.338669+0000","last_became_active":"2024-04-25T00:31:29.305825+0000","last_became_peered":"2024-04-25T00:31:29.305825+0000","last_unstale":"2024-04-25T00:31:32.338669+0000","last_undegraded":"2024-04-25T00:31:32.338669+0000","last_fullsized":"2024-04-25T00:31:32.338669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:12.847032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339405+0000","last_change":"2024-04-25T00:31:30.280578+0000","last_active":"2024-04-25T00:31:32.339405+0000","last_peered":"2024-04-25T00:31:32.339405+0000","last_clean":"2024-04-25T00:31:32.339405+0000","last_became_active":"2024-04-25T00:31:30.278616+0000","last_became_peered":"2024-04-25T00:31:30.278616+0000","last_unstale":"2024-04-25T00:31:32.339405+0000","last_undegraded":"2024-04-25T00:31:32.339405+0000","last_fullsized":"2024-04-25T00:31:32.339405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:24.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273089+0000","last_change":"2024-04-25T00:31:29.295550+0000","last_active":"2024-04-25T00:31:32.273089+0000","last_peered":"2024-04-25T00:31:32.273089+0000","last_clean":"2024-04-25T00:31:32.273089+0000","last_became_active":"2024-04-25T00:31:29.294791+0000","last_became_peered":"2024-04-25T00:31:29.294791+0000","last_unstale":"2024-04-25T00:31:32.273089+0000","last_undegraded":"2024-04-25T00:31:32.273089+0000","last_fullsized":"2024-04-25T00:31:32.273089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:28.847603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338214+0000","last_change":"2024-04-25T00:31:29.295660+0000","last_active":"2024-04-25T00:31:32.338214+0000","last_peered":"2024-04-25T00:31:32.338214+0000","last_clean":"2024-04-25T00:31:32.338214+0000","last_became_active":"2024-04-25T00:31:29.295125+0000","last_became_peered":"2024-04-25T00:31:29.295125+0000","last_unstale":"2024-04-25T00:31:32.338214+0000","last_undegraded":"2024-04-25T00:31:32.338214+0000","last_fullsized":"2024-04-25T00:31:32.338214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:43.126464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338169+0000","last_change":"2024-04-25T00:31:30.301078+0000","last_active":"2024-04-25T00:31:32.338169+0000","last_peered":"2024-04-25T00:31:32.338169+0000","last_clean":"2024-04-25T00:31:32.338169+0000","last_became_active":"2024-04-25T00:31:30.298382+0000","last_became_peered":"2024-04-25T00:31:30.298382+0000","last_unstale":"2024-04-25T00:31:32.338169+0000","last_undegraded":"2024-04-25T00:31:32.338169+0000","last_fullsized":"2024-04-25T00:31:32.338169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293054+0000","last_change":"2024-04-25T00:31:29.273641+0000","last_active":"2024-04-25T00:31:31.293054+0000","last_peered":"2024-04-25T00:31:31.293054+0000","last_clean":"2024-04-25T00:31:31.293054+0000","last_became_active":"2024-04-25T00:31:29.272664+0000","last_became_peered":"2024-04-25T00:31:29.272664+0000","last_unstale":"2024-04-25T00:31:31.293054+0000","last_undegraded":"2024-04-25T00:31:31.293054+0000","last_fullsized":"2024-04-25T00:31:31.293054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:12.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289635+0000","last_change":"2024-04-25T00:31:30.287860+0000","last_active":"2024-04-25T00:31:31.289635+0000","last_peered":"2024-04-25T00:31:31.289635+0000","last_clean":"2024-04-25T00:31:31.289635+0000","last_became_active":"2024-04-25T00:31:30.287518+0000","last_became_peered":"2024-04-25T00:31:30.287518+0000","last_unstale":"2024-04-25T00:31:31.289635+0000","last_undegraded":"2024-04-25T00:31:31.289635+0000","last_fullsized":"2024-04-25T00:31:31.289635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:24.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.274623+0000","last_change":"2024-04-25T00:31:29.300290+0000","last_active":"2024-04-25T00:31:32.274623+0000","last_peered":"2024-04-25T00:31:32.274623+0000","last_clean":"2024-04-25T00:31:32.274623+0000","last_became_active":"2024-04-25T00:31:29.297912+0000","last_became_peered":"2024-04-25T00:31:29.297912+0000","last_unstale":"2024-04-25T00:31:32.274623+0000","last_undegraded":"2024-04-25T00:31:32.274623+0000","last_fullsized":"2024-04-25T00:31:32.274623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:06.128468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294033+0000","last_change":"2024-04-25T00:31:30.323107+0000","last_active":"2024-04-25T00:31:31.294033+0000","last_peered":"2024-04-25T00:31:31.294033+0000","last_clean":"2024-04-25T00:31:31.294033+0000","last_became_active":"2024-04-25T00:31:30.322670+0000","last_became_peered":"2024-04-25T00:31:30.322670+0000","last_unstale":"2024-04-25T00:31:31.294033+0000","last_undegraded":"2024-04-25T00:31:31.294033+0000","last_fullsized":"2024-04-25T00:31:31.294033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:37.829118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294169+0000","last_change":"2024-04-25T00:31:29.297610+0000","last_active":"2024-04-25T00:31:31.294169+0000","last_peered":"2024-04-25T00:31:31.294169+0000","last_clean":"2024-04-25T00:31:31.294169+0000","last_became_active":"2024-04-25T00:31:29.296820+0000","last_became_peered":"2024-04-25T00:31:29.296820+0000","last_unstale":"2024-04-25T00:31:31.294169+0000","last_undegraded":"2024-04-25T00:31:31.294169+0000","last_fullsized":"2024-04-25T00:31:31.294169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:28.847603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339641+0000","last_change":"2024-04-25T00:31:30.306807+0000","last_active":"2024-04-25T00:31:32.339641+0000","last_peered":"2024-04-25T00:31:32.339641+0000","last_clean":"2024-04-25T00:31:32.339641+0000","last_became_active":"2024-04-25T00:31:30.306523+0000","last_became_peered":"2024-04-25T00:31:30.306523+0000","last_unstale":"2024-04-25T00:31:32.339641+0000","last_undegraded":"2024-04-25T00:31:32.339641+0000","last_fullsized":"2024-04-25T00:31:32.339641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289930+0000","last_change":"2024-04-25T00:31:29.296029+0000","last_active":"2024-04-25T00:31:31.289930+0000","last_peered":"2024-04-25T00:31:31.289930+0000","last_clean":"2024-04-25T00:31:31.289930+0000","last_became_active":"2024-04-25T00:31:29.295557+0000","last_became_peered":"2024-04-25T00:31:29.295557+0000","last_unstale":"2024-04-25T00:31:31.289930+0000","last_undegraded":"2024-04-25T00:31:31.289930+0000","last_fullsized":"2024-04-25T00:31:31.289930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:12.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295669+0000","last_change":"2024-04-25T00:31:30.311562+0000","last_active":"2024-04-25T00:31:31.295669+0000","last_peered":"2024-04-25T00:31:31.295669+0000","last_clean":"2024-04-25T00:31:31.295669+0000","last_became_active":"2024-04-25T00:31:30.309517+0000","last_became_peered":"2024-04-25T00:31:30.309517+0000","last_unstale":"2024-04-25T00:31:31.295669+0000","last_undegraded":"2024-04-25T00:31:31.295669+0000","last_fullsized":"2024-04-25T00:31:31.295669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:07.434483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273705+0000","last_change":"2024-04-25T00:31:29.295877+0000","last_active":"2024-04-25T00:31:32.273705+0000","last_peered":"2024-04-25T00:31:32.273705+0000","last_clean":"2024-04-25T00:31:32.273705+0000","last_became_active":"2024-04-25T00:31:29.295208+0000","last_became_peered":"2024-04-25T00:31:29.295208+0000","last_unstale":"2024-04-25T00:31:32.273705+0000","last_undegraded":"2024-04-25T00:31:32.273705+0000","last_fullsized":"2024-04-25T00:31:32.273705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:24.278463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339130+0000","last_change":"2024-04-25T00:31:30.307864+0000","last_active":"2024-04-25T00:31:32.339130+0000","last_peered":"2024-04-25T00:31:32.339130+0000","last_clean":"2024-04-25T00:31:32.339130+0000","last_became_active":"2024-04-25T00:31:30.306945+0000","last_became_peered":"2024-04-25T00:31:30.306945+0000","last_unstale":"2024-04-25T00:31:32.339130+0000","last_undegraded":"2024-04-25T00:31:32.339130+0000","last_fullsized":"2024-04-25T00:31:32.339130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:16.301733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339404+0000","last_change":"2024-04-25T00:31:29.291150+0000","last_active":"2024-04-25T00:31:32.339404+0000","last_peered":"2024-04-25T00:31:32.339404+0000","last_clean":"2024-04-25T00:31:32.339404+0000","last_became_active":"2024-04-25T00:31:29.290597+0000","last_became_peered":"2024-04-25T00:31:29.290597+0000","last_unstale":"2024-04-25T00:31:32.339404+0000","last_undegraded":"2024-04-25T00:31:32.339404+0000","last_fullsized":"2024-04-25T00:31:32.339404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:26.121929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339478+0000","last_change":"2024-04-25T00:31:30.322864+0000","last_active":"2024-04-25T00:31:32.339478+0000","last_peered":"2024-04-25T00:31:32.339478+0000","last_clean":"2024-04-25T00:31:32.339478+0000","last_became_active":"2024-04-25T00:31:30.322561+0000","last_became_peered":"2024-04-25T00:31:30.322561+0000","last_unstale":"2024-04-25T00:31:32.339478+0000","last_undegraded":"2024-04-25T00:31:32.339478+0000","last_fullsized":"2024-04-25T00:31:32.339478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294164+0000","last_change":"2024-04-25T00:31:29.289693+0000","last_active":"2024-04-25T00:31:31.294164+0000","last_peered":"2024-04-25T00:31:31.294164+0000","last_clean":"2024-04-25T00:31:31.294164+0000","last_became_active":"2024-04-25T00:31:29.286958+0000","last_became_peered":"2024-04-25T00:31:29.286958+0000","last_unstale":"2024-04-25T00:31:31.294164+0000","last_undegraded":"2024-04-25T00:31:31.294164+0000","last_fullsized":"2024-04-25T00:31:31.294164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:06.128468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339729+0000","last_change":"2024-04-25T00:31:30.327490+0000","last_active":"2024-04-25T00:31:32.339729+0000","last_peered":"2024-04-25T00:31:32.339729+0000","last_clean":"2024-04-25T00:31:32.339729+0000","last_became_active":"2024-04-25T00:31:30.324807+0000","last_became_peered":"2024-04-25T00:31:30.324807+0000","last_unstale":"2024-04-25T00:31:32.339729+0000","last_undegraded":"2024-04-25T00:31:32.339729+0000","last_fullsized":"2024-04-25T00:31:32.339729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:12.335111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273055+0000","last_change":"2024-04-25T00:31:29.292889+0000","last_active":"2024-04-25T00:31:32.273055+0000","last_peered":"2024-04-25T00:31:32.273055+0000","last_clean":"2024-04-25T00:31:32.273055+0000","last_became_active":"2024-04-25T00:31:29.292160+0000","last_became_peered":"2024-04-25T00:31:29.292160+0000","last_unstale":"2024-04-25T00:31:32.273055+0000","last_undegraded":"2024-04-25T00:31:32.273055+0000","last_fullsized":"2024-04-25T00:31:32.273055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:43.126464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273426+0000","last_change":"2024-04-25T00:31:30.306169+0000","last_active":"2024-04-25T00:31:32.273426+0000","last_peered":"2024-04-25T00:31:32.273426+0000","last_clean":"2024-04-25T00:31:32.273426+0000","last_became_active":"2024-04-25T00:31:30.299130+0000","last_became_peered":"2024-04-25T00:31:30.299130+0000","last_unstale":"2024-04-25T00:31:32.273426+0000","last_undegraded":"2024-04-25T00:31:32.273426+0000","last_fullsized":"2024-04-25T00:31:32.273426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:24.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.336849+0000","last_change":"2024-04-25T00:31:29.273896+0000","last_active":"2024-04-25T00:31:32.336849+0000","last_peered":"2024-04-25T00:31:32.336849+0000","last_clean":"2024-04-25T00:31:32.336849+0000","last_became_active":"2024-04-25T00:31:29.273531+0000","last_became_peered":"2024-04-25T00:31:29.273531+0000","last_unstale":"2024-04-25T00:31:32.336849+0000","last_undegraded":"2024-04-25T00:31:32.336849+0000","last_fullsized":"2024-04-25T00:31:32.336849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:43.126464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338915+0000","last_change":"2024-04-25T00:31:30.314038+0000","last_active":"2024-04-25T00:31:32.338915+0000","last_peered":"2024-04-25T00:31:32.338915+0000","last_clean":"2024-04-25T00:31:32.338915+0000","last_became_active":"2024-04-25T00:31:30.309018+0000","last_became_peered":"2024-04-25T00:31:30.309018+0000","last_unstale":"2024-04-25T00:31:32.338915+0000","last_undegraded":"2024-04-25T00:31:32.338915+0000","last_fullsized":"2024-04-25T00:31:32.338915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:56.905979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295834+0000","last_change":"2024-04-25T00:31:29.286381+0000","last_active":"2024-04-25T00:31:31.295834+0000","last_peered":"2024-04-25T00:31:31.295834+0000","last_clean":"2024-04-25T00:31:31.295834+0000","last_became_active":"2024-04-25T00:31:29.285385+0000","last_became_peered":"2024-04-25T00:31:29.285385+0000","last_unstale":"2024-04-25T00:31:31.295834+0000","last_undegraded":"2024-04-25T00:31:31.295834+0000","last_fullsized":"2024-04-25T00:31:31.295834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:33.874197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339927+0000","last_change":"2024-04-25T00:31:30.313477+0000","last_active":"2024-04-25T00:31:32.339927+0000","last_peered":"2024-04-25T00:31:32.339927+0000","last_clean":"2024-04-25T00:31:32.339927+0000","last_became_active":"2024-04-25T00:31:30.312831+0000","last_became_peered":"2024-04-25T00:31:30.312831+0000","last_unstale":"2024-04-25T00:31:32.339927+0000","last_undegraded":"2024-04-25T00:31:32.339927+0000","last_fullsized":"2024-04-25T00:31:32.339927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:36.378500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337220+0000","last_change":"2024-04-25T00:31:29.272826+0000","last_active":"2024-04-25T00:31:32.337220+0000","last_peered":"2024-04-25T00:31:32.337220+0000","last_clean":"2024-04-25T00:31:32.337220+0000","last_became_active":"2024-04-25T00:31:29.272432+0000","last_became_peered":"2024-04-25T00:31:29.272432+0000","last_unstale":"2024-04-25T00:31:32.337220+0000","last_undegraded":"2024-04-25T00:31:32.337220+0000","last_fullsized":"2024-04-25T00:31:32.337220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:24.278463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.340202+0000","last_change":"2024-04-25T00:31:30.312201+0000","last_active":"2024-04-25T00:31:32.340202+0000","last_peered":"2024-04-25T00:31:32.340202+0000","last_clean":"2024-04-25T00:31:32.340202+0000","last_became_active":"2024-04-25T00:31:30.311412+0000","last_became_peered":"2024-04-25T00:31:30.311412+0000","last_unstale":"2024-04-25T00:31:32.340202+0000","last_undegraded":"2024-04-25T00:31:32.340202+0000","last_fullsized":"2024-04-25T00:31:32.340202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:36.992645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.290023+0000","last_change":"2024-04-25T00:31:29.289893+0000","last_active":"2024-04-25T00:31:31.290023+0000","last_peered":"2024-04-25T00:31:31.290023+0000","last_clean":"2024-04-25T00:31:31.290023+0000","last_became_active":"2024-04-25T00:31:29.289495+0000","last_became_peered":"2024-04-25T00:31:29.289495+0000","last_unstale":"2024-04-25T00:31:31.290023+0000","last_undegraded":"2024-04-25T00:31:31.290023+0000","last_fullsized":"2024-04-25T00:31:31.290023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:06.128468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294386+0000","last_change":"2024-04-25T00:31:30.314359+0000","last_active":"2024-04-25T00:31:31.294386+0000","last_peered":"2024-04-25T00:31:31.294386+0000","last_clean":"2024-04-25T00:31:31.294386+0000","last_became_active":"2024-04-25T00:31:30.313625+0000","last_became_peered":"2024-04-25T00:31:30.313625+0000","last_unstale":"2024-04-25T00:31:31.294386+0000","last_undegraded":"2024-04-25T00:31:31.294386+0000","last_fullsized":"2024-04-25T00:31:31.294386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:16.301733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337138+0000","last_change":"2024-04-25T00:31:29.297325+0000","last_active":"2024-04-25T00:31:32.337138+0000","last_peered":"2024-04-25T00:31:32.337138+0000","last_clean":"2024-04-25T00:31:32.337138+0000","last_became_active":"2024-04-25T00:31:29.296822+0000","last_became_peered":"2024-04-25T00:31:29.296822+0000","last_unstale":"2024-04-25T00:31:32.337138+0000","last_undegraded":"2024-04-25T00:31:32.337138+0000","last_fullsized":"2024-04-25T00:31:32.337138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:28.847603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339313+0000","last_change":"2024-04-25T00:31:30.323788+0000","last_active":"2024-04-25T00:31:32.339313+0000","last_peered":"2024-04-25T00:31:32.339313+0000","last_clean":"2024-04-25T00:31:32.339313+0000","last_became_active":"2024-04-25T00:31:30.323115+0000","last_became_peered":"2024-04-25T00:31:30.323115+0000","last_unstale":"2024-04-25T00:31:32.339313+0000","last_undegraded":"2024-04-25T00:31:32.339313+0000","last_fullsized":"2024-04-25T00:31:32.339313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:51.764905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338672+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:31:32.338672+0000","last_peered":"2024-04-25T00:31:32.338672+0000","last_clean":"2024-04-25T00:31:32.338672+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:31:32.338672+0000","last_undegraded":"2024-04-25T00:31:32.338672+0000","last_fullsized":"2024-04-25T00:31:32.338672+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.292686+0000","last_change":"2024-04-25T00:31:29.287393+0000","last_active":"2024-04-25T00:31:31.292686+0000","last_peered":"2024-04-25T00:31:31.292686+0000","last_clean":"2024-04-25T00:31:31.292686+0000","last_became_active":"2024-04-25T00:31:29.286842+0000","last_became_peered":"2024-04-25T00:31:29.286842+0000","last_unstale":"2024-04-25T00:31:31.292686+0000","last_undegraded":"2024-04-25T00:31:31.292686+0000","last_fullsized":"2024-04-25T00:31:31.292686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:43.126464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338536+0000","last_change":"2024-04-25T00:31:30.302310+0000","last_active":"2024-04-25T00:31:32.338536+0000","last_peered":"2024-04-25T00:31:32.338536+0000","last_clean":"2024-04-25T00:31:32.338536+0000","last_became_active":"2024-04-25T00:31:30.302000+0000","last_became_peered":"2024-04-25T00:31:30.302000+0000","last_unstale":"2024-04-25T00:31:32.338536+0000","last_undegraded":"2024-04-25T00:31:32.338536+0000","last_fullsized":"2024-04-25T00:31:32.338536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338493+0000","last_change":"2024-04-25T00:31:29.308692+0000","last_active":"2024-04-25T00:31:32.338493+0000","last_peered":"2024-04-25T00:31:32.338493+0000","last_clean":"2024-04-25T00:31:32.338493+0000","last_became_active":"2024-04-25T00:31:29.306907+0000","last_became_peered":"2024-04-25T00:31:29.306907+0000","last_unstale":"2024-04-25T00:31:32.338493+0000","last_undegraded":"2024-04-25T00:31:32.338493+0000","last_fullsized":"2024-04-25T00:31:32.338493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:07.430509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339054+0000","last_change":"2024-04-25T00:31:30.278855+0000","last_active":"2024-04-25T00:31:32.339054+0000","last_peered":"2024-04-25T00:31:32.339054+0000","last_clean":"2024-04-25T00:31:32.339054+0000","last_became_active":"2024-04-25T00:31:30.278557+0000","last_became_peered":"2024-04-25T00:31:30.278557+0000","last_unstale":"2024-04-25T00:31:32.339054+0000","last_undegraded":"2024-04-25T00:31:32.339054+0000","last_fullsized":"2024-04-25T00:31:32.339054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:00.643971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337108+0000","last_change":"2024-04-25T00:31:29.271815+0000","last_active":"2024-04-25T00:31:32.337108+0000","last_peered":"2024-04-25T00:31:32.337108+0000","last_clean":"2024-04-25T00:31:32.337108+0000","last_became_active":"2024-04-25T00:31:29.271154+0000","last_became_peered":"2024-04-25T00:31:29.271154+0000","last_unstale":"2024-04-25T00:31:32.337108+0000","last_undegraded":"2024-04-25T00:31:32.337108+0000","last_fullsized":"2024-04-25T00:31:32.337108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:12.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289615+0000","last_change":"2024-04-25T00:31:30.287295+0000","last_active":"2024-04-25T00:31:31.289615+0000","last_peered":"2024-04-25T00:31:31.289615+0000","last_clean":"2024-04-25T00:31:31.289615+0000","last_became_active":"2024-04-25T00:31:30.285530+0000","last_became_peered":"2024-04-25T00:31:30.285530+0000","last_unstale":"2024-04-25T00:31:31.289615+0000","last_undegraded":"2024-04-25T00:31:31.289615+0000","last_fullsized":"2024-04-25T00:31:31.289615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:26.475549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294485+0000","last_change":"2024-04-25T00:31:29.288097+0000","last_active":"2024-04-25T00:31:31.294485+0000","last_peered":"2024-04-25T00:31:31.294485+0000","last_clean":"2024-04-25T00:31:31.294485+0000","last_became_active":"2024-04-25T00:31:29.286580+0000","last_became_peered":"2024-04-25T00:31:29.286580+0000","last_unstale":"2024-04-25T00:31:31.294485+0000","last_undegraded":"2024-04-25T00:31:31.294485+0000","last_fullsized":"2024-04-25T00:31:31.294485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:24.278463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294183+0000","last_change":"2024-04-25T00:31:30.323523+0000","last_active":"2024-04-25T00:31:31.294183+0000","last_peered":"2024-04-25T00:31:31.294183+0000","last_clean":"2024-04-25T00:31:31.294183+0000","last_became_active":"2024-04-25T00:31:30.310509+0000","last_became_peered":"2024-04-25T00:31:30.310509+0000","last_unstale":"2024-04-25T00:31:31.294183+0000","last_undegraded":"2024-04-25T00:31:31.294183+0000","last_fullsized":"2024-04-25T00:31:31.294183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:25.290315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289773+0000","last_change":"2024-04-25T00:31:29.283436+0000","last_active":"2024-04-25T00:31:31.289773+0000","last_peered":"2024-04-25T00:31:31.289773+0000","last_clean":"2024-04-25T00:31:31.289773+0000","last_became_active":"2024-04-25T00:31:29.283038+0000","last_became_peered":"2024-04-25T00:31:29.283038+0000","last_unstale":"2024-04-25T00:31:31.289773+0000","last_undegraded":"2024-04-25T00:31:31.289773+0000","last_fullsized":"2024-04-25T00:31:31.289773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:28.847603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295871+0000","last_change":"2024-04-25T00:31:30.311973+0000","last_active":"2024-04-25T00:31:31.295871+0000","last_peered":"2024-04-25T00:31:31.295871+0000","last_clean":"2024-04-25T00:31:31.295871+0000","last_became_active":"2024-04-25T00:31:30.311141+0000","last_became_peered":"2024-04-25T00:31:30.311141+0000","last_unstale":"2024-04-25T00:31:31.295871+0000","last_undegraded":"2024-04-25T00:31:31.295871+0000","last_fullsized":"2024-04-25T00:31:31.295871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:12.335111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339611+0000","last_change":"2024-04-25T00:31:29.309713+0000","last_active":"2024-04-25T00:31:32.339611+0000","last_peered":"2024-04-25T00:31:32.339611+0000","last_clean":"2024-04-25T00:31:32.339611+0000","last_became_active":"2024-04-25T00:31:29.309033+0000","last_became_peered":"2024-04-25T00:31:29.309033+0000","last_unstale":"2024-04-25T00:31:32.339611+0000","last_undegraded":"2024-04-25T00:31:32.339611+0000","last_fullsized":"2024-04-25T00:31:32.339611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:59.632119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289794+0000","last_change":"2024-04-25T00:31:30.285762+0000","last_active":"2024-04-25T00:31:31.289794+0000","last_peered":"2024-04-25T00:31:31.289794+0000","last_clean":"2024-04-25T00:31:31.289794+0000","last_became_active":"2024-04-25T00:31:30.284685+0000","last_became_peered":"2024-04-25T00:31:30.284685+0000","last_unstale":"2024-04-25T00:31:31.289794+0000","last_undegraded":"2024-04-25T00:31:31.289794+0000","last_fullsized":"2024-04-25T00:31:31.289794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.290064+0000","last_change":"2024-04-25T00:31:29.296589+0000","last_active":"2024-04-25T00:31:31.290064+0000","last_peered":"2024-04-25T00:31:31.290064+0000","last_clean":"2024-04-25T00:31:31.290064+0000","last_became_active":"2024-04-25T00:31:29.296082+0000","last_became_peered":"2024-04-25T00:31:29.296082+0000","last_unstale":"2024-04-25T00:31:31.290064+0000","last_undegraded":"2024-04-25T00:31:31.290064+0000","last_fullsized":"2024-04-25T00:31:31.290064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:06.422631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338590+0000","last_change":"2024-04-25T00:31:30.318568+0000","last_active":"2024-04-25T00:31:32.338590+0000","last_peered":"2024-04-25T00:31:32.338590+0000","last_clean":"2024-04-25T00:31:32.338590+0000","last_became_active":"2024-04-25T00:31:30.318203+0000","last_became_peered":"2024-04-25T00:31:30.318203+0000","last_unstale":"2024-04-25T00:31:32.338590+0000","last_undegraded":"2024-04-25T00:31:32.338590+0000","last_fullsized":"2024-04-25T00:31:32.338590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:12.015188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.274864+0000","last_change":"2024-04-25T00:31:29.310321+0000","last_active":"2024-04-25T00:31:32.274864+0000","last_peered":"2024-04-25T00:31:32.274864+0000","last_clean":"2024-04-25T00:31:32.274864+0000","last_became_active":"2024-04-25T00:31:29.309796+0000","last_became_peered":"2024-04-25T00:31:29.309796+0000","last_unstale":"2024-04-25T00:31:32.274864+0000","last_undegraded":"2024-04-25T00:31:32.274864+0000","last_fullsized":"2024-04-25T00:31:32.274864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:12.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339672+0000","last_change":"2024-04-25T00:31:30.306140+0000","last_active":"2024-04-25T00:31:32.339672+0000","last_peered":"2024-04-25T00:31:32.339672+0000","last_clean":"2024-04-25T00:31:32.339672+0000","last_became_active":"2024-04-25T00:31:30.294939+0000","last_became_peered":"2024-04-25T00:31:30.294939+0000","last_unstale":"2024-04-25T00:31:32.339672+0000","last_undegraded":"2024-04-25T00:31:32.339672+0000","last_fullsized":"2024-04-25T00:31:32.339672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338396+0000","last_change":"2024-04-25T00:31:29.313166+0000","last_active":"2024-04-25T00:31:32.338396+0000","last_peered":"2024-04-25T00:31:32.338396+0000","last_clean":"2024-04-25T00:31:32.338396+0000","last_became_active":"2024-04-25T00:31:29.308641+0000","last_became_peered":"2024-04-25T00:31:29.308641+0000","last_unstale":"2024-04-25T00:31:32.338396+0000","last_undegraded":"2024-04-25T00:31:32.338396+0000","last_fullsized":"2024-04-25T00:31:32.338396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:06.422631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.296415+0000","last_change":"2024-04-25T00:31:30.303363+0000","last_active":"2024-04-25T00:31:31.296415+0000","last_peered":"2024-04-25T00:31:31.296415+0000","last_clean":"2024-04-25T00:31:31.296415+0000","last_became_active":"2024-04-25T00:31:30.302602+0000","last_became_peered":"2024-04-25T00:31:30.302602+0000","last_unstale":"2024-04-25T00:31:31.296415+0000","last_undegraded":"2024-04-25T00:31:31.296415+0000","last_fullsized":"2024-04-25T00:31:31.296415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:08.442361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289690+0000","last_change":"2024-04-25T00:31:29.306640+0000","last_active":"2024-04-25T00:31:31.289690+0000","last_peered":"2024-04-25T00:31:31.289690+0000","last_clean":"2024-04-25T00:31:31.289690+0000","last_became_active":"2024-04-25T00:31:29.306117+0000","last_became_peered":"2024-04-25T00:31:29.306117+0000","last_unstale":"2024-04-25T00:31:31.289690+0000","last_undegraded":"2024-04-25T00:31:31.289690+0000","last_fullsized":"2024-04-25T00:31:31.289690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:07.430509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338904+0000","last_change":"2024-04-25T00:31:30.287149+0000","last_active":"2024-04-25T00:31:32.338904+0000","last_peered":"2024-04-25T00:31:32.338904+0000","last_clean":"2024-04-25T00:31:32.338904+0000","last_became_active":"2024-04-25T00:31:30.286140+0000","last_became_peered":"2024-04-25T00:31:30.286140+0000","last_unstale":"2024-04-25T00:31:32.338904+0000","last_undegraded":"2024-04-25T00:31:32.338904+0000","last_fullsized":"2024-04-25T00:31:32.338904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:37.829118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.290109+0000","last_change":"2024-04-25T00:31:29.278390+0000","last_active":"2024-04-25T00:31:31.290109+0000","last_peered":"2024-04-25T00:31:31.290109+0000","last_clean":"2024-04-25T00:31:31.290109+0000","last_became_active":"2024-04-25T00:31:29.277901+0000","last_became_peered":"2024-04-25T00:31:29.277901+0000","last_unstale":"2024-04-25T00:31:31.290109+0000","last_undegraded":"2024-04-25T00:31:31.290109+0000","last_fullsized":"2024-04-25T00:31:31.290109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:24.278463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338701+0000","last_change":"2024-04-25T00:31:30.298582+0000","last_active":"2024-04-25T00:31:32.338701+0000","last_peered":"2024-04-25T00:31:32.338701+0000","last_clean":"2024-04-25T00:31:32.338701+0000","last_became_active":"2024-04-25T00:31:30.298289+0000","last_became_peered":"2024-04-25T00:31:30.298289+0000","last_unstale":"2024-04-25T00:31:32.338701+0000","last_undegraded":"2024-04-25T00:31:32.338701+0000","last_fullsized":"2024-04-25T00:31:32.338701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:50.437582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339895+0000","last_change":"2024-04-25T00:31:29.309564+0000","last_active":"2024-04-25T00:31:32.339895+0000","last_peered":"2024-04-25T00:31:32.339895+0000","last_clean":"2024-04-25T00:31:32.339895+0000","last_became_active":"2024-04-25T00:31:29.308824+0000","last_became_peered":"2024-04-25T00:31:29.308824+0000","last_unstale":"2024-04-25T00:31:32.339895+0000","last_undegraded":"2024-04-25T00:31:32.339895+0000","last_fullsized":"2024-04-25T00:31:32.339895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:49.417800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273411+0000","last_change":"2024-04-25T00:31:30.316035+0000","last_active":"2024-04-25T00:31:32.273411+0000","last_peered":"2024-04-25T00:31:32.273411+0000","last_clean":"2024-04-25T00:31:32.273411+0000","last_became_active":"2024-04-25T00:31:30.315263+0000","last_became_peered":"2024-04-25T00:31:30.315263+0000","last_unstale":"2024-04-25T00:31:32.273411+0000","last_undegraded":"2024-04-25T00:31:32.273411+0000","last_fullsized":"2024-04-25T00:31:32.273411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:26.475549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295648+0000","last_change":"2024-04-25T00:31:30.329587+0000","last_active":"2024-04-25T00:31:31.295648+0000","last_peered":"2024-04-25T00:31:31.295648+0000","last_clean":"2024-04-25T00:31:31.295648+0000","last_became_active":"2024-04-25T00:31:30.329014+0000","last_became_peered":"2024-04-25T00:31:30.329014+0000","last_unstale":"2024-04-25T00:31:31.295648+0000","last_undegraded":"2024-04-25T00:31:31.295648+0000","last_fullsized":"2024-04-25T00:31:31.295648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:02.546759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338944+0000","last_change":"2024-04-25T00:31:29.307154+0000","last_active":"2024-04-25T00:31:32.338944+0000","last_peered":"2024-04-25T00:31:32.338944+0000","last_clean":"2024-04-25T00:31:32.338944+0000","last_became_active":"2024-04-25T00:31:29.306693+0000","last_became_peered":"2024-04-25T00:31:29.306693+0000","last_unstale":"2024-04-25T00:31:32.338944+0000","last_undegraded":"2024-04-25T00:31:32.338944+0000","last_fullsized":"2024-04-25T00:31:32.338944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:20.941312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293968+0000","last_change":"2024-04-25T00:31:30.329462+0000","last_active":"2024-04-25T00:31:31.293968+0000","last_peered":"2024-04-25T00:31:31.293968+0000","last_clean":"2024-04-25T00:31:31.293968+0000","last_became_active":"2024-04-25T00:31:30.328941+0000","last_became_peered":"2024-04-25T00:31:30.328941+0000","last_unstale":"2024-04-25T00:31:31.293968+0000","last_undegraded":"2024-04-25T00:31:31.293968+0000","last_fullsized":"2024-04-25T00:31:31.293968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:50.437582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339296+0000","last_change":"2024-04-25T00:31:29.311168+0000","last_active":"2024-04-25T00:31:32.339296+0000","last_peered":"2024-04-25T00:31:32.339296+0000","last_clean":"2024-04-25T00:31:32.339296+0000","last_became_active":"2024-04-25T00:31:29.310402+0000","last_became_peered":"2024-04-25T00:31:29.310402+0000","last_unstale":"2024-04-25T00:31:32.339296+0000","last_undegraded":"2024-04-25T00:31:32.339296+0000","last_fullsized":"2024-04-25T00:31:32.339296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:56.307057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339924+0000","last_change":"2024-04-25T00:31:30.307691+0000","last_active":"2024-04-25T00:31:32.339924+0000","last_peered":"2024-04-25T00:31:32.339924+0000","last_clean":"2024-04-25T00:31:32.339924+0000","last_became_active":"2024-04-25T00:31:30.307158+0000","last_became_peered":"2024-04-25T00:31:30.307158+0000","last_unstale":"2024-04-25T00:31:32.339924+0000","last_undegraded":"2024-04-25T00:31:32.339924+0000","last_fullsized":"2024-04-25T00:31:32.339924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.057143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338533+0000","last_change":"2024-04-25T00:31:29.282318+0000","last_active":"2024-04-25T00:31:32.338533+0000","last_peered":"2024-04-25T00:31:32.338533+0000","last_clean":"2024-04-25T00:31:32.338533+0000","last_became_active":"2024-04-25T00:31:29.281826+0000","last_became_peered":"2024-04-25T00:31:29.281826+0000","last_unstale":"2024-04-25T00:31:32.338533+0000","last_undegraded":"2024-04-25T00:31:32.338533+0000","last_fullsized":"2024-04-25T00:31:32.338533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:26.121929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338333+0000","last_change":"2024-04-25T00:31:30.302619+0000","last_active":"2024-04-25T00:31:32.338333+0000","last_peered":"2024-04-25T00:31:32.338333+0000","last_clean":"2024-04-25T00:31:32.338333+0000","last_became_active":"2024-04-25T00:31:30.301637+0000","last_became_peered":"2024-04-25T00:31:30.301637+0000","last_unstale":"2024-04-25T00:31:32.338333+0000","last_undegraded":"2024-04-25T00:31:32.338333+0000","last_fullsized":"2024-04-25T00:31:32.338333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339800+0000","last_change":"2024-04-25T00:31:29.311460+0000","last_active":"2024-04-25T00:31:32.339800+0000","last_peered":"2024-04-25T00:31:32.339800+0000","last_clean":"2024-04-25T00:31:32.339800+0000","last_became_active":"2024-04-25T00:31:29.311052+0000","last_became_peered":"2024-04-25T00:31:29.311052+0000","last_unstale":"2024-04-25T00:31:32.339800+0000","last_undegraded":"2024-04-25T00:31:32.339800+0000","last_fullsized":"2024-04-25T00:31:32.339800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:12.847032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:31:42.165 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:31:42.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc321d0640 1 Processor -- start 2024-04-25T00:31:42.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc321d0640 1 -- start start 2024-04-25T00:31:42.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc321d0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc2c15e170 0x7fbc2c161d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc321d0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbc2c15e550 0x7fbc2c15f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc321d0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc2c1622b0 con 0x7fbc2c15f570 2024-04-25T00:31:42.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc321d0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbc2c162580 con 0x7fbc2c15e550 2024-04-25T00:31:42.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc321d0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbc2c162860 con 0x7fbc2c15e170 2024-04-25T00:31:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc2affd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbc2c15e550 0x7fbc2c15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc2affd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbc2c15e550 0x7fbc2c15f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50044/0 (socket says 172.21.15.29:50044) 2024-04-25T00:31:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc2affd640 1 -- 172.21.15.29:0/2145384949 learned_addr learned my addr 172.21.15.29:0/2145384949 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc2b7fe640 1 --2- 172.21.15.29:0/2145384949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc2c15e170 0x7fbc2c161d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc2affd640 1 -- 172.21.15.29:0/2145384949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc2c15e170 msgr2=0x7fbc2c161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc2affd640 1 --2- 172.21.15.29:0/2145384949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc2c15e170 0x7fbc2c161d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc2affd640 1 -- 172.21.15.29:0/2145384949 >> v1:172.21.15.157:6789/0 conn(0x7fbc2c15f570 legacy=0x7fbc2c06b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:42.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.316+0000 7fbc2affd640 1 -- 172.21.15.29:0/2145384949 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c162b40 con 0x7fbc2c15e550 2024-04-25T00:31:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2affd640 1 --2- 172.21.15.29:0/2145384949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc2c15e550 0x7fbc2c15f030 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009870 tx=0x7fbc2001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ddba1fbc584283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2a7fc640 1 -- 172.21.15.29:0/2145384949 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200b5070 con 0x7fbc2c15e550 2024-04-25T00:31:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2a7fc640 1 -- 172.21.15.29:0/2145384949 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc200a89d0 con 0x7fbc2c15e550 2024-04-25T00:31:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2a7fc640 1 -- 172.21.15.29:0/2145384949 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200a8da0 con 0x7fbc2c15e550 2024-04-25T00:31:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 -- 172.21.15.29:0/2145384949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc2c15e550 msgr2=0x7fbc2c15f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 --2- 172.21.15.29:0/2145384949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc2c15e550 0x7fbc2c15f030 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009870 tx=0x7fbc2001ed10 comp rx=0 tx=0).stop 2024-04-25T00:31:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 -- 172.21.15.29:0/2145384949 shutdown_connections 2024-04-25T00:31:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 --2- 172.21.15.29:0/2145384949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc2c15e550 0x7fbc2c15f030 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 --2- 172.21.15.29:0/2145384949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc2c15e170 0x7fbc2c161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 -- 172.21.15.29:0/2145384949 >> 172.21.15.29:0/2145384949 conn(0x7fbc2c0d0c30 msgr2=0x7fbc2c0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 -- 172.21.15.29:0/2145384949 shutdown_connections 2024-04-25T00:31:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 -- 172.21.15.29:0/2145384949 wait complete. 2024-04-25T00:31:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 Processor -- start 2024-04-25T00:31:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 -- start start 2024-04-25T00:31:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc2c15e170 0x7fbc2c15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc2c15e550 0x7fbc2c147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc2c157b60 0x7fbc2c161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c06baa0 con 0x7fbc2c15e170 2024-04-25T00:31:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c06b920 con 0x7fbc2c15e550 2024-04-25T00:31:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbc2c0b01d0 con 0x7fbc2c157b60 2024-04-25T00:31:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc2c15e170 0x7fbc2c15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc2c15e170 0x7fbc2c15d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50052/0 (socket says 172.21.15.29:50052) 2024-04-25T00:31:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2b7fe640 1 -- 172.21.15.29:0/2239922979 learned_addr learned my addr 172.21.15.29:0/2239922979 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2affd640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc2c15e550 0x7fbc2c147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2bfff640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc2c157b60 0x7fbc2c161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2b7fe640 1 -- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc2c157b60 msgr2=0x7fbc2c161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2b7fe640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc2c157b60 0x7fbc2c161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2b7fe640 1 -- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc2c15e550 msgr2=0x7fbc2c147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2b7fe640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc2c15e550 0x7fbc2c147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2b7fe640 1 -- 172.21.15.29:0/2239922979 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c15ff90 con 0x7fbc2c15e170 2024-04-25T00:31:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2affd640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc2c15e550 0x7fbc2c147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc2b7fe640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc2c15e170 0x7fbc2c15d2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00d800 tx=0x7fbc1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc13fff640 1 -- 172.21.15.29:0/2239922979 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00b6c0 con 0x7fbc2c15e170 2024-04-25T00:31:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc13fff640 1 -- 172.21.15.29:0/2239922979 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc1c019070 con 0x7fbc2c15e170 2024-04-25T00:31:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 -- 172.21.15.29:0/2239922979 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c160270 con 0x7fbc2c15e170 2024-04-25T00:31:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc13fff640 1 -- 172.21.15.29:0/2239922979 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00bae0 con 0x7fbc2c15e170 2024-04-25T00:31:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.320+0000 7fbc321d0640 1 -- 172.21.15.29:0/2239922979 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c160830 con 0x7fbc2c15e170 2024-04-25T00:31:42.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.324+0000 7fbc321d0640 1 -- 172.21.15.29:0/2239922979 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2c069270 con 0x7fbc2c15e170 2024-04-25T00:31:42.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.324+0000 7fbc13fff640 1 -- 172.21.15.29:0/2239922979 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbc1c00bc80 con 0x7fbc2c15e170 2024-04-25T00:31:42.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.324+0000 7fbc13fff640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbc180678f0 0x7fbc18069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.324+0000 7fbc13fff640 1 -- 172.21.15.29:0/2239922979 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbc1c0860f0 con 0x7fbc2c15e170 2024-04-25T00:31:42.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.324+0000 7fbc2affd640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbc180678f0 0x7fbc18069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.324+0000 7fbc2affd640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbc180678f0 0x7fbc18069db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbc200b2000 tx=0x7fbc20008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:42.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.328+0000 7fbc13fff640 1 -- 172.21.15.29:0/2239922979 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbc1c056cc0 con 0x7fbc2c15e170 2024-04-25T00:31:42.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.480+0000 7fbc321d0640 1 -- 172.21.15.29:0/2239922979 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbc2c0b1960 con 0x7fbc180678f0 2024-04-25T00:31:42.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.496+0000 7fbc13fff640 1 -- 172.21.15.29:0/2239922979 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7fbc2c0b1960 con 0x7fbc180678f0 2024-04-25T00:31:42.501 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:42.502 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:31:42.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.500+0000 7fbc321d0640 1 -- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbc180678f0 msgr2=0x7fbc18069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.500+0000 7fbc321d0640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbc180678f0 0x7fbc18069db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbc200b2000 tx=0x7fbc20008040 comp rx=0 tx=0).stop 2024-04-25T00:31:42.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.500+0000 7fbc321d0640 1 -- 172.21.15.29:0/2239922979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc2c15e170 msgr2=0x7fbc2c15d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.500+0000 7fbc321d0640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc2c15e170 0x7fbc2c15d2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00d800 tx=0x7fbc1c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.500+0000 7fbc321d0640 1 -- 172.21.15.29:0/2239922979 shutdown_connections 2024-04-25T00:31:42.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.500+0000 7fbc321d0640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc2c157b60 0x7fbc2c161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.500+0000 7fbc321d0640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbc180678f0 0x7fbc18069db0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.500+0000 7fbc321d0640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc2c15e550 0x7fbc2c147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.500+0000 7fbc321d0640 1 --2- 172.21.15.29:0/2239922979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc2c15e170 0x7fbc2c15d2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.500+0000 7fbc321d0640 1 -- 172.21.15.29:0/2239922979 >> 172.21.15.29:0/2239922979 conn(0x7fbc2c0d0c30 msgr2=0x7fbc2c0c6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:42.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.500+0000 7fbc321d0640 1 -- 172.21.15.29:0/2239922979 shutdown_connections 2024-04-25T00:31:42.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.500+0000 7fbc321d0640 1 -- 172.21.15.29:0/2239922979 wait complete. 2024-04-25T00:31:42.517 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-25T00:31:41.447533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252264,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722456,"statfs":{"total":773094113280,"available":772835794944,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.158100"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339973+0000","last_change":"2024-04-25T00:31:30.311935+0000","last_active":"2024-04-25T00:31:32.339973+0000","last_peered":"2024-04-25T00:31:32.339973+0000","last_clean":"2024-04-25T00:31:32.339973+0000","last_became_active":"2024-04-25T00:31:30.311276+0000","last_became_peered":"2024-04-25T00:31:30.311276+0000","last_unstale":"2024-04-25T00:31:32.339973+0000","last_undegraded":"2024-04-25T00:31:32.339973+0000","last_fullsized":"2024-04-25T00:31:32.339973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:00.435231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289979+0000","last_change":"2024-04-25T00:31:30.286809+0000","last_active":"2024-04-25T00:31:31.289979+0000","last_peered":"2024-04-25T00:31:31.289979+0000","last_clean":"2024-04-25T00:31:31.289979+0000","last_became_active":"2024-04-25T00:31:30.286285+0000","last_became_peered":"2024-04-25T00:31:30.286285+0000","last_unstale":"2024-04-25T00:31:31.289979+0000","last_undegraded":"2024-04-25T00:31:31.289979+0000","last_fullsized":"2024-04-25T00:31:31.289979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339468+0000","last_change":"2024-04-25T00:31:30.295646+0000","last_active":"2024-04-25T00:31:32.339468+0000","last_peered":"2024-04-25T00:31:32.339468+0000","last_clean":"2024-04-25T00:31:32.339468+0000","last_became_active":"2024-04-25T00:31:30.295302+0000","last_became_peered":"2024-04-25T00:31:30.295302+0000","last_unstale":"2024-04-25T00:31:32.339468+0000","last_undegraded":"2024-04-25T00:31:32.339468+0000","last_fullsized":"2024-04-25T00:31:32.339468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293552+0000","last_change":"2024-04-25T00:31:30.317738+0000","last_active":"2024-04-25T00:31:31.293552+0000","last_peered":"2024-04-25T00:31:31.293552+0000","last_clean":"2024-04-25T00:31:31.293552+0000","last_became_active":"2024-04-25T00:31:30.315216+0000","last_became_peered":"2024-04-25T00:31:30.315216+0000","last_unstale":"2024-04-25T00:31:31.293552+0000","last_undegraded":"2024-04-25T00:31:31.293552+0000","last_fullsized":"2024-04-25T00:31:31.293552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.340299+0000","last_change":"2024-04-25T00:31:30.308169+0000","last_active":"2024-04-25T00:31:32.340299+0000","last_peered":"2024-04-25T00:31:32.340299+0000","last_clean":"2024-04-25T00:31:32.340299+0000","last_became_active":"2024-04-25T00:31:30.307068+0000","last_became_peered":"2024-04-25T00:31:30.307068+0000","last_unstale":"2024-04-25T00:31:32.340299+0000","last_undegraded":"2024-04-25T00:31:32.340299+0000","last_fullsized":"2024-04-25T00:31:32.340299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:44.328314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293940+0000","last_change":"2024-04-25T00:31:30.318086+0000","last_active":"2024-04-25T00:31:31.293940+0000","last_peered":"2024-04-25T00:31:31.293940+0000","last_clean":"2024-04-25T00:31:31.293940+0000","last_became_active":"2024-04-25T00:31:30.314553+0000","last_became_peered":"2024-04-25T00:31:30.314553+0000","last_unstale":"2024-04-25T00:31:31.293940+0000","last_undegraded":"2024-04-25T00:31:31.293940+0000","last_fullsized":"2024-04-25T00:31:31.293940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:12.015188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338903+0000","last_change":"2024-04-25T00:31:30.323525+0000","last_active":"2024-04-25T00:31:32.338903+0000","last_peered":"2024-04-25T00:31:32.338903+0000","last_clean":"2024-04-25T00:31:32.338903+0000","last_became_active":"2024-04-25T00:31:30.322985+0000","last_became_peered":"2024-04-25T00:31:30.322985+0000","last_unstale":"2024-04-25T00:31:32.338903+0000","last_undegraded":"2024-04-25T00:31:32.338903+0000","last_fullsized":"2024-04-25T00:31:32.338903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:37.829118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337996+0000","last_change":"2024-04-25T00:31:30.302700+0000","last_active":"2024-04-25T00:31:32.337996+0000","last_peered":"2024-04-25T00:31:32.337996+0000","last_clean":"2024-04-25T00:31:32.337996+0000","last_became_active":"2024-04-25T00:31:30.302417+0000","last_became_peered":"2024-04-25T00:31:30.302417+0000","last_unstale":"2024-04-25T00:31:32.337996+0000","last_undegraded":"2024-04-25T00:31:32.337996+0000","last_fullsized":"2024-04-25T00:31:32.337996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:26.475549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.340068+0000","last_change":"2024-04-25T00:31:30.322870+0000","last_active":"2024-04-25T00:31:32.340068+0000","last_peered":"2024-04-25T00:31:32.340068+0000","last_clean":"2024-04-25T00:31:32.340068+0000","last_became_active":"2024-04-25T00:31:30.322271+0000","last_became_peered":"2024-04-25T00:31:30.322271+0000","last_unstale":"2024-04-25T00:31:32.340068+0000","last_undegraded":"2024-04-25T00:31:32.340068+0000","last_fullsized":"2024-04-25T00:31:32.340068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:27.228158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273989+0000","last_change":"2024-04-25T00:31:30.299652+0000","last_active":"2024-04-25T00:31:32.273989+0000","last_peered":"2024-04-25T00:31:32.273989+0000","last_clean":"2024-04-25T00:31:32.273989+0000","last_became_active":"2024-04-25T00:31:30.298705+0000","last_became_peered":"2024-04-25T00:31:30.298705+0000","last_unstale":"2024-04-25T00:31:32.273989+0000","last_undegraded":"2024-04-25T00:31:32.273989+0000","last_fullsized":"2024-04-25T00:31:32.273989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273870+0000","last_change":"2024-04-25T00:31:30.305174+0000","last_active":"2024-04-25T00:31:32.273870+0000","last_peered":"2024-04-25T00:31:32.273870+0000","last_clean":"2024-04-25T00:31:32.273870+0000","last_became_active":"2024-04-25T00:31:30.299316+0000","last_became_peered":"2024-04-25T00:31:30.299316+0000","last_unstale":"2024-04-25T00:31:32.273870+0000","last_undegraded":"2024-04-25T00:31:32.273870+0000","last_fullsized":"2024-04-25T00:31:32.273870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295466+0000","last_change":"2024-04-25T00:31:30.313575+0000","last_active":"2024-04-25T00:31:31.295466+0000","last_peered":"2024-04-25T00:31:31.295466+0000","last_clean":"2024-04-25T00:31:31.295466+0000","last_became_active":"2024-04-25T00:31:30.312375+0000","last_became_peered":"2024-04-25T00:31:30.312375+0000","last_unstale":"2024-04-25T00:31:31.295466+0000","last_undegraded":"2024-04-25T00:31:31.295466+0000","last_fullsized":"2024-04-25T00:31:31.295466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:07.140320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338585+0000","last_change":"2024-04-25T00:31:30.301894+0000","last_active":"2024-04-25T00:31:32.338585+0000","last_peered":"2024-04-25T00:31:32.338585+0000","last_clean":"2024-04-25T00:31:32.338585+0000","last_became_active":"2024-04-25T00:31:30.301566+0000","last_became_peered":"2024-04-25T00:31:30.301566+0000","last_unstale":"2024-04-25T00:31:32.338585+0000","last_undegraded":"2024-04-25T00:31:32.338585+0000","last_fullsized":"2024-04-25T00:31:32.338585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.057143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339233+0000","last_change":"2024-04-25T00:31:30.295471+0000","last_active":"2024-04-25T00:31:32.339233+0000","last_peered":"2024-04-25T00:31:32.339233+0000","last_clean":"2024-04-25T00:31:32.339233+0000","last_became_active":"2024-04-25T00:31:30.295041+0000","last_became_peered":"2024-04-25T00:31:30.295041+0000","last_unstale":"2024-04-25T00:31:32.339233+0000","last_undegraded":"2024-04-25T00:31:32.339233+0000","last_fullsized":"2024-04-25T00:31:32.339233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:39.536187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289654+0000","last_change":"2024-04-25T00:31:30.329846+0000","last_active":"2024-04-25T00:31:31.289654+0000","last_peered":"2024-04-25T00:31:31.289654+0000","last_clean":"2024-04-25T00:31:31.289654+0000","last_became_active":"2024-04-25T00:31:30.329327+0000","last_became_peered":"2024-04-25T00:31:30.329327+0000","last_unstale":"2024-04-25T00:31:31.289654+0000","last_undegraded":"2024-04-25T00:31:31.289654+0000","last_fullsized":"2024-04-25T00:31:31.289654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293587+0000","last_change":"2024-04-25T00:31:30.317089+0000","last_active":"2024-04-25T00:31:31.293587+0000","last_peered":"2024-04-25T00:31:31.293587+0000","last_clean":"2024-04-25T00:31:31.293587+0000","last_became_active":"2024-04-25T00:31:30.314593+0000","last_became_peered":"2024-04-25T00:31:30.314593+0000","last_unstale":"2024-04-25T00:31:31.293587+0000","last_undegraded":"2024-04-25T00:31:31.293587+0000","last_fullsized":"2024-04-25T00:31:31.293587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.057143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295132+0000","last_change":"2024-04-25T00:31:30.330739+0000","last_active":"2024-04-25T00:31:31.295132+0000","last_peered":"2024-04-25T00:31:31.295132+0000","last_clean":"2024-04-25T00:31:31.295132+0000","last_became_active":"2024-04-25T00:31:30.330173+0000","last_became_peered":"2024-04-25T00:31:30.330173+0000","last_unstale":"2024-04-25T00:31:31.295132+0000","last_undegraded":"2024-04-25T00:31:31.295132+0000","last_fullsized":"2024-04-25T00:31:31.295132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:51.764905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337280+0000","last_change":"2024-04-25T00:31:30.310551+0000","last_active":"2024-04-25T00:31:32.337280+0000","last_peered":"2024-04-25T00:31:32.337280+0000","last_clean":"2024-04-25T00:31:32.337280+0000","last_became_active":"2024-04-25T00:31:30.310236+0000","last_became_peered":"2024-04-25T00:31:30.310236+0000","last_unstale":"2024-04-25T00:31:32.337280+0000","last_undegraded":"2024-04-25T00:31:32.337280+0000","last_fullsized":"2024-04-25T00:31:32.337280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:50.429652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339723+0000","last_change":"2024-04-25T00:31:30.318248+0000","last_active":"2024-04-25T00:31:32.339723+0000","last_peered":"2024-04-25T00:31:32.339723+0000","last_clean":"2024-04-25T00:31:32.339723+0000","last_became_active":"2024-04-25T00:31:30.317643+0000","last_became_peered":"2024-04-25T00:31:30.317643+0000","last_unstale":"2024-04-25T00:31:32.339723+0000","last_undegraded":"2024-04-25T00:31:32.339723+0000","last_fullsized":"2024-04-25T00:31:32.339723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:02.546759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273823+0000","last_change":"2024-04-25T00:31:30.316790+0000","last_active":"2024-04-25T00:31:32.273823+0000","last_peered":"2024-04-25T00:31:32.273823+0000","last_clean":"2024-04-25T00:31:32.273823+0000","last_became_active":"2024-04-25T00:31:30.315664+0000","last_became_peered":"2024-04-25T00:31:30.315664+0000","last_unstale":"2024-04-25T00:31:32.273823+0000","last_undegraded":"2024-04-25T00:31:32.273823+0000","last_fullsized":"2024-04-25T00:31:32.273823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339315+0000","last_change":"2024-04-25T00:31:30.322906+0000","last_active":"2024-04-25T00:31:32.339315+0000","last_peered":"2024-04-25T00:31:32.339315+0000","last_clean":"2024-04-25T00:31:32.339315+0000","last_became_active":"2024-04-25T00:31:30.322626+0000","last_became_peered":"2024-04-25T00:31:30.322626+0000","last_unstale":"2024-04-25T00:31:32.339315+0000","last_undegraded":"2024-04-25T00:31:32.339315+0000","last_fullsized":"2024-04-25T00:31:32.339315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:26.475549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293998+0000","last_change":"2024-04-25T00:31:30.327130+0000","last_active":"2024-04-25T00:31:31.293998+0000","last_peered":"2024-04-25T00:31:31.293998+0000","last_clean":"2024-04-25T00:31:31.293998+0000","last_became_active":"2024-04-25T00:31:30.326309+0000","last_became_peered":"2024-04-25T00:31:30.326309+0000","last_unstale":"2024-04-25T00:31:31.293998+0000","last_undegraded":"2024-04-25T00:31:31.293998+0000","last_fullsized":"2024-04-25T00:31:31.293998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:06.663363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338079+0000","last_change":"2024-04-25T00:31:30.329677+0000","last_active":"2024-04-25T00:31:32.338079+0000","last_peered":"2024-04-25T00:31:32.338079+0000","last_clean":"2024-04-25T00:31:32.338079+0000","last_became_active":"2024-04-25T00:31:30.329278+0000","last_became_peered":"2024-04-25T00:31:30.329278+0000","last_unstale":"2024-04-25T00:31:32.338079+0000","last_undegraded":"2024-04-25T00:31:32.338079+0000","last_fullsized":"2024-04-25T00:31:32.338079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:24.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.292924+0000","last_change":"2024-04-25T00:31:30.317304+0000","last_active":"2024-04-25T00:31:31.292924+0000","last_peered":"2024-04-25T00:31:31.292924+0000","last_clean":"2024-04-25T00:31:31.292924+0000","last_became_active":"2024-04-25T00:31:30.314963+0000","last_became_peered":"2024-04-25T00:31:30.314963+0000","last_unstale":"2024-04-25T00:31:31.292924+0000","last_undegraded":"2024-04-25T00:31:31.292924+0000","last_fullsized":"2024-04-25T00:31:31.292924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338403+0000","last_change":"2024-04-25T00:31:30.329416+0000","last_active":"2024-04-25T00:31:32.338403+0000","last_peered":"2024-04-25T00:31:32.338403+0000","last_clean":"2024-04-25T00:31:32.338403+0000","last_became_active":"2024-04-25T00:31:30.329055+0000","last_became_peered":"2024-04-25T00:31:30.329055+0000","last_unstale":"2024-04-25T00:31:32.338403+0000","last_undegraded":"2024-04-25T00:31:32.338403+0000","last_fullsized":"2024-04-25T00:31:32.338403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.274281+0000","last_change":"2024-04-25T00:31:30.325387+0000","last_active":"2024-04-25T00:31:32.274281+0000","last_peered":"2024-04-25T00:31:32.274281+0000","last_clean":"2024-04-25T00:31:32.274281+0000","last_became_active":"2024-04-25T00:31:30.324786+0000","last_became_peered":"2024-04-25T00:31:30.324786+0000","last_unstale":"2024-04-25T00:31:32.274281+0000","last_undegraded":"2024-04-25T00:31:32.274281+0000","last_fullsized":"2024-04-25T00:31:32.274281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.057143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289883+0000","last_change":"2024-04-25T00:31:30.288677+0000","last_active":"2024-04-25T00:31:31.289883+0000","last_peered":"2024-04-25T00:31:31.289883+0000","last_clean":"2024-04-25T00:31:31.289883+0000","last_became_active":"2024-04-25T00:31:30.288327+0000","last_became_peered":"2024-04-25T00:31:30.288327+0000","last_unstale":"2024-04-25T00:31:31.289883+0000","last_undegraded":"2024-04-25T00:31:31.289883+0000","last_fullsized":"2024-04-25T00:31:31.289883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338740+0000","last_change":"2024-04-25T00:31:30.287147+0000","last_active":"2024-04-25T00:31:32.338740+0000","last_peered":"2024-04-25T00:31:32.338740+0000","last_clean":"2024-04-25T00:31:32.338740+0000","last_became_active":"2024-04-25T00:31:30.286375+0000","last_became_peered":"2024-04-25T00:31:30.286375+0000","last_unstale":"2024-04-25T00:31:32.338740+0000","last_undegraded":"2024-04-25T00:31:32.338740+0000","last_fullsized":"2024-04-25T00:31:32.338740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:06.663363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339232+0000","last_change":"2024-04-25T00:31:30.306816+0000","last_active":"2024-04-25T00:31:32.339232+0000","last_peered":"2024-04-25T00:31:32.339232+0000","last_clean":"2024-04-25T00:31:32.339232+0000","last_became_active":"2024-04-25T00:31:30.306476+0000","last_became_peered":"2024-04-25T00:31:30.306476+0000","last_unstale":"2024-04-25T00:31:32.339232+0000","last_undegraded":"2024-04-25T00:31:32.339232+0000","last_fullsized":"2024-04-25T00:31:32.339232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:30.262696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294366+0000","last_change":"2024-04-25T00:31:30.317416+0000","last_active":"2024-04-25T00:31:31.294366+0000","last_peered":"2024-04-25T00:31:31.294366+0000","last_clean":"2024-04-25T00:31:31.294366+0000","last_became_active":"2024-04-25T00:31:30.315307+0000","last_became_peered":"2024-04-25T00:31:30.315307+0000","last_unstale":"2024-04-25T00:31:31.294366+0000","last_undegraded":"2024-04-25T00:31:31.294366+0000","last_fullsized":"2024-04-25T00:31:31.294366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:56.905979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337140+0000","last_change":"2024-04-25T00:31:30.299856+0000","last_active":"2024-04-25T00:31:32.337140+0000","last_peered":"2024-04-25T00:31:32.337140+0000","last_clean":"2024-04-25T00:31:32.337140+0000","last_became_active":"2024-04-25T00:31:30.299616+0000","last_became_peered":"2024-04-25T00:31:30.299616+0000","last_unstale":"2024-04-25T00:31:32.337140+0000","last_undegraded":"2024-04-25T00:31:32.337140+0000","last_fullsized":"2024-04-25T00:31:32.337140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:08.442361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273483+0000","last_change":"2024-04-25T00:31:30.304725+0000","last_active":"2024-04-25T00:31:32.273483+0000","last_peered":"2024-04-25T00:31:32.273483+0000","last_clean":"2024-04-25T00:31:32.273483+0000","last_became_active":"2024-04-25T00:31:30.298142+0000","last_became_peered":"2024-04-25T00:31:30.298142+0000","last_unstale":"2024-04-25T00:31:32.273483+0000","last_undegraded":"2024-04-25T00:31:32.273483+0000","last_fullsized":"2024-04-25T00:31:32.273483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293069+0000","last_change":"2024-04-25T00:31:30.304440+0000","last_active":"2024-04-25T00:31:31.293069+0000","last_peered":"2024-04-25T00:31:31.293069+0000","last_clean":"2024-04-25T00:31:31.293069+0000","last_became_active":"2024-04-25T00:31:30.303603+0000","last_became_peered":"2024-04-25T00:31:30.303603+0000","last_unstale":"2024-04-25T00:31:31.293069+0000","last_undegraded":"2024-04-25T00:31:31.293069+0000","last_fullsized":"2024-04-25T00:31:31.293069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:13.138372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338669+0000","last_change":"2024-04-25T00:31:29.306305+0000","last_active":"2024-04-25T00:31:32.338669+0000","last_peered":"2024-04-25T00:31:32.338669+0000","last_clean":"2024-04-25T00:31:32.338669+0000","last_became_active":"2024-04-25T00:31:29.305825+0000","last_became_peered":"2024-04-25T00:31:29.305825+0000","last_unstale":"2024-04-25T00:31:32.338669+0000","last_undegraded":"2024-04-25T00:31:32.338669+0000","last_fullsized":"2024-04-25T00:31:32.338669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:12.847032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339405+0000","last_change":"2024-04-25T00:31:30.280578+0000","last_active":"2024-04-25T00:31:32.339405+0000","last_peered":"2024-04-25T00:31:32.339405+0000","last_clean":"2024-04-25T00:31:32.339405+0000","last_became_active":"2024-04-25T00:31:30.278616+0000","last_became_peered":"2024-04-25T00:31:30.278616+0000","last_unstale":"2024-04-25T00:31:32.339405+0000","last_undegraded":"2024-04-25T00:31:32.339405+0000","last_fullsized":"2024-04-25T00:31:32.339405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:24.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273089+0000","last_change":"2024-04-25T00:31:29.295550+0000","last_active":"2024-04-25T00:31:32.273089+0000","last_peered":"2024-04-25T00:31:32.273089+0000","last_clean":"2024-04-25T00:31:32.273089+0000","last_became_active":"2024-04-25T00:31:29.294791+0000","last_became_peered":"2024-04-25T00:31:29.294791+0000","last_unstale":"2024-04-25T00:31:32.273089+0000","last_undegraded":"2024-04-25T00:31:32.273089+0000","last_fullsized":"2024-04-25T00:31:32.273089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:28.847603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338214+0000","last_change":"2024-04-25T00:31:29.295660+0000","last_active":"2024-04-25T00:31:32.338214+0000","last_peered":"2024-04-25T00:31:32.338214+0000","last_clean":"2024-04-25T00:31:32.338214+0000","last_became_active":"2024-04-25T00:31:29.295125+0000","last_became_peered":"2024-04-25T00:31:29.295125+0000","last_unstale":"2024-04-25T00:31:32.338214+0000","last_undegraded":"2024-04-25T00:31:32.338214+0000","last_fullsized":"2024-04-25T00:31:32.338214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:43.126464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338169+0000","last_change":"2024-04-25T00:31:30.301078+0000","last_active":"2024-04-25T00:31:32.338169+0000","last_peered":"2024-04-25T00:31:32.338169+0000","last_clean":"2024-04-25T00:31:32.338169+0000","last_became_active":"2024-04-25T00:31:30.298382+0000","last_became_peered":"2024-04-25T00:31:30.298382+0000","last_unstale":"2024-04-25T00:31:32.338169+0000","last_undegraded":"2024-04-25T00:31:32.338169+0000","last_fullsized":"2024-04-25T00:31:32.338169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293054+0000","last_change":"2024-04-25T00:31:29.273641+0000","last_active":"2024-04-25T00:31:31.293054+0000","last_peered":"2024-04-25T00:31:31.293054+0000","last_clean":"2024-04-25T00:31:31.293054+0000","last_became_active":"2024-04-25T00:31:29.272664+0000","last_became_peered":"2024-04-25T00:31:29.272664+0000","last_unstale":"2024-04-25T00:31:31.293054+0000","last_undegraded":"2024-04-25T00:31:31.293054+0000","last_fullsized":"2024-04-25T00:31:31.293054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:12.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289635+0000","last_change":"2024-04-25T00:31:30.287860+0000","last_active":"2024-04-25T00:31:31.289635+0000","last_peered":"2024-04-25T00:31:31.289635+0000","last_clean":"2024-04-25T00:31:31.289635+0000","last_became_active":"2024-04-25T00:31:30.287518+0000","last_became_peered":"2024-04-25T00:31:30.287518+0000","last_unstale":"2024-04-25T00:31:31.289635+0000","last_undegraded":"2024-04-25T00:31:31.289635+0000","last_fullsized":"2024-04-25T00:31:31.289635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:24.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.274623+0000","last_change":"2024-04-25T00:31:29.300290+0000","last_active":"2024-04-25T00:31:32.274623+0000","last_peered":"2024-04-25T00:31:32.274623+0000","last_clean":"2024-04-25T00:31:32.274623+0000","last_became_active":"2024-04-25T00:31:29.297912+0000","last_became_peered":"2024-04-25T00:31:29.297912+0000","last_unstale":"2024-04-25T00:31:32.274623+0000","last_undegraded":"2024-04-25T00:31:32.274623+0000","last_fullsized":"2024-04-25T00:31:32.274623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:06.128468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294033+0000","last_change":"2024-04-25T00:31:30.323107+0000","last_active":"2024-04-25T00:31:31.294033+0000","last_peered":"2024-04-25T00:31:31.294033+0000","last_clean":"2024-04-25T00:31:31.294033+0000","last_became_active":"2024-04-25T00:31:30.322670+0000","last_became_peered":"2024-04-25T00:31:30.322670+0000","last_unstale":"2024-04-25T00:31:31.294033+0000","last_undegraded":"2024-04-25T00:31:31.294033+0000","last_fullsized":"2024-04-25T00:31:31.294033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:37.829118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294169+0000","last_change":"2024-04-25T00:31:29.297610+0000","last_active":"2024-04-25T00:31:31.294169+0000","last_peered":"2024-04-25T00:31:31.294169+0000","last_clean":"2024-04-25T00:31:31.294169+0000","last_became_active":"2024-04-25T00:31:29.296820+0000","last_became_peered":"2024-04-25T00:31:29.296820+0000","last_unstale":"2024-04-25T00:31:31.294169+0000","last_undegraded":"2024-04-25T00:31:31.294169+0000","last_fullsized":"2024-04-25T00:31:31.294169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:28.847603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339641+0000","last_change":"2024-04-25T00:31:30.306807+0000","last_active":"2024-04-25T00:31:32.339641+0000","last_peered":"2024-04-25T00:31:32.339641+0000","last_clean":"2024-04-25T00:31:32.339641+0000","last_became_active":"2024-04-25T00:31:30.306523+0000","last_became_peered":"2024-04-25T00:31:30.306523+0000","last_unstale":"2024-04-25T00:31:32.339641+0000","last_undegraded":"2024-04-25T00:31:32.339641+0000","last_fullsized":"2024-04-25T00:31:32.339641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:37.782297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289930+0000","last_change":"2024-04-25T00:31:29.296029+0000","last_active":"2024-04-25T00:31:31.289930+0000","last_peered":"2024-04-25T00:31:31.289930+0000","last_clean":"2024-04-25T00:31:31.289930+0000","last_became_active":"2024-04-25T00:31:29.295557+0000","last_became_peered":"2024-04-25T00:31:29.295557+0000","last_unstale":"2024-04-25T00:31:31.289930+0000","last_undegraded":"2024-04-25T00:31:31.289930+0000","last_fullsized":"2024-04-25T00:31:31.289930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:12.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295669+0000","last_change":"2024-04-25T00:31:30.311562+0000","last_active":"2024-04-25T00:31:31.295669+0000","last_peered":"2024-04-25T00:31:31.295669+0000","last_clean":"2024-04-25T00:31:31.295669+0000","last_became_active":"2024-04-25T00:31:30.309517+0000","last_became_peered":"2024-04-25T00:31:30.309517+0000","last_unstale":"2024-04-25T00:31:31.295669+0000","last_undegraded":"2024-04-25T00:31:31.295669+0000","last_fullsized":"2024-04-25T00:31:31.295669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:07.434483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273705+0000","last_change":"2024-04-25T00:31:29.295877+0000","last_active":"2024-04-25T00:31:32.273705+0000","last_peered":"2024-04-25T00:31:32.273705+0000","last_clean":"2024-04-25T00:31:32.273705+0000","last_became_active":"2024-04-25T00:31:29.295208+0000","last_became_peered":"2024-04-25T00:31:29.295208+0000","last_unstale":"2024-04-25T00:31:32.273705+0000","last_undegraded":"2024-04-25T00:31:32.273705+0000","last_fullsized":"2024-04-25T00:31:32.273705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:24.278463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339130+0000","last_change":"2024-04-25T00:31:30.307864+0000","last_active":"2024-04-25T00:31:32.339130+0000","last_peered":"2024-04-25T00:31:32.339130+0000","last_clean":"2024-04-25T00:31:32.339130+0000","last_became_active":"2024-04-25T00:31:30.306945+0000","last_became_peered":"2024-04-25T00:31:30.306945+0000","last_unstale":"2024-04-25T00:31:32.339130+0000","last_undegraded":"2024-04-25T00:31:32.339130+0000","last_fullsized":"2024-04-25T00:31:32.339130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:16.301733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339404+0000","last_change":"2024-04-25T00:31:29.291150+0000","last_active":"2024-04-25T00:31:32.339404+0000","last_peered":"2024-04-25T00:31:32.339404+0000","last_clean":"2024-04-25T00:31:32.339404+0000","last_became_active":"2024-04-25T00:31:29.290597+0000","last_became_peered":"2024-04-25T00:31:29.290597+0000","last_unstale":"2024-04-25T00:31:32.339404+0000","last_undegraded":"2024-04-25T00:31:32.339404+0000","last_fullsized":"2024-04-25T00:31:32.339404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:26.121929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339478+0000","last_change":"2024-04-25T00:31:30.322864+0000","last_active":"2024-04-25T00:31:32.339478+0000","last_peered":"2024-04-25T00:31:32.339478+0000","last_clean":"2024-04-25T00:31:32.339478+0000","last_became_active":"2024-04-25T00:31:30.322561+0000","last_became_peered":"2024-04-25T00:31:30.322561+0000","last_unstale":"2024-04-25T00:31:32.339478+0000","last_undegraded":"2024-04-25T00:31:32.339478+0000","last_fullsized":"2024-04-25T00:31:32.339478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294164+0000","last_change":"2024-04-25T00:31:29.289693+0000","last_active":"2024-04-25T00:31:31.294164+0000","last_peered":"2024-04-25T00:31:31.294164+0000","last_clean":"2024-04-25T00:31:31.294164+0000","last_became_active":"2024-04-25T00:31:29.286958+0000","last_became_peered":"2024-04-25T00:31:29.286958+0000","last_unstale":"2024-04-25T00:31:31.294164+0000","last_undegraded":"2024-04-25T00:31:31.294164+0000","last_fullsized":"2024-04-25T00:31:31.294164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:06.128468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339729+0000","last_change":"2024-04-25T00:31:30.327490+0000","last_active":"2024-04-25T00:31:32.339729+0000","last_peered":"2024-04-25T00:31:32.339729+0000","last_clean":"2024-04-25T00:31:32.339729+0000","last_became_active":"2024-04-25T00:31:30.324807+0000","last_became_peered":"2024-04-25T00:31:30.324807+0000","last_unstale":"2024-04-25T00:31:32.339729+0000","last_undegraded":"2024-04-25T00:31:32.339729+0000","last_fullsized":"2024-04-25T00:31:32.339729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:12.335111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273055+0000","last_change":"2024-04-25T00:31:29.292889+0000","last_active":"2024-04-25T00:31:32.273055+0000","last_peered":"2024-04-25T00:31:32.273055+0000","last_clean":"2024-04-25T00:31:32.273055+0000","last_became_active":"2024-04-25T00:31:29.292160+0000","last_became_peered":"2024-04-25T00:31:29.292160+0000","last_unstale":"2024-04-25T00:31:32.273055+0000","last_undegraded":"2024-04-25T00:31:32.273055+0000","last_fullsized":"2024-04-25T00:31:32.273055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:43.126464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273426+0000","last_change":"2024-04-25T00:31:30.306169+0000","last_active":"2024-04-25T00:31:32.273426+0000","last_peered":"2024-04-25T00:31:32.273426+0000","last_clean":"2024-04-25T00:31:32.273426+0000","last_became_active":"2024-04-25T00:31:30.299130+0000","last_became_peered":"2024-04-25T00:31:30.299130+0000","last_unstale":"2024-04-25T00:31:32.273426+0000","last_undegraded":"2024-04-25T00:31:32.273426+0000","last_fullsized":"2024-04-25T00:31:32.273426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:24.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.336849+0000","last_change":"2024-04-25T00:31:29.273896+0000","last_active":"2024-04-25T00:31:32.336849+0000","last_peered":"2024-04-25T00:31:32.336849+0000","last_clean":"2024-04-25T00:31:32.336849+0000","last_became_active":"2024-04-25T00:31:29.273531+0000","last_became_peered":"2024-04-25T00:31:29.273531+0000","last_unstale":"2024-04-25T00:31:32.336849+0000","last_undegraded":"2024-04-25T00:31:32.336849+0000","last_fullsized":"2024-04-25T00:31:32.336849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:43.126464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338915+0000","last_change":"2024-04-25T00:31:30.314038+0000","last_active":"2024-04-25T00:31:32.338915+0000","last_peered":"2024-04-25T00:31:32.338915+0000","last_clean":"2024-04-25T00:31:32.338915+0000","last_became_active":"2024-04-25T00:31:30.309018+0000","last_became_peered":"2024-04-25T00:31:30.309018+0000","last_unstale":"2024-04-25T00:31:32.338915+0000","last_undegraded":"2024-04-25T00:31:32.338915+0000","last_fullsized":"2024-04-25T00:31:32.338915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:56.905979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295834+0000","last_change":"2024-04-25T00:31:29.286381+0000","last_active":"2024-04-25T00:31:31.295834+0000","last_peered":"2024-04-25T00:31:31.295834+0000","last_clean":"2024-04-25T00:31:31.295834+0000","last_became_active":"2024-04-25T00:31:29.285385+0000","last_became_peered":"2024-04-25T00:31:29.285385+0000","last_unstale":"2024-04-25T00:31:31.295834+0000","last_undegraded":"2024-04-25T00:31:31.295834+0000","last_fullsized":"2024-04-25T00:31:31.295834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:33.874197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339927+0000","last_change":"2024-04-25T00:31:30.313477+0000","last_active":"2024-04-25T00:31:32.339927+0000","last_peered":"2024-04-25T00:31:32.339927+0000","last_clean":"2024-04-25T00:31:32.339927+0000","last_became_active":"2024-04-25T00:31:30.312831+0000","last_became_peered":"2024-04-25T00:31:30.312831+0000","last_unstale":"2024-04-25T00:31:32.339927+0000","last_undegraded":"2024-04-25T00:31:32.339927+0000","last_fullsized":"2024-04-25T00:31:32.339927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:36.378500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337220+0000","last_change":"2024-04-25T00:31:29.272826+0000","last_active":"2024-04-25T00:31:32.337220+0000","last_peered":"2024-04-25T00:31:32.337220+0000","last_clean":"2024-04-25T00:31:32.337220+0000","last_became_active":"2024-04-25T00:31:29.272432+0000","last_became_peered":"2024-04-25T00:31:29.272432+0000","last_unstale":"2024-04-25T00:31:32.337220+0000","last_undegraded":"2024-04-25T00:31:32.337220+0000","last_fullsized":"2024-04-25T00:31:32.337220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:24.278463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.340202+0000","last_change":"2024-04-25T00:31:30.312201+0000","last_active":"2024-04-25T00:31:32.340202+0000","last_peered":"2024-04-25T00:31:32.340202+0000","last_clean":"2024-04-25T00:31:32.340202+0000","last_became_active":"2024-04-25T00:31:30.311412+0000","last_became_peered":"2024-04-25T00:31:30.311412+0000","last_unstale":"2024-04-25T00:31:32.340202+0000","last_undegraded":"2024-04-25T00:31:32.340202+0000","last_fullsized":"2024-04-25T00:31:32.340202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:36.992645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.290023+0000","last_change":"2024-04-25T00:31:29.289893+0000","last_active":"2024-04-25T00:31:31.290023+0000","last_peered":"2024-04-25T00:31:31.290023+0000","last_clean":"2024-04-25T00:31:31.290023+0000","last_became_active":"2024-04-25T00:31:29.289495+0000","last_became_peered":"2024-04-25T00:31:29.289495+0000","last_unstale":"2024-04-25T00:31:31.290023+0000","last_undegraded":"2024-04-25T00:31:31.290023+0000","last_fullsized":"2024-04-25T00:31:31.290023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:06.128468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294386+0000","last_change":"2024-04-25T00:31:30.314359+0000","last_active":"2024-04-25T00:31:31.294386+0000","last_peered":"2024-04-25T00:31:31.294386+0000","last_clean":"2024-04-25T00:31:31.294386+0000","last_became_active":"2024-04-25T00:31:30.313625+0000","last_became_peered":"2024-04-25T00:31:30.313625+0000","last_unstale":"2024-04-25T00:31:31.294386+0000","last_undegraded":"2024-04-25T00:31:31.294386+0000","last_fullsized":"2024-04-25T00:31:31.294386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:16.301733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337138+0000","last_change":"2024-04-25T00:31:29.297325+0000","last_active":"2024-04-25T00:31:32.337138+0000","last_peered":"2024-04-25T00:31:32.337138+0000","last_clean":"2024-04-25T00:31:32.337138+0000","last_became_active":"2024-04-25T00:31:29.296822+0000","last_became_peered":"2024-04-25T00:31:29.296822+0000","last_unstale":"2024-04-25T00:31:32.337138+0000","last_undegraded":"2024-04-25T00:31:32.337138+0000","last_fullsized":"2024-04-25T00:31:32.337138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:28.847603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339313+0000","last_change":"2024-04-25T00:31:30.323788+0000","last_active":"2024-04-25T00:31:32.339313+0000","last_peered":"2024-04-25T00:31:32.339313+0000","last_clean":"2024-04-25T00:31:32.339313+0000","last_became_active":"2024-04-25T00:31:30.323115+0000","last_became_peered":"2024-04-25T00:31:30.323115+0000","last_unstale":"2024-04-25T00:31:32.339313+0000","last_undegraded":"2024-04-25T00:31:32.339313+0000","last_fullsized":"2024-04-25T00:31:32.339313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:51.764905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338672+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:31:32.338672+0000","last_peered":"2024-04-25T00:31:32.338672+0000","last_clean":"2024-04-25T00:31:32.338672+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:31:32.338672+0000","last_undegraded":"2024-04-25T00:31:32.338672+0000","last_fullsized":"2024-04-25T00:31:32.338672+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.292686+0000","last_change":"2024-04-25T00:31:29.287393+0000","last_active":"2024-04-25T00:31:31.292686+0000","last_peered":"2024-04-25T00:31:31.292686+0000","last_clean":"2024-04-25T00:31:31.292686+0000","last_became_active":"2024-04-25T00:31:29.286842+0000","last_became_peered":"2024-04-25T00:31:29.286842+0000","last_unstale":"2024-04-25T00:31:31.292686+0000","last_undegraded":"2024-04-25T00:31:31.292686+0000","last_fullsized":"2024-04-25T00:31:31.292686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:43.126464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338536+0000","last_change":"2024-04-25T00:31:30.302310+0000","last_active":"2024-04-25T00:31:32.338536+0000","last_peered":"2024-04-25T00:31:32.338536+0000","last_clean":"2024-04-25T00:31:32.338536+0000","last_became_active":"2024-04-25T00:31:30.302000+0000","last_became_peered":"2024-04-25T00:31:30.302000+0000","last_unstale":"2024-04-25T00:31:32.338536+0000","last_undegraded":"2024-04-25T00:31:32.338536+0000","last_fullsized":"2024-04-25T00:31:32.338536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338493+0000","last_change":"2024-04-25T00:31:29.308692+0000","last_active":"2024-04-25T00:31:32.338493+0000","last_peered":"2024-04-25T00:31:32.338493+0000","last_clean":"2024-04-25T00:31:32.338493+0000","last_became_active":"2024-04-25T00:31:29.306907+0000","last_became_peered":"2024-04-25T00:31:29.306907+0000","last_unstale":"2024-04-25T00:31:32.338493+0000","last_undegraded":"2024-04-25T00:31:32.338493+0000","last_fullsized":"2024-04-25T00:31:32.338493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:07.430509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339054+0000","last_change":"2024-04-25T00:31:30.278855+0000","last_active":"2024-04-25T00:31:32.339054+0000","last_peered":"2024-04-25T00:31:32.339054+0000","last_clean":"2024-04-25T00:31:32.339054+0000","last_became_active":"2024-04-25T00:31:30.278557+0000","last_became_peered":"2024-04-25T00:31:30.278557+0000","last_unstale":"2024-04-25T00:31:32.339054+0000","last_undegraded":"2024-04-25T00:31:32.339054+0000","last_fullsized":"2024-04-25T00:31:32.339054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:00.643971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.337108+0000","last_change":"2024-04-25T00:31:29.271815+0000","last_active":"2024-04-25T00:31:32.337108+0000","last_peered":"2024-04-25T00:31:32.337108+0000","last_clean":"2024-04-25T00:31:32.337108+0000","last_became_active":"2024-04-25T00:31:29.271154+0000","last_became_peered":"2024-04-25T00:31:29.271154+0000","last_unstale":"2024-04-25T00:31:32.337108+0000","last_undegraded":"2024-04-25T00:31:32.337108+0000","last_fullsized":"2024-04-25T00:31:32.337108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:12.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289615+0000","last_change":"2024-04-25T00:31:30.287295+0000","last_active":"2024-04-25T00:31:31.289615+0000","last_peered":"2024-04-25T00:31:31.289615+0000","last_clean":"2024-04-25T00:31:31.289615+0000","last_became_active":"2024-04-25T00:31:30.285530+0000","last_became_peered":"2024-04-25T00:31:30.285530+0000","last_unstale":"2024-04-25T00:31:31.289615+0000","last_undegraded":"2024-04-25T00:31:31.289615+0000","last_fullsized":"2024-04-25T00:31:31.289615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:26.475549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294485+0000","last_change":"2024-04-25T00:31:29.288097+0000","last_active":"2024-04-25T00:31:31.294485+0000","last_peered":"2024-04-25T00:31:31.294485+0000","last_clean":"2024-04-25T00:31:31.294485+0000","last_became_active":"2024-04-25T00:31:29.286580+0000","last_became_peered":"2024-04-25T00:31:29.286580+0000","last_unstale":"2024-04-25T00:31:31.294485+0000","last_undegraded":"2024-04-25T00:31:31.294485+0000","last_fullsized":"2024-04-25T00:31:31.294485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:24.278463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.294183+0000","last_change":"2024-04-25T00:31:30.323523+0000","last_active":"2024-04-25T00:31:31.294183+0000","last_peered":"2024-04-25T00:31:31.294183+0000","last_clean":"2024-04-25T00:31:31.294183+0000","last_became_active":"2024-04-25T00:31:30.310509+0000","last_became_peered":"2024-04-25T00:31:30.310509+0000","last_unstale":"2024-04-25T00:31:31.294183+0000","last_undegraded":"2024-04-25T00:31:31.294183+0000","last_fullsized":"2024-04-25T00:31:31.294183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:25.290315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289773+0000","last_change":"2024-04-25T00:31:29.283436+0000","last_active":"2024-04-25T00:31:31.289773+0000","last_peered":"2024-04-25T00:31:31.289773+0000","last_clean":"2024-04-25T00:31:31.289773+0000","last_became_active":"2024-04-25T00:31:29.283038+0000","last_became_peered":"2024-04-25T00:31:29.283038+0000","last_unstale":"2024-04-25T00:31:31.289773+0000","last_undegraded":"2024-04-25T00:31:31.289773+0000","last_fullsized":"2024-04-25T00:31:31.289773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:28.847603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295871+0000","last_change":"2024-04-25T00:31:30.311973+0000","last_active":"2024-04-25T00:31:31.295871+0000","last_peered":"2024-04-25T00:31:31.295871+0000","last_clean":"2024-04-25T00:31:31.295871+0000","last_became_active":"2024-04-25T00:31:30.311141+0000","last_became_peered":"2024-04-25T00:31:30.311141+0000","last_unstale":"2024-04-25T00:31:31.295871+0000","last_undegraded":"2024-04-25T00:31:31.295871+0000","last_fullsized":"2024-04-25T00:31:31.295871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:12.335111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339611+0000","last_change":"2024-04-25T00:31:29.309713+0000","last_active":"2024-04-25T00:31:32.339611+0000","last_peered":"2024-04-25T00:31:32.339611+0000","last_clean":"2024-04-25T00:31:32.339611+0000","last_became_active":"2024-04-25T00:31:29.309033+0000","last_became_peered":"2024-04-25T00:31:29.309033+0000","last_unstale":"2024-04-25T00:31:32.339611+0000","last_undegraded":"2024-04-25T00:31:32.339611+0000","last_fullsized":"2024-04-25T00:31:32.339611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:59.632119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289794+0000","last_change":"2024-04-25T00:31:30.285762+0000","last_active":"2024-04-25T00:31:31.289794+0000","last_peered":"2024-04-25T00:31:31.289794+0000","last_clean":"2024-04-25T00:31:31.289794+0000","last_became_active":"2024-04-25T00:31:30.284685+0000","last_became_peered":"2024-04-25T00:31:30.284685+0000","last_unstale":"2024-04-25T00:31:31.289794+0000","last_undegraded":"2024-04-25T00:31:31.289794+0000","last_fullsized":"2024-04-25T00:31:31.289794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.290064+0000","last_change":"2024-04-25T00:31:29.296589+0000","last_active":"2024-04-25T00:31:31.290064+0000","last_peered":"2024-04-25T00:31:31.290064+0000","last_clean":"2024-04-25T00:31:31.290064+0000","last_became_active":"2024-04-25T00:31:29.296082+0000","last_became_peered":"2024-04-25T00:31:29.296082+0000","last_unstale":"2024-04-25T00:31:31.290064+0000","last_undegraded":"2024-04-25T00:31:31.290064+0000","last_fullsized":"2024-04-25T00:31:31.290064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:06.422631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338590+0000","last_change":"2024-04-25T00:31:30.318568+0000","last_active":"2024-04-25T00:31:32.338590+0000","last_peered":"2024-04-25T00:31:32.338590+0000","last_clean":"2024-04-25T00:31:32.338590+0000","last_became_active":"2024-04-25T00:31:30.318203+0000","last_became_peered":"2024-04-25T00:31:30.318203+0000","last_unstale":"2024-04-25T00:31:32.338590+0000","last_undegraded":"2024-04-25T00:31:32.338590+0000","last_fullsized":"2024-04-25T00:31:32.338590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:12.015188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.274864+0000","last_change":"2024-04-25T00:31:29.310321+0000","last_active":"2024-04-25T00:31:32.274864+0000","last_peered":"2024-04-25T00:31:32.274864+0000","last_clean":"2024-04-25T00:31:32.274864+0000","last_became_active":"2024-04-25T00:31:29.309796+0000","last_became_peered":"2024-04-25T00:31:29.309796+0000","last_unstale":"2024-04-25T00:31:32.274864+0000","last_undegraded":"2024-04-25T00:31:32.274864+0000","last_fullsized":"2024-04-25T00:31:32.274864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:12.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339672+0000","last_change":"2024-04-25T00:31:30.306140+0000","last_active":"2024-04-25T00:31:32.339672+0000","last_peered":"2024-04-25T00:31:32.339672+0000","last_clean":"2024-04-25T00:31:32.339672+0000","last_became_active":"2024-04-25T00:31:30.294939+0000","last_became_peered":"2024-04-25T00:31:30.294939+0000","last_unstale":"2024-04-25T00:31:32.339672+0000","last_undegraded":"2024-04-25T00:31:32.339672+0000","last_fullsized":"2024-04-25T00:31:32.339672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.913965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338396+0000","last_change":"2024-04-25T00:31:29.313166+0000","last_active":"2024-04-25T00:31:32.338396+0000","last_peered":"2024-04-25T00:31:32.338396+0000","last_clean":"2024-04-25T00:31:32.338396+0000","last_became_active":"2024-04-25T00:31:29.308641+0000","last_became_peered":"2024-04-25T00:31:29.308641+0000","last_unstale":"2024-04-25T00:31:32.338396+0000","last_undegraded":"2024-04-25T00:31:32.338396+0000","last_fullsized":"2024-04-25T00:31:32.338396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:06.422631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.296415+0000","last_change":"2024-04-25T00:31:30.303363+0000","last_active":"2024-04-25T00:31:31.296415+0000","last_peered":"2024-04-25T00:31:31.296415+0000","last_clean":"2024-04-25T00:31:31.296415+0000","last_became_active":"2024-04-25T00:31:30.302602+0000","last_became_peered":"2024-04-25T00:31:30.302602+0000","last_unstale":"2024-04-25T00:31:31.296415+0000","last_undegraded":"2024-04-25T00:31:31.296415+0000","last_fullsized":"2024-04-25T00:31:31.296415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:08.442361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.289690+0000","last_change":"2024-04-25T00:31:29.306640+0000","last_active":"2024-04-25T00:31:31.289690+0000","last_peered":"2024-04-25T00:31:31.289690+0000","last_clean":"2024-04-25T00:31:31.289690+0000","last_became_active":"2024-04-25T00:31:29.306117+0000","last_became_peered":"2024-04-25T00:31:29.306117+0000","last_unstale":"2024-04-25T00:31:31.289690+0000","last_undegraded":"2024-04-25T00:31:31.289690+0000","last_fullsized":"2024-04-25T00:31:31.289690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:07.430509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338904+0000","last_change":"2024-04-25T00:31:30.287149+0000","last_active":"2024-04-25T00:31:32.338904+0000","last_peered":"2024-04-25T00:31:32.338904+0000","last_clean":"2024-04-25T00:31:32.338904+0000","last_became_active":"2024-04-25T00:31:30.286140+0000","last_became_peered":"2024-04-25T00:31:30.286140+0000","last_unstale":"2024-04-25T00:31:32.338904+0000","last_undegraded":"2024-04-25T00:31:32.338904+0000","last_fullsized":"2024-04-25T00:31:32.338904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:37.829118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.290109+0000","last_change":"2024-04-25T00:31:29.278390+0000","last_active":"2024-04-25T00:31:31.290109+0000","last_peered":"2024-04-25T00:31:31.290109+0000","last_clean":"2024-04-25T00:31:31.290109+0000","last_became_active":"2024-04-25T00:31:29.277901+0000","last_became_peered":"2024-04-25T00:31:29.277901+0000","last_unstale":"2024-04-25T00:31:31.290109+0000","last_undegraded":"2024-04-25T00:31:31.290109+0000","last_fullsized":"2024-04-25T00:31:31.290109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:24.278463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338701+0000","last_change":"2024-04-25T00:31:30.298582+0000","last_active":"2024-04-25T00:31:32.338701+0000","last_peered":"2024-04-25T00:31:32.338701+0000","last_clean":"2024-04-25T00:31:32.338701+0000","last_became_active":"2024-04-25T00:31:30.298289+0000","last_became_peered":"2024-04-25T00:31:30.298289+0000","last_unstale":"2024-04-25T00:31:32.338701+0000","last_undegraded":"2024-04-25T00:31:32.338701+0000","last_fullsized":"2024-04-25T00:31:32.338701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:50.437582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339895+0000","last_change":"2024-04-25T00:31:29.309564+0000","last_active":"2024-04-25T00:31:32.339895+0000","last_peered":"2024-04-25T00:31:32.339895+0000","last_clean":"2024-04-25T00:31:32.339895+0000","last_became_active":"2024-04-25T00:31:29.308824+0000","last_became_peered":"2024-04-25T00:31:29.308824+0000","last_unstale":"2024-04-25T00:31:32.339895+0000","last_undegraded":"2024-04-25T00:31:32.339895+0000","last_fullsized":"2024-04-25T00:31:32.339895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:49.417800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.273411+0000","last_change":"2024-04-25T00:31:30.316035+0000","last_active":"2024-04-25T00:31:32.273411+0000","last_peered":"2024-04-25T00:31:32.273411+0000","last_clean":"2024-04-25T00:31:32.273411+0000","last_became_active":"2024-04-25T00:31:30.315263+0000","last_became_peered":"2024-04-25T00:31:30.315263+0000","last_unstale":"2024-04-25T00:31:32.273411+0000","last_undegraded":"2024-04-25T00:31:32.273411+0000","last_fullsized":"2024-04-25T00:31:32.273411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:26.475549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.295648+0000","last_change":"2024-04-25T00:31:30.329587+0000","last_active":"2024-04-25T00:31:31.295648+0000","last_peered":"2024-04-25T00:31:31.295648+0000","last_clean":"2024-04-25T00:31:31.295648+0000","last_became_active":"2024-04-25T00:31:30.329014+0000","last_became_peered":"2024-04-25T00:31:30.329014+0000","last_unstale":"2024-04-25T00:31:31.295648+0000","last_undegraded":"2024-04-25T00:31:31.295648+0000","last_fullsized":"2024-04-25T00:31:31.295648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:02.546759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338944+0000","last_change":"2024-04-25T00:31:29.307154+0000","last_active":"2024-04-25T00:31:32.338944+0000","last_peered":"2024-04-25T00:31:32.338944+0000","last_clean":"2024-04-25T00:31:32.338944+0000","last_became_active":"2024-04-25T00:31:29.306693+0000","last_became_peered":"2024-04-25T00:31:29.306693+0000","last_unstale":"2024-04-25T00:31:32.338944+0000","last_undegraded":"2024-04-25T00:31:32.338944+0000","last_fullsized":"2024-04-25T00:31:32.338944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:20.941312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:31.293968+0000","last_change":"2024-04-25T00:31:30.329462+0000","last_active":"2024-04-25T00:31:31.293968+0000","last_peered":"2024-04-25T00:31:31.293968+0000","last_clean":"2024-04-25T00:31:31.293968+0000","last_became_active":"2024-04-25T00:31:30.328941+0000","last_became_peered":"2024-04-25T00:31:30.328941+0000","last_unstale":"2024-04-25T00:31:31.293968+0000","last_undegraded":"2024-04-25T00:31:31.293968+0000","last_fullsized":"2024-04-25T00:31:31.293968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:50.437582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339296+0000","last_change":"2024-04-25T00:31:29.311168+0000","last_active":"2024-04-25T00:31:32.339296+0000","last_peered":"2024-04-25T00:31:32.339296+0000","last_clean":"2024-04-25T00:31:32.339296+0000","last_became_active":"2024-04-25T00:31:29.310402+0000","last_became_peered":"2024-04-25T00:31:29.310402+0000","last_unstale":"2024-04-25T00:31:32.339296+0000","last_undegraded":"2024-04-25T00:31:32.339296+0000","last_fullsized":"2024-04-25T00:31:32.339296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:56.307057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339924+0000","last_change":"2024-04-25T00:31:30.307691+0000","last_active":"2024-04-25T00:31:32.339924+0000","last_peered":"2024-04-25T00:31:32.339924+0000","last_clean":"2024-04-25T00:31:32.339924+0000","last_became_active":"2024-04-25T00:31:30.307158+0000","last_became_peered":"2024-04-25T00:31:30.307158+0000","last_unstale":"2024-04-25T00:31:32.339924+0000","last_undegraded":"2024-04-25T00:31:32.339924+0000","last_fullsized":"2024-04-25T00:31:32.339924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.057143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338533+0000","last_change":"2024-04-25T00:31:29.282318+0000","last_active":"2024-04-25T00:31:32.338533+0000","last_peered":"2024-04-25T00:31:32.338533+0000","last_clean":"2024-04-25T00:31:32.338533+0000","last_became_active":"2024-04-25T00:31:29.281826+0000","last_became_peered":"2024-04-25T00:31:29.281826+0000","last_unstale":"2024-04-25T00:31:32.338533+0000","last_undegraded":"2024-04-25T00:31:32.338533+0000","last_fullsized":"2024-04-25T00:31:32.338533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:26.121929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.338333+0000","last_change":"2024-04-25T00:31:30.302619+0000","last_active":"2024-04-25T00:31:32.338333+0000","last_peered":"2024-04-25T00:31:32.338333+0000","last_clean":"2024-04-25T00:31:32.338333+0000","last_became_active":"2024-04-25T00:31:30.301637+0000","last_became_peered":"2024-04-25T00:31:30.301637+0000","last_unstale":"2024-04-25T00:31:32.338333+0000","last_undegraded":"2024-04-25T00:31:32.338333+0000","last_fullsized":"2024-04-25T00:31:32.338333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:29.245289+0000","last_clean_scrub_stamp":"2024-04-25T00:31:29.245289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:31:32.339800+0000","last_change":"2024-04-25T00:31:29.311460+0000","last_active":"2024-04-25T00:31:32.339800+0000","last_peered":"2024-04-25T00:31:32.339800+0000","last_clean":"2024-04-25T00:31:32.339800+0000","last_became_active":"2024-04-25T00:31:29.311052+0000","last_became_peered":"2024-04-25T00:31:29.311052+0000","last_unstale":"2024-04-25T00:31:32.339800+0000","last_undegraded":"2024-04-25T00:31:32.339800+0000","last_fullsized":"2024-04-25T00:31:32.339800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:28.233437+0000","last_clean_scrub_stamp":"2024-04-25T00:31:28.233437+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:12.847032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:31:42.521 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T00:31:42.521 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T00:31:42.521 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T00:31:42.521 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T00:31:42.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.676+0000 7fb27e58f640 1 Processor -- start 2024-04-25T00:31:42.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.676+0000 7fb27e58f640 1 -- start start 2024-04-25T00:31:42.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb27e58f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb27815e8d0 0x7fb2780cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb27e58f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2780b3720 con 0x7fb27815f3c0 2024-04-25T00:31:42.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb27e58f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2780b39c0 con 0x7fb2780b4780 2024-04-25T00:31:42.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb27e58f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2780b42f0 con 0x7fb27815e8d0 2024-04-25T00:31:42.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb27cb2c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb27815f3c0 0x7fb27815c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50082/0 (socket says 172.21.15.29:50082) 2024-04-25T00:31:42.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb27cb2c640 1 -- 172.21.15.29:0/4031723098 learned_addr learned my addr 172.21.15.29:0/4031723098 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:42.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb277fff640 1 --2- 172.21.15.29:0/4031723098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb27815e8d0 0x7fb2780cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3380077478 0 0) 0x7fb2780b3720 con 0x7fb27815f3c0 2024-04-25T00:31:42.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb260005740 con 0x7fb27815f3c0 2024-04-25T00:31:42.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 424262988 0 0) 0x7fb260005740 con 0x7fb27815f3c0 2024-04-25T00:31:42.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb27815e8d0 msgr2=0x7fb2780cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:42.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 --2- 172.21.15.29:0/4031723098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb27815e8d0 0x7fb2780cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 >> v1:172.21.15.157:6789/0 conn(0x7fb2780b4780 legacy=0x7fb27815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2780b3ca0 con 0x7fb27815f3c0 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fb268001680 con 0x7fb27815f3c0 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 >> v1:172.21.15.29:6789/0 conn(0x7fb27815f3c0 legacy=0x7fb27815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 --2- 172.21.15.29:0/4031723098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb260005c00 0x7fb260006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 --2- 172.21.15.29:0/4031723098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb27815e8d0 0x7fb260006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 --2- 172.21.15.29:0/4031723098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb260006f60 0x7fb2600073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2780b42f0 con 0x7fb260006f60 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb26000dc90 con 0x7fb260005c00 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb26000de90 con 0x7fb27815e8d0 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb27cb2c640 1 --2- 172.21.15.29:0/4031723098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb260006f60 0x7fb2600073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb277fff640 1 --2- 172.21.15.29:0/4031723098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb260005c00 0x7fb260006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb2777fe640 1 --2- 172.21.15.29:0/4031723098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb27815e8d0 0x7fb260006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb27cb2c640 1 -- 172.21.15.29:0/4031723098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb27815e8d0 msgr2=0x7fb260006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:42.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb27cb2c640 1 --2- 172.21.15.29:0/4031723098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb27815e8d0 0x7fb260006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb27cb2c640 1 -- 172.21.15.29:0/4031723098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb260005c00 msgr2=0x7fb260006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:42.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb27cb2c640 1 --2- 172.21.15.29:0/4031723098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb260005c00 0x7fb260006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb27cb2c640 1 -- 172.21.15.29:0/4031723098 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb260005740 con 0x7fb260006f60 2024-04-25T00:31:42.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb27cb2c640 1 --2- 172.21.15.29:0/4031723098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb260006f60 0x7fb2600073e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb268001ec0 tx=0x7fb26801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1791a153d5a6f81c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:42.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268007d40 con 0x7fb260006f60 2024-04-25T00:31:42.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2680a9070 con 0x7fb260006f60 2024-04-25T00:31:42.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.680+0000 7fb276ffd640 1 -- 172.21.15.29:0/4031723098 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2680a5dc0 con 0x7fb260006f60 2024-04-25T00:31:42.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 -- 172.21.15.29:0/4031723098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb260006f60 msgr2=0x7fb2600073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 --2- 172.21.15.29:0/4031723098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb260006f60 0x7fb2600073e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb268001ec0 tx=0x7fb26801cc60 comp rx=0 tx=0).stop 2024-04-25T00:31:42.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27cb2c640 1 -- 172.21.15.29:0/4031723098 reap_dead start 2024-04-25T00:31:42.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 -- 172.21.15.29:0/4031723098 shutdown_connections 2024-04-25T00:31:42.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 -- 172.21.15.29:0/4031723098 >> 172.21.15.29:0/4031723098 conn(0x7fb2780b1870 msgr2=0x7fb2780b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:42.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 -- 172.21.15.29:0/4031723098 shutdown_connections 2024-04-25T00:31:42.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 -- 172.21.15.29:0/4031723098 wait complete. 2024-04-25T00:31:42.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 Processor -- start 2024-04-25T00:31:42.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 -- start start 2024-04-25T00:31:42.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb27815a0e0 0x7fb278137570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb27813dd90 0x7fb27813f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb27813fe20 0x7fb27813bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb278144a20 con 0x7fb27813fe20 2024-04-25T00:31:42.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb27806cf10 con 0x7fb27813dd90 2024-04-25T00:31:42.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb278137ec0 con 0x7fb27815a0e0 2024-04-25T00:31:42.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27cb2c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb27813fe20 0x7fb27813bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27cb2c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb27813fe20 0x7fb27813bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50072/0 (socket says 172.21.15.29:50072) 2024-04-25T00:31:42.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27cb2c640 1 -- 172.21.15.29:0/951391595 learned_addr learned my addr 172.21.15.29:0/951391595 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:42.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb2777fe640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb27813dd90 0x7fb27813f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb277fff640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb27815a0e0 0x7fb278137570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27cb2c640 1 -- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb27815a0e0 msgr2=0x7fb278137570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27cb2c640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb27815a0e0 0x7fb278137570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27cb2c640 1 -- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb27813dd90 msgr2=0x7fb27813f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27cb2c640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb27813dd90 0x7fb27813f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27cb2c640 1 -- 172.21.15.29:0/951391595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2781381a0 con 0x7fb27813fe20 2024-04-25T00:31:42.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27cb2c640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb27813fe20 0x7fb27813bf30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb26801c6d0 tx=0x7fb260007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:42.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb274ff9640 1 -- 172.21.15.29:0/951391595 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268007d20 con 0x7fb27813fe20 2024-04-25T00:31:42.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 -- 172.21.15.29:0/951391595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2781354d0 con 0x7fb27813fe20 2024-04-25T00:31:42.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb274ff9640 1 -- 172.21.15.29:0/951391595 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb268000e90 con 0x7fb27813fe20 2024-04-25T00:31:42.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb274ff9640 1 -- 172.21.15.29:0/951391595 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2680048b0 con 0x7fb27813fe20 2024-04-25T00:31:42.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.684+0000 7fb27e58f640 1 -- 172.21.15.29:0/951391595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb278135a90 con 0x7fb27813fe20 2024-04-25T00:31:42.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.688+0000 7fb27e58f640 1 -- 172.21.15.29:0/951391595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb278135c90 con 0x7fb27813fe20 2024-04-25T00:31:42.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.688+0000 7fb274ff9640 1 -- 172.21.15.29:0/951391595 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb268004a50 con 0x7fb27813fe20 2024-04-25T00:31:42.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.688+0000 7fb274ff9640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb2640678c0 0x7fb264069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:42.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.688+0000 7fb274ff9640 1 -- 172.21.15.29:0/951391595 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb26811f250 con 0x7fb27813fe20 2024-04-25T00:31:42.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.688+0000 7fb277fff640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb2640678c0 0x7fb264069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:42.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.688+0000 7fb277fff640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb2640678c0 0x7fb264069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb270007520 tx=0x7fb27000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:42.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.692+0000 7fb274ff9640 1 -- 172.21.15.29:0/951391595 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb278135c90 con 0x7fb27813fe20 2024-04-25T00:31:42.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.880+0000 7fb27e58f640 1 -- 172.21.15.29:0/951391595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb278069270 con 0x7fb27813fe20 2024-04-25T00:31:42.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.880+0000 7fb274ff9640 1 -- 172.21.15.29:0/951391595 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb2680bf030 con 0x7fb27813fe20 2024-04-25T00:31:42.884 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:42.884 INFO:teuthology.orchestra.run.smithi029.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T00:31:42.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.884+0000 7fb27e58f640 1 -- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb2640678c0 msgr2=0x7fb264069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.884+0000 7fb27e58f640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb2640678c0 0x7fb264069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb270007520 tx=0x7fb27000b040 comp rx=0 tx=0).stop 2024-04-25T00:31:42.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.884+0000 7fb27e58f640 1 -- 172.21.15.29:0/951391595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb27813fe20 msgr2=0x7fb27813bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:42.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.884+0000 7fb27e58f640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb27813fe20 0x7fb27813bf30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb26801c6d0 tx=0x7fb260007a50 comp rx=0 tx=0).stop 2024-04-25T00:31:42.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.884+0000 7fb27e58f640 1 -- 172.21.15.29:0/951391595 shutdown_connections 2024-04-25T00:31:42.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.884+0000 7fb27e58f640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb27813fe20 0x7fb27813bf30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.884+0000 7fb27e58f640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb2640678c0 0x7fb264069d80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.884+0000 7fb27e58f640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb27813dd90 0x7fb27813f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.884+0000 7fb27e58f640 1 --2- 172.21.15.29:0/951391595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb27815a0e0 0x7fb278137570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:42.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.884+0000 7fb27e58f640 1 -- 172.21.15.29:0/951391595 >> 172.21.15.29:0/951391595 conn(0x7fb2780b1870 msgr2=0x7fb2780b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:42.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.884+0000 7fb27e58f640 1 -- 172.21.15.29:0/951391595 shutdown_connections 2024-04-25T00:31:42.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:42.884+0000 7fb27e58f640 1 -- 172.21.15.29:0/951391595 wait complete. 2024-04-25T00:31:42.902 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T00:31:42.902 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:43.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.008+0000 7fb694549640 1 Processor -- start 2024-04-25T00:31:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.008+0000 7fb694549640 1 -- start start 2024-04-25T00:31:43.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb694549640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb68c0b3140 0x7fb68c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb694549640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb68c0b4b10 0x7fb68c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb694549640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb68c0b3a60 0x7fb68c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb694549640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb68c0b41e0 con 0x7fb68c0b3a60 2024-04-25T00:31:43.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb694549640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb68c06d960 con 0x7fb68c0b3140 2024-04-25T00:31:43.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb694549640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb68c06dc40 con 0x7fb68c0b4b10 2024-04-25T00:31:43.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb692ae6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb68c0b3a60 0x7fb68c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb692ae6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb68c0b3a60 0x7fb68c15c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50086/0 (socket says 172.21.15.29:50086) 2024-04-25T00:31:43.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb692ae6640 1 -- 172.21.15.29:0/119232507 learned_addr learned my addr 172.21.15.29:0/119232507 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:43.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb6922e5640 1 --2- 172.21.15.29:0/119232507 >> v2:172.21.15.157:3300/0 conn(0x7fb68c0b3140 0x7fb68c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb691ae4640 1 --2- 172.21.15.29:0/119232507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb68c0b4b10 0x7fb68c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb692ae6640 1 -- 172.21.15.29:0/119232507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb68c0b4b10 msgr2=0x7fb68c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:43.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb692ae6640 1 --2- 172.21.15.29:0/119232507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb68c0b4b10 0x7fb68c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb692ae6640 1 -- 172.21.15.29:0/119232507 >> v2:172.21.15.157:3300/0 conn(0x7fb68c0b3140 msgr2=0x7fb68c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:43.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb692ae6640 1 --2- 172.21.15.29:0/119232507 >> v2:172.21.15.157:3300/0 conn(0x7fb68c0b3140 0x7fb68c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.012+0000 7fb692ae6640 1 -- 172.21.15.29:0/119232507 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68c06df20 con 0x7fb68c0b3a60 2024-04-25T00:31:43.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb692ae6640 1 --2- 172.21.15.29:0/119232507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb68c0b3a60 0x7fb68c15c3e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0d0bf0 tx=0x7fb68401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a1c44e923ab94f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:43.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb6912e3640 1 -- 172.21.15.29:0/119232507 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6840a5670 con 0x7fb68c0b3a60 2024-04-25T00:31:43.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb6912e3640 1 -- 172.21.15.29:0/119232507 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6840a5810 con 0x7fb68c0b3a60 2024-04-25T00:31:43.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb6912e3640 1 -- 172.21.15.29:0/119232507 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6840a5be0 con 0x7fb68c0b3a60 2024-04-25T00:31:43.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 -- 172.21.15.29:0/119232507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb68c0b3a60 msgr2=0x7fb68c15c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 --2- 172.21.15.29:0/119232507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb68c0b3a60 0x7fb68c15c3e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0d0bf0 tx=0x7fb68401c9f0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 -- 172.21.15.29:0/119232507 shutdown_connections 2024-04-25T00:31:43.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 --2- 172.21.15.29:0/119232507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb68c0b3a60 0x7fb68c15c3e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 --2- 172.21.15.29:0/119232507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb68c0b4b10 0x7fb68c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 --2- 172.21.15.29:0/119232507 >> v2:172.21.15.157:3300/0 conn(0x7fb68c0b3140 0x7fb68c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 -- 172.21.15.29:0/119232507 >> 172.21.15.29:0/119232507 conn(0x7fb68c0b1870 msgr2=0x7fb68c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:43.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 -- 172.21.15.29:0/119232507 shutdown_connections 2024-04-25T00:31:43.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 -- 172.21.15.29:0/119232507 wait complete. 2024-04-25T00:31:43.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 Processor -- start 2024-04-25T00:31:43.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 -- start start 2024-04-25T00:31:43.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb68c0b3140 0x7fb68c159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb68c0b3a60 0x7fb68c13d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb68c0b4b10 0x7fb68c15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c0ce3c0 con 0x7fb68c0b4b10 2024-04-25T00:31:43.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c0ce240 con 0x7fb68c0b3140 2024-04-25T00:31:43.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb694549640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb68c0ce540 con 0x7fb68c0b3a60 2024-04-25T00:31:43.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb692ae6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb68c0b4b10 0x7fb68c15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb692ae6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb68c0b4b10 0x7fb68c15b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50100/0 (socket says 172.21.15.29:50100) 2024-04-25T00:31:43.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb692ae6640 1 -- 172.21.15.29:0/1242575490 learned_addr learned my addr 172.21.15.29:0/1242575490 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:43.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb6922e5640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb68c0b3140 0x7fb68c159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb691ae4640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb68c0b3a60 0x7fb68c13d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb692ae6640 1 -- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb68c0b3a60 msgr2=0x7fb68c13d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb692ae6640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb68c0b3a60 0x7fb68c13d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb692ae6640 1 -- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb68c0b3140 msgr2=0x7fb68c159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb692ae6640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb68c0b3140 0x7fb68c159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb692ae6640 1 -- 172.21.15.29:0/1242575490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68c137830 con 0x7fb68c0b4b10 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.016+0000 7fb6922e5640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb68c0b3140 0x7fb68c159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.020+0000 7fb692ae6640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb68c0b4b10 0x7fb68c15b1b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb68401b0a0 tx=0x7fb684019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.020+0000 7fb68affd640 1 -- 172.21.15.29:0/1242575490 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6840bc020 con 0x7fb68c0b4b10 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.020+0000 7fb68affd640 1 -- 172.21.15.29:0/1242575490 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6840b8670 con 0x7fb68c0b4b10 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.020+0000 7fb694549640 1 -- 172.21.15.29:0/1242575490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68c137b10 con 0x7fb68c0b4b10 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.020+0000 7fb68affd640 1 -- 172.21.15.29:0/1242575490 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684009510 con 0x7fb68c0b4b10 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.020+0000 7fb694549640 1 -- 172.21.15.29:0/1242575490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb68c138d40 con 0x7fb68c0b4b10 2024-04-25T00:31:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.020+0000 7fb694549640 1 -- 172.21.15.29:0/1242575490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb68c069270 con 0x7fb68c0b4b10 2024-04-25T00:31:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.020+0000 7fb68affd640 1 -- 172.21.15.29:0/1242575490 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb6840bf020 con 0x7fb68c0b4b10 2024-04-25T00:31:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.020+0000 7fb68affd640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb6780678c0 0x7fb678069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.020+0000 7fb68affd640 1 -- 172.21.15.29:0/1242575490 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb684120c30 con 0x7fb68c0b4b10 2024-04-25T00:31:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.020+0000 7fb6922e5640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb6780678c0 0x7fb678069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.020+0000 7fb6922e5640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb6780678c0 0x7fb678069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb67c004660 tx=0x7fb67c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:43.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.024+0000 7fb68affd640 1 -- 172.21.15.29:0/1242575490 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb6840c6030 con 0x7fb68c0b4b10 2024-04-25T00:31:43.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.180+0000 7fb694549640 1 -- 172.21.15.29:0/1242575490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb68c144740 con 0x7fb68c0b4b10 2024-04-25T00:31:43.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.184+0000 7fb68affd640 1 -- 172.21.15.29:0/1242575490 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fb6840165e0 con 0x7fb68c0b4b10 2024-04-25T00:31:43.186 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:43.186 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:43.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.184+0000 7fb694549640 1 -- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb6780678c0 msgr2=0x7fb678069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.184+0000 7fb694549640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb6780678c0 0x7fb678069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb67c004660 tx=0x7fb67c008040 comp rx=0 tx=0).stop 2024-04-25T00:31:43.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.184+0000 7fb694549640 1 -- 172.21.15.29:0/1242575490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb68c0b4b10 msgr2=0x7fb68c15b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.184+0000 7fb694549640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb68c0b4b10 0x7fb68c15b1b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb68401b0a0 tx=0x7fb684019790 comp rx=0 tx=0).stop 2024-04-25T00:31:43.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.184+0000 7fb694549640 1 -- 172.21.15.29:0/1242575490 shutdown_connections 2024-04-25T00:31:43.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.184+0000 7fb694549640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb68c0b4b10 0x7fb68c15b1b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.184+0000 7fb694549640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb68c0b3a60 0x7fb68c13d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.184+0000 7fb694549640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb6780678c0 0x7fb678069d80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.184+0000 7fb694549640 1 --2- 172.21.15.29:0/1242575490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb68c0b3140 0x7fb68c159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.184+0000 7fb694549640 1 -- 172.21.15.29:0/1242575490 >> 172.21.15.29:0/1242575490 conn(0x7fb68c0b1870 msgr2=0x7fb68c13c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:43.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.184+0000 7fb694549640 1 -- 172.21.15.29:0/1242575490 shutdown_connections 2024-04-25T00:31:43.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.184+0000 7fb694549640 1 -- 172.21.15.29:0/1242575490 wait complete. 2024-04-25T00:31:43.203 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:31:43.204 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:31:43.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff50ca9e640 1 Processor -- start 2024-04-25T00:31:43.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff50ca9e640 1 -- start start 2024-04-25T00:31:43.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff50ca9e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff50815e200 0x7ff5080d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff50ca9e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5080cf690 0x7ff5080d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff50ca9e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5080d2a90 0x7ff5080d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff50ca9e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff5080d2ee0 con 0x7ff5080d2a90 2024-04-25T00:31:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff50ca9e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff5080b38d0 con 0x7ff50815e200 2024-04-25T00:31:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff50ca9e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff5080cfc10 con 0x7ff5080cf690 2024-04-25T00:31:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff506d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5080d2a90 0x7ff5080d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff50659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff50815e200 0x7ff5080d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff506d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5080d2a90 0x7ff5080d05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50108/0 (socket says 172.21.15.29:50108) 2024-04-25T00:31:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff506d9d640 1 -- 172.21.15.29:0/1248416054 learned_addr learned my addr 172.21.15.29:0/1248416054 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff505d9b640 1 --2- 172.21.15.29:0/1248416054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5080cf690 0x7ff5080d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff506d9d640 1 -- 172.21.15.29:0/1248416054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5080cf690 msgr2=0x7ff5080d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff506d9d640 1 --2- 172.21.15.29:0/1248416054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5080cf690 0x7ff5080d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff506d9d640 1 -- 172.21.15.29:0/1248416054 >> v2:172.21.15.157:3300/0 conn(0x7ff50815e200 msgr2=0x7ff5080d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff506d9d640 1 --2- 172.21.15.29:0/1248416054 >> v2:172.21.15.157:3300/0 conn(0x7ff50815e200 0x7ff5080d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.324+0000 7ff506d9d640 1 -- 172.21.15.29:0/1248416054 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5080cfef0 con 0x7ff5080d2a90 2024-04-25T00:31:43.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff506d9d640 1 --2- 172.21.15.29:0/1248416054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5080d2a90 0x7ff5080d05a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff50808d8f0 tx=0x7ff50001c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d34dfb9bff02d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:43.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50559a640 1 -- 172.21.15.29:0/1248416054 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5000a5580 con 0x7ff5080d2a90 2024-04-25T00:31:43.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50559a640 1 -- 172.21.15.29:0/1248416054 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5000a5720 con 0x7ff5080d2a90 2024-04-25T00:31:43.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50559a640 1 -- 172.21.15.29:0/1248416054 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5000a5af0 con 0x7ff5080d2a90 2024-04-25T00:31:43.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 -- 172.21.15.29:0/1248416054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5080d2a90 msgr2=0x7ff5080d05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 --2- 172.21.15.29:0/1248416054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5080d2a90 0x7ff5080d05a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff50808d8f0 tx=0x7ff50001c700 comp rx=0 tx=0).stop 2024-04-25T00:31:43.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 -- 172.21.15.29:0/1248416054 shutdown_connections 2024-04-25T00:31:43.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 --2- 172.21.15.29:0/1248416054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5080d2a90 0x7ff5080d05a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 --2- 172.21.15.29:0/1248416054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5080cf690 0x7ff5080d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 --2- 172.21.15.29:0/1248416054 >> v2:172.21.15.157:3300/0 conn(0x7ff50815e200 0x7ff5080d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 -- 172.21.15.29:0/1248416054 >> 172.21.15.29:0/1248416054 conn(0x7ff50809d280 msgr2=0x7ff5080d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:43.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 -- 172.21.15.29:0/1248416054 shutdown_connections 2024-04-25T00:31:43.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 -- 172.21.15.29:0/1248416054 wait complete. 2024-04-25T00:31:43.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 Processor -- start 2024-04-25T00:31:43.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 -- start start 2024-04-25T00:31:43.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5080cf690 0x7ff50815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5080d2a90 0x7ff508156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50815e200 0x7ff5080ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff50815e760 con 0x7ff5080d2a90 2024-04-25T00:31:43.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff50815e5e0 con 0x7ff50815e200 2024-04-25T00:31:43.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50ca9e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff5080b4320 con 0x7ff5080cf690 2024-04-25T00:31:43.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff505d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5080d2a90 0x7ff508156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff505d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5080d2a90 0x7ff508156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50124/0 (socket says 172.21.15.29:50124) 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff505d9b640 1 -- 172.21.15.29:0/391894444 learned_addr learned my addr 172.21.15.29:0/391894444 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff506d9d640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50815e200 0x7ff5080ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff50659c640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5080cf690 0x7ff50815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff505d9b640 1 -- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5080cf690 msgr2=0x7ff50815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff505d9b640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5080cf690 0x7ff50815a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff505d9b640 1 -- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50815e200 msgr2=0x7ff5080ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff505d9b640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50815e200 0x7ff5080ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff505d9b640 1 -- 172.21.15.29:0/391894444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff508159790 con 0x7ff5080d2a90 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.328+0000 7ff505d9b640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5080d2a90 0x7ff508156f50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00d830 tx=0x7ff4fc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.332+0000 7ff4f6ffd640 1 -- 172.21.15.29:0/391894444 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc018020 con 0x7ff5080d2a90 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.332+0000 7ff4f6ffd640 1 -- 172.21.15.29:0/391894444 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4fc0096c0 con 0x7ff5080d2a90 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.332+0000 7ff50ca9e640 1 -- 172.21.15.29:0/391894444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff508159a70 con 0x7ff5080d2a90 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.332+0000 7ff4f6ffd640 1 -- 172.21.15.29:0/391894444 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc009ab0 con 0x7ff5080d2a90 2024-04-25T00:31:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.332+0000 7ff50ca9e640 1 -- 172.21.15.29:0/391894444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50815a310 con 0x7ff5080d2a90 2024-04-25T00:31:43.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.332+0000 7ff50ca9e640 1 -- 172.21.15.29:0/391894444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff50815a510 con 0x7ff5080d2a90 2024-04-25T00:31:43.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.332+0000 7ff4f6ffd640 1 -- 172.21.15.29:0/391894444 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff4fc01b020 con 0x7ff5080d2a90 2024-04-25T00:31:43.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.332+0000 7ff4f6ffd640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff4e80678f0 0x7ff4e8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.332+0000 7ff4f6ffd640 1 -- 172.21.15.29:0/391894444 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff4fc086180 con 0x7ff5080d2a90 2024-04-25T00:31:43.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.332+0000 7ff50659c640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff4e80678f0 0x7ff4e8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.332+0000 7ff50659c640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff4e80678f0 0x7ff4e8069db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff5080b2e50 tx=0x7ff4f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:43.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.336+0000 7ff4f6ffd640 1 -- 172.21.15.29:0/391894444 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff50815a510 con 0x7ff5080d2a90 2024-04-25T00:31:43.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff50ca9e640 1 -- 172.21.15.29:0/391894444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff508069270 con 0x7ff5080d2a90 2024-04-25T00:31:43.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff4f6ffd640 1 -- 172.21.15.29:0/391894444 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7ff4fc022020 con 0x7ff5080d2a90 2024-04-25T00:31:43.494 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:31:43.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff50ca9e640 1 -- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff4e80678f0 msgr2=0x7ff4e8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff50ca9e640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff4e80678f0 0x7ff4e8069db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff5080b2e50 tx=0x7ff4f8008040 comp rx=0 tx=0).stop 2024-04-25T00:31:43.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff50ca9e640 1 -- 172.21.15.29:0/391894444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5080d2a90 msgr2=0x7ff508156f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff50ca9e640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5080d2a90 0x7ff508156f50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00d830 tx=0x7ff4fc00dcf0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff50ca9e640 1 -- 172.21.15.29:0/391894444 shutdown_connections 2024-04-25T00:31:43.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff50ca9e640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff4e80678f0 0x7ff4e8069db0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff50ca9e640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50815e200 0x7ff5080ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff50ca9e640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5080d2a90 0x7ff508156f50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff50ca9e640 1 --2- 172.21.15.29:0/391894444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5080cf690 0x7ff50815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff50ca9e640 1 -- 172.21.15.29:0/391894444 >> 172.21.15.29:0/391894444 conn(0x7ff50809d280 msgr2=0x7ff5080d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:43.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff50ca9e640 1 -- 172.21.15.29:0/391894444 shutdown_connections 2024-04-25T00:31:43.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.492+0000 7ff50ca9e640 1 -- 172.21.15.29:0/391894444 wait complete. 2024-04-25T00:31:43.511 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:31:43.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c8a25b640 1 Processor -- start 2024-04-25T00:31:43.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c8a25b640 1 -- start start 2024-04-25T00:31:43.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c8a25b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c840aec60 0x7f7c840cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c8a25b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7c840af040 0x7f7c8415c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c8a25b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c8415ccd0 con 0x7f7c8415d720 2024-04-25T00:31:43.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c8a25b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7c8415caa0 con 0x7f7c840aec60 2024-04-25T00:31:43.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c8a25b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7c8415cfb0 con 0x7f7c840af040 2024-04-25T00:31:43.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c837fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c840aec60 0x7f7c840cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c837fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c840aec60 0x7f7c840cfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50132/0 (socket says 172.21.15.29:50132) 2024-04-25T00:31:43.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c837fe640 1 -- 172.21.15.29:0/2321097922 learned_addr learned my addr 172.21.15.29:0/2321097922 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:43.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c82ffd640 1 --2- 172.21.15.29:0/2321097922 >> v2:172.21.15.157:3300/0 conn(0x7f7c840af040 0x7f7c8415c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c837fe640 1 -- 172.21.15.29:0/2321097922 >> v2:172.21.15.157:3300/0 conn(0x7f7c840af040 msgr2=0x7f7c8415c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:43.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c837fe640 1 --2- 172.21.15.29:0/2321097922 >> v2:172.21.15.157:3300/0 conn(0x7f7c840af040 0x7f7c8415c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c837fe640 1 -- 172.21.15.29:0/2321097922 >> v1:172.21.15.157:6789/0 conn(0x7f7c8415d720 legacy=0x7f7c840b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:43.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c837fe640 1 -- 172.21.15.29:0/2321097922 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8415d290 con 0x7f7c840aec60 2024-04-25T00:31:43.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.632+0000 7f7c82ffd640 1 --2- 172.21.15.29:0/2321097922 >> v2:172.21.15.157:3300/0 conn(0x7f7c840af040 0x7f7c8415c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:43.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c837fe640 1 --2- 172.21.15.29:0/2321097922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c840aec60 0x7f7c840cfe40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7c840cf4b0 tx=0x7f7c7c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b815847a3905c4ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:43.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c827fc640 1 -- 172.21.15.29:0/2321097922 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c0a55e0 con 0x7f7c840aec60 2024-04-25T00:31:43.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c827fc640 1 -- 172.21.15.29:0/2321097922 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c7c0a5780 con 0x7f7c840aec60 2024-04-25T00:31:43.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c827fc640 1 -- 172.21.15.29:0/2321097922 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c0a5b50 con 0x7f7c840aec60 2024-04-25T00:31:43.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c8a25b640 1 -- 172.21.15.29:0/2321097922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c840aec60 msgr2=0x7f7c840cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c8a25b640 1 --2- 172.21.15.29:0/2321097922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c840aec60 0x7f7c840cfe40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7c840cf4b0 tx=0x7f7c7c01c920 comp rx=0 tx=0).stop 2024-04-25T00:31:43.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c8a25b640 1 -- 172.21.15.29:0/2321097922 shutdown_connections 2024-04-25T00:31:43.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c8a25b640 1 --2- 172.21.15.29:0/2321097922 >> v2:172.21.15.157:3300/0 conn(0x7f7c840af040 0x7f7c8415c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c8a25b640 1 --2- 172.21.15.29:0/2321097922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c840aec60 0x7f7c840cfe40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c8a25b640 1 -- 172.21.15.29:0/2321097922 >> 172.21.15.29:0/2321097922 conn(0x7f7c84075030 msgr2=0x7f7c840b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:43.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c8a25b640 1 -- 172.21.15.29:0/2321097922 shutdown_connections 2024-04-25T00:31:43.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c8a25b640 1 -- 172.21.15.29:0/2321097922 wait complete. 2024-04-25T00:31:43.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c8a25b640 1 Processor -- start 2024-04-25T00:31:43.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.636+0000 7f7c8a25b640 1 -- start start 2024-04-25T00:31:43.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c8a25b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c840aec60 0x7f7c8414ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c8a25b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c840af040 0x7f7c8413dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c8a25b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8415d720 0x7f7c840aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c8a25b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7c84168d20 con 0x7f7c8415d720 2024-04-25T00:31:43.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c8a25b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c84168ba0 con 0x7f7c840aec60 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c8a25b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c84168ea0 con 0x7f7c840af040 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c83fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8415d720 0x7f7c840aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c83fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8415d720 0x7f7c840aabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50138/0 (socket says 172.21.15.29:50138) 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c83fff640 1 -- 172.21.15.29:0/3480120934 learned_addr learned my addr 172.21.15.29:0/3480120934 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c837fe640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c840aec60 0x7f7c8414ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c82ffd640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c840af040 0x7f7c8413dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c83fff640 1 -- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c840af040 msgr2=0x7f7c8413dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c83fff640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c840af040 0x7f7c8413dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c83fff640 1 -- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c840aec60 msgr2=0x7f7c8414ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c83fff640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c840aec60 0x7f7c8414ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c83fff640 1 -- 172.21.15.29:0/3480120934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c840ac7c0 con 0x7f7c8415d720 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c83fff640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8415d720 0x7f7c840aabf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400c3c0 tx=0x7f7c7400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c7bfff640 1 -- 172.21.15.29:0/3480120934 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74011790 con 0x7f7c8415d720 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c7bfff640 1 -- 172.21.15.29:0/3480120934 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c74017070 con 0x7f7c8415d720 2024-04-25T00:31:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c8a25b640 1 -- 172.21.15.29:0/3480120934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c840ac200 con 0x7f7c8415d720 2024-04-25T00:31:43.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c7bfff640 1 -- 172.21.15.29:0/3480120934 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74011b70 con 0x7f7c8415d720 2024-04-25T00:31:43.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c8a25b640 1 -- 172.21.15.29:0/3480120934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c840aa3f0 con 0x7f7c8415d720 2024-04-25T00:31:43.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c8a25b640 1 -- 172.21.15.29:0/3480120934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c84069270 con 0x7f7c8415d720 2024-04-25T00:31:43.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c7bfff640 1 -- 172.21.15.29:0/3480120934 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7c74011d10 con 0x7f7c8415d720 2024-04-25T00:31:43.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c7bfff640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c680678c0 0x7f7c68069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c7bfff640 1 -- 172.21.15.29:0/3480120934 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7c74085a00 con 0x7f7c8415d720 2024-04-25T00:31:43.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c837fe640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c680678c0 0x7f7c68069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.640+0000 7f7c837fe640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c680678c0 0x7f7c68069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0ae000 tx=0x7f7c7c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:43.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.644+0000 7f7c7bfff640 1 -- 172.21.15.29:0/3480120934 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7c740565d0 con 0x7f7c8415d720 2024-04-25T00:31:43.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.800+0000 7f7c8a25b640 1 -- 172.21.15.29:0/3480120934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7c841502f0 con 0x7f7c8415d720 2024-04-25T00:31:43.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.804+0000 7f7c7bfff640 1 -- 172.21.15.29:0/3480120934 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f7c74055ed0 con 0x7f7c8415d720 2024-04-25T00:31:43.805 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:31:43.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.804+0000 7f7c8a25b640 1 -- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c680678c0 msgr2=0x7f7c68069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.804+0000 7f7c8a25b640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c680678c0 0x7f7c68069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0ae000 tx=0x7f7c7c0a7070 comp rx=0 tx=0).stop 2024-04-25T00:31:43.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.804+0000 7f7c8a25b640 1 -- 172.21.15.29:0/3480120934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8415d720 msgr2=0x7f7c840aabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.804+0000 7f7c8a25b640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8415d720 0x7f7c840aabf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400c3c0 tx=0x7f7c7400c880 comp rx=0 tx=0).stop 2024-04-25T00:31:43.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.804+0000 7f7c8a25b640 1 -- 172.21.15.29:0/3480120934 shutdown_connections 2024-04-25T00:31:43.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.804+0000 7f7c8a25b640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8415d720 0x7f7c840aabf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.804+0000 7f7c8a25b640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c840af040 0x7f7c8413dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.804+0000 7f7c8a25b640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c680678c0 0x7f7c68069d80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.804+0000 7f7c8a25b640 1 --2- 172.21.15.29:0/3480120934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c840aec60 0x7f7c8414ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.804+0000 7f7c8a25b640 1 -- 172.21.15.29:0/3480120934 >> 172.21.15.29:0/3480120934 conn(0x7f7c84075030 msgr2=0x7f7c84166320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:43.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.804+0000 7f7c8a25b640 1 -- 172.21.15.29:0/3480120934 shutdown_connections 2024-04-25T00:31:43.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.804+0000 7f7c8a25b640 1 -- 172.21.15.29:0/3480120934 wait complete. 2024-04-25T00:31:43.822 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:31:43.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.936+0000 7ff7a425e640 1 Processor -- start 2024-04-25T00:31:43.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.936+0000 7ff7a425e640 1 -- start start 2024-04-25T00:31:43.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.940+0000 7ff7a425e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff79c15e8d0 0x7ff79c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.940+0000 7ff7a425e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79c0b4780 0x7ff79c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.940+0000 7ff7a425e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff79c0b3720 con 0x7ff79c15f1f0 2024-04-25T00:31:43.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.940+0000 7ff7a425e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff79c0b42f0 con 0x7ff79c15e8d0 2024-04-25T00:31:43.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.940+0000 7ff7a425e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff79c15d7e0 con 0x7ff79c0b4780 2024-04-25T00:31:43.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.940+0000 7ff7a1ffa640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff79c15e8d0 0x7ff79c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.940+0000 7ff7a1ffa640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff79c15e8d0 0x7ff79c0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50146/0 (socket says 172.21.15.29:50146) 2024-04-25T00:31:43.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.940+0000 7ff7a1ffa640 1 -- 172.21.15.29:0/3397429692 learned_addr learned my addr 172.21.15.29:0/3397429692 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:43.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.940+0000 7ff7a17f9640 1 --2- 172.21.15.29:0/3397429692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79c0b4780 0x7ff79c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.940+0000 7ff7a1ffa640 1 -- 172.21.15.29:0/3397429692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79c0b4780 msgr2=0x7ff79c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:43.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.940+0000 7ff7a1ffa640 1 --2- 172.21.15.29:0/3397429692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79c0b4780 0x7ff79c15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.940+0000 7ff7a1ffa640 1 -- 172.21.15.29:0/3397429692 >> v1:172.21.15.157:6789/0 conn(0x7ff79c15f1f0 legacy=0x7ff79c15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.940+0000 7ff7a1ffa640 1 -- 172.21.15.29:0/3397429692 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79c0b39c0 con 0x7ff79c15e8d0 2024-04-25T00:31:43.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a1ffa640 1 --2- 172.21.15.29:0/3397429692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff79c15e8d0 0x7ff79c0cc240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff79800bb10 tx=0x7ff79801ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a61876b48a0c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:43.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a0ff8640 1 -- 172.21.15.29:0/3397429692 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980b4070 con 0x7ff79c15e8d0 2024-04-25T00:31:43.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a0ff8640 1 -- 172.21.15.29:0/3397429692 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7980a7cd0 con 0x7ff79c15e8d0 2024-04-25T00:31:43.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a0ff8640 1 -- 172.21.15.29:0/3397429692 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c15c1c0 con 0x7ff79c15e8d0 2024-04-25T00:31:43.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 -- 172.21.15.29:0/3397429692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff79c15e8d0 msgr2=0x7ff79c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 --2- 172.21.15.29:0/3397429692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff79c15e8d0 0x7ff79c0cc240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff79800bb10 tx=0x7ff79801ec00 comp rx=0 tx=0).stop 2024-04-25T00:31:43.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 -- 172.21.15.29:0/3397429692 shutdown_connections 2024-04-25T00:31:43.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 --2- 172.21.15.29:0/3397429692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79c0b4780 0x7ff79c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 --2- 172.21.15.29:0/3397429692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff79c15e8d0 0x7ff79c0cc240 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 -- 172.21.15.29:0/3397429692 >> 172.21.15.29:0/3397429692 conn(0x7ff79c0b1870 msgr2=0x7ff79c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:43.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 -- 172.21.15.29:0/3397429692 shutdown_connections 2024-04-25T00:31:43.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 -- 172.21.15.29:0/3397429692 wait complete. 2024-04-25T00:31:43.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 Processor -- start 2024-04-25T00:31:43.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 -- start start 2024-04-25T00:31:43.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79c0b4780 0x7ff79c158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff79c15e8d0 0x7ff79c15aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff79c15f1f0 0x7ff79c1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff79c144a20 con 0x7ff79c15f1f0 2024-04-25T00:31:43.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff79c15f600 con 0x7ff79c15e8d0 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a425e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff79c15aa20 con 0x7ff79c0b4780 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a27fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff79c15f1f0 0x7ff79c1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a27fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff79c15f1f0 0x7ff79c1593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50162/0 (socket says 172.21.15.29:50162) 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.944+0000 7ff7a27fb640 1 -- 172.21.15.29:0/2080798594 learned_addr learned my addr 172.21.15.29:0/2080798594 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7a17f9640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff79c15e8d0 0x7ff79c15aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7a1ffa640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79c0b4780 0x7ff79c158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7a27fb640 1 -- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79c0b4780 msgr2=0x7ff79c158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7a27fb640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79c0b4780 0x7ff79c158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7a27fb640 1 -- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff79c15e8d0 msgr2=0x7ff79c15aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7a27fb640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff79c15e8d0 0x7ff79c15aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7a27fb640 1 -- 172.21.15.29:0/2080798594 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79c159e10 con 0x7ff79c15f1f0 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7a27fb640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff79c15f1f0 0x7ff79c1593c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff79400c730 tx=0x7ff79400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7927fc640 1 -- 172.21.15.29:0/2080798594 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794003b80 con 0x7ff79c15f1f0 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7927fc640 1 -- 172.21.15.29:0/2080798594 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff794003d20 con 0x7ff79c15f1f0 2024-04-25T00:31:43.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7927fc640 1 -- 172.21.15.29:0/2080798594 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79400aa30 con 0x7ff79c15f1f0 2024-04-25T00:31:43.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7a425e640 1 -- 172.21.15.29:0/2080798594 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79c15a0f0 con 0x7ff79c15f1f0 2024-04-25T00:31:43.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7a425e640 1 -- 172.21.15.29:0/2080798594 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff79c15a6b0 con 0x7ff79c15f1f0 2024-04-25T00:31:43.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7a425e640 1 -- 172.21.15.29:0/2080798594 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff79c157f90 con 0x7ff79c15f1f0 2024-04-25T00:31:43.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7927fc640 1 -- 172.21.15.29:0/2080798594 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff79400abd0 con 0x7ff79c15f1f0 2024-04-25T00:31:43.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7927fc640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7800678c0 0x7ff780069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:43.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7927fc640 1 -- 172.21.15.29:0/2080798594 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff7940865a0 con 0x7ff79c15f1f0 2024-04-25T00:31:43.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7a1ffa640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7800678c0 0x7ff780069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:43.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.948+0000 7ff7a1ffa640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7800678c0 0x7ff780069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff7980b1000 tx=0x7ff7980a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:43.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:43.952+0000 7ff7927fc640 1 -- 172.21.15.29:0/2080798594 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff794057150 con 0x7ff79c15f1f0 2024-04-25T00:31:44.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7a425e640 1 -- 172.21.15.29:0/2080798594 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff79c069270 con 0x7ff79c15f1f0 2024-04-25T00:31:44.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7927fc640 1 -- 172.21.15.29:0/2080798594 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ff794056a50 con 0x7ff79c15f1f0 2024-04-25T00:31:44.110 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:31:44.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7a425e640 1 -- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7800678c0 msgr2=0x7ff780069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7a425e640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7800678c0 0x7ff780069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff7980b1000 tx=0x7ff7980a9070 comp rx=0 tx=0).stop 2024-04-25T00:31:44.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7a425e640 1 -- 172.21.15.29:0/2080798594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff79c15f1f0 msgr2=0x7ff79c1593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7a425e640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff79c15f1f0 0x7ff79c1593c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff79400c730 tx=0x7ff79400cbf0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7a425e640 1 -- 172.21.15.29:0/2080798594 shutdown_connections 2024-04-25T00:31:44.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7a425e640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff79c15f1f0 0x7ff79c1593c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7a425e640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7800678c0 0x7ff780069d80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7a425e640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff79c15e8d0 0x7ff79c15aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7a425e640 1 --2- 172.21.15.29:0/2080798594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff79c0b4780 0x7ff79c158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7a425e640 1 -- 172.21.15.29:0/2080798594 >> 172.21.15.29:0/2080798594 conn(0x7ff79c0b1870 msgr2=0x7ff79c0afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:44.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7a425e640 1 -- 172.21.15.29:0/2080798594 shutdown_connections 2024-04-25T00:31:44.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.108+0000 7ff7a425e640 1 -- 172.21.15.29:0/2080798594 wait complete. 2024-04-25T00:31:44.130 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:31:44.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f8459480640 1 Processor -- start 2024-04-25T00:31:44.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f8459480640 1 -- start start 2024-04-25T00:31:44.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f8459480640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f84540ccb70 0x7f84540ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f8459480640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84540d1340 con 0x7f84540b1290 2024-04-25T00:31:44.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f8459480640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f845406c4a0 con 0x7f845415e2f0 2024-04-25T00:31:44.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f8459480640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f845415e6d0 con 0x7f84540ccb70 2024-04-25T00:31:44.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f844bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f84540b1290 0x7f84540cc460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50084/0 (socket says 172.21.15.29:50084) 2024-04-25T00:31:44.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f844bfff640 1 -- 172.21.15.29:0/2316023257 learned_addr learned my addr 172.21.15.29:0/2316023257 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:44.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84537fe640 1 --2- 172.21.15.29:0/2316023257 >> v2:172.21.15.157:3300/0 conn(0x7f84540ccb70 0x7f84540ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1742814587 0 0) 0x7f84540d1340 con 0x7f84540b1290 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8440003530 con 0x7f84540b1290 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4070860068 0 0) 0x7f8440003530 con 0x7f84540b1290 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 >> v2:172.21.15.157:3300/0 conn(0x7f84540ccb70 msgr2=0x7f84540ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 --2- 172.21.15.29:0/2316023257 >> v2:172.21.15.157:3300/0 conn(0x7f84540ccb70 0x7f84540ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 >> v1:172.21.15.157:6789/0 conn(0x7f845415e2f0 legacy=0x7f845406cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845406c780 con 0x7f84540b1290 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8444001800 con 0x7f84540b1290 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 >> v1:172.21.15.29:6789/0 conn(0x7f84540b1290 legacy=0x7f84540cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 --2- 172.21.15.29:0/2316023257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400039c0 0x7f8440003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 --2- 172.21.15.29:0/2316023257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8440004810 0x7f8440004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 --2- 172.21.15.29:0/2316023257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400051d0 0x7f844000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f845415e6d0 con 0x7f84400051d0 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f844000c080 con 0x7f84400039c0 2024-04-25T00:31:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f844000c280 con 0x7f8440004810 2024-04-25T00:31:44.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84537fe640 1 --2- 172.21.15.29:0/2316023257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400051d0 0x7f844000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f844bfff640 1 --2- 172.21.15.29:0/2316023257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8440004810 0x7f8440004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84537fe640 1 -- 172.21.15.29:0/2316023257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8440004810 msgr2=0x7f8440004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:44.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84537fe640 1 --2- 172.21.15.29:0/2316023257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8440004810 0x7f8440004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84537fe640 1 -- 172.21.15.29:0/2316023257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400039c0 msgr2=0x7f8440003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:44.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f8452ffd640 1 --2- 172.21.15.29:0/2316023257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400039c0 0x7f8440003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84537fe640 1 --2- 172.21.15.29:0/2316023257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400039c0 0x7f8440003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f84537fe640 1 -- 172.21.15.29:0/2316023257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8440003530 con 0x7f84400051d0 2024-04-25T00:31:44.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.240+0000 7f844bfff640 1 --2- 172.21.15.29:0/2316023257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8440004810 0x7f8440004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84537fe640 1 -- 172.21.15.29:0/2316023257 reap_dead start 2024-04-25T00:31:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84537fe640 1 --2- 172.21.15.29:0/2316023257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400051d0 0x7f844000b9d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f844c009f70 tx=0x7f844c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a748146e3d668f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c0b2070 con 0x7f84400051d0 2024-04-25T00:31:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f844c0a5d30 con 0x7f84400051d0 2024-04-25T00:31:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84527fc640 1 -- 172.21.15.29:0/2316023257 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c009bf0 con 0x7f84400051d0 2024-04-25T00:31:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 -- 172.21.15.29:0/2316023257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400051d0 msgr2=0x7f844000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 --2- 172.21.15.29:0/2316023257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400051d0 0x7f844000b9d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f844c009f70 tx=0x7f844c01cc60 comp rx=0 tx=0).stop 2024-04-25T00:31:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 -- 172.21.15.29:0/2316023257 shutdown_connections 2024-04-25T00:31:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 --2- 172.21.15.29:0/2316023257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400051d0 0x7f844000b9d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 -- 172.21.15.29:0/2316023257 >> 172.21.15.29:0/2316023257 conn(0x7f84540d1e10 msgr2=0x7f84540d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 -- 172.21.15.29:0/2316023257 shutdown_connections 2024-04-25T00:31:44.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 -- 172.21.15.29:0/2316023257 wait complete. 2024-04-25T00:31:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 Processor -- start 2024-04-25T00:31:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 -- start start 2024-04-25T00:31:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400051d0 0x7f8454143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84541f6c70 0x7f8454145290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84541f70f0 0x7f8454157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f84540cf6c0 con 0x7f84541f70f0 2024-04-25T00:31:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f84540d1170 con 0x7f84400051d0 2024-04-25T00:31:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8459480640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f84540d0260 con 0x7f84541f6c70 2024-04-25T00:31:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84537fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84541f70f0 0x7f8454157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84537fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84541f70f0 0x7f8454157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50184/0 (socket says 172.21.15.29:50184) 2024-04-25T00:31:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84537fe640 1 -- 172.21.15.29:0/3829056356 learned_addr learned my addr 172.21.15.29:0/3829056356 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f8452ffd640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400051d0 0x7f8454143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f844bfff640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84541f6c70 0x7f8454145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84537fe640 1 -- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84541f6c70 msgr2=0x7f8454145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84537fe640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84541f6c70 0x7f8454145290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84537fe640 1 -- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400051d0 msgr2=0x7f8454143100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84537fe640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400051d0 0x7f8454143100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84537fe640 1 -- 172.21.15.29:0/3829056356 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84541534d0 con 0x7f84541f70f0 2024-04-25T00:31:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f844bfff640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84541f6c70 0x7f8454145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.244+0000 7f84537fe640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84541f70f0 0x7f8454157e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f844c01ca90 tx=0x7f844c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.248+0000 7f844b7fe640 1 -- 172.21.15.29:0/3829056356 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c0b2040 con 0x7f84541f70f0 2024-04-25T00:31:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.248+0000 7f844b7fe640 1 -- 172.21.15.29:0/3829056356 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f844c0a5b30 con 0x7f84541f70f0 2024-04-25T00:31:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.248+0000 7f8459480640 1 -- 172.21.15.29:0/3829056356 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84540c9990 con 0x7f84541f70f0 2024-04-25T00:31:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.248+0000 7f844b7fe640 1 -- 172.21.15.29:0/3829056356 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c009bf0 con 0x7f84541f70f0 2024-04-25T00:31:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.248+0000 7f8459480640 1 -- 172.21.15.29:0/3829056356 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8454153ff0 con 0x7f84541f70f0 2024-04-25T00:31:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.248+0000 7f8459480640 1 -- 172.21.15.29:0/3829056356 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8454069270 con 0x7f84541f70f0 2024-04-25T00:31:44.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.248+0000 7f844b7fe640 1 -- 172.21.15.29:0/3829056356 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f844c0c0030 con 0x7f84541f70f0 2024-04-25T00:31:44.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.248+0000 7f844b7fe640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84340678c0 0x7f8434069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.248+0000 7f844b7fe640 1 -- 172.21.15.29:0/3829056356 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f844c1206b0 con 0x7f84541f70f0 2024-04-25T00:31:44.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.248+0000 7f8452ffd640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84340678c0 0x7f8434069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.248+0000 7f8452ffd640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84340678c0 0x7f8434069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f843c007fd0 tx=0x7f843c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:44.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.252+0000 7f844b7fe640 1 -- 172.21.15.29:0/3829056356 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f844c0c7060 con 0x7f84541f70f0 2024-04-25T00:31:44.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.440+0000 7f8459480640 1 -- 172.21.15.29:0/3829056356 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84540d12f0 con 0x7f84541f70f0 2024-04-25T00:31:44.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.440+0000 7f844b7fe640 1 -- 172.21.15.29:0/3829056356 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f844c016970 con 0x7f84541f70f0 2024-04-25T00:31:44.444 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:44.444 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1532858457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532858457},{"type":"v1","addr":"172.21.15.29:6833","nonce":1532858457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/718997473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":718997473},{"type":"v1","addr":"172.21.15.157:6835","nonce":718997473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1617208213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1617208213},{"type":"v1","addr":"172.21.15.157:6837","nonce":1617208213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:31:29.640568+0000","modified":"2024-04-25T00:31:31.633585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4252":{"gid":4252,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1924706890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1924706890},{"type":"v1","addr":"172.21.15.29:6835","nonce":1924706890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4252,"qdb_cluster":[4252]},"id":1}]} 2024-04-25T00:31:44.444 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 6 2024-04-25T00:31:44.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.444+0000 7f8459480640 1 -- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84340678c0 msgr2=0x7f8434069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.444+0000 7f8459480640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84340678c0 0x7f8434069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f843c007fd0 tx=0x7f843c009040 comp rx=0 tx=0).stop 2024-04-25T00:31:44.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.444+0000 7f8459480640 1 -- 172.21.15.29:0/3829056356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84541f70f0 msgr2=0x7f8454157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.444+0000 7f8459480640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84541f70f0 0x7f8454157e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f844c01ca90 tx=0x7f844c009750 comp rx=0 tx=0).stop 2024-04-25T00:31:44.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.444+0000 7f8459480640 1 -- 172.21.15.29:0/3829056356 shutdown_connections 2024-04-25T00:31:44.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.444+0000 7f8459480640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84541f70f0 0x7f8454157e10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.444+0000 7f8459480640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84541f6c70 0x7f8454145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.444+0000 7f8459480640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84340678c0 0x7f8434069d80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.444+0000 7f8459480640 1 --2- 172.21.15.29:0/3829056356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400051d0 0x7f8454143100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.444+0000 7f8459480640 1 -- 172.21.15.29:0/3829056356 >> 172.21.15.29:0/3829056356 conn(0x7f84540d1e10 msgr2=0x7f845406cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:44.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.444+0000 7f8459480640 1 -- 172.21.15.29:0/3829056356 shutdown_connections 2024-04-25T00:31:44.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.444+0000 7f8459480640 1 -- 172.21.15.29:0/3829056356 wait complete. 2024-04-25T00:31:44.462 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:44.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.572+0000 7f4ec7414640 1 Processor -- start 2024-04-25T00:31:44.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.572+0000 7f4ec7414640 1 -- start start 2024-04-25T00:31:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec7414640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ec015f3c0 0x7f4ec00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec7414640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec00b3720 con 0x7f4ec015e8d0 2024-04-25T00:31:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec7414640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec00b39c0 con 0x7f4ec00b4780 2024-04-25T00:31:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec7414640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec00b42f0 con 0x7f4ec015f3c0 2024-04-25T00:31:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec51b0640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4ec015e8d0 0x7f4ec015ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50098/0 (socket says 172.21.15.29:50098) 2024-04-25T00:31:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec51b0640 1 -- 172.21.15.29:0/1336893198 learned_addr learned my addr 172.21.15.29:0/1336893198 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1190246886 0 0) 0x7f4ec00b3720 con 0x7f4ec015e8d0 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4eac003530 con 0x7f4ec015e8d0 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec59b1640 1 --2- 172.21.15.29:0/1336893198 >> v2:172.21.15.157:3300/0 conn(0x7f4ec015f3c0 0x7f4ec00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3158259112 0 0) 0x7f4eac003530 con 0x7f4ec015e8d0 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 >> v2:172.21.15.157:3300/0 conn(0x7f4ec015f3c0 msgr2=0x7f4ec00cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 --2- 172.21.15.29:0/1336893198 >> v2:172.21.15.157:3300/0 conn(0x7f4ec015f3c0 0x7f4ec00cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 >> v1:172.21.15.157:6789/0 conn(0x7f4ec00b4780 legacy=0x7f4ec015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec00b3ca0 con 0x7f4ec015e8d0 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4ebc001680 con 0x7f4ec015e8d0 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 >> v1:172.21.15.29:6789/0 conn(0x7f4ec015e8d0 legacy=0x7f4ec015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 --2- 172.21.15.29:0/1336893198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac0039c0 0x7f4eac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 --2- 172.21.15.29:0/1336893198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac004810 0x7f4eac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 --2- 172.21.15.29:0/1336893198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eac0051d0 0x7f4eac00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec00b42f0 con 0x7f4eac0051d0 2024-04-25T00:31:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac00e330 con 0x7f4eac0039c0 2024-04-25T00:31:44.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4eac00e530 con 0x7f4eac004810 2024-04-25T00:31:44.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec59b1640 1 --2- 172.21.15.29:0/1336893198 >> v2:172.21.15.157:3300/0 conn(0x7f4ec015f3c0 0x7f4ec00cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:44.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec59b1640 1 --2- 172.21.15.29:0/1336893198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eac0051d0 0x7f4eac00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec49af640 1 --2- 172.21.15.29:0/1336893198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac004810 0x7f4eac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec51b0640 1 --2- 172.21.15.29:0/1336893198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac0039c0 0x7f4eac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec59b1640 1 -- 172.21.15.29:0/1336893198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac004810 msgr2=0x7f4eac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:44.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec59b1640 1 --2- 172.21.15.29:0/1336893198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eac004810 0x7f4eac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec59b1640 1 -- 172.21.15.29:0/1336893198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac0039c0 msgr2=0x7f4eac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:44.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec59b1640 1 --2- 172.21.15.29:0/1336893198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac0039c0 0x7f4eac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec59b1640 1 -- 172.21.15.29:0/1336893198 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac003530 con 0x7f4eac0051d0 2024-04-25T00:31:44.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec51b0640 1 --2- 172.21.15.29:0/1336893198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac0039c0 0x7f4eac003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:44.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.576+0000 7f4ec59b1640 1 -- 172.21.15.29:0/1336893198 reap_dead start 2024-04-25T00:31:44.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec59b1640 1 --2- 172.21.15.29:0/1336893198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eac0051d0 0x7f4eac00dbe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80073e0 tx=0x7f4eb801cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c505d2bf6e8a4471 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:44.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8009c70 con 0x7f4eac0051d0 2024-04-25T00:31:44.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb80a5e40 con 0x7f4eac0051d0 2024-04-25T00:31:44.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4eb7fff640 1 -- 172.21.15.29:0/1336893198 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb80ad660 con 0x7f4eac0051d0 2024-04-25T00:31:44.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 -- 172.21.15.29:0/1336893198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eac0051d0 msgr2=0x7f4eac00dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 --2- 172.21.15.29:0/1336893198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eac0051d0 0x7f4eac00dbe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80073e0 tx=0x7f4eb801cf40 comp rx=0 tx=0).stop 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 -- 172.21.15.29:0/1336893198 shutdown_connections 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 --2- 172.21.15.29:0/1336893198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eac0051d0 0x7f4eac00dbe0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 -- 172.21.15.29:0/1336893198 >> 172.21.15.29:0/1336893198 conn(0x7f4ec00b1870 msgr2=0x7f4ec00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 -- 172.21.15.29:0/1336893198 shutdown_connections 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 -- 172.21.15.29:0/1336893198 wait complete. 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 Processor -- start 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 -- start start 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac0051d0 0x7f4ec013e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ec013f790 0x7f4ec01323c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ec013d310 0x7f4ec0141320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec006cf10 con 0x7f4ec013f790 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec00b3580 con 0x7f4eac0051d0 2024-04-25T00:31:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec0144a20 con 0x7f4ec013d310 2024-04-25T00:31:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec49af640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ec013f790 0x7f4ec01323c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec49af640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ec013f790 0x7f4ec01323c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50214/0 (socket says 172.21.15.29:50214) 2024-04-25T00:31:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec49af640 1 -- 172.21.15.29:0/1701385080 learned_addr learned my addr 172.21.15.29:0/1701385080 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec49af640 1 -- 172.21.15.29:0/1701385080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ec013d310 msgr2=0x7f4ec0141320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec49af640 1 --2- 172.21.15.29:0/1701385080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ec013d310 0x7f4ec0141320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec49af640 1 -- 172.21.15.29:0/1701385080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac0051d0 msgr2=0x7f4ec013e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec49af640 1 --2- 172.21.15.29:0/1701385080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac0051d0 0x7f4ec013e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec49af640 1 -- 172.21.15.29:0/1701385080 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec013daf0 con 0x7f4ec013f790 2024-04-25T00:31:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec49af640 1 --2- 172.21.15.29:0/1701385080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ec013f790 0x7f4ec01323c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000f5f0 tx=0x7f4eb000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4eb5ffb640 1 -- 172.21.15.29:0/1701385080 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00052e0 con 0x7f4ec013f790 2024-04-25T00:31:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4eb5ffb640 1 -- 172.21.15.29:0/1701385080 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb0018070 con 0x7f4ec013f790 2024-04-25T00:31:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 -- 172.21.15.29:0/1701385080 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec013e6f0 con 0x7f4ec013f790 2024-04-25T00:31:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4eb5ffb640 1 -- 172.21.15.29:0/1701385080 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0003820 con 0x7f4ec013f790 2024-04-25T00:31:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.580+0000 7f4ec7414640 1 -- 172.21.15.29:0/1701385080 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec013ecb0 con 0x7f4ec013f790 2024-04-25T00:31:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.584+0000 7f4ec7414640 1 -- 172.21.15.29:0/1701385080 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec0069270 con 0x7f4ec013f790 2024-04-25T00:31:44.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.584+0000 7f4eb5ffb640 1 -- 172.21.15.29:0/1701385080 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4eb00039c0 con 0x7f4ec013f790 2024-04-25T00:31:44.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.584+0000 7f4eb5ffb640 1 --2- 172.21.15.29:0/1701385080 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ea40678c0 0x7f4ea4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.584+0000 7f4eb5ffb640 1 -- 172.21.15.29:0/1701385080 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4eb0085a50 con 0x7f4ec013f790 2024-04-25T00:31:44.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.584+0000 7f4ec51b0640 1 --2- 172.21.15.29:0/1701385080 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ea40678c0 0x7f4ea4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.584+0000 7f4ec51b0640 1 --2- 172.21.15.29:0/1701385080 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ea40678c0 0x7f4ea4069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc001c00 tx=0x7f4ebc001160 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:44.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.588+0000 7f4eb5ffb640 1 -- 172.21.15.29:0/1701385080 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4eb0056590 con 0x7f4ec013f790 2024-04-25T00:31:44.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.752+0000 7f4ec7414640 1 -- 172.21.15.29:0/1701385080 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ec006e7b0 con 0x7f4ec013f790 2024-04-25T00:31:44.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.756+0000 7f4eb5ffb640 1 -- 172.21.15.29:0/1701385080 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f4eb0005480 con 0x7f4ec013f790 2024-04-25T00:31:44.757 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:44.757 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:44.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.756+0000 7f4ec7414640 1 -- 172.21.15.29:0/1701385080 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ea40678c0 msgr2=0x7f4ea4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.756+0000 7f4ec7414640 1 --2- 172.21.15.29:0/1701385080 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ea40678c0 0x7f4ea4069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc001c00 tx=0x7f4ebc001160 comp rx=0 tx=0).stop 2024-04-25T00:31:44.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.756+0000 7f4ec7414640 1 -- 172.21.15.29:0/1701385080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ec013f790 msgr2=0x7f4ec01323c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.756+0000 7f4ec7414640 1 --2- 172.21.15.29:0/1701385080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ec013f790 0x7f4ec01323c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000f5f0 tx=0x7f4eb000fab0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.756+0000 7f4ec7414640 1 -- 172.21.15.29:0/1701385080 shutdown_connections 2024-04-25T00:31:44.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.756+0000 7f4ec7414640 1 --2- 172.21.15.29:0/1701385080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ec013d310 0x7f4ec0141320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.756+0000 7f4ec7414640 1 --2- 172.21.15.29:0/1701385080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ec013f790 0x7f4ec01323c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.756+0000 7f4ec7414640 1 --2- 172.21.15.29:0/1701385080 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ea40678c0 0x7f4ea4069d80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.756+0000 7f4ec7414640 1 --2- 172.21.15.29:0/1701385080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eac0051d0 0x7f4ec013e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.756+0000 7f4ec7414640 1 -- 172.21.15.29:0/1701385080 >> 172.21.15.29:0/1701385080 conn(0x7f4ec00b1870 msgr2=0x7f4ec00b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:44.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.756+0000 7f4ec7414640 1 -- 172.21.15.29:0/1701385080 shutdown_connections 2024-04-25T00:31:44.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.756+0000 7f4ec7414640 1 -- 172.21.15.29:0/1701385080 wait complete. 2024-04-25T00:31:44.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:31:44.777 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:31:44.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe524f33640 1 Processor -- start 2024-04-25T00:31:44.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe524f33640 1 -- start start 2024-04-25T00:31:44.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe524f33640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5200b4e00 0x7fe5200b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe524f33640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe520071690 con 0x7fe5200b30b0 2024-04-25T00:31:44.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe524f33640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe520071910 con 0x7fe520162420 2024-04-25T00:31:44.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe524f33640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe5200b3500 con 0x7fe5200b4e00 2024-04-25T00:31:44.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51ed9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe5200b30b0 0x7fe52006cf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50110/0 (socket says 172.21.15.29:50110) 2024-04-25T00:31:44.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51ed9d640 1 -- 172.21.15.29:0/1277341526 learned_addr learned my addr 172.21.15.29:0/1277341526 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:44.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51dd9b640 1 --2- 172.21.15.29:0/1277341526 >> v2:172.21.15.157:3300/0 conn(0x7fe5200b4e00 0x7fe5200b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1325595032 0 0) 0x7fe5180014a0 con 0x7fe5200b30b0 2024-04-25T00:31:44.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe508003530 con 0x7fe5200b30b0 2024-04-25T00:31:44.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 16172175 0 0) 0x7fe508003530 con 0x7fe5200b30b0 2024-04-25T00:31:44.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 >> v2:172.21.15.157:3300/0 conn(0x7fe5200b4e00 msgr2=0x7fe5200b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:44.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 --2- 172.21.15.29:0/1277341526 >> v2:172.21.15.157:3300/0 conn(0x7fe5200b4e00 0x7fe5200b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 >> v1:172.21.15.157:6789/0 conn(0x7fe520162420 legacy=0x7fe520070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe520071bf0 con 0x7fe5200b30b0 2024-04-25T00:31:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fe518001bd0 con 0x7fe5200b30b0 2024-04-25T00:31:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 >> v1:172.21.15.29:6789/0 conn(0x7fe5200b30b0 legacy=0x7fe52006cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 --2- 172.21.15.29:0/1277341526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5080039c0 0x7fe508003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 --2- 172.21.15.29:0/1277341526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe508004810 0x7fe508004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 --2- 172.21.15.29:0/1277341526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5080051d0 0x7fe50800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe5200b3500 con 0x7fe5080051d0 2024-04-25T00:31:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe50800c070 con 0x7fe508004810 2024-04-25T00:31:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe50800c270 con 0x7fe5080039c0 2024-04-25T00:31:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51ed9d640 1 --2- 172.21.15.29:0/1277341526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5080051d0 0x7fe50800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51dd9b640 1 --2- 172.21.15.29:0/1277341526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe508004810 0x7fe508004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51e59c640 1 --2- 172.21.15.29:0/1277341526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5080039c0 0x7fe508003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51ed9d640 1 -- 172.21.15.29:0/1277341526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5080039c0 msgr2=0x7fe508003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51ed9d640 1 --2- 172.21.15.29:0/1277341526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5080039c0 0x7fe508003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51ed9d640 1 -- 172.21.15.29:0/1277341526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe508004810 msgr2=0x7fe508004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51ed9d640 1 --2- 172.21.15.29:0/1277341526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe508004810 0x7fe508004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51ed9d640 1 -- 172.21.15.29:0/1277341526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe508003530 con 0x7fe5080051d0 2024-04-25T00:31:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51e59c640 1 --2- 172.21.15.29:0/1277341526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5080039c0 0x7fe508003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.896+0000 7fe51dd9b640 1 --2- 172.21.15.29:0/1277341526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe508004810 0x7fe508004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.900+0000 7fe51ed9d640 1 -- 172.21.15.29:0/1277341526 reap_dead start 2024-04-25T00:31:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.900+0000 7fe51ed9d640 1 --2- 172.21.15.29:0/1277341526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5080051d0 0x7fe50800b9d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe51800aa20 tx=0x7fe51801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b7d2aac1f0cfcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.900+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51800a640 con 0x7fe5080051d0 2024-04-25T00:31:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.900+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5180ac640 con 0x7fe5080051d0 2024-04-25T00:31:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.900+0000 7fe51d59a640 1 -- 172.21.15.29:0/1277341526 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180aca30 con 0x7fe5080051d0 2024-04-25T00:31:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.900+0000 7fe524f33640 1 -- 172.21.15.29:0/1277341526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5080051d0 msgr2=0x7fe50800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.900+0000 7fe524f33640 1 --2- 172.21.15.29:0/1277341526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5080051d0 0x7fe50800b9d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe51800aa20 tx=0x7fe51801d360 comp rx=0 tx=0).stop 2024-04-25T00:31:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.900+0000 7fe524f33640 1 -- 172.21.15.29:0/1277341526 shutdown_connections 2024-04-25T00:31:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.900+0000 7fe524f33640 1 --2- 172.21.15.29:0/1277341526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5080051d0 0x7fe50800b9d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.900+0000 7fe524f33640 1 -- 172.21.15.29:0/1277341526 >> 172.21.15.29:0/1277341526 conn(0x7fe5200b1870 msgr2=0x7fe5200b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:44.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.900+0000 7fe524f33640 1 -- 172.21.15.29:0/1277341526 shutdown_connections 2024-04-25T00:31:44.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.900+0000 7fe524f33640 1 -- 172.21.15.29:0/1277341526 wait complete. 2024-04-25T00:31:44.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe524f33640 1 Processor -- start 2024-04-25T00:31:44.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe524f33640 1 -- start start 2024-04-25T00:31:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe524f33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5080051d0 0x7fe5201540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe524f33640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe520159930 0x7fe520155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe524f33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe520155d60 0x7fe52015ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe524f33640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe520162800 con 0x7fe520159930 2024-04-25T00:31:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe524f33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe5200721b0 con 0x7fe5080051d0 2024-04-25T00:31:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe524f33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe5200b2b70 con 0x7fe520155d60 2024-04-25T00:31:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe51dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe520159930 0x7fe520155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe51dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe520159930 0x7fe520155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50228/0 (socket says 172.21.15.29:50228) 2024-04-25T00:31:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe51dd9b640 1 -- 172.21.15.29:0/3393288738 learned_addr learned my addr 172.21.15.29:0/3393288738 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe51e59c640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5080051d0 0x7fe5201540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe51ed9d640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe520155d60 0x7fe52015ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe51dd9b640 1 -- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe520155d60 msgr2=0x7fe52015ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe51dd9b640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe520155d60 0x7fe52015ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe51dd9b640 1 -- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5080051d0 msgr2=0x7fe5201540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe51dd9b640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5080051d0 0x7fe5201540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe51dd9b640 1 -- 172.21.15.29:0/3393288738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe520156200 con 0x7fe520159930 2024-04-25T00:31:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe51dd9b640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe520159930 0x7fe520155820 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe514009fd0 tx=0x7fe514011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe50effd640 1 -- 172.21.15.29:0/3393288738 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514015870 con 0x7fe520159930 2024-04-25T00:31:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe50effd640 1 -- 172.21.15.29:0/3393288738 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe514015a10 con 0x7fe520159930 2024-04-25T00:31:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe524f33640 1 -- 172.21.15.29:0/3393288738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe520158b70 con 0x7fe520159930 2024-04-25T00:31:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe50effd640 1 -- 172.21.15.29:0/3393288738 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514015e00 con 0x7fe520159930 2024-04-25T00:31:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe524f33640 1 -- 172.21.15.29:0/3393288738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe520159130 con 0x7fe520159930 2024-04-25T00:31:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.904+0000 7fe524f33640 1 -- 172.21.15.29:0/3393288738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe520069270 con 0x7fe520159930 2024-04-25T00:31:44.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.908+0000 7fe50effd640 1 -- 172.21.15.29:0/3393288738 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe51401d400 con 0x7fe520159930 2024-04-25T00:31:44.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.908+0000 7fe50effd640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe5000678c0 0x7fe500069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:44.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.908+0000 7fe50effd640 1 -- 172.21.15.29:0/3393288738 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe5140868d0 con 0x7fe520159930 2024-04-25T00:31:44.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.908+0000 7fe51e59c640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe5000678c0 0x7fe500069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:44.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.908+0000 7fe51e59c640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe5000678c0 0x7fe500069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe510000f70 tx=0x7fe510009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:44.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:44.908+0000 7fe50effd640 1 -- 172.21.15.29:0/3393288738 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe514019030 con 0x7fe520159930 2024-04-25T00:31:45.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.064+0000 7fe524f33640 1 -- 172.21.15.29:0/3393288738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe520072fa0 con 0x7fe520159930 2024-04-25T00:31:45.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.064+0000 7fe50effd640 1 -- 172.21.15.29:0/3393288738 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fe5140574f0 con 0x7fe520159930 2024-04-25T00:31:45.069 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:31:45.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.068+0000 7fe524f33640 1 -- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe5000678c0 msgr2=0x7fe500069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.068+0000 7fe524f33640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe5000678c0 0x7fe500069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe510000f70 tx=0x7fe510009040 comp rx=0 tx=0).stop 2024-04-25T00:31:45.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.068+0000 7fe524f33640 1 -- 172.21.15.29:0/3393288738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe520159930 msgr2=0x7fe520155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.068+0000 7fe524f33640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe520159930 0x7fe520155820 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe514009fd0 tx=0x7fe514011a60 comp rx=0 tx=0).stop 2024-04-25T00:31:45.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.068+0000 7fe524f33640 1 -- 172.21.15.29:0/3393288738 shutdown_connections 2024-04-25T00:31:45.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.068+0000 7fe524f33640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe520155d60 0x7fe52015ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.068+0000 7fe524f33640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe520159930 0x7fe520155820 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.068+0000 7fe524f33640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe5000678c0 0x7fe500069d80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.068+0000 7fe524f33640 1 --2- 172.21.15.29:0/3393288738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5080051d0 0x7fe5201540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.068+0000 7fe524f33640 1 -- 172.21.15.29:0/3393288738 >> 172.21.15.29:0/3393288738 conn(0x7fe5200b1870 msgr2=0x7fe520166830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:45.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.068+0000 7fe524f33640 1 -- 172.21.15.29:0/3393288738 shutdown_connections 2024-04-25T00:31:45.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.068+0000 7fe524f33640 1 -- 172.21.15.29:0/3393288738 wait complete. 2024-04-25T00:31:45.086 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:31:45.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f66101a0640 1 Processor -- start 2024-04-25T00:31:45.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f66101a0640 1 -- start start 2024-04-25T00:31:45.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f66101a0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f660815df70 0x7f66080b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f66101a0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66080b4d30 0x7f66080ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f66101a0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f660806d980 con 0x7f660806d210 2024-04-25T00:31:45.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f66101a0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f660806dc60 con 0x7f660815df70 2024-04-25T00:31:45.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f66101a0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f660806df40 con 0x7f66080b4d30 2024-04-25T00:31:45.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f660df3c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f660815df70 0x7f66080b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f660d73b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f660806d210 0x7f66080b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50120/0 (socket says 172.21.15.29:50120) 2024-04-25T00:31:45.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f660d73b640 1 -- 172.21.15.29:0/3083516295 learned_addr learned my addr 172.21.15.29:0/3083516295 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:45.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f660e73d640 1 --2- 172.21.15.29:0/3083516295 >> v2:172.21.15.157:3300/0 conn(0x7f66080b4d30 0x7f66080ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f660df3c640 1 -- 172.21.15.29:0/3083516295 >> v2:172.21.15.157:3300/0 conn(0x7f66080b4d30 msgr2=0x7f66080ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:45.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f660df3c640 1 --2- 172.21.15.29:0/3083516295 >> v2:172.21.15.157:3300/0 conn(0x7f66080b4d30 0x7f66080ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f660df3c640 1 -- 172.21.15.29:0/3083516295 >> v1:172.21.15.29:6789/0 conn(0x7f660806d210 legacy=0x7f66080b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f660df3c640 1 -- 172.21.15.29:0/3083516295 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660806e220 con 0x7f660815df70 2024-04-25T00:31:45.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.204+0000 7f660cf3a640 1 -- 172.21.15.29:0/3083516295 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 930348402 0 0) 0x7f660806d980 con 0x7f660806d210 2024-04-25T00:31:45.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f660df3c640 1 --2- 172.21.15.29:0/3083516295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f660815df70 0x7f66080b3e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6608071fd0 tx=0x7f660001ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d04b9b84ea2b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:45.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f660cf3a640 1 -- 172.21.15.29:0/3083516295 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66000b1070 con 0x7f660815df70 2024-04-25T00:31:45.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f660cf3a640 1 -- 172.21.15.29:0/3083516295 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66000a8050 con 0x7f660815df70 2024-04-25T00:31:45.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f660cf3a640 1 -- 172.21.15.29:0/3083516295 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66000ac860 con 0x7f660815df70 2024-04-25T00:31:45.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f66101a0640 1 -- 172.21.15.29:0/3083516295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f660815df70 msgr2=0x7f66080b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f66101a0640 1 --2- 172.21.15.29:0/3083516295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f660815df70 0x7f66080b3e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6608071fd0 tx=0x7f660001ce20 comp rx=0 tx=0).stop 2024-04-25T00:31:45.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f66101a0640 1 -- 172.21.15.29:0/3083516295 shutdown_connections 2024-04-25T00:31:45.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f66101a0640 1 --2- 172.21.15.29:0/3083516295 >> v2:172.21.15.157:3300/0 conn(0x7f66080b4d30 0x7f66080ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f66101a0640 1 --2- 172.21.15.29:0/3083516295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f660815df70 0x7f66080b3e40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f66101a0640 1 -- 172.21.15.29:0/3083516295 >> 172.21.15.29:0/3083516295 conn(0x7f66080b1870 msgr2=0x7f66080b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:45.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f66101a0640 1 -- 172.21.15.29:0/3083516295 shutdown_connections 2024-04-25T00:31:45.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f66101a0640 1 -- 172.21.15.29:0/3083516295 wait complete. 2024-04-25T00:31:45.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f66101a0640 1 Processor -- start 2024-04-25T00:31:45.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.208+0000 7f66101a0640 1 -- start start 2024-04-25T00:31:45.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f66101a0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f660806d210 0x7f6608135cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f66101a0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66080b4d30 0x7f6608152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f66101a0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f660815df70 0x7f6608153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f66101a0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f660806eba0 con 0x7f660806d210 2024-04-25T00:31:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f66101a0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f660806ea20 con 0x7f660815df70 2024-04-25T00:31:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f66101a0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f66080ae5f0 con 0x7f66080b4d30 2024-04-25T00:31:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f660df3c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f660806d210 0x7f6608135cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f660df3c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f660806d210 0x7f6608135cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50244/0 (socket says 172.21.15.29:50244) 2024-04-25T00:31:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f660df3c640 1 -- 172.21.15.29:0/2289001704 learned_addr learned my addr 172.21.15.29:0/2289001704 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f660e73d640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f660815df70 0x7f6608153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f660df3c640 1 -- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66080b4d30 msgr2=0x7f6608152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f660d73b640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66080b4d30 0x7f6608152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f660df3c640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66080b4d30 0x7f6608152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f660df3c640 1 -- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f660815df70 msgr2=0x7f6608153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f660df3c640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f660815df70 0x7f6608153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f660df3c640 1 -- 172.21.15.29:0/2289001704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660814a9d0 con 0x7f660806d210 2024-04-25T00:31:45.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f660df3c640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f660806d210 0x7f6608135cf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f66000af040 tx=0x7f660001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:45.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f65fe7fc640 1 -- 172.21.15.29:0/2289001704 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66000b1040 con 0x7f660806d210 2024-04-25T00:31:45.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f65fe7fc640 1 -- 172.21.15.29:0/2289001704 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6600004020 con 0x7f660806d210 2024-04-25T00:31:45.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f66101a0640 1 -- 172.21.15.29:0/2289001704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660814acb0 con 0x7f660806d210 2024-04-25T00:31:45.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f65fe7fc640 1 -- 172.21.15.29:0/2289001704 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600006e20 con 0x7f660806d210 2024-04-25T00:31:45.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f66101a0640 1 -- 172.21.15.29:0/2289001704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6608136230 con 0x7f660806d210 2024-04-25T00:31:45.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f66101a0640 1 -- 172.21.15.29:0/2289001704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6608069270 con 0x7f660806d210 2024-04-25T00:31:45.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f65fe7fc640 1 -- 172.21.15.29:0/2289001704 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f66000b4070 con 0x7f660806d210 2024-04-25T00:31:45.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f65fe7fc640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f65f40678c0 0x7f65f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f65fe7fc640 1 -- 172.21.15.29:0/2289001704 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f66001209c0 con 0x7f660806d210 2024-04-25T00:31:45.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.212+0000 7f660d73b640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f65f40678c0 0x7f65f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.216+0000 7f660d73b640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f65f40678c0 0x7f65f4069d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f66081586f0 tx=0x7f6604009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:45.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.216+0000 7f65fe7fc640 1 -- 172.21.15.29:0/2289001704 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f66000c4030 con 0x7f660806d210 2024-04-25T00:31:45.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f66101a0640 1 -- 172.21.15.29:0/2289001704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6608071db0 con 0x7f660806d210 2024-04-25T00:31:45.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f65fe7fc640 1 -- 172.21.15.29:0/2289001704 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f66000bb030 con 0x7f660806d210 2024-04-25T00:31:45.378 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:31:45.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f66101a0640 1 -- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f65f40678c0 msgr2=0x7f65f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f66101a0640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f65f40678c0 0x7f65f4069d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f66081586f0 tx=0x7f6604009040 comp rx=0 tx=0).stop 2024-04-25T00:31:45.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f66101a0640 1 -- 172.21.15.29:0/2289001704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f660806d210 msgr2=0x7f6608135cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f66101a0640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f660806d210 0x7f6608135cf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f66000af040 tx=0x7f660001ac00 comp rx=0 tx=0).stop 2024-04-25T00:31:45.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f66101a0640 1 -- 172.21.15.29:0/2289001704 shutdown_connections 2024-04-25T00:31:45.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f66101a0640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f65f40678c0 0x7f65f4069d80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f66101a0640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f660815df70 0x7f6608153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f66101a0640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66080b4d30 0x7f6608152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f66101a0640 1 --2- 172.21.15.29:0/2289001704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f660806d210 0x7f6608135cf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f66101a0640 1 -- 172.21.15.29:0/2289001704 >> 172.21.15.29:0/2289001704 conn(0x7f66080b1870 msgr2=0x7f66080cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:45.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f66101a0640 1 -- 172.21.15.29:0/2289001704 shutdown_connections 2024-04-25T00:31:45.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.376+0000 7f66101a0640 1 -- 172.21.15.29:0/2289001704 wait complete. 2024-04-25T00:31:45.396 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:31:45.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.520+0000 7fccc4be7640 1 Processor -- start 2024-04-25T00:31:45.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.520+0000 7fccc4be7640 1 -- start start 2024-04-25T00:31:45.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccc4be7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fccc015e8d0 0x7fccc00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccc4be7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fccc015f3c0 0x7fccc015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccc4be7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccc00b3720 con 0x7fccc00b4780 2024-04-25T00:31:45.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccc4be7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fccc00b42f0 con 0x7fccc015f3c0 2024-04-25T00:31:45.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccc4be7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fccc00b39c0 con 0x7fccc015e8d0 2024-04-25T00:31:45.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fccc015f3c0 0x7fccc015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbdd9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fccc00b4780 0x7fccc015ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50124/0 (socket says 172.21.15.29:50124) 2024-04-25T00:31:45.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fccc015f3c0 0x7fccc015c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50258/0 (socket says 172.21.15.29:50258) 2024-04-25T00:31:45.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbdd9b640 1 -- 172.21.15.29:0/4142485051 learned_addr learned my addr 172.21.15.29:0/4142485051 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:45.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbe59c640 1 --2- 172.21.15.29:0/4142485051 >> v2:172.21.15.157:3300/0 conn(0x7fccc015e8d0 0x7fccc00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbed9d640 1 -- 172.21.15.29:0/4142485051 >> v2:172.21.15.157:3300/0 conn(0x7fccc015e8d0 msgr2=0x7fccc00cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:45.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbed9d640 1 --2- 172.21.15.29:0/4142485051 >> v2:172.21.15.157:3300/0 conn(0x7fccc015e8d0 0x7fccc00cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbed9d640 1 -- 172.21.15.29:0/4142485051 >> v1:172.21.15.29:6789/0 conn(0x7fccc00b4780 legacy=0x7fccc015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbed9d640 1 -- 172.21.15.29:0/4142485051 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc00b3ca0 con 0x7fccc015f3c0 2024-04-25T00:31:45.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbd59a640 1 -- 172.21.15.29:0/4142485051 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 333932579 0 0) 0x7fccc00b3720 con 0x7fccc00b4780 2024-04-25T00:31:45.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbe59c640 1 --2- 172.21.15.29:0/4142485051 >> v2:172.21.15.157:3300/0 conn(0x7fccc015e8d0 0x7fccc00cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:45.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbed9d640 1 --2- 172.21.15.29:0/4142485051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccc015f3c0 0x7fccc015c7d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fccc00b5260 tx=0x7fccb001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b149f27217c7bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:45.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbd59a640 1 -- 172.21.15.29:0/4142485051 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00b0070 con 0x7fccc015f3c0 2024-04-25T00:31:45.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbd59a640 1 -- 172.21.15.29:0/4142485051 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb0004230 con 0x7fccc015f3c0 2024-04-25T00:31:45.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.524+0000 7fccbd59a640 1 -- 172.21.15.29:0/4142485051 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0003620 con 0x7fccc015f3c0 2024-04-25T00:31:45.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 -- 172.21.15.29:0/4142485051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccc015f3c0 msgr2=0x7fccc015c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 --2- 172.21.15.29:0/4142485051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccc015f3c0 0x7fccc015c7d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fccc00b5260 tx=0x7fccb001c480 comp rx=0 tx=0).stop 2024-04-25T00:31:45.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 -- 172.21.15.29:0/4142485051 shutdown_connections 2024-04-25T00:31:45.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 --2- 172.21.15.29:0/4142485051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccc015f3c0 0x7fccc015c7d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 --2- 172.21.15.29:0/4142485051 >> v2:172.21.15.157:3300/0 conn(0x7fccc015e8d0 0x7fccc00cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 -- 172.21.15.29:0/4142485051 >> 172.21.15.29:0/4142485051 conn(0x7fccc00b1870 msgr2=0x7fccc00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:45.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 -- 172.21.15.29:0/4142485051 shutdown_connections 2024-04-25T00:31:45.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 -- 172.21.15.29:0/4142485051 wait complete. 2024-04-25T00:31:45.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 Processor -- start 2024-04-25T00:31:45.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 -- start start 2024-04-25T00:31:45.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccc00b4780 0x7fccc013b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccc015e8d0 0x7fccc013e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc015f3c0 0x7fccc013bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fccc015d6a0 con 0x7fccc015e8d0 2024-04-25T00:31:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccc015d520 con 0x7fccc00b4780 2024-04-25T00:31:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccc4be7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fccc0167c70 con 0x7fccc015f3c0 2024-04-25T00:31:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccbdd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccc015e8d0 0x7fccc013e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccbdd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccc015e8d0 0x7fccc013e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50264/0 (socket says 172.21.15.29:50264) 2024-04-25T00:31:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccbdd9b640 1 -- 172.21.15.29:0/1297340135 learned_addr learned my addr 172.21.15.29:0/1297340135 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccbe59c640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccc00b4780 0x7fccc013b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccbed9d640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc015f3c0 0x7fccc013bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccbdd9b640 1 -- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc015f3c0 msgr2=0x7fccc013bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccbdd9b640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc015f3c0 0x7fccc013bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccbdd9b640 1 -- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccc00b4780 msgr2=0x7fccc013b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccbdd9b640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccc00b4780 0x7fccc013b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccbdd9b640 1 -- 172.21.15.29:0/1297340135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc013e9c0 con 0x7fccc015e8d0 2024-04-25T00:31:45.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.528+0000 7fccbdd9b640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccc015e8d0 0x7fccc013e270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fccb40011f0 tx=0x7fccb400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:45.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.532+0000 7fccaeffd640 1 -- 172.21.15.29:0/1297340135 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4018070 con 0x7fccc015e8d0 2024-04-25T00:31:45.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.532+0000 7fccaeffd640 1 -- 172.21.15.29:0/1297340135 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb40074e0 con 0x7fccc015e8d0 2024-04-25T00:31:45.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.532+0000 7fccc4be7640 1 -- 172.21.15.29:0/1297340135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc013eca0 con 0x7fccc015e8d0 2024-04-25T00:31:45.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.532+0000 7fccaeffd640 1 -- 172.21.15.29:0/1297340135 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4004a10 con 0x7fccc015e8d0 2024-04-25T00:31:45.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.532+0000 7fccc4be7640 1 -- 172.21.15.29:0/1297340135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc013f260 con 0x7fccc015e8d0 2024-04-25T00:31:45.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.532+0000 7fccc4be7640 1 -- 172.21.15.29:0/1297340135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc0069270 con 0x7fccc015e8d0 2024-04-25T00:31:45.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.532+0000 7fccaeffd640 1 -- 172.21.15.29:0/1297340135 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fccb401b070 con 0x7fccc015e8d0 2024-04-25T00:31:45.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.532+0000 7fccaeffd640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcca00678f0 0x7fcca0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.532+0000 7fccaeffd640 1 -- 172.21.15.29:0/1297340135 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fccb4085060 con 0x7fccc015e8d0 2024-04-25T00:31:45.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.532+0000 7fccbe59c640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcca00678f0 0x7fcca0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.532+0000 7fccbe59c640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcca00678f0 0x7fcca0069db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009a80 tx=0x7fccb8003a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:45.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.536+0000 7fccaeffd640 1 -- 172.21.15.29:0/1297340135 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fccb4055b10 con 0x7fccc015e8d0 2024-04-25T00:31:45.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.696+0000 7fccc4be7640 1 -- 172.21.15.29:0/1297340135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fccc006e7b0 con 0x7fccc015e8d0 2024-04-25T00:31:45.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.696+0000 7fccaeffd640 1 -- 172.21.15.29:0/1297340135 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fccb4055410 con 0x7fccc015e8d0 2024-04-25T00:31:45.699 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:31:45.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.700+0000 7fccc4be7640 1 -- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcca00678f0 msgr2=0x7fcca0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.700+0000 7fccc4be7640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcca00678f0 0x7fcca0069db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009a80 tx=0x7fccb8003a00 comp rx=0 tx=0).stop 2024-04-25T00:31:45.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.700+0000 7fccc4be7640 1 -- 172.21.15.29:0/1297340135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccc015e8d0 msgr2=0x7fccc013e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.700+0000 7fccc4be7640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccc015e8d0 0x7fccc013e270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fccb40011f0 tx=0x7fccb400c9b0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.700+0000 7fccc4be7640 1 -- 172.21.15.29:0/1297340135 shutdown_connections 2024-04-25T00:31:45.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.700+0000 7fccc4be7640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccc015f3c0 0x7fccc013bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.700+0000 7fccc4be7640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccc015e8d0 0x7fccc013e270 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.700+0000 7fccc4be7640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcca00678f0 0x7fcca0069db0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.700+0000 7fccc4be7640 1 --2- 172.21.15.29:0/1297340135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccc00b4780 0x7fccc013b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.700+0000 7fccc4be7640 1 -- 172.21.15.29:0/1297340135 >> 172.21.15.29:0/1297340135 conn(0x7fccc00b1870 msgr2=0x7fccc006de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:45.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.700+0000 7fccc4be7640 1 -- 172.21.15.29:0/1297340135 shutdown_connections 2024-04-25T00:31:45.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.700+0000 7fccc4be7640 1 -- 172.21.15.29:0/1297340135 wait complete. 2024-04-25T00:31:45.719 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:31:45.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f103462c640 1 Processor -- start 2024-04-25T00:31:45.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f103462c640 1 -- start start 2024-04-25T00:31:45.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f103462c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f102c161700 0x7f102c0b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f103462c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f102c0b4d30 0x7f102c0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f103462c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f102c161ae0 con 0x7f102c0ceff0 2024-04-25T00:31:45.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f103462c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f102c0cf520 con 0x7f102c0b4d30 2024-04-25T00:31:45.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f103462c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f102c0cf800 con 0x7f102c161700 2024-04-25T00:31:45.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f1031bc7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f102c0b4d30 0x7f102c0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f1031bc7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f102c0b4d30 0x7f102c0ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50274/0 (socket says 172.21.15.29:50274) 2024-04-25T00:31:45.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f1031bc7640 1 -- 172.21.15.29:0/1692339598 learned_addr learned my addr 172.21.15.29:0/1692339598 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:45.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f10323c8640 1 --2- 172.21.15.29:0/1692339598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f102c161700 0x7f102c0b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f1031bc7640 1 -- 172.21.15.29:0/1692339598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f102c161700 msgr2=0x7f102c0b6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:45.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f1031bc7640 1 --2- 172.21.15.29:0/1692339598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f102c161700 0x7f102c0b6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f1031bc7640 1 -- 172.21.15.29:0/1692339598 >> v1:172.21.15.157:6789/0 conn(0x7f102c0ceff0 legacy=0x7f102c0d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:45.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f1031bc7640 1 -- 172.21.15.29:0/1692339598 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c0cfae0 con 0x7f102c0b4d30 2024-04-25T00:31:45.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f10323c8640 1 --2- 172.21.15.29:0/1692339598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f102c161700 0x7f102c0b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:45.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.844+0000 7f1031bc7640 1 --2- 172.21.15.29:0/1692339598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f102c0b4d30 0x7f102c0ceab0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f101c009870 tx=0x7f101c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87575abd52b14800 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:45.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f10313c6640 1 -- 172.21.15.29:0/1692339598 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c0aa610 con 0x7f102c0b4d30 2024-04-25T00:31:45.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f10313c6640 1 -- 172.21.15.29:0/1692339598 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101c0af070 con 0x7f102c0b4d30 2024-04-25T00:31:45.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f10313c6640 1 -- 172.21.15.29:0/1692339598 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c0aa9d0 con 0x7f102c0b4d30 2024-04-25T00:31:45.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 -- 172.21.15.29:0/1692339598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f102c0b4d30 msgr2=0x7f102c0ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 --2- 172.21.15.29:0/1692339598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f102c0b4d30 0x7f102c0ceab0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f101c009870 tx=0x7f101c01c4f0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 -- 172.21.15.29:0/1692339598 shutdown_connections 2024-04-25T00:31:45.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 --2- 172.21.15.29:0/1692339598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f102c0b4d30 0x7f102c0ceab0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 --2- 172.21.15.29:0/1692339598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f102c161700 0x7f102c0b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 -- 172.21.15.29:0/1692339598 >> 172.21.15.29:0/1692339598 conn(0x7f102c09d280 msgr2=0x7f102c0d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:45.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 -- 172.21.15.29:0/1692339598 shutdown_connections 2024-04-25T00:31:45.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 -- 172.21.15.29:0/1692339598 wait complete. 2024-04-25T00:31:45.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 Processor -- start 2024-04-25T00:31:45.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 -- start start 2024-04-25T00:31:45.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f102c0b4d30 0x7f102c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f102c0ceff0 0x7f102c150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f102c161700 0x7f102c14fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f102c0d28b0 con 0x7f102c0ceff0 2024-04-25T00:31:45.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f102c0d1280 con 0x7f102c0b4d30 2024-04-25T00:31:45.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f102c0d2a30 con 0x7f102c161700 2024-04-25T00:31:45.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f1031bc7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f102c0ceff0 0x7f102c150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f1031bc7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f102c0ceff0 0x7f102c150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50278/0 (socket says 172.21.15.29:50278) 2024-04-25T00:31:45.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f1031bc7640 1 -- 172.21.15.29:0/4028943855 learned_addr learned my addr 172.21.15.29:0/4028943855 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:45.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f10323c8640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f102c0b4d30 0x7f102c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f1032bc9640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f102c161700 0x7f102c14fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f1031bc7640 1 -- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f102c161700 msgr2=0x7f102c14fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f1031bc7640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f102c161700 0x7f102c14fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f1031bc7640 1 -- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f102c0b4d30 msgr2=0x7f102c0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:45.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f1031bc7640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f102c0b4d30 0x7f102c0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:45.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f1031bc7640 1 -- 172.21.15.29:0/4028943855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c15a120 con 0x7f102c0ceff0 2024-04-25T00:31:45.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f1031bc7640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f102c0ceff0 0x7f102c150280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f101c0ac040 tx=0x7f101c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:45.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f1022ffd640 1 -- 172.21.15.29:0/4028943855 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c0b3800 con 0x7f102c0ceff0 2024-04-25T00:31:45.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f1022ffd640 1 -- 172.21.15.29:0/4028943855 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101c0b39c0 con 0x7f102c0ceff0 2024-04-25T00:31:45.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 -- 172.21.15.29:0/4028943855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102c150090 con 0x7f102c0ceff0 2024-04-25T00:31:45.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f1022ffd640 1 -- 172.21.15.29:0/4028943855 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c0b3db0 con 0x7f102c0ceff0 2024-04-25T00:31:45.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.848+0000 7f103462c640 1 -- 172.21.15.29:0/4028943855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102c150c60 con 0x7f102c0ceff0 2024-04-25T00:31:45.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.852+0000 7f103462c640 1 -- 172.21.15.29:0/4028943855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102c069270 con 0x7f102c0ceff0 2024-04-25T00:31:45.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.852+0000 7f1022ffd640 1 -- 172.21.15.29:0/4028943855 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f101c0b2380 con 0x7f102c0ceff0 2024-04-25T00:31:45.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.852+0000 7f1022ffd640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f10100678c0 0x7f1010069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:45.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.852+0000 7f1022ffd640 1 -- 172.21.15.29:0/4028943855 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f101c120280 con 0x7f102c0ceff0 2024-04-25T00:31:45.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.852+0000 7f10323c8640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f10100678c0 0x7f1010069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:45.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.852+0000 7f10323c8640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f10100678c0 0x7f1010069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1028006fd0 tx=0x7f1028008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:45.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:45.856+0000 7f1022ffd640 1 -- 172.21.15.29:0/4028943855 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f101c0c3060 con 0x7f102c0ceff0 2024-04-25T00:31:46.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.052+0000 7f103462c640 1 -- 172.21.15.29:0/4028943855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f102c0cc920 con 0x7f102c0ceff0 2024-04-25T00:31:46.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.052+0000 7f1022ffd640 1 -- 172.21.15.29:0/4028943855 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f101c0a7070 con 0x7f102c0ceff0 2024-04-25T00:31:46.057 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:46.057 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1532858457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532858457},{"type":"v1","addr":"172.21.15.29:6833","nonce":1532858457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/718997473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":718997473},{"type":"v1","addr":"172.21.15.157:6835","nonce":718997473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1617208213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1617208213},{"type":"v1","addr":"172.21.15.157:6837","nonce":1617208213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:31:29.640568+0000","modified":"2024-04-25T00:31:31.633585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4252":{"gid":4252,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1924706890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1924706890},{"type":"v1","addr":"172.21.15.29:6835","nonce":1924706890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4252,"qdb_cluster":[4252]},"id":1}]} 2024-04-25T00:31:46.057 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 6 2024-04-25T00:31:46.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.056+0000 7f103462c640 1 -- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f10100678c0 msgr2=0x7f1010069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.056+0000 7f103462c640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f10100678c0 0x7f1010069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1028006fd0 tx=0x7f1028008040 comp rx=0 tx=0).stop 2024-04-25T00:31:46.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.056+0000 7f103462c640 1 -- 172.21.15.29:0/4028943855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f102c0ceff0 msgr2=0x7f102c150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.056+0000 7f103462c640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f102c0ceff0 0x7f102c150280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f101c0ac040 tx=0x7f101c01ad30 comp rx=0 tx=0).stop 2024-04-25T00:31:46.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.056+0000 7f103462c640 1 -- 172.21.15.29:0/4028943855 shutdown_connections 2024-04-25T00:31:46.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.056+0000 7f103462c640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f102c161700 0x7f102c14fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.056+0000 7f103462c640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f102c0ceff0 0x7f102c150280 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.056+0000 7f103462c640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f10100678c0 0x7f1010069d80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.056+0000 7f103462c640 1 --2- 172.21.15.29:0/4028943855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f102c0b4d30 0x7f102c0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.056+0000 7f103462c640 1 -- 172.21.15.29:0/4028943855 >> 172.21.15.29:0/4028943855 conn(0x7f102c09d280 msgr2=0x7f102c15edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:46.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.056+0000 7f103462c640 1 -- 172.21.15.29:0/4028943855 shutdown_connections 2024-04-25T00:31:46.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.056+0000 7f103462c640 1 -- 172.21.15.29:0/4028943855 wait complete. 2024-04-25T00:31:46.075 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:46.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.184+0000 7f60a84d7640 1 Processor -- start 2024-04-25T00:31:46.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.184+0000 7f60a84d7640 1 -- start start 2024-04-25T00:31:46.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.184+0000 7f60a84d7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f60a00b3c30 0x7f60a006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.184+0000 7f60a84d7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60a00b4b10 0x7f60a00ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.184+0000 7f60a84d7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a00b41e0 con 0x7f60a00b3140 2024-04-25T00:31:46.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.184+0000 7f60a84d7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f60a006d960 con 0x7f60a00b3c30 2024-04-25T00:31:46.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.184+0000 7f60a84d7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f60a006dc40 con 0x7f60a00b4b10 2024-04-25T00:31:46.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.184+0000 7f60a5a72640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f60a00b3c30 0x7f60a006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.184+0000 7f60a5a72640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f60a00b3c30 0x7f60a006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:32994/0 (socket says 172.21.15.29:32994) 2024-04-25T00:31:46.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.184+0000 7f60a5a72640 1 -- 172.21.15.29:0/1028060309 learned_addr learned my addr 172.21.15.29:0/1028060309 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:46.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.184+0000 7f60a6a74640 1 --2- 172.21.15.29:0/1028060309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60a00b4b10 0x7f60a00ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a5a72640 1 -- 172.21.15.29:0/1028060309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60a00b4b10 msgr2=0x7f60a00ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:46.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a5a72640 1 --2- 172.21.15.29:0/1028060309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60a00b4b10 0x7f60a00ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a5a72640 1 -- 172.21.15.29:0/1028060309 >> v1:172.21.15.157:6789/0 conn(0x7f60a00b3140 legacy=0x7f60a00b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a5a72640 1 -- 172.21.15.29:0/1028060309 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a006df20 con 0x7f60a00b3c30 2024-04-25T00:31:46.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a5a72640 1 --2- 172.21.15.29:0/1028060309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a00b3c30 0x7f60a006e440 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f60940098a0 tx=0x7f609401c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=318573231a7ee736 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:46.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a5271640 1 -- 172.21.15.29:0/1028060309 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940b0070 con 0x7f60a00b3c30 2024-04-25T00:31:46.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a5271640 1 -- 172.21.15.29:0/1028060309 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60940ab600 con 0x7f60a00b3c30 2024-04-25T00:31:46.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a5271640 1 -- 172.21.15.29:0/1028060309 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940ab9d0 con 0x7f60a00b3c30 2024-04-25T00:31:46.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a84d7640 1 -- 172.21.15.29:0/1028060309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a00b3c30 msgr2=0x7f60a006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a84d7640 1 --2- 172.21.15.29:0/1028060309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a00b3c30 0x7f60a006e440 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f60940098a0 tx=0x7f609401c540 comp rx=0 tx=0).stop 2024-04-25T00:31:46.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a84d7640 1 -- 172.21.15.29:0/1028060309 shutdown_connections 2024-04-25T00:31:46.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a84d7640 1 --2- 172.21.15.29:0/1028060309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60a00b4b10 0x7f60a00ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a84d7640 1 --2- 172.21.15.29:0/1028060309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a00b3c30 0x7f60a006e440 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a84d7640 1 -- 172.21.15.29:0/1028060309 >> 172.21.15.29:0/1028060309 conn(0x7f60a00b1870 msgr2=0x7f60a00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:46.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a84d7640 1 -- 172.21.15.29:0/1028060309 shutdown_connections 2024-04-25T00:31:46.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.188+0000 7f60a84d7640 1 -- 172.21.15.29:0/1028060309 wait complete. 2024-04-25T00:31:46.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a84d7640 1 Processor -- start 2024-04-25T00:31:46.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a84d7640 1 -- start start 2024-04-25T00:31:46.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a84d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a00b3140 0x7f60a0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a84d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60a00b3c30 0x7f60a0142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a84d7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60a00b4b10 0x7f60a0156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a84d7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0168d80 con 0x7f60a00b3c30 2024-04-25T00:31:46.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a84d7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0168c00 con 0x7f60a00b3140 2024-04-25T00:31:46.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a84d7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f60a0168f00 con 0x7f60a00b4b10 2024-04-25T00:31:46.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a5a72640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60a00b3c30 0x7f60a0142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a5a72640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60a00b3c30 0x7f60a0142cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50280/0 (socket says 172.21.15.29:50280) 2024-04-25T00:31:46.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a5a72640 1 -- 172.21.15.29:0/1516578453 learned_addr learned my addr 172.21.15.29:0/1516578453 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:46.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a6a74640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60a00b4b10 0x7f60a0156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a6273640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a00b3140 0x7f60a0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a5a72640 1 -- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60a00b4b10 msgr2=0x7f60a0156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a5a72640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60a00b4b10 0x7f60a0156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a5a72640 1 -- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a00b3140 msgr2=0x7f60a0157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a5a72640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a00b3140 0x7f60a0157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a5a72640 1 -- 172.21.15.29:0/1516578453 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a01581d0 con 0x7f60a00b3c30 2024-04-25T00:31:46.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a6273640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a00b3140 0x7f60a0157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:46.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a5a72640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60a00b3c30 0x7f60a0142cd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f609401abf0 tx=0x7f60940192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:46.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f6092ffd640 1 -- 172.21.15.29:0/1516578453 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940b9070 con 0x7f60a00b3c30 2024-04-25T00:31:46.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f6092ffd640 1 -- 172.21.15.29:0/1516578453 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60940b4420 con 0x7f60a00b3c30 2024-04-25T00:31:46.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a84d7640 1 -- 172.21.15.29:0/1516578453 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a0157c10 con 0x7f60a00b3c30 2024-04-25T00:31:46.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f6092ffd640 1 -- 172.21.15.29:0/1516578453 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940b0040 con 0x7f60a00b3c30 2024-04-25T00:31:46.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a84d7640 1 -- 172.21.15.29:0/1516578453 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a01562a0 con 0x7f60a00b3c30 2024-04-25T00:31:46.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f60a84d7640 1 -- 172.21.15.29:0/1516578453 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a0069270 con 0x7f60a00b3c30 2024-04-25T00:31:46.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.192+0000 7f6092ffd640 1 -- 172.21.15.29:0/1516578453 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f60940bc070 con 0x7f60a00b3c30 2024-04-25T00:31:46.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.196+0000 7f6092ffd640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f60840678c0 0x7f6084069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.196+0000 7f6092ffd640 1 -- 172.21.15.29:0/1516578453 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f609411ebc0 con 0x7f60a00b3c30 2024-04-25T00:31:46.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.196+0000 7f60a6273640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f60840678c0 0x7f6084069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.196+0000 7f60a6273640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f60840678c0 0x7f6084069d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f609c001650 tx=0x7f609c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:46.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.200+0000 7f6092ffd640 1 -- 172.21.15.29:0/1516578453 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6094016130 con 0x7f60a00b3c30 2024-04-25T00:31:46.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.364+0000 7f60a84d7640 1 -- 172.21.15.29:0/1516578453 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60a0140450 con 0x7f60a00b3c30 2024-04-25T00:31:46.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.364+0000 7f6092ffd640 1 -- 172.21.15.29:0/1516578453 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f60940ef010 con 0x7f60a00b3c30 2024-04-25T00:31:46.368 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:46.368 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:46.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.368+0000 7f60a84d7640 1 -- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f60840678c0 msgr2=0x7f6084069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.368+0000 7f60a84d7640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f60840678c0 0x7f6084069d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f609c001650 tx=0x7f609c009040 comp rx=0 tx=0).stop 2024-04-25T00:31:46.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.368+0000 7f60a84d7640 1 -- 172.21.15.29:0/1516578453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60a00b3c30 msgr2=0x7f60a0142cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.368+0000 7f60a84d7640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60a00b3c30 0x7f60a0142cd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f609401abf0 tx=0x7f60940192e0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.368+0000 7f60a84d7640 1 -- 172.21.15.29:0/1516578453 shutdown_connections 2024-04-25T00:31:46.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.368+0000 7f60a84d7640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60a00b4b10 0x7f60a0156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.368+0000 7f60a84d7640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60a00b3c30 0x7f60a0142cd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.368+0000 7f60a84d7640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f60840678c0 0x7f6084069d80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.368+0000 7f60a84d7640 1 --2- 172.21.15.29:0/1516578453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a00b3140 0x7f60a0157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.368+0000 7f60a84d7640 1 -- 172.21.15.29:0/1516578453 >> 172.21.15.29:0/1516578453 conn(0x7f60a00b1870 msgr2=0x7f60a00b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:46.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.368+0000 7f60a84d7640 1 -- 172.21.15.29:0/1516578453 shutdown_connections 2024-04-25T00:31:46.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.368+0000 7f60a84d7640 1 -- 172.21.15.29:0/1516578453 wait complete. 2024-04-25T00:31:46.387 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:31:46.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.508+0000 7fb7bbc48640 1 Processor -- start 2024-04-25T00:31:46.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.508+0000 7fb7bbc48640 1 -- start start 2024-04-25T00:31:46.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7bbc48640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7b415bff0 0x7fb7b40cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7bbc48640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb7b40d0ac0 0x7fb7b40b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7bbc48640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7b40cdaf0 con 0x7fb7b40d0660 2024-04-25T00:31:46.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7bbc48640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b40cf930 con 0x7fb7b40d0ac0 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7bbc48640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b40cd890 con 0x7fb7b415bff0 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b91e3640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb7b40d0660 0x7fb7b40b4bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50136/0 (socket says 172.21.15.29:50136) 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b91e3640 1 -- 172.21.15.29:0/1189123986 learned_addr learned my addr 172.21.15.29:0/1189123986 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b99e4640 1 --2- 172.21.15.29:0/1189123986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7b415bff0 0x7fb7b40cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3523162622 0 0) 0x7fb7b40cdaf0 con 0x7fb7b40d0660 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7ba1e5640 1 --2- 172.21.15.29:0/1189123986 >> v2:172.21.15.157:3300/0 conn(0x7fb7b40d0ac0 0x7fb7b40b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7a0003530 con 0x7fb7b40d0660 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 526472009 0 0) 0x7fb7a0003530 con 0x7fb7b40d0660 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7b415bff0 msgr2=0x7fb7b40cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 --2- 172.21.15.29:0/1189123986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7b415bff0 0x7fb7b40cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 >> v2:172.21.15.157:3300/0 conn(0x7fb7b40d0ac0 msgr2=0x7fb7b40b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 --2- 172.21.15.29:0/1189123986 >> v2:172.21.15.157:3300/0 conn(0x7fb7b40d0ac0 0x7fb7b40b52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b40cfb80 con 0x7fb7b40d0660 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fb7a4001680 con 0x7fb7b40d0660 2024-04-25T00:31:46.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 >> v1:172.21.15.29:6789/0 conn(0x7fb7b40d0660 legacy=0x7fb7b40b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 --2- 172.21.15.29:0/1189123986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7b415bff0 0x7fb7a00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 --2- 172.21.15.29:0/1189123986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7a0003f40 0x7fb7a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 --2- 172.21.15.29:0/1189123986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7a0004d50 0x7fb7a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b40cf930 con 0x7fb7a0003f40 2024-04-25T00:31:46.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b40cd890 con 0x7fb7a0004d50 2024-04-25T00:31:46.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb7a000b9d0 con 0x7fb7b415bff0 2024-04-25T00:31:46.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b91e3640 1 --2- 172.21.15.29:0/1189123986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7a0003f40 0x7fb7a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b99e4640 1 --2- 172.21.15.29:0/1189123986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7b415bff0 0x7fb7a00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b91e3640 1 -- 172.21.15.29:0/1189123986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7b415bff0 msgr2=0x7fb7a00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:46.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b91e3640 1 --2- 172.21.15.29:0/1189123986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7b415bff0 0x7fb7a00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b91e3640 1 -- 172.21.15.29:0/1189123986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7a0004d50 msgr2=0x7fb7a00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:46.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b91e3640 1 --2- 172.21.15.29:0/1189123986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7a0004d50 0x7fb7a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.512+0000 7fb7b91e3640 1 -- 172.21.15.29:0/1189123986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a0003530 con 0x7fb7a0003f40 2024-04-25T00:31:46.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7b91e3640 1 --2- 172.21.15.29:0/1189123986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7a0003f40 0x7fb7a0004810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4001060 tx=0x7fb7a401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e6fa8a34fbbc20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:46.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4007d80 con 0x7fb7a0003f40 2024-04-25T00:31:46.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a4005b00 con 0x7fb7a0003f40 2024-04-25T00:31:46.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7b89e2640 1 -- 172.21.15.29:0/1189123986 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4007070 con 0x7fb7a0003f40 2024-04-25T00:31:46.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 -- 172.21.15.29:0/1189123986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7a0003f40 msgr2=0x7fb7a0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 --2- 172.21.15.29:0/1189123986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7a0003f40 0x7fb7a0004810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4001060 tx=0x7fb7a401f3b0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7ba1e5640 1 -- 172.21.15.29:0/1189123986 reap_dead start 2024-04-25T00:31:46.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 -- 172.21.15.29:0/1189123986 shutdown_connections 2024-04-25T00:31:46.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 -- 172.21.15.29:0/1189123986 >> 172.21.15.29:0/1189123986 conn(0x7fb7b40b38b0 msgr2=0x7fb7b40b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:46.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 -- 172.21.15.29:0/1189123986 shutdown_connections 2024-04-25T00:31:46.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 -- 172.21.15.29:0/1189123986 wait complete. 2024-04-25T00:31:46.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 Processor -- start 2024-04-25T00:31:46.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 -- start start 2024-04-25T00:31:46.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7a0003f40 0x7fb7b4157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b41f4e10 0x7fb7b40b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b41581e0 0x7fb7b40b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b415c3d0 con 0x7fb7b41f4e10 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b40d0140 con 0x7fb7b41581e0 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7bbc48640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b40cd350 con 0x7fb7a0003f40 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7b91e3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b41f4e10 0x7fb7b40b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7b91e3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b41f4e10 0x7fb7b40b6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50288/0 (socket says 172.21.15.29:50288) 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7b91e3640 1 -- 172.21.15.29:0/2657065085 learned_addr learned my addr 172.21.15.29:0/2657065085 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7b99e4640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7a0003f40 0x7fb7b4157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7ba1e5640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b41581e0 0x7fb7b40b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7b91e3640 1 -- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7a0003f40 msgr2=0x7fb7b4157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7b91e3640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7a0003f40 0x7fb7b4157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7b91e3640 1 -- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b41581e0 msgr2=0x7fb7b40b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7b91e3640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b41581e0 0x7fb7b40b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.516+0000 7fb7b91e3640 1 -- 172.21.15.29:0/2657065085 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b4158760 con 0x7fb7b41f4e10 2024-04-25T00:31:46.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.520+0000 7fb7b91e3640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b41f4e10 0x7fb7b40b6ec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb7a401fc50 tx=0x7fb7a0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:46.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.520+0000 7fb7aa7fc640 1 -- 172.21.15.29:0/2657065085 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4005d70 con 0x7fb7b41f4e10 2024-04-25T00:31:46.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.520+0000 7fb7aa7fc640 1 -- 172.21.15.29:0/2657065085 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a40047f0 con 0x7fb7b41f4e10 2024-04-25T00:31:46.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.520+0000 7fb7bbc48640 1 -- 172.21.15.29:0/2657065085 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b4158a40 con 0x7fb7b41f4e10 2024-04-25T00:31:46.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.520+0000 7fb7aa7fc640 1 -- 172.21.15.29:0/2657065085 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4004be0 con 0x7fb7b41f4e10 2024-04-25T00:31:46.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.520+0000 7fb7bbc48640 1 -- 172.21.15.29:0/2657065085 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b4159000 con 0x7fb7b41f4e10 2024-04-25T00:31:46.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.520+0000 7fb7bbc48640 1 -- 172.21.15.29:0/2657065085 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b4069270 con 0x7fb7b41f4e10 2024-04-25T00:31:46.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.520+0000 7fb7aa7fc640 1 -- 172.21.15.29:0/2657065085 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb7a40ba030 con 0x7fb7b41f4e10 2024-04-25T00:31:46.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.520+0000 7fb7aa7fc640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7980678c0 0x7fb798069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:46.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.520+0000 7fb7aa7fc640 1 -- 172.21.15.29:0/2657065085 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb7a41228b0 con 0x7fb7b41f4e10 2024-04-25T00:31:46.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.520+0000 7fb7b99e4640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7980678c0 0x7fb798069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:46.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.520+0000 7fb7b99e4640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7980678c0 0x7fb798069d80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0007440 tx=0x7fb7b000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:46.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.524+0000 7fb7aa7fc640 1 -- 172.21.15.29:0/2657065085 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb7a4019120 con 0x7fb7b41f4e10 2024-04-25T00:31:46.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.720+0000 7fb7bbc48640 1 -- 172.21.15.29:0/2657065085 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7b4070ae0 con 0x7fb7b41f4e10 2024-04-25T00:31:46.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.724+0000 7fb7aa7fc640 1 -- 172.21.15.29:0/2657065085 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7fb7a40f34e0 con 0x7fb7b41f4e10 2024-04-25T00:31:46.725 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:46.726 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1532858457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532858457},{"type":"v1","addr":"172.21.15.29:6833","nonce":1532858457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/718997473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":718997473},{"type":"v1","addr":"172.21.15.157:6835","nonce":718997473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1617208213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1617208213},{"type":"v1","addr":"172.21.15.157:6837","nonce":1617208213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:31:29.640568+0000","modified":"2024-04-25T00:31:31.633585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4252":{"gid":4252,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1924706890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1924706890},{"type":"v1","addr":"172.21.15.29:6835","nonce":1924706890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4252,"qdb_cluster":[4252]},"id":1}]} 2024-04-25T00:31:46.726 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 6 2024-04-25T00:31:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.724+0000 7fb7bbc48640 1 -- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7980678c0 msgr2=0x7fb798069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.724+0000 7fb7bbc48640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7980678c0 0x7fb798069d80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0007440 tx=0x7fb7b000b040 comp rx=0 tx=0).stop 2024-04-25T00:31:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.724+0000 7fb7bbc48640 1 -- 172.21.15.29:0/2657065085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b41f4e10 msgr2=0x7fb7b40b6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.724+0000 7fb7bbc48640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b41f4e10 0x7fb7b40b6ec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb7a401fc50 tx=0x7fb7a0005840 comp rx=0 tx=0).stop 2024-04-25T00:31:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.724+0000 7fb7bbc48640 1 -- 172.21.15.29:0/2657065085 shutdown_connections 2024-04-25T00:31:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.724+0000 7fb7bbc48640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7980678c0 0x7fb798069d80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.724+0000 7fb7bbc48640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b41581e0 0x7fb7b40b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.724+0000 7fb7bbc48640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b41f4e10 0x7fb7b40b6ec0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.724+0000 7fb7bbc48640 1 --2- 172.21.15.29:0/2657065085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7a0003f40 0x7fb7b4157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.724+0000 7fb7bbc48640 1 -- 172.21.15.29:0/2657065085 >> 172.21.15.29:0/2657065085 conn(0x7fb7b40b38b0 msgr2=0x7fb7b40b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.724+0000 7fb7bbc48640 1 -- 172.21.15.29:0/2657065085 shutdown_connections 2024-04-25T00:31:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:46.724+0000 7fb7bbc48640 1 -- 172.21.15.29:0/2657065085 wait complete. 2024-04-25T00:31:47.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:31:29.640568+0000', 'modified': '2024-04-25T00:31:31.633585+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4252}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4252': {'gid': 4252, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/1924706890', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 1924706890}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 1924706890}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4252, 'qdb_cluster': [4252]} 2024-04-25T00:31:47.148 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:31:47.148 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:31:47.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.280+0000 7f5fea3d2640 1 Processor -- start 2024-04-25T00:31:47.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.280+0000 7f5fea3d2640 1 -- start start 2024-04-25T00:31:47.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.280+0000 7f5fea3d2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5fe40b4780 0x7f5fe415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.280+0000 7f5fea3d2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe40b3720 con 0x7f5fe415e8d0 2024-04-25T00:31:47.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.280+0000 7f5fea3d2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe40b39c0 con 0x7f5fe40cc240 2024-04-25T00:31:47.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.280+0000 7f5fea3d2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe40cc650 con 0x7f5fe40b4780 2024-04-25T00:31:47.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.280+0000 7f5fe3fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5fe415e8d0 0x7f5fe415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50142/0 (socket says 172.21.15.29:50142) 2024-04-25T00:31:47.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.280+0000 7f5fe3fff640 1 -- 172.21.15.29:0/2877406805 learned_addr learned my addr 172.21.15.29:0/2877406805 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:47.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.280+0000 7f5fe37fe640 1 --2- 172.21.15.29:0/2877406805 >> v2:172.21.15.157:3300/0 conn(0x7f5fe40b4780 0x7f5fe415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.280+0000 7f5fe2ffd640 1 -- 172.21.15.29:0/2877406805 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 517005643 0 0) 0x7f5fe40b3720 con 0x7f5fe415e8d0 2024-04-25T00:31:47.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.280+0000 7f5fe2ffd640 1 -- 172.21.15.29:0/2877406805 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fcc003530 con 0x7f5fe415e8d0 2024-04-25T00:31:47.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fe37fe640 1 -- 172.21.15.29:0/2877406805 >> v1:172.21.15.157:6789/0 conn(0x7f5fe40cc240 legacy=0x7f5fe415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fe37fe640 1 -- 172.21.15.29:0/2877406805 >> v1:172.21.15.29:6789/0 conn(0x7f5fe415e8d0 legacy=0x7f5fe415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fe37fe640 1 -- 172.21.15.29:0/2877406805 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe40b3ca0 con 0x7f5fe40b4780 2024-04-25T00:31:47.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fe37fe640 1 --2- 172.21.15.29:0/2877406805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe40b4780 0x7f5fe415f3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80097c0 tx=0x7f5fd801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7060069e9f26c0d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:47.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fe2ffd640 1 -- 172.21.15.29:0/2877406805 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80b0070 con 0x7f5fe40b4780 2024-04-25T00:31:47.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fe2ffd640 1 -- 172.21.15.29:0/2877406805 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd80ab600 con 0x7f5fe40b4780 2024-04-25T00:31:47.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fe2ffd640 1 -- 172.21.15.29:0/2877406805 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80ab9b0 con 0x7f5fe40b4780 2024-04-25T00:31:47.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fea3d2640 1 -- 172.21.15.29:0/2877406805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe40b4780 msgr2=0x7f5fe415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fea3d2640 1 --2- 172.21.15.29:0/2877406805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe40b4780 0x7f5fe415f3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80097c0 tx=0x7f5fd801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fea3d2640 1 -- 172.21.15.29:0/2877406805 shutdown_connections 2024-04-25T00:31:47.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fea3d2640 1 --2- 172.21.15.29:0/2877406805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe40b4780 0x7f5fe415f3c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fea3d2640 1 -- 172.21.15.29:0/2877406805 >> 172.21.15.29:0/2877406805 conn(0x7f5fe40b1870 msgr2=0x7f5fe40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:47.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fea3d2640 1 -- 172.21.15.29:0/2877406805 shutdown_connections 2024-04-25T00:31:47.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.284+0000 7f5fea3d2640 1 -- 172.21.15.29:0/2877406805 wait complete. 2024-04-25T00:31:47.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fea3d2640 1 Processor -- start 2024-04-25T00:31:47.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fea3d2640 1 -- start start 2024-04-25T00:31:47.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fea3d2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fe40b4780 0x7f5fe4070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fea3d2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fe40cc240 0x7f5fe406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fea3d2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe415e8d0 0x7f5fe41405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fea3d2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4168e00 con 0x7f5fe40b4780 2024-04-25T00:31:47.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fea3d2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4168c80 con 0x7f5fe415e8d0 2024-04-25T00:31:47.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fea3d2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe4168f80 con 0x7f5fe40cc240 2024-04-25T00:31:47.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fe40b4780 0x7f5fe4070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fe40b4780 0x7f5fe4070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50302/0 (socket says 172.21.15.29:50302) 2024-04-25T00:31:47.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe3fff640 1 -- 172.21.15.29:0/3102199501 learned_addr learned my addr 172.21.15.29:0/3102199501 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:47.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe896f640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe415e8d0 0x7f5fe41405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe37fe640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fe40cc240 0x7f5fe406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe3fff640 1 -- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fe40cc240 msgr2=0x7f5fe406d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fe40cc240 0x7f5fe406d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe3fff640 1 -- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe415e8d0 msgr2=0x7f5fe41405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe415e8d0 0x7f5fe41405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe3fff640 1 -- 172.21.15.29:0/3102199501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe4142430 con 0x7f5fe40b4780 2024-04-25T00:31:47.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fe40b4780 0x7f5fe4070ce0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400c420 tx=0x7f5fd400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:47.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe0ff9640 1 -- 172.21.15.29:0/3102199501 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4017070 con 0x7f5fe40b4780 2024-04-25T00:31:47.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe0ff9640 1 -- 172.21.15.29:0/3102199501 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd4005cf0 con 0x7f5fe40b4780 2024-04-25T00:31:47.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fea3d2640 1 -- 172.21.15.29:0/3102199501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe41426d0 con 0x7f5fe40b4780 2024-04-25T00:31:47.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fe0ff9640 1 -- 172.21.15.29:0/3102199501 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40072d0 con 0x7f5fe40b4780 2024-04-25T00:31:47.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.288+0000 7f5fea3d2640 1 -- 172.21.15.29:0/3102199501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe4142f70 con 0x7f5fe40b4780 2024-04-25T00:31:47.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe0ff9640 1 -- 172.21.15.29:0/3102199501 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5fd401a070 con 0x7f5fe40b4780 2024-04-25T00:31:47.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe0ff9640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5fd00678f0 0x7f5fd0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe0ff9640 1 -- 172.21.15.29:0/3102199501 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5fd4084ef0 con 0x7f5fe40b4780 2024-04-25T00:31:47.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe37fe640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5fd00678f0 0x7f5fd0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 Processor -- start 2024-04-25T00:31:47.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 -- start start 2024-04-25T00:31:47.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe37fe640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5fd00678f0 0x7f5fd0069db0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80ad000 tx=0x7f5fd80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:47.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 0x55c112ad92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad9810 0x55c112a8f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 0x55c112a8ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c1128dd5e0 con 0x55c112ad8ef0 2024-04-25T00:31:47.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c112864c50 con 0x55c112ad9810 2024-04-25T00:31:47.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55c1128d95a0 con 0x55c112a8fbc0 2024-04-25T00:31:47.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 0x55c112ad92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 0x55c112ad92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50306/0 (socket says 172.21.15.29:50306) 2024-04-25T00:31:47.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe3fff640 1 -- 172.21.15.29:0/3450311238 learned_addr learned my addr 172.21.15.29:0/3450311238 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:47.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe37fe640 1 --2- 172.21.15.29:0/3450311238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad9810 0x55c112a8f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe3fff640 1 -- 172.21.15.29:0/3450311238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 msgr2=0x55c112a8ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:47.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/3450311238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 0x55c112a8ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe3fff640 1 -- 172.21.15.29:0/3450311238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad9810 msgr2=0x55c112a8f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:47.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/3450311238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad9810 0x55c112a8f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe3fff640 1 -- 172.21.15.29:0/3450311238 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c112862d10 con 0x55c112ad8ef0 2024-04-25T00:31:47.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe37fe640 1 --2- 172.21.15.29:0/3450311238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad9810 0x55c112a8f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:47.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/3450311238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 0x55c112ad92d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4083450 tx=0x7f5fd4059770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72dea7bb2432eb20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:47.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fc9ffb640 1 -- 172.21.15.29:0/3450311238 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4053770 con 0x55c112ad8ef0 2024-04-25T00:31:47.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fc9ffb640 1 -- 172.21.15.29:0/3450311238 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd4051ae0 con 0x55c112ad8ef0 2024-04-25T00:31:47.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fc9ffb640 1 -- 172.21.15.29:0/3450311238 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd406d3d0 con 0x55c112ad8ef0 2024-04-25T00:31:47.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 -- 172.21.15.29:0/3450311238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 msgr2=0x55c112ad92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 --2- 172.21.15.29:0/3450311238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 0x55c112ad92d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4083450 tx=0x7f5fd4059770 comp rx=0 tx=0).stop 2024-04-25T00:31:47.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 -- 172.21.15.29:0/3450311238 shutdown_connections 2024-04-25T00:31:47.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 --2- 172.21.15.29:0/3450311238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 0x55c112a8ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 --2- 172.21.15.29:0/3450311238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad9810 0x55c112a8f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 --2- 172.21.15.29:0/3450311238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 0x55c112ad92d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 -- 172.21.15.29:0/3450311238 >> 172.21.15.29:0/3450311238 conn(0x55c112a8bb70 msgr2=0x55c112a8df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:47.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 -- 172.21.15.29:0/3450311238 shutdown_connections 2024-04-25T00:31:47.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.292+0000 7f5fed54b000 1 -- 172.21.15.29:0/3450311238 wait complete. 2024-04-25T00:31:47.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 -- start start 2024-04-25T00:31:47.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 Processor -- start 2024-04-25T00:31:47.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112a8fbc0 0x55c112abdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112ad8ef0 0x55c112abe460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 0x55c112abe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c112897520 con 0x55c112ad9810 2024-04-25T00:31:47.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c112883810 con 0x55c112a8fbc0 2024-04-25T00:31:47.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55c1128e1c10 con 0x55c112ad8ef0 2024-04-25T00:31:47.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 10 client.0 _ll_register_callbacks cb 0x55c112a7a5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:47.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe896f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 0x55c112abe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe896f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 0x55c112abe9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50312/0 (socket says 172.21.15.29:50312) 2024-04-25T00:31:47.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe896f640 1 -- 172.21.15.29:0/893110598 learned_addr learned my addr 172.21.15.29:0/893110598 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:47.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112a8fbc0 0x55c112abdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe37fe640 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112ad8ef0 0x55c112abe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe896f640 1 -- 172.21.15.29:0/893110598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112ad8ef0 msgr2=0x55c112abe460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:47.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe896f640 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112ad8ef0 0x55c112abe460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe896f640 1 -- 172.21.15.29:0/893110598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112a8fbc0 msgr2=0x55c112abdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:47.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe896f640 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112a8fbc0 0x55c112abdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe896f640 1 -- 172.21.15.29:0/893110598 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1128757b0 con 0x55c112ad9810 2024-04-25T00:31:47.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe37fe640 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112ad8ef0 0x55c112abe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:47.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe896f640 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 0x55c112abe9a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e4a0 tx=0x7f5fdc00e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d29ed8ae2ad2abf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:47.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:31:47.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fc2ffd640 1 -- 172.21.15.29:0/893110598 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc019060 con 0x55c112ad9810 2024-04-25T00:31:47.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fc2ffd640 1 -- 172.21.15.29:0/893110598 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5fdc003b90 con 0x55c112ad9810 2024-04-25T00:31:47.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 -- 172.21.15.29:0/893110598 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd4051ae0 con 0x55c112ad9810 2024-04-25T00:31:47.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fc2ffd640 1 -- 172.21.15.29:0/893110598 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f5fdc01f050 con 0x55c112ad9810 2024-04-25T00:31:47.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 10 client.4638 fetch_fsmap learned FSMap version 6 2024-04-25T00:31:47.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 -- 172.21.15.29:0/893110598 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55c112864ea0 con 0x55c112ad9810 2024-04-25T00:31:47.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fc2ffd640 1 -- 172.21.15.29:0/893110598 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd4051ae0 con 0x55c112ad9810 2024-04-25T00:31:47.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fc2ffd640 1 -- 172.21.15.29:0/893110598 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 6) ==== 3494+0+0 (secure 0 0 0) 0x7f5fdc022020 con 0x55c112ad9810 2024-04-25T00:31:47.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 10 client.4638 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T00:31:47.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 10 client.4638 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:31:47.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 20 client.4638 populate_metadata read hostname 'smithi029' 2024-04-25T00:31:47.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.29:6834/1924706890,v1:172.21.15.29:6835/1924706890] conn(0x55c112abf680 0x55c112aeafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 4 client.4638 mds_command: new command op to 4252 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:47.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 -- 172.21.15.29:0/893110598 --> [v2:172.21.15.29:6834/1924706890,v1:172.21.15.29:6835/1924706890] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c112878ab0 con 0x55c112abf680 2024-04-25T00:31:47.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.29:6834/1924706890,v1:172.21.15.29:6835/1924706890] conn(0x55c112abf680 0x55c112aeafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.29:6834/1924706890,v1:172.21.15.29:6835/1924706890] conn(0x55c112abf680 0x55c112aeafa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d04ebb8200b91fbd server_cookie=3750b1bbc8475062 in_seq=0 out_seq=0 2024-04-25T00:31:47.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fc2ffd640 10 client.4638 ms_handle_connect on v2:172.21.15.29:6834/1924706890 2024-04-25T00:31:47.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fc2ffd640 1 -- 172.21.15.29:0/893110598 <== mds.0 v2:172.21.15.29:6834/1924706890 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c112878ab0 con 0x55c112abf680 2024-04-25T00:31:47.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fc2ffd640 10 client.4638 handle_command_reply: tid=0 2024-04-25T00:31:47.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 client.4638 shutdown 2024-04-25T00:31:47.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 -- 172.21.15.29:0/893110598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 msgr2=0x55c112abe9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.296+0000 7f5fed54b000 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 0x55c112abe9a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e4a0 tx=0x7f5fdc00e960 comp rx=0 tx=0).stop 2024-04-25T00:31:47.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.300+0000 7f5fed54b000 1 -- 172.21.15.29:0/893110598 shutdown_connections 2024-04-25T00:31:47.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.300+0000 7f5fed54b000 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.29:6834/1924706890,v1:172.21.15.29:6835/1924706890] conn(0x55c112abf680 0x55c112aeafa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.300+0000 7f5fed54b000 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 0x55c112abe9a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.300+0000 7f5fed54b000 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112ad8ef0 0x55c112abe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.300+0000 7f5fed54b000 1 --2- 172.21.15.29:0/893110598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112a8fbc0 0x55c112abdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.300+0000 7f5fed54b000 1 -- 172.21.15.29:0/893110598 >> 172.21.15.29:0/893110598 conn(0x55c112a8bb70 msgr2=0x55c112a8e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:47.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.300+0000 7f5fc2ffd640 0 client.4638 ms_handle_reset on v2:172.21.15.29:6834/1924706890 2024-04-25T00:31:47.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.300+0000 7f5fed54b000 1 -- 172.21.15.29:0/893110598 shutdown_connections 2024-04-25T00:31:47.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.300+0000 7f5fed54b000 1 -- 172.21.15.29:0/893110598 wait complete. 2024-04-25T00:31:47.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.300+0000 7f5fed54b000 20 client.4638 trim_cache size 0 max 16384 2024-04-25T00:31:47.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 Processor -- start 2024-04-25T00:31:47.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 -- start start 2024-04-25T00:31:47.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 0x55c112b44750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 0x55c112b44c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad9810 0x55c112b451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4203560 con 0x55c112ad8ef0 2024-04-25T00:31:47.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe41fb220 con 0x55c112ad9810 2024-04-25T00:31:47.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe41fa580 con 0x55c112a8fbc0 2024-04-25T00:31:47.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fe37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 0x55c112b44c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fe37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 0x55c112b44c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50326/0 (socket says 172.21.15.29:50326) 2024-04-25T00:31:47.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fe37fe640 1 -- 172.21.15.29:0/718804894 learned_addr learned my addr 172.21.15.29:0/718804894 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:47.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fe896f640 1 --2- 172.21.15.29:0/718804894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad9810 0x55c112b451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/718804894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 0x55c112b44750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fe37fe640 1 -- 172.21.15.29:0/718804894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 msgr2=0x55c112b44750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:47.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fe37fe640 1 --2- 172.21.15.29:0/718804894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 0x55c112b44750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fe37fe640 1 -- 172.21.15.29:0/718804894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad9810 msgr2=0x55c112b451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:47.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fe37fe640 1 --2- 172.21.15.29:0/718804894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad9810 0x55c112b451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fe37fe640 1 -- 172.21.15.29:0/718804894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c112865e80 con 0x55c112ad8ef0 2024-04-25T00:31:47.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/718804894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 0x55c112b44750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:47.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fe37fe640 1 --2- 172.21.15.29:0/718804894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 0x55c112b44c90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5fd801a0d0 tx=0x7f5fd8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd0d5c32fc147b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:47.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fc97fa640 1 -- 172.21.15.29:0/718804894 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80b0050 con 0x55c112ad8ef0 2024-04-25T00:31:47.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fc97fa640 1 -- 172.21.15.29:0/718804894 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd80c9070 con 0x55c112ad8ef0 2024-04-25T00:31:47.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fc97fa640 1 -- 172.21.15.29:0/718804894 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80c4480 con 0x55c112ad8ef0 2024-04-25T00:31:47.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 -- 172.21.15.29:0/718804894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 msgr2=0x55c112b44c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 --2- 172.21.15.29:0/718804894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 0x55c112b44c90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5fd801a0d0 tx=0x7f5fd8004740 comp rx=0 tx=0).stop 2024-04-25T00:31:47.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 -- 172.21.15.29:0/718804894 shutdown_connections 2024-04-25T00:31:47.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 --2- 172.21.15.29:0/718804894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad9810 0x55c112b451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 --2- 172.21.15.29:0/718804894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad8ef0 0x55c112b44c90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 --2- 172.21.15.29:0/718804894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 0x55c112b44750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 -- 172.21.15.29:0/718804894 >> 172.21.15.29:0/718804894 conn(0x55c112a8bb70 msgr2=0x55c112ad7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:47.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 -- 172.21.15.29:0/718804894 shutdown_connections 2024-04-25T00:31:47.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.316+0000 7f5fed54b000 1 -- 172.21.15.29:0/718804894 wait complete. 2024-04-25T00:31:47.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 -- start start 2024-04-25T00:31:47.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 Processor -- start 2024-04-25T00:31:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 0x55c112b482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad8ef0 0x55c112aeb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 0x55c112aebc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c112865640 con 0x55c112ad9810 2024-04-25T00:31:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe41fabd0 con 0x55c112ad8ef0 2024-04-25T00:31:47.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55c112875dd0 con 0x55c112a8fbc0 2024-04-25T00:31:47.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 10 client.0 _ll_register_callbacks cb 0x7f5fdc001650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:47.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fe896f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 0x55c112aebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fe896f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 0x55c112aebc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50340/0 (socket says 172.21.15.29:50340) 2024-04-25T00:31:47.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fe896f640 1 -- 172.21.15.29:0/2131602532 learned_addr learned my addr 172.21.15.29:0/2131602532 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:47.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fe37fe640 1 --2- 172.21.15.29:0/2131602532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad8ef0 0x55c112aeb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/2131602532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 0x55c112b482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fe896f640 1 -- 172.21.15.29:0/2131602532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 msgr2=0x55c112b482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:47.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fe896f640 1 --2- 172.21.15.29:0/2131602532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 0x55c112b482d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fe896f640 1 -- 172.21.15.29:0/2131602532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad8ef0 msgr2=0x55c112aeb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:47.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fe896f640 1 --2- 172.21.15.29:0/2131602532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad8ef0 0x55c112aeb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fe896f640 1 -- 172.21.15.29:0/2131602532 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1128aa760 con 0x55c112ad9810 2024-04-25T00:31:47.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fe896f640 1 --2- 172.21.15.29:0/2131602532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 0x55c112aebc50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e470 tx=0x7f5fdc0014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b21bbdb3b0a50b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:47.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:31:47.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fc2ffd640 1 -- 172.21.15.29:0/2131602532 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc0056a0 con 0x55c112ad9810 2024-04-25T00:31:47.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fc2ffd640 1 -- 172.21.15.29:0/2131602532 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5fdc014480 con 0x55c112ad9810 2024-04-25T00:31:47.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 -- 172.21.15.29:0/2131602532 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd80c9070 con 0x55c112ad9810 2024-04-25T00:31:47.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fc2ffd640 1 -- 172.21.15.29:0/2131602532 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f5fdc005840 con 0x55c112ad9810 2024-04-25T00:31:47.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 10 client.4644 fetch_fsmap learned FSMap version 6 2024-04-25T00:31:47.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 -- 172.21.15.29:0/2131602532 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55c1129369e0 con 0x55c112ad9810 2024-04-25T00:31:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fc2ffd640 1 -- 172.21.15.29:0/2131602532 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd80c9070 con 0x55c112ad9810 2024-04-25T00:31:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fc2ffd640 1 -- 172.21.15.29:0/2131602532 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 6) ==== 3494+0+0 (secure 0 0 0) 0x7f5fdc022040 con 0x55c112ad9810 2024-04-25T00:31:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 10 client.4644 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T00:31:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 10 client.4644 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:31:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 20 client.4644 populate_metadata read hostname 'smithi029' 2024-04-25T00:31:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 --2- 172.21.15.29:0/2131602532 >> [v2:172.21.15.29:6834/1924706890,v1:172.21.15.29:6835/1924706890] conn(0x55c112b4fa10 0x55c112aecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 4 client.4644 mds_command: new command op to 4252 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:31:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 -- 172.21.15.29:0/2131602532 --> [v2:172.21.15.29:6834/1924706890,v1:172.21.15.29:6835/1924706890] -- command(tid 0: {"prefix": "status"}) -- 0x55c1128eab20 con 0x55c112b4fa10 2024-04-25T00:31:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/2131602532 >> [v2:172.21.15.29:6834/1924706890,v1:172.21.15.29:6835/1924706890] conn(0x55c112b4fa10 0x55c112aecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fc2ffd640 10 client.4644 ms_handle_connect on v2:172.21.15.29:6834/1924706890 2024-04-25T00:31:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/2131602532 >> [v2:172.21.15.29:6834/1924706890,v1:172.21.15.29:6835/1924706890] conn(0x55c112b4fa10 0x55c112aecae0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffda77eec089d0aa server_cookie=32b88a9218417b87 in_seq=0 out_seq=0 2024-04-25T00:31:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fc2ffd640 1 -- 172.21.15.29:0/2131602532 <== mds.0 v2:172.21.15.29:6834/1924706890 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55c1128eab20 con 0x55c112b4fa10 2024-04-25T00:31:47.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fc2ffd640 10 client.4644 handle_command_reply: tid=0 2024-04-25T00:31:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 client.4644 shutdown 2024-04-25T00:31:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 -- 172.21.15.29:0/2131602532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 msgr2=0x55c112aebc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 --2- 172.21.15.29:0/2131602532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 0x55c112aebc50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e470 tx=0x7f5fdc0014b0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 -- 172.21.15.29:0/2131602532 shutdown_connections 2024-04-25T00:31:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 --2- 172.21.15.29:0/2131602532 >> [v2:172.21.15.29:6834/1924706890,v1:172.21.15.29:6835/1924706890] conn(0x55c112b4fa10 0x55c112aecae0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 --2- 172.21.15.29:0/2131602532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c112ad9810 0x55c112aebc50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 --2- 172.21.15.29:0/2131602532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c112ad8ef0 0x55c112aeb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fc2ffd640 0 client.4644 ms_handle_reset on v2:172.21.15.29:6834/1924706890 2024-04-25T00:31:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 --2- 172.21.15.29:0/2131602532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c112a8fbc0 0x55c112b482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 -- 172.21.15.29:0/2131602532 >> 172.21.15.29:0/2131602532 conn(0x55c112a8bb70 msgr2=0x55c112ad7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:47.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 -- 172.21.15.29:0/2131602532 shutdown_connections 2024-04-25T00:31:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 1 -- 172.21.15.29:0/2131602532 wait complete. 2024-04-25T00:31:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.320+0000 7f5fed54b000 20 client.4644 trim_cache size 0 max 16384 2024-04-25T00:31:47.333 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:31:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "59afbe7d-aaa8-45e2-b468-69ee150cacea", 2024-04-25T00:31:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-04-25T00:31:47.333 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 4252, 2024-04-25T00:31:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-04-25T00:31:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-04-25T00:31:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-04-25T00:31:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 17.064380373999999, 2024-04-25T00:31:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 6, 2024-04-25T00:31:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 20, 2024-04-25T00:31:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T00:31:47.334 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 23.824850321 2024-04-25T00:31:47.334 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:31:47.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.324+0000 7f5fea3d2640 1 -- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5fd00678f0 msgr2=0x7f5fd0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.324+0000 7f5fea3d2640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5fd00678f0 0x7f5fd0069db0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80ad000 tx=0x7f5fd80a6000 comp rx=0 tx=0).stop 2024-04-25T00:31:47.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.324+0000 7f5fea3d2640 1 -- 172.21.15.29:0/3102199501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fe40b4780 msgr2=0x7f5fe4070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.324+0000 7f5fea3d2640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fe40b4780 0x7f5fe4070ce0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400c420 tx=0x7f5fd400c8e0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.324+0000 7f5fea3d2640 1 -- 172.21.15.29:0/3102199501 shutdown_connections 2024-04-25T00:31:47.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.324+0000 7f5fea3d2640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5fd00678f0 0x7f5fd0069db0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.324+0000 7f5fea3d2640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe415e8d0 0x7f5fe41405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.324+0000 7f5fea3d2640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fe40cc240 0x7f5fe406d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.324+0000 7f5fea3d2640 1 --2- 172.21.15.29:0/3102199501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fe40b4780 0x7f5fe4070ce0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.324+0000 7f5fea3d2640 1 -- 172.21.15.29:0/3102199501 >> 172.21.15.29:0/3102199501 conn(0x7f5fe40b1870 msgr2=0x7f5fe40ae740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:47.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.324+0000 7f5fea3d2640 1 -- 172.21.15.29:0/3102199501 shutdown_connections 2024-04-25T00:31:47.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.324+0000 7f5fea3d2640 1 -- 172.21.15.29:0/3102199501 wait complete. 2024-04-25T00:31:47.341 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T00:31:47.420 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T00:31:47.420 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T00:31:47.420 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T00:31:47.421 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T00:31:47.467 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T00:31:47.467 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T00:31:47.467 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T00:31:47.467 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:31:47.467 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:31:47.467 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T00:31:47.467 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:31:47.467 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T00:31:47.467 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T00:31:47.468 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:31:47.468 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:31:47.468 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T00:31:47.468 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:31:47.468 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T00:31:47.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:47.468 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:31:47.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:47.475 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-04-25T00:31:47.533 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2024-04-25T00:31:47.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:31:47.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:47.535 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:31:47.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:47.541 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link delete ceph-brx 2024-04-25T00:31:47.602 INFO:teuthology.orchestra.run.smithi029.stderr:Cannot find device "ceph-brx" 2024-04-25T00:31:47.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:31:47.604 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T00:31:47.604 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:31:47.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0f79d640 1 Processor -- start 2024-04-25T00:31:47.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0f79d640 1 -- start start 2024-04-25T00:31:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0f79d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8a0815df70 0x7f8a080b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0f79d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8a0806d210 0x7f8a080b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0f79d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0806d980 con 0x7f8a080b4b60 2024-04-25T00:31:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0f79d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8a080b4fd0 con 0x7f8a0806d210 2024-04-25T00:31:47.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0f79d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8a0806dc60 con 0x7f8a0815df70 2024-04-25T00:31:47.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0cd38640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8a0806d210 0x7f8a080b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0cd38640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8a0806d210 0x7f8a080b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47432/0 (socket says 172.21.15.29:47432) 2024-04-25T00:31:47.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0dd3a640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8a080b4b60 0x7f8a08070ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:45782/0 (socket says 172.21.15.29:45782) 2024-04-25T00:31:47.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0cd38640 1 -- 172.21.15.29:0/3599464699 learned_addr learned my addr 172.21.15.29:0/3599464699 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:47.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0d539640 1 --2- 172.21.15.29:0/3599464699 >> v2:172.21.15.157:3300/0 conn(0x7f8a0815df70 0x7f8a080b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0cd38640 1 -- 172.21.15.29:0/3599464699 >> v2:172.21.15.157:3300/0 conn(0x7f8a0815df70 msgr2=0x7f8a080b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:47.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0cd38640 1 --2- 172.21.15.29:0/3599464699 >> v2:172.21.15.157:3300/0 conn(0x7f8a0815df70 0x7f8a080b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0cd38640 1 -- 172.21.15.29:0/3599464699 >> v1:172.21.15.29:6789/0 conn(0x7f8a080b4b60 legacy=0x7f8a08070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f89fbfff640 1 -- 172.21.15.29:0/3599464699 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 441562103 0 0) 0x7f8a0806d980 con 0x7f8a080b4b60 2024-04-25T00:31:47.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.776+0000 7f8a0cd38640 1 -- 172.21.15.29:0/3599464699 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0806df40 con 0x7f8a0806d210 2024-04-25T00:31:47.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0cd38640 1 --2- 172.21.15.29:0/3599464699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0806d210 0x7f8a080b4620 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8a080d0bf0 tx=0x7f8a0401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8716fccdd04093c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f89fbfff640 1 -- 172.21.15.29:0/3599464699 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040b0070 con 0x7f8a0806d210 2024-04-25T00:31:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f89fbfff640 1 -- 172.21.15.29:0/3599464699 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a040ab600 con 0x7f8a0806d210 2024-04-25T00:31:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f89fbfff640 1 -- 172.21.15.29:0/3599464699 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040ab9d0 con 0x7f8a0806d210 2024-04-25T00:31:47.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 -- 172.21.15.29:0/3599464699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0806d210 msgr2=0x7f8a080b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 --2- 172.21.15.29:0/3599464699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0806d210 0x7f8a080b4620 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8a080d0bf0 tx=0x7f8a0401c520 comp rx=0 tx=0).stop 2024-04-25T00:31:47.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 -- 172.21.15.29:0/3599464699 shutdown_connections 2024-04-25T00:31:47.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 --2- 172.21.15.29:0/3599464699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0806d210 0x7f8a080b4620 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 --2- 172.21.15.29:0/3599464699 >> v2:172.21.15.157:3300/0 conn(0x7f8a0815df70 0x7f8a080b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 -- 172.21.15.29:0/3599464699 >> 172.21.15.29:0/3599464699 conn(0x7f8a080b1870 msgr2=0x7f8a080b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:47.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 -- 172.21.15.29:0/3599464699 shutdown_connections 2024-04-25T00:31:47.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 -- 172.21.15.29:0/3599464699 wait complete. 2024-04-25T00:31:47.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 Processor -- start 2024-04-25T00:31:47.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 -- start start 2024-04-25T00:31:47.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0806d210 0x7f8a08154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a080b4b60 0x7f8a080cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0815df70 0x7f8a0813d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0806ea20 con 0x7f8a0815df70 2024-04-25T00:31:47.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0815e350 con 0x7f8a0806d210 2024-04-25T00:31:47.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0f79d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0806eba0 con 0x7f8a080b4b60 2024-04-25T00:31:47.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0dd3a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0815df70 0x7f8a0813d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0dd3a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0815df70 0x7f8a0813d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47440/0 (socket says 172.21.15.29:47440) 2024-04-25T00:31:47.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.780+0000 7f8a0dd3a640 1 -- 172.21.15.29:0/256591143 learned_addr learned my addr 172.21.15.29:0/256591143 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:47.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f8a0cd38640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a080b4b60 0x7f8a080cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f8a0d539640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0806d210 0x7f8a08154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f8a0dd3a640 1 -- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a080b4b60 msgr2=0x7f8a080cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f8a0dd3a640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a080b4b60 0x7f8a080cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f8a0dd3a640 1 -- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0806d210 msgr2=0x7f8a08154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f8a0dd3a640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0806d210 0x7f8a08154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f8a0dd3a640 1 -- 172.21.15.29:0/256591143 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a080ce610 con 0x7f8a0815df70 2024-04-25T00:31:47.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f8a0dd3a640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0815df70 0x7f8a0813d370 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000c480 tx=0x7f8a0000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:47.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f89f9ffb640 1 -- 172.21.15.29:0/256591143 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00018070 con 0x7f8a0815df70 2024-04-25T00:31:47.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f89f9ffb640 1 -- 172.21.15.29:0/256591143 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a000074e0 con 0x7f8a0815df70 2024-04-25T00:31:47.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f89f9ffb640 1 -- 172.21.15.29:0/256591143 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00004a30 con 0x7f8a0815df70 2024-04-25T00:31:47.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f8a0f79d640 1 -- 172.21.15.29:0/256591143 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a080ce8f0 con 0x7f8a0815df70 2024-04-25T00:31:47.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f8a0f79d640 1 -- 172.21.15.29:0/256591143 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a080ceeb0 con 0x7f8a0815df70 2024-04-25T00:31:47.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f8a0f79d640 1 -- 172.21.15.29:0/256591143 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a08069270 con 0x7f8a0815df70 2024-04-25T00:31:47.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f89f9ffb640 1 -- 172.21.15.29:0/256591143 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8a00004bd0 con 0x7f8a0815df70 2024-04-25T00:31:47.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f89f9ffb640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f89ec0678f0 0x7f89ec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:47.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f89f9ffb640 1 -- 172.21.15.29:0/256591143 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8a00084e10 con 0x7f8a0815df70 2024-04-25T00:31:47.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f8a0d539640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f89ec0678f0 0x7f89ec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:47.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.784+0000 7f8a0d539640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f89ec0678f0 0x7f89ec069db0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f89fc003fd0 tx=0x7f89fc0039a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:47.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.788+0000 7f89f9ffb640 1 -- 172.21.15.29:0/256591143 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8a000559e0 con 0x7f8a0815df70 2024-04-25T00:31:47.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.988+0000 7f8a0f79d640 1 -- 172.21.15.29:0/256591143 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8a0806ef20 con 0x7f8a0815df70 2024-04-25T00:31:47.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.988+0000 7f89f9ffb640 1 -- 172.21.15.29:0/256591143 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f8a000552e0 con 0x7f8a0815df70 2024-04-25T00:31:47.991 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:31:47.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.992+0000 7f8a0f79d640 1 -- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f89ec0678f0 msgr2=0x7f89ec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.992+0000 7f8a0f79d640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f89ec0678f0 0x7f89ec069db0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f89fc003fd0 tx=0x7f89fc0039a0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.992+0000 7f8a0f79d640 1 -- 172.21.15.29:0/256591143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0815df70 msgr2=0x7f8a0813d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:47.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.992+0000 7f8a0f79d640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0815df70 0x7f8a0813d370 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000c480 tx=0x7f8a0000c940 comp rx=0 tx=0).stop 2024-04-25T00:31:47.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.992+0000 7f8a0f79d640 1 -- 172.21.15.29:0/256591143 shutdown_connections 2024-04-25T00:31:47.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.992+0000 7f8a0f79d640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0815df70 0x7f8a0813d370 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.992+0000 7f8a0f79d640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a080b4b60 0x7f8a080cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.992+0000 7f8a0f79d640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f89ec0678f0 0x7f89ec069db0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.992+0000 7f8a0f79d640 1 --2- 172.21.15.29:0/256591143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a0806d210 0x7f8a08154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:47.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.992+0000 7f8a0f79d640 1 -- 172.21.15.29:0/256591143 >> 172.21.15.29:0/256591143 conn(0x7f8a080b1870 msgr2=0x7f8a080b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:47.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.992+0000 7f8a0f79d640 1 -- 172.21.15.29:0/256591143 shutdown_connections 2024-04-25T00:31:47.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:47.992+0000 7f8a0f79d640 1 -- 172.21.15.29:0/256591143 wait complete. 2024-04-25T00:31:48.011 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:31:48.011 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:31:48.011 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-25T00:31:48.011 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:31:48.011 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:31:48.011 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:31:48.011 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:31:48.011 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:31:48.011 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:31:48.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:48.012 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-25T00:31:48.018 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:31:48.018 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:31:48.018 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:31:48.018 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:48.018 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:31:48.018 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:48.018 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:31:48.018 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:48.018 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-25T00:31:48.018 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:31:48.018 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:48.018 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42452sec preferred_lft 42452sec 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 02:42:28:a5:36:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:31:48.019 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:48.020 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:31:48.020 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:48.020 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:31:48.020 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:31:48.020 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip addr flush dev ceph-brx 2024-04-25T00:31:48.020 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set ceph-brx up 2024-04-25T00:31:48.020 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:31:48.020 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:31:48.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:48.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:48.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:48.150 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:31:48.159 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-25T00:31:48.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:48.160 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-25T00:31:48.232 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-25T00:31:48.232 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:31:48.232 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T00:31:48.232 INFO:teuthology.orchestra.run.smithi029.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:31:48.233 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T00:31:48.233 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:31:48.233 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:48.233 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:31:48.233 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:31:48.234 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:31:48.234 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:31:48.234 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:31:48.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:48.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:48.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:48.356 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:31:48.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:48.362 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list-id 2024-04-25T00:31:48.409 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:48.410 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:31:48.410 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:31:48.410 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T00:31:48.410 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:31:48.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:48.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:48.531 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T00:31:48.531 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:48.531 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:31:48.532 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T00:31:48.532 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:31:48.532 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T00:31:48.532 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T00:31:48.532 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T00:31:48.532 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:31:48.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:48.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:48.643 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:48.643 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:31:48.643 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-04-25T00:31:48.643 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:31:48.643 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:31:48.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:48.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:48.749 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:31:48.749 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:31:48.749 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:31:48.798 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:31:48.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:31:48.799 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:31:48.845 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:31:48.902 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:31:48.952 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:31:48.952 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:31:48.952 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:31:48.952 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:31:48.952 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:31:48.952 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:31:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:31:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:31:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:31:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:31:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:31:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:31:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:31:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:31:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:31:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:31:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:31:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:31:48.953 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:31:48.954 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:31:48.954 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:31:48.954 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:31:48.954 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:31:48.954 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:31:48.954 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:31:48.954 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:31:48.954 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:31:48.954 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:31:48.954 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:31:48.954 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:31:48.954 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:31:48.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:48.956 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:31:49.002 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:31:49.002 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:31:49.047 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:31:49.062 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:31:49.114 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:31:49.114 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:31:49.114 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:31:49.114 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:31:49.114 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:31:49.114 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:31:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:31:49.116 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:31:49.116 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:31:49.116 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:31:49.116 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:31:49.116 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:31:49.116 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:31:49.116 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:31:49.116 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:31:49.116 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:31:49.116 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:31:49.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:49.117 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:31:49.126 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:2024-04-25T00:31:49.124+0000 7f153958e580 -1 init, newargv = 0x5646de7570c0 newargc=15 2024-04-25T00:31:49.127 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[34433]: starting ceph client 2024-04-25T00:31:49.134 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[34433]: starting fuse 2024-04-25T00:31:50.234 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:31:50.251 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:31:50.298 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:31:50.298 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:31:50.299 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:31:50.300 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:31:50.301 INFO:teuthology.orchestra.run.smithi029.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:31:50.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:50.302 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:31:50.350 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:31:50.350 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:31:50.350 DEBUG:teuthology.orchestra.run.smithi029:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:31:50.350 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-04-25T00:31:50.350 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-04-25T00:31:50.350 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-04-25T00:31:50.350 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-04-25T00:31:50.350 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-04-25T00:31:50.351 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-04-25T00:31:50.352 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-04-25T00:31:50.352 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:31:50.352 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:31:50.352 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.0")) 2024-04-25T00:31:50.352 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-04-25T00:31:50.481 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.0.34433.asok 2024-04-25T00:31:50.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:50.495 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.34433.asok 2024-04-25T00:31:50.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:50.495 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.34433.asok status 2024-04-25T00:31:50.655 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:31:50.655 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-04-25T00:31:50.656 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:31:50.656 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:31:50.656 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "0", 2024-04-25T00:31:50.656 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-04-25T00:31:50.656 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:31:50.656 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "34433", 2024-04-25T00:31:50.656 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-04-25T00:31:50.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:31:50.656 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-04-25T00:31:50.656 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-04-25T00:31:50.656 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 4659, 2024-04-25T00:31:50.656 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-04-25T00:31:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-04-25T00:31:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-04-25T00:31:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 4659 2024-04-25T00:31:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:31:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:31:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:31:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:31:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 911034902 2024-04-25T00:31:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:31:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:31:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:31:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:31:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:31:50.658 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 911034902 2024-04-25T00:31:50.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:31:50.658 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.4659 192.168.0.1:0/911034902", 2024-04-25T00:31:50.658 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.1:0/911034902", 2024-04-25T00:31:50.658 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-04-25T00:31:50.658 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 6, 2024-04-25T00:31:50.658 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 20, 2024-04-25T00:31:50.658 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:31:50.658 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-04-25T00:31:50.658 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-04-25T00:31:50.658 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:31:50.670 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.792+0000 7fd21a143640 1 Processor -- start 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.792+0000 7fd21a143640 1 -- start start 2024-04-25T00:31:50.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd21a143640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd21415e8d0 0x7fd2140cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd21a143640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd2140b4780 0x7fd21415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd21a143640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2140b3720 con 0x7fd21415f1f0 2024-04-25T00:31:50.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd21a143640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd2140b42f0 con 0x7fd2140b4780 2024-04-25T00:31:50.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd21a143640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd21415d7e0 con 0x7fd21415e8d0 2024-04-25T00:31:50.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd212ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd2140b4780 0x7fd21415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd212ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd2140b4780 0x7fd21415ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47442/0 (socket says 172.21.15.29:47442) 2024-04-25T00:31:50.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd212ffd640 1 -- 172.21.15.29:0/2791137564 learned_addr learned my addr 172.21.15.29:0/2791137564 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:50.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd2137fe640 1 --2- 172.21.15.29:0/2791137564 >> v2:172.21.15.157:3300/0 conn(0x7fd21415e8d0 0x7fd2140cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd212ffd640 1 -- 172.21.15.29:0/2791137564 >> v2:172.21.15.157:3300/0 conn(0x7fd21415e8d0 msgr2=0x7fd2140cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd212ffd640 1 --2- 172.21.15.29:0/2791137564 >> v2:172.21.15.157:3300/0 conn(0x7fd21415e8d0 0x7fd2140cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd212ffd640 1 -- 172.21.15.29:0/2791137564 >> v1:172.21.15.157:6789/0 conn(0x7fd21415f1f0 legacy=0x7fd21415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd212ffd640 1 -- 172.21.15.29:0/2791137564 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2140b39c0 con 0x7fd2140b4780 2024-04-25T00:31:50.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd212ffd640 1 --2- 172.21.15.29:0/2791137564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2140b4780 0x7fd21415ecb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd2140b5260 tx=0x7fd20801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80234c511a43d165 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd2127fc640 1 -- 172.21.15.29:0/2791137564 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21415c1c0 con 0x7fd2140b4780 2024-04-25T00:31:50.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd2127fc640 1 -- 172.21.15.29:0/2791137564 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2080af070 con 0x7fd2140b4780 2024-04-25T00:31:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.796+0000 7fd2127fc640 1 -- 172.21.15.29:0/2791137564 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2080aa830 con 0x7fd2140b4780 2024-04-25T00:31:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 -- 172.21.15.29:0/2791137564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2140b4780 msgr2=0x7fd21415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 --2- 172.21.15.29:0/2791137564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2140b4780 0x7fd21415ecb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd2140b5260 tx=0x7fd20801c4b0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 -- 172.21.15.29:0/2791137564 shutdown_connections 2024-04-25T00:31:50.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 --2- 172.21.15.29:0/2791137564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2140b4780 0x7fd21415ecb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 --2- 172.21.15.29:0/2791137564 >> v2:172.21.15.157:3300/0 conn(0x7fd21415e8d0 0x7fd2140cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 -- 172.21.15.29:0/2791137564 >> 172.21.15.29:0/2791137564 conn(0x7fd2140b1870 msgr2=0x7fd2140b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 -- 172.21.15.29:0/2791137564 shutdown_connections 2024-04-25T00:31:50.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 -- 172.21.15.29:0/2791137564 wait complete. 2024-04-25T00:31:50.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 Processor -- start 2024-04-25T00:31:50.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 -- start start 2024-04-25T00:31:50.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2140b4780 0x7fd21414e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21415e8d0 0x7fd2140a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd21415f1f0 0x7fd2140b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd214144a20 con 0x7fd2140b4780 2024-04-25T00:31:50.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd21415f600 con 0x7fd21415e8d0 2024-04-25T00:31:50.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd21414ebe0 con 0x7fd21415f1f0 2024-04-25T00:31:50.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd2137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2140b4780 0x7fd21414e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd2137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2140b4780 0x7fd21414e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47456/0 (socket says 172.21.15.29:47456) 2024-04-25T00:31:50.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd2137fe640 1 -- 172.21.15.29:0/3920595464 learned_addr learned my addr 172.21.15.29:0/3920595464 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:50.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd212ffd640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21415e8d0 0x7fd2140a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd213fff640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd21415f1f0 0x7fd2140b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd2137fe640 1 -- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd21415f1f0 msgr2=0x7fd2140b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd2137fe640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd21415f1f0 0x7fd2140b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd2137fe640 1 -- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21415e8d0 msgr2=0x7fd2140a9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd2137fe640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21415e8d0 0x7fd2140a9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd2137fe640 1 -- 172.21.15.29:0/3920595464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2140aa810 con 0x7fd2140b4780 2024-04-25T00:31:50.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd2137fe640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2140b4780 0x7fd21414e6a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd20400d800 tx=0x7fd20400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd1fbfff640 1 -- 172.21.15.29:0/3920595464 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20400b6c0 con 0x7fd2140b4780 2024-04-25T00:31:50.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd1fbfff640 1 -- 172.21.15.29:0/3920595464 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd204019070 con 0x7fd2140b4780 2024-04-25T00:31:50.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 -- 172.21.15.29:0/3920595464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2140aaaf0 con 0x7fd2140b4780 2024-04-25T00:31:50.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd1fbfff640 1 -- 172.21.15.29:0/3920595464 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20400ba60 con 0x7fd2140b4780 2024-04-25T00:31:50.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.800+0000 7fd21a143640 1 -- 172.21.15.29:0/3920595464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2140ab0b0 con 0x7fd2140b4780 2024-04-25T00:31:50.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.804+0000 7fd21a143640 1 -- 172.21.15.29:0/3920595464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2140acb30 con 0x7fd2140b4780 2024-04-25T00:31:50.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.804+0000 7fd1fbfff640 1 -- 172.21.15.29:0/3920595464 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd20400bc00 con 0x7fd2140b4780 2024-04-25T00:31:50.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.804+0000 7fd1fbfff640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2000678c0 0x7fd200069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.804+0000 7fd1fbfff640 1 -- 172.21.15.29:0/3920595464 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd2040860d0 con 0x7fd2140b4780 2024-04-25T00:31:50.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.804+0000 7fd212ffd640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2000678c0 0x7fd200069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.804+0000 7fd212ffd640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2000678c0 0x7fd200069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd2080ae000 tx=0x7fd2080a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:50.808+0000 7fd1fbfff640 1 -- 172.21.15.29:0/3920595464 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd2140acb30 con 0x7fd2140b4780 2024-04-25T00:31:51.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.008+0000 7fd21a143640 1 -- 172.21.15.29:0/3920595464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd214069270 con 0x7fd2140b4780 2024-04-25T00:31:51.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.008+0000 7fd1fbfff640 1 -- 172.21.15.29:0/3920595464 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fd204056c10 con 0x7fd2140b4780 2024-04-25T00:31:51.011 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:31:51.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.012+0000 7fd21a143640 1 -- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2000678c0 msgr2=0x7fd200069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.012+0000 7fd21a143640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2000678c0 0x7fd200069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd2080ae000 tx=0x7fd2080a7000 comp rx=0 tx=0).stop 2024-04-25T00:31:51.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.012+0000 7fd21a143640 1 -- 172.21.15.29:0/3920595464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2140b4780 msgr2=0x7fd21414e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.012+0000 7fd21a143640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2140b4780 0x7fd21414e6a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd20400d800 tx=0x7fd20400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.012+0000 7fd21a143640 1 -- 172.21.15.29:0/3920595464 shutdown_connections 2024-04-25T00:31:51.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.012+0000 7fd21a143640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd21415f1f0 0x7fd2140b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.012+0000 7fd21a143640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2000678c0 0x7fd200069d80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.012+0000 7fd21a143640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21415e8d0 0x7fd2140a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.012+0000 7fd21a143640 1 --2- 172.21.15.29:0/3920595464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2140b4780 0x7fd21414e6a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.012+0000 7fd21a143640 1 -- 172.21.15.29:0/3920595464 >> 172.21.15.29:0/3920595464 conn(0x7fd2140b1870 msgr2=0x7fd2140b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.012+0000 7fd21a143640 1 -- 172.21.15.29:0/3920595464 shutdown_connections 2024-04-25T00:31:51.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51.012+0000 7fd21a143640 1 -- 172.21.15.29:0/3920595464 wait complete. 2024-04-25T00:31:51.031 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:31:51.032 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:31:51.032 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-25T00:31:51.032 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T00:31:51.032 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T00:31:51.032 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:31:51.032 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:31:51.032 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:31:51.032 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T00:31:51.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:51.032 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-25T00:31:51.038 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:31:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:31:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:31:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.038 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42449sec preferred_lft 42449sec 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-04-25T00:31:51.039 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 02:42:28:a5:36:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.040 INFO:teuthology.orchestra.run.smithi029.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:31:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:31:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:31:51.041 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:51.041 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:31:51.090 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-25T00:31:51.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:51.092 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-25T00:31:51.140 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-25T00:31:51.140 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:31:51.140 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T00:31:51.140 INFO:teuthology.orchestra.run.smithi029.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:31:51.141 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T00:31:51.141 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:31:51.141 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:51.141 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:31:51.141 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:31:51.141 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:31:51.141 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:31:51.141 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:31:51.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:51.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:51.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:51.270 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:31:51.275 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:31:51.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:51.276 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list-id 2024-04-25T00:31:51.322 INFO:teuthology.orchestra.run.smithi029.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T00:31:51.323 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:51.323 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:31:51.323 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:31:51.323 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T00:31:51.323 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:31:51.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:51.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:51.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:51.442 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T00:31:51.455 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:31:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:31:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:31:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:31:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.455 INFO:teuthology.orchestra.run.smithi029.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:31:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether d2:c8:f8:d9:0e:fd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T00:31:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T00:31:51.455 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::d0c8:f8ff:fed9:efd/64 scope link 2024-04-25T00:31:51.456 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:51.457 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T00:31:51.513 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:31:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:31:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:31:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:31:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.513 INFO:teuthology.orchestra.run.smithi029.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:31:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether d2:c8:f8:d9:0e:fd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T00:31:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T00:31:51.513 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::d0c8:f8ff:fed9:efd/64 scope link 2024-04-25T00:31:51.514 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:51.515 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T00:31:51.515 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:51.515 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:31:51.515 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T00:31:51.515 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T00:31:51.515 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T00:31:51.515 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T00:31:51.515 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T00:31:51.515 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:31:51.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:51.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:51.660 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:51.660 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:31:51.660 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.1 up 2024-04-25T00:31:51.660 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T00:31:51.660 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:31:51.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:51.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:51.721 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T00:31:51.721 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:31:51.721 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T00:31:51.769 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T00:31:51.770 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:31:51.770 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T00:31:51.816 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:31:51.869 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:31:51.920 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:31:51.920 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:31:51.920 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:31:51.920 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:31:51.920 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:31:51.920 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:31:51.920 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:31:51.920 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:31:51.920 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:31:51.921 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:31:51.922 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:31:51.922 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:31:51.922 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:31:51.922 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:31:51.922 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:31:51.922 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:31:51.922 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:31:51.922 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:31:51.922 INFO:teuthology.orchestra.run.smithi029.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:31:51.922 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:31:51.922 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:31:51.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:51.923 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:31:51.974 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:31:51.974 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T00:31:51.974 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T00:31:52.019 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:31:52.033 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:31:52.082 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:31:52.082 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:31:52.082 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:31:52.082 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:31:52.082 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:31:52.082 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:31:52.082 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:31:52.082 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:31:52.083 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:31:52.084 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:31:52.084 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:31:52.084 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:31:52.084 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:31:52.084 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:31:52.084 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:31:52.084 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:31:52.084 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:31:52.084 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:31:52.084 INFO:teuthology.orchestra.run.smithi029.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:31:52.084 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:31:52.084 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:31:52.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:52.085 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:31:52.098 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi029.stderr:2024-04-25T00:31:52.096+0000 7f4404906580 -1 init, newargv = 0x562e2729fe80 newargc=15 2024-04-25T00:31:52.098 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi029.stderr:ceph-fuse[34578]: starting ceph client 2024-04-25T00:31:52.104 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi029.stderr:ceph-fuse[34578]: starting fuse 2024-04-25T00:31:52.106 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:31:52.106 INFO:teuthology.orchestra.run.smithi029.stdout:46 2024-04-25T00:31:52.106 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T00:31:52.107 DEBUG:teuthology.orchestra.run.smithi029:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:31:52.107 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-04-25T00:31:52.107 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-04-25T00:31:52.107 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-04-25T00:31:52.107 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-04-25T00:31:52.107 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:31:52.107 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-04-25T00:31:52.107 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T00:31:52.107 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-04-25T00:31:52.107 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T00:31:52.107 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:31:52.107 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-04-25T00:31:52.107 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.2")) 2024-04-25T00:31:52.108 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-04-25T00:31:52.221 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.2.34578.asok 2024-04-25T00:31:52.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:52.233 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.34578.asok 2024-04-25T00:31:52.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:52.233 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.34578.asok status 2024-04-25T00:31:52.394 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:31:52.394 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-04-25T00:31:52.394 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:31:52.394 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:31:52.394 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "2", 2024-04-25T00:31:52.395 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-04-25T00:31:52.395 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T00:31:52.395 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "34578", 2024-04-25T00:31:52.395 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-04-25T00:31:52.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:31:52.395 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-04-25T00:31:52.395 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-04-25T00:31:52.396 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 4671, 2024-04-25T00:31:52.396 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-04-25T00:31:52.396 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-04-25T00:31:52.396 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-04-25T00:31:52.396 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 4671 2024-04-25T00:31:52.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:31:52.396 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:31:52.397 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:31:52.397 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.2:0", 2024-04-25T00:31:52.397 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 921331477 2024-04-25T00:31:52.397 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:31:52.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:31:52.397 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:31:52.397 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:31:52.397 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.2:0", 2024-04-25T00:31:52.397 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 921331477 2024-04-25T00:31:52.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:31:52.397 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.4671 192.168.0.2:0/921331477", 2024-04-25T00:31:52.397 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.2:0/921331477", 2024-04-25T00:31:52.397 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-04-25T00:31:52.398 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 6, 2024-04-25T00:31:52.398 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 20, 2024-04-25T00:31:52.398 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:31:52.398 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-04-25T00:31:52.398 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-04-25T00:31:52.398 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:31:52.409 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f369e8b2640 1 Processor -- start 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f369e8b2640 1 -- start start 2024-04-25T00:31:52.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f369e8b2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36980b3140 0x7f369806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f369e8b2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36980b41e0 con 0x7f36980b3c30 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f369e8b2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f369806d960 con 0x7f36980b4b10 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f369e8b2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f369806dc40 con 0x7f36980b3140 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f369ce4f640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f36980b3c30 0x7f36980ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:45792/0 (socket says 172.21.15.29:45792) 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f369ce4f640 1 -- 172.21.15.29:0/4219641212 learned_addr learned my addr 172.21.15.29:0/4219641212 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3697fff640 1 --2- 172.21.15.29:0/4219641212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36980b3140 0x7f369806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1206754497 0 0) 0x7f36980b41e0 con 0x7f36980b3c30 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3680003530 con 0x7f36980b3c30 2024-04-25T00:31:52.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3707033607 0 0) 0x7f3680003530 con 0x7f36980b3c30 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36980b3140 msgr2=0x7f369806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 --2- 172.21.15.29:0/4219641212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36980b3140 0x7f369806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 >> v1:172.21.15.157:6789/0 conn(0x7f36980b4b10 legacy=0x7f36980b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369806df20 con 0x7f36980b3c30 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f3690001a00 con 0x7f36980b3c30 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 >> v1:172.21.15.29:6789/0 conn(0x7f36980b3c30 legacy=0x7f36980ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 --2- 172.21.15.29:0/4219641212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36800039f0 0x7f3680003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 --2- 172.21.15.29:0/4219641212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36800043a0 0x7f3680004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 --2- 172.21.15.29:0/4219641212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36980b3140 0x7f36800050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f369806dc40 con 0x7f36800043a0 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f368000b9d0 con 0x7f36800039f0 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f368000bbd0 con 0x7f36980b3140 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f36977fe640 1 --2- 172.21.15.29:0/4219641212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36800043a0 0x7f3680004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f3697fff640 1 --2- 172.21.15.29:0/4219641212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36800039f0 0x7f3680003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f369ce4f640 1 --2- 172.21.15.29:0/4219641212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36980b3140 0x7f36800050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f36977fe640 1 -- 172.21.15.29:0/4219641212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36980b3140 msgr2=0x7f36800050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f36977fe640 1 --2- 172.21.15.29:0/4219641212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36980b3140 0x7f36800050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f36977fe640 1 -- 172.21.15.29:0/4219641212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36800039f0 msgr2=0x7f3680003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f36977fe640 1 --2- 172.21.15.29:0/4219641212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36800039f0 0x7f3680003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f36977fe640 1 -- 172.21.15.29:0/4219641212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3680003530 con 0x7f36800043a0 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.536+0000 7f369ce4f640 1 --2- 172.21.15.29:0/4219641212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36980b3140 0x7f36800050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:52.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.540+0000 7f36977fe640 1 --2- 172.21.15.29:0/4219641212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36800043a0 0x7f3680004870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f368c001860 tx=0x7f368c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea67f5fdf4f1d800 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.540+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c0b0070 con 0x7f36800043a0 2024-04-25T00:31:52.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.540+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f368c0ab600 con 0x7f36800043a0 2024-04-25T00:31:52.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.540+0000 7f3696ffd640 1 -- 172.21.15.29:0/4219641212 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c001460 con 0x7f36800043a0 2024-04-25T00:31:52.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.540+0000 7f369e8b2640 1 -- 172.21.15.29:0/4219641212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36800043a0 msgr2=0x7f3680004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.540+0000 7f369e8b2640 1 --2- 172.21.15.29:0/4219641212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36800043a0 0x7f3680004870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f368c001860 tx=0x7f368c01c500 comp rx=0 tx=0).stop 2024-04-25T00:31:52.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.540+0000 7f369ce4f640 1 -- 172.21.15.29:0/4219641212 reap_dead start 2024-04-25T00:31:52.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.540+0000 7f369e8b2640 1 -- 172.21.15.29:0/4219641212 shutdown_connections 2024-04-25T00:31:52.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.540+0000 7f369e8b2640 1 -- 172.21.15.29:0/4219641212 >> 172.21.15.29:0/4219641212 conn(0x7f36980b1870 msgr2=0x7f36980b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.540+0000 7f369e8b2640 1 -- 172.21.15.29:0/4219641212 shutdown_connections 2024-04-25T00:31:52.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.540+0000 7f369e8b2640 1 -- 172.21.15.29:0/4219641212 wait complete. 2024-04-25T00:31:52.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.540+0000 7f369e8b2640 1 Processor -- start 2024-04-25T00:31:52.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.540+0000 7f369e8b2640 1 -- start start 2024-04-25T00:31:52.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f369e8b2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3698156760 0x7f369815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f369e8b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3698156b40 0x7f369815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f369e8b2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369813c1d0 0x7f369813c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f369e8b2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3698167d80 con 0x7f3698156b40 2024-04-25T00:31:52.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f369e8b2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3698167c00 con 0x7f369813c1d0 2024-04-25T00:31:52.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f369e8b2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3698167f00 con 0x7f3698156760 2024-04-25T00:31:52.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f36977fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3698156b40 0x7f369815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f36977fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3698156b40 0x7f369815f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47470/0 (socket says 172.21.15.29:47470) 2024-04-25T00:31:52.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f36977fe640 1 -- 172.21.15.29:0/2591198170 learned_addr learned my addr 172.21.15.29:0/2591198170 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:52.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f3697fff640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3698156760 0x7f369815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f369ce4f640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369813c1d0 0x7f369813c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f36977fe640 1 -- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3698156760 msgr2=0x7f369815ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f36977fe640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3698156760 0x7f369815ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f36977fe640 1 -- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369813c1d0 msgr2=0x7f369813c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f36977fe640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369813c1d0 0x7f369813c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f36977fe640 1 -- 172.21.15.29:0/2591198170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369813cb90 con 0x7f3698156b40 2024-04-25T00:31:52.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f36977fe640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3698156b40 0x7f369815f0f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f368c01c330 tx=0x7f3680005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f3694ff9640 1 -- 172.21.15.29:0/2591198170 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c0b9070 con 0x7f3698156b40 2024-04-25T00:31:52.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f3694ff9640 1 -- 172.21.15.29:0/2591198170 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f368c0ab7b0 con 0x7f3698156b40 2024-04-25T00:31:52.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f369e8b2640 1 -- 172.21.15.29:0/2591198170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369815cfb0 con 0x7f3698156b40 2024-04-25T00:31:52.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f3694ff9640 1 -- 172.21.15.29:0/2591198170 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c0b0040 con 0x7f3698156b40 2024-04-25T00:31:52.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f369e8b2640 1 -- 172.21.15.29:0/2591198170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f369815d570 con 0x7f3698156b40 2024-04-25T00:31:52.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f369e8b2640 1 -- 172.21.15.29:0/2591198170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3698069270 con 0x7f3698156b40 2024-04-25T00:31:52.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f3694ff9640 1 -- 172.21.15.29:0/2591198170 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f368c0bc070 con 0x7f3698156b40 2024-04-25T00:31:52.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f3694ff9640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f36840678f0 0x7f3684069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f3694ff9640 1 -- 172.21.15.29:0/2591198170 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f368c11ed40 con 0x7f3698156b40 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.544+0000 7f3697fff640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f36840678f0 0x7f3684069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.548+0000 7f3697fff640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f36840678f0 0x7f3684069db0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f36981572c0 tx=0x7f3688008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.548+0000 7f3694ff9640 1 -- 172.21.15.29:0/2591198170 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f368c016210 con 0x7f3698156b40 2024-04-25T00:31:52.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.752+0000 7f369e8b2640 1 -- 172.21.15.29:0/2591198170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f369813d8d0 con 0x7f3698156b40 2024-04-25T00:31:52.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.752+0000 7f3694ff9640 1 -- 172.21.15.29:0/2591198170 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f368c0ef240 con 0x7f3698156b40 2024-04-25T00:31:52.755 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:31:52.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.752+0000 7f369e8b2640 1 -- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f36840678f0 msgr2=0x7f3684069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.752+0000 7f369e8b2640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f36840678f0 0x7f3684069db0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f36981572c0 tx=0x7f3688008040 comp rx=0 tx=0).stop 2024-04-25T00:31:52.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.752+0000 7f369e8b2640 1 -- 172.21.15.29:0/2591198170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3698156b40 msgr2=0x7f369815f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.752+0000 7f369e8b2640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3698156b40 0x7f369815f0f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f368c01c330 tx=0x7f3680005840 comp rx=0 tx=0).stop 2024-04-25T00:31:52.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.756+0000 7f369e8b2640 1 -- 172.21.15.29:0/2591198170 shutdown_connections 2024-04-25T00:31:52.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.756+0000 7f369e8b2640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f36840678f0 0x7f3684069db0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.756+0000 7f369e8b2640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369813c1d0 0x7f369813c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.756+0000 7f369e8b2640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3698156b40 0x7f369815f0f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.756+0000 7f369e8b2640 1 --2- 172.21.15.29:0/2591198170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3698156760 0x7f369815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.756+0000 7f369e8b2640 1 -- 172.21.15.29:0/2591198170 >> 172.21.15.29:0/2591198170 conn(0x7f36980b1870 msgr2=0x7f369815c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.756+0000 7f369e8b2640 1 -- 172.21.15.29:0/2591198170 shutdown_connections 2024-04-25T00:31:52.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:52.756+0000 7f369e8b2640 1 -- 172.21.15.29:0/2591198170 wait complete. 2024-04-25T00:31:52.774 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:31:52.774 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:31:52.774 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-25T00:31:52.774 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:31:52.774 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:31:52.774 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:31:52.774 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:31:52.774 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:31:52.774 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:31:52.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:52.775 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-25T00:31:52.782 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:31:52.782 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:31:52.782 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:31:52.782 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:52.782 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:31:52.782 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:52.782 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:31:52.782 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:52.782 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-25T00:31:52.782 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42442sec preferred_lft 42442sec 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:9c:8d:4b:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:31:52.783 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:52.784 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:31:52.784 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:52.784 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:31:52.784 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:31:52.784 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr flush dev ceph-brx 2024-04-25T00:31:52.784 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set ceph-brx up 2024-04-25T00:31:52.784 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:31:52.784 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:31:52.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:52.904 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:52.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:52.910 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:31:52.922 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:31:52.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:52.923 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-25T00:31:53.007 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-25T00:31:53.007 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:31:53.008 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:31:53.008 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:31:53.008 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:31:53.008 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:31:53.008 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:31:53.008 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:31:53.008 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:31:53.008 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:31:53.008 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:31:53.008 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:53.009 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:31:53.009 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:31:53.009 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:31:53.009 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:31:53.009 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:31:53.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:53.127 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:53.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:53.132 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:31:53.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:53.181 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-04-25T00:31:53.229 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:53.229 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:31:53.229 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:53.229 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T00:31:53.229 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:31:53.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:53.341 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:53.348 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T00:31:53.349 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:53.349 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:31:53.349 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T00:31:53.349 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:31:53.349 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T00:31:53.349 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T00:31:53.349 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T00:31:53.349 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:31:53.394 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:53.445 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:53.451 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:53.452 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:31:53.452 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-25T00:31:53.452 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:31:53.452 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:31:53.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:53.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:53.560 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T00:31:53.560 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:31:53.560 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:31:53.612 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:31:53.613 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:31:53.613 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:31:53.660 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:31:53.719 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:31:53.765 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:31:53.765 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:31:53.766 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:31:53.766 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:31:53.766 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:31:53.766 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:31:53.766 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:31:53.766 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:31:53.766 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:31:53.766 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:31:53.766 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:31:53.766 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:31:53.766 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:31:53.766 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:31:53.766 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:31:53.767 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:53.768 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:53.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:53.769 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:31:53.817 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:31:53.817 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T00:31:53.861 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:31:53.875 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:31:53.924 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:31:53.925 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:31:53.925 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:31:53.925 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:31:53.925 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:31:53.925 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:31:53.925 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:31:53.925 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:31:53.925 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:31:53.925 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:31:53.925 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:31:53.925 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:31:53.925 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:31:53.926 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:31:53.927 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:31:53.927 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:31:53.927 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:53.927 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:53.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:53.927 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:31:53.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-04-25T00:31:53.940+0000 7f22646ae580 -1 init, newargv = 0x561d0220ae10 newargc=15 2024-04-25T00:31:53.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[33387]: starting ceph client 2024-04-25T00:31:53.950 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[33387]: starting fuse 2024-04-25T00:31:55.046 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:31:55.062 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:31:55.109 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:31:55.109 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:31:55.110 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:31:55.110 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:31:55.110 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:31:55.110 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:31:55.110 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:31:55.110 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:31:55.110 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:31:55.110 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:31:55.110 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:31:55.110 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:31:55.110 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:31:55.110 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:31:55.110 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:31:55.111 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:55.112 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:55.112 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:31:55.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:55.113 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:31:55.160 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:31:55.161 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:31:55.161 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:31:55.161 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-25T00:31:55.161 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-25T00:31:55.161 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-25T00:31:55.161 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-25T00:31:55.161 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:31:55.161 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-25T00:31:55.161 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T00:31:55.161 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-25T00:31:55.161 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:31:55.162 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:31:55.163 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-04-25T00:31:55.163 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-25T00:31:55.285 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.33387.asok 2024-04-25T00:31:55.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:55.297 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33387.asok 2024-04-25T00:31:55.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:55.298 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33387.asok status 2024-04-25T00:31:55.447 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:31:55.447 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-25T00:31:55.448 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:31:55.448 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:31:55.448 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-04-25T00:31:55.448 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-25T00:31:55.448 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T00:31:55.448 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "33387", 2024-04-25T00:31:55.449 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-25T00:31:55.449 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:31:55.449 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-25T00:31:55.449 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-25T00:31:55.449 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4651, 2024-04-25T00:31:55.449 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-25T00:31:55.450 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-25T00:31:55.450 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-25T00:31:55.450 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 4651 2024-04-25T00:31:55.450 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:31:55.450 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:31:55.451 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:31:55.451 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:31:55.451 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2399822661 2024-04-25T00:31:55.451 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-25T00:31:55.451 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:31:55.451 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:31:55.451 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:31:55.452 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:31:55.452 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2399822661 2024-04-25T00:31:55.452 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:31:55.452 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.4651 192.168.0.1:0/2399822661", 2024-04-25T00:31:55.452 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/2399822661", 2024-04-25T00:31:55.452 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-25T00:31:55.452 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 6, 2024-04-25T00:31:55.452 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 20, 2024-04-25T00:31:55.452 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:31:55.452 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-25T00:31:55.452 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-25T00:31:55.452 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:31:55.461 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:31:55.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.592+0000 7fc892312640 1 Processor -- start 2024-04-25T00:31:55.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc892312640 1 -- start start 2024-04-25T00:31:55.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc892312640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc88c0b3140 0x7fc88c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:55.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc892312640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c0b4b10 0x7fc88c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:55.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc892312640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc88c0b3a60 0x7fc88c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:55.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc892312640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc88c0b41e0 con 0x7fc88c0b3140 2024-04-25T00:31:55.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc892312640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc88c06d960 con 0x7fc88c0b3a60 2024-04-25T00:31:55.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc892312640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc88c06dc40 con 0x7fc88c0b4b10 2024-04-25T00:31:55.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc88bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc88c0b3140 0x7fc88c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:55.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc88bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc88c0b3140 0x7fc88c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47476/0 (socket says 172.21.15.29:47476) 2024-04-25T00:31:55.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc88bfff640 1 -- 172.21.15.29:0/1903805738 learned_addr learned my addr 172.21.15.29:0/1903805738 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:55.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc8908af640 1 --2- 172.21.15.29:0/1903805738 >> v2:172.21.15.157:3300/0 conn(0x7fc88c0b3a60 0x7fc88c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:55.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc88b7fe640 1 --2- 172.21.15.29:0/1903805738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c0b4b10 0x7fc88c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:55.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc88bfff640 1 -- 172.21.15.29:0/1903805738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c0b4b10 msgr2=0x7fc88c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:55.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc88bfff640 1 --2- 172.21.15.29:0/1903805738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c0b4b10 0x7fc88c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:55.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc88bfff640 1 -- 172.21.15.29:0/1903805738 >> v2:172.21.15.157:3300/0 conn(0x7fc88c0b3a60 msgr2=0x7fc88c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:55.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc88bfff640 1 --2- 172.21.15.29:0/1903805738 >> v2:172.21.15.157:3300/0 conn(0x7fc88c0b3a60 0x7fc88c15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:55.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc88bfff640 1 -- 172.21.15.29:0/1903805738 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c06df20 con 0x7fc88c0b3140 2024-04-25T00:31:55.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc88bfff640 1 --2- 172.21.15.29:0/1903805738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc88c0b3140 0x7fc88c06e440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0b4790 tx=0x7fc88001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6df51c123673427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:55.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.596+0000 7fc88affd640 1 -- 172.21.15.29:0/1903805738 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8800b0070 con 0x7fc88c0b3140 2024-04-25T00:31:55.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc88affd640 1 -- 172.21.15.29:0/1903805738 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8800ab600 con 0x7fc88c0b3140 2024-04-25T00:31:55.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc88affd640 1 -- 172.21.15.29:0/1903805738 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8800ab9d0 con 0x7fc88c0b3140 2024-04-25T00:31:55.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 -- 172.21.15.29:0/1903805738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc88c0b3140 msgr2=0x7fc88c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:55.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 --2- 172.21.15.29:0/1903805738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc88c0b3140 0x7fc88c06e440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0b4790 tx=0x7fc88001c520 comp rx=0 tx=0).stop 2024-04-25T00:31:55.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 -- 172.21.15.29:0/1903805738 shutdown_connections 2024-04-25T00:31:55.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 --2- 172.21.15.29:0/1903805738 >> v2:172.21.15.157:3300/0 conn(0x7fc88c0b3a60 0x7fc88c15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:55.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 --2- 172.21.15.29:0/1903805738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c0b4b10 0x7fc88c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:55.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 --2- 172.21.15.29:0/1903805738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc88c0b3140 0x7fc88c06e440 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:55.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 -- 172.21.15.29:0/1903805738 >> 172.21.15.29:0/1903805738 conn(0x7fc88c0b1870 msgr2=0x7fc88c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:55.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 -- 172.21.15.29:0/1903805738 shutdown_connections 2024-04-25T00:31:55.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 -- 172.21.15.29:0/1903805738 wait complete. 2024-04-25T00:31:55.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 Processor -- start 2024-04-25T00:31:55.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 -- start start 2024-04-25T00:31:55.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc88c0b3140 0x7fc88c13ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:55.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc88c0b3a60 0x7fc88c071150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:55.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c0b4b10 0x7fc88c15d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:55.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c0ce3c0 con 0x7fc88c0b3a60 2024-04-25T00:31:55.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c0ce240 con 0x7fc88c0b3140 2024-04-25T00:31:55.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc892312640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc88c0ce540 con 0x7fc88c0b4b10 2024-04-25T00:31:55.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc88b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc88c0b3a60 0x7fc88c071150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:55.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc88b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc88c0b3a60 0x7fc88c071150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47478/0 (socket says 172.21.15.29:47478) 2024-04-25T00:31:55.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc88b7fe640 1 -- 172.21.15.29:0/1877599443 learned_addr learned my addr 172.21.15.29:0/1877599443 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:55.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc8908af640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c0b4b10 0x7fc88c15d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:55.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc88bfff640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc88c0b3140 0x7fc88c13ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:55.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc88b7fe640 1 -- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c0b4b10 msgr2=0x7fc88c15d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:55.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc88b7fe640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c0b4b10 0x7fc88c15d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:55.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc88b7fe640 1 -- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc88c0b3140 msgr2=0x7fc88c13ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:55.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc88b7fe640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc88c0b3140 0x7fc88c13ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:55.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc88b7fe640 1 -- 172.21.15.29:0/1877599443 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c13d1f0 con 0x7fc88c0b3a60 2024-04-25T00:31:55.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.600+0000 7fc88b7fe640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc88c0b3a60 0x7fc88c071150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00d5f0 tx=0x7fc87c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:55.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.604+0000 7fc888ff9640 1 -- 172.21.15.29:0/1877599443 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c0125f0 con 0x7fc88c0b3a60 2024-04-25T00:31:55.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.604+0000 7fc892312640 1 -- 172.21.15.29:0/1877599443 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88c071bb0 con 0x7fc88c0b3a60 2024-04-25T00:31:55.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.604+0000 7fc888ff9640 1 -- 172.21.15.29:0/1877599443 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc87c019070 con 0x7fc88c0b3a60 2024-04-25T00:31:55.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.604+0000 7fc888ff9640 1 -- 172.21.15.29:0/1877599443 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c0129d0 con 0x7fc88c0b3a60 2024-04-25T00:31:55.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.604+0000 7fc892312640 1 -- 172.21.15.29:0/1877599443 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc88c072170 con 0x7fc88c0b3a60 2024-04-25T00:31:55.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.604+0000 7fc892312640 1 -- 172.21.15.29:0/1877599443 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc88c069270 con 0x7fc88c0b3a60 2024-04-25T00:31:55.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.604+0000 7fc888ff9640 1 -- 172.21.15.29:0/1877599443 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc87c012b70 con 0x7fc88c0b3a60 2024-04-25T00:31:55.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.604+0000 7fc888ff9640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc8780678c0 0x7fc878069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:55.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.604+0000 7fc888ff9640 1 -- 172.21.15.29:0/1877599443 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc87c086020 con 0x7fc88c0b3a60 2024-04-25T00:31:55.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.604+0000 7fc88bfff640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc8780678c0 0x7fc878069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:55.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.604+0000 7fc88bfff640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc8780678c0 0x7fc878069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc8800ad000 tx=0x7fc8800a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:55.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.608+0000 7fc888ff9640 1 -- 172.21.15.29:0/1877599443 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc87c056490 con 0x7fc88c0b3a60 2024-04-25T00:31:55.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc892312640 1 -- 172.21.15.29:0/1877599443 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc88c06eda0 con 0x7fc88c0b3a60 2024-04-25T00:31:55.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc888ff9640 1 -- 172.21.15.29:0/1877599443 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fc87c05a790 con 0x7fc88c0b3a60 2024-04-25T00:31:55.810 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:31:55.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc892312640 1 -- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc8780678c0 msgr2=0x7fc878069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:55.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc892312640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc8780678c0 0x7fc878069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc8800ad000 tx=0x7fc8800a6000 comp rx=0 tx=0).stop 2024-04-25T00:31:55.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc892312640 1 -- 172.21.15.29:0/1877599443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc88c0b3a60 msgr2=0x7fc88c071150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:55.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc892312640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc88c0b3a60 0x7fc88c071150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00d5f0 tx=0x7fc87c00dab0 comp rx=0 tx=0).stop 2024-04-25T00:31:55.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc892312640 1 -- 172.21.15.29:0/1877599443 shutdown_connections 2024-04-25T00:31:55.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc892312640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c0b4b10 0x7fc88c15d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:55.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc892312640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc88c0b3a60 0x7fc88c071150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:55.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc892312640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc8780678c0 0x7fc878069d80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:55.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc892312640 1 --2- 172.21.15.29:0/1877599443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc88c0b3140 0x7fc88c13ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:55.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc892312640 1 -- 172.21.15.29:0/1877599443 >> 172.21.15.29:0/1877599443 conn(0x7fc88c0b1870 msgr2=0x7fc88c142d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:55.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc892312640 1 -- 172.21.15.29:0/1877599443 shutdown_connections 2024-04-25T00:31:55.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:55.808+0000 7fc892312640 1 -- 172.21.15.29:0/1877599443 wait complete. 2024-04-25T00:31:55.829 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:31:55.829 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:31:55.829 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-25T00:31:55.829 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T00:31:55.829 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T00:31:55.829 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:31:55.829 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:31:55.830 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:31:55.830 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T00:31:55.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:55.830 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-25T00:31:55.837 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:31:55.837 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:31:55.837 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:31:55.837 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:55.837 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:31:55.837 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:55.837 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:31:55.837 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:55.838 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-25T00:31:55.838 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:31:55.838 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:55.838 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-25T00:31:55.838 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:31:55.838 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:55.838 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:31:55.838 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:55.838 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:31:55.838 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42439sec preferred_lft 42439sec 2024-04-25T00:31:55.838 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-25T00:31:55.838 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:55.838 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:31:55.839 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:9c:8d:4b:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:55.839 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:31:55.839 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:55.839 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:31:55.839 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:31:55.839 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:31:55.839 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:55.839 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:31:55.839 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:55.839 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:31:55.839 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:55.839 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:31:55.839 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:55.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:55.840 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:31:55.891 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:31:55.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:55.893 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-25T00:31:55.945 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-25T00:31:55.945 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:31:55.946 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:31:55.946 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:31:55.946 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:31:55.946 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:31:55.946 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:31:55.946 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:31:55.946 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:31:55.946 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:31:55.946 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:31:55.946 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:55.947 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:31:55.947 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:31:55.947 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:31:55.947 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:31:55.947 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:31:56.034 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:56.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:56.070 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:31:56.075 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:31:56.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:56.076 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-04-25T00:31:56.124 INFO:teuthology.orchestra.run.smithi157.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T00:31:56.125 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:56.125 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:31:56.125 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:31:56.125 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T00:31:56.125 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:31:56.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:56.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:56.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:56.246 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T00:31:56.257 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:31:56.257 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:31:56.258 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:31:56.258 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:56.258 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:31:56.258 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:56.258 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:31:56.258 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 1a:d7:d1:86:55:37 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T00:31:56.258 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T00:31:56.258 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:56.258 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::18d7:d1ff:fe86:5537/64 scope link 2024-04-25T00:31:56.258 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:56.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:56.259 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T00:31:56.316 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:31:56.316 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:31:56.316 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:31:56.317 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:56.317 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:31:56.317 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:56.317 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:31:56.317 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 1a:d7:d1:86:55:37 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T00:31:56.317 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T00:31:56.317 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:56.317 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::18d7:d1ff:fe86:5537/64 scope link 2024-04-25T00:31:56.317 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:31:56.318 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T00:31:56.319 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:56.319 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:31:56.319 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T00:31:56.319 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T00:31:56.319 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T00:31:56.319 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T00:31:56.319 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T00:31:56.319 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:31:56.402 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:56.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:56.456 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:31:56.457 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:31:56.457 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.1 up 2024-04-25T00:31:56.457 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T00:31:56.457 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:31:56.539 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:31:56.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:56.562 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T00:31:56.562 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:31:56.563 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T00:31:56.608 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T00:31:56.608 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:31:56.609 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T00:31:56.656 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:31:56.712 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:31:56.761 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:31:56.762 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:31:56.762 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:31:56.762 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:31:56.762 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:31:56.762 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:31:56.762 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:31:56.762 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:31:56.762 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:31:56.762 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:31:56.762 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:31:56.762 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:31:56.762 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:31:56.763 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:31:56.764 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:31:56.764 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:56.764 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:56.764 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:31:56.764 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:31:56.764 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:31:56.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:56.765 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:31:56.812 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:31:56.813 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T00:31:56.813 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T00:31:56.858 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:31:56.873 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:31:56.925 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:31:56.926 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:31:56.926 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:31:56.926 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:31:56.926 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:31:56.926 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:31:56.926 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:31:56.926 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:31:56.926 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:31:56.926 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:31:56.926 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:31:56.926 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:31:56.927 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:31:56.927 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:31:56.927 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:31:56.927 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:31:56.927 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:31:56.927 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:31:56.927 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:31:56.927 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:31:56.927 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:31:56.927 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:31:56.927 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:31:56.928 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:31:56.928 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:31:56.928 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:31:56.928 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:31:56.928 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:31:56.928 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:31:56.928 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:56.928 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:56.928 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:31:56.928 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:31:56.928 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:31:56.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:56.929 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:31:56.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi157.stderr:2024-04-25T00:31:56.940+0000 7f036ac6f580 -1 init, newargv = 0x55aac64af070 newargc=15 2024-04-25T00:31:56.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi157.stderr:ceph-fuse[33506]: starting ceph client 2024-04-25T00:31:56.945 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:31:56.948 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi157.stderr:ceph-fuse[33506]: starting fuse 2024-04-25T00:31:58.047 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:31:58.063 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:31:58.115 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:31:58.115 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:31:58.115 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:31:58.115 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:31:58.115 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:31:58.115 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:31:58.115 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:31:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:31:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:31:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:31:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:31:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:31:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:31:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:31:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:31:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:31:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:31:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:31:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:31:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:31:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:31:58.118 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.3 2024-04-25T00:31:58.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:58.119 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:31:58.169 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:31:58.169 INFO:teuthology.orchestra.run.smithi157.stdout:46 2024-04-25T00:31:58.169 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-25T00:31:58.170 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-25T00:31:58.171 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-25T00:31:58.171 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:31:58.171 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-25T00:31:58.171 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:31:58.171 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:31:58.171 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:31:58.171 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-25T00:31:58.171 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-25T00:31:58.171 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-25T00:31:58.171 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:31:58.171 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:31:58.172 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.3")) 2024-04-25T00:31:58.172 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-25T00:31:58.293 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.3.33506.asok 2024-04-25T00:31:58.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:31:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:31:58.306 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33506.asok 2024-04-25T00:31:58.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:58.306 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33506.asok status 2024-04-25T00:31:58.459 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:31:58.459 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-25T00:31:58.459 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:31:58.460 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:31:58.460 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "3", 2024-04-25T00:31:58.460 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-25T00:31:58.460 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T00:31:58.460 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "33506", 2024-04-25T00:31:58.460 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-25T00:31:58.460 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:31:58.460 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-25T00:31:58.460 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-25T00:31:58.460 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4514, 2024-04-25T00:31:58.460 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-25T00:31:58.460 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-25T00:31:58.460 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 4514 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.2:0", 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1937461157 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.2:0", 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1937461157 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.4514 192.168.0.2:0/1937461157", 2024-04-25T00:31:58.461 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.2:0/1937461157", 2024-04-25T00:31:58.462 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-25T00:31:58.462 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 6, 2024-04-25T00:31:58.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 20, 2024-04-25T00:31:58.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:31:58.462 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-25T00:31:58.462 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-25T00:31:58.462 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:31:58.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:58.474 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-04-25T00:31:58.474 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:31:58.481 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-04-25T00:31:58.482 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:31:58.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:58.482 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:31:58.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:58.543 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-04-25T00:31:58.543 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T00:31:58.590 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-04-25T00:31:58.591 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T00:31:58.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:58.591 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T00:31:58.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:58.652 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-25T00:31:58.652 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T00:31:58.662 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-25T00:31:58.662 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T00:31:58.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:58.663 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:31:58.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:58.717 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-25T00:31:58.717 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T00:31:58.766 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-25T00:31:58.766 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T00:31:58.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:58.766 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T00:31:58.821 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T00:31:58.843 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:31:58.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8441471640 1 Processor -- start 2024-04-25T00:31:58.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8441471640 1 -- start start 2024-04-25T00:31:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8441471640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c06e440 0x7f843c0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8441471640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f843c0b41e0 con 0x7f843c0b3140 2024-04-25T00:31:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8441471640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f843c06d960 con 0x7f843c0b3c30 2024-04-25T00:31:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8441471640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f843c0b5050 con 0x7f843c06e440 2024-04-25T00:31:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f843affd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f843c0b3140 0x7f843c0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39558/0 (socket says 172.21.15.29:39558) 2024-04-25T00:31:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f843affd640 1 -- 172.21.15.29:0/146232162 learned_addr learned my addr 172.21.15.29:0/146232162 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f843b7fe640 1 --2- 172.21.15.29:0/146232162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c06e440 0x7f843c0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 501128664 0 0) 0x7f843c0b41e0 con 0x7f843c0b3140 2024-04-25T00:31:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8424003530 con 0x7f843c0b3140 2024-04-25T00:31:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3641278617 0 0) 0x7f8424003530 con 0x7f843c0b3140 2024-04-25T00:31:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c06e440 msgr2=0x7f843c0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 --2- 172.21.15.29:0/146232162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c06e440 0x7f843c0b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 >> v1:172.21.15.157:6789/0 conn(0x7f843c0b3c30 legacy=0x7f843c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843c06dc40 con 0x7f843c0b3140 2024-04-25T00:31:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f842c001680 con 0x7f843c0b3140 2024-04-25T00:31:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 >> v1:172.21.15.29:6789/0 conn(0x7f843c0b3140 legacy=0x7f843c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:58.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 --2- 172.21.15.29:0/146232162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c06e440 0x7f84240039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:58.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 --2- 172.21.15.29:0/146232162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8424003f70 0x7f8424004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:58.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 --2- 172.21.15.29:0/146232162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8424004d50 0x7f84240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:58.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f843c0b5050 con 0x7f8424004d50 2024-04-25T00:31:58.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f842400bab0 con 0x7f8424003f70 2024-04-25T00:31:58.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f842400bcb0 con 0x7f843c06e440 2024-04-25T00:31:58.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f843b7fe640 1 --2- 172.21.15.29:0/146232162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8424004d50 0x7f84240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:58.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f843a7fc640 1 --2- 172.21.15.29:0/146232162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8424003f70 0x7f8424004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:58.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f843b7fe640 1 -- 172.21.15.29:0/146232162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c06e440 msgr2=0x7f84240039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:58.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f843affd640 1 --2- 172.21.15.29:0/146232162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c06e440 0x7f84240039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:58.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f843b7fe640 1 --2- 172.21.15.29:0/146232162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c06e440 0x7f84240039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:58.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f843b7fe640 1 -- 172.21.15.29:0/146232162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8424003f70 msgr2=0x7f8424004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:58.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f843b7fe640 1 --2- 172.21.15.29:0/146232162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8424003f70 0x7f8424004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:58.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f843b7fe640 1 -- 172.21.15.29:0/146232162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8424003530 con 0x7f8424004d50 2024-04-25T00:31:58.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.976+0000 7f843affd640 1 --2- 172.21.15.29:0/146232162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c06e440 0x7f84240039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:58.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.980+0000 7f843b7fe640 1 --2- 172.21.15.29:0/146232162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8424004d50 0x7f84240051d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8434009fa0 tx=0x7f843401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d0438a56c5259a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:58.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.980+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84340b2070 con 0x7f8424004d50 2024-04-25T00:31:58.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.980+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84340a5d10 con 0x7f8424004d50 2024-04-25T00:31:58.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.980+0000 7f8439ffb640 1 -- 172.21.15.29:0/146232162 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434009bf0 con 0x7f8424004d50 2024-04-25T00:31:58.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.980+0000 7f8441471640 1 -- 172.21.15.29:0/146232162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8424004d50 msgr2=0x7f84240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:58.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.980+0000 7f8441471640 1 --2- 172.21.15.29:0/146232162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8424004d50 0x7f84240051d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8434009fa0 tx=0x7f843401cc40 comp rx=0 tx=0).stop 2024-04-25T00:31:58.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.980+0000 7f843b7fe640 1 -- 172.21.15.29:0/146232162 reap_dead start 2024-04-25T00:31:58.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.980+0000 7f8441471640 1 -- 172.21.15.29:0/146232162 shutdown_connections 2024-04-25T00:31:58.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.980+0000 7f8441471640 1 -- 172.21.15.29:0/146232162 >> 172.21.15.29:0/146232162 conn(0x7f843c0b1870 msgr2=0x7f843c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:58.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.980+0000 7f8441471640 1 -- 172.21.15.29:0/146232162 shutdown_connections 2024-04-25T00:31:58.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.980+0000 7f8441471640 1 -- 172.21.15.29:0/146232162 wait complete. 2024-04-25T00:31:58.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.980+0000 7f8441471640 1 Processor -- start 2024-04-25T00:31:58.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.980+0000 7f8441471640 1 -- start start 2024-04-25T00:31:58.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f8441471640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c13c510 0x7f843c156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:58.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f8441471640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f843c13c8f0 0x7f843c13f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:58.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f8441471640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c1f2510 0x7f843c141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:58.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f8441471640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f843c145a90 con 0x7f843c13c8f0 2024-04-25T00:31:58.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f8441471640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f843c145910 con 0x7f843c1f2510 2024-04-25T00:31:58.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f8441471640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f843c0b4b10 con 0x7f843c13c510 2024-04-25T00:31:58.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f843a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f843c13c8f0 0x7f843c13f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:58.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f843a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f843c13c8f0 0x7f843c13f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41048/0 (socket says 172.21.15.29:41048) 2024-04-25T00:31:58.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f843a7fc640 1 -- 172.21.15.29:0/93575970 learned_addr learned my addr 172.21.15.29:0/93575970 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:58.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f843affd640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c13c510 0x7f843c156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:58.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f843b7fe640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c1f2510 0x7f843c141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:58.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f843a7fc640 1 -- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c13c510 msgr2=0x7f843c156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:58.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f843a7fc640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c13c510 0x7f843c156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:58.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f843a7fc640 1 -- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c1f2510 msgr2=0x7f843c141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:58.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f843a7fc640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c1f2510 0x7f843c141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:58.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f843a7fc640 1 -- 172.21.15.29:0/93575970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843c140330 con 0x7f843c13c8f0 2024-04-25T00:31:58.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f843b7fe640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c1f2510 0x7f843c141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:58.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f843a7fc640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f843c13c8f0 0x7f843c13f9f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f843000e550 tx=0x7f843000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:58.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f84237fe640 1 -- 172.21.15.29:0/93575970 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84300052e0 con 0x7f843c13c8f0 2024-04-25T00:31:58.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f84237fe640 1 -- 172.21.15.29:0/93575970 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8430018070 con 0x7f843c13c8f0 2024-04-25T00:31:58.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f8441471640 1 -- 172.21.15.29:0/93575970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843c13d5a0 con 0x7f843c13c8f0 2024-04-25T00:31:58.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f84237fe640 1 -- 172.21.15.29:0/93575970 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430003820 con 0x7f843c13c8f0 2024-04-25T00:31:58.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f8441471640 1 -- 172.21.15.29:0/93575970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843c13db60 con 0x7f843c13c8f0 2024-04-25T00:31:58.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f8441471640 1 -- 172.21.15.29:0/93575970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843c069270 con 0x7f843c13c8f0 2024-04-25T00:31:58.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f84237fe640 1 -- 172.21.15.29:0/93575970 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f84300039c0 con 0x7f843c13c8f0 2024-04-25T00:31:58.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f84237fe640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84280678f0 0x7f8428069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:58.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f84237fe640 1 -- 172.21.15.29:0/93575970 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8430085950 con 0x7f843c13c8f0 2024-04-25T00:31:58.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.984+0000 7f843affd640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84280678f0 0x7f8428069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:58.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.988+0000 7f843affd640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84280678f0 0x7f8428069db0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f842c001060 tx=0x7f842c000ff0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:58.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:58.988+0000 7f84237fe640 1 -- 172.21.15.29:0/93575970 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8430056490 con 0x7f843c13c8f0 2024-04-25T00:31:59.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.152+0000 7f8441471640 1 -- 172.21.15.29:0/93575970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f843c15b980 con 0x7f843c13c8f0 2024-04-25T00:31:59.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.156+0000 7f84237fe640 1 -- 172.21.15.29:0/93575970 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f8430005480 con 0x7f843c13c8f0 2024-04-25T00:31:59.160 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:31:59.160 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:31:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.160+0000 7f8441471640 1 -- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84280678f0 msgr2=0x7f8428069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.160+0000 7f8441471640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84280678f0 0x7f8428069db0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f842c001060 tx=0x7f842c000ff0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.160+0000 7f8441471640 1 -- 172.21.15.29:0/93575970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f843c13c8f0 msgr2=0x7f843c13f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.160+0000 7f8441471640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f843c13c8f0 0x7f843c13f9f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f843000e550 tx=0x7f843000ea10 comp rx=0 tx=0).stop 2024-04-25T00:31:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.160+0000 7f8441471640 1 -- 172.21.15.29:0/93575970 shutdown_connections 2024-04-25T00:31:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.160+0000 7f8441471640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84280678f0 0x7f8428069db0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.160+0000 7f8441471640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843c1f2510 0x7f843c141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.160+0000 7f8441471640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f843c13c8f0 0x7f843c13f9f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.160+0000 7f8441471640 1 --2- 172.21.15.29:0/93575970 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f843c13c510 0x7f843c156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.160+0000 7f8441471640 1 -- 172.21.15.29:0/93575970 >> 172.21.15.29:0/93575970 conn(0x7f843c0b1870 msgr2=0x7f843c0cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:59.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.160+0000 7f8441471640 1 -- 172.21.15.29:0/93575970 shutdown_connections 2024-04-25T00:31:59.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.160+0000 7f8441471640 1 -- 172.21.15.29:0/93575970 wait complete. 2024-04-25T00:31:59.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:31:59.179 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:31:59.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f2bd4640 1 Processor -- start 2024-04-25T00:31:59.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f2bd4640 1 -- start start 2024-04-25T00:31:59.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f2bd4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35ec166d70 0x7f35ec06b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f2bd4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ec0d0a60 con 0x7f35ec0728b0 2024-04-25T00:31:59.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f2bd4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ec0d01c0 con 0x7f35ec162640 2024-04-25T00:31:59.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f2bd4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f35ec0cff90 con 0x7f35ec166d70 2024-04-25T00:31:59.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35ebfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f35ec0728b0 0x7f35ec166660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39574/0 (socket says 172.21.15.29:39574) 2024-04-25T00:31:59.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35ebfff640 1 -- 172.21.15.29:0/612849960 learned_addr learned my addr 172.21.15.29:0/612849960 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:59.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f1171640 1 --2- 172.21.15.29:0/612849960 >> v2:172.21.15.157:3300/0 conn(0x7f35ec166d70 0x7f35ec06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1467172825 0 0) 0x7f35ec0d0a60 con 0x7f35ec0728b0 2024-04-25T00:31:59.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35d4003530 con 0x7f35ec0728b0 2024-04-25T00:31:59.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2617083524 0 0) 0x7f35d4003530 con 0x7f35ec0728b0 2024-04-25T00:31:59.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 >> v2:172.21.15.157:3300/0 conn(0x7f35ec166d70 msgr2=0x7f35ec06b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:59.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 --2- 172.21.15.29:0/612849960 >> v2:172.21.15.157:3300/0 conn(0x7f35ec166d70 0x7f35ec06b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 >> v1:172.21.15.157:6789/0 conn(0x7f35ec162640 legacy=0x7f35ec06db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec0d04a0 con 0x7f35ec0728b0 2024-04-25T00:31:59.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f35e0001a00 con 0x7f35ec0728b0 2024-04-25T00:31:59.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 >> v1:172.21.15.29:6789/0 conn(0x7f35ec0728b0 legacy=0x7f35ec166660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 --2- 172.21.15.29:0/612849960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d40039c0 0x7f35d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 --2- 172.21.15.29:0/612849960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35d4004810 0x7f35d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 --2- 172.21.15.29:0/612849960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d40051d0 0x7f35d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f35d400c140 con 0x7f35d40039c0 2024-04-25T00:31:59.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35d400c2f0 con 0x7f35d40051d0 2024-04-25T00:31:59.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f35d400c4f0 con 0x7f35d4004810 2024-04-25T00:31:59.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f0970640 1 --2- 172.21.15.29:0/612849960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d40039c0 0x7f35d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f1171640 1 --2- 172.21.15.29:0/612849960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d40051d0 0x7f35d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35ebfff640 1 --2- 172.21.15.29:0/612849960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35d4004810 0x7f35d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f0970640 1 -- 172.21.15.29:0/612849960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35d4004810 msgr2=0x7f35d4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:59.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f0970640 1 --2- 172.21.15.29:0/612849960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35d4004810 0x7f35d4004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f0970640 1 -- 172.21.15.29:0/612849960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d40051d0 msgr2=0x7f35d400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:59.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f0970640 1 --2- 172.21.15.29:0/612849960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d40051d0 0x7f35d400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f0970640 1 -- 172.21.15.29:0/612849960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d4003530 con 0x7f35d40039c0 2024-04-25T00:31:59.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f1171640 1 --2- 172.21.15.29:0/612849960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d40051d0 0x7f35d400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:59.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.308+0000 7f35f1171640 1 -- 172.21.15.29:0/612849960 reap_dead start 2024-04-25T00:31:59.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.312+0000 7f35f0970640 1 --2- 172.21.15.29:0/612849960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d40039c0 0x7f35d4003de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f35e4001780 tx=0x7f35e401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe91990e176c80f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:59.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.312+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e40013a0 con 0x7f35d40039c0 2024-04-25T00:31:59.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.312+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e40052f0 con 0x7f35d40039c0 2024-04-25T00:31:59.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.312+0000 7f35eb7fe640 1 -- 172.21.15.29:0/612849960 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4003840 con 0x7f35d40039c0 2024-04-25T00:31:59.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.312+0000 7f35f2bd4640 1 -- 172.21.15.29:0/612849960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d40039c0 msgr2=0x7f35d4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.312+0000 7f35f2bd4640 1 --2- 172.21.15.29:0/612849960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d40039c0 0x7f35d4003de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f35e4001780 tx=0x7f35e401c400 comp rx=0 tx=0).stop 2024-04-25T00:31:59.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.312+0000 7f35f2bd4640 1 -- 172.21.15.29:0/612849960 shutdown_connections 2024-04-25T00:31:59.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.312+0000 7f35f2bd4640 1 --2- 172.21.15.29:0/612849960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d40039c0 0x7f35d4003de0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.312+0000 7f35f2bd4640 1 -- 172.21.15.29:0/612849960 >> 172.21.15.29:0/612849960 conn(0x7f35ec0b2920 msgr2=0x7f35ec0b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:59.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.312+0000 7f35f2bd4640 1 -- 172.21.15.29:0/612849960 shutdown_connections 2024-04-25T00:31:59.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.312+0000 7f35f2bd4640 1 -- 172.21.15.29:0/612849960 wait complete. 2024-04-25T00:31:59.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.312+0000 7f35f2bd4640 1 Processor -- start 2024-04-25T00:31:59.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.312+0000 7f35f2bd4640 1 -- start start 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35f2bd4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35d40039c0 0x7f35ec0b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35f2bd4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ec161350 0x7f35ec157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35f2bd4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ec0b5c70 0x7f35ec0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35f2bd4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec06b390 con 0x7f35ec161350 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35f2bd4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec06b210 con 0x7f35ec0b5c70 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35f2bd4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f35ec06b510 con 0x7f35d40039c0 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35ebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ec161350 0x7f35ec157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35ebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ec161350 0x7f35ec157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41064/0 (socket says 172.21.15.29:41064) 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35ebfff640 1 -- 172.21.15.29:0/4089965756 learned_addr learned my addr 172.21.15.29:0/4089965756 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35f1171640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ec0b5c70 0x7f35ec0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35f0970640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35d40039c0 0x7f35ec0b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35ebfff640 1 -- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35d40039c0 msgr2=0x7f35ec0b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35ebfff640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35d40039c0 0x7f35ec0b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35ebfff640 1 -- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ec0b5c70 msgr2=0x7f35ec0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35ebfff640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ec0b5c70 0x7f35ec0b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35ebfff640 1 -- 172.21.15.29:0/4089965756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec165780 con 0x7f35ec161350 2024-04-25T00:31:59.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35f0970640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35d40039c0 0x7f35ec0b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:59.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35ebfff640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ec161350 0x7f35ec157b70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f35e0002080 tx=0x7f35e000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:59.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35e97fa640 1 -- 172.21.15.29:0/4089965756 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0019070 con 0x7f35ec161350 2024-04-25T00:31:59.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35e97fa640 1 -- 172.21.15.29:0/4089965756 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e0004d70 con 0x7f35ec161350 2024-04-25T00:31:59.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35f2bd4640 1 -- 172.21.15.29:0/4089965756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ec165a60 con 0x7f35ec161350 2024-04-25T00:31:59.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35e97fa640 1 -- 172.21.15.29:0/4089965756 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e00072d0 con 0x7f35ec161350 2024-04-25T00:31:59.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35f2bd4640 1 -- 172.21.15.29:0/4089965756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ec0b4c70 con 0x7f35ec161350 2024-04-25T00:31:59.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35f2bd4640 1 -- 172.21.15.29:0/4089965756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ec069270 con 0x7f35ec161350 2024-04-25T00:31:59.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35e97fa640 1 -- 172.21.15.29:0/4089965756 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f35e001c070 con 0x7f35ec161350 2024-04-25T00:31:59.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35e97fa640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f35d80678f0 0x7f35d8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.316+0000 7f35e97fa640 1 -- 172.21.15.29:0/4089965756 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f35e0085b50 con 0x7f35ec161350 2024-04-25T00:31:59.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.320+0000 7f35f0970640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f35d80678f0 0x7f35d8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.320+0000 7f35f0970640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f35d80678f0 0x7f35d8069db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f35e4001750 tx=0x7f35e40011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:59.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.324+0000 7f35e97fa640 1 -- 172.21.15.29:0/4089965756 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f35e00560d0 con 0x7f35ec161350 2024-04-25T00:31:59.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.484+0000 7f35f2bd4640 1 -- 172.21.15.29:0/4089965756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35ec072460 con 0x7f35ec161350 2024-04-25T00:31:59.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.488+0000 7f35e97fa640 1 -- 172.21.15.29:0/4089965756 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f35e005a3d0 con 0x7f35ec161350 2024-04-25T00:31:59.489 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:31:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.488+0000 7f35f2bd4640 1 -- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f35d80678f0 msgr2=0x7f35d8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.488+0000 7f35f2bd4640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f35d80678f0 0x7f35d8069db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f35e4001750 tx=0x7f35e40011a0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.488+0000 7f35f2bd4640 1 -- 172.21.15.29:0/4089965756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ec161350 msgr2=0x7f35ec157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.488+0000 7f35f2bd4640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ec161350 0x7f35ec157b70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f35e0002080 tx=0x7f35e000b4a0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.488+0000 7f35f2bd4640 1 -- 172.21.15.29:0/4089965756 shutdown_connections 2024-04-25T00:31:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.488+0000 7f35f2bd4640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f35d80678f0 0x7f35d8069db0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.488+0000 7f35f2bd4640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ec0b5c70 0x7f35ec0b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.488+0000 7f35f2bd4640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ec161350 0x7f35ec157b70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.488+0000 7f35f2bd4640 1 --2- 172.21.15.29:0/4089965756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35d40039c0 0x7f35ec0b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.488+0000 7f35f2bd4640 1 -- 172.21.15.29:0/4089965756 >> 172.21.15.29:0/4089965756 conn(0x7f35ec0b2920 msgr2=0x7f35ec0cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:59.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.488+0000 7f35f2bd4640 1 -- 172.21.15.29:0/4089965756 shutdown_connections 2024-04-25T00:31:59.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.488+0000 7f35f2bd4640 1 -- 172.21.15.29:0/4089965756 wait complete. 2024-04-25T00:31:59.509 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:31:59.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa84d6de640 1 Processor -- start 2024-04-25T00:31:59.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa84d6de640 1 -- start start 2024-04-25T00:31:59.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa84d6de640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa84815e1b0 0x7fa8480cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa84d6de640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8480d01d0 con 0x7fa848071fa0 2024-04-25T00:31:59.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa84d6de640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8480d04b0 con 0x7fa8480cb7e0 2024-04-25T00:31:59.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa84d6de640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa8480d0a70 con 0x7fa84815e1b0 2024-04-25T00:31:59.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa846ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa84815e1b0 0x7fa8480cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa8467fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa848071fa0 0x7fa84815ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39586/0 (socket says 172.21.15.29:39586) 2024-04-25T00:31:59.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa8467fc640 1 -- 172.21.15.29:0/58674725 learned_addr learned my addr 172.21.15.29:0/58674725 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:59.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa845ffb640 1 -- 172.21.15.29:0/58674725 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 742454508 0 0) 0x7fa8480d01d0 con 0x7fa848071fa0 2024-04-25T00:31:59.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa845ffb640 1 -- 172.21.15.29:0/58674725 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa834003530 con 0x7fa848071fa0 2024-04-25T00:31:59.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa846ffd640 1 -- 172.21.15.29:0/58674725 >> v1:172.21.15.157:6789/0 conn(0x7fa8480cb7e0 legacy=0x7fa84806f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:59.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa846ffd640 1 -- 172.21.15.29:0/58674725 >> v1:172.21.15.29:6789/0 conn(0x7fa848071fa0 legacy=0x7fa84815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.640+0000 7fa846ffd640 1 -- 172.21.15.29:0/58674725 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8480d0790 con 0x7fa84815e1b0 2024-04-25T00:31:59.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa846ffd640 1 --2- 172.21.15.29:0/58674725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa84815e1b0 0x7fa8480cb2a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa8400097c0 tx=0x7fa84001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d6d5b90978d226 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:59.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa845ffb640 1 -- 172.21.15.29:0/58674725 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8400b0070 con 0x7fa84815e1b0 2024-04-25T00:31:59.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa845ffb640 1 -- 172.21.15.29:0/58674725 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8400ab600 con 0x7fa84815e1b0 2024-04-25T00:31:59.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa845ffb640 1 -- 172.21.15.29:0/58674725 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8400ab9d0 con 0x7fa84815e1b0 2024-04-25T00:31:59.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 -- 172.21.15.29:0/58674725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa84815e1b0 msgr2=0x7fa8480cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 --2- 172.21.15.29:0/58674725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa84815e1b0 0x7fa8480cb2a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa8400097c0 tx=0x7fa84001c460 comp rx=0 tx=0).stop 2024-04-25T00:31:59.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 -- 172.21.15.29:0/58674725 shutdown_connections 2024-04-25T00:31:59.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 --2- 172.21.15.29:0/58674725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa84815e1b0 0x7fa8480cb2a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 -- 172.21.15.29:0/58674725 >> 172.21.15.29:0/58674725 conn(0x7fa8480b2920 msgr2=0x7fa8480b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:59.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 -- 172.21.15.29:0/58674725 shutdown_connections 2024-04-25T00:31:59.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 -- 172.21.15.29:0/58674725 wait complete. 2024-04-25T00:31:59.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 Processor -- start 2024-04-25T00:31:59.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 -- start start 2024-04-25T00:31:59.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa848071fa0 0x7fa848141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8480cb7e0 0x7fa8481402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84815e1b0 0x7fa84813f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa84815e830 con 0x7fa8480cb7e0 2024-04-25T00:31:59.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8480cfed0 con 0x7fa84815e1b0 2024-04-25T00:31:59.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa84d6de640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa84816c460 con 0x7fa848071fa0 2024-04-25T00:31:59.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa8467fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8480cb7e0 0x7fa8481402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa8467fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8480cb7e0 0x7fa8481402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41088/0 (socket says 172.21.15.29:41088) 2024-04-25T00:31:59.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa8467fc640 1 -- 172.21.15.29:0/3624699157 learned_addr learned my addr 172.21.15.29:0/3624699157 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:59.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa846ffd640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa848071fa0 0x7fa848141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa8477fe640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84815e1b0 0x7fa84813f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa8467fc640 1 -- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa848071fa0 msgr2=0x7fa848141bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa8467fc640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa848071fa0 0x7fa848141bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa8467fc640 1 -- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84815e1b0 msgr2=0x7fa84813f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa8467fc640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84815e1b0 0x7fa84813f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa8467fc640 1 -- 172.21.15.29:0/3624699157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa848142110 con 0x7fa8480cb7e0 2024-04-25T00:31:59.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa8467fc640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8480cb7e0 0x7fa8481402e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa83800caa0 tx=0x7fa83800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:59.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.644+0000 7fa83f7fe640 1 -- 172.21.15.29:0/3624699157 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838017070 con 0x7fa8480cb7e0 2024-04-25T00:31:59.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.648+0000 7fa83f7fe640 1 -- 172.21.15.29:0/3624699157 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83800ac00 con 0x7fa8480cb7e0 2024-04-25T00:31:59.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.648+0000 7fa84d6de640 1 -- 172.21.15.29:0/3624699157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa848140b30 con 0x7fa8480cb7e0 2024-04-25T00:31:59.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.648+0000 7fa83f7fe640 1 -- 172.21.15.29:0/3624699157 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838012400 con 0x7fa8480cb7e0 2024-04-25T00:31:59.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.648+0000 7fa84d6de640 1 -- 172.21.15.29:0/3624699157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8481413d0 con 0x7fa8480cb7e0 2024-04-25T00:31:59.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.648+0000 7fa84d6de640 1 -- 172.21.15.29:0/3624699157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa848140f20 con 0x7fa8480cb7e0 2024-04-25T00:31:59.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.648+0000 7fa83f7fe640 1 -- 172.21.15.29:0/3624699157 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa83801a070 con 0x7fa8480cb7e0 2024-04-25T00:31:59.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.648+0000 7fa83f7fe640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa8280678f0 0x7fa828069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.648+0000 7fa83f7fe640 1 -- 172.21.15.29:0/3624699157 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa838085a50 con 0x7fa8480cb7e0 2024-04-25T00:31:59.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.648+0000 7fa846ffd640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa8280678f0 0x7fa828069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.648+0000 7fa846ffd640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa8280678f0 0x7fa828069db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8400ad000 tx=0x7fa8400a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:59.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.652+0000 7fa83f7fe640 1 -- 172.21.15.29:0/3624699157 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa838056590 con 0x7fa8480cb7e0 2024-04-25T00:31:59.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.812+0000 7fa84d6de640 1 -- 172.21.15.29:0/3624699157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa8480b2da0 con 0x7fa8480cb7e0 2024-04-25T00:31:59.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.816+0000 7fa83f7fe640 1 -- 172.21.15.29:0/3624699157 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa838055e90 con 0x7fa8480cb7e0 2024-04-25T00:31:59.817 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:31:59.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.816+0000 7fa84d6de640 1 -- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa8280678f0 msgr2=0x7fa828069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.816+0000 7fa84d6de640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa8280678f0 0x7fa828069db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8400ad000 tx=0x7fa8400a6000 comp rx=0 tx=0).stop 2024-04-25T00:31:59.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.816+0000 7fa84d6de640 1 -- 172.21.15.29:0/3624699157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8480cb7e0 msgr2=0x7fa8481402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.816+0000 7fa84d6de640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8480cb7e0 0x7fa8481402e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa83800caa0 tx=0x7fa83800cf60 comp rx=0 tx=0).stop 2024-04-25T00:31:59.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.816+0000 7fa84d6de640 1 -- 172.21.15.29:0/3624699157 shutdown_connections 2024-04-25T00:31:59.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.816+0000 7fa84d6de640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa8280678f0 0x7fa828069db0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.816+0000 7fa84d6de640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84815e1b0 0x7fa84813f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.816+0000 7fa84d6de640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8480cb7e0 0x7fa8481402e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.816+0000 7fa84d6de640 1 --2- 172.21.15.29:0/3624699157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa848071fa0 0x7fa848141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.816+0000 7fa84d6de640 1 -- 172.21.15.29:0/3624699157 >> 172.21.15.29:0/3624699157 conn(0x7fa8480b2920 msgr2=0x7fa8480cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:59.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.816+0000 7fa84d6de640 1 -- 172.21.15.29:0/3624699157 shutdown_connections 2024-04-25T00:31:59.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.816+0000 7fa84d6de640 1 -- 172.21.15.29:0/3624699157 wait complete. 2024-04-25T00:31:59.837 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:31:59.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febd616d640 1 Processor -- start 2024-04-25T00:31:59.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febd616d640 1 -- start start 2024-04-25T00:31:59.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febd616d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febd006b920 0x7febd0168f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febd616d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd00cba50 con 0x7febd0162730 2024-04-25T00:31:59.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febd616d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd00cbd30 con 0x7febd015df40 2024-04-25T00:31:59.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febd616d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7febd00addc0 con 0x7febd006b920 2024-04-25T00:31:59.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febc6ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7febd0162730 0x7febd006b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39588/0 (socket says 172.21.15.29:39588) 2024-04-25T00:31:59.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febc6ffd640 1 -- 172.21.15.29:0/3667077459 learned_addr learned my addr 172.21.15.29:0/3667077459 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:31:59.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febcffff640 1 --2- 172.21.15.29:0/3667077459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febd006b920 0x7febd0168f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3429469177 0 0) 0x7febd00cba50 con 0x7febd0162730 2024-04-25T00:31:59.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febbc003530 con 0x7febd0162730 2024-04-25T00:31:59.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1420915624 0 0) 0x7febbc003530 con 0x7febd0162730 2024-04-25T00:31:59.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febd006b920 msgr2=0x7febd0168f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:59.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 --2- 172.21.15.29:0/3667077459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febd006b920 0x7febd0168f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 >> v1:172.21.15.157:6789/0 conn(0x7febd015df40 legacy=0x7febd00cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd00ada50 con 0x7febd0162730 2024-04-25T00:31:59.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7febb8001680 con 0x7febd0162730 2024-04-25T00:31:59.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 >> v1:172.21.15.29:6789/0 conn(0x7febd0162730 legacy=0x7febd006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 --2- 172.21.15.29:0/3667077459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febd006b920 0x7febbc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 --2- 172.21.15.29:0/3667077459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febbc003f50 0x7febbc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 --2- 172.21.15.29:0/3667077459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febbc004d50 0x7febbc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7febd00addc0 con 0x7febbc004d50 2024-04-25T00:31:59.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7febbc00bae0 con 0x7febbc003f50 2024-04-25T00:31:59.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7febbc00bce0 con 0x7febd006b920 2024-04-25T00:31:59.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febcffff640 1 --2- 172.21.15.29:0/3667077459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febbc004d50 0x7febbc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febcf7fe640 1 --2- 172.21.15.29:0/3667077459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febd006b920 0x7febbc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febc6ffd640 1 --2- 172.21.15.29:0/3667077459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febbc003f50 0x7febbc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febcffff640 1 -- 172.21.15.29:0/3667077459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febd006b920 msgr2=0x7febbc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:59.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febcffff640 1 --2- 172.21.15.29:0/3667077459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febd006b920 0x7febbc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febcffff640 1 -- 172.21.15.29:0/3667077459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febbc003f50 msgr2=0x7febbc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:59.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febcffff640 1 --2- 172.21.15.29:0/3667077459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febbc003f50 0x7febbc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.964+0000 7febcffff640 1 -- 172.21.15.29:0/3667077459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febbc003530 con 0x7febbc004d50 2024-04-25T00:31:59.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.968+0000 7febcffff640 1 --2- 172.21.15.29:0/3667077459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febbc004d50 0x7febbc0051d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7febc8009ae0 tx=0x7febc801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bec7ccc16502b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:59.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.968+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc80b2070 con 0x7febbc004d50 2024-04-25T00:31:59.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.968+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc80a5660 con 0x7febbc004d50 2024-04-25T00:31:59.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.968+0000 7febceffd640 1 -- 172.21.15.29:0/3667077459 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc80096b0 con 0x7febbc004d50 2024-04-25T00:31:59.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.968+0000 7febd616d640 1 -- 172.21.15.29:0/3667077459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febbc004d50 msgr2=0x7febbc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.968+0000 7febd616d640 1 --2- 172.21.15.29:0/3667077459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febbc004d50 0x7febbc0051d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7febc8009ae0 tx=0x7febc801ca70 comp rx=0 tx=0).stop 2024-04-25T00:31:59.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.968+0000 7febcffff640 1 -- 172.21.15.29:0/3667077459 reap_dead start 2024-04-25T00:31:59.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.968+0000 7febd616d640 1 -- 172.21.15.29:0/3667077459 shutdown_connections 2024-04-25T00:31:59.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.968+0000 7febd616d640 1 -- 172.21.15.29:0/3667077459 >> 172.21.15.29:0/3667077459 conn(0x7febd00d0c10 msgr2=0x7febd00d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:59.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.968+0000 7febd616d640 1 -- 172.21.15.29:0/3667077459 shutdown_connections 2024-04-25T00:31:59.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.968+0000 7febd616d640 1 -- 172.21.15.29:0/3667077459 wait complete. 2024-04-25T00:31:59.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.968+0000 7febd616d640 1 Processor -- start 2024-04-25T00:31:59.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.968+0000 7febd616d640 1 -- start start 2024-04-25T00:31:59.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febd616d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febd00c87c0 0x7febd01e9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febd616d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febd01e9fc0 0x7febd00c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febd616d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febd00ca440 0x7febd01e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febd616d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7febd00af3e0 con 0x7febd00c87c0 2024-04-25T00:31:59.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febd616d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7febd00af260 con 0x7febd01e9fc0 2024-04-25T00:31:59.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febd616d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7febd00af560 con 0x7febd00ca440 2024-04-25T00:31:59.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febd00c87c0 0x7febd01e9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febd00c87c0 0x7febd01e9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41092/0 (socket says 172.21.15.29:41092) 2024-04-25T00:31:59.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febcf7fe640 1 -- 172.21.15.29:0/1310352014 learned_addr learned my addr 172.21.15.29:0/1310352014 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:31:59.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febcffff640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febd00ca440 0x7febd01e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febc6ffd640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febd01e9fc0 0x7febd00c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febcf7fe640 1 -- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febd00ca440 msgr2=0x7febd01e88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febcf7fe640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febd00ca440 0x7febd01e88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febcf7fe640 1 -- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febd01e9fc0 msgr2=0x7febd00c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:59.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febcf7fe640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febd01e9fc0 0x7febd00c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:59.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febcf7fe640 1 -- 172.21.15.29:0/1310352014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd014fb70 con 0x7febd00c87c0 2024-04-25T00:31:59.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febc6ffd640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febd01e9fc0 0x7febd00c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:59.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febcf7fe640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febd00c87c0 0x7febd01e9a80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7febc0010f40 tx=0x7febc000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:59.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febccff9640 1 -- 172.21.15.29:0/1310352014 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0018020 con 0x7febd00c87c0 2024-04-25T00:31:59.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febccff9640 1 -- 172.21.15.29:0/1310352014 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc000ae50 con 0x7febd00c87c0 2024-04-25T00:31:59.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febd616d640 1 -- 172.21.15.29:0/1310352014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd014fe50 con 0x7febd00c87c0 2024-04-25T00:31:59.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febccff9640 1 -- 172.21.15.29:0/1310352014 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0003e00 con 0x7febd00c87c0 2024-04-25T00:31:59.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febd616d640 1 -- 172.21.15.29:0/1310352014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd0150410 con 0x7febd00c87c0 2024-04-25T00:31:59.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febd616d640 1 -- 172.21.15.29:0/1310352014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd0069270 con 0x7febd00c87c0 2024-04-25T00:31:59.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febccff9640 1 -- 172.21.15.29:0/1310352014 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7febc001b020 con 0x7febd00c87c0 2024-04-25T00:31:59.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febccff9640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7febb00678f0 0x7febb0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:59.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febccff9640 1 -- 172.21.15.29:0/1310352014 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7febc0089290 con 0x7febd00c87c0 2024-04-25T00:31:59.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.972+0000 7febc6ffd640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7febb00678f0 0x7febb0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:59.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.976+0000 7febc6ffd640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7febb00678f0 0x7febb0069db0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7febb80010f0 tx=0x7febb8000ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:59.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:31:59.976+0000 7febccff9640 1 -- 172.21.15.29:0/1310352014 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7febc0022030 con 0x7febd00c87c0 2024-04-25T00:32:00.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.140+0000 7febd616d640 1 -- 172.21.15.29:0/1310352014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7febd0152630 con 0x7febd00c87c0 2024-04-25T00:32:00.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.140+0000 7febccff9640 1 -- 172.21.15.29:0/1310352014 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7febc0059dd0 con 0x7febd00c87c0 2024-04-25T00:32:00.143 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:00.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.144+0000 7febd616d640 1 -- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7febb00678f0 msgr2=0x7febb0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.144+0000 7febd616d640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7febb00678f0 0x7febb0069db0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7febb80010f0 tx=0x7febb8000ee0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.144+0000 7febd616d640 1 -- 172.21.15.29:0/1310352014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febd00c87c0 msgr2=0x7febd01e9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.144+0000 7febd616d640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febd00c87c0 0x7febd01e9a80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7febc0010f40 tx=0x7febc000e530 comp rx=0 tx=0).stop 2024-04-25T00:32:00.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.144+0000 7febd616d640 1 -- 172.21.15.29:0/1310352014 shutdown_connections 2024-04-25T00:32:00.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.144+0000 7febd616d640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febd00ca440 0x7febd01e88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.144+0000 7febd616d640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7febb00678f0 0x7febb0069db0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.144+0000 7febd616d640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febd01e9fc0 0x7febd00c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.144+0000 7febd616d640 1 --2- 172.21.15.29:0/1310352014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febd00c87c0 0x7febd01e9a80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.144+0000 7febd616d640 1 -- 172.21.15.29:0/1310352014 >> 172.21.15.29:0/1310352014 conn(0x7febd00d0c10 msgr2=0x7febd006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:00.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.144+0000 7febd616d640 1 -- 172.21.15.29:0/1310352014 shutdown_connections 2024-04-25T00:32:00.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.144+0000 7febd616d640 1 -- 172.21.15.29:0/1310352014 wait complete. 2024-04-25T00:32:00.161 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:00.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbeed058640 1 Processor -- start 2024-04-25T00:32:00.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbeed058640 1 -- start start 2024-04-25T00:32:00.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbeed058640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee8071e10 0x7fbee815ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbeed058640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbee80721f0 0x7fbee8072650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbeed058640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbee815e860 0x7fbee815ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbeed058640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbee806f5e0 con 0x7fbee80721f0 2024-04-25T00:32:00.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbeed058640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbee80b0370 con 0x7fbee815e860 2024-04-25T00:32:00.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbeed058640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbee806e580 con 0x7fbee8071e10 2024-04-25T00:32:00.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbee659c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbee80721f0 0x7fbee8072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbee659c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbee80721f0 0x7fbee8072650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41100/0 (socket says 172.21.15.29:41100) 2024-04-25T00:32:00.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbee659c640 1 -- 172.21.15.29:0/1289603390 learned_addr learned my addr 172.21.15.29:0/1289603390 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:00.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbee6d9d640 1 --2- 172.21.15.29:0/1289603390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee8071e10 0x7fbee815ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbee659c640 1 -- 172.21.15.29:0/1289603390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee8071e10 msgr2=0x7fbee815ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:00.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbee659c640 1 --2- 172.21.15.29:0/1289603390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee8071e10 0x7fbee815ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbee659c640 1 -- 172.21.15.29:0/1289603390 >> v2:172.21.15.157:3300/0 conn(0x7fbee815e860 msgr2=0x7fbee815ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:00.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbee759e640 1 --2- 172.21.15.29:0/1289603390 >> v2:172.21.15.157:3300/0 conn(0x7fbee815e860 0x7fbee815ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbee659c640 1 --2- 172.21.15.29:0/1289603390 >> v2:172.21.15.157:3300/0 conn(0x7fbee815e860 0x7fbee815ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbee659c640 1 -- 172.21.15.29:0/1289603390 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee806e820 con 0x7fbee80721f0 2024-04-25T00:32:00.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbee6d9d640 1 --2- 172.21.15.29:0/1289603390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee8071e10 0x7fbee815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:00.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.272+0000 7fbee759e640 1 --2- 172.21.15.29:0/1289603390 >> v2:172.21.15.157:3300/0 conn(0x7fbee815e860 0x7fbee815ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:00.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee659c640 1 --2- 172.21.15.29:0/1289603390 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee80721f0 0x7fbee8072650 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbee815f900 tx=0x7fbedc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c084e605a8474fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:00.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee5d9b640 1 -- 172.21.15.29:0/1289603390 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0aa610 con 0x7fbee80721f0 2024-04-25T00:32:00.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee5d9b640 1 -- 172.21.15.29:0/1289603390 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbedc0af070 con 0x7fbee80721f0 2024-04-25T00:32:00.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee5d9b640 1 -- 172.21.15.29:0/1289603390 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0aa9d0 con 0x7fbee80721f0 2024-04-25T00:32:00.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 -- 172.21.15.29:0/1289603390 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee80721f0 msgr2=0x7fbee8072650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 --2- 172.21.15.29:0/1289603390 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee80721f0 0x7fbee8072650 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbee815f900 tx=0x7fbedc01c4f0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 -- 172.21.15.29:0/1289603390 shutdown_connections 2024-04-25T00:32:00.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 --2- 172.21.15.29:0/1289603390 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee80721f0 0x7fbee8072650 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 --2- 172.21.15.29:0/1289603390 >> v2:172.21.15.157:3300/0 conn(0x7fbee815e860 0x7fbee815ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 --2- 172.21.15.29:0/1289603390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee8071e10 0x7fbee815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 -- 172.21.15.29:0/1289603390 >> 172.21.15.29:0/1289603390 conn(0x7fbee80b2920 msgr2=0x7fbee80b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:00.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 -- 172.21.15.29:0/1289603390 shutdown_connections 2024-04-25T00:32:00.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 -- 172.21.15.29:0/1289603390 wait complete. 2024-04-25T00:32:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 Processor -- start 2024-04-25T00:32:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 -- start start 2024-04-25T00:32:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8071e10 0x7fbee8149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee80721f0 0x7fbee81495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee815e860 0x7fbee814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbee8072b90 con 0x7fbee80721f0 2024-04-25T00:32:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbee815f620 con 0x7fbee8071e10 2024-04-25T00:32:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbeed058640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbee815bfd0 con 0x7fbee815e860 2024-04-25T00:32:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee80721f0 0x7fbee81495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee80721f0 0x7fbee81495e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41110/0 (socket says 172.21.15.29:41110) 2024-04-25T00:32:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee659c640 1 -- 172.21.15.29:0/4271188062 learned_addr learned my addr 172.21.15.29:0/4271188062 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee759e640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee815e860 0x7fbee814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee6d9d640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8071e10 0x7fbee8149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee659c640 1 -- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee815e860 msgr2=0x7fbee814da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee659c640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee815e860 0x7fbee814da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee659c640 1 -- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8071e10 msgr2=0x7fbee8149060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee659c640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8071e10 0x7fbee8149060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee659c640 1 -- 172.21.15.29:0/4271188062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee80cbe20 con 0x7fbee80721f0 2024-04-25T00:32:00.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.276+0000 7fbee6d9d640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8071e10 0x7fbee8149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:00.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.280+0000 7fbee659c640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee80721f0 0x7fbee81495e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbedc01aba0 tx=0x7fbedc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:00.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.280+0000 7fbecf7fe640 1 -- 172.21.15.29:0/4271188062 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0aae60 con 0x7fbee80721f0 2024-04-25T00:32:00.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.280+0000 7fbecf7fe640 1 -- 172.21.15.29:0/4271188062 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbedc0b3820 con 0x7fbee80721f0 2024-04-25T00:32:00.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.280+0000 7fbeed058640 1 -- 172.21.15.29:0/4271188062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee80cc100 con 0x7fbee80721f0 2024-04-25T00:32:00.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.280+0000 7fbecf7fe640 1 -- 172.21.15.29:0/4271188062 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0b3c10 con 0x7fbee80721f0 2024-04-25T00:32:00.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.280+0000 7fbeed058640 1 -- 172.21.15.29:0/4271188062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee8149b20 con 0x7fbee80721f0 2024-04-25T00:32:00.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.280+0000 7fbeed058640 1 -- 172.21.15.29:0/4271188062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee8069270 con 0x7fbee80721f0 2024-04-25T00:32:00.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.280+0000 7fbecf7fe640 1 -- 172.21.15.29:0/4271188062 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbedc0c6390 con 0x7fbee80721f0 2024-04-25T00:32:00.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.280+0000 7fbecf7fe640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbed40678c0 0x7fbed4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.280+0000 7fbecf7fe640 1 -- 172.21.15.29:0/4271188062 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbedc1203e0 con 0x7fbee80721f0 2024-04-25T00:32:00.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.280+0000 7fbee6d9d640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbed40678c0 0x7fbed4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.280+0000 7fbee6d9d640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbed40678c0 0x7fbed4069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbed8006fd0 tx=0x7fbed8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:00.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.284+0000 7fbecf7fe640 1 -- 172.21.15.29:0/4271188062 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbedc0c3060 con 0x7fbee80721f0 2024-04-25T00:32:00.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.444+0000 7fbeed058640 1 -- 172.21.15.29:0/4271188062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbee806c100 con 0x7fbee80721f0 2024-04-25T00:32:00.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.448+0000 7fbecf7fe640 1 -- 172.21.15.29:0/4271188062 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fbedc0a7070 con 0x7fbee80721f0 2024-04-25T00:32:00.451 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:00.451 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:00.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.448+0000 7fbeed058640 1 -- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbed40678c0 msgr2=0x7fbed4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.448+0000 7fbeed058640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbed40678c0 0x7fbed4069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbed8006fd0 tx=0x7fbed8008040 comp rx=0 tx=0).stop 2024-04-25T00:32:00.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.448+0000 7fbeed058640 1 -- 172.21.15.29:0/4271188062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee80721f0 msgr2=0x7fbee81495e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.448+0000 7fbeed058640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee80721f0 0x7fbee81495e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbedc01aba0 tx=0x7fbedc01ad30 comp rx=0 tx=0).stop 2024-04-25T00:32:00.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.448+0000 7fbeed058640 1 -- 172.21.15.29:0/4271188062 shutdown_connections 2024-04-25T00:32:00.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.448+0000 7fbeed058640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbee815e860 0x7fbee814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.448+0000 7fbeed058640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbee80721f0 0x7fbee81495e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.448+0000 7fbeed058640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbed40678c0 0x7fbed4069d80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.448+0000 7fbeed058640 1 --2- 172.21.15.29:0/4271188062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8071e10 0x7fbee8149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.448+0000 7fbeed058640 1 -- 172.21.15.29:0/4271188062 >> 172.21.15.29:0/4271188062 conn(0x7fbee80b2920 msgr2=0x7fbee8159090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:00.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.448+0000 7fbeed058640 1 -- 172.21.15.29:0/4271188062 shutdown_connections 2024-04-25T00:32:00.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.448+0000 7fbeed058640 1 -- 172.21.15.29:0/4271188062 wait complete. 2024-04-25T00:32:00.468 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:00.469 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:00.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a84e68640 1 Processor -- start 2024-04-25T00:32:00.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a84e68640 1 -- start start 2024-04-25T00:32:00.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a84e68640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8015d980 0x7f4a8006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a84e68640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a8006db10 0x7f4a80151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a84e68640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a800b3130 con 0x7f4a800b2b70 2024-04-25T00:32:00.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a84e68640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4a800cf560 con 0x7f4a8006db10 2024-04-25T00:32:00.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a84e68640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4a800b3410 con 0x7f4a8015d980 2024-04-25T00:32:00.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a7dd9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4a800b2b70 0x7f4a80150df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39596/0 (socket says 172.21.15.29:39596) 2024-04-25T00:32:00.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a7dd9b640 1 -- 172.21.15.29:0/2660729229 learned_addr learned my addr 172.21.15.29:0/2660729229 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:00.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a7e59c640 1 --2- 172.21.15.29:0/2660729229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8015d980 0x7f4a8006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a7ed9d640 1 --2- 172.21.15.29:0/2660729229 >> v2:172.21.15.157:3300/0 conn(0x7f4a8006db10 0x7f4a80151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a7ed9d640 1 -- 172.21.15.29:0/2660729229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8015d980 msgr2=0x7f4a8006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:00.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a7ed9d640 1 --2- 172.21.15.29:0/2660729229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8015d980 0x7f4a8006d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a7ed9d640 1 -- 172.21.15.29:0/2660729229 >> v1:172.21.15.29:6789/0 conn(0x7f4a800b2b70 legacy=0x7f4a80150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.588+0000 7f4a7ed9d640 1 -- 172.21.15.29:0/2660729229 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a800b36f0 con 0x7f4a8006db10 2024-04-25T00:32:00.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.592+0000 7f4a7ed9d640 1 --2- 172.21.15.29:0/2660729229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8006db10 0x7f4a80151500 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009d90 tx=0x7f4a7801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ad6f673a242b1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.592+0000 7f4a7d59a640 1 -- 172.21.15.29:0/2660729229 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a780b2070 con 0x7f4a8006db10 2024-04-25T00:32:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.592+0000 7f4a7d59a640 1 -- 172.21.15.29:0/2660729229 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a780a5940 con 0x7f4a8006db10 2024-04-25T00:32:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.592+0000 7f4a7d59a640 1 -- 172.21.15.29:0/2660729229 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a780a5d10 con 0x7f4a8006db10 2024-04-25T00:32:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.592+0000 7f4a84e68640 1 -- 172.21.15.29:0/2660729229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8006db10 msgr2=0x7f4a80151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.592+0000 7f4a84e68640 1 --2- 172.21.15.29:0/2660729229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8006db10 0x7f4a80151500 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009d90 tx=0x7f4a7801c970 comp rx=0 tx=0).stop 2024-04-25T00:32:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.592+0000 7f4a84e68640 1 -- 172.21.15.29:0/2660729229 shutdown_connections 2024-04-25T00:32:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.592+0000 7f4a84e68640 1 --2- 172.21.15.29:0/2660729229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8006db10 0x7f4a80151500 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.592+0000 7f4a84e68640 1 --2- 172.21.15.29:0/2660729229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8015d980 0x7f4a8006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.592+0000 7f4a84e68640 1 -- 172.21.15.29:0/2660729229 >> 172.21.15.29:0/2660729229 conn(0x7f4a800b1870 msgr2=0x7f4a800b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:00.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.592+0000 7f4a84e68640 1 -- 172.21.15.29:0/2660729229 shutdown_connections 2024-04-25T00:32:00.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.592+0000 7f4a84e68640 1 -- 172.21.15.29:0/2660729229 wait complete. 2024-04-25T00:32:00.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a84e68640 1 Processor -- start 2024-04-25T00:32:00.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a84e68640 1 -- start start 2024-04-25T00:32:00.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a84e68640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8006db10 0x7f4a80149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a84e68640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a800b2b70 0x7f4a80147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a84e68640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a8015d980 0x7f4a80150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a84e68640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8006cec0 con 0x7f4a8015d980 2024-04-25T00:32:00.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a84e68640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8006cd40 con 0x7f4a800b2b70 2024-04-25T00:32:00.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a84e68640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8006d040 con 0x7f4a8006db10 2024-04-25T00:32:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a7ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a8015d980 0x7f4a80150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a7ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a8015d980 0x7f4a80150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41114/0 (socket says 172.21.15.29:41114) 2024-04-25T00:32:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a7ed9d640 1 -- 172.21.15.29:0/2095302716 learned_addr learned my addr 172.21.15.29:0/2095302716 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a7e59c640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8006db10 0x7f4a80149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a7dd9b640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a800b2b70 0x7f4a80147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a7ed9d640 1 -- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8006db10 msgr2=0x7f4a80149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a7ed9d640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8006db10 0x7f4a80149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a7ed9d640 1 -- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a800b2b70 msgr2=0x7f4a80147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a7ed9d640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a800b2b70 0x7f4a80147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a7ed9d640 1 -- 172.21.15.29:0/2095302716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a80155190 con 0x7f4a8015d980 2024-04-25T00:32:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a7ed9d640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a8015d980 0x7f4a80150200 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4a7801b020 tx=0x7f4a78019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a6effd640 1 -- 172.21.15.29:0/2095302716 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a780b2050 con 0x7f4a8015d980 2024-04-25T00:32:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a6effd640 1 -- 172.21.15.29:0/2095302716 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a780be070 con 0x7f4a8015d980 2024-04-25T00:32:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a84e68640 1 -- 172.21.15.29:0/2095302716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a80155470 con 0x7f4a8015d980 2024-04-25T00:32:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a6effd640 1 -- 172.21.15.29:0/2095302716 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a780ada70 con 0x7f4a8015d980 2024-04-25T00:32:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.596+0000 7f4a84e68640 1 -- 172.21.15.29:0/2095302716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a80155a30 con 0x7f4a8015d980 2024-04-25T00:32:00.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.600+0000 7f4a84e68640 1 -- 172.21.15.29:0/2095302716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a80069270 con 0x7f4a8015d980 2024-04-25T00:32:00.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.600+0000 7f4a6effd640 1 -- 172.21.15.29:0/2095302716 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4a780adc10 con 0x7f4a8015d980 2024-04-25T00:32:00.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.600+0000 7f4a6effd640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4a600678c0 0x7f4a60069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.600+0000 7f4a6effd640 1 -- 172.21.15.29:0/2095302716 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4a78120c30 con 0x7f4a8015d980 2024-04-25T00:32:00.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.600+0000 7f4a7e59c640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4a600678c0 0x7f4a60069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.600+0000 7f4a7e59c640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4a600678c0 0x7f4a60069d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4a8014aab0 tx=0x7f4a70009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:00.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.604+0000 7f4a6effd640 1 -- 172.21.15.29:0/2095302716 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4a78016560 con 0x7f4a8015d980 2024-04-25T00:32:00.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.760+0000 7f4a84e68640 1 -- 172.21.15.29:0/2095302716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4a800cf910 con 0x7f4a8015d980 2024-04-25T00:32:00.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.760+0000 7f4a6effd640 1 -- 172.21.15.29:0/2095302716 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f4a780f1800 con 0x7f4a8015d980 2024-04-25T00:32:00.764 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.764+0000 7f4a84e68640 1 -- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4a600678c0 msgr2=0x7f4a60069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.764+0000 7f4a84e68640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4a600678c0 0x7f4a60069d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4a8014aab0 tx=0x7f4a70009190 comp rx=0 tx=0).stop 2024-04-25T00:32:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.764+0000 7f4a84e68640 1 -- 172.21.15.29:0/2095302716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a8015d980 msgr2=0x7f4a80150200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.764+0000 7f4a84e68640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a8015d980 0x7f4a80150200 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4a7801b020 tx=0x7f4a78019710 comp rx=0 tx=0).stop 2024-04-25T00:32:00.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.764+0000 7f4a84e68640 1 -- 172.21.15.29:0/2095302716 shutdown_connections 2024-04-25T00:32:00.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.764+0000 7f4a84e68640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a8015d980 0x7f4a80150200 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.764+0000 7f4a84e68640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4a600678c0 0x7f4a60069d80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.764+0000 7f4a84e68640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a800b2b70 0x7f4a80147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.764+0000 7f4a84e68640 1 --2- 172.21.15.29:0/2095302716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8006db10 0x7f4a80149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.764+0000 7f4a84e68640 1 -- 172.21.15.29:0/2095302716 >> 172.21.15.29:0/2095302716 conn(0x7f4a800b1870 msgr2=0x7f4a800b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:00.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.764+0000 7f4a84e68640 1 -- 172.21.15.29:0/2095302716 shutdown_connections 2024-04-25T00:32:00.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.764+0000 7f4a84e68640 1 -- 172.21.15.29:0/2095302716 wait complete. 2024-04-25T00:32:00.783 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:00.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd9806ca640 1 Processor -- start 2024-04-25T00:32:00.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd9806ca640 1 -- start start 2024-04-25T00:32:00.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd9806ca640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd97815e8d0 0x7fd9780cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd9806ca640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd9780b4780 0x7fd97815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd9806ca640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97815f1f0 0x7fd97815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd9806ca640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd9780b42f0 con 0x7fd9780b4780 2024-04-25T00:32:00.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd9806ca640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd9780b3720 con 0x7fd97815e8d0 2024-04-25T00:32:00.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd9806ca640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9780b39c0 con 0x7fd97815f1f0 2024-04-25T00:32:00.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd97dc65640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd9780b4780 0x7fd97815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd97dc65640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd9780b4780 0x7fd97815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41124/0 (socket says 172.21.15.29:41124) 2024-04-25T00:32:00.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd97dc65640 1 -- 172.21.15.29:0/4179629995 learned_addr learned my addr 172.21.15.29:0/4179629995 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:00.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd97ec67640 1 --2- 172.21.15.29:0/4179629995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97815f1f0 0x7fd97815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd97e466640 1 --2- 172.21.15.29:0/4179629995 >> v2:172.21.15.157:3300/0 conn(0x7fd97815e8d0 0x7fd9780cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd97dc65640 1 -- 172.21.15.29:0/4179629995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97815f1f0 msgr2=0x7fd97815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:00.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd97dc65640 1 --2- 172.21.15.29:0/4179629995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97815f1f0 0x7fd97815b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd97dc65640 1 -- 172.21.15.29:0/4179629995 >> v2:172.21.15.157:3300/0 conn(0x7fd97815e8d0 msgr2=0x7fd9780cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:00.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd97dc65640 1 --2- 172.21.15.29:0/4179629995 >> v2:172.21.15.157:3300/0 conn(0x7fd97815e8d0 0x7fd9780cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.912+0000 7fd97dc65640 1 -- 172.21.15.29:0/4179629995 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9780b3ca0 con 0x7fd9780b4780 2024-04-25T00:32:00.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd97dc65640 1 --2- 172.21.15.29:0/4179629995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9780b4780 0x7fd97815ecb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd9780b5260 tx=0x7fd96801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5937e30b638f8bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd97d464640 1 -- 172.21.15.29:0/4179629995 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9680aa5f0 con 0x7fd9780b4780 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd97d464640 1 -- 172.21.15.29:0/4179629995 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9680af070 con 0x7fd9780b4780 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd97d464640 1 -- 172.21.15.29:0/4179629995 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9680aa9b0 con 0x7fd9780b4780 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 -- 172.21.15.29:0/4179629995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9780b4780 msgr2=0x7fd97815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 --2- 172.21.15.29:0/4179629995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9780b4780 0x7fd97815ecb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd9780b5260 tx=0x7fd96801c4f0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 -- 172.21.15.29:0/4179629995 shutdown_connections 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 --2- 172.21.15.29:0/4179629995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9780b4780 0x7fd97815ecb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 --2- 172.21.15.29:0/4179629995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97815f1f0 0x7fd97815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 --2- 172.21.15.29:0/4179629995 >> v2:172.21.15.157:3300/0 conn(0x7fd97815e8d0 0x7fd9780cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 -- 172.21.15.29:0/4179629995 >> 172.21.15.29:0/4179629995 conn(0x7fd9780b1870 msgr2=0x7fd9780b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 -- 172.21.15.29:0/4179629995 shutdown_connections 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 -- 172.21.15.29:0/4179629995 wait complete. 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 Processor -- start 2024-04-25T00:32:00.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 -- start start 2024-04-25T00:32:00.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9780b4780 0x7fd978140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd97815e8d0 0x7fd97815a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97815f1f0 0x7fd978143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd97815f600 con 0x7fd97815e8d0 2024-04-25T00:32:00.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd97815c7d0 con 0x7fd9780b4780 2024-04-25T00:32:00.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd9806ca640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9780ceb10 con 0x7fd97815f1f0 2024-04-25T00:32:00.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd97dc65640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd97815e8d0 0x7fd97815a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd97dc65640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd97815e8d0 0x7fd97815a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41140/0 (socket says 172.21.15.29:41140) 2024-04-25T00:32:00.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.916+0000 7fd97dc65640 1 -- 172.21.15.29:0/1490182475 learned_addr learned my addr 172.21.15.29:0/1490182475 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:00.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd97ec67640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97815f1f0 0x7fd978143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd97e466640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9780b4780 0x7fd978140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd97dc65640 1 -- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97815f1f0 msgr2=0x7fd978143130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd97dc65640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97815f1f0 0x7fd978143130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd97dc65640 1 -- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9780b4780 msgr2=0x7fd978140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:00.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd97dc65640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9780b4780 0x7fd978140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:00.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd97dc65640 1 -- 172.21.15.29:0/1490182475 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9780aea20 con 0x7fd97815e8d0 2024-04-25T00:32:00.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd97e466640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9780b4780 0x7fd978140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:00.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd97dc65640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd97815e8d0 0x7fd97815a530 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd96801aba0 tx=0x7fd96801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:00.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd96effd640 1 -- 172.21.15.29:0/1490182475 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9680bd5f0 con 0x7fd97815e8d0 2024-04-25T00:32:00.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd96effd640 1 -- 172.21.15.29:0/1490182475 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9680bd790 con 0x7fd97815e8d0 2024-04-25T00:32:00.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd9806ca640 1 -- 172.21.15.29:0/1490182475 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97806ef10 con 0x7fd97815e8d0 2024-04-25T00:32:00.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd96effd640 1 -- 172.21.15.29:0/1490182475 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9680bdb80 con 0x7fd97815e8d0 2024-04-25T00:32:00.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd9806ca640 1 -- 172.21.15.29:0/1490182475 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd978140ff0 con 0x7fd97815e8d0 2024-04-25T00:32:00.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd9806ca640 1 -- 172.21.15.29:0/1490182475 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd978069270 con 0x7fd97815e8d0 2024-04-25T00:32:00.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd96effd640 1 -- 172.21.15.29:0/1490182475 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd9680b33f0 con 0x7fd97815e8d0 2024-04-25T00:32:00.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd96effd640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd95c0678c0 0x7fd95c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:00.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd96effd640 1 -- 172.21.15.29:0/1490182475 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd968120800 con 0x7fd97815e8d0 2024-04-25T00:32:00.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd97e466640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd95c0678c0 0x7fd95c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:00.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.920+0000 7fd97e466640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd95c0678c0 0x7fd95c069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9740097c0 tx=0x7fd974008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:00.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:00.924+0000 7fd96effd640 1 -- 172.21.15.29:0/1490182475 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd968016110 con 0x7fd97815e8d0 2024-04-25T00:32:01.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.084+0000 7fd9806ca640 1 -- 172.21.15.29:0/1490182475 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd97806e7b0 con 0x7fd97815e8d0 2024-04-25T00:32:01.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.084+0000 7fd96effd640 1 -- 172.21.15.29:0/1490182475 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fd9680f1330 con 0x7fd97815e8d0 2024-04-25T00:32:01.087 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:01.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.088+0000 7fd9806ca640 1 -- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd95c0678c0 msgr2=0x7fd95c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.088+0000 7fd9806ca640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd95c0678c0 0x7fd95c069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9740097c0 tx=0x7fd974008040 comp rx=0 tx=0).stop 2024-04-25T00:32:01.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.088+0000 7fd9806ca640 1 -- 172.21.15.29:0/1490182475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd97815e8d0 msgr2=0x7fd97815a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.088+0000 7fd9806ca640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd97815e8d0 0x7fd97815a530 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd96801aba0 tx=0x7fd96801ad30 comp rx=0 tx=0).stop 2024-04-25T00:32:01.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.088+0000 7fd9806ca640 1 -- 172.21.15.29:0/1490182475 shutdown_connections 2024-04-25T00:32:01.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.088+0000 7fd9806ca640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97815f1f0 0x7fd978143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.088+0000 7fd9806ca640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd97815e8d0 0x7fd97815a530 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.088+0000 7fd9806ca640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd95c0678c0 0x7fd95c069d80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.088+0000 7fd9806ca640 1 --2- 172.21.15.29:0/1490182475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9780b4780 0x7fd978140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.088+0000 7fd9806ca640 1 -- 172.21.15.29:0/1490182475 >> 172.21.15.29:0/1490182475 conn(0x7fd9780b1870 msgr2=0x7fd9780b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:01.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.088+0000 7fd9806ca640 1 -- 172.21.15.29:0/1490182475 shutdown_connections 2024-04-25T00:32:01.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.088+0000 7fd9806ca640 1 -- 172.21.15.29:0/1490182475 wait complete. 2024-04-25T00:32:01.104 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:01.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f839425e640 1 Processor -- start 2024-04-25T00:32:01.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f839425e640 1 -- start start 2024-04-25T00:32:01.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f839425e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f838c0721f0 0x7f838c072650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f839425e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f838c0ce820 0x7f838c0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f839425e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f838c06e580 con 0x7f838c071e10 2024-04-25T00:32:01.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f839425e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f838c06f5e0 con 0x7f838c0ce820 2024-04-25T00:32:01.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f839425e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f838c15c1a0 con 0x7f838c0721f0 2024-04-25T00:32:01.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f8391ffa640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f838c071e10 0x7f838c15ba90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39606/0 (socket says 172.21.15.29:39606) 2024-04-25T00:32:01.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f8391ffa640 1 -- 172.21.15.29:0/2757626745 learned_addr learned my addr 172.21.15.29:0/2757626745 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:01.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f83927fb640 1 --2- 172.21.15.29:0/2757626745 >> v2:172.21.15.157:3300/0 conn(0x7f838c0ce820 0x7f838c0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f83917f9640 1 --2- 172.21.15.29:0/2757626745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f838c0721f0 0x7f838c072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f8390ff8640 1 -- 172.21.15.29:0/2757626745 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3654382469 0 0) 0x7f838c06e580 con 0x7f838c071e10 2024-04-25T00:32:01.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f8390ff8640 1 -- 172.21.15.29:0/2757626745 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8374003530 con 0x7f838c071e10 2024-04-25T00:32:01.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f83927fb640 1 -- 172.21.15.29:0/2757626745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f838c0721f0 msgr2=0x7f838c072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f83927fb640 1 --2- 172.21.15.29:0/2757626745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f838c0721f0 0x7f838c072650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f83927fb640 1 -- 172.21.15.29:0/2757626745 >> v1:172.21.15.29:6789/0 conn(0x7f838c071e10 legacy=0x7f838c15ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f8390ff8640 1 -- 172.21.15.29:0/2757626745 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3614706204 0 0) 0x7f8374003530 con 0x7f838c071e10 2024-04-25T00:32:01.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.224+0000 7f83927fb640 1 -- 172.21.15.29:0/2757626745 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838c06e820 con 0x7f838c0ce820 2024-04-25T00:32:01.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f83927fb640 1 --2- 172.21.15.29:0/2757626745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f838c0ce820 0x7f838c0cec90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83840097c0 tx=0x7f838401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2681828d6aa881 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f8390ff8640 1 -- 172.21.15.29:0/2757626745 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83840b0070 con 0x7f838c0ce820 2024-04-25T00:32:01.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f8390ff8640 1 -- 172.21.15.29:0/2757626745 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83840ab400 con 0x7f838c0ce820 2024-04-25T00:32:01.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f8390ff8640 1 -- 172.21.15.29:0/2757626745 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c001a00 con 0x7f838c0ce820 2024-04-25T00:32:01.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f839425e640 1 -- 172.21.15.29:0/2757626745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f838c0ce820 msgr2=0x7f838c0cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f839425e640 1 --2- 172.21.15.29:0/2757626745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f838c0ce820 0x7f838c0cec90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83840097c0 tx=0x7f838401c3c0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f839425e640 1 -- 172.21.15.29:0/2757626745 shutdown_connections 2024-04-25T00:32:01.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f839425e640 1 --2- 172.21.15.29:0/2757626745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f838c0ce820 0x7f838c0cec90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f839425e640 1 --2- 172.21.15.29:0/2757626745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f838c0721f0 0x7f838c072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f839425e640 1 -- 172.21.15.29:0/2757626745 >> 172.21.15.29:0/2757626745 conn(0x7f838c0b2920 msgr2=0x7f838c0b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:01.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f839425e640 1 -- 172.21.15.29:0/2757626745 shutdown_connections 2024-04-25T00:32:01.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f839425e640 1 -- 172.21.15.29:0/2757626745 wait complete. 2024-04-25T00:32:01.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f839425e640 1 Processor -- start 2024-04-25T00:32:01.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.228+0000 7f839425e640 1 -- start start 2024-04-25T00:32:01.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f839425e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f838c071e10 0x7f838c159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f839425e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f838c0721f0 0x7f838c0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f839425e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f838c0ce820 0x7f838c14e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f839425e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f838c167c00 con 0x7f838c0721f0 2024-04-25T00:32:01.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f839425e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f838c072b90 con 0x7f838c0ce820 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f839425e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f838c167d80 con 0x7f838c071e10 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f83917f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f838c0721f0 0x7f838c0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f83917f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f838c0721f0 0x7f838c0b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41152/0 (socket says 172.21.15.29:41152) 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f83917f9640 1 -- 172.21.15.29:0/3373463227 learned_addr learned my addr 172.21.15.29:0/3373463227 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f83917f9640 1 -- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f838c071e10 msgr2=0x7f838c159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f8391ffa640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f838c071e10 0x7f838c159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f83927fb640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f838c0ce820 0x7f838c14e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f83917f9640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f838c071e10 0x7f838c159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f83917f9640 1 -- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f838c0ce820 msgr2=0x7f838c14e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f83917f9640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f838c0ce820 0x7f838c14e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f83917f9640 1 -- 172.21.15.29:0/3373463227 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838c159fe0 con 0x7f838c0721f0 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f83917f9640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f838c0721f0 0x7f838c0b3990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f838000e930 tx=0x7f838000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f838a7fc640 1 -- 172.21.15.29:0/3373463227 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380017070 con 0x7f838c0721f0 2024-04-25T00:32:01.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f838a7fc640 1 -- 172.21.15.29:0/3373463227 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838000ca00 con 0x7f838c0721f0 2024-04-25T00:32:01.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f838a7fc640 1 -- 172.21.15.29:0/3373463227 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838000cdb0 con 0x7f838c0721f0 2024-04-25T00:32:01.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f839425e640 1 -- 172.21.15.29:0/3373463227 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838c0b3ed0 con 0x7f838c0721f0 2024-04-25T00:32:01.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f839425e640 1 -- 172.21.15.29:0/3373463227 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f838c1fee30 con 0x7f838c0721f0 2024-04-25T00:32:01.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f839425e640 1 -- 172.21.15.29:0/3373463227 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f838c069270 con 0x7f838c0721f0 2024-04-25T00:32:01.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f838a7fc640 1 -- 172.21.15.29:0/3373463227 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f838001a070 con 0x7f838c0721f0 2024-04-25T00:32:01.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f838a7fc640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f83780678c0 0x7f8378069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f838a7fc640 1 -- 172.21.15.29:0/3373463227 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8380085c40 con 0x7f838c0721f0 2024-04-25T00:32:01.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f8391ffa640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f83780678c0 0x7f8378069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.232+0000 7f8391ffa640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f83780678c0 0x7f8378069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f838c1659d0 tx=0x7f837c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.236+0000 7f838a7fc640 1 -- 172.21.15.29:0/3373463227 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f838c159fe0 con 0x7f838c0721f0 2024-04-25T00:32:01.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.396+0000 7f839425e640 1 -- 172.21.15.29:0/3373463227 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f838c1fb100 con 0x7f838c0721f0 2024-04-25T00:32:01.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.396+0000 7f838a7fc640 1 -- 172.21.15.29:0/3373463227 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f83800568d0 con 0x7f838c0721f0 2024-04-25T00:32:01.401 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:01.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.400+0000 7f839425e640 1 -- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f83780678c0 msgr2=0x7f8378069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.400+0000 7f839425e640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f83780678c0 0x7f8378069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f838c1659d0 tx=0x7f837c009040 comp rx=0 tx=0).stop 2024-04-25T00:32:01.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.400+0000 7f839425e640 1 -- 172.21.15.29:0/3373463227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f838c0721f0 msgr2=0x7f838c0b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.400+0000 7f839425e640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f838c0721f0 0x7f838c0b3990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f838000e930 tx=0x7f838000edf0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.400+0000 7f839425e640 1 -- 172.21.15.29:0/3373463227 shutdown_connections 2024-04-25T00:32:01.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.400+0000 7f839425e640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f83780678c0 0x7f8378069d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.400+0000 7f839425e640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f838c0ce820 0x7f838c14e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.400+0000 7f839425e640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f838c0721f0 0x7f838c0b3990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.400+0000 7f839425e640 1 --2- 172.21.15.29:0/3373463227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f838c071e10 0x7f838c159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.400+0000 7f839425e640 1 -- 172.21.15.29:0/3373463227 >> 172.21.15.29:0/3373463227 conn(0x7f838c0b2920 msgr2=0x7f838c1560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:01.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.400+0000 7f839425e640 1 -- 172.21.15.29:0/3373463227 shutdown_connections 2024-04-25T00:32:01.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.400+0000 7f839425e640 1 -- 172.21.15.29:0/3373463227 wait complete. 2024-04-25T00:32:01.420 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:01.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b1be26640 1 Processor -- start 2024-04-25T00:32:01.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b1be26640 1 -- start start 2024-04-25T00:32:01.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b1be26640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1415e330 0x7f5b1406db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b1be26640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b1406b210 0x7f5b1406b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b1be26640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b140cfe90 con 0x7f5b1406f270 2024-04-25T00:32:01.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b1be26640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5b140d0170 con 0x7f5b1406b210 2024-04-25T00:32:01.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b1be26640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5b140d0450 con 0x7f5b1415e330 2024-04-25T00:32:01.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b193c1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1415e330 0x7f5b1406db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b1a3c3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b1406b210 0x7f5b1406b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b193c1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1415e330 0x7f5b1406db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:41696/0 (socket says 172.21.15.29:41696) 2024-04-25T00:32:01.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b1a3c3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b1406b210 0x7f5b1406b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:50092/0 (socket says 172.21.15.29:50092) 2024-04-25T00:32:01.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b193c1640 1 -- 172.21.15.29:0/1974410329 learned_addr learned my addr 172.21.15.29:0/1974410329 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:01.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b19bc2640 1 --1- 172.21.15.29:0/1974410329 >> v1:172.21.15.157:6789/0 conn(0x7f5b1406f270 0x7f5b1406f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:53156/0 (socket says 172.21.15.29:53156) 2024-04-25T00:32:01.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b1a3c3640 1 -- 172.21.15.29:0/1974410329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1415e330 msgr2=0x7f5b1406db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b1a3c3640 1 --2- 172.21.15.29:0/1974410329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1415e330 0x7f5b1406db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b1a3c3640 1 -- 172.21.15.29:0/1974410329 >> v1:172.21.15.157:6789/0 conn(0x7f5b1406f270 legacy=0x7f5b1406f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.540+0000 7f5b1a3c3640 1 -- 172.21.15.29:0/1974410329 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b140d0730 con 0x7f5b1406b210 2024-04-25T00:32:01.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.544+0000 7f5b1a3c3640 1 --2- 172.21.15.29:0/1974410329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1406b210 0x7f5b1406b680 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0097c0 tx=0x7f5b0c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7daa033d52cf316c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.544+0000 7f5b18bc0640 1 -- 172.21.15.29:0/1974410329 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0b0070 con 0x7f5b1406b210 2024-04-25T00:32:01.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.544+0000 7f5b18bc0640 1 -- 172.21.15.29:0/1974410329 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0ab600 con 0x7f5b1406b210 2024-04-25T00:32:01.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.544+0000 7f5b18bc0640 1 -- 172.21.15.29:0/1974410329 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0ab9d0 con 0x7f5b1406b210 2024-04-25T00:32:01.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.544+0000 7f5b1be26640 1 -- 172.21.15.29:0/1974410329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1406b210 msgr2=0x7f5b1406b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.544+0000 7f5b1be26640 1 --2- 172.21.15.29:0/1974410329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1406b210 0x7f5b1406b680 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0097c0 tx=0x7f5b0c01c410 comp rx=0 tx=0).stop 2024-04-25T00:32:01.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.544+0000 7f5b1be26640 1 -- 172.21.15.29:0/1974410329 shutdown_connections 2024-04-25T00:32:01.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.544+0000 7f5b1be26640 1 --2- 172.21.15.29:0/1974410329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1406b210 0x7f5b1406b680 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.544+0000 7f5b1be26640 1 --2- 172.21.15.29:0/1974410329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1415e330 0x7f5b1406db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.544+0000 7f5b1be26640 1 -- 172.21.15.29:0/1974410329 >> 172.21.15.29:0/1974410329 conn(0x7f5b140b2920 msgr2=0x7f5b140b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:01.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.544+0000 7f5b1be26640 1 -- 172.21.15.29:0/1974410329 shutdown_connections 2024-04-25T00:32:01.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.544+0000 7f5b1be26640 1 -- 172.21.15.29:0/1974410329 wait complete. 2024-04-25T00:32:01.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b1be26640 1 Processor -- start 2024-04-25T00:32:01.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b1be26640 1 -- start start 2024-04-25T00:32:01.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b1be26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1406b210 0x7f5b1414a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b1be26640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b1406f270 0x7f5b141530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b1be26640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b140cc660 0x7f5b140ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b1be26640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1415f580 con 0x7f5b1406f270 2024-04-25T00:32:01.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b1be26640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1415f400 con 0x7f5b1406b210 2024-04-25T00:32:01.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b1be26640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5b1416af40 con 0x7f5b140cc660 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b193c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b1406f270 0x7f5b141530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b193c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b1406f270 0x7f5b141530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41156/0 (socket says 172.21.15.29:41156) 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b193c1640 1 -- 172.21.15.29:0/1121911155 learned_addr learned my addr 172.21.15.29:0/1121911155 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b1a3c3640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b140cc660 0x7f5b140ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b19bc2640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1406b210 0x7f5b1414a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b193c1640 1 -- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b140cc660 msgr2=0x7f5b140ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b193c1640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b140cc660 0x7f5b140ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b193c1640 1 -- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1406b210 msgr2=0x7f5b1414a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b193c1640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1406b210 0x7f5b1414a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b193c1640 1 -- 172.21.15.29:0/1121911155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b141524f0 con 0x7f5b1406f270 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b193c1640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b1406f270 0x7f5b141530c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400e890 tx=0x7f5b0400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b0a7fc640 1 -- 172.21.15.29:0/1121911155 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04017070 con 0x7f5b1406f270 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b0a7fc640 1 -- 172.21.15.29:0/1121911155 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0400c940 con 0x7f5b1406f270 2024-04-25T00:32:01.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b1be26640 1 -- 172.21.15.29:0/1121911155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b140cb5c0 con 0x7f5b1406f270 2024-04-25T00:32:01.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b0a7fc640 1 -- 172.21.15.29:0/1121911155 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0400cd30 con 0x7f5b1406f270 2024-04-25T00:32:01.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b1be26640 1 -- 172.21.15.29:0/1121911155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b140cbb80 con 0x7f5b1406f270 2024-04-25T00:32:01.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b1be26640 1 -- 172.21.15.29:0/1121911155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b14069270 con 0x7f5b1406f270 2024-04-25T00:32:01.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.548+0000 7f5b0a7fc640 1 -- 172.21.15.29:0/1121911155 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5b0401a070 con 0x7f5b1406f270 2024-04-25T00:32:01.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.552+0000 7f5b0a7fc640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5af80678c0 0x7f5af8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.552+0000 7f5b0a7fc640 1 -- 172.21.15.29:0/1121911155 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5b04085fc0 con 0x7f5b1406f270 2024-04-25T00:32:01.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.552+0000 7f5b19bc2640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5af80678c0 0x7f5af8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.552+0000 7f5b19bc2640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5af80678c0 0x7f5af8069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5b100012f0 tx=0x7f5b10001640 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.556+0000 7f5b0a7fc640 1 -- 172.21.15.29:0/1121911155 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5b04056410 con 0x7f5b1406f270 2024-04-25T00:32:01.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.720+0000 7f5b1be26640 1 -- 172.21.15.29:0/1121911155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b14161000 con 0x7f5b1406f270 2024-04-25T00:32:01.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.724+0000 7f5b0a7fc640 1 -- 172.21.15.29:0/1121911155 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f5b0405a710 con 0x7f5b1406f270 2024-04-25T00:32:01.726 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:01.726 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:01.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.724+0000 7f5b1be26640 1 -- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5af80678c0 msgr2=0x7f5af8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.724+0000 7f5b1be26640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5af80678c0 0x7f5af8069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5b100012f0 tx=0x7f5b10001640 comp rx=0 tx=0).stop 2024-04-25T00:32:01.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.724+0000 7f5b1be26640 1 -- 172.21.15.29:0/1121911155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b1406f270 msgr2=0x7f5b141530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.724+0000 7f5b1be26640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b1406f270 0x7f5b141530c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400e890 tx=0x7f5b0400ed50 comp rx=0 tx=0).stop 2024-04-25T00:32:01.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.724+0000 7f5b1be26640 1 -- 172.21.15.29:0/1121911155 shutdown_connections 2024-04-25T00:32:01.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.724+0000 7f5b1be26640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b140cc660 0x7f5b140ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.724+0000 7f5b1be26640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b1406f270 0x7f5b141530c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.724+0000 7f5b1be26640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5af80678c0 0x7f5af8069d80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.724+0000 7f5b1be26640 1 --2- 172.21.15.29:0/1121911155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1406b210 0x7f5b1414a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.724+0000 7f5b1be26640 1 -- 172.21.15.29:0/1121911155 >> 172.21.15.29:0/1121911155 conn(0x7f5b140b2920 msgr2=0x7f5b1406f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:01.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.724+0000 7f5b1be26640 1 -- 172.21.15.29:0/1121911155 shutdown_connections 2024-04-25T00:32:01.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.724+0000 7f5b1be26640 1 -- 172.21.15.29:0/1121911155 wait complete. 2024-04-25T00:32:01.746 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:01.747 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:01.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.864+0000 7f9607af1640 1 Processor -- start 2024-04-25T00:32:01.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.864+0000 7f9607af1640 1 -- start start 2024-04-25T00:32:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f9607af1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f960006b210 0x7f9600072990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f9607af1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960015ecf0 0x7f96000ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f9607af1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f960015f2a0 con 0x7f960006b5f0 2024-04-25T00:32:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f9607af1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f96000b03f0 con 0x7f960006b210 2024-04-25T00:32:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f9607af1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f960015f580 con 0x7f960015ecf0 2024-04-25T00:32:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960508c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f960006b5f0 0x7f960015e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39614/0 (socket says 172.21.15.29:39614) 2024-04-25T00:32:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960508c640 1 -- 172.21.15.29:0/1078854389 learned_addr learned my addr 172.21.15.29:0/1078854389 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960588d640 1 --2- 172.21.15.29:0/1078854389 >> v2:172.21.15.157:3300/0 conn(0x7f960006b210 0x7f9600072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960608e640 1 --2- 172.21.15.29:0/1078854389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960015ecf0 0x7f96000ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 691329954 0 0) 0x7f960015f2a0 con 0x7f960006b5f0 2024-04-25T00:32:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95ec003530 con 0x7f960006b5f0 2024-04-25T00:32:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3568494821 0 0) 0x7f95ec003530 con 0x7f960006b5f0 2024-04-25T00:32:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960015ecf0 msgr2=0x7f96000ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 --2- 172.21.15.29:0/1078854389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960015ecf0 0x7f96000ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 >> v2:172.21.15.157:3300/0 conn(0x7f960006b210 msgr2=0x7f9600072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 --2- 172.21.15.29:0/1078854389 >> v2:172.21.15.157:3300/0 conn(0x7f960006b210 0x7f9600072990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9600071eb0 con 0x7f960006b5f0 2024-04-25T00:32:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f95f0001680 con 0x7f960006b5f0 2024-04-25T00:32:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 >> v1:172.21.15.29:6789/0 conn(0x7f960006b5f0 legacy=0x7f960015e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 --2- 172.21.15.29:0/1078854389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960015ecf0 0x7f95ec0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 --2- 172.21.15.29:0/1078854389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ec003f40 0x7f95ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 --2- 172.21.15.29:0/1078854389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95ec004d50 0x7f95ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f96000b03f0 con 0x7f95ec004d50 2024-04-25T00:32:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f960015f580 con 0x7f95ec003f40 2024-04-25T00:32:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f95ec00bab0 con 0x7f960015ecf0 2024-04-25T00:32:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960608e640 1 --2- 172.21.15.29:0/1078854389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95ec004d50 0x7f95ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960588d640 1 --2- 172.21.15.29:0/1078854389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960015ecf0 0x7f95ec0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960508c640 1 --2- 172.21.15.29:0/1078854389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ec003f40 0x7f95ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960608e640 1 -- 172.21.15.29:0/1078854389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960015ecf0 msgr2=0x7f95ec0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960608e640 1 --2- 172.21.15.29:0/1078854389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f960015ecf0 0x7f95ec0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960608e640 1 -- 172.21.15.29:0/1078854389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ec003f40 msgr2=0x7f95ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960608e640 1 --2- 172.21.15.29:0/1078854389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ec003f40 0x7f95ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.868+0000 7f960608e640 1 -- 172.21.15.29:0/1078854389 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ec003530 con 0x7f95ec004d50 2024-04-25T00:32:01.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.872+0000 7f960608e640 1 --2- 172.21.15.29:0/1078854389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95ec004d50 0x7f95ec0051d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009fc0 tx=0x7f95f801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845e8c68a5306307 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.872+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f80b2070 con 0x7f95ec004d50 2024-04-25T00:32:01.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.872+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f80a5a70 con 0x7f95ec004d50 2024-04-25T00:32:01.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.872+0000 7f960488b640 1 -- 172.21.15.29:0/1078854389 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8009c10 con 0x7f95ec004d50 2024-04-25T00:32:01.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 -- 172.21.15.29:0/1078854389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95ec004d50 msgr2=0x7f95ec0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 --2- 172.21.15.29:0/1078854389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95ec004d50 0x7f95ec0051d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009fc0 tx=0x7f95f801cc80 comp rx=0 tx=0).stop 2024-04-25T00:32:01.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f960608e640 1 -- 172.21.15.29:0/1078854389 reap_dead start 2024-04-25T00:32:01.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 -- 172.21.15.29:0/1078854389 shutdown_connections 2024-04-25T00:32:01.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 -- 172.21.15.29:0/1078854389 >> 172.21.15.29:0/1078854389 conn(0x7f96000d0c40 msgr2=0x7f96000d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:01.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 -- 172.21.15.29:0/1078854389 shutdown_connections 2024-04-25T00:32:01.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 -- 172.21.15.29:0/1078854389 wait complete. 2024-04-25T00:32:01.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 Processor -- start 2024-04-25T00:32:01.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 -- start start 2024-04-25T00:32:01.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96000cfad0 0x7f96001fba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96000b4950 0x7f96000caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96000cfeb0 0x7f96000cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f96000ae0c0 con 0x7f96000cfad0 2024-04-25T00:32:01.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96000cbf80 con 0x7f96000b4950 2024-04-25T00:32:01.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96000ae240 con 0x7f96000cfeb0 2024-04-25T00:32:01.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f960588d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96000cfad0 0x7f96001fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f960588d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96000cfad0 0x7f96001fba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41166/0 (socket says 172.21.15.29:41166) 2024-04-25T00:32:01.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f960588d640 1 -- 172.21.15.29:0/2016019952 learned_addr learned my addr 172.21.15.29:0/2016019952 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:01.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f960608e640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96000cfeb0 0x7f96000cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f960508c640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96000b4950 0x7f96000caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f960588d640 1 -- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96000cfeb0 msgr2=0x7f96000cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f960588d640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96000cfeb0 0x7f96000cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f960588d640 1 -- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96000b4950 msgr2=0x7f96000caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:01.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f960588d640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96000b4950 0x7f96000caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:01.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f960588d640 1 -- 172.21.15.29:0/2016019952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96001fbfb0 con 0x7f96000cfad0 2024-04-25T00:32:01.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f960608e640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96000cfeb0 0x7f96000cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:01.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f960588d640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96000cfad0 0x7f96001fba70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f95fc011690 tx=0x7f95fc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f95f67fc640 1 -- 172.21.15.29:0/2016019952 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc00f9c0 con 0x7f96000cfad0 2024-04-25T00:32:01.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f95f67fc640 1 -- 172.21.15.29:0/2016019952 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95fc00fb60 con 0x7f96000cfad0 2024-04-25T00:32:01.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 -- 172.21.15.29:0/2016019952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96000b4e80 con 0x7f96000cfad0 2024-04-25T00:32:01.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f95f67fc640 1 -- 172.21.15.29:0/2016019952 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc015460 con 0x7f96000cfad0 2024-04-25T00:32:01.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.876+0000 7f9607af1640 1 -- 172.21.15.29:0/2016019952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96000b5440 con 0x7f96000cfad0 2024-04-25T00:32:01.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.880+0000 7f9607af1640 1 -- 172.21.15.29:0/2016019952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9600069270 con 0x7f96000cfad0 2024-04-25T00:32:01.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.880+0000 7f95f67fc640 1 -- 172.21.15.29:0/2016019952 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f95fc015600 con 0x7f96000cfad0 2024-04-25T00:32:01.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.880+0000 7f95f67fc640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f95e40678f0 0x7f95e4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:01.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.880+0000 7f95f67fc640 1 -- 172.21.15.29:0/2016019952 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f95fc0865d0 con 0x7f96000cfad0 2024-04-25T00:32:01.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.880+0000 7f960508c640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f95e40678f0 0x7f95e4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:01.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.880+0000 7f960508c640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f95e40678f0 0x7f95e4069db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9600166ef0 tx=0x7f95f0001dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:01.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:01.884+0000 7f95f67fc640 1 -- 172.21.15.29:0/2016019952 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f96001fbfb0 con 0x7f96000cfad0 2024-04-25T00:32:02.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.044+0000 7f9607af1640 1 -- 172.21.15.29:0/2016019952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f96000d1a70 con 0x7f96000cfad0 2024-04-25T00:32:02.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.048+0000 7f95f67fc640 1 -- 172.21.15.29:0/2016019952 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f95fc019030 con 0x7f96000cfad0 2024-04-25T00:32:02.049 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:02.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.048+0000 7f9607af1640 1 -- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f95e40678f0 msgr2=0x7f95e4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.048+0000 7f9607af1640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f95e40678f0 0x7f95e4069db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9600166ef0 tx=0x7f95f0001dc0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.048+0000 7f9607af1640 1 -- 172.21.15.29:0/2016019952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96000cfad0 msgr2=0x7f96001fba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.048+0000 7f9607af1640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96000cfad0 0x7f96001fba70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f95fc011690 tx=0x7f95fc011b50 comp rx=0 tx=0).stop 2024-04-25T00:32:02.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.048+0000 7f9607af1640 1 -- 172.21.15.29:0/2016019952 shutdown_connections 2024-04-25T00:32:02.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.048+0000 7f9607af1640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96000cfeb0 0x7f96000cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.048+0000 7f9607af1640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f95e40678f0 0x7f95e4069db0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.048+0000 7f9607af1640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96000b4950 0x7f96000caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.048+0000 7f9607af1640 1 --2- 172.21.15.29:0/2016019952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96000cfad0 0x7f96001fba70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.048+0000 7f9607af1640 1 -- 172.21.15.29:0/2016019952 >> 172.21.15.29:0/2016019952 conn(0x7f96000d0c40 msgr2=0x7f96000ced50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:02.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.048+0000 7f9607af1640 1 -- 172.21.15.29:0/2016019952 shutdown_connections 2024-04-25T00:32:02.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.048+0000 7f9607af1640 1 -- 172.21.15.29:0/2016019952 wait complete. 2024-04-25T00:32:02.070 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:02.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f028d33d640 1 Processor -- start 2024-04-25T00:32:02.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f028d33d640 1 -- start start 2024-04-25T00:32:02.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f028d33d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f028806dce0 0x7f0288150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f028d33d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02880b3130 con 0x7f02880b2b70 2024-04-25T00:32:02.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f028d33d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02880b3410 con 0x7f028815d980 2024-04-25T00:32:02.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f028d33d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f02880cf560 con 0x7f028806dce0 2024-04-25T00:32:02.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f02877fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f02880b2b70 0x7f0288151330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39616/0 (socket says 172.21.15.29:39616) 2024-04-25T00:32:02.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f02877fe640 1 -- 172.21.15.29:0/2080095204 learned_addr learned my addr 172.21.15.29:0/2080095204 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:02.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f02867fc640 1 --2- 172.21.15.29:0/2080095204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f028806dce0 0x7f0288150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1001266109 0 0) 0x7f02880b3130 con 0x7f02880b2b70 2024-04-25T00:32:02.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0270003530 con 0x7f02880b2b70 2024-04-25T00:32:02.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 895264908 0 0) 0x7f0270003530 con 0x7f02880b2b70 2024-04-25T00:32:02.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f028806dce0 msgr2=0x7f0288150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:02.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 --2- 172.21.15.29:0/2080095204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f028806dce0 0x7f0288150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 >> v1:172.21.15.157:6789/0 conn(0x7f028815d980 legacy=0x7f028806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02880b36f0 con 0x7f02880b2b70 2024-04-25T00:32:02.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f0280001a00 con 0x7f02880b2b70 2024-04-25T00:32:02.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 >> v1:172.21.15.29:6789/0 conn(0x7f02880b2b70 legacy=0x7f0288151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 --2- 172.21.15.29:0/2080095204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02700039f0 0x7f0270003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 --2- 172.21.15.29:0/2080095204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0270004810 0x7f0270004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 --2- 172.21.15.29:0/2080095204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f028806dce0 0x7f0270005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f02880cf560 con 0x7f02700039f0 2024-04-25T00:32:02.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f027000bb00 con 0x7f0270004810 2024-04-25T00:32:02.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f027000df10 con 0x7f028806dce0 2024-04-25T00:32:02.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0286ffd640 1 --2- 172.21.15.29:0/2080095204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02700039f0 0x7f0270003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f02867fc640 1 --2- 172.21.15.29:0/2080095204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0270004810 0x7f0270004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f02877fe640 1 --2- 172.21.15.29:0/2080095204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f028806dce0 0x7f0270005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0286ffd640 1 -- 172.21.15.29:0/2080095204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f028806dce0 msgr2=0x7f0270005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:02.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0286ffd640 1 --2- 172.21.15.29:0/2080095204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f028806dce0 0x7f0270005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0286ffd640 1 -- 172.21.15.29:0/2080095204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0270004810 msgr2=0x7f0270004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:02.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0286ffd640 1 --2- 172.21.15.29:0/2080095204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0270004810 0x7f0270004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.200+0000 7f0286ffd640 1 -- 172.21.15.29:0/2080095204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0270003530 con 0x7f02700039f0 2024-04-25T00:32:02.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f0286ffd640 1 --2- 172.21.15.29:0/2080095204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02700039f0 0x7f0270003e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0278001780 tx=0x7f027801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9b99c9aeb0464 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:02.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02780013a0 con 0x7f02700039f0 2024-04-25T00:32:02.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02780052f0 con 0x7f02700039f0 2024-04-25T00:32:02.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f0285ffb640 1 -- 172.21.15.29:0/2080095204 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278003840 con 0x7f02700039f0 2024-04-25T00:32:02.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 -- 172.21.15.29:0/2080095204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02700039f0 msgr2=0x7f0270003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 --2- 172.21.15.29:0/2080095204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02700039f0 0x7f0270003e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0278001780 tx=0x7f027801c420 comp rx=0 tx=0).stop 2024-04-25T00:32:02.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f02877fe640 1 -- 172.21.15.29:0/2080095204 reap_dead start 2024-04-25T00:32:02.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 -- 172.21.15.29:0/2080095204 shutdown_connections 2024-04-25T00:32:02.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 -- 172.21.15.29:0/2080095204 >> 172.21.15.29:0/2080095204 conn(0x7f02880b1870 msgr2=0x7f02880b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:02.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 -- 172.21.15.29:0/2080095204 shutdown_connections 2024-04-25T00:32:02.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 -- 172.21.15.29:0/2080095204 wait complete. 2024-04-25T00:32:02.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 Processor -- start 2024-04-25T00:32:02.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 -- start start 2024-04-25T00:32:02.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02700039f0 0x7f02880ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02881f7230 0x7f028815fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02880cdb20 0x7f028814f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f028815b630 con 0x7f02700039f0 2024-04-25T00:32:02.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f028815b4b0 con 0x7f02881f7230 2024-04-25T00:32:02.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f028d33d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f028815b7b0 con 0x7f02880cdb20 2024-04-25T00:32:02.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f0286ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02700039f0 0x7f02880ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f0286ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02700039f0 0x7f02880ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41184/0 (socket says 172.21.15.29:41184) 2024-04-25T00:32:02.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.204+0000 7f0286ffd640 1 -- 172.21.15.29:0/603725167 learned_addr learned my addr 172.21.15.29:0/603725167 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:02.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f02867fc640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02881f7230 0x7f028815fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f02877fe640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02880cdb20 0x7f028814f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f0286ffd640 1 -- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02880cdb20 msgr2=0x7f028814f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f0286ffd640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02880cdb20 0x7f028814f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f0286ffd640 1 -- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02881f7230 msgr2=0x7f028815fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f0286ffd640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02881f7230 0x7f028815fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f0286ffd640 1 -- 172.21.15.29:0/603725167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0288160cc0 con 0x7f02700039f0 2024-04-25T00:32:02.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f0286ffd640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02700039f0 0x7f02880ae4b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0278005040 tx=0x7f0270005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:02.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f026f7fe640 1 -- 172.21.15.29:0/603725167 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278007a40 con 0x7f02700039f0 2024-04-25T00:32:02.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f026f7fe640 1 -- 172.21.15.29:0/603725167 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02780bb070 con 0x7f02700039f0 2024-04-25T00:32:02.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f028d33d640 1 -- 172.21.15.29:0/603725167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0288160420 con 0x7f02700039f0 2024-04-25T00:32:02.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f026f7fe640 1 -- 172.21.15.29:0/603725167 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02780013a0 con 0x7f02700039f0 2024-04-25T00:32:02.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f028d33d640 1 -- 172.21.15.29:0/603725167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02881609e0 con 0x7f02700039f0 2024-04-25T00:32:02.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f028d33d640 1 -- 172.21.15.29:0/603725167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0288069270 con 0x7f02700039f0 2024-04-25T00:32:02.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f026f7fe640 1 -- 172.21.15.29:0/603725167 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f02780b4040 con 0x7f02700039f0 2024-04-25T00:32:02.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f026f7fe640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02740678f0 0x7f0274069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f026f7fe640 1 -- 172.21.15.29:0/603725167 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f027811fb40 con 0x7f02700039f0 2024-04-25T00:32:02.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.208+0000 7f02867fc640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02740678f0 0x7f0274069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.212+0000 7f02867fc640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02740678f0 0x7f0274069db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f02881fa750 tx=0x7f027c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:02.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.212+0000 7f026f7fe640 1 -- 172.21.15.29:0/603725167 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0278016130 con 0x7f02700039f0 2024-04-25T00:32:02.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f028d33d640 1 -- 172.21.15.29:0/603725167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f02880cf910 con 0x7f02700039f0 2024-04-25T00:32:02.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f026f7fe640 1 -- 172.21.15.29:0/603725167 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f02780f0710 con 0x7f02700039f0 2024-04-25T00:32:02.386 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:02.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f028d33d640 1 -- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02740678f0 msgr2=0x7f0274069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f028d33d640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02740678f0 0x7f0274069db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f02881fa750 tx=0x7f027c008040 comp rx=0 tx=0).stop 2024-04-25T00:32:02.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f028d33d640 1 -- 172.21.15.29:0/603725167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02700039f0 msgr2=0x7f02880ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f028d33d640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02700039f0 0x7f02880ae4b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0278005040 tx=0x7f0270005840 comp rx=0 tx=0).stop 2024-04-25T00:32:02.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f028d33d640 1 -- 172.21.15.29:0/603725167 shutdown_connections 2024-04-25T00:32:02.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f028d33d640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02880cdb20 0x7f028814f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f028d33d640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02740678f0 0x7f0274069db0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f028d33d640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02881f7230 0x7f028815fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f028d33d640 1 --2- 172.21.15.29:0/603725167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02700039f0 0x7f02880ae4b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f028d33d640 1 -- 172.21.15.29:0/603725167 >> 172.21.15.29:0/603725167 conn(0x7f02880b1870 msgr2=0x7f0288158590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:02.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f028d33d640 1 -- 172.21.15.29:0/603725167 shutdown_connections 2024-04-25T00:32:02.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.384+0000 7f028d33d640 1 -- 172.21.15.29:0/603725167 wait complete. 2024-04-25T00:32:02.406 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:02.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f6425d15640 1 Processor -- start 2024-04-25T00:32:02.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f6425d15640 1 -- start start 2024-04-25T00:32:02.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f6425d15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f642015e200 0x7f64200d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f6425d15640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f64200d2c60 0x7f64200b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f6425d15640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64200cfc10 con 0x7f64200cf690 2024-04-25T00:32:02.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f6425d15640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f64200b38d0 con 0x7f64200d2c60 2024-04-25T00:32:02.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f6425d15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f64200cfef0 con 0x7f642015e200 2024-04-25T00:32:02.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f6417fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f64200cf690 0x7f64200d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39620/0 (socket says 172.21.15.29:39620) 2024-04-25T00:32:02.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f6417fff640 1 -- 172.21.15.29:0/3007815334 learned_addr learned my addr 172.21.15.29:0/3007815334 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:02.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f641ffff640 1 --2- 172.21.15.29:0/3007815334 >> v2:172.21.15.29:3300/0 conn(0x7f64200d2c60 0x7f64200b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f641f7fe640 1 --2- 172.21.15.29:0/3007815334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f642015e200 0x7f64200d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f641effd640 1 -- 172.21.15.29:0/3007815334 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3492427850 0 0) 0x7f64200cfc10 con 0x7f64200cf690 2024-04-25T00:32:02.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f641effd640 1 -- 172.21.15.29:0/3007815334 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f640c003530 con 0x7f64200cf690 2024-04-25T00:32:02.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f641ffff640 1 -- 172.21.15.29:0/3007815334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f642015e200 msgr2=0x7f64200d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:02.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f641ffff640 1 --2- 172.21.15.29:0/3007815334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f642015e200 0x7f64200d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f641ffff640 1 -- 172.21.15.29:0/3007815334 >> v1:172.21.15.29:6789/0 conn(0x7f64200cf690 legacy=0x7f64200d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.528+0000 7f641ffff640 1 -- 172.21.15.29:0/3007815334 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64200b3b20 con 0x7f64200d2c60 2024-04-25T00:32:02.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f641ffff640 1 --2- 172.21.15.29:0/3007815334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64200d2c60 0x7f64200b4320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6418009d90 tx=0x7f641801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09cda44110b4a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:02.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f641effd640 1 -- 172.21.15.29:0/3007815334 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64180b2070 con 0x7f64200d2c60 2024-04-25T00:32:02.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f641effd640 1 -- 172.21.15.29:0/3007815334 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64180a5a20 con 0x7f64200d2c60 2024-04-25T00:32:02.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f641effd640 1 -- 172.21.15.29:0/3007815334 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64180a5df0 con 0x7f64200d2c60 2024-04-25T00:32:02.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 -- 172.21.15.29:0/3007815334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64200d2c60 msgr2=0x7f64200b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 --2- 172.21.15.29:0/3007815334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64200d2c60 0x7f64200b4320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6418009d90 tx=0x7f641801c940 comp rx=0 tx=0).stop 2024-04-25T00:32:02.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 -- 172.21.15.29:0/3007815334 shutdown_connections 2024-04-25T00:32:02.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 --2- 172.21.15.29:0/3007815334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64200d2c60 0x7f64200b4320 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 --2- 172.21.15.29:0/3007815334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f642015e200 0x7f64200d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 -- 172.21.15.29:0/3007815334 >> 172.21.15.29:0/3007815334 conn(0x7f642009d280 msgr2=0x7f64200d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:02.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 -- 172.21.15.29:0/3007815334 shutdown_connections 2024-04-25T00:32:02.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 -- 172.21.15.29:0/3007815334 wait complete. 2024-04-25T00:32:02.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 Processor -- start 2024-04-25T00:32:02.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 -- start start 2024-04-25T00:32:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64200cf690 0x7f6420159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64200d2c60 0x7f64200b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642015e200 0x7f6420158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f64200b51f0 con 0x7f64200d2c60 2024-04-25T00:32:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f64200b5070 con 0x7f642015e200 2024-04-25T00:32:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6425d15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f64200b5370 con 0x7f64200cf690 2024-04-25T00:32:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6417fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64200d2c60 0x7f64200b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6417fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64200d2c60 0x7f64200b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41202/0 (socket says 172.21.15.29:41202) 2024-04-25T00:32:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f6417fff640 1 -- 172.21.15.29:0/4035128568 learned_addr learned my addr 172.21.15.29:0/4035128568 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.532+0000 7f641f7fe640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64200cf690 0x7f6420159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f6417fff640 1 -- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64200cf690 msgr2=0x7f6420159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f641ffff640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642015e200 0x7f6420158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f6417fff640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64200cf690 0x7f6420159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f6417fff640 1 -- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642015e200 msgr2=0x7f6420158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f6417fff640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642015e200 0x7f6420158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f6417fff640 1 -- 172.21.15.29:0/4035128568 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64200d3560 con 0x7f64200d2c60 2024-04-25T00:32:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f6417fff640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64200d2c60 0x7f64200b5a50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f640800c500 tx=0x7f640800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f641cff9640 1 -- 172.21.15.29:0/4035128568 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408017070 con 0x7f64200d2c60 2024-04-25T00:32:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f641cff9640 1 -- 172.21.15.29:0/4035128568 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6408005cf0 con 0x7f64200d2c60 2024-04-25T00:32:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f6425d15640 1 -- 172.21.15.29:0/4035128568 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64200ce0b0 con 0x7f64200d2c60 2024-04-25T00:32:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f641cff9640 1 -- 172.21.15.29:0/4035128568 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64080072f0 con 0x7f64200d2c60 2024-04-25T00:32:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f6425d15640 1 -- 172.21.15.29:0/4035128568 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6420158c70 con 0x7f64200d2c60 2024-04-25T00:32:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f6425d15640 1 -- 172.21.15.29:0/4035128568 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6420069270 con 0x7f64200d2c60 2024-04-25T00:32:02.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f641cff9640 1 -- 172.21.15.29:0/4035128568 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f640801a070 con 0x7f64200d2c60 2024-04-25T00:32:02.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f641cff9640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f64000678c0 0x7f6400069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f641cff9640 1 -- 172.21.15.29:0/4035128568 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6408084ef0 con 0x7f64200d2c60 2024-04-25T00:32:02.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f641f7fe640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f64000678c0 0x7f6400069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.536+0000 7f641f7fe640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f64000678c0 0x7f6400069d80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6410009a50 tx=0x7f6410008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:02.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.540+0000 7f641cff9640 1 -- 172.21.15.29:0/4035128568 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6408055470 con 0x7f64200d2c60 2024-04-25T00:32:02.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.704+0000 7f6425d15640 1 -- 172.21.15.29:0/4035128568 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6420067900 con 0x7f64200d2c60 2024-04-25T00:32:02.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.704+0000 7f641cff9640 1 -- 172.21.15.29:0/4035128568 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f6408059770 con 0x7f64200d2c60 2024-04-25T00:32:02.707 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:02.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.708+0000 7f6425d15640 1 -- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f64000678c0 msgr2=0x7f6400069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.708+0000 7f6425d15640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f64000678c0 0x7f6400069d80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6410009a50 tx=0x7f6410008040 comp rx=0 tx=0).stop 2024-04-25T00:32:02.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.708+0000 7f6425d15640 1 -- 172.21.15.29:0/4035128568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64200d2c60 msgr2=0x7f64200b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.708+0000 7f6425d15640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64200d2c60 0x7f64200b5a50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f640800c500 tx=0x7f640800c9c0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.708+0000 7f6425d15640 1 -- 172.21.15.29:0/4035128568 shutdown_connections 2024-04-25T00:32:02.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.708+0000 7f6425d15640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f64000678c0 0x7f6400069d80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.708+0000 7f6425d15640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f642015e200 0x7f6420158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.708+0000 7f6425d15640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64200d2c60 0x7f64200b5a50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.708+0000 7f6425d15640 1 --2- 172.21.15.29:0/4035128568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64200cf690 0x7f6420159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.708+0000 7f6425d15640 1 -- 172.21.15.29:0/4035128568 >> 172.21.15.29:0/4035128568 conn(0x7f642009d280 msgr2=0x7f642015cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:02.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.708+0000 7f6425d15640 1 -- 172.21.15.29:0/4035128568 shutdown_connections 2024-04-25T00:32:02.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.708+0000 7f6425d15640 1 -- 172.21.15.29:0/4035128568 wait complete. 2024-04-25T00:32:02.727 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:02.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3d1618640 1 Processor -- start 2024-04-25T00:32:02.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3d1618640 1 -- start start 2024-04-25T00:32:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3d1618640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc3cc15e440 0x7fc3cc0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3d1618640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3cc0cc3b0 con 0x7fc3cc0d0ff0 2024-04-25T00:32:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3d1618640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3cc0b11d0 con 0x7fc3cc06c950 2024-04-25T00:32:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3d1618640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc3cc0cc650 con 0x7fc3cc15e440 2024-04-25T00:32:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3c27fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc3cc15e440 0x7fc3cc0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3cb7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc3cc0d0ff0 0x7fc3cc0cd410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39632/0 (socket says 172.21.15.29:39632) 2024-04-25T00:32:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3cb7fe640 1 -- 172.21.15.29:0/1582174891 learned_addr learned my addr 172.21.15.29:0/1582174891 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3c27fc640 1 -- 172.21.15.29:0/1582174891 >> v1:172.21.15.157:6789/0 conn(0x7fc3cc06c950 legacy=0x7fc3cc06cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3c27fc640 1 -- 172.21.15.29:0/1582174891 >> v1:172.21.15.29:6789/0 conn(0x7fc3cc0d0ff0 legacy=0x7fc3cc0cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3c27fc640 1 -- 172.21.15.29:0/1582174891 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc0cc930 con 0x7fc3cc15e440 2024-04-25T00:32:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3ca7fc640 1 -- 172.21.15.29:0/1582174891 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2058775489 0 0) 0x7fc3cc0cc3b0 con 0x7fc3cc0d0ff0 2024-04-25T00:32:02.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.844+0000 7fc3c27fc640 1 --2- 172.21.15.29:0/1582174891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc15e440 0x7fc3cc0d0ab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40097c0 tx=0x7fc3b401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8daa22b305333d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:02.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3ca7fc640 1 -- 172.21.15.29:0/1582174891 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40b0070 con 0x7fc3cc15e440 2024-04-25T00:32:02.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3ca7fc640 1 -- 172.21.15.29:0/1582174891 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b4004230 con 0x7fc3cc15e440 2024-04-25T00:32:02.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3ca7fc640 1 -- 172.21.15.29:0/1582174891 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4003620 con 0x7fc3cc15e440 2024-04-25T00:32:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 -- 172.21.15.29:0/1582174891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc15e440 msgr2=0x7fc3cc0d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 --2- 172.21.15.29:0/1582174891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc15e440 0x7fc3cc0d0ab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40097c0 tx=0x7fc3b401c460 comp rx=0 tx=0).stop 2024-04-25T00:32:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 -- 172.21.15.29:0/1582174891 shutdown_connections 2024-04-25T00:32:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 --2- 172.21.15.29:0/1582174891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc15e440 0x7fc3cc0d0ab0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 -- 172.21.15.29:0/1582174891 >> 172.21.15.29:0/1582174891 conn(0x7fc3cc0d1d60 msgr2=0x7fc3cc0d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 -- 172.21.15.29:0/1582174891 shutdown_connections 2024-04-25T00:32:02.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 -- 172.21.15.29:0/1582174891 wait complete. 2024-04-25T00:32:02.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 Processor -- start 2024-04-25T00:32:02.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 -- start start 2024-04-25T00:32:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc06c950 0x7fc3cc0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3cc0d0ff0 0x7fc3cc070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc15e440 0x7fc3cc0c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc16bfb0 con 0x7fc3cc06c950 2024-04-25T00:32:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc16be30 con 0x7fc3cc15e440 2024-04-25T00:32:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc3cc16c130 con 0x7fc3cc0d0ff0 2024-04-25T00:32:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3caffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc06c950 0x7fc3cc0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3caffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc06c950 0x7fc3cc0cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41218/0 (socket says 172.21.15.29:41218) 2024-04-25T00:32:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3caffd640 1 -- 172.21.15.29:0/4281458169 learned_addr learned my addr 172.21.15.29:0/4281458169 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3c27fc640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3cc0d0ff0 0x7fc3cc070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3cb7fe640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc15e440 0x7fc3cc0c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3caffd640 1 -- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3cc0d0ff0 msgr2=0x7fc3cc070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3caffd640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3cc0d0ff0 0x7fc3cc070490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3caffd640 1 -- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc15e440 msgr2=0x7fc3cc0c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3caffd640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc15e440 0x7fc3cc0c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3caffd640 1 -- 172.21.15.29:0/4281458169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc0b3150 con 0x7fc3cc06c950 2024-04-25T00:32:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3cb7fe640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc15e440 0x7fc3cc0c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3caffd640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc06c950 0x7fc3cc0cbb80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00b410 tx=0x7fc3bc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3c3fff640 1 -- 172.21.15.29:0/4281458169 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc015030 con 0x7fc3cc06c950 2024-04-25T00:32:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3c3fff640 1 -- 172.21.15.29:0/4281458169 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3bc004020 con 0x7fc3cc06c950 2024-04-25T00:32:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 -- 172.21.15.29:0/4281458169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3cc0b3430 con 0x7fc3cc06c950 2024-04-25T00:32:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3c3fff640 1 -- 172.21.15.29:0/4281458169 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc004450 con 0x7fc3cc06c950 2024-04-25T00:32:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.848+0000 7fc3d1618640 1 -- 172.21.15.29:0/4281458169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3cc150cd0 con 0x7fc3cc06c950 2024-04-25T00:32:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.852+0000 7fc3d1618640 1 -- 172.21.15.29:0/4281458169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3cc069270 con 0x7fc3cc06c950 2024-04-25T00:32:02.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.852+0000 7fc3c3fff640 1 -- 172.21.15.29:0/4281458169 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc3bc018070 con 0x7fc3cc06c950 2024-04-25T00:32:02.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.852+0000 7fc3c3fff640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc3ac0678c0 0x7fc3ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:02.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.852+0000 7fc3c3fff640 1 -- 172.21.15.29:0/4281458169 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc3bc085960 con 0x7fc3cc06c950 2024-04-25T00:32:02.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.852+0000 7fc3c27fc640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc3ac0678c0 0x7fc3ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:02.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.852+0000 7fc3c27fc640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc3ac0678c0 0x7fc3ac069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40ad000 tx=0x7fc3b40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:02.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:02.856+0000 7fc3c3fff640 1 -- 172.21.15.29:0/4281458169 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc3bc056400 con 0x7fc3cc06c950 2024-04-25T00:32:03.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.016+0000 7fc3d1618640 1 -- 172.21.15.29:0/4281458169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3cc0d03b0 con 0x7fc3cc06c950 2024-04-25T00:32:03.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.020+0000 7fc3c3fff640 1 -- 172.21.15.29:0/4281458169 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fc3bc055d00 con 0x7fc3cc06c950 2024-04-25T00:32:03.022 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:03.022 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:03.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.020+0000 7fc3d1618640 1 -- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc3ac0678c0 msgr2=0x7fc3ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.020+0000 7fc3d1618640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc3ac0678c0 0x7fc3ac069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40ad000 tx=0x7fc3b40a6000 comp rx=0 tx=0).stop 2024-04-25T00:32:03.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.020+0000 7fc3d1618640 1 -- 172.21.15.29:0/4281458169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc06c950 msgr2=0x7fc3cc0cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.020+0000 7fc3d1618640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc06c950 0x7fc3cc0cbb80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00b410 tx=0x7fc3bc00b8d0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.020+0000 7fc3d1618640 1 -- 172.21.15.29:0/4281458169 shutdown_connections 2024-04-25T00:32:03.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.020+0000 7fc3d1618640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc3ac0678c0 0x7fc3ac069d80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.020+0000 7fc3d1618640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc15e440 0x7fc3cc0c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.020+0000 7fc3d1618640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3cc0d0ff0 0x7fc3cc070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.020+0000 7fc3d1618640 1 --2- 172.21.15.29:0/4281458169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc06c950 0x7fc3cc0cbb80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.020+0000 7fc3d1618640 1 -- 172.21.15.29:0/4281458169 >> 172.21.15.29:0/4281458169 conn(0x7fc3cc0d1d60 msgr2=0x7fc3cc158850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:03.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.020+0000 7fc3d1618640 1 -- 172.21.15.29:0/4281458169 shutdown_connections 2024-04-25T00:32:03.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.020+0000 7fc3d1618640 1 -- 172.21.15.29:0/4281458169 wait complete. 2024-04-25T00:32:03.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:03.040 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:03.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda4503c640 1 Processor -- start 2024-04-25T00:32:03.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda4503c640 1 -- start start 2024-04-25T00:32:03.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda4503c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fda400cb330 0x7fda400cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda4503c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda4006e1a0 con 0x7fda4015bd60 2024-04-25T00:32:03.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda4503c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda4006e480 con 0x7fda4006dd10 2024-04-25T00:32:03.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda4503c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fda4006edb0 con 0x7fda400cb330 2024-04-25T00:32:03.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda3ed9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fda4015bd60 0x7fda4006b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39644/0 (socket says 172.21.15.29:39644) 2024-04-25T00:32:03.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda3ed9d640 1 -- 172.21.15.29:0/320812181 learned_addr learned my addr 172.21.15.29:0/320812181 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:03.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda3f59e640 1 --2- 172.21.15.29:0/320812181 >> v2:172.21.15.29:3300/0 conn(0x7fda400cb330 0x7fda400cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda3dd9b640 1 -- 172.21.15.29:0/320812181 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865314843 0 0) 0x7fda4006e1a0 con 0x7fda4015bd60 2024-04-25T00:32:03.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda3dd9b640 1 -- 172.21.15.29:0/320812181 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda28003530 con 0x7fda4015bd60 2024-04-25T00:32:03.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda3f59e640 1 -- 172.21.15.29:0/320812181 >> v1:172.21.15.157:6789/0 conn(0x7fda4006dd10 legacy=0x7fda400cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:03.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda3f59e640 1 -- 172.21.15.29:0/320812181 >> v1:172.21.15.29:6789/0 conn(0x7fda4015bd60 legacy=0x7fda4006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.156+0000 7fda3f59e640 1 -- 172.21.15.29:0/320812181 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4006e760 con 0x7fda400cb330 2024-04-25T00:32:03.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3f59e640 1 --2- 172.21.15.29:0/320812181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda400cb330 0x7fda400cd190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda30009d90 tx=0x7fda3001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d9824f488f671b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:03.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3dd9b640 1 -- 172.21.15.29:0/320812181 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda300b2070 con 0x7fda400cb330 2024-04-25T00:32:03.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3dd9b640 1 -- 172.21.15.29:0/320812181 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda300a5a20 con 0x7fda400cb330 2024-04-25T00:32:03.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3dd9b640 1 -- 172.21.15.29:0/320812181 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda300a5df0 con 0x7fda400cb330 2024-04-25T00:32:03.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 -- 172.21.15.29:0/320812181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda400cb330 msgr2=0x7fda400cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 --2- 172.21.15.29:0/320812181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda400cb330 0x7fda400cd190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda30009d90 tx=0x7fda3001c940 comp rx=0 tx=0).stop 2024-04-25T00:32:03.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 -- 172.21.15.29:0/320812181 shutdown_connections 2024-04-25T00:32:03.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 --2- 172.21.15.29:0/320812181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda400cb330 0x7fda400cd190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 -- 172.21.15.29:0/320812181 >> 172.21.15.29:0/320812181 conn(0x7fda4006c320 msgr2=0x7fda4006c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:03.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 -- 172.21.15.29:0/320812181 shutdown_connections 2024-04-25T00:32:03.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 -- 172.21.15.29:0/320812181 wait complete. 2024-04-25T00:32:03.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 Processor -- start 2024-04-25T00:32:03.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 -- start start 2024-04-25T00:32:03.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda400cb330 0x7fda400c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda4015bd60 0x7fda401570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda40157630 0x7fda40158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fda40072bc0 con 0x7fda40157630 2024-04-25T00:32:03.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda4006bb30 con 0x7fda400cb330 2024-04-25T00:32:03.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda4503c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fda40072d40 con 0x7fda4015bd60 2024-04-25T00:32:03.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3f59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda40157630 0x7fda40158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3f59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda40157630 0x7fda40158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41236/0 (socket says 172.21.15.29:41236) 2024-04-25T00:32:03.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3f59e640 1 -- 172.21.15.29:0/2360855610 learned_addr learned my addr 172.21.15.29:0/2360855610 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:03.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3f59e640 1 -- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda4015bd60 msgr2=0x7fda401570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3e59c640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda4015bd60 0x7fda401570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3ed9d640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda400cb330 0x7fda400c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3f59e640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda4015bd60 0x7fda401570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3f59e640 1 -- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda400cb330 msgr2=0x7fda400c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3f59e640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda400cb330 0x7fda400c6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3f59e640 1 -- 172.21.15.29:0/2360855610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4015adf0 con 0x7fda40157630 2024-04-25T00:32:03.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.160+0000 7fda3e59c640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda4015bd60 0x7fda401570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:03.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.164+0000 7fda3f59e640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda40157630 0x7fda40158dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fda3001a570 tx=0x7fda300196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:03.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.164+0000 7fda277fe640 1 -- 172.21.15.29:0/2360855610 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda300b2040 con 0x7fda40157630 2024-04-25T00:32:03.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.164+0000 7fda277fe640 1 -- 172.21.15.29:0/2360855610 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda300bf070 con 0x7fda40157630 2024-04-25T00:32:03.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.164+0000 7fda277fe640 1 -- 172.21.15.29:0/2360855610 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda300b8870 con 0x7fda40157630 2024-04-25T00:32:03.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.164+0000 7fda4503c640 1 -- 172.21.15.29:0/2360855610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda40157d50 con 0x7fda40157630 2024-04-25T00:32:03.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.164+0000 7fda4503c640 1 -- 172.21.15.29:0/2360855610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda401582f0 con 0x7fda40157630 2024-04-25T00:32:03.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.164+0000 7fda4503c640 1 -- 172.21.15.29:0/2360855610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda40069270 con 0x7fda40157630 2024-04-25T00:32:03.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.164+0000 7fda277fe640 1 -- 172.21.15.29:0/2360855610 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fda300b8a10 con 0x7fda40157630 2024-04-25T00:32:03.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.164+0000 7fda277fe640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fda2c0678f0 0x7fda2c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.164+0000 7fda277fe640 1 -- 172.21.15.29:0/2360855610 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fda30120910 con 0x7fda40157630 2024-04-25T00:32:03.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.164+0000 7fda3ed9d640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fda2c0678f0 0x7fda2c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.164+0000 7fda3ed9d640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fda2c0678f0 0x7fda2c069db0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda380011f0 tx=0x7fda38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:03.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.168+0000 7fda277fe640 1 -- 172.21.15.29:0/2360855610 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fda300c8020 con 0x7fda40157630 2024-04-25T00:32:03.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.324+0000 7fda4503c640 1 -- 172.21.15.29:0/2360855610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda4006c0e0 con 0x7fda40157630 2024-04-25T00:32:03.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.324+0000 7fda277fe640 1 -- 172.21.15.29:0/2360855610 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fda30016560 con 0x7fda40157630 2024-04-25T00:32:03.327 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:03.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.328+0000 7fda4503c640 1 -- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fda2c0678f0 msgr2=0x7fda2c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.328+0000 7fda4503c640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fda2c0678f0 0x7fda2c069db0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda380011f0 tx=0x7fda38009040 comp rx=0 tx=0).stop 2024-04-25T00:32:03.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.328+0000 7fda4503c640 1 -- 172.21.15.29:0/2360855610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda40157630 msgr2=0x7fda40158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.328+0000 7fda4503c640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda40157630 0x7fda40158dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fda3001a570 tx=0x7fda300196e0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.328+0000 7fda4503c640 1 -- 172.21.15.29:0/2360855610 shutdown_connections 2024-04-25T00:32:03.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.328+0000 7fda4503c640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda40157630 0x7fda40158dd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.328+0000 7fda4503c640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda4015bd60 0x7fda401570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.328+0000 7fda4503c640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fda2c0678f0 0x7fda2c069db0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.328+0000 7fda4503c640 1 --2- 172.21.15.29:0/2360855610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda400cb330 0x7fda400c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.328+0000 7fda4503c640 1 -- 172.21.15.29:0/2360855610 >> 172.21.15.29:0/2360855610 conn(0x7fda4006c320 msgr2=0x7fda40067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:03.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.328+0000 7fda4503c640 1 -- 172.21.15.29:0/2360855610 shutdown_connections 2024-04-25T00:32:03.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.328+0000 7fda4503c640 1 -- 172.21.15.29:0/2360855610 wait complete. 2024-04-25T00:32:03.345 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:03.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.472+0000 7fc522e80640 1 Processor -- start 2024-04-25T00:32:03.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.472+0000 7fc522e80640 1 -- start start 2024-04-25T00:32:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc522e80640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc51c0d1a10 0x7fc51c06c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc522e80640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0b12d0 0x7fc51c0d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc522e80640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc51c0ce650 0x7fc51c0ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc522e80640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc51c0b1730 con 0x7fc51c0d1a10 2024-04-25T00:32:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc522e80640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc51c0d12a0 con 0x7fc51c0ce650 2024-04-25T00:32:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc522e80640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc51c0d2330 con 0x7fc51c0b12d0 2024-04-25T00:32:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc520c1c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc51c0d1a10 0x7fc51c06c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc520c1c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc51c0d1a10 0x7fc51c06c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41250/0 (socket says 172.21.15.29:41250) 2024-04-25T00:32:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc520c1c640 1 -- 172.21.15.29:0/105499381 learned_addr learned my addr 172.21.15.29:0/105499381 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc51bfff640 1 --2- 172.21.15.29:0/105499381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0b12d0 0x7fc51c0d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc52141d640 1 --2- 172.21.15.29:0/105499381 >> v2:172.21.15.157:3300/0 conn(0x7fc51c0ce650 0x7fc51c0ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc520c1c640 1 -- 172.21.15.29:0/105499381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0b12d0 msgr2=0x7fc51c0d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc520c1c640 1 --2- 172.21.15.29:0/105499381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0b12d0 0x7fc51c0d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc520c1c640 1 -- 172.21.15.29:0/105499381 >> v2:172.21.15.157:3300/0 conn(0x7fc51c0ce650 msgr2=0x7fc51c0ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc520c1c640 1 --2- 172.21.15.29:0/105499381 >> v2:172.21.15.157:3300/0 conn(0x7fc51c0ce650 0x7fc51c0ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc520c1c640 1 -- 172.21.15.29:0/105499381 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c06ced0 con 0x7fc51c0d1a10 2024-04-25T00:32:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc520c1c640 1 --2- 172.21.15.29:0/105499381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0d1a10 0x7fc51c06c530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0cf810 tx=0x7fc51001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ed5b586c5f341 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.476+0000 7fc51b7fe640 1 -- 172.21.15.29:0/105499381 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5100aa610 con 0x7fc51c0d1a10 2024-04-25T00:32:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc51b7fe640 1 -- 172.21.15.29:0/105499381 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5100af070 con 0x7fc51c0d1a10 2024-04-25T00:32:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc51b7fe640 1 -- 172.21.15.29:0/105499381 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5100aa9d0 con 0x7fc51c0d1a10 2024-04-25T00:32:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 -- 172.21.15.29:0/105499381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0d1a10 msgr2=0x7fc51c06c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 --2- 172.21.15.29:0/105499381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0d1a10 0x7fc51c06c530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0cf810 tx=0x7fc51001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 -- 172.21.15.29:0/105499381 shutdown_connections 2024-04-25T00:32:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 --2- 172.21.15.29:0/105499381 >> v2:172.21.15.157:3300/0 conn(0x7fc51c0ce650 0x7fc51c0ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 --2- 172.21.15.29:0/105499381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0b12d0 0x7fc51c0d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 --2- 172.21.15.29:0/105499381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0d1a10 0x7fc51c06c530 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 -- 172.21.15.29:0/105499381 >> 172.21.15.29:0/105499381 conn(0x7fc51c09aa30 msgr2=0x7fc51c09ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:03.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 -- 172.21.15.29:0/105499381 shutdown_connections 2024-04-25T00:32:03.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 -- 172.21.15.29:0/105499381 wait complete. 2024-04-25T00:32:03.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 Processor -- start 2024-04-25T00:32:03.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 -- start start 2024-04-25T00:32:03.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0b12d0 0x7fc51c0b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0ce650 0x7fc51c1f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51c0d1a10 0x7fc51c0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c15e5b0 con 0x7fc51c0b12d0 2024-04-25T00:32:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c0d0d80 con 0x7fc51c0d1a10 2024-04-25T00:32:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc51c15e730 con 0x7fc51c0ce650 2024-04-25T00:32:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc520c1c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0b12d0 0x7fc51c0b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc520c1c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0b12d0 0x7fc51c0b72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41266/0 (socket says 172.21.15.29:41266) 2024-04-25T00:32:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc520c1c640 1 -- 172.21.15.29:0/2803158890 learned_addr learned my addr 172.21.15.29:0/2803158890 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc51bfff640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0ce650 0x7fc51c1f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc52141d640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51c0d1a10 0x7fc51c0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc520c1c640 1 -- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0ce650 msgr2=0x7fc51c1f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc520c1c640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0ce650 0x7fc51c1f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc520c1c640 1 -- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51c0d1a10 msgr2=0x7fc51c0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc520c1c640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51c0d1a10 0x7fc51c0c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc520c1c640 1 -- 172.21.15.29:0/2803158890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c1f7b90 con 0x7fc51c0b12d0 2024-04-25T00:32:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc520c1c640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0b12d0 0x7fc51c0b72e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc51001aba0 tx=0x7fc51001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc5197fa640 1 -- 172.21.15.29:0/2803158890 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5100bd5f0 con 0x7fc51c0b12d0 2024-04-25T00:32:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc5197fa640 1 -- 172.21.15.29:0/2803158890 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5100bd790 con 0x7fc51c0b12d0 2024-04-25T00:32:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc5197fa640 1 -- 172.21.15.29:0/2803158890 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5100bdb80 con 0x7fc51c0b12d0 2024-04-25T00:32:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 -- 172.21.15.29:0/2803158890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51c1f7e70 con 0x7fc51c0b12d0 2024-04-25T00:32:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.480+0000 7fc522e80640 1 -- 172.21.15.29:0/2803158890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51c0c5aa0 con 0x7fc51c0b12d0 2024-04-25T00:32:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.484+0000 7fc522e80640 1 -- 172.21.15.29:0/2803158890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc51c1f8390 con 0x7fc51c0b12d0 2024-04-25T00:32:03.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.484+0000 7fc5197fa640 1 -- 172.21.15.29:0/2803158890 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc5100b33f0 con 0x7fc51c0b12d0 2024-04-25T00:32:03.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.484+0000 7fc5197fa640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc5000678c0 0x7fc500069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.484+0000 7fc5197fa640 1 -- 172.21.15.29:0/2803158890 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc5101206f0 con 0x7fc51c0b12d0 2024-04-25T00:32:03.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.484+0000 7fc51bfff640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc5000678c0 0x7fc500069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.484+0000 7fc51bfff640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc5000678c0 0x7fc500069d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0afc30 tx=0x7fc508008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.488+0000 7fc5197fa640 1 -- 172.21.15.29:0/2803158890 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc51c1f8390 con 0x7fc51c0b12d0 2024-04-25T00:32:03.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.652+0000 7fc522e80640 1 -- 172.21.15.29:0/2803158890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc51c069270 con 0x7fc51c0b12d0 2024-04-25T00:32:03.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.652+0000 7fc5197fa640 1 -- 172.21.15.29:0/2803158890 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc5100a7070 con 0x7fc51c0b12d0 2024-04-25T00:32:03.657 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:03.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.656+0000 7fc522e80640 1 -- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc5000678c0 msgr2=0x7fc500069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.656+0000 7fc522e80640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc5000678c0 0x7fc500069d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0afc30 tx=0x7fc508008040 comp rx=0 tx=0).stop 2024-04-25T00:32:03.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.656+0000 7fc522e80640 1 -- 172.21.15.29:0/2803158890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0b12d0 msgr2=0x7fc51c0b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.656+0000 7fc522e80640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0b12d0 0x7fc51c0b72e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc51001aba0 tx=0x7fc51001ad30 comp rx=0 tx=0).stop 2024-04-25T00:32:03.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.656+0000 7fc522e80640 1 -- 172.21.15.29:0/2803158890 shutdown_connections 2024-04-25T00:32:03.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.656+0000 7fc522e80640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc5000678c0 0x7fc500069d80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.656+0000 7fc522e80640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51c0d1a10 0x7fc51c0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.656+0000 7fc522e80640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0ce650 0x7fc51c1f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.656+0000 7fc522e80640 1 --2- 172.21.15.29:0/2803158890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0b12d0 0x7fc51c0b72e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.656+0000 7fc522e80640 1 -- 172.21.15.29:0/2803158890 >> 172.21.15.29:0/2803158890 conn(0x7fc51c09aa30 msgr2=0x7fc51c159330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:03.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.656+0000 7fc522e80640 1 -- 172.21.15.29:0/2803158890 shutdown_connections 2024-04-25T00:32:03.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.656+0000 7fc522e80640 1 -- 172.21.15.29:0/2803158890 wait complete. 2024-04-25T00:32:03.676 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:03.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ec6c640 1 Processor -- start 2024-04-25T00:32:03.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ec6c640 1 -- start start 2024-04-25T00:32:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ec6c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4f68156580 0x7f4f6806b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ec6c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f6815cc80 0x7f4f6806b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ec6c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f6815d2a0 con 0x7f4f68160820 2024-04-25T00:32:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ec6c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4f6815fd40 con 0x7f4f68156580 2024-04-25T00:32:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ec6c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4f68160020 con 0x7f4f6815cc80 2024-04-25T00:32:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f67fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4f68160820 0x7f4f6815c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39648/0 (socket says 172.21.15.29:39648) 2024-04-25T00:32:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f67fff640 1 -- 172.21.15.29:0/2934417074 learned_addr learned my addr 172.21.15.29:0/2934417074 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6d209640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f6815cc80 0x7f4f6806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ca08640 1 --2- 172.21.15.29:0/2934417074 >> v2:172.21.15.157:3300/0 conn(0x7f4f68156580 0x7f4f6806b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1147190727 0 0) 0x7f4f6815d2a0 con 0x7f4f68160820 2024-04-25T00:32:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f50003530 con 0x7f4f68160820 2024-04-25T00:32:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 763679457 0 0) 0x7f4f50003530 con 0x7f4f68160820 2024-04-25T00:32:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f6815cc80 msgr2=0x7f4f6806b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f6815cc80 0x7f4f6806b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 >> v2:172.21.15.157:3300/0 conn(0x7f4f68156580 msgr2=0x7f4f6806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 --2- 172.21.15.29:0/2934417074 >> v2:172.21.15.157:3300/0 conn(0x7f4f68156580 0x7f4f6806b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f68156960 con 0x7f4f68160820 2024-04-25T00:32:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4f5c001680 con 0x7f4f68160820 2024-04-25T00:32:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 >> v1:172.21.15.29:6789/0 conn(0x7f4f68160820 legacy=0x7f4f6815c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f500039c0 0x7f4f50003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f6815cc80 0x7f4f50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f50004d50 0x7f4f500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6815fd40 con 0x7f4f500039c0 2024-04-25T00:32:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f68160020 con 0x7f4f50004d50 2024-04-25T00:32:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5000b9d0 con 0x7f4f6815cc80 2024-04-25T00:32:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ca08640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f500039c0 0x7f4f50003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f67fff640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f6815cc80 0x7f4f50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6d209640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f50004d50 0x7f4f500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ca08640 1 -- 172.21.15.29:0/2934417074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f6815cc80 msgr2=0x7f4f50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ca08640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f6815cc80 0x7f4f50004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ca08640 1 -- 172.21.15.29:0/2934417074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f50004d50 msgr2=0x7f4f500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ca08640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f50004d50 0x7f4f500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6ca08640 1 -- 172.21.15.29:0/2934417074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f50003530 con 0x7f4f500039c0 2024-04-25T00:32:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f6d209640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f50004d50 0x7f4f500051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.800+0000 7f4f67fff640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f6815cc80 0x7f4f50004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:03.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.804+0000 7f4f6ca08640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f500039c0 0x7f4f50003de0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009fc0 tx=0x7f4f5801cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8388b766fb96208 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:03.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.804+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f580b2070 con 0x7f4f500039c0 2024-04-25T00:32:03.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.804+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f580a5850 con 0x7f4f500039c0 2024-04-25T00:32:03.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.804+0000 7f4f677fe640 1 -- 172.21.15.29:0/2934417074 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f58009c10 con 0x7f4f500039c0 2024-04-25T00:32:03.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.804+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2934417074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f500039c0 msgr2=0x7f4f50003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.804+0000 7f4f6ec6c640 1 --2- 172.21.15.29:0/2934417074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f500039c0 0x7f4f50003de0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009fc0 tx=0x7f4f5801cae0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.804+0000 7f4f6d209640 1 -- 172.21.15.29:0/2934417074 reap_dead start 2024-04-25T00:32:03.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.804+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2934417074 shutdown_connections 2024-04-25T00:32:03.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.804+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2934417074 >> 172.21.15.29:0/2934417074 conn(0x7f4f680d0c50 msgr2=0x7f4f680d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:03.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.804+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2934417074 shutdown_connections 2024-04-25T00:32:03.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.804+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2934417074 wait complete. 2024-04-25T00:32:03.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.804+0000 7f4f6ec6c640 1 Processor -- start 2024-04-25T00:32:03.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.804+0000 7f4f6ec6c640 1 -- start start 2024-04-25T00:32:03.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f6ec6c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f500039c0 0x7f4f6815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f6ec6c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f6813f410 0x7f4f68141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f6ec6c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f68142230 0x7f4f681426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f6ec6c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4f680cfc00 con 0x7f4f6813f410 2024-04-25T00:32:03.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f6ec6c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f680cfa80 con 0x7f4f500039c0 2024-04-25T00:32:03.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f6ec6c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4f680cfd80 con 0x7f4f68142230 2024-04-25T00:32:03.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f67fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f6813f410 0x7f4f68141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f67fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f6813f410 0x7f4f68141cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41284/0 (socket says 172.21.15.29:41284) 2024-04-25T00:32:03.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f67fff640 1 -- 172.21.15.29:0/2907469718 learned_addr learned my addr 172.21.15.29:0/2907469718 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:03.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f6ca08640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f500039c0 0x7f4f6815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f6d209640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f68142230 0x7f4f681426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f67fff640 1 -- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f68142230 msgr2=0x7f4f681426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f67fff640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f68142230 0x7f4f681426b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f67fff640 1 -- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f500039c0 msgr2=0x7f4f6815dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f67fff640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f500039c0 0x7f4f6815dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f67fff640 1 -- 172.21.15.29:0/2907469718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6815e2a0 con 0x7f4f6813f410 2024-04-25T00:32:03.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f6ca08640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f500039c0 0x7f4f6815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:03.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f67fff640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f6813f410 0x7f4f68141cf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c000f70 tx=0x7f4f5c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:03.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f657fa640 1 -- 172.21.15.29:0/2907469718 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c019070 con 0x7f4f6813f410 2024-04-25T00:32:03.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f657fa640 1 -- 172.21.15.29:0/2907469718 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5c0074e0 con 0x7f4f6813f410 2024-04-25T00:32:03.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2907469718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f6815e540 con 0x7f4f6813f410 2024-04-25T00:32:03.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f657fa640 1 -- 172.21.15.29:0/2907469718 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c004a10 con 0x7f4f6813f410 2024-04-25T00:32:03.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2907469718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f6815eb00 con 0x7f4f6813f410 2024-04-25T00:32:03.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2907469718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f68069270 con 0x7f4f6813f410 2024-04-25T00:32:03.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f657fa640 1 -- 172.21.15.29:0/2907469718 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4f5c01c070 con 0x7f4f6813f410 2024-04-25T00:32:03.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f657fa640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f540678c0 0x7f4f54069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:03.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f657fa640 1 -- 172.21.15.29:0/2907469718 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4f5c085710 con 0x7f4f6813f410 2024-04-25T00:32:03.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.808+0000 7f4f6ca08640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f540678c0 0x7f4f54069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:03.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.812+0000 7f4f6ca08640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f540678c0 0x7f4f54069d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4f580af000 tx=0x7f4f580a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:03.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.812+0000 7f4f657fa640 1 -- 172.21.15.29:0/2907469718 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4f5c0562e0 con 0x7f4f6813f410 2024-04-25T00:32:03.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.968+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2907469718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4f681533d0 con 0x7f4f6813f410 2024-04-25T00:32:03.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.968+0000 7f4f657fa640 1 -- 172.21.15.29:0/2907469718 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f4f5c055be0 con 0x7f4f6813f410 2024-04-25T00:32:03.971 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:03.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.968+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f540678c0 msgr2=0x7f4f54069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.968+0000 7f4f6ec6c640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f540678c0 0x7f4f54069d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4f580af000 tx=0x7f4f580a7040 comp rx=0 tx=0).stop 2024-04-25T00:32:03.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.968+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2907469718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f6813f410 msgr2=0x7f4f68141cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:03.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.968+0000 7f4f6ec6c640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f6813f410 0x7f4f68141cf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c000f70 tx=0x7f4f5c00dcd0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.968+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2907469718 shutdown_connections 2024-04-25T00:32:03.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.968+0000 7f4f6ec6c640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f68142230 0x7f4f681426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.968+0000 7f4f6ec6c640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f6813f410 0x7f4f68141cf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.968+0000 7f4f6ec6c640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f540678c0 0x7f4f54069d80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.968+0000 7f4f6ec6c640 1 --2- 172.21.15.29:0/2907469718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f500039c0 0x7f4f6815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:03.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.968+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2907469718 >> 172.21.15.29:0/2907469718 conn(0x7f4f680d0c50 msgr2=0x7f4f6815aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:03.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.972+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2907469718 shutdown_connections 2024-04-25T00:32:03.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:03.972+0000 7f4f6ec6c640 1 -- 172.21.15.29:0/2907469718 wait complete. 2024-04-25T00:32:04.132 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2024-04-25T00:32:04.132 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-04-25T00:32:04.133 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:04.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.260+0000 7f9dda54a640 1 Processor -- start 2024-04-25T00:32:04.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dda54a640 1 -- start start 2024-04-25T00:32:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dda54a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9dd415d980 0x7f9dd406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dda54a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9dd40b2b70 0x7f9dd406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dda54a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dd4151950 con 0x7f9dd4150df0 2024-04-25T00:32:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dda54a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd40cf560 con 0x7f9dd415d980 2024-04-25T00:32:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dda54a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd40b3130 con 0x7f9dd40b2b70 2024-04-25T00:32:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dd3fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9dd415d980 0x7f9dd406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dd8ae7640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9dd4150df0 0x7f9dd4151240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39656/0 (socket says 172.21.15.29:39656) 2024-04-25T00:32:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dd8ae7640 1 -- 172.21.15.29:0/211388479 learned_addr learned my addr 172.21.15.29:0/211388479 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dd3fff640 1 -- 172.21.15.29:0/211388479 >> v2:172.21.15.157:3300/0 conn(0x7f9dd40b2b70 msgr2=0x7f9dd406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dd3fff640 1 --2- 172.21.15.29:0/211388479 >> v2:172.21.15.157:3300/0 conn(0x7f9dd40b2b70 0x7f9dd406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dd3fff640 1 -- 172.21.15.29:0/211388479 >> v1:172.21.15.29:6789/0 conn(0x7f9dd4150df0 legacy=0x7f9dd4151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dd3fff640 1 -- 172.21.15.29:0/211388479 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd40b3410 con 0x7f9dd415d980 2024-04-25T00:32:04.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.264+0000 7f9dd37fe640 1 -- 172.21.15.29:0/211388479 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2774760653 0 0) 0x7f9dd4151950 con 0x7f9dd4150df0 2024-04-25T00:32:04.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd3fff640 1 --2- 172.21.15.29:0/211388479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd415d980 0x7f9dd406d5d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40b2720 tx=0x7f9dc401d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7460f235bff9653b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd37fe640 1 -- 172.21.15.29:0/211388479 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40b3070 con 0x7f9dd415d980 2024-04-25T00:32:04.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd37fe640 1 -- 172.21.15.29:0/211388479 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc40a6850 con 0x7f9dd415d980 2024-04-25T00:32:04.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd37fe640 1 -- 172.21.15.29:0/211388479 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc400b040 con 0x7f9dd415d980 2024-04-25T00:32:04.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 -- 172.21.15.29:0/211388479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd415d980 msgr2=0x7f9dd406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 --2- 172.21.15.29:0/211388479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd415d980 0x7f9dd406d5d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40b2720 tx=0x7f9dc401d820 comp rx=0 tx=0).stop 2024-04-25T00:32:04.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 -- 172.21.15.29:0/211388479 shutdown_connections 2024-04-25T00:32:04.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 --2- 172.21.15.29:0/211388479 >> v2:172.21.15.157:3300/0 conn(0x7f9dd40b2b70 0x7f9dd406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 --2- 172.21.15.29:0/211388479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd415d980 0x7f9dd406d5d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 -- 172.21.15.29:0/211388479 >> 172.21.15.29:0/211388479 conn(0x7f9dd40b1870 msgr2=0x7f9dd40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:04.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 -- 172.21.15.29:0/211388479 shutdown_connections 2024-04-25T00:32:04.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 -- 172.21.15.29:0/211388479 wait complete. 2024-04-25T00:32:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 Processor -- start 2024-04-25T00:32:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 -- start start 2024-04-25T00:32:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd40b2b70 0x7f9dd41467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd4150df0 0x7f9dd415bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd415d980 0x7f9dd415c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd415b4b0 con 0x7f9dd40b2b70 2024-04-25T00:32:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd415dd60 con 0x7f9dd415d980 2024-04-25T00:32:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd415b630 con 0x7f9dd4150df0 2024-04-25T00:32:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd40b2b70 0x7f9dd41467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd40b2b70 0x7f9dd41467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41304/0 (socket says 172.21.15.29:41304) 2024-04-25T00:32:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd3fff640 1 -- 172.21.15.29:0/487607195 learned_addr learned my addr 172.21.15.29:0/487607195 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dcbfff640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd4150df0 0x7f9dd415bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd3fff640 1 -- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd4150df0 msgr2=0x7f9dd415bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd3fff640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd4150df0 0x7f9dd415bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd3fff640 1 -- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd415d980 msgr2=0x7f9dd415c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd8ae7640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd415d980 0x7f9dd415c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd3fff640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd415d980 0x7f9dd415c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd3fff640 1 -- 172.21.15.29:0/487607195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd41f70e0 con 0x7f9dd40b2b70 2024-04-25T00:32:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dcbfff640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd4150df0 0x7f9dd415bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd3fff640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd40b2b70 0x7f9dd41467b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40b1040 tx=0x7f9dc401a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd17fa640 1 -- 172.21.15.29:0/487607195 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40b3040 con 0x7f9dd40b2b70 2024-04-25T00:32:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd17fa640 1 -- 172.21.15.29:0/487607195 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc40be070 con 0x7f9dd40b2b70 2024-04-25T00:32:04.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 -- 172.21.15.29:0/487607195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd41f73c0 con 0x7f9dd40b2b70 2024-04-25T00:32:04.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dd17fa640 1 -- 172.21.15.29:0/487607195 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40b9840 con 0x7f9dd40b2b70 2024-04-25T00:32:04.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.268+0000 7f9dda54a640 1 -- 172.21.15.29:0/487607195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd41f7980 con 0x7f9dd40b2b70 2024-04-25T00:32:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.272+0000 7f9dda54a640 1 -- 172.21.15.29:0/487607195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd4069270 con 0x7f9dd40b2b70 2024-04-25T00:32:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.272+0000 7f9dd17fa640 1 -- 172.21.15.29:0/487607195 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9dc40a6710 con 0x7f9dd40b2b70 2024-04-25T00:32:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.272+0000 7f9dd17fa640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9db40678c0 0x7f9db4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.272+0000 7f9dd17fa640 1 -- 172.21.15.29:0/487607195 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9dc41206a0 con 0x7f9dd40b2b70 2024-04-25T00:32:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.272+0000 7f9dcbfff640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9db40678c0 0x7f9db4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.272+0000 7f9dcbfff640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9db40678c0 0x7f9db4069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40cbf30 tx=0x7f9dbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.280+0000 7f9dd17fa640 1 -- 172.21.15.29:0/487607195 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9dc4017440 con 0x7f9dd40b2b70 2024-04-25T00:32:04.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.444+0000 7f9dda54a640 1 -- 172.21.15.29:0/487607195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9dd40cf910 con 0x7f9dd40b2b70 2024-04-25T00:32:04.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.444+0000 7f9dd17fa640 1 -- 172.21.15.29:0/487607195 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f9dc40f11e0 con 0x7f9dd40b2b70 2024-04-25T00:32:04.553 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:04.553 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.448+0000 7f9dda54a640 1 -- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9db40678c0 msgr2=0x7f9db4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.448+0000 7f9dda54a640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9db40678c0 0x7f9db4069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40cbf30 tx=0x7f9dbc008040 comp rx=0 tx=0).stop 2024-04-25T00:32:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.448+0000 7f9dda54a640 1 -- 172.21.15.29:0/487607195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd40b2b70 msgr2=0x7f9dd41467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.448+0000 7f9dda54a640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd40b2b70 0x7f9dd41467b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40b1040 tx=0x7f9dc401a5c0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.448+0000 7f9dda54a640 1 -- 172.21.15.29:0/487607195 shutdown_connections 2024-04-25T00:32:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.448+0000 7f9dda54a640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9db40678c0 0x7f9db4069d80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.448+0000 7f9dda54a640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd415d980 0x7f9dd415c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.448+0000 7f9dda54a640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd4150df0 0x7f9dd415bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.448+0000 7f9dda54a640 1 --2- 172.21.15.29:0/487607195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd40b2b70 0x7f9dd41467b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.448+0000 7f9dda54a640 1 -- 172.21.15.29:0/487607195 >> 172.21.15.29:0/487607195 conn(0x7f9dd40b1870 msgr2=0x7f9dd414c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.448+0000 7f9dda54a640 1 -- 172.21.15.29:0/487607195 shutdown_connections 2024-04-25T00:32:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.448+0000 7f9dda54a640 1 -- 172.21.15.29:0/487607195 wait complete. 2024-04-25T00:32:04.556 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:04.557 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:04.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.688+0000 7f078d3c4640 1 Processor -- start 2024-04-25T00:32:04.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.688+0000 7f078d3c4640 1 -- start start 2024-04-25T00:32:04.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.688+0000 7f078d3c4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f078815f5a0 0x7f07880d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.688+0000 7f078d3c4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07880cf9f0 0x7f07880cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f078d3c4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f07880d0300 0x7f07880d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f078d3c4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f07880cc8a0 con 0x7f07880d0300 2024-04-25T00:32:04.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f078d3c4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f078815f980 con 0x7f078815f5a0 2024-04-25T00:32:04.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f078d3c4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f07880cf3e0 con 0x7f07880cf9f0 2024-04-25T00:32:04.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f07877fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f07880d0300 0x7f07880d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f07877fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f07880d0300 0x7f07880d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41318/0 (socket says 172.21.15.29:41318) 2024-04-25T00:32:04.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f07877fe640 1 -- 172.21.15.29:0/77796019 learned_addr learned my addr 172.21.15.29:0/77796019 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:04.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f07867fc640 1 --2- 172.21.15.29:0/77796019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07880cf9f0 0x7f07880cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f0786ffd640 1 --2- 172.21.15.29:0/77796019 >> v2:172.21.15.157:3300/0 conn(0x7f078815f5a0 0x7f07880d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f07877fe640 1 -- 172.21.15.29:0/77796019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07880cf9f0 msgr2=0x7f07880cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:04.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f07877fe640 1 --2- 172.21.15.29:0/77796019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07880cf9f0 0x7f07880cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f07877fe640 1 -- 172.21.15.29:0/77796019 >> v2:172.21.15.157:3300/0 conn(0x7f078815f5a0 msgr2=0x7f07880d2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:04.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f07877fe640 1 --2- 172.21.15.29:0/77796019 >> v2:172.21.15.157:3300/0 conn(0x7f078815f5a0 0x7f07880d2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f07877fe640 1 -- 172.21.15.29:0/77796019 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07880ccad0 con 0x7f07880d0300 2024-04-25T00:32:04.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f0786ffd640 1 --2- 172.21.15.29:0/77796019 >> v2:172.21.15.157:3300/0 conn(0x7f078815f5a0 0x7f07880d2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:04.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.692+0000 7f07877fe640 1 --2- 172.21.15.29:0/77796019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07880d0300 0x7f07880d07b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f07880720a0 tx=0x7f078001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcdd929a7a4ca9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f0785ffb640 1 -- 172.21.15.29:0/77796019 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07800b5070 con 0x7f07880d0300 2024-04-25T00:32:04.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f0785ffb640 1 -- 172.21.15.29:0/77796019 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07800a89d0 con 0x7f07880d0300 2024-04-25T00:32:04.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f0785ffb640 1 -- 172.21.15.29:0/77796019 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07800a8da0 con 0x7f07880d0300 2024-04-25T00:32:04.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 -- 172.21.15.29:0/77796019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07880d0300 msgr2=0x7f07880d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 --2- 172.21.15.29:0/77796019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07880d0300 0x7f07880d07b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f07880720a0 tx=0x7f078001ed10 comp rx=0 tx=0).stop 2024-04-25T00:32:04.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 -- 172.21.15.29:0/77796019 shutdown_connections 2024-04-25T00:32:04.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 --2- 172.21.15.29:0/77796019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07880d0300 0x7f07880d07b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 --2- 172.21.15.29:0/77796019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07880cf9f0 0x7f07880cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 --2- 172.21.15.29:0/77796019 >> v2:172.21.15.157:3300/0 conn(0x7f078815f5a0 0x7f07880d2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 -- 172.21.15.29:0/77796019 >> 172.21.15.29:0/77796019 conn(0x7f07880d1d20 msgr2=0x7f07880d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:04.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 -- 172.21.15.29:0/77796019 shutdown_connections 2024-04-25T00:32:04.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 -- 172.21.15.29:0/77796019 wait complete. 2024-04-25T00:32:04.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 Processor -- start 2024-04-25T00:32:04.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 -- start start 2024-04-25T00:32:04.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07880cf9f0 0x7f07880cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07880d0300 0x7f07881f1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f078815f5a0 0x7f0788153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f07880cefb0 con 0x7f078815f5a0 2024-04-25T00:32:04.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07880cd090 con 0x7f07880d0300 2024-04-25T00:32:04.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f07880cf130 con 0x7f07880cf9f0 2024-04-25T00:32:04.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f07877fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f078815f5a0 0x7f0788153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f07877fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f078815f5a0 0x7f0788153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41334/0 (socket says 172.21.15.29:41334) 2024-04-25T00:32:04.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f07877fe640 1 -- 172.21.15.29:0/1005782544 learned_addr learned my addr 172.21.15.29:0/1005782544 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:04.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f0786ffd640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07880cf9f0 0x7f07880cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f07867fc640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07880d0300 0x7f07881f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f07877fe640 1 -- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07880cf9f0 msgr2=0x7f07880cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f07877fe640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07880cf9f0 0x7f07880cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f07877fe640 1 -- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07880d0300 msgr2=0x7f07881f1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f07877fe640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07880d0300 0x7f07881f1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f07877fe640 1 -- 172.21.15.29:0/1005782544 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0788152670 con 0x7f078815f5a0 2024-04-25T00:32:04.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f0786ffd640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07880cf9f0 0x7f07880cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:04.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f07877fe640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f078815f5a0 0x7f0788153710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f07800b3040 tx=0x7f078001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f077f7fe640 1 -- 172.21.15.29:0/1005782544 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07800c0070 con 0x7f078815f5a0 2024-04-25T00:32:04.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f077f7fe640 1 -- 172.21.15.29:0/1005782544 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07800b06c0 con 0x7f078815f5a0 2024-04-25T00:32:04.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.696+0000 7f078d3c4640 1 -- 172.21.15.29:0/1005782544 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0788152950 con 0x7f078815f5a0 2024-04-25T00:32:04.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.700+0000 7f077f7fe640 1 -- 172.21.15.29:0/1005782544 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07800b5040 con 0x7f078815f5a0 2024-04-25T00:32:04.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.700+0000 7f078d3c4640 1 -- 172.21.15.29:0/1005782544 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0788152f10 con 0x7f078815f5a0 2024-04-25T00:32:04.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.700+0000 7f078d3c4640 1 -- 172.21.15.29:0/1005782544 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0788069270 con 0x7f078815f5a0 2024-04-25T00:32:04.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.700+0000 7f077f7fe640 1 -- 172.21.15.29:0/1005782544 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f07800c3070 con 0x7f078815f5a0 2024-04-25T00:32:04.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.700+0000 7f077f7fe640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f07680678c0 0x7f0768069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.700+0000 7f077f7fe640 1 -- 172.21.15.29:0/1005782544 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0780122c50 con 0x7f078815f5a0 2024-04-25T00:32:04.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.700+0000 7f0786ffd640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f07680678c0 0x7f0768069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.700+0000 7f0786ffd640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f07680678c0 0x7f0768069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0778009b40 tx=0x7f0778003a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.704+0000 7f077f7fe640 1 -- 172.21.15.29:0/1005782544 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0780128020 con 0x7f078815f5a0 2024-04-25T00:32:04.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.864+0000 7f078d3c4640 1 -- 172.21.15.29:0/1005782544 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07880cbc00 con 0x7f078815f5a0 2024-04-25T00:32:04.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.868+0000 7f077f7fe640 1 -- 172.21.15.29:0/1005782544 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f0780018870 con 0x7f078815f5a0 2024-04-25T00:32:04.869 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:04.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.868+0000 7f078d3c4640 1 -- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f07680678c0 msgr2=0x7f0768069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.868+0000 7f078d3c4640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f07680678c0 0x7f0768069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0778009b40 tx=0x7f0778003a40 comp rx=0 tx=0).stop 2024-04-25T00:32:04.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.868+0000 7f078d3c4640 1 -- 172.21.15.29:0/1005782544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f078815f5a0 msgr2=0x7f0788153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.868+0000 7f078d3c4640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f078815f5a0 0x7f0788153710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f07800b3040 tx=0x7f078001bab0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.868+0000 7f078d3c4640 1 -- 172.21.15.29:0/1005782544 shutdown_connections 2024-04-25T00:32:04.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.868+0000 7f078d3c4640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f078815f5a0 0x7f0788153710 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.868+0000 7f078d3c4640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f07680678c0 0x7f0768069d80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.868+0000 7f078d3c4640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07880d0300 0x7f07881f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.868+0000 7f078d3c4640 1 --2- 172.21.15.29:0/1005782544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07880cf9f0 0x7f07880cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.868+0000 7f078d3c4640 1 -- 172.21.15.29:0/1005782544 >> 172.21.15.29:0/1005782544 conn(0x7f07880d1d20 msgr2=0x7f078806feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:04.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.868+0000 7f078d3c4640 1 -- 172.21.15.29:0/1005782544 shutdown_connections 2024-04-25T00:32:04.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:04.868+0000 7f078d3c4640 1 -- 172.21.15.29:0/1005782544 wait complete. 2024-04-25T00:32:04.889 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:05.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.008+0000 7f2415615640 1 Processor -- start 2024-04-25T00:32:05.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.008+0000 7f2415615640 1 -- start start 2024-04-25T00:32:05.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.008+0000 7f2415615640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f24100d2c60 0x7f2410168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.008+0000 7f2415615640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24100cfc10 con 0x7f241015e200 2024-04-25T00:32:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.008+0000 7f2415615640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24100cfef0 con 0x7f24100cf690 2024-04-25T00:32:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.008+0000 7f2415615640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f24100b38d0 con 0x7f24100d2c60 2024-04-25T00:32:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.008+0000 7f240f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f24100d2c60 0x7f2410168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.008+0000 7f240effd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f241015e200 0x7f24100d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39668/0 (socket says 172.21.15.29:39668) 2024-04-25T00:32:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.008+0000 7f240effd640 1 -- 172.21.15.29:0/1471249469 learned_addr learned my addr 172.21.15.29:0/1471249469 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:05.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.008+0000 7f240f7fe640 1 -- 172.21.15.29:0/1471249469 >> v1:172.21.15.157:6789/0 conn(0x7f24100cf690 legacy=0x7f24100d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:05.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.008+0000 7f240f7fe640 1 -- 172.21.15.29:0/1471249469 >> v1:172.21.15.29:6789/0 conn(0x7f241015e200 legacy=0x7f24100d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.008+0000 7f240f7fe640 1 -- 172.21.15.29:0/1471249469 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24100b3b20 con 0x7f24100d2c60 2024-04-25T00:32:05.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.012+0000 7f240f7fe640 1 --2- 172.21.15.29:0/1471249469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24100d2c60 0x7f2410168e20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2408009d90 tx=0x7f240801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4815bc29f5c7830 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:05.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.012+0000 7f240dffb640 1 -- 172.21.15.29:0/1471249469 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24080b2070 con 0x7f24100d2c60 2024-04-25T00:32:05.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.012+0000 7f240dffb640 1 -- 172.21.15.29:0/1471249469 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24080a5810 con 0x7f24100d2c60 2024-04-25T00:32:05.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.012+0000 7f240dffb640 1 -- 172.21.15.29:0/1471249469 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24080a5be0 con 0x7f24100d2c60 2024-04-25T00:32:05.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.012+0000 7f2415615640 1 -- 172.21.15.29:0/1471249469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24100d2c60 msgr2=0x7f2410168e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.012+0000 7f2415615640 1 --2- 172.21.15.29:0/1471249469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24100d2c60 0x7f2410168e20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2408009d90 tx=0x7f240801c930 comp rx=0 tx=0).stop 2024-04-25T00:32:05.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.012+0000 7f2415615640 1 -- 172.21.15.29:0/1471249469 shutdown_connections 2024-04-25T00:32:05.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.012+0000 7f2415615640 1 --2- 172.21.15.29:0/1471249469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24100d2c60 0x7f2410168e20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.012+0000 7f2415615640 1 -- 172.21.15.29:0/1471249469 >> 172.21.15.29:0/1471249469 conn(0x7f241009d280 msgr2=0x7f24100d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:05.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.012+0000 7f2415615640 1 -- 172.21.15.29:0/1471249469 shutdown_connections 2024-04-25T00:32:05.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.012+0000 7f2415615640 1 -- 172.21.15.29:0/1471249469 wait complete. 2024-04-25T00:32:05.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.012+0000 7f2415615640 1 Processor -- start 2024-04-25T00:32:05.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.012+0000 7f2415615640 1 -- start start 2024-04-25T00:32:05.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f2415615640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24100cf690 0x7f24100cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f2415615640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24100d2c60 0x7f2410152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f2415615640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f241015e200 0x7f24100ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f2415615640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f241016bf10 con 0x7f241015e200 2024-04-25T00:32:05.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f2415615640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f241016bd90 con 0x7f24100d2c60 2024-04-25T00:32:05.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f2415615640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f241016c090 con 0x7f24100cf690 2024-04-25T00:32:05.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f240f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f241015e200 0x7f24100ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f240f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f241015e200 0x7f24100ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41364/0 (socket says 172.21.15.29:41364) 2024-04-25T00:32:05.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f240f7fe640 1 -- 172.21.15.29:0/1688352911 learned_addr learned my addr 172.21.15.29:0/1688352911 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:05.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f240effd640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24100cf690 0x7f24100cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f240e7fc640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24100d2c60 0x7f2410152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f240f7fe640 1 -- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24100cf690 msgr2=0x7f24100cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f240f7fe640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24100cf690 0x7f24100cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f240f7fe640 1 -- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24100d2c60 msgr2=0x7f2410152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f240f7fe640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24100d2c60 0x7f2410152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f240f7fe640 1 -- 172.21.15.29:0/1688352911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24100cb690 con 0x7f241015e200 2024-04-25T00:32:05.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f240f7fe640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f241015e200 0x7f24100ca430 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f240801a560 tx=0x7f24080196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:05.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f23f77fe640 1 -- 172.21.15.29:0/1688352911 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24080b2050 con 0x7f241015e200 2024-04-25T00:32:05.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f23f77fe640 1 -- 172.21.15.29:0/1688352911 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24080be070 con 0x7f241015e200 2024-04-25T00:32:05.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f23f77fe640 1 -- 172.21.15.29:0/1688352911 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24080ada50 con 0x7f241015e200 2024-04-25T00:32:05.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f2415615640 1 -- 172.21.15.29:0/1688352911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24100cb970 con 0x7f241015e200 2024-04-25T00:32:05.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f2415615640 1 -- 172.21.15.29:0/1688352911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24100cbf30 con 0x7f241015e200 2024-04-25T00:32:05.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f2415615640 1 -- 172.21.15.29:0/1688352911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2410069270 con 0x7f241015e200 2024-04-25T00:32:05.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f23f77fe640 1 -- 172.21.15.29:0/1688352911 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f24080adbf0 con 0x7f241015e200 2024-04-25T00:32:05.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f23f77fe640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f23fc0678c0 0x7f23fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f23f77fe640 1 -- 172.21.15.29:0/1688352911 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2408120fb0 con 0x7f241015e200 2024-04-25T00:32:05.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.015+0000 7f240effd640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f23fc0678c0 0x7f23fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.019+0000 7f240effd640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f23fc0678c0 0x7f23fc069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2404000f70 tx=0x7f2404009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:05.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.019+0000 7f23f77fe640 1 -- 172.21.15.29:0/1688352911 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2408016520 con 0x7f241015e200 2024-04-25T00:32:05.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.183+0000 7f2415615640 1 -- 172.21.15.29:0/1688352911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f24101567f0 con 0x7f241015e200 2024-04-25T00:32:05.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.183+0000 7f23f77fe640 1 -- 172.21.15.29:0/1688352911 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f24080f1a60 con 0x7f241015e200 2024-04-25T00:32:05.187 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:05.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.187+0000 7f2415615640 1 -- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f23fc0678c0 msgr2=0x7f23fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.187+0000 7f2415615640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f23fc0678c0 0x7f23fc069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2404000f70 tx=0x7f2404009040 comp rx=0 tx=0).stop 2024-04-25T00:32:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.187+0000 7f2415615640 1 -- 172.21.15.29:0/1688352911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f241015e200 msgr2=0x7f24100ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.187+0000 7f2415615640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f241015e200 0x7f24100ca430 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f240801a560 tx=0x7f24080196d0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.187+0000 7f2415615640 1 -- 172.21.15.29:0/1688352911 shutdown_connections 2024-04-25T00:32:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.187+0000 7f2415615640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f241015e200 0x7f24100ca430 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.187+0000 7f2415615640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f23fc0678c0 0x7f23fc069d80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.187+0000 7f2415615640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24100d2c60 0x7f2410152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.187+0000 7f2415615640 1 --2- 172.21.15.29:0/1688352911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24100cf690 0x7f24100cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.187+0000 7f2415615640 1 -- 172.21.15.29:0/1688352911 >> 172.21.15.29:0/1688352911 conn(0x7f241009d280 msgr2=0x7f241006c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.187+0000 7f2415615640 1 -- 172.21.15.29:0/1688352911 shutdown_connections 2024-04-25T00:32:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.187+0000 7f2415615640 1 -- 172.21.15.29:0/1688352911 wait complete. 2024-04-25T00:32:05.205 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:05.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.331+0000 7fd2373e9640 1 Processor -- start 2024-04-25T00:32:05.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.331+0000 7fd2373e9640 1 -- start start 2024-04-25T00:32:05.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd2373e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2300b3140 0x7fd23006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd2373e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd2300b4b10 0x7fd2300b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd2373e9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2300b41e0 con 0x7fd2300b3a60 2024-04-25T00:32:05.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd2373e9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd23006d960 con 0x7fd2300b4b10 2024-04-25T00:32:05.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd2373e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd23006dc40 con 0x7fd2300b3140 2024-04-25T00:32:05.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd235986640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd2300b3a60 0x7fd23015c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39676/0 (socket says 172.21.15.29:39676) 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd235986640 1 -- 172.21.15.29:0/3363054929 learned_addr learned my addr 172.21.15.29:0/3363054929 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd235185640 1 --2- 172.21.15.29:0/3363054929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2300b3140 0x7fd23006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3484178308 0 0) 0x7fd2300b41e0 con 0x7fd2300b3a60 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd234984640 1 --2- 172.21.15.29:0/3363054929 >> v2:172.21.15.157:3300/0 conn(0x7fd2300b4b10 0x7fd2300b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd21c003530 con 0x7fd2300b3a60 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 491669566 0 0) 0x7fd21c003530 con 0x7fd2300b3a60 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2300b3140 msgr2=0x7fd23006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 --2- 172.21.15.29:0/3363054929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2300b3140 0x7fd23006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 >> v2:172.21.15.157:3300/0 conn(0x7fd2300b4b10 msgr2=0x7fd2300b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 --2- 172.21.15.29:0/3363054929 >> v2:172.21.15.157:3300/0 conn(0x7fd2300b4b10 0x7fd2300b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23006df20 con 0x7fd2300b3a60 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd228001680 con 0x7fd2300b3a60 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 >> v1:172.21.15.29:6789/0 conn(0x7fd2300b3a60 legacy=0x7fd23015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 --2- 172.21.15.29:0/3363054929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21c0039c0 0x7fd21c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 --2- 172.21.15.29:0/3363054929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd21c004810 0x7fd21c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 --2- 172.21.15.29:0/3363054929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2300b3140 0x7fd21c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd23006d960 con 0x7fd21c004810 2024-04-25T00:32:05.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd23006dc40 con 0x7fd21c0039c0 2024-04-25T00:32:05.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c00b9d0 con 0x7fd2300b3140 2024-04-25T00:32:05.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd234984640 1 --2- 172.21.15.29:0/3363054929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd21c004810 0x7fd21c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd235185640 1 --2- 172.21.15.29:0/3363054929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21c0039c0 0x7fd21c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd235986640 1 --2- 172.21.15.29:0/3363054929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2300b3140 0x7fd21c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd234984640 1 -- 172.21.15.29:0/3363054929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2300b3140 msgr2=0x7fd21c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:05.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd234984640 1 --2- 172.21.15.29:0/3363054929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2300b3140 0x7fd21c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd234984640 1 -- 172.21.15.29:0/3363054929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21c0039c0 msgr2=0x7fd21c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:05.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd234984640 1 --2- 172.21.15.29:0/3363054929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21c0039c0 0x7fd21c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd234984640 1 -- 172.21.15.29:0/3363054929 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c003530 con 0x7fd21c004810 2024-04-25T00:32:05.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd234984640 1 --2- 172.21.15.29:0/3363054929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd21c004810 0x7fd21c004c90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009fc0 tx=0x7fd22c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51be928da6161fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:05.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c0b2070 con 0x7fd21c004810 2024-04-25T00:32:05.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd22c0a5bf0 con 0x7fd21c004810 2024-04-25T00:32:05.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.335+0000 7fd227fff640 1 -- 172.21.15.29:0/3363054929 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c009c10 con 0x7fd21c004810 2024-04-25T00:32:05.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 -- 172.21.15.29:0/3363054929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd21c004810 msgr2=0x7fd21c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 --2- 172.21.15.29:0/3363054929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd21c004810 0x7fd21c004c90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009fc0 tx=0x7fd22c01ccc0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd235986640 1 -- 172.21.15.29:0/3363054929 reap_dead start 2024-04-25T00:32:05.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 -- 172.21.15.29:0/3363054929 shutdown_connections 2024-04-25T00:32:05.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 -- 172.21.15.29:0/3363054929 >> 172.21.15.29:0/3363054929 conn(0x7fd2300b1870 msgr2=0x7fd2300b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:05.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 -- 172.21.15.29:0/3363054929 shutdown_connections 2024-04-25T00:32:05.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 -- 172.21.15.29:0/3363054929 wait complete. 2024-04-25T00:32:05.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 Processor -- start 2024-04-25T00:32:05.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 -- start start 2024-04-25T00:32:05.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd21c004810 0x7fd2301f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd23013a4e0 0x7fd2301f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd23013eea0 0x7fd23013f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd230145910 con 0x7fd21c004810 2024-04-25T00:32:05.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd2300b4f70 con 0x7fd23013a4e0 2024-04-25T00:32:05.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd230145a90 con 0x7fd23013eea0 2024-04-25T00:32:05.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd235185640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd21c004810 0x7fd2301f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd235185640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd21c004810 0x7fd2301f3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41390/0 (socket says 172.21.15.29:41390) 2024-04-25T00:32:05.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd235185640 1 -- 172.21.15.29:0/1975306213 learned_addr learned my addr 172.21.15.29:0/1975306213 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:05.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd234984640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd23013a4e0 0x7fd2301f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd235986640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd23013eea0 0x7fd23013f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd235185640 1 -- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd23013eea0 msgr2=0x7fd23013f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd235185640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd23013eea0 0x7fd23013f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd235185640 1 -- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd23013a4e0 msgr2=0x7fd2301f6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd235185640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd23013a4e0 0x7fd2301f6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd235185640 1 -- 172.21.15.29:0/1975306213 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23013c4b0 con 0x7fd21c004810 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd234984640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd23013a4e0 0x7fd2301f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd235185640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd21c004810 0x7fd2301f3ea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd220011610 tx=0x7fd220011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd225ffb640 1 -- 172.21.15.29:0/1975306213 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220015870 con 0x7fd21c004810 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd225ffb640 1 -- 172.21.15.29:0/1975306213 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd220015a10 con 0x7fd21c004810 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 -- 172.21.15.29:0/1975306213 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23013c1d0 con 0x7fd21c004810 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd225ffb640 1 -- 172.21.15.29:0/1975306213 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220015dc0 con 0x7fd21c004810 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.339+0000 7fd2373e9640 1 -- 172.21.15.29:0/1975306213 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd230071060 con 0x7fd21c004810 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.343+0000 7fd2373e9640 1 -- 172.21.15.29:0/1975306213 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd230069270 con 0x7fd21c004810 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.343+0000 7fd225ffb640 1 -- 172.21.15.29:0/1975306213 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd22000f480 con 0x7fd21c004810 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.343+0000 7fd225ffb640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2140678f0 0x7fd214069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.343+0000 7fd225ffb640 1 -- 172.21.15.29:0/1975306213 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd220086440 con 0x7fd21c004810 2024-04-25T00:32:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.343+0000 7fd234984640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2140678f0 0x7fd214069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.343+0000 7fd234984640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2140678f0 0x7fd214069db0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0af000 tx=0x7fd22c0a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:05.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.347+0000 7fd225ffb640 1 -- 172.21.15.29:0/1975306213 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd220019030 con 0x7fd21c004810 2024-04-25T00:32:05.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.499+0000 7fd2373e9640 1 -- 172.21.15.29:0/1975306213 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd230143d30 con 0x7fd21c004810 2024-04-25T00:32:05.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.499+0000 7fd225ffb640 1 -- 172.21.15.29:0/1975306213 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fd220057010 con 0x7fd21c004810 2024-04-25T00:32:05.504 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:05.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.503+0000 7fd2373e9640 1 -- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2140678f0 msgr2=0x7fd214069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.503+0000 7fd2373e9640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2140678f0 0x7fd214069db0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0af000 tx=0x7fd22c0a7040 comp rx=0 tx=0).stop 2024-04-25T00:32:05.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.503+0000 7fd2373e9640 1 -- 172.21.15.29:0/1975306213 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd21c004810 msgr2=0x7fd2301f3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.503+0000 7fd2373e9640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd21c004810 0x7fd2301f3ea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd220011610 tx=0x7fd220011ad0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.503+0000 7fd2373e9640 1 -- 172.21.15.29:0/1975306213 shutdown_connections 2024-04-25T00:32:05.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.503+0000 7fd2373e9640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd23013eea0 0x7fd23013f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.503+0000 7fd2373e9640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2140678f0 0x7fd214069db0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.503+0000 7fd2373e9640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd23013a4e0 0x7fd2301f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.503+0000 7fd2373e9640 1 --2- 172.21.15.29:0/1975306213 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd21c004810 0x7fd2301f3ea0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.503+0000 7fd2373e9640 1 -- 172.21.15.29:0/1975306213 >> 172.21.15.29:0/1975306213 conn(0x7fd2300b1870 msgr2=0x7fd23015f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:05.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.503+0000 7fd2373e9640 1 -- 172.21.15.29:0/1975306213 shutdown_connections 2024-04-25T00:32:05.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.503+0000 7fd2373e9640 1 -- 172.21.15.29:0/1975306213 wait complete. 2024-04-25T00:32:05.522 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-04-25T00:32:05.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.631+0000 7ff752599640 1 Processor -- start 2024-04-25T00:32:05.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.631+0000 7ff752599640 1 -- start start 2024-04-25T00:32:05.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff752599640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff74c15e060 0x7ff74c15c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff752599640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15bce0 0x7ff74c071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff752599640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff74c15e850 con 0x7ff74c0722c0 2024-04-25T00:32:05.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff752599640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff74c15f700 con 0x7ff74c15e060 2024-04-25T00:32:05.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff752599640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff74c15eaf0 con 0x7ff74c15bce0 2024-04-25T00:32:05.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff74bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff74c15e060 0x7ff74c15c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff750b36640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff74c0722c0 0x7ff74c0cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39688/0 (socket says 172.21.15.29:39688) 2024-04-25T00:32:05.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff750b36640 1 -- 172.21.15.29:0/1892335522 learned_addr learned my addr 172.21.15.29:0/1892335522 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:05.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff74bfff640 1 -- 172.21.15.29:0/1892335522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15bce0 msgr2=0x7ff74c071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:05.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff74bfff640 1 --2- 172.21.15.29:0/1892335522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15bce0 0x7ff74c071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff74bfff640 1 -- 172.21.15.29:0/1892335522 >> v1:172.21.15.29:6789/0 conn(0x7ff74c0722c0 legacy=0x7ff74c0cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff74bfff640 1 -- 172.21.15.29:0/1892335522 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c15edd0 con 0x7ff74c15e060 2024-04-25T00:32:05.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff74affd640 1 -- 172.21.15.29:0/1892335522 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1895156327 0 0) 0x7ff74c15e850 con 0x7ff74c0722c0 2024-04-25T00:32:05.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff74bfff640 1 --2- 172.21.15.29:0/1892335522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74c15e060 0x7ff74c15c420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0097c0 tx=0x7ff73c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9613e6a890cb4bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:05.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff74affd640 1 -- 172.21.15.29:0/1892335522 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0b2070 con 0x7ff74c15e060 2024-04-25T00:32:05.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff74affd640 1 -- 172.21.15.29:0/1892335522 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73c0a9070 con 0x7ff74c15e060 2024-04-25T00:32:05.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff74affd640 1 -- 172.21.15.29:0/1892335522 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0ad880 con 0x7ff74c15e060 2024-04-25T00:32:05.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff752599640 1 -- 172.21.15.29:0/1892335522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74c15e060 msgr2=0x7ff74c15c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff752599640 1 --2- 172.21.15.29:0/1892335522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74c15e060 0x7ff74c15c420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0097c0 tx=0x7ff73c01c470 comp rx=0 tx=0).stop 2024-04-25T00:32:05.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff752599640 1 -- 172.21.15.29:0/1892335522 shutdown_connections 2024-04-25T00:32:05.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff752599640 1 --2- 172.21.15.29:0/1892335522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15bce0 0x7ff74c071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff752599640 1 --2- 172.21.15.29:0/1892335522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74c15e060 0x7ff74c15c420 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff752599640 1 -- 172.21.15.29:0/1892335522 >> 172.21.15.29:0/1892335522 conn(0x7ff74c06c320 msgr2=0x7ff74c06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:05.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff752599640 1 -- 172.21.15.29:0/1892335522 shutdown_connections 2024-04-25T00:32:05.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.635+0000 7ff752599640 1 -- 172.21.15.29:0/1892335522 wait complete. 2024-04-25T00:32:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff752599640 1 Processor -- start 2024-04-25T00:32:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff752599640 1 -- start start 2024-04-25T00:32:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff752599640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74c0722c0 0x7ff74c151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff752599640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15bce0 0x7ff74c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff752599640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c15e060 0x7ff74c14fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff752599640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c15e5c0 con 0x7ff74c0722c0 2024-04-25T00:32:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff752599640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c15e440 con 0x7ff74c15e060 2024-04-25T00:32:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff752599640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff74c06f610 con 0x7ff74c15bce0 2024-04-25T00:32:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff74bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74c0722c0 0x7ff74c151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff74bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74c0722c0 0x7ff74c151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41418/0 (socket says 172.21.15.29:41418) 2024-04-25T00:32:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff74bfff640 1 -- 172.21.15.29:0/1149119407 learned_addr learned my addr 172.21.15.29:0/1149119407 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff74bfff640 1 -- 172.21.15.29:0/1149119407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15bce0 msgr2=0x7ff74c0b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff74bfff640 1 --2- 172.21.15.29:0/1149119407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15bce0 0x7ff74c0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff74bfff640 1 -- 172.21.15.29:0/1149119407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c15e060 msgr2=0x7ff74c14fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff74bfff640 1 --2- 172.21.15.29:0/1149119407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c15e060 0x7ff74c14fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff74bfff640 1 -- 172.21.15.29:0/1149119407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c0ca330 con 0x7ff74c0722c0 2024-04-25T00:32:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff74bfff640 1 --2- 172.21.15.29:0/1149119407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74c0722c0 0x7ff74c151cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0097c0 tx=0x7ff73c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff748ff9640 1 -- 172.21.15.29:0/1149119407 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0bf070 con 0x7ff74c0722c0 2024-04-25T00:32:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff748ff9640 1 -- 172.21.15.29:0/1149119407 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73c0ade30 con 0x7ff74c0722c0 2024-04-25T00:32:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff752599640 1 -- 172.21.15.29:0/1149119407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74c0ca610 con 0x7ff74c0722c0 2024-04-25T00:32:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff748ff9640 1 -- 172.21.15.29:0/1149119407 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0b2040 con 0x7ff74c0722c0 2024-04-25T00:32:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff752599640 1 -- 172.21.15.29:0/1149119407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74c14f780 con 0x7ff74c0722c0 2024-04-25T00:32:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.639+0000 7ff752599640 1 -- 172.21.15.29:0/1149119407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff74c0caa00 con 0x7ff74c0722c0 2024-04-25T00:32:05.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.643+0000 7ff748ff9640 1 -- 172.21.15.29:0/1149119407 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff73c0c2070 con 0x7ff74c0722c0 2024-04-25T00:32:05.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.643+0000 7ff748ff9640 1 --2- 172.21.15.29:0/1149119407 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7380678c0 0x7ff738069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.643+0000 7ff748ff9640 1 -- 172.21.15.29:0/1149119407 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff73c11f860 con 0x7ff74c0722c0 2024-04-25T00:32:05.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.643+0000 7ff74b7fe640 1 --2- 172.21.15.29:0/1149119407 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7380678c0 0x7ff738069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.643+0000 7ff74b7fe640 1 --2- 172.21.15.29:0/1149119407 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7380678c0 0x7ff738069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0b12e0 tx=0x7ff740006ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.647+0000 7ff748ff9640 1 -- 172.21.15.29:0/1149119407 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff73c016090 con 0x7ff74c0722c0 2024-04-25T00:32:05.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:05.867+0000 7ff752599640 1 -- 172.21.15.29:0/1149119407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7ff74c069270 con 0x7ff74c0722c0 2024-04-25T00:32:06.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.615+0000 7ff748ff9640 1 -- 172.21.15.29:0/1149119407 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v50) ==== 161+0+0 (secure 0 0 0) 0x7ff73c0f0510 con 0x7ff74c0722c0 2024-04-25T00:32:06.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.619+0000 7ff752599640 1 -- 172.21.15.29:0/1149119407 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7380678c0 msgr2=0x7ff738069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:06.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.619+0000 7ff752599640 1 --2- 172.21.15.29:0/1149119407 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7380678c0 0x7ff738069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0b12e0 tx=0x7ff740006ca0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.619+0000 7ff752599640 1 -- 172.21.15.29:0/1149119407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74c0722c0 msgr2=0x7ff74c151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:06.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.619+0000 7ff752599640 1 --2- 172.21.15.29:0/1149119407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74c0722c0 0x7ff74c151cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0097c0 tx=0x7ff73c01acb0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.619+0000 7ff752599640 1 -- 172.21.15.29:0/1149119407 shutdown_connections 2024-04-25T00:32:06.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.619+0000 7ff752599640 1 --2- 172.21.15.29:0/1149119407 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7380678c0 0x7ff738069d80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.619+0000 7ff752599640 1 --2- 172.21.15.29:0/1149119407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff74c15e060 0x7ff74c14fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.619+0000 7ff752599640 1 --2- 172.21.15.29:0/1149119407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff74c15bce0 0x7ff74c0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.619+0000 7ff752599640 1 --2- 172.21.15.29:0/1149119407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74c0722c0 0x7ff74c151cf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.619+0000 7ff752599640 1 -- 172.21.15.29:0/1149119407 >> 172.21.15.29:0/1149119407 conn(0x7ff74c06c320 msgr2=0x7ff74c15b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:06.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.619+0000 7ff752599640 1 -- 172.21.15.29:0/1149119407 shutdown_connections 2024-04-25T00:32:06.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.619+0000 7ff752599640 1 -- 172.21.15.29:0/1149119407 wait complete. 2024-04-25T00:32:06.646 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:32:06.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e6cc73640 1 Processor -- start 2024-04-25T00:32:06.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e6cc73640 1 -- start start 2024-04-25T00:32:06.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e6cc73640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8e6815df70 0x7f8e680b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:06.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e6cc73640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8e680b4d30 0x7f8e680ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:06.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e6cc73640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e6806d980 con 0x7f8e6806d210 2024-04-25T00:32:06.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e6cc73640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8e6806dc60 con 0x7f8e680b4d30 2024-04-25T00:32:06.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e6cc73640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8e6806df40 con 0x7f8e6815df70 2024-04-25T00:32:06.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e66d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8e680b4d30 0x7f8e680ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:06.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e66d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8e680b4d30 0x7f8e680ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41420/0 (socket says 172.21.15.29:41420) 2024-04-25T00:32:06.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e66d9d640 1 -- 172.21.15.29:0/3607173622 learned_addr learned my addr 172.21.15.29:0/3607173622 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:06.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e6659c640 1 --2- 172.21.15.29:0/3607173622 >> v2:172.21.15.157:3300/0 conn(0x7f8e6815df70 0x7f8e680b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:06.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e66d9d640 1 -- 172.21.15.29:0/3607173622 >> v2:172.21.15.157:3300/0 conn(0x7f8e6815df70 msgr2=0x7f8e680b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:06.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e66d9d640 1 --2- 172.21.15.29:0/3607173622 >> v2:172.21.15.157:3300/0 conn(0x7f8e6815df70 0x7f8e680b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e66d9d640 1 -- 172.21.15.29:0/3607173622 >> v1:172.21.15.157:6789/0 conn(0x7f8e6806d210 legacy=0x7f8e680b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:06.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.775+0000 7f8e66d9d640 1 -- 172.21.15.29:0/3607173622 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6806e220 con 0x7f8e680b4d30 2024-04-25T00:32:06.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e66d9d640 1 --2- 172.21.15.29:0/3607173622 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e680b4d30 0x7f8e680ad8a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8e680d0bf0 tx=0x7f8e6001c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bab058983455447 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:06.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6559a640 1 -- 172.21.15.29:0/3607173622 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e600a57b0 con 0x7f8e680b4d30 2024-04-25T00:32:06.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6559a640 1 -- 172.21.15.29:0/3607173622 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e600a5950 con 0x7f8e680b4d30 2024-04-25T00:32:06.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6559a640 1 -- 172.21.15.29:0/3607173622 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e600a5d20 con 0x7f8e680b4d30 2024-04-25T00:32:06.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 -- 172.21.15.29:0/3607173622 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e680b4d30 msgr2=0x7f8e680ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:06.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 --2- 172.21.15.29:0/3607173622 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e680b4d30 0x7f8e680ad8a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8e680d0bf0 tx=0x7f8e6001c8f0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 -- 172.21.15.29:0/3607173622 shutdown_connections 2024-04-25T00:32:06.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 --2- 172.21.15.29:0/3607173622 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e680b4d30 0x7f8e680ad8a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 --2- 172.21.15.29:0/3607173622 >> v2:172.21.15.157:3300/0 conn(0x7f8e6815df70 0x7f8e680b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 -- 172.21.15.29:0/3607173622 >> 172.21.15.29:0/3607173622 conn(0x7f8e680b1870 msgr2=0x7f8e680b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:06.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 -- 172.21.15.29:0/3607173622 shutdown_connections 2024-04-25T00:32:06.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 -- 172.21.15.29:0/3607173622 wait complete. 2024-04-25T00:32:06.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 Processor -- start 2024-04-25T00:32:06.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 -- start start 2024-04-25T00:32:06.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e6806d210 0x7f8e681577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:06.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e680b4d30 0x7f8e68142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:06.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6815df70 0x7f8e680cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:06.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6806eba0 con 0x7f8e6815df70 2024-04-25T00:32:06.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6806ea20 con 0x7f8e680b4d30 2024-04-25T00:32:06.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6cc73640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8e680ae5f0 con 0x7f8e6806d210 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e66d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6815df70 0x7f8e680cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e66d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6815df70 0x7f8e680cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41424/0 (socket says 172.21.15.29:41424) 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e66d9d640 1 -- 172.21.15.29:0/2858522071 learned_addr learned my addr 172.21.15.29:0/2858522071 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e65d9b640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e680b4d30 0x7f8e68142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e6659c640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e6806d210 0x7f8e681577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e66d9d640 1 -- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e6806d210 msgr2=0x7f8e681577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e66d9d640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e6806d210 0x7f8e681577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e66d9d640 1 -- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e680b4d30 msgr2=0x7f8e68142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e66d9d640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e680b4d30 0x7f8e68142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e66d9d640 1 -- 172.21.15.29:0/2858522071 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6815cf40 con 0x7f8e6815df70 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.779+0000 7f8e65d9b640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e680b4d30 0x7f8e68142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.783+0000 7f8e66d9d640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6815df70 0x7f8e680cc450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8e600af040 tx=0x7f8e60019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.783+0000 7f8e56ffd640 1 -- 172.21.15.29:0/2858522071 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e600bc020 con 0x7f8e6815df70 2024-04-25T00:32:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.783+0000 7f8e56ffd640 1 -- 172.21.15.29:0/2858522071 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e600b8880 con 0x7f8e6815df70 2024-04-25T00:32:06.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.783+0000 7f8e6cc73640 1 -- 172.21.15.29:0/2858522071 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e68157da0 con 0x7f8e6815df70 2024-04-25T00:32:06.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.783+0000 7f8e56ffd640 1 -- 172.21.15.29:0/2858522071 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60009270 con 0x7f8e6815df70 2024-04-25T00:32:06.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.783+0000 7f8e6cc73640 1 -- 172.21.15.29:0/2858522071 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e68153440 con 0x7f8e6815df70 2024-04-25T00:32:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.783+0000 7f8e6cc73640 1 -- 172.21.15.29:0/2858522071 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e68069270 con 0x7f8e6815df70 2024-04-25T00:32:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.783+0000 7f8e56ffd640 1 -- 172.21.15.29:0/2858522071 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8e600bf020 con 0x7f8e6815df70 2024-04-25T00:32:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.783+0000 7f8e56ffd640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8e480678f0 0x7f8e48069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.783+0000 7f8e56ffd640 1 -- 172.21.15.29:0/2858522071 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8e60120eb0 con 0x7f8e6815df70 2024-04-25T00:32:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.783+0000 7f8e6659c640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8e480678f0 0x7f8e48069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.787+0000 7f8e6659c640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8e480678f0 0x7f8e48069db0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800a8d0 tx=0x7f8e58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:06.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.787+0000 7f8e56ffd640 1 -- 172.21.15.29:0/2858522071 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8e600c6030 con 0x7f8e6815df70 2024-04-25T00:32:06.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.947+0000 7f8e6cc73640 1 -- 172.21.15.29:0/2858522071 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8e68159000 con 0x7f8e6815df70 2024-04-25T00:32:06.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.951+0000 7f8e56ffd640 1 -- 172.21.15.29:0/2858522071 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f8e600165a0 con 0x7f8e6815df70 2024-04-25T00:32:06.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.955+0000 7f8e6cc73640 1 -- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8e480678f0 msgr2=0x7f8e48069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:06.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.955+0000 7f8e6cc73640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8e480678f0 0x7f8e48069db0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800a8d0 tx=0x7f8e58008040 comp rx=0 tx=0).stop 2024-04-25T00:32:06.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.955+0000 7f8e6cc73640 1 -- 172.21.15.29:0/2858522071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6815df70 msgr2=0x7f8e680cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:06.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.955+0000 7f8e6cc73640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6815df70 0x7f8e680cc450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8e600af040 tx=0x7f8e60019690 comp rx=0 tx=0).stop 2024-04-25T00:32:06.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.955+0000 7f8e6cc73640 1 -- 172.21.15.29:0/2858522071 shutdown_connections 2024-04-25T00:32:06.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.955+0000 7f8e6cc73640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6815df70 0x7f8e680cc450 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.955+0000 7f8e6cc73640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8e480678f0 0x7f8e48069db0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.955+0000 7f8e6cc73640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e680b4d30 0x7f8e68142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.955+0000 7f8e6cc73640 1 --2- 172.21.15.29:0/2858522071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e6806d210 0x7f8e681577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:06.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.955+0000 7f8e6cc73640 1 -- 172.21.15.29:0/2858522071 >> 172.21.15.29:0/2858522071 conn(0x7f8e680b1870 msgr2=0x7f8e680b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:06.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.955+0000 7f8e6cc73640 1 -- 172.21.15.29:0/2858522071 shutdown_connections 2024-04-25T00:32:06.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:06.955+0000 7f8e6cc73640 1 -- 172.21.15.29:0/2858522071 wait complete. 2024-04-25T00:32:06.974 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:32:06.974 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:32:07.010 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:32:07.058 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:32:07.058 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:32:07.058 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:07.064 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:32:07.064 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:07.113 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:07.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.279+0000 7f1ae514c640 1 Processor -- start 2024-04-25T00:32:07.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.279+0000 7f1ae514c640 1 -- start start 2024-04-25T00:32:07.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.279+0000 7f1ae514c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1ae006f270 0x7f1ae006f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.279+0000 7f1ae514c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ae006fb90 con 0x7f1ae015e330 2024-04-25T00:32:07.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.279+0000 7f1ae514c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ae00cfe90 con 0x7f1ae006b210 2024-04-25T00:32:07.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.279+0000 7f1ae514c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae00d0170 con 0x7f1ae006f270 2024-04-25T00:32:07.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.279+0000 7f1ade59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1ae015e330 0x7f1ae006db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39702/0 (socket says 172.21.15.29:39702) 2024-04-25T00:32:07.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.279+0000 7f1aded9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1ae006f270 0x7f1ae006f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.279+0000 7f1ade59c640 1 -- 172.21.15.29:0/933331072 learned_addr learned my addr 172.21.15.29:0/933331072 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.279+0000 7f1aded9d640 1 -- 172.21.15.29:0/933331072 >> v1:172.21.15.157:6789/0 conn(0x7f1ae006b210 legacy=0x7f1ae006b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.279+0000 7f1aded9d640 1 -- 172.21.15.29:0/933331072 >> v1:172.21.15.29:6789/0 conn(0x7f1ae015e330 legacy=0x7f1ae006db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.279+0000 7f1aded9d640 1 -- 172.21.15.29:0/933331072 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae00d0450 con 0x7f1ae006f270 2024-04-25T00:32:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.279+0000 7f1addd9b640 1 -- 172.21.15.29:0/933331072 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 470469053 0 0) 0x7f1ae006fb90 con 0x7f1ae015e330 2024-04-25T00:32:07.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.283+0000 7f1aded9d640 1 --2- 172.21.15.29:0/933331072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae006f270 0x7f1ae006f650 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80097c0 tx=0x7f1ad801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dd1b9677085515 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:07.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.283+0000 7f1addd9b640 1 -- 172.21.15.29:0/933331072 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80b0070 con 0x7f1ae006f270 2024-04-25T00:32:07.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.283+0000 7f1addd9b640 1 -- 172.21.15.29:0/933331072 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad80ab600 con 0x7f1ae006f270 2024-04-25T00:32:07.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.283+0000 7f1addd9b640 1 -- 172.21.15.29:0/933331072 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80a4e60 con 0x7f1ae006f270 2024-04-25T00:32:07.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.283+0000 7f1ae514c640 1 -- 172.21.15.29:0/933331072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae006f270 msgr2=0x7f1ae006f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.283+0000 7f1ae514c640 1 --2- 172.21.15.29:0/933331072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae006f270 0x7f1ae006f650 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80097c0 tx=0x7f1ad801c3d0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.283+0000 7f1ae514c640 1 -- 172.21.15.29:0/933331072 shutdown_connections 2024-04-25T00:32:07.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.283+0000 7f1ae514c640 1 --2- 172.21.15.29:0/933331072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae006f270 0x7f1ae006f650 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.283+0000 7f1ae514c640 1 -- 172.21.15.29:0/933331072 >> 172.21.15.29:0/933331072 conn(0x7f1ae00b2920 msgr2=0x7f1ae00b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:07.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.283+0000 7f1ae514c640 1 -- 172.21.15.29:0/933331072 shutdown_connections 2024-04-25T00:32:07.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.283+0000 7f1ae514c640 1 -- 172.21.15.29:0/933331072 wait complete. 2024-04-25T00:32:07.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ae514c640 1 Processor -- start 2024-04-25T00:32:07.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ae514c640 1 -- start start 2024-04-25T00:32:07.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ae514c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ae006b210 0x7f1ae013ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ae514c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae006f270 0x7f1ae00b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ae514c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae014dd30 0x7f1ae014e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ae514c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae016bf50 con 0x7f1ae006f270 2024-04-25T00:32:07.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ae514c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae015f4a0 con 0x7f1ae006b210 2024-04-25T00:32:07.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ae514c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae016c0d0 con 0x7f1ae014dd30 2024-04-25T00:32:07.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ade59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae006f270 0x7f1ae00b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ade59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae006f270 0x7f1ae00b1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41440/0 (socket says 172.21.15.29:41440) 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ade59c640 1 -- 172.21.15.29:0/1706650887 learned_addr learned my addr 172.21.15.29:0/1706650887 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1aded9d640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ae006b210 0x7f1ae013ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ade59c640 1 -- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae014dd30 msgr2=0x7f1ae014e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1adf59e640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae014dd30 0x7f1ae014e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ade59c640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae014dd30 0x7f1ae014e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ade59c640 1 -- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ae006b210 msgr2=0x7f1ae013ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ade59c640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ae006b210 0x7f1ae013ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ade59c640 1 -- 172.21.15.29:0/1706650887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae013b2d0 con 0x7f1ae006f270 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1aded9d640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ae006b210 0x7f1ae013ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1adf59e640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae014dd30 0x7f1ae014e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ade59c640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae006f270 0x7f1ae00b1a90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400c890 tx=0x7f1ad400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ac77fe640 1 -- 172.21.15.29:0/1706650887 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4018070 con 0x7f1ae006f270 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ac77fe640 1 -- 172.21.15.29:0/1706650887 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad400a5e0 con 0x7f1ae006f270 2024-04-25T00:32:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ac77fe640 1 -- 172.21.15.29:0/1706650887 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400a9d0 con 0x7f1ae006f270 2024-04-25T00:32:07.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ae514c640 1 -- 172.21.15.29:0/1706650887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae00b5bb0 con 0x7f1ae006f270 2024-04-25T00:32:07.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ae514c640 1 -- 172.21.15.29:0/1706650887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae00b6170 con 0x7f1ae006f270 2024-04-25T00:32:07.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.287+0000 7f1ae514c640 1 -- 172.21.15.29:0/1706650887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae0139470 con 0x7f1ae006f270 2024-04-25T00:32:07.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.291+0000 7f1ac77fe640 1 -- 172.21.15.29:0/1706650887 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1ad401b070 con 0x7f1ae006f270 2024-04-25T00:32:07.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.291+0000 7f1ac77fe640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1acc0678c0 0x7f1acc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.291+0000 7f1ac77fe640 1 -- 172.21.15.29:0/1706650887 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1ad40856f0 con 0x7f1ae006f270 2024-04-25T00:32:07.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.291+0000 7f1aded9d640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1acc0678c0 0x7f1acc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.291+0000 7f1aded9d640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1acc0678c0 0x7f1acc069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80ad000 tx=0x7f1ad8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:07.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.295+0000 7f1ac77fe640 1 -- 172.21.15.29:0/1706650887 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1ae013b2d0 con 0x7f1ae006f270 2024-04-25T00:32:07.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.455+0000 7f1ae514c640 1 -- 172.21.15.29:0/1706650887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ae0072200 con 0x7f1ae006f270 2024-04-25T00:32:07.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.459+0000 7f1ac77fe640 1 -- 172.21.15.29:0/1706650887 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f1ad4056210 con 0x7f1ae006f270 2024-04-25T00:32:07.463 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:07.463 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:07.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.459+0000 7f1ae514c640 1 -- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1acc0678c0 msgr2=0x7f1acc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.459+0000 7f1ae514c640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1acc0678c0 0x7f1acc069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80ad000 tx=0x7f1ad8008040 comp rx=0 tx=0).stop 2024-04-25T00:32:07.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.459+0000 7f1ae514c640 1 -- 172.21.15.29:0/1706650887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae006f270 msgr2=0x7f1ae00b1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.459+0000 7f1ae514c640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae006f270 0x7f1ae00b1a90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400c890 tx=0x7f1ad400cd50 comp rx=0 tx=0).stop 2024-04-25T00:32:07.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.459+0000 7f1ae514c640 1 -- 172.21.15.29:0/1706650887 shutdown_connections 2024-04-25T00:32:07.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.459+0000 7f1ae514c640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae014dd30 0x7f1ae014e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.459+0000 7f1ae514c640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae006f270 0x7f1ae00b1a90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.459+0000 7f1ae514c640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1acc0678c0 0x7f1acc069d80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.459+0000 7f1ae514c640 1 --2- 172.21.15.29:0/1706650887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ae006b210 0x7f1ae013ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.459+0000 7f1ae514c640 1 -- 172.21.15.29:0/1706650887 >> 172.21.15.29:0/1706650887 conn(0x7f1ae00b2920 msgr2=0x7f1ae015f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:07.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.463+0000 7f1ae514c640 1 -- 172.21.15.29:0/1706650887 shutdown_connections 2024-04-25T00:32:07.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.463+0000 7f1ae514c640 1 -- 172.21.15.29:0/1706650887 wait complete. 2024-04-25T00:32:07.482 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:07.483 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664beac640 1 Processor -- start 2024-04-25T00:32:07.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664beac640 1 -- start start 2024-04-25T00:32:07.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664beac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664415e060 0x7f664415c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664beac640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6644072490 0x7f66440cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664beac640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f664415e850 con 0x7f664415bce0 2024-04-25T00:32:07.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664beac640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f664415f700 con 0x7f6644072490 2024-04-25T00:32:07.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664beac640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f664415eaf0 con 0x7f664415e060 2024-04-25T00:32:07.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664a449640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6644072490 0x7f66440cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664a449640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6644072490 0x7f66440cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41446/0 (socket says 172.21.15.29:41446) 2024-04-25T00:32:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664a449640 1 -- 172.21.15.29:0/954922606 learned_addr learned my addr 172.21.15.29:0/954922606 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f6649c48640 1 --2- 172.21.15.29:0/954922606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664415e060 0x7f664415c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664a449640 1 -- 172.21.15.29:0/954922606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664415e060 msgr2=0x7f664415c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664a449640 1 --2- 172.21.15.29:0/954922606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664415e060 0x7f664415c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664a449640 1 -- 172.21.15.29:0/954922606 >> v1:172.21.15.157:6789/0 conn(0x7f664415bce0 legacy=0x7f6644071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.603+0000 7f664a449640 1 -- 172.21.15.29:0/954922606 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664415edd0 con 0x7f6644072490 2024-04-25T00:32:07.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664a449640 1 --2- 172.21.15.29:0/954922606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6644072490 0x7f66440cf100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f663c009e50 tx=0x7f663c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d555540c54b57f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:07.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f6648c46640 1 -- 172.21.15.29:0/954922606 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c0090d0 con 0x7f6644072490 2024-04-25T00:32:07.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f6648c46640 1 -- 172.21.15.29:0/954922606 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f663c0a5840 con 0x7f6644072490 2024-04-25T00:32:07.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f6648c46640 1 -- 172.21.15.29:0/954922606 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c009270 con 0x7f6644072490 2024-04-25T00:32:07.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 -- 172.21.15.29:0/954922606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6644072490 msgr2=0x7f66440cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 --2- 172.21.15.29:0/954922606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6644072490 0x7f66440cf100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f663c009e50 tx=0x7f663c01c9c0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 -- 172.21.15.29:0/954922606 shutdown_connections 2024-04-25T00:32:07.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 --2- 172.21.15.29:0/954922606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6644072490 0x7f66440cf100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 --2- 172.21.15.29:0/954922606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664415e060 0x7f664415c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 -- 172.21.15.29:0/954922606 >> 172.21.15.29:0/954922606 conn(0x7f664406c320 msgr2=0x7f664406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:07.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 -- 172.21.15.29:0/954922606 shutdown_connections 2024-04-25T00:32:07.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 -- 172.21.15.29:0/954922606 wait complete. 2024-04-25T00:32:07.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 Processor -- start 2024-04-25T00:32:07.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 -- start start 2024-04-25T00:32:07.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6644072490 0x7f6644152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664415bce0 0x7f66440ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f664415e060 0x7f6644153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f664415e5c0 con 0x7f664415e060 2024-04-25T00:32:07.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f664415e440 con 0x7f6644072490 2024-04-25T00:32:07.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664beac640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f664406f610 con 0x7f664415bce0 2024-04-25T00:32:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664a449640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f664415e060 0x7f6644153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664a449640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f664415e060 0x7f6644153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41450/0 (socket says 172.21.15.29:41450) 2024-04-25T00:32:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f664a449640 1 -- 172.21.15.29:0/3483960333 learned_addr learned my addr 172.21.15.29:0/3483960333 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f6649447640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664415bce0 0x7f66440ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.607+0000 7f6649c48640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6644072490 0x7f6644152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f664a449640 1 -- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664415bce0 msgr2=0x7f66440ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f664a449640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664415bce0 0x7f66440ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f664a449640 1 -- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6644072490 msgr2=0x7f6644152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f664a449640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6644072490 0x7f6644152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f664a449640 1 -- 172.21.15.29:0/3483960333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6644150c90 con 0x7f664415e060 2024-04-25T00:32:07.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f6649c48640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6644072490 0x7f6644152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:07.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f664a449640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f664415e060 0x7f6644153050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f663c0af040 tx=0x7f663c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:07.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f66367fc640 1 -- 172.21.15.29:0/3483960333 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c0bc020 con 0x7f664415e060 2024-04-25T00:32:07.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f66367fc640 1 -- 172.21.15.29:0/3483960333 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f663c0b8670 con 0x7f664415e060 2024-04-25T00:32:07.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f664beac640 1 -- 172.21.15.29:0/3483960333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66440ca6a0 con 0x7f664415e060 2024-04-25T00:32:07.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f66367fc640 1 -- 172.21.15.29:0/3483960333 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c0b8a40 con 0x7f664415e060 2024-04-25T00:32:07.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f664beac640 1 -- 172.21.15.29:0/3483960333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6644151e90 con 0x7f664415e060 2024-04-25T00:32:07.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f664beac640 1 -- 172.21.15.29:0/3483960333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6644069270 con 0x7f664415e060 2024-04-25T00:32:07.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f66367fc640 1 -- 172.21.15.29:0/3483960333 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f663c0bf020 con 0x7f664415e060 2024-04-25T00:32:07.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f66367fc640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f66280678c0 0x7f6628069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f66367fc640 1 -- 172.21.15.29:0/3483960333 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f663c120c50 con 0x7f664415e060 2024-04-25T00:32:07.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f6649c48640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f66280678c0 0x7f6628069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.611+0000 7f6649c48640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f66280678c0 0x7f6628069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f664000b9f0 tx=0x7f6640008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:07.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.615+0000 7f66367fc640 1 -- 172.21.15.29:0/3483960333 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f663c126020 con 0x7f664415e060 2024-04-25T00:32:07.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.779+0000 7f664beac640 1 -- 172.21.15.29:0/3483960333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f66440cb0a0 con 0x7f664415e060 2024-04-25T00:32:07.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.779+0000 7f66367fc640 1 -- 172.21.15.29:0/3483960333 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f663c0c6030 con 0x7f664415e060 2024-04-25T00:32:07.784 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:07.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.783+0000 7f664beac640 1 -- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f66280678c0 msgr2=0x7f6628069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.783+0000 7f664beac640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f66280678c0 0x7f6628069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f664000b9f0 tx=0x7f6640008040 comp rx=0 tx=0).stop 2024-04-25T00:32:07.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.783+0000 7f664beac640 1 -- 172.21.15.29:0/3483960333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f664415e060 msgr2=0x7f6644153050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.783+0000 7f664beac640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f664415e060 0x7f6644153050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f663c0af040 tx=0x7f663c019760 comp rx=0 tx=0).stop 2024-04-25T00:32:07.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.783+0000 7f664beac640 1 -- 172.21.15.29:0/3483960333 shutdown_connections 2024-04-25T00:32:07.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.783+0000 7f664beac640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f664415e060 0x7f6644153050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.783+0000 7f664beac640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664415bce0 0x7f66440ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.783+0000 7f664beac640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f66280678c0 0x7f6628069d80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.783+0000 7f664beac640 1 --2- 172.21.15.29:0/3483960333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6644072490 0x7f6644152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.783+0000 7f664beac640 1 -- 172.21.15.29:0/3483960333 >> 172.21.15.29:0/3483960333 conn(0x7f664406c320 msgr2=0x7f664406b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:07.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.783+0000 7f664beac640 1 -- 172.21.15.29:0/3483960333 shutdown_connections 2024-04-25T00:32:07.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.783+0000 7f664beac640 1 -- 172.21.15.29:0/3483960333 wait complete. 2024-04-25T00:32:07.802 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:07.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c419a3640 1 Processor -- start 2024-04-25T00:32:07.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c419a3640 1 -- start start 2024-04-25T00:32:07.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c419a3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7c3c15ad20 0x7f7c3c0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c419a3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3c0d0390 0x7f7c3c15b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c419a3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c3c15b640 0x7f7c3c071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c419a3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7c3c0cb6a0 con 0x7f7c3c15b640 2024-04-25T00:32:07.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c419a3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7c3c15bc30 con 0x7f7c3c15ad20 2024-04-25T00:32:07.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c419a3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3c15bf10 con 0x7f7c3c0d0390 2024-04-25T00:32:07.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c3b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c3c15b640 0x7f7c3c071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c3b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c3c15b640 0x7f7c3c071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40246/0 (socket says 172.21.15.29:40246) 2024-04-25T00:32:07.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c3b7fe640 1 -- 172.21.15.29:0/1655243943 learned_addr learned my addr 172.21.15.29:0/1655243943 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:07.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c3a7fc640 1 --2- 172.21.15.29:0/1655243943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3c0d0390 0x7f7c3c15b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c3affd640 1 --2- 172.21.15.29:0/1655243943 >> v2:172.21.15.157:3300/0 conn(0x7f7c3c15ad20 0x7f7c3c0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c3b7fe640 1 -- 172.21.15.29:0/1655243943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3c0d0390 msgr2=0x7f7c3c15b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:07.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c3b7fe640 1 --2- 172.21.15.29:0/1655243943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3c0d0390 0x7f7c3c15b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c3b7fe640 1 -- 172.21.15.29:0/1655243943 >> v2:172.21.15.157:3300/0 conn(0x7f7c3c15ad20 msgr2=0x7f7c3c0cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:07.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c3b7fe640 1 --2- 172.21.15.29:0/1655243943 >> v2:172.21.15.157:3300/0 conn(0x7f7c3c15ad20 0x7f7c3c0cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.923+0000 7f7c3b7fe640 1 -- 172.21.15.29:0/1655243943 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c15c1f0 con 0x7f7c3c15b640 2024-04-25T00:32:07.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c3b7fe640 1 --2- 172.21.15.29:0/1655243943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3c15b640 0x7f7c3c071d80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0cd620 tx=0x7f7c3401ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08af60504125e50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:07.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c39ffb640 1 -- 172.21.15.29:0/1655243943 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3400b2d0 con 0x7f7c3c15b640 2024-04-25T00:32:07.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c39ffb640 1 -- 172.21.15.29:0/1655243943 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c3400b470 con 0x7f7c3c15b640 2024-04-25T00:32:07.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c39ffb640 1 -- 172.21.15.29:0/1655243943 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c340a7b80 con 0x7f7c3c15b640 2024-04-25T00:32:07.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 -- 172.21.15.29:0/1655243943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3c15b640 msgr2=0x7f7c3c071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 --2- 172.21.15.29:0/1655243943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3c15b640 0x7f7c3c071d80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0cd620 tx=0x7f7c3401ebc0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 -- 172.21.15.29:0/1655243943 shutdown_connections 2024-04-25T00:32:07.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 --2- 172.21.15.29:0/1655243943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3c15b640 0x7f7c3c071d80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 --2- 172.21.15.29:0/1655243943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3c0d0390 0x7f7c3c15b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 --2- 172.21.15.29:0/1655243943 >> v2:172.21.15.157:3300/0 conn(0x7f7c3c15ad20 0x7f7c3c0cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 -- 172.21.15.29:0/1655243943 >> 172.21.15.29:0/1655243943 conn(0x7f7c3c0d0c40 msgr2=0x7f7c3c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:07.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 -- 172.21.15.29:0/1655243943 shutdown_connections 2024-04-25T00:32:07.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 -- 172.21.15.29:0/1655243943 wait complete. 2024-04-25T00:32:07.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 Processor -- start 2024-04-25T00:32:07.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 -- start start 2024-04-25T00:32:07.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c3c0d0390 0x7f7c3c14a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3c15ad20 0x7f7c3c14fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3c15b640 0x7f7c3c1503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c0ce810 con 0x7f7c3c15ad20 2024-04-25T00:32:07.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c06f740 con 0x7f7c3c0d0390 2024-04-25T00:32:07.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c419a3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3c0ce990 con 0x7f7c3c15b640 2024-04-25T00:32:07.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c3a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3c15ad20 0x7f7c3c14fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c3a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3c15ad20 0x7f7c3c14fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40250/0 (socket says 172.21.15.29:40250) 2024-04-25T00:32:07.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.927+0000 7f7c3a7fc640 1 -- 172.21.15.29:0/753082949 learned_addr learned my addr 172.21.15.29:0/753082949 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:07.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c3b7fe640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3c15b640 0x7f7c3c1503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c3affd640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c3c0d0390 0x7f7c3c14a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c3a7fc640 1 -- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3c15b640 msgr2=0x7f7c3c1503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c3a7fc640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3c15b640 0x7f7c3c1503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c3a7fc640 1 -- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c3c0d0390 msgr2=0x7f7c3c14a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:07.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c3a7fc640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c3c0d0390 0x7f7c3c14a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:07.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c3a7fc640 1 -- 172.21.15.29:0/753082949 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c1f2a10 con 0x7f7c3c15ad20 2024-04-25T00:32:07.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c3a7fc640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3c15ad20 0x7f7c3c14fe90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000d7d0 tx=0x7f7c3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:07.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c23fff640 1 -- 172.21.15.29:0/753082949 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000b6c0 con 0x7f7c3c15ad20 2024-04-25T00:32:07.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c23fff640 1 -- 172.21.15.29:0/753082949 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c30019070 con 0x7f7c3c15ad20 2024-04-25T00:32:07.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c419a3640 1 -- 172.21.15.29:0/753082949 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3c1f2cf0 con 0x7f7c3c15ad20 2024-04-25T00:32:07.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c23fff640 1 -- 172.21.15.29:0/753082949 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000baa0 con 0x7f7c3c15ad20 2024-04-25T00:32:07.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c419a3640 1 -- 172.21.15.29:0/753082949 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3c1f32b0 con 0x7f7c3c15ad20 2024-04-25T00:32:07.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c419a3640 1 -- 172.21.15.29:0/753082949 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c3c069270 con 0x7f7c3c15ad20 2024-04-25T00:32:07.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c23fff640 1 -- 172.21.15.29:0/753082949 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7c3000bc40 con 0x7f7c3c15ad20 2024-04-25T00:32:07.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c23fff640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c280678c0 0x7f7c28069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:07.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c23fff640 1 -- 172.21.15.29:0/753082949 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7c30086150 con 0x7f7c3c15ad20 2024-04-25T00:32:07.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c3affd640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c280678c0 0x7f7c28069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:07.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.931+0000 7f7c3affd640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c280678c0 0x7f7c28069d80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00b950 tx=0x7f7c2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:07.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:07.935+0000 7f7c23fff640 1 -- 172.21.15.29:0/753082949 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7c30056c90 con 0x7f7c3c15ad20 2024-04-25T00:32:08.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c419a3640 1 -- 172.21.15.29:0/753082949 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7c3c14daf0 con 0x7f7c3c15ad20 2024-04-25T00:32:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c23fff640 1 -- 172.21.15.29:0/753082949 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f7c30056590 con 0x7f7c3c15ad20 2024-04-25T00:32:08.103 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c419a3640 1 -- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c280678c0 msgr2=0x7f7c28069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c419a3640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c280678c0 0x7f7c28069d80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00b950 tx=0x7f7c2c008040 comp rx=0 tx=0).stop 2024-04-25T00:32:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c419a3640 1 -- 172.21.15.29:0/753082949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3c15ad20 msgr2=0x7f7c3c14fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c419a3640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3c15ad20 0x7f7c3c14fe90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000d7d0 tx=0x7f7c3000dc90 comp rx=0 tx=0).stop 2024-04-25T00:32:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c419a3640 1 -- 172.21.15.29:0/753082949 shutdown_connections 2024-04-25T00:32:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c419a3640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3c15b640 0x7f7c3c1503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c419a3640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3c15ad20 0x7f7c3c14fe90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c419a3640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c280678c0 0x7f7c28069d80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c419a3640 1 --2- 172.21.15.29:0/753082949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c3c0d0390 0x7f7c3c14a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c419a3640 1 -- 172.21.15.29:0/753082949 >> 172.21.15.29:0/753082949 conn(0x7f7c3c0d0c40 msgr2=0x7f7c3c0cc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:08.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c419a3640 1 -- 172.21.15.29:0/753082949 shutdown_connections 2024-04-25T00:32:08.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.099+0000 7f7c419a3640 1 -- 172.21.15.29:0/753082949 wait complete. 2024-04-25T00:32:08.121 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:08.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.235+0000 7f4b6b8f4640 1 Processor -- start 2024-04-25T00:32:08.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.235+0000 7f4b6b8f4640 1 -- start start 2024-04-25T00:32:08.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b6b8f4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4b640cf690 0x7f4b640d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b6b8f4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b640d2ea0 con 0x7f4b6415e200 2024-04-25T00:32:08.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b6b8f4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b640cfc10 con 0x7f4b640d2a90 2024-04-25T00:32:08.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b6b8f4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4b640b38d0 con 0x7f4b640cf690 2024-04-25T00:32:08.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b68e8f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4b640cf690 0x7f4b640d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b68e8f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4b640cf690 0x7f4b640d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40266/0 (socket says 172.21.15.29:40266) 2024-04-25T00:32:08.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b68e8f640 1 -- 172.21.15.29:0/568862275 learned_addr learned my addr 172.21.15.29:0/568862275 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:08.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b68e8f640 1 -- 172.21.15.29:0/568862275 >> v1:172.21.15.157:6789/0 conn(0x7f4b640d2a90 legacy=0x7f4b640b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:08.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b68e8f640 1 -- 172.21.15.29:0/568862275 >> v1:172.21.15.29:6789/0 conn(0x7f4b6415e200 legacy=0x7f4b640d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b68e8f640 1 -- 172.21.15.29:0/568862275 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b640cfef0 con 0x7f4b640cf690 2024-04-25T00:32:08.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b5bfff640 1 -- 172.21.15.29:0/568862275 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2161912050 0 0) 0x7f4b60001120 con 0x7f4b6415e200 2024-04-25T00:32:08.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b68e8f640 1 --2- 172.21.15.29:0/568862275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b640cf690 0x7f4b640d2550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009870 tx=0x7f4b5401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65aff1826dac35c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:08.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b5bfff640 1 -- 172.21.15.29:0/568862275 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b540ab610 con 0x7f4b640cf690 2024-04-25T00:32:08.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b5bfff640 1 -- 172.21.15.29:0/568862275 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b540b0070 con 0x7f4b640cf690 2024-04-25T00:32:08.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b5bfff640 1 -- 172.21.15.29:0/568862275 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b540ab9d0 con 0x7f4b640cf690 2024-04-25T00:32:08.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/568862275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b640cf690 msgr2=0x7f4b640d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b6b8f4640 1 --2- 172.21.15.29:0/568862275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b640cf690 0x7f4b640d2550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009870 tx=0x7f4b5401c520 comp rx=0 tx=0).stop 2024-04-25T00:32:08.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/568862275 shutdown_connections 2024-04-25T00:32:08.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b6b8f4640 1 --2- 172.21.15.29:0/568862275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b640cf690 0x7f4b640d2550 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/568862275 >> 172.21.15.29:0/568862275 conn(0x7f4b6409d280 msgr2=0x7f4b640d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:08.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/568862275 shutdown_connections 2024-04-25T00:32:08.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.239+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/568862275 wait complete. 2024-04-25T00:32:08.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b6b8f4640 1 Processor -- start 2024-04-25T00:32:08.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b6b8f4640 1 -- start start 2024-04-25T00:32:08.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b6b8f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b640cf690 0x7f4b64154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b6b8f4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b640d2a90 0x7f4b64159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b6b8f4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6415e200 0x7f4b64156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b6b8f4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4b640b53c0 con 0x7f4b6415e200 2024-04-25T00:32:08.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b6b8f4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b640b5240 con 0x7f4b640cf690 2024-04-25T00:32:08.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b6b8f4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b640b5540 con 0x7f4b640d2a90 2024-04-25T00:32:08.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b69e91640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6415e200 0x7f4b64156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b69e91640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6415e200 0x7f4b64156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40282/0 (socket says 172.21.15.29:40282) 2024-04-25T00:32:08.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b69e91640 1 -- 172.21.15.29:0/2163589967 learned_addr learned my addr 172.21.15.29:0/2163589967 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b69690640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b640cf690 0x7f4b64154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b68e8f640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b640d2a90 0x7f4b64159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b69e91640 1 -- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b640d2a90 msgr2=0x7f4b64159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b69e91640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b640d2a90 0x7f4b64159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b69e91640 1 -- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b640cf690 msgr2=0x7f4b64154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b69e91640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b640cf690 0x7f4b64154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b69e91640 1 -- 172.21.15.29:0/2163589967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b640c7cf0 con 0x7f4b6415e200 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b69690640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b640cf690 0x7f4b64154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b69e91640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6415e200 0x7f4b64156250 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00b970 tx=0x7f4b5c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b59ffb640 1 -- 172.21.15.29:0/2163589967 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c019070 con 0x7f4b6415e200 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b59ffb640 1 -- 172.21.15.29:0/2163589967 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b5c00cae0 con 0x7f4b6415e200 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b59ffb640 1 -- 172.21.15.29:0/2163589967 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c014440 con 0x7f4b6415e200 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/2163589967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b640c7fd0 con 0x7f4b6415e200 2024-04-25T00:32:08.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/2163589967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b640c8590 con 0x7f4b6415e200 2024-04-25T00:32:08.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/2163589967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b64069270 con 0x7f4b6415e200 2024-04-25T00:32:08.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b59ffb640 1 -- 172.21.15.29:0/2163589967 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4b5c01c070 con 0x7f4b6415e200 2024-04-25T00:32:08.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.243+0000 7f4b59ffb640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b480678c0 0x7f4b48069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.247+0000 7f4b59ffb640 1 -- 172.21.15.29:0/2163589967 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4b5c0857f0 con 0x7f4b6415e200 2024-04-25T00:32:08.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.247+0000 7f4b69690640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b480678c0 0x7f4b48069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.247+0000 7f4b69690640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b480678c0 0x7f4b48069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4b60000f70 tx=0x7f4b60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:08.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.251+0000 7f4b59ffb640 1 -- 172.21.15.29:0/2163589967 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4b5c0563c0 con 0x7f4b6415e200 2024-04-25T00:32:08.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.407+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/2163589967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4b641551b0 con 0x7f4b6415e200 2024-04-25T00:32:08.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.407+0000 7f4b59ffb640 1 -- 172.21.15.29:0/2163589967 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f4b5c055cc0 con 0x7f4b6415e200 2024-04-25T00:32:08.411 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.407+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b480678c0 msgr2=0x7f4b48069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.407+0000 7f4b6b8f4640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b480678c0 0x7f4b48069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4b60000f70 tx=0x7f4b60009040 comp rx=0 tx=0).stop 2024-04-25T00:32:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.407+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/2163589967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6415e200 msgr2=0x7f4b64156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.407+0000 7f4b6b8f4640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6415e200 0x7f4b64156250 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00b970 tx=0x7f4b5c00be30 comp rx=0 tx=0).stop 2024-04-25T00:32:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.411+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/2163589967 shutdown_connections 2024-04-25T00:32:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.411+0000 7f4b6b8f4640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6415e200 0x7f4b64156250 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.411+0000 7f4b6b8f4640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b640d2a90 0x7f4b64159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.411+0000 7f4b6b8f4640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b480678c0 0x7f4b48069d80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.411+0000 7f4b6b8f4640 1 --2- 172.21.15.29:0/2163589967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b640cf690 0x7f4b64154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.411+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/2163589967 >> 172.21.15.29:0/2163589967 conn(0x7f4b6409d280 msgr2=0x7f4b640b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.411+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/2163589967 shutdown_connections 2024-04-25T00:32:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.411+0000 7f4b6b8f4640 1 -- 172.21.15.29:0/2163589967 wait complete. 2024-04-25T00:32:08.429 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:32:08.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa34463c640 1 Processor -- start 2024-04-25T00:32:08.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa34463c640 1 -- start start 2024-04-25T00:32:08.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa34463c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa33c0d09c0 0x7fa33c15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa34463c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa33c0b3a90 0x7fa33c143d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa34463c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa33c1443f0 con 0x7fa33c143880 2024-04-25T00:32:08.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa34463c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa33c15ee20 con 0x7fa33c0b3a90 2024-04-25T00:32:08.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa34463c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa33c15e7d0 con 0x7fa33c0d09c0 2024-04-25T00:32:08.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa342bd9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa33c0b3a90 0x7fa33c143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa342bd9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa33c0b3a90 0x7fa33c143d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40286/0 (socket says 172.21.15.29:40286) 2024-04-25T00:32:08.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa342bd9640 1 -- 172.21.15.29:0/2974208697 learned_addr learned my addr 172.21.15.29:0/2974208697 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:08.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa3423d8640 1 --2- 172.21.15.29:0/2974208697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa33c0d09c0 0x7fa33c15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa342bd9640 1 -- 172.21.15.29:0/2974208697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa33c0d09c0 msgr2=0x7fa33c15f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:08.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa342bd9640 1 --2- 172.21.15.29:0/2974208697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa33c0d09c0 0x7fa33c15f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa342bd9640 1 -- 172.21.15.29:0/2974208697 >> v1:172.21.15.157:6789/0 conn(0x7fa33c143880 legacy=0x7fa33c0b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa342bd9640 1 -- 172.21.15.29:0/2974208697 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c15eab0 con 0x7fa33c0b3a90 2024-04-25T00:32:08.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.547+0000 7fa3423d8640 1 --2- 172.21.15.29:0/2974208697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa33c0d09c0 0x7fa33c15f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:08.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa342bd9640 1 --2- 172.21.15.29:0/2974208697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa33c0b3a90 0x7fa33c143d00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa330009870 tx=0x7fa33001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56571ef20a06271d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:08.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa3413d6640 1 -- 172.21.15.29:0/2974208697 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3300a7070 con 0x7fa33c0b3a90 2024-04-25T00:32:08.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa3413d6640 1 -- 172.21.15.29:0/2974208697 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3300b0070 con 0x7fa33c0b3a90 2024-04-25T00:32:08.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa3413d6640 1 -- 172.21.15.29:0/2974208697 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3300ab890 con 0x7fa33c0b3a90 2024-04-25T00:32:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa34463c640 1 -- 172.21.15.29:0/2974208697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa33c0b3a90 msgr2=0x7fa33c143d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa34463c640 1 --2- 172.21.15.29:0/2974208697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa33c0b3a90 0x7fa33c143d00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa330009870 tx=0x7fa33001c560 comp rx=0 tx=0).stop 2024-04-25T00:32:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa34463c640 1 -- 172.21.15.29:0/2974208697 shutdown_connections 2024-04-25T00:32:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa34463c640 1 --2- 172.21.15.29:0/2974208697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa33c0b3a90 0x7fa33c143d00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa34463c640 1 --2- 172.21.15.29:0/2974208697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa33c0d09c0 0x7fa33c15f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa34463c640 1 -- 172.21.15.29:0/2974208697 >> 172.21.15.29:0/2974208697 conn(0x7fa33c074c00 msgr2=0x7fa33c06b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa34463c640 1 -- 172.21.15.29:0/2974208697 shutdown_connections 2024-04-25T00:32:08.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa34463c640 1 -- 172.21.15.29:0/2974208697 wait complete. 2024-04-25T00:32:08.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa34463c640 1 Processor -- start 2024-04-25T00:32:08.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.551+0000 7fa34463c640 1 -- start start 2024-04-25T00:32:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa34463c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa33c0b3a90 0x7fa33c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa34463c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33c0d09c0 0x7fa33c157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa34463c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa33c143880 0x7fa33c1576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa34463c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c162850 con 0x7fa33c143880 2024-04-25T00:32:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa34463c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1626d0 con 0x7fa33c0d09c0 2024-04-25T00:32:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa34463c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa33c1629d0 con 0x7fa33c0b3a90 2024-04-25T00:32:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa342bd9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa33c143880 0x7fa33c1576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa342bd9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa33c143880 0x7fa33c1576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40298/0 (socket says 172.21.15.29:40298) 2024-04-25T00:32:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa342bd9640 1 -- 172.21.15.29:0/940329670 learned_addr learned my addr 172.21.15.29:0/940329670 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa3423d8640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa33c0b3a90 0x7fa33c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa341bd7640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33c0d09c0 0x7fa33c157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa342bd9640 1 -- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa33c0b3a90 msgr2=0x7fa33c156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa342bd9640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa33c0b3a90 0x7fa33c156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa342bd9640 1 -- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33c0d09c0 msgr2=0x7fa33c157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa342bd9640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33c0d09c0 0x7fa33c157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa342bd9640 1 -- 172.21.15.29:0/940329670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c160b80 con 0x7fa33c143880 2024-04-25T00:32:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa341bd7640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33c0d09c0 0x7fa33c157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa342bd9640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa33c143880 0x7fa33c1576a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa3300ad040 tx=0x7fa33001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa32effd640 1 -- 172.21.15.29:0/940329670 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3300be5f0 con 0x7fa33c143880 2024-04-25T00:32:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa32effd640 1 -- 172.21.15.29:0/940329670 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3300be790 con 0x7fa33c143880 2024-04-25T00:32:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa34463c640 1 -- 172.21.15.29:0/940329670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33c0ad940 con 0x7fa33c143880 2024-04-25T00:32:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa32effd640 1 -- 172.21.15.29:0/940329670 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3300beb80 con 0x7fa33c143880 2024-04-25T00:32:08.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa34463c640 1 -- 172.21.15.29:0/940329670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa33c0adf00 con 0x7fa33c143880 2024-04-25T00:32:08.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa34463c640 1 -- 172.21.15.29:0/940329670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33c069270 con 0x7fa33c143880 2024-04-25T00:32:08.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa32effd640 1 -- 172.21.15.29:0/940329670 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa3300b43f0 con 0x7fa33c143880 2024-04-25T00:32:08.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa32effd640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa3200678c0 0x7fa320069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa32effd640 1 -- 172.21.15.29:0/940329670 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3301206c0 con 0x7fa33c143880 2024-04-25T00:32:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.555+0000 7fa3423d8640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa3200678c0 0x7fa320069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.559+0000 7fa3423d8640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa3200678c0 0x7fa320069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa338009a80 tx=0x7fa338003a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:08.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.563+0000 7fa32effd640 1 -- 172.21.15.29:0/940329670 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa330016150 con 0x7fa33c143880 2024-04-25T00:32:08.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.735+0000 7fa34463c640 1 -- 172.21.15.29:0/940329670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa33c06b9f0 con 0x7fa33c143880 2024-04-25T00:32:08.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.735+0000 7fa32effd640 1 -- 172.21.15.29:0/940329670 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fa3300f13b0 con 0x7fa33c143880 2024-04-25T00:32:08.739 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:32:08.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.739+0000 7fa34463c640 1 -- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa3200678c0 msgr2=0x7fa320069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.739+0000 7fa34463c640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa3200678c0 0x7fa320069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa338009a80 tx=0x7fa338003a00 comp rx=0 tx=0).stop 2024-04-25T00:32:08.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.739+0000 7fa34463c640 1 -- 172.21.15.29:0/940329670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa33c143880 msgr2=0x7fa33c1576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.739+0000 7fa34463c640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa33c143880 0x7fa33c1576a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa3300ad040 tx=0x7fa33001ada0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.739+0000 7fa34463c640 1 -- 172.21.15.29:0/940329670 shutdown_connections 2024-04-25T00:32:08.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.739+0000 7fa34463c640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa33c143880 0x7fa33c1576a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.739+0000 7fa34463c640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa3200678c0 0x7fa320069d80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.739+0000 7fa34463c640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33c0d09c0 0x7fa33c157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.739+0000 7fa34463c640 1 --2- 172.21.15.29:0/940329670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa33c0b3a90 0x7fa33c156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.739+0000 7fa34463c640 1 -- 172.21.15.29:0/940329670 >> 172.21.15.29:0/940329670 conn(0x7fa33c074c00 msgr2=0x7fa33c0b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:08.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.739+0000 7fa34463c640 1 -- 172.21.15.29:0/940329670 shutdown_connections 2024-04-25T00:32:08.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.739+0000 7fa34463c640 1 -- 172.21.15.29:0/940329670 wait complete. 2024-04-25T00:32:08.759 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:32:08.759 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:08.764 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:08.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58bea23640 1 Processor -- start 2024-04-25T00:32:08.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58bea23640 1 -- start start 2024-04-25T00:32:08.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58bea23640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58b815c220 0x7f58b80cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58bea23640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f58b80d0670 0x7f58b80b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58bea23640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b80cdbd0 con 0x7f58b80cfa10 2024-04-25T00:32:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58bea23640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f58b815c600 con 0x7f58b80d0670 2024-04-25T00:32:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58bea23640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f58b80b1a30 con 0x7f58b815c220 2024-04-25T00:32:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58bcfc0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f58b80d0670 0x7f58b80b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58b77fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f58b80cfa10 0x7f58b80cff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35166/0 (socket says 172.21.15.29:35166) 2024-04-25T00:32:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58bcfc0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f58b80d0670 0x7f58b80b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40312/0 (socket says 172.21.15.29:40312) 2024-04-25T00:32:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58b77fe640 1 -- 172.21.15.29:0/2728774473 learned_addr learned my addr 172.21.15.29:0/2728774473 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58b7fff640 1 --2- 172.21.15.29:0/2728774473 >> v2:172.21.15.157:3300/0 conn(0x7f58b815c220 0x7f58b80cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58bcfc0640 1 -- 172.21.15.29:0/2728774473 >> v2:172.21.15.157:3300/0 conn(0x7f58b815c220 msgr2=0x7f58b80cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58bcfc0640 1 --2- 172.21.15.29:0/2728774473 >> v2:172.21.15.157:3300/0 conn(0x7f58b815c220 0x7f58b80cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58bcfc0640 1 -- 172.21.15.29:0/2728774473 >> v1:172.21.15.29:6789/0 conn(0x7f58b80cfa10 legacy=0x7f58b80cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58b6ffd640 1 -- 172.21.15.29:0/2728774473 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2489067249 0 0) 0x7f58b80cdbd0 con 0x7f58b80cfa10 2024-04-25T00:32:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58bcfc0640 1 -- 172.21.15.29:0/2728774473 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b80b13e0 con 0x7f58b80d0670 2024-04-25T00:32:08.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.927+0000 7f58b7fff640 1 --2- 172.21.15.29:0/2728774473 >> v2:172.21.15.157:3300/0 conn(0x7f58b815c220 0x7f58b80cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:08.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58bcfc0640 1 --2- 172.21.15.29:0/2728774473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58b80d0670 0x7f58b80b1ec0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f58b80cf860 tx=0x7f58b001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beddf323bf968561 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:08.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58b6ffd640 1 -- 172.21.15.29:0/2728774473 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b00b0070 con 0x7f58b80d0670 2024-04-25T00:32:08.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58b6ffd640 1 -- 172.21.15.29:0/2728774473 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b00ab600 con 0x7f58b80d0670 2024-04-25T00:32:08.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58b6ffd640 1 -- 172.21.15.29:0/2728774473 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b00ab9d0 con 0x7f58b80d0670 2024-04-25T00:32:08.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58bea23640 1 -- 172.21.15.29:0/2728774473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58b80d0670 msgr2=0x7f58b80b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58bea23640 1 --2- 172.21.15.29:0/2728774473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58b80d0670 0x7f58b80b1ec0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f58b80cf860 tx=0x7f58b001c480 comp rx=0 tx=0).stop 2024-04-25T00:32:08.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58bea23640 1 -- 172.21.15.29:0/2728774473 shutdown_connections 2024-04-25T00:32:08.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58bea23640 1 --2- 172.21.15.29:0/2728774473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58b80d0670 0x7f58b80b1ec0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58bea23640 1 --2- 172.21.15.29:0/2728774473 >> v2:172.21.15.157:3300/0 conn(0x7f58b815c220 0x7f58b80cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58bea23640 1 -- 172.21.15.29:0/2728774473 >> 172.21.15.29:0/2728774473 conn(0x7f58b80d1d60 msgr2=0x7f58b80d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:08.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58bea23640 1 -- 172.21.15.29:0/2728774473 shutdown_connections 2024-04-25T00:32:08.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58bea23640 1 -- 172.21.15.29:0/2728774473 wait complete. 2024-04-25T00:32:08.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58bea23640 1 Processor -- start 2024-04-25T00:32:08.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.931+0000 7f58bea23640 1 -- start start 2024-04-25T00:32:08.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58bea23640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58b80cfa10 0x7f58b814b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58bea23640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58b80d0670 0x7f58b815ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58bea23640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58b815c220 0x7f58b80c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58bea23640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f58b80b2fe0 con 0x7f58b80cfa10 2024-04-25T00:32:08.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58bea23640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58b80b2e60 con 0x7f58b815c220 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58bea23640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f58b80b3160 con 0x7f58b80d0670 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58b80cfa10 0x7f58b814b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58b80cfa10 0x7f58b814b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40326/0 (socket says 172.21.15.29:40326) 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b7fff640 1 -- 172.21.15.29:0/3295672644 learned_addr learned my addr 172.21.15.29:0/3295672644 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b77fe640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58b80d0670 0x7f58b815ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58bcfc0640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58b815c220 0x7f58b80c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b7fff640 1 -- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58b80d0670 msgr2=0x7f58b815ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b7fff640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58b80d0670 0x7f58b815ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b7fff640 1 -- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58b815c220 msgr2=0x7f58b80c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b7fff640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58b815c220 0x7f58b80c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b7fff640 1 -- 172.21.15.29:0/3295672644 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b815cd20 con 0x7f58b80cfa10 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b7fff640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58b80cfa10 0x7f58b814b2b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f58a800cf60 tx=0x7f58a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b4ff9640 1 -- 172.21.15.29:0/3295672644 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a801a070 con 0x7f58b80cfa10 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b4ff9640 1 -- 172.21.15.29:0/3295672644 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a800d840 con 0x7f58b80cfa10 2024-04-25T00:32:08.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58bea23640 1 -- 172.21.15.29:0/3295672644 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b815d000 con 0x7f58b80cfa10 2024-04-25T00:32:08.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b4ff9640 1 -- 172.21.15.29:0/3295672644 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a800dc10 con 0x7f58b80cfa10 2024-04-25T00:32:08.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58bea23640 1 -- 172.21.15.29:0/3295672644 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b815d8a0 con 0x7f58b80cfa10 2024-04-25T00:32:08.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58bea23640 1 -- 172.21.15.29:0/3295672644 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b815d1d0 con 0x7f58b80cfa10 2024-04-25T00:32:08.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b4ff9640 1 -- 172.21.15.29:0/3295672644 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f58a801d070 con 0x7f58b80cfa10 2024-04-25T00:32:08.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b4ff9640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f58a40678f0 0x7f58a4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:08.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b4ff9640 1 -- 172.21.15.29:0/3295672644 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f58a80863a0 con 0x7f58b80cfa10 2024-04-25T00:32:08.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.935+0000 7f58b77fe640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f58a40678f0 0x7f58a4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:08.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.939+0000 7f58b77fe640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f58a40678f0 0x7f58a4069db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f58ac007fd0 tx=0x7f58ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:08.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:08.939+0000 7f58b4ff9640 1 -- 172.21.15.29:0/3295672644 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f58a8056920 con 0x7f58b80cfa10 2024-04-25T00:32:09.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.103+0000 7f58bea23640 1 -- 172.21.15.29:0/3295672644 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58b8069270 con 0x7f58b80cfa10 2024-04-25T00:32:09.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.103+0000 7f58b4ff9640 1 -- 172.21.15.29:0/3295672644 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f58a805ac20 con 0x7f58b80cfa10 2024-04-25T00:32:09.112 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:09.112 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:09.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.107+0000 7f58bea23640 1 -- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f58a40678f0 msgr2=0x7f58a4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.107+0000 7f58bea23640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f58a40678f0 0x7f58a4069db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f58ac007fd0 tx=0x7f58ac009040 comp rx=0 tx=0).stop 2024-04-25T00:32:09.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.107+0000 7f58bea23640 1 -- 172.21.15.29:0/3295672644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58b80cfa10 msgr2=0x7f58b814b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.107+0000 7f58bea23640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58b80cfa10 0x7f58b814b2b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f58a800cf60 tx=0x7f58a800cf90 comp rx=0 tx=0).stop 2024-04-25T00:32:09.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.107+0000 7f58bea23640 1 -- 172.21.15.29:0/3295672644 shutdown_connections 2024-04-25T00:32:09.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.107+0000 7f58bea23640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f58a40678f0 0x7f58a4069db0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.107+0000 7f58bea23640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58b815c220 0x7f58b80c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.107+0000 7f58bea23640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58b80d0670 0x7f58b815ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.107+0000 7f58bea23640 1 --2- 172.21.15.29:0/3295672644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58b80cfa10 0x7f58b814b2b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.107+0000 7f58bea23640 1 -- 172.21.15.29:0/3295672644 >> 172.21.15.29:0/3295672644 conn(0x7f58b80d1d60 msgr2=0x7f58b8155450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:09.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.107+0000 7f58bea23640 1 -- 172.21.15.29:0/3295672644 shutdown_connections 2024-04-25T00:32:09.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.107+0000 7f58bea23640 1 -- 172.21.15.29:0/3295672644 wait complete. 2024-04-25T00:32:09.127 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:09.127 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.247+0000 7f810f95a640 1 Processor -- start 2024-04-25T00:32:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.247+0000 7f810f95a640 1 -- start start 2024-04-25T00:32:09.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810f95a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81080d06c0 0x7f8108072040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810f95a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f810806b210 0x7f810806b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810f95a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81080cd740 con 0x7f810815f9a0 2024-04-25T00:32:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810f95a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f810806bbb0 con 0x7f810806b210 2024-04-25T00:32:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810f95a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f81080d0aa0 con 0x7f81080d06c0 2024-04-25T00:32:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810def7640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f810815f9a0 0x7f810815fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35168/0 (socket says 172.21.15.29:35168) 2024-04-25T00:32:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810def7640 1 -- 172.21.15.29:0/326761807 learned_addr learned my addr 172.21.15.29:0/326761807 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810d6f6640 1 --2- 172.21.15.29:0/326761807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81080d06c0 0x7f8108072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810cef5640 1 --2- 172.21.15.29:0/326761807 >> v2:172.21.15.157:3300/0 conn(0x7f810806b210 0x7f810806b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2283683904 0 0) 0x7f81080cd740 con 0x7f810815f9a0 2024-04-25T00:32:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80f4003530 con 0x7f810815f9a0 2024-04-25T00:32:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4117108924 0 0) 0x7f80f4003530 con 0x7f810815f9a0 2024-04-25T00:32:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81080d06c0 msgr2=0x7f8108072040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 --2- 172.21.15.29:0/326761807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81080d06c0 0x7f8108072040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 >> v2:172.21.15.157:3300/0 conn(0x7f810806b210 msgr2=0x7f810806b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 --2- 172.21.15.29:0/326761807 >> v2:172.21.15.157:3300/0 conn(0x7f810806b210 0x7f810806b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81080cda20 con 0x7f810815f9a0 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8100001a00 con 0x7f810815f9a0 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 >> v1:172.21.15.29:6789/0 conn(0x7f810815f9a0 legacy=0x7f810815fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 --2- 172.21.15.29:0/326761807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80f40039c0 0x7f80f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 --2- 172.21.15.29:0/326761807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81080d06c0 0x7f80f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 --2- 172.21.15.29:0/326761807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80f4004d50 0x7f80f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81080d0aa0 con 0x7f80f4004d50 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f80f400b9d0 con 0x7f80f40039c0 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f80f400bb50 con 0x7f81080d06c0 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810def7640 1 --2- 172.21.15.29:0/326761807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80f4004d50 0x7f80f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810d6f6640 1 --2- 172.21.15.29:0/326761807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80f40039c0 0x7f80f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810cef5640 1 --2- 172.21.15.29:0/326761807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81080d06c0 0x7f80f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810def7640 1 -- 172.21.15.29:0/326761807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81080d06c0 msgr2=0x7f80f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810def7640 1 --2- 172.21.15.29:0/326761807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81080d06c0 0x7f80f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810def7640 1 -- 172.21.15.29:0/326761807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80f40039c0 msgr2=0x7f80f4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:09.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810def7640 1 --2- 172.21.15.29:0/326761807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80f40039c0 0x7f80f4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810def7640 1 -- 172.21.15.29:0/326761807 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f4003530 con 0x7f80f4004d50 2024-04-25T00:32:09.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.251+0000 7f810d6f6640 1 --2- 172.21.15.29:0/326761807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80f40039c0 0x7f80f4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810def7640 1 --2- 172.21.15.29:0/326761807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80f4004d50 0x7f80f40051d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f810000a880 tx=0x7f810001fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b77fb3e165b8aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810000a4a0 con 0x7f80f4004d50 2024-04-25T00:32:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81000ad020 con 0x7f80f4004d50 2024-04-25T00:32:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f80fffff640 1 -- 172.21.15.29:0/326761807 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81000a9740 con 0x7f80f4004d50 2024-04-25T00:32:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 -- 172.21.15.29:0/326761807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80f4004d50 msgr2=0x7f80f40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 --2- 172.21.15.29:0/326761807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80f4004d50 0x7f80f40051d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f810000a880 tx=0x7f810001fb00 comp rx=0 tx=0).stop 2024-04-25T00:32:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810def7640 1 -- 172.21.15.29:0/326761807 reap_dead start 2024-04-25T00:32:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 -- 172.21.15.29:0/326761807 shutdown_connections 2024-04-25T00:32:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 -- 172.21.15.29:0/326761807 >> 172.21.15.29:0/326761807 conn(0x7f81080b2920 msgr2=0x7f81080b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 -- 172.21.15.29:0/326761807 shutdown_connections 2024-04-25T00:32:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 -- 172.21.15.29:0/326761807 wait complete. 2024-04-25T00:32:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 Processor -- start 2024-04-25T00:32:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 -- start start 2024-04-25T00:32:09.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81081f5120 0x7f810815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f810815c9e0 0x7f81081403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81081f5500 0x7f8108140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81080cdfe0 con 0x7f81081f5500 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81080ce350 con 0x7f81081f5120 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810f95a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8108145960 con 0x7f810815c9e0 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810def7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81081f5500 0x7f8108140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810def7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81081f5500 0x7f8108140910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40356/0 (socket says 172.21.15.29:40356) 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810def7640 1 -- 172.21.15.29:0/2170236233 learned_addr learned my addr 172.21.15.29:0/2170236233 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810d6f6640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81081f5120 0x7f810815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810cef5640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f810815c9e0 0x7f81081403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810def7640 1 -- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f810815c9e0 msgr2=0x7f81081403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810def7640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f810815c9e0 0x7f81081403d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810def7640 1 -- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81081f5120 msgr2=0x7f810815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810def7640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81081f5120 0x7f810815c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810def7640 1 -- 172.21.15.29:0/2170236233 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8108140e50 con 0x7f81081f5500 2024-04-25T00:32:09.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810d6f6640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81081f5120 0x7f810815c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:09.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.255+0000 7f810cef5640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f810815c9e0 0x7f81081403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:09.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.259+0000 7f810def7640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81081f5500 0x7f8108140910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f810000a880 tx=0x7f80f4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:09.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.259+0000 7f80fdffb640 1 -- 172.21.15.29:0/2170236233 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810000a4a0 con 0x7f81081f5500 2024-04-25T00:32:09.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.259+0000 7f80fdffb640 1 -- 172.21.15.29:0/2170236233 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81000b6070 con 0x7f81081f5500 2024-04-25T00:32:09.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.259+0000 7f810f95a640 1 -- 172.21.15.29:0/2170236233 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810815cf40 con 0x7f81081f5500 2024-04-25T00:32:09.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.259+0000 7f80fdffb640 1 -- 172.21.15.29:0/2170236233 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100002000 con 0x7f81081f5500 2024-04-25T00:32:09.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.259+0000 7f810f95a640 1 -- 172.21.15.29:0/2170236233 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810815d500 con 0x7f81081f5500 2024-04-25T00:32:09.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.259+0000 7f810f95a640 1 -- 172.21.15.29:0/2170236233 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8108069270 con 0x7f81081f5500 2024-04-25T00:32:09.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.259+0000 7f80fdffb640 1 -- 172.21.15.29:0/2170236233 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f81000a9940 con 0x7f81081f5500 2024-04-25T00:32:09.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.259+0000 7f80fdffb640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f80ec0678f0 0x7f80ec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.259+0000 7f80fdffb640 1 -- 172.21.15.29:0/2170236233 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8100122d40 con 0x7f81081f5500 2024-04-25T00:32:09.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.259+0000 7f810d6f6640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f80ec0678f0 0x7f80ec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.259+0000 7f810d6f6640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f80ec0678f0 0x7f80ec069db0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8104006f90 tx=0x7f8104008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:09.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.263+0000 7f80fdffb640 1 -- 172.21.15.29:0/2170236233 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8100019810 con 0x7f81081f5500 2024-04-25T00:32:09.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f810f95a640 1 -- 172.21.15.29:0/2170236233 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f81080670f0 con 0x7f81081f5500 2024-04-25T00:32:09.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f80fdffb640 1 -- 172.21.15.29:0/2170236233 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f81000f3750 con 0x7f81081f5500 2024-04-25T00:32:09.434 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:09.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f810f95a640 1 -- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f80ec0678f0 msgr2=0x7f80ec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f810f95a640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f80ec0678f0 0x7f80ec069db0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8104006f90 tx=0x7f8104008040 comp rx=0 tx=0).stop 2024-04-25T00:32:09.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f810f95a640 1 -- 172.21.15.29:0/2170236233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81081f5500 msgr2=0x7f8108140910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f810f95a640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81081f5500 0x7f8108140910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f810000a880 tx=0x7f80f4005840 comp rx=0 tx=0).stop 2024-04-25T00:32:09.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f810f95a640 1 -- 172.21.15.29:0/2170236233 shutdown_connections 2024-04-25T00:32:09.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f810f95a640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81081f5500 0x7f8108140910 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f810f95a640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f810815c9e0 0x7f81081403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f810f95a640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f80ec0678f0 0x7f80ec069db0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f810f95a640 1 --2- 172.21.15.29:0/2170236233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81081f5120 0x7f810815c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f810f95a640 1 -- 172.21.15.29:0/2170236233 >> 172.21.15.29:0/2170236233 conn(0x7f81080b2920 msgr2=0x7f810815fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:09.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f810f95a640 1 -- 172.21.15.29:0/2170236233 shutdown_connections 2024-04-25T00:32:09.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.431+0000 7f810f95a640 1 -- 172.21.15.29:0/2170236233 wait complete. 2024-04-25T00:32:09.454 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:09.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a0d8d5640 1 Processor -- start 2024-04-25T00:32:09.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a0d8d5640 1 -- start start 2024-04-25T00:32:09.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a0d8d5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a0813deb0 0x7f3a080ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a0d8d5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a080cf680 con 0x7f3a080cf020 2024-04-25T00:32:09.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a0d8d5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a080cf960 con 0x7f3a08140ca0 2024-04-25T00:32:09.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a0d8d5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a080cfc40 con 0x7f3a0813deb0 2024-04-25T00:32:09.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a067fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3a080cf020 0x7f3a08140590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35180/0 (socket says 172.21.15.29:35180) 2024-04-25T00:32:09.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a067fc640 1 -- 172.21.15.29:0/189509310 learned_addr learned my addr 172.21.15.29:0/189509310 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:09.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a06ffd640 1 --2- 172.21.15.29:0/189509310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a0813deb0 0x7f3a080ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4111107672 0 0) 0x7f3a080cf680 con 0x7f3a080cf020 2024-04-25T00:32:09.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39f4003530 con 0x7f3a080cf020 2024-04-25T00:32:09.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3291874313 0 0) 0x7f39f4003530 con 0x7f3a080cf020 2024-04-25T00:32:09.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a0813deb0 msgr2=0x7f3a080ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:09.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 --2- 172.21.15.29:0/189509310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a0813deb0 0x7f3a080ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 >> v1:172.21.15.157:6789/0 conn(0x7f3a08140ca0 legacy=0x7f3a0806d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0806d4d0 con 0x7f3a080cf020 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f39f0001680 con 0x7f3a080cf020 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 >> v1:172.21.15.29:6789/0 conn(0x7f3a080cf020 legacy=0x7f3a08140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 --2- 172.21.15.29:0/189509310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a0813deb0 0x7f39f40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 --2- 172.21.15.29:0/189509310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39f4003f30 0x7f39f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 --2- 172.21.15.29:0/189509310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f4004d50 0x7f39f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a080cfc40 con 0x7f39f4004d50 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39f400ba80 con 0x7f39f4003f30 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f39f400bc80 con 0x7f3a0813deb0 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a077fe640 1 --2- 172.21.15.29:0/189509310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f4004d50 0x7f39f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a067fc640 1 --2- 172.21.15.29:0/189509310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39f4003f30 0x7f39f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a06ffd640 1 --2- 172.21.15.29:0/189509310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a0813deb0 0x7f39f40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a077fe640 1 -- 172.21.15.29:0/189509310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a0813deb0 msgr2=0x7f39f40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a077fe640 1 --2- 172.21.15.29:0/189509310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a0813deb0 0x7f39f40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a077fe640 1 -- 172.21.15.29:0/189509310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39f4003f30 msgr2=0x7f39f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:09.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a077fe640 1 --2- 172.21.15.29:0/189509310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39f4003f30 0x7f39f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.575+0000 7f3a077fe640 1 -- 172.21.15.29:0/189509310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f4003530 con 0x7f39f4004d50 2024-04-25T00:32:09.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.579+0000 7f3a077fe640 1 --2- 172.21.15.29:0/189509310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f4004d50 0x7f39f40051d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a00001860 tx=0x7f3a0001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994c2743481d4ccd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:09.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.579+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00001460 con 0x7f39f4004d50 2024-04-25T00:32:09.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.579+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a000b0070 con 0x7f39f4004d50 2024-04-25T00:32:09.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.579+0000 7f3a05ffb640 1 -- 172.21.15.29:0/189509310 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a000ab880 con 0x7f39f4004d50 2024-04-25T00:32:09.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.579+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/189509310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f4004d50 msgr2=0x7f39f40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.579+0000 7f3a0d8d5640 1 --2- 172.21.15.29:0/189509310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f4004d50 0x7f39f40051d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a00001860 tx=0x7f3a0001c500 comp rx=0 tx=0).stop 2024-04-25T00:32:09.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.579+0000 7f3a077fe640 1 -- 172.21.15.29:0/189509310 reap_dead start 2024-04-25T00:32:09.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.579+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/189509310 shutdown_connections 2024-04-25T00:32:09.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.579+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/189509310 >> 172.21.15.29:0/189509310 conn(0x7f3a08075030 msgr2=0x7f3a080b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:09.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.579+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/189509310 shutdown_connections 2024-04-25T00:32:09.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.579+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/189509310 wait complete. 2024-04-25T00:32:09.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.579+0000 7f3a0d8d5640 1 Processor -- start 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.579+0000 7f3a0d8d5640 1 -- start start 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a0d8d5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39f4004d50 0x7f3a081388a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a0d8d5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a080b4330 0x7f3a0815d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a0d8d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a080b47b0 0x7f3a0815d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a0d8d5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a080cec60 con 0x7f3a080b47b0 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a0d8d5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3a080ceae0 con 0x7f3a080b4330 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a0d8d5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a080cede0 con 0x7f39f4004d50 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a080b47b0 0x7f3a0815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a080b47b0 0x7f3a0815d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40386/0 (socket says 172.21.15.29:40386) 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a077fe640 1 -- 172.21.15.29:0/3619487510 learned_addr learned my addr 172.21.15.29:0/3619487510 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a06ffd640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39f4004d50 0x7f3a081388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a067fc640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a080b4330 0x7f3a0815d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a077fe640 1 -- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39f4004d50 msgr2=0x7f3a081388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a077fe640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39f4004d50 0x7f3a081388a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a077fe640 1 -- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a080b4330 msgr2=0x7f3a0815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a077fe640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a080b4330 0x7f3a0815d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a077fe640 1 -- 172.21.15.29:0/3619487510 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a08154940 con 0x7f3a080b47b0 2024-04-25T00:32:09.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a067fc640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a080b4330 0x7f3a0815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:09.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a077fe640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a080b47b0 0x7f3a0815d680 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3a0001c9e0 tx=0x7f39f4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:09.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f39fffff640 1 -- 172.21.15.29:0/3619487510 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00005480 con 0x7f3a080b47b0 2024-04-25T00:32:09.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f39fffff640 1 -- 172.21.15.29:0/3619487510 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a00001440 con 0x7f3a080b47b0 2024-04-25T00:32:09.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/3619487510 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a08154c20 con 0x7f3a080b47b0 2024-04-25T00:32:09.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f39fffff640 1 -- 172.21.15.29:0/3619487510 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00003820 con 0x7f3a080b47b0 2024-04-25T00:32:09.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/3619487510 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a081551e0 con 0x7f3a080b47b0 2024-04-25T00:32:09.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/3619487510 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a08069270 con 0x7f3a080b47b0 2024-04-25T00:32:09.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f39fffff640 1 -- 172.21.15.29:0/3619487510 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3a000039c0 con 0x7f3a080b47b0 2024-04-25T00:32:09.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f39fffff640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f39e8067890 0x7f39e8069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.583+0000 7f39fffff640 1 -- 172.21.15.29:0/3619487510 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3a0011fe70 con 0x7f3a080b47b0 2024-04-25T00:32:09.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.587+0000 7f3a06ffd640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f39e8067890 0x7f39e8069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.587+0000 7f3a06ffd640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f39e8067890 0x7f39e8069d50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3a080b59b0 tx=0x7f39f800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:09.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.591+0000 7f39fffff640 1 -- 172.21.15.29:0/3619487510 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3a00016210 con 0x7f3a080b47b0 2024-04-25T00:32:09.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.751+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/3619487510 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3a081593f0 con 0x7f3a080b47b0 2024-04-25T00:32:09.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.751+0000 7f39fffff640 1 -- 172.21.15.29:0/3619487510 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f3a000f0a40 con 0x7f3a080b47b0 2024-04-25T00:32:09.755 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:09.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.751+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f39e8067890 msgr2=0x7f39e8069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.751+0000 7f3a0d8d5640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f39e8067890 0x7f39e8069d50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3a080b59b0 tx=0x7f39f800b040 comp rx=0 tx=0).stop 2024-04-25T00:32:09.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.751+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/3619487510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a080b47b0 msgr2=0x7f3a0815d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.751+0000 7f3a0d8d5640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a080b47b0 0x7f3a0815d680 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3a0001c9e0 tx=0x7f39f4005840 comp rx=0 tx=0).stop 2024-04-25T00:32:09.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.751+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/3619487510 shutdown_connections 2024-04-25T00:32:09.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.751+0000 7f3a0d8d5640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a080b47b0 0x7f3a0815d680 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.751+0000 7f3a0d8d5640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f39e8067890 0x7f39e8069d50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.751+0000 7f3a0d8d5640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a080b4330 0x7f3a0815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.751+0000 7f3a0d8d5640 1 --2- 172.21.15.29:0/3619487510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39f4004d50 0x7f3a081388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.751+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/3619487510 >> 172.21.15.29:0/3619487510 conn(0x7f3a08075030 msgr2=0x7f3a08159910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:09.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.755+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/3619487510 shutdown_connections 2024-04-25T00:32:09.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.755+0000 7f3a0d8d5640 1 -- 172.21.15.29:0/3619487510 wait complete. 2024-04-25T00:32:09.773 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:09.863 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:32:09.866 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:32:09.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.883+0000 7f609a095640 1 Processor -- start 2024-04-25T00:32:09.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.883+0000 7f609a095640 1 -- start start 2024-04-25T00:32:09.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f609a095640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f609415a490 0x7f609406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f609a095640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 0x7f609415b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f609a095640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f609406e930 con 0x7f60940b3860 2024-04-25T00:32:09.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f609a095640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f609415ad60 con 0x7f609415a490 2024-04-25T00:32:09.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f609a095640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f609415b040 con 0x7f60940b4510 2024-04-25T00:32:09.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f6093fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 0x7f609415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f6093fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 0x7f609415b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:34482/0 (socket says 172.21.15.29:34482) 2024-04-25T00:32:09.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f6093fff640 1 -- 172.21.15.29:0/4073000404 learned_addr learned my addr 172.21.15.29:0/4073000404 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:09.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f60937fe640 1 --2- 172.21.15.29:0/4073000404 >> v2:172.21.15.157:3300/0 conn(0x7f609415a490 0x7f609406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f6093fff640 1 -- 172.21.15.29:0/4073000404 >> v2:172.21.15.157:3300/0 conn(0x7f609415a490 msgr2=0x7f609406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:09.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f6093fff640 1 --2- 172.21.15.29:0/4073000404 >> v2:172.21.15.157:3300/0 conn(0x7f609415a490 0x7f609406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f6093fff640 1 -- 172.21.15.29:0/4073000404 >> v1:172.21.15.157:6789/0 conn(0x7f60940b3860 legacy=0x7f60940b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f6093fff640 1 -- 172.21.15.29:0/4073000404 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609415b320 con 0x7f60940b4510 2024-04-25T00:32:09.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f6093fff640 1 --2- 172.21.15.29:0/4073000404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 0x7f609415b840 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f608c00d150 tx=0x7f608c01eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ffcc74d190ea75e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:09.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f60927fc640 1 -- 172.21.15.29:0/4073000404 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c0ad070 con 0x7f60940b4510 2024-04-25T00:32:09.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f60927fc640 1 -- 172.21.15.29:0/4073000404 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f609406e6e0 con 0x7f60940b4510 2024-04-25T00:32:09.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f60927fc640 1 -- 172.21.15.29:0/4073000404 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c0a7e30 con 0x7f60940b4510 2024-04-25T00:32:09.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f609a095640 1 -- 172.21.15.29:0/4073000404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 msgr2=0x7f609415b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.887+0000 7f609a095640 1 --2- 172.21.15.29:0/4073000404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 0x7f609415b840 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f608c00d150 tx=0x7f608c01eba0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 -- 172.21.15.29:0/4073000404 shutdown_connections 2024-04-25T00:32:09.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 --2- 172.21.15.29:0/4073000404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 0x7f609415b840 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 --2- 172.21.15.29:0/4073000404 >> v2:172.21.15.157:3300/0 conn(0x7f609415a490 0x7f609406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 -- 172.21.15.29:0/4073000404 >> 172.21.15.29:0/4073000404 conn(0x7f6094074c00 msgr2=0x7f609406b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:09.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 -- 172.21.15.29:0/4073000404 shutdown_connections 2024-04-25T00:32:09.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 -- 172.21.15.29:0/4073000404 wait complete. 2024-04-25T00:32:09.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 Processor -- start 2024-04-25T00:32:09.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 -- start start 2024-04-25T00:32:09.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60940b3860 0x7f60941450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 0x7f60940b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609415a490 0x7f60940b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f609415c110 con 0x7f60940b3860 2024-04-25T00:32:09.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f609415bf90 con 0x7f609415a490 2024-04-25T00:32:09.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f609415c290 con 0x7f60940b4510 2024-04-25T00:32:09.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f6092ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 0x7f60940b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f60937fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60940b3860 0x7f60941450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f6092ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 0x7f60940b05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:34486/0 (socket says 172.21.15.29:34486) 2024-04-25T00:32:09.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f6092ffd640 1 -- 172.21.15.29:0/2643210210 learned_addr learned my addr 172.21.15.29:0/2643210210 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:09.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f6093fff640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609415a490 0x7f60940b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f60937fe640 1 -- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 msgr2=0x7f60940b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f60937fe640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 0x7f60940b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f60937fe640 1 -- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609415a490 msgr2=0x7f60940b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:09.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f60937fe640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609415a490 0x7f60940b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:09.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f60937fe640 1 -- 172.21.15.29:0/2643210210 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60941455e0 con 0x7f60940b3860 2024-04-25T00:32:09.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f6092ffd640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 0x7f60940b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:09.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f6093fff640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609415a490 0x7f60940b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:09.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f60937fe640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60940b3860 0x7f60941450a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f608400e8a0 tx=0x7f608400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:09.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f607bfff640 1 -- 172.21.15.29:0/2643210210 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6084017070 con 0x7f60940b3860 2024-04-25T00:32:09.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f607bfff640 1 -- 172.21.15.29:0/2643210210 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f608400c940 con 0x7f60940b3860 2024-04-25T00:32:09.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f607bfff640 1 -- 172.21.15.29:0/2643210210 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608400cd10 con 0x7f60940b3860 2024-04-25T00:32:09.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 -- 172.21.15.29:0/2643210210 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60940ad940 con 0x7f60940b3860 2024-04-25T00:32:09.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.891+0000 7f609a095640 1 -- 172.21.15.29:0/2643210210 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60940adec0 con 0x7f60940b3860 2024-04-25T00:32:09.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.895+0000 7f609a095640 1 -- 172.21.15.29:0/2643210210 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6094069270 con 0x7f60940b3860 2024-04-25T00:32:09.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.895+0000 7f607bfff640 1 -- 172.21.15.29:0/2643210210 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f608401a070 con 0x7f60940b3860 2024-04-25T00:32:09.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.895+0000 7f607bfff640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f60800678c0 0x7f6080069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:09.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.895+0000 7f607bfff640 1 -- 172.21.15.29:0/2643210210 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f60840859c0 con 0x7f60940b3860 2024-04-25T00:32:09.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.895+0000 7f6092ffd640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f60800678c0 0x7f6080069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:09.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.895+0000 7f6092ffd640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f60800678c0 0x7f6080069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f609414a6b0 tx=0x7f6088009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:09.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:09.899+0000 7f607bfff640 1 -- 172.21.15.29:0/2643210210 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6084087e20 con 0x7f60940b3860 2024-04-25T00:32:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.059+0000 7f609a095640 1 -- 172.21.15.29:0/2643210210 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6094150830 con 0x7f60940b3860 2024-04-25T00:32:10.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.059+0000 7f607bfff640 1 -- 172.21.15.29:0/2643210210 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f6084055eb0 con 0x7f60940b3860 2024-04-25T00:32:10.063 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:10.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.063+0000 7f609a095640 1 -- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f60800678c0 msgr2=0x7f6080069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.063+0000 7f609a095640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f60800678c0 0x7f6080069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f609414a6b0 tx=0x7f6088009040 comp rx=0 tx=0).stop 2024-04-25T00:32:10.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.063+0000 7f609a095640 1 -- 172.21.15.29:0/2643210210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60940b3860 msgr2=0x7f60941450a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.063+0000 7f609a095640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60940b3860 0x7f60941450a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f608400e8a0 tx=0x7f608400ed60 comp rx=0 tx=0).stop 2024-04-25T00:32:10.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.063+0000 7f609a095640 1 -- 172.21.15.29:0/2643210210 shutdown_connections 2024-04-25T00:32:10.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.063+0000 7f609a095640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f60800678c0 0x7f6080069d80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.063+0000 7f609a095640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609415a490 0x7f60940b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.063+0000 7f609a095640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60940b4510 0x7f60940b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.063+0000 7f609a095640 1 --2- 172.21.15.29:0/2643210210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60940b3860 0x7f60941450a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.063+0000 7f609a095640 1 -- 172.21.15.29:0/2643210210 >> 172.21.15.29:0/2643210210 conn(0x7f6094074c00 msgr2=0x7f609415d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:10.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.063+0000 7f609a095640 1 -- 172.21.15.29:0/2643210210 shutdown_connections 2024-04-25T00:32:10.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.063+0000 7f609a095640 1 -- 172.21.15.29:0/2643210210 wait complete. 2024-04-25T00:32:10.083 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:32:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fc000d6e640 1 Processor -- start 2024-04-25T00:32:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fc000d6e640 1 -- start start 2024-04-25T00:32:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fc000d6e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbffc0b3140 0x7fbffc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fc000d6e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbffc0b4b10 0x7fbffc0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fc000d6e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbffc0b3a60 0x7fbffc15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fc000d6e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbffc0b41e0 con 0x7fbffc0b4b10 2024-04-25T00:32:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fc000d6e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbffc06d960 con 0x7fbffc0b3140 2024-04-25T00:32:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fc000d6e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbffc06dc40 con 0x7fbffc0b3a60 2024-04-25T00:32:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fbff1d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbffc0b4b10 0x7fbffc0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fbffad9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbffc0b3a60 0x7fbffc15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fbff1d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbffc0b4b10 0x7fbffc0b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40402/0 (socket says 172.21.15.29:40402) 2024-04-25T00:32:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fbff1d9b640 1 -- 172.21.15.29:0/1986131470 learned_addr learned my addr 172.21.15.29:0/1986131470 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fbffa59c640 1 --2- 172.21.15.29:0/1986131470 >> v2:172.21.15.157:3300/0 conn(0x7fbffc0b3140 0x7fbffc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fbff1d9b640 1 -- 172.21.15.29:0/1986131470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbffc0b3a60 msgr2=0x7fbffc15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:10.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fbff1d9b640 1 --2- 172.21.15.29:0/1986131470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbffc0b3a60 0x7fbffc15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fbff1d9b640 1 -- 172.21.15.29:0/1986131470 >> v2:172.21.15.157:3300/0 conn(0x7fbffc0b3140 msgr2=0x7fbffc06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:10.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fbff1d9b640 1 --2- 172.21.15.29:0/1986131470 >> v2:172.21.15.157:3300/0 conn(0x7fbffc0b3140 0x7fbffc06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.211+0000 7fbff1d9b640 1 -- 172.21.15.29:0/1986131470 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc06df20 con 0x7fbffc0b4b10 2024-04-25T00:32:10.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.215+0000 7fbff1d9b640 1 --2- 172.21.15.29:0/1986131470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbffc0b4b10 0x7fbffc0b3520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009870 tx=0x7fbfe401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a29237d9d0e18e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.215+0000 7fbff9d9b640 1 -- 172.21.15.29:0/1986131470 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40b0070 con 0x7fbffc0b4b10 2024-04-25T00:32:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.215+0000 7fbff9d9b640 1 -- 172.21.15.29:0/1986131470 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe40ab600 con 0x7fbffc0b4b10 2024-04-25T00:32:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.215+0000 7fbff9d9b640 1 -- 172.21.15.29:0/1986131470 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40ab9d0 con 0x7fbffc0b4b10 2024-04-25T00:32:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.215+0000 7fc000d6e640 1 -- 172.21.15.29:0/1986131470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbffc0b4b10 msgr2=0x7fbffc0b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.215+0000 7fc000d6e640 1 --2- 172.21.15.29:0/1986131470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbffc0b4b10 0x7fbffc0b3520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009870 tx=0x7fbfe401c520 comp rx=0 tx=0).stop 2024-04-25T00:32:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.215+0000 7fc000d6e640 1 -- 172.21.15.29:0/1986131470 shutdown_connections 2024-04-25T00:32:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.215+0000 7fc000d6e640 1 --2- 172.21.15.29:0/1986131470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbffc0b4b10 0x7fbffc0b3520 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.215+0000 7fc000d6e640 1 --2- 172.21.15.29:0/1986131470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbffc0b3a60 0x7fbffc15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.215+0000 7fc000d6e640 1 --2- 172.21.15.29:0/1986131470 >> v2:172.21.15.157:3300/0 conn(0x7fbffc0b3140 0x7fbffc06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.215+0000 7fc000d6e640 1 -- 172.21.15.29:0/1986131470 >> 172.21.15.29:0/1986131470 conn(0x7fbffc0b1870 msgr2=0x7fbffc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:10.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.215+0000 7fc000d6e640 1 -- 172.21.15.29:0/1986131470 shutdown_connections 2024-04-25T00:32:10.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.215+0000 7fc000d6e640 1 -- 172.21.15.29:0/1986131470 wait complete. 2024-04-25T00:32:10.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fc000d6e640 1 Processor -- start 2024-04-25T00:32:10.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fc000d6e640 1 -- start start 2024-04-25T00:32:10.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fc000d6e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbffc0b3140 0x7fbffc158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fc000d6e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbffc0b3a60 0x7fbffc1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fc000d6e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbffc0b4b10 0x7fbffc138810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fc000d6e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0ce3c0 con 0x7fbffc0b4b10 2024-04-25T00:32:10.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fc000d6e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0ce240 con 0x7fbffc0b3140 2024-04-25T00:32:10.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fc000d6e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbffc0ce540 con 0x7fbffc0b3a60 2024-04-25T00:32:10.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbffad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbffc0b4b10 0x7fbffc138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbffad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbffc0b4b10 0x7fbffc138810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40414/0 (socket says 172.21.15.29:40414) 2024-04-25T00:32:10.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbffad9d640 1 -- 172.21.15.29:0/3340281211 learned_addr learned my addr 172.21.15.29:0/3340281211 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbff1d9b640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbffc0b3a60 0x7fbffc1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbffa59c640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbffc0b3140 0x7fbffc158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbffad9d640 1 -- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbffc0b3a60 msgr2=0x7fbffc1569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbffad9d640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbffc0b3a60 0x7fbffc1569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbffad9d640 1 -- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbffc0b3140 msgr2=0x7fbffc158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbffad9d640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbffc0b3140 0x7fbffc158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbffad9d640 1 -- 172.21.15.29:0/3340281211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc13a1d0 con 0x7fbffc0b4b10 2024-04-25T00:32:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbffad9d640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbffc0b4b10 0x7fbffc138810 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbff400efc0 tx=0x7fbff400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbff37fe640 1 -- 172.21.15.29:0/3340281211 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4018020 con 0x7fbffc0b4b10 2024-04-25T00:32:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbff37fe640 1 -- 172.21.15.29:0/3340281211 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff4014480 con 0x7fbffc0b4b10 2024-04-25T00:32:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fc000d6e640 1 -- 172.21.15.29:0/3340281211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc13a4b0 con 0x7fbffc0b4b10 2024-04-25T00:32:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fbff37fe640 1 -- 172.21.15.29:0/3340281211 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4014870 con 0x7fbffc0b4b10 2024-04-25T00:32:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fc000d6e640 1 -- 172.21.15.29:0/3340281211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc13aa70 con 0x7fbffc0b4b10 2024-04-25T00:32:10.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.219+0000 7fc000d6e640 1 -- 172.21.15.29:0/3340281211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbffc069270 con 0x7fbffc0b4b10 2024-04-25T00:32:10.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.223+0000 7fbff37fe640 1 -- 172.21.15.29:0/3340281211 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbff401b020 con 0x7fbffc0b4b10 2024-04-25T00:32:10.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.223+0000 7fbff37fe640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfdc0678c0 0x7fbfdc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.223+0000 7fbff37fe640 1 -- 172.21.15.29:0/3340281211 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbff40866d0 con 0x7fbffc0b4b10 2024-04-25T00:32:10.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.223+0000 7fbffa59c640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfdc0678c0 0x7fbfdc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.223+0000 7fbffa59c640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfdc0678c0 0x7fbfdc069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009a70 tx=0x7fbfec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:10.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.227+0000 7fbff37fe640 1 -- 172.21.15.29:0/3340281211 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbff4022070 con 0x7fbffc0b4b10 2024-04-25T00:32:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.403+0000 7fc000d6e640 1 -- 172.21.15.29:0/3340281211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbffc0cdd40 con 0x7fbffc0b4b10 2024-04-25T00:32:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.403+0000 7fbff37fe640 1 -- 172.21.15.29:0/3340281211 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fbff4010040 con 0x7fbffc0b4b10 2024-04-25T00:32:10.408 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:32:10.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.407+0000 7fc000d6e640 1 -- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfdc0678c0 msgr2=0x7fbfdc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.407+0000 7fc000d6e640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfdc0678c0 0x7fbfdc069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009a70 tx=0x7fbfec008040 comp rx=0 tx=0).stop 2024-04-25T00:32:10.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.407+0000 7fc000d6e640 1 -- 172.21.15.29:0/3340281211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbffc0b4b10 msgr2=0x7fbffc138810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.407+0000 7fc000d6e640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbffc0b4b10 0x7fbffc138810 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbff400efc0 tx=0x7fbff400c490 comp rx=0 tx=0).stop 2024-04-25T00:32:10.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.407+0000 7fc000d6e640 1 -- 172.21.15.29:0/3340281211 shutdown_connections 2024-04-25T00:32:10.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.407+0000 7fc000d6e640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbffc0b4b10 0x7fbffc138810 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.407+0000 7fc000d6e640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbffc0b3a60 0x7fbffc1569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.407+0000 7fc000d6e640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfdc0678c0 0x7fbfdc069d80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.407+0000 7fc000d6e640 1 --2- 172.21.15.29:0/3340281211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbffc0b3140 0x7fbffc158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.407+0000 7fc000d6e640 1 -- 172.21.15.29:0/3340281211 >> 172.21.15.29:0/3340281211 conn(0x7fbffc0b1870 msgr2=0x7fbffc139250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:10.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.407+0000 7fc000d6e640 1 -- 172.21.15.29:0/3340281211 shutdown_connections 2024-04-25T00:32:10.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.407+0000 7fc000d6e640 1 -- 172.21.15.29:0/3340281211 wait complete. 2024-04-25T00:32:10.425 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-04-25T00:32:10.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:10.426 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:32:10.453 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:32:10.453 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:32:10.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:10.453 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:32:10.510 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:32:10.511 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:10.517 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:32:10.517 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:10.565 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:10.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.687+0000 7f437f439640 1 Processor -- start 2024-04-25T00:32:10.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.687+0000 7f437f439640 1 -- start start 2024-04-25T00:32:10.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437f439640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f437815e8d0 0x7f43780cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437f439640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43780b3720 con 0x7f43780b4780 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437f439640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43780b39c0 con 0x7f437815f3c0 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437f439640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f43780b42f0 con 0x7f437815e8d0 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437c9d4640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f43780b4780 0x7f437815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35182/0 (socket says 172.21.15.29:35182) 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437c9d4640 1 -- 172.21.15.29:0/3814047618 learned_addr learned my addr 172.21.15.29:0/3814047618 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437d1d5640 1 --2- 172.21.15.29:0/3814047618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f437815e8d0 0x7f43780cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 263956900 0 0) 0x7f43780b3720 con 0x7f43780b4780 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4364003530 con 0x7f43780b4780 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3322256141 0 0) 0x7f4364003530 con 0x7f43780b4780 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f437815e8d0 msgr2=0x7f43780cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 --2- 172.21.15.29:0/3814047618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f437815e8d0 0x7f43780cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 >> v1:172.21.15.157:6789/0 conn(0x7f437815f3c0 legacy=0x7f437815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43780b3ca0 con 0x7f43780b4780 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4368001680 con 0x7f43780b4780 2024-04-25T00:32:10.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437d1d5640 1 --2- 172.21.15.29:0/3814047618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f437815e8d0 0x7f43780cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 >> v1:172.21.15.29:6789/0 conn(0x7f43780b4780 legacy=0x7f437815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 --2- 172.21.15.29:0/3814047618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43640039f0 0x7f4364003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 --2- 172.21.15.29:0/3814047618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f437815e8d0 0x7f4364004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 --2- 172.21.15.29:0/3814047618 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4364004d50 0x7f43640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f43780b42f0 con 0x7f4364004d50 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f436400bae0 con 0x7f43640039f0 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f436400bce0 con 0x7f437815e8d0 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437d9d6640 1 --2- 172.21.15.29:0/3814047618 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4364004d50 0x7f43640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437d1d5640 1 --2- 172.21.15.29:0/3814047618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43640039f0 0x7f4364003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437c9d4640 1 --2- 172.21.15.29:0/3814047618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f437815e8d0 0x7f4364004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437d9d6640 1 -- 172.21.15.29:0/3814047618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f437815e8d0 msgr2=0x7f4364004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437d9d6640 1 --2- 172.21.15.29:0/3814047618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f437815e8d0 0x7f4364004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437d9d6640 1 -- 172.21.15.29:0/3814047618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43640039f0 msgr2=0x7f4364003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:10.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437d9d6640 1 --2- 172.21.15.29:0/3814047618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43640039f0 0x7f4364003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.691+0000 7f437d9d6640 1 -- 172.21.15.29:0/3814047618 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4364003530 con 0x7f4364004d50 2024-04-25T00:32:10.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.695+0000 7f437d9d6640 1 --2- 172.21.15.29:0/3814047618 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4364004d50 0x7f43640051d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4370001e40 tx=0x7f437001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b2289cc84a7560 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:10.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.695+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370001a90 con 0x7f4364004d50 2024-04-25T00:32:10.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.695+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4370003b80 con 0x7f4364004d50 2024-04-25T00:32:10.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.695+0000 7f436ffff640 1 -- 172.21.15.29:0/3814047618 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370005080 con 0x7f4364004d50 2024-04-25T00:32:10.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.695+0000 7f437f439640 1 -- 172.21.15.29:0/3814047618 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4364004d50 msgr2=0x7f43640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.695+0000 7f437f439640 1 --2- 172.21.15.29:0/3814047618 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4364004d50 0x7f43640051d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4370001e40 tx=0x7f437001c760 comp rx=0 tx=0).stop 2024-04-25T00:32:10.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.695+0000 7f437d9d6640 1 -- 172.21.15.29:0/3814047618 reap_dead start 2024-04-25T00:32:10.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.695+0000 7f437f439640 1 -- 172.21.15.29:0/3814047618 shutdown_connections 2024-04-25T00:32:10.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.695+0000 7f437f439640 1 -- 172.21.15.29:0/3814047618 >> 172.21.15.29:0/3814047618 conn(0x7f43780b1870 msgr2=0x7f43780b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:10.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.695+0000 7f437f439640 1 -- 172.21.15.29:0/3814047618 shutdown_connections 2024-04-25T00:32:10.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.695+0000 7f437f439640 1 -- 172.21.15.29:0/3814047618 wait complete. 2024-04-25T00:32:10.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437f439640 1 Processor -- start 2024-04-25T00:32:10.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437f439640 1 -- start start 2024-04-25T00:32:10.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437f439640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4378143a80 0x7f43781f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437f439640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43780b0960 0x7f43780b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437f439640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43781f4090 0x7f43781f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437f439640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4378144a20 con 0x7f43781f4090 2024-04-25T00:32:10.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437f439640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f437806cf10 con 0x7f4378143a80 2024-04-25T00:32:10.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437f439640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4378138e90 con 0x7f43780b0960 2024-04-25T00:32:10.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437d9d6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43781f4090 0x7f43781f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437d9d6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43781f4090 0x7f43781f4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40446/0 (socket says 172.21.15.29:40446) 2024-04-25T00:32:10.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437d9d6640 1 -- 172.21.15.29:0/4245256834 learned_addr learned my addr 172.21.15.29:0/4245256834 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:10.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437c9d4640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43780b0960 0x7f43780b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437d1d5640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4378143a80 0x7f43781f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437d9d6640 1 -- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43780b0960 msgr2=0x7f43780b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437d9d6640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43780b0960 0x7f43780b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437d9d6640 1 -- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4378143a80 msgr2=0x7f43781f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437d9d6640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4378143a80 0x7f43781f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437d9d6640 1 -- 172.21.15.29:0/4245256834 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43781f4dc0 con 0x7f43781f4090 2024-04-25T00:32:10.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437d9d6640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43781f4090 0x7f43781f4510 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f437001ccd0 tx=0x7f4364005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:10.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f436dffb640 1 -- 172.21.15.29:0/4245256834 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370001a90 con 0x7f43781f4090 2024-04-25T00:32:10.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f436dffb640 1 -- 172.21.15.29:0/4245256834 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4370006b60 con 0x7f43781f4090 2024-04-25T00:32:10.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437f439640 1 -- 172.21.15.29:0/4245256834 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4378140ec0 con 0x7f43781f4090 2024-04-25T00:32:10.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f436dffb640 1 -- 172.21.15.29:0/4245256834 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370007dc0 con 0x7f43781f4090 2024-04-25T00:32:10.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437f439640 1 -- 172.21.15.29:0/4245256834 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4378141480 con 0x7f43781f4090 2024-04-25T00:32:10.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.699+0000 7f437f439640 1 -- 172.21.15.29:0/4245256834 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4378069270 con 0x7f43781f4090 2024-04-25T00:32:10.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.703+0000 7f436dffb640 1 -- 172.21.15.29:0/4245256834 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f43700ba060 con 0x7f43781f4090 2024-04-25T00:32:10.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.703+0000 7f436dffb640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f435c0678c0 0x7f435c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:10.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.703+0000 7f436dffb640 1 -- 172.21.15.29:0/4245256834 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f437011eb50 con 0x7f43781f4090 2024-04-25T00:32:10.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.703+0000 7f437d1d5640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f435c0678c0 0x7f435c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:10.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.703+0000 7f437d1d5640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f435c0678c0 0x7f435c069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4374006f90 tx=0x7f4374008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:10.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.707+0000 7f436dffb640 1 -- 172.21.15.29:0/4245256834 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f43700ac020 con 0x7f43781f4090 2024-04-25T00:32:10.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f437f439640 1 -- 172.21.15.29:0/4245256834 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f437806e7b0 con 0x7f43781f4090 2024-04-25T00:32:10.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f436dffb640 1 -- 172.21.15.29:0/4245256834 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f4370016470 con 0x7f43781f4090 2024-04-25T00:32:10.867 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:10.867 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:10.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f437f439640 1 -- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f435c0678c0 msgr2=0x7f435c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f437f439640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f435c0678c0 0x7f435c069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4374006f90 tx=0x7f4374008040 comp rx=0 tx=0).stop 2024-04-25T00:32:10.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f437f439640 1 -- 172.21.15.29:0/4245256834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43781f4090 msgr2=0x7f43781f4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:10.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f437f439640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43781f4090 0x7f43781f4510 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f437001ccd0 tx=0x7f4364005840 comp rx=0 tx=0).stop 2024-04-25T00:32:10.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f437f439640 1 -- 172.21.15.29:0/4245256834 shutdown_connections 2024-04-25T00:32:10.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f437f439640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43781f4090 0x7f43781f4510 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f437f439640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43780b0960 0x7f43780b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f437f439640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f435c0678c0 0x7f435c069d80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f437f439640 1 --2- 172.21.15.29:0/4245256834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4378143a80 0x7f43781f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:10.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f437f439640 1 -- 172.21.15.29:0/4245256834 >> 172.21.15.29:0/4245256834 conn(0x7f43780b1870 msgr2=0x7f43780aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:10.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f437f439640 1 -- 172.21.15.29:0/4245256834 shutdown_connections 2024-04-25T00:32:10.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.863+0000 7f437f439640 1 -- 172.21.15.29:0/4245256834 wait complete. 2024-04-25T00:32:10.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:10.885 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:11.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.995+0000 7fbf56d78640 1 Processor -- start 2024-04-25T00:32:11.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf56d78640 1 -- start start 2024-04-25T00:32:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf56d78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5015e3c0 0x7fbf500d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf56d78640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf5015e7a0 con 0x7fbf5006d320 2024-04-25T00:32:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf56d78640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf500d10f0 con 0x7fbf5006c530 2024-04-25T00:32:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf56d78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbf500b4bc0 con 0x7fbf5015e3c0 2024-04-25T00:32:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf55315640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbf5006d320 0x7fbf500cd040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35184/0 (socket says 172.21.15.29:35184) 2024-04-25T00:32:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf55315640 1 -- 172.21.15.29:0/1440469877 learned_addr learned my addr 172.21.15.29:0/1440469877 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf54b14640 1 --2- 172.21.15.29:0/1440469877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5015e3c0 0x7fbf500d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf4f7fe640 1 -- 172.21.15.29:0/1440469877 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2844266694 0 0) 0x7fbf5015e7a0 con 0x7fbf5006d320 2024-04-25T00:32:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf4f7fe640 1 -- 172.21.15.29:0/1440469877 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf40003530 con 0x7fbf5006d320 2024-04-25T00:32:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf4f7fe640 1 -- 172.21.15.29:0/1440469877 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 224053141 0 0) 0x7fbf40003530 con 0x7fbf5006d320 2024-04-25T00:32:11.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf54b14640 1 -- 172.21.15.29:0/1440469877 >> v1:172.21.15.157:6789/0 conn(0x7fbf5006c530 legacy=0x7fbf5006cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf54b14640 1 -- 172.21.15.29:0/1440469877 >> v1:172.21.15.29:6789/0 conn(0x7fbf5006d320 legacy=0x7fbf500cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:10.999+0000 7fbf54b14640 1 -- 172.21.15.29:0/1440469877 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf500cc560 con 0x7fbf5015e3c0 2024-04-25T00:32:11.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf54b14640 1 --2- 172.21.15.29:0/1440469877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5015e3c0 0x7fbf500d0a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbf440097f0 tx=0x7fbf4401c2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35af9eb2e2fa331c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:11.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf4f7fe640 1 -- 172.21.15.29:0/1440469877 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf440b0070 con 0x7fbf5015e3c0 2024-04-25T00:32:11.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf4f7fe640 1 -- 172.21.15.29:0/1440469877 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf440ab400 con 0x7fbf5015e3c0 2024-04-25T00:32:11.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf4f7fe640 1 -- 172.21.15.29:0/1440469877 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48001a00 con 0x7fbf5015e3c0 2024-04-25T00:32:11.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 -- 172.21.15.29:0/1440469877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5015e3c0 msgr2=0x7fbf500d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 --2- 172.21.15.29:0/1440469877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5015e3c0 0x7fbf500d0a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbf440097f0 tx=0x7fbf4401c2b0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 -- 172.21.15.29:0/1440469877 shutdown_connections 2024-04-25T00:32:11.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 --2- 172.21.15.29:0/1440469877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5015e3c0 0x7fbf500d0a80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 -- 172.21.15.29:0/1440469877 >> 172.21.15.29:0/1440469877 conn(0x7fbf500d1d70 msgr2=0x7fbf500d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:11.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 -- 172.21.15.29:0/1440469877 shutdown_connections 2024-04-25T00:32:11.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 -- 172.21.15.29:0/1440469877 wait complete. 2024-04-25T00:32:11.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 Processor -- start 2024-04-25T00:32:11.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 -- start start 2024-04-25T00:32:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5006c530 0x7fbf500c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5006d320 0x7fbf5015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5015e3c0 0x7fbf50160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5006c970 con 0x7fbf5006c530 2024-04-25T00:32:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf500ccb20 con 0x7fbf5006d320 2024-04-25T00:32:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf56d78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbf500cd750 con 0x7fbf5015e3c0 2024-04-25T00:32:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf54b14640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5006c530 0x7fbf500c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf54b14640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5006c530 0x7fbf500c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40456/0 (socket says 172.21.15.29:40456) 2024-04-25T00:32:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf54b14640 1 -- 172.21.15.29:0/1503143413 learned_addr learned my addr 172.21.15.29:0/1503143413 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.003+0000 7fbf55315640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5015e3c0 0x7fbf50160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf4ffff640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5006d320 0x7fbf5015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf54b14640 1 -- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5015e3c0 msgr2=0x7fbf50160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf54b14640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5015e3c0 0x7fbf50160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf54b14640 1 -- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5006d320 msgr2=0x7fbf5015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf54b14640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5006d320 0x7fbf5015ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf54b14640 1 -- 172.21.15.29:0/1503143413 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5015d360 con 0x7fbf5006c530 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf55315640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5015e3c0 0x7fbf50160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf54b14640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5006c530 0x7fbf500c9ac0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbf440ae040 tx=0x7fbf44019050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf4d7fa640 1 -- 172.21.15.29:0/1503143413 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf440bb020 con 0x7fbf5006c530 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf4d7fa640 1 -- 172.21.15.29:0/1503143413 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf440044c0 con 0x7fbf5006c530 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf56d78640 1 -- 172.21.15.29:0/1503143413 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf501617c0 con 0x7fbf5006c530 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf4d7fa640 1 -- 172.21.15.29:0/1503143413 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf440b0040 con 0x7fbf5006c530 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf56d78640 1 -- 172.21.15.29:0/1503143413 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf500c92c0 con 0x7fbf5006c530 2024-04-25T00:32:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf56d78640 1 -- 172.21.15.29:0/1503143413 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf50069270 con 0x7fbf5006c530 2024-04-25T00:32:11.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf4d7fa640 1 -- 172.21.15.29:0/1503143413 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbf440be020 con 0x7fbf5006c530 2024-04-25T00:32:11.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf4d7fa640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbf340678c0 0x7fbf34069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf4d7fa640 1 -- 172.21.15.29:0/1503143413 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbf44120480 con 0x7fbf5006c530 2024-04-25T00:32:11.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf4ffff640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbf340678c0 0x7fbf34069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.007+0000 7fbf4ffff640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbf340678c0 0x7fbf34069d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c001ec0 tx=0x7fbf3c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:11.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.011+0000 7fbf4d7fa640 1 -- 172.21.15.29:0/1503143413 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbf440c5030 con 0x7fbf5006c530 2024-04-25T00:32:11.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.163+0000 7fbf56d78640 1 -- 172.21.15.29:0/1503143413 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf500d0860 con 0x7fbf5006c530 2024-04-25T00:32:11.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.167+0000 7fbf4d7fa640 1 -- 172.21.15.29:0/1503143413 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fbf44015ff0 con 0x7fbf5006c530 2024-04-25T00:32:11.170 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:11.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.167+0000 7fbf56d78640 1 -- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbf340678c0 msgr2=0x7fbf34069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.167+0000 7fbf56d78640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbf340678c0 0x7fbf34069d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c001ec0 tx=0x7fbf3c00b040 comp rx=0 tx=0).stop 2024-04-25T00:32:11.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.167+0000 7fbf56d78640 1 -- 172.21.15.29:0/1503143413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5006c530 msgr2=0x7fbf500c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.167+0000 7fbf56d78640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5006c530 0x7fbf500c9ac0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbf440ae040 tx=0x7fbf44019050 comp rx=0 tx=0).stop 2024-04-25T00:32:11.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.167+0000 7fbf56d78640 1 -- 172.21.15.29:0/1503143413 shutdown_connections 2024-04-25T00:32:11.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.167+0000 7fbf56d78640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5015e3c0 0x7fbf50160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.167+0000 7fbf56d78640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbf340678c0 0x7fbf34069d80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.167+0000 7fbf56d78640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf5006d320 0x7fbf5015ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.167+0000 7fbf56d78640 1 --2- 172.21.15.29:0/1503143413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf5006c530 0x7fbf500c9ac0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.167+0000 7fbf56d78640 1 -- 172.21.15.29:0/1503143413 >> 172.21.15.29:0/1503143413 conn(0x7fbf500d1d70 msgr2=0x7fbf500b76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:11.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.167+0000 7fbf56d78640 1 -- 172.21.15.29:0/1503143413 shutdown_connections 2024-04-25T00:32:11.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.167+0000 7fbf56d78640 1 -- 172.21.15.29:0/1503143413 wait complete. 2024-04-25T00:32:11.187 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801f0aa640 1 Processor -- start 2024-04-25T00:32:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801f0aa640 1 -- start start 2024-04-25T00:32:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801f0aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f801815c220 0x7f80180cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801f0aa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80180cfa10 0x7f80180cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801f0aa640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f80180d04a0 0x7f80180ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801f0aa640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f80180d0900 con 0x7f80180d04a0 2024-04-25T00:32:11.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801f0aa640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f801815c600 con 0x7f801815c220 2024-04-25T00:32:11.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801f0aa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f80180b1a30 con 0x7f80180cfa10 2024-04-25T00:32:11.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801d647640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f80180d04a0 0x7f80180ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801d647640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f80180d04a0 0x7f80180ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40472/0 (socket says 172.21.15.29:40472) 2024-04-25T00:32:11.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801d647640 1 -- 172.21.15.29:0/1822879861 learned_addr learned my addr 172.21.15.29:0/1822879861 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:11.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801ce46640 1 --2- 172.21.15.29:0/1822879861 >> v2:172.21.15.157:3300/0 conn(0x7f801815c220 0x7f80180cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f8017fff640 1 --2- 172.21.15.29:0/1822879861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80180cfa10 0x7f80180cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801d647640 1 -- 172.21.15.29:0/1822879861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80180cfa10 msgr2=0x7f80180cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:11.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801d647640 1 --2- 172.21.15.29:0/1822879861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80180cfa10 0x7f80180cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801d647640 1 -- 172.21.15.29:0/1822879861 >> v2:172.21.15.157:3300/0 conn(0x7f801815c220 msgr2=0x7f80180cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:11.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801d647640 1 --2- 172.21.15.29:0/1822879861 >> v2:172.21.15.157:3300/0 conn(0x7f801815c220 0x7f80180cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801d647640 1 -- 172.21.15.29:0/1822879861 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80180cdbd0 con 0x7f80180d04a0 2024-04-25T00:32:11.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f801ce46640 1 --2- 172.21.15.29:0/1822879861 >> v2:172.21.15.157:3300/0 conn(0x7f801815c220 0x7f80180cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:11.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.311+0000 7f8017fff640 1 --2- 172.21.15.29:0/1822879861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80180cfa10 0x7f80180cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:11.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801d647640 1 --2- 172.21.15.29:0/1822879861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80180d04a0 0x7f80180ce6c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f80180cf860 tx=0x7f801001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2a6929b6144c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:11.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f80177fe640 1 -- 172.21.15.29:0/1822879861 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80100ac070 con 0x7f80180d04a0 2024-04-25T00:32:11.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f80177fe640 1 -- 172.21.15.29:0/1822879861 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80100a8420 con 0x7f80180d04a0 2024-04-25T00:32:11.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f80177fe640 1 -- 172.21.15.29:0/1822879861 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80100a87f0 con 0x7f80180d04a0 2024-04-25T00:32:11.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 -- 172.21.15.29:0/1822879861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80180d04a0 msgr2=0x7f80180ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 --2- 172.21.15.29:0/1822879861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80180d04a0 0x7f80180ce6c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f80180cf860 tx=0x7f801001c730 comp rx=0 tx=0).stop 2024-04-25T00:32:11.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 -- 172.21.15.29:0/1822879861 shutdown_connections 2024-04-25T00:32:11.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 --2- 172.21.15.29:0/1822879861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80180d04a0 0x7f80180ce6c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 --2- 172.21.15.29:0/1822879861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80180cfa10 0x7f80180cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 --2- 172.21.15.29:0/1822879861 >> v2:172.21.15.157:3300/0 conn(0x7f801815c220 0x7f80180cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 -- 172.21.15.29:0/1822879861 >> 172.21.15.29:0/1822879861 conn(0x7f80180d1d60 msgr2=0x7f80180d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:11.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 -- 172.21.15.29:0/1822879861 shutdown_connections 2024-04-25T00:32:11.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 -- 172.21.15.29:0/1822879861 wait complete. 2024-04-25T00:32:11.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 Processor -- start 2024-04-25T00:32:11.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 -- start start 2024-04-25T00:32:11.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80180cfa10 0x7f80180c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80180d04a0 0x7f8018155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801815c220 0x7f801815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f80180b2020 con 0x7f801815c220 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f80180b1ea0 con 0x7f80180d04a0 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801f0aa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f80180b21a0 con 0x7f80180cfa10 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801d647640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801815c220 0x7f801815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801d647640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801815c220 0x7f801815ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40486/0 (socket says 172.21.15.29:40486) 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801d647640 1 -- 172.21.15.29:0/2443168882 learned_addr learned my addr 172.21.15.29:0/2443168882 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f8017fff640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80180d04a0 0x7f8018155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.315+0000 7f801ce46640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80180cfa10 0x7f80180c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f801d647640 1 -- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80180cfa10 msgr2=0x7f80180c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f801d647640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80180cfa10 0x7f80180c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f801d647640 1 -- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80180d04a0 msgr2=0x7f8018155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f801d647640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80180d04a0 0x7f8018155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f801d647640 1 -- 172.21.15.29:0/2443168882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80180c5b80 con 0x7f801815c220 2024-04-25T00:32:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f801d647640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801815c220 0x7f801815ef80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f801001ade0 tx=0x7f80100194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f80157fa640 1 -- 172.21.15.29:0/2443168882 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80100ac040 con 0x7f801815c220 2024-04-25T00:32:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f80157fa640 1 -- 172.21.15.29:0/2443168882 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80100bd670 con 0x7f801815c220 2024-04-25T00:32:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f801f0aa640 1 -- 172.21.15.29:0/2443168882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80180c5e60 con 0x7f801815c220 2024-04-25T00:32:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f80157fa640 1 -- 172.21.15.29:0/2443168882 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80100bda60 con 0x7f801815c220 2024-04-25T00:32:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f801f0aa640 1 -- 172.21.15.29:0/2443168882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80180c6420 con 0x7f801815c220 2024-04-25T00:32:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f801f0aa640 1 -- 172.21.15.29:0/2443168882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8018069270 con 0x7f801815c220 2024-04-25T00:32:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f80157fa640 1 -- 172.21.15.29:0/2443168882 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f80100af070 con 0x7f801815c220 2024-04-25T00:32:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f80157fa640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f80040678f0 0x7f8004069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f80157fa640 1 -- 172.21.15.29:0/2443168882 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8010120840 con 0x7f801815c220 2024-04-25T00:32:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f801ce46640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f80040678f0 0x7f8004069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.319+0000 7f801ce46640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f80040678f0 0x7f8004069db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f800c006fd0 tx=0x7f800c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.323+0000 7f80157fa640 1 -- 172.21.15.29:0/2443168882 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f80100c2060 con 0x7f801815c220 2024-04-25T00:32:11.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.487+0000 7f801f0aa640 1 -- 172.21.15.29:0/2443168882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f80181574a0 con 0x7f801815c220 2024-04-25T00:32:11.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.491+0000 7f80157fa640 1 -- 172.21.15.29:0/2443168882 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f80100165e0 con 0x7f801815c220 2024-04-25T00:32:11.493 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:11.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.491+0000 7f801f0aa640 1 -- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f80040678f0 msgr2=0x7f8004069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.491+0000 7f801f0aa640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f80040678f0 0x7f8004069db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f800c006fd0 tx=0x7f800c008040 comp rx=0 tx=0).stop 2024-04-25T00:32:11.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.491+0000 7f801f0aa640 1 -- 172.21.15.29:0/2443168882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801815c220 msgr2=0x7f801815ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.491+0000 7f801f0aa640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801815c220 0x7f801815ef80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f801001ade0 tx=0x7f80100194d0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.491+0000 7f801f0aa640 1 -- 172.21.15.29:0/2443168882 shutdown_connections 2024-04-25T00:32:11.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.491+0000 7f801f0aa640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801815c220 0x7f801815ef80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.491+0000 7f801f0aa640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f80040678f0 0x7f8004069db0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.491+0000 7f801f0aa640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80180d04a0 0x7f8018155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.491+0000 7f801f0aa640 1 --2- 172.21.15.29:0/2443168882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80180cfa10 0x7f80180c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.491+0000 7f801f0aa640 1 -- 172.21.15.29:0/2443168882 >> 172.21.15.29:0/2443168882 conn(0x7f80180d1d60 msgr2=0x7f80180aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:11.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.491+0000 7f801f0aa640 1 -- 172.21.15.29:0/2443168882 shutdown_connections 2024-04-25T00:32:11.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.491+0000 7f801f0aa640 1 -- 172.21.15.29:0/2443168882 wait complete. 2024-04-25T00:32:11.512 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:11.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe6bed5640 1 Processor -- start 2024-04-25T00:32:11.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe6bed5640 1 -- start start 2024-04-25T00:32:11.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe6bed5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efe6415c420 0x7efe64071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe6bed5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe640722c0 0x7efe640cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe6bed5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe6415e850 con 0x7efe6415e060 2024-04-25T00:32:11.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe6bed5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7efe6415f700 con 0x7efe6415c420 2024-04-25T00:32:11.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe6bed5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efe6415eaf0 con 0x7efe640722c0 2024-04-25T00:32:11.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe69470640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efe6415c420 0x7efe64071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe69c71640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7efe6415e060 0x7efe6415bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:36308/0 (socket says 172.21.15.29:36308) 2024-04-25T00:32:11.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe6a472640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe640722c0 0x7efe640cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe69c71640 1 -- 172.21.15.29:0/641816132 learned_addr learned my addr 172.21.15.29:0/641816132 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:11.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe69470640 1 -- 172.21.15.29:0/641816132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe640722c0 msgr2=0x7efe640cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:11.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe69470640 1 --2- 172.21.15.29:0/641816132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe640722c0 0x7efe640cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe69470640 1 -- 172.21.15.29:0/641816132 >> v1:172.21.15.157:6789/0 conn(0x7efe6415e060 legacy=0x7efe6415bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.635+0000 7efe69470640 1 -- 172.21.15.29:0/641816132 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6415edd0 con 0x7efe6415c420 2024-04-25T00:32:11.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.639+0000 7efe69470640 1 --2- 172.21.15.29:0/641816132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe6415c420 0x7efe64071d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efe6415f880 tx=0x7efe5401e6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=326a0a5055eee3c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:11.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.639+0000 7efe68c6f640 1 -- 172.21.15.29:0/641816132 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe540ab470 con 0x7efe6415c420 2024-04-25T00:32:11.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.639+0000 7efe68c6f640 1 -- 172.21.15.29:0/641816132 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe540ab610 con 0x7efe6415c420 2024-04-25T00:32:11.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.639+0000 7efe68c6f640 1 -- 172.21.15.29:0/641816132 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe540ab9e0 con 0x7efe6415c420 2024-04-25T00:32:11.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.639+0000 7efe6bed5640 1 -- 172.21.15.29:0/641816132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe6415c420 msgr2=0x7efe64071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.639+0000 7efe6bed5640 1 --2- 172.21.15.29:0/641816132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe6415c420 0x7efe64071d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efe6415f880 tx=0x7efe5401e6b0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.639+0000 7efe6bed5640 1 -- 172.21.15.29:0/641816132 shutdown_connections 2024-04-25T00:32:11.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.639+0000 7efe6bed5640 1 --2- 172.21.15.29:0/641816132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe640722c0 0x7efe640cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.639+0000 7efe6bed5640 1 --2- 172.21.15.29:0/641816132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe6415c420 0x7efe64071d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.639+0000 7efe6bed5640 1 -- 172.21.15.29:0/641816132 >> 172.21.15.29:0/641816132 conn(0x7efe6406c320 msgr2=0x7efe6406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:11.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.639+0000 7efe6bed5640 1 -- 172.21.15.29:0/641816132 shutdown_connections 2024-04-25T00:32:11.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.639+0000 7efe6bed5640 1 -- 172.21.15.29:0/641816132 wait complete. 2024-04-25T00:32:11.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe6bed5640 1 Processor -- start 2024-04-25T00:32:11.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe6bed5640 1 -- start start 2024-04-25T00:32:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe6bed5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe640722c0 0x7efe640ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe6bed5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe6415c420 0x7efe6414b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe6bed5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe6415e060 0x7efe6414bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe6bed5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efe6406f790 con 0x7efe640722c0 2024-04-25T00:32:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe6bed5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efe6406f610 con 0x7efe6415c420 2024-04-25T00:32:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe6bed5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efe6406b210 con 0x7efe6415e060 2024-04-25T00:32:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe69c71640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe640722c0 0x7efe640ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe69c71640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe640722c0 0x7efe640ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40500/0 (socket says 172.21.15.29:40500) 2024-04-25T00:32:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe69c71640 1 -- 172.21.15.29:0/3074774315 learned_addr learned my addr 172.21.15.29:0/3074774315 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe6a472640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe6415e060 0x7efe6414bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe69470640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe6415c420 0x7efe6414b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe69c71640 1 -- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe6415e060 msgr2=0x7efe6414bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe69c71640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe6415e060 0x7efe6414bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe69c71640 1 -- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe6415c420 msgr2=0x7efe6414b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe69c71640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe6415c420 0x7efe6414b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe69c71640 1 -- 172.21.15.29:0/3074774315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe640c4c10 con 0x7efe640722c0 2024-04-25T00:32:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe69c71640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe640722c0 0x7efe640ca5c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efe6000c370 tx=0x7efe6000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe5a7fc640 1 -- 172.21.15.29:0/3074774315 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60017070 con 0x7efe640722c0 2024-04-25T00:32:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe5a7fc640 1 -- 172.21.15.29:0/3074774315 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe60005cf0 con 0x7efe640722c0 2024-04-25T00:32:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe5a7fc640 1 -- 172.21.15.29:0/3074774315 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe600072f0 con 0x7efe640722c0 2024-04-25T00:32:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe6bed5640 1 -- 172.21.15.29:0/3074774315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe640c4ef0 con 0x7efe640722c0 2024-04-25T00:32:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe6bed5640 1 -- 172.21.15.29:0/3074774315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe640c54b0 con 0x7efe640722c0 2024-04-25T00:32:11.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.643+0000 7efe6bed5640 1 -- 172.21.15.29:0/3074774315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe64069270 con 0x7efe640722c0 2024-04-25T00:32:11.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.647+0000 7efe5a7fc640 1 -- 172.21.15.29:0/3074774315 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efe6001a070 con 0x7efe640722c0 2024-04-25T00:32:11.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.647+0000 7efe5a7fc640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efe480678c0 0x7efe48069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.647+0000 7efe5a7fc640 1 -- 172.21.15.29:0/3074774315 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efe60085070 con 0x7efe640722c0 2024-04-25T00:32:11.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.647+0000 7efe69470640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efe480678c0 0x7efe48069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.647+0000 7efe69470640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efe480678c0 0x7efe48069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe540b0000 tx=0x7efe540b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.651+0000 7efe5a7fc640 1 -- 172.21.15.29:0/3074774315 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efe60055ba0 con 0x7efe640722c0 2024-04-25T00:32:11.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.807+0000 7efe6bed5640 1 -- 172.21.15.29:0/3074774315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efe6406e090 con 0x7efe640722c0 2024-04-25T00:32:11.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.811+0000 7efe5a7fc640 1 -- 172.21.15.29:0/3074774315 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7efe600554a0 con 0x7efe640722c0 2024-04-25T00:32:11.814 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:11.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.811+0000 7efe6bed5640 1 -- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efe480678c0 msgr2=0x7efe48069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.811+0000 7efe6bed5640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efe480678c0 0x7efe48069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe540b0000 tx=0x7efe540b5040 comp rx=0 tx=0).stop 2024-04-25T00:32:11.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.811+0000 7efe6bed5640 1 -- 172.21.15.29:0/3074774315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe640722c0 msgr2=0x7efe640ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.811+0000 7efe6bed5640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe640722c0 0x7efe640ca5c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efe6000c370 tx=0x7efe6000c830 comp rx=0 tx=0).stop 2024-04-25T00:32:11.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.811+0000 7efe6bed5640 1 -- 172.21.15.29:0/3074774315 shutdown_connections 2024-04-25T00:32:11.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.811+0000 7efe6bed5640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe6415e060 0x7efe6414bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.811+0000 7efe6bed5640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efe480678c0 0x7efe48069d80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.811+0000 7efe6bed5640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe6415c420 0x7efe6414b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.811+0000 7efe6bed5640 1 --2- 172.21.15.29:0/3074774315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe640722c0 0x7efe640ca5c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.811+0000 7efe6bed5640 1 -- 172.21.15.29:0/3074774315 >> 172.21.15.29:0/3074774315 conn(0x7efe6406c320 msgr2=0x7efe640b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:11.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.811+0000 7efe6bed5640 1 -- 172.21.15.29:0/3074774315 shutdown_connections 2024-04-25T00:32:11.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.811+0000 7efe6bed5640 1 -- 172.21.15.29:0/3074774315 wait complete. 2024-04-25T00:32:11.833 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:32:11.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac2de8f640 1 Processor -- start 2024-04-25T00:32:11.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac2de8f640 1 -- start start 2024-04-25T00:32:11.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac2de8f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280b3140 0x7fac2806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac2de8f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fac280b4b10 0x7fac280b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac2de8f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fac280b3a60 0x7fac2815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac2de8f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fac280b41e0 con 0x7fac280b4b10 2024-04-25T00:32:11.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac2de8f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fac2806d960 con 0x7fac280b3a60 2024-04-25T00:32:11.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac2de8f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac2806dc40 con 0x7fac280b3140 2024-04-25T00:32:11.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac26ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fac280b4b10 0x7fac280b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac26ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fac280b4b10 0x7fac280b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40508/0 (socket says 172.21.15.29:40508) 2024-04-25T00:32:11.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac26ffd640 1 -- 172.21.15.29:0/810831580 learned_addr learned my addr 172.21.15.29:0/810831580 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:11.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac277fe640 1 --2- 172.21.15.29:0/810831580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280b3140 0x7fac2806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac27fff640 1 --2- 172.21.15.29:0/810831580 >> v2:172.21.15.157:3300/0 conn(0x7fac280b3a60 0x7fac2815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac26ffd640 1 -- 172.21.15.29:0/810831580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280b3140 msgr2=0x7fac2806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:11.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.951+0000 7fac26ffd640 1 --2- 172.21.15.29:0/810831580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280b3140 0x7fac2806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac26ffd640 1 -- 172.21.15.29:0/810831580 >> v2:172.21.15.157:3300/0 conn(0x7fac280b3a60 msgr2=0x7fac2815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:11.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac26ffd640 1 --2- 172.21.15.29:0/810831580 >> v2:172.21.15.157:3300/0 conn(0x7fac280b3a60 0x7fac2815c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac26ffd640 1 -- 172.21.15.29:0/810831580 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2806df20 con 0x7fac280b4b10 2024-04-25T00:32:11.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac26ffd640 1 --2- 172.21.15.29:0/810831580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac280b4b10 0x7fac280b3520 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fac280b4790 tx=0x7fac1c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a343100682b20d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:11.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac267fc640 1 -- 172.21.15.29:0/810831580 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0b0070 con 0x7fac280b4b10 2024-04-25T00:32:11.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac267fc640 1 -- 172.21.15.29:0/810831580 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac1c0ab600 con 0x7fac280b4b10 2024-04-25T00:32:11.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac267fc640 1 -- 172.21.15.29:0/810831580 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0ab9d0 con 0x7fac280b4b10 2024-04-25T00:32:11.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac2de8f640 1 -- 172.21.15.29:0/810831580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac280b4b10 msgr2=0x7fac280b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac2de8f640 1 --2- 172.21.15.29:0/810831580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac280b4b10 0x7fac280b3520 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fac280b4790 tx=0x7fac1c01c520 comp rx=0 tx=0).stop 2024-04-25T00:32:11.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac2de8f640 1 -- 172.21.15.29:0/810831580 shutdown_connections 2024-04-25T00:32:11.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac2de8f640 1 --2- 172.21.15.29:0/810831580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac280b4b10 0x7fac280b3520 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac2de8f640 1 --2- 172.21.15.29:0/810831580 >> v2:172.21.15.157:3300/0 conn(0x7fac280b3a60 0x7fac2815c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac2de8f640 1 --2- 172.21.15.29:0/810831580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280b3140 0x7fac2806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac2de8f640 1 -- 172.21.15.29:0/810831580 >> 172.21.15.29:0/810831580 conn(0x7fac280b1870 msgr2=0x7fac280b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:11.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac2de8f640 1 -- 172.21.15.29:0/810831580 shutdown_connections 2024-04-25T00:32:11.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.955+0000 7fac2de8f640 1 -- 172.21.15.29:0/810831580 wait complete. 2024-04-25T00:32:11.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac2de8f640 1 Processor -- start 2024-04-25T00:32:11.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac2de8f640 1 -- start start 2024-04-25T00:32:11.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac2de8f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac280b3140 0x7fac28139a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac2de8f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280b3a60 0x7fac2813ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac2de8f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac280b4b10 0x7fac2813b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac2de8f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac280ce3c0 con 0x7fac280b3140 2024-04-25T00:32:11.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac2de8f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac280ce240 con 0x7fac280b4b10 2024-04-25T00:32:11.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac2de8f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac280ce540 con 0x7fac280b3a60 2024-04-25T00:32:11.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac280b3140 0x7fac28139a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac280b3140 0x7fac28139a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40512/0 (socket says 172.21.15.29:40512) 2024-04-25T00:32:11.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac277fe640 1 -- 172.21.15.29:0/1590420411 learned_addr learned my addr 172.21.15.29:0/1590420411 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:11.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac26ffd640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280b3a60 0x7fac2813ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac27fff640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac280b4b10 0x7fac2813b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac277fe640 1 -- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280b3a60 msgr2=0x7fac2813ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac277fe640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280b3a60 0x7fac2813ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac277fe640 1 -- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac280b4b10 msgr2=0x7fac2813b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:11.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac277fe640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac280b4b10 0x7fac2813b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:11.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac277fe640 1 -- 172.21.15.29:0/1590420411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac28139f70 con 0x7fac280b3140 2024-04-25T00:32:11.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac27fff640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac280b4b10 0x7fac2813b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:11.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac277fe640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac280b3140 0x7fac28139a30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fac1800e8d0 tx=0x7fac1800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:11.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac0ffff640 1 -- 172.21.15.29:0/1590420411 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18017070 con 0x7fac280b3140 2024-04-25T00:32:11.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac0ffff640 1 -- 172.21.15.29:0/1590420411 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac1800c940 con 0x7fac280b3140 2024-04-25T00:32:11.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac2de8f640 1 -- 172.21.15.29:0/1590420411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac28156e50 con 0x7fac280b3140 2024-04-25T00:32:11.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac0ffff640 1 -- 172.21.15.29:0/1590420411 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1800cd10 con 0x7fac280b3140 2024-04-25T00:32:11.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac2de8f640 1 -- 172.21.15.29:0/1590420411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac28157410 con 0x7fac280b3140 2024-04-25T00:32:11.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac2de8f640 1 -- 172.21.15.29:0/1590420411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac28069270 con 0x7fac280b3140 2024-04-25T00:32:11.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac0ffff640 1 -- 172.21.15.29:0/1590420411 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fac1801a070 con 0x7fac280b3140 2024-04-25T00:32:11.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac0ffff640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fac140678c0 0x7fac14069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:11.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.959+0000 7fac0ffff640 1 -- 172.21.15.29:0/1590420411 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fac18085c60 con 0x7fac280b3140 2024-04-25T00:32:11.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.963+0000 7fac26ffd640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fac140678c0 0x7fac14069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:11.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.963+0000 7fac26ffd640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fac140678c0 0x7fac14069d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0ad000 tx=0x7fac1c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:11.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:11.963+0000 7fac0ffff640 1 -- 172.21.15.29:0/1590420411 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fac18056830 con 0x7fac280b3140 2024-04-25T00:32:12.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.135+0000 7fac2de8f640 1 -- 172.21.15.29:0/1590420411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fac280cecd0 con 0x7fac280b3140 2024-04-25T00:32:12.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.135+0000 7fac0ffff640 1 -- 172.21.15.29:0/1590420411 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fac18056130 con 0x7fac280b3140 2024-04-25T00:32:12.140 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:32:12.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.139+0000 7fac2de8f640 1 -- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fac140678c0 msgr2=0x7fac14069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.139+0000 7fac2de8f640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fac140678c0 0x7fac14069d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0ad000 tx=0x7fac1c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:32:12.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.139+0000 7fac2de8f640 1 -- 172.21.15.29:0/1590420411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac280b3140 msgr2=0x7fac28139a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.139+0000 7fac2de8f640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac280b3140 0x7fac28139a30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fac1800e8d0 tx=0x7fac1800ed90 comp rx=0 tx=0).stop 2024-04-25T00:32:12.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.139+0000 7fac2de8f640 1 -- 172.21.15.29:0/1590420411 shutdown_connections 2024-04-25T00:32:12.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.139+0000 7fac2de8f640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fac140678c0 0x7fac14069d80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.139+0000 7fac2de8f640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac280b4b10 0x7fac2813b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.139+0000 7fac2de8f640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac280b3a60 0x7fac2813ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.139+0000 7fac2de8f640 1 --2- 172.21.15.29:0/1590420411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac280b3140 0x7fac28139a30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.139+0000 7fac2de8f640 1 -- 172.21.15.29:0/1590420411 >> 172.21.15.29:0/1590420411 conn(0x7fac280b1870 msgr2=0x7fac2806cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:12.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.139+0000 7fac2de8f640 1 -- 172.21.15.29:0/1590420411 shutdown_connections 2024-04-25T00:32:12.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.139+0000 7fac2de8f640 1 -- 172.21.15.29:0/1590420411 wait complete. 2024-04-25T00:32:12.158 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:32:12.158 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:12.165 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:12.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc398b640 1 Processor -- start 2024-04-25T00:32:12.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc398b640 1 -- start start 2024-04-25T00:32:12.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc398b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4cbc0721f0 0x7f4cbc072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc398b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4cbc0ce820 0x7f4cbc0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc398b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cbc06e580 con 0x7f4cbc071e10 2024-04-25T00:32:12.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc398b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4cbc06f5e0 con 0x7f4cbc0721f0 2024-04-25T00:32:12.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc398b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4cbc15c1a0 con 0x7f4cbc0ce820 2024-04-25T00:32:12.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc1727640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4cbc071e10 0x7f4cbc15ba90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35194/0 (socket says 172.21.15.29:35194) 2024-04-25T00:32:12.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc1727640 1 -- 172.21.15.29:0/2250823721 learned_addr learned my addr 172.21.15.29:0/2250823721 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:12.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc1f28640 1 --2- 172.21.15.29:0/2250823721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4cbc0ce820 0x7f4cbc0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc0f26640 1 --2- 172.21.15.29:0/2250823721 >> v2:172.21.15.157:3300/0 conn(0x7f4cbc0721f0 0x7f4cbc072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2920504445 0 0) 0x7f4cbc06e580 con 0x7f4cbc071e10 2024-04-25T00:32:12.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ca8005740 con 0x7f4cbc071e10 2024-04-25T00:32:12.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2127723783 0 0) 0x7f4ca8005740 con 0x7f4cbc071e10 2024-04-25T00:32:12.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4cbc0ce820 msgr2=0x7f4cbc0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:12.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 --2- 172.21.15.29:0/2250823721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4cbc0ce820 0x7f4cbc0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 >> v2:172.21.15.157:3300/0 conn(0x7f4cbc0721f0 msgr2=0x7f4cbc072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:12.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 --2- 172.21.15.29:0/2250823721 >> v2:172.21.15.157:3300/0 conn(0x7f4cbc0721f0 0x7f4cbc072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cbc06e820 con 0x7f4cbc071e10 2024-04-25T00:32:12.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4cb8001a00 con 0x7f4cbc071e10 2024-04-25T00:32:12.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 >> v1:172.21.15.29:6789/0 conn(0x7f4cbc071e10 legacy=0x7f4cbc15ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 --2- 172.21.15.29:0/2250823721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4cbc0ce820 0x7f4ca8005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 --2- 172.21.15.29:0/2250823721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ca8006150 0x7f4ca8006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 --2- 172.21.15.29:0/2250823721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca8006f60 0x7f4ca80073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc06f5e0 con 0x7f4ca8006150 2024-04-25T00:32:12.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc15c1a0 con 0x7f4ca8006f60 2024-04-25T00:32:12.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca800dbe0 con 0x7f4cbc0ce820 2024-04-25T00:32:12.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc0f26640 1 --2- 172.21.15.29:0/2250823721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ca8006150 0x7f4ca8006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc1727640 1 --2- 172.21.15.29:0/2250823721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4cbc0ce820 0x7f4ca8005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc0f26640 1 -- 172.21.15.29:0/2250823721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4cbc0ce820 msgr2=0x7f4ca8005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:12.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc1f28640 1 --2- 172.21.15.29:0/2250823721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca8006f60 0x7f4ca80073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc0f26640 1 --2- 172.21.15.29:0/2250823721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4cbc0ce820 0x7f4ca8005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc0f26640 1 -- 172.21.15.29:0/2250823721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca8006f60 msgr2=0x7f4ca80073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:12.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc0f26640 1 --2- 172.21.15.29:0/2250823721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca8006f60 0x7f4ca80073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.275+0000 7f4cc0f26640 1 -- 172.21.15.29:0/2250823721 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca8005740 con 0x7f4ca8006150 2024-04-25T00:32:12.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc0f26640 1 --2- 172.21.15.29:0/2250823721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ca8006150 0x7f4ca8006a20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009ae0 tx=0x7f4cac01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca4de63a577e121 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0b2070 con 0x7f4ca8006150 2024-04-25T00:32:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cac0a5590 con 0x7f4ca8006150 2024-04-25T00:32:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cb3fff640 1 -- 172.21.15.29:0/2250823721 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0096b0 con 0x7f4ca8006150 2024-04-25T00:32:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 -- 172.21.15.29:0/2250823721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ca8006150 msgr2=0x7f4ca8006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 --2- 172.21.15.29:0/2250823721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ca8006150 0x7f4ca8006a20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009ae0 tx=0x7f4cac01ca50 comp rx=0 tx=0).stop 2024-04-25T00:32:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc1f28640 1 -- 172.21.15.29:0/2250823721 reap_dead start 2024-04-25T00:32:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 -- 172.21.15.29:0/2250823721 shutdown_connections 2024-04-25T00:32:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 -- 172.21.15.29:0/2250823721 >> 172.21.15.29:0/2250823721 conn(0x7f4cbc0b2920 msgr2=0x7f4cbc0b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 -- 172.21.15.29:0/2250823721 shutdown_connections 2024-04-25T00:32:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 -- 172.21.15.29:0/2250823721 wait complete. 2024-04-25T00:32:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 Processor -- start 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 -- start start 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca8006150 0x7f4cbc145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cbc0c9fa0 0x7f4cbc0ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cbc15d310 0x7f4cbc15d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc06ede0 con 0x7f4cbc0c9fa0 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc0b0370 con 0x7f4cbc15d310 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc398b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4cbc06f0c0 con 0x7f4ca8006150 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc0f26640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cbc0c9fa0 0x7f4cbc0ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc1727640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca8006150 0x7f4cbc145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc0f26640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cbc0c9fa0 0x7f4cbc0ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40526/0 (socket says 172.21.15.29:40526) 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc1727640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca8006150 0x7f4cbc145730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:34666/0 (socket says 172.21.15.29:34666) 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc0f26640 1 -- 172.21.15.29:0/1446146978 learned_addr learned my addr 172.21.15.29:0/1446146978 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.279+0000 7f4cc1f28640 1 --2- 172.21.15.29:0/1446146978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cbc15d310 0x7f4cbc15d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cc0f26640 1 -- 172.21.15.29:0/1446146978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca8006150 msgr2=0x7f4cbc145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cc0f26640 1 --2- 172.21.15.29:0/1446146978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca8006150 0x7f4cbc145730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cc0f26640 1 -- 172.21.15.29:0/1446146978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cbc15d310 msgr2=0x7f4cbc15d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cc0f26640 1 --2- 172.21.15.29:0/1446146978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cbc15d310 0x7f4cbc15d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cc0f26640 1 -- 172.21.15.29:0/1446146978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cbc15b310 con 0x7f4cbc0c9fa0 2024-04-25T00:32:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cc0f26640 1 --2- 172.21.15.29:0/1446146978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cbc0c9fa0 0x7f4cbc0ca420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4cac01d2f0 tx=0x7f4ca8007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cb1ffb640 1 -- 172.21.15.29:0/1446146978 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0bc020 con 0x7f4cbc0c9fa0 2024-04-25T00:32:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cc398b640 1 -- 172.21.15.29:0/1446146978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cbc0b3e50 con 0x7f4cbc0c9fa0 2024-04-25T00:32:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cb1ffb640 1 -- 172.21.15.29:0/1446146978 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cac004020 con 0x7f4cbc0c9fa0 2024-04-25T00:32:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cb1ffb640 1 -- 172.21.15.29:0/1446146978 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0b2040 con 0x7f4cbc0c9fa0 2024-04-25T00:32:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cc398b640 1 -- 172.21.15.29:0/1446146978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cbc15def0 con 0x7f4cbc0c9fa0 2024-04-25T00:32:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cc398b640 1 -- 172.21.15.29:0/1446146978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cbc069270 con 0x7f4cbc0c9fa0 2024-04-25T00:32:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cb1ffb640 1 -- 172.21.15.29:0/1446146978 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4cac003620 con 0x7f4cbc0c9fa0 2024-04-25T00:32:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cb1ffb640 1 --2- 172.21.15.29:0/1446146978 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ca00678c0 0x7f4ca0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cb1ffb640 1 -- 172.21.15.29:0/1446146978 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4cac120000 con 0x7f4cbc0c9fa0 2024-04-25T00:32:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cc1727640 1 --2- 172.21.15.29:0/1446146978 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ca00678c0 0x7f4ca0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.283+0000 7f4cc1727640 1 --2- 172.21.15.29:0/1446146978 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ca00678c0 0x7f4ca0069d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8001bf0 tx=0x7f4cb8001740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:12.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.287+0000 7f4cb1ffb640 1 -- 172.21.15.29:0/1446146978 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4cac0c6020 con 0x7f4cbc0c9fa0 2024-04-25T00:32:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.443+0000 7f4cc398b640 1 -- 172.21.15.29:0/1446146978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cbc06e050 con 0x7f4cbc0c9fa0 2024-04-25T00:32:12.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.447+0000 7f4cb1ffb640 1 -- 172.21.15.29:0/1446146978 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f4cbc0b3e50 con 0x7f4cbc0c9fa0 2024-04-25T00:32:12.451 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:12.451 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.447+0000 7f4cc398b640 1 -- 172.21.15.29:0/1446146978 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ca00678c0 msgr2=0x7f4ca0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.447+0000 7f4cc398b640 1 --2- 172.21.15.29:0/1446146978 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ca00678c0 0x7f4ca0069d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8001bf0 tx=0x7f4cb8001740 comp rx=0 tx=0).stop 2024-04-25T00:32:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.447+0000 7f4cc398b640 1 -- 172.21.15.29:0/1446146978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cbc0c9fa0 msgr2=0x7f4cbc0ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.447+0000 7f4cc398b640 1 --2- 172.21.15.29:0/1446146978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cbc0c9fa0 0x7f4cbc0ca420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4cac01d2f0 tx=0x7f4ca8007a50 comp rx=0 tx=0).stop 2024-04-25T00:32:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.447+0000 7f4cc398b640 1 -- 172.21.15.29:0/1446146978 shutdown_connections 2024-04-25T00:32:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.447+0000 7f4cc398b640 1 --2- 172.21.15.29:0/1446146978 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ca00678c0 0x7f4ca0069d80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.447+0000 7f4cc398b640 1 --2- 172.21.15.29:0/1446146978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cbc15d310 0x7f4cbc15d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.447+0000 7f4cc398b640 1 --2- 172.21.15.29:0/1446146978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cbc0c9fa0 0x7f4cbc0ca420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.447+0000 7f4cc398b640 1 --2- 172.21.15.29:0/1446146978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca8006150 0x7f4cbc145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.447+0000 7f4cc398b640 1 -- 172.21.15.29:0/1446146978 >> 172.21.15.29:0/1446146978 conn(0x7f4cbc0b2920 msgr2=0x7f4cbc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.447+0000 7f4cc398b640 1 -- 172.21.15.29:0/1446146978 shutdown_connections 2024-04-25T00:32:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.447+0000 7f4cc398b640 1 -- 172.21.15.29:0/1446146978 wait complete. 2024-04-25T00:32:12.468 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:12.469 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:12.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.587+0000 7fc3a5191640 1 Processor -- start 2024-04-25T00:32:12.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc3a5191640 1 -- start start 2024-04-25T00:32:12.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc3a5191640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc3a015e200 0x7fc3a00d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc3a5191640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc3a00d2c60 0x7fc3a00b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc3a5191640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3a00cfc10 con 0x7fc3a00cf690 2024-04-25T00:32:12.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc3a5191640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc3a00b38d0 con 0x7fc3a015e200 2024-04-25T00:32:12.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc3a5191640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc3a00cfef0 con 0x7fc3a00d2c60 2024-04-25T00:32:12.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc39ed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc3a015e200 0x7fc3a00d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc39ed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc3a015e200 0x7fc3a00d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40542/0 (socket says 172.21.15.29:40542) 2024-04-25T00:32:12.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc39ed9d640 1 -- 172.21.15.29:0/950121061 learned_addr learned my addr 172.21.15.29:0/950121061 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:12.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc39f59e640 1 --2- 172.21.15.29:0/950121061 >> v2:172.21.15.157:3300/0 conn(0x7fc3a00d2c60 0x7fc3a00b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc39ed9d640 1 -- 172.21.15.29:0/950121061 >> v2:172.21.15.157:3300/0 conn(0x7fc3a00d2c60 msgr2=0x7fc3a00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:12.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc39ed9d640 1 --2- 172.21.15.29:0/950121061 >> v2:172.21.15.157:3300/0 conn(0x7fc3a00d2c60 0x7fc3a00b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc39ed9d640 1 -- 172.21.15.29:0/950121061 >> v1:172.21.15.29:6789/0 conn(0x7fc3a00cf690 legacy=0x7fc3a00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc39ed9d640 1 -- 172.21.15.29:0/950121061 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a00b3b20 con 0x7fc3a015e200 2024-04-25T00:32:12.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc39f59e640 1 --2- 172.21.15.29:0/950121061 >> v2:172.21.15.157:3300/0 conn(0x7fc3a00d2c60 0x7fc3a00b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:12.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc39ed9d640 1 --2- 172.21.15.29:0/950121061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3a015e200 0x7fc3a00d47f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00cfb20 tx=0x7fc39001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bde5a3fb676efde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:12.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc39dd9b640 1 -- 172.21.15.29:0/950121061 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3900aa670 con 0x7fc3a015e200 2024-04-25T00:32:12.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc39dd9b640 1 -- 172.21.15.29:0/950121061 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3900af070 con 0x7fc3a015e200 2024-04-25T00:32:12.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.591+0000 7fc39dd9b640 1 -- 172.21.15.29:0/950121061 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3900aaa30 con 0x7fc3a015e200 2024-04-25T00:32:12.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 -- 172.21.15.29:0/950121061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3a015e200 msgr2=0x7fc3a00d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 --2- 172.21.15.29:0/950121061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3a015e200 0x7fc3a00d47f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00cfb20 tx=0x7fc39001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 -- 172.21.15.29:0/950121061 shutdown_connections 2024-04-25T00:32:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 --2- 172.21.15.29:0/950121061 >> v2:172.21.15.157:3300/0 conn(0x7fc3a00d2c60 0x7fc3a00b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 --2- 172.21.15.29:0/950121061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3a015e200 0x7fc3a00d47f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 -- 172.21.15.29:0/950121061 >> 172.21.15.29:0/950121061 conn(0x7fc3a009d280 msgr2=0x7fc3a00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 -- 172.21.15.29:0/950121061 shutdown_connections 2024-04-25T00:32:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 -- 172.21.15.29:0/950121061 wait complete. 2024-04-25T00:32:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 Processor -- start 2024-04-25T00:32:12.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 -- start start 2024-04-25T00:32:12.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3a00cf690 0x7fc3a015aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3a00d2c60 0x7fc3a0150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a015e200 0x7fc3a00c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a00b51f0 con 0x7fc3a00cf690 2024-04-25T00:32:12.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a00b5070 con 0x7fc3a015e200 2024-04-25T00:32:12.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a00b5370 con 0x7fc3a00d2c60 2024-04-25T00:32:12.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc39ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3a00cf690 0x7fc3a015aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc39ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3a00cf690 0x7fc3a015aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40550/0 (socket says 172.21.15.29:40550) 2024-04-25T00:32:12.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc39ed9d640 1 -- 172.21.15.29:0/2430853916 learned_addr learned my addr 172.21.15.29:0/2430853916 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:12.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc39ed9d640 1 -- 172.21.15.29:0/2430853916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3a00d2c60 msgr2=0x7fc3a0150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc39ed9d640 1 --2- 172.21.15.29:0/2430853916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3a00d2c60 0x7fc3a0150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc39ed9d640 1 -- 172.21.15.29:0/2430853916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a015e200 msgr2=0x7fc3a00c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc39ed9d640 1 --2- 172.21.15.29:0/2430853916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a015e200 0x7fc3a00c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc39ed9d640 1 -- 172.21.15.29:0/2430853916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a01517d0 con 0x7fc3a00cf690 2024-04-25T00:32:12.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc39ed9d640 1 --2- 172.21.15.29:0/2430853916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3a00cf690 0x7fc3a015aa70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc3900ac040 tx=0x7fc39001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:12.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3877fe640 1 -- 172.21.15.29:0/2430853916 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3900b3800 con 0x7fc3a00cf690 2024-04-25T00:32:12.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3877fe640 1 -- 172.21.15.29:0/2430853916 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3900b39c0 con 0x7fc3a00cf690 2024-04-25T00:32:12.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 -- 172.21.15.29:0/2430853916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a0151a70 con 0x7fc3a00cf690 2024-04-25T00:32:12.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3877fe640 1 -- 172.21.15.29:0/2430853916 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3900b3db0 con 0x7fc3a00cf690 2024-04-25T00:32:12.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 -- 172.21.15.29:0/2430853916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a0153a60 con 0x7fc3a00cf690 2024-04-25T00:32:12.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.595+0000 7fc3a5191640 1 -- 172.21.15.29:0/2430853916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a0069270 con 0x7fc3a00cf690 2024-04-25T00:32:12.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.599+0000 7fc3877fe640 1 -- 172.21.15.29:0/2430853916 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc3900b2380 con 0x7fc3a00cf690 2024-04-25T00:32:12.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.599+0000 7fc3877fe640 1 --2- 172.21.15.29:0/2430853916 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc38c0678f0 0x7fc38c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.599+0000 7fc3877fe640 1 -- 172.21.15.29:0/2430853916 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc390120020 con 0x7fc3a00cf690 2024-04-25T00:32:12.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.599+0000 7fc39e59c640 1 --2- 172.21.15.29:0/2430853916 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc38c0678f0 0x7fc38c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.599+0000 7fc39e59c640 1 --2- 172.21.15.29:0/2430853916 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc38c0678f0 0x7fc38c069db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc394000f70 tx=0x7fc3940079e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:12.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.603+0000 7fc3877fe640 1 -- 172.21.15.29:0/2430853916 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc3900c3060 con 0x7fc3a00cf690 2024-04-25T00:32:12.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3a5191640 1 -- 172.21.15.29:0/2430853916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc3a00d1ce0 con 0x7fc3a00cf690 2024-04-25T00:32:12.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3877fe640 1 -- 172.21.15.29:0/2430853916 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fc390016110 con 0x7fc3a00cf690 2024-04-25T00:32:12.762 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:12.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3a5191640 1 -- 172.21.15.29:0/2430853916 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc38c0678f0 msgr2=0x7fc38c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3a5191640 1 --2- 172.21.15.29:0/2430853916 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc38c0678f0 0x7fc38c069db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc394000f70 tx=0x7fc3940079e0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3a5191640 1 -- 172.21.15.29:0/2430853916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3a00cf690 msgr2=0x7fc3a015aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3a5191640 1 --2- 172.21.15.29:0/2430853916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3a00cf690 0x7fc3a015aa70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc3900ac040 tx=0x7fc39001ad30 comp rx=0 tx=0).stop 2024-04-25T00:32:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3a5191640 1 -- 172.21.15.29:0/2430853916 shutdown_connections 2024-04-25T00:32:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3a5191640 1 --2- 172.21.15.29:0/2430853916 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc38c0678f0 0x7fc38c069db0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3a5191640 1 --2- 172.21.15.29:0/2430853916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a015e200 0x7fc3a00c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3a5191640 1 --2- 172.21.15.29:0/2430853916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3a00d2c60 0x7fc3a0150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3a5191640 1 --2- 172.21.15.29:0/2430853916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3a00cf690 0x7fc3a015aa70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3a5191640 1 -- 172.21.15.29:0/2430853916 >> 172.21.15.29:0/2430853916 conn(0x7fc3a009d280 msgr2=0x7fc3a006d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3a5191640 1 -- 172.21.15.29:0/2430853916 shutdown_connections 2024-04-25T00:32:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.759+0000 7fc3a5191640 1 -- 172.21.15.29:0/2430853916 wait complete. 2024-04-25T00:32:12.779 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:12.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.887+0000 7f2906108640 1 Processor -- start 2024-04-25T00:32:12.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.887+0000 7f2906108640 1 -- start start 2024-04-25T00:32:12.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.887+0000 7f2906108640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000cf690 0x7f29000d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.887+0000 7f2906108640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29000cfc10 con 0x7f290015e200 2024-04-25T00:32:12.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.887+0000 7f2906108640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29000cfef0 con 0x7f29000d2a90 2024-04-25T00:32:12.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.887+0000 7f2906108640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f29000d2ee0 con 0x7f29000cf690 2024-04-25T00:32:12.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.887+0000 7f28ff7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f290015e200 0x7f29000d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35210/0 (socket says 172.21.15.29:35210) 2024-04-25T00:32:12.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.887+0000 7f28ff7fe640 1 -- 172.21.15.29:0/2951612982 learned_addr learned my addr 172.21.15.29:0/2951612982 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:12.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.887+0000 7f28feffd640 1 --2- 172.21.15.29:0/2951612982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000cf690 0x7f29000d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.887+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3963692731 0 0) 0x7f29000cfc10 con 0x7f290015e200 2024-04-25T00:32:12.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.887+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28e8003530 con 0x7f290015e200 2024-04-25T00:32:12.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3455941109 0 0) 0x7f28e8003530 con 0x7f290015e200 2024-04-25T00:32:12.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000cf690 msgr2=0x7f29000d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:12.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 --2- 172.21.15.29:0/2951612982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000cf690 0x7f29000d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 >> v1:172.21.15.157:6789/0 conn(0x7f29000d2a90 legacy=0x7f29000b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29000b3b20 con 0x7f290015e200 2024-04-25T00:32:12.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f28f0001680 con 0x7f290015e200 2024-04-25T00:32:12.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 >> v1:172.21.15.29:6789/0 conn(0x7f290015e200 legacy=0x7f29000d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 --2- 172.21.15.29:0/2951612982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e80039f0 0x7f28e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 --2- 172.21.15.29:0/2951612982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e8004810 0x7f28e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 --2- 172.21.15.29:0/2951612982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000cf690 0x7f28e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f29000d2ee0 con 0x7f28e8004810 2024-04-25T00:32:12.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f28e800ba80 con 0x7f28e80039f0 2024-04-25T00:32:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f28e800bc80 con 0x7f29000cf690 2024-04-25T00:32:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28feffd640 1 --2- 172.21.15.29:0/2951612982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e8004810 0x7f28e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28ff7fe640 1 --2- 172.21.15.29:0/2951612982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e80039f0 0x7f28e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fffff640 1 --2- 172.21.15.29:0/2951612982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000cf690 0x7f28e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28feffd640 1 -- 172.21.15.29:0/2951612982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000cf690 msgr2=0x7f28e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28feffd640 1 --2- 172.21.15.29:0/2951612982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000cf690 0x7f28e80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28feffd640 1 -- 172.21.15.29:0/2951612982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e80039f0 msgr2=0x7f28e8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28feffd640 1 --2- 172.21.15.29:0/2951612982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e80039f0 0x7f28e8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28feffd640 1 -- 172.21.15.29:0/2951612982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e8003530 con 0x7f28e8004810 2024-04-25T00:32:12.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28feffd640 1 --2- 172.21.15.29:0/2951612982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e8004810 0x7f28e8004c90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009ab0 tx=0x7f28f401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d1a2840cc63fed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:12.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f40b2070 con 0x7f28e8004810 2024-04-25T00:32:12.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f40a5510 con 0x7f28e8004810 2024-04-25T00:32:12.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fe7fc640 1 -- 172.21.15.29:0/2951612982 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f40096b0 con 0x7f28e8004810 2024-04-25T00:32:12.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f2906108640 1 -- 172.21.15.29:0/2951612982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e8004810 msgr2=0x7f28e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f2906108640 1 --2- 172.21.15.29:0/2951612982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e8004810 0x7f28e8004c90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009ab0 tx=0x7f28f401ca30 comp rx=0 tx=0).stop 2024-04-25T00:32:12.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f28fffff640 1 -- 172.21.15.29:0/2951612982 reap_dead start 2024-04-25T00:32:12.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f2906108640 1 -- 172.21.15.29:0/2951612982 shutdown_connections 2024-04-25T00:32:12.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f2906108640 1 -- 172.21.15.29:0/2951612982 >> 172.21.15.29:0/2951612982 conn(0x7f290009d280 msgr2=0x7f29000d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:12.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f2906108640 1 -- 172.21.15.29:0/2951612982 shutdown_connections 2024-04-25T00:32:12.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.891+0000 7f2906108640 1 -- 172.21.15.29:0/2951612982 wait complete. 2024-04-25T00:32:12.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f2906108640 1 Processor -- start 2024-04-25T00:32:12.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f2906108640 1 -- start start 2024-04-25T00:32:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f2906108640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e8004810 0x7f29000c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f2906108640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000b8fb0 0x7f29000c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f2906108640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29000c99b0 0x7f29000c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f2906108640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f290015e5e0 con 0x7f28e8004810 2024-04-25T00:32:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f2906108640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f29000b38d0 con 0x7f29000c99b0 2024-04-25T00:32:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f2906108640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f290015e760 con 0x7f29000b8fb0 2024-04-25T00:32:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28ff7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e8004810 0x7f29000c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28ff7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e8004810 0x7f29000c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40568/0 (socket says 172.21.15.29:40568) 2024-04-25T00:32:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28ff7fe640 1 -- 172.21.15.29:0/460227727 learned_addr learned my addr 172.21.15.29:0/460227727 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28feffd640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000b8fb0 0x7f29000c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28fffff640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29000c99b0 0x7f29000c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28ff7fe640 1 -- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000b8fb0 msgr2=0x7f29000c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28ff7fe640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000b8fb0 0x7f29000c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28ff7fe640 1 -- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29000c99b0 msgr2=0x7f29000c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28ff7fe640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29000c99b0 0x7f29000c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28ff7fe640 1 -- 172.21.15.29:0/460227727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29000b6e40 con 0x7f28e8004810 2024-04-25T00:32:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28fffff640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29000c99b0 0x7f29000c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28feffd640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000b8fb0 0x7f29000c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28ff7fe640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e8004810 0x7f29000c7aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f28f0011840 tx=0x7f28f0011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28e7fff640 1 -- 172.21.15.29:0/460227727 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0010040 con 0x7f28e8004810 2024-04-25T00:32:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28e7fff640 1 -- 172.21.15.29:0/460227727 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f001a070 con 0x7f28e8004810 2024-04-25T00:32:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f2906108640 1 -- 172.21.15.29:0/460227727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29000c7fe0 con 0x7f28e8004810 2024-04-25T00:32:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28e7fff640 1 -- 172.21.15.29:0/460227727 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f000fdc0 con 0x7f28e8004810 2024-04-25T00:32:12.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f2906108640 1 -- 172.21.15.29:0/460227727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29000ca9e0 con 0x7f28e8004810 2024-04-25T00:32:12.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f2906108640 1 -- 172.21.15.29:0/460227727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29000b7010 con 0x7f28e8004810 2024-04-25T00:32:12.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.895+0000 7f28e7fff640 1 -- 172.21.15.29:0/460227727 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f28f001f460 con 0x7f28e8004810 2024-04-25T00:32:12.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.899+0000 7f28e7fff640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f28ec0678c0 0x7f28ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:12.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.899+0000 7f28e7fff640 1 -- 172.21.15.29:0/460227727 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f28f0088f10 con 0x7f28e8004810 2024-04-25T00:32:12.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.899+0000 7f28feffd640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f28ec0678c0 0x7f28ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:12.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.899+0000 7f28feffd640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f28ec0678c0 0x7f28ec069d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f28f40af000 tx=0x7f28f40046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:12.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:12.903+0000 7f28e7fff640 1 -- 172.21.15.29:0/460227727 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f28f0059a50 con 0x7f28e8004810 2024-04-25T00:32:13.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.059+0000 7f2906108640 1 -- 172.21.15.29:0/460227727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f29000cf2f0 con 0x7f28e8004810 2024-04-25T00:32:13.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.059+0000 7f28e7fff640 1 -- 172.21.15.29:0/460227727 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f28f0059350 con 0x7f28e8004810 2024-04-25T00:32:13.064 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:13.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.063+0000 7f2906108640 1 -- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f28ec0678c0 msgr2=0x7f28ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:13.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.063+0000 7f2906108640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f28ec0678c0 0x7f28ec069d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f28f40af000 tx=0x7f28f40046d0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.063+0000 7f2906108640 1 -- 172.21.15.29:0/460227727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e8004810 msgr2=0x7f29000c7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:13.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.063+0000 7f2906108640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e8004810 0x7f29000c7aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f28f0011840 tx=0x7f28f0011d00 comp rx=0 tx=0).stop 2024-04-25T00:32:13.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.063+0000 7f2906108640 1 -- 172.21.15.29:0/460227727 shutdown_connections 2024-04-25T00:32:13.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.063+0000 7f2906108640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f28ec0678c0 0x7f28ec069d80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.063+0000 7f2906108640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29000c99b0 0x7f29000c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.063+0000 7f2906108640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29000b8fb0 0x7f29000c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.063+0000 7f2906108640 1 --2- 172.21.15.29:0/460227727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e8004810 0x7f29000c7aa0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.063+0000 7f2906108640 1 -- 172.21.15.29:0/460227727 >> 172.21.15.29:0/460227727 conn(0x7f290009d280 msgr2=0x7f29000d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:13.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.063+0000 7f2906108640 1 -- 172.21.15.29:0/460227727 shutdown_connections 2024-04-25T00:32:13.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.063+0000 7f2906108640 1 -- 172.21.15.29:0/460227727 wait complete. 2024-04-25T00:32:13.083 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:13.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa491c40640 1 Processor -- start 2024-04-25T00:32:13.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa491c40640 1 -- start start 2024-04-25T00:32:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa491c40640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa48c162420 0x7fa48c070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa491c40640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48c0b4e00 0x7fa48c0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa491c40640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa48c0b30b0 0x7fa48c166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa491c40640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa48c0b3500 con 0x7fa48c0b30b0 2024-04-25T00:32:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa491c40640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa48c071910 con 0x7fa48c162420 2024-04-25T00:32:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa491c40640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c071bf0 con 0x7fa48c0b4e00 2024-04-25T00:32:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa48bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa48c0b30b0 0x7fa48c166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa48bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa48c0b30b0 0x7fa48c166020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40580/0 (socket says 172.21.15.29:40580) 2024-04-25T00:32:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa48bfff640 1 -- 172.21.15.29:0/1241608584 learned_addr learned my addr 172.21.15.29:0/1241608584 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa48b7fe640 1 --2- 172.21.15.29:0/1241608584 >> v2:172.21.15.157:3300/0 conn(0x7fa48c162420 0x7fa48c070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa48affd640 1 --2- 172.21.15.29:0/1241608584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48c0b4e00 0x7fa48c0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa48bfff640 1 -- 172.21.15.29:0/1241608584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48c0b4e00 msgr2=0x7fa48c0b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa48bfff640 1 --2- 172.21.15.29:0/1241608584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48c0b4e00 0x7fa48c0b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa48bfff640 1 -- 172.21.15.29:0/1241608584 >> v2:172.21.15.157:3300/0 conn(0x7fa48c162420 msgr2=0x7fa48c070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa48bfff640 1 --2- 172.21.15.29:0/1241608584 >> v2:172.21.15.157:3300/0 conn(0x7fa48c162420 0x7fa48c070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.207+0000 7fa48bfff640 1 -- 172.21.15.29:0/1241608584 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c071ed0 con 0x7fa48c0b30b0 2024-04-25T00:32:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48bfff640 1 --2- 172.21.15.29:0/1241608584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa48c0b30b0 0x7fa48c166020 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa48c067e70 tx=0x7fa48401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33ee1825d9670cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48a7fc640 1 -- 172.21.15.29:0/1241608584 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4840b0070 con 0x7fa48c0b30b0 2024-04-25T00:32:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48a7fc640 1 -- 172.21.15.29:0/1241608584 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4840ab600 con 0x7fa48c0b30b0 2024-04-25T00:32:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48a7fc640 1 -- 172.21.15.29:0/1241608584 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4840ab9d0 con 0x7fa48c0b30b0 2024-04-25T00:32:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 -- 172.21.15.29:0/1241608584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa48c0b30b0 msgr2=0x7fa48c166020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 --2- 172.21.15.29:0/1241608584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa48c0b30b0 0x7fa48c166020 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa48c067e70 tx=0x7fa48401c260 comp rx=0 tx=0).stop 2024-04-25T00:32:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 -- 172.21.15.29:0/1241608584 shutdown_connections 2024-04-25T00:32:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 --2- 172.21.15.29:0/1241608584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa48c0b30b0 0x7fa48c166020 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 --2- 172.21.15.29:0/1241608584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48c0b4e00 0x7fa48c0b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 --2- 172.21.15.29:0/1241608584 >> v2:172.21.15.157:3300/0 conn(0x7fa48c162420 0x7fa48c070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 -- 172.21.15.29:0/1241608584 >> 172.21.15.29:0/1241608584 conn(0x7fa48c0b1870 msgr2=0x7fa48c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 -- 172.21.15.29:0/1241608584 shutdown_connections 2024-04-25T00:32:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 -- 172.21.15.29:0/1241608584 wait complete. 2024-04-25T00:32:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 Processor -- start 2024-04-25T00:32:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 -- start start 2024-04-25T00:32:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa48c0b30b0 0x7fa48c160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa48c0b4e00 0x7fa48c155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48c162420 0x7fa48c1fb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c06cf60 con 0x7fa48c0b4e00 2024-04-25T00:32:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c0716d0 con 0x7fa48c0b30b0 2024-04-25T00:32:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa491c40640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c06d0e0 con 0x7fa48c162420 2024-04-25T00:32:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa48c0b4e00 0x7fa48c155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa48c0b4e00 0x7fa48c155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40592/0 (socket says 172.21.15.29:40592) 2024-04-25T00:32:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48affd640 1 -- 172.21.15.29:0/4117247735 learned_addr learned my addr 172.21.15.29:0/4117247735 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48b7fe640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa48c0b30b0 0x7fa48c160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48bfff640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48c162420 0x7fa48c1fb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48affd640 1 -- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48c162420 msgr2=0x7fa48c1fb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48affd640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48c162420 0x7fa48c1fb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48affd640 1 -- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa48c0b30b0 msgr2=0x7fa48c160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48affd640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa48c0b30b0 0x7fa48c160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.211+0000 7fa48affd640 1 -- 172.21.15.29:0/4117247735 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c159100 con 0x7fa48c0b4e00 2024-04-25T00:32:13.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.215+0000 7fa48affd640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa48c0b4e00 0x7fa48c155790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa480007e10 tx=0x7fa48000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:13.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.215+0000 7fa473fff640 1 -- 172.21.15.29:0/4117247735 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480014620 con 0x7fa48c0b4e00 2024-04-25T00:32:13.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.215+0000 7fa473fff640 1 -- 172.21.15.29:0/4117247735 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa480019070 con 0x7fa48c0b4e00 2024-04-25T00:32:13.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.215+0000 7fa491c40640 1 -- 172.21.15.29:0/4117247735 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c1593e0 con 0x7fa48c0b4e00 2024-04-25T00:32:13.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.215+0000 7fa473fff640 1 -- 172.21.15.29:0/4117247735 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480014a00 con 0x7fa48c0b4e00 2024-04-25T00:32:13.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.215+0000 7fa491c40640 1 -- 172.21.15.29:0/4117247735 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c159c80 con 0x7fa48c0b4e00 2024-04-25T00:32:13.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.215+0000 7fa491c40640 1 -- 172.21.15.29:0/4117247735 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48c1595b0 con 0x7fa48c0b4e00 2024-04-25T00:32:13.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.215+0000 7fa473fff640 1 -- 172.21.15.29:0/4117247735 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa480014ba0 con 0x7fa48c0b4e00 2024-04-25T00:32:13.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.215+0000 7fa473fff640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa4780678c0 0x7fa478069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.215+0000 7fa473fff640 1 -- 172.21.15.29:0/4117247735 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa4800864f0 con 0x7fa48c0b4e00 2024-04-25T00:32:13.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.215+0000 7fa48b7fe640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa4780678c0 0x7fa478069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.215+0000 7fa48b7fe640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa4780678c0 0x7fa478069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa48c164c20 tx=0x7fa47c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:13.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.219+0000 7fa473fff640 1 -- 172.21.15.29:0/4117247735 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa480057030 con 0x7fa48c0b4e00 2024-04-25T00:32:13.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa491c40640 1 -- 172.21.15.29:0/4117247735 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa48c069270 con 0x7fa48c0b4e00 2024-04-25T00:32:13.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa473fff640 1 -- 172.21.15.29:0/4117247735 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa480056930 con 0x7fa48c0b4e00 2024-04-25T00:32:13.378 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:13.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa491c40640 1 -- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa4780678c0 msgr2=0x7fa478069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:13.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa491c40640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa4780678c0 0x7fa478069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa48c164c20 tx=0x7fa47c008040 comp rx=0 tx=0).stop 2024-04-25T00:32:13.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa491c40640 1 -- 172.21.15.29:0/4117247735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa48c0b4e00 msgr2=0x7fa48c155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:13.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa491c40640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa48c0b4e00 0x7fa48c155790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa480007e10 tx=0x7fa48000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa491c40640 1 -- 172.21.15.29:0/4117247735 shutdown_connections 2024-04-25T00:32:13.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa491c40640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48c162420 0x7fa48c1fb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa491c40640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa48c0b4e00 0x7fa48c155790 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa491c40640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa4780678c0 0x7fa478069d80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa491c40640 1 --2- 172.21.15.29:0/4117247735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa48c0b30b0 0x7fa48c160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa491c40640 1 -- 172.21.15.29:0/4117247735 >> 172.21.15.29:0/4117247735 conn(0x7fa48c0b1870 msgr2=0x7fa48c0cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:13.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa491c40640 1 -- 172.21.15.29:0/4117247735 shutdown_connections 2024-04-25T00:32:13.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.375+0000 7fa491c40640 1 -- 172.21.15.29:0/4117247735 wait complete. 2024-04-25T00:32:13.397 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:32:13.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.519+0000 7f840c564640 1 Processor -- start 2024-04-25T00:32:13.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.519+0000 7f840c564640 1 -- start start 2024-04-25T00:32:13.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.519+0000 7f840c564640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84040b4780 0x7f840415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.519+0000 7f840c564640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f84040cc240 0x7f840415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.519+0000 7f840c564640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84040b3720 con 0x7f840415e8d0 2024-04-25T00:32:13.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.519+0000 7f840c564640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f84040b42f0 con 0x7f84040cc240 2024-04-25T00:32:13.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.519+0000 7f840c564640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f84040b39c0 con 0x7f84040b4780 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.519+0000 7f840a300640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f840415e8d0 0x7f840415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35220/0 (socket says 172.21.15.29:35220) 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.519+0000 7f840a300640 1 -- 172.21.15.29:0/3849902294 learned_addr learned my addr 172.21.15.29:0/3849902294 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f840ab01640 1 --2- 172.21.15.29:0/3849902294 >> v2:172.21.15.157:3300/0 conn(0x7f84040cc240 0x7f840415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f8409aff640 1 --2- 172.21.15.29:0/3849902294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84040b4780 0x7f840415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2463918330 0 0) 0x7f84040b3720 con 0x7f840415e8d0 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83f0003530 con 0x7f840415e8d0 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 877393003 0 0) 0x7f83f0003530 con 0x7f840415e8d0 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84040b4780 msgr2=0x7f840415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 --2- 172.21.15.29:0/3849902294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84040b4780 0x7f840415f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 >> v2:172.21.15.157:3300/0 conn(0x7f84040cc240 msgr2=0x7f840415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 --2- 172.21.15.29:0/3849902294 >> v2:172.21.15.157:3300/0 conn(0x7f84040cc240 0x7f840415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84040b3ca0 con 0x7f840415e8d0 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8400001680 con 0x7f840415e8d0 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 >> v1:172.21.15.29:6789/0 conn(0x7f840415e8d0 legacy=0x7f840415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 --2- 172.21.15.29:0/3849902294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83f00039c0 0x7f83f0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 --2- 172.21.15.29:0/3849902294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84040b4780 0x7f83f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 --2- 172.21.15.29:0/3849902294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f0004d50 0x7f83f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f84040b42f0 con 0x7f83f00039c0 2024-04-25T00:32:13.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f84040b39c0 con 0x7f83f0004d50 2024-04-25T00:32:13.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83f000b9d0 con 0x7f84040b4780 2024-04-25T00:32:13.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f840a300640 1 --2- 172.21.15.29:0/3849902294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83f00039c0 0x7f83f0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f8409aff640 1 --2- 172.21.15.29:0/3849902294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84040b4780 0x7f83f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f840ab01640 1 --2- 172.21.15.29:0/3849902294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f0004d50 0x7f83f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f840a300640 1 -- 172.21.15.29:0/3849902294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84040b4780 msgr2=0x7f83f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:13.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f840a300640 1 --2- 172.21.15.29:0/3849902294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84040b4780 0x7f83f0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f840a300640 1 -- 172.21.15.29:0/3849902294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f0004d50 msgr2=0x7f83f00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:13.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f840a300640 1 --2- 172.21.15.29:0/3849902294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f0004d50 0x7f83f00051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f840a300640 1 -- 172.21.15.29:0/3849902294 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f0003530 con 0x7f83f00039c0 2024-04-25T00:32:13.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f840a300640 1 --2- 172.21.15.29:0/3849902294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83f00039c0 0x7f83f0003de0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f840000c210 tx=0x7f840001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9d2ec49261c9cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:13.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400007d60 con 0x7f83f00039c0 2024-04-25T00:32:13.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84000a9020 con 0x7f83f00039c0 2024-04-25T00:32:13.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f84092fe640 1 -- 172.21.15.29:0/3849902294 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400005b20 con 0x7f83f00039c0 2024-04-25T00:32:13.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f840c564640 1 -- 172.21.15.29:0/3849902294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83f00039c0 msgr2=0x7f83f0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:13.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f840c564640 1 --2- 172.21.15.29:0/3849902294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83f00039c0 0x7f83f0003de0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f840000c210 tx=0x7f840001cca0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.523+0000 7f840ab01640 1 -- 172.21.15.29:0/3849902294 reap_dead start 2024-04-25T00:32:13.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 -- 172.21.15.29:0/3849902294 shutdown_connections 2024-04-25T00:32:13.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 -- 172.21.15.29:0/3849902294 >> 172.21.15.29:0/3849902294 conn(0x7f84040b1870 msgr2=0x7f84040b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:13.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 -- 172.21.15.29:0/3849902294 shutdown_connections 2024-04-25T00:32:13.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 -- 172.21.15.29:0/3849902294 wait complete. 2024-04-25T00:32:13.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 Processor -- start 2024-04-25T00:32:13.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 -- start start 2024-04-25T00:32:13.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f00039c0 0x7f84041429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84041412d0 0x7f840413b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8404143430 0x7f840406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f840415d6a0 con 0x7f8404143430 2024-04-25T00:32:13.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f840415d520 con 0x7f83f00039c0 2024-04-25T00:32:13.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8404165c60 con 0x7f84041412d0 2024-04-25T00:32:13.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840ab01640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8404143430 0x7f840406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840ab01640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8404143430 0x7f840406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40608/0 (socket says 172.21.15.29:40608) 2024-04-25T00:32:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840ab01640 1 -- 172.21.15.29:0/3648130254 learned_addr learned my addr 172.21.15.29:0/3648130254 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840a300640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f00039c0 0x7f84041429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f8409aff640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84041412d0 0x7f840413b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840ab01640 1 -- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84041412d0 msgr2=0x7f840413b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840ab01640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84041412d0 0x7f840413b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840ab01640 1 -- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f00039c0 msgr2=0x7f84041429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840ab01640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f00039c0 0x7f84041429c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840ab01640 1 -- 172.21.15.29:0/3648130254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840413b8f0 con 0x7f8404143430 2024-04-25T00:32:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840ab01640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8404143430 0x7f840406d5d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f840415f9a0 tx=0x7f83fc00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f83faffd640 1 -- 172.21.15.29:0/3648130254 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc016620 con 0x7f8404143430 2024-04-25T00:32:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f83faffd640 1 -- 172.21.15.29:0/3648130254 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83fc01b070 con 0x7f8404143430 2024-04-25T00:32:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 -- 172.21.15.29:0/3648130254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8404141920 con 0x7f8404143430 2024-04-25T00:32:13.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f83faffd640 1 -- 172.21.15.29:0/3648130254 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc016a20 con 0x7f8404143430 2024-04-25T00:32:13.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 -- 172.21.15.29:0/3648130254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8404141ee0 con 0x7f8404143430 2024-04-25T00:32:13.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.527+0000 7f840c564640 1 -- 172.21.15.29:0/3648130254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84041416b0 con 0x7f8404143430 2024-04-25T00:32:13.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.531+0000 7f83faffd640 1 -- 172.21.15.29:0/3648130254 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f83fc016bc0 con 0x7f8404143430 2024-04-25T00:32:13.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.531+0000 7f83faffd640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f83e80678c0 0x7f83e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:13.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.531+0000 7f83faffd640 1 -- 172.21.15.29:0/3648130254 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f83fc0866c0 con 0x7f8404143430 2024-04-25T00:32:13.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.531+0000 7f840a300640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f83e80678c0 0x7f83e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:13.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.531+0000 7f840a300640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f83e80678c0 0x7f83e8069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f84000aa000 tx=0x7f8400000ff0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:13.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.535+0000 7f83faffd640 1 -- 172.21.15.29:0/3648130254 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f83fc024020 con 0x7f8404143430 2024-04-25T00:32:13.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.711+0000 7f840c564640 1 -- 172.21.15.29:0/3648130254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8404069270 con 0x7f8404143430 2024-04-25T00:32:13.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.711+0000 7f83faffd640 1 -- 172.21.15.29:0/3648130254 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f83fc0571f0 con 0x7f8404143430 2024-04-25T00:32:13.715 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:32:13.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.711+0000 7f840c564640 1 -- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f83e80678c0 msgr2=0x7f83e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:13.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.715+0000 7f840c564640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f83e80678c0 0x7f83e8069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f84000aa000 tx=0x7f8400000ff0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.715+0000 7f840c564640 1 -- 172.21.15.29:0/3648130254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8404143430 msgr2=0x7f840406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:13.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.715+0000 7f840c564640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8404143430 0x7f840406d5d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f840415f9a0 tx=0x7f83fc00f520 comp rx=0 tx=0).stop 2024-04-25T00:32:13.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.715+0000 7f840c564640 1 -- 172.21.15.29:0/3648130254 shutdown_connections 2024-04-25T00:32:13.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.715+0000 7f840c564640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8404143430 0x7f840406d5d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.715+0000 7f840c564640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84041412d0 0x7f840413b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.715+0000 7f840c564640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f83e80678c0 0x7f83e8069d80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.715+0000 7f840c564640 1 --2- 172.21.15.29:0/3648130254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f00039c0 0x7f84041429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:13.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.715+0000 7f840c564640 1 -- 172.21.15.29:0/3648130254 >> 172.21.15.29:0/3648130254 conn(0x7f84040b1870 msgr2=0x7f8404165310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:13.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.715+0000 7f840c564640 1 -- 172.21.15.29:0/3648130254 shutdown_connections 2024-04-25T00:32:13.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:13.715+0000 7f840c564640 1 -- 172.21.15.29:0/3648130254 wait complete. 2024-04-25T00:32:13.734 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-25T00:32:13.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:13.734 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T00:32:13.757 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:32:13.757 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:32:13.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:13.758 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:32:13.815 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:32:13.815 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:13.822 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:32:13.822 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:13.870 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:14.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.031+0000 7fe3ad452640 1 Processor -- start 2024-04-25T00:32:14.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.031+0000 7fe3ad452640 1 -- start start 2024-04-25T00:32:14.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3ad452640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe3a80b4780 0x7fe3a815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3ad452640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe3a80cc240 0x7fe3a815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3ad452640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a80b3720 con 0x7fe3a815e8d0 2024-04-25T00:32:14.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3ad452640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe3a80cc650 con 0x7fe3a80cc240 2024-04-25T00:32:14.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3ad452640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe3a80b42f0 con 0x7fe3a80b4780 2024-04-25T00:32:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3a77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe3a80cc240 0x7fe3a815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3a77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe3a80cc240 0x7fe3a815c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40616/0 (socket says 172.21.15.29:40616) 2024-04-25T00:32:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3a77fe640 1 -- 172.21.15.29:0/3545580136 learned_addr learned my addr 172.21.15.29:0/3545580136 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3a67fc640 1 --2- 172.21.15.29:0/3545580136 >> v2:172.21.15.157:3300/0 conn(0x7fe3a80b4780 0x7fe3a815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3a77fe640 1 -- 172.21.15.29:0/3545580136 >> v2:172.21.15.157:3300/0 conn(0x7fe3a80b4780 msgr2=0x7fe3a815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3a77fe640 1 --2- 172.21.15.29:0/3545580136 >> v2:172.21.15.157:3300/0 conn(0x7fe3a80b4780 0x7fe3a815f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3a77fe640 1 -- 172.21.15.29:0/3545580136 >> v1:172.21.15.157:6789/0 conn(0x7fe3a815e8d0 legacy=0x7fe3a815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3a77fe640 1 -- 172.21.15.29:0/3545580136 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a80b39c0 con 0x7fe3a80cc240 2024-04-25T00:32:14.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3a77fe640 1 --2- 172.21.15.29:0/3545580136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3a80cc240 0x7fe3a815c7d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8070910 tx=0x7fe3a001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d0d8a9d93f5e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:14.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3a5ffb640 1 -- 172.21.15.29:0/3545580136 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a00aa610 con 0x7fe3a80cc240 2024-04-25T00:32:14.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3a5ffb640 1 -- 172.21.15.29:0/3545580136 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a00af070 con 0x7fe3a80cc240 2024-04-25T00:32:14.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.035+0000 7fe3a5ffb640 1 -- 172.21.15.29:0/3545580136 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a00aa9d0 con 0x7fe3a80cc240 2024-04-25T00:32:14.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 -- 172.21.15.29:0/3545580136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3a80cc240 msgr2=0x7fe3a815c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 --2- 172.21.15.29:0/3545580136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3a80cc240 0x7fe3a815c7d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8070910 tx=0x7fe3a001c4b0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 -- 172.21.15.29:0/3545580136 shutdown_connections 2024-04-25T00:32:14.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 --2- 172.21.15.29:0/3545580136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3a80cc240 0x7fe3a815c7d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 --2- 172.21.15.29:0/3545580136 >> v2:172.21.15.157:3300/0 conn(0x7fe3a80b4780 0x7fe3a815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 -- 172.21.15.29:0/3545580136 >> 172.21.15.29:0/3545580136 conn(0x7fe3a80b1870 msgr2=0x7fe3a80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:14.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 -- 172.21.15.29:0/3545580136 shutdown_connections 2024-04-25T00:32:14.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 -- 172.21.15.29:0/3545580136 wait complete. 2024-04-25T00:32:14.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 Processor -- start 2024-04-25T00:32:14.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 -- start start 2024-04-25T00:32:14.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a80b4780 0x7fe3a81fa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3a80cc240 0x7fe3a8143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a815e8d0 0x7fe3a8143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8144a20 con 0x7fe3a80cc240 2024-04-25T00:32:14.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a815d520 con 0x7fe3a80b4780 2024-04-25T00:32:14.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a8071f40 con 0x7fe3a815e8d0 2024-04-25T00:32:14.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3a67fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3a80cc240 0x7fe3a8143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3a67fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3a80cc240 0x7fe3a8143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40630/0 (socket says 172.21.15.29:40630) 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3a67fc640 1 -- 172.21.15.29:0/3739525966 learned_addr learned my addr 172.21.15.29:0/3739525966 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3a77fe640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a815e8d0 0x7fe3a8143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3a6ffd640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a80b4780 0x7fe3a81fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3a67fc640 1 -- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a815e8d0 msgr2=0x7fe3a8143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3a67fc640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a815e8d0 0x7fe3a8143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3a67fc640 1 -- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a80b4780 msgr2=0x7fe3a81fa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3a67fc640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a80b4780 0x7fe3a81fa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3a67fc640 1 -- 172.21.15.29:0/3739525966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a8072220 con 0x7fe3a80cc240 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3a6ffd640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a80b4780 0x7fe3a81fa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3a77fe640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a815e8d0 0x7fe3a8143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3a67fc640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3a80cc240 0x7fe3a8143860 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe39c007ec0 tx=0x7fe39c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe38f7fe640 1 -- 172.21.15.29:0/3739525966 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c014620 con 0x7fe3a80cc240 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe38f7fe640 1 -- 172.21.15.29:0/3739525966 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe39c019070 con 0x7fe3a80cc240 2024-04-25T00:32:14.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe38f7fe640 1 -- 172.21.15.29:0/3739525966 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c014a20 con 0x7fe3a80cc240 2024-04-25T00:32:14.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 -- 172.21.15.29:0/3739525966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a8072500 con 0x7fe3a80cc240 2024-04-25T00:32:14.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.039+0000 7fe3ad452640 1 -- 172.21.15.29:0/3739525966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a8072ac0 con 0x7fe3a80cc240 2024-04-25T00:32:14.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.043+0000 7fe3ad452640 1 -- 172.21.15.29:0/3739525966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a8072cc0 con 0x7fe3a80cc240 2024-04-25T00:32:14.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.043+0000 7fe38f7fe640 1 -- 172.21.15.29:0/3739525966 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe39c014bc0 con 0x7fe3a80cc240 2024-04-25T00:32:14.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.043+0000 7fe38f7fe640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe3940678f0 0x7fe394069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.043+0000 7fe38f7fe640 1 -- 172.21.15.29:0/3739525966 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe39c0865e0 con 0x7fe3a80cc240 2024-04-25T00:32:14.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.043+0000 7fe3a6ffd640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe3940678f0 0x7fe394069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.043+0000 7fe3a6ffd640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe3940678f0 0x7fe394069db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe39800a850 tx=0x7fe398008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:14.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.047+0000 7fe38f7fe640 1 -- 172.21.15.29:0/3739525966 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe3a8072cc0 con 0x7fe3a80cc240 2024-04-25T00:32:14.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.211+0000 7fe3ad452640 1 -- 172.21.15.29:0/3739525966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3a8069270 con 0x7fe3a80cc240 2024-04-25T00:32:14.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.211+0000 7fe38f7fe640 1 -- 172.21.15.29:0/3739525966 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fe39c057110 con 0x7fe3a80cc240 2024-04-25T00:32:14.217 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:14.217 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:14.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.215+0000 7fe3ad452640 1 -- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe3940678f0 msgr2=0x7fe394069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.215+0000 7fe3ad452640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe3940678f0 0x7fe394069db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe39800a850 tx=0x7fe398008040 comp rx=0 tx=0).stop 2024-04-25T00:32:14.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.215+0000 7fe3ad452640 1 -- 172.21.15.29:0/3739525966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3a80cc240 msgr2=0x7fe3a8143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.215+0000 7fe3ad452640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3a80cc240 0x7fe3a8143860 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe39c007ec0 tx=0x7fe39c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.215+0000 7fe3ad452640 1 -- 172.21.15.29:0/3739525966 shutdown_connections 2024-04-25T00:32:14.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.215+0000 7fe3ad452640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a815e8d0 0x7fe3a8143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.215+0000 7fe3ad452640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3a80cc240 0x7fe3a8143860 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.215+0000 7fe3ad452640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe3940678f0 0x7fe394069db0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.215+0000 7fe3ad452640 1 --2- 172.21.15.29:0/3739525966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a80b4780 0x7fe3a81fa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.215+0000 7fe3ad452640 1 -- 172.21.15.29:0/3739525966 >> 172.21.15.29:0/3739525966 conn(0x7fe3a80b1870 msgr2=0x7fe3a813e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:14.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.215+0000 7fe3ad452640 1 -- 172.21.15.29:0/3739525966 shutdown_connections 2024-04-25T00:32:14.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.215+0000 7fe3ad452640 1 -- 172.21.15.29:0/3739525966 wait complete. 2024-04-25T00:32:14.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:14.236 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:14.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7ef2061640 1 Processor -- start 2024-04-25T00:32:14.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7ef2061640 1 -- start start 2024-04-25T00:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7ef2061640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7eec0b3140 0x7f7eec06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7ef2061640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7eec0b3c30 0x7f7eec0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7ef2061640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eec0b41e0 con 0x7f7eec0b4b10 2024-04-25T00:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7ef2061640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7eec06d960 con 0x7f7eec0b3140 2024-04-25T00:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7ef2061640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7eec06dc40 con 0x7f7eec0b3c30 2024-04-25T00:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7eeb7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7eec0b3140 0x7f7eec06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7eeaffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7eec0b4b10 0x7f7eec0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35224/0 (socket says 172.21.15.29:35224) 2024-04-25T00:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7eeaffd640 1 -- 172.21.15.29:0/1810143981 learned_addr learned my addr 172.21.15.29:0/1810143981 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7eebfff640 1 --2- 172.21.15.29:0/1810143981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7eec0b3c30 0x7f7eec0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7eeb7fe640 1 -- 172.21.15.29:0/1810143981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7eec0b3c30 msgr2=0x7f7eec0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7eeb7fe640 1 --2- 172.21.15.29:0/1810143981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7eec0b3c30 0x7f7eec0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7eeb7fe640 1 -- 172.21.15.29:0/1810143981 >> v1:172.21.15.29:6789/0 conn(0x7f7eec0b4b10 legacy=0x7f7eec0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.363+0000 7f7eeb7fe640 1 -- 172.21.15.29:0/1810143981 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec06df20 con 0x7f7eec0b3140 2024-04-25T00:32:14.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.367+0000 7f7eeb7fe640 1 --2- 172.21.15.29:0/1810143981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec0b3140 0x7f7eec06e440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40097f0 tx=0x7f7ee401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892b2c36be1d037f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:14.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.367+0000 7f7eea7fc640 1 -- 172.21.15.29:0/1810143981 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee40b0070 con 0x7f7eec0b3140 2024-04-25T00:32:14.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.367+0000 7f7eea7fc640 1 -- 172.21.15.29:0/1810143981 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee40ab600 con 0x7f7eec0b3140 2024-04-25T00:32:14.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.367+0000 7f7eea7fc640 1 -- 172.21.15.29:0/1810143981 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee40ab9d0 con 0x7f7eec0b3140 2024-04-25T00:32:14.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.367+0000 7f7ef2061640 1 -- 172.21.15.29:0/1810143981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec0b3140 msgr2=0x7f7eec06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.367+0000 7f7ef2061640 1 --2- 172.21.15.29:0/1810143981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec0b3140 0x7f7eec06e440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40097f0 tx=0x7f7ee401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.367+0000 7f7ef2061640 1 -- 172.21.15.29:0/1810143981 shutdown_connections 2024-04-25T00:32:14.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.367+0000 7f7ef2061640 1 --2- 172.21.15.29:0/1810143981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7eec0b3c30 0x7f7eec0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.367+0000 7f7ef2061640 1 --2- 172.21.15.29:0/1810143981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec0b3140 0x7f7eec06e440 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.367+0000 7f7ef2061640 1 -- 172.21.15.29:0/1810143981 >> 172.21.15.29:0/1810143981 conn(0x7f7eec0b1870 msgr2=0x7f7eec0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:14.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.367+0000 7f7ef2061640 1 -- 172.21.15.29:0/1810143981 shutdown_connections 2024-04-25T00:32:14.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.367+0000 7f7ef2061640 1 -- 172.21.15.29:0/1810143981 wait complete. 2024-04-25T00:32:14.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ef2061640 1 Processor -- start 2024-04-25T00:32:14.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ef2061640 1 -- start start 2024-04-25T00:32:14.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ef2061640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7eec0b3140 0x7f7eec138cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ef2061640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec0b3c30 0x7f7eec13d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ef2061640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec0b4b10 0x7f7eec13a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ef2061640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec168d80 con 0x7f7eec0b4b10 2024-04-25T00:32:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ef2061640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec168c00 con 0x7f7eec0b3c30 2024-04-25T00:32:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ef2061640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7eec168f00 con 0x7f7eec0b3140 2024-04-25T00:32:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7eebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec0b4b10 0x7f7eec13a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7eebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec0b4b10 0x7f7eec13a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40648/0 (socket says 172.21.15.29:40648) 2024-04-25T00:32:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7eebfff640 1 -- 172.21.15.29:0/462131011 learned_addr learned my addr 172.21.15.29:0/462131011 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7eeb7fe640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7eec0b3140 0x7f7eec138cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7eeaffd640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec0b3c30 0x7f7eec13d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7eebfff640 1 -- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7eec0b3140 msgr2=0x7f7eec138cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7eebfff640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7eec0b3140 0x7f7eec138cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7eebfff640 1 -- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec0b3c30 msgr2=0x7f7eec13d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7eebfff640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec0b3c30 0x7f7eec13d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7eebfff640 1 -- 172.21.15.29:0/462131011 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec13d960 con 0x7f7eec0b4b10 2024-04-25T00:32:14.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7eeaffd640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec0b3c30 0x7f7eec13d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:14.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7eebfff640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec0b4b10 0x7f7eec13a4a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7edc007e40 tx=0x7f7edc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:14.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ee3fff640 1 -- 172.21.15.29:0/462131011 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc014620 con 0x7f7eec0b4b10 2024-04-25T00:32:14.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ee3fff640 1 -- 172.21.15.29:0/462131011 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7edc019070 con 0x7f7eec0b4b10 2024-04-25T00:32:14.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ef2061640 1 -- 172.21.15.29:0/462131011 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eec139400 con 0x7f7eec0b4b10 2024-04-25T00:32:14.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ee3fff640 1 -- 172.21.15.29:0/462131011 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc014a20 con 0x7f7eec0b4b10 2024-04-25T00:32:14.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ef2061640 1 -- 172.21.15.29:0/462131011 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eec1399c0 con 0x7f7eec0b4b10 2024-04-25T00:32:14.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ef2061640 1 -- 172.21.15.29:0/462131011 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eec069270 con 0x7f7eec0b4b10 2024-04-25T00:32:14.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ee3fff640 1 -- 172.21.15.29:0/462131011 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7edc014bc0 con 0x7f7eec0b4b10 2024-04-25T00:32:14.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.371+0000 7f7ee3fff640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7ecc0678f0 0x7f7ecc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.375+0000 7f7ee3fff640 1 -- 172.21.15.29:0/462131011 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7edc0864c0 con 0x7f7eec0b4b10 2024-04-25T00:32:14.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.375+0000 7f7eeb7fe640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7ecc0678f0 0x7f7ecc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.375+0000 7f7eeb7fe640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7ecc0678f0 0x7f7ecc069db0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40ad000 tx=0x7f7ee4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:14.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.379+0000 7f7ee3fff640 1 -- 172.21.15.29:0/462131011 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7edc057090 con 0x7f7eec0b4b10 2024-04-25T00:32:14.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ef2061640 1 -- 172.21.15.29:0/462131011 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7eec0cdd40 con 0x7f7eec0b4b10 2024-04-25T00:32:14.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ee3fff640 1 -- 172.21.15.29:0/462131011 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f7edc056990 con 0x7f7eec0b4b10 2024-04-25T00:32:14.546 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:14.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ef2061640 1 -- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7ecc0678f0 msgr2=0x7f7ecc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ef2061640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7ecc0678f0 0x7f7ecc069db0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40ad000 tx=0x7f7ee4008040 comp rx=0 tx=0).stop 2024-04-25T00:32:14.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ef2061640 1 -- 172.21.15.29:0/462131011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec0b4b10 msgr2=0x7f7eec13a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ef2061640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec0b4b10 0x7f7eec13a4a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7edc007e40 tx=0x7f7edc00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ef2061640 1 -- 172.21.15.29:0/462131011 shutdown_connections 2024-04-25T00:32:14.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ef2061640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eec0b4b10 0x7f7eec13a4a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ef2061640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7ecc0678f0 0x7f7ecc069db0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ef2061640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec0b3c30 0x7f7eec13d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ef2061640 1 --2- 172.21.15.29:0/462131011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7eec0b3140 0x7f7eec138cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ef2061640 1 -- 172.21.15.29:0/462131011 >> 172.21.15.29:0/462131011 conn(0x7f7eec0b1870 msgr2=0x7f7eec142d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:14.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ef2061640 1 -- 172.21.15.29:0/462131011 shutdown_connections 2024-04-25T00:32:14.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.543+0000 7f7ef2061640 1 -- 172.21.15.29:0/462131011 wait complete. 2024-04-25T00:32:14.566 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:14.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f570607a640 1 Processor -- start 2024-04-25T00:32:14.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f570607a640 1 -- start start 2024-04-25T00:32:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f570607a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f570006d320 0x7f57000d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f570607a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f570006c530 0x7f57000cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f570607a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f570015e7a0 con 0x7f570015e3c0 2024-04-25T00:32:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f570607a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f57000b4bc0 con 0x7f570006c530 2024-04-25T00:32:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f570607a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f57000d10f0 con 0x7f570006d320 2024-04-25T00:32:14.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f56ff7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f570015e3c0 0x7f570006cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35236/0 (socket says 172.21.15.29:35236) 2024-04-25T00:32:14.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f56ff7fe640 1 -- 172.21.15.29:0/1489623604 learned_addr learned my addr 172.21.15.29:0/1489623604 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:14.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f56fffff640 1 --2- 172.21.15.29:0/1489623604 >> v2:172.21.15.29:3300/0 conn(0x7f570006c530 0x7f57000cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f56feffd640 1 --2- 172.21.15.29:0/1489623604 >> v2:172.21.15.157:3300/0 conn(0x7f570006d320 0x7f57000d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f56fe7fc640 1 -- 172.21.15.29:0/1489623604 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 945888000 0 0) 0x7f570015e7a0 con 0x7f570015e3c0 2024-04-25T00:32:14.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f56fe7fc640 1 -- 172.21.15.29:0/1489623604 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56e8003530 con 0x7f570015e3c0 2024-04-25T00:32:14.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f56fffff640 1 -- 172.21.15.29:0/1489623604 >> v2:172.21.15.157:3300/0 conn(0x7f570006d320 msgr2=0x7f57000d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:14.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.687+0000 7f56fffff640 1 --2- 172.21.15.29:0/1489623604 >> v2:172.21.15.157:3300/0 conn(0x7f570006d320 0x7f57000d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f56fffff640 1 -- 172.21.15.29:0/1489623604 >> v1:172.21.15.29:6789/0 conn(0x7f570015e3c0 legacy=0x7f570006cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f56fffff640 1 -- 172.21.15.29:0/1489623604 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57000cc560 con 0x7f570006c530 2024-04-25T00:32:14.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f56fffff640 1 --2- 172.21.15.29:0/1489623604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f570006c530 0x7f57000cd040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5700071fd0 tx=0x7f56f801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b087dddc915930 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:14.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f56fe7fc640 1 -- 172.21.15.29:0/1489623604 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f80b2070 con 0x7f570006c530 2024-04-25T00:32:14.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f56fe7fc640 1 -- 172.21.15.29:0/1489623604 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f80a58e0 con 0x7f570006c530 2024-04-25T00:32:14.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f56fe7fc640 1 -- 172.21.15.29:0/1489623604 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f80a5cb0 con 0x7f570006c530 2024-04-25T00:32:14.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f570607a640 1 -- 172.21.15.29:0/1489623604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f570006c530 msgr2=0x7f57000cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f570607a640 1 --2- 172.21.15.29:0/1489623604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f570006c530 0x7f57000cd040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5700071fd0 tx=0x7f56f801c930 comp rx=0 tx=0).stop 2024-04-25T00:32:14.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f570607a640 1 -- 172.21.15.29:0/1489623604 shutdown_connections 2024-04-25T00:32:14.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f570607a640 1 --2- 172.21.15.29:0/1489623604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f570006c530 0x7f57000cd040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f570607a640 1 --2- 172.21.15.29:0/1489623604 >> v2:172.21.15.157:3300/0 conn(0x7f570006d320 0x7f57000d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f570607a640 1 -- 172.21.15.29:0/1489623604 >> 172.21.15.29:0/1489623604 conn(0x7f57000d1d70 msgr2=0x7f57000d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:14.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f570607a640 1 -- 172.21.15.29:0/1489623604 shutdown_connections 2024-04-25T00:32:14.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.691+0000 7f570607a640 1 -- 172.21.15.29:0/1489623604 wait complete. 2024-04-25T00:32:14.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f570607a640 1 Processor -- start 2024-04-25T00:32:14.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f570607a640 1 -- start start 2024-04-25T00:32:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f570607a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f570006c530 0x7f57001491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f570607a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f570006d320 0x7f570015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f570607a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f570015e3c0 0x7f570015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f570607a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f57000cdf10 con 0x7f570006c530 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f570607a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57000cdd90 con 0x7f570015e3c0 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f570607a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f57000ce090 con 0x7f570006d320 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56ff7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f570006c530 0x7f57001491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56ff7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f570006c530 0x7f57001491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40664/0 (socket says 172.21.15.29:40664) 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56ff7fe640 1 -- 172.21.15.29:0/1619786595 learned_addr learned my addr 172.21.15.29:0/1619786595 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56fffff640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f570015e3c0 0x7f570015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56feffd640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f570006d320 0x7f570015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56ff7fe640 1 -- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f570006d320 msgr2=0x7f570015cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56ff7fe640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f570006d320 0x7f570015cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56ff7fe640 1 -- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f570015e3c0 msgr2=0x7f570015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56ff7fe640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f570015e3c0 0x7f570015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56ff7fe640 1 -- 172.21.15.29:0/1619786595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570014a360 con 0x7f570006c530 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56ff7fe640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f570006c530 0x7f57001491e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f56f000c480 tx=0x7f56f000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56e7fff640 1 -- 172.21.15.29:0/1619786595 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0017070 con 0x7f570006c530 2024-04-25T00:32:14.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56e7fff640 1 -- 172.21.15.29:0/1619786595 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f0005cf0 con 0x7f570006c530 2024-04-25T00:32:14.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f56e7fff640 1 -- 172.21.15.29:0/1619786595 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0007290 con 0x7f570006c530 2024-04-25T00:32:14.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f570607a640 1 -- 172.21.15.29:0/1619786595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57001616a0 con 0x7f570006c530 2024-04-25T00:32:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f570607a640 1 -- 172.21.15.29:0/1619786595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f570014a8e0 con 0x7f570006c530 2024-04-25T00:32:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.695+0000 7f570607a640 1 -- 172.21.15.29:0/1619786595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5700069270 con 0x7f570006c530 2024-04-25T00:32:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.699+0000 7f56e7fff640 1 -- 172.21.15.29:0/1619786595 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f56f001a070 con 0x7f570006c530 2024-04-25T00:32:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.699+0000 7f56e7fff640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f56ec0678c0 0x7f56ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.699+0000 7f56e7fff640 1 -- 172.21.15.29:0/1619786595 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f56f0085400 con 0x7f570006c530 2024-04-25T00:32:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.699+0000 7f56feffd640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f56ec0678c0 0x7f56ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.699+0000 7f56feffd640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f56ec0678c0 0x7f56ec069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f57000d2a00 tx=0x7f56f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.703+0000 7f56e7fff640 1 -- 172.21.15.29:0/1619786595 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f56f0021020 con 0x7f570006c530 2024-04-25T00:32:14.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.863+0000 7f570607a640 1 -- 172.21.15.29:0/1619786595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f570015e7a0 con 0x7f570006c530 2024-04-25T00:32:14.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.863+0000 7f56e7fff640 1 -- 172.21.15.29:0/1619786595 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f56f00560b0 con 0x7f570006c530 2024-04-25T00:32:14.867 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:14.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.863+0000 7f570607a640 1 -- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f56ec0678c0 msgr2=0x7f56ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.863+0000 7f570607a640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f56ec0678c0 0x7f56ec069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f57000d2a00 tx=0x7f56f4008040 comp rx=0 tx=0).stop 2024-04-25T00:32:14.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.863+0000 7f570607a640 1 -- 172.21.15.29:0/1619786595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f570006c530 msgr2=0x7f57001491e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:14.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.863+0000 7f570607a640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f570006c530 0x7f57001491e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f56f000c480 tx=0x7f56f000c940 comp rx=0 tx=0).stop 2024-04-25T00:32:14.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.863+0000 7f570607a640 1 -- 172.21.15.29:0/1619786595 shutdown_connections 2024-04-25T00:32:14.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.863+0000 7f570607a640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f56ec0678c0 0x7f56ec069d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.863+0000 7f570607a640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f570015e3c0 0x7f570015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.863+0000 7f570607a640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f570006d320 0x7f570015cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.863+0000 7f570607a640 1 --2- 172.21.15.29:0/1619786595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f570006c530 0x7f57001491e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:14.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.863+0000 7f570607a640 1 -- 172.21.15.29:0/1619786595 >> 172.21.15.29:0/1619786595 conn(0x7f57000d1d70 msgr2=0x7f57000cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:14.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.863+0000 7f570607a640 1 -- 172.21.15.29:0/1619786595 shutdown_connections 2024-04-25T00:32:14.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:14.867+0000 7f570607a640 1 -- 172.21.15.29:0/1619786595 wait complete. 2024-04-25T00:32:14.886 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:15.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.003+0000 7f66810a2640 1 Processor -- start 2024-04-25T00:32:15.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.003+0000 7f66810a2640 1 -- start start 2024-04-25T00:32:15.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f66810a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667c0b3140 0x7f667c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f66810a2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f667c0b4b10 0x7f667c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f66810a2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f667c0b41e0 con 0x7f667c0b3a60 2024-04-25T00:32:15.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f66810a2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f667c06d960 con 0x7f667c0b4b10 2024-04-25T00:32:15.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f66810a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f667c06dc40 con 0x7f667c0b3140 2024-04-25T00:32:15.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667b59e640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f667c0b3a60 0x7f667c15c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35240/0 (socket says 172.21.15.29:35240) 2024-04-25T00:32:15.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667b59e640 1 -- 172.21.15.29:0/2004296675 learned_addr learned my addr 172.21.15.29:0/2004296675 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:15.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667259c640 1 --2- 172.21.15.29:0/2004296675 >> v2:172.21.15.29:3300/0 conn(0x7f667c0b4b10 0x7f667c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667ad9d640 1 --2- 172.21.15.29:0/2004296675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667c0b3140 0x7f667c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667a59c640 1 -- 172.21.15.29:0/2004296675 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1374848980 0 0) 0x7f667c0b41e0 con 0x7f667c0b3a60 2024-04-25T00:32:15.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667a59c640 1 -- 172.21.15.29:0/2004296675 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6668003530 con 0x7f667c0b3a60 2024-04-25T00:32:15.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667259c640 1 -- 172.21.15.29:0/2004296675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667c0b3140 msgr2=0x7f667c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:15.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667259c640 1 --2- 172.21.15.29:0/2004296675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667c0b3140 0x7f667c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667259c640 1 -- 172.21.15.29:0/2004296675 >> v1:172.21.15.29:6789/0 conn(0x7f667c0b3a60 legacy=0x7f667c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667259c640 1 -- 172.21.15.29:0/2004296675 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c06df20 con 0x7f667c0b4b10 2024-04-25T00:32:15.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667ad9d640 1 --2- 172.21.15.29:0/2004296675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667c0b3140 0x7f667c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:15.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667259c640 1 --2- 172.21.15.29:0/2004296675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f667c0b4b10 0x7f667c0b3520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f66640097c0 tx=0x7f666401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6eeee4d25065a55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667a59c640 1 -- 172.21.15.29:0/2004296675 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66640b0070 con 0x7f667c0b4b10 2024-04-25T00:32:15.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667a59c640 1 -- 172.21.15.29:0/2004296675 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66640ab600 con 0x7f667c0b4b10 2024-04-25T00:32:15.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.007+0000 7f667a59c640 1 -- 172.21.15.29:0/2004296675 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66640ab9d0 con 0x7f667c0b4b10 2024-04-25T00:32:15.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 -- 172.21.15.29:0/2004296675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f667c0b4b10 msgr2=0x7f667c0b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 --2- 172.21.15.29:0/2004296675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f667c0b4b10 0x7f667c0b3520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f66640097c0 tx=0x7f666401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 -- 172.21.15.29:0/2004296675 shutdown_connections 2024-04-25T00:32:15.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 --2- 172.21.15.29:0/2004296675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f667c0b4b10 0x7f667c0b3520 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 --2- 172.21.15.29:0/2004296675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667c0b3140 0x7f667c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 -- 172.21.15.29:0/2004296675 >> 172.21.15.29:0/2004296675 conn(0x7f667c0b1870 msgr2=0x7f667c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:15.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 -- 172.21.15.29:0/2004296675 shutdown_connections 2024-04-25T00:32:15.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 -- 172.21.15.29:0/2004296675 wait complete. 2024-04-25T00:32:15.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 Processor -- start 2024-04-25T00:32:15.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 -- start start 2024-04-25T00:32:15.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f667c0b3140 0x7f667c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667c0b3a60 0x7f667c15e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667c0b4b10 0x7f667c15ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f667c168b70 con 0x7f667c0b3140 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f667c1689f0 con 0x7f667c0b4b10 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f667c168cf0 con 0x7f667c0b3a60 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f667ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f667c0b3140 0x7f667c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f667ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f667c0b3140 0x7f667c136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40686/0 (socket says 172.21.15.29:40686) 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f667ad9d640 1 -- 172.21.15.29:0/1923811771 learned_addr learned my addr 172.21.15.29:0/1923811771 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f667ad9d640 1 -- 172.21.15.29:0/1923811771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667c0b3a60 msgr2=0x7f667c15e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f667ad9d640 1 --2- 172.21.15.29:0/1923811771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667c0b3a60 0x7f667c15e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f667ad9d640 1 -- 172.21.15.29:0/1923811771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667c0b4b10 msgr2=0x7f667c15ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f667ad9d640 1 --2- 172.21.15.29:0/1923811771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667c0b4b10 0x7f667c15ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f667ad9d640 1 -- 172.21.15.29:0/1923811771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c166380 con 0x7f667c0b3140 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f667ad9d640 1 --2- 172.21.15.29:0/1923811771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f667c0b3140 0x7f667c136aa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f666c00d800 tx=0x7f666c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f6673fff640 1 -- 172.21.15.29:0/1923811771 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c018020 con 0x7f667c0b3140 2024-04-25T00:32:15.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f6673fff640 1 -- 172.21.15.29:0/1923811771 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f666c0096c0 con 0x7f667c0b3140 2024-04-25T00:32:15.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f6673fff640 1 -- 172.21.15.29:0/1923811771 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c009ab0 con 0x7f667c0b3140 2024-04-25T00:32:15.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 -- 172.21.15.29:0/1923811771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667c166660 con 0x7f667c0b3140 2024-04-25T00:32:15.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.011+0000 7f66810a2640 1 -- 172.21.15.29:0/1923811771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667c138970 con 0x7f667c0b3140 2024-04-25T00:32:15.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.015+0000 7f66810a2640 1 -- 172.21.15.29:0/1923811771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667c15bc60 con 0x7f667c0b3140 2024-04-25T00:32:15.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.015+0000 7f6673fff640 1 -- 172.21.15.29:0/1923811771 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f666c01b020 con 0x7f667c0b3140 2024-04-25T00:32:15.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.015+0000 7f6673fff640 1 --2- 172.21.15.29:0/1923811771 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f665c0678f0 0x7f665c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.015+0000 7f6673fff640 1 -- 172.21.15.29:0/1923811771 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f666c0861e0 con 0x7f667c0b3140 2024-04-25T00:32:15.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.015+0000 7f667259c640 1 --2- 172.21.15.29:0/1923811771 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f665c0678f0 0x7f665c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.015+0000 7f667259c640 1 --2- 172.21.15.29:0/1923811771 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f665c0678f0 0x7f665c069db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f66640ad000 tx=0x7f6664004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.019+0000 7f6673fff640 1 -- 172.21.15.29:0/1923811771 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f666c022020 con 0x7f667c0b3140 2024-04-25T00:32:15.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f66810a2640 1 -- 172.21.15.29:0/1923811771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f667c15c410 con 0x7f667c0b3140 2024-04-25T00:32:15.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f6673fff640 1 -- 172.21.15.29:0/1923811771 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f666c056d20 con 0x7f667c0b3140 2024-04-25T00:32:15.178 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:15.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f66810a2640 1 -- 172.21.15.29:0/1923811771 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f665c0678f0 msgr2=0x7f665c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f66810a2640 1 --2- 172.21.15.29:0/1923811771 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f665c0678f0 0x7f665c069db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f66640ad000 tx=0x7f6664004740 comp rx=0 tx=0).stop 2024-04-25T00:32:15.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f66810a2640 1 -- 172.21.15.29:0/1923811771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f667c0b3140 msgr2=0x7f667c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f66810a2640 1 --2- 172.21.15.29:0/1923811771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f667c0b3140 0x7f667c136aa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f666c00d800 tx=0x7f666c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f66810a2640 1 -- 172.21.15.29:0/1923811771 shutdown_connections 2024-04-25T00:32:15.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f66810a2640 1 --2- 172.21.15.29:0/1923811771 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f665c0678f0 0x7f665c069db0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f66810a2640 1 --2- 172.21.15.29:0/1923811771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667c0b4b10 0x7f667c15ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f66810a2640 1 --2- 172.21.15.29:0/1923811771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f667c0b3a60 0x7f667c15e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f66810a2640 1 --2- 172.21.15.29:0/1923811771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f667c0b3140 0x7f667c136aa0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f66810a2640 1 -- 172.21.15.29:0/1923811771 >> 172.21.15.29:0/1923811771 conn(0x7f667c0b1870 msgr2=0x7f667c13f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:15.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f66810a2640 1 -- 172.21.15.29:0/1923811771 shutdown_connections 2024-04-25T00:32:15.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.175+0000 7f66810a2640 1 -- 172.21.15.29:0/1923811771 wait complete. 2024-04-25T00:32:15.195 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:32:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f1858fe3640 1 Processor -- start 2024-04-25T00:32:15.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f1858fe3640 1 -- start start 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f1858fe3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f18540b3140 0x7f185406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f1858fe3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18540b3c30 0x7f18540ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f1858fe3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18540b41e0 con 0x7f18540b4b10 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f1858fe3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f185406d960 con 0x7f18540b3140 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f1858fe3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f185406dc40 con 0x7f18540b3c30 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f185259c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f18540b3140 0x7f185406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f1851d9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f18540b4b10 0x7f18540b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35254/0 (socket says 172.21.15.29:35254) 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f1851d9b640 1 -- 172.21.15.29:0/1879505055 learned_addr learned my addr 172.21.15.29:0/1879505055 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f1852d9d640 1 --2- 172.21.15.29:0/1879505055 >> v2:172.21.15.157:3300/0 conn(0x7f18540b3c30 0x7f18540ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f185259c640 1 -- 172.21.15.29:0/1879505055 >> v2:172.21.15.157:3300/0 conn(0x7f18540b3c30 msgr2=0x7f18540ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f185259c640 1 --2- 172.21.15.29:0/1879505055 >> v2:172.21.15.157:3300/0 conn(0x7f18540b3c30 0x7f18540ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f185259c640 1 -- 172.21.15.29:0/1879505055 >> v1:172.21.15.29:6789/0 conn(0x7f18540b4b10 legacy=0x7f18540b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f185259c640 1 -- 172.21.15.29:0/1879505055 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185406df20 con 0x7f18540b3140 2024-04-25T00:32:15.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f185159a640 1 -- 172.21.15.29:0/1879505055 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1622887070 0 0) 0x7f18540b41e0 con 0x7f18540b4b10 2024-04-25T00:32:15.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.315+0000 7f185259c640 1 --2- 172.21.15.29:0/1879505055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18540b3140 0x7f185406e440 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f183c0097c0 tx=0x7f183c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370e17189fe4f25a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f185159a640 1 -- 172.21.15.29:0/1879505055 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c0b0070 con 0x7f18540b3140 2024-04-25T00:32:15.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f185159a640 1 -- 172.21.15.29:0/1879505055 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f183c0ab600 con 0x7f18540b3140 2024-04-25T00:32:15.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f185159a640 1 -- 172.21.15.29:0/1879505055 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c0ab9d0 con 0x7f18540b3140 2024-04-25T00:32:15.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 -- 172.21.15.29:0/1879505055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18540b3140 msgr2=0x7f185406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 --2- 172.21.15.29:0/1879505055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18540b3140 0x7f185406e440 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f183c0097c0 tx=0x7f183c01c460 comp rx=0 tx=0).stop 2024-04-25T00:32:15.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 -- 172.21.15.29:0/1879505055 shutdown_connections 2024-04-25T00:32:15.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 --2- 172.21.15.29:0/1879505055 >> v2:172.21.15.157:3300/0 conn(0x7f18540b3c30 0x7f18540ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 --2- 172.21.15.29:0/1879505055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18540b3140 0x7f185406e440 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 -- 172.21.15.29:0/1879505055 >> 172.21.15.29:0/1879505055 conn(0x7f18540b1870 msgr2=0x7f18540b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:15.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 -- 172.21.15.29:0/1879505055 shutdown_connections 2024-04-25T00:32:15.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 -- 172.21.15.29:0/1879505055 wait complete. 2024-04-25T00:32:15.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 Processor -- start 2024-04-25T00:32:15.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 -- start start 2024-04-25T00:32:15.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18540b3140 0x7f1854156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18540b3c30 0x7f185413f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18540b4b10 0x7f185413a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1854168d80 con 0x7f18540b3140 2024-04-25T00:32:15.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1854168c00 con 0x7f18540b3c30 2024-04-25T00:32:15.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1854168f00 con 0x7f18540b4b10 2024-04-25T00:32:15.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f185259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18540b3140 0x7f1854156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f185259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18540b3140 0x7f1854156cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40690/0 (socket says 172.21.15.29:40690) 2024-04-25T00:32:15.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f185259c640 1 -- 172.21.15.29:0/250904944 learned_addr learned my addr 172.21.15.29:0/250904944 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:15.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1851d9b640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18540b3c30 0x7f185413f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1852d9d640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18540b4b10 0x7f185413a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f185259c640 1 -- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18540b4b10 msgr2=0x7f185413a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f185259c640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18540b4b10 0x7f185413a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f185259c640 1 -- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18540b3c30 msgr2=0x7f185413f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f185259c640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18540b3c30 0x7f185413f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f185259c640 1 -- 172.21.15.29:0/250904944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1854070ce0 con 0x7f18540b3140 2024-04-25T00:32:15.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1852d9d640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18540b4b10 0x7f185413a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:15.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f185259c640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18540b3140 0x7f1854156cd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f183c019070 tx=0x7f183c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f184affd640 1 -- 172.21.15.29:0/250904944 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c0b0050 con 0x7f18540b3140 2024-04-25T00:32:15.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f184affd640 1 -- 172.21.15.29:0/250904944 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f183c0bc070 con 0x7f18540b3140 2024-04-25T00:32:15.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f184affd640 1 -- 172.21.15.29:0/250904944 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c0b5850 con 0x7f18540b3140 2024-04-25T00:32:15.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 -- 172.21.15.29:0/250904944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18541572b0 con 0x7f18540b3140 2024-04-25T00:32:15.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.319+0000 7f1858fe3640 1 -- 172.21.15.29:0/250904944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1854157870 con 0x7f18540b3140 2024-04-25T00:32:15.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.323+0000 7f1858fe3640 1 -- 172.21.15.29:0/250904944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1854069270 con 0x7f18540b3140 2024-04-25T00:32:15.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.323+0000 7f184affd640 1 -- 172.21.15.29:0/250904944 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f183c0b59f0 con 0x7f18540b3140 2024-04-25T00:32:15.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.323+0000 7f184affd640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f18340678c0 0x7f1834069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.323+0000 7f184affd640 1 -- 172.21.15.29:0/250904944 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f183c1204b0 con 0x7f18540b3140 2024-04-25T00:32:15.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.323+0000 7f1851d9b640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f18340678c0 0x7f1834069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.323+0000 7f1851d9b640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f18340678c0 0x7f1834069d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f18540686b0 tx=0x7f1844007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.327+0000 7f184affd640 1 -- 172.21.15.29:0/250904944 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f183c016050 con 0x7f18540b3140 2024-04-25T00:32:15.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.507+0000 7f1858fe3640 1 -- 172.21.15.29:0/250904944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1854142670 con 0x7f18540b3140 2024-04-25T00:32:15.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.507+0000 7f184affd640 1 -- 172.21.15.29:0/250904944 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f183c0f0f50 con 0x7f18540b3140 2024-04-25T00:32:15.512 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:32:15.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.511+0000 7f1858fe3640 1 -- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f18340678c0 msgr2=0x7f1834069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.511+0000 7f1858fe3640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f18340678c0 0x7f1834069d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f18540686b0 tx=0x7f1844007b60 comp rx=0 tx=0).stop 2024-04-25T00:32:15.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.511+0000 7f1858fe3640 1 -- 172.21.15.29:0/250904944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18540b3140 msgr2=0x7f1854156cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.511+0000 7f1858fe3640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18540b3140 0x7f1854156cd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f183c019070 tx=0x7f183c01aca0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.511+0000 7f1858fe3640 1 -- 172.21.15.29:0/250904944 shutdown_connections 2024-04-25T00:32:15.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.511+0000 7f1858fe3640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18540b4b10 0x7f185413a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.511+0000 7f1858fe3640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f18340678c0 0x7f1834069d80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.511+0000 7f1858fe3640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18540b3c30 0x7f185413f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.511+0000 7f1858fe3640 1 --2- 172.21.15.29:0/250904944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18540b3140 0x7f1854156cd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.511+0000 7f1858fe3640 1 -- 172.21.15.29:0/250904944 >> 172.21.15.29:0/250904944 conn(0x7f18540b1870 msgr2=0x7f185413d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:15.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.511+0000 7f1858fe3640 1 -- 172.21.15.29:0/250904944 shutdown_connections 2024-04-25T00:32:15.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.511+0000 7f1858fe3640 1 -- 172.21.15.29:0/250904944 wait complete. 2024-04-25T00:32:15.529 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:32:15.529 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:15.535 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:15.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca81da0640 1 Processor -- start 2024-04-25T00:32:15.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca81da0640 1 -- start start 2024-04-25T00:32:15.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca81da0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fca7c15d980 0x7fca7c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca81da0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca7c0b3130 con 0x7fca7c06db10 2024-04-25T00:32:15.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca81da0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca7c0b3410 con 0x7fca7c0b2b70 2024-04-25T00:32:15.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca81da0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fca7c0cf560 con 0x7fca7c15d980 2024-04-25T00:32:15.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca7b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fca7c15d980 0x7fca7c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca7bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fca7c06db10 0x7fca7c15b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35266/0 (socket says 172.21.15.29:35266) 2024-04-25T00:32:15.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca7bfff640 1 -- 172.21.15.29:0/496309191 learned_addr learned my addr 172.21.15.29:0/496309191 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:15.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca7b7fe640 1 -- 172.21.15.29:0/496309191 >> v1:172.21.15.157:6789/0 conn(0x7fca7c0b2b70 legacy=0x7fca7c150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:15.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca7b7fe640 1 -- 172.21.15.29:0/496309191 >> v1:172.21.15.29:6789/0 conn(0x7fca7c06db10 legacy=0x7fca7c15b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca7b7fe640 1 -- 172.21.15.29:0/496309191 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c0b36f0 con 0x7fca7c15d980 2024-04-25T00:32:15.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca7b7fe640 1 --2- 172.21.15.29:0/496309191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca7c15d980 0x7fca7c06d5d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fca6400ba50 tx=0x7fca6401e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a5cb0836d5db25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca7affd640 1 -- 172.21.15.29:0/496309191 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca640ad070 con 0x7fca7c15d980 2024-04-25T00:32:15.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca7affd640 1 -- 172.21.15.29:0/496309191 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca640a78d0 con 0x7fca7c15d980 2024-04-25T00:32:15.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.703+0000 7fca7affd640 1 -- 172.21.15.29:0/496309191 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca640a7ca0 con 0x7fca7c15d980 2024-04-25T00:32:15.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 -- 172.21.15.29:0/496309191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca7c15d980 msgr2=0x7fca7c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 --2- 172.21.15.29:0/496309191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca7c15d980 0x7fca7c06d5d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fca6400ba50 tx=0x7fca6401e910 comp rx=0 tx=0).stop 2024-04-25T00:32:15.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 -- 172.21.15.29:0/496309191 shutdown_connections 2024-04-25T00:32:15.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 --2- 172.21.15.29:0/496309191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca7c15d980 0x7fca7c06d5d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 -- 172.21.15.29:0/496309191 >> 172.21.15.29:0/496309191 conn(0x7fca7c0b1870 msgr2=0x7fca7c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:15.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 -- 172.21.15.29:0/496309191 shutdown_connections 2024-04-25T00:32:15.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 -- 172.21.15.29:0/496309191 wait complete. 2024-04-25T00:32:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 Processor -- start 2024-04-25T00:32:15.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 -- start start 2024-04-25T00:32:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca7c06db10 0x7fca7c1436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca7c0b2b70 0x7fca7c151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca7c15d980 0x7fca7c152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1683d0 con 0x7fca7c0b2b70 2024-04-25T00:32:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c15dd60 con 0x7fca7c06db10 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca81da0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fca7c168550 con 0x7fca7c15d980 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca72ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca7c0b2b70 0x7fca7c151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca72ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca7c0b2b70 0x7fca7c151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40716/0 (socket says 172.21.15.29:40716) 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca72ffd640 1 -- 172.21.15.29:0/802006519 learned_addr learned my addr 172.21.15.29:0/802006519 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca7b7fe640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca7c06db10 0x7fca7c1436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca7bfff640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca7c15d980 0x7fca7c152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca72ffd640 1 -- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca7c15d980 msgr2=0x7fca7c152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca72ffd640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca7c15d980 0x7fca7c152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca72ffd640 1 -- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca7c06db10 msgr2=0x7fca7c1436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca72ffd640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca7c06db10 0x7fca7c1436b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca72ffd640 1 -- 172.21.15.29:0/802006519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c147620 con 0x7fca7c0b2b70 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca72ffd640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca7c0b2b70 0x7fca7c151b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00c450 tx=0x7fca6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.707+0000 7fca78ff9640 1 -- 172.21.15.29:0/802006519 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c012830 con 0x7fca7c0b2b70 2024-04-25T00:32:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.711+0000 7fca78ff9640 1 -- 172.21.15.29:0/802006519 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca6c017070 con 0x7fca7c0b2b70 2024-04-25T00:32:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.711+0000 7fca81da0640 1 -- 172.21.15.29:0/802006519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7c147900 con 0x7fca7c0b2b70 2024-04-25T00:32:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.711+0000 7fca78ff9640 1 -- 172.21.15.29:0/802006519 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c012c30 con 0x7fca7c0b2b70 2024-04-25T00:32:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.711+0000 7fca81da0640 1 -- 172.21.15.29:0/802006519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7c147ec0 con 0x7fca7c0b2b70 2024-04-25T00:32:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.711+0000 7fca81da0640 1 -- 172.21.15.29:0/802006519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca7c145a50 con 0x7fca7c0b2b70 2024-04-25T00:32:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.711+0000 7fca78ff9640 1 -- 172.21.15.29:0/802006519 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fca6c01c460 con 0x7fca7c0b2b70 2024-04-25T00:32:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.711+0000 7fca78ff9640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fca5c0678c0 0x7fca5c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.711+0000 7fca78ff9640 1 -- 172.21.15.29:0/802006519 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fca6c085a00 con 0x7fca7c0b2b70 2024-04-25T00:32:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.711+0000 7fca7b7fe640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fca5c0678c0 0x7fca5c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.711+0000 7fca7b7fe640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fca5c0678c0 0x7fca5c069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fca640b2000 tx=0x7fca640a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.715+0000 7fca78ff9640 1 -- 172.21.15.29:0/802006519 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fca6c0565d0 con 0x7fca7c0b2b70 2024-04-25T00:32:15.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.879+0000 7fca81da0640 1 -- 172.21.15.29:0/802006519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca7c0cf910 con 0x7fca7c0b2b70 2024-04-25T00:32:15.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.879+0000 7fca78ff9640 1 -- 172.21.15.29:0/802006519 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fca6c055ed0 con 0x7fca7c0b2b70 2024-04-25T00:32:15.884 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:15.884 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:15.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.883+0000 7fca81da0640 1 -- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fca5c0678c0 msgr2=0x7fca5c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.883+0000 7fca81da0640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fca5c0678c0 0x7fca5c069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fca640b2000 tx=0x7fca640a9070 comp rx=0 tx=0).stop 2024-04-25T00:32:15.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.883+0000 7fca81da0640 1 -- 172.21.15.29:0/802006519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca7c0b2b70 msgr2=0x7fca7c151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:15.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.883+0000 7fca81da0640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca7c0b2b70 0x7fca7c151b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00c450 tx=0x7fca6c00c910 comp rx=0 tx=0).stop 2024-04-25T00:32:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.883+0000 7fca81da0640 1 -- 172.21.15.29:0/802006519 shutdown_connections 2024-04-25T00:32:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.883+0000 7fca81da0640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca7c15d980 0x7fca7c152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.883+0000 7fca81da0640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca7c0b2b70 0x7fca7c151b50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.883+0000 7fca81da0640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fca5c0678c0 0x7fca5c069d80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.883+0000 7fca81da0640 1 --2- 172.21.15.29:0/802006519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca7c06db10 0x7fca7c1436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.883+0000 7fca81da0640 1 -- 172.21.15.29:0/802006519 >> 172.21.15.29:0/802006519 conn(0x7fca7c0b1870 msgr2=0x7fca7c0b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.883+0000 7fca81da0640 1 -- 172.21.15.29:0/802006519 shutdown_connections 2024-04-25T00:32:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:15.883+0000 7fca81da0640 1 -- 172.21.15.29:0/802006519 wait complete. 2024-04-25T00:32:15.903 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:15.903 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:16.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f108336b640 1 Processor -- start 2024-04-25T00:32:16.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f108336b640 1 -- start start 2024-04-25T00:32:16.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f108336b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f107c0d4bf0 0x7f107c0d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f108336b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f107c15e8c0 0x7f107c0b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f108336b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f107c0cfe30 con 0x7f107c15e4e0 2024-04-25T00:32:16.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f108336b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f107c0b6f80 con 0x7f107c0d4bf0 2024-04-25T00:32:16.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f108336b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f107c0b7260 con 0x7f107c15e8c0 2024-04-25T00:32:16.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f1080906640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f107c0d4bf0 0x7f107c0d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f1081107640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f107c15e4e0 0x7f107c0d1eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35278/0 (socket says 172.21.15.29:35278) 2024-04-25T00:32:16.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f1081107640 1 -- 172.21.15.29:0/445652981 learned_addr learned my addr 172.21.15.29:0/445652981 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:16.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f1081908640 1 --2- 172.21.15.29:0/445652981 >> v2:172.21.15.157:3300/0 conn(0x7f107c15e8c0 0x7f107c0b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f1080906640 1 -- 172.21.15.29:0/445652981 >> v2:172.21.15.157:3300/0 conn(0x7f107c15e8c0 msgr2=0x7f107c0b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:16.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f1080906640 1 --2- 172.21.15.29:0/445652981 >> v2:172.21.15.157:3300/0 conn(0x7f107c15e8c0 0x7f107c0b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f1073fff640 1 -- 172.21.15.29:0/445652981 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1278778852 0 0) 0x7f107c0cfe30 con 0x7f107c15e4e0 2024-04-25T00:32:16.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f1080906640 1 -- 172.21.15.29:0/445652981 >> v1:172.21.15.29:6789/0 conn(0x7f107c15e4e0 legacy=0x7f107c0d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.019+0000 7f1080906640 1 -- 172.21.15.29:0/445652981 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107c0ced50 con 0x7f107c0d4bf0 2024-04-25T00:32:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.023+0000 7f1080906640 1 --2- 172.21.15.29:0/445652981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f107c0d4bf0 0x7f107c0d25c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f107c073880 tx=0x7f106c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e08cbf08ee96241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.023+0000 7f1073fff640 1 -- 172.21.15.29:0/445652981 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0b5070 con 0x7f107c0d4bf0 2024-04-25T00:32:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.023+0000 7f1073fff640 1 -- 172.21.15.29:0/445652981 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106c0a8970 con 0x7f107c0d4bf0 2024-04-25T00:32:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.023+0000 7f1073fff640 1 -- 172.21.15.29:0/445652981 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0a8d40 con 0x7f107c0d4bf0 2024-04-25T00:32:16.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.023+0000 7f108336b640 1 -- 172.21.15.29:0/445652981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f107c0d4bf0 msgr2=0x7f107c0d25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.023+0000 7f108336b640 1 --2- 172.21.15.29:0/445652981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f107c0d4bf0 0x7f107c0d25c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f107c073880 tx=0x7f106c01ec70 comp rx=0 tx=0).stop 2024-04-25T00:32:16.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.023+0000 7f108336b640 1 -- 172.21.15.29:0/445652981 shutdown_connections 2024-04-25T00:32:16.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.023+0000 7f108336b640 1 --2- 172.21.15.29:0/445652981 >> v2:172.21.15.157:3300/0 conn(0x7f107c15e8c0 0x7f107c0b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.023+0000 7f108336b640 1 --2- 172.21.15.29:0/445652981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f107c0d4bf0 0x7f107c0d25c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.023+0000 7f108336b640 1 -- 172.21.15.29:0/445652981 >> 172.21.15.29:0/445652981 conn(0x7f107c0d3fb0 msgr2=0x7f107c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:16.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.023+0000 7f108336b640 1 -- 172.21.15.29:0/445652981 shutdown_connections 2024-04-25T00:32:16.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.023+0000 7f108336b640 1 -- 172.21.15.29:0/445652981 wait complete. 2024-04-25T00:32:16.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f108336b640 1 Processor -- start 2024-04-25T00:32:16.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f108336b640 1 -- start start 2024-04-25T00:32:16.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f108336b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f107c0d4bf0 0x7f107c15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f108336b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f107c15e4e0 0x7f107c06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f108336b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107c15e8c0 0x7f107c06d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f108336b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f107c0b7e40 con 0x7f107c15e4e0 2024-04-25T00:32:16.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f108336b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f107c0b7cc0 con 0x7f107c15e8c0 2024-04-25T00:32:16.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f108336b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f107c0b7fc0 con 0x7f107c0d4bf0 2024-04-25T00:32:16.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1080906640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f107c15e4e0 0x7f107c06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1080906640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f107c15e4e0 0x7f107c06cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40724/0 (socket says 172.21.15.29:40724) 2024-04-25T00:32:16.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1080906640 1 -- 172.21.15.29:0/3210990832 learned_addr learned my addr 172.21.15.29:0/3210990832 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:16.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1081908640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107c15e8c0 0x7f107c06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1081107640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f107c0d4bf0 0x7f107c15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1080906640 1 -- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f107c0d4bf0 msgr2=0x7f107c15a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1080906640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f107c0d4bf0 0x7f107c15a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1080906640 1 -- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107c15e8c0 msgr2=0x7f107c06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1080906640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107c15e8c0 0x7f107c06d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1080906640 1 -- 172.21.15.29:0/3210990832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107c1f41c0 con 0x7f107c15e4e0 2024-04-25T00:32:16.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1081908640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107c15e8c0 0x7f107c06d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:16.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1080906640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f107c15e4e0 0x7f107c06cb20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f106c01c8a0 tx=0x7f106c01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:16.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1071ffb640 1 -- 172.21.15.29:0/3210990832 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0bf070 con 0x7f107c15e4e0 2024-04-25T00:32:16.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1071ffb640 1 -- 172.21.15.29:0/3210990832 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106c0b0740 con 0x7f107c15e4e0 2024-04-25T00:32:16.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f108336b640 1 -- 172.21.15.29:0/3210990832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107c1f44a0 con 0x7f107c15e4e0 2024-04-25T00:32:16.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1071ffb640 1 -- 172.21.15.29:0/3210990832 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0b5040 con 0x7f107c15e4e0 2024-04-25T00:32:16.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f108336b640 1 -- 172.21.15.29:0/3210990832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f107c1f4a60 con 0x7f107c15e4e0 2024-04-25T00:32:16.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f108336b640 1 -- 172.21.15.29:0/3210990832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107c069270 con 0x7f107c15e4e0 2024-04-25T00:32:16.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.027+0000 7f1071ffb640 1 -- 172.21.15.29:0/3210990832 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f106c0c2070 con 0x7f107c15e4e0 2024-04-25T00:32:16.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.031+0000 7f1071ffb640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f10600678c0 0x7f1060069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.031+0000 7f1071ffb640 1 -- 172.21.15.29:0/3210990832 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f106c122510 con 0x7f107c15e4e0 2024-04-25T00:32:16.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.031+0000 7f1081107640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f10600678c0 0x7f1060069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.031+0000 7f1081107640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f10600678c0 0x7f1060069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f10740012f0 tx=0x7f1074009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.035+0000 7f1071ffb640 1 -- 172.21.15.29:0/3210990832 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f106c0187d0 con 0x7f107c15e4e0 2024-04-25T00:32:16.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.187+0000 7f108336b640 1 -- 172.21.15.29:0/3210990832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f107c0b8860 con 0x7f107c15e4e0 2024-04-25T00:32:16.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.191+0000 7f1071ffb640 1 -- 172.21.15.29:0/3210990832 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f106c0f3060 con 0x7f107c15e4e0 2024-04-25T00:32:16.193 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.191+0000 7f108336b640 1 -- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f10600678c0 msgr2=0x7f1060069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.191+0000 7f108336b640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f10600678c0 0x7f1060069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f10740012f0 tx=0x7f1074009040 comp rx=0 tx=0).stop 2024-04-25T00:32:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.191+0000 7f108336b640 1 -- 172.21.15.29:0/3210990832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f107c15e4e0 msgr2=0x7f107c06cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.191+0000 7f108336b640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f107c15e4e0 0x7f107c06cb20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f106c01c8a0 tx=0x7f106c01ba10 comp rx=0 tx=0).stop 2024-04-25T00:32:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.191+0000 7f108336b640 1 -- 172.21.15.29:0/3210990832 shutdown_connections 2024-04-25T00:32:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.191+0000 7f108336b640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f10600678c0 0x7f1060069d80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.191+0000 7f108336b640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107c15e8c0 0x7f107c06d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.191+0000 7f108336b640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f107c15e4e0 0x7f107c06cb20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.191+0000 7f108336b640 1 --2- 172.21.15.29:0/3210990832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f107c0d4bf0 0x7f107c15a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.191+0000 7f108336b640 1 -- 172.21.15.29:0/3210990832 >> 172.21.15.29:0/3210990832 conn(0x7f107c0d3fb0 msgr2=0x7f107c0b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:16.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.191+0000 7f108336b640 1 -- 172.21.15.29:0/3210990832 shutdown_connections 2024-04-25T00:32:16.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.191+0000 7f108336b640 1 -- 172.21.15.29:0/3210990832 wait complete. 2024-04-25T00:32:16.212 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:16.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81cdc57640 1 Processor -- start 2024-04-25T00:32:16.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81cdc57640 1 -- start start 2024-04-25T00:32:16.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81cdc57640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f81c80b3c30 0x7f81c806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81cdc57640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f81c80b4b10 0x7f81c80ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81cdc57640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81c80b41e0 con 0x7f81c80b3140 2024-04-25T00:32:16.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81cdc57640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f81c806d960 con 0x7f81c80b4b10 2024-04-25T00:32:16.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81cdc57640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f81c806dc40 con 0x7f81c80b3c30 2024-04-25T00:32:16.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81c7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f81c80b4b10 0x7f81c80ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81c7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f81c80b4b10 0x7f81c80ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40728/0 (socket says 172.21.15.29:40728) 2024-04-25T00:32:16.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81c7fff640 1 -- 172.21.15.29:0/2454743478 learned_addr learned my addr 172.21.15.29:0/2454743478 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:16.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81c6ffd640 1 --2- 172.21.15.29:0/2454743478 >> v2:172.21.15.157:3300/0 conn(0x7f81c80b3c30 0x7f81c806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81c7fff640 1 -- 172.21.15.29:0/2454743478 >> v2:172.21.15.157:3300/0 conn(0x7f81c80b3c30 msgr2=0x7f81c806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:16.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81c7fff640 1 --2- 172.21.15.29:0/2454743478 >> v2:172.21.15.157:3300/0 conn(0x7f81c80b3c30 0x7f81c806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81c7fff640 1 -- 172.21.15.29:0/2454743478 >> v1:172.21.15.29:6789/0 conn(0x7f81c80b3140 legacy=0x7f81c80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81c7fff640 1 -- 172.21.15.29:0/2454743478 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c806df20 con 0x7f81c80b4b10 2024-04-25T00:32:16.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.331+0000 7f81c6ffd640 1 --2- 172.21.15.29:0/2454743478 >> v2:172.21.15.157:3300/0 conn(0x7f81c80b3c30 0x7f81c806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:16.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81c7fff640 1 --2- 172.21.15.29:0/2454743478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81c80b4b10 0x7f81c80ce240 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009e50 tx=0x7f81c001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85979ec8b416d408 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:16.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81c67fc640 1 -- 172.21.15.29:0/2454743478 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c00a56c0 con 0x7f81c80b4b10 2024-04-25T00:32:16.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81c67fc640 1 -- 172.21.15.29:0/2454743478 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c00a5860 con 0x7f81c80b4b10 2024-04-25T00:32:16.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81c67fc640 1 -- 172.21.15.29:0/2454743478 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c00a5c30 con 0x7f81c80b4b10 2024-04-25T00:32:16.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 -- 172.21.15.29:0/2454743478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81c80b4b10 msgr2=0x7f81c80ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 --2- 172.21.15.29:0/2454743478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81c80b4b10 0x7f81c80ce240 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009e50 tx=0x7f81c001c9f0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 -- 172.21.15.29:0/2454743478 shutdown_connections 2024-04-25T00:32:16.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 --2- 172.21.15.29:0/2454743478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81c80b4b10 0x7f81c80ce240 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 --2- 172.21.15.29:0/2454743478 >> v2:172.21.15.157:3300/0 conn(0x7f81c80b3c30 0x7f81c806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 -- 172.21.15.29:0/2454743478 >> 172.21.15.29:0/2454743478 conn(0x7f81c80b1870 msgr2=0x7f81c80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:16.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 -- 172.21.15.29:0/2454743478 shutdown_connections 2024-04-25T00:32:16.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 -- 172.21.15.29:0/2454743478 wait complete. 2024-04-25T00:32:16.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 Processor -- start 2024-04-25T00:32:16.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 -- start start 2024-04-25T00:32:16.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81c80b3140 0x7f81c812c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81c80b3c30 0x7f81c80cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81c80b4b10 0x7f81c80d0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8168d80 con 0x7f81c80b3140 2024-04-25T00:32:16.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8168c00 con 0x7f81c80b4b10 2024-04-25T00:32:16.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81cdc57640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f81c8168f00 con 0x7f81c80b3c30 2024-04-25T00:32:16.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81c77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81c80b3140 0x7f81c812c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81c77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81c80b3140 0x7f81c812c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40730/0 (socket says 172.21.15.29:40730) 2024-04-25T00:32:16.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.335+0000 7f81c77fe640 1 -- 172.21.15.29:0/2815585575 learned_addr learned my addr 172.21.15.29:0/2815585575 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:16.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81c7fff640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81c80b4b10 0x7f81c80d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81c6ffd640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81c80b3c30 0x7f81c80cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81c77fe640 1 -- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81c80b3c30 msgr2=0x7f81c80cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81c77fe640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81c80b3c30 0x7f81c80cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81c77fe640 1 -- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81c80b4b10 msgr2=0x7f81c80d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81c77fe640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81c80b4b10 0x7f81c80d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81c77fe640 1 -- 172.21.15.29:0/2815585575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c8164b50 con 0x7f81c80b3140 2024-04-25T00:32:16.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81c77fe640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81c80b3140 0x7f81c812c380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f81b800c420 tx=0x7f81b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:16.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81affff640 1 -- 172.21.15.29:0/2815585575 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b80052d0 con 0x7f81c80b3140 2024-04-25T00:32:16.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81affff640 1 -- 172.21.15.29:0/2815585575 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b8005470 con 0x7f81c80b3140 2024-04-25T00:32:16.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81affff640 1 -- 172.21.15.29:0/2815585575 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8007260 con 0x7f81c80b3140 2024-04-25T00:32:16.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81cdc57640 1 -- 172.21.15.29:0/2815585575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c8164e30 con 0x7f81c80b3140 2024-04-25T00:32:16.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81cdc57640 1 -- 172.21.15.29:0/2815585575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c81656d0 con 0x7f81c80b3140 2024-04-25T00:32:16.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81cdc57640 1 -- 172.21.15.29:0/2815585575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c8069270 con 0x7f81c80b3140 2024-04-25T00:32:16.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81affff640 1 -- 172.21.15.29:0/2815585575 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f81b8007400 con 0x7f81c80b3140 2024-04-25T00:32:16.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81affff640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81b40678c0 0x7f81b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81affff640 1 -- 172.21.15.29:0/2815585575 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f81b80851d0 con 0x7f81c80b3140 2024-04-25T00:32:16.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81c6ffd640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81b40678c0 0x7f81b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.339+0000 7f81c6ffd640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81b40678c0 0x7f81b4069d80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f81c81437c0 tx=0x7f81bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:16.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.343+0000 7f81affff640 1 -- 172.21.15.29:0/2815585575 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f81b8055d10 con 0x7f81c80b3140 2024-04-25T00:32:16.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.503+0000 7f81cdc57640 1 -- 172.21.15.29:0/2815585575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f81c812c920 con 0x7f81c80b3140 2024-04-25T00:32:16.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.503+0000 7f81affff640 1 -- 172.21.15.29:0/2815585575 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f81b8055610 con 0x7f81c80b3140 2024-04-25T00:32:16.509 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:16.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.507+0000 7f81cdc57640 1 -- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81b40678c0 msgr2=0x7f81b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.507+0000 7f81cdc57640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81b40678c0 0x7f81b4069d80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f81c81437c0 tx=0x7f81bc008040 comp rx=0 tx=0).stop 2024-04-25T00:32:16.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.507+0000 7f81cdc57640 1 -- 172.21.15.29:0/2815585575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81c80b3140 msgr2=0x7f81c812c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.507+0000 7f81cdc57640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81c80b3140 0x7f81c812c380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f81b800c420 tx=0x7f81b800c8e0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.507+0000 7f81cdc57640 1 -- 172.21.15.29:0/2815585575 shutdown_connections 2024-04-25T00:32:16.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.507+0000 7f81cdc57640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81b40678c0 0x7f81b4069d80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.507+0000 7f81cdc57640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81c80b4b10 0x7f81c80d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.507+0000 7f81cdc57640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81c80b3c30 0x7f81c80cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.507+0000 7f81cdc57640 1 --2- 172.21.15.29:0/2815585575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81c80b3140 0x7f81c812c380 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.507+0000 7f81cdc57640 1 -- 172.21.15.29:0/2815585575 >> 172.21.15.29:0/2815585575 conn(0x7f81c80b1870 msgr2=0x7f81c80b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:16.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.507+0000 7f81cdc57640 1 -- 172.21.15.29:0/2815585575 shutdown_connections 2024-04-25T00:32:16.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.507+0000 7f81cdc57640 1 -- 172.21.15.29:0/2815585575 wait complete. 2024-04-25T00:32:16.528 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:16.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.651+0000 7fb17b456640 1 Processor -- start 2024-04-25T00:32:16.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.651+0000 7fb17b456640 1 -- start start 2024-04-25T00:32:16.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.651+0000 7fb17b456640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1740b4780 0x7fb17415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.651+0000 7fb17b456640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb1740cc240 0x7fb17415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.651+0000 7fb17b456640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1740b3720 con 0x7fb17415e8d0 2024-04-25T00:32:16.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.651+0000 7fb17b456640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb1740cc650 con 0x7fb1740cc240 2024-04-25T00:32:16.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.651+0000 7fb17b456640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb1740b42f0 con 0x7fb1740b4780 2024-04-25T00:32:16.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.651+0000 7fb1799f3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb1740cc240 0x7fb17415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.655+0000 7fb1799f3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb1740cc240 0x7fb17415c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40734/0 (socket says 172.21.15.29:40734) 2024-04-25T00:32:16.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.655+0000 7fb1791f2640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb17415e8d0 0x7fb17415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35298/0 (socket says 172.21.15.29:35298) 2024-04-25T00:32:16.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.655+0000 7fb1799f3640 1 -- 172.21.15.29:0/1026405311 learned_addr learned my addr 172.21.15.29:0/1026405311 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:16.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.655+0000 7fb1789f1640 1 --2- 172.21.15.29:0/1026405311 >> v2:172.21.15.157:3300/0 conn(0x7fb1740b4780 0x7fb17415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.655+0000 7fb1799f3640 1 -- 172.21.15.29:0/1026405311 >> v2:172.21.15.157:3300/0 conn(0x7fb1740b4780 msgr2=0x7fb17415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:16.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.655+0000 7fb1799f3640 1 --2- 172.21.15.29:0/1026405311 >> v2:172.21.15.157:3300/0 conn(0x7fb1740b4780 0x7fb17415f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.655+0000 7fb1799f3640 1 -- 172.21.15.29:0/1026405311 >> v1:172.21.15.29:6789/0 conn(0x7fb17415e8d0 legacy=0x7fb17415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.655+0000 7fb1799f3640 1 -- 172.21.15.29:0/1026405311 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1740b39c0 con 0x7fb1740cc240 2024-04-25T00:32:16.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.655+0000 7fb1799f3640 1 --2- 172.21.15.29:0/1026405311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1740cc240 0x7fb17415c7d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb174070910 tx=0x7fb16c01ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9c393d5674bbee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:16.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb16bfff640 1 -- 172.21.15.29:0/1026405311 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c0a7aa0 con 0x7fb1740cc240 2024-04-25T00:32:16.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb16bfff640 1 -- 172.21.15.29:0/1026405311 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16c0a7c40 con 0x7fb1740cc240 2024-04-25T00:32:16.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb16bfff640 1 -- 172.21.15.29:0/1026405311 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c0af3f0 con 0x7fb1740cc240 2024-04-25T00:32:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 -- 172.21.15.29:0/1026405311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1740cc240 msgr2=0x7fb17415c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 --2- 172.21.15.29:0/1026405311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1740cc240 0x7fb17415c7d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb174070910 tx=0x7fb16c01ebd0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 -- 172.21.15.29:0/1026405311 shutdown_connections 2024-04-25T00:32:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 --2- 172.21.15.29:0/1026405311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1740cc240 0x7fb17415c7d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 --2- 172.21.15.29:0/1026405311 >> v2:172.21.15.157:3300/0 conn(0x7fb1740b4780 0x7fb17415f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 -- 172.21.15.29:0/1026405311 >> 172.21.15.29:0/1026405311 conn(0x7fb1740b1870 msgr2=0x7fb1740b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:16.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 -- 172.21.15.29:0/1026405311 shutdown_connections 2024-04-25T00:32:16.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 -- 172.21.15.29:0/1026405311 wait complete. 2024-04-25T00:32:16.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 Processor -- start 2024-04-25T00:32:16.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 -- start start 2024-04-25T00:32:16.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1740b4780 0x7fb174137d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1740cc240 0x7fb1740ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb17415e8d0 0x7fb1740aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb174144a20 con 0x7fb1740b4780 2024-04-25T00:32:16.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb17415d520 con 0x7fb17415e8d0 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb1740ad8a0 con 0x7fb1740cc240 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb1791f2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1740b4780 0x7fb174137d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb1791f2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1740b4780 0x7fb174137d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40736/0 (socket says 172.21.15.29:40736) 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb1791f2640 1 -- 172.21.15.29:0/402044050 learned_addr learned my addr 172.21.15.29:0/402044050 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb1799f3640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb17415e8d0 0x7fb1740aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb1789f1640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1740cc240 0x7fb1740ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb1791f2640 1 -- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1740cc240 msgr2=0x7fb1740ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb1791f2640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1740cc240 0x7fb1740ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb1791f2640 1 -- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb17415e8d0 msgr2=0x7fb1740aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb1791f2640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb17415e8d0 0x7fb1740aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb1791f2640 1 -- 172.21.15.29:0/402044050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1740adb80 con 0x7fb1740b4780 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb1799f3640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb17415e8d0 0x7fb1740aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb1789f1640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1740cc240 0x7fb1740ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb1791f2640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1740b4780 0x7fb174137d70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb17000c3a0 tx=0x7fb17000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:16.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb169ffb640 1 -- 172.21.15.29:0/402044050 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1700074e0 con 0x7fb1740b4780 2024-04-25T00:32:16.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb169ffb640 1 -- 172.21.15.29:0/402044050 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb170007680 con 0x7fb1740b4780 2024-04-25T00:32:16.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 -- 172.21.15.29:0/402044050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1740ade60 con 0x7fb1740b4780 2024-04-25T00:32:16.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb169ffb640 1 -- 172.21.15.29:0/402044050 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170004a30 con 0x7fb1740b4780 2024-04-25T00:32:16.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.659+0000 7fb17b456640 1 -- 172.21.15.29:0/402044050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb174143770 con 0x7fb1740b4780 2024-04-25T00:32:16.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.663+0000 7fb17b456640 1 -- 172.21.15.29:0/402044050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb174069270 con 0x7fb1740b4780 2024-04-25T00:32:16.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.663+0000 7fb169ffb640 1 -- 172.21.15.29:0/402044050 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb17001a020 con 0x7fb1740b4780 2024-04-25T00:32:16.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.663+0000 7fb169ffb640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb1580678c0 0x7fb158069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.663+0000 7fb169ffb640 1 -- 172.21.15.29:0/402044050 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb170085420 con 0x7fb1740b4780 2024-04-25T00:32:16.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.663+0000 7fb1789f1640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb1580678c0 0x7fb158069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.663+0000 7fb1789f1640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb1580678c0 0x7fb158069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb1740cf720 tx=0x7fb164008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:16.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.667+0000 7fb169ffb640 1 -- 172.21.15.29:0/402044050 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb170055f60 con 0x7fb1740b4780 2024-04-25T00:32:16.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.827+0000 7fb17b456640 1 -- 172.21.15.29:0/402044050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb17406e7b0 con 0x7fb1740b4780 2024-04-25T00:32:16.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.827+0000 7fb169ffb640 1 -- 172.21.15.29:0/402044050 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fb170055860 con 0x7fb1740b4780 2024-04-25T00:32:16.832 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.831+0000 7fb17b456640 1 -- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb1580678c0 msgr2=0x7fb158069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.831+0000 7fb17b456640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb1580678c0 0x7fb158069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb1740cf720 tx=0x7fb164008040 comp rx=0 tx=0).stop 2024-04-25T00:32:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.831+0000 7fb17b456640 1 -- 172.21.15.29:0/402044050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1740b4780 msgr2=0x7fb174137d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.831+0000 7fb17b456640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1740b4780 0x7fb174137d70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb17000c3a0 tx=0x7fb17000c860 comp rx=0 tx=0).stop 2024-04-25T00:32:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.831+0000 7fb17b456640 1 -- 172.21.15.29:0/402044050 shutdown_connections 2024-04-25T00:32:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.831+0000 7fb17b456640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb1580678c0 0x7fb158069d80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.831+0000 7fb17b456640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb17415e8d0 0x7fb1740aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.831+0000 7fb17b456640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1740cc240 0x7fb1740ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.831+0000 7fb17b456640 1 --2- 172.21.15.29:0/402044050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1740b4780 0x7fb174137d70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.831+0000 7fb17b456640 1 -- 172.21.15.29:0/402044050 >> 172.21.15.29:0/402044050 conn(0x7fb1740b1870 msgr2=0x7fb17413c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:16.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.831+0000 7fb17b456640 1 -- 172.21.15.29:0/402044050 shutdown_connections 2024-04-25T00:32:16.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.831+0000 7fb17b456640 1 -- 172.21.15.29:0/402044050 wait complete. 2024-04-25T00:32:16.852 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:32:16.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f133452b640 1 Processor -- start 2024-04-25T00:32:16.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f133452b640 1 -- start start 2024-04-25T00:32:16.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f133452b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f132c0b3140 0x7f132c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f133452b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132c0b3c30 0x7f132c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f133452b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f132c0b41e0 con 0x7f132c0b4b10 2024-04-25T00:32:16.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f133452b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f132c06d960 con 0x7f132c0b3140 2024-04-25T00:32:16.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f133452b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f132c06dc40 con 0x7f132c0b3c30 2024-04-25T00:32:16.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f13322c7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f132c0b3140 0x7f132c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f13322c7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f132c0b3140 0x7f132c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40746/0 (socket says 172.21.15.29:40746) 2024-04-25T00:32:16.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f13322c7640 1 -- 172.21.15.29:0/1428469609 learned_addr learned my addr 172.21.15.29:0/1428469609 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:16.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f1332ac8640 1 --2- 172.21.15.29:0/1428469609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132c0b3c30 0x7f132c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f13322c7640 1 -- 172.21.15.29:0/1428469609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132c0b3c30 msgr2=0x7f132c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:16.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f13322c7640 1 --2- 172.21.15.29:0/1428469609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132c0b3c30 0x7f132c0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f13322c7640 1 -- 172.21.15.29:0/1428469609 >> v1:172.21.15.157:6789/0 conn(0x7f132c0b4b10 legacy=0x7f132c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.975+0000 7f13322c7640 1 -- 172.21.15.29:0/1428469609 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c06df20 con 0x7f132c0b3140 2024-04-25T00:32:16.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.979+0000 7f13322c7640 1 --2- 172.21.15.29:0/1428469609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f132c0b3140 0x7f132c06e440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f131c009e80 tx=0x7f131c01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edfec8ee2c8e5d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:16.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.979+0000 7f13312c5640 1 -- 172.21.15.29:0/1428469609 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c0a56e0 con 0x7f132c0b3140 2024-04-25T00:32:16.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.979+0000 7f13312c5640 1 -- 172.21.15.29:0/1428469609 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f131c0a5880 con 0x7f132c0b3140 2024-04-25T00:32:16.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.979+0000 7f13312c5640 1 -- 172.21.15.29:0/1428469609 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c0a5c50 con 0x7f132c0b3140 2024-04-25T00:32:16.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.979+0000 7f133452b640 1 -- 172.21.15.29:0/1428469609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f132c0b3140 msgr2=0x7f132c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.979+0000 7f133452b640 1 --2- 172.21.15.29:0/1428469609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f132c0b3140 0x7f132c06e440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f131c009e80 tx=0x7f131c01ca10 comp rx=0 tx=0).stop 2024-04-25T00:32:16.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.979+0000 7f133452b640 1 -- 172.21.15.29:0/1428469609 shutdown_connections 2024-04-25T00:32:16.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.979+0000 7f133452b640 1 --2- 172.21.15.29:0/1428469609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132c0b3c30 0x7f132c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.979+0000 7f133452b640 1 --2- 172.21.15.29:0/1428469609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f132c0b3140 0x7f132c06e440 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.979+0000 7f133452b640 1 -- 172.21.15.29:0/1428469609 >> 172.21.15.29:0/1428469609 conn(0x7f132c0b1870 msgr2=0x7f132c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:16.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.979+0000 7f133452b640 1 -- 172.21.15.29:0/1428469609 shutdown_connections 2024-04-25T00:32:16.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.979+0000 7f133452b640 1 -- 172.21.15.29:0/1428469609 wait complete. 2024-04-25T00:32:16.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f133452b640 1 Processor -- start 2024-04-25T00:32:16.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f133452b640 1 -- start start 2024-04-25T00:32:16.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f133452b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f132c0b3140 0x7f132c138960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f133452b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132c0b3c30 0x7f132c139160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f133452b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f132c0b4b10 0x7f132c157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f133452b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f132c168d80 con 0x7f132c0b3140 2024-04-25T00:32:16.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f133452b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f132c168c00 con 0x7f132c0b4b10 2024-04-25T00:32:16.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f133452b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f132c168f00 con 0x7f132c0b3c30 2024-04-25T00:32:16.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f13322c7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f132c0b3140 0x7f132c138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f13322c7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f132c0b3140 0x7f132c138960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40748/0 (socket says 172.21.15.29:40748) 2024-04-25T00:32:16.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f13322c7640 1 -- 172.21.15.29:0/3432118931 learned_addr learned my addr 172.21.15.29:0/3432118931 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:16.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f1332ac8640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f132c0b4b10 0x7f132c157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f1331ac6640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132c0b3c30 0x7f132c139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f13322c7640 1 -- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132c0b3c30 msgr2=0x7f132c139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f13322c7640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132c0b3c30 0x7f132c139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f13322c7640 1 -- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f132c0b4b10 msgr2=0x7f132c157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:16.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f13322c7640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f132c0b4b10 0x7f132c157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:16.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f13322c7640 1 -- 172.21.15.29:0/3432118931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c139a40 con 0x7f132c0b3140 2024-04-25T00:32:16.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f13322c7640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f132c0b3140 0x7f132c138960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f131c01a640 tx=0x7f131c0197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:16.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f132affd640 1 -- 172.21.15.29:0/3432118931 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c0bc020 con 0x7f132c0b3140 2024-04-25T00:32:16.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f132affd640 1 -- 172.21.15.29:0/3432118931 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f131c0b8670 con 0x7f132c0b3140 2024-04-25T00:32:16.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f133452b640 1 -- 172.21.15.29:0/3432118931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132c139d20 con 0x7f132c0b3140 2024-04-25T00:32:16.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f132affd640 1 -- 172.21.15.29:0/3432118931 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c009510 con 0x7f132c0b3140 2024-04-25T00:32:16.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f133452b640 1 -- 172.21.15.29:0/3432118931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132c13a2e0 con 0x7f132c0b3140 2024-04-25T00:32:16.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f133452b640 1 -- 172.21.15.29:0/3432118931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f132c069270 con 0x7f132c0b3140 2024-04-25T00:32:16.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f132affd640 1 -- 172.21.15.29:0/3432118931 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f131c0bf020 con 0x7f132c0b3140 2024-04-25T00:32:16.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.983+0000 7f132affd640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13180678c0 0x7f1318069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:16.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.987+0000 7f132affd640 1 -- 172.21.15.29:0/3432118931 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f131c120be0 con 0x7f132c0b3140 2024-04-25T00:32:16.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.987+0000 7f1331ac6640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13180678c0 0x7f1318069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:16.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.987+0000 7f1331ac6640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13180678c0 0x7f1318069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f132c15d5d0 tx=0x7f1320009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:16.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:16.991+0000 7f132affd640 1 -- 172.21.15.29:0/3432118931 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f131c0c6030 con 0x7f132c0b3140 2024-04-25T00:32:17.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.159+0000 7f133452b640 1 -- 172.21.15.29:0/3432118931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f132c155e50 con 0x7f132c0b3140 2024-04-25T00:32:17.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.163+0000 7f132affd640 1 -- 172.21.15.29:0/3432118931 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f131c016600 con 0x7f132c0b3140 2024-04-25T00:32:17.165 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:32:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.163+0000 7f133452b640 1 -- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13180678c0 msgr2=0x7f1318069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.163+0000 7f133452b640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13180678c0 0x7f1318069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f132c15d5d0 tx=0x7f1320009040 comp rx=0 tx=0).stop 2024-04-25T00:32:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.163+0000 7f133452b640 1 -- 172.21.15.29:0/3432118931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f132c0b3140 msgr2=0x7f132c138960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.163+0000 7f133452b640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f132c0b3140 0x7f132c138960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f131c01a640 tx=0x7f131c0197b0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.163+0000 7f133452b640 1 -- 172.21.15.29:0/3432118931 shutdown_connections 2024-04-25T00:32:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.163+0000 7f133452b640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13180678c0 0x7f1318069d80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.163+0000 7f133452b640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f132c0b4b10 0x7f132c157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.163+0000 7f133452b640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f132c0b3c30 0x7f132c139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.163+0000 7f133452b640 1 --2- 172.21.15.29:0/3432118931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f132c0b3140 0x7f132c138960 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.163+0000 7f133452b640 1 -- 172.21.15.29:0/3432118931 >> 172.21.15.29:0/3432118931 conn(0x7f132c0b1870 msgr2=0x7f132c0cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.163+0000 7f133452b640 1 -- 172.21.15.29:0/3432118931 shutdown_connections 2024-04-25T00:32:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.163+0000 7f133452b640 1 -- 172.21.15.29:0/3432118931 wait complete. 2024-04-25T00:32:17.184 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-04-25T00:32:17.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:17.184 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T00:32:17.197 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:32:17.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:32:17.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:17.197 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:32:17.246 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:32:17.246 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:17.253 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:32:17.253 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:17.301 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:17.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.423+0000 7fb73602e640 1 Processor -- start 2024-04-25T00:32:17.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.423+0000 7fb73602e640 1 -- start start 2024-04-25T00:32:17.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb73602e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb7300b3140 0x7fb73006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb73602e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300b3c30 0x7fb7300ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb73602e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7300b41e0 con 0x7fb7300b4b10 2024-04-25T00:32:17.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb73602e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb73006d960 con 0x7fb7300b3140 2024-04-25T00:32:17.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb73602e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb73006dc40 con 0x7fb7300b3c30 2024-04-25T00:32:17.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300b3c30 0x7fb7300ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb72f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb7300b3140 0x7fb73006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb727fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb7300b4b10 0x7fb7300b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:36336/0 (socket says 172.21.15.29:36336) 2024-04-25T00:32:17.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb72f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb7300b3140 0x7fb73006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:40848/0 (socket says 172.21.15.29:40848) 2024-04-25T00:32:17.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300b3c30 0x7fb7300ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:34848/0 (socket says 172.21.15.29:34848) 2024-04-25T00:32:17.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb727fff640 1 -- 172.21.15.29:0/619729174 learned_addr learned my addr 172.21.15.29:0/619729174 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:17.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb72ffff640 1 -- 172.21.15.29:0/619729174 >> v2:172.21.15.157:3300/0 conn(0x7fb7300b3140 msgr2=0x7fb73006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:17.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb72ffff640 1 --2- 172.21.15.29:0/619729174 >> v2:172.21.15.157:3300/0 conn(0x7fb7300b3140 0x7fb73006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb72ffff640 1 -- 172.21.15.29:0/619729174 >> v1:172.21.15.157:6789/0 conn(0x7fb7300b4b10 legacy=0x7fb7300b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb72ffff640 1 -- 172.21.15.29:0/619729174 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73006df20 con 0x7fb7300b3c30 2024-04-25T00:32:17.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.427+0000 7fb72f7fe640 1 --2- 172.21.15.29:0/619729174 >> v2:172.21.15.157:3300/0 conn(0x7fb7300b3140 0x7fb73006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:17.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb72ffff640 1 --2- 172.21.15.29:0/619729174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300b3c30 0x7fb7300ce240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb728009dc0 tx=0x7fb72801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbbb5ec425abc6a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:17.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb72effd640 1 -- 172.21.15.29:0/619729174 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7280b2070 con 0x7fb7300b3c30 2024-04-25T00:32:17.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb72effd640 1 -- 172.21.15.29:0/619729174 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7280a5810 con 0x7fb7300b3c30 2024-04-25T00:32:17.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb72effd640 1 -- 172.21.15.29:0/619729174 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7280a5be0 con 0x7fb7300b3c30 2024-04-25T00:32:17.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 -- 172.21.15.29:0/619729174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300b3c30 msgr2=0x7fb7300ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 --2- 172.21.15.29:0/619729174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300b3c30 0x7fb7300ce240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb728009dc0 tx=0x7fb72801c950 comp rx=0 tx=0).stop 2024-04-25T00:32:17.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 -- 172.21.15.29:0/619729174 shutdown_connections 2024-04-25T00:32:17.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 --2- 172.21.15.29:0/619729174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300b3c30 0x7fb7300ce240 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 --2- 172.21.15.29:0/619729174 >> v2:172.21.15.157:3300/0 conn(0x7fb7300b3140 0x7fb73006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 -- 172.21.15.29:0/619729174 >> 172.21.15.29:0/619729174 conn(0x7fb7300b1870 msgr2=0x7fb7300b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:17.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 -- 172.21.15.29:0/619729174 shutdown_connections 2024-04-25T00:32:17.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 -- 172.21.15.29:0/619729174 wait complete. 2024-04-25T00:32:17.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 Processor -- start 2024-04-25T00:32:17.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 -- start start 2024-04-25T00:32:17.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300b3140 0x7fb730137880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7300b3c30 0x7fb730142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7300b4b10 0x7fb7301431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb730168d80 con 0x7fb7300b4b10 2024-04-25T00:32:17.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb730168c00 con 0x7fb7300b3c30 2024-04-25T00:32:17.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.431+0000 7fb73602e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb730168f00 con 0x7fb7300b3140 2024-04-25T00:32:17.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7300b4b10 0x7fb7301431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7300b4b10 0x7fb7301431d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40760/0 (socket says 172.21.15.29:40760) 2024-04-25T00:32:17.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72ffff640 1 -- 172.21.15.29:0/4102061105 learned_addr learned my addr 172.21.15.29:0/4102061105 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:17.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72f7fe640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300b3140 0x7fb730137880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb727fff640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7300b3c30 0x7fb730142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72ffff640 1 -- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300b3140 msgr2=0x7fb730137880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72ffff640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300b3140 0x7fb730137880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72ffff640 1 -- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7300b3c30 msgr2=0x7fb730142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72ffff640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7300b3c30 0x7fb730142c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72ffff640 1 -- 172.21.15.29:0/4102061105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb730137ff0 con 0x7fb7300b4b10 2024-04-25T00:32:17.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72ffff640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7300b4b10 0x7fb7301431d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb7280b0040 tx=0x7fb7280196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:17.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72cff9640 1 -- 172.21.15.29:0/4102061105 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7280b2050 con 0x7fb7300b4b10 2024-04-25T00:32:17.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72cff9640 1 -- 172.21.15.29:0/4102061105 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7280be070 con 0x7fb7300b4b10 2024-04-25T00:32:17.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb73602e640 1 -- 172.21.15.29:0/4102061105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7301382d0 con 0x7fb7300b4b10 2024-04-25T00:32:17.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72cff9640 1 -- 172.21.15.29:0/4102061105 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7280ad870 con 0x7fb7300b4b10 2024-04-25T00:32:17.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb73602e640 1 -- 172.21.15.29:0/4102061105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb730138890 con 0x7fb7300b4b10 2024-04-25T00:32:17.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb73602e640 1 -- 172.21.15.29:0/4102061105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb730069270 con 0x7fb7300b4b10 2024-04-25T00:32:17.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72cff9640 1 -- 172.21.15.29:0/4102061105 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb7280ada10 con 0x7fb7300b4b10 2024-04-25T00:32:17.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72cff9640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7100678c0 0x7fb710069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72cff9640 1 -- 172.21.15.29:0/4102061105 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb728120c50 con 0x7fb7300b4b10 2024-04-25T00:32:17.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72f7fe640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7100678c0 0x7fb710069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.435+0000 7fb72f7fe640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7100678c0 0x7fb710069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb720009a50 tx=0x7fb720008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:17.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.439+0000 7fb72cff9640 1 -- 172.21.15.29:0/4102061105 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb728126020 con 0x7fb7300b4b10 2024-04-25T00:32:17.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.603+0000 7fb73602e640 1 -- 172.21.15.29:0/4102061105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7300cdd40 con 0x7fb7300b4b10 2024-04-25T00:32:17.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.607+0000 7fb72cff9640 1 -- 172.21.15.29:0/4102061105 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fb728016540 con 0x7fb7300b4b10 2024-04-25T00:32:17.611 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:17.611 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:31:31.279977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:17.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.611+0000 7fb73602e640 1 -- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7100678c0 msgr2=0x7fb710069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.611+0000 7fb73602e640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7100678c0 0x7fb710069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb720009a50 tx=0x7fb720008040 comp rx=0 tx=0).stop 2024-04-25T00:32:17.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.611+0000 7fb73602e640 1 -- 172.21.15.29:0/4102061105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7300b4b10 msgr2=0x7fb7301431d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.611+0000 7fb73602e640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7300b4b10 0x7fb7301431d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb7280b0040 tx=0x7fb7280196f0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.611+0000 7fb73602e640 1 -- 172.21.15.29:0/4102061105 shutdown_connections 2024-04-25T00:32:17.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.611+0000 7fb73602e640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7300b4b10 0x7fb7301431d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.611+0000 7fb73602e640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb7100678c0 0x7fb710069d80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.611+0000 7fb73602e640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7300b3c30 0x7fb730142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.611+0000 7fb73602e640 1 --2- 172.21.15.29:0/4102061105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300b3140 0x7fb730137880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.611+0000 7fb73602e640 1 -- 172.21.15.29:0/4102061105 >> 172.21.15.29:0/4102061105 conn(0x7fb7300b1870 msgr2=0x7fb7300cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:17.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.611+0000 7fb73602e640 1 -- 172.21.15.29:0/4102061105 shutdown_connections 2024-04-25T00:32:17.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.611+0000 7fb73602e640 1 -- 172.21.15.29:0/4102061105 wait complete. 2024-04-25T00:32:17.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:17.632 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:17.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4f733640 1 Processor -- start 2024-04-25T00:32:17.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4f733640 1 -- start start 2024-04-25T00:32:17.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4f733640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7a4815ad20 0x7f7a480cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4f733640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7a480d0390 0x7f7a4815b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4f733640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a4815bc30 con 0x7f7a4815b640 2024-04-25T00:32:17.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4f733640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7a480cb6a0 con 0x7f7a4815ad20 2024-04-25T00:32:17.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4f733640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7a4815bf10 con 0x7f7a480d0390 2024-04-25T00:32:17.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4d4cf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7a4815ad20 0x7f7a480cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4dcd0640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7a4815b640 0x7f7a48071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38384/0 (socket says 172.21.15.29:38384) 2024-04-25T00:32:17.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4dcd0640 1 -- 172.21.15.29:0/2898192007 learned_addr learned my addr 172.21.15.29:0/2898192007 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:17.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4ccce640 1 --2- 172.21.15.29:0/2898192007 >> v2:172.21.15.157:3300/0 conn(0x7f7a480d0390 0x7f7a4815b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4d4cf640 1 -- 172.21.15.29:0/2898192007 >> v2:172.21.15.157:3300/0 conn(0x7f7a480d0390 msgr2=0x7f7a4815b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:17.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4d4cf640 1 --2- 172.21.15.29:0/2898192007 >> v2:172.21.15.157:3300/0 conn(0x7f7a480d0390 0x7f7a4815b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4d4cf640 1 -- 172.21.15.29:0/2898192007 >> v1:172.21.15.29:6789/0 conn(0x7f7a4815b640 legacy=0x7f7a48071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a4d4cf640 1 -- 172.21.15.29:0/2898192007 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4815c1f0 con 0x7f7a4815ad20 2024-04-25T00:32:17.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.759+0000 7f7a3ffff640 1 -- 172.21.15.29:0/2898192007 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337586992 0 0) 0x7f7a4815bc30 con 0x7f7a4815b640 2024-04-25T00:32:17.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4d4cf640 1 --2- 172.21.15.29:0/2898192007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a4815ad20 0x7f7a480cbb30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7a48072d40 tx=0x7f7a3801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa0d886414e3701 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:17.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a3ffff640 1 -- 172.21.15.29:0/2898192007 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a380b1070 con 0x7f7a4815ad20 2024-04-25T00:32:17.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a3ffff640 1 -- 172.21.15.29:0/2898192007 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a380ac5f0 con 0x7f7a4815ad20 2024-04-25T00:32:17.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a3ffff640 1 -- 172.21.15.29:0/2898192007 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a380ac9c0 con 0x7f7a4815ad20 2024-04-25T00:32:17.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 -- 172.21.15.29:0/2898192007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a4815ad20 msgr2=0x7f7a480cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 --2- 172.21.15.29:0/2898192007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a4815ad20 0x7f7a480cbb30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7a48072d40 tx=0x7f7a3801c480 comp rx=0 tx=0).stop 2024-04-25T00:32:17.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 -- 172.21.15.29:0/2898192007 shutdown_connections 2024-04-25T00:32:17.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 --2- 172.21.15.29:0/2898192007 >> v2:172.21.15.157:3300/0 conn(0x7f7a480d0390 0x7f7a4815b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 --2- 172.21.15.29:0/2898192007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a4815ad20 0x7f7a480cbb30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 -- 172.21.15.29:0/2898192007 >> 172.21.15.29:0/2898192007 conn(0x7f7a480d0c40 msgr2=0x7f7a480d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:17.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 -- 172.21.15.29:0/2898192007 shutdown_connections 2024-04-25T00:32:17.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 -- 172.21.15.29:0/2898192007 wait complete. 2024-04-25T00:32:17.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 Processor -- start 2024-04-25T00:32:17.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 -- start start 2024-04-25T00:32:17.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a480d0390 0x7f7a480ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4815ad20 0x7f7a480c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a4815b640 0x7f7a480c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4806f530 con 0x7f7a4815b640 2024-04-25T00:32:17.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7a480726a0 con 0x7f7a480d0390 2024-04-25T00:32:17.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4f733640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4806f6b0 con 0x7f7a4815ad20 2024-04-25T00:32:17.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4dcd0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a4815b640 0x7f7a480c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4dcd0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a4815b640 0x7f7a480c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42612/0 (socket says 172.21.15.29:42612) 2024-04-25T00:32:17.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.763+0000 7f7a4dcd0640 1 -- 172.21.15.29:0/865304039 learned_addr learned my addr 172.21.15.29:0/865304039 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:17.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4d4cf640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a480d0390 0x7f7a480ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4ccce640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4815ad20 0x7f7a480c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4dcd0640 1 -- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4815ad20 msgr2=0x7f7a480c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4dcd0640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4815ad20 0x7f7a480c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4dcd0640 1 -- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a480d0390 msgr2=0x7f7a480ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4dcd0640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a480d0390 0x7f7a480ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4dcd0640 1 -- 172.21.15.29:0/865304039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a481521f0 con 0x7f7a4815b640 2024-04-25T00:32:17.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4ccce640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4815ad20 0x7f7a480c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:17.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4dcd0640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a4815b640 0x7f7a480c7d10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000c780 tx=0x7f7a4000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:17.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a3dffb640 1 -- 172.21.15.29:0/865304039 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40018070 con 0x7f7a4815b640 2024-04-25T00:32:17.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a3dffb640 1 -- 172.21.15.29:0/865304039 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a4000a640 con 0x7f7a4815b640 2024-04-25T00:32:17.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4f733640 1 -- 172.21.15.29:0/865304039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a480b3900 con 0x7f7a4815b640 2024-04-25T00:32:17.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a3dffb640 1 -- 172.21.15.29:0/865304039 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000aa30 con 0x7f7a4815b640 2024-04-25T00:32:17.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4f733640 1 -- 172.21.15.29:0/865304039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a48152490 con 0x7f7a4815b640 2024-04-25T00:32:17.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4f733640 1 -- 172.21.15.29:0/865304039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a48069270 con 0x7f7a4815b640 2024-04-25T00:32:17.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a3dffb640 1 -- 172.21.15.29:0/865304039 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7a4001b070 con 0x7f7a4815b640 2024-04-25T00:32:17.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a3dffb640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7a2c0678c0 0x7f7a2c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a3dffb640 1 -- 172.21.15.29:0/865304039 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5883+0+0 (secure 0 0 0) 0x7f7a40085370 con 0x7f7a4815b640 2024-04-25T00:32:17.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4d4cf640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7a2c0678c0 0x7f7a2c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.767+0000 7f7a4d4cf640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7a2c0678c0 0x7f7a2c069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a380ae000 tx=0x7f7a38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:17.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.771+0000 7f7a3dffb640 1 -- 172.21.15.29:0/865304039 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7a40055e00 con 0x7f7a4815b640 2024-04-25T00:32:17.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.927+0000 7f7a4f733640 1 -- 172.21.15.29:0/865304039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7a48075300 con 0x7f7a4815b640 2024-04-25T00:32:17.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.931+0000 7f7a3dffb640 1 -- 172.21.15.29:0/865304039 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f7a40055700 con 0x7f7a4815b640 2024-04-25T00:32:17.933 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:17.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.931+0000 7f7a4f733640 1 -- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7a2c0678c0 msgr2=0x7f7a2c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.931+0000 7f7a4f733640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7a2c0678c0 0x7f7a2c069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a380ae000 tx=0x7f7a38008040 comp rx=0 tx=0).stop 2024-04-25T00:32:17.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.931+0000 7f7a4f733640 1 -- 172.21.15.29:0/865304039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a4815b640 msgr2=0x7f7a480c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.931+0000 7f7a4f733640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a4815b640 0x7f7a480c7d10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000c780 tx=0x7f7a4000cc40 comp rx=0 tx=0).stop 2024-04-25T00:32:17.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.931+0000 7f7a4f733640 1 -- 172.21.15.29:0/865304039 shutdown_connections 2024-04-25T00:32:17.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.931+0000 7f7a4f733640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a4815b640 0x7f7a480c7d10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.931+0000 7f7a4f733640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4815ad20 0x7f7a480c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.931+0000 7f7a4f733640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7a2c0678c0 0x7f7a2c069d80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.931+0000 7f7a4f733640 1 --2- 172.21.15.29:0/865304039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a480d0390 0x7f7a480ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.931+0000 7f7a4f733640 1 -- 172.21.15.29:0/865304039 >> 172.21.15.29:0/865304039 conn(0x7f7a480d0c40 msgr2=0x7f7a480b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:17.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.931+0000 7f7a4f733640 1 -- 172.21.15.29:0/865304039 shutdown_connections 2024-04-25T00:32:17.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:17.931+0000 7f7a4f733640 1 -- 172.21.15.29:0/865304039 wait complete. 2024-04-25T00:32:17.952 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:18.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f4758fab640 1 Processor -- start 2024-04-25T00:32:18.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f4758fab640 1 -- start start 2024-04-25T00:32:18.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f4758fab640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f475415e2a0 0x7f47540d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f4758fab640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47540d4d10 0x7f47540d2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f4758fab640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47540cf8a0 con 0x7f47540d2680 2024-04-25T00:32:18.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f4758fab640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f47540cf650 con 0x7f475415e2a0 2024-04-25T00:32:18.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f4758fab640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f47540b38d0 con 0x7f47540d4d10 2024-04-25T00:32:18.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f475259c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f475415e2a0 0x7f47540d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f475259c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f475415e2a0 0x7f47540d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42620/0 (socket says 172.21.15.29:42620) 2024-04-25T00:32:18.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f475259c640 1 -- 172.21.15.29:0/742789485 learned_addr learned my addr 172.21.15.29:0/742789485 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:18.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f4751d9b640 1 --2- 172.21.15.29:0/742789485 >> v2:172.21.15.157:3300/0 conn(0x7f47540d4d10 0x7f47540d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f475259c640 1 -- 172.21.15.29:0/742789485 >> v2:172.21.15.157:3300/0 conn(0x7f47540d4d10 msgr2=0x7f47540d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:18.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f475259c640 1 --2- 172.21.15.29:0/742789485 >> v2:172.21.15.157:3300/0 conn(0x7f47540d4d10 0x7f47540d2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f475259c640 1 -- 172.21.15.29:0/742789485 >> v1:172.21.15.29:6789/0 conn(0x7f47540d2680 legacy=0x7f47540d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.075+0000 7f475259c640 1 -- 172.21.15.29:0/742789485 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47540cfb80 con 0x7f475415e2a0 2024-04-25T00:32:18.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f475259c640 1 --2- 172.21.15.29:0/742789485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f475415e2a0 0x7f47540d1c00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4754075430 tx=0x7f474401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadf14760bf70892 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:18.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f475159a640 1 -- 172.21.15.29:0/742789485 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47440a5730 con 0x7f475415e2a0 2024-04-25T00:32:18.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f475159a640 1 -- 172.21.15.29:0/742789485 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47440a58d0 con 0x7f475415e2a0 2024-04-25T00:32:18.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f475159a640 1 -- 172.21.15.29:0/742789485 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47440a5ca0 con 0x7f475415e2a0 2024-04-25T00:32:18.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f4758fab640 1 -- 172.21.15.29:0/742789485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f475415e2a0 msgr2=0x7f47540d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f4758fab640 1 --2- 172.21.15.29:0/742789485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f475415e2a0 0x7f47540d1c00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4754075430 tx=0x7f474401c960 comp rx=0 tx=0).stop 2024-04-25T00:32:18.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f4758fab640 1 -- 172.21.15.29:0/742789485 shutdown_connections 2024-04-25T00:32:18.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f4758fab640 1 --2- 172.21.15.29:0/742789485 >> v2:172.21.15.157:3300/0 conn(0x7f47540d4d10 0x7f47540d2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f4758fab640 1 --2- 172.21.15.29:0/742789485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f475415e2a0 0x7f47540d1c00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f4758fab640 1 -- 172.21.15.29:0/742789485 >> 172.21.15.29:0/742789485 conn(0x7f47540d4030 msgr2=0x7f47540d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:18.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f4758fab640 1 -- 172.21.15.29:0/742789485 shutdown_connections 2024-04-25T00:32:18.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f4758fab640 1 -- 172.21.15.29:0/742789485 wait complete. 2024-04-25T00:32:18.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f4758fab640 1 Processor -- start 2024-04-25T00:32:18.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.079+0000 7f4758fab640 1 -- start start 2024-04-25T00:32:18.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4758fab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47540d2680 0x7f47541509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4758fab640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47540d4d10 0x7f4754152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4758fab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f475415e2a0 0x7f47540b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4758fab640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f475416b280 con 0x7f475415e2a0 2024-04-25T00:32:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4758fab640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f475416b100 con 0x7f47540d2680 2024-04-25T00:32:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4758fab640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f475416b400 con 0x7f47540d4d10 2024-04-25T00:32:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4752d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f475415e2a0 0x7f47540b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4752d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f475415e2a0 0x7f47540b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42630/0 (socket says 172.21.15.29:42630) 2024-04-25T00:32:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4752d9d640 1 -- 172.21.15.29:0/3442149728 learned_addr learned my addr 172.21.15.29:0/3442149728 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f475259c640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47540d2680 0x7f47541509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4751d9b640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47540d4d10 0x7f4754152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4752d9d640 1 -- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47540d4d10 msgr2=0x7f4754152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4752d9d640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47540d4d10 0x7f4754152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4752d9d640 1 -- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47540d2680 msgr2=0x7f47541509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4752d9d640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47540d2680 0x7f47541509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4752d9d640 1 -- 172.21.15.29:0/3442149728 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47541510c0 con 0x7f475415e2a0 2024-04-25T00:32:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4752d9d640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f475415e2a0 0x7f47540b1ea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f474c00c3c0 tx=0x7f474c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4742ffd640 1 -- 172.21.15.29:0/3442149728 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0052d0 con 0x7f475415e2a0 2024-04-25T00:32:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4742ffd640 1 -- 172.21.15.29:0/3442149728 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474c005470 con 0x7f475415e2a0 2024-04-25T00:32:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4758fab640 1 -- 172.21.15.29:0/3442149728 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47541513a0 con 0x7f475415e2a0 2024-04-25T00:32:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4742ffd640 1 -- 172.21.15.29:0/3442149728 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c007240 con 0x7f475415e2a0 2024-04-25T00:32:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4758fab640 1 -- 172.21.15.29:0/3442149728 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4754151960 con 0x7f475415e2a0 2024-04-25T00:32:18.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4758fab640 1 -- 172.21.15.29:0/3442149728 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4754069270 con 0x7f475415e2a0 2024-04-25T00:32:18.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4742ffd640 1 -- 172.21.15.29:0/3442149728 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f474c0073e0 con 0x7f475415e2a0 2024-04-25T00:32:18.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4742ffd640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f47340678c0 0x7f4734069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f4742ffd640 1 -- 172.21.15.29:0/3442149728 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5883+0+0 (secure 0 0 0) 0x7f474c0850e0 con 0x7f475415e2a0 2024-04-25T00:32:18.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.083+0000 7f475259c640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f47340678c0 0x7f4734069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.087+0000 7f475259c640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f47340678c0 0x7f4734069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f47440ae000 tx=0x7f47440a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:18.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.087+0000 7f4742ffd640 1 -- 172.21.15.29:0/3442149728 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f474c055b70 con 0x7f475415e2a0 2024-04-25T00:32:18.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.247+0000 7f4758fab640 1 -- 172.21.15.29:0/3442149728 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f47540d3e10 con 0x7f475415e2a0 2024-04-25T00:32:18.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.251+0000 7f4742ffd640 1 -- 172.21.15.29:0/3442149728 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f474c055470 con 0x7f475415e2a0 2024-04-25T00:32:18.253 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:18.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.251+0000 7f4758fab640 1 -- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f47340678c0 msgr2=0x7f4734069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.251+0000 7f4758fab640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f47340678c0 0x7f4734069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f47440ae000 tx=0x7f47440a7070 comp rx=0 tx=0).stop 2024-04-25T00:32:18.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.251+0000 7f4758fab640 1 -- 172.21.15.29:0/3442149728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f475415e2a0 msgr2=0x7f47540b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.251+0000 7f4758fab640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f475415e2a0 0x7f47540b1ea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f474c00c3c0 tx=0x7f474c00c880 comp rx=0 tx=0).stop 2024-04-25T00:32:18.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.251+0000 7f4758fab640 1 -- 172.21.15.29:0/3442149728 shutdown_connections 2024-04-25T00:32:18.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.251+0000 7f4758fab640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f475415e2a0 0x7f47540b1ea0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.251+0000 7f4758fab640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47540d4d10 0x7f4754152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.251+0000 7f4758fab640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f47340678c0 0x7f4734069d80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.251+0000 7f4758fab640 1 --2- 172.21.15.29:0/3442149728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47540d2680 0x7f47541509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.251+0000 7f4758fab640 1 -- 172.21.15.29:0/3442149728 >> 172.21.15.29:0/3442149728 conn(0x7f47540d4030 msgr2=0x7f475406c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:18.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.251+0000 7f4758fab640 1 -- 172.21.15.29:0/3442149728 shutdown_connections 2024-04-25T00:32:18.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.251+0000 7f4758fab640 1 -- 172.21.15.29:0/3442149728 wait complete. 2024-04-25T00:32:18.272 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:18.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f48726d0640 1 Processor -- start 2024-04-25T00:32:18.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f48726d0640 1 -- start start 2024-04-25T00:32:18.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f48726d0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f486c15c220 0x7f486c0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f48726d0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f486c0d0670 0x7f486c0b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f48726d0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486c0cdbd0 con 0x7f486c0cfa10 2024-04-25T00:32:18.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f48726d0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f486c15c600 con 0x7f486c15c220 2024-04-25T00:32:18.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f48726d0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f486c0b1a30 con 0x7f486c0d0670 2024-04-25T00:32:18.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f4870c6d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f486c0d0670 0x7f486c0b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f486bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f486c15c220 0x7f486c0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f4870c6d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f486c0d0670 0x7f486c0b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:57078/0 (socket says 172.21.15.29:57078) 2024-04-25T00:32:18.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f486bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f486c15c220 0x7f486c0cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:44634/0 (socket says 172.21.15.29:44634) 2024-04-25T00:32:18.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f4870c6d640 1 -- 172.21.15.29:0/1334933413 learned_addr learned my addr 172.21.15.29:0/1334933413 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:18.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f486b7fe640 1 --1- 172.21.15.29:0/1334933413 >> v1:172.21.15.157:6789/0 conn(0x7f486c0cfa10 0x7f486c0cff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:53266/0 (socket says 172.21.15.29:53266) 2024-04-25T00:32:18.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f486bfff640 1 -- 172.21.15.29:0/1334933413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f486c0d0670 msgr2=0x7f486c0b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:18.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f486bfff640 1 --2- 172.21.15.29:0/1334933413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f486c0d0670 0x7f486c0b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f486bfff640 1 -- 172.21.15.29:0/1334933413 >> v1:172.21.15.157:6789/0 conn(0x7f486c0cfa10 legacy=0x7f486c0cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.395+0000 7f486bfff640 1 -- 172.21.15.29:0/1334933413 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c0b13e0 con 0x7f486c15c220 2024-04-25T00:32:18.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f486bfff640 1 --2- 172.21.15.29:0/1334933413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f486c15c220 0x7f486c0cd350 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f485c0097c0 tx=0x7f485c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe7c60403538ab52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:18.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f486affd640 1 -- 172.21.15.29:0/1334933413 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0b0070 con 0x7f486c15c220 2024-04-25T00:32:18.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f486affd640 1 -- 172.21.15.29:0/1334933413 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f485c0ab600 con 0x7f486c15c220 2024-04-25T00:32:18.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f486affd640 1 -- 172.21.15.29:0/1334933413 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0ab9d0 con 0x7f486c15c220 2024-04-25T00:32:18.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 -- 172.21.15.29:0/1334933413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f486c15c220 msgr2=0x7f486c0cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 --2- 172.21.15.29:0/1334933413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f486c15c220 0x7f486c0cd350 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f485c0097c0 tx=0x7f485c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 -- 172.21.15.29:0/1334933413 shutdown_connections 2024-04-25T00:32:18.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 --2- 172.21.15.29:0/1334933413 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f486c0d0670 0x7f486c0b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 --2- 172.21.15.29:0/1334933413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f486c15c220 0x7f486c0cd350 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 -- 172.21.15.29:0/1334933413 >> 172.21.15.29:0/1334933413 conn(0x7f486c0d1d60 msgr2=0x7f486c0d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:18.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 -- 172.21.15.29:0/1334933413 shutdown_connections 2024-04-25T00:32:18.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 -- 172.21.15.29:0/1334933413 wait complete. 2024-04-25T00:32:18.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 Processor -- start 2024-04-25T00:32:18.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 -- start start 2024-04-25T00:32:18.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f486c0cfa10 0x7f486c14e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f486c0d0670 0x7f486c0c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f486c15c220 0x7f486c156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f486c0b2fa0 con 0x7f486c15c220 2024-04-25T00:32:18.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f486c0b2e20 con 0x7f486c0cfa10 2024-04-25T00:32:18.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f48726d0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f486c0b3120 con 0x7f486c0d0670 2024-04-25T00:32:18.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.399+0000 7f4870c6d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f486c15c220 0x7f486c156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4870c6d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f486c15c220 0x7f486c156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42632/0 (socket says 172.21.15.29:42632) 2024-04-25T00:32:18.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4870c6d640 1 -- 172.21.15.29:0/2421437530 learned_addr learned my addr 172.21.15.29:0/2421437530 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:18.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f486bfff640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f486c0cfa10 0x7f486c14e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f486b7fe640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f486c0d0670 0x7f486c0c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4870c6d640 1 -- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f486c0d0670 msgr2=0x7f486c0c56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4870c6d640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f486c0d0670 0x7f486c0c56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4870c6d640 1 -- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f486c0cfa10 msgr2=0x7f486c14e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4870c6d640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f486c0cfa10 0x7f486c14e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4870c6d640 1 -- 172.21.15.29:0/2421437530 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c155a00 con 0x7f486c15c220 2024-04-25T00:32:18.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f486bfff640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f486c0cfa10 0x7f486c14e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:18.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4870c6d640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f486c15c220 0x7f486c156870 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f486400efc0 tx=0x7f486400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:18.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4868ff9640 1 -- 172.21.15.29:0/2421437530 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864009e60 con 0x7f486c15c220 2024-04-25T00:32:18.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4868ff9640 1 -- 172.21.15.29:0/2421437530 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486400cd20 con 0x7f486c15c220 2024-04-25T00:32:18.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f48726d0640 1 -- 172.21.15.29:0/2421437530 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c155ce0 con 0x7f486c15c220 2024-04-25T00:32:18.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4868ff9640 1 -- 172.21.15.29:0/2421437530 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864003be0 con 0x7f486c15c220 2024-04-25T00:32:18.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f48726d0640 1 -- 172.21.15.29:0/2421437530 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c14f8d0 con 0x7f486c15c220 2024-04-25T00:32:18.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f48726d0640 1 -- 172.21.15.29:0/2421437530 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486c069270 con 0x7f486c15c220 2024-04-25T00:32:18.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4868ff9640 1 -- 172.21.15.29:0/2421437530 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4864012050 con 0x7f486c15c220 2024-04-25T00:32:18.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4868ff9640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f48580678c0 0x7f4858069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f4868ff9640 1 -- 172.21.15.29:0/2421437530 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5883+0+0 (secure 0 0 0) 0x7f48640860e0 con 0x7f486c15c220 2024-04-25T00:32:18.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f486bfff640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f48580678c0 0x7f4858069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.403+0000 7f486bfff640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f48580678c0 0x7f4858069d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f485c0ad000 tx=0x7f485c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:18.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.407+0000 7f4868ff9640 1 -- 172.21.15.29:0/2421437530 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4864010040 con 0x7f486c15c220 2024-04-25T00:32:18.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.559+0000 7f48726d0640 1 -- 172.21.15.29:0/2421437530 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f486c068d90 con 0x7f486c15c220 2024-04-25T00:32:18.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.563+0000 7f4868ff9640 1 -- 172.21.15.29:0/2421437530 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f4864056c00 con 0x7f486c15c220 2024-04-25T00:32:18.566 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:18.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.563+0000 7f48726d0640 1 -- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f48580678c0 msgr2=0x7f4858069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.563+0000 7f48726d0640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f48580678c0 0x7f4858069d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f485c0ad000 tx=0x7f485c008040 comp rx=0 tx=0).stop 2024-04-25T00:32:18.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.563+0000 7f48726d0640 1 -- 172.21.15.29:0/2421437530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f486c15c220 msgr2=0x7f486c156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.563+0000 7f48726d0640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f486c15c220 0x7f486c156870 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f486400efc0 tx=0x7f486400c490 comp rx=0 tx=0).stop 2024-04-25T00:32:18.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.563+0000 7f48726d0640 1 -- 172.21.15.29:0/2421437530 shutdown_connections 2024-04-25T00:32:18.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.563+0000 7f48726d0640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f486c15c220 0x7f486c156870 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.563+0000 7f48726d0640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f486c0d0670 0x7f486c0c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.563+0000 7f48726d0640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f48580678c0 0x7f4858069d80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.563+0000 7f48726d0640 1 --2- 172.21.15.29:0/2421437530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f486c0cfa10 0x7f486c14e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.563+0000 7f48726d0640 1 -- 172.21.15.29:0/2421437530 >> 172.21.15.29:0/2421437530 conn(0x7f486c0d1d60 msgr2=0x7f486c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:18.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.563+0000 7f48726d0640 1 -- 172.21.15.29:0/2421437530 shutdown_connections 2024-04-25T00:32:18.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.563+0000 7f48726d0640 1 -- 172.21.15.29:0/2421437530 wait complete. 2024-04-25T00:32:18.584 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:32:18.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba817ca640 1 Processor -- start 2024-04-25T00:32:18.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba817ca640 1 -- start start 2024-04-25T00:32:18.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba817ca640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fba7c0b3140 0x7fba7c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba817ca640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba7c0b3c30 0x7fba7c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba817ca640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba7c0b41e0 con 0x7fba7c0b4b10 2024-04-25T00:32:18.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba817ca640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fba7c06d960 con 0x7fba7c0b3140 2024-04-25T00:32:18.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba817ca640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c06dc40 con 0x7fba7c0b3c30 2024-04-25T00:32:18.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba7b7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba7c0b3c30 0x7fba7c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba7affd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fba7c0b3140 0x7fba7c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba7a7fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fba7c0b4b10 0x7fba7c0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:53278/0 (socket says 172.21.15.29:53278) 2024-04-25T00:32:18.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba7a7fc640 1 -- 172.21.15.29:0/2181983501 learned_addr learned my addr 172.21.15.29:0/2181983501 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:18.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba7affd640 1 -- 172.21.15.29:0/2181983501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba7c0b3c30 msgr2=0x7fba7c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:18.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba7affd640 1 --2- 172.21.15.29:0/2181983501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba7c0b3c30 0x7fba7c0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba7affd640 1 -- 172.21.15.29:0/2181983501 >> v1:172.21.15.157:6789/0 conn(0x7fba7c0b4b10 legacy=0x7fba7c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.711+0000 7fba7affd640 1 -- 172.21.15.29:0/2181983501 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c06df20 con 0x7fba7c0b3140 2024-04-25T00:32:18.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.715+0000 7fba7affd640 1 --2- 172.21.15.29:0/2181983501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7c0b3140 0x7fba7c06e440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0097f0 tx=0x7fba6c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a21b09e2b25204 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:18.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.715+0000 7fba79ffb640 1 -- 172.21.15.29:0/2181983501 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0b0070 con 0x7fba7c0b3140 2024-04-25T00:32:18.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.715+0000 7fba79ffb640 1 -- 172.21.15.29:0/2181983501 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c0ab600 con 0x7fba7c0b3140 2024-04-25T00:32:18.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.715+0000 7fba79ffb640 1 -- 172.21.15.29:0/2181983501 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0ab9d0 con 0x7fba7c0b3140 2024-04-25T00:32:18.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.715+0000 7fba817ca640 1 -- 172.21.15.29:0/2181983501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7c0b3140 msgr2=0x7fba7c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.715+0000 7fba817ca640 1 --2- 172.21.15.29:0/2181983501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7c0b3140 0x7fba7c06e440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0097f0 tx=0x7fba6c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.715+0000 7fba817ca640 1 -- 172.21.15.29:0/2181983501 shutdown_connections 2024-04-25T00:32:18.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.715+0000 7fba817ca640 1 --2- 172.21.15.29:0/2181983501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba7c0b3c30 0x7fba7c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.715+0000 7fba817ca640 1 --2- 172.21.15.29:0/2181983501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7c0b3140 0x7fba7c06e440 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.715+0000 7fba817ca640 1 -- 172.21.15.29:0/2181983501 >> 172.21.15.29:0/2181983501 conn(0x7fba7c0b1870 msgr2=0x7fba7c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:18.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.715+0000 7fba817ca640 1 -- 172.21.15.29:0/2181983501 shutdown_connections 2024-04-25T00:32:18.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.715+0000 7fba817ca640 1 -- 172.21.15.29:0/2181983501 wait complete. 2024-04-25T00:32:18.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba817ca640 1 Processor -- start 2024-04-25T00:32:18.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba817ca640 1 -- start start 2024-04-25T00:32:18.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba817ca640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba7c0b3140 0x7fba7c15e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba817ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c0b3c30 0x7fba7c15b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba817ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7c0b4b10 0x7fba7c141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba817ca640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c168d80 con 0x7fba7c0b3c30 2024-04-25T00:32:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba817ca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c168c00 con 0x7fba7c0b4b10 2024-04-25T00:32:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba817ca640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c168f00 con 0x7fba7c0b3140 2024-04-25T00:32:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba7a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c0b3c30 0x7fba7c15b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba7a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c0b3c30 0x7fba7c15b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42634/0 (socket says 172.21.15.29:42634) 2024-04-25T00:32:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba7a7fc640 1 -- 172.21.15.29:0/4216094043 learned_addr learned my addr 172.21.15.29:0/4216094043 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba7affd640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba7c0b3140 0x7fba7c15e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba7b7fe640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7c0b4b10 0x7fba7c141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba7a7fc640 1 -- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba7c0b3140 msgr2=0x7fba7c15e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba7a7fc640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba7c0b3140 0x7fba7c15e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba7a7fc640 1 -- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7c0b4b10 msgr2=0x7fba7c141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba7a7fc640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7c0b4b10 0x7fba7c141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba7a7fc640 1 -- 172.21.15.29:0/4216094043 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c15ebb0 con 0x7fba7c0b3c30 2024-04-25T00:32:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba7affd640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba7c0b3140 0x7fba7c15e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba7b7fe640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7c0b4b10 0x7fba7c141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba7a7fc640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c0b3c30 0x7fba7c15b9e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fba7000c530 tx=0x7fba7000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba637fe640 1 -- 172.21.15.29:0/4216094043 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70017070 con 0x7fba7c0b3c30 2024-04-25T00:32:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba637fe640 1 -- 172.21.15.29:0/4216094043 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba70005cf0 con 0x7fba7c0b3c30 2024-04-25T00:32:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba637fe640 1 -- 172.21.15.29:0/4216094043 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70007270 con 0x7fba7c0b3c30 2024-04-25T00:32:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba817ca640 1 -- 172.21.15.29:0/4216094043 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c15d5b0 con 0x7fba7c0b3c30 2024-04-25T00:32:18.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba817ca640 1 -- 172.21.15.29:0/4216094043 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c15db70 con 0x7fba7c0b3c30 2024-04-25T00:32:18.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.719+0000 7fba817ca640 1 -- 172.21.15.29:0/4216094043 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7c069270 con 0x7fba7c0b3c30 2024-04-25T00:32:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.723+0000 7fba637fe640 1 -- 172.21.15.29:0/4216094043 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fba7001a070 con 0x7fba7c0b3c30 2024-04-25T00:32:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.723+0000 7fba637fe640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba680678c0 0x7fba68069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.723+0000 7fba637fe640 1 -- 172.21.15.29:0/4216094043 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fba70084fa0 con 0x7fba7c0b3c30 2024-04-25T00:32:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.723+0000 7fba7affd640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba680678c0 0x7fba68069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.723+0000 7fba7affd640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba680678c0 0x7fba68069d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0ad000 tx=0x7fba6c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:18.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.727+0000 7fba637fe640 1 -- 172.21.15.29:0/4216094043 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fba700553c0 con 0x7fba7c0b3c30 2024-04-25T00:32:18.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.903+0000 7fba817ca640 1 -- 172.21.15.29:0/4216094043 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fba7c154db0 con 0x7fba7c0b3c30 2024-04-25T00:32:18.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.903+0000 7fba637fe640 1 -- 172.21.15.29:0/4216094043 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fba700596c0 con 0x7fba7c0b3c30 2024-04-25T00:32:18.909 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:32:18.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.907+0000 7fba817ca640 1 -- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba680678c0 msgr2=0x7fba68069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.907+0000 7fba817ca640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba680678c0 0x7fba68069d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0ad000 tx=0x7fba6c008040 comp rx=0 tx=0).stop 2024-04-25T00:32:18.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.907+0000 7fba817ca640 1 -- 172.21.15.29:0/4216094043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c0b3c30 msgr2=0x7fba7c15b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.907+0000 7fba817ca640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c0b3c30 0x7fba7c15b9e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fba7000c530 tx=0x7fba7000c9f0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.907+0000 7fba817ca640 1 -- 172.21.15.29:0/4216094043 shutdown_connections 2024-04-25T00:32:18.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.907+0000 7fba817ca640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba680678c0 0x7fba68069d80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.907+0000 7fba817ca640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7c0b4b10 0x7fba7c141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.907+0000 7fba817ca640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c0b3c30 0x7fba7c15b9e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.907+0000 7fba817ca640 1 --2- 172.21.15.29:0/4216094043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba7c0b3140 0x7fba7c15e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.907+0000 7fba817ca640 1 -- 172.21.15.29:0/4216094043 >> 172.21.15.29:0/4216094043 conn(0x7fba7c0b1870 msgr2=0x7fba7c15f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:18.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.907+0000 7fba817ca640 1 -- 172.21.15.29:0/4216094043 shutdown_connections 2024-04-25T00:32:18.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:18.907+0000 7fba817ca640 1 -- 172.21.15.29:0/4216094043 wait complete. 2024-04-25T00:32:18.930 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:32:18.930 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:18.937 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:19.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.051+0000 7f99bf842640 1 Processor -- start 2024-04-25T00:32:19.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.051+0000 7f99bf842640 1 -- start start 2024-04-25T00:32:19.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.051+0000 7f99bf842640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f99b80aea40 0x7f99b80aee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.051+0000 7f99bf842640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99b80af770 con 0x7f99b815c290 2024-04-25T00:32:19.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.051+0000 7f99bf842640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99b80afa50 con 0x7f99b80d1c10 2024-04-25T00:32:19.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.051+0000 7f99bf842640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f99b80b0380 con 0x7f99b80aea40 2024-04-25T00:32:19.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.051+0000 7f99bdddf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f99b80aea40 0x7f99b80aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.051+0000 7f99bcddd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f99b815c290 0x7f99b80cc770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38394/0 (socket says 172.21.15.29:38394) 2024-04-25T00:32:19.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.051+0000 7f99bdddf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f99b80aea40 0x7f99b80aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42636/0 (socket says 172.21.15.29:42636) 2024-04-25T00:32:19.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.051+0000 7f99bcddd640 1 -- 172.21.15.29:0/1795118488 learned_addr learned my addr 172.21.15.29:0/1795118488 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:19.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99bdddf640 1 -- 172.21.15.29:0/1795118488 >> v1:172.21.15.157:6789/0 conn(0x7f99b80d1c10 legacy=0x7f99b80d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99bdddf640 1 -- 172.21.15.29:0/1795118488 >> v1:172.21.15.29:6789/0 conn(0x7f99b815c290 legacy=0x7f99b80cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99bdddf640 1 -- 172.21.15.29:0/1795118488 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b80afd30 con 0x7f99b80aea40 2024-04-25T00:32:19.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99bdddf640 1 --2- 172.21.15.29:0/1795118488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99b80aea40 0x7f99b80aee90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009d90 tx=0x7f99b001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a36208d06706a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:19.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99affff640 1 -- 172.21.15.29:0/1795118488 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b00b2070 con 0x7f99b80aea40 2024-04-25T00:32:19.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99affff640 1 -- 172.21.15.29:0/1795118488 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b00a5930 con 0x7f99b80aea40 2024-04-25T00:32:19.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99affff640 1 -- 172.21.15.29:0/1795118488 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b00a5d00 con 0x7f99b80aea40 2024-04-25T00:32:19.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99bf842640 1 -- 172.21.15.29:0/1795118488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99b80aea40 msgr2=0x7f99b80aee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99bf842640 1 --2- 172.21.15.29:0/1795118488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99b80aea40 0x7f99b80aee90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009d90 tx=0x7f99b001c940 comp rx=0 tx=0).stop 2024-04-25T00:32:19.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99bf842640 1 -- 172.21.15.29:0/1795118488 shutdown_connections 2024-04-25T00:32:19.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99bf842640 1 --2- 172.21.15.29:0/1795118488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99b80aea40 0x7f99b80aee90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99bf842640 1 -- 172.21.15.29:0/1795118488 >> 172.21.15.29:0/1795118488 conn(0x7f99b809aa30 msgr2=0x7f99b809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:19.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99bf842640 1 -- 172.21.15.29:0/1795118488 shutdown_connections 2024-04-25T00:32:19.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.055+0000 7f99bf842640 1 -- 172.21.15.29:0/1795118488 wait complete. 2024-04-25T00:32:19.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bf842640 1 Processor -- start 2024-04-25T00:32:19.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bf842640 1 -- start start 2024-04-25T00:32:19.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bf842640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99b80aea40 0x7f99b80c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bf842640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b80d1c10 0x7f99b80c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bf842640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99b815c290 0x7f99b815e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bf842640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f99b80b1410 con 0x7f99b815c290 2024-04-25T00:32:19.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bf842640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99b80b1290 con 0x7f99b80d1c10 2024-04-25T00:32:19.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bf842640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f99b8169c10 con 0x7f99b80aea40 2024-04-25T00:32:19.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bdddf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99b815c290 0x7f99b815e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bdddf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99b815c290 0x7f99b815e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42642/0 (socket says 172.21.15.29:42642) 2024-04-25T00:32:19.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bdddf640 1 -- 172.21.15.29:0/501221378 learned_addr learned my addr 172.21.15.29:0/501221378 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:19.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bcddd640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b80d1c10 0x7f99b80c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bd5de640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99b80aea40 0x7f99b80c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bdddf640 1 -- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99b80aea40 msgr2=0x7f99b80c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bdddf640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99b80aea40 0x7f99b80c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bdddf640 1 -- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b80d1c10 msgr2=0x7f99b80c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bdddf640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b80d1c10 0x7f99b80c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bdddf640 1 -- 172.21.15.29:0/501221378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b80c9df0 con 0x7f99b815c290 2024-04-25T00:32:19.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bdddf640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99b815c290 0x7f99b815e330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f99b00b0040 tx=0x7f99b00196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:19.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99adffb640 1 -- 172.21.15.29:0/501221378 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b00b2050 con 0x7f99b815c290 2024-04-25T00:32:19.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99adffb640 1 -- 172.21.15.29:0/501221378 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b00be070 con 0x7f99b815c290 2024-04-25T00:32:19.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bf842640 1 -- 172.21.15.29:0/501221378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b815d5d0 con 0x7f99b815c290 2024-04-25T00:32:19.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99adffb640 1 -- 172.21.15.29:0/501221378 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b00adb90 con 0x7f99b815c290 2024-04-25T00:32:19.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.059+0000 7f99bf842640 1 -- 172.21.15.29:0/501221378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b815d870 con 0x7f99b815c290 2024-04-25T00:32:19.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.063+0000 7f99bf842640 1 -- 172.21.15.29:0/501221378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b8069270 con 0x7f99b815c290 2024-04-25T00:32:19.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.063+0000 7f99adffb640 1 -- 172.21.15.29:0/501221378 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f99b00b9450 con 0x7f99b815c290 2024-04-25T00:32:19.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.063+0000 7f99adffb640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f999c0678c0 0x7f999c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.063+0000 7f99adffb640 1 -- 172.21.15.29:0/501221378 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f99b0121c80 con 0x7f99b815c290 2024-04-25T00:32:19.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.063+0000 7f99bd5de640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f999c0678c0 0x7f999c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.063+0000 7f99bd5de640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f999c0678c0 0x7f999c069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99a8001650 tx=0x7f99a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:19.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.067+0000 7f99adffb640 1 -- 172.21.15.29:0/501221378 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f99b0016560 con 0x7f99b815c290 2024-04-25T00:32:19.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.227+0000 7f99bf842640 1 -- 172.21.15.29:0/501221378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99b8156cc0 con 0x7f99b815c290 2024-04-25T00:32:19.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.231+0000 7f99adffb640 1 -- 172.21.15.29:0/501221378 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16838 (secure 0 0 0) 0x7f99b00f2840 con 0x7f99b815c290 2024-04-25T00:32:19.234 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:19.234 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":22,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:32:18.667140+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:19.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.231+0000 7f99bf842640 1 -- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f999c0678c0 msgr2=0x7f999c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.231+0000 7f99bf842640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f999c0678c0 0x7f999c069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99a8001650 tx=0x7f99a8009040 comp rx=0 tx=0).stop 2024-04-25T00:32:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.231+0000 7f99bf842640 1 -- 172.21.15.29:0/501221378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99b815c290 msgr2=0x7f99b815e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.231+0000 7f99bf842640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99b815c290 0x7f99b815e330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f99b00b0040 tx=0x7f99b00196e0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.231+0000 7f99bf842640 1 -- 172.21.15.29:0/501221378 shutdown_connections 2024-04-25T00:32:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.231+0000 7f99bf842640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99b815c290 0x7f99b815e330 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.231+0000 7f99bf842640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f999c0678c0 0x7f999c069d80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.231+0000 7f99bf842640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b80d1c10 0x7f99b80c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.231+0000 7f99bf842640 1 --2- 172.21.15.29:0/501221378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99b80aea40 0x7f99b80c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.231+0000 7f99bf842640 1 -- 172.21.15.29:0/501221378 >> 172.21.15.29:0/501221378 conn(0x7f99b809aa30 msgr2=0x7f99b80b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.231+0000 7f99bf842640 1 -- 172.21.15.29:0/501221378 shutdown_connections 2024-04-25T00:32:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.231+0000 7f99bf842640 1 -- 172.21.15.29:0/501221378 wait complete. 2024-04-25T00:32:19.252 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:19.253 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:19.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f6788edd640 1 Processor -- start 2024-04-25T00:32:19.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f6788edd640 1 -- start start 2024-04-25T00:32:19.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f6788edd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f678415c190 0x7f67840d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f6788edd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67840ccf30 0x7f67840d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f6788edd640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67840cd3e0 con 0x7f678415bdb0 2024-04-25T00:32:19.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f6788edd640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f67840cd6c0 con 0x7f678415c190 2024-04-25T00:32:19.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f6788edd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f67840cd9a0 con 0x7f67840ccf30 2024-04-25T00:32:19.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f6781d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f678415c190 0x7f67840d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f678259c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f678415bdb0 0x7f67840cc820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38396/0 (socket says 172.21.15.29:38396) 2024-04-25T00:32:19.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f678259c640 1 -- 172.21.15.29:0/969257930 learned_addr learned my addr 172.21.15.29:0/969257930 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:19.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f6782d9d640 1 --2- 172.21.15.29:0/969257930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67840ccf30 0x7f67840d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f6781d9b640 1 -- 172.21.15.29:0/969257930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67840ccf30 msgr2=0x7f67840d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:19.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f6781d9b640 1 --2- 172.21.15.29:0/969257930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67840ccf30 0x7f67840d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f6781d9b640 1 -- 172.21.15.29:0/969257930 >> v1:172.21.15.29:6789/0 conn(0x7f678415bdb0 legacy=0x7f67840cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.379+0000 7f6781d9b640 1 -- 172.21.15.29:0/969257930 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67840cdc80 con 0x7f678415c190 2024-04-25T00:32:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6781d9b640 1 --2- 172.21.15.29:0/969257930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f678415c190 0x7f67840d02d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f67780097c0 tx=0x7f677801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e87c4654cc01d67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f678159a640 1 -- 172.21.15.29:0/969257930 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67780b1070 con 0x7f678415c190 2024-04-25T00:32:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f678159a640 1 -- 172.21.15.29:0/969257930 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67780ac600 con 0x7f678415c190 2024-04-25T00:32:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f678159a640 1 -- 172.21.15.29:0/969257930 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67780ac9d0 con 0x7f678415c190 2024-04-25T00:32:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 -- 172.21.15.29:0/969257930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f678415c190 msgr2=0x7f67840d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 --2- 172.21.15.29:0/969257930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f678415c190 0x7f67840d02d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f67780097c0 tx=0x7f677801c3e0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 -- 172.21.15.29:0/969257930 shutdown_connections 2024-04-25T00:32:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 --2- 172.21.15.29:0/969257930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67840ccf30 0x7f67840d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 --2- 172.21.15.29:0/969257930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f678415c190 0x7f67840d02d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 -- 172.21.15.29:0/969257930 >> 172.21.15.29:0/969257930 conn(0x7f678409aa30 msgr2=0x7f678409ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:19.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 -- 172.21.15.29:0/969257930 shutdown_connections 2024-04-25T00:32:19.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 -- 172.21.15.29:0/969257930 wait complete. 2024-04-25T00:32:19.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 Processor -- start 2024-04-25T00:32:19.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 -- start start 2024-04-25T00:32:19.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67840ccf30 0x7f6784159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678415bdb0 0x7f67840b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67841582d0 0x7f6784153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f67840c6400 con 0x7f67840ccf30 2024-04-25T00:32:19.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f67840c6280 con 0x7f678415bdb0 2024-04-25T00:32:19.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6788edd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f67840c6580 con 0x7f67841582d0 2024-04-25T00:32:19.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f678259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67840ccf30 0x7f6784159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f678259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67840ccf30 0x7f6784159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42662/0 (socket says 172.21.15.29:42662) 2024-04-25T00:32:19.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f678259c640 1 -- 172.21.15.29:0/2477468412 learned_addr learned my addr 172.21.15.29:0/2477468412 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:19.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6781d9b640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678415bdb0 0x7f67840b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6782d9d640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67841582d0 0x7f6784153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f678259c640 1 -- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67841582d0 msgr2=0x7f6784153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f678259c640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67841582d0 0x7f6784153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f678259c640 1 -- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678415bdb0 msgr2=0x7f67840b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f678259c640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678415bdb0 0x7f67840b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f678259c640 1 -- 172.21.15.29:0/2477468412 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67841587d0 con 0x7f67840ccf30 2024-04-25T00:32:19.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.383+0000 7f6781d9b640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678415bdb0 0x7f67840b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:19.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.387+0000 7f678259c640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67840ccf30 0x7f6784159870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f677400c450 tx=0x7f677400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:19.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.387+0000 7f6772ffd640 1 -- 172.21.15.29:0/2477468412 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774017070 con 0x7f67840ccf30 2024-04-25T00:32:19.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.387+0000 7f6772ffd640 1 -- 172.21.15.29:0/2477468412 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6774005cf0 con 0x7f67840ccf30 2024-04-25T00:32:19.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.387+0000 7f6788edd640 1 -- 172.21.15.29:0/2477468412 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6784158ab0 con 0x7f67840ccf30 2024-04-25T00:32:19.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.387+0000 7f6772ffd640 1 -- 172.21.15.29:0/2477468412 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774007290 con 0x7f67840ccf30 2024-04-25T00:32:19.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.387+0000 7f6788edd640 1 -- 172.21.15.29:0/2477468412 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6784159070 con 0x7f67840ccf30 2024-04-25T00:32:19.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.387+0000 7f6788edd640 1 -- 172.21.15.29:0/2477468412 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6784069270 con 0x7f67840ccf30 2024-04-25T00:32:19.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.387+0000 7f6772ffd640 1 -- 172.21.15.29:0/2477468412 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f677401a070 con 0x7f67840ccf30 2024-04-25T00:32:19.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.387+0000 7f6772ffd640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f67640678c0 0x7f6764069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.387+0000 7f6772ffd640 1 -- 172.21.15.29:0/2477468412 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f6774085150 con 0x7f67840ccf30 2024-04-25T00:32:19.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.387+0000 7f6781d9b640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f67640678c0 0x7f6764069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.387+0000 7f6781d9b640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f67640678c0 0x7f6764069d80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f67780ae000 tx=0x7f67780a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:19.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.391+0000 7f6772ffd640 1 -- 172.21.15.29:0/2477468412 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6774055b40 con 0x7f67840ccf30 2024-04-25T00:32:19.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6788edd640 1 -- 172.21.15.29:0/2477468412 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f678415a080 con 0x7f67840ccf30 2024-04-25T00:32:19.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6772ffd640 1 -- 172.21.15.29:0/2477468412 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f6774055440 con 0x7f67840ccf30 2024-04-25T00:32:19.558 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:19.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6788edd640 1 -- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f67640678c0 msgr2=0x7f6764069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6788edd640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f67640678c0 0x7f6764069d80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f67780ae000 tx=0x7f67780a7000 comp rx=0 tx=0).stop 2024-04-25T00:32:19.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6788edd640 1 -- 172.21.15.29:0/2477468412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67840ccf30 msgr2=0x7f6784159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6788edd640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67840ccf30 0x7f6784159870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f677400c450 tx=0x7f677400c910 comp rx=0 tx=0).stop 2024-04-25T00:32:19.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6788edd640 1 -- 172.21.15.29:0/2477468412 shutdown_connections 2024-04-25T00:32:19.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6788edd640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67841582d0 0x7f6784153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6788edd640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f67640678c0 0x7f6764069d80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6788edd640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f678415bdb0 0x7f67840b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6788edd640 1 --2- 172.21.15.29:0/2477468412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67840ccf30 0x7f6784159870 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6788edd640 1 -- 172.21.15.29:0/2477468412 >> 172.21.15.29:0/2477468412 conn(0x7f678409aa30 msgr2=0x7f678406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:19.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6788edd640 1 -- 172.21.15.29:0/2477468412 shutdown_connections 2024-04-25T00:32:19.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.555+0000 7f6788edd640 1 -- 172.21.15.29:0/2477468412 wait complete. 2024-04-25T00:32:19.578 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:19.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb5a078a640 1 Processor -- start 2024-04-25T00:32:19.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb5a078a640 1 -- start start 2024-04-25T00:32:19.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb5a078a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb59815c190 0x7fb5980d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb5a078a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb5980ccf30 0x7fb5980d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb5a078a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5980cd3e0 con 0x7fb59815bdb0 2024-04-25T00:32:19.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb5a078a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb5980cd6c0 con 0x7fb5980ccf30 2024-04-25T00:32:19.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb5a078a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb5980cd9a0 con 0x7fb59815c190 2024-04-25T00:32:19.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb59ed27640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb5980ccf30 0x7fb5980d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb59ed27640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb5980ccf30 0x7fb5980d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42678/0 (socket says 172.21.15.29:42678) 2024-04-25T00:32:19.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb59ed27640 1 -- 172.21.15.29:0/468729613 learned_addr learned my addr 172.21.15.29:0/468729613 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:19.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb59dd25640 1 --2- 172.21.15.29:0/468729613 >> v2:172.21.15.157:3300/0 conn(0x7fb59815c190 0x7fb5980d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb59ed27640 1 -- 172.21.15.29:0/468729613 >> v2:172.21.15.157:3300/0 conn(0x7fb59815c190 msgr2=0x7fb5980d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:19.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb59ed27640 1 --2- 172.21.15.29:0/468729613 >> v2:172.21.15.157:3300/0 conn(0x7fb59815c190 0x7fb5980d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb59ed27640 1 -- 172.21.15.29:0/468729613 >> v1:172.21.15.157:6789/0 conn(0x7fb59815bdb0 legacy=0x7fb5980cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:19.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb59ed27640 1 -- 172.21.15.29:0/468729613 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5980cdc80 con 0x7fb5980ccf30 2024-04-25T00:32:19.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.699+0000 7fb59dd25640 1 --2- 172.21.15.29:0/468729613 >> v2:172.21.15.157:3300/0 conn(0x7fb59815c190 0x7fb5980d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:19.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb59ed27640 1 --2- 172.21.15.29:0/468729613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5980ccf30 0x7fb5980d0810 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb5980cc360 tx=0x7fb59001c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9ddd0c37c05fae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:19.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb59d524640 1 -- 172.21.15.29:0/468729613 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5900a5770 con 0x7fb5980ccf30 2024-04-25T00:32:19.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb59d524640 1 -- 172.21.15.29:0/468729613 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5900a5910 con 0x7fb5980ccf30 2024-04-25T00:32:19.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb59d524640 1 -- 172.21.15.29:0/468729613 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5900a5ce0 con 0x7fb5980ccf30 2024-04-25T00:32:19.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 -- 172.21.15.29:0/468729613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5980ccf30 msgr2=0x7fb5980d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 --2- 172.21.15.29:0/468729613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5980ccf30 0x7fb5980d0810 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb5980cc360 tx=0x7fb59001c8f0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 -- 172.21.15.29:0/468729613 shutdown_connections 2024-04-25T00:32:19.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 --2- 172.21.15.29:0/468729613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5980ccf30 0x7fb5980d0810 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 --2- 172.21.15.29:0/468729613 >> v2:172.21.15.157:3300/0 conn(0x7fb59815c190 0x7fb5980d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 -- 172.21.15.29:0/468729613 >> 172.21.15.29:0/468729613 conn(0x7fb59809aa30 msgr2=0x7fb59809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:19.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 -- 172.21.15.29:0/468729613 shutdown_connections 2024-04-25T00:32:19.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 -- 172.21.15.29:0/468729613 wait complete. 2024-04-25T00:32:19.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 Processor -- start 2024-04-25T00:32:19.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 -- start start 2024-04-25T00:32:19.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5980ccf30 0x7fb59815f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb59815bdb0 0x7fb59814c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5980cae40 0x7fb5980cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb5980c6400 con 0x7fb59815bdb0 2024-04-25T00:32:19.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5980c6280 con 0x7fb5980cae40 2024-04-25T00:32:19.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb5a078a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb5980c6580 con 0x7fb5980ccf30 2024-04-25T00:32:19.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb59dd25640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb59815bdb0 0x7fb59814c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb59dd25640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb59815bdb0 0x7fb59814c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42690/0 (socket says 172.21.15.29:42690) 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb59dd25640 1 -- 172.21.15.29:0/746141076 learned_addr learned my addr 172.21.15.29:0/746141076 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb59ed27640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5980cae40 0x7fb5980cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.703+0000 7fb59e526640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5980ccf30 0x7fb59815f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb59dd25640 1 -- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5980ccf30 msgr2=0x7fb59815f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb59dd25640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5980ccf30 0x7fb59815f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb59dd25640 1 -- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5980cae40 msgr2=0x7fb5980cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb59dd25640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5980cae40 0x7fb5980cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb59dd25640 1 -- 172.21.15.29:0/746141076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5980cb3d0 con 0x7fb59815bdb0 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb59e526640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5980ccf30 0x7fb59815f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb59ed27640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5980cae40 0x7fb5980cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb59dd25640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb59815bdb0 0x7fb59814c870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb59400c7f0 tx=0x7fb59400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb58affd640 1 -- 172.21.15.29:0/746141076 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594015070 con 0x7fb59815bdb0 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb58affd640 1 -- 172.21.15.29:0/746141076 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb594004220 con 0x7fb59815bdb0 2024-04-25T00:32:19.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb5a078a640 1 -- 172.21.15.29:0/746141076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5980cb6b0 con 0x7fb59815bdb0 2024-04-25T00:32:19.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb58affd640 1 -- 172.21.15.29:0/746141076 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594003620 con 0x7fb59815bdb0 2024-04-25T00:32:19.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb5a078a640 1 -- 172.21.15.29:0/746141076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5980cbc70 con 0x7fb59815bdb0 2024-04-25T00:32:19.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb5a078a640 1 -- 172.21.15.29:0/746141076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb598069270 con 0x7fb59815bdb0 2024-04-25T00:32:19.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb58affd640 1 -- 172.21.15.29:0/746141076 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb5940037c0 con 0x7fb59815bdb0 2024-04-25T00:32:19.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb58affd640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb57c0678f0 0x7fb57c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:19.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb58affd640 1 -- 172.21.15.29:0/746141076 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fb594085ae0 con 0x7fb59815bdb0 2024-04-25T00:32:19.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb59e526640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb57c0678f0 0x7fb57c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:19.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.707+0000 7fb59e526640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb57c0678f0 0x7fb57c069db0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0016e0 tx=0x7fb58c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:19.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.711+0000 7fb58affd640 1 -- 172.21.15.29:0/746141076 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb59408a070 con 0x7fb59815bdb0 2024-04-25T00:32:19.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.871+0000 7fb5a078a640 1 -- 172.21.15.29:0/746141076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb59815b090 con 0x7fb59815bdb0 2024-04-25T00:32:19.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.875+0000 7fb58affd640 1 -- 172.21.15.29:0/746141076 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fb594056570 con 0x7fb59815bdb0 2024-04-25T00:32:19.878 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:19.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.875+0000 7fb5a078a640 1 -- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb57c0678f0 msgr2=0x7fb57c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.875+0000 7fb5a078a640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb57c0678f0 0x7fb57c069db0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0016e0 tx=0x7fb58c009040 comp rx=0 tx=0).stop 2024-04-25T00:32:19.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.875+0000 7fb5a078a640 1 -- 172.21.15.29:0/746141076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb59815bdb0 msgr2=0x7fb59814c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:19.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.875+0000 7fb5a078a640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb59815bdb0 0x7fb59814c870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb59400c7f0 tx=0x7fb59400ccb0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.875+0000 7fb5a078a640 1 -- 172.21.15.29:0/746141076 shutdown_connections 2024-04-25T00:32:19.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.875+0000 7fb5a078a640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb57c0678f0 0x7fb57c069db0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.875+0000 7fb5a078a640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5980cae40 0x7fb5980cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.875+0000 7fb5a078a640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb59815bdb0 0x7fb59814c870 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.875+0000 7fb5a078a640 1 --2- 172.21.15.29:0/746141076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5980ccf30 0x7fb59815f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:19.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.875+0000 7fb5a078a640 1 -- 172.21.15.29:0/746141076 >> 172.21.15.29:0/746141076 conn(0x7fb59809aa30 msgr2=0x7fb5981532e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:19.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.875+0000 7fb5a078a640 1 -- 172.21.15.29:0/746141076 shutdown_connections 2024-04-25T00:32:19.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:19.875+0000 7fb5a078a640 1 -- 172.21.15.29:0/746141076 wait complete. 2024-04-25T00:32:19.898 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:20.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.015+0000 7f999f569640 1 Processor -- start 2024-04-25T00:32:20.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.015+0000 7f999f569640 1 -- start start 2024-04-25T00:32:20.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.015+0000 7f999f569640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f999815e1b0 0x7f9998072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.015+0000 7f999f569640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99980d0a30 con 0x7f99980ccfd0 2024-04-25T00:32:20.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.015+0000 7f999f569640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99980cfcb0 con 0x7f999806b920 2024-04-25T00:32:20.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.015+0000 7f999f569640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9998161bb0 con 0x7f999815e1b0 2024-04-25T00:32:20.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.015+0000 7f999cb04640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f99980ccfd0 0x7f999806b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38410/0 (socket says 172.21.15.29:38410) 2024-04-25T00:32:20.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.015+0000 7f999cb04640 1 -- 172.21.15.29:0/2221126340 learned_addr learned my addr 172.21.15.29:0/2221126340 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:20.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.015+0000 7f999d305640 1 --2- 172.21.15.29:0/2221126340 >> v2:172.21.15.157:3300/0 conn(0x7f999815e1b0 0x7f9998072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.015+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 769059698 0 0) 0x7f99980d0a30 con 0x7f99980ccfd0 2024-04-25T00:32:20.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.015+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9984003530 con 0x7f99980ccfd0 2024-04-25T00:32:20.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1563460072 0 0) 0x7f9984003530 con 0x7f99980ccfd0 2024-04-25T00:32:20.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 >> v2:172.21.15.157:3300/0 conn(0x7f999815e1b0 msgr2=0x7f9998072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:20.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 --2- 172.21.15.29:0/2221126340 >> v2:172.21.15.157:3300/0 conn(0x7f999815e1b0 0x7f9998072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 >> v1:172.21.15.157:6789/0 conn(0x7f999806b920 legacy=0x7f99980cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9998072e60 con 0x7f99980ccfd0 2024-04-25T00:32:20.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f9988001680 con 0x7f99980ccfd0 2024-04-25T00:32:20.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 >> v1:172.21.15.29:6789/0 conn(0x7f99980ccfd0 legacy=0x7f999806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 --2- 172.21.15.29:0/2221126340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840039c0 0x7f9984003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 --2- 172.21.15.29:0/2221126340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9984004810 0x7f9984004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 --2- 172.21.15.29:0/2221126340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99840051d0 0x7f998400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f998400c040 con 0x7f99840039c0 2024-04-25T00:32:20.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f998400c1f0 con 0x7f9984004810 2024-04-25T00:32:20.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f998400c3f0 con 0x7f99840051d0 2024-04-25T00:32:20.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999d305640 1 --2- 172.21.15.29:0/2221126340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840039c0 0x7f9984003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999cb04640 1 --2- 172.21.15.29:0/2221126340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9984004810 0x7f9984004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999db06640 1 --2- 172.21.15.29:0/2221126340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99840051d0 0x7f998400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999d305640 1 -- 172.21.15.29:0/2221126340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99840051d0 msgr2=0x7f998400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:20.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999d305640 1 --2- 172.21.15.29:0/2221126340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99840051d0 0x7f998400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999d305640 1 -- 172.21.15.29:0/2221126340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9984004810 msgr2=0x7f9984004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:20.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999d305640 1 --2- 172.21.15.29:0/2221126340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9984004810 0x7f9984004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999d305640 1 -- 172.21.15.29:0/2221126340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9984003530 con 0x7f99840039c0 2024-04-25T00:32:20.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999db06640 1 -- 172.21.15.29:0/2221126340 reap_dead start 2024-04-25T00:32:20.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999d305640 1 --2- 172.21.15.29:0/2221126340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840039c0 0x7f9984003de0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9994009fc0 tx=0x7f999401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf70638a79704c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:20.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99940b2070 con 0x7f99840039c0 2024-04-25T00:32:20.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99940a5b80 con 0x7f99840039c0 2024-04-25T00:32:20.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f998ffff640 1 -- 172.21.15.29:0/2221126340 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994009c10 con 0x7f99840039c0 2024-04-25T00:32:20.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999f569640 1 -- 172.21.15.29:0/2221126340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840039c0 msgr2=0x7f9984003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999f569640 1 --2- 172.21.15.29:0/2221126340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840039c0 0x7f9984003de0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9994009fc0 tx=0x7f999401cca0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999f569640 1 -- 172.21.15.29:0/2221126340 shutdown_connections 2024-04-25T00:32:20.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999f569640 1 --2- 172.21.15.29:0/2221126340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840039c0 0x7f9984003de0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999f569640 1 -- 172.21.15.29:0/2221126340 >> 172.21.15.29:0/2221126340 conn(0x7f99980b2920 msgr2=0x7f99980b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:20.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999f569640 1 -- 172.21.15.29:0/2221126340 shutdown_connections 2024-04-25T00:32:20.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.019+0000 7f999f569640 1 -- 172.21.15.29:0/2221126340 wait complete. 2024-04-25T00:32:20.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999f569640 1 Processor -- start 2024-04-25T00:32:20.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999f569640 1 -- start start 2024-04-25T00:32:20.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999f569640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99840039c0 0x7f999806e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999f569640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9998153c60 0x7f9998153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999f569640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9998155c50 0x7f9998156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999f569640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f999806b7a0 con 0x7f9998153c60 2024-04-25T00:32:20.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999f569640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f999806b620 con 0x7f9998155c50 2024-04-25T00:32:20.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999f569640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9998072a20 con 0x7f99840039c0 2024-04-25T00:32:20.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999cb04640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9998153c60 0x7f9998153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999cb04640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9998153c60 0x7f9998153140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42706/0 (socket says 172.21.15.29:42706) 2024-04-25T00:32:20.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999cb04640 1 -- 172.21.15.29:0/2658462319 learned_addr learned my addr 172.21.15.29:0/2658462319 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:20.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999d305640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99840039c0 0x7f999806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999db06640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9998155c50 0x7f9998156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999cb04640 1 -- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99840039c0 msgr2=0x7f999806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999cb04640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99840039c0 0x7f999806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999cb04640 1 -- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9998155c50 msgr2=0x7f9998156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999cb04640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9998155c50 0x7f9998156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999cb04640 1 -- 172.21.15.29:0/2658462319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99981539b0 con 0x7f9998153c60 2024-04-25T00:32:20.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999cb04640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9998153c60 0x7f9998153140 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f998800e740 tx=0x7f998800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:20.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f998dffb640 1 -- 172.21.15.29:0/2658462319 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998800ee50 con 0x7f9998153c60 2024-04-25T00:32:20.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f998dffb640 1 -- 172.21.15.29:0/2658462319 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9988018070 con 0x7f9998153c60 2024-04-25T00:32:20.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999f569640 1 -- 172.21.15.29:0/2658462319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999815aa60 con 0x7f9998153c60 2024-04-25T00:32:20.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f998dffb640 1 -- 172.21.15.29:0/2658462319 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998800cb10 con 0x7f9998153c60 2024-04-25T00:32:20.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.023+0000 7f999f569640 1 -- 172.21.15.29:0/2658462319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999815b020 con 0x7f9998153c60 2024-04-25T00:32:20.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.027+0000 7f999f569640 1 -- 172.21.15.29:0/2658462319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9998069270 con 0x7f9998153c60 2024-04-25T00:32:20.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.027+0000 7f998dffb640 1 -- 172.21.15.29:0/2658462319 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f998800ccb0 con 0x7f9998153c60 2024-04-25T00:32:20.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.027+0000 7f998dffb640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f997c0678c0 0x7f997c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.027+0000 7f998dffb640 1 -- 172.21.15.29:0/2658462319 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f9988085af0 con 0x7f9998153c60 2024-04-25T00:32:20.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.027+0000 7f999d305640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f997c0678c0 0x7f997c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.027+0000 7f999d305640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f997c0678c0 0x7f997c069d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f99940af000 tx=0x7f9994004ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:20.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.031+0000 7f998dffb640 1 -- 172.21.15.29:0/2658462319 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9988055e90 con 0x7f9998153c60 2024-04-25T00:32:20.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.199+0000 7f999f569640 1 -- 172.21.15.29:0/2658462319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f99980721c0 con 0x7f9998153c60 2024-04-25T00:32:20.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.203+0000 7f998dffb640 1 -- 172.21.15.29:0/2658462319 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f998805a190 con 0x7f9998153c60 2024-04-25T00:32:20.205 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:20.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.203+0000 7f999f569640 1 -- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f997c0678c0 msgr2=0x7f997c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.203+0000 7f999f569640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f997c0678c0 0x7f997c069d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f99940af000 tx=0x7f9994004ac0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.203+0000 7f999f569640 1 -- 172.21.15.29:0/2658462319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9998153c60 msgr2=0x7f9998153140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.203+0000 7f999f569640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9998153c60 0x7f9998153140 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f998800e740 tx=0x7f998800ec00 comp rx=0 tx=0).stop 2024-04-25T00:32:20.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.203+0000 7f999f569640 1 -- 172.21.15.29:0/2658462319 shutdown_connections 2024-04-25T00:32:20.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.203+0000 7f999f569640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f997c0678c0 0x7f997c069d80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.203+0000 7f999f569640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9998155c50 0x7f9998156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.203+0000 7f999f569640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9998153c60 0x7f9998153140 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.203+0000 7f999f569640 1 --2- 172.21.15.29:0/2658462319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99840039c0 0x7f999806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.203+0000 7f999f569640 1 -- 172.21.15.29:0/2658462319 >> 172.21.15.29:0/2658462319 conn(0x7f99980b2920 msgr2=0x7f99980afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:20.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.203+0000 7f999f569640 1 -- 172.21.15.29:0/2658462319 shutdown_connections 2024-04-25T00:32:20.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.203+0000 7f999f569640 1 -- 172.21.15.29:0/2658462319 wait complete. 2024-04-25T00:32:20.225 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:32:20.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc70e95f640 1 Processor -- start 2024-04-25T00:32:20.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc70e95f640 1 -- start start 2024-04-25T00:32:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc70e95f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc70815e8d0 0x7fc7080cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc70e95f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc7080b4780 0x7fc70815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc70e95f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc70815f1f0 0x7fc70815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc70e95f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc7080b42f0 con 0x7fc7080b4780 2024-04-25T00:32:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc70e95f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc7080b3720 con 0x7fc70815f1f0 2024-04-25T00:32:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc70e95f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc7080b39c0 con 0x7fc70815e8d0 2024-04-25T00:32:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc6fffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc7080b4780 0x7fc70815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc6fffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc7080b4780 0x7fc70815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42722/0 (socket says 172.21.15.29:42722) 2024-04-25T00:32:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc6fffff640 1 -- 172.21.15.29:0/1949808195 learned_addr learned my addr 172.21.15.29:0/1949808195 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc707fff640 1 --2- 172.21.15.29:0/1949808195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc70815e8d0 0x7fc7080cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc70cefc640 1 --2- 172.21.15.29:0/1949808195 >> v2:172.21.15.157:3300/0 conn(0x7fc70815f1f0 0x7fc70815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.351+0000 7fc6fffff640 1 -- 172.21.15.29:0/1949808195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc70815e8d0 msgr2=0x7fc7080cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc6fffff640 1 --2- 172.21.15.29:0/1949808195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc70815e8d0 0x7fc7080cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc6fffff640 1 -- 172.21.15.29:0/1949808195 >> v2:172.21.15.157:3300/0 conn(0x7fc70815f1f0 msgr2=0x7fc70815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc6fffff640 1 --2- 172.21.15.29:0/1949808195 >> v2:172.21.15.157:3300/0 conn(0x7fc70815f1f0 0x7fc70815b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc6fffff640 1 -- 172.21.15.29:0/1949808195 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7080b3ca0 con 0x7fc7080b4780 2024-04-25T00:32:20.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc6fffff640 1 --2- 172.21.15.29:0/1949808195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7080b4780 0x7fc70815ecb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc7080708a0 tx=0x7fc6f001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc3a75c45363890 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:20.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc7077fe640 1 -- 172.21.15.29:0/1949808195 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f00aa5f0 con 0x7fc7080b4780 2024-04-25T00:32:20.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc7077fe640 1 -- 172.21.15.29:0/1949808195 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6f00af070 con 0x7fc7080b4780 2024-04-25T00:32:20.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc7077fe640 1 -- 172.21.15.29:0/1949808195 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f00aa9b0 con 0x7fc7080b4780 2024-04-25T00:32:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc70e95f640 1 -- 172.21.15.29:0/1949808195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7080b4780 msgr2=0x7fc70815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc70e95f640 1 --2- 172.21.15.29:0/1949808195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7080b4780 0x7fc70815ecb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc7080708a0 tx=0x7fc6f001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc70e95f640 1 -- 172.21.15.29:0/1949808195 shutdown_connections 2024-04-25T00:32:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc70e95f640 1 --2- 172.21.15.29:0/1949808195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7080b4780 0x7fc70815ecb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc70e95f640 1 --2- 172.21.15.29:0/1949808195 >> v2:172.21.15.157:3300/0 conn(0x7fc70815f1f0 0x7fc70815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc70e95f640 1 --2- 172.21.15.29:0/1949808195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc70815e8d0 0x7fc7080cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc70e95f640 1 -- 172.21.15.29:0/1949808195 >> 172.21.15.29:0/1949808195 conn(0x7fc7080b1870 msgr2=0x7fc7080b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc70e95f640 1 -- 172.21.15.29:0/1949808195 shutdown_connections 2024-04-25T00:32:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.355+0000 7fc70e95f640 1 -- 172.21.15.29:0/1949808195 wait complete. 2024-04-25T00:32:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70e95f640 1 Processor -- start 2024-04-25T00:32:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70e95f640 1 -- start start 2024-04-25T00:32:20.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70e95f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7080b4780 0x7fc708138140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70e95f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc70815e8d0 0x7fc7080ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70e95f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc70815f1f0 0x7fc708137840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70e95f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc70815f600 con 0x7fc70815f1f0 2024-04-25T00:32:20.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70e95f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc70806d5d0 con 0x7fc70815e8d0 2024-04-25T00:32:20.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70e95f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc7080ce6e0 con 0x7fc7080b4780 2024-04-25T00:32:20.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70cefc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc70815f1f0 0x7fc708137840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70cefc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc70815f1f0 0x7fc708137840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42728/0 (socket says 172.21.15.29:42728) 2024-04-25T00:32:20.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70cefc640 1 -- 172.21.15.29:0/3963047832 learned_addr learned my addr 172.21.15.29:0/3963047832 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc707fff640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7080b4780 0x7fc708138140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc6fffff640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc70815e8d0 0x7fc7080ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70cefc640 1 -- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7080b4780 msgr2=0x7fc708138140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70cefc640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7080b4780 0x7fc708138140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70cefc640 1 -- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc70815e8d0 msgr2=0x7fc7080ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70cefc640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc70815e8d0 0x7fc7080ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70cefc640 1 -- 172.21.15.29:0/3963047832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc708138790 con 0x7fc70815f1f0 2024-04-25T00:32:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70cefc640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc70815f1f0 0x7fc708137840 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc700007e60 tx=0x7fc70000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc7057fa640 1 -- 172.21.15.29:0/3963047832 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700014420 con 0x7fc70815f1f0 2024-04-25T00:32:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc7057fa640 1 -- 172.21.15.29:0/3963047832 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc700019070 con 0x7fc70815f1f0 2024-04-25T00:32:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc7057fa640 1 -- 172.21.15.29:0/3963047832 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700014820 con 0x7fc70815f1f0 2024-04-25T00:32:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70e95f640 1 -- 172.21.15.29:0/3963047832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc708138a70 con 0x7fc70815f1f0 2024-04-25T00:32:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70e95f640 1 -- 172.21.15.29:0/3963047832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc708154d30 con 0x7fc70815f1f0 2024-04-25T00:32:20.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc70e95f640 1 -- 172.21.15.29:0/3963047832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc708069270 con 0x7fc70815f1f0 2024-04-25T00:32:20.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc7057fa640 1 -- 172.21.15.29:0/3963047832 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc7000149c0 con 0x7fc70815f1f0 2024-04-25T00:32:20.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc7057fa640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc6e80678c0 0x7fc6e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.359+0000 7fc7057fa640 1 -- 172.21.15.29:0/3963047832 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fc7000865e0 con 0x7fc70815f1f0 2024-04-25T00:32:20.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.363+0000 7fc707fff640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc6e80678c0 0x7fc6e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.363+0000 7fc707fff640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc6e80678c0 0x7fc6e8069d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800ba30 tx=0x7fc6f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:20.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.367+0000 7fc7057fa640 1 -- 172.21.15.29:0/3963047832 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc700057060 con 0x7fc70815f1f0 2024-04-25T00:32:20.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.539+0000 7fc70e95f640 1 -- 172.21.15.29:0/3963047832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc70806e7b0 con 0x7fc70815f1f0 2024-04-25T00:32:20.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.539+0000 7fc7057fa640 1 -- 172.21.15.29:0/3963047832 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fc700056960 con 0x7fc70815f1f0 2024-04-25T00:32:20.544 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:32:20.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.543+0000 7fc70e95f640 1 -- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc6e80678c0 msgr2=0x7fc6e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.543+0000 7fc70e95f640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc6e80678c0 0x7fc6e8069d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800ba30 tx=0x7fc6f8008040 comp rx=0 tx=0).stop 2024-04-25T00:32:20.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.543+0000 7fc70e95f640 1 -- 172.21.15.29:0/3963047832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc70815f1f0 msgr2=0x7fc708137840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.543+0000 7fc70e95f640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc70815f1f0 0x7fc708137840 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc700007e60 tx=0x7fc70000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.543+0000 7fc70e95f640 1 -- 172.21.15.29:0/3963047832 shutdown_connections 2024-04-25T00:32:20.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.543+0000 7fc70e95f640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc70815f1f0 0x7fc708137840 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.543+0000 7fc70e95f640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc6e80678c0 0x7fc6e8069d80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.543+0000 7fc70e95f640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc70815e8d0 0x7fc7080ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.543+0000 7fc70e95f640 1 --2- 172.21.15.29:0/3963047832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7080b4780 0x7fc708138140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.543+0000 7fc70e95f640 1 -- 172.21.15.29:0/3963047832 >> 172.21.15.29:0/3963047832 conn(0x7fc7080b1870 msgr2=0x7fc7080b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:20.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.543+0000 7fc70e95f640 1 -- 172.21.15.29:0/3963047832 shutdown_connections 2024-04-25T00:32:20.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.543+0000 7fc70e95f640 1 -- 172.21.15.29:0/3963047832 wait complete. 2024-04-25T00:32:20.563 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-25T00:32:20.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:20.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T00:32:20.577 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:32:20.578 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:32:20.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:20.578 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:32:20.626 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:32:20.627 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:32:20.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb15e84640 1 Processor -- start 2024-04-25T00:32:20.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb15e84640 1 -- start start 2024-04-25T00:32:20.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb15e84640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdb100cf690 0x7fdb100d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb15e84640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb100d2a90 0x7fdb100b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb15e84640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb100cfc10 con 0x7fdb1015e200 2024-04-25T00:32:20.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb15e84640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdb100d2ee0 con 0x7fdb100cf690 2024-04-25T00:32:20.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb15e84640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdb100b38d0 con 0x7fdb100d2a90 2024-04-25T00:32:20.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb06ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdb100cf690 0x7fdb100d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb0f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdb1015e200 0x7fdb100d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38426/0 (socket says 172.21.15.29:38426) 2024-04-25T00:32:20.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb0f7fe640 1 -- 172.21.15.29:0/1518357569 learned_addr learned my addr 172.21.15.29:0/1518357569 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:20.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb0ffff640 1 --2- 172.21.15.29:0/1518357569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb100d2a90 0x7fdb100b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb06ffd640 1 -- 172.21.15.29:0/1518357569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb100d2a90 msgr2=0x7fdb100b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:20.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb06ffd640 1 --2- 172.21.15.29:0/1518357569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb100d2a90 0x7fdb100b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb06ffd640 1 -- 172.21.15.29:0/1518357569 >> v1:172.21.15.29:6789/0 conn(0x7fdb1015e200 legacy=0x7fdb100d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb0effd640 1 -- 172.21.15.29:0/1518357569 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2195801345 0 0) 0x7fdb100cfc10 con 0x7fdb1015e200 2024-04-25T00:32:20.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.751+0000 7fdb06ffd640 1 -- 172.21.15.29:0/1518357569 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb100cfef0 con 0x7fdb100cf690 2024-04-25T00:32:20.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb06ffd640 1 --2- 172.21.15.29:0/1518357569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb100cf690 0x7fdb100d2550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80097c0 tx=0x7fdaf801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ceceec7fae44be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:20.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb0effd640 1 -- 172.21.15.29:0/1518357569 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80b2070 con 0x7fdb100cf690 2024-04-25T00:32:20.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb0effd640 1 -- 172.21.15.29:0/1518357569 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf80ad640 con 0x7fdb100cf690 2024-04-25T00:32:20.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb0effd640 1 -- 172.21.15.29:0/1518357569 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80ada10 con 0x7fdb100cf690 2024-04-25T00:32:20.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 -- 172.21.15.29:0/1518357569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb100cf690 msgr2=0x7fdb100d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 --2- 172.21.15.29:0/1518357569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb100cf690 0x7fdb100d2550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80097c0 tx=0x7fdaf801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 -- 172.21.15.29:0/1518357569 shutdown_connections 2024-04-25T00:32:20.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 --2- 172.21.15.29:0/1518357569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb100d2a90 0x7fdb100b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 --2- 172.21.15.29:0/1518357569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb100cf690 0x7fdb100d2550 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 -- 172.21.15.29:0/1518357569 >> 172.21.15.29:0/1518357569 conn(0x7fdb1009d280 msgr2=0x7fdb100d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:20.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 -- 172.21.15.29:0/1518357569 shutdown_connections 2024-04-25T00:32:20.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 -- 172.21.15.29:0/1518357569 wait complete. 2024-04-25T00:32:20.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 Processor -- start 2024-04-25T00:32:20.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 -- start start 2024-04-25T00:32:20.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb100cf690 0x7fdb10158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb100d2a90 0x7fdb10160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb1015e200 0x7fdb10155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdb100b51f0 con 0x7fdb1015e200 2024-04-25T00:32:20.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdb100b5070 con 0x7fdb100cf690 2024-04-25T00:32:20.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb15e84640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdb100b5370 con 0x7fdb100d2a90 2024-04-25T00:32:20.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb0ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb1015e200 0x7fdb10155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb0ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb1015e200 0x7fdb10155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42736/0 (socket says 172.21.15.29:42736) 2024-04-25T00:32:20.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb0ffff640 1 -- 172.21.15.29:0/1283938519 learned_addr learned my addr 172.21.15.29:0/1283938519 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:20.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb0f7fe640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb100cf690 0x7fdb10158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb06ffd640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb100d2a90 0x7fdb10160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb0ffff640 1 -- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb100d2a90 msgr2=0x7fdb10160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb0ffff640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb100d2a90 0x7fdb10160fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb0ffff640 1 -- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb100cf690 msgr2=0x7fdb10158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb0ffff640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb100cf690 0x7fdb10158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.755+0000 7fdb0ffff640 1 -- 172.21.15.29:0/1283938519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb10159600 con 0x7fdb1015e200 2024-04-25T00:32:20.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.759+0000 7fdb0ffff640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb1015e200 0x7fdb10155080 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdb08007e40 tx=0x7fdb0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:20.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.759+0000 7fdb0cff9640 1 -- 172.21.15.29:0/1283938519 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08014620 con 0x7fdb1015e200 2024-04-25T00:32:20.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.759+0000 7fdb0cff9640 1 -- 172.21.15.29:0/1283938519 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb08019070 con 0x7fdb1015e200 2024-04-25T00:32:20.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.759+0000 7fdb15e84640 1 -- 172.21.15.29:0/1283938519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb101598e0 con 0x7fdb1015e200 2024-04-25T00:32:20.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.759+0000 7fdb0cff9640 1 -- 172.21.15.29:0/1283938519 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08014a20 con 0x7fdb1015e200 2024-04-25T00:32:20.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.759+0000 7fdb15e84640 1 -- 172.21.15.29:0/1283938519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb10159ea0 con 0x7fdb1015e200 2024-04-25T00:32:20.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.759+0000 7fdb15e84640 1 -- 172.21.15.29:0/1283938519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb10069270 con 0x7fdb1015e200 2024-04-25T00:32:20.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.759+0000 7fdb0cff9640 1 -- 172.21.15.29:0/1283938519 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdb08014bc0 con 0x7fdb1015e200 2024-04-25T00:32:20.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.759+0000 7fdb0cff9640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdaf00678f0 0x7fdaf0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:20.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.759+0000 7fdb0cff9640 1 -- 172.21.15.29:0/1283938519 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fdb08086660 con 0x7fdb1015e200 2024-04-25T00:32:20.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.759+0000 7fdb0f7fe640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdaf00678f0 0x7fdaf0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:20.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.759+0000 7fdb0f7fe640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdaf00678f0 0x7fdaf0069db0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdb00000f70 tx=0x7fdb00007c10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:20.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.763+0000 7fdb0cff9640 1 -- 172.21.15.29:0/1283938519 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdb080570f0 con 0x7fdb1015e200 2024-04-25T00:32:20.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.959+0000 7fdb15e84640 1 -- 172.21.15.29:0/1283938519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb1015ac80 con 0x7fdb1015e200 2024-04-25T00:32:20.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.959+0000 7fdb0cff9640 1 -- 172.21.15.29:0/1283938519 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7fdb080569f0 con 0x7fdb1015e200 2024-04-25T00:32:20.964 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:20.965 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1532858457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532858457},{"type":"v1","addr":"172.21.15.29:6833","nonce":1532858457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/718997473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":718997473},{"type":"v1","addr":"172.21.15.157:6835","nonce":718997473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1617208213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1617208213},{"type":"v1","addr":"172.21.15.157:6837","nonce":1617208213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:31:29.640568+0000","modified":"2024-04-25T00:31:31.633585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4252":{"gid":4252,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1924706890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1924706890},{"type":"v1","addr":"172.21.15.29:6835","nonce":1924706890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4252,"qdb_cluster":[4252]},"id":1}]} 2024-04-25T00:32:20.965 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 6 2024-04-25T00:32:20.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.963+0000 7fdb15e84640 1 -- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdaf00678f0 msgr2=0x7fdaf0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.963+0000 7fdb15e84640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdaf00678f0 0x7fdaf0069db0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdb00000f70 tx=0x7fdb00007c10 comp rx=0 tx=0).stop 2024-04-25T00:32:20.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.963+0000 7fdb15e84640 1 -- 172.21.15.29:0/1283938519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb1015e200 msgr2=0x7fdb10155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:20.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.963+0000 7fdb15e84640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb1015e200 0x7fdb10155080 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdb08007e40 tx=0x7fdb0800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.963+0000 7fdb15e84640 1 -- 172.21.15.29:0/1283938519 shutdown_connections 2024-04-25T00:32:20.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.963+0000 7fdb15e84640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb1015e200 0x7fdb10155080 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.963+0000 7fdb15e84640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb100d2a90 0x7fdb10160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.963+0000 7fdb15e84640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdaf00678f0 0x7fdaf0069db0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.963+0000 7fdb15e84640 1 --2- 172.21.15.29:0/1283938519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb100cf690 0x7fdb10158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:20.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.963+0000 7fdb15e84640 1 -- 172.21.15.29:0/1283938519 >> 172.21.15.29:0/1283938519 conn(0x7fdb1009d280 msgr2=0x7fdb100d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:20.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.963+0000 7fdb15e84640 1 -- 172.21.15.29:0/1283938519 shutdown_connections 2024-04-25T00:32:20.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:20.963+0000 7fdb15e84640 1 -- 172.21.15.29:0/1283938519 wait complete. 2024-04-25T00:32:20.983 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:21.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc7f00f640 1 Processor -- start 2024-04-25T00:32:21.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc7f00f640 1 -- start start 2024-04-25T00:32:21.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc7f00f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbc780aea40 0x7fbc780aee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc7f00f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc780af770 con 0x7fbc7815c290 2024-04-25T00:32:21.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc7f00f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc780afa50 con 0x7fbc780d1c10 2024-04-25T00:32:21.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc7f00f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbc780b0380 con 0x7fbc780aea40 2024-04-25T00:32:21.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc77fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbc7815c290 0x7fbc780cc770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38434/0 (socket says 172.21.15.29:38434) 2024-04-25T00:32:21.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc77fff640 1 -- 172.21.15.29:0/3406795452 learned_addr learned my addr 172.21.15.29:0/3406795452 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:21.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc7d5ac640 1 --2- 172.21.15.29:0/3406795452 >> v2:172.21.15.157:3300/0 conn(0x7fbc780aea40 0x7fbc780aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2504376888 0 0) 0x7fbc780af770 con 0x7fbc7815c290 2024-04-25T00:32:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc60003530 con 0x7fbc7815c290 2024-04-25T00:32:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3675830029 0 0) 0x7fbc60003530 con 0x7fbc7815c290 2024-04-25T00:32:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 >> v2:172.21.15.157:3300/0 conn(0x7fbc780aea40 msgr2=0x7fbc780aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 --2- 172.21.15.29:0/3406795452 >> v2:172.21.15.157:3300/0 conn(0x7fbc780aea40 0x7fbc780aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 >> v1:172.21.15.157:6789/0 conn(0x7fbc780d1c10 legacy=0x7fbc780d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc780afd30 con 0x7fbc7815c290 2024-04-25T00:32:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fbc6c001680 con 0x7fbc7815c290 2024-04-25T00:32:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 >> v1:172.21.15.29:6789/0 conn(0x7fbc7815c290 legacy=0x7fbc780cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 --2- 172.21.15.29:0/3406795452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc600039c0 0x7fbc60003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 --2- 172.21.15.29:0/3406795452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc60004810 0x7fbc60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 --2- 172.21.15.29:0/3406795452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc600051d0 0x7fbc6000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbc780b0380 con 0x7fbc60004810 2024-04-25T00:32:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6000c070 con 0x7fbc600051d0 2024-04-25T00:32:21.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6000c270 con 0x7fbc600039c0 2024-04-25T00:32:21.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc77fff640 1 --2- 172.21.15.29:0/3406795452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc60004810 0x7fbc60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc7cdab640 1 --2- 172.21.15.29:0/3406795452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc600039c0 0x7fbc60003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc7d5ac640 1 --2- 172.21.15.29:0/3406795452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc600051d0 0x7fbc6000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc77fff640 1 -- 172.21.15.29:0/3406795452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc600039c0 msgr2=0x7fbc60003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:21.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc77fff640 1 --2- 172.21.15.29:0/3406795452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc600039c0 0x7fbc60003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc77fff640 1 -- 172.21.15.29:0/3406795452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc600051d0 msgr2=0x7fbc6000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:21.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc77fff640 1 --2- 172.21.15.29:0/3406795452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc600051d0 0x7fbc6000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc77fff640 1 -- 172.21.15.29:0/3406795452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc60003530 con 0x7fbc60004810 2024-04-25T00:32:21.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc7d5ac640 1 --2- 172.21.15.29:0/3406795452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc600051d0 0x7fbc6000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:21.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.095+0000 7fbc7d5ac640 1 -- 172.21.15.29:0/3406795452 reap_dead start 2024-04-25T00:32:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.099+0000 7fbc77fff640 1 --2- 172.21.15.29:0/3406795452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc60004810 0x7fbc60004c90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00c240 tx=0x7fbc6c01f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c147fd1cb558167c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.099+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c007d20 con 0x7fbc60004810 2024-04-25T00:32:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.099+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc6c005ae0 con 0x7fbc60004810 2024-04-25T00:32:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.099+0000 7fbc777fe640 1 -- 172.21.15.29:0/3406795452 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c007070 con 0x7fbc60004810 2024-04-25T00:32:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.099+0000 7fbc7f00f640 1 -- 172.21.15.29:0/3406795452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc60004810 msgr2=0x7fbc60004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.099+0000 7fbc7f00f640 1 --2- 172.21.15.29:0/3406795452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc60004810 0x7fbc60004c90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00c240 tx=0x7fbc6c01f370 comp rx=0 tx=0).stop 2024-04-25T00:32:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.099+0000 7fbc7f00f640 1 -- 172.21.15.29:0/3406795452 shutdown_connections 2024-04-25T00:32:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.099+0000 7fbc7f00f640 1 --2- 172.21.15.29:0/3406795452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc60004810 0x7fbc60004c90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.099+0000 7fbc7f00f640 1 -- 172.21.15.29:0/3406795452 >> 172.21.15.29:0/3406795452 conn(0x7fbc7809aa30 msgr2=0x7fbc7809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.099+0000 7fbc7f00f640 1 -- 172.21.15.29:0/3406795452 shutdown_connections 2024-04-25T00:32:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.099+0000 7fbc7f00f640 1 -- 172.21.15.29:0/3406795452 wait complete. 2024-04-25T00:32:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.099+0000 7fbc7f00f640 1 Processor -- start 2024-04-25T00:32:21.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.099+0000 7fbc7f00f640 1 -- start start 2024-04-25T00:32:21.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7f00f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc60004810 0x7fbc780cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7f00f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc780c9030 0x7fbc780c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7f00f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc7806c320 0x7fbc7806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7f00f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbc780d2170 con 0x7fbc7806c320 2024-04-25T00:32:21.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7f00f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbc780d1ff0 con 0x7fbc780c9030 2024-04-25T00:32:21.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7f00f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbc780d22f0 con 0x7fbc60004810 2024-04-25T00:32:21.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7d5ac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc7806c320 0x7fbc7806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7d5ac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc7806c320 0x7fbc7806c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42764/0 (socket says 172.21.15.29:42764) 2024-04-25T00:32:21.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7d5ac640 1 -- 172.21.15.29:0/2300703829 learned_addr learned my addr 172.21.15.29:0/2300703829 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:21.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc77fff640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc780c9030 0x7fbc780c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7cdab640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc60004810 0x7fbc780cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7d5ac640 1 -- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc60004810 msgr2=0x7fbc780cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7d5ac640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc60004810 0x7fbc780cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7d5ac640 1 -- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc780c9030 msgr2=0x7fbc780c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7d5ac640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc780c9030 0x7fbc780c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7d5ac640 1 -- 172.21.15.29:0/2300703829 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7806d010 con 0x7fbc7806c320 2024-04-25T00:32:21.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc77fff640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc780c9030 0x7fbc780c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:21.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7d5ac640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc7806c320 0x7fbc7806c7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009ea0 tx=0x7fbc70009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:21.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc757fa640 1 -- 172.21.15.29:0/2300703829 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70016420 con 0x7fbc7806c320 2024-04-25T00:32:21.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc757fa640 1 -- 172.21.15.29:0/2300703829 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc7001b070 con 0x7fbc7806c320 2024-04-25T00:32:21.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc757fa640 1 -- 172.21.15.29:0/2300703829 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc700167c0 con 0x7fbc7806c320 2024-04-25T00:32:21.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7f00f640 1 -- 172.21.15.29:0/2300703829 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7806d2f0 con 0x7fbc7806c320 2024-04-25T00:32:21.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7f00f640 1 -- 172.21.15.29:0/2300703829 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc7806d8b0 con 0x7fbc7806c320 2024-04-25T00:32:21.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7f00f640 1 -- 172.21.15.29:0/2300703829 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc78069270 con 0x7fbc7806c320 2024-04-25T00:32:21.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc757fa640 1 -- 172.21.15.29:0/2300703829 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbc70016960 con 0x7fbc7806c320 2024-04-25T00:32:21.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc757fa640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbc640678c0 0x7fbc64069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc757fa640 1 -- 172.21.15.29:0/2300703829 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fbc70086a90 con 0x7fbc7806c320 2024-04-25T00:32:21.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.103+0000 7fbc7cdab640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbc640678c0 0x7fbc64069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.107+0000 7fbc7cdab640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbc640678c0 0x7fbc64069d80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbc7815d000 tx=0x7fbc68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:21.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.107+0000 7fbc757fa640 1 -- 172.21.15.29:0/2300703829 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbc700575b0 con 0x7fbc7806c320 2024-04-25T00:32:21.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.271+0000 7fbc7f00f640 1 -- 172.21.15.29:0/2300703829 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc7806f520 con 0x7fbc7806c320 2024-04-25T00:32:21.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.271+0000 7fbc757fa640 1 -- 172.21.15.29:0/2300703829 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16838 (secure 0 0 0) 0x7fbc70056eb0 con 0x7fbc7806c320 2024-04-25T00:32:21.276 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:21.277 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":22,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:32:18.667140+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:21.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.275+0000 7fbc7f00f640 1 -- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbc640678c0 msgr2=0x7fbc64069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.275+0000 7fbc7f00f640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbc640678c0 0x7fbc64069d80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbc7815d000 tx=0x7fbc68009040 comp rx=0 tx=0).stop 2024-04-25T00:32:21.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.275+0000 7fbc7f00f640 1 -- 172.21.15.29:0/2300703829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc7806c320 msgr2=0x7fbc7806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.275+0000 7fbc7f00f640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc7806c320 0x7fbc7806c7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009ea0 tx=0x7fbc70009f90 comp rx=0 tx=0).stop 2024-04-25T00:32:21.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.275+0000 7fbc7f00f640 1 -- 172.21.15.29:0/2300703829 shutdown_connections 2024-04-25T00:32:21.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.275+0000 7fbc7f00f640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc7806c320 0x7fbc7806c7a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.275+0000 7fbc7f00f640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbc640678c0 0x7fbc64069d80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.275+0000 7fbc7f00f640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc780c9030 0x7fbc780c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.275+0000 7fbc7f00f640 1 --2- 172.21.15.29:0/2300703829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc60004810 0x7fbc780cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.275+0000 7fbc7f00f640 1 -- 172.21.15.29:0/2300703829 >> 172.21.15.29:0/2300703829 conn(0x7fbc7809aa30 msgr2=0x7fbc780cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:21.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.275+0000 7fbc7f00f640 1 -- 172.21.15.29:0/2300703829 shutdown_connections 2024-04-25T00:32:21.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.275+0000 7fbc7f00f640 1 -- 172.21.15.29:0/2300703829 wait complete. 2024-04-25T00:32:21.296 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:31:27.778241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:31:28.627385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:21.296 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:21.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.415+0000 7eff4b963640 1 Processor -- start 2024-04-25T00:32:21.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.415+0000 7eff4b963640 1 -- start start 2024-04-25T00:32:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff4b963640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7eff440b4780 0x7eff4415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff4b963640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff440cc240 0x7eff4415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff4b963640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff440b3720 con 0x7eff4415e8d0 2024-04-25T00:32:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff4b963640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7eff440b42f0 con 0x7eff440b4780 2024-04-25T00:32:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff4b963640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff440b39c0 con 0x7eff440cc240 2024-04-25T00:32:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff48efe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7eff440b4780 0x7eff4415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff48efe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7eff440b4780 0x7eff4415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42778/0 (socket says 172.21.15.29:42778) 2024-04-25T00:32:21.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff48efe640 1 -- 172.21.15.29:0/2880634314 learned_addr learned my addr 172.21.15.29:0/2880634314 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff496ff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7eff4415e8d0 0x7eff4415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38446/0 (socket says 172.21.15.29:38446) 2024-04-25T00:32:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff49f00640 1 --2- 172.21.15.29:0/2880634314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff440cc240 0x7eff4415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff48efe640 1 -- 172.21.15.29:0/2880634314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff440cc240 msgr2=0x7eff4415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff48efe640 1 --2- 172.21.15.29:0/2880634314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff440cc240 0x7eff4415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff48efe640 1 -- 172.21.15.29:0/2880634314 >> v1:172.21.15.29:6789/0 conn(0x7eff4415e8d0 legacy=0x7eff4415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff3bfff640 1 -- 172.21.15.29:0/2880634314 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2457696921 0 0) 0x7eff440b3720 con 0x7eff4415e8d0 2024-04-25T00:32:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff48efe640 1 -- 172.21.15.29:0/2880634314 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff440b3ca0 con 0x7eff440b4780 2024-04-25T00:32:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff49f00640 1 --2- 172.21.15.29:0/2880634314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff440cc240 0x7eff4415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff48efe640 1 --2- 172.21.15.29:0/2880634314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff440b4780 0x7eff4415f3c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7eff440b4470 tx=0x7eff3401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447c1d02ceb9f711 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff3bfff640 1 -- 172.21.15.29:0/2880634314 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff340ab610 con 0x7eff440b4780 2024-04-25T00:32:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff3bfff640 1 -- 172.21.15.29:0/2880634314 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff340b0070 con 0x7eff440b4780 2024-04-25T00:32:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff3bfff640 1 -- 172.21.15.29:0/2880634314 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff340ab9d0 con 0x7eff440b4780 2024-04-25T00:32:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff4b963640 1 -- 172.21.15.29:0/2880634314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff440b4780 msgr2=0x7eff4415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff4b963640 1 --2- 172.21.15.29:0/2880634314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff440b4780 0x7eff4415f3c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7eff440b4470 tx=0x7eff3401c540 comp rx=0 tx=0).stop 2024-04-25T00:32:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff4b963640 1 -- 172.21.15.29:0/2880634314 shutdown_connections 2024-04-25T00:32:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff4b963640 1 --2- 172.21.15.29:0/2880634314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff440cc240 0x7eff4415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff4b963640 1 --2- 172.21.15.29:0/2880634314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff440b4780 0x7eff4415f3c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff4b963640 1 -- 172.21.15.29:0/2880634314 >> 172.21.15.29:0/2880634314 conn(0x7eff440b1870 msgr2=0x7eff440b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff4b963640 1 -- 172.21.15.29:0/2880634314 shutdown_connections 2024-04-25T00:32:21.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.419+0000 7eff4b963640 1 -- 172.21.15.29:0/2880634314 wait complete. 2024-04-25T00:32:21.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff4b963640 1 Processor -- start 2024-04-25T00:32:21.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff4b963640 1 -- start start 2024-04-25T00:32:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff4b963640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff440b4780 0x7eff441366e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff4b963640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff440cc240 0x7eff44139c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff4b963640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff4415e8d0 0x7eff440cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff4b963640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7eff4415d6a0 con 0x7eff440b4780 2024-04-25T00:32:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff4b963640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff4415d520 con 0x7eff440cc240 2024-04-25T00:32:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff4b963640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff44167c70 con 0x7eff4415e8d0 2024-04-25T00:32:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff496ff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff440b4780 0x7eff441366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff496ff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff440b4780 0x7eff441366e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42788/0 (socket says 172.21.15.29:42788) 2024-04-25T00:32:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff496ff640 1 -- 172.21.15.29:0/3483129755 learned_addr learned my addr 172.21.15.29:0/3483129755 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff48efe640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff440cc240 0x7eff44139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff49f00640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff4415e8d0 0x7eff440cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff496ff640 1 -- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff4415e8d0 msgr2=0x7eff440cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff496ff640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff4415e8d0 0x7eff440cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff496ff640 1 -- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff440cc240 msgr2=0x7eff44139c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff496ff640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff440cc240 0x7eff44139c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff496ff640 1 -- 172.21.15.29:0/3483129755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff44135640 con 0x7eff440b4780 2024-04-25T00:32:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff48efe640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff440cc240 0x7eff44139c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff496ff640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff440b4780 0x7eff441366e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7eff4000c480 tx=0x7eff4000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff39ffb640 1 -- 172.21.15.29:0/3483129755 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40018070 con 0x7eff440b4780 2024-04-25T00:32:21.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff39ffb640 1 -- 172.21.15.29:0/3483129755 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff40007500 con 0x7eff440b4780 2024-04-25T00:32:21.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff4b963640 1 -- 172.21.15.29:0/3483129755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff44135920 con 0x7eff440b4780 2024-04-25T00:32:21.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff39ffb640 1 -- 172.21.15.29:0/3483129755 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40004a30 con 0x7eff440b4780 2024-04-25T00:32:21.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff4b963640 1 -- 172.21.15.29:0/3483129755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff44135ee0 con 0x7eff440b4780 2024-04-25T00:32:21.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff4b963640 1 -- 172.21.15.29:0/3483129755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff4406ed20 con 0x7eff440b4780 2024-04-25T00:32:21.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.423+0000 7eff39ffb640 1 -- 172.21.15.29:0/3483129755 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7eff4001b070 con 0x7eff440b4780 2024-04-25T00:32:21.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.427+0000 7eff39ffb640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff280678c0 0x7eff28069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.427+0000 7eff39ffb640 1 -- 172.21.15.29:0/3483129755 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7eff40084fc0 con 0x7eff440b4780 2024-04-25T00:32:21.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.427+0000 7eff48efe640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff280678c0 0x7eff28069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.427+0000 7eff48efe640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff280678c0 0x7eff28069d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7eff340af000 tx=0x7eff340a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:21.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.431+0000 7eff39ffb640 1 -- 172.21.15.29:0/3483129755 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7eff40055ae0 con 0x7eff440b4780 2024-04-25T00:32:21.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.591+0000 7eff4b963640 1 -- 172.21.15.29:0/3483129755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff4406e7b0 con 0x7eff440b4780 2024-04-25T00:32:21.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.591+0000 7eff39ffb640 1 -- 172.21.15.29:0/3483129755 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7eff400553e0 con 0x7eff440b4780 2024-04-25T00:32:21.595 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:21.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.595+0000 7eff4b963640 1 -- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff280678c0 msgr2=0x7eff28069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.595+0000 7eff4b963640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff280678c0 0x7eff28069d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7eff340af000 tx=0x7eff340a7000 comp rx=0 tx=0).stop 2024-04-25T00:32:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.595+0000 7eff4b963640 1 -- 172.21.15.29:0/3483129755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff440b4780 msgr2=0x7eff441366e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.595+0000 7eff4b963640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff440b4780 0x7eff441366e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7eff4000c480 tx=0x7eff4000c940 comp rx=0 tx=0).stop 2024-04-25T00:32:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.595+0000 7eff4b963640 1 -- 172.21.15.29:0/3483129755 shutdown_connections 2024-04-25T00:32:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.595+0000 7eff4b963640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff4415e8d0 0x7eff440cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.595+0000 7eff4b963640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff280678c0 0x7eff28069d80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.595+0000 7eff4b963640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff440cc240 0x7eff44139c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.595+0000 7eff4b963640 1 --2- 172.21.15.29:0/3483129755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff440b4780 0x7eff441366e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.595+0000 7eff4b963640 1 -- 172.21.15.29:0/3483129755 >> 172.21.15.29:0/3483129755 conn(0x7eff440b1870 msgr2=0x7eff440d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.595+0000 7eff4b963640 1 -- 172.21.15.29:0/3483129755 shutdown_connections 2024-04-25T00:32:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.595+0000 7eff4b963640 1 -- 172.21.15.29:0/3483129755 wait complete. 2024-04-25T00:32:21.615 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:21.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfc29d8640 1 Processor -- start 2024-04-25T00:32:21.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfc29d8640 1 -- start start 2024-04-25T00:32:21.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfc29d8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbfbc0b4d30 0x7fbfbc0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfc29d8640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfbc06d6c0 con 0x7fbfbc06d210 2024-04-25T00:32:21.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfc29d8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfbc06d980 con 0x7fbfbc15df70 2024-04-25T00:32:21.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfc29d8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbfbc06dc60 con 0x7fbfbc0b4d30 2024-04-25T00:32:21.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfc0f75640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbfbc06d210 0x7fbfbc0ad8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38452/0 (socket says 172.21.15.29:38452) 2024-04-25T00:32:21.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfc0f75640 1 -- 172.21.15.29:0/3011521026 learned_addr learned my addr 172.21.15.29:0/3011521026 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:21.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbb7fe640 1 --2- 172.21.15.29:0/3011521026 >> v2:172.21.15.157:3300/0 conn(0x7fbfbc0b4d30 0x7fbfbc0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2437207993 0 0) 0x7fbfbc06d6c0 con 0x7fbfbc06d210 2024-04-25T00:32:21.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfa4003530 con 0x7fbfbc06d210 2024-04-25T00:32:21.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4293260002 0 0) 0x7fbfbc06d980 con 0x7fbfbc15df70 2024-04-25T00:32:21.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfbc06d6c0 con 0x7fbfbc15df70 2024-04-25T00:32:21.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 223318984 0 0) 0x7fbfa4003530 con 0x7fbfbc06d210 2024-04-25T00:32:21.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 >> v2:172.21.15.157:3300/0 conn(0x7fbfbc0b4d30 msgr2=0x7fbfbc0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:21.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 --2- 172.21.15.29:0/3011521026 >> v2:172.21.15.157:3300/0 conn(0x7fbfbc0b4d30 0x7fbfbc0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 >> v1:172.21.15.157:6789/0 conn(0x7fbfbc15df70 legacy=0x7fbfbc0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc06df40 con 0x7fbfbc06d210 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fbfb4001a00 con 0x7fbfbc06d210 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 >> v1:172.21.15.29:6789/0 conn(0x7fbfbc06d210 legacy=0x7fbfbc0ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 --2- 172.21.15.29:0/3011521026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa4003b50 0x7fbfa4003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 --2- 172.21.15.29:0/3011521026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa4004810 0x7fbfa4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 --2- 172.21.15.29:0/3011521026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa40051d0 0x7fbfa400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc06dc60 con 0x7fbfa4003b50 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa400c050 con 0x7fbfa4004810 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa400c250 con 0x7fbfa40051d0 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.739+0000 7fbfbbfff640 1 --2- 172.21.15.29:0/3011521026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa4003b50 0x7fbfa4003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfbb7fe640 1 --2- 172.21.15.29:0/3011521026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa4004810 0x7fbfa4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfc0f75640 1 --2- 172.21.15.29:0/3011521026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa40051d0 0x7fbfa400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfbbfff640 1 -- 172.21.15.29:0/3011521026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa40051d0 msgr2=0x7fbfa400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfbbfff640 1 --2- 172.21.15.29:0/3011521026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa40051d0 0x7fbfa400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfbbfff640 1 -- 172.21.15.29:0/3011521026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa4004810 msgr2=0x7fbfa4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:21.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfbbfff640 1 --2- 172.21.15.29:0/3011521026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa4004810 0x7fbfa4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfbbfff640 1 -- 172.21.15.29:0/3011521026 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa4003530 con 0x7fbfa4003b50 2024-04-25T00:32:21.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfc0f75640 1 --2- 172.21.15.29:0/3011521026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa40051d0 0x7fbfa400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:21.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfc0f75640 1 -- 172.21.15.29:0/3011521026 reap_dead start 2024-04-25T00:32:21.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfbbfff640 1 --2- 172.21.15.29:0/3011521026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa4003b50 0x7fbfa4003f50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0018d0 tx=0x7fbfac01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9896b8233e6808 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:21.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac001520 con 0x7fbfa4003b50 2024-04-25T00:32:21.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfac0a9070 con 0x7fbfa4003b50 2024-04-25T00:32:21.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfbaffd640 1 -- 172.21.15.29:0/3011521026 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac003600 con 0x7fbfa4003b50 2024-04-25T00:32:21.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3011521026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa4003b50 msgr2=0x7fbfa4003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfc29d8640 1 --2- 172.21.15.29:0/3011521026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa4003b50 0x7fbfa4003f50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0018d0 tx=0x7fbfac01c530 comp rx=0 tx=0).stop 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3011521026 shutdown_connections 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfc29d8640 1 --2- 172.21.15.29:0/3011521026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa4003b50 0x7fbfa4003f50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3011521026 >> 172.21.15.29:0/3011521026 conn(0x7fbfbc0b1870 msgr2=0x7fbfbc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3011521026 shutdown_connections 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.743+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3011521026 wait complete. 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfc29d8640 1 Processor -- start 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfc29d8640 1 -- start start 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfc29d8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa4003b50 0x7fbfbc157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfc29d8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfbc140dd0 0x7fbfbc151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfc29d8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfbc1ec6e0 0x7fbfbc1ecb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfc29d8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc0b4bb0 con 0x7fbfa4003b50 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfc29d8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc0b4a30 con 0x7fbfbc140dd0 2024-04-25T00:32:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfc29d8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbfbc06ea20 con 0x7fbfbc1ec6e0 2024-04-25T00:32:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfbbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa4003b50 0x7fbfbc157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfbbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa4003b50 0x7fbfbc157510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42802/0 (socket says 172.21.15.29:42802) 2024-04-25T00:32:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfbbfff640 1 -- 172.21.15.29:0/3991334118 learned_addr learned my addr 172.21.15.29:0/3991334118 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfbb7fe640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfbc140dd0 0x7fbfbc151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfbbfff640 1 -- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfbc1ec6e0 msgr2=0x7fbfbc1ecb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfc0f75640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfbc1ec6e0 0x7fbfbc1ecb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfbbfff640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfbc1ec6e0 0x7fbfbc1ecb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfbbfff640 1 -- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfbc140dd0 msgr2=0x7fbfbc151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfbbfff640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfbc140dd0 0x7fbfbc151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfbbfff640 1 -- 172.21.15.29:0/3991334118 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc1515c0 con 0x7fbfa4003b50 2024-04-25T00:32:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfbb7fe640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfbc140dd0 0x7fbfbc151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfbbfff640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa4003b50 0x7fbfbc157510 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbfac01c360 tx=0x7fbfa4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfb8ff9640 1 -- 172.21.15.29:0/3991334118 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0ba070 con 0x7fbfa4003b50 2024-04-25T00:32:21.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfb8ff9640 1 -- 172.21.15.29:0/3991334118 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfac001500 con 0x7fbfa4003b50 2024-04-25T00:32:21.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfb8ff9640 1 -- 172.21.15.29:0/3991334118 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac006a20 con 0x7fbfa4003b50 2024-04-25T00:32:21.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3991334118 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfbc1eb640 con 0x7fbfa4003b50 2024-04-25T00:32:21.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3991334118 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfbc1ebc00 con 0x7fbfa4003b50 2024-04-25T00:32:21.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.747+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3991334118 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfbc069270 con 0x7fbfa4003b50 2024-04-25T00:32:21.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.751+0000 7fbfb8ff9640 1 -- 172.21.15.29:0/3991334118 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbfac0ab070 con 0x7fbfa4003b50 2024-04-25T00:32:21.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.751+0000 7fbfb8ff9640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfa80678f0 0x7fbfa8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:21.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.751+0000 7fbfb8ff9640 1 -- 172.21.15.29:0/3991334118 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fbfac11e8b0 con 0x7fbfa4003b50 2024-04-25T00:32:21.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.751+0000 7fbfbb7fe640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfa80678f0 0x7fbfa8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:21.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.751+0000 7fbfbb7fe640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfa80678f0 0x7fbfa8069db0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc1ed2a0 tx=0x7fbfb0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:21.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.755+0000 7fbfb8ff9640 1 -- 172.21.15.29:0/3991334118 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbfac016270 con 0x7fbfa4003b50 2024-04-25T00:32:21.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.915+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3991334118 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbfbc072190 con 0x7fbfa4003b50 2024-04-25T00:32:21.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.915+0000 7fbfb8ff9640 1 -- 172.21.15.29:0/3991334118 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fbfac0ef3d0 con 0x7fbfa4003b50 2024-04-25T00:32:21.921 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.919+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfa80678f0 msgr2=0x7fbfa8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.919+0000 7fbfc29d8640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfa80678f0 0x7fbfa8069db0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc1ed2a0 tx=0x7fbfb0008040 comp rx=0 tx=0).stop 2024-04-25T00:32:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.919+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3991334118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa4003b50 msgr2=0x7fbfbc157510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.919+0000 7fbfc29d8640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa4003b50 0x7fbfbc157510 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbfac01c360 tx=0x7fbfa4005840 comp rx=0 tx=0).stop 2024-04-25T00:32:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.919+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3991334118 shutdown_connections 2024-04-25T00:32:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.919+0000 7fbfc29d8640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfbc1ec6e0 0x7fbfbc1ecb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.919+0000 7fbfc29d8640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfa80678f0 0x7fbfa8069db0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.919+0000 7fbfc29d8640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfbc140dd0 0x7fbfbc151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.919+0000 7fbfc29d8640 1 --2- 172.21.15.29:0/3991334118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa4003b50 0x7fbfbc157510 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.919+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3991334118 >> 172.21.15.29:0/3991334118 conn(0x7fbfbc0b1870 msgr2=0x7fbfbc0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.919+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3991334118 shutdown_connections 2024-04-25T00:32:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:21.919+0000 7fbfc29d8640 1 -- 172.21.15.29:0/3991334118 wait complete. 2024-04-25T00:32:21.940 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:22.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.063+0000 7f94e1eec640 1 Processor -- start 2024-04-25T00:32:22.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.063+0000 7f94e1eec640 1 -- start start 2024-04-25T00:32:22.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.063+0000 7f94e1eec640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f94dc071e10 0x7f94dc15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.063+0000 7f94e1eec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94dc0ce820 0x7f94dc0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.063+0000 7f94e1eec640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94dc06e580 con 0x7f94dc0721f0 2024-04-25T00:32:22.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.063+0000 7f94e1eec640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f94dc06f5e0 con 0x7f94dc071e10 2024-04-25T00:32:22.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.063+0000 7f94e1eec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f94dc0b0370 con 0x7f94dc0ce820 2024-04-25T00:32:22.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94db7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f94dc071e10 0x7f94dc15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94db7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f94dc071e10 0x7f94dc15ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42812/0 (socket says 172.21.15.29:42812) 2024-04-25T00:32:22.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94db7fe640 1 -- 172.21.15.29:0/1062430497 learned_addr learned my addr 172.21.15.29:0/1062430497 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:22.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94dbfff640 1 --2- 172.21.15.29:0/1062430497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94dc0ce820 0x7f94dc0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94db7fe640 1 -- 172.21.15.29:0/1062430497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94dc0ce820 msgr2=0x7f94dc0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94db7fe640 1 --2- 172.21.15.29:0/1062430497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94dc0ce820 0x7f94dc0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94db7fe640 1 -- 172.21.15.29:0/1062430497 >> v1:172.21.15.157:6789/0 conn(0x7f94dc0721f0 legacy=0x7f94dc072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94db7fe640 1 -- 172.21.15.29:0/1062430497 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc06e820 con 0x7f94dc071e10 2024-04-25T00:32:22.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94db7fe640 1 --2- 172.21.15.29:0/1062430497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94dc071e10 0x7f94dc15ba90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009870 tx=0x7f94cc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67d955d83becde8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94da7fc640 1 -- 172.21.15.29:0/1062430497 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0aa600 con 0x7f94dc071e10 2024-04-25T00:32:22.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94da7fc640 1 -- 172.21.15.29:0/1062430497 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94cc0af070 con 0x7f94dc071e10 2024-04-25T00:32:22.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94da7fc640 1 -- 172.21.15.29:0/1062430497 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0aa9c0 con 0x7f94dc071e10 2024-04-25T00:32:22.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94e1eec640 1 -- 172.21.15.29:0/1062430497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94dc071e10 msgr2=0x7f94dc15ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94e1eec640 1 --2- 172.21.15.29:0/1062430497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94dc071e10 0x7f94dc15ba90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009870 tx=0x7f94cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94e1eec640 1 -- 172.21.15.29:0/1062430497 shutdown_connections 2024-04-25T00:32:22.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94e1eec640 1 --2- 172.21.15.29:0/1062430497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94dc0ce820 0x7f94dc0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94e1eec640 1 --2- 172.21.15.29:0/1062430497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94dc071e10 0x7f94dc15ba90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94e1eec640 1 -- 172.21.15.29:0/1062430497 >> 172.21.15.29:0/1062430497 conn(0x7f94dc0b2920 msgr2=0x7f94dc0b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:22.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94e1eec640 1 -- 172.21.15.29:0/1062430497 shutdown_connections 2024-04-25T00:32:22.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.067+0000 7f94e1eec640 1 -- 172.21.15.29:0/1062430497 wait complete. 2024-04-25T00:32:22.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94e1eec640 1 Processor -- start 2024-04-25T00:32:22.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94e1eec640 1 -- start start 2024-04-25T00:32:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94e1eec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94dc071e10 0x7f94dc143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94e1eec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94dc0721f0 0x7f94dc1453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94e1eec640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94dc0ce820 0x7f94dc14cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94e1eec640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc15c150 con 0x7f94dc0ce820 2024-04-25T00:32:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94e1eec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc15bfd0 con 0x7f94dc071e10 2024-04-25T00:32:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94e1eec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f94dc167c00 con 0x7f94dc0721f0 2024-04-25T00:32:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94dbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94dc0ce820 0x7f94dc14cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94dbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94dc0ce820 0x7f94dc14cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42828/0 (socket says 172.21.15.29:42828) 2024-04-25T00:32:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94dbfff640 1 -- 172.21.15.29:0/354961335 learned_addr learned my addr 172.21.15.29:0/354961335 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94db7fe640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94dc071e10 0x7f94dc143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94daffd640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94dc0721f0 0x7f94dc1453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94dbfff640 1 -- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94dc0721f0 msgr2=0x7f94dc1453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94dbfff640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94dc0721f0 0x7f94dc1453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94dbfff640 1 -- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94dc071e10 msgr2=0x7f94dc143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94dbfff640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94dc071e10 0x7f94dc143aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94dbfff640 1 -- 172.21.15.29:0/354961335 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc143fe0 con 0x7f94dc0ce820 2024-04-25T00:32:22.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94db7fe640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94dc071e10 0x7f94dc143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:22.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94dbfff640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94dc0ce820 0x7f94dc14cca0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f94d4007ed0 tx=0x7f94d40106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94c3fff640 1 -- 172.21.15.29:0/354961335 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4018070 con 0x7f94dc0ce820 2024-04-25T00:32:22.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94c3fff640 1 -- 172.21.15.29:0/354961335 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94d40138a0 con 0x7f94dc0ce820 2024-04-25T00:32:22.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94e1eec640 1 -- 172.21.15.29:0/354961335 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94dc0b4410 con 0x7f94dc0ce820 2024-04-25T00:32:22.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94c3fff640 1 -- 172.21.15.29:0/354961335 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4013c90 con 0x7f94dc0ce820 2024-04-25T00:32:22.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94e1eec640 1 -- 172.21.15.29:0/354961335 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94dc0b49d0 con 0x7f94dc0ce820 2024-04-25T00:32:22.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.071+0000 7f94e1eec640 1 -- 172.21.15.29:0/354961335 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94dc069270 con 0x7f94dc0ce820 2024-04-25T00:32:22.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.075+0000 7f94c3fff640 1 -- 172.21.15.29:0/354961335 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f94d401b070 con 0x7f94dc0ce820 2024-04-25T00:32:22.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.075+0000 7f94c3fff640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f94c80678c0 0x7f94c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.075+0000 7f94c3fff640 1 -- 172.21.15.29:0/354961335 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f94d4088c70 con 0x7f94dc0ce820 2024-04-25T00:32:22.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.075+0000 7f94db7fe640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f94c80678c0 0x7f94c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.075+0000 7f94db7fe640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f94c80678c0 0x7f94c8069d80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0ae000 tx=0x7f94cc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.079+0000 7f94c3fff640 1 -- 172.21.15.29:0/354961335 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f94d4059790 con 0x7f94dc0ce820 2024-04-25T00:32:22.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.243+0000 7f94e1eec640 1 -- 172.21.15.29:0/354961335 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f94dc06e050 con 0x7f94dc0ce820 2024-04-25T00:32:22.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.243+0000 7f94c3fff640 1 -- 172.21.15.29:0/354961335 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f94d4059090 con 0x7f94dc0ce820 2024-04-25T00:32:22.247 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:22.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.243+0000 7f94e1eec640 1 -- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f94c80678c0 msgr2=0x7f94c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.243+0000 7f94e1eec640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f94c80678c0 0x7f94c8069d80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0ae000 tx=0x7f94cc0a7000 comp rx=0 tx=0).stop 2024-04-25T00:32:22.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.243+0000 7f94e1eec640 1 -- 172.21.15.29:0/354961335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94dc0ce820 msgr2=0x7f94dc14cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.243+0000 7f94e1eec640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94dc0ce820 0x7f94dc14cca0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f94d4007ed0 tx=0x7f94d40106c0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.243+0000 7f94e1eec640 1 -- 172.21.15.29:0/354961335 shutdown_connections 2024-04-25T00:32:22.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.243+0000 7f94e1eec640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94dc0ce820 0x7f94dc14cca0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.243+0000 7f94e1eec640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94dc0721f0 0x7f94dc1453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.243+0000 7f94e1eec640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f94c80678c0 0x7f94c8069d80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.243+0000 7f94e1eec640 1 --2- 172.21.15.29:0/354961335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94dc071e10 0x7f94dc143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.243+0000 7f94e1eec640 1 -- 172.21.15.29:0/354961335 >> 172.21.15.29:0/354961335 conn(0x7f94dc0b2920 msgr2=0x7f94dc0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.247+0000 7f94e1eec640 1 -- 172.21.15.29:0/354961335 shutdown_connections 2024-04-25T00:32:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.247+0000 7f94e1eec640 1 -- 172.21.15.29:0/354961335 wait complete. 2024-04-25T00:32:22.266 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:32:22.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af3e26640 1 Processor -- start 2024-04-25T00:32:22.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af3e26640 1 -- start start 2024-04-25T00:32:22.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af3e26640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8aec15e200 0x7f8aec0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af3e26640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aec0cfc10 con 0x7f8aec0cf690 2024-04-25T00:32:22.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af3e26640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aec0cfef0 con 0x7f8aec0d2c60 2024-04-25T00:32:22.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af3e26640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8aec0b38d0 con 0x7f8aec15e200 2024-04-25T00:32:22.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af13c1640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8aec0cf690 0x7f8aec0d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38454/0 (socket says 172.21.15.29:38454) 2024-04-25T00:32:22.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af13c1640 1 -- 172.21.15.29:0/751676131 learned_addr learned my addr 172.21.15.29:0/751676131 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:22.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af1bc2640 1 --2- 172.21.15.29:0/751676131 >> v2:172.21.15.157:3300/0 conn(0x7f8aec15e200 0x7f8aec0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2798846679 0 0) 0x7f8aec0cfc10 con 0x7f8aec0cf690 2024-04-25T00:32:22.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ad8003530 con 0x7f8aec0cf690 2024-04-25T00:32:22.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4030724499 0 0) 0x7f8ad8003530 con 0x7f8aec0cf690 2024-04-25T00:32:22.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 >> v2:172.21.15.157:3300/0 conn(0x7f8aec15e200 msgr2=0x7f8aec0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 --2- 172.21.15.29:0/751676131 >> v2:172.21.15.157:3300/0 conn(0x7f8aec15e200 0x7f8aec0d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 >> v1:172.21.15.157:6789/0 conn(0x7f8aec0d2c60 legacy=0x7f8aec0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec0b3b20 con 0x7f8aec0cf690 2024-04-25T00:32:22.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8adc001680 con 0x7f8aec0cf690 2024-04-25T00:32:22.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 >> v1:172.21.15.29:6789/0 conn(0x7f8aec0cf690 legacy=0x7f8aec0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 --2- 172.21.15.29:0/751676131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ad80039c0 0x7f8ad8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 --2- 172.21.15.29:0/751676131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ad8004810 0x7f8ad8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 --2- 172.21.15.29:0/751676131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ad80051d0 0x7f8ad800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec0b38d0 con 0x7f8ad80039c0 2024-04-25T00:32:22.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad800c070 con 0x7f8ad8004810 2024-04-25T00:32:22.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad800c270 con 0x7f8ad80051d0 2024-04-25T00:32:22.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af1bc2640 1 --2- 172.21.15.29:0/751676131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ad80039c0 0x7f8ad8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af13c1640 1 --2- 172.21.15.29:0/751676131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ad8004810 0x7f8ad8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af23c3640 1 --2- 172.21.15.29:0/751676131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ad80051d0 0x7f8ad800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af1bc2640 1 -- 172.21.15.29:0/751676131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ad80051d0 msgr2=0x7f8ad800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af1bc2640 1 --2- 172.21.15.29:0/751676131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ad80051d0 0x7f8ad800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af1bc2640 1 -- 172.21.15.29:0/751676131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ad8004810 msgr2=0x7f8ad8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af1bc2640 1 --2- 172.21.15.29:0/751676131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ad8004810 0x7f8ad8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af1bc2640 1 -- 172.21.15.29:0/751676131 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad8003530 con 0x7f8ad80039c0 2024-04-25T00:32:22.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.387+0000 7f8af23c3640 1 -- 172.21.15.29:0/751676131 reap_dead start 2024-04-25T00:32:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.391+0000 7f8af1bc2640 1 --2- 172.21.15.29:0/751676131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ad80039c0 0x7f8ad8003de0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009ae0 tx=0x7f8ae801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc25ce1a3b67f158 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.391+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80b2070 con 0x7f8ad80039c0 2024-04-25T00:32:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.391+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae80a5540 con 0x7f8ad80039c0 2024-04-25T00:32:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.391+0000 7f8af0bc0640 1 -- 172.21.15.29:0/751676131 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80096b0 con 0x7f8ad80039c0 2024-04-25T00:32:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.391+0000 7f8af3e26640 1 -- 172.21.15.29:0/751676131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ad80039c0 msgr2=0x7f8ad8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.391+0000 7f8af3e26640 1 --2- 172.21.15.29:0/751676131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ad80039c0 0x7f8ad8003de0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009ae0 tx=0x7f8ae801ca40 comp rx=0 tx=0).stop 2024-04-25T00:32:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.391+0000 7f8af3e26640 1 -- 172.21.15.29:0/751676131 shutdown_connections 2024-04-25T00:32:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.391+0000 7f8af3e26640 1 --2- 172.21.15.29:0/751676131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ad80039c0 0x7f8ad8003de0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.391+0000 7f8af3e26640 1 -- 172.21.15.29:0/751676131 >> 172.21.15.29:0/751676131 conn(0x7f8aec09d280 msgr2=0x7f8aec0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.391+0000 7f8af3e26640 1 -- 172.21.15.29:0/751676131 shutdown_connections 2024-04-25T00:32:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.391+0000 7f8af3e26640 1 -- 172.21.15.29:0/751676131 wait complete. 2024-04-25T00:32:22.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af3e26640 1 Processor -- start 2024-04-25T00:32:22.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af3e26640 1 -- start start 2024-04-25T00:32:22.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af3e26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ad80039c0 0x7f8aec1603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af3e26640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aec0b6fe0 0x7f8aec0b8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af3e26640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8aec160d00 0x7f8aec161180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af3e26640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec0d0720 con 0x7f8aec160d00 2024-04-25T00:32:22.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af3e26640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec0d05a0 con 0x7f8ad80039c0 2024-04-25T00:32:22.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af3e26640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8aec0d08a0 con 0x7f8aec0b6fe0 2024-04-25T00:32:22.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af23c3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8aec160d00 0x7f8aec161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af23c3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8aec160d00 0x7f8aec161180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42852/0 (socket says 172.21.15.29:42852) 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af23c3640 1 -- 172.21.15.29:0/1610675554 learned_addr learned my addr 172.21.15.29:0/1610675554 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af1bc2640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ad80039c0 0x7f8aec1603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af13c1640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aec0b6fe0 0x7f8aec0b8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af23c3640 1 -- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aec0b6fe0 msgr2=0x7f8aec0b8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af23c3640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aec0b6fe0 0x7f8aec0b8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af23c3640 1 -- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ad80039c0 msgr2=0x7f8aec1603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af23c3640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ad80039c0 0x7f8aec1603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af23c3640 1 -- 172.21.15.29:0/1610675554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec1f6e00 con 0x7f8aec160d00 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af23c3640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8aec160d00 0x7f8aec161180 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8aec06b900 tx=0x7f8ae400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8ae27fc640 1 -- 172.21.15.29:0/1610675554 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4003b80 con 0x7f8aec160d00 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8ae27fc640 1 -- 172.21.15.29:0/1610675554 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae4017070 con 0x7f8aec160d00 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8ae27fc640 1 -- 172.21.15.29:0/1610675554 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4005070 con 0x7f8aec160d00 2024-04-25T00:32:22.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af3e26640 1 -- 172.21.15.29:0/1610675554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aec0b91a0 con 0x7f8aec160d00 2024-04-25T00:32:22.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af3e26640 1 -- 172.21.15.29:0/1610675554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aec0b9760 con 0x7f8aec160d00 2024-04-25T00:32:22.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.395+0000 7f8af3e26640 1 -- 172.21.15.29:0/1610675554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aec069270 con 0x7f8aec160d00 2024-04-25T00:32:22.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.399+0000 7f8ae27fc640 1 -- 172.21.15.29:0/1610675554 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8ae4005210 con 0x7f8aec160d00 2024-04-25T00:32:22.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.399+0000 7f8ae27fc640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ad00678c0 0x7f8ad0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.399+0000 7f8ae27fc640 1 -- 172.21.15.29:0/1610675554 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f8ae4085b10 con 0x7f8aec160d00 2024-04-25T00:32:22.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.399+0000 7f8af1bc2640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ad00678c0 0x7f8ad0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.399+0000 7f8af1bc2640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ad00678c0 0x7f8ad0069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80af000 tx=0x7f8ae80046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.403+0000 7f8ae27fc640 1 -- 172.21.15.29:0/1610675554 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8ae4056500 con 0x7f8aec160d00 2024-04-25T00:32:22.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.595+0000 7f8af3e26640 1 -- 172.21.15.29:0/1610675554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8aec0d43d0 con 0x7f8aec160d00 2024-04-25T00:32:22.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.599+0000 7f8ae27fc640 1 -- 172.21.15.29:0/1610675554 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f8ae4055e00 con 0x7f8aec160d00 2024-04-25T00:32:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:22.602 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1532858457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532858457},{"type":"v1","addr":"172.21.15.29:6833","nonce":1532858457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/718997473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":718997473},{"type":"v1","addr":"172.21.15.157:6835","nonce":718997473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1617208213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1617208213},{"type":"v1","addr":"172.21.15.157:6837","nonce":1617208213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:31:29.640568+0000","modified":"2024-04-25T00:31:31.633585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4252":{"gid":4252,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1924706890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1924706890},{"type":"v1","addr":"172.21.15.29:6835","nonce":1924706890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4252,"qdb_cluster":[4252]},"id":1}]} 2024-04-25T00:32:22.602 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 6 2024-04-25T00:32:22.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.599+0000 7f8af3e26640 1 -- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ad00678c0 msgr2=0x7f8ad0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.599+0000 7f8af3e26640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ad00678c0 0x7f8ad0069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80af000 tx=0x7f8ae80046d0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.599+0000 7f8af3e26640 1 -- 172.21.15.29:0/1610675554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8aec160d00 msgr2=0x7f8aec161180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.599+0000 7f8af3e26640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8aec160d00 0x7f8aec161180 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8aec06b900 tx=0x7f8ae400dc00 comp rx=0 tx=0).stop 2024-04-25T00:32:22.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.599+0000 7f8af3e26640 1 -- 172.21.15.29:0/1610675554 shutdown_connections 2024-04-25T00:32:22.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.599+0000 7f8af3e26640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8aec160d00 0x7f8aec161180 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.599+0000 7f8af3e26640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8aec0b6fe0 0x7f8aec0b8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.599+0000 7f8af3e26640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ad00678c0 0x7f8ad0069d80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.599+0000 7f8af3e26640 1 --2- 172.21.15.29:0/1610675554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ad80039c0 0x7f8aec1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.599+0000 7f8af3e26640 1 -- 172.21.15.29:0/1610675554 >> 172.21.15.29:0/1610675554 conn(0x7f8aec09d280 msgr2=0x7f8aec066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:22.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.599+0000 7f8af3e26640 1 -- 172.21.15.29:0/1610675554 shutdown_connections 2024-04-25T00:32:22.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.599+0000 7f8af3e26640 1 -- 172.21.15.29:0/1610675554 wait complete. 2024-04-25T00:32:22.622 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:22.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.747+0000 7f02ddcaf640 1 Processor -- start 2024-04-25T00:32:22.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.747+0000 7f02ddcaf640 1 -- start start 2024-04-25T00:32:22.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02ddcaf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f02d80b3140 0x7f02d806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02ddcaf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02d80b4b10 0x7f02d80b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02ddcaf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02d80b41e0 con 0x7f02d80b3a60 2024-04-25T00:32:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02ddcaf640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f02d806d960 con 0x7f02d80b3140 2024-04-25T00:32:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02ddcaf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f02d806dc40 con 0x7f02d80b4b10 2024-04-25T00:32:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02d77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f02d80b3140 0x7f02d806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02d77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f02d80b3140 0x7f02d806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42854/0 (socket says 172.21.15.29:42854) 2024-04-25T00:32:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02d77fe640 1 -- 172.21.15.29:0/1150334320 learned_addr learned my addr 172.21.15.29:0/1150334320 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02d6ffd640 1 --2- 172.21.15.29:0/1150334320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02d80b4b10 0x7f02d80b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02d77fe640 1 -- 172.21.15.29:0/1150334320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02d80b4b10 msgr2=0x7f02d80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02d77fe640 1 --2- 172.21.15.29:0/1150334320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02d80b4b10 0x7f02d80b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02d77fe640 1 -- 172.21.15.29:0/1150334320 >> v1:172.21.15.157:6789/0 conn(0x7f02d80b3a60 legacy=0x7f02d815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02d77fe640 1 -- 172.21.15.29:0/1150334320 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d806df20 con 0x7f02d80b3140 2024-04-25T00:32:22.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02d77fe640 1 --2- 172.21.15.29:0/1150334320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80b3140 0x7f02d806e440 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009870 tx=0x7f02c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a0cb71d696da22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02d67fc640 1 -- 172.21.15.29:0/1150334320 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c80b0070 con 0x7f02d80b3140 2024-04-25T00:32:22.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02d67fc640 1 -- 172.21.15.29:0/1150334320 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c80ab600 con 0x7f02d80b3140 2024-04-25T00:32:22.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.751+0000 7f02d67fc640 1 -- 172.21.15.29:0/1150334320 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c80a4e60 con 0x7f02d80b3140 2024-04-25T00:32:22.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 -- 172.21.15.29:0/1150334320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80b3140 msgr2=0x7f02d806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 --2- 172.21.15.29:0/1150334320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80b3140 0x7f02d806e440 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009870 tx=0x7f02c801c520 comp rx=0 tx=0).stop 2024-04-25T00:32:22.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 -- 172.21.15.29:0/1150334320 shutdown_connections 2024-04-25T00:32:22.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 --2- 172.21.15.29:0/1150334320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02d80b4b10 0x7f02d80b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 --2- 172.21.15.29:0/1150334320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80b3140 0x7f02d806e440 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 -- 172.21.15.29:0/1150334320 >> 172.21.15.29:0/1150334320 conn(0x7f02d80b1870 msgr2=0x7f02d80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:22.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 -- 172.21.15.29:0/1150334320 shutdown_connections 2024-04-25T00:32:22.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 -- 172.21.15.29:0/1150334320 wait complete. 2024-04-25T00:32:22.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 Processor -- start 2024-04-25T00:32:22.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 -- start start 2024-04-25T00:32:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80b3140 0x7f02d813c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02d80b3a60 0x7f02d8139b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d80b4b10 0x7f02d8072620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f02d8168b70 con 0x7f02d80b3140 2024-04-25T00:32:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f02d81689f0 con 0x7f02d80b4b10 2024-04-25T00:32:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f02d8168cf0 con 0x7f02d80b3a60 2024-04-25T00:32:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80b3140 0x7f02d813c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80b3140 0x7f02d813c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42866/0 (socket says 172.21.15.29:42866) 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02d77fe640 1 -- 172.21.15.29:0/4165154559 learned_addr learned my addr 172.21.15.29:0/4165154559 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02d7fff640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d80b4b10 0x7f02d8072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02d77fe640 1 -- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02d80b3a60 msgr2=0x7f02d8139b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02d6ffd640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02d80b3a60 0x7f02d8139b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02d77fe640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02d80b3a60 0x7f02d8139b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02d77fe640 1 -- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d80b4b10 msgr2=0x7f02d8072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02d77fe640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d80b4b10 0x7f02d8072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02d77fe640 1 -- 172.21.15.29:0/4165154559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d813aa20 con 0x7f02d80b3140 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02d7fff640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d80b4b10 0x7f02d8072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02d6ffd640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02d80b3a60 0x7f02d8139b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02d77fe640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80b3140 0x7f02d813c560 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f02c80ae040 tx=0x7f02c801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02bffff640 1 -- 172.21.15.29:0/4165154559 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c80ba020 con 0x7f02d80b3140 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02bffff640 1 -- 172.21.15.29:0/4165154559 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c80b6420 con 0x7f02d80b3140 2024-04-25T00:32:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02bffff640 1 -- 172.21.15.29:0/4165154559 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c80b0040 con 0x7f02d80b3140 2024-04-25T00:32:22.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 -- 172.21.15.29:0/4165154559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d813cc60 con 0x7f02d80b3140 2024-04-25T00:32:22.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.755+0000 7f02ddcaf640 1 -- 172.21.15.29:0/4165154559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d8071e20 con 0x7f02d80b3140 2024-04-25T00:32:22.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.759+0000 7f02ddcaf640 1 -- 172.21.15.29:0/4165154559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d8069270 con 0x7f02d80b3140 2024-04-25T00:32:22.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.759+0000 7f02bffff640 1 -- 172.21.15.29:0/4165154559 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f02c80bd020 con 0x7f02d80b3140 2024-04-25T00:32:22.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.759+0000 7f02bffff640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02c40678f0 0x7f02c4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.759+0000 7f02bffff640 1 -- 172.21.15.29:0/4165154559 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f02c811fe10 con 0x7f02d80b3140 2024-04-25T00:32:22.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.759+0000 7f02d6ffd640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02c40678f0 0x7f02c4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.759+0000 7f02d6ffd640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02c40678f0 0x7f02c4069db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f02d8145100 tx=0x7f02cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.763+0000 7f02bffff640 1 -- 172.21.15.29:0/4165154559 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f02c80c4030 con 0x7f02d80b3140 2024-04-25T00:32:22.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.927+0000 7f02ddcaf640 1 -- 172.21.15.29:0/4165154559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02d8155c90 con 0x7f02d80b3140 2024-04-25T00:32:22.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.927+0000 7f02bffff640 1 -- 172.21.15.29:0/4165154559 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16838 (secure 0 0 0) 0x7f02c8016110 con 0x7f02d80b3140 2024-04-25T00:32:22.932 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:22.932 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":22,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:32:18.667140+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:22.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.931+0000 7f02ddcaf640 1 -- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02c40678f0 msgr2=0x7f02c4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.931+0000 7f02ddcaf640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02c40678f0 0x7f02c4069db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f02d8145100 tx=0x7f02cc008040 comp rx=0 tx=0).stop 2024-04-25T00:32:22.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.931+0000 7f02ddcaf640 1 -- 172.21.15.29:0/4165154559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80b3140 msgr2=0x7f02d813c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.931+0000 7f02ddcaf640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80b3140 0x7f02d813c560 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f02c80ae040 tx=0x7f02c801ad60 comp rx=0 tx=0).stop 2024-04-25T00:32:22.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.931+0000 7f02ddcaf640 1 -- 172.21.15.29:0/4165154559 shutdown_connections 2024-04-25T00:32:22.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.931+0000 7f02ddcaf640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02c40678f0 0x7f02c4069db0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.931+0000 7f02ddcaf640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d80b4b10 0x7f02d8072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.931+0000 7f02ddcaf640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02d80b3a60 0x7f02d8139b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.931+0000 7f02ddcaf640 1 --2- 172.21.15.29:0/4165154559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80b3140 0x7f02d813c560 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.931+0000 7f02ddcaf640 1 -- 172.21.15.29:0/4165154559 >> 172.21.15.29:0/4165154559 conn(0x7f02d80b1870 msgr2=0x7f02d80ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:22.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.931+0000 7f02ddcaf640 1 -- 172.21.15.29:0/4165154559 shutdown_connections 2024-04-25T00:32:22.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:22.931+0000 7f02ddcaf640 1 -- 172.21.15.29:0/4165154559 wait complete. 2024-04-25T00:32:22.952 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:32:22.952 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:32:23.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.075+0000 7fcd5d6cc640 1 Processor -- start 2024-04-25T00:32:23.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.075+0000 7fcd5d6cc640 1 -- start start 2024-04-25T00:32:23.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.075+0000 7fcd5d6cc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd5815e1b0 0x7fcd580cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.075+0000 7fcd5d6cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd580cb7e0 0x7fcd5806f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.075+0000 7fcd5d6cc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd580d01d0 con 0x7fcd58071fa0 2024-04-25T00:32:23.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.075+0000 7fcd5d6cc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcd580d0a70 con 0x7fcd5815e1b0 2024-04-25T00:32:23.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.075+0000 7fcd5d6cc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5815e830 con 0x7fcd580cb7e0 2024-04-25T00:32:23.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.075+0000 7fcd567fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcd58071fa0 0x7fcd5815ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38464/0 (socket says 172.21.15.29:38464) 2024-04-25T00:32:23.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.075+0000 7fcd567fc640 1 -- 172.21.15.29:0/2702650899 learned_addr learned my addr 172.21.15.29:0/2702650899 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:23.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd56ffd640 1 --2- 172.21.15.29:0/2702650899 >> v2:172.21.15.157:3300/0 conn(0x7fcd5815e1b0 0x7fcd580cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd577fe640 1 --2- 172.21.15.29:0/2702650899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd580cb7e0 0x7fcd5806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 28358447 0 0) 0x7fcd580d01d0 con 0x7fcd58071fa0 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd40003530 con 0x7fcd58071fa0 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3286019056 0 0) 0x7fcd40003530 con 0x7fcd58071fa0 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd580cb7e0 msgr2=0x7fcd5806f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 --2- 172.21.15.29:0/2702650899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd580cb7e0 0x7fcd5806f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 >> v2:172.21.15.157:3300/0 conn(0x7fcd5815e1b0 msgr2=0x7fcd580cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 --2- 172.21.15.29:0/2702650899 >> v2:172.21.15.157:3300/0 conn(0x7fcd5815e1b0 0x7fcd580cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd580d04b0 con 0x7fcd58071fa0 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fcd4c001680 con 0x7fcd58071fa0 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 >> v1:172.21.15.29:6789/0 conn(0x7fcd58071fa0 legacy=0x7fcd5815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 --2- 172.21.15.29:0/2702650899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd400039c0 0x7fcd40003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 --2- 172.21.15.29:0/2702650899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd40004810 0x7fcd40004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 --2- 172.21.15.29:0/2702650899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd580cb7e0 0x7fcd400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcd580d0a70 con 0x7fcd40004810 2024-04-25T00:32:23.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5815e830 con 0x7fcd400039c0 2024-04-25T00:32:23.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4000b9d0 con 0x7fcd580cb7e0 2024-04-25T00:32:23.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd567fc640 1 --2- 172.21.15.29:0/2702650899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd40004810 0x7fcd40004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd577fe640 1 --2- 172.21.15.29:0/2702650899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd580cb7e0 0x7fcd400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd567fc640 1 -- 172.21.15.29:0/2702650899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd580cb7e0 msgr2=0x7fcd400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd567fc640 1 --2- 172.21.15.29:0/2702650899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd580cb7e0 0x7fcd400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd56ffd640 1 --2- 172.21.15.29:0/2702650899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd400039c0 0x7fcd40003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd567fc640 1 -- 172.21.15.29:0/2702650899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd400039c0 msgr2=0x7fcd40003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd567fc640 1 --2- 172.21.15.29:0/2702650899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd400039c0 0x7fcd40003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd567fc640 1 -- 172.21.15.29:0/2702650899 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd40003530 con 0x7fcd40004810 2024-04-25T00:32:23.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd567fc640 1 --2- 172.21.15.29:0/2702650899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd40004810 0x7fcd40004c90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00c2c0 tx=0x7fcd4c01f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8564400cdc696a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c007d60 con 0x7fcd40004810 2024-04-25T00:32:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c005b00 con 0x7fcd40004810 2024-04-25T00:32:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd55ffb640 1 -- 172.21.15.29:0/2702650899 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c007070 con 0x7fcd40004810 2024-04-25T00:32:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/2702650899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd40004810 msgr2=0x7fcd40004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.079+0000 7fcd5d6cc640 1 --2- 172.21.15.29:0/2702650899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd40004810 0x7fcd40004c90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00c2c0 tx=0x7fcd4c01f4b0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd577fe640 1 -- 172.21.15.29:0/2702650899 reap_dead start 2024-04-25T00:32:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/2702650899 shutdown_connections 2024-04-25T00:32:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/2702650899 >> 172.21.15.29:0/2702650899 conn(0x7fcd580b2920 msgr2=0x7fcd580b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/2702650899 shutdown_connections 2024-04-25T00:32:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/2702650899 wait complete. 2024-04-25T00:32:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 Processor -- start 2024-04-25T00:32:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 -- start start 2024-04-25T00:32:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd40004810 0x7fcd5815c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd580b1550 0x7fcd580b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd581e7660 0x7fcd581e7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5815f550 con 0x7fcd580b1550 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5815f3d0 con 0x7fcd581e7660 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd580728c0 con 0x7fcd40004810 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd567fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd580b1550 0x7fcd580b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd567fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd580b1550 0x7fcd580b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42894/0 (socket says 172.21.15.29:42894) 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd567fc640 1 -- 172.21.15.29:0/3787168995 learned_addr learned my addr 172.21.15.29:0/3787168995 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd577fe640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd581e7660 0x7fcd581e7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd56ffd640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd40004810 0x7fcd5815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd567fc640 1 -- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd40004810 msgr2=0x7fcd5815c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd567fc640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd40004810 0x7fcd5815c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd567fc640 1 -- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd581e7660 msgr2=0x7fcd581e7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd567fc640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd581e7660 0x7fcd581e7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd567fc640 1 -- 172.21.15.29:0/3787168995 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5815c750 con 0x7fcd580b1550 2024-04-25T00:32:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd567fc640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd580b1550 0x7fcd580b19d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c01f2e0 tx=0x7fcd40005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd3f7fe640 1 -- 172.21.15.29:0/3787168995 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c001c90 con 0x7fcd580b1550 2024-04-25T00:32:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd3f7fe640 1 -- 172.21.15.29:0/3787168995 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c005cb0 con 0x7fcd580b1550 2024-04-25T00:32:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/3787168995 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd581e8030 con 0x7fcd580b1550 2024-04-25T00:32:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd3f7fe640 1 -- 172.21.15.29:0/3787168995 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c005e60 con 0x7fcd580b1550 2024-04-25T00:32:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/3787168995 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd580b5a90 con 0x7fcd580b1550 2024-04-25T00:32:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.083+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/3787168995 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd58069270 con 0x7fcd580b1550 2024-04-25T00:32:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.087+0000 7fcd3f7fe640 1 -- 172.21.15.29:0/3787168995 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcd4c004a30 con 0x7fcd580b1550 2024-04-25T00:32:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.087+0000 7fcd3f7fe640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcd440678c0 0x7fcd44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.087+0000 7fcd3f7fe640 1 -- 172.21.15.29:0/3787168995 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fcd4c122d90 con 0x7fcd580b1550 2024-04-25T00:32:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.087+0000 7fcd56ffd640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcd440678c0 0x7fcd44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.087+0000 7fcd56ffd640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcd440678c0 0x7fcd44069d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcd581e7030 tx=0x7fcd5000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.091+0000 7fcd3f7fe640 1 -- 172.21.15.29:0/3787168995 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcd4c0191c0 con 0x7fcd580b1550 2024-04-25T00:32:23.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.283+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/3787168995 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd5806e420 con 0x7fcd580b1550 2024-04-25T00:32:23.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.283+0000 7fcd3f7fe640 1 -- 172.21.15.29:0/3787168995 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7fcd4c0f3830 con 0x7fcd580b1550 2024-04-25T00:32:23.288 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:23.289 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1532858457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532858457},{"type":"v1","addr":"172.21.15.29:6833","nonce":1532858457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/718997473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":718997473},{"type":"v1","addr":"172.21.15.157:6835","nonce":718997473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1617208213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1617208213},{"type":"v1","addr":"172.21.15.157:6837","nonce":1617208213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:31:29.640568+0000","modified":"2024-04-25T00:31:31.633585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4252":{"gid":4252,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1924706890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1924706890},{"type":"v1","addr":"172.21.15.29:6835","nonce":1924706890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4252,"qdb_cluster":[4252]},"id":1}]} 2024-04-25T00:32:23.289 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 6 2024-04-25T00:32:23.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.287+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcd440678c0 msgr2=0x7fcd44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.287+0000 7fcd5d6cc640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcd440678c0 0x7fcd44069d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcd581e7030 tx=0x7fcd5000b040 comp rx=0 tx=0).stop 2024-04-25T00:32:23.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.287+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/3787168995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd580b1550 msgr2=0x7fcd580b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.287+0000 7fcd5d6cc640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd580b1550 0x7fcd580b19d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c01f2e0 tx=0x7fcd40005840 comp rx=0 tx=0).stop 2024-04-25T00:32:23.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.287+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/3787168995 shutdown_connections 2024-04-25T00:32:23.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.287+0000 7fcd5d6cc640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcd440678c0 0x7fcd44069d80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.287+0000 7fcd5d6cc640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd581e7660 0x7fcd581e7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.287+0000 7fcd5d6cc640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd580b1550 0x7fcd580b19d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.287+0000 7fcd5d6cc640 1 --2- 172.21.15.29:0/3787168995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd40004810 0x7fcd5815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.287+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/3787168995 >> 172.21.15.29:0/3787168995 conn(0x7fcd580b2920 msgr2=0x7fcd580b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.287+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/3787168995 shutdown_connections 2024-04-25T00:32:23.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.287+0000 7fcd5d6cc640 1 -- 172.21.15.29:0/3787168995 wait complete. 2024-04-25T00:32:23.306 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:32:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.423+0000 7f96c7909640 1 Processor -- start 2024-04-25T00:32:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.423+0000 7f96c7909640 1 -- start start 2024-04-25T00:32:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c7909640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c00b2b70 0x7f96c0151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c7909640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c00b3130 con 0x7f96c015d980 2024-04-25T00:32:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c7909640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c00b3410 con 0x7f96c006dce0 2024-04-25T00:32:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c7909640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96c00cf560 con 0x7f96c00b2b70 2024-04-25T00:32:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c56a5640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f96c015d980 0x7f96c006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38476/0 (socket says 172.21.15.29:38476) 2024-04-25T00:32:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c56a5640 1 -- 172.21.15.29:0/2417005218 learned_addr learned my addr 172.21.15.29:0/2417005218 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c5ea6640 1 --2- 172.21.15.29:0/2417005218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c00b2b70 0x7f96c0151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 788045374 0 0) 0x7f96c00b3130 con 0x7f96c015d980 2024-04-25T00:32:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96ac003530 con 0x7f96c015d980 2024-04-25T00:32:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2490650700 0 0) 0x7f96c00b3410 con 0x7f96c006dce0 2024-04-25T00:32:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96c00b3130 con 0x7f96c006dce0 2024-04-25T00:32:23.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1807917793 0 0) 0x7f96ac003530 con 0x7f96c015d980 2024-04-25T00:32:23.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c00b2b70 msgr2=0x7f96c0151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 --2- 172.21.15.29:0/2417005218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c00b2b70 0x7f96c0151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 >> v1:172.21.15.157:6789/0 conn(0x7f96c006dce0 legacy=0x7f96c0150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c00b36f0 con 0x7f96c015d980 2024-04-25T00:32:23.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f96bc001680 con 0x7f96c015d980 2024-04-25T00:32:23.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 >> v1:172.21.15.29:6789/0 conn(0x7f96c015d980 legacy=0x7f96c006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 --2- 172.21.15.29:0/2417005218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96ac003b80 0x7f96ac003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 --2- 172.21.15.29:0/2417005218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c00b2b70 0x7f96ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 --2- 172.21.15.29:0/2417005218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96ac004d50 0x7f96ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f96c00cf560 con 0x7f96ac004d50 2024-04-25T00:32:23.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac00dc90 con 0x7f96ac003b80 2024-04-25T00:32:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96ac00de90 con 0x7f96c00b2b70 2024-04-25T00:32:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c5ea6640 1 --2- 172.21.15.29:0/2417005218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96ac004d50 0x7f96ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c56a5640 1 --2- 172.21.15.29:0/2417005218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96ac003b80 0x7f96ac003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c4ea4640 1 --2- 172.21.15.29:0/2417005218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c00b2b70 0x7f96ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c5ea6640 1 -- 172.21.15.29:0/2417005218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c00b2b70 msgr2=0x7f96ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c5ea6640 1 --2- 172.21.15.29:0/2417005218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c00b2b70 0x7f96ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c5ea6640 1 -- 172.21.15.29:0/2417005218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96ac003b80 msgr2=0x7f96ac003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c5ea6640 1 --2- 172.21.15.29:0/2417005218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96ac003b80 0x7f96ac003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c5ea6640 1 -- 172.21.15.29:0/2417005218 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac003530 con 0x7f96ac004d50 2024-04-25T00:32:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.427+0000 7f96c5ea6640 1 --2- 172.21.15.29:0/2417005218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96ac004d50 0x7f96ac0051d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f96b8007460 tx=0x7f96b801d180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a13780d401161de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b80b3070 con 0x7f96ac004d50 2024-04-25T00:32:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b80ae410 con 0x7f96ac004d50 2024-04-25T00:32:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96b7fff640 1 -- 172.21.15.29:0/2417005218 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8009d60 con 0x7f96ac004d50 2024-04-25T00:32:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 -- 172.21.15.29:0/2417005218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96ac004d50 msgr2=0x7f96ac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 --2- 172.21.15.29:0/2417005218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96ac004d50 0x7f96ac0051d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f96b8007460 tx=0x7f96b801d180 comp rx=0 tx=0).stop 2024-04-25T00:32:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c5ea6640 1 -- 172.21.15.29:0/2417005218 reap_dead start 2024-04-25T00:32:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 -- 172.21.15.29:0/2417005218 shutdown_connections 2024-04-25T00:32:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 -- 172.21.15.29:0/2417005218 >> 172.21.15.29:0/2417005218 conn(0x7f96c00b1870 msgr2=0x7f96c00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 -- 172.21.15.29:0/2417005218 shutdown_connections 2024-04-25T00:32:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 -- 172.21.15.29:0/2417005218 wait complete. 2024-04-25T00:32:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 Processor -- start 2024-04-25T00:32:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 -- start start 2024-04-25T00:32:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c0144590 0x7f96c01e3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c00cde00 0x7f96c006cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96c006d280 0x7f96c0159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f96c015dd60 con 0x7f96c00cde00 2024-04-25T00:32:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96c00b3cb0 con 0x7f96c006d280 2024-04-25T00:32:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c7909640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96c0151500 con 0x7f96c0144590 2024-04-25T00:32:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c4ea4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c00cde00 0x7f96c006cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c4ea4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c00cde00 0x7f96c006cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42916/0 (socket says 172.21.15.29:42916) 2024-04-25T00:32:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c4ea4640 1 -- 172.21.15.29:0/4113487778 learned_addr learned my addr 172.21.15.29:0/4113487778 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c56a5640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c0144590 0x7f96c01e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.431+0000 7f96c5ea6640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96c006d280 0x7f96c0159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96c4ea4640 1 -- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c0144590 msgr2=0x7f96c01e3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96c4ea4640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c0144590 0x7f96c01e3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96c4ea4640 1 -- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96c006d280 msgr2=0x7f96c0159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96c4ea4640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96c006d280 0x7f96c0159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96c4ea4640 1 -- 172.21.15.29:0/4113487778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c01e41c0 con 0x7f96c00cde00 2024-04-25T00:32:23.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96c4ea4640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c00cde00 0x7f96c006cd40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f96b000e7b0 tx=0x7f96b000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96b5ffb640 1 -- 172.21.15.29:0/4113487778 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b000ee60 con 0x7f96c00cde00 2024-04-25T00:32:23.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96b5ffb640 1 -- 172.21.15.29:0/4113487778 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b0018070 con 0x7f96c00cde00 2024-04-25T00:32:23.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96c7909640 1 -- 172.21.15.29:0/4113487778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c01e44a0 con 0x7f96c00cde00 2024-04-25T00:32:23.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96b5ffb640 1 -- 172.21.15.29:0/4113487778 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b000c6c0 con 0x7f96c00cde00 2024-04-25T00:32:23.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96c7909640 1 -- 172.21.15.29:0/4113487778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c01521f0 con 0x7f96c00cde00 2024-04-25T00:32:23.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96c7909640 1 -- 172.21.15.29:0/4113487778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c006d690 con 0x7f96c00cde00 2024-04-25T00:32:23.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96b5ffb640 1 -- 172.21.15.29:0/4113487778 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f96b000c860 con 0x7f96c00cde00 2024-04-25T00:32:23.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96b5ffb640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f96a40678f0 0x7f96a4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96b5ffb640 1 -- 172.21.15.29:0/4113487778 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f96b00857d0 con 0x7f96c00cde00 2024-04-25T00:32:23.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96c56a5640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f96a40678f0 0x7f96a4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.435+0000 7f96c56a5640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f96a40678f0 0x7f96a4069db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f96bc000c10 tx=0x7f96bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.439+0000 7f96b5ffb640 1 -- 172.21.15.29:0/4113487778 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f96c006d690 con 0x7f96c00cde00 2024-04-25T00:32:23.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.639+0000 7f96c7909640 1 -- 172.21.15.29:0/4113487778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96c01523c0 con 0x7f96c00cde00 2024-04-25T00:32:23.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.639+0000 7f96b5ffb640 1 -- 172.21.15.29:0/4113487778 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f96b00112e0 con 0x7f96c00cde00 2024-04-25T00:32:23.644 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:23.644 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1532858457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532858457},{"type":"v1","addr":"172.21.15.29:6833","nonce":1532858457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/718997473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":718997473},{"type":"v1","addr":"172.21.15.157:6835","nonce":718997473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1617208213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1617208213},{"type":"v1","addr":"172.21.15.157:6837","nonce":1617208213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:31:29.640568+0000","modified":"2024-04-25T00:31:31.633585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4252":{"gid":4252,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1924706890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1924706890},{"type":"v1","addr":"172.21.15.29:6835","nonce":1924706890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4252,"qdb_cluster":[4252]},"id":1}]} 2024-04-25T00:32:23.644 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 6 2024-04-25T00:32:23.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.643+0000 7f96c7909640 1 -- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f96a40678f0 msgr2=0x7f96a4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.643+0000 7f96c7909640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f96a40678f0 0x7f96a4069db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f96bc000c10 tx=0x7f96bc009040 comp rx=0 tx=0).stop 2024-04-25T00:32:23.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.643+0000 7f96c7909640 1 -- 172.21.15.29:0/4113487778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c00cde00 msgr2=0x7f96c006cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.643+0000 7f96c7909640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c00cde00 0x7f96c006cd40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f96b000e7b0 tx=0x7f96b000ec70 comp rx=0 tx=0).stop 2024-04-25T00:32:23.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.643+0000 7f96c7909640 1 -- 172.21.15.29:0/4113487778 shutdown_connections 2024-04-25T00:32:23.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.643+0000 7f96c7909640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f96a40678f0 0x7f96a4069db0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.643+0000 7f96c7909640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96c006d280 0x7f96c0159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.643+0000 7f96c7909640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c00cde00 0x7f96c006cd40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.643+0000 7f96c7909640 1 --2- 172.21.15.29:0/4113487778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c0144590 0x7f96c01e3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.643+0000 7f96c7909640 1 -- 172.21.15.29:0/4113487778 >> 172.21.15.29:0/4113487778 conn(0x7f96c00b1870 msgr2=0x7f96c0150c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.643+0000 7f96c7909640 1 -- 172.21.15.29:0/4113487778 shutdown_connections 2024-04-25T00:32:23.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.643+0000 7f96c7909640 1 -- 172.21.15.29:0/4113487778 wait complete. 2024-04-25T00:32:23.664 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:23.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d6263c640 1 Processor -- start 2024-04-25T00:32:23.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d6263c640 1 -- start start 2024-04-25T00:32:23.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d6263c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5d5c161700 0x7f5d5c0b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d6263c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5d5c0d0920 0x7f5d5c0d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d6263c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d5c0d1240 con 0x7f5d5c0b4d30 2024-04-25T00:32:23.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d6263c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5d5c161ae0 con 0x7f5d5c161700 2024-04-25T00:32:23.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d6263c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5d5c0cf520 con 0x7f5d5c0d0920 2024-04-25T00:32:23.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d5bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5d5c161700 0x7f5d5c0b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d5bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5d5c161700 0x7f5d5c0b6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42918/0 (socket says 172.21.15.29:42918) 2024-04-25T00:32:23.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d5bfff640 1 -- 172.21.15.29:0/256171091 learned_addr learned my addr 172.21.15.29:0/256171091 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:23.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d60bd9640 1 --2- 172.21.15.29:0/256171091 >> v2:172.21.15.157:3300/0 conn(0x7f5d5c0d0920 0x7f5d5c0d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d5bfff640 1 -- 172.21.15.29:0/256171091 >> v2:172.21.15.157:3300/0 conn(0x7f5d5c0d0920 msgr2=0x7f5d5c0d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d5bfff640 1 --2- 172.21.15.29:0/256171091 >> v2:172.21.15.157:3300/0 conn(0x7f5d5c0d0920 0x7f5d5c0d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d5bfff640 1 -- 172.21.15.29:0/256171091 >> v1:172.21.15.157:6789/0 conn(0x7f5d5c0b4d30 legacy=0x7f5d5c0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.787+0000 7f5d5bfff640 1 -- 172.21.15.29:0/256171091 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c0cf800 con 0x7f5d5c161700 2024-04-25T00:32:23.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.791+0000 7f5d5bfff640 1 --2- 172.21.15.29:0/256171091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d5c161700 0x7f5d5c0b6d60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c06bab0 tx=0x7f5d4c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b29f2de116994c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.791+0000 7f5d5affd640 1 -- 172.21.15.29:0/256171091 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0aa610 con 0x7f5d5c161700 2024-04-25T00:32:23.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.791+0000 7f5d5affd640 1 -- 172.21.15.29:0/256171091 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4c0af070 con 0x7f5d5c161700 2024-04-25T00:32:23.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.791+0000 7f5d5affd640 1 -- 172.21.15.29:0/256171091 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0aa9d0 con 0x7f5d5c161700 2024-04-25T00:32:23.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.791+0000 7f5d6263c640 1 -- 172.21.15.29:0/256171091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d5c161700 msgr2=0x7f5d5c0b6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.791+0000 7f5d6263c640 1 --2- 172.21.15.29:0/256171091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d5c161700 0x7f5d5c0b6d60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c06bab0 tx=0x7f5d4c01c4b0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.791+0000 7f5d6263c640 1 -- 172.21.15.29:0/256171091 shutdown_connections 2024-04-25T00:32:23.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.791+0000 7f5d6263c640 1 --2- 172.21.15.29:0/256171091 >> v2:172.21.15.157:3300/0 conn(0x7f5d5c0d0920 0x7f5d5c0d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.791+0000 7f5d6263c640 1 --2- 172.21.15.29:0/256171091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d5c161700 0x7f5d5c0b6d60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.791+0000 7f5d6263c640 1 -- 172.21.15.29:0/256171091 >> 172.21.15.29:0/256171091 conn(0x7f5d5c09d280 msgr2=0x7f5d5c0d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.791+0000 7f5d6263c640 1 -- 172.21.15.29:0/256171091 shutdown_connections 2024-04-25T00:32:23.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.791+0000 7f5d6263c640 1 -- 172.21.15.29:0/256171091 wait complete. 2024-04-25T00:32:23.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.791+0000 7f5d6263c640 1 Processor -- start 2024-04-25T00:32:23.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d6263c640 1 -- start start 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d6263c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d5c0b4d30 0x7f5d5c1555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d6263c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d5c0d0920 0x7f5d5c0c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d6263c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d5c161700 0x7f5d5c1f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d6263c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c0d28b0 con 0x7f5d5c161700 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d6263c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c0cf1c0 con 0x7f5d5c0d0920 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d6263c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5c0d2a30 con 0x7f5d5c0b4d30 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d60bd9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d5c161700 0x7f5d5c1f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d60bd9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d5c161700 0x7f5d5c1f6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42926/0 (socket says 172.21.15.29:42926) 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d60bd9640 1 -- 172.21.15.29:0/840518512 learned_addr learned my addr 172.21.15.29:0/840518512 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d5b7fe640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d5c0d0920 0x7f5d5c0c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d5bfff640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d5c0b4d30 0x7f5d5c1555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d60bd9640 1 -- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d5c0b4d30 msgr2=0x7f5d5c1555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d60bd9640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d5c0b4d30 0x7f5d5c1555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d60bd9640 1 -- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d5c0d0920 msgr2=0x7f5d5c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d60bd9640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d5c0d0920 0x7f5d5c0c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d60bd9640 1 -- 172.21.15.29:0/840518512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c0c7860 con 0x7f5d5c161700 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d5bfff640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d5c0b4d30 0x7f5d5c1555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d60bd9640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d5c161700 0x7f5d5c1f6f40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400cd70 tx=0x7f5d5400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d58ff9640 1 -- 172.21.15.29:0/840518512 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54003ba0 con 0x7f5d5c161700 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d58ff9640 1 -- 172.21.15.29:0/840518512 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d54003d40 con 0x7f5d5c161700 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d58ff9640 1 -- 172.21.15.29:0/840518512 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54004240 con 0x7f5d5c161700 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d6263c640 1 -- 172.21.15.29:0/840518512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5c0c7b40 con 0x7f5d5c161700 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d6263c640 1 -- 172.21.15.29:0/840518512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d5c0c8100 con 0x7f5d5c161700 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d6263c640 1 -- 172.21.15.29:0/840518512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d5c069270 con 0x7f5d5c161700 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d58ff9640 1 -- 172.21.15.29:0/840518512 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5d540043e0 con 0x7f5d5c161700 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d58ff9640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5d480678c0 0x7f5d48069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.795+0000 7f5d58ff9640 1 -- 172.21.15.29:0/840518512 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f5d540868f0 con 0x7f5d5c161700 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.799+0000 7f5d5bfff640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5d480678c0 0x7f5d48069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.799+0000 7f5d5bfff640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5d480678c0 0x7f5d48069d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0ae000 tx=0x7f5d4c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.799+0000 7f5d58ff9640 1 -- 172.21.15.29:0/840518512 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5d5401d340 con 0x7f5d5c161700 2024-04-25T00:32:23.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.963+0000 7f5d6263c640 1 -- 172.21.15.29:0/840518512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d5c0d1590 con 0x7f5d5c161700 2024-04-25T00:32:23.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.967+0000 7f5d58ff9640 1 -- 172.21.15.29:0/840518512 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16838 (secure 0 0 0) 0x7f5d540572e0 con 0x7f5d5c161700 2024-04-25T00:32:23.969 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:23.969 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":22,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:32:18.667140+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:31:27.778241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:31:28.627385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:23.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.967+0000 7f5d6263c640 1 -- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5d480678c0 msgr2=0x7f5d48069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.967+0000 7f5d6263c640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5d480678c0 0x7f5d48069d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0ae000 tx=0x7f5d4c008040 comp rx=0 tx=0).stop 2024-04-25T00:32:23.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.967+0000 7f5d6263c640 1 -- 172.21.15.29:0/840518512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d5c161700 msgr2=0x7f5d5c1f6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.967+0000 7f5d6263c640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d5c161700 0x7f5d5c1f6f40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400cd70 tx=0x7f5d5400a660 comp rx=0 tx=0).stop 2024-04-25T00:32:23.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.967+0000 7f5d6263c640 1 -- 172.21.15.29:0/840518512 shutdown_connections 2024-04-25T00:32:23.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.967+0000 7f5d6263c640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d5c161700 0x7f5d5c1f6f40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.967+0000 7f5d6263c640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5d480678c0 0x7f5d48069d80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.967+0000 7f5d6263c640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d5c0d0920 0x7f5d5c0c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.967+0000 7f5d6263c640 1 --2- 172.21.15.29:0/840518512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d5c0b4d30 0x7f5d5c1555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.967+0000 7f5d6263c640 1 -- 172.21.15.29:0/840518512 >> 172.21.15.29:0/840518512 conn(0x7f5d5c09d280 msgr2=0x7f5d5c06bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.967+0000 7f5d6263c640 1 -- 172.21.15.29:0/840518512 shutdown_connections 2024-04-25T00:32:23.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:23.967+0000 7f5d6263c640 1 -- 172.21.15.29:0/840518512 wait complete. 2024-04-25T00:32:23.989 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:32:24.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b607ce640 1 Processor -- start 2024-04-25T00:32:24.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b607ce640 1 -- start start 2024-04-25T00:32:24.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b607ce640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4b5815ad20 0x7f4b580cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b607ce640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4b580d0390 0x7f4b5815b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b607ce640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5815bc30 con 0x7f4b5815b640 2024-04-25T00:32:24.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b607ce640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4b580cb6a0 con 0x7f4b5815ad20 2024-04-25T00:32:24.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b607ce640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5815bf10 con 0x7f4b580d0390 2024-04-25T00:32:24.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b5e56a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4b5815ad20 0x7f4b580cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b5ed6b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4b5815b640 0x7f4b58071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38478/0 (socket says 172.21.15.29:38478) 2024-04-25T00:32:24.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b5e56a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4b5815ad20 0x7f4b580cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42936/0 (socket says 172.21.15.29:42936) 2024-04-25T00:32:24.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b5ed6b640 1 -- 172.21.15.29:0/3243745880 learned_addr learned my addr 172.21.15.29:0/3243745880 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:24.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b5dd69640 1 --2- 172.21.15.29:0/3243745880 >> v2:172.21.15.157:3300/0 conn(0x7f4b580d0390 0x7f4b5815b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b5e56a640 1 -- 172.21.15.29:0/3243745880 >> v2:172.21.15.157:3300/0 conn(0x7f4b580d0390 msgr2=0x7f4b5815b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:24.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b5e56a640 1 --2- 172.21.15.29:0/3243745880 >> v2:172.21.15.157:3300/0 conn(0x7f4b580d0390 0x7f4b5815b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b5e56a640 1 -- 172.21.15.29:0/3243745880 >> v1:172.21.15.29:6789/0 conn(0x7f4b5815b640 legacy=0x7f4b58071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:24.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b5e56a640 1 -- 172.21.15.29:0/3243745880 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5815c1f0 con 0x7f4b5815ad20 2024-04-25T00:32:24.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.103+0000 7f4b5dd69640 1 --2- 172.21.15.29:0/3243745880 >> v2:172.21.15.157:3300/0 conn(0x7f4b580d0390 0x7f4b5815b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:24.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b5e56a640 1 --2- 172.21.15.29:0/3243745880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b5815ad20 0x7f4b580cbb30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4b58072d40 tx=0x7f4b5401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f2240e4e722aba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:24.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b5d568640 1 -- 172.21.15.29:0/3243745880 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b540b0070 con 0x7f4b5815ad20 2024-04-25T00:32:24.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b5d568640 1 -- 172.21.15.29:0/3243745880 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b540ab600 con 0x7f4b5815ad20 2024-04-25T00:32:24.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b5d568640 1 -- 172.21.15.29:0/3243745880 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b540ab9d0 con 0x7f4b5815ad20 2024-04-25T00:32:24.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b607ce640 1 -- 172.21.15.29:0/3243745880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b5815ad20 msgr2=0x7f4b580cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:24.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b607ce640 1 --2- 172.21.15.29:0/3243745880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b5815ad20 0x7f4b580cbb30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4b58072d40 tx=0x7f4b5401c310 comp rx=0 tx=0).stop 2024-04-25T00:32:24.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b607ce640 1 -- 172.21.15.29:0/3243745880 shutdown_connections 2024-04-25T00:32:24.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b607ce640 1 --2- 172.21.15.29:0/3243745880 >> v2:172.21.15.157:3300/0 conn(0x7f4b580d0390 0x7f4b5815b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b607ce640 1 --2- 172.21.15.29:0/3243745880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b5815ad20 0x7f4b580cbb30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b607ce640 1 -- 172.21.15.29:0/3243745880 >> 172.21.15.29:0/3243745880 conn(0x7f4b580d0c40 msgr2=0x7f4b580d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:24.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b607ce640 1 -- 172.21.15.29:0/3243745880 shutdown_connections 2024-04-25T00:32:24.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b607ce640 1 -- 172.21.15.29:0/3243745880 wait complete. 2024-04-25T00:32:24.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b607ce640 1 Processor -- start 2024-04-25T00:32:24.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.107+0000 7f4b607ce640 1 -- start start 2024-04-25T00:32:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b607ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b580d0390 0x7f4b581462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b607ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b5815ad20 0x7f4b581548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b607ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b5815b640 0x7f4b580b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b607ce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4b580cc1f0 con 0x7f4b5815b640 2024-04-25T00:32:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b607ce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b580cc070 con 0x7f4b5815ad20 2024-04-25T00:32:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b607ce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b58072490 con 0x7f4b580d0390 2024-04-25T00:32:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b5ed6b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b5815b640 0x7f4b580b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b5ed6b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b5815b640 0x7f4b580b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42950/0 (socket says 172.21.15.29:42950) 2024-04-25T00:32:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b5ed6b640 1 -- 172.21.15.29:0/2572120793 learned_addr learned my addr 172.21.15.29:0/2572120793 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b5dd69640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b5815ad20 0x7f4b581548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b5e56a640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b580d0390 0x7f4b581462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b5ed6b640 1 -- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b580d0390 msgr2=0x7f4b581462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b5ed6b640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b580d0390 0x7f4b581462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b5ed6b640 1 -- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b5815ad20 msgr2=0x7f4b581548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b5ed6b640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b5815ad20 0x7f4b581548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b5ed6b640 1 -- 172.21.15.29:0/2572120793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b580b5770 con 0x7f4b5815b640 2024-04-25T00:32:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b5ed6b640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b5815b640 0x7f4b580b3900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009e00 tx=0x7f4b5000c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b4affd640 1 -- 172.21.15.29:0/2572120793 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50019070 con 0x7f4b5815b640 2024-04-25T00:32:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b4affd640 1 -- 172.21.15.29:0/2572120793 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b500078f0 con 0x7f4b5815b640 2024-04-25T00:32:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b607ce640 1 -- 172.21.15.29:0/2572120793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b580b1b30 con 0x7f4b5815b640 2024-04-25T00:32:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b4affd640 1 -- 172.21.15.29:0/2572120793 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50007ce0 con 0x7f4b5815b640 2024-04-25T00:32:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b607ce640 1 -- 172.21.15.29:0/2572120793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b580b5cf0 con 0x7f4b5815b640 2024-04-25T00:32:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b607ce640 1 -- 172.21.15.29:0/2572120793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b58069270 con 0x7f4b5815b640 2024-04-25T00:32:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b4affd640 1 -- 172.21.15.29:0/2572120793 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4b5001c070 con 0x7f4b5815b640 2024-04-25T00:32:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b4affd640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b3c0678c0 0x7f4b3c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b4affd640 1 -- 172.21.15.29:0/2572120793 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f4b50087a00 con 0x7f4b5815b640 2024-04-25T00:32:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.111+0000 7f4b5e56a640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b3c0678c0 0x7f4b3c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.115+0000 7f4b5e56a640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b3c0678c0 0x7f4b3c069d80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4b540ad000 tx=0x7f4b54008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:24.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.115+0000 7f4b4affd640 1 -- 172.21.15.29:0/2572120793 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4b50058490 con 0x7f4b5815b640 2024-04-25T00:32:24.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.311+0000 7f4b607ce640 1 -- 172.21.15.29:0/2572120793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4b580ce890 con 0x7f4b5815b640 2024-04-25T00:32:24.314 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-04-25T00:32:24.311+0000 7f3ac00b5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:32:24.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.311+0000 7f4b4affd640 1 -- 172.21.15.29:0/2572120793 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) ==== 161+0+0 (secure 0 0 0) 0x7f4b50057d90 con 0x7f4b5815b640 2024-04-25T00:32:24.316 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:32:24.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.315+0000 7f4b607ce640 1 -- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b3c0678c0 msgr2=0x7f4b3c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:24.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.315+0000 7f4b607ce640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b3c0678c0 0x7f4b3c069d80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4b540ad000 tx=0x7f4b54008040 comp rx=0 tx=0).stop 2024-04-25T00:32:24.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.315+0000 7f4b607ce640 1 -- 172.21.15.29:0/2572120793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b5815b640 msgr2=0x7f4b580b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:24.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.315+0000 7f4b607ce640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b5815b640 0x7f4b580b3900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009e00 tx=0x7f4b5000c430 comp rx=0 tx=0).stop 2024-04-25T00:32:24.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.319+0000 7f4b607ce640 1 -- 172.21.15.29:0/2572120793 shutdown_connections 2024-04-25T00:32:24.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.319+0000 7f4b607ce640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b5815b640 0x7f4b580b3900 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.319+0000 7f4b607ce640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b3c0678c0 0x7f4b3c069d80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.319+0000 7f4b607ce640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b5815ad20 0x7f4b581548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.319+0000 7f4b607ce640 1 --2- 172.21.15.29:0/2572120793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b580d0390 0x7f4b581462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.319+0000 7f4b607ce640 1 -- 172.21.15.29:0/2572120793 >> 172.21.15.29:0/2572120793 conn(0x7f4b580d0c40 msgr2=0x7f4b580b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:24.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.319+0000 7f4b607ce640 1 -- 172.21.15.29:0/2572120793 shutdown_connections 2024-04-25T00:32:24.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.319+0000 7f4b607ce640 1 -- 172.21.15.29:0/2572120793 wait complete. 2024-04-25T00:32:24.326 INFO:tasks.ceph.mds.c.smithi029.stderr: -3383> 2024-04-25T00:31:34.536+0000 7f469ae65640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:32:24.339 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:32:24.368 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:32:24.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23c3f1640 1 Processor -- start 2024-04-25T00:32:24.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23c3f1640 1 -- start start 2024-04-25T00:32:24.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23c3f1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2340b3140 0x7fc23406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23c3f1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc2340b4b10 0x7fc2340b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23c3f1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc2340b3a60 0x7fc23415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23c3f1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc2340b41e0 con 0x7fc2340b3a60 2024-04-25T00:32:24.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23c3f1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc23406d960 con 0x7fc2340b4b10 2024-04-25T00:32:24.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23c3f1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc23406dc40 con 0x7fc2340b3140 2024-04-25T00:32:24.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23a98e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc2340b3a60 0x7fc23415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23a98e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc2340b3a60 0x7fc23415c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42968/0 (socket says 172.21.15.29:42968) 2024-04-25T00:32:24.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23a98e640 1 -- 172.21.15.29:0/2029209158 learned_addr learned my addr 172.21.15.29:0/2029209158 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:24.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23998c640 1 --2- 172.21.15.29:0/2029209158 >> v2:172.21.15.157:3300/0 conn(0x7fc2340b4b10 0x7fc2340b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23a18d640 1 --2- 172.21.15.29:0/2029209158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2340b3140 0x7fc23406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23a98e640 1 -- 172.21.15.29:0/2029209158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2340b3140 msgr2=0x7fc23406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:24.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23a98e640 1 --2- 172.21.15.29:0/2029209158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2340b3140 0x7fc23406e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23a98e640 1 -- 172.21.15.29:0/2029209158 >> v2:172.21.15.157:3300/0 conn(0x7fc2340b4b10 msgr2=0x7fc2340b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:24.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23a98e640 1 --2- 172.21.15.29:0/2029209158 >> v2:172.21.15.157:3300/0 conn(0x7fc2340b4b10 0x7fc2340b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.447+0000 7fc23a98e640 1 -- 172.21.15.29:0/2029209158 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23406df20 con 0x7fc2340b3a60 2024-04-25T00:32:24.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23a98e640 1 --2- 172.21.15.29:0/2029209158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2340b3a60 0x7fc23415c3e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc234074fb0 tx=0x7fc22c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47587bf4e703839a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:24.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23918b640 1 -- 172.21.15.29:0/2029209158 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c0a5670 con 0x7fc2340b3a60 2024-04-25T00:32:24.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23918b640 1 -- 172.21.15.29:0/2029209158 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22c0a5810 con 0x7fc2340b3a60 2024-04-25T00:32:24.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23918b640 1 -- 172.21.15.29:0/2029209158 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c0a5be0 con 0x7fc2340b3a60 2024-04-25T00:32:24.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2029209158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2340b3a60 msgr2=0x7fc23415c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:24.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23c3f1640 1 --2- 172.21.15.29:0/2029209158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2340b3a60 0x7fc23415c3e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc234074fb0 tx=0x7fc22c01c9f0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2029209158 shutdown_connections 2024-04-25T00:32:24.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23c3f1640 1 --2- 172.21.15.29:0/2029209158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2340b3a60 0x7fc23415c3e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23c3f1640 1 --2- 172.21.15.29:0/2029209158 >> v2:172.21.15.157:3300/0 conn(0x7fc2340b4b10 0x7fc2340b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23c3f1640 1 --2- 172.21.15.29:0/2029209158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2340b3140 0x7fc23406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2029209158 >> 172.21.15.29:0/2029209158 conn(0x7fc2340b1870 msgr2=0x7fc2340b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:24.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2029209158 shutdown_connections 2024-04-25T00:32:24.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2029209158 wait complete. 2024-04-25T00:32:24.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23c3f1640 1 Processor -- start 2024-04-25T00:32:24.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.451+0000 7fc23c3f1640 1 -- start start 2024-04-25T00:32:24.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23c3f1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2340b3140 0x7fc23415d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23c3f1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2340b3a60 0x7fc23413ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23c3f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2340b4b10 0x7fc2341534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23c3f1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2340ce3c0 con 0x7fc2340b3a60 2024-04-25T00:32:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23c3f1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc2340ce240 con 0x7fc2340b4b10 2024-04-25T00:32:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23c3f1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc2340ce540 con 0x7fc2340b3140 2024-04-25T00:32:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23998c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2340b3a60 0x7fc23413ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23998c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2340b3a60 0x7fc23413ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42974/0 (socket says 172.21.15.29:42974) 2024-04-25T00:32:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23998c640 1 -- 172.21.15.29:0/2309938357 learned_addr learned my addr 172.21.15.29:0/2309938357 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23998c640 1 -- 172.21.15.29:0/2309938357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2340b3140 msgr2=0x7fc23415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23998c640 1 --2- 172.21.15.29:0/2309938357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2340b3140 0x7fc23415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23998c640 1 -- 172.21.15.29:0/2309938357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2340b4b10 msgr2=0x7fc2341534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23998c640 1 --2- 172.21.15.29:0/2309938357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2340b4b10 0x7fc2341534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23998c640 1 -- 172.21.15.29:0/2309938357 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc234154160 con 0x7fc2340b3a60 2024-04-25T00:32:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23998c640 1 --2- 172.21.15.29:0/2309938357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2340b3a60 0x7fc23413ea00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc22400c7f0 tx=0x7fc22400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc22affd640 1 -- 172.21.15.29:0/2309938357 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224015070 con 0x7fc2340b3a60 2024-04-25T00:32:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc22affd640 1 -- 172.21.15.29:0/2309938357 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc224007820 con 0x7fc2340b3a60 2024-04-25T00:32:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2309938357 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc234154440 con 0x7fc2340b3a60 2024-04-25T00:32:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc22affd640 1 -- 172.21.15.29:0/2309938357 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224007c10 con 0x7fc2340b3a60 2024-04-25T00:32:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2309938357 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc234154a00 con 0x7fc2340b3a60 2024-04-25T00:32:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2309938357 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc234069270 con 0x7fc2340b3a60 2024-04-25T00:32:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc22affd640 1 -- 172.21.15.29:0/2309938357 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc224018070 con 0x7fc2340b3a60 2024-04-25T00:32:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc22affd640 1 --2- 172.21.15.29:0/2309938357 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc2180678c0 0x7fc218069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc22affd640 1 -- 172.21.15.29:0/2309938357 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5969+0+0 (secure 0 0 0) 0x7fc224085e80 con 0x7fc2340b3a60 2024-04-25T00:32:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23a18d640 1 --2- 172.21.15.29:0/2309938357 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc2180678c0 0x7fc218069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.455+0000 7fc23a18d640 1 --2- 172.21.15.29:0/2309938357 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc2180678c0 0x7fc218069d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc230006b20 tx=0x7fc230008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:24.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.459+0000 7fc22affd640 1 -- 172.21.15.29:0/2309938357 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc2240568b0 con 0x7fc2340b3a60 2024-04-25T00:32:24.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:24.655+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2309938357 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc2340cdd20 con 0x7fc2340b3a60 2024-04-25T00:32:25.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.315+0000 7fc22affd640 1 -- 172.21.15.29:0/2309938357 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) ==== 104+0+0 (secure 0 0 0) 0x7fc2240561b0 con 0x7fc2340b3a60 2024-04-25T00:32:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.319+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2309938357 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc2180678c0 msgr2=0x7fc218069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.319+0000 7fc23c3f1640 1 --2- 172.21.15.29:0/2309938357 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc2180678c0 0x7fc218069d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc230006b20 tx=0x7fc230008040 comp rx=0 tx=0).stop 2024-04-25T00:32:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.319+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2309938357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2340b3a60 msgr2=0x7fc23413ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.319+0000 7fc23c3f1640 1 --2- 172.21.15.29:0/2309938357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2340b3a60 0x7fc23413ea00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc22400c7f0 tx=0x7fc22400ccb0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.319+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2309938357 shutdown_connections 2024-04-25T00:32:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.319+0000 7fc23c3f1640 1 --2- 172.21.15.29:0/2309938357 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc2180678c0 0x7fc218069d80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.319+0000 7fc23c3f1640 1 --2- 172.21.15.29:0/2309938357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2340b4b10 0x7fc2341534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.319+0000 7fc23c3f1640 1 --2- 172.21.15.29:0/2309938357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2340b3a60 0x7fc23413ea00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.319+0000 7fc23c3f1640 1 --2- 172.21.15.29:0/2309938357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2340b3140 0x7fc23415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.319+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2309938357 >> 172.21.15.29:0/2309938357 conn(0x7fc2340b1870 msgr2=0x7fc234143030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:25.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.319+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2309938357 shutdown_connections 2024-04-25T00:32:25.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.319+0000 7fc23c3f1640 1 -- 172.21.15.29:0/2309938357 wait complete. 2024-04-25T00:32:25.346 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:32:25.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe37fb640 1 Processor -- start 2024-04-25T00:32:25.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe37fb640 1 -- start start 2024-04-25T00:32:25.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe37fb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fafdc1616e0 0x7fafdc0d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe37fb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fafdc0d10f0 0x7fafdc0b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe37fb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafdc0cf740 con 0x7fafdc161ac0 2024-04-25T00:32:25.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe37fb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fafdc0b56b0 con 0x7fafdc1616e0 2024-04-25T00:32:25.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe37fb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fafdc0cf9e0 con 0x7fafdc0d10f0 2024-04-25T00:32:25.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe1597640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fafdc1616e0 0x7fafdc0d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe1597640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fafdc1616e0 0x7fafdc0d4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42976/0 (socket says 172.21.15.29:42976) 2024-04-25T00:32:25.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe1597640 1 -- 172.21.15.29:0/106556493 learned_addr learned my addr 172.21.15.29:0/106556493 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:25.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe1597640 1 -- 172.21.15.29:0/106556493 >> v2:172.21.15.157:3300/0 conn(0x7fafdc0d10f0 msgr2=0x7fafdc0b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:25.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe1d98640 1 --2- 172.21.15.29:0/106556493 >> v2:172.21.15.157:3300/0 conn(0x7fafdc0d10f0 0x7fafdc0b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe1597640 1 --2- 172.21.15.29:0/106556493 >> v2:172.21.15.157:3300/0 conn(0x7fafdc0d10f0 0x7fafdc0b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe1597640 1 -- 172.21.15.29:0/106556493 >> v1:172.21.15.157:6789/0 conn(0x7fafdc161ac0 legacy=0x7fafdc0d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:25.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe1597640 1 -- 172.21.15.29:0/106556493 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc0cfcc0 con 0x7fafdc1616e0 2024-04-25T00:32:25.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.527+0000 7fafe1597640 1 --2- 172.21.15.29:0/106556493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafdc1616e0 0x7fafdc0d4700 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fafdc08e3f0 tx=0x7fafd801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9625f0acd149378 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:25.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafd3fff640 1 -- 172.21.15.29:0/106556493 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd80b1070 con 0x7fafdc1616e0 2024-04-25T00:32:25.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafd3fff640 1 -- 172.21.15.29:0/106556493 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd80ac630 con 0x7fafdc1616e0 2024-04-25T00:32:25.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafd3fff640 1 -- 172.21.15.29:0/106556493 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd80a8050 con 0x7fafdc1616e0 2024-04-25T00:32:25.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 -- 172.21.15.29:0/106556493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafdc1616e0 msgr2=0x7fafdc0d4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 --2- 172.21.15.29:0/106556493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafdc1616e0 0x7fafdc0d4700 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fafdc08e3f0 tx=0x7fafd801c4c0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 -- 172.21.15.29:0/106556493 shutdown_connections 2024-04-25T00:32:25.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 --2- 172.21.15.29:0/106556493 >> v2:172.21.15.157:3300/0 conn(0x7fafdc0d10f0 0x7fafdc0b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 --2- 172.21.15.29:0/106556493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafdc1616e0 0x7fafdc0d4700 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 -- 172.21.15.29:0/106556493 >> 172.21.15.29:0/106556493 conn(0x7fafdc09d300 msgr2=0x7fafdc0d3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:25.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 -- 172.21.15.29:0/106556493 shutdown_connections 2024-04-25T00:32:25.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 -- 172.21.15.29:0/106556493 wait complete. 2024-04-25T00:32:25.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 Processor -- start 2024-04-25T00:32:25.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 -- start start 2024-04-25T00:32:25.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafdc0d10f0 0x7fafdc1facf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafdc1616e0 0x7fafdc0cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafdc161ac0 0x7fafdc0cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc0b7c60 con 0x7fafdc161ac0 2024-04-25T00:32:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc0b7ae0 con 0x7fafdc1616e0 2024-04-25T00:32:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe37fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fafdc0b7de0 con 0x7fafdc0d10f0 2024-04-25T00:32:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe1d98640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafdc161ac0 0x7fafdc0cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe1d98640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafdc161ac0 0x7fafdc0cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42990/0 (socket says 172.21.15.29:42990) 2024-04-25T00:32:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe1d98640 1 -- 172.21.15.29:0/912229379 learned_addr learned my addr 172.21.15.29:0/912229379 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe0d96640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafdc1616e0 0x7fafdc0cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe1597640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafdc0d10f0 0x7fafdc1facf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe1d98640 1 -- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafdc0d10f0 msgr2=0x7fafdc1facf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe1d98640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafdc0d10f0 0x7fafdc1facf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe1d98640 1 -- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafdc1616e0 msgr2=0x7fafdc0cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe1d98640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafdc1616e0 0x7fafdc0cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe1d98640 1 -- 172.21.15.29:0/912229379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc1fb230 con 0x7fafdc161ac0 2024-04-25T00:32:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe0d96640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafdc1616e0 0x7fafdc0cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafe1d98640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafdc161ac0 0x7fafdc0cd850 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fafd400cd60 tx=0x7fafd400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafd1ffb640 1 -- 172.21.15.29:0/912229379 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4003ba0 con 0x7fafdc161ac0 2024-04-25T00:32:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafd1ffb640 1 -- 172.21.15.29:0/912229379 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd4003d60 con 0x7fafdc161ac0 2024-04-25T00:32:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.531+0000 7fafd1ffb640 1 -- 172.21.15.29:0/912229379 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4004260 con 0x7fafdc161ac0 2024-04-25T00:32:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.535+0000 7fafe37fb640 1 -- 172.21.15.29:0/912229379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc1fb500 con 0x7fafdc161ac0 2024-04-25T00:32:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.535+0000 7fafe37fb640 1 -- 172.21.15.29:0/912229379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc1fbac0 con 0x7fafdc161ac0 2024-04-25T00:32:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.535+0000 7fafe37fb640 1 -- 172.21.15.29:0/912229379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafdc0692e0 con 0x7fafdc161ac0 2024-04-25T00:32:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.535+0000 7fafd1ffb640 1 -- 172.21.15.29:0/912229379 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fafd4004400 con 0x7fafdc161ac0 2024-04-25T00:32:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.535+0000 7fafd1ffb640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fafc00678c0 0x7fafc0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.535+0000 7fafd1ffb640 1 -- 172.21.15.29:0/912229379 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5969+0+0 (secure 0 0 0) 0x7fafd4086830 con 0x7fafdc161ac0 2024-04-25T00:32:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.535+0000 7fafe1597640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fafc00678c0 0x7fafc0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.535+0000 7fafe1597640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fafc00678c0 0x7fafc0069d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fafd80ae000 tx=0x7fafd8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.539+0000 7fafd1ffb640 1 -- 172.21.15.29:0/912229379 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fafd4057250 con 0x7fafdc161ac0 2024-04-25T00:32:25.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:25.695+0000 7fafe37fb640 1 -- 172.21.15.29:0/912229379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fafdc068710 con 0x7fafdc161ac0 2024-04-25T00:32:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.323+0000 7fafd1ffb640 1 -- 172.21.15.29:0/912229379 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7fafd4056b50 con 0x7fafdc161ac0 2024-04-25T00:32:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:32:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.331+0000 7fafe37fb640 1 -- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fafc00678c0 msgr2=0x7fafc0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.331+0000 7fafe37fb640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fafc00678c0 0x7fafc0069d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fafd80ae000 tx=0x7fafd8008040 comp rx=0 tx=0).stop 2024-04-25T00:32:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.331+0000 7fafe37fb640 1 -- 172.21.15.29:0/912229379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafdc161ac0 msgr2=0x7fafdc0cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.331+0000 7fafe37fb640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafdc161ac0 0x7fafdc0cd850 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fafd400cd60 tx=0x7fafd400a660 comp rx=0 tx=0).stop 2024-04-25T00:32:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.331+0000 7fafe37fb640 1 -- 172.21.15.29:0/912229379 shutdown_connections 2024-04-25T00:32:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.331+0000 7fafe37fb640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafdc161ac0 0x7fafdc0cd850 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.331+0000 7fafe37fb640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fafc00678c0 0x7fafc0069d80 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fafd80ae000 tx=0x7fafd8008040 comp rx=0 tx=0).stop 2024-04-25T00:32:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.331+0000 7fafe37fb640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafdc1616e0 0x7fafdc0cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.331+0000 7fafe37fb640 1 --2- 172.21.15.29:0/912229379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fafdc0d10f0 0x7fafdc1facf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.331+0000 7fafe37fb640 1 -- 172.21.15.29:0/912229379 >> 172.21.15.29:0/912229379 conn(0x7fafdc09d300 msgr2=0x7fafdc15a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.331+0000 7fafe37fb640 1 -- 172.21.15.29:0/912229379 shutdown_connections 2024-04-25T00:32:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.331+0000 7fafe37fb640 1 -- 172.21.15.29:0/912229379 wait complete. 2024-04-25T00:32:26.355 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:32:26.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.475+0000 7f1624024640 1 Processor -- start 2024-04-25T00:32:26.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.475+0000 7f1624024640 1 -- start start 2024-04-25T00:32:26.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.475+0000 7f1624024640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f161c15e9a0 0x7f161c070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.475+0000 7f1624024640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f161c0b4590 0x7f161c15ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.475+0000 7f1624024640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f161c15f2c0 con 0x7f161c0cd1c0 2024-04-25T00:32:26.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.475+0000 7f1624024640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f161c071ed0 con 0x7f161c0b4590 2024-04-25T00:32:26.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.475+0000 7f1624024640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f161c0712c0 con 0x7f161c15e9a0 2024-04-25T00:32:26.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.475+0000 7f16225c1640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f161c0cd1c0 0x7f161c072360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38492/0 (socket says 172.21.15.29:38492) 2024-04-25T00:32:26.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.475+0000 7f16225c1640 1 -- 172.21.15.29:0/3645448331 learned_addr learned my addr 172.21.15.29:0/3645448331 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:26.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.475+0000 7f16215bf640 1 --2- 172.21.15.29:0/3645448331 >> v2:172.21.15.29:3300/0 conn(0x7f161c0b4590 0x7f161c15ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:26.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.475+0000 7f1621dc0640 1 --2- 172.21.15.29:0/3645448331 >> v2:172.21.15.157:3300/0 conn(0x7f161c15e9a0 0x7f161c070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:26.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.475+0000 7f1620dbe640 1 -- 172.21.15.29:0/3645448331 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352205177 0 0) 0x7f161c15f2c0 con 0x7f161c0cd1c0 2024-04-25T00:32:26.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.475+0000 7f1620dbe640 1 -- 172.21.15.29:0/3645448331 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1608003530 con 0x7f161c0cd1c0 2024-04-25T00:32:26.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f16215bf640 1 -- 172.21.15.29:0/3645448331 >> v2:172.21.15.157:3300/0 conn(0x7f161c15e9a0 msgr2=0x7f161c070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:26.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f16215bf640 1 --2- 172.21.15.29:0/3645448331 >> v2:172.21.15.157:3300/0 conn(0x7f161c15e9a0 0x7f161c070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f16215bf640 1 -- 172.21.15.29:0/3645448331 >> v1:172.21.15.29:6789/0 conn(0x7f161c0cd1c0 legacy=0x7f161c072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f16215bf640 1 -- 172.21.15.29:0/3645448331 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161c0715a0 con 0x7f161c0b4590 2024-04-25T00:32:26.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f16215bf640 1 --2- 172.21.15.29:0/3645448331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c0b4590 0x7f161c15ed80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f161c0b5260 tx=0x7f160c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22eecc9c4bbba039 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:26.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f1620dbe640 1 -- 172.21.15.29:0/3645448331 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c0ab650 con 0x7f161c0b4590 2024-04-25T00:32:26.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f1620dbe640 1 -- 172.21.15.29:0/3645448331 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160c0b0070 con 0x7f161c0b4590 2024-04-25T00:32:26.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f1620dbe640 1 -- 172.21.15.29:0/3645448331 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c0aba10 con 0x7f161c0b4590 2024-04-25T00:32:26.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f1624024640 1 -- 172.21.15.29:0/3645448331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c0b4590 msgr2=0x7f161c15ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f1624024640 1 --2- 172.21.15.29:0/3645448331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c0b4590 0x7f161c15ed80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f161c0b5260 tx=0x7f160c01c450 comp rx=0 tx=0).stop 2024-04-25T00:32:26.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f1624024640 1 -- 172.21.15.29:0/3645448331 shutdown_connections 2024-04-25T00:32:26.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f1624024640 1 --2- 172.21.15.29:0/3645448331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c0b4590 0x7f161c15ed80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f1624024640 1 --2- 172.21.15.29:0/3645448331 >> v2:172.21.15.157:3300/0 conn(0x7f161c15e9a0 0x7f161c070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f1624024640 1 -- 172.21.15.29:0/3645448331 >> 172.21.15.29:0/3645448331 conn(0x7f161c0b1870 msgr2=0x7f161c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:26.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f1624024640 1 -- 172.21.15.29:0/3645448331 shutdown_connections 2024-04-25T00:32:26.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.479+0000 7f1624024640 1 -- 172.21.15.29:0/3645448331 wait complete. 2024-04-25T00:32:26.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1624024640 1 Processor -- start 2024-04-25T00:32:26.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1624024640 1 -- start start 2024-04-25T00:32:26.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1624024640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c0b4590 0x7f161c13d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1624024640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f161c0cd1c0 0x7f161c0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1624024640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f161c15e9a0 0x7f161c0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1624024640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f161c144870 con 0x7f161c0b4590 2024-04-25T00:32:26.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1624024640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f161c072e80 con 0x7f161c15e9a0 2024-04-25T00:32:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1624024640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f161c1449f0 con 0x7f161c0cd1c0 2024-04-25T00:32:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1621dc0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c0b4590 0x7f161c13d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1621dc0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c0b4590 0x7f161c13d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42994/0 (socket says 172.21.15.29:42994) 2024-04-25T00:32:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1621dc0640 1 -- 172.21.15.29:0/2520147548 learned_addr learned my addr 172.21.15.29:0/2520147548 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1621dc0640 1 -- 172.21.15.29:0/2520147548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f161c0cd1c0 msgr2=0x7f161c0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1621dc0640 1 --2- 172.21.15.29:0/2520147548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f161c0cd1c0 0x7f161c0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1621dc0640 1 -- 172.21.15.29:0/2520147548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f161c15e9a0 msgr2=0x7f161c0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1621dc0640 1 --2- 172.21.15.29:0/2520147548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f161c15e9a0 0x7f161c0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1621dc0640 1 -- 172.21.15.29:0/2520147548 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161c159a10 con 0x7f161c0b4590 2024-04-25T00:32:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1621dc0640 1 --2- 172.21.15.29:0/2520147548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c0b4590 0x7f161c13d020 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f161800e930 tx=0x7f161800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f16127fc640 1 -- 172.21.15.29:0/2520147548 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618017070 con 0x7f161c0b4590 2024-04-25T00:32:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f16127fc640 1 -- 172.21.15.29:0/2520147548 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161800ca00 con 0x7f161c0b4590 2024-04-25T00:32:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1624024640 1 -- 172.21.15.29:0/2520147548 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161c13aaa0 con 0x7f161c0b4590 2024-04-25T00:32:26.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f16127fc640 1 -- 172.21.15.29:0/2520147548 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161800cdb0 con 0x7f161c0b4590 2024-04-25T00:32:26.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1624024640 1 -- 172.21.15.29:0/2520147548 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f161c13b060 con 0x7f161c0b4590 2024-04-25T00:32:26.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f1624024640 1 -- 172.21.15.29:0/2520147548 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161c0692e0 con 0x7f161c0b4590 2024-04-25T00:32:26.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.483+0000 7f16127fc640 1 -- 172.21.15.29:0/2520147548 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f161801a070 con 0x7f161c0b4590 2024-04-25T00:32:26.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.487+0000 7f16127fc640 1 --2- 172.21.15.29:0/2520147548 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f16000678c0 0x7f1600069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.487+0000 7f16127fc640 1 -- 172.21.15.29:0/2520147548 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5531+0+0 (secure 0 0 0) 0x7f1618085c70 con 0x7f161c0b4590 2024-04-25T00:32:26.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.487+0000 7f16215bf640 1 --2- 172.21.15.29:0/2520147548 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f16000678c0 0x7f1600069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:26.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.487+0000 7f16215bf640 1 --2- 172.21.15.29:0/2520147548 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f16000678c0 0x7f1600069d80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f160c0af000 tx=0x7f160c004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:26.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.491+0000 7f16127fc640 1 -- 172.21.15.29:0/2520147548 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1618056920 con 0x7f161c0b4590 2024-04-25T00:32:26.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:26.651+0000 7f1624024640 1 -- 172.21.15.29:0/2520147548 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f161c06e7b0 con 0x7f161c0b4590 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.327+0000 7f16127fc640 1 -- 172.21.15.29:0/2520147548 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f1618056220 con 0x7f161c0b4590 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.331+0000 7f1624024640 1 -- 172.21.15.29:0/2520147548 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f16000678c0 msgr2=0x7f1600069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.331+0000 7f1624024640 1 --2- 172.21.15.29:0/2520147548 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f16000678c0 0x7f1600069d80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f160c0af000 tx=0x7f160c004740 comp rx=0 tx=0).stop 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.331+0000 7f1624024640 1 -- 172.21.15.29:0/2520147548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c0b4590 msgr2=0x7f161c13d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.331+0000 7f1624024640 1 --2- 172.21.15.29:0/2520147548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c0b4590 0x7f161c13d020 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f161800e930 tx=0x7f161800edf0 comp rx=0 tx=0).stop 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.331+0000 7f1624024640 1 -- 172.21.15.29:0/2520147548 shutdown_connections 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.331+0000 7f1624024640 1 --2- 172.21.15.29:0/2520147548 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f16000678c0 0x7f1600069d80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.331+0000 7f1624024640 1 --2- 172.21.15.29:0/2520147548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f161c15e9a0 0x7f161c0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.331+0000 7f1624024640 1 --2- 172.21.15.29:0/2520147548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f161c0cd1c0 0x7f161c0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.331+0000 7f1624024640 1 --2- 172.21.15.29:0/2520147548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f161c0b4590 0x7f161c13d020 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.331+0000 7f1624024640 1 -- 172.21.15.29:0/2520147548 >> 172.21.15.29:0/2520147548 conn(0x7f161c0b1870 msgr2=0x7f161c142e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.331+0000 7f1624024640 1 -- 172.21.15.29:0/2520147548 shutdown_connections 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.331+0000 7f1624024640 1 -- 172.21.15.29:0/2520147548 wait complete. 2024-04-25T00:32:27.363 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:32:27.363 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:32:27.363 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:32:27.363 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:32:27.363 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:32:27.364 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:32:27.364 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:32:27.364 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:32:27.364 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:32:27.364 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:32:27.365 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:32:27.365 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:32:27.374 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:32:27.374 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:32:27.376 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:32:27.376 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:32:27.383 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:32:27.384 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:32:27.384 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:32:27.385 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:32:27.418 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:32:27.419 DEBUG:teuthology.parallel:result is None 2024-04-25T00:32:27.420 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:32:27.420 DEBUG:teuthology.parallel:result is None 2024-04-25T00:32:27.430 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:32:27.430 DEBUG:teuthology.parallel:result is None 2024-04-25T00:32:27.430 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:32:27.431 DEBUG:teuthology.parallel:result is None 2024-04-25T00:32:27.431 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:32:27.486 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:32:27.487 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:32:27.496 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-25T00:32:27.498 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a3a611640 1 Processor -- start 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a3a611640 1 -- start start 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a3a611640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3a3415e200 0x7f3a340d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a3a611640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a340cf690 0x7f3a340d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a3a611640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a340cfc10 con 0x7f3a340d2a90 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a3a611640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f3a340d2ee0 con 0x7f3a3415e200 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a3a611640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a340b38d0 con 0x7f3a340cf690 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a33fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3a3415e200 0x7f3a340d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a33fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3a3415e200 0x7f3a340d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43030/0 (socket says 172.21.15.29:43030) 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a33fff640 1 -- 172.21.15.29:0/3660213772 learned_addr learned my addr 172.21.15.29:0/3660213772 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a337fe640 1 --2- 172.21.15.29:0/3660213772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a340cf690 0x7f3a340d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a33fff640 1 -- 172.21.15.29:0/3660213772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a340cf690 msgr2=0x7f3a340d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a33fff640 1 --2- 172.21.15.29:0/3660213772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a340cf690 0x7f3a340d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a33fff640 1 -- 172.21.15.29:0/3660213772 >> v1:172.21.15.157:6789/0 conn(0x7f3a340d2a90 legacy=0x7f3a340d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a33fff640 1 -- 172.21.15.29:0/3660213772 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a340cfef0 con 0x7f3a3415e200 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.555+0000 7f3a33fff640 1 --2- 172.21.15.29:0/3660213772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a3415e200 0x7f3a340d47f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009e50 tx=0x7f3a2401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b050df0a5991a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a32ffd640 1 -- 172.21.15.29:0/3660213772 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a240a58d0 con 0x7f3a3415e200 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a32ffd640 1 -- 172.21.15.29:0/3660213772 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a240a5a70 con 0x7f3a3415e200 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a32ffd640 1 -- 172.21.15.29:0/3660213772 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a240a5e40 con 0x7f3a3415e200 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 -- 172.21.15.29:0/3660213772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a3415e200 msgr2=0x7f3a340d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 --2- 172.21.15.29:0/3660213772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a3415e200 0x7f3a340d47f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009e50 tx=0x7f3a2401ca00 comp rx=0 tx=0).stop 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 -- 172.21.15.29:0/3660213772 shutdown_connections 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 --2- 172.21.15.29:0/3660213772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a340cf690 0x7f3a340d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 --2- 172.21.15.29:0/3660213772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a3415e200 0x7f3a340d47f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 -- 172.21.15.29:0/3660213772 >> 172.21.15.29:0/3660213772 conn(0x7f3a3409d280 msgr2=0x7f3a340d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 -- 172.21.15.29:0/3660213772 shutdown_connections 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 -- 172.21.15.29:0/3660213772 wait complete. 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 Processor -- start 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 -- start start 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a340cf690 0x7f3a340b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a340d2a90 0x7f3a340b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a3415e200 0x7f3a340b9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3415e760 con 0x7f3a3415e200 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3415e5e0 con 0x7f3a340cf690 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a3416ae30 con 0x7f3a340d2a90 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a38bae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a3415e200 0x7f3a340b9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a38bae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a3415e200 0x7f3a340b9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43038/0 (socket says 172.21.15.29:43038) 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a38bae640 1 -- 172.21.15.29:0/345053498 learned_addr learned my addr 172.21.15.29:0/345053498 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a33fff640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a340cf690 0x7f3a340b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a337fe640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a340d2a90 0x7f3a340b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a38bae640 1 -- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a340d2a90 msgr2=0x7f3a340b94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a38bae640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a340d2a90 0x7f3a340b94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a38bae640 1 -- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a340cf690 msgr2=0x7f3a340b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a38bae640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a340cf690 0x7f3a340b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a38bae640 1 -- 172.21.15.29:0/345053498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a34155960 con 0x7f3a3415e200 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a38bae640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a3415e200 0x7f3a340b9a00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00c340 tx=0x7f3a2c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a30ff9640 1 -- 172.21.15.29:0/345053498 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c005220 con 0x7f3a3415e200 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a30ff9640 1 -- 172.21.15.29:0/345053498 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a2c0053c0 con 0x7f3a3415e200 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a30ff9640 1 -- 172.21.15.29:0/345053498 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c007270 con 0x7f3a3415e200 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.559+0000 7f3a3a611640 1 -- 172.21.15.29:0/345053498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a34155c40 con 0x7f3a3415e200 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.563+0000 7f3a3a611640 1 -- 172.21.15.29:0/345053498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a341564e0 con 0x7f3a3415e200 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.563+0000 7f3a3a611640 1 -- 172.21.15.29:0/345053498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a34069270 con 0x7f3a3415e200 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.563+0000 7f3a30ff9640 1 -- 172.21.15.29:0/345053498 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3a2c019020 con 0x7f3a3415e200 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.563+0000 7f3a30ff9640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a200678f0 0x7f3a20069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.563+0000 7f3a30ff9640 1 -- 172.21.15.29:0/345053498 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f3a2c084a70 con 0x7f3a3415e200 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.563+0000 7f3a33fff640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a200678f0 0x7f3a20069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.563+0000 7f3a33fff640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a200678f0 0x7f3a20069db0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a240ae000 tx=0x7f3a240a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.567+0000 7f3a30ff9640 1 -- 172.21.15.29:0/345053498 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3a2c0551e0 con 0x7f3a3415e200 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:27.743+0000 7f3a3a611640 1 -- 172.21.15.29:0/345053498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3a34168c00 con 0x7f3a3415e200 2024-04-25T00:32:28.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.363+0000 7f3a30ff9640 1 -- 172.21.15.29:0/345053498 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f3a2c0594e0 con 0x7f3a3415e200 2024-04-25T00:32:28.369 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-04-25T00:32:28.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.367+0000 7f3a3a611640 1 -- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a200678f0 msgr2=0x7f3a20069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.367+0000 7f3a3a611640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a200678f0 0x7f3a20069db0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a240ae000 tx=0x7f3a240a7070 comp rx=0 tx=0).stop 2024-04-25T00:32:28.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.367+0000 7f3a3a611640 1 -- 172.21.15.29:0/345053498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a3415e200 msgr2=0x7f3a340b9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.367+0000 7f3a3a611640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a3415e200 0x7f3a340b9a00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00c340 tx=0x7f3a2c00c800 comp rx=0 tx=0).stop 2024-04-25T00:32:28.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.367+0000 7f3a3a611640 1 -- 172.21.15.29:0/345053498 shutdown_connections 2024-04-25T00:32:28.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.367+0000 7f3a3a611640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a3415e200 0x7f3a340b9a00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.367+0000 7f3a3a611640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a340d2a90 0x7f3a340b94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.367+0000 7f3a3a611640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a200678f0 0x7f3a20069db0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.367+0000 7f3a3a611640 1 --2- 172.21.15.29:0/345053498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a340cf690 0x7f3a340b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.367+0000 7f3a3a611640 1 -- 172.21.15.29:0/345053498 >> 172.21.15.29:0/345053498 conn(0x7f3a3409d280 msgr2=0x7f3a3406cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:28.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.367+0000 7f3a3a611640 1 -- 172.21.15.29:0/345053498 shutdown_connections 2024-04-25T00:32:28.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.367+0000 7f3a3a611640 1 -- 172.21.15.29:0/345053498 wait complete. 2024-04-25T00:32:28.399 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:32:28.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d19afa640 1 Processor -- start 2024-04-25T00:32:28.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d19afa640 1 -- start start 2024-04-25T00:32:28.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d19afa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3d1415e8d0 0x7f3d140cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d19afa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d140b4780 0x7f3d1415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d19afa640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d140b3720 con 0x7f3d1415f1f0 2024-04-25T00:32:28.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d19afa640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3d140b42f0 con 0x7f3d1415e8d0 2024-04-25T00:32:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d19afa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1415d7e0 con 0x7f3d140b4780 2024-04-25T00:32:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d13fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3d1415f1f0 0x7f3d1415b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37170/0 (socket says 172.21.15.29:37170) 2024-04-25T00:32:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d13fff640 1 -- 172.21.15.29:0/253774582 learned_addr learned my addr 172.21.15.29:0/253774582 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d12ffd640 1 --2- 172.21.15.29:0/253774582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d140b4780 0x7f3d1415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d137fe640 1 --2- 172.21.15.29:0/253774582 >> v2:172.21.15.157:3300/0 conn(0x7f3d1415e8d0 0x7f3d140cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d127fc640 1 -- 172.21.15.29:0/253774582 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 670682741 0 0) 0x7f3d140b3720 con 0x7f3d1415f1f0 2024-04-25T00:32:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d127fc640 1 -- 172.21.15.29:0/253774582 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cfc003530 con 0x7f3d1415f1f0 2024-04-25T00:32:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d12ffd640 1 -- 172.21.15.29:0/253774582 >> v2:172.21.15.157:3300/0 conn(0x7f3d1415e8d0 msgr2=0x7f3d140cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d12ffd640 1 --2- 172.21.15.29:0/253774582 >> v2:172.21.15.157:3300/0 conn(0x7f3d1415e8d0 0x7f3d140cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d12ffd640 1 -- 172.21.15.29:0/253774582 >> v1:172.21.15.29:6789/0 conn(0x7f3d1415f1f0 legacy=0x7f3d1415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d127fc640 1 -- 172.21.15.29:0/253774582 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2671010500 0 0) 0x7f3cfc003530 con 0x7f3d1415f1f0 2024-04-25T00:32:28.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.519+0000 7f3d12ffd640 1 -- 172.21.15.29:0/253774582 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d140b39c0 con 0x7f3d140b4780 2024-04-25T00:32:28.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d12ffd640 1 --2- 172.21.15.29:0/253774582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d140b4780 0x7f3d1415ecb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d040097c0 tx=0x7f3d0401c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=384cd1ae9581a4fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d127fc640 1 -- 172.21.15.29:0/253774582 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d040b0070 con 0x7f3d140b4780 2024-04-25T00:32:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d127fc640 1 -- 172.21.15.29:0/253774582 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d040ab400 con 0x7f3d140b4780 2024-04-25T00:32:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d127fc640 1 -- 172.21.15.29:0/253774582 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1415c1c0 con 0x7f3d140b4780 2024-04-25T00:32:28.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 -- 172.21.15.29:0/253774582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d140b4780 msgr2=0x7f3d1415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 --2- 172.21.15.29:0/253774582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d140b4780 0x7f3d1415ecb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d040097c0 tx=0x7f3d0401c390 comp rx=0 tx=0).stop 2024-04-25T00:32:28.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 -- 172.21.15.29:0/253774582 shutdown_connections 2024-04-25T00:32:28.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 --2- 172.21.15.29:0/253774582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d140b4780 0x7f3d1415ecb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 --2- 172.21.15.29:0/253774582 >> v2:172.21.15.157:3300/0 conn(0x7f3d1415e8d0 0x7f3d140cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 -- 172.21.15.29:0/253774582 >> 172.21.15.29:0/253774582 conn(0x7f3d140b1870 msgr2=0x7f3d140b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:28.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 -- 172.21.15.29:0/253774582 shutdown_connections 2024-04-25T00:32:28.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 -- 172.21.15.29:0/253774582 wait complete. 2024-04-25T00:32:28.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 Processor -- start 2024-04-25T00:32:28.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 -- start start 2024-04-25T00:32:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d140b4780 0x7f3d14152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d1415e8d0 0x7f3d14155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d1415f1f0 0x7f3d140ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14144a20 con 0x7f3d1415f1f0 2024-04-25T00:32:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1415f660 con 0x7f3d140b4780 2024-04-25T00:32:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.523+0000 7f3d19afa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3d14071010 con 0x7f3d1415e8d0 2024-04-25T00:32:28.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d13fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d1415f1f0 0x7f3d140ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d13fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d1415f1f0 0x7f3d140ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37348/0 (socket says 172.21.15.29:37348) 2024-04-25T00:32:28.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d12ffd640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d1415e8d0 0x7f3d14155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d13fff640 1 -- 172.21.15.29:0/2616806260 learned_addr learned my addr 172.21.15.29:0/2616806260 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:28.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d137fe640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d140b4780 0x7f3d14152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d13fff640 1 -- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d1415e8d0 msgr2=0x7f3d14155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d13fff640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d1415e8d0 0x7f3d14155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d13fff640 1 -- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d140b4780 msgr2=0x7f3d14152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d13fff640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d140b4780 0x7f3d14152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d13fff640 1 -- 172.21.15.29:0/2616806260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d141564a0 con 0x7f3d1415f1f0 2024-04-25T00:32:28.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d13fff640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d1415f1f0 0x7f3d140ce3a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00caf0 tx=0x7f3d0c007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3cfbfff640 1 -- 172.21.15.29:0/2616806260 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c005cf0 con 0x7f3d1415f1f0 2024-04-25T00:32:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3cfbfff640 1 -- 172.21.15.29:0/2616806260 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d0c007070 con 0x7f3d1415f1f0 2024-04-25T00:32:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d19afa640 1 -- 172.21.15.29:0/2616806260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d141591c0 con 0x7f3d1415f1f0 2024-04-25T00:32:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3cfbfff640 1 -- 172.21.15.29:0/2616806260 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c007440 con 0x7f3d1415f1f0 2024-04-25T00:32:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d19afa640 1 -- 172.21.15.29:0/2616806260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d14159780 con 0x7f3d1415f1f0 2024-04-25T00:32:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3d19afa640 1 -- 172.21.15.29:0/2616806260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d14069270 con 0x7f3d1415f1f0 2024-04-25T00:32:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3cfbfff640 1 -- 172.21.15.29:0/2616806260 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3d0c0075e0 con 0x7f3d1415f1f0 2024-04-25T00:32:28.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.527+0000 7f3cfbfff640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3d000678c0 0x7f3d00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.531+0000 7f3cfbfff640 1 -- 172.21.15.29:0/2616806260 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f3d0c086240 con 0x7f3d1415f1f0 2024-04-25T00:32:28.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.531+0000 7f3d137fe640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3d000678c0 0x7f3d00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.531+0000 7f3d137fe640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3d000678c0 0x7f3d00069d80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d08011340 tx=0x7f3d0800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:28.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.531+0000 7f3cfbfff640 1 -- 172.21.15.29:0/2616806260 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3d0c056a60 con 0x7f3d1415f1f0 2024-04-25T00:32:28.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3d19afa640 1 -- 172.21.15.29:0/2616806260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3d1406e7b0 con 0x7f3d1415f1f0 2024-04-25T00:32:28.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3cfbfff640 1 -- 172.21.15.29:0/2616806260 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) ==== 80+0+6645 (secure 0 0 0) 0x7f3d0c05ad60 con 0x7f3d1415f1f0 2024-04-25T00:32:28.759 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:28.759 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:32:28.759 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-04-25T00:32:28.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.759 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-04-25T00:32:28.759 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmyzgKAxAAU+nrUJgQhDX6xlARNFI/ng==", 2024-04-25T00:32:28.759 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.759 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:32:28.759 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:32:28.759 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:32:28.759 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.759 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.760 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-04-25T00:32:28.760 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmtyzPERAAxNUDANw1H/ZzOo+dhgzxlg==", 2024-04-25T00:32:28.760 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.760 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:32:28.760 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:32:28.760 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:32:28.760 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.760 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.760 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-04-25T00:32:28.761 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmYUl8CxAAj6pV1+s4a0rpWLq2EgZ+8w==", 2024-04-25T00:32:28.761 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.761 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:32:28.761 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:32:28.761 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:32:28.761 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.761 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.761 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-04-25T00:32:28.761 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmuAhKGhAAOLIcWUodW5TqtVgHWZl12A==", 2024-04-25T00:32:28.762 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.762 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:32:28.762 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:32:28.762 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:32:28.762 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.762 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.762 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-04-25T00:32:28.762 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAzpClmudk6MhAA6YFyd57jwtB2pdOaad7bxQ==", 2024-04-25T00:32:28.762 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.762 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:32:28.762 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA2pClm76q0NRAAxWhEP2ZGoFnZcuiQyGjglw==", 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.763 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA6pClmcgULABAA2F5WBR9Df0v4Sn875AzF7g==", 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA9pClm9QuRBhAALFLruv4JU/uNZYMX0yNuXw==", 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:32:28.764 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBBpClmNj4VDBAAxaxMvyaCvjynPY3f81udnw==", 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-04-25T00:32:28.765 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBEpClmzBybEBAArry1lM4RIsM/txKfgBqDpw==", 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBHpClm5HfjFhAAbFwJCpbhegLWDu0mbS8FoA==", 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.766 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBKpClme/7oHBAAOR2iQ6CweexR7sC1PgesZw==", 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A==", 2024-04-25T00:32:28.767 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.768 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:32:28.768 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:32:28.768 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:32:28.768 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:32:28.768 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.768 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-04-25T00:32:28.768 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm8W/EJxAA0HZW24Hhcwf+soSZn6jHJg==", 2024-04-25T00:32:28.768 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.768 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:32:28.768 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:32:28.768 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmGuQJJRAAhVd4D0jc6Whgyf+B6xgFMw==", 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.769 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmn+ciLRAAg4tQuZ13WP0RnWJaJq5fUQ==", 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmPVihJBAA7PQge/teyRZKSkBnufZSZw==", 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:32:28.770 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.bootstrap-mds", 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQpClmg9FKJxAAmo3BuHMtWLeTR88/Jd/EPw==", 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T00:32:28.771 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQpClmgexKJxAA3S0DPjECisrPG3kBUY+XIg==", 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.bootstrap-osd", 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQpClm8QNLJxAAWtoXQw3yLCqd5pR47ljLCw==", 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQpClmfxpLJxAAenXqsKIYrCUZeUt4+z0D8A==", 2024-04-25T00:32:28.772 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQpClmLTFLJxAAIEBkPcd9jMw7shiKrsAyhg==", 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQpClmsEhLJxAA0g3vh12QXYp/WvanoZwsBA==", 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.773 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClm6WrdOhAAsY9FLCxG7PGpq2NCY4hGhQ==", 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmeR4UOBAANs9SRolOm6E8+/4ciupk5w==", 2024-04-25T00:32:28.774 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:32:28.775 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:32:28.775 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:32:28.775 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:32:28.775 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.775 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:28.775 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:32:28.775 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:32:28.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3d19afa640 1 -- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3d000678c0 msgr2=0x7f3d00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3d19afa640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3d000678c0 0x7f3d00069d80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d08011340 tx=0x7f3d0800b040 comp rx=0 tx=0).stop 2024-04-25T00:32:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3d19afa640 1 -- 172.21.15.29:0/2616806260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d1415f1f0 msgr2=0x7f3d140ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3d19afa640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d1415f1f0 0x7f3d140ce3a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00caf0 tx=0x7f3d0c007e60 comp rx=0 tx=0).stop 2024-04-25T00:32:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3d19afa640 1 -- 172.21.15.29:0/2616806260 shutdown_connections 2024-04-25T00:32:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3d19afa640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d1415f1f0 0x7f3d140ce3a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3d19afa640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d1415e8d0 0x7f3d14155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3d19afa640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3d000678c0 0x7f3d00069d80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3d19afa640 1 --2- 172.21.15.29:0/2616806260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d140b4780 0x7f3d14152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3d19afa640 1 -- 172.21.15.29:0/2616806260 >> 172.21.15.29:0/2616806260 conn(0x7f3d140b1870 msgr2=0x7f3d140b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3d19afa640 1 -- 172.21.15.29:0/2616806260 shutdown_connections 2024-04-25T00:32:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.755+0000 7f3d19afa640 1 -- 172.21.15.29:0/2616806260 wait complete. 2024-04-25T00:32:28.777 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T00:32:28.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f37671e5640 1 Processor -- start 2024-04-25T00:32:28.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f37671e5640 1 -- start start 2024-04-25T00:32:28.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f37671e5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37600b3140 0x7f376006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f37671e5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f37600b4b10 0x7f37600b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f37671e5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37600b41e0 con 0x7f37600b3a60 2024-04-25T00:32:28.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f37671e5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f376006d960 con 0x7f37600b4b10 2024-04-25T00:32:28.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f37671e5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f376006dc40 con 0x7f37600b3140 2024-04-25T00:32:28.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f375ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f37600b4b10 0x7f37600b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f375ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f37600b4b10 0x7f37600b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37350/0 (socket says 172.21.15.29:37350) 2024-04-25T00:32:28.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f375ffff640 1 -- 172.21.15.29:0/1469184322 learned_addr learned my addr 172.21.15.29:0/1469184322 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:28.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f3764f81640 1 --2- 172.21.15.29:0/1469184322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37600b3140 0x7f376006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f375ffff640 1 -- 172.21.15.29:0/1469184322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37600b3140 msgr2=0x7f376006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:28.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f375ffff640 1 --2- 172.21.15.29:0/1469184322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37600b3140 0x7f376006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f375ffff640 1 -- 172.21.15.29:0/1469184322 >> v1:172.21.15.157:6789/0 conn(0x7f37600b3a60 legacy=0x7f376015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f375ffff640 1 -- 172.21.15.29:0/1469184322 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376006df20 con 0x7f37600b4b10 2024-04-25T00:32:28.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.899+0000 7f3764f81640 1 --2- 172.21.15.29:0/1469184322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37600b3140 0x7f376006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:28.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.903+0000 7f375ffff640 1 --2- 172.21.15.29:0/1469184322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37600b4b10 0x7f37600b3520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3754009870 tx=0x7f375401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b227cdf4dde8abc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:28.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.903+0000 7f375f7fe640 1 -- 172.21.15.29:0/1469184322 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37540b0070 con 0x7f37600b4b10 2024-04-25T00:32:28.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.903+0000 7f375f7fe640 1 -- 172.21.15.29:0/1469184322 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37540ab600 con 0x7f37600b4b10 2024-04-25T00:32:28.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.903+0000 7f375f7fe640 1 -- 172.21.15.29:0/1469184322 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37540ab9d0 con 0x7f37600b4b10 2024-04-25T00:32:28.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.903+0000 7f37671e5640 1 -- 172.21.15.29:0/1469184322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37600b4b10 msgr2=0x7f37600b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.903+0000 7f37671e5640 1 --2- 172.21.15.29:0/1469184322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37600b4b10 0x7f37600b3520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3754009870 tx=0x7f375401c560 comp rx=0 tx=0).stop 2024-04-25T00:32:28.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.903+0000 7f37671e5640 1 -- 172.21.15.29:0/1469184322 shutdown_connections 2024-04-25T00:32:28.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.903+0000 7f37671e5640 1 --2- 172.21.15.29:0/1469184322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37600b4b10 0x7f37600b3520 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.903+0000 7f37671e5640 1 --2- 172.21.15.29:0/1469184322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37600b3140 0x7f376006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.903+0000 7f37671e5640 1 -- 172.21.15.29:0/1469184322 >> 172.21.15.29:0/1469184322 conn(0x7f37600b1870 msgr2=0x7f37600b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:28.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.903+0000 7f37671e5640 1 -- 172.21.15.29:0/1469184322 shutdown_connections 2024-04-25T00:32:28.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.903+0000 7f37671e5640 1 -- 172.21.15.29:0/1469184322 wait complete. 2024-04-25T00:32:28.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f37671e5640 1 Processor -- start 2024-04-25T00:32:28.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f37671e5640 1 -- start start 2024-04-25T00:32:28.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f37671e5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37600b3140 0x7f376015af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f37671e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37600b3a60 0x7f376013a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f37671e5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37600b4b10 0x7f3760140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f37671e5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3760168b70 con 0x7f37600b3140 2024-04-25T00:32:28.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f37671e5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f37601689f0 con 0x7f37600b3a60 2024-04-25T00:32:28.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f37671e5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3760168cf0 con 0x7f37600b4b10 2024-04-25T00:32:28.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f3764f81640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37600b3140 0x7f376015af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f3764f81640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37600b3140 0x7f376015af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37362/0 (socket says 172.21.15.29:37362) 2024-04-25T00:32:28.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f3764f81640 1 -- 172.21.15.29:0/3076588635 learned_addr learned my addr 172.21.15.29:0/3076588635 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:28.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f375ffff640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37600b3a60 0x7f376013a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f3765782640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37600b4b10 0x7f3760140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f3764f81640 1 -- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37600b4b10 msgr2=0x7f3760140330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f3764f81640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37600b4b10 0x7f3760140330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f3764f81640 1 -- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37600b3a60 msgr2=0x7f376013a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f3764f81640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37600b3a60 0x7f376013a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f3764f81640 1 -- 172.21.15.29:0/3076588635 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37601398d0 con 0x7f37600b3140 2024-04-25T00:32:28.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f3764f81640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37600b3140 0x7f376015af50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f375000d800 tx=0x7f375000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:28.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f375d7fa640 1 -- 172.21.15.29:0/3076588635 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750018020 con 0x7f37600b3140 2024-04-25T00:32:28.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f375d7fa640 1 -- 172.21.15.29:0/3076588635 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37500096c0 con 0x7f37600b3140 2024-04-25T00:32:28.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f37671e5640 1 -- 172.21.15.29:0/3076588635 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3760139bb0 con 0x7f37600b3140 2024-04-25T00:32:28.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f375d7fa640 1 -- 172.21.15.29:0/3076588635 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750009ab0 con 0x7f37600b3140 2024-04-25T00:32:28.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f37671e5640 1 -- 172.21.15.29:0/3076588635 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376013a170 con 0x7f37600b3140 2024-04-25T00:32:28.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.907+0000 7f37671e5640 1 -- 172.21.15.29:0/3076588635 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3760069270 con 0x7f37600b3140 2024-04-25T00:32:28.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.911+0000 7f375d7fa640 1 -- 172.21.15.29:0/3076588635 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f375001b020 con 0x7f37600b3140 2024-04-25T00:32:28.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.911+0000 7f375d7fa640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f374c0678c0 0x7f374c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:28.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.911+0000 7f375d7fa640 1 -- 172.21.15.29:0/3076588635 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f3750085e80 con 0x7f37600b3140 2024-04-25T00:32:28.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.911+0000 7f375ffff640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f374c0678c0 0x7f374c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:28.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.911+0000 7f375ffff640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f374c0678c0 0x7f374c069d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f37540ad000 tx=0x7f37540a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:28.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:28.915+0000 7f375d7fa640 1 -- 172.21.15.29:0/3076588635 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3750022020 con 0x7f37600b3140 2024-04-25T00:32:29.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.127+0000 7f37671e5640 1 -- 172.21.15.29:0/3076588635 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f376015c510 con 0x7f37600b3140 2024-04-25T00:32:29.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.131+0000 7f375d7fa640 1 -- 172.21.15.29:0/3076588635 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f3750056d00 con 0x7f37600b3140 2024-04-25T00:32:29.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.135+0000 7f37671e5640 1 -- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f374c0678c0 msgr2=0x7f374c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.135+0000 7f37671e5640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f374c0678c0 0x7f374c069d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f37540ad000 tx=0x7f37540a6000 comp rx=0 tx=0).stop 2024-04-25T00:32:29.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.135+0000 7f37671e5640 1 -- 172.21.15.29:0/3076588635 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37600b3140 msgr2=0x7f376015af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.135+0000 7f37671e5640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37600b3140 0x7f376015af50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f375000d800 tx=0x7f375000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.135+0000 7f37671e5640 1 -- 172.21.15.29:0/3076588635 shutdown_connections 2024-04-25T00:32:29.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.135+0000 7f37671e5640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37600b4b10 0x7f3760140330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.135+0000 7f37671e5640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f374c0678c0 0x7f374c069d80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.135+0000 7f37671e5640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37600b3a60 0x7f376013a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.135+0000 7f37671e5640 1 --2- 172.21.15.29:0/3076588635 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37600b3140 0x7f376015af50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.135+0000 7f37671e5640 1 -- 172.21.15.29:0/3076588635 >> 172.21.15.29:0/3076588635 conn(0x7f37600b1870 msgr2=0x7f37601428f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:29.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.135+0000 7f37671e5640 1 -- 172.21.15.29:0/3076588635 shutdown_connections 2024-04-25T00:32:29.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.135+0000 7f37671e5640 1 -- 172.21.15.29:0/3076588635 wait complete. 2024-04-25T00:32:29.153 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T00:32:29.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9b00b7640 1 Processor -- start 2024-04-25T00:32:29.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9b00b7640 1 -- start start 2024-04-25T00:32:29.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9b00b7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a815df70 0x7fd9a80b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9b00b7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9a806d210 0x7fd9a80b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9b00b7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a806d980 con 0x7fd9a80b4b60 2024-04-25T00:32:29.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9b00b7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a80b4fa0 con 0x7fd9a806d210 2024-04-25T00:32:29.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9b00b7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a806dc60 con 0x7fd9a815df70 2024-04-25T00:32:29.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ae654640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd9a80b4b60 0x7fd9a8070ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37182/0 (socket says 172.21.15.29:37182) 2024-04-25T00:32:29.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ae654640 1 -- 172.21.15.29:0/3346395344 learned_addr learned my addr 172.21.15.29:0/3346395344 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:29.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ade53640 1 --2- 172.21.15.29:0/3346395344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a815df70 0x7fd9a80b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ad652640 1 --2- 172.21.15.29:0/3346395344 >> v2:172.21.15.157:3300/0 conn(0x7fd9a806d210 0x7fd9a80b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3411353128 0 0) 0x7fd9a806d980 con 0x7fd9a80b4b60 2024-04-25T00:32:29.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd994003530 con 0x7fd9a80b4b60 2024-04-25T00:32:29.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 778495924 0 0) 0x7fd994003530 con 0x7fd9a80b4b60 2024-04-25T00:32:29.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a815df70 msgr2=0x7fd9a80b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:29.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 --2- 172.21.15.29:0/3346395344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a815df70 0x7fd9a80b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 >> v2:172.21.15.157:3300/0 conn(0x7fd9a806d210 msgr2=0x7fd9a80b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:29.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 --2- 172.21.15.29:0/3346395344 >> v2:172.21.15.157:3300/0 conn(0x7fd9a806d210 0x7fd9a80b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a806df40 con 0x7fd9a80b4b60 2024-04-25T00:32:29.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd9a0001a00 con 0x7fd9a80b4b60 2024-04-25T00:32:29.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 >> v1:172.21.15.29:6789/0 conn(0x7fd9a80b4b60 legacy=0x7fd9a8070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 --2- 172.21.15.29:0/3346395344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9940039c0 0x7fd994003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 --2- 172.21.15.29:0/3346395344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd994004810 0x7fd994004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 --2- 172.21.15.29:0/3346395344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a815df70 0x7fd9940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a80b4fa0 con 0x7fd994004810 2024-04-25T00:32:29.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a806dc60 con 0x7fd9940039c0 2024-04-25T00:32:29.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd99400b9d0 con 0x7fd9a815df70 2024-04-25T00:32:29.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.267+0000 7fd9ad652640 1 --2- 172.21.15.29:0/3346395344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd994004810 0x7fd994004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9ade53640 1 --2- 172.21.15.29:0/3346395344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9940039c0 0x7fd994003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9ae654640 1 --2- 172.21.15.29:0/3346395344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a815df70 0x7fd9940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9ad652640 1 -- 172.21.15.29:0/3346395344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a815df70 msgr2=0x7fd9940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:29.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9ad652640 1 --2- 172.21.15.29:0/3346395344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a815df70 0x7fd9940051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9ad652640 1 -- 172.21.15.29:0/3346395344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9940039c0 msgr2=0x7fd994003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:29.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9ad652640 1 --2- 172.21.15.29:0/3346395344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9940039c0 0x7fd994003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9ad652640 1 -- 172.21.15.29:0/3346395344 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd994003530 con 0x7fd994004810 2024-04-25T00:32:29.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9ade53640 1 --2- 172.21.15.29:0/3346395344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9940039c0 0x7fd994003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:29.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9ad652640 1 --2- 172.21.15.29:0/3346395344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd994004810 0x7fd994004c90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd998009ae0 tx=0x7fd99801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed362cdb4aa6b3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:29.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980b2070 con 0x7fd994004810 2024-04-25T00:32:29.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9980a55c0 con 0x7fd994004810 2024-04-25T00:32:29.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9ace51640 1 -- 172.21.15.29:0/3346395344 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980096b0 con 0x7fd994004810 2024-04-25T00:32:29.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9b00b7640 1 -- 172.21.15.29:0/3346395344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd994004810 msgr2=0x7fd994004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9b00b7640 1 --2- 172.21.15.29:0/3346395344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd994004810 0x7fd994004c90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd998009ae0 tx=0x7fd99801cab0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9ae654640 1 -- 172.21.15.29:0/3346395344 reap_dead start 2024-04-25T00:32:29.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9b00b7640 1 -- 172.21.15.29:0/3346395344 shutdown_connections 2024-04-25T00:32:29.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9b00b7640 1 -- 172.21.15.29:0/3346395344 >> 172.21.15.29:0/3346395344 conn(0x7fd9a80b1870 msgr2=0x7fd9a80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:29.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9b00b7640 1 -- 172.21.15.29:0/3346395344 shutdown_connections 2024-04-25T00:32:29.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.271+0000 7fd9b00b7640 1 -- 172.21.15.29:0/3346395344 wait complete. 2024-04-25T00:32:29.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9b00b7640 1 Processor -- start 2024-04-25T00:32:29.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9b00b7640 1 -- start start 2024-04-25T00:32:29.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9b00b7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd994004810 0x7fd9a80ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9b00b7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a80af2f0 0x7fd9a813f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9b00b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a815c890 0x7fd9a815cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9b00b7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a806d670 con 0x7fd9a80af2f0 2024-04-25T00:32:29.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9b00b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a806e500 con 0x7fd9a815c890 2024-04-25T00:32:29.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9b00b7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a815e350 con 0x7fd994004810 2024-04-25T00:32:29.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9ad652640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a80af2f0 0x7fd9a813f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9ad652640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a80af2f0 0x7fd9a813f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37380/0 (socket says 172.21.15.29:37380) 2024-04-25T00:32:29.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9ad652640 1 -- 172.21.15.29:0/2972446611 learned_addr learned my addr 172.21.15.29:0/2972446611 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:29.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9ade53640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd994004810 0x7fd9a80ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9ad652640 1 -- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd994004810 msgr2=0x7fd9a80ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9ad652640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd994004810 0x7fd9a80ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9ad652640 1 -- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a815c890 msgr2=0x7fd9a815cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9ad652640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a815c890 0x7fd9a815cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9ad652640 1 -- 172.21.15.29:0/2972446611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a80ae7f0 con 0x7fd9a80af2f0 2024-04-25T00:32:29.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9ade53640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd994004810 0x7fd9a80ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:29.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9ad652640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a80af2f0 0x7fd9a813f480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd998009ae0 tx=0x7fd994005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:29.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd99e7fc640 1 -- 172.21.15.29:0/2972446611 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980b2040 con 0x7fd9a80af2f0 2024-04-25T00:32:29.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd99e7fc640 1 -- 172.21.15.29:0/2972446611 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9980a5df0 con 0x7fd9a80af2f0 2024-04-25T00:32:29.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9b00b7640 1 -- 172.21.15.29:0/2972446611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a80af850 con 0x7fd9a80af2f0 2024-04-25T00:32:29.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd99e7fc640 1 -- 172.21.15.29:0/2972446611 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998004020 con 0x7fd9a80af2f0 2024-04-25T00:32:29.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9b00b7640 1 -- 172.21.15.29:0/2972446611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a80afe10 con 0x7fd9a80af2f0 2024-04-25T00:32:29.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd9b00b7640 1 -- 172.21.15.29:0/2972446611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a8069270 con 0x7fd9a80af2f0 2024-04-25T00:32:29.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.275+0000 7fd99e7fc640 1 -- 172.21.15.29:0/2972446611 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd9980c0030 con 0x7fd9a80af2f0 2024-04-25T00:32:29.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.279+0000 7fd99e7fc640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd98c0678f0 0x7fd98c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.279+0000 7fd99e7fc640 1 -- 172.21.15.29:0/2972446611 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fd99811f520 con 0x7fd9a80af2f0 2024-04-25T00:32:29.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.279+0000 7fd9ade53640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd98c0678f0 0x7fd98c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.279+0000 7fd9ade53640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd98c0678f0 0x7fd98c069db0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40099e0 tx=0x7fd9a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:29.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.283+0000 7fd99e7fc640 1 -- 172.21.15.29:0/2972446611 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd9980c7060 con 0x7fd9a80af2f0 2024-04-25T00:32:29.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.499+0000 7fd9b00b7640 1 -- 172.21.15.29:0/2972446611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fd9a815b1c0 con 0x7fd9a80af2f0 2024-04-25T00:32:29.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.503+0000 7fd99e7fc640 1 -- 172.21.15.29:0/2972446611 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fd9980167c0 con 0x7fd9a80af2f0 2024-04-25T00:32:29.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.503+0000 7fd9b00b7640 1 -- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd98c0678f0 msgr2=0x7fd98c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.503+0000 7fd9b00b7640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd98c0678f0 0x7fd98c069db0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40099e0 tx=0x7fd9a4008040 comp rx=0 tx=0).stop 2024-04-25T00:32:29.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.503+0000 7fd9b00b7640 1 -- 172.21.15.29:0/2972446611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a80af2f0 msgr2=0x7fd9a813f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.503+0000 7fd9b00b7640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a80af2f0 0x7fd9a813f480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd998009ae0 tx=0x7fd994005840 comp rx=0 tx=0).stop 2024-04-25T00:32:29.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.503+0000 7fd9b00b7640 1 -- 172.21.15.29:0/2972446611 shutdown_connections 2024-04-25T00:32:29.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.503+0000 7fd9b00b7640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd98c0678f0 0x7fd98c069db0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.503+0000 7fd9b00b7640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a815c890 0x7fd9a815cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.503+0000 7fd9b00b7640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a80af2f0 0x7fd9a813f480 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.503+0000 7fd9b00b7640 1 --2- 172.21.15.29:0/2972446611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd994004810 0x7fd9a80ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.503+0000 7fd9b00b7640 1 -- 172.21.15.29:0/2972446611 >> 172.21.15.29:0/2972446611 conn(0x7fd9a80b1870 msgr2=0x7fd9a80cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:29.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.507+0000 7fd9b00b7640 1 -- 172.21.15.29:0/2972446611 shutdown_connections 2024-04-25T00:32:29.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.507+0000 7fd9b00b7640 1 -- 172.21.15.29:0/2972446611 wait complete. 2024-04-25T00:32:29.525 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T00:32:29.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd529b2640 1 Processor -- start 2024-04-25T00:32:29.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd529b2640 1 -- start start 2024-04-25T00:32:29.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd529b2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c06c950 0x7fbd4c06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd529b2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd4c0cc3b0 con 0x7fbd4c15e440 2024-04-25T00:32:29.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd529b2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd4c06d270 con 0x7fbd4c0cd410 2024-04-25T00:32:29.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd529b2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4c0b11d0 con 0x7fbd4c06c950 2024-04-25T00:32:29.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4b7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbd4c15e440 0x7fbd4c0d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37198/0 (socket says 172.21.15.29:37198) 2024-04-25T00:32:29.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4b7fe640 1 -- 172.21.15.29:0/1334207047 learned_addr learned my addr 172.21.15.29:0/1334207047 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:29.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4bfff640 1 --2- 172.21.15.29:0/1334207047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c06c950 0x7fbd4c06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2076545381 0 0) 0x7fbd4c0cc3b0 con 0x7fbd4c15e440 2024-04-25T00:32:29.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd34003530 con 0x7fbd4c15e440 2024-04-25T00:32:29.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 630660967 0 0) 0x7fbd34003530 con 0x7fbd4c15e440 2024-04-25T00:32:29.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c06c950 msgr2=0x7fbd4c06cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:29.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 --2- 172.21.15.29:0/1334207047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c06c950 0x7fbd4c06cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 >> v1:172.21.15.157:6789/0 conn(0x7fbd4c0cd410 legacy=0x7fbd4c0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c0cc650 con 0x7fbd4c15e440 2024-04-25T00:32:29.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fbd40001680 con 0x7fbd4c15e440 2024-04-25T00:32:29.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 >> v1:172.21.15.29:6789/0 conn(0x7fbd4c15e440 legacy=0x7fbd4c0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 --2- 172.21.15.29:0/1334207047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c06c950 0x7fbd340039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 --2- 172.21.15.29:0/1334207047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd34003f30 0x7fbd34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 --2- 172.21.15.29:0/1334207047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd34004d50 0x7fbd340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c0b11d0 con 0x7fbd34003f30 2024-04-25T00:32:29.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3400ba30 con 0x7fbd34004d50 2024-04-25T00:32:29.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3400bc30 con 0x7fbd4c06c950 2024-04-25T00:32:29.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4b7fe640 1 --2- 172.21.15.29:0/1334207047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd34003f30 0x7fbd34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4bfff640 1 --2- 172.21.15.29:0/1334207047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c06c950 0x7fbd340039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd50f4f640 1 --2- 172.21.15.29:0/1334207047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd34004d50 0x7fbd340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4b7fe640 1 -- 172.21.15.29:0/1334207047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c06c950 msgr2=0x7fbd340039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:29.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4b7fe640 1 --2- 172.21.15.29:0/1334207047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c06c950 0x7fbd340039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4b7fe640 1 -- 172.21.15.29:0/1334207047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd34004d50 msgr2=0x7fbd340051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:29.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4b7fe640 1 --2- 172.21.15.29:0/1334207047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd34004d50 0x7fbd340051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.647+0000 7fbd4b7fe640 1 -- 172.21.15.29:0/1334207047 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd34003530 con 0x7fbd34003f30 2024-04-25T00:32:29.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.651+0000 7fbd4b7fe640 1 --2- 172.21.15.29:0/1334207047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd34003f30 0x7fbd34004810 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000c160 tx=0x7fbd4001cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f249dca9d569a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:29.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.651+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40007cd0 con 0x7fbd34003f30 2024-04-25T00:32:29.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.651+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd400a9070 con 0x7fbd34003f30 2024-04-25T00:32:29.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.651+0000 7fbd4affd640 1 -- 172.21.15.29:0/1334207047 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40005d60 con 0x7fbd34003f30 2024-04-25T00:32:29.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.651+0000 7fbd529b2640 1 -- 172.21.15.29:0/1334207047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd34003f30 msgr2=0x7fbd34004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.651+0000 7fbd529b2640 1 --2- 172.21.15.29:0/1334207047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd34003f30 0x7fbd34004810 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000c160 tx=0x7fbd4001cb20 comp rx=0 tx=0).stop 2024-04-25T00:32:29.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.651+0000 7fbd50f4f640 1 -- 172.21.15.29:0/1334207047 reap_dead start 2024-04-25T00:32:29.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.651+0000 7fbd529b2640 1 -- 172.21.15.29:0/1334207047 shutdown_connections 2024-04-25T00:32:29.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.651+0000 7fbd529b2640 1 -- 172.21.15.29:0/1334207047 >> 172.21.15.29:0/1334207047 conn(0x7fbd4c0d1d60 msgr2=0x7fbd4c0d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:29.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.651+0000 7fbd529b2640 1 -- 172.21.15.29:0/1334207047 shutdown_connections 2024-04-25T00:32:29.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.651+0000 7fbd529b2640 1 -- 172.21.15.29:0/1334207047 wait complete. 2024-04-25T00:32:29.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.651+0000 7fbd529b2640 1 Processor -- start 2024-04-25T00:32:29.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.651+0000 7fbd529b2640 1 -- start start 2024-04-25T00:32:29.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd529b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd34003f30 0x7fbd4c0af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd529b2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd4c0b0c90 0x7fbd4c1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd529b2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c159de0 0x7fbd4c152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd529b2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c0d11c0 con 0x7fbd34003f30 2024-04-25T00:32:29.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd529b2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c06c520 con 0x7fbd4c0b0c90 2024-04-25T00:32:29.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd529b2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4c0ce3c0 con 0x7fbd4c159de0 2024-04-25T00:32:29.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd34003f30 0x7fbd4c0af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd34003f30 0x7fbd4c0af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37388/0 (socket says 172.21.15.29:37388) 2024-04-25T00:32:29.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd4bfff640 1 -- 172.21.15.29:0/3866436192 learned_addr learned my addr 172.21.15.29:0/3866436192 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:29.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd4b7fe640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd4c0b0c90 0x7fbd4c1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd50f4f640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c159de0 0x7fbd4c152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd4bfff640 1 -- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c159de0 msgr2=0x7fbd4c152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd4bfff640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c159de0 0x7fbd4c152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd4bfff640 1 -- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd4c0b0c90 msgr2=0x7fbd4c1598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd4bfff640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd4c0b0c90 0x7fbd4c1598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd4bfff640 1 -- 172.21.15.29:0/3866436192 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c0c7080 con 0x7fbd34003f30 2024-04-25T00:32:29.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd50f4f640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c159de0 0x7fbd4c152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:29.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd4bfff640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd34003f30 0x7fbd4c0af480 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0119d0 tx=0x7fbd3c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:29.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd48ff9640 1 -- 172.21.15.29:0/3866436192 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c00faf0 con 0x7fbd34003f30 2024-04-25T00:32:29.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd48ff9640 1 -- 172.21.15.29:0/3866436192 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd3c00fc90 con 0x7fbd34003f30 2024-04-25T00:32:29.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd529b2640 1 -- 172.21.15.29:0/3866436192 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c0afbf0 con 0x7fbd34003f30 2024-04-25T00:32:29.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd48ff9640 1 -- 172.21.15.29:0/3866436192 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c014440 con 0x7fbd34003f30 2024-04-25T00:32:29.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd529b2640 1 -- 172.21.15.29:0/3866436192 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c0b01b0 con 0x7fbd34003f30 2024-04-25T00:32:29.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd529b2640 1 -- 172.21.15.29:0/3866436192 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4c069270 con 0x7fbd34003f30 2024-04-25T00:32:29.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd48ff9640 1 -- 172.21.15.29:0/3866436192 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbd3c012040 con 0x7fbd34003f30 2024-04-25T00:32:29.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd48ff9640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbd380678c0 0x7fbd38069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:29.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd48ff9640 1 -- 172.21.15.29:0/3866436192 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fbd3c085ff0 con 0x7fbd34003f30 2024-04-25T00:32:29.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.655+0000 7fbd4b7fe640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbd380678c0 0x7fbd38069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:29.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.659+0000 7fbd4b7fe640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbd380678c0 0x7fbd38069d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbd400ab000 tx=0x7fbd400b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:29.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.659+0000 7fbd48ff9640 1 -- 172.21.15.29:0/3866436192 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbd3c019030 con 0x7fbd34003f30 2024-04-25T00:32:29.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.879+0000 7fbd529b2640 1 -- 172.21.15.29:0/3866436192 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fbd4c070c40 con 0x7fbd34003f30 2024-04-25T00:32:29.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.883+0000 7fbd48ff9640 1 -- 172.21.15.29:0/3866436192 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7fbd3c056f40 con 0x7fbd34003f30 2024-04-25T00:32:29.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.887+0000 7fbd529b2640 1 -- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbd380678c0 msgr2=0x7fbd38069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.887+0000 7fbd529b2640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbd380678c0 0x7fbd38069d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbd400ab000 tx=0x7fbd400b1040 comp rx=0 tx=0).stop 2024-04-25T00:32:29.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.887+0000 7fbd529b2640 1 -- 172.21.15.29:0/3866436192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd34003f30 msgr2=0x7fbd4c0af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:29.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.887+0000 7fbd529b2640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd34003f30 0x7fbd4c0af480 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0119d0 tx=0x7fbd3c011e90 comp rx=0 tx=0).stop 2024-04-25T00:32:29.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.887+0000 7fbd529b2640 1 -- 172.21.15.29:0/3866436192 shutdown_connections 2024-04-25T00:32:29.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.887+0000 7fbd529b2640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c159de0 0x7fbd4c152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.887+0000 7fbd529b2640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbd380678c0 0x7fbd38069d80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.887+0000 7fbd529b2640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd4c0b0c90 0x7fbd4c1598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.887+0000 7fbd529b2640 1 --2- 172.21.15.29:0/3866436192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd34003f30 0x7fbd4c0af480 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:29.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.887+0000 7fbd529b2640 1 -- 172.21.15.29:0/3866436192 >> 172.21.15.29:0/3866436192 conn(0x7fbd4c0d1d60 msgr2=0x7fbd4c0b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:29.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.887+0000 7fbd529b2640 1 -- 172.21.15.29:0/3866436192 shutdown_connections 2024-04-25T00:32:29.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:29.887+0000 7fbd529b2640 1 -- 172.21.15.29:0/3866436192 wait complete. 2024-04-25T00:32:29.908 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T00:32:30.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349edbb640 1 Processor -- start 2024-04-25T00:32:30.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349edbb640 1 -- start start 2024-04-25T00:32:30.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349edbb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f34980b3140 0x7f349806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349edbb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f34980b4b10 0x7f34980b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349edbb640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34980b41e0 con 0x7f34980b3a60 2024-04-25T00:32:30.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349edbb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f349806d960 con 0x7f34980b3140 2024-04-25T00:32:30.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349edbb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f349806dc40 con 0x7f34980b4b10 2024-04-25T00:32:30.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349cb57640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f34980b3140 0x7f349806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349d358640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f34980b3a60 0x7f349815c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37200/0 (socket says 172.21.15.29:37200) 2024-04-25T00:32:30.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349d358640 1 -- 172.21.15.29:0/2253074761 learned_addr learned my addr 172.21.15.29:0/2253074761 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:30.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349cb57640 1 -- 172.21.15.29:0/2253074761 >> v2:172.21.15.157:3300/0 conn(0x7f34980b4b10 msgr2=0x7f34980b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:30.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349cb57640 1 --2- 172.21.15.29:0/2253074761 >> v2:172.21.15.157:3300/0 conn(0x7f34980b4b10 0x7f34980b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349cb57640 1 -- 172.21.15.29:0/2253074761 >> v1:172.21.15.29:6789/0 conn(0x7f34980b3a60 legacy=0x7f349815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.023+0000 7f349cb57640 1 -- 172.21.15.29:0/2253074761 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349806df20 con 0x7f34980b3140 2024-04-25T00:32:30.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f349cb57640 1 --2- 172.21.15.29:0/2253074761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34980b3140 0x7f349806e440 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f34900097c0 tx=0x7f349001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09771160aa8bb48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:30.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f34977fe640 1 -- 172.21.15.29:0/2253074761 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34900b0070 con 0x7f34980b3140 2024-04-25T00:32:30.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f34977fe640 1 -- 172.21.15.29:0/2253074761 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34900ab600 con 0x7f34980b3140 2024-04-25T00:32:30.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f34977fe640 1 -- 172.21.15.29:0/2253074761 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34900ab9d0 con 0x7f34980b3140 2024-04-25T00:32:30.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f349edbb640 1 -- 172.21.15.29:0/2253074761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34980b3140 msgr2=0x7f349806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f349edbb640 1 --2- 172.21.15.29:0/2253074761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34980b3140 0x7f349806e440 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f34900097c0 tx=0x7f349001c460 comp rx=0 tx=0).stop 2024-04-25T00:32:30.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f349edbb640 1 -- 172.21.15.29:0/2253074761 shutdown_connections 2024-04-25T00:32:30.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f349edbb640 1 --2- 172.21.15.29:0/2253074761 >> v2:172.21.15.157:3300/0 conn(0x7f34980b4b10 0x7f34980b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f349edbb640 1 --2- 172.21.15.29:0/2253074761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34980b3140 0x7f349806e440 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f349edbb640 1 -- 172.21.15.29:0/2253074761 >> 172.21.15.29:0/2253074761 conn(0x7f34980b1870 msgr2=0x7f34980b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:30.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f349edbb640 1 -- 172.21.15.29:0/2253074761 shutdown_connections 2024-04-25T00:32:30.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f349edbb640 1 -- 172.21.15.29:0/2253074761 wait complete. 2024-04-25T00:32:30.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f349edbb640 1 Processor -- start 2024-04-25T00:32:30.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.027+0000 7f349edbb640 1 -- start start 2024-04-25T00:32:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349edbb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34980b3140 0x7f349813f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349edbb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34980b3a60 0x7f349813d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349edbb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34980b4b10 0x7f34981402b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349edbb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3498168b70 con 0x7f34980b3140 2024-04-25T00:32:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349edbb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34981689f0 con 0x7f34980b3a60 2024-04-25T00:32:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349edbb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3498168cf0 con 0x7f34980b4b10 2024-04-25T00:32:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349cb57640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34980b3140 0x7f349813f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349cb57640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34980b3140 0x7f349813f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37396/0 (socket says 172.21.15.29:37396) 2024-04-25T00:32:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349cb57640 1 -- 172.21.15.29:0/1079651096 learned_addr learned my addr 172.21.15.29:0/1079651096 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349cb57640 1 -- 172.21.15.29:0/1079651096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34980b4b10 msgr2=0x7f34981402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349cb57640 1 --2- 172.21.15.29:0/1079651096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34980b4b10 0x7f34981402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349cb57640 1 -- 172.21.15.29:0/1079651096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34980b3a60 msgr2=0x7f349813d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349cb57640 1 --2- 172.21.15.29:0/1079651096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34980b3a60 0x7f349813d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349cb57640 1 -- 172.21.15.29:0/1079651096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349813e180 con 0x7f34980b3140 2024-04-25T00:32:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349cb57640 1 --2- 172.21.15.29:0/1079651096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34980b3140 0x7f349813f220 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f34900ae040 tx=0x7f349001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f34957fa640 1 -- 172.21.15.29:0/1079651096 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34900b0050 con 0x7f34980b3140 2024-04-25T00:32:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f34957fa640 1 -- 172.21.15.29:0/1079651096 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34900bc070 con 0x7f34980b3140 2024-04-25T00:32:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349edbb640 1 -- 172.21.15.29:0/1079651096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349813e460 con 0x7f34980b3140 2024-04-25T00:32:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f34957fa640 1 -- 172.21.15.29:0/1079651096 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34900b5830 con 0x7f34980b3140 2024-04-25T00:32:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349edbb640 1 -- 172.21.15.29:0/1079651096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f349813ea20 con 0x7f34980b3140 2024-04-25T00:32:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f349edbb640 1 -- 172.21.15.29:0/1079651096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3498069270 con 0x7f34980b3140 2024-04-25T00:32:30.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f34957fa640 1 -- 172.21.15.29:0/1079651096 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f34900b59d0 con 0x7f34980b3140 2024-04-25T00:32:30.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f34957fa640 1 --2- 172.21.15.29:0/1079651096 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f34780678c0 0x7f3478069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f34957fa640 1 -- 172.21.15.29:0/1079651096 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f3490120cc0 con 0x7f34980b3140 2024-04-25T00:32:30.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.031+0000 7f3497fff640 1 --2- 172.21.15.29:0/1079651096 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f34780678c0 0x7f3478069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.035+0000 7f3497fff640 1 --2- 172.21.15.29:0/1079651096 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f34780678c0 0x7f3478069d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f34980cf2f0 tx=0x7f3480006ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.035+0000 7f34957fa640 1 -- 172.21.15.29:0/1079651096 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3490016050 con 0x7f34980b3140 2024-04-25T00:32:30.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.247+0000 7f349edbb640 1 -- 172.21.15.29:0/1079651096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f3498144750 con 0x7f34980b3140 2024-04-25T00:32:30.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.251+0000 7f34957fa640 1 -- 172.21.15.29:0/1079651096 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f34900f1c30 con 0x7f34980b3140 2024-04-25T00:32:30.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.251+0000 7f349edbb640 1 -- 172.21.15.29:0/1079651096 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f34780678c0 msgr2=0x7f3478069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.251+0000 7f349edbb640 1 --2- 172.21.15.29:0/1079651096 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f34780678c0 0x7f3478069d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f34980cf2f0 tx=0x7f3480006ca0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.251+0000 7f349edbb640 1 -- 172.21.15.29:0/1079651096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34980b3140 msgr2=0x7f349813f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.251+0000 7f349edbb640 1 --2- 172.21.15.29:0/1079651096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34980b3140 0x7f349813f220 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f34900ae040 tx=0x7f349001aca0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.251+0000 7f349edbb640 1 -- 172.21.15.29:0/1079651096 shutdown_connections 2024-04-25T00:32:30.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.251+0000 7f349edbb640 1 --2- 172.21.15.29:0/1079651096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34980b4b10 0x7f34981402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.251+0000 7f349edbb640 1 --2- 172.21.15.29:0/1079651096 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f34780678c0 0x7f3478069d80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.251+0000 7f349edbb640 1 --2- 172.21.15.29:0/1079651096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34980b3a60 0x7f349813d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.251+0000 7f349edbb640 1 --2- 172.21.15.29:0/1079651096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34980b3140 0x7f349813f220 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.251+0000 7f349edbb640 1 -- 172.21.15.29:0/1079651096 >> 172.21.15.29:0/1079651096 conn(0x7f34980b1870 msgr2=0x7f3498142be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:30.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.251+0000 7f349edbb640 1 -- 172.21.15.29:0/1079651096 shutdown_connections 2024-04-25T00:32:30.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.251+0000 7f349edbb640 1 -- 172.21.15.29:0/1079651096 wait complete. 2024-04-25T00:32:30.271 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T00:32:30.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.379+0000 7f1e90db1640 1 Processor -- start 2024-04-25T00:32:30.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.379+0000 7f1e90db1640 1 -- start start 2024-04-25T00:32:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e90db1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1e8c15a790 0x7f1e8c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e90db1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1e8c0cfe70 0x7f1e8c0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e90db1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e8c0c3c90 con 0x7f1e8c0b0000 2024-04-25T00:32:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e90db1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1e8c0ae610 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e90db1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1e8c0af8c0 con 0x7f1e8c15a790 2024-04-25T00:32:30.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e89d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1e8c0cfe70 0x7f1e8c0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e89d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1e8c0cfe70 0x7f1e8c0cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37398/0 (socket says 172.21.15.29:37398) 2024-04-25T00:32:30.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e89d9b640 1 -- 172.21.15.29:0/2320860490 learned_addr learned my addr 172.21.15.29:0/2320860490 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:30.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e8ad9d640 1 --1- 172.21.15.29:0/2320860490 >> v1:172.21.15.29:6789/0 conn(0x7f1e8c0b0000 0x7f1e8c0aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37212/0 (socket says 172.21.15.29:37212) 2024-04-25T00:32:30.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e8a59c640 1 --2- 172.21.15.29:0/2320860490 >> v2:172.21.15.157:3300/0 conn(0x7f1e8c15a790 0x7f1e8c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e89d9b640 1 -- 172.21.15.29:0/2320860490 >> v2:172.21.15.157:3300/0 conn(0x7f1e8c15a790 msgr2=0x7f1e8c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:30.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e89d9b640 1 --2- 172.21.15.29:0/2320860490 >> v2:172.21.15.157:3300/0 conn(0x7f1e8c15a790 0x7f1e8c0afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e89d9b640 1 -- 172.21.15.29:0/2320860490 >> v1:172.21.15.29:6789/0 conn(0x7f1e8c0b0000 legacy=0x7f1e8c0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e89d9b640 1 -- 172.21.15.29:0/2320860490 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c0adce0 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e89d9b640 1 --2- 172.21.15.29:0/2320860490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e8c0cfe70 0x7f1e8c0cdf60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0704d0 tx=0x7f1e8001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfbf4b40f0ee220 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:30.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e8959a640 1 -- 172.21.15.29:0/2320860490 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e800b0070 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e8959a640 1 -- 172.21.15.29:0/2320860490 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e800ab600 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e8959a640 1 -- 172.21.15.29:0/2320860490 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e800a4e60 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e90db1640 1 -- 172.21.15.29:0/2320860490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e8c0cfe70 msgr2=0x7f1e8c0cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e90db1640 1 --2- 172.21.15.29:0/2320860490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e8c0cfe70 0x7f1e8c0cdf60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0704d0 tx=0x7f1e8001c3d0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e90db1640 1 -- 172.21.15.29:0/2320860490 shutdown_connections 2024-04-25T00:32:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e90db1640 1 --2- 172.21.15.29:0/2320860490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e8c0cfe70 0x7f1e8c0cdf60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e90db1640 1 --2- 172.21.15.29:0/2320860490 >> v2:172.21.15.157:3300/0 conn(0x7f1e8c15a790 0x7f1e8c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e90db1640 1 -- 172.21.15.29:0/2320860490 >> 172.21.15.29:0/2320860490 conn(0x7f1e8c0b1ec0 msgr2=0x7f1e8c0b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e90db1640 1 -- 172.21.15.29:0/2320860490 shutdown_connections 2024-04-25T00:32:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.383+0000 7f1e90db1640 1 -- 172.21.15.29:0/2320860490 wait complete. 2024-04-25T00:32:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e90db1640 1 Processor -- start 2024-04-25T00:32:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e90db1640 1 -- start start 2024-04-25T00:32:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e90db1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e8c0b0000 0x7f1e8c168400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e90db1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e8c0cfe70 0x7f1e8c0c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e90db1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c15a790 0x7f1e8c0c3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e90db1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c15ab70 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e90db1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c0b2bf0 con 0x7f1e8c15a790 2024-04-25T00:32:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e90db1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8c158b90 con 0x7f1e8c0b0000 2024-04-25T00:32:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e89d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e8c0cfe70 0x7f1e8c0c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e89d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e8c0cfe70 0x7f1e8c0c2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37408/0 (socket says 172.21.15.29:37408) 2024-04-25T00:32:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e89d9b640 1 -- 172.21.15.29:0/3876126482 learned_addr learned my addr 172.21.15.29:0/3876126482 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e8a59c640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e8c0b0000 0x7f1e8c168400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e8ad9d640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c15a790 0x7f1e8c0c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e89d9b640 1 -- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e8c0b0000 msgr2=0x7f1e8c168400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e89d9b640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e8c0b0000 0x7f1e8c168400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e89d9b640 1 -- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c15a790 msgr2=0x7f1e8c0c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e89d9b640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c15a790 0x7f1e8c0c3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e89d9b640 1 -- 172.21.15.29:0/3876126482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c0c3ac0 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e8a59c640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e8c0b0000 0x7f1e8c168400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e89d9b640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e8c0cfe70 0x7f1e8c0c2be0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1e800ae040 tx=0x7f1e80019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e7affd640 1 -- 172.21.15.29:0/3876126482 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e800bb070 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e7affd640 1 -- 172.21.15.29:0/3876126482 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e800a4e60 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e90db1640 1 -- 172.21.15.29:0/3876126482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c158580 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e7affd640 1 -- 172.21.15.29:0/3876126482 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e800b0040 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e90db1640 1 -- 172.21.15.29:0/3876126482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c0b4920 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.387+0000 7f1e90db1640 1 -- 172.21.15.29:0/3876126482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8c069270 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.391+0000 7f1e7affd640 1 -- 172.21.15.29:0/3876126482 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1e800be070 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.391+0000 7f1e7affd640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1e6c0678c0 0x7f1e6c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.391+0000 7f1e7affd640 1 -- 172.21.15.29:0/3876126482 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f1e8011efe0 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.391+0000 7f1e8a59c640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1e6c0678c0 0x7f1e6c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.391+0000 7f1e8a59c640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1e6c0678c0 0x7f1e6c069d80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c06e0e0 tx=0x7f1e7c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:30.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.395+0000 7f1e7affd640 1 -- 172.21.15.29:0/3876126482 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1e80016110 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.607+0000 7f1e90db1640 1 -- 172.21.15.29:0/3876126482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f1e8c0b6310 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.611+0000 7f1e7affd640 1 -- 172.21.15.29:0/3876126482 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) ==== 97+0+0 (secure 0 0 0) 0x7f1e800efe70 con 0x7f1e8c0cfe70 2024-04-25T00:32:30.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.611+0000 7f1e90db1640 1 -- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1e6c0678c0 msgr2=0x7f1e6c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.611+0000 7f1e90db1640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1e6c0678c0 0x7f1e6c069d80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c06e0e0 tx=0x7f1e7c008040 comp rx=0 tx=0).stop 2024-04-25T00:32:30.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.611+0000 7f1e90db1640 1 -- 172.21.15.29:0/3876126482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e8c0cfe70 msgr2=0x7f1e8c0c2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.611+0000 7f1e90db1640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e8c0cfe70 0x7f1e8c0c2be0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1e800ae040 tx=0x7f1e80019170 comp rx=0 tx=0).stop 2024-04-25T00:32:30.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.611+0000 7f1e90db1640 1 -- 172.21.15.29:0/3876126482 shutdown_connections 2024-04-25T00:32:30.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.611+0000 7f1e90db1640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1e6c0678c0 0x7f1e6c069d80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.611+0000 7f1e90db1640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c15a790 0x7f1e8c0c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.611+0000 7f1e90db1640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e8c0cfe70 0x7f1e8c0c2be0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.611+0000 7f1e90db1640 1 --2- 172.21.15.29:0/3876126482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e8c0b0000 0x7f1e8c168400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.611+0000 7f1e90db1640 1 -- 172.21.15.29:0/3876126482 >> 172.21.15.29:0/3876126482 conn(0x7f1e8c0b1ec0 msgr2=0x7f1e8c06d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:30.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.611+0000 7f1e90db1640 1 -- 172.21.15.29:0/3876126482 shutdown_connections 2024-04-25T00:32:30.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.611+0000 7f1e90db1640 1 -- 172.21.15.29:0/3876126482 wait complete. 2024-04-25T00:32:30.631 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T00:32:30.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f11f26640 1 Processor -- start 2024-04-25T00:32:30.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f11f26640 1 -- start start 2024-04-25T00:32:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f11f26640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2f0c15e8d0 0x7f2f0c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f11f26640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f0c0b3720 con 0x7f2f0c15f3c0 2024-04-25T00:32:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f11f26640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f0c0b39c0 con 0x7f2f0c0b4780 2024-04-25T00:32:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f11f26640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2f0c0b42f0 con 0x7f2f0c15e8d0 2024-04-25T00:32:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2f0c15f3c0 0x7f2f0c15c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37216/0 (socket says 172.21.15.29:37216) 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0bfff640 1 -- 172.21.15.29:0/3991691866 learned_addr learned my addr 172.21.15.29:0/3991691866 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0b7fe640 1 --2- 172.21.15.29:0/3991691866 >> v2:172.21.15.157:3300/0 conn(0x7f2f0c15e8d0 0x7f2f0c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3655966452 0 0) 0x7f2f0c0b3720 con 0x7f2f0c15f3c0 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ef4003530 con 0x7f2f0c15f3c0 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 844469835 0 0) 0x7f2ef4003530 con 0x7f2f0c15f3c0 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 >> v2:172.21.15.157:3300/0 conn(0x7f2f0c15e8d0 msgr2=0x7f2f0c0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0a7fc640 1 --2- 172.21.15.29:0/3991691866 >> v2:172.21.15.157:3300/0 conn(0x7f2f0c15e8d0 0x7f2f0c0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 >> v1:172.21.15.157:6789/0 conn(0x7f2f0c0b4780 legacy=0x7f2f0c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0c0b3ca0 con 0x7f2f0c15f3c0 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2f04001a00 con 0x7f2f0c15f3c0 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 >> v1:172.21.15.29:6789/0 conn(0x7f2f0c15f3c0 legacy=0x7f2f0c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0a7fc640 1 --2- 172.21.15.29:0/3991691866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ef40039c0 0x7f2ef4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0a7fc640 1 --2- 172.21.15.29:0/3991691866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ef4004810 0x7f2ef4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.739+0000 7f2f0a7fc640 1 --2- 172.21.15.29:0/3991691866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40051d0 0x7f2ef400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c0b42f0 con 0x7f2ef4004810 2024-04-25T00:32:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef400c070 con 0x7f2ef40051d0 2024-04-25T00:32:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef400c270 con 0x7f2ef40039c0 2024-04-25T00:32:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0affd640 1 --2- 172.21.15.29:0/3991691866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ef4004810 0x7f2ef4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0b7fe640 1 --2- 172.21.15.29:0/3991691866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ef40039c0 0x7f2ef4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0bfff640 1 --2- 172.21.15.29:0/3991691866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40051d0 0x7f2ef400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0affd640 1 -- 172.21.15.29:0/3991691866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ef40039c0 msgr2=0x7f2ef4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0affd640 1 --2- 172.21.15.29:0/3991691866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ef40039c0 0x7f2ef4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0affd640 1 -- 172.21.15.29:0/3991691866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40051d0 msgr2=0x7f2ef400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0affd640 1 --2- 172.21.15.29:0/3991691866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40051d0 0x7f2ef400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0affd640 1 -- 172.21.15.29:0/3991691866 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef4003530 con 0x7f2ef4004810 2024-04-25T00:32:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0bfff640 1 --2- 172.21.15.29:0/3991691866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40051d0 0x7f2ef400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:30.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0bfff640 1 -- 172.21.15.29:0/3991691866 reap_dead start 2024-04-25T00:32:30.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0affd640 1 --2- 172.21.15.29:0/3991691866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ef4004810 0x7f2ef4004c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2f00001780 tx=0x7f2f0001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545075a50b76cdc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:30.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f000013a0 con 0x7f2ef4004810 2024-04-25T00:32:30.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f000052f0 con 0x7f2ef4004810 2024-04-25T00:32:30.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f0a7fc640 1 -- 172.21.15.29:0/3991691866 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00003840 con 0x7f2ef4004810 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f11f26640 1 -- 172.21.15.29:0/3991691866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ef4004810 msgr2=0x7f2ef4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f11f26640 1 --2- 172.21.15.29:0/3991691866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ef4004810 0x7f2ef4004c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2f00001780 tx=0x7f2f0001c400 comp rx=0 tx=0).stop 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f11f26640 1 -- 172.21.15.29:0/3991691866 shutdown_connections 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f11f26640 1 --2- 172.21.15.29:0/3991691866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ef4004810 0x7f2ef4004c90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f11f26640 1 -- 172.21.15.29:0/3991691866 >> 172.21.15.29:0/3991691866 conn(0x7f2f0c0b1870 msgr2=0x7f2f0c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f11f26640 1 -- 172.21.15.29:0/3991691866 shutdown_connections 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.743+0000 7f2f11f26640 1 -- 172.21.15.29:0/3991691866 wait complete. 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f11f26640 1 Processor -- start 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f11f26640 1 -- start start 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f11f26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef4004810 0x7f2f0c0b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f11f26640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f0c072790 0x7f2f0c1f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f11f26640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f0c15ba30 0x7f2f0c1f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f11f26640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c144a20 con 0x7f2f0c15ba30 2024-04-25T00:32:30.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f11f26640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c06cf10 con 0x7f2ef4004810 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f11f26640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0c072f00 con 0x7f2f0c072790 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f0bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f0c15ba30 0x7f2f0c1f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f0bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f0c15ba30 0x7f2f0c1f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37426/0 (socket says 172.21.15.29:37426) 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f0bfff640 1 -- 172.21.15.29:0/3234323085 learned_addr learned my addr 172.21.15.29:0/3234323085 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f0b7fe640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef4004810 0x7f2f0c0b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f0affd640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f0c072790 0x7f2f0c1f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f0bfff640 1 -- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f0c072790 msgr2=0x7f2f0c1f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f0bfff640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f0c072790 0x7f2f0c1f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f0bfff640 1 -- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef4004810 msgr2=0x7f2f0c0b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f0bfff640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef4004810 0x7f2f0c0b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f0bfff640 1 -- 172.21.15.29:0/3234323085 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0c15b7a0 con 0x7f2f0c15ba30 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f0bfff640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f0c15ba30 0x7f2f0c1f6150 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2f04001340 tx=0x7f2f04010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2ef3fff640 1 -- 172.21.15.29:0/3234323085 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0401a070 con 0x7f2f0c15ba30 2024-04-25T00:32:30.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2ef3fff640 1 -- 172.21.15.29:0/3234323085 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f04011040 con 0x7f2f0c15ba30 2024-04-25T00:32:30.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f11f26640 1 -- 172.21.15.29:0/3234323085 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0c12a7a0 con 0x7f2f0c15ba30 2024-04-25T00:32:30.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2ef3fff640 1 -- 172.21.15.29:0/3234323085 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400ece0 con 0x7f2f0c15ba30 2024-04-25T00:32:30.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f11f26640 1 -- 172.21.15.29:0/3234323085 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0c12aa40 con 0x7f2f0c15ba30 2024-04-25T00:32:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f11f26640 1 -- 172.21.15.29:0/3234323085 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0c069270 con 0x7f2f0c15ba30 2024-04-25T00:32:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2ef3fff640 1 -- 172.21.15.29:0/3234323085 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2f0401d070 con 0x7f2f0c15ba30 2024-04-25T00:32:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2ef3fff640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2ef80678c0 0x7f2ef8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2ef3fff640 1 -- 172.21.15.29:0/3234323085 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f2f04088060 con 0x7f2f0c15ba30 2024-04-25T00:32:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.747+0000 7f2f0b7fe640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2ef80678c0 0x7f2ef8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:30.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.751+0000 7f2f0b7fe640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2ef80678c0 0x7f2ef8069d80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009ab0 tx=0x7f2efc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:30.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.751+0000 7f2ef3fff640 1 -- 172.21.15.29:0/3234323085 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2f04058ef0 con 0x7f2f0c15ba30 2024-04-25T00:32:30.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.967+0000 7f2f11f26640 1 -- 172.21.15.29:0/3234323085 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f2f0c06efd0 con 0x7f2f0c15ba30 2024-04-25T00:32:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.971+0000 7f2ef3fff640 1 -- 172.21.15.29:0/3234323085 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) ==== 90+0+0 (secure 0 0 0) 0x7f2f040587f0 con 0x7f2f0c15ba30 2024-04-25T00:32:30.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.971+0000 7f2f11f26640 1 -- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2ef80678c0 msgr2=0x7f2ef8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.971+0000 7f2f11f26640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2ef80678c0 0x7f2ef8069d80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009ab0 tx=0x7f2efc008040 comp rx=0 tx=0).stop 2024-04-25T00:32:30.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.971+0000 7f2f11f26640 1 -- 172.21.15.29:0/3234323085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f0c15ba30 msgr2=0x7f2f0c1f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:30.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.971+0000 7f2f11f26640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f0c15ba30 0x7f2f0c1f6150 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2f04001340 tx=0x7f2f04010cf0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.971+0000 7f2f11f26640 1 -- 172.21.15.29:0/3234323085 shutdown_connections 2024-04-25T00:32:30.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.971+0000 7f2f11f26640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f0c15ba30 0x7f2f0c1f6150 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.971+0000 7f2f11f26640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f0c072790 0x7f2f0c1f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.971+0000 7f2f11f26640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2ef80678c0 0x7f2ef8069d80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.971+0000 7f2f11f26640 1 --2- 172.21.15.29:0/3234323085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef4004810 0x7f2f0c0b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:30.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.971+0000 7f2f11f26640 1 -- 172.21.15.29:0/3234323085 >> 172.21.15.29:0/3234323085 conn(0x7f2f0c0b1870 msgr2=0x7f2f0c15ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:30.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.971+0000 7f2f11f26640 1 -- 172.21.15.29:0/3234323085 shutdown_connections 2024-04-25T00:32:30.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:30.971+0000 7f2f11f26640 1 -- 172.21.15.29:0/3234323085 wait complete. 2024-04-25T00:32:30.992 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:31.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e77568640 1 Processor -- start 2024-04-25T00:32:31.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e77568640 1 -- start start 2024-04-25T00:32:31.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e77568640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e700b1290 0x7f9e700cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e77568640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9e700d0260 0x7f9e700cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e77568640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e700d1340 con 0x7f9e7015e2f0 2024-04-25T00:32:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e77568640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9e7015e6d0 con 0x7f9e700d0260 2024-04-25T00:32:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e77568640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9e7006c4a0 con 0x7f9e700b1290 2024-04-25T00:32:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e75304640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9e7015e2f0 0x7f9e7006cf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37228/0 (socket says 172.21.15.29:37228) 2024-04-25T00:32:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e75304640 1 -- 172.21.15.29:0/2562905617 learned_addr learned my addr 172.21.15.29:0/2562905617 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e75b05640 1 --2- 172.21.15.29:0/2562905617 >> v2:172.21.15.29:3300/0 conn(0x7f9e700d0260 0x7f9e700cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e74b03640 1 --2- 172.21.15.29:0/2562905617 >> v2:172.21.15.157:3300/0 conn(0x7f9e700b1290 0x7f9e700cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e67fff640 1 -- 172.21.15.29:0/2562905617 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 625264151 0 0) 0x7f9e700d1340 con 0x7f9e7015e2f0 2024-04-25T00:32:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e67fff640 1 -- 172.21.15.29:0/2562905617 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e5c003530 con 0x7f9e7015e2f0 2024-04-25T00:32:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e75b05640 1 -- 172.21.15.29:0/2562905617 >> v2:172.21.15.157:3300/0 conn(0x7f9e700b1290 msgr2=0x7f9e700cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e75b05640 1 --2- 172.21.15.29:0/2562905617 >> v2:172.21.15.157:3300/0 conn(0x7f9e700b1290 0x7f9e700cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e75b05640 1 -- 172.21.15.29:0/2562905617 >> v1:172.21.15.29:6789/0 conn(0x7f9e7015e2f0 legacy=0x7f9e7006cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.107+0000 7f9e75b05640 1 -- 172.21.15.29:0/2562905617 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7006c780 con 0x7f9e700d0260 2024-04-25T00:32:31.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e75b05640 1 --2- 172.21.15.29:0/2562905617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e700d0260 0x7f9e700cc9a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9e70071b90 tx=0x7f9e6801f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1530ac418837a6d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e67fff640 1 -- 172.21.15.29:0/2562905617 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e680b4070 con 0x7f9e700d0260 2024-04-25T00:32:31.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e67fff640 1 -- 172.21.15.29:0/2562905617 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e680af600 con 0x7f9e700d0260 2024-04-25T00:32:31.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e67fff640 1 -- 172.21.15.29:0/2562905617 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e680af9d0 con 0x7f9e700d0260 2024-04-25T00:32:31.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 -- 172.21.15.29:0/2562905617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e700d0260 msgr2=0x7f9e700cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 --2- 172.21.15.29:0/2562905617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e700d0260 0x7f9e700cc9a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9e70071b90 tx=0x7f9e6801f110 comp rx=0 tx=0).stop 2024-04-25T00:32:31.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 -- 172.21.15.29:0/2562905617 shutdown_connections 2024-04-25T00:32:31.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 --2- 172.21.15.29:0/2562905617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e700d0260 0x7f9e700cc9a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 --2- 172.21.15.29:0/2562905617 >> v2:172.21.15.157:3300/0 conn(0x7f9e700b1290 0x7f9e700cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 -- 172.21.15.29:0/2562905617 >> 172.21.15.29:0/2562905617 conn(0x7f9e700d1e10 msgr2=0x7f9e700d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:31.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 -- 172.21.15.29:0/2562905617 shutdown_connections 2024-04-25T00:32:31.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 -- 172.21.15.29:0/2562905617 wait complete. 2024-04-25T00:32:31.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 Processor -- start 2024-04-25T00:32:31.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 -- start start 2024-04-25T00:32:31.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e700b1290 0x7f9e701f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e700d0260 0x7f9e701591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7015e2f0 0x7f9e700afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7006d690 con 0x7f9e700b1290 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9e700cf6c0 con 0x7f9e7015e2f0 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e77568640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7006d810 con 0x7f9e700d0260 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e75304640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e700b1290 0x7f9e701f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e75304640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e700b1290 0x7f9e701f4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37438/0 (socket says 172.21.15.29:37438) 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.111+0000 7f9e75304640 1 -- 172.21.15.29:0/4246723612 learned_addr learned my addr 172.21.15.29:0/4246723612 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e74b03640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e700d0260 0x7f9e701591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e75b05640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7015e2f0 0x7f9e700afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e75304640 1 -- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e700d0260 msgr2=0x7f9e701591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e75304640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e700d0260 0x7f9e701591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e75304640 1 -- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7015e2f0 msgr2=0x7f9e700afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e75304640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7015e2f0 0x7f9e700afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e75304640 1 -- 172.21.15.29:0/4246723612 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e701f4650 con 0x7f9e700b1290 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e74b03640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e700d0260 0x7f9e701591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:31.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e75304640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e700b1290 0x7f9e701f4c30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c450 tx=0x7f9e6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e65ffb640 1 -- 172.21.15.29:0/4246723612 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c017070 con 0x7f9e700b1290 2024-04-25T00:32:31.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e65ffb640 1 -- 172.21.15.29:0/4246723612 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e6c005cf0 con 0x7f9e700b1290 2024-04-25T00:32:31.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e65ffb640 1 -- 172.21.15.29:0/4246723612 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c007290 con 0x7f9e700b1290 2024-04-25T00:32:31.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e77568640 1 -- 172.21.15.29:0/4246723612 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e701f4930 con 0x7f9e700b1290 2024-04-25T00:32:31.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e77568640 1 -- 172.21.15.29:0/4246723612 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7009c380 con 0x7f9e700b1290 2024-04-25T00:32:31.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e77568640 1 -- 172.21.15.29:0/4246723612 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e70069270 con 0x7f9e700b1290 2024-04-25T00:32:31.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e65ffb640 1 -- 172.21.15.29:0/4246723612 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9e6c01a070 con 0x7f9e700b1290 2024-04-25T00:32:31.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e65ffb640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9e540678c0 0x7f9e54069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e65ffb640 1 -- 172.21.15.29:0/4246723612 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f9e6c084d50 con 0x7f9e700b1290 2024-04-25T00:32:31.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.115+0000 7f9e74b03640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9e540678c0 0x7f9e54069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.119+0000 7f9e74b03640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9e540678c0 0x7f9e54069d80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9e701f6fc0 tx=0x7f9e60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.123+0000 7f9e65ffb640 1 -- 172.21.15.29:0/4246723612 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9e6c055bd0 con 0x7f9e700b1290 2024-04-25T00:32:31.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.283+0000 7f9e77568640 1 -- 172.21.15.29:0/4246723612 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e7009ec20 con 0x7f9e700b1290 2024-04-25T00:32:31.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.287+0000 7f9e65ffb640 1 -- 172.21.15.29:0/4246723612 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12843 (secure 0 0 0) 0x7f9e7009ec20 con 0x7f9e700b1290 2024-04-25T00:32:31.289 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:31.289 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":29,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:32:28.382987+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:31.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.287+0000 7f9e77568640 1 -- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9e540678c0 msgr2=0x7f9e54069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.287+0000 7f9e77568640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9e540678c0 0x7f9e54069d80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9e701f6fc0 tx=0x7f9e60008040 comp rx=0 tx=0).stop 2024-04-25T00:32:31.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.287+0000 7f9e77568640 1 -- 172.21.15.29:0/4246723612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e700b1290 msgr2=0x7f9e701f4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.287+0000 7f9e77568640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e700b1290 0x7f9e701f4c30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c450 tx=0x7f9e6c00c910 comp rx=0 tx=0).stop 2024-04-25T00:32:31.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.287+0000 7f9e77568640 1 -- 172.21.15.29:0/4246723612 shutdown_connections 2024-04-25T00:32:31.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.287+0000 7f9e77568640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9e540678c0 0x7f9e54069d80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.287+0000 7f9e77568640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7015e2f0 0x7f9e700afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.287+0000 7f9e77568640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e700d0260 0x7f9e701591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.287+0000 7f9e77568640 1 --2- 172.21.15.29:0/4246723612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e700b1290 0x7f9e701f4c30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.287+0000 7f9e77568640 1 -- 172.21.15.29:0/4246723612 >> 172.21.15.29:0/4246723612 conn(0x7f9e700d1e10 msgr2=0x7f9e700b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:31.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.287+0000 7f9e77568640 1 -- 172.21.15.29:0/4246723612 shutdown_connections 2024-04-25T00:32:31.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.287+0000 7f9e77568640 1 -- 172.21.15.29:0/4246723612 wait complete. 2024-04-25T00:32:31.309 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:31.310 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:31.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f173218e640 1 Processor -- start 2024-04-25T00:32:31.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f173218e640 1 -- start start 2024-04-25T00:32:31.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f173218e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172c0b4780 0x7f172c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f173218e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f172c0cc240 0x7f172c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f173218e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f172c0b3720 con 0x7f172c15e8d0 2024-04-25T00:32:31.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f173218e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f172c0b42f0 con 0x7f172c0cc240 2024-04-25T00:32:31.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f173218e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f172c0b39c0 con 0x7f172c0b4780 2024-04-25T00:32:31.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f172bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f172c0cc240 0x7f172c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f172bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f172c0cc240 0x7f172c15c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37450/0 (socket says 172.21.15.29:37450) 2024-04-25T00:32:31.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f172bfff640 1 -- 172.21.15.29:0/3855135541 learned_addr learned my addr 172.21.15.29:0/3855135541 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:31.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f172affd640 1 --2- 172.21.15.29:0/3855135541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172c0b4780 0x7f172c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f172bfff640 1 -- 172.21.15.29:0/3855135541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172c0b4780 msgr2=0x7f172c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:31.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f172bfff640 1 --2- 172.21.15.29:0/3855135541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172c0b4780 0x7f172c15f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f172bfff640 1 -- 172.21.15.29:0/3855135541 >> v1:172.21.15.157:6789/0 conn(0x7f172c15e8d0 legacy=0x7f172c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:31.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.431+0000 7f172bfff640 1 -- 172.21.15.29:0/3855135541 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c0b3ca0 con 0x7f172c0cc240 2024-04-25T00:32:31.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172bfff640 1 --2- 172.21.15.29:0/3855135541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f172c0cc240 0x7f172c15c7d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1724009e50 tx=0x7f172401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1434ba68d2dcad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172a7fc640 1 -- 172.21.15.29:0/3855135541 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17240a5740 con 0x7f172c0cc240 2024-04-25T00:32:31.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172a7fc640 1 -- 172.21.15.29:0/3855135541 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17240a58e0 con 0x7f172c0cc240 2024-04-25T00:32:31.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172a7fc640 1 -- 172.21.15.29:0/3855135541 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17240a5cb0 con 0x7f172c0cc240 2024-04-25T00:32:31.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 -- 172.21.15.29:0/3855135541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f172c0cc240 msgr2=0x7f172c15c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 --2- 172.21.15.29:0/3855135541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f172c0cc240 0x7f172c15c7d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1724009e50 tx=0x7f172401c9f0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 -- 172.21.15.29:0/3855135541 shutdown_connections 2024-04-25T00:32:31.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 --2- 172.21.15.29:0/3855135541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f172c0cc240 0x7f172c15c7d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 --2- 172.21.15.29:0/3855135541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172c0b4780 0x7f172c15f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 -- 172.21.15.29:0/3855135541 >> 172.21.15.29:0/3855135541 conn(0x7f172c0b1870 msgr2=0x7f172c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:31.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 -- 172.21.15.29:0/3855135541 shutdown_connections 2024-04-25T00:32:31.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 -- 172.21.15.29:0/3855135541 wait complete. 2024-04-25T00:32:31.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 Processor -- start 2024-04-25T00:32:31.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 -- start start 2024-04-25T00:32:31.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172c0b4780 0x7f172c168920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f172c0cc240 0x7f172c1537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172c15e8d0 0x7f172c0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f172c15d6a0 con 0x7f172c0cc240 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f172c15d520 con 0x7f172c15e8d0 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f173218e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f172c165c60 con 0x7f172c0b4780 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f172c0cc240 0x7f172c1537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f172c0cc240 0x7f172c1537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37460/0 (socket says 172.21.15.29:37460) 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172affd640 1 -- 172.21.15.29:0/635682577 learned_addr learned my addr 172.21.15.29:0/635682577 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172bfff640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172c15e8d0 0x7f172c0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172b7fe640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172c0b4780 0x7f172c168920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172affd640 1 -- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172c0b4780 msgr2=0x7f172c168920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172affd640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172c0b4780 0x7f172c168920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172affd640 1 -- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172c15e8d0 msgr2=0x7f172c0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172affd640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172c15e8d0 0x7f172c0cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172affd640 1 -- 172.21.15.29:0/635682577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c167a20 con 0x7f172c0cc240 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f172affd640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f172c0cc240 0x7f172c1537d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f172000d800 tx=0x7f172000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f1713fff640 1 -- 172.21.15.29:0/635682577 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172000b6c0 con 0x7f172c0cc240 2024-04-25T00:32:31.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.435+0000 7f1713fff640 1 -- 172.21.15.29:0/635682577 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1720019070 con 0x7f172c0cc240 2024-04-25T00:32:31.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.439+0000 7f173218e640 1 -- 172.21.15.29:0/635682577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172c167d00 con 0x7f172c0cc240 2024-04-25T00:32:31.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.439+0000 7f1713fff640 1 -- 172.21.15.29:0/635682577 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172000bae0 con 0x7f172c0cc240 2024-04-25T00:32:31.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.439+0000 7f173218e640 1 -- 172.21.15.29:0/635682577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172c1546d0 con 0x7f172c0cc240 2024-04-25T00:32:31.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.439+0000 7f173218e640 1 -- 172.21.15.29:0/635682577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172c069270 con 0x7f172c0cc240 2024-04-25T00:32:31.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.439+0000 7f1713fff640 1 -- 172.21.15.29:0/635682577 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f172000bc80 con 0x7f172c0cc240 2024-04-25T00:32:31.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.439+0000 7f1713fff640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f17180678c0 0x7f1718069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.439+0000 7f1713fff640 1 -- 172.21.15.29:0/635682577 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f1720085e30 con 0x7f172c0cc240 2024-04-25T00:32:31.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.439+0000 7f172b7fe640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f17180678c0 0x7f1718069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.439+0000 7f172b7fe640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f17180678c0 0x7f1718069d80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f171c00b810 tx=0x7f171c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.443+0000 7f1713fff640 1 -- 172.21.15.29:0/635682577 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1720056cc0 con 0x7f172c0cc240 2024-04-25T00:32:31.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.599+0000 7f173218e640 1 -- 172.21.15.29:0/635682577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f172c06e7b0 con 0x7f172c0cc240 2024-04-25T00:32:31.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.599+0000 7f1713fff640 1 -- 172.21.15.29:0/635682577 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f17200565c0 con 0x7f172c0cc240 2024-04-25T00:32:31.603 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:31.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.603+0000 7f173218e640 1 -- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f17180678c0 msgr2=0x7f1718069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.603+0000 7f173218e640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f17180678c0 0x7f1718069d80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f171c00b810 tx=0x7f171c008040 comp rx=0 tx=0).stop 2024-04-25T00:32:31.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.603+0000 7f173218e640 1 -- 172.21.15.29:0/635682577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f172c0cc240 msgr2=0x7f172c1537d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.603+0000 7f173218e640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f172c0cc240 0x7f172c1537d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f172000d800 tx=0x7f172000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.603+0000 7f173218e640 1 -- 172.21.15.29:0/635682577 shutdown_connections 2024-04-25T00:32:31.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.603+0000 7f173218e640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f17180678c0 0x7f1718069d80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.603+0000 7f173218e640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172c15e8d0 0x7f172c0cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.603+0000 7f173218e640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f172c0cc240 0x7f172c1537d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.603+0000 7f173218e640 1 --2- 172.21.15.29:0/635682577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172c0b4780 0x7f172c168920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.603+0000 7f173218e640 1 -- 172.21.15.29:0/635682577 >> 172.21.15.29:0/635682577 conn(0x7f172c0b1870 msgr2=0x7f172c06dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:31.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.603+0000 7f173218e640 1 -- 172.21.15.29:0/635682577 shutdown_connections 2024-04-25T00:32:31.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.603+0000 7f173218e640 1 -- 172.21.15.29:0/635682577 wait complete. 2024-04-25T00:32:31.621 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:32:31.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.735+0000 7ffa55c97640 1 Processor -- start 2024-04-25T00:32:31.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.735+0000 7ffa55c97640 1 -- start start 2024-04-25T00:32:31.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa55c97640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa500b3140 0x7ffa5006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa55c97640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa500b3c30 0x7ffa500ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa55c97640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa500b41e0 con 0x7ffa500b4b10 2024-04-25T00:32:31.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa55c97640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ffa5006d960 con 0x7ffa500b3140 2024-04-25T00:32:31.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa55c97640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5006dc40 con 0x7ffa500b3c30 2024-04-25T00:32:31.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa4f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa500b3140 0x7ffa5006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa4f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa500b3140 0x7ffa5006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37470/0 (socket says 172.21.15.29:37470) 2024-04-25T00:32:31.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa4f7fe640 1 -- 172.21.15.29:0/1628189135 learned_addr learned my addr 172.21.15.29:0/1628189135 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:31.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa4ffff640 1 --2- 172.21.15.29:0/1628189135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa500b3c30 0x7ffa500ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa4f7fe640 1 -- 172.21.15.29:0/1628189135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa500b3c30 msgr2=0x7ffa500ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:31.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa4f7fe640 1 --2- 172.21.15.29:0/1628189135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa500b3c30 0x7ffa500ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa4f7fe640 1 -- 172.21.15.29:0/1628189135 >> v1:172.21.15.157:6789/0 conn(0x7ffa500b4b10 legacy=0x7ffa500b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa4f7fe640 1 -- 172.21.15.29:0/1628189135 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5006df20 con 0x7ffa500b3140 2024-04-25T00:32:31.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa4f7fe640 1 --2- 172.21.15.29:0/1628189135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa500b3140 0x7ffa5006e440 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffa400098a0 tx=0x7ffa4001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a72a9de9213bba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa4e7fc640 1 -- 172.21.15.29:0/1628189135 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa400b0070 con 0x7ffa500b3140 2024-04-25T00:32:31.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa4e7fc640 1 -- 172.21.15.29:0/1628189135 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa400ab600 con 0x7ffa500b3140 2024-04-25T00:32:31.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.739+0000 7ffa4e7fc640 1 -- 172.21.15.29:0/1628189135 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa400ab9d0 con 0x7ffa500b3140 2024-04-25T00:32:31.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 -- 172.21.15.29:0/1628189135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa500b3140 msgr2=0x7ffa5006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 --2- 172.21.15.29:0/1628189135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa500b3140 0x7ffa5006e440 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffa400098a0 tx=0x7ffa4001c540 comp rx=0 tx=0).stop 2024-04-25T00:32:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 -- 172.21.15.29:0/1628189135 shutdown_connections 2024-04-25T00:32:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 --2- 172.21.15.29:0/1628189135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa500b3c30 0x7ffa500ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 --2- 172.21.15.29:0/1628189135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa500b3140 0x7ffa5006e440 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 -- 172.21.15.29:0/1628189135 >> 172.21.15.29:0/1628189135 conn(0x7ffa500b1870 msgr2=0x7ffa500b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 -- 172.21.15.29:0/1628189135 shutdown_connections 2024-04-25T00:32:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 -- 172.21.15.29:0/1628189135 wait complete. 2024-04-25T00:32:31.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 Processor -- start 2024-04-25T00:32:31.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 -- start start 2024-04-25T00:32:31.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa500b3140 0x7ffa50152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa500b3c30 0x7ffa5015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa500b4b10 0x7ffa5015ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50168d80 con 0x7ffa500b3140 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50168c00 con 0x7ffa500b4b10 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffa50168f00 con 0x7ffa500b3c30 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa4f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa500b3140 0x7ffa50152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa4f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa500b3140 0x7ffa50152ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37486/0 (socket says 172.21.15.29:37486) 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa4f7fe640 1 -- 172.21.15.29:0/237913417 learned_addr learned my addr 172.21.15.29:0/237913417 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa4effd640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa500b3c30 0x7ffa5015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa4f7fe640 1 -- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa500b3c30 msgr2=0x7ffa5015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa4f7fe640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa500b3c30 0x7ffa5015ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa4f7fe640 1 -- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa500b4b10 msgr2=0x7ffa5015ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa4ffff640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa500b4b10 0x7ffa5015ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa4f7fe640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa500b4b10 0x7ffa5015ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa4f7fe640 1 -- 172.21.15.29:0/237913417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5015f270 con 0x7ffa500b3140 2024-04-25T00:32:31.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa4f7fe640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa500b3140 0x7ffa50152ba0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffa4001abf0 tx=0x7ffa400192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa37fff640 1 -- 172.21.15.29:0/237913417 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa400b9070 con 0x7ffa500b3140 2024-04-25T00:32:31.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa37fff640 1 -- 172.21.15.29:0/237913417 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa400b4420 con 0x7ffa500b3140 2024-04-25T00:32:31.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 -- 172.21.15.29:0/237913417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa50154240 con 0x7ffa500b3140 2024-04-25T00:32:31.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa37fff640 1 -- 172.21.15.29:0/237913417 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa400b0040 con 0x7ffa500b3140 2024-04-25T00:32:31.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 -- 172.21.15.29:0/237913417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa50135110 con 0x7ffa500b3140 2024-04-25T00:32:31.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.743+0000 7ffa55c97640 1 -- 172.21.15.29:0/237913417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa50069270 con 0x7ffa500b3140 2024-04-25T00:32:31.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.747+0000 7ffa37fff640 1 -- 172.21.15.29:0/237913417 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffa400b33c0 con 0x7ffa500b3140 2024-04-25T00:32:31.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.747+0000 7ffa37fff640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffa3c0678c0 0x7ffa3c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.747+0000 7ffa37fff640 1 -- 172.21.15.29:0/237913417 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7ffa4011eaf0 con 0x7ffa500b3140 2024-04-25T00:32:31.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.747+0000 7ffa4effd640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffa3c0678c0 0x7ffa3c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.747+0000 7ffa4effd640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffa3c0678c0 0x7ffa3c069d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffa44007fd0 tx=0x7ffa44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.751+0000 7ffa37fff640 1 -- 172.21.15.29:0/237913417 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffa40016130 con 0x7ffa500b3140 2024-04-25T00:32:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.907+0000 7ffa55c97640 1 -- 172.21.15.29:0/237913417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ffa50142bd0 con 0x7ffa500b3140 2024-04-25T00:32:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.911+0000 7ffa37fff640 1 -- 172.21.15.29:0/237913417 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7ffa400ef990 con 0x7ffa500b3140 2024-04-25T00:32:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:31.914 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:32:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-04-25T00:32:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": ".mgr" 2024-04-25T00:32:31.914 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:31.914 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:32:31.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.911+0000 7ffa55c97640 1 -- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffa3c0678c0 msgr2=0x7ffa3c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.911+0000 7ffa55c97640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffa3c0678c0 0x7ffa3c069d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffa44007fd0 tx=0x7ffa44009040 comp rx=0 tx=0).stop 2024-04-25T00:32:31.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.911+0000 7ffa55c97640 1 -- 172.21.15.29:0/237913417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa500b3140 msgr2=0x7ffa50152ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.911+0000 7ffa55c97640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa500b3140 0x7ffa50152ba0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffa4001abf0 tx=0x7ffa400192e0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.911+0000 7ffa55c97640 1 -- 172.21.15.29:0/237913417 shutdown_connections 2024-04-25T00:32:31.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.911+0000 7ffa55c97640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffa3c0678c0 0x7ffa3c069d80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.911+0000 7ffa55c97640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa500b4b10 0x7ffa5015ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.911+0000 7ffa55c97640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa500b3c30 0x7ffa5015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.911+0000 7ffa55c97640 1 --2- 172.21.15.29:0/237913417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa500b3140 0x7ffa50152ba0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.911+0000 7ffa55c97640 1 -- 172.21.15.29:0/237913417 >> 172.21.15.29:0/237913417 conn(0x7ffa500b1870 msgr2=0x7ffa500cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.911+0000 7ffa55c97640 1 -- 172.21.15.29:0/237913417 shutdown_connections 2024-04-25T00:32:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:31.911+0000 7ffa55c97640 1 -- 172.21.15.29:0/237913417 wait complete. 2024-04-25T00:32:31.931 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:32:31.932 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:32:32.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f193b5ca640 1 Processor -- start 2024-04-25T00:32:32.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f193b5ca640 1 -- start start 2024-04-25T00:32:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f193b5ca640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19340b3b50 0x7f19340b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f193b5ca640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f193406cd40 0x7f193406d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f193b5ca640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19340cf670 con 0x7f19341456d0 2024-04-25T00:32:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f193b5ca640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f19340b4380 con 0x7f19340b3b50 2024-04-25T00:32:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f193b5ca640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f19340b40a0 con 0x7f193406cd40 2024-04-25T00:32:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f1938b65640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19340b3b50 0x7f19340b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f1938b65640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19340b3b50 0x7f19340b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37500/0 (socket says 172.21.15.29:37500) 2024-04-25T00:32:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f1938b65640 1 -- 172.21.15.29:0/1482829921 learned_addr learned my addr 172.21.15.29:0/1482829921 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f1939b67640 1 --2- 172.21.15.29:0/1482829921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f193406cd40 0x7f193406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f1938b65640 1 -- 172.21.15.29:0/1482829921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f193406cd40 msgr2=0x7f193406d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f1938b65640 1 --2- 172.21.15.29:0/1482829921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f193406cd40 0x7f193406d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f1938b65640 1 -- 172.21.15.29:0/1482829921 >> v1:172.21.15.157:6789/0 conn(0x7f19341456d0 legacy=0x7f19340cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:32.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.043+0000 7f1938b65640 1 -- 172.21.15.29:0/1482829921 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19340ce480 con 0x7f19340b3b50 2024-04-25T00:32:32.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f1938b65640 1 --2- 172.21.15.29:0/1482829921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19340b3b50 0x7f19340b2b70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1924009870 tx=0x7f192401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b2c4c04e5f4b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:32.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f192bfff640 1 -- 172.21.15.29:0/1482829921 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19240b1070 con 0x7f19340b3b50 2024-04-25T00:32:32.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f192bfff640 1 -- 172.21.15.29:0/1482829921 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19240ac630 con 0x7f19340b3b50 2024-04-25T00:32:32.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f192bfff640 1 -- 172.21.15.29:0/1482829921 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19240a8050 con 0x7f19340b3b50 2024-04-25T00:32:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f193b5ca640 1 -- 172.21.15.29:0/1482829921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19340b3b50 msgr2=0x7f19340b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f193b5ca640 1 --2- 172.21.15.29:0/1482829921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19340b3b50 0x7f19340b2b70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1924009870 tx=0x7f192401c520 comp rx=0 tx=0).stop 2024-04-25T00:32:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f193b5ca640 1 -- 172.21.15.29:0/1482829921 shutdown_connections 2024-04-25T00:32:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f193b5ca640 1 --2- 172.21.15.29:0/1482829921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f193406cd40 0x7f193406d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f193b5ca640 1 --2- 172.21.15.29:0/1482829921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19340b3b50 0x7f19340b2b70 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f193b5ca640 1 -- 172.21.15.29:0/1482829921 >> 172.21.15.29:0/1482829921 conn(0x7f19340b1870 msgr2=0x7f19340b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f193b5ca640 1 -- 172.21.15.29:0/1482829921 shutdown_connections 2024-04-25T00:32:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f193b5ca640 1 -- 172.21.15.29:0/1482829921 wait complete. 2024-04-25T00:32:32.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f193b5ca640 1 Processor -- start 2024-04-25T00:32:32.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.047+0000 7f193b5ca640 1 -- start start 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f193b5ca640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f193406cd40 0x7f193415f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f193b5ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19340b3b50 0x7f19340af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f193b5ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19341456d0 0x7f19340afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f193b5ca640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f19340b5000 con 0x7f19341456d0 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f193b5ca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1934145ab0 con 0x7f19340b3b50 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f193b5ca640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1934168b70 con 0x7f193406cd40 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1939b67640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19341456d0 0x7f19340afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1939b67640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19341456d0 0x7f19340afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37504/0 (socket says 172.21.15.29:37504) 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1939b67640 1 -- 172.21.15.29:0/1372598875 learned_addr learned my addr 172.21.15.29:0/1372598875 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1938b65640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19340b3b50 0x7f19340af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1939366640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f193406cd40 0x7f193415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1939b67640 1 -- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f193406cd40 msgr2=0x7f193415f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1939b67640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f193406cd40 0x7f193415f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1939b67640 1 -- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19340b3b50 msgr2=0x7f19340af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:32.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1939b67640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19340b3b50 0x7f19340af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1939b67640 1 -- 172.21.15.29:0/1372598875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193406efc0 con 0x7f19341456d0 2024-04-25T00:32:32.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1939b67640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19341456d0 0x7f19340afd20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f192c007e60 tx=0x7f192c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:32.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1929ffb640 1 -- 172.21.15.29:0/1372598875 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c014420 con 0x7f19341456d0 2024-04-25T00:32:32.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1929ffb640 1 -- 172.21.15.29:0/1372598875 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f192c019070 con 0x7f19341456d0 2024-04-25T00:32:32.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f193b5ca640 1 -- 172.21.15.29:0/1372598875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193413c420 con 0x7f19341456d0 2024-04-25T00:32:32.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1929ffb640 1 -- 172.21.15.29:0/1372598875 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c014820 con 0x7f19341456d0 2024-04-25T00:32:32.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f193b5ca640 1 -- 172.21.15.29:0/1372598875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193413c9e0 con 0x7f19341456d0 2024-04-25T00:32:32.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f193b5ca640 1 -- 172.21.15.29:0/1372598875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f193413c1b0 con 0x7f19341456d0 2024-04-25T00:32:32.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1929ffb640 1 -- 172.21.15.29:0/1372598875 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f192c0149c0 con 0x7f19341456d0 2024-04-25T00:32:32.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1929ffb640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f19180678c0 0x7f1918069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:32.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1929ffb640 1 -- 172.21.15.29:0/1372598875 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f192c086140 con 0x7f19341456d0 2024-04-25T00:32:32.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.051+0000 7f1939366640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f19180678c0 0x7f1918069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:32.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.055+0000 7f1939366640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f19180678c0 0x7f1918069d80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1930004e30 tx=0x7f1930009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:32.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.055+0000 7f1929ffb640 1 -- 172.21.15.29:0/1372598875 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f192c057060 con 0x7f19341456d0 2024-04-25T00:32:32.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.211+0000 7f193b5ca640 1 -- 172.21.15.29:0/1372598875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f19340692e0 con 0x7f19341456d0 2024-04-25T00:32:32.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.383+0000 7f1929ffb640 1 -- 172.21.15.29:0/1372598875 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f192c056960 con 0x7f19341456d0 2024-04-25T00:32:32.389 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-04-25T00:32:32.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.387+0000 7f193b5ca640 1 -- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f19180678c0 msgr2=0x7f1918069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:32.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.387+0000 7f193b5ca640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f19180678c0 0x7f1918069d80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1930004e30 tx=0x7f1930009040 comp rx=0 tx=0).stop 2024-04-25T00:32:32.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.387+0000 7f193b5ca640 1 -- 172.21.15.29:0/1372598875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19341456d0 msgr2=0x7f19340afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:32.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.387+0000 7f193b5ca640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19341456d0 0x7f19340afd20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f192c007e60 tx=0x7f192c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.387+0000 7f193b5ca640 1 -- 172.21.15.29:0/1372598875 shutdown_connections 2024-04-25T00:32:32.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.387+0000 7f193b5ca640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19341456d0 0x7f19340afd20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.387+0000 7f193b5ca640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f19180678c0 0x7f1918069d80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.387+0000 7f193b5ca640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19340b3b50 0x7f19340af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.387+0000 7f193b5ca640 1 --2- 172.21.15.29:0/1372598875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f193406cd40 0x7f193415f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.387+0000 7f193b5ca640 1 -- 172.21.15.29:0/1372598875 >> 172.21.15.29:0/1372598875 conn(0x7f19340b1870 msgr2=0x7f19340b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:32.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.387+0000 7f193b5ca640 1 -- 172.21.15.29:0/1372598875 shutdown_connections 2024-04-25T00:32:32.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.387+0000 7f193b5ca640 1 -- 172.21.15.29:0/1372598875 wait complete. 2024-04-25T00:32:32.416 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:32:32.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.551+0000 7f3865165640 1 Processor -- start 2024-04-25T00:32:32.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.551+0000 7f3865165640 1 -- start start 2024-04-25T00:32:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f3865165640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38600b3140 0x7f386006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f3865165640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38600b3c30 0x7f38600ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f3865165640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38600b41e0 con 0x7f38600b4b10 2024-04-25T00:32:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f3865165640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f386006d960 con 0x7f38600b3c30 2024-04-25T00:32:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f3865165640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f386006dc40 con 0x7f38600b3140 2024-04-25T00:32:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f385f59e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38600b3c30 0x7f38600ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f385f59e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38600b3c30 0x7f38600ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37518/0 (socket says 172.21.15.29:37518) 2024-04-25T00:32:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f385f59e640 1 -- 172.21.15.29:0/1969355231 learned_addr learned my addr 172.21.15.29:0/1969355231 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f385ed9d640 1 --2- 172.21.15.29:0/1969355231 >> v2:172.21.15.157:3300/0 conn(0x7f38600b3140 0x7f386006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f385f59e640 1 -- 172.21.15.29:0/1969355231 >> v2:172.21.15.157:3300/0 conn(0x7f38600b3140 msgr2=0x7f386006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f385f59e640 1 --2- 172.21.15.29:0/1969355231 >> v2:172.21.15.157:3300/0 conn(0x7f38600b3140 0x7f386006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f385f59e640 1 -- 172.21.15.29:0/1969355231 >> v1:172.21.15.157:6789/0 conn(0x7f38600b4b10 legacy=0x7f38600b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f385f59e640 1 -- 172.21.15.29:0/1969355231 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386006df20 con 0x7f38600b3c30 2024-04-25T00:32:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f385f59e640 1 --2- 172.21.15.29:0/1969355231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38600b3c30 0x7f38600ce240 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3858009e50 tx=0x7f385801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43afed75975f107e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f385dd9b640 1 -- 172.21.15.29:0/1969355231 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38580a5670 con 0x7f38600b3c30 2024-04-25T00:32:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f385dd9b640 1 -- 172.21.15.29:0/1969355231 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38580a5810 con 0x7f38600b3c30 2024-04-25T00:32:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.555+0000 7f385dd9b640 1 -- 172.21.15.29:0/1969355231 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38580a5be0 con 0x7f38600b3c30 2024-04-25T00:32:32.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 -- 172.21.15.29:0/1969355231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38600b3c30 msgr2=0x7f38600ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:32.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 --2- 172.21.15.29:0/1969355231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38600b3c30 0x7f38600ce240 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3858009e50 tx=0x7f385801c9b0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 -- 172.21.15.29:0/1969355231 shutdown_connections 2024-04-25T00:32:32.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 --2- 172.21.15.29:0/1969355231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38600b3c30 0x7f38600ce240 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 --2- 172.21.15.29:0/1969355231 >> v2:172.21.15.157:3300/0 conn(0x7f38600b3140 0x7f386006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 -- 172.21.15.29:0/1969355231 >> 172.21.15.29:0/1969355231 conn(0x7f38600b1870 msgr2=0x7f38600b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:32.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 -- 172.21.15.29:0/1969355231 shutdown_connections 2024-04-25T00:32:32.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 -- 172.21.15.29:0/1969355231 wait complete. 2024-04-25T00:32:32.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 Processor -- start 2024-04-25T00:32:32.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 -- start start 2024-04-25T00:32:32.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38600b3140 0x7f38601545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:32.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38600b3c30 0x7f38601584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:32.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38600b4b10 0x7f38601589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:32.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3860168d80 con 0x7f38600b4b10 2024-04-25T00:32:32.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3860168c00 con 0x7f38600b3140 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3860168f00 con 0x7f38600b3c30 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f385f59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38600b4b10 0x7f38601589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f385f59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38600b4b10 0x7f38601589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37532/0 (socket says 172.21.15.29:37532) 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f385f59e640 1 -- 172.21.15.29:0/2797306092 learned_addr learned my addr 172.21.15.29:0/2797306092 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f385e59c640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38600b3c30 0x7f38601584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f385ed9d640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38600b3140 0x7f38601545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f385f59e640 1 -- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38600b3c30 msgr2=0x7f38601584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f385f59e640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38600b3c30 0x7f38601584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f385f59e640 1 -- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38600b3140 msgr2=0x7f38601545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f385f59e640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38600b3140 0x7f38601545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f385f59e640 1 -- 172.21.15.29:0/2797306092 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3860139710 con 0x7f38600b4b10 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f385f59e640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38600b4b10 0x7f38601589e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f385801a5e0 tx=0x7f3858019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f38477fe640 1 -- 172.21.15.29:0/2797306092 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858009510 con 0x7f38600b4b10 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f38477fe640 1 -- 172.21.15.29:0/2797306092 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38580be070 con 0x7f38600b4b10 2024-04-25T00:32:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 -- 172.21.15.29:0/2797306092 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38601399f0 con 0x7f38600b4b10 2024-04-25T00:32:32.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f38477fe640 1 -- 172.21.15.29:0/2797306092 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38580b88b0 con 0x7f38600b4b10 2024-04-25T00:32:32.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.559+0000 7f3865165640 1 -- 172.21.15.29:0/2797306092 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3860139fb0 con 0x7f38600b4b10 2024-04-25T00:32:32.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.563+0000 7f3865165640 1 -- 172.21.15.29:0/2797306092 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3860069270 con 0x7f38600b4b10 2024-04-25T00:32:32.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.563+0000 7f38477fe640 1 -- 172.21.15.29:0/2797306092 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f38580b8a50 con 0x7f38600b4b10 2024-04-25T00:32:32.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.563+0000 7f38477fe640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f384c0678c0 0x7f384c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:32.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.563+0000 7f38477fe640 1 -- 172.21.15.29:0/2797306092 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f3858120c10 con 0x7f38600b4b10 2024-04-25T00:32:32.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.563+0000 7f385ed9d640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f384c0678c0 0x7f384c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:32.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.563+0000 7f385ed9d640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f384c0678c0 0x7f384c069d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f385000b930 tx=0x7f3850008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:32.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.567+0000 7f38477fe640 1 -- 172.21.15.29:0/2797306092 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f38580165a0 con 0x7f38600b4b10 2024-04-25T00:32:32.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:32.735+0000 7f3865165640 1 -- 172.21.15.29:0/2797306092 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3860144330 con 0x7f38600b4b10 2024-04-25T00:32:33.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.391+0000 7f38477fe640 1 -- 172.21.15.29:0/2797306092 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f3858124020 con 0x7f38600b4b10 2024-04-25T00:32:33.394 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-04-25T00:32:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.395+0000 7f3865165640 1 -- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f384c0678c0 msgr2=0x7f384c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.395+0000 7f3865165640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f384c0678c0 0x7f384c069d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f385000b930 tx=0x7f3850008040 comp rx=0 tx=0).stop 2024-04-25T00:32:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.395+0000 7f3865165640 1 -- 172.21.15.29:0/2797306092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38600b4b10 msgr2=0x7f38601589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.395+0000 7f3865165640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38600b4b10 0x7f38601589e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f385801a5e0 tx=0x7f3858019750 comp rx=0 tx=0).stop 2024-04-25T00:32:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.395+0000 7f3865165640 1 -- 172.21.15.29:0/2797306092 shutdown_connections 2024-04-25T00:32:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.395+0000 7f3865165640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38600b4b10 0x7f38601589e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.395+0000 7f3865165640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38600b3c30 0x7f38601584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.395+0000 7f3865165640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f384c0678c0 0x7f384c069d80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.395+0000 7f3865165640 1 --2- 172.21.15.29:0/2797306092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38600b3140 0x7f38601545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.395+0000 7f3865165640 1 -- 172.21.15.29:0/2797306092 >> 172.21.15.29:0/2797306092 conn(0x7f38600b1870 msgr2=0x7f38600b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.395+0000 7f3865165640 1 -- 172.21.15.29:0/2797306092 shutdown_connections 2024-04-25T00:32:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.395+0000 7f3865165640 1 -- 172.21.15.29:0/2797306092 wait complete. 2024-04-25T00:32:33.434 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:32:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9ee1693640 1 Processor -- start 2024-04-25T00:32:33.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9ee1693640 1 -- start start 2024-04-25T00:32:33.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9ee1693640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9edc0b3140 0x7f9edc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:33.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9ee1693640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9edc0b4b10 0x7f9edc0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:33.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9ee1693640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9edc0b3a60 0x7f9edc15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:33.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9ee1693640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9edc0b41e0 con 0x7f9edc0b3a60 2024-04-25T00:32:33.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9ee1693640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9edc06d960 con 0x7f9edc0b4b10 2024-04-25T00:32:33.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9ee1693640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9edc06dc40 con 0x7f9edc0b3140 2024-04-25T00:32:33.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9edb7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9edc0b3a60 0x7f9edc15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:33.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9edb7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9edc0b3a60 0x7f9edc15c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37546/0 (socket says 172.21.15.29:37546) 2024-04-25T00:32:33.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9edb7fe640 1 -- 172.21.15.29:0/3598626336 learned_addr learned my addr 172.21.15.29:0/3598626336 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:33.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9eda7fc640 1 --2- 172.21.15.29:0/3598626336 >> v2:172.21.15.157:3300/0 conn(0x7f9edc0b4b10 0x7f9edc0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:33.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9edaffd640 1 --2- 172.21.15.29:0/3598626336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9edc0b3140 0x7f9edc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:33.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9edb7fe640 1 -- 172.21.15.29:0/3598626336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9edc0b3140 msgr2=0x7f9edc06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:33.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9edb7fe640 1 --2- 172.21.15.29:0/3598626336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9edc0b3140 0x7f9edc06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:33.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9edb7fe640 1 -- 172.21.15.29:0/3598626336 >> v2:172.21.15.157:3300/0 conn(0x7f9edc0b4b10 msgr2=0x7f9edc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:33.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9edb7fe640 1 --2- 172.21.15.29:0/3598626336 >> v2:172.21.15.157:3300/0 conn(0x7f9edc0b4b10 0x7f9edc0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:33.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.559+0000 7f9edb7fe640 1 -- 172.21.15.29:0/3598626336 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc06df20 con 0x7f9edc0b3a60 2024-04-25T00:32:33.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9edb7fe640 1 --2- 172.21.15.29:0/3598626336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9edc0b3a60 0x7f9edc15c3e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9edc074fb0 tx=0x7f9ed401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7212e174a2d5866e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:33.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ed9ffb640 1 -- 172.21.15.29:0/3598626336 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40b0070 con 0x7f9edc0b3a60 2024-04-25T00:32:33.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ed9ffb640 1 -- 172.21.15.29:0/3598626336 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed40ab600 con 0x7f9edc0b3a60 2024-04-25T00:32:33.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ed9ffb640 1 -- 172.21.15.29:0/3598626336 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40ab9d0 con 0x7f9edc0b3a60 2024-04-25T00:32:33.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ee1693640 1 -- 172.21.15.29:0/3598626336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9edc0b3a60 msgr2=0x7f9edc15c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:33.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ee1693640 1 --2- 172.21.15.29:0/3598626336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9edc0b3a60 0x7f9edc15c3e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9edc074fb0 tx=0x7f9ed401c520 comp rx=0 tx=0).stop 2024-04-25T00:32:33.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ee1693640 1 -- 172.21.15.29:0/3598626336 shutdown_connections 2024-04-25T00:32:33.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ee1693640 1 --2- 172.21.15.29:0/3598626336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9edc0b3a60 0x7f9edc15c3e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:33.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ee1693640 1 --2- 172.21.15.29:0/3598626336 >> v2:172.21.15.157:3300/0 conn(0x7f9edc0b4b10 0x7f9edc0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:33.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ee1693640 1 --2- 172.21.15.29:0/3598626336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9edc0b3140 0x7f9edc06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:33.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ee1693640 1 -- 172.21.15.29:0/3598626336 >> 172.21.15.29:0/3598626336 conn(0x7f9edc0b1870 msgr2=0x7f9edc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:33.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ee1693640 1 -- 172.21.15.29:0/3598626336 shutdown_connections 2024-04-25T00:32:33.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ee1693640 1 -- 172.21.15.29:0/3598626336 wait complete. 2024-04-25T00:32:33.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ee1693640 1 Processor -- start 2024-04-25T00:32:33.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.563+0000 7f9ee1693640 1 -- start start 2024-04-25T00:32:33.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9ee1693640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9edc0b3140 0x7f9edc1313d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:33.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9ee1693640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9edc0b3a60 0x7f9edc157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:33.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9ee1693640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9edc0b4b10 0x7f9edc157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:33.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9ee1693640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc0ce3c0 con 0x7f9edc0b3a60 2024-04-25T00:32:33.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9ee1693640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc0ce240 con 0x7f9edc0b3140 2024-04-25T00:32:33.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9ee1693640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9edc0ce540 con 0x7f9edc0b4b10 2024-04-25T00:32:33.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9eda7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9edc0b3a60 0x7f9edc157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:33.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9eda7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9edc0b3a60 0x7f9edc157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37556/0 (socket says 172.21.15.29:37556) 2024-04-25T00:32:33.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9eda7fc640 1 -- 172.21.15.29:0/3257214473 learned_addr learned my addr 172.21.15.29:0/3257214473 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:33.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9edb7fe640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9edc0b4b10 0x7f9edc157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9edaffd640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9edc0b3140 0x7f9edc1313d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9eda7fc640 1 -- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9edc0b4b10 msgr2=0x7f9edc157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9eda7fc640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9edc0b4b10 0x7f9edc157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9eda7fc640 1 -- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9edc0b3140 msgr2=0x7f9edc1313d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9eda7fc640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9edc0b3140 0x7f9edc1313d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9eda7fc640 1 -- 172.21.15.29:0/3257214473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc13a0d0 con 0x7f9edc0b3a60 2024-04-25T00:32:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9edaffd640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9edc0b3140 0x7f9edc1313d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9eda7fc640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9edc0b3a60 0x7f9edc157a70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0007e40 tx=0x7f9ed000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9ec37fe640 1 -- 172.21.15.29:0/3257214473 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0014620 con 0x7f9edc0b3a60 2024-04-25T00:32:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9ec37fe640 1 -- 172.21.15.29:0/3257214473 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed0019070 con 0x7f9edc0b3a60 2024-04-25T00:32:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9ec37fe640 1 -- 172.21.15.29:0/3257214473 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0014a00 con 0x7f9edc0b3a60 2024-04-25T00:32:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9ee1693640 1 -- 172.21.15.29:0/3257214473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9edc13a3b0 con 0x7f9edc0b3a60 2024-04-25T00:32:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9ee1693640 1 -- 172.21.15.29:0/3257214473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9edc13a970 con 0x7f9edc0b3a60 2024-04-25T00:32:33.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9ee1693640 1 -- 172.21.15.29:0/3257214473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9edc069270 con 0x7f9edc0b3a60 2024-04-25T00:32:33.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.567+0000 7f9ec37fe640 1 -- 172.21.15.29:0/3257214473 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9ed0014ba0 con 0x7f9edc0b3a60 2024-04-25T00:32:33.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.571+0000 7f9ec37fe640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9ec80678c0 0x7f9ec8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:33.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.571+0000 7f9ec37fe640 1 -- 172.21.15.29:0/3257214473 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f9ed0086560 con 0x7f9edc0b3a60 2024-04-25T00:32:33.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.571+0000 7f9edaffd640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9ec80678c0 0x7f9ec8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:33.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.571+0000 7f9edaffd640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9ec80678c0 0x7f9ec8069d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9edc1f8110 tx=0x7f9ecc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:33.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.575+0000 7f9ec37fe640 1 -- 172.21.15.29:0/3257214473 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9ed0057060 con 0x7f9edc0b3a60 2024-04-25T00:32:33.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:33.771+0000 7f9ee1693640 1 -- 172.21.15.29:0/3257214473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9edc0d0380 con 0x7f9edc0b3a60 2024-04-25T00:32:34.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.423+0000 7f9ec37fe640 1 -- 172.21.15.29:0/3257214473 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) ==== 172+0+0 (secure 0 0 0) 0x7f9ed0056960 con 0x7f9edc0b3a60 2024-04-25T00:32:34.426 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T00:32:34.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.427+0000 7f9ee1693640 1 -- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9ec80678c0 msgr2=0x7f9ec8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:34.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.427+0000 7f9ee1693640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9ec80678c0 0x7f9ec8069d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9edc1f8110 tx=0x7f9ecc008040 comp rx=0 tx=0).stop 2024-04-25T00:32:34.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.427+0000 7f9ee1693640 1 -- 172.21.15.29:0/3257214473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9edc0b3a60 msgr2=0x7f9edc157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:34.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.427+0000 7f9ee1693640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9edc0b3a60 0x7f9edc157a70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0007e40 tx=0x7f9ed000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.427+0000 7f9ee1693640 1 -- 172.21.15.29:0/3257214473 shutdown_connections 2024-04-25T00:32:34.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.427+0000 7f9ee1693640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9edc0b4b10 0x7f9edc157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.427+0000 7f9ee1693640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9edc0b3a60 0x7f9edc157a70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.427+0000 7f9ee1693640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9ec80678c0 0x7f9ec8069d80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.427+0000 7f9ee1693640 1 --2- 172.21.15.29:0/3257214473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9edc0b3140 0x7f9edc1313d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.427+0000 7f9ee1693640 1 -- 172.21.15.29:0/3257214473 >> 172.21.15.29:0/3257214473 conn(0x7f9edc0b1870 msgr2=0x7f9edc15d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:34.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.431+0000 7f9ee1693640 1 -- 172.21.15.29:0/3257214473 shutdown_connections 2024-04-25T00:32:34.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.431+0000 7f9ee1693640 1 -- 172.21.15.29:0/3257214473 wait complete. 2024-04-25T00:32:34.474 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:34.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3f91c640 1 Processor -- start 2024-04-25T00:32:34.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3f91c640 1 -- start start 2024-04-25T00:32:34.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3f91c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc3815e200 0x7fcc380d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:34.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3f91c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcc380cf690 0x7fcc380d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:34.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3f91c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc380d2ea0 con 0x7fcc380d2a90 2024-04-25T00:32:34.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3f91c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fcc380b38d0 con 0x7fcc3815e200 2024-04-25T00:32:34.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3f91c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcc380cfc10 con 0x7fcc380cf690 2024-04-25T00:32:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3d6b8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc3815e200 0x7fcc380d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3deb9640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcc380d2a90 0x7fcc380d05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37240/0 (socket says 172.21.15.29:37240) 2024-04-25T00:32:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3d6b8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc3815e200 0x7fcc380d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37564/0 (socket says 172.21.15.29:37564) 2024-04-25T00:32:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3deb9640 1 -- 172.21.15.29:0/1042283314 learned_addr learned my addr 172.21.15.29:0/1042283314 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3ceb7640 1 --2- 172.21.15.29:0/1042283314 >> v2:172.21.15.157:3300/0 conn(0x7fcc380cf690 0x7fcc380d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3d6b8640 1 -- 172.21.15.29:0/1042283314 >> v2:172.21.15.157:3300/0 conn(0x7fcc380cf690 msgr2=0x7fcc380d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3d6b8640 1 --2- 172.21.15.29:0/1042283314 >> v2:172.21.15.157:3300/0 conn(0x7fcc380cf690 0x7fcc380d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3d6b8640 1 -- 172.21.15.29:0/1042283314 >> v1:172.21.15.29:6789/0 conn(0x7fcc380d2a90 legacy=0x7fcc380d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc2ffff640 1 -- 172.21.15.29:0/1042283314 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2420946250 0 0) 0x7fcc300014a0 con 0x7fcc380d2a90 2024-04-25T00:32:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3d6b8640 1 -- 172.21.15.29:0/1042283314 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc380cfef0 con 0x7fcc3815e200 2024-04-25T00:32:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.587+0000 7fcc3ceb7640 1 --2- 172.21.15.29:0/1042283314 >> v2:172.21.15.157:3300/0 conn(0x7fcc380cf690 0x7fcc380d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:34.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.591+0000 7fcc3d6b8640 1 --2- 172.21.15.29:0/1042283314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815e200 0x7fcc380d47f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcc380d24a0 tx=0x7fcc3401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80dfda844d8905e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:34.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.591+0000 7fcc2ffff640 1 -- 172.21.15.29:0/1042283314 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc340ab650 con 0x7fcc3815e200 2024-04-25T00:32:34.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.591+0000 7fcc2ffff640 1 -- 172.21.15.29:0/1042283314 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc340b0070 con 0x7fcc3815e200 2024-04-25T00:32:34.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.591+0000 7fcc2ffff640 1 -- 172.21.15.29:0/1042283314 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc340aba10 con 0x7fcc3815e200 2024-04-25T00:32:34.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.591+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1042283314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815e200 msgr2=0x7fcc380d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:34.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.591+0000 7fcc3f91c640 1 --2- 172.21.15.29:0/1042283314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815e200 0x7fcc380d47f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcc380d24a0 tx=0x7fcc3401c450 comp rx=0 tx=0).stop 2024-04-25T00:32:34.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.591+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1042283314 shutdown_connections 2024-04-25T00:32:34.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.591+0000 7fcc3f91c640 1 --2- 172.21.15.29:0/1042283314 >> v2:172.21.15.157:3300/0 conn(0x7fcc380cf690 0x7fcc380d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.591+0000 7fcc3f91c640 1 --2- 172.21.15.29:0/1042283314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815e200 0x7fcc380d47f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.591+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1042283314 >> 172.21.15.29:0/1042283314 conn(0x7fcc3809d280 msgr2=0x7fcc380d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:34.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.591+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1042283314 shutdown_connections 2024-04-25T00:32:34.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.591+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1042283314 wait complete. 2024-04-25T00:32:34.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3f91c640 1 Processor -- start 2024-04-25T00:32:34.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3f91c640 1 -- start start 2024-04-25T00:32:34.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3f91c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380cf690 0x7fcc380cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:34.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3f91c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380d2a90 0x7fcc38157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:34.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3f91c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815e200 0x7fcc380b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:34.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3f91c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3815e760 con 0x7fcc380cf690 2024-04-25T00:32:34.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3f91c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3815e5e0 con 0x7fcc3815e200 2024-04-25T00:32:34.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3f91c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3816ae30 con 0x7fcc380d2a90 2024-04-25T00:32:34.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3d6b8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380cf690 0x7fcc380cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:34.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3d6b8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380cf690 0x7fcc380cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37580/0 (socket says 172.21.15.29:37580) 2024-04-25T00:32:34.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3d6b8640 1 -- 172.21.15.29:0/1631754840 learned_addr learned my addr 172.21.15.29:0/1631754840 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3deb9640 1 --2- 172.21.15.29:0/1631754840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815e200 0x7fcc380b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3d6b8640 1 -- 172.21.15.29:0/1631754840 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380d2a90 msgr2=0x7fcc38157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3d6b8640 1 --2- 172.21.15.29:0/1631754840 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380d2a90 0x7fcc38157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3d6b8640 1 -- 172.21.15.29:0/1631754840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815e200 msgr2=0x7fcc380b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3d6b8640 1 --2- 172.21.15.29:0/1631754840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815e200 0x7fcc380b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3d6b8640 1 -- 172.21.15.29:0/1631754840 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc380c9e50 con 0x7fcc380cf690 2024-04-25T00:32:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3d6b8640 1 --2- 172.21.15.29:0/1631754840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380cf690 0x7fcc380cb040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcc34019060 tx=0x7fcc3401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc2dffb640 1 -- 172.21.15.29:0/1631754840 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34004220 con 0x7fcc380cf690 2024-04-25T00:32:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc2dffb640 1 -- 172.21.15.29:0/1631754840 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc340043c0 con 0x7fcc380cf690 2024-04-25T00:32:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1631754840 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc380c8db0 con 0x7fcc380cf690 2024-04-25T00:32:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc2dffb640 1 -- 172.21.15.29:0/1631754840 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34003820 con 0x7fcc380cf690 2024-04-25T00:32:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1631754840 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc380c9370 con 0x7fcc380cf690 2024-04-25T00:32:34.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1631754840 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc38069270 con 0x7fcc380cf690 2024-04-25T00:32:34.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc2dffb640 1 -- 172.21.15.29:0/1631754840 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcc340039c0 con 0x7fcc380cf690 2024-04-25T00:32:34.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc2dffb640 1 --2- 172.21.15.29:0/1631754840 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc1c0678f0 0x7fcc1c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:34.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.595+0000 7fcc2dffb640 1 -- 172.21.15.29:0/1631754840 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fcc341204a0 con 0x7fcc380cf690 2024-04-25T00:32:34.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.599+0000 7fcc3ceb7640 1 --2- 172.21.15.29:0/1631754840 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc1c0678f0 0x7fcc1c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:34.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.599+0000 7fcc3ceb7640 1 --2- 172.21.15.29:0/1631754840 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc1c0678f0 0x7fcc1c069db0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcc380b4fe0 tx=0x7fcc28008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.603+0000 7fcc2dffb640 1 -- 172.21.15.29:0/1631754840 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcc34016070 con 0x7fcc380cf690 2024-04-25T00:32:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.763+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1631754840 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc380cef90 con 0x7fcc380cf690 2024-04-25T00:32:34.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.767+0000 7fcc2dffb640 1 -- 172.21.15.29:0/1631754840 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16634 (secure 0 0 0) 0x7fcc340f1070 con 0x7fcc380cf690 2024-04-25T00:32:34.772 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:34.772 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":32,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:32:33.777299+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:32:32.254456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:32:32.779372+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:34.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.771+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1631754840 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc1c0678f0 msgr2=0x7fcc1c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:34.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.771+0000 7fcc3f91c640 1 --2- 172.21.15.29:0/1631754840 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc1c0678f0 0x7fcc1c069db0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcc380b4fe0 tx=0x7fcc28008040 comp rx=0 tx=0).stop 2024-04-25T00:32:34.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.771+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1631754840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380cf690 msgr2=0x7fcc380cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:34.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.771+0000 7fcc3f91c640 1 --2- 172.21.15.29:0/1631754840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380cf690 0x7fcc380cb040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcc34019060 tx=0x7fcc3401ac90 comp rx=0 tx=0).stop 2024-04-25T00:32:34.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.771+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1631754840 shutdown_connections 2024-04-25T00:32:34.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.771+0000 7fcc3f91c640 1 --2- 172.21.15.29:0/1631754840 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc1c0678f0 0x7fcc1c069db0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.771+0000 7fcc3f91c640 1 --2- 172.21.15.29:0/1631754840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815e200 0x7fcc380b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.771+0000 7fcc3f91c640 1 --2- 172.21.15.29:0/1631754840 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380d2a90 0x7fcc38157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.771+0000 7fcc3f91c640 1 --2- 172.21.15.29:0/1631754840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380cf690 0x7fcc380cb040 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.771+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1631754840 >> 172.21.15.29:0/1631754840 conn(0x7fcc3809d280 msgr2=0x7fcc3815d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:34.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.771+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1631754840 shutdown_connections 2024-04-25T00:32:34.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.771+0000 7fcc3f91c640 1 -- 172.21.15.29:0/1631754840 wait complete. 2024-04-25T00:32:34.791 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:34.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf4752b640 1 Processor -- start 2024-04-25T00:32:34.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf4752b640 1 -- start start 2024-04-25T00:32:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf4752b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf400b3140 0x7faf4006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf4752b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faf400b4b10 0x7faf400b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf4752b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf400b3a60 0x7faf4015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf4752b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7faf400b41e0 con 0x7faf400b4b10 2024-04-25T00:32:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf4752b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faf4006d960 con 0x7faf400b3140 2024-04-25T00:32:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf4752b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faf4006dc40 con 0x7faf400b3a60 2024-04-25T00:32:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf44ac6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faf400b4b10 0x7faf400b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf44ac6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faf400b4b10 0x7faf400b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37584/0 (socket says 172.21.15.29:37584) 2024-04-25T00:32:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf44ac6640 1 -- 172.21.15.29:0/3446473766 learned_addr learned my addr 172.21.15.29:0/3446473766 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf452c7640 1 --2- 172.21.15.29:0/3446473766 >> v2:172.21.15.157:3300/0 conn(0x7faf400b3140 0x7faf4006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf45ac8640 1 --2- 172.21.15.29:0/3446473766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf400b3a60 0x7faf4015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf44ac6640 1 -- 172.21.15.29:0/3446473766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf400b3a60 msgr2=0x7faf4015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf44ac6640 1 --2- 172.21.15.29:0/3446473766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf400b3a60 0x7faf4015c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf44ac6640 1 -- 172.21.15.29:0/3446473766 >> v2:172.21.15.157:3300/0 conn(0x7faf400b3140 msgr2=0x7faf4006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf44ac6640 1 --2- 172.21.15.29:0/3446473766 >> v2:172.21.15.157:3300/0 conn(0x7faf400b3140 0x7faf4006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.911+0000 7faf44ac6640 1 -- 172.21.15.29:0/3446473766 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4006df20 con 0x7faf400b4b10 2024-04-25T00:32:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf44ac6640 1 --2- 172.21.15.29:0/3446473766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf400b4b10 0x7faf400b3520 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faf30009870 tx=0x7faf3001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3dc4c3718b1e8c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf37fff640 1 -- 172.21.15.29:0/3446473766 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf300b0070 con 0x7faf400b4b10 2024-04-25T00:32:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf37fff640 1 -- 172.21.15.29:0/3446473766 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf300ab600 con 0x7faf400b4b10 2024-04-25T00:32:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf37fff640 1 -- 172.21.15.29:0/3446473766 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf300ab9d0 con 0x7faf400b4b10 2024-04-25T00:32:34.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 -- 172.21.15.29:0/3446473766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf400b4b10 msgr2=0x7faf400b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:34.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 --2- 172.21.15.29:0/3446473766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf400b4b10 0x7faf400b3520 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faf30009870 tx=0x7faf3001c520 comp rx=0 tx=0).stop 2024-04-25T00:32:34.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 -- 172.21.15.29:0/3446473766 shutdown_connections 2024-04-25T00:32:34.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 --2- 172.21.15.29:0/3446473766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf400b4b10 0x7faf400b3520 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 --2- 172.21.15.29:0/3446473766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf400b3a60 0x7faf4015c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 --2- 172.21.15.29:0/3446473766 >> v2:172.21.15.157:3300/0 conn(0x7faf400b3140 0x7faf4006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 -- 172.21.15.29:0/3446473766 >> 172.21.15.29:0/3446473766 conn(0x7faf400b1870 msgr2=0x7faf400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:34.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 -- 172.21.15.29:0/3446473766 shutdown_connections 2024-04-25T00:32:34.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 -- 172.21.15.29:0/3446473766 wait complete. 2024-04-25T00:32:34.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 Processor -- start 2024-04-25T00:32:34.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 -- start start 2024-04-25T00:32:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf400b3140 0x7faf4013f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf400b3a60 0x7faf401418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf400b4b10 0x7faf40141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faf400ce3c0 con 0x7faf400b3140 2024-04-25T00:32:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faf400ce240 con 0x7faf400b3a60 2024-04-25T00:32:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf4752b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faf400ce540 con 0x7faf400b4b10 2024-04-25T00:32:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf452c7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf400b3140 0x7faf4013f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf452c7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf400b3140 0x7faf4013f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37586/0 (socket says 172.21.15.29:37586) 2024-04-25T00:32:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf452c7640 1 -- 172.21.15.29:0/227466404 learned_addr learned my addr 172.21.15.29:0/227466404 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf44ac6640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf400b3a60 0x7faf401418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf45ac8640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf400b4b10 0x7faf40141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf452c7640 1 -- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf400b4b10 msgr2=0x7faf40141e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf452c7640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf400b4b10 0x7faf40141e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf452c7640 1 -- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf400b3a60 msgr2=0x7faf401418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf452c7640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf400b3a60 0x7faf401418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.915+0000 7faf452c7640 1 -- 172.21.15.29:0/227466404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf40154a80 con 0x7faf400b3140 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf45ac8640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf400b4b10 0x7faf40141e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf44ac6640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf400b3a60 0x7faf401418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf452c7640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf400b3140 0x7faf4013f000 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00d830 tx=0x7faf3c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf35ffb640 1 -- 172.21.15.29:0/227466404 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c018020 con 0x7faf400b3140 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf35ffb640 1 -- 172.21.15.29:0/227466404 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf3c0096a0 con 0x7faf400b3140 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf4752b640 1 -- 172.21.15.29:0/227466404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf40135c50 con 0x7faf400b3140 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf35ffb640 1 -- 172.21.15.29:0/227466404 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c009a50 con 0x7faf400b3140 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf4752b640 1 -- 172.21.15.29:0/227466404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4013d420 con 0x7faf400b3140 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf4752b640 1 -- 172.21.15.29:0/227466404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf40069270 con 0x7faf400b3140 2024-04-25T00:32:34.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf35ffb640 1 -- 172.21.15.29:0/227466404 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faf3c01b020 con 0x7faf400b3140 2024-04-25T00:32:34.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf35ffb640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faf240678f0 0x7faf24069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:34.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf35ffb640 1 -- 172.21.15.29:0/227466404 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7faf3c0864b0 con 0x7faf400b3140 2024-04-25T00:32:34.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf44ac6640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faf240678f0 0x7faf24069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:34.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.919+0000 7faf44ac6640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faf240678f0 0x7faf24069db0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faf300ad000 tx=0x7faf300a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:34.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:34.923+0000 7faf35ffb640 1 -- 172.21.15.29:0/227466404 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faf3c022020 con 0x7faf400b3140 2024-04-25T00:32:35.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.087+0000 7faf4752b640 1 -- 172.21.15.29:0/227466404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf40155420 con 0x7faf400b3140 2024-04-25T00:32:35.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.087+0000 7faf35ffb640 1 -- 172.21.15.29:0/227466404 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16634 (secure 0 0 0) 0x7faf3c011340 con 0x7faf400b3140 2024-04-25T00:32:35.092 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:35.092 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":32,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:32:33.777299+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:32:32.254456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:32:32.779372+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:35.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.091+0000 7faf4752b640 1 -- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faf240678f0 msgr2=0x7faf24069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:35.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.091+0000 7faf4752b640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faf240678f0 0x7faf24069db0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faf300ad000 tx=0x7faf300a6000 comp rx=0 tx=0).stop 2024-04-25T00:32:35.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.091+0000 7faf4752b640 1 -- 172.21.15.29:0/227466404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf400b3140 msgr2=0x7faf4013f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:35.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.091+0000 7faf4752b640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf400b3140 0x7faf4013f000 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00d830 tx=0x7faf3c00dcf0 comp rx=0 tx=0).stop 2024-04-25T00:32:35.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.091+0000 7faf4752b640 1 -- 172.21.15.29:0/227466404 shutdown_connections 2024-04-25T00:32:35.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.091+0000 7faf4752b640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf400b4b10 0x7faf40141e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:35.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.091+0000 7faf4752b640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faf240678f0 0x7faf24069db0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:35.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.091+0000 7faf4752b640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf400b3a60 0x7faf401418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:35.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.091+0000 7faf4752b640 1 --2- 172.21.15.29:0/227466404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf400b3140 0x7faf4013f000 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:35.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.091+0000 7faf4752b640 1 -- 172.21.15.29:0/227466404 >> 172.21.15.29:0/227466404 conn(0x7faf400b1870 msgr2=0x7faf40140ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:35.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.091+0000 7faf4752b640 1 -- 172.21.15.29:0/227466404 shutdown_connections 2024-04-25T00:32:35.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.091+0000 7faf4752b640 1 -- 172.21.15.29:0/227466404 wait complete. 2024-04-25T00:32:35.112 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:32:35.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.239+0000 7f73427c4640 1 Processor -- start 2024-04-25T00:32:35.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.239+0000 7f73427c4640 1 -- start start 2024-04-25T00:32:35.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.239+0000 7f73427c4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f733c0b8790 0x7f733c1692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:35.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.239+0000 7f73427c4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f733c0d2cd0 con 0x7f733c0d4bf0 2024-04-25T00:32:35.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.239+0000 7f73427c4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f733c0cfe30 con 0x7f733c15e4e0 2024-04-25T00:32:35.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.239+0000 7f73427c4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f733c0b6f80 con 0x7f733c0b8790 2024-04-25T00:32:35.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.239+0000 7f7340d61640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f733c0b8790 0x7f733c1692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:35.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.239+0000 7f73337fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f733c0d4bf0 0x7f733c0d25c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37254/0 (socket says 172.21.15.29:37254) 2024-04-25T00:32:35.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.239+0000 7f73337fe640 1 -- 172.21.15.29:0/902098244 learned_addr learned my addr 172.21.15.29:0/902098244 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:35.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.239+0000 7f7340d61640 1 -- 172.21.15.29:0/902098244 >> v1:172.21.15.157:6789/0 conn(0x7f733c15e4e0 legacy=0x7f733c0d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:35.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.239+0000 7f7340d61640 1 -- 172.21.15.29:0/902098244 >> v1:172.21.15.29:6789/0 conn(0x7f733c0d4bf0 legacy=0x7f733c0d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:35.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.239+0000 7f7340d61640 1 -- 172.21.15.29:0/902098244 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c0b7260 con 0x7f733c0b8790 2024-04-25T00:32:35.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.243+0000 7f7340d61640 1 --2- 172.21.15.29:0/902098244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733c0b8790 0x7f733c1692c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7334009d90 tx=0x7f733401f120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ade648cd1d58f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:35.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.243+0000 7f733b7fe640 1 -- 172.21.15.29:0/902098244 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340b4070 con 0x7f733c0b8790 2024-04-25T00:32:35.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.243+0000 7f733b7fe640 1 -- 172.21.15.29:0/902098244 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73340af600 con 0x7f733c0b8790 2024-04-25T00:32:35.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.243+0000 7f733b7fe640 1 -- 172.21.15.29:0/902098244 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340af9d0 con 0x7f733c0b8790 2024-04-25T00:32:35.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.243+0000 7f73427c4640 1 -- 172.21.15.29:0/902098244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733c0b8790 msgr2=0x7f733c1692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:35.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.243+0000 7f73427c4640 1 --2- 172.21.15.29:0/902098244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733c0b8790 0x7f733c1692c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7334009d90 tx=0x7f733401f120 comp rx=0 tx=0).stop 2024-04-25T00:32:35.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.243+0000 7f73427c4640 1 -- 172.21.15.29:0/902098244 shutdown_connections 2024-04-25T00:32:35.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.243+0000 7f73427c4640 1 --2- 172.21.15.29:0/902098244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733c0b8790 0x7f733c1692c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:35.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.243+0000 7f73427c4640 1 -- 172.21.15.29:0/902098244 >> 172.21.15.29:0/902098244 conn(0x7f733c0d3fb0 msgr2=0x7f733c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:35.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.243+0000 7f73427c4640 1 -- 172.21.15.29:0/902098244 shutdown_connections 2024-04-25T00:32:35.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.243+0000 7f73427c4640 1 -- 172.21.15.29:0/902098244 wait complete. 2024-04-25T00:32:35.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.243+0000 7f73427c4640 1 Processor -- start 2024-04-25T00:32:35.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.243+0000 7f73427c4640 1 -- start start 2024-04-25T00:32:35.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73427c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f733c0b8790 0x7f733c159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:35.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73427c4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733c0d4bf0 0x7f733c153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:35.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73427c4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733c15e4e0 0x7f733c1583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:35.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73427c4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f733c15ea40 con 0x7f733c15e4e0 2024-04-25T00:32:35.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73427c4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f733c15e8c0 con 0x7f733c0b8790 2024-04-25T00:32:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73427c4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f733c15ebc0 con 0x7f733c0d4bf0 2024-04-25T00:32:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f7340d61640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733c15e4e0 0x7f733c1583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f7340d61640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733c15e4e0 0x7f733c1583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37616/0 (socket says 172.21.15.29:37616) 2024-04-25T00:32:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f7340d61640 1 -- 172.21.15.29:0/3297125753 learned_addr learned my addr 172.21.15.29:0/3297125753 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73337fe640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733c0d4bf0 0x7f733c153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f733bfff640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f733c0b8790 0x7f733c159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f7340d61640 1 -- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733c0d4bf0 msgr2=0x7f733c153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f7340d61640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733c0d4bf0 0x7f733c153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f7340d61640 1 -- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f733c0b8790 msgr2=0x7f733c159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f7340d61640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f733c0b8790 0x7f733c159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f7340d61640 1 -- 172.21.15.29:0/3297125753 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c15abc0 con 0x7f733c15e4e0 2024-04-25T00:32:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f7340d61640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733c15e4e0 0x7f733c1583e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7334009d90 tx=0x7f733401bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73397fa640 1 -- 172.21.15.29:0/3297125753 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340b4040 con 0x7f733c15e4e0 2024-04-25T00:32:35.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73397fa640 1 -- 172.21.15.29:0/3297125753 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73340c1070 con 0x7f733c15e4e0 2024-04-25T00:32:35.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73397fa640 1 -- 172.21.15.29:0/3297125753 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340ba6b0 con 0x7f733c15e4e0 2024-04-25T00:32:35.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73427c4640 1 -- 172.21.15.29:0/3297125753 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733c15a600 con 0x7f733c15e4e0 2024-04-25T00:32:35.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73427c4640 1 -- 172.21.15.29:0/3297125753 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733c0c7bd0 con 0x7f733c15e4e0 2024-04-25T00:32:35.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73427c4640 1 -- 172.21.15.29:0/3297125753 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f733c069270 con 0x7f733c15e4e0 2024-04-25T00:32:35.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.247+0000 7f73397fa640 1 -- 172.21.15.29:0/3297125753 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f73340ba850 con 0x7f733c15e4e0 2024-04-25T00:32:35.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.251+0000 7f73397fa640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f73200678f0 0x7f7320069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:35.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.251+0000 7f73397fa640 1 -- 172.21.15.29:0/3297125753 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f73341247d0 con 0x7f733c15e4e0 2024-04-25T00:32:35.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.251+0000 7f733bfff640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f73200678f0 0x7f7320069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:35.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.251+0000 7f733bfff640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f73200678f0 0x7f7320069db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f733c0caf80 tx=0x7f7324008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:35.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.255+0000 7f73397fa640 1 -- 172.21.15.29:0/3297125753 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7334018c80 con 0x7f733c15e4e0 2024-04-25T00:32:35.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:35.451+0000 7f73427c4640 1 -- 172.21.15.29:0/3297125753 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f733c161d70 con 0x7f733c15e4e0 2024-04-25T00:32:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.463+0000 7f73397fa640 1 -- 172.21.15.29:0/3297125753 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) ==== 118+0+0 (secure 0 0 0) 0x7f73340f5330 con 0x7f733c15e4e0 2024-04-25T00:32:36.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.467+0000 7f73427c4640 1 -- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f73200678f0 msgr2=0x7f7320069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:36.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.467+0000 7f73427c4640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f73200678f0 0x7f7320069db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f733c0caf80 tx=0x7f7324008040 comp rx=0 tx=0).stop 2024-04-25T00:32:36.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.467+0000 7f73427c4640 1 -- 172.21.15.29:0/3297125753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733c15e4e0 msgr2=0x7f733c1583e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:36.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.467+0000 7f73427c4640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733c15e4e0 0x7f733c1583e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7334009d90 tx=0x7f733401bec0 comp rx=0 tx=0).stop 2024-04-25T00:32:36.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.467+0000 7f73427c4640 1 -- 172.21.15.29:0/3297125753 shutdown_connections 2024-04-25T00:32:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.467+0000 7f73427c4640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733c15e4e0 0x7f733c1583e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.467+0000 7f73427c4640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733c0d4bf0 0x7f733c153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.467+0000 7f73427c4640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f73200678f0 0x7f7320069db0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.467+0000 7f73427c4640 1 --2- 172.21.15.29:0/3297125753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f733c0b8790 0x7f733c159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.467+0000 7f73427c4640 1 -- 172.21.15.29:0/3297125753 >> 172.21.15.29:0/3297125753 conn(0x7f733c0d3fb0 msgr2=0x7f733c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.467+0000 7f73427c4640 1 -- 172.21.15.29:0/3297125753 shutdown_connections 2024-04-25T00:32:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.467+0000 7f73427c4640 1 -- 172.21.15.29:0/3297125753 wait complete. 2024-04-25T00:32:36.490 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:32:36.490 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:32:36.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f3011495640 1 Processor -- start 2024-04-25T00:32:36.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f3011495640 1 -- start start 2024-04-25T00:32:36.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f3011495640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f300c15e200 0x7f300c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:36.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f3011495640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f300c0cfc10 con 0x7f300c0d2c60 2024-04-25T00:32:36.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f3011495640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f300c0cfef0 con 0x7f300c0cf690 2024-04-25T00:32:36.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f3011495640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f300c0b38d0 con 0x7f300c15e200 2024-04-25T00:32:36.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300b7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f300c0d2c60 0x7f300c0b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37260/0 (socket says 172.21.15.29:37260) 2024-04-25T00:32:36.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300b7fe640 1 -- 172.21.15.29:0/1303619158 learned_addr learned my addr 172.21.15.29:0/1303619158 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:36.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300affd640 1 --2- 172.21.15.29:0/1303619158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f300c15e200 0x7f300c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:36.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1805969158 0 0) 0x7f300c0cfc10 con 0x7f300c0d2c60 2024-04-25T00:32:36.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ff8003530 con 0x7f300c0d2c60 2024-04-25T00:32:36.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2467416589 0 0) 0x7f2ff8003530 con 0x7f300c0d2c60 2024-04-25T00:32:36.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f300c15e200 msgr2=0x7f300c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:36.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 --2- 172.21.15.29:0/1303619158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f300c15e200 0x7f300c0d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:36.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 >> v1:172.21.15.157:6789/0 conn(0x7f300c0cf690 legacy=0x7f300c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:36.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c0b3b20 con 0x7f300c0d2c60 2024-04-25T00:32:36.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f3004001a00 con 0x7f300c0d2c60 2024-04-25T00:32:36.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 >> v1:172.21.15.29:6789/0 conn(0x7f300c0d2c60 legacy=0x7f300c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:36.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 --2- 172.21.15.29:0/1303619158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ff80039f0 0x7f2ff8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:36.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 --2- 172.21.15.29:0/1303619158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff8004810 0x7f2ff8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:36.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 --2- 172.21.15.29:0/1303619158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f300c15e200 0x7f2ff80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:36.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f300c0b38d0 con 0x7f2ff80039f0 2024-04-25T00:32:36.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff800ba80 con 0x7f2ff8004810 2024-04-25T00:32:36.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff800bc80 con 0x7f300c15e200 2024-04-25T00:32:36.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300affd640 1 --2- 172.21.15.29:0/1303619158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ff80039f0 0x7f2ff8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:36.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f3003fff640 1 --2- 172.21.15.29:0/1303619158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff8004810 0x7f2ff8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:36.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300b7fe640 1 --2- 172.21.15.29:0/1303619158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f300c15e200 0x7f2ff80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:36.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300affd640 1 -- 172.21.15.29:0/1303619158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f300c15e200 msgr2=0x7f2ff80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:36.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300affd640 1 --2- 172.21.15.29:0/1303619158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f300c15e200 0x7f2ff80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:36.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300affd640 1 -- 172.21.15.29:0/1303619158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff8004810 msgr2=0x7f2ff8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:36.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300affd640 1 --2- 172.21.15.29:0/1303619158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff8004810 0x7f2ff8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:36.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.619+0000 7f300affd640 1 -- 172.21.15.29:0/1303619158 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff8003530 con 0x7f2ff80039f0 2024-04-25T00:32:36.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f300affd640 1 --2- 172.21.15.29:0/1303619158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ff80039f0 0x7f2ff8003e10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009ab0 tx=0x7f2ffc01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8573940fb82f0f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:36.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0096b0 con 0x7f2ff80039f0 2024-04-25T00:32:36.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc0a6510 con 0x7f2ff80039f0 2024-04-25T00:32:36.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f300a7fc640 1 -- 172.21.15.29:0/1303619158 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0a6900 con 0x7f2ff80039f0 2024-04-25T00:32:36.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 -- 172.21.15.29:0/1303619158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ff80039f0 msgr2=0x7f2ff8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:36.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 --2- 172.21.15.29:0/1303619158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ff80039f0 0x7f2ff8003e10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009ab0 tx=0x7f2ffc01ca00 comp rx=0 tx=0).stop 2024-04-25T00:32:36.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f300b7fe640 1 -- 172.21.15.29:0/1303619158 reap_dead start 2024-04-25T00:32:36.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 -- 172.21.15.29:0/1303619158 shutdown_connections 2024-04-25T00:32:36.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 -- 172.21.15.29:0/1303619158 >> 172.21.15.29:0/1303619158 conn(0x7f300c09d280 msgr2=0x7f300c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:36.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 -- 172.21.15.29:0/1303619158 shutdown_connections 2024-04-25T00:32:36.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 -- 172.21.15.29:0/1303619158 wait complete. 2024-04-25T00:32:36.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 Processor -- start 2024-04-25T00:32:36.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 -- start start 2024-04-25T00:32:36.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ff80039f0 0x7f300c0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:36.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f300c159ef0 0x7f300c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:36.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f300c06c5b0 0x7f300c06ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f300c0b4e40 con 0x7f2ff80039f0 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f300c0b4cc0 con 0x7f300c06c5b0 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f3011495640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f300c0b4fc0 con 0x7f300c159ef0 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f300affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ff80039f0 0x7f300c0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f300affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ff80039f0 0x7f300c0ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37636/0 (socket says 172.21.15.29:37636) 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.623+0000 7f300affd640 1 -- 172.21.15.29:0/3459533181 learned_addr learned my addr 172.21.15.29:0/3459533181 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f300b7fe640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f300c06c5b0 0x7f300c06ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f3003fff640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f300c159ef0 0x7f300c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f300affd640 1 -- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f300c159ef0 msgr2=0x7f300c15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f300affd640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f300c159ef0 0x7f300c15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f300affd640 1 -- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f300c06c5b0 msgr2=0x7f300c06ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f300affd640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f300c06c5b0 0x7f300c06ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f300affd640 1 -- 172.21.15.29:0/3459533181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c15b9d0 con 0x7f2ff80039f0 2024-04-25T00:32:36.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f300affd640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ff80039f0 0x7f300c0ccaf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc01c830 tx=0x7f2ff8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:36.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f30037fe640 1 -- 172.21.15.29:0/3459533181 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0a66c0 con 0x7f2ff80039f0 2024-04-25T00:32:36.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f30037fe640 1 -- 172.21.15.29:0/3459533181 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc0bf070 con 0x7f2ff80039f0 2024-04-25T00:32:36.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f3011495640 1 -- 172.21.15.29:0/3459533181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c15a930 con 0x7f2ff80039f0 2024-04-25T00:32:36.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f30037fe640 1 -- 172.21.15.29:0/3459533181 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0a6ac0 con 0x7f2ff80039f0 2024-04-25T00:32:36.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f3011495640 1 -- 172.21.15.29:0/3459533181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c15aef0 con 0x7f2ff80039f0 2024-04-25T00:32:36.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f3011495640 1 -- 172.21.15.29:0/3459533181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f300c06cf70 con 0x7f2ff80039f0 2024-04-25T00:32:36.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f30037fe640 1 -- 172.21.15.29:0/3459533181 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2ffc0a6c60 con 0x7f2ff80039f0 2024-04-25T00:32:36.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f30037fe640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2fec0678c0 0x7f2fec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:36.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f30037fe640 1 -- 172.21.15.29:0/3459533181 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2ffc1206c0 con 0x7f2ff80039f0 2024-04-25T00:32:36.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f3003fff640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2fec0678c0 0x7f2fec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:36.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.627+0000 7f3003fff640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2fec0678c0 0x7f2fec069d80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f300c0cc7d0 tx=0x7f2ff4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.631+0000 7f30037fe640 1 -- 172.21.15.29:0/3459533181 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2ffc016740 con 0x7f2ff80039f0 2024-04-25T00:32:36.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:36.831+0000 7f3011495640 1 -- 172.21.15.29:0/3459533181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f300c069270 con 0x7f2ff80039f0 2024-04-25T00:32:37.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.471+0000 7f30037fe640 1 -- 172.21.15.29:0/3459533181 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) ==== 122+0+0 (secure 0 0 0) 0x7f2ffc0f1170 con 0x7f2ff80039f0 2024-04-25T00:32:37.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.475+0000 7f3011495640 1 -- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2fec0678c0 msgr2=0x7f2fec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:37.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.475+0000 7f3011495640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2fec0678c0 0x7f2fec069d80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f300c0cc7d0 tx=0x7f2ff4009040 comp rx=0 tx=0).stop 2024-04-25T00:32:37.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.475+0000 7f3011495640 1 -- 172.21.15.29:0/3459533181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ff80039f0 msgr2=0x7f300c0ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:37.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.475+0000 7f3011495640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ff80039f0 0x7f300c0ccaf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc01c830 tx=0x7f2ff8005840 comp rx=0 tx=0).stop 2024-04-25T00:32:37.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.475+0000 7f3011495640 1 -- 172.21.15.29:0/3459533181 shutdown_connections 2024-04-25T00:32:37.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.475+0000 7f3011495640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2fec0678c0 0x7f2fec069d80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.475+0000 7f3011495640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f300c06c5b0 0x7f300c06ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.475+0000 7f3011495640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f300c159ef0 0x7f300c15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.475+0000 7f3011495640 1 --2- 172.21.15.29:0/3459533181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ff80039f0 0x7f300c0ccaf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.475+0000 7f3011495640 1 -- 172.21.15.29:0/3459533181 >> 172.21.15.29:0/3459533181 conn(0x7f300c09d280 msgr2=0x7f300c0b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:37.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.475+0000 7f3011495640 1 -- 172.21.15.29:0/3459533181 shutdown_connections 2024-04-25T00:32:37.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.475+0000 7f3011495640 1 -- 172.21.15.29:0/3459533181 wait complete. 2024-04-25T00:32:37.501 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:32:37.501 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:32:37.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf116e6640 1 Processor -- start 2024-04-25T00:32:37.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf116e6640 1 -- start start 2024-04-25T00:32:37.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf116e6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdf0c0b3c30 0x7fdf0c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:37.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf116e6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdf0c0b4b10 0x7fdf0c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:37.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf116e6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf0c0b41e0 con 0x7fdf0c0b3140 2024-04-25T00:32:37.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf116e6640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdf0c06d960 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf116e6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdf0c06dc40 con 0x7fdf0c0b4b10 2024-04-25T00:32:37.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf0affd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdf0c0b3140 0x7fdf0c0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37262/0 (socket says 172.21.15.29:37262) 2024-04-25T00:32:37.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf0affd640 1 -- 172.21.15.29:0/2956017475 learned_addr learned my addr 172.21.15.29:0/2956017475 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:37.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf0a7fc640 1 --2- 172.21.15.29:0/2956017475 >> v2:172.21.15.29:3300/0 conn(0x7fdf0c0b3c30 0x7fdf0c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:37.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf0b7fe640 1 --2- 172.21.15.29:0/2956017475 >> v2:172.21.15.157:3300/0 conn(0x7fdf0c0b4b10 0x7fdf0c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:37.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf0a7fc640 1 -- 172.21.15.29:0/2956017475 >> v2:172.21.15.157:3300/0 conn(0x7fdf0c0b4b10 msgr2=0x7fdf0c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:37.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf0a7fc640 1 --2- 172.21.15.29:0/2956017475 >> v2:172.21.15.157:3300/0 conn(0x7fdf0c0b4b10 0x7fdf0c0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf0a7fc640 1 -- 172.21.15.29:0/2956017475 >> v1:172.21.15.29:6789/0 conn(0x7fdf0c0b3140 legacy=0x7fdf0c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:37.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf0a7fc640 1 -- 172.21.15.29:0/2956017475 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c06df20 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.619+0000 7fdf09ffb640 1 -- 172.21.15.29:0/2956017475 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2131498625 0 0) 0x7fdf0c0b41e0 con 0x7fdf0c0b3140 2024-04-25T00:32:37.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf0a7fc640 1 --2- 172.21.15.29:0/2956017475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf0c0b3c30 0x7fdf0c06e440 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdf000097c0 tx=0x7fdf0001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dea8a9a4ae4229 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:37.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf09ffb640 1 -- 172.21.15.29:0/2956017475 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf000b0070 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf09ffb640 1 -- 172.21.15.29:0/2956017475 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf000ab600 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf09ffb640 1 -- 172.21.15.29:0/2956017475 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf000ab9d0 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 -- 172.21.15.29:0/2956017475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf0c0b3c30 msgr2=0x7fdf0c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:37.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 --2- 172.21.15.29:0/2956017475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf0c0b3c30 0x7fdf0c06e440 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdf000097c0 tx=0x7fdf0001c460 comp rx=0 tx=0).stop 2024-04-25T00:32:37.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 -- 172.21.15.29:0/2956017475 shutdown_connections 2024-04-25T00:32:37.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 --2- 172.21.15.29:0/2956017475 >> v2:172.21.15.157:3300/0 conn(0x7fdf0c0b4b10 0x7fdf0c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 --2- 172.21.15.29:0/2956017475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf0c0b3c30 0x7fdf0c06e440 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 -- 172.21.15.29:0/2956017475 >> 172.21.15.29:0/2956017475 conn(0x7fdf0c0b1870 msgr2=0x7fdf0c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:37.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 -- 172.21.15.29:0/2956017475 shutdown_connections 2024-04-25T00:32:37.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 -- 172.21.15.29:0/2956017475 wait complete. 2024-04-25T00:32:37.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 Processor -- start 2024-04-25T00:32:37.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 -- start start 2024-04-25T00:32:37.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf0c0b3140 0x7fdf0c142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:37.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf0c0b3c30 0x7fdf0c139100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:37.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf0c0b4b10 0x7fdf0c15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:37.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c168d80 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c168c00 con 0x7fdf0c0b4b10 2024-04-25T00:32:37.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.623+0000 7fdf116e6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0c168f00 con 0x7fdf0c0b3140 2024-04-25T00:32:37.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf0a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf0c0b3c30 0x7fdf0c139100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:37.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf0a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf0c0b3c30 0x7fdf0c139100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37648/0 (socket says 172.21.15.29:37648) 2024-04-25T00:32:37.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf0a7fc640 1 -- 172.21.15.29:0/692104614 learned_addr learned my addr 172.21.15.29:0/692104614 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:37.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf0affd640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf0c0b3140 0x7fdf0c142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:37.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf0b7fe640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf0c0b4b10 0x7fdf0c15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:37.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf0a7fc640 1 -- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf0c0b3140 msgr2=0x7fdf0c142420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:37.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf0a7fc640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf0c0b3140 0x7fdf0c142420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf0a7fc640 1 -- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf0c0b4b10 msgr2=0x7fdf0c15b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:37.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf0a7fc640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf0c0b4b10 0x7fdf0c15b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf0a7fc640 1 -- 172.21.15.29:0/692104614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c1565c0 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf0a7fc640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf0c0b3c30 0x7fdf0c139100 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdf000ae040 tx=0x7fdf0001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdef37fe640 1 -- 172.21.15.29:0/692104614 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf000b0050 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdef37fe640 1 -- 172.21.15.29:0/692104614 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf000b1040 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf116e6640 1 -- 172.21.15.29:0/692104614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0c142960 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdef37fe640 1 -- 172.21.15.29:0/692104614 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00006e20 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf116e6640 1 -- 172.21.15.29:0/692104614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0c13fd10 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf116e6640 1 -- 172.21.15.29:0/692104614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0c069270 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdef37fe640 1 -- 172.21.15.29:0/692104614 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdf000be060 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdef37fe640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdef80678c0 0x7fdef8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdef37fe640 1 -- 172.21.15.29:0/692104614 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdf0011f6a0 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf0affd640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdef80678c0 0x7fdef8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:37.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.627+0000 7fdf0affd640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdef80678c0 0x7fdef8069d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdefc000f70 tx=0x7fdefc007c10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:37.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.631+0000 7fdef37fe640 1 -- 172.21.15.29:0/692104614 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdf00016050 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.827+0000 7fdf116e6640 1 -- 172.21.15.29:0/692104614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf0c0d09d0 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.827+0000 7fdef37fe640 1 -- 172.21.15.29:0/692104614 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5186 (secure 0 0 0) 0x7fdf000f01e0 con 0x7fdf0c0b3c30 2024-04-25T00:32:37.832 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:37.833 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2100384916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2100384916},{"type":"v1","addr":"172.21.15.157:6835","nonce":2100384916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4984,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1430513194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1430513194},{"type":"v1","addr":"172.21.15.157:6837","nonce":1430513194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3354831881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3354831881},{"type":"v1","addr":"172.21.15.29:6833","nonce":3354831881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:32:33.777640+0000","modified":"2024-04-25T00:32:36.837733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1468158138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1468158138},{"type":"v1","addr":"172.21.15.29:6835","nonce":1468158138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":2}]} 2024-04-25T00:32:37.833 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 15 2024-04-25T00:32:37.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.831+0000 7fdf116e6640 1 -- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdef80678c0 msgr2=0x7fdef8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:37.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.831+0000 7fdf116e6640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdef80678c0 0x7fdef8069d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdefc000f70 tx=0x7fdefc007c10 comp rx=0 tx=0).stop 2024-04-25T00:32:37.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.831+0000 7fdf116e6640 1 -- 172.21.15.29:0/692104614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf0c0b3c30 msgr2=0x7fdf0c139100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:37.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.831+0000 7fdf116e6640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf0c0b3c30 0x7fdf0c139100 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdf000ae040 tx=0x7fdf0001aca0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.831+0000 7fdf116e6640 1 -- 172.21.15.29:0/692104614 shutdown_connections 2024-04-25T00:32:37.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.831+0000 7fdf116e6640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdef80678c0 0x7fdef8069d80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.831+0000 7fdf116e6640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf0c0b4b10 0x7fdf0c15b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.831+0000 7fdf116e6640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf0c0b3c30 0x7fdf0c139100 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.831+0000 7fdf116e6640 1 --2- 172.21.15.29:0/692104614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf0c0b3140 0x7fdf0c142420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.831+0000 7fdf116e6640 1 -- 172.21.15.29:0/692104614 >> 172.21.15.29:0/692104614 conn(0x7fdf0c0b1870 msgr2=0x7fdf0c1661b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:37.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.831+0000 7fdf116e6640 1 -- 172.21.15.29:0/692104614 shutdown_connections 2024-04-25T00:32:37.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.831+0000 7fdf116e6640 1 -- 172.21.15.29:0/692104614 wait complete. 2024-04-25T00:32:37.850 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca93d4640 1 Processor -- start 2024-04-25T00:32:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca93d4640 1 -- start start 2024-04-25T00:32:37.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca93d4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ca406b210 0x7f7ca4072990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:37.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca93d4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7ca415ecf0 0x7f7ca40ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:37.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca93d4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ca415f2a0 con 0x7f7ca406b5f0 2024-04-25T00:32:37.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca93d4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7ca40b03f0 con 0x7f7ca406b210 2024-04-25T00:32:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca93d4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7ca415f580 con 0x7f7ca415ecf0 2024-04-25T00:32:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca2ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ca406b210 0x7f7ca4072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca27fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7ca406b5f0 0x7f7ca415e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38292/0 (socket says 172.21.15.29:38292) 2024-04-25T00:32:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca27fc640 1 -- 172.21.15.29:0/1615664303 learned_addr learned my addr 172.21.15.29:0/1615664303 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca37fe640 1 --2- 172.21.15.29:0/1615664303 >> v2:172.21.15.157:3300/0 conn(0x7f7ca415ecf0 0x7f7ca40ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca2ffd640 1 -- 172.21.15.29:0/1615664303 >> v2:172.21.15.157:3300/0 conn(0x7f7ca415ecf0 msgr2=0x7f7ca40ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca2ffd640 1 --2- 172.21.15.29:0/1615664303 >> v2:172.21.15.157:3300/0 conn(0x7f7ca415ecf0 0x7f7ca40ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca2ffd640 1 -- 172.21.15.29:0/1615664303 >> v1:172.21.15.29:6789/0 conn(0x7f7ca406b5f0 legacy=0x7f7ca415e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca2ffd640 1 -- 172.21.15.29:0/1615664303 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca4071eb0 con 0x7f7ca406b210 2024-04-25T00:32:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.959+0000 7f7ca1ffb640 1 -- 172.21.15.29:0/1615664303 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 905010636 0 0) 0x7f7ca415f2a0 con 0x7f7ca406b5f0 2024-04-25T00:32:37.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca2ffd640 1 --2- 172.21.15.29:0/1615664303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca406b210 0x7f7ca4072990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4071d10 tx=0x7f7c9401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb166f0fcf7d405 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:37.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca1ffb640 1 -- 172.21.15.29:0/1615664303 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c940b0070 con 0x7f7ca406b210 2024-04-25T00:32:37.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca1ffb640 1 -- 172.21.15.29:0/1615664303 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c940ab400 con 0x7f7ca406b210 2024-04-25T00:32:37.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca1ffb640 1 -- 172.21.15.29:0/1615664303 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c940ab7d0 con 0x7f7ca406b210 2024-04-25T00:32:37.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 -- 172.21.15.29:0/1615664303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca406b210 msgr2=0x7f7ca4072990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:37.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 --2- 172.21.15.29:0/1615664303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca406b210 0x7f7ca4072990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4071d10 tx=0x7f7c9401c360 comp rx=0 tx=0).stop 2024-04-25T00:32:37.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 -- 172.21.15.29:0/1615664303 shutdown_connections 2024-04-25T00:32:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 --2- 172.21.15.29:0/1615664303 >> v2:172.21.15.157:3300/0 conn(0x7f7ca415ecf0 0x7f7ca40ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 --2- 172.21.15.29:0/1615664303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca406b210 0x7f7ca4072990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 -- 172.21.15.29:0/1615664303 >> 172.21.15.29:0/1615664303 conn(0x7f7ca40d0c40 msgr2=0x7f7ca40d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 -- 172.21.15.29:0/1615664303 shutdown_connections 2024-04-25T00:32:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 -- 172.21.15.29:0/1615664303 wait complete. 2024-04-25T00:32:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 Processor -- start 2024-04-25T00:32:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 -- start start 2024-04-25T00:32:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ca406b210 0x7f7ca4156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ca406b5f0 0x7f7ca4145740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca415ecf0 0x7f7ca40c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca416af70 con 0x7f7ca415ecf0 2024-04-25T00:32:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca40cbf80 con 0x7f7ca406b5f0 2024-04-25T00:32:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7ca416b0f0 con 0x7f7ca406b210 2024-04-25T00:32:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca415ecf0 0x7f7ca40c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca415ecf0 0x7f7ca40c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54182/0 (socket says 172.21.15.29:54182) 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca37fe640 1 -- 172.21.15.29:0/4065721180 learned_addr learned my addr 172.21.15.29:0/4065721180 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca2ffd640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ca406b210 0x7f7ca4156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca27fc640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ca406b5f0 0x7f7ca4145740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca37fe640 1 -- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ca406b210 msgr2=0x7f7ca4156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca37fe640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ca406b210 0x7f7ca4156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca37fe640 1 -- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ca406b5f0 msgr2=0x7f7ca4145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca37fe640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ca406b5f0 0x7f7ca4145740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca37fe640 1 -- 172.21.15.29:0/4065721180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca4157420 con 0x7f7ca415ecf0 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca37fe640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca415ecf0 0x7f7ca40c6760 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c007e40 tx=0x7f7c9c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7c8b7fe640 1 -- 172.21.15.29:0/4065721180 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c014620 con 0x7f7ca415ecf0 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7c8b7fe640 1 -- 172.21.15.29:0/4065721180 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c9c019070 con 0x7f7ca415ecf0 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 -- 172.21.15.29:0/4065721180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca4157700 con 0x7f7ca415ecf0 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7c8b7fe640 1 -- 172.21.15.29:0/4065721180 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c014a20 con 0x7f7ca415ecf0 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.963+0000 7f7ca93d4640 1 -- 172.21.15.29:0/4065721180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca4157cc0 con 0x7f7ca415ecf0 2024-04-25T00:32:37.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.967+0000 7f7ca93d4640 1 -- 172.21.15.29:0/4065721180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ca4069270 con 0x7f7ca415ecf0 2024-04-25T00:32:37.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.967+0000 7f7c8b7fe640 1 -- 172.21.15.29:0/4065721180 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7c9c014bc0 con 0x7f7ca415ecf0 2024-04-25T00:32:37.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.967+0000 7f7c8b7fe640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c900678f0 0x7f7c90069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:37.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.967+0000 7f7c8b7fe640 1 -- 172.21.15.29:0/4065721180 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7c9c086550 con 0x7f7ca415ecf0 2024-04-25T00:32:37.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.967+0000 7f7ca2ffd640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c900678f0 0x7f7c90069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:37.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.967+0000 7f7ca2ffd640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c900678f0 0x7f7c90069db0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7c940ad000 tx=0x7f7c940a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:37.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:37.971+0000 7f7c8b7fe640 1 -- 172.21.15.29:0/4065721180 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7c9c057090 con 0x7f7ca415ecf0 2024-04-25T00:32:38.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.139+0000 7f7ca93d4640 1 -- 172.21.15.29:0/4065721180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ca40d1a70 con 0x7f7ca415ecf0 2024-04-25T00:32:38.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.139+0000 7f7c8b7fe640 1 -- 172.21.15.29:0/4065721180 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16621 (secure 0 0 0) 0x7f7c9c056990 con 0x7f7ca415ecf0 2024-04-25T00:32:38.144 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:38.144 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":33,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:32:35.426396+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:32:32.254456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:32:32.779372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:38.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.143+0000 7f7ca93d4640 1 -- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c900678f0 msgr2=0x7f7c90069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.143+0000 7f7ca93d4640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c900678f0 0x7f7c90069db0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7c940ad000 tx=0x7f7c940a6000 comp rx=0 tx=0).stop 2024-04-25T00:32:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.143+0000 7f7ca93d4640 1 -- 172.21.15.29:0/4065721180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca415ecf0 msgr2=0x7f7ca40c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.143+0000 7f7ca93d4640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca415ecf0 0x7f7ca40c6760 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c007e40 tx=0x7f7c9c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.143+0000 7f7ca93d4640 1 -- 172.21.15.29:0/4065721180 shutdown_connections 2024-04-25T00:32:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.143+0000 7f7ca93d4640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca415ecf0 0x7f7ca40c6760 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.143+0000 7f7ca93d4640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c900678f0 0x7f7c90069db0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.143+0000 7f7ca93d4640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ca406b5f0 0x7f7ca4145740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.143+0000 7f7ca93d4640 1 --2- 172.21.15.29:0/4065721180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ca406b210 0x7f7ca4156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.143+0000 7f7ca93d4640 1 -- 172.21.15.29:0/4065721180 >> 172.21.15.29:0/4065721180 conn(0x7f7ca40d0c40 msgr2=0x7f7ca40b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.143+0000 7f7ca93d4640 1 -- 172.21.15.29:0/4065721180 shutdown_connections 2024-04-25T00:32:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.143+0000 7f7ca93d4640 1 -- 172.21.15.29:0/4065721180 wait complete. 2024-04-25T00:32:38.163 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:32:38.163 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:32:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.271+0000 7faab6a35640 1 Processor -- start 2024-04-25T00:32:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.271+0000 7faab6a35640 1 -- start start 2024-04-25T00:32:38.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faab6a35640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faab015f3c0 0x7faab00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faab6a35640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faab00b3720 con 0x7faab00b4780 2024-04-25T00:32:38.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faab6a35640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faab00b39c0 con 0x7faab015e8d0 2024-04-25T00:32:38.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faab6a35640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faab00b42f0 con 0x7faab015f3c0 2024-04-25T00:32:38.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaf7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7faab00b4780 0x7faab015c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38300/0 (socket says 172.21.15.29:38300) 2024-04-25T00:32:38.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaf7fe640 1 -- 172.21.15.29:0/1963893858 learned_addr learned my addr 172.21.15.29:0/1963893858 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:38.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faab4fd2640 1 --2- 172.21.15.29:0/1963893858 >> v2:172.21.15.157:3300/0 conn(0x7faab015f3c0 0x7faab00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 319455580 0 0) 0x7faab00b3720 con 0x7faab00b4780 2024-04-25T00:32:38.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa98003530 con 0x7faab00b4780 2024-04-25T00:32:38.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 620603628 0 0) 0x7faa98003530 con 0x7faab00b4780 2024-04-25T00:32:38.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 >> v2:172.21.15.157:3300/0 conn(0x7faab015f3c0 msgr2=0x7faab00cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:38.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 --2- 172.21.15.29:0/1963893858 >> v2:172.21.15.157:3300/0 conn(0x7faab015f3c0 0x7faab00cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 >> v1:172.21.15.157:6789/0 conn(0x7faab015e8d0 legacy=0x7faab015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab00b3ca0 con 0x7faab00b4780 2024-04-25T00:32:38.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7faaa4001680 con 0x7faab00b4780 2024-04-25T00:32:38.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 >> v1:172.21.15.29:6789/0 conn(0x7faab00b4780 legacy=0x7faab015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 --2- 172.21.15.29:0/1963893858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa980039c0 0x7faa98003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 --2- 172.21.15.29:0/1963893858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa98004810 0x7faa98004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 --2- 172.21.15.29:0/1963893858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa980051d0 0x7faa9800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faab00b42f0 con 0x7faa980051d0 2024-04-25T00:32:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faa9800c070 con 0x7faa98004810 2024-04-25T00:32:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faa9800c270 con 0x7faa980039c0 2024-04-25T00:32:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faab4fd2640 1 --2- 172.21.15.29:0/1963893858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa980051d0 0x7faa9800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaf7fe640 1 --2- 172.21.15.29:0/1963893858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa98004810 0x7faa98004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faaaffff640 1 --2- 172.21.15.29:0/1963893858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa980039c0 0x7faa98003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faab4fd2640 1 -- 172.21.15.29:0/1963893858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa980039c0 msgr2=0x7faa98003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faab4fd2640 1 --2- 172.21.15.29:0/1963893858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa980039c0 0x7faa98003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faab4fd2640 1 -- 172.21.15.29:0/1963893858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa98004810 msgr2=0x7faa98004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faab4fd2640 1 --2- 172.21.15.29:0/1963893858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa98004810 0x7faa98004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.275+0000 7faab4fd2640 1 -- 172.21.15.29:0/1963893858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa98003530 con 0x7faa980051d0 2024-04-25T00:32:38.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab4fd2640 1 -- 172.21.15.29:0/1963893858 reap_dead start 2024-04-25T00:32:38.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab4fd2640 1 --2- 172.21.15.29:0/1963893858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa980051d0 0x7faa9800b9d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009fc0 tx=0x7faaa801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d36779ab11c7c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:38.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8009c10 con 0x7faa980051d0 2024-04-25T00:32:38.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa80a5cd0 con 0x7faa980051d0 2024-04-25T00:32:38.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faaaeffd640 1 -- 172.21.15.29:0/1963893858 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8003be0 con 0x7faa980051d0 2024-04-25T00:32:38.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 -- 172.21.15.29:0/1963893858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa980051d0 msgr2=0x7faa9800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 --2- 172.21.15.29:0/1963893858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa980051d0 0x7faa9800b9d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009fc0 tx=0x7faaa801ccc0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 -- 172.21.15.29:0/1963893858 shutdown_connections 2024-04-25T00:32:38.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 --2- 172.21.15.29:0/1963893858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa980051d0 0x7faa9800b9d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 -- 172.21.15.29:0/1963893858 >> 172.21.15.29:0/1963893858 conn(0x7faab00b1870 msgr2=0x7faab00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 -- 172.21.15.29:0/1963893858 shutdown_connections 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 -- 172.21.15.29:0/1963893858 wait complete. 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 Processor -- start 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 -- start start 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa980051d0 0x7faab015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab00727a0 0x7faab013ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faab013bea0 0x7faab013c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faab006cf10 con 0x7faab013bea0 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faab00b3580 con 0x7faab00727a0 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab6a35640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faab0144a20 con 0x7faa980051d0 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab4fd2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faab013bea0 0x7faab013c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab4fd2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faab013bea0 0x7faab013c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54206/0 (socket says 172.21.15.29:54206) 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab4fd2640 1 -- 172.21.15.29:0/89549170 learned_addr learned my addr 172.21.15.29:0/89549170 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faaaffff640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa980051d0 0x7faab015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faaaf7fe640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab00727a0 0x7faab013ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.279+0000 7faab4fd2640 1 -- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa980051d0 msgr2=0x7faab015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faab4fd2640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa980051d0 0x7faab015a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faab4fd2640 1 -- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab00727a0 msgr2=0x7faab013ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faab4fd2640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab00727a0 0x7faab013ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faab4fd2640 1 -- 172.21.15.29:0/89549170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab013c870 con 0x7faab013bea0 2024-04-25T00:32:38.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faaaf7fe640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab00727a0 0x7faab013ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:38.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faab4fd2640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faab013bea0 0x7faab013c330 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7faaa801c730 tx=0x7faaa8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:38.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faaacff9640 1 -- 172.21.15.29:0/89549170 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa80aed70 con 0x7faab013bea0 2024-04-25T00:32:38.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faaacff9640 1 -- 172.21.15.29:0/89549170 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa80bd070 con 0x7faab013bea0 2024-04-25T00:32:38.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faaacff9640 1 -- 172.21.15.29:0/89549170 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8003ba0 con 0x7faab013bea0 2024-04-25T00:32:38.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faab6a35640 1 -- 172.21.15.29:0/89549170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab01395a0 con 0x7faab013bea0 2024-04-25T00:32:38.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faab6a35640 1 -- 172.21.15.29:0/89549170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab013f640 con 0x7faab013bea0 2024-04-25T00:32:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faab6a35640 1 -- 172.21.15.29:0/89549170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab0069270 con 0x7faab013bea0 2024-04-25T00:32:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faaacff9640 1 -- 172.21.15.29:0/89549170 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faaa8004030 con 0x7faab013bea0 2024-04-25T00:32:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faaacff9640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faa9c0678f0 0x7faa9c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faaacff9640 1 -- 172.21.15.29:0/89549170 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7faaa81216c0 con 0x7faab013bea0 2024-04-25T00:32:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faaaffff640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faa9c0678f0 0x7faa9c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.283+0000 7faaaffff640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faa9c0678f0 0x7faa9c069db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faaa0000c10 tx=0x7faaa0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.287+0000 7faaacff9640 1 -- 172.21.15.29:0/89549170 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faaa80169d0 con 0x7faab013bea0 2024-04-25T00:32:38.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.439+0000 7faab6a35640 1 -- 172.21.15.29:0/89549170 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faab006e7b0 con 0x7faa9c0678f0 2024-04-25T00:32:38.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.459+0000 7faaacff9640 1 -- 172.21.15.29:0/89549170 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7faab006e7b0 con 0x7faa9c0678f0 2024-04-25T00:32:38.465 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:38.468 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:32:38.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.463+0000 7faab6a35640 1 -- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faa9c0678f0 msgr2=0x7faa9c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.463+0000 7faab6a35640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faa9c0678f0 0x7faa9c069db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faaa0000c10 tx=0x7faaa0009040 comp rx=0 tx=0).stop 2024-04-25T00:32:38.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.463+0000 7faab6a35640 1 -- 172.21.15.29:0/89549170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faab013bea0 msgr2=0x7faab013c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.463+0000 7faab6a35640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faab013bea0 0x7faab013c330 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7faaa801c730 tx=0x7faaa8009750 comp rx=0 tx=0).stop 2024-04-25T00:32:38.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.463+0000 7faab6a35640 1 -- 172.21.15.29:0/89549170 shutdown_connections 2024-04-25T00:32:38.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.463+0000 7faab6a35640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faab013bea0 0x7faab013c330 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.463+0000 7faab6a35640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faa9c0678f0 0x7faa9c069db0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.463+0000 7faab6a35640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab00727a0 0x7faab013ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.463+0000 7faab6a35640 1 --2- 172.21.15.29:0/89549170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa980051d0 0x7faab015a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.463+0000 7faab6a35640 1 -- 172.21.15.29:0/89549170 >> 172.21.15.29:0/89549170 conn(0x7faab00b1870 msgr2=0x7faab006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:38.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.463+0000 7faab6a35640 1 -- 172.21.15.29:0/89549170 shutdown_connections 2024-04-25T00:32:38.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.463+0000 7faab6a35640 1 -- 172.21.15.29:0/89549170 wait complete. 2024-04-25T00:32:38.486 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-04-25T00:32:37.466967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253272,"kb_used_data":3820,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721448,"statfs":{"total":773094113280,"available":772834762752,"internally_reserved":0,"allocated":3911680,"data_stored":2220449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.075330"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.421587+0000","last_change":"2024-04-25T00:32:33.421587+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.421587+0000","last_undegraded":"2024-04-25T00:32:33.421587+0000","last_fullsized":"2024-04-25T00:32:33.421587+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.416958+0000","last_change":"2024-04-25T00:32:33.416958+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.416958+0000","last_undegraded":"2024-04-25T00:32:33.416958+0000","last_fullsized":"2024-04-25T00:32:33.416958+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.401783+0000","last_change":"2024-04-25T00:32:33.401783+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.401783+0000","last_undegraded":"2024-04-25T00:32:33.401783+0000","last_fullsized":"2024-04-25T00:32:33.401783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410014+0000","last_change":"2024-04-25T00:32:33.410014+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410014+0000","last_undegraded":"2024-04-25T00:32:33.410014+0000","last_fullsized":"2024-04-25T00:32:33.410014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.406582+0000","last_change":"2024-04-25T00:32:33.406582+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.406582+0000","last_undegraded":"2024-04-25T00:32:33.406582+0000","last_fullsized":"2024-04-25T00:32:33.406582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.416330+0000","last_change":"2024-04-25T00:32:33.416330+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.416330+0000","last_undegraded":"2024-04-25T00:32:33.416330+0000","last_fullsized":"2024-04-25T00:32:33.416330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.415789+0000","last_change":"2024-04-25T00:32:33.415789+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.415789+0000","last_undegraded":"2024-04-25T00:32:33.415789+0000","last_fullsized":"2024-04-25T00:32:33.415789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.428371+0000","last_change":"2024-04-25T00:32:33.428371+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.428371+0000","last_undegraded":"2024-04-25T00:32:33.428371+0000","last_fullsized":"2024-04-25T00:32:33.428371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402082+0000","last_change":"2024-04-25T00:32:33.402082+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402082+0000","last_undegraded":"2024-04-25T00:32:33.402082+0000","last_fullsized":"2024-04-25T00:32:33.402082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410001+0000","last_change":"2024-04-25T00:32:33.410001+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410001+0000","last_undegraded":"2024-04-25T00:32:33.410001+0000","last_fullsized":"2024-04-25T00:32:33.410001+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.411695+0000","last_change":"2024-04-25T00:32:33.411695+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.411695+0000","last_undegraded":"2024-04-25T00:32:33.411695+0000","last_fullsized":"2024-04-25T00:32:33.411695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402278+0000","last_change":"2024-04-25T00:32:33.402278+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402278+0000","last_undegraded":"2024-04-25T00:32:33.402278+0000","last_fullsized":"2024-04-25T00:32:33.402278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.406087+0000","last_change":"2024-04-25T00:32:33.406087+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.406087+0000","last_undegraded":"2024-04-25T00:32:33.406087+0000","last_fullsized":"2024-04-25T00:32:33.406087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.412316+0000","last_change":"2024-04-25T00:32:33.412316+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.412316+0000","last_undegraded":"2024-04-25T00:32:33.412316+0000","last_fullsized":"2024-04-25T00:32:33.412316+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402227+0000","last_change":"2024-04-25T00:32:33.402227+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402227+0000","last_undegraded":"2024-04-25T00:32:33.402227+0000","last_fullsized":"2024-04-25T00:32:33.402227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.412924+0000","last_change":"2024-04-25T00:32:33.412924+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.412924+0000","last_undegraded":"2024-04-25T00:32:33.412924+0000","last_fullsized":"2024-04-25T00:32:33.412924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.413406+0000","last_change":"2024-04-25T00:32:33.413406+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.413406+0000","last_undegraded":"2024-04-25T00:32:33.413406+0000","last_fullsized":"2024-04-25T00:32:33.413406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.408069+0000","last_change":"2024-04-25T00:32:33.408069+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.408069+0000","last_undegraded":"2024-04-25T00:32:33.408069+0000","last_fullsized":"2024-04-25T00:32:33.408069+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404172+0000","last_change":"2024-04-25T00:32:33.404172+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404172+0000","last_undegraded":"2024-04-25T00:32:33.404172+0000","last_fullsized":"2024-04-25T00:32:33.404172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.405470+0000","last_change":"2024-04-25T00:32:33.405470+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.405470+0000","last_undegraded":"2024-04-25T00:32:33.405470+0000","last_fullsized":"2024-04-25T00:32:33.405470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.408165+0000","last_change":"2024-04-25T00:32:33.408165+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.408165+0000","last_undegraded":"2024-04-25T00:32:33.408165+0000","last_fullsized":"2024-04-25T00:32:33.408165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.400602+0000","last_change":"2024-04-25T00:32:33.400602+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.400602+0000","last_undegraded":"2024-04-25T00:32:33.400602+0000","last_fullsized":"2024-04-25T00:32:33.400602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.405760+0000","last_change":"2024-04-25T00:32:33.405760+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.405760+0000","last_undegraded":"2024-04-25T00:32:33.405760+0000","last_fullsized":"2024-04-25T00:32:33.405760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402840+0000","last_change":"2024-04-25T00:32:33.402840+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402840+0000","last_undegraded":"2024-04-25T00:32:33.402840+0000","last_fullsized":"2024-04-25T00:32:33.402840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410219+0000","last_change":"2024-04-25T00:32:33.410219+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410219+0000","last_undegraded":"2024-04-25T00:32:33.410219+0000","last_fullsized":"2024-04-25T00:32:33.410219+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.407507+0000","last_change":"2024-04-25T00:32:33.407507+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.407507+0000","last_undegraded":"2024-04-25T00:32:33.407507+0000","last_fullsized":"2024-04-25T00:32:33.407507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403049+0000","last_change":"2024-04-25T00:32:33.403049+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403049+0000","last_undegraded":"2024-04-25T00:32:33.403049+0000","last_fullsized":"2024-04-25T00:32:33.403049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.400778+0000","last_change":"2024-04-25T00:32:33.400778+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.400778+0000","last_undegraded":"2024-04-25T00:32:33.400778+0000","last_fullsized":"2024-04-25T00:32:33.400778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.406659+0000","last_change":"2024-04-25T00:32:33.406659+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.406659+0000","last_undegraded":"2024-04-25T00:32:33.406659+0000","last_fullsized":"2024-04-25T00:32:33.406659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.411758+0000","last_change":"2024-04-25T00:32:33.411758+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.411758+0000","last_undegraded":"2024-04-25T00:32:33.411758+0000","last_fullsized":"2024-04-25T00:32:33.411758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.417701+0000","last_change":"2024-04-25T00:32:33.417701+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.417701+0000","last_undegraded":"2024-04-25T00:32:33.417701+0000","last_fullsized":"2024-04-25T00:32:33.417701+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.426820+0000","last_change":"2024-04-25T00:32:33.426820+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.426820+0000","last_undegraded":"2024-04-25T00:32:33.426820+0000","last_fullsized":"2024-04-25T00:32:33.426820+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410800+0000","last_change":"2024-04-25T00:32:33.410800+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410800+0000","last_undegraded":"2024-04-25T00:32:33.410800+0000","last_fullsized":"2024-04-25T00:32:33.410800+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.440516+0000","last_change":"2024-04-25T00:32:33.439097+0000","last_active":"2024-04-25T00:32:33.440516+0000","last_peered":"2024-04-25T00:32:33.440516+0000","last_clean":"2024-04-25T00:32:33.440516+0000","last_became_active":"2024-04-25T00:32:33.438390+0000","last_became_peered":"2024-04-25T00:32:33.438390+0000","last_unstale":"2024-04-25T00:32:33.440516+0000","last_undegraded":"2024-04-25T00:32:33.440516+0000","last_fullsized":"2024-04-25T00:32:33.440516+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:33.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.419679+0000","last_change":"2024-04-25T00:32:33.419679+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.419679+0000","last_undegraded":"2024-04-25T00:32:33.419679+0000","last_fullsized":"2024-04-25T00:32:33.419679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.440528+0000","last_change":"2024-04-25T00:32:33.439749+0000","last_active":"2024-04-25T00:32:33.440528+0000","last_peered":"2024-04-25T00:32:33.440528+0000","last_clean":"2024-04-25T00:32:33.440528+0000","last_became_active":"2024-04-25T00:32:33.439548+0000","last_became_peered":"2024-04-25T00:32:33.439548+0000","last_unstale":"2024-04-25T00:32:33.440528+0000","last_undegraded":"2024-04-25T00:32:33.440528+0000","last_fullsized":"2024-04-25T00:32:33.440528+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:10.605816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404042+0000","last_change":"2024-04-25T00:32:33.404042+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404042+0000","last_undegraded":"2024-04-25T00:32:33.404042+0000","last_fullsized":"2024-04-25T00:32:33.404042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422465+0000","last_change":"2024-04-25T00:32:33.419772+0000","last_active":"2024-04-25T00:32:33.422465+0000","last_peered":"2024-04-25T00:32:33.422465+0000","last_clean":"2024-04-25T00:32:33.422465+0000","last_became_active":"2024-04-25T00:32:33.419459+0000","last_became_peered":"2024-04-25T00:32:33.419459+0000","last_unstale":"2024-04-25T00:32:33.422465+0000","last_undegraded":"2024-04-25T00:32:33.422465+0000","last_fullsized":"2024-04-25T00:32:33.422465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:24.119844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402369+0000","last_change":"2024-04-25T00:32:33.402369+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402369+0000","last_undegraded":"2024-04-25T00:32:33.402369+0000","last_fullsized":"2024-04-25T00:32:33.402369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.420930+0000","last_change":"2024-04-25T00:32:33.419126+0000","last_active":"2024-04-25T00:32:33.420930+0000","last_peered":"2024-04-25T00:32:33.420930+0000","last_clean":"2024-04-25T00:32:33.420930+0000","last_became_active":"2024-04-25T00:32:33.418834+0000","last_became_peered":"2024-04-25T00:32:33.418834+0000","last_unstale":"2024-04-25T00:32:33.420930+0000","last_undegraded":"2024-04-25T00:32:33.420930+0000","last_fullsized":"2024-04-25T00:32:33.420930+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:39.513615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410063+0000","last_change":"2024-04-25T00:32:33.410063+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410063+0000","last_undegraded":"2024-04-25T00:32:33.410063+0000","last_fullsized":"2024-04-25T00:32:33.410063+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.433292+0000","last_change":"2024-04-25T00:32:33.429088+0000","last_active":"2024-04-25T00:32:33.433292+0000","last_peered":"2024-04-25T00:32:33.433292+0000","last_clean":"2024-04-25T00:32:33.433292+0000","last_became_active":"2024-04-25T00:32:33.427379+0000","last_became_peered":"2024-04-25T00:32:33.427379+0000","last_unstale":"2024-04-25T00:32:33.433292+0000","last_undegraded":"2024-04-25T00:32:33.433292+0000","last_fullsized":"2024-04-25T00:32:33.433292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:05.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403511+0000","last_change":"2024-04-25T00:32:33.403511+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403511+0000","last_undegraded":"2024-04-25T00:32:33.403511+0000","last_fullsized":"2024-04-25T00:32:33.403511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.441233+0000","last_change":"2024-04-25T00:32:33.429965+0000","last_active":"2024-04-25T00:32:33.441233+0000","last_peered":"2024-04-25T00:32:33.441233+0000","last_clean":"2024-04-25T00:32:33.441233+0000","last_became_active":"2024-04-25T00:32:33.427356+0000","last_became_peered":"2024-04-25T00:32:33.427356+0000","last_unstale":"2024-04-25T00:32:33.441233+0000","last_undegraded":"2024-04-25T00:32:33.441233+0000","last_fullsized":"2024-04-25T00:32:33.441233+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:39.513615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404138+0000","last_change":"2024-04-25T00:32:33.404138+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404138+0000","last_undegraded":"2024-04-25T00:32:33.404138+0000","last_fullsized":"2024-04-25T00:32:33.404138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422459+0000","last_change":"2024-04-25T00:32:33.419720+0000","last_active":"2024-04-25T00:32:33.422459+0000","last_peered":"2024-04-25T00:32:33.422459+0000","last_clean":"2024-04-25T00:32:33.422459+0000","last_became_active":"2024-04-25T00:32:33.419338+0000","last_became_peered":"2024-04-25T00:32:33.419338+0000","last_unstale":"2024-04-25T00:32:33.422459+0000","last_undegraded":"2024-04-25T00:32:33.422459+0000","last_fullsized":"2024-04-25T00:32:33.422459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:07.836095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402370+0000","last_change":"2024-04-25T00:32:33.402370+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402370+0000","last_undegraded":"2024-04-25T00:32:33.402370+0000","last_fullsized":"2024-04-25T00:32:33.402370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.441077+0000","last_change":"2024-04-25T00:32:33.433081+0000","last_active":"2024-04-25T00:32:33.441077+0000","last_peered":"2024-04-25T00:32:33.441077+0000","last_clean":"2024-04-25T00:32:33.441077+0000","last_became_active":"2024-04-25T00:32:33.432236+0000","last_became_peered":"2024-04-25T00:32:33.432236+0000","last_unstale":"2024-04-25T00:32:33.441077+0000","last_undegraded":"2024-04-25T00:32:33.441077+0000","last_fullsized":"2024-04-25T00:32:33.441077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:03.570347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403517+0000","last_change":"2024-04-25T00:32:33.403517+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403517+0000","last_undegraded":"2024-04-25T00:32:33.403517+0000","last_fullsized":"2024-04-25T00:32:33.403517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.432184+0000","last_change":"2024-04-25T00:32:33.431456+0000","last_active":"2024-04-25T00:32:33.432184+0000","last_peered":"2024-04-25T00:32:33.432184+0000","last_clean":"2024-04-25T00:32:33.432184+0000","last_became_active":"2024-04-25T00:32:33.431235+0000","last_became_peered":"2024-04-25T00:32:33.431235+0000","last_unstale":"2024-04-25T00:32:33.432184+0000","last_undegraded":"2024-04-25T00:32:33.432184+0000","last_fullsized":"2024-04-25T00:32:33.432184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:48.933581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.409317+0000","last_change":"2024-04-25T00:32:33.409317+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.409317+0000","last_undegraded":"2024-04-25T00:32:33.409317+0000","last_fullsized":"2024-04-25T00:32:33.409317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.417776+0000","last_change":"2024-04-25T00:32:33.416642+0000","last_active":"2024-04-25T00:32:33.417776+0000","last_peered":"2024-04-25T00:32:33.417776+0000","last_clean":"2024-04-25T00:32:33.417776+0000","last_became_active":"2024-04-25T00:32:33.416316+0000","last_became_peered":"2024-04-25T00:32:33.416316+0000","last_unstale":"2024-04-25T00:32:33.417776+0000","last_undegraded":"2024-04-25T00:32:33.417776+0000","last_fullsized":"2024-04-25T00:32:33.417776+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:05.605371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.405584+0000","last_change":"2024-04-25T00:32:33.405584+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.405584+0000","last_undegraded":"2024-04-25T00:32:33.405584+0000","last_fullsized":"2024-04-25T00:32:33.405584+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.429349+0000","last_change":"2024-04-25T00:32:33.424885+0000","last_active":"2024-04-25T00:32:33.429349+0000","last_peered":"2024-04-25T00:32:33.429349+0000","last_clean":"2024-04-25T00:32:33.429349+0000","last_became_active":"2024-04-25T00:32:33.424174+0000","last_became_peered":"2024-04-25T00:32:33.424174+0000","last_unstale":"2024-04-25T00:32:33.429349+0000","last_undegraded":"2024-04-25T00:32:33.429349+0000","last_fullsized":"2024-04-25T00:32:33.429349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:28.291849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402897+0000","last_change":"2024-04-25T00:32:33.402897+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402897+0000","last_undegraded":"2024-04-25T00:32:33.402897+0000","last_fullsized":"2024-04-25T00:32:33.402897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422426+0000","last_change":"2024-04-25T00:32:33.420265+0000","last_active":"2024-04-25T00:32:33.422426+0000","last_peered":"2024-04-25T00:32:33.422426+0000","last_clean":"2024-04-25T00:32:33.422426+0000","last_became_active":"2024-04-25T00:32:33.419977+0000","last_became_peered":"2024-04-25T00:32:33.419977+0000","last_unstale":"2024-04-25T00:32:33.422426+0000","last_undegraded":"2024-04-25T00:32:33.422426+0000","last_fullsized":"2024-04-25T00:32:33.422426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:34.696377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402398+0000","last_change":"2024-04-25T00:32:33.402398+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402398+0000","last_undegraded":"2024-04-25T00:32:33.402398+0000","last_fullsized":"2024-04-25T00:32:33.402398+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.442465+0000","last_change":"2024-04-25T00:32:33.434692+0000","last_active":"2024-04-25T00:32:33.442465+0000","last_peered":"2024-04-25T00:32:33.442465+0000","last_clean":"2024-04-25T00:32:33.442465+0000","last_became_active":"2024-04-25T00:32:33.433973+0000","last_became_peered":"2024-04-25T00:32:33.433973+0000","last_unstale":"2024-04-25T00:32:33.442465+0000","last_undegraded":"2024-04-25T00:32:33.442465+0000","last_fullsized":"2024-04-25T00:32:33.442465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:30.363273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403586+0000","last_change":"2024-04-25T00:32:33.403586+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403586+0000","last_undegraded":"2024-04-25T00:32:33.403586+0000","last_fullsized":"2024-04-25T00:32:33.403586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.432758+0000","last_change":"2024-04-25T00:32:33.432224+0000","last_active":"2024-04-25T00:32:33.432758+0000","last_peered":"2024-04-25T00:32:33.432758+0000","last_clean":"2024-04-25T00:32:33.432758+0000","last_became_active":"2024-04-25T00:32:33.431863+0000","last_became_peered":"2024-04-25T00:32:33.431863+0000","last_unstale":"2024-04-25T00:32:33.432758+0000","last_undegraded":"2024-04-25T00:32:33.432758+0000","last_fullsized":"2024-04-25T00:32:33.432758+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:33.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.425836+0000","last_change":"2024-04-25T00:32:33.425836+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.425836+0000","last_undegraded":"2024-04-25T00:32:33.425836+0000","last_fullsized":"2024-04-25T00:32:33.425836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.432174+0000","last_change":"2024-04-25T00:32:33.431495+0000","last_active":"2024-04-25T00:32:33.432174+0000","last_peered":"2024-04-25T00:32:33.432174+0000","last_clean":"2024-04-25T00:32:33.432174+0000","last_became_active":"2024-04-25T00:32:33.431231+0000","last_became_peered":"2024-04-25T00:32:33.431231+0000","last_unstale":"2024-04-25T00:32:33.432174+0000","last_undegraded":"2024-04-25T00:32:33.432174+0000","last_fullsized":"2024-04-25T00:32:33.432174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:28.291849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.416393+0000","last_change":"2024-04-25T00:32:33.416393+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.416393+0000","last_undegraded":"2024-04-25T00:32:33.416393+0000","last_fullsized":"2024-04-25T00:32:33.416393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422373+0000","last_change":"2024-04-25T00:32:33.419970+0000","last_active":"2024-04-25T00:32:33.422373+0000","last_peered":"2024-04-25T00:32:33.422373+0000","last_clean":"2024-04-25T00:32:33.422373+0000","last_became_active":"2024-04-25T00:32:33.419465+0000","last_became_peered":"2024-04-25T00:32:33.419465+0000","last_unstale":"2024-04-25T00:32:33.422373+0000","last_undegraded":"2024-04-25T00:32:33.422373+0000","last_fullsized":"2024-04-25T00:32:33.422373+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.651396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.405505+0000","last_change":"2024-04-25T00:32:33.405505+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.405505+0000","last_undegraded":"2024-04-25T00:32:33.405505+0000","last_fullsized":"2024-04-25T00:32:33.405505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.421058+0000","last_change":"2024-04-25T00:32:33.418814+0000","last_active":"2024-04-25T00:32:33.421058+0000","last_peered":"2024-04-25T00:32:33.421058+0000","last_clean":"2024-04-25T00:32:33.421058+0000","last_became_active":"2024-04-25T00:32:33.418270+0000","last_became_peered":"2024-04-25T00:32:33.418270+0000","last_unstale":"2024-04-25T00:32:33.421058+0000","last_undegraded":"2024-04-25T00:32:33.421058+0000","last_fullsized":"2024-04-25T00:32:33.421058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:18.360499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.409511+0000","last_change":"2024-04-25T00:32:33.409511+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.409511+0000","last_undegraded":"2024-04-25T00:32:33.409511+0000","last_fullsized":"2024-04-25T00:32:33.409511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.442615+0000","last_change":"2024-04-25T00:32:33.434714+0000","last_active":"2024-04-25T00:32:33.442615+0000","last_peered":"2024-04-25T00:32:33.442615+0000","last_clean":"2024-04-25T00:32:33.442615+0000","last_became_active":"2024-04-25T00:32:33.433950+0000","last_became_peered":"2024-04-25T00:32:33.433950+0000","last_unstale":"2024-04-25T00:32:33.442615+0000","last_undegraded":"2024-04-25T00:32:33.442615+0000","last_fullsized":"2024-04-25T00:32:33.442615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:40.127761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402374+0000","last_change":"2024-04-25T00:32:33.402374+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402374+0000","last_undegraded":"2024-04-25T00:32:33.402374+0000","last_fullsized":"2024-04-25T00:32:33.402374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422095+0000","last_change":"2024-04-25T00:32:33.412615+0000","last_active":"2024-04-25T00:32:33.422095+0000","last_peered":"2024-04-25T00:32:33.422095+0000","last_clean":"2024-04-25T00:32:33.422095+0000","last_became_active":"2024-04-25T00:32:33.411820+0000","last_became_peered":"2024-04-25T00:32:33.411820+0000","last_unstale":"2024-04-25T00:32:33.422095+0000","last_undegraded":"2024-04-25T00:32:33.422095+0000","last_fullsized":"2024-04-25T00:32:33.422095+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:00.041094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403654+0000","last_change":"2024-04-25T00:32:33.403654+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403654+0000","last_undegraded":"2024-04-25T00:32:33.403654+0000","last_fullsized":"2024-04-25T00:32:33.403654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.433365+0000","last_change":"2024-04-25T00:32:33.429101+0000","last_active":"2024-04-25T00:32:33.433365+0000","last_peered":"2024-04-25T00:32:33.433365+0000","last_clean":"2024-04-25T00:32:33.433365+0000","last_became_active":"2024-04-25T00:32:33.428344+0000","last_became_peered":"2024-04-25T00:32:33.428344+0000","last_unstale":"2024-04-25T00:32:33.433365+0000","last_undegraded":"2024-04-25T00:32:33.433365+0000","last_fullsized":"2024-04-25T00:32:33.433365+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:15.470226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403525+0000","last_change":"2024-04-25T00:32:33.403525+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403525+0000","last_undegraded":"2024-04-25T00:32:33.403525+0000","last_fullsized":"2024-04-25T00:32:33.403525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.420940+0000","last_change":"2024-04-25T00:32:33.419832+0000","last_active":"2024-04-25T00:32:33.420940+0000","last_peered":"2024-04-25T00:32:33.420940+0000","last_clean":"2024-04-25T00:32:33.420940+0000","last_became_active":"2024-04-25T00:32:33.419551+0000","last_became_peered":"2024-04-25T00:32:33.419551+0000","last_unstale":"2024-04-25T00:32:33.420940+0000","last_undegraded":"2024-04-25T00:32:33.420940+0000","last_fullsized":"2024-04-25T00:32:33.420940+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:18.360499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403524+0000","last_change":"2024-04-25T00:32:33.403524+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403524+0000","last_undegraded":"2024-04-25T00:32:33.403524+0000","last_fullsized":"2024-04-25T00:32:33.403524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.434530+0000","last_change":"2024-04-25T00:32:33.432483+0000","last_active":"2024-04-25T00:32:33.434530+0000","last_peered":"2024-04-25T00:32:33.434530+0000","last_clean":"2024-04-25T00:32:33.434530+0000","last_became_active":"2024-04-25T00:32:33.431774+0000","last_became_peered":"2024-04-25T00:32:33.431774+0000","last_unstale":"2024-04-25T00:32:33.434530+0000","last_undegraded":"2024-04-25T00:32:33.434530+0000","last_fullsized":"2024-04-25T00:32:33.434530+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:39.207154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404371+0000","last_change":"2024-04-25T00:32:33.404371+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404371+0000","last_undegraded":"2024-04-25T00:32:33.404371+0000","last_fullsized":"2024-04-25T00:32:33.404371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.429004+0000","last_change":"2024-04-25T00:32:33.426645+0000","last_active":"2024-04-25T00:32:33.429004+0000","last_peered":"2024-04-25T00:32:33.429004+0000","last_clean":"2024-04-25T00:32:33.429004+0000","last_became_active":"2024-04-25T00:32:33.425875+0000","last_became_peered":"2024-04-25T00:32:33.425875+0000","last_unstale":"2024-04-25T00:32:33.429004+0000","last_undegraded":"2024-04-25T00:32:33.429004+0000","last_fullsized":"2024-04-25T00:32:33.429004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:54.900021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.416854+0000","last_change":"2024-04-25T00:32:33.416854+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.416854+0000","last_undegraded":"2024-04-25T00:32:33.416854+0000","last_fullsized":"2024-04-25T00:32:33.416854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.443276+0000","last_change":"2024-04-25T00:32:33.441614+0000","last_active":"2024-04-25T00:32:33.443276+0000","last_peered":"2024-04-25T00:32:33.443276+0000","last_clean":"2024-04-25T00:32:33.443276+0000","last_became_active":"2024-04-25T00:32:33.441004+0000","last_became_peered":"2024-04-25T00:32:33.441004+0000","last_unstale":"2024-04-25T00:32:33.443276+0000","last_undegraded":"2024-04-25T00:32:33.443276+0000","last_fullsized":"2024-04-25T00:32:33.443276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:32.656831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404823+0000","last_change":"2024-04-25T00:32:33.404823+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404823+0000","last_undegraded":"2024-04-25T00:32:33.404823+0000","last_fullsized":"2024-04-25T00:32:33.404823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.418691+0000","last_change":"2024-04-25T00:32:33.412949+0000","last_active":"2024-04-25T00:32:33.418691+0000","last_peered":"2024-04-25T00:32:33.418691+0000","last_clean":"2024-04-25T00:32:33.418691+0000","last_became_active":"2024-04-25T00:32:33.412675+0000","last_became_peered":"2024-04-25T00:32:33.412675+0000","last_unstale":"2024-04-25T00:32:33.418691+0000","last_undegraded":"2024-04-25T00:32:33.418691+0000","last_fullsized":"2024-04-25T00:32:33.418691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:39.207154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.408491+0000","last_change":"2024-04-25T00:32:33.408491+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.408491+0000","last_undegraded":"2024-04-25T00:32:33.408491+0000","last_fullsized":"2024-04-25T00:32:33.408491+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.421639+0000","last_change":"2024-04-25T00:32:33.420075+0000","last_active":"2024-04-25T00:32:33.421639+0000","last_peered":"2024-04-25T00:32:33.421639+0000","last_clean":"2024-04-25T00:32:33.421639+0000","last_became_active":"2024-04-25T00:32:33.419783+0000","last_became_peered":"2024-04-25T00:32:33.419783+0000","last_unstale":"2024-04-25T00:32:33.421639+0000","last_undegraded":"2024-04-25T00:32:33.421639+0000","last_fullsized":"2024-04-25T00:32:33.421639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:33.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.407661+0000","last_change":"2024-04-25T00:32:33.407661+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.407661+0000","last_undegraded":"2024-04-25T00:32:33.407661+0000","last_fullsized":"2024-04-25T00:32:33.407661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.418654+0000","last_change":"2024-04-25T00:32:33.411847+0000","last_active":"2024-04-25T00:32:33.418654+0000","last_peered":"2024-04-25T00:32:33.418654+0000","last_clean":"2024-04-25T00:32:33.418654+0000","last_became_active":"2024-04-25T00:32:33.411429+0000","last_became_peered":"2024-04-25T00:32:33.411429+0000","last_unstale":"2024-04-25T00:32:33.418654+0000","last_undegraded":"2024-04-25T00:32:33.418654+0000","last_fullsized":"2024-04-25T00:32:33.418654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:25.652624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.420253+0000","last_change":"2024-04-25T00:32:33.420253+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.420253+0000","last_undegraded":"2024-04-25T00:32:33.420253+0000","last_fullsized":"2024-04-25T00:32:33.420253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.418712+0000","last_change":"2024-04-25T00:32:33.417550+0000","last_active":"2024-04-25T00:32:33.418712+0000","last_peered":"2024-04-25T00:32:33.418712+0000","last_clean":"2024-04-25T00:32:33.418712+0000","last_became_active":"2024-04-25T00:32:33.417040+0000","last_became_peered":"2024-04-25T00:32:33.417040+0000","last_unstale":"2024-04-25T00:32:33.418712+0000","last_undegraded":"2024-04-25T00:32:33.418712+0000","last_fullsized":"2024-04-25T00:32:33.418712+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:39.207154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.408161+0000","last_change":"2024-04-25T00:32:33.408161+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.408161+0000","last_undegraded":"2024-04-25T00:32:33.408161+0000","last_fullsized":"2024-04-25T00:32:33.408161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.418721+0000","last_change":"2024-04-25T00:32:33.412644+0000","last_active":"2024-04-25T00:32:33.418721+0000","last_peered":"2024-04-25T00:32:33.418721+0000","last_clean":"2024-04-25T00:32:33.418721+0000","last_became_active":"2024-04-25T00:32:33.412193+0000","last_became_peered":"2024-04-25T00:32:33.412193+0000","last_unstale":"2024-04-25T00:32:33.418721+0000","last_undegraded":"2024-04-25T00:32:33.418721+0000","last_fullsized":"2024-04-25T00:32:33.418721+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:05.605371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.406800+0000","last_change":"2024-04-25T00:32:33.406800+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.406800+0000","last_undegraded":"2024-04-25T00:32:33.406800+0000","last_fullsized":"2024-04-25T00:32:33.406800+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.421360+0000","last_change":"2024-04-25T00:32:33.416074+0000","last_active":"2024-04-25T00:32:33.421360+0000","last_peered":"2024-04-25T00:32:33.421360+0000","last_clean":"2024-04-25T00:32:33.421360+0000","last_became_active":"2024-04-25T00:32:33.415336+0000","last_became_peered":"2024-04-25T00:32:33.415336+0000","last_unstale":"2024-04-25T00:32:33.421360+0000","last_undegraded":"2024-04-25T00:32:33.421360+0000","last_fullsized":"2024-04-25T00:32:33.421360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:19.436849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410397+0000","last_change":"2024-04-25T00:32:33.410397+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410397+0000","last_undegraded":"2024-04-25T00:32:33.410397+0000","last_fullsized":"2024-04-25T00:32:33.410397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.441555+0000","last_change":"2024-04-25T00:32:33.440955+0000","last_active":"2024-04-25T00:32:33.441555+0000","last_peered":"2024-04-25T00:32:33.441555+0000","last_clean":"2024-04-25T00:32:33.441555+0000","last_became_active":"2024-04-25T00:32:33.440603+0000","last_became_peered":"2024-04-25T00:32:33.440603+0000","last_unstale":"2024-04-25T00:32:33.441555+0000","last_undegraded":"2024-04-25T00:32:33.441555+0000","last_fullsized":"2024-04-25T00:32:33.441555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:32.656831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.411828+0000","last_change":"2024-04-25T00:32:33.411828+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.411828+0000","last_undegraded":"2024-04-25T00:32:33.411828+0000","last_fullsized":"2024-04-25T00:32:33.411828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.436571+0000","last_change":"2024-04-25T00:32:33.432559+0000","last_active":"2024-04-25T00:32:33.436571+0000","last_peered":"2024-04-25T00:32:33.436571+0000","last_clean":"2024-04-25T00:32:33.436571+0000","last_became_active":"2024-04-25T00:32:33.431829+0000","last_became_peered":"2024-04-25T00:32:33.431829+0000","last_unstale":"2024-04-25T00:32:33.436571+0000","last_undegraded":"2024-04-25T00:32:33.436571+0000","last_fullsized":"2024-04-25T00:32:33.436571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:18.360499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.436634+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:32:33.436634+0000","last_peered":"2024-04-25T00:32:33.436634+0000","last_clean":"2024-04-25T00:32:33.436634+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:32:33.436634+0000","last_undegraded":"2024-04-25T00:32:33.436634+0000","last_fullsized":"2024-04-25T00:32:33.436634+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":724249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":348160,"data_stored":133881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":724249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:32:38.491 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:32:38.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.643+0000 7f8b54f4d640 1 Processor -- start 2024-04-25T00:32:38.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.643+0000 7f8b54f4d640 1 -- start start 2024-04-25T00:32:38.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.647+0000 7f8b54f4d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b5006dce0 0x7f8b50150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.647+0000 7f8b54f4d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b500b2b70 0x7f8b50151330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.647+0000 7f8b54f4d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b500b3130 con 0x7f8b5015d980 2024-04-25T00:32:38.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.647+0000 7f8b54f4d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8b500cf560 con 0x7f8b5006dce0 2024-04-25T00:32:38.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.647+0000 7f8b54f4d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b500b3410 con 0x7f8b500b2b70 2024-04-25T00:32:38.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.647+0000 7f8b4ed9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b500b2b70 0x7f8b50151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.647+0000 7f8b45d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b5006dce0 0x7f8b50150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.647+0000 7f8b4ed9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b500b2b70 0x7f8b50151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:50336/0 (socket says 172.21.15.29:50336) 2024-04-25T00:32:38.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.647+0000 7f8b4ed9d640 1 -- 172.21.15.29:0/2069792579 learned_addr learned my addr 172.21.15.29:0/2069792579 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:38.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.647+0000 7f8b4ed9d640 1 -- 172.21.15.29:0/2069792579 >> v2:172.21.15.157:3300/0 conn(0x7f8b5006dce0 msgr2=0x7f8b50150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:38.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.647+0000 7f8b4ed9d640 1 --2- 172.21.15.29:0/2069792579 >> v2:172.21.15.157:3300/0 conn(0x7f8b5006dce0 0x7f8b50150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.647+0000 7f8b4ed9d640 1 -- 172.21.15.29:0/2069792579 >> v1:172.21.15.157:6789/0 conn(0x7f8b5015d980 legacy=0x7f8b5006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.647+0000 7f8b4ed9d640 1 -- 172.21.15.29:0/2069792579 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b500b36f0 con 0x7f8b500b2b70 2024-04-25T00:32:38.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b4ed9d640 1 --2- 172.21.15.29:0/2069792579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b500b2b70 0x7f8b50151330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800af60 tx=0x7f8b4801db00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf9cdf1849856c08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:38.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b4dd9b640 1 -- 172.21.15.29:0/2069792579 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4800a2d0 con 0x7f8b500b2b70 2024-04-25T00:32:38.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b4dd9b640 1 -- 172.21.15.29:0/2069792579 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b480a6940 con 0x7f8b500b2b70 2024-04-25T00:32:38.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b4dd9b640 1 -- 172.21.15.29:0/2069792579 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b480a6d10 con 0x7f8b500b2b70 2024-04-25T00:32:38.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 -- 172.21.15.29:0/2069792579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b500b2b70 msgr2=0x7f8b50151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 --2- 172.21.15.29:0/2069792579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b500b2b70 0x7f8b50151330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800af60 tx=0x7f8b4801db00 comp rx=0 tx=0).stop 2024-04-25T00:32:38.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 -- 172.21.15.29:0/2069792579 shutdown_connections 2024-04-25T00:32:38.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 --2- 172.21.15.29:0/2069792579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b500b2b70 0x7f8b50151330 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 --2- 172.21.15.29:0/2069792579 >> v2:172.21.15.157:3300/0 conn(0x7f8b5006dce0 0x7f8b50150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 -- 172.21.15.29:0/2069792579 >> 172.21.15.29:0/2069792579 conn(0x7f8b500b1870 msgr2=0x7f8b500b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:38.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 -- 172.21.15.29:0/2069792579 shutdown_connections 2024-04-25T00:32:38.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 -- 172.21.15.29:0/2069792579 wait complete. 2024-04-25T00:32:38.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 Processor -- start 2024-04-25T00:32:38.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 -- start start 2024-04-25T00:32:38.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b5006dce0 0x7f8b5013d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b500b2b70 0x7f8b500cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b5015d980 0x7f8b500ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5015b4b0 con 0x7f8b500b2b70 2024-04-25T00:32:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50151870 con 0x7f8b5015d980 2024-04-25T00:32:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b5015b630 con 0x7f8b5006dce0 2024-04-25T00:32:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b45d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b500b2b70 0x7f8b500cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b45d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b500b2b70 0x7f8b500cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54212/0 (socket says 172.21.15.29:54212) 2024-04-25T00:32:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b45d9b640 1 -- 172.21.15.29:0/473574313 learned_addr learned my addr 172.21.15.29:0/473574313 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b4ed9d640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b5015d980 0x7f8b500ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b4e59c640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b5006dce0 0x7f8b5013d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b45d9b640 1 -- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b5006dce0 msgr2=0x7f8b5013d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b45d9b640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b5006dce0 0x7f8b5013d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b45d9b640 1 -- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b5015d980 msgr2=0x7f8b500ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b45d9b640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b5015d980 0x7f8b500ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b45d9b640 1 -- 172.21.15.29:0/473574313 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5014fde0 con 0x7f8b500b2b70 2024-04-25T00:32:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b45d9b640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b500b2b70 0x7f8b500cc870 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800e930 tx=0x7f8b3800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b477fe640 1 -- 172.21.15.29:0/473574313 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38017070 con 0x7f8b500b2b70 2024-04-25T00:32:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b477fe640 1 -- 172.21.15.29:0/473574313 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3800ca00 con 0x7f8b500b2b70 2024-04-25T00:32:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 -- 172.21.15.29:0/473574313 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5014ed40 con 0x7f8b500b2b70 2024-04-25T00:32:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b477fe640 1 -- 172.21.15.29:0/473574313 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3800cdb0 con 0x7f8b500b2b70 2024-04-25T00:32:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.651+0000 7f8b54f4d640 1 -- 172.21.15.29:0/473574313 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5014f300 con 0x7f8b500b2b70 2024-04-25T00:32:38.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.655+0000 7f8b54f4d640 1 -- 172.21.15.29:0/473574313 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b50069270 con 0x7f8b500b2b70 2024-04-25T00:32:38.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.655+0000 7f8b477fe640 1 -- 172.21.15.29:0/473574313 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8b3801a070 con 0x7f8b500b2b70 2024-04-25T00:32:38.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.655+0000 7f8b477fe640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b300678f0 0x7f8b30069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:38.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.655+0000 7f8b477fe640 1 -- 172.21.15.29:0/473574313 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8b38085d20 con 0x7f8b500b2b70 2024-04-25T00:32:38.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.655+0000 7f8b4e59c640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b300678f0 0x7f8b30069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:38.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.655+0000 7f8b4e59c640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b300678f0 0x7f8b30069db0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b400015b0 tx=0x7f8b40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:38.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.659+0000 7f8b477fe640 1 -- 172.21.15.29:0/473574313 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8b380568a0 con 0x7f8b500b2b70 2024-04-25T00:32:38.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.811+0000 7f8b54f4d640 1 -- 172.21.15.29:0/473574313 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b5006e1d0 con 0x7f8b300678f0 2024-04-25T00:32:38.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.831+0000 7f8b477fe640 1 -- 172.21.15.29:0/473574313 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7f8b5006e1d0 con 0x7f8b300678f0 2024-04-25T00:32:38.833 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:38.835 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:32:38.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.831+0000 7f8b54f4d640 1 -- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b300678f0 msgr2=0x7f8b30069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.831+0000 7f8b54f4d640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b300678f0 0x7f8b30069db0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b400015b0 tx=0x7f8b40009040 comp rx=0 tx=0).stop 2024-04-25T00:32:38.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.831+0000 7f8b54f4d640 1 -- 172.21.15.29:0/473574313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b500b2b70 msgr2=0x7f8b500cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:38.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.831+0000 7f8b54f4d640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b500b2b70 0x7f8b500cc870 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800e930 tx=0x7f8b3800edf0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.831+0000 7f8b54f4d640 1 -- 172.21.15.29:0/473574313 shutdown_connections 2024-04-25T00:32:38.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.831+0000 7f8b54f4d640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b300678f0 0x7f8b30069db0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.831+0000 7f8b54f4d640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b5015d980 0x7f8b500ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.831+0000 7f8b54f4d640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b500b2b70 0x7f8b500cc870 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.831+0000 7f8b54f4d640 1 --2- 172.21.15.29:0/473574313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b5006dce0 0x7f8b5013d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:38.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.831+0000 7f8b54f4d640 1 -- 172.21.15.29:0/473574313 >> 172.21.15.29:0/473574313 conn(0x7f8b500b1870 msgr2=0x7f8b50151110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:38.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.831+0000 7f8b54f4d640 1 -- 172.21.15.29:0/473574313 shutdown_connections 2024-04-25T00:32:38.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:38.831+0000 7f8b54f4d640 1 -- 172.21.15.29:0/473574313 wait complete. 2024-04-25T00:32:38.851 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-04-25T00:32:37.466967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253272,"kb_used_data":3820,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721448,"statfs":{"total":773094113280,"available":772834762752,"internally_reserved":0,"allocated":3911680,"data_stored":2220449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.075330"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.421587+0000","last_change":"2024-04-25T00:32:33.421587+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.421587+0000","last_undegraded":"2024-04-25T00:32:33.421587+0000","last_fullsized":"2024-04-25T00:32:33.421587+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.416958+0000","last_change":"2024-04-25T00:32:33.416958+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.416958+0000","last_undegraded":"2024-04-25T00:32:33.416958+0000","last_fullsized":"2024-04-25T00:32:33.416958+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.401783+0000","last_change":"2024-04-25T00:32:33.401783+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.401783+0000","last_undegraded":"2024-04-25T00:32:33.401783+0000","last_fullsized":"2024-04-25T00:32:33.401783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410014+0000","last_change":"2024-04-25T00:32:33.410014+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410014+0000","last_undegraded":"2024-04-25T00:32:33.410014+0000","last_fullsized":"2024-04-25T00:32:33.410014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.406582+0000","last_change":"2024-04-25T00:32:33.406582+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.406582+0000","last_undegraded":"2024-04-25T00:32:33.406582+0000","last_fullsized":"2024-04-25T00:32:33.406582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.416330+0000","last_change":"2024-04-25T00:32:33.416330+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.416330+0000","last_undegraded":"2024-04-25T00:32:33.416330+0000","last_fullsized":"2024-04-25T00:32:33.416330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.415789+0000","last_change":"2024-04-25T00:32:33.415789+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.415789+0000","last_undegraded":"2024-04-25T00:32:33.415789+0000","last_fullsized":"2024-04-25T00:32:33.415789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.428371+0000","last_change":"2024-04-25T00:32:33.428371+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.428371+0000","last_undegraded":"2024-04-25T00:32:33.428371+0000","last_fullsized":"2024-04-25T00:32:33.428371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402082+0000","last_change":"2024-04-25T00:32:33.402082+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402082+0000","last_undegraded":"2024-04-25T00:32:33.402082+0000","last_fullsized":"2024-04-25T00:32:33.402082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410001+0000","last_change":"2024-04-25T00:32:33.410001+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410001+0000","last_undegraded":"2024-04-25T00:32:33.410001+0000","last_fullsized":"2024-04-25T00:32:33.410001+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.411695+0000","last_change":"2024-04-25T00:32:33.411695+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.411695+0000","last_undegraded":"2024-04-25T00:32:33.411695+0000","last_fullsized":"2024-04-25T00:32:33.411695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402278+0000","last_change":"2024-04-25T00:32:33.402278+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402278+0000","last_undegraded":"2024-04-25T00:32:33.402278+0000","last_fullsized":"2024-04-25T00:32:33.402278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.406087+0000","last_change":"2024-04-25T00:32:33.406087+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.406087+0000","last_undegraded":"2024-04-25T00:32:33.406087+0000","last_fullsized":"2024-04-25T00:32:33.406087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.412316+0000","last_change":"2024-04-25T00:32:33.412316+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.412316+0000","last_undegraded":"2024-04-25T00:32:33.412316+0000","last_fullsized":"2024-04-25T00:32:33.412316+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402227+0000","last_change":"2024-04-25T00:32:33.402227+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402227+0000","last_undegraded":"2024-04-25T00:32:33.402227+0000","last_fullsized":"2024-04-25T00:32:33.402227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.412924+0000","last_change":"2024-04-25T00:32:33.412924+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.412924+0000","last_undegraded":"2024-04-25T00:32:33.412924+0000","last_fullsized":"2024-04-25T00:32:33.412924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.413406+0000","last_change":"2024-04-25T00:32:33.413406+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.413406+0000","last_undegraded":"2024-04-25T00:32:33.413406+0000","last_fullsized":"2024-04-25T00:32:33.413406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.408069+0000","last_change":"2024-04-25T00:32:33.408069+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.408069+0000","last_undegraded":"2024-04-25T00:32:33.408069+0000","last_fullsized":"2024-04-25T00:32:33.408069+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404172+0000","last_change":"2024-04-25T00:32:33.404172+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404172+0000","last_undegraded":"2024-04-25T00:32:33.404172+0000","last_fullsized":"2024-04-25T00:32:33.404172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.405470+0000","last_change":"2024-04-25T00:32:33.405470+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.405470+0000","last_undegraded":"2024-04-25T00:32:33.405470+0000","last_fullsized":"2024-04-25T00:32:33.405470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.408165+0000","last_change":"2024-04-25T00:32:33.408165+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.408165+0000","last_undegraded":"2024-04-25T00:32:33.408165+0000","last_fullsized":"2024-04-25T00:32:33.408165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.400602+0000","last_change":"2024-04-25T00:32:33.400602+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.400602+0000","last_undegraded":"2024-04-25T00:32:33.400602+0000","last_fullsized":"2024-04-25T00:32:33.400602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.405760+0000","last_change":"2024-04-25T00:32:33.405760+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.405760+0000","last_undegraded":"2024-04-25T00:32:33.405760+0000","last_fullsized":"2024-04-25T00:32:33.405760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402840+0000","last_change":"2024-04-25T00:32:33.402840+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402840+0000","last_undegraded":"2024-04-25T00:32:33.402840+0000","last_fullsized":"2024-04-25T00:32:33.402840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410219+0000","last_change":"2024-04-25T00:32:33.410219+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410219+0000","last_undegraded":"2024-04-25T00:32:33.410219+0000","last_fullsized":"2024-04-25T00:32:33.410219+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.407507+0000","last_change":"2024-04-25T00:32:33.407507+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.407507+0000","last_undegraded":"2024-04-25T00:32:33.407507+0000","last_fullsized":"2024-04-25T00:32:33.407507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403049+0000","last_change":"2024-04-25T00:32:33.403049+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403049+0000","last_undegraded":"2024-04-25T00:32:33.403049+0000","last_fullsized":"2024-04-25T00:32:33.403049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.400778+0000","last_change":"2024-04-25T00:32:33.400778+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.400778+0000","last_undegraded":"2024-04-25T00:32:33.400778+0000","last_fullsized":"2024-04-25T00:32:33.400778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.406659+0000","last_change":"2024-04-25T00:32:33.406659+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.406659+0000","last_undegraded":"2024-04-25T00:32:33.406659+0000","last_fullsized":"2024-04-25T00:32:33.406659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.411758+0000","last_change":"2024-04-25T00:32:33.411758+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.411758+0000","last_undegraded":"2024-04-25T00:32:33.411758+0000","last_fullsized":"2024-04-25T00:32:33.411758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.417701+0000","last_change":"2024-04-25T00:32:33.417701+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.417701+0000","last_undegraded":"2024-04-25T00:32:33.417701+0000","last_fullsized":"2024-04-25T00:32:33.417701+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.426820+0000","last_change":"2024-04-25T00:32:33.426820+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.426820+0000","last_undegraded":"2024-04-25T00:32:33.426820+0000","last_fullsized":"2024-04-25T00:32:33.426820+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410800+0000","last_change":"2024-04-25T00:32:33.410800+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410800+0000","last_undegraded":"2024-04-25T00:32:33.410800+0000","last_fullsized":"2024-04-25T00:32:33.410800+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.440516+0000","last_change":"2024-04-25T00:32:33.439097+0000","last_active":"2024-04-25T00:32:33.440516+0000","last_peered":"2024-04-25T00:32:33.440516+0000","last_clean":"2024-04-25T00:32:33.440516+0000","last_became_active":"2024-04-25T00:32:33.438390+0000","last_became_peered":"2024-04-25T00:32:33.438390+0000","last_unstale":"2024-04-25T00:32:33.440516+0000","last_undegraded":"2024-04-25T00:32:33.440516+0000","last_fullsized":"2024-04-25T00:32:33.440516+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:33.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.419679+0000","last_change":"2024-04-25T00:32:33.419679+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.419679+0000","last_undegraded":"2024-04-25T00:32:33.419679+0000","last_fullsized":"2024-04-25T00:32:33.419679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.440528+0000","last_change":"2024-04-25T00:32:33.439749+0000","last_active":"2024-04-25T00:32:33.440528+0000","last_peered":"2024-04-25T00:32:33.440528+0000","last_clean":"2024-04-25T00:32:33.440528+0000","last_became_active":"2024-04-25T00:32:33.439548+0000","last_became_peered":"2024-04-25T00:32:33.439548+0000","last_unstale":"2024-04-25T00:32:33.440528+0000","last_undegraded":"2024-04-25T00:32:33.440528+0000","last_fullsized":"2024-04-25T00:32:33.440528+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:10.605816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404042+0000","last_change":"2024-04-25T00:32:33.404042+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404042+0000","last_undegraded":"2024-04-25T00:32:33.404042+0000","last_fullsized":"2024-04-25T00:32:33.404042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422465+0000","last_change":"2024-04-25T00:32:33.419772+0000","last_active":"2024-04-25T00:32:33.422465+0000","last_peered":"2024-04-25T00:32:33.422465+0000","last_clean":"2024-04-25T00:32:33.422465+0000","last_became_active":"2024-04-25T00:32:33.419459+0000","last_became_peered":"2024-04-25T00:32:33.419459+0000","last_unstale":"2024-04-25T00:32:33.422465+0000","last_undegraded":"2024-04-25T00:32:33.422465+0000","last_fullsized":"2024-04-25T00:32:33.422465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:24.119844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402369+0000","last_change":"2024-04-25T00:32:33.402369+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402369+0000","last_undegraded":"2024-04-25T00:32:33.402369+0000","last_fullsized":"2024-04-25T00:32:33.402369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.420930+0000","last_change":"2024-04-25T00:32:33.419126+0000","last_active":"2024-04-25T00:32:33.420930+0000","last_peered":"2024-04-25T00:32:33.420930+0000","last_clean":"2024-04-25T00:32:33.420930+0000","last_became_active":"2024-04-25T00:32:33.418834+0000","last_became_peered":"2024-04-25T00:32:33.418834+0000","last_unstale":"2024-04-25T00:32:33.420930+0000","last_undegraded":"2024-04-25T00:32:33.420930+0000","last_fullsized":"2024-04-25T00:32:33.420930+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:39.513615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410063+0000","last_change":"2024-04-25T00:32:33.410063+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410063+0000","last_undegraded":"2024-04-25T00:32:33.410063+0000","last_fullsized":"2024-04-25T00:32:33.410063+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.433292+0000","last_change":"2024-04-25T00:32:33.429088+0000","last_active":"2024-04-25T00:32:33.433292+0000","last_peered":"2024-04-25T00:32:33.433292+0000","last_clean":"2024-04-25T00:32:33.433292+0000","last_became_active":"2024-04-25T00:32:33.427379+0000","last_became_peered":"2024-04-25T00:32:33.427379+0000","last_unstale":"2024-04-25T00:32:33.433292+0000","last_undegraded":"2024-04-25T00:32:33.433292+0000","last_fullsized":"2024-04-25T00:32:33.433292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:05.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403511+0000","last_change":"2024-04-25T00:32:33.403511+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403511+0000","last_undegraded":"2024-04-25T00:32:33.403511+0000","last_fullsized":"2024-04-25T00:32:33.403511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.441233+0000","last_change":"2024-04-25T00:32:33.429965+0000","last_active":"2024-04-25T00:32:33.441233+0000","last_peered":"2024-04-25T00:32:33.441233+0000","last_clean":"2024-04-25T00:32:33.441233+0000","last_became_active":"2024-04-25T00:32:33.427356+0000","last_became_peered":"2024-04-25T00:32:33.427356+0000","last_unstale":"2024-04-25T00:32:33.441233+0000","last_undegraded":"2024-04-25T00:32:33.441233+0000","last_fullsized":"2024-04-25T00:32:33.441233+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:39.513615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404138+0000","last_change":"2024-04-25T00:32:33.404138+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404138+0000","last_undegraded":"2024-04-25T00:32:33.404138+0000","last_fullsized":"2024-04-25T00:32:33.404138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422459+0000","last_change":"2024-04-25T00:32:33.419720+0000","last_active":"2024-04-25T00:32:33.422459+0000","last_peered":"2024-04-25T00:32:33.422459+0000","last_clean":"2024-04-25T00:32:33.422459+0000","last_became_active":"2024-04-25T00:32:33.419338+0000","last_became_peered":"2024-04-25T00:32:33.419338+0000","last_unstale":"2024-04-25T00:32:33.422459+0000","last_undegraded":"2024-04-25T00:32:33.422459+0000","last_fullsized":"2024-04-25T00:32:33.422459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:07.836095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402370+0000","last_change":"2024-04-25T00:32:33.402370+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402370+0000","last_undegraded":"2024-04-25T00:32:33.402370+0000","last_fullsized":"2024-04-25T00:32:33.402370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.441077+0000","last_change":"2024-04-25T00:32:33.433081+0000","last_active":"2024-04-25T00:32:33.441077+0000","last_peered":"2024-04-25T00:32:33.441077+0000","last_clean":"2024-04-25T00:32:33.441077+0000","last_became_active":"2024-04-25T00:32:33.432236+0000","last_became_peered":"2024-04-25T00:32:33.432236+0000","last_unstale":"2024-04-25T00:32:33.441077+0000","last_undegraded":"2024-04-25T00:32:33.441077+0000","last_fullsized":"2024-04-25T00:32:33.441077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:03.570347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403517+0000","last_change":"2024-04-25T00:32:33.403517+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403517+0000","last_undegraded":"2024-04-25T00:32:33.403517+0000","last_fullsized":"2024-04-25T00:32:33.403517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.432184+0000","last_change":"2024-04-25T00:32:33.431456+0000","last_active":"2024-04-25T00:32:33.432184+0000","last_peered":"2024-04-25T00:32:33.432184+0000","last_clean":"2024-04-25T00:32:33.432184+0000","last_became_active":"2024-04-25T00:32:33.431235+0000","last_became_peered":"2024-04-25T00:32:33.431235+0000","last_unstale":"2024-04-25T00:32:33.432184+0000","last_undegraded":"2024-04-25T00:32:33.432184+0000","last_fullsized":"2024-04-25T00:32:33.432184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:48.933581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.409317+0000","last_change":"2024-04-25T00:32:33.409317+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.409317+0000","last_undegraded":"2024-04-25T00:32:33.409317+0000","last_fullsized":"2024-04-25T00:32:33.409317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.417776+0000","last_change":"2024-04-25T00:32:33.416642+0000","last_active":"2024-04-25T00:32:33.417776+0000","last_peered":"2024-04-25T00:32:33.417776+0000","last_clean":"2024-04-25T00:32:33.417776+0000","last_became_active":"2024-04-25T00:32:33.416316+0000","last_became_peered":"2024-04-25T00:32:33.416316+0000","last_unstale":"2024-04-25T00:32:33.417776+0000","last_undegraded":"2024-04-25T00:32:33.417776+0000","last_fullsized":"2024-04-25T00:32:33.417776+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:05.605371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.405584+0000","last_change":"2024-04-25T00:32:33.405584+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.405584+0000","last_undegraded":"2024-04-25T00:32:33.405584+0000","last_fullsized":"2024-04-25T00:32:33.405584+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.429349+0000","last_change":"2024-04-25T00:32:33.424885+0000","last_active":"2024-04-25T00:32:33.429349+0000","last_peered":"2024-04-25T00:32:33.429349+0000","last_clean":"2024-04-25T00:32:33.429349+0000","last_became_active":"2024-04-25T00:32:33.424174+0000","last_became_peered":"2024-04-25T00:32:33.424174+0000","last_unstale":"2024-04-25T00:32:33.429349+0000","last_undegraded":"2024-04-25T00:32:33.429349+0000","last_fullsized":"2024-04-25T00:32:33.429349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:28.291849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402897+0000","last_change":"2024-04-25T00:32:33.402897+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402897+0000","last_undegraded":"2024-04-25T00:32:33.402897+0000","last_fullsized":"2024-04-25T00:32:33.402897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422426+0000","last_change":"2024-04-25T00:32:33.420265+0000","last_active":"2024-04-25T00:32:33.422426+0000","last_peered":"2024-04-25T00:32:33.422426+0000","last_clean":"2024-04-25T00:32:33.422426+0000","last_became_active":"2024-04-25T00:32:33.419977+0000","last_became_peered":"2024-04-25T00:32:33.419977+0000","last_unstale":"2024-04-25T00:32:33.422426+0000","last_undegraded":"2024-04-25T00:32:33.422426+0000","last_fullsized":"2024-04-25T00:32:33.422426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:34.696377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402398+0000","last_change":"2024-04-25T00:32:33.402398+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402398+0000","last_undegraded":"2024-04-25T00:32:33.402398+0000","last_fullsized":"2024-04-25T00:32:33.402398+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.442465+0000","last_change":"2024-04-25T00:32:33.434692+0000","last_active":"2024-04-25T00:32:33.442465+0000","last_peered":"2024-04-25T00:32:33.442465+0000","last_clean":"2024-04-25T00:32:33.442465+0000","last_became_active":"2024-04-25T00:32:33.433973+0000","last_became_peered":"2024-04-25T00:32:33.433973+0000","last_unstale":"2024-04-25T00:32:33.442465+0000","last_undegraded":"2024-04-25T00:32:33.442465+0000","last_fullsized":"2024-04-25T00:32:33.442465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:30.363273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403586+0000","last_change":"2024-04-25T00:32:33.403586+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403586+0000","last_undegraded":"2024-04-25T00:32:33.403586+0000","last_fullsized":"2024-04-25T00:32:33.403586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.432758+0000","last_change":"2024-04-25T00:32:33.432224+0000","last_active":"2024-04-25T00:32:33.432758+0000","last_peered":"2024-04-25T00:32:33.432758+0000","last_clean":"2024-04-25T00:32:33.432758+0000","last_became_active":"2024-04-25T00:32:33.431863+0000","last_became_peered":"2024-04-25T00:32:33.431863+0000","last_unstale":"2024-04-25T00:32:33.432758+0000","last_undegraded":"2024-04-25T00:32:33.432758+0000","last_fullsized":"2024-04-25T00:32:33.432758+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:33.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.425836+0000","last_change":"2024-04-25T00:32:33.425836+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.425836+0000","last_undegraded":"2024-04-25T00:32:33.425836+0000","last_fullsized":"2024-04-25T00:32:33.425836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.432174+0000","last_change":"2024-04-25T00:32:33.431495+0000","last_active":"2024-04-25T00:32:33.432174+0000","last_peered":"2024-04-25T00:32:33.432174+0000","last_clean":"2024-04-25T00:32:33.432174+0000","last_became_active":"2024-04-25T00:32:33.431231+0000","last_became_peered":"2024-04-25T00:32:33.431231+0000","last_unstale":"2024-04-25T00:32:33.432174+0000","last_undegraded":"2024-04-25T00:32:33.432174+0000","last_fullsized":"2024-04-25T00:32:33.432174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:28.291849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.416393+0000","last_change":"2024-04-25T00:32:33.416393+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.416393+0000","last_undegraded":"2024-04-25T00:32:33.416393+0000","last_fullsized":"2024-04-25T00:32:33.416393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422373+0000","last_change":"2024-04-25T00:32:33.419970+0000","last_active":"2024-04-25T00:32:33.422373+0000","last_peered":"2024-04-25T00:32:33.422373+0000","last_clean":"2024-04-25T00:32:33.422373+0000","last_became_active":"2024-04-25T00:32:33.419465+0000","last_became_peered":"2024-04-25T00:32:33.419465+0000","last_unstale":"2024-04-25T00:32:33.422373+0000","last_undegraded":"2024-04-25T00:32:33.422373+0000","last_fullsized":"2024-04-25T00:32:33.422373+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.651396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.405505+0000","last_change":"2024-04-25T00:32:33.405505+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.405505+0000","last_undegraded":"2024-04-25T00:32:33.405505+0000","last_fullsized":"2024-04-25T00:32:33.405505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.421058+0000","last_change":"2024-04-25T00:32:33.418814+0000","last_active":"2024-04-25T00:32:33.421058+0000","last_peered":"2024-04-25T00:32:33.421058+0000","last_clean":"2024-04-25T00:32:33.421058+0000","last_became_active":"2024-04-25T00:32:33.418270+0000","last_became_peered":"2024-04-25T00:32:33.418270+0000","last_unstale":"2024-04-25T00:32:33.421058+0000","last_undegraded":"2024-04-25T00:32:33.421058+0000","last_fullsized":"2024-04-25T00:32:33.421058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:18.360499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.409511+0000","last_change":"2024-04-25T00:32:33.409511+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.409511+0000","last_undegraded":"2024-04-25T00:32:33.409511+0000","last_fullsized":"2024-04-25T00:32:33.409511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.442615+0000","last_change":"2024-04-25T00:32:33.434714+0000","last_active":"2024-04-25T00:32:33.442615+0000","last_peered":"2024-04-25T00:32:33.442615+0000","last_clean":"2024-04-25T00:32:33.442615+0000","last_became_active":"2024-04-25T00:32:33.433950+0000","last_became_peered":"2024-04-25T00:32:33.433950+0000","last_unstale":"2024-04-25T00:32:33.442615+0000","last_undegraded":"2024-04-25T00:32:33.442615+0000","last_fullsized":"2024-04-25T00:32:33.442615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:40.127761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402374+0000","last_change":"2024-04-25T00:32:33.402374+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402374+0000","last_undegraded":"2024-04-25T00:32:33.402374+0000","last_fullsized":"2024-04-25T00:32:33.402374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422095+0000","last_change":"2024-04-25T00:32:33.412615+0000","last_active":"2024-04-25T00:32:33.422095+0000","last_peered":"2024-04-25T00:32:33.422095+0000","last_clean":"2024-04-25T00:32:33.422095+0000","last_became_active":"2024-04-25T00:32:33.411820+0000","last_became_peered":"2024-04-25T00:32:33.411820+0000","last_unstale":"2024-04-25T00:32:33.422095+0000","last_undegraded":"2024-04-25T00:32:33.422095+0000","last_fullsized":"2024-04-25T00:32:33.422095+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:00.041094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403654+0000","last_change":"2024-04-25T00:32:33.403654+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403654+0000","last_undegraded":"2024-04-25T00:32:33.403654+0000","last_fullsized":"2024-04-25T00:32:33.403654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.433365+0000","last_change":"2024-04-25T00:32:33.429101+0000","last_active":"2024-04-25T00:32:33.433365+0000","last_peered":"2024-04-25T00:32:33.433365+0000","last_clean":"2024-04-25T00:32:33.433365+0000","last_became_active":"2024-04-25T00:32:33.428344+0000","last_became_peered":"2024-04-25T00:32:33.428344+0000","last_unstale":"2024-04-25T00:32:33.433365+0000","last_undegraded":"2024-04-25T00:32:33.433365+0000","last_fullsized":"2024-04-25T00:32:33.433365+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:15.470226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403525+0000","last_change":"2024-04-25T00:32:33.403525+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403525+0000","last_undegraded":"2024-04-25T00:32:33.403525+0000","last_fullsized":"2024-04-25T00:32:33.403525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.420940+0000","last_change":"2024-04-25T00:32:33.419832+0000","last_active":"2024-04-25T00:32:33.420940+0000","last_peered":"2024-04-25T00:32:33.420940+0000","last_clean":"2024-04-25T00:32:33.420940+0000","last_became_active":"2024-04-25T00:32:33.419551+0000","last_became_peered":"2024-04-25T00:32:33.419551+0000","last_unstale":"2024-04-25T00:32:33.420940+0000","last_undegraded":"2024-04-25T00:32:33.420940+0000","last_fullsized":"2024-04-25T00:32:33.420940+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:18.360499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403524+0000","last_change":"2024-04-25T00:32:33.403524+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403524+0000","last_undegraded":"2024-04-25T00:32:33.403524+0000","last_fullsized":"2024-04-25T00:32:33.403524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.434530+0000","last_change":"2024-04-25T00:32:33.432483+0000","last_active":"2024-04-25T00:32:33.434530+0000","last_peered":"2024-04-25T00:32:33.434530+0000","last_clean":"2024-04-25T00:32:33.434530+0000","last_became_active":"2024-04-25T00:32:33.431774+0000","last_became_peered":"2024-04-25T00:32:33.431774+0000","last_unstale":"2024-04-25T00:32:33.434530+0000","last_undegraded":"2024-04-25T00:32:33.434530+0000","last_fullsized":"2024-04-25T00:32:33.434530+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:39.207154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404371+0000","last_change":"2024-04-25T00:32:33.404371+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404371+0000","last_undegraded":"2024-04-25T00:32:33.404371+0000","last_fullsized":"2024-04-25T00:32:33.404371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.429004+0000","last_change":"2024-04-25T00:32:33.426645+0000","last_active":"2024-04-25T00:32:33.429004+0000","last_peered":"2024-04-25T00:32:33.429004+0000","last_clean":"2024-04-25T00:32:33.429004+0000","last_became_active":"2024-04-25T00:32:33.425875+0000","last_became_peered":"2024-04-25T00:32:33.425875+0000","last_unstale":"2024-04-25T00:32:33.429004+0000","last_undegraded":"2024-04-25T00:32:33.429004+0000","last_fullsized":"2024-04-25T00:32:33.429004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:54.900021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.416854+0000","last_change":"2024-04-25T00:32:33.416854+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.416854+0000","last_undegraded":"2024-04-25T00:32:33.416854+0000","last_fullsized":"2024-04-25T00:32:33.416854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.443276+0000","last_change":"2024-04-25T00:32:33.441614+0000","last_active":"2024-04-25T00:32:33.443276+0000","last_peered":"2024-04-25T00:32:33.443276+0000","last_clean":"2024-04-25T00:32:33.443276+0000","last_became_active":"2024-04-25T00:32:33.441004+0000","last_became_peered":"2024-04-25T00:32:33.441004+0000","last_unstale":"2024-04-25T00:32:33.443276+0000","last_undegraded":"2024-04-25T00:32:33.443276+0000","last_fullsized":"2024-04-25T00:32:33.443276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:32.656831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404823+0000","last_change":"2024-04-25T00:32:33.404823+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404823+0000","last_undegraded":"2024-04-25T00:32:33.404823+0000","last_fullsized":"2024-04-25T00:32:33.404823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.418691+0000","last_change":"2024-04-25T00:32:33.412949+0000","last_active":"2024-04-25T00:32:33.418691+0000","last_peered":"2024-04-25T00:32:33.418691+0000","last_clean":"2024-04-25T00:32:33.418691+0000","last_became_active":"2024-04-25T00:32:33.412675+0000","last_became_peered":"2024-04-25T00:32:33.412675+0000","last_unstale":"2024-04-25T00:32:33.418691+0000","last_undegraded":"2024-04-25T00:32:33.418691+0000","last_fullsized":"2024-04-25T00:32:33.418691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:39.207154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.408491+0000","last_change":"2024-04-25T00:32:33.408491+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.408491+0000","last_undegraded":"2024-04-25T00:32:33.408491+0000","last_fullsized":"2024-04-25T00:32:33.408491+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.421639+0000","last_change":"2024-04-25T00:32:33.420075+0000","last_active":"2024-04-25T00:32:33.421639+0000","last_peered":"2024-04-25T00:32:33.421639+0000","last_clean":"2024-04-25T00:32:33.421639+0000","last_became_active":"2024-04-25T00:32:33.419783+0000","last_became_peered":"2024-04-25T00:32:33.419783+0000","last_unstale":"2024-04-25T00:32:33.421639+0000","last_undegraded":"2024-04-25T00:32:33.421639+0000","last_fullsized":"2024-04-25T00:32:33.421639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:33.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.407661+0000","last_change":"2024-04-25T00:32:33.407661+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.407661+0000","last_undegraded":"2024-04-25T00:32:33.407661+0000","last_fullsized":"2024-04-25T00:32:33.407661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.418654+0000","last_change":"2024-04-25T00:32:33.411847+0000","last_active":"2024-04-25T00:32:33.418654+0000","last_peered":"2024-04-25T00:32:33.418654+0000","last_clean":"2024-04-25T00:32:33.418654+0000","last_became_active":"2024-04-25T00:32:33.411429+0000","last_became_peered":"2024-04-25T00:32:33.411429+0000","last_unstale":"2024-04-25T00:32:33.418654+0000","last_undegraded":"2024-04-25T00:32:33.418654+0000","last_fullsized":"2024-04-25T00:32:33.418654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:25.652624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.420253+0000","last_change":"2024-04-25T00:32:33.420253+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.420253+0000","last_undegraded":"2024-04-25T00:32:33.420253+0000","last_fullsized":"2024-04-25T00:32:33.420253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.418712+0000","last_change":"2024-04-25T00:32:33.417550+0000","last_active":"2024-04-25T00:32:33.418712+0000","last_peered":"2024-04-25T00:32:33.418712+0000","last_clean":"2024-04-25T00:32:33.418712+0000","last_became_active":"2024-04-25T00:32:33.417040+0000","last_became_peered":"2024-04-25T00:32:33.417040+0000","last_unstale":"2024-04-25T00:32:33.418712+0000","last_undegraded":"2024-04-25T00:32:33.418712+0000","last_fullsized":"2024-04-25T00:32:33.418712+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:39.207154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.408161+0000","last_change":"2024-04-25T00:32:33.408161+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.408161+0000","last_undegraded":"2024-04-25T00:32:33.408161+0000","last_fullsized":"2024-04-25T00:32:33.408161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.418721+0000","last_change":"2024-04-25T00:32:33.412644+0000","last_active":"2024-04-25T00:32:33.418721+0000","last_peered":"2024-04-25T00:32:33.418721+0000","last_clean":"2024-04-25T00:32:33.418721+0000","last_became_active":"2024-04-25T00:32:33.412193+0000","last_became_peered":"2024-04-25T00:32:33.412193+0000","last_unstale":"2024-04-25T00:32:33.418721+0000","last_undegraded":"2024-04-25T00:32:33.418721+0000","last_fullsized":"2024-04-25T00:32:33.418721+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:05.605371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.406800+0000","last_change":"2024-04-25T00:32:33.406800+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.406800+0000","last_undegraded":"2024-04-25T00:32:33.406800+0000","last_fullsized":"2024-04-25T00:32:33.406800+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.421360+0000","last_change":"2024-04-25T00:32:33.416074+0000","last_active":"2024-04-25T00:32:33.421360+0000","last_peered":"2024-04-25T00:32:33.421360+0000","last_clean":"2024-04-25T00:32:33.421360+0000","last_became_active":"2024-04-25T00:32:33.415336+0000","last_became_peered":"2024-04-25T00:32:33.415336+0000","last_unstale":"2024-04-25T00:32:33.421360+0000","last_undegraded":"2024-04-25T00:32:33.421360+0000","last_fullsized":"2024-04-25T00:32:33.421360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:19.436849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410397+0000","last_change":"2024-04-25T00:32:33.410397+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410397+0000","last_undegraded":"2024-04-25T00:32:33.410397+0000","last_fullsized":"2024-04-25T00:32:33.410397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.441555+0000","last_change":"2024-04-25T00:32:33.440955+0000","last_active":"2024-04-25T00:32:33.441555+0000","last_peered":"2024-04-25T00:32:33.441555+0000","last_clean":"2024-04-25T00:32:33.441555+0000","last_became_active":"2024-04-25T00:32:33.440603+0000","last_became_peered":"2024-04-25T00:32:33.440603+0000","last_unstale":"2024-04-25T00:32:33.441555+0000","last_undegraded":"2024-04-25T00:32:33.441555+0000","last_fullsized":"2024-04-25T00:32:33.441555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:32.656831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.411828+0000","last_change":"2024-04-25T00:32:33.411828+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.411828+0000","last_undegraded":"2024-04-25T00:32:33.411828+0000","last_fullsized":"2024-04-25T00:32:33.411828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.436571+0000","last_change":"2024-04-25T00:32:33.432559+0000","last_active":"2024-04-25T00:32:33.436571+0000","last_peered":"2024-04-25T00:32:33.436571+0000","last_clean":"2024-04-25T00:32:33.436571+0000","last_became_active":"2024-04-25T00:32:33.431829+0000","last_became_peered":"2024-04-25T00:32:33.431829+0000","last_unstale":"2024-04-25T00:32:33.436571+0000","last_undegraded":"2024-04-25T00:32:33.436571+0000","last_fullsized":"2024-04-25T00:32:33.436571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:18.360499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.436634+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:32:33.436634+0000","last_peered":"2024-04-25T00:32:33.436634+0000","last_clean":"2024-04-25T00:32:33.436634+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:32:33.436634+0000","last_undegraded":"2024-04-25T00:32:33.436634+0000","last_fullsized":"2024-04-25T00:32:33.436634+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":724249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":348160,"data_stored":133881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":724249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:32:38.856 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:32:39.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f8446152640 1 Processor -- start 2024-04-25T00:32:39.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f8446152640 1 -- start start 2024-04-25T00:32:39.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f8446152640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f84400d2bb0 0x7f84400ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f8446152640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f844015e630 0x7f84400cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f8446152640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84400c8690 con 0x7f844015e250 2024-04-25T00:32:39.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f8446152640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f84400c8440 con 0x7f84400d2bb0 2024-04-25T00:32:39.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f8446152640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f84400c8970 con 0x7f844015e630 2024-04-25T00:32:39.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f843f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f844015e250 0x7f84400d24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38306/0 (socket says 172.21.15.29:38306) 2024-04-25T00:32:39.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f843f7fe640 1 -- 172.21.15.29:0/3024893934 learned_addr learned my addr 172.21.15.29:0/3024893934 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:39.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f8437fff640 1 --2- 172.21.15.29:0/3024893934 >> v2:172.21.15.29:3300/0 conn(0x7f84400d2bb0 0x7f84400ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f843ffff640 1 --2- 172.21.15.29:0/3024893934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f844015e630 0x7f84400cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f843effd640 1 -- 172.21.15.29:0/3024893934 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1192397692 0 0) 0x7f84400c8690 con 0x7f844015e250 2024-04-25T00:32:39.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f843effd640 1 -- 172.21.15.29:0/3024893934 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f842c003530 con 0x7f844015e250 2024-04-25T00:32:39.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f8437fff640 1 -- 172.21.15.29:0/3024893934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f844015e630 msgr2=0x7f84400cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:39.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f8437fff640 1 --2- 172.21.15.29:0/3024893934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f844015e630 0x7f84400cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f8437fff640 1 -- 172.21.15.29:0/3024893934 >> v1:172.21.15.29:6789/0 conn(0x7f844015e250 legacy=0x7f84400d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f8437fff640 1 -- 172.21.15.29:0/3024893934 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84400c8c50 con 0x7f84400d2bb0 2024-04-25T00:32:39.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.011+0000 7f843ffff640 1 --2- 172.21.15.29:0/3024893934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f844015e630 0x7f84400cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:39.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f8437fff640 1 --2- 172.21.15.29:0/3024893934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400d2bb0 0x7f84400ceaf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f84300097c0 tx=0x7f843001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7467292a70172be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:39.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f843effd640 1 -- 172.21.15.29:0/3024893934 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84300b0070 con 0x7f84400d2bb0 2024-04-25T00:32:39.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f843effd640 1 -- 172.21.15.29:0/3024893934 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84300ab600 con 0x7f84400d2bb0 2024-04-25T00:32:39.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f843effd640 1 -- 172.21.15.29:0/3024893934 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84300ab9d0 con 0x7f84400d2bb0 2024-04-25T00:32:39.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f8446152640 1 -- 172.21.15.29:0/3024893934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400d2bb0 msgr2=0x7f84400ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f8446152640 1 --2- 172.21.15.29:0/3024893934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400d2bb0 0x7f84400ceaf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f84300097c0 tx=0x7f843001c410 comp rx=0 tx=0).stop 2024-04-25T00:32:39.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f8446152640 1 -- 172.21.15.29:0/3024893934 shutdown_connections 2024-04-25T00:32:39.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f8446152640 1 --2- 172.21.15.29:0/3024893934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f844015e630 0x7f84400cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f8446152640 1 --2- 172.21.15.29:0/3024893934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400d2bb0 0x7f84400ceaf0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f8446152640 1 -- 172.21.15.29:0/3024893934 >> 172.21.15.29:0/3024893934 conn(0x7f84400d3f70 msgr2=0x7f84400d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:39.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f8446152640 1 -- 172.21.15.29:0/3024893934 shutdown_connections 2024-04-25T00:32:39.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f8446152640 1 -- 172.21.15.29:0/3024893934 wait complete. 2024-04-25T00:32:39.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f8446152640 1 Processor -- start 2024-04-25T00:32:39.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.015+0000 7f8446152640 1 -- start start 2024-04-25T00:32:39.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f8446152640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400d2bb0 0x7f8440140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f8446152640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f844015e250 0x7f8440154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f8446152640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f844015e630 0x7f8440160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f8446152640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f844016aff0 con 0x7f84400d2bb0 2024-04-25T00:32:39.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f8446152640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f84400d1ed0 con 0x7f844015e630 2024-04-25T00:32:39.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f8446152640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f844016b170 con 0x7f844015e250 2024-04-25T00:32:39.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400d2bb0 0x7f8440140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400d2bb0 0x7f8440140d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54240/0 (socket says 172.21.15.29:54240) 2024-04-25T00:32:39.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843f7fe640 1 -- 172.21.15.29:0/3400433603 learned_addr learned my addr 172.21.15.29:0/3400433603 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:39.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f8437fff640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f844015e250 0x7f8440154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843ffff640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f844015e630 0x7f8440160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843f7fe640 1 -- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f844015e250 msgr2=0x7f8440154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843f7fe640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f844015e250 0x7f8440154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843f7fe640 1 -- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f844015e630 msgr2=0x7f8440160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843f7fe640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f844015e630 0x7f8440160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843f7fe640 1 -- 172.21.15.29:0/3400433603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8440154c50 con 0x7f84400d2bb0 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f8437fff640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f844015e250 0x7f8440154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843ffff640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f844015e630 0x7f8440160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843f7fe640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400d2bb0 0x7f8440140d20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f842800c450 tx=0x7f842800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843cff9640 1 -- 172.21.15.29:0/3400433603 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428017070 con 0x7f84400d2bb0 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843cff9640 1 -- 172.21.15.29:0/3400433603 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8428005cf0 con 0x7f84400d2bb0 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f8446152640 1 -- 172.21.15.29:0/3400433603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84401614b0 con 0x7f84400d2bb0 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843cff9640 1 -- 172.21.15.29:0/3400433603 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428007290 con 0x7f84400d2bb0 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f8446152640 1 -- 172.21.15.29:0/3400433603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8440161a70 con 0x7f84400d2bb0 2024-04-25T00:32:39.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f8446152640 1 -- 172.21.15.29:0/3400433603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8440069270 con 0x7f84400d2bb0 2024-04-25T00:32:39.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843cff9640 1 -- 172.21.15.29:0/3400433603 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f842801a070 con 0x7f84400d2bb0 2024-04-25T00:32:39.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843cff9640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84200678f0 0x7f8420069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f843cff9640 1 -- 172.21.15.29:0/3400433603 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8428085080 con 0x7f84400d2bb0 2024-04-25T00:32:39.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.019+0000 7f8437fff640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84200678f0 0x7f8420069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.023+0000 7f8437fff640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84200678f0 0x7f8420069db0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f84300ad000 tx=0x7f8430008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:39.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.023+0000 7f843cff9640 1 -- 172.21.15.29:0/3400433603 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8440154c50 con 0x7f84400d2bb0 2024-04-25T00:32:39.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.175+0000 7f8446152640 1 -- 172.21.15.29:0/3400433603 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84400c7bb0 con 0x7f84200678f0 2024-04-25T00:32:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.195+0000 7f843cff9640 1 -- 172.21.15.29:0/3400433603 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7f84400c7bb0 con 0x7f84200678f0 2024-04-25T00:32:39.200 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.199+0000 7f8446152640 1 -- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84200678f0 msgr2=0x7f8420069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.199+0000 7f8446152640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84200678f0 0x7f8420069db0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f84300ad000 tx=0x7f8430008040 comp rx=0 tx=0).stop 2024-04-25T00:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.199+0000 7f8446152640 1 -- 172.21.15.29:0/3400433603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400d2bb0 msgr2=0x7f8440140d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.199+0000 7f8446152640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400d2bb0 0x7f8440140d20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f842800c450 tx=0x7f842800c910 comp rx=0 tx=0).stop 2024-04-25T00:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.199+0000 7f8446152640 1 -- 172.21.15.29:0/3400433603 shutdown_connections 2024-04-25T00:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.199+0000 7f8446152640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f84200678f0 0x7f8420069db0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.199+0000 7f8446152640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f844015e630 0x7f8440160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.199+0000 7f8446152640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f844015e250 0x7f8440154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.199+0000 7f8446152640 1 --2- 172.21.15.29:0/3400433603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84400d2bb0 0x7f8440140d20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.199+0000 7f8446152640 1 -- 172.21.15.29:0/3400433603 >> 172.21.15.29:0/3400433603 conn(0x7f84400d3f70 msgr2=0x7f844015eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.199+0000 7f8446152640 1 -- 172.21.15.29:0/3400433603 shutdown_connections 2024-04-25T00:32:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.199+0000 7f8446152640 1 -- 172.21.15.29:0/3400433603 wait complete. 2024-04-25T00:32:39.218 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-04-25T00:32:37.466967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253272,"kb_used_data":3820,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721448,"statfs":{"total":773094113280,"available":772834762752,"internally_reserved":0,"allocated":3911680,"data_stored":2220449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.075330"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.421587+0000","last_change":"2024-04-25T00:32:33.421587+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.421587+0000","last_undegraded":"2024-04-25T00:32:33.421587+0000","last_fullsized":"2024-04-25T00:32:33.421587+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.416958+0000","last_change":"2024-04-25T00:32:33.416958+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.416958+0000","last_undegraded":"2024-04-25T00:32:33.416958+0000","last_fullsized":"2024-04-25T00:32:33.416958+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.401783+0000","last_change":"2024-04-25T00:32:33.401783+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.401783+0000","last_undegraded":"2024-04-25T00:32:33.401783+0000","last_fullsized":"2024-04-25T00:32:33.401783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410014+0000","last_change":"2024-04-25T00:32:33.410014+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410014+0000","last_undegraded":"2024-04-25T00:32:33.410014+0000","last_fullsized":"2024-04-25T00:32:33.410014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.406582+0000","last_change":"2024-04-25T00:32:33.406582+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.406582+0000","last_undegraded":"2024-04-25T00:32:33.406582+0000","last_fullsized":"2024-04-25T00:32:33.406582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.416330+0000","last_change":"2024-04-25T00:32:33.416330+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.416330+0000","last_undegraded":"2024-04-25T00:32:33.416330+0000","last_fullsized":"2024-04-25T00:32:33.416330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.415789+0000","last_change":"2024-04-25T00:32:33.415789+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.415789+0000","last_undegraded":"2024-04-25T00:32:33.415789+0000","last_fullsized":"2024-04-25T00:32:33.415789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.428371+0000","last_change":"2024-04-25T00:32:33.428371+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.428371+0000","last_undegraded":"2024-04-25T00:32:33.428371+0000","last_fullsized":"2024-04-25T00:32:33.428371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402082+0000","last_change":"2024-04-25T00:32:33.402082+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402082+0000","last_undegraded":"2024-04-25T00:32:33.402082+0000","last_fullsized":"2024-04-25T00:32:33.402082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410001+0000","last_change":"2024-04-25T00:32:33.410001+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410001+0000","last_undegraded":"2024-04-25T00:32:33.410001+0000","last_fullsized":"2024-04-25T00:32:33.410001+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.411695+0000","last_change":"2024-04-25T00:32:33.411695+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.411695+0000","last_undegraded":"2024-04-25T00:32:33.411695+0000","last_fullsized":"2024-04-25T00:32:33.411695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402278+0000","last_change":"2024-04-25T00:32:33.402278+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402278+0000","last_undegraded":"2024-04-25T00:32:33.402278+0000","last_fullsized":"2024-04-25T00:32:33.402278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.406087+0000","last_change":"2024-04-25T00:32:33.406087+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.406087+0000","last_undegraded":"2024-04-25T00:32:33.406087+0000","last_fullsized":"2024-04-25T00:32:33.406087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.412316+0000","last_change":"2024-04-25T00:32:33.412316+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.412316+0000","last_undegraded":"2024-04-25T00:32:33.412316+0000","last_fullsized":"2024-04-25T00:32:33.412316+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402227+0000","last_change":"2024-04-25T00:32:33.402227+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402227+0000","last_undegraded":"2024-04-25T00:32:33.402227+0000","last_fullsized":"2024-04-25T00:32:33.402227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.412924+0000","last_change":"2024-04-25T00:32:33.412924+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.412924+0000","last_undegraded":"2024-04-25T00:32:33.412924+0000","last_fullsized":"2024-04-25T00:32:33.412924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.413406+0000","last_change":"2024-04-25T00:32:33.413406+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.413406+0000","last_undegraded":"2024-04-25T00:32:33.413406+0000","last_fullsized":"2024-04-25T00:32:33.413406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.408069+0000","last_change":"2024-04-25T00:32:33.408069+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.408069+0000","last_undegraded":"2024-04-25T00:32:33.408069+0000","last_fullsized":"2024-04-25T00:32:33.408069+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404172+0000","last_change":"2024-04-25T00:32:33.404172+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404172+0000","last_undegraded":"2024-04-25T00:32:33.404172+0000","last_fullsized":"2024-04-25T00:32:33.404172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.405470+0000","last_change":"2024-04-25T00:32:33.405470+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.405470+0000","last_undegraded":"2024-04-25T00:32:33.405470+0000","last_fullsized":"2024-04-25T00:32:33.405470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.408165+0000","last_change":"2024-04-25T00:32:33.408165+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.408165+0000","last_undegraded":"2024-04-25T00:32:33.408165+0000","last_fullsized":"2024-04-25T00:32:33.408165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.400602+0000","last_change":"2024-04-25T00:32:33.400602+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.400602+0000","last_undegraded":"2024-04-25T00:32:33.400602+0000","last_fullsized":"2024-04-25T00:32:33.400602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.405760+0000","last_change":"2024-04-25T00:32:33.405760+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.405760+0000","last_undegraded":"2024-04-25T00:32:33.405760+0000","last_fullsized":"2024-04-25T00:32:33.405760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402840+0000","last_change":"2024-04-25T00:32:33.402840+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402840+0000","last_undegraded":"2024-04-25T00:32:33.402840+0000","last_fullsized":"2024-04-25T00:32:33.402840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410219+0000","last_change":"2024-04-25T00:32:33.410219+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410219+0000","last_undegraded":"2024-04-25T00:32:33.410219+0000","last_fullsized":"2024-04-25T00:32:33.410219+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.407507+0000","last_change":"2024-04-25T00:32:33.407507+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.407507+0000","last_undegraded":"2024-04-25T00:32:33.407507+0000","last_fullsized":"2024-04-25T00:32:33.407507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403049+0000","last_change":"2024-04-25T00:32:33.403049+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403049+0000","last_undegraded":"2024-04-25T00:32:33.403049+0000","last_fullsized":"2024-04-25T00:32:33.403049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.400778+0000","last_change":"2024-04-25T00:32:33.400778+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.400778+0000","last_undegraded":"2024-04-25T00:32:33.400778+0000","last_fullsized":"2024-04-25T00:32:33.400778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.406659+0000","last_change":"2024-04-25T00:32:33.406659+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.406659+0000","last_undegraded":"2024-04-25T00:32:33.406659+0000","last_fullsized":"2024-04-25T00:32:33.406659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.411758+0000","last_change":"2024-04-25T00:32:33.411758+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.411758+0000","last_undegraded":"2024-04-25T00:32:33.411758+0000","last_fullsized":"2024-04-25T00:32:33.411758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.417701+0000","last_change":"2024-04-25T00:32:33.417701+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.417701+0000","last_undegraded":"2024-04-25T00:32:33.417701+0000","last_fullsized":"2024-04-25T00:32:33.417701+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.426820+0000","last_change":"2024-04-25T00:32:33.426820+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.426820+0000","last_undegraded":"2024-04-25T00:32:33.426820+0000","last_fullsized":"2024-04-25T00:32:33.426820+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410800+0000","last_change":"2024-04-25T00:32:33.410800+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410800+0000","last_undegraded":"2024-04-25T00:32:33.410800+0000","last_fullsized":"2024-04-25T00:32:33.410800+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.440516+0000","last_change":"2024-04-25T00:32:33.439097+0000","last_active":"2024-04-25T00:32:33.440516+0000","last_peered":"2024-04-25T00:32:33.440516+0000","last_clean":"2024-04-25T00:32:33.440516+0000","last_became_active":"2024-04-25T00:32:33.438390+0000","last_became_peered":"2024-04-25T00:32:33.438390+0000","last_unstale":"2024-04-25T00:32:33.440516+0000","last_undegraded":"2024-04-25T00:32:33.440516+0000","last_fullsized":"2024-04-25T00:32:33.440516+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:33.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.419679+0000","last_change":"2024-04-25T00:32:33.419679+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.419679+0000","last_undegraded":"2024-04-25T00:32:33.419679+0000","last_fullsized":"2024-04-25T00:32:33.419679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.440528+0000","last_change":"2024-04-25T00:32:33.439749+0000","last_active":"2024-04-25T00:32:33.440528+0000","last_peered":"2024-04-25T00:32:33.440528+0000","last_clean":"2024-04-25T00:32:33.440528+0000","last_became_active":"2024-04-25T00:32:33.439548+0000","last_became_peered":"2024-04-25T00:32:33.439548+0000","last_unstale":"2024-04-25T00:32:33.440528+0000","last_undegraded":"2024-04-25T00:32:33.440528+0000","last_fullsized":"2024-04-25T00:32:33.440528+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:10.605816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404042+0000","last_change":"2024-04-25T00:32:33.404042+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404042+0000","last_undegraded":"2024-04-25T00:32:33.404042+0000","last_fullsized":"2024-04-25T00:32:33.404042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422465+0000","last_change":"2024-04-25T00:32:33.419772+0000","last_active":"2024-04-25T00:32:33.422465+0000","last_peered":"2024-04-25T00:32:33.422465+0000","last_clean":"2024-04-25T00:32:33.422465+0000","last_became_active":"2024-04-25T00:32:33.419459+0000","last_became_peered":"2024-04-25T00:32:33.419459+0000","last_unstale":"2024-04-25T00:32:33.422465+0000","last_undegraded":"2024-04-25T00:32:33.422465+0000","last_fullsized":"2024-04-25T00:32:33.422465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:24.119844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402369+0000","last_change":"2024-04-25T00:32:33.402369+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402369+0000","last_undegraded":"2024-04-25T00:32:33.402369+0000","last_fullsized":"2024-04-25T00:32:33.402369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.420930+0000","last_change":"2024-04-25T00:32:33.419126+0000","last_active":"2024-04-25T00:32:33.420930+0000","last_peered":"2024-04-25T00:32:33.420930+0000","last_clean":"2024-04-25T00:32:33.420930+0000","last_became_active":"2024-04-25T00:32:33.418834+0000","last_became_peered":"2024-04-25T00:32:33.418834+0000","last_unstale":"2024-04-25T00:32:33.420930+0000","last_undegraded":"2024-04-25T00:32:33.420930+0000","last_fullsized":"2024-04-25T00:32:33.420930+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:39.513615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410063+0000","last_change":"2024-04-25T00:32:33.410063+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410063+0000","last_undegraded":"2024-04-25T00:32:33.410063+0000","last_fullsized":"2024-04-25T00:32:33.410063+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.433292+0000","last_change":"2024-04-25T00:32:33.429088+0000","last_active":"2024-04-25T00:32:33.433292+0000","last_peered":"2024-04-25T00:32:33.433292+0000","last_clean":"2024-04-25T00:32:33.433292+0000","last_became_active":"2024-04-25T00:32:33.427379+0000","last_became_peered":"2024-04-25T00:32:33.427379+0000","last_unstale":"2024-04-25T00:32:33.433292+0000","last_undegraded":"2024-04-25T00:32:33.433292+0000","last_fullsized":"2024-04-25T00:32:33.433292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:05.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403511+0000","last_change":"2024-04-25T00:32:33.403511+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403511+0000","last_undegraded":"2024-04-25T00:32:33.403511+0000","last_fullsized":"2024-04-25T00:32:33.403511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.441233+0000","last_change":"2024-04-25T00:32:33.429965+0000","last_active":"2024-04-25T00:32:33.441233+0000","last_peered":"2024-04-25T00:32:33.441233+0000","last_clean":"2024-04-25T00:32:33.441233+0000","last_became_active":"2024-04-25T00:32:33.427356+0000","last_became_peered":"2024-04-25T00:32:33.427356+0000","last_unstale":"2024-04-25T00:32:33.441233+0000","last_undegraded":"2024-04-25T00:32:33.441233+0000","last_fullsized":"2024-04-25T00:32:33.441233+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:39.513615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404138+0000","last_change":"2024-04-25T00:32:33.404138+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404138+0000","last_undegraded":"2024-04-25T00:32:33.404138+0000","last_fullsized":"2024-04-25T00:32:33.404138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422459+0000","last_change":"2024-04-25T00:32:33.419720+0000","last_active":"2024-04-25T00:32:33.422459+0000","last_peered":"2024-04-25T00:32:33.422459+0000","last_clean":"2024-04-25T00:32:33.422459+0000","last_became_active":"2024-04-25T00:32:33.419338+0000","last_became_peered":"2024-04-25T00:32:33.419338+0000","last_unstale":"2024-04-25T00:32:33.422459+0000","last_undegraded":"2024-04-25T00:32:33.422459+0000","last_fullsized":"2024-04-25T00:32:33.422459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:07.836095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402370+0000","last_change":"2024-04-25T00:32:33.402370+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402370+0000","last_undegraded":"2024-04-25T00:32:33.402370+0000","last_fullsized":"2024-04-25T00:32:33.402370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.441077+0000","last_change":"2024-04-25T00:32:33.433081+0000","last_active":"2024-04-25T00:32:33.441077+0000","last_peered":"2024-04-25T00:32:33.441077+0000","last_clean":"2024-04-25T00:32:33.441077+0000","last_became_active":"2024-04-25T00:32:33.432236+0000","last_became_peered":"2024-04-25T00:32:33.432236+0000","last_unstale":"2024-04-25T00:32:33.441077+0000","last_undegraded":"2024-04-25T00:32:33.441077+0000","last_fullsized":"2024-04-25T00:32:33.441077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:03.570347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403517+0000","last_change":"2024-04-25T00:32:33.403517+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403517+0000","last_undegraded":"2024-04-25T00:32:33.403517+0000","last_fullsized":"2024-04-25T00:32:33.403517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.432184+0000","last_change":"2024-04-25T00:32:33.431456+0000","last_active":"2024-04-25T00:32:33.432184+0000","last_peered":"2024-04-25T00:32:33.432184+0000","last_clean":"2024-04-25T00:32:33.432184+0000","last_became_active":"2024-04-25T00:32:33.431235+0000","last_became_peered":"2024-04-25T00:32:33.431235+0000","last_unstale":"2024-04-25T00:32:33.432184+0000","last_undegraded":"2024-04-25T00:32:33.432184+0000","last_fullsized":"2024-04-25T00:32:33.432184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:48.933581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.409317+0000","last_change":"2024-04-25T00:32:33.409317+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.409317+0000","last_undegraded":"2024-04-25T00:32:33.409317+0000","last_fullsized":"2024-04-25T00:32:33.409317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.417776+0000","last_change":"2024-04-25T00:32:33.416642+0000","last_active":"2024-04-25T00:32:33.417776+0000","last_peered":"2024-04-25T00:32:33.417776+0000","last_clean":"2024-04-25T00:32:33.417776+0000","last_became_active":"2024-04-25T00:32:33.416316+0000","last_became_peered":"2024-04-25T00:32:33.416316+0000","last_unstale":"2024-04-25T00:32:33.417776+0000","last_undegraded":"2024-04-25T00:32:33.417776+0000","last_fullsized":"2024-04-25T00:32:33.417776+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:05.605371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.405584+0000","last_change":"2024-04-25T00:32:33.405584+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.405584+0000","last_undegraded":"2024-04-25T00:32:33.405584+0000","last_fullsized":"2024-04-25T00:32:33.405584+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.429349+0000","last_change":"2024-04-25T00:32:33.424885+0000","last_active":"2024-04-25T00:32:33.429349+0000","last_peered":"2024-04-25T00:32:33.429349+0000","last_clean":"2024-04-25T00:32:33.429349+0000","last_became_active":"2024-04-25T00:32:33.424174+0000","last_became_peered":"2024-04-25T00:32:33.424174+0000","last_unstale":"2024-04-25T00:32:33.429349+0000","last_undegraded":"2024-04-25T00:32:33.429349+0000","last_fullsized":"2024-04-25T00:32:33.429349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:28.291849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402897+0000","last_change":"2024-04-25T00:32:33.402897+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402897+0000","last_undegraded":"2024-04-25T00:32:33.402897+0000","last_fullsized":"2024-04-25T00:32:33.402897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422426+0000","last_change":"2024-04-25T00:32:33.420265+0000","last_active":"2024-04-25T00:32:33.422426+0000","last_peered":"2024-04-25T00:32:33.422426+0000","last_clean":"2024-04-25T00:32:33.422426+0000","last_became_active":"2024-04-25T00:32:33.419977+0000","last_became_peered":"2024-04-25T00:32:33.419977+0000","last_unstale":"2024-04-25T00:32:33.422426+0000","last_undegraded":"2024-04-25T00:32:33.422426+0000","last_fullsized":"2024-04-25T00:32:33.422426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:34.696377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402398+0000","last_change":"2024-04-25T00:32:33.402398+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402398+0000","last_undegraded":"2024-04-25T00:32:33.402398+0000","last_fullsized":"2024-04-25T00:32:33.402398+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.442465+0000","last_change":"2024-04-25T00:32:33.434692+0000","last_active":"2024-04-25T00:32:33.442465+0000","last_peered":"2024-04-25T00:32:33.442465+0000","last_clean":"2024-04-25T00:32:33.442465+0000","last_became_active":"2024-04-25T00:32:33.433973+0000","last_became_peered":"2024-04-25T00:32:33.433973+0000","last_unstale":"2024-04-25T00:32:33.442465+0000","last_undegraded":"2024-04-25T00:32:33.442465+0000","last_fullsized":"2024-04-25T00:32:33.442465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:30.363273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403586+0000","last_change":"2024-04-25T00:32:33.403586+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403586+0000","last_undegraded":"2024-04-25T00:32:33.403586+0000","last_fullsized":"2024-04-25T00:32:33.403586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.432758+0000","last_change":"2024-04-25T00:32:33.432224+0000","last_active":"2024-04-25T00:32:33.432758+0000","last_peered":"2024-04-25T00:32:33.432758+0000","last_clean":"2024-04-25T00:32:33.432758+0000","last_became_active":"2024-04-25T00:32:33.431863+0000","last_became_peered":"2024-04-25T00:32:33.431863+0000","last_unstale":"2024-04-25T00:32:33.432758+0000","last_undegraded":"2024-04-25T00:32:33.432758+0000","last_fullsized":"2024-04-25T00:32:33.432758+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:33.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.425836+0000","last_change":"2024-04-25T00:32:33.425836+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.425836+0000","last_undegraded":"2024-04-25T00:32:33.425836+0000","last_fullsized":"2024-04-25T00:32:33.425836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.432174+0000","last_change":"2024-04-25T00:32:33.431495+0000","last_active":"2024-04-25T00:32:33.432174+0000","last_peered":"2024-04-25T00:32:33.432174+0000","last_clean":"2024-04-25T00:32:33.432174+0000","last_became_active":"2024-04-25T00:32:33.431231+0000","last_became_peered":"2024-04-25T00:32:33.431231+0000","last_unstale":"2024-04-25T00:32:33.432174+0000","last_undegraded":"2024-04-25T00:32:33.432174+0000","last_fullsized":"2024-04-25T00:32:33.432174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:28.291849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.416393+0000","last_change":"2024-04-25T00:32:33.416393+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.416393+0000","last_undegraded":"2024-04-25T00:32:33.416393+0000","last_fullsized":"2024-04-25T00:32:33.416393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422373+0000","last_change":"2024-04-25T00:32:33.419970+0000","last_active":"2024-04-25T00:32:33.422373+0000","last_peered":"2024-04-25T00:32:33.422373+0000","last_clean":"2024-04-25T00:32:33.422373+0000","last_became_active":"2024-04-25T00:32:33.419465+0000","last_became_peered":"2024-04-25T00:32:33.419465+0000","last_unstale":"2024-04-25T00:32:33.422373+0000","last_undegraded":"2024-04-25T00:32:33.422373+0000","last_fullsized":"2024-04-25T00:32:33.422373+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.651396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.405505+0000","last_change":"2024-04-25T00:32:33.405505+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.405505+0000","last_undegraded":"2024-04-25T00:32:33.405505+0000","last_fullsized":"2024-04-25T00:32:33.405505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.421058+0000","last_change":"2024-04-25T00:32:33.418814+0000","last_active":"2024-04-25T00:32:33.421058+0000","last_peered":"2024-04-25T00:32:33.421058+0000","last_clean":"2024-04-25T00:32:33.421058+0000","last_became_active":"2024-04-25T00:32:33.418270+0000","last_became_peered":"2024-04-25T00:32:33.418270+0000","last_unstale":"2024-04-25T00:32:33.421058+0000","last_undegraded":"2024-04-25T00:32:33.421058+0000","last_fullsized":"2024-04-25T00:32:33.421058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:18.360499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.409511+0000","last_change":"2024-04-25T00:32:33.409511+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.409511+0000","last_undegraded":"2024-04-25T00:32:33.409511+0000","last_fullsized":"2024-04-25T00:32:33.409511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.442615+0000","last_change":"2024-04-25T00:32:33.434714+0000","last_active":"2024-04-25T00:32:33.442615+0000","last_peered":"2024-04-25T00:32:33.442615+0000","last_clean":"2024-04-25T00:32:33.442615+0000","last_became_active":"2024-04-25T00:32:33.433950+0000","last_became_peered":"2024-04-25T00:32:33.433950+0000","last_unstale":"2024-04-25T00:32:33.442615+0000","last_undegraded":"2024-04-25T00:32:33.442615+0000","last_fullsized":"2024-04-25T00:32:33.442615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:40.127761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.402374+0000","last_change":"2024-04-25T00:32:33.402374+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.402374+0000","last_undegraded":"2024-04-25T00:32:33.402374+0000","last_fullsized":"2024-04-25T00:32:33.402374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.422095+0000","last_change":"2024-04-25T00:32:33.412615+0000","last_active":"2024-04-25T00:32:33.422095+0000","last_peered":"2024-04-25T00:32:33.422095+0000","last_clean":"2024-04-25T00:32:33.422095+0000","last_became_active":"2024-04-25T00:32:33.411820+0000","last_became_peered":"2024-04-25T00:32:33.411820+0000","last_unstale":"2024-04-25T00:32:33.422095+0000","last_undegraded":"2024-04-25T00:32:33.422095+0000","last_fullsized":"2024-04-25T00:32:33.422095+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:00.041094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403654+0000","last_change":"2024-04-25T00:32:33.403654+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403654+0000","last_undegraded":"2024-04-25T00:32:33.403654+0000","last_fullsized":"2024-04-25T00:32:33.403654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.433365+0000","last_change":"2024-04-25T00:32:33.429101+0000","last_active":"2024-04-25T00:32:33.433365+0000","last_peered":"2024-04-25T00:32:33.433365+0000","last_clean":"2024-04-25T00:32:33.433365+0000","last_became_active":"2024-04-25T00:32:33.428344+0000","last_became_peered":"2024-04-25T00:32:33.428344+0000","last_unstale":"2024-04-25T00:32:33.433365+0000","last_undegraded":"2024-04-25T00:32:33.433365+0000","last_fullsized":"2024-04-25T00:32:33.433365+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:15.470226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403525+0000","last_change":"2024-04-25T00:32:33.403525+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403525+0000","last_undegraded":"2024-04-25T00:32:33.403525+0000","last_fullsized":"2024-04-25T00:32:33.403525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.420940+0000","last_change":"2024-04-25T00:32:33.419832+0000","last_active":"2024-04-25T00:32:33.420940+0000","last_peered":"2024-04-25T00:32:33.420940+0000","last_clean":"2024-04-25T00:32:33.420940+0000","last_became_active":"2024-04-25T00:32:33.419551+0000","last_became_peered":"2024-04-25T00:32:33.419551+0000","last_unstale":"2024-04-25T00:32:33.420940+0000","last_undegraded":"2024-04-25T00:32:33.420940+0000","last_fullsized":"2024-04-25T00:32:33.420940+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:18.360499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.403524+0000","last_change":"2024-04-25T00:32:33.403524+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.403524+0000","last_undegraded":"2024-04-25T00:32:33.403524+0000","last_fullsized":"2024-04-25T00:32:33.403524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.434530+0000","last_change":"2024-04-25T00:32:33.432483+0000","last_active":"2024-04-25T00:32:33.434530+0000","last_peered":"2024-04-25T00:32:33.434530+0000","last_clean":"2024-04-25T00:32:33.434530+0000","last_became_active":"2024-04-25T00:32:33.431774+0000","last_became_peered":"2024-04-25T00:32:33.431774+0000","last_unstale":"2024-04-25T00:32:33.434530+0000","last_undegraded":"2024-04-25T00:32:33.434530+0000","last_fullsized":"2024-04-25T00:32:33.434530+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:39.207154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404371+0000","last_change":"2024-04-25T00:32:33.404371+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404371+0000","last_undegraded":"2024-04-25T00:32:33.404371+0000","last_fullsized":"2024-04-25T00:32:33.404371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.429004+0000","last_change":"2024-04-25T00:32:33.426645+0000","last_active":"2024-04-25T00:32:33.429004+0000","last_peered":"2024-04-25T00:32:33.429004+0000","last_clean":"2024-04-25T00:32:33.429004+0000","last_became_active":"2024-04-25T00:32:33.425875+0000","last_became_peered":"2024-04-25T00:32:33.425875+0000","last_unstale":"2024-04-25T00:32:33.429004+0000","last_undegraded":"2024-04-25T00:32:33.429004+0000","last_fullsized":"2024-04-25T00:32:33.429004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:54.900021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.416854+0000","last_change":"2024-04-25T00:32:33.416854+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.416854+0000","last_undegraded":"2024-04-25T00:32:33.416854+0000","last_fullsized":"2024-04-25T00:32:33.416854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.443276+0000","last_change":"2024-04-25T00:32:33.441614+0000","last_active":"2024-04-25T00:32:33.443276+0000","last_peered":"2024-04-25T00:32:33.443276+0000","last_clean":"2024-04-25T00:32:33.443276+0000","last_became_active":"2024-04-25T00:32:33.441004+0000","last_became_peered":"2024-04-25T00:32:33.441004+0000","last_unstale":"2024-04-25T00:32:33.443276+0000","last_undegraded":"2024-04-25T00:32:33.443276+0000","last_fullsized":"2024-04-25T00:32:33.443276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:32.656831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.404823+0000","last_change":"2024-04-25T00:32:33.404823+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.404823+0000","last_undegraded":"2024-04-25T00:32:33.404823+0000","last_fullsized":"2024-04-25T00:32:33.404823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.418691+0000","last_change":"2024-04-25T00:32:33.412949+0000","last_active":"2024-04-25T00:32:33.418691+0000","last_peered":"2024-04-25T00:32:33.418691+0000","last_clean":"2024-04-25T00:32:33.418691+0000","last_became_active":"2024-04-25T00:32:33.412675+0000","last_became_peered":"2024-04-25T00:32:33.412675+0000","last_unstale":"2024-04-25T00:32:33.418691+0000","last_undegraded":"2024-04-25T00:32:33.418691+0000","last_fullsized":"2024-04-25T00:32:33.418691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:39.207154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.408491+0000","last_change":"2024-04-25T00:32:33.408491+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.408491+0000","last_undegraded":"2024-04-25T00:32:33.408491+0000","last_fullsized":"2024-04-25T00:32:33.408491+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.421639+0000","last_change":"2024-04-25T00:32:33.420075+0000","last_active":"2024-04-25T00:32:33.421639+0000","last_peered":"2024-04-25T00:32:33.421639+0000","last_clean":"2024-04-25T00:32:33.421639+0000","last_became_active":"2024-04-25T00:32:33.419783+0000","last_became_peered":"2024-04-25T00:32:33.419783+0000","last_unstale":"2024-04-25T00:32:33.421639+0000","last_undegraded":"2024-04-25T00:32:33.421639+0000","last_fullsized":"2024-04-25T00:32:33.421639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:33.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.407661+0000","last_change":"2024-04-25T00:32:33.407661+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.407661+0000","last_undegraded":"2024-04-25T00:32:33.407661+0000","last_fullsized":"2024-04-25T00:32:33.407661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.418654+0000","last_change":"2024-04-25T00:32:33.411847+0000","last_active":"2024-04-25T00:32:33.418654+0000","last_peered":"2024-04-25T00:32:33.418654+0000","last_clean":"2024-04-25T00:32:33.418654+0000","last_became_active":"2024-04-25T00:32:33.411429+0000","last_became_peered":"2024-04-25T00:32:33.411429+0000","last_unstale":"2024-04-25T00:32:33.418654+0000","last_undegraded":"2024-04-25T00:32:33.418654+0000","last_fullsized":"2024-04-25T00:32:33.418654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:25.652624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.420253+0000","last_change":"2024-04-25T00:32:33.420253+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.420253+0000","last_undegraded":"2024-04-25T00:32:33.420253+0000","last_fullsized":"2024-04-25T00:32:33.420253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.418712+0000","last_change":"2024-04-25T00:32:33.417550+0000","last_active":"2024-04-25T00:32:33.418712+0000","last_peered":"2024-04-25T00:32:33.418712+0000","last_clean":"2024-04-25T00:32:33.418712+0000","last_became_active":"2024-04-25T00:32:33.417040+0000","last_became_peered":"2024-04-25T00:32:33.417040+0000","last_unstale":"2024-04-25T00:32:33.418712+0000","last_undegraded":"2024-04-25T00:32:33.418712+0000","last_fullsized":"2024-04-25T00:32:33.418712+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:39.207154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.408161+0000","last_change":"2024-04-25T00:32:33.408161+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.408161+0000","last_undegraded":"2024-04-25T00:32:33.408161+0000","last_fullsized":"2024-04-25T00:32:33.408161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.418721+0000","last_change":"2024-04-25T00:32:33.412644+0000","last_active":"2024-04-25T00:32:33.418721+0000","last_peered":"2024-04-25T00:32:33.418721+0000","last_clean":"2024-04-25T00:32:33.418721+0000","last_became_active":"2024-04-25T00:32:33.412193+0000","last_became_peered":"2024-04-25T00:32:33.412193+0000","last_unstale":"2024-04-25T00:32:33.418721+0000","last_undegraded":"2024-04-25T00:32:33.418721+0000","last_fullsized":"2024-04-25T00:32:33.418721+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:05.605371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.406800+0000","last_change":"2024-04-25T00:32:33.406800+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.406800+0000","last_undegraded":"2024-04-25T00:32:33.406800+0000","last_fullsized":"2024-04-25T00:32:33.406800+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.421360+0000","last_change":"2024-04-25T00:32:33.416074+0000","last_active":"2024-04-25T00:32:33.421360+0000","last_peered":"2024-04-25T00:32:33.421360+0000","last_clean":"2024-04-25T00:32:33.421360+0000","last_became_active":"2024-04-25T00:32:33.415336+0000","last_became_peered":"2024-04-25T00:32:33.415336+0000","last_unstale":"2024-04-25T00:32:33.421360+0000","last_undegraded":"2024-04-25T00:32:33.421360+0000","last_fullsized":"2024-04-25T00:32:33.421360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:19.436849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.410397+0000","last_change":"2024-04-25T00:32:33.410397+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.410397+0000","last_undegraded":"2024-04-25T00:32:33.410397+0000","last_fullsized":"2024-04-25T00:32:33.410397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.441555+0000","last_change":"2024-04-25T00:32:33.440955+0000","last_active":"2024-04-25T00:32:33.441555+0000","last_peered":"2024-04-25T00:32:33.441555+0000","last_clean":"2024-04-25T00:32:33.441555+0000","last_became_active":"2024-04-25T00:32:33.440603+0000","last_became_peered":"2024-04-25T00:32:33.440603+0000","last_unstale":"2024-04-25T00:32:33.441555+0000","last_undegraded":"2024-04-25T00:32:33.441555+0000","last_fullsized":"2024-04-25T00:32:33.441555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:32.656831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T00:32:33.411828+0000","last_change":"2024-04-25T00:32:33.411828+0000","last_active":"2024-04-25T00:32:33.389607+0000","last_peered":"2024-04-25T00:32:33.389607+0000","last_clean":"2024-04-25T00:32:33.389607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:32:33.411828+0000","last_undegraded":"2024-04-25T00:32:33.411828+0000","last_fullsized":"2024-04-25T00:32:33.411828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.436571+0000","last_change":"2024-04-25T00:32:33.432559+0000","last_active":"2024-04-25T00:32:33.436571+0000","last_peered":"2024-04-25T00:32:33.436571+0000","last_clean":"2024-04-25T00:32:33.436571+0000","last_became_active":"2024-04-25T00:32:33.431829+0000","last_became_peered":"2024-04-25T00:32:33.431829+0000","last_unstale":"2024-04-25T00:32:33.436571+0000","last_undegraded":"2024-04-25T00:32:33.436571+0000","last_fullsized":"2024-04-25T00:32:33.436571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:18.360499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T00:32:33.436634+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:32:33.436634+0000","last_peered":"2024-04-25T00:32:33.436634+0000","last_clean":"2024-04-25T00:32:33.436634+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:32:33.436634+0000","last_undegraded":"2024-04-25T00:32:33.436634+0000","last_fullsized":"2024-04-25T00:32:33.436634+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":724249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":348160,"data_stored":133881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":724249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:32:39.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-04-25T00:32:39.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.421587+0000', 'last_change': '2024-04-25T00:32:33.421587+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.421587+0000', 'last_undegraded': '2024-04-25T00:32:33.421587+0000', 'last_fullsized': '2024-04-25T00:32:33.421587+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:32:39.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-25T00:32:39.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.416958+0000', 'last_change': '2024-04-25T00:32:33.416958+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.416958+0000', 'last_undegraded': '2024-04-25T00:32:33.416958+0000', 'last_fullsized': '2024-04-25T00:32:33.416958+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:32:39.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-25T00:32:39.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.401783+0000', 'last_change': '2024-04-25T00:32:33.401783+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.401783+0000', 'last_undegraded': '2024-04-25T00:32:33.401783+0000', 'last_fullsized': '2024-04-25T00:32:33.401783+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:32:39.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-25T00:32:39.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.410014+0000', 'last_change': '2024-04-25T00:32:33.410014+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.410014+0000', 'last_undegraded': '2024-04-25T00:32:33.410014+0000', 'last_fullsized': '2024-04-25T00:32:33.410014+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:32:39.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-25T00:32:39.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.406582+0000', 'last_change': '2024-04-25T00:32:33.406582+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.406582+0000', 'last_undegraded': '2024-04-25T00:32:33.406582+0000', 'last_fullsized': '2024-04-25T00:32:33.406582+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:32:39.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-04-25T00:32:39.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.416330+0000', 'last_change': '2024-04-25T00:32:33.416330+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.416330+0000', 'last_undegraded': '2024-04-25T00:32:33.416330+0000', 'last_fullsized': '2024-04-25T00:32:33.416330+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:32:39.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-04-25T00:32:39.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.415789+0000', 'last_change': '2024-04-25T00:32:33.415789+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.415789+0000', 'last_undegraded': '2024-04-25T00:32:33.415789+0000', 'last_fullsized': '2024-04-25T00:32:33.415789+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:32:39.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-04-25T00:32:39.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.428371+0000', 'last_change': '2024-04-25T00:32:33.428371+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.428371+0000', 'last_undegraded': '2024-04-25T00:32:33.428371+0000', 'last_fullsized': '2024-04-25T00:32:33.428371+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:32:39.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-04-25T00:32:39.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.402082+0000', 'last_change': '2024-04-25T00:32:33.402082+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.402082+0000', 'last_undegraded': '2024-04-25T00:32:33.402082+0000', 'last_fullsized': '2024-04-25T00:32:33.402082+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:32:39.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-25T00:32:39.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.410001+0000', 'last_change': '2024-04-25T00:32:33.410001+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.410001+0000', 'last_undegraded': '2024-04-25T00:32:33.410001+0000', 'last_fullsized': '2024-04-25T00:32:33.410001+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:32:39.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-25T00:32:39.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.411695+0000', 'last_change': '2024-04-25T00:32:33.411695+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.411695+0000', 'last_undegraded': '2024-04-25T00:32:33.411695+0000', 'last_fullsized': '2024-04-25T00:32:33.411695+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:32:39.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-04-25T00:32:39.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.402278+0000', 'last_change': '2024-04-25T00:32:33.402278+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.402278+0000', 'last_undegraded': '2024-04-25T00:32:33.402278+0000', 'last_fullsized': '2024-04-25T00:32:33.402278+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:32:39.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-25T00:32:39.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.406087+0000', 'last_change': '2024-04-25T00:32:33.406087+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.406087+0000', 'last_undegraded': '2024-04-25T00:32:33.406087+0000', 'last_fullsized': '2024-04-25T00:32:33.406087+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:32:39.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-04-25T00:32:39.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.412316+0000', 'last_change': '2024-04-25T00:32:33.412316+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.412316+0000', 'last_undegraded': '2024-04-25T00:32:33.412316+0000', 'last_fullsized': '2024-04-25T00:32:33.412316+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:32:39.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-04-25T00:32:39.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.402227+0000', 'last_change': '2024-04-25T00:32:33.402227+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.402227+0000', 'last_undegraded': '2024-04-25T00:32:33.402227+0000', 'last_fullsized': '2024-04-25T00:32:33.402227+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:32:39.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-04-25T00:32:39.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.412924+0000', 'last_change': '2024-04-25T00:32:33.412924+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.412924+0000', 'last_undegraded': '2024-04-25T00:32:33.412924+0000', 'last_fullsized': '2024-04-25T00:32:33.412924+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:32:39.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-04-25T00:32:39.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.413406+0000', 'last_change': '2024-04-25T00:32:33.413406+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.413406+0000', 'last_undegraded': '2024-04-25T00:32:33.413406+0000', 'last_fullsized': '2024-04-25T00:32:33.413406+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:32:39.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-04-25T00:32:39.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.408069+0000', 'last_change': '2024-04-25T00:32:33.408069+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.408069+0000', 'last_undegraded': '2024-04-25T00:32:33.408069+0000', 'last_fullsized': '2024-04-25T00:32:33.408069+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:32:39.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-04-25T00:32:39.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.404172+0000', 'last_change': '2024-04-25T00:32:33.404172+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.404172+0000', 'last_undegraded': '2024-04-25T00:32:33.404172+0000', 'last_fullsized': '2024-04-25T00:32:33.404172+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:32:39.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-04-25T00:32:39.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.405470+0000', 'last_change': '2024-04-25T00:32:33.405470+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.405470+0000', 'last_undegraded': '2024-04-25T00:32:33.405470+0000', 'last_fullsized': '2024-04-25T00:32:33.405470+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:32:39.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-04-25T00:32:39.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.408165+0000', 'last_change': '2024-04-25T00:32:33.408165+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.408165+0000', 'last_undegraded': '2024-04-25T00:32:33.408165+0000', 'last_fullsized': '2024-04-25T00:32:33.408165+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:32:39.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-04-25T00:32:39.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.400602+0000', 'last_change': '2024-04-25T00:32:33.400602+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.400602+0000', 'last_undegraded': '2024-04-25T00:32:33.400602+0000', 'last_fullsized': '2024-04-25T00:32:33.400602+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:32:39.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-04-25T00:32:39.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.405760+0000', 'last_change': '2024-04-25T00:32:33.405760+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.405760+0000', 'last_undegraded': '2024-04-25T00:32:33.405760+0000', 'last_fullsized': '2024-04-25T00:32:33.405760+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:32:39.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-04-25T00:32:39.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.402840+0000', 'last_change': '2024-04-25T00:32:33.402840+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.402840+0000', 'last_undegraded': '2024-04-25T00:32:33.402840+0000', 'last_fullsized': '2024-04-25T00:32:33.402840+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:32:39.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-25T00:32:39.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.410219+0000', 'last_change': '2024-04-25T00:32:33.410219+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.410219+0000', 'last_undegraded': '2024-04-25T00:32:33.410219+0000', 'last_fullsized': '2024-04-25T00:32:33.410219+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:32:39.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-04-25T00:32:39.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.407507+0000', 'last_change': '2024-04-25T00:32:33.407507+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.407507+0000', 'last_undegraded': '2024-04-25T00:32:33.407507+0000', 'last_fullsized': '2024-04-25T00:32:33.407507+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:32:39.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-25T00:32:39.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.403049+0000', 'last_change': '2024-04-25T00:32:33.403049+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.403049+0000', 'last_undegraded': '2024-04-25T00:32:33.403049+0000', 'last_fullsized': '2024-04-25T00:32:33.403049+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:32:39.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-04-25T00:32:39.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.400778+0000', 'last_change': '2024-04-25T00:32:33.400778+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.400778+0000', 'last_undegraded': '2024-04-25T00:32:33.400778+0000', 'last_fullsized': '2024-04-25T00:32:33.400778+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:32:39.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-04-25T00:32:39.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.406659+0000', 'last_change': '2024-04-25T00:32:33.406659+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.406659+0000', 'last_undegraded': '2024-04-25T00:32:33.406659+0000', 'last_fullsized': '2024-04-25T00:32:33.406659+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:32:39.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-04-25T00:32:39.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.411758+0000', 'last_change': '2024-04-25T00:32:33.411758+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.411758+0000', 'last_undegraded': '2024-04-25T00:32:33.411758+0000', 'last_fullsized': '2024-04-25T00:32:33.411758+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:32:39.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-04-25T00:32:39.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.417701+0000', 'last_change': '2024-04-25T00:32:33.417701+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.417701+0000', 'last_undegraded': '2024-04-25T00:32:33.417701+0000', 'last_fullsized': '2024-04-25T00:32:33.417701+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:32:39.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-04-25T00:32:39.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.426820+0000', 'last_change': '2024-04-25T00:32:33.426820+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.426820+0000', 'last_undegraded': '2024-04-25T00:32:33.426820+0000', 'last_fullsized': '2024-04-25T00:32:33.426820+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:32:39.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-25T00:32:39.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.410800+0000', 'last_change': '2024-04-25T00:32:33.410800+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.410800+0000', 'last_undegraded': '2024-04-25T00:32:33.410800+0000', 'last_fullsized': '2024-04-25T00:32:33.410800+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:32:39.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-25T00:32:39.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.419679+0000', 'last_change': '2024-04-25T00:32:33.419679+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.419679+0000', 'last_undegraded': '2024-04-25T00:32:33.419679+0000', 'last_fullsized': '2024-04-25T00:32:33.419679+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:32:39.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-25T00:32:39.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.404042+0000', 'last_change': '2024-04-25T00:32:33.404042+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.404042+0000', 'last_undegraded': '2024-04-25T00:32:33.404042+0000', 'last_fullsized': '2024-04-25T00:32:33.404042+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:32:39.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-25T00:32:39.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.402369+0000', 'last_change': '2024-04-25T00:32:33.402369+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.402369+0000', 'last_undegraded': '2024-04-25T00:32:33.402369+0000', 'last_fullsized': '2024-04-25T00:32:33.402369+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:32:39.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-25T00:32:39.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.410063+0000', 'last_change': '2024-04-25T00:32:33.410063+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.410063+0000', 'last_undegraded': '2024-04-25T00:32:33.410063+0000', 'last_fullsized': '2024-04-25T00:32:33.410063+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:32:39.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-25T00:32:39.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.403511+0000', 'last_change': '2024-04-25T00:32:33.403511+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.403511+0000', 'last_undegraded': '2024-04-25T00:32:33.403511+0000', 'last_fullsized': '2024-04-25T00:32:33.403511+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:32:39.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-25T00:32:39.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.404138+0000', 'last_change': '2024-04-25T00:32:33.404138+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.404138+0000', 'last_undegraded': '2024-04-25T00:32:33.404138+0000', 'last_fullsized': '2024-04-25T00:32:33.404138+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:32:39.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-25T00:32:39.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.402370+0000', 'last_change': '2024-04-25T00:32:33.402370+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.402370+0000', 'last_undegraded': '2024-04-25T00:32:33.402370+0000', 'last_fullsized': '2024-04-25T00:32:33.402370+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:32:39.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-25T00:32:39.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.403517+0000', 'last_change': '2024-04-25T00:32:33.403517+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.403517+0000', 'last_undegraded': '2024-04-25T00:32:33.403517+0000', 'last_fullsized': '2024-04-25T00:32:33.403517+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:32:39.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-25T00:32:39.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.409317+0000', 'last_change': '2024-04-25T00:32:33.409317+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.409317+0000', 'last_undegraded': '2024-04-25T00:32:33.409317+0000', 'last_fullsized': '2024-04-25T00:32:33.409317+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:32:39.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-25T00:32:39.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.405584+0000', 'last_change': '2024-04-25T00:32:33.405584+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.405584+0000', 'last_undegraded': '2024-04-25T00:32:33.405584+0000', 'last_fullsized': '2024-04-25T00:32:33.405584+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:32:39.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-25T00:32:39.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.402897+0000', 'last_change': '2024-04-25T00:32:33.402897+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.402897+0000', 'last_undegraded': '2024-04-25T00:32:33.402897+0000', 'last_fullsized': '2024-04-25T00:32:33.402897+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:32:39.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-25T00:32:39.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.402398+0000', 'last_change': '2024-04-25T00:32:33.402398+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.402398+0000', 'last_undegraded': '2024-04-25T00:32:33.402398+0000', 'last_fullsized': '2024-04-25T00:32:33.402398+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:32:39.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-25T00:32:39.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.403586+0000', 'last_change': '2024-04-25T00:32:33.403586+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.403586+0000', 'last_undegraded': '2024-04-25T00:32:33.403586+0000', 'last_fullsized': '2024-04-25T00:32:33.403586+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:32:39.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-25T00:32:39.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.425836+0000', 'last_change': '2024-04-25T00:32:33.425836+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.425836+0000', 'last_undegraded': '2024-04-25T00:32:33.425836+0000', 'last_fullsized': '2024-04-25T00:32:33.425836+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:32:39.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-25T00:32:39.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.416393+0000', 'last_change': '2024-04-25T00:32:33.416393+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.416393+0000', 'last_undegraded': '2024-04-25T00:32:33.416393+0000', 'last_fullsized': '2024-04-25T00:32:33.416393+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:32:39.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-25T00:32:39.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.405505+0000', 'last_change': '2024-04-25T00:32:33.405505+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.405505+0000', 'last_undegraded': '2024-04-25T00:32:33.405505+0000', 'last_fullsized': '2024-04-25T00:32:33.405505+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:32:39.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-25T00:32:39.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.409511+0000', 'last_change': '2024-04-25T00:32:33.409511+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.409511+0000', 'last_undegraded': '2024-04-25T00:32:33.409511+0000', 'last_fullsized': '2024-04-25T00:32:33.409511+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:32:39.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-25T00:32:39.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.402374+0000', 'last_change': '2024-04-25T00:32:33.402374+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.402374+0000', 'last_undegraded': '2024-04-25T00:32:33.402374+0000', 'last_fullsized': '2024-04-25T00:32:33.402374+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:32:39.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-25T00:32:39.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.403654+0000', 'last_change': '2024-04-25T00:32:33.403654+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.403654+0000', 'last_undegraded': '2024-04-25T00:32:33.403654+0000', 'last_fullsized': '2024-04-25T00:32:33.403654+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:32:39.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-25T00:32:39.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.403525+0000', 'last_change': '2024-04-25T00:32:33.403525+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.403525+0000', 'last_undegraded': '2024-04-25T00:32:33.403525+0000', 'last_fullsized': '2024-04-25T00:32:33.403525+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:32:39.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-25T00:32:39.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.403524+0000', 'last_change': '2024-04-25T00:32:33.403524+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.403524+0000', 'last_undegraded': '2024-04-25T00:32:33.403524+0000', 'last_fullsized': '2024-04-25T00:32:33.403524+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:32:39.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-25T00:32:39.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.404371+0000', 'last_change': '2024-04-25T00:32:33.404371+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.404371+0000', 'last_undegraded': '2024-04-25T00:32:33.404371+0000', 'last_fullsized': '2024-04-25T00:32:33.404371+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:32:39.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-25T00:32:39.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.416854+0000', 'last_change': '2024-04-25T00:32:33.416854+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.416854+0000', 'last_undegraded': '2024-04-25T00:32:33.416854+0000', 'last_fullsized': '2024-04-25T00:32:33.416854+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:32:39.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-25T00:32:39.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.404823+0000', 'last_change': '2024-04-25T00:32:33.404823+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.404823+0000', 'last_undegraded': '2024-04-25T00:32:33.404823+0000', 'last_fullsized': '2024-04-25T00:32:33.404823+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:32:39.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-25T00:32:39.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.408491+0000', 'last_change': '2024-04-25T00:32:33.408491+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.408491+0000', 'last_undegraded': '2024-04-25T00:32:33.408491+0000', 'last_fullsized': '2024-04-25T00:32:33.408491+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:32:39.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-25T00:32:39.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.407661+0000', 'last_change': '2024-04-25T00:32:33.407661+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.407661+0000', 'last_undegraded': '2024-04-25T00:32:33.407661+0000', 'last_fullsized': '2024-04-25T00:32:33.407661+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:32:39.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-25T00:32:39.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.420253+0000', 'last_change': '2024-04-25T00:32:33.420253+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.420253+0000', 'last_undegraded': '2024-04-25T00:32:33.420253+0000', 'last_fullsized': '2024-04-25T00:32:33.420253+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:32:39.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-25T00:32:39.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.408161+0000', 'last_change': '2024-04-25T00:32:33.408161+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.408161+0000', 'last_undegraded': '2024-04-25T00:32:33.408161+0000', 'last_fullsized': '2024-04-25T00:32:33.408161+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:32:39.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-25T00:32:39.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.406800+0000', 'last_change': '2024-04-25T00:32:33.406800+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.406800+0000', 'last_undegraded': '2024-04-25T00:32:33.406800+0000', 'last_fullsized': '2024-04-25T00:32:33.406800+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:32:39.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-25T00:32:39.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.410397+0000', 'last_change': '2024-04-25T00:32:33.410397+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.410397+0000', 'last_undegraded': '2024-04-25T00:32:33.410397+0000', 'last_fullsized': '2024-04-25T00:32:33.410397+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:32:39.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-25T00:32:39.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:32:33.411828+0000', 'last_change': '2024-04-25T00:32:33.411828+0000', 'last_active': '2024-04-25T00:32:33.389607+0000', 'last_peered': '2024-04-25T00:32:33.389607+0000', 'last_clean': '2024-04-25T00:32:33.389607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:32:33.411828+0000', 'last_undegraded': '2024-04-25T00:32:33.411828+0000', 'last_fullsized': '2024-04-25T00:32:33.411828+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'last_clean_scrub_stamp': '2024-04-25T00:32:33.389607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:32:39.234 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:32:39.379 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-04-25T00:32:39.375+0000 7f5eb8c4f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:32:39.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.375+0000 7ff99eb0c640 1 Processor -- start 2024-04-25T00:32:39.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.375+0000 7ff99eb0c640 1 -- start start 2024-04-25T00:32:39.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.375+0000 7ff99eb0c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff99806f270 0x7ff99806f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.375+0000 7ff99eb0c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff99806fb90 con 0x7ff99806b210 2024-04-25T00:32:39.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.375+0000 7ff99eb0c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9980cfe90 con 0x7ff99815e330 2024-04-25T00:32:39.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.375+0000 7ff99eb0c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff9980d0170 con 0x7ff99806f270 2024-04-25T00:32:39.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff99d0a9640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff99806b210 0x7ff99806b640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38318/0 (socket says 172.21.15.29:38318) 2024-04-25T00:32:39.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff99d0a9640 1 -- 172.21.15.29:0/2930619100 learned_addr learned my addr 172.21.15.29:0/2930619100 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:39.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff99c8a8640 1 --2- 172.21.15.29:0/2930619100 >> v2:172.21.15.157:3300/0 conn(0x7ff99806f270 0x7ff99806f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3924719457 0 0) 0x7ff99806fb90 con 0x7ff99806b210 2024-04-25T00:32:39.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff984003530 con 0x7ff99806b210 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2732294341 0 0) 0x7ff984003530 con 0x7ff99806b210 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 >> v2:172.21.15.157:3300/0 conn(0x7ff99806f270 msgr2=0x7ff99806f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 --2- 172.21.15.29:0/2930619100 >> v2:172.21.15.157:3300/0 conn(0x7ff99806f270 0x7ff99806f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 >> v1:172.21.15.157:6789/0 conn(0x7ff99815e330 legacy=0x7ff99806db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9980d0450 con 0x7ff99806b210 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ff988001680 con 0x7ff99806b210 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff99c8a8640 1 --2- 172.21.15.29:0/2930619100 >> v2:172.21.15.157:3300/0 conn(0x7ff99806f270 0x7ff99806f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 >> v1:172.21.15.29:6789/0 conn(0x7ff99806b210 legacy=0x7ff99806b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 --2- 172.21.15.29:0/2930619100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9840039c0 0x7ff984003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 --2- 172.21.15.29:0/2930619100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff984004810 0x7ff984004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 --2- 172.21.15.29:0/2930619100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9840051d0 0x7ff98400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff9980d0170 con 0x7ff984004810 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff98400c170 con 0x7ff9840051d0 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff98400c370 con 0x7ff9840039c0 2024-04-25T00:32:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff997fff640 1 --2- 172.21.15.29:0/2930619100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff984004810 0x7ff984004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff99c8a8640 1 --2- 172.21.15.29:0/2930619100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9840039c0 0x7ff984003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff99d0a9640 1 --2- 172.21.15.29:0/2930619100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9840051d0 0x7ff98400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff997fff640 1 -- 172.21.15.29:0/2930619100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9840039c0 msgr2=0x7ff984003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:39.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff997fff640 1 --2- 172.21.15.29:0/2930619100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9840039c0 0x7ff984003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff997fff640 1 -- 172.21.15.29:0/2930619100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9840051d0 msgr2=0x7ff98400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:39.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff997fff640 1 --2- 172.21.15.29:0/2930619100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9840051d0 0x7ff98400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff997fff640 1 -- 172.21.15.29:0/2930619100 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff984003530 con 0x7ff984004810 2024-04-25T00:32:39.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff99d0a9640 1 -- 172.21.15.29:0/2930619100 reap_dead start 2024-04-25T00:32:39.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff997fff640 1 --2- 172.21.15.29:0/2930619100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff984004810 0x7ff984004c90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff980001780 tx=0x7ff98001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d39f3987a9c967 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:39.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9800013a0 con 0x7ff984004810 2024-04-25T00:32:39.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9800052f0 con 0x7ff984004810 2024-04-25T00:32:39.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.379+0000 7ff9977fe640 1 -- 172.21.15.29:0/2930619100 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980003840 con 0x7ff984004810 2024-04-25T00:32:39.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 -- 172.21.15.29:0/2930619100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff984004810 msgr2=0x7ff984004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 --2- 172.21.15.29:0/2930619100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff984004810 0x7ff984004c90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff980001780 tx=0x7ff98001c400 comp rx=0 tx=0).stop 2024-04-25T00:32:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 -- 172.21.15.29:0/2930619100 shutdown_connections 2024-04-25T00:32:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 --2- 172.21.15.29:0/2930619100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff984004810 0x7ff984004c90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 -- 172.21.15.29:0/2930619100 >> 172.21.15.29:0/2930619100 conn(0x7ff9980b2920 msgr2=0x7ff9980b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 -- 172.21.15.29:0/2930619100 shutdown_connections 2024-04-25T00:32:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 -- 172.21.15.29:0/2930619100 wait complete. 2024-04-25T00:32:39.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 Processor -- start 2024-04-25T00:32:39.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 -- start start 2024-04-25T00:32:39.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff984004810 0x7ff9981507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9980b5920 0x7ff998153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9980cb720 0x7ff9980cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff99815ed70 con 0x7ff9980b5920 2024-04-25T00:32:39.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff99815ebf0 con 0x7ff984004810 2024-04-25T00:32:39.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff99815eef0 con 0x7ff9980cb720 2024-04-25T00:32:39.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff997fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9980b5920 0x7ff998153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff997fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9980b5920 0x7ff998153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54254/0 (socket says 172.21.15.29:54254) 2024-04-25T00:32:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff997fff640 1 -- 172.21.15.29:0/4288913646 learned_addr learned my addr 172.21.15.29:0/4288913646 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99c8a8640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff984004810 0x7ff9981507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99d0a9640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9980cb720 0x7ff9980cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff997fff640 1 -- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9980cb720 msgr2=0x7ff9980cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff997fff640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9980cb720 0x7ff9980cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff997fff640 1 -- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff984004810 msgr2=0x7ff9981507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff997fff640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff984004810 0x7ff9981507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff997fff640 1 -- 172.21.15.29:0/4288913646 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9980b5e20 con 0x7ff9980b5920 2024-04-25T00:32:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99c8a8640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff984004810 0x7ff9981507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff997fff640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9980b5920 0x7ff998153f10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff980005040 tx=0x7ff984005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff9957fa640 1 -- 172.21.15.29:0/4288913646 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980007a40 con 0x7ff9980b5920 2024-04-25T00:32:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff9957fa640 1 -- 172.21.15.29:0/4288913646 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9800ba070 con 0x7ff9980b5920 2024-04-25T00:32:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 -- 172.21.15.29:0/4288913646 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9980b6100 con 0x7ff9980b5920 2024-04-25T00:32:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff9957fa640 1 -- 172.21.15.29:0/4288913646 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980003e60 con 0x7ff9980b5920 2024-04-25T00:32:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.383+0000 7ff99eb0c640 1 -- 172.21.15.29:0/4288913646 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9980b66c0 con 0x7ff9980b5920 2024-04-25T00:32:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.387+0000 7ff99eb0c640 1 -- 172.21.15.29:0/4288913646 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9980b68c0 con 0x7ff9980b5920 2024-04-25T00:32:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.387+0000 7ff9957fa640 1 -- 172.21.15.29:0/4288913646 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff9800052f0 con 0x7ff9980b5920 2024-04-25T00:32:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.387+0000 7ff9957fa640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff9780678f0 0x7ff978069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.387+0000 7ff9957fa640 1 -- 172.21.15.29:0/4288913646 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff98011ffd0 con 0x7ff9980b5920 2024-04-25T00:32:39.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.387+0000 7ff99c8a8640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff9780678f0 0x7ff978069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.387+0000 7ff99c8a8640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff9780678f0 0x7ff978069db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff990007620 tx=0x7ff99000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:39.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.391+0000 7ff9957fa640 1 -- 172.21.15.29:0/4288913646 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff9980b68c0 con 0x7ff9980b5920 2024-04-25T00:32:39.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.611+0000 7ff99eb0c640 1 -- 172.21.15.29:0/4288913646 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff998069270 con 0x7ff9980b5920 2024-04-25T00:32:39.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.615+0000 7ff9957fa640 1 -- 172.21.15.29:0/4288913646 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1173 (secure 0 0 0) 0x7ff980016110 con 0x7ff9980b5920 2024-04-25T00:32:39.617 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:39.617 INFO:teuthology.orchestra.run.smithi029.stdout:{"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":86,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1714005083,"num_in_osds":8,"osd_in_since":1714005074,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":259694592,"bytes_avail":772834418688,"bytes_total":773094113280,"write_bytes_sec":2361,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5214}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-25T00:32:25.463697+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:32:39.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.615+0000 7ff99eb0c640 1 -- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff9780678f0 msgr2=0x7ff978069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.615+0000 7ff99eb0c640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff9780678f0 0x7ff978069db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff990007620 tx=0x7ff99000b040 comp rx=0 tx=0).stop 2024-04-25T00:32:39.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.615+0000 7ff99eb0c640 1 -- 172.21.15.29:0/4288913646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9980b5920 msgr2=0x7ff998153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.615+0000 7ff99eb0c640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9980b5920 0x7ff998153f10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff980005040 tx=0x7ff984005840 comp rx=0 tx=0).stop 2024-04-25T00:32:39.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.615+0000 7ff99eb0c640 1 -- 172.21.15.29:0/4288913646 shutdown_connections 2024-04-25T00:32:39.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.615+0000 7ff99eb0c640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9980cb720 0x7ff9980cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.615+0000 7ff99eb0c640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9980b5920 0x7ff998153f10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.615+0000 7ff99eb0c640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff9780678f0 0x7ff978069db0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.615+0000 7ff99eb0c640 1 --2- 172.21.15.29:0/4288913646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff984004810 0x7ff9981507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.615+0000 7ff99eb0c640 1 -- 172.21.15.29:0/4288913646 >> 172.21.15.29:0/4288913646 conn(0x7ff9980b2920 msgr2=0x7ff998161230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:39.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.615+0000 7ff99eb0c640 1 -- 172.21.15.29:0/4288913646 shutdown_connections 2024-04-25T00:32:39.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.615+0000 7ff99eb0c640 1 -- 172.21.15.29:0/4288913646 wait complete. 2024-04-25T00:32:39.635 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T00:32:39.635 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:32:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.739+0000 7fdc5ec26640 1 Processor -- start 2024-04-25T00:32:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.739+0000 7fdc5ec26640 1 -- start start 2024-04-25T00:32:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5ec26640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdc5815c4a0 0x7fdc5815c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5ec26640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc580cecd0 con 0x7fdc5815cdc0 2024-04-25T00:32:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5ec26640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc580cefb0 con 0x7fdc5806cc60 2024-04-25T00:32:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5ec26640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdc580b25c0 con 0x7fdc5815c4a0 2024-04-25T00:32:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5dc24640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdc5815c4a0 0x7fdc5815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5d423640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdc5815cdc0 0x7fdc580cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38334/0 (socket says 172.21.15.29:38334) 2024-04-25T00:32:39.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5d423640 1 -- 172.21.15.29:0/1216044835 learned_addr learned my addr 172.21.15.29:0/1216044835 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:39.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5dc24640 1 --2- 172.21.15.29:0/1216044835 >> v2:172.21.15.29:3300/0 conn(0x7fdc5815c4a0 0x7fdc5815c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54270/0 (socket says 172.21.15.29:54270) 2024-04-25T00:32:39.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5dc24640 1 -- 172.21.15.29:0/1216044835 >> v1:172.21.15.157:6789/0 conn(0x7fdc5806cc60 legacy=0x7fdc5815f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:39.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5dc24640 1 -- 172.21.15.29:0/1216044835 >> v1:172.21.15.29:6789/0 conn(0x7fdc5815cdc0 legacy=0x7fdc580cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5dc24640 1 -- 172.21.15.29:0/1216044835 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc580cf290 con 0x7fdc5815c4a0 2024-04-25T00:32:39.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5cc22640 1 -- 172.21.15.29:0/1216044835 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3522786576 0 0) 0x7fdc580cecd0 con 0x7fdc5815cdc0 2024-04-25T00:32:39.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5dc24640 1 --2- 172.21.15.29:0/1216044835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5815c4a0 0x7fdc5815c880 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdc540097c0 tx=0x7fdc5401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350bdc67e737a7c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:39.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5cc22640 1 -- 172.21.15.29:0/1216044835 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc540b0070 con 0x7fdc5815c4a0 2024-04-25T00:32:39.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5cc22640 1 -- 172.21.15.29:0/1216044835 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc540ab600 con 0x7fdc5815c4a0 2024-04-25T00:32:39.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5cc22640 1 -- 172.21.15.29:0/1216044835 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc540ab9d0 con 0x7fdc5815c4a0 2024-04-25T00:32:39.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5ec26640 1 -- 172.21.15.29:0/1216044835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5815c4a0 msgr2=0x7fdc5815c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.743+0000 7fdc5ec26640 1 --2- 172.21.15.29:0/1216044835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5815c4a0 0x7fdc5815c880 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdc540097c0 tx=0x7fdc5401c3d0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 -- 172.21.15.29:0/1216044835 shutdown_connections 2024-04-25T00:32:39.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 --2- 172.21.15.29:0/1216044835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5815c4a0 0x7fdc5815c880 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 -- 172.21.15.29:0/1216044835 >> 172.21.15.29:0/1216044835 conn(0x7fdc580b1770 msgr2=0x7fdc580b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:39.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 -- 172.21.15.29:0/1216044835 shutdown_connections 2024-04-25T00:32:39.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 -- 172.21.15.29:0/1216044835 wait complete. 2024-04-25T00:32:39.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 Processor -- start 2024-04-25T00:32:39.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 -- start start 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5806cc60 0x7fdc5814e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc5815c4a0 0x7fdc58150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc5815cdc0 0x7fdc58159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5816a0f0 con 0x7fdc5806cc60 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc58169f70 con 0x7fdc5815c4a0 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5816a270 con 0x7fdc5815cdc0 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5dc24640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5806cc60 0x7fdc5814e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5dc24640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5806cc60 0x7fdc5814e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54284/0 (socket says 172.21.15.29:54284) 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5dc24640 1 -- 172.21.15.29:0/3956889754 learned_addr learned my addr 172.21.15.29:0/3956889754 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5d423640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc5815c4a0 0x7fdc58150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5dc24640 1 -- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc5815cdc0 msgr2=0x7fdc58159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5dc24640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc5815cdc0 0x7fdc58159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5dc24640 1 -- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc5815c4a0 msgr2=0x7fdc58150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5dc24640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc5815c4a0 0x7fdc58150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5dc24640 1 -- 172.21.15.29:0/3956889754 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc58151440 con 0x7fdc5806cc60 2024-04-25T00:32:39.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5d423640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc5815c4a0 0x7fdc58150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:39.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5dc24640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5806cc60 0x7fdc5814e060 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdc540ae040 tx=0x7fdc5401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:39.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc4e7fc640 1 -- 172.21.15.29:0/3956889754 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc540b0050 con 0x7fdc5806cc60 2024-04-25T00:32:39.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc4e7fc640 1 -- 172.21.15.29:0/3956889754 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc540bc070 con 0x7fdc5806cc60 2024-04-25T00:32:39.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 -- 172.21.15.29:0/3956889754 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc58151720 con 0x7fdc5806cc60 2024-04-25T00:32:39.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc4e7fc640 1 -- 172.21.15.29:0/3956889754 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc540b7670 con 0x7fdc5806cc60 2024-04-25T00:32:39.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 -- 172.21.15.29:0/3956889754 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc58138330 con 0x7fdc5806cc60 2024-04-25T00:32:39.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.747+0000 7fdc5ec26640 1 -- 172.21.15.29:0/3956889754 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc58069200 con 0x7fdc5806cc60 2024-04-25T00:32:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.751+0000 7fdc4e7fc640 1 -- 172.21.15.29:0/3956889754 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdc540b7810 con 0x7fdc5806cc60 2024-04-25T00:32:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.751+0000 7fdc4e7fc640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc3c0678f0 0x7fdc3c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.751+0000 7fdc4e7fc640 1 -- 172.21.15.29:0/3956889754 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdc541203a0 con 0x7fdc5806cc60 2024-04-25T00:32:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.751+0000 7fdc5d423640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc3c0678f0 0x7fdc3c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.751+0000 7fdc5d423640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc3c0678f0 0x7fdc3c069db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc48007fd0 tx=0x7fdc48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:39.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.755+0000 7fdc4e7fc640 1 -- 172.21.15.29:0/3956889754 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdc54016050 con 0x7fdc5806cc60 2024-04-25T00:32:39.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.907+0000 7fdc5ec26640 1 -- 172.21.15.29:0/3956889754 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdc5815c960 con 0x7fdc3c0678f0 2024-04-25T00:32:39.917 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:32:39.919 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:32:39.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.927+0000 7fdc4e7fc640 1 -- 172.21.15.29:0/3956889754 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7fdc5815c960 con 0x7fdc3c0678f0 2024-04-25T00:32:39.933 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:39.935 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.931+0000 7fdc5ec26640 1 -- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc3c0678f0 msgr2=0x7fdc3c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.931+0000 7fdc5ec26640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc3c0678f0 0x7fdc3c069db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc48007fd0 tx=0x7fdc48009040 comp rx=0 tx=0).stop 2024-04-25T00:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.931+0000 7fdc5ec26640 1 -- 172.21.15.29:0/3956889754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5806cc60 msgr2=0x7fdc5814e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.931+0000 7fdc5ec26640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5806cc60 0x7fdc5814e060 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdc540ae040 tx=0x7fdc5401ac10 comp rx=0 tx=0).stop 2024-04-25T00:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.931+0000 7fdc5ec26640 1 -- 172.21.15.29:0/3956889754 shutdown_connections 2024-04-25T00:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.931+0000 7fdc5ec26640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc5815cdc0 0x7fdc58159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.931+0000 7fdc5ec26640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc3c0678f0 0x7fdc3c069db0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.931+0000 7fdc5ec26640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc5815c4a0 0x7fdc58150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.931+0000 7fdc5ec26640 1 --2- 172.21.15.29:0/3956889754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5806cc60 0x7fdc5814e060 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.931+0000 7fdc5ec26640 1 -- 172.21.15.29:0/3956889754 >> 172.21.15.29:0/3956889754 conn(0x7fdc580b1770 msgr2=0x7fdc5806e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.931+0000 7fdc5ec26640 1 -- 172.21.15.29:0/3956889754 shutdown_connections 2024-04-25T00:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:39.931+0000 7fdc5ec26640 1 -- 172.21.15.29:0/3956889754 wait complete. 2024-04-25T00:32:39.950 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-04-25T00:32:39.467644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253608,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721112,"statfs":{"total":773094113280,"available":772834418688,"internally_reserved":0,"allocated":4235264,"data_stored":2361204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.071416"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.049924+0000","last_change":"2024-04-25T00:32:34.486883+0000","last_active":"2024-04-25T00:32:36.049924+0000","last_peered":"2024-04-25T00:32:36.049924+0000","last_clean":"2024-04-25T00:32:36.049924+0000","last_became_active":"2024-04-25T00:32:34.486381+0000","last_became_peered":"2024-04-25T00:32:34.486381+0000","last_unstale":"2024-04-25T00:32:36.049924+0000","last_undegraded":"2024-04-25T00:32:36.049924+0000","last_fullsized":"2024-04-25T00:32:36.049924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:17.530979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524331+0000","last_change":"2024-04-25T00:32:34.452939+0000","last_active":"2024-04-25T00:32:35.524331+0000","last_peered":"2024-04-25T00:32:35.524331+0000","last_clean":"2024-04-25T00:32:35.524331+0000","last_became_active":"2024-04-25T00:32:34.447660+0000","last_became_peered":"2024-04-25T00:32:34.447660+0000","last_unstale":"2024-04-25T00:32:35.524331+0000","last_undegraded":"2024-04-25T00:32:35.524331+0000","last_fullsized":"2024-04-25T00:32:35.524331+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:04.291110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436237+0000","last_change":"2024-04-25T00:32:34.476785+0000","last_active":"2024-04-25T00:32:35.436237+0000","last_peered":"2024-04-25T00:32:35.436237+0000","last_clean":"2024-04-25T00:32:35.436237+0000","last_became_active":"2024-04-25T00:32:34.475274+0000","last_became_peered":"2024-04-25T00:32:34.475274+0000","last_unstale":"2024-04-25T00:32:35.436237+0000","last_undegraded":"2024-04-25T00:32:35.436237+0000","last_fullsized":"2024-04-25T00:32:35.436237+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:12.912933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.432748+0000","last_change":"2024-04-25T00:32:34.461013+0000","last_active":"2024-04-25T00:32:35.432748+0000","last_peered":"2024-04-25T00:32:35.432748+0000","last_clean":"2024-04-25T00:32:35.432748+0000","last_became_active":"2024-04-25T00:32:34.460700+0000","last_became_peered":"2024-04-25T00:32:34.460700+0000","last_unstale":"2024-04-25T00:32:35.432748+0000","last_undegraded":"2024-04-25T00:32:35.432748+0000","last_fullsized":"2024-04-25T00:32:35.432748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:47.539395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658773+0000","last_change":"2024-04-25T00:32:34.446249+0000","last_active":"2024-04-25T00:32:36.658773+0000","last_peered":"2024-04-25T00:32:36.658773+0000","last_clean":"2024-04-25T00:32:36.658773+0000","last_became_active":"2024-04-25T00:32:34.445950+0000","last_became_peered":"2024-04-25T00:32:34.445950+0000","last_unstale":"2024-04-25T00:32:36.658773+0000","last_undegraded":"2024-04-25T00:32:36.658773+0000","last_fullsized":"2024-04-25T00:32:36.658773+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:34.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.523749+0000","last_change":"2024-04-25T00:32:34.473559+0000","last_active":"2024-04-25T00:32:35.523749+0000","last_peered":"2024-04-25T00:32:35.523749+0000","last_clean":"2024-04-25T00:32:35.523749+0000","last_became_active":"2024-04-25T00:32:34.473242+0000","last_became_peered":"2024-04-25T00:32:34.473242+0000","last_unstale":"2024-04-25T00:32:35.523749+0000","last_undegraded":"2024-04-25T00:32:35.523749+0000","last_fullsized":"2024-04-25T00:32:35.523749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:12.912933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547759+0000","last_change":"2024-04-25T00:32:34.487797+0000","last_active":"2024-04-25T00:32:35.547759+0000","last_peered":"2024-04-25T00:32:35.547759+0000","last_clean":"2024-04-25T00:32:35.547759+0000","last_became_active":"2024-04-25T00:32:34.487034+0000","last_became_peered":"2024-04-25T00:32:34.487034+0000","last_unstale":"2024-04-25T00:32:35.547759+0000","last_undegraded":"2024-04-25T00:32:35.547759+0000","last_fullsized":"2024-04-25T00:32:35.547759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:14.211499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547296+0000","last_change":"2024-04-25T00:32:34.481305+0000","last_active":"2024-04-25T00:32:35.547296+0000","last_peered":"2024-04-25T00:32:35.547296+0000","last_clean":"2024-04-25T00:32:35.547296+0000","last_became_active":"2024-04-25T00:32:34.480558+0000","last_became_peered":"2024-04-25T00:32:34.480558+0000","last_unstale":"2024-04-25T00:32:35.547296+0000","last_undegraded":"2024-04-25T00:32:35.547296+0000","last_fullsized":"2024-04-25T00:32:35.547296+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:18.311016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436415+0000","last_change":"2024-04-25T00:32:34.461475+0000","last_active":"2024-04-25T00:32:35.436415+0000","last_peered":"2024-04-25T00:32:35.436415+0000","last_clean":"2024-04-25T00:32:35.436415+0000","last_became_active":"2024-04-25T00:32:34.460713+0000","last_became_peered":"2024-04-25T00:32:34.460713+0000","last_unstale":"2024-04-25T00:32:35.436415+0000","last_undegraded":"2024-04-25T00:32:35.436415+0000","last_fullsized":"2024-04-25T00:32:35.436415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:38.765850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433632+0000","last_change":"2024-04-25T00:32:34.453887+0000","last_active":"2024-04-25T00:32:35.433632+0000","last_peered":"2024-04-25T00:32:35.433632+0000","last_clean":"2024-04-25T00:32:35.433632+0000","last_became_active":"2024-04-25T00:32:34.450130+0000","last_became_peered":"2024-04-25T00:32:34.450130+0000","last_unstale":"2024-04-25T00:32:35.433632+0000","last_undegraded":"2024-04-25T00:32:35.433632+0000","last_fullsized":"2024-04-25T00:32:35.433632+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:09.955792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.548011+0000","last_change":"2024-04-25T00:32:34.483296+0000","last_active":"2024-04-25T00:32:35.548011+0000","last_peered":"2024-04-25T00:32:35.548011+0000","last_clean":"2024-04-25T00:32:35.548011+0000","last_became_active":"2024-04-25T00:32:34.481034+0000","last_became_peered":"2024-04-25T00:32:34.481034+0000","last_unstale":"2024-04-25T00:32:35.548011+0000","last_undegraded":"2024-04-25T00:32:35.548011+0000","last_fullsized":"2024-04-25T00:32:35.548011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:33.114509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434876+0000","last_change":"2024-04-25T00:32:34.475382+0000","last_active":"2024-04-25T00:32:35.434876+0000","last_peered":"2024-04-25T00:32:35.434876+0000","last_clean":"2024-04-25T00:32:35.434876+0000","last_became_active":"2024-04-25T00:32:34.474197+0000","last_became_peered":"2024-04-25T00:32:34.474197+0000","last_unstale":"2024-04-25T00:32:35.434876+0000","last_undegraded":"2024-04-25T00:32:35.434876+0000","last_fullsized":"2024-04-25T00:32:35.434876+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:48.062329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658880+0000","last_change":"2024-04-25T00:32:34.439134+0000","last_active":"2024-04-25T00:32:36.658880+0000","last_peered":"2024-04-25T00:32:36.658880+0000","last_clean":"2024-04-25T00:32:36.658880+0000","last_became_active":"2024-04-25T00:32:34.438834+0000","last_became_peered":"2024-04-25T00:32:34.438834+0000","last_unstale":"2024-04-25T00:32:36.658880+0000","last_undegraded":"2024-04-25T00:32:36.658880+0000","last_fullsized":"2024-04-25T00:32:36.658880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:35.705579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524075+0000","last_change":"2024-04-25T00:32:34.448199+0000","last_active":"2024-04-25T00:32:35.524075+0000","last_peered":"2024-04-25T00:32:35.524075+0000","last_clean":"2024-04-25T00:32:35.524075+0000","last_became_active":"2024-04-25T00:32:34.441867+0000","last_became_peered":"2024-04-25T00:32:34.441867+0000","last_unstale":"2024-04-25T00:32:35.524075+0000","last_undegraded":"2024-04-25T00:32:35.524075+0000","last_fullsized":"2024-04-25T00:32:35.524075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:54.889078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434188+0000","last_change":"2024-04-25T00:32:34.476484+0000","last_active":"2024-04-25T00:32:35.434188+0000","last_peered":"2024-04-25T00:32:35.434188+0000","last_clean":"2024-04-25T00:32:35.434188+0000","last_became_active":"2024-04-25T00:32:34.475640+0000","last_became_peered":"2024-04-25T00:32:34.475640+0000","last_unstale":"2024-04-25T00:32:35.434188+0000","last_undegraded":"2024-04-25T00:32:35.434188+0000","last_fullsized":"2024-04-25T00:32:35.434188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:02.419942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.548042+0000","last_change":"2024-04-25T00:32:34.491665+0000","last_active":"2024-04-25T00:32:35.548042+0000","last_peered":"2024-04-25T00:32:35.548042+0000","last_clean":"2024-04-25T00:32:35.548042+0000","last_became_active":"2024-04-25T00:32:34.489466+0000","last_became_peered":"2024-04-25T00:32:34.489466+0000","last_unstale":"2024-04-25T00:32:35.548042+0000","last_undegraded":"2024-04-25T00:32:35.548042+0000","last_fullsized":"2024-04-25T00:32:35.548042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:00.562101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.548631+0000","last_change":"2024-04-25T00:32:34.487734+0000","last_active":"2024-04-25T00:32:35.548631+0000","last_peered":"2024-04-25T00:32:35.548631+0000","last_clean":"2024-04-25T00:32:35.548631+0000","last_became_active":"2024-04-25T00:32:34.487116+0000","last_became_peered":"2024-04-25T00:32:34.487116+0000","last_unstale":"2024-04-25T00:32:35.548631+0000","last_undegraded":"2024-04-25T00:32:35.548631+0000","last_fullsized":"2024-04-25T00:32:35.548631+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:52.301520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524588+0000","last_change":"2024-04-25T00:32:34.447795+0000","last_active":"2024-04-25T00:32:35.524588+0000","last_peered":"2024-04-25T00:32:35.524588+0000","last_clean":"2024-04-25T00:32:35.524588+0000","last_became_active":"2024-04-25T00:32:34.447273+0000","last_became_peered":"2024-04-25T00:32:34.447273+0000","last_unstale":"2024-04-25T00:32:35.524588+0000","last_undegraded":"2024-04-25T00:32:35.524588+0000","last_fullsized":"2024-04-25T00:32:35.524588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:08.570179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524684+0000","last_change":"2024-04-25T00:32:34.445862+0000","last_active":"2024-04-25T00:32:35.524684+0000","last_peered":"2024-04-25T00:32:35.524684+0000","last_clean":"2024-04-25T00:32:35.524684+0000","last_became_active":"2024-04-25T00:32:34.444856+0000","last_became_peered":"2024-04-25T00:32:34.444856+0000","last_unstale":"2024-04-25T00:32:35.524684+0000","last_undegraded":"2024-04-25T00:32:35.524684+0000","last_fullsized":"2024-04-25T00:32:35.524684+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:52.301520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658952+0000","last_change":"2024-04-25T00:32:34.478706+0000","last_active":"2024-04-25T00:32:36.658952+0000","last_peered":"2024-04-25T00:32:36.658952+0000","last_clean":"2024-04-25T00:32:36.658952+0000","last_became_active":"2024-04-25T00:32:34.478390+0000","last_became_peered":"2024-04-25T00:32:34.478390+0000","last_unstale":"2024-04-25T00:32:36.658952+0000","last_undegraded":"2024-04-25T00:32:36.658952+0000","last_fullsized":"2024-04-25T00:32:36.658952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:49.942783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435859+0000","last_change":"2024-04-25T00:32:34.464425+0000","last_active":"2024-04-25T00:32:35.435859+0000","last_peered":"2024-04-25T00:32:35.435859+0000","last_clean":"2024-04-25T00:32:35.435859+0000","last_became_active":"2024-04-25T00:32:34.456321+0000","last_became_peered":"2024-04-25T00:32:34.456321+0000","last_unstale":"2024-04-25T00:32:35.435859+0000","last_undegraded":"2024-04-25T00:32:35.435859+0000","last_fullsized":"2024-04-25T00:32:35.435859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:05.464400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435515+0000","last_change":"2024-04-25T00:32:34.449881+0000","last_active":"2024-04-25T00:32:35.435515+0000","last_peered":"2024-04-25T00:32:35.435515+0000","last_clean":"2024-04-25T00:32:35.435515+0000","last_became_active":"2024-04-25T00:32:34.448725+0000","last_became_peered":"2024-04-25T00:32:34.448725+0000","last_unstale":"2024-04-25T00:32:35.435515+0000","last_undegraded":"2024-04-25T00:32:35.435515+0000","last_fullsized":"2024-04-25T00:32:35.435515+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:17.530979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.436714+0000","last_change":"2024-04-25T00:32:34.467564+0000","last_active":"2024-04-25T00:32:36.436714+0000","last_peered":"2024-04-25T00:32:36.436714+0000","last_clean":"2024-04-25T00:32:36.436714+0000","last_became_active":"2024-04-25T00:32:34.467264+0000","last_became_peered":"2024-04-25T00:32:34.467264+0000","last_unstale":"2024-04-25T00:32:36.436714+0000","last_undegraded":"2024-04-25T00:32:36.436714+0000","last_fullsized":"2024-04-25T00:32:36.436714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:37.803710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433848+0000","last_change":"2024-04-25T00:32:34.474541+0000","last_active":"2024-04-25T00:32:35.433848+0000","last_peered":"2024-04-25T00:32:35.433848+0000","last_clean":"2024-04-25T00:32:35.433848+0000","last_became_active":"2024-04-25T00:32:34.473805+0000","last_became_peered":"2024-04-25T00:32:34.473805+0000","last_unstale":"2024-04-25T00:32:35.433848+0000","last_undegraded":"2024-04-25T00:32:35.433848+0000","last_fullsized":"2024-04-25T00:32:35.433848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:45.660093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433310+0000","last_change":"2024-04-25T00:32:34.469626+0000","last_active":"2024-04-25T00:32:35.433310+0000","last_peered":"2024-04-25T00:32:35.433310+0000","last_clean":"2024-04-25T00:32:35.433310+0000","last_became_active":"2024-04-25T00:32:34.469259+0000","last_became_peered":"2024-04-25T00:32:34.469259+0000","last_unstale":"2024-04-25T00:32:35.433310+0000","last_undegraded":"2024-04-25T00:32:35.433310+0000","last_fullsized":"2024-04-25T00:32:35.433310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:47.677244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524630+0000","last_change":"2024-04-25T00:32:34.469477+0000","last_active":"2024-04-25T00:32:35.524630+0000","last_peered":"2024-04-25T00:32:35.524630+0000","last_clean":"2024-04-25T00:32:35.524630+0000","last_became_active":"2024-04-25T00:32:34.469157+0000","last_became_peered":"2024-04-25T00:32:34.469157+0000","last_unstale":"2024-04-25T00:32:35.524630+0000","last_undegraded":"2024-04-25T00:32:35.524630+0000","last_fullsized":"2024-04-25T00:32:35.524630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:16.527471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433333+0000","last_change":"2024-04-25T00:32:34.461083+0000","last_active":"2024-04-25T00:32:35.433333+0000","last_peered":"2024-04-25T00:32:35.433333+0000","last_clean":"2024-04-25T00:32:35.433333+0000","last_became_active":"2024-04-25T00:32:34.460777+0000","last_became_peered":"2024-04-25T00:32:34.460777+0000","last_unstale":"2024-04-25T00:32:35.433333+0000","last_undegraded":"2024-04-25T00:32:35.433333+0000","last_fullsized":"2024-04-25T00:32:35.433333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:54.672586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434880+0000","last_change":"2024-04-25T00:32:34.474938+0000","last_active":"2024-04-25T00:32:35.434880+0000","last_peered":"2024-04-25T00:32:35.434880+0000","last_clean":"2024-04-25T00:32:35.434880+0000","last_became_active":"2024-04-25T00:32:34.473570+0000","last_became_peered":"2024-04-25T00:32:34.473570+0000","last_unstale":"2024-04-25T00:32:35.434880+0000","last_undegraded":"2024-04-25T00:32:35.434880+0000","last_fullsized":"2024-04-25T00:32:35.434880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:39.687967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435695+0000","last_change":"2024-04-25T00:32:34.457058+0000","last_active":"2024-04-25T00:32:35.435695+0000","last_peered":"2024-04-25T00:32:35.435695+0000","last_clean":"2024-04-25T00:32:35.435695+0000","last_became_active":"2024-04-25T00:32:34.456275+0000","last_became_peered":"2024-04-25T00:32:34.456275+0000","last_unstale":"2024-04-25T00:32:35.435695+0000","last_undegraded":"2024-04-25T00:32:35.435695+0000","last_fullsized":"2024-04-25T00:32:35.435695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:25.019780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547711+0000","last_change":"2024-04-25T00:32:34.487878+0000","last_active":"2024-04-25T00:32:35.547711+0000","last_peered":"2024-04-25T00:32:35.547711+0000","last_clean":"2024-04-25T00:32:35.547711+0000","last_became_active":"2024-04-25T00:32:34.487179+0000","last_became_peered":"2024-04-25T00:32:34.487179+0000","last_unstale":"2024-04-25T00:32:35.547711+0000","last_undegraded":"2024-04-25T00:32:35.547711+0000","last_fullsized":"2024-04-25T00:32:35.547711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:04.291110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.049480+0000","last_change":"2024-04-25T00:32:34.453427+0000","last_active":"2024-04-25T00:32:36.049480+0000","last_peered":"2024-04-25T00:32:36.049480+0000","last_clean":"2024-04-25T00:32:36.049480+0000","last_became_active":"2024-04-25T00:32:34.452661+0000","last_became_peered":"2024-04-25T00:32:34.452661+0000","last_unstale":"2024-04-25T00:32:36.049480+0000","last_undegraded":"2024-04-25T00:32:36.049480+0000","last_fullsized":"2024-04-25T00:32:36.049480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:40.112885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547273+0000","last_change":"2024-04-25T00:32:34.483634+0000","last_active":"2024-04-25T00:32:35.547273+0000","last_peered":"2024-04-25T00:32:35.547273+0000","last_clean":"2024-04-25T00:32:35.547273+0000","last_became_active":"2024-04-25T00:32:34.481912+0000","last_became_peered":"2024-04-25T00:32:34.481912+0000","last_unstale":"2024-04-25T00:32:35.547273+0000","last_undegraded":"2024-04-25T00:32:35.547273+0000","last_fullsized":"2024-04-25T00:32:35.547273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:48.062329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437295+0000","last_change":"2024-04-25T00:32:34.481329+0000","last_active":"2024-04-25T00:32:36.437295+0000","last_peered":"2024-04-25T00:32:36.437295+0000","last_clean":"2024-04-25T00:32:36.437295+0000","last_became_active":"2024-04-25T00:32:34.480962+0000","last_became_peered":"2024-04-25T00:32:34.480962+0000","last_unstale":"2024-04-25T00:32:36.437295+0000","last_undegraded":"2024-04-25T00:32:36.437295+0000","last_fullsized":"2024-04-25T00:32:36.437295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:05.464400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.050080+0000","last_change":"2024-04-25T00:32:33.439097+0000","last_active":"2024-04-25T00:32:36.050080+0000","last_peered":"2024-04-25T00:32:36.050080+0000","last_clean":"2024-04-25T00:32:36.050080+0000","last_became_active":"2024-04-25T00:32:33.438390+0000","last_became_peered":"2024-04-25T00:32:33.438390+0000","last_unstale":"2024-04-25T00:32:36.050080+0000","last_undegraded":"2024-04-25T00:32:36.050080+0000","last_fullsized":"2024-04-25T00:32:36.050080+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.651396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.050023+0000","last_change":"2024-04-25T00:32:34.477891+0000","last_active":"2024-04-25T00:32:36.050023+0000","last_peered":"2024-04-25T00:32:36.050023+0000","last_clean":"2024-04-25T00:32:36.050023+0000","last_became_active":"2024-04-25T00:32:34.477107+0000","last_became_peered":"2024-04-25T00:32:34.477107+0000","last_unstale":"2024-04-25T00:32:36.050023+0000","last_undegraded":"2024-04-25T00:32:36.050023+0000","last_fullsized":"2024-04-25T00:32:36.050023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:25.019780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437181+0000","last_change":"2024-04-25T00:32:33.439749+0000","last_active":"2024-04-25T00:32:36.437181+0000","last_peered":"2024-04-25T00:32:36.437181+0000","last_clean":"2024-04-25T00:32:36.437181+0000","last_became_active":"2024-04-25T00:32:33.439548+0000","last_became_peered":"2024-04-25T00:32:33.439548+0000","last_unstale":"2024-04-25T00:32:36.437181+0000","last_undegraded":"2024-04-25T00:32:36.437181+0000","last_fullsized":"2024-04-25T00:32:36.437181+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:07.836095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436388+0000","last_change":"2024-04-25T00:32:34.449587+0000","last_active":"2024-04-25T00:32:35.436388+0000","last_peered":"2024-04-25T00:32:35.436388+0000","last_clean":"2024-04-25T00:32:35.436388+0000","last_became_active":"2024-04-25T00:32:34.448891+0000","last_became_peered":"2024-04-25T00:32:34.448891+0000","last_unstale":"2024-04-25T00:32:35.436388+0000","last_undegraded":"2024-04-25T00:32:35.436388+0000","last_fullsized":"2024-04-25T00:32:35.436388+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:47.539395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524834+0000","last_change":"2024-04-25T00:32:33.419772+0000","last_active":"2024-04-25T00:32:35.524834+0000","last_peered":"2024-04-25T00:32:35.524834+0000","last_clean":"2024-04-25T00:32:35.524834+0000","last_became_active":"2024-04-25T00:32:33.419459+0000","last_became_peered":"2024-04-25T00:32:33.419459+0000","last_unstale":"2024-04-25T00:32:35.524834+0000","last_undegraded":"2024-04-25T00:32:35.524834+0000","last_fullsized":"2024-04-25T00:32:35.524834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:24.010578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433709+0000","last_change":"2024-04-25T00:32:34.460897+0000","last_active":"2024-04-25T00:32:35.433709+0000","last_peered":"2024-04-25T00:32:35.433709+0000","last_clean":"2024-04-25T00:32:35.433709+0000","last_became_active":"2024-04-25T00:32:34.460596+0000","last_became_peered":"2024-04-25T00:32:34.460596+0000","last_unstale":"2024-04-25T00:32:35.433709+0000","last_undegraded":"2024-04-25T00:32:35.433709+0000","last_fullsized":"2024-04-25T00:32:35.433709+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:45.660093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658987+0000","last_change":"2024-04-25T00:32:33.419126+0000","last_active":"2024-04-25T00:32:36.658987+0000","last_peered":"2024-04-25T00:32:36.658987+0000","last_clean":"2024-04-25T00:32:36.658987+0000","last_became_active":"2024-04-25T00:32:33.418834+0000","last_became_peered":"2024-04-25T00:32:33.418834+0000","last_unstale":"2024-04-25T00:32:36.658987+0000","last_undegraded":"2024-04-25T00:32:36.658987+0000","last_fullsized":"2024-04-25T00:32:36.658987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:30.363273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.548312+0000","last_change":"2024-04-25T00:32:34.468459+0000","last_active":"2024-04-25T00:32:35.548312+0000","last_peered":"2024-04-25T00:32:35.548312+0000","last_clean":"2024-04-25T00:32:35.548312+0000","last_became_active":"2024-04-25T00:32:34.466447+0000","last_became_peered":"2024-04-25T00:32:34.466447+0000","last_unstale":"2024-04-25T00:32:35.548312+0000","last_undegraded":"2024-04-25T00:32:35.548312+0000","last_fullsized":"2024-04-25T00:32:35.548312+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:08.570179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435957+0000","last_change":"2024-04-25T00:32:33.429088+0000","last_active":"2024-04-25T00:32:35.435957+0000","last_peered":"2024-04-25T00:32:35.435957+0000","last_clean":"2024-04-25T00:32:35.435957+0000","last_became_active":"2024-04-25T00:32:33.427379+0000","last_became_peered":"2024-04-25T00:32:33.427379+0000","last_unstale":"2024-04-25T00:32:35.435957+0000","last_undegraded":"2024-04-25T00:32:35.435957+0000","last_fullsized":"2024-04-25T00:32:35.435957+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:10.605816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434676+0000","last_change":"2024-04-25T00:32:34.457417+0000","last_active":"2024-04-25T00:32:35.434676+0000","last_peered":"2024-04-25T00:32:35.434676+0000","last_clean":"2024-04-25T00:32:35.434676+0000","last_became_active":"2024-04-25T00:32:34.456659+0000","last_became_peered":"2024-04-25T00:32:34.456659+0000","last_unstale":"2024-04-25T00:32:35.434676+0000","last_undegraded":"2024-04-25T00:32:35.434676+0000","last_fullsized":"2024-04-25T00:32:35.434676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.038817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435766+0000","last_change":"2024-04-25T00:32:33.429965+0000","last_active":"2024-04-25T00:32:35.435766+0000","last_peered":"2024-04-25T00:32:35.435766+0000","last_clean":"2024-04-25T00:32:35.435766+0000","last_became_active":"2024-04-25T00:32:33.427356+0000","last_became_peered":"2024-04-25T00:32:33.427356+0000","last_unstale":"2024-04-25T00:32:35.435766+0000","last_undegraded":"2024-04-25T00:32:35.435766+0000","last_fullsized":"2024-04-25T00:32:35.435766+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:33.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434419+0000","last_change":"2024-04-25T00:32:34.454184+0000","last_active":"2024-04-25T00:32:35.434419+0000","last_peered":"2024-04-25T00:32:35.434419+0000","last_clean":"2024-04-25T00:32:35.434419+0000","last_became_active":"2024-04-25T00:32:34.453126+0000","last_became_peered":"2024-04-25T00:32:34.453126+0000","last_unstale":"2024-04-25T00:32:35.434419+0000","last_undegraded":"2024-04-25T00:32:35.434419+0000","last_fullsized":"2024-04-25T00:32:35.434419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:16.435951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.523449+0000","last_change":"2024-04-25T00:32:33.419720+0000","last_active":"2024-04-25T00:32:35.523449+0000","last_peered":"2024-04-25T00:32:35.523449+0000","last_clean":"2024-04-25T00:32:35.523449+0000","last_became_active":"2024-04-25T00:32:33.419338+0000","last_became_peered":"2024-04-25T00:32:33.419338+0000","last_unstale":"2024-04-25T00:32:35.523449+0000","last_undegraded":"2024-04-25T00:32:35.523449+0000","last_fullsized":"2024-04-25T00:32:35.523449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:15.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.432209+0000","last_change":"2024-04-25T00:32:34.473522+0000","last_active":"2024-04-25T00:32:35.432209+0000","last_peered":"2024-04-25T00:32:35.432209+0000","last_clean":"2024-04-25T00:32:35.432209+0000","last_became_active":"2024-04-25T00:32:34.473212+0000","last_became_peered":"2024-04-25T00:32:34.473212+0000","last_unstale":"2024-04-25T00:32:35.432209+0000","last_undegraded":"2024-04-25T00:32:35.432209+0000","last_fullsized":"2024-04-25T00:32:35.432209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:05.464400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434873+0000","last_change":"2024-04-25T00:32:33.433081+0000","last_active":"2024-04-25T00:32:35.434873+0000","last_peered":"2024-04-25T00:32:35.434873+0000","last_clean":"2024-04-25T00:32:35.434873+0000","last_became_active":"2024-04-25T00:32:33.432236+0000","last_became_peered":"2024-04-25T00:32:33.432236+0000","last_unstale":"2024-04-25T00:32:35.434873+0000","last_undegraded":"2024-04-25T00:32:35.434873+0000","last_fullsized":"2024-04-25T00:32:35.434873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:25.652624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436121+0000","last_change":"2024-04-25T00:32:34.478755+0000","last_active":"2024-04-25T00:32:35.436121+0000","last_peered":"2024-04-25T00:32:35.436121+0000","last_clean":"2024-04-25T00:32:35.436121+0000","last_became_active":"2024-04-25T00:32:34.478125+0000","last_became_peered":"2024-04-25T00:32:34.478125+0000","last_unstale":"2024-04-25T00:32:35.436121+0000","last_undegraded":"2024-04-25T00:32:35.436121+0000","last_fullsized":"2024-04-25T00:32:35.436121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:18.311016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524685+0000","last_change":"2024-04-25T00:32:33.431456+0000","last_active":"2024-04-25T00:32:35.524685+0000","last_peered":"2024-04-25T00:32:35.524685+0000","last_clean":"2024-04-25T00:32:35.524685+0000","last_became_active":"2024-04-25T00:32:33.431235+0000","last_became_peered":"2024-04-25T00:32:33.431235+0000","last_unstale":"2024-04-25T00:32:35.524685+0000","last_undegraded":"2024-04-25T00:32:35.524685+0000","last_fullsized":"2024-04-25T00:32:35.524685+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:04.455198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524303+0000","last_change":"2024-04-25T00:32:34.445434+0000","last_active":"2024-04-25T00:32:35.524303+0000","last_peered":"2024-04-25T00:32:35.524303+0000","last_clean":"2024-04-25T00:32:35.524303+0000","last_became_active":"2024-04-25T00:32:34.445151+0000","last_became_peered":"2024-04-25T00:32:34.445151+0000","last_unstale":"2024-04-25T00:32:35.524303+0000","last_undegraded":"2024-04-25T00:32:35.524303+0000","last_fullsized":"2024-04-25T00:32:35.524303+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:33.114509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437574+0000","last_change":"2024-04-25T00:32:33.416642+0000","last_active":"2024-04-25T00:32:36.437574+0000","last_peered":"2024-04-25T00:32:36.437574+0000","last_clean":"2024-04-25T00:32:36.437574+0000","last_became_active":"2024-04-25T00:32:33.416316+0000","last_became_peered":"2024-04-25T00:32:33.416316+0000","last_unstale":"2024-04-25T00:32:36.437574+0000","last_undegraded":"2024-04-25T00:32:36.437574+0000","last_fullsized":"2024-04-25T00:32:36.437574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:48.933581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524467+0000","last_change":"2024-04-25T00:32:34.468094+0000","last_active":"2024-04-25T00:32:35.524467+0000","last_peered":"2024-04-25T00:32:35.524467+0000","last_clean":"2024-04-25T00:32:35.524467+0000","last_became_active":"2024-04-25T00:32:34.467807+0000","last_became_peered":"2024-04-25T00:32:34.467807+0000","last_unstale":"2024-04-25T00:32:35.524467+0000","last_undegraded":"2024-04-25T00:32:35.524467+0000","last_fullsized":"2024-04-25T00:32:35.524467+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:00.562101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436467+0000","last_change":"2024-04-25T00:32:33.424885+0000","last_active":"2024-04-25T00:32:35.436467+0000","last_peered":"2024-04-25T00:32:35.436467+0000","last_clean":"2024-04-25T00:32:35.436467+0000","last_became_active":"2024-04-25T00:32:33.424174+0000","last_became_peered":"2024-04-25T00:32:33.424174+0000","last_unstale":"2024-04-25T00:32:35.436467+0000","last_undegraded":"2024-04-25T00:32:35.436467+0000","last_fullsized":"2024-04-25T00:32:35.436467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:15.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436953+0000","last_change":"2024-04-25T00:32:34.477357+0000","last_active":"2024-04-25T00:32:35.436953+0000","last_peered":"2024-04-25T00:32:35.436953+0000","last_clean":"2024-04-25T00:32:35.436953+0000","last_became_active":"2024-04-25T00:32:34.476024+0000","last_became_peered":"2024-04-25T00:32:34.476024+0000","last_unstale":"2024-04-25T00:32:35.436953+0000","last_undegraded":"2024-04-25T00:32:35.436953+0000","last_fullsized":"2024-04-25T00:32:35.436953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:14.211499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524398+0000","last_change":"2024-04-25T00:32:33.420265+0000","last_active":"2024-04-25T00:32:35.524398+0000","last_peered":"2024-04-25T00:32:35.524398+0000","last_clean":"2024-04-25T00:32:35.524398+0000","last_became_active":"2024-04-25T00:32:33.419977+0000","last_became_peered":"2024-04-25T00:32:33.419977+0000","last_unstale":"2024-04-25T00:32:35.524398+0000","last_undegraded":"2024-04-25T00:32:35.524398+0000","last_fullsized":"2024-04-25T00:32:35.524398+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:16.521777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.432381+0000","last_change":"2024-04-25T00:32:34.473507+0000","last_active":"2024-04-25T00:32:35.432381+0000","last_peered":"2024-04-25T00:32:35.432381+0000","last_clean":"2024-04-25T00:32:35.432381+0000","last_became_active":"2024-04-25T00:32:34.473224+0000","last_became_peered":"2024-04-25T00:32:34.473224+0000","last_unstale":"2024-04-25T00:32:35.432381+0000","last_undegraded":"2024-04-25T00:32:35.432381+0000","last_fullsized":"2024-04-25T00:32:35.432381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:04.587403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436609+0000","last_change":"2024-04-25T00:32:33.434692+0000","last_active":"2024-04-25T00:32:35.436609+0000","last_peered":"2024-04-25T00:32:35.436609+0000","last_clean":"2024-04-25T00:32:35.436609+0000","last_became_active":"2024-04-25T00:32:33.433973+0000","last_became_peered":"2024-04-25T00:32:33.433973+0000","last_unstale":"2024-04-25T00:32:35.436609+0000","last_undegraded":"2024-04-25T00:32:35.436609+0000","last_fullsized":"2024-04-25T00:32:35.436609+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.651396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524304+0000","last_change":"2024-04-25T00:32:34.447270+0000","last_active":"2024-04-25T00:32:35.524304+0000","last_peered":"2024-04-25T00:32:35.524304+0000","last_clean":"2024-04-25T00:32:35.524304+0000","last_became_active":"2024-04-25T00:32:34.446955+0000","last_became_peered":"2024-04-25T00:32:34.446955+0000","last_unstale":"2024-04-25T00:32:35.524304+0000","last_undegraded":"2024-04-25T00:32:35.524304+0000","last_fullsized":"2024-04-25T00:32:35.524304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:14.211499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.432017+0000","last_change":"2024-04-25T00:32:33.432224+0000","last_active":"2024-04-25T00:32:35.432017+0000","last_peered":"2024-04-25T00:32:35.432017+0000","last_clean":"2024-04-25T00:32:35.432017+0000","last_became_active":"2024-04-25T00:32:33.431863+0000","last_became_peered":"2024-04-25T00:32:33.431863+0000","last_unstale":"2024-04-25T00:32:35.432017+0000","last_undegraded":"2024-04-25T00:32:35.432017+0000","last_fullsized":"2024-04-25T00:32:35.432017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.651396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547641+0000","last_change":"2024-04-25T00:32:34.488122+0000","last_active":"2024-04-25T00:32:35.547641+0000","last_peered":"2024-04-25T00:32:35.547641+0000","last_clean":"2024-04-25T00:32:35.547641+0000","last_became_active":"2024-04-25T00:32:34.482551+0000","last_became_peered":"2024-04-25T00:32:34.482551+0000","last_unstale":"2024-04-25T00:32:35.547641+0000","last_undegraded":"2024-04-25T00:32:35.547641+0000","last_fullsized":"2024-04-25T00:32:35.547641+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:12.912933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.523649+0000","last_change":"2024-04-25T00:32:33.431495+0000","last_active":"2024-04-25T00:32:35.523649+0000","last_peered":"2024-04-25T00:32:35.523649+0000","last_clean":"2024-04-25T00:32:35.523649+0000","last_became_active":"2024-04-25T00:32:33.431231+0000","last_became_peered":"2024-04-25T00:32:33.431231+0000","last_unstale":"2024-04-25T00:32:35.523649+0000","last_undegraded":"2024-04-25T00:32:35.523649+0000","last_fullsized":"2024-04-25T00:32:35.523649+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:15.602236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.049859+0000","last_change":"2024-04-25T00:32:34.473569+0000","last_active":"2024-04-25T00:32:36.049859+0000","last_peered":"2024-04-25T00:32:36.049859+0000","last_clean":"2024-04-25T00:32:36.049859+0000","last_became_active":"2024-04-25T00:32:34.472936+0000","last_became_peered":"2024-04-25T00:32:34.472936+0000","last_unstale":"2024-04-25T00:32:36.049859+0000","last_undegraded":"2024-04-25T00:32:36.049859+0000","last_fullsized":"2024-04-25T00:32:36.049859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:37.803710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.523514+0000","last_change":"2024-04-25T00:32:33.419970+0000","last_active":"2024-04-25T00:32:35.523514+0000","last_peered":"2024-04-25T00:32:35.523514+0000","last_clean":"2024-04-25T00:32:35.523514+0000","last_became_active":"2024-04-25T00:32:33.419465+0000","last_became_peered":"2024-04-25T00:32:33.419465+0000","last_unstale":"2024-04-25T00:32:35.523514+0000","last_undegraded":"2024-04-25T00:32:35.523514+0000","last_fullsized":"2024-04-25T00:32:35.523514+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:39.103683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436268+0000","last_change":"2024-04-25T00:32:34.484992+0000","last_active":"2024-04-25T00:32:35.436268+0000","last_peered":"2024-04-25T00:32:35.436268+0000","last_clean":"2024-04-25T00:32:35.436268+0000","last_became_active":"2024-04-25T00:32:34.484250+0000","last_became_peered":"2024-04-25T00:32:34.484250+0000","last_unstale":"2024-04-25T00:32:35.436268+0000","last_undegraded":"2024-04-25T00:32:35.436268+0000","last_fullsized":"2024-04-25T00:32:35.436268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:04.587403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433441+0000","last_change":"2024-04-25T00:32:33.418814+0000","last_active":"2024-04-25T00:32:35.433441+0000","last_peered":"2024-04-25T00:32:35.433441+0000","last_clean":"2024-04-25T00:32:35.433441+0000","last_became_active":"2024-04-25T00:32:33.418270+0000","last_became_peered":"2024-04-25T00:32:33.418270+0000","last_unstale":"2024-04-25T00:32:35.433441+0000","last_undegraded":"2024-04-25T00:32:35.433441+0000","last_fullsized":"2024-04-25T00:32:35.433441+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:48.933581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436577+0000","last_change":"2024-04-25T00:32:34.485344+0000","last_active":"2024-04-25T00:32:35.436577+0000","last_peered":"2024-04-25T00:32:35.436577+0000","last_clean":"2024-04-25T00:32:35.436577+0000","last_became_active":"2024-04-25T00:32:34.484661+0000","last_became_peered":"2024-04-25T00:32:34.484661+0000","last_unstale":"2024-04-25T00:32:35.436577+0000","last_undegraded":"2024-04-25T00:32:35.436577+0000","last_fullsized":"2024-04-25T00:32:35.436577+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:54.889078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436426+0000","last_change":"2024-04-25T00:32:33.434714+0000","last_active":"2024-04-25T00:32:35.436426+0000","last_peered":"2024-04-25T00:32:35.436426+0000","last_clean":"2024-04-25T00:32:35.436426+0000","last_became_active":"2024-04-25T00:32:33.433950+0000","last_became_peered":"2024-04-25T00:32:33.433950+0000","last_unstale":"2024-04-25T00:32:35.436426+0000","last_undegraded":"2024-04-25T00:32:35.436426+0000","last_fullsized":"2024-04-25T00:32:35.436426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:18.360499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433004+0000","last_change":"2024-04-25T00:32:34.455386+0000","last_active":"2024-04-25T00:32:35.433004+0000","last_peered":"2024-04-25T00:32:35.433004+0000","last_clean":"2024-04-25T00:32:35.433004+0000","last_became_active":"2024-04-25T00:32:34.454053+0000","last_became_peered":"2024-04-25T00:32:34.454053+0000","last_unstale":"2024-04-25T00:32:35.433004+0000","last_undegraded":"2024-04-25T00:32:35.433004+0000","last_fullsized":"2024-04-25T00:32:35.433004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:38.765850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436715+0000","last_change":"2024-04-25T00:32:33.412615+0000","last_active":"2024-04-25T00:32:35.436715+0000","last_peered":"2024-04-25T00:32:35.436715+0000","last_clean":"2024-04-25T00:32:35.436715+0000","last_became_active":"2024-04-25T00:32:33.411820+0000","last_became_peered":"2024-04-25T00:32:33.411820+0000","last_unstale":"2024-04-25T00:32:35.436715+0000","last_undegraded":"2024-04-25T00:32:35.436715+0000","last_fullsized":"2024-04-25T00:32:35.436715+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:26.408279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658882+0000","last_change":"2024-04-25T00:32:34.467502+0000","last_active":"2024-04-25T00:32:36.658882+0000","last_peered":"2024-04-25T00:32:36.658882+0000","last_clean":"2024-04-25T00:32:36.658882+0000","last_became_active":"2024-04-25T00:32:34.467168+0000","last_became_peered":"2024-04-25T00:32:34.467168+0000","last_unstale":"2024-04-25T00:32:36.658882+0000","last_undegraded":"2024-04-25T00:32:36.658882+0000","last_fullsized":"2024-04-25T00:32:36.658882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:08.845297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436398+0000","last_change":"2024-04-25T00:32:33.429101+0000","last_active":"2024-04-25T00:32:35.436398+0000","last_peered":"2024-04-25T00:32:35.436398+0000","last_clean":"2024-04-25T00:32:35.436398+0000","last_became_active":"2024-04-25T00:32:33.428344+0000","last_became_peered":"2024-04-25T00:32:33.428344+0000","last_unstale":"2024-04-25T00:32:35.436398+0000","last_undegraded":"2024-04-25T00:32:35.436398+0000","last_fullsized":"2024-04-25T00:32:35.436398+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:39.207154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658835+0000","last_change":"2024-04-25T00:32:34.463991+0000","last_active":"2024-04-25T00:32:36.658835+0000","last_peered":"2024-04-25T00:32:36.658835+0000","last_clean":"2024-04-25T00:32:36.658835+0000","last_became_active":"2024-04-25T00:32:34.463661+0000","last_became_peered":"2024-04-25T00:32:34.463661+0000","last_unstale":"2024-04-25T00:32:36.658835+0000","last_undegraded":"2024-04-25T00:32:36.658835+0000","last_fullsized":"2024-04-25T00:32:36.658835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:11.615017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437175+0000","last_change":"2024-04-25T00:32:33.419832+0000","last_active":"2024-04-25T00:32:36.437175+0000","last_peered":"2024-04-25T00:32:36.437175+0000","last_clean":"2024-04-25T00:32:36.437175+0000","last_became_active":"2024-04-25T00:32:33.419551+0000","last_became_peered":"2024-04-25T00:32:33.419551+0000","last_unstale":"2024-04-25T00:32:36.437175+0000","last_undegraded":"2024-04-25T00:32:36.437175+0000","last_fullsized":"2024-04-25T00:32:36.437175+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.651396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.659043+0000","last_change":"2024-04-25T00:32:34.447073+0000","last_active":"2024-04-25T00:32:36.659043+0000","last_peered":"2024-04-25T00:32:36.659043+0000","last_clean":"2024-04-25T00:32:36.659043+0000","last_became_active":"2024-04-25T00:32:34.446798+0000","last_became_peered":"2024-04-25T00:32:34.446798+0000","last_unstale":"2024-04-25T00:32:36.659043+0000","last_undegraded":"2024-04-25T00:32:36.659043+0000","last_fullsized":"2024-04-25T00:32:36.659043+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:29.301051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.049651+0000","last_change":"2024-04-25T00:32:33.432483+0000","last_active":"2024-04-25T00:32:36.049651+0000","last_peered":"2024-04-25T00:32:36.049651+0000","last_clean":"2024-04-25T00:32:36.049651+0000","last_became_active":"2024-04-25T00:32:33.431774+0000","last_became_peered":"2024-04-25T00:32:33.431774+0000","last_unstale":"2024-04-25T00:32:36.049651+0000","last_undegraded":"2024-04-25T00:32:36.049651+0000","last_fullsized":"2024-04-25T00:32:36.049651+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:34.696377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658919+0000","last_change":"2024-04-25T00:32:34.441629+0000","last_active":"2024-04-25T00:32:36.658919+0000","last_peered":"2024-04-25T00:32:36.658919+0000","last_clean":"2024-04-25T00:32:36.658919+0000","last_became_active":"2024-04-25T00:32:34.441306+0000","last_became_peered":"2024-04-25T00:32:34.441306+0000","last_unstale":"2024-04-25T00:32:36.658919+0000","last_undegraded":"2024-04-25T00:32:36.658919+0000","last_fullsized":"2024-04-25T00:32:36.658919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:25.129046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435133+0000","last_change":"2024-04-25T00:32:33.426645+0000","last_active":"2024-04-25T00:32:35.435133+0000","last_peered":"2024-04-25T00:32:35.435133+0000","last_clean":"2024-04-25T00:32:35.435133+0000","last_became_active":"2024-04-25T00:32:33.425875+0000","last_became_peered":"2024-04-25T00:32:33.425875+0000","last_unstale":"2024-04-25T00:32:35.435133+0000","last_undegraded":"2024-04-25T00:32:35.435133+0000","last_fullsized":"2024-04-25T00:32:35.435133+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:18.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547722+0000","last_change":"2024-04-25T00:32:34.482697+0000","last_active":"2024-04-25T00:32:35.547722+0000","last_peered":"2024-04-25T00:32:35.547722+0000","last_clean":"2024-04-25T00:32:35.547722+0000","last_became_active":"2024-04-25T00:32:34.482147+0000","last_became_peered":"2024-04-25T00:32:34.482147+0000","last_unstale":"2024-04-25T00:32:35.547722+0000","last_undegraded":"2024-04-25T00:32:35.547722+0000","last_fullsized":"2024-04-25T00:32:35.547722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:54.889078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434106+0000","last_change":"2024-04-25T00:32:33.441614+0000","last_active":"2024-04-25T00:32:35.434106+0000","last_peered":"2024-04-25T00:32:35.434106+0000","last_clean":"2024-04-25T00:32:35.434106+0000","last_became_active":"2024-04-25T00:32:33.441004+0000","last_became_peered":"2024-04-25T00:32:33.441004+0000","last_unstale":"2024-04-25T00:32:35.434106+0000","last_undegraded":"2024-04-25T00:32:35.434106+0000","last_fullsized":"2024-04-25T00:32:35.434106+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:39.103683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658870+0000","last_change":"2024-04-25T00:32:34.478773+0000","last_active":"2024-04-25T00:32:36.658870+0000","last_peered":"2024-04-25T00:32:36.658870+0000","last_clean":"2024-04-25T00:32:36.658870+0000","last_became_active":"2024-04-25T00:32:34.478484+0000","last_became_peered":"2024-04-25T00:32:34.478484+0000","last_unstale":"2024-04-25T00:32:36.658870+0000","last_undegraded":"2024-04-25T00:32:36.658870+0000","last_fullsized":"2024-04-25T00:32:36.658870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:29.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433516+0000","last_change":"2024-04-25T00:32:33.412949+0000","last_active":"2024-04-25T00:32:35.433516+0000","last_peered":"2024-04-25T00:32:35.433516+0000","last_clean":"2024-04-25T00:32:35.433516+0000","last_became_active":"2024-04-25T00:32:33.412675+0000","last_became_peered":"2024-04-25T00:32:33.412675+0000","last_unstale":"2024-04-25T00:32:35.433516+0000","last_undegraded":"2024-04-25T00:32:35.433516+0000","last_fullsized":"2024-04-25T00:32:35.433516+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:24.119844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436830+0000","last_change":"2024-04-25T00:32:34.485250+0000","last_active":"2024-04-25T00:32:35.436830+0000","last_peered":"2024-04-25T00:32:35.436830+0000","last_clean":"2024-04-25T00:32:35.436830+0000","last_became_active":"2024-04-25T00:32:34.484511+0000","last_became_peered":"2024-04-25T00:32:34.484511+0000","last_unstale":"2024-04-25T00:32:35.436830+0000","last_undegraded":"2024-04-25T00:32:35.436830+0000","last_fullsized":"2024-04-25T00:32:35.436830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:04.291110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437468+0000","last_change":"2024-04-25T00:32:33.420075+0000","last_active":"2024-04-25T00:32:36.437468+0000","last_peered":"2024-04-25T00:32:36.437468+0000","last_clean":"2024-04-25T00:32:36.437468+0000","last_became_active":"2024-04-25T00:32:33.419783+0000","last_became_peered":"2024-04-25T00:32:33.419783+0000","last_unstale":"2024-04-25T00:32:36.437468+0000","last_undegraded":"2024-04-25T00:32:36.437468+0000","last_fullsized":"2024-04-25T00:32:36.437468+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:24.119844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437057+0000","last_change":"2024-04-25T00:32:34.456626+0000","last_active":"2024-04-25T00:32:36.437057+0000","last_peered":"2024-04-25T00:32:36.437057+0000","last_clean":"2024-04-25T00:32:36.437057+0000","last_became_active":"2024-04-25T00:32:34.456328+0000","last_became_peered":"2024-04-25T00:32:34.456328+0000","last_unstale":"2024-04-25T00:32:36.437057+0000","last_undegraded":"2024-04-25T00:32:36.437057+0000","last_fullsized":"2024-04-25T00:32:36.437057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:17.530979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.432988+0000","last_change":"2024-04-25T00:32:33.411847+0000","last_active":"2024-04-25T00:32:35.432988+0000","last_peered":"2024-04-25T00:32:35.432988+0000","last_clean":"2024-04-25T00:32:35.432988+0000","last_became_active":"2024-04-25T00:32:33.411429+0000","last_became_peered":"2024-04-25T00:32:33.411429+0000","last_unstale":"2024-04-25T00:32:35.432988+0000","last_undegraded":"2024-04-25T00:32:35.432988+0000","last_fullsized":"2024-04-25T00:32:35.432988+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:07.836095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.049585+0000","last_change":"2024-04-25T00:32:34.465264+0000","last_active":"2024-04-25T00:32:36.049585+0000","last_peered":"2024-04-25T00:32:36.049585+0000","last_clean":"2024-04-25T00:32:36.049585+0000","last_became_active":"2024-04-25T00:32:34.461301+0000","last_became_peered":"2024-04-25T00:32:34.461301+0000","last_unstale":"2024-04-25T00:32:36.049585+0000","last_undegraded":"2024-04-25T00:32:36.049585+0000","last_fullsized":"2024-04-25T00:32:36.049585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:16.435951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437486+0000","last_change":"2024-04-25T00:32:33.417550+0000","last_active":"2024-04-25T00:32:36.437486+0000","last_peered":"2024-04-25T00:32:36.437486+0000","last_clean":"2024-04-25T00:32:36.437486+0000","last_became_active":"2024-04-25T00:32:33.417040+0000","last_became_peered":"2024-04-25T00:32:33.417040+0000","last_unstale":"2024-04-25T00:32:36.437486+0000","last_undegraded":"2024-04-25T00:32:36.437486+0000","last_fullsized":"2024-04-25T00:32:36.437486+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:34.696377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437106+0000","last_change":"2024-04-25T00:32:34.453155+0000","last_active":"2024-04-25T00:32:36.437106+0000","last_peered":"2024-04-25T00:32:36.437106+0000","last_clean":"2024-04-25T00:32:36.437106+0000","last_became_active":"2024-04-25T00:32:34.452888+0000","last_became_peered":"2024-04-25T00:32:34.452888+0000","last_unstale":"2024-04-25T00:32:36.437106+0000","last_undegraded":"2024-04-25T00:32:36.437106+0000","last_fullsized":"2024-04-25T00:32:36.437106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:25.019780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.432193+0000","last_change":"2024-04-25T00:32:33.412644+0000","last_active":"2024-04-25T00:32:35.432193+0000","last_peered":"2024-04-25T00:32:35.432193+0000","last_clean":"2024-04-25T00:32:35.432193+0000","last_became_active":"2024-04-25T00:32:33.412193+0000","last_became_peered":"2024-04-25T00:32:33.412193+0000","last_unstale":"2024-04-25T00:32:35.432193+0000","last_undegraded":"2024-04-25T00:32:35.432193+0000","last_fullsized":"2024-04-25T00:32:35.432193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:34.696377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.523517+0000","last_change":"2024-04-25T00:32:34.445525+0000","last_active":"2024-04-25T00:32:35.523517+0000","last_peered":"2024-04-25T00:32:35.523517+0000","last_clean":"2024-04-25T00:32:35.523517+0000","last_became_active":"2024-04-25T00:32:34.444600+0000","last_became_peered":"2024-04-25T00:32:34.444600+0000","last_unstale":"2024-04-25T00:32:35.523517+0000","last_undegraded":"2024-04-25T00:32:35.523517+0000","last_fullsized":"2024-04-25T00:32:35.523517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:18.311016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435449+0000","last_change":"2024-04-25T00:32:33.416074+0000","last_active":"2024-04-25T00:32:35.435449+0000","last_peered":"2024-04-25T00:32:35.435449+0000","last_clean":"2024-04-25T00:32:35.435449+0000","last_became_active":"2024-04-25T00:32:33.415336+0000","last_became_peered":"2024-04-25T00:32:33.415336+0000","last_unstale":"2024-04-25T00:32:35.435449+0000","last_undegraded":"2024-04-25T00:32:35.435449+0000","last_fullsized":"2024-04-25T00:32:35.435449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:05.605371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435235+0000","last_change":"2024-04-25T00:32:34.448880+0000","last_active":"2024-04-25T00:32:35.435235+0000","last_peered":"2024-04-25T00:32:35.435235+0000","last_clean":"2024-04-25T00:32:35.435235+0000","last_became_active":"2024-04-25T00:32:34.448082+0000","last_became_peered":"2024-04-25T00:32:34.448082+0000","last_unstale":"2024-04-25T00:32:35.435235+0000","last_undegraded":"2024-04-25T00:32:35.435235+0000","last_fullsized":"2024-04-25T00:32:35.435235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:37.803710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.523812+0000","last_change":"2024-04-25T00:32:33.440955+0000","last_active":"2024-04-25T00:32:35.523812+0000","last_peered":"2024-04-25T00:32:35.523812+0000","last_clean":"2024-04-25T00:32:35.523812+0000","last_became_active":"2024-04-25T00:32:33.440603+0000","last_became_peered":"2024-04-25T00:32:33.440603+0000","last_unstale":"2024-04-25T00:32:35.523812+0000","last_undegraded":"2024-04-25T00:32:35.523812+0000","last_fullsized":"2024-04-25T00:32:35.523812+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:36.794509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.548197+0000","last_change":"2024-04-25T00:32:34.488479+0000","last_active":"2024-04-25T00:32:35.548197+0000","last_peered":"2024-04-25T00:32:35.548197+0000","last_clean":"2024-04-25T00:32:35.548197+0000","last_became_active":"2024-04-25T00:32:34.483778+0000","last_became_peered":"2024-04-25T00:32:34.483778+0000","last_unstale":"2024-04-25T00:32:35.548197+0000","last_undegraded":"2024-04-25T00:32:35.548197+0000","last_fullsized":"2024-04-25T00:32:35.548197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:16.527471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.049715+0000","last_change":"2024-04-25T00:32:33.432559+0000","last_active":"2024-04-25T00:32:36.049715+0000","last_peered":"2024-04-25T00:32:36.049715+0000","last_clean":"2024-04-25T00:32:36.049715+0000","last_became_active":"2024-04-25T00:32:33.431829+0000","last_became_peered":"2024-04-25T00:32:33.431829+0000","last_unstale":"2024-04-25T00:32:36.049715+0000","last_undegraded":"2024-04-25T00:32:36.049715+0000","last_fullsized":"2024-04-25T00:32:36.049715+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:10.605816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":194,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547965+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:32:35.547965+0000","last_peered":"2024-04-25T00:32:35.547965+0000","last_clean":"2024-04-25T00:32:35.547965+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:32:35.547965+0000","last_undegraded":"2024-04-25T00:32:35.547965+0000","last_fullsized":"2024-04-25T00:32:35.547965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":737414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":147572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":147628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":147060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":389120,"data_stored":148590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":376832,"data_stored":147026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":389120,"data_stored":148590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":966656,"data_stored":737324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:32:42.954 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:32:43.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c9245640 1 Processor -- start 2024-04-25T00:32:43.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c9245640 1 -- start start 2024-04-25T00:32:43.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c9245640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2c406d320 0x7ff2c40d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c9245640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff2c406c530 0x7ff2c40cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c9245640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c415e7a0 con 0x7ff2c415e3c0 2024-04-25T00:32:43.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c9245640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c40b4bc0 con 0x7ff2c406c530 2024-04-25T00:32:43.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c9245640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c406c9c0 con 0x7ff2c406d320 2024-04-25T00:32:43.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c359e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff2c406c530 0x7ff2c40cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c359e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff2c406c530 0x7ff2c40cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54294/0 (socket says 172.21.15.29:54294) 2024-04-25T00:32:43.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c359e640 1 -- 172.21.15.29:0/79398297 learned_addr learned my addr 172.21.15.29:0/79398297 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:43.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2ba59c640 1 --2- 172.21.15.29:0/79398297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2c406d320 0x7ff2c40d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c359e640 1 -- 172.21.15.29:0/79398297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2c406d320 msgr2=0x7ff2c40d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:43.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c359e640 1 --2- 172.21.15.29:0/79398297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2c406d320 0x7ff2c40d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c359e640 1 -- 172.21.15.29:0/79398297 >> v1:172.21.15.157:6789/0 conn(0x7ff2c415e3c0 legacy=0x7ff2c406cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.087+0000 7ff2c359e640 1 -- 172.21.15.29:0/79398297 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c40d10f0 con 0x7ff2c406c530 2024-04-25T00:32:43.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c359e640 1 --2- 172.21.15.29:0/79398297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2c406c530 0x7ff2c40cd040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc009e50 tx=0x7ff2bc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d166779376d2726b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:43.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c259c640 1 -- 172.21.15.29:0/79398297 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc0a57f0 con 0x7ff2c406c530 2024-04-25T00:32:43.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c259c640 1 -- 172.21.15.29:0/79398297 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2bc0a5990 con 0x7ff2c406c530 2024-04-25T00:32:43.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c259c640 1 -- 172.21.15.29:0/79398297 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc0a5d60 con 0x7ff2c406c530 2024-04-25T00:32:43.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 -- 172.21.15.29:0/79398297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2c406c530 msgr2=0x7ff2c40cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 --2- 172.21.15.29:0/79398297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2c406c530 0x7ff2c40cd040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc009e50 tx=0x7ff2bc01ca30 comp rx=0 tx=0).stop 2024-04-25T00:32:43.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 -- 172.21.15.29:0/79398297 shutdown_connections 2024-04-25T00:32:43.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 --2- 172.21.15.29:0/79398297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2c406c530 0x7ff2c40cd040 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 --2- 172.21.15.29:0/79398297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2c406d320 0x7ff2c40d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 -- 172.21.15.29:0/79398297 >> 172.21.15.29:0/79398297 conn(0x7ff2c40d1d70 msgr2=0x7ff2c40d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:43.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 -- 172.21.15.29:0/79398297 shutdown_connections 2024-04-25T00:32:43.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 -- 172.21.15.29:0/79398297 wait complete. 2024-04-25T00:32:43.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 Processor -- start 2024-04-25T00:32:43.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 -- start start 2024-04-25T00:32:43.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2c406c530 0x7ff2c4152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2c406d320 0x7ff2c4160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2c415e3c0 0x7ff2c4141020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c40cdf10 con 0x7ff2c406c530 2024-04-25T00:32:43.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c40cdd90 con 0x7ff2c406d320 2024-04-25T00:32:43.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c9245640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c40ce090 con 0x7ff2c415e3c0 2024-04-25T00:32:43.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2c406c530 0x7ff2c4152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2c406c530 0x7ff2c4152960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54310/0 (socket says 172.21.15.29:54310) 2024-04-25T00:32:43.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c2d9d640 1 -- 172.21.15.29:0/2690346067 learned_addr learned my addr 172.21.15.29:0/2690346067 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:43.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c359e640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2c415e3c0 0x7ff2c4141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2ba59c640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2c406d320 0x7ff2c4160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c2d9d640 1 -- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2c415e3c0 msgr2=0x7ff2c4141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c2d9d640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2c415e3c0 0x7ff2c4141020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c2d9d640 1 -- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2c406d320 msgr2=0x7ff2c4160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c2d9d640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2c406d320 0x7ff2c4160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c2d9d640 1 -- 172.21.15.29:0/2690346067 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c4152ea0 con 0x7ff2c406c530 2024-04-25T00:32:43.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2c2d9d640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2c406c530 0x7ff2c4152960 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400c420 tx=0x7ff2b400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:43.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.091+0000 7ff2bbfff640 1 -- 172.21.15.29:0/2690346067 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40052d0 con 0x7ff2c406c530 2024-04-25T00:32:43.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.095+0000 7ff2bbfff640 1 -- 172.21.15.29:0/2690346067 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b4005470 con 0x7ff2c406c530 2024-04-25T00:32:43.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.095+0000 7ff2c9245640 1 -- 172.21.15.29:0/2690346067 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c41610d0 con 0x7ff2c406c530 2024-04-25T00:32:43.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.095+0000 7ff2c9245640 1 -- 172.21.15.29:0/2690346067 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c4161680 con 0x7ff2c406c530 2024-04-25T00:32:43.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.095+0000 7ff2bbfff640 1 -- 172.21.15.29:0/2690346067 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4007260 con 0x7ff2c406c530 2024-04-25T00:32:43.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.095+0000 7ff2c9245640 1 -- 172.21.15.29:0/2690346067 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a80052a0 con 0x7ff2c406c530 2024-04-25T00:32:43.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.095+0000 7ff2bbfff640 1 -- 172.21.15.29:0/2690346067 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff2b4007400 con 0x7ff2c406c530 2024-04-25T00:32:43.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.095+0000 7ff2bbfff640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff2a40678f0 0x7ff2a4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.095+0000 7ff2bbfff640 1 -- 172.21.15.29:0/2690346067 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff2b4085050 con 0x7ff2c406c530 2024-04-25T00:32:43.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.095+0000 7ff2ba59c640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff2a40678f0 0x7ff2a4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.095+0000 7ff2ba59c640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff2a40678f0 0x7ff2a4069db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4073350 tx=0x7ff2ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:43.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.099+0000 7ff2bbfff640 1 -- 172.21.15.29:0/2690346067 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff2c4152ea0 con 0x7ff2c406c530 2024-04-25T00:32:43.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.255+0000 7ff2c9245640 1 -- 172.21.15.29:0/2690346067 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2a8000bd0 con 0x7ff2a40678f0 2024-04-25T00:32:43.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.275+0000 7ff2bbfff640 1 -- 172.21.15.29:0/2690346067 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7ff2a8000bd0 con 0x7ff2a40678f0 2024-04-25T00:32:43.281 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:32:43.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.279+0000 7ff2c9245640 1 -- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff2a40678f0 msgr2=0x7ff2a4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.279+0000 7ff2c9245640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff2a40678f0 0x7ff2a4069db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4073350 tx=0x7ff2ac008040 comp rx=0 tx=0).stop 2024-04-25T00:32:43.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.279+0000 7ff2c9245640 1 -- 172.21.15.29:0/2690346067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2c406c530 msgr2=0x7ff2c4152960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.279+0000 7ff2c9245640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2c406c530 0x7ff2c4152960 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400c420 tx=0x7ff2b400c8e0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.279+0000 7ff2c9245640 1 -- 172.21.15.29:0/2690346067 shutdown_connections 2024-04-25T00:32:43.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.279+0000 7ff2c9245640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2c415e3c0 0x7ff2c4141020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.279+0000 7ff2c9245640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff2a40678f0 0x7ff2a4069db0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.279+0000 7ff2c9245640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2c406d320 0x7ff2c4160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.279+0000 7ff2c9245640 1 --2- 172.21.15.29:0/2690346067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2c406c530 0x7ff2c4152960 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.279+0000 7ff2c9245640 1 -- 172.21.15.29:0/2690346067 >> 172.21.15.29:0/2690346067 conn(0x7ff2c40d1d70 msgr2=0x7ff2c406ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.279+0000 7ff2c9245640 1 -- 172.21.15.29:0/2690346067 shutdown_connections 2024-04-25T00:32:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.279+0000 7ff2c9245640 1 -- 172.21.15.29:0/2690346067 wait complete. 2024-04-25T00:32:43.298 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-04-25T00:32:41.468702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253608,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721112,"statfs":{"total":773094113280,"available":772834418688,"internally_reserved":0,"allocated":4235264,"data_stored":2361204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003002"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.049924+0000","last_change":"2024-04-25T00:32:34.486883+0000","last_active":"2024-04-25T00:32:36.049924+0000","last_peered":"2024-04-25T00:32:36.049924+0000","last_clean":"2024-04-25T00:32:36.049924+0000","last_became_active":"2024-04-25T00:32:34.486381+0000","last_became_peered":"2024-04-25T00:32:34.486381+0000","last_unstale":"2024-04-25T00:32:36.049924+0000","last_undegraded":"2024-04-25T00:32:36.049924+0000","last_fullsized":"2024-04-25T00:32:36.049924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:17.530979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524331+0000","last_change":"2024-04-25T00:32:34.452939+0000","last_active":"2024-04-25T00:32:35.524331+0000","last_peered":"2024-04-25T00:32:35.524331+0000","last_clean":"2024-04-25T00:32:35.524331+0000","last_became_active":"2024-04-25T00:32:34.447660+0000","last_became_peered":"2024-04-25T00:32:34.447660+0000","last_unstale":"2024-04-25T00:32:35.524331+0000","last_undegraded":"2024-04-25T00:32:35.524331+0000","last_fullsized":"2024-04-25T00:32:35.524331+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:04.291110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436237+0000","last_change":"2024-04-25T00:32:34.476785+0000","last_active":"2024-04-25T00:32:35.436237+0000","last_peered":"2024-04-25T00:32:35.436237+0000","last_clean":"2024-04-25T00:32:35.436237+0000","last_became_active":"2024-04-25T00:32:34.475274+0000","last_became_peered":"2024-04-25T00:32:34.475274+0000","last_unstale":"2024-04-25T00:32:35.436237+0000","last_undegraded":"2024-04-25T00:32:35.436237+0000","last_fullsized":"2024-04-25T00:32:35.436237+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:12.912933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.432748+0000","last_change":"2024-04-25T00:32:34.461013+0000","last_active":"2024-04-25T00:32:35.432748+0000","last_peered":"2024-04-25T00:32:35.432748+0000","last_clean":"2024-04-25T00:32:35.432748+0000","last_became_active":"2024-04-25T00:32:34.460700+0000","last_became_peered":"2024-04-25T00:32:34.460700+0000","last_unstale":"2024-04-25T00:32:35.432748+0000","last_undegraded":"2024-04-25T00:32:35.432748+0000","last_fullsized":"2024-04-25T00:32:35.432748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:47.539395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658773+0000","last_change":"2024-04-25T00:32:34.446249+0000","last_active":"2024-04-25T00:32:36.658773+0000","last_peered":"2024-04-25T00:32:36.658773+0000","last_clean":"2024-04-25T00:32:36.658773+0000","last_became_active":"2024-04-25T00:32:34.445950+0000","last_became_peered":"2024-04-25T00:32:34.445950+0000","last_unstale":"2024-04-25T00:32:36.658773+0000","last_undegraded":"2024-04-25T00:32:36.658773+0000","last_fullsized":"2024-04-25T00:32:36.658773+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:34.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.523749+0000","last_change":"2024-04-25T00:32:34.473559+0000","last_active":"2024-04-25T00:32:35.523749+0000","last_peered":"2024-04-25T00:32:35.523749+0000","last_clean":"2024-04-25T00:32:35.523749+0000","last_became_active":"2024-04-25T00:32:34.473242+0000","last_became_peered":"2024-04-25T00:32:34.473242+0000","last_unstale":"2024-04-25T00:32:35.523749+0000","last_undegraded":"2024-04-25T00:32:35.523749+0000","last_fullsized":"2024-04-25T00:32:35.523749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:12.912933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547759+0000","last_change":"2024-04-25T00:32:34.487797+0000","last_active":"2024-04-25T00:32:35.547759+0000","last_peered":"2024-04-25T00:32:35.547759+0000","last_clean":"2024-04-25T00:32:35.547759+0000","last_became_active":"2024-04-25T00:32:34.487034+0000","last_became_peered":"2024-04-25T00:32:34.487034+0000","last_unstale":"2024-04-25T00:32:35.547759+0000","last_undegraded":"2024-04-25T00:32:35.547759+0000","last_fullsized":"2024-04-25T00:32:35.547759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:14.211499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547296+0000","last_change":"2024-04-25T00:32:34.481305+0000","last_active":"2024-04-25T00:32:35.547296+0000","last_peered":"2024-04-25T00:32:35.547296+0000","last_clean":"2024-04-25T00:32:35.547296+0000","last_became_active":"2024-04-25T00:32:34.480558+0000","last_became_peered":"2024-04-25T00:32:34.480558+0000","last_unstale":"2024-04-25T00:32:35.547296+0000","last_undegraded":"2024-04-25T00:32:35.547296+0000","last_fullsized":"2024-04-25T00:32:35.547296+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:18.311016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436415+0000","last_change":"2024-04-25T00:32:34.461475+0000","last_active":"2024-04-25T00:32:35.436415+0000","last_peered":"2024-04-25T00:32:35.436415+0000","last_clean":"2024-04-25T00:32:35.436415+0000","last_became_active":"2024-04-25T00:32:34.460713+0000","last_became_peered":"2024-04-25T00:32:34.460713+0000","last_unstale":"2024-04-25T00:32:35.436415+0000","last_undegraded":"2024-04-25T00:32:35.436415+0000","last_fullsized":"2024-04-25T00:32:35.436415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:38.765850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433632+0000","last_change":"2024-04-25T00:32:34.453887+0000","last_active":"2024-04-25T00:32:35.433632+0000","last_peered":"2024-04-25T00:32:35.433632+0000","last_clean":"2024-04-25T00:32:35.433632+0000","last_became_active":"2024-04-25T00:32:34.450130+0000","last_became_peered":"2024-04-25T00:32:34.450130+0000","last_unstale":"2024-04-25T00:32:35.433632+0000","last_undegraded":"2024-04-25T00:32:35.433632+0000","last_fullsized":"2024-04-25T00:32:35.433632+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:09.955792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.548011+0000","last_change":"2024-04-25T00:32:34.483296+0000","last_active":"2024-04-25T00:32:35.548011+0000","last_peered":"2024-04-25T00:32:35.548011+0000","last_clean":"2024-04-25T00:32:35.548011+0000","last_became_active":"2024-04-25T00:32:34.481034+0000","last_became_peered":"2024-04-25T00:32:34.481034+0000","last_unstale":"2024-04-25T00:32:35.548011+0000","last_undegraded":"2024-04-25T00:32:35.548011+0000","last_fullsized":"2024-04-25T00:32:35.548011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:33.114509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434876+0000","last_change":"2024-04-25T00:32:34.475382+0000","last_active":"2024-04-25T00:32:35.434876+0000","last_peered":"2024-04-25T00:32:35.434876+0000","last_clean":"2024-04-25T00:32:35.434876+0000","last_became_active":"2024-04-25T00:32:34.474197+0000","last_became_peered":"2024-04-25T00:32:34.474197+0000","last_unstale":"2024-04-25T00:32:35.434876+0000","last_undegraded":"2024-04-25T00:32:35.434876+0000","last_fullsized":"2024-04-25T00:32:35.434876+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:48.062329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658880+0000","last_change":"2024-04-25T00:32:34.439134+0000","last_active":"2024-04-25T00:32:36.658880+0000","last_peered":"2024-04-25T00:32:36.658880+0000","last_clean":"2024-04-25T00:32:36.658880+0000","last_became_active":"2024-04-25T00:32:34.438834+0000","last_became_peered":"2024-04-25T00:32:34.438834+0000","last_unstale":"2024-04-25T00:32:36.658880+0000","last_undegraded":"2024-04-25T00:32:36.658880+0000","last_fullsized":"2024-04-25T00:32:36.658880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:35.705579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524075+0000","last_change":"2024-04-25T00:32:34.448199+0000","last_active":"2024-04-25T00:32:35.524075+0000","last_peered":"2024-04-25T00:32:35.524075+0000","last_clean":"2024-04-25T00:32:35.524075+0000","last_became_active":"2024-04-25T00:32:34.441867+0000","last_became_peered":"2024-04-25T00:32:34.441867+0000","last_unstale":"2024-04-25T00:32:35.524075+0000","last_undegraded":"2024-04-25T00:32:35.524075+0000","last_fullsized":"2024-04-25T00:32:35.524075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:54.889078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434188+0000","last_change":"2024-04-25T00:32:34.476484+0000","last_active":"2024-04-25T00:32:35.434188+0000","last_peered":"2024-04-25T00:32:35.434188+0000","last_clean":"2024-04-25T00:32:35.434188+0000","last_became_active":"2024-04-25T00:32:34.475640+0000","last_became_peered":"2024-04-25T00:32:34.475640+0000","last_unstale":"2024-04-25T00:32:35.434188+0000","last_undegraded":"2024-04-25T00:32:35.434188+0000","last_fullsized":"2024-04-25T00:32:35.434188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:02.419942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.548042+0000","last_change":"2024-04-25T00:32:34.491665+0000","last_active":"2024-04-25T00:32:35.548042+0000","last_peered":"2024-04-25T00:32:35.548042+0000","last_clean":"2024-04-25T00:32:35.548042+0000","last_became_active":"2024-04-25T00:32:34.489466+0000","last_became_peered":"2024-04-25T00:32:34.489466+0000","last_unstale":"2024-04-25T00:32:35.548042+0000","last_undegraded":"2024-04-25T00:32:35.548042+0000","last_fullsized":"2024-04-25T00:32:35.548042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:00.562101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.548631+0000","last_change":"2024-04-25T00:32:34.487734+0000","last_active":"2024-04-25T00:32:35.548631+0000","last_peered":"2024-04-25T00:32:35.548631+0000","last_clean":"2024-04-25T00:32:35.548631+0000","last_became_active":"2024-04-25T00:32:34.487116+0000","last_became_peered":"2024-04-25T00:32:34.487116+0000","last_unstale":"2024-04-25T00:32:35.548631+0000","last_undegraded":"2024-04-25T00:32:35.548631+0000","last_fullsized":"2024-04-25T00:32:35.548631+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:52.301520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524588+0000","last_change":"2024-04-25T00:32:34.447795+0000","last_active":"2024-04-25T00:32:35.524588+0000","last_peered":"2024-04-25T00:32:35.524588+0000","last_clean":"2024-04-25T00:32:35.524588+0000","last_became_active":"2024-04-25T00:32:34.447273+0000","last_became_peered":"2024-04-25T00:32:34.447273+0000","last_unstale":"2024-04-25T00:32:35.524588+0000","last_undegraded":"2024-04-25T00:32:35.524588+0000","last_fullsized":"2024-04-25T00:32:35.524588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:08.570179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524684+0000","last_change":"2024-04-25T00:32:34.445862+0000","last_active":"2024-04-25T00:32:35.524684+0000","last_peered":"2024-04-25T00:32:35.524684+0000","last_clean":"2024-04-25T00:32:35.524684+0000","last_became_active":"2024-04-25T00:32:34.444856+0000","last_became_peered":"2024-04-25T00:32:34.444856+0000","last_unstale":"2024-04-25T00:32:35.524684+0000","last_undegraded":"2024-04-25T00:32:35.524684+0000","last_fullsized":"2024-04-25T00:32:35.524684+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:52.301520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658952+0000","last_change":"2024-04-25T00:32:34.478706+0000","last_active":"2024-04-25T00:32:36.658952+0000","last_peered":"2024-04-25T00:32:36.658952+0000","last_clean":"2024-04-25T00:32:36.658952+0000","last_became_active":"2024-04-25T00:32:34.478390+0000","last_became_peered":"2024-04-25T00:32:34.478390+0000","last_unstale":"2024-04-25T00:32:36.658952+0000","last_undegraded":"2024-04-25T00:32:36.658952+0000","last_fullsized":"2024-04-25T00:32:36.658952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:49.942783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435859+0000","last_change":"2024-04-25T00:32:34.464425+0000","last_active":"2024-04-25T00:32:35.435859+0000","last_peered":"2024-04-25T00:32:35.435859+0000","last_clean":"2024-04-25T00:32:35.435859+0000","last_became_active":"2024-04-25T00:32:34.456321+0000","last_became_peered":"2024-04-25T00:32:34.456321+0000","last_unstale":"2024-04-25T00:32:35.435859+0000","last_undegraded":"2024-04-25T00:32:35.435859+0000","last_fullsized":"2024-04-25T00:32:35.435859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:05.464400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435515+0000","last_change":"2024-04-25T00:32:34.449881+0000","last_active":"2024-04-25T00:32:35.435515+0000","last_peered":"2024-04-25T00:32:35.435515+0000","last_clean":"2024-04-25T00:32:35.435515+0000","last_became_active":"2024-04-25T00:32:34.448725+0000","last_became_peered":"2024-04-25T00:32:34.448725+0000","last_unstale":"2024-04-25T00:32:35.435515+0000","last_undegraded":"2024-04-25T00:32:35.435515+0000","last_fullsized":"2024-04-25T00:32:35.435515+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:17.530979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.436714+0000","last_change":"2024-04-25T00:32:34.467564+0000","last_active":"2024-04-25T00:32:36.436714+0000","last_peered":"2024-04-25T00:32:36.436714+0000","last_clean":"2024-04-25T00:32:36.436714+0000","last_became_active":"2024-04-25T00:32:34.467264+0000","last_became_peered":"2024-04-25T00:32:34.467264+0000","last_unstale":"2024-04-25T00:32:36.436714+0000","last_undegraded":"2024-04-25T00:32:36.436714+0000","last_fullsized":"2024-04-25T00:32:36.436714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:37.803710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433848+0000","last_change":"2024-04-25T00:32:34.474541+0000","last_active":"2024-04-25T00:32:35.433848+0000","last_peered":"2024-04-25T00:32:35.433848+0000","last_clean":"2024-04-25T00:32:35.433848+0000","last_became_active":"2024-04-25T00:32:34.473805+0000","last_became_peered":"2024-04-25T00:32:34.473805+0000","last_unstale":"2024-04-25T00:32:35.433848+0000","last_undegraded":"2024-04-25T00:32:35.433848+0000","last_fullsized":"2024-04-25T00:32:35.433848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:45.660093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433310+0000","last_change":"2024-04-25T00:32:34.469626+0000","last_active":"2024-04-25T00:32:35.433310+0000","last_peered":"2024-04-25T00:32:35.433310+0000","last_clean":"2024-04-25T00:32:35.433310+0000","last_became_active":"2024-04-25T00:32:34.469259+0000","last_became_peered":"2024-04-25T00:32:34.469259+0000","last_unstale":"2024-04-25T00:32:35.433310+0000","last_undegraded":"2024-04-25T00:32:35.433310+0000","last_fullsized":"2024-04-25T00:32:35.433310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:47.677244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524630+0000","last_change":"2024-04-25T00:32:34.469477+0000","last_active":"2024-04-25T00:32:35.524630+0000","last_peered":"2024-04-25T00:32:35.524630+0000","last_clean":"2024-04-25T00:32:35.524630+0000","last_became_active":"2024-04-25T00:32:34.469157+0000","last_became_peered":"2024-04-25T00:32:34.469157+0000","last_unstale":"2024-04-25T00:32:35.524630+0000","last_undegraded":"2024-04-25T00:32:35.524630+0000","last_fullsized":"2024-04-25T00:32:35.524630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:16.527471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433333+0000","last_change":"2024-04-25T00:32:34.461083+0000","last_active":"2024-04-25T00:32:35.433333+0000","last_peered":"2024-04-25T00:32:35.433333+0000","last_clean":"2024-04-25T00:32:35.433333+0000","last_became_active":"2024-04-25T00:32:34.460777+0000","last_became_peered":"2024-04-25T00:32:34.460777+0000","last_unstale":"2024-04-25T00:32:35.433333+0000","last_undegraded":"2024-04-25T00:32:35.433333+0000","last_fullsized":"2024-04-25T00:32:35.433333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:54.672586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434880+0000","last_change":"2024-04-25T00:32:34.474938+0000","last_active":"2024-04-25T00:32:35.434880+0000","last_peered":"2024-04-25T00:32:35.434880+0000","last_clean":"2024-04-25T00:32:35.434880+0000","last_became_active":"2024-04-25T00:32:34.473570+0000","last_became_peered":"2024-04-25T00:32:34.473570+0000","last_unstale":"2024-04-25T00:32:35.434880+0000","last_undegraded":"2024-04-25T00:32:35.434880+0000","last_fullsized":"2024-04-25T00:32:35.434880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:39.687967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435695+0000","last_change":"2024-04-25T00:32:34.457058+0000","last_active":"2024-04-25T00:32:35.435695+0000","last_peered":"2024-04-25T00:32:35.435695+0000","last_clean":"2024-04-25T00:32:35.435695+0000","last_became_active":"2024-04-25T00:32:34.456275+0000","last_became_peered":"2024-04-25T00:32:34.456275+0000","last_unstale":"2024-04-25T00:32:35.435695+0000","last_undegraded":"2024-04-25T00:32:35.435695+0000","last_fullsized":"2024-04-25T00:32:35.435695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:25.019780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547711+0000","last_change":"2024-04-25T00:32:34.487878+0000","last_active":"2024-04-25T00:32:35.547711+0000","last_peered":"2024-04-25T00:32:35.547711+0000","last_clean":"2024-04-25T00:32:35.547711+0000","last_became_active":"2024-04-25T00:32:34.487179+0000","last_became_peered":"2024-04-25T00:32:34.487179+0000","last_unstale":"2024-04-25T00:32:35.547711+0000","last_undegraded":"2024-04-25T00:32:35.547711+0000","last_fullsized":"2024-04-25T00:32:35.547711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:04.291110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.049480+0000","last_change":"2024-04-25T00:32:34.453427+0000","last_active":"2024-04-25T00:32:36.049480+0000","last_peered":"2024-04-25T00:32:36.049480+0000","last_clean":"2024-04-25T00:32:36.049480+0000","last_became_active":"2024-04-25T00:32:34.452661+0000","last_became_peered":"2024-04-25T00:32:34.452661+0000","last_unstale":"2024-04-25T00:32:36.049480+0000","last_undegraded":"2024-04-25T00:32:36.049480+0000","last_fullsized":"2024-04-25T00:32:36.049480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:40.112885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547273+0000","last_change":"2024-04-25T00:32:34.483634+0000","last_active":"2024-04-25T00:32:35.547273+0000","last_peered":"2024-04-25T00:32:35.547273+0000","last_clean":"2024-04-25T00:32:35.547273+0000","last_became_active":"2024-04-25T00:32:34.481912+0000","last_became_peered":"2024-04-25T00:32:34.481912+0000","last_unstale":"2024-04-25T00:32:35.547273+0000","last_undegraded":"2024-04-25T00:32:35.547273+0000","last_fullsized":"2024-04-25T00:32:35.547273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:48.062329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437295+0000","last_change":"2024-04-25T00:32:34.481329+0000","last_active":"2024-04-25T00:32:36.437295+0000","last_peered":"2024-04-25T00:32:36.437295+0000","last_clean":"2024-04-25T00:32:36.437295+0000","last_became_active":"2024-04-25T00:32:34.480962+0000","last_became_peered":"2024-04-25T00:32:34.480962+0000","last_unstale":"2024-04-25T00:32:36.437295+0000","last_undegraded":"2024-04-25T00:32:36.437295+0000","last_fullsized":"2024-04-25T00:32:36.437295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:05.464400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.050080+0000","last_change":"2024-04-25T00:32:33.439097+0000","last_active":"2024-04-25T00:32:36.050080+0000","last_peered":"2024-04-25T00:32:36.050080+0000","last_clean":"2024-04-25T00:32:36.050080+0000","last_became_active":"2024-04-25T00:32:33.438390+0000","last_became_peered":"2024-04-25T00:32:33.438390+0000","last_unstale":"2024-04-25T00:32:36.050080+0000","last_undegraded":"2024-04-25T00:32:36.050080+0000","last_fullsized":"2024-04-25T00:32:36.050080+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.651396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.050023+0000","last_change":"2024-04-25T00:32:34.477891+0000","last_active":"2024-04-25T00:32:36.050023+0000","last_peered":"2024-04-25T00:32:36.050023+0000","last_clean":"2024-04-25T00:32:36.050023+0000","last_became_active":"2024-04-25T00:32:34.477107+0000","last_became_peered":"2024-04-25T00:32:34.477107+0000","last_unstale":"2024-04-25T00:32:36.050023+0000","last_undegraded":"2024-04-25T00:32:36.050023+0000","last_fullsized":"2024-04-25T00:32:36.050023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:25.019780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437181+0000","last_change":"2024-04-25T00:32:33.439749+0000","last_active":"2024-04-25T00:32:36.437181+0000","last_peered":"2024-04-25T00:32:36.437181+0000","last_clean":"2024-04-25T00:32:36.437181+0000","last_became_active":"2024-04-25T00:32:33.439548+0000","last_became_peered":"2024-04-25T00:32:33.439548+0000","last_unstale":"2024-04-25T00:32:36.437181+0000","last_undegraded":"2024-04-25T00:32:36.437181+0000","last_fullsized":"2024-04-25T00:32:36.437181+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:07.836095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436388+0000","last_change":"2024-04-25T00:32:34.449587+0000","last_active":"2024-04-25T00:32:35.436388+0000","last_peered":"2024-04-25T00:32:35.436388+0000","last_clean":"2024-04-25T00:32:35.436388+0000","last_became_active":"2024-04-25T00:32:34.448891+0000","last_became_peered":"2024-04-25T00:32:34.448891+0000","last_unstale":"2024-04-25T00:32:35.436388+0000","last_undegraded":"2024-04-25T00:32:35.436388+0000","last_fullsized":"2024-04-25T00:32:35.436388+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:47.539395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524834+0000","last_change":"2024-04-25T00:32:33.419772+0000","last_active":"2024-04-25T00:32:35.524834+0000","last_peered":"2024-04-25T00:32:35.524834+0000","last_clean":"2024-04-25T00:32:35.524834+0000","last_became_active":"2024-04-25T00:32:33.419459+0000","last_became_peered":"2024-04-25T00:32:33.419459+0000","last_unstale":"2024-04-25T00:32:35.524834+0000","last_undegraded":"2024-04-25T00:32:35.524834+0000","last_fullsized":"2024-04-25T00:32:35.524834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:24.010578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433709+0000","last_change":"2024-04-25T00:32:34.460897+0000","last_active":"2024-04-25T00:32:35.433709+0000","last_peered":"2024-04-25T00:32:35.433709+0000","last_clean":"2024-04-25T00:32:35.433709+0000","last_became_active":"2024-04-25T00:32:34.460596+0000","last_became_peered":"2024-04-25T00:32:34.460596+0000","last_unstale":"2024-04-25T00:32:35.433709+0000","last_undegraded":"2024-04-25T00:32:35.433709+0000","last_fullsized":"2024-04-25T00:32:35.433709+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:45.660093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658987+0000","last_change":"2024-04-25T00:32:33.419126+0000","last_active":"2024-04-25T00:32:36.658987+0000","last_peered":"2024-04-25T00:32:36.658987+0000","last_clean":"2024-04-25T00:32:36.658987+0000","last_became_active":"2024-04-25T00:32:33.418834+0000","last_became_peered":"2024-04-25T00:32:33.418834+0000","last_unstale":"2024-04-25T00:32:36.658987+0000","last_undegraded":"2024-04-25T00:32:36.658987+0000","last_fullsized":"2024-04-25T00:32:36.658987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:30.363273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.548312+0000","last_change":"2024-04-25T00:32:34.468459+0000","last_active":"2024-04-25T00:32:35.548312+0000","last_peered":"2024-04-25T00:32:35.548312+0000","last_clean":"2024-04-25T00:32:35.548312+0000","last_became_active":"2024-04-25T00:32:34.466447+0000","last_became_peered":"2024-04-25T00:32:34.466447+0000","last_unstale":"2024-04-25T00:32:35.548312+0000","last_undegraded":"2024-04-25T00:32:35.548312+0000","last_fullsized":"2024-04-25T00:32:35.548312+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:08.570179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435957+0000","last_change":"2024-04-25T00:32:33.429088+0000","last_active":"2024-04-25T00:32:35.435957+0000","last_peered":"2024-04-25T00:32:35.435957+0000","last_clean":"2024-04-25T00:32:35.435957+0000","last_became_active":"2024-04-25T00:32:33.427379+0000","last_became_peered":"2024-04-25T00:32:33.427379+0000","last_unstale":"2024-04-25T00:32:35.435957+0000","last_undegraded":"2024-04-25T00:32:35.435957+0000","last_fullsized":"2024-04-25T00:32:35.435957+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:10.605816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434676+0000","last_change":"2024-04-25T00:32:34.457417+0000","last_active":"2024-04-25T00:32:35.434676+0000","last_peered":"2024-04-25T00:32:35.434676+0000","last_clean":"2024-04-25T00:32:35.434676+0000","last_became_active":"2024-04-25T00:32:34.456659+0000","last_became_peered":"2024-04-25T00:32:34.456659+0000","last_unstale":"2024-04-25T00:32:35.434676+0000","last_undegraded":"2024-04-25T00:32:35.434676+0000","last_fullsized":"2024-04-25T00:32:35.434676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.038817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435766+0000","last_change":"2024-04-25T00:32:33.429965+0000","last_active":"2024-04-25T00:32:35.435766+0000","last_peered":"2024-04-25T00:32:35.435766+0000","last_clean":"2024-04-25T00:32:35.435766+0000","last_became_active":"2024-04-25T00:32:33.427356+0000","last_became_peered":"2024-04-25T00:32:33.427356+0000","last_unstale":"2024-04-25T00:32:35.435766+0000","last_undegraded":"2024-04-25T00:32:35.435766+0000","last_fullsized":"2024-04-25T00:32:35.435766+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:33.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434419+0000","last_change":"2024-04-25T00:32:34.454184+0000","last_active":"2024-04-25T00:32:35.434419+0000","last_peered":"2024-04-25T00:32:35.434419+0000","last_clean":"2024-04-25T00:32:35.434419+0000","last_became_active":"2024-04-25T00:32:34.453126+0000","last_became_peered":"2024-04-25T00:32:34.453126+0000","last_unstale":"2024-04-25T00:32:35.434419+0000","last_undegraded":"2024-04-25T00:32:35.434419+0000","last_fullsized":"2024-04-25T00:32:35.434419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:16.435951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.523449+0000","last_change":"2024-04-25T00:32:33.419720+0000","last_active":"2024-04-25T00:32:35.523449+0000","last_peered":"2024-04-25T00:32:35.523449+0000","last_clean":"2024-04-25T00:32:35.523449+0000","last_became_active":"2024-04-25T00:32:33.419338+0000","last_became_peered":"2024-04-25T00:32:33.419338+0000","last_unstale":"2024-04-25T00:32:35.523449+0000","last_undegraded":"2024-04-25T00:32:35.523449+0000","last_fullsized":"2024-04-25T00:32:35.523449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:15.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.432209+0000","last_change":"2024-04-25T00:32:34.473522+0000","last_active":"2024-04-25T00:32:35.432209+0000","last_peered":"2024-04-25T00:32:35.432209+0000","last_clean":"2024-04-25T00:32:35.432209+0000","last_became_active":"2024-04-25T00:32:34.473212+0000","last_became_peered":"2024-04-25T00:32:34.473212+0000","last_unstale":"2024-04-25T00:32:35.432209+0000","last_undegraded":"2024-04-25T00:32:35.432209+0000","last_fullsized":"2024-04-25T00:32:35.432209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:05.464400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434873+0000","last_change":"2024-04-25T00:32:33.433081+0000","last_active":"2024-04-25T00:32:35.434873+0000","last_peered":"2024-04-25T00:32:35.434873+0000","last_clean":"2024-04-25T00:32:35.434873+0000","last_became_active":"2024-04-25T00:32:33.432236+0000","last_became_peered":"2024-04-25T00:32:33.432236+0000","last_unstale":"2024-04-25T00:32:35.434873+0000","last_undegraded":"2024-04-25T00:32:35.434873+0000","last_fullsized":"2024-04-25T00:32:35.434873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:25.652624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436121+0000","last_change":"2024-04-25T00:32:34.478755+0000","last_active":"2024-04-25T00:32:35.436121+0000","last_peered":"2024-04-25T00:32:35.436121+0000","last_clean":"2024-04-25T00:32:35.436121+0000","last_became_active":"2024-04-25T00:32:34.478125+0000","last_became_peered":"2024-04-25T00:32:34.478125+0000","last_unstale":"2024-04-25T00:32:35.436121+0000","last_undegraded":"2024-04-25T00:32:35.436121+0000","last_fullsized":"2024-04-25T00:32:35.436121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:18.311016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524685+0000","last_change":"2024-04-25T00:32:33.431456+0000","last_active":"2024-04-25T00:32:35.524685+0000","last_peered":"2024-04-25T00:32:35.524685+0000","last_clean":"2024-04-25T00:32:35.524685+0000","last_became_active":"2024-04-25T00:32:33.431235+0000","last_became_peered":"2024-04-25T00:32:33.431235+0000","last_unstale":"2024-04-25T00:32:35.524685+0000","last_undegraded":"2024-04-25T00:32:35.524685+0000","last_fullsized":"2024-04-25T00:32:35.524685+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:04.455198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524303+0000","last_change":"2024-04-25T00:32:34.445434+0000","last_active":"2024-04-25T00:32:35.524303+0000","last_peered":"2024-04-25T00:32:35.524303+0000","last_clean":"2024-04-25T00:32:35.524303+0000","last_became_active":"2024-04-25T00:32:34.445151+0000","last_became_peered":"2024-04-25T00:32:34.445151+0000","last_unstale":"2024-04-25T00:32:35.524303+0000","last_undegraded":"2024-04-25T00:32:35.524303+0000","last_fullsized":"2024-04-25T00:32:35.524303+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:33.114509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437574+0000","last_change":"2024-04-25T00:32:33.416642+0000","last_active":"2024-04-25T00:32:36.437574+0000","last_peered":"2024-04-25T00:32:36.437574+0000","last_clean":"2024-04-25T00:32:36.437574+0000","last_became_active":"2024-04-25T00:32:33.416316+0000","last_became_peered":"2024-04-25T00:32:33.416316+0000","last_unstale":"2024-04-25T00:32:36.437574+0000","last_undegraded":"2024-04-25T00:32:36.437574+0000","last_fullsized":"2024-04-25T00:32:36.437574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:48.933581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524467+0000","last_change":"2024-04-25T00:32:34.468094+0000","last_active":"2024-04-25T00:32:35.524467+0000","last_peered":"2024-04-25T00:32:35.524467+0000","last_clean":"2024-04-25T00:32:35.524467+0000","last_became_active":"2024-04-25T00:32:34.467807+0000","last_became_peered":"2024-04-25T00:32:34.467807+0000","last_unstale":"2024-04-25T00:32:35.524467+0000","last_undegraded":"2024-04-25T00:32:35.524467+0000","last_fullsized":"2024-04-25T00:32:35.524467+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:00.562101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436467+0000","last_change":"2024-04-25T00:32:33.424885+0000","last_active":"2024-04-25T00:32:35.436467+0000","last_peered":"2024-04-25T00:32:35.436467+0000","last_clean":"2024-04-25T00:32:35.436467+0000","last_became_active":"2024-04-25T00:32:33.424174+0000","last_became_peered":"2024-04-25T00:32:33.424174+0000","last_unstale":"2024-04-25T00:32:35.436467+0000","last_undegraded":"2024-04-25T00:32:35.436467+0000","last_fullsized":"2024-04-25T00:32:35.436467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:15.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436953+0000","last_change":"2024-04-25T00:32:34.477357+0000","last_active":"2024-04-25T00:32:35.436953+0000","last_peered":"2024-04-25T00:32:35.436953+0000","last_clean":"2024-04-25T00:32:35.436953+0000","last_became_active":"2024-04-25T00:32:34.476024+0000","last_became_peered":"2024-04-25T00:32:34.476024+0000","last_unstale":"2024-04-25T00:32:35.436953+0000","last_undegraded":"2024-04-25T00:32:35.436953+0000","last_fullsized":"2024-04-25T00:32:35.436953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:14.211499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524398+0000","last_change":"2024-04-25T00:32:33.420265+0000","last_active":"2024-04-25T00:32:35.524398+0000","last_peered":"2024-04-25T00:32:35.524398+0000","last_clean":"2024-04-25T00:32:35.524398+0000","last_became_active":"2024-04-25T00:32:33.419977+0000","last_became_peered":"2024-04-25T00:32:33.419977+0000","last_unstale":"2024-04-25T00:32:35.524398+0000","last_undegraded":"2024-04-25T00:32:35.524398+0000","last_fullsized":"2024-04-25T00:32:35.524398+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:16.521777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.432381+0000","last_change":"2024-04-25T00:32:34.473507+0000","last_active":"2024-04-25T00:32:35.432381+0000","last_peered":"2024-04-25T00:32:35.432381+0000","last_clean":"2024-04-25T00:32:35.432381+0000","last_became_active":"2024-04-25T00:32:34.473224+0000","last_became_peered":"2024-04-25T00:32:34.473224+0000","last_unstale":"2024-04-25T00:32:35.432381+0000","last_undegraded":"2024-04-25T00:32:35.432381+0000","last_fullsized":"2024-04-25T00:32:35.432381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:04.587403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436609+0000","last_change":"2024-04-25T00:32:33.434692+0000","last_active":"2024-04-25T00:32:35.436609+0000","last_peered":"2024-04-25T00:32:35.436609+0000","last_clean":"2024-04-25T00:32:35.436609+0000","last_became_active":"2024-04-25T00:32:33.433973+0000","last_became_peered":"2024-04-25T00:32:33.433973+0000","last_unstale":"2024-04-25T00:32:35.436609+0000","last_undegraded":"2024-04-25T00:32:35.436609+0000","last_fullsized":"2024-04-25T00:32:35.436609+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.651396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.524304+0000","last_change":"2024-04-25T00:32:34.447270+0000","last_active":"2024-04-25T00:32:35.524304+0000","last_peered":"2024-04-25T00:32:35.524304+0000","last_clean":"2024-04-25T00:32:35.524304+0000","last_became_active":"2024-04-25T00:32:34.446955+0000","last_became_peered":"2024-04-25T00:32:34.446955+0000","last_unstale":"2024-04-25T00:32:35.524304+0000","last_undegraded":"2024-04-25T00:32:35.524304+0000","last_fullsized":"2024-04-25T00:32:35.524304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:14.211499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.432017+0000","last_change":"2024-04-25T00:32:33.432224+0000","last_active":"2024-04-25T00:32:35.432017+0000","last_peered":"2024-04-25T00:32:35.432017+0000","last_clean":"2024-04-25T00:32:35.432017+0000","last_became_active":"2024-04-25T00:32:33.431863+0000","last_became_peered":"2024-04-25T00:32:33.431863+0000","last_unstale":"2024-04-25T00:32:35.432017+0000","last_undegraded":"2024-04-25T00:32:35.432017+0000","last_fullsized":"2024-04-25T00:32:35.432017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.651396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547641+0000","last_change":"2024-04-25T00:32:34.488122+0000","last_active":"2024-04-25T00:32:35.547641+0000","last_peered":"2024-04-25T00:32:35.547641+0000","last_clean":"2024-04-25T00:32:35.547641+0000","last_became_active":"2024-04-25T00:32:34.482551+0000","last_became_peered":"2024-04-25T00:32:34.482551+0000","last_unstale":"2024-04-25T00:32:35.547641+0000","last_undegraded":"2024-04-25T00:32:35.547641+0000","last_fullsized":"2024-04-25T00:32:35.547641+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:12.912933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.523649+0000","last_change":"2024-04-25T00:32:33.431495+0000","last_active":"2024-04-25T00:32:35.523649+0000","last_peered":"2024-04-25T00:32:35.523649+0000","last_clean":"2024-04-25T00:32:35.523649+0000","last_became_active":"2024-04-25T00:32:33.431231+0000","last_became_peered":"2024-04-25T00:32:33.431231+0000","last_unstale":"2024-04-25T00:32:35.523649+0000","last_undegraded":"2024-04-25T00:32:35.523649+0000","last_fullsized":"2024-04-25T00:32:35.523649+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:15.602236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.049859+0000","last_change":"2024-04-25T00:32:34.473569+0000","last_active":"2024-04-25T00:32:36.049859+0000","last_peered":"2024-04-25T00:32:36.049859+0000","last_clean":"2024-04-25T00:32:36.049859+0000","last_became_active":"2024-04-25T00:32:34.472936+0000","last_became_peered":"2024-04-25T00:32:34.472936+0000","last_unstale":"2024-04-25T00:32:36.049859+0000","last_undegraded":"2024-04-25T00:32:36.049859+0000","last_fullsized":"2024-04-25T00:32:36.049859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:37.803710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.523514+0000","last_change":"2024-04-25T00:32:33.419970+0000","last_active":"2024-04-25T00:32:35.523514+0000","last_peered":"2024-04-25T00:32:35.523514+0000","last_clean":"2024-04-25T00:32:35.523514+0000","last_became_active":"2024-04-25T00:32:33.419465+0000","last_became_peered":"2024-04-25T00:32:33.419465+0000","last_unstale":"2024-04-25T00:32:35.523514+0000","last_undegraded":"2024-04-25T00:32:35.523514+0000","last_fullsized":"2024-04-25T00:32:35.523514+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:39.103683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436268+0000","last_change":"2024-04-25T00:32:34.484992+0000","last_active":"2024-04-25T00:32:35.436268+0000","last_peered":"2024-04-25T00:32:35.436268+0000","last_clean":"2024-04-25T00:32:35.436268+0000","last_became_active":"2024-04-25T00:32:34.484250+0000","last_became_peered":"2024-04-25T00:32:34.484250+0000","last_unstale":"2024-04-25T00:32:35.436268+0000","last_undegraded":"2024-04-25T00:32:35.436268+0000","last_fullsized":"2024-04-25T00:32:35.436268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:04.587403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433441+0000","last_change":"2024-04-25T00:32:33.418814+0000","last_active":"2024-04-25T00:32:35.433441+0000","last_peered":"2024-04-25T00:32:35.433441+0000","last_clean":"2024-04-25T00:32:35.433441+0000","last_became_active":"2024-04-25T00:32:33.418270+0000","last_became_peered":"2024-04-25T00:32:33.418270+0000","last_unstale":"2024-04-25T00:32:35.433441+0000","last_undegraded":"2024-04-25T00:32:35.433441+0000","last_fullsized":"2024-04-25T00:32:35.433441+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:48.933581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436577+0000","last_change":"2024-04-25T00:32:34.485344+0000","last_active":"2024-04-25T00:32:35.436577+0000","last_peered":"2024-04-25T00:32:35.436577+0000","last_clean":"2024-04-25T00:32:35.436577+0000","last_became_active":"2024-04-25T00:32:34.484661+0000","last_became_peered":"2024-04-25T00:32:34.484661+0000","last_unstale":"2024-04-25T00:32:35.436577+0000","last_undegraded":"2024-04-25T00:32:35.436577+0000","last_fullsized":"2024-04-25T00:32:35.436577+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:54.889078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436426+0000","last_change":"2024-04-25T00:32:33.434714+0000","last_active":"2024-04-25T00:32:35.436426+0000","last_peered":"2024-04-25T00:32:35.436426+0000","last_clean":"2024-04-25T00:32:35.436426+0000","last_became_active":"2024-04-25T00:32:33.433950+0000","last_became_peered":"2024-04-25T00:32:33.433950+0000","last_unstale":"2024-04-25T00:32:35.436426+0000","last_undegraded":"2024-04-25T00:32:35.436426+0000","last_fullsized":"2024-04-25T00:32:35.436426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:18.360499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433004+0000","last_change":"2024-04-25T00:32:34.455386+0000","last_active":"2024-04-25T00:32:35.433004+0000","last_peered":"2024-04-25T00:32:35.433004+0000","last_clean":"2024-04-25T00:32:35.433004+0000","last_became_active":"2024-04-25T00:32:34.454053+0000","last_became_peered":"2024-04-25T00:32:34.454053+0000","last_unstale":"2024-04-25T00:32:35.433004+0000","last_undegraded":"2024-04-25T00:32:35.433004+0000","last_fullsized":"2024-04-25T00:32:35.433004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:38.765850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436715+0000","last_change":"2024-04-25T00:32:33.412615+0000","last_active":"2024-04-25T00:32:35.436715+0000","last_peered":"2024-04-25T00:32:35.436715+0000","last_clean":"2024-04-25T00:32:35.436715+0000","last_became_active":"2024-04-25T00:32:33.411820+0000","last_became_peered":"2024-04-25T00:32:33.411820+0000","last_unstale":"2024-04-25T00:32:35.436715+0000","last_undegraded":"2024-04-25T00:32:35.436715+0000","last_fullsized":"2024-04-25T00:32:35.436715+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:26.408279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658882+0000","last_change":"2024-04-25T00:32:34.467502+0000","last_active":"2024-04-25T00:32:36.658882+0000","last_peered":"2024-04-25T00:32:36.658882+0000","last_clean":"2024-04-25T00:32:36.658882+0000","last_became_active":"2024-04-25T00:32:34.467168+0000","last_became_peered":"2024-04-25T00:32:34.467168+0000","last_unstale":"2024-04-25T00:32:36.658882+0000","last_undegraded":"2024-04-25T00:32:36.658882+0000","last_fullsized":"2024-04-25T00:32:36.658882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:08.845297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436398+0000","last_change":"2024-04-25T00:32:33.429101+0000","last_active":"2024-04-25T00:32:35.436398+0000","last_peered":"2024-04-25T00:32:35.436398+0000","last_clean":"2024-04-25T00:32:35.436398+0000","last_became_active":"2024-04-25T00:32:33.428344+0000","last_became_peered":"2024-04-25T00:32:33.428344+0000","last_unstale":"2024-04-25T00:32:35.436398+0000","last_undegraded":"2024-04-25T00:32:35.436398+0000","last_fullsized":"2024-04-25T00:32:35.436398+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:39.207154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658835+0000","last_change":"2024-04-25T00:32:34.463991+0000","last_active":"2024-04-25T00:32:36.658835+0000","last_peered":"2024-04-25T00:32:36.658835+0000","last_clean":"2024-04-25T00:32:36.658835+0000","last_became_active":"2024-04-25T00:32:34.463661+0000","last_became_peered":"2024-04-25T00:32:34.463661+0000","last_unstale":"2024-04-25T00:32:36.658835+0000","last_undegraded":"2024-04-25T00:32:36.658835+0000","last_fullsized":"2024-04-25T00:32:36.658835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:11.615017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437175+0000","last_change":"2024-04-25T00:32:33.419832+0000","last_active":"2024-04-25T00:32:36.437175+0000","last_peered":"2024-04-25T00:32:36.437175+0000","last_clean":"2024-04-25T00:32:36.437175+0000","last_became_active":"2024-04-25T00:32:33.419551+0000","last_became_peered":"2024-04-25T00:32:33.419551+0000","last_unstale":"2024-04-25T00:32:36.437175+0000","last_undegraded":"2024-04-25T00:32:36.437175+0000","last_fullsized":"2024-04-25T00:32:36.437175+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.651396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.659043+0000","last_change":"2024-04-25T00:32:34.447073+0000","last_active":"2024-04-25T00:32:36.659043+0000","last_peered":"2024-04-25T00:32:36.659043+0000","last_clean":"2024-04-25T00:32:36.659043+0000","last_became_active":"2024-04-25T00:32:34.446798+0000","last_became_peered":"2024-04-25T00:32:34.446798+0000","last_unstale":"2024-04-25T00:32:36.659043+0000","last_undegraded":"2024-04-25T00:32:36.659043+0000","last_fullsized":"2024-04-25T00:32:36.659043+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:29.301051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.049651+0000","last_change":"2024-04-25T00:32:33.432483+0000","last_active":"2024-04-25T00:32:36.049651+0000","last_peered":"2024-04-25T00:32:36.049651+0000","last_clean":"2024-04-25T00:32:36.049651+0000","last_became_active":"2024-04-25T00:32:33.431774+0000","last_became_peered":"2024-04-25T00:32:33.431774+0000","last_unstale":"2024-04-25T00:32:36.049651+0000","last_undegraded":"2024-04-25T00:32:36.049651+0000","last_fullsized":"2024-04-25T00:32:36.049651+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:34.696377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658919+0000","last_change":"2024-04-25T00:32:34.441629+0000","last_active":"2024-04-25T00:32:36.658919+0000","last_peered":"2024-04-25T00:32:36.658919+0000","last_clean":"2024-04-25T00:32:36.658919+0000","last_became_active":"2024-04-25T00:32:34.441306+0000","last_became_peered":"2024-04-25T00:32:34.441306+0000","last_unstale":"2024-04-25T00:32:36.658919+0000","last_undegraded":"2024-04-25T00:32:36.658919+0000","last_fullsized":"2024-04-25T00:32:36.658919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:25.129046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435133+0000","last_change":"2024-04-25T00:32:33.426645+0000","last_active":"2024-04-25T00:32:35.435133+0000","last_peered":"2024-04-25T00:32:35.435133+0000","last_clean":"2024-04-25T00:32:35.435133+0000","last_became_active":"2024-04-25T00:32:33.425875+0000","last_became_peered":"2024-04-25T00:32:33.425875+0000","last_unstale":"2024-04-25T00:32:35.435133+0000","last_undegraded":"2024-04-25T00:32:35.435133+0000","last_fullsized":"2024-04-25T00:32:35.435133+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:18.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547722+0000","last_change":"2024-04-25T00:32:34.482697+0000","last_active":"2024-04-25T00:32:35.547722+0000","last_peered":"2024-04-25T00:32:35.547722+0000","last_clean":"2024-04-25T00:32:35.547722+0000","last_became_active":"2024-04-25T00:32:34.482147+0000","last_became_peered":"2024-04-25T00:32:34.482147+0000","last_unstale":"2024-04-25T00:32:35.547722+0000","last_undegraded":"2024-04-25T00:32:35.547722+0000","last_fullsized":"2024-04-25T00:32:35.547722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:54.889078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.434106+0000","last_change":"2024-04-25T00:32:33.441614+0000","last_active":"2024-04-25T00:32:35.434106+0000","last_peered":"2024-04-25T00:32:35.434106+0000","last_clean":"2024-04-25T00:32:35.434106+0000","last_became_active":"2024-04-25T00:32:33.441004+0000","last_became_peered":"2024-04-25T00:32:33.441004+0000","last_unstale":"2024-04-25T00:32:35.434106+0000","last_undegraded":"2024-04-25T00:32:35.434106+0000","last_fullsized":"2024-04-25T00:32:35.434106+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:39.103683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.658870+0000","last_change":"2024-04-25T00:32:34.478773+0000","last_active":"2024-04-25T00:32:36.658870+0000","last_peered":"2024-04-25T00:32:36.658870+0000","last_clean":"2024-04-25T00:32:36.658870+0000","last_became_active":"2024-04-25T00:32:34.478484+0000","last_became_peered":"2024-04-25T00:32:34.478484+0000","last_unstale":"2024-04-25T00:32:36.658870+0000","last_undegraded":"2024-04-25T00:32:36.658870+0000","last_fullsized":"2024-04-25T00:32:36.658870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:29.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.433516+0000","last_change":"2024-04-25T00:32:33.412949+0000","last_active":"2024-04-25T00:32:35.433516+0000","last_peered":"2024-04-25T00:32:35.433516+0000","last_clean":"2024-04-25T00:32:35.433516+0000","last_became_active":"2024-04-25T00:32:33.412675+0000","last_became_peered":"2024-04-25T00:32:33.412675+0000","last_unstale":"2024-04-25T00:32:35.433516+0000","last_undegraded":"2024-04-25T00:32:35.433516+0000","last_fullsized":"2024-04-25T00:32:35.433516+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:24.119844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.436830+0000","last_change":"2024-04-25T00:32:34.485250+0000","last_active":"2024-04-25T00:32:35.436830+0000","last_peered":"2024-04-25T00:32:35.436830+0000","last_clean":"2024-04-25T00:32:35.436830+0000","last_became_active":"2024-04-25T00:32:34.484511+0000","last_became_peered":"2024-04-25T00:32:34.484511+0000","last_unstale":"2024-04-25T00:32:35.436830+0000","last_undegraded":"2024-04-25T00:32:35.436830+0000","last_fullsized":"2024-04-25T00:32:35.436830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:04.291110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437468+0000","last_change":"2024-04-25T00:32:33.420075+0000","last_active":"2024-04-25T00:32:36.437468+0000","last_peered":"2024-04-25T00:32:36.437468+0000","last_clean":"2024-04-25T00:32:36.437468+0000","last_became_active":"2024-04-25T00:32:33.419783+0000","last_became_peered":"2024-04-25T00:32:33.419783+0000","last_unstale":"2024-04-25T00:32:36.437468+0000","last_undegraded":"2024-04-25T00:32:36.437468+0000","last_fullsized":"2024-04-25T00:32:36.437468+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:24.119844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437057+0000","last_change":"2024-04-25T00:32:34.456626+0000","last_active":"2024-04-25T00:32:36.437057+0000","last_peered":"2024-04-25T00:32:36.437057+0000","last_clean":"2024-04-25T00:32:36.437057+0000","last_became_active":"2024-04-25T00:32:34.456328+0000","last_became_peered":"2024-04-25T00:32:34.456328+0000","last_unstale":"2024-04-25T00:32:36.437057+0000","last_undegraded":"2024-04-25T00:32:36.437057+0000","last_fullsized":"2024-04-25T00:32:36.437057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:17.530979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.432988+0000","last_change":"2024-04-25T00:32:33.411847+0000","last_active":"2024-04-25T00:32:35.432988+0000","last_peered":"2024-04-25T00:32:35.432988+0000","last_clean":"2024-04-25T00:32:35.432988+0000","last_became_active":"2024-04-25T00:32:33.411429+0000","last_became_peered":"2024-04-25T00:32:33.411429+0000","last_unstale":"2024-04-25T00:32:35.432988+0000","last_undegraded":"2024-04-25T00:32:35.432988+0000","last_fullsized":"2024-04-25T00:32:35.432988+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:07.836095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.049585+0000","last_change":"2024-04-25T00:32:34.465264+0000","last_active":"2024-04-25T00:32:36.049585+0000","last_peered":"2024-04-25T00:32:36.049585+0000","last_clean":"2024-04-25T00:32:36.049585+0000","last_became_active":"2024-04-25T00:32:34.461301+0000","last_became_peered":"2024-04-25T00:32:34.461301+0000","last_unstale":"2024-04-25T00:32:36.049585+0000","last_undegraded":"2024-04-25T00:32:36.049585+0000","last_fullsized":"2024-04-25T00:32:36.049585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:16.435951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437486+0000","last_change":"2024-04-25T00:32:33.417550+0000","last_active":"2024-04-25T00:32:36.437486+0000","last_peered":"2024-04-25T00:32:36.437486+0000","last_clean":"2024-04-25T00:32:36.437486+0000","last_became_active":"2024-04-25T00:32:33.417040+0000","last_became_peered":"2024-04-25T00:32:33.417040+0000","last_unstale":"2024-04-25T00:32:36.437486+0000","last_undegraded":"2024-04-25T00:32:36.437486+0000","last_fullsized":"2024-04-25T00:32:36.437486+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:34.696377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.437106+0000","last_change":"2024-04-25T00:32:34.453155+0000","last_active":"2024-04-25T00:32:36.437106+0000","last_peered":"2024-04-25T00:32:36.437106+0000","last_clean":"2024-04-25T00:32:36.437106+0000","last_became_active":"2024-04-25T00:32:34.452888+0000","last_became_peered":"2024-04-25T00:32:34.452888+0000","last_unstale":"2024-04-25T00:32:36.437106+0000","last_undegraded":"2024-04-25T00:32:36.437106+0000","last_fullsized":"2024-04-25T00:32:36.437106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:25.019780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.432193+0000","last_change":"2024-04-25T00:32:33.412644+0000","last_active":"2024-04-25T00:32:35.432193+0000","last_peered":"2024-04-25T00:32:35.432193+0000","last_clean":"2024-04-25T00:32:35.432193+0000","last_became_active":"2024-04-25T00:32:33.412193+0000","last_became_peered":"2024-04-25T00:32:33.412193+0000","last_unstale":"2024-04-25T00:32:35.432193+0000","last_undegraded":"2024-04-25T00:32:35.432193+0000","last_fullsized":"2024-04-25T00:32:35.432193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:34.696377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.523517+0000","last_change":"2024-04-25T00:32:34.445525+0000","last_active":"2024-04-25T00:32:35.523517+0000","last_peered":"2024-04-25T00:32:35.523517+0000","last_clean":"2024-04-25T00:32:35.523517+0000","last_became_active":"2024-04-25T00:32:34.444600+0000","last_became_peered":"2024-04-25T00:32:34.444600+0000","last_unstale":"2024-04-25T00:32:35.523517+0000","last_undegraded":"2024-04-25T00:32:35.523517+0000","last_fullsized":"2024-04-25T00:32:35.523517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:18.311016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435449+0000","last_change":"2024-04-25T00:32:33.416074+0000","last_active":"2024-04-25T00:32:35.435449+0000","last_peered":"2024-04-25T00:32:35.435449+0000","last_clean":"2024-04-25T00:32:35.435449+0000","last_became_active":"2024-04-25T00:32:33.415336+0000","last_became_peered":"2024-04-25T00:32:33.415336+0000","last_unstale":"2024-04-25T00:32:35.435449+0000","last_undegraded":"2024-04-25T00:32:35.435449+0000","last_fullsized":"2024-04-25T00:32:35.435449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:05.605371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.435235+0000","last_change":"2024-04-25T00:32:34.448880+0000","last_active":"2024-04-25T00:32:35.435235+0000","last_peered":"2024-04-25T00:32:35.435235+0000","last_clean":"2024-04-25T00:32:35.435235+0000","last_became_active":"2024-04-25T00:32:34.448082+0000","last_became_peered":"2024-04-25T00:32:34.448082+0000","last_unstale":"2024-04-25T00:32:35.435235+0000","last_undegraded":"2024-04-25T00:32:35.435235+0000","last_fullsized":"2024-04-25T00:32:35.435235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:37.803710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.523812+0000","last_change":"2024-04-25T00:32:33.440955+0000","last_active":"2024-04-25T00:32:35.523812+0000","last_peered":"2024-04-25T00:32:35.523812+0000","last_clean":"2024-04-25T00:32:35.523812+0000","last_became_active":"2024-04-25T00:32:33.440603+0000","last_became_peered":"2024-04-25T00:32:33.440603+0000","last_unstale":"2024-04-25T00:32:35.523812+0000","last_undegraded":"2024-04-25T00:32:35.523812+0000","last_fullsized":"2024-04-25T00:32:35.523812+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:36.794509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.548197+0000","last_change":"2024-04-25T00:32:34.488479+0000","last_active":"2024-04-25T00:32:35.548197+0000","last_peered":"2024-04-25T00:32:35.548197+0000","last_clean":"2024-04-25T00:32:35.548197+0000","last_became_active":"2024-04-25T00:32:34.483778+0000","last_became_peered":"2024-04-25T00:32:34.483778+0000","last_unstale":"2024-04-25T00:32:35.548197+0000","last_undegraded":"2024-04-25T00:32:35.548197+0000","last_fullsized":"2024-04-25T00:32:35.548197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:33.389607+0000","last_clean_scrub_stamp":"2024-04-25T00:32:33.389607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:16.527471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:36.049715+0000","last_change":"2024-04-25T00:32:33.432559+0000","last_active":"2024-04-25T00:32:36.049715+0000","last_peered":"2024-04-25T00:32:36.049715+0000","last_clean":"2024-04-25T00:32:36.049715+0000","last_became_active":"2024-04-25T00:32:33.431829+0000","last_became_peered":"2024-04-25T00:32:33.431829+0000","last_unstale":"2024-04-25T00:32:36.049715+0000","last_undegraded":"2024-04-25T00:32:36.049715+0000","last_fullsized":"2024-04-25T00:32:36.049715+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:32:32.380405+0000","last_clean_scrub_stamp":"2024-04-25T00:32:32.380405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:10.605816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":194,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T00:32:35.547965+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:32:35.547965+0000","last_peered":"2024-04-25T00:32:35.547965+0000","last_clean":"2024-04-25T00:32:35.547965+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:32:35.547965+0000","last_undegraded":"2024-04-25T00:32:35.547965+0000","last_fullsized":"2024-04-25T00:32:35.547965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":737414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":147572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":147628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":147060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":389120,"data_stored":148590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":376832,"data_stored":147026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":389120,"data_stored":148590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":966656,"data_stored":737324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:32:43.302 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:32:43.303 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:32:43.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.471+0000 7f13554a6640 1 Processor -- start 2024-04-25T00:32:43.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.471+0000 7f13554a6640 1 -- start start 2024-04-25T00:32:43.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f13554a6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f13500d0670 0x7f1350166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f13554a6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13500cdbd0 con 0x7f13500cfa10 2024-04-25T00:32:43.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f13554a6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13500b13e0 con 0x7f135015c220 2024-04-25T00:32:43.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f13554a6640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f135015c600 con 0x7f13500d0670 2024-04-25T00:32:43.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f134f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f13500d0670 0x7f1350166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f134e7fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f13500cfa10 0x7f13500cff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38348/0 (socket says 172.21.15.29:38348) 2024-04-25T00:32:43.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f134e7fc640 1 -- 172.21.15.29:0/358807735 learned_addr learned my addr 172.21.15.29:0/358807735 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:43.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f134f7fe640 1 -- 172.21.15.29:0/358807735 >> v1:172.21.15.157:6789/0 conn(0x7f135015c220 legacy=0x7f13500cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:43.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f134f7fe640 1 -- 172.21.15.29:0/358807735 >> v1:172.21.15.29:6789/0 conn(0x7f13500cfa10 legacy=0x7f13500cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f134dffb640 1 -- 172.21.15.29:0/358807735 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2670739470 0 0) 0x7f13500cdbd0 con 0x7f13500cfa10 2024-04-25T00:32:43.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f134f7fe640 1 -- 172.21.15.29:0/358807735 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13500b16c0 con 0x7f13500d0670 2024-04-25T00:32:43.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f134f7fe640 1 --2- 172.21.15.29:0/358807735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13500d0670 0x7f1350166cf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f13400097c0 tx=0x7f134001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f504da3e4f18df08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:43.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f134dffb640 1 -- 172.21.15.29:0/358807735 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13400b0070 con 0x7f13500d0670 2024-04-25T00:32:43.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f134dffb640 1 -- 172.21.15.29:0/358807735 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13400ab600 con 0x7f13500d0670 2024-04-25T00:32:43.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f134dffb640 1 -- 172.21.15.29:0/358807735 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13400ab9d0 con 0x7f13500d0670 2024-04-25T00:32:43.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f13554a6640 1 -- 172.21.15.29:0/358807735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13500d0670 msgr2=0x7f1350166cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f13554a6640 1 --2- 172.21.15.29:0/358807735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13500d0670 0x7f1350166cf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f13400097c0 tx=0x7f134001c460 comp rx=0 tx=0).stop 2024-04-25T00:32:43.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f13554a6640 1 -- 172.21.15.29:0/358807735 shutdown_connections 2024-04-25T00:32:43.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f13554a6640 1 --2- 172.21.15.29:0/358807735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13500d0670 0x7f1350166cf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f13554a6640 1 -- 172.21.15.29:0/358807735 >> 172.21.15.29:0/358807735 conn(0x7f13500d1d60 msgr2=0x7f13500d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:43.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f13554a6640 1 -- 172.21.15.29:0/358807735 shutdown_connections 2024-04-25T00:32:43.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.475+0000 7f13554a6640 1 -- 172.21.15.29:0/358807735 wait complete. 2024-04-25T00:32:43.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f13554a6640 1 Processor -- start 2024-04-25T00:32:43.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f13554a6640 1 -- start start 2024-04-25T00:32:43.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f13554a6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cfa10 0x7f13500c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f13554a6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13500d0670 0x7f1350153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f13554a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135015c220 0x7f135015e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f13554a6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1350169c60 con 0x7f13500d0670 2024-04-25T00:32:43.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f13554a6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13500b1a30 con 0x7f135015c220 2024-04-25T00:32:43.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f13554a6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1350169de0 con 0x7f13500cfa10 2024-04-25T00:32:43.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f134e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13500d0670 0x7f1350153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f134e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13500d0670 0x7f1350153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54330/0 (socket says 172.21.15.29:54330) 2024-04-25T00:32:43.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f134e7fc640 1 -- 172.21.15.29:0/3601573109 learned_addr learned my addr 172.21.15.29:0/3601573109 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:43.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f134effd640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cfa10 0x7f13500c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f134f7fe640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135015c220 0x7f135015e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f134e7fc640 1 -- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cfa10 msgr2=0x7f13500c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f134e7fc640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cfa10 0x7f13500c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f134e7fc640 1 -- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135015c220 msgr2=0x7f135015e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f134e7fc640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135015c220 0x7f135015e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f134e7fc640 1 -- 172.21.15.29:0/3601573109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13500c5860 con 0x7f13500d0670 2024-04-25T00:32:43.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f134f7fe640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135015c220 0x7f135015e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:43.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f134e7fc640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13500d0670 0x7f1350153ca0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f133800c480 tx=0x7f133800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:43.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f13477fe640 1 -- 172.21.15.29:0/3601573109 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338018070 con 0x7f13500d0670 2024-04-25T00:32:43.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f13477fe640 1 -- 172.21.15.29:0/3601573109 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1338007500 con 0x7f13500d0670 2024-04-25T00:32:43.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f13477fe640 1 -- 172.21.15.29:0/3601573109 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338004a30 con 0x7f13500d0670 2024-04-25T00:32:43.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f13554a6640 1 -- 172.21.15.29:0/3601573109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13500c89b0 con 0x7f13500d0670 2024-04-25T00:32:43.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.479+0000 7f13554a6640 1 -- 172.21.15.29:0/3601573109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1350154790 con 0x7f13500d0670 2024-04-25T00:32:43.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.483+0000 7f13554a6640 1 -- 172.21.15.29:0/3601573109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13500c5d40 con 0x7f13500d0670 2024-04-25T00:32:43.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.483+0000 7f13477fe640 1 -- 172.21.15.29:0/3601573109 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1338004bd0 con 0x7f13500d0670 2024-04-25T00:32:43.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.483+0000 7f13477fe640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13300678f0 0x7f1330069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.483+0000 7f13477fe640 1 -- 172.21.15.29:0/3601573109 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1338084f00 con 0x7f13500d0670 2024-04-25T00:32:43.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.483+0000 7f134effd640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13300678f0 0x7f1330069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.483+0000 7f134effd640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13300678f0 0x7f1330069db0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1348005530 tx=0x7f1348009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:43.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.487+0000 7f13477fe640 1 -- 172.21.15.29:0/3601573109 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f13500c5d40 con 0x7f13500d0670 2024-04-25T00:32:43.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.699+0000 7f13554a6640 1 -- 172.21.15.29:0/3601573109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1350069270 con 0x7f13500d0670 2024-04-25T00:32:43.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.703+0000 7f13477fe640 1 -- 172.21.15.29:0/3601573109 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f1338055a30 con 0x7f13500d0670 2024-04-25T00:32:43.708 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-04-25T00:32:43.708 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A== 2024-04-25T00:32:43.708 INFO:teuthology.orchestra.run.smithi029.stdout: caps mds = "allow" 2024-04-25T00:32:43.708 INFO:teuthology.orchestra.run.smithi029.stdout: caps mon = "allow r" 2024-04-25T00:32:43.708 INFO:teuthology.orchestra.run.smithi029.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:32:43.708 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-04-25T00:32:43.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.707+0000 7f13554a6640 1 -- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13300678f0 msgr2=0x7f1330069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.707+0000 7f13554a6640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13300678f0 0x7f1330069db0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1348005530 tx=0x7f1348009040 comp rx=0 tx=0).stop 2024-04-25T00:32:43.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.707+0000 7f13554a6640 1 -- 172.21.15.29:0/3601573109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13500d0670 msgr2=0x7f1350153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.707+0000 7f13554a6640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13500d0670 0x7f1350153ca0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f133800c480 tx=0x7f133800c940 comp rx=0 tx=0).stop 2024-04-25T00:32:43.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.707+0000 7f13554a6640 1 -- 172.21.15.29:0/3601573109 shutdown_connections 2024-04-25T00:32:43.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.707+0000 7f13554a6640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13300678f0 0x7f1330069db0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.707+0000 7f13554a6640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135015c220 0x7f135015e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.707+0000 7f13554a6640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13500d0670 0x7f1350153ca0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.707+0000 7f13554a6640 1 --2- 172.21.15.29:0/3601573109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cfa10 0x7f13500c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.707+0000 7f13554a6640 1 -- 172.21.15.29:0/3601573109 >> 172.21.15.29:0/3601573109 conn(0x7f13500d1d60 msgr2=0x7f13500cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:43.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.707+0000 7f13554a6640 1 -- 172.21.15.29:0/3601573109 shutdown_connections 2024-04-25T00:32:43.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.707+0000 7f13554a6640 1 -- 172.21.15.29:0/3601573109 wait complete. 2024-04-25T00:32:43.726 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:32:43.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40ff2cc640 1 Processor -- start 2024-04-25T00:32:43.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40ff2cc640 1 -- start start 2024-04-25T00:32:43.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40ff2cc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f40f81584c0 0x7f40f80715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40ff2cc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f40f81588a0 0x7f40f8071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40ff2cc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f815e930 con 0x7f40f80b4060 2024-04-25T00:32:43.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40ff2cc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f40f806d330 con 0x7f40f81588a0 2024-04-25T00:32:43.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40ff2cc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f40f80ae3b0 con 0x7f40f81584c0 2024-04-25T00:32:43.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40fc867640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f40f81588a0 0x7f40f8071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40fc867640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f40f81588a0 0x7f40f8071af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54336/0 (socket says 172.21.15.29:54336) 2024-04-25T00:32:43.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40fc867640 1 -- 172.21.15.29:0/1619004003 learned_addr learned my addr 172.21.15.29:0/1619004003 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:43.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40fd068640 1 --2- 172.21.15.29:0/1619004003 >> v2:172.21.15.157:3300/0 conn(0x7f40f81584c0 0x7f40f80715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40fc867640 1 -- 172.21.15.29:0/1619004003 >> v2:172.21.15.157:3300/0 conn(0x7f40f81584c0 msgr2=0x7f40f80715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:43.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40fc867640 1 --2- 172.21.15.29:0/1619004003 >> v2:172.21.15.157:3300/0 conn(0x7f40f81584c0 0x7f40f80715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40fc867640 1 -- 172.21.15.29:0/1619004003 >> v1:172.21.15.157:6789/0 conn(0x7f40f80b4060 legacy=0x7f40f8161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:43.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.843+0000 7f40fc867640 1 -- 172.21.15.29:0/1619004003 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f80b4c40 con 0x7f40f81588a0 2024-04-25T00:32:43.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40fc867640 1 --2- 172.21.15.29:0/1619004003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f81588a0 0x7f40f8071af0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f40f80b2f90 tx=0x7f40e801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee533a417b13e1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:43.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40effff640 1 -- 172.21.15.29:0/1619004003 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e80b1070 con 0x7f40f81588a0 2024-04-25T00:32:43.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40effff640 1 -- 172.21.15.29:0/1619004003 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e80ac630 con 0x7f40f81588a0 2024-04-25T00:32:43.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40effff640 1 -- 172.21.15.29:0/1619004003 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e80a8050 con 0x7f40f81588a0 2024-04-25T00:32:43.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1619004003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f81588a0 msgr2=0x7f40f8071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40ff2cc640 1 --2- 172.21.15.29:0/1619004003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f81588a0 0x7f40f8071af0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f40f80b2f90 tx=0x7f40e801c4e0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1619004003 shutdown_connections 2024-04-25T00:32:43.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40ff2cc640 1 --2- 172.21.15.29:0/1619004003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f81588a0 0x7f40f8071af0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40ff2cc640 1 --2- 172.21.15.29:0/1619004003 >> v2:172.21.15.157:3300/0 conn(0x7f40f81584c0 0x7f40f80715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1619004003 >> 172.21.15.29:0/1619004003 conn(0x7f40f8075030 msgr2=0x7f40f80b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:43.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1619004003 shutdown_connections 2024-04-25T00:32:43.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1619004003 wait complete. 2024-04-25T00:32:43.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40ff2cc640 1 Processor -- start 2024-04-25T00:32:43.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.847+0000 7f40ff2cc640 1 -- start start 2024-04-25T00:32:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40ff2cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40f80b4060 0x7f40f814ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40ff2cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40f81584c0 0x7f40f80b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40ff2cc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f81588a0 0x7f40f80b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40ff2cc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f40f80afdf0 con 0x7f40f81588a0 2024-04-25T00:32:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40ff2cc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f40f80b4920 con 0x7f40f80b4060 2024-04-25T00:32:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40ff2cc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f40f80aff70 con 0x7f40f81584c0 2024-04-25T00:32:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40fd869640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f81588a0 0x7f40f80b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40fd869640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f81588a0 0x7f40f80b07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54352/0 (socket says 172.21.15.29:54352) 2024-04-25T00:32:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40fd869640 1 -- 172.21.15.29:0/1740863845 learned_addr learned my addr 172.21.15.29:0/1740863845 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40fd068640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40f80b4060 0x7f40f814ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40fc867640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40f81584c0 0x7f40f80b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40fd869640 1 -- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40f81584c0 msgr2=0x7f40f80b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40fd869640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40f81584c0 0x7f40f80b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40fd869640 1 -- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40f80b4060 msgr2=0x7f40f814ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:43.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40fd869640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40f80b4060 0x7f40f814ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:43.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40fd869640 1 -- 172.21.15.29:0/1740863845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f814d370 con 0x7f40f81588a0 2024-04-25T00:32:43.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40fd869640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f81588a0 0x7f40f80b07b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f40f000c3c0 tx=0x7f40f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:43.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40edffb640 1 -- 172.21.15.29:0/1740863845 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0011790 con 0x7f40f81588a0 2024-04-25T00:32:43.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40edffb640 1 -- 172.21.15.29:0/1740863845 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f0017070 con 0x7f40f81588a0 2024-04-25T00:32:43.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1740863845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f80cf250 con 0x7f40f81588a0 2024-04-25T00:32:43.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40edffb640 1 -- 172.21.15.29:0/1740863845 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0011b70 con 0x7f40f81588a0 2024-04-25T00:32:43.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1740863845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f80cf810 con 0x7f40f81588a0 2024-04-25T00:32:43.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1740863845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f8069270 con 0x7f40f81588a0 2024-04-25T00:32:43.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40edffb640 1 -- 172.21.15.29:0/1740863845 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f40f0011d10 con 0x7f40f81588a0 2024-04-25T00:32:43.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40edffb640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f40dc0678c0 0x7f40dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:43.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.851+0000 7f40edffb640 1 -- 172.21.15.29:0/1740863845 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f40f00864d0 con 0x7f40f81588a0 2024-04-25T00:32:43.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.855+0000 7f40fd068640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f40dc0678c0 0x7f40dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:43.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.855+0000 7f40fd068640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f40dc0678c0 0x7f40dc069d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f40f4003fd0 tx=0x7f40f40039a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:43.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:43.859+0000 7f40edffb640 1 -- 172.21.15.29:0/1740863845 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f40f00570f0 con 0x7f40f81588a0 2024-04-25T00:32:44.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.047+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1740863845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40f80743d0 con 0x7f40f81588a0 2024-04-25T00:32:44.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.047+0000 7f40edffb640 1 -- 172.21.15.29:0/1740863845 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5186 (secure 0 0 0) 0x7f40f00569f0 con 0x7f40f81588a0 2024-04-25T00:32:44.051 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:44.051 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2100384916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2100384916},{"type":"v1","addr":"172.21.15.157:6835","nonce":2100384916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4984,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1430513194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1430513194},{"type":"v1","addr":"172.21.15.157:6837","nonce":1430513194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3354831881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3354831881},{"type":"v1","addr":"172.21.15.29:6833","nonce":3354831881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:32:33.777640+0000","modified":"2024-04-25T00:32:36.837733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1468158138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1468158138},{"type":"v1","addr":"172.21.15.29:6835","nonce":1468158138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":2}]} 2024-04-25T00:32:44.052 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 15 2024-04-25T00:32:44.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.051+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f40dc0678c0 msgr2=0x7f40dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.051+0000 7f40ff2cc640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f40dc0678c0 0x7f40dc069d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f40f4003fd0 tx=0x7f40f40039a0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.051+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1740863845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f81588a0 msgr2=0x7f40f80b07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.051+0000 7f40ff2cc640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f81588a0 0x7f40f80b07b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f40f000c3c0 tx=0x7f40f000c880 comp rx=0 tx=0).stop 2024-04-25T00:32:44.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.051+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1740863845 shutdown_connections 2024-04-25T00:32:44.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.051+0000 7f40ff2cc640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f81588a0 0x7f40f80b07b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.051+0000 7f40ff2cc640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40f81584c0 0x7f40f80b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.051+0000 7f40ff2cc640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f40dc0678c0 0x7f40dc069d80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.051+0000 7f40ff2cc640 1 --2- 172.21.15.29:0/1740863845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40f80b4060 0x7f40f814ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.051+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1740863845 >> 172.21.15.29:0/1740863845 conn(0x7f40f8075030 msgr2=0x7f40f80b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:44.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.051+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1740863845 shutdown_connections 2024-04-25T00:32:44.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.051+0000 7f40ff2cc640 1 -- 172.21.15.29:0/1740863845 wait complete. 2024-04-25T00:32:44.472 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:32:33.777640+0000', 'modified': '2024-04-25T00:32:36.837733+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5214}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5214': {'gid': 5214, 'name': 'c', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/1468158138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 1468158138}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 1468158138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5214, 'qdb_cluster': [5214]} 2024-04-25T00:32:44.472 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:32:44.473 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:32:44.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.603+0000 7f1d4aa78640 1 Processor -- start 2024-04-25T00:32:44.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.603+0000 7f1d4aa78640 1 -- start start 2024-04-25T00:32:44.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.603+0000 7f1d4aa78640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1d440b4780 0x7f1d4415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.603+0000 7f1d4aa78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d440cc240 0x7f1d4415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.603+0000 7f1d4aa78640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d440b3720 con 0x7f1d4415e8d0 2024-04-25T00:32:44.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.603+0000 7f1d4aa78640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1d440b42f0 con 0x7f1d440b4780 2024-04-25T00:32:44.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.603+0000 7f1d4aa78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1d440b39c0 con 0x7f1d440cc240 2024-04-25T00:32:44.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.603+0000 7f1d48814640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1d4415e8d0 0x7f1d4415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38350/0 (socket says 172.21.15.29:38350) 2024-04-25T00:32:44.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.603+0000 7f1d48814640 1 -- 172.21.15.29:0/142596263 learned_addr learned my addr 172.21.15.29:0/142596263 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:44.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.603+0000 7f1d43fff640 1 --2- 172.21.15.29:0/142596263 >> v2:172.21.15.157:3300/0 conn(0x7f1d440b4780 0x7f1d4415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.603+0000 7f1d49015640 1 --2- 172.21.15.29:0/142596263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d440cc240 0x7f1d4415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.603+0000 7f1d437fe640 1 -- 172.21.15.29:0/142596263 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4276357375 0 0) 0x7f1d440b3720 con 0x7f1d4415e8d0 2024-04-25T00:32:44.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.603+0000 7f1d437fe640 1 -- 172.21.15.29:0/142596263 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d34003530 con 0x7f1d4415e8d0 2024-04-25T00:32:44.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d43fff640 1 -- 172.21.15.29:0/142596263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d440cc240 msgr2=0x7f1d4415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:44.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d43fff640 1 --2- 172.21.15.29:0/142596263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d440cc240 0x7f1d4415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d43fff640 1 -- 172.21.15.29:0/142596263 >> v1:172.21.15.29:6789/0 conn(0x7f1d4415e8d0 legacy=0x7f1d4415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d437fe640 1 -- 172.21.15.29:0/142596263 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4171937978 0 0) 0x7f1d34003530 con 0x7f1d4415e8d0 2024-04-25T00:32:44.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d43fff640 1 -- 172.21.15.29:0/142596263 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d440b3ca0 con 0x7f1d440b4780 2024-04-25T00:32:44.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d49015640 1 --2- 172.21.15.29:0/142596263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d440cc240 0x7f1d4415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:44.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d43fff640 1 --2- 172.21.15.29:0/142596263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d440b4780 0x7f1d4415f3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d440b4470 tx=0x7f1d3001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65ff265c0c63db76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d437fe640 1 -- 172.21.15.29:0/142596263 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d300b0070 con 0x7f1d440b4780 2024-04-25T00:32:44.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d437fe640 1 -- 172.21.15.29:0/142596263 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d300ab400 con 0x7f1d440b4780 2024-04-25T00:32:44.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d437fe640 1 -- 172.21.15.29:0/142596263 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d300a4e60 con 0x7f1d440b4780 2024-04-25T00:32:44.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d4aa78640 1 -- 172.21.15.29:0/142596263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d440b4780 msgr2=0x7f1d4415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d4aa78640 1 --2- 172.21.15.29:0/142596263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d440b4780 0x7f1d4415f3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d440b4470 tx=0x7f1d3001c3c0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d4aa78640 1 -- 172.21.15.29:0/142596263 shutdown_connections 2024-04-25T00:32:44.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d4aa78640 1 --2- 172.21.15.29:0/142596263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d440cc240 0x7f1d4415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d4aa78640 1 --2- 172.21.15.29:0/142596263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d440b4780 0x7f1d4415f3c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d4aa78640 1 -- 172.21.15.29:0/142596263 >> 172.21.15.29:0/142596263 conn(0x7f1d440b1870 msgr2=0x7f1d440b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:44.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d4aa78640 1 -- 172.21.15.29:0/142596263 shutdown_connections 2024-04-25T00:32:44.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.607+0000 7f1d4aa78640 1 -- 172.21.15.29:0/142596263 wait complete. 2024-04-25T00:32:44.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d4aa78640 1 Processor -- start 2024-04-25T00:32:44.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d4aa78640 1 -- start start 2024-04-25T00:32:44.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d4aa78640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d440b4780 0x7f1d44143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d4aa78640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d440cc240 0x7f1d4415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d4aa78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4415e8d0 0x7f1d4415a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d4aa78640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4415d6a0 con 0x7f1d440cc240 2024-04-25T00:32:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d4aa78640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4415d520 con 0x7f1d440b4780 2024-04-25T00:32:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d4aa78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1d44167c70 con 0x7f1d4415e8d0 2024-04-25T00:32:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d43fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d440cc240 0x7f1d4415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d43fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d440cc240 0x7f1d4415a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54368/0 (socket says 172.21.15.29:54368) 2024-04-25T00:32:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d43fff640 1 -- 172.21.15.29:0/2058761307 learned_addr learned my addr 172.21.15.29:0/2058761307 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d49015640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4415e8d0 0x7f1d4415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d48814640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d440b4780 0x7f1d44143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d43fff640 1 -- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4415e8d0 msgr2=0x7f1d4415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d43fff640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4415e8d0 0x7f1d4415a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d43fff640 1 -- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d440b4780 msgr2=0x7f1d44143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d43fff640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d440b4780 0x7f1d44143b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d43fff640 1 -- 172.21.15.29:0/2058761307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d44136130 con 0x7f1d440cc240 2024-04-25T00:32:44.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d48814640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d440b4780 0x7f1d44143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:44.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d43fff640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d440cc240 0x7f1d4415a240 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1d3001aa70 tx=0x7f1d3001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d417fa640 1 -- 172.21.15.29:0/2058761307 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d300b0050 con 0x7f1d440cc240 2024-04-25T00:32:44.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d417fa640 1 -- 172.21.15.29:0/2058761307 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d300bc070 con 0x7f1d440cc240 2024-04-25T00:32:44.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d417fa640 1 -- 172.21.15.29:0/2058761307 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38001680 con 0x7f1d440cc240 2024-04-25T00:32:44.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d4aa78640 1 -- 172.21.15.29:0/2058761307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d44136410 con 0x7f1d440cc240 2024-04-25T00:32:44.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.611+0000 7f1d4aa78640 1 -- 172.21.15.29:0/2058761307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d441369d0 con 0x7f1d440cc240 2024-04-25T00:32:44.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d417fa640 1 -- 172.21.15.29:0/2058761307 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1d300b7870 con 0x7f1d440cc240 2024-04-25T00:32:44.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d417fa640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d280678c0 0x7f1d28069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d417fa640 1 -- 172.21.15.29:0/2058761307 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1d30120050 con 0x7f1d440cc240 2024-04-25T00:32:44.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d48814640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d280678c0 0x7f1d28069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d4dbf1000 1 Processor -- start 2024-04-25T00:32:44.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d4dbf1000 1 -- start start 2024-04-25T00:32:44.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d48814640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d280678c0 0x7f1d28069d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007b70 tx=0x7f1d38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d4dbf1000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa747ef0 0x55cafa7482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d4dbf1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa6fe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d4dbf1000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 0x55cafa6fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d4dbf1000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cafa54c5e0 con 0x55cafa6febc0 2024-04-25T00:32:44.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d4dbf1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cafa4d3c50 con 0x55cafa748810 2024-04-25T00:32:44.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d4dbf1000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55cafa5485a0 con 0x55cafa747ef0 2024-04-25T00:32:44.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d49015640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 0x55cafa6fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d49015640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 0x55cafa6fefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54378/0 (socket says 172.21.15.29:54378) 2024-04-25T00:32:44.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d49015640 1 -- 172.21.15.29:0/1808912436 learned_addr learned my addr 172.21.15.29:0/1808912436 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d43fff640 1 --2- 172.21.15.29:0/1808912436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa6fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d48814640 1 --2- 172.21.15.29:0/1808912436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa747ef0 0x55cafa7482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d49015640 1 -- 172.21.15.29:0/1808912436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa747ef0 msgr2=0x55cafa7482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d49015640 1 --2- 172.21.15.29:0/1808912436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa747ef0 0x55cafa7482d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d49015640 1 -- 172.21.15.29:0/1808912436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 msgr2=0x55cafa6fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d49015640 1 --2- 172.21.15.29:0/1808912436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa6fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d49015640 1 -- 172.21.15.29:0/1808912436 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cafa4d1d10 con 0x55cafa6febc0 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.615+0000 7f1d43fff640 1 --2- 172.21.15.29:0/1808912436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa6fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d49015640 1 --2- 172.21.15.29:0/1808912436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 0x55cafa6fefa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c011580 tx=0x7f1d3c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76f64af26bc04a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d2e7fc640 1 -- 172.21.15.29:0/1808912436 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c003ba0 con 0x55cafa6febc0 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d2e7fc640 1 -- 172.21.15.29:0/1808912436 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d3c003d40 con 0x55cafa6febc0 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d2e7fc640 1 -- 172.21.15.29:0/1808912436 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c015920 con 0x55cafa6febc0 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/1808912436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 msgr2=0x55cafa6fefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/1808912436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 0x55cafa6fefa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c011580 tx=0x7f1d3c009750 comp rx=0 tx=0).stop 2024-04-25T00:32:44.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/1808912436 shutdown_connections 2024-04-25T00:32:44.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/1808912436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 0x55cafa6fefa0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/1808912436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa6fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/1808912436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa747ef0 0x55cafa7482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/1808912436 >> 172.21.15.29:0/1808912436 conn(0x55cafa6fab70 msgr2=0x55cafa6fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:44.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/1808912436 shutdown_connections 2024-04-25T00:32:44.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/1808912436 wait complete. 2024-04-25T00:32:44.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 -- start start 2024-04-25T00:32:44.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 Processor -- start 2024-04-25T00:32:44.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa6febc0 0x55cafa72cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 0x55cafa72d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa72d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cafa506520 con 0x55cafa747ef0 2024-04-25T00:32:44.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cafa519340 con 0x55cafa748810 2024-04-25T00:32:44.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55cafa4f2810 con 0x55cafa6febc0 2024-04-25T00:32:44.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 10 client.0 _ll_register_callbacks cb 0x55cafa6e95c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:44.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d43fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 0x55cafa72d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d43fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 0x55cafa72d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54386/0 (socket says 172.21.15.29:54386) 2024-04-25T00:32:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d43fff640 1 -- 172.21.15.29:0/3253806629 learned_addr learned my addr 172.21.15.29:0/3253806629 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d49015640 1 --2- 172.21.15.29:0/3253806629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa72d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d48814640 1 --2- 172.21.15.29:0/3253806629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa6febc0 0x55cafa72cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d43fff640 1 -- 172.21.15.29:0/3253806629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa6febc0 msgr2=0x55cafa72cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d43fff640 1 --2- 172.21.15.29:0/3253806629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa6febc0 0x55cafa72cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d43fff640 1 -- 172.21.15.29:0/3253806629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 msgr2=0x55cafa72d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d43fff640 1 --2- 172.21.15.29:0/3253806629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa72d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d43fff640 1 -- 172.21.15.29:0/3253806629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cafa4e47b0 con 0x55cafa747ef0 2024-04-25T00:32:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d43fff640 1 --2- 172.21.15.29:0/3253806629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 0x55cafa72d460 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1d30000c10 tx=0x7f1d30006ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cf1a0db03370a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d237fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:32:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d237fe640 1 -- 172.21.15.29:0/3253806629 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3011c060 con 0x55cafa747ef0 2024-04-25T00:32:44.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d237fe640 1 -- 172.21.15.29:0/3253806629 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1d300f8be0 con 0x55cafa747ef0 2024-04-25T00:32:44.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/3253806629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d3c003d40 con 0x55cafa747ef0 2024-04-25T00:32:44.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d237fe640 1 -- 172.21.15.29:0/3253806629 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f1d3010d050 con 0x55cafa747ef0 2024-04-25T00:32:44.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 10 client.5430 fetch_fsmap learned FSMap version 15 2024-04-25T00:32:44.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/3253806629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55cafa4d3ea0 con 0x55cafa747ef0 2024-04-25T00:32:44.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d237fe640 1 -- 172.21.15.29:0/3253806629 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d3c003d40 con 0x55cafa747ef0 2024-04-25T00:32:44.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d237fe640 1 -- 172.21.15.29:0/3253806629 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f1d300b7b60 con 0x55cafa747ef0 2024-04-25T00:32:44.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 10 client.5430 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T00:32:44.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 10 client.5430 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:32:44.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 20 client.5430 populate_metadata read hostname 'smithi029' 2024-04-25T00:32:44.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/3253806629 >> [v2:172.21.15.29:6834/1468158138,v1:172.21.15.29:6835/1468158138] conn(0x55cafa757f90 0x55cafa75a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 4 client.5430 mds_command: new command op to 5214 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:44.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.619+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/3253806629 --> [v2:172.21.15.29:6834/1468158138,v1:172.21.15.29:6835/1468158138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cafa4e7ab0 con 0x55cafa757f90 2024-04-25T00:32:44.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d48814640 1 --2- 172.21.15.29:0/3253806629 >> [v2:172.21.15.29:6834/1468158138,v1:172.21.15.29:6835/1468158138] conn(0x55cafa757f90 0x55cafa75a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d237fe640 10 client.5430 ms_handle_connect on v2:172.21.15.29:6834/1468158138 2024-04-25T00:32:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d48814640 1 --2- 172.21.15.29:0/3253806629 >> [v2:172.21.15.29:6834/1468158138,v1:172.21.15.29:6835/1468158138] conn(0x55cafa757f90 0x55cafa75a380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=524303a4123d5462 server_cookie=69c8aab8c35f7795 in_seq=0 out_seq=0 2024-04-25T00:32:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d237fe640 1 -- 172.21.15.29:0/3253806629 <== mds.0 v2:172.21.15.29:6834/1468158138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cafa4e7ab0 con 0x55cafa757f90 2024-04-25T00:32:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d237fe640 10 client.5430 handle_command_reply: tid=0 2024-04-25T00:32:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d4dbf1000 1 client.5430 shutdown 2024-04-25T00:32:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/3253806629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 msgr2=0x55cafa72d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/3253806629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 0x55cafa72d460 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1d30000c10 tx=0x7f1d30006ee0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/3253806629 shutdown_connections 2024-04-25T00:32:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/3253806629 >> [v2:172.21.15.29:6834/1468158138,v1:172.21.15.29:6835/1468158138] conn(0x55cafa757f90 0x55cafa75a380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/3253806629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa72d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/3253806629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 0x55cafa72d460 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d237fe640 0 client.5430 ms_handle_reset on v2:172.21.15.29:6834/1468158138 2024-04-25T00:32:44.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/3253806629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa6febc0 0x55cafa72cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/3253806629 >> 172.21.15.29:0/3253806629 conn(0x55cafa6fab70 msgr2=0x55cafa6fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:44.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/3253806629 shutdown_connections 2024-04-25T00:32:44.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/3253806629 wait complete. 2024-04-25T00:32:44.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.623+0000 7f1d4dbf1000 20 client.5430 trim_cache size 0 max 16384 2024-04-25T00:32:44.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.639+0000 7f1d4dbf1000 1 Processor -- start 2024-04-25T00:32:44.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.639+0000 7f1d4dbf1000 1 -- start start 2024-04-25T00:32:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 0x55cafa747090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa747ef0 0x55cafa7475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa758370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44202270 con 0x55cafa6febc0 2024-04-25T00:32:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441fb4d0 con 0x55cafa748810 2024-04-25T00:32:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1d44201c80 con 0x55cafa747ef0 2024-04-25T00:32:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d48814640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 0x55cafa747090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d48814640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 0x55cafa747090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54390/0 (socket says 172.21.15.29:54390) 2024-04-25T00:32:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d48814640 1 -- 172.21.15.29:0/3089866510 learned_addr learned my addr 172.21.15.29:0/3089866510 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d49015640 1 --2- 172.21.15.29:0/3089866510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa758370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d43fff640 1 --2- 172.21.15.29:0/3089866510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa747ef0 0x55cafa7475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d48814640 1 -- 172.21.15.29:0/3089866510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa747ef0 msgr2=0x55cafa7475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d48814640 1 --2- 172.21.15.29:0/3089866510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa747ef0 0x55cafa7475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d48814640 1 -- 172.21.15.29:0/3089866510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 msgr2=0x55cafa758370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d48814640 1 --2- 172.21.15.29:0/3089866510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa758370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d48814640 1 -- 172.21.15.29:0/3089866510 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cafa4d4e80 con 0x55cafa6febc0 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d49015640 1 --2- 172.21.15.29:0/3089866510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa758370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d43fff640 1 --2- 172.21.15.29:0/3089866510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa747ef0 0x55cafa7475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d48814640 1 --2- 172.21.15.29:0/3089866510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 0x55cafa747090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55cafa501d30 tx=0x7f1d38012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55d463607e6a46c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d2dffb640 1 -- 172.21.15.29:0/3089866510 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3802c070 con 0x55cafa6febc0 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d2dffb640 1 -- 172.21.15.29:0/3089866510 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d38005e30 con 0x55cafa6febc0 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d2dffb640 1 -- 172.21.15.29:0/3089866510 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800e960 con 0x55cafa6febc0 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/3089866510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 msgr2=0x55cafa747090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/3089866510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 0x55cafa747090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55cafa501d30 tx=0x7f1d38012000 comp rx=0 tx=0).stop 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/3089866510 shutdown_connections 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/3089866510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa748810 0x55cafa758370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/3089866510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa747ef0 0x55cafa7475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/3089866510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa6febc0 0x55cafa747090 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/3089866510 >> 172.21.15.29:0/3089866510 conn(0x55cafa6fab70 msgr2=0x55cafa7552c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:44.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/3089866510 shutdown_connections 2024-04-25T00:32:44.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/3089866510 wait complete. 2024-04-25T00:32:44.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 -- start start 2024-04-25T00:32:44.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 Processor -- start 2024-04-25T00:32:44.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa6febc0 0x55cafa7b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 0x55cafa7b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa748810 0x55cafa75aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cafa515810 con 0x55cafa747ef0 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cafa506520 con 0x55cafa6febc0 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55cafa558180 con 0x55cafa748810 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d4dbf1000 10 client.0 _ll_register_callbacks cb 0x7f1d300abec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d43fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 0x55cafa7b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d43fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 0x55cafa7b90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54404/0 (socket says 172.21.15.29:54404) 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d43fff640 1 -- 172.21.15.29:0/1730381272 learned_addr learned my addr 172.21.15.29:0/1730381272 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d48814640 1 --2- 172.21.15.29:0/1730381272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa6febc0 0x55cafa7b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.643+0000 7f1d49015640 1 --2- 172.21.15.29:0/1730381272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa748810 0x55cafa75aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d43fff640 1 -- 172.21.15.29:0/1730381272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa748810 msgr2=0x55cafa75aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d43fff640 1 --2- 172.21.15.29:0/1730381272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa748810 0x55cafa75aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d43fff640 1 -- 172.21.15.29:0/1730381272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa6febc0 msgr2=0x55cafa7b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d43fff640 1 --2- 172.21.15.29:0/1730381272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa6febc0 0x55cafa7b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d43fff640 1 -- 172.21.15.29:0/1730381272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cafa519760 con 0x55cafa747ef0 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d43fff640 1 --2- 172.21.15.29:0/1730381272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 0x55cafa7b90d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1d300b8fd0 tx=0x7f1d3011e450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240ff86b18bd3838 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d237fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d237fe640 1 -- 172.21.15.29:0/1730381272 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d300efdf0 con 0x55cafa747ef0 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d237fe640 1 -- 172.21.15.29:0/1730381272 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1d300f8be0 con 0x55cafa747ef0 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/1730381272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d38005e30 con 0x55cafa747ef0 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d237fe640 1 -- 172.21.15.29:0/1730381272 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f1d300ff050 con 0x55cafa747ef0 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 10 client.5436 fetch_fsmap learned FSMap version 15 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/1730381272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55cafa5a59e0 con 0x55cafa747ef0 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d237fe640 1 -- 172.21.15.29:0/1730381272 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d38005e30 con 0x55cafa747ef0 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d237fe640 1 -- 172.21.15.29:0/1730381272 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f1d30102020 con 0x55cafa747ef0 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 10 client.5436 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 10 client.5436 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 20 client.5436 populate_metadata read hostname 'smithi029' 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/1730381272 >> [v2:172.21.15.29:6834/1468158138,v1:172.21.15.29:6835/1468158138] conn(0x55cafa75b420 0x55cafa75b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 4 client.5436 mds_command: new command op to 5214 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/1730381272 --> [v2:172.21.15.29:6834/1468158138,v1:172.21.15.29:6835/1468158138] -- command(tid 0: {"prefix": "status"}) -- 0x55cafa559b20 con 0x55cafa75b420 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d48814640 1 --2- 172.21.15.29:0/1730381272 >> [v2:172.21.15.29:6834/1468158138,v1:172.21.15.29:6835/1468158138] conn(0x55cafa75b420 0x55cafa75b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d237fe640 10 client.5436 ms_handle_connect on v2:172.21.15.29:6834/1468158138 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d48814640 1 --2- 172.21.15.29:0/1730381272 >> [v2:172.21.15.29:6834/1468158138,v1:172.21.15.29:6835/1468158138] conn(0x55cafa75b420 0x55cafa75b800 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c4e788a167693b server_cookie=3969a84b29aa0684 in_seq=0 out_seq=0 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d237fe640 1 -- 172.21.15.29:0/1730381272 <== mds.0 v2:172.21.15.29:6834/1468158138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55cafa559b20 con 0x55cafa75b420 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d237fe640 10 client.5436 handle_command_reply: tid=0 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 client.5436 shutdown 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/1730381272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 msgr2=0x55cafa7b90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/1730381272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 0x55cafa7b90d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1d300b8fd0 tx=0x7f1d3011e450 comp rx=0 tx=0).stop 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/1730381272 shutdown_connections 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/1730381272 >> [v2:172.21.15.29:6834/1468158138,v1:172.21.15.29:6835/1468158138] conn(0x55cafa75b420 0x55cafa75b800 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/1730381272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cafa748810 0x55cafa75aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/1730381272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cafa747ef0 0x55cafa7b90d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d237fe640 0 client.5436 ms_handle_reset on v2:172.21.15.29:6834/1468158138 2024-04-25T00:32:44.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 --2- 172.21.15.29:0/1730381272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cafa6febc0 0x55cafa7b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/1730381272 >> 172.21.15.29:0/1730381272 conn(0x55cafa6fab70 msgr2=0x55cafa747090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:44.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/1730381272 shutdown_connections 2024-04-25T00:32:44.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 1 -- 172.21.15.29:0/1730381272 wait complete. 2024-04-25T00:32:44.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4dbf1000 20 client.5436 trim_cache size 0 max 16384 2024-04-25T00:32:44.657 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:32:44.657 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "59afbe7d-aaa8-45e2-b468-69ee150cacea", 2024-04-25T00:32:44.657 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-04-25T00:32:44.657 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 5214, 2024-04-25T00:32:44.658 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-04-25T00:32:44.658 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-04-25T00:32:44.658 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-04-25T00:32:44.658 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 10.223992494000001, 2024-04-25T00:32:44.658 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 15, 2024-04-25T00:32:44.658 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 33, 2024-04-25T00:32:44.658 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 33, 2024-04-25T00:32:44.658 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 17.161740813000002 2024-04-25T00:32:44.658 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4aa78640 1 -- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d280678c0 msgr2=0x7f1d28069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4aa78640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d280678c0 0x7f1d28069d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007b70 tx=0x7f1d38009040 comp rx=0 tx=0).stop 2024-04-25T00:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4aa78640 1 -- 172.21.15.29:0/2058761307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d440cc240 msgr2=0x7f1d4415a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4aa78640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d440cc240 0x7f1d4415a240 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1d3001aa70 tx=0x7f1d3001ac00 comp rx=0 tx=0).stop 2024-04-25T00:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4aa78640 1 -- 172.21.15.29:0/2058761307 shutdown_connections 2024-04-25T00:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4aa78640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d4415e8d0 0x7f1d4415a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4aa78640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d440cc240 0x7f1d4415a240 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4aa78640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d280678c0 0x7f1d28069d80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4aa78640 1 --2- 172.21.15.29:0/2058761307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d440b4780 0x7f1d44143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4aa78640 1 -- 172.21.15.29:0/2058761307 >> 172.21.15.29:0/2058761307 conn(0x7f1d440b1870 msgr2=0x7f1d440cfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4aa78640 1 -- 172.21.15.29:0/2058761307 shutdown_connections 2024-04-25T00:32:44.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.647+0000 7f1d4aa78640 1 -- 172.21.15.29:0/2058761307 wait complete. 2024-04-25T00:32:44.668 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:32:44.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.775+0000 7f4bb8159640 1 Processor -- start 2024-04-25T00:32:44.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb8159640 1 -- start start 2024-04-25T00:32:44.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb8159640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4bb006dce0 0x7f4bb0150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb8159640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb00b3130 con 0x7f4bb015d980 2024-04-25T00:32:44.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb8159640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb00b3410 con 0x7f4bb00b2b70 2024-04-25T00:32:44.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb8159640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb00cf560 con 0x7f4bb006dce0 2024-04-25T00:32:44.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb56f4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4bb006dce0 0x7f4bb0150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb56f4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4bb006dce0 0x7f4bb0150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54418/0 (socket says 172.21.15.29:54418) 2024-04-25T00:32:44.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb56f4640 1 -- 172.21.15.29:0/2693082171 learned_addr learned my addr 172.21.15.29:0/2693082171 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:44.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb56f4640 1 -- 172.21.15.29:0/2693082171 >> v1:172.21.15.157:6789/0 conn(0x7f4bb00b2b70 legacy=0x7f4bb0151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb56f4640 1 -- 172.21.15.29:0/2693082171 >> v1:172.21.15.29:6789/0 conn(0x7f4bb015d980 legacy=0x7f4bb006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb56f4640 1 -- 172.21.15.29:0/2693082171 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb00b36f0 con 0x7f4bb006dce0 2024-04-25T00:32:44.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb4ef3640 1 -- 172.21.15.29:0/2693082171 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2119180744 0 0) 0x7f4bb00b3130 con 0x7f4bb015d980 2024-04-25T00:32:44.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb56f4640 1 --2- 172.21.15.29:0/2693082171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb006dce0 0x7f4bb0150df0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009870 tx=0x7f4bac01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d96d34ebbbae1d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb4ef3640 1 -- 172.21.15.29:0/2693082171 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac0b0070 con 0x7f4bb006dce0 2024-04-25T00:32:44.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb4ef3640 1 -- 172.21.15.29:0/2693082171 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bac0ab600 con 0x7f4bb006dce0 2024-04-25T00:32:44.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.779+0000 7f4bb4ef3640 1 -- 172.21.15.29:0/2693082171 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac0ab9d0 con 0x7f4bb006dce0 2024-04-25T00:32:44.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 -- 172.21.15.29:0/2693082171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb006dce0 msgr2=0x7f4bb0150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 --2- 172.21.15.29:0/2693082171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb006dce0 0x7f4bb0150df0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009870 tx=0x7f4bac01c560 comp rx=0 tx=0).stop 2024-04-25T00:32:44.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 -- 172.21.15.29:0/2693082171 shutdown_connections 2024-04-25T00:32:44.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 --2- 172.21.15.29:0/2693082171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb006dce0 0x7f4bb0150df0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 -- 172.21.15.29:0/2693082171 >> 172.21.15.29:0/2693082171 conn(0x7f4bb00b1870 msgr2=0x7f4bb00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:44.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 -- 172.21.15.29:0/2693082171 shutdown_connections 2024-04-25T00:32:44.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 -- 172.21.15.29:0/2693082171 wait complete. 2024-04-25T00:32:44.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 Processor -- start 2024-04-25T00:32:44.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 -- start start 2024-04-25T00:32:44.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bb006dce0 0x7f4bb014a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb00b2b70 0x7f4bb00b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb015d980 0x7f4bb00cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb0167550 con 0x7f4bb015d980 2024-04-25T00:32:44.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb01673d0 con 0x7f4bb00b2b70 2024-04-25T00:32:44.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb01676d0 con 0x7f4bb006dce0 2024-04-25T00:32:44.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb66f6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb015d980 0x7f4bb00cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb66f6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb015d980 0x7f4bb00cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54420/0 (socket says 172.21.15.29:54420) 2024-04-25T00:32:44.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb66f6640 1 -- 172.21.15.29:0/1089198332 learned_addr learned my addr 172.21.15.29:0/1089198332 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:44.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb56f4640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb00b2b70 0x7f4bb00b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb5ef5640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bb006dce0 0x7f4bb014a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb66f6640 1 -- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bb006dce0 msgr2=0x7f4bb014a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb66f6640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bb006dce0 0x7f4bb014a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb66f6640 1 -- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb00b2b70 msgr2=0x7f4bb00b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb66f6640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb00b2b70 0x7f4bb00b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb66f6640 1 -- 172.21.15.29:0/1089198332 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb00afc80 con 0x7f4bb015d980 2024-04-25T00:32:44.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb5ef5640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bb006dce0 0x7f4bb014a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:44.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb66f6640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb015d980 0x7f4bb00cdd70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800c770 tx=0x7f4ba800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4ba67fc640 1 -- 172.21.15.29:0/1089198332 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8003b80 con 0x7f4bb015d980 2024-04-25T00:32:44.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4ba67fc640 1 -- 172.21.15.29:0/1089198332 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba8003d20 con 0x7f4bb015d980 2024-04-25T00:32:44.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 -- 172.21.15.29:0/1089198332 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb013a880 con 0x7f4bb015d980 2024-04-25T00:32:44.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4ba67fc640 1 -- 172.21.15.29:0/1089198332 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba800aa90 con 0x7f4bb015d980 2024-04-25T00:32:44.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.783+0000 7f4bb8159640 1 -- 172.21.15.29:0/1089198332 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb013b120 con 0x7f4bb015d980 2024-04-25T00:32:44.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.787+0000 7f4bb8159640 1 -- 172.21.15.29:0/1089198332 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb0069270 con 0x7f4bb015d980 2024-04-25T00:32:44.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.787+0000 7f4ba67fc640 1 -- 172.21.15.29:0/1089198332 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4ba800ac30 con 0x7f4bb015d980 2024-04-25T00:32:44.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.787+0000 7f4ba67fc640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b940678c0 0x7f4b94069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.787+0000 7f4ba67fc640 1 -- 172.21.15.29:0/1089198332 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f4ba8085c10 con 0x7f4bb015d980 2024-04-25T00:32:44.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.787+0000 7f4bb5ef5640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b940678c0 0x7f4b94069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.787+0000 7f4bb5ef5640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b940678c0 0x7f4b94069d80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0000f70 tx=0x7f4ba0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.791+0000 7f4ba67fc640 1 -- 172.21.15.29:0/1089198332 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4ba8056750 con 0x7f4bb015d980 2024-04-25T00:32:44.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4bb8159640 1 -- 172.21.15.29:0/1089198332 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4bb01671b0 con 0x7f4bb015d980 2024-04-25T00:32:44.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4ba67fc640 1 -- 172.21.15.29:0/1089198332 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7f4ba8056050 con 0x7f4bb015d980 2024-04-25T00:32:44.994 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:32:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4bb8159640 1 -- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b940678c0 msgr2=0x7f4b94069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4bb8159640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b940678c0 0x7f4b94069d80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0000f70 tx=0x7f4ba0009040 comp rx=0 tx=0).stop 2024-04-25T00:32:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4bb8159640 1 -- 172.21.15.29:0/1089198332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb015d980 msgr2=0x7f4bb00cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4bb8159640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb015d980 0x7f4bb00cdd70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800c770 tx=0x7f4ba800cc30 comp rx=0 tx=0).stop 2024-04-25T00:32:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4bb8159640 1 -- 172.21.15.29:0/1089198332 shutdown_connections 2024-04-25T00:32:44.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4bb8159640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb015d980 0x7f4bb00cdd70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4bb8159640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b940678c0 0x7f4b94069d80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4bb8159640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb00b2b70 0x7f4bb00b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4bb8159640 1 --2- 172.21.15.29:0/1089198332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bb006dce0 0x7f4bb014a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4bb8159640 1 -- 172.21.15.29:0/1089198332 >> 172.21.15.29:0/1089198332 conn(0x7f4bb00b1870 msgr2=0x7f4bb006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:44.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4bb8159640 1 -- 172.21.15.29:0/1089198332 shutdown_connections 2024-04-25T00:32:44.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:44.991+0000 7f4bb8159640 1 -- 172.21.15.29:0/1089198332 wait complete. 2024-04-25T00:32:45.012 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:32:45.012 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:32:45.012 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-25T00:32:45.012 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:32:45.012 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:32:45.013 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:32:45.013 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:32:45.013 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:32:45.013 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:32:45.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:45.013 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-25T00:32:45.019 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:32:45.019 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:32:45.019 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:32:45.019 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:45.019 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:32:45.019 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:45.019 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:32:45.020 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:32:45.020 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-25T00:32:45.020 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:32:45.020 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:32:45.020 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-25T00:32:45.020 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:32:45.020 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:32:45.020 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-25T00:32:45.020 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:32:45.020 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42395sec preferred_lft 42395sec 2024-04-25T00:32:45.020 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-04-25T00:32:45.020 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:45.020 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:32:45.021 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:32:45.021 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-25T00:32:45.021 INFO:teuthology.orchestra.run.smithi029.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:32:45.021 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 02:42:28:a5:36:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:32:45.021 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:32:45.021 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:45.021 INFO:teuthology.orchestra.run.smithi029.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:32:45.021 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:32:45.021 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:32:45.021 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:45.021 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:32:45.021 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:45.021 INFO:teuthology.orchestra.run.smithi029.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:32:45.022 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:32:45.022 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:32:45.022 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:45.022 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:32:45.022 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:32:45.022 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:32:45.022 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:45.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:45.022 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:32:45.070 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-25T00:32:45.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:45.072 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-25T00:32:45.122 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-25T00:32:45.123 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:32:45.123 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T00:32:45.123 INFO:teuthology.orchestra.run.smithi029.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:32:45.123 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T00:32:45.123 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:32:45.123 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:32:45.123 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:32:45.123 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:32:45.124 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:32:45.124 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:32:45.124 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:32:45.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:32:45.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:32:45.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:45.254 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:32:45.260 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:32:45.261 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:32:45.261 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:32:45.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:45.261 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-04-25T00:32:45.319 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:32:45.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:32:45.319 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:32:45.366 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:32:45.367 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:32:45.367 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:32:45.413 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:32:45.471 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:32:45.524 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:32:45.524 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:32:45.524 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:32:45.524 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:32:45.524 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:32:45.524 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:32:45.524 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:32:45.524 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:32:45.524 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:32:45.524 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:32:45.524 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:32:45.524 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:32:45.525 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:32:45.526 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:32:45.526 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:32:45.526 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:32:45.526 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:32:45.526 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:32:45.526 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:32:45.526 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:32:45.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:45.526 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:32:45.570 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:32:45.570 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:32:45.615 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:32:45.628 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:32:45.679 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:32:45.679 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:32:45.679 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:32:45.679 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:32:45.679 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:32:45.679 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:32:45.679 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:32:45.679 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:32:45.679 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:32:45.680 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:32:45.681 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:32:45.681 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:32:45.681 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:32:45.681 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:32:45.681 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:32:45.681 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:32:45.681 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:32:45.681 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:32:45.681 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:32:45.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:45.681 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:32:45.699 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:2024-04-25T00:32:45.695+0000 7fc362b49580 -1 init, newargv = 0x56510eadf2e0 newargc=15 2024-04-25T00:32:45.699 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[37598]: starting ceph client 2024-04-25T00:32:45.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[37598]: starting fuse 2024-04-25T00:32:46.805 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:32:46.820 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:32:46.871 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:32:46.871 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:32:46.871 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:32:46.871 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:32:46.871 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:32:46.871 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:32:46.871 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:32:46.871 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:32:46.872 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:32:46.873 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:32:46.873 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:32:46.873 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:32:46.873 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:32:46.873 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:32:46.873 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:32:46.873 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:32:46.873 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:32:46.873 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:32:46.873 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:32:46.873 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:32:46.873 INFO:teuthology.orchestra.run.smithi029.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:32:46.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:46.874 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:32:46.926 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:32:46.926 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:32:46.926 DEBUG:teuthology.orchestra.run.smithi029:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:32:46.927 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-04-25T00:32:46.928 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:32:46.928 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:32:46.928 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:32:46.928 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-04-25T00:32:46.928 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-04-25T00:32:46.928 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-04-25T00:32:46.928 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:32:46.928 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:32:46.928 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.0")) 2024-04-25T00:32:46.928 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-04-25T00:32:47.054 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.0.37598.asok 2024-04-25T00:32:47.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:32:47.067 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37598.asok 2024-04-25T00:32:47.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:47.067 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37598.asok status 2024-04-25T00:32:47.227 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:32:47.227 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-04-25T00:32:47.228 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:32:47.228 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:32:47.228 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "0", 2024-04-25T00:32:47.228 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-04-25T00:32:47.228 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:32:47.228 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "37598", 2024-04-25T00:32:47.228 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-04-25T00:32:47.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:47.228 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-04-25T00:32:47.228 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-04-25T00:32:47.228 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 5451, 2024-04-25T00:32:47.228 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 5451 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 265205701 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:32:47.229 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 265205701 2024-04-25T00:32:47.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:32:47.230 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.5451 192.168.0.1:0/265205701", 2024-04-25T00:32:47.230 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.1:0/265205701", 2024-04-25T00:32:47.230 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-04-25T00:32:47.230 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 15, 2024-04-25T00:32:47.230 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 33, 2024-04-25T00:32:47.230 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:32:47.230 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-04-25T00:32:47.230 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-04-25T00:32:47.230 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:32:47.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:47.241 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-04-25T00:32:47.241 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:32:47.248 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-04-25T00:32:47.248 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:32:47.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:47.248 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:32:47.301 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:32:47.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdba7112640 1 Processor -- start 2024-04-25T00:32:47.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdba7112640 1 -- start start 2024-04-25T00:32:47.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdba7112640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdba015c190 0x7fdba00d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:47.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdba7112640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdba00cd3e0 con 0x7fdba00ccf30 2024-04-25T00:32:47.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdba7112640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdba00cd6c0 con 0x7fdba015bdb0 2024-04-25T00:32:47.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdba7112640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdba00cd9a0 con 0x7fdba015c190 2024-04-25T00:32:47.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdba56af640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdba00ccf30 0x7fdba00c5860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38366/0 (socket says 172.21.15.29:38366) 2024-04-25T00:32:47.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdba56af640 1 -- 172.21.15.29:0/13142176 learned_addr learned my addr 172.21.15.29:0/13142176 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:47.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9ffff640 1 --2- 172.21.15.29:0/13142176 >> v2:172.21.15.157:3300/0 conn(0x7fdba015c190 0x7fdba00d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:47.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3122788805 0 0) 0x7fdba00cd3e0 con 0x7fdba00ccf30 2024-04-25T00:32:47.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb90003530 con 0x7fdba00ccf30 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2672638227 0 0) 0x7fdb90003530 con 0x7fdba00ccf30 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 >> v2:172.21.15.157:3300/0 conn(0x7fdba015c190 msgr2=0x7fdba00d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 --2- 172.21.15.29:0/13142176 >> v2:172.21.15.157:3300/0 conn(0x7fdba015c190 0x7fdba00d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 >> v1:172.21.15.157:6789/0 conn(0x7fdba015bdb0 legacy=0x7fdba00cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba00cdc80 con 0x7fdba00ccf30 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fdb98001a00 con 0x7fdba00ccf30 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 >> v1:172.21.15.29:6789/0 conn(0x7fdba00ccf30 legacy=0x7fdba00c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 --2- 172.21.15.29:0/13142176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb900039c0 0x7fdb90003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 --2- 172.21.15.29:0/13142176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb90004810 0x7fdb90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 --2- 172.21.15.29:0/13142176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb900051d0 0x7fdb9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdba00cd9a0 con 0x7fdb90004810 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9000c170 con 0x7fdb900051d0 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9000c370 con 0x7fdb900039c0 2024-04-25T00:32:47.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9ffff640 1 --2- 172.21.15.29:0/13142176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb90004810 0x7fdb90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:47.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdba56af640 1 --2- 172.21.15.29:0/13142176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb900051d0 0x7fdb9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:47.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdba4eae640 1 --2- 172.21.15.29:0/13142176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb900039c0 0x7fdb90003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:47.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9ffff640 1 -- 172.21.15.29:0/13142176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb900039c0 msgr2=0x7fdb90003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:47.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9ffff640 1 --2- 172.21.15.29:0/13142176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb900039c0 0x7fdb90003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:47.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9ffff640 1 -- 172.21.15.29:0/13142176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb900051d0 msgr2=0x7fdb9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:47.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9ffff640 1 --2- 172.21.15.29:0/13142176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb900051d0 0x7fdb9000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:47.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdb9ffff640 1 -- 172.21.15.29:0/13142176 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb90003530 con 0x7fdb90004810 2024-04-25T00:32:47.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.471+0000 7fdba56af640 1 -- 172.21.15.29:0/13142176 reap_dead start 2024-04-25T00:32:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.475+0000 7fdb9ffff640 1 --2- 172.21.15.29:0/13142176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb90004810 0x7fdb90004c90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009ab0 tx=0x7fdb9401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1d55cfa167c3e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.475+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940b2070 con 0x7fdb90004810 2024-04-25T00:32:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.475+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb940a55b0 con 0x7fdb90004810 2024-04-25T00:32:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.475+0000 7fdb9f7fe640 1 -- 172.21.15.29:0/13142176 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940096b0 con 0x7fdb90004810 2024-04-25T00:32:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.475+0000 7fdba7112640 1 -- 172.21.15.29:0/13142176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb90004810 msgr2=0x7fdb90004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.475+0000 7fdba7112640 1 --2- 172.21.15.29:0/13142176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb90004810 0x7fdb90004c90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009ab0 tx=0x7fdb9401ca20 comp rx=0 tx=0).stop 2024-04-25T00:32:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.475+0000 7fdba7112640 1 -- 172.21.15.29:0/13142176 shutdown_connections 2024-04-25T00:32:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.475+0000 7fdba7112640 1 --2- 172.21.15.29:0/13142176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb90004810 0x7fdb90004c90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.475+0000 7fdba7112640 1 -- 172.21.15.29:0/13142176 >> 172.21.15.29:0/13142176 conn(0x7fdba009aa30 msgr2=0x7fdba009ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.475+0000 7fdba7112640 1 -- 172.21.15.29:0/13142176 shutdown_connections 2024-04-25T00:32:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.475+0000 7fdba7112640 1 -- 172.21.15.29:0/13142176 wait complete. 2024-04-25T00:32:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.475+0000 7fdba7112640 1 Processor -- start 2024-04-25T00:32:47.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.475+0000 7fdba7112640 1 -- start start 2024-04-25T00:32:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba7112640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb90004810 0x7fdba00aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:47.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba7112640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba01561e0 0x7fdba015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba7112640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdba00cb0c0 0x7fdba00cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba7112640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdba00d1b40 con 0x7fdb90004810 2024-04-25T00:32:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba7112640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdba00d19c0 con 0x7fdba00cb0c0 2024-04-25T00:32:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba7112640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdba00d1cc0 con 0x7fdba01561e0 2024-04-25T00:32:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba4eae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb90004810 0x7fdba00aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba4eae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb90004810 0x7fdba00aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54444/0 (socket says 172.21.15.29:54444) 2024-04-25T00:32:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba4eae640 1 -- 172.21.15.29:0/2953259266 learned_addr learned my addr 172.21.15.29:0/2953259266 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:47.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdb9ffff640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba01561e0 0x7fdba015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba56af640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdba00cb0c0 0x7fdba00cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba4eae640 1 -- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba01561e0 msgr2=0x7fdba015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba4eae640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba01561e0 0x7fdba015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba4eae640 1 -- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdba00cb0c0 msgr2=0x7fdba00cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba4eae640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdba00cb0c0 0x7fdba00cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba4eae640 1 -- 172.21.15.29:0/2953259266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba0156660 con 0x7fdb90004810 2024-04-25T00:32:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdb9ffff640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba01561e0 0x7fdba015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba4eae640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb90004810 0x7fdba00aeb40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00e530 tx=0x7fdb8c00e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdb9d7fa640 1 -- 172.21.15.29:0/2953259266 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0053a0 con 0x7fdb90004810 2024-04-25T00:32:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdb9d7fa640 1 -- 172.21.15.29:0/2953259266 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb8c018070 con 0x7fdb90004810 2024-04-25T00:32:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba7112640 1 -- 172.21.15.29:0/2953259266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba015dd40 con 0x7fdb90004810 2024-04-25T00:32:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdb9d7fa640 1 -- 172.21.15.29:0/2953259266 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c003820 con 0x7fdb90004810 2024-04-25T00:32:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba7112640 1 -- 172.21.15.29:0/2953259266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba015e300 con 0x7fdb90004810 2024-04-25T00:32:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.479+0000 7fdba7112640 1 -- 172.21.15.29:0/2953259266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba0069270 con 0x7fdb90004810 2024-04-25T00:32:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.483+0000 7fdb9d7fa640 1 -- 172.21.15.29:0/2953259266 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdb8c0039c0 con 0x7fdb90004810 2024-04-25T00:32:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.483+0000 7fdb9d7fa640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdb840678c0 0x7fdb84069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:47.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.483+0000 7fdb9d7fa640 1 -- 172.21.15.29:0/2953259266 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdb8c086350 con 0x7fdb90004810 2024-04-25T00:32:47.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.483+0000 7fdb9ffff640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdb840678c0 0x7fdb84069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:47.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.483+0000 7fdb9ffff640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdb840678c0 0x7fdb84069d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdb940af000 tx=0x7fdb94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:47.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.487+0000 7fdb9d7fa640 1 -- 172.21.15.29:0/2953259266 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdb8c056f70 con 0x7fdb90004810 2024-04-25T00:32:47.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.687+0000 7fdba7112640 1 -- 172.21.15.29:0/2953259266 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdba009b470 con 0x7fdb90004810 2024-04-25T00:32:47.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.687+0000 7fdb9d7fa640 1 -- 172.21.15.29:0/2953259266 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7fdb8c056870 con 0x7fdb90004810 2024-04-25T00:32:47.691 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:32:47.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.687+0000 7fdba7112640 1 -- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdb840678c0 msgr2=0x7fdb84069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:47.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.687+0000 7fdba7112640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdb840678c0 0x7fdb84069d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdb940af000 tx=0x7fdb94008040 comp rx=0 tx=0).stop 2024-04-25T00:32:47.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.687+0000 7fdba7112640 1 -- 172.21.15.29:0/2953259266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb90004810 msgr2=0x7fdba00aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:47.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.687+0000 7fdba7112640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb90004810 0x7fdba00aeb40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00e530 tx=0x7fdb8c00e9f0 comp rx=0 tx=0).stop 2024-04-25T00:32:47.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.687+0000 7fdba7112640 1 -- 172.21.15.29:0/2953259266 shutdown_connections 2024-04-25T00:32:47.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.687+0000 7fdba7112640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdb840678c0 0x7fdb84069d80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:47.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.687+0000 7fdba7112640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdba00cb0c0 0x7fdba00cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:47.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.691+0000 7fdba7112640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdba01561e0 0x7fdba015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:47.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.691+0000 7fdba7112640 1 --2- 172.21.15.29:0/2953259266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb90004810 0x7fdba00aeb40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:47.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.691+0000 7fdba7112640 1 -- 172.21.15.29:0/2953259266 >> 172.21.15.29:0/2953259266 conn(0x7fdba009aa30 msgr2=0x7fdba0070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:47.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.691+0000 7fdba7112640 1 -- 172.21.15.29:0/2953259266 shutdown_connections 2024-04-25T00:32:47.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:47.691+0000 7fdba7112640 1 -- 172.21.15.29:0/2953259266 wait complete. 2024-04-25T00:32:47.710 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:32:47.711 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:32:47.711 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-25T00:32:47.711 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:32:47.711 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:32:47.711 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:32:47.711 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:32:47.711 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:32:47.712 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:32:47.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:47.712 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-25T00:32:47.719 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:32:47.719 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:32:47.719 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:32:47.719 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:47.719 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:32:47.719 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:47.719 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:32:47.719 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:32:47.719 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-25T00:32:47.719 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42387sec preferred_lft 42387sec 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:9c:8d:4b:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:47.720 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:32:47.721 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:32:47.721 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:32:47.721 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:47.721 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:32:47.721 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:47.721 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:32:47.721 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:32:47.721 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:32:47.721 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:47.721 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:32:47.721 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:32:47.721 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:32:47.721 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:32:47.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:47.722 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:32:47.772 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:32:47.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:47.774 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-25T00:32:47.829 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-25T00:32:47.829 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:32:47.829 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:32:47.829 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:32:47.829 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:32:47.830 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:32:47.830 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:32:47.830 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:32:47.830 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:32:47.830 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:32:47.830 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:32:47.830 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:32:47.830 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:32:47.830 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:32:47.830 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:32:47.831 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:32:47.831 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:32:47.918 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:32:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:32:47.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:32:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:32:47.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:47.954 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:32:47.960 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:32:47.960 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:32:47.960 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:32:47.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:47.960 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-25T00:32:48.017 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T00:32:48.017 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:32:48.017 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:32:48.064 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:32:48.065 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:32:48.065 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:32:48.113 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:32:48.169 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:32:48.219 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:32:48.219 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:32:48.219 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:32:48.219 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:32:48.219 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:32:48.219 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:32:48.219 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:32:48.219 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:32:48.219 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:32:48.219 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:32:48.219 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:32:48.219 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:32:48.219 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:32:48.220 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:32:48.221 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:32:48.221 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:32:48.221 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:32:48.221 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:32:48.221 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:32:48.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:48.221 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:32:48.265 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:32:48.265 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T00:32:48.310 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:32:48.325 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:32:48.373 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:32:48.374 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:32:48.374 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:32:48.374 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:32:48.374 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:32:48.374 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:32:48.374 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:32:48.374 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:32:48.374 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:32:48.374 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:32:48.374 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:32:48.375 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:32:48.375 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:32:48.375 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:32:48.375 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:32:48.375 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:32:48.375 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:32:48.375 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:32:48.375 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:32:48.375 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:32:48.375 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:32:48.375 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:32:48.375 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:32:48.376 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:32:48.376 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:32:48.376 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:32:48.376 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:32:48.376 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:32:48.376 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:32:48.376 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:32:48.376 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:32:48.376 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:32:48.376 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:32:48.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:48.377 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:32:48.392 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-04-25T00:32:48.391+0000 7f680a4a7580 -1 init, newargv = 0x558f67887f20 newargc=15 2024-04-25T00:32:48.392 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[33664]: starting ceph client 2024-04-25T00:32:48.400 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[33664]: starting fuse 2024-04-25T00:32:49.497 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:32:49.514 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:32:49.566 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:32:49.566 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:32:49.566 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:32:49.566 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:32:49.566 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:32:49.566 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:32:49.566 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:32:49.567 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:32:49.568 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:32:49.568 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:32:49.568 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:32:49.568 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:32:49.568 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:32:49.568 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:32:49.568 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:32:49.568 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:32:49.568 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:32:49.568 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:32:49.568 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:32:49.568 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:32:49.568 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:32:49.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:49.569 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:32:49.621 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:32:49.621 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:32:49.621 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:32:49.621 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-25T00:32:49.621 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-25T00:32:49.621 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-25T00:32:49.621 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-25T00:32:49.621 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:32:49.621 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-25T00:32:49.621 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T00:32:49.621 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:32:49.622 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:32:49.623 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-04-25T00:32:49.623 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-25T00:32:49.746 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.33664.asok 2024-04-25T00:32:49.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:32:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:32:49.758 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33664.asok 2024-04-25T00:32:49.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:49.758 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33664.asok status 2024-04-25T00:32:49.911 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:32:49.911 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-25T00:32:49.911 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:32:49.911 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:32:49.911 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-04-25T00:32:49.911 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "33664", 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4910, 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 4910 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:32:49.912 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:32:49.913 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:32:49.913 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3857704513 2024-04-25T00:32:49.913 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-25T00:32:49.913 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:32:49.913 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:32:49.913 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:32:49.913 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:32:49.913 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3857704513 2024-04-25T00:32:49.913 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:32:49.913 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.4910 192.168.0.1:0/3857704513", 2024-04-25T00:32:49.913 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/3857704513", 2024-04-25T00:32:49.913 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-25T00:32:49.914 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 15, 2024-04-25T00:32:49.914 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 33, 2024-04-25T00:32:49.914 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:32:49.914 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-25T00:32:49.914 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-25T00:32:49.914 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:32:49.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:49.926 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-25T00:32:49.926 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T00:32:49.932 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-25T00:32:49.933 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T00:32:49.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:49.933 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:32:49.988 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-25T00:32:50.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.123+0000 7f8b7fb78640 1 Processor -- start 2024-04-25T00:32:50.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.123+0000 7f8b7fb78640 1 -- start start 2024-04-25T00:32:50.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.123+0000 7f8b7fb78640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b781456d0 0x7f8b780b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.123+0000 7f8b7fb78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b780b3b50 0x7f8b780cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.123+0000 7f8b7fb78640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b780b4380 con 0x7f8b7806cd40 2024-04-25T00:32:50.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.123+0000 7f8b7fb78640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8b780b40a0 con 0x7f8b781456d0 2024-04-25T00:32:50.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.123+0000 7f8b7fb78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b780ce480 con 0x7f8b780b3b50 2024-04-25T00:32:50.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.127+0000 7f8b7e115640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8b7806cd40 0x7f8b7806d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:41640/0 (socket says 172.21.15.29:41640) 2024-04-25T00:32:50.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.127+0000 7f8b7d914640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b781456d0 0x7f8b780b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.127+0000 7f8b7e115640 1 -- 172.21.15.29:0/194218759 learned_addr learned my addr 172.21.15.29:0/194218759 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:50.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.127+0000 7f8b7d113640 1 --2- 172.21.15.29:0/194218759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b780b3b50 0x7f8b780cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.127+0000 7f8b7c912640 1 -- 172.21.15.29:0/194218759 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3642439708 0 0) 0x7f8b780b4380 con 0x7f8b7806cd40 2024-04-25T00:32:50.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.127+0000 7f8b7c912640 1 -- 172.21.15.29:0/194218759 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b64003530 con 0x7f8b7806cd40 2024-04-25T00:32:50.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.127+0000 7f8b7d914640 1 -- 172.21.15.29:0/194218759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b780b3b50 msgr2=0x7f8b780cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:50.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.127+0000 7f8b7d914640 1 --2- 172.21.15.29:0/194218759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b780b3b50 0x7f8b780cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.127+0000 7f8b7d914640 1 -- 172.21.15.29:0/194218759 >> v1:172.21.15.157:6789/0 conn(0x7f8b7806cd40 legacy=0x7f8b7806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:50.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.127+0000 7f8b7d914640 1 -- 172.21.15.29:0/194218759 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b780ce760 con 0x7f8b781456d0 2024-04-25T00:32:50.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.127+0000 7f8b7d113640 1 --2- 172.21.15.29:0/194218759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b780b3b50 0x7f8b780cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:50.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.131+0000 7f8b7d914640 1 --2- 172.21.15.29:0/194218759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b781456d0 0x7f8b780b2b70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0097c0 tx=0x7f8b6c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=702cc39c550b5c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:50.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.131+0000 7f8b7c912640 1 -- 172.21.15.29:0/194218759 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c0b0070 con 0x7f8b781456d0 2024-04-25T00:32:50.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.131+0000 7f8b7c912640 1 -- 172.21.15.29:0/194218759 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b6c0ab600 con 0x7f8b781456d0 2024-04-25T00:32:50.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.131+0000 7f8b7c912640 1 -- 172.21.15.29:0/194218759 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c0ab9d0 con 0x7f8b781456d0 2024-04-25T00:32:50.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.131+0000 7f8b7fb78640 1 -- 172.21.15.29:0/194218759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b781456d0 msgr2=0x7f8b780b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:50.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.131+0000 7f8b7fb78640 1 --2- 172.21.15.29:0/194218759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b781456d0 0x7f8b780b2b70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0097c0 tx=0x7f8b6c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.131+0000 7f8b7fb78640 1 -- 172.21.15.29:0/194218759 shutdown_connections 2024-04-25T00:32:50.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.131+0000 7f8b7fb78640 1 --2- 172.21.15.29:0/194218759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b780b3b50 0x7f8b780cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.131+0000 7f8b7fb78640 1 --2- 172.21.15.29:0/194218759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b781456d0 0x7f8b780b2b70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.131+0000 7f8b7fb78640 1 -- 172.21.15.29:0/194218759 >> 172.21.15.29:0/194218759 conn(0x7f8b780b1870 msgr2=0x7f8b780b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:50.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.131+0000 7f8b7fb78640 1 -- 172.21.15.29:0/194218759 shutdown_connections 2024-04-25T00:32:50.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.131+0000 7f8b7fb78640 1 -- 172.21.15.29:0/194218759 wait complete. 2024-04-25T00:32:50.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.131+0000 7f8b7fb78640 1 Processor -- start 2024-04-25T00:32:50.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7fb78640 1 -- start start 2024-04-25T00:32:50.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7fb78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7806cd40 0x7f8b78151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7fb78640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b780b3b50 0x7f8b781540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7fb78640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b781456d0 0x7f8b78152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7fb78640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8b780cf620 con 0x7f8b781456d0 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7fb78640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b780cf4a0 con 0x7f8b780b3b50 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7fb78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b78168b70 con 0x7f8b7806cd40 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7e115640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b781456d0 0x7f8b78152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7e115640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b781456d0 0x7f8b78152320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33562/0 (socket says 172.21.15.29:33562) 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7e115640 1 -- 172.21.15.29:0/461513978 learned_addr learned my addr 172.21.15.29:0/461513978 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7d113640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b780b3b50 0x7f8b781540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7d914640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7806cd40 0x7f8b78151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7e115640 1 -- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7806cd40 msgr2=0x7f8b78151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7e115640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7806cd40 0x7f8b78151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7e115640 1 -- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b780b3b50 msgr2=0x7f8b781540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7e115640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b780b3b50 0x7f8b781540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7e115640 1 -- 172.21.15.29:0/461513978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b78155d50 con 0x7f8b781456d0 2024-04-25T00:32:50.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7d914640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7806cd40 0x7f8b78151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:50.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7e115640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b781456d0 0x7f8b78152320 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000c3c0 tx=0x7f8b7000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:50.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/461513978 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70017070 con 0x7f8b781456d0 2024-04-25T00:32:50.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/461513978 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b70005cf0 con 0x7f8b781456d0 2024-04-25T00:32:50.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7fb78640 1 -- 172.21.15.29:0/461513978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7813be20 con 0x7f8b781456d0 2024-04-25T00:32:50.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/461513978 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70007310 con 0x7f8b781456d0 2024-04-25T00:32:50.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7fb78640 1 -- 172.21.15.29:0/461513978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b781562a0 con 0x7f8b781456d0 2024-04-25T00:32:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b7fb78640 1 -- 172.21.15.29:0/461513978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b780692e0 con 0x7f8b781456d0 2024-04-25T00:32:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.135+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/461513978 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8b7001a070 con 0x7f8b781456d0 2024-04-25T00:32:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.139+0000 7f8b6a7fc640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b5c0678c0 0x7f8b5c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.139+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/461513978 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8b70084e00 con 0x7f8b781456d0 2024-04-25T00:32:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.139+0000 7f8b7d914640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b5c0678c0 0x7f8b5c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.139+0000 7f8b7d914640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b5c0678c0 0x7f8b5c069d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0ad000 tx=0x7f8b6c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.143+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/461513978 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8b7813be20 con 0x7f8b781456d0 2024-04-25T00:32:50.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.339+0000 7f8b7fb78640 1 -- 172.21.15.29:0/461513978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f8b780b4de0 con 0x7f8b781456d0 2024-04-25T00:32:50.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.511+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/461513978 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v16) ==== 136+0+0 (secure 0 0 0) 0x7f8b70055380 con 0x7f8b781456d0 2024-04-25T00:32:50.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.515+0000 7f8b7fb78640 1 -- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b5c0678c0 msgr2=0x7f8b5c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:50.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.515+0000 7f8b7fb78640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b5c0678c0 0x7f8b5c069d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0ad000 tx=0x7f8b6c008040 comp rx=0 tx=0).stop 2024-04-25T00:32:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.515+0000 7f8b7fb78640 1 -- 172.21.15.29:0/461513978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b781456d0 msgr2=0x7f8b78152320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.515+0000 7f8b7fb78640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b781456d0 0x7f8b78152320 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000c3c0 tx=0x7f8b7000c880 comp rx=0 tx=0).stop 2024-04-25T00:32:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.515+0000 7f8b7fb78640 1 -- 172.21.15.29:0/461513978 shutdown_connections 2024-04-25T00:32:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.515+0000 7f8b7fb78640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b781456d0 0x7f8b78152320 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.515+0000 7f8b7fb78640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b5c0678c0 0x7f8b5c069d80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.515+0000 7f8b7fb78640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b780b3b50 0x7f8b781540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.515+0000 7f8b7fb78640 1 --2- 172.21.15.29:0/461513978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7806cd40 0x7f8b78151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.515+0000 7f8b7fb78640 1 -- 172.21.15.29:0/461513978 >> 172.21.15.29:0/461513978 conn(0x7f8b780b1870 msgr2=0x7f8b7806d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.515+0000 7f8b7fb78640 1 -- 172.21.15.29:0/461513978 shutdown_connections 2024-04-25T00:32:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.515+0000 7f8b7fb78640 1 -- 172.21.15.29:0/461513978 wait complete. 2024-04-25T00:32:50.542 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-25T00:32:50.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.667+0000 7f4b8fae1640 1 Processor -- start 2024-04-25T00:32:50.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.667+0000 7f4b8fae1640 1 -- start start 2024-04-25T00:32:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8fae1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b8815e2a0 0x7f4b880d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8fae1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b880cf8a0 con 0x7f4b880b4040 2024-04-25T00:32:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8fae1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b880cfb80 con 0x7f4b880d4d10 2024-04-25T00:32:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8fae1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b880d19a0 con 0x7f4b8815e2a0 2024-04-25T00:32:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8e07e640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4b880b4040 0x7f4b880b44f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50050/0 (socket says 172.21.15.29:50050) 2024-04-25T00:32:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8e07e640 1 -- 172.21.15.29:0/3302845690 learned_addr learned my addr 172.21.15.29:0/3302845690 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:50.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1636164576 0 0) 0x7f4b880cf8a0 con 0x7f4b880b4040 2024-04-25T00:32:50.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8d87d640 1 --2- 172.21.15.29:0/3302845690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b8815e2a0 0x7f4b880d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b74003530 con 0x7f4b880b4040 2024-04-25T00:32:50.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3216734026 0 0) 0x7f4b74003530 con 0x7f4b880b4040 2024-04-25T00:32:50.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b8815e2a0 msgr2=0x7f4b880d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:50.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 --2- 172.21.15.29:0/3302845690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b8815e2a0 0x7f4b880d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 >> v1:172.21.15.157:6789/0 conn(0x7f4b880d4d10 legacy=0x7f4b880d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b880cfe60 con 0x7f4b880b4040 2024-04-25T00:32:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4b80001a00 con 0x7f4b880b4040 2024-04-25T00:32:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 >> v1:172.21.15.29:6789/0 conn(0x7f4b880b4040 legacy=0x7f4b880b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 --2- 172.21.15.29:0/3302845690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b8815e2a0 0x7f4b740039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 --2- 172.21.15.29:0/3302845690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b74003f70 0x7f4b74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 --2- 172.21.15.29:0/3302845690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b74004d50 0x7f4b740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7400bb60 con 0x7f4b74003f70 2024-04-25T00:32:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7400bce0 con 0x7f4b74004d50 2024-04-25T00:32:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b7400bee0 con 0x7f4b8815e2a0 2024-04-25T00:32:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8d07c640 1 --2- 172.21.15.29:0/3302845690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b74003f70 0x7f4b74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8e07e640 1 --2- 172.21.15.29:0/3302845690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b74004d50 0x7f4b740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8d87d640 1 --2- 172.21.15.29:0/3302845690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b8815e2a0 0x7f4b740039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8d07c640 1 -- 172.21.15.29:0/3302845690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b8815e2a0 msgr2=0x7f4b740039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8d07c640 1 --2- 172.21.15.29:0/3302845690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b8815e2a0 0x7f4b740039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8d07c640 1 -- 172.21.15.29:0/3302845690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b74004d50 msgr2=0x7f4b740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8d07c640 1 --2- 172.21.15.29:0/3302845690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b74004d50 0x7f4b740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8d07c640 1 -- 172.21.15.29:0/3302845690 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b74003530 con 0x7f4b74003f70 2024-04-25T00:32:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.671+0000 7f4b8d07c640 1 --2- 172.21.15.29:0/3302845690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b74003f70 0x7f4b74004810 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4b78001750 tx=0x7f4b7801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694f407c7eea0dc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b780013a0 con 0x7f4b74003f70 2024-04-25T00:32:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b780052f0 con 0x7f4b74003f70 2024-04-25T00:32:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8c87b640 1 -- 172.21.15.29:0/3302845690 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78003840 con 0x7f4b74003f70 2024-04-25T00:32:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 -- 172.21.15.29:0/3302845690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b74003f70 msgr2=0x7f4b74004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 --2- 172.21.15.29:0/3302845690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b74003f70 0x7f4b74004810 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4b78001750 tx=0x7f4b7801c3e0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8e07e640 1 -- 172.21.15.29:0/3302845690 reap_dead start 2024-04-25T00:32:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 -- 172.21.15.29:0/3302845690 shutdown_connections 2024-04-25T00:32:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 -- 172.21.15.29:0/3302845690 >> 172.21.15.29:0/3302845690 conn(0x7f4b880d4030 msgr2=0x7f4b880d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:50.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 -- 172.21.15.29:0/3302845690 shutdown_connections 2024-04-25T00:32:50.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 -- 172.21.15.29:0/3302845690 wait complete. 2024-04-25T00:32:50.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 Processor -- start 2024-04-25T00:32:50.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 -- start start 2024-04-25T00:32:50.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b74003f70 0x7f4b880cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b880cc560 0x7f4b880b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b88153540 0x7f4b881583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4b880b3bb0 con 0x7f4b74003f70 2024-04-25T00:32:50.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b880b38d0 con 0x7f4b880cc560 2024-04-25T00:32:50.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b880d2a60 con 0x7f4b88153540 2024-04-25T00:32:50.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8d87d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b74003f70 0x7f4b880cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8d87d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b74003f70 0x7f4b880cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33578/0 (socket says 172.21.15.29:33578) 2024-04-25T00:32:50.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8d87d640 1 -- 172.21.15.29:0/947616445 learned_addr learned my addr 172.21.15.29:0/947616445 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:50.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8d07c640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b880cc560 0x7f4b880b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8e07e640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b88153540 0x7f4b881583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8d87d640 1 -- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b88153540 msgr2=0x7f4b881583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:50.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8d87d640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b88153540 0x7f4b881583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8d87d640 1 -- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b880cc560 msgr2=0x7f4b880b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:50.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8d87d640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b880cc560 0x7f4b880b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8d87d640 1 -- 172.21.15.29:0/947616445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b880ccb20 con 0x7f4b74003f70 2024-04-25T00:32:50.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8d87d640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b74003f70 0x7f4b880cdbc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4b84011a00 tx=0x7f4b84011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:50.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b7e7fc640 1 -- 172.21.15.29:0/947616445 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8400faf0 con 0x7f4b74003f70 2024-04-25T00:32:50.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b7e7fc640 1 -- 172.21.15.29:0/947616445 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b8400fc90 con 0x7f4b74003f70 2024-04-25T00:32:50.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 -- 172.21.15.29:0/947616445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b880cce00 con 0x7f4b74003f70 2024-04-25T00:32:50.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b7e7fc640 1 -- 172.21.15.29:0/947616445 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84014440 con 0x7f4b74003f70 2024-04-25T00:32:50.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.675+0000 7f4b8fae1640 1 -- 172.21.15.29:0/947616445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b880cd3c0 con 0x7f4b74003f70 2024-04-25T00:32:50.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.679+0000 7f4b8fae1640 1 -- 172.21.15.29:0/947616445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b88069270 con 0x7f4b74003f70 2024-04-25T00:32:50.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.679+0000 7f4b7e7fc640 1 -- 172.21.15.29:0/947616445 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4b84012040 con 0x7f4b74003f70 2024-04-25T00:32:50.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.679+0000 7f4b7e7fc640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b6c0678f0 0x7f4b6c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:50.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.679+0000 7f4b7e7fc640 1 -- 172.21.15.29:0/947616445 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f4b84086380 con 0x7f4b74003f70 2024-04-25T00:32:50.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.679+0000 7f4b8d07c640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b6c0678f0 0x7f4b6c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:50.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.679+0000 7f4b8d07c640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b6c0678f0 0x7f4b6c069db0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4b780ac000 tx=0x7f4b780011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:50.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.683+0000 7f4b7e7fc640 1 -- 172.21.15.29:0/947616445 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4b84019030 con 0x7f4b74003f70 2024-04-25T00:32:50.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.867+0000 7f4b8fae1640 1 -- 172.21.15.29:0/947616445 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f4b880d3e10 con 0x7f4b6c0678f0 2024-04-25T00:32:50.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.871+0000 7f4b7e7fc640 1 -- 172.21.15.29:0/947616445 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f4b880d3e10 con 0x7f4b6c0678f0 2024-04-25T00:32:50.874 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:32:50.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:32:50.874 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-04-25T00:32:50.874 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:32:50.874 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:32:50.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.871+0000 7f4b8fae1640 1 -- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b6c0678f0 msgr2=0x7f4b6c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:50.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.871+0000 7f4b8fae1640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b6c0678f0 0x7f4b6c069db0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4b780ac000 tx=0x7f4b780011a0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.871+0000 7f4b8fae1640 1 -- 172.21.15.29:0/947616445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b74003f70 msgr2=0x7f4b880cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:50.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.871+0000 7f4b8fae1640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b74003f70 0x7f4b880cdbc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4b84011a00 tx=0x7f4b84011ec0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.871+0000 7f4b8fae1640 1 -- 172.21.15.29:0/947616445 shutdown_connections 2024-04-25T00:32:50.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.871+0000 7f4b8fae1640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b88153540 0x7f4b881583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.871+0000 7f4b8fae1640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b6c0678f0 0x7f4b6c069db0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.871+0000 7f4b8fae1640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b880cc560 0x7f4b880b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.871+0000 7f4b8fae1640 1 --2- 172.21.15.29:0/947616445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b74003f70 0x7f4b880cdbc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:50.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.871+0000 7f4b8fae1640 1 -- 172.21.15.29:0/947616445 >> 172.21.15.29:0/947616445 conn(0x7f4b880d4030 msgr2=0x7f4b880b5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:50.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.871+0000 7f4b8fae1640 1 -- 172.21.15.29:0/947616445 shutdown_connections 2024-04-25T00:32:50.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:50.871+0000 7f4b8fae1640 1 -- 172.21.15.29:0/947616445 wait complete. 2024-04-25T00:32:50.893 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:32:51.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd324160640 1 Processor -- start 2024-04-25T00:32:51.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd324160640 1 -- start start 2024-04-25T00:32:51.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd324160640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c15d980 0x7fd31c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:51.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd324160640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd31c0b2b70 0x7fd31c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:51.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd324160640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31c0b3130 con 0x7fd31c150df0 2024-04-25T00:32:51.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd324160640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd31c0cf560 con 0x7fd31c0b2b70 2024-04-25T00:32:51.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd324160640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c151990 con 0x7fd31c15d980 2024-04-25T00:32:51.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd321efc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c15d980 0x7fd31c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:51.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd3216fb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd31c0b2b70 0x7fd31c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:51.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd321efc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c15d980 0x7fd31c06d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:43316/0 (socket says 172.21.15.29:43316) 2024-04-25T00:32:51.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd321efc640 1 -- 172.21.15.29:0/983901197 learned_addr learned my addr 172.21.15.29:0/983901197 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:51.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd321efc640 1 -- 172.21.15.29:0/983901197 >> v2:172.21.15.157:3300/0 conn(0x7fd31c0b2b70 msgr2=0x7fd31c06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:51.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd321efc640 1 --2- 172.21.15.29:0/983901197 >> v2:172.21.15.157:3300/0 conn(0x7fd31c0b2b70 0x7fd31c06db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd321efc640 1 -- 172.21.15.29:0/983901197 >> v1:172.21.15.157:6789/0 conn(0x7fd31c150df0 legacy=0x7fd31c151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:51.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.015+0000 7fd321efc640 1 -- 172.21.15.29:0/983901197 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c0b3410 con 0x7fd31c15d980 2024-04-25T00:32:51.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.019+0000 7fd321efc640 1 --2- 172.21.15.29:0/983901197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c15d980 0x7fd31c06d5d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd318009870 tx=0x7fd31801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c62f61d008c311e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:51.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.019+0000 7fd320efa640 1 -- 172.21.15.29:0/983901197 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3180aa430 con 0x7fd31c15d980 2024-04-25T00:32:51.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.019+0000 7fd320efa640 1 -- 172.21.15.29:0/983901197 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3180af070 con 0x7fd31c15d980 2024-04-25T00:32:51.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.019+0000 7fd320efa640 1 -- 172.21.15.29:0/983901197 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3180aa7f0 con 0x7fd31c15d980 2024-04-25T00:32:51.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.019+0000 7fd324160640 1 -- 172.21.15.29:0/983901197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c15d980 msgr2=0x7fd31c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:51.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.019+0000 7fd324160640 1 --2- 172.21.15.29:0/983901197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c15d980 0x7fd31c06d5d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd318009870 tx=0x7fd31801c530 comp rx=0 tx=0).stop 2024-04-25T00:32:51.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.019+0000 7fd324160640 1 -- 172.21.15.29:0/983901197 shutdown_connections 2024-04-25T00:32:51.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.019+0000 7fd324160640 1 --2- 172.21.15.29:0/983901197 >> v2:172.21.15.157:3300/0 conn(0x7fd31c0b2b70 0x7fd31c06db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.019+0000 7fd324160640 1 --2- 172.21.15.29:0/983901197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c15d980 0x7fd31c06d5d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.019+0000 7fd324160640 1 -- 172.21.15.29:0/983901197 >> 172.21.15.29:0/983901197 conn(0x7fd31c0b1870 msgr2=0x7fd31c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:51.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.019+0000 7fd324160640 1 -- 172.21.15.29:0/983901197 shutdown_connections 2024-04-25T00:32:51.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.019+0000 7fd324160640 1 -- 172.21.15.29:0/983901197 wait complete. 2024-04-25T00:32:51.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd324160640 1 Processor -- start 2024-04-25T00:32:51.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd324160640 1 -- start start 2024-04-25T00:32:51.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd324160640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd31c0b2b70 0x7fd31c0cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:51.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd324160640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c150df0 0x7fd31c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd324160640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd31c15d980 0x7fd31c166630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd324160640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c15b4b0 con 0x7fd31c0b2b70 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd324160640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c15dd60 con 0x7fd31c15d980 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd324160640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c15b630 con 0x7fd31c150df0 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd321efc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd31c0b2b70 0x7fd31c0cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd321efc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd31c0b2b70 0x7fd31c0cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33580/0 (socket says 172.21.15.29:33580) 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd321efc640 1 -- 172.21.15.29:0/931217481 learned_addr learned my addr 172.21.15.29:0/931217481 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd3216fb640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c150df0 0x7fd31c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd3226fd640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd31c15d980 0x7fd31c166630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd321efc640 1 -- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c150df0 msgr2=0x7fd31c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd321efc640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c150df0 0x7fd31c0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd321efc640 1 -- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd31c15d980 msgr2=0x7fd31c166630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd321efc640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd31c15d980 0x7fd31c166630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd321efc640 1 -- 172.21.15.29:0/931217481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c1f8fa0 con 0x7fd31c0b2b70 2024-04-25T00:32:51.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd321efc640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd31c0b2b70 0x7fd31c0cdee0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd3180ac040 tx=0x7fd3180192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:51.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd3127fc640 1 -- 172.21.15.29:0/931217481 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3180be610 con 0x7fd31c0b2b70 2024-04-25T00:32:51.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd3127fc640 1 -- 172.21.15.29:0/931217481 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3180be7f0 con 0x7fd31c0b2b70 2024-04-25T00:32:51.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd324160640 1 -- 172.21.15.29:0/931217481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c0ce5e0 con 0x7fd31c0b2b70 2024-04-25T00:32:51.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd3127fc640 1 -- 172.21.15.29:0/931217481 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3180bebe0 con 0x7fd31c0b2b70 2024-04-25T00:32:51.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd324160640 1 -- 172.21.15.29:0/931217481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c1f84c0 con 0x7fd31c0b2b70 2024-04-25T00:32:51.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd324160640 1 -- 172.21.15.29:0/931217481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31c069270 con 0x7fd31c0b2b70 2024-04-25T00:32:51.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd3127fc640 1 -- 172.21.15.29:0/931217481 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd3180bc020 con 0x7fd31c0b2b70 2024-04-25T00:32:51.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd3127fc640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd3000678c0 0x7fd300069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:51.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.023+0000 7fd3127fc640 1 -- 172.21.15.29:0/931217481 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd318120060 con 0x7fd31c0b2b70 2024-04-25T00:32:51.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.027+0000 7fd3216fb640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd3000678c0 0x7fd300069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:51.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.027+0000 7fd3216fb640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd3000678c0 0x7fd300069d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd31c1f8050 tx=0x7fd30c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:51.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.031+0000 7fd3127fc640 1 -- 172.21.15.29:0/931217481 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd318016150 con 0x7fd31c0b2b70 2024-04-25T00:32:51.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.191+0000 7fd324160640 1 -- 172.21.15.29:0/931217481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd31c0cf910 con 0x7fd31c0b2b70 2024-04-25T00:32:51.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.191+0000 7fd3127fc640 1 -- 172.21.15.29:0/931217481 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fd3180f0d50 con 0x7fd31c0b2b70 2024-04-25T00:32:51.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.195+0000 7fd324160640 1 -- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd3000678c0 msgr2=0x7fd300069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:51.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.195+0000 7fd324160640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd3000678c0 0x7fd300069d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd31c1f8050 tx=0x7fd30c008040 comp rx=0 tx=0).stop 2024-04-25T00:32:51.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.195+0000 7fd324160640 1 -- 172.21.15.29:0/931217481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd31c0b2b70 msgr2=0x7fd31c0cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:51.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.195+0000 7fd324160640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd31c0b2b70 0x7fd31c0cdee0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd3180ac040 tx=0x7fd3180192d0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.195+0000 7fd324160640 1 -- 172.21.15.29:0/931217481 shutdown_connections 2024-04-25T00:32:51.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.195+0000 7fd324160640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd3000678c0 0x7fd300069d80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.195+0000 7fd324160640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd31c15d980 0x7fd31c166630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.195+0000 7fd324160640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c150df0 0x7fd31c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.195+0000 7fd324160640 1 --2- 172.21.15.29:0/931217481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd31c0b2b70 0x7fd31c0cdee0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.195+0000 7fd324160640 1 -- 172.21.15.29:0/931217481 >> 172.21.15.29:0/931217481 conn(0x7fd31c0b1870 msgr2=0x7fd31c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:51.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.195+0000 7fd324160640 1 -- 172.21.15.29:0/931217481 shutdown_connections 2024-04-25T00:32:51.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.195+0000 7fd324160640 1 -- 172.21.15.29:0/931217481 wait complete. 2024-04-25T00:32:51.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:51.217 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp95260/subvol25790) 2024-04-25T00:32:51.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:51.242 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-25T00:32:51.310 INFO:teuthology.orchestra.run.smithi029.stdout:cephfs_data 2024-04-25T00:32:51.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:51.311 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp95260/subvol25790) 2024-04-25T00:32:51.373 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:32:51.373 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp95260/subvol25790/.meta 2024-04-25T00:32:51.443 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol25790 subvol_grp95260 2024-04-25T00:32:51.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d945cb640 1 Processor -- start 2024-04-25T00:32:51.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d945cb640 1 -- start start 2024-04-25T00:32:51.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d945cb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2d8c0b3b50 0x7f2d8c0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:51.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d945cb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2d8c06cd40 0x7f2d8c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:51.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d945cb640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d8c0cf670 con 0x7f2d8c1456d0 2024-04-25T00:32:51.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d945cb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2d8c0b4380 con 0x7f2d8c0b3b50 2024-04-25T00:32:51.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d945cb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2d8c0b40a0 con 0x7f2d8c06cd40 2024-04-25T00:32:51.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d91b66640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2d8c0b3b50 0x7f2d8c0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:51.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d92367640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2d8c1456d0 0x7f2d8c0cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50060/0 (socket says 172.21.15.29:50060) 2024-04-25T00:32:51.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d92367640 1 -- 172.21.15.29:0/1677328471 learned_addr learned my addr 172.21.15.29:0/1677328471 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:51.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d92b68640 1 --2- 172.21.15.29:0/1677328471 >> v2:172.21.15.157:3300/0 conn(0x7f2d8c06cd40 0x7f2d8c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:51.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d91b66640 1 -- 172.21.15.29:0/1677328471 >> v2:172.21.15.157:3300/0 conn(0x7f2d8c06cd40 msgr2=0x7f2d8c06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:51.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d91b66640 1 --2- 172.21.15.29:0/1677328471 >> v2:172.21.15.157:3300/0 conn(0x7f2d8c06cd40 0x7f2d8c06d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d91b66640 1 -- 172.21.15.29:0/1677328471 >> v1:172.21.15.29:6789/0 conn(0x7f2d8c1456d0 legacy=0x7f2d8c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:51.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d91b66640 1 -- 172.21.15.29:0/1677328471 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8c0ce480 con 0x7f2d8c0b3b50 2024-04-25T00:32:51.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.610+0000 7f2d91365640 1 -- 172.21.15.29:0/1677328471 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1700934611 0 0) 0x7f2d8c0cf670 con 0x7f2d8c1456d0 2024-04-25T00:32:51.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d91b66640 1 --2- 172.21.15.29:0/1677328471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c0b3b50 0x7f2d8c0b2b70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0730a0 tx=0x7f2d7c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86731fcdc68faf5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:51.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d91365640 1 -- 172.21.15.29:0/1677328471 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0b0070 con 0x7f2d8c0b3b50 2024-04-25T00:32:51.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d91365640 1 -- 172.21.15.29:0/1677328471 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d7c0ab600 con 0x7f2d8c0b3b50 2024-04-25T00:32:51.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d91365640 1 -- 172.21.15.29:0/1677328471 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0ab9d0 con 0x7f2d8c0b3b50 2024-04-25T00:32:51.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d945cb640 1 -- 172.21.15.29:0/1677328471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c0b3b50 msgr2=0x7f2d8c0b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:51.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d945cb640 1 --2- 172.21.15.29:0/1677328471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c0b3b50 0x7f2d8c0b2b70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0730a0 tx=0x7f2d7c01c480 comp rx=0 tx=0).stop 2024-04-25T00:32:51.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d945cb640 1 -- 172.21.15.29:0/1677328471 shutdown_connections 2024-04-25T00:32:51.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d945cb640 1 --2- 172.21.15.29:0/1677328471 >> v2:172.21.15.157:3300/0 conn(0x7f2d8c06cd40 0x7f2d8c06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d945cb640 1 --2- 172.21.15.29:0/1677328471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c0b3b50 0x7f2d8c0b2b70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d945cb640 1 -- 172.21.15.29:0/1677328471 >> 172.21.15.29:0/1677328471 conn(0x7f2d8c0b1870 msgr2=0x7f2d8c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:51.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d945cb640 1 -- 172.21.15.29:0/1677328471 shutdown_connections 2024-04-25T00:32:51.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d945cb640 1 -- 172.21.15.29:0/1677328471 wait complete. 2024-04-25T00:32:51.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d945cb640 1 Processor -- start 2024-04-25T00:32:51.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.614+0000 7f2d945cb640 1 -- start start 2024-04-25T00:32:51.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d945cb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d8c06cd40 0x7f2d8c139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:51.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d945cb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d8c0b3b50 0x7f2d8c1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:51.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d945cb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c1456d0 0x7f2d8c154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:51.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d945cb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c06d690 con 0x7f2d8c1456d0 2024-04-25T00:32:51.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d945cb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c0b5000 con 0x7f2d8c06cd40 2024-04-25T00:32:51.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d945cb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8c168b70 con 0x7f2d8c0b3b50 2024-04-25T00:32:51.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d92b68640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c1456d0 0x7f2d8c154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:51.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d92b68640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c1456d0 0x7f2d8c154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33612/0 (socket says 172.21.15.29:33612) 2024-04-25T00:32:51.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d92b68640 1 -- 172.21.15.29:0/1580417477 learned_addr learned my addr 172.21.15.29:0/1580417477 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:51.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d92367640 1 --2- 172.21.15.29:0/1580417477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d8c06cd40 0x7f2d8c139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:51.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d92b68640 1 -- 172.21.15.29:0/1580417477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d8c0b3b50 msgr2=0x7f2d8c1569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:51.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d92b68640 1 --2- 172.21.15.29:0/1580417477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d8c0b3b50 0x7f2d8c1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d92b68640 1 -- 172.21.15.29:0/1580417477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d8c06cd40 msgr2=0x7f2d8c139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:51.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d92b68640 1 --2- 172.21.15.29:0/1580417477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d8c06cd40 0x7f2d8c139e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d92b68640 1 -- 172.21.15.29:0/1580417477 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8c1572d0 con 0x7f2d8c1456d0 2024-04-25T00:32:51.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d92b68640 1 --2- 172.21.15.29:0/1580417477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c1456d0 0x7f2d8c154c30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2d84007e60 tx=0x7f2d8400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:51.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d82ffd640 1 -- 172.21.15.29:0/1580417477 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84014420 con 0x7f2d8c1456d0 2024-04-25T00:32:51.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d82ffd640 1 -- 172.21.15.29:0/1580417477 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d84019070 con 0x7f2d8c1456d0 2024-04-25T00:32:51.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d945cb640 1 -- 172.21.15.29:0/1580417477 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8c1575b0 con 0x7f2d8c1456d0 2024-04-25T00:32:51.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d82ffd640 1 -- 172.21.15.29:0/1580417477 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84014820 con 0x7f2d8c1456d0 2024-04-25T00:32:51.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d945cb640 1 -- 172.21.15.29:0/1580417477 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d8c13aaa0 con 0x7f2d8c1456d0 2024-04-25T00:32:51.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d945cb640 1 -- 172.21.15.29:0/1580417477 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d8c0692e0 con 0x7f2d8c1456d0 2024-04-25T00:32:51.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d82ffd640 1 -- 172.21.15.29:0/1580417477 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2d840149c0 con 0x7f2d8c1456d0 2024-04-25T00:32:51.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d82ffd640 1 --2- 172.21.15.29:0/1580417477 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2d700678f0 0x7f2d70069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:51.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d82ffd640 1 -- 172.21.15.29:0/1580417477 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2d840864b0 con 0x7f2d8c1456d0 2024-04-25T00:32:51.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.618+0000 7f2d92367640 1 --2- 172.21.15.29:0/1580417477 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2d700678f0 0x7f2d70069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:51.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.622+0000 7f2d92367640 1 --2- 172.21.15.29:0/1580417477 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2d700678f0 0x7f2d70069db0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2d88000f70 tx=0x7f2d88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:51.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.622+0000 7f2d82ffd640 1 -- 172.21.15.29:0/1580417477 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2d84056fe0 con 0x7f2d8c1456d0 2024-04-25T00:32:51.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.810+0000 7f2d945cb640 1 -- 172.21.15.29:0/1580417477 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol25790", "group_name": "subvol_grp95260", "target": ["mon-mgr", ""]}) -- 0x7f2d8c0b4de0 con 0x7f2d700678f0 2024-04-25T00:32:51.824 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:51.823+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:51.824 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:51.823+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:51.824 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:51.823+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:51.824 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:51.823+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:51.824 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:51.823+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:51.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.950+0000 7f2d82ffd640 1 -- 172.21.15.29:0/1580417477 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2d8c0b4de0 con 0x7f2d700678f0 2024-04-25T00:32:51.955 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/subvol_grp95260/subvol25790 2024-04-25T00:32:51.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.954+0000 7f2d945cb640 1 -- 172.21.15.29:0/1580417477 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2d700678f0 msgr2=0x7f2d70069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:51.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.954+0000 7f2d945cb640 1 --2- 172.21.15.29:0/1580417477 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2d700678f0 0x7f2d70069db0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2d88000f70 tx=0x7f2d88009040 comp rx=0 tx=0).stop 2024-04-25T00:32:51.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.954+0000 7f2d945cb640 1 -- 172.21.15.29:0/1580417477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c1456d0 msgr2=0x7f2d8c154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:51.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.954+0000 7f2d945cb640 1 --2- 172.21.15.29:0/1580417477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c1456d0 0x7f2d8c154c30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2d84007e60 tx=0x7f2d8400e4b0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.954+0000 7f2d945cb640 1 -- 172.21.15.29:0/1580417477 shutdown_connections 2024-04-25T00:32:51.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.954+0000 7f2d945cb640 1 --2- 172.21.15.29:0/1580417477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c1456d0 0x7f2d8c154c30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.954+0000 7f2d945cb640 1 --2- 172.21.15.29:0/1580417477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d8c0b3b50 0x7f2d8c1569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.954+0000 7f2d945cb640 1 --2- 172.21.15.29:0/1580417477 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2d700678f0 0x7f2d70069db0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.954+0000 7f2d945cb640 1 --2- 172.21.15.29:0/1580417477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d8c06cd40 0x7f2d8c139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:51.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.954+0000 7f2d945cb640 1 -- 172.21.15.29:0/1580417477 >> 172.21.15.29:0/1580417477 conn(0x7f2d8c0b1870 msgr2=0x7f2d8c0b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:51.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.954+0000 7f2d945cb640 1 -- 172.21.15.29:0/1580417477 shutdown_connections 2024-04-25T00:32:51.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:51.954+0000 7f2d945cb640 1 -- 172.21.15.29:0/1580417477 wait complete. 2024-04-25T00:32:51.976 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol25790 subvol_grp95260 2024-04-25T00:32:52.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1bed0640 1 Processor -- start 2024-04-25T00:32:52.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1bed0640 1 -- start start 2024-04-25T00:32:52.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1bed0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1406d450 0x7f8f140b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:52.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1bed0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f140cf670 con 0x7f8f141456d0 2024-04-25T00:32:52.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1bed0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f140b4380 con 0x7f8f140b3b50 2024-04-25T00:32:52.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1bed0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f140b40a0 con 0x7f8f1406d450 2024-04-25T00:32:52.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f19c6c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8f141456d0 0x7f8f140cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50068/0 (socket says 172.21.15.29:50068) 2024-04-25T00:32:52.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f19c6c640 1 -- 172.21.15.29:0/4095052777 learned_addr learned my addr 172.21.15.29:0/4095052777 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:52.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1a46d640 1 --2- 172.21.15.29:0/4095052777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1406d450 0x7f8f140b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2924642132 0 0) 0x7f8f140cf670 con 0x7f8f141456d0 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f00003530 con 0x7f8f141456d0 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2403683061 0 0) 0x7f8f00003530 con 0x7f8f141456d0 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1406d450 msgr2=0x7f8f140b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 --2- 172.21.15.29:0/4095052777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1406d450 0x7f8f140b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 >> v1:172.21.15.157:6789/0 conn(0x7f8f140b3b50 legacy=0x7f8f1406cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f140ce480 con 0x7f8f141456d0 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8f10001680 con 0x7f8f141456d0 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 >> v1:172.21.15.29:6789/0 conn(0x7f8f141456d0 legacy=0x7f8f140cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 --2- 172.21.15.29:0/4095052777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1406d450 0x7f8f000039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 --2- 172.21.15.29:0/4095052777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f00003f70 0x7f8f00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 --2- 172.21.15.29:0/4095052777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f00004d50 0x7f8f000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f140b40a0 con 0x7f8f00003f70 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0000bb60 con 0x7f8f00004d50 2024-04-25T00:32:52.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f0000bd60 con 0x7f8f1406d450 2024-04-25T00:32:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1946b640 1 --2- 172.21.15.29:0/4095052777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f00003f70 0x7f8f00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f19c6c640 1 --2- 172.21.15.29:0/4095052777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1406d450 0x7f8f000039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1a46d640 1 --2- 172.21.15.29:0/4095052777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f00004d50 0x7f8f000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1946b640 1 -- 172.21.15.29:0/4095052777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1406d450 msgr2=0x7f8f000039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1946b640 1 --2- 172.21.15.29:0/4095052777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f1406d450 0x7f8f000039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1946b640 1 -- 172.21.15.29:0/4095052777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f00004d50 msgr2=0x7f8f000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1946b640 1 --2- 172.21.15.29:0/4095052777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f00004d50 0x7f8f000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1946b640 1 -- 172.21.15.29:0/4095052777 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f00003530 con 0x7f8f00003f70 2024-04-25T00:32:52.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.106+0000 7f8f1a46d640 1 --2- 172.21.15.29:0/4095052777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f00004d50 0x7f8f000051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:52.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.110+0000 7f8f1946b640 1 --2- 172.21.15.29:0/4095052777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f00003f70 0x7f8f00004810 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8f04001780 tx=0x7f8f0401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5d025d2919eff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:52.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.110+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f040013a0 con 0x7f8f00003f70 2024-04-25T00:32:52.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.110+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f040052f0 con 0x7f8f00003f70 2024-04-25T00:32:52.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.110+0000 7f8f18c6a640 1 -- 172.21.15.29:0/4095052777 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04003840 con 0x7f8f00003f70 2024-04-25T00:32:52.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.110+0000 7f8f1bed0640 1 -- 172.21.15.29:0/4095052777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f00003f70 msgr2=0x7f8f00004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.110+0000 7f8f1bed0640 1 --2- 172.21.15.29:0/4095052777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f00003f70 0x7f8f00004810 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8f04001780 tx=0x7f8f0401c420 comp rx=0 tx=0).stop 2024-04-25T00:32:52.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.110+0000 7f8f1a46d640 1 -- 172.21.15.29:0/4095052777 reap_dead start 2024-04-25T00:32:52.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.110+0000 7f8f1bed0640 1 -- 172.21.15.29:0/4095052777 shutdown_connections 2024-04-25T00:32:52.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.110+0000 7f8f1bed0640 1 -- 172.21.15.29:0/4095052777 >> 172.21.15.29:0/4095052777 conn(0x7f8f140b1870 msgr2=0x7f8f140b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:52.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.110+0000 7f8f1bed0640 1 -- 172.21.15.29:0/4095052777 shutdown_connections 2024-04-25T00:32:52.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.110+0000 7f8f1bed0640 1 -- 172.21.15.29:0/4095052777 wait complete. 2024-04-25T00:32:52.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.110+0000 7f8f1bed0640 1 Processor -- start 2024-04-25T00:32:52.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.110+0000 7f8f1bed0640 1 -- start start 2024-04-25T00:32:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f1bed0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f00003f70 0x7f8f1414d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f1bed0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f14157950 0x7f8f1413e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f1bed0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f14157db0 0x7f8f1413e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f1bed0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14167b70 con 0x7f8f00003f70 2024-04-25T00:32:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f1bed0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f140b5000 con 0x7f8f14157db0 2024-04-25T00:32:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f1bed0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f14167cf0 con 0x7f8f14157950 2024-04-25T00:32:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f19c6c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f00003f70 0x7f8f1414d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f19c6c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f00003f70 0x7f8f1414d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33638/0 (socket says 172.21.15.29:33638) 2024-04-25T00:32:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f19c6c640 1 -- 172.21.15.29:0/708564804 learned_addr learned my addr 172.21.15.29:0/708564804 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f1a46d640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f14157db0 0x7f8f1413e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:52.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f1946b640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f14157950 0x7f8f1413e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:52.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f19c6c640 1 -- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f14157950 msgr2=0x7f8f1413e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f19c6c640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f14157950 0x7f8f1413e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f19c6c640 1 -- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f14157db0 msgr2=0x7f8f1413e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f19c6c640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f14157db0 0x7f8f1413e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f19c6c640 1 -- 172.21.15.29:0/708564804 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1413dc70 con 0x7f8f00003f70 2024-04-25T00:32:52.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f19c6c640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f00003f70 0x7f8f1414d8d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8f10001ec0 tx=0x7f8f1000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:52.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f0a7fc640 1 -- 172.21.15.29:0/708564804 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1000ee00 con 0x7f8f00003f70 2024-04-25T00:32:52.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f0a7fc640 1 -- 172.21.15.29:0/708564804 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f10018070 con 0x7f8f00003f70 2024-04-25T00:32:52.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f1bed0640 1 -- 172.21.15.29:0/708564804 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1413efc0 con 0x7f8f00003f70 2024-04-25T00:32:52.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f0a7fc640 1 -- 172.21.15.29:0/708564804 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1000ca90 con 0x7f8f00003f70 2024-04-25T00:32:52.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f1bed0640 1 -- 172.21.15.29:0/708564804 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1413b620 con 0x7f8f00003f70 2024-04-25T00:32:52.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f1bed0640 1 -- 172.21.15.29:0/708564804 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f140692e0 con 0x7f8f00003f70 2024-04-25T00:32:52.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f0a7fc640 1 -- 172.21.15.29:0/708564804 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8f1000cc30 con 0x7f8f00003f70 2024-04-25T00:32:52.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f0a7fc640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ef80678c0 0x7f8ef8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:52.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f0a7fc640 1 -- 172.21.15.29:0/708564804 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8f100859d0 con 0x7f8f00003f70 2024-04-25T00:32:52.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.114+0000 7f8f1946b640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ef80678c0 0x7f8ef8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:52.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.118+0000 7f8f1946b640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ef80678c0 0x7f8ef8069d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8f040ac000 tx=0x7f8f040011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:52.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.118+0000 7f8f0a7fc640 1 -- 172.21.15.29:0/708564804 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8f100565a0 con 0x7f8f00003f70 2024-04-25T00:32:52.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.306+0000 7f8f1bed0640 1 -- 172.21.15.29:0/708564804 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol25790", "group_name": "subvol_grp95260", "target": ["mon-mgr", ""]}) -- 0x7f8f140b4de0 con 0x7f8ef80678c0 2024-04-25T00:32:52.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.322+0000 7f8f0a7fc640 1 -- 172.21.15.29:0/708564804 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8f140b4de0 con 0x7f8ef80678c0 2024-04-25T00:32:52.325 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/subvol_grp95260/subvol25790 2024-04-25T00:32:52.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.322+0000 7f8f1bed0640 1 -- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ef80678c0 msgr2=0x7f8ef8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.322+0000 7f8f1bed0640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ef80678c0 0x7f8ef8069d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8f040ac000 tx=0x7f8f040011a0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.322+0000 7f8f1bed0640 1 -- 172.21.15.29:0/708564804 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f00003f70 msgr2=0x7f8f1414d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.322+0000 7f8f1bed0640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f00003f70 0x7f8f1414d8d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8f10001ec0 tx=0x7f8f1000ec10 comp rx=0 tx=0).stop 2024-04-25T00:32:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.322+0000 7f8f1bed0640 1 -- 172.21.15.29:0/708564804 shutdown_connections 2024-04-25T00:32:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.322+0000 7f8f1bed0640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ef80678c0 0x7f8ef8069d80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.322+0000 7f8f1bed0640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f14157db0 0x7f8f1413e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.322+0000 7f8f1bed0640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f14157950 0x7f8f1413e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.322+0000 7f8f1bed0640 1 --2- 172.21.15.29:0/708564804 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f00003f70 0x7f8f1414d8d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.322+0000 7f8f1bed0640 1 -- 172.21.15.29:0/708564804 >> 172.21.15.29:0/708564804 conn(0x7f8f140b1870 msgr2=0x7f8f140af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.322+0000 7f8f1bed0640 1 -- 172.21.15.29:0/708564804 shutdown_connections 2024-04-25T00:32:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.322+0000 7f8f1bed0640 1 -- 172.21.15.29:0/708564804 wait complete. 2024-04-25T00:32:52.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:52.344 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/3fb9a8c4eabf564c7403134e37ad4dd5.meta) 2024-04-25T00:32:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:[GLOBAL] 2024-04-25T00:32:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:version = 1 2024-04-25T00:32:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:type = subvolume 2024-04-25T00:32:52.500 INFO:teuthology.orchestra.run.smithi029.stdout:path = /volumes/subvol_grp95260/subvol25790 2024-04-25T00:32:52.501 INFO:teuthology.orchestra.run.smithi029.stdout:state = complete 2024-04-25T00:32:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:52.502 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol25790 subvol_grp95260 2024-04-25T00:32:52.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.638+0000 7f230d814640 1 Processor -- start 2024-04-25T00:32:52.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.638+0000 7f230d814640 1 -- start start 2024-04-25T00:32:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.642+0000 7f230d814640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2308070ce0 0x7f2308072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.642+0000 7f230d814640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23080712c0 con 0x7f23080b4590 2024-04-25T00:32:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.642+0000 7f230d814640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23080715a0 con 0x7f230815e9a0 2024-04-25T00:32:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.642+0000 7f230d814640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2308071ed0 con 0x7f2308070ce0 2024-04-25T00:32:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.642+0000 7f22fffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f23080b4590 0x7f23080cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50078/0 (socket says 172.21.15.29:50078) 2024-04-25T00:32:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.642+0000 7f22fffff640 1 -- 172.21.15.29:0/4077681510 learned_addr learned my addr 172.21.15.29:0/4077681510 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.642+0000 7f23077fe640 1 --2- 172.21.15.29:0/4077681510 >> v2:172.21.15.29:3300/0 conn(0x7f2308070ce0 0x7f2308072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.642+0000 7f23067fc640 1 -- 172.21.15.29:0/4077681510 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 63543561 0 0) 0x7f23080712c0 con 0x7f23080b4590 2024-04-25T00:32:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.642+0000 7f23067fc640 1 -- 172.21.15.29:0/4077681510 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22f4003530 con 0x7f23080b4590 2024-04-25T00:32:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.642+0000 7f23067fc640 1 -- 172.21.15.29:0/4077681510 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2655260783 0 0) 0x7f23080715a0 con 0x7f230815e9a0 2024-04-25T00:32:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.642+0000 7f23077fe640 1 -- 172.21.15.29:0/4077681510 >> v1:172.21.15.157:6789/0 conn(0x7f230815e9a0 legacy=0x7f230815ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.642+0000 7f23077fe640 1 -- 172.21.15.29:0/4077681510 >> v1:172.21.15.29:6789/0 conn(0x7f23080b4590 legacy=0x7f23080cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.642+0000 7f23077fe640 1 -- 172.21.15.29:0/4077681510 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2308071880 con 0x7f2308070ce0 2024-04-25T00:32:52.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.646+0000 7f23077fe640 1 --2- 172.21.15.29:0/4077681510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2308070ce0 0x7f2308072360 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2300009d90 tx=0x7f230001c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76dc6dbb281df88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:52.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.646+0000 7f23067fc640 1 -- 172.21.15.29:0/4077681510 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23000b2070 con 0x7f2308070ce0 2024-04-25T00:32:52.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.646+0000 7f23067fc640 1 -- 172.21.15.29:0/4077681510 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23000a5590 con 0x7f2308070ce0 2024-04-25T00:32:52.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.646+0000 7f23067fc640 1 -- 172.21.15.29:0/4077681510 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0001680 con 0x7f2308070ce0 2024-04-25T00:32:52.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.646+0000 7f230d814640 1 -- 172.21.15.29:0/4077681510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2308070ce0 msgr2=0x7f2308072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.646+0000 7f230d814640 1 --2- 172.21.15.29:0/4077681510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2308070ce0 0x7f2308072360 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2300009d90 tx=0x7f230001c6f0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.646+0000 7f230d814640 1 -- 172.21.15.29:0/4077681510 shutdown_connections 2024-04-25T00:32:52.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.646+0000 7f230d814640 1 --2- 172.21.15.29:0/4077681510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2308070ce0 0x7f2308072360 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.646+0000 7f230d814640 1 -- 172.21.15.29:0/4077681510 >> 172.21.15.29:0/4077681510 conn(0x7f23080b1870 msgr2=0x7f23080b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:52.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.646+0000 7f230d814640 1 -- 172.21.15.29:0/4077681510 shutdown_connections 2024-04-25T00:32:52.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.646+0000 7f230d814640 1 -- 172.21.15.29:0/4077681510 wait complete. 2024-04-25T00:32:52.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f230d814640 1 Processor -- start 2024-04-25T00:32:52.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f230d814640 1 -- start start 2024-04-25T00:32:52.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f230d814640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2308070ce0 0x7f23081589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:52.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f230d814640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23080b4590 0x7f230806d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:52.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f230d814640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f230815e9a0 0x7f230813c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:52.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f230d814640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2308168ed0 con 0x7f23080b4590 2024-04-25T00:32:52.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f230d814640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2308168d50 con 0x7f2308070ce0 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f230d814640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2308169050 con 0x7f230815e9a0 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f22fffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23080b4590 0x7f230806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f22fffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23080b4590 0x7f230806d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33662/0 (socket says 172.21.15.29:33662) 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f22fffff640 1 -- 172.21.15.29:0/177370235 learned_addr learned my addr 172.21.15.29:0/177370235 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f2306ffd640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2308070ce0 0x7f23081589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f23077fe640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f230815e9a0 0x7f230813c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f22fffff640 1 -- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f230815e9a0 msgr2=0x7f230813c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f22fffff640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f230815e9a0 0x7f230813c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f22fffff640 1 -- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2308070ce0 msgr2=0x7f23081589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f22fffff640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2308070ce0 0x7f23081589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f22fffff640 1 -- 172.21.15.29:0/177370235 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230806e3f0 con 0x7f23080b4590 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f23077fe640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f230815e9a0 0x7f230813c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f22fffff640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23080b4590 0x7f230806d830 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f22f000c790 tx=0x7f22f000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f2304ff9640 1 -- 172.21.15.29:0/177370235 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0018070 con 0x7f23080b4590 2024-04-25T00:32:52.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f2304ff9640 1 -- 172.21.15.29:0/177370235 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f00074e0 con 0x7f23080b4590 2024-04-25T00:32:52.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f230d814640 1 -- 172.21.15.29:0/177370235 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23080cdac0 con 0x7f23080b4590 2024-04-25T00:32:52.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f2304ff9640 1 -- 172.21.15.29:0/177370235 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0004a10 con 0x7f23080b4590 2024-04-25T00:32:52.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f230d814640 1 -- 172.21.15.29:0/177370235 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23080ce080 con 0x7f23080b4590 2024-04-25T00:32:52.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f230d814640 1 -- 172.21.15.29:0/177370235 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23080692e0 con 0x7f23080b4590 2024-04-25T00:32:52.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.650+0000 7f2304ff9640 1 -- 172.21.15.29:0/177370235 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7f22f001b070 con 0x7f23080b4590 2024-04-25T00:32:52.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.654+0000 7f2304ff9640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f22e8067950 0x7f22e8069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:52.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.654+0000 7f2304ff9640 1 -- 172.21.15.29:0/177370235 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f22f00853f0 con 0x7f23080b4590 2024-04-25T00:32:52.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.654+0000 7f2306ffd640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f22e8067950 0x7f22e8069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:52.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.654+0000 7f2306ffd640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f22e8067950 0x7f22e8069e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2308158040 tx=0x7f22f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:52.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.658+0000 7f2304ff9640 1 -- 172.21.15.29:0/177370235 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f22f0055e40 con 0x7f23080b4590 2024-04-25T00:32:52.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.846+0000 7f230d814640 1 -- 172.21.15.29:0/177370235 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol25790", "group_name": "subvol_grp95260", "target": ["mon-mgr", ""]}) -- 0x7f230806e7b0 con 0x7f22e8067950 2024-04-25T00:32:52.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.878+0000 7f2304ff9640 1 -- 172.21.15.29:0/177370235 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f230806e7b0 con 0x7f22e8067950 2024-04-25T00:32:52.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.882+0000 7f230d814640 1 -- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f22e8067950 msgr2=0x7f22e8069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.882+0000 7f230d814640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f22e8067950 0x7f22e8069e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2308158040 tx=0x7f22f8009040 comp rx=0 tx=0).stop 2024-04-25T00:32:52.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.882+0000 7f230d814640 1 -- 172.21.15.29:0/177370235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23080b4590 msgr2=0x7f230806d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:52.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.882+0000 7f230d814640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23080b4590 0x7f230806d830 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f22f000c790 tx=0x7f22f000cc50 comp rx=0 tx=0).stop 2024-04-25T00:32:52.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.882+0000 7f230d814640 1 -- 172.21.15.29:0/177370235 shutdown_connections 2024-04-25T00:32:52.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.882+0000 7f230d814640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f230815e9a0 0x7f230813c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.882+0000 7f230d814640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23080b4590 0x7f230806d830 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.882+0000 7f230d814640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f22e8067950 0x7f22e8069e10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.882+0000 7f230d814640 1 --2- 172.21.15.29:0/177370235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2308070ce0 0x7f23081589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:52.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.882+0000 7f230d814640 1 -- 172.21.15.29:0/177370235 >> 172.21.15.29:0/177370235 conn(0x7f23080b1870 msgr2=0x7f23081424e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:52.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.882+0000 7f230d814640 1 -- 172.21.15.29:0/177370235 shutdown_connections 2024-04-25T00:32:52.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52.882+0000 7f230d814640 1 -- 172.21.15.29:0/177370235 wait complete. 2024-04-25T00:32:52.891 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:52.887+0000 7f47fa062640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:52.891 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:52.887+0000 7f47fa062640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:52.891 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:52.887+0000 7f47fa062640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:52.891 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:52.887+0000 7f47fa062640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:52.891 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:52.887+0000 7f47fa062640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:52.907 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-25T00:32:52.911 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:52.907+0000 7f47f9861640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:52.911 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:52.907+0000 7f47f9861640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:52.911 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:52.907+0000 7f47f9861640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:52.911 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:52.907+0000 7f47f9861640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:52.911 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:32:52.907+0000 7f47f9861640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:32:52.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:32:52.953 INFO:teuthology.orchestra.run.smithi029.stdout:2 2024-04-25T00:32:52.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:32:52.960 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp95260 2024-04-25T00:32:53.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.126+0000 7f419dd05640 1 Processor -- start 2024-04-25T00:32:53.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.126+0000 7f419dd05640 1 -- start start 2024-04-25T00:32:53.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f419dd05640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f419815e200 0x7f41980d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f419dd05640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f41980cf690 0x7f41980d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f419dd05640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f41980d2a90 0x7f41980d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f419dd05640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f41980d2ee0 con 0x7f41980d2a90 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f419dd05640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f41980b38d0 con 0x7f41980cf690 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f419dd05640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f41980cfc10 con 0x7f419815e200 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f4197fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f41980d2a90 0x7f41980d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f4197fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f41980d2a90 0x7f41980d05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33672/0 (socket says 172.21.15.29:33672) 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f4197fff640 1 -- 172.21.15.29:0/2837733151 learned_addr learned my addr 172.21.15.29:0/2837733151 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f41977fe640 1 --2- 172.21.15.29:0/2837733151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f419815e200 0x7f41980d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f4196ffd640 1 --2- 172.21.15.29:0/2837733151 >> v2:172.21.15.157:3300/0 conn(0x7f41980cf690 0x7f41980d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f4197fff640 1 -- 172.21.15.29:0/2837733151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f419815e200 msgr2=0x7f41980d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f4197fff640 1 --2- 172.21.15.29:0/2837733151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f419815e200 0x7f41980d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f4197fff640 1 -- 172.21.15.29:0/2837733151 >> v2:172.21.15.157:3300/0 conn(0x7f41980cf690 msgr2=0x7f41980d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f4197fff640 1 --2- 172.21.15.29:0/2837733151 >> v2:172.21.15.157:3300/0 conn(0x7f41980cf690 0x7f41980d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f4197fff640 1 -- 172.21.15.29:0/2837733151 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41980cfef0 con 0x7f41980d2a90 2024-04-25T00:32:53.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f4196ffd640 1 --2- 172.21.15.29:0/2837733151 >> v2:172.21.15.157:3300/0 conn(0x7f41980cf690 0x7f41980d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:53.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f4197fff640 1 --2- 172.21.15.29:0/2837733151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41980d2a90 0x7f41980d05a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f419808e7a0 tx=0x7f419001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b999b08f4f8794 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:53.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f41967fc640 1 -- 172.21.15.29:0/2837733151 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41900b0070 con 0x7f41980d2a90 2024-04-25T00:32:53.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f41967fc640 1 -- 172.21.15.29:0/2837733151 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41900ab600 con 0x7f41980d2a90 2024-04-25T00:32:53.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.130+0000 7f41967fc640 1 -- 172.21.15.29:0/2837733151 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41900ab9d0 con 0x7f41980d2a90 2024-04-25T00:32:53.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 -- 172.21.15.29:0/2837733151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41980d2a90 msgr2=0x7f41980d05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 --2- 172.21.15.29:0/2837733151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41980d2a90 0x7f41980d05a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f419808e7a0 tx=0x7f419001c230 comp rx=0 tx=0).stop 2024-04-25T00:32:53.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 -- 172.21.15.29:0/2837733151 shutdown_connections 2024-04-25T00:32:53.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 --2- 172.21.15.29:0/2837733151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41980d2a90 0x7f41980d05a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 --2- 172.21.15.29:0/2837733151 >> v2:172.21.15.157:3300/0 conn(0x7f41980cf690 0x7f41980d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 --2- 172.21.15.29:0/2837733151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f419815e200 0x7f41980d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 -- 172.21.15.29:0/2837733151 >> 172.21.15.29:0/2837733151 conn(0x7f419809d280 msgr2=0x7f41980d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:53.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 -- 172.21.15.29:0/2837733151 shutdown_connections 2024-04-25T00:32:53.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 -- 172.21.15.29:0/2837733151 wait complete. 2024-04-25T00:32:53.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 Processor -- start 2024-04-25T00:32:53.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 -- start start 2024-04-25T00:32:53.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41980cf690 0x7f4198155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41980d2a90 0x7f4198147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f419815e200 0x7f41980b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f419815e760 con 0x7f41980d2a90 2024-04-25T00:32:53.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f419815e5e0 con 0x7f41980cf690 2024-04-25T00:32:53.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f41980b4320 con 0x7f419815e200 2024-04-25T00:32:53.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f4196ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41980d2a90 0x7f4198147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f4196ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41980d2a90 0x7f4198147010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33678/0 (socket says 172.21.15.29:33678) 2024-04-25T00:32:53.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f4196ffd640 1 -- 172.21.15.29:0/1664722296 learned_addr learned my addr 172.21.15.29:0/1664722296 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:53.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f4197fff640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f419815e200 0x7f41980b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f41977fe640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41980cf690 0x7f4198155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f4196ffd640 1 -- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f419815e200 msgr2=0x7f41980b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f4196ffd640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f419815e200 0x7f41980b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f4196ffd640 1 -- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41980cf690 msgr2=0x7f4198155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f4196ffd640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41980cf690 0x7f4198155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f4196ffd640 1 -- 172.21.15.29:0/1664722296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41980cbf20 con 0x7f41980d2a90 2024-04-25T00:32:53.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f4197fff640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f419815e200 0x7f41980b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:53.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f4196ffd640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41980d2a90 0x7f4198147010 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f418c00c7c0 tx=0x7f418c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:53.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f417ffff640 1 -- 172.21.15.29:0/1664722296 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c015070 con 0x7f41980d2a90 2024-04-25T00:32:53.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f417ffff640 1 -- 172.21.15.29:0/1664722296 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f418c007a30 con 0x7f41980d2a90 2024-04-25T00:32:53.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 -- 172.21.15.29:0/1664722296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41980cc200 con 0x7f41980d2a90 2024-04-25T00:32:53.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f417ffff640 1 -- 172.21.15.29:0/1664722296 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c007e00 con 0x7f41980d2a90 2024-04-25T00:32:53.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.134+0000 7f419dd05640 1 -- 172.21.15.29:0/1664722296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4198156760 con 0x7f41980d2a90 2024-04-25T00:32:53.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.138+0000 7f419dd05640 1 -- 172.21.15.29:0/1664722296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41980cc8c0 con 0x7f41980d2a90 2024-04-25T00:32:53.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.138+0000 7f417ffff640 1 -- 172.21.15.29:0/1664722296 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7f418c018070 con 0x7f41980d2a90 2024-04-25T00:32:53.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.138+0000 7f417ffff640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4184067950 0x7f4184069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.138+0000 7f417ffff640 1 -- 172.21.15.29:0/1664722296 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f418c085c90 con 0x7f41980d2a90 2024-04-25T00:32:53.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.138+0000 7f41977fe640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4184067950 0x7f4184069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.138+0000 7f41977fe640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4184067950 0x7f4184069e10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4188009e90 tx=0x7f418800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:53.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.142+0000 7f417ffff640 1 -- 172.21.15.29:0/1664722296 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f41980cc8c0 con 0x7f41980d2a90 2024-04-25T00:32:53.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.326+0000 7f419dd05640 1 -- 172.21.15.29:0/1664722296 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp95260", "target": ["mon-mgr", ""]}) -- 0x7f4198069270 con 0x7f4184067950 2024-04-25T00:32:53.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.334+0000 7f417ffff640 1 -- 172.21.15.29:0/1664722296 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4198069270 con 0x7f4184067950 2024-04-25T00:32:53.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.334+0000 7f419dd05640 1 -- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4184067950 msgr2=0x7f4184069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.334+0000 7f419dd05640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4184067950 0x7f4184069e10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4188009e90 tx=0x7f418800b040 comp rx=0 tx=0).stop 2024-04-25T00:32:53.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.334+0000 7f419dd05640 1 -- 172.21.15.29:0/1664722296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41980d2a90 msgr2=0x7f4198147010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.334+0000 7f419dd05640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41980d2a90 0x7f4198147010 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f418c00c7c0 tx=0x7f418c00cc80 comp rx=0 tx=0).stop 2024-04-25T00:32:53.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.334+0000 7f419dd05640 1 -- 172.21.15.29:0/1664722296 shutdown_connections 2024-04-25T00:32:53.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.334+0000 7f419dd05640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f419815e200 0x7f41980b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.334+0000 7f419dd05640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41980d2a90 0x7f4198147010 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.334+0000 7f419dd05640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4184067950 0x7f4184069e10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.334+0000 7f419dd05640 1 --2- 172.21.15.29:0/1664722296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41980cf690 0x7f4198155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.334+0000 7f419dd05640 1 -- 172.21.15.29:0/1664722296 >> 172.21.15.29:0/1664722296 conn(0x7f419809d280 msgr2=0x7f419815d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:53.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.334+0000 7f419dd05640 1 -- 172.21.15.29:0/1664722296 shutdown_connections 2024-04-25T00:32:53.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.334+0000 7f419dd05640 1 -- 172.21.15.29:0/1664722296 wait complete. 2024-04-25T00:32:53.356 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:32:53.356 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:32:53.407 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:32:53.462 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:32:53.462 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:32:53.462 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:32:53.463 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:53.468 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:53.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.626+0000 7f524e074640 1 Processor -- start 2024-04-25T00:32:53.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.626+0000 7f524e074640 1 -- start start 2024-04-25T00:32:53.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.626+0000 7f524e074640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52480b3c30 0x7f524806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.626+0000 7f524e074640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f52480b4b10 0x7f52480ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.626+0000 7f524e074640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52480b41e0 con 0x7f52480b3140 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.626+0000 7f524e074640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f524806d960 con 0x7f52480b4b10 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.626+0000 7f524e074640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f524806dc40 con 0x7f52480b3c30 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.626+0000 7f52477fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f52480b3140 0x7f52480b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50086/0 (socket says 172.21.15.29:50086) 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.626+0000 7f52477fe640 1 -- 172.21.15.29:0/2676781749 learned_addr learned my addr 172.21.15.29:0/2676781749 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.626+0000 7f5247fff640 1 --2- 172.21.15.29:0/2676781749 >> v2:172.21.15.157:3300/0 conn(0x7f52480b4b10 0x7f52480ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.626+0000 7f523effd640 1 --2- 172.21.15.29:0/2676781749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52480b3c30 0x7f524806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.626+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3930034099 0 0) 0x7f52480b41e0 con 0x7f52480b3140 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.626+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5234003530 con 0x7f52480b3140 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2706773378 0 0) 0x7f5234003530 con 0x7f52480b3140 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52480b3c30 msgr2=0x7f524806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 --2- 172.21.15.29:0/2676781749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52480b3c30 0x7f524806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 >> v2:172.21.15.157:3300/0 conn(0x7f52480b4b10 msgr2=0x7f52480ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 --2- 172.21.15.29:0/2676781749 >> v2:172.21.15.157:3300/0 conn(0x7f52480b4b10 0x7f52480ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524806df20 con 0x7f52480b3140 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f5238001680 con 0x7f52480b3140 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5247fff640 1 --2- 172.21.15.29:0/2676781749 >> v2:172.21.15.157:3300/0 conn(0x7f52480b4b10 0x7f52480ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 >> v1:172.21.15.29:6789/0 conn(0x7f52480b3140 legacy=0x7f52480b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 --2- 172.21.15.29:0/2676781749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52340039c0 0x7f5234003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 --2- 172.21.15.29:0/2676781749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5234004810 0x7f5234004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 --2- 172.21.15.29:0/2676781749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52480b3c30 0x7f52340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f524806d960 con 0x7f5234004810 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f524806dc40 con 0x7f52340039c0 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f523400b9d0 con 0x7f52480b3c30 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f523effd640 1 --2- 172.21.15.29:0/2676781749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5234004810 0x7f5234004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f52477fe640 1 --2- 172.21.15.29:0/2676781749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52340039c0 0x7f5234003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5247fff640 1 --2- 172.21.15.29:0/2676781749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52480b3c30 0x7f52340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f523effd640 1 -- 172.21.15.29:0/2676781749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52480b3c30 msgr2=0x7f52340051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:53.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f523effd640 1 --2- 172.21.15.29:0/2676781749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52480b3c30 0x7f52340051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f523effd640 1 -- 172.21.15.29:0/2676781749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52340039c0 msgr2=0x7f5234003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:53.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f523effd640 1 --2- 172.21.15.29:0/2676781749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52340039c0 0x7f5234003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f523effd640 1 -- 172.21.15.29:0/2676781749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5234003530 con 0x7f5234004810 2024-04-25T00:32:53.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5247fff640 1 --2- 172.21.15.29:0/2676781749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52480b3c30 0x7f52340051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f523effd640 1 --2- 172.21.15.29:0/2676781749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5234004810 0x7f5234004c90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5230009ae0 tx=0x7f523001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd103cd8961175d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300096b0 con 0x7f5234004810 2024-04-25T00:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52300a6480 con 0x7f5234004810 2024-04-25T00:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5246ffd640 1 -- 172.21.15.29:0/2676781749 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300a6870 con 0x7f5234004810 2024-04-25T00:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f524e074640 1 -- 172.21.15.29:0/2676781749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5234004810 msgr2=0x7f5234004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f524e074640 1 --2- 172.21.15.29:0/2676781749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5234004810 0x7f5234004c90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5230009ae0 tx=0x7f523001ca70 comp rx=0 tx=0).stop 2024-04-25T00:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f5247fff640 1 -- 172.21.15.29:0/2676781749 reap_dead start 2024-04-25T00:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f524e074640 1 -- 172.21.15.29:0/2676781749 shutdown_connections 2024-04-25T00:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f524e074640 1 -- 172.21.15.29:0/2676781749 >> 172.21.15.29:0/2676781749 conn(0x7f52480b1870 msgr2=0x7f52480b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f524e074640 1 -- 172.21.15.29:0/2676781749 shutdown_connections 2024-04-25T00:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.630+0000 7f524e074640 1 -- 172.21.15.29:0/2676781749 wait complete. 2024-04-25T00:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f524e074640 1 Processor -- start 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f524e074640 1 -- start start 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f524e074640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5234004810 0x7f52480cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f524e074640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52481f3f40 0x7f524813e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f524e074640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524813c4f0 0x7f5248071a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f524e074640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5248145a90 con 0x7f524813c4f0 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f524e074640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5248145910 con 0x7f52481f3f40 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f524e074640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f524806e440 con 0x7f5234004810 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f5247fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524813c4f0 0x7f5248071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f5247fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524813c4f0 0x7f5248071a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33698/0 (socket says 172.21.15.29:33698) 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f5247fff640 1 -- 172.21.15.29:0/3360489215 learned_addr learned my addr 172.21.15.29:0/3360489215 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f52477fe640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5234004810 0x7f52480cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f523effd640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52481f3f40 0x7f524813e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f5247fff640 1 -- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5234004810 msgr2=0x7f52480cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f5247fff640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5234004810 0x7f52480cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f5247fff640 1 -- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52481f3f40 msgr2=0x7f524813e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f5247fff640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52481f3f40 0x7f524813e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f5247fff640 1 -- 172.21.15.29:0/3360489215 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52481f4760 con 0x7f524813c4f0 2024-04-25T00:32:53.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f523effd640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52481f3f40 0x7f524813e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:53.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f5247fff640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524813c4f0 0x7f5248071a70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5248074fe0 tx=0x7f524000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:53.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f5244ff9640 1 -- 172.21.15.29:0/3360489215 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240016620 con 0x7f524813c4f0 2024-04-25T00:32:53.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f5244ff9640 1 -- 172.21.15.29:0/3360489215 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f524001b070 con 0x7f524813c4f0 2024-04-25T00:32:53.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f524e074640 1 -- 172.21.15.29:0/3360489215 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52481f4a40 con 0x7f524813c4f0 2024-04-25T00:32:53.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f5244ff9640 1 -- 172.21.15.29:0/3360489215 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52400169e0 con 0x7f524813c4f0 2024-04-25T00:32:53.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f524e074640 1 -- 172.21.15.29:0/3360489215 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f524815c530 con 0x7f524813c4f0 2024-04-25T00:32:53.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.634+0000 7f524e074640 1 -- 172.21.15.29:0/3360489215 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f524815c730 con 0x7f524813c4f0 2024-04-25T00:32:53.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.638+0000 7f5244ff9640 1 -- 172.21.15.29:0/3360489215 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7f5240016b80 con 0x7f524813c4f0 2024-04-25T00:32:53.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.638+0000 7f5244ff9640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5228067950 0x7f5228069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.638+0000 7f5244ff9640 1 -- 172.21.15.29:0/3360489215 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5240086c90 con 0x7f524813c4f0 2024-04-25T00:32:53.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.638+0000 7f52477fe640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5228067950 0x7f5228069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.638+0000 7f52477fe640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5228067950 0x7f5228069e10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5238001060 tx=0x7f5238005630 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:53.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.642+0000 7f5244ff9640 1 -- 172.21.15.29:0/3360489215 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f524815c730 con 0x7f524813c4f0 2024-04-25T00:32:53.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.802+0000 7f524e074640 1 -- 172.21.15.29:0/3360489215 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5248069270 con 0x7f524813c4f0 2024-04-25T00:32:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.806+0000 7f5244ff9640 1 -- 172.21.15.29:0/3360489215 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16621 (secure 0 0 0) 0x7f5240025020 con 0x7f524813c4f0 2024-04-25T00:32:53.810 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:53.810 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":33,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:32:35.426396+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:32:32.254456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:32:32.779372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:53.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.806+0000 7f524e074640 1 -- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5228067950 msgr2=0x7f5228069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.806+0000 7f524e074640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5228067950 0x7f5228069e10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5238001060 tx=0x7f5238005630 comp rx=0 tx=0).stop 2024-04-25T00:32:53.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.806+0000 7f524e074640 1 -- 172.21.15.29:0/3360489215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524813c4f0 msgr2=0x7f5248071a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.806+0000 7f524e074640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524813c4f0 0x7f5248071a70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5248074fe0 tx=0x7f524000f810 comp rx=0 tx=0).stop 2024-04-25T00:32:53.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.806+0000 7f524e074640 1 -- 172.21.15.29:0/3360489215 shutdown_connections 2024-04-25T00:32:53.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.806+0000 7f524e074640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524813c4f0 0x7f5248071a70 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.806+0000 7f524e074640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5228067950 0x7f5228069e10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.806+0000 7f524e074640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52481f3f40 0x7f524813e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.806+0000 7f524e074640 1 --2- 172.21.15.29:0/3360489215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5234004810 0x7f52480cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.806+0000 7f524e074640 1 -- 172.21.15.29:0/3360489215 >> 172.21.15.29:0/3360489215 conn(0x7f52480b1870 msgr2=0x7f5248142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.806+0000 7f524e074640 1 -- 172.21.15.29:0/3360489215 shutdown_connections 2024-04-25T00:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.806+0000 7f524e074640 1 -- 172.21.15.29:0/3360489215 wait complete. 2024-04-25T00:32:53.829 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:32:32.254456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:32:32.779372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:53.829 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:53.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.942+0000 7fd484760640 1 Processor -- start 2024-04-25T00:32:53.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.942+0000 7fd484760640 1 -- start start 2024-04-25T00:32:53.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd484760640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd47c0b3140 0x7fd47c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd484760640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd47c0b3c30 0x7fd47c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd484760640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd47c0b41e0 con 0x7fd47c0b4b10 2024-04-25T00:32:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd484760640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd47c06d960 con 0x7fd47c0b3c30 2024-04-25T00:32:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd484760640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd47c06dc40 con 0x7fd47c0b3140 2024-04-25T00:32:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd482cfd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd47c0b3c30 0x7fd47c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd482cfd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd47c0b3c30 0x7fd47c0ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33700/0 (socket says 172.21.15.29:33700) 2024-04-25T00:32:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd482cfd640 1 -- 172.21.15.29:0/545039840 learned_addr learned my addr 172.21.15.29:0/545039840 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd4824fc640 1 --2- 172.21.15.29:0/545039840 >> v2:172.21.15.157:3300/0 conn(0x7fd47c0b3140 0x7fd47c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd482cfd640 1 -- 172.21.15.29:0/545039840 >> v2:172.21.15.157:3300/0 conn(0x7fd47c0b3140 msgr2=0x7fd47c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd482cfd640 1 --2- 172.21.15.29:0/545039840 >> v2:172.21.15.157:3300/0 conn(0x7fd47c0b3140 0x7fd47c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd482cfd640 1 -- 172.21.15.29:0/545039840 >> v1:172.21.15.157:6789/0 conn(0x7fd47c0b4b10 legacy=0x7fd47c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd482cfd640 1 -- 172.21.15.29:0/545039840 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47c06df20 con 0x7fd47c0b3c30 2024-04-25T00:32:53.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd4824fc640 1 --2- 172.21.15.29:0/545039840 >> v2:172.21.15.157:3300/0 conn(0x7fd47c0b3140 0x7fd47c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:53.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.946+0000 7fd482cfd640 1 --2- 172.21.15.29:0/545039840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd47c0b3c30 0x7fd47c0ce240 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd474009e50 tx=0x7fd47401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7fd19ab135550d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:53.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd4814fa640 1 -- 172.21.15.29:0/545039840 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740a5670 con 0x7fd47c0b3c30 2024-04-25T00:32:53.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd4814fa640 1 -- 172.21.15.29:0/545039840 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4740a5810 con 0x7fd47c0b3c30 2024-04-25T00:32:53.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd4814fa640 1 -- 172.21.15.29:0/545039840 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740a5be0 con 0x7fd47c0b3c30 2024-04-25T00:32:53.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 -- 172.21.15.29:0/545039840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd47c0b3c30 msgr2=0x7fd47c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 --2- 172.21.15.29:0/545039840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd47c0b3c30 0x7fd47c0ce240 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd474009e50 tx=0x7fd47401c9b0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 -- 172.21.15.29:0/545039840 shutdown_connections 2024-04-25T00:32:53.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 --2- 172.21.15.29:0/545039840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd47c0b3c30 0x7fd47c0ce240 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 --2- 172.21.15.29:0/545039840 >> v2:172.21.15.157:3300/0 conn(0x7fd47c0b3140 0x7fd47c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 -- 172.21.15.29:0/545039840 >> 172.21.15.29:0/545039840 conn(0x7fd47c0b1870 msgr2=0x7fd47c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:53.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 -- 172.21.15.29:0/545039840 shutdown_connections 2024-04-25T00:32:53.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 -- 172.21.15.29:0/545039840 wait complete. 2024-04-25T00:32:53.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 Processor -- start 2024-04-25T00:32:53.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 -- start start 2024-04-25T00:32:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd47c0b3140 0x7fd47c15a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd47c0b3c30 0x7fd47c142a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd47c0b4b10 0x7fd47c13e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd47c168d80 con 0x7fd47c0b3140 2024-04-25T00:32:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd47c168c00 con 0x7fd47c0b3c30 2024-04-25T00:32:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd47c168f00 con 0x7fd47c0b4b10 2024-04-25T00:32:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd4824fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd47c0b3140 0x7fd47c15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd4824fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd47c0b3140 0x7fd47c15a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33706/0 (socket says 172.21.15.29:33706) 2024-04-25T00:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd4824fc640 1 -- 172.21.15.29:0/3437414403 learned_addr learned my addr 172.21.15.29:0/3437414403 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd4824fc640 1 -- 172.21.15.29:0/3437414403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd47c0b4b10 msgr2=0x7fd47c13e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd4824fc640 1 --2- 172.21.15.29:0/3437414403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd47c0b4b10 0x7fd47c13e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd4824fc640 1 -- 172.21.15.29:0/3437414403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd47c0b3c30 msgr2=0x7fd47c142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd4824fc640 1 --2- 172.21.15.29:0/3437414403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd47c0b3c30 0x7fd47c142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd4824fc640 1 -- 172.21.15.29:0/3437414403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47c13f5e0 con 0x7fd47c0b3140 2024-04-25T00:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd4824fc640 1 --2- 172.21.15.29:0/3437414403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd47c0b3140 0x7fd47c15a270 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c820 tx=0x7fd47800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd472ffd640 1 -- 172.21.15.29:0/3437414403 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478015070 con 0x7fd47c0b3140 2024-04-25T00:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd472ffd640 1 -- 172.21.15.29:0/3437414403 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd478007a30 con 0x7fd47c0b3140 2024-04-25T00:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 -- 172.21.15.29:0/3437414403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47c159790 con 0x7fd47c0b3140 2024-04-25T00:32:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd472ffd640 1 -- 172.21.15.29:0/3437414403 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478007e40 con 0x7fd47c0b3140 2024-04-25T00:32:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 -- 172.21.15.29:0/3437414403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd47c159d50 con 0x7fd47c0b3140 2024-04-25T00:32:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.950+0000 7fd484760640 1 -- 172.21.15.29:0/3437414403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd47c069270 con 0x7fd47c0b3140 2024-04-25T00:32:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.954+0000 7fd472ffd640 1 -- 172.21.15.29:0/3437414403 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7fd478018070 con 0x7fd47c0b3140 2024-04-25T00:32:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.954+0000 7fd472ffd640 1 --2- 172.21.15.29:0/3437414403 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd460067950 0x7fd460069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.954+0000 7fd472ffd640 1 -- 172.21.15.29:0/3437414403 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd478085c70 con 0x7fd47c0b3140 2024-04-25T00:32:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.954+0000 7fd481cfb640 1 --2- 172.21.15.29:0/3437414403 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd460067950 0x7fd460069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.954+0000 7fd481cfb640 1 --2- 172.21.15.29:0/3437414403 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd460067950 0x7fd460069e10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd47c1408c0 tx=0x7fd46c0079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:53.958+0000 7fd472ffd640 1 -- 172.21.15.29:0/3437414403 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd478056740 con 0x7fd47c0b3140 2024-04-25T00:32:54.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.110+0000 7fd484760640 1 -- 172.21.15.29:0/3437414403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd47c06efe0 con 0x7fd47c0b3140 2024-04-25T00:32:54.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.114+0000 7fd472ffd640 1 -- 172.21.15.29:0/3437414403 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7fd478056040 con 0x7fd47c0b3140 2024-04-25T00:32:54.117 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:54.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.114+0000 7fd484760640 1 -- 172.21.15.29:0/3437414403 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd460067950 msgr2=0x7fd460069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.114+0000 7fd484760640 1 --2- 172.21.15.29:0/3437414403 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd460067950 0x7fd460069e10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd47c1408c0 tx=0x7fd46c0079c0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.114+0000 7fd484760640 1 -- 172.21.15.29:0/3437414403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd47c0b3140 msgr2=0x7fd47c15a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.114+0000 7fd484760640 1 --2- 172.21.15.29:0/3437414403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd47c0b3140 0x7fd47c15a270 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c820 tx=0x7fd47800cce0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.114+0000 7fd484760640 1 -- 172.21.15.29:0/3437414403 shutdown_connections 2024-04-25T00:32:54.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.114+0000 7fd484760640 1 --2- 172.21.15.29:0/3437414403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd47c0b4b10 0x7fd47c13e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.114+0000 7fd484760640 1 --2- 172.21.15.29:0/3437414403 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd460067950 0x7fd460069e10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.114+0000 7fd484760640 1 --2- 172.21.15.29:0/3437414403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd47c0b3c30 0x7fd47c142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.114+0000 7fd484760640 1 --2- 172.21.15.29:0/3437414403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd47c0b3140 0x7fd47c15a270 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.114+0000 7fd484760640 1 -- 172.21.15.29:0/3437414403 >> 172.21.15.29:0/3437414403 conn(0x7fd47c0b1870 msgr2=0x7fd47c158dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:54.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.114+0000 7fd484760640 1 -- 172.21.15.29:0/3437414403 shutdown_connections 2024-04-25T00:32:54.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.114+0000 7fd484760640 1 -- 172.21.15.29:0/3437414403 wait complete. 2024-04-25T00:32:54.135 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.254+0000 7f48c2310640 1 Processor -- start 2024-04-25T00:32:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.254+0000 7f48c2310640 1 -- start start 2024-04-25T00:32:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48c2310640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f48bc0d0a80 0x7f48bc06c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48c2310640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48bc15e7a0 con 0x7f48bc06d320 2024-04-25T00:32:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48c2310640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48bc0d10f0 con 0x7f48bc15e3c0 2024-04-25T00:32:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48c2310640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f48bc06ca70 con 0x7f48bc0d0a80 2024-04-25T00:32:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48bb7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f48bc06d320 0x7f48bc0cd040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50088/0 (socket says 172.21.15.29:50088) 2024-04-25T00:32:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48bb7fe640 1 -- 172.21.15.29:0/2009233106 learned_addr learned my addr 172.21.15.29:0/2009233106 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48c08ad640 1 --2- 172.21.15.29:0/2009233106 >> v2:172.21.15.157:3300/0 conn(0x7f48bc0d0a80 0x7f48bc06c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1577491136 0 0) 0x7f48bc15e7a0 con 0x7f48bc06d320 2024-04-25T00:32:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48a4003530 con 0x7f48bc06d320 2024-04-25T00:32:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1060431168 0 0) 0x7f48a4003530 con 0x7f48bc06d320 2024-04-25T00:32:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 >> v2:172.21.15.157:3300/0 conn(0x7f48bc0d0a80 msgr2=0x7f48bc06c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 --2- 172.21.15.29:0/2009233106 >> v2:172.21.15.157:3300/0 conn(0x7f48bc0d0a80 0x7f48bc06c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 >> v1:172.21.15.157:6789/0 conn(0x7f48bc15e3c0 legacy=0x7f48bc06cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48bc0cc560 con 0x7f48bc06d320 2024-04-25T00:32:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f48b0001680 con 0x7f48bc06d320 2024-04-25T00:32:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 >> v1:172.21.15.29:6789/0 conn(0x7f48bc06d320 legacy=0x7f48bc0cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 --2- 172.21.15.29:0/2009233106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48a40039c0 0x7f48a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 --2- 172.21.15.29:0/2009233106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a4004810 0x7f48a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 --2- 172.21.15.29:0/2009233106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48a40051d0 0x7f48a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc06ca70 con 0x7f48a40051d0 2024-04-25T00:32:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f48a400c080 con 0x7f48a40039c0 2024-04-25T00:32:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f48a400c280 con 0x7f48a4004810 2024-04-25T00:32:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48c08ad640 1 --2- 172.21.15.29:0/2009233106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48a40051d0 0x7f48a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48bbfff640 1 --2- 172.21.15.29:0/2009233106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48a40039c0 0x7f48a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48bb7fe640 1 --2- 172.21.15.29:0/2009233106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a4004810 0x7f48a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48c08ad640 1 -- 172.21.15.29:0/2009233106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a4004810 msgr2=0x7f48a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48c08ad640 1 --2- 172.21.15.29:0/2009233106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a4004810 0x7f48a4004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48c08ad640 1 -- 172.21.15.29:0/2009233106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48a40039c0 msgr2=0x7f48a4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48c08ad640 1 --2- 172.21.15.29:0/2009233106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48a40039c0 0x7f48a4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.258+0000 7f48c08ad640 1 -- 172.21.15.29:0/2009233106 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a4003530 con 0x7f48a40051d0 2024-04-25T00:32:54.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c08ad640 1 -- 172.21.15.29:0/2009233106 reap_dead start 2024-04-25T00:32:54.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c08ad640 1 --2- 172.21.15.29:0/2009233106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48a40051d0 0x7f48a400b9d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009fa0 tx=0x7f48b401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814216d12ec9db8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b40b2070 con 0x7f48a40051d0 2024-04-25T00:32:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b40a5d50 con 0x7f48a40051d0 2024-04-25T00:32:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48baffd640 1 -- 172.21.15.29:0/2009233106 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4009bf0 con 0x7f48a40051d0 2024-04-25T00:32:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 -- 172.21.15.29:0/2009233106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48a40051d0 msgr2=0x7f48a400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 --2- 172.21.15.29:0/2009233106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48a40051d0 0x7f48a400b9d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009fa0 tx=0x7f48b401cc80 comp rx=0 tx=0).stop 2024-04-25T00:32:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 -- 172.21.15.29:0/2009233106 shutdown_connections 2024-04-25T00:32:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 --2- 172.21.15.29:0/2009233106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48a40051d0 0x7f48a400b9d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 -- 172.21.15.29:0/2009233106 >> 172.21.15.29:0/2009233106 conn(0x7f48bc0d1d70 msgr2=0x7f48bc0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 -- 172.21.15.29:0/2009233106 shutdown_connections 2024-04-25T00:32:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 -- 172.21.15.29:0/2009233106 wait complete. 2024-04-25T00:32:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 Processor -- start 2024-04-25T00:32:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 -- start start 2024-04-25T00:32:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a40051d0 0x7f48bc157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48bc0b7360 0x7f48bc0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48bc0b77e0 0x7f48bc0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc0ccb20 con 0x7f48bc0b7360 2024-04-25T00:32:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc0d0e90 con 0x7f48bc0b77e0 2024-04-25T00:32:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c2310640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f48bc06c530 con 0x7f48a40051d0 2024-04-25T00:32:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48bb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48bc0b7360 0x7f48bc0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48bb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48bc0b7360 0x7f48bc0cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33718/0 (socket says 172.21.15.29:33718) 2024-04-25T00:32:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48bb7fe640 1 -- 172.21.15.29:0/1912174021 learned_addr learned my addr 172.21.15.29:0/1912174021 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c08ad640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48bc0b77e0 0x7f48bc0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48bb7fe640 1 -- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a40051d0 msgr2=0x7f48bc157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48bbfff640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a40051d0 0x7f48bc157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48bb7fe640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a40051d0 0x7f48bc157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48bb7fe640 1 -- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48bc0b77e0 msgr2=0x7f48bc0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48bb7fe640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48bc0b77e0 0x7f48bc0cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48bb7fe640 1 -- 172.21.15.29:0/1912174021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48bc157770 con 0x7f48bc0b7360 2024-04-25T00:32:54.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.262+0000 7f48c08ad640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48bc0b77e0 0x7f48bc0cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:54.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.266+0000 7f48bb7fe640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48bc0b7360 0x7f48bc0cda90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f48b000d7a0 tx=0x7f48b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.266+0000 7f48b8ff9640 1 -- 172.21.15.29:0/1912174021 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0019070 con 0x7f48bc0b7360 2024-04-25T00:32:54.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.266+0000 7f48b8ff9640 1 -- 172.21.15.29:0/1912174021 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b000de50 con 0x7f48bc0b7360 2024-04-25T00:32:54.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.266+0000 7f48b8ff9640 1 -- 172.21.15.29:0/1912174021 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b00047f0 con 0x7f48bc0b7360 2024-04-25T00:32:54.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.266+0000 7f48c2310640 1 -- 172.21.15.29:0/1912174021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48bc1582b0 con 0x7f48bc0b7360 2024-04-25T00:32:54.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.266+0000 7f48c2310640 1 -- 172.21.15.29:0/1912174021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48bc0b53a0 con 0x7f48bc0b7360 2024-04-25T00:32:54.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.266+0000 7f48c2310640 1 -- 172.21.15.29:0/1912174021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48bc069270 con 0x7f48bc0b7360 2024-04-25T00:32:54.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.266+0000 7f48b8ff9640 1 -- 172.21.15.29:0/1912174021 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7f48b001c070 con 0x7f48bc0b7360 2024-04-25T00:32:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.266+0000 7f48b8ff9640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f48a8067950 0x7f48a8069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.266+0000 7f48b8ff9640 1 -- 172.21.15.29:0/1912174021 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f48b0085790 con 0x7f48bc0b7360 2024-04-25T00:32:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.266+0000 7f48bbfff640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f48a8067950 0x7f48a8069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.266+0000 7f48bbfff640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f48a8067950 0x7f48a8069e10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f48ac0015a0 tx=0x7f48ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.270+0000 7f48b8ff9640 1 -- 172.21.15.29:0/1912174021 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f48b00561d0 con 0x7f48bc0b7360 2024-04-25T00:32:54.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48c2310640 1 -- 172.21.15.29:0/1912174021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f48bc158480 con 0x7f48bc0b7360 2024-04-25T00:32:54.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48b8ff9640 1 -- 172.21.15.29:0/1912174021 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f48b0083e20 con 0x7f48bc0b7360 2024-04-25T00:32:54.430 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:54.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48c2310640 1 -- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f48a8067950 msgr2=0x7f48a8069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48c2310640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f48a8067950 0x7f48a8069e10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f48ac0015a0 tx=0x7f48ac009040 comp rx=0 tx=0).stop 2024-04-25T00:32:54.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48c2310640 1 -- 172.21.15.29:0/1912174021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48bc0b7360 msgr2=0x7f48bc0cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48c2310640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48bc0b7360 0x7f48bc0cda90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f48b000d7a0 tx=0x7f48b000dc60 comp rx=0 tx=0).stop 2024-04-25T00:32:54.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48c2310640 1 -- 172.21.15.29:0/1912174021 shutdown_connections 2024-04-25T00:32:54.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48c2310640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f48a8067950 0x7f48a8069e10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48c2310640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48bc0b77e0 0x7f48bc0cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48c2310640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48bc0b7360 0x7f48bc0cda90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48c2310640 1 --2- 172.21.15.29:0/1912174021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48a40051d0 0x7f48bc157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48c2310640 1 -- 172.21.15.29:0/1912174021 >> 172.21.15.29:0/1912174021 conn(0x7f48bc0d1d70 msgr2=0x7f48bc0d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:54.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48c2310640 1 -- 172.21.15.29:0/1912174021 shutdown_connections 2024-04-25T00:32:54.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.426+0000 7f48c2310640 1 -- 172.21.15.29:0/1912174021 wait complete. 2024-04-25T00:32:54.448 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.562+0000 7f158b36e640 1 Processor -- start 2024-04-25T00:32:54.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.566+0000 7f158b36e640 1 -- start start 2024-04-25T00:32:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.566+0000 7f158b36e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f158415bd60 0x7f158406b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.566+0000 7f158b36e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f158406dd10 0x7f158406b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.566+0000 7f158b36e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f158406e1a0 con 0x7f15840cb330 2024-04-25T00:32:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.566+0000 7f158b36e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f158406edb0 con 0x7f158415bd60 2024-04-25T00:32:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.566+0000 7f158b36e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f158406e480 con 0x7f158406dd10 2024-04-25T00:32:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.566+0000 7f158910a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f158415bd60 0x7f158406b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.566+0000 7f158910a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f158415bd60 0x7f158406b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33728/0 (socket says 172.21.15.29:33728) 2024-04-25T00:32:54.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.566+0000 7f158910a640 1 -- 172.21.15.29:0/2360437951 learned_addr learned my addr 172.21.15.29:0/2360437951 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:54.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.566+0000 7f158910a640 1 -- 172.21.15.29:0/2360437951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f158406dd10 msgr2=0x7f158406b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.566+0000 7f158910a640 1 --2- 172.21.15.29:0/2360437951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f158406dd10 0x7f158406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.566+0000 7f158910a640 1 -- 172.21.15.29:0/2360437951 >> v1:172.21.15.157:6789/0 conn(0x7f15840cb330 legacy=0x7f158415f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.566+0000 7f158910a640 1 -- 172.21.15.29:0/2360437951 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158406e760 con 0x7f158415bd60 2024-04-25T00:32:54.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f158910a640 1 --2- 172.21.15.29:0/2360437951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f158415bd60 0x7f158406b210 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1580009870 tx=0x7f158001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c25c6be77d6b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f157bfff640 1 -- 172.21.15.29:0/2360437951 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15800aa430 con 0x7f158415bd60 2024-04-25T00:32:54.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f157bfff640 1 -- 172.21.15.29:0/2360437951 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15800af070 con 0x7f158415bd60 2024-04-25T00:32:54.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f157bfff640 1 -- 172.21.15.29:0/2360437951 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15800aa7f0 con 0x7f158415bd60 2024-04-25T00:32:54.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f158b36e640 1 -- 172.21.15.29:0/2360437951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f158415bd60 msgr2=0x7f158406b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f158b36e640 1 --2- 172.21.15.29:0/2360437951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f158415bd60 0x7f158406b210 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1580009870 tx=0x7f158001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f158b36e640 1 -- 172.21.15.29:0/2360437951 shutdown_connections 2024-04-25T00:32:54.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f158b36e640 1 --2- 172.21.15.29:0/2360437951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f158406dd10 0x7f158406b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f158b36e640 1 --2- 172.21.15.29:0/2360437951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f158415bd60 0x7f158406b210 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f158b36e640 1 -- 172.21.15.29:0/2360437951 >> 172.21.15.29:0/2360437951 conn(0x7f158406c320 msgr2=0x7f158406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:54.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f158b36e640 1 -- 172.21.15.29:0/2360437951 shutdown_connections 2024-04-25T00:32:54.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f158b36e640 1 -- 172.21.15.29:0/2360437951 wait complete. 2024-04-25T00:32:54.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f158b36e640 1 Processor -- start 2024-04-25T00:32:54.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.570+0000 7f158b36e640 1 -- start start 2024-04-25T00:32:54.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158b36e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15840cb330 0x7f1584159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158b36e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f158415bd60 0x7f1584153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158b36e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1584152220 0x7f15840cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158b36e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1584168f30 con 0x7f15840cb330 2024-04-25T00:32:54.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158b36e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15840cdea0 con 0x7f1584152220 2024-04-25T00:32:54.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158b36e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f15841690b0 con 0x7f158415bd60 2024-04-25T00:32:54.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158910a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15840cb330 0x7f1584159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158910a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15840cb330 0x7f1584159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33740/0 (socket says 172.21.15.29:33740) 2024-04-25T00:32:54.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158910a640 1 -- 172.21.15.29:0/3000150415 learned_addr learned my addr 172.21.15.29:0/3000150415 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:54.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158910a640 1 -- 172.21.15.29:0/3000150415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f158415bd60 msgr2=0x7f1584153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158910a640 1 --2- 172.21.15.29:0/3000150415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f158415bd60 0x7f1584153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158910a640 1 -- 172.21.15.29:0/3000150415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1584152220 msgr2=0x7f15840cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158910a640 1 --2- 172.21.15.29:0/3000150415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1584152220 0x7f15840cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158910a640 1 -- 172.21.15.29:0/3000150415 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1584151fd0 con 0x7f15840cb330 2024-04-25T00:32:54.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158910a640 1 --2- 172.21.15.29:0/3000150415 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15840cb330 0x7f1584159f30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f158001cd90 tx=0x7f158001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f1579ffb640 1 -- 172.21.15.29:0/3000150415 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15800bd5f0 con 0x7f15840cb330 2024-04-25T00:32:54.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f1579ffb640 1 -- 172.21.15.29:0/3000150415 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15800bd7b0 con 0x7f15840cb330 2024-04-25T00:32:54.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158b36e640 1 -- 172.21.15.29:0/3000150415 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1584151cf0 con 0x7f15840cb330 2024-04-25T00:32:54.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f1579ffb640 1 -- 172.21.15.29:0/3000150415 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15800bdba0 con 0x7f15840cb330 2024-04-25T00:32:54.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158b36e640 1 -- 172.21.15.29:0/3000150415 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15840cebc0 con 0x7f15840cb330 2024-04-25T00:32:54.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f158b36e640 1 -- 172.21.15.29:0/3000150415 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1584069270 con 0x7f15840cb330 2024-04-25T00:32:54.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f1579ffb640 1 -- 172.21.15.29:0/3000150415 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f15800b33f0 con 0x7f15840cb330 2024-04-25T00:32:54.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f1579ffb640 1 --2- 172.21.15.29:0/3000150415 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1568067a70 0x7f1568069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f1579ffb640 1 -- 172.21.15.29:0/3000150415 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1580120560 con 0x7f15840cb330 2024-04-25T00:32:54.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.574+0000 7f1588909640 1 --2- 172.21.15.29:0/3000150415 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1568067a70 0x7f1568069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.578+0000 7f1588909640 1 --2- 172.21.15.29:0/3000150415 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1568067a70 0x7f1568069f30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1584154e10 tx=0x7f1574006ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.578+0000 7f1579ffb640 1 -- 172.21.15.29:0/3000150415 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f15800c3060 con 0x7f15840cb330 2024-04-25T00:32:54.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.742+0000 7f158b36e640 1 -- 172.21.15.29:0/3000150415 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f158415fe40 con 0x7f15840cb330 2024-04-25T00:32:54.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.746+0000 7f1579ffb640 1 -- 172.21.15.29:0/3000150415 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f1580016110 con 0x7f15840cb330 2024-04-25T00:32:54.750 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:54.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.746+0000 7f158b36e640 1 -- 172.21.15.29:0/3000150415 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1568067a70 msgr2=0x7f1568069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.746+0000 7f158b36e640 1 --2- 172.21.15.29:0/3000150415 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1568067a70 0x7f1568069f30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1584154e10 tx=0x7f1574006ca0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.746+0000 7f158b36e640 1 -- 172.21.15.29:0/3000150415 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15840cb330 msgr2=0x7f1584159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.746+0000 7f158b36e640 1 --2- 172.21.15.29:0/3000150415 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15840cb330 0x7f1584159f30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f158001cd90 tx=0x7f158001ad30 comp rx=0 tx=0).stop 2024-04-25T00:32:54.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.746+0000 7f158b36e640 1 -- 172.21.15.29:0/3000150415 shutdown_connections 2024-04-25T00:32:54.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.746+0000 7f158b36e640 1 --2- 172.21.15.29:0/3000150415 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1568067a70 0x7f1568069f30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.746+0000 7f158b36e640 1 --2- 172.21.15.29:0/3000150415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1584152220 0x7f15840cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.746+0000 7f158b36e640 1 --2- 172.21.15.29:0/3000150415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f158415bd60 0x7f1584153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.746+0000 7f158b36e640 1 --2- 172.21.15.29:0/3000150415 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15840cb330 0x7f1584159f30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.746+0000 7f158b36e640 1 -- 172.21.15.29:0/3000150415 >> 172.21.15.29:0/3000150415 conn(0x7f158406c320 msgr2=0x7f15841557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:54.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.746+0000 7f158b36e640 1 -- 172.21.15.29:0/3000150415 shutdown_connections 2024-04-25T00:32:54.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.746+0000 7f158b36e640 1 -- 172.21.15.29:0/3000150415 wait complete. 2024-04-25T00:32:54.770 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:32:54.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.890+0000 7f04538f0640 1 Processor -- start 2024-04-25T00:32:54.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.890+0000 7f04538f0640 1 -- start start 2024-04-25T00:32:54.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f04538f0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f044c0b3140 0x7f044c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f04538f0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f044c0b4b10 0x7f044c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f04538f0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f044c0b41e0 con 0x7f044c0b3a60 2024-04-25T00:32:54.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f04538f0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f044c06d960 con 0x7f044c0b4b10 2024-04-25T00:32:54.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f04538f0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f044c06dc40 con 0x7f044c0b3140 2024-04-25T00:32:54.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f0450e8b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f044c0b4b10 0x7f044c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f0450e8b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f044c0b4b10 0x7f044c0b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33752/0 (socket says 172.21.15.29:33752) 2024-04-25T00:32:54.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f0450e8b640 1 -- 172.21.15.29:0/1216941478 learned_addr learned my addr 172.21.15.29:0/1216941478 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:54.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f045168c640 1 --2- 172.21.15.29:0/1216941478 >> v2:172.21.15.157:3300/0 conn(0x7f044c0b3140 0x7f044c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f0450e8b640 1 -- 172.21.15.29:0/1216941478 >> v2:172.21.15.157:3300/0 conn(0x7f044c0b3140 msgr2=0x7f044c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f0450e8b640 1 --2- 172.21.15.29:0/1216941478 >> v2:172.21.15.157:3300/0 conn(0x7f044c0b3140 0x7f044c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f0450e8b640 1 -- 172.21.15.29:0/1216941478 >> v1:172.21.15.157:6789/0 conn(0x7f044c0b3a60 legacy=0x7f044c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:54.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f0450e8b640 1 -- 172.21.15.29:0/1216941478 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c06df20 con 0x7f044c0b4b10 2024-04-25T00:32:54.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f0450e8b640 1 --2- 172.21.15.29:0/1216941478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f044c0b4b10 0x7f044c0b3520 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f043c009870 tx=0x7f043c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d2cdacc23268e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f0443fff640 1 -- 172.21.15.29:0/1216941478 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c0b0070 con 0x7f044c0b4b10 2024-04-25T00:32:54.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f0443fff640 1 -- 172.21.15.29:0/1216941478 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f043c0ab600 con 0x7f044c0b4b10 2024-04-25T00:32:54.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.894+0000 7f0443fff640 1 -- 172.21.15.29:0/1216941478 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c0a4e60 con 0x7f044c0b4b10 2024-04-25T00:32:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 -- 172.21.15.29:0/1216941478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f044c0b4b10 msgr2=0x7f044c0b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 --2- 172.21.15.29:0/1216941478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f044c0b4b10 0x7f044c0b3520 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f043c009870 tx=0x7f043c01c4e0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 -- 172.21.15.29:0/1216941478 shutdown_connections 2024-04-25T00:32:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 --2- 172.21.15.29:0/1216941478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f044c0b4b10 0x7f044c0b3520 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 --2- 172.21.15.29:0/1216941478 >> v2:172.21.15.157:3300/0 conn(0x7f044c0b3140 0x7f044c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 -- 172.21.15.29:0/1216941478 >> 172.21.15.29:0/1216941478 conn(0x7f044c0b1870 msgr2=0x7f044c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 -- 172.21.15.29:0/1216941478 shutdown_connections 2024-04-25T00:32:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 -- 172.21.15.29:0/1216941478 wait complete. 2024-04-25T00:32:54.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 Processor -- start 2024-04-25T00:32:54.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 -- start start 2024-04-25T00:32:54.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f044c0b3140 0x7f044c139210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f044c0b3a60 0x7f044c154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f044c0b4b10 0x7f044c15a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f044c168b70 con 0x7f044c0b3a60 2024-04-25T00:32:54.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1689f0 con 0x7f044c0b3140 2024-04-25T00:32:54.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f044c168cf0 con 0x7f044c0b4b10 2024-04-25T00:32:54.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f0450e8b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f044c0b3a60 0x7f044c154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f0450e8b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f044c0b3a60 0x7f044c154790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33762/0 (socket says 172.21.15.29:33762) 2024-04-25T00:32:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f0450e8b640 1 -- 172.21.15.29:0/3862600096 learned_addr learned my addr 172.21.15.29:0/3862600096 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f045168c640 1 --2- 172.21.15.29:0/3862600096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f044c0b3140 0x7f044c139210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f0450e8b640 1 -- 172.21.15.29:0/3862600096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f044c0b4b10 msgr2=0x7f044c15a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f0450e8b640 1 --2- 172.21.15.29:0/3862600096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f044c0b4b10 0x7f044c15a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f0450e8b640 1 -- 172.21.15.29:0/3862600096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f044c0b3140 msgr2=0x7f044c139210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f0450e8b640 1 --2- 172.21.15.29:0/3862600096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f044c0b3140 0x7f044c139210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f0450e8b640 1 -- 172.21.15.29:0/3862600096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c158f40 con 0x7f044c0b3a60 2024-04-25T00:32:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f0450e8b640 1 --2- 172.21.15.29:0/3862600096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f044c0b3a60 0x7f044c154790 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f043c01ab90 tx=0x7f043c019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f0441ffb640 1 -- 172.21.15.29:0/3862600096 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c0b0040 con 0x7f044c0b3a60 2024-04-25T00:32:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f0441ffb640 1 -- 172.21.15.29:0/3862600096 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f043c0bc070 con 0x7f044c0b3a60 2024-04-25T00:32:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f04538f0640 1 -- 172.21.15.29:0/3862600096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044c139f40 con 0x7f044c0b3a60 2024-04-25T00:32:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.898+0000 7f0441ffb640 1 -- 172.21.15.29:0/3862600096 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c0b6660 con 0x7f044c0b3a60 2024-04-25T00:32:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.902+0000 7f04538f0640 1 -- 172.21.15.29:0/3862600096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044c13a220 con 0x7f044c0b3a60 2024-04-25T00:32:54.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.902+0000 7f04538f0640 1 -- 172.21.15.29:0/3862600096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f044c069270 con 0x7f044c0b3a60 2024-04-25T00:32:54.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.902+0000 7f0441ffb640 1 -- 172.21.15.29:0/3862600096 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f043c0b6800 con 0x7f044c0b3a60 2024-04-25T00:32:54.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.902+0000 7f0441ffb640 1 --2- 172.21.15.29:0/3862600096 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0430067aa0 0x7f0430069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:54.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.902+0000 7f0441ffb640 1 -- 172.21.15.29:0/3862600096 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f043c120400 con 0x7f044c0b3a60 2024-04-25T00:32:54.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.902+0000 7f045168c640 1 --2- 172.21.15.29:0/3862600096 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0430067aa0 0x7f0430069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:54.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.902+0000 7f045168c640 1 --2- 172.21.15.29:0/3862600096 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0430067aa0 0x7f0430069f60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f044800a8d0 tx=0x7f0448008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:54.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:54.906+0000 7f0441ffb640 1 -- 172.21.15.29:0/3862600096 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f043c0160d0 con 0x7f044c0b3a60 2024-04-25T00:32:55.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.078+0000 7f04538f0640 1 -- 172.21.15.29:0/3862600096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f044c154cd0 con 0x7f044c0b3a60 2024-04-25T00:32:55.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.078+0000 7f0441ffb640 1 -- 172.21.15.29:0/3862600096 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f043c0f0b10 con 0x7f044c0b3a60 2024-04-25T00:32:55.085 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:32:55.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.082+0000 7f04538f0640 1 -- 172.21.15.29:0/3862600096 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0430067aa0 msgr2=0x7f0430069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.082+0000 7f04538f0640 1 --2- 172.21.15.29:0/3862600096 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0430067aa0 0x7f0430069f60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f044800a8d0 tx=0x7f0448008040 comp rx=0 tx=0).stop 2024-04-25T00:32:55.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.082+0000 7f04538f0640 1 -- 172.21.15.29:0/3862600096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f044c0b3a60 msgr2=0x7f044c154790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.082+0000 7f04538f0640 1 --2- 172.21.15.29:0/3862600096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f044c0b3a60 0x7f044c154790 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f043c01ab90 tx=0x7f043c019280 comp rx=0 tx=0).stop 2024-04-25T00:32:55.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.082+0000 7f04538f0640 1 -- 172.21.15.29:0/3862600096 shutdown_connections 2024-04-25T00:32:55.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.082+0000 7f04538f0640 1 --2- 172.21.15.29:0/3862600096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f044c0b4b10 0x7f044c15a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.082+0000 7f04538f0640 1 --2- 172.21.15.29:0/3862600096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f044c0b3a60 0x7f044c154790 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.082+0000 7f04538f0640 1 --2- 172.21.15.29:0/3862600096 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0430067aa0 0x7f0430069f60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.082+0000 7f04538f0640 1 --2- 172.21.15.29:0/3862600096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f044c0b3140 0x7f044c139210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.082+0000 7f04538f0640 1 -- 172.21.15.29:0/3862600096 >> 172.21.15.29:0/3862600096 conn(0x7f044c0b1870 msgr2=0x7f044c154fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:55.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.082+0000 7f04538f0640 1 -- 172.21.15.29:0/3862600096 shutdown_connections 2024-04-25T00:32:55.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.082+0000 7f04538f0640 1 -- 172.21.15.29:0/3862600096 wait complete. 2024-04-25T00:32:55.104 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-04-25T00:32:55.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:55.104 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:32:55.113 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[37598]: fuse finished with error 0 and tester_r 0 2024-04-25T00:32:55.115 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:32:55.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:55.115 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:32:55.282 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:32:55.282 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:32:55.282 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:55.289 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:55.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d23b12640 1 Processor -- start 2024-04-25T00:32:55.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d23b12640 1 -- start start 2024-04-25T00:32:55.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d23b12640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d1c0b2b70 0x7f7d1c151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d23b12640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d1c0b3130 con 0x7f7d1c15d980 2024-04-25T00:32:55.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d23b12640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d1c0b3410 con 0x7f7d1c06dce0 2024-04-25T00:32:55.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d23b12640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7d1c0cf560 con 0x7f7d1c0b2b70 2024-04-25T00:32:55.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d218ae640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7d1c15d980 0x7f7d1c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50100/0 (socket says 172.21.15.29:50100) 2024-04-25T00:32:55.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d218ae640 1 -- 172.21.15.29:0/3369046850 learned_addr learned my addr 172.21.15.29:0/3369046850 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:55.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d220af640 1 --2- 172.21.15.29:0/3369046850 >> v2:172.21.15.157:3300/0 conn(0x7f7d1c0b2b70 0x7f7d1c151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:55.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2677295356 0 0) 0x7f7d1c0b3130 con 0x7f7d1c15d980 2024-04-25T00:32:55.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d04004640 con 0x7f7d1c15d980 2024-04-25T00:32:55.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 819996374 0 0) 0x7f7d04004640 con 0x7f7d1c15d980 2024-04-25T00:32:55.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 >> v2:172.21.15.157:3300/0 conn(0x7f7d1c0b2b70 msgr2=0x7f7d1c151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:55.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 --2- 172.21.15.29:0/3369046850 >> v2:172.21.15.157:3300/0 conn(0x7f7d1c0b2b70 0x7f7d1c151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 >> v1:172.21.15.157:6789/0 conn(0x7f7d1c06dce0 legacy=0x7f7d1c150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c0b36f0 con 0x7f7d1c15d980 2024-04-25T00:32:55.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f7d0c001a00 con 0x7f7d1c15d980 2024-04-25T00:32:55.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 >> v1:172.21.15.29:6789/0 conn(0x7f7d1c15d980 legacy=0x7f7d1c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 --2- 172.21.15.29:0/3369046850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d04004ad0 0x7f7d04004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 --2- 172.21.15.29:0/3369046850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d04005920 0x7f7d04005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 --2- 172.21.15.29:0/3369046850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d040062e0 0x7f7d04003450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c0cf560 con 0x7f7d040062e0 2024-04-25T00:32:55.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04003af0 con 0x7f7d04004ad0 2024-04-25T00:32:55.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7d04003cf0 con 0x7f7d04005920 2024-04-25T00:32:55.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d220af640 1 --2- 172.21.15.29:0/3369046850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d040062e0 0x7f7d04003450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:55.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d210ad640 1 --2- 172.21.15.29:0/3369046850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d04005920 0x7f7d04005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:55.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d220af640 1 -- 172.21.15.29:0/3369046850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d04005920 msgr2=0x7f7d04005da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:55.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d220af640 1 --2- 172.21.15.29:0/3369046850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d04005920 0x7f7d04005da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d220af640 1 -- 172.21.15.29:0/3369046850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d04004ad0 msgr2=0x7f7d04004ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:55.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d220af640 1 --2- 172.21.15.29:0/3369046850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d04004ad0 0x7f7d04004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.446+0000 7f7d220af640 1 -- 172.21.15.29:0/3369046850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d04004640 con 0x7f7d040062e0 2024-04-25T00:32:55.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.450+0000 7f7d220af640 1 -- 172.21.15.29:0/3369046850 reap_dead start 2024-04-25T00:32:55.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.450+0000 7f7d220af640 1 --2- 172.21.15.29:0/3369046850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d040062e0 0x7f7d04003450 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009ae0 tx=0x7f7d1401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765c6406fde640ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:55.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.450+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d140096b0 con 0x7f7d040062e0 2024-04-25T00:32:55.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.450+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d14004020 con 0x7f7d040062e0 2024-04-25T00:32:55.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.450+0000 7f7d208ac640 1 -- 172.21.15.29:0/3369046850 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14004410 con 0x7f7d040062e0 2024-04-25T00:32:55.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.450+0000 7f7d23b12640 1 -- 172.21.15.29:0/3369046850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d040062e0 msgr2=0x7f7d04003450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.450+0000 7f7d23b12640 1 --2- 172.21.15.29:0/3369046850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d040062e0 0x7f7d04003450 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009ae0 tx=0x7f7d1401cab0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.450+0000 7f7d23b12640 1 -- 172.21.15.29:0/3369046850 shutdown_connections 2024-04-25T00:32:55.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.450+0000 7f7d23b12640 1 --2- 172.21.15.29:0/3369046850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d040062e0 0x7f7d04003450 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.450+0000 7f7d23b12640 1 -- 172.21.15.29:0/3369046850 >> 172.21.15.29:0/3369046850 conn(0x7f7d1c0b1870 msgr2=0x7f7d1c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:55.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.450+0000 7f7d23b12640 1 -- 172.21.15.29:0/3369046850 shutdown_connections 2024-04-25T00:32:55.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.450+0000 7f7d23b12640 1 -- 172.21.15.29:0/3369046850 wait complete. 2024-04-25T00:32:55.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d23b12640 1 Processor -- start 2024-04-25T00:32:55.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d23b12640 1 -- start start 2024-04-25T00:32:55.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d23b12640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d040062e0 0x7f7d1c0ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d23b12640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c148430 0x7f7d1c1424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d23b12640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d1c15bc40 0x7f7d1c15c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d23b12640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c15dd60 con 0x7f7d1c148430 2024-04-25T00:32:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d23b12640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c0b3cb0 con 0x7f7d040062e0 2024-04-25T00:32:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d23b12640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1c150df0 con 0x7f7d1c15bc40 2024-04-25T00:32:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d210ad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c148430 0x7f7d1c1424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d210ad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c148430 0x7f7d1c1424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33782/0 (socket says 172.21.15.29:33782) 2024-04-25T00:32:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d210ad640 1 -- 172.21.15.29:0/2829515472 learned_addr learned my addr 172.21.15.29:0/2829515472 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d218ae640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d040062e0 0x7f7d1c0ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d220af640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d1c15bc40 0x7f7d1c15c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d210ad640 1 -- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d1c15bc40 msgr2=0x7f7d1c15c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d210ad640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d1c15bc40 0x7f7d1c15c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d210ad640 1 -- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d040062e0 msgr2=0x7f7d1c0ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d210ad640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d040062e0 0x7f7d1c0ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d210ad640 1 -- 172.21.15.29:0/2829515472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c0ae7f0 con 0x7f7d1c148430 2024-04-25T00:32:55.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d210ad640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c148430 0x7f7d1c1424b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000d470 tx=0x7f7d1000d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:55.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d1a7fc640 1 -- 172.21.15.29:0/2829515472 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10018070 con 0x7f7d1c148430 2024-04-25T00:32:55.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d1a7fc640 1 -- 172.21.15.29:0/2829515472 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d10003620 con 0x7f7d1c148430 2024-04-25T00:32:55.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d23b12640 1 -- 172.21.15.29:0/2829515472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1c0aea90 con 0x7f7d1c148430 2024-04-25T00:32:55.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d1a7fc640 1 -- 172.21.15.29:0/2829515472 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d100039d0 con 0x7f7d1c148430 2024-04-25T00:32:55.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d23b12640 1 -- 172.21.15.29:0/2829515472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d1c0af050 con 0x7f7d1c148430 2024-04-25T00:32:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d23b12640 1 -- 172.21.15.29:0/2829515472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d1c148c90 con 0x7f7d1c148430 2024-04-25T00:32:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.454+0000 7f7d1a7fc640 1 -- 172.21.15.29:0/2829515472 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f7d10003b70 con 0x7f7d1c148430 2024-04-25T00:32:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.458+0000 7f7d1a7fc640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7d08067aa0 0x7f7d08069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.458+0000 7f7d1a7fc640 1 -- 172.21.15.29:0/2829515472 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7d10085830 con 0x7f7d1c148430 2024-04-25T00:32:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.458+0000 7f7d218ae640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7d08067aa0 0x7f7d08069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.458+0000 7f7d218ae640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7d08067aa0 0x7f7d08069f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c007fd0 tx=0x7f7d0c007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:55.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.462+0000 7f7d1a7fc640 1 -- 172.21.15.29:0/2829515472 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7d1c148c90 con 0x7f7d1c148430 2024-04-25T00:32:55.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.622+0000 7f7d23b12640 1 -- 172.21.15.29:0/2829515472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d1c069270 con 0x7f7d1c148430 2024-04-25T00:32:55.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.626+0000 7f7d1a7fc640 1 -- 172.21.15.29:0/2829515472 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16621 (secure 0 0 0) 0x7f7d10055900 con 0x7f7d1c148430 2024-04-25T00:32:55.632 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:55.632 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":33,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:32:35.426396+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:32:32.254456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:32:32.779372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:55.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.630+0000 7f7d23b12640 1 -- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7d08067aa0 msgr2=0x7f7d08069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.630+0000 7f7d23b12640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7d08067aa0 0x7f7d08069f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c007fd0 tx=0x7f7d0c007dc0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.630+0000 7f7d23b12640 1 -- 172.21.15.29:0/2829515472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c148430 msgr2=0x7f7d1c1424b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.630+0000 7f7d23b12640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c148430 0x7f7d1c1424b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000d470 tx=0x7f7d1000d930 comp rx=0 tx=0).stop 2024-04-25T00:32:55.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.630+0000 7f7d23b12640 1 -- 172.21.15.29:0/2829515472 shutdown_connections 2024-04-25T00:32:55.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.630+0000 7f7d23b12640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d1c15bc40 0x7f7d1c15c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.630+0000 7f7d23b12640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c148430 0x7f7d1c1424b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.630+0000 7f7d23b12640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7d08067aa0 0x7f7d08069f60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.630+0000 7f7d23b12640 1 --2- 172.21.15.29:0/2829515472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d040062e0 0x7f7d1c0ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.630+0000 7f7d23b12640 1 -- 172.21.15.29:0/2829515472 >> 172.21.15.29:0/2829515472 conn(0x7f7d1c0b1870 msgr2=0x7f7d1c0afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:55.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.630+0000 7f7d23b12640 1 -- 172.21.15.29:0/2829515472 shutdown_connections 2024-04-25T00:32:55.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.630+0000 7f7d23b12640 1 -- 172.21.15.29:0/2829515472 wait complete. 2024-04-25T00:32:55.651 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:32:32.254456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:32:32.779372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:55.652 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:55.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.774+0000 7fd45a12c640 1 Processor -- start 2024-04-25T00:32:55.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.774+0000 7fd45a12c640 1 -- start start 2024-04-25T00:32:55.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd45a12c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd45406dce0 0x7fd454150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd45a12c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd4540b2b70 0x7fd454151330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd45a12c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4540b3130 con 0x7fd45415d980 2024-04-25T00:32:55.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd45a12c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd4540cf560 con 0x7fd4540b2b70 2024-04-25T00:32:55.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd45a12c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd4540b3410 con 0x7fd45406dce0 2024-04-25T00:32:55.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd452ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd45406dce0 0x7fd454150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:55.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd453fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd4540b2b70 0x7fd454151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:55.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd4537fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd45415d980 0x7fd45406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:41724/0 (socket says 172.21.15.29:41724) 2024-04-25T00:32:55.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd4537fe640 1 -- 172.21.15.29:0/508916616 learned_addr learned my addr 172.21.15.29:0/508916616 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:55.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd4527fc640 1 -- 172.21.15.29:0/508916616 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2917963829 0 0) 0x7fd4540b3130 con 0x7fd45415d980 2024-04-25T00:32:55.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd4527fc640 1 -- 172.21.15.29:0/508916616 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd43c003530 con 0x7fd45415d980 2024-04-25T00:32:55.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd453fff640 1 -- 172.21.15.29:0/508916616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd45406dce0 msgr2=0x7fd454150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:55.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd453fff640 1 --2- 172.21.15.29:0/508916616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd45406dce0 0x7fd454150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd453fff640 1 -- 172.21.15.29:0/508916616 >> v1:172.21.15.157:6789/0 conn(0x7fd45415d980 legacy=0x7fd45406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd453fff640 1 -- 172.21.15.29:0/508916616 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4540b36f0 con 0x7fd4540b2b70 2024-04-25T00:32:55.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.778+0000 7fd452ffd640 1 --2- 172.21.15.29:0/508916616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd45406dce0 0x7fd454150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:55.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd453fff640 1 --2- 172.21.15.29:0/508916616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4540b2b70 0x7fd454151330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009d90 tx=0x7fd44c01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7654c3d294b713f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:55.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd4527fc640 1 -- 172.21.15.29:0/508916616 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0b2070 con 0x7fd4540b2b70 2024-04-25T00:32:55.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd4527fc640 1 -- 172.21.15.29:0/508916616 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd44c0a58f0 con 0x7fd4540b2b70 2024-04-25T00:32:55.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd4527fc640 1 -- 172.21.15.29:0/508916616 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0a5ca0 con 0x7fd4540b2b70 2024-04-25T00:32:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 -- 172.21.15.29:0/508916616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4540b2b70 msgr2=0x7fd454151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 --2- 172.21.15.29:0/508916616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4540b2b70 0x7fd454151330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009d90 tx=0x7fd44c01c8b0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 -- 172.21.15.29:0/508916616 shutdown_connections 2024-04-25T00:32:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 --2- 172.21.15.29:0/508916616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4540b2b70 0x7fd454151330 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 --2- 172.21.15.29:0/508916616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd45406dce0 0x7fd454150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 -- 172.21.15.29:0/508916616 >> 172.21.15.29:0/508916616 conn(0x7fd4540b1870 msgr2=0x7fd4540b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 -- 172.21.15.29:0/508916616 shutdown_connections 2024-04-25T00:32:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 -- 172.21.15.29:0/508916616 wait complete. 2024-04-25T00:32:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 Processor -- start 2024-04-25T00:32:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 -- start start 2024-04-25T00:32:55.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45406dce0 0x7fd454157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4540b2b70 0x7fd4541527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd45415d980 0x7fd4540cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd4541663c0 con 0x7fd4540b2b70 2024-04-25T00:32:55.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd454151870 con 0x7fd45406dce0 2024-04-25T00:32:55.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.782+0000 7fd45a12c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd454166540 con 0x7fd45415d980 2024-04-25T00:32:55.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd452ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4540b2b70 0x7fd4541527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:55.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd452ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4540b2b70 0x7fd4541527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33784/0 (socket says 172.21.15.29:33784) 2024-04-25T00:32:55.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd452ffd640 1 -- 172.21.15.29:0/822199190 learned_addr learned my addr 172.21.15.29:0/822199190 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:55.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd4537fe640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45406dce0 0x7fd454157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:55.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd453fff640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd45415d980 0x7fd4540cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd452ffd640 1 -- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd45415d980 msgr2=0x7fd4540cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd452ffd640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd45415d980 0x7fd4540cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd452ffd640 1 -- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45406dce0 msgr2=0x7fd454157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd452ffd640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45406dce0 0x7fd454157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd452ffd640 1 -- 172.21.15.29:0/822199190 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd454151a90 con 0x7fd4540b2b70 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd4537fe640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45406dce0 0x7fd454157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd452ffd640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4540b2b70 0x7fd4541527e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd44800ec10 tx=0x7fd44800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd43bfff640 1 -- 172.21.15.29:0/822199190 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448017070 con 0x7fd4540b2b70 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd43bfff640 1 -- 172.21.15.29:0/822199190 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd44800cc30 con 0x7fd4540b2b70 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd45a12c640 1 -- 172.21.15.29:0/822199190 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd454151d00 con 0x7fd4540b2b70 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd43bfff640 1 -- 172.21.15.29:0/822199190 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448013470 con 0x7fd4540b2b70 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd45a12c640 1 -- 172.21.15.29:0/822199190 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4541522c0 con 0x7fd4540b2b70 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd45a12c640 1 -- 172.21.15.29:0/822199190 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd454069270 con 0x7fd4540b2b70 2024-04-25T00:32:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd43bfff640 1 -- 172.21.15.29:0/822199190 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fd44801a070 con 0x7fd4540b2b70 2024-04-25T00:32:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd43bfff640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd440067a70 0x7fd440069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd43bfff640 1 -- 172.21.15.29:0/822199190 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd448086350 con 0x7fd4540b2b70 2024-04-25T00:32:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.786+0000 7fd4537fe640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd440067a70 0x7fd440069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.790+0000 7fd4537fe640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd440067a70 0x7fd440069f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd444005890 tx=0x7fd444009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:55.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.790+0000 7fd43bfff640 1 -- 172.21.15.29:0/822199190 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd448056c10 con 0x7fd4540b2b70 2024-04-25T00:32:55.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd45a12c640 1 -- 172.21.15.29:0/822199190 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd4540cf910 con 0x7fd4540b2b70 2024-04-25T00:32:55.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd43bfff640 1 -- 172.21.15.29:0/822199190 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7fd448056510 con 0x7fd4540b2b70 2024-04-25T00:32:55.958 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:55.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd45a12c640 1 -- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd440067a70 msgr2=0x7fd440069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd45a12c640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd440067a70 0x7fd440069f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd444005890 tx=0x7fd444009040 comp rx=0 tx=0).stop 2024-04-25T00:32:55.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd45a12c640 1 -- 172.21.15.29:0/822199190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4540b2b70 msgr2=0x7fd4541527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:55.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd45a12c640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4540b2b70 0x7fd4541527e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd44800ec10 tx=0x7fd44800c520 comp rx=0 tx=0).stop 2024-04-25T00:32:55.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd45a12c640 1 -- 172.21.15.29:0/822199190 shutdown_connections 2024-04-25T00:32:55.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd45a12c640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd45415d980 0x7fd4540cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd45a12c640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4540b2b70 0x7fd4541527e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd45a12c640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd440067a70 0x7fd440069f30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd45a12c640 1 --2- 172.21.15.29:0/822199190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45406dce0 0x7fd454157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:55.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd45a12c640 1 -- 172.21.15.29:0/822199190 >> 172.21.15.29:0/822199190 conn(0x7fd4540b1870 msgr2=0x7fd4540afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:55.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd45a12c640 1 -- 172.21.15.29:0/822199190 shutdown_connections 2024-04-25T00:32:55.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:55.954+0000 7fd45a12c640 1 -- 172.21.15.29:0/822199190 wait complete. 2024-04-25T00:32:55.978 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:56.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52edcd6640 1 Processor -- start 2024-04-25T00:32:56.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52edcd6640 1 -- start start 2024-04-25T00:32:56.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52edcd6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f52e815e200 0x7f52e80d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52edcd6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e80cfc10 con 0x7f52e80cf690 2024-04-25T00:32:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52edcd6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e80cfef0 con 0x7f52e80d2c60 2024-04-25T00:32:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52edcd6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f52e80b38d0 con 0x7f52e815e200 2024-04-25T00:32:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52dffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f52e80cf690 0x7f52e80d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50108/0 (socket says 172.21.15.29:50108) 2024-04-25T00:32:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52dffff640 1 -- 172.21.15.29:0/2714013770 learned_addr learned my addr 172.21.15.29:0/2714013770 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e77fe640 1 --2- 172.21.15.29:0/2714013770 >> v2:172.21.15.157:3300/0 conn(0x7f52e815e200 0x7f52e80d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3944614370 0 0) 0x7f52e80cfc10 con 0x7f52e80cf690 2024-04-25T00:32:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52d4003530 con 0x7f52e80cf690 2024-04-25T00:32:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3625368820 0 0) 0x7f52d4003530 con 0x7f52e80cf690 2024-04-25T00:32:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 >> v2:172.21.15.157:3300/0 conn(0x7f52e815e200 msgr2=0x7f52e80d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 --2- 172.21.15.29:0/2714013770 >> v2:172.21.15.157:3300/0 conn(0x7f52e815e200 0x7f52e80d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 >> v1:172.21.15.157:6789/0 conn(0x7f52e80d2c60 legacy=0x7f52e80b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e80b3b20 con 0x7f52e80cf690 2024-04-25T00:32:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f52d0001680 con 0x7f52e80cf690 2024-04-25T00:32:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 >> v1:172.21.15.29:6789/0 conn(0x7f52e80cf690 legacy=0x7f52e80d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 --2- 172.21.15.29:0/2714013770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d40039c0 0x7f52d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 --2- 172.21.15.29:0/2714013770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d4004810 0x7f52d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 --2- 172.21.15.29:0/2714013770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d40051d0 0x7f52d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80b38d0 con 0x7f52d40039c0 2024-04-25T00:32:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52d400c070 con 0x7f52d40051d0 2024-04-25T00:32:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f52d400c270 con 0x7f52d4004810 2024-04-25T00:32:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e77fe640 1 --2- 172.21.15.29:0/2714013770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d40039c0 0x7f52d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e7fff640 1 --2- 172.21.15.29:0/2714013770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d40051d0 0x7f52d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52dffff640 1 --2- 172.21.15.29:0/2714013770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d4004810 0x7f52d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e77fe640 1 -- 172.21.15.29:0/2714013770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d4004810 msgr2=0x7f52d4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e77fe640 1 --2- 172.21.15.29:0/2714013770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d4004810 0x7f52d4004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e77fe640 1 -- 172.21.15.29:0/2714013770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d40051d0 msgr2=0x7f52d400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e77fe640 1 --2- 172.21.15.29:0/2714013770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d40051d0 0x7f52d400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e77fe640 1 -- 172.21.15.29:0/2714013770 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d4003530 con 0x7f52d40039c0 2024-04-25T00:32:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.098+0000 7f52e7fff640 1 -- 172.21.15.29:0/2714013770 reap_dead start 2024-04-25T00:32:56.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.102+0000 7f52e77fe640 1 --2- 172.21.15.29:0/2714013770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d40039c0 0x7f52d4003de0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009ae0 tx=0x7f52d801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ad4cdd3a700a1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:56.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.102+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d80b2070 con 0x7f52d40039c0 2024-04-25T00:32:56.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.102+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d80a5540 con 0x7f52d40039c0 2024-04-25T00:32:56.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.102+0000 7f52e6ffd640 1 -- 172.21.15.29:0/2714013770 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d80096b0 con 0x7f52d40039c0 2024-04-25T00:32:56.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.102+0000 7f52edcd6640 1 -- 172.21.15.29:0/2714013770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d40039c0 msgr2=0x7f52d4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.102+0000 7f52edcd6640 1 --2- 172.21.15.29:0/2714013770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d40039c0 0x7f52d4003de0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009ae0 tx=0x7f52d801ca40 comp rx=0 tx=0).stop 2024-04-25T00:32:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.102+0000 7f52edcd6640 1 -- 172.21.15.29:0/2714013770 shutdown_connections 2024-04-25T00:32:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.102+0000 7f52edcd6640 1 --2- 172.21.15.29:0/2714013770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d40039c0 0x7f52d4003de0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.102+0000 7f52edcd6640 1 -- 172.21.15.29:0/2714013770 >> 172.21.15.29:0/2714013770 conn(0x7f52e809d280 msgr2=0x7f52e80d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.102+0000 7f52edcd6640 1 -- 172.21.15.29:0/2714013770 shutdown_connections 2024-04-25T00:32:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.102+0000 7f52edcd6640 1 -- 172.21.15.29:0/2714013770 wait complete. 2024-04-25T00:32:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.102+0000 7f52edcd6640 1 Processor -- start 2024-04-25T00:32:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.102+0000 7f52edcd6640 1 -- start start 2024-04-25T00:32:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52edcd6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d40039c0 0x7f52e814fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52edcd6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52e80b8680 0x7f52e8160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52edcd6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52e806c550 0x7f52e806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52edcd6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80d0720 con 0x7f52e80b8680 2024-04-25T00:32:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52edcd6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80d05a0 con 0x7f52d40039c0 2024-04-25T00:32:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52edcd6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f52e80d08a0 con 0x7f52e806c550 2024-04-25T00:32:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52dffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52e80b8680 0x7f52e8160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52dffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52e80b8680 0x7f52e8160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33794/0 (socket says 172.21.15.29:33794) 2024-04-25T00:32:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52dffff640 1 -- 172.21.15.29:0/1022099046 learned_addr learned my addr 172.21.15.29:0/1022099046 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52e77fe640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d40039c0 0x7f52e814fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52e7fff640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52e806c550 0x7f52e806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52dffff640 1 -- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52e806c550 msgr2=0x7f52e806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52dffff640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52e806c550 0x7f52e806c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52dffff640 1 -- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d40039c0 msgr2=0x7f52e814fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52dffff640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d40039c0 0x7f52e814fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52dffff640 1 -- 172.21.15.29:0/1022099046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e81614d0 con 0x7f52e80b8680 2024-04-25T00:32:56.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52dffff640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52e80b8680 0x7f52e8160d60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f52d000d810 tx=0x7f52d000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:56.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52e4ff9640 1 -- 172.21.15.29:0/1022099046 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0019070 con 0x7f52e80b8680 2024-04-25T00:32:56.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52e4ff9640 1 -- 172.21.15.29:0/1022099046 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d00074e0 con 0x7f52e80b8680 2024-04-25T00:32:56.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52edcd6640 1 -- 172.21.15.29:0/1022099046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e814d2c0 con 0x7f52e80b8680 2024-04-25T00:32:56.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52e4ff9640 1 -- 172.21.15.29:0/1022099046 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d00049f0 con 0x7f52e80b8680 2024-04-25T00:32:56.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52edcd6640 1 -- 172.21.15.29:0/1022099046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e815e8e0 con 0x7f52e80b8680 2024-04-25T00:32:56.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52edcd6640 1 -- 172.21.15.29:0/1022099046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e8069270 con 0x7f52e80b8680 2024-04-25T00:32:56.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52e4ff9640 1 -- 172.21.15.29:0/1022099046 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f52d001c070 con 0x7f52e80b8680 2024-04-25T00:32:56.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52e4ff9640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f52c8067a70 0x7f52c8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52e4ff9640 1 -- 172.21.15.29:0/1022099046 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f52d0085bd0 con 0x7f52e80b8680 2024-04-25T00:32:56.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.106+0000 7f52e77fe640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f52c8067a70 0x7f52c8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.110+0000 7f52e77fe640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f52c8067a70 0x7f52c8069f30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f52d80af000 tx=0x7f52d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:56.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.110+0000 7f52e4ff9640 1 -- 172.21.15.29:0/1022099046 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f52d0056380 con 0x7f52e80b8680 2024-04-25T00:32:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.266+0000 7f52edcd6640 1 -- 172.21.15.29:0/1022099046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f52e80d22b0 con 0x7f52e80b8680 2024-04-25T00:32:56.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.270+0000 7f52e4ff9640 1 -- 172.21.15.29:0/1022099046 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f52d0055c80 con 0x7f52e80b8680 2024-04-25T00:32:56.273 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.270+0000 7f52edcd6640 1 -- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f52c8067a70 msgr2=0x7f52c8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.270+0000 7f52edcd6640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f52c8067a70 0x7f52c8069f30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f52d80af000 tx=0x7f52d8008040 comp rx=0 tx=0).stop 2024-04-25T00:32:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.270+0000 7f52edcd6640 1 -- 172.21.15.29:0/1022099046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52e80b8680 msgr2=0x7f52e8160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.270+0000 7f52edcd6640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52e80b8680 0x7f52e8160d60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f52d000d810 tx=0x7f52d000dcd0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.270+0000 7f52edcd6640 1 -- 172.21.15.29:0/1022099046 shutdown_connections 2024-04-25T00:32:56.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.270+0000 7f52edcd6640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52e806c550 0x7f52e806c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.270+0000 7f52edcd6640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52e80b8680 0x7f52e8160d60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.270+0000 7f52edcd6640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f52c8067a70 0x7f52c8069f30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.270+0000 7f52edcd6640 1 --2- 172.21.15.29:0/1022099046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d40039c0 0x7f52e814fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.270+0000 7f52edcd6640 1 -- 172.21.15.29:0/1022099046 >> 172.21.15.29:0/1022099046 conn(0x7f52e809d280 msgr2=0x7f52e815d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:56.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.270+0000 7f52edcd6640 1 -- 172.21.15.29:0/1022099046 shutdown_connections 2024-04-25T00:32:56.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.270+0000 7f52edcd6640 1 -- 172.21.15.29:0/1022099046 wait complete. 2024-04-25T00:32:56.291 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:56.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc11222640 1 Processor -- start 2024-04-25T00:32:56.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc11222640 1 -- start start 2024-04-25T00:32:56.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc11222640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efc0c0b3c30 0x7efc0c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc11222640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc0c0b4b10 0x7efc0c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc11222640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc0c0b41e0 con 0x7efc0c0b3140 2024-04-25T00:32:56.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc11222640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7efc0c06d960 con 0x7efc0c0b3c30 2024-04-25T00:32:56.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc11222640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc0c06dc40 con 0x7efc0c0b4b10 2024-04-25T00:32:56.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc0ad9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7efc0c0b3140 0x7efc0c0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50116/0 (socket says 172.21.15.29:50116) 2024-04-25T00:32:56.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc0ad9d640 1 -- 172.21.15.29:0/1783847395 learned_addr learned my addr 172.21.15.29:0/1783847395 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:56.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc0259c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efc0c0b3c30 0x7efc0c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc0b59e640 1 --2- 172.21.15.29:0/1783847395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc0c0b4b10 0x7efc0c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc0259c640 1 -- 172.21.15.29:0/1783847395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc0c0b4b10 msgr2=0x7efc0c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:56.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc0259c640 1 --2- 172.21.15.29:0/1783847395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc0c0b4b10 0x7efc0c0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc0259c640 1 -- 172.21.15.29:0/1783847395 >> v1:172.21.15.29:6789/0 conn(0x7efc0c0b3140 legacy=0x7efc0c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc0259c640 1 -- 172.21.15.29:0/1783847395 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c06df20 con 0x7efc0c0b3c30 2024-04-25T00:32:56.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc0259c640 1 --2- 172.21.15.29:0/1783847395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0c0b3c30 0x7efc0c06e440 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efbf40097f0 tx=0x7efbf401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde59d3363d3dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc0a59c640 1 -- 172.21.15.29:0/1783847395 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf40b0070 con 0x7efc0c0b3c30 2024-04-25T00:32:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc0a59c640 1 -- 172.21.15.29:0/1783847395 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf40ab600 con 0x7efc0c0b3c30 2024-04-25T00:32:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.402+0000 7efc0a59c640 1 -- 172.21.15.29:0/1783847395 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf40ab9d0 con 0x7efc0c0b3c30 2024-04-25T00:32:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 -- 172.21.15.29:0/1783847395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0c0b3c30 msgr2=0x7efc0c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 --2- 172.21.15.29:0/1783847395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0c0b3c30 0x7efc0c06e440 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efbf40097f0 tx=0x7efbf401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 -- 172.21.15.29:0/1783847395 shutdown_connections 2024-04-25T00:32:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 --2- 172.21.15.29:0/1783847395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc0c0b4b10 0x7efc0c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 --2- 172.21.15.29:0/1783847395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0c0b3c30 0x7efc0c06e440 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 -- 172.21.15.29:0/1783847395 >> 172.21.15.29:0/1783847395 conn(0x7efc0c0b1870 msgr2=0x7efc0c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 -- 172.21.15.29:0/1783847395 shutdown_connections 2024-04-25T00:32:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 -- 172.21.15.29:0/1783847395 wait complete. 2024-04-25T00:32:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 Processor -- start 2024-04-25T00:32:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 -- start start 2024-04-25T00:32:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc0c0b3140 0x7efc0c15e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc0c0b3c30 0x7efc0c141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0c0b4b10 0x7efc0c1423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c168d80 con 0x7efc0c0b4b10 2024-04-25T00:32:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c168c00 con 0x7efc0c0b3c30 2024-04-25T00:32:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc0c168f00 con 0x7efc0c0b3140 2024-04-25T00:32:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc0b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0c0b4b10 0x7efc0c1423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc0b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0c0b4b10 0x7efc0c1423a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33816/0 (socket says 172.21.15.29:33816) 2024-04-25T00:32:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc0b59e640 1 -- 172.21.15.29:0/192613141 learned_addr learned my addr 172.21.15.29:0/192613141 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:56.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc0ad9d640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc0c0b3140 0x7efc0c15e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc0259c640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc0c0b3c30 0x7efc0c141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc0b59e640 1 -- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc0c0b3140 msgr2=0x7efc0c15e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc0b59e640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc0c0b3140 0x7efc0c15e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc0b59e640 1 -- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc0c0b3c30 msgr2=0x7efc0c141e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc0b59e640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc0c0b3c30 0x7efc0c141e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc0b59e640 1 -- 172.21.15.29:0/192613141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c1428e0 con 0x7efc0c0b4b10 2024-04-25T00:32:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc0b59e640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0c0b4b10 0x7efc0c1423a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc04007e40 tx=0x7efc0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc03fff640 1 -- 172.21.15.29:0/192613141 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc04014620 con 0x7efc0c0b4b10 2024-04-25T00:32:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc03fff640 1 -- 172.21.15.29:0/192613141 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc04019070 con 0x7efc0c0b4b10 2024-04-25T00:32:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc03fff640 1 -- 172.21.15.29:0/192613141 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc04014a20 con 0x7efc0c0b4b10 2024-04-25T00:32:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 -- 172.21.15.29:0/192613141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0c15d660 con 0x7efc0c0b4b10 2024-04-25T00:32:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.406+0000 7efc11222640 1 -- 172.21.15.29:0/192613141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0c15dc20 con 0x7efc0c0b4b10 2024-04-25T00:32:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.410+0000 7efc11222640 1 -- 172.21.15.29:0/192613141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc0c069270 con 0x7efc0c0b4b10 2024-04-25T00:32:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.410+0000 7efc03fff640 1 -- 172.21.15.29:0/192613141 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7efc04014bc0 con 0x7efc0c0b4b10 2024-04-25T00:32:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.410+0000 7efc03fff640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efbec067aa0 0x7efbec069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.410+0000 7efc03fff640 1 -- 172.21.15.29:0/192613141 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efc040867d0 con 0x7efc0c0b4b10 2024-04-25T00:32:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.410+0000 7efc0ad9d640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efbec067aa0 0x7efbec069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.410+0000 7efc0ad9d640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efbec067aa0 0x7efbec069f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efbfc000f70 tx=0x7efbfc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.414+0000 7efc03fff640 1 -- 172.21.15.29:0/192613141 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efc04057120 con 0x7efc0c0b4b10 2024-04-25T00:32:56.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.578+0000 7efc11222640 1 -- 172.21.15.29:0/192613141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efc0c0b37c0 con 0x7efc0c0b4b10 2024-04-25T00:32:56.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.578+0000 7efc03fff640 1 -- 172.21.15.29:0/192613141 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7efc04056a20 con 0x7efc0c0b4b10 2024-04-25T00:32:56.583 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:56.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.582+0000 7efc11222640 1 -- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efbec067aa0 msgr2=0x7efbec069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.582+0000 7efc11222640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efbec067aa0 0x7efbec069f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efbfc000f70 tx=0x7efbfc009040 comp rx=0 tx=0).stop 2024-04-25T00:32:56.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.582+0000 7efc11222640 1 -- 172.21.15.29:0/192613141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0c0b4b10 msgr2=0x7efc0c1423a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.582+0000 7efc11222640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0c0b4b10 0x7efc0c1423a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc04007e40 tx=0x7efc0400e4b0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.582+0000 7efc11222640 1 -- 172.21.15.29:0/192613141 shutdown_connections 2024-04-25T00:32:56.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.582+0000 7efc11222640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0c0b4b10 0x7efc0c1423a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.582+0000 7efc11222640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efbec067aa0 0x7efbec069f60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.582+0000 7efc11222640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc0c0b3c30 0x7efc0c141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.582+0000 7efc11222640 1 --2- 172.21.15.29:0/192613141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc0c0b3140 0x7efc0c15e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.582+0000 7efc11222640 1 -- 172.21.15.29:0/192613141 >> 172.21.15.29:0/192613141 conn(0x7efc0c0b1870 msgr2=0x7efc0c140ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:56.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.582+0000 7efc11222640 1 -- 172.21.15.29:0/192613141 shutdown_connections 2024-04-25T00:32:56.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.582+0000 7efc11222640 1 -- 172.21.15.29:0/192613141 wait complete. 2024-04-25T00:32:56.603 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:32:56.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e444d640 1 Processor -- start 2024-04-25T00:32:56.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e444d640 1 -- start start 2024-04-25T00:32:56.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e444d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f53dc06dce0 0x7f53dc150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e444d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53dc0b2b70 0x7f53dc151330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e444d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53dc0b3130 con 0x7f53dc15d980 2024-04-25T00:32:56.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e444d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f53dc0cf560 con 0x7f53dc06dce0 2024-04-25T00:32:56.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e444d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f53dc0b3410 con 0x7f53dc0b2b70 2024-04-25T00:32:56.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e19e8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f53dc06dce0 0x7f53dc150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e19e8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f53dc06dce0 0x7f53dc150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33830/0 (socket says 172.21.15.29:33830) 2024-04-25T00:32:56.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e19e8640 1 -- 172.21.15.29:0/1854891647 learned_addr learned my addr 172.21.15.29:0/1854891647 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:56.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e29ea640 1 --2- 172.21.15.29:0/1854891647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53dc0b2b70 0x7f53dc151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e19e8640 1 -- 172.21.15.29:0/1854891647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53dc0b2b70 msgr2=0x7f53dc151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:56.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e19e8640 1 --2- 172.21.15.29:0/1854891647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53dc0b2b70 0x7f53dc151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e19e8640 1 -- 172.21.15.29:0/1854891647 >> v1:172.21.15.157:6789/0 conn(0x7f53dc15d980 legacy=0x7f53dc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.714+0000 7f53e19e8640 1 -- 172.21.15.29:0/1854891647 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc0b36f0 con 0x7f53dc06dce0 2024-04-25T00:32:56.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.718+0000 7f53e19e8640 1 --2- 172.21.15.29:0/1854891647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53dc06dce0 0x7f53dc150df0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53cc006fd0 tx=0x7f53cc01e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d447c2dc1593809 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:56.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.718+0000 7f53e11e7640 1 -- 172.21.15.29:0/1854891647 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0ad070 con 0x7f53dc06dce0 2024-04-25T00:32:56.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.718+0000 7f53e11e7640 1 -- 172.21.15.29:0/1854891647 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53cc004230 con 0x7f53dc06dce0 2024-04-25T00:32:56.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.718+0000 7f53e11e7640 1 -- 172.21.15.29:0/1854891647 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0a7720 con 0x7f53dc06dce0 2024-04-25T00:32:56.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.718+0000 7f53e444d640 1 -- 172.21.15.29:0/1854891647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53dc06dce0 msgr2=0x7f53dc150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.718+0000 7f53e444d640 1 --2- 172.21.15.29:0/1854891647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53dc06dce0 0x7f53dc150df0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53cc006fd0 tx=0x7f53cc01e9c0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.718+0000 7f53e444d640 1 -- 172.21.15.29:0/1854891647 shutdown_connections 2024-04-25T00:32:56.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.718+0000 7f53e444d640 1 --2- 172.21.15.29:0/1854891647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53dc0b2b70 0x7f53dc151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.718+0000 7f53e444d640 1 --2- 172.21.15.29:0/1854891647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53dc06dce0 0x7f53dc150df0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.718+0000 7f53e444d640 1 -- 172.21.15.29:0/1854891647 >> 172.21.15.29:0/1854891647 conn(0x7f53dc0b1870 msgr2=0x7f53dc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:56.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.718+0000 7f53e444d640 1 -- 172.21.15.29:0/1854891647 shutdown_connections 2024-04-25T00:32:56.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.718+0000 7f53e444d640 1 -- 172.21.15.29:0/1854891647 wait complete. 2024-04-25T00:32:56.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e444d640 1 Processor -- start 2024-04-25T00:32:56.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e444d640 1 -- start start 2024-04-25T00:32:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e444d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53dc06dce0 0x7f53dc1415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e444d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53dc0b2b70 0x7f53dc1530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e444d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53dc15d980 0x7f53dc14e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e444d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc15b4b0 con 0x7f53dc06dce0 2024-04-25T00:32:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e444d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc151870 con 0x7f53dc0b2b70 2024-04-25T00:32:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e444d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f53dc15b630 con 0x7f53dc15d980 2024-04-25T00:32:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e21e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53dc06dce0 0x7f53dc1415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e21e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53dc06dce0 0x7f53dc1415d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33840/0 (socket says 172.21.15.29:33840) 2024-04-25T00:32:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e21e9640 1 -- 172.21.15.29:0/589473871 learned_addr learned my addr 172.21.15.29:0/589473871 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e29ea640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53dc15d980 0x7f53dc14e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e19e8640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53dc0b2b70 0x7f53dc1530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e21e9640 1 -- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53dc15d980 msgr2=0x7f53dc14e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e21e9640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53dc15d980 0x7f53dc14e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e21e9640 1 -- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53dc0b2b70 msgr2=0x7f53dc1530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e21e9640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53dc0b2b70 0x7f53dc1530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e21e9640 1 -- 172.21.15.29:0/589473871 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc14f210 con 0x7f53dc06dce0 2024-04-25T00:32:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e19e8640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53dc0b2b70 0x7f53dc1530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e21e9640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53dc06dce0 0x7f53dc1415d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f53d800c730 tx=0x7f53d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53d2ffd640 1 -- 172.21.15.29:0/589473871 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8016020 con 0x7f53dc06dce0 2024-04-25T00:32:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53d2ffd640 1 -- 172.21.15.29:0/589473871 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d8006d60 con 0x7f53dc06dce0 2024-04-25T00:32:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53d2ffd640 1 -- 172.21.15.29:0/589473871 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8003bc0 con 0x7f53dc06dce0 2024-04-25T00:32:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e444d640 1 -- 172.21.15.29:0/589473871 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc14f4f0 con 0x7f53dc06dce0 2024-04-25T00:32:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e444d640 1 -- 172.21.15.29:0/589473871 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53dc0cca10 con 0x7f53dc06dce0 2024-04-25T00:32:56.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.722+0000 7f53e444d640 1 -- 172.21.15.29:0/589473871 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53dc069270 con 0x7f53dc06dce0 2024-04-25T00:32:56.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.726+0000 7f53d2ffd640 1 -- 172.21.15.29:0/589473871 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f53d800a850 con 0x7f53dc06dce0 2024-04-25T00:32:56.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.726+0000 7f53d2ffd640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f53c0067a70 0x7f53c0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:56.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.726+0000 7f53d2ffd640 1 -- 172.21.15.29:0/589473871 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f53d8085c80 con 0x7f53dc06dce0 2024-04-25T00:32:56.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.726+0000 7f53e19e8640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f53c0067a70 0x7f53c0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:56.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.726+0000 7f53e19e8640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f53c0067a70 0x7f53c0069f30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0b1000 tx=0x7f53cc0a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:56.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.730+0000 7f53d2ffd640 1 -- 172.21.15.29:0/589473871 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f53d8020020 con 0x7f53dc06dce0 2024-04-25T00:32:56.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.898+0000 7f53e444d640 1 -- 172.21.15.29:0/589473871 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f53dc150a90 con 0x7f53dc06dce0 2024-04-25T00:32:56.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.898+0000 7f53d2ffd640 1 -- 172.21.15.29:0/589473871 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f53d8056530 con 0x7f53dc06dce0 2024-04-25T00:32:56.903 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:32:56.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.898+0000 7f53e444d640 1 -- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f53c0067a70 msgr2=0x7f53c0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.898+0000 7f53e444d640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f53c0067a70 0x7f53c0069f30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0b1000 tx=0x7f53cc0a9070 comp rx=0 tx=0).stop 2024-04-25T00:32:56.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.898+0000 7f53e444d640 1 -- 172.21.15.29:0/589473871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53dc06dce0 msgr2=0x7f53dc1415d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:56.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.898+0000 7f53e444d640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53dc06dce0 0x7f53dc1415d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f53d800c730 tx=0x7f53d800cbf0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.898+0000 7f53e444d640 1 -- 172.21.15.29:0/589473871 shutdown_connections 2024-04-25T00:32:56.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.898+0000 7f53e444d640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53dc15d980 0x7f53dc14e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.898+0000 7f53e444d640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f53c0067a70 0x7f53c0069f30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.898+0000 7f53e444d640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53dc0b2b70 0x7f53dc1530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.898+0000 7f53e444d640 1 --2- 172.21.15.29:0/589473871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53dc06dce0 0x7f53dc1415d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:56.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.898+0000 7f53e444d640 1 -- 172.21.15.29:0/589473871 >> 172.21.15.29:0/589473871 conn(0x7f53dc0b1870 msgr2=0x7f53dc06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:56.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.898+0000 7f53e444d640 1 -- 172.21.15.29:0/589473871 shutdown_connections 2024-04-25T00:32:56.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:56.902+0000 7f53e444d640 1 -- 172.21.15.29:0/589473871 wait complete. 2024-04-25T00:32:56.920 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-25T00:32:56.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:56.921 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T00:32:56.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[33664]: fuse finished with error 0 and tester_r 0 2024-04-25T00:32:56.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:32:56.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:56.936 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:32:56.992 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:32:56.992 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:32:56.992 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:56.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:32:56.999 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:32:56.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:56.999 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:32:57.045 INFO:teuthology.orchestra.run.smithi029.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T00:32:57.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:32:57.046 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:32:57.050 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:32:57.050 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:32:57.056 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:32:57.056 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:32:57.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:32:57.056 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:32:57.105 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T00:32:57.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:32:57.106 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:32:57.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f36151640 1 Processor -- start 2024-04-25T00:32:57.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f36151640 1 -- start start 2024-04-25T00:32:57.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f36151640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f300b3c30 0x7f8f3006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f36151640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f300b41e0 con 0x7f8f300b3140 2024-04-25T00:32:57.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f36151640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f3006d960 con 0x7f8f300b4b10 2024-04-25T00:32:57.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f36151640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3006dc40 con 0x7f8f300b3c30 2024-04-25T00:32:57.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8f300b3140 0x7f8f300b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50122/0 (socket says 172.21.15.29:50122) 2024-04-25T00:32:57.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2f7fe640 1 -- 172.21.15.29:0/2340763113 learned_addr learned my addr 172.21.15.29:0/2340763113 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:57.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f27fff640 1 --2- 172.21.15.29:0/2340763113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f300b3c30 0x7f8f3006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1214848843 0 0) 0x7f8f300b41e0 con 0x7f8f300b3140 2024-04-25T00:32:57.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f1c003530 con 0x7f8f300b3140 2024-04-25T00:32:57.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 49220294 0 0) 0x7f8f1c003530 con 0x7f8f300b3140 2024-04-25T00:32:57.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f300b3c30 msgr2=0x7f8f3006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:57.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 --2- 172.21.15.29:0/2340763113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f300b3c30 0x7f8f3006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 >> v1:172.21.15.157:6789/0 conn(0x7f8f300b4b10 legacy=0x7f8f300ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3006df20 con 0x7f8f300b3140 2024-04-25T00:32:57.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8f20001680 con 0x7f8f300b3140 2024-04-25T00:32:57.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 >> v1:172.21.15.29:6789/0 conn(0x7f8f300b3140 legacy=0x7f8f300b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 --2- 172.21.15.29:0/2340763113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f1c0039f0 0x7f8f1c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 --2- 172.21.15.29:0/2340763113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c004810 0x7f8f1c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 --2- 172.21.15.29:0/2340763113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f300b3c30 0x7f8f1c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3006dc40 con 0x7f8f1c0039f0 2024-04-25T00:32:57.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c00bab0 con 0x7f8f1c004810 2024-04-25T00:32:57.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1c00bcb0 con 0x7f8f300b3c30 2024-04-25T00:32:57.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2f7fe640 1 --2- 172.21.15.29:0/2340763113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f1c0039f0 0x7f8f1c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f27fff640 1 --2- 172.21.15.29:0/2340763113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c004810 0x7f8f1c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2ffff640 1 --2- 172.21.15.29:0/2340763113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f300b3c30 0x7f8f1c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2f7fe640 1 -- 172.21.15.29:0/2340763113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f300b3c30 msgr2=0x7f8f1c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:57.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2f7fe640 1 --2- 172.21.15.29:0/2340763113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f300b3c30 0x7f8f1c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2f7fe640 1 -- 172.21.15.29:0/2340763113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c004810 msgr2=0x7f8f1c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:57.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2f7fe640 1 --2- 172.21.15.29:0/2340763113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c004810 0x7f8f1c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.230+0000 7f8f2f7fe640 1 -- 172.21.15.29:0/2340763113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c003530 con 0x7f8f1c0039f0 2024-04-25T00:32:57.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.234+0000 7f8f2f7fe640 1 --2- 172.21.15.29:0/2340763113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f1c0039f0 0x7f8f1c003e10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8f20001ec0 tx=0x7f8f2001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b7d4e1071350e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:57.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.234+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20007d40 con 0x7f8f1c0039f0 2024-04-25T00:32:57.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.234+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f200a9070 con 0x7f8f1c0039f0 2024-04-25T00:32:57.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.234+0000 7f8f2effd640 1 -- 172.21.15.29:0/2340763113 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20005d40 con 0x7f8f1c0039f0 2024-04-25T00:32:57.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.234+0000 7f8f36151640 1 -- 172.21.15.29:0/2340763113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f1c0039f0 msgr2=0x7f8f1c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.234+0000 7f8f36151640 1 --2- 172.21.15.29:0/2340763113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f1c0039f0 0x7f8f1c003e10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8f20001ec0 tx=0x7f8f2001cc40 comp rx=0 tx=0).stop 2024-04-25T00:32:57.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.234+0000 7f8f2ffff640 1 -- 172.21.15.29:0/2340763113 reap_dead start 2024-04-25T00:32:57.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.234+0000 7f8f36151640 1 -- 172.21.15.29:0/2340763113 shutdown_connections 2024-04-25T00:32:57.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.234+0000 7f8f36151640 1 -- 172.21.15.29:0/2340763113 >> 172.21.15.29:0/2340763113 conn(0x7f8f300b1870 msgr2=0x7f8f300b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:57.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.234+0000 7f8f36151640 1 -- 172.21.15.29:0/2340763113 shutdown_connections 2024-04-25T00:32:57.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.234+0000 7f8f36151640 1 -- 172.21.15.29:0/2340763113 wait complete. 2024-04-25T00:32:57.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.234+0000 7f8f36151640 1 Processor -- start 2024-04-25T00:32:57.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.234+0000 7f8f36151640 1 -- start start 2024-04-25T00:32:57.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f36151640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c0039f0 0x7f8f3013c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f36151640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f30141b70 0x7f8f3015d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f36151640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f30141ff0 0x7f8f3015d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f36151640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30167d80 con 0x7f8f30141b70 2024-04-25T00:32:57.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f36151640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30167c00 con 0x7f8f1c0039f0 2024-04-25T00:32:57.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f36151640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f30167f00 con 0x7f8f30141ff0 2024-04-25T00:32:57.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f27fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f30141b70 0x7f8f3015d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f27fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f30141b70 0x7f8f3015d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33858/0 (socket says 172.21.15.29:33858) 2024-04-25T00:32:57.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f27fff640 1 -- 172.21.15.29:0/1483004902 learned_addr learned my addr 172.21.15.29:0/1483004902 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:57.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f2ffff640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f30141ff0 0x7f8f3015d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f2f7fe640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c0039f0 0x7f8f3013c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f27fff640 1 -- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f30141ff0 msgr2=0x7f8f3015d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f27fff640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f30141ff0 0x7f8f3015d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f27fff640 1 -- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c0039f0 msgr2=0x7f8f3013c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f27fff640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c0039f0 0x7f8f3013c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f27fff640 1 -- 172.21.15.29:0/1483004902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3015fad0 con 0x7f8f30141b70 2024-04-25T00:32:57.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f27fff640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f30141b70 0x7f8f3015d470 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8f18011610 tx=0x7f8f18011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:57.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f2cff9640 1 -- 172.21.15.29:0/1483004902 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1800f8f0 con 0x7f8f30141b70 2024-04-25T00:32:57.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f2cff9640 1 -- 172.21.15.29:0/1483004902 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1800fa90 con 0x7f8f30141b70 2024-04-25T00:32:57.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f2cff9640 1 -- 172.21.15.29:0/1483004902 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1800fe40 con 0x7f8f30141b70 2024-04-25T00:32:57.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f36151640 1 -- 172.21.15.29:0/1483004902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3015fd70 con 0x7f8f30141b70 2024-04-25T00:32:57.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f36151640 1 -- 172.21.15.29:0/1483004902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f30160330 con 0x7f8f30141b70 2024-04-25T00:32:57.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f36151640 1 -- 172.21.15.29:0/1483004902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f30069270 con 0x7f8f30141b70 2024-04-25T00:32:57.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f2cff9640 1 -- 172.21.15.29:0/1483004902 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f8f18015460 con 0x7f8f30141b70 2024-04-25T00:32:57.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f2cff9640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8f10067a70 0x7f8f10069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.238+0000 7f8f2cff9640 1 -- 172.21.15.29:0/1483004902 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8f180868a0 con 0x7f8f30141b70 2024-04-25T00:32:57.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.242+0000 7f8f2f7fe640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8f10067a70 0x7f8f10069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.242+0000 7f8f2f7fe640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8f10067a70 0x7f8f10069f30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8f200ab000 tx=0x7f8f200a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:57.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.246+0000 7f8f2cff9640 1 -- 172.21.15.29:0/1483004902 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8f18019030 con 0x7f8f30141b70 2024-04-25T00:32:57.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.442+0000 7f8f36151640 1 -- 172.21.15.29:0/1483004902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f30159340 con 0x7f8f30141b70 2024-04-25T00:32:57.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.446+0000 7f8f2cff9640 1 -- 172.21.15.29:0/1483004902 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5186 (secure 0 0 0) 0x7f8f18057150 con 0x7f8f30141b70 2024-04-25T00:32:57.449 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:57.450 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2100384916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2100384916},{"type":"v1","addr":"172.21.15.157:6835","nonce":2100384916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4984,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1430513194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1430513194},{"type":"v1","addr":"172.21.15.157:6837","nonce":1430513194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3354831881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3354831881},{"type":"v1","addr":"172.21.15.29:6833","nonce":3354831881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:32:33.777640+0000","modified":"2024-04-25T00:32:36.837733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1468158138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1468158138},{"type":"v1","addr":"172.21.15.29:6835","nonce":1468158138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":2}]} 2024-04-25T00:32:57.450 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 16 2024-04-25T00:32:57.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.446+0000 7f8f36151640 1 -- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8f10067a70 msgr2=0x7f8f10069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.446+0000 7f8f36151640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8f10067a70 0x7f8f10069f30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8f200ab000 tx=0x7f8f200a7070 comp rx=0 tx=0).stop 2024-04-25T00:32:57.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.446+0000 7f8f36151640 1 -- 172.21.15.29:0/1483004902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f30141b70 msgr2=0x7f8f3015d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.446+0000 7f8f36151640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f30141b70 0x7f8f3015d470 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8f18011610 tx=0x7f8f18011ad0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.446+0000 7f8f36151640 1 -- 172.21.15.29:0/1483004902 shutdown_connections 2024-04-25T00:32:57.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.446+0000 7f8f36151640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f30141ff0 0x7f8f3015d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.446+0000 7f8f36151640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f30141b70 0x7f8f3015d470 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.446+0000 7f8f36151640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8f10067a70 0x7f8f10069f30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.446+0000 7f8f36151640 1 --2- 172.21.15.29:0/1483004902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c0039f0 0x7f8f3013c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.446+0000 7f8f36151640 1 -- 172.21.15.29:0/1483004902 >> 172.21.15.29:0/1483004902 conn(0x7f8f300b1870 msgr2=0x7f8f301649d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:57.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.446+0000 7f8f36151640 1 -- 172.21.15.29:0/1483004902 shutdown_connections 2024-04-25T00:32:57.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.446+0000 7f8f36151640 1 -- 172.21.15.29:0/1483004902 wait complete. 2024-04-25T00:32:57.468 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.582+0000 7f2a6f743640 1 Processor -- start 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6f743640 1 -- start start 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6f743640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2a680b3140 0x7f2a6806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6f743640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a680b41e0 con 0x7f2a680b4b10 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6f743640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a6806d960 con 0x7f2a680b3c30 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6f743640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2a6806dc40 con 0x7f2a680b3140 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6ccde640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2a680b4b10 0x7f2a680b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:50130/0 (socket says 172.21.15.29:50130) 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6ccde640 1 -- 172.21.15.29:0/1806917400 learned_addr learned my addr 172.21.15.29:0/1806917400 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6d4df640 1 --2- 172.21.15.29:0/1806917400 >> v2:172.21.15.157:3300/0 conn(0x7f2a680b3140 0x7f2a6806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2370022514 0 0) 0x7f2a680b41e0 con 0x7f2a680b4b10 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a54003530 con 0x7f2a680b4b10 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 713642795 0 0) 0x7f2a54003530 con 0x7f2a680b4b10 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 >> v2:172.21.15.157:3300/0 conn(0x7f2a680b3140 msgr2=0x7f2a6806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 --2- 172.21.15.29:0/1806917400 >> v2:172.21.15.157:3300/0 conn(0x7f2a680b3140 0x7f2a6806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 >> v1:172.21.15.157:6789/0 conn(0x7f2a680b3c30 legacy=0x7f2a680ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6806df20 con 0x7f2a680b4b10 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2a64001a00 con 0x7f2a680b4b10 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 >> v1:172.21.15.29:6789/0 conn(0x7f2a680b4b10 legacy=0x7f2a680b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 --2- 172.21.15.29:0/1806917400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a540039c0 0x7f2a54003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 --2- 172.21.15.29:0/1806917400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a54004810 0x7f2a54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 --2- 172.21.15.29:0/1806917400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a540051d0 0x7f2a5400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6806dc40 con 0x7f2a54004810 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5400c070 con 0x7f2a540051d0 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5400c270 con 0x7f2a540039c0 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6ccde640 1 --2- 172.21.15.29:0/1806917400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a54004810 0x7f2a54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6d4df640 1 --2- 172.21.15.29:0/1806917400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a540039c0 0x7f2a54003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6dce0640 1 --2- 172.21.15.29:0/1806917400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a540051d0 0x7f2a5400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6ccde640 1 -- 172.21.15.29:0/1806917400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a540039c0 msgr2=0x7f2a54003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6ccde640 1 --2- 172.21.15.29:0/1806917400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a540039c0 0x7f2a54003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6ccde640 1 -- 172.21.15.29:0/1806917400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a540051d0 msgr2=0x7f2a5400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6ccde640 1 --2- 172.21.15.29:0/1806917400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a540051d0 0x7f2a5400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6ccde640 1 -- 172.21.15.29:0/1806917400 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a54003530 con 0x7f2a54004810 2024-04-25T00:32:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6d4df640 1 --2- 172.21.15.29:0/1806917400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a540039c0 0x7f2a54003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6dce0640 1 --2- 172.21.15.29:0/1806917400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a540051d0 0x7f2a5400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.586+0000 7f2a6dce0640 1 -- 172.21.15.29:0/1806917400 reap_dead start 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6ccde640 1 --2- 172.21.15.29:0/1806917400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a54004810 0x7f2a54004c90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400a830 tx=0x7f2a6401fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9eb4f3887b638 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6400a480 con 0x7f2a54004810 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a640aa710 con 0x7f2a54004810 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a5ffff640 1 -- 172.21.15.29:0/1806917400 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a640aab00 con 0x7f2a54004810 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 -- 172.21.15.29:0/1806917400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a54004810 msgr2=0x7f2a54004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 --2- 172.21.15.29:0/1806917400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a54004810 0x7f2a54004c90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400a830 tx=0x7f2a6401fac0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 -- 172.21.15.29:0/1806917400 shutdown_connections 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 --2- 172.21.15.29:0/1806917400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a54004810 0x7f2a54004c90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 -- 172.21.15.29:0/1806917400 >> 172.21.15.29:0/1806917400 conn(0x7f2a680b1870 msgr2=0x7f2a680b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 -- 172.21.15.29:0/1806917400 shutdown_connections 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 -- 172.21.15.29:0/1806917400 wait complete. 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 Processor -- start 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 -- start start 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a54004810 0x7f2a6814d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a68159840 0x7f2a6813a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a68159cc0 0x7f2a6813eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68145a90 con 0x7f2a68159cc0 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68145910 con 0x7f2a54004810 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6f743640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6806e440 con 0x7f2a68159840 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6dce0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a68159cc0 0x7f2a6813eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6dce0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a68159cc0 0x7f2a6813eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33884/0 (socket says 172.21.15.29:33884) 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6dce0640 1 -- 172.21.15.29:0/1349343922 learned_addr learned my addr 172.21.15.29:0/1349343922 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6ccde640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a68159840 0x7f2a6813a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6dce0640 1 -- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a68159840 msgr2=0x7f2a6813a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6d4df640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a54004810 0x7f2a6814d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6dce0640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a68159840 0x7f2a6813a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6dce0640 1 -- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a54004810 msgr2=0x7f2a6814d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6dce0640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a54004810 0x7f2a6814d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6dce0640 1 -- 172.21.15.29:0/1349343922 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a68156b60 con 0x7f2a68159cc0 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.590+0000 7f2a6ccde640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a68159840 0x7f2a6813a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.594+0000 7f2a6dce0640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a68159cc0 0x7f2a6813eb90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2a68074fe0 tx=0x7f2a6000d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.594+0000 7f2a5dffb640 1 -- 172.21.15.29:0/1349343922 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60018070 con 0x7f2a68159cc0 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.594+0000 7f2a5dffb640 1 -- 172.21.15.29:0/1349343922 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a600052f0 con 0x7f2a68159cc0 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.594+0000 7f2a5dffb640 1 -- 172.21.15.29:0/1349343922 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600054a0 con 0x7f2a68159cc0 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.594+0000 7f2a6f743640 1 -- 172.21.15.29:0/1349343922 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6814db60 con 0x7f2a68159cc0 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.594+0000 7f2a6f743640 1 -- 172.21.15.29:0/1349343922 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a68139750 con 0x7f2a68159cc0 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.594+0000 7f2a6f743640 1 -- 172.21.15.29:0/1349343922 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a68069270 con 0x7f2a68159cc0 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.594+0000 7f2a5dffb640 1 -- 172.21.15.29:0/1349343922 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f2a6001b070 con 0x7f2a68159cc0 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.594+0000 7f2a5dffb640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2a4c067a70 0x7f2a4c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.594+0000 7f2a5dffb640 1 -- 172.21.15.29:0/1349343922 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2a600854a0 con 0x7f2a68159cc0 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.594+0000 7f2a6d4df640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2a4c067a70 0x7f2a4c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.594+0000 7f2a6d4df640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2a4c067a70 0x7f2a4c069f30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009a60 tx=0x7f2a58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.598+0000 7f2a5dffb640 1 -- 172.21.15.29:0/1349343922 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2a60055c50 con 0x7f2a68159cc0 2024-04-25T00:32:57.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.762+0000 7f2a6f743640 1 -- 172.21.15.29:0/1349343922 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a68142910 con 0x7f2a68159cc0 2024-04-25T00:32:57.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.762+0000 7f2a5dffb640 1 -- 172.21.15.29:0/1349343922 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16621 (secure 0 0 0) 0x7f2a60055550 con 0x7f2a68159cc0 2024-04-25T00:32:57.768 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:57.769 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":33,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:32:35.426396+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:32:32.254456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:32:32.779372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:57.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.766+0000 7f2a6f743640 1 -- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2a4c067a70 msgr2=0x7f2a4c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.766+0000 7f2a6f743640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2a4c067a70 0x7f2a4c069f30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009a60 tx=0x7f2a58008040 comp rx=0 tx=0).stop 2024-04-25T00:32:57.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.766+0000 7f2a6f743640 1 -- 172.21.15.29:0/1349343922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a68159cc0 msgr2=0x7f2a6813eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.766+0000 7f2a6f743640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a68159cc0 0x7f2a6813eb90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2a68074fe0 tx=0x7f2a6000d5e0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.766+0000 7f2a6f743640 1 -- 172.21.15.29:0/1349343922 shutdown_connections 2024-04-25T00:32:57.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.766+0000 7f2a6f743640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a68159cc0 0x7f2a6813eb90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.766+0000 7f2a6f743640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a68159840 0x7f2a6813a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.766+0000 7f2a6f743640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2a4c067a70 0x7f2a4c069f30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.766+0000 7f2a6f743640 1 --2- 172.21.15.29:0/1349343922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a54004810 0x7f2a6814d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.766+0000 7f2a6f743640 1 -- 172.21.15.29:0/1349343922 >> 172.21.15.29:0/1349343922 conn(0x7f2a680b1870 msgr2=0x7f2a6813fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:57.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.766+0000 7f2a6f743640 1 -- 172.21.15.29:0/1349343922 shutdown_connections 2024-04-25T00:32:57.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.766+0000 7f2a6f743640 1 -- 172.21.15.29:0/1349343922 wait complete. 2024-04-25T00:32:57.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:32:32.254456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:32:32.779372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:32:57.788 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:57.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f243567c640 1 Processor -- start 2024-04-25T00:32:57.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f243567c640 1 -- start start 2024-04-25T00:32:57.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f243567c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f243015c220 0x7f24300cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f243567c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f24300cfa10 0x7f24300cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f243567c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24300cdbd0 con 0x7f24300d04a0 2024-04-25T00:32:57.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f243567c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f24300d0910 con 0x7f24300cfa10 2024-04-25T00:32:57.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f243567c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f243015c600 con 0x7f243015c220 2024-04-25T00:32:57.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f242e7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f24300cfa10 0x7f24300cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f242f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f24300d04a0 0x7f24300ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46454/0 (socket says 172.21.15.29:46454) 2024-04-25T00:32:57.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f242f7fe640 1 -- 172.21.15.29:0/2332299329 learned_addr learned my addr 172.21.15.29:0/2332299329 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:57.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f242effd640 1 --2- 172.21.15.29:0/2332299329 >> v2:172.21.15.157:3300/0 conn(0x7f243015c220 0x7f24300cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f242e7fc640 1 -- 172.21.15.29:0/2332299329 >> v2:172.21.15.157:3300/0 conn(0x7f243015c220 msgr2=0x7f24300cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:57.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f242e7fc640 1 --2- 172.21.15.29:0/2332299329 >> v2:172.21.15.157:3300/0 conn(0x7f243015c220 0x7f24300cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f242e7fc640 1 -- 172.21.15.29:0/2332299329 >> v1:172.21.15.29:6789/0 conn(0x7f24300d04a0 legacy=0x7f24300ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f242dffb640 1 -- 172.21.15.29:0/2332299329 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2103915544 0 0) 0x7f24300cdbd0 con 0x7f24300d04a0 2024-04-25T00:32:57.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.910+0000 7f242e7fc640 1 -- 172.21.15.29:0/2332299329 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24300b13e0 con 0x7f24300cfa10 2024-04-25T00:32:57.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f242e7fc640 1 --2- 172.21.15.29:0/2332299329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24300cfa10 0x7f24300cff60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f24300cf860 tx=0x7f242401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed13bc9ba146a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:57.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f242dffb640 1 -- 172.21.15.29:0/2332299329 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24240b0070 con 0x7f24300cfa10 2024-04-25T00:32:57.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f242dffb640 1 -- 172.21.15.29:0/2332299329 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24240ab600 con 0x7f24300cfa10 2024-04-25T00:32:57.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f242dffb640 1 -- 172.21.15.29:0/2332299329 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24240ab9d0 con 0x7f24300cfa10 2024-04-25T00:32:57.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 -- 172.21.15.29:0/2332299329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24300cfa10 msgr2=0x7f24300cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 --2- 172.21.15.29:0/2332299329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24300cfa10 0x7f24300cff60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f24300cf860 tx=0x7f242401c480 comp rx=0 tx=0).stop 2024-04-25T00:32:57.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 -- 172.21.15.29:0/2332299329 shutdown_connections 2024-04-25T00:32:57.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 --2- 172.21.15.29:0/2332299329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24300cfa10 0x7f24300cff60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 --2- 172.21.15.29:0/2332299329 >> v2:172.21.15.157:3300/0 conn(0x7f243015c220 0x7f24300cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 -- 172.21.15.29:0/2332299329 >> 172.21.15.29:0/2332299329 conn(0x7f24300d1d60 msgr2=0x7f24300d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:57.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 -- 172.21.15.29:0/2332299329 shutdown_connections 2024-04-25T00:32:57.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 -- 172.21.15.29:0/2332299329 wait complete. 2024-04-25T00:32:57.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 Processor -- start 2024-04-25T00:32:57.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 -- start start 2024-04-25T00:32:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24300cfa10 0x7f2430153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24300d04a0 0x7f243014a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f243015c220 0x7f2430159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2430169090 con 0x7f243015c220 2024-04-25T00:32:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2430168f10 con 0x7f24300cfa10 2024-04-25T00:32:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f243567c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2430169210 con 0x7f24300d04a0 2024-04-25T00:32:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f242f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f243015c220 0x7f2430159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f242f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f243015c220 0x7f2430159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38304/0 (socket says 172.21.15.29:38304) 2024-04-25T00:32:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.914+0000 7f242f7fe640 1 -- 172.21.15.29:0/833763926 learned_addr learned my addr 172.21.15.29:0/833763926 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f242effd640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24300cfa10 0x7f2430153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f242e7fc640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24300d04a0 0x7f243014a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f242f7fe640 1 -- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24300d04a0 msgr2=0x7f243014a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f242f7fe640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24300d04a0 0x7f243014a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f242f7fe640 1 -- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24300cfa10 msgr2=0x7f2430153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f242f7fe640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24300cfa10 0x7f2430153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f242f7fe640 1 -- 172.21.15.29:0/833763926 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2430154540 con 0x7f243015c220 2024-04-25T00:32:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f242f7fe640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f243015c220 0x7f2430159930 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f242800c700 tx=0x7f242800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f24177fe640 1 -- 172.21.15.29:0/833763926 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428018070 con 0x7f243015c220 2024-04-25T00:32:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f24177fe640 1 -- 172.21.15.29:0/833763926 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242800ce10 con 0x7f243015c220 2024-04-25T00:32:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f243567c640 1 -- 172.21.15.29:0/833763926 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2430154820 con 0x7f243015c220 2024-04-25T00:32:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f24177fe640 1 -- 172.21.15.29:0/833763926 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242800a860 con 0x7f243015c220 2024-04-25T00:32:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f243567c640 1 -- 172.21.15.29:0/833763926 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243014bee0 con 0x7f243015c220 2024-04-25T00:32:57.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f243567c640 1 -- 172.21.15.29:0/833763926 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2430069270 con 0x7f243015c220 2024-04-25T00:32:57.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f24177fe640 1 -- 172.21.15.29:0/833763926 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f242801b070 con 0x7f243015c220 2024-04-25T00:32:57.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f24177fe640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f241c067a70 0x7f241c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:57.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f24177fe640 1 -- 172.21.15.29:0/833763926 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2428085550 con 0x7f243015c220 2024-04-25T00:32:57.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f242effd640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f241c067a70 0x7f241c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:57.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.918+0000 7f242effd640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f241c067a70 0x7f241c069f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f24301566b0 tx=0x7f2420008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:57.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:57.922+0000 7f24177fe640 1 -- 172.21.15.29:0/833763926 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2428055e10 con 0x7f243015c220 2024-04-25T00:32:58.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.078+0000 7f243567c640 1 -- 172.21.15.29:0/833763926 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f243006f3c0 con 0x7f243015c220 2024-04-25T00:32:58.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.078+0000 7f24177fe640 1 -- 172.21.15.29:0/833763926 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f2428055710 con 0x7f243015c220 2024-04-25T00:32:58.083 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:32:58.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.082+0000 7f243567c640 1 -- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f241c067a70 msgr2=0x7f241c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.082+0000 7f243567c640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f241c067a70 0x7f241c069f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f24301566b0 tx=0x7f2420008040 comp rx=0 tx=0).stop 2024-04-25T00:32:58.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.082+0000 7f243567c640 1 -- 172.21.15.29:0/833763926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f243015c220 msgr2=0x7f2430159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.082+0000 7f243567c640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f243015c220 0x7f2430159930 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f242800c700 tx=0x7f242800cbc0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.082+0000 7f243567c640 1 -- 172.21.15.29:0/833763926 shutdown_connections 2024-04-25T00:32:58.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.082+0000 7f243567c640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f243015c220 0x7f2430159930 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.082+0000 7f243567c640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24300d04a0 0x7f243014a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.082+0000 7f243567c640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f241c067a70 0x7f241c069f30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.082+0000 7f243567c640 1 --2- 172.21.15.29:0/833763926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24300cfa10 0x7f2430153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.082+0000 7f243567c640 1 -- 172.21.15.29:0/833763926 >> 172.21.15.29:0/833763926 conn(0x7f24300d1d60 msgr2=0x7f24300aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:58.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.082+0000 7f243567c640 1 -- 172.21.15.29:0/833763926 shutdown_connections 2024-04-25T00:32:58.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.082+0000 7f243567c640 1 -- 172.21.15.29:0/833763926 wait complete. 2024-04-25T00:32:58.101 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:58.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.210+0000 7f5debbe9640 1 Processor -- start 2024-04-25T00:32:58.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.210+0000 7f5debbe9640 1 -- start start 2024-04-25T00:32:58.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5debbe9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5de415e8d0 0x7f5de40cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5debbe9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5de40b4780 0x7f5de415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5debbe9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5de40b3720 con 0x7f5de415f1f0 2024-04-25T00:32:58.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5debbe9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5de40b42f0 con 0x7f5de415e8d0 2024-04-25T00:32:58.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5debbe9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5de415d7e0 con 0x7f5de40b4780 2024-04-25T00:32:58.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5de9985640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5de415e8d0 0x7f5de40cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5de9985640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5de415e8d0 0x7f5de40cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38320/0 (socket says 172.21.15.29:38320) 2024-04-25T00:32:58.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5de9985640 1 -- 172.21.15.29:0/568135765 learned_addr learned my addr 172.21.15.29:0/568135765 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:58.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5dea186640 1 --1- 172.21.15.29:0/568135765 >> v1:172.21.15.29:6789/0 conn(0x7f5de415f1f0 0x7f5de415b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46466/0 (socket says 172.21.15.29:46466) 2024-04-25T00:32:58.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5de9985640 1 -- 172.21.15.29:0/568135765 >> v2:172.21.15.157:3300/0 conn(0x7f5de40b4780 msgr2=0x7f5de415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:58.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5de9184640 1 --2- 172.21.15.29:0/568135765 >> v2:172.21.15.157:3300/0 conn(0x7f5de40b4780 0x7f5de415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5de9985640 1 --2- 172.21.15.29:0/568135765 >> v2:172.21.15.157:3300/0 conn(0x7f5de40b4780 0x7f5de415ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5de9985640 1 -- 172.21.15.29:0/568135765 >> v1:172.21.15.29:6789/0 conn(0x7f5de415f1f0 legacy=0x7f5de415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5de9985640 1 -- 172.21.15.29:0/568135765 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de40b39c0 con 0x7f5de415e8d0 2024-04-25T00:32:58.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5de9985640 1 --2- 172.21.15.29:0/568135765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5de415e8d0 0x7f5de40cc240 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5de4070910 tx=0x7f5de0020aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d574749e4dae76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5de8983640 1 -- 172.21.15.29:0/568135765 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00b6070 con 0x7f5de415e8d0 2024-04-25T00:32:58.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5de8983640 1 -- 172.21.15.29:0/568135765 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de00a9b70 con 0x7f5de415e8d0 2024-04-25T00:32:58.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.214+0000 7f5de8983640 1 -- 172.21.15.29:0/568135765 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de415c1c0 con 0x7f5de415e8d0 2024-04-25T00:32:58.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 -- 172.21.15.29:0/568135765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5de415e8d0 msgr2=0x7f5de40cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 --2- 172.21.15.29:0/568135765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5de415e8d0 0x7f5de40cc240 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5de4070910 tx=0x7f5de0020aa0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 -- 172.21.15.29:0/568135765 shutdown_connections 2024-04-25T00:32:58.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 --2- 172.21.15.29:0/568135765 >> v2:172.21.15.157:3300/0 conn(0x7f5de40b4780 0x7f5de415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 --2- 172.21.15.29:0/568135765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5de415e8d0 0x7f5de40cc240 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 -- 172.21.15.29:0/568135765 >> 172.21.15.29:0/568135765 conn(0x7f5de40b1870 msgr2=0x7f5de40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:58.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 -- 172.21.15.29:0/568135765 shutdown_connections 2024-04-25T00:32:58.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 -- 172.21.15.29:0/568135765 wait complete. 2024-04-25T00:32:58.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 Processor -- start 2024-04-25T00:32:58.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 -- start start 2024-04-25T00:32:58.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de40b4780 0x7f5de4158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5de415e8d0 0x7f5de41321f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5de415f1f0 0x7f5de4137e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4144a20 con 0x7f5de415e8d0 2024-04-25T00:32:58.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5de415f600 con 0x7f5de415f1f0 2024-04-25T00:32:58.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5de4131d60 con 0x7f5de40b4780 2024-04-25T00:32:58.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5de9184640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5de415e8d0 0x7f5de41321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5de9184640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5de415e8d0 0x7f5de41321f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38322/0 (socket says 172.21.15.29:38322) 2024-04-25T00:32:58.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5de9184640 1 -- 172.21.15.29:0/3297156322 learned_addr learned my addr 172.21.15.29:0/3297156322 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:58.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5de9985640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de40b4780 0x7f5de4158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5dea186640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5de415f1f0 0x7f5de4137e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5de9184640 1 -- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de40b4780 msgr2=0x7f5de4158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5de9184640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de40b4780 0x7f5de4158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5de9184640 1 -- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5de415f1f0 msgr2=0x7f5de4137e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5de9184640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5de415f1f0 0x7f5de4137e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5de9184640 1 -- 172.21.15.29:0/3297156322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de4131150 con 0x7f5de415e8d0 2024-04-25T00:32:58.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5de9184640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5de415e8d0 0x7f5de41321f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400c820 tx=0x7f5dd400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5dda7fc640 1 -- 172.21.15.29:0/3297156322 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4015070 con 0x7f5de415e8d0 2024-04-25T00:32:58.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5dda7fc640 1 -- 172.21.15.29:0/3297156322 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd4004240 con 0x7f5de415e8d0 2024-04-25T00:32:58.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 -- 172.21.15.29:0/3297156322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de4131430 con 0x7f5de415e8d0 2024-04-25T00:32:58.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5dda7fc640 1 -- 172.21.15.29:0/3297156322 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400ce10 con 0x7f5de415e8d0 2024-04-25T00:32:58.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.218+0000 7f5debbe9640 1 -- 172.21.15.29:0/3297156322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de41319f0 con 0x7f5de415e8d0 2024-04-25T00:32:58.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.222+0000 7f5debbe9640 1 -- 172.21.15.29:0/3297156322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de4069270 con 0x7f5de415e8d0 2024-04-25T00:32:58.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.222+0000 7f5dda7fc640 1 -- 172.21.15.29:0/3297156322 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f5dd4007830 con 0x7f5de415e8d0 2024-04-25T00:32:58.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.222+0000 7f5dda7fc640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5dc8067a70 0x7f5dc8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.222+0000 7f5dda7fc640 1 -- 172.21.15.29:0/3297156322 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5dd4085f80 con 0x7f5de415e8d0 2024-04-25T00:32:58.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.222+0000 7f5de9985640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5dc8067a70 0x7f5dc8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.222+0000 7f5de9985640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5dc8067a70 0x7f5dc8069f30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5de00b3000 tx=0x7f5de00ab070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.226+0000 7f5dda7fc640 1 -- 172.21.15.29:0/3297156322 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5dd4056730 con 0x7f5de415e8d0 2024-04-25T00:32:58.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.378+0000 7f5debbe9640 1 -- 172.21.15.29:0/3297156322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5de406e7b0 con 0x7f5de415e8d0 2024-04-25T00:32:58.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.378+0000 7f5dda7fc640 1 -- 172.21.15.29:0/3297156322 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f5dd4056030 con 0x7f5de415e8d0 2024-04-25T00:32:58.384 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:32:58.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.382+0000 7f5debbe9640 1 -- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5dc8067a70 msgr2=0x7f5dc8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.382+0000 7f5debbe9640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5dc8067a70 0x7f5dc8069f30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5de00b3000 tx=0x7f5de00ab070 comp rx=0 tx=0).stop 2024-04-25T00:32:58.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.382+0000 7f5debbe9640 1 -- 172.21.15.29:0/3297156322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5de415e8d0 msgr2=0x7f5de41321f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.382+0000 7f5debbe9640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5de415e8d0 0x7f5de41321f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400c820 tx=0x7f5dd400cce0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.382+0000 7f5debbe9640 1 -- 172.21.15.29:0/3297156322 shutdown_connections 2024-04-25T00:32:58.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.382+0000 7f5debbe9640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5dc8067a70 0x7f5dc8069f30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.382+0000 7f5debbe9640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5de415f1f0 0x7f5de4137e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.382+0000 7f5debbe9640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5de415e8d0 0x7f5de41321f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.382+0000 7f5debbe9640 1 --2- 172.21.15.29:0/3297156322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de40b4780 0x7f5de4158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.382+0000 7f5debbe9640 1 -- 172.21.15.29:0/3297156322 >> 172.21.15.29:0/3297156322 conn(0x7f5de40b1870 msgr2=0x7f5de40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:58.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.382+0000 7f5debbe9640 1 -- 172.21.15.29:0/3297156322 shutdown_connections 2024-04-25T00:32:58.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.382+0000 7f5debbe9640 1 -- 172.21.15.29:0/3297156322 wait complete. 2024-04-25T00:32:58.401 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:58.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.514+0000 7fa6c73d7640 1 Processor -- start 2024-04-25T00:32:58.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.514+0000 7fa6c73d7640 1 -- start start 2024-04-25T00:32:58.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6c73d7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa6c015d980 0x7fa6c0150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6c73d7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6c0151330 0x7fa6c006cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6c73d7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c00b3130 con 0x7fa6c00b2b70 2024-04-25T00:32:58.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6c73d7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c00cf560 con 0x7fa6c015d980 2024-04-25T00:32:58.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6c73d7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c015de30 con 0x7fa6c0151330 2024-04-25T00:32:58.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6c5173640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa6c00b2b70 0x7fa6c015c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46480/0 (socket says 172.21.15.29:46480) 2024-04-25T00:32:58.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6c4972640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa6c015d980 0x7fa6c0150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6c5173640 1 -- 172.21.15.29:0/693708877 learned_addr learned my addr 172.21.15.29:0/693708877 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:58.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6c4972640 1 -- 172.21.15.29:0/693708877 >> v2:172.21.15.157:3300/0 conn(0x7fa6c0151330 msgr2=0x7fa6c006cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:58.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6c4972640 1 --2- 172.21.15.29:0/693708877 >> v2:172.21.15.157:3300/0 conn(0x7fa6c0151330 0x7fa6c006cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6c4972640 1 -- 172.21.15.29:0/693708877 >> v1:172.21.15.29:6789/0 conn(0x7fa6c00b2b70 legacy=0x7fa6c015c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6c4972640 1 -- 172.21.15.29:0/693708877 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c00b3410 con 0x7fa6c015d980 2024-04-25T00:32:58.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6b7fff640 1 -- 172.21.15.29:0/693708877 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510017790 0 0) 0x7fa6c00b3130 con 0x7fa6c00b2b70 2024-04-25T00:32:58.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.518+0000 7fa6c4972640 1 --2- 172.21.15.29:0/693708877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c015d980 0x7fa6c0150df0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0158570 tx=0x7fa6b001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9725d0440a83663d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6b7fff640 1 -- 172.21.15.29:0/693708877 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00b0070 con 0x7fa6c015d980 2024-04-25T00:32:58.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6b7fff640 1 -- 172.21.15.29:0/693708877 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b00ab600 con 0x7fa6c015d980 2024-04-25T00:32:58.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6b7fff640 1 -- 172.21.15.29:0/693708877 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00ab9d0 con 0x7fa6c015d980 2024-04-25T00:32:58.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 -- 172.21.15.29:0/693708877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c015d980 msgr2=0x7fa6c0150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 --2- 172.21.15.29:0/693708877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c015d980 0x7fa6c0150df0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0158570 tx=0x7fa6b001c480 comp rx=0 tx=0).stop 2024-04-25T00:32:58.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 -- 172.21.15.29:0/693708877 shutdown_connections 2024-04-25T00:32:58.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 --2- 172.21.15.29:0/693708877 >> v2:172.21.15.157:3300/0 conn(0x7fa6c0151330 0x7fa6c006cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 --2- 172.21.15.29:0/693708877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c015d980 0x7fa6c0150df0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 -- 172.21.15.29:0/693708877 >> 172.21.15.29:0/693708877 conn(0x7fa6c00b1870 msgr2=0x7fa6c00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:58.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 -- 172.21.15.29:0/693708877 shutdown_connections 2024-04-25T00:32:58.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 -- 172.21.15.29:0/693708877 wait complete. 2024-04-25T00:32:58.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 Processor -- start 2024-04-25T00:32:58.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 -- start start 2024-04-25T00:32:58.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c00b2b70 0x7fa6c0144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c0151330 0x7fa6c01451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c015d980 0x7fa6c015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c015ce50 con 0x7fa6c015d980 2024-04-25T00:32:58.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c015ccd0 con 0x7fa6c00b2b70 2024-04-25T00:32:58.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c73d7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c0151760 con 0x7fa6c0151330 2024-04-25T00:32:58.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c5974640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c015d980 0x7fa6c015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c5974640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c015d980 0x7fa6c015b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38336/0 (socket says 172.21.15.29:38336) 2024-04-25T00:32:58.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c5974640 1 -- 172.21.15.29:0/533271117 learned_addr learned my addr 172.21.15.29:0/533271117 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:58.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c5173640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c00b2b70 0x7fa6c0144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c4972640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c0151330 0x7fa6c01451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c5974640 1 -- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c0151330 msgr2=0x7fa6c01451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c5974640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c0151330 0x7fa6c01451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c5974640 1 -- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c00b2b70 msgr2=0x7fa6c0144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c5974640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c00b2b70 0x7fa6c0144c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c5974640 1 -- 172.21.15.29:0/533271117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c01548e0 con 0x7fa6c015d980 2024-04-25T00:32:58.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.522+0000 7fa6c4972640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c0151330 0x7fa6c01451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:58.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.526+0000 7fa6c5173640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c00b2b70 0x7fa6c0144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:58.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.526+0000 7fa6c5974640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c015d980 0x7fa6c015b4b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800b970 tx=0x7fa6b800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.526+0000 7fa6b5ffb640 1 -- 172.21.15.29:0/533271117 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8019070 con 0x7fa6c015d980 2024-04-25T00:32:58.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.526+0000 7fa6b5ffb640 1 -- 172.21.15.29:0/533271117 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b800cb60 con 0x7fa6c015d980 2024-04-25T00:32:58.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.526+0000 7fa6c73d7640 1 -- 172.21.15.29:0/533271117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c00ccbf0 con 0x7fa6c015d980 2024-04-25T00:32:58.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.526+0000 7fa6b5ffb640 1 -- 172.21.15.29:0/533271117 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8014440 con 0x7fa6c015d980 2024-04-25T00:32:58.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.526+0000 7fa6c73d7640 1 -- 172.21.15.29:0/533271117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c00cc910 con 0x7fa6c015d980 2024-04-25T00:32:58.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.526+0000 7fa6c73d7640 1 -- 172.21.15.29:0/533271117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c0069270 con 0x7fa6c015d980 2024-04-25T00:32:58.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.526+0000 7fa6b5ffb640 1 -- 172.21.15.29:0/533271117 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fa6b801c070 con 0x7fa6c015d980 2024-04-25T00:32:58.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.526+0000 7fa6b5ffb640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa6a4067aa0 0x7fa6a4069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.526+0000 7fa6b5ffb640 1 -- 172.21.15.29:0/533271117 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa6b8085c50 con 0x7fa6c015d980 2024-04-25T00:32:58.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.526+0000 7fa6c5173640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa6a4067aa0 0x7fa6a4069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.526+0000 7fa6c5173640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa6a4067aa0 0x7fa6a4069f60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc000f70 tx=0x7fa6bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.530+0000 7fa6b5ffb640 1 -- 172.21.15.29:0/533271117 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa6b808b020 con 0x7fa6c015d980 2024-04-25T00:32:58.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.686+0000 7fa6c73d7640 1 -- 172.21.15.29:0/533271117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa6c00cf910 con 0x7fa6c015d980 2024-04-25T00:32:58.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.686+0000 7fa6b5ffb640 1 -- 172.21.15.29:0/533271117 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7fa6c01548e0 con 0x7fa6c015d980 2024-04-25T00:32:58.693 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:32:58.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.690+0000 7fa6c73d7640 1 -- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa6a4067aa0 msgr2=0x7fa6a4069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.690+0000 7fa6c73d7640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa6a4067aa0 0x7fa6a4069f60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc000f70 tx=0x7fa6bc009040 comp rx=0 tx=0).stop 2024-04-25T00:32:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.690+0000 7fa6c73d7640 1 -- 172.21.15.29:0/533271117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c015d980 msgr2=0x7fa6c015b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.690+0000 7fa6c73d7640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c015d980 0x7fa6c015b4b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800b970 tx=0x7fa6b800be30 comp rx=0 tx=0).stop 2024-04-25T00:32:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.690+0000 7fa6c73d7640 1 -- 172.21.15.29:0/533271117 shutdown_connections 2024-04-25T00:32:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.690+0000 7fa6c73d7640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c015d980 0x7fa6c015b4b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.690+0000 7fa6c73d7640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6c0151330 0x7fa6c01451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.690+0000 7fa6c73d7640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa6a4067aa0 0x7fa6a4069f60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.690+0000 7fa6c73d7640 1 --2- 172.21.15.29:0/533271117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c00b2b70 0x7fa6c0144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.690+0000 7fa6c73d7640 1 -- 172.21.15.29:0/533271117 >> 172.21.15.29:0/533271117 conn(0x7fa6c00b1870 msgr2=0x7fa6c015c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.690+0000 7fa6c73d7640 1 -- 172.21.15.29:0/533271117 shutdown_connections 2024-04-25T00:32:58.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.690+0000 7fa6c73d7640 1 -- 172.21.15.29:0/533271117 wait complete. 2024-04-25T00:32:58.711 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:32:58.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.826+0000 7fba6555a640 1 Processor -- start 2024-04-25T00:32:58.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.826+0000 7fba6555a640 1 -- start start 2024-04-25T00:32:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.826+0000 7fba6555a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600b3c30 0x7fba6006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.826+0000 7fba6555a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba600b41e0 con 0x7fba600b4b10 2024-04-25T00:32:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.826+0000 7fba6555a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba6006d960 con 0x7fba600b3140 2024-04-25T00:32:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.826+0000 7fba6555a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba6006dc40 con 0x7fba600b3c30 2024-04-25T00:32:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.826+0000 7fba5f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fba600b4b10 0x7fba600ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46496/0 (socket says 172.21.15.29:46496) 2024-04-25T00:32:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.826+0000 7fba5f7fe640 1 -- 172.21.15.29:0/3013041812 learned_addr learned my addr 172.21.15.29:0/3013041812 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5e7fc640 1 --2- 172.21.15.29:0/3013041812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600b3c30 0x7fba6006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2450556867 0 0) 0x7fba600b41e0 con 0x7fba600b4b10 2024-04-25T00:32:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba48003530 con 0x7fba600b4b10 2024-04-25T00:32:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3514164071 0 0) 0x7fba48003530 con 0x7fba600b4b10 2024-04-25T00:32:58.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600b3c30 msgr2=0x7fba6006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 --2- 172.21.15.29:0/3013041812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600b3c30 0x7fba6006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 >> v1:172.21.15.157:6789/0 conn(0x7fba600b3140 legacy=0x7fba600b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6006df20 con 0x7fba600b4b10 2024-04-25T00:32:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fba58001a00 con 0x7fba600b4b10 2024-04-25T00:32:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 >> v1:172.21.15.29:6789/0 conn(0x7fba600b4b10 legacy=0x7fba600ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 --2- 172.21.15.29:0/3013041812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600b3c30 0x7fba480039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 --2- 172.21.15.29:0/3013041812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba48003f70 0x7fba48004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 --2- 172.21.15.29:0/3013041812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba48004d50 0x7fba480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fba6006dc40 con 0x7fba48004d50 2024-04-25T00:32:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba4800bab0 con 0x7fba48003f70 2024-04-25T00:32:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba4800bcb0 con 0x7fba600b3c30 2024-04-25T00:32:58.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5f7fe640 1 --2- 172.21.15.29:0/3013041812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba48004d50 0x7fba480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5effd640 1 --2- 172.21.15.29:0/3013041812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600b3c30 0x7fba480039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5f7fe640 1 -- 172.21.15.29:0/3013041812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600b3c30 msgr2=0x7fba480039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5f7fe640 1 --2- 172.21.15.29:0/3013041812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600b3c30 0x7fba480039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5f7fe640 1 -- 172.21.15.29:0/3013041812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba48003f70 msgr2=0x7fba48004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5f7fe640 1 --2- 172.21.15.29:0/3013041812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba48003f70 0x7fba48004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5f7fe640 1 -- 172.21.15.29:0/3013041812 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba48003530 con 0x7fba48004d50 2024-04-25T00:32:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5effd640 1 --2- 172.21.15.29:0/3013041812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600b3c30 0x7fba480039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:58.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5f7fe640 1 --2- 172.21.15.29:0/3013041812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba48004d50 0x7fba480051d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fba5800a850 tx=0x7fba5801d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b0bd5e63ff8298 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5800a4a0 con 0x7fba48004d50 2024-04-25T00:32:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba580ad650 con 0x7fba48004d50 2024-04-25T00:32:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5dffb640 1 -- 172.21.15.29:0/3013041812 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba580ada80 con 0x7fba48004d50 2024-04-25T00:32:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba6555a640 1 -- 172.21.15.29:0/3013041812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba48004d50 msgr2=0x7fba480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba6555a640 1 --2- 172.21.15.29:0/3013041812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba48004d50 0x7fba480051d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fba5800a850 tx=0x7fba5801d3a0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.830+0000 7fba5f7fe640 1 -- 172.21.15.29:0/3013041812 reap_dead start 2024-04-25T00:32:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 -- 172.21.15.29:0/3013041812 shutdown_connections 2024-04-25T00:32:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 -- 172.21.15.29:0/3013041812 >> 172.21.15.29:0/3013041812 conn(0x7fba600b1870 msgr2=0x7fba600b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 -- 172.21.15.29:0/3013041812 shutdown_connections 2024-04-25T00:32:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 -- 172.21.15.29:0/3013041812 wait complete. 2024-04-25T00:32:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 Processor -- start 2024-04-25T00:32:58.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 -- start start 2024-04-25T00:32:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba48004d50 0x7fba6013c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba60142520 0x7fba60142980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba60143370 0x7fba601437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fba60145a90 con 0x7fba48004d50 2024-04-25T00:32:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba60145910 con 0x7fba60143370 2024-04-25T00:32:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba6006e440 con 0x7fba60142520 2024-04-25T00:32:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba5effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba48004d50 0x7fba6013c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba5effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba48004d50 0x7fba6013c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38352/0 (socket says 172.21.15.29:38352) 2024-04-25T00:32:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba5effd640 1 -- 172.21.15.29:0/3456879164 learned_addr learned my addr 172.21.15.29:0/3456879164 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba5f7fe640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba60143370 0x7fba601437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba5e7fc640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba60142520 0x7fba60142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba5effd640 1 -- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba60142520 msgr2=0x7fba60142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba5effd640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba60142520 0x7fba60142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba5effd640 1 -- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba60143370 msgr2=0x7fba601437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba5effd640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba60143370 0x7fba601437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba5effd640 1 -- 172.21.15.29:0/3456879164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6013cac0 con 0x7fba48004d50 2024-04-25T00:32:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba5effd640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba48004d50 0x7fba6013c580 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fba5000d770 tx=0x7fba5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba477fe640 1 -- 172.21.15.29:0/3456879164 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50005070 con 0x7fba48004d50 2024-04-25T00:32:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba477fe640 1 -- 172.21.15.29:0/3456879164 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba50005230 con 0x7fba48004d50 2024-04-25T00:32:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 -- 172.21.15.29:0/3456879164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba60158870 con 0x7fba48004d50 2024-04-25T00:32:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba477fe640 1 -- 172.21.15.29:0/3456879164 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50007030 con 0x7fba48004d50 2024-04-25T00:32:58.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.834+0000 7fba6555a640 1 -- 172.21.15.29:0/3456879164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba60158df0 con 0x7fba48004d50 2024-04-25T00:32:58.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.838+0000 7fba6555a640 1 -- 172.21.15.29:0/3456879164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba60069270 con 0x7fba48004d50 2024-04-25T00:32:58.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.838+0000 7fba477fe640 1 -- 172.21.15.29:0/3456879164 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fba5001a020 con 0x7fba48004d50 2024-04-25T00:32:58.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.838+0000 7fba477fe640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba4c067aa0 0x7fba4c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.838+0000 7fba477fe640 1 -- 172.21.15.29:0/3456879164 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fba500858a0 con 0x7fba48004d50 2024-04-25T00:32:58.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.838+0000 7fba5e7fc640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba4c067aa0 0x7fba4c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.838+0000 7fba5e7fc640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba4c067aa0 0x7fba4c069f60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fba540047c0 tx=0x7fba54008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:58.842+0000 7fba477fe640 1 -- 172.21.15.29:0/3456879164 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fba50055fc0 con 0x7fba48004d50 2024-04-25T00:32:59.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.034+0000 7fba6555a640 1 -- 172.21.15.29:0/3456879164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba6015e8d0 con 0x7fba48004d50 2024-04-25T00:32:59.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.038+0000 7fba477fe640 1 -- 172.21.15.29:0/3456879164 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5186 (secure 0 0 0) 0x7fba500558c0 con 0x7fba48004d50 2024-04-25T00:32:59.041 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:59.042 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2100384916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2100384916},{"type":"v1","addr":"172.21.15.157:6835","nonce":2100384916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4984,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1430513194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1430513194},{"type":"v1","addr":"172.21.15.157:6837","nonce":1430513194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3354831881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3354831881},{"type":"v1","addr":"172.21.15.29:6833","nonce":3354831881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:32:33.777640+0000","modified":"2024-04-25T00:32:36.837733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1468158138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1468158138},{"type":"v1","addr":"172.21.15.29:6835","nonce":1468158138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":2}]} 2024-04-25T00:32:59.042 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 16 2024-04-25T00:32:59.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.038+0000 7fba6555a640 1 -- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba4c067aa0 msgr2=0x7fba4c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.038+0000 7fba6555a640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba4c067aa0 0x7fba4c069f60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fba540047c0 tx=0x7fba54008040 comp rx=0 tx=0).stop 2024-04-25T00:32:59.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.038+0000 7fba6555a640 1 -- 172.21.15.29:0/3456879164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba48004d50 msgr2=0x7fba6013c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.038+0000 7fba6555a640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba48004d50 0x7fba6013c580 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fba5000d770 tx=0x7fba5000dc30 comp rx=0 tx=0).stop 2024-04-25T00:32:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.038+0000 7fba6555a640 1 -- 172.21.15.29:0/3456879164 shutdown_connections 2024-04-25T00:32:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.038+0000 7fba6555a640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba4c067aa0 0x7fba4c069f60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.038+0000 7fba6555a640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba60143370 0x7fba601437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.038+0000 7fba6555a640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba60142520 0x7fba60142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.038+0000 7fba6555a640 1 --2- 172.21.15.29:0/3456879164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba48004d50 0x7fba6013c580 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.038+0000 7fba6555a640 1 -- 172.21.15.29:0/3456879164 >> 172.21.15.29:0/3456879164 conn(0x7fba600b1870 msgr2=0x7fba6015edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.038+0000 7fba6555a640 1 -- 172.21.15.29:0/3456879164 shutdown_connections 2024-04-25T00:32:59.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.038+0000 7fba6555a640 1 -- 172.21.15.29:0/3456879164 wait complete. 2024-04-25T00:32:59.059 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:59.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.166+0000 7f5c49cec640 1 Processor -- start 2024-04-25T00:32:59.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c49cec640 1 -- start start 2024-04-25T00:32:59.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c49cec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c4415e8d0 0x7f5c440cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c49cec640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5c440b4780 0x7f5c4415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c49cec640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c440b3720 con 0x7f5c4415f1f0 2024-04-25T00:32:59.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c49cec640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5c440b42f0 con 0x7f5c440b4780 2024-04-25T00:32:59.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c49cec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5c4415d7e0 con 0x7f5c4415e8d0 2024-04-25T00:32:59.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c42ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5c440b4780 0x7f5c4415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c43fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5c4415f1f0 0x7f5c4415b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46502/0 (socket says 172.21.15.29:46502) 2024-04-25T00:32:59.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c43fff640 1 -- 172.21.15.29:0/974437655 learned_addr learned my addr 172.21.15.29:0/974437655 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:59.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c437fe640 1 --2- 172.21.15.29:0/974437655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c4415e8d0 0x7f5c440cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c42ffd640 1 -- 172.21.15.29:0/974437655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c4415e8d0 msgr2=0x7f5c440cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:59.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c42ffd640 1 --2- 172.21.15.29:0/974437655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c4415e8d0 0x7f5c440cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c42ffd640 1 -- 172.21.15.29:0/974437655 >> v1:172.21.15.29:6789/0 conn(0x7f5c4415f1f0 legacy=0x7f5c4415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c42ffd640 1 -- 172.21.15.29:0/974437655 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c440b39c0 con 0x7f5c440b4780 2024-04-25T00:32:59.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c427fc640 1 -- 172.21.15.29:0/974437655 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 353263043 0 0) 0x7f5c440b3720 con 0x7f5c4415f1f0 2024-04-25T00:32:59.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.170+0000 7f5c437fe640 1 --2- 172.21.15.29:0/974437655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c4415e8d0 0x7f5c440cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:59.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c42ffd640 1 --2- 172.21.15.29:0/974437655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c440b4780 0x7f5c4415ecb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5c380097c0 tx=0x7f5c3801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda5a20cf3d144a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:59.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c427fc640 1 -- 172.21.15.29:0/974437655 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c380a7070 con 0x7f5c440b4780 2024-04-25T00:32:59.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c427fc640 1 -- 172.21.15.29:0/974437655 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c380b0070 con 0x7f5c440b4780 2024-04-25T00:32:59.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c427fc640 1 -- 172.21.15.29:0/974437655 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4415c1c0 con 0x7f5c440b4780 2024-04-25T00:32:59.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 -- 172.21.15.29:0/974437655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c440b4780 msgr2=0x7f5c4415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 --2- 172.21.15.29:0/974437655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c440b4780 0x7f5c4415ecb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5c380097c0 tx=0x7f5c3801c470 comp rx=0 tx=0).stop 2024-04-25T00:32:59.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 -- 172.21.15.29:0/974437655 shutdown_connections 2024-04-25T00:32:59.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 --2- 172.21.15.29:0/974437655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c440b4780 0x7f5c4415ecb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 --2- 172.21.15.29:0/974437655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c4415e8d0 0x7f5c440cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 -- 172.21.15.29:0/974437655 >> 172.21.15.29:0/974437655 conn(0x7f5c440b1870 msgr2=0x7f5c440b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:59.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 -- 172.21.15.29:0/974437655 shutdown_connections 2024-04-25T00:32:59.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 -- 172.21.15.29:0/974437655 wait complete. 2024-04-25T00:32:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 Processor -- start 2024-04-25T00:32:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 -- start start 2024-04-25T00:32:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c440b4780 0x7f5c4415a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4415e8d0 0x7f5c441f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c4415f1f0 0x7f5c44139d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5c44144a20 con 0x7f5c440b4780 2024-04-25T00:32:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4415f620 con 0x7f5c4415e8d0 2024-04-25T00:32:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5c441f5f70 con 0x7f5c4415f1f0 2024-04-25T00:32:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c437fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c440b4780 0x7f5c4415a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c437fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c440b4780 0x7f5c4415a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38356/0 (socket says 172.21.15.29:38356) 2024-04-25T00:32:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c437fe640 1 -- 172.21.15.29:0/3047725917 learned_addr learned my addr 172.21.15.29:0/3047725917 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c43fff640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c4415f1f0 0x7f5c44139d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c437fe640 1 -- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c4415f1f0 msgr2=0x7f5c44139d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c42ffd640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4415e8d0 0x7f5c441f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c437fe640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c4415f1f0 0x7f5c44139d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c437fe640 1 -- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4415e8d0 msgr2=0x7f5c441f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c437fe640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4415e8d0 0x7f5c441f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c437fe640 1 -- 172.21.15.29:0/3047725917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c441f5530 con 0x7f5c440b4780 2024-04-25T00:32:59.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c43fff640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c4415f1f0 0x7f5c44139d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:59.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c437fe640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c440b4780 0x7f5c4415a0e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400e840 tx=0x7f5c3400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:59.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c2bfff640 1 -- 172.21.15.29:0/3047725917 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34017070 con 0x7f5c440b4780 2024-04-25T00:32:59.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c2bfff640 1 -- 172.21.15.29:0/3047725917 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c3400c8d0 con 0x7f5c440b4780 2024-04-25T00:32:59.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c2bfff640 1 -- 172.21.15.29:0/3047725917 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400f040 con 0x7f5c440b4780 2024-04-25T00:32:59.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 -- 172.21.15.29:0/3047725917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c441f5810 con 0x7f5c440b4780 2024-04-25T00:32:59.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.174+0000 7f5c49cec640 1 -- 172.21.15.29:0/3047725917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c4415a840 con 0x7f5c440b4780 2024-04-25T00:32:59.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.178+0000 7f5c49cec640 1 -- 172.21.15.29:0/3047725917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c44069270 con 0x7f5c440b4780 2024-04-25T00:32:59.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.178+0000 7f5c2bfff640 1 -- 172.21.15.29:0/3047725917 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f5c3401a070 con 0x7f5c440b4780 2024-04-25T00:32:59.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.178+0000 7f5c2bfff640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5c30067aa0 0x7f5c30069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.178+0000 7f5c2bfff640 1 -- 172.21.15.29:0/3047725917 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5c34085d60 con 0x7f5c440b4780 2024-04-25T00:32:59.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.178+0000 7f5c42ffd640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5c30067aa0 0x7f5c30069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.178+0000 7f5c42ffd640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5c30067aa0 0x7f5c30069f60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5c380af000 tx=0x7f5c38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:59.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.182+0000 7f5c2bfff640 1 -- 172.21.15.29:0/3047725917 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5c34056470 con 0x7f5c440b4780 2024-04-25T00:32:59.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.338+0000 7f5c49cec640 1 -- 172.21.15.29:0/3047725917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c4406e7b0 con 0x7f5c440b4780 2024-04-25T00:32:59.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.342+0000 7f5c2bfff640 1 -- 172.21.15.29:0/3047725917 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16621 (secure 0 0 0) 0x7f5c34055d70 con 0x7f5c440b4780 2024-04-25T00:32:59.346 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:59.346 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":33,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:32:35.426396+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:32:32.254456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:32:32.779372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.342+0000 7f5c49cec640 1 -- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5c30067aa0 msgr2=0x7f5c30069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.342+0000 7f5c49cec640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5c30067aa0 0x7f5c30069f60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5c380af000 tx=0x7f5c38008040 comp rx=0 tx=0).stop 2024-04-25T00:32:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.342+0000 7f5c49cec640 1 -- 172.21.15.29:0/3047725917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c440b4780 msgr2=0x7f5c4415a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.342+0000 7f5c49cec640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c440b4780 0x7f5c4415a0e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400e840 tx=0x7f5c3400ed00 comp rx=0 tx=0).stop 2024-04-25T00:32:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.342+0000 7f5c49cec640 1 -- 172.21.15.29:0/3047725917 shutdown_connections 2024-04-25T00:32:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.342+0000 7f5c49cec640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c4415f1f0 0x7f5c44139d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.342+0000 7f5c49cec640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5c30067aa0 0x7f5c30069f60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.342+0000 7f5c49cec640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4415e8d0 0x7f5c441f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.342+0000 7f5c49cec640 1 --2- 172.21.15.29:0/3047725917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c440b4780 0x7f5c4415a0e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.342+0000 7f5c49cec640 1 -- 172.21.15.29:0/3047725917 >> 172.21.15.29:0/3047725917 conn(0x7f5c440b1870 msgr2=0x7f5c4415b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.342+0000 7f5c49cec640 1 -- 172.21.15.29:0/3047725917 shutdown_connections 2024-04-25T00:32:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.342+0000 7f5c49cec640 1 -- 172.21.15.29:0/3047725917 wait complete. 2024-04-25T00:32:59.364 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:32:59.364 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:32:59.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.474+0000 7f14b9029640 1 Processor -- start 2024-04-25T00:32:59.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.474+0000 7f14b9029640 1 -- start start 2024-04-25T00:32:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b9029640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f14b406dce0 0x7f14b4150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b9029640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f14b40b2b70 0x7f14b4151330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b9029640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b40b3130 con 0x7f14b415d980 2024-04-25T00:32:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b9029640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f14b40cf560 con 0x7f14b40b2b70 2024-04-25T00:32:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b9029640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f14b40b3410 con 0x7f14b406dce0 2024-04-25T00:32:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b359e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f14b40b2b70 0x7f14b4151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b359e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f14b40b2b70 0x7f14b4151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38358/0 (socket says 172.21.15.29:38358) 2024-04-25T00:32:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b359e640 1 -- 172.21.15.29:0/1614442787 learned_addr learned my addr 172.21.15.29:0/1614442787 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14abfff640 1 --2- 172.21.15.29:0/1614442787 >> v2:172.21.15.157:3300/0 conn(0x7f14b406dce0 0x7f14b4150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b359e640 1 -- 172.21.15.29:0/1614442787 >> v2:172.21.15.157:3300/0 conn(0x7f14b406dce0 msgr2=0x7f14b4150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:59.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b359e640 1 --2- 172.21.15.29:0/1614442787 >> v2:172.21.15.157:3300/0 conn(0x7f14b406dce0 0x7f14b4150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b359e640 1 -- 172.21.15.29:0/1614442787 >> v1:172.21.15.157:6789/0 conn(0x7f14b415d980 legacy=0x7f14b406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:59.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b359e640 1 -- 172.21.15.29:0/1614442787 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b40b36f0 con 0x7f14b40b2b70 2024-04-25T00:32:59.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b359e640 1 --2- 172.21.15.29:0/1614442787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14b40b2b70 0x7f14b4151330 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f14b40b2720 tx=0x7f14ac01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4206a7b2810800 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:59.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.478+0000 7f14b259c640 1 -- 172.21.15.29:0/1614442787 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0a57c0 con 0x7f14b40b2b70 2024-04-25T00:32:59.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b259c640 1 -- 172.21.15.29:0/1614442787 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14ac0a5960 con 0x7f14b40b2b70 2024-04-25T00:32:59.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b259c640 1 -- 172.21.15.29:0/1614442787 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0a5d30 con 0x7f14b40b2b70 2024-04-25T00:32:59.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 -- 172.21.15.29:0/1614442787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14b40b2b70 msgr2=0x7f14b4151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 --2- 172.21.15.29:0/1614442787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14b40b2b70 0x7f14b4151330 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f14b40b2720 tx=0x7f14ac01c980 comp rx=0 tx=0).stop 2024-04-25T00:32:59.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 -- 172.21.15.29:0/1614442787 shutdown_connections 2024-04-25T00:32:59.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 --2- 172.21.15.29:0/1614442787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14b40b2b70 0x7f14b4151330 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 --2- 172.21.15.29:0/1614442787 >> v2:172.21.15.157:3300/0 conn(0x7f14b406dce0 0x7f14b4150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 -- 172.21.15.29:0/1614442787 >> 172.21.15.29:0/1614442787 conn(0x7f14b40b1870 msgr2=0x7f14b40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:59.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 -- 172.21.15.29:0/1614442787 shutdown_connections 2024-04-25T00:32:59.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 -- 172.21.15.29:0/1614442787 wait complete. 2024-04-25T00:32:59.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 Processor -- start 2024-04-25T00:32:59.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 -- start start 2024-04-25T00:32:59.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14b406dce0 0x7f14b41519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14b40b2b70 0x7f14b4153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14b415d980 0x7f14b413f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f14b415b4b0 con 0x7f14b415d980 2024-04-25T00:32:59.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4151870 con 0x7f14b40b2b70 2024-04-25T00:32:59.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f14b415b630 con 0x7f14b406dce0 2024-04-25T00:32:59.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b359e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14b415d980 0x7f14b413f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b359e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14b415d980 0x7f14b413f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38370/0 (socket says 172.21.15.29:38370) 2024-04-25T00:32:59.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b359e640 1 -- 172.21.15.29:0/2034408223 learned_addr learned my addr 172.21.15.29:0/2034408223 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b2d9d640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14b406dce0 0x7f14b41519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14abfff640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14b40b2b70 0x7f14b4153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b359e640 1 -- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14b406dce0 msgr2=0x7f14b41519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b359e640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14b406dce0 0x7f14b41519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b359e640 1 -- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14b40b2b70 msgr2=0x7f14b4153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b359e640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14b40b2b70 0x7f14b4153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b359e640 1 -- 172.21.15.29:0/2034408223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b4151fd0 con 0x7f14b415d980 2024-04-25T00:32:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b359e640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14b415d980 0x7f14b413f480 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f14ac01a5b0 tx=0x7f14ac019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14ab7fe640 1 -- 172.21.15.29:0/2034408223 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0b8880 con 0x7f14b415d980 2024-04-25T00:32:59.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14ab7fe640 1 -- 172.21.15.29:0/2034408223 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14ac0be070 con 0x7f14b415d980 2024-04-25T00:32:59.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 -- 172.21.15.29:0/2034408223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b41522b0 con 0x7f14b415d980 2024-04-25T00:32:59.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14ab7fe640 1 -- 172.21.15.29:0/2034408223 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0b8c80 con 0x7f14b415d980 2024-04-25T00:32:59.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.482+0000 7f14b9029640 1 -- 172.21.15.29:0/2034408223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b4152870 con 0x7f14b415d980 2024-04-25T00:32:59.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.486+0000 7f14b9029640 1 -- 172.21.15.29:0/2034408223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b4069270 con 0x7f14b415d980 2024-04-25T00:32:59.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.486+0000 7f14ab7fe640 1 -- 172.21.15.29:0/2034408223 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f14ac0c3460 con 0x7f14b415d980 2024-04-25T00:32:59.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.486+0000 7f14ab7fe640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1494067a70 0x7f1494069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.486+0000 7f14ab7fe640 1 -- 172.21.15.29:0/2034408223 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f14ac121250 con 0x7f14b415d980 2024-04-25T00:32:59.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.486+0000 7f14b2d9d640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1494067a70 0x7f1494069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.486+0000 7f14b2d9d640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1494067a70 0x7f1494069f30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14a400b810 tx=0x7f14a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:59.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.490+0000 7f14ab7fe640 1 -- 172.21.15.29:0/2034408223 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f14ac0165a0 con 0x7f14b415d980 2024-04-25T00:32:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.686+0000 7f14b9029640 1 -- 172.21.15.29:0/2034408223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14b40cf910 con 0x7f14b415d980 2024-04-25T00:32:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.690+0000 7f14ab7fe640 1 -- 172.21.15.29:0/2034408223 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5186 (secure 0 0 0) 0x7f14ac0f1b10 con 0x7f14b415d980 2024-04-25T00:32:59.694 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:32:59.694 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2100384916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2100384916},{"type":"v1","addr":"172.21.15.157:6835","nonce":2100384916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4984,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1430513194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1430513194},{"type":"v1","addr":"172.21.15.157:6837","nonce":1430513194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3354831881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3354831881},{"type":"v1","addr":"172.21.15.29:6833","nonce":3354831881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:32:33.777640+0000","modified":"2024-04-25T00:32:36.837733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1468158138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1468158138},{"type":"v1","addr":"172.21.15.29:6835","nonce":1468158138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":2}]} 2024-04-25T00:32:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 16 2024-04-25T00:32:59.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.690+0000 7f14b9029640 1 -- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1494067a70 msgr2=0x7f1494069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.690+0000 7f14b9029640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1494067a70 0x7f1494069f30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14a400b810 tx=0x7f14a4008040 comp rx=0 tx=0).stop 2024-04-25T00:32:59.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.690+0000 7f14b9029640 1 -- 172.21.15.29:0/2034408223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14b415d980 msgr2=0x7f14b413f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.690+0000 7f14b9029640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14b415d980 0x7f14b413f480 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f14ac01a5b0 tx=0x7f14ac019720 comp rx=0 tx=0).stop 2024-04-25T00:32:59.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.690+0000 7f14b9029640 1 -- 172.21.15.29:0/2034408223 shutdown_connections 2024-04-25T00:32:59.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.690+0000 7f14b9029640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14b415d980 0x7f14b413f480 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.690+0000 7f14b9029640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1494067a70 0x7f1494069f30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.690+0000 7f14b9029640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14b40b2b70 0x7f14b4153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.690+0000 7f14b9029640 1 --2- 172.21.15.29:0/2034408223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14b406dce0 0x7f14b41519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.690+0000 7f14b9029640 1 -- 172.21.15.29:0/2034408223 >> 172.21.15.29:0/2034408223 conn(0x7f14b40b1870 msgr2=0x7f14b415df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:59.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.690+0000 7f14b9029640 1 -- 172.21.15.29:0/2034408223 shutdown_connections 2024-04-25T00:32:59.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.690+0000 7f14b9029640 1 -- 172.21.15.29:0/2034408223 wait complete. 2024-04-25T00:32:59.712 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:32:59.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.826+0000 7f8fee1d9640 1 Processor -- start 2024-04-25T00:32:59.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.826+0000 7f8fee1d9640 1 -- start start 2024-04-25T00:32:59.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fee1d9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8fe806e440 0x7f8fe80b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fee1d9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe80b41e0 con 0x7f8fe80b3140 2024-04-25T00:32:59.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fee1d9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe806d960 con 0x7f8fe80b3c30 2024-04-25T00:32:59.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fee1d9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe80b5050 con 0x7f8fe806e440 2024-04-25T00:32:59.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fe7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8fe806e440 0x7f8fe80b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fe77fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8fe80b3140 0x7f8fe80b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46506/0 (socket says 172.21.15.29:46506) 2024-04-25T00:32:59.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fe77fe640 1 -- 172.21.15.29:0/1484990441 learned_addr learned my addr 172.21.15.29:0/1484990441 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:32:59.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fe7fff640 1 -- 172.21.15.29:0/1484990441 >> v1:172.21.15.157:6789/0 conn(0x7f8fe80b3c30 legacy=0x7f8fe80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:59.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fe7fff640 1 -- 172.21.15.29:0/1484990441 >> v1:172.21.15.29:6789/0 conn(0x7f8fe80b3140 legacy=0x7f8fe80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fe7fff640 1 -- 172.21.15.29:0/1484990441 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe806dc40 con 0x7f8fe806e440 2024-04-25T00:32:59.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fe67fc640 1 -- 172.21.15.29:0/1484990441 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1046152919 0 0) 0x7f8fe80b41e0 con 0x7f8fe80b3140 2024-04-25T00:32:59.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fe7fff640 1 --2- 172.21.15.29:0/1484990441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe806e440 0x7f8fe80b4b10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009dc0 tx=0x7f8fe001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccbfd1e75819bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:59.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fe67fc640 1 -- 172.21.15.29:0/1484990441 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00b2070 con 0x7f8fe806e440 2024-04-25T00:32:59.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fe67fc640 1 -- 172.21.15.29:0/1484990441 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe00a57d0 con 0x7f8fe806e440 2024-04-25T00:32:59.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.830+0000 7f8fe67fc640 1 -- 172.21.15.29:0/1484990441 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00a5ba0 con 0x7f8fe806e440 2024-04-25T00:32:59.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 -- 172.21.15.29:0/1484990441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe806e440 msgr2=0x7f8fe80b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 --2- 172.21.15.29:0/1484990441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe806e440 0x7f8fe80b4b10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009dc0 tx=0x7f8fe001c910 comp rx=0 tx=0).stop 2024-04-25T00:32:59.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 -- 172.21.15.29:0/1484990441 shutdown_connections 2024-04-25T00:32:59.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 --2- 172.21.15.29:0/1484990441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe806e440 0x7f8fe80b4b10 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 -- 172.21.15.29:0/1484990441 >> 172.21.15.29:0/1484990441 conn(0x7f8fe80b1870 msgr2=0x7f8fe80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:59.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 -- 172.21.15.29:0/1484990441 shutdown_connections 2024-04-25T00:32:59.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 -- 172.21.15.29:0/1484990441 wait complete. 2024-04-25T00:32:59.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 Processor -- start 2024-04-25T00:32:59.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 -- start start 2024-04-25T00:32:59.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fe806e440 0x7f8fe815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe80b3140 0x7f8fe813af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe80b3c30 0x7f8fe815f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe8169d90 con 0x7f8fe80b3c30 2024-04-25T00:32:59.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe8169c10 con 0x7f8fe806e440 2024-04-25T00:32:59.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe8169f10 con 0x7f8fe80b3140 2024-04-25T00:32:59.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fe7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe80b3c30 0x7f8fe815f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fe7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe80b3c30 0x7f8fe815f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38398/0 (socket says 172.21.15.29:38398) 2024-04-25T00:32:59.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fe7fff640 1 -- 172.21.15.29:0/680088621 learned_addr learned my addr 172.21.15.29:0/680088621 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:32:59.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fe6ffd640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe80b3140 0x7f8fe813af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fe77fe640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fe806e440 0x7f8fe815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fe7fff640 1 -- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe80b3140 msgr2=0x7f8fe813af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fe7fff640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe80b3140 0x7f8fe813af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fe7fff640 1 -- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fe806e440 msgr2=0x7f8fe815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:59.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fe7fff640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fe806e440 0x7f8fe815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:59.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fe7fff640 1 -- 172.21.15.29:0/680088621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe815dfc0 con 0x7f8fe80b3c30 2024-04-25T00:32:59.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fe6ffd640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe80b3140 0x7f8fe813af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:59.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fe7fff640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe80b3c30 0x7f8fe815f060 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8fe001cdf0 tx=0x7f8fe00196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:59.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fcffff640 1 -- 172.21.15.29:0/680088621 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00b2050 con 0x7f8fe80b3c30 2024-04-25T00:32:59.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fcffff640 1 -- 172.21.15.29:0/680088621 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe00be070 con 0x7f8fe80b3c30 2024-04-25T00:32:59.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fcffff640 1 -- 172.21.15.29:0/680088621 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00adab0 con 0x7f8fe80b3c30 2024-04-25T00:32:59.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 -- 172.21.15.29:0/680088621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe815e2a0 con 0x7f8fe80b3c30 2024-04-25T00:32:59.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.834+0000 7f8fee1d9640 1 -- 172.21.15.29:0/680088621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe815e860 con 0x7f8fe80b3c30 2024-04-25T00:32:59.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.838+0000 7f8fee1d9640 1 -- 172.21.15.29:0/680088621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe8069270 con 0x7f8fe80b3c30 2024-04-25T00:32:59.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.838+0000 7f8fcffff640 1 -- 172.21.15.29:0/680088621 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f8fe00a5650 con 0x7f8fe80b3c30 2024-04-25T00:32:59.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.838+0000 7f8fcffff640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fd4067a70 0x7f8fd4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:59.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.838+0000 7f8fcffff640 1 -- 172.21.15.29:0/680088621 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8fe0121ba0 con 0x7f8fe80b3c30 2024-04-25T00:32:59.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.838+0000 7f8fe77fe640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fd4067a70 0x7f8fd4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:59.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.838+0000 7f8fe77fe640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fd4067a70 0x7f8fd4069f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8000f70 tx=0x7f8fd8007b80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:59.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:32:59.842+0000 7f8fcffff640 1 -- 172.21.15.29:0/680088621 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8fe0121e20 con 0x7f8fe80b3c30 2024-04-25T00:33:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fee1d9640 1 -- 172.21.15.29:0/680088621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fe80cf7d0 con 0x7f8fe80b3c30 2024-04-25T00:33:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fcffff640 1 -- 172.21.15.29:0/680088621 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5186 (secure 0 0 0) 0x7f8fe0016500 con 0x7f8fe80b3c30 2024-04-25T00:33:00.035 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:00.035 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2100384916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2100384916},{"type":"v1","addr":"172.21.15.157:6835","nonce":2100384916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4984,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1430513194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1430513194},{"type":"v1","addr":"172.21.15.157:6837","nonce":1430513194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3354831881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3354831881},{"type":"v1","addr":"172.21.15.29:6833","nonce":3354831881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:32:33.777640+0000","modified":"2024-04-25T00:32:36.837733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1468158138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1468158138},{"type":"v1","addr":"172.21.15.29:6835","nonce":1468158138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":2}]} 2024-04-25T00:33:00.035 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 16 2024-04-25T00:33:00.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fee1d9640 1 -- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fd4067a70 msgr2=0x7f8fd4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fee1d9640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fd4067a70 0x7f8fd4069f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8000f70 tx=0x7f8fd8007b80 comp rx=0 tx=0).stop 2024-04-25T00:33:00.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fee1d9640 1 -- 172.21.15.29:0/680088621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe80b3c30 msgr2=0x7f8fe815f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fee1d9640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe80b3c30 0x7f8fe815f060 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8fe001cdf0 tx=0x7f8fe00196b0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fee1d9640 1 -- 172.21.15.29:0/680088621 shutdown_connections 2024-04-25T00:33:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fee1d9640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe80b3c30 0x7f8fe815f060 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fee1d9640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fe80b3140 0x7f8fe813af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fee1d9640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fd4067a70 0x7f8fd4069f30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fee1d9640 1 --2- 172.21.15.29:0/680088621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fe806e440 0x7f8fe815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fee1d9640 1 -- 172.21.15.29:0/680088621 >> 172.21.15.29:0/680088621 conn(0x7f8fe80b1870 msgr2=0x7f8fe8144920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fee1d9640 1 -- 172.21.15.29:0/680088621 shutdown_connections 2024-04-25T00:33:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.030+0000 7f8fee1d9640 1 -- 172.21.15.29:0/680088621 wait complete. 2024-04-25T00:33:00.051 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:33:00.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae9786d640 1 Processor -- start 2024-04-25T00:33:00.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae9786d640 1 -- start start 2024-04-25T00:33:00.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae9786d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9015e060 0x7fae9015c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae9786d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fae9015bce0 0x7fae90071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae9786d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fae900722c0 0x7fae900cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae9786d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fae9015f700 con 0x7fae900722c0 2024-04-25T00:33:00.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae9786d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fae9015e850 con 0x7fae9015bce0 2024-04-25T00:33:00.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae9786d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fae9015eaf0 con 0x7fae9015e060 2024-04-25T00:33:00.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae95e0a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fae900722c0 0x7fae900cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae95e0a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fae900722c0 0x7fae900cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38402/0 (socket says 172.21.15.29:38402) 2024-04-25T00:33:00.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae95e0a640 1 -- 172.21.15.29:0/2058078716 learned_addr learned my addr 172.21.15.29:0/2058078716 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:00.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae95609640 1 --2- 172.21.15.29:0/2058078716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9015e060 0x7fae9015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae94e08640 1 --2- 172.21.15.29:0/2058078716 >> v2:172.21.15.157:3300/0 conn(0x7fae9015bce0 0x7fae90071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae95e0a640 1 -- 172.21.15.29:0/2058078716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9015e060 msgr2=0x7fae9015c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:00.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae95e0a640 1 --2- 172.21.15.29:0/2058078716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9015e060 0x7fae9015c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae95e0a640 1 -- 172.21.15.29:0/2058078716 >> v2:172.21.15.157:3300/0 conn(0x7fae9015bce0 msgr2=0x7fae90071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:00.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae95e0a640 1 --2- 172.21.15.29:0/2058078716 >> v2:172.21.15.157:3300/0 conn(0x7fae9015bce0 0x7fae90071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.166+0000 7fae95e0a640 1 -- 172.21.15.29:0/2058078716 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9015edd0 con 0x7fae900722c0 2024-04-25T00:33:00.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae95e0a640 1 --2- 172.21.15.29:0/2058078716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae900722c0 0x7fae900cf100 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fae90072e40 tx=0x7fae8c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1b7ec145d21628 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:00.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae87fff640 1 -- 172.21.15.29:0/2058078716 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0090d0 con 0x7fae900722c0 2024-04-25T00:33:00.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae87fff640 1 -- 172.21.15.29:0/2058078716 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae8c0a5760 con 0x7fae900722c0 2024-04-25T00:33:00.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae87fff640 1 -- 172.21.15.29:0/2058078716 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c009270 con 0x7fae900722c0 2024-04-25T00:33:00.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 -- 172.21.15.29:0/2058078716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae900722c0 msgr2=0x7fae900cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 --2- 172.21.15.29:0/2058078716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae900722c0 0x7fae900cf100 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fae90072e40 tx=0x7fae8c01c9f0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 -- 172.21.15.29:0/2058078716 shutdown_connections 2024-04-25T00:33:00.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 --2- 172.21.15.29:0/2058078716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae900722c0 0x7fae900cf100 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 --2- 172.21.15.29:0/2058078716 >> v2:172.21.15.157:3300/0 conn(0x7fae9015bce0 0x7fae90071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 --2- 172.21.15.29:0/2058078716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9015e060 0x7fae9015c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 -- 172.21.15.29:0/2058078716 >> 172.21.15.29:0/2058078716 conn(0x7fae9006c320 msgr2=0x7fae9006c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:00.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 -- 172.21.15.29:0/2058078716 shutdown_connections 2024-04-25T00:33:00.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 -- 172.21.15.29:0/2058078716 wait complete. 2024-04-25T00:33:00.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 Processor -- start 2024-04-25T00:33:00.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 -- start start 2024-04-25T00:33:00.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae900722c0 0x7fae900c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9015bce0 0x7fae9015a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae9015e060 0x7fae9014e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fae9015e5c0 con 0x7fae9015e060 2024-04-25T00:33:00.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fae9015e440 con 0x7fae900722c0 2024-04-25T00:33:00.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae9786d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fae9006f610 con 0x7fae9015bce0 2024-04-25T00:33:00.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae95e0a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae9015e060 0x7fae9014e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae95e0a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae9015e060 0x7fae9014e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38406/0 (socket says 172.21.15.29:38406) 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae95e0a640 1 -- 172.21.15.29:0/4234495130 learned_addr learned my addr 172.21.15.29:0/4234495130 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae94e08640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9015bce0 0x7fae9015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.170+0000 7fae95609640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae900722c0 0x7fae900c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae95e0a640 1 -- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9015bce0 msgr2=0x7fae9015a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae95e0a640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9015bce0 0x7fae9015a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae95e0a640 1 -- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae900722c0 msgr2=0x7fae900c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae95e0a640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae900722c0 0x7fae900c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae95e0a640 1 -- 172.21.15.29:0/4234495130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae900cc7f0 con 0x7fae9015e060 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae95e0a640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae9015e060 0x7fae9014e5b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fae8c01b0a0 tx=0x7fae8c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae85ffb640 1 -- 172.21.15.29:0/4234495130 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0b8670 con 0x7fae9015e060 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae85ffb640 1 -- 172.21.15.29:0/4234495130 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae8c0be070 con 0x7fae9015e060 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae85ffb640 1 -- 172.21.15.29:0/4234495130 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0b8a30 con 0x7fae9015e060 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae9786d640 1 -- 172.21.15.29:0/4234495130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae900ccad0 con 0x7fae9015e060 2024-04-25T00:33:00.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae9786d640 1 -- 172.21.15.29:0/4234495130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae900b4e10 con 0x7fae9015e060 2024-04-25T00:33:00.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae9786d640 1 -- 172.21.15.29:0/4234495130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae900ccca0 con 0x7fae9015e060 2024-04-25T00:33:00.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae85ffb640 1 -- 172.21.15.29:0/4234495130 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fae8c0b8bd0 con 0x7fae9015e060 2024-04-25T00:33:00.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae85ffb640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fae74067a70 0x7fae74069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae85ffb640 1 -- 172.21.15.29:0/4234495130 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fae8c121090 con 0x7fae9015e060 2024-04-25T00:33:00.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae95609640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fae74067a70 0x7fae74069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.174+0000 7fae95609640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fae74067a70 0x7fae74069f30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fae8800d8d0 tx=0x7fae88008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:00.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.178+0000 7fae85ffb640 1 -- 172.21.15.29:0/4234495130 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fae8c0165e0 con 0x7fae9015e060 2024-04-25T00:33:00.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.342+0000 7fae9786d640 1 -- 172.21.15.29:0/4234495130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae90069270 con 0x7fae9015e060 2024-04-25T00:33:00.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.346+0000 7fae85ffb640 1 -- 172.21.15.29:0/4234495130 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16621 (secure 0 0 0) 0x7fae8c0f1950 con 0x7fae9015e060 2024-04-25T00:33:00.351 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:00.351 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":33,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:32:35.426396+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:32:32.254456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:32:32.779372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:33:00.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.346+0000 7fae9786d640 1 -- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fae74067a70 msgr2=0x7fae74069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.346+0000 7fae9786d640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fae74067a70 0x7fae74069f30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fae8800d8d0 tx=0x7fae88008040 comp rx=0 tx=0).stop 2024-04-25T00:33:00.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.346+0000 7fae9786d640 1 -- 172.21.15.29:0/4234495130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae9015e060 msgr2=0x7fae9014e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.346+0000 7fae9786d640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae9015e060 0x7fae9014e5b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fae8c01b0a0 tx=0x7fae8c019790 comp rx=0 tx=0).stop 2024-04-25T00:33:00.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.346+0000 7fae9786d640 1 -- 172.21.15.29:0/4234495130 shutdown_connections 2024-04-25T00:33:00.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.346+0000 7fae9786d640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae9015e060 0x7fae9014e5b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.346+0000 7fae9786d640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae9015bce0 0x7fae9015a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.346+0000 7fae9786d640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fae74067a70 0x7fae74069f30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.346+0000 7fae9786d640 1 --2- 172.21.15.29:0/4234495130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae900722c0 0x7fae900c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.346+0000 7fae9786d640 1 -- 172.21.15.29:0/4234495130 >> 172.21.15.29:0/4234495130 conn(0x7fae9006c320 msgr2=0x7fae9006ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:00.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.346+0000 7fae9786d640 1 -- 172.21.15.29:0/4234495130 shutdown_connections 2024-04-25T00:33:00.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.346+0000 7fae9786d640 1 -- 172.21.15.29:0/4234495130 wait complete. 2024-04-25T00:33:00.369 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:33:00.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f3480838640 1 Processor -- start 2024-04-25T00:33:00.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f3480838640 1 -- start start 2024-04-25T00:33:00.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f3480838640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f347c06b920 0x7f347c0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f3480838640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347c160190 con 0x7f347c156420 2024-04-25T00:33:00.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f3480838640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347c15c680 con 0x7f347c15ce60 2024-04-25T00:33:00.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f3480838640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f347c15d2f0 con 0x7f347c06b920 2024-04-25T00:33:00.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f347a59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f347c156420 0x7f347c15fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46522/0 (socket says 172.21.15.29:46522) 2024-04-25T00:33:00.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f347a59c640 1 -- 172.21.15.29:0/508262454 learned_addr learned my addr 172.21.15.29:0/508262454 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:00.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f347ad9d640 1 --2- 172.21.15.29:0/508262454 >> v2:172.21.15.157:3300/0 conn(0x7f347c06b920 0x7f347c0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1881021525 0 0) 0x7f347c160190 con 0x7f347c156420 2024-04-25T00:33:00.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3464003530 con 0x7f347c156420 2024-04-25T00:33:00.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2871276746 0 0) 0x7f3464003530 con 0x7f347c156420 2024-04-25T00:33:00.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 >> v2:172.21.15.157:3300/0 conn(0x7f347c06b920 msgr2=0x7f347c0cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:00.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f347959a640 1 --2- 172.21.15.29:0/508262454 >> v2:172.21.15.157:3300/0 conn(0x7f347c06b920 0x7f347c0cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 >> v1:172.21.15.157:6789/0 conn(0x7f347c15ce60 legacy=0x7f347c06b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.482+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c160440 con 0x7f347c156420 2024-04-25T00:33:00.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f346c001a00 con 0x7f347c156420 2024-04-25T00:33:00.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 >> v1:172.21.15.29:6789/0 conn(0x7f347c156420 legacy=0x7f347c15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347959a640 1 --2- 172.21.15.29:0/508262454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34640039c0 0x7f3464003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347959a640 1 --2- 172.21.15.29:0/508262454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3464004810 0x7f3464004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347959a640 1 --2- 172.21.15.29:0/508262454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34640051d0 0x7f346400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f347c15d2f0 con 0x7f34640051d0 2024-04-25T00:33:00.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f346400c070 con 0x7f3464004810 2024-04-25T00:33:00.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f346400c270 con 0x7f34640039c0 2024-04-25T00:33:00.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347ad9d640 1 --2- 172.21.15.29:0/508262454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34640051d0 0x7f346400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347a59c640 1 --2- 172.21.15.29:0/508262454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34640039c0 0x7f3464003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f3479d9b640 1 --2- 172.21.15.29:0/508262454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3464004810 0x7f3464004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347ad9d640 1 -- 172.21.15.29:0/508262454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34640039c0 msgr2=0x7f3464003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:00.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347ad9d640 1 --2- 172.21.15.29:0/508262454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34640039c0 0x7f3464003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347ad9d640 1 -- 172.21.15.29:0/508262454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3464004810 msgr2=0x7f3464004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:00.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347ad9d640 1 --2- 172.21.15.29:0/508262454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3464004810 0x7f3464004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347ad9d640 1 -- 172.21.15.29:0/508262454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3464003530 con 0x7f34640051d0 2024-04-25T00:33:00.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347ad9d640 1 -- 172.21.15.29:0/508262454 reap_dead start 2024-04-25T00:33:00.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347ad9d640 1 --2- 172.21.15.29:0/508262454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34640051d0 0x7f346400b9d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3474009ae0 tx=0x7f347401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54fdf52c861d4c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:00.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740096b0 con 0x7f34640051d0 2024-04-25T00:33:00.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3474004020 con 0x7f34640051d0 2024-04-25T00:33:00.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f347959a640 1 -- 172.21.15.29:0/508262454 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474004410 con 0x7f34640051d0 2024-04-25T00:33:00.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f3480838640 1 -- 172.21.15.29:0/508262454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34640051d0 msgr2=0x7f346400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f3480838640 1 --2- 172.21.15.29:0/508262454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34640051d0 0x7f346400b9d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3474009ae0 tx=0x7f347401cab0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f3480838640 1 -- 172.21.15.29:0/508262454 shutdown_connections 2024-04-25T00:33:00.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f3480838640 1 --2- 172.21.15.29:0/508262454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34640051d0 0x7f346400b9d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f3480838640 1 -- 172.21.15.29:0/508262454 >> 172.21.15.29:0/508262454 conn(0x7f347c0d0c40 msgr2=0x7f347c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:00.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f3480838640 1 -- 172.21.15.29:0/508262454 shutdown_connections 2024-04-25T00:33:00.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.486+0000 7f3480838640 1 -- 172.21.15.29:0/508262454 wait complete. 2024-04-25T00:33:00.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f3480838640 1 Processor -- start 2024-04-25T00:33:00.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f3480838640 1 -- start start 2024-04-25T00:33:00.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f3480838640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34640051d0 0x7f347c0cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f3480838640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f347c0b0f60 0x7f347c0b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f3480838640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f347c0c8470 0x7f347c0c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f3480838640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f347c156800 con 0x7f34640051d0 2024-04-25T00:33:00.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f3480838640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f347c15c940 con 0x7f347c0b0f60 2024-04-25T00:33:00.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f3480838640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f347c156980 con 0x7f347c0c8470 2024-04-25T00:33:00.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f347a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34640051d0 0x7f347c0cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f347a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34640051d0 0x7f347c0cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38434/0 (socket says 172.21.15.29:38434) 2024-04-25T00:33:00.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f347a59c640 1 -- 172.21.15.29:0/3727674589 learned_addr learned my addr 172.21.15.29:0/3727674589 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:00.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f3479d9b640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f347c0b0f60 0x7f347c0b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f347ad9d640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f347c0c8470 0x7f347c0c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f347a59c640 1 -- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f347c0c8470 msgr2=0x7f347c0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f347a59c640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f347c0c8470 0x7f347c0c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f347a59c640 1 -- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f347c0b0f60 msgr2=0x7f347c0b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f347a59c640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f347c0b0f60 0x7f347c0b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f347a59c640 1 -- 172.21.15.29:0/3727674589 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c0ccb00 con 0x7f34640051d0 2024-04-25T00:33:00.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f347ad9d640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f347c0c8470 0x7f347c0c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:00.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f347a59c640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34640051d0 0x7f347c0cc5c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f346c007aa0 tx=0x7f346c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:00.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f346affd640 1 -- 172.21.15.29:0/3727674589 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c00ee50 con 0x7f34640051d0 2024-04-25T00:33:00.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f346affd640 1 -- 172.21.15.29:0/3727674589 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346c018070 con 0x7f34640051d0 2024-04-25T00:33:00.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f3480838640 1 -- 172.21.15.29:0/3727674589 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347c0c9080 con 0x7f34640051d0 2024-04-25T00:33:00.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f346affd640 1 -- 172.21.15.29:0/3727674589 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c013660 con 0x7f34640051d0 2024-04-25T00:33:00.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f3480838640 1 -- 172.21.15.29:0/3727674589 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347c0c9640 con 0x7f34640051d0 2024-04-25T00:33:00.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.490+0000 7f3480838640 1 -- 172.21.15.29:0/3727674589 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347c0c4a40 con 0x7f34640051d0 2024-04-25T00:33:00.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.494+0000 7f346affd640 1 -- 172.21.15.29:0/3727674589 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f346c013800 con 0x7f34640051d0 2024-04-25T00:33:00.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.494+0000 7f346affd640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f345c067aa0 0x7f345c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.494+0000 7f346affd640 1 -- 172.21.15.29:0/3727674589 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f346c086540 con 0x7f34640051d0 2024-04-25T00:33:00.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.494+0000 7f3479d9b640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f345c067aa0 0x7f345c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.494+0000 7f3479d9b640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f345c067aa0 0x7f345c069f60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f347c0c44b0 tx=0x7f3470009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:00.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.498+0000 7f346affd640 1 -- 172.21.15.29:0/3727674589 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f346c056e90 con 0x7f34640051d0 2024-04-25T00:33:00.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.694+0000 7f3480838640 1 -- 172.21.15.29:0/3727674589 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f347c069270 con 0x7f34640051d0 2024-04-25T00:33:00.700 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-04-25T00:33:00.694+0000 7f3ac00b5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:33:00.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.698+0000 7f346affd640 1 -- 172.21.15.29:0/3727674589 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f346c056790 con 0x7f34640051d0 2024-04-25T00:33:00.703 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:33:00.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.698+0000 7f3480838640 1 -- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f345c067aa0 msgr2=0x7f345c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.698+0000 7f3480838640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f345c067aa0 0x7f345c069f60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f347c0c44b0 tx=0x7f3470009040 comp rx=0 tx=0).stop 2024-04-25T00:33:00.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.698+0000 7f3480838640 1 -- 172.21.15.29:0/3727674589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34640051d0 msgr2=0x7f347c0cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.698+0000 7f3480838640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34640051d0 0x7f347c0cc5c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f346c007aa0 tx=0x7f346c00e530 comp rx=0 tx=0).stop 2024-04-25T00:33:00.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.702+0000 7f3480838640 1 -- 172.21.15.29:0/3727674589 shutdown_connections 2024-04-25T00:33:00.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.702+0000 7f3480838640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f347c0c8470 0x7f347c0c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.702+0000 7f3480838640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f345c067aa0 0x7f345c069f60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.702+0000 7f3480838640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f347c0b0f60 0x7f347c0b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.702+0000 7f3480838640 1 --2- 172.21.15.29:0/3727674589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34640051d0 0x7f347c0cc5c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.702+0000 7f3480838640 1 -- 172.21.15.29:0/3727674589 >> 172.21.15.29:0/3727674589 conn(0x7f347c0d0c40 msgr2=0x7f347c155960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:00.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.702+0000 7f3480838640 1 -- 172.21.15.29:0/3727674589 shutdown_connections 2024-04-25T00:33:00.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.702+0000 7f3480838640 1 -- 172.21.15.29:0/3727674589 wait complete. 2024-04-25T00:33:00.724 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:33:00.744 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:33:00.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedec6b640 1 Processor -- start 2024-04-25T00:33:00.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedec6b640 1 -- start start 2024-04-25T00:33:00.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedec6b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fced815e8d0 0x7fced80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedec6b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fced80b4780 0x7fced815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedec6b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fced815f1f0 0x7fced815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedec6b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fced80b42f0 con 0x7fced815f1f0 2024-04-25T00:33:00.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedec6b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fced80b3720 con 0x7fced80b4780 2024-04-25T00:33:00.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedec6b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fced80b39c0 con 0x7fced815e8d0 2024-04-25T00:33:00.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedd208640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fced815f1f0 0x7fced815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedd208640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fced815f1f0 0x7fced815b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38464/0 (socket says 172.21.15.29:38464) 2024-04-25T00:33:00.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedd208640 1 -- 172.21.15.29:0/347780204 learned_addr learned my addr 172.21.15.29:0/347780204 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:00.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedca07640 1 --2- 172.21.15.29:0/347780204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fced815e8d0 0x7fced80cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fced7fff640 1 --2- 172.21.15.29:0/347780204 >> v2:172.21.15.157:3300/0 conn(0x7fced80b4780 0x7fced815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedd208640 1 -- 172.21.15.29:0/347780204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fced815e8d0 msgr2=0x7fced80cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:00.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedd208640 1 --2- 172.21.15.29:0/347780204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fced815e8d0 0x7fced80cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedd208640 1 -- 172.21.15.29:0/347780204 >> v2:172.21.15.157:3300/0 conn(0x7fced80b4780 msgr2=0x7fced815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:00.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedd208640 1 --2- 172.21.15.29:0/347780204 >> v2:172.21.15.157:3300/0 conn(0x7fced80b4780 0x7fced815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fcedd208640 1 -- 172.21.15.29:0/347780204 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced80b3ca0 con 0x7fced815f1f0 2024-04-25T00:33:00.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.834+0000 7fced7fff640 1 --2- 172.21.15.29:0/347780204 >> v2:172.21.15.157:3300/0 conn(0x7fced80b4780 0x7fced815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:00.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fcedd208640 1 --2- 172.21.15.29:0/347780204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fced815f1f0 0x7fced815b2a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fced8074fb0 tx=0x7fced001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df77b388d988c57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:00.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fced77fe640 1 -- 172.21.15.29:0/347780204 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced00a5820 con 0x7fced815f1f0 2024-04-25T00:33:00.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fced77fe640 1 -- 172.21.15.29:0/347780204 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced00a59c0 con 0x7fced815f1f0 2024-04-25T00:33:00.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fced77fe640 1 -- 172.21.15.29:0/347780204 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced00a5d90 con 0x7fced815f1f0 2024-04-25T00:33:00.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fcedec6b640 1 -- 172.21.15.29:0/347780204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fced815f1f0 msgr2=0x7fced815b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fcedec6b640 1 --2- 172.21.15.29:0/347780204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fced815f1f0 0x7fced815b2a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fced8074fb0 tx=0x7fced001c9c0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fcedec6b640 1 -- 172.21.15.29:0/347780204 shutdown_connections 2024-04-25T00:33:00.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fcedec6b640 1 --2- 172.21.15.29:0/347780204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fced815f1f0 0x7fced815b2a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fcedec6b640 1 --2- 172.21.15.29:0/347780204 >> v2:172.21.15.157:3300/0 conn(0x7fced80b4780 0x7fced815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fcedec6b640 1 --2- 172.21.15.29:0/347780204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fced815e8d0 0x7fced80cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fcedec6b640 1 -- 172.21.15.29:0/347780204 >> 172.21.15.29:0/347780204 conn(0x7fced80b1870 msgr2=0x7fced80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:00.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fcedec6b640 1 -- 172.21.15.29:0/347780204 shutdown_connections 2024-04-25T00:33:00.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fcedec6b640 1 -- 172.21.15.29:0/347780204 wait complete. 2024-04-25T00:33:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fcedec6b640 1 Processor -- start 2024-04-25T00:33:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.838+0000 7fcedec6b640 1 -- start start 2024-04-25T00:33:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fcedec6b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fced80b4780 0x7fced8143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fcedec6b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fced815e8d0 0x7fced8154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fcedec6b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fced815f1f0 0x7fced815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fcedec6b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fced8144a20 con 0x7fced815e8d0 2024-04-25T00:33:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fcedec6b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fced806d590 con 0x7fced80b4780 2024-04-25T00:33:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fcedec6b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fced8159b30 con 0x7fced815f1f0 2024-04-25T00:33:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fced815e8d0 0x7fced8154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fced815e8d0 0x7fced8154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38478/0 (socket says 172.21.15.29:38478) 2024-04-25T00:33:00.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced7fff640 1 -- 172.21.15.29:0/683861398 learned_addr learned my addr 172.21.15.29:0/683861398 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fcedd208640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fced815f1f0 0x7fced815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fcedca07640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fced80b4780 0x7fced8143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced7fff640 1 -- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fced815f1f0 msgr2=0x7fced815cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced7fff640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fced815f1f0 0x7fced815cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced7fff640 1 -- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fced80b4780 msgr2=0x7fced8143ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced7fff640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fced80b4780 0x7fced8143ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced7fff640 1 -- 172.21.15.29:0/683861398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced8154ed0 con 0x7fced815e8d0 2024-04-25T00:33:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced7fff640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fced815e8d0 0x7fced8154990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00c7c0 tx=0x7fcecc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced57fa640 1 -- 172.21.15.29:0/683861398 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc015070 con 0x7fced815e8d0 2024-04-25T00:33:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced57fa640 1 -- 172.21.15.29:0/683861398 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcecc007a30 con 0x7fced815e8d0 2024-04-25T00:33:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fcedec6b640 1 -- 172.21.15.29:0/683861398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced813a700 con 0x7fced815e8d0 2024-04-25T00:33:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced57fa640 1 -- 172.21.15.29:0/683861398 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc007e00 con 0x7fced815e8d0 2024-04-25T00:33:00.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fcedec6b640 1 -- 172.21.15.29:0/683861398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced81597c0 con 0x7fced815e8d0 2024-04-25T00:33:00.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fcedec6b640 1 -- 172.21.15.29:0/683861398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced8159300 con 0x7fced815e8d0 2024-04-25T00:33:00.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced57fa640 1 -- 172.21.15.29:0/683861398 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fcecc018070 con 0x7fced815e8d0 2024-04-25T00:33:00.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced57fa640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcec4067a70 0x7fcec4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:00.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fced57fa640 1 -- 172.21.15.29:0/683861398 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fcecc085f80 con 0x7fced815e8d0 2024-04-25T00:33:00.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.842+0000 7fcedca07640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcec4067a70 0x7fcec4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:00.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.846+0000 7fcedca07640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcec4067a70 0x7fcec4069f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fced8072070 tx=0x7fcec8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:00.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:00.846+0000 7fced57fa640 1 -- 172.21.15.29:0/683861398 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fced8159300 con 0x7fced815e8d0 2024-04-25T00:33:01.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.042+0000 7fcedec6b640 1 -- 172.21.15.29:0/683861398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fced8069270 con 0x7fced815e8d0 2024-04-25T00:33:01.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.706+0000 7fced57fa640 1 -- 172.21.15.29:0/683861398 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7fcecc056760 con 0x7fced815e8d0 2024-04-25T00:33:01.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.710+0000 7fcedec6b640 1 -- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcec4067a70 msgr2=0x7fcec4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.710+0000 7fcedec6b640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcec4067a70 0x7fcec4069f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fced8072070 tx=0x7fcec8008040 comp rx=0 tx=0).stop 2024-04-25T00:33:01.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.710+0000 7fcedec6b640 1 -- 172.21.15.29:0/683861398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fced815e8d0 msgr2=0x7fced8154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.710+0000 7fcedec6b640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fced815e8d0 0x7fced8154990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00c7c0 tx=0x7fcecc00cc80 comp rx=0 tx=0).stop 2024-04-25T00:33:01.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.710+0000 7fcedec6b640 1 -- 172.21.15.29:0/683861398 shutdown_connections 2024-04-25T00:33:01.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.710+0000 7fcedec6b640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fced815f1f0 0x7fced815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.710+0000 7fcedec6b640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fced815e8d0 0x7fced8154990 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.710+0000 7fcedec6b640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcec4067a70 0x7fcec4069f30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.710+0000 7fcedec6b640 1 --2- 172.21.15.29:0/683861398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fced80b4780 0x7fced8143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.710+0000 7fcedec6b640 1 -- 172.21.15.29:0/683861398 >> 172.21.15.29:0/683861398 conn(0x7fced80b1870 msgr2=0x7fced80b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:01.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.710+0000 7fcedec6b640 1 -- 172.21.15.29:0/683861398 shutdown_connections 2024-04-25T00:33:01.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.710+0000 7fcedec6b640 1 -- 172.21.15.29:0/683861398 wait complete. 2024-04-25T00:33:01.739 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:33:01.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff2171a3640 1 Processor -- start 2024-04-25T00:33:01.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff2171a3640 1 -- start start 2024-04-25T00:33:01.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff2171a3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff2101456d0 0x7ff2100b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff2171a3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2100b3b50 0x7ff2100cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff2171a3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff21006cd40 0x7ff21006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff2171a3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff2100b4380 con 0x7ff2101456d0 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff2171a3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff2100b40a0 con 0x7ff21006cd40 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff2171a3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff2100ce480 con 0x7ff2100b3b50 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff214f3f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff2101456d0 0x7ff2100b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff214f3f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff2101456d0 0x7ff2100b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38490/0 (socket says 172.21.15.29:38490) 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff214f3f640 1 -- 172.21.15.29:0/872053549 learned_addr learned my addr 172.21.15.29:0/872053549 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff215740640 1 --2- 172.21.15.29:0/872053549 >> v2:172.21.15.157:3300/0 conn(0x7ff21006cd40 0x7ff21006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff20ffff640 1 --2- 172.21.15.29:0/872053549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2100b3b50 0x7ff2100cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff214f3f640 1 -- 172.21.15.29:0/872053549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2100b3b50 msgr2=0x7ff2100cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff214f3f640 1 --2- 172.21.15.29:0/872053549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2100b3b50 0x7ff2100cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff214f3f640 1 -- 172.21.15.29:0/872053549 >> v2:172.21.15.157:3300/0 conn(0x7ff21006cd40 msgr2=0x7ff21006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff214f3f640 1 --2- 172.21.15.29:0/872053549 >> v2:172.21.15.157:3300/0 conn(0x7ff21006cd40 0x7ff21006d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff214f3f640 1 -- 172.21.15.29:0/872053549 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2100ce760 con 0x7ff2101456d0 2024-04-25T00:33:01.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.922+0000 7ff215740640 1 --2- 172.21.15.29:0/872053549 >> v2:172.21.15.157:3300/0 conn(0x7ff21006cd40 0x7ff21006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:01.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff214f3f640 1 --2- 172.21.15.29:0/872053549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2101456d0 0x7ff2100b2b70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff2100cfb50 tx=0x7ff20001cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1b941e26c4dc3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:01.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff20f7fe640 1 -- 172.21.15.29:0/872053549 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2000ab5f0 con 0x7ff2101456d0 2024-04-25T00:33:01.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff20f7fe640 1 -- 172.21.15.29:0/872053549 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2000b0070 con 0x7ff2101456d0 2024-04-25T00:33:01.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff20f7fe640 1 -- 172.21.15.29:0/872053549 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2000ab9b0 con 0x7ff2101456d0 2024-04-25T00:33:01.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff2171a3640 1 -- 172.21.15.29:0/872053549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2101456d0 msgr2=0x7ff2100b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff2171a3640 1 --2- 172.21.15.29:0/872053549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2101456d0 0x7ff2100b2b70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff2100cfb50 tx=0x7ff20001cf80 comp rx=0 tx=0).stop 2024-04-25T00:33:01.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff2171a3640 1 -- 172.21.15.29:0/872053549 shutdown_connections 2024-04-25T00:33:01.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff2171a3640 1 --2- 172.21.15.29:0/872053549 >> v2:172.21.15.157:3300/0 conn(0x7ff21006cd40 0x7ff21006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff2171a3640 1 --2- 172.21.15.29:0/872053549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2100b3b50 0x7ff2100cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff2171a3640 1 --2- 172.21.15.29:0/872053549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2101456d0 0x7ff2100b2b70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff2171a3640 1 -- 172.21.15.29:0/872053549 >> 172.21.15.29:0/872053549 conn(0x7ff2100b1870 msgr2=0x7ff2100b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:01.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff2171a3640 1 -- 172.21.15.29:0/872053549 shutdown_connections 2024-04-25T00:33:01.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff2171a3640 1 -- 172.21.15.29:0/872053549 wait complete. 2024-04-25T00:33:01.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff2171a3640 1 Processor -- start 2024-04-25T00:33:01.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.926+0000 7ff2171a3640 1 -- start start 2024-04-25T00:33:01.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff2171a3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff21006cd40 0x7ff210160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff2171a3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2100b3b50 0x7ff210164f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff2171a3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2101456d0 0x7ff210165460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff2171a3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff2100cf4a0 con 0x7ff2101456d0 2024-04-25T00:33:01.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff2171a3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff2100b5000 con 0x7ff21006cd40 2024-04-25T00:33:01.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff2171a3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff2100cf620 con 0x7ff2100b3b50 2024-04-25T00:33:01.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff215740640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2101456d0 0x7ff210165460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff215740640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2101456d0 0x7ff210165460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38498/0 (socket says 172.21.15.29:38498) 2024-04-25T00:33:01.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff215740640 1 -- 172.21.15.29:0/2855799490 learned_addr learned my addr 172.21.15.29:0/2855799490 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:01.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff20ffff640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2100b3b50 0x7ff210164f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff214f3f640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff21006cd40 0x7ff210160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff215740640 1 -- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2100b3b50 msgr2=0x7ff210164f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff215740640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2100b3b50 0x7ff210164f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff215740640 1 -- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff21006cd40 msgr2=0x7ff210160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff215740640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff21006cd40 0x7ff210160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff215740640 1 -- 172.21.15.29:0/2855799490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21006e610 con 0x7ff2101456d0 2024-04-25T00:33:01.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff20ffff640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2100b3b50 0x7ff210164f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:01.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff215740640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2101456d0 0x7ff210165460 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff20800cd50 tx=0x7ff20800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:01.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff20d7fa640 1 -- 172.21.15.29:0/2855799490 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20800f680 con 0x7ff2101456d0 2024-04-25T00:33:01.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff20d7fa640 1 -- 172.21.15.29:0/2855799490 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff20800f840 con 0x7ff2101456d0 2024-04-25T00:33:01.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff2171a3640 1 -- 172.21.15.29:0/2855799490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21015da20 con 0x7ff2101456d0 2024-04-25T00:33:01.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff20d7fa640 1 -- 172.21.15.29:0/2855799490 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20800fc30 con 0x7ff2101456d0 2024-04-25T00:33:01.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff2171a3640 1 -- 172.21.15.29:0/2855799490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff210071160 con 0x7ff2101456d0 2024-04-25T00:33:01.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff2171a3640 1 -- 172.21.15.29:0/2855799490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2100692e0 con 0x7ff2101456d0 2024-04-25T00:33:01.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff20d7fa640 1 -- 172.21.15.29:0/2855799490 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7ff2080213f0 con 0x7ff2101456d0 2024-04-25T00:33:01.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff20d7fa640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1f0067aa0 0x7ff1f0069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff20d7fa640 1 -- 172.21.15.29:0/2855799490 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7ff208086e70 con 0x7ff2101456d0 2024-04-25T00:33:01.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.930+0000 7ff214f3f640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1f0067aa0 0x7ff1f0069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.934+0000 7ff214f3f640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1f0067aa0 0x7ff1f0069f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff2000af000 tx=0x7ff200008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:01.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:01.934+0000 7ff20d7fa640 1 -- 172.21.15.29:0/2855799490 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff208014030 con 0x7ff2101456d0 2024-04-25T00:33:02.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.098+0000 7ff2171a3640 1 -- 172.21.15.29:0/2855799490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff2100cf280 con 0x7ff2101456d0 2024-04-25T00:33:02.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.706+0000 7ff20d7fa640 1 -- 172.21.15.29:0/2855799490 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) ==== 181+0+0 (secure 0 0 0) 0x7ff2080575c0 con 0x7ff2101456d0 2024-04-25T00:33:02.713 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:33:02.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.710+0000 7ff2171a3640 1 -- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1f0067aa0 msgr2=0x7ff1f0069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.710+0000 7ff2171a3640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1f0067aa0 0x7ff1f0069f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff2000af000 tx=0x7ff200008040 comp rx=0 tx=0).stop 2024-04-25T00:33:02.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.710+0000 7ff2171a3640 1 -- 172.21.15.29:0/2855799490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2101456d0 msgr2=0x7ff210165460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.710+0000 7ff2171a3640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2101456d0 0x7ff210165460 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff20800cd50 tx=0x7ff20800a650 comp rx=0 tx=0).stop 2024-04-25T00:33:02.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.710+0000 7ff2171a3640 1 -- 172.21.15.29:0/2855799490 shutdown_connections 2024-04-25T00:33:02.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.710+0000 7ff2171a3640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2101456d0 0x7ff210165460 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.710+0000 7ff2171a3640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2100b3b50 0x7ff210164f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.710+0000 7ff2171a3640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff1f0067aa0 0x7ff1f0069f60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.710+0000 7ff2171a3640 1 --2- 172.21.15.29:0/2855799490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff21006cd40 0x7ff210160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.710+0000 7ff2171a3640 1 -- 172.21.15.29:0/2855799490 >> 172.21.15.29:0/2855799490 conn(0x7ff2100b1870 msgr2=0x7ff210141800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:02.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.710+0000 7ff2171a3640 1 -- 172.21.15.29:0/2855799490 shutdown_connections 2024-04-25T00:33:02.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.710+0000 7ff2171a3640 1 -- 172.21.15.29:0/2855799490 wait complete. 2024-04-25T00:33:02.736 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:33:02.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.862+0000 7f225dfa6640 1 Processor -- start 2024-04-25T00:33:02.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.862+0000 7f225dfa6640 1 -- start start 2024-04-25T00:33:02.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.866+0000 7f225dfa6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f225806c530 0x7f225806c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.866+0000 7f225dfa6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f225806d5f0 con 0x7f22580d0090 2024-04-25T00:33:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.866+0000 7f225dfa6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f225806d030 con 0x7f2258161730 2024-04-25T00:33:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.866+0000 7f225dfa6640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f22580b2100 con 0x7f225806c530 2024-04-25T00:33:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.866+0000 7f2256ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f225806c530 0x7f225806c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.866+0000 7f2257fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f22580d0090 0x7f22580d1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46544/0 (socket says 172.21.15.29:46544) 2024-04-25T00:33:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.866+0000 7f2257fff640 1 -- 172.21.15.29:0/3481610366 learned_addr learned my addr 172.21.15.29:0/3481610366 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.866+0000 7f2256ffd640 1 -- 172.21.15.29:0/3481610366 >> v1:172.21.15.157:6789/0 conn(0x7f2258161730 legacy=0x7f22580b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.866+0000 7f2256ffd640 1 -- 172.21.15.29:0/3481610366 >> v1:172.21.15.29:6789/0 conn(0x7f22580d0090 legacy=0x7f22580d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.866+0000 7f2256ffd640 1 -- 172.21.15.29:0/3481610366 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225806d310 con 0x7f225806c530 2024-04-25T00:33:02.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.866+0000 7f22567fc640 1 -- 172.21.15.29:0/3481610366 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 794743251 0 0) 0x7f2250001120 con 0x7f22580d0090 2024-04-25T00:33:02.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.866+0000 7f2256ffd640 1 --2- 172.21.15.29:0/3481610366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f225806c530 0x7f225806c9e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f224c0097c0 tx=0x7f224c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18353dfc3e1c7af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:02.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.866+0000 7f22567fc640 1 -- 172.21.15.29:0/3481610366 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c0ab600 con 0x7f225806c530 2024-04-25T00:33:02.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f22567fc640 1 -- 172.21.15.29:0/3481610366 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f224c0b0070 con 0x7f225806c530 2024-04-25T00:33:02.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f22567fc640 1 -- 172.21.15.29:0/3481610366 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c0ab9c0 con 0x7f225806c530 2024-04-25T00:33:02.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 -- 172.21.15.29:0/3481610366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f225806c530 msgr2=0x7f225806c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 --2- 172.21.15.29:0/3481610366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f225806c530 0x7f225806c9e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f224c0097c0 tx=0x7f224c01c3e0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 -- 172.21.15.29:0/3481610366 shutdown_connections 2024-04-25T00:33:02.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 --2- 172.21.15.29:0/3481610366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f225806c530 0x7f225806c9e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 -- 172.21.15.29:0/3481610366 >> 172.21.15.29:0/3481610366 conn(0x7f225809aa30 msgr2=0x7f225809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:02.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 -- 172.21.15.29:0/3481610366 shutdown_connections 2024-04-25T00:33:02.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 -- 172.21.15.29:0/3481610366 wait complete. 2024-04-25T00:33:02.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 Processor -- start 2024-04-25T00:33:02.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 -- start start 2024-04-25T00:33:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f225806c530 0x7f22581540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22580d0090 0x7f22580d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2258161730 0x7f22580b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f22580b2e20 con 0x7f225806c530 2024-04-25T00:33:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f22580b2ca0 con 0x7f22580d0090 2024-04-25T00:33:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f22580d2510 con 0x7f2258161730 2024-04-25T00:33:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f22577fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f225806c530 0x7f22581540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f22577fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f225806c530 0x7f22581540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38518/0 (socket says 172.21.15.29:38518) 2024-04-25T00:33:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f22577fe640 1 -- 172.21.15.29:0/674642692 learned_addr learned my addr 172.21.15.29:0/674642692 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f2257fff640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2258161730 0x7f22580b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f2256ffd640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22580d0090 0x7f22580d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f22577fe640 1 -- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2258161730 msgr2=0x7f22580b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f22577fe640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2258161730 0x7f22580b5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f22577fe640 1 -- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22580d0090 msgr2=0x7f22580d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f22577fe640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22580d0090 0x7f22580d18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f22577fe640 1 -- 172.21.15.29:0/674642692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22580b7840 con 0x7f225806c530 2024-04-25T00:33:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f22577fe640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f225806c530 0x7f22581540b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f224800c7a0 tx=0x7f224800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f223ffff640 1 -- 172.21.15.29:0/674642692 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248003b80 con 0x7f225806c530 2024-04-25T00:33:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f223ffff640 1 -- 172.21.15.29:0/674642692 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2248003d20 con 0x7f225806c530 2024-04-25T00:33:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 -- 172.21.15.29:0/674642692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22580b4be0 con 0x7f225806c530 2024-04-25T00:33:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f223ffff640 1 -- 172.21.15.29:0/674642692 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224800aa90 con 0x7f225806c530 2024-04-25T00:33:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.870+0000 7f225dfa6640 1 -- 172.21.15.29:0/674642692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22580b5160 con 0x7f225806c530 2024-04-25T00:33:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.874+0000 7f225dfa6640 1 -- 172.21.15.29:0/674642692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22580692e0 con 0x7f225806c530 2024-04-25T00:33:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.874+0000 7f223ffff640 1 -- 172.21.15.29:0/674642692 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f224800ac30 con 0x7f225806c530 2024-04-25T00:33:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.874+0000 7f223ffff640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2244067a70 0x7f2244069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.874+0000 7f223ffff640 1 -- 172.21.15.29:0/674642692 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5357+0+0 (secure 0 0 0) 0x7f2248085d00 con 0x7f225806c530 2024-04-25T00:33:02.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.874+0000 7f2256ffd640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2244067a70 0x7f2244069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.874+0000 7f2256ffd640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2244067a70 0x7f2244069f30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f224c0af000 tx=0x7f224c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:02.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:02.878+0000 7f223ffff640 1 -- 172.21.15.29:0/674642692 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f22480567b0 con 0x7f225806c530 2024-04-25T00:33:03.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.054+0000 7f225dfa6640 1 -- 172.21.15.29:0/674642692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2258155e70 con 0x7f225806c530 2024-04-25T00:33:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.714+0000 7f223ffff640 1 -- 172.21.15.29:0/674642692 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) ==== 169+0+0 (secure 0 0 0) 0x7f22480560b0 con 0x7f225806c530 2024-04-25T00:33:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-04-25T00:33:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.718+0000 7f225dfa6640 1 -- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2244067a70 msgr2=0x7f2244069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.718+0000 7f225dfa6640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2244067a70 0x7f2244069f30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f224c0af000 tx=0x7f224c0a7000 comp rx=0 tx=0).stop 2024-04-25T00:33:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.718+0000 7f225dfa6640 1 -- 172.21.15.29:0/674642692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f225806c530 msgr2=0x7f22581540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.718+0000 7f225dfa6640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f225806c530 0x7f22581540b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f224800c7a0 tx=0x7f224800cc60 comp rx=0 tx=0).stop 2024-04-25T00:33:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.718+0000 7f225dfa6640 1 -- 172.21.15.29:0/674642692 shutdown_connections 2024-04-25T00:33:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.718+0000 7f225dfa6640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2258161730 0x7f22580b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.718+0000 7f225dfa6640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2244067a70 0x7f2244069f30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.718+0000 7f225dfa6640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22580d0090 0x7f22580d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.718+0000 7f225dfa6640 1 --2- 172.21.15.29:0/674642692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f225806c530 0x7f22581540b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.718+0000 7f225dfa6640 1 -- 172.21.15.29:0/674642692 >> 172.21.15.29:0/674642692 conn(0x7f225809aa30 msgr2=0x7f22580cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.718+0000 7f225dfa6640 1 -- 172.21.15.29:0/674642692 shutdown_connections 2024-04-25T00:33:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.718+0000 7f225dfa6640 1 -- 172.21.15.29:0/674642692 wait complete. 2024-04-25T00:33:03.746 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:33:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.870+0000 7ffa38429640 1 Processor -- start 2024-04-25T00:33:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa38429640 1 -- start start 2024-04-25T00:33:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa38429640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa3006dce0 0x7ffa30150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa38429640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa300b2b70 0x7ffa30151330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa38429640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa300b3130 con 0x7ffa3015d980 2024-04-25T00:33:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa38429640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ffa300cf560 con 0x7ffa300b2b70 2024-04-25T00:33:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa38429640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ffa300b3410 con 0x7ffa3006dce0 2024-04-25T00:33:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa369c6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa300b2b70 0x7ffa30151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa361c5640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ffa3015d980 0x7ffa3006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46554/0 (socket says 172.21.15.29:46554) 2024-04-25T00:33:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa361c5640 1 -- 172.21.15.29:0/3351124874 learned_addr learned my addr 172.21.15.29:0/3351124874 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa359c4640 1 --2- 172.21.15.29:0/3351124874 >> v2:172.21.15.157:3300/0 conn(0x7ffa3006dce0 0x7ffa30150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa369c6640 1 -- 172.21.15.29:0/3351124874 >> v2:172.21.15.157:3300/0 conn(0x7ffa3006dce0 msgr2=0x7ffa30150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa369c6640 1 --2- 172.21.15.29:0/3351124874 >> v2:172.21.15.157:3300/0 conn(0x7ffa3006dce0 0x7ffa30150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa369c6640 1 -- 172.21.15.29:0/3351124874 >> v1:172.21.15.29:6789/0 conn(0x7ffa3015d980 legacy=0x7ffa3006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa369c6640 1 -- 172.21.15.29:0/3351124874 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa300b36f0 con 0x7ffa300b2b70 2024-04-25T00:33:03.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa351c3640 1 -- 172.21.15.29:0/3351124874 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 645878910 0 0) 0x7ffa300b3130 con 0x7ffa3015d980 2024-04-25T00:33:03.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa359c4640 1 --2- 172.21.15.29:0/3351124874 >> v2:172.21.15.157:3300/0 conn(0x7ffa3006dce0 0x7ffa30150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.874+0000 7ffa369c6640 1 --2- 172.21.15.29:0/3351124874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa300b2b70 0x7ffa30151330 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffa300b2720 tx=0x7ffa2801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd293e24c8f18f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa351c3640 1 -- 172.21.15.29:0/3351124874 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa280b2070 con 0x7ffa300b2b70 2024-04-25T00:33:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa351c3640 1 -- 172.21.15.29:0/3351124874 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa280a59c0 con 0x7ffa300b2b70 2024-04-25T00:33:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa351c3640 1 -- 172.21.15.29:0/3351124874 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa280a5d90 con 0x7ffa300b2b70 2024-04-25T00:33:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 -- 172.21.15.29:0/3351124874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa300b2b70 msgr2=0x7ffa30151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 --2- 172.21.15.29:0/3351124874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa300b2b70 0x7ffa30151330 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffa300b2720 tx=0x7ffa2801c900 comp rx=0 tx=0).stop 2024-04-25T00:33:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 -- 172.21.15.29:0/3351124874 shutdown_connections 2024-04-25T00:33:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 --2- 172.21.15.29:0/3351124874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa300b2b70 0x7ffa30151330 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 --2- 172.21.15.29:0/3351124874 >> v2:172.21.15.157:3300/0 conn(0x7ffa3006dce0 0x7ffa30150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 -- 172.21.15.29:0/3351124874 >> 172.21.15.29:0/3351124874 conn(0x7ffa300b1870 msgr2=0x7ffa300b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 -- 172.21.15.29:0/3351124874 shutdown_connections 2024-04-25T00:33:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 -- 172.21.15.29:0/3351124874 wait complete. 2024-04-25T00:33:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 Processor -- start 2024-04-25T00:33:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 -- start start 2024-04-25T00:33:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa3006dce0 0x7ffa301386b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa300b2b70 0x7ffa300cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa3015d980 0x7ffa300ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3006cd40 con 0x7ffa300b2b70 2024-04-25T00:33:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa30151870 con 0x7ffa3015d980 2024-04-25T00:33:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3006cec0 con 0x7ffa3006dce0 2024-04-25T00:33:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa359c4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa300b2b70 0x7ffa300cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa359c4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa300b2b70 0x7ffa300cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38536/0 (socket says 172.21.15.29:38536) 2024-04-25T00:33:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa359c4640 1 -- 172.21.15.29:0/96368282 learned_addr learned my addr 172.21.15.29:0/96368282 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa369c6640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa3015d980 0x7ffa300ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa361c5640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa3006dce0 0x7ffa301386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa359c4640 1 -- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa3006dce0 msgr2=0x7ffa301386b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa359c4640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa3006dce0 0x7ffa301386b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa359c4640 1 -- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa3015d980 msgr2=0x7ffa300ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa359c4640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa3015d980 0x7ffa300ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa359c4640 1 -- 172.21.15.29:0/96368282 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa30151660 con 0x7ffa300b2b70 2024-04-25T00:33:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa359c4640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa300b2b70 0x7ffa300cc870 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c7c0 tx=0x7ffa2000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa26ffd640 1 -- 172.21.15.29:0/96368282 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20007a30 con 0x7ffa300b2b70 2024-04-25T00:33:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa26ffd640 1 -- 172.21.15.29:0/96368282 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa20007bd0 con 0x7ffa300b2b70 2024-04-25T00:33:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 -- 172.21.15.29:0/96368282 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa30138bf0 con 0x7ffa300b2b70 2024-04-25T00:33:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa26ffd640 1 -- 172.21.15.29:0/96368282 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa200113f0 con 0x7ffa300b2b70 2024-04-25T00:33:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.878+0000 7ffa38429640 1 -- 172.21.15.29:0/96368282 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3014ed50 con 0x7ffa300b2b70 2024-04-25T00:33:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.882+0000 7ffa38429640 1 -- 172.21.15.29:0/96368282 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa30069270 con 0x7ffa300b2b70 2024-04-25T00:33:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.882+0000 7ffa26ffd640 1 -- 172.21.15.29:0/96368282 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7ffa20011590 con 0x7ffa300b2b70 2024-04-25T00:33:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.882+0000 7ffa26ffd640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffa14067a70 0x7ffa14069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.882+0000 7ffa26ffd640 1 -- 172.21.15.29:0/96368282 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7ffa20085f30 con 0x7ffa300b2b70 2024-04-25T00:33:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.882+0000 7ffa361c5640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffa14067a70 0x7ffa14069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.882+0000 7ffa361c5640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffa14067a70 0x7ffa14069f30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c000f70 tx=0x7ffa2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:03.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:03.886+0000 7ffa26ffd640 1 -- 172.21.15.29:0/96368282 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffa30151660 con 0x7ffa300b2b70 2024-04-25T00:33:04.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.042+0000 7ffa38429640 1 -- 172.21.15.29:0/96368282 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ffa3006e1d0 con 0x7ffa300b2b70 2024-04-25T00:33:04.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.046+0000 7ffa26ffd640 1 -- 172.21.15.29:0/96368282 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7ffa20015020 con 0x7ffa300b2b70 2024-04-25T00:33:04.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.050+0000 7ffa38429640 1 -- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffa14067a70 msgr2=0x7ffa14069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.050+0000 7ffa38429640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffa14067a70 0x7ffa14069f30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c000f70 tx=0x7ffa2c009040 comp rx=0 tx=0).stop 2024-04-25T00:33:04.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.050+0000 7ffa38429640 1 -- 172.21.15.29:0/96368282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa300b2b70 msgr2=0x7ffa300cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.050+0000 7ffa38429640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa300b2b70 0x7ffa300cc870 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c7c0 tx=0x7ffa2000cc80 comp rx=0 tx=0).stop 2024-04-25T00:33:04.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.050+0000 7ffa38429640 1 -- 172.21.15.29:0/96368282 shutdown_connections 2024-04-25T00:33:04.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.050+0000 7ffa38429640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffa14067a70 0x7ffa14069f30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.050+0000 7ffa38429640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa3015d980 0x7ffa300ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.050+0000 7ffa38429640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa300b2b70 0x7ffa300cc870 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.050+0000 7ffa38429640 1 --2- 172.21.15.29:0/96368282 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa3006dce0 0x7ffa301386b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.050+0000 7ffa38429640 1 -- 172.21.15.29:0/96368282 >> 172.21.15.29:0/96368282 conn(0x7ffa300b1870 msgr2=0x7ffa300cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:04.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.050+0000 7ffa38429640 1 -- 172.21.15.29:0/96368282 shutdown_connections 2024-04-25T00:33:04.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.050+0000 7ffa38429640 1 -- 172.21.15.29:0/96368282 wait complete. 2024-04-25T00:33:04.071 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-04-25T00:33:04.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65be9d5640 1 Processor -- start 2024-04-25T00:33:04.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65be9d5640 1 -- start start 2024-04-25T00:33:04.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65be9d5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f65b80b3140 0x7f65b806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65be9d5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b4b10 0x7f65b80b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65be9d5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65b80b41e0 con 0x7f65b80b3a60 2024-04-25T00:33:04.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65be9d5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f65b806d960 con 0x7f65b80b3140 2024-04-25T00:33:04.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65be9d5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65b806dc40 con 0x7f65b80b4b10 2024-04-25T00:33:04.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65b7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f65b80b3140 0x7f65b806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65b7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f65b80b3140 0x7f65b806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38542/0 (socket says 172.21.15.29:38542) 2024-04-25T00:33:04.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65b7fff640 1 -- 172.21.15.29:0/43483775 learned_addr learned my addr 172.21.15.29:0/43483775 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:04.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65b77fe640 1 --2- 172.21.15.29:0/43483775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b4b10 0x7f65b80b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65b7fff640 1 -- 172.21.15.29:0/43483775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b4b10 msgr2=0x7f65b80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:04.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65b7fff640 1 --2- 172.21.15.29:0/43483775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b4b10 0x7f65b80b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65b7fff640 1 -- 172.21.15.29:0/43483775 >> v1:172.21.15.157:6789/0 conn(0x7f65b80b3a60 legacy=0x7f65b815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:04.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.182+0000 7f65b7fff640 1 -- 172.21.15.29:0/43483775 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b806df20 con 0x7f65b80b3140 2024-04-25T00:33:04.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65b7fff640 1 --2- 172.21.15.29:0/43483775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65b80b3140 0x7f65b806e440 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009870 tx=0x7f65a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514511fdd62d8762 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:04.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65b6ffd640 1 -- 172.21.15.29:0/43483775 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a80b0070 con 0x7f65b80b3140 2024-04-25T00:33:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65b6ffd640 1 -- 172.21.15.29:0/43483775 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a80ab600 con 0x7f65b80b3140 2024-04-25T00:33:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65b6ffd640 1 -- 172.21.15.29:0/43483775 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a80a4e60 con 0x7f65b80b3140 2024-04-25T00:33:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 -- 172.21.15.29:0/43483775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65b80b3140 msgr2=0x7f65b806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 --2- 172.21.15.29:0/43483775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65b80b3140 0x7f65b806e440 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009870 tx=0x7f65a801c520 comp rx=0 tx=0).stop 2024-04-25T00:33:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 -- 172.21.15.29:0/43483775 shutdown_connections 2024-04-25T00:33:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 --2- 172.21.15.29:0/43483775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b4b10 0x7f65b80b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 --2- 172.21.15.29:0/43483775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65b80b3140 0x7f65b806e440 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 -- 172.21.15.29:0/43483775 >> 172.21.15.29:0/43483775 conn(0x7f65b80b1870 msgr2=0x7f65b80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 -- 172.21.15.29:0/43483775 shutdown_connections 2024-04-25T00:33:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 -- 172.21.15.29:0/43483775 wait complete. 2024-04-25T00:33:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 Processor -- start 2024-04-25T00:33:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 -- start start 2024-04-25T00:33:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b80b3140 0x7f65b81545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65b80b3a60 0x7f65b8138120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b4b10 0x7f65b8138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8168b70 con 0x7f65b80b3a60 2024-04-25T00:33:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f65b81689f0 con 0x7f65b80b3140 2024-04-25T00:33:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65be9d5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65b8168cf0 con 0x7f65b80b4b10 2024-04-25T00:33:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65b77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65b80b3a60 0x7f65b8138120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65b77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65b80b3a60 0x7f65b8138120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38550/0 (socket says 172.21.15.29:38550) 2024-04-25T00:33:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65b77fe640 1 -- 172.21.15.29:0/2712439587 learned_addr learned my addr 172.21.15.29:0/2712439587 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65b7fff640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b80b3140 0x7f65b81545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65bcf72640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b4b10 0x7f65b8138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65b77fe640 1 -- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b4b10 msgr2=0x7f65b8138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.186+0000 7f65b77fe640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b4b10 0x7f65b8138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65b77fe640 1 -- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b80b3140 msgr2=0x7f65b81545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65b77fe640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b80b3140 0x7f65b81545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65b77fe640 1 -- 172.21.15.29:0/2712439587 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b813a790 con 0x7f65b80b3a60 2024-04-25T00:33:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65b77fe640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65b80b3a60 0x7f65b8138120 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00c7f0 tx=0x7f65ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65b4ff9640 1 -- 172.21.15.29:0/2712439587 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac015070 con 0x7f65b80b3a60 2024-04-25T00:33:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65b4ff9640 1 -- 172.21.15.29:0/2712439587 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ac007820 con 0x7f65b80b3a60 2024-04-25T00:33:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65be9d5640 1 -- 172.21.15.29:0/2712439587 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b8159c40 con 0x7f65b80b3a60 2024-04-25T00:33:04.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65b4ff9640 1 -- 172.21.15.29:0/2712439587 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac007c10 con 0x7f65b80b3a60 2024-04-25T00:33:04.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65be9d5640 1 -- 172.21.15.29:0/2712439587 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b815a200 con 0x7f65b80b3a60 2024-04-25T00:33:04.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65be9d5640 1 -- 172.21.15.29:0/2712439587 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b8069270 con 0x7f65b80b3a60 2024-04-25T00:33:04.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65b4ff9640 1 -- 172.21.15.29:0/2712439587 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f65ac018070 con 0x7f65b80b3a60 2024-04-25T00:33:04.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65b4ff9640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f65a4067a70 0x7f65a4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65b4ff9640 1 -- 172.21.15.29:0/2712439587 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f65ac085e90 con 0x7f65b80b3a60 2024-04-25T00:33:04.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65b7fff640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f65a4067a70 0x7f65a4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.190+0000 7f65b7fff640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f65a4067a70 0x7f65a4069f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f65a80ad000 tx=0x7f65a80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:04.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.194+0000 7f65b4ff9640 1 -- 172.21.15.29:0/2712439587 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f65ac056940 con 0x7f65b80b3a60 2024-04-25T00:33:04.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.414+0000 7f65be9d5640 1 -- 172.21.15.29:0/2712439587 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f65b81599f0 con 0x7f65b80b3a60 2024-04-25T00:33:04.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.718+0000 7f65b4ff9640 1 -- 172.21.15.29:0/2712439587 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v104) ==== 158+0+0 (secure 0 0 0) 0x7f65ac056240 con 0x7f65b80b3a60 2024-04-25T00:33:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.722+0000 7f65be9d5640 1 -- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f65a4067a70 msgr2=0x7f65a4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.722+0000 7f65be9d5640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f65a4067a70 0x7f65a4069f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f65a80ad000 tx=0x7f65a80a6000 comp rx=0 tx=0).stop 2024-04-25T00:33:04.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.722+0000 7f65be9d5640 1 -- 172.21.15.29:0/2712439587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65b80b3a60 msgr2=0x7f65b8138120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.722+0000 7f65be9d5640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65b80b3a60 0x7f65b8138120 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00c7f0 tx=0x7f65ac00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.722+0000 7f65be9d5640 1 -- 172.21.15.29:0/2712439587 shutdown_connections 2024-04-25T00:33:04.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.722+0000 7f65be9d5640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b4b10 0x7f65b8138660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.722+0000 7f65be9d5640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65b80b3a60 0x7f65b8138120 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.722+0000 7f65be9d5640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f65a4067a70 0x7f65a4069f30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.722+0000 7f65be9d5640 1 --2- 172.21.15.29:0/2712439587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b80b3140 0x7f65b81545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.722+0000 7f65be9d5640 1 -- 172.21.15.29:0/2712439587 >> 172.21.15.29:0/2712439587 conn(0x7f65b80b1870 msgr2=0x7f65b815d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:04.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.722+0000 7f65be9d5640 1 -- 172.21.15.29:0/2712439587 shutdown_connections 2024-04-25T00:33:04.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.722+0000 7f65be9d5640 1 -- 172.21.15.29:0/2712439587 wait complete. 2024-04-25T00:33:04.750 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2024-04-25T00:33:04.750 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-04-25T00:33:04.751 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:33:04.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.870+0000 7f198a0dc640 1 Processor -- start 2024-04-25T00:33:04.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.870+0000 7f198a0dc640 1 -- start start 2024-04-25T00:33:04.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f198a0dc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19840b3c30 0x7f198406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f198a0dc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19840b4b10 0x7f19840ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f198a0dc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19840b41e0 con 0x7f19840b3140 2024-04-25T00:33:04.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f198a0dc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f198406d960 con 0x7f19840b4b10 2024-04-25T00:33:04.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f198a0dc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f198406dc40 con 0x7f19840b3c30 2024-04-25T00:33:04.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f1983fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19840b4b10 0x7f19840ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f1982ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19840b3c30 0x7f198406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f19837fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f19840b3140 0x7f19840b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:58696/0 (socket says 172.21.15.29:58696) 2024-04-25T00:33:04.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f1983fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19840b4b10 0x7f19840ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:38748/0 (socket says 172.21.15.29:38748) 2024-04-25T00:33:04.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f19837fe640 1 -- 172.21.15.29:0/2202272643 learned_addr learned my addr 172.21.15.29:0/2202272643 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:04.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f1982ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19840b3c30 0x7f198406e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:36896/0 (socket says 172.21.15.29:36896) 2024-04-25T00:33:04.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f1983fff640 1 -- 172.21.15.29:0/2202272643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19840b3c30 msgr2=0x7f198406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:04.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f1983fff640 1 --2- 172.21.15.29:0/2202272643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19840b3c30 0x7f198406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f1983fff640 1 -- 172.21.15.29:0/2202272643 >> v1:172.21.15.157:6789/0 conn(0x7f19840b3140 legacy=0x7f19840b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f1983fff640 1 -- 172.21.15.29:0/2202272643 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198406df20 con 0x7f19840b4b10 2024-04-25T00:33:04.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.874+0000 7f1982ffd640 1 --2- 172.21.15.29:0/2202272643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19840b3c30 0x7f198406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:33:04.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f1983fff640 1 --2- 172.21.15.29:0/2202272643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19840b4b10 0x7f19840ce240 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f197c0097c0 tx=0x7f197c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dec2b22f857986dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:04.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f19827fc640 1 -- 172.21.15.29:0/2202272643 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c0b0070 con 0x7f19840b4b10 2024-04-25T00:33:04.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f19827fc640 1 -- 172.21.15.29:0/2202272643 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f197c0ab600 con 0x7f19840b4b10 2024-04-25T00:33:04.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f19827fc640 1 -- 172.21.15.29:0/2202272643 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c0ab9d0 con 0x7f19840b4b10 2024-04-25T00:33:04.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 -- 172.21.15.29:0/2202272643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19840b4b10 msgr2=0x7f19840ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 --2- 172.21.15.29:0/2202272643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19840b4b10 0x7f19840ce240 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f197c0097c0 tx=0x7f197c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 -- 172.21.15.29:0/2202272643 shutdown_connections 2024-04-25T00:33:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 --2- 172.21.15.29:0/2202272643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19840b4b10 0x7f19840ce240 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 --2- 172.21.15.29:0/2202272643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19840b3c30 0x7f198406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 -- 172.21.15.29:0/2202272643 >> 172.21.15.29:0/2202272643 conn(0x7f19840b1870 msgr2=0x7f19840b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 -- 172.21.15.29:0/2202272643 shutdown_connections 2024-04-25T00:33:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 -- 172.21.15.29:0/2202272643 wait complete. 2024-04-25T00:33:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 Processor -- start 2024-04-25T00:33:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 -- start start 2024-04-25T00:33:04.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19840b3140 0x7f198415bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19840b3c30 0x7f19840715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19840b4b10 0x7f1984142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1984168d80 con 0x7f19840b4b10 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1984168c00 con 0x7f19840b3140 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f198a0dc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1984168f00 con 0x7f19840b3c30 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f1983fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19840b4b10 0x7f1984142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f1983fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19840b4b10 0x7f1984142ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38560/0 (socket says 172.21.15.29:38560) 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f1983fff640 1 -- 172.21.15.29:0/1377057903 learned_addr learned my addr 172.21.15.29:0/1377057903 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.878+0000 7f1982ffd640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19840b3c30 0x7f19840715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f19837fe640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19840b3140 0x7f198415bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f1983fff640 1 -- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19840b3c30 msgr2=0x7f19840715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f1983fff640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19840b3c30 0x7f19840715d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f1983fff640 1 -- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19840b3140 msgr2=0x7f198415bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f1983fff640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19840b3140 0x7f198415bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f1983fff640 1 -- 172.21.15.29:0/1377057903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19840af330 con 0x7f19840b4b10 2024-04-25T00:33:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f1983fff640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19840b4b10 0x7f1984142ca0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f197c01a0d0 tx=0x7f197c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f196bfff640 1 -- 172.21.15.29:0/1377057903 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c0b0040 con 0x7f19840b4b10 2024-04-25T00:33:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f196bfff640 1 -- 172.21.15.29:0/1377057903 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f197c0bc070 con 0x7f19840b4b10 2024-04-25T00:33:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f196bfff640 1 -- 172.21.15.29:0/1377057903 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c006e20 con 0x7f19840b4b10 2024-04-25T00:33:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f198a0dc640 1 -- 172.21.15.29:0/1377057903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19840ae7b0 con 0x7f19840b4b10 2024-04-25T00:33:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f198a0dc640 1 -- 172.21.15.29:0/1377057903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19840aed70 con 0x7f19840b4b10 2024-04-25T00:33:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f198a0dc640 1 -- 172.21.15.29:0/1377057903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1984069270 con 0x7f19840b4b10 2024-04-25T00:33:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f196bfff640 1 -- 172.21.15.29:0/1377057903 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f197c0b3070 con 0x7f19840b4b10 2024-04-25T00:33:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f196bfff640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1970067a70 0x7f1970069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f196bfff640 1 -- 172.21.15.29:0/1377057903 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f197c11fef0 con 0x7f19840b4b10 2024-04-25T00:33:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f19837fe640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1970067a70 0x7f1970069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.882+0000 7f19837fe640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1970067a70 0x7f1970069f30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1978005890 tx=0x7f1978009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:04.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:04.886+0000 7f196bfff640 1 -- 172.21.15.29:0/1377057903 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f197c016090 con 0x7f19840b4b10 2024-04-25T00:33:05.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.046+0000 7f198a0dc640 1 -- 172.21.15.29:0/1377057903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19840ce200 con 0x7f19840b4b10 2024-04-25T00:33:05.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.050+0000 7f196bfff640 1 -- 172.21.15.29:0/1377057903 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+12972 (secure 0 0 0) 0x7f197c0f0ac0 con 0x7f19840b4b10 2024-04-25T00:33:05.055 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:05.055 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":36,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:33:03.713977+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6835/1468158138":"2024-04-26T00:33:00.700768+0000","172.21.15.29:6834/1468158138":"2024-04-26T00:33:00.700768+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:33:05.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.050+0000 7f198a0dc640 1 -- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1970067a70 msgr2=0x7f1970069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.050+0000 7f198a0dc640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1970067a70 0x7f1970069f30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1978005890 tx=0x7f1978009040 comp rx=0 tx=0).stop 2024-04-25T00:33:05.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.050+0000 7f198a0dc640 1 -- 172.21.15.29:0/1377057903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19840b4b10 msgr2=0x7f1984142ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.050+0000 7f198a0dc640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19840b4b10 0x7f1984142ca0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f197c01a0d0 tx=0x7f197c019240 comp rx=0 tx=0).stop 2024-04-25T00:33:05.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.050+0000 7f198a0dc640 1 -- 172.21.15.29:0/1377057903 shutdown_connections 2024-04-25T00:33:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.050+0000 7f198a0dc640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19840b4b10 0x7f1984142ca0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.050+0000 7f198a0dc640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19840b3c30 0x7f19840715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.050+0000 7f198a0dc640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1970067a70 0x7f1970069f30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.050+0000 7f198a0dc640 1 --2- 172.21.15.29:0/1377057903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19840b3140 0x7f198415bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.050+0000 7f198a0dc640 1 -- 172.21.15.29:0/1377057903 >> 172.21.15.29:0/1377057903 conn(0x7f19840b1870 msgr2=0x7f19840cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.050+0000 7f198a0dc640 1 -- 172.21.15.29:0/1377057903 shutdown_connections 2024-04-25T00:33:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.050+0000 7f198a0dc640 1 -- 172.21.15.29:0/1377057903 wait complete. 2024-04-25T00:33:05.074 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:33:05.075 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:33:05.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.194+0000 7f484574e640 1 Processor -- start 2024-04-25T00:33:05.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.194+0000 7f484574e640 1 -- start start 2024-04-25T00:33:05.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f484574e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4840162420 0x7f4840070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f484574e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48400b4e00 0x7f48400b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f484574e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4840071910 con 0x7f48400b30b0 2024-04-25T00:33:05.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f484574e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f48400b3500 con 0x7f4840162420 2024-04-25T00:33:05.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f484574e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4840071bf0 con 0x7f48400b4e00 2024-04-25T00:33:05.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f483e7fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48400b4e00 0x7f48400b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f483effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4840162420 0x7f4840070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f483effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4840162420 0x7f4840070f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:38760/0 (socket says 172.21.15.29:38760) 2024-04-25T00:33:05.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f483e7fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48400b4e00 0x7f48400b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:36908/0 (socket says 172.21.15.29:36908) 2024-04-25T00:33:05.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f483f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f48400b30b0 0x7f4840166020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:58704/0 (socket says 172.21.15.29:58704) 2024-04-25T00:33:05.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f483effd640 1 -- 172.21.15.29:0/144781402 learned_addr learned my addr 172.21.15.29:0/144781402 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:05.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f483effd640 1 -- 172.21.15.29:0/144781402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48400b4e00 msgr2=0x7f48400b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:05.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f483effd640 1 --2- 172.21.15.29:0/144781402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48400b4e00 0x7f48400b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f483effd640 1 -- 172.21.15.29:0/144781402 >> v1:172.21.15.157:6789/0 conn(0x7f48400b30b0 legacy=0x7f4840166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f483effd640 1 -- 172.21.15.29:0/144781402 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4840071ed0 con 0x7f4840162420 2024-04-25T00:33:05.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.198+0000 7f483e7fc640 1 --2- 172.21.15.29:0/144781402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48400b4e00 0x7f48400b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:33:05.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f483effd640 1 --2- 172.21.15.29:0/144781402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840162420 0x7f4840070f80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4838009e50 tx=0x7f483801ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=215e6a52f5b398df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:05.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f483dffb640 1 -- 172.21.15.29:0/144781402 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48380a5780 con 0x7f4840162420 2024-04-25T00:33:05.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f483dffb640 1 -- 172.21.15.29:0/144781402 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48380a5920 con 0x7f4840162420 2024-04-25T00:33:05.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f483dffb640 1 -- 172.21.15.29:0/144781402 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48380a5cf0 con 0x7f4840162420 2024-04-25T00:33:05.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 -- 172.21.15.29:0/144781402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840162420 msgr2=0x7f4840070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 --2- 172.21.15.29:0/144781402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840162420 0x7f4840070f80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4838009e50 tx=0x7f483801ca30 comp rx=0 tx=0).stop 2024-04-25T00:33:05.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 -- 172.21.15.29:0/144781402 shutdown_connections 2024-04-25T00:33:05.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 --2- 172.21.15.29:0/144781402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48400b4e00 0x7f48400b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 --2- 172.21.15.29:0/144781402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840162420 0x7f4840070f80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 -- 172.21.15.29:0/144781402 >> 172.21.15.29:0/144781402 conn(0x7f48400b1870 msgr2=0x7f48400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:05.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 -- 172.21.15.29:0/144781402 shutdown_connections 2024-04-25T00:33:05.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 -- 172.21.15.29:0/144781402 wait complete. 2024-04-25T00:33:05.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 Processor -- start 2024-04-25T00:33:05.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 -- start start 2024-04-25T00:33:05.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48400b30b0 0x7f4840155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48400b4e00 0x7f4840155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840162420 0x7f48400b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f484016f3c0 con 0x7f48400b4e00 2024-04-25T00:33:05.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f48400716d0 con 0x7f4840162420 2024-04-25T00:33:05.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f484574e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f484016f540 con 0x7f48400b30b0 2024-04-25T00:33:05.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f483e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48400b4e00 0x7f4840155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f483e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48400b4e00 0x7f4840155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38568/0 (socket says 172.21.15.29:38568) 2024-04-25T00:33:05.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.202+0000 7f483e7fc640 1 -- 172.21.15.29:0/3816713190 learned_addr learned my addr 172.21.15.29:0/3816713190 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:05.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f483effd640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48400b30b0 0x7f4840155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f483f7fe640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840162420 0x7f48400b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f483e7fc640 1 -- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48400b30b0 msgr2=0x7f4840155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f483e7fc640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48400b30b0 0x7f4840155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f483e7fc640 1 -- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840162420 msgr2=0x7f48400b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f483e7fc640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840162420 0x7f48400b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f483e7fc640 1 -- 172.21.15.29:0/3816713190 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4840156390 con 0x7f48400b4e00 2024-04-25T00:33:05.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f483e7fc640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48400b4e00 0x7f4840155450 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f483400e9d0 tx=0x7f483400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:05.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f48277fe640 1 -- 172.21.15.29:0/3816713190 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834017070 con 0x7f48400b4e00 2024-04-25T00:33:05.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f48277fe640 1 -- 172.21.15.29:0/3816713190 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f483400ca60 con 0x7f48400b4e00 2024-04-25T00:33:05.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f484574e640 1 -- 172.21.15.29:0/3816713190 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4840160bf0 con 0x7f48400b4e00 2024-04-25T00:33:05.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f48277fe640 1 -- 172.21.15.29:0/3816713190 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483400ce50 con 0x7f48400b4e00 2024-04-25T00:33:05.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f484574e640 1 -- 172.21.15.29:0/3816713190 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f484015f720 con 0x7f48400b4e00 2024-04-25T00:33:05.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f484574e640 1 -- 172.21.15.29:0/3816713190 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4840069270 con 0x7f48400b4e00 2024-04-25T00:33:05.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f48277fe640 1 -- 172.21.15.29:0/3816713190 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f483401a070 con 0x7f48400b4e00 2024-04-25T00:33:05.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f48277fe640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f482c067c10 0x7f482c06a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f48277fe640 1 -- 172.21.15.29:0/3816713190 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f4834085900 con 0x7f48400b4e00 2024-04-25T00:33:05.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f483effd640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f482c067c10 0x7f482c06a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.206+0000 7f483effd640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f482c067c10 0x7f482c06a0d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f48380ae000 tx=0x7f48380a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:05.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.210+0000 7f48277fe640 1 -- 172.21.15.29:0/3816713190 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4834056420 con 0x7f48400b4e00 2024-04-25T00:33:05.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.374+0000 7f484574e640 1 -- 172.21.15.29:0/3816713190 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4840072fa0 con 0x7f48400b4e00 2024-04-25T00:33:05.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.374+0000 7f48277fe640 1 -- 172.21.15.29:0/3816713190 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f4834055d20 con 0x7f48400b4e00 2024-04-25T00:33:05.379 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:33:05.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.378+0000 7f484574e640 1 -- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f482c067c10 msgr2=0x7f482c06a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.378+0000 7f484574e640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f482c067c10 0x7f482c06a0d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f48380ae000 tx=0x7f48380a7070 comp rx=0 tx=0).stop 2024-04-25T00:33:05.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.378+0000 7f484574e640 1 -- 172.21.15.29:0/3816713190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48400b4e00 msgr2=0x7f4840155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.378+0000 7f484574e640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48400b4e00 0x7f4840155450 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f483400e9d0 tx=0x7f483400ee90 comp rx=0 tx=0).stop 2024-04-25T00:33:05.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.378+0000 7f484574e640 1 -- 172.21.15.29:0/3816713190 shutdown_connections 2024-04-25T00:33:05.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.378+0000 7f484574e640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f482c067c10 0x7f482c06a0d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.378+0000 7f484574e640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840162420 0x7f48400b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.378+0000 7f484574e640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48400b4e00 0x7f4840155450 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.378+0000 7f484574e640 1 --2- 172.21.15.29:0/3816713190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48400b30b0 0x7f4840155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.378+0000 7f484574e640 1 -- 172.21.15.29:0/3816713190 >> 172.21.15.29:0/3816713190 conn(0x7f48400b1870 msgr2=0x7f48400cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:05.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.378+0000 7f484574e640 1 -- 172.21.15.29:0/3816713190 shutdown_connections 2024-04-25T00:33:05.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.378+0000 7f484574e640 1 -- 172.21.15.29:0/3816713190 wait complete. 2024-04-25T00:33:05.398 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-04-25T00:33:05.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83b12c0640 1 Processor -- start 2024-04-25T00:33:05.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83b12c0640 1 -- start start 2024-04-25T00:33:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83b12c0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f83ac15e8d0 0x7f83ac0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83b12c0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac15f3c0 0x7f83ac15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83b12c0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83ac0b3720 con 0x7f83ac0b4780 2024-04-25T00:33:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83b12c0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f83ac0b42f0 con 0x7f83ac15e8d0 2024-04-25T00:33:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83b12c0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83ac0b39c0 con 0x7f83ac15f3c0 2024-04-25T00:33:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83aaffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f83ac15e8d0 0x7f83ac0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83aaffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f83ac15e8d0 0x7f83ac0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38582/0 (socket says 172.21.15.29:38582) 2024-04-25T00:33:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83aaffd640 1 -- 172.21.15.29:0/3287895981 learned_addr learned my addr 172.21.15.29:0/3287895981 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83ab7fe640 1 --2- 172.21.15.29:0/3287895981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac15f3c0 0x7f83ac15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83aaffd640 1 -- 172.21.15.29:0/3287895981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac15f3c0 msgr2=0x7f83ac15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83aaffd640 1 --2- 172.21.15.29:0/3287895981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac15f3c0 0x7f83ac15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83aaffd640 1 -- 172.21.15.29:0/3287895981 >> v1:172.21.15.157:6789/0 conn(0x7f83ac0b4780 legacy=0x7f83ac15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83aaffd640 1 -- 172.21.15.29:0/3287895981 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac0b3ca0 con 0x7f83ac15e8d0 2024-04-25T00:33:05.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.514+0000 7f83aaffd640 1 --2- 172.21.15.29:0/3287895981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac15e8d0 0x7f83ac0cc240 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8394009870 tx=0x7f839401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a99f43cd87d23c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:05.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83a9ffb640 1 -- 172.21.15.29:0/3287895981 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940b1070 con 0x7f83ac15e8d0 2024-04-25T00:33:05.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83a9ffb640 1 -- 172.21.15.29:0/3287895981 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83940ac620 con 0x7f83ac15e8d0 2024-04-25T00:33:05.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83a9ffb640 1 -- 172.21.15.29:0/3287895981 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940ac9f0 con 0x7f83ac15e8d0 2024-04-25T00:33:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 -- 172.21.15.29:0/3287895981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac15e8d0 msgr2=0x7f83ac0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 --2- 172.21.15.29:0/3287895981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac15e8d0 0x7f83ac0cc240 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8394009870 tx=0x7f839401c4f0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 -- 172.21.15.29:0/3287895981 shutdown_connections 2024-04-25T00:33:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 --2- 172.21.15.29:0/3287895981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac15f3c0 0x7f83ac15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 --2- 172.21.15.29:0/3287895981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac15e8d0 0x7f83ac0cc240 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 -- 172.21.15.29:0/3287895981 >> 172.21.15.29:0/3287895981 conn(0x7f83ac0b1870 msgr2=0x7f83ac0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 -- 172.21.15.29:0/3287895981 shutdown_connections 2024-04-25T00:33:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 -- 172.21.15.29:0/3287895981 wait complete. 2024-04-25T00:33:05.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 Processor -- start 2024-04-25T00:33:05.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 -- start start 2024-04-25T00:33:05.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac0b4780 0x7f83ac13ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83ac15e8d0 0x7f83ac143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac15f3c0 0x7f83ac13d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac15d6a0 con 0x7f83ac0b4780 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac15d520 con 0x7f83ac15e8d0 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.518+0000 7f83b12c0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83ac167c70 con 0x7f83ac15f3c0 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83aaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac0b4780 0x7f83ac13ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83aaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac0b4780 0x7f83ac13ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38596/0 (socket says 172.21.15.29:38596) 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83aaffd640 1 -- 172.21.15.29:0/3818274445 learned_addr learned my addr 172.21.15.29:0/3818274445 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83ab7fe640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac15f3c0 0x7f83ac13d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83aa7fc640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83ac15e8d0 0x7f83ac143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83aaffd640 1 -- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac15f3c0 msgr2=0x7f83ac13d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83aaffd640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac15f3c0 0x7f83ac13d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83aaffd640 1 -- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83ac15e8d0 msgr2=0x7f83ac143a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83aaffd640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83ac15e8d0 0x7f83ac143a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83aaffd640 1 -- 172.21.15.29:0/3818274445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac1420c0 con 0x7f83ac0b4780 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83aa7fc640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83ac15e8d0 0x7f83ac143a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:05.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83aaffd640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac0b4780 0x7f83ac13ab40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f839401aba0 tx=0x7f839401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:05.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83a37fe640 1 -- 172.21.15.29:0/3818274445 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940bb020 con 0x7f83ac0b4780 2024-04-25T00:33:05.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83a37fe640 1 -- 172.21.15.29:0/3818274445 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83940b7640 con 0x7f83ac0b4780 2024-04-25T00:33:05.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83a37fe640 1 -- 172.21.15.29:0/3818274445 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940b1040 con 0x7f83ac0b4780 2024-04-25T00:33:05.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83b12c0640 1 -- 172.21.15.29:0/3818274445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ac1423a0 con 0x7f83ac0b4780 2024-04-25T00:33:05.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83b12c0640 1 -- 172.21.15.29:0/3818274445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ac142960 con 0x7f83ac0b4780 2024-04-25T00:33:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83b12c0640 1 -- 172.21.15.29:0/3818274445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ac069270 con 0x7f83ac0b4780 2024-04-25T00:33:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83a37fe640 1 -- 172.21.15.29:0/3818274445 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f83940be020 con 0x7f83ac0b4780 2024-04-25T00:33:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83a37fe640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f838c067aa0 0x7f838c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83a37fe640 1 -- 172.21.15.29:0/3818274445 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f8394120190 con 0x7f83ac0b4780 2024-04-25T00:33:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.522+0000 7f83aa7fc640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f838c067aa0 0x7f838c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.526+0000 7f83aa7fc640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f838c067aa0 0x7f838c069f60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0b13a0 tx=0x7f839c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:05.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.526+0000 7f83a37fe640 1 -- 172.21.15.29:0/3818274445 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f83940c5030 con 0x7f83ac0b4780 2024-04-25T00:33:05.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:05.738+0000 7f83b12c0640 1 -- 172.21.15.29:0/3818274445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f83ac15c090 con 0x7f83ac0b4780 2024-04-25T00:33:06.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.726+0000 7f83a37fe640 1 -- 172.21.15.29:0/3818274445 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v106) ==== 143+0+0 (secure 0 0 0) 0x7f8394016110 con 0x7f83ac0b4780 2024-04-25T00:33:06.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.730+0000 7f83b12c0640 1 -- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f838c067aa0 msgr2=0x7f838c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:06.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.730+0000 7f83b12c0640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f838c067aa0 0x7f838c069f60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0b13a0 tx=0x7f839c009040 comp rx=0 tx=0).stop 2024-04-25T00:33:06.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.730+0000 7f83b12c0640 1 -- 172.21.15.29:0/3818274445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac0b4780 msgr2=0x7f83ac13ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:06.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.730+0000 7f83b12c0640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac0b4780 0x7f83ac13ab40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f839401aba0 tx=0x7f839401ad30 comp rx=0 tx=0).stop 2024-04-25T00:33:06.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.730+0000 7f83b12c0640 1 -- 172.21.15.29:0/3818274445 shutdown_connections 2024-04-25T00:33:06.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.730+0000 7f83b12c0640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac15f3c0 0x7f83ac13d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.730+0000 7f83b12c0640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f838c067aa0 0x7f838c069f60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.730+0000 7f83b12c0640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83ac15e8d0 0x7f83ac143a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.730+0000 7f83b12c0640 1 --2- 172.21.15.29:0/3818274445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac0b4780 0x7f83ac13ab40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.730+0000 7f83b12c0640 1 -- 172.21.15.29:0/3818274445 >> 172.21.15.29:0/3818274445 conn(0x7f83ac0b1870 msgr2=0x7f83ac13ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:06.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.730+0000 7f83b12c0640 1 -- 172.21.15.29:0/3818274445 shutdown_connections 2024-04-25T00:33:06.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.730+0000 7f83b12c0640 1 -- 172.21.15.29:0/3818274445 wait complete. 2024-04-25T00:33:06.757 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:33:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecf6896640 1 Processor -- start 2024-04-25T00:33:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecf6896640 1 -- start start 2024-04-25T00:33:06.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecf6896640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf00b4d30 0x7fecf00b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:06.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecf6896640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fecf006d210 0x7fecf00ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:06.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecf6896640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecf006d980 con 0x7fecf015df70 2024-04-25T00:33:06.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecf6896640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fecf006dc60 con 0x7fecf006d210 2024-04-25T00:33:06.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecf6896640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fecf006df40 con 0x7fecf00b4d30 2024-04-25T00:33:06.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fecf015df70 0x7fecf00b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46564/0 (socket says 172.21.15.29:46564) 2024-04-25T00:33:06.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceffff640 1 -- 172.21.15.29:0/3394008628 learned_addr learned my addr 172.21.15.29:0/3394008628 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecf4e33640 1 --2- 172.21.15.29:0/3394008628 >> v2:172.21.15.157:3300/0 conn(0x7fecf006d210 0x7fecf00ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecef7fe640 1 --2- 172.21.15.29:0/3394008628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf00b4d30 0x7fecf00b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1378147439 0 0) 0x7fecf006d980 con 0x7fecf015df70 2024-04-25T00:33:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecd8003530 con 0x7fecf015df70 2024-04-25T00:33:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3482109685 0 0) 0x7fecd8003530 con 0x7fecf015df70 2024-04-25T00:33:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf00b4d30 msgr2=0x7fecf00b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 --2- 172.21.15.29:0/3394008628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf00b4d30 0x7fecf00b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 >> v2:172.21.15.157:3300/0 conn(0x7fecf006d210 msgr2=0x7fecf00ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 --2- 172.21.15.29:0/3394008628 >> v2:172.21.15.157:3300/0 conn(0x7fecf006d210 0x7fecf00ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf006e220 con 0x7fecf015df70 2024-04-25T00:33:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fece0001680 con 0x7fecf015df70 2024-04-25T00:33:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 >> v1:172.21.15.29:6789/0 conn(0x7fecf015df70 legacy=0x7fecf00b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 --2- 172.21.15.29:0/3394008628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf00b4d30 0x7fecd80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 --2- 172.21.15.29:0/3394008628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecd8003f40 0x7fecd8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 --2- 172.21.15.29:0/3394008628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d50 0x7fecd80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fecf006dc60 con 0x7fecd8003f40 2024-04-25T00:33:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fecf006df40 con 0x7fecd8004d50 2024-04-25T00:33:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fecd800b9d0 con 0x7fecf00b4d30 2024-04-25T00:33:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecef7fe640 1 --2- 172.21.15.29:0/3394008628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecd8003f40 0x7fecd8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecf4e33640 1 --2- 172.21.15.29:0/3394008628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d50 0x7fecd80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7feceffff640 1 --2- 172.21.15.29:0/3394008628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf00b4d30 0x7fecd80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecef7fe640 1 -- 172.21.15.29:0/3394008628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf00b4d30 msgr2=0x7fecd80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecef7fe640 1 --2- 172.21.15.29:0/3394008628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf00b4d30 0x7fecd80039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecef7fe640 1 -- 172.21.15.29:0/3394008628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d50 msgr2=0x7fecd80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecef7fe640 1 --2- 172.21.15.29:0/3394008628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d50 0x7fecd80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecef7fe640 1 -- 172.21.15.29:0/3394008628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd8003530 con 0x7fecd8003f40 2024-04-25T00:33:06.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.886+0000 7fecf4e33640 1 --2- 172.21.15.29:0/3394008628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d50 0x7fecd80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:06.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.890+0000 7fecef7fe640 1 --2- 172.21.15.29:0/3394008628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecd8003f40 0x7fecd8004810 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fece4009ae0 tx=0x7fece401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833943ee3c52bef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:06.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.890+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece40b2070 con 0x7fecd8003f40 2024-04-25T00:33:06.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.890+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece40a5660 con 0x7fecd8003f40 2024-04-25T00:33:06.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.890+0000 7feceeffd640 1 -- 172.21.15.29:0/3394008628 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece40096b0 con 0x7fecd8003f40 2024-04-25T00:33:06.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.890+0000 7fecf6896640 1 -- 172.21.15.29:0/3394008628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecd8003f40 msgr2=0x7fecd8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:06.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.890+0000 7fecf6896640 1 --2- 172.21.15.29:0/3394008628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecd8003f40 0x7fecd8004810 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fece4009ae0 tx=0x7fece401ca10 comp rx=0 tx=0).stop 2024-04-25T00:33:06.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.890+0000 7fecf4e33640 1 -- 172.21.15.29:0/3394008628 reap_dead start 2024-04-25T00:33:06.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.890+0000 7fecf6896640 1 -- 172.21.15.29:0/3394008628 shutdown_connections 2024-04-25T00:33:06.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.890+0000 7fecf6896640 1 -- 172.21.15.29:0/3394008628 >> 172.21.15.29:0/3394008628 conn(0x7fecf00b1870 msgr2=0x7fecf00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:06.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.890+0000 7fecf6896640 1 -- 172.21.15.29:0/3394008628 shutdown_connections 2024-04-25T00:33:06.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.890+0000 7fecf6896640 1 -- 172.21.15.29:0/3394008628 wait complete. 2024-04-25T00:33:06.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.890+0000 7fecf6896640 1 Processor -- start 2024-04-25T00:33:06.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.890+0000 7fecf6896640 1 -- start start 2024-04-25T00:33:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fecf6896640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecd8003f40 0x7fecf01532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fecf6896640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf015b4b0 0x7fecf01e9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fecf6896640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf015b930 0x7fecf01e9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fecf6896640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fecf015e350 con 0x7fecd8003f40 2024-04-25T00:33:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fecf6896640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fecf006d680 con 0x7fecf015b4b0 2024-04-25T00:33:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fecf6896640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fecf006ea20 con 0x7fecf015b930 2024-04-25T00:33:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7feceffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecd8003f40 0x7fecf01532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7feceffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecd8003f40 0x7fecf01532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38622/0 (socket says 172.21.15.29:38622) 2024-04-25T00:33:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7feceffff640 1 -- 172.21.15.29:0/1199547513 learned_addr learned my addr 172.21.15.29:0/1199547513 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fecef7fe640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf015b4b0 0x7fecf01e9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fecf4e33640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf015b930 0x7fecf01e9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7feceffff640 1 -- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf015b930 msgr2=0x7fecf01e9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7feceffff640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf015b930 0x7fecf01e9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7feceffff640 1 -- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf015b4b0 msgr2=0x7fecf01e9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:06.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7feceffff640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf015b4b0 0x7fecf01e9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7feceffff640 1 -- 172.21.15.29:0/1199547513 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf00684e0 con 0x7fecd8003f40 2024-04-25T00:33:06.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fecef7fe640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf015b4b0 0x7fecf01e9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:06.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7feceffff640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecd8003f40 0x7fecf01532b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fece0000fc0 tx=0x7fece0010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:06.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fececff9640 1 -- 172.21.15.29:0/1199547513 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece001b070 con 0x7fecd8003f40 2024-04-25T00:33:06.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fececff9640 1 -- 172.21.15.29:0/1199547513 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece0011040 con 0x7fecd8003f40 2024-04-25T00:33:06.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fecf6896640 1 -- 172.21.15.29:0/1199547513 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf0152510 con 0x7fecd8003f40 2024-04-25T00:33:06.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fececff9640 1 -- 172.21.15.29:0/1199547513 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece00143a0 con 0x7fecd8003f40 2024-04-25T00:33:06.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fecf6896640 1 -- 172.21.15.29:0/1199547513 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf0152ad0 con 0x7fecd8003f40 2024-04-25T00:33:06.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fecf6896640 1 -- 172.21.15.29:0/1199547513 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf0069270 con 0x7fecd8003f40 2024-04-25T00:33:06.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fececff9640 1 -- 172.21.15.29:0/1199547513 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fece001e070 con 0x7fecd8003f40 2024-04-25T00:33:06.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fececff9640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fecdc067a70 0x7fecdc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fececff9640 1 -- 172.21.15.29:0/1199547513 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fece0088800 con 0x7fecd8003f40 2024-04-25T00:33:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.894+0000 7fecef7fe640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fecdc067a70 0x7fecdc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.898+0000 7fecef7fe640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fecdc067a70 0x7fecdc069f30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fece40af000 tx=0x7fece40046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:06.898+0000 7fececff9640 1 -- 172.21.15.29:0/1199547513 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fece00593a0 con 0x7fecd8003f40 2024-04-25T00:33:07.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.062+0000 7fecf6896640 1 -- 172.21.15.29:0/1199547513 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fecf00b3bd0 con 0x7fecd8003f40 2024-04-25T00:33:07.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.066+0000 7fececff9640 1 -- 172.21.15.29:0/1199547513 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7fece0058ca0 con 0x7fecd8003f40 2024-04-25T00:33:07.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.066+0000 7fecf6896640 1 -- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fecdc067a70 msgr2=0x7fecdc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.066+0000 7fecf6896640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fecdc067a70 0x7fecdc069f30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fece40af000 tx=0x7fece40046d0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.066+0000 7fecf6896640 1 -- 172.21.15.29:0/1199547513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecd8003f40 msgr2=0x7fecf01532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.066+0000 7fecf6896640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecd8003f40 0x7fecf01532b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fece0000fc0 tx=0x7fece0010f90 comp rx=0 tx=0).stop 2024-04-25T00:33:07.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.066+0000 7fecf6896640 1 -- 172.21.15.29:0/1199547513 shutdown_connections 2024-04-25T00:33:07.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.066+0000 7fecf6896640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf015b930 0x7fecf01e9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.066+0000 7fecf6896640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fecdc067a70 0x7fecdc069f30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.066+0000 7fecf6896640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf015b4b0 0x7fecf01e9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.066+0000 7fecf6896640 1 --2- 172.21.15.29:0/1199547513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecd8003f40 0x7fecf01532b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.066+0000 7fecf6896640 1 -- 172.21.15.29:0/1199547513 >> 172.21.15.29:0/1199547513 conn(0x7fecf00b1870 msgr2=0x7fecf00b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.066+0000 7fecf6896640 1 -- 172.21.15.29:0/1199547513 shutdown_connections 2024-04-25T00:33:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.066+0000 7fecf6896640 1 -- 172.21.15.29:0/1199547513 wait complete. 2024-04-25T00:33:07.090 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:33:07.090 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:33:07.139 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:33:07.190 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:33:07.191 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:33:07.191 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:33:07.196 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:33:07.196 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:33:07.196 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:33:07.202 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:33:07.202 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:33:07.202 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:33:07.241 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:33:07.241 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:33:07.241 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:33:07.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:33:07.249 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:33:07.249 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:33:07.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.406+0000 7fc6f0a76640 1 Processor -- start 2024-04-25T00:33:07.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6f0a76640 1 -- start start 2024-04-25T00:33:07.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6f0a76640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6ec0b3c30 0x7fc6ec06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6f0a76640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc6ec0b4b10 0x7fc6ec0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6f0a76640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ec0b41e0 con 0x7fc6ec0b3140 2024-04-25T00:33:07.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6f0a76640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc6ec06d960 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6f0a76640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ec06dc40 con 0x7fc6ec0b3c30 2024-04-25T00:33:07.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6ead9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc6ec0b4b10 0x7fc6ec0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6e9d9b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6ec0b3c30 0x7fc6ec06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6ea59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc6ec0b3140 0x7fc6ec0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:58720/0 (socket says 172.21.15.29:58720) 2024-04-25T00:33:07.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6ea59c640 1 -- 172.21.15.29:0/3974227153 learned_addr learned my addr 172.21.15.29:0/3974227153 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:07.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6ead9d640 1 --2- 172.21.15.29:0/3974227153 >> v2:172.21.15.157:3300/0 conn(0x7fc6ec0b4b10 0x7fc6ec0ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:38798/0 (socket says 172.21.15.29:38798) 2024-04-25T00:33:07.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6ead9d640 1 -- 172.21.15.29:0/3974227153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6ec0b3c30 msgr2=0x7fc6ec06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:07.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6ead9d640 1 --2- 172.21.15.29:0/3974227153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6ec0b3c30 0x7fc6ec06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6ead9d640 1 -- 172.21.15.29:0/3974227153 >> v1:172.21.15.157:6789/0 conn(0x7fc6ec0b3140 legacy=0x7fc6ec0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.410+0000 7fc6ead9d640 1 -- 172.21.15.29:0/3974227153 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec06df20 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6ead9d640 1 --2- 172.21.15.29:0/3974227153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ec0b4b10 0x7fc6ec0ce240 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009d90 tx=0x7fc6e401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b79b78de21cd9558 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6e959a640 1 -- 172.21.15.29:0/3974227153 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40b2070 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6e959a640 1 -- 172.21.15.29:0/3974227153 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e40a5830 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6e959a640 1 -- 172.21.15.29:0/3974227153 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40a5c00 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 -- 172.21.15.29:0/3974227153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ec0b4b10 msgr2=0x7fc6ec0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 --2- 172.21.15.29:0/3974227153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ec0b4b10 0x7fc6ec0ce240 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009d90 tx=0x7fc6e401c970 comp rx=0 tx=0).stop 2024-04-25T00:33:07.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 -- 172.21.15.29:0/3974227153 shutdown_connections 2024-04-25T00:33:07.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 --2- 172.21.15.29:0/3974227153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ec0b4b10 0x7fc6ec0ce240 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 --2- 172.21.15.29:0/3974227153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6ec0b3c30 0x7fc6ec06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 -- 172.21.15.29:0/3974227153 >> 172.21.15.29:0/3974227153 conn(0x7fc6ec0b1870 msgr2=0x7fc6ec0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:07.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 -- 172.21.15.29:0/3974227153 shutdown_connections 2024-04-25T00:33:07.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 -- 172.21.15.29:0/3974227153 wait complete. 2024-04-25T00:33:07.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 Processor -- start 2024-04-25T00:33:07.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 -- start start 2024-04-25T00:33:07.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6ec0b3140 0x7fc6ec13a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ec0b3c30 0x7fc6ec13c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec0b4b10 0x7fc6ec0b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec168d80 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec168c00 con 0x7fc6ec0b3c30 2024-04-25T00:33:07.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6f0a76640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ec168f00 con 0x7fc6ec0b3140 2024-04-25T00:33:07.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6ead9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec0b4b10 0x7fc6ec0b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6ead9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec0b4b10 0x7fc6ec0b0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38636/0 (socket says 172.21.15.29:38636) 2024-04-25T00:33:07.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6ead9d640 1 -- 172.21.15.29:0/2329450212 learned_addr learned my addr 172.21.15.29:0/2329450212 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:07.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.414+0000 7fc6ea59c640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6ec0b3140 0x7fc6ec13a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6e9d9b640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ec0b3c30 0x7fc6ec13c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6ead9d640 1 -- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6ec0b3140 msgr2=0x7fc6ec13a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6ead9d640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6ec0b3140 0x7fc6ec13a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6ead9d640 1 -- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ec0b3c30 msgr2=0x7fc6ec13c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6ead9d640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ec0b3c30 0x7fc6ec13c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6ead9d640 1 -- 172.21.15.29:0/2329450212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec13afc0 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6e9d9b640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ec0b3c30 0x7fc6ec13c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:07.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6ead9d640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec0b4b10 0x7fc6ec0b0240 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40b0040 tx=0x7fc6e4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6daffd640 1 -- 172.21.15.29:0/2329450212 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40b2050 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6daffd640 1 -- 172.21.15.29:0/2329450212 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e40be070 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6f0a76640 1 -- 172.21.15.29:0/2329450212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ec13b260 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6daffd640 1 -- 172.21.15.29:0/2329450212 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40ad870 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6f0a76640 1 -- 172.21.15.29:0/2329450212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ec13b820 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6f0a76640 1 -- 172.21.15.29:0/2329450212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ec069270 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6daffd640 1 -- 172.21.15.29:0/2329450212 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fc6e40ada10 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6daffd640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc6cc067a70 0x7fc6cc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6daffd640 1 -- 172.21.15.29:0/2329450212 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fc6e4120ce0 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6ea59c640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc6cc067a70 0x7fc6cc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.418+0000 7fc6ea59c640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc6cc067a70 0x7fc6cc069f30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec158a10 tx=0x7fc6dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.422+0000 7fc6daffd640 1 -- 172.21.15.29:0/2329450212 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc6e4016560 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.614+0000 7fc6f0a76640 1 -- 172.21.15.29:0/2329450212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6ec159150 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.618+0000 7fc6daffd640 1 -- 172.21.15.29:0/2329450212 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3874 (secure 0 0 0) 0x7fc6e40add00 con 0x7fc6ec0b4b10 2024-04-25T00:33:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:07.621 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2100384916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2100384916},{"type":"v1","addr":"172.21.15.157:6835","nonce":2100384916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4984,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1430513194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1430513194},{"type":"v1","addr":"172.21.15.157:6837","nonce":1430513194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3354831881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3354831881},{"type":"v1","addr":"172.21.15.29:6833","nonce":3354831881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":5667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2707564103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":2707564103},{"type":"v1","addr":"172.21.15.29:6835","nonce":2707564103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-04-25T00:33:07.622 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 18 2024-04-25T00:33:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.618+0000 7fc6f0a76640 1 -- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc6cc067a70 msgr2=0x7fc6cc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.618+0000 7fc6f0a76640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc6cc067a70 0x7fc6cc069f30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec158a10 tx=0x7fc6dc009040 comp rx=0 tx=0).stop 2024-04-25T00:33:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.618+0000 7fc6f0a76640 1 -- 172.21.15.29:0/2329450212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec0b4b10 msgr2=0x7fc6ec0b0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.618+0000 7fc6f0a76640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec0b4b10 0x7fc6ec0b0240 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40b0040 tx=0x7fc6e4019710 comp rx=0 tx=0).stop 2024-04-25T00:33:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.618+0000 7fc6f0a76640 1 -- 172.21.15.29:0/2329450212 shutdown_connections 2024-04-25T00:33:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.618+0000 7fc6f0a76640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6ec0b4b10 0x7fc6ec0b0240 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.618+0000 7fc6f0a76640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc6cc067a70 0x7fc6cc069f30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.618+0000 7fc6f0a76640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ec0b3c30 0x7fc6ec13c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.618+0000 7fc6f0a76640 1 --2- 172.21.15.29:0/2329450212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6ec0b3140 0x7fc6ec13a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.618+0000 7fc6f0a76640 1 -- 172.21.15.29:0/2329450212 >> 172.21.15.29:0/2329450212 conn(0x7fc6ec0b1870 msgr2=0x7fc6ec1447b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:07.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.618+0000 7fc6f0a76640 1 -- 172.21.15.29:0/2329450212 shutdown_connections 2024-04-25T00:33:07.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.618+0000 7fc6f0a76640 1 -- 172.21.15.29:0/2329450212 wait complete. 2024-04-25T00:33:07.641 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:33:07.641 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:33:07.641 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:33:07.641 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:33:07.641 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:33:07.641 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:33:07.642 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:33:07.642 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:33:07.642 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:33:07.642 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:33:07.642 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:33:07.643 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:33:07.643 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:33:07.643 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:33:07.643 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:33:07.643 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:33:07.651 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:33:07.651 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:33:07.653 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:33:07.653 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:33:07.656 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:33:07.656 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:33:07.662 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:33:07.662 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:33:07.698 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:33:07.699 DEBUG:teuthology.parallel:result is None 2024-04-25T00:33:07.699 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:33:07.700 DEBUG:teuthology.parallel:result is None 2024-04-25T00:33:07.705 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:33:07.705 DEBUG:teuthology.parallel:result is None 2024-04-25T00:33:07.706 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:33:07.707 DEBUG:teuthology.parallel:result is None 2024-04-25T00:33:07.707 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:33:07.756 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:33:07.756 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:33:07.778 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-25T00:33:07.780 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-25T00:33:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.818+0000 7f22be053640 1 Processor -- start 2024-04-25T00:33:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.818+0000 7f22be053640 1 -- start start 2024-04-25T00:33:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22be053640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f22b815e1b0 0x7f22b8072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22be053640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22b80ccfd0 0x7f22b806b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22be053640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22b806bb60 con 0x7f22b806b750 2024-04-25T00:33:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22be053640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f22b8161bb0 con 0x7f22b815e1b0 2024-04-25T00:33:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22be053640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f22b80d0a30 con 0x7f22b80ccfd0 2024-04-25T00:33:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f22b815e1b0 0x7f22b8072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f22b815e1b0 0x7f22b8072610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59708/0 (socket says 172.21.15.29:59708) 2024-04-25T00:33:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b77fe640 1 -- 172.21.15.29:0/3717878517 learned_addr learned my addr 172.21.15.29:0/3717878517 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b6ffd640 1 --2- 172.21.15.29:0/3717878517 >> v2:172.21.15.157:3300/0 conn(0x7f22b80ccfd0 0x7f22b806b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b77fe640 1 -- 172.21.15.29:0/3717878517 >> v2:172.21.15.157:3300/0 conn(0x7f22b80ccfd0 msgr2=0x7f22b806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b77fe640 1 --2- 172.21.15.29:0/3717878517 >> v2:172.21.15.157:3300/0 conn(0x7f22b80ccfd0 0x7f22b806b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b77fe640 1 -- 172.21.15.29:0/3717878517 >> v1:172.21.15.29:6789/0 conn(0x7f22b806b750 legacy=0x7f22b806db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b67fc640 1 -- 172.21.15.29:0/3717878517 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4172422451 0 0) 0x7f22b00014a0 con 0x7f22b806b750 2024-04-25T00:33:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b77fe640 1 -- 172.21.15.29:0/3717878517 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b80cfcb0 con 0x7f22b815e1b0 2024-04-25T00:33:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b6ffd640 1 --2- 172.21.15.29:0/3717878517 >> v2:172.21.15.157:3300/0 conn(0x7f22b80ccfd0 0x7f22b806b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b77fe640 1 --2- 172.21.15.29:0/3717878517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22b815e1b0 0x7f22b8072610 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f22b8067f70 tx=0x7f22a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59e94db0606d5f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b67fc640 1 -- 172.21.15.29:0/3717878517 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a80b0070 con 0x7f22b815e1b0 2024-04-25T00:33:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b67fc640 1 -- 172.21.15.29:0/3717878517 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a80ab600 con 0x7f22b815e1b0 2024-04-25T00:33:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22b67fc640 1 -- 172.21.15.29:0/3717878517 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a80ab9d0 con 0x7f22b815e1b0 2024-04-25T00:33:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22be053640 1 -- 172.21.15.29:0/3717878517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22b815e1b0 msgr2=0x7f22b8072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22be053640 1 --2- 172.21.15.29:0/3717878517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22b815e1b0 0x7f22b8072610 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f22b8067f70 tx=0x7f22a801c520 comp rx=0 tx=0).stop 2024-04-25T00:33:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22be053640 1 -- 172.21.15.29:0/3717878517 shutdown_connections 2024-04-25T00:33:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22be053640 1 --2- 172.21.15.29:0/3717878517 >> v2:172.21.15.157:3300/0 conn(0x7f22b80ccfd0 0x7f22b806b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22be053640 1 --2- 172.21.15.29:0/3717878517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22b815e1b0 0x7f22b8072610 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22be053640 1 -- 172.21.15.29:0/3717878517 >> 172.21.15.29:0/3717878517 conn(0x7f22b80b2920 msgr2=0x7f22b80b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22be053640 1 -- 172.21.15.29:0/3717878517 shutdown_connections 2024-04-25T00:33:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.822+0000 7f22be053640 1 -- 172.21.15.29:0/3717878517 wait complete. 2024-04-25T00:33:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22be053640 1 Processor -- start 2024-04-25T00:33:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22be053640 1 -- start start 2024-04-25T00:33:07.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22be053640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b806b750 0x7f22b80cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22be053640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22b80ccfd0 0x7f22b8152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22be053640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22b815e1b0 0x7f22b80ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22be053640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f22b8072b50 con 0x7f22b80ccfd0 2024-04-25T00:33:07.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22be053640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f22b80d0510 con 0x7f22b806b750 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22be053640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f22b816af50 con 0x7f22b815e1b0 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22b6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22b80ccfd0 0x7f22b8152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22b6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22b80ccfd0 0x7f22b8152a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59712/0 (socket says 172.21.15.29:59712) 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22b6ffd640 1 -- 172.21.15.29:0/303445481 learned_addr learned my addr 172.21.15.29:0/303445481 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22b77fe640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b806b750 0x7f22b80cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22b7fff640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22b815e1b0 0x7f22b80ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22b6ffd640 1 -- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22b815e1b0 msgr2=0x7f22b80ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22b6ffd640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22b815e1b0 0x7f22b80ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22b6ffd640 1 -- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b806b750 msgr2=0x7f22b80cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22b6ffd640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b806b750 0x7f22b80cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22b6ffd640 1 -- 172.21.15.29:0/303445481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b8153710 con 0x7f22b80ccfd0 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22b6ffd640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22b80ccfd0 0x7f22b8152a70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c7c0 tx=0x7f22ac00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f229ffff640 1 -- 172.21.15.29:0/303445481 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac007a30 con 0x7f22b80ccfd0 2024-04-25T00:33:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f229ffff640 1 -- 172.21.15.29:0/303445481 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ac007bd0 con 0x7f22b80ccfd0 2024-04-25T00:33:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22be053640 1 -- 172.21.15.29:0/303445481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b81539f0 con 0x7f22b80ccfd0 2024-04-25T00:33:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f229ffff640 1 -- 172.21.15.29:0/303445481 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac0113f0 con 0x7f22b80ccfd0 2024-04-25T00:33:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22be053640 1 -- 172.21.15.29:0/303445481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b8153fb0 con 0x7f22b80ccfd0 2024-04-25T00:33:07.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.826+0000 7f22be053640 1 -- 172.21.15.29:0/303445481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b8069270 con 0x7f22b80ccfd0 2024-04-25T00:33:07.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.830+0000 7f229ffff640 1 -- 172.21.15.29:0/303445481 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f22ac011590 con 0x7f22b80ccfd0 2024-04-25T00:33:07.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.830+0000 7f229ffff640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f22a4067aa0 0x7f22a4069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.830+0000 7f229ffff640 1 -- 172.21.15.29:0/303445481 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f22ac085fa0 con 0x7f22b80ccfd0 2024-04-25T00:33:07.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.830+0000 7f22b77fe640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f22a4067aa0 0x7f22a4069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.830+0000 7f22b77fe640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f22a4067aa0 0x7f22a4069f60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f22a80ad000 tx=0x7f22a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:07.834+0000 7f229ffff640 1 -- 172.21.15.29:0/303445481 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f22ac015020 con 0x7f22b80ccfd0 2024-04-25T00:33:08.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:08.002+0000 7f22be053640 1 -- 172.21.15.29:0/303445481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f22b80721c0 con 0x7f22b80ccfd0 2024-04-25T00:33:09.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.090+0000 7f229ffff640 1 -- 172.21.15.29:0/303445481 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) ==== 97+0+0 (secure 0 0 0) 0x7f22ac056360 con 0x7f22b80ccfd0 2024-04-25T00:33:09.095 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-04-25T00:33:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.094+0000 7f22be053640 1 -- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f22a4067aa0 msgr2=0x7f22a4069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.094+0000 7f22be053640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f22a4067aa0 0x7f22a4069f60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f22a80ad000 tx=0x7f22a8008040 comp rx=0 tx=0).stop 2024-04-25T00:33:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.094+0000 7f22be053640 1 -- 172.21.15.29:0/303445481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22b80ccfd0 msgr2=0x7f22b8152a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.094+0000 7f22be053640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22b80ccfd0 0x7f22b8152a70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c7c0 tx=0x7f22ac00cc80 comp rx=0 tx=0).stop 2024-04-25T00:33:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.094+0000 7f22be053640 1 -- 172.21.15.29:0/303445481 shutdown_connections 2024-04-25T00:33:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.094+0000 7f22be053640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22b815e1b0 0x7f22b80ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.094+0000 7f22be053640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22b80ccfd0 0x7f22b8152a70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.094+0000 7f22be053640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f22a4067aa0 0x7f22a4069f60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.094+0000 7f22be053640 1 --2- 172.21.15.29:0/303445481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b806b750 0x7f22b80cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.094+0000 7f22be053640 1 -- 172.21.15.29:0/303445481 >> 172.21.15.29:0/303445481 conn(0x7f22b80b2920 msgr2=0x7f22b80b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:09.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.094+0000 7f22be053640 1 -- 172.21.15.29:0/303445481 shutdown_connections 2024-04-25T00:33:09.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.094+0000 7f22be053640 1 -- 172.21.15.29:0/303445481 wait complete. 2024-04-25T00:33:09.120 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:33:09.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.242+0000 7f688423e640 1 Processor -- start 2024-04-25T00:33:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.242+0000 7f688423e640 1 -- start start 2024-04-25T00:33:09.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f688423e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f687c15e2a0 0x7f687c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f688423e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f687c0cf8a0 con 0x7f687c0d4d10 2024-04-25T00:33:09.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f688423e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f687c0cfb80 con 0x7f687c0b4040 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f688423e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f687c0d19a0 con 0x7f687c15e2a0 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f68817d9640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f687c0d4d10 0x7f687c0d2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38770/0 (socket says 172.21.15.29:38770) 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f68817d9640 1 -- 172.21.15.29:0/233858124 learned_addr learned my addr 172.21.15.29:0/233858124 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6881fda640 1 --2- 172.21.15.29:0/233858124 >> v2:172.21.15.157:3300/0 conn(0x7f687c15e2a0 0x7f687c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4162887811 0 0) 0x7f687c0cf8a0 con 0x7f687c0d4d10 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6868003530 con 0x7f687c0d4d10 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3396770862 0 0) 0x7f6868003530 con 0x7f687c0d4d10 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 >> v2:172.21.15.157:3300/0 conn(0x7f687c15e2a0 msgr2=0x7f687c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 --2- 172.21.15.29:0/233858124 >> v2:172.21.15.157:3300/0 conn(0x7f687c15e2a0 0x7f687c0d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 >> v1:172.21.15.157:6789/0 conn(0x7f687c0b4040 legacy=0x7f687c0b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c0cfe60 con 0x7f687c0d4d10 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f686c001680 con 0x7f687c0d4d10 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 >> v1:172.21.15.29:6789/0 conn(0x7f687c0d4d10 legacy=0x7f687c0d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 --2- 172.21.15.29:0/233858124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68680039c0 0x7f6868003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 --2- 172.21.15.29:0/233858124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6868004810 0x7f6868004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 --2- 172.21.15.29:0/233858124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68680051d0 0x7f686800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f686800c140 con 0x7f68680051d0 2024-04-25T00:33:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f686800c2f0 con 0x7f68680039c0 2024-04-25T00:33:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f686800c4f0 con 0x7f6868004810 2024-04-25T00:33:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f68827db640 1 --2- 172.21.15.29:0/233858124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68680051d0 0x7f686800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6881fda640 1 --2- 172.21.15.29:0/233858124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68680039c0 0x7f6868003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f68817d9640 1 --2- 172.21.15.29:0/233858124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6868004810 0x7f6868004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f68827db640 1 -- 172.21.15.29:0/233858124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6868004810 msgr2=0x7f6868004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f68827db640 1 --2- 172.21.15.29:0/233858124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6868004810 0x7f6868004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f68827db640 1 -- 172.21.15.29:0/233858124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68680039c0 msgr2=0x7f6868003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f68827db640 1 --2- 172.21.15.29:0/233858124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68680039c0 0x7f6868003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f68827db640 1 -- 172.21.15.29:0/233858124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6868003530 con 0x7f68680051d0 2024-04-25T00:33:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.246+0000 7f6881fda640 1 --2- 172.21.15.29:0/233858124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68680039c0 0x7f6868003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:09.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f68827db640 1 -- 172.21.15.29:0/233858124 reap_dead start 2024-04-25T00:33:09.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f68827db640 1 --2- 172.21.15.29:0/233858124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68680051d0 0x7f686800b9d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6874001e70 tx=0x7f687401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cf8047caa1a8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:09.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874001a90 con 0x7f68680051d0 2024-04-25T00:33:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6874003bc0 con 0x7f68680051d0 2024-04-25T00:33:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f6880fd8640 1 -- 172.21.15.29:0/233858124 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68740050d0 con 0x7f68680051d0 2024-04-25T00:33:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 -- 172.21.15.29:0/233858124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68680051d0 msgr2=0x7f686800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 --2- 172.21.15.29:0/233858124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68680051d0 0x7f686800b9d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6874001e70 tx=0x7f687401c780 comp rx=0 tx=0).stop 2024-04-25T00:33:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 -- 172.21.15.29:0/233858124 shutdown_connections 2024-04-25T00:33:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 --2- 172.21.15.29:0/233858124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68680051d0 0x7f686800b9d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 -- 172.21.15.29:0/233858124 >> 172.21.15.29:0/233858124 conn(0x7f687c0d4030 msgr2=0x7f687c0d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 -- 172.21.15.29:0/233858124 shutdown_connections 2024-04-25T00:33:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 -- 172.21.15.29:0/233858124 wait complete. 2024-04-25T00:33:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 Processor -- start 2024-04-25T00:33:09.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 -- start start 2024-04-25T00:33:09.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68680051d0 0x7f687c0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c1605c0 0x7f687c153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f687c160b10 0x7f687c160f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f687c0b3bb0 con 0x7f687c1605c0 2024-04-25T00:33:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f687c0b38d0 con 0x7f68680051d0 2024-04-25T00:33:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f688423e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f687c15e680 con 0x7f687c160b10 2024-04-25T00:33:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f68817d9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c1605c0 0x7f687c153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f68817d9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c1605c0 0x7f687c153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59734/0 (socket says 172.21.15.29:59734) 2024-04-25T00:33:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f68817d9640 1 -- 172.21.15.29:0/4026461773 learned_addr learned my addr 172.21.15.29:0/4026461773 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f68827db640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f687c160b10 0x7f687c160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f6881fda640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68680051d0 0x7f687c0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f68817d9640 1 -- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f687c160b10 msgr2=0x7f687c160f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f68817d9640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f687c160b10 0x7f687c160f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f68817d9640 1 -- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68680051d0 msgr2=0x7f687c0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f68817d9640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68680051d0 0x7f687c0cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f68817d9640 1 -- 172.21.15.29:0/4026461773 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c161840 con 0x7f687c1605c0 2024-04-25T00:33:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.250+0000 7f6881fda640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68680051d0 0x7f687c0cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.254+0000 7f68817d9640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c1605c0 0x7f687c153520 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f686c011810 tx=0x7f686c011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.254+0000 7f68727fc640 1 -- 172.21.15.29:0/4026461773 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c010040 con 0x7f687c1605c0 2024-04-25T00:33:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.254+0000 7f68727fc640 1 -- 172.21.15.29:0/4026461773 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686c01a070 con 0x7f687c1605c0 2024-04-25T00:33:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.254+0000 7f68727fc640 1 -- 172.21.15.29:0/4026461773 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c00fd60 con 0x7f687c1605c0 2024-04-25T00:33:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.254+0000 7f688423e640 1 -- 172.21.15.29:0/4026461773 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687c153a60 con 0x7f687c1605c0 2024-04-25T00:33:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.254+0000 7f688423e640 1 -- 172.21.15.29:0/4026461773 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687c0cdac0 con 0x7f687c1605c0 2024-04-25T00:33:09.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.254+0000 7f688423e640 1 -- 172.21.15.29:0/4026461773 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f687c069270 con 0x7f687c1605c0 2024-04-25T00:33:09.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.254+0000 7f68727fc640 1 -- 172.21.15.29:0/4026461773 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f686c01f460 con 0x7f687c1605c0 2024-04-25T00:33:09.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.254+0000 7f68727fc640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6860067aa0 0x7f6860069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.254+0000 7f68727fc640 1 -- 172.21.15.29:0/4026461773 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f686c088c60 con 0x7f687c1605c0 2024-04-25T00:33:09.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.254+0000 7f6881fda640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6860067aa0 0x7f6860069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.254+0000 7f6881fda640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6860067aa0 0x7f6860069f60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6878009a70 tx=0x7f6878008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:09.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.258+0000 7f68727fc640 1 -- 172.21.15.29:0/4026461773 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f687c161840 con 0x7f687c1605c0 2024-04-25T00:33:09.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f688423e640 1 -- 172.21.15.29:0/4026461773 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f687c0b6c50 con 0x7f687c1605c0 2024-04-25T00:33:09.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f68727fc640 1 -- 172.21.15.29:0/4026461773 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f686c059870 con 0x7f687c1605c0 2024-04-25T00:33:09.479 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:09.479 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:33:09.479 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-04-25T00:33:09.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.479 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-04-25T00:33:09.479 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmyzgKAxAAU+nrUJgQhDX6xlARNFI/ng==", 2024-04-25T00:33:09.479 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.479 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:33:09.479 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:33:09.479 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:33:09.479 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.480 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.480 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-04-25T00:33:09.480 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmtyzPERAAxNUDANw1H/ZzOo+dhgzxlg==", 2024-04-25T00:33:09.480 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.480 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:33:09.480 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:33:09.480 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:33:09.480 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.480 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmYUl8CxAAj6pV1+s4a0rpWLq2EgZ+8w==", 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmuAhKGhAAOLIcWUodW5TqtVgHWZl12A==", 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:33:09.481 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAzpClmudk6MhAA6YFyd57jwtB2pdOaad7bxQ==", 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.483 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-04-25T00:33:09.483 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA2pClm76q0NRAAxWhEP2ZGoFnZcuiQyGjglw==", 2024-04-25T00:33:09.483 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.483 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:33:09.483 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:33:09.483 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.483 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.483 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA6pClmcgULABAA2F5WBR9Df0v4Sn875AzF7g==", 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA9pClm9QuRBhAALFLruv4JU/uNZYMX0yNuXw==", 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:33:09.484 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.485 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-04-25T00:33:09.485 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBBpClmNj4VDBAAxaxMvyaCvjynPY3f81udnw==", 2024-04-25T00:33:09.485 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.485 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:33:09.485 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:33:09.485 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.485 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-04-25T00:33:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBEpClmzBybEBAArry1lM4RIsM/txKfgBqDpw==", 2024-04-25T00:33:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:33:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:33:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-04-25T00:33:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBHpClm5HfjFhAAbFwJCpbhegLWDu0mbS8FoA==", 2024-04-25T00:33:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:33:09.486 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:33:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-04-25T00:33:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBKpClme/7oHBAAOR2iQ6CweexR7sC1PgesZw==", 2024-04-25T00:33:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:33:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:33:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-04-25T00:33:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A==", 2024-04-25T00:33:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:33:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-04-25T00:33:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:33:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-04-25T00:33:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm8W/EJxAA0HZW24Hhcwf+soSZn6jHJg==", 2024-04-25T00:33:09.488 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmGuQJJRAAhVd4D0jc6Whgyf+B6xgFMw==", 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:33:09.489 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmn+ciLRAAg4tQuZ13WP0RnWJaJq5fUQ==", 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.490 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmPVihJBAA7PQge/teyRZKSkBnufZSZw==", 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClm6WrdOhAAsY9FLCxG7PGpq2NCY4hGhQ==", 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:33:09.491 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmeR4UOBAANs9SRolOm6E8+/4ciupk5w==", 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:33:09.492 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:33:09.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f688423e640 1 -- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6860067aa0 msgr2=0x7f6860069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f688423e640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6860067aa0 0x7f6860069f60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6878009a70 tx=0x7f6878008040 comp rx=0 tx=0).stop 2024-04-25T00:33:09.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f688423e640 1 -- 172.21.15.29:0/4026461773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c1605c0 msgr2=0x7f687c153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f688423e640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c1605c0 0x7f687c153520 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f686c011810 tx=0x7f686c011cd0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f688423e640 1 -- 172.21.15.29:0/4026461773 shutdown_connections 2024-04-25T00:33:09.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f688423e640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f687c160b10 0x7f687c160f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f688423e640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c1605c0 0x7f687c153520 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f688423e640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6860067aa0 0x7f6860069f60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f688423e640 1 --2- 172.21.15.29:0/4026461773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68680051d0 0x7f687c0cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f688423e640 1 -- 172.21.15.29:0/4026461773 >> 172.21.15.29:0/4026461773 conn(0x7f687c0d4030 msgr2=0x7f687c06cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:09.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f688423e640 1 -- 172.21.15.29:0/4026461773 shutdown_connections 2024-04-25T00:33:09.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.474+0000 7f688423e640 1 -- 172.21.15.29:0/4026461773 wait complete. 2024-04-25T00:33:09.497 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:33:09.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac969fc640 1 Processor -- start 2024-04-25T00:33:09.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac969fc640 1 -- start start 2024-04-25T00:33:09.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac969fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fac900cf690 0x7fac900d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac969fc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac900d2ea0 con 0x7fac900d2a90 2024-04-25T00:33:09.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac969fc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac900cfc10 con 0x7fac9015e200 2024-04-25T00:33:09.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac969fc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fac900b38d0 con 0x7fac900cf690 2024-04-25T00:33:09.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac94f99640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fac900d2a90 0x7fac900b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38778/0 (socket says 172.21.15.29:38778) 2024-04-25T00:33:09.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac94f99640 1 -- 172.21.15.29:0/1495855659 learned_addr learned my addr 172.21.15.29:0/1495855659 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:09.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8f7fe640 1 --2- 172.21.15.29:0/1495855659 >> v2:172.21.15.157:3300/0 conn(0x7fac900cf690 0x7fac900d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1198535131 0 0) 0x7fac880014a0 con 0x7fac900d2a90 2024-04-25T00:33:09.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac78003530 con 0x7fac900d2a90 2024-04-25T00:33:09.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2811475787 0 0) 0x7fac78003530 con 0x7fac900d2a90 2024-04-25T00:33:09.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 >> v2:172.21.15.157:3300/0 conn(0x7fac900cf690 msgr2=0x7fac900d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:09.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 --2- 172.21.15.29:0/1495855659 >> v2:172.21.15.157:3300/0 conn(0x7fac900cf690 0x7fac900d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 >> v1:172.21.15.157:6789/0 conn(0x7fac9015e200 legacy=0x7fac900d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac900cfef0 con 0x7fac900d2a90 2024-04-25T00:33:09.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fac88001bd0 con 0x7fac900d2a90 2024-04-25T00:33:09.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 >> v1:172.21.15.29:6789/0 conn(0x7fac900d2a90 legacy=0x7fac900b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 --2- 172.21.15.29:0/1495855659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac780039c0 0x7fac78003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 --2- 172.21.15.29:0/1495855659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac78004810 0x7fac78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 --2- 172.21.15.29:0/1495855659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac780051d0 0x7fac7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac900b38d0 con 0x7fac780039c0 2024-04-25T00:33:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac7800c070 con 0x7fac78004810 2024-04-25T00:33:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac7800c270 con 0x7fac780051d0 2024-04-25T00:33:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8ffff640 1 --2- 172.21.15.29:0/1495855659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac780039c0 0x7fac78003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8f7fe640 1 --2- 172.21.15.29:0/1495855659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac78004810 0x7fac78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac94f99640 1 --2- 172.21.15.29:0/1495855659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac780051d0 0x7fac7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8ffff640 1 -- 172.21.15.29:0/1495855659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac780051d0 msgr2=0x7fac7800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8ffff640 1 --2- 172.21.15.29:0/1495855659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac780051d0 0x7fac7800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8ffff640 1 -- 172.21.15.29:0/1495855659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac78004810 msgr2=0x7fac78004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8ffff640 1 --2- 172.21.15.29:0/1495855659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac78004810 0x7fac78004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac94f99640 1 -- 172.21.15.29:0/1495855659 reap_dead start 2024-04-25T00:33:09.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.606+0000 7fac8ffff640 1 -- 172.21.15.29:0/1495855659 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac78003530 con 0x7fac780039c0 2024-04-25T00:33:09.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.610+0000 7fac8ffff640 1 --2- 172.21.15.29:0/1495855659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac780039c0 0x7fac78003de0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fac80001780 tx=0x7fac8001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3837b7f7c33fd2a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:09.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.610+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac800013a0 con 0x7fac780039c0 2024-04-25T00:33:09.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.610+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac800052f0 con 0x7fac780039c0 2024-04-25T00:33:09.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.610+0000 7fac8effd640 1 -- 172.21.15.29:0/1495855659 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac80003840 con 0x7fac780039c0 2024-04-25T00:33:09.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.610+0000 7fac969fc640 1 -- 172.21.15.29:0/1495855659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac780039c0 msgr2=0x7fac78003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.610+0000 7fac969fc640 1 --2- 172.21.15.29:0/1495855659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac780039c0 0x7fac78003de0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fac80001780 tx=0x7fac8001c3f0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.610+0000 7fac969fc640 1 -- 172.21.15.29:0/1495855659 shutdown_connections 2024-04-25T00:33:09.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.610+0000 7fac969fc640 1 --2- 172.21.15.29:0/1495855659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac780039c0 0x7fac78003de0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.610+0000 7fac969fc640 1 -- 172.21.15.29:0/1495855659 >> 172.21.15.29:0/1495855659 conn(0x7fac9009d280 msgr2=0x7fac900d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:09.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.610+0000 7fac969fc640 1 -- 172.21.15.29:0/1495855659 shutdown_connections 2024-04-25T00:33:09.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.610+0000 7fac969fc640 1 -- 172.21.15.29:0/1495855659 wait complete. 2024-04-25T00:33:09.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac969fc640 1 Processor -- start 2024-04-25T00:33:09.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac969fc640 1 -- start start 2024-04-25T00:33:09.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac969fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac780039c0 0x7fac901f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac969fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9015a060 0x7fac9015f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac969fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac9015fd80 0x7fac9015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac969fc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac900d0720 con 0x7fac780039c0 2024-04-25T00:33:09.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac969fc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac900d05a0 con 0x7fac9015fd80 2024-04-25T00:33:09.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac969fc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac900d08a0 con 0x7fac9015a060 2024-04-25T00:33:09.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac780039c0 0x7fac901f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac780039c0 0x7fac901f76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59752/0 (socket says 172.21.15.29:59752) 2024-04-25T00:33:09.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8ffff640 1 -- 172.21.15.29:0/806403433 learned_addr learned my addr 172.21.15.29:0/806403433 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:09.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8f7fe640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9015a060 0x7fac9015f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac94f99640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac9015fd80 0x7fac9015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8ffff640 1 -- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9015a060 msgr2=0x7fac9015f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8ffff640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9015a060 0x7fac9015f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8ffff640 1 -- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac9015fd80 msgr2=0x7fac9015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8ffff640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac9015fd80 0x7fac9015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8ffff640 1 -- 172.21.15.29:0/806403433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac90160290 con 0x7fac780039c0 2024-04-25T00:33:09.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8f7fe640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9015a060 0x7fac9015f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:09.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8ffff640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac780039c0 0x7fac901f76d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fac8001c220 tx=0x7fac78005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:09.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8cff9640 1 -- 172.21.15.29:0/806403433 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac800b8070 con 0x7fac780039c0 2024-04-25T00:33:09.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8cff9640 1 -- 172.21.15.29:0/806403433 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac80000ee0 con 0x7fac780039c0 2024-04-25T00:33:09.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac8cff9640 1 -- 172.21.15.29:0/806403433 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac80007a40 con 0x7fac780039c0 2024-04-25T00:33:09.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac969fc640 1 -- 172.21.15.29:0/806403433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac90160570 con 0x7fac780039c0 2024-04-25T00:33:09.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.614+0000 7fac969fc640 1 -- 172.21.15.29:0/806403433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac901f7c10 con 0x7fac780039c0 2024-04-25T00:33:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.618+0000 7fac969fc640 1 -- 172.21.15.29:0/806403433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac901f6ce0 con 0x7fac780039c0 2024-04-25T00:33:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.618+0000 7fac8cff9640 1 -- 172.21.15.29:0/806403433 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fac800052d0 con 0x7fac780039c0 2024-04-25T00:33:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.618+0000 7fac8cff9640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fac7c067a70 0x7fac7c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.618+0000 7fac8cff9640 1 -- 172.21.15.29:0/806403433 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7fac8011f860 con 0x7fac780039c0 2024-04-25T00:33:09.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.618+0000 7fac8f7fe640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fac7c067a70 0x7fac7c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.618+0000 7fac8f7fe640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fac7c067a70 0x7fac7c069f30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fac900cbc90 tx=0x7fac84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:09.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.622+0000 7fac8cff9640 1 -- 172.21.15.29:0/806403433 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fac80016130 con 0x7fac780039c0 2024-04-25T00:33:09.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.786+0000 7fac969fc640 1 -- 172.21.15.29:0/806403433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac90069270 con 0x7fac780039c0 2024-04-25T00:33:09.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.786+0000 7fac8cff9640 1 -- 172.21.15.29:0/806403433 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+12843 (secure 0 0 0) 0x7fac800f03f0 con 0x7fac780039c0 2024-04-25T00:33:09.792 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:09.793 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":41,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:33:09.092669+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:33:09.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.790+0000 7fac969fc640 1 -- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fac7c067a70 msgr2=0x7fac7c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.790+0000 7fac969fc640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fac7c067a70 0x7fac7c069f30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fac900cbc90 tx=0x7fac84008040 comp rx=0 tx=0).stop 2024-04-25T00:33:09.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.790+0000 7fac969fc640 1 -- 172.21.15.29:0/806403433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac780039c0 msgr2=0x7fac901f76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.790+0000 7fac969fc640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac780039c0 0x7fac901f76d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fac8001c220 tx=0x7fac78005840 comp rx=0 tx=0).stop 2024-04-25T00:33:09.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.790+0000 7fac969fc640 1 -- 172.21.15.29:0/806403433 shutdown_connections 2024-04-25T00:33:09.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.790+0000 7fac969fc640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fac7c067a70 0x7fac7c069f30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.790+0000 7fac969fc640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac9015fd80 0x7fac9015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.790+0000 7fac969fc640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac9015a060 0x7fac9015f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.790+0000 7fac969fc640 1 --2- 172.21.15.29:0/806403433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac780039c0 0x7fac901f76d0 secure :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fac8001c220 tx=0x7fac78005840 comp rx=0 tx=0).stop 2024-04-25T00:33:09.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.790+0000 7fac969fc640 1 -- 172.21.15.29:0/806403433 >> 172.21.15.29:0/806403433 conn(0x7fac9009d280 msgr2=0x7fac900d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:09.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.790+0000 7fac969fc640 1 -- 172.21.15.29:0/806403433 shutdown_connections 2024-04-25T00:33:09.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.790+0000 7fac969fc640 1 -- 172.21.15.29:0/806403433 wait complete. 2024-04-25T00:33:09.811 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:33:09.811 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:33:09.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f550315a640 1 Processor -- start 2024-04-25T00:33:09.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f550315a640 1 -- start start 2024-04-25T00:33:09.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f550315a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f54fc06e440 0x7f54fc0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f550315a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54fc0b41e0 con 0x7f54fc0b3c30 2024-04-25T00:33:09.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f550315a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54fc06d960 con 0x7f54fc0b3140 2024-04-25T00:33:09.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f550315a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f54fc0b5050 con 0x7f54fc06e440 2024-04-25T00:33:09.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f55016f7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f54fc06e440 0x7f54fc0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f54fbfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f54fc0b3c30 0x7f54fc0ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38794/0 (socket says 172.21.15.29:38794) 2024-04-25T00:33:09.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f54fbfff640 1 -- 172.21.15.29:0/1475656155 learned_addr learned my addr 172.21.15.29:0/1475656155 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:09.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f55016f7640 1 -- 172.21.15.29:0/1475656155 >> v1:172.21.15.157:6789/0 conn(0x7f54fc0b3140 legacy=0x7f54fc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:09.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f55016f7640 1 -- 172.21.15.29:0/1475656155 >> v1:172.21.15.29:6789/0 conn(0x7f54fc0b3c30 legacy=0x7f54fc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f55016f7640 1 -- 172.21.15.29:0/1475656155 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc06dc40 con 0x7f54fc06e440 2024-04-25T00:33:09.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f55016f7640 1 --2- 172.21.15.29:0/1475656155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54fc06e440 0x7f54fc0b4b10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009dc0 tx=0x7f54f401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94eff33f56d8332c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:09.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f54fb7fe640 1 -- 172.21.15.29:0/1475656155 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f40b2070 con 0x7f54fc06e440 2024-04-25T00:33:09.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f54fb7fe640 1 -- 172.21.15.29:0/1475656155 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f40a57d0 con 0x7f54fc06e440 2024-04-25T00:33:09.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.922+0000 7f54fb7fe640 1 -- 172.21.15.29:0/1475656155 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f40a5ba0 con 0x7f54fc06e440 2024-04-25T00:33:09.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 -- 172.21.15.29:0/1475656155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54fc06e440 msgr2=0x7f54fc0b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 --2- 172.21.15.29:0/1475656155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54fc06e440 0x7f54fc0b4b10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009dc0 tx=0x7f54f401c910 comp rx=0 tx=0).stop 2024-04-25T00:33:09.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 -- 172.21.15.29:0/1475656155 shutdown_connections 2024-04-25T00:33:09.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 --2- 172.21.15.29:0/1475656155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54fc06e440 0x7f54fc0b4b10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 -- 172.21.15.29:0/1475656155 >> 172.21.15.29:0/1475656155 conn(0x7f54fc0b1870 msgr2=0x7f54fc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:09.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 -- 172.21.15.29:0/1475656155 shutdown_connections 2024-04-25T00:33:09.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 -- 172.21.15.29:0/1475656155 wait complete. 2024-04-25T00:33:09.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 Processor -- start 2024-04-25T00:33:09.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 -- start start 2024-04-25T00:33:09.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc06e440 0x7f54fc13fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc0b3140 0x7f54fc0d0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54fc0b3c30 0x7f54fc139550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc169d90 con 0x7f54fc0b3c30 2024-04-25T00:33:09.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc169c10 con 0x7f54fc06e440 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f54fc169f10 con 0x7f54fc0b3140 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f55016f7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54fc0b3c30 0x7f54fc139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f55016f7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54fc0b3c30 0x7f54fc139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59776/0 (socket says 172.21.15.29:59776) 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f55016f7640 1 -- 172.21.15.29:0/3634086879 learned_addr learned my addr 172.21.15.29:0/3634086879 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f5500ef6640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc06e440 0x7f54fc13fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f54fbfff640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc0b3140 0x7f54fc0d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f55016f7640 1 -- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc0b3140 msgr2=0x7f54fc0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f55016f7640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc0b3140 0x7f54fc0d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f55016f7640 1 -- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc06e440 msgr2=0x7f54fc13fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f55016f7640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc06e440 0x7f54fc13fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f55016f7640 1 -- 172.21.15.29:0/3634086879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc166ff0 con 0x7f54fc0b3c30 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f54fbfff640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc0b3140 0x7f54fc0d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f55016f7640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54fc0b3c30 0x7f54fc139550 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f54f40b0040 tx=0x7f54f40196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:09.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f54f97fa640 1 -- 172.21.15.29:0/3634086879 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f40bd020 con 0x7f54fc0b3c30 2024-04-25T00:33:09.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f54f97fa640 1 -- 172.21.15.29:0/3634086879 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f40b8670 con 0x7f54fc0b3c30 2024-04-25T00:33:09.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 -- 172.21.15.29:0/3634086879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54fc1672d0 con 0x7f54fc0b3c30 2024-04-25T00:33:09.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f54f97fa640 1 -- 172.21.15.29:0/3634086879 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f40b2040 con 0x7f54fc0b3c30 2024-04-25T00:33:09.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.926+0000 7f550315a640 1 -- 172.21.15.29:0/3634086879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54fc13f7b0 con 0x7f54fc0b3c30 2024-04-25T00:33:09.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.930+0000 7f550315a640 1 -- 172.21.15.29:0/3634086879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54fc069270 con 0x7f54fc0b3c30 2024-04-25T00:33:09.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.930+0000 7f54f97fa640 1 -- 172.21.15.29:0/3634086879 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f54f40c0020 con 0x7f54fc0b3c30 2024-04-25T00:33:09.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.930+0000 7f54f97fa640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f54e8067a70 0x7f54e8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:09.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.930+0000 7f54f97fa640 1 -- 172.21.15.29:0/3634086879 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f54f4120b50 con 0x7f54fc0b3c30 2024-04-25T00:33:09.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.930+0000 7f5500ef6640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f54e8067a70 0x7f54e8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:09.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.930+0000 7f5500ef6640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f54e8067a70 0x7f54e8069f30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f54ec004e30 tx=0x7f54ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:09.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:09.934+0000 7f54f97fa640 1 -- 172.21.15.29:0/3634086879 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f54f40c7030 con 0x7f54fc0b3c30 2024-04-25T00:33:09.972 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:33:09.975 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:33:10.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.094+0000 7f550315a640 1 -- 172.21.15.29:0/3634086879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f54fc0675c0 con 0x7f54fc0b3c30 2024-04-25T00:33:10.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.094+0000 7f54f97fa640 1 -- 172.21.15.29:0/3634086879 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f54f4016500 con 0x7f54fc0b3c30 2024-04-25T00:33:10.099 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:33:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.098+0000 7f550315a640 1 -- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f54e8067a70 msgr2=0x7f54e8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.098+0000 7f550315a640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f54e8067a70 0x7f54e8069f30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f54ec004e30 tx=0x7f54ec009040 comp rx=0 tx=0).stop 2024-04-25T00:33:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.098+0000 7f550315a640 1 -- 172.21.15.29:0/3634086879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54fc0b3c30 msgr2=0x7f54fc139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.098+0000 7f550315a640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54fc0b3c30 0x7f54fc139550 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f54f40b0040 tx=0x7f54f40196b0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.098+0000 7f550315a640 1 -- 172.21.15.29:0/3634086879 shutdown_connections 2024-04-25T00:33:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.098+0000 7f550315a640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54fc0b3c30 0x7f54fc139550 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.098+0000 7f550315a640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc0b3140 0x7f54fc0d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.098+0000 7f550315a640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f54e8067a70 0x7f54e8069f30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.098+0000 7f550315a640 1 --2- 172.21.15.29:0/3634086879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc06e440 0x7f54fc13fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.098+0000 7f550315a640 1 -- 172.21.15.29:0/3634086879 >> 172.21.15.29:0/3634086879 conn(0x7f54fc0b1870 msgr2=0x7f54fc0cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.098+0000 7f550315a640 1 -- 172.21.15.29:0/3634086879 shutdown_connections 2024-04-25T00:33:10.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.098+0000 7f550315a640 1 -- 172.21.15.29:0/3634086879 wait complete. 2024-04-25T00:33:10.118 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:33:10.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b8848640 1 Processor -- start 2024-04-25T00:33:10.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b8848640 1 -- start start 2024-04-25T00:33:10.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b8848640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b415e8d0 0x7fd9b40cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b8848640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9b40b4780 0x7fd9b415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b8848640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9b40b3720 con 0x7fd9b415f1f0 2024-04-25T00:33:10.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b8848640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd9b40b42f0 con 0x7fd9b40b4780 2024-04-25T00:33:10.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b8848640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b415d7e0 con 0x7fd9b415e8d0 2024-04-25T00:33:10.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b2d9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd9b415f1f0 0x7fd9b415b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38796/0 (socket says 172.21.15.29:38796) 2024-04-25T00:33:10.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b2d9d640 1 -- 172.21.15.29:0/92210827 learned_addr learned my addr 172.21.15.29:0/92210827 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:10.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b1d9b640 1 --2- 172.21.15.29:0/92210827 >> v2:172.21.15.157:3300/0 conn(0x7fd9b40b4780 0x7fd9b415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b259c640 1 --2- 172.21.15.29:0/92210827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b415e8d0 0x7fd9b40cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2682923544 0 0) 0x7fd9b40b3720 con 0x7fd9b415f1f0 2024-04-25T00:33:10.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd99c003530 con 0x7fd9b415f1f0 2024-04-25T00:33:10.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1626595080 0 0) 0x7fd99c003530 con 0x7fd9b415f1f0 2024-04-25T00:33:10.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b415e8d0 msgr2=0x7fd9b40cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:10.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 --2- 172.21.15.29:0/92210827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b415e8d0 0x7fd9b40cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 >> v2:172.21.15.157:3300/0 conn(0x7fd9b40b4780 msgr2=0x7fd9b415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:10.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 --2- 172.21.15.29:0/92210827 >> v2:172.21.15.157:3300/0 conn(0x7fd9b40b4780 0x7fd9b415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b40b39c0 con 0x7fd9b415f1f0 2024-04-25T00:33:10.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd9ac001a00 con 0x7fd9b415f1f0 2024-04-25T00:33:10.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 >> v1:172.21.15.29:6789/0 conn(0x7fd9b415f1f0 legacy=0x7fd9b415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:10.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 --2- 172.21.15.29:0/92210827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd99c0039c0 0x7fd99c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 --2- 172.21.15.29:0/92210827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd99c004810 0x7fd99c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 --2- 172.21.15.29:0/92210827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b415e8d0 0x7fd99c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b40b42f0 con 0x7fd99c004810 2024-04-25T00:33:10.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b415d7e0 con 0x7fd99c0039c0 2024-04-25T00:33:10.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd99c00b9d0 con 0x7fd9b415e8d0 2024-04-25T00:33:10.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b1d9b640 1 --2- 172.21.15.29:0/92210827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd99c004810 0x7fd99c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b259c640 1 --2- 172.21.15.29:0/92210827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd99c0039c0 0x7fd99c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b2d9d640 1 --2- 172.21.15.29:0/92210827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b415e8d0 0x7fd99c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b1d9b640 1 -- 172.21.15.29:0/92210827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b415e8d0 msgr2=0x7fd99c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:10.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b1d9b640 1 --2- 172.21.15.29:0/92210827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b415e8d0 0x7fd99c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b1d9b640 1 -- 172.21.15.29:0/92210827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd99c0039c0 msgr2=0x7fd99c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:10.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b1d9b640 1 --2- 172.21.15.29:0/92210827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd99c0039c0 0x7fd99c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b1d9b640 1 -- 172.21.15.29:0/92210827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c003530 con 0x7fd99c004810 2024-04-25T00:33:10.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.246+0000 7fd9b2d9d640 1 --2- 172.21.15.29:0/92210827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b415e8d0 0x7fd99c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:10.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.250+0000 7fd9b1d9b640 1 --2- 172.21.15.29:0/92210827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd99c004810 0x7fd99c004c90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009ae0 tx=0x7fd9a801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ff00948104d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:10.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.250+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80096b0 con 0x7fd99c004810 2024-04-25T00:33:10.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.250+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a80a64d0 con 0x7fd99c004810 2024-04-25T00:33:10.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.250+0000 7fd9b159a640 1 -- 172.21.15.29:0/92210827 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80a68c0 con 0x7fd99c004810 2024-04-25T00:33:10.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.250+0000 7fd9b8848640 1 -- 172.21.15.29:0/92210827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd99c004810 msgr2=0x7fd99c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:10.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.250+0000 7fd9b8848640 1 --2- 172.21.15.29:0/92210827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd99c004810 0x7fd99c004c90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009ae0 tx=0x7fd9a801cab0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.250+0000 7fd9b2d9d640 1 -- 172.21.15.29:0/92210827 reap_dead start 2024-04-25T00:33:10.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.250+0000 7fd9b8848640 1 -- 172.21.15.29:0/92210827 shutdown_connections 2024-04-25T00:33:10.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.250+0000 7fd9b8848640 1 -- 172.21.15.29:0/92210827 >> 172.21.15.29:0/92210827 conn(0x7fd9b40b1870 msgr2=0x7fd9b40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:10.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.250+0000 7fd9b8848640 1 -- 172.21.15.29:0/92210827 shutdown_connections 2024-04-25T00:33:10.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.250+0000 7fd9b8848640 1 -- 172.21.15.29:0/92210827 wait complete. 2024-04-25T00:33:10.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.250+0000 7fd9b8848640 1 Processor -- start 2024-04-25T00:33:10.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.250+0000 7fd9b8848640 1 -- start start 2024-04-25T00:33:10.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b8848640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd99c004810 0x7fd9b41f8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b8848640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b413c940 0x7fd9b413cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b8848640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9b4071640 0x7fd9b4071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b8848640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b406cf10 con 0x7fd9b4071640 2024-04-25T00:33:10.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b8848640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b40b3f80 con 0x7fd99c004810 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b8848640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b415f660 con 0x7fd9b413c940 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9b4071640 0x7fd9b4071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9b4071640 0x7fd9b4071af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59796/0 (socket says 172.21.15.29:59796) 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b2d9d640 1 -- 172.21.15.29:0/1814103076 learned_addr learned my addr 172.21.15.29:0/1814103076 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b1d9b640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b413c940 0x7fd9b413cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b259c640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd99c004810 0x7fd9b41f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b2d9d640 1 -- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b413c940 msgr2=0x7fd9b413cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b2d9d640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b413c940 0x7fd9b413cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b2d9d640 1 -- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd99c004810 msgr2=0x7fd9b41f8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b2d9d640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd99c004810 0x7fd9b41f8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b2d9d640 1 -- 172.21.15.29:0/1814103076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b4071190 con 0x7fd9b4071640 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b2d9d640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9b4071640 0x7fd9b4071af0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40709b0 tx=0x7fd9ac010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9a2ffd640 1 -- 172.21.15.29:0/1814103076 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac01a070 con 0x7fd9b4071640 2024-04-25T00:33:10.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9a2ffd640 1 -- 172.21.15.29:0/1814103076 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9ac011040 con 0x7fd9b4071640 2024-04-25T00:33:10.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b8848640 1 -- 172.21.15.29:0/1814103076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b413d300 con 0x7fd9b4071640 2024-04-25T00:33:10.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9a2ffd640 1 -- 172.21.15.29:0/1814103076 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00ecc0 con 0x7fd9b4071640 2024-04-25T00:33:10.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b8848640 1 -- 172.21.15.29:0/1814103076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b41f5390 con 0x7fd9b4071640 2024-04-25T00:33:10.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b8848640 1 -- 172.21.15.29:0/1814103076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b4069270 con 0x7fd9b4071640 2024-04-25T00:33:10.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9a2ffd640 1 -- 172.21.15.29:0/1814103076 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fd9ac01d070 con 0x7fd9b4071640 2024-04-25T00:33:10.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9a2ffd640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd994067a70 0x7fd994069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9a2ffd640 1 -- 172.21.15.29:0/1814103076 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7fd9ac088360 con 0x7fd9b4071640 2024-04-25T00:33:10.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.254+0000 7fd9b259c640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd994067a70 0x7fd994069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.258+0000 7fd9b259c640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd994067a70 0x7fd994069f30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd9b406d9b0 tx=0x7fd9a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:10.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.258+0000 7fd9a2ffd640 1 -- 172.21.15.29:0/1814103076 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd9ac00ee60 con 0x7fd9b4071640 2024-04-25T00:33:10.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.422+0000 7fd9b8848640 1 -- 172.21.15.29:0/1814103076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd9b40b15e0 con 0x7fd9b4071640 2024-04-25T00:33:10.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.426+0000 7fd9a2ffd640 1 -- 172.21.15.29:0/1814103076 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) ==== 84+0+65 (secure 0 0 0) 0x7fd9ac058920 con 0x7fd9b4071640 2024-04-25T00:33:10.430 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:10.430 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:33:10.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:10.430 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-04-25T00:33:10.430 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": ".mgr" 2024-04-25T00:33:10.430 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:10.430 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:33:10.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.426+0000 7fd9b8848640 1 -- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd994067a70 msgr2=0x7fd994069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:10.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.426+0000 7fd9b8848640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd994067a70 0x7fd994069f30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd9b406d9b0 tx=0x7fd9a4008040 comp rx=0 tx=0).stop 2024-04-25T00:33:10.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.426+0000 7fd9b8848640 1 -- 172.21.15.29:0/1814103076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9b4071640 msgr2=0x7fd9b4071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:10.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.426+0000 7fd9b8848640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9b4071640 0x7fd9b4071af0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40709b0 tx=0x7fd9ac010d00 comp rx=0 tx=0).stop 2024-04-25T00:33:10.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.426+0000 7fd9b8848640 1 -- 172.21.15.29:0/1814103076 shutdown_connections 2024-04-25T00:33:10.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.426+0000 7fd9b8848640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9b4071640 0x7fd9b4071af0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.426+0000 7fd9b8848640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9b413c940 0x7fd9b413cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.426+0000 7fd9b8848640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd994067a70 0x7fd994069f30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.426+0000 7fd9b8848640 1 --2- 172.21.15.29:0/1814103076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd99c004810 0x7fd9b41f8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.426+0000 7fd9b8848640 1 -- 172.21.15.29:0/1814103076 >> 172.21.15.29:0/1814103076 conn(0x7fd9b40b1870 msgr2=0x7fd9b413e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:10.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.426+0000 7fd9b8848640 1 -- 172.21.15.29:0/1814103076 shutdown_connections 2024-04-25T00:33:10.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.426+0000 7fd9b8848640 1 -- 172.21.15.29:0/1814103076 wait complete. 2024-04-25T00:33:10.448 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:33:10.448 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:33:10.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.570+0000 7fd1ec60f640 1 Processor -- start 2024-04-25T00:33:10.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.570+0000 7fd1ec60f640 1 -- start start 2024-04-25T00:33:10.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.570+0000 7fd1ec60f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1e40b4590 0x7fd1e4070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.570+0000 7fd1ec60f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1e40712c0 con 0x7fd1e40cd1c0 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.570+0000 7fd1ec60f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1e40715a0 con 0x7fd1e415e9a0 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.570+0000 7fd1ec60f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e4071ed0 con 0x7fd1e40b4590 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.570+0000 7fd1eabac640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd1e40cd1c0 0x7fd1e4072360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38802/0 (socket says 172.21.15.29:38802) 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.570+0000 7fd1eabac640 1 -- 172.21.15.29:0/2849555374 learned_addr learned my addr 172.21.15.29:0/2849555374 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e9baa640 1 --2- 172.21.15.29:0/2849555374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1e40b4590 0x7fd1e4070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3915100639 0 0) 0x7fd1e40712c0 con 0x7fd1e40cd1c0 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1d0005740 con 0x7fd1e40cd1c0 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4063605294 0 0) 0x7fd1d0005740 con 0x7fd1e40cd1c0 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1e40b4590 msgr2=0x7fd1e4070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 --2- 172.21.15.29:0/2849555374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1e40b4590 0x7fd1e4070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 >> v1:172.21.15.157:6789/0 conn(0x7fd1e415e9a0 legacy=0x7fd1e415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e4071880 con 0x7fd1e40cd1c0 2024-04-25T00:33:10.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd1dc001a00 con 0x7fd1e40cd1c0 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 >> v1:172.21.15.29:6789/0 conn(0x7fd1e40cd1c0 legacy=0x7fd1e4072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 --2- 172.21.15.29:0/2849555374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1e40b4590 0x7fd1d0005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 --2- 172.21.15.29:0/2849555374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1d0006180 0x7fd1d0006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 --2- 172.21.15.29:0/2849555374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1d0006f60 0x7fd1d00073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e4071ed0 con 0x7fd1d0006180 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d000dcc0 con 0x7fd1d0006f60 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d000dec0 con 0x7fd1e40b4590 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e9baa640 1 --2- 172.21.15.29:0/2849555374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1d0006180 0x7fd1d0006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1ea3ab640 1 --2- 172.21.15.29:0/2849555374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1e40b4590 0x7fd1d0005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1eabac640 1 --2- 172.21.15.29:0/2849555374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1d0006f60 0x7fd1d00073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e9baa640 1 -- 172.21.15.29:0/2849555374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1e40b4590 msgr2=0x7fd1d0005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e9baa640 1 --2- 172.21.15.29:0/2849555374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1e40b4590 0x7fd1d0005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e9baa640 1 -- 172.21.15.29:0/2849555374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1d0006f60 msgr2=0x7fd1d00073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e9baa640 1 --2- 172.21.15.29:0/2849555374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1d0006f60 0x7fd1d00073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e9baa640 1 -- 172.21.15.29:0/2849555374 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d0005740 con 0x7fd1d0006180 2024-04-25T00:33:10.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1ea3ab640 1 --2- 172.21.15.29:0/2849555374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1e40b4590 0x7fd1d0005c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:10.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e9baa640 1 --2- 172.21.15.29:0/2849555374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1d0006180 0x7fd1d0006a20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009ae0 tx=0x7fd1e001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba744d9d6958d479 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:10.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00b2070 con 0x7fd1d0006180 2024-04-25T00:33:10.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e00a5550 con 0x7fd1d0006180 2024-04-25T00:33:10.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1e93a9640 1 -- 172.21.15.29:0/2849555374 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00096b0 con 0x7fd1d0006180 2024-04-25T00:33:10.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2849555374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1d0006180 msgr2=0x7fd1d0006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:10.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1ec60f640 1 --2- 172.21.15.29:0/2849555374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1d0006180 0x7fd1d0006a20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009ae0 tx=0x7fd1e001ca70 comp rx=0 tx=0).stop 2024-04-25T00:33:10.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1eabac640 1 -- 172.21.15.29:0/2849555374 reap_dead start 2024-04-25T00:33:10.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2849555374 shutdown_connections 2024-04-25T00:33:10.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2849555374 >> 172.21.15.29:0/2849555374 conn(0x7fd1e40b1870 msgr2=0x7fd1e40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:10.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2849555374 shutdown_connections 2024-04-25T00:33:10.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.574+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2849555374 wait complete. 2024-04-25T00:33:10.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1ec60f640 1 Processor -- start 2024-04-25T00:33:10.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1ec60f640 1 -- start start 2024-04-25T00:33:10.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1ec60f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1d0006180 0x7fd1e4159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1ec60f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1e40cf3e0 0x7fd1e4131950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1ec60f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41f7830 0x7fd1e413e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1ec60f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e40cd650 con 0x7fd1e41f7830 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1ec60f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e415f700 con 0x7fd1e40cf3e0 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1ec60f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e40cd7d0 con 0x7fd1d0006180 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1eabac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41f7830 0x7fd1e413e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1eabac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41f7830 0x7fd1e413e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59816/0 (socket says 172.21.15.29:59816) 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1eabac640 1 -- 172.21.15.29:0/2613578748 learned_addr learned my addr 172.21.15.29:0/2613578748 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1ea3ab640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1d0006180 0x7fd1e4159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1e9baa640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1e40cf3e0 0x7fd1e4131950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1eabac640 1 -- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1d0006180 msgr2=0x7fd1e4159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1eabac640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1d0006180 0x7fd1e4159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1eabac640 1 -- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1e40cf3e0 msgr2=0x7fd1e4131950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1eabac640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1e40cf3e0 0x7fd1e4131950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1eabac640 1 -- 172.21.15.29:0/2613578748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e41f8130 con 0x7fd1e41f7830 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1e9baa640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1e40cf3e0 0x7fd1e4131950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1eabac640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41f7830 0x7fd1e413e280 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40d0bb0 tx=0x7fd1dc011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1daffd640 1 -- 172.21.15.29:0/2613578748 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc010040 con 0x7fd1e41f7830 2024-04-25T00:33:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1daffd640 1 -- 172.21.15.29:0/2613578748 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1dc01a070 con 0x7fd1e41f7830 2024-04-25T00:33:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2613578748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e41f8410 con 0x7fd1e41f7830 2024-04-25T00:33:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1daffd640 1 -- 172.21.15.29:0/2613578748 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc00fb80 con 0x7fd1e41f7830 2024-04-25T00:33:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2613578748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e4131430 con 0x7fd1e41f7830 2024-04-25T00:33:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.578+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2613578748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e40692e0 con 0x7fd1e41f7830 2024-04-25T00:33:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.582+0000 7fd1daffd640 1 -- 172.21.15.29:0/2613578748 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fd1dc00fd20 con 0x7fd1e41f7830 2024-04-25T00:33:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.582+0000 7fd1daffd640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd1c8067a70 0x7fd1c8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.582+0000 7fd1daffd640 1 -- 172.21.15.29:0/2613578748 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7fd1dc088ae0 con 0x7fd1e41f7830 2024-04-25T00:33:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.582+0000 7fd1ea3ab640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd1c8067a70 0x7fd1c8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.582+0000 7fd1ea3ab640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd1c8067a70 0x7fd1c8069f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4001760 tx=0x7fd1d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:10.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.586+0000 7fd1daffd640 1 -- 172.21.15.29:0/2613578748 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd1dc059780 con 0x7fd1e41f7830 2024-04-25T00:33:10.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:10.742+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2613578748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd1e41418e0 con 0x7fd1e41f7830 2024-04-25T00:33:11.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.750+0000 7fd1daffd640 1 -- 172.21.15.29:0/2613578748 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) ==== 138+0+0 (secure 0 0 0) 0x7fd1dc059080 con 0x7fd1e41f7830 2024-04-25T00:33:11.755 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-04-25T00:33:11.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.754+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd1c8067a70 msgr2=0x7fd1c8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:11.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.754+0000 7fd1ec60f640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd1c8067a70 0x7fd1c8069f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4001760 tx=0x7fd1d4009040 comp rx=0 tx=0).stop 2024-04-25T00:33:11.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.754+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2613578748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41f7830 msgr2=0x7fd1e413e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:11.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.754+0000 7fd1ec60f640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41f7830 0x7fd1e413e280 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40d0bb0 tx=0x7fd1dc011c20 comp rx=0 tx=0).stop 2024-04-25T00:33:11.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.754+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2613578748 shutdown_connections 2024-04-25T00:33:11.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.754+0000 7fd1ec60f640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41f7830 0x7fd1e413e280 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.754+0000 7fd1ec60f640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd1c8067a70 0x7fd1c8069f30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.754+0000 7fd1ec60f640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1e40cf3e0 0x7fd1e4131950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.754+0000 7fd1ec60f640 1 --2- 172.21.15.29:0/2613578748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1d0006180 0x7fd1e4159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.754+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2613578748 >> 172.21.15.29:0/2613578748 conn(0x7fd1e40b1870 msgr2=0x7fd1e40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:11.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.754+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2613578748 shutdown_connections 2024-04-25T00:33:11.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.754+0000 7fd1ec60f640 1 -- 172.21.15.29:0/2613578748 wait complete. 2024-04-25T00:33:11.784 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:33:11.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec22298640 1 Processor -- start 2024-04-25T00:33:11.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec22298640 1 -- start start 2024-04-25T00:33:11.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec22298640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1c0b3c30 0x7fec1c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec22298640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fec1c0b4b10 0x7fec1c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec22298640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec1c0b41e0 con 0x7fec1c0b3140 2024-04-25T00:33:11.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec22298640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fec1c06d960 con 0x7fec1c0b4b10 2024-04-25T00:33:11.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec22298640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fec1c06dc40 con 0x7fec1c0b3c30 2024-04-25T00:33:11.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec1b7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1c0b3c30 0x7fec1c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec20835640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fec1c0b4b10 0x7fec1c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec1b7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1c0b3c30 0x7fec1c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:35400/0 (socket says 172.21.15.29:35400) 2024-04-25T00:33:11.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec1b7fe640 1 -- 172.21.15.29:0/1681908068 learned_addr learned my addr 172.21.15.29:0/1681908068 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:11.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec20835640 1 -- 172.21.15.29:0/1681908068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1c0b3c30 msgr2=0x7fec1c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:11.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec20835640 1 --2- 172.21.15.29:0/1681908068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1c0b3c30 0x7fec1c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec20835640 1 -- 172.21.15.29:0/1681908068 >> v1:172.21.15.157:6789/0 conn(0x7fec1c0b3140 legacy=0x7fec1c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:11.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec20835640 1 -- 172.21.15.29:0/1681908068 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c06df20 con 0x7fec1c0b4b10 2024-04-25T00:33:11.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.914+0000 7fec1b7fe640 1 --2- 172.21.15.29:0/1681908068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1c0b3c30 0x7fec1c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:33:11.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.918+0000 7fec20835640 1 --2- 172.21.15.29:0/1681908068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c0b4b10 0x7fec1c0ce240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fec14009d90 tx=0x7fec1401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5649aa50a3769e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:11.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.918+0000 7fec1affd640 1 -- 172.21.15.29:0/1681908068 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec140b2070 con 0x7fec1c0b4b10 2024-04-25T00:33:11.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.918+0000 7fec1affd640 1 -- 172.21.15.29:0/1681908068 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec140a5830 con 0x7fec1c0b4b10 2024-04-25T00:33:11.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.918+0000 7fec1affd640 1 -- 172.21.15.29:0/1681908068 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec140a5c00 con 0x7fec1c0b4b10 2024-04-25T00:33:11.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.918+0000 7fec22298640 1 -- 172.21.15.29:0/1681908068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c0b4b10 msgr2=0x7fec1c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:11.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.918+0000 7fec22298640 1 --2- 172.21.15.29:0/1681908068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c0b4b10 0x7fec1c0ce240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fec14009d90 tx=0x7fec1401c970 comp rx=0 tx=0).stop 2024-04-25T00:33:11.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.918+0000 7fec22298640 1 -- 172.21.15.29:0/1681908068 shutdown_connections 2024-04-25T00:33:11.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.918+0000 7fec22298640 1 --2- 172.21.15.29:0/1681908068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c0b4b10 0x7fec1c0ce240 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.918+0000 7fec22298640 1 --2- 172.21.15.29:0/1681908068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1c0b3c30 0x7fec1c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.918+0000 7fec22298640 1 -- 172.21.15.29:0/1681908068 >> 172.21.15.29:0/1681908068 conn(0x7fec1c0b1870 msgr2=0x7fec1c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:11.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.918+0000 7fec22298640 1 -- 172.21.15.29:0/1681908068 shutdown_connections 2024-04-25T00:33:11.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.918+0000 7fec22298640 1 -- 172.21.15.29:0/1681908068 wait complete. 2024-04-25T00:33:11.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec22298640 1 Processor -- start 2024-04-25T00:33:11.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec22298640 1 -- start start 2024-04-25T00:33:11.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec22298640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c0b3140 0x7fec1c13cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec22298640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1c0b3c30 0x7fec1c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec22298640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec1c0b4b10 0x7fec1c15d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec22298640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c168d80 con 0x7fec1c0b4b10 2024-04-25T00:33:11.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec22298640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c168c00 con 0x7fec1c0b3140 2024-04-25T00:33:11.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec22298640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fec1c168f00 con 0x7fec1c0b3c30 2024-04-25T00:33:11.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec20835640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec1c0b4b10 0x7fec1c15d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec20835640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec1c0b4b10 0x7fec1c15d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59824/0 (socket says 172.21.15.29:59824) 2024-04-25T00:33:11.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec20835640 1 -- 172.21.15.29:0/241280977 learned_addr learned my addr 172.21.15.29:0/241280977 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:11.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec1b7fe640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1c0b3c30 0x7fec1c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec1bfff640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c0b3140 0x7fec1c13cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec20835640 1 -- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1c0b3c30 msgr2=0x7fec1c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:11.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec20835640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1c0b3c30 0x7fec1c15b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec20835640 1 -- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c0b3140 msgr2=0x7fec1c13cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:11.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec20835640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c0b3140 0x7fec1c13cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec20835640 1 -- 172.21.15.29:0/241280977 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c071f00 con 0x7fec1c0b4b10 2024-04-25T00:33:11.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec20835640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec1c0b4b10 0x7fec1c15d410 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fec140b0040 tx=0x7fec14019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:11.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec18ff9640 1 -- 172.21.15.29:0/241280977 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec140b2050 con 0x7fec1c0b4b10 2024-04-25T00:33:11.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec18ff9640 1 -- 172.21.15.29:0/241280977 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec140be070 con 0x7fec1c0b4b10 2024-04-25T00:33:11.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec22298640 1 -- 172.21.15.29:0/241280977 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c0721e0 con 0x7fec1c0b4b10 2024-04-25T00:33:11.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec18ff9640 1 -- 172.21.15.29:0/241280977 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec140ad870 con 0x7fec1c0b4b10 2024-04-25T00:33:11.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec22298640 1 -- 172.21.15.29:0/241280977 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c0727a0 con 0x7fec1c0b4b10 2024-04-25T00:33:11.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.922+0000 7fec22298640 1 -- 172.21.15.29:0/241280977 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1c069270 con 0x7fec1c0b4b10 2024-04-25T00:33:11.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.926+0000 7fec18ff9640 1 -- 172.21.15.29:0/241280977 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fec140ada10 con 0x7fec1c0b4b10 2024-04-25T00:33:11.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.926+0000 7fec18ff9640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fec08067aa0 0x7fec08069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.926+0000 7fec18ff9640 1 -- 172.21.15.29:0/241280977 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5227+0+0 (secure 0 0 0) 0x7fec14121800 con 0x7fec1c0b4b10 2024-04-25T00:33:11.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.926+0000 7fec1bfff640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fec08067aa0 0x7fec08069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.926+0000 7fec1bfff640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fec08067aa0 0x7fec08069f60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fec1c072ec0 tx=0x7fec0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:11.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:11.930+0000 7fec18ff9640 1 -- 172.21.15.29:0/241280977 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fec14016560 con 0x7fec1c0b4b10 2024-04-25T00:33:12.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.082+0000 7fec22298640 1 -- 172.21.15.29:0/241280977 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fec1c155150 con 0x7fec1c0b4b10 2024-04-25T00:33:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.762+0000 7fec18ff9640 1 -- 172.21.15.29:0/241280977 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) ==== 170+0+0 (secure 0 0 0) 0x7fec140f1c50 con 0x7fec1c0b4b10 2024-04-25T00:33:12.766 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-04-25T00:33:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.770+0000 7fec22298640 1 -- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fec08067aa0 msgr2=0x7fec08069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.770+0000 7fec22298640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fec08067aa0 0x7fec08069f60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fec1c072ec0 tx=0x7fec0c009040 comp rx=0 tx=0).stop 2024-04-25T00:33:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.770+0000 7fec22298640 1 -- 172.21.15.29:0/241280977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec1c0b4b10 msgr2=0x7fec1c15d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.770+0000 7fec22298640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec1c0b4b10 0x7fec1c15d410 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fec140b0040 tx=0x7fec14019710 comp rx=0 tx=0).stop 2024-04-25T00:33:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.770+0000 7fec22298640 1 -- 172.21.15.29:0/241280977 shutdown_connections 2024-04-25T00:33:12.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.770+0000 7fec22298640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec1c0b4b10 0x7fec1c15d410 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.770+0000 7fec22298640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1c0b3c30 0x7fec1c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.770+0000 7fec22298640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fec08067aa0 0x7fec08069f60 secure :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fec1c072ec0 tx=0x7fec0c009040 comp rx=0 tx=0).stop 2024-04-25T00:33:12.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.770+0000 7fec22298640 1 --2- 172.21.15.29:0/241280977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec1c0b3140 0x7fec1c13cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.770+0000 7fec22298640 1 -- 172.21.15.29:0/241280977 >> 172.21.15.29:0/241280977 conn(0x7fec1c0b1870 msgr2=0x7fec1c155950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:12.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.774+0000 7fec22298640 1 -- 172.21.15.29:0/241280977 shutdown_connections 2024-04-25T00:33:12.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.774+0000 7fec22298640 1 -- 172.21.15.29:0/241280977 wait complete. 2024-04-25T00:33:12.813 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:33:12.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.934+0000 7f1c0dc6b640 1 Processor -- start 2024-04-25T00:33:12.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.934+0000 7f1c0dc6b640 1 -- start start 2024-04-25T00:33:12.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.934+0000 7f1c0dc6b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1c0815e200 0x7f1c080d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:12.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.934+0000 7f1c0dc6b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c080cfc10 con 0x7f1c080d2c60 2024-04-25T00:33:12.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.934+0000 7f1c0dc6b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c080cfef0 con 0x7f1c080cf690 2024-04-25T00:33:12.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.934+0000 7f1c0dc6b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1c080b38d0 con 0x7f1c0815e200 2024-04-25T00:33:12.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.934+0000 7f1c077fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1c0815e200 0x7f1c080d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:12.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.934+0000 7f1c07fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1c080d2c60 0x7f1c080b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38806/0 (socket says 172.21.15.29:38806) 2024-04-25T00:33:12.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.934+0000 7f1c077fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1c0815e200 0x7f1c080d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59826/0 (socket says 172.21.15.29:59826) 2024-04-25T00:33:12.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.934+0000 7f1c07fff640 1 -- 172.21.15.29:0/1692623359 learned_addr learned my addr 172.21.15.29:0/1692623359 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:12.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.934+0000 7f1c077fe640 1 -- 172.21.15.29:0/1692623359 >> v1:172.21.15.157:6789/0 conn(0x7f1c080cf690 legacy=0x7f1c080d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:12.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.934+0000 7f1c077fe640 1 -- 172.21.15.29:0/1692623359 >> v1:172.21.15.29:6789/0 conn(0x7f1c080d2c60 legacy=0x7f1c080b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:12.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.934+0000 7f1c077fe640 1 -- 172.21.15.29:0/1692623359 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c080b3b20 con 0x7f1c0815e200 2024-04-25T00:33:12.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.938+0000 7f1c077fe640 1 --2- 172.21.15.29:0/1692623359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0815e200 0x7f1c080d47f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80097c0 tx=0x7f1bf801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6393883cc75e055f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:12.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.938+0000 7f1c067fc640 1 -- 172.21.15.29:0/1692623359 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80ab650 con 0x7f1c0815e200 2024-04-25T00:33:12.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.938+0000 7f1c067fc640 1 -- 172.21.15.29:0/1692623359 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf80b0070 con 0x7f1c0815e200 2024-04-25T00:33:12.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.938+0000 7f1c067fc640 1 -- 172.21.15.29:0/1692623359 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80aba10 con 0x7f1c0815e200 2024-04-25T00:33:12.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.938+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/1692623359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0815e200 msgr2=0x7f1c080d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:12.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.938+0000 7f1c0dc6b640 1 --2- 172.21.15.29:0/1692623359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0815e200 0x7f1c080d47f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80097c0 tx=0x7f1bf801c430 comp rx=0 tx=0).stop 2024-04-25T00:33:12.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.938+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/1692623359 shutdown_connections 2024-04-25T00:33:12.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.938+0000 7f1c0dc6b640 1 --2- 172.21.15.29:0/1692623359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0815e200 0x7f1c080d47f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.938+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/1692623359 >> 172.21.15.29:0/1692623359 conn(0x7f1c0809d280 msgr2=0x7f1c080d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:12.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.938+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/1692623359 shutdown_connections 2024-04-25T00:33:12.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.938+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/1692623359 wait complete. 2024-04-25T00:33:12.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c0dc6b640 1 Processor -- start 2024-04-25T00:33:12.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c0dc6b640 1 -- start start 2024-04-25T00:33:12.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c0dc6b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c080cf690 0x7f1c08154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:12.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c0dc6b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c080d2c60 0x7f1c080b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:12.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c0dc6b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c0815e200 0x7f1c080b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:12.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c0dc6b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1c080b53c0 con 0x7f1c080d2c60 2024-04-25T00:33:12.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c0dc6b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c080b5240 con 0x7f1c0815e200 2024-04-25T00:33:12.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c0dc6b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1c080b5540 con 0x7f1c080cf690 2024-04-25T00:33:12.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c06ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c080d2c60 0x7f1c080b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:12.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c06ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c080d2c60 0x7f1c080b2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59830/0 (socket says 172.21.15.29:59830) 2024-04-25T00:33:12.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c06ffd640 1 -- 172.21.15.29:0/2755976887 learned_addr learned my addr 172.21.15.29:0/2755976887 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:12.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c077fe640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c080cf690 0x7f1c08154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:12.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c07fff640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c0815e200 0x7f1c080b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:12.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c06ffd640 1 -- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c080cf690 msgr2=0x7f1c08154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:12.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c06ffd640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c080cf690 0x7f1c08154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c06ffd640 1 -- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c0815e200 msgr2=0x7f1c080b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:12.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c06ffd640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c0815e200 0x7f1c080b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c06ffd640 1 -- 172.21.15.29:0/2755976887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c080c8b90 con 0x7f1c080d2c60 2024-04-25T00:33:12.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c07fff640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c0815e200 0x7f1c080b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:12.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c06ffd640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c080d2c60 0x7f1c080b2080 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b410 tx=0x7f1bfc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:12.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1beffff640 1 -- 172.21.15.29:0/2755976887 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc015020 con 0x7f1c080d2c60 2024-04-25T00:33:12.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1beffff640 1 -- 172.21.15.29:0/2755976887 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bfc011830 con 0x7f1c080d2c60 2024-04-25T00:33:12.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/2755976887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c080c8e30 con 0x7f1c080d2c60 2024-04-25T00:33:12.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1beffff640 1 -- 172.21.15.29:0/2755976887 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc011c20 con 0x7f1c080d2c60 2024-04-25T00:33:12.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/2755976887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c081551b0 con 0x7f1c080d2c60 2024-04-25T00:33:12.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/2755976887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c08069270 con 0x7f1c080d2c60 2024-04-25T00:33:12.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.942+0000 7f1beffff640 1 -- 172.21.15.29:0/2755976887 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f1bfc018020 con 0x7f1c080d2c60 2024-04-25T00:33:12.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.946+0000 7f1beffff640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1bf4067aa0 0x7f1bf4069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:12.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.946+0000 7f1beffff640 1 -- 172.21.15.29:0/2755976887 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5609+0+0 (secure 0 0 0) 0x7f1bfc085c90 con 0x7f1c080d2c60 2024-04-25T00:33:12.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.946+0000 7f1c077fe640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1bf4067aa0 0x7f1bf4069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:12.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.946+0000 7f1c077fe640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1bf4067aa0 0x7f1bf4069f60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80af000 tx=0x7f1bf80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:12.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:12.950+0000 7f1beffff640 1 -- 172.21.15.29:0/2755976887 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1bfc01f030 con 0x7f1c080d2c60 2024-04-25T00:33:13.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.150+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/2755976887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1c080b4430 con 0x7f1c080d2c60 2024-04-25T00:33:13.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.814+0000 7f1beffff640 1 -- 172.21.15.29:0/2755976887 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) ==== 172+0+0 (secure 0 0 0) 0x7f1bfc056490 con 0x7f1c080d2c60 2024-04-25T00:33:13.819 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 6 and data pool 7 2024-04-25T00:33:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.818+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1bf4067aa0 msgr2=0x7f1bf4069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.818+0000 7f1c0dc6b640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1bf4067aa0 0x7f1bf4069f60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80af000 tx=0x7f1bf80a7000 comp rx=0 tx=0).stop 2024-04-25T00:33:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.818+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/2755976887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c080d2c60 msgr2=0x7f1c080b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.818+0000 7f1c0dc6b640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c080d2c60 0x7f1c080b2080 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b410 tx=0x7f1bfc00b8d0 comp rx=0 tx=0).stop 2024-04-25T00:33:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.818+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/2755976887 shutdown_connections 2024-04-25T00:33:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.818+0000 7f1c0dc6b640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1bf4067aa0 0x7f1bf4069f60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.818+0000 7f1c0dc6b640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c0815e200 0x7f1c080b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:13.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.818+0000 7f1c0dc6b640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c080d2c60 0x7f1c080b2080 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.818+0000 7f1c0dc6b640 1 --2- 172.21.15.29:0/2755976887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c080cf690 0x7f1c08154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.818+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/2755976887 >> 172.21.15.29:0/2755976887 conn(0x7f1c0809d280 msgr2=0x7f1c08159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.818+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/2755976887 shutdown_connections 2024-04-25T00:33:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.818+0000 7f1c0dc6b640 1 -- 172.21.15.29:0/2755976887 wait complete. 2024-04-25T00:33:13.847 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:33:13.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f741bf5d640 1 Processor -- start 2024-04-25T00:33:13.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f741bf5d640 1 -- start start 2024-04-25T00:33:13.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f741bf5d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7414070580 0x7f741413f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:13.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f741bf5d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f741415f930 0x7f741406b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:13.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f741bf5d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f741415fef0 con 0x7f74140701a0 2024-04-25T00:33:13.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f741bf5d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f74141601d0 con 0x7f7414070580 2024-04-25T00:33:13.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f741bf5d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f74141604b0 con 0x7f741415f930 2024-04-25T00:33:13.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f7419cf9640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f74140701a0 0x7f741415cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38810/0 (socket says 172.21.15.29:38810) 2024-04-25T00:33:13.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f74194f8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7414070580 0x7f741413f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:13.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f7419cf9640 1 -- 172.21.15.29:0/3324757102 learned_addr learned my addr 172.21.15.29:0/3324757102 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:13.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f741a4fa640 1 --2- 172.21.15.29:0/3324757102 >> v2:172.21.15.157:3300/0 conn(0x7f741415f930 0x7f741406b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:13.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f74194f8640 1 -- 172.21.15.29:0/3324757102 >> v2:172.21.15.157:3300/0 conn(0x7f741415f930 msgr2=0x7f741406b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:13.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f74194f8640 1 --2- 172.21.15.29:0/3324757102 >> v2:172.21.15.157:3300/0 conn(0x7f741415f930 0x7f741406b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:13.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f74194f8640 1 -- 172.21.15.29:0/3324757102 >> v1:172.21.15.29:6789/0 conn(0x7f74140701a0 legacy=0x7f741415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:13.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f74194f8640 1 -- 172.21.15.29:0/3324757102 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7414160790 con 0x7f7414070580 2024-04-25T00:33:13.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f7418cf7640 1 -- 172.21.15.29:0/3324757102 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1231702301 0 0) 0x7f741415fef0 con 0x7f74140701a0 2024-04-25T00:33:13.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.958+0000 7f741a4fa640 1 --2- 172.21.15.29:0/3324757102 >> v2:172.21.15.157:3300/0 conn(0x7f741415f930 0x7f741406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:13.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f74194f8640 1 --2- 172.21.15.29:0/3324757102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7414070580 0x7f741413f430 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f741415f870 tx=0x7f740401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d46ffe50b7a3197 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:13.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f7418cf7640 1 -- 172.21.15.29:0/3324757102 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74040b0070 con 0x7f7414070580 2024-04-25T00:33:13.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f7418cf7640 1 -- 172.21.15.29:0/3324757102 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74040ab600 con 0x7f7414070580 2024-04-25T00:33:13.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f7418cf7640 1 -- 172.21.15.29:0/3324757102 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74040ab9d0 con 0x7f7414070580 2024-04-25T00:33:13.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 -- 172.21.15.29:0/3324757102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7414070580 msgr2=0x7f741413f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:13.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 --2- 172.21.15.29:0/3324757102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7414070580 0x7f741413f430 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f741415f870 tx=0x7f740401c480 comp rx=0 tx=0).stop 2024-04-25T00:33:13.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 -- 172.21.15.29:0/3324757102 shutdown_connections 2024-04-25T00:33:13.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 --2- 172.21.15.29:0/3324757102 >> v2:172.21.15.157:3300/0 conn(0x7f741415f930 0x7f741406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:13.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 --2- 172.21.15.29:0/3324757102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7414070580 0x7f741413f430 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:13.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 -- 172.21.15.29:0/3324757102 >> 172.21.15.29:0/3324757102 conn(0x7f74140d0c30 msgr2=0x7f74140d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:13.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 -- 172.21.15.29:0/3324757102 shutdown_connections 2024-04-25T00:33:13.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 -- 172.21.15.29:0/3324757102 wait complete. 2024-04-25T00:33:13.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 Processor -- start 2024-04-25T00:33:13.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 -- start start 2024-04-25T00:33:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74140701a0 0x7f741413c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7414070580 0x7f741413de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f741415f930 0x7f74140ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f74140726e0 con 0x7f74140701a0 2024-04-25T00:33:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7414072560 con 0x7f7414070580 2024-04-25T00:33:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f741bf5d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7414072860 con 0x7f741415f930 2024-04-25T00:33:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f7419cf9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74140701a0 0x7f741413c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f7419cf9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74140701a0 0x7f741413c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59848/0 (socket says 172.21.15.29:59848) 2024-04-25T00:33:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.962+0000 7f7419cf9640 1 -- 172.21.15.29:0/360693012 learned_addr learned my addr 172.21.15.29:0/360693012 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f7419cf9640 1 -- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f741415f930 msgr2=0x7f74140ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f741a4fa640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f741415f930 0x7f74140ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f74194f8640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7414070580 0x7f741413de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f7419cf9640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f741415f930 0x7f74140ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f7419cf9640 1 -- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7414070580 msgr2=0x7f741413de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f7419cf9640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7414070580 0x7f741413de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f7419cf9640 1 -- 172.21.15.29:0/360693012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74140c5c90 con 0x7f74140701a0 2024-04-25T00:33:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f7419cf9640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74140701a0 0x7f741413c2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f741000c4b0 tx=0x7f741000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f740a7fc640 1 -- 172.21.15.29:0/360693012 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410018070 con 0x7f74140701a0 2024-04-25T00:33:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f740a7fc640 1 -- 172.21.15.29:0/360693012 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7410007500 con 0x7f74140701a0 2024-04-25T00:33:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f740a7fc640 1 -- 172.21.15.29:0/360693012 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410004a30 con 0x7f74140701a0 2024-04-25T00:33:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f741bf5d640 1 -- 172.21.15.29:0/360693012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74140c5f70 con 0x7f74140701a0 2024-04-25T00:33:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f741bf5d640 1 -- 172.21.15.29:0/360693012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74140c6530 con 0x7f74140701a0 2024-04-25T00:33:13.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f741bf5d640 1 -- 172.21.15.29:0/360693012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7414069270 con 0x7f74140701a0 2024-04-25T00:33:13.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f740a7fc640 1 -- 172.21.15.29:0/360693012 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f741001b070 con 0x7f74140701a0 2024-04-25T00:33:13.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f740a7fc640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f73f8067a70 0x7f73f8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:13.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f740a7fc640 1 -- 172.21.15.29:0/360693012 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f7410085310 con 0x7f74140701a0 2024-04-25T00:33:13.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f74194f8640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f73f8067a70 0x7f73f8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:13.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.966+0000 7f74194f8640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f73f8067a70 0x7f73f8069f30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f74040ad000 tx=0x7f7404008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:13.970+0000 7f740a7fc640 1 -- 172.21.15.29:0/360693012 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7410055b30 con 0x7f74140701a0 2024-04-25T00:33:14.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.134+0000 7f741bf5d640 1 -- 172.21.15.29:0/360693012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74140d1890 con 0x7f74140701a0 2024-04-25T00:33:14.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.138+0000 7f740a7fc640 1 -- 172.21.15.29:0/360693012 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16578 (secure 0 0 0) 0x7f7410055430 con 0x7f74140701a0 2024-04-25T00:33:14.141 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:14.142 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":44,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:33:13.156008+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:33:10.789862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:33:12.129841+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:33:14.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.138+0000 7f741bf5d640 1 -- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f73f8067a70 msgr2=0x7f73f8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.138+0000 7f741bf5d640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f73f8067a70 0x7f73f8069f30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f74040ad000 tx=0x7f7404008040 comp rx=0 tx=0).stop 2024-04-25T00:33:14.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.138+0000 7f741bf5d640 1 -- 172.21.15.29:0/360693012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74140701a0 msgr2=0x7f741413c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.138+0000 7f741bf5d640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74140701a0 0x7f741413c2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f741000c4b0 tx=0x7f741000c970 comp rx=0 tx=0).stop 2024-04-25T00:33:14.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.138+0000 7f741bf5d640 1 -- 172.21.15.29:0/360693012 shutdown_connections 2024-04-25T00:33:14.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.138+0000 7f741bf5d640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f741415f930 0x7f74140ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.138+0000 7f741bf5d640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f73f8067a70 0x7f73f8069f30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.138+0000 7f741bf5d640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7414070580 0x7f741413de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.138+0000 7f741bf5d640 1 --2- 172.21.15.29:0/360693012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74140701a0 0x7f741413c2f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.138+0000 7f741bf5d640 1 -- 172.21.15.29:0/360693012 >> 172.21.15.29:0/360693012 conn(0x7f74140d0c30 msgr2=0x7f74140b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:14.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.138+0000 7f741bf5d640 1 -- 172.21.15.29:0/360693012 shutdown_connections 2024-04-25T00:33:14.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.138+0000 7f741bf5d640 1 -- 172.21.15.29:0/360693012 wait complete. 2024-04-25T00:33:14.162 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:33:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.282+0000 7eff192f7640 1 Processor -- start 2024-04-25T00:33:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.282+0000 7eff192f7640 1 -- start start 2024-04-25T00:33:14.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff192f7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7eff140b3140 0x7eff1406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:14.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff192f7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff140b41e0 con 0x7eff140b3c30 2024-04-25T00:33:14.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff192f7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff1406d960 con 0x7eff140b4b10 2024-04-25T00:33:14.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff192f7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7eff1406dc40 con 0x7eff140b3140 2024-04-25T00:33:14.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff12ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7eff140b3140 0x7eff1406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff137fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7eff140b3c30 0x7eff140ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38826/0 (socket says 172.21.15.29:38826) 2024-04-25T00:33:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff137fe640 1 -- 172.21.15.29:0/3922023249 learned_addr learned my addr 172.21.15.29:0/3922023249 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff12ffd640 1 -- 172.21.15.29:0/3922023249 >> v1:172.21.15.157:6789/0 conn(0x7eff140b4b10 legacy=0x7eff140b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff12ffd640 1 -- 172.21.15.29:0/3922023249 >> v1:172.21.15.29:6789/0 conn(0x7eff140b3c30 legacy=0x7eff140ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff12ffd640 1 -- 172.21.15.29:0/3922023249 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1406df20 con 0x7eff140b3140 2024-04-25T00:33:14.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff12ffd640 1 --2- 172.21.15.29:0/3922023249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff140b3140 0x7eff1406e440 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7eff040097f0 tx=0x7eff0401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf784c6707e23013 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:14.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff11ffb640 1 -- 172.21.15.29:0/3922023249 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff040b0070 con 0x7eff140b3140 2024-04-25T00:33:14.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff11ffb640 1 -- 172.21.15.29:0/3922023249 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff040ab600 con 0x7eff140b3140 2024-04-25T00:33:14.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.286+0000 7eff11ffb640 1 -- 172.21.15.29:0/3922023249 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff040ab9d0 con 0x7eff140b3140 2024-04-25T00:33:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 -- 172.21.15.29:0/3922023249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff140b3140 msgr2=0x7eff1406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 --2- 172.21.15.29:0/3922023249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff140b3140 0x7eff1406e440 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7eff040097f0 tx=0x7eff0401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 -- 172.21.15.29:0/3922023249 shutdown_connections 2024-04-25T00:33:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 --2- 172.21.15.29:0/3922023249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff140b3140 0x7eff1406e440 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 -- 172.21.15.29:0/3922023249 >> 172.21.15.29:0/3922023249 conn(0x7eff140b1870 msgr2=0x7eff140b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 -- 172.21.15.29:0/3922023249 shutdown_connections 2024-04-25T00:33:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 -- 172.21.15.29:0/3922023249 wait complete. 2024-04-25T00:33:14.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 Processor -- start 2024-04-25T00:33:14.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 -- start start 2024-04-25T00:33:14.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff140b3140 0x7eff1413a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:14.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff140b3c30 0x7eff1415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:14.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff140b4b10 0x7eff141586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:14.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7eff14169d90 con 0x7eff140b4b10 2024-04-25T00:33:14.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff14169c10 con 0x7eff140b3c30 2024-04-25T00:33:14.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff14169f10 con 0x7eff140b3140 2024-04-25T00:33:14.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff140b4b10 0x7eff141586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:14.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff137fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff140b4b10 0x7eff141586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59868/0 (socket says 172.21.15.29:59868) 2024-04-25T00:33:14.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff137fe640 1 -- 172.21.15.29:0/1517539981 learned_addr learned my addr 172.21.15.29:0/1517539981 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:14.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff12ffd640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff140b3140 0x7eff1413a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff127fc640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff140b3c30 0x7eff1415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff137fe640 1 -- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff140b3140 msgr2=0x7eff1413a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff137fe640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff140b3140 0x7eff1413a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff137fe640 1 -- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff140b3c30 msgr2=0x7eff1415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff137fe640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff140b3c30 0x7eff1415a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff137fe640 1 -- 172.21.15.29:0/1517539981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1413ceb0 con 0x7eff140b4b10 2024-04-25T00:33:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff137fe640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff140b4b10 0x7eff141586d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00ca80 tx=0x7eff0c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7efefb7fe640 1 -- 172.21.15.29:0/1517539981 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c017070 con 0x7eff140b4b10 2024-04-25T00:33:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7efefb7fe640 1 -- 172.21.15.29:0/1517539981 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff0c00ac30 con 0x7eff140b4b10 2024-04-25T00:33:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 -- 172.21.15.29:0/1517539981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1413be10 con 0x7eff140b4b10 2024-04-25T00:33:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7efefb7fe640 1 -- 172.21.15.29:0/1517539981 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c012600 con 0x7eff140b4b10 2024-04-25T00:33:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.290+0000 7eff192f7640 1 -- 172.21.15.29:0/1517539981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1413c3d0 con 0x7eff140b4b10 2024-04-25T00:33:14.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.294+0000 7eff192f7640 1 -- 172.21.15.29:0/1517539981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff14069270 con 0x7eff140b4b10 2024-04-25T00:33:14.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.294+0000 7efefb7fe640 1 -- 172.21.15.29:0/1517539981 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7eff0c01a070 con 0x7eff140b4b10 2024-04-25T00:33:14.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.294+0000 7efefb7fe640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff00067a70 0x7eff00069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:14.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.294+0000 7efefb7fe640 1 -- 172.21.15.29:0/1517539981 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7eff0c085ba0 con 0x7eff140b4b10 2024-04-25T00:33:14.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.294+0000 7eff12ffd640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff00067a70 0x7eff00069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:14.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.294+0000 7eff12ffd640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff00067a70 0x7eff00069f30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7eff040ad000 tx=0x7eff040a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:14.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.298+0000 7efefb7fe640 1 -- 172.21.15.29:0/1517539981 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7eff0c00ae20 con 0x7eff140b4b10 2024-04-25T00:33:14.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.458+0000 7eff192f7640 1 -- 172.21.15.29:0/1517539981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff1413cae0 con 0x7eff140b4b10 2024-04-25T00:33:14.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.462+0000 7efefb7fe640 1 -- 172.21.15.29:0/1517539981 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16578 (secure 0 0 0) 0x7eff0c055e00 con 0x7eff140b4b10 2024-04-25T00:33:14.467 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:14.468 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":44,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:33:13.156008+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:33:10.789862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:33:12.129841+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:33:14.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.466+0000 7eff192f7640 1 -- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff00067a70 msgr2=0x7eff00069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.466+0000 7eff192f7640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff00067a70 0x7eff00069f30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7eff040ad000 tx=0x7eff040a6000 comp rx=0 tx=0).stop 2024-04-25T00:33:14.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.466+0000 7eff192f7640 1 -- 172.21.15.29:0/1517539981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff140b4b10 msgr2=0x7eff141586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.466+0000 7eff192f7640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff140b4b10 0x7eff141586d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00ca80 tx=0x7eff0c00cf40 comp rx=0 tx=0).stop 2024-04-25T00:33:14.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.466+0000 7eff192f7640 1 -- 172.21.15.29:0/1517539981 shutdown_connections 2024-04-25T00:33:14.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.466+0000 7eff192f7640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff140b4b10 0x7eff141586d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.466+0000 7eff192f7640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7eff00067a70 0x7eff00069f30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.466+0000 7eff192f7640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff140b3c30 0x7eff1415a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.466+0000 7eff192f7640 1 --2- 172.21.15.29:0/1517539981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff140b3140 0x7eff1413a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.466+0000 7eff192f7640 1 -- 172.21.15.29:0/1517539981 >> 172.21.15.29:0/1517539981 conn(0x7eff140b1870 msgr2=0x7eff14157380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:14.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.466+0000 7eff192f7640 1 -- 172.21.15.29:0/1517539981 shutdown_connections 2024-04-25T00:33:14.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.466+0000 7eff192f7640 1 -- 172.21.15.29:0/1517539981 wait complete. 2024-04-25T00:33:14.487 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:33:14.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f355e63a640 1 Processor -- start 2024-04-25T00:33:14.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f355e63a640 1 -- start start 2024-04-25T00:33:14.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f355e63a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355815bf70 0x7f35580b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:14.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f355e63a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35580cdbe0 con 0x7f35580cece0 2024-04-25T00:33:14.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f355e63a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35580cf6e0 con 0x7f355815c350 2024-04-25T00:33:14.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f355e63a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f35580cf9e0 con 0x7f355815bf70 2024-04-25T00:33:14.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f355cbd7640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f35580cece0 0x7f355806f390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38836/0 (socket says 172.21.15.29:38836) 2024-04-25T00:33:14.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f355cbd7640 1 -- 172.21.15.29:0/3445676952 learned_addr learned my addr 172.21.15.29:0/3445676952 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:14.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3557fff640 1 --2- 172.21.15.29:0/3445676952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355815bf70 0x7f35580b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:14.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1162556146 0 0) 0x7f35580cdbe0 con 0x7f35580cece0 2024-04-25T00:33:14.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3544003530 con 0x7f35580cece0 2024-04-25T00:33:14.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3938566949 0 0) 0x7f3544003530 con 0x7f35580cece0 2024-04-25T00:33:14.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355815bf70 msgr2=0x7f35580b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:14.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 --2- 172.21.15.29:0/3445676952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355815bf70 0x7f35580b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 >> v1:172.21.15.157:6789/0 conn(0x7f355815c350 legacy=0x7f35580ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35580cfcc0 con 0x7f35580cece0 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f3550001680 con 0x7f35580cece0 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 >> v1:172.21.15.29:6789/0 conn(0x7f35580cece0 legacy=0x7f355806f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 --2- 172.21.15.29:0/3445676952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35440039f0 0x7f3544003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 --2- 172.21.15.29:0/3445676952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544004810 0x7f3544004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 --2- 172.21.15.29:0/3445676952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355815bf70 0x7f35440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f35580cf9e0 con 0x7f35440039f0 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f354400bb10 con 0x7f3544004810 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f354400bd10 con 0x7f355815bf70 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3557fff640 1 --2- 172.21.15.29:0/3445676952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35440039f0 0x7f3544003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f35577fe640 1 --2- 172.21.15.29:0/3445676952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544004810 0x7f3544004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f355cbd7640 1 --2- 172.21.15.29:0/3445676952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355815bf70 0x7f35440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3557fff640 1 -- 172.21.15.29:0/3445676952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355815bf70 msgr2=0x7f35440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3557fff640 1 --2- 172.21.15.29:0/3445676952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355815bf70 0x7f35440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3557fff640 1 -- 172.21.15.29:0/3445676952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544004810 msgr2=0x7f3544004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:14.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3557fff640 1 --2- 172.21.15.29:0/3445676952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544004810 0x7f3544004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f3557fff640 1 -- 172.21.15.29:0/3445676952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3544003530 con 0x7f35440039f0 2024-04-25T00:33:14.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.606+0000 7f355cbd7640 1 --2- 172.21.15.29:0/3445676952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f355815bf70 0x7f35440051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.610+0000 7f3557fff640 1 --2- 172.21.15.29:0/3445676952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35440039f0 0x7f3544003e10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3548009f90 tx=0x7f354801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bb5682336bda64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.610+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35480b2070 con 0x7f35440039f0 2024-04-25T00:33:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.610+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35480a5c30 con 0x7f35440039f0 2024-04-25T00:33:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.610+0000 7f3556ffd640 1 -- 172.21.15.29:0/3445676952 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548009c10 con 0x7f35440039f0 2024-04-25T00:33:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.610+0000 7f355e63a640 1 -- 172.21.15.29:0/3445676952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35440039f0 msgr2=0x7f3544003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.610+0000 7f355e63a640 1 --2- 172.21.15.29:0/3445676952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35440039f0 0x7f3544003e10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3548009f90 tx=0x7f354801cc20 comp rx=0 tx=0).stop 2024-04-25T00:33:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.610+0000 7f355cbd7640 1 -- 172.21.15.29:0/3445676952 reap_dead start 2024-04-25T00:33:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.610+0000 7f355e63a640 1 -- 172.21.15.29:0/3445676952 shutdown_connections 2024-04-25T00:33:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.610+0000 7f355e63a640 1 -- 172.21.15.29:0/3445676952 >> 172.21.15.29:0/3445676952 conn(0x7f35580b3920 msgr2=0x7f35580b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.610+0000 7f355e63a640 1 -- 172.21.15.29:0/3445676952 shutdown_connections 2024-04-25T00:33:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.610+0000 7f355e63a640 1 -- 172.21.15.29:0/3445676952 wait complete. 2024-04-25T00:33:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f355e63a640 1 Processor -- start 2024-04-25T00:33:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f355e63a640 1 -- start start 2024-04-25T00:33:14.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f355e63a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35440039f0 0x7f3558152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:14.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f355e63a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35580c58a0 0x7f355806d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:14.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f355e63a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f355806d970 0x7f355806de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:14.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f355e63a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f35580cf310 con 0x7f35440039f0 2024-04-25T00:33:14.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f355e63a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35580cf190 con 0x7f355806d970 2024-04-25T00:33:14.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f355e63a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f35580cf490 con 0x7f35580c58a0 2024-04-25T00:33:14.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f3557fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35440039f0 0x7f3558152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:14.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f3557fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35440039f0 0x7f3558152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59894/0 (socket says 172.21.15.29:59894) 2024-04-25T00:33:14.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f3557fff640 1 -- 172.21.15.29:0/395784450 learned_addr learned my addr 172.21.15.29:0/395784450 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:14.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f35577fe640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35580c58a0 0x7f355806d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:14.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f355cbd7640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f355806d970 0x7f355806de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:14.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f3557fff640 1 -- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35580c58a0 msgr2=0x7f355806d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f3557fff640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35580c58a0 0x7f355806d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f3557fff640 1 -- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f355806d970 msgr2=0x7f355806de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:14.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f3557fff640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f355806d970 0x7f355806de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:14.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f3557fff640 1 -- 172.21.15.29:0/395784450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35580b2050 con 0x7f35440039f0 2024-04-25T00:33:14.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f3557fff640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35440039f0 0x7f3558152590 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f354801ca50 tx=0x7f3548009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:14.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f3554ff9640 1 -- 172.21.15.29:0/395784450 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35480b2040 con 0x7f35440039f0 2024-04-25T00:33:14.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f3554ff9640 1 -- 172.21.15.29:0/395784450 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35480bd070 con 0x7f35440039f0 2024-04-25T00:33:14.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f3554ff9640 1 -- 172.21.15.29:0/395784450 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35480adc90 con 0x7f35440039f0 2024-04-25T00:33:14.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f355e63a640 1 -- 172.21.15.29:0/395784450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35580c60f0 con 0x7f35440039f0 2024-04-25T00:33:14.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f355e63a640 1 -- 172.21.15.29:0/395784450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35580c66b0 con 0x7f35440039f0 2024-04-25T00:33:14.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f355e63a640 1 -- 172.21.15.29:0/395784450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3558069270 con 0x7f35440039f0 2024-04-25T00:33:14.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.614+0000 7f3554ff9640 1 -- 172.21.15.29:0/395784450 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f35480c1410 con 0x7f35440039f0 2024-04-25T00:33:14.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.618+0000 7f3554ff9640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3538067aa0 0x7f3538069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:14.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.618+0000 7f3554ff9640 1 -- 172.21.15.29:0/395784450 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f3548120940 con 0x7f35440039f0 2024-04-25T00:33:14.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.618+0000 7f35577fe640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3538067aa0 0x7f3538069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:14.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.618+0000 7f35577fe640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3538067aa0 0x7f3538069f60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f35580c45a0 tx=0x7f3540009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:14.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.622+0000 7f3554ff9640 1 -- 172.21.15.29:0/395784450 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3548016930 con 0x7f35440039f0 2024-04-25T00:33:14.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:14.822+0000 7f355e63a640 1 -- 172.21.15.29:0/395784450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f355815f9c0 con 0x7f35440039f0 2024-04-25T00:33:15.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.798+0000 7f3554ff9640 1 -- 172.21.15.29:0/395784450 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7f35480f1320 con 0x7f35440039f0 2024-04-25T00:33:15.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.802+0000 7f355e63a640 1 -- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3538067aa0 msgr2=0x7f3538069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:15.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.802+0000 7f355e63a640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3538067aa0 0x7f3538069f60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f35580c45a0 tx=0x7f3540009040 comp rx=0 tx=0).stop 2024-04-25T00:33:15.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.802+0000 7f355e63a640 1 -- 172.21.15.29:0/395784450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35440039f0 msgr2=0x7f3558152590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:15.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.802+0000 7f355e63a640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35440039f0 0x7f3558152590 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f354801ca50 tx=0x7f3548009750 comp rx=0 tx=0).stop 2024-04-25T00:33:15.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.802+0000 7f355e63a640 1 -- 172.21.15.29:0/395784450 shutdown_connections 2024-04-25T00:33:15.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.802+0000 7f355e63a640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3538067aa0 0x7f3538069f60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:15.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.802+0000 7f355e63a640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f355806d970 0x7f355806de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:15.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.802+0000 7f355e63a640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35580c58a0 0x7f355806d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:15.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.802+0000 7f355e63a640 1 --2- 172.21.15.29:0/395784450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35440039f0 0x7f3558152590 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:15.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.802+0000 7f355e63a640 1 -- 172.21.15.29:0/395784450 >> 172.21.15.29:0/395784450 conn(0x7f35580b3920 msgr2=0x7f35580b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:15.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.802+0000 7f355e63a640 1 -- 172.21.15.29:0/395784450 shutdown_connections 2024-04-25T00:33:15.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.806+0000 7f355e63a640 1 -- 172.21.15.29:0/395784450 wait complete. 2024-04-25T00:33:15.833 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:33:15.834 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:33:15.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f6b10e640 1 Processor -- start 2024-04-25T00:33:15.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f6b10e640 1 -- start start 2024-04-25T00:33:15.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f6b10e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b4d30 0x7f6f640b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:15.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f6b10e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f6406d980 con 0x7f6f6406d210 2024-04-25T00:33:15.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f6b10e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f6406dc60 con 0x7f6f6415df70 2024-04-25T00:33:15.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f6b10e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6406df40 con 0x7f6f640b4d30 2024-04-25T00:33:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f696ab640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6f6406d210 0x7f6f640ad8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38842/0 (socket says 172.21.15.29:38842) 2024-04-25T00:33:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f696ab640 1 -- 172.21.15.29:0/1732523360 learned_addr learned my addr 172.21.15.29:0/1732523360 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f63fff640 1 --2- 172.21.15.29:0/1732523360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b4d30 0x7f6f640b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f637fe640 1 -- 172.21.15.29:0/1732523360 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1203677947 0 0) 0x7f6f6406d980 con 0x7f6f6406d210 2024-04-25T00:33:15.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f637fe640 1 -- 172.21.15.29:0/1732523360 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f4c003530 con 0x7f6f6406d210 2024-04-25T00:33:15.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f637fe640 1 -- 172.21.15.29:0/1732523360 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2648777517 0 0) 0x7f6f6406dc60 con 0x7f6f6415df70 2024-04-25T00:33:15.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f637fe640 1 -- 172.21.15.29:0/1732523360 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f6406d980 con 0x7f6f6415df70 2024-04-25T00:33:15.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f637fe640 1 -- 172.21.15.29:0/1732523360 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1062067111 0 0) 0x7f6f4c003530 con 0x7f6f6406d210 2024-04-25T00:33:15.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f63fff640 1 -- 172.21.15.29:0/1732523360 >> v1:172.21.15.157:6789/0 conn(0x7f6f6415df70 legacy=0x7f6f640b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:15.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f63fff640 1 -- 172.21.15.29:0/1732523360 >> v1:172.21.15.29:6789/0 conn(0x7f6f6406d210 legacy=0x7f6f640ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:15.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.962+0000 7f6f63fff640 1 -- 172.21.15.29:0/1732523360 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6406e220 con 0x7f6f640b4d30 2024-04-25T00:33:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.966+0000 7f6f63fff640 1 --2- 172.21.15.29:0/1732523360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b4d30 0x7f6f640b3e40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f580097f0 tx=0x7f6f5801c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a773f23abc90fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.966+0000 7f6f637fe640 1 -- 172.21.15.29:0/1732523360 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f580b0070 con 0x7f6f640b4d30 2024-04-25T00:33:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.966+0000 7f6f637fe640 1 -- 172.21.15.29:0/1732523360 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f580ab400 con 0x7f6f640b4d30 2024-04-25T00:33:15.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.966+0000 7f6f637fe640 1 -- 172.21.15.29:0/1732523360 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c001a00 con 0x7f6f640b4d30 2024-04-25T00:33:15.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.966+0000 7f6f6b10e640 1 -- 172.21.15.29:0/1732523360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b4d30 msgr2=0x7f6f640b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:15.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.966+0000 7f6f6b10e640 1 --2- 172.21.15.29:0/1732523360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b4d30 0x7f6f640b3e40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f580097f0 tx=0x7f6f5801c3d0 comp rx=0 tx=0).stop 2024-04-25T00:33:15.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.966+0000 7f6f6b10e640 1 -- 172.21.15.29:0/1732523360 shutdown_connections 2024-04-25T00:33:15.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.966+0000 7f6f6b10e640 1 --2- 172.21.15.29:0/1732523360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b4d30 0x7f6f640b3e40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:15.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.966+0000 7f6f6b10e640 1 -- 172.21.15.29:0/1732523360 >> 172.21.15.29:0/1732523360 conn(0x7f6f640b1870 msgr2=0x7f6f640b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:15.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.966+0000 7f6f6b10e640 1 -- 172.21.15.29:0/1732523360 shutdown_connections 2024-04-25T00:33:15.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.966+0000 7f6f6b10e640 1 -- 172.21.15.29:0/1732523360 wait complete. 2024-04-25T00:33:15.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.966+0000 7f6f6b10e640 1 Processor -- start 2024-04-25T00:33:15.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.966+0000 7f6f6b10e640 1 -- start start 2024-04-25T00:33:15.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f6b10e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6406d210 0x7f6f64157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:15.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f6b10e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f640b4d30 0x7f6f640b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:15.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f6b10e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6415df70 0x7f6f64156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:15.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f6b10e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6406eba0 con 0x7f6f640b4d30 2024-04-25T00:33:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f6b10e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6406ea20 con 0x7f6f6415df70 2024-04-25T00:33:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f6b10e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6416bbc0 con 0x7f6f6406d210 2024-04-25T00:33:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f63fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f640b4d30 0x7f6f640b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f63fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f640b4d30 0x7f6f640b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59896/0 (socket says 172.21.15.29:59896) 2024-04-25T00:33:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f63fff640 1 -- 172.21.15.29:0/663876775 learned_addr learned my addr 172.21.15.29:0/663876775 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f68eaa640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6406d210 0x7f6f64157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f696ab640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6415df70 0x7f6f64156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f63fff640 1 -- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6406d210 msgr2=0x7f6f64157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f63fff640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6406d210 0x7f6f64157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f63fff640 1 -- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6415df70 msgr2=0x7f6f64156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f63fff640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6415df70 0x7f6f64156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f63fff640 1 -- 172.21.15.29:0/663876775 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f641582a0 con 0x7f6f640b4d30 2024-04-25T00:33:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f696ab640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6415df70 0x7f6f64156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f63fff640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f640b4d30 0x7f6f640b0e80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6f580097f0 tx=0x7f6f58019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f617fa640 1 -- 172.21.15.29:0/663876775 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f580b0040 con 0x7f6f640b4d30 2024-04-25T00:33:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f617fa640 1 -- 172.21.15.29:0/663876775 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f580b4430 con 0x7f6f640b4d30 2024-04-25T00:33:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f6b10e640 1 -- 172.21.15.29:0/663876775 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f641555a0 con 0x7f6f640b4d30 2024-04-25T00:33:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f617fa640 1 -- 172.21.15.29:0/663876775 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f580b47e0 con 0x7f6f640b4d30 2024-04-25T00:33:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f6b10e640 1 -- 172.21.15.29:0/663876775 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f64155b60 con 0x7f6f640b4d30 2024-04-25T00:33:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f6b10e640 1 -- 172.21.15.29:0/663876775 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f64069270 con 0x7f6f640b4d30 2024-04-25T00:33:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f617fa640 1 -- 172.21.15.29:0/663876775 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f6f580bb020 con 0x7f6f640b4d30 2024-04-25T00:33:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f617fa640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f50067a70 0x7f6f50069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f617fa640 1 -- 172.21.15.29:0/663876775 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f6f58120510 con 0x7f6f640b4d30 2024-04-25T00:33:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.970+0000 7f6f68eaa640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f50067a70 0x7f6f50069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.974+0000 7f6f68eaa640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f50067a70 0x7f6f50069f30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6f640b0e10 tx=0x7f6f54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:15.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:15.974+0000 7f6f617fa640 1 -- 172.21.15.29:0/663876775 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6f58015ff0 con 0x7f6f640b4d30 2024-04-25T00:33:16.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.174+0000 7f6f6b10e640 1 -- 172.21.15.29:0/663876775 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6f640b3610 con 0x7f6f640b4d30 2024-04-25T00:33:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.806+0000 7f6f617fa640 1 -- 172.21.15.29:0/663876775 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f6f580f0cc0 con 0x7f6f640b4d30 2024-04-25T00:33:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.810+0000 7f6f6b10e640 1 -- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f50067a70 msgr2=0x7f6f50069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.810+0000 7f6f6b10e640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f50067a70 0x7f6f50069f30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6f640b0e10 tx=0x7f6f54009040 comp rx=0 tx=0).stop 2024-04-25T00:33:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.810+0000 7f6f6b10e640 1 -- 172.21.15.29:0/663876775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f640b4d30 msgr2=0x7f6f640b0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.810+0000 7f6f6b10e640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f640b4d30 0x7f6f640b0e80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6f580097f0 tx=0x7f6f58019170 comp rx=0 tx=0).stop 2024-04-25T00:33:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.810+0000 7f6f6b10e640 1 -- 172.21.15.29:0/663876775 shutdown_connections 2024-04-25T00:33:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.810+0000 7f6f6b10e640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f50067a70 0x7f6f50069f30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.810+0000 7f6f6b10e640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6415df70 0x7f6f64156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.810+0000 7f6f6b10e640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f640b4d30 0x7f6f640b0e80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.810+0000 7f6f6b10e640 1 --2- 172.21.15.29:0/663876775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6406d210 0x7f6f64157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.810+0000 7f6f6b10e640 1 -- 172.21.15.29:0/663876775 >> 172.21.15.29:0/663876775 conn(0x7f6f640b1870 msgr2=0x7f6f6415ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:16.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.810+0000 7f6f6b10e640 1 -- 172.21.15.29:0/663876775 shutdown_connections 2024-04-25T00:33:16.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.810+0000 7f6f6b10e640 1 -- 172.21.15.29:0/663876775 wait complete. 2024-04-25T00:33:16.836 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:33:16.836 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:33:16.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e4162d640 1 Processor -- start 2024-04-25T00:33:16.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e4162d640 1 -- start start 2024-04-25T00:33:16.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e4162d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5e3c15e550 0x7f5e3c0b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:16.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e4162d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5e3c0cf770 0x7f5e3c0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:16.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e4162d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e3c0cfc70 con 0x7f5e3c15e170 2024-04-25T00:33:16.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e4162d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5e3c0d1c60 con 0x7f5e3c15e550 2024-04-25T00:33:16.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e4162d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5e3c0d2220 con 0x7f5e3c0cf770 2024-04-25T00:33:16.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e33fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5e3c15e550 0x7f5e3c0b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:16.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e3affd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5e3c15e170 0x7f5e3c0d49b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38854/0 (socket says 172.21.15.29:38854) 2024-04-25T00:33:16.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e3affd640 1 -- 172.21.15.29:0/692443021 learned_addr learned my addr 172.21.15.29:0/692443021 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:16.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e3b7fe640 1 --2- 172.21.15.29:0/692443021 >> v2:172.21.15.157:3300/0 conn(0x7f5e3c0cf770 0x7f5e3c0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:16.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e33fff640 1 -- 172.21.15.29:0/692443021 >> v2:172.21.15.157:3300/0 conn(0x7f5e3c0cf770 msgr2=0x7f5e3c0d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:16.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e33fff640 1 --2- 172.21.15.29:0/692443021 >> v2:172.21.15.157:3300/0 conn(0x7f5e3c0cf770 0x7f5e3c0d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:16.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e33fff640 1 -- 172.21.15.29:0/692443021 >> v1:172.21.15.29:6789/0 conn(0x7f5e3c15e170 legacy=0x7f5e3c0d49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:16.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e3a7fc640 1 -- 172.21.15.29:0/692443021 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 426765459 0 0) 0x7f5e3c0cfc70 con 0x7f5e3c15e170 2024-04-25T00:33:16.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e33fff640 1 -- 172.21.15.29:0/692443021 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c0cff50 con 0x7f5e3c15e550 2024-04-25T00:33:16.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.962+0000 7f5e3b7fe640 1 --2- 172.21.15.29:0/692443021 >> v2:172.21.15.157:3300/0 conn(0x7f5e3c0cf770 0x7f5e3c0d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:16.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e33fff640 1 --2- 172.21.15.29:0/692443021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e3c15e550 0x7f5e3c0b0c10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0d2190 tx=0x7f5e2401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0de6b3791451e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:16.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e3a7fc640 1 -- 172.21.15.29:0/692443021 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e240b0070 con 0x7f5e3c15e550 2024-04-25T00:33:16.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e3a7fc640 1 -- 172.21.15.29:0/692443021 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e240ab600 con 0x7f5e3c15e550 2024-04-25T00:33:16.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e3a7fc640 1 -- 172.21.15.29:0/692443021 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e240ab9d0 con 0x7f5e3c15e550 2024-04-25T00:33:16.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 -- 172.21.15.29:0/692443021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e3c15e550 msgr2=0x7f5e3c0b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:16.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 --2- 172.21.15.29:0/692443021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e3c15e550 0x7f5e3c0b0c10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0d2190 tx=0x7f5e2401c3c0 comp rx=0 tx=0).stop 2024-04-25T00:33:16.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 -- 172.21.15.29:0/692443021 shutdown_connections 2024-04-25T00:33:16.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 --2- 172.21.15.29:0/692443021 >> v2:172.21.15.157:3300/0 conn(0x7f5e3c0cf770 0x7f5e3c0d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:16.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 --2- 172.21.15.29:0/692443021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e3c15e550 0x7f5e3c0b0c10 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:16.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 -- 172.21.15.29:0/692443021 >> 172.21.15.29:0/692443021 conn(0x7f5e3c09d320 msgr2=0x7f5e3c0d4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:16.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 -- 172.21.15.29:0/692443021 shutdown_connections 2024-04-25T00:33:16.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 -- 172.21.15.29:0/692443021 wait complete. 2024-04-25T00:33:16.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 Processor -- start 2024-04-25T00:33:16.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 -- start start 2024-04-25T00:33:16.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e3c0cf770 0x7f5e3c0b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:16.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3c15e170 0x7f5e3c0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:16.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3c15e550 0x7f5e3c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:16.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c16b3c0 con 0x7f5e3c0cf770 2024-04-25T00:33:16.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c0b1360 con 0x7f5e3c15e170 2024-04-25T00:33:16.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5e3c16b540 con 0x7f5e3c15e550 2024-04-25T00:33:16.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e3affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e3c0cf770 0x7f5e3c0b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:16.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e3affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e3c0cf770 0x7f5e3c0b8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59922/0 (socket says 172.21.15.29:59922) 2024-04-25T00:33:16.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e3affd640 1 -- 172.21.15.29:0/968032436 learned_addr learned my addr 172.21.15.29:0/968032436 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:16.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e33fff640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3c15e170 0x7f5e3c0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:16.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e3b7fe640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3c15e550 0x7f5e3c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:16.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e3affd640 1 -- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3c15e550 msgr2=0x7f5e3c0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:16.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e3affd640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3c15e550 0x7f5e3c0cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:16.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e3affd640 1 -- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3c15e170 msgr2=0x7f5e3c0cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:16.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e3affd640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3c15e170 0x7f5e3c0cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:16.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e3affd640 1 -- 172.21.15.29:0/968032436 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c158380 con 0x7f5e3c0cf770 2024-04-25T00:33:16.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e3affd640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e3c0cf770 0x7f5e3c0b8df0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00c520 tx=0x7f5e2c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:16.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e337fe640 1 -- 172.21.15.29:0/968032436 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c018070 con 0x7f5e3c0cf770 2024-04-25T00:33:16.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e337fe640 1 -- 172.21.15.29:0/968032436 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e2c0074e0 con 0x7f5e3c0cf770 2024-04-25T00:33:16.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 -- 172.21.15.29:0/968032436 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3c158660 con 0x7f5e3c0cf770 2024-04-25T00:33:16.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e337fe640 1 -- 172.21.15.29:0/968032436 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0049f0 con 0x7f5e3c0cf770 2024-04-25T00:33:16.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.966+0000 7f5e4162d640 1 -- 172.21.15.29:0/968032436 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e3c158c20 con 0x7f5e3c0cf770 2024-04-25T00:33:16.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.970+0000 7f5e4162d640 1 -- 172.21.15.29:0/968032436 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e3c069270 con 0x7f5e3c0cf770 2024-04-25T00:33:16.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.970+0000 7f5e337fe640 1 -- 172.21.15.29:0/968032436 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f5e2c01b070 con 0x7f5e3c0cf770 2024-04-25T00:33:16.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.970+0000 7f5e337fe640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5e1c067a70 0x7f5e1c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:16.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.970+0000 7f5e337fe640 1 -- 172.21.15.29:0/968032436 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f5e2c0853f0 con 0x7f5e3c0cf770 2024-04-25T00:33:16.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.970+0000 7f5e33fff640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5e1c067a70 0x7f5e1c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:16.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.970+0000 7f5e33fff640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5e1c067a70 0x7f5e1c069f30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5e240ad000 tx=0x7f5e240a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:16.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:16.974+0000 7f5e337fe640 1 -- 172.21.15.29:0/968032436 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5e2c055cb0 con 0x7f5e3c0cf770 2024-04-25T00:33:17.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.170+0000 7f5e4162d640 1 -- 172.21.15.29:0/968032436 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e3c0cb410 con 0x7f5e3c0cf770 2024-04-25T00:33:17.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.170+0000 7f5e337fe640 1 -- 172.21.15.29:0/968032436 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7f5e2c0555b0 con 0x7f5e3c0cf770 2024-04-25T00:33:17.176 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:17.177 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:33:17.177 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 24 2024-04-25T00:33:17.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.174+0000 7f5e4162d640 1 -- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5e1c067a70 msgr2=0x7f5e1c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.174+0000 7f5e4162d640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5e1c067a70 0x7f5e1c069f30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5e240ad000 tx=0x7f5e240a6000 comp rx=0 tx=0).stop 2024-04-25T00:33:17.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.174+0000 7f5e4162d640 1 -- 172.21.15.29:0/968032436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e3c0cf770 msgr2=0x7f5e3c0b8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.174+0000 7f5e4162d640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e3c0cf770 0x7f5e3c0b8df0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00c520 tx=0x7f5e2c00c9e0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.174+0000 7f5e4162d640 1 -- 172.21.15.29:0/968032436 shutdown_connections 2024-04-25T00:33:17.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.174+0000 7f5e4162d640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3c15e550 0x7f5e3c0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.174+0000 7f5e4162d640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5e1c067a70 0x7f5e1c069f30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.174+0000 7f5e4162d640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3c15e170 0x7f5e3c0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.174+0000 7f5e4162d640 1 --2- 172.21.15.29:0/968032436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e3c0cf770 0x7f5e3c0b8df0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.174+0000 7f5e4162d640 1 -- 172.21.15.29:0/968032436 >> 172.21.15.29:0/968032436 conn(0x7f5e3c09d320 msgr2=0x7f5e3c0b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:17.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.174+0000 7f5e4162d640 1 -- 172.21.15.29:0/968032436 shutdown_connections 2024-04-25T00:33:17.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.174+0000 7f5e4162d640 1 -- 172.21.15.29:0/968032436 wait complete. 2024-04-25T00:33:17.196 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:33:17.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.310+0000 7fdbfe1eb640 1 Processor -- start 2024-04-25T00:33:17.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.310+0000 7fdbfe1eb640 1 -- start start 2024-04-25T00:33:17.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.310+0000 7fdbfe1eb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdbf813d140 0x7fdbf813d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.310+0000 7fdbfe1eb640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbf806cf90 con 0x7fdbf80b2e80 2024-04-25T00:33:17.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.310+0000 7fdbfe1eb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbf806d270 con 0x7fdbf806e460 2024-04-25T00:33:17.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.310+0000 7fdbfe1eb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf806cd40 con 0x7fdbf813d140 2024-04-25T00:33:17.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.310+0000 7fdbf6ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdbf80b2e80 0x7fdbf806dd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38860/0 (socket says 172.21.15.29:38860) 2024-04-25T00:33:17.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.310+0000 7fdbf6ffd640 1 -- 172.21.15.29:0/1888798694 learned_addr learned my addr 172.21.15.29:0/1888798694 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:17.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.310+0000 7fdbf77fe640 1 --2- 172.21.15.29:0/1888798694 >> v2:172.21.15.157:3300/0 conn(0x7fdbf813d140 0x7fdbf813d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.310+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819151157 0 0) 0x7fdbf806cf90 con 0x7fdbf80b2e80 2024-04-25T00:33:17.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.310+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbe0003530 con 0x7fdbf80b2e80 2024-04-25T00:33:17.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1873259954 0 0) 0x7fdbe0003530 con 0x7fdbf80b2e80 2024-04-25T00:33:17.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 >> v2:172.21.15.157:3300/0 conn(0x7fdbf813d140 msgr2=0x7fdbf813d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:17.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 --2- 172.21.15.29:0/1888798694 >> v2:172.21.15.157:3300/0 conn(0x7fdbf813d140 0x7fdbf813d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 >> v1:172.21.15.157:6789/0 conn(0x7fdbf806e460 legacy=0x7fdbf815a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf806d550 con 0x7fdbf80b2e80 2024-04-25T00:33:17.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fdbec001680 con 0x7fdbf80b2e80 2024-04-25T00:33:17.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 >> v1:172.21.15.29:6789/0 conn(0x7fdbf80b2e80 legacy=0x7fdbf806dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 --2- 172.21.15.29:0/1888798694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbe00039c0 0x7fdbe0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 --2- 172.21.15.29:0/1888798694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00043e0 0x7fdbe00048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 --2- 172.21.15.29:0/1888798694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe00050b0 0x7fdbe000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf806cd40 con 0x7fdbe00043e0 2024-04-25T00:33:17.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe000bff0 con 0x7fdbe00050b0 2024-04-25T00:33:17.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe000c1f0 con 0x7fdbe00039c0 2024-04-25T00:33:17.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf6ffd640 1 --2- 172.21.15.29:0/1888798694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00043e0 0x7fdbe00048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf7fff640 1 --2- 172.21.15.29:0/1888798694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe00050b0 0x7fdbe000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf77fe640 1 --2- 172.21.15.29:0/1888798694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbe00039c0 0x7fdbe0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf6ffd640 1 -- 172.21.15.29:0/1888798694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbe00039c0 msgr2=0x7fdbe0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:17.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf6ffd640 1 --2- 172.21.15.29:0/1888798694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbe00039c0 0x7fdbe0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf6ffd640 1 -- 172.21.15.29:0/1888798694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe00050b0 msgr2=0x7fdbe000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:17.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf6ffd640 1 --2- 172.21.15.29:0/1888798694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe00050b0 0x7fdbe000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf7fff640 1 -- 172.21.15.29:0/1888798694 reap_dead start 2024-04-25T00:33:17.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf6ffd640 1 -- 172.21.15.29:0/1888798694 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe0003530 con 0x7fdbe00043e0 2024-04-25T00:33:17.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf6ffd640 1 --2- 172.21.15.29:0/1888798694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00043e0 0x7fdbe00048b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c240 tx=0x7fdbec01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9007e2992b3e2a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:17.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec007d20 con 0x7fdbe00043e0 2024-04-25T00:33:17.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbec0a9020 con 0x7fdbe00043e0 2024-04-25T00:33:17.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbf67fc640 1 -- 172.21.15.29:0/1888798694 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec005b00 con 0x7fdbe00043e0 2024-04-25T00:33:17.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/1888798694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00043e0 msgr2=0x7fdbe00048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbfe1eb640 1 --2- 172.21.15.29:0/1888798694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00043e0 0x7fdbe00048b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c240 tx=0x7fdbec01cc60 comp rx=0 tx=0).stop 2024-04-25T00:33:17.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/1888798694 shutdown_connections 2024-04-25T00:33:17.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbfe1eb640 1 --2- 172.21.15.29:0/1888798694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00043e0 0x7fdbe00048b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/1888798694 >> 172.21.15.29:0/1888798694 conn(0x7fdbf80b1870 msgr2=0x7fdbf80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:17.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/1888798694 shutdown_connections 2024-04-25T00:33:17.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.314+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/1888798694 wait complete. 2024-04-25T00:33:17.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbfe1eb640 1 Processor -- start 2024-04-25T00:33:17.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbfe1eb640 1 -- start start 2024-04-25T00:33:17.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbfe1eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe00043e0 0x7fdbf80af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbfe1eb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf8158970 0x7fdbf81570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbfe1eb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbf813be50 0x7fdbf80abe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbfe1eb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf806dd50 con 0x7fdbf8158970 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbfe1eb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf815b3c0 con 0x7fdbe00043e0 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbfe1eb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf806ded0 con 0x7fdbf813be50 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbf6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf8158970 0x7fdbf81570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbf6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf8158970 0x7fdbf81570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59940/0 (socket says 172.21.15.29:59940) 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbf6ffd640 1 -- 172.21.15.29:0/2061548606 learned_addr learned my addr 172.21.15.29:0/2061548606 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbf77fe640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe00043e0 0x7fdbf80af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbf7fff640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbf813be50 0x7fdbf80abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbf6ffd640 1 -- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbf813be50 msgr2=0x7fdbf80abe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbf6ffd640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbf813be50 0x7fdbf80abe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbf6ffd640 1 -- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe00043e0 msgr2=0x7fdbf80af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbf6ffd640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe00043e0 0x7fdbf80af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbf6ffd640 1 -- 172.21.15.29:0/2061548606 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf8157630 con 0x7fdbf8158970 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbf6ffd640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf8158970 0x7fdbf81570f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0ab040 tx=0x7fdbe0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:17.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbdffff640 1 -- 172.21.15.29:0/2061548606 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0b4070 con 0x7fdbf8158970 2024-04-25T00:33:17.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbdffff640 1 -- 172.21.15.29:0/2061548606 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbec0072d0 con 0x7fdbf8158970 2024-04-25T00:33:17.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/2061548606 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf81578d0 con 0x7fdbf8158970 2024-04-25T00:33:17.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbdffff640 1 -- 172.21.15.29:0/2061548606 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec007d20 con 0x7fdbf8158970 2024-04-25T00:33:17.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.318+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/2061548606 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf8157e90 con 0x7fdbf8158970 2024-04-25T00:33:17.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.322+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/2061548606 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf8069270 con 0x7fdbf8158970 2024-04-25T00:33:17.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.322+0000 7fdbdffff640 1 -- 172.21.15.29:0/2061548606 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fdbec0bf060 con 0x7fdbf8158970 2024-04-25T00:33:17.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.322+0000 7fdbdffff640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdbe4067a70 0x7fdbe4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.322+0000 7fdbdffff640 1 -- 172.21.15.29:0/2061548606 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fdbec11ef20 con 0x7fdbf8158970 2024-04-25T00:33:17.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.322+0000 7fdbf77fe640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdbe4067a70 0x7fdbe4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.322+0000 7fdbf77fe640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdbe4067a70 0x7fdbe4069f30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8158fc0 tx=0x7fdbe8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:17.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.326+0000 7fdbdffff640 1 -- 172.21.15.29:0/2061548606 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdbec016970 con 0x7fdbf8158970 2024-04-25T00:33:17.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.490+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/2061548606 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdbf80b2740 con 0x7fdbf8158970 2024-04-25T00:33:17.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.494+0000 7fdbdffff640 1 -- 172.21.15.29:0/2061548606 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16565 (secure 0 0 0) 0x7fdbec0ef6b0 con 0x7fdbf8158970 2024-04-25T00:33:17.498 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:17.498 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":45,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:33:14.784398+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:33:10.789862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:33:12.129841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:33:17.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.494+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdbe4067a70 msgr2=0x7fdbe4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.494+0000 7fdbfe1eb640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdbe4067a70 0x7fdbe4069f30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8158fc0 tx=0x7fdbe8008040 comp rx=0 tx=0).stop 2024-04-25T00:33:17.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.494+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/2061548606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf8158970 msgr2=0x7fdbf81570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.494+0000 7fdbfe1eb640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf8158970 0x7fdbf81570f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0ab040 tx=0x7fdbe0005840 comp rx=0 tx=0).stop 2024-04-25T00:33:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.494+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/2061548606 shutdown_connections 2024-04-25T00:33:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.494+0000 7fdbfe1eb640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbf813be50 0x7fdbf80abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.494+0000 7fdbfe1eb640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf8158970 0x7fdbf81570f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.494+0000 7fdbfe1eb640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdbe4067a70 0x7fdbe4069f30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.494+0000 7fdbfe1eb640 1 --2- 172.21.15.29:0/2061548606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe00043e0 0x7fdbf80af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.494+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/2061548606 >> 172.21.15.29:0/2061548606 conn(0x7fdbf80b1870 msgr2=0x7fdbf8151b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.494+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/2061548606 shutdown_connections 2024-04-25T00:33:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.494+0000 7fdbfe1eb640 1 -- 172.21.15.29:0/2061548606 wait complete. 2024-04-25T00:33:17.517 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:33:17.518 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:33:17.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f5154216640 1 Processor -- start 2024-04-25T00:33:17.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f5154216640 1 -- start start 2024-04-25T00:33:17.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f5154216640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f514c15e440 0x7f514c0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f5154216640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f514c0d0ff0 0x7f514c0cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f5154216640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f514c0cc3b0 con 0x7f514c06c950 2024-04-25T00:33:17.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f5154216640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f514c0b11d0 con 0x7f514c0d0ff0 2024-04-25T00:33:17.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f5154216640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f514c0cc650 con 0x7f514c15e440 2024-04-25T00:33:17.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f51527b3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f514c0d0ff0 0x7f514c0cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f51527b3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f514c0d0ff0 0x7f514c0cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43340/0 (socket says 172.21.15.29:43340) 2024-04-25T00:33:17.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f51527b3640 1 -- 172.21.15.29:0/2670532132 learned_addr learned my addr 172.21.15.29:0/2670532132 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:17.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f51517b1640 1 --2- 172.21.15.29:0/2670532132 >> v2:172.21.15.157:3300/0 conn(0x7f514c15e440 0x7f514c0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f51527b3640 1 -- 172.21.15.29:0/2670532132 >> v2:172.21.15.157:3300/0 conn(0x7f514c15e440 msgr2=0x7f514c0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:17.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f51527b3640 1 --2- 172.21.15.29:0/2670532132 >> v2:172.21.15.157:3300/0 conn(0x7f514c15e440 0x7f514c0d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f51527b3640 1 -- 172.21.15.29:0/2670532132 >> v1:172.21.15.157:6789/0 conn(0x7f514c06c950 legacy=0x7f514c06cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.638+0000 7f51527b3640 1 -- 172.21.15.29:0/2670532132 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514c0cc930 con 0x7f514c0d0ff0 2024-04-25T00:33:17.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f51527b3640 1 --2- 172.21.15.29:0/2670532132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f514c0d0ff0 0x7f514c0cd410 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f514c071f70 tx=0x7f514401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa2213429dcda3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:17.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f5150fb0640 1 -- 172.21.15.29:0/2670532132 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51440a57a0 con 0x7f514c0d0ff0 2024-04-25T00:33:17.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f5150fb0640 1 -- 172.21.15.29:0/2670532132 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51440a5940 con 0x7f514c0d0ff0 2024-04-25T00:33:17.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f5150fb0640 1 -- 172.21.15.29:0/2670532132 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51440a5d10 con 0x7f514c0d0ff0 2024-04-25T00:33:17.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f5154216640 1 -- 172.21.15.29:0/2670532132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f514c0d0ff0 msgr2=0x7f514c0cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f5154216640 1 --2- 172.21.15.29:0/2670532132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f514c0d0ff0 0x7f514c0cd410 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f514c071f70 tx=0x7f514401c9f0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f5154216640 1 -- 172.21.15.29:0/2670532132 shutdown_connections 2024-04-25T00:33:17.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f5154216640 1 --2- 172.21.15.29:0/2670532132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f514c0d0ff0 0x7f514c0cd410 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f5154216640 1 --2- 172.21.15.29:0/2670532132 >> v2:172.21.15.157:3300/0 conn(0x7f514c15e440 0x7f514c0d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f5154216640 1 -- 172.21.15.29:0/2670532132 >> 172.21.15.29:0/2670532132 conn(0x7f514c0d1d60 msgr2=0x7f514c0d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:17.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f5154216640 1 -- 172.21.15.29:0/2670532132 shutdown_connections 2024-04-25T00:33:17.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f5154216640 1 -- 172.21.15.29:0/2670532132 wait complete. 2024-04-25T00:33:17.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f5154216640 1 Processor -- start 2024-04-25T00:33:17.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.642+0000 7f5154216640 1 -- start start 2024-04-25T00:33:17.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f5154216640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c06c950 0x7f514c1538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f5154216640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f514c0d0ff0 0x7f514c0c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f5154216640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f514c15e440 0x7f514c15bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f5154216640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f514c0ce2e0 con 0x7f514c15e440 2024-04-25T00:33:17.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f5154216640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f514c0ce160 con 0x7f514c0d0ff0 2024-04-25T00:33:17.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f5154216640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f514c0ce460 con 0x7f514c06c950 2024-04-25T00:33:17.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f51527b3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f514c15e440 0x7f514c15bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f51527b3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f514c15e440 0x7f514c15bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43346/0 (socket says 172.21.15.29:43346) 2024-04-25T00:33:17.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f51527b3640 1 -- 172.21.15.29:0/2132158318 learned_addr learned my addr 172.21.15.29:0/2132158318 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:17.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f51517b1640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f514c0d0ff0 0x7f514c0c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f5151fb2640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c06c950 0x7f514c1538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f51527b3640 1 -- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c06c950 msgr2=0x7f514c1538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f51527b3640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c06c950 0x7f514c1538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f51527b3640 1 -- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f514c0d0ff0 msgr2=0x7f514c0c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f51527b3640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f514c0d0ff0 0x7f514c0c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f51527b3640 1 -- 172.21.15.29:0/2132158318 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514c0cd410 con 0x7f514c15e440 2024-04-25T00:33:17.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f51527b3640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f514c15e440 0x7f514c15bad0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f514401a620 tx=0x7f5144019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:17.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f513e7fc640 1 -- 172.21.15.29:0/2132158318 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51440b8670 con 0x7f514c15e440 2024-04-25T00:33:17.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f513e7fc640 1 -- 172.21.15.29:0/2132158318 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51440be070 con 0x7f514c15e440 2024-04-25T00:33:17.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f5154216640 1 -- 172.21.15.29:0/2132158318 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514c0c8e30 con 0x7f514c15e440 2024-04-25T00:33:17.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f513e7fc640 1 -- 172.21.15.29:0/2132158318 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51440b8a90 con 0x7f514c15e440 2024-04-25T00:33:17.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f5154216640 1 -- 172.21.15.29:0/2132158318 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f514c0c7b90 con 0x7f514c15e440 2024-04-25T00:33:17.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f5154216640 1 -- 172.21.15.29:0/2132158318 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514c069270 con 0x7f514c15e440 2024-04-25T00:33:17.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f513e7fc640 1 -- 172.21.15.29:0/2132158318 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f51440b8c30 con 0x7f514c15e440 2024-04-25T00:33:17.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f513e7fc640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5130067a70 0x7f5130069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:17.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.646+0000 7f513e7fc640 1 -- 172.21.15.29:0/2132158318 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f5144121200 con 0x7f514c15e440 2024-04-25T00:33:17.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.650+0000 7f5151fb2640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5130067a70 0x7f5130069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:17.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.650+0000 7f5151fb2640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5130067a70 0x7f5130069f30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f51480015b0 tx=0x7f5148009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:17.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.654+0000 7f513e7fc640 1 -- 172.21.15.29:0/2132158318 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f51440165e0 con 0x7f514c15e440 2024-04-25T00:33:17.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.806+0000 7f5154216640 1 -- 172.21.15.29:0/2132158318 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f514c159e50 con 0x7f5130067a70 2024-04-25T00:33:17.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.826+0000 7f513e7fc640 1 -- 172.21.15.29:0/2132158318 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246641 (secure 0 0 0) 0x7f514c159e50 con 0x7f5130067a70 2024-04-25T00:33:17.833 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:17.834 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:33:17.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.830+0000 7f5154216640 1 -- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5130067a70 msgr2=0x7f5130069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.830+0000 7f5154216640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5130067a70 0x7f5130069f30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f51480015b0 tx=0x7f5148009040 comp rx=0 tx=0).stop 2024-04-25T00:33:17.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.830+0000 7f5154216640 1 -- 172.21.15.29:0/2132158318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f514c15e440 msgr2=0x7f514c15bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:17.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.830+0000 7f5154216640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f514c15e440 0x7f514c15bad0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f514401a620 tx=0x7f5144019790 comp rx=0 tx=0).stop 2024-04-25T00:33:17.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.830+0000 7f5154216640 1 -- 172.21.15.29:0/2132158318 shutdown_connections 2024-04-25T00:33:17.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.830+0000 7f5154216640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f514c15e440 0x7f514c15bad0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.830+0000 7f5154216640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5130067a70 0x7f5130069f30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.830+0000 7f5154216640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f514c0d0ff0 0x7f514c0c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.830+0000 7f5154216640 1 --2- 172.21.15.29:0/2132158318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c06c950 0x7f514c1538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:17.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.830+0000 7f5154216640 1 -- 172.21.15.29:0/2132158318 >> 172.21.15.29:0/2132158318 conn(0x7f514c0d1d60 msgr2=0x7f514c15ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:17.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.830+0000 7f5154216640 1 -- 172.21.15.29:0/2132158318 shutdown_connections 2024-04-25T00:33:17.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.830+0000 7f5154216640 1 -- 172.21.15.29:0/2132158318 wait complete. 2024-04-25T00:33:17.849 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-04-25T00:33:17.480927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258736,"kb_used_data":5096,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715984,"statfs":{"total":773094113280,"available":772829167616,"internally_reserved":0,"allocated":5218304,"data_stored":2799840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.723742"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.786161+0000","last_change":"2024-04-25T00:33:12.786161+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.786161+0000","last_undegraded":"2024-04-25T00:33:12.786161+0000","last_fullsized":"2024-04-25T00:33:12.786161+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778155+0000","last_change":"2024-04-25T00:33:12.778155+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778155+0000","last_undegraded":"2024-04-25T00:33:12.778155+0000","last_fullsized":"2024-04-25T00:33:12.778155+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772955+0000","last_change":"2024-04-25T00:33:12.772955+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772955+0000","last_undegraded":"2024-04-25T00:33:12.772955+0000","last_fullsized":"2024-04-25T00:33:12.772955+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779053+0000","last_change":"2024-04-25T00:33:12.779053+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779053+0000","last_undegraded":"2024-04-25T00:33:12.779053+0000","last_fullsized":"2024-04-25T00:33:12.779053+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779330+0000","last_change":"2024-04-25T00:33:12.779330+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779330+0000","last_undegraded":"2024-04-25T00:33:12.779330+0000","last_fullsized":"2024-04-25T00:33:12.779330+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785451+0000","last_change":"2024-04-25T00:33:12.785451+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785451+0000","last_undegraded":"2024-04-25T00:33:12.785451+0000","last_fullsized":"2024-04-25T00:33:12.785451+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781016+0000","last_change":"2024-04-25T00:33:12.781016+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781016+0000","last_undegraded":"2024-04-25T00:33:12.781016+0000","last_fullsized":"2024-04-25T00:33:12.781016+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779106+0000","last_change":"2024-04-25T00:33:12.779106+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779106+0000","last_undegraded":"2024-04-25T00:33:12.779106+0000","last_fullsized":"2024-04-25T00:33:12.779106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776612+0000","last_change":"2024-04-25T00:33:12.776612+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776612+0000","last_undegraded":"2024-04-25T00:33:12.776612+0000","last_fullsized":"2024-04-25T00:33:12.776612+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.777743+0000","last_change":"2024-04-25T00:33:12.777743+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.777743+0000","last_undegraded":"2024-04-25T00:33:12.777743+0000","last_fullsized":"2024-04-25T00:33:12.777743+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773992+0000","last_change":"2024-04-25T00:33:12.773992+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773992+0000","last_undegraded":"2024-04-25T00:33:12.773992+0000","last_fullsized":"2024-04-25T00:33:12.773992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.786788+0000","last_change":"2024-04-25T00:33:12.786788+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.786788+0000","last_undegraded":"2024-04-25T00:33:12.786788+0000","last_fullsized":"2024-04-25T00:33:12.786788+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776214+0000","last_change":"2024-04-25T00:33:12.776214+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776214+0000","last_undegraded":"2024-04-25T00:33:12.776214+0000","last_fullsized":"2024-04-25T00:33:12.776214+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779230+0000","last_change":"2024-04-25T00:33:12.779230+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779230+0000","last_undegraded":"2024-04-25T00:33:12.779230+0000","last_fullsized":"2024-04-25T00:33:12.779230+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779729+0000","last_change":"2024-04-25T00:33:12.779729+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779729+0000","last_undegraded":"2024-04-25T00:33:12.779729+0000","last_fullsized":"2024-04-25T00:33:12.779729+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781755+0000","last_change":"2024-04-25T00:33:12.781755+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781755+0000","last_undegraded":"2024-04-25T00:33:12.781755+0000","last_fullsized":"2024-04-25T00:33:12.781755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.775241+0000","last_change":"2024-04-25T00:33:12.775241+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.775241+0000","last_undegraded":"2024-04-25T00:33:12.775241+0000","last_fullsized":"2024-04-25T00:33:12.775241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.774539+0000","last_change":"2024-04-25T00:33:12.774539+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.774539+0000","last_undegraded":"2024-04-25T00:33:12.774539+0000","last_fullsized":"2024-04-25T00:33:12.774539+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779300+0000","last_change":"2024-04-25T00:33:12.779300+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779300+0000","last_undegraded":"2024-04-25T00:33:12.779300+0000","last_fullsized":"2024-04-25T00:33:12.779300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776553+0000","last_change":"2024-04-25T00:33:12.776553+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776553+0000","last_undegraded":"2024-04-25T00:33:12.776553+0000","last_fullsized":"2024-04-25T00:33:12.776553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784989+0000","last_change":"2024-04-25T00:33:12.784989+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784989+0000","last_undegraded":"2024-04-25T00:33:12.784989+0000","last_fullsized":"2024-04-25T00:33:12.784989+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771659+0000","last_change":"2024-04-25T00:33:12.771659+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771659+0000","last_undegraded":"2024-04-25T00:33:12.771659+0000","last_fullsized":"2024-04-25T00:33:12.771659+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778391+0000","last_change":"2024-04-25T00:33:12.778391+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778391+0000","last_undegraded":"2024-04-25T00:33:12.778391+0000","last_fullsized":"2024-04-25T00:33:12.778391+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780527+0000","last_change":"2024-04-25T00:33:12.780527+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780527+0000","last_undegraded":"2024-04-25T00:33:12.780527+0000","last_fullsized":"2024-04-25T00:33:12.780527+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.786781+0000","last_change":"2024-04-25T00:33:12.786781+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.786781+0000","last_undegraded":"2024-04-25T00:33:12.786781+0000","last_fullsized":"2024-04-25T00:33:12.786781+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776393+0000","last_change":"2024-04-25T00:33:12.776393+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776393+0000","last_undegraded":"2024-04-25T00:33:12.776393+0000","last_fullsized":"2024-04-25T00:33:12.776393+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771755+0000","last_change":"2024-04-25T00:33:12.771755+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771755+0000","last_undegraded":"2024-04-25T00:33:12.771755+0000","last_fullsized":"2024-04-25T00:33:12.771755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785983+0000","last_change":"2024-04-25T00:33:12.785983+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785983+0000","last_undegraded":"2024-04-25T00:33:12.785983+0000","last_fullsized":"2024-04-25T00:33:12.785983+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781215+0000","last_change":"2024-04-25T00:33:12.781215+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781215+0000","last_undegraded":"2024-04-25T00:33:12.781215+0000","last_fullsized":"2024-04-25T00:33:12.781215+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773770+0000","last_change":"2024-04-25T00:33:12.773770+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773770+0000","last_undegraded":"2024-04-25T00:33:12.773770+0000","last_fullsized":"2024-04-25T00:33:12.773770+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778731+0000","last_change":"2024-04-25T00:33:12.778731+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778731+0000","last_undegraded":"2024-04-25T00:33:12.778731+0000","last_fullsized":"2024-04-25T00:33:12.778731+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781274+0000","last_change":"2024-04-25T00:33:12.781274+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781274+0000","last_undegraded":"2024-04-25T00:33:12.781274+0000","last_fullsized":"2024-04-25T00:33:12.781274+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.783167+0000","last_change":"2024-04-25T00:33:12.783167+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.783167+0000","last_undegraded":"2024-04-25T00:33:12.783167+0000","last_fullsized":"2024-04-25T00:33:12.783167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802167+0000","last_change":"2024-04-25T00:33:12.801022+0000","last_active":"2024-04-25T00:33:12.802167+0000","last_peered":"2024-04-25T00:33:12.802167+0000","last_clean":"2024-04-25T00:33:12.802167+0000","last_became_active":"2024-04-25T00:33:12.800641+0000","last_became_peered":"2024-04-25T00:33:12.800641+0000","last_unstale":"2024-04-25T00:33:12.802167+0000","last_undegraded":"2024-04-25T00:33:12.802167+0000","last_fullsized":"2024-04-25T00:33:12.802167+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:10.074900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785776+0000","last_change":"2024-04-25T00:33:12.785776+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785776+0000","last_undegraded":"2024-04-25T00:33:12.785776+0000","last_fullsized":"2024-04-25T00:33:12.785776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799161+0000","last_change":"2024-04-25T00:33:12.797818+0000","last_active":"2024-04-25T00:33:12.799161+0000","last_peered":"2024-04-25T00:33:12.799161+0000","last_clean":"2024-04-25T00:33:12.799161+0000","last_became_active":"2024-04-25T00:33:12.796615+0000","last_became_peered":"2024-04-25T00:33:12.796615+0000","last_unstale":"2024-04-25T00:33:12.799161+0000","last_undegraded":"2024-04-25T00:33:12.799161+0000","last_fullsized":"2024-04-25T00:33:12.799161+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:38.919514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785220+0000","last_change":"2024-04-25T00:33:12.785220+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785220+0000","last_undegraded":"2024-04-25T00:33:12.785220+0000","last_fullsized":"2024-04-25T00:33:12.785220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799612+0000","last_change":"2024-04-25T00:33:12.797893+0000","last_active":"2024-04-25T00:33:12.799612+0000","last_peered":"2024-04-25T00:33:12.799612+0000","last_clean":"2024-04-25T00:33:12.799612+0000","last_became_active":"2024-04-25T00:33:12.797499+0000","last_became_peered":"2024-04-25T00:33:12.797499+0000","last_unstale":"2024-04-25T00:33:12.799612+0000","last_undegraded":"2024-04-25T00:33:12.799612+0000","last_fullsized":"2024-04-25T00:33:12.799612+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:54.884884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780797+0000","last_change":"2024-04-25T00:33:12.780797+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780797+0000","last_undegraded":"2024-04-25T00:33:12.780797+0000","last_fullsized":"2024-04-25T00:33:12.780797+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.809109+0000","last_change":"2024-04-25T00:33:12.808013+0000","last_active":"2024-04-25T00:33:12.809109+0000","last_peered":"2024-04-25T00:33:12.809109+0000","last_clean":"2024-04-25T00:33:12.809109+0000","last_became_active":"2024-04-25T00:33:12.807497+0000","last_became_peered":"2024-04-25T00:33:12.807497+0000","last_unstale":"2024-04-25T00:33:12.809109+0000","last_undegraded":"2024-04-25T00:33:12.809109+0000","last_fullsized":"2024-04-25T00:33:12.809109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:13.091178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772640+0000","last_change":"2024-04-25T00:33:12.772640+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772640+0000","last_undegraded":"2024-04-25T00:33:12.772640+0000","last_fullsized":"2024-04-25T00:33:12.772640+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.803068+0000","last_change":"2024-04-25T00:33:12.801886+0000","last_active":"2024-04-25T00:33:12.803068+0000","last_peered":"2024-04-25T00:33:12.803068+0000","last_clean":"2024-04-25T00:33:12.803068+0000","last_became_active":"2024-04-25T00:33:12.801570+0000","last_became_peered":"2024-04-25T00:33:12.801570+0000","last_unstale":"2024-04-25T00:33:12.803068+0000","last_undegraded":"2024-04-25T00:33:12.803068+0000","last_fullsized":"2024-04-25T00:33:12.803068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:37.757927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772470+0000","last_change":"2024-04-25T00:33:12.772470+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772470+0000","last_undegraded":"2024-04-25T00:33:12.772470+0000","last_fullsized":"2024-04-25T00:33:12.772470+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.797304+0000","last_change":"2024-04-25T00:33:12.795400+0000","last_active":"2024-04-25T00:33:12.797304+0000","last_peered":"2024-04-25T00:33:12.797304+0000","last_clean":"2024-04-25T00:33:12.797304+0000","last_became_active":"2024-04-25T00:33:12.794892+0000","last_became_peered":"2024-04-25T00:33:12.794892+0000","last_unstale":"2024-04-25T00:33:12.797304+0000","last_undegraded":"2024-04-25T00:33:12.797304+0000","last_fullsized":"2024-04-25T00:33:12.797304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:11.471922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772301+0000","last_change":"2024-04-25T00:33:12.772301+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772301+0000","last_undegraded":"2024-04-25T00:33:12.772301+0000","last_fullsized":"2024-04-25T00:33:12.772301+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.801950+0000","last_change":"2024-04-25T00:33:12.794993+0000","last_active":"2024-04-25T00:33:12.801950+0000","last_peered":"2024-04-25T00:33:12.801950+0000","last_clean":"2024-04-25T00:33:12.801950+0000","last_became_active":"2024-04-25T00:33:12.794251+0000","last_became_peered":"2024-04-25T00:33:12.794251+0000","last_unstale":"2024-04-25T00:33:12.801950+0000","last_undegraded":"2024-04-25T00:33:12.801950+0000","last_fullsized":"2024-04-25T00:33:12.801950+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:05.611837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779496+0000","last_change":"2024-04-25T00:33:12.779496+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779496+0000","last_undegraded":"2024-04-25T00:33:12.779496+0000","last_fullsized":"2024-04-25T00:33:12.779496+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.795732+0000","last_change":"2024-04-25T00:33:12.793905+0000","last_active":"2024-04-25T00:33:12.795732+0000","last_peered":"2024-04-25T00:33:12.795732+0000","last_clean":"2024-04-25T00:33:12.795732+0000","last_became_active":"2024-04-25T00:33:12.791067+0000","last_became_peered":"2024-04-25T00:33:12.791067+0000","last_unstale":"2024-04-25T00:33:12.795732+0000","last_undegraded":"2024-04-25T00:33:12.795732+0000","last_fullsized":"2024-04-25T00:33:12.795732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:26.419741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773456+0000","last_change":"2024-04-25T00:33:12.773456+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773456+0000","last_undegraded":"2024-04-25T00:33:12.773456+0000","last_fullsized":"2024-04-25T00:33:12.773456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.793147+0000","last_change":"2024-04-25T00:33:12.791338+0000","last_active":"2024-04-25T00:33:12.793147+0000","last_peered":"2024-04-25T00:33:12.793147+0000","last_clean":"2024-04-25T00:33:12.793147+0000","last_became_active":"2024-04-25T00:33:12.790547+0000","last_became_peered":"2024-04-25T00:33:12.790547+0000","last_unstale":"2024-04-25T00:33:12.793147+0000","last_undegraded":"2024-04-25T00:33:12.793147+0000","last_fullsized":"2024-04-25T00:33:12.793147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:42.648522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785118+0000","last_change":"2024-04-25T00:33:12.785118+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785118+0000","last_undegraded":"2024-04-25T00:33:12.785118+0000","last_fullsized":"2024-04-25T00:33:12.785118+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802404+0000","last_change":"2024-04-25T00:33:12.800591+0000","last_active":"2024-04-25T00:33:12.802404+0000","last_peered":"2024-04-25T00:33:12.802404+0000","last_clean":"2024-04-25T00:33:12.802404+0000","last_became_active":"2024-04-25T00:33:12.800301+0000","last_became_peered":"2024-04-25T00:33:12.800301+0000","last_unstale":"2024-04-25T00:33:12.802404+0000","last_undegraded":"2024-04-25T00:33:12.802404+0000","last_fullsized":"2024-04-25T00:33:12.802404+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:51.270346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.782338+0000","last_change":"2024-04-25T00:33:12.782338+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.782338+0000","last_undegraded":"2024-04-25T00:33:12.782338+0000","last_fullsized":"2024-04-25T00:33:12.782338+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.811725+0000","last_change":"2024-04-25T00:33:12.810318+0000","last_active":"2024-04-25T00:33:12.811725+0000","last_peered":"2024-04-25T00:33:12.811725+0000","last_clean":"2024-04-25T00:33:12.811725+0000","last_became_active":"2024-04-25T00:33:12.809691+0000","last_became_peered":"2024-04-25T00:33:12.809691+0000","last_unstale":"2024-04-25T00:33:12.811725+0000","last_undegraded":"2024-04-25T00:33:12.811725+0000","last_fullsized":"2024-04-25T00:33:12.811725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:31.459566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784334+0000","last_change":"2024-04-25T00:33:12.784334+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784334+0000","last_undegraded":"2024-04-25T00:33:12.784334+0000","last_fullsized":"2024-04-25T00:33:12.784334+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.788652+0000","last_change":"2024-04-25T00:33:12.787615+0000","last_active":"2024-04-25T00:33:12.788652+0000","last_peered":"2024-04-25T00:33:12.788652+0000","last_clean":"2024-04-25T00:33:12.788652+0000","last_became_active":"2024-04-25T00:33:12.787329+0000","last_became_peered":"2024-04-25T00:33:12.787329+0000","last_unstale":"2024-04-25T00:33:12.788652+0000","last_undegraded":"2024-04-25T00:33:12.788652+0000","last_fullsized":"2024-04-25T00:33:12.788652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:26.034657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778244+0000","last_change":"2024-04-25T00:33:12.778244+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778244+0000","last_undegraded":"2024-04-25T00:33:12.778244+0000","last_fullsized":"2024-04-25T00:33:12.778244+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.798527+0000","last_change":"2024-04-25T00:33:12.795864+0000","last_active":"2024-04-25T00:33:12.798527+0000","last_peered":"2024-04-25T00:33:12.798527+0000","last_clean":"2024-04-25T00:33:12.798527+0000","last_became_active":"2024-04-25T00:33:12.795591+0000","last_became_peered":"2024-04-25T00:33:12.795591+0000","last_unstale":"2024-04-25T00:33:12.798527+0000","last_undegraded":"2024-04-25T00:33:12.798527+0000","last_fullsized":"2024-04-25T00:33:12.798527+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:41.044585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771469+0000","last_change":"2024-04-25T00:33:12.771469+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771469+0000","last_undegraded":"2024-04-25T00:33:12.771469+0000","last_fullsized":"2024-04-25T00:33:12.771469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.807386+0000","last_change":"2024-04-25T00:33:12.806163+0000","last_active":"2024-04-25T00:33:12.807386+0000","last_peered":"2024-04-25T00:33:12.807386+0000","last_clean":"2024-04-25T00:33:12.807386+0000","last_became_active":"2024-04-25T00:33:12.802549+0000","last_became_peered":"2024-04-25T00:33:12.802549+0000","last_unstale":"2024-04-25T00:33:12.807386+0000","last_undegraded":"2024-04-25T00:33:12.807386+0000","last_fullsized":"2024-04-25T00:33:12.807386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:41.044585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.777997+0000","last_change":"2024-04-25T00:33:12.777997+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.777997+0000","last_undegraded":"2024-04-25T00:33:12.777997+0000","last_fullsized":"2024-04-25T00:33:12.777997+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799122+0000","last_change":"2024-04-25T00:33:12.797153+0000","last_active":"2024-04-25T00:33:12.799122+0000","last_peered":"2024-04-25T00:33:12.799122+0000","last_clean":"2024-04-25T00:33:12.799122+0000","last_became_active":"2024-04-25T00:33:12.796727+0000","last_became_peered":"2024-04-25T00:33:12.796727+0000","last_unstale":"2024-04-25T00:33:12.799122+0000","last_undegraded":"2024-04-25T00:33:12.799122+0000","last_fullsized":"2024-04-25T00:33:12.799122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:13.091178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779937+0000","last_change":"2024-04-25T00:33:12.779937+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779937+0000","last_undegraded":"2024-04-25T00:33:12.779937+0000","last_fullsized":"2024-04-25T00:33:12.779937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.800089+0000","last_change":"2024-04-25T00:33:12.799231+0000","last_active":"2024-04-25T00:33:12.800089+0000","last_peered":"2024-04-25T00:33:12.800089+0000","last_clean":"2024-04-25T00:33:12.800089+0000","last_became_active":"2024-04-25T00:33:12.798864+0000","last_became_peered":"2024-04-25T00:33:12.798864+0000","last_unstale":"2024-04-25T00:33:12.800089+0000","last_undegraded":"2024-04-25T00:33:12.800089+0000","last_fullsized":"2024-04-25T00:33:12.800089+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:56.668429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.775446+0000","last_change":"2024-04-25T00:33:12.775446+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.775446+0000","last_undegraded":"2024-04-25T00:33:12.775446+0000","last_fullsized":"2024-04-25T00:33:12.775446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.792247+0000","last_change":"2024-04-25T00:33:12.788985+0000","last_active":"2024-04-25T00:33:12.792247+0000","last_peered":"2024-04-25T00:33:12.792247+0000","last_clean":"2024-04-25T00:33:12.792247+0000","last_became_active":"2024-04-25T00:33:12.788443+0000","last_became_peered":"2024-04-25T00:33:12.788443+0000","last_unstale":"2024-04-25T00:33:12.792247+0000","last_undegraded":"2024-04-25T00:33:12.792247+0000","last_fullsized":"2024-04-25T00:33:12.792247+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:33.029999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773102+0000","last_change":"2024-04-25T00:33:12.773102+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773102+0000","last_undegraded":"2024-04-25T00:33:12.773102+0000","last_fullsized":"2024-04-25T00:33:12.773102+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.797229+0000","last_change":"2024-04-25T00:33:12.794976+0000","last_active":"2024-04-25T00:33:12.797229+0000","last_peered":"2024-04-25T00:33:12.797229+0000","last_clean":"2024-04-25T00:33:12.797229+0000","last_became_active":"2024-04-25T00:33:12.793226+0000","last_became_peered":"2024-04-25T00:33:12.793226+0000","last_unstale":"2024-04-25T00:33:12.797229+0000","last_undegraded":"2024-04-25T00:33:12.797229+0000","last_fullsized":"2024-04-25T00:33:12.797229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:52.568911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780454+0000","last_change":"2024-04-25T00:33:12.780454+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780454+0000","last_undegraded":"2024-04-25T00:33:12.780454+0000","last_fullsized":"2024-04-25T00:33:12.780454+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802028+0000","last_change":"2024-04-25T00:33:12.794824+0000","last_active":"2024-04-25T00:33:12.802028+0000","last_peered":"2024-04-25T00:33:12.802028+0000","last_clean":"2024-04-25T00:33:12.802028+0000","last_became_active":"2024-04-25T00:33:12.794088+0000","last_became_peered":"2024-04-25T00:33:12.794088+0000","last_unstale":"2024-04-25T00:33:12.802028+0000","last_undegraded":"2024-04-25T00:33:12.802028+0000","last_fullsized":"2024-04-25T00:33:12.802028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:33.703646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785147+0000","last_change":"2024-04-25T00:33:12.785147+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785147+0000","last_undegraded":"2024-04-25T00:33:12.785147+0000","last_fullsized":"2024-04-25T00:33:12.785147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802238+0000","last_change":"2024-04-25T00:33:12.801012+0000","last_active":"2024-04-25T00:33:12.802238+0000","last_peered":"2024-04-25T00:33:12.802238+0000","last_clean":"2024-04-25T00:33:12.802238+0000","last_became_active":"2024-04-25T00:33:12.800555+0000","last_became_peered":"2024-04-25T00:33:12.800555+0000","last_unstale":"2024-04-25T00:33:12.802238+0000","last_undegraded":"2024-04-25T00:33:12.802238+0000","last_fullsized":"2024-04-25T00:33:12.802238+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:31.459566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785063+0000","last_change":"2024-04-25T00:33:12.785063+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785063+0000","last_undegraded":"2024-04-25T00:33:12.785063+0000","last_fullsized":"2024-04-25T00:33:12.785063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802176+0000","last_change":"2024-04-25T00:33:12.801088+0000","last_active":"2024-04-25T00:33:12.802176+0000","last_peered":"2024-04-25T00:33:12.802176+0000","last_clean":"2024-04-25T00:33:12.802176+0000","last_became_active":"2024-04-25T00:33:12.800686+0000","last_became_peered":"2024-04-25T00:33:12.800686+0000","last_unstale":"2024-04-25T00:33:12.802176+0000","last_undegraded":"2024-04-25T00:33:12.802176+0000","last_fullsized":"2024-04-25T00:33:12.802176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:04.740368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785871+0000","last_change":"2024-04-25T00:33:12.785871+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785871+0000","last_undegraded":"2024-04-25T00:33:12.785871+0000","last_fullsized":"2024-04-25T00:33:12.785871+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.806654+0000","last_change":"2024-04-25T00:33:12.804216+0000","last_active":"2024-04-25T00:33:12.806654+0000","last_peered":"2024-04-25T00:33:12.806654+0000","last_clean":"2024-04-25T00:33:12.806654+0000","last_became_active":"2024-04-25T00:33:12.800818+0000","last_became_peered":"2024-04-25T00:33:12.800818+0000","last_unstale":"2024-04-25T00:33:12.806654+0000","last_undegraded":"2024-04-25T00:33:12.806654+0000","last_fullsized":"2024-04-25T00:33:12.806654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:04.901463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780423+0000","last_change":"2024-04-25T00:33:12.780423+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780423+0000","last_undegraded":"2024-04-25T00:33:12.780423+0000","last_fullsized":"2024-04-25T00:33:12.780423+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.795832+0000","last_change":"2024-04-25T00:33:12.793774+0000","last_active":"2024-04-25T00:33:12.795832+0000","last_peered":"2024-04-25T00:33:12.795832+0000","last_clean":"2024-04-25T00:33:12.795832+0000","last_became_active":"2024-04-25T00:33:12.793317+0000","last_became_peered":"2024-04-25T00:33:12.793317+0000","last_unstale":"2024-04-25T00:33:12.795832+0000","last_undegraded":"2024-04-25T00:33:12.795832+0000","last_fullsized":"2024-04-25T00:33:12.795832+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:18.045380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773072+0000","last_change":"2024-04-25T00:33:12.773072+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773072+0000","last_undegraded":"2024-04-25T00:33:12.773072+0000","last_fullsized":"2024-04-25T00:33:12.773072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.807370+0000","last_change":"2024-04-25T00:33:12.805396+0000","last_active":"2024-04-25T00:33:12.807370+0000","last_peered":"2024-04-25T00:33:12.807370+0000","last_clean":"2024-04-25T00:33:12.807370+0000","last_became_active":"2024-04-25T00:33:12.803923+0000","last_became_peered":"2024-04-25T00:33:12.803923+0000","last_unstale":"2024-04-25T00:33:12.807370+0000","last_undegraded":"2024-04-25T00:33:12.807370+0000","last_fullsized":"2024-04-25T00:33:12.807370+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.189080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780611+0000","last_change":"2024-04-25T00:33:12.780611+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780611+0000","last_undegraded":"2024-04-25T00:33:12.780611+0000","last_fullsized":"2024-04-25T00:33:12.780611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.798496+0000","last_change":"2024-04-25T00:33:12.795318+0000","last_active":"2024-04-25T00:33:12.798496+0000","last_peered":"2024-04-25T00:33:12.798496+0000","last_clean":"2024-04-25T00:33:12.798496+0000","last_became_active":"2024-04-25T00:33:12.794708+0000","last_became_peered":"2024-04-25T00:33:12.794708+0000","last_unstale":"2024-04-25T00:33:12.798496+0000","last_undegraded":"2024-04-25T00:33:12.798496+0000","last_fullsized":"2024-04-25T00:33:12.798496+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.189080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773274+0000","last_change":"2024-04-25T00:33:12.773274+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773274+0000","last_undegraded":"2024-04-25T00:33:12.773274+0000","last_fullsized":"2024-04-25T00:33:12.773274+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.812310+0000","last_change":"2024-04-25T00:33:12.810928+0000","last_active":"2024-04-25T00:33:12.812310+0000","last_peered":"2024-04-25T00:33:12.812310+0000","last_clean":"2024-04-25T00:33:12.812310+0000","last_became_active":"2024-04-25T00:33:12.810284+0000","last_became_peered":"2024-04-25T00:33:12.810284+0000","last_unstale":"2024-04-25T00:33:12.812310+0000","last_undegraded":"2024-04-25T00:33:12.812310+0000","last_fullsized":"2024-04-25T00:33:12.812310+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:04.740368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.783434+0000","last_change":"2024-04-25T00:33:12.783434+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.783434+0000","last_undegraded":"2024-04-25T00:33:12.783434+0000","last_fullsized":"2024-04-25T00:33:12.783434+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.804755+0000","last_change":"2024-04-25T00:33:12.803739+0000","last_active":"2024-04-25T00:33:12.804755+0000","last_peered":"2024-04-25T00:33:12.804755+0000","last_clean":"2024-04-25T00:33:12.804755+0000","last_became_active":"2024-04-25T00:33:12.803381+0000","last_became_peered":"2024-04-25T00:33:12.803381+0000","last_unstale":"2024-04-25T00:33:12.804755+0000","last_undegraded":"2024-04-25T00:33:12.804755+0000","last_fullsized":"2024-04-25T00:33:12.804755+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:46.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784539+0000","last_change":"2024-04-25T00:33:12.784539+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784539+0000","last_undegraded":"2024-04-25T00:33:12.784539+0000","last_fullsized":"2024-04-25T00:33:12.784539+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.811877+0000","last_change":"2024-04-25T00:33:12.810436+0000","last_active":"2024-04-25T00:33:12.811877+0000","last_peered":"2024-04-25T00:33:12.811877+0000","last_clean":"2024-04-25T00:33:12.811877+0000","last_became_active":"2024-04-25T00:33:12.809773+0000","last_became_peered":"2024-04-25T00:33:12.809773+0000","last_unstale":"2024-04-25T00:33:12.811877+0000","last_undegraded":"2024-04-25T00:33:12.811877+0000","last_fullsized":"2024-04-25T00:33:12.811877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:10.074900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784359+0000","last_change":"2024-04-25T00:33:12.784359+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784359+0000","last_undegraded":"2024-04-25T00:33:12.784359+0000","last_fullsized":"2024-04-25T00:33:12.784359+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.793065+0000","last_change":"2024-04-25T00:33:12.791171+0000","last_active":"2024-04-25T00:33:12.793065+0000","last_peered":"2024-04-25T00:33:12.793065+0000","last_clean":"2024-04-25T00:33:12.793065+0000","last_became_active":"2024-04-25T00:33:12.790272+0000","last_became_peered":"2024-04-25T00:33:12.790272+0000","last_unstale":"2024-04-25T00:33:12.793065+0000","last_undegraded":"2024-04-25T00:33:12.793065+0000","last_fullsized":"2024-04-25T00:33:12.793065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:33.246491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779265+0000","last_change":"2024-04-25T00:33:12.779265+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779265+0000","last_undegraded":"2024-04-25T00:33:12.779265+0000","last_fullsized":"2024-04-25T00:33:12.779265+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799547+0000","last_change":"2024-04-25T00:33:12.797798+0000","last_active":"2024-04-25T00:33:12.799547+0000","last_peered":"2024-04-25T00:33:12.799547+0000","last_clean":"2024-04-25T00:33:12.799547+0000","last_became_active":"2024-04-25T00:33:12.797086+0000","last_became_peered":"2024-04-25T00:33:12.797086+0000","last_unstale":"2024-04-25T00:33:12.799547+0000","last_undegraded":"2024-04-25T00:33:12.799547+0000","last_fullsized":"2024-04-25T00:33:12.799547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:38.919514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771834+0000","last_change":"2024-04-25T00:33:12.771834+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771834+0000","last_undegraded":"2024-04-25T00:33:12.771834+0000","last_fullsized":"2024-04-25T00:33:12.771834+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.795110+0000","last_change":"2024-04-25T00:33:12.789836+0000","last_active":"2024-04-25T00:33:12.795110+0000","last_peered":"2024-04-25T00:33:12.795110+0000","last_clean":"2024-04-25T00:33:12.795110+0000","last_became_active":"2024-04-25T00:33:12.789493+0000","last_became_peered":"2024-04-25T00:33:12.789493+0000","last_unstale":"2024-04-25T00:33:12.795110+0000","last_undegraded":"2024-04-25T00:33:12.795110+0000","last_fullsized":"2024-04-25T00:33:12.795110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:03.396230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.777158+0000","last_change":"2024-04-25T00:33:12.777158+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.777158+0000","last_undegraded":"2024-04-25T00:33:12.777158+0000","last_fullsized":"2024-04-25T00:33:12.777158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802012+0000","last_change":"2024-04-25T00:33:12.795705+0000","last_active":"2024-04-25T00:33:12.802012+0000","last_peered":"2024-04-25T00:33:12.802012+0000","last_clean":"2024-04-25T00:33:12.802012+0000","last_became_active":"2024-04-25T00:33:12.795091+0000","last_became_peered":"2024-04-25T00:33:12.795091+0000","last_unstale":"2024-04-25T00:33:12.802012+0000","last_undegraded":"2024-04-25T00:33:12.802012+0000","last_fullsized":"2024-04-25T00:33:12.802012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:31.148769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.791555+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:33:12.791555+0000","last_peered":"2024-04-25T00:33:12.791555+0000","last_clean":"2024-04-25T00:33:12.791555+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:33:12.791555+0000","last_undegraded":"2024-04-25T00:33:12.791555+0000","last_fullsized":"2024-04-25T00:33:12.791555+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":792756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335424,"statfs":{"total":96636764160,"available":96599474176,"internally_reserved":0,"allocated":1097728,"data_stored":792756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:33:17.854 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:33:18.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.998+0000 7f0e65766640 1 Processor -- start 2024-04-25T00:33:18.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:17.998+0000 7f0e65766640 1 -- start start 2024-04-25T00:33:18.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e65766640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e600b4d30 0x7f0e600b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e65766640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e6006d210 0x7f0e600ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e65766640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e6006d980 con 0x7f0e6015df70 2024-04-25T00:33:18.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e65766640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f0e6006dc60 con 0x7f0e6006d210 2024-04-25T00:33:18.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e65766640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0e6006df40 con 0x7f0e600b4d30 2024-04-25T00:33:18.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e5f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e6006d210 0x7f0e600ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e5f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e6006d210 0x7f0e600ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43356/0 (socket says 172.21.15.29:43356) 2024-04-25T00:33:18.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e5f7fe640 1 -- 172.21.15.29:0/1617888922 learned_addr learned my addr 172.21.15.29:0/1617888922 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:18.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e5f7fe640 1 -- 172.21.15.29:0/1617888922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e600b4d30 msgr2=0x7f0e600b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:18.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e5f7fe640 1 --2- 172.21.15.29:0/1617888922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e600b4d30 0x7f0e600b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e5f7fe640 1 -- 172.21.15.29:0/1617888922 >> v1:172.21.15.157:6789/0 conn(0x7f0e6015df70 legacy=0x7f0e600b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:18.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e5f7fe640 1 -- 172.21.15.29:0/1617888922 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6006e220 con 0x7f0e6006d210 2024-04-25T00:33:18.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e5f7fe640 1 --2- 172.21.15.29:0/1617888922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e6006d210 0x7f0e600ad8a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009e50 tx=0x7f0e5801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441a0cc30410c5d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e5e7fc640 1 -- 172.21.15.29:0/1617888922 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e580a5860 con 0x7f0e6006d210 2024-04-25T00:33:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e5e7fc640 1 -- 172.21.15.29:0/1617888922 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e580a5a00 con 0x7f0e6006d210 2024-04-25T00:33:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.002+0000 7f0e5e7fc640 1 -- 172.21.15.29:0/1617888922 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e580a5dd0 con 0x7f0e6006d210 2024-04-25T00:33:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 -- 172.21.15.29:0/1617888922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e6006d210 msgr2=0x7f0e600ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 --2- 172.21.15.29:0/1617888922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e6006d210 0x7f0e600ad8a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009e50 tx=0x7f0e5801c9c0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 -- 172.21.15.29:0/1617888922 shutdown_connections 2024-04-25T00:33:18.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 --2- 172.21.15.29:0/1617888922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e6006d210 0x7f0e600ad8a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 --2- 172.21.15.29:0/1617888922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e600b4d30 0x7f0e600b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 -- 172.21.15.29:0/1617888922 >> 172.21.15.29:0/1617888922 conn(0x7f0e600b1870 msgr2=0x7f0e600b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:18.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 -- 172.21.15.29:0/1617888922 shutdown_connections 2024-04-25T00:33:18.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 -- 172.21.15.29:0/1617888922 wait complete. 2024-04-25T00:33:18.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 Processor -- start 2024-04-25T00:33:18.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 -- start start 2024-04-25T00:33:18.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e6006d210 0x7f0e6014cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e600b4d30 0x7f0e601571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e6015df70 0x7f0e60157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6006eba0 con 0x7f0e6006d210 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6006ea20 con 0x7f0e600b4d30 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0e600ae5f0 con 0x7f0e6015df70 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e5effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e6006d210 0x7f0e6014cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e5effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e6006d210 0x7f0e6014cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43366/0 (socket says 172.21.15.29:43366) 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e5effd640 1 -- 172.21.15.29:0/4116230122 learned_addr learned my addr 172.21.15.29:0/4116230122 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e5f7fe640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e6015df70 0x7f0e60157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e567fc640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e600b4d30 0x7f0e601571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e5effd640 1 -- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e6015df70 msgr2=0x7f0e60157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e5effd640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e6015df70 0x7f0e60157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e5effd640 1 -- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e600b4d30 msgr2=0x7f0e601571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e5effd640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e600b4d30 0x7f0e601571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e5effd640 1 -- 172.21.15.29:0/4116230122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e60151180 con 0x7f0e6006d210 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e5f7fe640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e6015df70 0x7f0e60157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e567fc640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e600b4d30 0x7f0e601571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:18.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e5effd640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e6006d210 0x7f0e6014cd00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000b3e0 tx=0x7f0e5000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:18.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e57fff640 1 -- 172.21.15.29:0/4116230122 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50015020 con 0x7f0e6006d210 2024-04-25T00:33:18.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e57fff640 1 -- 172.21.15.29:0/4116230122 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e50011850 con 0x7f0e6006d210 2024-04-25T00:33:18.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 -- 172.21.15.29:0/4116230122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e60151460 con 0x7f0e6006d210 2024-04-25T00:33:18.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e57fff640 1 -- 172.21.15.29:0/4116230122 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50011c40 con 0x7f0e6006d210 2024-04-25T00:33:18.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 -- 172.21.15.29:0/4116230122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e6014d240 con 0x7f0e6006d210 2024-04-25T00:33:18.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.006+0000 7f0e65766640 1 -- 172.21.15.29:0/4116230122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e60069270 con 0x7f0e6006d210 2024-04-25T00:33:18.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.010+0000 7f0e57fff640 1 -- 172.21.15.29:0/4116230122 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f0e50018020 con 0x7f0e6006d210 2024-04-25T00:33:18.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.010+0000 7f0e57fff640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0e40067a70 0x7f0e40069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.010+0000 7f0e57fff640 1 -- 172.21.15.29:0/4116230122 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f0e50085f80 con 0x7f0e6006d210 2024-04-25T00:33:18.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.010+0000 7f0e567fc640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0e40067a70 0x7f0e40069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.010+0000 7f0e567fc640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0e40067a70 0x7f0e40069f30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0e600b3270 tx=0x7f0e48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:18.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.014+0000 7f0e57fff640 1 -- 172.21.15.29:0/4116230122 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0e5001f020 con 0x7f0e6006d210 2024-04-25T00:33:18.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.170+0000 7f0e65766640 1 -- 172.21.15.29:0/4116230122 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e60156fb0 con 0x7f0e40067a70 2024-04-25T00:33:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.186+0000 7f0e57fff640 1 -- 172.21.15.29:0/4116230122 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246641 (secure 0 0 0) 0x7f0e60156fb0 con 0x7f0e40067a70 2024-04-25T00:33:18.194 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:18.196 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:33:18.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.190+0000 7f0e65766640 1 -- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0e40067a70 msgr2=0x7f0e40069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.190+0000 7f0e65766640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0e40067a70 0x7f0e40069f30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0e600b3270 tx=0x7f0e48008040 comp rx=0 tx=0).stop 2024-04-25T00:33:18.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.190+0000 7f0e65766640 1 -- 172.21.15.29:0/4116230122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e6006d210 msgr2=0x7f0e6014cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.190+0000 7f0e65766640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e6006d210 0x7f0e6014cd00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000b3e0 tx=0x7f0e5000b8a0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.190+0000 7f0e65766640 1 -- 172.21.15.29:0/4116230122 shutdown_connections 2024-04-25T00:33:18.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.190+0000 7f0e65766640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e6015df70 0x7f0e60157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.190+0000 7f0e65766640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0e40067a70 0x7f0e40069f30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.190+0000 7f0e65766640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e600b4d30 0x7f0e601571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.190+0000 7f0e65766640 1 --2- 172.21.15.29:0/4116230122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e6006d210 0x7f0e6014cd00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.190+0000 7f0e65766640 1 -- 172.21.15.29:0/4116230122 >> 172.21.15.29:0/4116230122 conn(0x7f0e600b1870 msgr2=0x7f0e600b4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:18.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.190+0000 7f0e65766640 1 -- 172.21.15.29:0/4116230122 shutdown_connections 2024-04-25T00:33:18.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.190+0000 7f0e65766640 1 -- 172.21.15.29:0/4116230122 wait complete. 2024-04-25T00:33:18.212 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-04-25T00:33:17.480927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258736,"kb_used_data":5096,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715984,"statfs":{"total":773094113280,"available":772829167616,"internally_reserved":0,"allocated":5218304,"data_stored":2799840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.723742"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.786161+0000","last_change":"2024-04-25T00:33:12.786161+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.786161+0000","last_undegraded":"2024-04-25T00:33:12.786161+0000","last_fullsized":"2024-04-25T00:33:12.786161+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778155+0000","last_change":"2024-04-25T00:33:12.778155+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778155+0000","last_undegraded":"2024-04-25T00:33:12.778155+0000","last_fullsized":"2024-04-25T00:33:12.778155+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772955+0000","last_change":"2024-04-25T00:33:12.772955+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772955+0000","last_undegraded":"2024-04-25T00:33:12.772955+0000","last_fullsized":"2024-04-25T00:33:12.772955+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779053+0000","last_change":"2024-04-25T00:33:12.779053+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779053+0000","last_undegraded":"2024-04-25T00:33:12.779053+0000","last_fullsized":"2024-04-25T00:33:12.779053+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779330+0000","last_change":"2024-04-25T00:33:12.779330+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779330+0000","last_undegraded":"2024-04-25T00:33:12.779330+0000","last_fullsized":"2024-04-25T00:33:12.779330+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785451+0000","last_change":"2024-04-25T00:33:12.785451+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785451+0000","last_undegraded":"2024-04-25T00:33:12.785451+0000","last_fullsized":"2024-04-25T00:33:12.785451+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781016+0000","last_change":"2024-04-25T00:33:12.781016+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781016+0000","last_undegraded":"2024-04-25T00:33:12.781016+0000","last_fullsized":"2024-04-25T00:33:12.781016+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779106+0000","last_change":"2024-04-25T00:33:12.779106+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779106+0000","last_undegraded":"2024-04-25T00:33:12.779106+0000","last_fullsized":"2024-04-25T00:33:12.779106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776612+0000","last_change":"2024-04-25T00:33:12.776612+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776612+0000","last_undegraded":"2024-04-25T00:33:12.776612+0000","last_fullsized":"2024-04-25T00:33:12.776612+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.777743+0000","last_change":"2024-04-25T00:33:12.777743+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.777743+0000","last_undegraded":"2024-04-25T00:33:12.777743+0000","last_fullsized":"2024-04-25T00:33:12.777743+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773992+0000","last_change":"2024-04-25T00:33:12.773992+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773992+0000","last_undegraded":"2024-04-25T00:33:12.773992+0000","last_fullsized":"2024-04-25T00:33:12.773992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.786788+0000","last_change":"2024-04-25T00:33:12.786788+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.786788+0000","last_undegraded":"2024-04-25T00:33:12.786788+0000","last_fullsized":"2024-04-25T00:33:12.786788+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776214+0000","last_change":"2024-04-25T00:33:12.776214+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776214+0000","last_undegraded":"2024-04-25T00:33:12.776214+0000","last_fullsized":"2024-04-25T00:33:12.776214+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779230+0000","last_change":"2024-04-25T00:33:12.779230+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779230+0000","last_undegraded":"2024-04-25T00:33:12.779230+0000","last_fullsized":"2024-04-25T00:33:12.779230+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779729+0000","last_change":"2024-04-25T00:33:12.779729+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779729+0000","last_undegraded":"2024-04-25T00:33:12.779729+0000","last_fullsized":"2024-04-25T00:33:12.779729+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781755+0000","last_change":"2024-04-25T00:33:12.781755+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781755+0000","last_undegraded":"2024-04-25T00:33:12.781755+0000","last_fullsized":"2024-04-25T00:33:12.781755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.775241+0000","last_change":"2024-04-25T00:33:12.775241+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.775241+0000","last_undegraded":"2024-04-25T00:33:12.775241+0000","last_fullsized":"2024-04-25T00:33:12.775241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.774539+0000","last_change":"2024-04-25T00:33:12.774539+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.774539+0000","last_undegraded":"2024-04-25T00:33:12.774539+0000","last_fullsized":"2024-04-25T00:33:12.774539+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779300+0000","last_change":"2024-04-25T00:33:12.779300+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779300+0000","last_undegraded":"2024-04-25T00:33:12.779300+0000","last_fullsized":"2024-04-25T00:33:12.779300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776553+0000","last_change":"2024-04-25T00:33:12.776553+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776553+0000","last_undegraded":"2024-04-25T00:33:12.776553+0000","last_fullsized":"2024-04-25T00:33:12.776553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784989+0000","last_change":"2024-04-25T00:33:12.784989+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784989+0000","last_undegraded":"2024-04-25T00:33:12.784989+0000","last_fullsized":"2024-04-25T00:33:12.784989+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771659+0000","last_change":"2024-04-25T00:33:12.771659+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771659+0000","last_undegraded":"2024-04-25T00:33:12.771659+0000","last_fullsized":"2024-04-25T00:33:12.771659+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778391+0000","last_change":"2024-04-25T00:33:12.778391+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778391+0000","last_undegraded":"2024-04-25T00:33:12.778391+0000","last_fullsized":"2024-04-25T00:33:12.778391+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780527+0000","last_change":"2024-04-25T00:33:12.780527+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780527+0000","last_undegraded":"2024-04-25T00:33:12.780527+0000","last_fullsized":"2024-04-25T00:33:12.780527+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.786781+0000","last_change":"2024-04-25T00:33:12.786781+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.786781+0000","last_undegraded":"2024-04-25T00:33:12.786781+0000","last_fullsized":"2024-04-25T00:33:12.786781+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776393+0000","last_change":"2024-04-25T00:33:12.776393+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776393+0000","last_undegraded":"2024-04-25T00:33:12.776393+0000","last_fullsized":"2024-04-25T00:33:12.776393+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771755+0000","last_change":"2024-04-25T00:33:12.771755+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771755+0000","last_undegraded":"2024-04-25T00:33:12.771755+0000","last_fullsized":"2024-04-25T00:33:12.771755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785983+0000","last_change":"2024-04-25T00:33:12.785983+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785983+0000","last_undegraded":"2024-04-25T00:33:12.785983+0000","last_fullsized":"2024-04-25T00:33:12.785983+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781215+0000","last_change":"2024-04-25T00:33:12.781215+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781215+0000","last_undegraded":"2024-04-25T00:33:12.781215+0000","last_fullsized":"2024-04-25T00:33:12.781215+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773770+0000","last_change":"2024-04-25T00:33:12.773770+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773770+0000","last_undegraded":"2024-04-25T00:33:12.773770+0000","last_fullsized":"2024-04-25T00:33:12.773770+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778731+0000","last_change":"2024-04-25T00:33:12.778731+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778731+0000","last_undegraded":"2024-04-25T00:33:12.778731+0000","last_fullsized":"2024-04-25T00:33:12.778731+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781274+0000","last_change":"2024-04-25T00:33:12.781274+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781274+0000","last_undegraded":"2024-04-25T00:33:12.781274+0000","last_fullsized":"2024-04-25T00:33:12.781274+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.783167+0000","last_change":"2024-04-25T00:33:12.783167+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.783167+0000","last_undegraded":"2024-04-25T00:33:12.783167+0000","last_fullsized":"2024-04-25T00:33:12.783167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802167+0000","last_change":"2024-04-25T00:33:12.801022+0000","last_active":"2024-04-25T00:33:12.802167+0000","last_peered":"2024-04-25T00:33:12.802167+0000","last_clean":"2024-04-25T00:33:12.802167+0000","last_became_active":"2024-04-25T00:33:12.800641+0000","last_became_peered":"2024-04-25T00:33:12.800641+0000","last_unstale":"2024-04-25T00:33:12.802167+0000","last_undegraded":"2024-04-25T00:33:12.802167+0000","last_fullsized":"2024-04-25T00:33:12.802167+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:10.074900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785776+0000","last_change":"2024-04-25T00:33:12.785776+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785776+0000","last_undegraded":"2024-04-25T00:33:12.785776+0000","last_fullsized":"2024-04-25T00:33:12.785776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799161+0000","last_change":"2024-04-25T00:33:12.797818+0000","last_active":"2024-04-25T00:33:12.799161+0000","last_peered":"2024-04-25T00:33:12.799161+0000","last_clean":"2024-04-25T00:33:12.799161+0000","last_became_active":"2024-04-25T00:33:12.796615+0000","last_became_peered":"2024-04-25T00:33:12.796615+0000","last_unstale":"2024-04-25T00:33:12.799161+0000","last_undegraded":"2024-04-25T00:33:12.799161+0000","last_fullsized":"2024-04-25T00:33:12.799161+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:38.919514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785220+0000","last_change":"2024-04-25T00:33:12.785220+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785220+0000","last_undegraded":"2024-04-25T00:33:12.785220+0000","last_fullsized":"2024-04-25T00:33:12.785220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799612+0000","last_change":"2024-04-25T00:33:12.797893+0000","last_active":"2024-04-25T00:33:12.799612+0000","last_peered":"2024-04-25T00:33:12.799612+0000","last_clean":"2024-04-25T00:33:12.799612+0000","last_became_active":"2024-04-25T00:33:12.797499+0000","last_became_peered":"2024-04-25T00:33:12.797499+0000","last_unstale":"2024-04-25T00:33:12.799612+0000","last_undegraded":"2024-04-25T00:33:12.799612+0000","last_fullsized":"2024-04-25T00:33:12.799612+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:54.884884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780797+0000","last_change":"2024-04-25T00:33:12.780797+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780797+0000","last_undegraded":"2024-04-25T00:33:12.780797+0000","last_fullsized":"2024-04-25T00:33:12.780797+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.809109+0000","last_change":"2024-04-25T00:33:12.808013+0000","last_active":"2024-04-25T00:33:12.809109+0000","last_peered":"2024-04-25T00:33:12.809109+0000","last_clean":"2024-04-25T00:33:12.809109+0000","last_became_active":"2024-04-25T00:33:12.807497+0000","last_became_peered":"2024-04-25T00:33:12.807497+0000","last_unstale":"2024-04-25T00:33:12.809109+0000","last_undegraded":"2024-04-25T00:33:12.809109+0000","last_fullsized":"2024-04-25T00:33:12.809109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:13.091178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772640+0000","last_change":"2024-04-25T00:33:12.772640+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772640+0000","last_undegraded":"2024-04-25T00:33:12.772640+0000","last_fullsized":"2024-04-25T00:33:12.772640+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.803068+0000","last_change":"2024-04-25T00:33:12.801886+0000","last_active":"2024-04-25T00:33:12.803068+0000","last_peered":"2024-04-25T00:33:12.803068+0000","last_clean":"2024-04-25T00:33:12.803068+0000","last_became_active":"2024-04-25T00:33:12.801570+0000","last_became_peered":"2024-04-25T00:33:12.801570+0000","last_unstale":"2024-04-25T00:33:12.803068+0000","last_undegraded":"2024-04-25T00:33:12.803068+0000","last_fullsized":"2024-04-25T00:33:12.803068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:37.757927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772470+0000","last_change":"2024-04-25T00:33:12.772470+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772470+0000","last_undegraded":"2024-04-25T00:33:12.772470+0000","last_fullsized":"2024-04-25T00:33:12.772470+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.797304+0000","last_change":"2024-04-25T00:33:12.795400+0000","last_active":"2024-04-25T00:33:12.797304+0000","last_peered":"2024-04-25T00:33:12.797304+0000","last_clean":"2024-04-25T00:33:12.797304+0000","last_became_active":"2024-04-25T00:33:12.794892+0000","last_became_peered":"2024-04-25T00:33:12.794892+0000","last_unstale":"2024-04-25T00:33:12.797304+0000","last_undegraded":"2024-04-25T00:33:12.797304+0000","last_fullsized":"2024-04-25T00:33:12.797304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:11.471922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772301+0000","last_change":"2024-04-25T00:33:12.772301+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772301+0000","last_undegraded":"2024-04-25T00:33:12.772301+0000","last_fullsized":"2024-04-25T00:33:12.772301+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.801950+0000","last_change":"2024-04-25T00:33:12.794993+0000","last_active":"2024-04-25T00:33:12.801950+0000","last_peered":"2024-04-25T00:33:12.801950+0000","last_clean":"2024-04-25T00:33:12.801950+0000","last_became_active":"2024-04-25T00:33:12.794251+0000","last_became_peered":"2024-04-25T00:33:12.794251+0000","last_unstale":"2024-04-25T00:33:12.801950+0000","last_undegraded":"2024-04-25T00:33:12.801950+0000","last_fullsized":"2024-04-25T00:33:12.801950+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:05.611837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779496+0000","last_change":"2024-04-25T00:33:12.779496+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779496+0000","last_undegraded":"2024-04-25T00:33:12.779496+0000","last_fullsized":"2024-04-25T00:33:12.779496+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.795732+0000","last_change":"2024-04-25T00:33:12.793905+0000","last_active":"2024-04-25T00:33:12.795732+0000","last_peered":"2024-04-25T00:33:12.795732+0000","last_clean":"2024-04-25T00:33:12.795732+0000","last_became_active":"2024-04-25T00:33:12.791067+0000","last_became_peered":"2024-04-25T00:33:12.791067+0000","last_unstale":"2024-04-25T00:33:12.795732+0000","last_undegraded":"2024-04-25T00:33:12.795732+0000","last_fullsized":"2024-04-25T00:33:12.795732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:26.419741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773456+0000","last_change":"2024-04-25T00:33:12.773456+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773456+0000","last_undegraded":"2024-04-25T00:33:12.773456+0000","last_fullsized":"2024-04-25T00:33:12.773456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.793147+0000","last_change":"2024-04-25T00:33:12.791338+0000","last_active":"2024-04-25T00:33:12.793147+0000","last_peered":"2024-04-25T00:33:12.793147+0000","last_clean":"2024-04-25T00:33:12.793147+0000","last_became_active":"2024-04-25T00:33:12.790547+0000","last_became_peered":"2024-04-25T00:33:12.790547+0000","last_unstale":"2024-04-25T00:33:12.793147+0000","last_undegraded":"2024-04-25T00:33:12.793147+0000","last_fullsized":"2024-04-25T00:33:12.793147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:42.648522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785118+0000","last_change":"2024-04-25T00:33:12.785118+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785118+0000","last_undegraded":"2024-04-25T00:33:12.785118+0000","last_fullsized":"2024-04-25T00:33:12.785118+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802404+0000","last_change":"2024-04-25T00:33:12.800591+0000","last_active":"2024-04-25T00:33:12.802404+0000","last_peered":"2024-04-25T00:33:12.802404+0000","last_clean":"2024-04-25T00:33:12.802404+0000","last_became_active":"2024-04-25T00:33:12.800301+0000","last_became_peered":"2024-04-25T00:33:12.800301+0000","last_unstale":"2024-04-25T00:33:12.802404+0000","last_undegraded":"2024-04-25T00:33:12.802404+0000","last_fullsized":"2024-04-25T00:33:12.802404+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:51.270346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.782338+0000","last_change":"2024-04-25T00:33:12.782338+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.782338+0000","last_undegraded":"2024-04-25T00:33:12.782338+0000","last_fullsized":"2024-04-25T00:33:12.782338+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.811725+0000","last_change":"2024-04-25T00:33:12.810318+0000","last_active":"2024-04-25T00:33:12.811725+0000","last_peered":"2024-04-25T00:33:12.811725+0000","last_clean":"2024-04-25T00:33:12.811725+0000","last_became_active":"2024-04-25T00:33:12.809691+0000","last_became_peered":"2024-04-25T00:33:12.809691+0000","last_unstale":"2024-04-25T00:33:12.811725+0000","last_undegraded":"2024-04-25T00:33:12.811725+0000","last_fullsized":"2024-04-25T00:33:12.811725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:31.459566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784334+0000","last_change":"2024-04-25T00:33:12.784334+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784334+0000","last_undegraded":"2024-04-25T00:33:12.784334+0000","last_fullsized":"2024-04-25T00:33:12.784334+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.788652+0000","last_change":"2024-04-25T00:33:12.787615+0000","last_active":"2024-04-25T00:33:12.788652+0000","last_peered":"2024-04-25T00:33:12.788652+0000","last_clean":"2024-04-25T00:33:12.788652+0000","last_became_active":"2024-04-25T00:33:12.787329+0000","last_became_peered":"2024-04-25T00:33:12.787329+0000","last_unstale":"2024-04-25T00:33:12.788652+0000","last_undegraded":"2024-04-25T00:33:12.788652+0000","last_fullsized":"2024-04-25T00:33:12.788652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:26.034657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778244+0000","last_change":"2024-04-25T00:33:12.778244+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778244+0000","last_undegraded":"2024-04-25T00:33:12.778244+0000","last_fullsized":"2024-04-25T00:33:12.778244+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.798527+0000","last_change":"2024-04-25T00:33:12.795864+0000","last_active":"2024-04-25T00:33:12.798527+0000","last_peered":"2024-04-25T00:33:12.798527+0000","last_clean":"2024-04-25T00:33:12.798527+0000","last_became_active":"2024-04-25T00:33:12.795591+0000","last_became_peered":"2024-04-25T00:33:12.795591+0000","last_unstale":"2024-04-25T00:33:12.798527+0000","last_undegraded":"2024-04-25T00:33:12.798527+0000","last_fullsized":"2024-04-25T00:33:12.798527+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:41.044585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771469+0000","last_change":"2024-04-25T00:33:12.771469+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771469+0000","last_undegraded":"2024-04-25T00:33:12.771469+0000","last_fullsized":"2024-04-25T00:33:12.771469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.807386+0000","last_change":"2024-04-25T00:33:12.806163+0000","last_active":"2024-04-25T00:33:12.807386+0000","last_peered":"2024-04-25T00:33:12.807386+0000","last_clean":"2024-04-25T00:33:12.807386+0000","last_became_active":"2024-04-25T00:33:12.802549+0000","last_became_peered":"2024-04-25T00:33:12.802549+0000","last_unstale":"2024-04-25T00:33:12.807386+0000","last_undegraded":"2024-04-25T00:33:12.807386+0000","last_fullsized":"2024-04-25T00:33:12.807386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:41.044585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.777997+0000","last_change":"2024-04-25T00:33:12.777997+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.777997+0000","last_undegraded":"2024-04-25T00:33:12.777997+0000","last_fullsized":"2024-04-25T00:33:12.777997+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799122+0000","last_change":"2024-04-25T00:33:12.797153+0000","last_active":"2024-04-25T00:33:12.799122+0000","last_peered":"2024-04-25T00:33:12.799122+0000","last_clean":"2024-04-25T00:33:12.799122+0000","last_became_active":"2024-04-25T00:33:12.796727+0000","last_became_peered":"2024-04-25T00:33:12.796727+0000","last_unstale":"2024-04-25T00:33:12.799122+0000","last_undegraded":"2024-04-25T00:33:12.799122+0000","last_fullsized":"2024-04-25T00:33:12.799122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:13.091178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779937+0000","last_change":"2024-04-25T00:33:12.779937+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779937+0000","last_undegraded":"2024-04-25T00:33:12.779937+0000","last_fullsized":"2024-04-25T00:33:12.779937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.800089+0000","last_change":"2024-04-25T00:33:12.799231+0000","last_active":"2024-04-25T00:33:12.800089+0000","last_peered":"2024-04-25T00:33:12.800089+0000","last_clean":"2024-04-25T00:33:12.800089+0000","last_became_active":"2024-04-25T00:33:12.798864+0000","last_became_peered":"2024-04-25T00:33:12.798864+0000","last_unstale":"2024-04-25T00:33:12.800089+0000","last_undegraded":"2024-04-25T00:33:12.800089+0000","last_fullsized":"2024-04-25T00:33:12.800089+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:56.668429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.775446+0000","last_change":"2024-04-25T00:33:12.775446+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.775446+0000","last_undegraded":"2024-04-25T00:33:12.775446+0000","last_fullsized":"2024-04-25T00:33:12.775446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.792247+0000","last_change":"2024-04-25T00:33:12.788985+0000","last_active":"2024-04-25T00:33:12.792247+0000","last_peered":"2024-04-25T00:33:12.792247+0000","last_clean":"2024-04-25T00:33:12.792247+0000","last_became_active":"2024-04-25T00:33:12.788443+0000","last_became_peered":"2024-04-25T00:33:12.788443+0000","last_unstale":"2024-04-25T00:33:12.792247+0000","last_undegraded":"2024-04-25T00:33:12.792247+0000","last_fullsized":"2024-04-25T00:33:12.792247+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:33.029999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773102+0000","last_change":"2024-04-25T00:33:12.773102+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773102+0000","last_undegraded":"2024-04-25T00:33:12.773102+0000","last_fullsized":"2024-04-25T00:33:12.773102+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.797229+0000","last_change":"2024-04-25T00:33:12.794976+0000","last_active":"2024-04-25T00:33:12.797229+0000","last_peered":"2024-04-25T00:33:12.797229+0000","last_clean":"2024-04-25T00:33:12.797229+0000","last_became_active":"2024-04-25T00:33:12.793226+0000","last_became_peered":"2024-04-25T00:33:12.793226+0000","last_unstale":"2024-04-25T00:33:12.797229+0000","last_undegraded":"2024-04-25T00:33:12.797229+0000","last_fullsized":"2024-04-25T00:33:12.797229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:52.568911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780454+0000","last_change":"2024-04-25T00:33:12.780454+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780454+0000","last_undegraded":"2024-04-25T00:33:12.780454+0000","last_fullsized":"2024-04-25T00:33:12.780454+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802028+0000","last_change":"2024-04-25T00:33:12.794824+0000","last_active":"2024-04-25T00:33:12.802028+0000","last_peered":"2024-04-25T00:33:12.802028+0000","last_clean":"2024-04-25T00:33:12.802028+0000","last_became_active":"2024-04-25T00:33:12.794088+0000","last_became_peered":"2024-04-25T00:33:12.794088+0000","last_unstale":"2024-04-25T00:33:12.802028+0000","last_undegraded":"2024-04-25T00:33:12.802028+0000","last_fullsized":"2024-04-25T00:33:12.802028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:33.703646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785147+0000","last_change":"2024-04-25T00:33:12.785147+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785147+0000","last_undegraded":"2024-04-25T00:33:12.785147+0000","last_fullsized":"2024-04-25T00:33:12.785147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802238+0000","last_change":"2024-04-25T00:33:12.801012+0000","last_active":"2024-04-25T00:33:12.802238+0000","last_peered":"2024-04-25T00:33:12.802238+0000","last_clean":"2024-04-25T00:33:12.802238+0000","last_became_active":"2024-04-25T00:33:12.800555+0000","last_became_peered":"2024-04-25T00:33:12.800555+0000","last_unstale":"2024-04-25T00:33:12.802238+0000","last_undegraded":"2024-04-25T00:33:12.802238+0000","last_fullsized":"2024-04-25T00:33:12.802238+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:31.459566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785063+0000","last_change":"2024-04-25T00:33:12.785063+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785063+0000","last_undegraded":"2024-04-25T00:33:12.785063+0000","last_fullsized":"2024-04-25T00:33:12.785063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802176+0000","last_change":"2024-04-25T00:33:12.801088+0000","last_active":"2024-04-25T00:33:12.802176+0000","last_peered":"2024-04-25T00:33:12.802176+0000","last_clean":"2024-04-25T00:33:12.802176+0000","last_became_active":"2024-04-25T00:33:12.800686+0000","last_became_peered":"2024-04-25T00:33:12.800686+0000","last_unstale":"2024-04-25T00:33:12.802176+0000","last_undegraded":"2024-04-25T00:33:12.802176+0000","last_fullsized":"2024-04-25T00:33:12.802176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:04.740368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785871+0000","last_change":"2024-04-25T00:33:12.785871+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785871+0000","last_undegraded":"2024-04-25T00:33:12.785871+0000","last_fullsized":"2024-04-25T00:33:12.785871+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.806654+0000","last_change":"2024-04-25T00:33:12.804216+0000","last_active":"2024-04-25T00:33:12.806654+0000","last_peered":"2024-04-25T00:33:12.806654+0000","last_clean":"2024-04-25T00:33:12.806654+0000","last_became_active":"2024-04-25T00:33:12.800818+0000","last_became_peered":"2024-04-25T00:33:12.800818+0000","last_unstale":"2024-04-25T00:33:12.806654+0000","last_undegraded":"2024-04-25T00:33:12.806654+0000","last_fullsized":"2024-04-25T00:33:12.806654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:04.901463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780423+0000","last_change":"2024-04-25T00:33:12.780423+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780423+0000","last_undegraded":"2024-04-25T00:33:12.780423+0000","last_fullsized":"2024-04-25T00:33:12.780423+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.795832+0000","last_change":"2024-04-25T00:33:12.793774+0000","last_active":"2024-04-25T00:33:12.795832+0000","last_peered":"2024-04-25T00:33:12.795832+0000","last_clean":"2024-04-25T00:33:12.795832+0000","last_became_active":"2024-04-25T00:33:12.793317+0000","last_became_peered":"2024-04-25T00:33:12.793317+0000","last_unstale":"2024-04-25T00:33:12.795832+0000","last_undegraded":"2024-04-25T00:33:12.795832+0000","last_fullsized":"2024-04-25T00:33:12.795832+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:18.045380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773072+0000","last_change":"2024-04-25T00:33:12.773072+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773072+0000","last_undegraded":"2024-04-25T00:33:12.773072+0000","last_fullsized":"2024-04-25T00:33:12.773072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.807370+0000","last_change":"2024-04-25T00:33:12.805396+0000","last_active":"2024-04-25T00:33:12.807370+0000","last_peered":"2024-04-25T00:33:12.807370+0000","last_clean":"2024-04-25T00:33:12.807370+0000","last_became_active":"2024-04-25T00:33:12.803923+0000","last_became_peered":"2024-04-25T00:33:12.803923+0000","last_unstale":"2024-04-25T00:33:12.807370+0000","last_undegraded":"2024-04-25T00:33:12.807370+0000","last_fullsized":"2024-04-25T00:33:12.807370+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.189080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780611+0000","last_change":"2024-04-25T00:33:12.780611+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780611+0000","last_undegraded":"2024-04-25T00:33:12.780611+0000","last_fullsized":"2024-04-25T00:33:12.780611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.798496+0000","last_change":"2024-04-25T00:33:12.795318+0000","last_active":"2024-04-25T00:33:12.798496+0000","last_peered":"2024-04-25T00:33:12.798496+0000","last_clean":"2024-04-25T00:33:12.798496+0000","last_became_active":"2024-04-25T00:33:12.794708+0000","last_became_peered":"2024-04-25T00:33:12.794708+0000","last_unstale":"2024-04-25T00:33:12.798496+0000","last_undegraded":"2024-04-25T00:33:12.798496+0000","last_fullsized":"2024-04-25T00:33:12.798496+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.189080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773274+0000","last_change":"2024-04-25T00:33:12.773274+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773274+0000","last_undegraded":"2024-04-25T00:33:12.773274+0000","last_fullsized":"2024-04-25T00:33:12.773274+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.812310+0000","last_change":"2024-04-25T00:33:12.810928+0000","last_active":"2024-04-25T00:33:12.812310+0000","last_peered":"2024-04-25T00:33:12.812310+0000","last_clean":"2024-04-25T00:33:12.812310+0000","last_became_active":"2024-04-25T00:33:12.810284+0000","last_became_peered":"2024-04-25T00:33:12.810284+0000","last_unstale":"2024-04-25T00:33:12.812310+0000","last_undegraded":"2024-04-25T00:33:12.812310+0000","last_fullsized":"2024-04-25T00:33:12.812310+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:04.740368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.783434+0000","last_change":"2024-04-25T00:33:12.783434+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.783434+0000","last_undegraded":"2024-04-25T00:33:12.783434+0000","last_fullsized":"2024-04-25T00:33:12.783434+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.804755+0000","last_change":"2024-04-25T00:33:12.803739+0000","last_active":"2024-04-25T00:33:12.804755+0000","last_peered":"2024-04-25T00:33:12.804755+0000","last_clean":"2024-04-25T00:33:12.804755+0000","last_became_active":"2024-04-25T00:33:12.803381+0000","last_became_peered":"2024-04-25T00:33:12.803381+0000","last_unstale":"2024-04-25T00:33:12.804755+0000","last_undegraded":"2024-04-25T00:33:12.804755+0000","last_fullsized":"2024-04-25T00:33:12.804755+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:46.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784539+0000","last_change":"2024-04-25T00:33:12.784539+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784539+0000","last_undegraded":"2024-04-25T00:33:12.784539+0000","last_fullsized":"2024-04-25T00:33:12.784539+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.811877+0000","last_change":"2024-04-25T00:33:12.810436+0000","last_active":"2024-04-25T00:33:12.811877+0000","last_peered":"2024-04-25T00:33:12.811877+0000","last_clean":"2024-04-25T00:33:12.811877+0000","last_became_active":"2024-04-25T00:33:12.809773+0000","last_became_peered":"2024-04-25T00:33:12.809773+0000","last_unstale":"2024-04-25T00:33:12.811877+0000","last_undegraded":"2024-04-25T00:33:12.811877+0000","last_fullsized":"2024-04-25T00:33:12.811877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:10.074900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784359+0000","last_change":"2024-04-25T00:33:12.784359+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784359+0000","last_undegraded":"2024-04-25T00:33:12.784359+0000","last_fullsized":"2024-04-25T00:33:12.784359+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.793065+0000","last_change":"2024-04-25T00:33:12.791171+0000","last_active":"2024-04-25T00:33:12.793065+0000","last_peered":"2024-04-25T00:33:12.793065+0000","last_clean":"2024-04-25T00:33:12.793065+0000","last_became_active":"2024-04-25T00:33:12.790272+0000","last_became_peered":"2024-04-25T00:33:12.790272+0000","last_unstale":"2024-04-25T00:33:12.793065+0000","last_undegraded":"2024-04-25T00:33:12.793065+0000","last_fullsized":"2024-04-25T00:33:12.793065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:33.246491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779265+0000","last_change":"2024-04-25T00:33:12.779265+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779265+0000","last_undegraded":"2024-04-25T00:33:12.779265+0000","last_fullsized":"2024-04-25T00:33:12.779265+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799547+0000","last_change":"2024-04-25T00:33:12.797798+0000","last_active":"2024-04-25T00:33:12.799547+0000","last_peered":"2024-04-25T00:33:12.799547+0000","last_clean":"2024-04-25T00:33:12.799547+0000","last_became_active":"2024-04-25T00:33:12.797086+0000","last_became_peered":"2024-04-25T00:33:12.797086+0000","last_unstale":"2024-04-25T00:33:12.799547+0000","last_undegraded":"2024-04-25T00:33:12.799547+0000","last_fullsized":"2024-04-25T00:33:12.799547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:38.919514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771834+0000","last_change":"2024-04-25T00:33:12.771834+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771834+0000","last_undegraded":"2024-04-25T00:33:12.771834+0000","last_fullsized":"2024-04-25T00:33:12.771834+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.795110+0000","last_change":"2024-04-25T00:33:12.789836+0000","last_active":"2024-04-25T00:33:12.795110+0000","last_peered":"2024-04-25T00:33:12.795110+0000","last_clean":"2024-04-25T00:33:12.795110+0000","last_became_active":"2024-04-25T00:33:12.789493+0000","last_became_peered":"2024-04-25T00:33:12.789493+0000","last_unstale":"2024-04-25T00:33:12.795110+0000","last_undegraded":"2024-04-25T00:33:12.795110+0000","last_fullsized":"2024-04-25T00:33:12.795110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:03.396230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.777158+0000","last_change":"2024-04-25T00:33:12.777158+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.777158+0000","last_undegraded":"2024-04-25T00:33:12.777158+0000","last_fullsized":"2024-04-25T00:33:12.777158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802012+0000","last_change":"2024-04-25T00:33:12.795705+0000","last_active":"2024-04-25T00:33:12.802012+0000","last_peered":"2024-04-25T00:33:12.802012+0000","last_clean":"2024-04-25T00:33:12.802012+0000","last_became_active":"2024-04-25T00:33:12.795091+0000","last_became_peered":"2024-04-25T00:33:12.795091+0000","last_unstale":"2024-04-25T00:33:12.802012+0000","last_undegraded":"2024-04-25T00:33:12.802012+0000","last_fullsized":"2024-04-25T00:33:12.802012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:31.148769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.791555+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:33:12.791555+0000","last_peered":"2024-04-25T00:33:12.791555+0000","last_clean":"2024-04-25T00:33:12.791555+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:33:12.791555+0000","last_undegraded":"2024-04-25T00:33:12.791555+0000","last_fullsized":"2024-04-25T00:33:12.791555+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":792756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335424,"statfs":{"total":96636764160,"available":96599474176,"internally_reserved":0,"allocated":1097728,"data_stored":792756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:33:18.216 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:33:18.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7afd80d640 1 Processor -- start 2024-04-25T00:33:18.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7afd80d640 1 -- start start 2024-04-25T00:33:18.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7afd80d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7af815bdb0 0x7f7af80cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7afd80d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7af815c190 0x7f7af80ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7afd80d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af80cd3e0 con 0x7f7af80d02d0 2024-04-25T00:33:18.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7afd80d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7af80cd6c0 con 0x7f7af815bdb0 2024-04-25T00:33:18.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7afd80d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7af80cd9a0 con 0x7f7af815c190 2024-04-25T00:33:18.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7af6ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7af815bdb0 0x7f7af80cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7af6ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7af815bdb0 0x7f7af80cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43380/0 (socket says 172.21.15.29:43380) 2024-04-25T00:33:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7af6ffd640 1 -- 172.21.15.29:0/3284343879 learned_addr learned my addr 172.21.15.29:0/3284343879 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7af67fc640 1 --2- 172.21.15.29:0/3284343879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7af815c190 0x7f7af80ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7af6ffd640 1 -- 172.21.15.29:0/3284343879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7af815c190 msgr2=0x7f7af80ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7af6ffd640 1 --2- 172.21.15.29:0/3284343879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7af815c190 0x7f7af80ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7af6ffd640 1 -- 172.21.15.29:0/3284343879 >> v1:172.21.15.157:6789/0 conn(0x7f7af80d02d0 legacy=0x7f7af80d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.378+0000 7f7af6ffd640 1 -- 172.21.15.29:0/3284343879 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af80cdc80 con 0x7f7af815bdb0 2024-04-25T00:33:18.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af6ffd640 1 --2- 172.21.15.29:0/3284343879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7af815bdb0 0x7f7af80cc820 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009e50 tx=0x7f7ae801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d142194bd0109b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:18.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af5ffb640 1 -- 172.21.15.29:0/3284343879 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae80a5770 con 0x7f7af815bdb0 2024-04-25T00:33:18.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af5ffb640 1 -- 172.21.15.29:0/3284343879 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae80a5910 con 0x7f7af815bdb0 2024-04-25T00:33:18.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af5ffb640 1 -- 172.21.15.29:0/3284343879 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae80a5ce0 con 0x7f7af815bdb0 2024-04-25T00:33:18.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 -- 172.21.15.29:0/3284343879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7af815bdb0 msgr2=0x7f7af80cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 --2- 172.21.15.29:0/3284343879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7af815bdb0 0x7f7af80cc820 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009e50 tx=0x7f7ae801c930 comp rx=0 tx=0).stop 2024-04-25T00:33:18.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 -- 172.21.15.29:0/3284343879 shutdown_connections 2024-04-25T00:33:18.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 --2- 172.21.15.29:0/3284343879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7af815c190 0x7f7af80ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 --2- 172.21.15.29:0/3284343879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7af815bdb0 0x7f7af80cc820 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 -- 172.21.15.29:0/3284343879 >> 172.21.15.29:0/3284343879 conn(0x7f7af809aa30 msgr2=0x7f7af809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:18.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 -- 172.21.15.29:0/3284343879 shutdown_connections 2024-04-25T00:33:18.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 -- 172.21.15.29:0/3284343879 wait complete. 2024-04-25T00:33:18.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 Processor -- start 2024-04-25T00:33:18.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 -- start start 2024-04-25T00:33:18.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7af80d02d0 0x7f7af815ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7af815bdb0 0x7f7af8154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af81553f0 0x7f7af815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7af8168ad0 con 0x7f7af80d02d0 2024-04-25T00:33:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7af8168c50 con 0x7f7af81553f0 2024-04-25T00:33:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7afd80d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7af8168dd0 con 0x7f7af815bdb0 2024-04-25T00:33:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7af80d02d0 0x7f7af815ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7af80d02d0 0x7f7af815ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43386/0 (socket says 172.21.15.29:43386) 2024-04-25T00:33:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af6ffd640 1 -- 172.21.15.29:0/1196149238 learned_addr learned my addr 172.21.15.29:0/1196149238 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af67fc640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7af815bdb0 0x7f7af8154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af77fe640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af81553f0 0x7f7af815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af6ffd640 1 -- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7af815bdb0 msgr2=0x7f7af8154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af6ffd640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7af815bdb0 0x7f7af8154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af6ffd640 1 -- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af81553f0 msgr2=0x7f7af815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af6ffd640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af81553f0 0x7f7af815c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af6ffd640 1 -- 172.21.15.29:0/1196149238 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af8155a70 con 0x7f7af80d02d0 2024-04-25T00:33:18.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af67fc640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7af815bdb0 0x7f7af8154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:18.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af77fe640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af81553f0 0x7f7af815c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:18.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7af6ffd640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7af80d02d0 0x7f7af815ee30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7ae801afe0 tx=0x7f7ae80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:18.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.382+0000 7f7adffff640 1 -- 172.21.15.29:0/1196149238 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae80bc020 con 0x7f7af80d02d0 2024-04-25T00:33:18.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.386+0000 7f7adffff640 1 -- 172.21.15.29:0/1196149238 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae80b8670 con 0x7f7af80d02d0 2024-04-25T00:33:18.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.386+0000 7f7afd80d640 1 -- 172.21.15.29:0/1196149238 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af8155d50 con 0x7f7af80d02d0 2024-04-25T00:33:18.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.386+0000 7f7adffff640 1 -- 172.21.15.29:0/1196149238 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae80b8a60 con 0x7f7af80d02d0 2024-04-25T00:33:18.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.386+0000 7f7afd80d640 1 -- 172.21.15.29:0/1196149238 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af8156310 con 0x7f7af80d02d0 2024-04-25T00:33:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.386+0000 7f7afd80d640 1 -- 172.21.15.29:0/1196149238 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af8155800 con 0x7f7af80d02d0 2024-04-25T00:33:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.386+0000 7f7adffff640 1 -- 172.21.15.29:0/1196149238 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f7ae80bf020 con 0x7f7af80d02d0 2024-04-25T00:33:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.386+0000 7f7adffff640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7ae4067a70 0x7f7ae4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.386+0000 7f7adffff640 1 -- 172.21.15.29:0/1196149238 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7ae8121210 con 0x7f7af80d02d0 2024-04-25T00:33:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.386+0000 7f7af67fc640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7ae4067a70 0x7f7ae4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.386+0000 7f7af67fc640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7ae4067a70 0x7f7ae4069f30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7aec006fd0 tx=0x7f7aec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:18.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.390+0000 7f7adffff640 1 -- 172.21.15.29:0/1196149238 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7ae80c6030 con 0x7f7af80d02d0 2024-04-25T00:33:18.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.550+0000 7f7afd80d640 1 -- 172.21.15.29:0/1196149238 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7af8069270 con 0x7f7ae4067a70 2024-04-25T00:33:18.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.570+0000 7f7adffff640 1 -- 172.21.15.29:0/1196149238 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246641 (secure 0 0 0) 0x7f7af8069270 con 0x7f7ae4067a70 2024-04-25T00:33:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:18.578 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:33:18.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.574+0000 7f7afd80d640 1 -- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7ae4067a70 msgr2=0x7f7ae4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.574+0000 7f7afd80d640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7ae4067a70 0x7f7ae4069f30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7aec006fd0 tx=0x7f7aec008040 comp rx=0 tx=0).stop 2024-04-25T00:33:18.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.574+0000 7f7afd80d640 1 -- 172.21.15.29:0/1196149238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7af80d02d0 msgr2=0x7f7af815ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.574+0000 7f7afd80d640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7af80d02d0 0x7f7af815ee30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7ae801afe0 tx=0x7f7ae80196d0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.574+0000 7f7afd80d640 1 -- 172.21.15.29:0/1196149238 shutdown_connections 2024-04-25T00:33:18.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.574+0000 7f7afd80d640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7ae4067a70 0x7f7ae4069f30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.574+0000 7f7afd80d640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af81553f0 0x7f7af815c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.574+0000 7f7afd80d640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7af815bdb0 0x7f7af8154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.574+0000 7f7afd80d640 1 --2- 172.21.15.29:0/1196149238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7af80d02d0 0x7f7af815ee30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.574+0000 7f7afd80d640 1 -- 172.21.15.29:0/1196149238 >> 172.21.15.29:0/1196149238 conn(0x7f7af809aa30 msgr2=0x7f7af806f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:18.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.574+0000 7f7afd80d640 1 -- 172.21.15.29:0/1196149238 shutdown_connections 2024-04-25T00:33:18.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.574+0000 7f7afd80d640 1 -- 172.21.15.29:0/1196149238 wait complete. 2024-04-25T00:33:18.595 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-04-25T00:33:17.480927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258736,"kb_used_data":5096,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715984,"statfs":{"total":773094113280,"available":772829167616,"internally_reserved":0,"allocated":5218304,"data_stored":2799840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.723742"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.786161+0000","last_change":"2024-04-25T00:33:12.786161+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.786161+0000","last_undegraded":"2024-04-25T00:33:12.786161+0000","last_fullsized":"2024-04-25T00:33:12.786161+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778155+0000","last_change":"2024-04-25T00:33:12.778155+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778155+0000","last_undegraded":"2024-04-25T00:33:12.778155+0000","last_fullsized":"2024-04-25T00:33:12.778155+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772955+0000","last_change":"2024-04-25T00:33:12.772955+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772955+0000","last_undegraded":"2024-04-25T00:33:12.772955+0000","last_fullsized":"2024-04-25T00:33:12.772955+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779053+0000","last_change":"2024-04-25T00:33:12.779053+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779053+0000","last_undegraded":"2024-04-25T00:33:12.779053+0000","last_fullsized":"2024-04-25T00:33:12.779053+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779330+0000","last_change":"2024-04-25T00:33:12.779330+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779330+0000","last_undegraded":"2024-04-25T00:33:12.779330+0000","last_fullsized":"2024-04-25T00:33:12.779330+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785451+0000","last_change":"2024-04-25T00:33:12.785451+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785451+0000","last_undegraded":"2024-04-25T00:33:12.785451+0000","last_fullsized":"2024-04-25T00:33:12.785451+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781016+0000","last_change":"2024-04-25T00:33:12.781016+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781016+0000","last_undegraded":"2024-04-25T00:33:12.781016+0000","last_fullsized":"2024-04-25T00:33:12.781016+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779106+0000","last_change":"2024-04-25T00:33:12.779106+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779106+0000","last_undegraded":"2024-04-25T00:33:12.779106+0000","last_fullsized":"2024-04-25T00:33:12.779106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776612+0000","last_change":"2024-04-25T00:33:12.776612+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776612+0000","last_undegraded":"2024-04-25T00:33:12.776612+0000","last_fullsized":"2024-04-25T00:33:12.776612+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.777743+0000","last_change":"2024-04-25T00:33:12.777743+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.777743+0000","last_undegraded":"2024-04-25T00:33:12.777743+0000","last_fullsized":"2024-04-25T00:33:12.777743+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773992+0000","last_change":"2024-04-25T00:33:12.773992+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773992+0000","last_undegraded":"2024-04-25T00:33:12.773992+0000","last_fullsized":"2024-04-25T00:33:12.773992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.786788+0000","last_change":"2024-04-25T00:33:12.786788+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.786788+0000","last_undegraded":"2024-04-25T00:33:12.786788+0000","last_fullsized":"2024-04-25T00:33:12.786788+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776214+0000","last_change":"2024-04-25T00:33:12.776214+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776214+0000","last_undegraded":"2024-04-25T00:33:12.776214+0000","last_fullsized":"2024-04-25T00:33:12.776214+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779230+0000","last_change":"2024-04-25T00:33:12.779230+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779230+0000","last_undegraded":"2024-04-25T00:33:12.779230+0000","last_fullsized":"2024-04-25T00:33:12.779230+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779729+0000","last_change":"2024-04-25T00:33:12.779729+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779729+0000","last_undegraded":"2024-04-25T00:33:12.779729+0000","last_fullsized":"2024-04-25T00:33:12.779729+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781755+0000","last_change":"2024-04-25T00:33:12.781755+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781755+0000","last_undegraded":"2024-04-25T00:33:12.781755+0000","last_fullsized":"2024-04-25T00:33:12.781755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.775241+0000","last_change":"2024-04-25T00:33:12.775241+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.775241+0000","last_undegraded":"2024-04-25T00:33:12.775241+0000","last_fullsized":"2024-04-25T00:33:12.775241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.774539+0000","last_change":"2024-04-25T00:33:12.774539+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.774539+0000","last_undegraded":"2024-04-25T00:33:12.774539+0000","last_fullsized":"2024-04-25T00:33:12.774539+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779300+0000","last_change":"2024-04-25T00:33:12.779300+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779300+0000","last_undegraded":"2024-04-25T00:33:12.779300+0000","last_fullsized":"2024-04-25T00:33:12.779300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776553+0000","last_change":"2024-04-25T00:33:12.776553+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776553+0000","last_undegraded":"2024-04-25T00:33:12.776553+0000","last_fullsized":"2024-04-25T00:33:12.776553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784989+0000","last_change":"2024-04-25T00:33:12.784989+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784989+0000","last_undegraded":"2024-04-25T00:33:12.784989+0000","last_fullsized":"2024-04-25T00:33:12.784989+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771659+0000","last_change":"2024-04-25T00:33:12.771659+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771659+0000","last_undegraded":"2024-04-25T00:33:12.771659+0000","last_fullsized":"2024-04-25T00:33:12.771659+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778391+0000","last_change":"2024-04-25T00:33:12.778391+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778391+0000","last_undegraded":"2024-04-25T00:33:12.778391+0000","last_fullsized":"2024-04-25T00:33:12.778391+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780527+0000","last_change":"2024-04-25T00:33:12.780527+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780527+0000","last_undegraded":"2024-04-25T00:33:12.780527+0000","last_fullsized":"2024-04-25T00:33:12.780527+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.786781+0000","last_change":"2024-04-25T00:33:12.786781+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.786781+0000","last_undegraded":"2024-04-25T00:33:12.786781+0000","last_fullsized":"2024-04-25T00:33:12.786781+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776393+0000","last_change":"2024-04-25T00:33:12.776393+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776393+0000","last_undegraded":"2024-04-25T00:33:12.776393+0000","last_fullsized":"2024-04-25T00:33:12.776393+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771755+0000","last_change":"2024-04-25T00:33:12.771755+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771755+0000","last_undegraded":"2024-04-25T00:33:12.771755+0000","last_fullsized":"2024-04-25T00:33:12.771755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785983+0000","last_change":"2024-04-25T00:33:12.785983+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785983+0000","last_undegraded":"2024-04-25T00:33:12.785983+0000","last_fullsized":"2024-04-25T00:33:12.785983+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781215+0000","last_change":"2024-04-25T00:33:12.781215+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781215+0000","last_undegraded":"2024-04-25T00:33:12.781215+0000","last_fullsized":"2024-04-25T00:33:12.781215+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773770+0000","last_change":"2024-04-25T00:33:12.773770+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773770+0000","last_undegraded":"2024-04-25T00:33:12.773770+0000","last_fullsized":"2024-04-25T00:33:12.773770+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778731+0000","last_change":"2024-04-25T00:33:12.778731+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778731+0000","last_undegraded":"2024-04-25T00:33:12.778731+0000","last_fullsized":"2024-04-25T00:33:12.778731+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781274+0000","last_change":"2024-04-25T00:33:12.781274+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781274+0000","last_undegraded":"2024-04-25T00:33:12.781274+0000","last_fullsized":"2024-04-25T00:33:12.781274+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.783167+0000","last_change":"2024-04-25T00:33:12.783167+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.783167+0000","last_undegraded":"2024-04-25T00:33:12.783167+0000","last_fullsized":"2024-04-25T00:33:12.783167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802167+0000","last_change":"2024-04-25T00:33:12.801022+0000","last_active":"2024-04-25T00:33:12.802167+0000","last_peered":"2024-04-25T00:33:12.802167+0000","last_clean":"2024-04-25T00:33:12.802167+0000","last_became_active":"2024-04-25T00:33:12.800641+0000","last_became_peered":"2024-04-25T00:33:12.800641+0000","last_unstale":"2024-04-25T00:33:12.802167+0000","last_undegraded":"2024-04-25T00:33:12.802167+0000","last_fullsized":"2024-04-25T00:33:12.802167+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:10.074900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785776+0000","last_change":"2024-04-25T00:33:12.785776+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785776+0000","last_undegraded":"2024-04-25T00:33:12.785776+0000","last_fullsized":"2024-04-25T00:33:12.785776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799161+0000","last_change":"2024-04-25T00:33:12.797818+0000","last_active":"2024-04-25T00:33:12.799161+0000","last_peered":"2024-04-25T00:33:12.799161+0000","last_clean":"2024-04-25T00:33:12.799161+0000","last_became_active":"2024-04-25T00:33:12.796615+0000","last_became_peered":"2024-04-25T00:33:12.796615+0000","last_unstale":"2024-04-25T00:33:12.799161+0000","last_undegraded":"2024-04-25T00:33:12.799161+0000","last_fullsized":"2024-04-25T00:33:12.799161+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:38.919514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785220+0000","last_change":"2024-04-25T00:33:12.785220+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785220+0000","last_undegraded":"2024-04-25T00:33:12.785220+0000","last_fullsized":"2024-04-25T00:33:12.785220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799612+0000","last_change":"2024-04-25T00:33:12.797893+0000","last_active":"2024-04-25T00:33:12.799612+0000","last_peered":"2024-04-25T00:33:12.799612+0000","last_clean":"2024-04-25T00:33:12.799612+0000","last_became_active":"2024-04-25T00:33:12.797499+0000","last_became_peered":"2024-04-25T00:33:12.797499+0000","last_unstale":"2024-04-25T00:33:12.799612+0000","last_undegraded":"2024-04-25T00:33:12.799612+0000","last_fullsized":"2024-04-25T00:33:12.799612+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:54.884884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780797+0000","last_change":"2024-04-25T00:33:12.780797+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780797+0000","last_undegraded":"2024-04-25T00:33:12.780797+0000","last_fullsized":"2024-04-25T00:33:12.780797+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.809109+0000","last_change":"2024-04-25T00:33:12.808013+0000","last_active":"2024-04-25T00:33:12.809109+0000","last_peered":"2024-04-25T00:33:12.809109+0000","last_clean":"2024-04-25T00:33:12.809109+0000","last_became_active":"2024-04-25T00:33:12.807497+0000","last_became_peered":"2024-04-25T00:33:12.807497+0000","last_unstale":"2024-04-25T00:33:12.809109+0000","last_undegraded":"2024-04-25T00:33:12.809109+0000","last_fullsized":"2024-04-25T00:33:12.809109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:13.091178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772640+0000","last_change":"2024-04-25T00:33:12.772640+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772640+0000","last_undegraded":"2024-04-25T00:33:12.772640+0000","last_fullsized":"2024-04-25T00:33:12.772640+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.803068+0000","last_change":"2024-04-25T00:33:12.801886+0000","last_active":"2024-04-25T00:33:12.803068+0000","last_peered":"2024-04-25T00:33:12.803068+0000","last_clean":"2024-04-25T00:33:12.803068+0000","last_became_active":"2024-04-25T00:33:12.801570+0000","last_became_peered":"2024-04-25T00:33:12.801570+0000","last_unstale":"2024-04-25T00:33:12.803068+0000","last_undegraded":"2024-04-25T00:33:12.803068+0000","last_fullsized":"2024-04-25T00:33:12.803068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:37.757927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772470+0000","last_change":"2024-04-25T00:33:12.772470+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772470+0000","last_undegraded":"2024-04-25T00:33:12.772470+0000","last_fullsized":"2024-04-25T00:33:12.772470+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.797304+0000","last_change":"2024-04-25T00:33:12.795400+0000","last_active":"2024-04-25T00:33:12.797304+0000","last_peered":"2024-04-25T00:33:12.797304+0000","last_clean":"2024-04-25T00:33:12.797304+0000","last_became_active":"2024-04-25T00:33:12.794892+0000","last_became_peered":"2024-04-25T00:33:12.794892+0000","last_unstale":"2024-04-25T00:33:12.797304+0000","last_undegraded":"2024-04-25T00:33:12.797304+0000","last_fullsized":"2024-04-25T00:33:12.797304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:11.471922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772301+0000","last_change":"2024-04-25T00:33:12.772301+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772301+0000","last_undegraded":"2024-04-25T00:33:12.772301+0000","last_fullsized":"2024-04-25T00:33:12.772301+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.801950+0000","last_change":"2024-04-25T00:33:12.794993+0000","last_active":"2024-04-25T00:33:12.801950+0000","last_peered":"2024-04-25T00:33:12.801950+0000","last_clean":"2024-04-25T00:33:12.801950+0000","last_became_active":"2024-04-25T00:33:12.794251+0000","last_became_peered":"2024-04-25T00:33:12.794251+0000","last_unstale":"2024-04-25T00:33:12.801950+0000","last_undegraded":"2024-04-25T00:33:12.801950+0000","last_fullsized":"2024-04-25T00:33:12.801950+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:05.611837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779496+0000","last_change":"2024-04-25T00:33:12.779496+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779496+0000","last_undegraded":"2024-04-25T00:33:12.779496+0000","last_fullsized":"2024-04-25T00:33:12.779496+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.795732+0000","last_change":"2024-04-25T00:33:12.793905+0000","last_active":"2024-04-25T00:33:12.795732+0000","last_peered":"2024-04-25T00:33:12.795732+0000","last_clean":"2024-04-25T00:33:12.795732+0000","last_became_active":"2024-04-25T00:33:12.791067+0000","last_became_peered":"2024-04-25T00:33:12.791067+0000","last_unstale":"2024-04-25T00:33:12.795732+0000","last_undegraded":"2024-04-25T00:33:12.795732+0000","last_fullsized":"2024-04-25T00:33:12.795732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:26.419741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773456+0000","last_change":"2024-04-25T00:33:12.773456+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773456+0000","last_undegraded":"2024-04-25T00:33:12.773456+0000","last_fullsized":"2024-04-25T00:33:12.773456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.793147+0000","last_change":"2024-04-25T00:33:12.791338+0000","last_active":"2024-04-25T00:33:12.793147+0000","last_peered":"2024-04-25T00:33:12.793147+0000","last_clean":"2024-04-25T00:33:12.793147+0000","last_became_active":"2024-04-25T00:33:12.790547+0000","last_became_peered":"2024-04-25T00:33:12.790547+0000","last_unstale":"2024-04-25T00:33:12.793147+0000","last_undegraded":"2024-04-25T00:33:12.793147+0000","last_fullsized":"2024-04-25T00:33:12.793147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:42.648522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785118+0000","last_change":"2024-04-25T00:33:12.785118+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785118+0000","last_undegraded":"2024-04-25T00:33:12.785118+0000","last_fullsized":"2024-04-25T00:33:12.785118+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802404+0000","last_change":"2024-04-25T00:33:12.800591+0000","last_active":"2024-04-25T00:33:12.802404+0000","last_peered":"2024-04-25T00:33:12.802404+0000","last_clean":"2024-04-25T00:33:12.802404+0000","last_became_active":"2024-04-25T00:33:12.800301+0000","last_became_peered":"2024-04-25T00:33:12.800301+0000","last_unstale":"2024-04-25T00:33:12.802404+0000","last_undegraded":"2024-04-25T00:33:12.802404+0000","last_fullsized":"2024-04-25T00:33:12.802404+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:51.270346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.782338+0000","last_change":"2024-04-25T00:33:12.782338+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.782338+0000","last_undegraded":"2024-04-25T00:33:12.782338+0000","last_fullsized":"2024-04-25T00:33:12.782338+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.811725+0000","last_change":"2024-04-25T00:33:12.810318+0000","last_active":"2024-04-25T00:33:12.811725+0000","last_peered":"2024-04-25T00:33:12.811725+0000","last_clean":"2024-04-25T00:33:12.811725+0000","last_became_active":"2024-04-25T00:33:12.809691+0000","last_became_peered":"2024-04-25T00:33:12.809691+0000","last_unstale":"2024-04-25T00:33:12.811725+0000","last_undegraded":"2024-04-25T00:33:12.811725+0000","last_fullsized":"2024-04-25T00:33:12.811725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:31.459566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784334+0000","last_change":"2024-04-25T00:33:12.784334+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784334+0000","last_undegraded":"2024-04-25T00:33:12.784334+0000","last_fullsized":"2024-04-25T00:33:12.784334+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.788652+0000","last_change":"2024-04-25T00:33:12.787615+0000","last_active":"2024-04-25T00:33:12.788652+0000","last_peered":"2024-04-25T00:33:12.788652+0000","last_clean":"2024-04-25T00:33:12.788652+0000","last_became_active":"2024-04-25T00:33:12.787329+0000","last_became_peered":"2024-04-25T00:33:12.787329+0000","last_unstale":"2024-04-25T00:33:12.788652+0000","last_undegraded":"2024-04-25T00:33:12.788652+0000","last_fullsized":"2024-04-25T00:33:12.788652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:26.034657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778244+0000","last_change":"2024-04-25T00:33:12.778244+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778244+0000","last_undegraded":"2024-04-25T00:33:12.778244+0000","last_fullsized":"2024-04-25T00:33:12.778244+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.798527+0000","last_change":"2024-04-25T00:33:12.795864+0000","last_active":"2024-04-25T00:33:12.798527+0000","last_peered":"2024-04-25T00:33:12.798527+0000","last_clean":"2024-04-25T00:33:12.798527+0000","last_became_active":"2024-04-25T00:33:12.795591+0000","last_became_peered":"2024-04-25T00:33:12.795591+0000","last_unstale":"2024-04-25T00:33:12.798527+0000","last_undegraded":"2024-04-25T00:33:12.798527+0000","last_fullsized":"2024-04-25T00:33:12.798527+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:41.044585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771469+0000","last_change":"2024-04-25T00:33:12.771469+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771469+0000","last_undegraded":"2024-04-25T00:33:12.771469+0000","last_fullsized":"2024-04-25T00:33:12.771469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.807386+0000","last_change":"2024-04-25T00:33:12.806163+0000","last_active":"2024-04-25T00:33:12.807386+0000","last_peered":"2024-04-25T00:33:12.807386+0000","last_clean":"2024-04-25T00:33:12.807386+0000","last_became_active":"2024-04-25T00:33:12.802549+0000","last_became_peered":"2024-04-25T00:33:12.802549+0000","last_unstale":"2024-04-25T00:33:12.807386+0000","last_undegraded":"2024-04-25T00:33:12.807386+0000","last_fullsized":"2024-04-25T00:33:12.807386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:41.044585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.777997+0000","last_change":"2024-04-25T00:33:12.777997+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.777997+0000","last_undegraded":"2024-04-25T00:33:12.777997+0000","last_fullsized":"2024-04-25T00:33:12.777997+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799122+0000","last_change":"2024-04-25T00:33:12.797153+0000","last_active":"2024-04-25T00:33:12.799122+0000","last_peered":"2024-04-25T00:33:12.799122+0000","last_clean":"2024-04-25T00:33:12.799122+0000","last_became_active":"2024-04-25T00:33:12.796727+0000","last_became_peered":"2024-04-25T00:33:12.796727+0000","last_unstale":"2024-04-25T00:33:12.799122+0000","last_undegraded":"2024-04-25T00:33:12.799122+0000","last_fullsized":"2024-04-25T00:33:12.799122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:13.091178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779937+0000","last_change":"2024-04-25T00:33:12.779937+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779937+0000","last_undegraded":"2024-04-25T00:33:12.779937+0000","last_fullsized":"2024-04-25T00:33:12.779937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.800089+0000","last_change":"2024-04-25T00:33:12.799231+0000","last_active":"2024-04-25T00:33:12.800089+0000","last_peered":"2024-04-25T00:33:12.800089+0000","last_clean":"2024-04-25T00:33:12.800089+0000","last_became_active":"2024-04-25T00:33:12.798864+0000","last_became_peered":"2024-04-25T00:33:12.798864+0000","last_unstale":"2024-04-25T00:33:12.800089+0000","last_undegraded":"2024-04-25T00:33:12.800089+0000","last_fullsized":"2024-04-25T00:33:12.800089+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:56.668429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.775446+0000","last_change":"2024-04-25T00:33:12.775446+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.775446+0000","last_undegraded":"2024-04-25T00:33:12.775446+0000","last_fullsized":"2024-04-25T00:33:12.775446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.792247+0000","last_change":"2024-04-25T00:33:12.788985+0000","last_active":"2024-04-25T00:33:12.792247+0000","last_peered":"2024-04-25T00:33:12.792247+0000","last_clean":"2024-04-25T00:33:12.792247+0000","last_became_active":"2024-04-25T00:33:12.788443+0000","last_became_peered":"2024-04-25T00:33:12.788443+0000","last_unstale":"2024-04-25T00:33:12.792247+0000","last_undegraded":"2024-04-25T00:33:12.792247+0000","last_fullsized":"2024-04-25T00:33:12.792247+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:33.029999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773102+0000","last_change":"2024-04-25T00:33:12.773102+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773102+0000","last_undegraded":"2024-04-25T00:33:12.773102+0000","last_fullsized":"2024-04-25T00:33:12.773102+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.797229+0000","last_change":"2024-04-25T00:33:12.794976+0000","last_active":"2024-04-25T00:33:12.797229+0000","last_peered":"2024-04-25T00:33:12.797229+0000","last_clean":"2024-04-25T00:33:12.797229+0000","last_became_active":"2024-04-25T00:33:12.793226+0000","last_became_peered":"2024-04-25T00:33:12.793226+0000","last_unstale":"2024-04-25T00:33:12.797229+0000","last_undegraded":"2024-04-25T00:33:12.797229+0000","last_fullsized":"2024-04-25T00:33:12.797229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:52.568911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780454+0000","last_change":"2024-04-25T00:33:12.780454+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780454+0000","last_undegraded":"2024-04-25T00:33:12.780454+0000","last_fullsized":"2024-04-25T00:33:12.780454+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802028+0000","last_change":"2024-04-25T00:33:12.794824+0000","last_active":"2024-04-25T00:33:12.802028+0000","last_peered":"2024-04-25T00:33:12.802028+0000","last_clean":"2024-04-25T00:33:12.802028+0000","last_became_active":"2024-04-25T00:33:12.794088+0000","last_became_peered":"2024-04-25T00:33:12.794088+0000","last_unstale":"2024-04-25T00:33:12.802028+0000","last_undegraded":"2024-04-25T00:33:12.802028+0000","last_fullsized":"2024-04-25T00:33:12.802028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:33.703646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785147+0000","last_change":"2024-04-25T00:33:12.785147+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785147+0000","last_undegraded":"2024-04-25T00:33:12.785147+0000","last_fullsized":"2024-04-25T00:33:12.785147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802238+0000","last_change":"2024-04-25T00:33:12.801012+0000","last_active":"2024-04-25T00:33:12.802238+0000","last_peered":"2024-04-25T00:33:12.802238+0000","last_clean":"2024-04-25T00:33:12.802238+0000","last_became_active":"2024-04-25T00:33:12.800555+0000","last_became_peered":"2024-04-25T00:33:12.800555+0000","last_unstale":"2024-04-25T00:33:12.802238+0000","last_undegraded":"2024-04-25T00:33:12.802238+0000","last_fullsized":"2024-04-25T00:33:12.802238+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:31.459566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785063+0000","last_change":"2024-04-25T00:33:12.785063+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785063+0000","last_undegraded":"2024-04-25T00:33:12.785063+0000","last_fullsized":"2024-04-25T00:33:12.785063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802176+0000","last_change":"2024-04-25T00:33:12.801088+0000","last_active":"2024-04-25T00:33:12.802176+0000","last_peered":"2024-04-25T00:33:12.802176+0000","last_clean":"2024-04-25T00:33:12.802176+0000","last_became_active":"2024-04-25T00:33:12.800686+0000","last_became_peered":"2024-04-25T00:33:12.800686+0000","last_unstale":"2024-04-25T00:33:12.802176+0000","last_undegraded":"2024-04-25T00:33:12.802176+0000","last_fullsized":"2024-04-25T00:33:12.802176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:04.740368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785871+0000","last_change":"2024-04-25T00:33:12.785871+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785871+0000","last_undegraded":"2024-04-25T00:33:12.785871+0000","last_fullsized":"2024-04-25T00:33:12.785871+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.806654+0000","last_change":"2024-04-25T00:33:12.804216+0000","last_active":"2024-04-25T00:33:12.806654+0000","last_peered":"2024-04-25T00:33:12.806654+0000","last_clean":"2024-04-25T00:33:12.806654+0000","last_became_active":"2024-04-25T00:33:12.800818+0000","last_became_peered":"2024-04-25T00:33:12.800818+0000","last_unstale":"2024-04-25T00:33:12.806654+0000","last_undegraded":"2024-04-25T00:33:12.806654+0000","last_fullsized":"2024-04-25T00:33:12.806654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:04.901463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780423+0000","last_change":"2024-04-25T00:33:12.780423+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780423+0000","last_undegraded":"2024-04-25T00:33:12.780423+0000","last_fullsized":"2024-04-25T00:33:12.780423+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.795832+0000","last_change":"2024-04-25T00:33:12.793774+0000","last_active":"2024-04-25T00:33:12.795832+0000","last_peered":"2024-04-25T00:33:12.795832+0000","last_clean":"2024-04-25T00:33:12.795832+0000","last_became_active":"2024-04-25T00:33:12.793317+0000","last_became_peered":"2024-04-25T00:33:12.793317+0000","last_unstale":"2024-04-25T00:33:12.795832+0000","last_undegraded":"2024-04-25T00:33:12.795832+0000","last_fullsized":"2024-04-25T00:33:12.795832+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:18.045380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773072+0000","last_change":"2024-04-25T00:33:12.773072+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773072+0000","last_undegraded":"2024-04-25T00:33:12.773072+0000","last_fullsized":"2024-04-25T00:33:12.773072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.807370+0000","last_change":"2024-04-25T00:33:12.805396+0000","last_active":"2024-04-25T00:33:12.807370+0000","last_peered":"2024-04-25T00:33:12.807370+0000","last_clean":"2024-04-25T00:33:12.807370+0000","last_became_active":"2024-04-25T00:33:12.803923+0000","last_became_peered":"2024-04-25T00:33:12.803923+0000","last_unstale":"2024-04-25T00:33:12.807370+0000","last_undegraded":"2024-04-25T00:33:12.807370+0000","last_fullsized":"2024-04-25T00:33:12.807370+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.189080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780611+0000","last_change":"2024-04-25T00:33:12.780611+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780611+0000","last_undegraded":"2024-04-25T00:33:12.780611+0000","last_fullsized":"2024-04-25T00:33:12.780611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.798496+0000","last_change":"2024-04-25T00:33:12.795318+0000","last_active":"2024-04-25T00:33:12.798496+0000","last_peered":"2024-04-25T00:33:12.798496+0000","last_clean":"2024-04-25T00:33:12.798496+0000","last_became_active":"2024-04-25T00:33:12.794708+0000","last_became_peered":"2024-04-25T00:33:12.794708+0000","last_unstale":"2024-04-25T00:33:12.798496+0000","last_undegraded":"2024-04-25T00:33:12.798496+0000","last_fullsized":"2024-04-25T00:33:12.798496+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.189080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773274+0000","last_change":"2024-04-25T00:33:12.773274+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773274+0000","last_undegraded":"2024-04-25T00:33:12.773274+0000","last_fullsized":"2024-04-25T00:33:12.773274+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.812310+0000","last_change":"2024-04-25T00:33:12.810928+0000","last_active":"2024-04-25T00:33:12.812310+0000","last_peered":"2024-04-25T00:33:12.812310+0000","last_clean":"2024-04-25T00:33:12.812310+0000","last_became_active":"2024-04-25T00:33:12.810284+0000","last_became_peered":"2024-04-25T00:33:12.810284+0000","last_unstale":"2024-04-25T00:33:12.812310+0000","last_undegraded":"2024-04-25T00:33:12.812310+0000","last_fullsized":"2024-04-25T00:33:12.812310+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:04.740368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.783434+0000","last_change":"2024-04-25T00:33:12.783434+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.783434+0000","last_undegraded":"2024-04-25T00:33:12.783434+0000","last_fullsized":"2024-04-25T00:33:12.783434+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.804755+0000","last_change":"2024-04-25T00:33:12.803739+0000","last_active":"2024-04-25T00:33:12.804755+0000","last_peered":"2024-04-25T00:33:12.804755+0000","last_clean":"2024-04-25T00:33:12.804755+0000","last_became_active":"2024-04-25T00:33:12.803381+0000","last_became_peered":"2024-04-25T00:33:12.803381+0000","last_unstale":"2024-04-25T00:33:12.804755+0000","last_undegraded":"2024-04-25T00:33:12.804755+0000","last_fullsized":"2024-04-25T00:33:12.804755+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:46.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784539+0000","last_change":"2024-04-25T00:33:12.784539+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784539+0000","last_undegraded":"2024-04-25T00:33:12.784539+0000","last_fullsized":"2024-04-25T00:33:12.784539+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.811877+0000","last_change":"2024-04-25T00:33:12.810436+0000","last_active":"2024-04-25T00:33:12.811877+0000","last_peered":"2024-04-25T00:33:12.811877+0000","last_clean":"2024-04-25T00:33:12.811877+0000","last_became_active":"2024-04-25T00:33:12.809773+0000","last_became_peered":"2024-04-25T00:33:12.809773+0000","last_unstale":"2024-04-25T00:33:12.811877+0000","last_undegraded":"2024-04-25T00:33:12.811877+0000","last_fullsized":"2024-04-25T00:33:12.811877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:10.074900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784359+0000","last_change":"2024-04-25T00:33:12.784359+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784359+0000","last_undegraded":"2024-04-25T00:33:12.784359+0000","last_fullsized":"2024-04-25T00:33:12.784359+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.793065+0000","last_change":"2024-04-25T00:33:12.791171+0000","last_active":"2024-04-25T00:33:12.793065+0000","last_peered":"2024-04-25T00:33:12.793065+0000","last_clean":"2024-04-25T00:33:12.793065+0000","last_became_active":"2024-04-25T00:33:12.790272+0000","last_became_peered":"2024-04-25T00:33:12.790272+0000","last_unstale":"2024-04-25T00:33:12.793065+0000","last_undegraded":"2024-04-25T00:33:12.793065+0000","last_fullsized":"2024-04-25T00:33:12.793065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:33.246491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779265+0000","last_change":"2024-04-25T00:33:12.779265+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779265+0000","last_undegraded":"2024-04-25T00:33:12.779265+0000","last_fullsized":"2024-04-25T00:33:12.779265+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799547+0000","last_change":"2024-04-25T00:33:12.797798+0000","last_active":"2024-04-25T00:33:12.799547+0000","last_peered":"2024-04-25T00:33:12.799547+0000","last_clean":"2024-04-25T00:33:12.799547+0000","last_became_active":"2024-04-25T00:33:12.797086+0000","last_became_peered":"2024-04-25T00:33:12.797086+0000","last_unstale":"2024-04-25T00:33:12.799547+0000","last_undegraded":"2024-04-25T00:33:12.799547+0000","last_fullsized":"2024-04-25T00:33:12.799547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:38.919514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771834+0000","last_change":"2024-04-25T00:33:12.771834+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771834+0000","last_undegraded":"2024-04-25T00:33:12.771834+0000","last_fullsized":"2024-04-25T00:33:12.771834+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.795110+0000","last_change":"2024-04-25T00:33:12.789836+0000","last_active":"2024-04-25T00:33:12.795110+0000","last_peered":"2024-04-25T00:33:12.795110+0000","last_clean":"2024-04-25T00:33:12.795110+0000","last_became_active":"2024-04-25T00:33:12.789493+0000","last_became_peered":"2024-04-25T00:33:12.789493+0000","last_unstale":"2024-04-25T00:33:12.795110+0000","last_undegraded":"2024-04-25T00:33:12.795110+0000","last_fullsized":"2024-04-25T00:33:12.795110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:03.396230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.777158+0000","last_change":"2024-04-25T00:33:12.777158+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.777158+0000","last_undegraded":"2024-04-25T00:33:12.777158+0000","last_fullsized":"2024-04-25T00:33:12.777158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802012+0000","last_change":"2024-04-25T00:33:12.795705+0000","last_active":"2024-04-25T00:33:12.802012+0000","last_peered":"2024-04-25T00:33:12.802012+0000","last_clean":"2024-04-25T00:33:12.802012+0000","last_became_active":"2024-04-25T00:33:12.795091+0000","last_became_peered":"2024-04-25T00:33:12.795091+0000","last_unstale":"2024-04-25T00:33:12.802012+0000","last_undegraded":"2024-04-25T00:33:12.802012+0000","last_fullsized":"2024-04-25T00:33:12.802012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:31.148769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.791555+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:33:12.791555+0000","last_peered":"2024-04-25T00:33:12.791555+0000","last_clean":"2024-04-25T00:33:12.791555+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:33:12.791555+0000","last_undegraded":"2024-04-25T00:33:12.791555+0000","last_fullsized":"2024-04-25T00:33:12.791555+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":792756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335424,"statfs":{"total":96636764160,"available":96599474176,"internally_reserved":0,"allocated":1097728,"data_stored":792756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:33:18.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2024-04-25T00:33:18.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.786161+0000', 'last_change': '2024-04-25T00:33:12.786161+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.786161+0000', 'last_undegraded': '2024-04-25T00:33:12.786161+0000', 'last_fullsized': '2024-04-25T00:33:12.786161+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:33:18.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2024-04-25T00:33:18.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.778155+0000', 'last_change': '2024-04-25T00:33:12.778155+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.778155+0000', 'last_undegraded': '2024-04-25T00:33:12.778155+0000', 'last_fullsized': '2024-04-25T00:33:12.778155+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:33:18.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2024-04-25T00:33:18.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.772955+0000', 'last_change': '2024-04-25T00:33:12.772955+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.772955+0000', 'last_undegraded': '2024-04-25T00:33:12.772955+0000', 'last_fullsized': '2024-04-25T00:33:12.772955+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:33:18.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2024-04-25T00:33:18.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.779053+0000', 'last_change': '2024-04-25T00:33:12.779053+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.779053+0000', 'last_undegraded': '2024-04-25T00:33:12.779053+0000', 'last_fullsized': '2024-04-25T00:33:12.779053+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2024-04-25T00:33:18.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.779330+0000', 'last_change': '2024-04-25T00:33:12.779330+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.779330+0000', 'last_undegraded': '2024-04-25T00:33:12.779330+0000', 'last_fullsized': '2024-04-25T00:33:12.779330+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:33:18.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2024-04-25T00:33:18.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.785451+0000', 'last_change': '2024-04-25T00:33:12.785451+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.785451+0000', 'last_undegraded': '2024-04-25T00:33:12.785451+0000', 'last_fullsized': '2024-04-25T00:33:12.785451+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:33:18.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2024-04-25T00:33:18.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.781016+0000', 'last_change': '2024-04-25T00:33:12.781016+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.781016+0000', 'last_undegraded': '2024-04-25T00:33:12.781016+0000', 'last_fullsized': '2024-04-25T00:33:12.781016+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:33:18.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2024-04-25T00:33:18.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.779106+0000', 'last_change': '2024-04-25T00:33:12.779106+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.779106+0000', 'last_undegraded': '2024-04-25T00:33:12.779106+0000', 'last_fullsized': '2024-04-25T00:33:12.779106+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2024-04-25T00:33:18.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.776612+0000', 'last_change': '2024-04-25T00:33:12.776612+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.776612+0000', 'last_undegraded': '2024-04-25T00:33:12.776612+0000', 'last_fullsized': '2024-04-25T00:33:12.776612+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2024-04-25T00:33:18.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.777743+0000', 'last_change': '2024-04-25T00:33:12.777743+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.777743+0000', 'last_undegraded': '2024-04-25T00:33:12.777743+0000', 'last_fullsized': '2024-04-25T00:33:12.777743+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2024-04-25T00:33:18.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.773992+0000', 'last_change': '2024-04-25T00:33:12.773992+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.773992+0000', 'last_undegraded': '2024-04-25T00:33:12.773992+0000', 'last_fullsized': '2024-04-25T00:33:12.773992+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2024-04-25T00:33:18.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.786788+0000', 'last_change': '2024-04-25T00:33:12.786788+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.786788+0000', 'last_undegraded': '2024-04-25T00:33:12.786788+0000', 'last_fullsized': '2024-04-25T00:33:12.786788+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:33:18.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2024-04-25T00:33:18.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.776214+0000', 'last_change': '2024-04-25T00:33:12.776214+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.776214+0000', 'last_undegraded': '2024-04-25T00:33:12.776214+0000', 'last_fullsized': '2024-04-25T00:33:12.776214+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:33:18.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2024-04-25T00:33:18.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.779230+0000', 'last_change': '2024-04-25T00:33:12.779230+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.779230+0000', 'last_undegraded': '2024-04-25T00:33:12.779230+0000', 'last_fullsized': '2024-04-25T00:33:12.779230+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-04-25T00:33:18.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.779729+0000', 'last_change': '2024-04-25T00:33:12.779729+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.779729+0000', 'last_undegraded': '2024-04-25T00:33:12.779729+0000', 'last_fullsized': '2024-04-25T00:33:12.779729+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:33:18.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2024-04-25T00:33:18.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.781755+0000', 'last_change': '2024-04-25T00:33:12.781755+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.781755+0000', 'last_undegraded': '2024-04-25T00:33:12.781755+0000', 'last_fullsized': '2024-04-25T00:33:12.781755+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:33:18.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2024-04-25T00:33:18.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.775241+0000', 'last_change': '2024-04-25T00:33:12.775241+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.775241+0000', 'last_undegraded': '2024-04-25T00:33:12.775241+0000', 'last_fullsized': '2024-04-25T00:33:12.775241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:33:18.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2024-04-25T00:33:18.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.774539+0000', 'last_change': '2024-04-25T00:33:12.774539+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.774539+0000', 'last_undegraded': '2024-04-25T00:33:12.774539+0000', 'last_fullsized': '2024-04-25T00:33:12.774539+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:33:18.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2024-04-25T00:33:18.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.779300+0000', 'last_change': '2024-04-25T00:33:12.779300+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.779300+0000', 'last_undegraded': '2024-04-25T00:33:12.779300+0000', 'last_fullsized': '2024-04-25T00:33:12.779300+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:33:18.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2024-04-25T00:33:18.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.776553+0000', 'last_change': '2024-04-25T00:33:12.776553+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.776553+0000', 'last_undegraded': '2024-04-25T00:33:12.776553+0000', 'last_fullsized': '2024-04-25T00:33:12.776553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:33:18.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-04-25T00:33:18.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.784989+0000', 'last_change': '2024-04-25T00:33:12.784989+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.784989+0000', 'last_undegraded': '2024-04-25T00:33:12.784989+0000', 'last_fullsized': '2024-04-25T00:33:12.784989+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:33:18.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2024-04-25T00:33:18.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.771659+0000', 'last_change': '2024-04-25T00:33:12.771659+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.771659+0000', 'last_undegraded': '2024-04-25T00:33:12.771659+0000', 'last_fullsized': '2024-04-25T00:33:12.771659+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2024-04-25T00:33:18.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.778391+0000', 'last_change': '2024-04-25T00:33:12.778391+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.778391+0000', 'last_undegraded': '2024-04-25T00:33:12.778391+0000', 'last_fullsized': '2024-04-25T00:33:12.778391+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:33:18.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2024-04-25T00:33:18.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.780527+0000', 'last_change': '2024-04-25T00:33:12.780527+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.780527+0000', 'last_undegraded': '2024-04-25T00:33:12.780527+0000', 'last_fullsized': '2024-04-25T00:33:12.780527+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:33:18.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2024-04-25T00:33:18.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.786781+0000', 'last_change': '2024-04-25T00:33:12.786781+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.786781+0000', 'last_undegraded': '2024-04-25T00:33:12.786781+0000', 'last_fullsized': '2024-04-25T00:33:12.786781+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:33:18.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-04-25T00:33:18.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.776393+0000', 'last_change': '2024-04-25T00:33:12.776393+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.776393+0000', 'last_undegraded': '2024-04-25T00:33:12.776393+0000', 'last_fullsized': '2024-04-25T00:33:12.776393+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:33:18.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2024-04-25T00:33:18.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.771755+0000', 'last_change': '2024-04-25T00:33:12.771755+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.771755+0000', 'last_undegraded': '2024-04-25T00:33:12.771755+0000', 'last_fullsized': '2024-04-25T00:33:12.771755+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:33:18.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2024-04-25T00:33:18.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.785983+0000', 'last_change': '2024-04-25T00:33:12.785983+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.785983+0000', 'last_undegraded': '2024-04-25T00:33:12.785983+0000', 'last_fullsized': '2024-04-25T00:33:12.785983+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:33:18.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2024-04-25T00:33:18.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.781215+0000', 'last_change': '2024-04-25T00:33:12.781215+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.781215+0000', 'last_undegraded': '2024-04-25T00:33:12.781215+0000', 'last_fullsized': '2024-04-25T00:33:12.781215+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:33:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2024-04-25T00:33:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.773770+0000', 'last_change': '2024-04-25T00:33:12.773770+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.773770+0000', 'last_undegraded': '2024-04-25T00:33:12.773770+0000', 'last_fullsized': '2024-04-25T00:33:12.773770+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:33:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2024-04-25T00:33:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.778731+0000', 'last_change': '2024-04-25T00:33:12.778731+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.778731+0000', 'last_undegraded': '2024-04-25T00:33:12.778731+0000', 'last_fullsized': '2024-04-25T00:33:12.778731+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:33:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-04-25T00:33:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.781274+0000', 'last_change': '2024-04-25T00:33:12.781274+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.781274+0000', 'last_undegraded': '2024-04-25T00:33:12.781274+0000', 'last_fullsized': '2024-04-25T00:33:12.781274+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:33:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-04-25T00:33:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.783167+0000', 'last_change': '2024-04-25T00:33:12.783167+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.783167+0000', 'last_undegraded': '2024-04-25T00:33:12.783167+0000', 'last_fullsized': '2024-04-25T00:33:12.783167+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:33:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-04-25T00:33:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.785776+0000', 'last_change': '2024-04-25T00:33:12.785776+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.785776+0000', 'last_undegraded': '2024-04-25T00:33:12.785776+0000', 'last_fullsized': '2024-04-25T00:33:12.785776+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:33:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-04-25T00:33:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.785220+0000', 'last_change': '2024-04-25T00:33:12.785220+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.785220+0000', 'last_undegraded': '2024-04-25T00:33:12.785220+0000', 'last_fullsized': '2024-04-25T00:33:12.785220+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:33:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-25T00:33:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.780797+0000', 'last_change': '2024-04-25T00:33:12.780797+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.780797+0000', 'last_undegraded': '2024-04-25T00:33:12.780797+0000', 'last_fullsized': '2024-04-25T00:33:12.780797+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:33:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-04-25T00:33:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.772640+0000', 'last_change': '2024-04-25T00:33:12.772640+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.772640+0000', 'last_undegraded': '2024-04-25T00:33:12.772640+0000', 'last_fullsized': '2024-04-25T00:33:12.772640+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:33:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-04-25T00:33:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.772470+0000', 'last_change': '2024-04-25T00:33:12.772470+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.772470+0000', 'last_undegraded': '2024-04-25T00:33:12.772470+0000', 'last_fullsized': '2024-04-25T00:33:12.772470+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-04-25T00:33:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.772301+0000', 'last_change': '2024-04-25T00:33:12.772301+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.772301+0000', 'last_undegraded': '2024-04-25T00:33:12.772301+0000', 'last_fullsized': '2024-04-25T00:33:12.772301+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-04-25T00:33:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.779496+0000', 'last_change': '2024-04-25T00:33:12.779496+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.779496+0000', 'last_undegraded': '2024-04-25T00:33:12.779496+0000', 'last_fullsized': '2024-04-25T00:33:12.779496+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:33:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-04-25T00:33:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.773456+0000', 'last_change': '2024-04-25T00:33:12.773456+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.773456+0000', 'last_undegraded': '2024-04-25T00:33:12.773456+0000', 'last_fullsized': '2024-04-25T00:33:12.773456+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:33:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-04-25T00:33:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.785118+0000', 'last_change': '2024-04-25T00:33:12.785118+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.785118+0000', 'last_undegraded': '2024-04-25T00:33:12.785118+0000', 'last_fullsized': '2024-04-25T00:33:12.785118+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:33:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-25T00:33:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.782338+0000', 'last_change': '2024-04-25T00:33:12.782338+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.782338+0000', 'last_undegraded': '2024-04-25T00:33:12.782338+0000', 'last_fullsized': '2024-04-25T00:33:12.782338+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:33:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-04-25T00:33:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.784334+0000', 'last_change': '2024-04-25T00:33:12.784334+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.784334+0000', 'last_undegraded': '2024-04-25T00:33:12.784334+0000', 'last_fullsized': '2024-04-25T00:33:12.784334+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:33:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-04-25T00:33:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.778244+0000', 'last_change': '2024-04-25T00:33:12.778244+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.778244+0000', 'last_undegraded': '2024-04-25T00:33:12.778244+0000', 'last_fullsized': '2024-04-25T00:33:12.778244+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:33:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-04-25T00:33:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.771469+0000', 'last_change': '2024-04-25T00:33:12.771469+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.771469+0000', 'last_undegraded': '2024-04-25T00:33:12.771469+0000', 'last_fullsized': '2024-04-25T00:33:12.771469+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-04-25T00:33:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.777997+0000', 'last_change': '2024-04-25T00:33:12.777997+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.777997+0000', 'last_undegraded': '2024-04-25T00:33:12.777997+0000', 'last_fullsized': '2024-04-25T00:33:12.777997+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:33:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-04-25T00:33:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.779937+0000', 'last_change': '2024-04-25T00:33:12.779937+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.779937+0000', 'last_undegraded': '2024-04-25T00:33:12.779937+0000', 'last_fullsized': '2024-04-25T00:33:12.779937+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:33:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-04-25T00:33:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.775446+0000', 'last_change': '2024-04-25T00:33:12.775446+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.775446+0000', 'last_undegraded': '2024-04-25T00:33:12.775446+0000', 'last_fullsized': '2024-04-25T00:33:12.775446+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:33:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-04-25T00:33:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.773102+0000', 'last_change': '2024-04-25T00:33:12.773102+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.773102+0000', 'last_undegraded': '2024-04-25T00:33:12.773102+0000', 'last_fullsized': '2024-04-25T00:33:12.773102+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-04-25T00:33:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.780454+0000', 'last_change': '2024-04-25T00:33:12.780454+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.780454+0000', 'last_undegraded': '2024-04-25T00:33:12.780454+0000', 'last_fullsized': '2024-04-25T00:33:12.780454+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:33:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-04-25T00:33:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.785147+0000', 'last_change': '2024-04-25T00:33:12.785147+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.785147+0000', 'last_undegraded': '2024-04-25T00:33:12.785147+0000', 'last_fullsized': '2024-04-25T00:33:12.785147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:33:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-04-25T00:33:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.785063+0000', 'last_change': '2024-04-25T00:33:12.785063+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.785063+0000', 'last_undegraded': '2024-04-25T00:33:12.785063+0000', 'last_fullsized': '2024-04-25T00:33:12.785063+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:33:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-04-25T00:33:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.785871+0000', 'last_change': '2024-04-25T00:33:12.785871+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.785871+0000', 'last_undegraded': '2024-04-25T00:33:12.785871+0000', 'last_fullsized': '2024-04-25T00:33:12.785871+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:33:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-25T00:33:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.780423+0000', 'last_change': '2024-04-25T00:33:12.780423+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.780423+0000', 'last_undegraded': '2024-04-25T00:33:12.780423+0000', 'last_fullsized': '2024-04-25T00:33:12.780423+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:33:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-25T00:33:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.773072+0000', 'last_change': '2024-04-25T00:33:12.773072+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.773072+0000', 'last_undegraded': '2024-04-25T00:33:12.773072+0000', 'last_fullsized': '2024-04-25T00:33:12.773072+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-04-25T00:33:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.780611+0000', 'last_change': '2024-04-25T00:33:12.780611+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.780611+0000', 'last_undegraded': '2024-04-25T00:33:12.780611+0000', 'last_fullsized': '2024-04-25T00:33:12.780611+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:33:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-04-25T00:33:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.773274+0000', 'last_change': '2024-04-25T00:33:12.773274+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.773274+0000', 'last_undegraded': '2024-04-25T00:33:12.773274+0000', 'last_fullsized': '2024-04-25T00:33:12.773274+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-04-25T00:33:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.783434+0000', 'last_change': '2024-04-25T00:33:12.783434+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.783434+0000', 'last_undegraded': '2024-04-25T00:33:12.783434+0000', 'last_fullsized': '2024-04-25T00:33:12.783434+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:33:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-04-25T00:33:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.784539+0000', 'last_change': '2024-04-25T00:33:12.784539+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.784539+0000', 'last_undegraded': '2024-04-25T00:33:12.784539+0000', 'last_fullsized': '2024-04-25T00:33:12.784539+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:33:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-25T00:33:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.784359+0000', 'last_change': '2024-04-25T00:33:12.784359+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.784359+0000', 'last_undegraded': '2024-04-25T00:33:12.784359+0000', 'last_fullsized': '2024-04-25T00:33:12.784359+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:33:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-04-25T00:33:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.779265+0000', 'last_change': '2024-04-25T00:33:12.779265+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.779265+0000', 'last_undegraded': '2024-04-25T00:33:12.779265+0000', 'last_fullsized': '2024-04-25T00:33:12.779265+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:33:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-04-25T00:33:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.771834+0000', 'last_change': '2024-04-25T00:33:12.771834+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.771834+0000', 'last_undegraded': '2024-04-25T00:33:12.771834+0000', 'last_fullsized': '2024-04-25T00:33:12.771834+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:33:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-25T00:33:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:33:12.777158+0000', 'last_change': '2024-04-25T00:33:12.777158+0000', 'last_active': '2024-04-25T00:33:12.755846+0000', 'last_peered': '2024-04-25T00:33:12.755846+0000', 'last_clean': '2024-04-25T00:33:12.755846+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:33:12.777158+0000', 'last_undegraded': '2024-04-25T00:33:12.777158+0000', 'last_fullsized': '2024-04-25T00:33:12.777158+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'last_clean_scrub_stamp': '2024-04-25T00:33:12.755846+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:33:18.610 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:33:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.746+0000 7fcceeda7640 1 Processor -- start 2024-04-25T00:33:18.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.746+0000 7fcceeda7640 1 -- start start 2024-04-25T00:33:18.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.746+0000 7fcceeda7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcce80b4d30 0x7fcce80b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fcceeda7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce806d210 0x7fcce80ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fcceeda7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce806d6c0 con 0x7fcce815df70 2024-04-25T00:33:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fcceeda7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcce806d980 con 0x7fcce80b4d30 2024-04-25T00:33:18.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fcceeda7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcce806dc60 con 0x7fcce806d210 2024-04-25T00:33:18.754 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fe2b8365640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:33:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fcce7fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcce80b4d30 0x7fcce80b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fcced344640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce806d210 0x7fcce80ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fcced344640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce806d210 0x7fcce80ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:44154/0 (socket says 172.21.15.29:44154) 2024-04-25T00:33:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fccecb43640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcce815df70 0x7fcce80b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:55858/0 (socket says 172.21.15.29:55858) 2024-04-25T00:33:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fcced344640 1 -- 172.21.15.29:0/338901981 learned_addr learned my addr 172.21.15.29:0/338901981 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fcce7fff640 1 -- 172.21.15.29:0/338901981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce806d210 msgr2=0x7fcce80ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fcce7fff640 1 --2- 172.21.15.29:0/338901981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce806d210 0x7fcce80ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fcce7fff640 1 -- 172.21.15.29:0/338901981 >> v1:172.21.15.157:6789/0 conn(0x7fcce815df70 legacy=0x7fcce80b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fcce7fff640 1 -- 172.21.15.29:0/338901981 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce806df40 con 0x7fcce80b4d30 2024-04-25T00:33:18.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.750+0000 7fcced344640 1 --2- 172.21.15.29:0/338901981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce806d210 0x7fcce80ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:33:18.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcce7fff640 1 --2- 172.21.15.29:0/338901981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce80b4d30 0x7fcce80b3e40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0097c0 tx=0x7fccdc01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ba8a4b04b075123 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:18.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcce77fe640 1 -- 172.21.15.29:0/338901981 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0a7070 con 0x7fcce80b4d30 2024-04-25T00:33:18.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcce77fe640 1 -- 172.21.15.29:0/338901981 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccdc0b0070 con 0x7fcce80b4d30 2024-04-25T00:33:18.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcce77fe640 1 -- 172.21.15.29:0/338901981 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0ab890 con 0x7fcce80b4d30 2024-04-25T00:33:18.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcceeda7640 1 -- 172.21.15.29:0/338901981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce80b4d30 msgr2=0x7fcce80b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcceeda7640 1 --2- 172.21.15.29:0/338901981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce80b4d30 0x7fcce80b3e40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0097c0 tx=0x7fccdc01c470 comp rx=0 tx=0).stop 2024-04-25T00:33:18.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcceeda7640 1 -- 172.21.15.29:0/338901981 shutdown_connections 2024-04-25T00:33:18.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcceeda7640 1 --2- 172.21.15.29:0/338901981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce806d210 0x7fcce80ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcceeda7640 1 --2- 172.21.15.29:0/338901981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce80b4d30 0x7fcce80b3e40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcceeda7640 1 -- 172.21.15.29:0/338901981 >> 172.21.15.29:0/338901981 conn(0x7fcce80b1870 msgr2=0x7fcce80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:18.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcceeda7640 1 -- 172.21.15.29:0/338901981 shutdown_connections 2024-04-25T00:33:18.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcceeda7640 1 -- 172.21.15.29:0/338901981 wait complete. 2024-04-25T00:33:18.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcceeda7640 1 Processor -- start 2024-04-25T00:33:18.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcceeda7640 1 -- start start 2024-04-25T00:33:18.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcceeda7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce806d210 0x7fcce8142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcceeda7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce80b4d30 0x7fcce80ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.754+0000 7fcceeda7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcce815df70 0x7fcce81520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcceeda7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcce806eba0 con 0x7fcce815df70 2024-04-25T00:33:18.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcceeda7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcce806ea20 con 0x7fcce80b4d30 2024-04-25T00:33:18.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcceeda7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcce80ae5f0 con 0x7fcce806d210 2024-04-25T00:33:18.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcced344640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcce815df70 0x7fcce81520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcced344640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcce815df70 0x7fcce81520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43394/0 (socket says 172.21.15.29:43394) 2024-04-25T00:33:18.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcced344640 1 -- 172.21.15.29:0/2077831130 learned_addr learned my addr 172.21.15.29:0/2077831130 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:18.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcce7fff640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce80b4d30 0x7fcce80ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fccecb43640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce806d210 0x7fcce8142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcced344640 1 -- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce806d210 msgr2=0x7fcce8142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcced344640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce806d210 0x7fcce8142570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcced344640 1 -- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce80b4d30 msgr2=0x7fcce80ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcced344640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce80b4d30 0x7fcce80ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcced344640 1 -- 172.21.15.29:0/2077831130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce81518a0 con 0x7fcce815df70 2024-04-25T00:33:18.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fccecb43640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce806d210 0x7fcce8142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:18.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcced344640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcce815df70 0x7fcce81520a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fcce000efc0 tx=0x7fcce000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:18.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcce57fa640 1 -- 172.21.15.29:0/2077831130 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce000cbe0 con 0x7fcce815df70 2024-04-25T00:33:18.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcce57fa640 1 -- 172.21.15.29:0/2077831130 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce000cd80 con 0x7fcce815df70 2024-04-25T00:33:18.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcce57fa640 1 -- 172.21.15.29:0/2077831130 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0003bc0 con 0x7fcce815df70 2024-04-25T00:33:18.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcceeda7640 1 -- 172.21.15.29:0/2077831130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce8150800 con 0x7fcce815df70 2024-04-25T00:33:18.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcceeda7640 1 -- 172.21.15.29:0/2077831130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce8150dc0 con 0x7fcce815df70 2024-04-25T00:33:18.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcceeda7640 1 -- 172.21.15.29:0/2077831130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce8069270 con 0x7fcce815df70 2024-04-25T00:33:18.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcce57fa640 1 -- 172.21.15.29:0/2077831130 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fcce001b030 con 0x7fcce815df70 2024-04-25T00:33:18.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcce57fa640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fccd4067a70 0x7fccd4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:18.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fcce57fa640 1 -- 172.21.15.29:0/2077831130 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fcce00864c0 con 0x7fcce815df70 2024-04-25T00:33:18.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.758+0000 7fccecb43640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fccd4067a70 0x7fccd4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:18.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.762+0000 7fccecb43640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fccd4067a70 0x7fccd4069f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fccd8000f70 tx=0x7fccd8007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:18.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.762+0000 7fcce57fa640 1 -- 172.21.15.29:0/2077831130 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcce0022020 con 0x7fcce815df70 2024-04-25T00:33:18.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.974+0000 7fcceeda7640 1 -- 172.21.15.29:0/2077831130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fcce80722a0 con 0x7fcce815df70 2024-04-25T00:33:18.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.978+0000 7fcce57fa640 1 -- 172.21.15.29:0/2077831130 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1453 (secure 0 0 0) 0x7fcce0010040 con 0x7fcce815df70 2024-04-25T00:33:18.981 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:18.981 INFO:teuthology.orchestra.run.smithi029.stdout:{"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":126,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":8,"num_up_osds":8,"osd_up_since":1714005083,"num_in_osds":8,"osd_in_since":1714005074,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":264945664,"bytes_avail":772829167616,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":24,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5733}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-04-25T00:33:01.476447+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:33:18.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.978+0000 7fcceeda7640 1 -- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fccd4067a70 msgr2=0x7fccd4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.978+0000 7fcceeda7640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fccd4067a70 0x7fccd4069f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fccd8000f70 tx=0x7fccd8007d50 comp rx=0 tx=0).stop 2024-04-25T00:33:18.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.978+0000 7fcceeda7640 1 -- 172.21.15.29:0/2077831130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcce815df70 msgr2=0x7fcce81520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:18.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.978+0000 7fcceeda7640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcce815df70 0x7fcce81520a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fcce000efc0 tx=0x7fcce000c490 comp rx=0 tx=0).stop 2024-04-25T00:33:18.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.978+0000 7fcceeda7640 1 -- 172.21.15.29:0/2077831130 shutdown_connections 2024-04-25T00:33:18.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.978+0000 7fcceeda7640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcce815df70 0x7fcce81520a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.978+0000 7fcceeda7640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fccd4067a70 0x7fccd4069f30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.978+0000 7fcceeda7640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce80b4d30 0x7fcce80ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.978+0000 7fcceeda7640 1 --2- 172.21.15.29:0/2077831130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce806d210 0x7fcce8142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:18.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.978+0000 7fcceeda7640 1 -- 172.21.15.29:0/2077831130 >> 172.21.15.29:0/2077831130 conn(0x7fcce80b1870 msgr2=0x7fcce815d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:18.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.978+0000 7fcceeda7640 1 -- 172.21.15.29:0/2077831130 shutdown_connections 2024-04-25T00:33:18.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:18.978+0000 7fcceeda7640 1 -- 172.21.15.29:0/2077831130 wait complete. 2024-04-25T00:33:18.999 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T00:33:18.999 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:33:19.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42d5132640 1 Processor -- start 2024-04-25T00:33:19.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42d5132640 1 -- start start 2024-04-25T00:33:19.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42d5132640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42d015e200 0x7f42d00d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:19.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42d5132640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d00cfc10 con 0x7f42d00d2c60 2024-04-25T00:33:19.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42d5132640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d00cfef0 con 0x7f42d00cf690 2024-04-25T00:33:19.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42d5132640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f42d00b38d0 con 0x7f42d015e200 2024-04-25T00:33:19.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42cf59e640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f42d00d2c60 0x7f42d00b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59026/0 (socket says 172.21.15.29:59026) 2024-04-25T00:33:19.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42cf59e640 1 -- 172.21.15.29:0/431146892 learned_addr learned my addr 172.21.15.29:0/431146892 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:19.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1522417531 0 0) 0x7f42d00cfc10 con 0x7f42d00d2c60 2024-04-25T00:33:19.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42bc003530 con 0x7f42d00d2c60 2024-04-25T00:33:19.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ced9d640 1 --2- 172.21.15.29:0/431146892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42d015e200 0x7f42d00d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:19.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3934734147 0 0) 0x7f42bc003530 con 0x7f42d00d2c60 2024-04-25T00:33:19.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42d015e200 msgr2=0x7f42d00d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:19.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 --2- 172.21.15.29:0/431146892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42d015e200 0x7f42d00d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:19.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 >> v1:172.21.15.157:6789/0 conn(0x7f42d00cf690 legacy=0x7f42d00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:19.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d00b3b20 con 0x7f42d00d2c60 2024-04-25T00:33:19.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f42c8001a00 con 0x7f42d00d2c60 2024-04-25T00:33:19.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 >> v1:172.21.15.29:6789/0 conn(0x7f42d00d2c60 legacy=0x7f42d00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:19.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 --2- 172.21.15.29:0/431146892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42bc0039f0 0x7f42bc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:19.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 --2- 172.21.15.29:0/431146892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42d015e200 0x7f42bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:19.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 --2- 172.21.15.29:0/431146892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42bc004d50 0x7f42bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:19.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f42d00b38d0 con 0x7f42bc0039f0 2024-04-25T00:33:19.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc00ba80 con 0x7f42bc004d50 2024-04-25T00:33:19.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f42bc00bc80 con 0x7f42d015e200 2024-04-25T00:33:19.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ced9d640 1 --2- 172.21.15.29:0/431146892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42bc0039f0 0x7f42bc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:19.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ced9d640 1 -- 172.21.15.29:0/431146892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42d015e200 msgr2=0x7f42bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:19.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ced9d640 1 --2- 172.21.15.29:0/431146892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42d015e200 0x7f42bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:19.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ced9d640 1 -- 172.21.15.29:0/431146892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42bc004d50 msgr2=0x7f42bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:19.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42cf59e640 1 --2- 172.21.15.29:0/431146892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42bc004d50 0x7f42bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:19.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ced9d640 1 --2- 172.21.15.29:0/431146892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42bc004d50 0x7f42bc0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:19.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.106+0000 7f42ced9d640 1 -- 172.21.15.29:0/431146892 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc003530 con 0x7f42bc0039f0 2024-04-25T00:33:19.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42ced9d640 1 --2- 172.21.15.29:0/431146892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42bc0039f0 0x7f42bc003e10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009ab0 tx=0x7f42c001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba93e3879c540a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:19.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c00b2070 con 0x7f42bc0039f0 2024-04-25T00:33:19.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c00a5560 con 0x7f42bc0039f0 2024-04-25T00:33:19.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42ce59c640 1 -- 172.21.15.29:0/431146892 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c00096b0 con 0x7f42bc0039f0 2024-04-25T00:33:19.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 -- 172.21.15.29:0/431146892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42bc0039f0 msgr2=0x7f42bc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:19.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 --2- 172.21.15.29:0/431146892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42bc0039f0 0x7f42bc003e10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009ab0 tx=0x7f42c001ca60 comp rx=0 tx=0).stop 2024-04-25T00:33:19.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42cf59e640 1 -- 172.21.15.29:0/431146892 reap_dead start 2024-04-25T00:33:19.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 -- 172.21.15.29:0/431146892 shutdown_connections 2024-04-25T00:33:19.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 -- 172.21.15.29:0/431146892 >> 172.21.15.29:0/431146892 conn(0x7f42d009d280 msgr2=0x7f42d00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:19.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 -- 172.21.15.29:0/431146892 shutdown_connections 2024-04-25T00:33:19.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 -- 172.21.15.29:0/431146892 wait complete. 2024-04-25T00:33:19.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 Processor -- start 2024-04-25T00:33:19.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 -- start start 2024-04-25T00:33:19.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42bc0039f0 0x7f42d00b94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:19.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42d0151f90 0x7f42d00cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:19.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d015fe40 0x7f42d01602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:19.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f42d00b4e40 con 0x7f42bc0039f0 2024-04-25T00:33:19.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f42d00b4cc0 con 0x7f42d015fe40 2024-04-25T00:33:19.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42d5132640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f42d00b4fc0 con 0x7f42d0151f90 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42ced9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42bc0039f0 0x7f42d00b94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42ced9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42bc0039f0 0x7f42d00b94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43410/0 (socket says 172.21.15.29:43410) 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42ced9d640 1 -- 172.21.15.29:0/1427158983 learned_addr learned my addr 172.21.15.29:0/1427158983 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42ced9d640 1 -- 172.21.15.29:0/1427158983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42d0151f90 msgr2=0x7f42d00cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42ced9d640 1 --2- 172.21.15.29:0/1427158983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42d0151f90 0x7f42d00cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42ced9d640 1 -- 172.21.15.29:0/1427158983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d015fe40 msgr2=0x7f42d01602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42ced9d640 1 --2- 172.21.15.29:0/1427158983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d015fe40 0x7f42d01602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42ced9d640 1 -- 172.21.15.29:0/1427158983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d0152490 con 0x7f42bc0039f0 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.110+0000 7f42ced9d640 1 --2- 172.21.15.29:0/1427158983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42bc0039f0 0x7f42d00b94d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f42c001c890 tx=0x7f42bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.114+0000 7f42c77fe640 1 -- 172.21.15.29:0/1427158983 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c00bd070 con 0x7f42bc0039f0 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.114+0000 7f42c77fe640 1 -- 172.21.15.29:0/1427158983 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c00a5d90 con 0x7f42bc0039f0 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.114+0000 7f42c77fe640 1 -- 172.21.15.29:0/1427158983 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c00b2040 con 0x7f42bc0039f0 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.114+0000 7f42d5132640 1 -- 172.21.15.29:0/1427158983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d00c7b40 con 0x7f42bc0039f0 2024-04-25T00:33:19.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.114+0000 7f42d5132640 1 -- 172.21.15.29:0/1427158983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d0152770 con 0x7f42bc0039f0 2024-04-25T00:33:19.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.114+0000 7f42d5132640 1 -- 172.21.15.29:0/1427158983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d0069270 con 0x7f42bc0039f0 2024-04-25T00:33:19.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.114+0000 7f42c77fe640 1 -- 172.21.15.29:0/1427158983 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f42c00c0070 con 0x7f42bc0039f0 2024-04-25T00:33:19.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.114+0000 7f42c77fe640 1 --2- 172.21.15.29:0/1427158983 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42b0067aa0 0x7f42b0069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:19.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.114+0000 7f42c77fe640 1 -- 172.21.15.29:0/1427158983 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f42c011f930 con 0x7f42bc0039f0 2024-04-25T00:33:19.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.114+0000 7f42c7fff640 1 --2- 172.21.15.29:0/1427158983 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42b0067aa0 0x7f42b0069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:19.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.114+0000 7f42c7fff640 1 --2- 172.21.15.29:0/1427158983 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42b0067aa0 0x7f42b0069f60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f42b8007fd0 tx=0x7f42b80079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:19.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.118+0000 7f42c77fe640 1 -- 172.21.15.29:0/1427158983 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f42c00167a0 con 0x7f42bc0039f0 2024-04-25T00:33:19.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.270+0000 7f42d5132640 1 -- 172.21.15.29:0/1427158983 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42d00d1ce0 con 0x7f42b0067aa0 2024-04-25T00:33:19.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.290+0000 7f42c77fe640 1 -- 172.21.15.29:0/1427158983 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246641 (secure 0 0 0) 0x7f42d00d1ce0 con 0x7f42b0067aa0 2024-04-25T00:33:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:19.296 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:33:19.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.290+0000 7f42d5132640 1 -- 172.21.15.29:0/1427158983 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42b0067aa0 msgr2=0x7f42b0069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:19.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.290+0000 7f42d5132640 1 --2- 172.21.15.29:0/1427158983 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42b0067aa0 0x7f42b0069f60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f42b8007fd0 tx=0x7f42b80079c0 comp rx=0 tx=0).stop 2024-04-25T00:33:19.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.290+0000 7f42d5132640 1 -- 172.21.15.29:0/1427158983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42bc0039f0 msgr2=0x7f42d00b94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:19.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.290+0000 7f42d5132640 1 --2- 172.21.15.29:0/1427158983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42bc0039f0 0x7f42d00b94d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f42c001c890 tx=0x7f42bc005840 comp rx=0 tx=0).stop 2024-04-25T00:33:19.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.290+0000 7f42d5132640 1 -- 172.21.15.29:0/1427158983 shutdown_connections 2024-04-25T00:33:19.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.290+0000 7f42d5132640 1 --2- 172.21.15.29:0/1427158983 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42b0067aa0 0x7f42b0069f60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:19.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.290+0000 7f42d5132640 1 --2- 172.21.15.29:0/1427158983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d015fe40 0x7f42d01602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:19.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.290+0000 7f42d5132640 1 --2- 172.21.15.29:0/1427158983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42d0151f90 0x7f42d00cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:19.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.290+0000 7f42d5132640 1 --2- 172.21.15.29:0/1427158983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42bc0039f0 0x7f42d00b94d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:19.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.290+0000 7f42d5132640 1 -- 172.21.15.29:0/1427158983 >> 172.21.15.29:0/1427158983 conn(0x7f42d009d280 msgr2=0x7f42d006cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:19.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.290+0000 7f42d5132640 1 -- 172.21.15.29:0/1427158983 shutdown_connections 2024-04-25T00:33:19.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:19.290+0000 7f42d5132640 1 -- 172.21.15.29:0/1427158983 wait complete. 2024-04-25T00:33:19.310 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-04-25T00:33:17.480927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258736,"kb_used_data":5096,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715984,"statfs":{"total":773094113280,"available":772829167616,"internally_reserved":0,"allocated":5218304,"data_stored":2799840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.723742"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.786161+0000","last_change":"2024-04-25T00:33:12.786161+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.786161+0000","last_undegraded":"2024-04-25T00:33:12.786161+0000","last_fullsized":"2024-04-25T00:33:12.786161+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778155+0000","last_change":"2024-04-25T00:33:12.778155+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778155+0000","last_undegraded":"2024-04-25T00:33:12.778155+0000","last_fullsized":"2024-04-25T00:33:12.778155+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772955+0000","last_change":"2024-04-25T00:33:12.772955+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772955+0000","last_undegraded":"2024-04-25T00:33:12.772955+0000","last_fullsized":"2024-04-25T00:33:12.772955+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779053+0000","last_change":"2024-04-25T00:33:12.779053+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779053+0000","last_undegraded":"2024-04-25T00:33:12.779053+0000","last_fullsized":"2024-04-25T00:33:12.779053+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779330+0000","last_change":"2024-04-25T00:33:12.779330+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779330+0000","last_undegraded":"2024-04-25T00:33:12.779330+0000","last_fullsized":"2024-04-25T00:33:12.779330+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785451+0000","last_change":"2024-04-25T00:33:12.785451+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785451+0000","last_undegraded":"2024-04-25T00:33:12.785451+0000","last_fullsized":"2024-04-25T00:33:12.785451+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781016+0000","last_change":"2024-04-25T00:33:12.781016+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781016+0000","last_undegraded":"2024-04-25T00:33:12.781016+0000","last_fullsized":"2024-04-25T00:33:12.781016+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779106+0000","last_change":"2024-04-25T00:33:12.779106+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779106+0000","last_undegraded":"2024-04-25T00:33:12.779106+0000","last_fullsized":"2024-04-25T00:33:12.779106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776612+0000","last_change":"2024-04-25T00:33:12.776612+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776612+0000","last_undegraded":"2024-04-25T00:33:12.776612+0000","last_fullsized":"2024-04-25T00:33:12.776612+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.777743+0000","last_change":"2024-04-25T00:33:12.777743+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.777743+0000","last_undegraded":"2024-04-25T00:33:12.777743+0000","last_fullsized":"2024-04-25T00:33:12.777743+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773992+0000","last_change":"2024-04-25T00:33:12.773992+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773992+0000","last_undegraded":"2024-04-25T00:33:12.773992+0000","last_fullsized":"2024-04-25T00:33:12.773992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.786788+0000","last_change":"2024-04-25T00:33:12.786788+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.786788+0000","last_undegraded":"2024-04-25T00:33:12.786788+0000","last_fullsized":"2024-04-25T00:33:12.786788+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776214+0000","last_change":"2024-04-25T00:33:12.776214+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776214+0000","last_undegraded":"2024-04-25T00:33:12.776214+0000","last_fullsized":"2024-04-25T00:33:12.776214+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779230+0000","last_change":"2024-04-25T00:33:12.779230+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779230+0000","last_undegraded":"2024-04-25T00:33:12.779230+0000","last_fullsized":"2024-04-25T00:33:12.779230+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779729+0000","last_change":"2024-04-25T00:33:12.779729+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779729+0000","last_undegraded":"2024-04-25T00:33:12.779729+0000","last_fullsized":"2024-04-25T00:33:12.779729+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781755+0000","last_change":"2024-04-25T00:33:12.781755+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781755+0000","last_undegraded":"2024-04-25T00:33:12.781755+0000","last_fullsized":"2024-04-25T00:33:12.781755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.775241+0000","last_change":"2024-04-25T00:33:12.775241+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.775241+0000","last_undegraded":"2024-04-25T00:33:12.775241+0000","last_fullsized":"2024-04-25T00:33:12.775241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.774539+0000","last_change":"2024-04-25T00:33:12.774539+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.774539+0000","last_undegraded":"2024-04-25T00:33:12.774539+0000","last_fullsized":"2024-04-25T00:33:12.774539+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779300+0000","last_change":"2024-04-25T00:33:12.779300+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779300+0000","last_undegraded":"2024-04-25T00:33:12.779300+0000","last_fullsized":"2024-04-25T00:33:12.779300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776553+0000","last_change":"2024-04-25T00:33:12.776553+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776553+0000","last_undegraded":"2024-04-25T00:33:12.776553+0000","last_fullsized":"2024-04-25T00:33:12.776553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784989+0000","last_change":"2024-04-25T00:33:12.784989+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784989+0000","last_undegraded":"2024-04-25T00:33:12.784989+0000","last_fullsized":"2024-04-25T00:33:12.784989+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771659+0000","last_change":"2024-04-25T00:33:12.771659+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771659+0000","last_undegraded":"2024-04-25T00:33:12.771659+0000","last_fullsized":"2024-04-25T00:33:12.771659+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778391+0000","last_change":"2024-04-25T00:33:12.778391+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778391+0000","last_undegraded":"2024-04-25T00:33:12.778391+0000","last_fullsized":"2024-04-25T00:33:12.778391+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780527+0000","last_change":"2024-04-25T00:33:12.780527+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780527+0000","last_undegraded":"2024-04-25T00:33:12.780527+0000","last_fullsized":"2024-04-25T00:33:12.780527+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.786781+0000","last_change":"2024-04-25T00:33:12.786781+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.786781+0000","last_undegraded":"2024-04-25T00:33:12.786781+0000","last_fullsized":"2024-04-25T00:33:12.786781+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.776393+0000","last_change":"2024-04-25T00:33:12.776393+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.776393+0000","last_undegraded":"2024-04-25T00:33:12.776393+0000","last_fullsized":"2024-04-25T00:33:12.776393+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771755+0000","last_change":"2024-04-25T00:33:12.771755+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771755+0000","last_undegraded":"2024-04-25T00:33:12.771755+0000","last_fullsized":"2024-04-25T00:33:12.771755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785983+0000","last_change":"2024-04-25T00:33:12.785983+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785983+0000","last_undegraded":"2024-04-25T00:33:12.785983+0000","last_fullsized":"2024-04-25T00:33:12.785983+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781215+0000","last_change":"2024-04-25T00:33:12.781215+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781215+0000","last_undegraded":"2024-04-25T00:33:12.781215+0000","last_fullsized":"2024-04-25T00:33:12.781215+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773770+0000","last_change":"2024-04-25T00:33:12.773770+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773770+0000","last_undegraded":"2024-04-25T00:33:12.773770+0000","last_fullsized":"2024-04-25T00:33:12.773770+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778731+0000","last_change":"2024-04-25T00:33:12.778731+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778731+0000","last_undegraded":"2024-04-25T00:33:12.778731+0000","last_fullsized":"2024-04-25T00:33:12.778731+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.781274+0000","last_change":"2024-04-25T00:33:12.781274+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.781274+0000","last_undegraded":"2024-04-25T00:33:12.781274+0000","last_fullsized":"2024-04-25T00:33:12.781274+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.783167+0000","last_change":"2024-04-25T00:33:12.783167+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.783167+0000","last_undegraded":"2024-04-25T00:33:12.783167+0000","last_fullsized":"2024-04-25T00:33:12.783167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802167+0000","last_change":"2024-04-25T00:33:12.801022+0000","last_active":"2024-04-25T00:33:12.802167+0000","last_peered":"2024-04-25T00:33:12.802167+0000","last_clean":"2024-04-25T00:33:12.802167+0000","last_became_active":"2024-04-25T00:33:12.800641+0000","last_became_peered":"2024-04-25T00:33:12.800641+0000","last_unstale":"2024-04-25T00:33:12.802167+0000","last_undegraded":"2024-04-25T00:33:12.802167+0000","last_fullsized":"2024-04-25T00:33:12.802167+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:10.074900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785776+0000","last_change":"2024-04-25T00:33:12.785776+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785776+0000","last_undegraded":"2024-04-25T00:33:12.785776+0000","last_fullsized":"2024-04-25T00:33:12.785776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799161+0000","last_change":"2024-04-25T00:33:12.797818+0000","last_active":"2024-04-25T00:33:12.799161+0000","last_peered":"2024-04-25T00:33:12.799161+0000","last_clean":"2024-04-25T00:33:12.799161+0000","last_became_active":"2024-04-25T00:33:12.796615+0000","last_became_peered":"2024-04-25T00:33:12.796615+0000","last_unstale":"2024-04-25T00:33:12.799161+0000","last_undegraded":"2024-04-25T00:33:12.799161+0000","last_fullsized":"2024-04-25T00:33:12.799161+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:38.919514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785220+0000","last_change":"2024-04-25T00:33:12.785220+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785220+0000","last_undegraded":"2024-04-25T00:33:12.785220+0000","last_fullsized":"2024-04-25T00:33:12.785220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799612+0000","last_change":"2024-04-25T00:33:12.797893+0000","last_active":"2024-04-25T00:33:12.799612+0000","last_peered":"2024-04-25T00:33:12.799612+0000","last_clean":"2024-04-25T00:33:12.799612+0000","last_became_active":"2024-04-25T00:33:12.797499+0000","last_became_peered":"2024-04-25T00:33:12.797499+0000","last_unstale":"2024-04-25T00:33:12.799612+0000","last_undegraded":"2024-04-25T00:33:12.799612+0000","last_fullsized":"2024-04-25T00:33:12.799612+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:54.884884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780797+0000","last_change":"2024-04-25T00:33:12.780797+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780797+0000","last_undegraded":"2024-04-25T00:33:12.780797+0000","last_fullsized":"2024-04-25T00:33:12.780797+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.809109+0000","last_change":"2024-04-25T00:33:12.808013+0000","last_active":"2024-04-25T00:33:12.809109+0000","last_peered":"2024-04-25T00:33:12.809109+0000","last_clean":"2024-04-25T00:33:12.809109+0000","last_became_active":"2024-04-25T00:33:12.807497+0000","last_became_peered":"2024-04-25T00:33:12.807497+0000","last_unstale":"2024-04-25T00:33:12.809109+0000","last_undegraded":"2024-04-25T00:33:12.809109+0000","last_fullsized":"2024-04-25T00:33:12.809109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:13.091178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772640+0000","last_change":"2024-04-25T00:33:12.772640+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772640+0000","last_undegraded":"2024-04-25T00:33:12.772640+0000","last_fullsized":"2024-04-25T00:33:12.772640+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.803068+0000","last_change":"2024-04-25T00:33:12.801886+0000","last_active":"2024-04-25T00:33:12.803068+0000","last_peered":"2024-04-25T00:33:12.803068+0000","last_clean":"2024-04-25T00:33:12.803068+0000","last_became_active":"2024-04-25T00:33:12.801570+0000","last_became_peered":"2024-04-25T00:33:12.801570+0000","last_unstale":"2024-04-25T00:33:12.803068+0000","last_undegraded":"2024-04-25T00:33:12.803068+0000","last_fullsized":"2024-04-25T00:33:12.803068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:37.757927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772470+0000","last_change":"2024-04-25T00:33:12.772470+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772470+0000","last_undegraded":"2024-04-25T00:33:12.772470+0000","last_fullsized":"2024-04-25T00:33:12.772470+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.797304+0000","last_change":"2024-04-25T00:33:12.795400+0000","last_active":"2024-04-25T00:33:12.797304+0000","last_peered":"2024-04-25T00:33:12.797304+0000","last_clean":"2024-04-25T00:33:12.797304+0000","last_became_active":"2024-04-25T00:33:12.794892+0000","last_became_peered":"2024-04-25T00:33:12.794892+0000","last_unstale":"2024-04-25T00:33:12.797304+0000","last_undegraded":"2024-04-25T00:33:12.797304+0000","last_fullsized":"2024-04-25T00:33:12.797304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:11.471922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.772301+0000","last_change":"2024-04-25T00:33:12.772301+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.772301+0000","last_undegraded":"2024-04-25T00:33:12.772301+0000","last_fullsized":"2024-04-25T00:33:12.772301+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.801950+0000","last_change":"2024-04-25T00:33:12.794993+0000","last_active":"2024-04-25T00:33:12.801950+0000","last_peered":"2024-04-25T00:33:12.801950+0000","last_clean":"2024-04-25T00:33:12.801950+0000","last_became_active":"2024-04-25T00:33:12.794251+0000","last_became_peered":"2024-04-25T00:33:12.794251+0000","last_unstale":"2024-04-25T00:33:12.801950+0000","last_undegraded":"2024-04-25T00:33:12.801950+0000","last_fullsized":"2024-04-25T00:33:12.801950+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:05.611837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779496+0000","last_change":"2024-04-25T00:33:12.779496+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779496+0000","last_undegraded":"2024-04-25T00:33:12.779496+0000","last_fullsized":"2024-04-25T00:33:12.779496+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.795732+0000","last_change":"2024-04-25T00:33:12.793905+0000","last_active":"2024-04-25T00:33:12.795732+0000","last_peered":"2024-04-25T00:33:12.795732+0000","last_clean":"2024-04-25T00:33:12.795732+0000","last_became_active":"2024-04-25T00:33:12.791067+0000","last_became_peered":"2024-04-25T00:33:12.791067+0000","last_unstale":"2024-04-25T00:33:12.795732+0000","last_undegraded":"2024-04-25T00:33:12.795732+0000","last_fullsized":"2024-04-25T00:33:12.795732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:26.419741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773456+0000","last_change":"2024-04-25T00:33:12.773456+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773456+0000","last_undegraded":"2024-04-25T00:33:12.773456+0000","last_fullsized":"2024-04-25T00:33:12.773456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.793147+0000","last_change":"2024-04-25T00:33:12.791338+0000","last_active":"2024-04-25T00:33:12.793147+0000","last_peered":"2024-04-25T00:33:12.793147+0000","last_clean":"2024-04-25T00:33:12.793147+0000","last_became_active":"2024-04-25T00:33:12.790547+0000","last_became_peered":"2024-04-25T00:33:12.790547+0000","last_unstale":"2024-04-25T00:33:12.793147+0000","last_undegraded":"2024-04-25T00:33:12.793147+0000","last_fullsized":"2024-04-25T00:33:12.793147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:42.648522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785118+0000","last_change":"2024-04-25T00:33:12.785118+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785118+0000","last_undegraded":"2024-04-25T00:33:12.785118+0000","last_fullsized":"2024-04-25T00:33:12.785118+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802404+0000","last_change":"2024-04-25T00:33:12.800591+0000","last_active":"2024-04-25T00:33:12.802404+0000","last_peered":"2024-04-25T00:33:12.802404+0000","last_clean":"2024-04-25T00:33:12.802404+0000","last_became_active":"2024-04-25T00:33:12.800301+0000","last_became_peered":"2024-04-25T00:33:12.800301+0000","last_unstale":"2024-04-25T00:33:12.802404+0000","last_undegraded":"2024-04-25T00:33:12.802404+0000","last_fullsized":"2024-04-25T00:33:12.802404+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:51.270346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.782338+0000","last_change":"2024-04-25T00:33:12.782338+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.782338+0000","last_undegraded":"2024-04-25T00:33:12.782338+0000","last_fullsized":"2024-04-25T00:33:12.782338+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.811725+0000","last_change":"2024-04-25T00:33:12.810318+0000","last_active":"2024-04-25T00:33:12.811725+0000","last_peered":"2024-04-25T00:33:12.811725+0000","last_clean":"2024-04-25T00:33:12.811725+0000","last_became_active":"2024-04-25T00:33:12.809691+0000","last_became_peered":"2024-04-25T00:33:12.809691+0000","last_unstale":"2024-04-25T00:33:12.811725+0000","last_undegraded":"2024-04-25T00:33:12.811725+0000","last_fullsized":"2024-04-25T00:33:12.811725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:31.459566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784334+0000","last_change":"2024-04-25T00:33:12.784334+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784334+0000","last_undegraded":"2024-04-25T00:33:12.784334+0000","last_fullsized":"2024-04-25T00:33:12.784334+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.788652+0000","last_change":"2024-04-25T00:33:12.787615+0000","last_active":"2024-04-25T00:33:12.788652+0000","last_peered":"2024-04-25T00:33:12.788652+0000","last_clean":"2024-04-25T00:33:12.788652+0000","last_became_active":"2024-04-25T00:33:12.787329+0000","last_became_peered":"2024-04-25T00:33:12.787329+0000","last_unstale":"2024-04-25T00:33:12.788652+0000","last_undegraded":"2024-04-25T00:33:12.788652+0000","last_fullsized":"2024-04-25T00:33:12.788652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:26.034657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.778244+0000","last_change":"2024-04-25T00:33:12.778244+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.778244+0000","last_undegraded":"2024-04-25T00:33:12.778244+0000","last_fullsized":"2024-04-25T00:33:12.778244+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.798527+0000","last_change":"2024-04-25T00:33:12.795864+0000","last_active":"2024-04-25T00:33:12.798527+0000","last_peered":"2024-04-25T00:33:12.798527+0000","last_clean":"2024-04-25T00:33:12.798527+0000","last_became_active":"2024-04-25T00:33:12.795591+0000","last_became_peered":"2024-04-25T00:33:12.795591+0000","last_unstale":"2024-04-25T00:33:12.798527+0000","last_undegraded":"2024-04-25T00:33:12.798527+0000","last_fullsized":"2024-04-25T00:33:12.798527+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:41.044585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771469+0000","last_change":"2024-04-25T00:33:12.771469+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771469+0000","last_undegraded":"2024-04-25T00:33:12.771469+0000","last_fullsized":"2024-04-25T00:33:12.771469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.807386+0000","last_change":"2024-04-25T00:33:12.806163+0000","last_active":"2024-04-25T00:33:12.807386+0000","last_peered":"2024-04-25T00:33:12.807386+0000","last_clean":"2024-04-25T00:33:12.807386+0000","last_became_active":"2024-04-25T00:33:12.802549+0000","last_became_peered":"2024-04-25T00:33:12.802549+0000","last_unstale":"2024-04-25T00:33:12.807386+0000","last_undegraded":"2024-04-25T00:33:12.807386+0000","last_fullsized":"2024-04-25T00:33:12.807386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:41.044585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.777997+0000","last_change":"2024-04-25T00:33:12.777997+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.777997+0000","last_undegraded":"2024-04-25T00:33:12.777997+0000","last_fullsized":"2024-04-25T00:33:12.777997+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799122+0000","last_change":"2024-04-25T00:33:12.797153+0000","last_active":"2024-04-25T00:33:12.799122+0000","last_peered":"2024-04-25T00:33:12.799122+0000","last_clean":"2024-04-25T00:33:12.799122+0000","last_became_active":"2024-04-25T00:33:12.796727+0000","last_became_peered":"2024-04-25T00:33:12.796727+0000","last_unstale":"2024-04-25T00:33:12.799122+0000","last_undegraded":"2024-04-25T00:33:12.799122+0000","last_fullsized":"2024-04-25T00:33:12.799122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:13.091178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779937+0000","last_change":"2024-04-25T00:33:12.779937+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779937+0000","last_undegraded":"2024-04-25T00:33:12.779937+0000","last_fullsized":"2024-04-25T00:33:12.779937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.800089+0000","last_change":"2024-04-25T00:33:12.799231+0000","last_active":"2024-04-25T00:33:12.800089+0000","last_peered":"2024-04-25T00:33:12.800089+0000","last_clean":"2024-04-25T00:33:12.800089+0000","last_became_active":"2024-04-25T00:33:12.798864+0000","last_became_peered":"2024-04-25T00:33:12.798864+0000","last_unstale":"2024-04-25T00:33:12.800089+0000","last_undegraded":"2024-04-25T00:33:12.800089+0000","last_fullsized":"2024-04-25T00:33:12.800089+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:56.668429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.775446+0000","last_change":"2024-04-25T00:33:12.775446+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.775446+0000","last_undegraded":"2024-04-25T00:33:12.775446+0000","last_fullsized":"2024-04-25T00:33:12.775446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.792247+0000","last_change":"2024-04-25T00:33:12.788985+0000","last_active":"2024-04-25T00:33:12.792247+0000","last_peered":"2024-04-25T00:33:12.792247+0000","last_clean":"2024-04-25T00:33:12.792247+0000","last_became_active":"2024-04-25T00:33:12.788443+0000","last_became_peered":"2024-04-25T00:33:12.788443+0000","last_unstale":"2024-04-25T00:33:12.792247+0000","last_undegraded":"2024-04-25T00:33:12.792247+0000","last_fullsized":"2024-04-25T00:33:12.792247+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:33.029999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773102+0000","last_change":"2024-04-25T00:33:12.773102+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773102+0000","last_undegraded":"2024-04-25T00:33:12.773102+0000","last_fullsized":"2024-04-25T00:33:12.773102+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.797229+0000","last_change":"2024-04-25T00:33:12.794976+0000","last_active":"2024-04-25T00:33:12.797229+0000","last_peered":"2024-04-25T00:33:12.797229+0000","last_clean":"2024-04-25T00:33:12.797229+0000","last_became_active":"2024-04-25T00:33:12.793226+0000","last_became_peered":"2024-04-25T00:33:12.793226+0000","last_unstale":"2024-04-25T00:33:12.797229+0000","last_undegraded":"2024-04-25T00:33:12.797229+0000","last_fullsized":"2024-04-25T00:33:12.797229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:52.568911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780454+0000","last_change":"2024-04-25T00:33:12.780454+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780454+0000","last_undegraded":"2024-04-25T00:33:12.780454+0000","last_fullsized":"2024-04-25T00:33:12.780454+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802028+0000","last_change":"2024-04-25T00:33:12.794824+0000","last_active":"2024-04-25T00:33:12.802028+0000","last_peered":"2024-04-25T00:33:12.802028+0000","last_clean":"2024-04-25T00:33:12.802028+0000","last_became_active":"2024-04-25T00:33:12.794088+0000","last_became_peered":"2024-04-25T00:33:12.794088+0000","last_unstale":"2024-04-25T00:33:12.802028+0000","last_undegraded":"2024-04-25T00:33:12.802028+0000","last_fullsized":"2024-04-25T00:33:12.802028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:33.703646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785147+0000","last_change":"2024-04-25T00:33:12.785147+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785147+0000","last_undegraded":"2024-04-25T00:33:12.785147+0000","last_fullsized":"2024-04-25T00:33:12.785147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802238+0000","last_change":"2024-04-25T00:33:12.801012+0000","last_active":"2024-04-25T00:33:12.802238+0000","last_peered":"2024-04-25T00:33:12.802238+0000","last_clean":"2024-04-25T00:33:12.802238+0000","last_became_active":"2024-04-25T00:33:12.800555+0000","last_became_peered":"2024-04-25T00:33:12.800555+0000","last_unstale":"2024-04-25T00:33:12.802238+0000","last_undegraded":"2024-04-25T00:33:12.802238+0000","last_fullsized":"2024-04-25T00:33:12.802238+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:31.459566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785063+0000","last_change":"2024-04-25T00:33:12.785063+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785063+0000","last_undegraded":"2024-04-25T00:33:12.785063+0000","last_fullsized":"2024-04-25T00:33:12.785063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802176+0000","last_change":"2024-04-25T00:33:12.801088+0000","last_active":"2024-04-25T00:33:12.802176+0000","last_peered":"2024-04-25T00:33:12.802176+0000","last_clean":"2024-04-25T00:33:12.802176+0000","last_became_active":"2024-04-25T00:33:12.800686+0000","last_became_peered":"2024-04-25T00:33:12.800686+0000","last_unstale":"2024-04-25T00:33:12.802176+0000","last_undegraded":"2024-04-25T00:33:12.802176+0000","last_fullsized":"2024-04-25T00:33:12.802176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:04.740368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.785871+0000","last_change":"2024-04-25T00:33:12.785871+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.785871+0000","last_undegraded":"2024-04-25T00:33:12.785871+0000","last_fullsized":"2024-04-25T00:33:12.785871+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.806654+0000","last_change":"2024-04-25T00:33:12.804216+0000","last_active":"2024-04-25T00:33:12.806654+0000","last_peered":"2024-04-25T00:33:12.806654+0000","last_clean":"2024-04-25T00:33:12.806654+0000","last_became_active":"2024-04-25T00:33:12.800818+0000","last_became_peered":"2024-04-25T00:33:12.800818+0000","last_unstale":"2024-04-25T00:33:12.806654+0000","last_undegraded":"2024-04-25T00:33:12.806654+0000","last_fullsized":"2024-04-25T00:33:12.806654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:04.901463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780423+0000","last_change":"2024-04-25T00:33:12.780423+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780423+0000","last_undegraded":"2024-04-25T00:33:12.780423+0000","last_fullsized":"2024-04-25T00:33:12.780423+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.795832+0000","last_change":"2024-04-25T00:33:12.793774+0000","last_active":"2024-04-25T00:33:12.795832+0000","last_peered":"2024-04-25T00:33:12.795832+0000","last_clean":"2024-04-25T00:33:12.795832+0000","last_became_active":"2024-04-25T00:33:12.793317+0000","last_became_peered":"2024-04-25T00:33:12.793317+0000","last_unstale":"2024-04-25T00:33:12.795832+0000","last_undegraded":"2024-04-25T00:33:12.795832+0000","last_fullsized":"2024-04-25T00:33:12.795832+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:18.045380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773072+0000","last_change":"2024-04-25T00:33:12.773072+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773072+0000","last_undegraded":"2024-04-25T00:33:12.773072+0000","last_fullsized":"2024-04-25T00:33:12.773072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.807370+0000","last_change":"2024-04-25T00:33:12.805396+0000","last_active":"2024-04-25T00:33:12.807370+0000","last_peered":"2024-04-25T00:33:12.807370+0000","last_clean":"2024-04-25T00:33:12.807370+0000","last_became_active":"2024-04-25T00:33:12.803923+0000","last_became_peered":"2024-04-25T00:33:12.803923+0000","last_unstale":"2024-04-25T00:33:12.807370+0000","last_undegraded":"2024-04-25T00:33:12.807370+0000","last_fullsized":"2024-04-25T00:33:12.807370+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.189080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.780611+0000","last_change":"2024-04-25T00:33:12.780611+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.780611+0000","last_undegraded":"2024-04-25T00:33:12.780611+0000","last_fullsized":"2024-04-25T00:33:12.780611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.798496+0000","last_change":"2024-04-25T00:33:12.795318+0000","last_active":"2024-04-25T00:33:12.798496+0000","last_peered":"2024-04-25T00:33:12.798496+0000","last_clean":"2024-04-25T00:33:12.798496+0000","last_became_active":"2024-04-25T00:33:12.794708+0000","last_became_peered":"2024-04-25T00:33:12.794708+0000","last_unstale":"2024-04-25T00:33:12.798496+0000","last_undegraded":"2024-04-25T00:33:12.798496+0000","last_fullsized":"2024-04-25T00:33:12.798496+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.189080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.773274+0000","last_change":"2024-04-25T00:33:12.773274+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.773274+0000","last_undegraded":"2024-04-25T00:33:12.773274+0000","last_fullsized":"2024-04-25T00:33:12.773274+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.812310+0000","last_change":"2024-04-25T00:33:12.810928+0000","last_active":"2024-04-25T00:33:12.812310+0000","last_peered":"2024-04-25T00:33:12.812310+0000","last_clean":"2024-04-25T00:33:12.812310+0000","last_became_active":"2024-04-25T00:33:12.810284+0000","last_became_peered":"2024-04-25T00:33:12.810284+0000","last_unstale":"2024-04-25T00:33:12.812310+0000","last_undegraded":"2024-04-25T00:33:12.812310+0000","last_fullsized":"2024-04-25T00:33:12.812310+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:04.740368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.783434+0000","last_change":"2024-04-25T00:33:12.783434+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.783434+0000","last_undegraded":"2024-04-25T00:33:12.783434+0000","last_fullsized":"2024-04-25T00:33:12.783434+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.804755+0000","last_change":"2024-04-25T00:33:12.803739+0000","last_active":"2024-04-25T00:33:12.804755+0000","last_peered":"2024-04-25T00:33:12.804755+0000","last_clean":"2024-04-25T00:33:12.804755+0000","last_became_active":"2024-04-25T00:33:12.803381+0000","last_became_peered":"2024-04-25T00:33:12.803381+0000","last_unstale":"2024-04-25T00:33:12.804755+0000","last_undegraded":"2024-04-25T00:33:12.804755+0000","last_fullsized":"2024-04-25T00:33:12.804755+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:46.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784539+0000","last_change":"2024-04-25T00:33:12.784539+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784539+0000","last_undegraded":"2024-04-25T00:33:12.784539+0000","last_fullsized":"2024-04-25T00:33:12.784539+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.811877+0000","last_change":"2024-04-25T00:33:12.810436+0000","last_active":"2024-04-25T00:33:12.811877+0000","last_peered":"2024-04-25T00:33:12.811877+0000","last_clean":"2024-04-25T00:33:12.811877+0000","last_became_active":"2024-04-25T00:33:12.809773+0000","last_became_peered":"2024-04-25T00:33:12.809773+0000","last_unstale":"2024-04-25T00:33:12.811877+0000","last_undegraded":"2024-04-25T00:33:12.811877+0000","last_fullsized":"2024-04-25T00:33:12.811877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:10.074900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.784359+0000","last_change":"2024-04-25T00:33:12.784359+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.784359+0000","last_undegraded":"2024-04-25T00:33:12.784359+0000","last_fullsized":"2024-04-25T00:33:12.784359+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.793065+0000","last_change":"2024-04-25T00:33:12.791171+0000","last_active":"2024-04-25T00:33:12.793065+0000","last_peered":"2024-04-25T00:33:12.793065+0000","last_clean":"2024-04-25T00:33:12.793065+0000","last_became_active":"2024-04-25T00:33:12.790272+0000","last_became_peered":"2024-04-25T00:33:12.790272+0000","last_unstale":"2024-04-25T00:33:12.793065+0000","last_undegraded":"2024-04-25T00:33:12.793065+0000","last_fullsized":"2024-04-25T00:33:12.793065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:33.246491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.779265+0000","last_change":"2024-04-25T00:33:12.779265+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.779265+0000","last_undegraded":"2024-04-25T00:33:12.779265+0000","last_fullsized":"2024-04-25T00:33:12.779265+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.799547+0000","last_change":"2024-04-25T00:33:12.797798+0000","last_active":"2024-04-25T00:33:12.799547+0000","last_peered":"2024-04-25T00:33:12.799547+0000","last_clean":"2024-04-25T00:33:12.799547+0000","last_became_active":"2024-04-25T00:33:12.797086+0000","last_became_peered":"2024-04-25T00:33:12.797086+0000","last_unstale":"2024-04-25T00:33:12.799547+0000","last_undegraded":"2024-04-25T00:33:12.799547+0000","last_fullsized":"2024-04-25T00:33:12.799547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:38.919514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.771834+0000","last_change":"2024-04-25T00:33:12.771834+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.771834+0000","last_undegraded":"2024-04-25T00:33:12.771834+0000","last_fullsized":"2024-04-25T00:33:12.771834+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.795110+0000","last_change":"2024-04-25T00:33:12.789836+0000","last_active":"2024-04-25T00:33:12.795110+0000","last_peered":"2024-04-25T00:33:12.795110+0000","last_clean":"2024-04-25T00:33:12.795110+0000","last_became_active":"2024-04-25T00:33:12.789493+0000","last_became_peered":"2024-04-25T00:33:12.789493+0000","last_unstale":"2024-04-25T00:33:12.795110+0000","last_undegraded":"2024-04-25T00:33:12.795110+0000","last_fullsized":"2024-04-25T00:33:12.795110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:03.396230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T00:33:12.777158+0000","last_change":"2024-04-25T00:33:12.777158+0000","last_active":"2024-04-25T00:33:12.755846+0000","last_peered":"2024-04-25T00:33:12.755846+0000","last_clean":"2024-04-25T00:33:12.755846+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:33:12.777158+0000","last_undegraded":"2024-04-25T00:33:12.777158+0000","last_fullsized":"2024-04-25T00:33:12.777158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.802012+0000","last_change":"2024-04-25T00:33:12.795705+0000","last_active":"2024-04-25T00:33:12.802012+0000","last_peered":"2024-04-25T00:33:12.802012+0000","last_clean":"2024-04-25T00:33:12.802012+0000","last_became_active":"2024-04-25T00:33:12.795091+0000","last_became_peered":"2024-04-25T00:33:12.795091+0000","last_unstale":"2024-04-25T00:33:12.802012+0000","last_undegraded":"2024-04-25T00:33:12.802012+0000","last_fullsized":"2024-04-25T00:33:12.802012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:31.148769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T00:33:12.791555+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:33:12.791555+0000","last_peered":"2024-04-25T00:33:12.791555+0000","last_clean":"2024-04-25T00:33:12.791555+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:33:12.791555+0000","last_undegraded":"2024-04-25T00:33:12.791555+0000","last_fullsized":"2024-04-25T00:33:12.791555+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":792756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":202388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335424,"statfs":{"total":96636764160,"available":96599474176,"internally_reserved":0,"allocated":1097728,"data_stored":792756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:33:22.319 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:33:22.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7ff13ad640 1 Processor -- start 2024-04-25T00:33:22.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7ff13ad640 1 -- start start 2024-04-25T00:33:22.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7ff13ad640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fec06c950 0x7f7fec06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7ff13ad640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7fec15e440 0x7f7fec0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7ff13ad640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fec0cc3b0 con 0x7f7fec0d0ff0 2024-04-25T00:33:22.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7ff13ad640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7fec06d270 con 0x7f7fec15e440 2024-04-25T00:33:22.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7ff13ad640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7fec0b11d0 con 0x7f7fec06c950 2024-04-25T00:33:22.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7feb7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7fec0d0ff0 0x7f7fec0ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59028/0 (socket says 172.21.15.29:59028) 2024-04-25T00:33:22.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7feb7fe640 1 -- 172.21.15.29:0/1576858689 learned_addr learned my addr 172.21.15.29:0/1576858689 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:22.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7feaffd640 1 --2- 172.21.15.29:0/1576858689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fec06c950 0x7f7fec06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fe3fff640 1 --2- 172.21.15.29:0/1576858689 >> v2:172.21.15.157:3300/0 conn(0x7f7fec15e440 0x7f7fec0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1039570225 0 0) 0x7f7fec0cc3b0 con 0x7f7fec0d0ff0 2024-04-25T00:33:22.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fd8003530 con 0x7f7fec0d0ff0 2024-04-25T00:33:22.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3597891843 0 0) 0x7f7fd8003530 con 0x7f7fec0d0ff0 2024-04-25T00:33:22.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fec06c950 msgr2=0x7f7fec06cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:22.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 --2- 172.21.15.29:0/1576858689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fec06c950 0x7f7fec06cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 >> v2:172.21.15.157:3300/0 conn(0x7f7fec15e440 msgr2=0x7f7fec0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 --2- 172.21.15.29:0/1576858689 >> v2:172.21.15.157:3300/0 conn(0x7f7fec15e440 0x7f7fec0d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec0cc650 con 0x7f7fec0d0ff0 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f7fe4001a00 con 0x7f7fec0d0ff0 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 >> v1:172.21.15.29:6789/0 conn(0x7f7fec0d0ff0 legacy=0x7f7fec0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 --2- 172.21.15.29:0/1576858689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fec06c950 0x7f7fd80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 --2- 172.21.15.29:0/1576858689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd8003f40 0x7f7fd8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 --2- 172.21.15.29:0/1576858689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd8004d50 0x7f7fd80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec06d270 con 0x7f7fd8004d50 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec0b11d0 con 0x7f7fd8003f40 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd800b9d0 con 0x7f7fec06c950 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7feb7fe640 1 --2- 172.21.15.29:0/1576858689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd8004d50 0x7f7fd80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7fe3fff640 1 --2- 172.21.15.29:0/1576858689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd8003f40 0x7f7fd8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7feaffd640 1 --2- 172.21.15.29:0/1576858689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fec06c950 0x7f7fd80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7feb7fe640 1 -- 172.21.15.29:0/1576858689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fec06c950 msgr2=0x7f7fd80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:22.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7feb7fe640 1 --2- 172.21.15.29:0/1576858689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fec06c950 0x7f7fd80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7feb7fe640 1 -- 172.21.15.29:0/1576858689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd8003f40 msgr2=0x7f7fd8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:22.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7feb7fe640 1 --2- 172.21.15.29:0/1576858689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd8003f40 0x7f7fd8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.450+0000 7f7feb7fe640 1 -- 172.21.15.29:0/1576858689 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd8003530 con 0x7f7fd8004d50 2024-04-25T00:33:22.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.454+0000 7f7feb7fe640 1 --2- 172.21.15.29:0/1576858689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd8004d50 0x7f7fd80051d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7fe40013e0 tx=0x7f7fe401d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e5fec5fbe0d1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:22.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.454+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400a4e0 con 0x7f7fd8004d50 2024-04-25T00:33:22.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.454+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe40ad650 con 0x7f7fd8004d50 2024-04-25T00:33:22.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.454+0000 7f7fea7fc640 1 -- 172.21.15.29:0/1576858689 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe40ada40 con 0x7f7fd8004d50 2024-04-25T00:33:22.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.454+0000 7f7ff13ad640 1 -- 172.21.15.29:0/1576858689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd8004d50 msgr2=0x7f7fd80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:22.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.454+0000 7f7ff13ad640 1 --2- 172.21.15.29:0/1576858689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd8004d50 0x7f7fd80051d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7fe40013e0 tx=0x7f7fe401d320 comp rx=0 tx=0).stop 2024-04-25T00:33:22.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.454+0000 7f7feb7fe640 1 -- 172.21.15.29:0/1576858689 reap_dead start 2024-04-25T00:33:22.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.454+0000 7f7ff13ad640 1 -- 172.21.15.29:0/1576858689 shutdown_connections 2024-04-25T00:33:22.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.454+0000 7f7ff13ad640 1 -- 172.21.15.29:0/1576858689 >> 172.21.15.29:0/1576858689 conn(0x7f7fec0d1d60 msgr2=0x7f7fec0d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:22.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.454+0000 7f7ff13ad640 1 -- 172.21.15.29:0/1576858689 shutdown_connections 2024-04-25T00:33:22.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.454+0000 7f7ff13ad640 1 -- 172.21.15.29:0/1576858689 wait complete. 2024-04-25T00:33:22.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.454+0000 7f7ff13ad640 1 Processor -- start 2024-04-25T00:33:22.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.454+0000 7f7ff13ad640 1 -- start start 2024-04-25T00:33:22.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7ff13ad640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fd8004d50 0x7f7fec149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7ff13ad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fec14a0a0 0x7f7fec0cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7ff13ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fec0cbd90 0x7f7fec0c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7ff13ad640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec0ccef0 con 0x7f7fec14a0a0 2024-04-25T00:33:22.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7ff13ad640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec0ccc10 con 0x7f7fec0cbd90 2024-04-25T00:33:22.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7ff13ad640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7fec0ce420 con 0x7f7fd8004d50 2024-04-25T00:33:22.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fec14a0a0 0x7f7fec0cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fec14a0a0 0x7f7fec0cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43422/0 (socket says 172.21.15.29:43422) 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe3fff640 1 -- 172.21.15.29:0/2677207850 learned_addr learned my addr 172.21.15.29:0/2677207850 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7feaffd640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fd8004d50 0x7f7fec149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7feb7fe640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fec0cbd90 0x7f7fec0c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe3fff640 1 -- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fd8004d50 msgr2=0x7f7fec149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe3fff640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fd8004d50 0x7f7fec149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe3fff640 1 -- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fec0cbd90 msgr2=0x7f7fec0c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe3fff640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fec0cbd90 0x7f7fec0c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe3fff640 1 -- 172.21.15.29:0/2677207850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec149dd0 con 0x7f7fec14a0a0 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7feb7fe640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fec0cbd90 0x7f7fec0c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe3fff640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fec14a0a0 0x7f7fec0cb850 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4011590 tx=0x7f7fd4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe37fe640 1 -- 172.21.15.29:0/2677207850 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4004220 con 0x7f7fec14a0a0 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe37fe640 1 -- 172.21.15.29:0/2677207850 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd40043c0 con 0x7f7fec14a0a0 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7ff13ad640 1 -- 172.21.15.29:0/2677207850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec0cc390 con 0x7f7fec14a0a0 2024-04-25T00:33:22.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe37fe640 1 -- 172.21.15.29:0/2677207850 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4015890 con 0x7f7fec14a0a0 2024-04-25T00:33:22.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7ff13ad640 1 -- 172.21.15.29:0/2677207850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec1e6c40 con 0x7f7fec14a0a0 2024-04-25T00:33:22.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7ff13ad640 1 -- 172.21.15.29:0/2677207850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fec069270 con 0x7f7fec14a0a0 2024-04-25T00:33:22.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe37fe640 1 -- 172.21.15.29:0/2677207850 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f7fd4015a30 con 0x7f7fec14a0a0 2024-04-25T00:33:22.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe37fe640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7fcc067a70 0x7f7fcc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7fe37fe640 1 -- 172.21.15.29:0/2677207850 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7fd40861f0 con 0x7f7fec14a0a0 2024-04-25T00:33:22.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.458+0000 7f7feaffd640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7fcc067a70 0x7f7fcc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.462+0000 7f7feaffd640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7fcc067a70 0x7f7fcc069f30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009a30 tx=0x7f7fdc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:22.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.462+0000 7f7fe37fe640 1 -- 172.21.15.29:0/2677207850 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7fd4019030 con 0x7f7fec14a0a0 2024-04-25T00:33:22.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.622+0000 7f7ff13ad640 1 -- 172.21.15.29:0/2677207850 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7fec06ffa0 con 0x7f7fcc067a70 2024-04-25T00:33:22.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.646+0000 7f7fe37fe640 1 -- 172.21.15.29:0/2677207850 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251967 (secure 0 0 0) 0x7f7fec06ffa0 con 0x7f7fcc067a70 2024-04-25T00:33:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:22.650 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:33:22.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.646+0000 7f7ff13ad640 1 -- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7fcc067a70 msgr2=0x7f7fcc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:22.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.646+0000 7f7ff13ad640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7fcc067a70 0x7f7fcc069f30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009a30 tx=0x7f7fdc008040 comp rx=0 tx=0).stop 2024-04-25T00:33:22.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.646+0000 7f7ff13ad640 1 -- 172.21.15.29:0/2677207850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fec14a0a0 msgr2=0x7f7fec0cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:22.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.646+0000 7f7ff13ad640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fec14a0a0 0x7f7fec0cb850 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4011590 tx=0x7f7fd4011a50 comp rx=0 tx=0).stop 2024-04-25T00:33:22.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.646+0000 7f7ff13ad640 1 -- 172.21.15.29:0/2677207850 shutdown_connections 2024-04-25T00:33:22.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.646+0000 7f7ff13ad640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7fcc067a70 0x7f7fcc069f30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.646+0000 7f7ff13ad640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fec0cbd90 0x7f7fec0c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.646+0000 7f7ff13ad640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fec14a0a0 0x7f7fec0cb850 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.646+0000 7f7ff13ad640 1 --2- 172.21.15.29:0/2677207850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fd8004d50 0x7f7fec149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.646+0000 7f7ff13ad640 1 -- 172.21.15.29:0/2677207850 >> 172.21.15.29:0/2677207850 conn(0x7f7fec0d1d60 msgr2=0x7f7fec15d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:22.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.646+0000 7f7ff13ad640 1 -- 172.21.15.29:0/2677207850 shutdown_connections 2024-04-25T00:33:22.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.646+0000 7f7ff13ad640 1 -- 172.21.15.29:0/2677207850 wait complete. 2024-04-25T00:33:22.669 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-04-25T00:33:21.482587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263136,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754711584,"statfs":{"total":773094113280,"available":772824662016,"internally_reserved":0,"allocated":5480448,"data_stored":2909260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":60,"apply_latency_ms":60,"commit_latency_ns":60000000,"apply_latency_ns":60000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002851"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.457178+0000","last_change":"2024-04-25T00:33:13.789541+0000","last_active":"2024-04-25T00:33:15.457178+0000","last_peered":"2024-04-25T00:33:15.457178+0000","last_clean":"2024-04-25T00:33:15.457178+0000","last_became_active":"2024-04-25T00:33:13.788777+0000","last_became_peered":"2024-04-25T00:33:13.788777+0000","last_unstale":"2024-04-25T00:33:15.457178+0000","last_undegraded":"2024-04-25T00:33:15.457178+0000","last_fullsized":"2024-04-25T00:33:15.457178+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:29.079459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.968529+0000","last_change":"2024-04-25T00:33:13.819122+0000","last_active":"2024-04-25T00:33:14.968529+0000","last_peered":"2024-04-25T00:33:14.968529+0000","last_clean":"2024-04-25T00:33:14.968529+0000","last_became_active":"2024-04-25T00:33:13.818392+0000","last_became_peered":"2024-04-25T00:33:13.818392+0000","last_unstale":"2024-04-25T00:33:14.968529+0000","last_undegraded":"2024-04-25T00:33:14.968529+0000","last_fullsized":"2024-04-25T00:33:14.968529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:48.744582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.803961+0000","last_change":"2024-04-25T00:33:13.808909+0000","last_active":"2024-04-25T00:33:15.803961+0000","last_peered":"2024-04-25T00:33:15.803961+0000","last_clean":"2024-04-25T00:33:15.803961+0000","last_became_active":"2024-04-25T00:33:13.807527+0000","last_became_peered":"2024-04-25T00:33:13.807527+0000","last_unstale":"2024-04-25T00:33:15.803961+0000","last_undegraded":"2024-04-25T00:33:15.803961+0000","last_fullsized":"2024-04-25T00:33:15.803961+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:20.197907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792045+0000","last_change":"2024-04-25T00:33:13.804020+0000","last_active":"2024-04-25T00:33:14.792045+0000","last_peered":"2024-04-25T00:33:14.792045+0000","last_clean":"2024-04-25T00:33:14.792045+0000","last_became_active":"2024-04-25T00:33:13.803699+0000","last_became_peered":"2024-04-25T00:33:13.803699+0000","last_unstale":"2024-04-25T00:33:14.792045+0000","last_undegraded":"2024-04-25T00:33:14.792045+0000","last_fullsized":"2024-04-25T00:33:14.792045+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:32.468393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.968090+0000","last_change":"2024-04-25T00:33:13.826987+0000","last_active":"2024-04-25T00:33:14.968090+0000","last_peered":"2024-04-25T00:33:14.968090+0000","last_clean":"2024-04-25T00:33:14.968090+0000","last_became_active":"2024-04-25T00:33:13.818897+0000","last_became_peered":"2024-04-25T00:33:13.818897+0000","last_unstale":"2024-04-25T00:33:14.968090+0000","last_undegraded":"2024-04-25T00:33:14.968090+0000","last_fullsized":"2024-04-25T00:33:14.968090+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:21.572087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.798896+0000","last_change":"2024-04-25T00:33:13.821986+0000","last_active":"2024-04-25T00:33:15.798896+0000","last_peered":"2024-04-25T00:33:15.798896+0000","last_clean":"2024-04-25T00:33:15.798896+0000","last_became_active":"2024-04-25T00:33:13.821654+0000","last_became_peered":"2024-04-25T00:33:13.821654+0000","last_unstale":"2024-04-25T00:33:15.798896+0000","last_undegraded":"2024-04-25T00:33:15.798896+0000","last_fullsized":"2024-04-25T00:33:15.798896+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.582395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.456662+0000","last_change":"2024-04-25T00:33:13.802217+0000","last_active":"2024-04-25T00:33:15.456662+0000","last_peered":"2024-04-25T00:33:15.456662+0000","last_clean":"2024-04-25T00:33:15.456662+0000","last_became_active":"2024-04-25T00:33:13.801526+0000","last_became_peered":"2024-04-25T00:33:13.801526+0000","last_unstale":"2024-04-25T00:33:15.456662+0000","last_undegraded":"2024-04-25T00:33:15.456662+0000","last_fullsized":"2024-04-25T00:33:15.456662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:26.844815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.791211+0000","last_change":"2024-04-25T00:33:13.813088+0000","last_active":"2024-04-25T00:33:14.791211+0000","last_peered":"2024-04-25T00:33:14.791211+0000","last_clean":"2024-04-25T00:33:14.791211+0000","last_became_active":"2024-04-25T00:33:13.812317+0000","last_became_peered":"2024-04-25T00:33:13.812317+0000","last_unstale":"2024-04-25T00:33:14.791211+0000","last_undegraded":"2024-04-25T00:33:14.791211+0000","last_fullsized":"2024-04-25T00:33:14.791211+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:37.533819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.793509+0000","last_change":"2024-04-25T00:33:13.806744+0000","last_active":"2024-04-25T00:33:14.793509+0000","last_peered":"2024-04-25T00:33:14.793509+0000","last_clean":"2024-04-25T00:33:14.793509+0000","last_became_active":"2024-04-25T00:33:13.805147+0000","last_became_peered":"2024-04-25T00:33:13.805147+0000","last_unstale":"2024-04-25T00:33:14.793509+0000","last_undegraded":"2024-04-25T00:33:14.793509+0000","last_fullsized":"2024-04-25T00:33:14.793509+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:18.290080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.793170+0000","last_change":"2024-04-25T00:33:13.808389+0000","last_active":"2024-04-25T00:33:14.793170+0000","last_peered":"2024-04-25T00:33:14.793170+0000","last_clean":"2024-04-25T00:33:14.793170+0000","last_became_active":"2024-04-25T00:33:13.807895+0000","last_became_peered":"2024-04-25T00:33:13.807895+0000","last_unstale":"2024-04-25T00:33:14.793170+0000","last_undegraded":"2024-04-25T00:33:14.793170+0000","last_fullsized":"2024-04-25T00:33:14.793170+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:25.756449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.791772+0000","last_change":"2024-04-25T00:33:13.802882+0000","last_active":"2024-04-25T00:33:14.791772+0000","last_peered":"2024-04-25T00:33:14.791772+0000","last_clean":"2024-04-25T00:33:14.791772+0000","last_became_active":"2024-04-25T00:33:13.802543+0000","last_became_peered":"2024-04-25T00:33:13.802543+0000","last_unstale":"2024-04-25T00:33:14.791772+0000","last_undegraded":"2024-04-25T00:33:14.791772+0000","last_fullsized":"2024-04-25T00:33:14.791772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:42.053412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.968401+0000","last_change":"2024-04-25T00:33:13.830983+0000","last_active":"2024-04-25T00:33:14.968401+0000","last_peered":"2024-04-25T00:33:14.968401+0000","last_clean":"2024-04-25T00:33:14.968401+0000","last_became_active":"2024-04-25T00:33:13.827416+0000","last_became_peered":"2024-04-25T00:33:13.827416+0000","last_unstale":"2024-04-25T00:33:14.968401+0000","last_undegraded":"2024-04-25T00:33:14.968401+0000","last_fullsized":"2024-04-25T00:33:14.968401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:46.980802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.793091+0000","last_change":"2024-04-25T00:33:13.815381+0000","last_active":"2024-04-25T00:33:14.793091+0000","last_peered":"2024-04-25T00:33:14.793091+0000","last_clean":"2024-04-25T00:33:14.793091+0000","last_became_active":"2024-04-25T00:33:13.814885+0000","last_became_peered":"2024-04-25T00:33:13.814885+0000","last_unstale":"2024-04-25T00:33:14.793091+0000","last_undegraded":"2024-04-25T00:33:14.793091+0000","last_fullsized":"2024-04-25T00:33:14.793091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:47.936419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792477+0000","last_change":"2024-04-25T00:33:13.799128+0000","last_active":"2024-04-25T00:33:14.792477+0000","last_peered":"2024-04-25T00:33:14.792477+0000","last_clean":"2024-04-25T00:33:14.792477+0000","last_became_active":"2024-04-25T00:33:13.798644+0000","last_became_peered":"2024-04-25T00:33:13.798644+0000","last_unstale":"2024-04-25T00:33:14.792477+0000","last_undegraded":"2024-04-25T00:33:14.792477+0000","last_fullsized":"2024-04-25T00:33:14.792477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:05.749195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.798718+0000","last_change":"2024-04-25T00:33:13.812237+0000","last_active":"2024-04-25T00:33:15.798718+0000","last_peered":"2024-04-25T00:33:15.798718+0000","last_clean":"2024-04-25T00:33:15.798718+0000","last_became_active":"2024-04-25T00:33:13.811670+0000","last_became_peered":"2024-04-25T00:33:13.811670+0000","last_unstale":"2024-04-25T00:33:15.798718+0000","last_undegraded":"2024-04-25T00:33:15.798718+0000","last_fullsized":"2024-04-25T00:33:15.798718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:53.025254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792286+0000","last_change":"2024-04-25T00:33:13.806693+0000","last_active":"2024-04-25T00:33:14.792286+0000","last_peered":"2024-04-25T00:33:14.792286+0000","last_clean":"2024-04-25T00:33:14.792286+0000","last_became_active":"2024-04-25T00:33:13.804709+0000","last_became_peered":"2024-04-25T00:33:13.804709+0000","last_unstale":"2024-04-25T00:33:14.792286+0000","last_undegraded":"2024-04-25T00:33:14.792286+0000","last_fullsized":"2024-04-25T00:33:14.792286+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:43.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.804595+0000","last_change":"2024-04-25T00:33:13.808800+0000","last_active":"2024-04-25T00:33:15.804595+0000","last_peered":"2024-04-25T00:33:15.804595+0000","last_clean":"2024-04-25T00:33:15.804595+0000","last_became_active":"2024-04-25T00:33:13.807258+0000","last_became_peered":"2024-04-25T00:33:13.807258+0000","last_unstale":"2024-04-25T00:33:15.804595+0000","last_undegraded":"2024-04-25T00:33:15.804595+0000","last_fullsized":"2024-04-25T00:33:15.804595+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:05.749195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.804314+0000","last_change":"2024-04-25T00:33:13.798525+0000","last_active":"2024-04-25T00:33:15.804314+0000","last_peered":"2024-04-25T00:33:15.804314+0000","last_clean":"2024-04-25T00:33:15.804314+0000","last_became_active":"2024-04-25T00:33:13.796009+0000","last_became_peered":"2024-04-25T00:33:13.796009+0000","last_unstale":"2024-04-25T00:33:15.804314+0000","last_undegraded":"2024-04-25T00:33:15.804314+0000","last_fullsized":"2024-04-25T00:33:15.804314+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:32.468393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.793352+0000","last_change":"2024-04-25T00:33:13.803537+0000","last_active":"2024-04-25T00:33:14.793352+0000","last_peered":"2024-04-25T00:33:14.793352+0000","last_clean":"2024-04-25T00:33:14.793352+0000","last_became_active":"2024-04-25T00:33:13.802789+0000","last_became_peered":"2024-04-25T00:33:13.802789+0000","last_unstale":"2024-04-25T00:33:14.793352+0000","last_undegraded":"2024-04-25T00:33:14.793352+0000","last_fullsized":"2024-04-25T00:33:14.793352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:32.157596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792654+0000","last_change":"2024-04-25T00:33:13.795258+0000","last_active":"2024-04-25T00:33:14.792654+0000","last_peered":"2024-04-25T00:33:14.792654+0000","last_clean":"2024-04-25T00:33:14.792654+0000","last_became_active":"2024-04-25T00:33:13.794954+0000","last_became_peered":"2024-04-25T00:33:13.794954+0000","last_unstale":"2024-04-25T00:33:14.792654+0000","last_undegraded":"2024-04-25T00:33:14.792654+0000","last_fullsized":"2024-04-25T00:33:14.792654+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:39.928341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.799715+0000","last_change":"2024-04-25T00:33:13.813736+0000","last_active":"2024-04-25T00:33:15.799715+0000","last_peered":"2024-04-25T00:33:15.799715+0000","last_clean":"2024-04-25T00:33:15.799715+0000","last_became_active":"2024-04-25T00:33:13.812217+0000","last_became_peered":"2024-04-25T00:33:13.812217+0000","last_unstale":"2024-04-25T00:33:15.799715+0000","last_undegraded":"2024-04-25T00:33:15.799715+0000","last_fullsized":"2024-04-25T00:33:15.799715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:53.621737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792793+0000","last_change":"2024-04-25T00:33:13.811528+0000","last_active":"2024-04-25T00:33:14.792793+0000","last_peered":"2024-04-25T00:33:14.792793+0000","last_clean":"2024-04-25T00:33:14.792793+0000","last_became_active":"2024-04-25T00:33:13.811034+0000","last_became_peered":"2024-04-25T00:33:13.811034+0000","last_unstale":"2024-04-25T00:33:14.792793+0000","last_undegraded":"2024-04-25T00:33:14.792793+0000","last_fullsized":"2024-04-25T00:33:14.792793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:38.766754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792553+0000","last_change":"2024-04-25T00:33:13.825421+0000","last_active":"2024-04-25T00:33:14.792553+0000","last_peered":"2024-04-25T00:33:14.792553+0000","last_clean":"2024-04-25T00:33:14.792553+0000","last_became_active":"2024-04-25T00:33:13.823305+0000","last_became_peered":"2024-04-25T00:33:13.823305+0000","last_unstale":"2024-04-25T00:33:14.792553+0000","last_undegraded":"2024-04-25T00:33:14.792553+0000","last_fullsized":"2024-04-25T00:33:14.792553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:31.944166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792284+0000","last_change":"2024-04-25T00:33:13.825937+0000","last_active":"2024-04-25T00:33:14.792284+0000","last_peered":"2024-04-25T00:33:14.792284+0000","last_clean":"2024-04-25T00:33:14.792284+0000","last_became_active":"2024-04-25T00:33:13.823496+0000","last_became_peered":"2024-04-25T00:33:13.823496+0000","last_unstale":"2024-04-25T00:33:14.792284+0000","last_undegraded":"2024-04-25T00:33:14.792284+0000","last_fullsized":"2024-04-25T00:33:14.792284+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:12.446867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.793095+0000","last_change":"2024-04-25T00:33:13.809480+0000","last_active":"2024-04-25T00:33:14.793095+0000","last_peered":"2024-04-25T00:33:14.793095+0000","last_clean":"2024-04-25T00:33:14.793095+0000","last_became_active":"2024-04-25T00:33:13.808766+0000","last_became_peered":"2024-04-25T00:33:13.808766+0000","last_unstale":"2024-04-25T00:33:14.793095+0000","last_undegraded":"2024-04-25T00:33:14.793095+0000","last_fullsized":"2024-04-25T00:33:14.793095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:06.620664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792590+0000","last_change":"2024-04-25T00:33:13.813975+0000","last_active":"2024-04-25T00:33:14.792590+0000","last_peered":"2024-04-25T00:33:14.792590+0000","last_clean":"2024-04-25T00:33:14.792590+0000","last_became_active":"2024-04-25T00:33:13.813707+0000","last_became_peered":"2024-04-25T00:33:13.813707+0000","last_unstale":"2024-04-25T00:33:14.792590+0000","last_undegraded":"2024-04-25T00:33:14.792590+0000","last_fullsized":"2024-04-25T00:33:14.792590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:12.480749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.804081+0000","last_change":"2024-04-25T00:33:13.808004+0000","last_active":"2024-04-25T00:33:15.804081+0000","last_peered":"2024-04-25T00:33:15.804081+0000","last_clean":"2024-04-25T00:33:15.804081+0000","last_became_active":"2024-04-25T00:33:13.806545+0000","last_became_peered":"2024-04-25T00:33:13.806545+0000","last_unstale":"2024-04-25T00:33:15.804081+0000","last_undegraded":"2024-04-25T00:33:15.804081+0000","last_fullsized":"2024-04-25T00:33:15.804081+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:42.053412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.968335+0000","last_change":"2024-04-25T00:33:13.830536+0000","last_active":"2024-04-25T00:33:14.968335+0000","last_peered":"2024-04-25T00:33:14.968335+0000","last_clean":"2024-04-25T00:33:14.968335+0000","last_became_active":"2024-04-25T00:33:13.826582+0000","last_became_peered":"2024-04-25T00:33:13.826582+0000","last_unstale":"2024-04-25T00:33:14.968335+0000","last_undegraded":"2024-04-25T00:33:14.968335+0000","last_fullsized":"2024-04-25T00:33:14.968335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:59.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792860+0000","last_change":"2024-04-25T00:33:13.809556+0000","last_active":"2024-04-25T00:33:14.792860+0000","last_peered":"2024-04-25T00:33:14.792860+0000","last_clean":"2024-04-25T00:33:14.792860+0000","last_became_active":"2024-04-25T00:33:13.808799+0000","last_became_peered":"2024-04-25T00:33:13.808799+0000","last_unstale":"2024-04-25T00:33:14.792860+0000","last_undegraded":"2024-04-25T00:33:14.792860+0000","last_fullsized":"2024-04-25T00:33:14.792860+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:50.571121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.803729+0000","last_change":"2024-04-25T00:33:13.785791+0000","last_active":"2024-04-25T00:33:15.803729+0000","last_peered":"2024-04-25T00:33:15.803729+0000","last_clean":"2024-04-25T00:33:15.803729+0000","last_became_active":"2024-04-25T00:33:13.784233+0000","last_became_peered":"2024-04-25T00:33:13.784233+0000","last_unstale":"2024-04-25T00:33:15.803729+0000","last_undegraded":"2024-04-25T00:33:15.803729+0000","last_fullsized":"2024-04-25T00:33:15.803729+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:37.533819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.798782+0000","last_change":"2024-04-25T00:33:13.827220+0000","last_active":"2024-04-25T00:33:15.798782+0000","last_peered":"2024-04-25T00:33:15.798782+0000","last_clean":"2024-04-25T00:33:15.798782+0000","last_became_active":"2024-04-25T00:33:13.826887+0000","last_became_peered":"2024-04-25T00:33:13.826887+0000","last_unstale":"2024-04-25T00:33:15.798782+0000","last_undegraded":"2024-04-25T00:33:15.798782+0000","last_fullsized":"2024-04-25T00:33:15.798782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:22.944277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.790724+0000","last_change":"2024-04-25T00:33:13.808472+0000","last_active":"2024-04-25T00:33:14.790724+0000","last_peered":"2024-04-25T00:33:14.790724+0000","last_clean":"2024-04-25T00:33:14.790724+0000","last_became_active":"2024-04-25T00:33:13.807854+0000","last_became_peered":"2024-04-25T00:33:13.807854+0000","last_unstale":"2024-04-25T00:33:14.790724+0000","last_undegraded":"2024-04-25T00:33:14.790724+0000","last_fullsized":"2024-04-25T00:33:14.790724+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:43.657349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.832724+0000","last_change":"2024-04-25T00:33:13.819007+0000","last_active":"2024-04-25T00:33:15.832724+0000","last_peered":"2024-04-25T00:33:15.832724+0000","last_clean":"2024-04-25T00:33:15.832724+0000","last_became_active":"2024-04-25T00:33:13.818556+0000","last_became_peered":"2024-04-25T00:33:13.818556+0000","last_unstale":"2024-04-25T00:33:15.832724+0000","last_undegraded":"2024-04-25T00:33:15.832724+0000","last_fullsized":"2024-04-25T00:33:15.832724+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:34.712473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.799916+0000","last_change":"2024-04-25T00:33:12.801022+0000","last_active":"2024-04-25T00:33:15.799916+0000","last_peered":"2024-04-25T00:33:15.799916+0000","last_clean":"2024-04-25T00:33:15.799916+0000","last_became_active":"2024-04-25T00:33:12.800641+0000","last_became_peered":"2024-04-25T00:33:12.800641+0000","last_unstale":"2024-04-25T00:33:15.799916+0000","last_undegraded":"2024-04-25T00:33:15.799916+0000","last_fullsized":"2024-04-25T00:33:15.799916+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:58.103145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.793058+0000","last_change":"2024-04-25T00:33:13.806914+0000","last_active":"2024-04-25T00:33:14.793058+0000","last_peered":"2024-04-25T00:33:14.793058+0000","last_clean":"2024-04-25T00:33:14.793058+0000","last_became_active":"2024-04-25T00:33:13.805545+0000","last_became_peered":"2024-04-25T00:33:13.805545+0000","last_unstale":"2024-04-25T00:33:14.793058+0000","last_undegraded":"2024-04-25T00:33:14.793058+0000","last_fullsized":"2024-04-25T00:33:14.793058+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:34.712473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.804679+0000","last_change":"2024-04-25T00:33:12.797818+0000","last_active":"2024-04-25T00:33:15.804679+0000","last_peered":"2024-04-25T00:33:15.804679+0000","last_clean":"2024-04-25T00:33:15.804679+0000","last_became_active":"2024-04-25T00:33:12.796615+0000","last_became_peered":"2024-04-25T00:33:12.796615+0000","last_unstale":"2024-04-25T00:33:15.804679+0000","last_undegraded":"2024-04-25T00:33:15.804679+0000","last_fullsized":"2024-04-25T00:33:15.804679+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:11.438040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.968689+0000","last_change":"2024-04-25T00:33:13.832212+0000","last_active":"2024-04-25T00:33:14.968689+0000","last_peered":"2024-04-25T00:33:14.968689+0000","last_clean":"2024-04-25T00:33:14.968689+0000","last_became_active":"2024-04-25T00:33:13.831653+0000","last_became_peered":"2024-04-25T00:33:13.831653+0000","last_unstale":"2024-04-25T00:33:14.968689+0000","last_undegraded":"2024-04-25T00:33:14.968689+0000","last_fullsized":"2024-04-25T00:33:14.968689+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:42.249837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.832970+0000","last_change":"2024-04-25T00:33:12.797893+0000","last_active":"2024-04-25T00:33:15.832970+0000","last_peered":"2024-04-25T00:33:15.832970+0000","last_clean":"2024-04-25T00:33:15.832970+0000","last_became_active":"2024-04-25T00:33:12.797499+0000","last_became_peered":"2024-04-25T00:33:12.797499+0000","last_unstale":"2024-04-25T00:33:15.832970+0000","last_undegraded":"2024-04-25T00:33:15.832970+0000","last_fullsized":"2024-04-25T00:33:15.832970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:11.438040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.833114+0000","last_change":"2024-04-25T00:33:13.807724+0000","last_active":"2024-04-25T00:33:15.833114+0000","last_peered":"2024-04-25T00:33:15.833114+0000","last_clean":"2024-04-25T00:33:15.833114+0000","last_became_active":"2024-04-25T00:33:13.807441+0000","last_became_peered":"2024-04-25T00:33:13.807441+0000","last_unstale":"2024-04-25T00:33:15.833114+0000","last_undegraded":"2024-04-25T00:33:15.833114+0000","last_fullsized":"2024-04-25T00:33:15.833114+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:06.620664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.456851+0000","last_change":"2024-04-25T00:33:12.808013+0000","last_active":"2024-04-25T00:33:15.456851+0000","last_peered":"2024-04-25T00:33:15.456851+0000","last_clean":"2024-04-25T00:33:15.456851+0000","last_became_active":"2024-04-25T00:33:12.807497+0000","last_became_peered":"2024-04-25T00:33:12.807497+0000","last_unstale":"2024-04-25T00:33:15.456851+0000","last_undegraded":"2024-04-25T00:33:15.456851+0000","last_fullsized":"2024-04-25T00:33:15.456851+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:17.281253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.803755+0000","last_change":"2024-04-25T00:33:13.811125+0000","last_active":"2024-04-25T00:33:15.803755+0000","last_peered":"2024-04-25T00:33:15.803755+0000","last_clean":"2024-04-25T00:33:15.803755+0000","last_became_active":"2024-04-25T00:33:13.810205+0000","last_became_peered":"2024-04-25T00:33:13.810205+0000","last_unstale":"2024-04-25T00:33:15.803755+0000","last_undegraded":"2024-04-25T00:33:15.803755+0000","last_fullsized":"2024-04-25T00:33:15.803755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:20.770452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.799496+0000","last_change":"2024-04-25T00:33:12.801886+0000","last_active":"2024-04-25T00:33:15.799496+0000","last_peered":"2024-04-25T00:33:15.799496+0000","last_clean":"2024-04-25T00:33:15.799496+0000","last_became_active":"2024-04-25T00:33:12.801570+0000","last_became_peered":"2024-04-25T00:33:12.801570+0000","last_unstale":"2024-04-25T00:33:15.799496+0000","last_undegraded":"2024-04-25T00:33:15.799496+0000","last_fullsized":"2024-04-25T00:33:15.799496+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:17.281253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792361+0000","last_change":"2024-04-25T00:33:13.798328+0000","last_active":"2024-04-25T00:33:14.792361+0000","last_peered":"2024-04-25T00:33:14.792361+0000","last_clean":"2024-04-25T00:33:14.792361+0000","last_became_active":"2024-04-25T00:33:13.797902+0000","last_became_peered":"2024-04-25T00:33:13.797902+0000","last_unstale":"2024-04-25T00:33:14.792361+0000","last_undegraded":"2024-04-25T00:33:14.792361+0000","last_fullsized":"2024-04-25T00:33:14.792361+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:21.572087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.804306+0000","last_change":"2024-04-25T00:33:12.795400+0000","last_active":"2024-04-25T00:33:15.804306+0000","last_peered":"2024-04-25T00:33:15.804306+0000","last_clean":"2024-04-25T00:33:15.804306+0000","last_became_active":"2024-04-25T00:33:12.794892+0000","last_became_peered":"2024-04-25T00:33:12.794892+0000","last_unstale":"2024-04-25T00:33:15.804306+0000","last_undegraded":"2024-04-25T00:33:15.804306+0000","last_fullsized":"2024-04-25T00:33:15.804306+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:54.884884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.791372+0000","last_change":"2024-04-25T00:33:13.810905+0000","last_active":"2024-04-25T00:33:14.791372+0000","last_peered":"2024-04-25T00:33:14.791372+0000","last_clean":"2024-04-25T00:33:14.791372+0000","last_became_active":"2024-04-25T00:33:13.810549+0000","last_became_peered":"2024-04-25T00:33:13.810549+0000","last_unstale":"2024-04-25T00:33:14.791372+0000","last_undegraded":"2024-04-25T00:33:14.791372+0000","last_fullsized":"2024-04-25T00:33:14.791372+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:20.197907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.967672+0000","last_change":"2024-04-25T00:33:12.794993+0000","last_active":"2024-04-25T00:33:14.967672+0000","last_peered":"2024-04-25T00:33:14.967672+0000","last_clean":"2024-04-25T00:33:14.967672+0000","last_became_active":"2024-04-25T00:33:12.794251+0000","last_became_peered":"2024-04-25T00:33:12.794251+0000","last_unstale":"2024-04-25T00:33:14.967672+0000","last_undegraded":"2024-04-25T00:33:14.967672+0000","last_fullsized":"2024-04-25T00:33:14.967672+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.189080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792142+0000","last_change":"2024-04-25T00:33:13.825969+0000","last_active":"2024-04-25T00:33:14.792142+0000","last_peered":"2024-04-25T00:33:14.792142+0000","last_clean":"2024-04-25T00:33:14.792142+0000","last_became_active":"2024-04-25T00:33:13.825397+0000","last_became_peered":"2024-04-25T00:33:13.825397+0000","last_unstale":"2024-04-25T00:33:14.792142+0000","last_undegraded":"2024-04-25T00:33:14.792142+0000","last_fullsized":"2024-04-25T00:33:14.792142+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:31.667760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.791467+0000","last_change":"2024-04-25T00:33:12.793905+0000","last_active":"2024-04-25T00:33:14.791467+0000","last_peered":"2024-04-25T00:33:14.791467+0000","last_clean":"2024-04-25T00:33:14.791467+0000","last_became_active":"2024-04-25T00:33:12.791067+0000","last_became_peered":"2024-04-25T00:33:12.791067+0000","last_unstale":"2024-04-25T00:33:14.791467+0000","last_undegraded":"2024-04-25T00:33:14.791467+0000","last_fullsized":"2024-04-25T00:33:14.791467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:42.648522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.804339+0000","last_change":"2024-04-25T00:33:13.797603+0000","last_active":"2024-04-25T00:33:15.804339+0000","last_peered":"2024-04-25T00:33:15.804339+0000","last_clean":"2024-04-25T00:33:15.804339+0000","last_became_active":"2024-04-25T00:33:13.794895+0000","last_became_peered":"2024-04-25T00:33:13.794895+0000","last_unstale":"2024-04-25T00:33:15.804339+0000","last_undegraded":"2024-04-25T00:33:15.804339+0000","last_fullsized":"2024-04-25T00:33:15.804339+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:11.083727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.804667+0000","last_change":"2024-04-25T00:33:12.791338+0000","last_active":"2024-04-25T00:33:15.804667+0000","last_peered":"2024-04-25T00:33:15.804667+0000","last_clean":"2024-04-25T00:33:15.804667+0000","last_became_active":"2024-04-25T00:33:12.790547+0000","last_became_peered":"2024-04-25T00:33:12.790547+0000","last_unstale":"2024-04-25T00:33:15.804667+0000","last_undegraded":"2024-04-25T00:33:15.804667+0000","last_fullsized":"2024-04-25T00:33:15.804667+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:46.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.457220+0000","last_change":"2024-04-25T00:33:13.815389+0000","last_active":"2024-04-25T00:33:15.457220+0000","last_peered":"2024-04-25T00:33:15.457220+0000","last_clean":"2024-04-25T00:33:15.457220+0000","last_became_active":"2024-04-25T00:33:13.814699+0000","last_became_peered":"2024-04-25T00:33:13.814699+0000","last_unstale":"2024-04-25T00:33:15.457220+0000","last_undegraded":"2024-04-25T00:33:15.457220+0000","last_fullsized":"2024-04-25T00:33:15.457220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:12.180645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.793265+0000","last_change":"2024-04-25T00:33:12.800591+0000","last_active":"2024-04-25T00:33:14.793265+0000","last_peered":"2024-04-25T00:33:14.793265+0000","last_clean":"2024-04-25T00:33:14.793265+0000","last_became_active":"2024-04-25T00:33:12.800301+0000","last_became_peered":"2024-04-25T00:33:12.800301+0000","last_unstale":"2024-04-25T00:33:14.793265+0000","last_undegraded":"2024-04-25T00:33:14.793265+0000","last_fullsized":"2024-04-25T00:33:14.793265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:54.884884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.799812+0000","last_change":"2024-04-25T00:33:13.802918+0000","last_active":"2024-04-25T00:33:15.799812+0000","last_peered":"2024-04-25T00:33:15.799812+0000","last_clean":"2024-04-25T00:33:15.799812+0000","last_became_active":"2024-04-25T00:33:13.802599+0000","last_became_peered":"2024-04-25T00:33:13.802599+0000","last_unstale":"2024-04-25T00:33:15.799812+0000","last_undegraded":"2024-04-25T00:33:15.799812+0000","last_fullsized":"2024-04-25T00:33:15.799812+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:00.256448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.457441+0000","last_change":"2024-04-25T00:33:12.810318+0000","last_active":"2024-04-25T00:33:15.457441+0000","last_peered":"2024-04-25T00:33:15.457441+0000","last_clean":"2024-04-25T00:33:15.457441+0000","last_became_active":"2024-04-25T00:33:12.809691+0000","last_became_peered":"2024-04-25T00:33:12.809691+0000","last_unstale":"2024-04-25T00:33:15.457441+0000","last_undegraded":"2024-04-25T00:33:15.457441+0000","last_fullsized":"2024-04-25T00:33:15.457441+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:20.563260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.457170+0000","last_change":"2024-04-25T00:33:13.803828+0000","last_active":"2024-04-25T00:33:15.457170+0000","last_peered":"2024-04-25T00:33:15.457170+0000","last_clean":"2024-04-25T00:33:15.457170+0000","last_became_active":"2024-04-25T00:33:13.802150+0000","last_became_peered":"2024-04-25T00:33:13.802150+0000","last_unstale":"2024-04-25T00:33:15.457170+0000","last_undegraded":"2024-04-25T00:33:15.457170+0000","last_fullsized":"2024-04-25T00:33:15.457170+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:31.068606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792770+0000","last_change":"2024-04-25T00:33:12.787615+0000","last_active":"2024-04-25T00:33:14.792770+0000","last_peered":"2024-04-25T00:33:14.792770+0000","last_clean":"2024-04-25T00:33:14.792770+0000","last_became_active":"2024-04-25T00:33:12.787329+0000","last_became_peered":"2024-04-25T00:33:12.787329+0000","last_unstale":"2024-04-25T00:33:14.792770+0000","last_undegraded":"2024-04-25T00:33:14.792770+0000","last_fullsized":"2024-04-25T00:33:14.792770+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:40.777355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792901+0000","last_change":"2024-04-25T00:33:13.812426+0000","last_active":"2024-04-25T00:33:14.792901+0000","last_peered":"2024-04-25T00:33:14.792901+0000","last_clean":"2024-04-25T00:33:14.792901+0000","last_became_active":"2024-04-25T00:33:13.811727+0000","last_became_peered":"2024-04-25T00:33:13.811727+0000","last_unstale":"2024-04-25T00:33:14.792901+0000","last_undegraded":"2024-04-25T00:33:14.792901+0000","last_fullsized":"2024-04-25T00:33:14.792901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:14.990491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.799890+0000","last_change":"2024-04-25T00:33:12.795864+0000","last_active":"2024-04-25T00:33:15.799890+0000","last_peered":"2024-04-25T00:33:15.799890+0000","last_clean":"2024-04-25T00:33:15.799890+0000","last_became_active":"2024-04-25T00:33:12.795591+0000","last_became_peered":"2024-04-25T00:33:12.795591+0000","last_unstale":"2024-04-25T00:33:15.799890+0000","last_undegraded":"2024-04-25T00:33:15.799890+0000","last_fullsized":"2024-04-25T00:33:15.799890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:45.971975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.794066+0000","last_change":"2024-04-25T00:33:13.786017+0000","last_active":"2024-04-25T00:33:14.794066+0000","last_peered":"2024-04-25T00:33:14.794066+0000","last_clean":"2024-04-25T00:33:14.794066+0000","last_became_active":"2024-04-25T00:33:13.785288+0000","last_became_peered":"2024-04-25T00:33:13.785288+0000","last_unstale":"2024-04-25T00:33:14.794066+0000","last_undegraded":"2024-04-25T00:33:14.794066+0000","last_fullsized":"2024-04-25T00:33:14.794066+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:59.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.456695+0000","last_change":"2024-04-25T00:33:12.806163+0000","last_active":"2024-04-25T00:33:15.456695+0000","last_peered":"2024-04-25T00:33:15.456695+0000","last_clean":"2024-04-25T00:33:15.456695+0000","last_became_active":"2024-04-25T00:33:12.802549+0000","last_became_peered":"2024-04-25T00:33:12.802549+0000","last_unstale":"2024-04-25T00:33:15.456695+0000","last_undegraded":"2024-04-25T00:33:15.456695+0000","last_fullsized":"2024-04-25T00:33:15.456695+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:49.661777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.798717+0000","last_change":"2024-04-25T00:33:13.803914+0000","last_active":"2024-04-25T00:33:15.798717+0000","last_peered":"2024-04-25T00:33:15.798717+0000","last_clean":"2024-04-25T00:33:15.798717+0000","last_became_active":"2024-04-25T00:33:13.803237+0000","last_became_peered":"2024-04-25T00:33:13.803237+0000","last_unstale":"2024-04-25T00:33:15.798717+0000","last_undegraded":"2024-04-25T00:33:15.798717+0000","last_fullsized":"2024-04-25T00:33:15.798717+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:12.180645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.799123+0000","last_change":"2024-04-25T00:33:12.797153+0000","last_active":"2024-04-25T00:33:15.799123+0000","last_peered":"2024-04-25T00:33:15.799123+0000","last_clean":"2024-04-25T00:33:15.799123+0000","last_became_active":"2024-04-25T00:33:12.796727+0000","last_became_peered":"2024-04-25T00:33:12.796727+0000","last_unstale":"2024-04-25T00:33:15.799123+0000","last_undegraded":"2024-04-25T00:33:15.799123+0000","last_fullsized":"2024-04-25T00:33:15.799123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:49.661777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.456829+0000","last_change":"2024-04-25T00:33:13.804721+0000","last_active":"2024-04-25T00:33:15.456829+0000","last_peered":"2024-04-25T00:33:15.456829+0000","last_clean":"2024-04-25T00:33:15.456829+0000","last_became_active":"2024-04-25T00:33:13.804050+0000","last_became_peered":"2024-04-25T00:33:13.804050+0000","last_unstale":"2024-04-25T00:33:15.456829+0000","last_undegraded":"2024-04-25T00:33:15.456829+0000","last_fullsized":"2024-04-25T00:33:15.456829+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:24.853710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":22,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.791565+0000","last_change":"2024-04-25T00:33:12.799231+0000","last_active":"2024-04-25T00:33:14.791565+0000","last_peered":"2024-04-25T00:33:14.791565+0000","last_clean":"2024-04-25T00:33:14.791565+0000","last_became_active":"2024-04-25T00:33:12.798864+0000","last_became_peered":"2024-04-25T00:33:12.798864+0000","last_unstale":"2024-04-25T00:33:14.791565+0000","last_undegraded":"2024-04-25T00:33:14.791565+0000","last_fullsized":"2024-04-25T00:33:14.791565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:52.568911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792993+0000","last_change":"2024-04-25T00:33:13.808157+0000","last_active":"2024-04-25T00:33:14.792993+0000","last_peered":"2024-04-25T00:33:14.792993+0000","last_clean":"2024-04-25T00:33:14.792993+0000","last_became_active":"2024-04-25T00:33:13.807845+0000","last_became_peered":"2024-04-25T00:33:13.807845+0000","last_unstale":"2024-04-25T00:33:14.792993+0000","last_undegraded":"2024-04-25T00:33:14.792993+0000","last_fullsized":"2024-04-25T00:33:14.792993+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:55.893711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.793680+0000","last_change":"2024-04-25T00:33:12.788985+0000","last_active":"2024-04-25T00:33:14.793680+0000","last_peered":"2024-04-25T00:33:14.793680+0000","last_clean":"2024-04-25T00:33:14.793680+0000","last_became_active":"2024-04-25T00:33:12.788443+0000","last_became_peered":"2024-04-25T00:33:12.788443+0000","last_unstale":"2024-04-25T00:33:14.793680+0000","last_undegraded":"2024-04-25T00:33:14.793680+0000","last_fullsized":"2024-04-25T00:33:14.793680+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:24.017506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.793658+0000","last_change":"2024-04-25T00:33:13.806015+0000","last_active":"2024-04-25T00:33:14.793658+0000","last_peered":"2024-04-25T00:33:14.793658+0000","last_clean":"2024-04-25T00:33:14.793658+0000","last_became_active":"2024-04-25T00:33:13.804206+0000","last_became_peered":"2024-04-25T00:33:13.804206+0000","last_unstale":"2024-04-25T00:33:14.793658+0000","last_undegraded":"2024-04-25T00:33:14.793658+0000","last_fullsized":"2024-04-25T00:33:14.793658+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:47.255920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.804696+0000","last_change":"2024-04-25T00:33:12.794976+0000","last_active":"2024-04-25T00:33:15.804696+0000","last_peered":"2024-04-25T00:33:15.804696+0000","last_clean":"2024-04-25T00:33:15.804696+0000","last_became_active":"2024-04-25T00:33:12.793226+0000","last_became_peered":"2024-04-25T00:33:12.793226+0000","last_unstale":"2024-04-25T00:33:15.804696+0000","last_undegraded":"2024-04-25T00:33:15.804696+0000","last_fullsized":"2024-04-25T00:33:15.804696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:42.839016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792843+0000","last_change":"2024-04-25T00:33:13.818618+0000","last_active":"2024-04-25T00:33:14.792843+0000","last_peered":"2024-04-25T00:33:14.792843+0000","last_clean":"2024-04-25T00:33:14.792843+0000","last_became_active":"2024-04-25T00:33:13.812587+0000","last_became_peered":"2024-04-25T00:33:13.812587+0000","last_unstale":"2024-04-25T00:33:14.792843+0000","last_undegraded":"2024-04-25T00:33:14.792843+0000","last_fullsized":"2024-04-25T00:33:14.792843+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:55.668698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.968271+0000","last_change":"2024-04-25T00:33:12.794824+0000","last_active":"2024-04-25T00:33:14.968271+0000","last_peered":"2024-04-25T00:33:14.968271+0000","last_clean":"2024-04-25T00:33:14.968271+0000","last_became_active":"2024-04-25T00:33:12.794088+0000","last_became_peered":"2024-04-25T00:33:12.794088+0000","last_unstale":"2024-04-25T00:33:14.968271+0000","last_undegraded":"2024-04-25T00:33:14.968271+0000","last_fullsized":"2024-04-25T00:33:14.968271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:31.459566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792964+0000","last_change":"2024-04-25T00:33:13.811136+0000","last_active":"2024-04-25T00:33:14.792964+0000","last_peered":"2024-04-25T00:33:14.792964+0000","last_clean":"2024-04-25T00:33:14.792964+0000","last_became_active":"2024-04-25T00:33:13.809868+0000","last_became_peered":"2024-04-25T00:33:13.809868+0000","last_unstale":"2024-04-25T00:33:14.792964+0000","last_undegraded":"2024-04-25T00:33:14.792964+0000","last_fullsized":"2024-04-25T00:33:14.792964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:00.082802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.799796+0000","last_change":"2024-04-25T00:33:12.801012+0000","last_active":"2024-04-25T00:33:15.799796+0000","last_peered":"2024-04-25T00:33:15.799796+0000","last_clean":"2024-04-25T00:33:15.799796+0000","last_became_active":"2024-04-25T00:33:12.800555+0000","last_became_peered":"2024-04-25T00:33:12.800555+0000","last_unstale":"2024-04-25T00:33:15.799796+0000","last_undegraded":"2024-04-25T00:33:15.799796+0000","last_fullsized":"2024-04-25T00:33:15.799796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:20.563260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.798937+0000","last_change":"2024-04-25T00:33:13.813541+0000","last_active":"2024-04-25T00:33:15.798937+0000","last_peered":"2024-04-25T00:33:15.798937+0000","last_clean":"2024-04-25T00:33:15.798937+0000","last_became_active":"2024-04-25T00:33:13.811864+0000","last_became_peered":"2024-04-25T00:33:13.811864+0000","last_unstale":"2024-04-25T00:33:15.798937+0000","last_undegraded":"2024-04-25T00:33:15.798937+0000","last_fullsized":"2024-04-25T00:33:15.798937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:46.179677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.800170+0000","last_change":"2024-04-25T00:33:12.801088+0000","last_active":"2024-04-25T00:33:15.800170+0000","last_peered":"2024-04-25T00:33:15.800170+0000","last_clean":"2024-04-25T00:33:15.800170+0000","last_became_active":"2024-04-25T00:33:12.800686+0000","last_became_peered":"2024-04-25T00:33:12.800686+0000","last_unstale":"2024-04-25T00:33:15.800170+0000","last_undegraded":"2024-04-25T00:33:15.800170+0000","last_fullsized":"2024-04-25T00:33:15.800170+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:47.735755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.832666+0000","last_change":"2024-04-25T00:33:13.833684+0000","last_active":"2024-04-25T00:33:15.832666+0000","last_peered":"2024-04-25T00:33:15.832666+0000","last_clean":"2024-04-25T00:33:15.832666+0000","last_became_active":"2024-04-25T00:33:13.833362+0000","last_became_peered":"2024-04-25T00:33:13.833362+0000","last_unstale":"2024-04-25T00:33:15.832666+0000","last_undegraded":"2024-04-25T00:33:15.832666+0000","last_fullsized":"2024-04-25T00:33:15.832666+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:00.082802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.967784+0000","last_change":"2024-04-25T00:33:12.804216+0000","last_active":"2024-04-25T00:33:14.967784+0000","last_peered":"2024-04-25T00:33:14.967784+0000","last_clean":"2024-04-25T00:33:14.967784+0000","last_became_active":"2024-04-25T00:33:12.800818+0000","last_became_peered":"2024-04-25T00:33:12.800818+0000","last_unstale":"2024-04-25T00:33:14.967784+0000","last_undegraded":"2024-04-25T00:33:14.967784+0000","last_fullsized":"2024-04-25T00:33:14.967784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:36.524992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.791004+0000","last_change":"2024-04-25T00:33:13.826876+0000","last_active":"2024-04-25T00:33:14.791004+0000","last_peered":"2024-04-25T00:33:14.791004+0000","last_clean":"2024-04-25T00:33:14.791004+0000","last_became_active":"2024-04-25T00:33:13.826414+0000","last_became_peered":"2024-04-25T00:33:13.826414+0000","last_unstale":"2024-04-25T00:33:14.791004+0000","last_undegraded":"2024-04-25T00:33:14.791004+0000","last_fullsized":"2024-04-25T00:33:14.791004+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:53.577738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792000+0000","last_change":"2024-04-25T00:33:12.793774+0000","last_active":"2024-04-25T00:33:14.792000+0000","last_peered":"2024-04-25T00:33:14.792000+0000","last_clean":"2024-04-25T00:33:14.792000+0000","last_became_active":"2024-04-25T00:33:12.793317+0000","last_became_peered":"2024-04-25T00:33:12.793317+0000","last_unstale":"2024-04-25T00:33:14.792000+0000","last_undegraded":"2024-04-25T00:33:14.792000+0000","last_fullsized":"2024-04-25T00:33:14.792000+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:38.919514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.791732+0000","last_change":"2024-04-25T00:33:13.802505+0000","last_active":"2024-04-25T00:33:14.791732+0000","last_peered":"2024-04-25T00:33:14.791732+0000","last_clean":"2024-04-25T00:33:14.791732+0000","last_became_active":"2024-04-25T00:33:13.802163+0000","last_became_peered":"2024-04-25T00:33:13.802163+0000","last_unstale":"2024-04-25T00:33:14.791732+0000","last_undegraded":"2024-04-25T00:33:14.791732+0000","last_fullsized":"2024-04-25T00:33:14.791732+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:14.100005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.456262+0000","last_change":"2024-04-25T00:33:12.805396+0000","last_active":"2024-04-25T00:33:15.456262+0000","last_peered":"2024-04-25T00:33:15.456262+0000","last_clean":"2024-04-25T00:33:15.456262+0000","last_became_active":"2024-04-25T00:33:12.803923+0000","last_became_peered":"2024-04-25T00:33:12.803923+0000","last_unstale":"2024-04-25T00:33:15.456262+0000","last_undegraded":"2024-04-25T00:33:15.456262+0000","last_fullsized":"2024-04-25T00:33:15.456262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:24.747622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.793360+0000","last_change":"2024-04-25T00:33:13.806306+0000","last_active":"2024-04-25T00:33:14.793360+0000","last_peered":"2024-04-25T00:33:14.793360+0000","last_clean":"2024-04-25T00:33:14.793360+0000","last_became_active":"2024-04-25T00:33:13.804739+0000","last_became_peered":"2024-04-25T00:33:13.804739+0000","last_unstale":"2024-04-25T00:33:14.793360+0000","last_undegraded":"2024-04-25T00:33:14.793360+0000","last_fullsized":"2024-04-25T00:33:14.793360+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:05.910290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.799744+0000","last_change":"2024-04-25T00:33:12.795318+0000","last_active":"2024-04-25T00:33:15.799744+0000","last_peered":"2024-04-25T00:33:15.799744+0000","last_clean":"2024-04-25T00:33:15.799744+0000","last_became_active":"2024-04-25T00:33:12.794708+0000","last_became_peered":"2024-04-25T00:33:12.794708+0000","last_unstale":"2024-04-25T00:33:15.799744+0000","last_undegraded":"2024-04-25T00:33:15.799744+0000","last_fullsized":"2024-04-25T00:33:15.799744+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:46.247093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.792409+0000","last_change":"2024-04-25T00:33:13.793233+0000","last_active":"2024-04-25T00:33:14.792409+0000","last_peered":"2024-04-25T00:33:14.792409+0000","last_clean":"2024-04-25T00:33:14.792409+0000","last_became_active":"2024-04-25T00:33:13.792685+0000","last_became_peered":"2024-04-25T00:33:13.792685+0000","last_unstale":"2024-04-25T00:33:14.792409+0000","last_undegraded":"2024-04-25T00:33:14.792409+0000","last_fullsized":"2024-04-25T00:33:14.792409+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:11.083727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.457529+0000","last_change":"2024-04-25T00:33:12.810928+0000","last_active":"2024-04-25T00:33:15.457529+0000","last_peered":"2024-04-25T00:33:15.457529+0000","last_clean":"2024-04-25T00:33:15.457529+0000","last_became_active":"2024-04-25T00:33:12.810284+0000","last_became_peered":"2024-04-25T00:33:12.810284+0000","last_unstale":"2024-04-25T00:33:15.457529+0000","last_undegraded":"2024-04-25T00:33:15.457529+0000","last_fullsized":"2024-04-25T00:33:15.457529+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:58.103145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.457342+0000","last_change":"2024-04-25T00:33:13.804400+0000","last_active":"2024-04-25T00:33:15.457342+0000","last_peered":"2024-04-25T00:33:15.457342+0000","last_clean":"2024-04-25T00:33:15.457342+0000","last_became_active":"2024-04-25T00:33:13.803089+0000","last_became_peered":"2024-04-25T00:33:13.803089+0000","last_unstale":"2024-04-25T00:33:15.457342+0000","last_undegraded":"2024-04-25T00:33:15.457342+0000","last_fullsized":"2024-04-25T00:33:15.457342+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:53.025254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.832626+0000","last_change":"2024-04-25T00:33:12.803739+0000","last_active":"2024-04-25T00:33:15.832626+0000","last_peered":"2024-04-25T00:33:15.832626+0000","last_clean":"2024-04-25T00:33:15.832626+0000","last_became_active":"2024-04-25T00:33:12.803381+0000","last_became_peered":"2024-04-25T00:33:12.803381+0000","last_unstale":"2024-04-25T00:33:15.832626+0000","last_undegraded":"2024-04-25T00:33:15.832626+0000","last_fullsized":"2024-04-25T00:33:15.832626+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:30.658933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.799398+0000","last_change":"2024-04-25T00:33:13.816305+0000","last_active":"2024-04-25T00:33:15.799398+0000","last_peered":"2024-04-25T00:33:15.799398+0000","last_clean":"2024-04-25T00:33:15.799398+0000","last_became_active":"2024-04-25T00:33:13.814617+0000","last_became_peered":"2024-04-25T00:33:13.814617+0000","last_unstale":"2024-04-25T00:33:15.799398+0000","last_undegraded":"2024-04-25T00:33:15.799398+0000","last_fullsized":"2024-04-25T00:33:15.799398+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:56.360368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.456828+0000","last_change":"2024-04-25T00:33:12.810436+0000","last_active":"2024-04-25T00:33:15.456828+0000","last_peered":"2024-04-25T00:33:15.456828+0000","last_clean":"2024-04-25T00:33:15.456828+0000","last_became_active":"2024-04-25T00:33:12.809773+0000","last_became_peered":"2024-04-25T00:33:12.809773+0000","last_unstale":"2024-04-25T00:33:15.456828+0000","last_undegraded":"2024-04-25T00:33:15.456828+0000","last_fullsized":"2024-04-25T00:33:15.456828+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:46.247093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.799197+0000","last_change":"2024-04-25T00:33:13.811631+0000","last_active":"2024-04-25T00:33:15.799197+0000","last_peered":"2024-04-25T00:33:15.799197+0000","last_clean":"2024-04-25T00:33:15.799197+0000","last_became_active":"2024-04-25T00:33:13.811316+0000","last_became_peered":"2024-04-25T00:33:13.811316+0000","last_unstale":"2024-04-25T00:33:15.799197+0000","last_undegraded":"2024-04-25T00:33:15.799197+0000","last_fullsized":"2024-04-25T00:33:15.799197+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:30.958863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.804134+0000","last_change":"2024-04-25T00:33:12.791171+0000","last_active":"2024-04-25T00:33:15.804134+0000","last_peered":"2024-04-25T00:33:15.804134+0000","last_clean":"2024-04-25T00:33:15.804134+0000","last_became_active":"2024-04-25T00:33:12.790272+0000","last_became_peered":"2024-04-25T00:33:12.790272+0000","last_unstale":"2024-04-25T00:33:15.804134+0000","last_undegraded":"2024-04-25T00:33:15.804134+0000","last_fullsized":"2024-04-25T00:33:15.804134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:30.658933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.967801+0000","last_change":"2024-04-25T00:33:13.825088+0000","last_active":"2024-04-25T00:33:14.967801+0000","last_peered":"2024-04-25T00:33:14.967801+0000","last_clean":"2024-04-25T00:33:14.967801+0000","last_became_active":"2024-04-25T00:33:13.819551+0000","last_became_peered":"2024-04-25T00:33:13.819551+0000","last_unstale":"2024-04-25T00:33:14.967801+0000","last_undegraded":"2024-04-25T00:33:14.967801+0000","last_fullsized":"2024-04-25T00:33:14.967801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:47.255920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.832554+0000","last_change":"2024-04-25T00:33:12.797798+0000","last_active":"2024-04-25T00:33:15.832554+0000","last_peered":"2024-04-25T00:33:15.832554+0000","last_clean":"2024-04-25T00:33:15.832554+0000","last_became_active":"2024-04-25T00:33:12.797086+0000","last_became_peered":"2024-04-25T00:33:12.797086+0000","last_unstale":"2024-04-25T00:33:15.832554+0000","last_undegraded":"2024-04-25T00:33:15.832554+0000","last_fullsized":"2024-04-25T00:33:15.832554+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:42.839016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.803732+0000","last_change":"2024-04-25T00:33:13.815143+0000","last_active":"2024-04-25T00:33:15.803732+0000","last_peered":"2024-04-25T00:33:15.803732+0000","last_clean":"2024-04-25T00:33:15.803732+0000","last_became_active":"2024-04-25T00:33:13.813674+0000","last_became_peered":"2024-04-25T00:33:13.813674+0000","last_unstale":"2024-04-25T00:33:15.803732+0000","last_undegraded":"2024-04-25T00:33:15.803732+0000","last_fullsized":"2024-04-25T00:33:15.803732+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:14.100005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.791857+0000","last_change":"2024-04-25T00:33:12.789836+0000","last_active":"2024-04-25T00:33:14.791857+0000","last_peered":"2024-04-25T00:33:14.791857+0000","last_clean":"2024-04-25T00:33:14.791857+0000","last_became_active":"2024-04-25T00:33:12.789493+0000","last_became_peered":"2024-04-25T00:33:12.789493+0000","last_unstale":"2024-04-25T00:33:14.791857+0000","last_undegraded":"2024-04-25T00:33:14.791857+0000","last_fullsized":"2024-04-25T00:33:14.791857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:11.471922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.793442+0000","last_change":"2024-04-25T00:33:13.790582+0000","last_active":"2024-04-25T00:33:14.793442+0000","last_peered":"2024-04-25T00:33:14.793442+0000","last_clean":"2024-04-25T00:33:14.793442+0000","last_became_active":"2024-04-25T00:33:13.789842+0000","last_became_peered":"2024-04-25T00:33:13.789842+0000","last_unstale":"2024-04-25T00:33:14.793442+0000","last_undegraded":"2024-04-25T00:33:14.793442+0000","last_fullsized":"2024-04-25T00:33:14.793442+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:12.755846+0000","last_clean_scrub_stamp":"2024-04-25T00:33:12.755846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:50.670604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:14.968638+0000","last_change":"2024-04-25T00:33:12.795705+0000","last_active":"2024-04-25T00:33:14.968638+0000","last_peered":"2024-04-25T00:33:14.968638+0000","last_clean":"2024-04-25T00:33:14.968638+0000","last_became_active":"2024-04-25T00:33:12.795091+0000","last_became_peered":"2024-04-25T00:33:12.795091+0000","last_unstale":"2024-04-25T00:33:14.968638+0000","last_undegraded":"2024-04-25T00:33:14.968638+0000","last_fullsized":"2024-04-25T00:33:14.968638+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:33:11.747019+0000","last_clean_scrub_stamp":"2024-04-25T00:33:11.747019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:13.091178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":216,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:33:15.456401+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:33:15.456401+0000","last_peered":"2024-04-25T00:33:15.456401+0000","last_clean":"2024-04-25T00:33:15.456401+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:33:15.456401+0000","last_undegraded":"2024-04-25T00:33:15.456401+0000","last_fullsized":"2024-04-25T00:33:15.456401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":805921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":536576,"data_stored":216135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":215533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":540672,"data_stored":217075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35872,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335968,"statfs":{"total":96636764160,"available":96600031232,"internally_reserved":0,"allocated":540672,"data_stored":216101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":217097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":532480,"data_stored":215533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36444,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335396,"statfs":{"total":96636764160,"available":96599445504,"internally_reserved":0,"allocated":1126400,"data_stored":805865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:33:22.675 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:33:22.676 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:33:22.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5ccbb3640 1 Processor -- start 2024-04-25T00:33:22.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5ccbb3640 1 -- start start 2024-04-25T00:33:22.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5ccbb3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5c80b2b70 0x7fa5c8151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5ccbb3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5c80b3130 con 0x7fa5c815d980 2024-04-25T00:33:22.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5ccbb3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5c80b3410 con 0x7fa5c806dce0 2024-04-25T00:33:22.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5ccbb3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c80cf560 con 0x7fa5c80b2b70 2024-04-25T00:33:22.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c659c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa5c815d980 0x7fa5c806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59042/0 (socket says 172.21.15.29:59042) 2024-04-25T00:33:22.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c659c640 1 -- 172.21.15.29:0/123317141 learned_addr learned my addr 172.21.15.29:0/123317141 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:22.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c6d9d640 1 --2- 172.21.15.29:0/123317141 >> v2:172.21.15.157:3300/0 conn(0x7fa5c80b2b70 0x7fa5c8151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1682658354 0 0) 0x7fa5c80b3130 con 0x7fa5c815d980 2024-04-25T00:33:22.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5b0004640 con 0x7fa5c815d980 2024-04-25T00:33:22.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1844924075 0 0) 0x7fa5b0004640 con 0x7fa5c815d980 2024-04-25T00:33:22.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 >> v2:172.21.15.157:3300/0 conn(0x7fa5c80b2b70 msgr2=0x7fa5c8151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:22.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 --2- 172.21.15.29:0/123317141 >> v2:172.21.15.157:3300/0 conn(0x7fa5c80b2b70 0x7fa5c8151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 >> v1:172.21.15.157:6789/0 conn(0x7fa5c806dce0 legacy=0x7fa5c8150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:22.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c80b36f0 con 0x7fa5c815d980 2024-04-25T00:33:22.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fa5b8001680 con 0x7fa5c815d980 2024-04-25T00:33:22.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 >> v1:172.21.15.29:6789/0 conn(0x7fa5c815d980 legacy=0x7fa5c806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:22.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 --2- 172.21.15.29:0/123317141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5b0004ad0 0x7fa5b0004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 --2- 172.21.15.29:0/123317141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5b0005920 0x7fa5b0005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 --2- 172.21.15.29:0/123317141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5b00062e0 0x7fa5b000cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c80cf560 con 0x7fa5b00062e0 2024-04-25T00:33:22.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b000d180 con 0x7fa5b0005920 2024-04-25T00:33:22.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b000d380 con 0x7fa5b0004ad0 2024-04-25T00:33:22.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c6d9d640 1 --2- 172.21.15.29:0/123317141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5b00062e0 0x7fa5b000cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c659c640 1 --2- 172.21.15.29:0/123317141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5b0004ad0 0x7fa5b0004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c5d9b640 1 --2- 172.21.15.29:0/123317141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5b0005920 0x7fa5b0005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c6d9d640 1 -- 172.21.15.29:0/123317141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5b0004ad0 msgr2=0x7fa5b0004ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:22.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c6d9d640 1 --2- 172.21.15.29:0/123317141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5b0004ad0 0x7fa5b0004ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c6d9d640 1 -- 172.21.15.29:0/123317141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5b0005920 msgr2=0x7fa5b0005da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:22.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c6d9d640 1 --2- 172.21.15.29:0/123317141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5b0005920 0x7fa5b0005da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c6d9d640 1 -- 172.21.15.29:0/123317141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b0004640 con 0x7fa5b00062e0 2024-04-25T00:33:22.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c5d9b640 1 --2- 172.21.15.29:0/123317141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5b0005920 0x7fa5b0005da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:22.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.846+0000 7fa5c659c640 1 --2- 172.21.15.29:0/123317141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5b0004ad0 0x7fa5b0004ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:22.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5c6d9d640 1 -- 172.21.15.29:0/123317141 reap_dead start 2024-04-25T00:33:22.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5c6d9d640 1 --2- 172.21.15.29:0/123317141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5b00062e0 0x7fa5b000cae0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009fc0 tx=0x7fa5c001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73968cab73b67c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:22.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0009c10 con 0x7fa5b00062e0 2024-04-25T00:33:22.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c00a5d10 con 0x7fa5b00062e0 2024-04-25T00:33:22.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5c559a640 1 -- 172.21.15.29:0/123317141 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0003be0 con 0x7fa5b00062e0 2024-04-25T00:33:22.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/123317141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5b00062e0 msgr2=0x7fa5b000cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:22.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5ccbb3640 1 --2- 172.21.15.29:0/123317141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5b00062e0 0x7fa5b000cae0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009fc0 tx=0x7fa5c001ccc0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/123317141 shutdown_connections 2024-04-25T00:33:22.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5ccbb3640 1 --2- 172.21.15.29:0/123317141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5b00062e0 0x7fa5b000cae0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/123317141 >> 172.21.15.29:0/123317141 conn(0x7fa5c80b1870 msgr2=0x7fa5c80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:22.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/123317141 shutdown_connections 2024-04-25T00:33:22.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/123317141 wait complete. 2024-04-25T00:33:22.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5ccbb3640 1 Processor -- start 2024-04-25T00:33:22.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.850+0000 7fa5ccbb3640 1 -- start start 2024-04-25T00:33:22.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5ccbb3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5b00062e0 0x7fa5c81f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5ccbb3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c8148740 0x7fa5c80ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5ccbb3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5c815c470 0x7fa5c815c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5ccbb3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c815dd60 con 0x7fa5c815c470 2024-04-25T00:33:22.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5ccbb3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c80b3cb0 con 0x7fa5b00062e0 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5ccbb3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c8150df0 con 0x7fa5c8148740 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5c6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5c815c470 0x7fa5c815c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5c6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5c815c470 0x7fa5c815c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43440/0 (socket says 172.21.15.29:43440) 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5c6d9d640 1 -- 172.21.15.29:0/2313689497 learned_addr learned my addr 172.21.15.29:0/2313689497 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5c659c640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5b00062e0 0x7fa5c81f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5c5d9b640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c8148740 0x7fa5c80ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5c6d9d640 1 -- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c8148740 msgr2=0x7fa5c80ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5c6d9d640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c8148740 0x7fa5c80ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5c6d9d640 1 -- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5b00062e0 msgr2=0x7fa5c81f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5c6d9d640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5b00062e0 0x7fa5c81f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5c6d9d640 1 -- 172.21.15.29:0/2313689497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c8148bc0 con 0x7fa5c815c470 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5c6d9d640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5c815c470 0x7fa5c815c920 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa5c001d1a0 tx=0x7fa5c0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5b6ffd640 1 -- 172.21.15.29:0/2313689497 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0009c10 con 0x7fa5c815c470 2024-04-25T00:33:22.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5b6ffd640 1 -- 172.21.15.29:0/2313689497 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c00bd070 con 0x7fa5c815c470 2024-04-25T00:33:22.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/2313689497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c80ccac0 con 0x7fa5c815c470 2024-04-25T00:33:22.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5b6ffd640 1 -- 172.21.15.29:0/2313689497 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00039a0 con 0x7fa5c815c470 2024-04-25T00:33:22.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/2313689497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c80ad8a0 con 0x7fa5c815c470 2024-04-25T00:33:22.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/2313689497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c8069270 con 0x7fa5c815c470 2024-04-25T00:33:22.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5b6ffd640 1 -- 172.21.15.29:0/2313689497 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fa5c0003b40 con 0x7fa5c815c470 2024-04-25T00:33:22.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5b6ffd640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa5a8067a40 0x7fa5a8069f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:22.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5b6ffd640 1 -- 172.21.15.29:0/2313689497 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fa5c0121940 con 0x7fa5c815c470 2024-04-25T00:33:22.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.854+0000 7fa5c659c640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa5a8067a40 0x7fa5a8069f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:22.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.858+0000 7fa5c659c640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa5a8067a40 0x7fa5a8069f00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8000c10 tx=0x7fa5b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:22.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:22.858+0000 7fa5b6ffd640 1 -- 172.21.15.29:0/2313689497 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa5c00169d0 con 0x7fa5c815c470 2024-04-25T00:33:23.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.074+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/2313689497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa5c80cf910 con 0x7fa5c815c470 2024-04-25T00:33:23.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.078+0000 7fa5b6ffd640 1 -- 172.21.15.29:0/2313689497 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fa5c00f2290 con 0x7fa5c815c470 2024-04-25T00:33:23.083 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-04-25T00:33:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A== 2024-04-25T00:33:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: caps mds = "allow" 2024-04-25T00:33:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: caps mon = "allow r" 2024-04-25T00:33:23.083 INFO:teuthology.orchestra.run.smithi029.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:33:23.084 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-04-25T00:33:23.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.078+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa5a8067a40 msgr2=0x7fa5a8069f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:23.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.078+0000 7fa5ccbb3640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa5a8067a40 0x7fa5a8069f00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8000c10 tx=0x7fa5b8009040 comp rx=0 tx=0).stop 2024-04-25T00:33:23.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.078+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/2313689497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5c815c470 msgr2=0x7fa5c815c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:23.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.078+0000 7fa5ccbb3640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5c815c470 0x7fa5c815c920 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa5c001d1a0 tx=0x7fa5c0009750 comp rx=0 tx=0).stop 2024-04-25T00:33:23.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.078+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/2313689497 shutdown_connections 2024-04-25T00:33:23.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.078+0000 7fa5ccbb3640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5c815c470 0x7fa5c815c920 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.078+0000 7fa5ccbb3640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5c8148740 0x7fa5c80ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.078+0000 7fa5ccbb3640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa5a8067a40 0x7fa5a8069f00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.078+0000 7fa5ccbb3640 1 --2- 172.21.15.29:0/2313689497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5b00062e0 0x7fa5c81f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.078+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/2313689497 >> 172.21.15.29:0/2313689497 conn(0x7fa5c80b1870 msgr2=0x7fa5c80d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:23.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.078+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/2313689497 shutdown_connections 2024-04-25T00:33:23.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.078+0000 7fa5ccbb3640 1 -- 172.21.15.29:0/2313689497 wait complete. 2024-04-25T00:33:23.101 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:33:23.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f54449640 1 Processor -- start 2024-04-25T00:33:23.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f54449640 1 -- start start 2024-04-25T00:33:23.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f54449640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4f4c0b4780 0x7f4f4c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:23.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f54449640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f4c0b3720 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f54449640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f4c0b39c0 con 0x7f4f4c0cc240 2024-04-25T00:33:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f54449640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4f4c0cc650 con 0x7f4f4c0b4780 2024-04-25T00:33:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f521e5640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4f4c15e8d0 0x7f4f4c15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59044/0 (socket says 172.21.15.29:59044) 2024-04-25T00:33:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f521e5640 1 -- 172.21.15.29:0/1469517377 learned_addr learned my addr 172.21.15.29:0/1469517377 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f519e4640 1 --2- 172.21.15.29:0/1469517377 >> v2:172.21.15.29:3300/0 conn(0x7f4f4c0b4780 0x7f4f4c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f511e3640 1 -- 172.21.15.29:0/1469517377 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884924178 0 0) 0x7f4f4c0b3720 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f511e3640 1 -- 172.21.15.29:0/1469517377 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f38003530 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f519e4640 1 -- 172.21.15.29:0/1469517377 >> v1:172.21.15.157:6789/0 conn(0x7f4f4c0cc240 legacy=0x7f4f4c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f519e4640 1 -- 172.21.15.29:0/1469517377 >> v1:172.21.15.29:6789/0 conn(0x7f4f4c15e8d0 legacy=0x7f4f4c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.214+0000 7f4f519e4640 1 -- 172.21.15.29:0/1469517377 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c0b3ca0 con 0x7f4f4c0b4780 2024-04-25T00:33:23.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f519e4640 1 --2- 172.21.15.29:0/1469517377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f4c0b4780 0x7f4f4c15f3c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4f480097c0 tx=0x7f4f4801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac918bfd5cb7edfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:23.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f511e3640 1 -- 172.21.15.29:0/1469517377 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f480b0070 con 0x7f4f4c0b4780 2024-04-25T00:33:23.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f511e3640 1 -- 172.21.15.29:0/1469517377 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f480ab600 con 0x7f4f4c0b4780 2024-04-25T00:33:23.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f511e3640 1 -- 172.21.15.29:0/1469517377 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f480ab9d0 con 0x7f4f4c0b4780 2024-04-25T00:33:23.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 -- 172.21.15.29:0/1469517377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f4c0b4780 msgr2=0x7f4f4c15f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:23.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 --2- 172.21.15.29:0/1469517377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f4c0b4780 0x7f4f4c15f3c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4f480097c0 tx=0x7f4f4801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 -- 172.21.15.29:0/1469517377 shutdown_connections 2024-04-25T00:33:23.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 --2- 172.21.15.29:0/1469517377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f4c0b4780 0x7f4f4c15f3c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 -- 172.21.15.29:0/1469517377 >> 172.21.15.29:0/1469517377 conn(0x7f4f4c0b1870 msgr2=0x7f4f4c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:23.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 -- 172.21.15.29:0/1469517377 shutdown_connections 2024-04-25T00:33:23.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 -- 172.21.15.29:0/1469517377 wait complete. 2024-04-25T00:33:23.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 Processor -- start 2024-04-25T00:33:23.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 -- start start 2024-04-25T00:33:23.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0b4780 0x7f4f4c159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:23.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4c0cc240 0x7f4f4c15b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:23.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f4c15e8d0 0x7f4f4c1fb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:23.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c06d750 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c06d5d0 con 0x7f4f4c0b4780 2024-04-25T00:33:23.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f54449640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4c06d8d0 con 0x7f4f4c0cc240 2024-04-25T00:33:23.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f529e6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f4c15e8d0 0x7f4f4c1fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:23.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f529e6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f4c15e8d0 0x7f4f4c1fb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43452/0 (socket says 172.21.15.29:43452) 2024-04-25T00:33:23.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f529e6640 1 -- 172.21.15.29:0/1209531281 learned_addr learned my addr 172.21.15.29:0/1209531281 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:23.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f521e5640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0b4780 0x7f4f4c159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:23.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f519e4640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4c0cc240 0x7f4f4c15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:23.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f529e6640 1 -- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4c0cc240 msgr2=0x7f4f4c15b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:23.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f529e6640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4c0cc240 0x7f4f4c15b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f529e6640 1 -- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0b4780 msgr2=0x7f4f4c159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:23.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f529e6640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0b4780 0x7f4f4c159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f529e6640 1 -- 172.21.15.29:0/1209531281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c0cbef0 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.218+0000 7f4f529e6640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f4c15e8d0 0x7f4f4c1fb030 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400c770 tx=0x7f4f4400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.222+0000 7f4f42ffd640 1 -- 172.21.15.29:0/1209531281 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44003b80 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.222+0000 7f4f42ffd640 1 -- 172.21.15.29:0/1209531281 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f44003d20 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.222+0000 7f4f54449640 1 -- 172.21.15.29:0/1209531281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c13d710 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.222+0000 7f4f42ffd640 1 -- 172.21.15.29:0/1209531281 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400aa90 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.222+0000 7f4f54449640 1 -- 172.21.15.29:0/1209531281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c13dcd0 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.222+0000 7f4f54449640 1 -- 172.21.15.29:0/1209531281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4c069270 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.222+0000 7f4f42ffd640 1 -- 172.21.15.29:0/1209531281 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f4f4400ac30 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.222+0000 7f4f42ffd640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f30067a70 0x7f4f30069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:23.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.222+0000 7f4f42ffd640 1 -- 172.21.15.29:0/1209531281 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f4f44085ee0 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.222+0000 7f4f521e5640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f30067a70 0x7f4f30069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:23.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.222+0000 7f4f521e5640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f30067a70 0x7f4f30069f30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0011f0 tx=0x7f4f3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:23.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.226+0000 7f4f42ffd640 1 -- 172.21.15.29:0/1209531281 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4f440567a0 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.426+0000 7f4f54449640 1 -- 172.21.15.29:0/1209531281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f4c06e7b0 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.430+0000 7f4f42ffd640 1 -- 172.21.15.29:0/1209531281 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7f4f440560a0 con 0x7f4f4c15e8d0 2024-04-25T00:33:23.435 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:23.435 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:33:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 24 2024-04-25T00:33:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.430+0000 7f4f54449640 1 -- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f30067a70 msgr2=0x7f4f30069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.430+0000 7f4f54449640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f30067a70 0x7f4f30069f30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0011f0 tx=0x7f4f3c009040 comp rx=0 tx=0).stop 2024-04-25T00:33:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.430+0000 7f4f54449640 1 -- 172.21.15.29:0/1209531281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f4c15e8d0 msgr2=0x7f4f4c1fb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.430+0000 7f4f54449640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f4c15e8d0 0x7f4f4c1fb030 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400c770 tx=0x7f4f4400cc30 comp rx=0 tx=0).stop 2024-04-25T00:33:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.430+0000 7f4f54449640 1 -- 172.21.15.29:0/1209531281 shutdown_connections 2024-04-25T00:33:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.430+0000 7f4f54449640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f4c15e8d0 0x7f4f4c1fb030 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.430+0000 7f4f54449640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f4c0cc240 0x7f4f4c15b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.430+0000 7f4f54449640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f30067a70 0x7f4f30069f30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.430+0000 7f4f54449640 1 --2- 172.21.15.29:0/1209531281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0b4780 0x7f4f4c159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.430+0000 7f4f54449640 1 -- 172.21.15.29:0/1209531281 >> 172.21.15.29:0/1209531281 conn(0x7f4f4c0b1870 msgr2=0x7f4f4c0b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.430+0000 7f4f54449640 1 -- 172.21.15.29:0/1209531281 shutdown_connections 2024-04-25T00:33:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.430+0000 7f4f54449640 1 -- 172.21.15.29:0/1209531281 wait complete. 2024-04-25T00:33:23.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:33:13.156560+0000', 'modified': '2024-04-25T00:33:16.179568+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5733}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5733': {'gid': 5733, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/628132809', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 628132809}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 628132809}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5733, 'qdb_cluster': [5733]} 2024-04-25T00:33:23.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:33:23.855 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:33:23.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f307dd19640 1 Processor -- start 2024-04-25T00:33:23.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f307dd19640 1 -- start start 2024-04-25T00:33:23.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f307dd19640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f307815c0e0 0x7f30780c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:23.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f307dd19640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30780d0010 0x7f30780c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:23.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f307dd19640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30780cf540 0x7f30780cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:23.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f307dd19640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f307815c4c0 con 0x7f30780cf540 2024-04-25T00:33:23.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f307dd19640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f30780d04f0 con 0x7f307815c0e0 2024-04-25T00:33:23.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f307dd19640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f30780d07d0 con 0x7f30780d0010 2024-04-25T00:33:23.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f3077fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30780cf540 0x7f30780cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:23.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f3077fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30780cf540 0x7f30780cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43454/0 (socket says 172.21.15.29:43454) 2024-04-25T00:33:23.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f3077fff640 1 -- 172.21.15.29:0/4106626336 learned_addr learned my addr 172.21.15.29:0/4106626336 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:23.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f3076ffd640 1 --2- 172.21.15.29:0/4106626336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30780d0010 0x7f30780c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:23.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f30777fe640 1 --2- 172.21.15.29:0/4106626336 >> v2:172.21.15.157:3300/0 conn(0x7f307815c0e0 0x7f30780c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:23.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f3077fff640 1 -- 172.21.15.29:0/4106626336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30780d0010 msgr2=0x7f30780c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:23.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f3077fff640 1 --2- 172.21.15.29:0/4106626336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30780d0010 0x7f30780c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f3077fff640 1 -- 172.21.15.29:0/4106626336 >> v2:172.21.15.157:3300/0 conn(0x7f307815c0e0 msgr2=0x7f30780c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:23.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f3077fff640 1 --2- 172.21.15.29:0/4106626336 >> v2:172.21.15.157:3300/0 conn(0x7f307815c0e0 0x7f30780c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.986+0000 7f3077fff640 1 -- 172.21.15.29:0/4106626336 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30780d0ab0 con 0x7f30780cf540 2024-04-25T00:33:23.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f3077fff640 1 --2- 172.21.15.29:0/4106626336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30780cf540 0x7f30780cc420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f307806c1f0 tx=0x7f307001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a457280e1ace1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:23.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f30767fc640 1 -- 172.21.15.29:0/4106626336 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30700a5840 con 0x7f30780cf540 2024-04-25T00:33:23.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f30767fc640 1 -- 172.21.15.29:0/4106626336 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30700a59e0 con 0x7f30780cf540 2024-04-25T00:33:23.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f30767fc640 1 -- 172.21.15.29:0/4106626336 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30700a5db0 con 0x7f30780cf540 2024-04-25T00:33:23.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f307dd19640 1 -- 172.21.15.29:0/4106626336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30780cf540 msgr2=0x7f30780cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:23.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f307dd19640 1 --2- 172.21.15.29:0/4106626336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30780cf540 0x7f30780cc420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f307806c1f0 tx=0x7f307001c9c0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f307dd19640 1 -- 172.21.15.29:0/4106626336 shutdown_connections 2024-04-25T00:33:23.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f307dd19640 1 --2- 172.21.15.29:0/4106626336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30780cf540 0x7f30780cc420 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f307dd19640 1 --2- 172.21.15.29:0/4106626336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30780d0010 0x7f30780c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f307dd19640 1 --2- 172.21.15.29:0/4106626336 >> v2:172.21.15.157:3300/0 conn(0x7f307815c0e0 0x7f30780c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:23.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f307dd19640 1 -- 172.21.15.29:0/4106626336 >> 172.21.15.29:0/4106626336 conn(0x7f30780b3a00 msgr2=0x7f30780b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:23.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f307dd19640 1 -- 172.21.15.29:0/4106626336 shutdown_connections 2024-04-25T00:33:23.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f307dd19640 1 -- 172.21.15.29:0/4106626336 wait complete. 2024-04-25T00:33:23.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f307dd19640 1 Processor -- start 2024-04-25T00:33:23.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.990+0000 7f307dd19640 1 -- start start 2024-04-25T00:33:24.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f307dd19640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30780cf540 0x7f30781535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f307dd19640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30780d0010 0x7f30780b7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f307dd19640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307815c0e0 0x7f30781555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f307dd19640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f307815fcd0 con 0x7f30780cf540 2024-04-25T00:33:24.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f307dd19640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f307815fb50 con 0x7f30780d0010 2024-04-25T00:33:24.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f307dd19640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f307815fe50 con 0x7f307815c0e0 2024-04-25T00:33:24.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f30777fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30780cf540 0x7f30781535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f30777fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30780cf540 0x7f30781535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43460/0 (socket says 172.21.15.29:43460) 2024-04-25T00:33:24.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f30777fe640 1 -- 172.21.15.29:0/3180579651 learned_addr learned my addr 172.21.15.29:0/3180579651 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:24.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f3077fff640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307815c0e0 0x7f30781555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f3076ffd640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30780d0010 0x7f30780b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f30777fe640 1 -- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307815c0e0 msgr2=0x7f30781555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f30777fe640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307815c0e0 0x7f30781555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f30777fe640 1 -- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30780d0010 msgr2=0x7f30780b7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f30777fe640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30780d0010 0x7f30780b7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f30777fe640 1 -- 172.21.15.29:0/3180579651 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3078153c20 con 0x7f30780cf540 2024-04-25T00:33:24.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f30777fe640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30780cf540 0x7f30781535b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f306800d620 tx=0x7f306800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:24.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f306ffff640 1 -- 172.21.15.29:0/3180579651 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306800b620 con 0x7f30780cf540 2024-04-25T00:33:24.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f306ffff640 1 -- 172.21.15.29:0/3180579651 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3068019070 con 0x7f30780cf540 2024-04-25T00:33:24.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f307dd19640 1 -- 172.21.15.29:0/3180579651 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307815c780 con 0x7f30780cf540 2024-04-25T00:33:24.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f306ffff640 1 -- 172.21.15.29:0/3180579651 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306800b9e0 con 0x7f30780cf540 2024-04-25T00:33:24.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f307dd19640 1 -- 172.21.15.29:0/3180579651 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3078155af0 con 0x7f30780cf540 2024-04-25T00:33:24.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f306ffff640 1 -- 172.21.15.29:0/3180579651 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f306800bb80 con 0x7f30780cf540 2024-04-25T00:33:24.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f306ffff640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3058067a70 0x7f3058069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.994+0000 7f306ffff640 1 -- 172.21.15.29:0/3180579651 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f3068086420 con 0x7f30780cf540 2024-04-25T00:33:24.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3076ffd640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3058067a70 0x7f3058069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 Processor -- start 2024-04-25T00:33:24.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 -- start start 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3076ffd640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3058067a70 0x7f3058069f30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f307815f270 tx=0x7f3060008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 0x55b4d08382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d0838810 0x55b4d07ee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 0x55b4d07eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b4d063c5e0 con 0x55b4d07eebc0 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b4d05c3c50 con 0x55b4d0837ef0 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55b4d06385a0 con 0x55b4d0838810 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3077fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 0x55b4d07eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3077fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 0x55b4d07eefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43474/0 (socket says 172.21.15.29:43474) 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3077fff640 1 -- 172.21.15.29:0/179976733 learned_addr learned my addr 172.21.15.29:0/179976733 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f30777fe640 1 --2- 172.21.15.29:0/179976733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 0x55b4d08382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3076ffd640 1 --2- 172.21.15.29:0/179976733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d0838810 0x55b4d07ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3077fff640 1 -- 172.21.15.29:0/179976733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d0838810 msgr2=0x55b4d07ee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3077fff640 1 --2- 172.21.15.29:0/179976733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d0838810 0x55b4d07ee680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3077fff640 1 -- 172.21.15.29:0/179976733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 msgr2=0x55b4d08382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3077fff640 1 --2- 172.21.15.29:0/179976733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 0x55b4d08382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3077fff640 1 -- 172.21.15.29:0/179976733 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4d05c1d10 con 0x55b4d07eebc0 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f30777fe640 1 --2- 172.21.15.29:0/179976733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 0x55b4d08382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3077fff640 1 --2- 172.21.15.29:0/179976733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 0x55b4d07eefa0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f307001d260 tx=0x7f3070004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a02f04b85e84e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f306d7fa640 1 -- 172.21.15.29:0/179976733 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30700bd070 con 0x55b4d07eebc0 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f306d7fa640 1 -- 172.21.15.29:0/179976733 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30700a5840 con 0x55b4d07eebc0 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f306d7fa640 1 -- 172.21.15.29:0/179976733 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30700a5bf0 con 0x55b4d07eebc0 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 -- 172.21.15.29:0/179976733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 msgr2=0x55b4d07eefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 --2- 172.21.15.29:0/179976733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 0x55b4d07eefa0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f307001d260 tx=0x7f3070004ac0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 -- 172.21.15.29:0/179976733 shutdown_connections 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 --2- 172.21.15.29:0/179976733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 0x55b4d07eefa0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 --2- 172.21.15.29:0/179976733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d0838810 0x55b4d07ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 --2- 172.21.15.29:0/179976733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 0x55b4d08382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 -- 172.21.15.29:0/179976733 >> 172.21.15.29:0/179976733 conn(0x55b4d07eab70 msgr2=0x55b4d07ecf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 -- 172.21.15.29:0/179976733 shutdown_connections 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 -- 172.21.15.29:0/179976733 wait complete. 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 -- start start 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:23.998+0000 7f3080e92000 1 Processor -- start 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d07eebc0 0x55b4d081cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0837ef0 0x55b4d081d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0838810 0x55b4d081d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b4d05f6520 con 0x55b4d0837ef0 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b4d07ea9f0 con 0x55b4d0838810 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55b4d081bb20 con 0x55b4d07eebc0 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 10 client.0 _ll_register_callbacks cb 0x55b4d07d95c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3076ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0837ef0 0x55b4d081d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3076ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0837ef0 0x55b4d081d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43490/0 (socket says 172.21.15.29:43490) 2024-04-25T00:33:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3076ffd640 1 -- 172.21.15.29:0/375898562 learned_addr learned my addr 172.21.15.29:0/375898562 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f30777fe640 1 --2- 172.21.15.29:0/375898562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d07eebc0 0x55b4d081cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3077fff640 1 --2- 172.21.15.29:0/375898562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0838810 0x55b4d081d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3076ffd640 1 -- 172.21.15.29:0/375898562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d07eebc0 msgr2=0x55b4d081cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3076ffd640 1 --2- 172.21.15.29:0/375898562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d07eebc0 0x55b4d081cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3076ffd640 1 -- 172.21.15.29:0/375898562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0838810 msgr2=0x55b4d081d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3076ffd640 1 --2- 172.21.15.29:0/375898562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0838810 0x55b4d081d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3076ffd640 1 -- 172.21.15.29:0/375898562 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4d05d47b0 con 0x55b4d0837ef0 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3076ffd640 1 --2- 172.21.15.29:0/375898562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0837ef0 0x55b4d081d460 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3060014460 tx=0x7f3060004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec6c917be60ee5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f30567fc640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f30567fc640 1 -- 172.21.15.29:0/375898562 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060015820 con 0x55b4d0837ef0 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f30567fc640 1 -- 172.21.15.29:0/375898562 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f30600159c0 con 0x55b4d0837ef0 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 -- 172.21.15.29:0/375898562 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30700a5840 con 0x55b4d0837ef0 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f30567fc640 1 -- 172.21.15.29:0/375898562 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f306002e050 con 0x55b4d0837ef0 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 10 client.5904 fetch_fsmap learned FSMap version 24 2024-04-25T00:33:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 -- 172.21.15.29:0/375898562 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55b4d05c3ea0 con 0x55b4d0837ef0 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f30567fc640 1 -- 172.21.15.29:0/375898562 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30700a5840 con 0x55b4d0837ef0 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f30567fc640 1 -- 172.21.15.29:0/375898562 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7f3060031020 con 0x55b4d0837ef0 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 10 client.5904 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 10 client.5904 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 20 client.5904 populate_metadata read hostname 'smithi029' 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 --2- 172.21.15.29:0/375898562 >> [v2:172.21.15.29:6833/628132809,v1:172.21.15.29:6835/628132809] conn(0x55b4d0847f90 0x55b4d084a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 4 client.5904 mds_command: new command op to 5733 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 -- 172.21.15.29:0/375898562 --> [v2:172.21.15.29:6833/628132809,v1:172.21.15.29:6835/628132809] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4d05d7ab0 con 0x55b4d0847f90 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f30777fe640 1 --2- 172.21.15.29:0/375898562 >> [v2:172.21.15.29:6833/628132809,v1:172.21.15.29:6835/628132809] conn(0x55b4d0847f90 0x55b4d084a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f30777fe640 1 --2- 172.21.15.29:0/375898562 >> [v2:172.21.15.29:6833/628132809,v1:172.21.15.29:6835/628132809] conn(0x55b4d0847f90 0x55b4d084a380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91902a93597829fe server_cookie=e924b6222dd3cd1c in_seq=0 out_seq=0 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f30567fc640 10 client.5904 ms_handle_connect on v2:172.21.15.29:6833/628132809 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f30567fc640 1 -- 172.21.15.29:0/375898562 <== mds.0 v2:172.21.15.29:6833/628132809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4d05d7ab0 con 0x55b4d0847f90 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f30567fc640 10 client.5904 handle_command_reply: tid=0 2024-04-25T00:33:24.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 client.5904 shutdown 2024-04-25T00:33:24.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 -- 172.21.15.29:0/375898562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0837ef0 msgr2=0x55b4d081d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 --2- 172.21.15.29:0/375898562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0837ef0 0x55b4d081d460 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3060014460 tx=0x7f3060004740 comp rx=0 tx=0).stop 2024-04-25T00:33:24.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 -- 172.21.15.29:0/375898562 shutdown_connections 2024-04-25T00:33:24.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 --2- 172.21.15.29:0/375898562 >> [v2:172.21.15.29:6833/628132809,v1:172.21.15.29:6835/628132809] conn(0x55b4d0847f90 0x55b4d084a380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 --2- 172.21.15.29:0/375898562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0838810 0x55b4d081d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 --2- 172.21.15.29:0/375898562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0837ef0 0x55b4d081d460 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f30567fc640 0 client.5904 ms_handle_reset on v2:172.21.15.29:6833/628132809 2024-04-25T00:33:24.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 --2- 172.21.15.29:0/375898562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d07eebc0 0x55b4d081cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 -- 172.21.15.29:0/375898562 >> 172.21.15.29:0/375898562 conn(0x55b4d07eab70 msgr2=0x55b4d07ed7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:24.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 -- 172.21.15.29:0/375898562 shutdown_connections 2024-04-25T00:33:24.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 1 -- 172.21.15.29:0/375898562 wait complete. 2024-04-25T00:33:24.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.002+0000 7f3080e92000 20 client.5904 trim_cache size 0 max 16384 2024-04-25T00:33:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 Processor -- start 2024-04-25T00:33:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 -- start start 2024-04-25T00:33:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d07eebc0 0x55b4d08a3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 0x55b4d08a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0838810 0x55b4d08a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3078200730 con 0x55b4d0838810 2024-04-25T00:33:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f30781ffa40 con 0x55b4d0837ef0 2024-04-25T00:33:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3078200060 con 0x55b4d07eebc0 2024-04-25T00:33:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3077fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0838810 0x55b4d08a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3077fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0838810 0x55b4d08a3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43506/0 (socket says 172.21.15.29:43506) 2024-04-25T00:33:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3077fff640 1 -- 172.21.15.29:0/3213937896 learned_addr learned my addr 172.21.15.29:0/3213937896 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3077fff640 1 -- 172.21.15.29:0/3213937896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d07eebc0 msgr2=0x55b4d08a3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3077fff640 1 --2- 172.21.15.29:0/3213937896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d07eebc0 0x55b4d08a3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3077fff640 1 -- 172.21.15.29:0/3213937896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 msgr2=0x55b4d08a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3077fff640 1 --2- 172.21.15.29:0/3213937896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 0x55b4d08a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3077fff640 1 -- 172.21.15.29:0/3213937896 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4d05c4e80 con 0x55b4d0838810 2024-04-25T00:33:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3077fff640 1 --2- 172.21.15.29:0/3213937896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0838810 0x55b4d08a3f80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3070009c90 tx=0x7f30700a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1a539cec664562 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f306cff9640 1 -- 172.21.15.29:0/3213937896 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30700bd070 con 0x55b4d0838810 2024-04-25T00:33:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f306cff9640 1 -- 172.21.15.29:0/3213937896 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30700090d0 con 0x55b4d0838810 2024-04-25T00:33:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f306cff9640 1 -- 172.21.15.29:0/3213937896 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30700ba040 con 0x55b4d0838810 2024-04-25T00:33:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 -- 172.21.15.29:0/3213937896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0838810 msgr2=0x55b4d08a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 --2- 172.21.15.29:0/3213937896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0838810 0x55b4d08a3f80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3070009c90 tx=0x7f30700a7000 comp rx=0 tx=0).stop 2024-04-25T00:33:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 -- 172.21.15.29:0/3213937896 shutdown_connections 2024-04-25T00:33:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 --2- 172.21.15.29:0/3213937896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d0838810 0x55b4d08a3f80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 --2- 172.21.15.29:0/3213937896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 0x55b4d08a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 --2- 172.21.15.29:0/3213937896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d07eebc0 0x55b4d08a3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 -- 172.21.15.29:0/3213937896 >> 172.21.15.29:0/3213937896 conn(0x55b4d07eab70 msgr2=0x55b4d08454e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:24.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 -- 172.21.15.29:0/3213937896 shutdown_connections 2024-04-25T00:33:24.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.022+0000 7f3080e92000 1 -- 172.21.15.29:0/3213937896 wait complete. 2024-04-25T00:33:24.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 -- start start 2024-04-25T00:33:24.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 Processor -- start 2024-04-25T00:33:24.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 0x55b4d08a7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 0x55b4d08a8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d0838810 0x55b4d084a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b4d06eee80 con 0x55b4d07eebc0 2024-04-25T00:33:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b4d0605810 con 0x55b4d0837ef0 2024-04-25T00:33:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55b4d063b4d0 con 0x55b4d0838810 2024-04-25T00:33:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 10 client.0 _ll_register_callbacks cb 0x7f306000b620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:24.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30777fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 0x55b4d08a7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30777fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 0x55b4d08a7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43516/0 (socket says 172.21.15.29:43516) 2024-04-25T00:33:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30777fe640 1 -- 172.21.15.29:0/188607818 learned_addr learned my addr 172.21.15.29:0/188607818 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30777fe640 1 -- 172.21.15.29:0/188607818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d0838810 msgr2=0x55b4d084a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30777fe640 1 --2- 172.21.15.29:0/188607818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d0838810 0x55b4d084a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30777fe640 1 -- 172.21.15.29:0/188607818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 msgr2=0x55b4d08a8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30777fe640 1 --2- 172.21.15.29:0/188607818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 0x55b4d08a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30777fe640 1 -- 172.21.15.29:0/188607818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4d0609760 con 0x55b4d07eebc0 2024-04-25T00:33:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30777fe640 1 --2- 172.21.15.29:0/188607818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 0x55b4d08a7dd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f306800b0e0 tx=0x7f30680561d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204f30d75ce2c796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30567fc640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:33:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30567fc640 1 -- 172.21.15.29:0/188607818 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068082030 con 0x55b4d07eebc0 2024-04-25T00:33:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30567fc640 1 -- 172.21.15.29:0/188607818 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f30680842c0 con 0x55b4d07eebc0 2024-04-25T00:33:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 -- 172.21.15.29:0/188607818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30700090d0 con 0x55b4d07eebc0 2024-04-25T00:33:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30567fc640 1 -- 172.21.15.29:0/188607818 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f306805f1b0 con 0x55b4d07eebc0 2024-04-25T00:33:24.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 10 client.5910 fetch_fsmap learned FSMap version 24 2024-04-25T00:33:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 -- 172.21.15.29:0/188607818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55b4d06959e0 con 0x55b4d07eebc0 2024-04-25T00:33:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30567fc640 1 -- 172.21.15.29:0/188607818 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30700090d0 con 0x55b4d07eebc0 2024-04-25T00:33:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30567fc640 1 -- 172.21.15.29:0/188607818 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7f3068013070 con 0x55b4d07eebc0 2024-04-25T00:33:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 10 client.5910 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:33:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 10 client.5910 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:33:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 20 client.5910 populate_metadata read hostname 'smithi029' 2024-04-25T00:33:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 --2- 172.21.15.29:0/188607818 >> [v2:172.21.15.29:6833/628132809,v1:172.21.15.29:6835/628132809] conn(0x55b4d08aeca0 0x55b4d084b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 4 client.5910 mds_command: new command op to 5733 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:33:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 -- 172.21.15.29:0/188607818 --> [v2:172.21.15.29:6833/628132809,v1:172.21.15.29:6835/628132809] -- command(tid 0: {"prefix": "status"}) -- 0x55b4d0649b20 con 0x55b4d08aeca0 2024-04-25T00:33:24.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3076ffd640 1 --2- 172.21.15.29:0/188607818 >> [v2:172.21.15.29:6833/628132809,v1:172.21.15.29:6835/628132809] conn(0x55b4d08aeca0 0x55b4d084b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3076ffd640 1 --2- 172.21.15.29:0/188607818 >> [v2:172.21.15.29:6833/628132809,v1:172.21.15.29:6835/628132809] conn(0x55b4d08aeca0 0x55b4d084b480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dc6af0b27039726 server_cookie=c0e787653d39288e in_seq=0 out_seq=0 2024-04-25T00:33:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30567fc640 10 client.5910 ms_handle_connect on v2:172.21.15.29:6833/628132809 2024-04-25T00:33:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30567fc640 1 -- 172.21.15.29:0/188607818 <== mds.0 v2:172.21.15.29:6833/628132809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b4d0649b20 con 0x55b4d08aeca0 2024-04-25T00:33:24.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30567fc640 10 client.5910 handle_command_reply: tid=0 2024-04-25T00:33:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 client.5910 shutdown 2024-04-25T00:33:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 -- 172.21.15.29:0/188607818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 msgr2=0x55b4d08a7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 --2- 172.21.15.29:0/188607818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 0x55b4d08a7dd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f306800b0e0 tx=0x7f30680561d0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 -- 172.21.15.29:0/188607818 shutdown_connections 2024-04-25T00:33:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 --2- 172.21.15.29:0/188607818 >> [v2:172.21.15.29:6833/628132809,v1:172.21.15.29:6835/628132809] conn(0x55b4d08aeca0 0x55b4d084b480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 --2- 172.21.15.29:0/188607818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b4d0838810 0x55b4d084a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f30567fc640 0 client.5910 ms_handle_reset on v2:172.21.15.29:6833/628132809 2024-04-25T00:33:24.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 --2- 172.21.15.29:0/188607818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4d0837ef0 0x55b4d08a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 --2- 172.21.15.29:0/188607818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b4d07eebc0 0x55b4d08a7dd0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 -- 172.21.15.29:0/188607818 >> 172.21.15.29:0/188607818 conn(0x55b4d07eab70 msgr2=0x55b4d0836800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:24.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 -- 172.21.15.29:0/188607818 shutdown_connections 2024-04-25T00:33:24.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 1 -- 172.21.15.29:0/188607818 wait complete. 2024-04-25T00:33:24.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.026+0000 7f3080e92000 20 client.5910 trim_cache size 0 max 16384 2024-04-25T00:33:24.049 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:33:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "59afbe7d-aaa8-45e2-b468-69ee150cacea", 2024-04-25T00:33:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-04-25T00:33:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 5733, 2024-04-25T00:33:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-04-25T00:33:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-04-25T00:33:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-04-25T00:33:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 10.246327587, 2024-04-25T00:33:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 24, 2024-04-25T00:33:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 45, 2024-04-25T00:33:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 45, 2024-04-25T00:33:24.049 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 16.274710245000001 2024-04-25T00:33:24.050 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:33:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.030+0000 7f307dd19640 1 -- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3058067a70 msgr2=0x7f3058069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.030+0000 7f307dd19640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3058067a70 0x7f3058069f30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f307815f270 tx=0x7f3060008040 comp rx=0 tx=0).stop 2024-04-25T00:33:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.030+0000 7f307dd19640 1 -- 172.21.15.29:0/3180579651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30780cf540 msgr2=0x7f30781535b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.030+0000 7f307dd19640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30780cf540 0x7f30781535b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f306800d620 tx=0x7f306800dae0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.030+0000 7f307dd19640 1 -- 172.21.15.29:0/3180579651 shutdown_connections 2024-04-25T00:33:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.030+0000 7f307dd19640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307815c0e0 0x7f30781555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.030+0000 7f307dd19640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3058067a70 0x7f3058069f30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.030+0000 7f307dd19640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30780d0010 0x7f30780b7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.030+0000 7f307dd19640 1 --2- 172.21.15.29:0/3180579651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30780cf540 0x7f30781535b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.030+0000 7f307dd19640 1 -- 172.21.15.29:0/3180579651 >> 172.21.15.29:0/3180579651 conn(0x7f30780b3a00 msgr2=0x7f3078066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.030+0000 7f307dd19640 1 -- 172.21.15.29:0/3180579651 shutdown_connections 2024-04-25T00:33:24.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.030+0000 7f307dd19640 1 -- 172.21.15.29:0/3180579651 wait complete. 2024-04-25T00:33:24.052 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:33:24.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f9989319640 1 Processor -- start 2024-04-25T00:33:24.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f9989319640 1 -- start start 2024-04-25T00:33:24.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f9989319640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9984162420 0x7f9984070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f9989319640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f99840b3280 0x7f998406cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f9989319640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9984071910 con 0x7f99840b4e00 2024-04-25T00:33:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f9989319640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9984071bf0 con 0x7f99840b3280 2024-04-25T00:33:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f9989319640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9984071ed0 con 0x7f9984162420 2024-04-25T00:33:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f99837fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f99840b3280 0x7f998406cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f99827fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f99840b4e00 0x7f99840b2b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59052/0 (socket says 172.21.15.29:59052) 2024-04-25T00:33:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f99827fc640 1 -- 172.21.15.29:0/1433350253 learned_addr learned my addr 172.21.15.29:0/1433350253 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f9982ffd640 1 --2- 172.21.15.29:0/1433350253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9984162420 0x7f9984070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f99837fe640 1 -- 172.21.15.29:0/1433350253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9984162420 msgr2=0x7f9984070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f99837fe640 1 --2- 172.21.15.29:0/1433350253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9984162420 0x7f9984070f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f99837fe640 1 -- 172.21.15.29:0/1433350253 >> v1:172.21.15.29:6789/0 conn(0x7f99840b4e00 legacy=0x7f99840b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f9981ffb640 1 -- 172.21.15.29:0/1433350253 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1521257221 0 0) 0x7f9984071910 con 0x7f99840b4e00 2024-04-25T00:33:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.162+0000 7f99837fe640 1 -- 172.21.15.29:0/1433350253 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99840721b0 con 0x7f99840b3280 2024-04-25T00:33:24.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f99837fe640 1 --2- 172.21.15.29:0/1433350253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840b3280 0x7f998406cf80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9974009d90 tx=0x7f997401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6d8f0fbde14ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:24.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9981ffb640 1 -- 172.21.15.29:0/1433350253 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99740b2070 con 0x7f99840b3280 2024-04-25T00:33:24.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9981ffb640 1 -- 172.21.15.29:0/1433350253 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99740a5940 con 0x7f99840b3280 2024-04-25T00:33:24.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9981ffb640 1 -- 172.21.15.29:0/1433350253 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99840716d0 con 0x7f99840b3280 2024-04-25T00:33:24.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 -- 172.21.15.29:0/1433350253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840b3280 msgr2=0x7f998406cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 --2- 172.21.15.29:0/1433350253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840b3280 0x7f998406cf80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9974009d90 tx=0x7f997401c970 comp rx=0 tx=0).stop 2024-04-25T00:33:24.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 -- 172.21.15.29:0/1433350253 shutdown_connections 2024-04-25T00:33:24.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 --2- 172.21.15.29:0/1433350253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840b3280 0x7f998406cf80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 --2- 172.21.15.29:0/1433350253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9984162420 0x7f9984070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 -- 172.21.15.29:0/1433350253 >> 172.21.15.29:0/1433350253 conn(0x7f99840b1870 msgr2=0x7f99840b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:24.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 -- 172.21.15.29:0/1433350253 shutdown_connections 2024-04-25T00:33:24.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 -- 172.21.15.29:0/1433350253 wait complete. 2024-04-25T00:33:24.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 Processor -- start 2024-04-25T00:33:24.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 -- start start 2024-04-25T00:33:24.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99840b3280 0x7f9984160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840b4e00 0x7f9984167280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9984162420 0x7f99841677c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f998406e0e0 con 0x7f99840b4e00 2024-04-25T00:33:24.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f998406df60 con 0x7f99840b3280 2024-04-25T00:33:24.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9989319640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f998406e260 con 0x7f9984162420 2024-04-25T00:33:24.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f99827fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840b4e00 0x7f9984167280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f99827fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840b4e00 0x7f9984167280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43548/0 (socket says 172.21.15.29:43548) 2024-04-25T00:33:24.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f99827fc640 1 -- 172.21.15.29:0/483584862 learned_addr learned my addr 172.21.15.29:0/483584862 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:24.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f99837fe640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9984162420 0x7f99841677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f9982ffd640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99840b3280 0x7f9984160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f99827fc640 1 -- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9984162420 msgr2=0x7f99841677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f99827fc640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9984162420 0x7f99841677c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f99827fc640 1 -- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99840b3280 msgr2=0x7f9984160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f99827fc640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99840b3280 0x7f9984160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.166+0000 7f99827fc640 1 -- 172.21.15.29:0/483584862 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9984167ff0 con 0x7f99840b4e00 2024-04-25T00:33:24.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.170+0000 7f99827fc640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840b4e00 0x7f9984167280 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f996c00c4b0 tx=0x7f996c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:24.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.170+0000 7f997b7fe640 1 -- 172.21.15.29:0/483584862 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c018070 con 0x7f99840b4e00 2024-04-25T00:33:24.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.170+0000 7f997b7fe640 1 -- 172.21.15.29:0/483584862 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996c007500 con 0x7f99840b4e00 2024-04-25T00:33:24.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.170+0000 7f9989319640 1 -- 172.21.15.29:0/483584862 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9984168ec0 con 0x7f99840b4e00 2024-04-25T00:33:24.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.170+0000 7f997b7fe640 1 -- 172.21.15.29:0/483584862 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c004a30 con 0x7f99840b4e00 2024-04-25T00:33:24.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.170+0000 7f9989319640 1 -- 172.21.15.29:0/483584862 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9984167d00 con 0x7f99840b4e00 2024-04-25T00:33:24.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.170+0000 7f9989319640 1 -- 172.21.15.29:0/483584862 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9984072fa0 con 0x7f99840b4e00 2024-04-25T00:33:24.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.170+0000 7f997b7fe640 1 -- 172.21.15.29:0/483584862 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f996c01b070 con 0x7f99840b4e00 2024-04-25T00:33:24.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.170+0000 7f997b7fe640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9964067a70 0x7f9964069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:24.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.170+0000 7f997b7fe640 1 -- 172.21.15.29:0/483584862 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f996c0855c0 con 0x7f99840b4e00 2024-04-25T00:33:24.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.170+0000 7f9982ffd640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9964067a70 0x7f9964069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:24.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.170+0000 7f9982ffd640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9964067a70 0x7f9964069f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f997c00b7c0 tx=0x7f997c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:24.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.174+0000 7f997b7fe640 1 -- 172.21.15.29:0/483584862 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f996c056020 con 0x7f99840b4e00 2024-04-25T00:33:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.370+0000 7f9989319640 1 -- 172.21.15.29:0/483584862 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f99841650f0 con 0x7f99840b4e00 2024-04-25T00:33:24.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.370+0000 7f997b7fe640 1 -- 172.21.15.29:0/483584862 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f996c055920 con 0x7f99840b4e00 2024-04-25T00:33:24.377 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:33:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.374+0000 7f9989319640 1 -- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9964067a70 msgr2=0x7f9964069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.374+0000 7f9989319640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9964067a70 0x7f9964069f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f997c00b7c0 tx=0x7f997c008040 comp rx=0 tx=0).stop 2024-04-25T00:33:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.374+0000 7f9989319640 1 -- 172.21.15.29:0/483584862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840b4e00 msgr2=0x7f9984167280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.374+0000 7f9989319640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840b4e00 0x7f9984167280 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f996c00c4b0 tx=0x7f996c00c970 comp rx=0 tx=0).stop 2024-04-25T00:33:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.374+0000 7f9989319640 1 -- 172.21.15.29:0/483584862 shutdown_connections 2024-04-25T00:33:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.374+0000 7f9989319640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9984162420 0x7f99841677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.374+0000 7f9989319640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99840b4e00 0x7f9984167280 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.374+0000 7f9989319640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9964067a70 0x7f9964069f30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.374+0000 7f9989319640 1 --2- 172.21.15.29:0/483584862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99840b3280 0x7f9984160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.374+0000 7f9989319640 1 -- 172.21.15.29:0/483584862 >> 172.21.15.29:0/483584862 conn(0x7f99840b1870 msgr2=0x7f9984165610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.374+0000 7f9989319640 1 -- 172.21.15.29:0/483584862 shutdown_connections 2024-04-25T00:33:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24.374+0000 7f9989319640 1 -- 172.21.15.29:0/483584862 wait complete. 2024-04-25T00:33:24.397 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:33:24.397 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:33:24.397 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-25T00:33:24.397 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:33:24.397 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:33:24.398 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:33:24.398 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:33:24.398 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:33:24.398 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:33:24.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:24.399 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-25T00:33:24.405 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:33:24.405 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:33:24.406 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:33:24.406 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:24.406 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:33:24.406 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:24.406 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:33:24.407 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:33:24.407 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-25T00:33:24.407 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:33:24.407 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:33:24.407 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-25T00:33:24.407 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:33:24.407 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:33:24.407 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-25T00:33:24.407 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:33:24.407 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42355sec preferred_lft 42355sec 2024-04-25T00:33:24.407 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 02:42:28:a5:36:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:33:24.408 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:24.409 INFO:teuthology.orchestra.run.smithi029.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:33:24.409 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:33:24.409 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:33:24.409 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:24.409 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:33:24.409 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:33:24.409 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:33:24.410 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:24.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:24.410 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:33:24.459 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-25T00:33:24.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:24.461 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-25T00:33:24.511 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-25T00:33:24.511 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:33:24.511 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T00:33:24.511 INFO:teuthology.orchestra.run.smithi029.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:33:24.511 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T00:33:24.511 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:33:24.511 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:33:24.511 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:33:24.512 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:33:24.512 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:33:24.512 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:33:24.512 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:33:24.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:33:24.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:33:24.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:24.638 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:33:24.643 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:33:24.644 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:33:24.644 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:33:24.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:24.644 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-04-25T00:33:24.701 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:33:24.701 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:33:24.701 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:33:24.749 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:33:24.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:33:24.750 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:33:24.798 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:33:24.856 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:33:24.903 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:33:24.903 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:33:24.903 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:33:24.903 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:33:24.903 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:33:24.903 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:33:24.903 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:33:24.903 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:33:24.904 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:33:24.904 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:33:24.904 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:33:24.904 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:33:24.904 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:33:24.904 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:33:24.904 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:33:24.904 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:33:24.904 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:33:24.905 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:33:24.905 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:33:24.905 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:33:24.905 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:33:24.905 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:33:24.905 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:33:24.905 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:33:24.905 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:33:24.905 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:33:24.905 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:33:24.905 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:33:24.906 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:33:24.906 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:33:24.906 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:33:24.906 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:33:24.906 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:33:24.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:24.907 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:33:24.955 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:33:24.955 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:33:24.999 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:33:25.014 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:33:25.063 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:33:25.063 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:33:25.063 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:33:25.063 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:33:25.063 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:33:25.063 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:33:25.063 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:33:25.063 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:33:25.063 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:33:25.063 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:33:25.063 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:33:25.063 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:33:25.064 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:33:25.065 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:33:25.065 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:33:25.065 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:33:25.065 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:33:25.065 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:33:25.065 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:33:25.065 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:33:25.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:25.069 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:33:25.081 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:2024-04-25T00:33:25.074+0000 7f3268eb9580 -1 init, newargv = 0x557b1496d4b0 newargc=15 2024-04-25T00:33:25.081 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[39466]: starting ceph client 2024-04-25T00:33:25.087 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[39466]: starting fuse 2024-04-25T00:33:26.188 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:33:26.205 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:33:26.256 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:33:26.256 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:33:26.256 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:33:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:33:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:33:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:33:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:33:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:33:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:33:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:33:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:33:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:33:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:33:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:33:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:33:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:33:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:33:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:33:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:33:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:33:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:33:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:33:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:33:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:33:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:33:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:33:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:33:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:33:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:33:26.259 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:33:26.259 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:33:26.259 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:33:26.259 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:33:26.259 INFO:teuthology.orchestra.run.smithi029.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:33:26.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:26.260 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:33:26.306 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:33:26.306 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:33:26.307 DEBUG:teuthology.orchestra.run.smithi029:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:33:26.307 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-04-25T00:33:26.307 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-04-25T00:33:26.307 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-04-25T00:33:26.307 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-04-25T00:33:26.307 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:33:26.307 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-04-25T00:33:26.307 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:33:26.307 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-04-25T00:33:26.307 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:33:26.307 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:33:26.307 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.0")) 2024-04-25T00:33:26.308 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-04-25T00:33:26.433 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.0.39466.asok 2024-04-25T00:33:26.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:33:26.446 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39466.asok 2024-04-25T00:33:26.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:26.446 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39466.asok status 2024-04-25T00:33:26.605 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:33:26.605 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-04-25T00:33:26.605 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:33:26.606 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:33:26.606 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "0", 2024-04-25T00:33:26.606 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-04-25T00:33:26.606 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:33:26.606 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "39466", 2024-04-25T00:33:26.606 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-04-25T00:33:26.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:26.606 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-04-25T00:33:26.606 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-04-25T00:33:26.606 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 5925, 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 5925 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1329756079 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:33:26.607 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1329756079 2024-04-25T00:33:26.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:33:26.608 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.5925 192.168.0.1:0/1329756079", 2024-04-25T00:33:26.608 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.1:0/1329756079", 2024-04-25T00:33:26.608 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-04-25T00:33:26.608 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 24, 2024-04-25T00:33:26.608 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 45, 2024-04-25T00:33:26.608 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:33:26.608 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-04-25T00:33:26.608 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-04-25T00:33:26.608 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:33:26.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:26.620 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-04-25T00:33:26.620 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:33:26.627 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-04-25T00:33:26.627 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:33:26.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:26.627 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:33:26.683 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:33:26.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42f7792640 1 Processor -- start 2024-04-25T00:33:26.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42f7792640 1 -- start start 2024-04-25T00:33:26.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42f7792640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f42f00b3c30 0x7f42f006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:26.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42f7792640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42f00b41e0 con 0x7f42f00b4b10 2024-04-25T00:33:26.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42f7792640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42f006d960 con 0x7f42f00b3140 2024-04-25T00:33:26.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42f7792640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f42f006dc40 con 0x7f42f00b3c30 2024-04-25T00:33:26.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42f4d2d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f42f00b3c30 0x7f42f006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:26.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42f4d2d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f42f00b3c30 0x7f42f006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43562/0 (socket says 172.21.15.29:43562) 2024-04-25T00:33:26.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42f4d2d640 1 -- 172.21.15.29:0/2205752031 learned_addr learned my addr 172.21.15.29:0/2205752031 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:26.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42f5d2f640 1 --1- 172.21.15.29:0/2205752031 >> v1:172.21.15.29:6789/0 conn(0x7f42f00b4b10 0x7f42f00ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59066/0 (socket says 172.21.15.29:59066) 2024-04-25T00:33:26.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42f4d2d640 1 -- 172.21.15.29:0/2205752031 >> v1:172.21.15.157:6789/0 conn(0x7f42f00b3140 legacy=0x7f42f00b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:26.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42f4d2d640 1 -- 172.21.15.29:0/2205752031 >> v1:172.21.15.29:6789/0 conn(0x7f42f00b4b10 legacy=0x7f42f00ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:26.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42f4d2d640 1 -- 172.21.15.29:0/2205752031 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f006df20 con 0x7f42f00b3c30 2024-04-25T00:33:26.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.846+0000 7f42e3fff640 1 -- 172.21.15.29:0/2205752031 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4130493665 0 0) 0x7f42f00b41e0 con 0x7f42f00b4b10 2024-04-25T00:33:26.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f4d2d640 1 --2- 172.21.15.29:0/2205752031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42f00b3c30 0x7f42f006e440 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f42e40098a0 tx=0x7f42e401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03c795e23999542 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:26.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42e3fff640 1 -- 172.21.15.29:0/2205752031 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e40b0070 con 0x7f42f00b3c30 2024-04-25T00:33:26.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42e3fff640 1 -- 172.21.15.29:0/2205752031 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e40ab600 con 0x7f42f00b3c30 2024-04-25T00:33:26.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42e3fff640 1 -- 172.21.15.29:0/2205752031 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e40ab9d0 con 0x7f42f00b3c30 2024-04-25T00:33:26.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 -- 172.21.15.29:0/2205752031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42f00b3c30 msgr2=0x7f42f006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:26.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 --2- 172.21.15.29:0/2205752031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42f00b3c30 0x7f42f006e440 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f42e40098a0 tx=0x7f42e401c540 comp rx=0 tx=0).stop 2024-04-25T00:33:26.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 -- 172.21.15.29:0/2205752031 shutdown_connections 2024-04-25T00:33:26.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 --2- 172.21.15.29:0/2205752031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42f00b3c30 0x7f42f006e440 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:26.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 -- 172.21.15.29:0/2205752031 >> 172.21.15.29:0/2205752031 conn(0x7f42f00b1870 msgr2=0x7f42f00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:26.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 -- 172.21.15.29:0/2205752031 shutdown_connections 2024-04-25T00:33:26.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 -- 172.21.15.29:0/2205752031 wait complete. 2024-04-25T00:33:26.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 Processor -- start 2024-04-25T00:33:26.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 -- start start 2024-04-25T00:33:26.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42f00b3140 0x7f42f0153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:26.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42f00b3c30 0x7f42f013cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:26.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42f00b4b10 0x7f42f013e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:26.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f42f0169d90 con 0x7f42f00b3c30 2024-04-25T00:33:26.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f42f0169c10 con 0x7f42f00b4b10 2024-04-25T00:33:26.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f7792640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f42f0169f10 con 0x7f42f00b3140 2024-04-25T00:33:26.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f4d2d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42f00b3c30 0x7f42f013cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:26.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f4d2d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42f00b3c30 0x7f42f013cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43578/0 (socket says 172.21.15.29:43578) 2024-04-25T00:33:26.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f4d2d640 1 -- 172.21.15.29:0/3815792107 learned_addr learned my addr 172.21.15.29:0/3815792107 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:26.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f5d2f640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42f00b4b10 0x7f42f013e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:26.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f4d2d640 1 -- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42f00b3140 msgr2=0x7f42f0153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:26.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f4d2d640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42f00b3140 0x7f42f0153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:26.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f4d2d640 1 -- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42f00b4b10 msgr2=0x7f42f013e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:26.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f4d2d640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42f00b4b10 0x7f42f013e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:26.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f4d2d640 1 -- 172.21.15.29:0/3815792107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f0070e80 con 0x7f42f00b3c30 2024-04-25T00:33:26.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.850+0000 7f42f5d2f640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42f00b4b10 0x7f42f013e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:26.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.854+0000 7f42f4d2d640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42f00b3c30 0x7f42f013cc70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f42e40098a0 tx=0x7f42e401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:26.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.854+0000 7f42e1ffb640 1 -- 172.21.15.29:0/3815792107 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e40b0040 con 0x7f42f00b3c30 2024-04-25T00:33:26.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.854+0000 7f42e1ffb640 1 -- 172.21.15.29:0/3815792107 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e4004020 con 0x7f42f00b3c30 2024-04-25T00:33:26.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.854+0000 7f42f7792640 1 -- 172.21.15.29:0/3815792107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f0071160 con 0x7f42f00b3c30 2024-04-25T00:33:26.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.854+0000 7f42e1ffb640 1 -- 172.21.15.29:0/3815792107 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e40043d0 con 0x7f42f00b3c30 2024-04-25T00:33:26.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.854+0000 7f42f7792640 1 -- 172.21.15.29:0/3815792107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f0071720 con 0x7f42f00b3c30 2024-04-25T00:33:26.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.854+0000 7f42f7792640 1 -- 172.21.15.29:0/3815792107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f0069270 con 0x7f42f00b3c30 2024-04-25T00:33:26.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.854+0000 7f42e1ffb640 1 -- 172.21.15.29:0/3815792107 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f42e4003620 con 0x7f42f00b3c30 2024-04-25T00:33:26.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.854+0000 7f42e1ffb640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42d4067a70 0x7f42d4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:26.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.854+0000 7f42e1ffb640 1 -- 172.21.15.29:0/3815792107 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f42e411ff90 con 0x7f42f00b3c30 2024-04-25T00:33:26.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.854+0000 7f42f552e640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42d4067a70 0x7f42d4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:26.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.854+0000 7f42f552e640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42d4067a70 0x7f42d4069f30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42ec000f70 tx=0x7f42ec0079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:26.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:26.858+0000 7f42e1ffb640 1 -- 172.21.15.29:0/3815792107 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f42e40c1030 con 0x7f42f00b3c30 2024-04-25T00:33:27.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.054+0000 7f42f7792640 1 -- 172.21.15.29:0/3815792107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f42f0144130 con 0x7f42f00b3c30 2024-04-25T00:33:27.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.054+0000 7f42e1ffb640 1 -- 172.21.15.29:0/3815792107 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f42e40b8030 con 0x7f42f00b3c30 2024-04-25T00:33:27.061 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:33:27.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.058+0000 7f42f7792640 1 -- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42d4067a70 msgr2=0x7f42d4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:27.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.058+0000 7f42f7792640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42d4067a70 0x7f42d4069f30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42ec000f70 tx=0x7f42ec0079c0 comp rx=0 tx=0).stop 2024-04-25T00:33:27.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.058+0000 7f42f7792640 1 -- 172.21.15.29:0/3815792107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42f00b3c30 msgr2=0x7f42f013cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:27.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.058+0000 7f42f7792640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42f00b3c30 0x7f42f013cc70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f42e40098a0 tx=0x7f42e401ad80 comp rx=0 tx=0).stop 2024-04-25T00:33:27.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.058+0000 7f42f7792640 1 -- 172.21.15.29:0/3815792107 shutdown_connections 2024-04-25T00:33:27.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.058+0000 7f42f7792640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42d4067a70 0x7f42d4069f30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:27.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.058+0000 7f42f7792640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42f00b4b10 0x7f42f013e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:27.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.058+0000 7f42f7792640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42f00b3c30 0x7f42f013cc70 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:27.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.058+0000 7f42f7792640 1 --2- 172.21.15.29:0/3815792107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42f00b3140 0x7f42f0153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:27.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.058+0000 7f42f7792640 1 -- 172.21.15.29:0/3815792107 >> 172.21.15.29:0/3815792107 conn(0x7f42f00b1870 msgr2=0x7f42f0142d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:27.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.058+0000 7f42f7792640 1 -- 172.21.15.29:0/3815792107 shutdown_connections 2024-04-25T00:33:27.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:27.058+0000 7f42f7792640 1 -- 172.21.15.29:0/3815792107 wait complete. 2024-04-25T00:33:27.078 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:33:27.078 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:33:27.078 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-25T00:33:27.078 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:33:27.078 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:33:27.078 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:33:27.079 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:33:27.079 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:33:27.079 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:33:27.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:27.079 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-25T00:33:27.086 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:33:27.086 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:33:27.086 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:33:27.086 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:27.086 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:33:27.086 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:27.086 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42348sec preferred_lft 42348sec 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:33:27.087 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:9c:8d:4b:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:33:27.088 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:33:27.088 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:27.088 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:33:27.088 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:33:27.088 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:33:27.088 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:27.088 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:33:27.088 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:27.088 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:33:27.088 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:33:27.088 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:33:27.088 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:27.088 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:33:27.089 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:33:27.089 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:33:27.089 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:33:27.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:27.089 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:33:27.144 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:33:27.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:27.146 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-25T00:33:27.198 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-25T00:33:27.198 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:33:27.198 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:33:27.198 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:33:27.198 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:33:27.199 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:33:27.199 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:33:27.199 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:33:27.199 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:33:27.199 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:33:27.199 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:33:27.199 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:33:27.199 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:33:27.199 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:33:27.199 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:33:27.199 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:33:27.200 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:33:27.289 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:33:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:33:27.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:33:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:33:27.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:27.326 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:33:27.331 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:33:27.331 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:33:27.331 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:33:27.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:27.332 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-25T00:33:27.382 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T00:33:27.382 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:33:27.382 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:33:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:33:27.429 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:33:27.429 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:33:27.477 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:33:27.535 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:33:27.586 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:33:27.586 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:33:27.586 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:33:27.586 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:33:27.586 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:33:27.587 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:33:27.588 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:33:27.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:27.589 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:33:27.641 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:33:27.641 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T00:33:27.686 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:33:27.701 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:33:27.753 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:33:27.753 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:33:27.753 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:33:27.754 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:33:27.754 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:33:27.754 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:33:27.754 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:33:27.754 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:33:27.754 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:33:27.754 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:33:27.754 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:33:27.754 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:33:27.754 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:33:27.754 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:33:27.755 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:33:27.755 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:33:27.755 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:33:27.755 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:33:27.755 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:33:27.755 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:33:27.755 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:33:27.756 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:33:27.756 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:33:27.756 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:33:27.756 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:33:27.756 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:33:27.756 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:33:27.757 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:33:27.757 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:33:27.757 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:33:27.757 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:33:27.757 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:33:27.757 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:33:27.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:27.759 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:33:27.763 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-04-25T00:33:27.758+0000 7f2ce3f7c580 -1 init, newargv = 0x557cb60cb030 newargc=15 2024-04-25T00:33:27.763 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[33858]: starting ceph client 2024-04-25T00:33:27.770 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[33858]: starting fuse 2024-04-25T00:33:28.869 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:33:28.886 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:33:28.936 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:33:28.936 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:33:28.936 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:33:28.936 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:33:28.936 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:33:28.936 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:33:28.936 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:33:28.936 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:33:28.936 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:33:28.936 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:33:28.936 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:33:28.937 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:33:28.938 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:33:28.938 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:33:28.938 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:33:28.938 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:33:28.938 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:33:28.938 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:33:28.938 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:33:28.938 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:33:28.938 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:33:28.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:28.939 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:33:28.985 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:33:28.986 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:33:28.986 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:33:28.986 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-25T00:33:28.986 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-25T00:33:28.986 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-25T00:33:28.986 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-25T00:33:28.986 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-25T00:33:28.987 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-25T00:33:28.988 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-25T00:33:28.988 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:33:28.988 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:33:28.988 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-04-25T00:33:28.988 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-25T00:33:29.110 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.33858.asok 2024-04-25T00:33:29.113 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:33:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:33:29.122 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33858.asok 2024-04-25T00:33:29.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:29.122 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33858.asok status 2024-04-25T00:33:29.274 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "33858", 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 5147, 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-25T00:33:29.275 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 5147 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3260141677 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3260141677 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.5147 192.168.0.1:0/3260141677", 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/3260141677", 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-25T00:33:29.276 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 24, 2024-04-25T00:33:29.277 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 45, 2024-04-25T00:33:29.277 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:33:29.277 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-25T00:33:29.277 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-25T00:33:29.277 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:33:29.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:29.288 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-25T00:33:29.289 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T00:33:29.295 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-25T00:33:29.295 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T00:33:29.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:29.295 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:33:29.347 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-25T00:33:29.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.482+0000 7f1393bc2640 1 Processor -- start 2024-04-25T00:33:29.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.482+0000 7f1393bc2640 1 -- start start 2024-04-25T00:33:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f1393bc2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f138c071590 0x7f138c06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f1393bc2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f138c0ce240 0x7f138c0ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f1393bc2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f138c06d640 con 0x7f138c0b5200 2024-04-25T00:33:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f1393bc2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f138c06efe0 con 0x7f138c0ce240 2024-04-25T00:33:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f1393bc2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f138c06d900 con 0x7f138c071590 2024-04-25T00:33:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f139215f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f138c0ce240 0x7f138c0ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:29.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f139215f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f138c0ce240 0x7f138c0ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37534/0 (socket says 172.21.15.29:37534) 2024-04-25T00:33:29.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f139215f640 1 -- 172.21.15.29:0/1162746966 learned_addr learned my addr 172.21.15.29:0/1162746966 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:29.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f139195e640 1 --2- 172.21.15.29:0/1162746966 >> v2:172.21.15.157:3300/0 conn(0x7f138c071590 0x7f138c06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:29.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f139215f640 1 -- 172.21.15.29:0/1162746966 >> v2:172.21.15.157:3300/0 conn(0x7f138c071590 msgr2=0x7f138c06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:29.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f139215f640 1 --2- 172.21.15.29:0/1162746966 >> v2:172.21.15.157:3300/0 conn(0x7f138c071590 0x7f138c06e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:29.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f139215f640 1 -- 172.21.15.29:0/1162746966 >> v1:172.21.15.157:6789/0 conn(0x7f138c0b5200 legacy=0x7f138c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:29.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f139215f640 1 -- 172.21.15.29:0/1162746966 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c06dbe0 con 0x7f138c0ce240 2024-04-25T00:33:29.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f139215f640 1 --2- 172.21.15.29:0/1162746966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f138c0ce240 0x7f138c0ce690 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f138c0b0110 tx=0x7f138401d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdac2b8e8e11e626 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:29.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f139095c640 1 -- 172.21.15.29:0/1162746966 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13840a5860 con 0x7f138c0ce240 2024-04-25T00:33:29.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f139095c640 1 -- 172.21.15.29:0/1162746966 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13840a5a00 con 0x7f138c0ce240 2024-04-25T00:33:29.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.486+0000 7f139095c640 1 -- 172.21.15.29:0/1162746966 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13840a5dd0 con 0x7f138c0ce240 2024-04-25T00:33:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 -- 172.21.15.29:0/1162746966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f138c0ce240 msgr2=0x7f138c0ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 --2- 172.21.15.29:0/1162746966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f138c0ce240 0x7f138c0ce690 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f138c0b0110 tx=0x7f138401d450 comp rx=0 tx=0).stop 2024-04-25T00:33:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 -- 172.21.15.29:0/1162746966 shutdown_connections 2024-04-25T00:33:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 --2- 172.21.15.29:0/1162746966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f138c0ce240 0x7f138c0ce690 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 --2- 172.21.15.29:0/1162746966 >> v2:172.21.15.157:3300/0 conn(0x7f138c071590 0x7f138c06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 -- 172.21.15.29:0/1162746966 >> 172.21.15.29:0/1162746966 conn(0x7f138c0b21d0 msgr2=0x7f138c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 -- 172.21.15.29:0/1162746966 shutdown_connections 2024-04-25T00:33:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 -- 172.21.15.29:0/1162746966 wait complete. 2024-04-25T00:33:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 Processor -- start 2024-04-25T00:33:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 -- start start 2024-04-25T00:33:29.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f138c071590 0x7f138c140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:29.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f138c0b5200 0x7f138c0b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:29.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f138c0ce240 0x7f138c0b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:29.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f138c06ed80 con 0x7f138c0ce240 2024-04-25T00:33:29.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f138c06ec00 con 0x7f138c071590 2024-04-25T00:33:29.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f138c0b5610 con 0x7f138c0b5200 2024-04-25T00:33:29.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f139215f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f138c0ce240 0x7f138c0b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:29.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f139215f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f138c0ce240 0x7f138c0b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37536/0 (socket says 172.21.15.29:37536) 2024-04-25T00:33:29.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f139215f640 1 -- 172.21.15.29:0/734644310 learned_addr learned my addr 172.21.15.29:0/734644310 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:29.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f139195e640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f138c071590 0x7f138c140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:29.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f139115d640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f138c0b5200 0x7f138c0b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:29.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f139215f640 1 -- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f138c0b5200 msgr2=0x7f138c0b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:29.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f139215f640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f138c0b5200 0x7f138c0b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:29.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f139215f640 1 -- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f138c071590 msgr2=0x7f138c140ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:29.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f139215f640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f138c071590 0x7f138c140ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:29.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f139215f640 1 -- 172.21.15.29:0/734644310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c141220 con 0x7f138c0ce240 2024-04-25T00:33:29.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f139215f640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f138c0ce240 0x7f138c0b1660 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f13840af040 tx=0x7f1384019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:29.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f13827fc640 1 -- 172.21.15.29:0/734644310 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13840b8680 con 0x7f138c0ce240 2024-04-25T00:33:29.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f13827fc640 1 -- 172.21.15.29:0/734644310 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13840b8820 con 0x7f138c0ce240 2024-04-25T00:33:29.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 -- 172.21.15.29:0/734644310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138c1568b0 con 0x7f138c0ce240 2024-04-25T00:33:29.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f13827fc640 1 -- 172.21.15.29:0/734644310 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13840b8c10 con 0x7f138c0ce240 2024-04-25T00:33:29.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.490+0000 7f1393bc2640 1 -- 172.21.15.29:0/734644310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138c156e70 con 0x7f138c0ce240 2024-04-25T00:33:29.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.494+0000 7f1393bc2640 1 -- 172.21.15.29:0/734644310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138c0692e0 con 0x7f138c0ce240 2024-04-25T00:33:29.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.494+0000 7f13827fc640 1 -- 172.21.15.29:0/734644310 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f13840c0020 con 0x7f138c0ce240 2024-04-25T00:33:29.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.494+0000 7f13827fc640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1370067a70 0x7f1370069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:29.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.494+0000 7f13827fc640 1 -- 172.21.15.29:0/734644310 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f13841208b0 con 0x7f138c0ce240 2024-04-25T00:33:29.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.494+0000 7f139195e640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1370067a70 0x7f1370069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:29.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.494+0000 7f139195e640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1370067a70 0x7f1370069f30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f137c00b9d0 tx=0x7f137c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:29.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.498+0000 7f13827fc640 1 -- 172.21.15.29:0/734644310 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f13840c7020 con 0x7f138c0ce240 2024-04-25T00:33:29.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:29.698+0000 7f1393bc2640 1 -- 172.21.15.29:0/734644310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f138c0724e0 con 0x7f138c0ce240 2024-04-25T00:33:30.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.562+0000 7f13827fc640 1 -- 172.21.15.29:0/734644310 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v25) ==== 136+0+0 (secure 0 0 0) 0x7f13840165e0 con 0x7f138c0ce240 2024-04-25T00:33:30.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.566+0000 7f1393bc2640 1 -- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1370067a70 msgr2=0x7f1370069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:30.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.566+0000 7f1393bc2640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1370067a70 0x7f1370069f30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f137c00b9d0 tx=0x7f137c008040 comp rx=0 tx=0).stop 2024-04-25T00:33:30.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.566+0000 7f1393bc2640 1 -- 172.21.15.29:0/734644310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f138c0ce240 msgr2=0x7f138c0b1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:30.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.566+0000 7f1393bc2640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f138c0ce240 0x7f138c0b1660 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f13840af040 tx=0x7f1384019790 comp rx=0 tx=0).stop 2024-04-25T00:33:30.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.566+0000 7f1393bc2640 1 -- 172.21.15.29:0/734644310 shutdown_connections 2024-04-25T00:33:30.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.566+0000 7f1393bc2640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f138c0ce240 0x7f138c0b1660 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.566+0000 7f1393bc2640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f138c0b5200 0x7f138c0b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.566+0000 7f1393bc2640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1370067a70 0x7f1370069f30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.566+0000 7f1393bc2640 1 --2- 172.21.15.29:0/734644310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f138c071590 0x7f138c140ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.566+0000 7f1393bc2640 1 -- 172.21.15.29:0/734644310 >> 172.21.15.29:0/734644310 conn(0x7f138c0b21d0 msgr2=0x7f138c0b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:30.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.566+0000 7f1393bc2640 1 -- 172.21.15.29:0/734644310 shutdown_connections 2024-04-25T00:33:30.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.566+0000 7f1393bc2640 1 -- 172.21.15.29:0/734644310 wait complete. 2024-04-25T00:33:30.593 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-25T00:33:30.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.714+0000 7f71889e5640 1 Processor -- start 2024-04-25T00:33:30.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.714+0000 7f71889e5640 1 -- start start 2024-04-25T00:33:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.714+0000 7f71889e5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f718415e200 0x7f71840d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.714+0000 7f71889e5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f71840cf690 0x7f71840d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.714+0000 7f71889e5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71840d2ea0 con 0x7f71840d2a90 2024-04-25T00:33:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.714+0000 7f71889e5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f71840b38d0 con 0x7f71840cf690 2024-04-25T00:33:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.714+0000 7f71889e5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f71840cfc10 con 0x7f718415e200 2024-04-25T00:33:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.714+0000 7f7179d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f71840cf690 0x7f71840d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.714+0000 7f7179d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f71840cf690 0x7f71840d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37548/0 (socket says 172.21.15.29:37548) 2024-04-25T00:33:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.714+0000 7f7179d9b640 1 -- 172.21.15.29:0/1632886826 learned_addr learned my addr 172.21.15.29:0/1632886826 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f718259c640 1 --2- 172.21.15.29:0/1632886826 >> v2:172.21.15.157:3300/0 conn(0x7f718415e200 0x7f71840d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f7179d9b640 1 -- 172.21.15.29:0/1632886826 >> v2:172.21.15.157:3300/0 conn(0x7f718415e200 msgr2=0x7f71840d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f7179d9b640 1 --2- 172.21.15.29:0/1632886826 >> v2:172.21.15.157:3300/0 conn(0x7f718415e200 0x7f71840d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f7179d9b640 1 -- 172.21.15.29:0/1632886826 >> v1:172.21.15.157:6789/0 conn(0x7f71840d2a90 legacy=0x7f71840d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:30.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f7179d9b640 1 -- 172.21.15.29:0/1632886826 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71840cfef0 con 0x7f71840cf690 2024-04-25T00:33:30.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f7179d9b640 1 --2- 172.21.15.29:0/1632886826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71840cf690 0x7f71840d2550 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f718408d8f0 tx=0x7f716c01c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da16b500a944b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:30.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f7181d9b640 1 -- 172.21.15.29:0/1632886826 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c0b0070 con 0x7f71840cf690 2024-04-25T00:33:30.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f7181d9b640 1 -- 172.21.15.29:0/1632886826 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716c0ab600 con 0x7f71840cf690 2024-04-25T00:33:30.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f7181d9b640 1 -- 172.21.15.29:0/1632886826 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c0ab9d0 con 0x7f71840cf690 2024-04-25T00:33:30.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f71889e5640 1 -- 172.21.15.29:0/1632886826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71840cf690 msgr2=0x7f71840d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:30.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f71889e5640 1 --2- 172.21.15.29:0/1632886826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71840cf690 0x7f71840d2550 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f718408d8f0 tx=0x7f716c01c2d0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f71889e5640 1 -- 172.21.15.29:0/1632886826 shutdown_connections 2024-04-25T00:33:30.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f71889e5640 1 --2- 172.21.15.29:0/1632886826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71840cf690 0x7f71840d2550 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f71889e5640 1 --2- 172.21.15.29:0/1632886826 >> v2:172.21.15.157:3300/0 conn(0x7f718415e200 0x7f71840d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f71889e5640 1 -- 172.21.15.29:0/1632886826 >> 172.21.15.29:0/1632886826 conn(0x7f718409d280 msgr2=0x7f71840d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:30.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f71889e5640 1 -- 172.21.15.29:0/1632886826 shutdown_connections 2024-04-25T00:33:30.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.718+0000 7f71889e5640 1 -- 172.21.15.29:0/1632886826 wait complete. 2024-04-25T00:33:30.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f71889e5640 1 Processor -- start 2024-04-25T00:33:30.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f71889e5640 1 -- start start 2024-04-25T00:33:30.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f71889e5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71840cf690 0x7f71840ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:30.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f71889e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71840d2a90 0x7f7184155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:30.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f71889e5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f718415e200 0x7f718415f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:30.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f71889e5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f718415e760 con 0x7f718415e200 2024-04-25T00:33:30.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f71889e5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f718415e5e0 con 0x7f71840d2a90 2024-04-25T00:33:30.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f71889e5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f718416ae30 con 0x7f71840cf690 2024-04-25T00:33:30.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f7182d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f718415e200 0x7f718415f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:30.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f7182d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f718415e200 0x7f718415f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37550/0 (socket says 172.21.15.29:37550) 2024-04-25T00:33:30.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f7182d9d640 1 -- 172.21.15.29:0/2016911 learned_addr learned my addr 172.21.15.29:0/2016911 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:30.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f718259c640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71840cf690 0x7f71840ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:30.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f7179d9b640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71840d2a90 0x7f7184155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:30.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f7182d9d640 1 -- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71840cf690 msgr2=0x7f71840ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:30.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f7182d9d640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71840cf690 0x7f71840ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f7182d9d640 1 -- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71840d2a90 msgr2=0x7f7184155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:30.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f7182d9d640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71840d2a90 0x7f7184155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f7182d9d640 1 -- 172.21.15.29:0/2016911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71841602a0 con 0x7f718415e200 2024-04-25T00:33:30.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f7182d9d640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f718415e200 0x7f718415f930 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f717c00c780 tx=0x7f717c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:30.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f717b7fe640 1 -- 172.21.15.29:0/2016911 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c004b70 con 0x7f718415e200 2024-04-25T00:33:30.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f717b7fe640 1 -- 172.21.15.29:0/2016911 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f717c004d10 con 0x7f718415e200 2024-04-25T00:33:30.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f71889e5640 1 -- 172.21.15.29:0/2016911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7184160580 con 0x7f718415e200 2024-04-25T00:33:30.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f717b7fe640 1 -- 172.21.15.29:0/2016911 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c00aa90 con 0x7f718415e200 2024-04-25T00:33:30.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f71889e5640 1 -- 172.21.15.29:0/2016911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7184160b40 con 0x7f718415e200 2024-04-25T00:33:30.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f71889e5640 1 -- 172.21.15.29:0/2016911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7184069270 con 0x7f718415e200 2024-04-25T00:33:30.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.722+0000 7f717b7fe640 1 -- 172.21.15.29:0/2016911 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f717c00ac30 con 0x7f718415e200 2024-04-25T00:33:30.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.726+0000 7f717b7fe640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7164067aa0 0x7f7164069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:30.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.726+0000 7f717b7fe640 1 -- 172.21.15.29:0/2016911 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f717c086910 con 0x7f718415e200 2024-04-25T00:33:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.726+0000 7f718259c640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7164067aa0 0x7f7164069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.726+0000 7f718259c640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7164067aa0 0x7f7164069f60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f717400b9d0 tx=0x7f7174008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:30.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.726+0000 7f717b7fe640 1 -- 172.21.15.29:0/2016911 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f717c020070 con 0x7f718415e200 2024-04-25T00:33:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.914+0000 7f71889e5640 1 -- 172.21.15.29:0/2016911 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f71840b5d20 con 0x7f7164067aa0 2024-04-25T00:33:30.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.918+0000 7f717b7fe640 1 -- 172.21.15.29:0/2016911 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f71840b5d20 con 0x7f7164067aa0 2024-04-25T00:33:30.921 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:33:30.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:33:30.921 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-04-25T00:33:30.921 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:33:30.921 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:33:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.918+0000 7f71889e5640 1 -- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7164067aa0 msgr2=0x7f7164069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.918+0000 7f71889e5640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7164067aa0 0x7f7164069f60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f717400b9d0 tx=0x7f7174008040 comp rx=0 tx=0).stop 2024-04-25T00:33:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.918+0000 7f71889e5640 1 -- 172.21.15.29:0/2016911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f718415e200 msgr2=0x7f718415f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.918+0000 7f71889e5640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f718415e200 0x7f718415f930 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f717c00c780 tx=0x7f717c00cc40 comp rx=0 tx=0).stop 2024-04-25T00:33:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.918+0000 7f71889e5640 1 -- 172.21.15.29:0/2016911 shutdown_connections 2024-04-25T00:33:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.918+0000 7f71889e5640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f718415e200 0x7f718415f930 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.918+0000 7f71889e5640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7164067aa0 0x7f7164069f60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.918+0000 7f71889e5640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71840d2a90 0x7f7184155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.918+0000 7f71889e5640 1 --2- 172.21.15.29:0/2016911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71840cf690 0x7f71840ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.918+0000 7f71889e5640 1 -- 172.21.15.29:0/2016911 >> 172.21.15.29:0/2016911 conn(0x7f718409d280 msgr2=0x7f71840b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.918+0000 7f71889e5640 1 -- 172.21.15.29:0/2016911 shutdown_connections 2024-04-25T00:33:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:30.918+0000 7f71889e5640 1 -- 172.21.15.29:0/2016911 wait complete. 2024-04-25T00:33:30.941 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:33:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f68733f9640 1 Processor -- start 2024-04-25T00:33:31.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f68733f9640 1 -- start start 2024-04-25T00:33:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f68733f9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0ce880 0x7f686c0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f68733f9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f686c0b40c0 0x7f686c1602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f68733f9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f686c0cef10 con 0x7f686c15c160 2024-04-25T00:33:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f68733f9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f686c0cf7b0 con 0x7f686c0b40c0 2024-04-25T00:33:31.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f68733f9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f686c15fe40 con 0x7f686c0ce880 2024-04-25T00:33:31.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6871195640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f686c15c160 0x7f686c0cd000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59610/0 (socket says 172.21.15.29:59610) 2024-04-25T00:33:31.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6871195640 1 -- 172.21.15.29:0/2928232049 learned_addr learned my addr 172.21.15.29:0/2928232049 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:31.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6870994640 1 --2- 172.21.15.29:0/2928232049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0ce880 0x7f686c0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6871996640 1 --2- 172.21.15.29:0/2928232049 >> v2:172.21.15.157:3300/0 conn(0x7f686c0b40c0 0x7f686c1602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1219599240 0 0) 0x7f686c0cef10 con 0x7f686c15c160 2024-04-25T00:33:31.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6858003530 con 0x7f686c15c160 2024-04-25T00:33:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3923674220 0 0) 0x7f6858003530 con 0x7f686c15c160 2024-04-25T00:33:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0ce880 msgr2=0x7f686c0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 --2- 172.21.15.29:0/2928232049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0ce880 0x7f686c0b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 >> v2:172.21.15.157:3300/0 conn(0x7f686c0b40c0 msgr2=0x7f686c1602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 --2- 172.21.15.29:0/2928232049 >> v2:172.21.15.157:3300/0 conn(0x7f686c0b40c0 0x7f686c1602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c0cf1f0 con 0x7f686c15c160 2024-04-25T00:33:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f6868001680 con 0x7f686c15c160 2024-04-25T00:33:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 >> v1:172.21.15.29:6789/0 conn(0x7f686c15c160 legacy=0x7f686c0cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 --2- 172.21.15.29:0/2928232049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68580039c0 0x7f6858003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 --2- 172.21.15.29:0/2928232049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6858004810 0x7f6858004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 --2- 172.21.15.29:0/2928232049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0ce880 0x7f68580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f686c0cf7b0 con 0x7f6858004810 2024-04-25T00:33:31.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f686c15fe40 con 0x7f68580039c0 2024-04-25T00:33:31.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f685800ba20 con 0x7f686c0ce880 2024-04-25T00:33:31.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6870994640 1 --2- 172.21.15.29:0/2928232049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6858004810 0x7f6858004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6871195640 1 --2- 172.21.15.29:0/2928232049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68580039c0 0x7f6858003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6871996640 1 --2- 172.21.15.29:0/2928232049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0ce880 0x7f68580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6870994640 1 -- 172.21.15.29:0/2928232049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0ce880 msgr2=0x7f68580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:31.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6870994640 1 --2- 172.21.15.29:0/2928232049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0ce880 0x7f68580051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6870994640 1 -- 172.21.15.29:0/2928232049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68580039c0 msgr2=0x7f6858003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:31.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6870994640 1 --2- 172.21.15.29:0/2928232049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68580039c0 0x7f6858003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6870994640 1 -- 172.21.15.29:0/2928232049 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6858003530 con 0x7f6858004810 2024-04-25T00:33:31.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.062+0000 7f6871996640 1 --2- 172.21.15.29:0/2928232049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0ce880 0x7f68580051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:31.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.066+0000 7f6870994640 1 --2- 172.21.15.29:0/2928232049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6858004810 0x7f6858004c90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f685c009b10 tx=0x7f685c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3fbea75300a8d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:31.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.066+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c0096b0 con 0x7f6858004810 2024-04-25T00:33:31.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.066+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685c0a9470 con 0x7f6858004810 2024-04-25T00:33:31.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.066+0000 7f6863fff640 1 -- 172.21.15.29:0/2928232049 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c0a9860 con 0x7f6858004810 2024-04-25T00:33:31.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.066+0000 7f68733f9640 1 -- 172.21.15.29:0/2928232049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6858004810 msgr2=0x7f6858004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.066+0000 7f68733f9640 1 --2- 172.21.15.29:0/2928232049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6858004810 0x7f6858004c90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f685c009b10 tx=0x7f685c01ca70 comp rx=0 tx=0).stop 2024-04-25T00:33:31.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.066+0000 7f6871996640 1 -- 172.21.15.29:0/2928232049 reap_dead start 2024-04-25T00:33:31.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.066+0000 7f68733f9640 1 -- 172.21.15.29:0/2928232049 shutdown_connections 2024-04-25T00:33:31.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.066+0000 7f68733f9640 1 -- 172.21.15.29:0/2928232049 >> 172.21.15.29:0/2928232049 conn(0x7f686c0d0cb0 msgr2=0x7f686c0d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:31.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.066+0000 7f68733f9640 1 -- 172.21.15.29:0/2928232049 shutdown_connections 2024-04-25T00:33:31.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.066+0000 7f68733f9640 1 -- 172.21.15.29:0/2928232049 wait complete. 2024-04-25T00:33:31.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.066+0000 7f68733f9640 1 Processor -- start 2024-04-25T00:33:31.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.066+0000 7f68733f9640 1 -- start start 2024-04-25T00:33:31.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f68733f9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6858004810 0x7f686c0b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f68733f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686c071df0 0x7f686c1f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f68733f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b4ee0 0x7f686c1f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f68733f9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f686c167010 con 0x7f686c0b4ee0 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f68733f9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f686c15c540 con 0x7f686c071df0 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f68733f9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f686c167190 con 0x7f6858004810 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6871996640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b4ee0 0x7f686c1f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6871996640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b4ee0 0x7f686c1f8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37566/0 (socket says 172.21.15.29:37566) 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6871996640 1 -- 172.21.15.29:0/3344126110 learned_addr learned my addr 172.21.15.29:0/3344126110 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6871195640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6858004810 0x7f686c0b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6870994640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686c071df0 0x7f686c1f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6871996640 1 -- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6858004810 msgr2=0x7f686c0b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6871996640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6858004810 0x7f686c0b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6871996640 1 -- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686c071df0 msgr2=0x7f686c1f8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6871996640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686c071df0 0x7f686c1f8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6871996640 1 -- 172.21.15.29:0/3344126110 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c1620e0 con 0x7f686c0b4ee0 2024-04-25T00:33:31.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6871996640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b4ee0 0x7f686c1f8fc0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f686c15fbe0 tx=0x7f686400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:31.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6861ffb640 1 -- 172.21.15.29:0/3344126110 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686401c070 con 0x7f686c0b4ee0 2024-04-25T00:33:31.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6861ffb640 1 -- 172.21.15.29:0/3344126110 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6864017400 con 0x7f686c0b4ee0 2024-04-25T00:33:31.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f68733f9640 1 -- 172.21.15.29:0/3344126110 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686c1623c0 con 0x7f686c0b4ee0 2024-04-25T00:33:31.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6861ffb640 1 -- 172.21.15.29:0/3344126110 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68640177d0 con 0x7f686c0b4ee0 2024-04-25T00:33:31.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f68733f9640 1 -- 172.21.15.29:0/3344126110 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686c162980 con 0x7f686c0b4ee0 2024-04-25T00:33:31.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f68733f9640 1 -- 172.21.15.29:0/3344126110 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f686c15cb10 con 0x7f686c0b4ee0 2024-04-25T00:33:31.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6861ffb640 1 -- 172.21.15.29:0/3344126110 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f686401f070 con 0x7f686c0b4ee0 2024-04-25T00:33:31.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6861ffb640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6850067a70 0x7f6850069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6861ffb640 1 -- 172.21.15.29:0/3344126110 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f68640862c0 con 0x7f686c0b4ee0 2024-04-25T00:33:31.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.070+0000 7f6871195640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6850067a70 0x7f6850069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.074+0000 7f6871195640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6850067a70 0x7f6850069f30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6868001ec0 tx=0x7f6868000f60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:31.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.078+0000 7f6861ffb640 1 -- 172.21.15.29:0/3344126110 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f686c15cb10 con 0x7f686c0b4ee0 2024-04-25T00:33:31.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.242+0000 7f68733f9640 1 -- 172.21.15.29:0/3344126110 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f686c069270 con 0x7f686c0b4ee0 2024-04-25T00:33:31.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.242+0000 7f6861ffb640 1 -- 172.21.15.29:0/3344126110 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f6864012070 con 0x7f686c0b4ee0 2024-04-25T00:33:31.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.246+0000 7f68733f9640 1 -- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6850067a70 msgr2=0x7f6850069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.246+0000 7f68733f9640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6850067a70 0x7f6850069f30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6868001ec0 tx=0x7f6868000f60 comp rx=0 tx=0).stop 2024-04-25T00:33:31.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.246+0000 7f68733f9640 1 -- 172.21.15.29:0/3344126110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b4ee0 msgr2=0x7f686c1f8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.246+0000 7f68733f9640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b4ee0 0x7f686c1f8fc0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f686c15fbe0 tx=0x7f686400f4a0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.246+0000 7f68733f9640 1 -- 172.21.15.29:0/3344126110 shutdown_connections 2024-04-25T00:33:31.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.246+0000 7f68733f9640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b4ee0 0x7f686c1f8fc0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.246+0000 7f68733f9640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6850067a70 0x7f6850069f30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.246+0000 7f68733f9640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686c071df0 0x7f686c1f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.246+0000 7f68733f9640 1 --2- 172.21.15.29:0/3344126110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6858004810 0x7f686c0b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.246+0000 7f68733f9640 1 -- 172.21.15.29:0/3344126110 >> 172.21.15.29:0/3344126110 conn(0x7f686c0d0cb0 msgr2=0x7f686c15b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:31.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.246+0000 7f68733f9640 1 -- 172.21.15.29:0/3344126110 shutdown_connections 2024-04-25T00:33:31.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.246+0000 7f68733f9640 1 -- 172.21.15.29:0/3344126110 wait complete. 2024-04-25T00:33:31.267 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:33:31.268 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:33:31.273 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:33:31.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.438+0000 7f8b7fbda640 1 Processor -- start 2024-04-25T00:33:31.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.438+0000 7f8b7fbda640 1 -- start start 2024-04-25T00:33:31.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7fbda640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7815f3c0 0x7f8b780cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7fbda640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b780b3720 con 0x7f8b7815e8d0 2024-04-25T00:33:31.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7fbda640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b780b39c0 con 0x7f8b780b4780 2024-04-25T00:33:31.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7fbda640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b780b42f0 con 0x7f8b7815f3c0 2024-04-25T00:33:31.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7d976640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8b7815e8d0 0x7f8b7815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59612/0 (socket says 172.21.15.29:59612) 2024-04-25T00:33:31.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7d976640 1 -- 172.21.15.29:0/2289697011 learned_addr learned my addr 172.21.15.29:0/2289697011 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:31.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7e177640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7815f3c0 0x7f8b780cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 286429065 0 0) 0x7f8b780b3720 con 0x7f8b7815e8d0 2024-04-25T00:33:31.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b64003530 con 0x7f8b7815e8d0 2024-04-25T00:33:31.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2934934771 0 0) 0x7f8b64003530 con 0x7f8b7815e8d0 2024-04-25T00:33:31.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7815f3c0 msgr2=0x7f8b780cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:31.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7815f3c0 0x7f8b780cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 >> v1:172.21.15.157:6789/0 conn(0x7f8b780b4780 legacy=0x7f8b7815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b780b3ca0 con 0x7f8b7815e8d0 2024-04-25T00:33:31.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8b68001680 con 0x7f8b7815e8d0 2024-04-25T00:33:31.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 >> v1:172.21.15.29:6789/0 conn(0x7f8b7815e8d0 legacy=0x7f8b7815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b640039f0 0x7f8b64003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b64004810 0x7f8b64004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7815f3c0 0x7f8b640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8b780b42f0 con 0x7f8b640039f0 2024-04-25T00:33:31.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6400dc90 con 0x7f8b64004810 2024-04-25T00:33:31.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6400de90 con 0x7f8b7815f3c0 2024-04-25T00:33:31.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7d976640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b640039f0 0x7f8b64003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7e177640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7815f3c0 0x7f8b640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7d976640 1 -- 172.21.15.29:0/2289697011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7815f3c0 msgr2=0x7f8b640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:31.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7d976640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7815f3c0 0x7f8b640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7d175640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b64004810 0x7f8b64004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7d976640 1 -- 172.21.15.29:0/2289697011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b64004810 msgr2=0x7f8b64004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:31.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7d976640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b64004810 0x7f8b64004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7d976640 1 -- 172.21.15.29:0/2289697011 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b64003530 con 0x7f8b640039f0 2024-04-25T00:33:31.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7e177640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b7815f3c0 0x7f8b640051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:31.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7d175640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b64004810 0x7f8b64004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:31.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.442+0000 7f8b7d976640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b640039f0 0x7f8b64003e10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8b68001ec0 tx=0x7f8b6801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7282337dba24d31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:31.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68007d40 con 0x7f8b640039f0 2024-04-25T00:33:31.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b680a9070 con 0x7f8b640039f0 2024-04-25T00:33:31.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7c974640 1 -- 172.21.15.29:0/2289697011 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68005d40 con 0x7f8b640039f0 2024-04-25T00:33:31.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 -- 172.21.15.29:0/2289697011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b640039f0 msgr2=0x7f8b64003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 --2- 172.21.15.29:0/2289697011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b640039f0 0x7f8b64003e10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8b68001ec0 tx=0x7f8b6801cc10 comp rx=0 tx=0).stop 2024-04-25T00:33:31.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7e177640 1 -- 172.21.15.29:0/2289697011 reap_dead start 2024-04-25T00:33:31.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 -- 172.21.15.29:0/2289697011 shutdown_connections 2024-04-25T00:33:31.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 -- 172.21.15.29:0/2289697011 >> 172.21.15.29:0/2289697011 conn(0x7f8b780b1870 msgr2=0x7f8b780b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:31.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 -- 172.21.15.29:0/2289697011 shutdown_connections 2024-04-25T00:33:31.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 -- 172.21.15.29:0/2289697011 wait complete. 2024-04-25T00:33:31.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 Processor -- start 2024-04-25T00:33:31.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 -- start start 2024-04-25T00:33:31.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b640039f0 0x7f8b78155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b7813bee0 0x7f8b78071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b780723a0 0x7f8b781570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7806d710 con 0x7f8b7813bee0 2024-04-25T00:33:31.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7806d590 con 0x7f8b640039f0 2024-04-25T00:33:31.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7806d890 con 0x7f8b780723a0 2024-04-25T00:33:31.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7d175640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b7813bee0 0x7f8b78071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7d175640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b7813bee0 0x7f8b78071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37574/0 (socket says 172.21.15.29:37574) 2024-04-25T00:33:31.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7d175640 1 -- 172.21.15.29:0/3141917898 learned_addr learned my addr 172.21.15.29:0/3141917898 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7e177640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b780723a0 0x7f8b781570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7d976640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b640039f0 0x7f8b78155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7d175640 1 -- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b780723a0 msgr2=0x7f8b781570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7d175640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b780723a0 0x7f8b781570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7d175640 1 -- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b640039f0 msgr2=0x7f8b78155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7d175640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b640039f0 0x7f8b78155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7d175640 1 -- 172.21.15.29:0/3141917898 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7813d090 con 0x7f8b7813bee0 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7d976640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b640039f0 0x7f8b78155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7d175640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b7813bee0 0x7f8b78071e60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400d4f0 tx=0x7f8b7400d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b6e7fc640 1 -- 172.21.15.29:0/3141917898 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74005480 con 0x7f8b7813bee0 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b6e7fc640 1 -- 172.21.15.29:0/3141917898 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b74017070 con 0x7f8b7813bee0 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 -- 172.21.15.29:0/3141917898 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7813c480 con 0x7f8b7813bee0 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b6e7fc640 1 -- 172.21.15.29:0/3141917898 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74003840 con 0x7f8b7813bee0 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.446+0000 7f8b7fbda640 1 -- 172.21.15.29:0/3141917898 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b7813ca40 con 0x7f8b7813bee0 2024-04-25T00:33:31.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.450+0000 7f8b7fbda640 1 -- 172.21.15.29:0/3141917898 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b78069270 con 0x7f8b7813bee0 2024-04-25T00:33:31.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.450+0000 7f8b6e7fc640 1 -- 172.21.15.29:0/3141917898 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f8b740039e0 con 0x7f8b7813bee0 2024-04-25T00:33:31.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.450+0000 7f8b6e7fc640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b5c067aa0 0x7f8b5c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.450+0000 7f8b6e7fc640 1 -- 172.21.15.29:0/3141917898 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f8b74085d20 con 0x7f8b7813bee0 2024-04-25T00:33:31.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.450+0000 7f8b7d976640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b5c067aa0 0x7f8b5c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.450+0000 7f8b7d976640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b5c067aa0 0x7f8b5c069f60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8b680ab000 tx=0x7f8b680a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:31.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.454+0000 7f8b6e7fc640 1 -- 172.21.15.29:0/3141917898 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8b7400f2f0 con 0x7f8b7813bee0 2024-04-25T00:33:31.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.614+0000 7f8b7fbda640 1 -- 172.21.15.29:0/3141917898 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b7815e5f0 con 0x7f8b7813bee0 2024-04-25T00:33:31.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.618+0000 7f8b6e7fc640 1 -- 172.21.15.29:0/3141917898 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16565 (secure 0 0 0) 0x7f8b74056560 con 0x7f8b7813bee0 2024-04-25T00:33:31.622 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:31.622 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":45,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:33:14.784398+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:33:10.789862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:33:12.129841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:33:31.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.618+0000 7f8b7fbda640 1 -- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b5c067aa0 msgr2=0x7f8b5c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.618+0000 7f8b7fbda640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b5c067aa0 0x7f8b5c069f60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8b680ab000 tx=0x7f8b680a7070 comp rx=0 tx=0).stop 2024-04-25T00:33:31.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.618+0000 7f8b7fbda640 1 -- 172.21.15.29:0/3141917898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b7813bee0 msgr2=0x7f8b78071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.618+0000 7f8b7fbda640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b7813bee0 0x7f8b78071e60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400d4f0 tx=0x7f8b7400d9b0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.618+0000 7f8b7fbda640 1 -- 172.21.15.29:0/3141917898 shutdown_connections 2024-04-25T00:33:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.618+0000 7f8b7fbda640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b780723a0 0x7f8b781570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.618+0000 7f8b7fbda640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b7813bee0 0x7f8b78071e60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.618+0000 7f8b7fbda640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b5c067aa0 0x7f8b5c069f60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.618+0000 7f8b7fbda640 1 --2- 172.21.15.29:0/3141917898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b640039f0 0x7f8b78155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.618+0000 7f8b7fbda640 1 -- 172.21.15.29:0/3141917898 >> 172.21.15.29:0/3141917898 conn(0x7f8b780b1870 msgr2=0x7f8b780b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.618+0000 7f8b7fbda640 1 -- 172.21.15.29:0/3141917898 shutdown_connections 2024-04-25T00:33:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.618+0000 7f8b7fbda640 1 -- 172.21.15.29:0/3141917898 wait complete. 2024-04-25T00:33:31.640 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:33:10.789862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:33:12.129841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:33:31.641 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:33:31.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8bb640640 1 Processor -- start 2024-04-25T00:33:31.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8bb640640 1 -- start start 2024-04-25T00:33:31.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8bb640640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc8b40b1440 0x7fc8b415e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8bb640640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8b40b1970 con 0x7fc8b40ce550 2024-04-25T00:33:31.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8bb640640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8b415eb10 con 0x7fc8b415e6c0 2024-04-25T00:33:31.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8bb640640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b40b0f80 con 0x7fc8b40b1440 2024-04-25T00:33:31.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8b93dc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc8b40ce550 0x7fc8b40ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59624/0 (socket says 172.21.15.29:59624) 2024-04-25T00:33:31.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8b93dc640 1 -- 172.21.15.29:0/1506480615 learned_addr learned my addr 172.21.15.29:0/1506480615 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:31.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 836089772 0 0) 0x7fc8b40b1970 con 0x7fc8b40ce550 2024-04-25T00:33:31.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8b8bdb640 1 --2- 172.21.15.29:0/1506480615 >> v2:172.21.15.157:3300/0 conn(0x7fc8b40b1440 0x7fc8b415e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8a0003530 con 0x7fc8b40ce550 2024-04-25T00:33:31.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1466244844 0 0) 0x7fc8a0003530 con 0x7fc8b40ce550 2024-04-25T00:33:31.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 >> v2:172.21.15.157:3300/0 conn(0x7fc8b40b1440 msgr2=0x7fc8b415e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:31.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 --2- 172.21.15.29:0/1506480615 >> v2:172.21.15.157:3300/0 conn(0x7fc8b40b1440 0x7fc8b415e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 >> v1:172.21.15.157:6789/0 conn(0x7fc8b415e6c0 legacy=0x7fc8b40ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b40b0670 con 0x7fc8b40ce550 2024-04-25T00:33:31.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fc8b0001680 con 0x7fc8b40ce550 2024-04-25T00:33:31.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 >> v1:172.21.15.29:6789/0 conn(0x7fc8b40ce550 legacy=0x7fc8b40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 --2- 172.21.15.29:0/1506480615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8a0003a10 0x7fc8a0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 --2- 172.21.15.29:0/1506480615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8a0004810 0x7fc8a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 --2- 172.21.15.29:0/1506480615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8a00051d0 0x7fc8a000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a000e2b0 con 0x7fc8a0003a10 2024-04-25T00:33:31.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a000e460 con 0x7fc8a00051d0 2024-04-25T00:33:31.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a000e660 con 0x7fc8a0004810 2024-04-25T00:33:31.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8b93dc640 1 --2- 172.21.15.29:0/1506480615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8a0003a10 0x7fc8a0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8b8bdb640 1 --2- 172.21.15.29:0/1506480615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8a0004810 0x7fc8a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8b9bdd640 1 --2- 172.21.15.29:0/1506480615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8a00051d0 0x7fc8a000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8b93dc640 1 -- 172.21.15.29:0/1506480615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8a0004810 msgr2=0x7fc8a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:31.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8b93dc640 1 --2- 172.21.15.29:0/1506480615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8a0004810 0x7fc8a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8b93dc640 1 -- 172.21.15.29:0/1506480615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8a00051d0 msgr2=0x7fc8a000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:31.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8b93dc640 1 --2- 172.21.15.29:0/1506480615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8a00051d0 0x7fc8a000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8b9bdd640 1 -- 172.21.15.29:0/1506480615 reap_dead start 2024-04-25T00:33:31.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.766+0000 7fc8b93dc640 1 -- 172.21.15.29:0/1506480615 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a0003530 con 0x7fc8a0003a10 2024-04-25T00:33:31.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.770+0000 7fc8b93dc640 1 --2- 172.21.15.29:0/1506480615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8a0003a10 0x7fc8a0003e30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000c110 tx=0x7fc8b001cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f669e4086a5012da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:31.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.770+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0007c50 con 0x7fc8a0003a10 2024-04-25T00:33:31.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.770+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b00a9070 con 0x7fc8a0003a10 2024-04-25T00:33:31.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.770+0000 7fc8abfff640 1 -- 172.21.15.29:0/1506480615 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0005d80 con 0x7fc8a0003a10 2024-04-25T00:33:31.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.770+0000 7fc8bb640640 1 -- 172.21.15.29:0/1506480615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8a0003a10 msgr2=0x7fc8a0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.770+0000 7fc8bb640640 1 --2- 172.21.15.29:0/1506480615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8a0003a10 0x7fc8a0003e30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000c110 tx=0x7fc8b001cbc0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.770+0000 7fc8bb640640 1 -- 172.21.15.29:0/1506480615 shutdown_connections 2024-04-25T00:33:31.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.770+0000 7fc8bb640640 1 --2- 172.21.15.29:0/1506480615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8a0003a10 0x7fc8a0003e30 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.770+0000 7fc8bb640640 1 -- 172.21.15.29:0/1506480615 >> 172.21.15.29:0/1506480615 conn(0x7fc8b4072930 msgr2=0x7fc8b4072d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:31.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.770+0000 7fc8bb640640 1 -- 172.21.15.29:0/1506480615 shutdown_connections 2024-04-25T00:33:31.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.770+0000 7fc8bb640640 1 -- 172.21.15.29:0/1506480615 wait complete. 2024-04-25T00:33:31.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.770+0000 7fc8bb640640 1 Processor -- start 2024-04-25T00:33:31.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.770+0000 7fc8bb640640 1 -- start start 2024-04-25T00:33:31.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8bb640640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8a0003a10 0x7fc8b40cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8bb640640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8b40c8030 0x7fc8b40cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8bb640640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b40cbb30 0x7fc8b41f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8bb640640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b415ef10 con 0x7fc8b40cbb30 2024-04-25T00:33:31.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8bb640640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b415ed90 con 0x7fc8b40c8030 2024-04-25T00:33:31.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8bb640640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b415f090 con 0x7fc8a0003a10 2024-04-25T00:33:31.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8b9bdd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b40cbb30 0x7fc8b41f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8b9bdd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b40cbb30 0x7fc8b41f6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37582/0 (socket says 172.21.15.29:37582) 2024-04-25T00:33:31.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8b9bdd640 1 -- 172.21.15.29:0/4073329526 learned_addr learned my addr 172.21.15.29:0/4073329526 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:31.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8b8bdb640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8b40c8030 0x7fc8b40cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8b93dc640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8a0003a10 0x7fc8b40cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8b9bdd640 1 -- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8a0003a10 msgr2=0x7fc8b40cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8b9bdd640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8a0003a10 0x7fc8b40cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8b9bdd640 1 -- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8b40c8030 msgr2=0x7fc8b40cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8b9bdd640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8b40c8030 0x7fc8b40cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8b9bdd640 1 -- 172.21.15.29:0/4073329526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b40cc040 con 0x7fc8b40cbb30 2024-04-25T00:33:31.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8b8bdb640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8b40c8030 0x7fc8b40cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:31.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8b9bdd640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b40cbb30 0x7fc8b41f6940 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc8b415eaa0 tx=0x7fc8ac00e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:31.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8a9ffb640 1 -- 172.21.15.29:0/4073329526 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac019070 con 0x7fc8b40cbb30 2024-04-25T00:33:31.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8a9ffb640 1 -- 172.21.15.29:0/4073329526 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8ac006e30 con 0x7fc8b40cbb30 2024-04-25T00:33:31.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8bb640640 1 -- 172.21.15.29:0/4073329526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b40cc320 con 0x7fc8b40cbb30 2024-04-25T00:33:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8a9ffb640 1 -- 172.21.15.29:0/4073329526 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac003ba0 con 0x7fc8b40cbb30 2024-04-25T00:33:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8bb640640 1 -- 172.21.15.29:0/4073329526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b40cc8e0 con 0x7fc8b40cbb30 2024-04-25T00:33:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8bb640640 1 -- 172.21.15.29:0/4073329526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b4069270 con 0x7fc8b40cbb30 2024-04-25T00:33:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8a9ffb640 1 -- 172.21.15.29:0/4073329526 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fc8ac01c070 con 0x7fc8b40cbb30 2024-04-25T00:33:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8a9ffb640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc898067a70 0x7fc898069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8a9ffb640 1 -- 172.21.15.29:0/4073329526 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc8ac0855d0 con 0x7fc8b40cbb30 2024-04-25T00:33:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.774+0000 7fc8b93dc640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc898067a70 0x7fc898069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.778+0000 7fc8b93dc640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc898067a70 0x7fc898069f30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00ab000 tx=0x7fc8b00b0040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:31.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.778+0000 7fc8a9ffb640 1 -- 172.21.15.29:0/4073329526 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc8ac055df0 con 0x7fc8b40cbb30 2024-04-25T00:33:31.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.934+0000 7fc8bb640640 1 -- 172.21.15.29:0/4073329526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8b40cf6d0 con 0x7fc8b40cbb30 2024-04-25T00:33:31.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.934+0000 7fc8a9ffb640 1 -- 172.21.15.29:0/4073329526 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7fc8ac0556f0 con 0x7fc8b40cbb30 2024-04-25T00:33:31.941 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:33:31.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.938+0000 7fc8bb640640 1 -- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc898067a70 msgr2=0x7fc898069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.938+0000 7fc8bb640640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc898067a70 0x7fc898069f30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00ab000 tx=0x7fc8b00b0040 comp rx=0 tx=0).stop 2024-04-25T00:33:31.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.938+0000 7fc8bb640640 1 -- 172.21.15.29:0/4073329526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b40cbb30 msgr2=0x7fc8b41f6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:31.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.938+0000 7fc8bb640640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b40cbb30 0x7fc8b41f6940 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc8b415eaa0 tx=0x7fc8ac00e960 comp rx=0 tx=0).stop 2024-04-25T00:33:31.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.938+0000 7fc8bb640640 1 -- 172.21.15.29:0/4073329526 shutdown_connections 2024-04-25T00:33:31.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.938+0000 7fc8bb640640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b40cbb30 0x7fc8b41f6940 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.938+0000 7fc8bb640640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc898067a70 0x7fc898069f30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.938+0000 7fc8bb640640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8b40c8030 0x7fc8b40cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.938+0000 7fc8bb640640 1 --2- 172.21.15.29:0/4073329526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8a0003a10 0x7fc8b40cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:31.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.938+0000 7fc8bb640640 1 -- 172.21.15.29:0/4073329526 >> 172.21.15.29:0/4073329526 conn(0x7fc8b4072930 msgr2=0x7fc8b415b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:31.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.938+0000 7fc8bb640640 1 -- 172.21.15.29:0/4073329526 shutdown_connections 2024-04-25T00:33:31.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:31.938+0000 7fc8bb640640 1 -- 172.21.15.29:0/4073329526 wait complete. 2024-04-25T00:33:31.958 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:33:32.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f7d951640 1 Processor -- start 2024-04-25T00:33:32.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f7d951640 1 -- start start 2024-04-25T00:33:32.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f7d951640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f7806cd30 0x7f8f78162450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f7d951640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f7806d4a0 con 0x7f8f7815e2b0 2024-04-25T00:33:32.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f7d951640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f780b18a0 con 0x7f8f780b2380 2024-04-25T00:33:32.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f7d951640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7806d780 con 0x7f8f7806cd30 2024-04-25T00:33:32.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f76ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8f7815e2b0 0x7f8f78161d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59638/0 (socket says 172.21.15.29:59638) 2024-04-25T00:33:32.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f76ffd640 1 -- 172.21.15.29:0/846442540 learned_addr learned my addr 172.21.15.29:0/846442540 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:32.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f767fc640 1 --2- 172.21.15.29:0/846442540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f7806cd30 0x7f8f78162450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4023441232 0 0) 0x7f8f7806d4a0 con 0x7f8f7815e2b0 2024-04-25T00:33:32.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f60003530 con 0x7f8f7815e2b0 2024-04-25T00:33:32.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3195183303 0 0) 0x7f8f60003530 con 0x7f8f7815e2b0 2024-04-25T00:33:32.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f7806cd30 msgr2=0x7f8f78162450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:32.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 --2- 172.21.15.29:0/846442540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f7806cd30 0x7f8f78162450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 >> v1:172.21.15.157:6789/0 conn(0x7f8f780b2380 legacy=0x7f8f780b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f780b1b80 con 0x7f8f7815e2b0 2024-04-25T00:33:32.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8f68001a00 con 0x7f8f7815e2b0 2024-04-25T00:33:32.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 >> v1:172.21.15.29:6789/0 conn(0x7f8f7815e2b0 legacy=0x7f8f78161d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 --2- 172.21.15.29:0/846442540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f600039f0 0x7f8f60003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 --2- 172.21.15.29:0/846442540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f60004810 0x7f8f60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 --2- 172.21.15.29:0/846442540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f7806cd30 0x7f8f600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7806d780 con 0x7f8f60004810 2024-04-25T00:33:32.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6000bab0 con 0x7f8f600039f0 2024-04-25T00:33:32.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6000bcb0 con 0x7f8f7806cd30 2024-04-25T00:33:32.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f767fc640 1 --2- 172.21.15.29:0/846442540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f60004810 0x7f8f60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f76ffd640 1 --2- 172.21.15.29:0/846442540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f600039f0 0x7f8f60003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f777fe640 1 --2- 172.21.15.29:0/846442540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f7806cd30 0x7f8f600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f767fc640 1 -- 172.21.15.29:0/846442540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f7806cd30 msgr2=0x7f8f600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:32.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f767fc640 1 --2- 172.21.15.29:0/846442540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f7806cd30 0x7f8f600051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f767fc640 1 -- 172.21.15.29:0/846442540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f600039f0 msgr2=0x7f8f60003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:32.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f767fc640 1 --2- 172.21.15.29:0/846442540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f600039f0 0x7f8f60003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.082+0000 7f8f767fc640 1 -- 172.21.15.29:0/846442540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f60003530 con 0x7f8f60004810 2024-04-25T00:33:32.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.086+0000 7f8f767fc640 1 --2- 172.21.15.29:0/846442540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f60004810 0x7f8f60004c90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009ab0 tx=0x7f8f6c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcb3687eda933d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:32.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.086+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0b2070 con 0x7f8f60004810 2024-04-25T00:33:32.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.086+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6c0a5510 con 0x7f8f60004810 2024-04-25T00:33:32.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.086+0000 7f8f75ffb640 1 -- 172.21.15.29:0/846442540 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0096b0 con 0x7f8f60004810 2024-04-25T00:33:32.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.086+0000 7f8f7d951640 1 -- 172.21.15.29:0/846442540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f60004810 msgr2=0x7f8f60004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.086+0000 7f8f7d951640 1 --2- 172.21.15.29:0/846442540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f60004810 0x7f8f60004c90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009ab0 tx=0x7f8f6c01ca30 comp rx=0 tx=0).stop 2024-04-25T00:33:32.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.086+0000 7f8f777fe640 1 -- 172.21.15.29:0/846442540 reap_dead start 2024-04-25T00:33:32.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.086+0000 7f8f7d951640 1 -- 172.21.15.29:0/846442540 shutdown_connections 2024-04-25T00:33:32.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.086+0000 7f8f7d951640 1 -- 172.21.15.29:0/846442540 >> 172.21.15.29:0/846442540 conn(0x7f8f780b38b0 msgr2=0x7f8f780b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:32.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.086+0000 7f8f7d951640 1 -- 172.21.15.29:0/846442540 shutdown_connections 2024-04-25T00:33:32.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.086+0000 7f8f7d951640 1 -- 172.21.15.29:0/846442540 wait complete. 2024-04-25T00:33:32.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.086+0000 7f8f7d951640 1 Processor -- start 2024-04-25T00:33:32.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.086+0000 7f8f7d951640 1 -- start start 2024-04-25T00:33:32.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f7d951640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f60004810 0x7f8f781544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f7d951640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f7815fa80 0x7f8f780c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f7d951640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f780c6d80 0x7f8f78161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f7d951640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7806d170 con 0x7f8f7815fa80 2024-04-25T00:33:32.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f7d951640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7815e690 con 0x7f8f780c6d80 2024-04-25T00:33:32.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f7d951640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f78162450 con 0x7f8f60004810 2024-04-25T00:33:32.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f767fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f7815fa80 0x7f8f780c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f767fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f7815fa80 0x7f8f780c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37596/0 (socket says 172.21.15.29:37596) 2024-04-25T00:33:32.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f767fc640 1 -- 172.21.15.29:0/2869611906 learned_addr learned my addr 172.21.15.29:0/2869611906 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:32.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f76ffd640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f60004810 0x7f8f781544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f777fe640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f780c6d80 0x7f8f78161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f767fc640 1 -- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f60004810 msgr2=0x7f8f781544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f767fc640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f60004810 0x7f8f781544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f767fc640 1 -- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f780c6d80 msgr2=0x7f8f78161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f767fc640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f780c6d80 0x7f8f78161000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f767fc640 1 -- 172.21.15.29:0/2869611906 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7806ded0 con 0x7f8f7815fa80 2024-04-25T00:33:32.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f767fc640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f7815fa80 0x7f8f780c6840 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c01c860 tx=0x7f8f60005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:32.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f5ffff640 1 -- 172.21.15.29:0/2869611906 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0b2040 con 0x7f8f7815fa80 2024-04-25T00:33:32.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f5ffff640 1 -- 172.21.15.29:0/2869611906 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6c004020 con 0x7f8f7815fa80 2024-04-25T00:33:32.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f7d951640 1 -- 172.21.15.29:0/2869611906 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7815ff60 con 0x7f8f7815fa80 2024-04-25T00:33:32.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f5ffff640 1 -- 172.21.15.29:0/2869611906 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0043d0 con 0x7f8f7815fa80 2024-04-25T00:33:32.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f7d951640 1 -- 172.21.15.29:0/2869611906 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f78160520 con 0x7f8f7815fa80 2024-04-25T00:33:32.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f7d951640 1 -- 172.21.15.29:0/2869611906 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f78069270 con 0x7f8f7815fa80 2024-04-25T00:33:32.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f5ffff640 1 -- 172.21.15.29:0/2869611906 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f8f6c0c0060 con 0x7f8f7815fa80 2024-04-25T00:33:32.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f5ffff640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8f64067a70 0x7f8f64069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f5ffff640 1 -- 172.21.15.29:0/2869611906 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f8f6c11fbc0 con 0x7f8f7815fa80 2024-04-25T00:33:32.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.090+0000 7f8f76ffd640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8f64067a70 0x7f8f64069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.094+0000 7f8f76ffd640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8f64067a70 0x7f8f64069f30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8f68002060 tx=0x7f8f68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:32.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.094+0000 7f8f5ffff640 1 -- 172.21.15.29:0/2869611906 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8f6c016740 con 0x7f8f7815fa80 2024-04-25T00:33:32.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f7d951640 1 -- 172.21.15.29:0/2869611906 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8f78154a00 con 0x7f8f7815fa80 2024-04-25T00:33:32.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f5ffff640 1 -- 172.21.15.29:0/2869611906 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) ==== 104+0+11 (secure 0 0 0) 0x7f8f6c0f03e0 con 0x7f8f7815fa80 2024-04-25T00:33:32.255 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:33:32.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f7d951640 1 -- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8f64067a70 msgr2=0x7f8f64069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f7d951640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8f64067a70 0x7f8f64069f30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8f68002060 tx=0x7f8f68009040 comp rx=0 tx=0).stop 2024-04-25T00:33:32.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f7d951640 1 -- 172.21.15.29:0/2869611906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f7815fa80 msgr2=0x7f8f780c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f7d951640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f7815fa80 0x7f8f780c6840 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c01c860 tx=0x7f8f60005840 comp rx=0 tx=0).stop 2024-04-25T00:33:32.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f7d951640 1 -- 172.21.15.29:0/2869611906 shutdown_connections 2024-04-25T00:33:32.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f7d951640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8f64067a70 0x7f8f64069f30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f7d951640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f780c6d80 0x7f8f78161000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f7d951640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f7815fa80 0x7f8f780c6840 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f7d951640 1 --2- 172.21.15.29:0/2869611906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f60004810 0x7f8f781544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f7d951640 1 -- 172.21.15.29:0/2869611906 >> 172.21.15.29:0/2869611906 conn(0x7f8f780b38b0 msgr2=0x7f8f7815d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:32.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f7d951640 1 -- 172.21.15.29:0/2869611906 shutdown_connections 2024-04-25T00:33:32.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.250+0000 7f8f7d951640 1 -- 172.21.15.29:0/2869611906 wait complete. 2024-04-25T00:33:32.272 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:33:32.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.394+0000 7f8ee1516640 1 Processor -- start 2024-04-25T00:33:32.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.394+0000 7f8ee1516640 1 -- start start 2024-04-25T00:33:32.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8ee1516640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8edc15e250 0x7f8edc0d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8ee1516640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc0cfb00 0x7f8edc0d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8ee1516640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8edc0c8690 con 0x7f8edc15e630 2024-04-25T00:33:32.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8ee1516640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8edc0c8440 con 0x7f8edc15e250 2024-04-25T00:33:32.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8ee1516640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8edc0c8970 con 0x7f8edc0cfb00 2024-04-25T00:33:32.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8edaffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8edc15e250 0x7f8edc0d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8edaffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8edc15e250 0x7f8edc0d24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:34960/0 (socket says 172.21.15.29:34960) 2024-04-25T00:33:32.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8eda7fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8edc15e630 0x7f8edc0c9450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:42726/0 (socket says 172.21.15.29:42726) 2024-04-25T00:33:32.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8edaffd640 1 -- 172.21.15.29:0/2037297471 learned_addr learned my addr 172.21.15.29:0/2037297471 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:32.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8edb7fe640 1 --2- 172.21.15.29:0/2037297471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc0cfb00 0x7f8edc0d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8edaffd640 1 -- 172.21.15.29:0/2037297471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc0cfb00 msgr2=0x7f8edc0d29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:32.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8edaffd640 1 --2- 172.21.15.29:0/2037297471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc0cfb00 0x7f8edc0d29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8edaffd640 1 -- 172.21.15.29:0/2037297471 >> v1:172.21.15.157:6789/0 conn(0x7f8edc15e630 legacy=0x7f8edc0c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8edaffd640 1 -- 172.21.15.29:0/2037297471 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc0c8c50 con 0x7f8edc15e250 2024-04-25T00:33:32.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.398+0000 7f8edaffd640 1 --2- 172.21.15.29:0/2037297471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc15e250 0x7f8edc0d24a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009870 tx=0x7f8ecc01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ac34bda6c457c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:32.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ed9ffb640 1 -- 172.21.15.29:0/2037297471 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0b0070 con 0x7f8edc15e250 2024-04-25T00:33:32.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ed9ffb640 1 -- 172.21.15.29:0/2037297471 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ecc0ab600 con 0x7f8edc15e250 2024-04-25T00:33:32.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ed9ffb640 1 -- 172.21.15.29:0/2037297471 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0ab9d0 con 0x7f8edc15e250 2024-04-25T00:33:32.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 -- 172.21.15.29:0/2037297471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc15e250 msgr2=0x7f8edc0d24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 --2- 172.21.15.29:0/2037297471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc15e250 0x7f8edc0d24a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009870 tx=0x7f8ecc01c4d0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 -- 172.21.15.29:0/2037297471 shutdown_connections 2024-04-25T00:33:32.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 --2- 172.21.15.29:0/2037297471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc0cfb00 0x7f8edc0d29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 --2- 172.21.15.29:0/2037297471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc15e250 0x7f8edc0d24a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 -- 172.21.15.29:0/2037297471 >> 172.21.15.29:0/2037297471 conn(0x7f8edc0d3f70 msgr2=0x7f8edc0d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:32.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 -- 172.21.15.29:0/2037297471 shutdown_connections 2024-04-25T00:33:32.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 -- 172.21.15.29:0/2037297471 wait complete. 2024-04-25T00:33:32.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 Processor -- start 2024-04-25T00:33:32.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 -- start start 2024-04-25T00:33:32.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0cfb00 0x7f8edc15b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8edc15e630 0x7f8edc0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc154c70 0x7f8edc0b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc0ca370 con 0x7f8edc15e630 2024-04-25T00:33:32.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc0d1ed0 con 0x7f8edc0cfb00 2024-04-25T00:33:32.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8ee1516640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8edc0ca4f0 con 0x7f8edc154c70 2024-04-25T00:33:32.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8eda7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8edc15e630 0x7f8edc0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8eda7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8edc15e630 0x7f8edc0cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37600/0 (socket says 172.21.15.29:37600) 2024-04-25T00:33:32.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.402+0000 7f8eda7fc640 1 -- 172.21.15.29:0/1439722045 learned_addr learned my addr 172.21.15.29:0/1439722045 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:32.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8edaffd640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0cfb00 0x7f8edc15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8edb7fe640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc154c70 0x7f8edc0b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8eda7fc640 1 -- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc154c70 msgr2=0x7f8edc0b93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8eda7fc640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc154c70 0x7f8edc0b93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8eda7fc640 1 -- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0cfb00 msgr2=0x7f8edc15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8eda7fc640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0cfb00 0x7f8edc15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8eda7fc640 1 -- 172.21.15.29:0/1439722045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc1551e0 con 0x7f8edc15e630 2024-04-25T00:33:32.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8eda7fc640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8edc15e630 0x7f8edc0cd780 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000c460 tx=0x7f8ed000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:32.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8ec37fe640 1 -- 172.21.15.29:0/1439722045 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0017070 con 0x7f8edc15e630 2024-04-25T00:33:32.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8ec37fe640 1 -- 172.21.15.29:0/1439722045 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed0005c50 con 0x7f8edc15e630 2024-04-25T00:33:32.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8ee1516640 1 -- 172.21.15.29:0/1439722045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8edc1554c0 con 0x7f8edc15e630 2024-04-25T00:33:32.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8ec37fe640 1 -- 172.21.15.29:0/1439722045 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed00072b0 con 0x7f8edc15e630 2024-04-25T00:33:32.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8ee1516640 1 -- 172.21.15.29:0/1439722045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8edc06c650 con 0x7f8edc15e630 2024-04-25T00:33:32.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8ee1516640 1 -- 172.21.15.29:0/1439722045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8edc069270 con 0x7f8edc15e630 2024-04-25T00:33:32.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8ec37fe640 1 -- 172.21.15.29:0/1439722045 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f8ed001a070 con 0x7f8edc15e630 2024-04-25T00:33:32.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8ec37fe640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ec8067a70 0x7f8ec8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8ec37fe640 1 -- 172.21.15.29:0/1439722045 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f8ed0085290 con 0x7f8edc15e630 2024-04-25T00:33:32.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8edaffd640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ec8067a70 0x7f8ec8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.406+0000 7f8edaffd640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ec8067a70 0x7f8ec8069f30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0ad000 tx=0x7f8ecc0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:32.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.410+0000 7f8ec37fe640 1 -- 172.21.15.29:0/1439722045 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8ed0055b50 con 0x7f8edc15e630 2024-04-25T00:33:32.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.570+0000 7f8ee1516640 1 -- 172.21.15.29:0/1439722045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8edc15b740 con 0x7f8edc15e630 2024-04-25T00:33:32.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.574+0000 7f8ec37fe640 1 -- 172.21.15.29:0/1439722045 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) ==== 100+0+11 (secure 0 0 0) 0x7f8ed0055450 con 0x7f8edc15e630 2024-04-25T00:33:32.580 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:33:32.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.578+0000 7f8ee1516640 1 -- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ec8067a70 msgr2=0x7f8ec8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.578+0000 7f8ee1516640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ec8067a70 0x7f8ec8069f30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0ad000 tx=0x7f8ecc0a6000 comp rx=0 tx=0).stop 2024-04-25T00:33:32.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.578+0000 7f8ee1516640 1 -- 172.21.15.29:0/1439722045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8edc15e630 msgr2=0x7f8edc0cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.578+0000 7f8ee1516640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8edc15e630 0x7f8edc0cd780 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000c460 tx=0x7f8ed000c920 comp rx=0 tx=0).stop 2024-04-25T00:33:32.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.578+0000 7f8ee1516640 1 -- 172.21.15.29:0/1439722045 shutdown_connections 2024-04-25T00:33:32.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.578+0000 7f8ee1516640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc154c70 0x7f8edc0b93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.578+0000 7f8ee1516640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8edc15e630 0x7f8edc0cd780 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.578+0000 7f8ee1516640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8ec8067a70 0x7f8ec8069f30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.578+0000 7f8ee1516640 1 --2- 172.21.15.29:0/1439722045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0cfb00 0x7f8edc15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.578+0000 7f8ee1516640 1 -- 172.21.15.29:0/1439722045 >> 172.21.15.29:0/1439722045 conn(0x7f8edc0d3f70 msgr2=0x7f8edc0d2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:32.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.578+0000 7f8ee1516640 1 -- 172.21.15.29:0/1439722045 shutdown_connections 2024-04-25T00:33:32.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.578+0000 7f8ee1516640 1 -- 172.21.15.29:0/1439722045 wait complete. 2024-04-25T00:33:32.600 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:33:32.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.718+0000 7efc66f07640 1 Processor -- start 2024-04-25T00:33:32.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.718+0000 7efc66f07640 1 -- start start 2024-04-25T00:33:32.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc66f07640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc60158af0 0x7efc600c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc66f07640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efc600725d0 0x7efc600cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc66f07640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc60158ed0 con 0x7efc600c70b0 2024-04-25T00:33:32.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc66f07640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7efc600c7530 con 0x7efc600725d0 2024-04-25T00:33:32.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc66f07640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc6015f650 con 0x7efc60158af0 2024-04-25T00:33:32.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc654a4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efc600725d0 0x7efc600cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc654a4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efc600725d0 0x7efc600cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37608/0 (socket says 172.21.15.29:37608) 2024-04-25T00:33:32.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc654a4640 1 -- 172.21.15.29:0/3248878747 learned_addr learned my addr 172.21.15.29:0/3248878747 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:32.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc64ca3640 1 --2- 172.21.15.29:0/3248878747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc60158af0 0x7efc600c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc654a4640 1 -- 172.21.15.29:0/3248878747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc60158af0 msgr2=0x7efc600c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:32.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc654a4640 1 --2- 172.21.15.29:0/3248878747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc60158af0 0x7efc600c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc654a4640 1 -- 172.21.15.29:0/3248878747 >> v1:172.21.15.157:6789/0 conn(0x7efc600c70b0 legacy=0x7efc60071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:32.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc654a4640 1 -- 172.21.15.29:0/3248878747 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6015ed20 con 0x7efc600725d0 2024-04-25T00:33:32.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc654a4640 1 --2- 172.21.15.29:0/3248878747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc600725d0 0x7efc600cc270 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7efc58009e50 tx=0x7efc5801c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bbe2d5c3658861 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:32.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc5f7fe640 1 -- 172.21.15.29:0/3248878747 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580ac020 con 0x7efc600725d0 2024-04-25T00:33:32.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc5f7fe640 1 -- 172.21.15.29:0/3248878747 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc580a64a0 con 0x7efc600725d0 2024-04-25T00:33:32.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc5f7fe640 1 -- 172.21.15.29:0/3248878747 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580a6870 con 0x7efc600725d0 2024-04-25T00:33:32.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc66f07640 1 -- 172.21.15.29:0/3248878747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc600725d0 msgr2=0x7efc600cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc66f07640 1 --2- 172.21.15.29:0/3248878747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc600725d0 0x7efc600cc270 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7efc58009e50 tx=0x7efc5801c810 comp rx=0 tx=0).stop 2024-04-25T00:33:32.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc66f07640 1 -- 172.21.15.29:0/3248878747 shutdown_connections 2024-04-25T00:33:32.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc66f07640 1 --2- 172.21.15.29:0/3248878747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc600725d0 0x7efc600cc270 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc66f07640 1 --2- 172.21.15.29:0/3248878747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc60158af0 0x7efc600c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.722+0000 7efc66f07640 1 -- 172.21.15.29:0/3248878747 >> 172.21.15.29:0/3248878747 conn(0x7efc600d0c50 msgr2=0x7efc600d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:32.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc66f07640 1 -- 172.21.15.29:0/3248878747 shutdown_connections 2024-04-25T00:33:32.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc66f07640 1 -- 172.21.15.29:0/3248878747 wait complete. 2024-04-25T00:33:32.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc66f07640 1 Processor -- start 2024-04-25T00:33:32.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc66f07640 1 -- start start 2024-04-25T00:33:32.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc66f07640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc600725d0 0x7efc600b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc66f07640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc600c70b0 0x7efc600cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc66f07640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc60158af0 0x7efc6013e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc66f07640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc600cd350 con 0x7efc60158af0 2024-04-25T00:33:32.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc66f07640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc600cd1d0 con 0x7efc600c70b0 2024-04-25T00:33:32.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc66f07640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc600cd4d0 con 0x7efc600725d0 2024-04-25T00:33:32.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc654a4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc60158af0 0x7efc6013e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc64ca3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc600725d0 0x7efc600b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc5ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc600c70b0 0x7efc600cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc654a4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc60158af0 0x7efc6013e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37614/0 (socket says 172.21.15.29:37614) 2024-04-25T00:33:32.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc654a4640 1 -- 172.21.15.29:0/2384337328 learned_addr learned my addr 172.21.15.29:0/2384337328 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:32.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc654a4640 1 -- 172.21.15.29:0/2384337328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc600725d0 msgr2=0x7efc600b6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc654a4640 1 --2- 172.21.15.29:0/2384337328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc600725d0 0x7efc600b6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc654a4640 1 -- 172.21.15.29:0/2384337328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc600c70b0 msgr2=0x7efc600cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc654a4640 1 --2- 172.21.15.29:0/2384337328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc600c70b0 0x7efc600cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc654a4640 1 -- 172.21.15.29:0/2384337328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc600b4860 con 0x7efc60158af0 2024-04-25T00:33:32.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc654a4640 1 --2- 172.21.15.29:0/2384337328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc60158af0 0x7efc6013e6f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7efc5801aec0 tx=0x7efc580195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:32.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc5d7fa640 1 -- 172.21.15.29:0/2384337328 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58009510 con 0x7efc60158af0 2024-04-25T00:33:32.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc5d7fa640 1 -- 172.21.15.29:0/2384337328 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc580a7680 con 0x7efc60158af0 2024-04-25T00:33:32.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc66f07640 1 -- 172.21.15.29:0/2384337328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc600b54d0 con 0x7efc60158af0 2024-04-25T00:33:32.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc5d7fa640 1 -- 172.21.15.29:0/2384337328 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580a7a50 con 0x7efc60158af0 2024-04-25T00:33:32.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.726+0000 7efc66f07640 1 -- 172.21.15.29:0/2384337328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc600b5a90 con 0x7efc60158af0 2024-04-25T00:33:32.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.730+0000 7efc66f07640 1 -- 172.21.15.29:0/2384337328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc60069270 con 0x7efc60158af0 2024-04-25T00:33:32.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.730+0000 7efc5d7fa640 1 -- 172.21.15.29:0/2384337328 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7efc580af020 con 0x7efc60158af0 2024-04-25T00:33:32.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.730+0000 7efc5d7fa640 1 --2- 172.21.15.29:0/2384337328 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efc4c067aa0 0x7efc4c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:32.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.730+0000 7efc5d7fa640 1 -- 172.21.15.29:0/2384337328 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7efc58120050 con 0x7efc60158af0 2024-04-25T00:33:32.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.730+0000 7efc64ca3640 1 --2- 172.21.15.29:0/2384337328 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efc4c067aa0 0x7efc4c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:32.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.730+0000 7efc64ca3640 1 --2- 172.21.15.29:0/2384337328 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efc4c067aa0 0x7efc4c069f60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efc500079c0 tx=0x7efc50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:32.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.734+0000 7efc5d7fa640 1 -- 172.21.15.29:0/2384337328 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efc60158ed0 con 0x7efc60158af0 2024-04-25T00:33:32.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc66f07640 1 -- 172.21.15.29:0/2384337328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efc6015ea90 con 0x7efc60158af0 2024-04-25T00:33:32.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc5d7fa640 1 -- 172.21.15.29:0/2384337328 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7efc580f01b0 con 0x7efc60158af0 2024-04-25T00:33:32.906 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:33:32.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc66f07640 1 -- 172.21.15.29:0/2384337328 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efc4c067aa0 msgr2=0x7efc4c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc66f07640 1 --2- 172.21.15.29:0/2384337328 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efc4c067aa0 0x7efc4c069f60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efc500079c0 tx=0x7efc50008040 comp rx=0 tx=0).stop 2024-04-25T00:33:32.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc66f07640 1 -- 172.21.15.29:0/2384337328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc60158af0 msgr2=0x7efc6013e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:32.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc66f07640 1 --2- 172.21.15.29:0/2384337328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc60158af0 0x7efc6013e6f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7efc5801aec0 tx=0x7efc580195b0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc66f07640 1 -- 172.21.15.29:0/2384337328 shutdown_connections 2024-04-25T00:33:32.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc66f07640 1 --2- 172.21.15.29:0/2384337328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc60158af0 0x7efc6013e6f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc66f07640 1 --2- 172.21.15.29:0/2384337328 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efc4c067aa0 0x7efc4c069f60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc66f07640 1 --2- 172.21.15.29:0/2384337328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc600c70b0 0x7efc600cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc66f07640 1 --2- 172.21.15.29:0/2384337328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc600725d0 0x7efc600b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:32.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc66f07640 1 -- 172.21.15.29:0/2384337328 >> 172.21.15.29:0/2384337328 conn(0x7efc600d0c50 msgr2=0x7efc601592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:32.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc66f07640 1 -- 172.21.15.29:0/2384337328 shutdown_connections 2024-04-25T00:33:32.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:32.902+0000 7efc66f07640 1 -- 172.21.15.29:0/2384337328 wait complete. 2024-04-25T00:33:32.923 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:33:32.923 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:33:32.930 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:33:33.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.090+0000 7f1f145be640 1 Processor -- start 2024-04-25T00:33:33.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.090+0000 7f1f145be640 1 -- start start 2024-04-25T00:33:33.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f145be640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0c15c110 0x7f1f0c0d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f145be640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f0c0cdda0 con 0x7f1f0c0d29a0 2024-04-25T00:33:33.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f145be640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f0c0cfa80 con 0x7f1f0c0ce750 2024-04-25T00:33:33.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f145be640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1f0c0cfd60 con 0x7f1f0c15c110 2024-04-25T00:33:33.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11b59640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1f0c0d29a0 0x7f1f0c0cd4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59642/0 (socket says 172.21.15.29:59642) 2024-04-25T00:33:33.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11b59640 1 -- 172.21.15.29:0/3089075362 learned_addr learned my addr 172.21.15.29:0/3089075362 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:33.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f1235a640 1 --2- 172.21.15.29:0/3089075362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0c15c110 0x7f1f0c0d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1127354254 0 0) 0x7f1f0c0cdda0 con 0x7f1f0c0d29a0 2024-04-25T00:33:33.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ef8003530 con 0x7f1f0c0d29a0 2024-04-25T00:33:33.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3239824136 0 0) 0x7f1ef8003530 con 0x7f1f0c0d29a0 2024-04-25T00:33:33.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0c15c110 msgr2=0x7f1f0c0d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 --2- 172.21.15.29:0/3089075362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0c15c110 0x7f1f0c0d0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 >> v1:172.21.15.157:6789/0 conn(0x7f1f0c0ce750 legacy=0x7f1f0c0c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c0d0040 con 0x7f1f0c0d29a0 2024-04-25T00:33:33.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f1efc0016d0 con 0x7f1f0c0d29a0 2024-04-25T00:33:33.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 >> v1:172.21.15.29:6789/0 conn(0x7f1f0c0d29a0 legacy=0x7f1f0c0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 --2- 172.21.15.29:0/3089075362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0c15c110 0x7f1ef80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 --2- 172.21.15.29:0/3089075362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8003f70 0x7f1ef8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 --2- 172.21.15.29:0/3089075362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ef8004d50 0x7f1ef80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c0cfd60 con 0x7f1ef8004d50 2024-04-25T00:33:33.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef800bb40 con 0x7f1ef8003f70 2024-04-25T00:33:33.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef800bd40 con 0x7f1f0c15c110 2024-04-25T00:33:33.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f12b5b640 1 --2- 172.21.15.29:0/3089075362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ef8004d50 0x7f1ef80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f1235a640 1 --2- 172.21.15.29:0/3089075362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0c15c110 0x7f1ef80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11b59640 1 --2- 172.21.15.29:0/3089075362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8003f70 0x7f1ef8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f12b5b640 1 -- 172.21.15.29:0/3089075362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0c15c110 msgr2=0x7f1ef80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f12b5b640 1 --2- 172.21.15.29:0/3089075362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0c15c110 0x7f1ef80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f12b5b640 1 -- 172.21.15.29:0/3089075362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8003f70 msgr2=0x7f1ef8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f12b5b640 1 --2- 172.21.15.29:0/3089075362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8003f70 0x7f1ef8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f12b5b640 1 -- 172.21.15.29:0/3089075362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef8003530 con 0x7f1ef8004d50 2024-04-25T00:33:33.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f12b5b640 1 --2- 172.21.15.29:0/3089075362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ef8004d50 0x7f1ef80051d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1f04001e70 tx=0x7f1f0401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932539f8956f4dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04001a90 con 0x7f1ef8004d50 2024-04-25T00:33:33.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f04003b80 con 0x7f1ef8004d50 2024-04-25T00:33:33.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.094+0000 7f1f11358640 1 -- 172.21.15.29:0/3089075362 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04005080 con 0x7f1ef8004d50 2024-04-25T00:33:33.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 -- 172.21.15.29:0/3089075362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ef8004d50 msgr2=0x7f1ef80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 --2- 172.21.15.29:0/3089075362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ef8004d50 0x7f1ef80051d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1f04001e70 tx=0x7f1f0401c780 comp rx=0 tx=0).stop 2024-04-25T00:33:33.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f12b5b640 1 -- 172.21.15.29:0/3089075362 reap_dead start 2024-04-25T00:33:33.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 -- 172.21.15.29:0/3089075362 shutdown_connections 2024-04-25T00:33:33.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 -- 172.21.15.29:0/3089075362 >> 172.21.15.29:0/3089075362 conn(0x7f1f0c0d1e10 msgr2=0x7f1f0c0d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:33.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 -- 172.21.15.29:0/3089075362 shutdown_connections 2024-04-25T00:33:33.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 -- 172.21.15.29:0/3089075362 wait complete. 2024-04-25T00:33:33.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 Processor -- start 2024-04-25T00:33:33.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 -- start start 2024-04-25T00:33:33.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f0c155390 0x7f1f0c06c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f0c06c860 0x7f1f0c1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0c158710 0x7f1f0c1f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c15c670 con 0x7f1f0c155390 2024-04-25T00:33:33.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c15c4f0 con 0x7f1f0c06c860 2024-04-25T00:33:33.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1f0c0c6070 con 0x7f1f0c158710 2024-04-25T00:33:33.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f1235a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f0c155390 0x7f1f0c06c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f1235a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f0c155390 0x7f1f0c06c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37638/0 (socket says 172.21.15.29:37638) 2024-04-25T00:33:33.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f1235a640 1 -- 172.21.15.29:0/3506187281 learned_addr learned my addr 172.21.15.29:0/3506187281 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:33.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f12b5b640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0c158710 0x7f1f0c1f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f11b59640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f0c06c860 0x7f1f0c1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f1235a640 1 -- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0c158710 msgr2=0x7f1f0c1f7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f1235a640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0c158710 0x7f1f0c1f7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f1235a640 1 -- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f0c06c860 msgr2=0x7f1f0c1581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f1235a640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f0c06c860 0x7f1f0c1581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f1235a640 1 -- 172.21.15.29:0/3506187281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c158b90 con 0x7f1f0c155390 2024-04-25T00:33:33.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f11b59640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f0c06c860 0x7f1f0c1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:33.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f1235a640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f0c155390 0x7f1f0c06c320 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1f080115e0 tx=0x7f1f08011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f02ffd640 1 -- 172.21.15.29:0/3506187281 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08015870 con 0x7f1f0c155390 2024-04-25T00:33:33.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f02ffd640 1 -- 172.21.15.29:0/3506187281 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f08015a10 con 0x7f1f0c155390 2024-04-25T00:33:33.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 -- 172.21.15.29:0/3506187281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0c06dda0 con 0x7f1f0c155390 2024-04-25T00:33:33.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f02ffd640 1 -- 172.21.15.29:0/3506187281 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08015dc0 con 0x7f1f0c155390 2024-04-25T00:33:33.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.098+0000 7f1f145be640 1 -- 172.21.15.29:0/3506187281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0c06cfe0 con 0x7f1f0c155390 2024-04-25T00:33:33.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.102+0000 7f1f145be640 1 -- 172.21.15.29:0/3506187281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f0c069270 con 0x7f1f0c155390 2024-04-25T00:33:33.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.102+0000 7f1f02ffd640 1 -- 172.21.15.29:0/3506187281 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f1f0800f480 con 0x7f1f0c155390 2024-04-25T00:33:33.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.102+0000 7f1f02ffd640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1ef0067aa0 0x7f1ef0069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.102+0000 7f1f02ffd640 1 -- 172.21.15.29:0/3506187281 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f1f08086970 con 0x7f1f0c155390 2024-04-25T00:33:33.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.102+0000 7f1f11b59640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1ef0067aa0 0x7f1ef0069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.102+0000 7f1f11b59640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1ef0067aa0 0x7f1ef0069f60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0010b0 tx=0x7f1efc001560 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.106+0000 7f1f02ffd640 1 -- 172.21.15.29:0/3506187281 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1f08019030 con 0x7f1f0c155390 2024-04-25T00:33:33.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.270+0000 7f1f145be640 1 -- 172.21.15.29:0/3506187281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f0c0cf3c0 con 0x7f1f0c155390 2024-04-25T00:33:33.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.270+0000 7f1f02ffd640 1 -- 172.21.15.29:0/3506187281 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16565 (secure 0 0 0) 0x7f1f080571a0 con 0x7f1f0c155390 2024-04-25T00:33:33.277 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:33.277 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":45,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:33:14.784398+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:33:10.789862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:33:12.129841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:33:33.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.274+0000 7f1f145be640 1 -- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1ef0067aa0 msgr2=0x7f1ef0069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.274+0000 7f1f145be640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1ef0067aa0 0x7f1ef0069f60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0010b0 tx=0x7f1efc001560 comp rx=0 tx=0).stop 2024-04-25T00:33:33.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.274+0000 7f1f145be640 1 -- 172.21.15.29:0/3506187281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f0c155390 msgr2=0x7f1f0c06c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.274+0000 7f1f145be640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f0c155390 0x7f1f0c06c320 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1f080115e0 tx=0x7f1f08011aa0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.274+0000 7f1f145be640 1 -- 172.21.15.29:0/3506187281 shutdown_connections 2024-04-25T00:33:33.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.274+0000 7f1f145be640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f0c158710 0x7f1f0c1f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.274+0000 7f1f145be640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1ef0067aa0 0x7f1ef0069f60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.274+0000 7f1f145be640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f0c06c860 0x7f1f0c1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.274+0000 7f1f145be640 1 --2- 172.21.15.29:0/3506187281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f0c155390 0x7f1f0c06c320 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.274+0000 7f1f145be640 1 -- 172.21.15.29:0/3506187281 >> 172.21.15.29:0/3506187281 conn(0x7f1f0c0d1e10 msgr2=0x7f1f0c0c7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:33.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.274+0000 7f1f145be640 1 -- 172.21.15.29:0/3506187281 shutdown_connections 2024-04-25T00:33:33.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.274+0000 7f1f145be640 1 -- 172.21.15.29:0/3506187281 wait complete. 2024-04-25T00:33:33.296 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:33:10.789862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:33:12.129841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:33:33.297 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:33:33.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1fa7b0640 1 Processor -- start 2024-04-25T00:33:33.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1fa7b0640 1 -- start start 2024-04-25T00:33:33.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1fa7b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa1f415df70 0x7fa1f40b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1fa7b0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f406d980 con 0x7fa1f406d210 2024-04-25T00:33:33.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1fa7b0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f406dc60 con 0x7fa1f40b4d30 2024-04-25T00:33:33.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1fa7b0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f406df40 con 0x7fa1f415df70 2024-04-25T00:33:33.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f37fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa1f406d210 0x7fa1f40b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59650/0 (socket says 172.21.15.29:59650) 2024-04-25T00:33:33.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f37fe640 1 -- 172.21.15.29:0/3777331883 learned_addr learned my addr 172.21.15.29:0/3777331883 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:33.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f3fff640 1 --2- 172.21.15.29:0/3777331883 >> v2:172.21.15.157:3300/0 conn(0x7fa1f415df70 0x7fa1f40b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3329763060 0 0) 0x7fa1f406d980 con 0x7fa1f406d210 2024-04-25T00:33:33.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1dc003530 con 0x7fa1f406d210 2024-04-25T00:33:33.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4068608667 0 0) 0x7fa1dc003530 con 0x7fa1f406d210 2024-04-25T00:33:33.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 >> v2:172.21.15.157:3300/0 conn(0x7fa1f415df70 msgr2=0x7fa1f40b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 --2- 172.21.15.29:0/3777331883 >> v2:172.21.15.157:3300/0 conn(0x7fa1f415df70 0x7fa1f40b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 >> v1:172.21.15.157:6789/0 conn(0x7fa1f40b4d30 legacy=0x7fa1f40ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f406e220 con 0x7fa1f406d210 2024-04-25T00:33:33.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fa1e8001680 con 0x7fa1f406d210 2024-04-25T00:33:33.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f3fff640 1 --2- 172.21.15.29:0/3777331883 >> v2:172.21.15.157:3300/0 conn(0x7fa1f415df70 0x7fa1f40b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:33.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 >> v1:172.21.15.29:6789/0 conn(0x7fa1f406d210 legacy=0x7fa1f40b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 --2- 172.21.15.29:0/3777331883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1dc0039c0 0x7fa1dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 --2- 172.21.15.29:0/3777331883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1dc004810 0x7fa1dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 --2- 172.21.15.29:0/3777331883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1dc0052f0 0x7fa1dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f406df40 con 0x7fa1dc0052f0 2024-04-25T00:33:33.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc00c1c0 con 0x7fa1dc004810 2024-04-25T00:33:33.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa1dc00c3c0 con 0x7fa1dc0039c0 2024-04-25T00:33:33.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f8d4d640 1 --2- 172.21.15.29:0/3777331883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1dc0052f0 0x7fa1dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f3fff640 1 --2- 172.21.15.29:0/3777331883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1dc0039c0 0x7fa1dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f37fe640 1 --2- 172.21.15.29:0/3777331883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1dc004810 0x7fa1dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f8d4d640 1 -- 172.21.15.29:0/3777331883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1dc0039c0 msgr2=0x7fa1dc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f8d4d640 1 --2- 172.21.15.29:0/3777331883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1dc0039c0 0x7fa1dc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f8d4d640 1 -- 172.21.15.29:0/3777331883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1dc004810 msgr2=0x7fa1dc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f8d4d640 1 --2- 172.21.15.29:0/3777331883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1dc004810 0x7fa1dc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f8d4d640 1 -- 172.21.15.29:0/3777331883 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1dc003530 con 0x7fa1dc0052f0 2024-04-25T00:33:33.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.418+0000 7fa1f37fe640 1 --2- 172.21.15.29:0/3777331883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1dc004810 0x7fa1dc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:33.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1f8d4d640 1 -- 172.21.15.29:0/3777331883 reap_dead start 2024-04-25T00:33:33.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1f8d4d640 1 --2- 172.21.15.29:0/3777331883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1dc0052f0 0x7fa1dc00b9d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec001e40 tx=0x7fa1ec01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c114277923d50f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec001a90 con 0x7fa1dc0052f0 2024-04-25T00:33:33.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1ec003b80 con 0x7fa1dc0052f0 2024-04-25T00:33:33.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1f2ffd640 1 -- 172.21.15.29:0/3777331883 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec005020 con 0x7fa1dc0052f0 2024-04-25T00:33:33.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3777331883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1dc0052f0 msgr2=0x7fa1dc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1fa7b0640 1 --2- 172.21.15.29:0/3777331883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1dc0052f0 0x7fa1dc00b9d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec001e40 tx=0x7fa1ec01c760 comp rx=0 tx=0).stop 2024-04-25T00:33:33.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3777331883 shutdown_connections 2024-04-25T00:33:33.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1fa7b0640 1 --2- 172.21.15.29:0/3777331883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1dc0052f0 0x7fa1dc00b9d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3777331883 >> 172.21.15.29:0/3777331883 conn(0x7fa1f40b1870 msgr2=0x7fa1f40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:33.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3777331883 shutdown_connections 2024-04-25T00:33:33.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3777331883 wait complete. 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1fa7b0640 1 Processor -- start 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.422+0000 7fa1fa7b0640 1 -- start start 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1fa7b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1dc0052f0 0x7fa1f414e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1fa7b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1f4156b80 0x7fa1f4157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1fa7b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1f4154e60 0x7fa1f4155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1fa7b0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f406d6a0 con 0x7fa1dc0052f0 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1fa7b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f415e350 con 0x7fa1f4156b80 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1fa7b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f406ea20 con 0x7fa1f4154e60 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1dc0052f0 0x7fa1f414e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1dc0052f0 0x7fa1f414e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37666/0 (socket says 172.21.15.29:37666) 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f3fff640 1 -- 172.21.15.29:0/3010989826 learned_addr learned my addr 172.21.15.29:0/3010989826 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f8d4d640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1f4154e60 0x7fa1f4155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f37fe640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1f4156b80 0x7fa1f4157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f3fff640 1 -- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1f4154e60 msgr2=0x7fa1f4155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f3fff640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1f4154e60 0x7fa1f4155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f3fff640 1 -- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1f4156b80 msgr2=0x7fa1f4157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f3fff640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1f4156b80 0x7fa1f4157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f3fff640 1 -- 172.21.15.29:0/3010989826 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f4155bd0 con 0x7fa1dc0052f0 2024-04-25T00:33:33.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f3fff640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1dc0052f0 0x7fa1f414e610 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40108c0 tx=0x7fa1e4010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f0ff9640 1 -- 172.21.15.29:0/3010989826 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e401a070 con 0x7fa1dc0052f0 2024-04-25T00:33:33.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f0ff9640 1 -- 172.21.15.29:0/3010989826 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e400cb40 con 0x7fa1dc0052f0 2024-04-25T00:33:33.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3010989826 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f4155eb0 con 0x7fa1dc0052f0 2024-04-25T00:33:33.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f0ff9640 1 -- 172.21.15.29:0/3010989826 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e400f040 con 0x7fa1dc0052f0 2024-04-25T00:33:33.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3010989826 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f40adb80 con 0x7fa1dc0052f0 2024-04-25T00:33:33.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3010989826 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f4069270 con 0x7fa1dc0052f0 2024-04-25T00:33:33.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f0ff9640 1 -- 172.21.15.29:0/3010989826 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fa1e401d070 con 0x7fa1dc0052f0 2024-04-25T00:33:33.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f0ff9640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa1e0067aa0 0x7fa1e0069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f0ff9640 1 -- 172.21.15.29:0/3010989826 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fa1e40865b0 con 0x7fa1dc0052f0 2024-04-25T00:33:33.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.426+0000 7fa1f37fe640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa1e0067aa0 0x7fa1e0069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.430+0000 7fa1f37fe640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa1e0067aa0 0x7fa1e0069f60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80017f0 tx=0x7fa1e8000ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.434+0000 7fa1f0ff9640 1 -- 172.21.15.29:0/3010989826 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa1e400ee60 con 0x7fa1dc0052f0 2024-04-25T00:33:33.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.602+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3010989826 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1f40b1340 con 0x7fa1dc0052f0 2024-04-25T00:33:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.602+0000 7fa1f0ff9640 1 -- 172.21.15.29:0/3010989826 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7fa1e4056d60 con 0x7fa1dc0052f0 2024-04-25T00:33:33.608 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:33:33.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.606+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa1e0067aa0 msgr2=0x7fa1e0069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.606+0000 7fa1fa7b0640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa1e0067aa0 0x7fa1e0069f60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80017f0 tx=0x7fa1e8000ee0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.606+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3010989826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1dc0052f0 msgr2=0x7fa1f414e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.606+0000 7fa1fa7b0640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1dc0052f0 0x7fa1f414e610 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40108c0 tx=0x7fa1e4010d80 comp rx=0 tx=0).stop 2024-04-25T00:33:33.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.606+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3010989826 shutdown_connections 2024-04-25T00:33:33.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.606+0000 7fa1fa7b0640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1f4154e60 0x7fa1f4155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.606+0000 7fa1fa7b0640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa1e0067aa0 0x7fa1e0069f60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.606+0000 7fa1fa7b0640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1f4156b80 0x7fa1f4157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.606+0000 7fa1fa7b0640 1 --2- 172.21.15.29:0/3010989826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1dc0052f0 0x7fa1f414e610 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.606+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3010989826 >> 172.21.15.29:0/3010989826 conn(0x7fa1f40b1870 msgr2=0x7fa1f40cc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:33.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.606+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3010989826 shutdown_connections 2024-04-25T00:33:33.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.606+0000 7fa1fa7b0640 1 -- 172.21.15.29:0/3010989826 wait complete. 2024-04-25T00:33:33.627 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:33:33.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2cd91f640 1 Processor -- start 2024-04-25T00:33:33.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2cd91f640 1 -- start start 2024-04-25T00:33:33.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2cd91f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb2c80b3140 0x7fb2c806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2cd91f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2c80b3c30 0x7fb2c80ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2cd91f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c80b41e0 con 0x7fb2c80b4b10 2024-04-25T00:33:33.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2cd91f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb2c806d960 con 0x7fb2c80b3140 2024-04-25T00:33:33.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2cd91f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c806dc40 con 0x7fb2c80b3c30 2024-04-25T00:33:33.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2c6ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb2c80b3140 0x7fb2c806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2c6ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb2c80b3140 0x7fb2c806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37670/0 (socket says 172.21.15.29:37670) 2024-04-25T00:33:33.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2c6ffd640 1 -- 172.21.15.29:0/2039470761 learned_addr learned my addr 172.21.15.29:0/2039470761 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:33.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2c77fe640 1 --2- 172.21.15.29:0/2039470761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2c80b3c30 0x7fb2c80ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2c6ffd640 1 -- 172.21.15.29:0/2039470761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2c80b3c30 msgr2=0x7fb2c80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:33.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2c6ffd640 1 --2- 172.21.15.29:0/2039470761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2c80b3c30 0x7fb2c80ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2c6ffd640 1 -- 172.21.15.29:0/2039470761 >> v1:172.21.15.157:6789/0 conn(0x7fb2c80b4b10 legacy=0x7fb2c80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2c6ffd640 1 -- 172.21.15.29:0/2039470761 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c806df20 con 0x7fb2c80b3140 2024-04-25T00:33:33.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.746+0000 7fb2c77fe640 1 --2- 172.21.15.29:0/2039470761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2c80b3c30 0x7fb2c80ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:33.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2c6ffd640 1 --2- 172.21.15.29:0/2039470761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c80b3140 0x7fb2c806e440 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009e80 tx=0x7fb2b801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a009509b0714a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2c5ffb640 1 -- 172.21.15.29:0/2039470761 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80a56e0 con 0x7fb2c80b3140 2024-04-25T00:33:33.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2c5ffb640 1 -- 172.21.15.29:0/2039470761 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b80a5880 con 0x7fb2c80b3140 2024-04-25T00:33:33.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2c5ffb640 1 -- 172.21.15.29:0/2039470761 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80a5c50 con 0x7fb2c80b3140 2024-04-25T00:33:33.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2039470761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c80b3140 msgr2=0x7fb2c806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2cd91f640 1 --2- 172.21.15.29:0/2039470761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c80b3140 0x7fb2c806e440 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009e80 tx=0x7fb2b801ca10 comp rx=0 tx=0).stop 2024-04-25T00:33:33.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2039470761 shutdown_connections 2024-04-25T00:33:33.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2cd91f640 1 --2- 172.21.15.29:0/2039470761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2c80b3c30 0x7fb2c80ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2cd91f640 1 --2- 172.21.15.29:0/2039470761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c80b3140 0x7fb2c806e440 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2039470761 >> 172.21.15.29:0/2039470761 conn(0x7fb2c80b1870 msgr2=0x7fb2c80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:33.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2039470761 shutdown_connections 2024-04-25T00:33:33.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2039470761 wait complete. 2024-04-25T00:33:33.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2cd91f640 1 Processor -- start 2024-04-25T00:33:33.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.750+0000 7fb2cd91f640 1 -- start start 2024-04-25T00:33:33.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2cd91f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2c80b3140 0x7fb2c813df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2cd91f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c80b3c30 0x7fb2c815ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2cd91f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c80b4b10 0x7fb2c815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2cd91f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8168d80 con 0x7fb2c80b4b10 2024-04-25T00:33:33.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2cd91f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8168c00 con 0x7fb2c80b3c30 2024-04-25T00:33:33.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2cd91f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c8168f00 con 0x7fb2c80b3140 2024-04-25T00:33:33.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2c77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c80b4b10 0x7fb2c815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2c77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c80b4b10 0x7fb2c815a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37672/0 (socket says 172.21.15.29:37672) 2024-04-25T00:33:33.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2c77fe640 1 -- 172.21.15.29:0/2016750590 learned_addr learned my addr 172.21.15.29:0/2016750590 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:33.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2c6ffd640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2c80b3140 0x7fb2c813df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2c67fc640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c80b3c30 0x7fb2c815ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2c77fe640 1 -- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2c80b3140 msgr2=0x7fb2c813df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2c77fe640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2c80b3140 0x7fb2c813df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2c77fe640 1 -- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c80b3c30 msgr2=0x7fb2c815ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2c77fe640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c80b3c30 0x7fb2c815ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2c77fe640 1 -- 172.21.15.29:0/2016750590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c815e8e0 con 0x7fb2c80b4b10 2024-04-25T00:33:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2c6ffd640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2c80b3140 0x7fb2c813df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2c77fe640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c80b4b10 0x7fb2c815a960 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000d800 tx=0x7fb2c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2affff640 1 -- 172.21.15.29:0/2016750590 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00096a0 con 0x7fb2c80b4b10 2024-04-25T00:33:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2affff640 1 -- 172.21.15.29:0/2016750590 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c0019070 con 0x7fb2c80b4b10 2024-04-25T00:33:33.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2016750590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c815ebc0 con 0x7fb2c80b4b10 2024-04-25T00:33:33.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2affff640 1 -- 172.21.15.29:0/2016750590 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0009a80 con 0x7fb2c80b4b10 2024-04-25T00:33:33.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2016750590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c813a420 con 0x7fb2c80b4b10 2024-04-25T00:33:33.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2016750590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c8159f50 con 0x7fb2c80b4b10 2024-04-25T00:33:33.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2affff640 1 -- 172.21.15.29:0/2016750590 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fb2c0009c20 con 0x7fb2c80b4b10 2024-04-25T00:33:33.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2affff640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb2b4067a70 0x7fb2b4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:33.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2affff640 1 -- 172.21.15.29:0/2016750590 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fb2c0086350 con 0x7fb2c80b4b10 2024-04-25T00:33:33.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.754+0000 7fb2c6ffd640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb2b4067a70 0x7fb2b4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:33.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.758+0000 7fb2c6ffd640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb2b4067a70 0x7fb2b4069f30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80ae000 tx=0x7fb2b80a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:33.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.758+0000 7fb2affff640 1 -- 172.21.15.29:0/2016750590 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb2c0056ca0 con 0x7fb2c80b4b10 2024-04-25T00:33:33.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2016750590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb2c80d0790 con 0x7fb2c80b4b10 2024-04-25T00:33:33.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2affff640 1 -- 172.21.15.29:0/2016750590 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) ==== 104+0+11 (secure 0 0 0) 0x7fb2c00565a0 con 0x7fb2c80b4b10 2024-04-25T00:33:33.927 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:33:33.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb2b4067a70 msgr2=0x7fb2b4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2cd91f640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb2b4067a70 0x7fb2b4069f30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80ae000 tx=0x7fb2b80a7070 comp rx=0 tx=0).stop 2024-04-25T00:33:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2016750590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c80b4b10 msgr2=0x7fb2c815a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2cd91f640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c80b4b10 0x7fb2c815a960 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000d800 tx=0x7fb2c000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2016750590 shutdown_connections 2024-04-25T00:33:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2cd91f640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c80b4b10 0x7fb2c815a960 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2cd91f640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb2b4067a70 0x7fb2b4069f30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2cd91f640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c80b3c30 0x7fb2c815ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2cd91f640 1 --2- 172.21.15.29:0/2016750590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2c80b3140 0x7fb2c813df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2016750590 >> 172.21.15.29:0/2016750590 conn(0x7fb2c80b1870 msgr2=0x7fb2c81430f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2016750590 shutdown_connections 2024-04-25T00:33:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:33.922+0000 7fb2cd91f640 1 -- 172.21.15.29:0/2016750590 wait complete. 2024-04-25T00:33:33.946 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:33:34.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdfa650b640 1 Processor -- start 2024-04-25T00:33:34.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdfa650b640 1 -- start start 2024-04-25T00:33:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdfa650b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e200 0x7fdfa00d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdfa650b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa00cfc10 con 0x7fdfa00d2c60 2024-04-25T00:33:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdfa650b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa00cfef0 con 0x7fdfa00cf690 2024-04-25T00:33:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdfa650b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa00b38d0 con 0x7fdfa015e200 2024-04-25T00:33:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdfa4aa8640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdfa00d2c60 0x7fdfa00b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59664/0 (socket says 172.21.15.29:59664) 2024-04-25T00:33:34.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdfa4aa8640 1 -- 172.21.15.29:0/943125344 learned_addr learned my addr 172.21.15.29:0/943125344 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:34.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9ffff640 1 --2- 172.21.15.29:0/943125344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e200 0x7fdfa00d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1889183548 0 0) 0x7fdfa00cfc10 con 0x7fdfa00d2c60 2024-04-25T00:33:34.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf88003530 con 0x7fdfa00d2c60 2024-04-25T00:33:34.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 181199865 0 0) 0x7fdf88003530 con 0x7fdfa00d2c60 2024-04-25T00:33:34.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e200 msgr2=0x7fdfa00d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:34.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 --2- 172.21.15.29:0/943125344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e200 0x7fdfa00d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 >> v1:172.21.15.157:6789/0 conn(0x7fdfa00cf690 legacy=0x7fdfa00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa00b3b20 con 0x7fdfa00d2c60 2024-04-25T00:33:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fdf98001680 con 0x7fdfa00d2c60 2024-04-25T00:33:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 >> v1:172.21.15.29:6789/0 conn(0x7fdfa00d2c60 legacy=0x7fdfa00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 --2- 172.21.15.29:0/943125344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf880039f0 0x7fdf88003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 --2- 172.21.15.29:0/943125344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e200 0x7fdf88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 --2- 172.21.15.29:0/943125344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf88004d50 0x7fdf880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa00b38d0 con 0x7fdf880039f0 2024-04-25T00:33:34.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8800ba80 con 0x7fdf88004d50 2024-04-25T00:33:34.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8800bc80 con 0x7fdfa015e200 2024-04-25T00:33:34.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9ffff640 1 --2- 172.21.15.29:0/943125344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf880039f0 0x7fdf88003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9f7fe640 1 --2- 172.21.15.29:0/943125344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e200 0x7fdf88004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdfa4aa8640 1 --2- 172.21.15.29:0/943125344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf88004d50 0x7fdf880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9ffff640 1 -- 172.21.15.29:0/943125344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e200 msgr2=0x7fdf88004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:34.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9ffff640 1 --2- 172.21.15.29:0/943125344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e200 0x7fdf88004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9ffff640 1 -- 172.21.15.29:0/943125344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf88004d50 msgr2=0x7fdf880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9ffff640 1 --2- 172.21.15.29:0/943125344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf88004d50 0x7fdf880051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdf9ffff640 1 -- 172.21.15.29:0/943125344 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf88003530 con 0x7fdf880039f0 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.058+0000 7fdfa4aa8640 1 --2- 172.21.15.29:0/943125344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf88004d50 0x7fdf880051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.062+0000 7fdf9ffff640 1 --2- 172.21.15.29:0/943125344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf880039f0 0x7fdf88003e10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009ab0 tx=0x7fdf9001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22adc0976fc79784 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.062+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900b2070 con 0x7fdf880039f0 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.062+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf900a5520 con 0x7fdf880039f0 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.062+0000 7fdf9effd640 1 -- 172.21.15.29:0/943125344 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900096b0 con 0x7fdf880039f0 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.062+0000 7fdfa650b640 1 -- 172.21.15.29:0/943125344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf880039f0 msgr2=0x7fdf88003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.062+0000 7fdfa650b640 1 --2- 172.21.15.29:0/943125344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf880039f0 0x7fdf88003e10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009ab0 tx=0x7fdf9001ca20 comp rx=0 tx=0).stop 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.062+0000 7fdfa4aa8640 1 -- 172.21.15.29:0/943125344 reap_dead start 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.062+0000 7fdfa650b640 1 -- 172.21.15.29:0/943125344 shutdown_connections 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.062+0000 7fdfa650b640 1 -- 172.21.15.29:0/943125344 >> 172.21.15.29:0/943125344 conn(0x7fdfa009d280 msgr2=0x7fdfa00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.062+0000 7fdfa650b640 1 -- 172.21.15.29:0/943125344 shutdown_connections 2024-04-25T00:33:34.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.062+0000 7fdfa650b640 1 -- 172.21.15.29:0/943125344 wait complete. 2024-04-25T00:33:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.062+0000 7fdfa650b640 1 Processor -- start 2024-04-25T00:33:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.062+0000 7fdfa650b640 1 -- start start 2024-04-25T00:33:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdfa650b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf880039f0 0x7fdfa014f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdfa650b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdfa00cc170 0x7fdfa015fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdfa650b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa00cc5d0 0x7fdfa0160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdfa650b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa00b4e40 con 0x7fdfa00cc170 2024-04-25T00:33:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdfa650b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa00b4cc0 con 0x7fdf880039f0 2024-04-25T00:33:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdfa650b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa00b4fc0 con 0x7fdfa00cc5d0 2024-04-25T00:33:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdfa00cc170 0x7fdfa015fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdfa00cc170 0x7fdfa015fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37692/0 (socket says 172.21.15.29:37692) 2024-04-25T00:33:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9f7fe640 1 -- 172.21.15.29:0/3961993593 learned_addr learned my addr 172.21.15.29:0/3961993593 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdfa4aa8640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa00cc5d0 0x7fdfa0160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9ffff640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf880039f0 0x7fdfa014f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9f7fe640 1 -- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa00cc5d0 msgr2=0x7fdfa0160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9f7fe640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa00cc5d0 0x7fdfa0160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9f7fe640 1 -- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf880039f0 msgr2=0x7fdfa014f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9f7fe640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf880039f0 0x7fdfa014f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9f7fe640 1 -- 172.21.15.29:0/3961993593 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa00ca980 con 0x7fdfa00cc170 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9f7fe640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdfa00cc170 0x7fdfa015fe40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400e900 tx=0x7fdf9400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9cff9640 1 -- 172.21.15.29:0/3961993593 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94003b90 con 0x7fdfa00cc170 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9cff9640 1 -- 172.21.15.29:0/3961993593 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf94018070 con 0x7fdfa00cc170 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdfa650b640 1 -- 172.21.15.29:0/3961993593 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa00cbf00 con 0x7fdfa00cc170 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9cff9640 1 -- 172.21.15.29:0/3961993593 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9400c780 con 0x7fdfa00cc170 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdfa650b640 1 -- 172.21.15.29:0/3961993593 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa00c7b40 con 0x7fdfa00cc170 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdfa650b640 1 -- 172.21.15.29:0/3961993593 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa0069270 con 0x7fdfa00cc170 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9cff9640 1 -- 172.21.15.29:0/3961993593 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fdf9400c920 con 0x7fdfa00cc170 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9cff9640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdf8c067a70 0x7fdf8c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9cff9640 1 -- 172.21.15.29:0/3961993593 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fdf940860c0 con 0x7fdfa00cc170 2024-04-25T00:33:34.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.066+0000 7fdf9ffff640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdf8c067a70 0x7fdf8c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.070+0000 7fdf9ffff640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdf8c067a70 0x7fdf8c069f30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdf900af000 tx=0x7fdf900046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:34.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.070+0000 7fdf9cff9640 1 -- 172.21.15.29:0/3961993593 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdfa00ca980 con 0x7fdfa00cc170 2024-04-25T00:33:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.226+0000 7fdfa650b640 1 -- 172.21.15.29:0/3961993593 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdfa00d40c0 con 0x7fdfa00cc170 2024-04-25T00:33:34.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.230+0000 7fdf9cff9640 1 -- 172.21.15.29:0/3961993593 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) ==== 100+0+11 (secure 0 0 0) 0x7fdf9401a310 con 0x7fdfa00cc170 2024-04-25T00:33:34.234 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:33:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.230+0000 7fdfa650b640 1 -- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdf8c067a70 msgr2=0x7fdf8c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.230+0000 7fdfa650b640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdf8c067a70 0x7fdf8c069f30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdf900af000 tx=0x7fdf900046d0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.230+0000 7fdfa650b640 1 -- 172.21.15.29:0/3961993593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdfa00cc170 msgr2=0x7fdfa015fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.230+0000 7fdfa650b640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdfa00cc170 0x7fdfa015fe40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400e900 tx=0x7fdf9400edc0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.230+0000 7fdfa650b640 1 -- 172.21.15.29:0/3961993593 shutdown_connections 2024-04-25T00:33:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.230+0000 7fdfa650b640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa00cc5d0 0x7fdfa0160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.230+0000 7fdfa650b640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdfa00cc170 0x7fdfa015fe40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.230+0000 7fdfa650b640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdf8c067a70 0x7fdf8c069f30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.230+0000 7fdfa650b640 1 --2- 172.21.15.29:0/3961993593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf880039f0 0x7fdfa014f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.230+0000 7fdfa650b640 1 -- 172.21.15.29:0/3961993593 >> 172.21.15.29:0/3961993593 conn(0x7fdfa009d280 msgr2=0x7fdfa00d2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:34.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.230+0000 7fdfa650b640 1 -- 172.21.15.29:0/3961993593 shutdown_connections 2024-04-25T00:33:34.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.230+0000 7fdfa650b640 1 -- 172.21.15.29:0/3961993593 wait complete. 2024-04-25T00:33:34.252 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:33:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.374+0000 7fc40bfcc640 1 Processor -- start 2024-04-25T00:33:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.374+0000 7fc40bfcc640 1 -- start start 2024-04-25T00:33:34.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.374+0000 7fc40bfcc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4040d11f0 0x7fc4040cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.374+0000 7fc40bfcc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc404070060 0x7fc40415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc40bfcc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4040ce1a0 con 0x7fc4040cddc0 2024-04-25T00:33:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc40bfcc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc4040d0d60 con 0x7fc4040d11f0 2024-04-25T00:33:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc40bfcc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc4040cf540 con 0x7fc404070060 2024-04-25T00:33:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc409567640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4040d11f0 0x7fc4040cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc409567640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4040d11f0 0x7fc4040cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37702/0 (socket says 172.21.15.29:37702) 2024-04-25T00:33:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc409567640 1 -- 172.21.15.29:0/1126647092 learned_addr learned my addr 172.21.15.29:0/1126647092 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc40a569640 1 --2- 172.21.15.29:0/1126647092 >> v2:172.21.15.157:3300/0 conn(0x7fc404070060 0x7fc40415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc409567640 1 -- 172.21.15.29:0/1126647092 >> v2:172.21.15.157:3300/0 conn(0x7fc404070060 msgr2=0x7fc40415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc409567640 1 --2- 172.21.15.29:0/1126647092 >> v2:172.21.15.157:3300/0 conn(0x7fc404070060 0x7fc40415d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc409567640 1 -- 172.21.15.29:0/1126647092 >> v1:172.21.15.157:6789/0 conn(0x7fc4040cddc0 legacy=0x7fc40406f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc409567640 1 -- 172.21.15.29:0/1126647092 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4040d0150 con 0x7fc4040d11f0 2024-04-25T00:33:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc409567640 1 --2- 172.21.15.29:0/1126647092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4040d11f0 0x7fc4040cf000 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc404162ed0 tx=0x7fc3f401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1c0dfd4745d4bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc408d66640 1 -- 172.21.15.29:0/1126647092 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f40b0070 con 0x7fc4040d11f0 2024-04-25T00:33:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc408d66640 1 -- 172.21.15.29:0/1126647092 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f40ab400 con 0x7fc4040d11f0 2024-04-25T00:33:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc408d66640 1 -- 172.21.15.29:0/1126647092 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f40ab7d0 con 0x7fc4040d11f0 2024-04-25T00:33:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc40bfcc640 1 -- 172.21.15.29:0/1126647092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4040d11f0 msgr2=0x7fc4040cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc40bfcc640 1 --2- 172.21.15.29:0/1126647092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4040d11f0 0x7fc4040cf000 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc404162ed0 tx=0x7fc3f401c450 comp rx=0 tx=0).stop 2024-04-25T00:33:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc40bfcc640 1 -- 172.21.15.29:0/1126647092 shutdown_connections 2024-04-25T00:33:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc40bfcc640 1 --2- 172.21.15.29:0/1126647092 >> v2:172.21.15.157:3300/0 conn(0x7fc404070060 0x7fc40415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc40bfcc640 1 --2- 172.21.15.29:0/1126647092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4040d11f0 0x7fc4040cf000 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc40bfcc640 1 -- 172.21.15.29:0/1126647092 >> 172.21.15.29:0/1126647092 conn(0x7fc4040b3920 msgr2=0x7fc4040b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc40bfcc640 1 -- 172.21.15.29:0/1126647092 shutdown_connections 2024-04-25T00:33:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.378+0000 7fc40bfcc640 1 -- 172.21.15.29:0/1126647092 wait complete. 2024-04-25T00:33:34.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40bfcc640 1 Processor -- start 2024-04-25T00:33:34.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40bfcc640 1 -- start start 2024-04-25T00:33:34.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40bfcc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc404070060 0x7fc4040b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40bfcc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4040cddc0 0x7fc4041563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40bfcc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4040d11f0 0x7fc4041568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40bfcc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc4040cc540 con 0x7fc4040d11f0 2024-04-25T00:33:34.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40bfcc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4040cc3c0 con 0x7fc404070060 2024-04-25T00:33:34.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40bfcc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4040cc6c0 con 0x7fc4040cddc0 2024-04-25T00:33:34.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40a569640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4040d11f0 0x7fc4041568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40a569640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4040d11f0 0x7fc4041568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37704/0 (socket says 172.21.15.29:37704) 2024-04-25T00:33:34.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40a569640 1 -- 172.21.15.29:0/2518419860 learned_addr learned my addr 172.21.15.29:0/2518419860 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:34.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc409567640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4040cddc0 0x7fc4041563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc409d68640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc404070060 0x7fc4040b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40a569640 1 -- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4040cddc0 msgr2=0x7fc4041563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40a569640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4040cddc0 0x7fc4041563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40a569640 1 -- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc404070060 msgr2=0x7fc4040b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40a569640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc404070060 0x7fc4040b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40a569640 1 -- 172.21.15.29:0/2518419860 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc404154ca0 con 0x7fc4040d11f0 2024-04-25T00:33:34.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc409d68640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc404070060 0x7fc4040b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:34.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40a569640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4040d11f0 0x7fc4041568f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00cd70 tx=0x7fc3fc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:34.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc3fa7fc640 1 -- 172.21.15.29:0/2518419860 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc003ba0 con 0x7fc4040d11f0 2024-04-25T00:33:34.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc3fa7fc640 1 -- 172.21.15.29:0/2518419860 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3fc003d40 con 0x7fc4040d11f0 2024-04-25T00:33:34.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40bfcc640 1 -- 172.21.15.29:0/2518419860 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc404154f80 con 0x7fc4040d11f0 2024-04-25T00:33:34.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc3fa7fc640 1 -- 172.21.15.29:0/2518419860 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc004240 con 0x7fc4040d11f0 2024-04-25T00:33:34.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.382+0000 7fc40bfcc640 1 -- 172.21.15.29:0/2518419860 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc404155540 con 0x7fc4040d11f0 2024-04-25T00:33:34.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.386+0000 7fc40bfcc640 1 -- 172.21.15.29:0/2518419860 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc404069270 con 0x7fc4040d11f0 2024-04-25T00:33:34.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.386+0000 7fc3fa7fc640 1 -- 172.21.15.29:0/2518419860 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fc3fc0043e0 con 0x7fc4040d11f0 2024-04-25T00:33:34.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.386+0000 7fc3fa7fc640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc3e8067a70 0x7fc3e8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.386+0000 7fc3fa7fc640 1 -- 172.21.15.29:0/2518419860 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc3fc086b90 con 0x7fc4040d11f0 2024-04-25T00:33:34.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.386+0000 7fc409d68640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc3e8067a70 0x7fc3e8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.386+0000 7fc409d68640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc3e8067a70 0x7fc3e8069f30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc4040b7e60 tx=0x7fc400009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:34.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.390+0000 7fc3fa7fc640 1 -- 172.21.15.29:0/2518419860 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc3fc0573d0 con 0x7fc4040d11f0 2024-04-25T00:33:34.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.562+0000 7fc40bfcc640 1 -- 172.21.15.29:0/2518419860 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc4040b4440 con 0x7fc4040d11f0 2024-04-25T00:33:34.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.566+0000 7fc3fa7fc640 1 -- 172.21.15.29:0/2518419860 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7fc3fc056cd0 con 0x7fc4040d11f0 2024-04-25T00:33:34.569 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:33:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.566+0000 7fc40bfcc640 1 -- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc3e8067a70 msgr2=0x7fc3e8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.566+0000 7fc40bfcc640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc3e8067a70 0x7fc3e8069f30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc4040b7e60 tx=0x7fc400009040 comp rx=0 tx=0).stop 2024-04-25T00:33:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.566+0000 7fc40bfcc640 1 -- 172.21.15.29:0/2518419860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4040d11f0 msgr2=0x7fc4041568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.566+0000 7fc40bfcc640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4040d11f0 0x7fc4041568f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00cd70 tx=0x7fc3fc00a660 comp rx=0 tx=0).stop 2024-04-25T00:33:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.566+0000 7fc40bfcc640 1 -- 172.21.15.29:0/2518419860 shutdown_connections 2024-04-25T00:33:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.566+0000 7fc40bfcc640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4040d11f0 0x7fc4041568f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.566+0000 7fc40bfcc640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4040cddc0 0x7fc4041563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.566+0000 7fc40bfcc640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc3e8067a70 0x7fc3e8069f30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.566+0000 7fc40bfcc640 1 --2- 172.21.15.29:0/2518419860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc404070060 0x7fc4040b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.566+0000 7fc40bfcc640 1 -- 172.21.15.29:0/2518419860 >> 172.21.15.29:0/2518419860 conn(0x7fc4040b3920 msgr2=0x7fc40406f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:34.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.566+0000 7fc40bfcc640 1 -- 172.21.15.29:0/2518419860 shutdown_connections 2024-04-25T00:33:34.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.566+0000 7fc40bfcc640 1 -- 172.21.15.29:0/2518419860 wait complete. 2024-04-25T00:33:34.588 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-04-25T00:33:34.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:34.588 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:33:34.599 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[39466]: fuse finished with error 0 and tester_r 0 2024-04-25T00:33:34.600 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:33:34.687 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:33:34.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:34.687 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:33:34.694 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:33:34.694 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:33:34.701 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:33:34.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2d0923640 1 Processor -- start 2024-04-25T00:33:34.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2d0923640 1 -- start start 2024-04-25T00:33:34.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2d0923640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd2cc15bd60 0x7fd2cc06b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2d0923640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd2cc06dd10 0x7fd2cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2d0923640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2cc0cb330 0x7fd2cc15f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2d0923640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd2cc06edb0 con 0x7fd2cc15bd60 2024-04-25T00:33:34.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2d0923640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd2cc06e1a0 con 0x7fd2cc06dd10 2024-04-25T00:33:34.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2d0923640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd2cc06e480 con 0x7fd2cc0cb330 2024-04-25T00:33:34.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2ca59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd2cc15bd60 0x7fd2cc06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2ca59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd2cc15bd60 0x7fd2cc06b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37710/0 (socket says 172.21.15.29:37710) 2024-04-25T00:33:34.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2ca59c640 1 -- 172.21.15.29:0/882788173 learned_addr learned my addr 172.21.15.29:0/882788173 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:34.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2ca59c640 1 -- 172.21.15.29:0/882788173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2cc0cb330 msgr2=0x7fd2cc15f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:34.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2cad9d640 1 --2- 172.21.15.29:0/882788173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2cc0cb330 0x7fd2cc15f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2c9d9b640 1 --2- 172.21.15.29:0/882788173 >> v2:172.21.15.157:3300/0 conn(0x7fd2cc06dd10 0x7fd2cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2ca59c640 1 --2- 172.21.15.29:0/882788173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2cc0cb330 0x7fd2cc15f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2ca59c640 1 -- 172.21.15.29:0/882788173 >> v2:172.21.15.157:3300/0 conn(0x7fd2cc06dd10 msgr2=0x7fd2cc06b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:34.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2ca59c640 1 --2- 172.21.15.29:0/882788173 >> v2:172.21.15.157:3300/0 conn(0x7fd2cc06dd10 0x7fd2cc06b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2ca59c640 1 -- 172.21.15.29:0/882788173 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc06e760 con 0x7fd2cc15bd60 2024-04-25T00:33:34.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.866+0000 7fd2c9d9b640 1 --2- 172.21.15.29:0/882788173 >> v2:172.21.15.157:3300/0 conn(0x7fd2cc06dd10 0x7fd2cc06b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:34.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.870+0000 7fd2ca59c640 1 --2- 172.21.15.29:0/882788173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2cc15bd60 0x7fd2cc06b210 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0ce730 tx=0x7fd2bc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eceddb9db426d931 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:34.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.870+0000 7fd2c959a640 1 -- 172.21.15.29:0/882788173 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0aa430 con 0x7fd2cc15bd60 2024-04-25T00:33:34.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.870+0000 7fd2c959a640 1 -- 172.21.15.29:0/882788173 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2bc0af070 con 0x7fd2cc15bd60 2024-04-25T00:33:34.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.870+0000 7fd2c959a640 1 -- 172.21.15.29:0/882788173 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0aa7f0 con 0x7fd2cc15bd60 2024-04-25T00:33:34.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.870+0000 7fd2d0923640 1 -- 172.21.15.29:0/882788173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2cc15bd60 msgr2=0x7fd2cc06b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.870+0000 7fd2d0923640 1 --2- 172.21.15.29:0/882788173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2cc15bd60 0x7fd2cc06b210 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0ce730 tx=0x7fd2bc01c4f0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.870+0000 7fd2d0923640 1 -- 172.21.15.29:0/882788173 shutdown_connections 2024-04-25T00:33:34.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.870+0000 7fd2d0923640 1 --2- 172.21.15.29:0/882788173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2cc0cb330 0x7fd2cc15f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.870+0000 7fd2d0923640 1 --2- 172.21.15.29:0/882788173 >> v2:172.21.15.157:3300/0 conn(0x7fd2cc06dd10 0x7fd2cc06b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.870+0000 7fd2d0923640 1 --2- 172.21.15.29:0/882788173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2cc15bd60 0x7fd2cc06b210 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.870+0000 7fd2d0923640 1 -- 172.21.15.29:0/882788173 >> 172.21.15.29:0/882788173 conn(0x7fd2cc06c320 msgr2=0x7fd2cc06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:34.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.870+0000 7fd2d0923640 1 -- 172.21.15.29:0/882788173 shutdown_connections 2024-04-25T00:33:34.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.870+0000 7fd2d0923640 1 -- 172.21.15.29:0/882788173 wait complete. 2024-04-25T00:33:34.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2d0923640 1 Processor -- start 2024-04-25T00:33:34.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2d0923640 1 -- start start 2024-04-25T00:33:34.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2d0923640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2cc0cb330 0x7fd2cc0c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2d0923640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2cc15bd60 0x7fd2cc153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2d0923640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2cc154210 0x7fd2cc14b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2d0923640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc0cce10 con 0x7fd2cc0cb330 2024-04-25T00:33:34.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2d0923640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc0ccc90 con 0x7fd2cc154210 2024-04-25T00:33:34.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2d0923640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd2cc0ccf90 con 0x7fd2cc15bd60 2024-04-25T00:33:34.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2ca59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2cc0cb330 0x7fd2cc0c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2ca59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2cc0cb330 0x7fd2cc0c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37720/0 (socket says 172.21.15.29:37720) 2024-04-25T00:33:34.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2ca59c640 1 -- 172.21.15.29:0/179984139 learned_addr learned my addr 172.21.15.29:0/179984139 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:34.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2c9d9b640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2cc15bd60 0x7fd2cc153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2cad9d640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2cc154210 0x7fd2cc14b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2ca59c640 1 -- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2cc15bd60 msgr2=0x7fd2cc153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2ca59c640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2cc15bd60 0x7fd2cc153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2ca59c640 1 -- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2cc154210 msgr2=0x7fd2cc14b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:34.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2ca59c640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2cc154210 0x7fd2cc14b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:34.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2ca59c640 1 -- 172.21.15.29:0/179984139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc1546a0 con 0x7fd2cc0cb330 2024-04-25T00:33:34.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2ca59c640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2cc0cb330 0x7fd2cc0c7d40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc01aba0 tx=0x7fd2bc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:34.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2baffd640 1 -- 172.21.15.29:0/179984139 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0bd5f0 con 0x7fd2cc0cb330 2024-04-25T00:33:34.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2baffd640 1 -- 172.21.15.29:0/179984139 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2bc0bd7b0 con 0x7fd2cc0cb330 2024-04-25T00:33:34.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2d0923640 1 -- 172.21.15.29:0/179984139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2cc152f10 con 0x7fd2cc0cb330 2024-04-25T00:33:34.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2baffd640 1 -- 172.21.15.29:0/179984139 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0bdba0 con 0x7fd2cc0cb330 2024-04-25T00:33:34.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.874+0000 7fd2d0923640 1 -- 172.21.15.29:0/179984139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2cc1534d0 con 0x7fd2cc0cb330 2024-04-25T00:33:34.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.878+0000 7fd2d0923640 1 -- 172.21.15.29:0/179984139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2cc069270 con 0x7fd2cc0cb330 2024-04-25T00:33:34.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.878+0000 7fd2baffd640 1 -- 172.21.15.29:0/179984139 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fd2bc0b33f0 con 0x7fd2cc0cb330 2024-04-25T00:33:34.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.878+0000 7fd2baffd640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2ac067a70 0x7fd2ac069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:34.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.878+0000 7fd2baffd640 1 -- 172.21.15.29:0/179984139 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fd2bc120d10 con 0x7fd2cc0cb330 2024-04-25T00:33:34.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.878+0000 7fd2c9d9b640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2ac067a70 0x7fd2ac069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:34.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.878+0000 7fd2c9d9b640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2ac067a70 0x7fd2ac069f30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0006fd0 tx=0x7fd2c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:34.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:34.882+0000 7fd2baffd640 1 -- 172.21.15.29:0/179984139 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd2bc126020 con 0x7fd2cc0cb330 2024-04-25T00:33:35.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.042+0000 7fd2d0923640 1 -- 172.21.15.29:0/179984139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2cc0b2580 con 0x7fd2cc0cb330 2024-04-25T00:33:35.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.046+0000 7fd2baffd640 1 -- 172.21.15.29:0/179984139 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16565 (secure 0 0 0) 0x7fd2bc0c3060 con 0x7fd2cc0cb330 2024-04-25T00:33:35.052 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:35.052 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":45,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:33:14.784398+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:33:10.789862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:33:12.129841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:33:35.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.050+0000 7fd2d0923640 1 -- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2ac067a70 msgr2=0x7fd2ac069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.050+0000 7fd2d0923640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2ac067a70 0x7fd2ac069f30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0006fd0 tx=0x7fd2c0008040 comp rx=0 tx=0).stop 2024-04-25T00:33:35.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.050+0000 7fd2d0923640 1 -- 172.21.15.29:0/179984139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2cc0cb330 msgr2=0x7fd2cc0c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.050+0000 7fd2d0923640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2cc0cb330 0x7fd2cc0c7d40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc01aba0 tx=0x7fd2bc01ad30 comp rx=0 tx=0).stop 2024-04-25T00:33:35.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.050+0000 7fd2d0923640 1 -- 172.21.15.29:0/179984139 shutdown_connections 2024-04-25T00:33:35.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.050+0000 7fd2d0923640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd2ac067a70 0x7fd2ac069f30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.050+0000 7fd2d0923640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2cc154210 0x7fd2cc14b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.050+0000 7fd2d0923640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2cc15bd60 0x7fd2cc153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.050+0000 7fd2d0923640 1 --2- 172.21.15.29:0/179984139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2cc0cb330 0x7fd2cc0c7d40 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.050+0000 7fd2d0923640 1 -- 172.21.15.29:0/179984139 >> 172.21.15.29:0/179984139 conn(0x7fd2cc06c320 msgr2=0x7fd2cc0b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:35.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.050+0000 7fd2d0923640 1 -- 172.21.15.29:0/179984139 shutdown_connections 2024-04-25T00:33:35.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.050+0000 7fd2d0923640 1 -- 172.21.15.29:0/179984139 wait complete. 2024-04-25T00:33:35.071 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:33:10.789862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:33:12.129841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:33:35.071 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:33:35.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 Processor -- start 2024-04-25T00:33:35.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 -- start start 2024-04-25T00:33:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce700d4d10 0x7fce700d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fce700b4040 0x7fce700b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce700cf8a0 con 0x7fce7015e2a0 2024-04-25T00:33:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fce700d19a0 con 0x7fce700b4040 2024-04-25T00:33:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fce700cf650 con 0x7fce700d4d10 2024-04-25T00:33:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce6f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fce700b4040 0x7fce700b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce6effd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fce7015e2a0 0x7fce700d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59678/0 (socket says 172.21.15.29:59678) 2024-04-25T00:33:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce6effd640 1 -- 172.21.15.29:0/1912058025 learned_addr learned my addr 172.21.15.29:0/1912058025 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:35.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce6e7fc640 1 --2- 172.21.15.29:0/1912058025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce700d4d10 0x7fce700d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce6f7fe640 1 -- 172.21.15.29:0/1912058025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce700d4d10 msgr2=0x7fce700d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:35.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce6f7fe640 1 --2- 172.21.15.29:0/1912058025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce700d4d10 0x7fce700d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce6f7fe640 1 -- 172.21.15.29:0/1912058025 >> v1:172.21.15.29:6789/0 conn(0x7fce7015e2a0 legacy=0x7fce700d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce6f7fe640 1 -- 172.21.15.29:0/1912058025 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce700cfb80 con 0x7fce700b4040 2024-04-25T00:33:35.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce6e7fc640 1 --2- 172.21.15.29:0/1912058025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce700d4d10 0x7fce700d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:35.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce6f7fe640 1 --2- 172.21.15.29:0/1912058025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce700b4040 0x7fce700b44f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fce68009d90 tx=0x7fce6801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2533cca2c2b0f7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:35.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce6dffb640 1 -- 172.21.15.29:0/1912058025 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce680b2070 con 0x7fce700b4040 2024-04-25T00:33:35.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce6dffb640 1 -- 172.21.15.29:0/1912058025 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce680a57a0 con 0x7fce700b4040 2024-04-25T00:33:35.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce6dffb640 1 -- 172.21.15.29:0/1912058025 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce680a5b70 con 0x7fce700b4040 2024-04-25T00:33:35.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 -- 172.21.15.29:0/1912058025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce700b4040 msgr2=0x7fce700b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 --2- 172.21.15.29:0/1912058025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce700b4040 0x7fce700b44f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fce68009d90 tx=0x7fce6801c8e0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 -- 172.21.15.29:0/1912058025 shutdown_connections 2024-04-25T00:33:35.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 --2- 172.21.15.29:0/1912058025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce700b4040 0x7fce700b44f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 --2- 172.21.15.29:0/1912058025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce700d4d10 0x7fce700d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 -- 172.21.15.29:0/1912058025 >> 172.21.15.29:0/1912058025 conn(0x7fce700d4030 msgr2=0x7fce700d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:35.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 -- 172.21.15.29:0/1912058025 shutdown_connections 2024-04-25T00:33:35.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.198+0000 7fce75878640 1 -- 172.21.15.29:0/1912058025 wait complete. 2024-04-25T00:33:35.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce75878640 1 Processor -- start 2024-04-25T00:33:35.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce75878640 1 -- start start 2024-04-25T00:33:35.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce75878640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce700b4040 0x7fce7015c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce75878640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce700d4d10 0x7fce7014d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce75878640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7015e2a0 0x7fce7015e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce75878640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fce7016aef0 con 0x7fce700b4040 2024-04-25T00:33:35.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce75878640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fce700d2a60 con 0x7fce700d4d10 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce75878640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fce7016b070 con 0x7fce7015e2a0 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce6effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce700b4040 0x7fce7015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce6effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce700b4040 0x7fce7015c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37730/0 (socket says 172.21.15.29:37730) 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce6effd640 1 -- 172.21.15.29:0/3273867292 learned_addr learned my addr 172.21.15.29:0/3273867292 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce6f7fe640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7015e2a0 0x7fce7015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce6e7fc640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce700d4d10 0x7fce7014d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce6effd640 1 -- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7015e2a0 msgr2=0x7fce7015e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce6effd640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7015e2a0 0x7fce7015e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce6effd640 1 -- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce700d4d10 msgr2=0x7fce7014d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce6effd640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce700d4d10 0x7fce7014d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce6effd640 1 -- 172.21.15.29:0/3273867292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7014da20 con 0x7fce700b4040 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce6effd640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce700b4040 0x7fce7015c060 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fce6000c450 tx=0x7fce6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce67fff640 1 -- 172.21.15.29:0/3273867292 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60017070 con 0x7fce700b4040 2024-04-25T00:33:35.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce67fff640 1 -- 172.21.15.29:0/3273867292 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce60005cf0 con 0x7fce700b4040 2024-04-25T00:33:35.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce67fff640 1 -- 172.21.15.29:0/3273867292 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60007290 con 0x7fce700b4040 2024-04-25T00:33:35.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce75878640 1 -- 172.21.15.29:0/3273867292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7015d400 con 0x7fce700b4040 2024-04-25T00:33:35.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce75878640 1 -- 172.21.15.29:0/3273867292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce7015d9c0 con 0x7fce700b4040 2024-04-25T00:33:35.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.202+0000 7fce75878640 1 -- 172.21.15.29:0/3273867292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce70069270 con 0x7fce700b4040 2024-04-25T00:33:35.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.206+0000 7fce67fff640 1 -- 172.21.15.29:0/3273867292 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fce6001a070 con 0x7fce700b4040 2024-04-25T00:33:35.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.206+0000 7fce67fff640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fce50067a70 0x7fce50069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.206+0000 7fce67fff640 1 -- 172.21.15.29:0/3273867292 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fce600852b0 con 0x7fce700b4040 2024-04-25T00:33:35.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.206+0000 7fce6e7fc640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fce50067a70 0x7fce50069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.206+0000 7fce6e7fc640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fce50067a70 0x7fce50069f30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce58006fd0 tx=0x7fce58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:35.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.214+0000 7fce67fff640 1 -- 172.21.15.29:0/3273867292 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fce600554a0 con 0x7fce700b4040 2024-04-25T00:33:35.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.370+0000 7fce75878640 1 -- 172.21.15.29:0/3273867292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fce700d3e10 con 0x7fce700b4040 2024-04-25T00:33:35.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.370+0000 7fce67fff640 1 -- 172.21.15.29:0/3273867292 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7fce600597a0 con 0x7fce700b4040 2024-04-25T00:33:35.376 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:33:35.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.378+0000 7fce75878640 1 -- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fce50067a70 msgr2=0x7fce50069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.378+0000 7fce75878640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fce50067a70 0x7fce50069f30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce58006fd0 tx=0x7fce58008040 comp rx=0 tx=0).stop 2024-04-25T00:33:35.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.378+0000 7fce75878640 1 -- 172.21.15.29:0/3273867292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce700b4040 msgr2=0x7fce7015c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.378+0000 7fce75878640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce700b4040 0x7fce7015c060 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fce6000c450 tx=0x7fce6000c910 comp rx=0 tx=0).stop 2024-04-25T00:33:35.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.378+0000 7fce75878640 1 -- 172.21.15.29:0/3273867292 shutdown_connections 2024-04-25T00:33:35.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.378+0000 7fce75878640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce7015e2a0 0x7fce7015e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.378+0000 7fce75878640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fce50067a70 0x7fce50069f30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.378+0000 7fce75878640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce700d4d10 0x7fce7014d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.378+0000 7fce75878640 1 --2- 172.21.15.29:0/3273867292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce700b4040 0x7fce7015c060 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.378+0000 7fce75878640 1 -- 172.21.15.29:0/3273867292 >> 172.21.15.29:0/3273867292 conn(0x7fce700d4030 msgr2=0x7fce7015a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:35.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.378+0000 7fce75878640 1 -- 172.21.15.29:0/3273867292 shutdown_connections 2024-04-25T00:33:35.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.378+0000 7fce75878640 1 -- 172.21.15.29:0/3273867292 wait complete. 2024-04-25T00:33:35.396 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:33:35.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 Processor -- start 2024-04-25T00:33:35.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 -- start start 2024-04-25T00:33:35.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4da015f2b0 0x7f4da00b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4da00b38c0 0x7f4da0161cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4da01443f0 con 0x7f4da00d09c0 2024-04-25T00:33:35.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4da015ee20 con 0x7f4da015f2b0 2024-04-25T00:33:35.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4da00b3cf0 con 0x7f4da00b38c0 2024-04-25T00:33:35.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4d9ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4da015f2b0 0x7f4da00b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4d9ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4da015f2b0 0x7f4da00b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37734/0 (socket says 172.21.15.29:37734) 2024-04-25T00:33:35.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4d9ffff640 1 -- 172.21.15.29:0/1338105870 learned_addr learned my addr 172.21.15.29:0/1338105870 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:35.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da532f640 1 --2- 172.21.15.29:0/1338105870 >> v2:172.21.15.157:3300/0 conn(0x7f4da00b38c0 0x7f4da0161cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4d9ffff640 1 -- 172.21.15.29:0/1338105870 >> v2:172.21.15.157:3300/0 conn(0x7f4da00b38c0 msgr2=0x7f4da0161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:35.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4d9ffff640 1 --2- 172.21.15.29:0/1338105870 >> v2:172.21.15.157:3300/0 conn(0x7f4da00b38c0 0x7f4da0161cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4d9ffff640 1 -- 172.21.15.29:0/1338105870 >> v1:172.21.15.157:6789/0 conn(0x7f4da00d09c0 legacy=0x7f4da0143880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4d9ffff640 1 -- 172.21.15.29:0/1338105870 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da015e7d0 con 0x7f4da015f2b0 2024-04-25T00:33:35.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4d9ffff640 1 --2- 172.21.15.29:0/1338105870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4da015f2b0 0x7f4da00b3380 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4da015c290 tx=0x7f4d8c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794d6885acde4bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:35.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4d9f7fe640 1 -- 172.21.15.29:0/1338105870 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0aa660 con 0x7f4da015f2b0 2024-04-25T00:33:35.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4d9f7fe640 1 -- 172.21.15.29:0/1338105870 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d8c0af070 con 0x7f4da015f2b0 2024-04-25T00:33:35.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4d9f7fe640 1 -- 172.21.15.29:0/1338105870 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0aaa20 con 0x7f4da015f2b0 2024-04-25T00:33:35.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 -- 172.21.15.29:0/1338105870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4da015f2b0 msgr2=0x7f4da00b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 --2- 172.21.15.29:0/1338105870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4da015f2b0 0x7f4da00b3380 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4da015c290 tx=0x7f4d8c01c520 comp rx=0 tx=0).stop 2024-04-25T00:33:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 -- 172.21.15.29:0/1338105870 shutdown_connections 2024-04-25T00:33:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 --2- 172.21.15.29:0/1338105870 >> v2:172.21.15.157:3300/0 conn(0x7f4da00b38c0 0x7f4da0161cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 --2- 172.21.15.29:0/1338105870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4da015f2b0 0x7f4da00b3380 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 -- 172.21.15.29:0/1338105870 >> 172.21.15.29:0/1338105870 conn(0x7f4da0074c00 msgr2=0x7f4da006b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 -- 172.21.15.29:0/1338105870 shutdown_connections 2024-04-25T00:33:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.518+0000 7f4da6d92640 1 -- 172.21.15.29:0/1338105870 wait complete. 2024-04-25T00:33:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da6d92640 1 Processor -- start 2024-04-25T00:33:35.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da6d92640 1 -- start start 2024-04-25T00:33:35.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da6d92640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4da00b38c0 0x7f4da0140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da6d92640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da00d09c0 0x7f4da013a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da6d92640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da015f2b0 0x7f4da0157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da6d92640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0144110 con 0x7f4da00b38c0 2024-04-25T00:33:35.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da6d92640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0143f90 con 0x7f4da015f2b0 2024-04-25T00:33:35.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da6d92640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4da016b050 con 0x7f4da00d09c0 2024-04-25T00:33:35.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da4b2e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4da00b38c0 0x7f4da0140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da4b2e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4da00b38c0 0x7f4da0140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37740/0 (socket says 172.21.15.29:37740) 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da4b2e640 1 -- 172.21.15.29:0/2741928435 learned_addr learned my addr 172.21.15.29:0/2741928435 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da532f640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da015f2b0 0x7f4da0157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4d9ffff640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da00d09c0 0x7f4da013a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da4b2e640 1 -- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da00d09c0 msgr2=0x7f4da013a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da4b2e640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da00d09c0 0x7f4da013a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da4b2e640 1 -- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da015f2b0 msgr2=0x7f4da0157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da4b2e640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da015f2b0 0x7f4da0157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da4b2e640 1 -- 172.21.15.29:0/2741928435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da013ffe0 con 0x7f4da00b38c0 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4d9ffff640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da00d09c0 0x7f4da013a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da532f640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da015f2b0 0x7f4da0157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da4b2e640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4da00b38c0 0x7f4da0140da0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400c450 tx=0x7f4d9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4d9d7fa640 1 -- 172.21.15.29:0/2741928435 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d94016020 con 0x7f4da00b38c0 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da6d92640 1 -- 172.21.15.29:0/2741928435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da01402c0 con 0x7f4da00b38c0 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4d9d7fa640 1 -- 172.21.15.29:0/2741928435 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d940052d0 con 0x7f4da00b38c0 2024-04-25T00:33:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4d9d7fa640 1 -- 172.21.15.29:0/2741928435 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d94007260 con 0x7f4da00b38c0 2024-04-25T00:33:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da6d92640 1 -- 172.21.15.29:0/2741928435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da0140880 con 0x7f4da00b38c0 2024-04-25T00:33:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4da6d92640 1 -- 172.21.15.29:0/2741928435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da0069270 con 0x7f4da00b38c0 2024-04-25T00:33:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.522+0000 7f4d9d7fa640 1 -- 172.21.15.29:0/2741928435 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f4d94007400 con 0x7f4da00b38c0 2024-04-25T00:33:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.526+0000 7f4d9d7fa640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4d84067a70 0x7f4d84069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.526+0000 7f4d9d7fa640 1 -- 172.21.15.29:0/2741928435 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f4d94085960 con 0x7f4da00b38c0 2024-04-25T00:33:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.526+0000 7f4d9ffff640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4d84067a70 0x7f4d84069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.526+0000 7f4d9ffff640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4d84067a70 0x7f4d84069f30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0ae000 tx=0x7f4d8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:35.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.530+0000 7f4d9d7fa640 1 -- 172.21.15.29:0/2741928435 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4d94020020 con 0x7f4da00b38c0 2024-04-25T00:33:35.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.690+0000 7f4da6d92640 1 -- 172.21.15.29:0/2741928435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4da0158a90 con 0x7f4da00b38c0 2024-04-25T00:33:35.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.690+0000 7f4d9d7fa640 1 -- 172.21.15.29:0/2741928435 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) ==== 104+0+11 (secure 0 0 0) 0x7f4d94005480 con 0x7f4da00b38c0 2024-04-25T00:33:35.695 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:33:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.690+0000 7f4da6d92640 1 -- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4d84067a70 msgr2=0x7f4d84069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.690+0000 7f4da6d92640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4d84067a70 0x7f4d84069f30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0ae000 tx=0x7f4d8c008040 comp rx=0 tx=0).stop 2024-04-25T00:33:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.690+0000 7f4da6d92640 1 -- 172.21.15.29:0/2741928435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4da00b38c0 msgr2=0x7f4da0140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.690+0000 7f4da6d92640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4da00b38c0 0x7f4da0140da0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400c450 tx=0x7f4d9400c910 comp rx=0 tx=0).stop 2024-04-25T00:33:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.690+0000 7f4da6d92640 1 -- 172.21.15.29:0/2741928435 shutdown_connections 2024-04-25T00:33:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.690+0000 7f4da6d92640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4d84067a70 0x7f4d84069f30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.690+0000 7f4da6d92640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da015f2b0 0x7f4da0157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.690+0000 7f4da6d92640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da00d09c0 0x7f4da013a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.690+0000 7f4da6d92640 1 --2- 172.21.15.29:0/2741928435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4da00b38c0 0x7f4da0140da0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.694+0000 7f4da6d92640 1 -- 172.21.15.29:0/2741928435 >> 172.21.15.29:0/2741928435 conn(0x7f4da0074c00 msgr2=0x7f4da0160d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.694+0000 7f4da6d92640 1 -- 172.21.15.29:0/2741928435 shutdown_connections 2024-04-25T00:33:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.694+0000 7f4da6d92640 1 -- 172.21.15.29:0/2741928435 wait complete. 2024-04-25T00:33:35.714 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:33:35.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 Processor -- start 2024-04-25T00:33:35.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 -- start start 2024-04-25T00:33:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f12380715b0 0x7f1238071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12380aec20 con 0x7f12380afcc0 2024-04-25T00:33:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12380aef00 con 0x7f123815e160 2024-04-25T00:33:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f123815e540 con 0x7f12380715b0 2024-04-25T00:33:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123caf5640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f12380afcc0 0x7f12380ae420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59684/0 (socket says 172.21.15.29:59684) 2024-04-25T00:33:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123caf5640 1 -- 172.21.15.29:0/1189242966 learned_addr learned my addr 172.21.15.29:0/1189242966 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:35.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123daf7640 1 --2- 172.21.15.29:0/1189242966 >> v2:172.21.15.157:3300/0 conn(0x7f12380715b0 0x7f1238071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1524193788 0 0) 0x7f12380aec20 con 0x7f12380afcc0 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1224003530 con 0x7f12380afcc0 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1104789486 0 0) 0x7f1224003530 con 0x7f12380afcc0 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 >> v2:172.21.15.157:3300/0 conn(0x7f12380715b0 msgr2=0x7f1238071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 --2- 172.21.15.29:0/1189242966 >> v2:172.21.15.157:3300/0 conn(0x7f12380715b0 0x7f1238071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 >> v1:172.21.15.157:6789/0 conn(0x7f123815e160 legacy=0x7f12381626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12380af1e0 con 0x7f12380afcc0 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f122c001680 con 0x7f12380afcc0 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 >> v1:172.21.15.29:6789/0 conn(0x7f12380afcc0 legacy=0x7f12380ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 --2- 172.21.15.29:0/1189242966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12240039c0 0x7f1224003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 --2- 172.21.15.29:0/1189242966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1224004810 0x7f1224004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 --2- 172.21.15.29:0/1189242966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12240051d0 0x7f122400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f123815e540 con 0x7f12240039c0 2024-04-25T00:33:35.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f122400c070 con 0x7f1224004810 2024-04-25T00:33:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f122400c270 con 0x7f12240051d0 2024-04-25T00:33:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123d2f6640 1 --2- 172.21.15.29:0/1189242966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12240039c0 0x7f1224003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123caf5640 1 --2- 172.21.15.29:0/1189242966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1224004810 0x7f1224004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123d2f6640 1 -- 172.21.15.29:0/1189242966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12240051d0 msgr2=0x7f122400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123daf7640 1 --2- 172.21.15.29:0/1189242966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12240051d0 0x7f122400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123d2f6640 1 --2- 172.21.15.29:0/1189242966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12240051d0 0x7f122400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123d2f6640 1 -- 172.21.15.29:0/1189242966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1224004810 msgr2=0x7f1224004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123d2f6640 1 --2- 172.21.15.29:0/1189242966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1224004810 0x7f1224004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123d2f6640 1 -- 172.21.15.29:0/1189242966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1224003530 con 0x7f12240039c0 2024-04-25T00:33:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123daf7640 1 -- 172.21.15.29:0/1189242966 reap_dead start 2024-04-25T00:33:35.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123caf5640 1 --2- 172.21.15.29:0/1189242966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1224004810 0x7f1224004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123d2f6640 1 --2- 172.21.15.29:0/1189242966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12240039c0 0x7f1224003de0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1234001780 tx=0x7f123401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856fc0cdc1f93851 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12340013a0 con 0x7f12240039c0 2024-04-25T00:33:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12340052f0 con 0x7f12240039c0 2024-04-25T00:33:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f122bfff640 1 -- 172.21.15.29:0/1189242966 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234003840 con 0x7f12240039c0 2024-04-25T00:33:35.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 -- 172.21.15.29:0/1189242966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12240039c0 msgr2=0x7f1224003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 --2- 172.21.15.29:0/1189242966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12240039c0 0x7f1224003de0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1234001780 tx=0x7f123401c3f0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 -- 172.21.15.29:0/1189242966 shutdown_connections 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 --2- 172.21.15.29:0/1189242966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12240039c0 0x7f1224003de0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 -- 172.21.15.29:0/1189242966 >> 172.21.15.29:0/1189242966 conn(0x7f1238075030 msgr2=0x7f12380b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 -- 172.21.15.29:0/1189242966 shutdown_connections 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 -- 172.21.15.29:0/1189242966 wait complete. 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 Processor -- start 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 -- start start 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12240039c0 0x7f12381faf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12381f6e00 0x7f123815feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1238160ac0 0x7f12381597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1238162c10 con 0x7f1238160ac0 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1238162a90 con 0x7f12240039c0 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.830+0000 7f123f55a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1238071990 con 0x7f12381f6e00 2024-04-25T00:33:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123daf7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1238160ac0 0x7f12381597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123daf7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1238160ac0 0x7f12381597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37756/0 (socket says 172.21.15.29:37756) 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123daf7640 1 -- 172.21.15.29:0/4033626337 learned_addr learned my addr 172.21.15.29:0/4033626337 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123d2f6640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12240039c0 0x7f12381faf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123caf5640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12381f6e00 0x7f123815feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123daf7640 1 -- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12381f6e00 msgr2=0x7f123815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123daf7640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12381f6e00 0x7f123815feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123daf7640 1 -- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12240039c0 msgr2=0x7f12381faf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123daf7640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12240039c0 0x7f12381faf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123daf7640 1 -- 172.21.15.29:0/4033626337 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12381f75e0 con 0x7f1238160ac0 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123d2f6640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12240039c0 0x7f12381faf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123daf7640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1238160ac0 0x7f12381597c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f123806c310 tx=0x7f123000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f1229ffb640 1 -- 172.21.15.29:0/4033626337 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123000fe50 con 0x7f1238160ac0 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f1229ffb640 1 -- 172.21.15.29:0/4033626337 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12300039a0 con 0x7f1238160ac0 2024-04-25T00:33:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123f55a640 1 -- 172.21.15.29:0/4033626337 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12381f78c0 con 0x7f1238160ac0 2024-04-25T00:33:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f1229ffb640 1 -- 172.21.15.29:0/4033626337 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230003d90 con 0x7f1238160ac0 2024-04-25T00:33:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123f55a640 1 -- 172.21.15.29:0/4033626337 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12381603f0 con 0x7f1238160ac0 2024-04-25T00:33:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123f55a640 1 -- 172.21.15.29:0/4033626337 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1238069270 con 0x7f1238160ac0 2024-04-25T00:33:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f1229ffb640 1 -- 172.21.15.29:0/4033626337 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f123001d020 con 0x7f1238160ac0 2024-04-25T00:33:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f1229ffb640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f121c067a70 0x7f121c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f1229ffb640 1 -- 172.21.15.29:0/4033626337 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f1230087530 con 0x7f1238160ac0 2024-04-25T00:33:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123d2f6640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f121c067a70 0x7f121c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.834+0000 7f123d2f6640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f121c067a70 0x7f121c069f30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1234001750 tx=0x7f12340011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.838+0000 7f1229ffb640 1 -- 172.21.15.29:0/4033626337 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1230024020 con 0x7f1238160ac0 2024-04-25T00:33:35.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.994+0000 7f123f55a640 1 -- 172.21.15.29:0/4033626337 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1238072650 con 0x7f1238160ac0 2024-04-25T00:33:35.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.998+0000 7f1229ffb640 1 -- 172.21.15.29:0/4033626337 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) ==== 100+0+11 (secure 0 0 0) 0x7f1230057dd0 con 0x7f1238160ac0 2024-04-25T00:33:35.998 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:33:36.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.998+0000 7f123f55a640 1 -- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f121c067a70 msgr2=0x7f121c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.998+0000 7f123f55a640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f121c067a70 0x7f121c069f30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1234001750 tx=0x7f12340011a0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.998+0000 7f123f55a640 1 -- 172.21.15.29:0/4033626337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1238160ac0 msgr2=0x7f12381597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.998+0000 7f123f55a640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1238160ac0 0x7f12381597c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f123806c310 tx=0x7f123000f520 comp rx=0 tx=0).stop 2024-04-25T00:33:36.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.998+0000 7f123f55a640 1 -- 172.21.15.29:0/4033626337 shutdown_connections 2024-04-25T00:33:36.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.998+0000 7f123f55a640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1238160ac0 0x7f12381597c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.998+0000 7f123f55a640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12381f6e00 0x7f123815feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.998+0000 7f123f55a640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f121c067a70 0x7f121c069f30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.998+0000 7f123f55a640 1 --2- 172.21.15.29:0/4033626337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12240039c0 0x7f12381faf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.998+0000 7f123f55a640 1 -- 172.21.15.29:0/4033626337 >> 172.21.15.29:0/4033626337 conn(0x7f1238075030 msgr2=0x7f12380b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:36.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.998+0000 7f123f55a640 1 -- 172.21.15.29:0/4033626337 shutdown_connections 2024-04-25T00:33:36.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:35.998+0000 7f123f55a640 1 -- 172.21.15.29:0/4033626337 wait complete. 2024-04-25T00:33:36.016 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:33:36.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f7644bfc640 1 Processor -- start 2024-04-25T00:33:36.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f7644bfc640 1 -- start start 2024-04-25T00:33:36.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f7644bfc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f764015e8d0 0x7f76400cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f7644bfc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f76400b4780 0x7f764015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f7644bfc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76400b3720 con 0x7f764015f1f0 2024-04-25T00:33:36.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f7644bfc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f76400b42f0 con 0x7f76400b4780 2024-04-25T00:33:36.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f7644bfc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f764015d7e0 con 0x7f764015e8d0 2024-04-25T00:33:36.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f763dd9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f76400b4780 0x7f764015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f763ed9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f764015f1f0 0x7f764015b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59692/0 (socket says 172.21.15.29:59692) 2024-04-25T00:33:36.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f763ed9d640 1 -- 172.21.15.29:0/1782936595 learned_addr learned my addr 172.21.15.29:0/1782936595 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:36.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f763e59c640 1 --2- 172.21.15.29:0/1782936595 >> v2:172.21.15.157:3300/0 conn(0x7f764015e8d0 0x7f76400cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f763dd9b640 1 -- 172.21.15.29:0/1782936595 >> v2:172.21.15.157:3300/0 conn(0x7f764015e8d0 msgr2=0x7f76400cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:36.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f763dd9b640 1 --2- 172.21.15.29:0/1782936595 >> v2:172.21.15.157:3300/0 conn(0x7f764015e8d0 0x7f76400cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f763dd9b640 1 -- 172.21.15.29:0/1782936595 >> v1:172.21.15.29:6789/0 conn(0x7f764015f1f0 legacy=0x7f764015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f763d59a640 1 -- 172.21.15.29:0/1782936595 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2266981933 0 0) 0x7f76400b3720 con 0x7f764015f1f0 2024-04-25T00:33:36.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f763dd9b640 1 -- 172.21.15.29:0/1782936595 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76400b39c0 con 0x7f76400b4780 2024-04-25T00:33:36.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.138+0000 7f763e59c640 1 --2- 172.21.15.29:0/1782936595 >> v2:172.21.15.157:3300/0 conn(0x7f764015e8d0 0x7f76400cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:36.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763dd9b640 1 --2- 172.21.15.29:0/1782936595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76400b4780 0x7f764015ecb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f76400b5260 tx=0x7f763401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939c31799879dc93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:36.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763d59a640 1 -- 172.21.15.29:0/1782936595 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764015c1c0 con 0x7f76400b4780 2024-04-25T00:33:36.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763d59a640 1 -- 172.21.15.29:0/1782936595 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76340b0070 con 0x7f76400b4780 2024-04-25T00:33:36.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763d59a640 1 -- 172.21.15.29:0/1782936595 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76340ab870 con 0x7f76400b4780 2024-04-25T00:33:36.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 -- 172.21.15.29:0/1782936595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76400b4780 msgr2=0x7f764015ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 --2- 172.21.15.29:0/1782936595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76400b4780 0x7f764015ecb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f76400b5260 tx=0x7f763401c450 comp rx=0 tx=0).stop 2024-04-25T00:33:36.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 -- 172.21.15.29:0/1782936595 shutdown_connections 2024-04-25T00:33:36.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 --2- 172.21.15.29:0/1782936595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76400b4780 0x7f764015ecb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 --2- 172.21.15.29:0/1782936595 >> v2:172.21.15.157:3300/0 conn(0x7f764015e8d0 0x7f76400cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 -- 172.21.15.29:0/1782936595 >> 172.21.15.29:0/1782936595 conn(0x7f76400b1870 msgr2=0x7f76400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:36.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 -- 172.21.15.29:0/1782936595 shutdown_connections 2024-04-25T00:33:36.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 -- 172.21.15.29:0/1782936595 wait complete. 2024-04-25T00:33:36.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 Processor -- start 2024-04-25T00:33:36.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 -- start start 2024-04-25T00:33:36.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76400b4780 0x7f7640142910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f764015e8d0 0x7f76401f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764015f1f0 0x7f76400cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7640144a20 con 0x7f76400b4780 2024-04-25T00:33:36.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f764015f600 con 0x7f764015f1f0 2024-04-25T00:33:36.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7640142f10 con 0x7f764015e8d0 2024-04-25T00:33:36.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76400b4780 0x7f7640142910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76400b4780 0x7f7640142910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37780/0 (socket says 172.21.15.29:37780) 2024-04-25T00:33:36.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763e59c640 1 -- 172.21.15.29:0/2161725609 learned_addr learned my addr 172.21.15.29:0/2161725609 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:36.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763dd9b640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f764015e8d0 0x7f76401f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763e59c640 1 -- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f764015e8d0 msgr2=0x7f76401f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763e59c640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f764015e8d0 0x7f76401f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763ed9d640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764015f1f0 0x7f76400cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763e59c640 1 -- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764015f1f0 msgr2=0x7f76400cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763e59c640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764015f1f0 0x7f76400cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763e59c640 1 -- 172.21.15.29:0/2161725609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76401431f0 con 0x7f76400b4780 2024-04-25T00:33:36.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f763e59c640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76400b4780 0x7f7640142910 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f763800cf60 tx=0x7f763800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:36.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f762effd640 1 -- 172.21.15.29:0/2161725609 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763801a070 con 0x7f76400b4780 2024-04-25T00:33:36.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f762effd640 1 -- 172.21.15.29:0/2161725609 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f763800d840 con 0x7f76400b4780 2024-04-25T00:33:36.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 -- 172.21.15.29:0/2161725609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76401434d0 con 0x7f76400b4780 2024-04-25T00:33:36.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f762effd640 1 -- 172.21.15.29:0/2161725609 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763800dc10 con 0x7f76400b4780 2024-04-25T00:33:36.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.142+0000 7f7644bfc640 1 -- 172.21.15.29:0/2161725609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764015aa30 con 0x7f76400b4780 2024-04-25T00:33:36.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.146+0000 7f7644bfc640 1 -- 172.21.15.29:0/2161725609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7640069270 con 0x7f76400b4780 2024-04-25T00:33:36.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.146+0000 7f762effd640 1 -- 172.21.15.29:0/2161725609 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f763801d070 con 0x7f76400b4780 2024-04-25T00:33:36.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.146+0000 7f762effd640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7620067a70 0x7f7620069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.146+0000 7f762effd640 1 -- 172.21.15.29:0/2161725609 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f76380866d0 con 0x7f76400b4780 2024-04-25T00:33:36.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.146+0000 7f763dd9b640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7620067a70 0x7f7620069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.146+0000 7f763dd9b640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7620067a70 0x7f7620069f30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f76340af000 tx=0x7f76340a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:36.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.150+0000 7f762effd640 1 -- 172.21.15.29:0/2161725609 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f76380569b0 con 0x7f76400b4780 2024-04-25T00:33:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.318+0000 7f7644bfc640 1 -- 172.21.15.29:0/2161725609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7640140700 con 0x7f76400b4780 2024-04-25T00:33:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.322+0000 7f762effd640 1 -- 172.21.15.29:0/2161725609 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7f763805acb0 con 0x7f76400b4780 2024-04-25T00:33:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:33:36.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.322+0000 7f7644bfc640 1 -- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7620067a70 msgr2=0x7f7620069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.322+0000 7f7644bfc640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7620067a70 0x7f7620069f30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f76340af000 tx=0x7f76340a7000 comp rx=0 tx=0).stop 2024-04-25T00:33:36.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.322+0000 7f7644bfc640 1 -- 172.21.15.29:0/2161725609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76400b4780 msgr2=0x7f7640142910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.322+0000 7f7644bfc640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76400b4780 0x7f7640142910 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f763800cf60 tx=0x7f763800cf90 comp rx=0 tx=0).stop 2024-04-25T00:33:36.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.322+0000 7f7644bfc640 1 -- 172.21.15.29:0/2161725609 shutdown_connections 2024-04-25T00:33:36.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.322+0000 7f7644bfc640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7620067a70 0x7f7620069f30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.322+0000 7f7644bfc640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764015f1f0 0x7f76400cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.322+0000 7f7644bfc640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f764015e8d0 0x7f76401f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.322+0000 7f7644bfc640 1 --2- 172.21.15.29:0/2161725609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76400b4780 0x7f7640142910 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.322+0000 7f7644bfc640 1 -- 172.21.15.29:0/2161725609 >> 172.21.15.29:0/2161725609 conn(0x7f76400b1870 msgr2=0x7f76400b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:36.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.322+0000 7f7644bfc640 1 -- 172.21.15.29:0/2161725609 shutdown_connections 2024-04-25T00:33:36.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.322+0000 7f7644bfc640 1 -- 172.21.15.29:0/2161725609 wait complete. 2024-04-25T00:33:36.340 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:33:36.340 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:33:36.347 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:33:36.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b3201b640 1 Processor -- start 2024-04-25T00:33:36.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b3201b640 1 -- start start 2024-04-25T00:33:36.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b3201b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b2c0b4780 0x7f8b2c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b3201b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b2c0b3720 con 0x7f8b2c15e8d0 2024-04-25T00:33:36.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b3201b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b2c0b39c0 con 0x7f8b2c0cc240 2024-04-25T00:33:36.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b3201b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8b2c0cc650 con 0x7f8b2c0b4780 2024-04-25T00:33:36.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b2affd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b2c0b4780 0x7f8b2c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b2b7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8b2c15e8d0 0x7f8b2c15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59698/0 (socket says 172.21.15.29:59698) 2024-04-25T00:33:36.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b2b7fe640 1 -- 172.21.15.29:0/4177566287 learned_addr learned my addr 172.21.15.29:0/4177566287 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:36.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b2affd640 1 -- 172.21.15.29:0/4177566287 >> v1:172.21.15.157:6789/0 conn(0x7f8b2c0cc240 legacy=0x7f8b2c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:36.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b2affd640 1 -- 172.21.15.29:0/4177566287 >> v1:172.21.15.29:6789/0 conn(0x7f8b2c15e8d0 legacy=0x7f8b2c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b2affd640 1 -- 172.21.15.29:0/4177566287 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c0b3ca0 con 0x7f8b2c0b4780 2024-04-25T00:33:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b2affd640 1 --2- 172.21.15.29:0/4177566287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b2c0b4780 0x7f8b2c15f3c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8b200097c0 tx=0x7f8b2001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933a2c9653a44991 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b2a7fc640 1 -- 172.21.15.29:0/4177566287 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b200b0070 con 0x7f8b2c0b4780 2024-04-25T00:33:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b2a7fc640 1 -- 172.21.15.29:0/4177566287 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b200ab600 con 0x7f8b2c0b4780 2024-04-25T00:33:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.458+0000 7f8b2a7fc640 1 -- 172.21.15.29:0/4177566287 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b200ab9d0 con 0x7f8b2c0b4780 2024-04-25T00:33:36.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 -- 172.21.15.29:0/4177566287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b2c0b4780 msgr2=0x7f8b2c15f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 --2- 172.21.15.29:0/4177566287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b2c0b4780 0x7f8b2c15f3c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8b200097c0 tx=0x7f8b2001c460 comp rx=0 tx=0).stop 2024-04-25T00:33:36.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 -- 172.21.15.29:0/4177566287 shutdown_connections 2024-04-25T00:33:36.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 --2- 172.21.15.29:0/4177566287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b2c0b4780 0x7f8b2c15f3c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 -- 172.21.15.29:0/4177566287 >> 172.21.15.29:0/4177566287 conn(0x7f8b2c0b1870 msgr2=0x7f8b2c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:36.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 -- 172.21.15.29:0/4177566287 shutdown_connections 2024-04-25T00:33:36.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 -- 172.21.15.29:0/4177566287 wait complete. 2024-04-25T00:33:36.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 Processor -- start 2024-04-25T00:33:36.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 -- start start 2024-04-25T00:33:36.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b2c0b4780 0x7f8b2c0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b2c0cc240 0x7f8b2c0713a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b2c15e8d0 0x7f8b2c1375c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c168e00 con 0x7f8b2c0cc240 2024-04-25T00:33:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c168c80 con 0x7f8b2c15e8d0 2024-04-25T00:33:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b3201b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2c168f80 con 0x7f8b2c0b4780 2024-04-25T00:33:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b2affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b2c0cc240 0x7f8b2c0713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b2affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b2c0cc240 0x7f8b2c0713a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37804/0 (socket says 172.21.15.29:37804) 2024-04-25T00:33:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b2affd640 1 -- 172.21.15.29:0/4178281230 learned_addr learned my addr 172.21.15.29:0/4178281230 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b2b7fe640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b2c0b4780 0x7f8b2c0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b2bfff640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b2c15e8d0 0x7f8b2c1375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b2affd640 1 -- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b2c0b4780 msgr2=0x7f8b2c0ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b2affd640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b2c0b4780 0x7f8b2c0ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b2affd640 1 -- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b2c15e8d0 msgr2=0x7f8b2c1375c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b2affd640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b2c15e8d0 0x7f8b2c1375c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b2affd640 1 -- 172.21.15.29:0/4178281230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c071aa0 con 0x7f8b2c0cc240 2024-04-25T00:33:36.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b2bfff640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b2c15e8d0 0x7f8b2c1375c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:36.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b2affd640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b2c0cc240 0x7f8b2c0713a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8b20019070 tx=0x7f8b20019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:36.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b13fff640 1 -- 172.21.15.29:0/4178281230 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b200b0040 con 0x7f8b2c0cc240 2024-04-25T00:33:36.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b13fff640 1 -- 172.21.15.29:0/4178281230 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b200bb070 con 0x7f8b2c0cc240 2024-04-25T00:33:36.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.462+0000 7f8b13fff640 1 -- 172.21.15.29:0/4178281230 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b200b4870 con 0x7f8b2c0cc240 2024-04-25T00:33:36.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.466+0000 7f8b3201b640 1 -- 172.21.15.29:0/4178281230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c071d80 con 0x7f8b2c0cc240 2024-04-25T00:33:36.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.466+0000 7f8b3201b640 1 -- 172.21.15.29:0/4178281230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c072340 con 0x7f8b2c0cc240 2024-04-25T00:33:36.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.466+0000 7f8b3201b640 1 -- 172.21.15.29:0/4178281230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c069270 con 0x7f8b2c0cc240 2024-04-25T00:33:36.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.466+0000 7f8b13fff640 1 -- 172.21.15.29:0/4178281230 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f8b200ab4c0 con 0x7f8b2c0cc240 2024-04-25T00:33:36.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.466+0000 7f8b13fff640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b18067a70 0x7f8b18069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.466+0000 7f8b13fff640 1 -- 172.21.15.29:0/4178281230 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f8b2011fed0 con 0x7f8b2c0cc240 2024-04-25T00:33:36.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.466+0000 7f8b2b7fe640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b18067a70 0x7f8b18069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.466+0000 7f8b2b7fe640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b18067a70 0x7f8b18069f30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0012f0 tx=0x7f8b1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:36.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.470+0000 7f8b13fff640 1 -- 172.21.15.29:0/4178281230 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8b20016050 con 0x7f8b2c0cc240 2024-04-25T00:33:36.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.630+0000 7f8b3201b640 1 -- 172.21.15.29:0/4178281230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b2c06e7b0 con 0x7f8b2c0cc240 2024-04-25T00:33:36.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.630+0000 7f8b13fff640 1 -- 172.21.15.29:0/4178281230 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16565 (secure 0 0 0) 0x7f8b200f0710 con 0x7f8b2c0cc240 2024-04-25T00:33:36.633 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:36.633 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":45,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:33:14.784398+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:33:10.789862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:33:12.129841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:33:36.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.634+0000 7f8b3201b640 1 -- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b18067a70 msgr2=0x7f8b18069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.634+0000 7f8b3201b640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b18067a70 0x7f8b18069f30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0012f0 tx=0x7f8b1c009040 comp rx=0 tx=0).stop 2024-04-25T00:33:36.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.634+0000 7f8b3201b640 1 -- 172.21.15.29:0/4178281230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b2c0cc240 msgr2=0x7f8b2c0713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.634+0000 7f8b3201b640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b2c0cc240 0x7f8b2c0713a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8b20019070 tx=0x7f8b20019200 comp rx=0 tx=0).stop 2024-04-25T00:33:36.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.634+0000 7f8b3201b640 1 -- 172.21.15.29:0/4178281230 shutdown_connections 2024-04-25T00:33:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.634+0000 7f8b3201b640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8b18067a70 0x7f8b18069f30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.634+0000 7f8b3201b640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b2c15e8d0 0x7f8b2c1375c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.634+0000 7f8b3201b640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b2c0cc240 0x7f8b2c0713a0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.634+0000 7f8b3201b640 1 --2- 172.21.15.29:0/4178281230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b2c0b4780 0x7f8b2c0ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.634+0000 7f8b3201b640 1 -- 172.21.15.29:0/4178281230 >> 172.21.15.29:0/4178281230 conn(0x7f8b2c0b1870 msgr2=0x7f8b2c0b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.634+0000 7f8b3201b640 1 -- 172.21.15.29:0/4178281230 shutdown_connections 2024-04-25T00:33:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.634+0000 7f8b3201b640 1 -- 172.21.15.29:0/4178281230 wait complete. 2024-04-25T00:33:36.651 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:33:10.789862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:33:12.129841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:33:36.651 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:33:36.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5827674640 1 Processor -- start 2024-04-25T00:33:36.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5827674640 1 -- start start 2024-04-25T00:33:36.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5827674640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820162420 0x7f5820070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5827674640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f58200b3280 0x7f582006cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5827674640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5820071910 con 0x7f58200b4e00 2024-04-25T00:33:36.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5827674640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5820071bf0 con 0x7f58200b3280 2024-04-25T00:33:36.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5827674640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5820071ed0 con 0x7f5820162420 2024-04-25T00:33:36.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5825c11640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f58200b3280 0x7f582006cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5825c11640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f58200b3280 0x7f582006cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37806/0 (socket says 172.21.15.29:37806) 2024-04-25T00:33:36.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5825c11640 1 -- 172.21.15.29:0/2919626139 learned_addr learned my addr 172.21.15.29:0/2919626139 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:36.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5824c0f640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f58200b4e00 0x7f58200b2b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59704/0 (socket says 172.21.15.29:59704) 2024-04-25T00:33:36.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5825410640 1 --2- 172.21.15.29:0/2919626139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820162420 0x7f5820070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5825c11640 1 -- 172.21.15.29:0/2919626139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820162420 msgr2=0x7f5820070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:36.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5825c11640 1 --2- 172.21.15.29:0/2919626139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820162420 0x7f5820070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5825c11640 1 -- 172.21.15.29:0/2919626139 >> v1:172.21.15.29:6789/0 conn(0x7f58200b4e00 legacy=0x7f58200b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5825c11640 1 -- 172.21.15.29:0/2919626139 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58200721b0 con 0x7f58200b3280 2024-04-25T00:33:36.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.766+0000 7f5813fff640 1 -- 172.21.15.29:0/2919626139 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1074437082 0 0) 0x7f5820071910 con 0x7f58200b4e00 2024-04-25T00:33:36.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5825c11640 1 --2- 172.21.15.29:0/2919626139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58200b3280 0x7f582006cf80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5818009e50 tx=0x7f581801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f1adc385c996df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:36.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5813fff640 1 -- 172.21.15.29:0/2919626139 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58200716d0 con 0x7f58200b3280 2024-04-25T00:33:36.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5813fff640 1 -- 172.21.15.29:0/2919626139 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58180a5810 con 0x7f58200b3280 2024-04-25T00:33:36.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5813fff640 1 -- 172.21.15.29:0/2919626139 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58180a5be0 con 0x7f58200b3280 2024-04-25T00:33:36.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 -- 172.21.15.29:0/2919626139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58200b3280 msgr2=0x7f582006cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 --2- 172.21.15.29:0/2919626139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58200b3280 0x7f582006cf80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5818009e50 tx=0x7f581801ca30 comp rx=0 tx=0).stop 2024-04-25T00:33:36.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 -- 172.21.15.29:0/2919626139 shutdown_connections 2024-04-25T00:33:36.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 --2- 172.21.15.29:0/2919626139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58200b3280 0x7f582006cf80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 --2- 172.21.15.29:0/2919626139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820162420 0x7f5820070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 -- 172.21.15.29:0/2919626139 >> 172.21.15.29:0/2919626139 conn(0x7f58200b1870 msgr2=0x7f58200b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:36.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 -- 172.21.15.29:0/2919626139 shutdown_connections 2024-04-25T00:33:36.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 -- 172.21.15.29:0/2919626139 wait complete. 2024-04-25T00:33:36.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 Processor -- start 2024-04-25T00:33:36.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 -- start start 2024-04-25T00:33:36.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200b3280 0x7f582014f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200b4e00 0x7f5820154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5820162420 0x7f582015fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f582006e0e0 con 0x7f5820162420 2024-04-25T00:33:36.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f582006df60 con 0x7f58200b4e00 2024-04-25T00:33:36.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5827674640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f582006e260 con 0x7f58200b3280 2024-04-25T00:33:36.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5825c11640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5820162420 0x7f582015fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5825c11640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5820162420 0x7f582015fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37812/0 (socket says 172.21.15.29:37812) 2024-04-25T00:33:36.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.770+0000 7f5825c11640 1 -- 172.21.15.29:0/1054303579 learned_addr learned my addr 172.21.15.29:0/1054303579 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:36.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5825410640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200b3280 0x7f582014f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5824c0f640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200b4e00 0x7f5820154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5825c11640 1 -- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200b3280 msgr2=0x7f582014f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5825c11640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200b3280 0x7f582014f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5825c11640 1 -- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200b4e00 msgr2=0x7f5820154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5825c11640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200b4e00 0x7f5820154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5825c11640 1 -- 172.21.15.29:0/1054303579 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582014f7b0 con 0x7f5820162420 2024-04-25T00:33:36.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5825410640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200b3280 0x7f582014f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:36.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5824c0f640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200b4e00 0x7f5820154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:36.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5825c11640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5820162420 0x7f582015fa50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f581801a660 tx=0x7f58180197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:36.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5811ffb640 1 -- 172.21.15.29:0/1054303579 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58180bc030 con 0x7f5820162420 2024-04-25T00:33:36.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5811ffb640 1 -- 172.21.15.29:0/1054303579 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58180b8470 con 0x7f5820162420 2024-04-25T00:33:36.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5811ffb640 1 -- 172.21.15.29:0/1054303579 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58180b8860 con 0x7f5820162420 2024-04-25T00:33:36.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5827674640 1 -- 172.21.15.29:0/1054303579 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5820154ed0 con 0x7f5820162420 2024-04-25T00:33:36.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5827674640 1 -- 172.21.15.29:0/1054303579 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58201531b0 con 0x7f5820162420 2024-04-25T00:33:36.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5827674640 1 -- 172.21.15.29:0/1054303579 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5820069270 con 0x7f5820162420 2024-04-25T00:33:36.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5811ffb640 1 -- 172.21.15.29:0/1054303579 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f58180bf070 con 0x7f5820162420 2024-04-25T00:33:36.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5811ffb640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5804067aa0 0x7f5804069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:36.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5811ffb640 1 -- 172.21.15.29:0/1054303579 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f58181210c0 con 0x7f5820162420 2024-04-25T00:33:36.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5825410640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5804067aa0 0x7f5804069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:36.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.774+0000 7f5825410640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5804067aa0 0x7f5804069f60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f581c0097a0 tx=0x7f581c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:36.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.778+0000 7f5811ffb640 1 -- 172.21.15.29:0/1054303579 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f582014f7b0 con 0x7f5820162420 2024-04-25T00:33:36.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.934+0000 7f5827674640 1 -- 172.21.15.29:0/1054303579 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f582015ecf0 con 0x7f5820162420 2024-04-25T00:33:36.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.934+0000 7f5811ffb640 1 -- 172.21.15.29:0/1054303579 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f5818016620 con 0x7f5820162420 2024-04-25T00:33:36.937 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:33:36.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.938+0000 7f5827674640 1 -- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5804067aa0 msgr2=0x7f5804069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.938+0000 7f5827674640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5804067aa0 0x7f5804069f60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f581c0097a0 tx=0x7f581c008040 comp rx=0 tx=0).stop 2024-04-25T00:33:36.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.938+0000 7f5827674640 1 -- 172.21.15.29:0/1054303579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5820162420 msgr2=0x7f582015fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:36.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.938+0000 7f5827674640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5820162420 0x7f582015fa50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f581801a660 tx=0x7f58180197d0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.938+0000 7f5827674640 1 -- 172.21.15.29:0/1054303579 shutdown_connections 2024-04-25T00:33:36.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.938+0000 7f5827674640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5820162420 0x7f582015fa50 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.938+0000 7f5827674640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5804067aa0 0x7f5804069f60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.938+0000 7f5827674640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200b4e00 0x7f5820154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.938+0000 7f5827674640 1 --2- 172.21.15.29:0/1054303579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200b3280 0x7f582014f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:36.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.938+0000 7f5827674640 1 -- 172.21.15.29:0/1054303579 >> 172.21.15.29:0/1054303579 conn(0x7f58200b1870 msgr2=0x7f5820163b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:36.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.938+0000 7f5827674640 1 -- 172.21.15.29:0/1054303579 shutdown_connections 2024-04-25T00:33:36.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:36.938+0000 7f5827674640 1 -- 172.21.15.29:0/1054303579 wait complete. 2024-04-25T00:33:36.954 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:33:37.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca9ca0640 1 Processor -- start 2024-04-25T00:33:37.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca9ca0640 1 -- start start 2024-04-25T00:33:37.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca9ca0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1ca40b3140 0x7f1ca406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca9ca0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca40b41e0 con 0x7f1ca40b3c30 2024-04-25T00:33:37.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca9ca0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca406d960 con 0x7f1ca40b4b10 2024-04-25T00:33:37.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca9ca0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca406dc40 con 0x7f1ca40b3140 2024-04-25T00:33:37.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca3fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1ca40b3c30 0x7f1ca40ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59708/0 (socket says 172.21.15.29:59708) 2024-04-25T00:33:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca3fff640 1 -- 172.21.15.29:0/3362311074 learned_addr learned my addr 172.21.15.29:0/3362311074 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca37fe640 1 --2- 172.21.15.29:0/3362311074 >> v2:172.21.15.157:3300/0 conn(0x7f1ca40b3140 0x7f1ca406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1085450354 0 0) 0x7f1ca40b41e0 con 0x7f1ca40b3c30 2024-04-25T00:33:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c8c003530 con 0x7f1ca40b3c30 2024-04-25T00:33:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4293960460 0 0) 0x7f1c8c003530 con 0x7f1ca40b3c30 2024-04-25T00:33:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 >> v2:172.21.15.157:3300/0 conn(0x7f1ca40b3140 msgr2=0x7f1ca406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 --2- 172.21.15.29:0/3362311074 >> v2:172.21.15.157:3300/0 conn(0x7f1ca40b3140 0x7f1ca406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 >> v1:172.21.15.157:6789/0 conn(0x7f1ca40b4b10 legacy=0x7f1ca40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca406df20 con 0x7f1ca40b3c30 2024-04-25T00:33:37.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f1c9c001a00 con 0x7f1ca40b3c30 2024-04-25T00:33:37.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca37fe640 1 --2- 172.21.15.29:0/3362311074 >> v2:172.21.15.157:3300/0 conn(0x7f1ca40b3140 0x7f1ca406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:37.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 >> v1:172.21.15.29:6789/0 conn(0x7f1ca40b3c30 legacy=0x7f1ca40ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 --2- 172.21.15.29:0/3362311074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c0039c0 0x7f1c8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 --2- 172.21.15.29:0/3362311074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c8c004810 0x7f1c8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 --2- 172.21.15.29:0/3362311074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c8c0051d0 0x7f1c8c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca406dc40 con 0x7f1c8c0051d0 2024-04-25T00:33:37.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c00c120 con 0x7f1c8c004810 2024-04-25T00:33:37.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1c8c00c320 con 0x7f1c8c0039c0 2024-04-25T00:33:37.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca3fff640 1 --2- 172.21.15.29:0/3362311074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c8c0051d0 0x7f1c8c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca37fe640 1 --2- 172.21.15.29:0/3362311074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c0039c0 0x7f1c8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca2ffd640 1 --2- 172.21.15.29:0/3362311074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c8c004810 0x7f1c8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca3fff640 1 -- 172.21.15.29:0/3362311074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c0039c0 msgr2=0x7f1c8c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:37.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca3fff640 1 --2- 172.21.15.29:0/3362311074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c0039c0 0x7f1c8c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca3fff640 1 -- 172.21.15.29:0/3362311074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c8c004810 msgr2=0x7f1c8c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:37.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca3fff640 1 --2- 172.21.15.29:0/3362311074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c8c004810 0x7f1c8c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.066+0000 7f1ca3fff640 1 -- 172.21.15.29:0/3362311074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8c003530 con 0x7f1c8c0051d0 2024-04-25T00:33:37.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca3fff640 1 -- 172.21.15.29:0/3362311074 reap_dead start 2024-04-25T00:33:37.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca3fff640 1 --2- 172.21.15.29:0/3362311074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c8c0051d0 0x7f1c8c00b9d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00a860 tx=0x7f1c9c01d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148d3091b0ab16f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:37.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00a480 con 0x7f1c8c0051d0 2024-04-25T00:33:37.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9c0ad630 con 0x7f1c8c0051d0 2024-04-25T00:33:37.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca27fc640 1 -- 172.21.15.29:0/3362311074 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0ada20 con 0x7f1c8c0051d0 2024-04-25T00:33:37.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/3362311074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c8c0051d0 msgr2=0x7f1c8c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca9ca0640 1 --2- 172.21.15.29:0/3362311074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c8c0051d0 0x7f1c8c00b9d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00a860 tx=0x7f1c9c01d380 comp rx=0 tx=0).stop 2024-04-25T00:33:37.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/3362311074 shutdown_connections 2024-04-25T00:33:37.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca9ca0640 1 --2- 172.21.15.29:0/3362311074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c8c0051d0 0x7f1c8c00b9d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/3362311074 >> 172.21.15.29:0/3362311074 conn(0x7f1ca40b1870 msgr2=0x7f1ca40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:37.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/3362311074 shutdown_connections 2024-04-25T00:33:37.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/3362311074 wait complete. 2024-04-25T00:33:37.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca9ca0640 1 Processor -- start 2024-04-25T00:33:37.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.070+0000 7f1ca9ca0640 1 -- start start 2024-04-25T00:33:37.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca9ca0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c8c0051d0 0x7f1ca4159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca9ca0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca413d770 0x7f1ca4139880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca9ca0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca415e590 0x7f1ca415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca9ca0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca4145a90 con 0x7f1c8c0051d0 2024-04-25T00:33:37.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca9ca0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca4145910 con 0x7f1ca415e590 2024-04-25T00:33:37.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca9ca0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca406e440 con 0x7f1ca413d770 2024-04-25T00:33:37.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c8c0051d0 0x7f1ca4159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c8c0051d0 0x7f1ca4159300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37826/0 (socket says 172.21.15.29:37826) 2024-04-25T00:33:37.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca37fe640 1 -- 172.21.15.29:0/2888055477 learned_addr learned my addr 172.21.15.29:0/2888055477 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:37.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca37fe640 1 -- 172.21.15.29:0/2888055477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca413d770 msgr2=0x7f1ca4139880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca37fe640 1 --2- 172.21.15.29:0/2888055477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca413d770 0x7f1ca4139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca37fe640 1 -- 172.21.15.29:0/2888055477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca415e590 msgr2=0x7f1ca415ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca37fe640 1 --2- 172.21.15.29:0/2888055477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca415e590 0x7f1ca415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca37fe640 1 -- 172.21.15.29:0/2888055477 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca4139dc0 con 0x7f1c8c0051d0 2024-04-25T00:33:37.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca37fe640 1 --2- 172.21.15.29:0/2888055477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c8c0051d0 0x7f1ca4159300 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400e560 tx=0x7f1c9400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:37.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1c8bfff640 1 -- 172.21.15.29:0/2888055477 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94017070 con 0x7f1c8c0051d0 2024-04-25T00:33:37.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1c8bfff640 1 -- 172.21.15.29:0/2888055477 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c94013610 con 0x7f1c8c0051d0 2024-04-25T00:33:37.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1c8bfff640 1 -- 172.21.15.29:0/2888055477 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400f040 con 0x7f1c8c0051d0 2024-04-25T00:33:37.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/2888055477 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca413c3f0 con 0x7f1c8c0051d0 2024-04-25T00:33:37.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/2888055477 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca413c9b0 con 0x7f1c8c0051d0 2024-04-25T00:33:37.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/2888055477 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca4069270 con 0x7f1c8c0051d0 2024-04-25T00:33:37.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1c8bfff640 1 -- 172.21.15.29:0/2888055477 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f1c9401a070 con 0x7f1c8c0051d0 2024-04-25T00:33:37.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1c8bfff640 1 --2- 172.21.15.29:0/2888055477 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1c90067aa0 0x7f1c90069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1c8bfff640 1 -- 172.21.15.29:0/2888055477 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f1c94085ac0 con 0x7f1c8c0051d0 2024-04-25T00:33:37.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca2ffd640 1 --2- 172.21.15.29:0/2888055477 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1c90067aa0 0x7f1c90069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.074+0000 7f1ca2ffd640 1 --2- 172.21.15.29:0/2888055477 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1c90067aa0 0x7f1c90069f60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1c98007fd0 tx=0x7f1c98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:37.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.078+0000 7f1c8bfff640 1 -- 172.21.15.29:0/2888055477 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1c94056300 con 0x7f1c8c0051d0 2024-04-25T00:33:37.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/2888055477 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1ca4140770 con 0x7f1c8c0051d0 2024-04-25T00:33:37.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1c8bfff640 1 -- 172.21.15.29:0/2888055477 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) ==== 104+0+11 (secure 0 0 0) 0x7f1c94055c00 con 0x7f1c8c0051d0 2024-04-25T00:33:37.243 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:33:37.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/2888055477 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1c90067aa0 msgr2=0x7f1c90069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1ca9ca0640 1 --2- 172.21.15.29:0/2888055477 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1c90067aa0 0x7f1c90069f60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1c98007fd0 tx=0x7f1c98009040 comp rx=0 tx=0).stop 2024-04-25T00:33:37.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/2888055477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c8c0051d0 msgr2=0x7f1ca4159300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1ca9ca0640 1 --2- 172.21.15.29:0/2888055477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c8c0051d0 0x7f1ca4159300 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400e560 tx=0x7f1c9400ea20 comp rx=0 tx=0).stop 2024-04-25T00:33:37.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/2888055477 shutdown_connections 2024-04-25T00:33:37.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1ca9ca0640 1 --2- 172.21.15.29:0/2888055477 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1c90067aa0 0x7f1c90069f60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1ca9ca0640 1 --2- 172.21.15.29:0/2888055477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca415e590 0x7f1ca415ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1ca9ca0640 1 --2- 172.21.15.29:0/2888055477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ca413d770 0x7f1ca4139880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1ca9ca0640 1 --2- 172.21.15.29:0/2888055477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c8c0051d0 0x7f1ca4159300 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/2888055477 >> 172.21.15.29:0/2888055477 conn(0x7f1ca40b1870 msgr2=0x7f1ca413f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:37.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/2888055477 shutdown_connections 2024-04-25T00:33:37.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.242+0000 7f1ca9ca0640 1 -- 172.21.15.29:0/2888055477 wait complete. 2024-04-25T00:33:37.262 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:33:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5be5eb640 1 Processor -- start 2024-04-25T00:33:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5be5eb640 1 -- start start 2024-04-25T00:33:37.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5be5eb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5b815df70 0x7ff5b80b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5be5eb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff5b806d210 0x7ff5b80b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5be5eb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5b806d980 con 0x7ff5b80b4b60 2024-04-25T00:33:37.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5be5eb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff5b80b4fd0 con 0x7ff5b815df70 2024-04-25T00:33:37.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5be5eb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff5b806dc60 con 0x7ff5b806d210 2024-04-25T00:33:37.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5b7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5b815df70 0x7ff5b80b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5b7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5b815df70 0x7ff5b80b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37832/0 (socket says 172.21.15.29:37832) 2024-04-25T00:33:37.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5b7fff640 1 -- 172.21.15.29:0/2581263231 learned_addr learned my addr 172.21.15.29:0/2581263231 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:37.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5b77fe640 1 --2- 172.21.15.29:0/2581263231 >> v2:172.21.15.157:3300/0 conn(0x7ff5b806d210 0x7ff5b80b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5b7fff640 1 -- 172.21.15.29:0/2581263231 >> v2:172.21.15.157:3300/0 conn(0x7ff5b806d210 msgr2=0x7ff5b80b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:37.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5b7fff640 1 --2- 172.21.15.29:0/2581263231 >> v2:172.21.15.157:3300/0 conn(0x7ff5b806d210 0x7ff5b80b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5b7fff640 1 -- 172.21.15.29:0/2581263231 >> v1:172.21.15.157:6789/0 conn(0x7ff5b80b4b60 legacy=0x7ff5b8070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:37.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.390+0000 7ff5b7fff640 1 -- 172.21.15.29:0/2581263231 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b806df40 con 0x7ff5b815df70 2024-04-25T00:33:37.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.394+0000 7ff5b7fff640 1 --2- 172.21.15.29:0/2581263231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b815df70 0x7ff5b80b3e40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80cfb70 tx=0x7ff5a801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bed5a0e87c8e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:37.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.394+0000 7ff5b6ffd640 1 -- 172.21.15.29:0/2581263231 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80b0070 con 0x7ff5b815df70 2024-04-25T00:33:37.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.394+0000 7ff5b6ffd640 1 -- 172.21.15.29:0/2581263231 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a80ab600 con 0x7ff5b815df70 2024-04-25T00:33:37.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.394+0000 7ff5b6ffd640 1 -- 172.21.15.29:0/2581263231 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80ab9d0 con 0x7ff5b815df70 2024-04-25T00:33:37.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.394+0000 7ff5be5eb640 1 -- 172.21.15.29:0/2581263231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b815df70 msgr2=0x7ff5b80b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.394+0000 7ff5be5eb640 1 --2- 172.21.15.29:0/2581263231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b815df70 0x7ff5b80b3e40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80cfb70 tx=0x7ff5a801c4e0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.394+0000 7ff5be5eb640 1 -- 172.21.15.29:0/2581263231 shutdown_connections 2024-04-25T00:33:37.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.394+0000 7ff5be5eb640 1 --2- 172.21.15.29:0/2581263231 >> v2:172.21.15.157:3300/0 conn(0x7ff5b806d210 0x7ff5b80b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.394+0000 7ff5be5eb640 1 --2- 172.21.15.29:0/2581263231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b815df70 0x7ff5b80b3e40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.394+0000 7ff5be5eb640 1 -- 172.21.15.29:0/2581263231 >> 172.21.15.29:0/2581263231 conn(0x7ff5b80b1870 msgr2=0x7ff5b80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:37.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.394+0000 7ff5be5eb640 1 -- 172.21.15.29:0/2581263231 shutdown_connections 2024-04-25T00:33:37.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.394+0000 7ff5be5eb640 1 -- 172.21.15.29:0/2581263231 wait complete. 2024-04-25T00:33:37.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5be5eb640 1 Processor -- start 2024-04-25T00:33:37.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5be5eb640 1 -- start start 2024-04-25T00:33:37.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5be5eb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b806d210 0x7ff5b8142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5be5eb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b80b4b60 0x7ff5b8151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5be5eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b815df70 0x7ff5b80cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5be5eb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b806ea20 con 0x7ff5b806d210 2024-04-25T00:33:37.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5be5eb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b806d620 con 0x7ff5b815df70 2024-04-25T00:33:37.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5be5eb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b806eba0 con 0x7ff5b80b4b60 2024-04-25T00:33:37.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b806d210 0x7ff5b8142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b806d210 0x7ff5b8142530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37846/0 (socket says 172.21.15.29:37846) 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b7fff640 1 -- 172.21.15.29:0/3761259563 learned_addr learned my addr 172.21.15.29:0/3761259563 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b77fe640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b80b4b60 0x7ff5b8151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5bcb88640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b815df70 0x7ff5b80cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b7fff640 1 -- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b80b4b60 msgr2=0x7ff5b8151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b7fff640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b80b4b60 0x7ff5b8151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b7fff640 1 -- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b815df70 msgr2=0x7ff5b80cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b7fff640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b815df70 0x7ff5b80cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b7fff640 1 -- 172.21.15.29:0/3761259563 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b80cd300 con 0x7ff5b806d210 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b77fe640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b80b4b60 0x7ff5b8151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5bcb88640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b815df70 0x7ff5b80cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b7fff640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b806d210 0x7ff5b8142530 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80ae040 tx=0x7ff5a801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b4ff9640 1 -- 172.21.15.29:0/3761259563 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80b0040 con 0x7ff5b806d210 2024-04-25T00:33:37.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b4ff9640 1 -- 172.21.15.29:0/3761259563 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a80bc070 con 0x7ff5b806d210 2024-04-25T00:33:37.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5be5eb640 1 -- 172.21.15.29:0/3761259563 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b80cd5e0 con 0x7ff5b806d210 2024-04-25T00:33:37.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5b4ff9640 1 -- 172.21.15.29:0/3761259563 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80b6880 con 0x7ff5b806d210 2024-04-25T00:33:37.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.398+0000 7ff5be5eb640 1 -- 172.21.15.29:0/3761259563 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b8150980 con 0x7ff5b806d210 2024-04-25T00:33:37.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.402+0000 7ff5be5eb640 1 -- 172.21.15.29:0/3761259563 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b8069270 con 0x7ff5b806d210 2024-04-25T00:33:37.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.402+0000 7ff5b4ff9640 1 -- 172.21.15.29:0/3761259563 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7ff5a80ab480 con 0x7ff5b806d210 2024-04-25T00:33:37.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.402+0000 7ff5b4ff9640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff5a4067a70 0x7ff5a4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.402+0000 7ff5b4ff9640 1 -- 172.21.15.29:0/3761259563 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7ff5a81204f0 con 0x7ff5b806d210 2024-04-25T00:33:37.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.402+0000 7ff5b77fe640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff5a4067a70 0x7ff5a4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.402+0000 7ff5b77fe640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff5a4067a70 0x7ff5a4069f30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5b81505b0 tx=0x7ff5ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:37.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.406+0000 7ff5b4ff9640 1 -- 172.21.15.29:0/3761259563 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff5a80160d0 con 0x7ff5b806d210 2024-04-25T00:33:37.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.570+0000 7ff5be5eb640 1 -- 172.21.15.29:0/3761259563 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff5b8154990 con 0x7ff5b806d210 2024-04-25T00:33:37.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.570+0000 7ff5b4ff9640 1 -- 172.21.15.29:0/3761259563 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) ==== 100+0+11 (secure 0 0 0) 0x7ff5a80f0c90 con 0x7ff5b806d210 2024-04-25T00:33:37.572 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:33:37.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.574+0000 7ff5be5eb640 1 -- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff5a4067a70 msgr2=0x7ff5a4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.574+0000 7ff5be5eb640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff5a4067a70 0x7ff5a4069f30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5b81505b0 tx=0x7ff5ac008040 comp rx=0 tx=0).stop 2024-04-25T00:33:37.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.574+0000 7ff5be5eb640 1 -- 172.21.15.29:0/3761259563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b806d210 msgr2=0x7ff5b8142530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.574+0000 7ff5be5eb640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b806d210 0x7ff5b8142530 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80ae040 tx=0x7ff5a801ad20 comp rx=0 tx=0).stop 2024-04-25T00:33:37.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.574+0000 7ff5be5eb640 1 -- 172.21.15.29:0/3761259563 shutdown_connections 2024-04-25T00:33:37.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.574+0000 7ff5be5eb640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff5a4067a70 0x7ff5a4069f30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.574+0000 7ff5be5eb640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b815df70 0x7ff5b80cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.574+0000 7ff5be5eb640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b80b4b60 0x7ff5b8151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.574+0000 7ff5be5eb640 1 --2- 172.21.15.29:0/3761259563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b806d210 0x7ff5b8142530 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.574+0000 7ff5be5eb640 1 -- 172.21.15.29:0/3761259563 >> 172.21.15.29:0/3761259563 conn(0x7ff5b80b1870 msgr2=0x7ff5b8157b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:37.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.574+0000 7ff5be5eb640 1 -- 172.21.15.29:0/3761259563 shutdown_connections 2024-04-25T00:33:37.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.574+0000 7ff5be5eb640 1 -- 172.21.15.29:0/3761259563 wait complete. 2024-04-25T00:33:37.592 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:33:37.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc9353fa640 1 Processor -- start 2024-04-25T00:33:37.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc9353fa640 1 -- start start 2024-04-25T00:33:37.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc9353fa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc93006c950 0x7fc93006cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc9353fa640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9300cc3b0 con 0x7fc9300cd410 2024-04-25T00:33:37.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc9353fa640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc93006d270 con 0x7fc93015e440 2024-04-25T00:33:37.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc9353fa640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc9300b11d0 con 0x7fc93006c950 2024-04-25T00:33:37.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc92f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc9300cd410 0x7fc9300cd8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34314/0 (socket says 172.21.15.29:34314) 2024-04-25T00:33:37.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc92f7fe640 1 -- 172.21.15.29:0/711906787 learned_addr learned my addr 172.21.15.29:0/711906787 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:37.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc92effd640 1 --2- 172.21.15.29:0/711906787 >> v2:172.21.15.157:3300/0 conn(0x7fc93006c950 0x7fc93006cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc92dffb640 1 -- 172.21.15.29:0/711906787 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2211024395 0 0) 0x7fc9300cc3b0 con 0x7fc9300cd410 2024-04-25T00:33:37.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc92dffb640 1 -- 172.21.15.29:0/711906787 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc918003530 con 0x7fc9300cd410 2024-04-25T00:33:37.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc92effd640 1 -- 172.21.15.29:0/711906787 >> v1:172.21.15.157:6789/0 conn(0x7fc93015e440 legacy=0x7fc9300d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc92dffb640 1 -- 172.21.15.29:0/711906787 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2099149883 0 0) 0x7fc918003530 con 0x7fc9300cd410 2024-04-25T00:33:37.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc92effd640 1 -- 172.21.15.29:0/711906787 >> v1:172.21.15.29:6789/0 conn(0x7fc9300cd410 legacy=0x7fc9300cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.722+0000 7fc92effd640 1 -- 172.21.15.29:0/711906787 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9300cc650 con 0x7fc93006c950 2024-04-25T00:33:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.726+0000 7fc92effd640 1 --2- 172.21.15.29:0/711906787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc93006c950 0x7fc93006cd30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc9200097c0 tx=0x7fc92001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=302c56291ce89790 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.726+0000 7fc92dffb640 1 -- 172.21.15.29:0/711906787 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9200b0070 con 0x7fc93006c950 2024-04-25T00:33:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.726+0000 7fc92dffb640 1 -- 172.21.15.29:0/711906787 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9200ab400 con 0x7fc93006c950 2024-04-25T00:33:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.726+0000 7fc92dffb640 1 -- 172.21.15.29:0/711906787 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928001680 con 0x7fc93006c950 2024-04-25T00:33:37.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.726+0000 7fc9353fa640 1 -- 172.21.15.29:0/711906787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc93006c950 msgr2=0x7fc93006cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.726+0000 7fc9353fa640 1 --2- 172.21.15.29:0/711906787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc93006c950 0x7fc93006cd30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc9200097c0 tx=0x7fc92001c3c0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.726+0000 7fc9353fa640 1 -- 172.21.15.29:0/711906787 shutdown_connections 2024-04-25T00:33:37.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.726+0000 7fc9353fa640 1 --2- 172.21.15.29:0/711906787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc93006c950 0x7fc93006cd30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.726+0000 7fc9353fa640 1 -- 172.21.15.29:0/711906787 >> 172.21.15.29:0/711906787 conn(0x7fc9300d1d60 msgr2=0x7fc9300d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:37.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.726+0000 7fc9353fa640 1 -- 172.21.15.29:0/711906787 shutdown_connections 2024-04-25T00:33:37.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.726+0000 7fc9353fa640 1 -- 172.21.15.29:0/711906787 wait complete. 2024-04-25T00:33:37.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9353fa640 1 Processor -- start 2024-04-25T00:33:37.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9353fa640 1 -- start start 2024-04-25T00:33:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9353fa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc93006c950 0x7fc930153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9353fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9300cd410 0x7fc9300c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9353fa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc93015e440 0x7fc9300c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9353fa640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc9300d11c0 con 0x7fc93015e440 2024-04-25T00:33:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9353fa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc93006c520 con 0x7fc9300cd410 2024-04-25T00:33:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9353fa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc9300ce3c0 con 0x7fc93006c950 2024-04-25T00:33:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc92f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc93015e440 0x7fc9300c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc92f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc93015e440 0x7fc9300c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38412/0 (socket says 172.21.15.29:38412) 2024-04-25T00:33:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc92f7fe640 1 -- 172.21.15.29:0/2976281945 learned_addr learned my addr 172.21.15.29:0/2976281945 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc92effd640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc93006c950 0x7fc930153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc92e7fc640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9300cd410 0x7fc9300c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc92f7fe640 1 -- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc93006c950 msgr2=0x7fc930153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc92f7fe640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc93006c950 0x7fc930153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc92f7fe640 1 -- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9300cd410 msgr2=0x7fc9300c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc92f7fe640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9300cd410 0x7fc9300c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc92f7fe640 1 -- 172.21.15.29:0/2976281945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93014fd00 con 0x7fc93015e440 2024-04-25T00:33:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc92e7fc640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9300cd410 0x7fc9300c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc92f7fe640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc93015e440 0x7fc9300c9370 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c5d0 tx=0x7fc92800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9177fe640 1 -- 172.21.15.29:0/2976281945 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928005cf0 con 0x7fc93015e440 2024-04-25T00:33:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9177fe640 1 -- 172.21.15.29:0/2976281945 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc928007070 con 0x7fc93015e440 2024-04-25T00:33:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9353fa640 1 -- 172.21.15.29:0/2976281945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9301545d0 con 0x7fc93015e440 2024-04-25T00:33:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9177fe640 1 -- 172.21.15.29:0/2976281945 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928007460 con 0x7fc93015e440 2024-04-25T00:33:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9353fa640 1 -- 172.21.15.29:0/2976281945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc930148da0 con 0x7fc93015e440 2024-04-25T00:33:37.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9353fa640 1 -- 172.21.15.29:0/2976281945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc930069270 con 0x7fc93015e440 2024-04-25T00:33:37.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.730+0000 7fc9177fe640 1 -- 172.21.15.29:0/2976281945 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fc928007600 con 0x7fc93015e440 2024-04-25T00:33:37.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.734+0000 7fc9177fe640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc91c067a70 0x7fc91c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:37.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.734+0000 7fc9177fe640 1 -- 172.21.15.29:0/2976281945 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc928085cd0 con 0x7fc93015e440 2024-04-25T00:33:37.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.734+0000 7fc92effd640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc91c067a70 0x7fc91c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:37.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.734+0000 7fc92effd640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc91c067a70 0x7fc91c069f30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc9200ad000 tx=0x7fc9200a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:37.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.738+0000 7fc9177fe640 1 -- 172.21.15.29:0/2976281945 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc928055fd0 con 0x7fc93015e440 2024-04-25T00:33:37.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.914+0000 7fc9353fa640 1 -- 172.21.15.29:0/2976281945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc9300d21e0 con 0x7fc93015e440 2024-04-25T00:33:37.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.914+0000 7fc9177fe640 1 -- 172.21.15.29:0/2976281945 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7fc92805a2d0 con 0x7fc93015e440 2024-04-25T00:33:37.917 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:33:37.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.918+0000 7fc9353fa640 1 -- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc91c067a70 msgr2=0x7fc91c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.918+0000 7fc9353fa640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc91c067a70 0x7fc91c069f30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc9200ad000 tx=0x7fc9200a6000 comp rx=0 tx=0).stop 2024-04-25T00:33:37.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.918+0000 7fc9353fa640 1 -- 172.21.15.29:0/2976281945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc93015e440 msgr2=0x7fc9300c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:37.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.918+0000 7fc9353fa640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc93015e440 0x7fc9300c9370 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c5d0 tx=0x7fc92800ca90 comp rx=0 tx=0).stop 2024-04-25T00:33:37.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.918+0000 7fc9353fa640 1 -- 172.21.15.29:0/2976281945 shutdown_connections 2024-04-25T00:33:37.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.918+0000 7fc9353fa640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc93015e440 0x7fc9300c9370 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.918+0000 7fc9353fa640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc91c067a70 0x7fc91c069f30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.918+0000 7fc9353fa640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9300cd410 0x7fc9300c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.918+0000 7fc9353fa640 1 --2- 172.21.15.29:0/2976281945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc93006c950 0x7fc930153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:37.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.918+0000 7fc9353fa640 1 -- 172.21.15.29:0/2976281945 >> 172.21.15.29:0/2976281945 conn(0x7fc9300d1d60 msgr2=0x7fc9300b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:37.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.918+0000 7fc9353fa640 1 -- 172.21.15.29:0/2976281945 shutdown_connections 2024-04-25T00:33:37.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:37.918+0000 7fc9353fa640 1 -- 172.21.15.29:0/2976281945 wait complete. 2024-04-25T00:33:37.935 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-25T00:33:37.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:37.935 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T00:33:37.948 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[33858]: fuse finished with error 0 and tester_r 0 2024-04-25T00:33:37.950 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:33:37.965 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:33:37.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:37.966 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:33:38.014 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:33:38.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b9323640 1 Processor -- start 2024-04-25T00:33:38.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b9323640 1 -- start start 2024-04-25T00:33:38.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b9323640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5b415df70 0x7ff5b40b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b9323640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff5b406d210 0x7ff5b40b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b9323640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b40b4b60 0x7ff5b4070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b9323640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff5b40b4fd0 con 0x7ff5b415df70 2024-04-25T00:33:38.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b9323640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff5b406d980 con 0x7ff5b406d210 2024-04-25T00:33:38.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b9323640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b406dc60 con 0x7ff5b40b4b60 2024-04-25T00:33:38.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b2ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5b415df70 0x7ff5b40b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b2ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5b415df70 0x7ff5b40b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38416/0 (socket says 172.21.15.29:38416) 2024-04-25T00:33:38.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b2ffd640 1 -- 172.21.15.29:0/2355662407 learned_addr learned my addr 172.21.15.29:0/2355662407 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:38.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b37fe640 1 --2- 172.21.15.29:0/2355662407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b40b4b60 0x7ff5b4070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b27fc640 1 --2- 172.21.15.29:0/2355662407 >> v2:172.21.15.157:3300/0 conn(0x7ff5b406d210 0x7ff5b40b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b2ffd640 1 -- 172.21.15.29:0/2355662407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b40b4b60 msgr2=0x7ff5b4070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:38.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b2ffd640 1 --2- 172.21.15.29:0/2355662407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b40b4b60 0x7ff5b4070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b2ffd640 1 -- 172.21.15.29:0/2355662407 >> v2:172.21.15.157:3300/0 conn(0x7ff5b406d210 msgr2=0x7ff5b40b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:38.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b2ffd640 1 --2- 172.21.15.29:0/2355662407 >> v2:172.21.15.157:3300/0 conn(0x7ff5b406d210 0x7ff5b40b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b2ffd640 1 -- 172.21.15.29:0/2355662407 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b406df40 con 0x7ff5b415df70 2024-04-25T00:33:38.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b37fe640 1 --2- 172.21.15.29:0/2355662407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b40b4b60 0x7ff5b4070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:38.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.142+0000 7ff5b27fc640 1 --2- 172.21.15.29:0/2355662407 >> v2:172.21.15.157:3300/0 conn(0x7ff5b406d210 0x7ff5b40b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.146+0000 7ff5b2ffd640 1 --2- 172.21.15.29:0/2355662407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b415df70 0x7ff5b40b3e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40cfb70 tx=0x7ff5a401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465ae7db5aed6511 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.146+0000 7ff5b1ffb640 1 -- 172.21.15.29:0/2355662407 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40b0070 con 0x7ff5b415df70 2024-04-25T00:33:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.146+0000 7ff5b1ffb640 1 -- 172.21.15.29:0/2355662407 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a40ab600 con 0x7ff5b415df70 2024-04-25T00:33:38.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.146+0000 7ff5b1ffb640 1 -- 172.21.15.29:0/2355662407 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40ab9d0 con 0x7ff5b415df70 2024-04-25T00:33:38.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.146+0000 7ff5b9323640 1 -- 172.21.15.29:0/2355662407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b415df70 msgr2=0x7ff5b40b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.146+0000 7ff5b9323640 1 --2- 172.21.15.29:0/2355662407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b415df70 0x7ff5b40b3e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40cfb70 tx=0x7ff5a401c230 comp rx=0 tx=0).stop 2024-04-25T00:33:38.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.146+0000 7ff5b9323640 1 -- 172.21.15.29:0/2355662407 shutdown_connections 2024-04-25T00:33:38.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.146+0000 7ff5b9323640 1 --2- 172.21.15.29:0/2355662407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b40b4b60 0x7ff5b4070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.146+0000 7ff5b9323640 1 --2- 172.21.15.29:0/2355662407 >> v2:172.21.15.157:3300/0 conn(0x7ff5b406d210 0x7ff5b40b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.146+0000 7ff5b9323640 1 --2- 172.21.15.29:0/2355662407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b415df70 0x7ff5b40b3e40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.146+0000 7ff5b9323640 1 -- 172.21.15.29:0/2355662407 >> 172.21.15.29:0/2355662407 conn(0x7ff5b40b1870 msgr2=0x7ff5b40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:38.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.146+0000 7ff5b9323640 1 -- 172.21.15.29:0/2355662407 shutdown_connections 2024-04-25T00:33:38.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.146+0000 7ff5b9323640 1 -- 172.21.15.29:0/2355662407 wait complete. 2024-04-25T00:33:38.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b9323640 1 Processor -- start 2024-04-25T00:33:38.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b9323640 1 -- start start 2024-04-25T00:33:38.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b9323640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b406d210 0x7ff5b413b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b9323640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b40b4b60 0x7ff5b415ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b9323640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b415df70 0x7ff5b40ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b9323640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b406ea20 con 0x7ff5b40b4b60 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b9323640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b406d620 con 0x7ff5b406d210 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b9323640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b406eba0 con 0x7ff5b415df70 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b40b4b60 0x7ff5b415ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b40b4b60 0x7ff5b415ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38424/0 (socket says 172.21.15.29:38424) 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b27fc640 1 -- 172.21.15.29:0/3725294097 learned_addr learned my addr 172.21.15.29:0/3725294097 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b27fc640 1 -- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b415df70 msgr2=0x7ff5b40ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b37fe640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b415df70 0x7ff5b40ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b2ffd640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b406d210 0x7ff5b413b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b27fc640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b415df70 0x7ff5b40ce910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b27fc640 1 -- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b406d210 msgr2=0x7ff5b413b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b27fc640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b406d210 0x7ff5b413b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b27fc640 1 -- 172.21.15.29:0/3725294097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b4159c40 con 0x7ff5b40b4b60 2024-04-25T00:33:38.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b27fc640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b40b4b60 0x7ff5b415ace0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00e840 tx=0x7ff59c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:38.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5ab7fe640 1 -- 172.21.15.29:0/3725294097 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c00f040 con 0x7ff5b40b4b60 2024-04-25T00:33:38.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5ab7fe640 1 -- 172.21.15.29:0/3725294097 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff59c004030 con 0x7ff5b40b4b60 2024-04-25T00:33:38.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5ab7fe640 1 -- 172.21.15.29:0/3725294097 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c004400 con 0x7ff5b40b4b60 2024-04-25T00:33:38.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b9323640 1 -- 172.21.15.29:0/3725294097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b4159f20 con 0x7ff5b40b4b60 2024-04-25T00:33:38.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b9323640 1 -- 172.21.15.29:0/3725294097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b415a4e0 con 0x7ff5b40b4b60 2024-04-25T00:33:38.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.150+0000 7ff5b9323640 1 -- 172.21.15.29:0/3725294097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b4069270 con 0x7ff5b40b4b60 2024-04-25T00:33:38.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.154+0000 7ff5ab7fe640 1 -- 172.21.15.29:0/3725294097 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7ff59c00c6d0 con 0x7ff5b40b4b60 2024-04-25T00:33:38.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.154+0000 7ff5ab7fe640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff594067a70 0x7ff594069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.154+0000 7ff5ab7fe640 1 -- 172.21.15.29:0/3725294097 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7ff59c085d80 con 0x7ff5b40b4b60 2024-04-25T00:33:38.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.154+0000 7ff5b2ffd640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff594067a70 0x7ff594069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.154+0000 7ff5b2ffd640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff594067a70 0x7ff594069f30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40ad000 tx=0x7ff5a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:38.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.158+0000 7ff5ab7fe640 1 -- 172.21.15.29:0/3725294097 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff59c017020 con 0x7ff5b40b4b60 2024-04-25T00:33:38.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.358+0000 7ff5b9323640 1 -- 172.21.15.29:0/3725294097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5b4155250 con 0x7ff5b40b4b60 2024-04-25T00:33:38.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.358+0000 7ff5ab7fe640 1 -- 172.21.15.29:0/3725294097 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7ff59c056530 con 0x7ff5b40b4b60 2024-04-25T00:33:38.361 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:38.361 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:33:38.362 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:33:38.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.362+0000 7ff5b9323640 1 -- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff594067a70 msgr2=0x7ff594069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.362+0000 7ff5b9323640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff594067a70 0x7ff594069f30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40ad000 tx=0x7ff5a4008040 comp rx=0 tx=0).stop 2024-04-25T00:33:38.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.362+0000 7ff5b9323640 1 -- 172.21.15.29:0/3725294097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b40b4b60 msgr2=0x7ff5b415ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.362+0000 7ff5b9323640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b40b4b60 0x7ff5b415ace0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00e840 tx=0x7ff59c00ed00 comp rx=0 tx=0).stop 2024-04-25T00:33:38.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.362+0000 7ff5b9323640 1 -- 172.21.15.29:0/3725294097 shutdown_connections 2024-04-25T00:33:38.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.362+0000 7ff5b9323640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5b415df70 0x7ff5b40ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.362+0000 7ff5b9323640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b40b4b60 0x7ff5b415ace0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.362+0000 7ff5b9323640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff594067a70 0x7ff594069f30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.362+0000 7ff5b9323640 1 --2- 172.21.15.29:0/3725294097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b406d210 0x7ff5b413b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.362+0000 7ff5b9323640 1 -- 172.21.15.29:0/3725294097 >> 172.21.15.29:0/3725294097 conn(0x7ff5b40b1870 msgr2=0x7ff5b4154710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:38.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.362+0000 7ff5b9323640 1 -- 172.21.15.29:0/3725294097 shutdown_connections 2024-04-25T00:33:38.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.362+0000 7ff5b9323640 1 -- 172.21.15.29:0/3725294097 wait complete. 2024-04-25T00:33:38.381 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:33:38.519 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-04-25T00:33:38.520 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-04-25T00:33:38.520 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol96490 2024-04-25T00:33:38.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66b486e640 1 Processor -- start 2024-04-25T00:33:38.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66b486e640 1 -- start start 2024-04-25T00:33:38.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66b486e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f66b00b3140 0x7f66b006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66b486e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66b00b4b10 0x7f66b00b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66b486e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b00b41e0 con 0x7f66b00b3a60 2024-04-25T00:33:38.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66b486e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f66b006d960 con 0x7f66b00b3140 2024-04-25T00:33:38.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66b486e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f66b006dc40 con 0x7f66b00b4b10 2024-04-25T00:33:38.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66ae59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f66b00b3140 0x7f66b006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66aed9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f66b00b3a60 0x7f66b015c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34330/0 (socket says 172.21.15.29:34330) 2024-04-25T00:33:38.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66aed9d640 1 -- 172.21.15.29:0/1589305562 learned_addr learned my addr 172.21.15.29:0/1589305562 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:38.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66a5d9b640 1 --2- 172.21.15.29:0/1589305562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66b00b4b10 0x7f66b00b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66add9b640 1 -- 172.21.15.29:0/1589305562 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 135696977 0 0) 0x7f66b00b41e0 con 0x7f66b00b3a60 2024-04-25T00:33:38.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66add9b640 1 -- 172.21.15.29:0/1589305562 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6690003530 con 0x7f66b00b3a60 2024-04-25T00:33:38.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66ae59c640 1 -- 172.21.15.29:0/1589305562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66b00b4b10 msgr2=0x7f66b00b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:38.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66ae59c640 1 --2- 172.21.15.29:0/1589305562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66b00b4b10 0x7f66b00b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66ae59c640 1 -- 172.21.15.29:0/1589305562 >> v1:172.21.15.29:6789/0 conn(0x7f66b00b3a60 legacy=0x7f66b015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.654+0000 7f66ae59c640 1 -- 172.21.15.29:0/1589305562 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b006df20 con 0x7f66b00b3140 2024-04-25T00:33:38.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66ae59c640 1 --2- 172.21.15.29:0/1589305562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b00b3140 0x7f66b006e440 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f66a00097c0 tx=0x7f66a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74fb5ec0725c790 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66add9b640 1 -- 172.21.15.29:0/1589305562 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a00b0070 con 0x7f66b00b3140 2024-04-25T00:33:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66add9b640 1 -- 172.21.15.29:0/1589305562 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a00ab600 con 0x7f66b00b3140 2024-04-25T00:33:38.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66add9b640 1 -- 172.21.15.29:0/1589305562 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a00ab9d0 con 0x7f66b00b3140 2024-04-25T00:33:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 -- 172.21.15.29:0/1589305562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b00b3140 msgr2=0x7f66b006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 --2- 172.21.15.29:0/1589305562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b00b3140 0x7f66b006e440 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f66a00097c0 tx=0x7f66a001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 -- 172.21.15.29:0/1589305562 shutdown_connections 2024-04-25T00:33:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 --2- 172.21.15.29:0/1589305562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66b00b4b10 0x7f66b00b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 --2- 172.21.15.29:0/1589305562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b00b3140 0x7f66b006e440 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 -- 172.21.15.29:0/1589305562 >> 172.21.15.29:0/1589305562 conn(0x7f66b00b1870 msgr2=0x7f66b00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 -- 172.21.15.29:0/1589305562 shutdown_connections 2024-04-25T00:33:38.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 -- 172.21.15.29:0/1589305562 wait complete. 2024-04-25T00:33:38.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 Processor -- start 2024-04-25T00:33:38.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 -- start start 2024-04-25T00:33:38.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66b00b3140 0x7f66b0166250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b00b3a60 0x7f66b013de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66b00b4b10 0x7f66b015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0168b70 con 0x7f66b00b3a60 2024-04-25T00:33:38.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01689f0 con 0x7f66b00b4b10 2024-04-25T00:33:38.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66b486e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f66b0168cf0 con 0x7f66b00b3140 2024-04-25T00:33:38.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66a5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b00b3a60 0x7f66b013de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66a5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b00b3a60 0x7f66b013de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38446/0 (socket says 172.21.15.29:38446) 2024-04-25T00:33:38.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66a5d9b640 1 -- 172.21.15.29:0/2302603515 learned_addr learned my addr 172.21.15.29:0/2302603515 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:38.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66ae59c640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66b00b3140 0x7f66b0166250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66aed9d640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66b00b4b10 0x7f66b015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66a5d9b640 1 -- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66b00b3140 msgr2=0x7f66b0166250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66a5d9b640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66b00b3140 0x7f66b0166250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66a5d9b640 1 -- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66b00b4b10 msgr2=0x7f66b015ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66a5d9b640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66b00b4b10 0x7f66b015ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66a5d9b640 1 -- 172.21.15.29:0/2302603515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b0166790 con 0x7f66b00b3a60 2024-04-25T00:33:38.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.658+0000 7f66ae59c640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66b00b3140 0x7f66b0166250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:38.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.662+0000 7f66a5d9b640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b00b3a60 0x7f66b013de30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f669800d5f0 tx=0x7f669800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:38.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.662+0000 7f66a77fe640 1 -- 172.21.15.29:0/2302603515 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66980125f0 con 0x7f66b00b3a60 2024-04-25T00:33:38.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.662+0000 7f66a77fe640 1 -- 172.21.15.29:0/2302603515 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6698019070 con 0x7f66b00b3a60 2024-04-25T00:33:38.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.662+0000 7f66b486e640 1 -- 172.21.15.29:0/2302603515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b013ed40 con 0x7f66b00b3a60 2024-04-25T00:33:38.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.662+0000 7f66a77fe640 1 -- 172.21.15.29:0/2302603515 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66980129b0 con 0x7f66b00b3a60 2024-04-25T00:33:38.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.662+0000 7f66b486e640 1 -- 172.21.15.29:0/2302603515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b013ea60 con 0x7f66b00b3a60 2024-04-25T00:33:38.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.662+0000 7f66b486e640 1 -- 172.21.15.29:0/2302603515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b0069270 con 0x7f66b00b3a60 2024-04-25T00:33:38.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.662+0000 7f66a77fe640 1 -- 172.21.15.29:0/2302603515 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f6698012b50 con 0x7f66b00b3a60 2024-04-25T00:33:38.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.662+0000 7f66a77fe640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6694067a70 0x7f6694069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.662+0000 7f66a77fe640 1 -- 172.21.15.29:0/2302603515 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f66980863a0 con 0x7f66b00b3a60 2024-04-25T00:33:38.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.662+0000 7f66ae59c640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6694067a70 0x7f6694069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.662+0000 7f66ae59c640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6694067a70 0x7f6694069f30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f66a00ad000 tx=0x7f66a00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:38.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.666+0000 7f66a77fe640 1 -- 172.21.15.29:0/2302603515 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6698056c60 con 0x7f66b00b3a60 2024-04-25T00:33:38.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.854+0000 7f66b486e640 1 -- 172.21.15.29:0/2302603515 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol96490", "target": ["mon-mgr", ""]}) -- 0x7f66b006d530 con 0x7f6694067a70 2024-04-25T00:33:38.866 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:33:38.861+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:33:38.866 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:33:38.861+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:33:38.866 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:33:38.861+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:33:38.866 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:33:38.861+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:33:38.866 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:33:38.861+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:33:38.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.906+0000 7f66a77fe640 1 -- 172.21.15.29:0/2302603515 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f66b006d530 con 0x7f6694067a70 2024-04-25T00:33:38.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.906+0000 7f66b486e640 1 -- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6694067a70 msgr2=0x7f6694069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.906+0000 7f66b486e640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6694067a70 0x7f6694069f30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f66a00ad000 tx=0x7f66a00a6000 comp rx=0 tx=0).stop 2024-04-25T00:33:38.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.906+0000 7f66b486e640 1 -- 172.21.15.29:0/2302603515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b00b3a60 msgr2=0x7f66b013de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.906+0000 7f66b486e640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b00b3a60 0x7f66b013de30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f669800d5f0 tx=0x7f669800dab0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.906+0000 7f66b486e640 1 -- 172.21.15.29:0/2302603515 shutdown_connections 2024-04-25T00:33:38.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.906+0000 7f66b486e640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6694067a70 0x7f6694069f30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.906+0000 7f66b486e640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66b00b4b10 0x7f66b015ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.906+0000 7f66b486e640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b00b3a60 0x7f66b013de30 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.906+0000 7f66b486e640 1 --2- 172.21.15.29:0/2302603515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66b00b3140 0x7f66b0166250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.906+0000 7f66b486e640 1 -- 172.21.15.29:0/2302603515 >> 172.21.15.29:0/2302603515 conn(0x7f66b00b1870 msgr2=0x7f66b013afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:38.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.906+0000 7f66b486e640 1 -- 172.21.15.29:0/2302603515 shutdown_connections 2024-04-25T00:33:38.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:38.906+0000 7f66b486e640 1 -- 172.21.15.29:0/2302603515 wait complete. 2024-04-25T00:33:38.925 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:33:39.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdcab637640 1 Processor -- start 2024-04-25T00:33:39.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdcab637640 1 -- start start 2024-04-25T00:33:39.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdcab637640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e8d0 0x7fdca40cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdcab637640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdca40b3720 con 0x7fdca40b4780 2024-04-25T00:33:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdcab637640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdca40b39c0 con 0x7fdca415f3c0 2024-04-25T00:33:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdcab637640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdca40b42f0 con 0x7fdca415e8d0 2024-04-25T00:33:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdca8bd2640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdca40b4780 0x7fdca415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34332/0 (socket says 172.21.15.29:34332) 2024-04-25T00:33:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdca8bd2640 1 -- 172.21.15.29:0/1346321371 learned_addr learned my addr 172.21.15.29:0/1346321371 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:33:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdca93d3640 1 --2- 172.21.15.29:0/1346321371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e8d0 0x7fdca40cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 103923211 0 0) 0x7fdca40b3720 con 0x7fdca40b4780 2024-04-25T00:33:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc90003530 con 0x7fdca40b4780 2024-04-25T00:33:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 306727244 0 0) 0x7fdc90003530 con 0x7fdca40b4780 2024-04-25T00:33:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e8d0 msgr2=0x7fdca40cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 --2- 172.21.15.29:0/1346321371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e8d0 0x7fdca40cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 >> v1:172.21.15.157:6789/0 conn(0x7fdca415f3c0 legacy=0x7fdca415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca40b3ca0 con 0x7fdca40b4780 2024-04-25T00:33:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fdc94001680 con 0x7fdca40b4780 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 >> v1:172.21.15.29:6789/0 conn(0x7fdca40b4780 legacy=0x7fdca415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 --2- 172.21.15.29:0/1346321371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc900039f0 0x7fdc90003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 --2- 172.21.15.29:0/1346321371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc90004810 0x7fdc90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 --2- 172.21.15.29:0/1346321371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e8d0 0x7fdc900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdca40b42f0 con 0x7fdc90004810 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9000ba80 con 0x7fdc900039f0 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc9000bc80 con 0x7fdca415e8d0 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdca8bd2640 1 --2- 172.21.15.29:0/1346321371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc90004810 0x7fdc90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdca93d3640 1 --2- 172.21.15.29:0/1346321371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc900039f0 0x7fdc90003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdca9bd4640 1 --2- 172.21.15.29:0/1346321371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e8d0 0x7fdc900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdca8bd2640 1 -- 172.21.15.29:0/1346321371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e8d0 msgr2=0x7fdc900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdca8bd2640 1 --2- 172.21.15.29:0/1346321371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e8d0 0x7fdc900051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdca8bd2640 1 -- 172.21.15.29:0/1346321371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc900039f0 msgr2=0x7fdc90003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdca8bd2640 1 --2- 172.21.15.29:0/1346321371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc900039f0 0x7fdc90003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdca8bd2640 1 -- 172.21.15.29:0/1346321371 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc90003530 con 0x7fdc90004810 2024-04-25T00:33:39.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.042+0000 7fdca9bd4640 1 --2- 172.21.15.29:0/1346321371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e8d0 0x7fdc900051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:39.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.046+0000 7fdca8bd2640 1 --2- 172.21.15.29:0/1346321371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc90004810 0x7fdc90004c90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdc94001ec0 tx=0x7fdc9401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4abbdf7fbe0dd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:39.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.046+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94007d40 con 0x7fdc90004810 2024-04-25T00:33:39.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.046+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc94005b00 con 0x7fdc90004810 2024-04-25T00:33:39.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.046+0000 7fdc9bfff640 1 -- 172.21.15.29:0/1346321371 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94007070 con 0x7fdc90004810 2024-04-25T00:33:39.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.046+0000 7fdcab637640 1 -- 172.21.15.29:0/1346321371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc90004810 msgr2=0x7fdc90004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.046+0000 7fdcab637640 1 --2- 172.21.15.29:0/1346321371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc90004810 0x7fdc90004c90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdc94001ec0 tx=0x7fdc9401f3b0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.046+0000 7fdca9bd4640 1 -- 172.21.15.29:0/1346321371 reap_dead start 2024-04-25T00:33:39.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.046+0000 7fdcab637640 1 -- 172.21.15.29:0/1346321371 shutdown_connections 2024-04-25T00:33:39.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.046+0000 7fdcab637640 1 -- 172.21.15.29:0/1346321371 >> 172.21.15.29:0/1346321371 conn(0x7fdca40b1870 msgr2=0x7fdca40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:39.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.046+0000 7fdcab637640 1 -- 172.21.15.29:0/1346321371 shutdown_connections 2024-04-25T00:33:39.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.046+0000 7fdcab637640 1 -- 172.21.15.29:0/1346321371 wait complete. 2024-04-25T00:33:39.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.046+0000 7fdcab637640 1 Processor -- start 2024-04-25T00:33:39.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.046+0000 7fdcab637640 1 -- start start 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdcab637640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc90004810 0x7fdca414c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdcab637640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca413b4c0 0x7fdca4141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdcab637640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca41423f0 0x7fdca4142870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdcab637640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4144a20 con 0x7fdc90004810 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdcab637640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdca406cf10 con 0x7fdca413b4c0 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdcab637640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdca414c6b0 con 0x7fdca41423f0 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdca93d3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc90004810 0x7fdca414c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdca93d3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc90004810 0x7fdca414c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38460/0 (socket says 172.21.15.29:38460) 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdca93d3640 1 -- 172.21.15.29:0/1672636219 learned_addr learned my addr 172.21.15.29:0/1672636219 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdca93d3640 1 -- 172.21.15.29:0/1672636219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca41423f0 msgr2=0x7fdca4142870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdca93d3640 1 --2- 172.21.15.29:0/1672636219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca41423f0 0x7fdca4142870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdca93d3640 1 -- 172.21.15.29:0/1672636219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca413b4c0 msgr2=0x7fdca4141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdca93d3640 1 --2- 172.21.15.29:0/1672636219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca413b4c0 0x7fdca4141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdca93d3640 1 -- 172.21.15.29:0/1672636219 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca414c990 con 0x7fdc90004810 2024-04-25T00:33:39.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdca93d3640 1 --2- 172.21.15.29:0/1672636219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc90004810 0x7fdca414c0d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdca00115e0 tx=0x7fdca0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:39.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdc99ffb640 1 -- 172.21.15.29:0/1672636219 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca000f8f0 con 0x7fdc90004810 2024-04-25T00:33:39.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdc99ffb640 1 -- 172.21.15.29:0/1672636219 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdca000fa90 con 0x7fdc90004810 2024-04-25T00:33:39.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdcab637640 1 -- 172.21.15.29:0/1672636219 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca414cc70 con 0x7fdc90004810 2024-04-25T00:33:39.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdc99ffb640 1 -- 172.21.15.29:0/1672636219 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca000fe40 con 0x7fdc90004810 2024-04-25T00:33:39.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdcab637640 1 -- 172.21.15.29:0/1672636219 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca414d230 con 0x7fdc90004810 2024-04-25T00:33:39.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdcab637640 1 -- 172.21.15.29:0/1672636219 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca4069270 con 0x7fdc90004810 2024-04-25T00:33:39.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdc99ffb640 1 -- 172.21.15.29:0/1672636219 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fdca0015460 con 0x7fdc90004810 2024-04-25T00:33:39.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdc99ffb640 1 --2- 172.21.15.29:0/1672636219 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc88067a70 0x7fdc88069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdc99ffb640 1 -- 172.21.15.29:0/1672636219 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fdca00868e0 con 0x7fdc90004810 2024-04-25T00:33:39.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.050+0000 7fdca8bd2640 1 --2- 172.21.15.29:0/1672636219 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc88067a70 0x7fdc88069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.054+0000 7fdca8bd2640 1 --2- 172.21.15.29:0/1672636219 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc88067a70 0x7fdc88069f30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdc940af000 tx=0x7fdc94001a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:39.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.054+0000 7fdc99ffb640 1 -- 172.21.15.29:0/1672636219 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdca0019030 con 0x7fdc90004810 2024-04-25T00:33:39.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.246+0000 7fdcab637640 1 -- 172.21.15.29:0/1672636219 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdca406e7b0 con 0x7fdc90004810 2024-04-25T00:33:39.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.246+0000 7fdc99ffb640 1 -- 172.21.15.29:0/1672636219 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7fdca00571a0 con 0x7fdc90004810 2024-04-25T00:33:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:39.248 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:33:39.248 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:33:39.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.250+0000 7fdcab637640 1 -- 172.21.15.29:0/1672636219 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc88067a70 msgr2=0x7fdc88069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.250+0000 7fdcab637640 1 --2- 172.21.15.29:0/1672636219 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc88067a70 0x7fdc88069f30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdc940af000 tx=0x7fdc94001a50 comp rx=0 tx=0).stop 2024-04-25T00:33:39.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.250+0000 7fdcab637640 1 -- 172.21.15.29:0/1672636219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc90004810 msgr2=0x7fdca414c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.250+0000 7fdcab637640 1 --2- 172.21.15.29:0/1672636219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc90004810 0x7fdca414c0d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdca00115e0 tx=0x7fdca0011aa0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.250+0000 7fdcab637640 1 -- 172.21.15.29:0/1672636219 shutdown_connections 2024-04-25T00:33:39.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.250+0000 7fdcab637640 1 --2- 172.21.15.29:0/1672636219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca41423f0 0x7fdca4142870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.250+0000 7fdcab637640 1 --2- 172.21.15.29:0/1672636219 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdc88067a70 0x7fdc88069f30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.250+0000 7fdcab637640 1 --2- 172.21.15.29:0/1672636219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca413b4c0 0x7fdca4141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.250+0000 7fdcab637640 1 --2- 172.21.15.29:0/1672636219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc90004810 0x7fdca414c0d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.250+0000 7fdcab637640 1 -- 172.21.15.29:0/1672636219 >> 172.21.15.29:0/1672636219 conn(0x7fdca40b1870 msgr2=0x7fdca406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:39.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.250+0000 7fdcab637640 1 -- 172.21.15.29:0/1672636219 shutdown_connections 2024-04-25T00:33:39.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.250+0000 7fdcab637640 1 -- 172.21.15.29:0/1672636219 wait complete. 2024-04-25T00:33:39.266 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:33:39.410 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.52399204300000002,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.974027343569468,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:33:39.410 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.974027343569468 }, "state": "open", "uptime": 0.523992043 } ] 2024-04-25T00:33:39.411 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:33:39.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.534+0000 7f25a5736640 1 Processor -- start 2024-04-25T00:33:39.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.534+0000 7f25a5736640 1 -- start start 2024-04-25T00:33:39.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f25a5736640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f25a015e8d0 0x7f25a00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f25a5736640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f25a015f3c0 0x7f25a015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f25a5736640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25a00b3720 con 0x7f25a00b4780 2024-04-25T00:33:39.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f25a5736640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f25a00b42f0 con 0x7f25a015f3c0 2024-04-25T00:33:39.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f25a5736640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f25a00b39c0 con 0x7f25a015e8d0 2024-04-25T00:33:39.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f259f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f25a015f3c0 0x7f25a015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f259f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f25a015f3c0 0x7f25a015c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38476/0 (socket says 172.21.15.29:38476) 2024-04-25T00:33:39.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f259f7fe640 1 -- 172.21.15.29:0/1607498288 learned_addr learned my addr 172.21.15.29:0/1607498288 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:39.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f259effd640 1 --2- 172.21.15.29:0/1607498288 >> v2:172.21.15.157:3300/0 conn(0x7f25a015e8d0 0x7f25a00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f259f7fe640 1 -- 172.21.15.29:0/1607498288 >> v2:172.21.15.157:3300/0 conn(0x7f25a015e8d0 msgr2=0x7f25a00cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:39.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f259f7fe640 1 --2- 172.21.15.29:0/1607498288 >> v2:172.21.15.157:3300/0 conn(0x7f25a015e8d0 0x7f25a00cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f259f7fe640 1 -- 172.21.15.29:0/1607498288 >> v1:172.21.15.157:6789/0 conn(0x7f25a00b4780 legacy=0x7f25a015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:39.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f259f7fe640 1 -- 172.21.15.29:0/1607498288 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a00b3ca0 con 0x7f25a015f3c0 2024-04-25T00:33:39.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f259f7fe640 1 --2- 172.21.15.29:0/1607498288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25a015f3c0 0x7f25a015c7d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f25a00b5260 tx=0x7f259801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43376d044acfdab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:39.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f259e7fc640 1 -- 172.21.15.29:0/1607498288 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25980a5740 con 0x7f25a015f3c0 2024-04-25T00:33:39.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f259e7fc640 1 -- 172.21.15.29:0/1607498288 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25980a58e0 con 0x7f25a015f3c0 2024-04-25T00:33:39.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f259e7fc640 1 -- 172.21.15.29:0/1607498288 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25980a5cb0 con 0x7f25a015f3c0 2024-04-25T00:33:39.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f25a5736640 1 -- 172.21.15.29:0/1607498288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25a015f3c0 msgr2=0x7f25a015c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f25a5736640 1 --2- 172.21.15.29:0/1607498288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25a015f3c0 0x7f25a015c7d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f25a00b5260 tx=0x7f259801c9b0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f25a5736640 1 -- 172.21.15.29:0/1607498288 shutdown_connections 2024-04-25T00:33:39.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f25a5736640 1 --2- 172.21.15.29:0/1607498288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25a015f3c0 0x7f25a015c7d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f25a5736640 1 --2- 172.21.15.29:0/1607498288 >> v2:172.21.15.157:3300/0 conn(0x7f25a015e8d0 0x7f25a00cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.538+0000 7f25a5736640 1 -- 172.21.15.29:0/1607498288 >> 172.21.15.29:0/1607498288 conn(0x7f25a00b1870 msgr2=0x7f25a00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:39.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25a5736640 1 -- 172.21.15.29:0/1607498288 shutdown_connections 2024-04-25T00:33:39.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25a5736640 1 -- 172.21.15.29:0/1607498288 wait complete. 2024-04-25T00:33:39.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25a5736640 1 Processor -- start 2024-04-25T00:33:39.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25a5736640 1 -- start start 2024-04-25T00:33:39.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25a5736640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25a00b4780 0x7f25a015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25a5736640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25a015e8d0 0x7f25a00d0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25a5736640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25a015f3c0 0x7f25a00d0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25a5736640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f25a015d6a0 con 0x7f25a015e8d0 2024-04-25T00:33:39.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25a5736640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f25a015d520 con 0x7f25a00b4780 2024-04-25T00:33:39.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25a5736640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f25a0167c70 con 0x7f25a015f3c0 2024-04-25T00:33:39.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25967fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25a015e8d0 0x7f25a00d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f259effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25a00b4780 0x7f25a015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25967fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25a015e8d0 0x7f25a00d0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38480/0 (socket says 172.21.15.29:38480) 2024-04-25T00:33:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f259f7fe640 1 --2- 172.21.15.29:0/689328672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25a015f3c0 0x7f25a00d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25967fc640 1 -- 172.21.15.29:0/689328672 learned_addr learned my addr 172.21.15.29:0/689328672 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25967fc640 1 -- 172.21.15.29:0/689328672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25a015f3c0 msgr2=0x7f25a00d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25967fc640 1 --2- 172.21.15.29:0/689328672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25a015f3c0 0x7f25a00d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25967fc640 1 -- 172.21.15.29:0/689328672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25a00b4780 msgr2=0x7f25a015a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25967fc640 1 --2- 172.21.15.29:0/689328672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25a00b4780 0x7f25a015a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25967fc640 1 -- 172.21.15.29:0/689328672 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a00b09a0 con 0x7f25a015e8d0 2024-04-25T00:33:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25967fc640 1 --2- 172.21.15.29:0/689328672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25a015e8d0 0x7f25a00d0140 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f258800c450 tx=0x7f258800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f2597fff640 1 -- 172.21.15.29:0/689328672 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588012830 con 0x7f25a015e8d0 2024-04-25T00:33:39.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f2597fff640 1 -- 172.21.15.29:0/689328672 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2588017070 con 0x7f25a015e8d0 2024-04-25T00:33:39.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25a5736640 1 -- 172.21.15.29:0/689328672 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a0139580 con 0x7f25a015e8d0 2024-04-25T00:33:39.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f2597fff640 1 -- 172.21.15.29:0/689328672 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588012c30 con 0x7f25a015e8d0 2024-04-25T00:33:39.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.542+0000 7f25a5736640 1 -- 172.21.15.29:0/689328672 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a00ce070 con 0x7f25a015e8d0 2024-04-25T00:33:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.546+0000 7f25a5736640 1 -- 172.21.15.29:0/689328672 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a0069270 con 0x7f25a015e8d0 2024-04-25T00:33:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.546+0000 7f2597fff640 1 -- 172.21.15.29:0/689328672 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f258801c460 con 0x7f25a015e8d0 2024-04-25T00:33:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.546+0000 7f2597fff640 1 --2- 172.21.15.29:0/689328672 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2580067aa0 0x7f2580069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.546+0000 7f2597fff640 1 -- 172.21.15.29:0/689328672 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f2588085c00 con 0x7f25a015e8d0 2024-04-25T00:33:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.546+0000 7f259effd640 1 --2- 172.21.15.29:0/689328672 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2580067aa0 0x7f2580069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.546+0000 7f259effd640 1 --2- 172.21.15.29:0/689328672 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2580067aa0 0x7f2580069f60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2590006fd0 tx=0x7f2590008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.550+0000 7f2597fff640 1 -- 172.21.15.29:0/689328672 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f258808c080 con 0x7f25a015e8d0 2024-04-25T00:33:39.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.738+0000 7f25a5736640 1 -- 172.21.15.29:0/689328672 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25a00afac0 con 0x7f25a015e8d0 2024-04-25T00:33:39.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.742+0000 7f2597fff640 1 -- 172.21.15.29:0/689328672 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7f25a0139580 con 0x7f25a015e8d0 2024-04-25T00:33:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:39.742 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:33:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:33:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.742+0000 7f25a5736640 1 -- 172.21.15.29:0/689328672 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2580067aa0 msgr2=0x7f2580069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.742+0000 7f25a5736640 1 --2- 172.21.15.29:0/689328672 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2580067aa0 0x7f2580069f60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2590006fd0 tx=0x7f2590008040 comp rx=0 tx=0).stop 2024-04-25T00:33:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.742+0000 7f25a5736640 1 -- 172.21.15.29:0/689328672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25a015e8d0 msgr2=0x7f25a00d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.742+0000 7f25a5736640 1 --2- 172.21.15.29:0/689328672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25a015e8d0 0x7f25a00d0140 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f258800c450 tx=0x7f258800c910 comp rx=0 tx=0).stop 2024-04-25T00:33:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.742+0000 7f25a5736640 1 -- 172.21.15.29:0/689328672 shutdown_connections 2024-04-25T00:33:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.742+0000 7f25a5736640 1 --2- 172.21.15.29:0/689328672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25a015f3c0 0x7f25a00d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.742+0000 7f25a5736640 1 --2- 172.21.15.29:0/689328672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25a015e8d0 0x7f25a00d0140 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.742+0000 7f25a5736640 1 --2- 172.21.15.29:0/689328672 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2580067aa0 0x7f2580069f60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.742+0000 7f25a5736640 1 --2- 172.21.15.29:0/689328672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25a00b4780 0x7f25a015a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.742+0000 7f25a5736640 1 -- 172.21.15.29:0/689328672 >> 172.21.15.29:0/689328672 conn(0x7f25a00b1870 msgr2=0x7f25a00b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.742+0000 7f25a5736640 1 -- 172.21.15.29:0/689328672 shutdown_connections 2024-04-25T00:33:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:39.742+0000 7f25a5736640 1 -- 172.21.15.29:0/689328672 wait complete. 2024-04-25T00:33:39.760 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:33:39.902 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":1.0159846960000001,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.949062682927831,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:33:39.903 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.94906268292783 }, "state": "open", "uptime": 1.015984696 } ] 2024-04-25T00:33:39.903 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-25T00:33:40.025 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:33:40.028 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:33:44.904 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:33:45.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0be5fa640 1 Processor -- start 2024-04-25T00:33:45.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0be5fa640 1 -- start start 2024-04-25T00:33:45.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0be5fa640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb0b815e200 0x7fb0b80d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:45.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0be5fa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb0b80cf690 0x7fb0b80d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:45.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0be5fa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0b80d2a90 0x7fb0b80d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:45.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0be5fa640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb0b80d2ee0 con 0x7fb0b815e200 2024-04-25T00:33:45.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0be5fa640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb0b80b38d0 con 0x7fb0b80cf690 2024-04-25T00:33:45.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0be5fa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb0b80cfc10 con 0x7fb0b80d2a90 2024-04-25T00:33:45.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0b7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb0b815e200 0x7fb0b80d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:45.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0b7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb0b815e200 0x7fb0b80d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38482/0 (socket says 172.21.15.29:38482) 2024-04-25T00:33:45.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0b7fff640 1 -- 172.21.15.29:0/4124010362 learned_addr learned my addr 172.21.15.29:0/4124010362 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:45.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0b77fe640 1 --2- 172.21.15.29:0/4124010362 >> v2:172.21.15.157:3300/0 conn(0x7fb0b80cf690 0x7fb0b80d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:45.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0b7fff640 1 -- 172.21.15.29:0/4124010362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0b80d2a90 msgr2=0x7fb0b80d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:45.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0b7fff640 1 --2- 172.21.15.29:0/4124010362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0b80d2a90 0x7fb0b80d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:45.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0b7fff640 1 -- 172.21.15.29:0/4124010362 >> v2:172.21.15.157:3300/0 conn(0x7fb0b80cf690 msgr2=0x7fb0b80d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:45.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0b7fff640 1 --2- 172.21.15.29:0/4124010362 >> v2:172.21.15.157:3300/0 conn(0x7fb0b80cf690 0x7fb0b80d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:45.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0b7fff640 1 -- 172.21.15.29:0/4124010362 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b80cfef0 con 0x7fb0b815e200 2024-04-25T00:33:45.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.042+0000 7fb0b77fe640 1 --2- 172.21.15.29:0/4124010362 >> v2:172.21.15.157:3300/0 conn(0x7fb0b80cf690 0x7fb0b80d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:45.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0b7fff640 1 --2- 172.21.15.29:0/4124010362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0b815e200 0x7fb0b80d47f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80d24a0 tx=0x7fb0a801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37472089e036c1e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:45.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0b6ffd640 1 -- 172.21.15.29:0/4124010362 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a80b0070 con 0x7fb0b815e200 2024-04-25T00:33:45.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0b6ffd640 1 -- 172.21.15.29:0/4124010362 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a80ab600 con 0x7fb0b815e200 2024-04-25T00:33:45.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0b6ffd640 1 -- 172.21.15.29:0/4124010362 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a80ab9d0 con 0x7fb0b815e200 2024-04-25T00:33:45.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 -- 172.21.15.29:0/4124010362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0b815e200 msgr2=0x7fb0b80d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:45.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 --2- 172.21.15.29:0/4124010362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0b815e200 0x7fb0b80d47f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80d24a0 tx=0x7fb0a801c230 comp rx=0 tx=0).stop 2024-04-25T00:33:45.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 -- 172.21.15.29:0/4124010362 shutdown_connections 2024-04-25T00:33:45.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 --2- 172.21.15.29:0/4124010362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0b80d2a90 0x7fb0b80d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:45.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 --2- 172.21.15.29:0/4124010362 >> v2:172.21.15.157:3300/0 conn(0x7fb0b80cf690 0x7fb0b80d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:45.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 --2- 172.21.15.29:0/4124010362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0b815e200 0x7fb0b80d47f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:45.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 -- 172.21.15.29:0/4124010362 >> 172.21.15.29:0/4124010362 conn(0x7fb0b809d280 msgr2=0x7fb0b80d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:45.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 -- 172.21.15.29:0/4124010362 shutdown_connections 2024-04-25T00:33:45.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 -- 172.21.15.29:0/4124010362 wait complete. 2024-04-25T00:33:45.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 Processor -- start 2024-04-25T00:33:45.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 -- start start 2024-04-25T00:33:45.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0b80cf690 0x7fb0b815d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:45.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0b80d2a90 0x7fb0b80ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:45.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b815e200 0x7fb0b8143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:45.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b815e760 con 0x7fb0b80d2a90 2024-04-25T00:33:45.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b815e5e0 con 0x7fb0b815e200 2024-04-25T00:33:45.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0be5fa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb0b80b4320 con 0x7fb0b80cf690 2024-04-25T00:33:45.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0b77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0b80d2a90 0x7fb0b80ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:45.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0b77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0b80d2a90 0x7fb0b80ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38494/0 (socket says 172.21.15.29:38494) 2024-04-25T00:33:45.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0b77fe640 1 -- 172.21.15.29:0/2630625506 learned_addr learned my addr 172.21.15.29:0/2630625506 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:45.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.046+0000 7fb0b7fff640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0b80cf690 0x7fb0b815d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:45.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0bcb97640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b815e200 0x7fb0b8143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:45.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0b77fe640 1 -- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0b80cf690 msgr2=0x7fb0b815d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:45.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0b77fe640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0b80cf690 0x7fb0b815d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:45.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0b77fe640 1 -- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b815e200 msgr2=0x7fb0b8143080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:45.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0b77fe640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b815e200 0x7fb0b8143080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:45.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0b77fe640 1 -- 172.21.15.29:0/2630625506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b815bd70 con 0x7fb0b80d2a90 2024-04-25T00:33:45.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0bcb97640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b815e200 0x7fb0b8143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:45.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0b77fe640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0b80d2a90 0x7fb0b80ca1d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00c7c0 tx=0x7fb0ac00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:45.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0b4ff9640 1 -- 172.21.15.29:0/2630625506 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac007a30 con 0x7fb0b80d2a90 2024-04-25T00:33:45.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0b4ff9640 1 -- 172.21.15.29:0/2630625506 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ac007bd0 con 0x7fb0b80d2a90 2024-04-25T00:33:45.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0b4ff9640 1 -- 172.21.15.29:0/2630625506 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0113f0 con 0x7fb0b80d2a90 2024-04-25T00:33:45.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0be5fa640 1 -- 172.21.15.29:0/2630625506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b80cd650 con 0x7fb0b80d2a90 2024-04-25T00:33:45.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0be5fa640 1 -- 172.21.15.29:0/2630625506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b815c2d0 con 0x7fb0b80d2a90 2024-04-25T00:33:45.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0be5fa640 1 -- 172.21.15.29:0/2630625506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b8069270 con 0x7fb0b80d2a90 2024-04-25T00:33:45.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0b4ff9640 1 -- 172.21.15.29:0/2630625506 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7fb0ac011590 con 0x7fb0b80d2a90 2024-04-25T00:33:45.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0b4ff9640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb0a4067b00 0x7fb0a4069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:45.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0b4ff9640 1 -- 172.21.15.29:0/2630625506 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fb0ac086390 con 0x7fb0b80d2a90 2024-04-25T00:33:45.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.050+0000 7fb0b7fff640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb0a4067b00 0x7fb0a4069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:45.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.054+0000 7fb0b7fff640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb0a4067b00 0x7fb0a4069fc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80ad000 tx=0x7fb0a80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:45.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.054+0000 7fb0b4ff9640 1 -- 172.21.15.29:0/2630625506 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb0b815bd70 con 0x7fb0b80d2a90 2024-04-25T00:33:45.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.250+0000 7fb0be5fa640 1 -- 172.21.15.29:0/2630625506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0b80d32b0 con 0x7fb0b80d2a90 2024-04-25T00:33:45.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.250+0000 7fb0b4ff9640 1 -- 172.21.15.29:0/2630625506 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7fb0ac015020 con 0x7fb0b80d2a90 2024-04-25T00:33:45.252 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:45.252 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:33:45.252 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:33:45.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.254+0000 7fb0be5fa640 1 -- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb0a4067b00 msgr2=0x7fb0a4069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:45.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.254+0000 7fb0be5fa640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb0a4067b00 0x7fb0a4069fc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80ad000 tx=0x7fb0a80a6000 comp rx=0 tx=0).stop 2024-04-25T00:33:45.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.254+0000 7fb0be5fa640 1 -- 172.21.15.29:0/2630625506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0b80d2a90 msgr2=0x7fb0b80ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:45.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.254+0000 7fb0be5fa640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0b80d2a90 0x7fb0b80ca1d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00c7c0 tx=0x7fb0ac00cc80 comp rx=0 tx=0).stop 2024-04-25T00:33:45.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.254+0000 7fb0be5fa640 1 -- 172.21.15.29:0/2630625506 shutdown_connections 2024-04-25T00:33:45.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.254+0000 7fb0be5fa640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb0a4067b00 0x7fb0a4069fc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:45.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.254+0000 7fb0be5fa640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b815e200 0x7fb0b8143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:45.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.254+0000 7fb0be5fa640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0b80d2a90 0x7fb0b80ca1d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:45.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.254+0000 7fb0be5fa640 1 --2- 172.21.15.29:0/2630625506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0b80cf690 0x7fb0b815d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:45.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.254+0000 7fb0be5fa640 1 -- 172.21.15.29:0/2630625506 >> 172.21.15.29:0/2630625506 conn(0x7fb0b809d280 msgr2=0x7fb0b80d3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:45.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.254+0000 7fb0be5fa640 1 -- 172.21.15.29:0/2630625506 shutdown_connections 2024-04-25T00:33:45.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:45.254+0000 7fb0be5fa640 1 -- 172.21.15.29:0/2630625506 wait complete. 2024-04-25T00:33:45.270 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:33:45.407 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":6.5239038320000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.67150883254418,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:33:45.408 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.67150883254418 }, "state": "open", "uptime": 6.523903832 } ] 2024-04-25T00:33:45.408 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-25T00:33:50.410 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:33:50.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3328aec640 1 Processor -- start 2024-04-25T00:33:50.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3328aec640 1 -- start start 2024-04-25T00:33:50.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3328aec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33240cf690 0x7f33240d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:50.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3328aec640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33240d2a90 0x7f33240b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:50.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3328aec640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33240cfc10 con 0x7f332415e200 2024-04-25T00:33:50.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3328aec640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f33240d2ee0 con 0x7f33240d2a90 2024-04-25T00:33:50.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3328aec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f33240b38d0 con 0x7f33240cf690 2024-04-25T00:33:50.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3322d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33240d2a90 0x7f33240b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:50.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3322d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33240d2a90 0x7f33240b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51606/0 (socket says 172.21.15.29:51606) 2024-04-25T00:33:50.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3322d9d640 1 -- 172.21.15.29:0/3479829091 learned_addr learned my addr 172.21.15.29:0/3479829091 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:50.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3321d9b640 1 --2- 172.21.15.29:0/3479829091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33240cf690 0x7f33240d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:50.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3322d9d640 1 -- 172.21.15.29:0/3479829091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33240cf690 msgr2=0x7f33240d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:50.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3322d9d640 1 --2- 172.21.15.29:0/3479829091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33240cf690 0x7f33240d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:50.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3322d9d640 1 -- 172.21.15.29:0/3479829091 >> v1:172.21.15.157:6789/0 conn(0x7f332415e200 legacy=0x7f33240d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:50.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.545+0000 7f3322d9d640 1 -- 172.21.15.29:0/3479829091 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33240cfef0 con 0x7f33240d2a90 2024-04-25T00:33:50.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3322d9d640 1 --2- 172.21.15.29:0/3479829091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33240d2a90 0x7f33240b4320 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f331c009870 tx=0x7f331c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25450b731ad1d1c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:50.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f332159a640 1 -- 172.21.15.29:0/3479829091 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c0ab650 con 0x7f33240d2a90 2024-04-25T00:33:50.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f332159a640 1 -- 172.21.15.29:0/3479829091 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f331c0b0070 con 0x7f33240d2a90 2024-04-25T00:33:50.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f332159a640 1 -- 172.21.15.29:0/3479829091 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c0aba10 con 0x7f33240d2a90 2024-04-25T00:33:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 -- 172.21.15.29:0/3479829091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33240d2a90 msgr2=0x7f33240b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 --2- 172.21.15.29:0/3479829091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33240d2a90 0x7f33240b4320 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f331c009870 tx=0x7f331c01c560 comp rx=0 tx=0).stop 2024-04-25T00:33:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 -- 172.21.15.29:0/3479829091 shutdown_connections 2024-04-25T00:33:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 --2- 172.21.15.29:0/3479829091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33240d2a90 0x7f33240b4320 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 --2- 172.21.15.29:0/3479829091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33240cf690 0x7f33240d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 -- 172.21.15.29:0/3479829091 >> 172.21.15.29:0/3479829091 conn(0x7f332409d280 msgr2=0x7f33240d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 -- 172.21.15.29:0/3479829091 shutdown_connections 2024-04-25T00:33:50.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 -- 172.21.15.29:0/3479829091 wait complete. 2024-04-25T00:33:50.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 Processor -- start 2024-04-25T00:33:50.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 -- start start 2024-04-25T00:33:50.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33240cf690 0x7f33241573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:50.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33240d2a90 0x7f332415fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:50.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f332415e200 0x7f332415c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:50.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f33240b51f0 con 0x7f33240d2a90 2024-04-25T00:33:50.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33240b5070 con 0x7f33240cf690 2024-04-25T00:33:50.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3328aec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f33240b5370 con 0x7f332415e200 2024-04-25T00:33:50.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3321d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33240d2a90 0x7f332415fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:50.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3321d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33240d2a90 0x7f332415fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51622/0 (socket says 172.21.15.29:51622) 2024-04-25T00:33:50.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3321d9b640 1 -- 172.21.15.29:0/2479537811 learned_addr learned my addr 172.21.15.29:0/2479537811 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:50.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f332259c640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33240cf690 0x7f33241573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:50.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.549+0000 7f3322d9d640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f332415e200 0x7f332415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:50.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f3321d9b640 1 -- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f332415e200 msgr2=0x7f332415c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:50.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f3321d9b640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f332415e200 0x7f332415c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:50.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f3321d9b640 1 -- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33240cf690 msgr2=0x7f33241573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:50.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f3321d9b640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33240cf690 0x7f33241573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:50.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f3321d9b640 1 -- 172.21.15.29:0/2479537811 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3324161720 con 0x7f33240d2a90 2024-04-25T00:33:50.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f3321d9b640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33240d2a90 0x7f332415fd10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f330c007ef0 tx=0x7f330c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:50.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f331affd640 1 -- 172.21.15.29:0/2479537811 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c014620 con 0x7f33240d2a90 2024-04-25T00:33:50.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f331affd640 1 -- 172.21.15.29:0/2479537811 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f330c019070 con 0x7f33240d2a90 2024-04-25T00:33:50.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f3328aec640 1 -- 172.21.15.29:0/2479537811 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3324160680 con 0x7f33240d2a90 2024-04-25T00:33:50.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f331affd640 1 -- 172.21.15.29:0/2479537811 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c014a00 con 0x7f33240d2a90 2024-04-25T00:33:50.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f3328aec640 1 -- 172.21.15.29:0/2479537811 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3324160c40 con 0x7f33240d2a90 2024-04-25T00:33:50.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f3328aec640 1 -- 172.21.15.29:0/2479537811 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3324069270 con 0x7f33240d2a90 2024-04-25T00:33:50.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f331affd640 1 -- 172.21.15.29:0/2479537811 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f330c014ba0 con 0x7f33240d2a90 2024-04-25T00:33:50.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f331affd640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3304067b00 0x7f3304069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:50.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f331affd640 1 -- 172.21.15.29:0/2479537811 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f330c0868d0 con 0x7f33240d2a90 2024-04-25T00:33:50.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f332259c640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3304067b00 0x7f3304069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:50.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.553+0000 7f332259c640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3304067b00 0x7f3304069fc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f33140015b0 tx=0x7f3314009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:50.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.557+0000 7f331affd640 1 -- 172.21.15.29:0/2479537811 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f330c057210 con 0x7f33240d2a90 2024-04-25T00:33:50.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.753+0000 7f3328aec640 1 -- 172.21.15.29:0/2479537811 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33240c7230 con 0x7f33240d2a90 2024-04-25T00:33:50.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.757+0000 7f331affd640 1 -- 172.21.15.29:0/2479537811 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7f330c056b10 con 0x7f33240d2a90 2024-04-25T00:33:50.758 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:50.758 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:33:50.758 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:33:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.757+0000 7f3328aec640 1 -- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3304067b00 msgr2=0x7f3304069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.757+0000 7f3328aec640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3304067b00 0x7f3304069fc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f33140015b0 tx=0x7f3314009040 comp rx=0 tx=0).stop 2024-04-25T00:33:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.757+0000 7f3328aec640 1 -- 172.21.15.29:0/2479537811 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33240d2a90 msgr2=0x7f332415fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.757+0000 7f3328aec640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33240d2a90 0x7f332415fd10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f330c007ef0 tx=0x7f330c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:33:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.757+0000 7f3328aec640 1 -- 172.21.15.29:0/2479537811 shutdown_connections 2024-04-25T00:33:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.757+0000 7f3328aec640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f332415e200 0x7f332415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.757+0000 7f3328aec640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33240d2a90 0x7f332415fd10 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.757+0000 7f3328aec640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3304067b00 0x7f3304069fc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.757+0000 7f3328aec640 1 --2- 172.21.15.29:0/2479537811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33240cf690 0x7f33241573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.757+0000 7f3328aec640 1 -- 172.21.15.29:0/2479537811 >> 172.21.15.29:0/2479537811 conn(0x7f332409d280 msgr2=0x7f33240d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.757+0000 7f3328aec640 1 -- 172.21.15.29:0/2479537811 shutdown_connections 2024-04-25T00:33:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:50.757+0000 7f3328aec640 1 -- 172.21.15.29:0/2479537811 wait complete. 2024-04-25T00:33:50.775 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:33:50.917 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":12.031827992,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.397464502784409,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:33:50.918 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.39746450278441 }, "state": "open", "uptime": 12.031827992 } ] 2024-04-25T00:33:50.918 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-25T00:33:55.919 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:33:56.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.053+0000 7fefce690640 1 Processor -- start 2024-04-25T00:33:56.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.053+0000 7fefce690640 1 -- start start 2024-04-25T00:33:56.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefce690640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc815e200 0x7fefc80d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:56.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefce690640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fefc80cf690 0x7fefc80d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:56.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefce690640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc80cfc10 con 0x7fefc80d2a90 2024-04-25T00:33:56.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefce690640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fefc80d2ee0 con 0x7fefc80cf690 2024-04-25T00:33:56.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefce690640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fefc80b38d0 con 0x7fefc815e200 2024-04-25T00:33:56.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefc77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fefc80cf690 0x7fefc80d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:56.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefc77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fefc80cf690 0x7fefc80d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51636/0 (socket says 172.21.15.29:51636) 2024-04-25T00:33:56.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefc77fe640 1 -- 172.21.15.29:0/2722307394 learned_addr learned my addr 172.21.15.29:0/2722307394 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:56.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefc7fff640 1 --2- 172.21.15.29:0/2722307394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc815e200 0x7fefc80d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:56.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefc77fe640 1 -- 172.21.15.29:0/2722307394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc815e200 msgr2=0x7fefc80d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:56.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefc77fe640 1 --2- 172.21.15.29:0/2722307394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc815e200 0x7fefc80d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:56.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefc77fe640 1 -- 172.21.15.29:0/2722307394 >> v1:172.21.15.157:6789/0 conn(0x7fefc80d2a90 legacy=0x7fefc80d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:56.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefc77fe640 1 -- 172.21.15.29:0/2722307394 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc80cfef0 con 0x7fefc80cf690 2024-04-25T00:33:56.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefc77fe640 1 --2- 172.21.15.29:0/2722307394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefc80cf690 0x7fefc80d2550 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009870 tx=0x7fefbc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f916d5eeb1e031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:56.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefc6ffd640 1 -- 172.21.15.29:0/2722307394 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc0aa610 con 0x7fefc80cf690 2024-04-25T00:33:56.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefc6ffd640 1 -- 172.21.15.29:0/2722307394 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefbc0af070 con 0x7fefc80cf690 2024-04-25T00:33:56.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.057+0000 7fefc6ffd640 1 -- 172.21.15.29:0/2722307394 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc0aa9d0 con 0x7fefc80cf690 2024-04-25T00:33:56.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 -- 172.21.15.29:0/2722307394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefc80cf690 msgr2=0x7fefc80d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:56.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 --2- 172.21.15.29:0/2722307394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefc80cf690 0x7fefc80d2550 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009870 tx=0x7fefbc01c4f0 comp rx=0 tx=0).stop 2024-04-25T00:33:56.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 -- 172.21.15.29:0/2722307394 shutdown_connections 2024-04-25T00:33:56.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 --2- 172.21.15.29:0/2722307394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefc80cf690 0x7fefc80d2550 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:56.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 --2- 172.21.15.29:0/2722307394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc815e200 0x7fefc80d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:56.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 -- 172.21.15.29:0/2722307394 >> 172.21.15.29:0/2722307394 conn(0x7fefc809d280 msgr2=0x7fefc80d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:56.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 -- 172.21.15.29:0/2722307394 shutdown_connections 2024-04-25T00:33:56.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 -- 172.21.15.29:0/2722307394 wait complete. 2024-04-25T00:33:56.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 Processor -- start 2024-04-25T00:33:56.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 -- start start 2024-04-25T00:33:56.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc80cf690 0x7fefc8153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:56.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc80d2a90 0x7fefc8152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:56.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefc815e200 0x7fefc80cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:56.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fefc815e760 con 0x7fefc815e200 2024-04-25T00:33:56.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fefc815e5e0 con 0x7fefc80cf690 2024-04-25T00:33:56.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefce690640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fefc816ae30 con 0x7fefc80d2a90 2024-04-25T00:33:56.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefccc2d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefc815e200 0x7fefc80cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:56.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefccc2d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefc815e200 0x7fefc80cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51640/0 (socket says 172.21.15.29:51640) 2024-04-25T00:33:56.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefccc2d640 1 -- 172.21.15.29:0/2150142338 learned_addr learned my addr 172.21.15.29:0/2150142338 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:33:56.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefc77fe640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc80d2a90 0x7fefc8152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:56.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.061+0000 7fefc7fff640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc80cf690 0x7fefc8153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:56.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefccc2d640 1 -- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc80d2a90 msgr2=0x7fefc8152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:56.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefccc2d640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc80d2a90 0x7fefc8152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:56.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefccc2d640 1 -- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc80cf690 msgr2=0x7fefc8153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:56.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefccc2d640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc80cf690 0x7fefc8153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:56.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefccc2d640 1 -- 172.21.15.29:0/2150142338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc8154a50 con 0x7fefc815e200 2024-04-25T00:33:56.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefccc2d640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefc815e200 0x7fefc80cd440 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fefc000c7b0 tx=0x7fefc000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:56.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefc4ff9640 1 -- 172.21.15.29:0/2150142338 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0004b70 con 0x7fefc815e200 2024-04-25T00:33:56.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefc4ff9640 1 -- 172.21.15.29:0/2150142338 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc0004d10 con 0x7fefc815e200 2024-04-25T00:33:56.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefce690640 1 -- 172.21.15.29:0/2150142338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc8153d90 con 0x7fefc815e200 2024-04-25T00:33:56.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefc4ff9640 1 -- 172.21.15.29:0/2150142338 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc000aa90 con 0x7fefc815e200 2024-04-25T00:33:56.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefce690640 1 -- 172.21.15.29:0/2150142338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc80cc680 con 0x7fefc815e200 2024-04-25T00:33:56.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefce690640 1 -- 172.21.15.29:0/2150142338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc8069270 con 0x7fefc815e200 2024-04-25T00:33:56.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefc4ff9640 1 -- 172.21.15.29:0/2150142338 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7fefc000ac30 con 0x7fefc815e200 2024-04-25T00:33:56.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefc4ff9640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fefb4067b30 0x7fefb4069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:56.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefc4ff9640 1 -- 172.21.15.29:0/2150142338 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fefc00869e0 con 0x7fefc815e200 2024-04-25T00:33:56.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefc7fff640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fefb4067b30 0x7fefb4069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:56.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.065+0000 7fefc7fff640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fefb4067b30 0x7fefb4069ff0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fefb800b9d0 tx=0x7fefb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:56.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.069+0000 7fefc4ff9640 1 -- 172.21.15.29:0/2150142338 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fefc0020070 con 0x7fefc815e200 2024-04-25T00:33:56.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.265+0000 7fefce690640 1 -- 172.21.15.29:0/2150142338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefc8157420 con 0x7fefc815e200 2024-04-25T00:33:56.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.269+0000 7fefc4ff9640 1 -- 172.21.15.29:0/2150142338 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7fefc8154a50 con 0x7fefc815e200 2024-04-25T00:33:56.269 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:33:56.270 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:33:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:33:56.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.269+0000 7fefce690640 1 -- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fefb4067b30 msgr2=0x7fefb4069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.269+0000 7fefce690640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fefb4067b30 0x7fefb4069ff0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fefb800b9d0 tx=0x7fefb8008040 comp rx=0 tx=0).stop 2024-04-25T00:33:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.269+0000 7fefce690640 1 -- 172.21.15.29:0/2150142338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefc815e200 msgr2=0x7fefc80cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.269+0000 7fefce690640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefc815e200 0x7fefc80cd440 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fefc000c7b0 tx=0x7fefc000cc70 comp rx=0 tx=0).stop 2024-04-25T00:33:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.269+0000 7fefce690640 1 -- 172.21.15.29:0/2150142338 shutdown_connections 2024-04-25T00:33:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.269+0000 7fefce690640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefc815e200 0x7fefc80cd440 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.269+0000 7fefce690640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefc80d2a90 0x7fefc8152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.269+0000 7fefce690640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fefb4067b30 0x7fefb4069ff0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.269+0000 7fefce690640 1 --2- 172.21.15.29:0/2150142338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc80cf690 0x7fefc8153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.269+0000 7fefce690640 1 -- 172.21.15.29:0/2150142338 >> 172.21.15.29:0/2150142338 conn(0x7fefc809d280 msgr2=0x7fefc806d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.269+0000 7fefce690640 1 -- 172.21.15.29:0/2150142338 shutdown_connections 2024-04-25T00:33:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:33:56.269+0000 7fefce690640 1 -- 172.21.15.29:0/2150142338 wait complete. 2024-04-25T00:33:56.289 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:33:56.424 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":17.539756745999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.126885341760129,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:33:56.425 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.12688534176013 }, "state": "open", "uptime": 17.539756746 } ] 2024-04-25T00:33:56.425 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-25T00:34:01.426 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:01.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.557+0000 7f30c4bfb640 1 Processor -- start 2024-04-25T00:34:01.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.557+0000 7f30c4bfb640 1 -- start start 2024-04-25T00:34:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30c4bfb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c015e200 0x7f30c00d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:01.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30c4bfb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30c00cf690 0x7f30c00d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30c4bfb640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30c00cfc10 con 0x7f30c00d2a90 2024-04-25T00:34:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30c4bfb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f30c00d2ee0 con 0x7f30c00cf690 2024-04-25T00:34:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30c4bfb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f30c00b38d0 con 0x7f30c015e200 2024-04-25T00:34:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30bdd9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30c00cf690 0x7f30c00d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30bed9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f30c00d2a90 0x7f30c00d05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52472/0 (socket says 172.21.15.29:52472) 2024-04-25T00:34:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30bed9d640 1 -- 172.21.15.29:0/141275313 learned_addr learned my addr 172.21.15.29:0/141275313 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30be59c640 1 --2- 172.21.15.29:0/141275313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c015e200 0x7f30c00d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30bdd9b640 1 -- 172.21.15.29:0/141275313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c015e200 msgr2=0x7f30c00d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:01.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30bdd9b640 1 --2- 172.21.15.29:0/141275313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c015e200 0x7f30c00d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:01.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30bdd9b640 1 -- 172.21.15.29:0/141275313 >> v1:172.21.15.29:6789/0 conn(0x7f30c00d2a90 legacy=0x7f30c00d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:01.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30bdd9b640 1 -- 172.21.15.29:0/141275313 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c00cfef0 con 0x7f30c00cf690 2024-04-25T00:34:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30bdd9b640 1 --2- 172.21.15.29:0/141275313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30c00cf690 0x7f30c00d2550 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009d90 tx=0x7f30a801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b4f7220ac724f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30bd59a640 1 -- 172.21.15.29:0/141275313 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a80b2070 con 0x7f30c00cf690 2024-04-25T00:34:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30bd59a640 1 -- 172.21.15.29:0/141275313 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30a80a5a00 con 0x7f30c00cf690 2024-04-25T00:34:01.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.561+0000 7f30bd59a640 1 -- 172.21.15.29:0/141275313 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a80a5dd0 con 0x7f30c00cf690 2024-04-25T00:34:01.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 -- 172.21.15.29:0/141275313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30c00cf690 msgr2=0x7f30c00d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:01.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 --2- 172.21.15.29:0/141275313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30c00cf690 0x7f30c00d2550 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009d90 tx=0x7f30a801c940 comp rx=0 tx=0).stop 2024-04-25T00:34:01.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 -- 172.21.15.29:0/141275313 shutdown_connections 2024-04-25T00:34:01.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 --2- 172.21.15.29:0/141275313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30c00cf690 0x7f30c00d2550 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:01.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 --2- 172.21.15.29:0/141275313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c015e200 0x7f30c00d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:01.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 -- 172.21.15.29:0/141275313 >> 172.21.15.29:0/141275313 conn(0x7f30c009d280 msgr2=0x7f30c00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:01.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 -- 172.21.15.29:0/141275313 shutdown_connections 2024-04-25T00:34:01.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 -- 172.21.15.29:0/141275313 wait complete. 2024-04-25T00:34:01.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 Processor -- start 2024-04-25T00:34:01.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 -- start start 2024-04-25T00:34:01.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c00cf690 0x7f30c014b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:01.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c00d2a90 0x7f30c0155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:01.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30c015e200 0x7f30c0155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:01.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30c015e760 con 0x7f30c015e200 2024-04-25T00:34:01.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f30c015e5e0 con 0x7f30c00cf690 2024-04-25T00:34:01.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f30c016ae30 con 0x7f30c00d2a90 2024-04-25T00:34:01.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30bed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30c015e200 0x7f30c0155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:01.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30bed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30c015e200 0x7f30c0155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60290/0 (socket says 172.21.15.29:60290) 2024-04-25T00:34:01.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30bed9d640 1 -- 172.21.15.29:0/1282308169 learned_addr learned my addr 172.21.15.29:0/1282308169 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:01.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30be59c640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c00cf690 0x7f30c014b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:01.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30bdd9b640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c00d2a90 0x7f30c0155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:01.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30bed9d640 1 -- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c00d2a90 msgr2=0x7f30c0155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:01.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30bed9d640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c00d2a90 0x7f30c0155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:01.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30bed9d640 1 -- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c00cf690 msgr2=0x7f30c014b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:01.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30bed9d640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c00cf690 0x7f30c014b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:01.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30bed9d640 1 -- 172.21.15.29:0/1282308169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c0156c50 con 0x7f30c015e200 2024-04-25T00:34:01.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30bed9d640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30c015e200 0x7f30c0155bd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f30b800c3f0 tx=0x7f30b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:01.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30b6ffd640 1 -- 172.21.15.29:0/1282308169 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8017070 con 0x7f30c015e200 2024-04-25T00:34:01.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30b6ffd640 1 -- 172.21.15.29:0/1282308169 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b8005cf0 con 0x7f30c015e200 2024-04-25T00:34:01.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30b6ffd640 1 -- 172.21.15.29:0/1282308169 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b80072f0 con 0x7f30c015e200 2024-04-25T00:34:01.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.565+0000 7f30c4bfb640 1 -- 172.21.15.29:0/1282308169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c015f8a0 con 0x7f30c015e200 2024-04-25T00:34:01.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.569+0000 7f30c4bfb640 1 -- 172.21.15.29:0/1282308169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c0156970 con 0x7f30c015e200 2024-04-25T00:34:01.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.569+0000 7f30c4bfb640 1 -- 172.21.15.29:0/1282308169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c0069270 con 0x7f30c015e200 2024-04-25T00:34:01.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.569+0000 7f30b6ffd640 1 -- 172.21.15.29:0/1282308169 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f30b801a070 con 0x7f30c015e200 2024-04-25T00:34:01.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.569+0000 7f30b6ffd640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f30a0067b00 0x7f30a0069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:01.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.569+0000 7f30b6ffd640 1 -- 172.21.15.29:0/1282308169 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f30b8085460 con 0x7f30c015e200 2024-04-25T00:34:01.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.569+0000 7f30be59c640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f30a0067b00 0x7f30a0069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:01.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.569+0000 7f30be59c640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f30a0067b00 0x7f30a0069fc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f30c014db70 tx=0x7f30b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:01.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.573+0000 7f30b6ffd640 1 -- 172.21.15.29:0/1282308169 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f30b8055c40 con 0x7f30c015e200 2024-04-25T00:34:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.773+0000 7f30c4bfb640 1 -- 172.21.15.29:0/1282308169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30c0067900 con 0x7f30c015e200 2024-04-25T00:34:01.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.773+0000 7f30b6ffd640 1 -- 172.21.15.29:0/1282308169 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7f30b8055540 con 0x7f30c015e200 2024-04-25T00:34:01.775 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:01.775 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:01.775 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.777+0000 7f30c4bfb640 1 -- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f30a0067b00 msgr2=0x7f30a0069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.777+0000 7f30c4bfb640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f30a0067b00 0x7f30a0069fc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f30c014db70 tx=0x7f30b0008040 comp rx=0 tx=0).stop 2024-04-25T00:34:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.777+0000 7f30c4bfb640 1 -- 172.21.15.29:0/1282308169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30c015e200 msgr2=0x7f30c0155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.777+0000 7f30c4bfb640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30c015e200 0x7f30c0155bd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f30b800c3f0 tx=0x7f30b800c8b0 comp rx=0 tx=0).stop 2024-04-25T00:34:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.777+0000 7f30c4bfb640 1 -- 172.21.15.29:0/1282308169 shutdown_connections 2024-04-25T00:34:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.777+0000 7f30c4bfb640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30c015e200 0x7f30c0155bd0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.777+0000 7f30c4bfb640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30c00d2a90 0x7f30c0155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.777+0000 7f30c4bfb640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f30a0067b00 0x7f30a0069fc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.777+0000 7f30c4bfb640 1 --2- 172.21.15.29:0/1282308169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30c00cf690 0x7f30c014b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.777+0000 7f30c4bfb640 1 -- 172.21.15.29:0/1282308169 >> 172.21.15.29:0/1282308169 conn(0x7f30c009d280 msgr2=0x7f30c00b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.777+0000 7f30c4bfb640 1 -- 172.21.15.29:0/1282308169 shutdown_connections 2024-04-25T00:34:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:01.777+0000 7f30c4bfb640 1 -- 172.21.15.29:0/1282308169 wait complete. 2024-04-25T00:34:01.795 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:34:01.939 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":23.051688380000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.859534835469532,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:34:01.940 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.859534835469532 }, "state": "open", "uptime": 23.05168838 } ] 2024-04-25T00:34:01.940 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-25T00:34:06.940 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:07.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbc21a7640 1 Processor -- start 2024-04-25T00:34:07.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbc21a7640 1 -- start start 2024-04-25T00:34:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbc21a7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcbbc0b3140 0x7fcbbc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbc21a7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbbc0b4b10 0x7fcbbc0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbc21a7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbbc0b41e0 con 0x7fcbbc0b3a60 2024-04-25T00:34:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbc21a7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fcbbc06d960 con 0x7fcbbc0b3140 2024-04-25T00:34:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbc21a7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcbbc06dc40 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbbb7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcbbc0b3140 0x7fcbbc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbbb7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcbbc0b3140 0x7fcbbc06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60306/0 (socket says 172.21.15.29:60306) 2024-04-25T00:34:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbbb7fe640 1 -- 172.21.15.29:0/454904989 learned_addr learned my addr 172.21.15.29:0/454904989 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbbaffd640 1 --2- 172.21.15.29:0/454904989 >> v2:172.21.15.157:3300/0 conn(0x7fcbbc0b4b10 0x7fcbbc0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbbb7fe640 1 -- 172.21.15.29:0/454904989 >> v2:172.21.15.157:3300/0 conn(0x7fcbbc0b4b10 msgr2=0x7fcbbc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbbb7fe640 1 --2- 172.21.15.29:0/454904989 >> v2:172.21.15.157:3300/0 conn(0x7fcbbc0b4b10 0x7fcbbc0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbbb7fe640 1 -- 172.21.15.29:0/454904989 >> v1:172.21.15.157:6789/0 conn(0x7fcbbc0b3a60 legacy=0x7fcbbc15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:07.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.073+0000 7fcbbb7fe640 1 -- 172.21.15.29:0/454904989 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc06df20 con 0x7fcbbc0b3140 2024-04-25T00:34:07.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbbb7fe640 1 --2- 172.21.15.29:0/454904989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbbc0b3140 0x7fcbbc06e440 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009870 tx=0x7fcbac01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e059a253b7fd5ee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:07.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbba7fc640 1 -- 172.21.15.29:0/454904989 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0b0070 con 0x7fcbbc0b3140 2024-04-25T00:34:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbba7fc640 1 -- 172.21.15.29:0/454904989 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbac0ab600 con 0x7fcbbc0b3140 2024-04-25T00:34:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbba7fc640 1 -- 172.21.15.29:0/454904989 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0a4e60 con 0x7fcbbc0b3140 2024-04-25T00:34:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 -- 172.21.15.29:0/454904989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbbc0b3140 msgr2=0x7fcbbc06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 --2- 172.21.15.29:0/454904989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbbc0b3140 0x7fcbbc06e440 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009870 tx=0x7fcbac01c4e0 comp rx=0 tx=0).stop 2024-04-25T00:34:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 -- 172.21.15.29:0/454904989 shutdown_connections 2024-04-25T00:34:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 --2- 172.21.15.29:0/454904989 >> v2:172.21.15.157:3300/0 conn(0x7fcbbc0b4b10 0x7fcbbc0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 --2- 172.21.15.29:0/454904989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbbc0b3140 0x7fcbbc06e440 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 -- 172.21.15.29:0/454904989 >> 172.21.15.29:0/454904989 conn(0x7fcbbc0b1870 msgr2=0x7fcbbc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 -- 172.21.15.29:0/454904989 shutdown_connections 2024-04-25T00:34:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 -- 172.21.15.29:0/454904989 wait complete. 2024-04-25T00:34:07.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 Processor -- start 2024-04-25T00:34:07.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 -- start start 2024-04-25T00:34:07.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbbc0b3140 0x7fcbbc138410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:07.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbbc0b3a60 0x7fcbbc157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:07.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbbc0b4b10 0x7fcbbc15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:07.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc168b70 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc1689f0 con 0x7fcbbc0b3140 2024-04-25T00:34:07.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbc21a7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc168cf0 con 0x7fcbbc0b3a60 2024-04-25T00:34:07.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbbbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbbc0b4b10 0x7fcbbc15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:07.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbbbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbbc0b4b10 0x7fcbbc15f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60308/0 (socket says 172.21.15.29:60308) 2024-04-25T00:34:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbbbfff640 1 -- 172.21.15.29:0/580384090 learned_addr learned my addr 172.21.15.29:0/580384090 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbbb7fe640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbbc0b3140 0x7fcbbc138410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.077+0000 7fcbbaffd640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbbc0b3a60 0x7fcbbc157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcbbbfff640 1 -- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbbc0b3a60 msgr2=0x7fcbbc157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcbbbfff640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbbc0b3a60 0x7fcbbc157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcbbbfff640 1 -- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbbc0b3140 msgr2=0x7fcbbc138410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcbbbfff640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbbc0b3140 0x7fcbbc138410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcbbbfff640 1 -- 172.21.15.29:0/580384090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc158260 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcbbbfff640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbbc0b4b10 0x7fcbbc15f0c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400b970 tx=0x7fcbb400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcba3fff640 1 -- 172.21.15.29:0/580384090 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4019070 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcba3fff640 1 -- 172.21.15.29:0/580384090 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb400cb40 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcbc21a7640 1 -- 172.21.15.29:0/580384090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbbc158540 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcba3fff640 1 -- 172.21.15.29:0/580384090 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4014440 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcbc21a7640 1 -- 172.21.15.29:0/580384090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbbc158b00 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcbc21a7640 1 -- 172.21.15.29:0/580384090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbbc069270 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcba3fff640 1 -- 172.21.15.29:0/580384090 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7fcbb401c070 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcba3fff640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcba8067b00 0x7fcba8069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcba3fff640 1 -- 172.21.15.29:0/580384090 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fcbb4085c90 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcbbb7fe640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcba8067b00 0x7fcba8069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.081+0000 7fcbbb7fe640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcba8067b00 0x7fcba8069fc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0ad000 tx=0x7fcbac0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:07.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.085+0000 7fcba3fff640 1 -- 172.21.15.29:0/580384090 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcbb4056500 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.281+0000 7fcbc21a7640 1 -- 172.21.15.29:0/580384090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbbc1447c0 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.281+0000 7fcba3fff640 1 -- 172.21.15.29:0/580384090 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7fcbb4055e00 con 0x7fcbbc0b4b10 2024-04-25T00:34:07.283 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:07.283 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:07.283 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:07.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.285+0000 7fcbc21a7640 1 -- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcba8067b00 msgr2=0x7fcba8069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.285+0000 7fcbc21a7640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcba8067b00 0x7fcba8069fc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0ad000 tx=0x7fcbac0a6000 comp rx=0 tx=0).stop 2024-04-25T00:34:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.285+0000 7fcbc21a7640 1 -- 172.21.15.29:0/580384090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbbc0b4b10 msgr2=0x7fcbbc15f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.285+0000 7fcbc21a7640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbbc0b4b10 0x7fcbbc15f0c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400b970 tx=0x7fcbb400be30 comp rx=0 tx=0).stop 2024-04-25T00:34:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.285+0000 7fcbc21a7640 1 -- 172.21.15.29:0/580384090 shutdown_connections 2024-04-25T00:34:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.285+0000 7fcbc21a7640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbbc0b4b10 0x7fcbbc15f0c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.285+0000 7fcbc21a7640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbbc0b3a60 0x7fcbbc157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.285+0000 7fcbc21a7640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcba8067b00 0x7fcba8069fc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.285+0000 7fcbc21a7640 1 --2- 172.21.15.29:0/580384090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbbc0b3140 0x7fcbbc138410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.285+0000 7fcbc21a7640 1 -- 172.21.15.29:0/580384090 >> 172.21.15.29:0/580384090 conn(0x7fcbbc0b1870 msgr2=0x7fcbbc142960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.285+0000 7fcbc21a7640 1 -- 172.21.15.29:0/580384090 shutdown_connections 2024-04-25T00:34:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:07.285+0000 7fcbc21a7640 1 -- 172.21.15.29:0/580384090 wait complete. 2024-04-25T00:34:07.301 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:34:07.437 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":28.551622944999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.596138419229881,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:34:07.438 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.59613841922988 }, "state": "open", "uptime": 28.551622945 } ] 2024-04-25T00:34:07.438 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-25T00:34:10.152 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:34:10.155 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:34:12.439 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:12.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.569+0000 7f6f6a2de640 1 Processor -- start 2024-04-25T00:34:12.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f6a2de640 1 -- start start 2024-04-25T00:34:12.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f6a2de640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b3140 0x7f6f6406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:12.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f6a2de640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f640b41e0 con 0x7f6f640b3c30 2024-04-25T00:34:12.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f6a2de640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f6406d960 con 0x7f6f640b4b10 2024-04-25T00:34:12.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f6a2de640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6406dc40 con 0x7f6f640b3140 2024-04-25T00:34:12.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f6887b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6f640b3c30 0x7f6f640ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35474/0 (socket says 172.21.15.29:35474) 2024-04-25T00:34:12.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f6887b640 1 -- 172.21.15.29:0/1845372484 learned_addr learned my addr 172.21.15.29:0/1845372484 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:12.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f63fff640 1 --2- 172.21.15.29:0/1845372484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b3140 0x7f6f6406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4040317513 0 0) 0x7f6f640b41e0 con 0x7f6f640b3c30 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f50003530 con 0x7f6f640b3c30 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2010660656 0 0) 0x7f6f50003530 con 0x7f6f640b3c30 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b3140 msgr2=0x7f6f6406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 --2- 172.21.15.29:0/1845372484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b3140 0x7f6f6406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 >> v1:172.21.15.157:6789/0 conn(0x7f6f640b4b10 legacy=0x7f6f640b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6406df20 con 0x7f6f640b3c30 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f6f54001680 con 0x7f6f640b3c30 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 >> v1:172.21.15.29:6789/0 conn(0x7f6f640b3c30 legacy=0x7f6f640ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 --2- 172.21.15.29:0/1845372484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f500039f0 0x7f6f50003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 --2- 172.21.15.29:0/1845372484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b3140 0x7f6f50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 --2- 172.21.15.29:0/1845372484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f50004d50 0x7f6f500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6406dc40 con 0x7f6f500039f0 2024-04-25T00:34:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5000ba80 con 0x7f6f50004d50 2024-04-25T00:34:12.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f5000bc80 con 0x7f6f640b3140 2024-04-25T00:34:12.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f63fff640 1 --2- 172.21.15.29:0/1845372484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f500039f0 0x7f6f50003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:12.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f637fe640 1 --2- 172.21.15.29:0/1845372484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b3140 0x7f6f50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:12.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f6887b640 1 --2- 172.21.15.29:0/1845372484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f50004d50 0x7f6f500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:12.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f63fff640 1 -- 172.21.15.29:0/1845372484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b3140 msgr2=0x7f6f50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:12.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f63fff640 1 --2- 172.21.15.29:0/1845372484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b3140 0x7f6f50004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:12.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f63fff640 1 -- 172.21.15.29:0/1845372484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f50004d50 msgr2=0x7f6f500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:12.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f63fff640 1 --2- 172.21.15.29:0/1845372484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f50004d50 0x7f6f500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:12.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f63fff640 1 -- 172.21.15.29:0/1845372484 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f50003530 con 0x7f6f500039f0 2024-04-25T00:34:12.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.573+0000 7f6f6887b640 1 --2- 172.21.15.29:0/1845372484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f50004d50 0x7f6f500051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:12.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f63fff640 1 --2- 172.21.15.29:0/1845372484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f500039f0 0x7f6f50003e10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009f90 tx=0x7f6f5c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667f1678a93dd9b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:12.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0b2070 con 0x7f6f500039f0 2024-04-25T00:34:12.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c0a5a30 con 0x7f6f500039f0 2024-04-25T00:34:12.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f62ffd640 1 -- 172.21.15.29:0/1845372484 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c009c10 con 0x7f6f500039f0 2024-04-25T00:34:12.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 -- 172.21.15.29:0/1845372484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f500039f0 msgr2=0x7f6f50003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:12.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 --2- 172.21.15.29:0/1845372484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f500039f0 0x7f6f50003e10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009f90 tx=0x7f6f5c01cc40 comp rx=0 tx=0).stop 2024-04-25T00:34:12.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6887b640 1 -- 172.21.15.29:0/1845372484 reap_dead start 2024-04-25T00:34:12.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 -- 172.21.15.29:0/1845372484 shutdown_connections 2024-04-25T00:34:12.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 -- 172.21.15.29:0/1845372484 >> 172.21.15.29:0/1845372484 conn(0x7f6f640b1870 msgr2=0x7f6f640b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:12.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 -- 172.21.15.29:0/1845372484 shutdown_connections 2024-04-25T00:34:12.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 -- 172.21.15.29:0/1845372484 wait complete. 2024-04-25T00:34:12.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 Processor -- start 2024-04-25T00:34:12.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 -- start start 2024-04-25T00:34:12.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f500039f0 0x7f6f6413a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:12.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f641fb6e0 0x7f6f64156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:12.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f64156cc0 0x7f6f6413d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:12.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6f64167d80 con 0x7f6f500039f0 2024-04-25T00:34:12.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f64167c00 con 0x7f6f641fb6e0 2024-04-25T00:34:12.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f6a2de640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f64167f00 con 0x7f6f64156cc0 2024-04-25T00:34:12.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f63fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f500039f0 0x7f6f6413a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:12.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f63fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f500039f0 0x7f6f6413a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50222/0 (socket says 172.21.15.29:50222) 2024-04-25T00:34:12.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.577+0000 7f6f63fff640 1 -- 172.21.15.29:0/66594021 learned_addr learned my addr 172.21.15.29:0/66594021 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:12.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f6887b640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f64156cc0 0x7f6f6413d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:12.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f637fe640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f641fb6e0 0x7f6f64156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f63fff640 1 -- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f64156cc0 msgr2=0x7f6f6413d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f63fff640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f64156cc0 0x7f6f6413d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f63fff640 1 -- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f641fb6e0 msgr2=0x7f6f64156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f63fff640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f641fb6e0 0x7f6f64156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f63fff640 1 -- 172.21.15.29:0/66594021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f64072900 con 0x7f6f500039f0 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f6887b640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f64156cc0 0x7f6f6413d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f637fe640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f641fb6e0 0x7f6f64156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f63fff640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f500039f0 0x7f6f6413a8b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0b0040 tx=0x7f6f5c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f60ff9640 1 -- 172.21.15.29:0/66594021 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0bc070 con 0x7f6f500039f0 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f60ff9640 1 -- 172.21.15.29:0/66594021 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c0ad640 con 0x7f6f500039f0 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f60ff9640 1 -- 172.21.15.29:0/66594021 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0b2040 con 0x7f6f500039f0 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f6a2de640 1 -- 172.21.15.29:0/66594021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6413af40 con 0x7f6f500039f0 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f6a2de640 1 -- 172.21.15.29:0/66594021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6413b500 con 0x7f6f500039f0 2024-04-25T00:34:12.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f6a2de640 1 -- 172.21.15.29:0/66594021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f64069270 con 0x7f6f500039f0 2024-04-25T00:34:12.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f60ff9640 1 -- 172.21.15.29:0/66594021 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f6f5c0bf070 con 0x7f6f500039f0 2024-04-25T00:34:12.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f60ff9640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f44067b30 0x7f6f44069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:12.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f60ff9640 1 -- 172.21.15.29:0/66594021 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f6f5c11fc50 con 0x7f6f500039f0 2024-04-25T00:34:12.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f637fe640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f44067b30 0x7f6f44069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:12.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.581+0000 7f6f637fe640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f44067b30 0x7f6f44069ff0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c007fd0 tx=0x7f6f4c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:12.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.585+0000 7f6f60ff9640 1 -- 172.21.15.29:0/66594021 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6f5c125020 con 0x7f6f500039f0 2024-04-25T00:34:12.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.781+0000 7f6f6a2de640 1 -- 172.21.15.29:0/66594021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f64140fc0 con 0x7f6f500039f0 2024-04-25T00:34:12.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.785+0000 7f6f60ff9640 1 -- 172.21.15.29:0/66594021 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7f6f5c016950 con 0x7f6f500039f0 2024-04-25T00:34:12.786 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:12.786 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:12.786 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:12.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.785+0000 7f6f6a2de640 1 -- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f44067b30 msgr2=0x7f6f44069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:12.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.785+0000 7f6f6a2de640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f44067b30 0x7f6f44069ff0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c007fd0 tx=0x7f6f4c009040 comp rx=0 tx=0).stop 2024-04-25T00:34:12.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.785+0000 7f6f6a2de640 1 -- 172.21.15.29:0/66594021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f500039f0 msgr2=0x7f6f6413a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:12.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.785+0000 7f6f6a2de640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f500039f0 0x7f6f6413a8b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0b0040 tx=0x7f6f5c009750 comp rx=0 tx=0).stop 2024-04-25T00:34:12.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.785+0000 7f6f6a2de640 1 -- 172.21.15.29:0/66594021 shutdown_connections 2024-04-25T00:34:12.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.785+0000 7f6f6a2de640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f64156cc0 0x7f6f6413d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:12.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.785+0000 7f6f6a2de640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f44067b30 0x7f6f44069ff0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:12.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.785+0000 7f6f6a2de640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f641fb6e0 0x7f6f64156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:12.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.785+0000 7f6f6a2de640 1 --2- 172.21.15.29:0/66594021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f500039f0 0x7f6f6413a8b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:12.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.785+0000 7f6f6a2de640 1 -- 172.21.15.29:0/66594021 >> 172.21.15.29:0/66594021 conn(0x7f6f640b1870 msgr2=0x7f6f6406cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:12.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.785+0000 7f6f6a2de640 1 -- 172.21.15.29:0/66594021 shutdown_connections 2024-04-25T00:34:12.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:12.785+0000 7f6f6a2de640 1 -- 172.21.15.29:0/66594021 wait complete. 2024-04-25T00:34:12.804 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:34:12.947 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":34.059562163999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.33569184275461,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:34:12.948 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.33569184275461 }, "state": "open", "uptime": 34.059562164 } ] 2024-04-25T00:34:12.948 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-25T00:34:17.949 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:18.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33c502d640 1 Processor -- start 2024-04-25T00:34:18.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33c502d640 1 -- start start 2024-04-25T00:34:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33c502d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33c015bdb0 0x7f33c00cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33c502d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33c015c190 0x7f33c00ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33c502d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33c00d02d0 0x7f33c00d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33c502d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f33c00d0c50 con 0x7f33c015bdb0 2024-04-25T00:34:18.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33c502d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f33c00cd3e0 con 0x7f33c00d02d0 2024-04-25T00:34:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33c502d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f33c00cd6c0 con 0x7f33c015c190 2024-04-25T00:34:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33bed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33c015bdb0 0x7f33c00cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33bed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33c015bdb0 0x7f33c00cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55932/0 (socket says 172.21.15.29:55932) 2024-04-25T00:34:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33bed9d640 1 -- 172.21.15.29:0/682012826 learned_addr learned my addr 172.21.15.29:0/682012826 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33bf59e640 1 --2- 172.21.15.29:0/682012826 >> v2:172.21.15.157:3300/0 conn(0x7f33c00d02d0 0x7f33c00d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33be59c640 1 --2- 172.21.15.29:0/682012826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33c015c190 0x7f33c00ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33bed9d640 1 -- 172.21.15.29:0/682012826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33c015c190 msgr2=0x7f33c00ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33bed9d640 1 --2- 172.21.15.29:0/682012826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33c015c190 0x7f33c00ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33bed9d640 1 -- 172.21.15.29:0/682012826 >> v2:172.21.15.157:3300/0 conn(0x7f33c00d02d0 msgr2=0x7f33c00d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33bed9d640 1 --2- 172.21.15.29:0/682012826 >> v2:172.21.15.157:3300/0 conn(0x7f33c00d02d0 0x7f33c00d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:18.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.085+0000 7f33bed9d640 1 -- 172.21.15.29:0/682012826 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c00cd9a0 con 0x7f33c015bdb0 2024-04-25T00:34:18.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33bed9d640 1 --2- 172.21.15.29:0/682012826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c015bdb0 0x7f33c00cc820 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f33c00c57e0 tx=0x7f33b001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff7d5292fbc262 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:18.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33bdd9b640 1 -- 172.21.15.29:0/682012826 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b00ab620 con 0x7f33c015bdb0 2024-04-25T00:34:18.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33bdd9b640 1 -- 172.21.15.29:0/682012826 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b00b0070 con 0x7f33c015bdb0 2024-04-25T00:34:18.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33bdd9b640 1 -- 172.21.15.29:0/682012826 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b00ab9e0 con 0x7f33c015bdb0 2024-04-25T00:34:18.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 -- 172.21.15.29:0/682012826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c015bdb0 msgr2=0x7f33c00cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:18.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 --2- 172.21.15.29:0/682012826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c015bdb0 0x7f33c00cc820 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f33c00c57e0 tx=0x7f33b001c460 comp rx=0 tx=0).stop 2024-04-25T00:34:18.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 -- 172.21.15.29:0/682012826 shutdown_connections 2024-04-25T00:34:18.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 --2- 172.21.15.29:0/682012826 >> v2:172.21.15.157:3300/0 conn(0x7f33c00d02d0 0x7f33c00d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:18.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 --2- 172.21.15.29:0/682012826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33c015c190 0x7f33c00ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:18.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 --2- 172.21.15.29:0/682012826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c015bdb0 0x7f33c00cc820 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:18.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 -- 172.21.15.29:0/682012826 >> 172.21.15.29:0/682012826 conn(0x7f33c009aa30 msgr2=0x7f33c009ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:18.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 -- 172.21.15.29:0/682012826 shutdown_connections 2024-04-25T00:34:18.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 -- 172.21.15.29:0/682012826 wait complete. 2024-04-25T00:34:18.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 Processor -- start 2024-04-25T00:34:18.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 -- start start 2024-04-25T00:34:18.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c00d02d0 0x7f33c014dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:18.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33c015bdb0 0x7f33c00c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:18.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33c015c5f0 0x7f33c00ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:18.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f33c00d2350 con 0x7f33c00d02d0 2024-04-25T00:34:18.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33c00d21d0 con 0x7f33c015c5f0 2024-04-25T00:34:18.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33c502d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f33c00d24d0 con 0x7f33c015bdb0 2024-04-25T00:34:18.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33bed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c00d02d0 0x7f33c014dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:18.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33bed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c00d02d0 0x7f33c014dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55938/0 (socket says 172.21.15.29:55938) 2024-04-25T00:34:18.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33bed9d640 1 -- 172.21.15.29:0/2905220091 learned_addr learned my addr 172.21.15.29:0/2905220091 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:18.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33bf59e640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33c015c5f0 0x7f33c00ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:18.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33be59c640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33c015bdb0 0x7f33c00c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:18.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33bed9d640 1 -- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33c015bdb0 msgr2=0x7f33c00c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:18.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33bed9d640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33c015bdb0 0x7f33c00c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:18.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33bed9d640 1 -- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33c015c5f0 msgr2=0x7f33c00ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:18.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33bed9d640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33c015c5f0 0x7f33c00ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:18.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.089+0000 7f33bed9d640 1 -- 172.21.15.29:0/2905220091 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c014e250 con 0x7f33c00d02d0 2024-04-25T00:34:18.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.093+0000 7f33bed9d640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c00d02d0 0x7f33c014dd10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f33b00ad040 tx=0x7f33b001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:18.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.093+0000 7f33a77fe640 1 -- 172.21.15.29:0/2905220091 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b00abd90 con 0x7f33c00d02d0 2024-04-25T00:34:18.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.093+0000 7f33a77fe640 1 -- 172.21.15.29:0/2905220091 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b00be3f0 con 0x7f33c00d02d0 2024-04-25T00:34:18.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.093+0000 7f33c502d640 1 -- 172.21.15.29:0/2905220091 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c015f300 con 0x7f33c00d02d0 2024-04-25T00:34:18.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.093+0000 7f33a77fe640 1 -- 172.21.15.29:0/2905220091 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b00be7c0 con 0x7f33c00d02d0 2024-04-25T00:34:18.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.093+0000 7f33c502d640 1 -- 172.21.15.29:0/2905220091 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c00b4dd0 con 0x7f33c00d02d0 2024-04-25T00:34:18.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.093+0000 7f33c502d640 1 -- 172.21.15.29:0/2905220091 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c0069270 con 0x7f33c00d02d0 2024-04-25T00:34:18.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.093+0000 7f33a77fe640 1 -- 172.21.15.29:0/2905220091 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f33b00b43f0 con 0x7f33c00d02d0 2024-04-25T00:34:18.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.093+0000 7f33a77fe640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f33ac067b50 0x7f33ac06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:18.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.093+0000 7f33a77fe640 1 -- 172.21.15.29:0/2905220091 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f33b01203d0 con 0x7f33c00d02d0 2024-04-25T00:34:18.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.093+0000 7f33be59c640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f33ac067b50 0x7f33ac06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:18.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.093+0000 7f33be59c640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f33ac067b50 0x7f33ac06a010 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f33c006dc00 tx=0x7f33b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:18.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.101+0000 7f33a77fe640 1 -- 172.21.15.29:0/2905220091 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f33b00c4060 con 0x7f33c00d02d0 2024-04-25T00:34:18.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.301+0000 7f33c502d640 1 -- 172.21.15.29:0/2905220091 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33c00cef40 con 0x7f33c00d02d0 2024-04-25T00:34:18.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.301+0000 7f33a77fe640 1 -- 172.21.15.29:0/2905220091 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7f33b0016110 con 0x7f33c00d02d0 2024-04-25T00:34:18.303 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:18.303 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:18.303 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:18.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.301+0000 7f33c502d640 1 -- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f33ac067b50 msgr2=0x7f33ac06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:18.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.301+0000 7f33c502d640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f33ac067b50 0x7f33ac06a010 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f33c006dc00 tx=0x7f33b4008040 comp rx=0 tx=0).stop 2024-04-25T00:34:18.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.301+0000 7f33c502d640 1 -- 172.21.15.29:0/2905220091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c00d02d0 msgr2=0x7f33c014dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:18.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.301+0000 7f33c502d640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c00d02d0 0x7f33c014dd10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f33b00ad040 tx=0x7f33b001aca0 comp rx=0 tx=0).stop 2024-04-25T00:34:18.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.305+0000 7f33c502d640 1 -- 172.21.15.29:0/2905220091 shutdown_connections 2024-04-25T00:34:18.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.305+0000 7f33c502d640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f33ac067b50 0x7f33ac06a010 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:18.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.305+0000 7f33c502d640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33c015c5f0 0x7f33c00ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:18.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.305+0000 7f33c502d640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33c015bdb0 0x7f33c00c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:18.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.305+0000 7f33c502d640 1 --2- 172.21.15.29:0/2905220091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c00d02d0 0x7f33c014dd10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:18.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.305+0000 7f33c502d640 1 -- 172.21.15.29:0/2905220091 >> 172.21.15.29:0/2905220091 conn(0x7f33c009aa30 msgr2=0x7f33c0070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:18.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.305+0000 7f33c502d640 1 -- 172.21.15.29:0/2905220091 shutdown_connections 2024-04-25T00:34:18.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:18.305+0000 7f33c502d640 1 -- 172.21.15.29:0/2905220091 wait complete. 2024-04-25T00:34:18.322 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:34:18.452 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":39.567503317000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.078538629817512,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:34:18.453 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.078538629817512 }, "state": "open", "uptime": 39.567503317 } ] 2024-04-25T00:34:18.453 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-25T00:34:23.454 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:23.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc39deb3640 1 Processor -- start 2024-04-25T00:34:23.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc39deb3640 1 -- start start 2024-04-25T00:34:23.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc39deb3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc3980b3140 0x7fc39806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:23.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc39deb3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980b4b10 0x7fc3980b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:23.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc39deb3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3980b41e0 con 0x7fc3980b3a60 2024-04-25T00:34:23.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc39deb3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc39806d960 con 0x7fc3980b3140 2024-04-25T00:34:23.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc39deb3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc39806dc40 con 0x7fc3980b4b10 2024-04-25T00:34:23.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc3977fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc3980b3140 0x7fc39806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:23.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc3977fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc3980b3140 0x7fc39806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55940/0 (socket says 172.21.15.29:55940) 2024-04-25T00:34:23.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc3977fe640 1 -- 172.21.15.29:0/1750717273 learned_addr learned my addr 172.21.15.29:0/1750717273 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:23.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc396ffd640 1 --2- 172.21.15.29:0/1750717273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980b4b10 0x7fc3980b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:23.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc3977fe640 1 -- 172.21.15.29:0/1750717273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980b4b10 msgr2=0x7fc3980b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:23.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc3977fe640 1 --2- 172.21.15.29:0/1750717273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980b4b10 0x7fc3980b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:23.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc3977fe640 1 -- 172.21.15.29:0/1750717273 >> v1:172.21.15.157:6789/0 conn(0x7fc3980b3a60 legacy=0x7fc39815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:23.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.589+0000 7fc3977fe640 1 -- 172.21.15.29:0/1750717273 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39806df20 con 0x7fc3980b3140 2024-04-25T00:34:23.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.593+0000 7fc3977fe640 1 --2- 172.21.15.29:0/1750717273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3980b3140 0x7fc39806e440 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc388009870 tx=0x7fc38801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cc8b89788910da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:23.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.593+0000 7fc3967fc640 1 -- 172.21.15.29:0/1750717273 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3880b0070 con 0x7fc3980b3140 2024-04-25T00:34:23.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.593+0000 7fc3967fc640 1 -- 172.21.15.29:0/1750717273 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3880ab600 con 0x7fc3980b3140 2024-04-25T00:34:23.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.593+0000 7fc3967fc640 1 -- 172.21.15.29:0/1750717273 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3880ab9d0 con 0x7fc3980b3140 2024-04-25T00:34:23.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.593+0000 7fc39deb3640 1 -- 172.21.15.29:0/1750717273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3980b3140 msgr2=0x7fc39806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:23.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.593+0000 7fc39deb3640 1 --2- 172.21.15.29:0/1750717273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3980b3140 0x7fc39806e440 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc388009870 tx=0x7fc38801c560 comp rx=0 tx=0).stop 2024-04-25T00:34:23.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.593+0000 7fc39deb3640 1 -- 172.21.15.29:0/1750717273 shutdown_connections 2024-04-25T00:34:23.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.593+0000 7fc39deb3640 1 --2- 172.21.15.29:0/1750717273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980b4b10 0x7fc3980b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:23.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.593+0000 7fc39deb3640 1 --2- 172.21.15.29:0/1750717273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3980b3140 0x7fc39806e440 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:23.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.593+0000 7fc39deb3640 1 -- 172.21.15.29:0/1750717273 >> 172.21.15.29:0/1750717273 conn(0x7fc3980b1870 msgr2=0x7fc3980b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:23.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.593+0000 7fc39deb3640 1 -- 172.21.15.29:0/1750717273 shutdown_connections 2024-04-25T00:34:23.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.593+0000 7fc39deb3640 1 -- 172.21.15.29:0/1750717273 wait complete. 2024-04-25T00:34:23.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc39deb3640 1 Processor -- start 2024-04-25T00:34:23.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc39deb3640 1 -- start start 2024-04-25T00:34:23.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc39deb3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980b3140 0x7fc39815d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:23.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc39deb3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3980b3a60 0x7fc3981f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:23.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc39deb3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3980b4b10 0x7fc3981f8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:23.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc39deb3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc398168b70 con 0x7fc3980b4b10 2024-04-25T00:34:23.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc39deb3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3981689f0 con 0x7fc3980b3a60 2024-04-25T00:34:23.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc39deb3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc398168cf0 con 0x7fc3980b3140 2024-04-25T00:34:23.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc397fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3980b4b10 0x7fc3981f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:23.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc397fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3980b4b10 0x7fc3981f8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55950/0 (socket says 172.21.15.29:55950) 2024-04-25T00:34:23.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc397fff640 1 -- 172.21.15.29:0/3872976277 learned_addr learned my addr 172.21.15.29:0/3872976277 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:23.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc3977fe640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980b3140 0x7fc39815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:23.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc396ffd640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3980b3a60 0x7fc3981f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:23.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc397fff640 1 -- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980b3140 msgr2=0x7fc39815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:23.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc397fff640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980b3140 0x7fc39815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:23.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc397fff640 1 -- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3980b3a60 msgr2=0x7fc3981f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:23.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc397fff640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3980b3a60 0x7fc3981f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:23.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc397fff640 1 -- 172.21.15.29:0/3872976277 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39813dfd0 con 0x7fc3980b4b10 2024-04-25T00:34:23.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc396ffd640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3980b3a60 0x7fc3981f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:34:23.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc397fff640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3980b4b10 0x7fc3981f8780 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc39000c770 tx=0x7fc39000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:23.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc37ffff640 1 -- 172.21.15.29:0/3872976277 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390003b80 con 0x7fc3980b4b10 2024-04-25T00:34:23.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc37ffff640 1 -- 172.21.15.29:0/3872976277 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc390003d20 con 0x7fc3980b4b10 2024-04-25T00:34:23.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc37ffff640 1 -- 172.21.15.29:0/3872976277 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39000aa90 con 0x7fc3980b4b10 2024-04-25T00:34:23.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc39deb3640 1 -- 172.21.15.29:0/3872976277 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39815eca0 con 0x7fc3980b4b10 2024-04-25T00:34:23.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc39deb3640 1 -- 172.21.15.29:0/3872976277 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc398159940 con 0x7fc3980b4b10 2024-04-25T00:34:23.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc39deb3640 1 -- 172.21.15.29:0/3872976277 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc398069270 con 0x7fc3980b4b10 2024-04-25T00:34:23.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.597+0000 7fc37ffff640 1 -- 172.21.15.29:0/3872976277 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7fc39000ac30 con 0x7fc3980b4b10 2024-04-25T00:34:23.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.601+0000 7fc37ffff640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc384067b00 0x7fc384069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:23.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.601+0000 7fc37ffff640 1 -- 172.21.15.29:0/3872976277 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fc390086110 con 0x7fc3980b4b10 2024-04-25T00:34:23.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.601+0000 7fc3977fe640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc384067b00 0x7fc384069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:23.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.601+0000 7fc3977fe640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc384067b00 0x7fc384069fc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc3880ad000 tx=0x7fc3880a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:23.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.605+0000 7fc37ffff640 1 -- 172.21.15.29:0/3872976277 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc390056820 con 0x7fc3980b4b10 2024-04-25T00:34:23.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc39deb3640 1 -- 172.21.15.29:0/3872976277 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc398159b10 con 0x7fc3980b4b10 2024-04-25T00:34:23.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc37ffff640 1 -- 172.21.15.29:0/3872976277 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7fc390056120 con 0x7fc3980b4b10 2024-04-25T00:34:23.806 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:23.806 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:23.807 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:23.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc39deb3640 1 -- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc384067b00 msgr2=0x7fc384069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:23.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc39deb3640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc384067b00 0x7fc384069fc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc3880ad000 tx=0x7fc3880a6000 comp rx=0 tx=0).stop 2024-04-25T00:34:23.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc39deb3640 1 -- 172.21.15.29:0/3872976277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3980b4b10 msgr2=0x7fc3981f8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:23.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc39deb3640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3980b4b10 0x7fc3981f8780 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc39000c770 tx=0x7fc39000cc30 comp rx=0 tx=0).stop 2024-04-25T00:34:23.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc39deb3640 1 -- 172.21.15.29:0/3872976277 shutdown_connections 2024-04-25T00:34:23.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc39deb3640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3980b4b10 0x7fc3981f8780 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:23.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc39deb3640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc384067b00 0x7fc384069fc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:23.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc39deb3640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3980b3a60 0x7fc3981f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:23.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc39deb3640 1 --2- 172.21.15.29:0/3872976277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980b3140 0x7fc39815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:23.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc39deb3640 1 -- 172.21.15.29:0/3872976277 >> 172.21.15.29:0/3872976277 conn(0x7fc3980b1870 msgr2=0x7fc3980ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:23.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc39deb3640 1 -- 172.21.15.29:0/3872976277 shutdown_connections 2024-04-25T00:34:23.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:23.805+0000 7fc39deb3640 1 -- 172.21.15.29:0/3872976277 wait complete. 2024-04-25T00:34:23.825 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:34:23.962 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":45.075444654999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.824637216873914,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:34:23.963 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.824637216873914 }, "state": "open", "uptime": 45.075444655 } ] 2024-04-25T00:34:23.963 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-25T00:34:28.964 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:29.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe146502640 1 Processor -- start 2024-04-25T00:34:29.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe146502640 1 -- start start 2024-04-25T00:34:29.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe146502640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe1400b3140 0x7fe14006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe146502640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1400b3c30 0x7fe1400ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe146502640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1400b41e0 con 0x7fe1400b4b10 2024-04-25T00:34:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe146502640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe14006d960 con 0x7fe1400b3140 2024-04-25T00:34:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe146502640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe14006dc40 con 0x7fe1400b3c30 2024-04-25T00:34:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe13ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe1400b3140 0x7fe14006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe13f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe1400b4b10 0x7fe1400b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42980/0 (socket says 172.21.15.29:42980) 2024-04-25T00:34:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe13f7fe640 1 -- 172.21.15.29:0/3647858654 learned_addr learned my addr 172.21.15.29:0/3647858654 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:29.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe144a9f640 1 --2- 172.21.15.29:0/3647858654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1400b3c30 0x7fe1400ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:29.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe13effd640 1 -- 172.21.15.29:0/3647858654 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 833066984 0 0) 0x7fe1400b41e0 con 0x7fe1400b4b10 2024-04-25T00:34:29.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe13effd640 1 -- 172.21.15.29:0/3647858654 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe12c003530 con 0x7fe1400b4b10 2024-04-25T00:34:29.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe13ffff640 1 -- 172.21.15.29:0/3647858654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1400b3c30 msgr2=0x7fe1400ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:29.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe13ffff640 1 --2- 172.21.15.29:0/3647858654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1400b3c30 0x7fe1400ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:29.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe13ffff640 1 -- 172.21.15.29:0/3647858654 >> v1:172.21.15.29:6789/0 conn(0x7fe1400b4b10 legacy=0x7fe1400b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:29.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.101+0000 7fe13ffff640 1 -- 172.21.15.29:0/3647858654 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14006df20 con 0x7fe1400b3140 2024-04-25T00:34:29.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe13ffff640 1 --2- 172.21.15.29:0/3647858654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1400b3140 0x7fe14006e440 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe1300097f0 tx=0x7fe13001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe7e85dfd0377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:29.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe13effd640 1 -- 172.21.15.29:0/3647858654 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1300b0070 con 0x7fe1400b3140 2024-04-25T00:34:29.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe13effd640 1 -- 172.21.15.29:0/3647858654 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1300ab600 con 0x7fe1400b3140 2024-04-25T00:34:29.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe13effd640 1 -- 172.21.15.29:0/3647858654 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1300ab9b0 con 0x7fe1400b3140 2024-04-25T00:34:29.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 -- 172.21.15.29:0/3647858654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1400b3140 msgr2=0x7fe14006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:29.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 --2- 172.21.15.29:0/3647858654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1400b3140 0x7fe14006e440 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe1300097f0 tx=0x7fe13001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:34:29.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 -- 172.21.15.29:0/3647858654 shutdown_connections 2024-04-25T00:34:29.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 --2- 172.21.15.29:0/3647858654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1400b3c30 0x7fe1400ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:29.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 --2- 172.21.15.29:0/3647858654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1400b3140 0x7fe14006e440 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:29.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 -- 172.21.15.29:0/3647858654 >> 172.21.15.29:0/3647858654 conn(0x7fe1400b1870 msgr2=0x7fe1400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:29.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 -- 172.21.15.29:0/3647858654 shutdown_connections 2024-04-25T00:34:29.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 -- 172.21.15.29:0/3647858654 wait complete. 2024-04-25T00:34:29.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 Processor -- start 2024-04-25T00:34:29.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 -- start start 2024-04-25T00:34:29.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1400b3140 0x7fe14013a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:29.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1400b3c30 0x7fe140071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:29.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1400b4b10 0x7fe140141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:29.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe140168d80 con 0x7fe1400b4b10 2024-04-25T00:34:29.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe140168c00 con 0x7fe1400b3c30 2024-04-25T00:34:29.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe140168f00 con 0x7fe1400b3140 2024-04-25T00:34:29.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe144a9f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1400b4b10 0x7fe140141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:29.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe144a9f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1400b4b10 0x7fe140141830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55678/0 (socket says 172.21.15.29:55678) 2024-04-25T00:34:29.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe144a9f640 1 -- 172.21.15.29:0/2193776980 learned_addr learned my addr 172.21.15.29:0/2193776980 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:29.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe13ffff640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1400b3140 0x7fe14013a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:29.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe13f7fe640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1400b3c30 0x7fe140071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:29.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe144a9f640 1 -- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1400b3140 msgr2=0x7fe14013a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:29.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe144a9f640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1400b3140 0x7fe14013a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:29.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe144a9f640 1 -- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1400b3c30 msgr2=0x7fe140071ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe144a9f640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1400b3c30 0x7fe140071ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe144a9f640 1 -- 172.21.15.29:0/2193776980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe140070d80 con 0x7fe1400b4b10 2024-04-25T00:34:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe13f7fe640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1400b3c30 0x7fe140071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe144a9f640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1400b4b10 0x7fe140141830 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe13800d560 tx=0x7fe13800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe13cff9640 1 -- 172.21.15.29:0/2193776980 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1380125f0 con 0x7fe1400b4b10 2024-04-25T00:34:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe13cff9640 1 -- 172.21.15.29:0/2193776980 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe138019070 con 0x7fe1400b4b10 2024-04-25T00:34:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.105+0000 7fe146502640 1 -- 172.21.15.29:0/2193776980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe140070ff0 con 0x7fe1400b4b10 2024-04-25T00:34:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.109+0000 7fe13cff9640 1 -- 172.21.15.29:0/2193776980 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1380129f0 con 0x7fe1400b4b10 2024-04-25T00:34:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.109+0000 7fe146502640 1 -- 172.21.15.29:0/2193776980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14013ab10 con 0x7fe1400b4b10 2024-04-25T00:34:29.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.109+0000 7fe146502640 1 -- 172.21.15.29:0/2193776980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1400711c0 con 0x7fe1400b4b10 2024-04-25T00:34:29.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.109+0000 7fe13cff9640 1 -- 172.21.15.29:0/2193776980 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7fe138012b90 con 0x7fe1400b4b10 2024-04-25T00:34:29.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.109+0000 7fe13cff9640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe120067b00 0x7fe120069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:29.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.109+0000 7fe13cff9640 1 -- 172.21.15.29:0/2193776980 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fe1380864f0 con 0x7fe1400b4b10 2024-04-25T00:34:29.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.109+0000 7fe13ffff640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe120067b00 0x7fe120069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:29.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.109+0000 7fe13ffff640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe120067b00 0x7fe120069fc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe1300ad000 tx=0x7fe130008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:29.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.113+0000 7fe13cff9640 1 -- 172.21.15.29:0/2193776980 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe138056c90 con 0x7fe1400b4b10 2024-04-25T00:34:29.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.309+0000 7fe146502640 1 -- 172.21.15.29:0/2193776980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe140069270 con 0x7fe1400b4b10 2024-04-25T00:34:29.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.313+0000 7fe13cff9640 1 -- 172.21.15.29:0/2193776980 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7fe138056590 con 0x7fe1400b4b10 2024-04-25T00:34:29.314 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:29.314 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:29.314 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:29.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.313+0000 7fe146502640 1 -- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe120067b00 msgr2=0x7fe120069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:29.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.313+0000 7fe146502640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe120067b00 0x7fe120069fc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe1300ad000 tx=0x7fe130008040 comp rx=0 tx=0).stop 2024-04-25T00:34:29.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.313+0000 7fe146502640 1 -- 172.21.15.29:0/2193776980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1400b4b10 msgr2=0x7fe140141830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:29.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.313+0000 7fe146502640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1400b4b10 0x7fe140141830 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe13800d560 tx=0x7fe13800da20 comp rx=0 tx=0).stop 2024-04-25T00:34:29.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.313+0000 7fe146502640 1 -- 172.21.15.29:0/2193776980 shutdown_connections 2024-04-25T00:34:29.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.313+0000 7fe146502640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1400b4b10 0x7fe140141830 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:29.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.313+0000 7fe146502640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe120067b00 0x7fe120069fc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:29.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.313+0000 7fe146502640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1400b3c30 0x7fe140071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:29.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.313+0000 7fe146502640 1 --2- 172.21.15.29:0/2193776980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1400b3140 0x7fe14013a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:29.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.313+0000 7fe146502640 1 -- 172.21.15.29:0/2193776980 >> 172.21.15.29:0/2193776980 conn(0x7fe1400b1870 msgr2=0x7fe140154df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:29.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.313+0000 7fe146502640 1 -- 172.21.15.29:0/2193776980 shutdown_connections 2024-04-25T00:34:29.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:29.313+0000 7fe146502640 1 -- 172.21.15.29:0/2193776980 wait complete. 2024-04-25T00:34:29.333 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:34:29.469 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":50.583386173000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.573946484011792,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:34:29.470 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.573946484011792 }, "state": "open", "uptime": 50.583386173 } ] 2024-04-25T00:34:29.470 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-25T00:34:34.471 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:34.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3f6e1640 1 Processor -- start 2024-04-25T00:34:34.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3f6e1640 1 -- start start 2024-04-25T00:34:34.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3f6e1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc3815df70 0x7fcc380b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3f6e1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380b4d30 0x7fcc380ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3f6e1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc3806d980 con 0x7fcc3806d210 2024-04-25T00:34:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3f6e1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fcc3806dc60 con 0x7fcc3815df70 2024-04-25T00:34:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3f6e1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3806df40 con 0x7fcc380b4d30 2024-04-25T00:34:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3d47d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc3815df70 0x7fcc380b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3d47d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc3815df70 0x7fcc380b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55688/0 (socket says 172.21.15.29:55688) 2024-04-25T00:34:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3d47d640 1 -- 172.21.15.29:0/580224765 learned_addr learned my addr 172.21.15.29:0/580224765 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3d47d640 1 -- 172.21.15.29:0/580224765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380b4d30 msgr2=0x7fcc380ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3dc7e640 1 --2- 172.21.15.29:0/580224765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380b4d30 0x7fcc380ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3d47d640 1 --2- 172.21.15.29:0/580224765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380b4d30 0x7fcc380ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3d47d640 1 -- 172.21.15.29:0/580224765 >> v1:172.21.15.157:6789/0 conn(0x7fcc3806d210 legacy=0x7fcc380b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.605+0000 7fcc3d47d640 1 -- 172.21.15.29:0/580224765 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3806e220 con 0x7fcc3815df70 2024-04-25T00:34:34.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.609+0000 7fcc3d47d640 1 --2- 172.21.15.29:0/580224765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815df70 0x7fcc380b3e40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009870 tx=0x7fcc3401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218469f4727f2235 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:34.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.609+0000 7fcc2ffff640 1 -- 172.21.15.29:0/580224765 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc340aa610 con 0x7fcc3815df70 2024-04-25T00:34:34.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.609+0000 7fcc2ffff640 1 -- 172.21.15.29:0/580224765 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc340af070 con 0x7fcc3815df70 2024-04-25T00:34:34.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.609+0000 7fcc2ffff640 1 -- 172.21.15.29:0/580224765 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc340aa9d0 con 0x7fcc3815df70 2024-04-25T00:34:34.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.609+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/580224765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815df70 msgr2=0x7fcc380b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:34.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.609+0000 7fcc3f6e1640 1 --2- 172.21.15.29:0/580224765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815df70 0x7fcc380b3e40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009870 tx=0x7fcc3401c4f0 comp rx=0 tx=0).stop 2024-04-25T00:34:34.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.609+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/580224765 shutdown_connections 2024-04-25T00:34:34.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.609+0000 7fcc3f6e1640 1 --2- 172.21.15.29:0/580224765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380b4d30 0x7fcc380ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:34.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.609+0000 7fcc3f6e1640 1 --2- 172.21.15.29:0/580224765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815df70 0x7fcc380b3e40 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:34.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.609+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/580224765 >> 172.21.15.29:0/580224765 conn(0x7fcc380b1870 msgr2=0x7fcc380b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:34.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.609+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/580224765 shutdown_connections 2024-04-25T00:34:34.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.609+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/580224765 wait complete. 2024-04-25T00:34:34.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.609+0000 7fcc3f6e1640 1 Processor -- start 2024-04-25T00:34:34.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3f6e1640 1 -- start start 2024-04-25T00:34:34.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3f6e1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3806d210 0x7fcc38151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:34.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3f6e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380b4d30 0x7fcc381425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:34.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3f6e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815df70 0x7fcc381561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:34.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3f6e1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3806eba0 con 0x7fcc380b4d30 2024-04-25T00:34:34.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3f6e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3806ea20 con 0x7fcc3815df70 2024-04-25T00:34:34.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3f6e1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcc380ae5f0 con 0x7fcc3806d210 2024-04-25T00:34:34.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3cc7c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380b4d30 0x7fcc381425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:34.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3cc7c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380b4d30 0x7fcc381425e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55694/0 (socket says 172.21.15.29:55694) 2024-04-25T00:34:34.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3cc7c640 1 -- 172.21.15.29:0/4181814713 learned_addr learned my addr 172.21.15.29:0/4181814713 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:34.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3d47d640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3806d210 0x7fcc38151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:34.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3dc7e640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815df70 0x7fcc381561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:34.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3cc7c640 1 -- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3806d210 msgr2=0x7fcc38151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:34.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3cc7c640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3806d210 0x7fcc38151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:34.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3cc7c640 1 -- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815df70 msgr2=0x7fcc381561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:34.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3cc7c640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815df70 0x7fcc381561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:34.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3cc7c640 1 -- 172.21.15.29:0/4181814713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc38142b20 con 0x7fcc380b4d30 2024-04-25T00:34:34.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3cc7c640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380b4d30 0x7fcc381425e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800c450 tx=0x7fcc2800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:34.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc2dffb640 1 -- 172.21.15.29:0/4181814713 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28012830 con 0x7fcc380b4d30 2024-04-25T00:34:34.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc2dffb640 1 -- 172.21.15.29:0/4181814713 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc28017070 con 0x7fcc380b4d30 2024-04-25T00:34:34.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/4181814713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc381585d0 con 0x7fcc380b4d30 2024-04-25T00:34:34.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc2dffb640 1 -- 172.21.15.29:0/4181814713 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28012c30 con 0x7fcc380b4d30 2024-04-25T00:34:34.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/4181814713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc38150ca0 con 0x7fcc380b4d30 2024-04-25T00:34:34.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/4181814713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc38069270 con 0x7fcc380b4d30 2024-04-25T00:34:34.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.613+0000 7fcc2dffb640 1 -- 172.21.15.29:0/4181814713 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7fcc2801c460 con 0x7fcc380b4d30 2024-04-25T00:34:34.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.617+0000 7fcc2dffb640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc1c067b00 0x7fcc1c069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:34.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.617+0000 7fcc2dffb640 1 -- 172.21.15.29:0/4181814713 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fcc28085ec0 con 0x7fcc380b4d30 2024-04-25T00:34:34.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.617+0000 7fcc3d47d640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc1c067b00 0x7fcc1c069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:34.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.617+0000 7fcc3d47d640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc1c067b00 0x7fcc1c069fc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcc340ae000 tx=0x7fcc340a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:34.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.621+0000 7fcc2dffb640 1 -- 172.21.15.29:0/4181814713 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcc28056650 con 0x7fcc380b4d30 2024-04-25T00:34:34.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/4181814713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc380b08e0 con 0x7fcc380b4d30 2024-04-25T00:34:34.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc2dffb640 1 -- 172.21.15.29:0/4181814713 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7fcc28055f50 con 0x7fcc380b4d30 2024-04-25T00:34:34.823 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:34.823 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:34.823 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:34.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc1c067b00 msgr2=0x7fcc1c069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:34.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc3f6e1640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc1c067b00 0x7fcc1c069fc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcc340ae000 tx=0x7fcc340a7000 comp rx=0 tx=0).stop 2024-04-25T00:34:34.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/4181814713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380b4d30 msgr2=0x7fcc381425e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:34.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc3f6e1640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380b4d30 0x7fcc381425e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800c450 tx=0x7fcc2800c910 comp rx=0 tx=0).stop 2024-04-25T00:34:34.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/4181814713 shutdown_connections 2024-04-25T00:34:34.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc3f6e1640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc1c067b00 0x7fcc1c069fc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:34.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc3f6e1640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815df70 0x7fcc381561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:34.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc3f6e1640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380b4d30 0x7fcc381425e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:34.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc3f6e1640 1 --2- 172.21.15.29:0/4181814713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3806d210 0x7fcc38151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:34.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/4181814713 >> 172.21.15.29:0/4181814713 conn(0x7fcc380b1870 msgr2=0x7fcc380b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:34.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/4181814713 shutdown_connections 2024-04-25T00:34:34.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:34.821+0000 7fcc3f6e1640 1 -- 172.21.15.29:0/4181814713 wait complete. 2024-04-25T00:34:34.842 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:34:34.980 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":56.095327828000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.326247219705916,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:34:34.980 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.326247219705916 }, "state": "open", "uptime": 56.095327828 } ] 2024-04-25T00:34:34.981 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-25T00:34:39.981 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:40.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.117+0000 7f264b984640 1 Processor -- start 2024-04-25T00:34:40.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.117+0000 7f264b984640 1 -- start start 2024-04-25T00:34:40.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f264b984640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f264406e440 0x7f26440b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:40.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f264b984640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26440b41e0 con 0x7f26440b3c30 2024-04-25T00:34:40.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f264b984640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264406d960 con 0x7f26440b3140 2024-04-25T00:34:40.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f264b984640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f26440b5050 con 0x7f264406e440 2024-04-25T00:34:40.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f2648f1f640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f26440b3c30 0x7f26440ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54686/0 (socket says 172.21.15.29:54686) 2024-04-25T00:34:40.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f2648f1f640 1 -- 172.21.15.29:0/1453748496 learned_addr learned my addr 172.21.15.29:0/1453748496 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:40.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f2649f21640 1 --2- 172.21.15.29:0/1453748496 >> v2:172.21.15.157:3300/0 conn(0x7f264406e440 0x7f26440b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:40.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1838701408 0 0) 0x7f26440b41e0 con 0x7f26440b3c30 2024-04-25T00:34:40.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2630003530 con 0x7f26440b3c30 2024-04-25T00:34:40.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4080566989 0 0) 0x7f2630003530 con 0x7f26440b3c30 2024-04-25T00:34:40.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 >> v2:172.21.15.157:3300/0 conn(0x7f264406e440 msgr2=0x7f26440b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:40.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 --2- 172.21.15.29:0/1453748496 >> v2:172.21.15.157:3300/0 conn(0x7f264406e440 0x7f26440b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:40.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 >> v1:172.21.15.157:6789/0 conn(0x7f26440b3140 legacy=0x7f26440b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:40.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264406dc40 con 0x7f26440b3c30 2024-04-25T00:34:40.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2634001680 con 0x7f26440b3c30 2024-04-25T00:34:40.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 >> v1:172.21.15.29:6789/0 conn(0x7f26440b3c30 legacy=0x7f26440ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:40.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 --2- 172.21.15.29:0/1453748496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26300039c0 0x7f2630003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:40.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 --2- 172.21.15.29:0/1453748496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2630004810 0x7f2630004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:40.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 --2- 172.21.15.29:0/1453748496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26300051d0 0x7f263000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:40.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f26440b5050 con 0x7f26300039c0 2024-04-25T00:34:40.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f263000c070 con 0x7f2630004810 2024-04-25T00:34:40.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f263000c270 con 0x7f26300051d0 2024-04-25T00:34:40.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f2649720640 1 --2- 172.21.15.29:0/1453748496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26300039c0 0x7f2630003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:40.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f2648f1f640 1 --2- 172.21.15.29:0/1453748496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2630004810 0x7f2630004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:40.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f2649720640 1 -- 172.21.15.29:0/1453748496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26300051d0 msgr2=0x7f263000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:40.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f2649720640 1 --2- 172.21.15.29:0/1453748496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26300051d0 0x7f263000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:40.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f2649720640 1 -- 172.21.15.29:0/1453748496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2630004810 msgr2=0x7f2630004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:40.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f2649720640 1 --2- 172.21.15.29:0/1453748496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2630004810 0x7f2630004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:40.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f2649720640 1 -- 172.21.15.29:0/1453748496 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2630003530 con 0x7f26300039c0 2024-04-25T00:34:40.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f2649f21640 1 -- 172.21.15.29:0/1453748496 reap_dead start 2024-04-25T00:34:40.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.121+0000 7f2648f1f640 1 --2- 172.21.15.29:0/1453748496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2630004810 0x7f2630004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:40.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f2649720640 1 --2- 172.21.15.29:0/1453748496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26300039c0 0x7f2630003de0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2640001780 tx=0x7f264001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf7552e3c1889fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:40.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26400013a0 con 0x7f26300039c0 2024-04-25T00:34:40.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26400052f0 con 0x7f26300039c0 2024-04-25T00:34:40.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f263bfff640 1 -- 172.21.15.29:0/1453748496 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640003840 con 0x7f26300039c0 2024-04-25T00:34:40.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 -- 172.21.15.29:0/1453748496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26300039c0 msgr2=0x7f2630003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:40.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 --2- 172.21.15.29:0/1453748496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26300039c0 0x7f2630003de0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2640001780 tx=0x7f264001c400 comp rx=0 tx=0).stop 2024-04-25T00:34:40.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 -- 172.21.15.29:0/1453748496 shutdown_connections 2024-04-25T00:34:40.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 --2- 172.21.15.29:0/1453748496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26300039c0 0x7f2630003de0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:40.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 -- 172.21.15.29:0/1453748496 >> 172.21.15.29:0/1453748496 conn(0x7f26440b1870 msgr2=0x7f26440b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:40.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 -- 172.21.15.29:0/1453748496 shutdown_connections 2024-04-25T00:34:40.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 -- 172.21.15.29:0/1453748496 wait complete. 2024-04-25T00:34:40.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 Processor -- start 2024-04-25T00:34:40.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 -- start start 2024-04-25T00:34:40.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26300039c0 0x7f26441f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:40.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f264413e6d0 0x7f26441f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:40.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26440b0710 0x7f26440b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:40.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f26440b3520 con 0x7f26300039c0 2024-04-25T00:34:40.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f264406e820 con 0x7f26440b0710 2024-04-25T00:34:40.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f264b984640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f26440b36a0 con 0x7f264413e6d0 2024-04-25T00:34:40.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f2649720640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26300039c0 0x7f26441f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:40.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f2649720640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26300039c0 0x7f26441f46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44588/0 (socket says 172.21.15.29:44588) 2024-04-25T00:34:40.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f2649720640 1 -- 172.21.15.29:0/3100556241 learned_addr learned my addr 172.21.15.29:0/3100556241 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:40.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f2649f21640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26440b0710 0x7f26440b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:40.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.125+0000 7f2648f1f640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f264413e6d0 0x7f26441f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:40.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2649720640 1 -- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f264413e6d0 msgr2=0x7f26441f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:40.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2649720640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f264413e6d0 0x7f26441f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:40.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2649720640 1 -- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26440b0710 msgr2=0x7f26440b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:40.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2649720640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26440b0710 0x7f26440b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:40.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2649720640 1 -- 172.21.15.29:0/3100556241 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264413f060 con 0x7f26300039c0 2024-04-25T00:34:40.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2649720640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26300039c0 0x7f26441f46b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2640005040 tx=0x7f2630005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:40.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2639ffb640 1 -- 172.21.15.29:0/3100556241 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640007a40 con 0x7f26300039c0 2024-04-25T00:34:40.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2639ffb640 1 -- 172.21.15.29:0/3100556241 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26400bb070 con 0x7f26300039c0 2024-04-25T00:34:40.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f264b984640 1 -- 172.21.15.29:0/3100556241 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26441f4c90 con 0x7f26300039c0 2024-04-25T00:34:40.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2639ffb640 1 -- 172.21.15.29:0/3100556241 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26400013a0 con 0x7f26300039c0 2024-04-25T00:34:40.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f264b984640 1 -- 172.21.15.29:0/3100556241 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26441f5250 con 0x7f26300039c0 2024-04-25T00:34:40.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f264b984640 1 -- 172.21.15.29:0/3100556241 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2644069270 con 0x7f26300039c0 2024-04-25T00:34:40.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2639ffb640 1 -- 172.21.15.29:0/3100556241 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f26400b4040 con 0x7f26300039c0 2024-04-25T00:34:40.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2639ffb640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2628067b00 0x7f2628069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:40.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2639ffb640 1 -- 172.21.15.29:0/3100556241 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f2640120f10 con 0x7f26300039c0 2024-04-25T00:34:40.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2648f1f640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2628067b00 0x7f2628069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:40.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.129+0000 7f2648f1f640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2628067b00 0x7f2628069fc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2634000c10 tx=0x7f2634009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:40.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.133+0000 7f2639ffb640 1 -- 172.21.15.29:0/3100556241 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2640016110 con 0x7f26300039c0 2024-04-25T00:34:40.198 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:34:40.201 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:34:40.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.333+0000 7f264b984640 1 -- 172.21.15.29:0/3100556241 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f264413ee40 con 0x7f26300039c0 2024-04-25T00:34:40.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.333+0000 7f2639ffb640 1 -- 172.21.15.29:0/3100556241 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7f26400f1080 con 0x7f26300039c0 2024-04-25T00:34:40.335 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:40.335 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:40.335 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:40.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.333+0000 7f264b984640 1 -- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2628067b00 msgr2=0x7f2628069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:40.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.333+0000 7f264b984640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2628067b00 0x7f2628069fc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2634000c10 tx=0x7f2634009040 comp rx=0 tx=0).stop 2024-04-25T00:34:40.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.337+0000 7f264b984640 1 -- 172.21.15.29:0/3100556241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26300039c0 msgr2=0x7f26441f46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:40.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.337+0000 7f264b984640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26300039c0 0x7f26441f46b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2640005040 tx=0x7f2630005840 comp rx=0 tx=0).stop 2024-04-25T00:34:40.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.337+0000 7f264b984640 1 -- 172.21.15.29:0/3100556241 shutdown_connections 2024-04-25T00:34:40.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.337+0000 7f264b984640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2628067b00 0x7f2628069fc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:40.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.337+0000 7f264b984640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26440b0710 0x7f26440b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:40.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.337+0000 7f264b984640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f264413e6d0 0x7f26441f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:40.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.337+0000 7f264b984640 1 --2- 172.21.15.29:0/3100556241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26300039c0 0x7f26441f46b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:40.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.337+0000 7f264b984640 1 -- 172.21.15.29:0/3100556241 >> 172.21.15.29:0/3100556241 conn(0x7f26440b1870 msgr2=0x7f264415b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:40.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.337+0000 7f264b984640 1 -- 172.21.15.29:0/3100556241 shutdown_connections 2024-04-25T00:34:40.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:40.337+0000 7f264b984640 1 -- 172.21.15.29:0/3100556241 wait complete. 2024-04-25T00:34:40.355 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:34:40.504 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":61.615269576000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.081329770114301,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:34:40.505 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.0813297701143 }, "state": "open", "uptime": 61.615269576 } ] 2024-04-25T00:34:40.505 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-25T00:34:45.506 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:45.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.641+0000 7f3eacda7640 1 Processor -- start 2024-04-25T00:34:45.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.641+0000 7f3eacda7640 1 -- start start 2024-04-25T00:34:45.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3eacda7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3ea80b3c30 0x7f3ea806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:45.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3eacda7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3ea80b4b10 0x7f3ea80ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:45.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3eacda7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea80b41e0 con 0x7f3ea80b3140 2024-04-25T00:34:45.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3eacda7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea806d960 con 0x7f3ea80b3c30 2024-04-25T00:34:45.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3eacda7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea806dc40 con 0x7f3ea80b4b10 2024-04-25T00:34:45.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3ea5d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3ea80b3c30 0x7f3ea806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:45.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3ea659c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3ea80b3140 0x7f3ea80b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54692/0 (socket says 172.21.15.29:54692) 2024-04-25T00:34:45.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3ea659c640 1 -- 172.21.15.29:0/2898860318 learned_addr learned my addr 172.21.15.29:0/2898860318 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:45.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3ea6d9d640 1 --2- 172.21.15.29:0/2898860318 >> v2:172.21.15.157:3300/0 conn(0x7f3ea80b4b10 0x7f3ea80ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:45.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3ea5d9b640 1 -- 172.21.15.29:0/2898860318 >> v2:172.21.15.157:3300/0 conn(0x7f3ea80b4b10 msgr2=0x7f3ea80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:45.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3ea5d9b640 1 --2- 172.21.15.29:0/2898860318 >> v2:172.21.15.157:3300/0 conn(0x7f3ea80b4b10 0x7f3ea80ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:45.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3ea5d9b640 1 -- 172.21.15.29:0/2898860318 >> v1:172.21.15.29:6789/0 conn(0x7f3ea80b3140 legacy=0x7f3ea80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:45.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3ea5d9b640 1 -- 172.21.15.29:0/2898860318 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea806df20 con 0x7f3ea80b3c30 2024-04-25T00:34:45.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3ea559a640 1 -- 172.21.15.29:0/2898860318 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1041385200 0 0) 0x7f3ea80b41e0 con 0x7f3ea80b3140 2024-04-25T00:34:45.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3ea5d9b640 1 --2- 172.21.15.29:0/2898860318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ea80b3c30 0x7f3ea806e440 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0097c0 tx=0x7f3e9c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7603a89f7c5a1de7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:45.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3ea559a640 1 -- 172.21.15.29:0/2898860318 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c0b0070 con 0x7f3ea80b3c30 2024-04-25T00:34:45.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3ea559a640 1 -- 172.21.15.29:0/2898860318 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e9c0ab600 con 0x7f3ea80b3c30 2024-04-25T00:34:45.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.645+0000 7f3ea559a640 1 -- 172.21.15.29:0/2898860318 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c0ab9d0 con 0x7f3ea80b3c30 2024-04-25T00:34:45.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 -- 172.21.15.29:0/2898860318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ea80b3c30 msgr2=0x7f3ea806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:45.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 --2- 172.21.15.29:0/2898860318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ea80b3c30 0x7f3ea806e440 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0097c0 tx=0x7f3e9c01c460 comp rx=0 tx=0).stop 2024-04-25T00:34:45.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 -- 172.21.15.29:0/2898860318 shutdown_connections 2024-04-25T00:34:45.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 --2- 172.21.15.29:0/2898860318 >> v2:172.21.15.157:3300/0 conn(0x7f3ea80b4b10 0x7f3ea80ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:45.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 --2- 172.21.15.29:0/2898860318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ea80b3c30 0x7f3ea806e440 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:45.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 -- 172.21.15.29:0/2898860318 >> 172.21.15.29:0/2898860318 conn(0x7f3ea80b1870 msgr2=0x7f3ea80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:45.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 -- 172.21.15.29:0/2898860318 shutdown_connections 2024-04-25T00:34:45.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 -- 172.21.15.29:0/2898860318 wait complete. 2024-04-25T00:34:45.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 Processor -- start 2024-04-25T00:34:45.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 -- start start 2024-04-25T00:34:45.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea80b3140 0x7f3ea81431b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:45.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea80b3c30 0x7f3ea8152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:45.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ea80b4b10 0x7f3ea8152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:45.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea8168d80 con 0x7f3ea80b4b10 2024-04-25T00:34:45.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea8168c00 con 0x7f3ea80b3c30 2024-04-25T00:34:45.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea8168f00 con 0x7f3ea80b3140 2024-04-25T00:34:45.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3ea6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ea80b4b10 0x7f3ea8152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:45.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3ea6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ea80b4b10 0x7f3ea8152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44614/0 (socket says 172.21.15.29:44614) 2024-04-25T00:34:45.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3ea6d9d640 1 -- 172.21.15.29:0/2152448575 learned_addr learned my addr 172.21.15.29:0/2152448575 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:45.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3ea6d9d640 1 -- 172.21.15.29:0/2152448575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea80b3140 msgr2=0x7f3ea81431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:45.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3ea6d9d640 1 --2- 172.21.15.29:0/2152448575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea80b3140 0x7f3ea81431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:45.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3ea6d9d640 1 -- 172.21.15.29:0/2152448575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea80b3c30 msgr2=0x7f3ea8152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:45.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3ea5d9b640 1 --2- 172.21.15.29:0/2152448575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea80b3c30 0x7f3ea8152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:45.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3ea6d9d640 1 --2- 172.21.15.29:0/2152448575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea80b3c30 0x7f3ea8152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:45.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3ea6d9d640 1 -- 172.21.15.29:0/2152448575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea815d620 con 0x7f3ea80b4b10 2024-04-25T00:34:45.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3ea6d9d640 1 --2- 172.21.15.29:0/2152448575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ea80b4b10 0x7f3ea8152570 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0007e70 tx=0x7f3ea000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:45.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3e96ffd640 1 -- 172.21.15.29:0/2152448575 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0014620 con 0x7f3ea80b4b10 2024-04-25T00:34:45.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3e96ffd640 1 -- 172.21.15.29:0/2152448575 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea0019070 con 0x7f3ea80b4b10 2024-04-25T00:34:45.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 -- 172.21.15.29:0/2152448575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea8140c30 con 0x7f3ea80b4b10 2024-04-25T00:34:45.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3e96ffd640 1 -- 172.21.15.29:0/2152448575 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0014a20 con 0x7f3ea80b4b10 2024-04-25T00:34:45.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.649+0000 7f3eacda7640 1 -- 172.21.15.29:0/2152448575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea815d8c0 con 0x7f3ea80b4b10 2024-04-25T00:34:45.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.653+0000 7f3eacda7640 1 -- 172.21.15.29:0/2152448575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea8069270 con 0x7f3ea80b4b10 2024-04-25T00:34:45.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.653+0000 7f3e96ffd640 1 -- 172.21.15.29:0/2152448575 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f3ea0014bc0 con 0x7f3ea80b4b10 2024-04-25T00:34:45.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.653+0000 7f3e96ffd640 1 --2- 172.21.15.29:0/2152448575 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3e88067b00 0x7f3e88069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:45.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.653+0000 7f3e96ffd640 1 -- 172.21.15.29:0/2152448575 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f3ea0086950 con 0x7f3ea80b4b10 2024-04-25T00:34:45.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.653+0000 7f3ea659c640 1 --2- 172.21.15.29:0/2152448575 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3e88067b00 0x7f3e88069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:45.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.653+0000 7f3ea659c640 1 --2- 172.21.15.29:0/2152448575 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3e88067b00 0x7f3e88069fc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80cf3e0 tx=0x7f3e98007970 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:45.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.657+0000 7f3e96ffd640 1 -- 172.21.15.29:0/2152448575 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3ea0011260 con 0x7f3ea80b4b10 2024-04-25T00:34:45.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.853+0000 7f3eacda7640 1 -- 172.21.15.29:0/2152448575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ea8154100 con 0x7f3ea80b4b10 2024-04-25T00:34:45.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.853+0000 7f3e96ffd640 1 -- 172.21.15.29:0/2152448575 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7f3ea00571c0 con 0x7f3ea80b4b10 2024-04-25T00:34:45.855 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:45.855 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:45.856 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:45.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.857+0000 7f3eacda7640 1 -- 172.21.15.29:0/2152448575 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3e88067b00 msgr2=0x7f3e88069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:45.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.857+0000 7f3eacda7640 1 --2- 172.21.15.29:0/2152448575 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3e88067b00 0x7f3e88069fc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80cf3e0 tx=0x7f3e98007970 comp rx=0 tx=0).stop 2024-04-25T00:34:45.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.857+0000 7f3eacda7640 1 -- 172.21.15.29:0/2152448575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ea80b4b10 msgr2=0x7f3ea8152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:45.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.857+0000 7f3eacda7640 1 --2- 172.21.15.29:0/2152448575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ea80b4b10 0x7f3ea8152570 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0007e70 tx=0x7f3ea000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:34:45.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.857+0000 7f3eacda7640 1 -- 172.21.15.29:0/2152448575 shutdown_connections 2024-04-25T00:34:45.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.857+0000 7f3eacda7640 1 --2- 172.21.15.29:0/2152448575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ea80b4b10 0x7f3ea8152570 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:45.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.857+0000 7f3eacda7640 1 --2- 172.21.15.29:0/2152448575 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3e88067b00 0x7f3e88069fc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:45.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.857+0000 7f3eacda7640 1 --2- 172.21.15.29:0/2152448575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea80b3c30 0x7f3ea8152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:45.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.857+0000 7f3eacda7640 1 --2- 172.21.15.29:0/2152448575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea80b3140 0x7f3ea81431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:45.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.857+0000 7f3eacda7640 1 -- 172.21.15.29:0/2152448575 >> 172.21.15.29:0/2152448575 conn(0x7f3ea80b1870 msgr2=0x7f3ea8144550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:45.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.857+0000 7f3eacda7640 1 -- 172.21.15.29:0/2152448575 shutdown_connections 2024-04-25T00:34:45.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:45.857+0000 7f3eacda7640 1 -- 172.21.15.29:0/2152448575 wait complete. 2024-04-25T00:34:45.873 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:34:46.008 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5240,"entity":{"name":{"type":"client","num":5240},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":3047473862}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":67.123211628000007,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.840038443945488,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5240 172.21.15.157:0/3047473862","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:34:46.009 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 3047473862, "type": "any" }, "name": { "num": 5240, "type": "client" } }, "id": 5240, "inst": "client.5240 172.21.15.157:0/3047473862", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.840038443945488 }, "state": "open", "uptime": 67.123211628 } ] 2024-04-25T00:34:46.009 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-04-25T00:34:51.010 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:51.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.145+0000 7fbbec610640 1 Processor -- start 2024-04-25T00:34:51.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.145+0000 7fbbec610640 1 -- start start 2024-04-25T00:34:51.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.145+0000 7fbbec610640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbbe40cf690 0x7fbbe40d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.145+0000 7fbbec610640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbe40d2ea0 con 0x7fbbe40d2a90 2024-04-25T00:34:51.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.145+0000 7fbbec610640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbe40cfc10 con 0x7fbbe415e200 2024-04-25T00:34:51.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.145+0000 7fbbec610640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbbe40b38d0 con 0x7fbbe40cf690 2024-04-25T00:34:51.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.145+0000 7fbbe9bab640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbbe40cf690 0x7fbbe40d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:51.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.145+0000 7fbbeabad640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbbe40d2a90 0x7fbbe40b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35990/0 (socket says 172.21.15.29:35990) 2024-04-25T00:34:51.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.145+0000 7fbbeabad640 1 -- 172.21.15.29:0/692623372 learned_addr learned my addr 172.21.15.29:0/692623372 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:51.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.145+0000 7fbbe9bab640 1 -- 172.21.15.29:0/692623372 >> v1:172.21.15.157:6789/0 conn(0x7fbbe415e200 legacy=0x7fbbe40d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:51.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.145+0000 7fbbe9bab640 1 -- 172.21.15.29:0/692623372 >> v1:172.21.15.29:6789/0 conn(0x7fbbe40d2a90 legacy=0x7fbbe40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:51.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.145+0000 7fbbe93aa640 1 -- 172.21.15.29:0/692623372 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3040426749 0 0) 0x7fbbdc0014a0 con 0x7fbbe40d2a90 2024-04-25T00:34:51.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.145+0000 7fbbe9bab640 1 -- 172.21.15.29:0/692623372 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe40cfef0 con 0x7fbbe40cf690 2024-04-25T00:34:51.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbe9bab640 1 --2- 172.21.15.29:0/692623372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbe40cf690 0x7fbbe40d2550 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40097c0 tx=0x7fbbd401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a772944d0904fe35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:51.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbe93aa640 1 -- 172.21.15.29:0/692623372 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40b0070 con 0x7fbbe40cf690 2024-04-25T00:34:51.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbe93aa640 1 -- 172.21.15.29:0/692623372 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd4004230 con 0x7fbbe40cf690 2024-04-25T00:34:51.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbe93aa640 1 -- 172.21.15.29:0/692623372 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4003620 con 0x7fbbe40cf690 2024-04-25T00:34:51.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 -- 172.21.15.29:0/692623372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbe40cf690 msgr2=0x7fbbe40d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:51.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 --2- 172.21.15.29:0/692623372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbe40cf690 0x7fbbe40d2550 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40097c0 tx=0x7fbbd401c460 comp rx=0 tx=0).stop 2024-04-25T00:34:51.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 -- 172.21.15.29:0/692623372 shutdown_connections 2024-04-25T00:34:51.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 --2- 172.21.15.29:0/692623372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbe40cf690 0x7fbbe40d2550 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 -- 172.21.15.29:0/692623372 >> 172.21.15.29:0/692623372 conn(0x7fbbe409d280 msgr2=0x7fbbe40d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:51.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 -- 172.21.15.29:0/692623372 shutdown_connections 2024-04-25T00:34:51.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 -- 172.21.15.29:0/692623372 wait complete. 2024-04-25T00:34:51.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 Processor -- start 2024-04-25T00:34:51.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 -- start start 2024-04-25T00:34:51.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbe40cf690 0x7fbbe40ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbe40d2a90 0x7fbbe41552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbe415e200 0x7fbbe40b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe40b53c0 con 0x7fbbe415e200 2024-04-25T00:34:51.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe40b5240 con 0x7fbbe40cf690 2024-04-25T00:34:51.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbec610640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe40b5540 con 0x7fbbe40d2a90 2024-04-25T00:34:51.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbeabad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbe415e200 0x7fbbe40b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:51.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbeabad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbe415e200 0x7fbbe40b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40236/0 (socket says 172.21.15.29:40236) 2024-04-25T00:34:51.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbeabad640 1 -- 172.21.15.29:0/1296687434 learned_addr learned my addr 172.21.15.29:0/1296687434 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:51.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbea3ac640 1 --2- 172.21.15.29:0/1296687434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbe40cf690 0x7fbbe40ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:51.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbeabad640 1 -- 172.21.15.29:0/1296687434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbe40d2a90 msgr2=0x7fbbe41552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:51.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbeabad640 1 --2- 172.21.15.29:0/1296687434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbe40d2a90 0x7fbbe41552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbeabad640 1 -- 172.21.15.29:0/1296687434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbe40cf690 msgr2=0x7fbbe40ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:51.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbeabad640 1 --2- 172.21.15.29:0/1296687434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbe40cf690 0x7fbbe40ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.149+0000 7fbbeabad640 1 -- 172.21.15.29:0/1296687434 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe41527c0 con 0x7fbbe415e200 2024-04-25T00:34:51.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.153+0000 7fbbeabad640 1 --2- 172.21.15.29:0/1296687434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbe415e200 0x7fbbe40b15c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c700 tx=0x7fbbdc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:51.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.153+0000 7fbbdaffd640 1 -- 172.21.15.29:0/1296687434 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc018070 con 0x7fbbe415e200 2024-04-25T00:34:51.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.153+0000 7fbbdaffd640 1 -- 172.21.15.29:0/1296687434 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbdc00ce10 con 0x7fbbe415e200 2024-04-25T00:34:51.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.153+0000 7fbbec610640 1 -- 172.21.15.29:0/1296687434 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe415f790 con 0x7fbbe415e200 2024-04-25T00:34:51.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.153+0000 7fbbdaffd640 1 -- 172.21.15.29:0/1296687434 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc00a860 con 0x7fbbe415e200 2024-04-25T00:34:51.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.153+0000 7fbbec610640 1 -- 172.21.15.29:0/1296687434 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe415fd10 con 0x7fbbe415e200 2024-04-25T00:34:51.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.153+0000 7fbbec610640 1 -- 172.21.15.29:0/1296687434 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe4069270 con 0x7fbbe415e200 2024-04-25T00:34:51.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.153+0000 7fbbdaffd640 1 -- 172.21.15.29:0/1296687434 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fbbdc01b070 con 0x7fbbe415e200 2024-04-25T00:34:51.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.153+0000 7fbbdaffd640 1 --2- 172.21.15.29:0/1296687434 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbbc8067a70 0x7fbbc8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.153+0000 7fbbdaffd640 1 -- 172.21.15.29:0/1296687434 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fbbdc0854d0 con 0x7fbbe415e200 2024-04-25T00:34:51.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.153+0000 7fbbea3ac640 1 --2- 172.21.15.29:0/1296687434 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbbc8067a70 0x7fbbc8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:51.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.157+0000 7fbbea3ac640 1 --2- 172.21.15.29:0/1296687434 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbbc8067a70 0x7fbbc8069f30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40d49d0 tx=0x7fbbe0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:51.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.157+0000 7fbbdaffd640 1 -- 172.21.15.29:0/1296687434 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbbe40d2ea0 con 0x7fbbe415e200 2024-04-25T00:34:51.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.357+0000 7fbbec610640 1 -- 172.21.15.29:0/1296687434 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbe4159f90 con 0x7fbbe415e200 2024-04-25T00:34:51.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.357+0000 7fbbdaffd640 1 -- 172.21.15.29:0/1296687434 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7fbbdc055de0 con 0x7fbbe415e200 2024-04-25T00:34:51.360 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:51.360 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:51.361 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:51.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.361+0000 7fbbec610640 1 -- 172.21.15.29:0/1296687434 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbbc8067a70 msgr2=0x7fbbc8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:51.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.361+0000 7fbbec610640 1 --2- 172.21.15.29:0/1296687434 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbbc8067a70 0x7fbbc8069f30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40d49d0 tx=0x7fbbe0009040 comp rx=0 tx=0).stop 2024-04-25T00:34:51.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.361+0000 7fbbec610640 1 -- 172.21.15.29:0/1296687434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbe415e200 msgr2=0x7fbbe40b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:51.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.361+0000 7fbbec610640 1 --2- 172.21.15.29:0/1296687434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbe415e200 0x7fbbe40b15c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c700 tx=0x7fbbdc00cbc0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.361+0000 7fbbec610640 1 -- 172.21.15.29:0/1296687434 shutdown_connections 2024-04-25T00:34:51.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.361+0000 7fbbec610640 1 --2- 172.21.15.29:0/1296687434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbe415e200 0x7fbbe40b15c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.361+0000 7fbbec610640 1 --2- 172.21.15.29:0/1296687434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbe40d2a90 0x7fbbe41552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.361+0000 7fbbec610640 1 --2- 172.21.15.29:0/1296687434 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbbc8067a70 0x7fbbc8069f30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.361+0000 7fbbec610640 1 --2- 172.21.15.29:0/1296687434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbe40cf690 0x7fbbe40ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.361+0000 7fbbec610640 1 -- 172.21.15.29:0/1296687434 >> 172.21.15.29:0/1296687434 conn(0x7fbbe409d280 msgr2=0x7fbbe40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:51.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.361+0000 7fbbec610640 1 -- 172.21.15.29:0/1296687434 shutdown_connections 2024-04-25T00:34:51.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.361+0000 7fbbec610640 1 -- 172.21.15.29:0/1296687434 wait complete. 2024-04-25T00:34:51.380 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:34:51.519 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-04-25T00:34:51.519 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-04-25T00:34:51.520 DEBUG:tasks.ceph_test_case:wait_until_true: success in 65s and 0 retries 2024-04-25T00:34:51.520 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:34:51.520 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:34:51.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:34:51.611 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:34:51.611 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:34:51.612 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:34:51.612 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:34:51.618 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:34:51.618 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:34:51.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:34:51.619 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:34:51.666 INFO:teuthology.orchestra.run.smithi029.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T00:34:51.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:34:51.667 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:34:51.667 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:34:51.667 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:34:51.673 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:34:51.673 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:34:51.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:34:51.673 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:34:51.721 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T00:34:51.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:34:51.722 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:34:51.722 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:34:51.722 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:34:51.729 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:34:51.729 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:34:51.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:34:51.729 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:34:51.778 INFO:teuthology.orchestra.run.smithi029.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T00:34:51.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:34:51.779 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:34:51.779 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:34:51.779 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:34:51.785 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:34:51.785 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:34:51.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:34:51.786 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:34:51.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:34:51.833 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T00:34:51.833 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:51.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa56f54640 1 Processor -- start 2024-04-25T00:34:51.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa56f54640 1 -- start start 2024-04-25T00:34:51.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa56f54640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5015e250 0x7faa500d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa56f54640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faa5015e630 0x7faa500d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa56f54640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa500c8690 con 0x7faa500c9450 2024-04-25T00:34:51.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa56f54640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faa500c8440 con 0x7faa5015e630 2024-04-25T00:34:51.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa56f54640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faa500c8970 con 0x7faa5015e250 2024-04-25T00:34:51.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa554f1640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7faa500c9450 0x7faa500c98e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36002/0 (socket says 172.21.15.29:36002) 2024-04-25T00:34:51.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa554f1640 1 -- 172.21.15.29:0/3306590439 learned_addr learned my addr 172.21.15.29:0/3306590439 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:51.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4ffff640 1 --2- 172.21.15.29:0/3306590439 >> v2:172.21.15.157:3300/0 conn(0x7faa5015e630 0x7faa500d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:51.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa54cf0640 1 --2- 172.21.15.29:0/3306590439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5015e250 0x7faa500d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:51.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1066814110 0 0) 0x7faa500c8690 con 0x7faa500c9450 2024-04-25T00:34:51.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa38003530 con 0x7faa500c9450 2024-04-25T00:34:51.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1850820782 0 0) 0x7faa38003530 con 0x7faa500c9450 2024-04-25T00:34:51.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5015e250 msgr2=0x7faa500d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:51.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 --2- 172.21.15.29:0/3306590439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5015e250 0x7faa500d24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 >> v2:172.21.15.157:3300/0 conn(0x7faa5015e630 msgr2=0x7faa500d29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:51.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 --2- 172.21.15.29:0/3306590439 >> v2:172.21.15.157:3300/0 conn(0x7faa5015e630 0x7faa500d29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa500c8c50 con 0x7faa500c9450 2024-04-25T00:34:51.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7faa48001a00 con 0x7faa500c9450 2024-04-25T00:34:51.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 >> v1:172.21.15.29:6789/0 conn(0x7faa500c9450 legacy=0x7faa500c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:51.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 --2- 172.21.15.29:0/3306590439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5015e250 0x7faa380039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 --2- 172.21.15.29:0/3306590439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa38003f60 0x7faa38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 --2- 172.21.15.29:0/3306590439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa38004d50 0x7faa380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faa500c8440 con 0x7faa38004d50 2024-04-25T00:34:51.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faa500c8970 con 0x7faa38003f60 2024-04-25T00:34:51.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faa3800ba50 con 0x7faa5015e250 2024-04-25T00:34:51.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa554f1640 1 --2- 172.21.15.29:0/3306590439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa38004d50 0x7faa380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:51.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa4ffff640 1 --2- 172.21.15.29:0/3306590439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa38003f60 0x7faa38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:51.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa54cf0640 1 --2- 172.21.15.29:0/3306590439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5015e250 0x7faa380039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:51.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa554f1640 1 -- 172.21.15.29:0/3306590439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5015e250 msgr2=0x7faa380039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:51.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa554f1640 1 --2- 172.21.15.29:0/3306590439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5015e250 0x7faa380039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa554f1640 1 -- 172.21.15.29:0/3306590439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa38003f60 msgr2=0x7faa38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:51.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa554f1640 1 --2- 172.21.15.29:0/3306590439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa38003f60 0x7faa38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.965+0000 7faa554f1640 1 -- 172.21.15.29:0/3306590439 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa38003530 con 0x7faa38004d50 2024-04-25T00:34:51.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.969+0000 7faa554f1640 1 --2- 172.21.15.29:0/3306590439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa38004d50 0x7faa380051d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7faa4800a870 tx=0x7faa4801fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4256d13b9a6905a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:51.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.969+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4800a4c0 con 0x7faa38004d50 2024-04-25T00:34:51.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.969+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa480ad020 con 0x7faa38004d50 2024-04-25T00:34:51.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.969+0000 7faa4f7fe640 1 -- 172.21.15.29:0/3306590439 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa480a9760 con 0x7faa38004d50 2024-04-25T00:34:51.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.969+0000 7faa56f54640 1 -- 172.21.15.29:0/3306590439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa38004d50 msgr2=0x7faa380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:51.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.969+0000 7faa56f54640 1 --2- 172.21.15.29:0/3306590439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa38004d50 0x7faa380051d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7faa4800a870 tx=0x7faa4801fb00 comp rx=0 tx=0).stop 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.969+0000 7faa554f1640 1 -- 172.21.15.29:0/3306590439 reap_dead start 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.969+0000 7faa56f54640 1 -- 172.21.15.29:0/3306590439 shutdown_connections 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.969+0000 7faa56f54640 1 -- 172.21.15.29:0/3306590439 >> 172.21.15.29:0/3306590439 conn(0x7faa500d3f70 msgr2=0x7faa500d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.969+0000 7faa56f54640 1 -- 172.21.15.29:0/3306590439 shutdown_connections 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.969+0000 7faa56f54640 1 -- 172.21.15.29:0/3306590439 wait complete. 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.969+0000 7faa56f54640 1 Processor -- start 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.969+0000 7faa56f54640 1 -- start start 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa56f54640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa5009ded0 0x7faa500b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa56f54640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5009e2b0 0x7faa50152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa56f54640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa50152590 0x7faa500a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa56f54640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faa50074fc0 con 0x7faa50152590 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa56f54640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faa50074e40 con 0x7faa5009ded0 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa56f54640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faa500d1cc0 con 0x7faa5009e2b0 2024-04-25T00:34:51.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa554f1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa50152590 0x7faa500a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:51.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa554f1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa50152590 0x7faa500a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40254/0 (socket says 172.21.15.29:40254) 2024-04-25T00:34:51.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa554f1640 1 -- 172.21.15.29:0/11145599 learned_addr learned my addr 172.21.15.29:0/11145599 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:51.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa54cf0640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa5009ded0 0x7faa500b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:51.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa554f1640 1 -- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5009e2b0 msgr2=0x7faa50152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:51.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa4ffff640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5009e2b0 0x7faa50152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:51.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa554f1640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5009e2b0 0x7faa50152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa554f1640 1 -- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa5009ded0 msgr2=0x7faa500b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:51.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa554f1640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa5009ded0 0x7faa500b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:51.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa554f1640 1 -- 172.21.15.29:0/11145599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5009fc80 con 0x7faa50152590 2024-04-25T00:34:51.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa554f1640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa50152590 0x7faa500a0ce0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7faa48000f90 tx=0x7faa38005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:51.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa4d7fa640 1 -- 172.21.15.29:0/11145599 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4800a4c0 con 0x7faa50152590 2024-04-25T00:34:51.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa4d7fa640 1 -- 172.21.15.29:0/11145599 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa480a9720 con 0x7faa50152590 2024-04-25T00:34:51.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa56f54640 1 -- 172.21.15.29:0/11145599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5009ff20 con 0x7faa50152590 2024-04-25T00:34:51.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa4d7fa640 1 -- 172.21.15.29:0/11145599 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48002040 con 0x7faa50152590 2024-04-25T00:34:51.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa56f54640 1 -- 172.21.15.29:0/11145599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa500a04e0 con 0x7faa50152590 2024-04-25T00:34:51.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa56f54640 1 -- 172.21.15.29:0/11145599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa50069270 con 0x7faa50152590 2024-04-25T00:34:51.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa4d7fa640 1 -- 172.21.15.29:0/11145599 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7faa480a9ad0 con 0x7faa50152590 2024-04-25T00:34:51.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa4d7fa640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faa3c067a70 0x7faa3c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:51.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa4d7fa640 1 -- 172.21.15.29:0/11145599 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7faa481230b0 con 0x7faa50152590 2024-04-25T00:34:51.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.973+0000 7faa54cf0640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faa3c067a70 0x7faa3c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:51.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.977+0000 7faa54cf0640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faa3c067a70 0x7faa3c069f30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faa40009a30 tx=0x7faa40008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:51.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:51.977+0000 7faa4d7fa640 1 -- 172.21.15.29:0/11145599 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faa48019810 con 0x7faa50152590 2024-04-25T00:34:52.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.177+0000 7faa56f54640 1 -- 172.21.15.29:0/11145599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa5015eaa0 con 0x7faa50152590 2024-04-25T00:34:52.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.181+0000 7faa4d7fa640 1 -- 172.21.15.29:0/11145599 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7faa480f3800 con 0x7faa50152590 2024-04-25T00:34:52.182 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:52.182 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:52.183 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:52.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.181+0000 7faa56f54640 1 -- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faa3c067a70 msgr2=0x7faa3c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.181+0000 7faa56f54640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faa3c067a70 0x7faa3c069f30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faa40009a30 tx=0x7faa40008040 comp rx=0 tx=0).stop 2024-04-25T00:34:52.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.181+0000 7faa56f54640 1 -- 172.21.15.29:0/11145599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa50152590 msgr2=0x7faa500a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.181+0000 7faa56f54640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa50152590 0x7faa500a0ce0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7faa48000f90 tx=0x7faa38005840 comp rx=0 tx=0).stop 2024-04-25T00:34:52.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.181+0000 7faa56f54640 1 -- 172.21.15.29:0/11145599 shutdown_connections 2024-04-25T00:34:52.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.181+0000 7faa56f54640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa50152590 0x7faa500a0ce0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.181+0000 7faa56f54640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5009e2b0 0x7faa50152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.181+0000 7faa56f54640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faa3c067a70 0x7faa3c069f30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.181+0000 7faa56f54640 1 --2- 172.21.15.29:0/11145599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa5009ded0 0x7faa500b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.181+0000 7faa56f54640 1 -- 172.21.15.29:0/11145599 >> 172.21.15.29:0/11145599 conn(0x7faa500d3f70 msgr2=0x7faa5006bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:52.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.181+0000 7faa56f54640 1 -- 172.21.15.29:0/11145599 shutdown_connections 2024-04-25T00:34:52.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.181+0000 7faa56f54640 1 -- 172.21.15.29:0/11145599 wait complete. 2024-04-25T00:34:52.202 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:34:52.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f75861bd640 1 Processor -- start 2024-04-25T00:34:52.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f75861bd640 1 -- start start 2024-04-25T00:34:52.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f75861bd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7580153030 0x7f758006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f75861bd640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f758015d580 con 0x7f758006b920 2024-04-25T00:34:52.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f75861bd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f758015c610 con 0x7f75800cd200 2024-04-25T00:34:52.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f75861bd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f758015d3d0 con 0x7f7580153030 2024-04-25T00:34:52.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757ffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f758006b920 0x7f75800b5ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36010/0 (socket says 172.21.15.29:36010) 2024-04-25T00:34:52.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757ffff640 1 -- 172.21.15.29:0/2085351153 learned_addr learned my addr 172.21.15.29:0/2085351153 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757f7fe640 1 --2- 172.21.15.29:0/2085351153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7580153030 0x7f758006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1004031723 0 0) 0x7f758015d580 con 0x7f758006b920 2024-04-25T00:34:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f756c003530 con 0x7f758006b920 2024-04-25T00:34:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2230132054 0 0) 0x7f758015c610 con 0x7f75800cd200 2024-04-25T00:34:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f758015d580 con 0x7f75800cd200 2024-04-25T00:34:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2222971148 0 0) 0x7f756c003530 con 0x7f758006b920 2024-04-25T00:34:52.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7580153030 msgr2=0x7f758006fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:52.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 --2- 172.21.15.29:0/2085351153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7580153030 0x7f758006fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 >> v1:172.21.15.157:6789/0 conn(0x7f75800cd200 legacy=0x7f758006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758015c8f0 con 0x7f758006b920 2024-04-25T00:34:52.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f7578001680 con 0x7f758006b920 2024-04-25T00:34:52.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 >> v1:172.21.15.29:6789/0 conn(0x7f758006b920 legacy=0x7f75800b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 --2- 172.21.15.29:0/2085351153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f756c003b80 0x7f756c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 --2- 172.21.15.29:0/2085351153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c004810 0x7f756c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 --2- 172.21.15.29:0/2085351153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7580153030 0x7f756c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f756c00ba80 con 0x7f756c003b80 2024-04-25T00:34:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f756c00bc00 con 0x7f756c004810 2024-04-25T00:34:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f756c00bdd0 con 0x7f7580153030 2024-04-25T00:34:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757f7fe640 1 --2- 172.21.15.29:0/2085351153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f756c003b80 0x7f756c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757effd640 1 --2- 172.21.15.29:0/2085351153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c004810 0x7f756c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757ffff640 1 --2- 172.21.15.29:0/2085351153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7580153030 0x7f756c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757f7fe640 1 -- 172.21.15.29:0/2085351153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7580153030 msgr2=0x7f756c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757f7fe640 1 --2- 172.21.15.29:0/2085351153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7580153030 0x7f756c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757f7fe640 1 -- 172.21.15.29:0/2085351153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c004810 msgr2=0x7f756c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757f7fe640 1 --2- 172.21.15.29:0/2085351153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c004810 0x7f756c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.325+0000 7f757f7fe640 1 -- 172.21.15.29:0/2085351153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756c003530 con 0x7f756c003b80 2024-04-25T00:34:52.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.329+0000 7f757f7fe640 1 --2- 172.21.15.29:0/2085351153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f756c003b80 0x7f756c003f80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7570009ae0 tx=0x7f757001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec1b30be29f4372 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.329+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700096b0 con 0x7f756c003b80 2024-04-25T00:34:52.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.329+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75700a6500 con 0x7f756c003b80 2024-04-25T00:34:52.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.329+0000 7f757e7fc640 1 -- 172.21.15.29:0/2085351153 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700a68f0 con 0x7f756c003b80 2024-04-25T00:34:52.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.329+0000 7f75861bd640 1 -- 172.21.15.29:0/2085351153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f756c003b80 msgr2=0x7f756c003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.329+0000 7f75861bd640 1 --2- 172.21.15.29:0/2085351153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f756c003b80 0x7f756c003f80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7570009ae0 tx=0x7f757001ca10 comp rx=0 tx=0).stop 2024-04-25T00:34:52.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.329+0000 7f757ffff640 1 -- 172.21.15.29:0/2085351153 reap_dead start 2024-04-25T00:34:52.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.329+0000 7f75861bd640 1 -- 172.21.15.29:0/2085351153 shutdown_connections 2024-04-25T00:34:52.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.329+0000 7f75861bd640 1 -- 172.21.15.29:0/2085351153 >> 172.21.15.29:0/2085351153 conn(0x7f75800d0c10 msgr2=0x7f75800d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.329+0000 7f75861bd640 1 -- 172.21.15.29:0/2085351153 shutdown_connections 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.329+0000 7f75861bd640 1 -- 172.21.15.29:0/2085351153 wait complete. 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.329+0000 7f75861bd640 1 Processor -- start 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.329+0000 7f75861bd640 1 -- start start 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f75861bd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c003b80 0x7f7580151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f75861bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75801510f0 0x7f75800d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f75861bd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758015ea50 0x7f75800b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f75861bd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f75800702e0 con 0x7f75801510f0 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f75861bd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7580070160 con 0x7f756c003b80 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f75861bd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7580070460 con 0x7f758015ea50 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f757effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75801510f0 0x7f75800d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f757effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75801510f0 0x7f75800d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40268/0 (socket says 172.21.15.29:40268) 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f757effd640 1 -- 172.21.15.29:0/190527243 learned_addr learned my addr 172.21.15.29:0/190527243 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:52.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f757f7fe640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c003b80 0x7f7580151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f757ffff640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758015ea50 0x7f75800b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f757effd640 1 -- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758015ea50 msgr2=0x7f75800b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f757effd640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758015ea50 0x7f75800b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f757effd640 1 -- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c003b80 msgr2=0x7f7580151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f757effd640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c003b80 0x7f7580151760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f757effd640 1 -- 172.21.15.29:0/190527243 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7580160c70 con 0x7f75801510f0 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f757effd640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75801510f0 0x7f75800d02c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f756800eb50 tx=0x7f756800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f7577fff640 1 -- 172.21.15.29:0/190527243 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756800ee50 con 0x7f75801510f0 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f7577fff640 1 -- 172.21.15.29:0/190527243 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7568018070 con 0x7f75801510f0 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f75861bd640 1 -- 172.21.15.29:0/190527243 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7580150050 con 0x7f75801510f0 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f7577fff640 1 -- 172.21.15.29:0/190527243 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756800ce00 con 0x7f75801510f0 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f75861bd640 1 -- 172.21.15.29:0/190527243 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7580150610 con 0x7f75801510f0 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f75861bd640 1 -- 172.21.15.29:0/190527243 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7580069270 con 0x7f75801510f0 2024-04-25T00:34:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f7577fff640 1 -- 172.21.15.29:0/190527243 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f7568003980 con 0x7f75801510f0 2024-04-25T00:34:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f7577fff640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7560067aa0 0x7f7560069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f7577fff640 1 -- 172.21.15.29:0/190527243 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f7568086130 con 0x7f75801510f0 2024-04-25T00:34:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.333+0000 7f757f7fe640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7560067aa0 0x7f7560069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.337+0000 7f757f7fe640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7560067aa0 0x7f7560069f60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f75700af000 tx=0x7f75700046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.337+0000 7f7577fff640 1 -- 172.21.15.29:0/190527243 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7568003c70 con 0x7f75801510f0 2024-04-25T00:34:52.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.497+0000 7f75861bd640 1 -- 172.21.15.29:0/190527243 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7580075340 con 0x7f75801510f0 2024-04-25T00:34:52.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.501+0000 7f7577fff640 1 -- 172.21.15.29:0/190527243 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16657 (secure 0 0 0) 0x7f7568056a40 con 0x7f75801510f0 2024-04-25T00:34:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:52.503 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":47,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:34:18.731770+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:33:10.789862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:33:12.129841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.36","mappings":[{"from":0,"to":4}]},{"pgid":"7.38","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:34:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.501+0000 7f75861bd640 1 -- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7560067aa0 msgr2=0x7f7560069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.501+0000 7f75861bd640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7560067aa0 0x7f7560069f60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f75700af000 tx=0x7f75700046d0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.501+0000 7f75861bd640 1 -- 172.21.15.29:0/190527243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75801510f0 msgr2=0x7f75800d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.501+0000 7f75861bd640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75801510f0 0x7f75800d02c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f756800eb50 tx=0x7f756800c4b0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.505+0000 7f75861bd640 1 -- 172.21.15.29:0/190527243 shutdown_connections 2024-04-25T00:34:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.505+0000 7f75861bd640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758015ea50 0x7f75800b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.505+0000 7f75861bd640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75801510f0 0x7f75800d02c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.505+0000 7f75861bd640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7560067aa0 0x7f7560069f60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.505+0000 7f75861bd640 1 --2- 172.21.15.29:0/190527243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756c003b80 0x7f7580151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.505+0000 7f75861bd640 1 -- 172.21.15.29:0/190527243 >> 172.21.15.29:0/190527243 conn(0x7f75800d0c10 msgr2=0x7f75800cb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.505+0000 7f75861bd640 1 -- 172.21.15.29:0/190527243 shutdown_connections 2024-04-25T00:34:52.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.505+0000 7f75861bd640 1 -- 172.21.15.29:0/190527243 wait complete. 2024-04-25T00:34:52.521 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:33:10.789862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:33:12.129841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:34:52.521 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:34:52.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.637+0000 7f8d7f9d3640 1 Processor -- start 2024-04-25T00:34:52.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.637+0000 7f8d7f9d3640 1 -- start start 2024-04-25T00:34:52.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.637+0000 7f8d7f9d3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8d780d2c60 0x7f8d78168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.637+0000 7f8d7f9d3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d780cfc10 con 0x7f8d7815e200 2024-04-25T00:34:52.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.637+0000 7f8d7f9d3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d780cfef0 con 0x7f8d780cf690 2024-04-25T00:34:52.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.637+0000 7f8d7f9d3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8d780b38d0 con 0x7f8d780d2c60 2024-04-25T00:34:52.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.637+0000 7f8d7df70640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8d780d2c60 0x7f8d78168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.637+0000 7f8d7d76f640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8d7815e200 0x7f8d780d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36014/0 (socket says 172.21.15.29:36014) 2024-04-25T00:34:52.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.637+0000 7f8d7d76f640 1 -- 172.21.15.29:0/2166609134 learned_addr learned my addr 172.21.15.29:0/2166609134 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:52.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.637+0000 7f8d7df70640 1 -- 172.21.15.29:0/2166609134 >> v1:172.21.15.157:6789/0 conn(0x7f8d780cf690 legacy=0x7f8d780d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.637+0000 7f8d7df70640 1 -- 172.21.15.29:0/2166609134 >> v1:172.21.15.29:6789/0 conn(0x7f8d7815e200 legacy=0x7f8d780d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.637+0000 7f8d7df70640 1 -- 172.21.15.29:0/2166609134 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d780b3b20 con 0x7f8d780d2c60 2024-04-25T00:34:52.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.637+0000 7f8d6ffff640 1 -- 172.21.15.29:0/2166609134 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3960088748 0 0) 0x7f8d780cfc10 con 0x7f8d7815e200 2024-04-25T00:34:52.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7df70640 1 --2- 172.21.15.29:0/2166609134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d780d2c60 0x7f8d78168e20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009d90 tx=0x7f8d7001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e46897238ed6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d6ffff640 1 -- 172.21.15.29:0/2166609134 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d700b2070 con 0x7f8d780d2c60 2024-04-25T00:34:52.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d6ffff640 1 -- 172.21.15.29:0/2166609134 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d700a5850 con 0x7f8d780d2c60 2024-04-25T00:34:52.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d6ffff640 1 -- 172.21.15.29:0/2166609134 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d700a5c20 con 0x7f8d780d2c60 2024-04-25T00:34:52.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/2166609134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d780d2c60 msgr2=0x7f8d78168e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 --2- 172.21.15.29:0/2166609134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d780d2c60 0x7f8d78168e20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009d90 tx=0x7f8d7001c970 comp rx=0 tx=0).stop 2024-04-25T00:34:52.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/2166609134 shutdown_connections 2024-04-25T00:34:52.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 --2- 172.21.15.29:0/2166609134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d780d2c60 0x7f8d78168e20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/2166609134 >> 172.21.15.29:0/2166609134 conn(0x7f8d7809d280 msgr2=0x7f8d780d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:52.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/2166609134 shutdown_connections 2024-04-25T00:34:52.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/2166609134 wait complete. 2024-04-25T00:34:52.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 Processor -- start 2024-04-25T00:34:52.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 -- start start 2024-04-25T00:34:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d780cf690 0x7f8d780b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d780d2c60 0x7f8d7814c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d7815e200 0x7f8d7815fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7816bf10 con 0x7f8d780cf690 2024-04-25T00:34:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7816bd90 con 0x7f8d780d2c60 2024-04-25T00:34:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7816c090 con 0x7f8d7815e200 2024-04-25T00:34:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7d76f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d780cf690 0x7f8d780b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7d76f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d780cf690 0x7f8d780b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40274/0 (socket says 172.21.15.29:40274) 2024-04-25T00:34:52.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7d76f640 1 -- 172.21.15.29:0/1452877782 learned_addr learned my addr 172.21.15.29:0/1452877782 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7cf6e640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d780d2c60 0x7f8d7814c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7df70640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d7815e200 0x7f8d7815fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7d76f640 1 -- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d7815e200 msgr2=0x7f8d7815fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7d76f640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d7815e200 0x7f8d7815fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7d76f640 1 -- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d780d2c60 msgr2=0x7f8d7814c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7d76f640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d780d2c60 0x7f8d7814c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7d76f640 1 -- 172.21.15.29:0/1452877782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7815eb30 con 0x7f8d780cf690 2024-04-25T00:34:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7cf6e640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d780d2c60 0x7f8d7814c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7d76f640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d780cf690 0x7f8d780b6050 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400c4a0 tx=0x7f8d7400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d6dffb640 1 -- 172.21.15.29:0/1452877782 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74018070 con 0x7f8d780cf690 2024-04-25T00:34:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d6dffb640 1 -- 172.21.15.29:0/1452877782 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d740074e0 con 0x7f8d780cf690 2024-04-25T00:34:52.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d6dffb640 1 -- 172.21.15.29:0/1452877782 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74004a30 con 0x7f8d780cf690 2024-04-25T00:34:52.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.641+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/1452877782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7815ee10 con 0x7f8d780cf690 2024-04-25T00:34:52.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.645+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/1452877782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7815f3d0 con 0x7f8d780cf690 2024-04-25T00:34:52.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.645+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/1452877782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d780b9e60 con 0x7f8d780cf690 2024-04-25T00:34:52.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.645+0000 7f8d6dffb640 1 -- 172.21.15.29:0/1452877782 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f8d7401b070 con 0x7f8d780cf690 2024-04-25T00:34:52.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.645+0000 7f8d6dffb640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8d5c067a70 0x7f8d5c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.645+0000 7f8d6dffb640 1 -- 172.21.15.29:0/1452877782 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f8d740853b0 con 0x7f8d780cf690 2024-04-25T00:34:52.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.645+0000 7f8d7cf6e640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8d5c067a70 0x7f8d5c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.645+0000 7f8d7cf6e640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8d5c067a70 0x7f8d5c069f30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8d780b9e10 tx=0x7f8d68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.649+0000 7f8d6dffb640 1 -- 172.21.15.29:0/1452877782 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8d74055b90 con 0x7f8d780cf690 2024-04-25T00:34:52.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.805+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/1452877782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8d78069270 con 0x7f8d780cf690 2024-04-25T00:34:52.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.805+0000 7f8d6dffb640 1 -- 172.21.15.29:0/1452877782 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7f8d74055490 con 0x7f8d780cf690 2024-04-25T00:34:52.807 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:34:52.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.809+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8d5c067a70 msgr2=0x7f8d5c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.809+0000 7f8d7f9d3640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8d5c067a70 0x7f8d5c069f30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8d780b9e10 tx=0x7f8d68009040 comp rx=0 tx=0).stop 2024-04-25T00:34:52.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.809+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/1452877782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d780cf690 msgr2=0x7f8d780b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.809+0000 7f8d7f9d3640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d780cf690 0x7f8d780b6050 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400c4a0 tx=0x7f8d7400c960 comp rx=0 tx=0).stop 2024-04-25T00:34:52.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.809+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/1452877782 shutdown_connections 2024-04-25T00:34:52.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.809+0000 7f8d7f9d3640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d7815e200 0x7f8d7815fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.809+0000 7f8d7f9d3640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8d5c067a70 0x7f8d5c069f30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.809+0000 7f8d7f9d3640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d780d2c60 0x7f8d7814c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.809+0000 7f8d7f9d3640 1 --2- 172.21.15.29:0/1452877782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d780cf690 0x7f8d780b6050 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.809+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/1452877782 >> 172.21.15.29:0/1452877782 conn(0x7f8d7809d280 msgr2=0x7f8d78156430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:52.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.809+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/1452877782 shutdown_connections 2024-04-25T00:34:52.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.809+0000 7f8d7f9d3640 1 -- 172.21.15.29:0/1452877782 wait complete. 2024-04-25T00:34:52.824 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:34:52.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.941+0000 7fc5247d6640 1 Processor -- start 2024-04-25T00:34:52.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.941+0000 7fc5247d6640 1 -- start start 2024-04-25T00:34:52.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.941+0000 7fc5247d6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc51c0cf690 0x7fc51c0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.941+0000 7fc5247d6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0d2a90 0x7fc51c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.941+0000 7fc5247d6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc51c0cfc10 con 0x7fc51c15e200 2024-04-25T00:34:52.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.941+0000 7fc5247d6640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc51c0d2ee0 con 0x7fc51c0cf690 2024-04-25T00:34:52.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc5247d6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc51c0b38d0 con 0x7fc51c0d2a90 2024-04-25T00:34:52.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc521d71640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc51c0cf690 0x7fc51c0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc521d71640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc51c0cf690 0x7fc51c0d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40290/0 (socket says 172.21.15.29:40290) 2024-04-25T00:34:52.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc521d71640 1 -- 172.21.15.29:0/1403856813 learned_addr learned my addr 172.21.15.29:0/1403856813 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:52.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc522d73640 1 --2- 172.21.15.29:0/1403856813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0d2a90 0x7fc51c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc521d71640 1 -- 172.21.15.29:0/1403856813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0d2a90 msgr2=0x7fc51c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:52.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc521d71640 1 --2- 172.21.15.29:0/1403856813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0d2a90 0x7fc51c0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc521d71640 1 -- 172.21.15.29:0/1403856813 >> v1:172.21.15.29:6789/0 conn(0x7fc51c15e200 legacy=0x7fc51c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc521570640 1 -- 172.21.15.29:0/1403856813 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3618734751 0 0) 0x7fc51c0cfc10 con 0x7fc51c15e200 2024-04-25T00:34:52.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc521d71640 1 -- 172.21.15.29:0/1403856813 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c0cfef0 con 0x7fc51c0cf690 2024-04-25T00:34:52.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc521d71640 1 --2- 172.21.15.29:0/1403856813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0cf690 0x7fc51c0d2550 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009870 tx=0x7fc50c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735ef2d8efb8fc50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc521570640 1 -- 172.21.15.29:0/1403856813 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0ab650 con 0x7fc51c0cf690 2024-04-25T00:34:52.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc521570640 1 -- 172.21.15.29:0/1403856813 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc50c0b0070 con 0x7fc51c0cf690 2024-04-25T00:34:52.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc521570640 1 -- 172.21.15.29:0/1403856813 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0aba10 con 0x7fc51c0cf690 2024-04-25T00:34:52.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc5247d6640 1 -- 172.21.15.29:0/1403856813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0cf690 msgr2=0x7fc51c0d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc5247d6640 1 --2- 172.21.15.29:0/1403856813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0cf690 0x7fc51c0d2550 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009870 tx=0x7fc50c01c560 comp rx=0 tx=0).stop 2024-04-25T00:34:52.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc5247d6640 1 -- 172.21.15.29:0/1403856813 shutdown_connections 2024-04-25T00:34:52.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc5247d6640 1 --2- 172.21.15.29:0/1403856813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0d2a90 0x7fc51c0b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc5247d6640 1 --2- 172.21.15.29:0/1403856813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c0cf690 0x7fc51c0d2550 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc5247d6640 1 -- 172.21.15.29:0/1403856813 >> 172.21.15.29:0/1403856813 conn(0x7fc51c09d280 msgr2=0x7fc51c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:52.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc5247d6640 1 -- 172.21.15.29:0/1403856813 shutdown_connections 2024-04-25T00:34:52.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.945+0000 7fc5247d6640 1 -- 172.21.15.29:0/1403856813 wait complete. 2024-04-25T00:34:52.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc5247d6640 1 Processor -- start 2024-04-25T00:34:52.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc5247d6640 1 -- start start 2024-04-25T00:34:52.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc5247d6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0cf690 0x7fc51c0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc5247d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51c0d2a90 0x7fc51c1f9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc5247d6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c15e200 0x7fc51c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc5247d6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c0b51f0 con 0x7fc51c15e200 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc5247d6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c0b5070 con 0x7fc51c0d2a90 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc5247d6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc51c0b5370 con 0x7fc51c0cf690 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc522d73640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c15e200 0x7fc51c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc522d73640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c15e200 0x7fc51c0cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40292/0 (socket says 172.21.15.29:40292) 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc522d73640 1 -- 172.21.15.29:0/4228235488 learned_addr learned my addr 172.21.15.29:0/4228235488 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc521d71640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51c0d2a90 0x7fc51c1f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc522572640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0cf690 0x7fc51c0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc522d73640 1 -- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0cf690 msgr2=0x7fc51c0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc522d73640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0cf690 0x7fc51c0cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc522d73640 1 -- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51c0d2a90 msgr2=0x7fc51c1f9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc522d73640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51c0d2a90 0x7fc51c1f9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:52.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc522d73640 1 -- 172.21.15.29:0/4228235488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c0b7600 con 0x7fc51c15e200 2024-04-25T00:34:52.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc521d71640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51c0d2a90 0x7fc51c1f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:52.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc522d73640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c15e200 0x7fc51c0cb040 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc51400cd70 tx=0x7fc51400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc512ffd640 1 -- 172.21.15.29:0/4228235488 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514003ba0 con 0x7fc51c15e200 2024-04-25T00:34:52.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc512ffd640 1 -- 172.21.15.29:0/4228235488 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc514003d40 con 0x7fc51c15e200 2024-04-25T00:34:52.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc5247d6640 1 -- 172.21.15.29:0/4228235488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51c0b78e0 con 0x7fc51c15e200 2024-04-25T00:34:52.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc512ffd640 1 -- 172.21.15.29:0/4228235488 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514004240 con 0x7fc51c15e200 2024-04-25T00:34:52.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc5247d6640 1 -- 172.21.15.29:0/4228235488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51c0ce410 con 0x7fc51c15e200 2024-04-25T00:34:52.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc5247d6640 1 -- 172.21.15.29:0/4228235488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc51c069270 con 0x7fc51c15e200 2024-04-25T00:34:52.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.949+0000 7fc512ffd640 1 -- 172.21.15.29:0/4228235488 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fc5140043e0 con 0x7fc51c15e200 2024-04-25T00:34:52.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.953+0000 7fc512ffd640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc500067a70 0x7fc500069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:52.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.953+0000 7fc512ffd640 1 -- 172.21.15.29:0/4228235488 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fc514086bd0 con 0x7fc51c15e200 2024-04-25T00:34:52.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.953+0000 7fc522572640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc500067a70 0x7fc500069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:52.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.953+0000 7fc522572640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc500067a70 0x7fc500069f30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc518000f70 tx=0x7fc518009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:52.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:52.957+0000 7fc512ffd640 1 -- 172.21.15.29:0/4228235488 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc5140573d0 con 0x7fc51c15e200 2024-04-25T00:34:53.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.117+0000 7fc5247d6640 1 -- 172.21.15.29:0/4228235488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc51c15a040 con 0x7fc51c15e200 2024-04-25T00:34:53.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.117+0000 7fc512ffd640 1 -- 172.21.15.29:0/4228235488 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7fc514056cd0 con 0x7fc51c15e200 2024-04-25T00:34:53.119 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:34:53.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.121+0000 7fc5247d6640 1 -- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc500067a70 msgr2=0x7fc500069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.121+0000 7fc5247d6640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc500067a70 0x7fc500069f30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc518000f70 tx=0x7fc518009040 comp rx=0 tx=0).stop 2024-04-25T00:34:53.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.121+0000 7fc5247d6640 1 -- 172.21.15.29:0/4228235488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c15e200 msgr2=0x7fc51c0cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.121+0000 7fc5247d6640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c15e200 0x7fc51c0cb040 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc51400cd70 tx=0x7fc51400a660 comp rx=0 tx=0).stop 2024-04-25T00:34:53.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.121+0000 7fc5247d6640 1 -- 172.21.15.29:0/4228235488 shutdown_connections 2024-04-25T00:34:53.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.121+0000 7fc5247d6640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51c15e200 0x7fc51c0cb040 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.121+0000 7fc5247d6640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc500067a70 0x7fc500069f30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.121+0000 7fc5247d6640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51c0d2a90 0x7fc51c1f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.121+0000 7fc5247d6640 1 --2- 172.21.15.29:0/4228235488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc51c0cf690 0x7fc51c0cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.121+0000 7fc5247d6640 1 -- 172.21.15.29:0/4228235488 >> 172.21.15.29:0/4228235488 conn(0x7fc51c09d280 msgr2=0x7fc51c0d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:53.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.121+0000 7fc5247d6640 1 -- 172.21.15.29:0/4228235488 shutdown_connections 2024-04-25T00:34:53.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.121+0000 7fc5247d6640 1 -- 172.21.15.29:0/4228235488 wait complete. 2024-04-25T00:34:53.138 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:34:53.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad2891f640 1 Processor -- start 2024-04-25T00:34:53.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad2891f640 1 -- start start 2024-04-25T00:34:53.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad2891f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2415e8d0 0x7fad240cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad2891f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fad2415f3c0 0x7fad2415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad2891f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad240b3720 con 0x7fad240b4780 2024-04-25T00:34:53.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad2891f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fad240b42f0 con 0x7fad2415f3c0 2024-04-25T00:34:53.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad2891f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fad240b39c0 con 0x7fad2415e8d0 2024-04-25T00:34:53.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad22d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fad2415f3c0 0x7fad2415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad1bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fad240b4780 0x7fad2415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:39210/0 (socket says 172.21.15.29:39210) 2024-04-25T00:34:53.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad22d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fad2415f3c0 0x7fad2415c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:33562/0 (socket says 172.21.15.29:33562) 2024-04-25T00:34:53.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad1bfff640 1 -- 172.21.15.29:0/4169867665 learned_addr learned my addr 172.21.15.29:0/4169867665 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:53.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad2259c640 1 --2- 172.21.15.29:0/4169867665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2415e8d0 0x7fad240cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad22d9d640 1 -- 172.21.15.29:0/4169867665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2415e8d0 msgr2=0x7fad240cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:53.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad22d9d640 1 --2- 172.21.15.29:0/4169867665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2415e8d0 0x7fad240cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad22d9d640 1 -- 172.21.15.29:0/4169867665 >> v1:172.21.15.157:6789/0 conn(0x7fad240b4780 legacy=0x7fad2415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.257+0000 7fad22d9d640 1 -- 172.21.15.29:0/4169867665 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad240b3ca0 con 0x7fad2415f3c0 2024-04-25T00:34:53.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.261+0000 7fad22d9d640 1 --2- 172.21.15.29:0/4169867665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2415f3c0 0x7fad2415c7d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00bfa0 tx=0x7fad1c01eb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96b6fb5833855739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:53.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.261+0000 7fad21d9b640 1 -- 172.21.15.29:0/4169867665 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c00b2d0 con 0x7fad2415f3c0 2024-04-25T00:34:53.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad21d9b640 1 -- 172.21.15.29:0/4169867665 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad1c0a79f0 con 0x7fad2415f3c0 2024-04-25T00:34:53.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad21d9b640 1 -- 172.21.15.29:0/4169867665 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c0a7dc0 con 0x7fad2415f3c0 2024-04-25T00:34:53.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 -- 172.21.15.29:0/4169867665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2415f3c0 msgr2=0x7fad2415c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 --2- 172.21.15.29:0/4169867665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2415f3c0 0x7fad2415c7d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00bfa0 tx=0x7fad1c01eb40 comp rx=0 tx=0).stop 2024-04-25T00:34:53.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 -- 172.21.15.29:0/4169867665 shutdown_connections 2024-04-25T00:34:53.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 --2- 172.21.15.29:0/4169867665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2415f3c0 0x7fad2415c7d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 --2- 172.21.15.29:0/4169867665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2415e8d0 0x7fad240cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 -- 172.21.15.29:0/4169867665 >> 172.21.15.29:0/4169867665 conn(0x7fad240b1870 msgr2=0x7fad240b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:53.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 -- 172.21.15.29:0/4169867665 shutdown_connections 2024-04-25T00:34:53.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 -- 172.21.15.29:0/4169867665 wait complete. 2024-04-25T00:34:53.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 Processor -- start 2024-04-25T00:34:53.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 -- start start 2024-04-25T00:34:53.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad240b4780 0x7fad241383f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2415e8d0 0x7fad24154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2415f3c0 0x7fad24157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fad2415d6a0 con 0x7fad240b4780 2024-04-25T00:34:53.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad2415d520 con 0x7fad2415e8d0 2024-04-25T00:34:53.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2891f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fad2406d590 con 0x7fad2415f3c0 2024-04-25T00:34:53.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad240b4780 0x7fad241383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad240b4780 0x7fad241383f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40298/0 (socket says 172.21.15.29:40298) 2024-04-25T00:34:53.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2259c640 1 -- 172.21.15.29:0/1712921849 learned_addr learned my addr 172.21.15.29:0/1712921849 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:53.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad22d9d640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2415f3c0 0x7fad24157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad1bfff640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2415e8d0 0x7fad24154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2259c640 1 -- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2415f3c0 msgr2=0x7fad24157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2259c640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2415f3c0 0x7fad24157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2259c640 1 -- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2415e8d0 msgr2=0x7fad24154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2259c640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2415e8d0 0x7fad24154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad2259c640 1 -- 172.21.15.29:0/1712921849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad240afe70 con 0x7fad240b4780 2024-04-25T00:34:53.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.265+0000 7fad1bfff640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2415e8d0 0x7fad24154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:53.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.269+0000 7fad2259c640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad240b4780 0x7fad241383f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fad1400e930 tx=0x7fad1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:53.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.269+0000 7fad1affd640 1 -- 172.21.15.29:0/1712921849 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14017070 con 0x7fad240b4780 2024-04-25T00:34:53.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.269+0000 7fad1affd640 1 -- 172.21.15.29:0/1712921849 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad1400ca00 con 0x7fad240b4780 2024-04-25T00:34:53.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.269+0000 7fad1affd640 1 -- 172.21.15.29:0/1712921849 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1400cdb0 con 0x7fad240b4780 2024-04-25T00:34:53.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.269+0000 7fad2891f640 1 -- 172.21.15.29:0/1712921849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad24155d70 con 0x7fad240b4780 2024-04-25T00:34:53.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.269+0000 7fad2891f640 1 -- 172.21.15.29:0/1712921849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad24156610 con 0x7fad240b4780 2024-04-25T00:34:53.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.269+0000 7fad2891f640 1 -- 172.21.15.29:0/1712921849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad24069270 con 0x7fad240b4780 2024-04-25T00:34:53.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.269+0000 7fad1affd640 1 -- 172.21.15.29:0/1712921849 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fad1401a070 con 0x7fad240b4780 2024-04-25T00:34:53.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.269+0000 7fad1affd640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fad04067a70 0x7fad04069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.269+0000 7fad1affd640 1 -- 172.21.15.29:0/1712921849 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fad14086160 con 0x7fad240b4780 2024-04-25T00:34:53.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.269+0000 7fad1bfff640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fad04067a70 0x7fad04069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.269+0000 7fad1bfff640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fad04067a70 0x7fad04069f30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fad2406dbe0 tx=0x7fad0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:53.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.273+0000 7fad1affd640 1 -- 172.21.15.29:0/1712921849 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fad14056980 con 0x7fad240b4780 2024-04-25T00:34:53.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.429+0000 7fad2891f640 1 -- 172.21.15.29:0/1712921849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad2406e7b0 con 0x7fad240b4780 2024-04-25T00:34:53.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.429+0000 7fad1affd640 1 -- 172.21.15.29:0/1712921849 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7fad14056280 con 0x7fad240b4780 2024-04-25T00:34:53.431 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:34:53.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.433+0000 7fad2891f640 1 -- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fad04067a70 msgr2=0x7fad04069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.433+0000 7fad2891f640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fad04067a70 0x7fad04069f30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fad2406dbe0 tx=0x7fad0c009040 comp rx=0 tx=0).stop 2024-04-25T00:34:53.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.433+0000 7fad2891f640 1 -- 172.21.15.29:0/1712921849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad240b4780 msgr2=0x7fad241383f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.433+0000 7fad2891f640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad240b4780 0x7fad241383f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fad1400e930 tx=0x7fad1400edf0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.433+0000 7fad2891f640 1 -- 172.21.15.29:0/1712921849 shutdown_connections 2024-04-25T00:34:53.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.433+0000 7fad2891f640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2415f3c0 0x7fad24157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.433+0000 7fad2891f640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fad04067a70 0x7fad04069f30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.433+0000 7fad2891f640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2415e8d0 0x7fad24154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.433+0000 7fad2891f640 1 --2- 172.21.15.29:0/1712921849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad240b4780 0x7fad241383f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.433+0000 7fad2891f640 1 -- 172.21.15.29:0/1712921849 >> 172.21.15.29:0/1712921849 conn(0x7fad240b1870 msgr2=0x7fad2415b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:53.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.433+0000 7fad2891f640 1 -- 172.21.15.29:0/1712921849 shutdown_connections 2024-04-25T00:34:53.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.433+0000 7fad2891f640 1 -- 172.21.15.29:0/1712921849 wait complete. 2024-04-25T00:34:53.449 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.557+0000 7fc0dc2e1640 1 Processor -- start 2024-04-25T00:34:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.557+0000 7fc0dc2e1640 1 -- start start 2024-04-25T00:34:53.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0dc2e1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc0d40b3140 0x7fc0d406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0dc2e1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0d40b3c30 0x7fc0d40ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0dc2e1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0d40b41e0 con 0x7fc0d40b4b10 2024-04-25T00:34:53.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0dc2e1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc0d406d960 con 0x7fc0d40b3140 2024-04-25T00:34:53.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0dc2e1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d406dc40 con 0x7fc0d40b3c30 2024-04-25T00:34:53.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0da07d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc0d40b3140 0x7fc0d406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0da07d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc0d40b3140 0x7fc0d406e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40306/0 (socket says 172.21.15.29:40306) 2024-04-25T00:34:53.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0da07d640 1 -- 172.21.15.29:0/893854673 learned_addr learned my addr 172.21.15.29:0/893854673 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:53.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0da07d640 1 -- 172.21.15.29:0/893854673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0d40b3c30 msgr2=0x7fc0d40ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:53.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0da07d640 1 --2- 172.21.15.29:0/893854673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0d40b3c30 0x7fc0d40ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0da07d640 1 -- 172.21.15.29:0/893854673 >> v1:172.21.15.157:6789/0 conn(0x7fc0d40b4b10 legacy=0x7fc0d40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:53.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0da07d640 1 -- 172.21.15.29:0/893854673 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d406df20 con 0x7fc0d40b3140 2024-04-25T00:34:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0da07d640 1 --2- 172.21.15.29:0/893854673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0d40b3140 0x7fc0d406e440 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009e80 tx=0x7fc0d001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e50f41ed06ed2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0d907b640 1 -- 172.21.15.29:0/893854673 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d00a5650 con 0x7fc0d40b3140 2024-04-25T00:34:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0d907b640 1 -- 172.21.15.29:0/893854673 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d00a57f0 con 0x7fc0d40b3140 2024-04-25T00:34:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0d907b640 1 -- 172.21.15.29:0/893854673 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d00a5bc0 con 0x7fc0d40b3140 2024-04-25T00:34:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/893854673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0d40b3140 msgr2=0x7fc0d406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0dc2e1640 1 --2- 172.21.15.29:0/893854673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0d40b3140 0x7fc0d406e440 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009e80 tx=0x7fc0d001c9d0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/893854673 shutdown_connections 2024-04-25T00:34:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0dc2e1640 1 --2- 172.21.15.29:0/893854673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0d40b3c30 0x7fc0d40ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0dc2e1640 1 --2- 172.21.15.29:0/893854673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0d40b3140 0x7fc0d406e440 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/893854673 >> 172.21.15.29:0/893854673 conn(0x7fc0d40b1870 msgr2=0x7fc0d40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/893854673 shutdown_connections 2024-04-25T00:34:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.561+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/893854673 wait complete. 2024-04-25T00:34:53.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0dc2e1640 1 Processor -- start 2024-04-25T00:34:53.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0dc2e1640 1 -- start start 2024-04-25T00:34:53.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0dc2e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0d40b3140 0x7fc0d41380b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0dc2e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0d40b3c30 0x7fc0d41f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0dc2e1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0d40b4b10 0x7fc0d4155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0dc2e1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4168d80 con 0x7fc0d40b3c30 2024-04-25T00:34:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0dc2e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4168c00 con 0x7fc0d40b3140 2024-04-25T00:34:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0dc2e1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d4168f00 con 0x7fc0d40b4b10 2024-04-25T00:34:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0d987c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0d40b3c30 0x7fc0d41f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0d987c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0d40b3c30 0x7fc0d41f81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40316/0 (socket says 172.21.15.29:40316) 2024-04-25T00:34:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0d987c640 1 -- 172.21.15.29:0/1360177137 learned_addr learned my addr 172.21.15.29:0/1360177137 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0da07d640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0d40b3140 0x7fc0d41380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0da87e640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0d40b4b10 0x7fc0d4155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0d987c640 1 -- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0d40b4b10 msgr2=0x7fc0d4155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0d987c640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0d40b4b10 0x7fc0d4155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0d987c640 1 -- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0d40b3140 msgr2=0x7fc0d41380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0d987c640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0d40b3140 0x7fc0d41380b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0d987c640 1 -- 172.21.15.29:0/1360177137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d4156330 con 0x7fc0d40b3c30 2024-04-25T00:34:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0d987c640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0d40b3c30 0x7fc0d41f81e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400c490 tx=0x7fc0c4001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0caffd640 1 -- 172.21.15.29:0/1360177137 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4017070 con 0x7fc0d40b3c30 2024-04-25T00:34:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0caffd640 1 -- 172.21.15.29:0/1360177137 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c4003600 con 0x7fc0d40b3c30 2024-04-25T00:34:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/1360177137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d4156610 con 0x7fc0d40b3c30 2024-04-25T00:34:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0caffd640 1 -- 172.21.15.29:0/1360177137 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4003a30 con 0x7fc0d40b3c30 2024-04-25T00:34:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/1360177137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d4156bd0 con 0x7fc0d40b3c30 2024-04-25T00:34:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.565+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/1360177137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d4069270 con 0x7fc0d40b3c30 2024-04-25T00:34:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.569+0000 7fc0caffd640 1 -- 172.21.15.29:0/1360177137 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fc0c401a070 con 0x7fc0d40b3c30 2024-04-25T00:34:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.569+0000 7fc0caffd640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc0b8067a70 0x7fc0b8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.569+0000 7fc0caffd640 1 -- 172.21.15.29:0/1360177137 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fc0c4085440 con 0x7fc0d40b3c30 2024-04-25T00:34:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.569+0000 7fc0da07d640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc0b8067a70 0x7fc0b8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.569+0000 7fc0da07d640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc0b8067a70 0x7fc0b8069f30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00ae000 tx=0x7fc0d00a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:53.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.573+0000 7fc0caffd640 1 -- 172.21.15.29:0/1360177137 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc0c4055cc0 con 0x7fc0d40b3c30 2024-04-25T00:34:53.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.761+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/1360177137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0d40cd860 con 0x7fc0d40b3c30 2024-04-25T00:34:53.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.761+0000 7fc0caffd640 1 -- 172.21.15.29:0/1360177137 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7fc0c40555c0 con 0x7fc0d40b3c30 2024-04-25T00:34:53.763 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:53.763 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:53.763 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:53.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.761+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc0b8067a70 msgr2=0x7fc0b8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.761+0000 7fc0dc2e1640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc0b8067a70 0x7fc0b8069f30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00ae000 tx=0x7fc0d00a7070 comp rx=0 tx=0).stop 2024-04-25T00:34:53.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.761+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/1360177137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0d40b3c30 msgr2=0x7fc0d41f81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.761+0000 7fc0dc2e1640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0d40b3c30 0x7fc0d41f81e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400c490 tx=0x7fc0c4001e80 comp rx=0 tx=0).stop 2024-04-25T00:34:53.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.765+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/1360177137 shutdown_connections 2024-04-25T00:34:53.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.765+0000 7fc0dc2e1640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0d40b4b10 0x7fc0d4155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.765+0000 7fc0dc2e1640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0d40b3c30 0x7fc0d41f81e0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.765+0000 7fc0dc2e1640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc0b8067a70 0x7fc0b8069f30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.765+0000 7fc0dc2e1640 1 --2- 172.21.15.29:0/1360177137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0d40b3140 0x7fc0d41380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.765+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/1360177137 >> 172.21.15.29:0/1360177137 conn(0x7fc0d40b1870 msgr2=0x7fc0d415b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:53.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.765+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/1360177137 shutdown_connections 2024-04-25T00:34:53.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.765+0000 7fc0dc2e1640 1 -- 172.21.15.29:0/1360177137 wait complete. 2024-04-25T00:34:53.781 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:34:53.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d6ed4640 1 Processor -- start 2024-04-25T00:34:53.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d6ed4640 1 -- start start 2024-04-25T00:34:53.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d6ed4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27d015e200 0x7f27d00d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d6ed4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d00cf690 0x7f27d00d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d6ed4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27d00cfc10 con 0x7f27d00d2a90 2024-04-25T00:34:53.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d6ed4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f27d00d2ee0 con 0x7f27d015e200 2024-04-25T00:34:53.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d6ed4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f27d00b38d0 con 0x7f27d00cf690 2024-04-25T00:34:53.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d4c70640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27d015e200 0x7f27d00d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d5471640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f27d00d2a90 0x7f27d00d05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36044/0 (socket says 172.21.15.29:36044) 2024-04-25T00:34:53.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d5471640 1 -- 172.21.15.29:0/2837661662 learned_addr learned my addr 172.21.15.29:0/2837661662 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:53.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27cffff640 1 --2- 172.21.15.29:0/2837661662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d00cf690 0x7f27d00d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d4c70640 1 -- 172.21.15.29:0/2837661662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d00cf690 msgr2=0x7f27d00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:53.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d4c70640 1 --2- 172.21.15.29:0/2837661662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d00cf690 0x7f27d00d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d4c70640 1 -- 172.21.15.29:0/2837661662 >> v1:172.21.15.29:6789/0 conn(0x7f27d00d2a90 legacy=0x7f27d00d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d4c70640 1 -- 172.21.15.29:0/2837661662 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d00cfef0 con 0x7f27d015e200 2024-04-25T00:34:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.901+0000 7f27d4c70640 1 --2- 172.21.15.29:0/2837661662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27d015e200 0x7f27d00d47f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009d90 tx=0x7f27c001c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04010b5db150aa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cf7fe640 1 -- 172.21.15.29:0/2837661662 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c00b2070 con 0x7f27d015e200 2024-04-25T00:34:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cf7fe640 1 -- 172.21.15.29:0/2837661662 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27c00a5820 con 0x7f27d015e200 2024-04-25T00:34:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cf7fe640 1 -- 172.21.15.29:0/2837661662 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c00a5bf0 con 0x7f27d015e200 2024-04-25T00:34:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2837661662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27d015e200 msgr2=0x7f27d00d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 --2- 172.21.15.29:0/2837661662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27d015e200 0x7f27d00d47f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009d90 tx=0x7f27c001c800 comp rx=0 tx=0).stop 2024-04-25T00:34:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2837661662 shutdown_connections 2024-04-25T00:34:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 --2- 172.21.15.29:0/2837661662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d00cf690 0x7f27d00d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 --2- 172.21.15.29:0/2837661662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27d015e200 0x7f27d00d47f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2837661662 >> 172.21.15.29:0/2837661662 conn(0x7f27d009d280 msgr2=0x7f27d00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2837661662 shutdown_connections 2024-04-25T00:34:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2837661662 wait complete. 2024-04-25T00:34:53.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 Processor -- start 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 -- start start 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d00cf690 0x7f27d00c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27d00d2a90 0x7f27d01f7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d015e200 0x7f27d01f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f27d00d4d30 con 0x7f27d00d2a90 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27d00b3e00 con 0x7f27d00cf690 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f27d015e5e0 con 0x7f27d015e200 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27d00d2a90 0x7f27d01f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27d00d2a90 0x7f27d01f7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40338/0 (socket says 172.21.15.29:40338) 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cffff640 1 -- 172.21.15.29:0/2424911717 learned_addr learned my addr 172.21.15.29:0/2424911717 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d4c70640 1 --2- 172.21.15.29:0/2424911717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d00cf690 0x7f27d00c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cffff640 1 -- 172.21.15.29:0/2424911717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d015e200 msgr2=0x7f27d01f7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cffff640 1 --2- 172.21.15.29:0/2424911717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d015e200 0x7f27d01f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cffff640 1 -- 172.21.15.29:0/2424911717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d00cf690 msgr2=0x7f27d00c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:53.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cffff640 1 --2- 172.21.15.29:0/2424911717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d00cf690 0x7f27d00c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:53.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cffff640 1 -- 172.21.15.29:0/2424911717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d00c7bd0 con 0x7f27d00d2a90 2024-04-25T00:34:53.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cffff640 1 --2- 172.21.15.29:0/2424911717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27d00d2a90 0x7f27d01f7400 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f27c400d620 tx=0x7f27c400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:53.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cd7fa640 1 -- 172.21.15.29:0/2424911717 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c400b620 con 0x7f27d00d2a90 2024-04-25T00:34:53.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cd7fa640 1 -- 172.21.15.29:0/2424911717 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27c4019070 con 0x7f27d00d2a90 2024-04-25T00:34:53.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2424911717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d00c7eb0 con 0x7f27d00d2a90 2024-04-25T00:34:53.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27cd7fa640 1 -- 172.21.15.29:0/2424911717 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c400ba20 con 0x7f27d00d2a90 2024-04-25T00:34:53.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.905+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2424911717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d00c8470 con 0x7f27d00d2a90 2024-04-25T00:34:53.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.909+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2424911717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d0069270 con 0x7f27d00d2a90 2024-04-25T00:34:53.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.909+0000 7f27cd7fa640 1 -- 172.21.15.29:0/2424911717 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f27c400bbc0 con 0x7f27d00d2a90 2024-04-25T00:34:53.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.909+0000 7f27cd7fa640 1 --2- 172.21.15.29:0/2424911717 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f27bc067aa0 0x7f27bc069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:53.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.909+0000 7f27cd7fa640 1 -- 172.21.15.29:0/2424911717 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f27c40863a0 con 0x7f27d00d2a90 2024-04-25T00:34:53.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.909+0000 7f27d4c70640 1 --2- 172.21.15.29:0/2424911717 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f27bc067aa0 0x7f27bc069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:53.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.909+0000 7f27d4c70640 1 --2- 172.21.15.29:0/2424911717 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f27bc067aa0 0x7f27bc069f60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f27c00af000 tx=0x7f27c00a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:53.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:53.913+0000 7f27cd7fa640 1 -- 172.21.15.29:0/2424911717 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f27c4056cb0 con 0x7f27d00d2a90 2024-04-25T00:34:54.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.073+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2424911717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27d00d0fc0 con 0x7f27d00d2a90 2024-04-25T00:34:54.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.077+0000 7f27cd7fa640 1 -- 172.21.15.29:0/2424911717 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16657 (secure 0 0 0) 0x7f27c40565b0 con 0x7f27d00d2a90 2024-04-25T00:34:54.079 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:54.079 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":47,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:34:18.731770+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:33:10.789862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:33:12.129841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.36","mappings":[{"from":0,"to":4}]},{"pgid":"7.38","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:34:54.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.077+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2424911717 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f27bc067aa0 msgr2=0x7f27bc069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:54.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.077+0000 7f27d6ed4640 1 --2- 172.21.15.29:0/2424911717 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f27bc067aa0 0x7f27bc069f60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f27c00af000 tx=0x7f27c00a7070 comp rx=0 tx=0).stop 2024-04-25T00:34:54.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.077+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2424911717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27d00d2a90 msgr2=0x7f27d01f7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:54.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.077+0000 7f27d6ed4640 1 --2- 172.21.15.29:0/2424911717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27d00d2a90 0x7f27d01f7400 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f27c400d620 tx=0x7f27c400dae0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.081+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2424911717 shutdown_connections 2024-04-25T00:34:54.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.081+0000 7f27d6ed4640 1 --2- 172.21.15.29:0/2424911717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27d015e200 0x7f27d01f7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.081+0000 7f27d6ed4640 1 --2- 172.21.15.29:0/2424911717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27d00d2a90 0x7f27d01f7400 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.081+0000 7f27d6ed4640 1 --2- 172.21.15.29:0/2424911717 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f27bc067aa0 0x7f27bc069f60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.081+0000 7f27d6ed4640 1 --2- 172.21.15.29:0/2424911717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d00cf690 0x7f27d00c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.081+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2424911717 >> 172.21.15.29:0/2424911717 conn(0x7f27d009d280 msgr2=0x7f27d00b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:54.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.081+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2424911717 shutdown_connections 2024-04-25T00:34:54.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.081+0000 7f27d6ed4640 1 -- 172.21.15.29:0/2424911717 wait complete. 2024-04-25T00:34:54.098 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:34:54.098 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:54.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.221+0000 7f2a95757640 1 Processor -- start 2024-04-25T00:34:54.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.221+0000 7f2a95757640 1 -- start start 2024-04-25T00:34:54.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a95757640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2a900b3140 0x7f2a9006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:54.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a95757640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2a900b4b10 0x7f2a900b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:54.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a95757640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a900b3a60 0x7f2a9015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:54.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a95757640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2a900b41e0 con 0x7f2a900b4b10 2024-04-25T00:34:54.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a95757640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9006d960 con 0x7f2a900b3140 2024-04-25T00:34:54.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a95757640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9006dc40 con 0x7f2a900b3a60 2024-04-25T00:34:54.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a8e7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2a900b4b10 0x7f2a900b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:54.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a8e7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2a900b4b10 0x7f2a900b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40344/0 (socket says 172.21.15.29:40344) 2024-04-25T00:34:54.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a8e7fc640 1 -- 172.21.15.29:0/1082962278 learned_addr learned my addr 172.21.15.29:0/1082962278 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:54.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a8e7fc640 1 -- 172.21.15.29:0/1082962278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a900b3a60 msgr2=0x7f2a9015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:54.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a8e7fc640 1 --2- 172.21.15.29:0/1082962278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a900b3a60 0x7f2a9015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a8e7fc640 1 -- 172.21.15.29:0/1082962278 >> v2:172.21.15.157:3300/0 conn(0x7f2a900b3140 msgr2=0x7f2a9006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:54.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a8effd640 1 --2- 172.21.15.29:0/1082962278 >> v2:172.21.15.157:3300/0 conn(0x7f2a900b3140 0x7f2a9006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:54.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a8e7fc640 1 --2- 172.21.15.29:0/1082962278 >> v2:172.21.15.157:3300/0 conn(0x7f2a900b3140 0x7f2a9006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a8e7fc640 1 -- 172.21.15.29:0/1082962278 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9006df20 con 0x7f2a900b4b10 2024-04-25T00:34:54.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a8e7fc640 1 --2- 172.21.15.29:0/1082962278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a900b4b10 0x7f2a900b3520 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009870 tx=0x7f2a7801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06a7668b96119fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:54.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a8dffb640 1 -- 172.21.15.29:0/1082962278 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a780b0070 con 0x7f2a900b4b10 2024-04-25T00:34:54.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a8dffb640 1 -- 172.21.15.29:0/1082962278 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a780ab600 con 0x7f2a900b4b10 2024-04-25T00:34:54.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.225+0000 7f2a8dffb640 1 -- 172.21.15.29:0/1082962278 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a780ab9d0 con 0x7f2a900b4b10 2024-04-25T00:34:54.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 -- 172.21.15.29:0/1082962278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a900b4b10 msgr2=0x7f2a900b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:54.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 --2- 172.21.15.29:0/1082962278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a900b4b10 0x7f2a900b3520 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009870 tx=0x7f2a7801c520 comp rx=0 tx=0).stop 2024-04-25T00:34:54.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 -- 172.21.15.29:0/1082962278 shutdown_connections 2024-04-25T00:34:54.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 --2- 172.21.15.29:0/1082962278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a900b4b10 0x7f2a900b3520 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 --2- 172.21.15.29:0/1082962278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a900b3a60 0x7f2a9015c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 --2- 172.21.15.29:0/1082962278 >> v2:172.21.15.157:3300/0 conn(0x7f2a900b3140 0x7f2a9006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 -- 172.21.15.29:0/1082962278 >> 172.21.15.29:0/1082962278 conn(0x7f2a900b1870 msgr2=0x7f2a900b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:54.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 -- 172.21.15.29:0/1082962278 shutdown_connections 2024-04-25T00:34:54.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 -- 172.21.15.29:0/1082962278 wait complete. 2024-04-25T00:34:54.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 Processor -- start 2024-04-25T00:34:54.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 -- start start 2024-04-25T00:34:54.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a900b3140 0x7f2a90136d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:54.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a900b3a60 0x7f2a9013a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:54.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a900b4b10 0x7f2a9013a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:54.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2a900ce3c0 con 0x7f2a900b3140 2024-04-25T00:34:54.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a900ce240 con 0x7f2a900b3a60 2024-04-25T00:34:54.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a95757640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2a900ce540 con 0x7f2a900b4b10 2024-04-25T00:34:54.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a8effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a900b3140 0x7f2a90136d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:54.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a8effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a900b3140 0x7f2a90136d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40352/0 (socket says 172.21.15.29:40352) 2024-04-25T00:34:54.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a8effd640 1 -- 172.21.15.29:0/1463721849 learned_addr learned my addr 172.21.15.29:0/1463721849 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:54.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a8e7fc640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a900b3a60 0x7f2a9013a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:54.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a8effd640 1 -- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a900b4b10 msgr2=0x7f2a9013a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:54.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a8f7fe640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a900b4b10 0x7f2a9013a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:54.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a8effd640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a900b4b10 0x7f2a9013a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a8effd640 1 -- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a900b3a60 msgr2=0x7f2a9013a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a8effd640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a900b3a60 0x7f2a9013a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a8effd640 1 -- 172.21.15.29:0/1463721849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a90135f70 con 0x7f2a900b3140 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a8e7fc640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a900b3a60 0x7f2a9013a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a8f7fe640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a900b4b10 0x7f2a9013a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a8effd640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a900b3140 0x7f2a90136d30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c820 tx=0x7f2a8000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a877fe640 1 -- 172.21.15.29:0/1463721849 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80015070 con 0x7f2a900b3140 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a877fe640 1 -- 172.21.15.29:0/1463721849 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a80004240 con 0x7f2a900b3140 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.229+0000 7f2a877fe640 1 -- 172.21.15.29:0/1463721849 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8000ce10 con 0x7f2a900b3140 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.233+0000 7f2a95757640 1 -- 172.21.15.29:0/1463721849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a90136250 con 0x7f2a900b3140 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.233+0000 7f2a95757640 1 -- 172.21.15.29:0/1463721849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a90137cd0 con 0x7f2a900b3140 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.233+0000 7f2a95757640 1 -- 172.21.15.29:0/1463721849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a90069270 con 0x7f2a900b3140 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.233+0000 7f2a877fe640 1 -- 172.21.15.29:0/1463721849 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f2a80007830 con 0x7f2a900b3140 2024-04-25T00:34:54.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.233+0000 7f2a877fe640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2a70067a70 0x7f2a70069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:54.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.233+0000 7f2a877fe640 1 -- 172.21.15.29:0/1463721849 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f2a80085fa0 con 0x7f2a900b3140 2024-04-25T00:34:54.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.233+0000 7f2a8e7fc640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2a70067a70 0x7f2a70069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:54.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.233+0000 7f2a8e7fc640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2a70067a70 0x7f2a70069f30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2a780ad000 tx=0x7f2a780a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:54.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.237+0000 7f2a877fe640 1 -- 172.21.15.29:0/1463721849 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2a80056710 con 0x7f2a900b3140 2024-04-25T00:34:54.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.433+0000 7f2a95757640 1 -- 172.21.15.29:0/1463721849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a901446e0 con 0x7f2a900b3140 2024-04-25T00:34:54.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.433+0000 7f2a877fe640 1 -- 172.21.15.29:0/1463721849 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7f2a80056010 con 0x7f2a900b3140 2024-04-25T00:34:54.565 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:54.566 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:54.567 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.437+0000 7f2a95757640 1 -- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2a70067a70 msgr2=0x7f2a70069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.437+0000 7f2a95757640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2a70067a70 0x7f2a70069f30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2a780ad000 tx=0x7f2a780a6000 comp rx=0 tx=0).stop 2024-04-25T00:34:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.437+0000 7f2a95757640 1 -- 172.21.15.29:0/1463721849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a900b3140 msgr2=0x7f2a90136d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.437+0000 7f2a95757640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a900b3140 0x7f2a90136d30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c820 tx=0x7f2a8000cce0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.437+0000 7f2a95757640 1 -- 172.21.15.29:0/1463721849 shutdown_connections 2024-04-25T00:34:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.437+0000 7f2a95757640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a900b4b10 0x7f2a9013a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.437+0000 7f2a95757640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2a70067a70 0x7f2a70069f30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.437+0000 7f2a95757640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a900b3a60 0x7f2a9013a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.437+0000 7f2a95757640 1 --2- 172.21.15.29:0/1463721849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a900b3140 0x7f2a90136d30 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.437+0000 7f2a95757640 1 -- 172.21.15.29:0/1463721849 >> 172.21.15.29:0/1463721849 conn(0x7f2a900b1870 msgr2=0x7f2a90140f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.437+0000 7f2a95757640 1 -- 172.21.15.29:0/1463721849 shutdown_connections 2024-04-25T00:34:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.437+0000 7f2a95757640 1 -- 172.21.15.29:0/1463721849 wait complete. 2024-04-25T00:34:54.569 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:34:54.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.697+0000 7ff013949640 1 Processor -- start 2024-04-25T00:34:54.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.697+0000 7ff013949640 1 -- start start 2024-04-25T00:34:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff013949640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0b3c30 0x7ff00c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff013949640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff00c0b4b10 0x7ff00c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff013949640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff00c0b41e0 con 0x7ff00c0b3140 2024-04-25T00:34:54.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff013949640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff00c06d960 con 0x7ff00c0b4b10 2024-04-25T00:34:54.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff013949640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff00c06dc40 con 0x7ff00c0b3c30 2024-04-25T00:34:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff010ee4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0b3c30 0x7ff00c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff010ee4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0b3c30 0x7ff00c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:59404/0 (socket says 172.21.15.29:59404) 2024-04-25T00:34:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff010ee4640 1 -- 172.21.15.29:0/881067867 learned_addr learned my addr 172.21.15.29:0/881067867 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff011ee6640 1 --2- 172.21.15.29:0/881067867 >> v2:172.21.15.157:3300/0 conn(0x7ff00c0b4b10 0x7ff00c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff011ee6640 1 -- 172.21.15.29:0/881067867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0b3c30 msgr2=0x7ff00c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff011ee6640 1 --2- 172.21.15.29:0/881067867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0b3c30 0x7ff00c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff011ee6640 1 -- 172.21.15.29:0/881067867 >> v1:172.21.15.157:6789/0 conn(0x7ff00c0b3140 legacy=0x7ff00c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff011ee6640 1 -- 172.21.15.29:0/881067867 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00c06df20 con 0x7ff00c0b4b10 2024-04-25T00:34:54.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff010ee4640 1 --2- 172.21.15.29:0/881067867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0b3c30 0x7ff00c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:34:54.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff011ee6640 1 --2- 172.21.15.29:0/881067867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c0b4b10 0x7ff00c0ce240 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff004009d90 tx=0x7ff00401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22da7f1c6234b924 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:54.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.701+0000 7ff003fff640 1 -- 172.21.15.29:0/881067867 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0040b2070 con 0x7ff00c0b4b10 2024-04-25T00:34:54.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff003fff640 1 -- 172.21.15.29:0/881067867 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0040a5830 con 0x7ff00c0b4b10 2024-04-25T00:34:54.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff003fff640 1 -- 172.21.15.29:0/881067867 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0040a5c00 con 0x7ff00c0b4b10 2024-04-25T00:34:54.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 -- 172.21.15.29:0/881067867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c0b4b10 msgr2=0x7ff00c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:54.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 --2- 172.21.15.29:0/881067867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c0b4b10 0x7ff00c0ce240 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff004009d90 tx=0x7ff00401c970 comp rx=0 tx=0).stop 2024-04-25T00:34:54.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 -- 172.21.15.29:0/881067867 shutdown_connections 2024-04-25T00:34:54.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 --2- 172.21.15.29:0/881067867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c0b4b10 0x7ff00c0ce240 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 --2- 172.21.15.29:0/881067867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0b3c30 0x7ff00c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 -- 172.21.15.29:0/881067867 >> 172.21.15.29:0/881067867 conn(0x7ff00c0b1870 msgr2=0x7ff00c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:54.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 -- 172.21.15.29:0/881067867 shutdown_connections 2024-04-25T00:34:54.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 -- 172.21.15.29:0/881067867 wait complete. 2024-04-25T00:34:54.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 Processor -- start 2024-04-25T00:34:54.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 -- start start 2024-04-25T00:34:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff00c0b3140 0x7ff00c138150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:54.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c0b3c30 0x7ff00c13f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0b4b10 0x7ff00c13f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c168d80 con 0x7ff00c0b3140 2024-04-25T00:34:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c168c00 con 0x7ff00c0b3c30 2024-04-25T00:34:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff00c168f00 con 0x7ff00c0b4b10 2024-04-25T00:34:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff0116e5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff00c0b3140 0x7ff00c138150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff0116e5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff00c0b3140 0x7ff00c138150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40358/0 (socket says 172.21.15.29:40358) 2024-04-25T00:34:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff0116e5640 1 -- 172.21.15.29:0/3741344033 learned_addr learned my addr 172.21.15.29:0/3741344033 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff010ee4640 1 --2- 172.21.15.29:0/3741344033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c0b3c30 0x7ff00c13f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff0116e5640 1 -- 172.21.15.29:0/3741344033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0b4b10 msgr2=0x7ff00c13f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff0116e5640 1 --2- 172.21.15.29:0/3741344033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0b4b10 0x7ff00c13f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff0116e5640 1 -- 172.21.15.29:0/3741344033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c0b3c30 msgr2=0x7ff00c13f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff0116e5640 1 --2- 172.21.15.29:0/3741344033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c0b3c30 0x7ff00c13f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff0116e5640 1 -- 172.21.15.29:0/3741344033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00c0ce330 con 0x7ff00c0b3140 2024-04-25T00:34:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff0116e5640 1 --2- 172.21.15.29:0/3741344033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff00c0b3140 0x7ff00c138150 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff00800c420 tx=0x7ff00800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff001ffb640 1 -- 172.21.15.29:0/3741344033 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0080052d0 con 0x7ff00c0b3140 2024-04-25T00:34:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff001ffb640 1 -- 172.21.15.29:0/3741344033 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff008005470 con 0x7ff00c0b3140 2024-04-25T00:34:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff001ffb640 1 -- 172.21.15.29:0/3741344033 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008007260 con 0x7ff00c0b3140 2024-04-25T00:34:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 -- 172.21.15.29:0/3741344033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00c13e610 con 0x7ff00c0b3140 2024-04-25T00:34:54.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.705+0000 7ff013949640 1 -- 172.21.15.29:0/3741344033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00c158f00 con 0x7ff00c0b3140 2024-04-25T00:34:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.709+0000 7ff013949640 1 -- 172.21.15.29:0/3741344033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00c069270 con 0x7ff00c0b3140 2024-04-25T00:34:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.709+0000 7ff001ffb640 1 -- 172.21.15.29:0/3741344033 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7ff008007400 con 0x7ff00c0b3140 2024-04-25T00:34:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.709+0000 7ff001ffb640 1 --2- 172.21.15.29:0/3741344033 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7feff0067a70 0x7feff0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.709+0000 7ff001ffb640 1 -- 172.21.15.29:0/3741344033 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7ff0080854d0 con 0x7ff00c0b3140 2024-04-25T00:34:54.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.709+0000 7ff010ee4640 1 --2- 172.21.15.29:0/3741344033 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7feff0067a70 0x7feff0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:54.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.709+0000 7ff010ee4640 1 --2- 172.21.15.29:0/3741344033 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7feff0067a70 0x7feff0069f30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0b1680 tx=0x7feffc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:54.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.713+0000 7ff001ffb640 1 -- 172.21.15.29:0/3741344033 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff008055de0 con 0x7ff00c0b3140 2024-04-25T00:34:54.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.909+0000 7ff013949640 1 -- 172.21.15.29:0/3741344033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff00c0b37e0 con 0x7ff00c0b3140 2024-04-25T00:34:54.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.909+0000 7ff001ffb640 1 -- 172.21.15.29:0/3741344033 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5181 (secure 0 0 0) 0x7ff0080556e0 con 0x7ff00c0b3140 2024-04-25T00:34:54.912 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:54.912 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:33:13.156560+0000","modified":"2024-04-25T00:33:16.179568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/628132809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":628132809},{"type":"v1","addr":"172.21.15.29:6835","nonce":628132809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5733,"qdb_cluster":[5733]},"id":3}]} 2024-04-25T00:34:54.913 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 25 2024-04-25T00:34:54.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.913+0000 7ff013949640 1 -- 172.21.15.29:0/3741344033 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7feff0067a70 msgr2=0x7feff0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:54.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.913+0000 7ff013949640 1 --2- 172.21.15.29:0/3741344033 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7feff0067a70 0x7feff0069f30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0b1680 tx=0x7feffc008040 comp rx=0 tx=0).stop 2024-04-25T00:34:54.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.913+0000 7ff013949640 1 -- 172.21.15.29:0/3741344033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff00c0b3140 msgr2=0x7ff00c138150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:54.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.913+0000 7ff013949640 1 --2- 172.21.15.29:0/3741344033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff00c0b3140 0x7ff00c138150 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff00800c420 tx=0x7ff00800c8e0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.913+0000 7ff013949640 1 -- 172.21.15.29:0/3741344033 shutdown_connections 2024-04-25T00:34:54.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.913+0000 7ff013949640 1 --2- 172.21.15.29:0/3741344033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0b4b10 0x7ff00c13f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.913+0000 7ff013949640 1 --2- 172.21.15.29:0/3741344033 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7feff0067a70 0x7feff0069f30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.913+0000 7ff013949640 1 --2- 172.21.15.29:0/3741344033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c0b3c30 0x7ff00c13f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.913+0000 7ff013949640 1 --2- 172.21.15.29:0/3741344033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff00c0b3140 0x7ff00c138150 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:54.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.913+0000 7ff013949640 1 -- 172.21.15.29:0/3741344033 >> 172.21.15.29:0/3741344033 conn(0x7ff00c0b1870 msgr2=0x7ff00c144810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:54.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.913+0000 7ff013949640 1 -- 172.21.15.29:0/3741344033 shutdown_connections 2024-04-25T00:34:54.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:54.913+0000 7ff013949640 1 -- 172.21.15.29:0/3741344033 wait complete. 2024-04-25T00:34:54.935 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:34:55.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.049+0000 7fe2002e9640 1 Processor -- start 2024-04-25T00:34:55.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.049+0000 7fe2002e9640 1 -- start start 2024-04-25T00:34:55.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe2002e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe1f80b3140 0x7fe1f806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe2002e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1f80b4b10 0x7fe1f80b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe2002e9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1f80b41e0 con 0x7fe1f80b3a60 2024-04-25T00:34:55.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe2002e9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe1f806d960 con 0x7fe1f80b3140 2024-04-25T00:34:55.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe2002e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f806dc40 con 0x7fe1f80b4b10 2024-04-25T00:34:55.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe1fd884640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1f80b4b10 0x7fe1f80b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe1fe085640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe1f80b3140 0x7fe1f806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe1fe886640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe1f80b3a60 0x7fe1f815c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:39228/0 (socket says 172.21.15.29:39228) 2024-04-25T00:34:55.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe1fd884640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1f80b4b10 0x7fe1f80b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:59430/0 (socket says 172.21.15.29:59430) 2024-04-25T00:34:55.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe1fe886640 1 -- 172.21.15.29:0/3816678090 learned_addr learned my addr 172.21.15.29:0/3816678090 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:55.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe1fd884640 1 -- 172.21.15.29:0/3816678090 >> v2:172.21.15.157:3300/0 conn(0x7fe1f80b3140 msgr2=0x7fe1f806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:55.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe1fd884640 1 --2- 172.21.15.29:0/3816678090 >> v2:172.21.15.157:3300/0 conn(0x7fe1f80b3140 0x7fe1f806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe1fd884640 1 -- 172.21.15.29:0/3816678090 >> v1:172.21.15.157:6789/0 conn(0x7fe1f80b3a60 legacy=0x7fe1f815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe1fd884640 1 -- 172.21.15.29:0/3816678090 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f806df20 con 0x7fe1f80b4b10 2024-04-25T00:34:55.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.053+0000 7fe1fd884640 1 --2- 172.21.15.29:0/3816678090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1f80b4b10 0x7fe1f80b3520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40097c0 tx=0x7fe1f401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5559fdfd38fd81ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:55.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fd083640 1 -- 172.21.15.29:0/3816678090 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40b0070 con 0x7fe1f80b4b10 2024-04-25T00:34:55.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fd083640 1 -- 172.21.15.29:0/3816678090 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f40ab600 con 0x7fe1f80b4b10 2024-04-25T00:34:55.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fd083640 1 -- 172.21.15.29:0/3816678090 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40ab9d0 con 0x7fe1f80b4b10 2024-04-25T00:34:55.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 -- 172.21.15.29:0/3816678090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1f80b4b10 msgr2=0x7fe1f80b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 --2- 172.21.15.29:0/3816678090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1f80b4b10 0x7fe1f80b3520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40097c0 tx=0x7fe1f401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 -- 172.21.15.29:0/3816678090 shutdown_connections 2024-04-25T00:34:55.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 --2- 172.21.15.29:0/3816678090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1f80b4b10 0x7fe1f80b3520 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 --2- 172.21.15.29:0/3816678090 >> v2:172.21.15.157:3300/0 conn(0x7fe1f80b3140 0x7fe1f806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 -- 172.21.15.29:0/3816678090 >> 172.21.15.29:0/3816678090 conn(0x7fe1f80b1870 msgr2=0x7fe1f80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:55.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 -- 172.21.15.29:0/3816678090 shutdown_connections 2024-04-25T00:34:55.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 -- 172.21.15.29:0/3816678090 wait complete. 2024-04-25T00:34:55.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 Processor -- start 2024-04-25T00:34:55.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 -- start start 2024-04-25T00:34:55.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1f80b3140 0x7fe1f80c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f80b3a60 0x7fe1f80ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1f80b4b10 0x7fe1f813e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8168b70 con 0x7fe1f80b3a60 2024-04-25T00:34:55.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81689f0 con 0x7fe1f80b4b10 2024-04-25T00:34:55.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe2002e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f8168cf0 con 0x7fe1f80b3140 2024-04-25T00:34:55.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fd884640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f80b3a60 0x7fe1f80ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fd884640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f80b3a60 0x7fe1f80ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40364/0 (socket says 172.21.15.29:40364) 2024-04-25T00:34:55.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fd884640 1 -- 172.21.15.29:0/1235005387 learned_addr learned my addr 172.21.15.29:0/1235005387 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:55.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fe085640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1f80b3140 0x7fe1f80c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fe886640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1f80b4b10 0x7fe1f813e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fd884640 1 -- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1f80b3140 msgr2=0x7fe1f80c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fd884640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1f80b3140 0x7fe1f80c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fd884640 1 -- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1f80b4b10 msgr2=0x7fe1f813e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fd884640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1f80b4b10 0x7fe1f813e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fd884640 1 -- 172.21.15.29:0/1235005387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f80c5e60 con 0x7fe1f80b3a60 2024-04-25T00:34:55.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1fd884640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f80b3a60 0x7fe1f80ce4d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe1f401ab50 tx=0x7fe1f4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:55.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.057+0000 7fe1eeffd640 1 -- 172.21.15.29:0/1235005387 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40bb070 con 0x7fe1f80b3a60 2024-04-25T00:34:55.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.061+0000 7fe1eeffd640 1 -- 172.21.15.29:0/1235005387 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f40b7460 con 0x7fe1f80b3a60 2024-04-25T00:34:55.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.061+0000 7fe2002e9640 1 -- 172.21.15.29:0/1235005387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f80c6140 con 0x7fe1f80b3a60 2024-04-25T00:34:55.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.061+0000 7fe1eeffd640 1 -- 172.21.15.29:0/1235005387 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40b0040 con 0x7fe1f80b3a60 2024-04-25T00:34:55.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.061+0000 7fe2002e9640 1 -- 172.21.15.29:0/1235005387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f81f6f30 con 0x7fe1f80b3a60 2024-04-25T00:34:55.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.061+0000 7fe2002e9640 1 -- 172.21.15.29:0/1235005387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f8069270 con 0x7fe1f80b3a60 2024-04-25T00:34:55.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.061+0000 7fe1eeffd640 1 -- 172.21.15.29:0/1235005387 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fe1f40be070 con 0x7fe1f80b3a60 2024-04-25T00:34:55.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.061+0000 7fe1eeffd640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe1dc067a70 0x7fe1dc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.061+0000 7fe1eeffd640 1 -- 172.21.15.29:0/1235005387 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fe1f41204a0 con 0x7fe1f80b3a60 2024-04-25T00:34:55.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.061+0000 7fe1fe085640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe1dc067a70 0x7fe1dc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.061+0000 7fe1fe085640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe1dc067a70 0x7fe1dc069f30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009ac0 tx=0x7fe1e8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:55.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.065+0000 7fe1eeffd640 1 -- 172.21.15.29:0/1235005387 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe1f40a4e20 con 0x7fe1f80b3a60 2024-04-25T00:34:55.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.221+0000 7fe2002e9640 1 -- 172.21.15.29:0/1235005387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1f815f060 con 0x7fe1f80b3a60 2024-04-25T00:34:55.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.225+0000 7fe1eeffd640 1 -- 172.21.15.29:0/1235005387 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16657 (secure 0 0 0) 0x7fe1f4016090 con 0x7fe1f80b3a60 2024-04-25T00:34:55.225 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:34:55.226 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":47,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:34:18.731770+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:33:10.789862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:33:12.129841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.36","mappings":[{"from":0,"to":4}]},{"pgid":"7.38","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:34:55.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.225+0000 7fe2002e9640 1 -- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe1dc067a70 msgr2=0x7fe1dc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.225+0000 7fe2002e9640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe1dc067a70 0x7fe1dc069f30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009ac0 tx=0x7fe1e8009190 comp rx=0 tx=0).stop 2024-04-25T00:34:55.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.225+0000 7fe2002e9640 1 -- 172.21.15.29:0/1235005387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f80b3a60 msgr2=0x7fe1f80ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.225+0000 7fe2002e9640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f80b3a60 0x7fe1f80ce4d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe1f401ab50 tx=0x7fe1f4019240 comp rx=0 tx=0).stop 2024-04-25T00:34:55.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.225+0000 7fe2002e9640 1 -- 172.21.15.29:0/1235005387 shutdown_connections 2024-04-25T00:34:55.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.225+0000 7fe2002e9640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe1dc067a70 0x7fe1dc069f30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.225+0000 7fe2002e9640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1f80b4b10 0x7fe1f813e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.225+0000 7fe2002e9640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1f80b3a60 0x7fe1f80ce4d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.225+0000 7fe2002e9640 1 --2- 172.21.15.29:0/1235005387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1f80b3140 0x7fe1f80c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.225+0000 7fe2002e9640 1 -- 172.21.15.29:0/1235005387 >> 172.21.15.29:0/1235005387 conn(0x7fe1f80b1870 msgr2=0x7fe1f8072700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:55.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.225+0000 7fe2002e9640 1 -- 172.21.15.29:0/1235005387 shutdown_connections 2024-04-25T00:34:55.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.225+0000 7fe2002e9640 1 -- 172.21.15.29:0/1235005387 wait complete. 2024-04-25T00:34:55.243 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:34:55.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f9093440640 1 Processor -- start 2024-04-25T00:34:55.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f9093440640 1 -- start start 2024-04-25T00:34:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f9093440640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f908c15e200 0x7f908c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f9093440640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f908c0cf690 0x7f908c0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f9093440640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f908c0d2a90 0x7f908c0d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f9093440640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f908c0d2ee0 con 0x7f908c0cf690 2024-04-25T00:34:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f9093440640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f908c0b38d0 con 0x7f908c15e200 2024-04-25T00:34:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f9093440640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f908c0cfc10 con 0x7f908c0d2a90 2024-04-25T00:34:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f90909db640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f908c0cf690 0x7f908c0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f90909db640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f908c0cf690 0x7f908c0d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40374/0 (socket says 172.21.15.29:40374) 2024-04-25T00:34:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f90909db640 1 -- 172.21.15.29:0/453789699 learned_addr learned my addr 172.21.15.29:0/453789699 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f90911dc640 1 --2- 172.21.15.29:0/453789699 >> v2:172.21.15.157:3300/0 conn(0x7f908c15e200 0x7f908c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f90919dd640 1 --2- 172.21.15.29:0/453789699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f908c0d2a90 0x7f908c0d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f90909db640 1 -- 172.21.15.29:0/453789699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f908c0d2a90 msgr2=0x7f908c0d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f90909db640 1 --2- 172.21.15.29:0/453789699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f908c0d2a90 0x7f908c0d05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f90909db640 1 -- 172.21.15.29:0/453789699 >> v2:172.21.15.157:3300/0 conn(0x7f908c15e200 msgr2=0x7f908c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f90909db640 1 --2- 172.21.15.29:0/453789699 >> v2:172.21.15.157:3300/0 conn(0x7f908c15e200 0x7f908c0d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.353+0000 7f90909db640 1 -- 172.21.15.29:0/453789699 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c0cfef0 con 0x7f908c0cf690 2024-04-25T00:34:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.357+0000 7f90909db640 1 --2- 172.21.15.29:0/453789699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f908c0cf690 0x7f908c0d2550 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f908c08d8f0 tx=0x7f907c01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bd8b0788f2c162 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.357+0000 7f9083fff640 1 -- 172.21.15.29:0/453789699 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c0b0070 con 0x7f908c0cf690 2024-04-25T00:34:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.357+0000 7f9083fff640 1 -- 172.21.15.29:0/453789699 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f907c0ab600 con 0x7f908c0cf690 2024-04-25T00:34:55.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.357+0000 7f9083fff640 1 -- 172.21.15.29:0/453789699 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c0ab9d0 con 0x7f908c0cf690 2024-04-25T00:34:55.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.357+0000 7f9093440640 1 -- 172.21.15.29:0/453789699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f908c0cf690 msgr2=0x7f908c0d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.357+0000 7f9093440640 1 --2- 172.21.15.29:0/453789699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f908c0cf690 0x7f908c0d2550 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f908c08d8f0 tx=0x7f907c01c230 comp rx=0 tx=0).stop 2024-04-25T00:34:55.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.357+0000 7f9093440640 1 -- 172.21.15.29:0/453789699 shutdown_connections 2024-04-25T00:34:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.357+0000 7f9093440640 1 --2- 172.21.15.29:0/453789699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f908c0cf690 0x7f908c0d2550 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.357+0000 7f9093440640 1 --2- 172.21.15.29:0/453789699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f908c0d2a90 0x7f908c0d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.357+0000 7f9093440640 1 --2- 172.21.15.29:0/453789699 >> v2:172.21.15.157:3300/0 conn(0x7f908c15e200 0x7f908c0d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.357+0000 7f9093440640 1 -- 172.21.15.29:0/453789699 >> 172.21.15.29:0/453789699 conn(0x7f908c09d280 msgr2=0x7f908c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.357+0000 7f9093440640 1 -- 172.21.15.29:0/453789699 shutdown_connections 2024-04-25T00:34:55.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.357+0000 7f9093440640 1 -- 172.21.15.29:0/453789699 wait complete. 2024-04-25T00:34:55.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9093440640 1 Processor -- start 2024-04-25T00:34:55.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9093440640 1 -- start start 2024-04-25T00:34:55.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9093440640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f908c0cf690 0x7f908c154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9093440640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f908c0d2a90 0x7f908c0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9093440640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908c15e200 0x7f908c155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9093440640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f908c15e760 con 0x7f908c0d2a90 2024-04-25T00:34:55.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9093440640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f908c15e5e0 con 0x7f908c15e200 2024-04-25T00:34:55.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9093440640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f908c0b4320 con 0x7f908c0cf690 2024-04-25T00:34:55.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f90909db640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f908c0d2a90 0x7f908c0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f90909db640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f908c0d2a90 0x7f908c0cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40378/0 (socket says 172.21.15.29:40378) 2024-04-25T00:34:55.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f90909db640 1 -- 172.21.15.29:0/355685410 learned_addr learned my addr 172.21.15.29:0/355685410 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:55.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f90919dd640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908c15e200 0x7f908c155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f90909db640 1 -- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f908c0cf690 msgr2=0x7f908c154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f90911dc640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f908c0cf690 0x7f908c154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f90909db640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f908c0cf690 0x7f908c154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f90909db640 1 -- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908c15e200 msgr2=0x7f908c155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f90909db640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908c15e200 0x7f908c155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f90909db640 1 -- 172.21.15.29:0/355685410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c15d490 con 0x7f908c0d2a90 2024-04-25T00:34:55.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f90909db640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f908c0d2a90 0x7f908c0cd620 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f907c01a8e0 tx=0x7f907c018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:55.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9081ffb640 1 -- 172.21.15.29:0/355685410 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c0b9070 con 0x7f908c0d2a90 2024-04-25T00:34:55.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9081ffb640 1 -- 172.21.15.29:0/355685410 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f907c0b4400 con 0x7f908c0d2a90 2024-04-25T00:34:55.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9081ffb640 1 -- 172.21.15.29:0/355685410 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c0b0040 con 0x7f908c0d2a90 2024-04-25T00:34:55.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9093440640 1 -- 172.21.15.29:0/355685410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908c15d9c0 con 0x7f908c0d2a90 2024-04-25T00:34:55.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9093440640 1 -- 172.21.15.29:0/355685410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908c15c9b0 con 0x7f908c0d2a90 2024-04-25T00:34:55.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9093440640 1 -- 172.21.15.29:0/355685410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908c069270 con 0x7f908c0d2a90 2024-04-25T00:34:55.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.361+0000 7f9081ffb640 1 -- 172.21.15.29:0/355685410 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f907c0b33a0 con 0x7f908c0d2a90 2024-04-25T00:34:55.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.365+0000 7f9081ffb640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9070067a70 0x7f9070069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.365+0000 7f9081ffb640 1 -- 172.21.15.29:0/355685410 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f907c11f740 con 0x7f908c0d2a90 2024-04-25T00:34:55.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.365+0000 7f90911dc640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9070067a70 0x7f9070069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.365+0000 7f90911dc640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9070067a70 0x7f9070069f30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9088009e90 tx=0x7f908800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:55.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.365+0000 7f9081ffb640 1 -- 172.21.15.29:0/355685410 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f907c0c2020 con 0x7f908c0d2a90 2024-04-25T00:34:55.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.557+0000 7f9093440640 1 -- 172.21.15.29:0/355685410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f908c1570d0 con 0x7f908c0d2a90 2024-04-25T00:34:55.559 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-04-25T00:34:55.557+0000 7f3ac00b5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:34:55.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.561+0000 7f9081ffb640 1 -- 172.21.15.29:0/355685410 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7f907c016110 con 0x7f908c0d2a90 2024-04-25T00:34:55.562 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:34:55.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.565+0000 7f9093440640 1 -- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9070067a70 msgr2=0x7f9070069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.565+0000 7f9093440640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9070067a70 0x7f9070069f30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9088009e90 tx=0x7f908800b040 comp rx=0 tx=0).stop 2024-04-25T00:34:55.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.565+0000 7f9093440640 1 -- 172.21.15.29:0/355685410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f908c0d2a90 msgr2=0x7f908c0cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.565+0000 7f9093440640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f908c0d2a90 0x7f908c0cd620 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f907c01a8e0 tx=0x7f907c018fd0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.565+0000 7f9093440640 1 -- 172.21.15.29:0/355685410 shutdown_connections 2024-04-25T00:34:55.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.565+0000 7f9093440640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9070067a70 0x7f9070069f30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.565+0000 7f9093440640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908c15e200 0x7f908c155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.565+0000 7f9093440640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f908c0d2a90 0x7f908c0cd620 secure :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f907c01a8e0 tx=0x7f907c018fd0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.565+0000 7f9093440640 1 --2- 172.21.15.29:0/355685410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f908c0cf690 0x7f908c154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.565+0000 7f9093440640 1 -- 172.21.15.29:0/355685410 >> 172.21.15.29:0/355685410 conn(0x7f908c09d280 msgr2=0x7f908c1575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:55.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.565+0000 7f9093440640 1 -- 172.21.15.29:0/355685410 shutdown_connections 2024-04-25T00:34:55.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.565+0000 7f9093440640 1 -- 172.21.15.29:0/355685410 wait complete. 2024-04-25T00:34:55.568 INFO:tasks.ceph.mds.c.smithi029.stderr: -9578> 2024-04-25T00:33:18.750+0000 7fe2b8365640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:34:55.583 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:34:55.601 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:34:55.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f4186e2f640 1 Processor -- start 2024-04-25T00:34:55.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f4186e2f640 1 -- start start 2024-04-25T00:34:55.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f4186e2f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f41801555b0 0x7f418015e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f4186e2f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f418015bfb0 0x7f418006d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f4186e2f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f418015f170 con 0x7f41800b5000 2024-04-25T00:34:55.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f4186e2f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f418015f450 con 0x7f418015bfb0 2024-04-25T00:34:55.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f4186e2f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f418015f730 con 0x7f41801555b0 2024-04-25T00:34:55.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f417ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f418015bfb0 0x7f418006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f417ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f418015bfb0 0x7f418006d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40404/0 (socket says 172.21.15.29:40404) 2024-04-25T00:34:55.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f417ffff640 1 -- 172.21.15.29:0/3549486682 learned_addr learned my addr 172.21.15.29:0/3549486682 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:55.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f417ffff640 1 -- 172.21.15.29:0/3549486682 >> v2:172.21.15.157:3300/0 conn(0x7f41801555b0 msgr2=0x7f418015e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:55.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f417ffff640 1 --2- 172.21.15.29:0/3549486682 >> v2:172.21.15.157:3300/0 conn(0x7f41801555b0 0x7f418015e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f417ffff640 1 -- 172.21.15.29:0/3549486682 >> v1:172.21.15.157:6789/0 conn(0x7f41800b5000 legacy=0x7f41800cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:55.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.697+0000 7f417ffff640 1 -- 172.21.15.29:0/3549486682 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41800cef00 con 0x7f418015bfb0 2024-04-25T00:34:55.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417ffff640 1 --2- 172.21.15.29:0/3549486682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f418015bfb0 0x7f418006d330 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f41800b5bc0 tx=0x7f416c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53014a1ba7847ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:55.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417f7fe640 1 -- 172.21.15.29:0/3549486682 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c0aa600 con 0x7f418015bfb0 2024-04-25T00:34:55.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417f7fe640 1 -- 172.21.15.29:0/3549486682 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416c0af070 con 0x7f418015bfb0 2024-04-25T00:34:55.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417f7fe640 1 -- 172.21.15.29:0/3549486682 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c0aa9c0 con 0x7f418015bfb0 2024-04-25T00:34:55.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 -- 172.21.15.29:0/3549486682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f418015bfb0 msgr2=0x7f418006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 --2- 172.21.15.29:0/3549486682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f418015bfb0 0x7f418006d330 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f41800b5bc0 tx=0x7f416c01c480 comp rx=0 tx=0).stop 2024-04-25T00:34:55.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 -- 172.21.15.29:0/3549486682 shutdown_connections 2024-04-25T00:34:55.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 --2- 172.21.15.29:0/3549486682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f418015bfb0 0x7f418006d330 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 --2- 172.21.15.29:0/3549486682 >> v2:172.21.15.157:3300/0 conn(0x7f41801555b0 0x7f418015e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 -- 172.21.15.29:0/3549486682 >> 172.21.15.29:0/3549486682 conn(0x7f4180075030 msgr2=0x7f41800b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:55.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 -- 172.21.15.29:0/3549486682 shutdown_connections 2024-04-25T00:34:55.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 -- 172.21.15.29:0/3549486682 wait complete. 2024-04-25T00:34:55.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 Processor -- start 2024-04-25T00:34:55.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 -- start start 2024-04-25T00:34:55.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41800b5000 0x7f418013f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41801555b0 0x7f4180140b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f418015bfb0 0x7f41801559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f418006d9f0 con 0x7f41801555b0 2024-04-25T00:34:55.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f418006d870 con 0x7f41800b5000 2024-04-25T00:34:55.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f41801684d0 con 0x7f418015bfb0 2024-04-25T00:34:55.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41801555b0 0x7f4180140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41801555b0 0x7f4180140b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40412/0 (socket says 172.21.15.29:40412) 2024-04-25T00:34:55.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417ffff640 1 -- 172.21.15.29:0/3678153051 learned_addr learned my addr 172.21.15.29:0/3678153051 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:55.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4184bcb640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41800b5000 0x7f418013f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f41853cc640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f418015bfb0 0x7f41801559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417ffff640 1 -- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f418015bfb0 msgr2=0x7f41801559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417ffff640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f418015bfb0 0x7f41801559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417ffff640 1 -- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41800b5000 msgr2=0x7f418013f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:55.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417ffff640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41800b5000 0x7f418013f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:55.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417ffff640 1 -- 172.21.15.29:0/3678153051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4180159120 con 0x7f41801555b0 2024-04-25T00:34:55.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4184bcb640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41800b5000 0x7f418013f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:55.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417ffff640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41801555b0 0x7f4180140b20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f416c0ac040 tx=0x7f416c01ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:55.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417d7fa640 1 -- 172.21.15.29:0/3678153051 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c0bd5f0 con 0x7f41801555b0 2024-04-25T00:34:55.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417d7fa640 1 -- 172.21.15.29:0/3678153051 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416c0bd790 con 0x7f41801555b0 2024-04-25T00:34:55.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 -- 172.21.15.29:0/3678153051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4180159400 con 0x7f41801555b0 2024-04-25T00:34:55.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f417d7fa640 1 -- 172.21.15.29:0/3678153051 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c0bdb80 con 0x7f41801555b0 2024-04-25T00:34:55.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.701+0000 7f4186e2f640 1 -- 172.21.15.29:0/3678153051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41801599c0 con 0x7f41801555b0 2024-04-25T00:34:55.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.705+0000 7f4186e2f640 1 -- 172.21.15.29:0/3678153051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4180069270 con 0x7f41801555b0 2024-04-25T00:34:55.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.705+0000 7f417d7fa640 1 -- 172.21.15.29:0/3678153051 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f416c0a9070 con 0x7f41801555b0 2024-04-25T00:34:55.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.705+0000 7f417d7fa640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4164067a70 0x7f4164069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:55.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.705+0000 7f417d7fa640 1 -- 172.21.15.29:0/3678153051 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7f416c120b90 con 0x7f41801555b0 2024-04-25T00:34:55.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.705+0000 7f4184bcb640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4164067a70 0x7f4164069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:55.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.705+0000 7f4184bcb640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4164067a70 0x7f4164069f30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f41801f9100 tx=0x7f4174008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:55.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.709+0000 7f417d7fa640 1 -- 172.21.15.29:0/3678153051 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f416c0c3060 con 0x7f41801555b0 2024-04-25T00:34:55.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:55.909+0000 7f4186e2f640 1 -- 172.21.15.29:0/3678153051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4180069270 con 0x7f41801555b0 2024-04-25T00:34:56.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.561+0000 7f417d7fa640 1 -- 172.21.15.29:0/3678153051 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f416c0160d0 con 0x7f41801555b0 2024-04-25T00:34:56.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.565+0000 7f4186e2f640 1 -- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4164067a70 msgr2=0x7f4164069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:56.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.565+0000 7f4186e2f640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4164067a70 0x7f4164069f30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f41801f9100 tx=0x7f4174008040 comp rx=0 tx=0).stop 2024-04-25T00:34:56.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.565+0000 7f4186e2f640 1 -- 172.21.15.29:0/3678153051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41801555b0 msgr2=0x7f4180140b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:56.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.565+0000 7f4186e2f640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41801555b0 0x7f4180140b20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f416c0ac040 tx=0x7f416c01ad20 comp rx=0 tx=0).stop 2024-04-25T00:34:56.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.565+0000 7f4186e2f640 1 -- 172.21.15.29:0/3678153051 shutdown_connections 2024-04-25T00:34:56.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.565+0000 7f4186e2f640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f418015bfb0 0x7f41801559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.565+0000 7f4186e2f640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41801555b0 0x7f4180140b20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.565+0000 7f4186e2f640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4164067a70 0x7f4164069f30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.565+0000 7f4186e2f640 1 --2- 172.21.15.29:0/3678153051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41800b5000 0x7f418013f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.565+0000 7f4186e2f640 1 -- 172.21.15.29:0/3678153051 >> 172.21.15.29:0/3678153051 conn(0x7f4180075030 msgr2=0x7f418006dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.565+0000 7f4186e2f640 1 -- 172.21.15.29:0/3678153051 shutdown_connections 2024-04-25T00:34:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.565+0000 7f4186e2f640 1 -- 172.21.15.29:0/3678153051 wait complete. 2024-04-25T00:34:56.593 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:34:56.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb10181640 1 Processor -- start 2024-04-25T00:34:56.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb10181640 1 -- start start 2024-04-25T00:34:56.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb10181640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb081456d0 0x7ffb080b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:56.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb10181640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffb080b3b50 0x7ffb080cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:56.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb10181640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb080b4380 con 0x7ffb0806cd40 2024-04-25T00:34:56.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb10181640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ffb080b40a0 con 0x7ffb080b3b50 2024-04-25T00:34:56.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb10181640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffb080ce480 con 0x7ffb081456d0 2024-04-25T00:34:56.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb0d71c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffb080b3b50 0x7ffb080cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:56.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb0e71e640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ffb0806cd40 0x7ffb0806d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36052/0 (socket says 172.21.15.29:36052) 2024-04-25T00:34:56.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb0e71e640 1 -- 172.21.15.29:0/2863353447 learned_addr learned my addr 172.21.15.29:0/2863353447 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:56.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb0df1d640 1 --2- 172.21.15.29:0/2863353447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb081456d0 0x7ffb080b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:56.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb0d71c640 1 -- 172.21.15.29:0/2863353447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb081456d0 msgr2=0x7ffb080b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:56.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb0d71c640 1 --2- 172.21.15.29:0/2863353447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb081456d0 0x7ffb080b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:56.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb0d71c640 1 -- 172.21.15.29:0/2863353447 >> v1:172.21.15.29:6789/0 conn(0x7ffb0806cd40 legacy=0x7ffb0806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:56.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb0d71c640 1 -- 172.21.15.29:0/2863353447 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb080ce760 con 0x7ffb080b3b50 2024-04-25T00:34:56.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb0df1d640 1 --2- 172.21.15.29:0/2863353447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb081456d0 0x7ffb080b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:56.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.777+0000 7ffb0cf1b640 1 -- 172.21.15.29:0/2863353447 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2691199839 0 0) 0x7ffb080b4380 con 0x7ffb0806cd40 2024-04-25T00:34:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb0d71c640 1 --2- 172.21.15.29:0/2863353447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb080b3b50 0x7ffb080cef60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ffaf80097c0 tx=0x7ffaf801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12221f54b3e72a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb0cf1b640 1 -- 172.21.15.29:0/2863353447 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf80b0070 con 0x7ffb080b3b50 2024-04-25T00:34:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb0cf1b640 1 -- 172.21.15.29:0/2863353447 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf80ab600 con 0x7ffb080b3b50 2024-04-25T00:34:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb0cf1b640 1 -- 172.21.15.29:0/2863353447 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf80ab9d0 con 0x7ffb080b3b50 2024-04-25T00:34:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb10181640 1 -- 172.21.15.29:0/2863353447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb080b3b50 msgr2=0x7ffb080cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb10181640 1 --2- 172.21.15.29:0/2863353447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb080b3b50 0x7ffb080cef60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ffaf80097c0 tx=0x7ffaf801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:34:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb10181640 1 -- 172.21.15.29:0/2863353447 shutdown_connections 2024-04-25T00:34:56.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb10181640 1 --2- 172.21.15.29:0/2863353447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb080b3b50 0x7ffb080cef60 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:56.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb10181640 1 --2- 172.21.15.29:0/2863353447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb081456d0 0x7ffb080b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:56.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb10181640 1 -- 172.21.15.29:0/2863353447 >> 172.21.15.29:0/2863353447 conn(0x7ffb080b1870 msgr2=0x7ffb080b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:56.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb10181640 1 -- 172.21.15.29:0/2863353447 shutdown_connections 2024-04-25T00:34:56.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb10181640 1 -- 172.21.15.29:0/2863353447 wait complete. 2024-04-25T00:34:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb10181640 1 Processor -- start 2024-04-25T00:34:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.781+0000 7ffb10181640 1 -- start start 2024-04-25T00:34:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb10181640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb0806cd40 0x7ffb08156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb10181640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb080b3b50 0x7ffb0813baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb10181640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb081456d0 0x7ffb08142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb10181640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffb080cf620 con 0x7ffb081456d0 2024-04-25T00:34:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb10181640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb080cf4a0 con 0x7ffb080b3b50 2024-04-25T00:34:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb10181640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffb08168b70 con 0x7ffb0806cd40 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb0e71e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb081456d0 0x7ffb08142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb0e71e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb081456d0 0x7ffb08142cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40434/0 (socket says 172.21.15.29:40434) 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb0e71e640 1 -- 172.21.15.29:0/1265848716 learned_addr learned my addr 172.21.15.29:0/1265848716 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb0df1d640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb0806cd40 0x7ffb08156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb0e71e640 1 -- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb0806cd40 msgr2=0x7ffb08156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb0e71e640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb0806cd40 0x7ffb08156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb0d71c640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb080b3b50 0x7ffb0813baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb0e71e640 1 -- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb080b3b50 msgr2=0x7ffb0813baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb0e71e640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb080b3b50 0x7ffb0813baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb0e71e640 1 -- 172.21.15.29:0/1265848716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb081555f0 con 0x7ffb081456d0 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb0e71e640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb081456d0 0x7ffb08142cd0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000c700 tx=0x7ffb0000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffafe7fc640 1 -- 172.21.15.29:0/1265848716 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00018070 con 0x7ffb081456d0 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffafe7fc640 1 -- 172.21.15.29:0/1265848716 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb0000ce10 con 0x7ffb081456d0 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb10181640 1 -- 172.21.15.29:0/1265848716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb081558d0 con 0x7ffb081456d0 2024-04-25T00:34:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffafe7fc640 1 -- 172.21.15.29:0/1265848716 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0000a860 con 0x7ffb081456d0 2024-04-25T00:34:56.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb10181640 1 -- 172.21.15.29:0/1265848716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb08155e90 con 0x7ffb081456d0 2024-04-25T00:34:56.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb10181640 1 -- 172.21.15.29:0/1265848716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb0813bfe0 con 0x7ffb081456d0 2024-04-25T00:34:56.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffafe7fc640 1 -- 172.21.15.29:0/1265848716 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7ffb0001b070 con 0x7ffb081456d0 2024-04-25T00:34:56.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffafe7fc640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffaec067aa0 0x7ffaec069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:56.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffafe7fc640 1 -- 172.21.15.29:0/1265848716 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7ffb000855c0 con 0x7ffb081456d0 2024-04-25T00:34:56.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.785+0000 7ffb0df1d640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffaec067aa0 0x7ffaec069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:56.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.789+0000 7ffb0df1d640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffaec067aa0 0x7ffaec069f60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400a9a0 tx=0x7ffb04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:56.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.789+0000 7ffafe7fc640 1 -- 172.21.15.29:0/1265848716 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffb00055df0 con 0x7ffb081456d0 2024-04-25T00:34:56.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:56.953+0000 7ffb10181640 1 -- 172.21.15.29:0/1265848716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ffb080b4de0 con 0x7ffb081456d0 2024-04-25T00:34:57.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.677+0000 7ffafe7fc640 1 -- 172.21.15.29:0/1265848716 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) ==== 181+0+0 (secure 0 0 0) 0x7ffb000556f0 con 0x7ffb081456d0 2024-04-25T00:34:57.680 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:34:57.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.681+0000 7ffb10181640 1 -- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffaec067aa0 msgr2=0x7ffaec069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:57.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.681+0000 7ffb10181640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffaec067aa0 0x7ffaec069f60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400a9a0 tx=0x7ffb04008040 comp rx=0 tx=0).stop 2024-04-25T00:34:57.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.681+0000 7ffb10181640 1 -- 172.21.15.29:0/1265848716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb081456d0 msgr2=0x7ffb08142cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:57.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.681+0000 7ffb10181640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb081456d0 0x7ffb08142cd0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000c700 tx=0x7ffb0000cbc0 comp rx=0 tx=0).stop 2024-04-25T00:34:57.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.681+0000 7ffb10181640 1 -- 172.21.15.29:0/1265848716 shutdown_connections 2024-04-25T00:34:57.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.681+0000 7ffb10181640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb081456d0 0x7ffb08142cd0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:57.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.681+0000 7ffb10181640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ffaec067aa0 0x7ffaec069f60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:57.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.681+0000 7ffb10181640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb080b3b50 0x7ffb0813baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.681+0000 7ffb10181640 1 --2- 172.21.15.29:0/1265848716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb0806cd40 0x7ffb08156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.681+0000 7ffb10181640 1 -- 172.21.15.29:0/1265848716 >> 172.21.15.29:0/1265848716 conn(0x7ffb080b1870 msgr2=0x7ffb08144680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.685+0000 7ffb10181640 1 -- 172.21.15.29:0/1265848716 shutdown_connections 2024-04-25T00:34:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.685+0000 7ffb10181640 1 -- 172.21.15.29:0/1265848716 wait complete. 2024-04-25T00:34:57.708 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:34:57.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.837+0000 7f58c435c640 1 Processor -- start 2024-04-25T00:34:57.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.837+0000 7f58c435c640 1 -- start start 2024-04-25T00:34:57.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.837+0000 7f58c435c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58bc0b4590 0x7f58bc070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:57.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.837+0000 7f58c435c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58bc0cd1c0 0x7f58bc072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:57.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.837+0000 7f58c435c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58bc0712c0 con 0x7f58bc15e9a0 2024-04-25T00:34:57.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.837+0000 7f58c435c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f58bc071ed0 con 0x7f58bc0cd1c0 2024-04-25T00:34:57.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.837+0000 7f58c435c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f58bc0715a0 con 0x7f58bc0b4590 2024-04-25T00:34:57.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.837+0000 7f58c18f7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58bc0b4590 0x7f58bc070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:57.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.837+0000 7f58c28f9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58bc0cd1c0 0x7f58bc072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:57.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.837+0000 7f58c20f8640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f58bc15e9a0 0x7f58bc15ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:36164/0 (socket says 172.21.15.29:36164) 2024-04-25T00:34:57.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.837+0000 7f58c20f8640 1 -- 172.21.15.29:0/1245485300 learned_addr learned my addr 172.21.15.29:0/1245485300 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:57.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c18f7640 1 -- 172.21.15.29:0/1245485300 >> v2:172.21.15.157:3300/0 conn(0x7f58bc0cd1c0 msgr2=0x7f58bc072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:57.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c18f7640 1 --2- 172.21.15.29:0/1245485300 >> v2:172.21.15.157:3300/0 conn(0x7f58bc0cd1c0 0x7f58bc072360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:57.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c18f7640 1 -- 172.21.15.29:0/1245485300 >> v1:172.21.15.157:6789/0 conn(0x7f58bc15e9a0 legacy=0x7f58bc15ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:57.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c18f7640 1 -- 172.21.15.29:0/1245485300 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc071880 con 0x7f58bc0b4590 2024-04-25T00:34:57.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c28f9640 1 --2- 172.21.15.29:0/1245485300 >> v2:172.21.15.157:3300/0 conn(0x7f58bc0cd1c0 0x7f58bc072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:34:57.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c18f7640 1 --2- 172.21.15.29:0/1245485300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58bc0b4590 0x7f58bc070ce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0097c0 tx=0x7f58ac01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de089c01965592d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:57.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c10f6640 1 -- 172.21.15.29:0/1245485300 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0ab600 con 0x7f58bc0b4590 2024-04-25T00:34:57.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c10f6640 1 -- 172.21.15.29:0/1245485300 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ac0b0070 con 0x7f58bc0b4590 2024-04-25T00:34:57.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c10f6640 1 -- 172.21.15.29:0/1245485300 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0ab9c0 con 0x7f58bc0b4590 2024-04-25T00:34:57.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c435c640 1 -- 172.21.15.29:0/1245485300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58bc0b4590 msgr2=0x7f58bc070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:57.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c435c640 1 --2- 172.21.15.29:0/1245485300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58bc0b4590 0x7f58bc070ce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0097c0 tx=0x7f58ac01c3e0 comp rx=0 tx=0).stop 2024-04-25T00:34:57.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c435c640 1 -- 172.21.15.29:0/1245485300 shutdown_connections 2024-04-25T00:34:57.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c435c640 1 --2- 172.21.15.29:0/1245485300 >> v2:172.21.15.157:3300/0 conn(0x7f58bc0cd1c0 0x7f58bc072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:57.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c435c640 1 --2- 172.21.15.29:0/1245485300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58bc0b4590 0x7f58bc070ce0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:57.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c435c640 1 -- 172.21.15.29:0/1245485300 >> 172.21.15.29:0/1245485300 conn(0x7f58bc0b1870 msgr2=0x7f58bc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:57.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c435c640 1 -- 172.21.15.29:0/1245485300 shutdown_connections 2024-04-25T00:34:57.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.841+0000 7f58c435c640 1 -- 172.21.15.29:0/1245485300 wait complete. 2024-04-25T00:34:57.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c435c640 1 Processor -- start 2024-04-25T00:34:57.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c435c640 1 -- start start 2024-04-25T00:34:57.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c435c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58bc0b4590 0x7f58bc0cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c435c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58bc0cd1c0 0x7f58bc159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c435c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58bc15e9a0 0x7f58bc13a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c435c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc0cd5d0 con 0x7f58bc15e9a0 2024-04-25T00:34:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c435c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc0cd750 con 0x7f58bc0b4590 2024-04-25T00:34:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c435c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f58bc06d5d0 con 0x7f58bc0cd1c0 2024-04-25T00:34:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c28f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58bc15e9a0 0x7f58bc13a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c28f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58bc15e9a0 0x7f58bc13a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48034/0 (socket says 172.21.15.29:48034) 2024-04-25T00:34:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c20f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58bc0b4590 0x7f58bc0cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c28f9640 1 -- 172.21.15.29:0/3377580385 learned_addr learned my addr 172.21.15.29:0/3377580385 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c18f7640 1 --2- 172.21.15.29:0/3377580385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58bc0cd1c0 0x7f58bc159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c28f9640 1 -- 172.21.15.29:0/3377580385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58bc0cd1c0 msgr2=0x7f58bc159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c28f9640 1 --2- 172.21.15.29:0/3377580385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58bc0cd1c0 0x7f58bc159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c28f9640 1 -- 172.21.15.29:0/3377580385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58bc0b4590 msgr2=0x7f58bc0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:57.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c28f9640 1 --2- 172.21.15.29:0/3377580385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58bc0b4590 0x7f58bc0cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:57.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c28f9640 1 -- 172.21.15.29:0/3377580385 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc13cb50 con 0x7f58bc15e9a0 2024-04-25T00:34:57.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c28f9640 1 --2- 172.21.15.29:0/3377580385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58bc15e9a0 0x7f58bc13a940 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f58b400efc0 tx=0x7f58b4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:57.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58b2ffd640 1 -- 172.21.15.29:0/3377580385 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4018030 con 0x7f58bc15e9a0 2024-04-25T00:34:57.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58b2ffd640 1 -- 172.21.15.29:0/3377580385 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b400cde0 con 0x7f58bc15e9a0 2024-04-25T00:34:57.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c435c640 1 -- 172.21.15.29:0/3377580385 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58bc1f2f40 con 0x7f58bc15e9a0 2024-04-25T00:34:57.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58b2ffd640 1 -- 172.21.15.29:0/3377580385 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4003bc0 con 0x7f58bc15e9a0 2024-04-25T00:34:57.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c435c640 1 -- 172.21.15.29:0/3377580385 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58bc0d0000 con 0x7f58bc15e9a0 2024-04-25T00:34:57.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.845+0000 7f58c435c640 1 -- 172.21.15.29:0/3377580385 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58bc137470 con 0x7f58bc15e9a0 2024-04-25T00:34:57.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.849+0000 7f58b2ffd640 1 -- 172.21.15.29:0/3377580385 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f58b401b070 con 0x7f58bc15e9a0 2024-04-25T00:34:57.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.849+0000 7f58b2ffd640 1 --2- 172.21.15.29:0/3377580385 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f58a0067a70 0x7f58a0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:57.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.849+0000 7f58b2ffd640 1 -- 172.21.15.29:0/3377580385 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5415+0+0 (secure 0 0 0) 0x7f58b4086840 con 0x7f58bc15e9a0 2024-04-25T00:34:57.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.849+0000 7f58c20f8640 1 --2- 172.21.15.29:0/3377580385 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f58a0067a70 0x7f58a0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:57.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.849+0000 7f58c20f8640 1 --2- 172.21.15.29:0/3377580385 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f58a0067a70 0x7f58a0069f30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f58bc137350 tx=0x7f58b8007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:57.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:57.853+0000 7f58b2ffd640 1 -- 172.21.15.29:0/3377580385 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f58b4010040 con 0x7f58bc15e9a0 2024-04-25T00:34:58.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.013+0000 7f58c435c640 1 -- 172.21.15.29:0/3377580385 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f58bc0692e0 con 0x7f58bc15e9a0 2024-04-25T00:34:58.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.681+0000 7f58b2ffd640 1 -- 172.21.15.29:0/3377580385 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) ==== 169+0+0 (secure 0 0 0) 0x7f58b4056b50 con 0x7f58bc15e9a0 2024-04-25T00:34:58.683 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-04-25T00:34:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.685+0000 7f58c435c640 1 -- 172.21.15.29:0/3377580385 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f58a0067a70 msgr2=0x7f58a0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.689+0000 7f58c435c640 1 --2- 172.21.15.29:0/3377580385 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f58a0067a70 0x7f58a0069f30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f58bc137350 tx=0x7f58b8007e90 comp rx=0 tx=0).stop 2024-04-25T00:34:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.689+0000 7f58c435c640 1 -- 172.21.15.29:0/3377580385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58bc15e9a0 msgr2=0x7f58bc13a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.689+0000 7f58c435c640 1 --2- 172.21.15.29:0/3377580385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58bc15e9a0 0x7f58bc13a940 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f58b400efc0 tx=0x7f58b4009f90 comp rx=0 tx=0).stop 2024-04-25T00:34:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.689+0000 7f58c435c640 1 -- 172.21.15.29:0/3377580385 shutdown_connections 2024-04-25T00:34:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.689+0000 7f58c435c640 1 --2- 172.21.15.29:0/3377580385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58bc15e9a0 0x7f58bc13a940 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:58.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.689+0000 7f58c435c640 1 --2- 172.21.15.29:0/3377580385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58bc0cd1c0 0x7f58bc159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.689+0000 7f58c435c640 1 --2- 172.21.15.29:0/3377580385 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f58a0067a70 0x7f58a0069f30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.689+0000 7f58c435c640 1 --2- 172.21.15.29:0/3377580385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58bc0b4590 0x7f58bc0cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.689+0000 7f58c435c640 1 -- 172.21.15.29:0/3377580385 >> 172.21.15.29:0/3377580385 conn(0x7f58bc0b1870 msgr2=0x7f58bc15a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.689+0000 7f58c435c640 1 -- 172.21.15.29:0/3377580385 shutdown_connections 2024-04-25T00:34:58.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.689+0000 7f58c435c640 1 -- 172.21.15.29:0/3377580385 wait complete. 2024-04-25T00:34:58.714 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:34:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc98ad01640 1 Processor -- start 2024-04-25T00:34:58.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc98ad01640 1 -- start start 2024-04-25T00:34:58.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc98ad01640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc98415e200 0x7fc9840d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc98ad01640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc9840d2c60 0x7fc9840b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc98ad01640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9840cfc10 con 0x7fc9840cf690 2024-04-25T00:34:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc98ad01640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc9840b38d0 con 0x7fc9840d2c60 2024-04-25T00:34:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc98ad01640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc9840cfef0 con 0x7fc98415e200 2024-04-25T00:34:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc98929e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc9840d2c60 0x7fc9840b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc983fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc9840cf690 0x7fc9840d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43404/0 (socket says 172.21.15.29:43404) 2024-04-25T00:34:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc98929e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc9840d2c60 0x7fc9840b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48038/0 (socket says 172.21.15.29:48038) 2024-04-25T00:34:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc983fff640 1 -- 172.21.15.29:0/2981525327 learned_addr learned my addr 172.21.15.29:0/2981525327 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:34:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc988a9d640 1 --2- 172.21.15.29:0/2981525327 >> v2:172.21.15.157:3300/0 conn(0x7fc98415e200 0x7fc9840d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc98929e640 1 -- 172.21.15.29:0/2981525327 >> v2:172.21.15.157:3300/0 conn(0x7fc98415e200 msgr2=0x7fc9840d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc98929e640 1 --2- 172.21.15.29:0/2981525327 >> v2:172.21.15.157:3300/0 conn(0x7fc98415e200 0x7fc9840d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc98929e640 1 -- 172.21.15.29:0/2981525327 >> v1:172.21.15.29:6789/0 conn(0x7fc9840cf690 legacy=0x7fc9840d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:58.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.837+0000 7fc98929e640 1 -- 172.21.15.29:0/2981525327 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9840b3b20 con 0x7fc9840d2c60 2024-04-25T00:34:58.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98929e640 1 --2- 172.21.15.29:0/2981525327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9840d2c60 0x7fc9840b4320 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc98408d8f0 tx=0x7fc97c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ddb88a1b87b916 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:58.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc9837fe640 1 -- 172.21.15.29:0/2981525327 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0b2070 con 0x7fc9840d2c60 2024-04-25T00:34:58.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc9837fe640 1 -- 172.21.15.29:0/2981525327 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97c0a59e0 con 0x7fc9840d2c60 2024-04-25T00:34:58.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc9837fe640 1 -- 172.21.15.29:0/2981525327 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0a5db0 con 0x7fc9840d2c60 2024-04-25T00:34:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 -- 172.21.15.29:0/2981525327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9840d2c60 msgr2=0x7fc9840b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 --2- 172.21.15.29:0/2981525327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9840d2c60 0x7fc9840b4320 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc98408d8f0 tx=0x7fc97c01c900 comp rx=0 tx=0).stop 2024-04-25T00:34:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 -- 172.21.15.29:0/2981525327 shutdown_connections 2024-04-25T00:34:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 --2- 172.21.15.29:0/2981525327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9840d2c60 0x7fc9840b4320 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:58.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 --2- 172.21.15.29:0/2981525327 >> v2:172.21.15.157:3300/0 conn(0x7fc98415e200 0x7fc9840d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 -- 172.21.15.29:0/2981525327 >> 172.21.15.29:0/2981525327 conn(0x7fc98409d280 msgr2=0x7fc9840d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 -- 172.21.15.29:0/2981525327 shutdown_connections 2024-04-25T00:34:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 -- 172.21.15.29:0/2981525327 wait complete. 2024-04-25T00:34:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 Processor -- start 2024-04-25T00:34:58.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 -- start start 2024-04-25T00:34:58.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9840cf690 0x7fc9840c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:58.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9840d2c60 0x7fc9840b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:58.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc98415e200 0x7fc984154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:58.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc9840b51f0 con 0x7fc9840cf690 2024-04-25T00:34:58.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc9840b5070 con 0x7fc98415e200 2024-04-25T00:34:58.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98ad01640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc9840b5370 con 0x7fc9840d2c60 2024-04-25T00:34:58.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc988a9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9840cf690 0x7fc9840c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:58.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc988a9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9840cf690 0x7fc9840c84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48042/0 (socket says 172.21.15.29:48042) 2024-04-25T00:34:58.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc988a9d640 1 -- 172.21.15.29:0/2161242515 learned_addr learned my addr 172.21.15.29:0/2161242515 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:58.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc983fff640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9840d2c60 0x7fc9840b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:58.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc988a9d640 1 -- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9840d2c60 msgr2=0x7fc9840b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:58.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc988a9d640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9840d2c60 0x7fc9840b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:58.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98929e640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc98415e200 0x7fc984154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:58.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc988a9d640 1 -- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc98415e200 msgr2=0x7fc984154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:58.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc988a9d640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc98415e200 0x7fc984154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:58.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc988a9d640 1 -- 172.21.15.29:0/2161242515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc984141ee0 con 0x7fc9840cf690 2024-04-25T00:34:58.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.841+0000 7fc98929e640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc98415e200 0x7fc984154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:58.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.845+0000 7fc988a9d640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9840cf690 0x7fc9840c84b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc97800d5f0 tx=0x7fc97800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:58.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.845+0000 7fc9817fa640 1 -- 172.21.15.29:0/2161242515 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9780125f0 con 0x7fc9840cf690 2024-04-25T00:34:58.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.845+0000 7fc9817fa640 1 -- 172.21.15.29:0/2161242515 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc978019070 con 0x7fc9840cf690 2024-04-25T00:34:58.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.845+0000 7fc98ad01640 1 -- 172.21.15.29:0/2161242515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9841421c0 con 0x7fc9840cf690 2024-04-25T00:34:58.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.845+0000 7fc9817fa640 1 -- 172.21.15.29:0/2161242515 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9780129d0 con 0x7fc9840cf690 2024-04-25T00:34:58.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.845+0000 7fc98ad01640 1 -- 172.21.15.29:0/2161242515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc984142780 con 0x7fc9840cf690 2024-04-25T00:34:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.845+0000 7fc98ad01640 1 -- 172.21.15.29:0/2161242515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc984069270 con 0x7fc9840cf690 2024-04-25T00:34:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.845+0000 7fc9817fa640 1 -- 172.21.15.29:0/2161242515 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fc978012b70 con 0x7fc9840cf690 2024-04-25T00:34:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.845+0000 7fc9817fa640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc968067a70 0x7fc968069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.845+0000 7fc9817fa640 1 -- 172.21.15.29:0/2161242515 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fc978086130 con 0x7fc9840cf690 2024-04-25T00:34:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.845+0000 7fc983fff640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc968067a70 0x7fc968069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.845+0000 7fc983fff640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc968067a70 0x7fc968069f30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc9840cef70 tx=0x7fc970009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:58.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:58.849+0000 7fc9817fa640 1 -- 172.21.15.29:0/2161242515 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc978056cf0 con 0x7fc9840cf690 2024-04-25T00:34:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.013+0000 7fc98ad01640 1 -- 172.21.15.29:0/2161242515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc9840d2960 con 0x7fc9840cf690 2024-04-25T00:34:59.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.017+0000 7fc9817fa640 1 -- 172.21.15.29:0/2161242515 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fc9780565f0 con 0x7fc9840cf690 2024-04-25T00:34:59.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.021+0000 7fc98ad01640 1 -- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc968067a70 msgr2=0x7fc968069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:59.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.021+0000 7fc98ad01640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc968067a70 0x7fc968069f30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc9840cef70 tx=0x7fc970009040 comp rx=0 tx=0).stop 2024-04-25T00:34:59.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.021+0000 7fc98ad01640 1 -- 172.21.15.29:0/2161242515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9840cf690 msgr2=0x7fc9840c84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:59.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.021+0000 7fc98ad01640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9840cf690 0x7fc9840c84b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc97800d5f0 tx=0x7fc97800dab0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.021+0000 7fc98ad01640 1 -- 172.21.15.29:0/2161242515 shutdown_connections 2024-04-25T00:34:59.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.021+0000 7fc98ad01640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc968067a70 0x7fc968069f30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.021+0000 7fc98ad01640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc98415e200 0x7fc984154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.021+0000 7fc98ad01640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9840d2c60 0x7fc9840b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.021+0000 7fc98ad01640 1 --2- 172.21.15.29:0/2161242515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9840cf690 0x7fc9840c84b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.021+0000 7fc98ad01640 1 -- 172.21.15.29:0/2161242515 >> 172.21.15.29:0/2161242515 conn(0x7fc98409d280 msgr2=0x7fc98415d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:59.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.021+0000 7fc98ad01640 1 -- 172.21.15.29:0/2161242515 shutdown_connections 2024-04-25T00:34:59.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.021+0000 7fc98ad01640 1 -- 172.21.15.29:0/2161242515 wait complete. 2024-04-25T00:34:59.039 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-04-25T00:34:59.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.165+0000 7f3e645c2640 1 Processor -- start 2024-04-25T00:34:59.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.165+0000 7f3e645c2640 1 -- start start 2024-04-25T00:34:59.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e645c2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e5c15e200 0x7f3e5c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:59.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e645c2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3e5c0d2c60 0x7f3e5c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:59.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e645c2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e5c0cfc10 con 0x7f3e5c0cf690 2024-04-25T00:34:59.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e645c2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f3e5c0b38d0 con 0x7f3e5c0d2c60 2024-04-25T00:34:59.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e645c2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3e5c0cfef0 con 0x7f3e5c15e200 2024-04-25T00:34:59.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e62b5f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3e5c0d2c60 0x7f3e5c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:59.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e62b5f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3e5c0d2c60 0x7f3e5c0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48052/0 (socket says 172.21.15.29:48052) 2024-04-25T00:34:59.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e61b5d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3e5c0cf690 0x7f3e5c0d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43420/0 (socket says 172.21.15.29:43420) 2024-04-25T00:34:59.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e62b5f640 1 -- 172.21.15.29:0/990100714 learned_addr learned my addr 172.21.15.29:0/990100714 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:59.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e6235e640 1 --2- 172.21.15.29:0/990100714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e5c15e200 0x7f3e5c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:59.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e62b5f640 1 -- 172.21.15.29:0/990100714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e5c15e200 msgr2=0x7f3e5c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:59.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e62b5f640 1 --2- 172.21.15.29:0/990100714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e5c15e200 0x7f3e5c0d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e62b5f640 1 -- 172.21.15.29:0/990100714 >> v1:172.21.15.29:6789/0 conn(0x7f3e5c0cf690 legacy=0x7f3e5c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:59.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.169+0000 7f3e62b5f640 1 -- 172.21.15.29:0/990100714 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c0b3b20 con 0x7f3e5c0d2c60 2024-04-25T00:34:59.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e62b5f640 1 --2- 172.21.15.29:0/990100714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c0d2c60 0x7f3e5c0b4320 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009870 tx=0x7f3e5401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f7045542a2180b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:59.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e6135c640 1 -- 172.21.15.29:0/990100714 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e540a9610 con 0x7f3e5c0d2c60 2024-04-25T00:34:59.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e6135c640 1 -- 172.21.15.29:0/990100714 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e540a97b0 con 0x7f3e5c0d2c60 2024-04-25T00:34:59.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e6135c640 1 -- 172.21.15.29:0/990100714 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e540a9b80 con 0x7f3e5c0d2c60 2024-04-25T00:34:59.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 -- 172.21.15.29:0/990100714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c0d2c60 msgr2=0x7f3e5c0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:59.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 --2- 172.21.15.29:0/990100714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c0d2c60 0x7f3e5c0b4320 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009870 tx=0x7f3e5401c530 comp rx=0 tx=0).stop 2024-04-25T00:34:59.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 -- 172.21.15.29:0/990100714 shutdown_connections 2024-04-25T00:34:59.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 --2- 172.21.15.29:0/990100714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c0d2c60 0x7f3e5c0b4320 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 --2- 172.21.15.29:0/990100714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e5c15e200 0x7f3e5c0d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 -- 172.21.15.29:0/990100714 >> 172.21.15.29:0/990100714 conn(0x7f3e5c09d280 msgr2=0x7f3e5c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:59.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 -- 172.21.15.29:0/990100714 shutdown_connections 2024-04-25T00:34:59.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 -- 172.21.15.29:0/990100714 wait complete. 2024-04-25T00:34:59.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 Processor -- start 2024-04-25T00:34:59.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 -- start start 2024-04-25T00:34:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e5c0cf690 0x7f3e5c151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e5c0d2c60 0x7f3e5c0c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c15e200 0x7f3e5c0cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c0b51f0 con 0x7f3e5c15e200 2024-04-25T00:34:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c0b5070 con 0x7f3e5c0cf690 2024-04-25T00:34:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e645c2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3e5c0b5370 con 0x7f3e5c0d2c60 2024-04-25T00:34:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e62b5f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c15e200 0x7f3e5c0cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e62b5f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c15e200 0x7f3e5c0cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48062/0 (socket says 172.21.15.29:48062) 2024-04-25T00:34:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.173+0000 7f3e62b5f640 1 -- 172.21.15.29:0/540944964 learned_addr learned my addr 172.21.15.29:0/540944964 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e6235e640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e5c0cf690 0x7f3e5c151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e61b5d640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e5c0d2c60 0x7f3e5c0c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e62b5f640 1 -- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e5c0d2c60 msgr2=0x7f3e5c0c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:59.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e62b5f640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e5c0d2c60 0x7f3e5c0c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e62b5f640 1 -- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e5c0cf690 msgr2=0x7f3e5c151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e62b5f640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e5c0cf690 0x7f3e5c151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e62b5f640 1 -- 172.21.15.29:0/540944964 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c0c7fe0 con 0x7f3e5c15e200 2024-04-25T00:34:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e62b5f640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c15e200 0x7f3e5c0cb570 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3e5401a160 tx=0x7f3e5401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e52ffd640 1 -- 172.21.15.29:0/540944964 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e540bc650 con 0x7f3e5c15e200 2024-04-25T00:34:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e52ffd640 1 -- 172.21.15.29:0/540944964 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e540bc7f0 con 0x7f3e5c15e200 2024-04-25T00:34:59.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e645c2640 1 -- 172.21.15.29:0/540944964 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5c0ca310 con 0x7f3e5c15e200 2024-04-25T00:34:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e52ffd640 1 -- 172.21.15.29:0/540944964 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e540bcba0 con 0x7f3e5c15e200 2024-04-25T00:34:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e645c2640 1 -- 172.21.15.29:0/540944964 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5c152530 con 0x7f3e5c15e200 2024-04-25T00:34:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e645c2640 1 -- 172.21.15.29:0/540944964 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e5c069270 con 0x7f3e5c15e200 2024-04-25T00:34:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e52ffd640 1 -- 172.21.15.29:0/540944964 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f3e540a9620 con 0x7f3e5c15e200 2024-04-25T00:34:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e52ffd640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3e40067a70 0x7f3e40069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e52ffd640 1 -- 172.21.15.29:0/540944964 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f3e541205e0 con 0x7f3e5c15e200 2024-04-25T00:34:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e6235e640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3e40067a70 0x7f3e40069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:59.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.177+0000 7f3e6235e640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3e40067a70 0x7f3e40069f30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0caaa0 tx=0x7f3e58009800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:59.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.181+0000 7f3e52ffd640 1 -- 172.21.15.29:0/540944964 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3e540ad020 con 0x7f3e5c15e200 2024-04-25T00:34:59.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.405+0000 7f3e645c2640 1 -- 172.21.15.29:0/540944964 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f3e5c0d10d0 con 0x7f3e5c15e200 2024-04-25T00:34:59.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.693+0000 7f3e52ffd640 1 -- 172.21.15.29:0/540944964 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v187) ==== 140+0+0 (secure 0 0 0) 0x7f3e54016150 con 0x7f3e5c15e200 2024-04-25T00:34:59.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.697+0000 7f3e645c2640 1 -- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3e40067a70 msgr2=0x7f3e40069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:59.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.697+0000 7f3e645c2640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3e40067a70 0x7f3e40069f30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0caaa0 tx=0x7f3e58009800 comp rx=0 tx=0).stop 2024-04-25T00:34:59.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.697+0000 7f3e645c2640 1 -- 172.21.15.29:0/540944964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c15e200 msgr2=0x7f3e5c0cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:59.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.697+0000 7f3e645c2640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c15e200 0x7f3e5c0cb570 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3e5401a160 tx=0x7f3e5401ad70 comp rx=0 tx=0).stop 2024-04-25T00:34:59.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.697+0000 7f3e645c2640 1 -- 172.21.15.29:0/540944964 shutdown_connections 2024-04-25T00:34:59.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.697+0000 7f3e645c2640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c15e200 0x7f3e5c0cb570 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.697+0000 7f3e645c2640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e5c0d2c60 0x7f3e5c0c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.697+0000 7f3e645c2640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3e40067a70 0x7f3e40069f30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.697+0000 7f3e645c2640 1 --2- 172.21.15.29:0/540944964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e5c0cf690 0x7f3e5c151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.697+0000 7f3e645c2640 1 -- 172.21.15.29:0/540944964 >> 172.21.15.29:0/540944964 conn(0x7f3e5c09d280 msgr2=0x7f3e5c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:59.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.697+0000 7f3e645c2640 1 -- 172.21.15.29:0/540944964 shutdown_connections 2024-04-25T00:34:59.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.697+0000 7f3e645c2640 1 -- 172.21.15.29:0/540944964 wait complete. 2024-04-25T00:34:59.718 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-04-25T00:34:59.718 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2024-04-25T00:34:59.718 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-04-25T00:34:59.719 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:34:59.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.861+0000 7fb5fd82d640 1 Processor -- start 2024-04-25T00:34:59.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.861+0000 7fb5fd82d640 1 -- start start 2024-04-25T00:34:59.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.861+0000 7fb5fd82d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f80b3a90 0x7fb5f8161d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:59.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.861+0000 7fb5fd82d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb5f806d580 0x7fb5f8162280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:59.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.861+0000 7fb5fd82d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5f8143a90 con 0x7fb5f80d07a0 2024-04-25T00:34:59.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.861+0000 7fb5fd82d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb5f8143d30 con 0x7fb5f806d580 2024-04-25T00:34:59.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.861+0000 7fb5fd82d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f8144010 con 0x7fb5f80b3a90 2024-04-25T00:34:59.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.861+0000 7fb5f77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb5f806d580 0x7fb5f8162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:59.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.861+0000 7fb5f77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb5f806d580 0x7fb5f8162280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48064/0 (socket says 172.21.15.29:48064) 2024-04-25T00:34:59.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.861+0000 7fb5f77fe640 1 -- 172.21.15.29:0/3391379067 learned_addr learned my addr 172.21.15.29:0/3391379067 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:59.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5f67fc640 1 --2- 172.21.15.29:0/3391379067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f80b3a90 0x7fb5f8161d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:59.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5f77fe640 1 -- 172.21.15.29:0/3391379067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f80b3a90 msgr2=0x7fb5f8161d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:59.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5f77fe640 1 --2- 172.21.15.29:0/3391379067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f80b3a90 0x7fb5f8161d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5f77fe640 1 -- 172.21.15.29:0/3391379067 >> v1:172.21.15.157:6789/0 conn(0x7fb5f80d07a0 legacy=0x7fb5f80b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:59.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5f77fe640 1 -- 172.21.15.29:0/3391379067 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f81442f0 con 0x7fb5f806d580 2024-04-25T00:34:59.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5f67fc640 1 --2- 172.21.15.29:0/3391379067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f80b3a90 0x7fb5f8161d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:59.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5f77fe640 1 --2- 172.21.15.29:0/3391379067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5f806d580 0x7fb5f8162280 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009e50 tx=0x7fb5f001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f53f63005457e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:59.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5f5ffb640 1 -- 172.21.15.29:0/3391379067 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00a58f0 con 0x7fb5f806d580 2024-04-25T00:34:59.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5f5ffb640 1 -- 172.21.15.29:0/3391379067 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f00a5a90 con 0x7fb5f806d580 2024-04-25T00:34:59.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5f5ffb640 1 -- 172.21.15.29:0/3391379067 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00a5e60 con 0x7fb5f806d580 2024-04-25T00:34:59.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5fd82d640 1 -- 172.21.15.29:0/3391379067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5f806d580 msgr2=0x7fb5f8162280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:59.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5fd82d640 1 --2- 172.21.15.29:0/3391379067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5f806d580 0x7fb5f8162280 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009e50 tx=0x7fb5f001ca00 comp rx=0 tx=0).stop 2024-04-25T00:34:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5fd82d640 1 -- 172.21.15.29:0/3391379067 shutdown_connections 2024-04-25T00:34:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5fd82d640 1 --2- 172.21.15.29:0/3391379067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5f806d580 0x7fb5f8162280 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5fd82d640 1 --2- 172.21.15.29:0/3391379067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f80b3a90 0x7fb5f8161d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5fd82d640 1 -- 172.21.15.29:0/3391379067 >> 172.21.15.29:0/3391379067 conn(0x7fb5f8074c00 msgr2=0x7fb5f806b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5fd82d640 1 -- 172.21.15.29:0/3391379067 shutdown_connections 2024-04-25T00:34:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.865+0000 7fb5fd82d640 1 -- 172.21.15.29:0/3391379067 wait complete. 2024-04-25T00:34:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5fd82d640 1 Processor -- start 2024-04-25T00:34:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5fd82d640 1 -- start start 2024-04-25T00:34:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5fd82d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f806d580 0x7fb5f8146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5fd82d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5f80b3a90 0x7fb5f815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5fd82d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80d07a0 0x7fb5f806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5fd82d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f8162940 con 0x7fb5f80b3a90 2024-04-25T00:34:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5fd82d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f81627c0 con 0x7fb5f80d07a0 2024-04-25T00:34:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5fd82d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f8162ac0 con 0x7fb5f806d580 2024-04-25T00:34:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5f67fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5f80b3a90 0x7fb5f815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5f67fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5f80b3a90 0x7fb5f815bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48068/0 (socket says 172.21.15.29:48068) 2024-04-25T00:34:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5f67fc640 1 -- 172.21.15.29:0/728448387 learned_addr learned my addr 172.21.15.29:0/728448387 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:34:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5f67fc640 1 -- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f806d580 msgr2=0x7fb5f8146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5f77fe640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80d07a0 0x7fb5f806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5f6ffd640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f806d580 0x7fb5f8146ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5f67fc640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f806d580 0x7fb5f8146ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5f67fc640 1 -- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80d07a0 msgr2=0x7fb5f806d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5f67fc640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80d07a0 0x7fb5f806d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:59.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5f67fc640 1 -- 172.21.15.29:0/728448387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f813f040 con 0x7fb5f80b3a90 2024-04-25T00:34:59.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5f77fe640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80d07a0 0x7fb5f806d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:59.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5f67fc640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5f80b3a90 0x7fb5f815bb60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000c7f0 tx=0x7fb5e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:59.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5effff640 1 -- 172.21.15.29:0/728448387 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0007a30 con 0x7fb5f80b3a90 2024-04-25T00:34:59.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5effff640 1 -- 172.21.15.29:0/728448387 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e0007bd0 con 0x7fb5f80b3a90 2024-04-25T00:34:59.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5fd82d640 1 -- 172.21.15.29:0/728448387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f813f320 con 0x7fb5f80b3a90 2024-04-25T00:34:59.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5effff640 1 -- 172.21.15.29:0/728448387 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e00113f0 con 0x7fb5f80b3a90 2024-04-25T00:34:59.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5fd82d640 1 -- 172.21.15.29:0/728448387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f813f8e0 con 0x7fb5f80b3a90 2024-04-25T00:34:59.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5fd82d640 1 -- 172.21.15.29:0/728448387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f8069270 con 0x7fb5f80b3a90 2024-04-25T00:34:59.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5effff640 1 -- 172.21.15.29:0/728448387 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fb5e0011590 con 0x7fb5f80b3a90 2024-04-25T00:34:59.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5effff640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb5d8067a70 0x7fb5d8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:59.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.869+0000 7fb5effff640 1 -- 172.21.15.29:0/728448387 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fb5e0086040 con 0x7fb5f80b3a90 2024-04-25T00:34:59.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.873+0000 7fb5f6ffd640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb5d8067a70 0x7fb5d8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:59.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.873+0000 7fb5f6ffd640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb5d8067a70 0x7fb5d8069f30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8007fd0 tx=0x7fb5e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:59.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:34:59.877+0000 7fb5effff640 1 -- 172.21.15.29:0/728448387 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb5e0015020 con 0x7fb5f80b3a90 2024-04-25T00:35:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.029+0000 7fb5fd82d640 1 -- 172.21.15.29:0/728448387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5f806b9f0 con 0x7fb5f80b3a90 2024-04-25T00:35:00.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.033+0000 7fb5effff640 1 -- 172.21.15.29:0/728448387 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+12970 (secure 0 0 0) 0x7fb5e0056a60 con 0x7fb5f80b3a90 2024-04-25T00:35:00.036 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:00.036 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":50,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:34:58.680811+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6835/628132809":"2024-04-26T00:34:55.560109+0000","172.21.15.29:6833/628132809":"2024-04-26T00:34:55.560109+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.033+0000 7fb5fd82d640 1 -- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb5d8067a70 msgr2=0x7fb5d8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.033+0000 7fb5fd82d640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb5d8067a70 0x7fb5d8069f30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8007fd0 tx=0x7fb5e8009040 comp rx=0 tx=0).stop 2024-04-25T00:35:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.033+0000 7fb5fd82d640 1 -- 172.21.15.29:0/728448387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5f80b3a90 msgr2=0x7fb5f815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.033+0000 7fb5fd82d640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5f80b3a90 0x7fb5f815bb60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000c7f0 tx=0x7fb5e000ccb0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.033+0000 7fb5fd82d640 1 -- 172.21.15.29:0/728448387 shutdown_connections 2024-04-25T00:35:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.033+0000 7fb5fd82d640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb5d8067a70 0x7fb5d8069f30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.033+0000 7fb5fd82d640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f80d07a0 0x7fb5f806d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.033+0000 7fb5fd82d640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5f80b3a90 0x7fb5f815bb60 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.033+0000 7fb5fd82d640 1 --2- 172.21.15.29:0/728448387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f806d580 0x7fb5f8146ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.033+0000 7fb5fd82d640 1 -- 172.21.15.29:0/728448387 >> 172.21.15.29:0/728448387 conn(0x7fb5f8074c00 msgr2=0x7fb5f8162070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.033+0000 7fb5fd82d640 1 -- 172.21.15.29:0/728448387 shutdown_connections 2024-04-25T00:35:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.033+0000 7fb5fd82d640 1 -- 172.21.15.29:0/728448387 wait complete. 2024-04-25T00:35:00.052 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:35:00.052 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:35:00.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b2f42640 1 Processor -- start 2024-04-25T00:35:00.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b2f42640 1 -- start start 2024-04-25T00:35:00.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b2f42640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44ac071c70 0x7f44ac0b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b2f42640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44ac0af8d0 0x7f44ac06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b2f42640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44ac15c560 con 0x7f44ac15aec0 2024-04-25T00:35:00.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b2f42640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f44ac15b2a0 con 0x7f44ac0af8d0 2024-04-25T00:35:00.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b2f42640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44ac15b4c0 con 0x7f44ac071c70 2024-04-25T00:35:00.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b14df640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44ac0af8d0 0x7f44ac06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44abfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44ac071c70 0x7f44ac0b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b14df640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44ac0af8d0 0x7f44ac06d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:45432/0 (socket says 172.21.15.29:45432) 2024-04-25T00:35:00.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b14df640 1 -- 172.21.15.29:0/1330967776 learned_addr learned my addr 172.21.15.29:0/1330967776 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:00.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b14df640 1 -- 172.21.15.29:0/1330967776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44ac071c70 msgr2=0x7f44ac0b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:00.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b14df640 1 --2- 172.21.15.29:0/1330967776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44ac071c70 0x7f44ac0b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b14df640 1 -- 172.21.15.29:0/1330967776 >> v1:172.21.15.157:6789/0 conn(0x7f44ac15aec0 legacy=0x7f44ac071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.185+0000 7f44b14df640 1 -- 172.21.15.29:0/1330967776 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac15b7a0 con 0x7f44ac0af8d0 2024-04-25T00:35:00.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44b14df640 1 --2- 172.21.15.29:0/1330967776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44ac0af8d0 0x7f44ac06d580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f449c009e50 tx=0x7f449c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6268e25c1e8547 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:00.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44ab7fe640 1 -- 172.21.15.29:0/1330967776 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c0a57a0 con 0x7f44ac0af8d0 2024-04-25T00:35:00.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44ab7fe640 1 -- 172.21.15.29:0/1330967776 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449c0a5940 con 0x7f44ac0af8d0 2024-04-25T00:35:00.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44ab7fe640 1 -- 172.21.15.29:0/1330967776 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c0a5d10 con 0x7f44ac0af8d0 2024-04-25T00:35:00.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44b2f42640 1 -- 172.21.15.29:0/1330967776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44ac0af8d0 msgr2=0x7f44ac06d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44b2f42640 1 --2- 172.21.15.29:0/1330967776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44ac0af8d0 0x7f44ac06d580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f449c009e50 tx=0x7f449c01c8e0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44b2f42640 1 -- 172.21.15.29:0/1330967776 shutdown_connections 2024-04-25T00:35:00.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44b2f42640 1 --2- 172.21.15.29:0/1330967776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44ac0af8d0 0x7f44ac06d580 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44b2f42640 1 --2- 172.21.15.29:0/1330967776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44ac071c70 0x7f44ac0b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44b2f42640 1 -- 172.21.15.29:0/1330967776 >> 172.21.15.29:0/1330967776 conn(0x7f44ac074c00 msgr2=0x7f44ac06b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:00.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44b2f42640 1 -- 172.21.15.29:0/1330967776 shutdown_connections 2024-04-25T00:35:00.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44b2f42640 1 -- 172.21.15.29:0/1330967776 wait complete. 2024-04-25T00:35:00.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44b2f42640 1 Processor -- start 2024-04-25T00:35:00.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.189+0000 7f44b2f42640 1 -- start start 2024-04-25T00:35:00.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b2f42640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44ac071c70 0x7f44ac1521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b2f42640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44ac0af8d0 0x7f44ac0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b2f42640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44ac15aec0 0x7f44ac0cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b2f42640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac06e2d0 con 0x7f44ac15aec0 2024-04-25T00:35:00.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b2f42640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac0b3f30 con 0x7f44ac0af8d0 2024-04-25T00:35:00.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b2f42640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44ac06e450 con 0x7f44ac071c70 2024-04-25T00:35:00.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b14df640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44ac15aec0 0x7f44ac0cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b14df640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44ac15aec0 0x7f44ac0cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48082/0 (socket says 172.21.15.29:48082) 2024-04-25T00:35:00.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b14df640 1 -- 172.21.15.29:0/1416552351 learned_addr learned my addr 172.21.15.29:0/1416552351 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:00.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44abfff640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44ac0af8d0 0x7f44ac0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b0cde640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44ac071c70 0x7f44ac1521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b14df640 1 -- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44ac071c70 msgr2=0x7f44ac1521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b14df640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44ac071c70 0x7f44ac1521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b14df640 1 -- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44ac0af8d0 msgr2=0x7f44ac0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b14df640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44ac0af8d0 0x7f44ac0ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b14df640 1 -- 172.21.15.29:0/1416552351 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac15a500 con 0x7f44ac15aec0 2024-04-25T00:35:00.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b0cde640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44ac071c70 0x7f44ac1521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:00.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b14df640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44ac15aec0 0x7f44ac0cede0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f449c0af040 tx=0x7f449c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:00.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44a97fa640 1 -- 172.21.15.29:0/1416552351 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c0bc020 con 0x7f44ac15aec0 2024-04-25T00:35:00.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44a97fa640 1 -- 172.21.15.29:0/1416552351 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449c0b8670 con 0x7f44ac15aec0 2024-04-25T00:35:00.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44a97fa640 1 -- 172.21.15.29:0/1416552351 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c0b8a40 con 0x7f44ac15aec0 2024-04-25T00:35:00.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b2f42640 1 -- 172.21.15.29:0/1416552351 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac158c50 con 0x7f44ac15aec0 2024-04-25T00:35:00.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b2f42640 1 -- 172.21.15.29:0/1416552351 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac157e90 con 0x7f44ac15aec0 2024-04-25T00:35:00.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.193+0000 7f44b2f42640 1 -- 172.21.15.29:0/1416552351 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ac069270 con 0x7f44ac15aec0 2024-04-25T00:35:00.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.197+0000 7f44a97fa640 1 -- 172.21.15.29:0/1416552351 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f449c0bf020 con 0x7f44ac15aec0 2024-04-25T00:35:00.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.197+0000 7f44a97fa640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f448c067a70 0x7f448c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.197+0000 7f44a97fa640 1 -- 172.21.15.29:0/1416552351 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f449c120d40 con 0x7f44ac15aec0 2024-04-25T00:35:00.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.197+0000 7f44b0cde640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f448c067a70 0x7f448c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.197+0000 7f44b0cde640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f448c067a70 0x7f448c069f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0cf3f0 tx=0x7f44a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:00.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.201+0000 7f44a97fa640 1 -- 172.21.15.29:0/1416552351 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f449c0c6030 con 0x7f44ac15aec0 2024-04-25T00:35:00.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.361+0000 7f44b2f42640 1 -- 172.21.15.29:0/1416552351 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f44ac0b4e00 con 0x7f44ac15aec0 2024-04-25T00:35:00.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.365+0000 7f44a97fa640 1 -- 172.21.15.29:0/1416552351 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7f449c016620 con 0x7f44ac15aec0 2024-04-25T00:35:00.365 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:35:00.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.365+0000 7f44b2f42640 1 -- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f448c067a70 msgr2=0x7f448c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.365+0000 7f44b2f42640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f448c067a70 0x7f448c069f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0cf3f0 tx=0x7f44a4009040 comp rx=0 tx=0).stop 2024-04-25T00:35:00.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.365+0000 7f44b2f42640 1 -- 172.21.15.29:0/1416552351 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44ac15aec0 msgr2=0x7f44ac0cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.365+0000 7f44b2f42640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44ac15aec0 0x7f44ac0cede0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f449c0af040 tx=0x7f449c019680 comp rx=0 tx=0).stop 2024-04-25T00:35:00.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.365+0000 7f44b2f42640 1 -- 172.21.15.29:0/1416552351 shutdown_connections 2024-04-25T00:35:00.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.365+0000 7f44b2f42640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44ac15aec0 0x7f44ac0cede0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.365+0000 7f44b2f42640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f448c067a70 0x7f448c069f30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.365+0000 7f44b2f42640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44ac0af8d0 0x7f44ac0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.365+0000 7f44b2f42640 1 --2- 172.21.15.29:0/1416552351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44ac071c70 0x7f44ac1521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.365+0000 7f44b2f42640 1 -- 172.21.15.29:0/1416552351 >> 172.21.15.29:0/1416552351 conn(0x7f44ac074c00 msgr2=0x7f44ac155d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:00.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.365+0000 7f44b2f42640 1 -- 172.21.15.29:0/1416552351 shutdown_connections 2024-04-25T00:35:00.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.365+0000 7f44b2f42640 1 -- 172.21.15.29:0/1416552351 wait complete. 2024-04-25T00:35:00.385 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-04-25T00:35:00.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc65e9c3640 1 Processor -- start 2024-04-25T00:35:00.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc65e9c3640 1 -- start start 2024-04-25T00:35:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc65e9c3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65815d980 0x7fc65806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc65e9c3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc6580b2b70 0x7fc65806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc65e9c3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6580b3130 con 0x7fc658150df0 2024-04-25T00:35:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc65e9c3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc6580cf560 con 0x7fc6580b2b70 2024-04-25T00:35:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc65e9c3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc658151990 con 0x7fc65815d980 2024-04-25T00:35:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc6577fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc6580b2b70 0x7fc65806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc6577fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc6580b2b70 0x7fc65806db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48090/0 (socket says 172.21.15.29:48090) 2024-04-25T00:35:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc6577fe640 1 -- 172.21.15.29:0/3933288610 learned_addr learned my addr 172.21.15.29:0/3933288610 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc657fff640 1 --2- 172.21.15.29:0/3933288610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65815d980 0x7fc65806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc6577fe640 1 -- 172.21.15.29:0/3933288610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65815d980 msgr2=0x7fc65806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:00.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc6577fe640 1 --2- 172.21.15.29:0/3933288610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65815d980 0x7fc65806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc6577fe640 1 -- 172.21.15.29:0/3933288610 >> v1:172.21.15.157:6789/0 conn(0x7fc658150df0 legacy=0x7fc658151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:00.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc6577fe640 1 -- 172.21.15.29:0/3933288610 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6580b3410 con 0x7fc6580b2b70 2024-04-25T00:35:00.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.505+0000 7fc657fff640 1 --2- 172.21.15.29:0/3933288610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65815d980 0x7fc65806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc6577fe640 1 --2- 172.21.15.29:0/3933288610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6580b2b70 0x7fc65806db10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009870 tx=0x7fc64c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc23760f07af6db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc656ffd640 1 -- 172.21.15.29:0/3933288610 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0aa610 con 0x7fc6580b2b70 2024-04-25T00:35:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc656ffd640 1 -- 172.21.15.29:0/3933288610 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc64c0af070 con 0x7fc6580b2b70 2024-04-25T00:35:00.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc656ffd640 1 -- 172.21.15.29:0/3933288610 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0aa9d0 con 0x7fc6580b2b70 2024-04-25T00:35:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 -- 172.21.15.29:0/3933288610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6580b2b70 msgr2=0x7fc65806db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 --2- 172.21.15.29:0/3933288610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6580b2b70 0x7fc65806db10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009870 tx=0x7fc64c01c4f0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 -- 172.21.15.29:0/3933288610 shutdown_connections 2024-04-25T00:35:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 --2- 172.21.15.29:0/3933288610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6580b2b70 0x7fc65806db10 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 --2- 172.21.15.29:0/3933288610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65815d980 0x7fc65806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 -- 172.21.15.29:0/3933288610 >> 172.21.15.29:0/3933288610 conn(0x7fc6580b1870 msgr2=0x7fc6580b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 -- 172.21.15.29:0/3933288610 shutdown_connections 2024-04-25T00:35:00.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 -- 172.21.15.29:0/3933288610 wait complete. 2024-04-25T00:35:00.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 Processor -- start 2024-04-25T00:35:00.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 -- start start 2024-04-25T00:35:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6580b2b70 0x7fc65814b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc658150df0 0x7fc6580ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65815d980 0x7fc6581570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc65806cd40 con 0x7fc6580b2b70 2024-04-25T00:35:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc65815dd60 con 0x7fc65815d980 2024-04-25T00:35:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc65806cec0 con 0x7fc658150df0 2024-04-25T00:35:00.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc657fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6580b2b70 0x7fc65814b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc657fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6580b2b70 0x7fc65814b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48100/0 (socket says 172.21.15.29:48100) 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc657fff640 1 -- 172.21.15.29:0/2353962833 learned_addr learned my addr 172.21.15.29:0/2353962833 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc6577fe640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc658150df0 0x7fc6580ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65cf60640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65815d980 0x7fc6581570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc657fff640 1 -- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc658150df0 msgr2=0x7fc6580ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc657fff640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc658150df0 0x7fc6580ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc657fff640 1 -- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65815d980 msgr2=0x7fc6581570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc657fff640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65815d980 0x7fc6581570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc657fff640 1 -- 172.21.15.29:0/2353962833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6581f6d30 con 0x7fc6580b2b70 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc657fff640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6580b2b70 0x7fc65814b220 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc64800ce00 tx=0x7fc64800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc654ff9640 1 -- 172.21.15.29:0/2353962833 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64800f890 con 0x7fc6580b2b70 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc654ff9640 1 -- 172.21.15.29:0/2353962833 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc64800fa30 con 0x7fc6580b2b70 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 -- 172.21.15.29:0/2353962833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6580af420 con 0x7fc6580b2b70 2024-04-25T00:35:00.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc654ff9640 1 -- 172.21.15.29:0/2353962833 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64800fe40 con 0x7fc6580b2b70 2024-04-25T00:35:00.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.509+0000 7fc65e9c3640 1 -- 172.21.15.29:0/2353962833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc658156300 con 0x7fc6580b2b70 2024-04-25T00:35:00.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.513+0000 7fc65e9c3640 1 -- 172.21.15.29:0/2353962833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc658069270 con 0x7fc6580b2b70 2024-04-25T00:35:00.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.513+0000 7fc654ff9640 1 -- 172.21.15.29:0/2353962833 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fc648018400 con 0x7fc6580b2b70 2024-04-25T00:35:00.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.513+0000 7fc654ff9640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc644067a70 0x7fc644069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.513+0000 7fc654ff9640 1 -- 172.21.15.29:0/2353962833 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fc648086b10 con 0x7fc6580b2b70 2024-04-25T00:35:00.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.513+0000 7fc6577fe640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc644067a70 0x7fc644069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.513+0000 7fc6577fe640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc644067a70 0x7fc644069f30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc64c0ae000 tx=0x7fc64c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:00.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.517+0000 7fc654ff9640 1 -- 172.21.15.29:0/2353962833 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc648014030 con 0x7fc6580b2b70 2024-04-25T00:35:00.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:00.733+0000 7fc65e9c3640 1 -- 172.21.15.29:0/2353962833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fc6581661a0 con 0x7fc6580b2b70 2024-04-25T00:35:01.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.701+0000 7fc654ff9640 1 -- 172.21.15.29:0/2353962833 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v189) ==== 164+0+0 (secure 0 0 0) 0x7fc64800ae20 con 0x7fc6580b2b70 2024-04-25T00:35:01.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.705+0000 7fc65e9c3640 1 -- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc644067a70 msgr2=0x7fc644069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:01.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.705+0000 7fc65e9c3640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc644067a70 0x7fc644069f30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc64c0ae000 tx=0x7fc64c0a7000 comp rx=0 tx=0).stop 2024-04-25T00:35:01.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.705+0000 7fc65e9c3640 1 -- 172.21.15.29:0/2353962833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6580b2b70 msgr2=0x7fc65814b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:01.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.705+0000 7fc65e9c3640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6580b2b70 0x7fc65814b220 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc64800ce00 tx=0x7fc64800a650 comp rx=0 tx=0).stop 2024-04-25T00:35:01.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.705+0000 7fc65e9c3640 1 -- 172.21.15.29:0/2353962833 shutdown_connections 2024-04-25T00:35:01.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.705+0000 7fc65e9c3640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc644067a70 0x7fc644069f30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:01.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.705+0000 7fc65e9c3640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65815d980 0x7fc6581570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:01.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.705+0000 7fc65e9c3640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc658150df0 0x7fc6580ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:01.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.705+0000 7fc65e9c3640 1 --2- 172.21.15.29:0/2353962833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6580b2b70 0x7fc65814b220 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:01.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.705+0000 7fc65e9c3640 1 -- 172.21.15.29:0/2353962833 >> 172.21.15.29:0/2353962833 conn(0x7fc6580b1870 msgr2=0x7fc6580cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:01.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.705+0000 7fc65e9c3640 1 -- 172.21.15.29:0/2353962833 shutdown_connections 2024-04-25T00:35:01.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.705+0000 7fc65e9c3640 1 -- 172.21.15.29:0/2353962833 wait complete. 2024-04-25T00:35:01.730 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:35:01.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.861+0000 7fc7a190e640 1 Processor -- start 2024-04-25T00:35:01.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.861+0000 7fc7a190e640 1 -- start start 2024-04-25T00:35:01.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.861+0000 7fc7a190e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc79c15d980 0x7fc79c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:01.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.861+0000 7fc7a190e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc79c0b3130 con 0x7fc79c06db10 2024-04-25T00:35:01.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.861+0000 7fc7a190e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc79c0b3410 con 0x7fc79c0b2b70 2024-04-25T00:35:01.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.861+0000 7fc7a190e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc79c0cf560 con 0x7fc79c15d980 2024-04-25T00:35:01.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.861+0000 7fc79affd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc79c15d980 0x7fc79c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:01.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.861+0000 7fc79affd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc79c15d980 0x7fc79c06d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48102/0 (socket says 172.21.15.29:48102) 2024-04-25T00:35:01.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.861+0000 7fc79b7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc79c06db10 0x7fc79c15b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43434/0 (socket says 172.21.15.29:43434) 2024-04-25T00:35:01.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.861+0000 7fc79affd640 1 -- 172.21.15.29:0/2219428245 learned_addr learned my addr 172.21.15.29:0/2219428245 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:01.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.861+0000 7fc79affd640 1 -- 172.21.15.29:0/2219428245 >> v1:172.21.15.157:6789/0 conn(0x7fc79c0b2b70 legacy=0x7fc79c150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:01.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.861+0000 7fc79affd640 1 -- 172.21.15.29:0/2219428245 >> v1:172.21.15.29:6789/0 conn(0x7fc79c06db10 legacy=0x7fc79c15b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:01.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.861+0000 7fc79affd640 1 -- 172.21.15.29:0/2219428245 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c0b36f0 con 0x7fc79c15d980 2024-04-25T00:35:01.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc79affd640 1 --2- 172.21.15.29:0/2219428245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc79c15d980 0x7fc79c06d5d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc790009870 tx=0x7fc79001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871440aa68f446e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:01.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc799ffb640 1 -- 172.21.15.29:0/2219428245 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7900ab610 con 0x7fc79c15d980 2024-04-25T00:35:01.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc799ffb640 1 -- 172.21.15.29:0/2219428245 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7900b0070 con 0x7fc79c15d980 2024-04-25T00:35:01.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc799ffb640 1 -- 172.21.15.29:0/2219428245 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7900ab9d0 con 0x7fc79c15d980 2024-04-25T00:35:01.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 -- 172.21.15.29:0/2219428245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc79c15d980 msgr2=0x7fc79c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:01.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 --2- 172.21.15.29:0/2219428245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc79c15d980 0x7fc79c06d5d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc790009870 tx=0x7fc79001c520 comp rx=0 tx=0).stop 2024-04-25T00:35:01.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 -- 172.21.15.29:0/2219428245 shutdown_connections 2024-04-25T00:35:01.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 --2- 172.21.15.29:0/2219428245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc79c15d980 0x7fc79c06d5d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:01.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 -- 172.21.15.29:0/2219428245 >> 172.21.15.29:0/2219428245 conn(0x7fc79c0b1870 msgr2=0x7fc79c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:01.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 -- 172.21.15.29:0/2219428245 shutdown_connections 2024-04-25T00:35:01.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 -- 172.21.15.29:0/2219428245 wait complete. 2024-04-25T00:35:01.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 Processor -- start 2024-04-25T00:35:01.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 -- start start 2024-04-25T00:35:01.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79c06db10 0x7fc79c146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:01.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc79c0b2b70 0x7fc79c145350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:01.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79c15d980 0x7fc79c0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:01.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1683d0 con 0x7fc79c0b2b70 2024-04-25T00:35:01.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c15dd60 con 0x7fc79c06db10 2024-04-25T00:35:01.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc7a190e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc79c168550 con 0x7fc79c15d980 2024-04-25T00:35:01.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc79a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc79c0b2b70 0x7fc79c145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:01.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc79a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc79c0b2b70 0x7fc79c145350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48108/0 (socket says 172.21.15.29:48108) 2024-04-25T00:35:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.865+0000 7fc79a7fc640 1 -- 172.21.15.29:0/3475380619 learned_addr learned my addr 172.21.15.29:0/3475380619 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc79affd640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79c06db10 0x7fc79c146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc79a7fc640 1 -- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79c15d980 msgr2=0x7fc79c0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc79b7fe640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79c15d980 0x7fc79c0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc79a7fc640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79c15d980 0x7fc79c0cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc79a7fc640 1 -- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79c06db10 msgr2=0x7fc79c146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc79a7fc640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79c06db10 0x7fc79c146060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc79a7fc640 1 -- 172.21.15.29:0/3475380619 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c15abc0 con 0x7fc79c0b2b70 2024-04-25T00:35:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc79affd640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79c06db10 0x7fc79c146060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc79a7fc640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc79c0b2b70 0x7fc79c145350 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00c450 tx=0x7fc78c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc783fff640 1 -- 172.21.15.29:0/3475380619 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c012830 con 0x7fc79c0b2b70 2024-04-25T00:35:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc783fff640 1 -- 172.21.15.29:0/3475380619 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc78c017070 con 0x7fc79c0b2b70 2024-04-25T00:35:01.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc7a190e640 1 -- 172.21.15.29:0/3475380619 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79c1465a0 con 0x7fc79c0b2b70 2024-04-25T00:35:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc783fff640 1 -- 172.21.15.29:0/3475380619 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c012c30 con 0x7fc79c0b2b70 2024-04-25T00:35:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc7a190e640 1 -- 172.21.15.29:0/3475380619 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79c0cc870 con 0x7fc79c0b2b70 2024-04-25T00:35:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc7a190e640 1 -- 172.21.15.29:0/3475380619 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc79c069270 con 0x7fc79c0b2b70 2024-04-25T00:35:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc783fff640 1 -- 172.21.15.29:0/3475380619 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fc78c01c460 con 0x7fc79c0b2b70 2024-04-25T00:35:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc783fff640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc788067a70 0x7fc788069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc783fff640 1 -- 172.21.15.29:0/3475380619 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fc78c085a50 con 0x7fc79c0b2b70 2024-04-25T00:35:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.869+0000 7fc79affd640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc788067a70 0x7fc788069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:01.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.873+0000 7fc79affd640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc788067a70 0x7fc788069f30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc7900af000 tx=0x7fc790008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:01.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:01.873+0000 7fc783fff640 1 -- 172.21.15.29:0/3475380619 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc78c056610 con 0x7fc79c0b2b70 2024-04-25T00:35:02.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.037+0000 7fc7a190e640 1 -- 172.21.15.29:0/3475380619 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc79c1661a0 con 0x7fc79c0b2b70 2024-04-25T00:35:02.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.041+0000 7fc783fff640 1 -- 172.21.15.29:0/3475380619 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fc78c055f10 con 0x7fc79c0b2b70 2024-04-25T00:35:02.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.041+0000 7fc7a190e640 1 -- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc788067a70 msgr2=0x7fc788069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:02.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.041+0000 7fc7a190e640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc788067a70 0x7fc788069f30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc7900af000 tx=0x7fc790008040 comp rx=0 tx=0).stop 2024-04-25T00:35:02.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.041+0000 7fc7a190e640 1 -- 172.21.15.29:0/3475380619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc79c0b2b70 msgr2=0x7fc79c145350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:02.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.041+0000 7fc7a190e640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc79c0b2b70 0x7fc79c145350 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00c450 tx=0x7fc78c00c910 comp rx=0 tx=0).stop 2024-04-25T00:35:02.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.041+0000 7fc7a190e640 1 -- 172.21.15.29:0/3475380619 shutdown_connections 2024-04-25T00:35:02.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.041+0000 7fc7a190e640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79c15d980 0x7fc79c0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.041+0000 7fc7a190e640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc79c0b2b70 0x7fc79c145350 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.041+0000 7fc7a190e640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc788067a70 0x7fc788069f30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.041+0000 7fc7a190e640 1 --2- 172.21.15.29:0/3475380619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79c06db10 0x7fc79c146060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.041+0000 7fc7a190e640 1 -- 172.21.15.29:0/3475380619 >> 172.21.15.29:0/3475380619 conn(0x7fc79c0b1870 msgr2=0x7fc79c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:02.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.041+0000 7fc7a190e640 1 -- 172.21.15.29:0/3475380619 shutdown_connections 2024-04-25T00:35:02.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.041+0000 7fc7a190e640 1 -- 172.21.15.29:0/3475380619 wait complete. 2024-04-25T00:35:02.062 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:35:02.062 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:35:02.122 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:35:02.186 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:35:02.186 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:35:02.186 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:35:02.192 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:35:02.192 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:35:02.192 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:35:02.198 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:35:02.198 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:35:02.198 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:35:02.242 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:35:02.242 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:35:02.242 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:35:02.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:35:02.247 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:35:02.247 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:35:02.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.417+0000 7ff60f012640 1 Processor -- start 2024-04-25T00:35:02.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.417+0000 7ff60f012640 1 -- start start 2024-04-25T00:35:02.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.417+0000 7ff60f012640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6080b4e00 0x7ff6080b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.417+0000 7ff60f012640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff6080b30b0 0x7ff60806cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.417+0000 7ff60f012640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff608071690 con 0x7ff608162420 2024-04-25T00:35:02.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.417+0000 7ff60f012640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff6080b3500 con 0x7ff6080b30b0 2024-04-25T00:35:02.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.417+0000 7ff60f012640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff608071910 con 0x7ff6080b4e00 2024-04-25T00:35:02.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.417+0000 7ff60cdae640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff608162420 0x7ff608070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43446/0 (socket says 172.21.15.29:43446) 2024-04-25T00:35:02.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.417+0000 7ff60cdae640 1 -- 172.21.15.29:0/1579690522 learned_addr learned my addr 172.21.15.29:0/1579690522 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:02.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.417+0000 7ff60d5af640 1 --2- 172.21.15.29:0/1579690522 >> v2:172.21.15.157:3300/0 conn(0x7ff6080b30b0 0x7ff60806cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5fffff640 1 --2- 172.21.15.29:0/1579690522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6080b4e00 0x7ff6080b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1230308895 0 0) 0x7ff604001120 con 0x7ff608162420 2024-04-25T00:35:02.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5f0003530 con 0x7ff608162420 2024-04-25T00:35:02.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2821035175 0 0) 0x7ff5f0003530 con 0x7ff608162420 2024-04-25T00:35:02.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6080b4e00 msgr2=0x7ff6080b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:02.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 --2- 172.21.15.29:0/1579690522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6080b4e00 0x7ff6080b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 >> v2:172.21.15.157:3300/0 conn(0x7ff6080b30b0 msgr2=0x7ff60806cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:02.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 --2- 172.21.15.29:0/1579690522 >> v2:172.21.15.157:3300/0 conn(0x7ff6080b30b0 0x7ff60806cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff608071bf0 con 0x7ff608162420 2024-04-25T00:35:02.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ff604001850 con 0x7ff608162420 2024-04-25T00:35:02.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 >> v1:172.21.15.29:6789/0 conn(0x7ff608162420 legacy=0x7ff608070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:02.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 --2- 172.21.15.29:0/1579690522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6080b4e00 0x7ff5f00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 --2- 172.21.15.29:0/1579690522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f0003f40 0x7ff5f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 --2- 172.21.15.29:0/1579690522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f0004d50 0x7ff5f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff6080b3500 con 0x7ff5f0003f40 2024-04-25T00:35:02.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff608071910 con 0x7ff5f0004d50 2024-04-25T00:35:02.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f000b9d0 con 0x7ff6080b4e00 2024-04-25T00:35:02.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5fffff640 1 --2- 172.21.15.29:0/1579690522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f0003f40 0x7ff5f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff60cdae640 1 --2- 172.21.15.29:0/1579690522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6080b4e00 0x7ff5f00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff60d5af640 1 --2- 172.21.15.29:0/1579690522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f0004d50 0x7ff5f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5fffff640 1 -- 172.21.15.29:0/1579690522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6080b4e00 msgr2=0x7ff5f00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:02.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5fffff640 1 --2- 172.21.15.29:0/1579690522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6080b4e00 0x7ff5f00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5fffff640 1 -- 172.21.15.29:0/1579690522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f0004d50 msgr2=0x7ff5f00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:02.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5fffff640 1 --2- 172.21.15.29:0/1579690522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f0004d50 0x7ff5f00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5fffff640 1 -- 172.21.15.29:0/1579690522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f0003530 con 0x7ff5f0003f40 2024-04-25T00:35:02.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff60d5af640 1 --2- 172.21.15.29:0/1579690522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f0004d50 0x7ff5f00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:02.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5fffff640 1 --2- 172.21.15.29:0/1579690522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f0003f40 0x7ff5f0004810 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009a90 tx=0x7ff5f801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a1c64a7d3ece4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:02.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80b2070 con 0x7ff5f0003f40 2024-04-25T00:35:02.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f80a5620 con 0x7ff5f0003f40 2024-04-25T00:35:02.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.421+0000 7ff5ff7fe640 1 -- 172.21.15.29:0/1579690522 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8009690 con 0x7ff5f0003f40 2024-04-25T00:35:02.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 -- 172.21.15.29:0/1579690522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f0003f40 msgr2=0x7ff5f0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:02.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 --2- 172.21.15.29:0/1579690522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f0003f40 0x7ff5f0004810 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009a90 tx=0x7ff5f801c9d0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60d5af640 1 -- 172.21.15.29:0/1579690522 reap_dead start 2024-04-25T00:35:02.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 -- 172.21.15.29:0/1579690522 shutdown_connections 2024-04-25T00:35:02.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 -- 172.21.15.29:0/1579690522 >> 172.21.15.29:0/1579690522 conn(0x7ff6080b1870 msgr2=0x7ff6080b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:02.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 -- 172.21.15.29:0/1579690522 shutdown_connections 2024-04-25T00:35:02.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 -- 172.21.15.29:0/1579690522 wait complete. 2024-04-25T00:35:02.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 Processor -- start 2024-04-25T00:35:02.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 -- start start 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5f0003f40 0x7ff608149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6081f8330 0x7ff6081f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff608155030 0x7ff6081554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff608162800 con 0x7ff608155030 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff6080721b0 con 0x7ff6081f8330 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff6080b2b70 con 0x7ff5f0003f40 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60d5af640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff608155030 0x7ff6081554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60d5af640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff608155030 0x7ff6081554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48122/0 (socket says 172.21.15.29:48122) 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60d5af640 1 -- 172.21.15.29:0/1979442428 learned_addr learned my addr 172.21.15.29:0/1979442428 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff5fffff640 1 --2- 172.21.15.29:0/1979442428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6081f8330 0x7ff6081f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60d5af640 1 -- 172.21.15.29:0/1979442428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5f0003f40 msgr2=0x7ff608149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60d5af640 1 --2- 172.21.15.29:0/1979442428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5f0003f40 0x7ff608149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60d5af640 1 -- 172.21.15.29:0/1979442428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6081f8330 msgr2=0x7ff6081f87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60d5af640 1 --2- 172.21.15.29:0/1979442428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6081f8330 0x7ff6081f87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60d5af640 1 -- 172.21.15.29:0/1979442428 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60815d4c0 con 0x7ff608155030 2024-04-25T00:35:02.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60d5af640 1 --2- 172.21.15.29:0/1979442428 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff608155030 0x7ff6081554b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff6080cfac0 tx=0x7ff60000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:02.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff5fd7fa640 1 -- 172.21.15.29:0/1979442428 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60001c070 con 0x7ff608155030 2024-04-25T00:35:02.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff5fd7fa640 1 -- 172.21.15.29:0/1979442428 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff600017400 con 0x7ff608155030 2024-04-25T00:35:02.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff5fd7fa640 1 -- 172.21.15.29:0/1979442428 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6000177d0 con 0x7ff608155030 2024-04-25T00:35:02.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 -- 172.21.15.29:0/1979442428 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff608155ba0 con 0x7ff608155030 2024-04-25T00:35:02.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.425+0000 7ff60f012640 1 -- 172.21.15.29:0/1979442428 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60815aba0 con 0x7ff608155030 2024-04-25T00:35:02.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.429+0000 7ff60f012640 1 -- 172.21.15.29:0/1979442428 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff608069270 con 0x7ff608155030 2024-04-25T00:35:02.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.429+0000 7ff5fd7fa640 1 -- 172.21.15.29:0/1979442428 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7ff60001f070 con 0x7ff608155030 2024-04-25T00:35:02.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.429+0000 7ff5fd7fa640 1 --2- 172.21.15.29:0/1979442428 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff5f4067aa0 0x7ff5f4069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.429+0000 7ff5fd7fa640 1 -- 172.21.15.29:0/1979442428 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7ff600085f70 con 0x7ff608155030 2024-04-25T00:35:02.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.429+0000 7ff60cdae640 1 --2- 172.21.15.29:0/1979442428 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff5f4067aa0 0x7ff5f4069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.429+0000 7ff60cdae640 1 --2- 172.21.15.29:0/1979442428 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff5f4067aa0 0x7ff5f4069f60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff604004880 tx=0x7ff6040047f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:02.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.433+0000 7ff5fd7fa640 1 -- 172.21.15.29:0/1979442428 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff600012070 con 0x7ff608155030 2024-04-25T00:35:02.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.629+0000 7ff60f012640 1 -- 172.21.15.29:0/1979442428 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff608067230 con 0x7ff608155030 2024-04-25T00:35:02.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.633+0000 7ff5fd7fa640 1 -- 172.21.15.29:0/1979442428 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+3872 (secure 0 0 0) 0x7ff600056a20 con 0x7ff608155030 2024-04-25T00:35:02.634 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:02.634 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2689633119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2689633119},{"type":"v1","addr":"172.21.15.157:6837","nonce":2689633119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/263626273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":263626273},{"type":"v1","addr":"172.21.15.157:6835","nonce":263626273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4096505822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4096505822},{"type":"v1","addr":"172.21.15.29:6834","nonce":4096505822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":6282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3893843666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3893843666},{"type":"v1","addr":"172.21.15.29:6835","nonce":3893843666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[]} 2024-04-25T00:35:02.634 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 27 2024-04-25T00:35:02.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.633+0000 7ff60f012640 1 -- 172.21.15.29:0/1979442428 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff5f4067aa0 msgr2=0x7ff5f4069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:02.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.633+0000 7ff60f012640 1 --2- 172.21.15.29:0/1979442428 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff5f4067aa0 0x7ff5f4069f60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff604004880 tx=0x7ff6040047f0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.633+0000 7ff60f012640 1 -- 172.21.15.29:0/1979442428 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff608155030 msgr2=0x7ff6081554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:02.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.633+0000 7ff60f012640 1 --2- 172.21.15.29:0/1979442428 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff608155030 0x7ff6081554b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff6080cfac0 tx=0x7ff60000f4a0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.633+0000 7ff60f012640 1 -- 172.21.15.29:0/1979442428 shutdown_connections 2024-04-25T00:35:02.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.633+0000 7ff60f012640 1 --2- 172.21.15.29:0/1979442428 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff608155030 0x7ff6081554b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.633+0000 7ff60f012640 1 --2- 172.21.15.29:0/1979442428 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff5f4067aa0 0x7ff5f4069f60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.633+0000 7ff60f012640 1 --2- 172.21.15.29:0/1979442428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6081f8330 0x7ff6081f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.633+0000 7ff60f012640 1 --2- 172.21.15.29:0/1979442428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5f0003f40 0x7ff608149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.633+0000 7ff60f012640 1 -- 172.21.15.29:0/1979442428 >> 172.21.15.29:0/1979442428 conn(0x7ff6080b1870 msgr2=0x7ff608165790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:02.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.633+0000 7ff60f012640 1 -- 172.21.15.29:0/1979442428 shutdown_connections 2024-04-25T00:35:02.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.633+0000 7ff60f012640 1 -- 172.21.15.29:0/1979442428 wait complete. 2024-04-25T00:35:02.655 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:35:02.655 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:35:02.655 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:35:02.655 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:35:02.655 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:35:02.655 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:35:02.656 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:35:02.656 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:35:02.656 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:35:02.656 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:35:02.656 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:35:02.656 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:35:02.656 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:35:02.657 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:35:02.657 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:35:02.657 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:35:02.667 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:35:02.667 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:35:02.668 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:35:02.668 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:35:02.673 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:35:02.673 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:35:02.674 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:35:02.674 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:35:02.714 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:35:02.715 DEBUG:teuthology.parallel:result is None 2024-04-25T00:35:02.715 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:35:02.715 DEBUG:teuthology.parallel:result is None 2024-04-25T00:35:02.717 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:35:02.717 DEBUG:teuthology.parallel:result is None 2024-04-25T00:35:02.718 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:35:02.718 DEBUG:teuthology.parallel:result is None 2024-04-25T00:35:02.718 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:35:02.778 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:35:02.785 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:35:02.786 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-25T00:35:02.787 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-25T00:35:02.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.841+0000 7fdcc26a7640 1 Processor -- start 2024-04-25T00:35:02.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.841+0000 7fdcc26a7640 1 -- start start 2024-04-25T00:35:02.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.841+0000 7fdcc26a7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdcbc1555b0 0x7fdcbc15e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.841+0000 7fdcc26a7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdcbc15bfb0 0x7fdcbc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.841+0000 7fdcc26a7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcbc15f170 con 0x7fdcbc0b5000 2024-04-25T00:35:02.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.841+0000 7fdcc26a7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdcbc15f450 con 0x7fdcbc1555b0 2024-04-25T00:35:02.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.841+0000 7fdcc26a7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdcbc15f730 con 0x7fdcbc15bfb0 2024-04-25T00:35:02.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.841+0000 7fdcbbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdcbc1555b0 0x7fdcbc15e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.841+0000 7fdcc0c44640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdcbc0b5000 0x7fdcbc0cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43468/0 (socket says 172.21.15.29:43468) 2024-04-25T00:35:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.841+0000 7fdcc0c44640 1 -- 172.21.15.29:0/1870078872 learned_addr learned my addr 172.21.15.29:0/1870078872 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcbb7fe640 1 --2- 172.21.15.29:0/1870078872 >> v2:172.21.15.157:3300/0 conn(0x7fdcbc15bfb0 0x7fdcbc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcbbfff640 1 -- 172.21.15.29:0/1870078872 >> v2:172.21.15.157:3300/0 conn(0x7fdcbc15bfb0 msgr2=0x7fdcbc06d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcbbfff640 1 --2- 172.21.15.29:0/1870078872 >> v2:172.21.15.157:3300/0 conn(0x7fdcbc15bfb0 0x7fdcbc06d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcbbfff640 1 -- 172.21.15.29:0/1870078872 >> v1:172.21.15.29:6789/0 conn(0x7fdcbc0b5000 legacy=0x7fdcbc0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:02.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcbbfff640 1 -- 172.21.15.29:0/1870078872 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc0cef00 con 0x7fdcbc1555b0 2024-04-25T00:35:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcbbfff640 1 --2- 172.21.15.29:0/1870078872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc1555b0 0x7fdcbc15e920 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc072830 tx=0x7fdcac01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6633a08d7616827b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcbaffd640 1 -- 172.21.15.29:0/1870078872 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac0ab640 con 0x7fdcbc1555b0 2024-04-25T00:35:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcbaffd640 1 -- 172.21.15.29:0/1870078872 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcac0b0070 con 0x7fdcbc1555b0 2024-04-25T00:35:02.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcbaffd640 1 -- 172.21.15.29:0/1870078872 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac0aba00 con 0x7fdcbc1555b0 2024-04-25T00:35:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcc26a7640 1 -- 172.21.15.29:0/1870078872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc1555b0 msgr2=0x7fdcbc15e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcc26a7640 1 --2- 172.21.15.29:0/1870078872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc1555b0 0x7fdcbc15e920 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc072830 tx=0x7fdcac01c420 comp rx=0 tx=0).stop 2024-04-25T00:35:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcc26a7640 1 -- 172.21.15.29:0/1870078872 shutdown_connections 2024-04-25T00:35:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcc26a7640 1 --2- 172.21.15.29:0/1870078872 >> v2:172.21.15.157:3300/0 conn(0x7fdcbc15bfb0 0x7fdcbc06d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcc26a7640 1 --2- 172.21.15.29:0/1870078872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc1555b0 0x7fdcbc15e920 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcc26a7640 1 -- 172.21.15.29:0/1870078872 >> 172.21.15.29:0/1870078872 conn(0x7fdcbc075030 msgr2=0x7fdcbc0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:02.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcc26a7640 1 -- 172.21.15.29:0/1870078872 shutdown_connections 2024-04-25T00:35:02.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.845+0000 7fdcc26a7640 1 -- 172.21.15.29:0/1870078872 wait complete. 2024-04-25T00:35:02.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcc26a7640 1 Processor -- start 2024-04-25T00:35:02.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcc26a7640 1 -- start start 2024-04-25T00:35:02.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcc26a7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc0b5000 0x7fdcbc0a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcc26a7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc1555b0 0x7fdcbc144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcc26a7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcbc15bfb0 0x7fdcbc156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcc26a7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc06d9f0 con 0x7fdcbc1555b0 2024-04-25T00:35:02.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcc26a7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc06d870 con 0x7fdcbc0b5000 2024-04-25T00:35:02.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcc26a7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdcbc1684d0 con 0x7fdcbc15bfb0 2024-04-25T00:35:02.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcbb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc1555b0 0x7fdcbc144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcbb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc1555b0 0x7fdcbc144eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48194/0 (socket says 172.21.15.29:48194) 2024-04-25T00:35:02.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcbb7fe640 1 -- 172.21.15.29:0/635710482 learned_addr learned my addr 172.21.15.29:0/635710482 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:02.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcbbfff640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc0b5000 0x7fdcbc0a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcc0c44640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcbc15bfb0 0x7fdcbc156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcbb7fe640 1 -- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcbc15bfb0 msgr2=0x7fdcbc156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:02.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcbb7fe640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcbc15bfb0 0x7fdcbc156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcbb7fe640 1 -- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc0b5000 msgr2=0x7fdcbc0a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcbb7fe640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc0b5000 0x7fdcbc0a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcbb7fe640 1 -- 172.21.15.29:0/635710482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc15a860 con 0x7fdcbc1555b0 2024-04-25T00:35:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcbbfff640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc0b5000 0x7fdcbc0a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcbb7fe640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc1555b0 0x7fdcbc144eb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000d5f0 tx=0x7fdcb000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcb8ff9640 1 -- 172.21.15.29:0/635710482 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00125f0 con 0x7fdcbc1555b0 2024-04-25T00:35:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcb8ff9640 1 -- 172.21.15.29:0/635710482 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb0019070 con 0x7fdcbc1555b0 2024-04-25T00:35:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcc26a7640 1 -- 172.21.15.29:0/635710482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcbc164020 con 0x7fdcbc1555b0 2024-04-25T00:35:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcb8ff9640 1 -- 172.21.15.29:0/635710482 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00129b0 con 0x7fdcbc1555b0 2024-04-25T00:35:02.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcc26a7640 1 -- 172.21.15.29:0/635710482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcbc1645e0 con 0x7fdcbc1555b0 2024-04-25T00:35:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcc26a7640 1 -- 172.21.15.29:0/635710482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcbc069270 con 0x7fdcbc1555b0 2024-04-25T00:35:02.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.849+0000 7fdcb8ff9640 1 -- 172.21.15.29:0/635710482 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fdcb0012b50 con 0x7fdcbc1555b0 2024-04-25T00:35:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.853+0000 7fdcb8ff9640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdca8067a70 0x7fdca8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.853+0000 7fdcb8ff9640 1 -- 172.21.15.29:0/635710482 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fdcb0085f50 con 0x7fdcbc1555b0 2024-04-25T00:35:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.853+0000 7fdcbbfff640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdca8067a70 0x7fdca8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:02.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.853+0000 7fdcbbfff640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdca8067a70 0x7fdca8069f30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0af000 tx=0x7fdcac0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:02.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:02.857+0000 7fdcb8ff9640 1 -- 172.21.15.29:0/635710482 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdcb0056ba0 con 0x7fdcbc1555b0 2024-04-25T00:35:03.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.029+0000 7fdcc26a7640 1 -- 172.21.15.29:0/635710482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fdcbc0afe20 con 0x7fdcbc1555b0 2024-04-25T00:35:03.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.721+0000 7fdcb8ff9640 1 -- 172.21.15.29:0/635710482 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v52) ==== 97+0+0 (secure 0 0 0) 0x7fdcb00564a0 con 0x7fdcbc1555b0 2024-04-25T00:35:03.722 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-04-25T00:35:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.725+0000 7fdcc26a7640 1 -- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdca8067a70 msgr2=0x7fdca8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.725+0000 7fdcc26a7640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdca8067a70 0x7fdca8069f30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0af000 tx=0x7fdcac0a7000 comp rx=0 tx=0).stop 2024-04-25T00:35:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.725+0000 7fdcc26a7640 1 -- 172.21.15.29:0/635710482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc1555b0 msgr2=0x7fdcbc144eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.725+0000 7fdcc26a7640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc1555b0 0x7fdcbc144eb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000d5f0 tx=0x7fdcb000dab0 comp rx=0 tx=0).stop 2024-04-25T00:35:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.725+0000 7fdcc26a7640 1 -- 172.21.15.29:0/635710482 shutdown_connections 2024-04-25T00:35:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.725+0000 7fdcc26a7640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcbc15bfb0 0x7fdcbc156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.725+0000 7fdcc26a7640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc1555b0 0x7fdcbc144eb0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.725+0000 7fdcc26a7640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdca8067a70 0x7fdca8069f30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.725+0000 7fdcc26a7640 1 --2- 172.21.15.29:0/635710482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc0b5000 0x7fdcbc0a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.725+0000 7fdcc26a7640 1 -- 172.21.15.29:0/635710482 >> 172.21.15.29:0/635710482 conn(0x7fdcbc075030 msgr2=0x7fdcbc06dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.725+0000 7fdcc26a7640 1 -- 172.21.15.29:0/635710482 shutdown_connections 2024-04-25T00:35:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.725+0000 7fdcc26a7640 1 -- 172.21.15.29:0/635710482 wait complete. 2024-04-25T00:35:03.750 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:35:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.877+0000 7fe315ec9640 1 Processor -- start 2024-04-25T00:35:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.877+0000 7fe315ec9640 1 -- start start 2024-04-25T00:35:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.877+0000 7fe315ec9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe31015e060 0x7fe31015c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.877+0000 7fe315ec9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe31015e850 con 0x7fe31015bce0 2024-04-25T00:35:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.877+0000 7fe315ec9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe31015eaf0 con 0x7fe310072490 2024-04-25T00:35:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.877+0000 7fe315ec9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe31015f700 con 0x7fe31015e060 2024-04-25T00:35:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.877+0000 7fe30f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe31015e060 0x7fe31015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.877+0000 7fe30f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe31015e060 0x7fe31015c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48208/0 (socket says 172.21.15.29:48208) 2024-04-25T00:35:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.877+0000 7fe30f7fe640 1 -- 172.21.15.29:0/1603276627 learned_addr learned my addr 172.21.15.29:0/1603276627 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.877+0000 7fe30f7fe640 1 -- 172.21.15.29:0/1603276627 >> v1:172.21.15.157:6789/0 conn(0x7fe310072490 legacy=0x7fe3100cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe30f7fe640 1 -- 172.21.15.29:0/1603276627 >> v1:172.21.15.29:6789/0 conn(0x7fe31015bce0 legacy=0x7fe310071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe30f7fe640 1 -- 172.21.15.29:0/1603276627 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31015edd0 con 0x7fe31015e060 2024-04-25T00:35:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe30e7fc640 1 -- 172.21.15.29:0/1603276627 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 813129700 0 0) 0x7fe31015e850 con 0x7fe31015bce0 2024-04-25T00:35:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe30f7fe640 1 --2- 172.21.15.29:0/1603276627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe31015e060 0x7fe31015c420 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe30400bb10 tx=0x7fe30401eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c2e81c11213b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe30e7fc640 1 -- 172.21.15.29:0/1603276627 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3040b4070 con 0x7fe31015e060 2024-04-25T00:35:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe30e7fc640 1 -- 172.21.15.29:0/1603276627 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3040a7bf0 con 0x7fe31015e060 2024-04-25T00:35:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe30e7fc640 1 -- 172.21.15.29:0/1603276627 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3040af440 con 0x7fe31015e060 2024-04-25T00:35:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe315ec9640 1 -- 172.21.15.29:0/1603276627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe31015e060 msgr2=0x7fe31015c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe315ec9640 1 --2- 172.21.15.29:0/1603276627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe31015e060 0x7fe31015c420 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe30400bb10 tx=0x7fe30401eb90 comp rx=0 tx=0).stop 2024-04-25T00:35:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe315ec9640 1 -- 172.21.15.29:0/1603276627 shutdown_connections 2024-04-25T00:35:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe315ec9640 1 --2- 172.21.15.29:0/1603276627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe31015e060 0x7fe31015c420 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe315ec9640 1 -- 172.21.15.29:0/1603276627 >> 172.21.15.29:0/1603276627 conn(0x7fe31006c320 msgr2=0x7fe31006c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe315ec9640 1 -- 172.21.15.29:0/1603276627 shutdown_connections 2024-04-25T00:35:03.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.881+0000 7fe315ec9640 1 -- 172.21.15.29:0/1603276627 wait complete. 2024-04-25T00:35:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe315ec9640 1 Processor -- start 2024-04-25T00:35:03.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe315ec9640 1 -- start start 2024-04-25T00:35:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe315ec9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe310072490 0x7fe3100b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe315ec9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe31015bce0 0x7fe3100c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe315ec9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe31015e060 0x7fe3100c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe315ec9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe31015e5c0 con 0x7fe31015bce0 2024-04-25T00:35:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe315ec9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe31015e440 con 0x7fe31015e060 2024-04-25T00:35:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe315ec9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe31006f610 con 0x7fe310072490 2024-04-25T00:35:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe30effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe31015bce0 0x7fe3100c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe30effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe31015bce0 0x7fe3100c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48224/0 (socket says 172.21.15.29:48224) 2024-04-25T00:35:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe30effd640 1 -- 172.21.15.29:0/1671749830 learned_addr learned my addr 172.21.15.29:0/1671749830 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe30ffff640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe31015e060 0x7fe3100c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe30f7fe640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe310072490 0x7fe3100b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe30effd640 1 -- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe310072490 msgr2=0x7fe3100b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe30effd640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe310072490 0x7fe3100b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe30effd640 1 -- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe31015e060 msgr2=0x7fe3100c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe30effd640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe31015e060 0x7fe3100c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe30effd640 1 -- 172.21.15.29:0/1671749830 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3100c5560 con 0x7fe31015bce0 2024-04-25T00:35:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe30ffff640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe31015e060 0x7fe3100c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe30effd640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe31015bce0 0x7fe3100c81f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe30000c440 tx=0x7fe30000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe2f7fff640 1 -- 172.21.15.29:0/1671749830 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300005480 con 0x7fe31015bce0 2024-04-25T00:35:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe2f7fff640 1 -- 172.21.15.29:0/1671749830 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe300003600 con 0x7fe31015bce0 2024-04-25T00:35:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe2f7fff640 1 -- 172.21.15.29:0/1671749830 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3000039f0 con 0x7fe31015bce0 2024-04-25T00:35:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe315ec9640 1 -- 172.21.15.29:0/1671749830 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3100c5840 con 0x7fe31015bce0 2024-04-25T00:35:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe315ec9640 1 -- 172.21.15.29:0/1671749830 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3100c5e00 con 0x7fe31015bce0 2024-04-25T00:35:03.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe315ec9640 1 -- 172.21.15.29:0/1671749830 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe310069270 con 0x7fe31015bce0 2024-04-25T00:35:03.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.885+0000 7fe2f7fff640 1 -- 172.21.15.29:0/1671749830 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fe300003b90 con 0x7fe31015bce0 2024-04-25T00:35:03.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.889+0000 7fe2f7fff640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe2fc067a70 0x7fe2fc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:03.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.889+0000 7fe2f7fff640 1 -- 172.21.15.29:0/1671749830 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fe300085820 con 0x7fe31015bce0 2024-04-25T00:35:03.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.889+0000 7fe30f7fe640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe2fc067a70 0x7fe2fc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:03.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.889+0000 7fe30f7fe640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe2fc067a70 0x7fe2fc069f30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe3040b1000 tx=0x7fe3040a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:03.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:03.893+0000 7fe2f7fff640 1 -- 172.21.15.29:0/1671749830 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe300055de0 con 0x7fe31015bce0 2024-04-25T00:35:04.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.117+0000 7fe315ec9640 1 -- 172.21.15.29:0/1671749830 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe3100cf3b0 con 0x7fe31015bce0 2024-04-25T00:35:04.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.117+0000 7fe2f7fff640 1 -- 172.21.15.29:0/1671749830 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fe30005a0e0 con 0x7fe31015bce0 2024-04-25T00:35:04.120 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:04.120 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:35:04.120 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-04-25T00:35:04.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmyzgKAxAAU+nrUJgQhDX6xlARNFI/ng==", 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmtyzPERAAxNUDANw1H/ZzOo+dhgzxlg==", 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:04.121 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:35:04.122 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:35:04.122 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.122 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.122 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-04-25T00:35:04.122 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmYUl8CxAAj6pV1+s4a0rpWLq2EgZ+8w==", 2024-04-25T00:35:04.122 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.122 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:04.122 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:35:04.122 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:35:04.122 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.122 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmuAhKGhAAOLIcWUodW5TqtVgHWZl12A==", 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAzpClmudk6MhAA6YFyd57jwtB2pdOaad7bxQ==", 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.123 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:04.124 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:04.124 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.124 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.124 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-04-25T00:35:04.124 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA2pClm76q0NRAAxWhEP2ZGoFnZcuiQyGjglw==", 2024-04-25T00:35:04.124 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.124 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:04.124 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:04.124 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.124 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.125 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-04-25T00:35:04.125 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA6pClmcgULABAA2F5WBR9Df0v4Sn875AzF7g==", 2024-04-25T00:35:04.125 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.125 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:04.125 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:04.125 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.125 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.125 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-04-25T00:35:04.125 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA9pClm9QuRBhAALFLruv4JU/uNZYMX0yNuXw==", 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBBpClmNj4VDBAAxaxMvyaCvjynPY3f81udnw==", 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBEpClmzBybEBAArry1lM4RIsM/txKfgBqDpw==", 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBHpClm5HfjFhAAbFwJCpbhegLWDu0mbS8FoA==", 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.127 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBKpClme/7oHBAAOR2iQ6CweexR7sC1PgesZw==", 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A==", 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm8W/EJxAA0HZW24Hhcwf+soSZn6jHJg==", 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:04.129 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmGuQJJRAAhVd4D0jc6Whgyf+B6xgFMw==", 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmn+ciLRAAg4tQuZ13WP0RnWJaJq5fUQ==", 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmPVihJBAA7PQge/teyRZKSkBnufZSZw==", 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.131 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClm6WrdOhAAsY9FLCxG7PGpq2NCY4hGhQ==", 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:04.132 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-04-25T00:35:04.133 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmeR4UOBAANs9SRolOm6E8+/4ciupk5w==", 2024-04-25T00:35:04.133 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:04.133 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:35:04.133 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:35:04.133 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:04.133 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.133 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:04.133 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:35:04.133 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:35:04.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.121+0000 7fe315ec9640 1 -- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe2fc067a70 msgr2=0x7fe2fc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.121+0000 7fe315ec9640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe2fc067a70 0x7fe2fc069f30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe3040b1000 tx=0x7fe3040a9070 comp rx=0 tx=0).stop 2024-04-25T00:35:04.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.121+0000 7fe315ec9640 1 -- 172.21.15.29:0/1671749830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe31015bce0 msgr2=0x7fe3100c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.121+0000 7fe315ec9640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe31015bce0 0x7fe3100c81f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe30000c440 tx=0x7fe30000c900 comp rx=0 tx=0).stop 2024-04-25T00:35:04.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.121+0000 7fe315ec9640 1 -- 172.21.15.29:0/1671749830 shutdown_connections 2024-04-25T00:35:04.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.121+0000 7fe315ec9640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe2fc067a70 0x7fe2fc069f30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.121+0000 7fe315ec9640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe31015e060 0x7fe3100c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.121+0000 7fe315ec9640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe31015bce0 0x7fe3100c81f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.121+0000 7fe315ec9640 1 --2- 172.21.15.29:0/1671749830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe310072490 0x7fe3100b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.121+0000 7fe315ec9640 1 -- 172.21.15.29:0/1671749830 >> 172.21.15.29:0/1671749830 conn(0x7fe31006c320 msgr2=0x7fe310066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:04.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.121+0000 7fe315ec9640 1 -- 172.21.15.29:0/1671749830 shutdown_connections 2024-04-25T00:35:04.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.121+0000 7fe315ec9640 1 -- 172.21.15.29:0/1671749830 wait complete. 2024-04-25T00:35:04.140 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:04.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.257+0000 7fd66fc67640 1 Processor -- start 2024-04-25T00:35:04.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.257+0000 7fd66fc67640 1 -- start start 2024-04-25T00:35:04.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66fc67640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd6680cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66fc67640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6680b3720 con 0x7fd66815e8d0 2024-04-25T00:35:04.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66fc67640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6680b39c0 con 0x7fd6680b4780 2024-04-25T00:35:04.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66fc67640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd6680b42f0 con 0x7fd66815f3c0 2024-04-25T00:35:04.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66da03640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd66815e8d0 0x7fd66815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43484/0 (socket says 172.21.15.29:43484) 2024-04-25T00:35:04.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66da03640 1 -- 172.21.15.29:0/764327242 learned_addr learned my addr 172.21.15.29:0/764327242 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:04.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66e204640 1 --2- 172.21.15.29:0/764327242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd6680cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66ca01640 1 -- 172.21.15.29:0/764327242 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2183658617 0 0) 0x7fd6680b3720 con 0x7fd66815e8d0 2024-04-25T00:35:04.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66ca01640 1 -- 172.21.15.29:0/764327242 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd654005740 con 0x7fd66815e8d0 2024-04-25T00:35:04.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66ca01640 1 -- 172.21.15.29:0/764327242 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31458317 0 0) 0x7fd6680b39c0 con 0x7fd6680b4780 2024-04-25T00:35:04.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66ca01640 1 -- 172.21.15.29:0/764327242 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6680b3720 con 0x7fd6680b4780 2024-04-25T00:35:04.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66e204640 1 -- 172.21.15.29:0/764327242 >> v1:172.21.15.157:6789/0 conn(0x7fd6680b4780 legacy=0x7fd66815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66e204640 1 -- 172.21.15.29:0/764327242 >> v1:172.21.15.29:6789/0 conn(0x7fd66815e8d0 legacy=0x7fd66815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.261+0000 7fd66e204640 1 -- 172.21.15.29:0/764327242 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6680b3ca0 con 0x7fd66815f3c0 2024-04-25T00:35:04.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66e204640 1 --2- 172.21.15.29:0/764327242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd6680cc240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd660009dc0 tx=0x7fd66001c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ba235067e3806e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:04.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66ca01640 1 -- 172.21.15.29:0/764327242 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6600b2070 con 0x7fd66815f3c0 2024-04-25T00:35:04.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66ca01640 1 -- 172.21.15.29:0/764327242 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6600a5a00 con 0x7fd66815f3c0 2024-04-25T00:35:04.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66ca01640 1 -- 172.21.15.29:0/764327242 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6600a5db0 con 0x7fd66815f3c0 2024-04-25T00:35:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 -- 172.21.15.29:0/764327242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 msgr2=0x7fd6680cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 --2- 172.21.15.29:0/764327242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd6680cc240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd660009dc0 tx=0x7fd66001c960 comp rx=0 tx=0).stop 2024-04-25T00:35:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 -- 172.21.15.29:0/764327242 shutdown_connections 2024-04-25T00:35:04.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 --2- 172.21.15.29:0/764327242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd6680cc240 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 -- 172.21.15.29:0/764327242 >> 172.21.15.29:0/764327242 conn(0x7fd6680b1870 msgr2=0x7fd6680b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:04.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 -- 172.21.15.29:0/764327242 shutdown_connections 2024-04-25T00:35:04.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 -- 172.21.15.29:0/764327242 wait complete. 2024-04-25T00:35:04.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 Processor -- start 2024-04-25T00:35:04.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 -- start start 2024-04-25T00:35:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6680b4780 0x7fd668158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd66815e8d0 0x7fd66815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd66813b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd66806d710 con 0x7fd66815e8d0 2024-04-25T00:35:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd66806d590 con 0x7fd6680b4780 2024-04-25T00:35:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66fc67640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd66806d890 con 0x7fd66815f3c0 2024-04-25T00:35:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66d202640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd66815e8d0 0x7fd66815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66d202640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd66815e8d0 0x7fd66815a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48228/0 (socket says 172.21.15.29:48228) 2024-04-25T00:35:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66d202640 1 -- 172.21.15.29:0/2476914595 learned_addr learned my addr 172.21.15.29:0/2476914595 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66e204640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd66813b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66da03640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6680b4780 0x7fd668158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66d202640 1 -- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 msgr2=0x7fd66813b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66d202640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd66813b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66d202640 1 -- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6680b4780 msgr2=0x7fd668158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66d202640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6680b4780 0x7fd668158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.265+0000 7fd66d202640 1 -- 172.21.15.29:0/2476914595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66813b9d0 con 0x7fd66815e8d0 2024-04-25T00:35:04.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.269+0000 7fd66d202640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd66815e8d0 0x7fd66815a9d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c500 tx=0x7fd66400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:04.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.269+0000 7fd65e7fc640 1 -- 172.21.15.29:0/2476914595 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6640052d0 con 0x7fd66815e8d0 2024-04-25T00:35:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.269+0000 7fd65e7fc640 1 -- 172.21.15.29:0/2476914595 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd664005490 con 0x7fd66815e8d0 2024-04-25T00:35:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.269+0000 7fd66fc67640 1 -- 172.21.15.29:0/2476914595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd668158e60 con 0x7fd66815e8d0 2024-04-25T00:35:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.269+0000 7fd65e7fc640 1 -- 172.21.15.29:0/2476914595 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664012850 con 0x7fd66815e8d0 2024-04-25T00:35:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.269+0000 7fd66fc67640 1 -- 172.21.15.29:0/2476914595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6681581d0 con 0x7fd66815e8d0 2024-04-25T00:35:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.269+0000 7fd66fc67640 1 -- 172.21.15.29:0/2476914595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd668069270 con 0x7fd66815e8d0 2024-04-25T00:35:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.269+0000 7fd65e7fc640 1 -- 172.21.15.29:0/2476914595 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fd6640129f0 con 0x7fd66815e8d0 2024-04-25T00:35:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.269+0000 7fd65e7fc640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd64c067a70 0x7fd64c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.269+0000 7fd65e7fc640 1 -- 172.21.15.29:0/2476914595 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fd6640858d0 con 0x7fd66815e8d0 2024-04-25T00:35:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.269+0000 7fd66da03640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd64c067a70 0x7fd64c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.269+0000 7fd66da03640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd64c067a70 0x7fd64c069f30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd6580011f0 tx=0x7fd658009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.273+0000 7fd65e7fc640 1 -- 172.21.15.29:0/2476914595 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd6640564f0 con 0x7fd66815e8d0 2024-04-25T00:35:04.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.433+0000 7fd66fc67640 1 -- 172.21.15.29:0/2476914595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd66815af60 con 0x7fd66815e8d0 2024-04-25T00:35:04.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.437+0000 7fd65e7fc640 1 -- 172.21.15.29:0/2476914595 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+12843 (secure 0 0 0) 0x7fd664055df0 con 0x7fd66815e8d0 2024-04-25T00:35:04.438 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:04.438 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":54,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:03.732134+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:04.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.437+0000 7fd66fc67640 1 -- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd64c067a70 msgr2=0x7fd64c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.437+0000 7fd66fc67640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd64c067a70 0x7fd64c069f30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd6580011f0 tx=0x7fd658009040 comp rx=0 tx=0).stop 2024-04-25T00:35:04.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.437+0000 7fd66fc67640 1 -- 172.21.15.29:0/2476914595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd66815e8d0 msgr2=0x7fd66815a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.437+0000 7fd66fc67640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd66815e8d0 0x7fd66815a9d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c500 tx=0x7fd66400c9c0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.437+0000 7fd66fc67640 1 -- 172.21.15.29:0/2476914595 shutdown_connections 2024-04-25T00:35:04.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.437+0000 7fd66fc67640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd66813b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.437+0000 7fd66fc67640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd66815e8d0 0x7fd66815a9d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.437+0000 7fd66fc67640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd64c067a70 0x7fd64c069f30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.437+0000 7fd66fc67640 1 --2- 172.21.15.29:0/2476914595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6680b4780 0x7fd668158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.437+0000 7fd66fc67640 1 -- 172.21.15.29:0/2476914595 >> 172.21.15.29:0/2476914595 conn(0x7fd6680b1870 msgr2=0x7fd6681386f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:04.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.437+0000 7fd66fc67640 1 -- 172.21.15.29:0/2476914595 shutdown_connections 2024-04-25T00:35:04.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.437+0000 7fd66fc67640 1 -- 172.21.15.29:0/2476914595 wait complete. 2024-04-25T00:35:04.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:35:04.459 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:35:04.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.589+0000 7ff4ccdf7640 1 Processor -- start 2024-04-25T00:35:04.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.589+0000 7ff4ccdf7640 1 -- start start 2024-04-25T00:35:04.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.589+0000 7ff4ccdf7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff4c806b920 0x7ff4c80cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.589+0000 7ff4ccdf7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c8160190 con 0x7ff4c815ce60 2024-04-25T00:35:04.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.589+0000 7ff4ccdf7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c815c680 con 0x7ff4c8156420 2024-04-25T00:35:04.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.589+0000 7ff4ccdf7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c815d2f0 con 0x7ff4c806b920 2024-04-25T00:35:04.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.589+0000 7ff4c6d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff4c806b920 0x7ff4c80cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.589+0000 7ff4c6d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff4c806b920 0x7ff4c80cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48238/0 (socket says 172.21.15.29:48238) 2024-04-25T00:35:04.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.589+0000 7ff4c6d9d640 1 -- 172.21.15.29:0/3918645998 learned_addr learned my addr 172.21.15.29:0/3918645998 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:04.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.589+0000 7ff4c6d9d640 1 -- 172.21.15.29:0/3918645998 >> v1:172.21.15.157:6789/0 conn(0x7ff4c8156420 legacy=0x7ff4c815fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:04.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.589+0000 7ff4c6d9d640 1 -- 172.21.15.29:0/3918645998 >> v1:172.21.15.29:6789/0 conn(0x7ff4c815ce60 legacy=0x7ff4c806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.589+0000 7ff4c6d9d640 1 -- 172.21.15.29:0/3918645998 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c8160440 con 0x7ff4c806b920 2024-04-25T00:35:04.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4c6d9d640 1 --2- 172.21.15.29:0/3918645998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4c806b920 0x7ff4c80cb290 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009e50 tx=0x7ff4c001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc06fdb12df0eb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:04.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4c559a640 1 -- 172.21.15.29:0/3918645998 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c00a57d0 con 0x7ff4c806b920 2024-04-25T00:35:04.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4c559a640 1 -- 172.21.15.29:0/3918645998 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c00a5970 con 0x7ff4c806b920 2024-04-25T00:35:04.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4c559a640 1 -- 172.21.15.29:0/3918645998 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c00a5d40 con 0x7ff4c806b920 2024-04-25T00:35:04.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3918645998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4c806b920 msgr2=0x7ff4c80cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4ccdf7640 1 --2- 172.21.15.29:0/3918645998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4c806b920 0x7ff4c80cb290 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009e50 tx=0x7ff4c001c9f0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3918645998 shutdown_connections 2024-04-25T00:35:04.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4ccdf7640 1 --2- 172.21.15.29:0/3918645998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4c806b920 0x7ff4c80cb290 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3918645998 >> 172.21.15.29:0/3918645998 conn(0x7ff4c80d0c40 msgr2=0x7ff4c80d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:04.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3918645998 shutdown_connections 2024-04-25T00:35:04.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3918645998 wait complete. 2024-04-25T00:35:04.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4ccdf7640 1 Processor -- start 2024-04-25T00:35:04.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4ccdf7640 1 -- start start 2024-04-25T00:35:04.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.593+0000 7ff4ccdf7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c806b920 0x7ff4c814ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4ccdf7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c8156420 0x7ff4c814cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4ccdf7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4c815ce60 0x7ff4c8154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4ccdf7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c80702e0 con 0x7ff4c815ce60 2024-04-25T00:35:04.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4ccdf7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c8070160 con 0x7ff4c8156420 2024-04-25T00:35:04.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4ccdf7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c8169e30 con 0x7ff4c806b920 2024-04-25T00:35:04.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4c6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4c815ce60 0x7ff4c8154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4c6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4c815ce60 0x7ff4c8154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48254/0 (socket says 172.21.15.29:48254) 2024-04-25T00:35:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4c6d9d640 1 -- 172.21.15.29:0/3479463751 learned_addr learned my addr 172.21.15.29:0/3479463751 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4c5d9b640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c8156420 0x7ff4c814cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4c659c640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c806b920 0x7ff4c814ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4c6d9d640 1 -- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c806b920 msgr2=0x7ff4c814ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4c6d9d640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c806b920 0x7ff4c814ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4c6d9d640 1 -- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c8156420 msgr2=0x7ff4c814cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4c6d9d640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c8156420 0x7ff4c814cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4c6d9d640 1 -- 172.21.15.29:0/3479463751 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c814d240 con 0x7ff4c815ce60 2024-04-25T00:35:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4c5d9b640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c8156420 0x7ff4c814cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4c6d9d640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4c815ce60 0x7ff4c8154940 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00af040 tx=0x7ff4c0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4b6ffd640 1 -- 172.21.15.29:0/3479463751 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0009250 con 0x7ff4c815ce60 2024-04-25T00:35:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4b6ffd640 1 -- 172.21.15.29:0/3479463751 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c00b8680 con 0x7ff4c815ce60 2024-04-25T00:35:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4b6ffd640 1 -- 172.21.15.29:0/3479463751 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c00b8a70 con 0x7ff4c815ce60 2024-04-25T00:35:04.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3479463751 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c8156b80 con 0x7ff4c815ce60 2024-04-25T00:35:04.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3479463751 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c8157100 con 0x7ff4c815ce60 2024-04-25T00:35:04.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3479463751 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c8069270 con 0x7ff4c815ce60 2024-04-25T00:35:04.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4b6ffd640 1 -- 172.21.15.29:0/3479463751 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7ff4c00c0020 con 0x7ff4c815ce60 2024-04-25T00:35:04.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4b6ffd640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff4a8067a70 0x7ff4a8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4b6ffd640 1 -- 172.21.15.29:0/3479463751 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7ff4c0120590 con 0x7ff4c815ce60 2024-04-25T00:35:04.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.597+0000 7ff4c659c640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff4a8067a70 0x7ff4a8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.601+0000 7ff4c659c640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff4a8067a70 0x7ff4a8069f30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80016c0 tx=0x7ff4b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:04.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.601+0000 7ff4b6ffd640 1 -- 172.21.15.29:0/3479463751 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff4c00c7020 con 0x7ff4c815ce60 2024-04-25T00:35:04.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.765+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3479463751 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4c80ce320 con 0x7ff4c815ce60 2024-04-25T00:35:04.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.769+0000 7ff4b6ffd640 1 -- 172.21.15.29:0/3479463751 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7ff4c00165e0 con 0x7ff4c815ce60 2024-04-25T00:35:04.769 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:35:04.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.769+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff4a8067a70 msgr2=0x7ff4a8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.769+0000 7ff4ccdf7640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff4a8067a70 0x7ff4a8069f30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80016c0 tx=0x7ff4b8009040 comp rx=0 tx=0).stop 2024-04-25T00:35:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.769+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3479463751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4c815ce60 msgr2=0x7ff4c8154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.769+0000 7ff4ccdf7640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4c815ce60 0x7ff4c8154940 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00af040 tx=0x7ff4c0019790 comp rx=0 tx=0).stop 2024-04-25T00:35:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.769+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3479463751 shutdown_connections 2024-04-25T00:35:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.769+0000 7ff4ccdf7640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4c815ce60 0x7ff4c8154940 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.769+0000 7ff4ccdf7640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff4a8067a70 0x7ff4a8069f30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.769+0000 7ff4ccdf7640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4c8156420 0x7ff4c814cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.769+0000 7ff4ccdf7640 1 --2- 172.21.15.29:0/3479463751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4c806b920 0x7ff4c814ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.769+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3479463751 >> 172.21.15.29:0/3479463751 conn(0x7ff4c80d0c40 msgr2=0x7ff4c80b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.769+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3479463751 shutdown_connections 2024-04-25T00:35:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.769+0000 7ff4ccdf7640 1 -- 172.21.15.29:0/3479463751 wait complete. 2024-04-25T00:35:04.789 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:35:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f0875fdc640 1 Processor -- start 2024-04-25T00:35:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f0875fdc640 1 -- start start 2024-04-25T00:35:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f0875fdc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08700b2840 0x7f08700b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f0875fdc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08700b5050 con 0x7f08700d2250 2024-04-25T00:35:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f0875fdc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08700b5330 con 0x7f08700d2d40 2024-04-25T00:35:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f0875fdc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f08700b0de0 con 0x7f08700b2840 2024-04-25T00:35:04.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f08700d2250 0x7f08700d2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43508/0 (socket says 172.21.15.29:43508) 2024-04-25T00:35:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086f7fe640 1 -- 172.21.15.29:0/1496721358 learned_addr learned my addr 172.21.15.29:0/1496721358 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086ffff640 1 --2- 172.21.15.29:0/1496721358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08700b2840 0x7f08700b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1266780537 0 0) 0x7f08700b5050 con 0x7f08700d2250 2024-04-25T00:35:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f085c003530 con 0x7f08700d2250 2024-04-25T00:35:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1211924570 0 0) 0x7f085c003530 con 0x7f08700d2250 2024-04-25T00:35:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08700b2840 msgr2=0x7f08700b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 --2- 172.21.15.29:0/1496721358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08700b2840 0x7f08700b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 >> v1:172.21.15.157:6789/0 conn(0x7f08700d2d40 legacy=0x7f08700b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08700b1030 con 0x7f08700d2250 2024-04-25T00:35:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f0860001680 con 0x7f08700d2250 2024-04-25T00:35:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 >> v1:172.21.15.29:6789/0 conn(0x7f08700d2250 legacy=0x7f08700d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 --2- 172.21.15.29:0/1496721358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08700b2840 0x7f085c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 --2- 172.21.15.29:0/1496721358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f085c003f70 0x7f085c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 --2- 172.21.15.29:0/1496721358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c004d50 0x7f085c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f08700b0de0 con 0x7f085c003f70 2024-04-25T00:35:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f085c00bb60 con 0x7f085c004d50 2024-04-25T00:35:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f085c00bd60 con 0x7f08700b2840 2024-04-25T00:35:04.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086effd640 1 --2- 172.21.15.29:0/1496721358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f085c003f70 0x7f085c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086f7fe640 1 --2- 172.21.15.29:0/1496721358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08700b2840 0x7f085c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086ffff640 1 --2- 172.21.15.29:0/1496721358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c004d50 0x7f085c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086effd640 1 -- 172.21.15.29:0/1496721358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08700b2840 msgr2=0x7f085c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:04.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086effd640 1 --2- 172.21.15.29:0/1496721358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08700b2840 0x7f085c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086effd640 1 -- 172.21.15.29:0/1496721358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c004d50 msgr2=0x7f085c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:04.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086effd640 1 --2- 172.21.15.29:0/1496721358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c004d50 0x7f085c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086effd640 1 -- 172.21.15.29:0/1496721358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c003530 con 0x7f085c003f70 2024-04-25T00:35:04.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.917+0000 7f086f7fe640 1 --2- 172.21.15.29:0/1496721358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08700b2840 0x7f085c0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:04.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.921+0000 7f086effd640 1 --2- 172.21.15.29:0/1496721358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f085c003f70 0x7f085c004810 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0858001780 tx=0x7f085801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a101ff629046b15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:04.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.921+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08580013a0 con 0x7f085c003f70 2024-04-25T00:35:04.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.921+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08580052f0 con 0x7f085c003f70 2024-04-25T00:35:04.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.921+0000 7f086e7fc640 1 -- 172.21.15.29:0/1496721358 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858003840 con 0x7f085c003f70 2024-04-25T00:35:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.921+0000 7f0875fdc640 1 -- 172.21.15.29:0/1496721358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f085c003f70 msgr2=0x7f085c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.921+0000 7f0875fdc640 1 --2- 172.21.15.29:0/1496721358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f085c003f70 0x7f085c004810 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0858001780 tx=0x7f085801c420 comp rx=0 tx=0).stop 2024-04-25T00:35:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.921+0000 7f086ffff640 1 -- 172.21.15.29:0/1496721358 reap_dead start 2024-04-25T00:35:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.921+0000 7f0875fdc640 1 -- 172.21.15.29:0/1496721358 shutdown_connections 2024-04-25T00:35:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.921+0000 7f0875fdc640 1 -- 172.21.15.29:0/1496721358 >> 172.21.15.29:0/1496721358 conn(0x7f087009d580 msgr2=0x7f08700d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.921+0000 7f0875fdc640 1 -- 172.21.15.29:0/1496721358 shutdown_connections 2024-04-25T00:35:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.921+0000 7f0875fdc640 1 -- 172.21.15.29:0/1496721358 wait complete. 2024-04-25T00:35:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.921+0000 7f0875fdc640 1 Processor -- start 2024-04-25T00:35:04.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.921+0000 7f0875fdc640 1 -- start start 2024-04-25T00:35:04.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0875fdc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f085c003f70 0x7f087015c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0875fdc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f087015e8d0 0x7f087015ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0875fdc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08700b7e50 0x7f08700b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0875fdc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f08700b4df0 con 0x7f085c003f70 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0875fdc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08700b4c70 con 0x7f08700b7e50 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0875fdc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f08701694c0 con 0x7f087015e8d0 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f086f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f085c003f70 0x7f087015c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f086f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f085c003f70 0x7f087015c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48274/0 (socket says 172.21.15.29:48274) 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f086f7fe640 1 -- 172.21.15.29:0/2332042178 learned_addr learned my addr 172.21.15.29:0/2332042178 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f086effd640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f087015e8d0 0x7f087015ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f086ffff640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08700b7e50 0x7f08700b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f086f7fe640 1 -- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f087015e8d0 msgr2=0x7f087015ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f086f7fe640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f087015e8d0 0x7f087015ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f086f7fe640 1 -- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08700b7e50 msgr2=0x7f08700b82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:04.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f086f7fe640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08700b7e50 0x7f08700b82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:04.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f086f7fe640 1 -- 172.21.15.29:0/2332042178 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08700cdae0 con 0x7f085c003f70 2024-04-25T00:35:04.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f086f7fe640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f085c003f70 0x7f087015c0a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f086000d840 tx=0x7f086000dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:04.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0867fff640 1 -- 172.21.15.29:0/2332042178 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860019070 con 0x7f085c003f70 2024-04-25T00:35:04.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0867fff640 1 -- 172.21.15.29:0/2332042178 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08600074e0 con 0x7f085c003f70 2024-04-25T00:35:04.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0875fdc640 1 -- 172.21.15.29:0/2332042178 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08700b8b30 con 0x7f085c003f70 2024-04-25T00:35:04.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0867fff640 1 -- 172.21.15.29:0/2332042178 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08600049f0 con 0x7f085c003f70 2024-04-25T00:35:04.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0875fdc640 1 -- 172.21.15.29:0/2332042178 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08700b90f0 con 0x7f085c003f70 2024-04-25T00:35:04.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0875fdc640 1 -- 172.21.15.29:0/2332042178 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0870069270 con 0x7f085c003f70 2024-04-25T00:35:04.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0867fff640 1 -- 172.21.15.29:0/2332042178 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f086001c070 con 0x7f085c003f70 2024-04-25T00:35:04.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0867fff640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0850067a70 0x7f0850069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:04.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f0867fff640 1 -- 172.21.15.29:0/2332042178 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f08600857a0 con 0x7f085c003f70 2024-04-25T00:35:04.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.925+0000 7f086effd640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0850067a70 0x7f0850069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:04.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.929+0000 7f086effd640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0850067a70 0x7f0850069f30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f08580ac000 tx=0x7f08580011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:04.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:04.929+0000 7f0867fff640 1 -- 172.21.15.29:0/2332042178 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f08700cdae0 con 0x7f085c003f70 2024-04-25T00:35:05.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.097+0000 7f0875fdc640 1 -- 172.21.15.29:0/2332042178 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0870072d10 con 0x7f085c003f70 2024-04-25T00:35:05.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.097+0000 7f0867fff640 1 -- 172.21.15.29:0/2332042178 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) ==== 84+0+65 (secure 0 0 0) 0x7f08600562a0 con 0x7f085c003f70 2024-04-25T00:35:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:05.099 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:35:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-04-25T00:35:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": ".mgr" 2024-04-25T00:35:05.099 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:05.100 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:35:05.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.101+0000 7f0875fdc640 1 -- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0850067a70 msgr2=0x7f0850069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:05.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.101+0000 7f0875fdc640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0850067a70 0x7f0850069f30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f08580ac000 tx=0x7f08580011a0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.101+0000 7f0875fdc640 1 -- 172.21.15.29:0/2332042178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f085c003f70 msgr2=0x7f087015c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:05.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.101+0000 7f0875fdc640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f085c003f70 0x7f087015c0a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f086000d840 tx=0x7f086000dd00 comp rx=0 tx=0).stop 2024-04-25T00:35:05.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.101+0000 7f0875fdc640 1 -- 172.21.15.29:0/2332042178 shutdown_connections 2024-04-25T00:35:05.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.101+0000 7f0875fdc640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0850067a70 0x7f0850069f30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.101+0000 7f0875fdc640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08700b7e50 0x7f08700b82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.101+0000 7f0875fdc640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f087015e8d0 0x7f087015ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.101+0000 7f0875fdc640 1 --2- 172.21.15.29:0/2332042178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f085c003f70 0x7f087015c0a0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.101+0000 7f0875fdc640 1 -- 172.21.15.29:0/2332042178 >> 172.21.15.29:0/2332042178 conn(0x7f087009d580 msgr2=0x7f08700d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:05.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.101+0000 7f0875fdc640 1 -- 172.21.15.29:0/2332042178 shutdown_connections 2024-04-25T00:35:05.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.101+0000 7f0875fdc640 1 -- 172.21.15.29:0/2332042178 wait complete. 2024-04-25T00:35:05.120 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:35:05.121 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:35:05.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.237+0000 7f104c595640 1 Processor -- start 2024-04-25T00:35:05.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.237+0000 7f104c595640 1 -- start start 2024-04-25T00:35:05.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104c595640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1044161b30 0x7f10440b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:05.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104c595640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10440d46e0 0x7f10440d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:05.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104c595640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10440d1970 con 0x7f1044161750 2024-04-25T00:35:05.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104c595640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f10440d1460 con 0x7f1044161b30 2024-04-25T00:35:05.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104c595640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f10440cf720 con 0x7f10440d46e0 2024-04-25T00:35:05.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f1049b30640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1044161b30 0x7f10440b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:05.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f1049b30640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1044161b30 0x7f10440b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48290/0 (socket says 172.21.15.29:48290) 2024-04-25T00:35:05.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f1049b30640 1 -- 172.21.15.29:0/2130338558 learned_addr learned my addr 172.21.15.29:0/2130338558 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:05.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104ab32640 1 --2- 172.21.15.29:0/2130338558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10440d46e0 0x7f10440d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:05.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f1049b30640 1 -- 172.21.15.29:0/2130338558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10440d46e0 msgr2=0x7f10440d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:05.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f1049b30640 1 --2- 172.21.15.29:0/2130338558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10440d46e0 0x7f10440d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f1049b30640 1 -- 172.21.15.29:0/2130338558 >> v1:172.21.15.157:6789/0 conn(0x7f1044161750 legacy=0x7f10440d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:05.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f1049b30640 1 -- 172.21.15.29:0/2130338558 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10440d1690 con 0x7f1044161b30 2024-04-25T00:35:05.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f1049b30640 1 --2- 172.21.15.29:0/2130338558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1044161b30 0x7f10440b4e80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1034009870 tx=0x7f103401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f197b95a79bed53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:05.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104932f640 1 -- 172.21.15.29:0/2130338558 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10340b0070 con 0x7f1044161b30 2024-04-25T00:35:05.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104932f640 1 -- 172.21.15.29:0/2130338558 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10340a4e50 con 0x7f1044161b30 2024-04-25T00:35:05.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104932f640 1 -- 172.21.15.29:0/2130338558 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10340ab820 con 0x7f1044161b30 2024-04-25T00:35:05.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104c595640 1 -- 172.21.15.29:0/2130338558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1044161b30 msgr2=0x7f10440b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:05.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104c595640 1 --2- 172.21.15.29:0/2130338558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1044161b30 0x7f10440b4e80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1034009870 tx=0x7f103401c400 comp rx=0 tx=0).stop 2024-04-25T00:35:05.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104c595640 1 -- 172.21.15.29:0/2130338558 shutdown_connections 2024-04-25T00:35:05.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104c595640 1 --2- 172.21.15.29:0/2130338558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10440d46e0 0x7f10440d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104c595640 1 --2- 172.21.15.29:0/2130338558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1044161b30 0x7f10440b4e80 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104c595640 1 -- 172.21.15.29:0/2130338558 >> 172.21.15.29:0/2130338558 conn(0x7f104409d2c0 msgr2=0x7f10440d3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:05.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104c595640 1 -- 172.21.15.29:0/2130338558 shutdown_connections 2024-04-25T00:35:05.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.241+0000 7f104c595640 1 -- 172.21.15.29:0/2130338558 wait complete. 2024-04-25T00:35:05.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104c595640 1 Processor -- start 2024-04-25T00:35:05.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104c595640 1 -- start start 2024-04-25T00:35:05.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104c595640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10440d46e0 0x7f10440cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:05.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104c595640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1044161750 0x7f104414efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:05.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104c595640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1044161b30 0x7f10440cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:05.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104c595640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f10440b7ac0 con 0x7f10440d46e0 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104c595640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f10440b7940 con 0x7f1044161750 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104c595640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f10440b7c40 con 0x7f1044161b30 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104a331640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10440d46e0 0x7f10440cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104a331640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10440d46e0 0x7f10440cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48298/0 (socket says 172.21.15.29:48298) 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104a331640 1 -- 172.21.15.29:0/2508288070 learned_addr learned my addr 172.21.15.29:0/2508288070 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104a331640 1 -- 172.21.15.29:0/2508288070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1044161b30 msgr2=0x7f10440cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104a331640 1 --2- 172.21.15.29:0/2508288070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1044161b30 0x7f10440cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104a331640 1 -- 172.21.15.29:0/2508288070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1044161750 msgr2=0x7f104414efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104a331640 1 --2- 172.21.15.29:0/2508288070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1044161750 0x7f104414efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104a331640 1 -- 172.21.15.29:0/2508288070 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10440c8480 con 0x7f10440d46e0 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104a331640 1 --2- 172.21.15.29:0/2508288070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10440d46e0 0x7f10440cde40 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f104000c450 tx=0x7f104000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f103affd640 1 -- 172.21.15.29:0/2508288070 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040012830 con 0x7f10440d46e0 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f103affd640 1 -- 172.21.15.29:0/2508288070 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1040017070 con 0x7f10440d46e0 2024-04-25T00:35:05.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f103affd640 1 -- 172.21.15.29:0/2508288070 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040012c30 con 0x7f10440d46e0 2024-04-25T00:35:05.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104c595640 1 -- 172.21.15.29:0/2508288070 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10440c8760 con 0x7f10440d46e0 2024-04-25T00:35:05.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104c595640 1 -- 172.21.15.29:0/2508288070 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104415d890 con 0x7f10440d46e0 2024-04-25T00:35:05.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.245+0000 7f104c595640 1 -- 172.21.15.29:0/2508288070 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10440692e0 con 0x7f10440d46e0 2024-04-25T00:35:05.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.249+0000 7f103affd640 1 -- 172.21.15.29:0/2508288070 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f104001c460 con 0x7f10440d46e0 2024-04-25T00:35:05.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.249+0000 7f103affd640 1 --2- 172.21.15.29:0/2508288070 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1028067a70 0x7f1028069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:05.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.249+0000 7f103affd640 1 -- 172.21.15.29:0/2508288070 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f10400859c0 con 0x7f10440d46e0 2024-04-25T00:35:05.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.249+0000 7f1049b30640 1 --2- 172.21.15.29:0/2508288070 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1028067a70 0x7f1028069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:05.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.249+0000 7f1049b30640 1 --2- 172.21.15.29:0/2508288070 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1028067a70 0x7f1028069f30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f10340ad000 tx=0x7f1034008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:05.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.253+0000 7f103affd640 1 -- 172.21.15.29:0/2508288070 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1040087e20 con 0x7f10440d46e0 2024-04-25T00:35:05.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.417+0000 7f104c595640 1 -- 172.21.15.29:0/2508288070 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f10440d3db0 con 0x7f10440d46e0 2024-04-25T00:35:05.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.729+0000 7f103affd640 1 -- 172.21.15.29:0/2508288070 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) ==== 138+0+0 (secure 0 0 0) 0x7f1040056660 con 0x7f10440d46e0 2024-04-25T00:35:05.731 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-04-25T00:35:05.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.733+0000 7f104c595640 1 -- 172.21.15.29:0/2508288070 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1028067a70 msgr2=0x7f1028069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:05.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.733+0000 7f104c595640 1 --2- 172.21.15.29:0/2508288070 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1028067a70 0x7f1028069f30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f10340ad000 tx=0x7f1034008040 comp rx=0 tx=0).stop 2024-04-25T00:35:05.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.733+0000 7f104c595640 1 -- 172.21.15.29:0/2508288070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10440d46e0 msgr2=0x7f10440cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:05.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.733+0000 7f104c595640 1 --2- 172.21.15.29:0/2508288070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10440d46e0 0x7f10440cde40 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f104000c450 tx=0x7f104000c910 comp rx=0 tx=0).stop 2024-04-25T00:35:05.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.733+0000 7f104c595640 1 -- 172.21.15.29:0/2508288070 shutdown_connections 2024-04-25T00:35:05.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.733+0000 7f104c595640 1 --2- 172.21.15.29:0/2508288070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1044161b30 0x7f10440cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.733+0000 7f104c595640 1 --2- 172.21.15.29:0/2508288070 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1028067a70 0x7f1028069f30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.733+0000 7f104c595640 1 --2- 172.21.15.29:0/2508288070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1044161750 0x7f104414efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.733+0000 7f104c595640 1 --2- 172.21.15.29:0/2508288070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10440d46e0 0x7f10440cde40 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.733+0000 7f104c595640 1 -- 172.21.15.29:0/2508288070 >> 172.21.15.29:0/2508288070 conn(0x7f104409d2c0 msgr2=0x7f10440d3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:05.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.733+0000 7f104c595640 1 -- 172.21.15.29:0/2508288070 shutdown_connections 2024-04-25T00:35:05.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.733+0000 7f104c595640 1 -- 172.21.15.29:0/2508288070 wait complete. 2024-04-25T00:35:05.763 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:35:05.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.893+0000 7f29553f8640 1 Processor -- start 2024-04-25T00:35:05.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.893+0000 7f29553f8640 1 -- start start 2024-04-25T00:35:05.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.893+0000 7f29553f8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29500b3140 0x7f295006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:05.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.893+0000 7f29553f8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b3c30 0x7f29500ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:05.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.893+0000 7f29553f8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29500b41e0 con 0x7f29500b4b10 2024-04-25T00:35:05.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.893+0000 7f29553f8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f295006d960 con 0x7f29500b3140 2024-04-25T00:35:05.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.893+0000 7f29553f8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f295006dc40 con 0x7f29500b3c30 2024-04-25T00:35:05.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.893+0000 7f294effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29500b3140 0x7f295006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:05.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.893+0000 7f294f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b3c30 0x7f29500ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:05.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.893+0000 7f294f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b3c30 0x7f29500ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:48590/0 (socket says 172.21.15.29:48590) 2024-04-25T00:35:05.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.893+0000 7f294effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29500b3140 0x7f295006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:45592/0 (socket says 172.21.15.29:45592) 2024-04-25T00:35:05.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.893+0000 7f294f7fe640 1 -- 172.21.15.29:0/1083896178 learned_addr learned my addr 172.21.15.29:0/1083896178 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:05.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f294effd640 1 -- 172.21.15.29:0/1083896178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b3c30 msgr2=0x7f29500ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:05.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f294effd640 1 --2- 172.21.15.29:0/1083896178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b3c30 0x7f29500ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f294effd640 1 -- 172.21.15.29:0/1083896178 >> v1:172.21.15.157:6789/0 conn(0x7f29500b4b10 legacy=0x7f29500b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:05.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f294effd640 1 -- 172.21.15.29:0/1083896178 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295006df20 con 0x7f29500b3140 2024-04-25T00:35:05.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f294f7fe640 1 --2- 172.21.15.29:0/1083896178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b3c30 0x7f29500ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:05.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f294effd640 1 --2- 172.21.15.29:0/1083896178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29500b3140 0x7f295006e440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f29400097f0 tx=0x7f294001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=844917d4a454f429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:05.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f294e7fc640 1 -- 172.21.15.29:0/1083896178 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400b0070 con 0x7f29500b3140 2024-04-25T00:35:05.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f294e7fc640 1 -- 172.21.15.29:0/1083896178 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29400ab600 con 0x7f29500b3140 2024-04-25T00:35:05.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f294e7fc640 1 -- 172.21.15.29:0/1083896178 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400ab9d0 con 0x7f29500b3140 2024-04-25T00:35:05.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f29553f8640 1 -- 172.21.15.29:0/1083896178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29500b3140 msgr2=0x7f295006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:05.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f29553f8640 1 --2- 172.21.15.29:0/1083896178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29500b3140 0x7f295006e440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f29400097f0 tx=0x7f294001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f29553f8640 1 -- 172.21.15.29:0/1083896178 shutdown_connections 2024-04-25T00:35:05.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f29553f8640 1 --2- 172.21.15.29:0/1083896178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b3c30 0x7f29500ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f29553f8640 1 --2- 172.21.15.29:0/1083896178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29500b3140 0x7f295006e440 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f29553f8640 1 -- 172.21.15.29:0/1083896178 >> 172.21.15.29:0/1083896178 conn(0x7f29500b1870 msgr2=0x7f29500b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:05.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f29553f8640 1 -- 172.21.15.29:0/1083896178 shutdown_connections 2024-04-25T00:35:05.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.897+0000 7f29553f8640 1 -- 172.21.15.29:0/1083896178 wait complete. 2024-04-25T00:35:05.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29553f8640 1 Processor -- start 2024-04-25T00:35:05.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29553f8640 1 -- start start 2024-04-25T00:35:05.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29553f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29500b3c30 0x7f295013ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29553f8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b4b10 0x7f2950159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29553f8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295015a4d0 0x7f2950070f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29553f8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2950168d80 con 0x7f295015a4d0 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29553f8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2950168c00 con 0x7f29500b3c30 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29553f8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2950168f00 con 0x7f29500b4b10 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f294f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295015a4d0 0x7f2950070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f294f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295015a4d0 0x7f2950070f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48302/0 (socket says 172.21.15.29:48302) 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f294f7fe640 1 -- 172.21.15.29:0/4253705325 learned_addr learned my addr 172.21.15.29:0/4253705325 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f2947fff640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b4b10 0x7f2950159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f294effd640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29500b3c30 0x7f295013ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f294f7fe640 1 -- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b4b10 msgr2=0x7f2950159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f294f7fe640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b4b10 0x7f2950159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f294f7fe640 1 -- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29500b3c30 msgr2=0x7f295013ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:05.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f294f7fe640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29500b3c30 0x7f295013ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:05.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f294f7fe640 1 -- 172.21.15.29:0/4253705325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2950156550 con 0x7f295015a4d0 2024-04-25T00:35:05.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f294f7fe640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295015a4d0 0x7f2950070f60 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f294800efc0 tx=0x7f294800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:05.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29477fe640 1 -- 172.21.15.29:0/4253705325 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294800cbe0 con 0x7f295015a4d0 2024-04-25T00:35:05.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29477fe640 1 -- 172.21.15.29:0/4253705325 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294800cd80 con 0x7f295015a4d0 2024-04-25T00:35:05.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29477fe640 1 -- 172.21.15.29:0/4253705325 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948003bc0 con 0x7f295015a4d0 2024-04-25T00:35:05.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29553f8640 1 -- 172.21.15.29:0/4253705325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2950071cb0 con 0x7f295015a4d0 2024-04-25T00:35:05.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29553f8640 1 -- 172.21.15.29:0/4253705325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2950072270 con 0x7f295015a4d0 2024-04-25T00:35:05.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29553f8640 1 -- 172.21.15.29:0/4253705325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2950069270 con 0x7f295015a4d0 2024-04-25T00:35:05.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.901+0000 7f29477fe640 1 -- 172.21.15.29:0/4253705325 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f294801b030 con 0x7f295015a4d0 2024-04-25T00:35:05.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.905+0000 7f29477fe640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2934067a70 0x7f2934069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:05.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.905+0000 7f29477fe640 1 -- 172.21.15.29:0/4253705325 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5227+0+0 (secure 0 0 0) 0x7f29480861c0 con 0x7f295015a4d0 2024-04-25T00:35:05.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.905+0000 7f294effd640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2934067a70 0x7f2934069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:05.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.905+0000 7f294effd640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2934067a70 0x7f2934069f30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f29400ad000 tx=0x7f2940008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:05.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:05.909+0000 7f29477fe640 1 -- 172.21.15.29:0/4253705325 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2948022020 con 0x7f295015a4d0 2024-04-25T00:35:06.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.061+0000 7f29553f8640 1 -- 172.21.15.29:0/4253705325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f29500cdb40 con 0x7f295015a4d0 2024-04-25T00:35:06.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.733+0000 7f29477fe640 1 -- 172.21.15.29:0/4253705325 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) ==== 170+0+0 (secure 0 0 0) 0x7f2948010040 con 0x7f295015a4d0 2024-04-25T00:35:06.735 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-04-25T00:35:06.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.737+0000 7f29553f8640 1 -- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2934067a70 msgr2=0x7f2934069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.737+0000 7f29553f8640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2934067a70 0x7f2934069f30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f29400ad000 tx=0x7f2940008040 comp rx=0 tx=0).stop 2024-04-25T00:35:06.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.737+0000 7f29553f8640 1 -- 172.21.15.29:0/4253705325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295015a4d0 msgr2=0x7f2950070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.737+0000 7f29553f8640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295015a4d0 0x7f2950070f60 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f294800efc0 tx=0x7f294800c490 comp rx=0 tx=0).stop 2024-04-25T00:35:06.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.737+0000 7f29553f8640 1 -- 172.21.15.29:0/4253705325 shutdown_connections 2024-04-25T00:35:06.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.737+0000 7f29553f8640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295015a4d0 0x7f2950070f60 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.737+0000 7f29553f8640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29500b4b10 0x7f2950159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.737+0000 7f29553f8640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2934067a70 0x7f2934069f30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.737+0000 7f29553f8640 1 --2- 172.21.15.29:0/4253705325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29500b3c30 0x7f295013ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.737+0000 7f29553f8640 1 -- 172.21.15.29:0/4253705325 >> 172.21.15.29:0/4253705325 conn(0x7f29500b1870 msgr2=0x7f2950143b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:06.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.737+0000 7f29553f8640 1 -- 172.21.15.29:0/4253705325 shutdown_connections 2024-04-25T00:35:06.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.737+0000 7f29553f8640 1 -- 172.21.15.29:0/4253705325 wait complete. 2024-04-25T00:35:06.776 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:35:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f59735640 1 Processor -- start 2024-04-25T00:35:06.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f59735640 1 -- start start 2024-04-25T00:35:06.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f59735640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2f540b3140 0x7f2f5406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f59735640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f540b41e0 con 0x7f2f540b4b10 2024-04-25T00:35:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f59735640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f5406d960 con 0x7f2f540b3c30 2024-04-25T00:35:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f59735640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2f5406dc40 con 0x7f2f540b3140 2024-04-25T00:35:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f527fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2f540b4b10 0x7f2f540b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:43524/0 (socket says 172.21.15.29:43524) 2024-04-25T00:35:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f527fc640 1 -- 172.21.15.29:0/2169188365 learned_addr learned my addr 172.21.15.29:0/2169188365 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f52ffd640 1 --2- 172.21.15.29:0/2169188365 >> v2:172.21.15.157:3300/0 conn(0x7f2f540b3140 0x7f2f5406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3526349941 0 0) 0x7f2f540b41e0 con 0x7f2f540b4b10 2024-04-25T00:35:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f3c003530 con 0x7f2f540b4b10 2024-04-25T00:35:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1534013812 0 0) 0x7f2f3c003530 con 0x7f2f540b4b10 2024-04-25T00:35:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 >> v2:172.21.15.157:3300/0 conn(0x7f2f540b3140 msgr2=0x7f2f5406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 --2- 172.21.15.29:0/2169188365 >> v2:172.21.15.157:3300/0 conn(0x7f2f540b3140 0x7f2f5406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 >> v1:172.21.15.157:6789/0 conn(0x7f2f540b3c30 legacy=0x7f2f540ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5406df20 con 0x7f2f540b4b10 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2f48001680 con 0x7f2f540b4b10 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 >> v1:172.21.15.29:6789/0 conn(0x7f2f540b4b10 legacy=0x7f2f540b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 --2- 172.21.15.29:0/2169188365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f3c0039c0 0x7f2f3c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 --2- 172.21.15.29:0/2169188365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f3c004810 0x7f2f3c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 --2- 172.21.15.29:0/2169188365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c0051d0 0x7f2f3c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5406dc40 con 0x7f2f3c0051d0 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c00c070 con 0x7f2f3c0039c0 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3c00c270 con 0x7f2f3c004810 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f537fe640 1 --2- 172.21.15.29:0/2169188365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c0051d0 0x7f2f3c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f52ffd640 1 --2- 172.21.15.29:0/2169188365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f3c0039c0 0x7f2f3c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f527fc640 1 --2- 172.21.15.29:0/2169188365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f3c004810 0x7f2f3c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f537fe640 1 -- 172.21.15.29:0/2169188365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f3c004810 msgr2=0x7f2f3c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:06.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f537fe640 1 --2- 172.21.15.29:0/2169188365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f3c004810 0x7f2f3c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f537fe640 1 -- 172.21.15.29:0/2169188365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f3c0039c0 msgr2=0x7f2f3c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f537fe640 1 --2- 172.21.15.29:0/2169188365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f3c0039c0 0x7f2f3c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.901+0000 7f2f537fe640 1 -- 172.21.15.29:0/2169188365 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c003530 con 0x7f2f3c0051d0 2024-04-25T00:35:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f537fe640 1 -- 172.21.15.29:0/2169188365 reap_dead start 2024-04-25T00:35:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f537fe640 1 --2- 172.21.15.29:0/2169188365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c0051d0 0x7f2f3c00b9d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c001e70 tx=0x7f2f4c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f6b4316186ca9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c001a90 con 0x7f2f3c0051d0 2024-04-25T00:35:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c003bc0 con 0x7f2f3c0051d0 2024-04-25T00:35:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f51ffb640 1 -- 172.21.15.29:0/2169188365 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c005020 con 0x7f2f3c0051d0 2024-04-25T00:35:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f59735640 1 -- 172.21.15.29:0/2169188365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c0051d0 msgr2=0x7f2f3c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f59735640 1 --2- 172.21.15.29:0/2169188365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c0051d0 0x7f2f3c00b9d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c001e70 tx=0x7f2f4c01c780 comp rx=0 tx=0).stop 2024-04-25T00:35:06.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f59735640 1 -- 172.21.15.29:0/2169188365 shutdown_connections 2024-04-25T00:35:06.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f59735640 1 --2- 172.21.15.29:0/2169188365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c0051d0 0x7f2f3c00b9d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f59735640 1 -- 172.21.15.29:0/2169188365 >> 172.21.15.29:0/2169188365 conn(0x7f2f540b1870 msgr2=0x7f2f540b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:06.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f59735640 1 -- 172.21.15.29:0/2169188365 shutdown_connections 2024-04-25T00:35:06.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f59735640 1 -- 172.21.15.29:0/2169188365 wait complete. 2024-04-25T00:35:06.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f59735640 1 Processor -- start 2024-04-25T00:35:06.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.905+0000 7f2f59735640 1 -- start start 2024-04-25T00:35:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f59735640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c0051d0 0x7f2f541f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f59735640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f5415d100 0x7f2f5415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f59735640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f5415e2e0 0x7f2f54140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f59735640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54145a90 con 0x7f2f3c0051d0 2024-04-25T00:35:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f59735640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54145910 con 0x7f2f5415d100 2024-04-25T00:35:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f59735640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2f5406e440 con 0x7f2f5415e2e0 2024-04-25T00:35:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f52ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c0051d0 0x7f2f541f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f52ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c0051d0 0x7f2f541f14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48314/0 (socket says 172.21.15.29:48314) 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f52ffd640 1 -- 172.21.15.29:0/1989282164 learned_addr learned my addr 172.21.15.29:0/1989282164 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f527fc640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f5415d100 0x7f2f5415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f537fe640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f5415e2e0 0x7f2f54140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f52ffd640 1 -- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f5415e2e0 msgr2=0x7f2f54140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f52ffd640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f5415e2e0 0x7f2f54140d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f52ffd640 1 -- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f5415d100 msgr2=0x7f2f5415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f52ffd640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f5415d100 0x7f2f5415dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f52ffd640 1 -- 172.21.15.29:0/1989282164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f54150b90 con 0x7f2f3c0051d0 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f527fc640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f5415d100 0x7f2f5415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f52ffd640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c0051d0 0x7f2f541f14d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2f440115c0 tx=0x7f2f44011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f3b7fe640 1 -- 172.21.15.29:0/1989282164 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44015870 con 0x7f2f3c0051d0 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f3b7fe640 1 -- 172.21.15.29:0/1989282164 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f44015a10 con 0x7f2f3c0051d0 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f59735640 1 -- 172.21.15.29:0/1989282164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5415c380 con 0x7f2f3c0051d0 2024-04-25T00:35:06.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f3b7fe640 1 -- 172.21.15.29:0/1989282164 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44015dc0 con 0x7f2f3c0051d0 2024-04-25T00:35:06.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f59735640 1 -- 172.21.15.29:0/1989282164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f5415c620 con 0x7f2f3c0051d0 2024-04-25T00:35:06.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f59735640 1 -- 172.21.15.29:0/1989282164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f54069270 con 0x7f2f3c0051d0 2024-04-25T00:35:06.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f3b7fe640 1 -- 172.21.15.29:0/1989282164 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f2f44013400 con 0x7f2f3c0051d0 2024-04-25T00:35:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f3b7fe640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2f40067aa0 0x7f2f40069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f3b7fe640 1 -- 172.21.15.29:0/1989282164 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5609+0+0 (secure 0 0 0) 0x7f2f44086760 con 0x7f2f3c0051d0 2024-04-25T00:35:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.909+0000 7f2f527fc640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2f40067aa0 0x7f2f40069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:06.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.913+0000 7f2f527fc640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2f40067aa0 0x7f2f40069f60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2f48001ec0 tx=0x7f2f48000f80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:06.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:06.917+0000 7f2f3b7fe640 1 -- 172.21.15.29:0/1989282164 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2f44019030 con 0x7f2f3c0051d0 2024-04-25T00:35:07.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.109+0000 7f2f59735640 1 -- 172.21.15.29:0/1989282164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2f5413fcf0 con 0x7f2f3c0051d0 2024-04-25T00:35:07.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.817+0000 7f2f3b7fe640 1 -- 172.21.15.29:0/1989282164 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) ==== 172+0+0 (secure 0 0 0) 0x7f2f44057080 con 0x7f2f3c0051d0 2024-04-25T00:35:07.822 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 8 and data pool 9 2024-04-25T00:35:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.825+0000 7f2f59735640 1 -- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2f40067aa0 msgr2=0x7f2f40069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.825+0000 7f2f59735640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2f40067aa0 0x7f2f40069f60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2f48001ec0 tx=0x7f2f48000f80 comp rx=0 tx=0).stop 2024-04-25T00:35:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.825+0000 7f2f59735640 1 -- 172.21.15.29:0/1989282164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c0051d0 msgr2=0x7f2f541f14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.825+0000 7f2f59735640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c0051d0 0x7f2f541f14d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2f440115c0 tx=0x7f2f44011a80 comp rx=0 tx=0).stop 2024-04-25T00:35:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.825+0000 7f2f59735640 1 -- 172.21.15.29:0/1989282164 shutdown_connections 2024-04-25T00:35:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.825+0000 7f2f59735640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f5415e2e0 0x7f2f54140d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.825+0000 7f2f59735640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2f40067aa0 0x7f2f40069f60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.825+0000 7f2f59735640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f5415d100 0x7f2f5415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.825+0000 7f2f59735640 1 --2- 172.21.15.29:0/1989282164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c0051d0 0x7f2f541f14d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.825+0000 7f2f59735640 1 -- 172.21.15.29:0/1989282164 >> 172.21.15.29:0/1989282164 conn(0x7f2f540b1870 msgr2=0x7f2f5406cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.825+0000 7f2f59735640 1 -- 172.21.15.29:0/1989282164 shutdown_connections 2024-04-25T00:35:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.825+0000 7f2f59735640 1 -- 172.21.15.29:0/1989282164 wait complete. 2024-04-25T00:35:07.854 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:07.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.969+0000 7f6b23ea6640 1 Processor -- start 2024-04-25T00:35:07.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.969+0000 7f6b23ea6640 1 -- start start 2024-04-25T00:35:07.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b23ea6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b1c0b3c30 0x7f6b1c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:07.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b23ea6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c0b4b10 0x7f6b1c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:07.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b23ea6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b1c0b41e0 con 0x7f6b1c0b3140 2024-04-25T00:35:07.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b23ea6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f6b1c06d960 con 0x7f6b1c0b3c30 2024-04-25T00:35:07.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b23ea6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1c06dc40 con 0x7f6b1c0b4b10 2024-04-25T00:35:07.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b22443640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c0b4b10 0x7f6b1c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:07.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b21441640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b1c0b3c30 0x7f6b1c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:07.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b22443640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c0b4b10 0x7f6b1c0ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:37250/0 (socket says 172.21.15.29:37250) 2024-04-25T00:35:07.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b22443640 1 -- 172.21.15.29:0/3569445601 learned_addr learned my addr 172.21.15.29:0/3569445601 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:07.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b22443640 1 -- 172.21.15.29:0/3569445601 >> v2:172.21.15.157:3300/0 conn(0x7f6b1c0b3c30 msgr2=0x7f6b1c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:07.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b22443640 1 --2- 172.21.15.29:0/3569445601 >> v2:172.21.15.157:3300/0 conn(0x7f6b1c0b3c30 0x7f6b1c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:07.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b22443640 1 -- 172.21.15.29:0/3569445601 >> v1:172.21.15.157:6789/0 conn(0x7f6b1c0b3140 legacy=0x7f6b1c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:07.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b22443640 1 -- 172.21.15.29:0/3569445601 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c06df20 con 0x7f6b1c0b4b10 2024-04-25T00:35:07.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.973+0000 7f6b21441640 1 --2- 172.21.15.29:0/3569445601 >> v2:172.21.15.157:3300/0 conn(0x7f6b1c0b3c30 0x7f6b1c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:07.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b22443640 1 --2- 172.21.15.29:0/3569445601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c0b4b10 0x7f6b1c0ce240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009e80 tx=0x7f6b1401ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18285d50236b8534 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:07.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b20c40640 1 -- 172.21.15.29:0/3569445601 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b140a56e0 con 0x7f6b1c0b4b10 2024-04-25T00:35:07.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b20c40640 1 -- 172.21.15.29:0/3569445601 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b140a5880 con 0x7f6b1c0b4b10 2024-04-25T00:35:07.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b20c40640 1 -- 172.21.15.29:0/3569445601 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b140a5c50 con 0x7f6b1c0b4b10 2024-04-25T00:35:07.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 -- 172.21.15.29:0/3569445601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c0b4b10 msgr2=0x7f6b1c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:07.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 --2- 172.21.15.29:0/3569445601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c0b4b10 0x7f6b1c0ce240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009e80 tx=0x7f6b1401ca10 comp rx=0 tx=0).stop 2024-04-25T00:35:07.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 -- 172.21.15.29:0/3569445601 shutdown_connections 2024-04-25T00:35:07.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 --2- 172.21.15.29:0/3569445601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c0b4b10 0x7f6b1c0ce240 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:07.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 --2- 172.21.15.29:0/3569445601 >> v2:172.21.15.157:3300/0 conn(0x7f6b1c0b3c30 0x7f6b1c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:07.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 -- 172.21.15.29:0/3569445601 >> 172.21.15.29:0/3569445601 conn(0x7f6b1c0b1870 msgr2=0x7f6b1c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:07.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 -- 172.21.15.29:0/3569445601 shutdown_connections 2024-04-25T00:35:07.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 -- 172.21.15.29:0/3569445601 wait complete. 2024-04-25T00:35:07.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 Processor -- start 2024-04-25T00:35:07.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 -- start start 2024-04-25T00:35:07.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c0b3140 0x7f6b1c15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:07.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b1c0b3c30 0x7f6b1c1fa610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:07.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1c0b4b10 0x7f6b1c1fab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:07.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c168d80 con 0x7f6b1c0b3c30 2024-04-25T00:35:07.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c168c00 con 0x7f6b1c0b4b10 2024-04-25T00:35:07.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b23ea6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1c168f00 con 0x7f6b1c0b3140 2024-04-25T00:35:07.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b21441640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b1c0b3c30 0x7f6b1c1fa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:07.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b21441640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b1c0b3c30 0x7f6b1c1fa610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59110/0 (socket says 172.21.15.29:59110) 2024-04-25T00:35:07.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b21441640 1 -- 172.21.15.29:0/2816561960 learned_addr learned my addr 172.21.15.29:0/2816561960 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:07.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.977+0000 7f6b22443640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1c0b4b10 0x7f6b1c1fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:07.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b21c42640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c0b3140 0x7f6b1c15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:07.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b21441640 1 -- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c0b3140 msgr2=0x7f6b1c15aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:07.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b21441640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c0b3140 0x7f6b1c15aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:07.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b21441640 1 -- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1c0b4b10 msgr2=0x7f6b1c1fab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:07.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b21441640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1c0b4b10 0x7f6b1c1fab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:07.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b21441640 1 -- 172.21.15.29:0/2816561960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c1649a0 con 0x7f6b1c0b3c30 2024-04-25T00:35:07.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b22443640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1c0b4b10 0x7f6b1c1fab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:07.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b21441640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b1c0b3c30 0x7f6b1c1fa610 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e980 tx=0x7f6b0c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:07.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b127fc640 1 -- 172.21.15.29:0/2816561960 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c017070 con 0x7f6b1c0b3c30 2024-04-25T00:35:07.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b127fc640 1 -- 172.21.15.29:0/2816561960 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b0c00ca00 con 0x7f6b1c0b3c30 2024-04-25T00:35:07.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b23ea6640 1 -- 172.21.15.29:0/2816561960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c164c80 con 0x7f6b1c0b3c30 2024-04-25T00:35:07.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b127fc640 1 -- 172.21.15.29:0/2816561960 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00cdd0 con 0x7f6b1c0b3c30 2024-04-25T00:35:07.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b23ea6640 1 -- 172.21.15.29:0/2816561960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c165240 con 0x7f6b1c0b3c30 2024-04-25T00:35:07.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b23ea6640 1 -- 172.21.15.29:0/2816561960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c069270 con 0x7f6b1c0b3c30 2024-04-25T00:35:07.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b127fc640 1 -- 172.21.15.29:0/2816561960 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f6b0c01a070 con 0x7f6b1c0b3c30 2024-04-25T00:35:07.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b127fc640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6b00067a70 0x7f6b00069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:07.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b127fc640 1 -- 172.21.15.29:0/2816561960 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f6b0c086200 con 0x7f6b1c0b3c30 2024-04-25T00:35:07.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.981+0000 7f6b21c42640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6b00067a70 0x7f6b00069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:07.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.985+0000 7f6b21c42640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6b00067a70 0x7f6b00069f30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6b180015b0 tx=0x7f6b18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:07.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:07.985+0000 7f6b127fc640 1 -- 172.21.15.29:0/2816561960 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6b0c056ad0 con 0x7f6b1c0b3c30 2024-04-25T00:35:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.149+0000 7f6b23ea6640 1 -- 172.21.15.29:0/2816561960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b1c13d050 con 0x7f6b1c0b3c30 2024-04-25T00:35:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.149+0000 7f6b127fc640 1 -- 172.21.15.29:0/2816561960 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16630 (secure 0 0 0) 0x7f6b0c0563d0 con 0x7f6b1c0b3c30 2024-04-25T00:35:08.152 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:08.152 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":57,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:07.113709+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:05.458923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:06.104267+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.153+0000 7f6b23ea6640 1 -- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6b00067a70 msgr2=0x7f6b00069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.153+0000 7f6b23ea6640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6b00067a70 0x7f6b00069f30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6b180015b0 tx=0x7f6b18009040 comp rx=0 tx=0).stop 2024-04-25T00:35:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.153+0000 7f6b23ea6640 1 -- 172.21.15.29:0/2816561960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b1c0b3c30 msgr2=0x7f6b1c1fa610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.153+0000 7f6b23ea6640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b1c0b3c30 0x7f6b1c1fa610 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e980 tx=0x7f6b0c00ee40 comp rx=0 tx=0).stop 2024-04-25T00:35:08.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.153+0000 7f6b23ea6640 1 -- 172.21.15.29:0/2816561960 shutdown_connections 2024-04-25T00:35:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.153+0000 7f6b23ea6640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6b00067a70 0x7f6b00069f30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.153+0000 7f6b23ea6640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b1c0b4b10 0x7f6b1c1fab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.153+0000 7f6b23ea6640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b1c0b3c30 0x7f6b1c1fa610 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.153+0000 7f6b23ea6640 1 --2- 172.21.15.29:0/2816561960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b1c0b3140 0x7f6b1c15aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.153+0000 7f6b23ea6640 1 -- 172.21.15.29:0/2816561960 >> 172.21.15.29:0/2816561960 conn(0x7f6b1c0b1870 msgr2=0x7f6b1c1436e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.153+0000 7f6b23ea6640 1 -- 172.21.15.29:0/2816561960 shutdown_connections 2024-04-25T00:35:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.153+0000 7f6b23ea6640 1 -- 172.21.15.29:0/2816561960 wait complete. 2024-04-25T00:35:08.171 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:08.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8fc09e640 1 Processor -- start 2024-04-25T00:35:08.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8fc09e640 1 -- start start 2024-04-25T00:35:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8fc09e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd8f415e8d0 0x7fd8f40cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8fc09e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f415f3c0 0x7fd8f415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8fc09e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8f40b3720 con 0x7fd8f40b4780 2024-04-25T00:35:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8fc09e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd8f40b42f0 con 0x7fd8f415e8d0 2024-04-25T00:35:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8fc09e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f40b39c0 con 0x7fd8f415f3c0 2024-04-25T00:35:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f9639640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd8f40b4780 0x7fd8f415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46784/0 (socket says 172.21.15.29:46784) 2024-04-25T00:35:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f9639640 1 -- 172.21.15.29:0/128688095 learned_addr learned my addr 172.21.15.29:0/128688095 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8fa63b640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f415f3c0 0x7fd8f415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f9e3a640 1 --2- 172.21.15.29:0/128688095 >> v2:172.21.15.157:3300/0 conn(0x7fd8f415e8d0 0x7fd8f40cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2496514420 0 0) 0x7fd8f40b3720 con 0x7fd8f40b4780 2024-04-25T00:35:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8e0003530 con 0x7fd8f40b4780 2024-04-25T00:35:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2270054479 0 0) 0x7fd8e0003530 con 0x7fd8f40b4780 2024-04-25T00:35:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f415f3c0 msgr2=0x7fd8f415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f415f3c0 0x7fd8f415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 >> v2:172.21.15.157:3300/0 conn(0x7fd8f415e8d0 msgr2=0x7fd8f40cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 --2- 172.21.15.29:0/128688095 >> v2:172.21.15.157:3300/0 conn(0x7fd8f415e8d0 0x7fd8f40cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f40b3ca0 con 0x7fd8f40b4780 2024-04-25T00:35:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd8e4001680 con 0x7fd8f40b4780 2024-04-25T00:35:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 >> v1:172.21.15.29:6789/0 conn(0x7fd8f40b4780 legacy=0x7fd8f415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f415f3c0 0x7fd8e00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8e0003f40 0x7fd8e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8e0004d50 0x7fd8e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f40b42f0 con 0x7fd8e0003f40 2024-04-25T00:35:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f40b39c0 con 0x7fd8e0004d50 2024-04-25T00:35:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e000b9d0 con 0x7fd8f415f3c0 2024-04-25T00:35:08.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f9639640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8e0003f40 0x7fd8e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f9e3a640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f415f3c0 0x7fd8e00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8fa63b640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8e0004d50 0x7fd8e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f9639640 1 -- 172.21.15.29:0/128688095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f415f3c0 msgr2=0x7fd8e00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:08.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f9639640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f415f3c0 0x7fd8e00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f9639640 1 -- 172.21.15.29:0/128688095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8e0004d50 msgr2=0x7fd8e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:08.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f9639640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8e0004d50 0x7fd8e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f9639640 1 -- 172.21.15.29:0/128688095 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e0003530 con 0x7fd8e0003f40 2024-04-25T00:35:08.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8f9e3a640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f415f3c0 0x7fd8e00039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:08.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.289+0000 7fd8fa63b640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8e0004d50 0x7fd8e00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:08.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8f9639640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8e0003f40 0x7fd8e0004810 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400c240 tx=0x7fd8e401f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72760bfc7e436a24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:08.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4007d60 con 0x7fd8e0003f40 2024-04-25T00:35:08.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e4005b00 con 0x7fd8e0003f40 2024-04-25T00:35:08.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8f8e38640 1 -- 172.21.15.29:0/128688095 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4007070 con 0x7fd8e0003f40 2024-04-25T00:35:08.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 -- 172.21.15.29:0/128688095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8e0003f40 msgr2=0x7fd8e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:08.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 --2- 172.21.15.29:0/128688095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8e0003f40 0x7fd8e0004810 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400c240 tx=0x7fd8e401f3d0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fa63b640 1 -- 172.21.15.29:0/128688095 reap_dead start 2024-04-25T00:35:08.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 -- 172.21.15.29:0/128688095 shutdown_connections 2024-04-25T00:35:08.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 -- 172.21.15.29:0/128688095 >> 172.21.15.29:0/128688095 conn(0x7fd8f40b1870 msgr2=0x7fd8f40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:08.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 -- 172.21.15.29:0/128688095 shutdown_connections 2024-04-25T00:35:08.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 -- 172.21.15.29:0/128688095 wait complete. 2024-04-25T00:35:08.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 Processor -- start 2024-04-25T00:35:08.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 -- start start 2024-04-25T00:35:08.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f415a7f0 0x7fd8f413c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f413e1f0 0x7fd8f4138630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f4138b70 0x7fd8f406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4144a20 con 0x7fd8f4138b70 2024-04-25T00:35:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f406cf10 con 0x7fd8f415a7f0 2024-04-25T00:35:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fc09e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f41409f0 con 0x7fd8f413e1f0 2024-04-25T00:35:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.293+0000 7fd8fa63b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f4138b70 0x7fd8f406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8fa63b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f4138b70 0x7fd8f406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59122/0 (socket says 172.21.15.29:59122) 2024-04-25T00:35:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8fa63b640 1 -- 172.21.15.29:0/4094533061 learned_addr learned my addr 172.21.15.29:0/4094533061 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8f9639640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f413e1f0 0x7fd8f4138630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8f9e3a640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f415a7f0 0x7fd8f413c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8fa63b640 1 -- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f413e1f0 msgr2=0x7fd8f4138630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8fa63b640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f413e1f0 0x7fd8f4138630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8fa63b640 1 -- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f415a7f0 msgr2=0x7fd8f413c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:08.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8fa63b640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f415a7f0 0x7fd8f413c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8fa63b640 1 -- 172.21.15.29:0/4094533061 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f413a650 con 0x7fd8f4138b70 2024-04-25T00:35:08.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8fa63b640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f4138b70 0x7fd8f406d5d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4067ea0 tx=0x7fd8ec00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:08.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8ea7fc640 1 -- 172.21.15.29:0/4094533061 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec01c070 con 0x7fd8f4138b70 2024-04-25T00:35:08.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8ea7fc640 1 -- 172.21.15.29:0/4094533061 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8ec00fe90 con 0x7fd8f4138b70 2024-04-25T00:35:08.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8ea7fc640 1 -- 172.21.15.29:0/4094533061 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec017620 con 0x7fd8f4138b70 2024-04-25T00:35:08.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8fc09e640 1 -- 172.21.15.29:0/4094533061 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f413a930 con 0x7fd8f4138b70 2024-04-25T00:35:08.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8fc09e640 1 -- 172.21.15.29:0/4094533061 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f413e670 con 0x7fd8f4138b70 2024-04-25T00:35:08.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8fc09e640 1 -- 172.21.15.29:0/4094533061 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f4069270 con 0x7fd8f4138b70 2024-04-25T00:35:08.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8ea7fc640 1 -- 172.21.15.29:0/4094533061 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fd8ec01f070 con 0x7fd8f4138b70 2024-04-25T00:35:08.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8ea7fc640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd8d8067aa0 0x7fd8d8069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8ea7fc640 1 -- 172.21.15.29:0/4094533061 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fd8ec085fa0 con 0x7fd8f4138b70 2024-04-25T00:35:08.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.297+0000 7fd8f9e3a640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd8d8067aa0 0x7fd8d8069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.301+0000 7fd8f9e3a640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd8d8067aa0 0x7fd8d8069f60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4139180 tx=0x7fd8f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:08.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.301+0000 7fd8ea7fc640 1 -- 172.21.15.29:0/4094533061 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd8ec012070 con 0x7fd8f4138b70 2024-04-25T00:35:08.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.469+0000 7fd8fc09e640 1 -- 172.21.15.29:0/4094533061 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8f406e7b0 con 0x7fd8f4138b70 2024-04-25T00:35:08.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.469+0000 7fd8ea7fc640 1 -- 172.21.15.29:0/4094533061 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16630 (secure 0 0 0) 0x7fd8ec0567e0 con 0x7fd8f4138b70 2024-04-25T00:35:08.472 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:08.472 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":57,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:07.113709+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:05.458923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:06.104267+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:08.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.473+0000 7fd8fc09e640 1 -- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd8d8067aa0 msgr2=0x7fd8d8069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:08.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.473+0000 7fd8fc09e640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd8d8067aa0 0x7fd8d8069f60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4139180 tx=0x7fd8f0008040 comp rx=0 tx=0).stop 2024-04-25T00:35:08.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.473+0000 7fd8fc09e640 1 -- 172.21.15.29:0/4094533061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f4138b70 msgr2=0x7fd8f406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:08.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.473+0000 7fd8fc09e640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f4138b70 0x7fd8f406d5d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4067ea0 tx=0x7fd8ec00f4a0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.473+0000 7fd8fc09e640 1 -- 172.21.15.29:0/4094533061 shutdown_connections 2024-04-25T00:35:08.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.473+0000 7fd8fc09e640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f4138b70 0x7fd8f406d5d0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.473+0000 7fd8fc09e640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f413e1f0 0x7fd8f4138630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.473+0000 7fd8fc09e640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd8d8067aa0 0x7fd8d8069f60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.473+0000 7fd8fc09e640 1 --2- 172.21.15.29:0/4094533061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f415a7f0 0x7fd8f413c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.473+0000 7fd8fc09e640 1 -- 172.21.15.29:0/4094533061 >> 172.21.15.29:0/4094533061 conn(0x7fd8f40b1870 msgr2=0x7fd8f40cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:08.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.473+0000 7fd8fc09e640 1 -- 172.21.15.29:0/4094533061 shutdown_connections 2024-04-25T00:35:08.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.473+0000 7fd8fc09e640 1 -- 172.21.15.29:0/4094533061 wait complete. 2024-04-25T00:35:08.492 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:35:08.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed4f245640 1 Processor -- start 2024-04-25T00:35:08.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed4f245640 1 -- start start 2024-04-25T00:35:08.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed4f245640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fed4815e200 0x7fed480d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed4f245640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed480cfc10 con 0x7fed480d2c60 2024-04-25T00:35:08.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed4f245640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed480cfef0 con 0x7fed480cf690 2024-04-25T00:35:08.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed4f245640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fed480b38d0 con 0x7fed4815e200 2024-04-25T00:35:08.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed4d7e2640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fed480d2c60 0x7fed480b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46788/0 (socket says 172.21.15.29:46788) 2024-04-25T00:35:08.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed4d7e2640 1 -- 172.21.15.29:0/3101634977 learned_addr learned my addr 172.21.15.29:0/3101634977 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:08.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed4cfe1640 1 --2- 172.21.15.29:0/3101634977 >> v2:172.21.15.157:3300/0 conn(0x7fed4815e200 0x7fed480d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3665006088 0 0) 0x7fed480cfc10 con 0x7fed480d2c60 2024-04-25T00:35:08.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed30003530 con 0x7fed480d2c60 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 199351256 0 0) 0x7fed30003530 con 0x7fed480d2c60 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 >> v2:172.21.15.157:3300/0 conn(0x7fed4815e200 msgr2=0x7fed480d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 --2- 172.21.15.29:0/3101634977 >> v2:172.21.15.157:3300/0 conn(0x7fed4815e200 0x7fed480d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 >> v1:172.21.15.157:6789/0 conn(0x7fed480cf690 legacy=0x7fed480d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed480b3b20 con 0x7fed480d2c60 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fed40001a00 con 0x7fed480d2c60 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 >> v1:172.21.15.29:6789/0 conn(0x7fed480d2c60 legacy=0x7fed480b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 --2- 172.21.15.29:0/3101634977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed300039c0 0x7fed30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 --2- 172.21.15.29:0/3101634977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed30004810 0x7fed30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 --2- 172.21.15.29:0/3101634977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed300051d0 0x7fed3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fed480b38d0 con 0x7fed30004810 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed3000c070 con 0x7fed300039c0 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fed3000c270 con 0x7fed300051d0 2024-04-25T00:35:08.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed47fff640 1 --2- 172.21.15.29:0/3101634977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed30004810 0x7fed30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed4d7e2640 1 --2- 172.21.15.29:0/3101634977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed300051d0 0x7fed3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed4cfe1640 1 --2- 172.21.15.29:0/3101634977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed300039c0 0x7fed30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed47fff640 1 -- 172.21.15.29:0/3101634977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed300051d0 msgr2=0x7fed3000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:08.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed47fff640 1 --2- 172.21.15.29:0/3101634977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed300051d0 0x7fed3000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed47fff640 1 -- 172.21.15.29:0/3101634977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed300039c0 msgr2=0x7fed30003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:08.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed47fff640 1 --2- 172.21.15.29:0/3101634977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed300039c0 0x7fed30003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed47fff640 1 -- 172.21.15.29:0/3101634977 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed30003530 con 0x7fed30004810 2024-04-25T00:35:08.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.621+0000 7fed4d7e2640 1 -- 172.21.15.29:0/3101634977 reap_dead start 2024-04-25T00:35:08.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.625+0000 7fed47fff640 1 --2- 172.21.15.29:0/3101634977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed30004810 0x7fed30004c90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fed3c001780 tx=0x7fed3c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83d5ab4872a5608 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:08.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.625+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0013a0 con 0x7fed30004810 2024-04-25T00:35:08.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.625+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed3c0052f0 con 0x7fed30004810 2024-04-25T00:35:08.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.625+0000 7fed477fe640 1 -- 172.21.15.29:0/3101634977 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c003840 con 0x7fed30004810 2024-04-25T00:35:08.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.625+0000 7fed4f245640 1 -- 172.21.15.29:0/3101634977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed30004810 msgr2=0x7fed30004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:08.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.625+0000 7fed4f245640 1 --2- 172.21.15.29:0/3101634977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed30004810 0x7fed30004c90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fed3c001780 tx=0x7fed3c01c400 comp rx=0 tx=0).stop 2024-04-25T00:35:08.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.625+0000 7fed4f245640 1 -- 172.21.15.29:0/3101634977 shutdown_connections 2024-04-25T00:35:08.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.625+0000 7fed4f245640 1 --2- 172.21.15.29:0/3101634977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed30004810 0x7fed30004c90 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.625+0000 7fed4f245640 1 -- 172.21.15.29:0/3101634977 >> 172.21.15.29:0/3101634977 conn(0x7fed4809d280 msgr2=0x7fed480d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:08.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.625+0000 7fed4f245640 1 -- 172.21.15.29:0/3101634977 shutdown_connections 2024-04-25T00:35:08.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.625+0000 7fed4f245640 1 -- 172.21.15.29:0/3101634977 wait complete. 2024-04-25T00:35:08.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.625+0000 7fed4f245640 1 Processor -- start 2024-04-25T00:35:08.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.625+0000 7fed4f245640 1 -- start start 2024-04-25T00:35:08.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed4f245640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed30004810 0x7fed480c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed4f245640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed48153460 0x7fed480c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed4f245640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed480c8520 0x7fed4815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed4f245640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fed4815e760 con 0x7fed48153460 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed4f245640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed4815e5e0 con 0x7fed480c8520 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed4f245640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fed480d2550 con 0x7fed30004810 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed47fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed48153460 0x7fed480c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed47fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed48153460 0x7fed480c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59140/0 (socket says 172.21.15.29:59140) 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed47fff640 1 -- 172.21.15.29:0/129290569 learned_addr learned my addr 172.21.15.29:0/129290569 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed4cfe1640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed30004810 0x7fed480c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed4d7e2640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed480c8520 0x7fed4815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed47fff640 1 -- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed30004810 msgr2=0x7fed480c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed47fff640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed30004810 0x7fed480c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed47fff640 1 -- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed480c8520 msgr2=0x7fed4815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:08.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed47fff640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed480c8520 0x7fed4815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:08.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed47fff640 1 -- 172.21.15.29:0/129290569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed480c9dc0 con 0x7fed48153460 2024-04-25T00:35:08.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed4d7e2640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed480c8520 0x7fed4815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:08.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed47fff640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed48153460 0x7fed480c7fe0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fed3c005040 tx=0x7fed30005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:08.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed457fa640 1 -- 172.21.15.29:0/129290569 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0013a0 con 0x7fed48153460 2024-04-25T00:35:08.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed457fa640 1 -- 172.21.15.29:0/129290569 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed3c0ba070 con 0x7fed48153460 2024-04-25T00:35:08.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed4f245640 1 -- 172.21.15.29:0/129290569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed480ca0a0 con 0x7fed48153460 2024-04-25T00:35:08.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed457fa640 1 -- 172.21.15.29:0/129290569 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c007a40 con 0x7fed48153460 2024-04-25T00:35:08.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed4f245640 1 -- 172.21.15.29:0/129290569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed480ca660 con 0x7fed48153460 2024-04-25T00:35:08.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed4f245640 1 -- 172.21.15.29:0/129290569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed480ca8a0 con 0x7fed48153460 2024-04-25T00:35:08.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed457fa640 1 -- 172.21.15.29:0/129290569 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fed3c007490 con 0x7fed48153460 2024-04-25T00:35:08.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed457fa640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fed34067aa0 0x7fed34069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:08.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed457fa640 1 -- 172.21.15.29:0/129290569 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fed3c1203c0 con 0x7fed48153460 2024-04-25T00:35:08.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.629+0000 7fed4cfe1640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fed34067aa0 0x7fed34069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:08.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.633+0000 7fed4cfe1640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fed34067aa0 0x7fed34069f60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fed48050eb0 tx=0x7fed38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:08.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.637+0000 7fed457fa640 1 -- 172.21.15.29:0/129290569 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fed480ca8a0 con 0x7fed48153460 2024-04-25T00:35:08.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:08.841+0000 7fed4f245640 1 -- 172.21.15.29:0/129290569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fed48069270 con 0x7fed48153460 2024-04-25T00:35:09.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.809+0000 7fed457fa640 1 -- 172.21.15.29:0/129290569 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) ==== 118+0+0 (secure 0 0 0) 0x7fed3c016110 con 0x7fed48153460 2024-04-25T00:35:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.813+0000 7fed4f245640 1 -- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fed34067aa0 msgr2=0x7fed34069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.813+0000 7fed4f245640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fed34067aa0 0x7fed34069f60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fed48050eb0 tx=0x7fed38008040 comp rx=0 tx=0).stop 2024-04-25T00:35:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.813+0000 7fed4f245640 1 -- 172.21.15.29:0/129290569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed48153460 msgr2=0x7fed480c7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.813+0000 7fed4f245640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed48153460 0x7fed480c7fe0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fed3c005040 tx=0x7fed30005840 comp rx=0 tx=0).stop 2024-04-25T00:35:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.813+0000 7fed4f245640 1 -- 172.21.15.29:0/129290569 shutdown_connections 2024-04-25T00:35:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.813+0000 7fed4f245640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fed34067aa0 0x7fed34069f60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.813+0000 7fed4f245640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed480c8520 0x7fed4815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.813+0000 7fed4f245640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed48153460 0x7fed480c7fe0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.813+0000 7fed4f245640 1 --2- 172.21.15.29:0/129290569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed30004810 0x7fed480c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.813+0000 7fed4f245640 1 -- 172.21.15.29:0/129290569 >> 172.21.15.29:0/129290569 conn(0x7fed4809d280 msgr2=0x7fed4815d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:09.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.813+0000 7fed4f245640 1 -- 172.21.15.29:0/129290569 shutdown_connections 2024-04-25T00:35:09.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.813+0000 7fed4f245640 1 -- 172.21.15.29:0/129290569 wait complete. 2024-04-25T00:35:09.836 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:35:09.837 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:35:09.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c99283640 1 Processor -- start 2024-04-25T00:35:09.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c99283640 1 -- start start 2024-04-25T00:35:09.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c99283640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3c9415e8d0 0x7f3c940cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:09.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c99283640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c9415f3c0 0x7f3c9415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:09.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c99283640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c940b3720 con 0x7f3c940b4780 2024-04-25T00:35:09.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c99283640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3c940b42f0 con 0x7f3c9415e8d0 2024-04-25T00:35:09.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c99283640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3c940b39c0 con 0x7f3c9415f3c0 2024-04-25T00:35:09.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c937fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c9415f3c0 0x7f3c9415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:09.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c92ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3c9415e8d0 0x7f3c940cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:09.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c937fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c9415f3c0 0x7f3c9415c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:37300/0 (socket says 172.21.15.29:37300) 2024-04-25T00:35:09.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c937fe640 1 -- 172.21.15.29:0/3724791092 learned_addr learned my addr 172.21.15.29:0/3724791092 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:09.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c937fe640 1 -- 172.21.15.29:0/3724791092 >> v2:172.21.15.157:3300/0 conn(0x7f3c9415e8d0 msgr2=0x7f3c940cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:09.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c937fe640 1 --2- 172.21.15.29:0/3724791092 >> v2:172.21.15.157:3300/0 conn(0x7f3c9415e8d0 0x7f3c940cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:09.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c937fe640 1 -- 172.21.15.29:0/3724791092 >> v1:172.21.15.157:6789/0 conn(0x7f3c940b4780 legacy=0x7f3c9415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:09.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.969+0000 7f3c937fe640 1 -- 172.21.15.29:0/3724791092 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c940b3ca0 con 0x7f3c9415f3c0 2024-04-25T00:35:09.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.973+0000 7f3c937fe640 1 --2- 172.21.15.29:0/3724791092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c9415f3c0 0x7f3c9415c7d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c009e50 tx=0x7f3c8c01ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd53613de4e0abe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.973+0000 7f3c91ffb640 1 -- 172.21.15.29:0/3724791092 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c0a58b0 con 0x7f3c9415f3c0 2024-04-25T00:35:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.973+0000 7f3c91ffb640 1 -- 172.21.15.29:0/3724791092 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8c0a5a50 con 0x7f3c9415f3c0 2024-04-25T00:35:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.973+0000 7f3c91ffb640 1 -- 172.21.15.29:0/3724791092 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c0a5e20 con 0x7f3c9415f3c0 2024-04-25T00:35:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.973+0000 7f3c99283640 1 -- 172.21.15.29:0/3724791092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c9415f3c0 msgr2=0x7f3c9415c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.973+0000 7f3c99283640 1 --2- 172.21.15.29:0/3724791092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c9415f3c0 0x7f3c9415c7d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c009e50 tx=0x7f3c8c01ca00 comp rx=0 tx=0).stop 2024-04-25T00:35:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.973+0000 7f3c99283640 1 -- 172.21.15.29:0/3724791092 shutdown_connections 2024-04-25T00:35:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.973+0000 7f3c99283640 1 --2- 172.21.15.29:0/3724791092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c9415f3c0 0x7f3c9415c7d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.973+0000 7f3c99283640 1 --2- 172.21.15.29:0/3724791092 >> v2:172.21.15.157:3300/0 conn(0x7f3c9415e8d0 0x7f3c940cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.973+0000 7f3c99283640 1 -- 172.21.15.29:0/3724791092 >> 172.21.15.29:0/3724791092 conn(0x7f3c940b1870 msgr2=0x7f3c940b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.973+0000 7f3c99283640 1 -- 172.21.15.29:0/3724791092 shutdown_connections 2024-04-25T00:35:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.973+0000 7f3c99283640 1 -- 172.21.15.29:0/3724791092 wait complete. 2024-04-25T00:35:09.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c99283640 1 Processor -- start 2024-04-25T00:35:09.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c99283640 1 -- start start 2024-04-25T00:35:09.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c99283640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c940b4780 0x7f3c9414e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:09.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c99283640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c9415e8d0 0x7f3c941431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:09.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c99283640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c9415f3c0 0x7f3c94130d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:09.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c99283640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9415d6a0 con 0x7f3c9415e8d0 2024-04-25T00:35:09.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c99283640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9415d520 con 0x7f3c940b4780 2024-04-25T00:35:09.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c99283640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3c94167c70 con 0x7f3c9415f3c0 2024-04-25T00:35:09.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c927fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c9415e8d0 0x7f3c941431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:09.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c927fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c9415e8d0 0x7f3c941431a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59142/0 (socket says 172.21.15.29:59142) 2024-04-25T00:35:09.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c927fc640 1 -- 172.21.15.29:0/1339592640 learned_addr learned my addr 172.21.15.29:0/1339592640 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:09.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c937fe640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c9415f3c0 0x7f3c94130d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:09.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c92ffd640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c940b4780 0x7f3c9414e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c927fc640 1 -- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c9415f3c0 msgr2=0x7f3c94130d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c927fc640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c9415f3c0 0x7f3c94130d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c927fc640 1 -- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c940b4780 msgr2=0x7f3c9414e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c927fc640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c940b4780 0x7f3c9414e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c927fc640 1 -- 172.21.15.29:0/1339592640 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9414ea80 con 0x7f3c9415e8d0 2024-04-25T00:35:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c927fc640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c9415e8d0 0x7f3c941431a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800c450 tx=0x7f3c8800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c7b7fe640 1 -- 172.21.15.29:0/1339592640 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88016020 con 0x7f3c9415e8d0 2024-04-25T00:35:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c7b7fe640 1 -- 172.21.15.29:0/1339592640 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c880052d0 con 0x7f3c9415e8d0 2024-04-25T00:35:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c99283640 1 -- 172.21.15.29:0/1339592640 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c941436e0 con 0x7f3c9415e8d0 2024-04-25T00:35:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c7b7fe640 1 -- 172.21.15.29:0/1339592640 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88007260 con 0x7f3c9415e8d0 2024-04-25T00:35:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c99283640 1 -- 172.21.15.29:0/1339592640 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c940cd250 con 0x7f3c9415e8d0 2024-04-25T00:35:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c99283640 1 -- 172.21.15.29:0/1339592640 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c94069270 con 0x7f3c9415e8d0 2024-04-25T00:35:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.977+0000 7f3c7b7fe640 1 -- 172.21.15.29:0/1339592640 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f3c88007400 con 0x7f3c9415e8d0 2024-04-25T00:35:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.981+0000 7f3c7b7fe640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3c80067a70 0x7f3c80069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.981+0000 7f3c7b7fe640 1 -- 172.21.15.29:0/1339592640 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3c880858e0 con 0x7f3c9415e8d0 2024-04-25T00:35:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.981+0000 7f3c92ffd640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3c80067a70 0x7f3c80069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.981+0000 7f3c92ffd640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3c80067a70 0x7f3c80069f30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009b60 tx=0x7f3c84009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:09.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:09.985+0000 7f3c7b7fe640 1 -- 172.21.15.29:0/1339592640 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3c88020020 con 0x7f3c9415e8d0 2024-04-25T00:35:10.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.181+0000 7f3c99283640 1 -- 172.21.15.29:0/1339592640 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3c940cd530 con 0x7f3c9415e8d0 2024-04-25T00:35:10.340 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:35:10.343 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:35:10.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.821+0000 7f3c7b7fe640 1 -- 172.21.15.29:0/1339592640 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f3c88005480 con 0x7f3c9415e8d0 2024-04-25T00:35:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.821+0000 7f3c99283640 1 -- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3c80067a70 msgr2=0x7f3c80069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.821+0000 7f3c99283640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3c80067a70 0x7f3c80069f30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009b60 tx=0x7f3c84009190 comp rx=0 tx=0).stop 2024-04-25T00:35:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.821+0000 7f3c99283640 1 -- 172.21.15.29:0/1339592640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c9415e8d0 msgr2=0x7f3c941431a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.821+0000 7f3c99283640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c9415e8d0 0x7f3c941431a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800c450 tx=0x7f3c8800c910 comp rx=0 tx=0).stop 2024-04-25T00:35:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.821+0000 7f3c99283640 1 -- 172.21.15.29:0/1339592640 shutdown_connections 2024-04-25T00:35:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.821+0000 7f3c99283640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c9415f3c0 0x7f3c94130d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.821+0000 7f3c99283640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c9415e8d0 0x7f3c941431a0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.821+0000 7f3c99283640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3c80067a70 0x7f3c80069f30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:10.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.821+0000 7f3c99283640 1 --2- 172.21.15.29:0/1339592640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c940b4780 0x7f3c9414e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:10.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.821+0000 7f3c99283640 1 -- 172.21.15.29:0/1339592640 >> 172.21.15.29:0/1339592640 conn(0x7f3c940b1870 msgr2=0x7f3c9415b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:10.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.821+0000 7f3c99283640 1 -- 172.21.15.29:0/1339592640 shutdown_connections 2024-04-25T00:35:10.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.821+0000 7f3c99283640 1 -- 172.21.15.29:0/1339592640 wait complete. 2024-04-25T00:35:10.844 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:35:10.844 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:35:10.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f56232aa640 1 Processor -- start 2024-04-25T00:35:10.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f56232aa640 1 -- start start 2024-04-25T00:35:10.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f56232aa640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f561c15e200 0x7f561c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:10.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f56232aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f561c0cf690 0x7f561c0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:10.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f56232aa640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f561c0d2ea0 con 0x7f561c0d2a90 2024-04-25T00:35:10.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f56232aa640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f561c0b38d0 con 0x7f561c15e200 2024-04-25T00:35:10.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f56232aa640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f561c0cfc10 con 0x7f561c0cf690 2024-04-25T00:35:10.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f5621046640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f561c15e200 0x7f561c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:10.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f5621046640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f561c15e200 0x7f561c0d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59148/0 (socket says 172.21.15.29:59148) 2024-04-25T00:35:10.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f5621046640 1 -- 172.21.15.29:0/1325299459 learned_addr learned my addr 172.21.15.29:0/1325299459 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:10.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f5620845640 1 --2- 172.21.15.29:0/1325299459 >> v2:172.21.15.157:3300/0 conn(0x7f561c0cf690 0x7f561c0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:10.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f5621046640 1 -- 172.21.15.29:0/1325299459 >> v2:172.21.15.157:3300/0 conn(0x7f561c0cf690 msgr2=0x7f561c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:10.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f5621046640 1 --2- 172.21.15.29:0/1325299459 >> v2:172.21.15.157:3300/0 conn(0x7f561c0cf690 0x7f561c0d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:10.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f5621046640 1 -- 172.21.15.29:0/1325299459 >> v1:172.21.15.157:6789/0 conn(0x7f561c0d2a90 legacy=0x7f561c0d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:10.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f5621046640 1 -- 172.21.15.29:0/1325299459 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c0cfef0 con 0x7f561c15e200 2024-04-25T00:35:10.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.973+0000 7f5620845640 1 --2- 172.21.15.29:0/1325299459 >> v2:172.21.15.157:3300/0 conn(0x7f561c0cf690 0x7f561c0d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:10.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.977+0000 7f5621046640 1 --2- 172.21.15.29:0/1325299459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f561c15e200 0x7f561c0d47f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f561c0d24a0 tx=0x7f561801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d292f1da90402125 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.977+0000 7f5613fff640 1 -- 172.21.15.29:0/1325299459 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56180a5880 con 0x7f561c15e200 2024-04-25T00:35:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.977+0000 7f5613fff640 1 -- 172.21.15.29:0/1325299459 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56180a5a20 con 0x7f561c15e200 2024-04-25T00:35:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.977+0000 7f5613fff640 1 -- 172.21.15.29:0/1325299459 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56180a5df0 con 0x7f561c15e200 2024-04-25T00:35:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.977+0000 7f56232aa640 1 -- 172.21.15.29:0/1325299459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f561c15e200 msgr2=0x7f561c0d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.977+0000 7f56232aa640 1 --2- 172.21.15.29:0/1325299459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f561c15e200 0x7f561c0d47f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f561c0d24a0 tx=0x7f561801c9c0 comp rx=0 tx=0).stop 2024-04-25T00:35:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.977+0000 7f56232aa640 1 -- 172.21.15.29:0/1325299459 shutdown_connections 2024-04-25T00:35:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.977+0000 7f56232aa640 1 --2- 172.21.15.29:0/1325299459 >> v2:172.21.15.157:3300/0 conn(0x7f561c0cf690 0x7f561c0d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.977+0000 7f56232aa640 1 --2- 172.21.15.29:0/1325299459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f561c15e200 0x7f561c0d47f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.977+0000 7f56232aa640 1 -- 172.21.15.29:0/1325299459 >> 172.21.15.29:0/1325299459 conn(0x7f561c09d280 msgr2=0x7f561c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.977+0000 7f56232aa640 1 -- 172.21.15.29:0/1325299459 shutdown_connections 2024-04-25T00:35:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.977+0000 7f56232aa640 1 -- 172.21.15.29:0/1325299459 wait complete. 2024-04-25T00:35:10.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f56232aa640 1 Processor -- start 2024-04-25T00:35:10.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f56232aa640 1 -- start start 2024-04-25T00:35:10.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f56232aa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561c0cf690 0x7f561c155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:10.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f56232aa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f561c0d2a90 0x7f561c0b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:10.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f56232aa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f561c15e200 0x7f561c0b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:10.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f56232aa640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f561c15e760 con 0x7f561c0d2a90 2024-04-25T00:35:10.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f56232aa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f561c15e5e0 con 0x7f561c0cf690 2024-04-25T00:35:10.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f56232aa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f561c16ae30 con 0x7f561c15e200 2024-04-25T00:35:10.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5620845640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f561c0d2a90 0x7f561c0b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:10.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5620845640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f561c0d2a90 0x7f561c0b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59158/0 (socket says 172.21.15.29:59158) 2024-04-25T00:35:10.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5620845640 1 -- 172.21.15.29:0/357652692 learned_addr learned my addr 172.21.15.29:0/357652692 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:10.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5621046640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561c0cf690 0x7f561c155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:10.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5621847640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f561c15e200 0x7f561c0b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:10.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5620845640 1 -- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f561c15e200 msgr2=0x7f561c0b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5620845640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f561c15e200 0x7f561c0b7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5620845640 1 -- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561c0cf690 msgr2=0x7f561c155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5620845640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561c0cf690 0x7f561c155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5620845640 1 -- 172.21.15.29:0/357652692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c06c2f0 con 0x7f561c0d2a90 2024-04-25T00:35:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5621847640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f561c15e200 0x7f561c0b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5621046640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561c0cf690 0x7f561c155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5620845640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f561c0d2a90 0x7f561c0b2460 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f560c00d800 tx=0x7f560c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5611ffb640 1 -- 172.21.15.29:0/357652692 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c018020 con 0x7f561c0d2a90 2024-04-25T00:35:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5611ffb640 1 -- 172.21.15.29:0/357652692 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f560c0096c0 con 0x7f561c0d2a90 2024-04-25T00:35:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f56232aa640 1 -- 172.21.15.29:0/357652692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561c0b8930 con 0x7f561c0d2a90 2024-04-25T00:35:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5611ffb640 1 -- 172.21.15.29:0/357652692 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c009ab0 con 0x7f561c0d2a90 2024-04-25T00:35:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f56232aa640 1 -- 172.21.15.29:0/357652692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f561c0b8ef0 con 0x7f561c0d2a90 2024-04-25T00:35:10.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f56232aa640 1 -- 172.21.15.29:0/357652692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f561c069270 con 0x7f561c0d2a90 2024-04-25T00:35:10.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5611ffb640 1 -- 172.21.15.29:0/357652692 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f560c01b020 con 0x7f561c0d2a90 2024-04-25T00:35:10.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5611ffb640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5600067a70 0x7f5600069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:10.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.981+0000 7f5611ffb640 1 -- 172.21.15.29:0/357652692 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f560c086970 con 0x7f561c0d2a90 2024-04-25T00:35:10.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.985+0000 7f5621046640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5600067a70 0x7f5600069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:10.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.985+0000 7f5621046640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5600067a70 0x7f5600069f30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f56180ae000 tx=0x7f56180a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:10.985+0000 7f5611ffb640 1 -- 172.21.15.29:0/357652692 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f560c022020 con 0x7f561c0d2a90 2024-04-25T00:35:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f56232aa640 1 -- 172.21.15.29:0/357652692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f561c0b1ce0 con 0x7f561c0d2a90 2024-04-25T00:35:11.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f5611ffb640 1 -- 172.21.15.29:0/357652692 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5175 (secure 0 0 0) 0x7f560c0573a0 con 0x7f561c0d2a90 2024-04-25T00:35:11.183 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:11.183 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/56623075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":56623075},{"type":"v1","addr":"172.21.15.157:6836","nonce":56623075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/748098725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":748098725},{"type":"v1","addr":"172.21.15.157:6837","nonce":748098725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2264413556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2264413556},{"type":"v1","addr":"172.21.15.29:6833","nonce":2264413556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:07.114520+0000","modified":"2024-04-25T00:35:10.185595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6372},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/802824676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":802824676},{"type":"v1","addr":"172.21.15.29:6835","nonce":802824676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372]},"id":4}]} 2024-04-25T00:35:11.183 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 34 2024-04-25T00:35:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f56232aa640 1 -- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5600067a70 msgr2=0x7f5600069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f56232aa640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5600067a70 0x7f5600069f30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f56180ae000 tx=0x7f56180a7070 comp rx=0 tx=0).stop 2024-04-25T00:35:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f56232aa640 1 -- 172.21.15.29:0/357652692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f561c0d2a90 msgr2=0x7f561c0b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f56232aa640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f561c0d2a90 0x7f561c0b2460 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f560c00d800 tx=0x7f560c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f56232aa640 1 -- 172.21.15.29:0/357652692 shutdown_connections 2024-04-25T00:35:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f56232aa640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f561c15e200 0x7f561c0b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f56232aa640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f561c0d2a90 0x7f561c0b2460 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f56232aa640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5600067a70 0x7f5600069f30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f56232aa640 1 --2- 172.21.15.29:0/357652692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561c0cf690 0x7f561c155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f56232aa640 1 -- 172.21.15.29:0/357652692 >> 172.21.15.29:0/357652692 conn(0x7f561c09d280 msgr2=0x7f561c15d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:11.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f56232aa640 1 -- 172.21.15.29:0/357652692 shutdown_connections 2024-04-25T00:35:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.181+0000 7f56232aa640 1 -- 172.21.15.29:0/357652692 wait complete. 2024-04-25T00:35:11.200 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.321+0000 7f38a94f7640 1 Processor -- start 2024-04-25T00:35:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.321+0000 7f38a94f7640 1 -- start start 2024-04-25T00:35:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.321+0000 7f38a94f7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a406dce0 0x7f38a4150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.321+0000 7f38a94f7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38a40b3130 con 0x7f38a415d980 2024-04-25T00:35:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.321+0000 7f38a94f7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38a40b3410 con 0x7f38a40b2b70 2024-04-25T00:35:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.321+0000 7f38a94f7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f38a40cf560 con 0x7f38a406dce0 2024-04-25T00:35:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.325+0000 7f38a2ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f38a415d980 0x7f38a406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46790/0 (socket says 172.21.15.29:46790) 2024-04-25T00:35:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.325+0000 7f38a2ffd640 1 -- 172.21.15.29:0/2750225257 learned_addr learned my addr 172.21.15.29:0/2750225257 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.325+0000 7f38a27fc640 1 --2- 172.21.15.29:0/2750225257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a406dce0 0x7f38a4150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:11.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.325+0000 7f38a1ffb640 1 -- 172.21.15.29:0/2750225257 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 119961451 0 0) 0x7f38a40b3130 con 0x7f38a415d980 2024-04-25T00:35:11.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.325+0000 7f38a1ffb640 1 -- 172.21.15.29:0/2750225257 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f388c004640 con 0x7f38a415d980 2024-04-25T00:35:11.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.325+0000 7f38a1ffb640 1 -- 172.21.15.29:0/2750225257 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4214071909 0 0) 0x7f388c004640 con 0x7f38a415d980 2024-04-25T00:35:11.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.325+0000 7f38a27fc640 1 -- 172.21.15.29:0/2750225257 >> v1:172.21.15.157:6789/0 conn(0x7f38a40b2b70 legacy=0x7f38a4151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.325+0000 7f38a27fc640 1 -- 172.21.15.29:0/2750225257 >> v1:172.21.15.29:6789/0 conn(0x7f38a415d980 legacy=0x7f38a406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.325+0000 7f38a27fc640 1 -- 172.21.15.29:0/2750225257 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a40b36f0 con 0x7f38a406dce0 2024-04-25T00:35:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.325+0000 7f38a27fc640 1 --2- 172.21.15.29:0/2750225257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a406dce0 0x7f38a4150df0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f38980097f0 tx=0x7f389801e430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4b123fe40a31e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.325+0000 7f38a1ffb640 1 -- 172.21.15.29:0/2750225257 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38980b2070 con 0x7f38a406dce0 2024-04-25T00:35:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.325+0000 7f38a1ffb640 1 -- 172.21.15.29:0/2750225257 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38980ad400 con 0x7f38a406dce0 2024-04-25T00:35:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.325+0000 7f38a1ffb640 1 -- 172.21.15.29:0/2750225257 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894001680 con 0x7f38a406dce0 2024-04-25T00:35:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 -- 172.21.15.29:0/2750225257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a406dce0 msgr2=0x7f38a4150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 --2- 172.21.15.29:0/2750225257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a406dce0 0x7f38a4150df0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f38980097f0 tx=0x7f389801e430 comp rx=0 tx=0).stop 2024-04-25T00:35:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 -- 172.21.15.29:0/2750225257 shutdown_connections 2024-04-25T00:35:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 --2- 172.21.15.29:0/2750225257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a406dce0 0x7f38a4150df0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 -- 172.21.15.29:0/2750225257 >> 172.21.15.29:0/2750225257 conn(0x7f38a40b1870 msgr2=0x7f38a40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 -- 172.21.15.29:0/2750225257 shutdown_connections 2024-04-25T00:35:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 -- 172.21.15.29:0/2750225257 wait complete. 2024-04-25T00:35:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 Processor -- start 2024-04-25T00:35:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 -- start start 2024-04-25T00:35:11.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a406dce0 0x7f38a41f5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:11.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40b2b70 0x7f38a414df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:11.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38a415d980 0x7f38a414e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:11.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f38a415c640 con 0x7f38a415d980 2024-04-25T00:35:11.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38a415c4c0 con 0x7f38a40b2b70 2024-04-25T00:35:11.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f38a415c7c0 con 0x7f38a406dce0 2024-04-25T00:35:11.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38a415d980 0x7f38a414e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:11.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38a415d980 0x7f38a414e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59160/0 (socket says 172.21.15.29:59160) 2024-04-25T00:35:11.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a37fe640 1 -- 172.21.15.29:0/2341451310 learned_addr learned my addr 172.21.15.29:0/2341451310 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:11.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a2ffd640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a406dce0 0x7f38a41f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:11.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a27fc640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40b2b70 0x7f38a414df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:11.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a37fe640 1 -- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a406dce0 msgr2=0x7f38a41f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a37fe640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a406dce0 0x7f38a41f5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a37fe640 1 -- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40b2b70 msgr2=0x7f38a414df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a37fe640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40b2b70 0x7f38a414df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a37fe640 1 -- 172.21.15.29:0/2341451310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a41509c0 con 0x7f38a415d980 2024-04-25T00:35:11.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a37fe640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38a415d980 0x7f38a414e470 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c8a0 tx=0x7f389c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:11.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f388b7fe640 1 -- 172.21.15.29:0/2341451310 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c017070 con 0x7f38a415d980 2024-04-25T00:35:11.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f388b7fe640 1 -- 172.21.15.29:0/2341451310 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f389c003db0 con 0x7f38a415d980 2024-04-25T00:35:11.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 -- 172.21.15.29:0/2341451310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a4150ca0 con 0x7f38a415d980 2024-04-25T00:35:11.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f388b7fe640 1 -- 172.21.15.29:0/2341451310 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c0052b0 con 0x7f38a415d980 2024-04-25T00:35:11.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 -- 172.21.15.29:0/2341451310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a4151260 con 0x7f38a415d980 2024-04-25T00:35:11.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.329+0000 7f38a94f7640 1 -- 172.21.15.29:0/2341451310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a4069270 con 0x7f38a415d980 2024-04-25T00:35:11.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.333+0000 7f388b7fe640 1 -- 172.21.15.29:0/2341451310 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f389c01a070 con 0x7f38a415d980 2024-04-25T00:35:11.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.333+0000 7f388b7fe640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3890067a70 0x7f3890069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:11.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.333+0000 7f388b7fe640 1 -- 172.21.15.29:0/2341451310 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f389c085700 con 0x7f38a415d980 2024-04-25T00:35:11.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.333+0000 7f38a2ffd640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3890067a70 0x7f3890069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:11.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.333+0000 7f38a2ffd640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3890067a70 0x7f3890069f30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3894001ec0 tx=0x7f3894001c30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:11.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.337+0000 7f388b7fe640 1 -- 172.21.15.29:0/2341451310 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f389c056050 con 0x7f38a415d980 2024-04-25T00:35:11.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.497+0000 7f38a94f7640 1 -- 172.21.15.29:0/2341451310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38a406e1d0 con 0x7f38a415d980 2024-04-25T00:35:11.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.497+0000 7f388b7fe640 1 -- 172.21.15.29:0/2341451310 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16617 (secure 0 0 0) 0x7f389c055950 con 0x7f38a415d980 2024-04-25T00:35:11.499 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:11.499 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":58,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:08.767613+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:05.458923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:06.104267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:11.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.501+0000 7f38a94f7640 1 -- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3890067a70 msgr2=0x7f3890069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.501+0000 7f38a94f7640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3890067a70 0x7f3890069f30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3894001ec0 tx=0x7f3894001c30 comp rx=0 tx=0).stop 2024-04-25T00:35:11.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.501+0000 7f38a94f7640 1 -- 172.21.15.29:0/2341451310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38a415d980 msgr2=0x7f38a414e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.501+0000 7f38a94f7640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38a415d980 0x7f38a414e470 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c8a0 tx=0x7f389c00cd60 comp rx=0 tx=0).stop 2024-04-25T00:35:11.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.501+0000 7f38a94f7640 1 -- 172.21.15.29:0/2341451310 shutdown_connections 2024-04-25T00:35:11.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.501+0000 7f38a94f7640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38a415d980 0x7f38a414e470 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.501+0000 7f38a94f7640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3890067a70 0x7f3890069f30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.501+0000 7f38a94f7640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40b2b70 0x7f38a414df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.501+0000 7f38a94f7640 1 --2- 172.21.15.29:0/2341451310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38a406dce0 0x7f38a41f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.501+0000 7f38a94f7640 1 -- 172.21.15.29:0/2341451310 >> 172.21.15.29:0/2341451310 conn(0x7f38a40b1870 msgr2=0x7f38a40b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:11.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.501+0000 7f38a94f7640 1 -- 172.21.15.29:0/2341451310 shutdown_connections 2024-04-25T00:35:11.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.501+0000 7f38a94f7640 1 -- 172.21.15.29:0/2341451310 wait complete. 2024-04-25T00:35:11.517 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:35:11.518 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:35:11.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.641+0000 7f4b7364a640 1 Processor -- start 2024-04-25T00:35:11.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.641+0000 7f4b7364a640 1 -- start start 2024-04-25T00:35:11.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.641+0000 7f4b7364a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6c15e8d0 0x7f4b6c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:11.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.641+0000 7f4b7364a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4b6c0b4780 0x7f4b6c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:11.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.641+0000 7f4b7364a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b6c0b3720 con 0x7f4b6c15f1f0 2024-04-25T00:35:11.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.641+0000 7f4b7364a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4b6c0b42f0 con 0x7f4b6c0b4780 2024-04-25T00:35:11.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.641+0000 7f4b7364a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6c15d7e0 con 0x7f4b6c15e8d0 2024-04-25T00:35:11.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.645+0000 7f4b713e6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6c15e8d0 0x7f4b6c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:11.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.645+0000 7f4b70be5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4b6c0b4780 0x7f4b6c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:11.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.645+0000 7f4b71be7640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4b6c15f1f0 0x7f4b6c15b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:33072/0 (socket says 172.21.15.29:33072) 2024-04-25T00:35:11.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.645+0000 7f4b71be7640 1 -- 172.21.15.29:0/2644957553 learned_addr learned my addr 172.21.15.29:0/2644957553 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:11.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.645+0000 7f4b713e6640 1 -- 172.21.15.29:0/2644957553 >> v2:172.21.15.157:3300/0 conn(0x7f4b6c0b4780 msgr2=0x7f4b6c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:11.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.645+0000 7f4b713e6640 1 --2- 172.21.15.29:0/2644957553 >> v2:172.21.15.157:3300/0 conn(0x7f4b6c0b4780 0x7f4b6c15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.645+0000 7f4b713e6640 1 -- 172.21.15.29:0/2644957553 >> v1:172.21.15.157:6789/0 conn(0x7f4b6c15f1f0 legacy=0x7f4b6c15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.645+0000 7f4b713e6640 1 -- 172.21.15.29:0/2644957553 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c0b39c0 con 0x7f4b6c15e8d0 2024-04-25T00:35:11.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.645+0000 7f4b70be5640 1 --2- 172.21.15.29:0/2644957553 >> v2:172.21.15.157:3300/0 conn(0x7f4b6c0b4780 0x7f4b6c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.645+0000 7f4b713e6640 1 --2- 172.21.15.29:0/2644957553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6c15e8d0 0x7f4b6c0cc240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b680097c0 tx=0x7f4b6801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c140a6cf720f7b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.645+0000 7f4b63fff640 1 -- 172.21.15.29:0/2644957553 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b680a7070 con 0x7f4b6c15e8d0 2024-04-25T00:35:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.645+0000 7f4b63fff640 1 -- 172.21.15.29:0/2644957553 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b680b0070 con 0x7f4b6c15e8d0 2024-04-25T00:35:11.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.645+0000 7f4b63fff640 1 -- 172.21.15.29:0/2644957553 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c15c1c0 con 0x7f4b6c15e8d0 2024-04-25T00:35:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 -- 172.21.15.29:0/2644957553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6c15e8d0 msgr2=0x7f4b6c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 --2- 172.21.15.29:0/2644957553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6c15e8d0 0x7f4b6c0cc240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b680097c0 tx=0x7f4b6801c470 comp rx=0 tx=0).stop 2024-04-25T00:35:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 -- 172.21.15.29:0/2644957553 shutdown_connections 2024-04-25T00:35:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 --2- 172.21.15.29:0/2644957553 >> v2:172.21.15.157:3300/0 conn(0x7f4b6c0b4780 0x7f4b6c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 --2- 172.21.15.29:0/2644957553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6c15e8d0 0x7f4b6c0cc240 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 -- 172.21.15.29:0/2644957553 >> 172.21.15.29:0/2644957553 conn(0x7f4b6c0b1870 msgr2=0x7f4b6c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 -- 172.21.15.29:0/2644957553 shutdown_connections 2024-04-25T00:35:11.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 -- 172.21.15.29:0/2644957553 wait complete. 2024-04-25T00:35:11.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 Processor -- start 2024-04-25T00:35:11.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 -- start start 2024-04-25T00:35:11.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6c0b4780 0x7f4b6c157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:11.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6c15e8d0 0x7f4b6c15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:11.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6c15f1f0 0x7f4b6c138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:11.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c144a20 con 0x7f4b6c0b4780 2024-04-25T00:35:11.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c15f600 con 0x7f4b6c15e8d0 2024-04-25T00:35:11.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6c139b60 con 0x7f4b6c15f1f0 2024-04-25T00:35:11.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b713e6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6c0b4780 0x7f4b6c157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:11.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b713e6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6c0b4780 0x7f4b6c157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59170/0 (socket says 172.21.15.29:59170) 2024-04-25T00:35:11.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b713e6640 1 -- 172.21.15.29:0/1508013819 learned_addr learned my addr 172.21.15.29:0/1508013819 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:11.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b71be7640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6c15f1f0 0x7f4b6c138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:11.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b713e6640 1 -- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6c15f1f0 msgr2=0x7f4b6c138ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b713e6640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6c15f1f0 0x7f4b6c138ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b70be5640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6c15e8d0 0x7f4b6c15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:11.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b713e6640 1 -- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6c15e8d0 msgr2=0x7f4b6c15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b713e6640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6c15e8d0 0x7f4b6c15aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b713e6640 1 -- 172.21.15.29:0/1508013819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c15a620 con 0x7f4b6c0b4780 2024-04-25T00:35:11.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b71be7640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6c15f1f0 0x7f4b6c138ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:11.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b70be5640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6c15e8d0 0x7f4b6c15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:11.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b713e6640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6c0b4780 0x7f4b6c157b50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4b6801cd10 tx=0x7f4b6801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:11.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b61ffb640 1 -- 172.21.15.29:0/1508013819 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b68006e20 con 0x7f4b6c0b4780 2024-04-25T00:35:11.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b61ffb640 1 -- 172.21.15.29:0/1508013819 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b680b5600 con 0x7f4b6c0b4780 2024-04-25T00:35:11.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 -- 172.21.15.29:0/1508013819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6c15a900 con 0x7f4b6c0b4780 2024-04-25T00:35:11.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b61ffb640 1 -- 172.21.15.29:0/1508013819 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b680b59d0 con 0x7f4b6c0b4780 2024-04-25T00:35:11.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 -- 172.21.15.29:0/1508013819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b6c160910 con 0x7f4b6c0b4780 2024-04-25T00:35:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.649+0000 7f4b7364a640 1 -- 172.21.15.29:0/1508013819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6c069270 con 0x7f4b6c0b4780 2024-04-25T00:35:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.653+0000 7f4b61ffb640 1 -- 172.21.15.29:0/1508013819 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f4b680bd070 con 0x7f4b6c0b4780 2024-04-25T00:35:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.653+0000 7f4b61ffb640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b50067a70 0x7f4b50069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.653+0000 7f4b61ffb640 1 -- 172.21.15.29:0/1508013819 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4b68120000 con 0x7f4b6c0b4780 2024-04-25T00:35:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.653+0000 7f4b70be5640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b50067a70 0x7f4b50069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.653+0000 7f4b70be5640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b50067a70 0x7f4b50069f30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c06ee50 tx=0x7f4b5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:11.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.657+0000 7f4b61ffb640 1 -- 172.21.15.29:0/1508013819 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4b68016090 con 0x7f4b6c0b4780 2024-04-25T00:35:11.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.809+0000 7f4b7364a640 1 -- 172.21.15.29:0/1508013819 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b6c15ae20 con 0x7f4b50067a70 2024-04-25T00:35:11.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.829+0000 7f4b61ffb640 1 -- 172.21.15.29:0/1508013819 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251889 (secure 0 0 0) 0x7f4b6c15ae20 con 0x7f4b50067a70 2024-04-25T00:35:11.833 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:11.835 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:35:11.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.833+0000 7f4b7364a640 1 -- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b50067a70 msgr2=0x7f4b50069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.833+0000 7f4b7364a640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b50067a70 0x7f4b50069f30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c06ee50 tx=0x7f4b5c008040 comp rx=0 tx=0).stop 2024-04-25T00:35:11.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.833+0000 7f4b7364a640 1 -- 172.21.15.29:0/1508013819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6c0b4780 msgr2=0x7f4b6c157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:11.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.833+0000 7f4b7364a640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6c0b4780 0x7f4b6c157b50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4b6801cd10 tx=0x7f4b6801acb0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.833+0000 7f4b7364a640 1 -- 172.21.15.29:0/1508013819 shutdown_connections 2024-04-25T00:35:11.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.833+0000 7f4b7364a640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6c15f1f0 0x7f4b6c138ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.833+0000 7f4b7364a640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4b50067a70 0x7f4b50069f30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.833+0000 7f4b7364a640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6c15e8d0 0x7f4b6c15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.833+0000 7f4b7364a640 1 --2- 172.21.15.29:0/1508013819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6c0b4780 0x7f4b6c157b50 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:11.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.833+0000 7f4b7364a640 1 -- 172.21.15.29:0/1508013819 >> 172.21.15.29:0/1508013819 conn(0x7f4b6c0b1870 msgr2=0x7f4b6c0b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:11.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.833+0000 7f4b7364a640 1 -- 172.21.15.29:0/1508013819 shutdown_connections 2024-04-25T00:35:11.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:11.833+0000 7f4b7364a640 1 -- 172.21.15.29:0/1508013819 wait complete. 2024-04-25T00:35:11.849 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-04-25T00:35:11.522201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268468,"kb_used_data":6548,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754706252,"statfs":{"total":773094113280,"available":772819202048,"internally_reserved":0,"allocated":6705152,"data_stored":3459396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.789282"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.823842+0000","last_change":"2024-04-25T00:35:07.822239+0000","last_active":"2024-04-25T00:35:07.823842+0000","last_peered":"2024-04-25T00:35:07.823842+0000","last_clean":"2024-04-25T00:35:07.823842+0000","last_became_active":"2024-04-25T00:35:07.821746+0000","last_became_peered":"2024-04-25T00:35:07.821746+0000","last_unstale":"2024-04-25T00:35:07.823842+0000","last_undegraded":"2024-04-25T00:35:07.823842+0000","last_fullsized":"2024-04-25T00:35:07.823842+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:07.309543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822832+0000","last_change":"2024-04-25T00:35:07.809147+0000","last_active":"2024-04-25T00:35:07.822832+0000","last_peered":"2024-04-25T00:35:07.822832+0000","last_clean":"2024-04-25T00:35:07.822832+0000","last_became_active":"2024-04-25T00:35:07.804819+0000","last_became_peered":"2024-04-25T00:35:07.804819+0000","last_unstale":"2024-04-25T00:35:07.822832+0000","last_undegraded":"2024-04-25T00:35:07.822832+0000","last_fullsized":"2024-04-25T00:35:07.822832+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:09.219797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.825218+0000","last_change":"2024-04-25T00:35:07.824004+0000","last_active":"2024-04-25T00:35:07.825218+0000","last_peered":"2024-04-25T00:35:07.825218+0000","last_clean":"2024-04-25T00:35:07.825218+0000","last_became_active":"2024-04-25T00:35:07.822761+0000","last_became_peered":"2024-04-25T00:35:07.822761+0000","last_unstale":"2024-04-25T00:35:07.825218+0000","last_undegraded":"2024-04-25T00:35:07.825218+0000","last_fullsized":"2024-04-25T00:35:07.825218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:28.555554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.815188+0000","last_change":"2024-04-25T00:35:07.804678+0000","last_active":"2024-04-25T00:35:07.815188+0000","last_peered":"2024-04-25T00:35:07.815188+0000","last_clean":"2024-04-25T00:35:07.815188+0000","last_became_active":"2024-04-25T00:35:07.803986+0000","last_became_peered":"2024-04-25T00:35:07.803986+0000","last_unstale":"2024-04-25T00:35:07.815188+0000","last_undegraded":"2024-04-25T00:35:07.815188+0000","last_fullsized":"2024-04-25T00:35:07.815188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:41.263988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.821054+0000","last_change":"2024-04-25T00:35:07.803137+0000","last_active":"2024-04-25T00:35:07.821054+0000","last_peered":"2024-04-25T00:35:07.821054+0000","last_clean":"2024-04-25T00:35:07.821054+0000","last_became_active":"2024-04-25T00:35:07.801969+0000","last_became_peered":"2024-04-25T00:35:07.801969+0000","last_unstale":"2024-04-25T00:35:07.821054+0000","last_undegraded":"2024-04-25T00:35:07.821054+0000","last_fullsized":"2024-04-25T00:35:07.821054+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:54.231727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822027+0000","last_change":"2024-04-25T00:35:07.810933+0000","last_active":"2024-04-25T00:35:07.822027+0000","last_peered":"2024-04-25T00:35:07.822027+0000","last_clean":"2024-04-25T00:35:07.822027+0000","last_became_active":"2024-04-25T00:35:07.796146+0000","last_became_peered":"2024-04-25T00:35:07.796146+0000","last_unstale":"2024-04-25T00:35:07.822027+0000","last_undegraded":"2024-04-25T00:35:07.822027+0000","last_fullsized":"2024-04-25T00:35:07.822027+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.734173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.795189+0000","last_change":"2024-04-25T00:35:07.788292+0000","last_active":"2024-04-25T00:35:07.795189+0000","last_peered":"2024-04-25T00:35:07.795189+0000","last_clean":"2024-04-25T00:35:07.795189+0000","last_became_active":"2024-04-25T00:35:07.787495+0000","last_became_peered":"2024-04-25T00:35:07.787495+0000","last_unstale":"2024-04-25T00:35:07.795189+0000","last_undegraded":"2024-04-25T00:35:07.795189+0000","last_fullsized":"2024-04-25T00:35:07.795189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:35.987615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822267+0000","last_change":"2024-04-25T00:35:07.804218+0000","last_active":"2024-04-25T00:35:07.822267+0000","last_peered":"2024-04-25T00:35:07.822267+0000","last_clean":"2024-04-25T00:35:07.822267+0000","last_became_active":"2024-04-25T00:35:07.803430+0000","last_became_peered":"2024-04-25T00:35:07.803430+0000","last_unstale":"2024-04-25T00:35:07.822267+0000","last_undegraded":"2024-04-25T00:35:07.822267+0000","last_fullsized":"2024-04-25T00:35:07.822267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:47.139485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.814680+0000","last_change":"2024-04-25T00:35:07.804978+0000","last_active":"2024-04-25T00:35:07.814680+0000","last_peered":"2024-04-25T00:35:07.814680+0000","last_clean":"2024-04-25T00:35:07.814680+0000","last_became_active":"2024-04-25T00:35:07.804265+0000","last_became_peered":"2024-04-25T00:35:07.804265+0000","last_unstale":"2024-04-25T00:35:07.814680+0000","last_undegraded":"2024-04-25T00:35:07.814680+0000","last_fullsized":"2024-04-25T00:35:07.814680+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:02.770380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.800415+0000","last_change":"2024-04-25T00:35:07.794455+0000","last_active":"2024-04-25T00:35:07.800415+0000","last_peered":"2024-04-25T00:35:07.800415+0000","last_clean":"2024-04-25T00:35:07.800415+0000","last_became_active":"2024-04-25T00:35:07.793690+0000","last_became_peered":"2024-04-25T00:35:07.793690+0000","last_unstale":"2024-04-25T00:35:07.800415+0000","last_undegraded":"2024-04-25T00:35:07.800415+0000","last_fullsized":"2024-04-25T00:35:07.800415+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:23.889772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.818055+0000","last_change":"2024-04-25T00:35:07.813286+0000","last_active":"2024-04-25T00:35:07.818055+0000","last_peered":"2024-04-25T00:35:07.818055+0000","last_clean":"2024-04-25T00:35:07.818055+0000","last_became_active":"2024-04-25T00:35:07.812547+0000","last_became_peered":"2024-04-25T00:35:07.812547+0000","last_unstale":"2024-04-25T00:35:07.818055+0000","last_undegraded":"2024-04-25T00:35:07.818055+0000","last_fullsized":"2024-04-25T00:35:07.818055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:35.987615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.793287+0000","last_change":"2024-04-25T00:35:07.787035+0000","last_active":"2024-04-25T00:35:07.793287+0000","last_peered":"2024-04-25T00:35:07.793287+0000","last_clean":"2024-04-25T00:35:07.793287+0000","last_became_active":"2024-04-25T00:35:07.786332+0000","last_became_peered":"2024-04-25T00:35:07.786332+0000","last_unstale":"2024-04-25T00:35:07.793287+0000","last_undegraded":"2024-04-25T00:35:07.793287+0000","last_fullsized":"2024-04-25T00:35:07.793287+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:34.426589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.810050+0000","last_change":"2024-04-25T00:35:07.807593+0000","last_active":"2024-04-25T00:35:07.810050+0000","last_peered":"2024-04-25T00:35:07.810050+0000","last_clean":"2024-04-25T00:35:07.810050+0000","last_became_active":"2024-04-25T00:35:07.806877+0000","last_became_peered":"2024-04-25T00:35:07.806877+0000","last_unstale":"2024-04-25T00:35:07.810050+0000","last_undegraded":"2024-04-25T00:35:07.810050+0000","last_fullsized":"2024-04-25T00:35:07.810050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:23.690728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.800571+0000","last_change":"2024-04-25T00:35:07.798753+0000","last_active":"2024-04-25T00:35:07.800571+0000","last_peered":"2024-04-25T00:35:07.800571+0000","last_clean":"2024-04-25T00:35:07.800571+0000","last_became_active":"2024-04-25T00:35:07.797765+0000","last_became_peered":"2024-04-25T00:35:07.797765+0000","last_unstale":"2024-04-25T00:35:07.800571+0000","last_undegraded":"2024-04-25T00:35:07.800571+0000","last_fullsized":"2024-04-25T00:35:07.800571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:35.412434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.798543+0000","last_change":"2024-04-25T00:35:07.794567+0000","last_active":"2024-04-25T00:35:07.798543+0000","last_peered":"2024-04-25T00:35:07.798543+0000","last_clean":"2024-04-25T00:35:07.798543+0000","last_became_active":"2024-04-25T00:35:07.793650+0000","last_became_peered":"2024-04-25T00:35:07.793650+0000","last_unstale":"2024-04-25T00:35:07.798543+0000","last_undegraded":"2024-04-25T00:35:07.798543+0000","last_fullsized":"2024-04-25T00:35:07.798543+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:32.706100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.825339+0000","last_change":"2024-04-25T00:35:07.806476+0000","last_active":"2024-04-25T00:35:07.825339+0000","last_peered":"2024-04-25T00:35:07.825339+0000","last_clean":"2024-04-25T00:35:07.825339+0000","last_became_active":"2024-04-25T00:35:07.805284+0000","last_became_peered":"2024-04-25T00:35:07.805284+0000","last_unstale":"2024-04-25T00:35:07.825339+0000","last_undegraded":"2024-04-25T00:35:07.825339+0000","last_fullsized":"2024-04-25T00:35:07.825339+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:34.426589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.823713+0000","last_change":"2024-04-25T00:35:07.809793+0000","last_active":"2024-04-25T00:35:07.823713+0000","last_peered":"2024-04-25T00:35:07.823713+0000","last_clean":"2024-04-25T00:35:07.823713+0000","last_became_active":"2024-04-25T00:35:07.809006+0000","last_became_peered":"2024-04-25T00:35:07.809006+0000","last_unstale":"2024-04-25T00:35:07.823713+0000","last_undegraded":"2024-04-25T00:35:07.823713+0000","last_fullsized":"2024-04-25T00:35:07.823713+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:13.114459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824636+0000","last_change":"2024-04-25T00:35:07.818709+0000","last_active":"2024-04-25T00:35:07.824636+0000","last_peered":"2024-04-25T00:35:07.824636+0000","last_clean":"2024-04-25T00:35:07.824636+0000","last_became_active":"2024-04-25T00:35:07.817911+0000","last_became_peered":"2024-04-25T00:35:07.817911+0000","last_unstale":"2024-04-25T00:35:07.824636+0000","last_undegraded":"2024-04-25T00:35:07.824636+0000","last_fullsized":"2024-04-25T00:35:07.824636+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:22.804662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822489+0000","last_change":"2024-04-25T00:35:07.820860+0000","last_active":"2024-04-25T00:35:07.822489+0000","last_peered":"2024-04-25T00:35:07.822489+0000","last_clean":"2024-04-25T00:35:07.822489+0000","last_became_active":"2024-04-25T00:35:07.820496+0000","last_became_peered":"2024-04-25T00:35:07.820496+0000","last_unstale":"2024-04-25T00:35:07.822489+0000","last_undegraded":"2024-04-25T00:35:07.822489+0000","last_fullsized":"2024-04-25T00:35:07.822489+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:51.679806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822883+0000","last_change":"2024-04-25T00:35:07.809234+0000","last_active":"2024-04-25T00:35:07.822883+0000","last_peered":"2024-04-25T00:35:07.822883+0000","last_clean":"2024-04-25T00:35:07.822883+0000","last_became_active":"2024-04-25T00:35:07.808481+0000","last_became_peered":"2024-04-25T00:35:07.808481+0000","last_unstale":"2024-04-25T00:35:07.822883+0000","last_undegraded":"2024-04-25T00:35:07.822883+0000","last_fullsized":"2024-04-25T00:35:07.822883+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:44.325402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.812488+0000","last_change":"2024-04-25T00:35:07.809751+0000","last_active":"2024-04-25T00:35:07.812488+0000","last_peered":"2024-04-25T00:35:07.812488+0000","last_clean":"2024-04-25T00:35:07.812488+0000","last_became_active":"2024-04-25T00:35:07.806831+0000","last_became_peered":"2024-04-25T00:35:07.806831+0000","last_unstale":"2024-04-25T00:35:07.812488+0000","last_undegraded":"2024-04-25T00:35:07.812488+0000","last_fullsized":"2024-04-25T00:35:07.812488+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:16.351363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.827253+0000","last_change":"2024-04-25T00:35:07.825780+0000","last_active":"2024-04-25T00:35:07.827253+0000","last_peered":"2024-04-25T00:35:07.827253+0000","last_clean":"2024-04-25T00:35:07.827253+0000","last_became_active":"2024-04-25T00:35:07.824515+0000","last_became_peered":"2024-04-25T00:35:07.824515+0000","last_unstale":"2024-04-25T00:35:07.827253+0000","last_undegraded":"2024-04-25T00:35:07.827253+0000","last_fullsized":"2024-04-25T00:35:07.827253+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:20.156676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824441+0000","last_change":"2024-04-25T00:35:07.817376+0000","last_active":"2024-04-25T00:35:07.824441+0000","last_peered":"2024-04-25T00:35:07.824441+0000","last_clean":"2024-04-25T00:35:07.824441+0000","last_became_active":"2024-04-25T00:35:07.815097+0000","last_became_peered":"2024-04-25T00:35:07.815097+0000","last_unstale":"2024-04-25T00:35:07.824441+0000","last_undegraded":"2024-04-25T00:35:07.824441+0000","last_fullsized":"2024-04-25T00:35:07.824441+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:41.263988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.816741+0000","last_change":"2024-04-25T00:35:07.811010+0000","last_active":"2024-04-25T00:35:07.816741+0000","last_peered":"2024-04-25T00:35:07.816741+0000","last_clean":"2024-04-25T00:35:07.816741+0000","last_became_active":"2024-04-25T00:35:07.809910+0000","last_became_peered":"2024-04-25T00:35:07.809910+0000","last_unstale":"2024-04-25T00:35:07.816741+0000","last_undegraded":"2024-04-25T00:35:07.816741+0000","last_fullsized":"2024-04-25T00:35:07.816741+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:02.293518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.820802+0000","last_change":"2024-04-25T00:35:07.802933+0000","last_active":"2024-04-25T00:35:07.820802+0000","last_peered":"2024-04-25T00:35:07.820802+0000","last_clean":"2024-04-25T00:35:07.820802+0000","last_became_active":"2024-04-25T00:35:07.801022+0000","last_became_peered":"2024-04-25T00:35:07.801022+0000","last_unstale":"2024-04-25T00:35:07.820802+0000","last_undegraded":"2024-04-25T00:35:07.820802+0000","last_fullsized":"2024-04-25T00:35:07.820802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:47.597016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.794498+0000","last_change":"2024-04-25T00:35:07.791035+0000","last_active":"2024-04-25T00:35:07.794498+0000","last_peered":"2024-04-25T00:35:07.794498+0000","last_clean":"2024-04-25T00:35:07.794498+0000","last_became_active":"2024-04-25T00:35:07.790301+0000","last_became_peered":"2024-04-25T00:35:07.790301+0000","last_unstale":"2024-04-25T00:35:07.794498+0000","last_undegraded":"2024-04-25T00:35:07.794498+0000","last_fullsized":"2024-04-25T00:35:07.794498+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:44.227824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822778+0000","last_change":"2024-04-25T00:35:07.812129+0000","last_active":"2024-04-25T00:35:07.822778+0000","last_peered":"2024-04-25T00:35:07.822778+0000","last_clean":"2024-04-25T00:35:07.822778+0000","last_became_active":"2024-04-25T00:35:07.806228+0000","last_became_peered":"2024-04-25T00:35:07.806228+0000","last_unstale":"2024-04-25T00:35:07.822778+0000","last_undegraded":"2024-04-25T00:35:07.822778+0000","last_fullsized":"2024-04-25T00:35:07.822778+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:44.227824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.821098+0000","last_change":"2024-04-25T00:35:07.793709+0000","last_active":"2024-04-25T00:35:07.821098+0000","last_peered":"2024-04-25T00:35:07.821098+0000","last_clean":"2024-04-25T00:35:07.821098+0000","last_became_active":"2024-04-25T00:35:07.793060+0000","last_became_peered":"2024-04-25T00:35:07.793060+0000","last_unstale":"2024-04-25T00:35:07.821098+0000","last_undegraded":"2024-04-25T00:35:07.821098+0000","last_fullsized":"2024-04-25T00:35:07.821098+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:24.934142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824073+0000","last_change":"2024-04-25T00:35:07.819244+0000","last_active":"2024-04-25T00:35:07.824073+0000","last_peered":"2024-04-25T00:35:07.824073+0000","last_clean":"2024-04-25T00:35:07.824073+0000","last_became_active":"2024-04-25T00:35:07.818420+0000","last_became_peered":"2024-04-25T00:35:07.818420+0000","last_unstale":"2024-04-25T00:35:07.824073+0000","last_undegraded":"2024-04-25T00:35:07.824073+0000","last_fullsized":"2024-04-25T00:35:07.824073+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:43.644264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.811021+0000","last_change":"2024-04-25T00:35:07.808596+0000","last_active":"2024-04-25T00:35:07.811021+0000","last_peered":"2024-04-25T00:35:07.811021+0000","last_clean":"2024-04-25T00:35:07.811021+0000","last_became_active":"2024-04-25T00:35:07.806801+0000","last_became_peered":"2024-04-25T00:35:07.806801+0000","last_unstale":"2024-04-25T00:35:07.811021+0000","last_undegraded":"2024-04-25T00:35:07.811021+0000","last_fullsized":"2024-04-25T00:35:07.811021+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:16.590960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.820386+0000","last_change":"2024-04-25T00:35:07.794804+0000","last_active":"2024-04-25T00:35:07.820386+0000","last_peered":"2024-04-25T00:35:07.820386+0000","last_clean":"2024-04-25T00:35:07.820386+0000","last_became_active":"2024-04-25T00:35:07.793931+0000","last_became_peered":"2024-04-25T00:35:07.793931+0000","last_unstale":"2024-04-25T00:35:07.820386+0000","last_undegraded":"2024-04-25T00:35:07.820386+0000","last_fullsized":"2024-04-25T00:35:07.820386+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:50.335647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.825761+0000","last_change":"2024-04-25T00:35:07.824155+0000","last_active":"2024-04-25T00:35:07.825761+0000","last_peered":"2024-04-25T00:35:07.825761+0000","last_clean":"2024-04-25T00:35:07.825761+0000","last_became_active":"2024-04-25T00:35:07.823593+0000","last_became_peered":"2024-04-25T00:35:07.823593+0000","last_unstale":"2024-04-25T00:35:07.825761+0000","last_undegraded":"2024-04-25T00:35:07.825761+0000","last_fullsized":"2024-04-25T00:35:07.825761+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:06.584791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822639+0000","last_change":"2024-04-25T00:35:07.812833+0000","last_active":"2024-04-25T00:35:07.822639+0000","last_peered":"2024-04-25T00:35:07.822639+0000","last_clean":"2024-04-25T00:35:07.822639+0000","last_became_active":"2024-04-25T00:35:07.812060+0000","last_became_peered":"2024-04-25T00:35:07.812060+0000","last_unstale":"2024-04-25T00:35:07.822639+0000","last_undegraded":"2024-04-25T00:35:07.822639+0000","last_fullsized":"2024-04-25T00:35:07.822639+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:16.351363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.836837+0000","last_change":"2024-04-25T00:35:06.765335+0000","last_active":"2024-04-25T00:35:07.836837+0000","last_peered":"2024-04-25T00:35:07.836837+0000","last_clean":"2024-04-25T00:35:07.836837+0000","last_became_active":"2024-04-25T00:35:06.764727+0000","last_became_peered":"2024-04-25T00:35:06.764727+0000","last_unstale":"2024-04-25T00:35:07.836837+0000","last_undegraded":"2024-04-25T00:35:07.836837+0000","last_fullsized":"2024-04-25T00:35:07.836837+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:53.228452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.798497+0000","last_change":"2024-04-25T00:35:07.789834+0000","last_active":"2024-04-25T00:35:07.798497+0000","last_peered":"2024-04-25T00:35:07.798497+0000","last_clean":"2024-04-25T00:35:07.798497+0000","last_became_active":"2024-04-25T00:35:07.787542+0000","last_became_peered":"2024-04-25T00:35:07.787542+0000","last_unstale":"2024-04-25T00:35:07.798497+0000","last_undegraded":"2024-04-25T00:35:07.798497+0000","last_fullsized":"2024-04-25T00:35:07.798497+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:23.690728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.834281+0000","last_change":"2024-04-25T00:35:06.768866+0000","last_active":"2024-04-25T00:35:07.834281+0000","last_peered":"2024-04-25T00:35:07.834281+0000","last_clean":"2024-04-25T00:35:07.834281+0000","last_became_active":"2024-04-25T00:35:06.768452+0000","last_became_peered":"2024-04-25T00:35:06.768452+0000","last_unstale":"2024-04-25T00:35:07.834281+0000","last_undegraded":"2024-04-25T00:35:07.834281+0000","last_fullsized":"2024-04-25T00:35:07.834281+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:38.952798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.815281+0000","last_change":"2024-04-25T00:35:07.797375+0000","last_active":"2024-04-25T00:35:07.815281+0000","last_peered":"2024-04-25T00:35:07.815281+0000","last_clean":"2024-04-25T00:35:07.815281+0000","last_became_active":"2024-04-25T00:35:07.796362+0000","last_became_peered":"2024-04-25T00:35:07.796362+0000","last_unstale":"2024-04-25T00:35:07.815281+0000","last_undegraded":"2024-04-25T00:35:07.815281+0000","last_fullsized":"2024-04-25T00:35:07.815281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:48.598127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.835858+0000","last_change":"2024-04-25T00:35:06.772078+0000","last_active":"2024-04-25T00:35:07.835858+0000","last_peered":"2024-04-25T00:35:07.835858+0000","last_clean":"2024-04-25T00:35:07.835858+0000","last_became_active":"2024-04-25T00:35:06.767244+0000","last_became_peered":"2024-04-25T00:35:06.767244+0000","last_unstale":"2024-04-25T00:35:07.835858+0000","last_undegraded":"2024-04-25T00:35:07.835858+0000","last_fullsized":"2024-04-25T00:35:07.835858+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:16.049571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822822+0000","last_change":"2024-04-25T00:35:07.812414+0000","last_active":"2024-04-25T00:35:07.822822+0000","last_peered":"2024-04-25T00:35:07.822822+0000","last_clean":"2024-04-25T00:35:07.822822+0000","last_became_active":"2024-04-25T00:35:07.805561+0000","last_became_peered":"2024-04-25T00:35:07.805561+0000","last_unstale":"2024-04-25T00:35:07.822822+0000","last_undegraded":"2024-04-25T00:35:07.822822+0000","last_fullsized":"2024-04-25T00:35:07.822822+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:33.255653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.841549+0000","last_change":"2024-04-25T00:35:06.776301+0000","last_active":"2024-04-25T00:35:07.841549+0000","last_peered":"2024-04-25T00:35:07.841549+0000","last_clean":"2024-04-25T00:35:07.841549+0000","last_became_active":"2024-04-25T00:35:06.773770+0000","last_became_peered":"2024-04-25T00:35:06.773770+0000","last_unstale":"2024-04-25T00:35:07.841549+0000","last_undegraded":"2024-04-25T00:35:07.841549+0000","last_fullsized":"2024-04-25T00:35:07.841549+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:47.587617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.827198+0000","last_change":"2024-04-25T00:35:07.825128+0000","last_active":"2024-04-25T00:35:07.827198+0000","last_peered":"2024-04-25T00:35:07.827198+0000","last_clean":"2024-04-25T00:35:07.827198+0000","last_became_active":"2024-04-25T00:35:07.823608+0000","last_became_peered":"2024-04-25T00:35:07.823608+0000","last_unstale":"2024-04-25T00:35:07.827198+0000","last_undegraded":"2024-04-25T00:35:07.827198+0000","last_fullsized":"2024-04-25T00:35:07.827198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:56.521301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.837008+0000","last_change":"2024-04-25T00:35:06.773548+0000","last_active":"2024-04-25T00:35:07.837008+0000","last_peered":"2024-04-25T00:35:07.837008+0000","last_clean":"2024-04-25T00:35:07.837008+0000","last_became_active":"2024-04-25T00:35:06.772497+0000","last_became_peered":"2024-04-25T00:35:06.772497+0000","last_unstale":"2024-04-25T00:35:07.837008+0000","last_undegraded":"2024-04-25T00:35:07.837008+0000","last_fullsized":"2024-04-25T00:35:07.837008+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:22.879261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824041+0000","last_change":"2024-04-25T00:35:07.815053+0000","last_active":"2024-04-25T00:35:07.824041+0000","last_peered":"2024-04-25T00:35:07.824041+0000","last_clean":"2024-04-25T00:35:07.824041+0000","last_became_active":"2024-04-25T00:35:07.814314+0000","last_became_peered":"2024-04-25T00:35:07.814314+0000","last_unstale":"2024-04-25T00:35:07.824041+0000","last_undegraded":"2024-04-25T00:35:07.824041+0000","last_fullsized":"2024-04-25T00:35:07.824041+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:09.827741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.836595+0000","last_change":"2024-04-25T00:35:06.778251+0000","last_active":"2024-04-25T00:35:07.836595+0000","last_peered":"2024-04-25T00:35:07.836595+0000","last_clean":"2024-04-25T00:35:07.836595+0000","last_became_active":"2024-04-25T00:35:06.777809+0000","last_became_peered":"2024-04-25T00:35:06.777809+0000","last_unstale":"2024-04-25T00:35:07.836595+0000","last_undegraded":"2024-04-25T00:35:07.836595+0000","last_fullsized":"2024-04-25T00:35:07.836595+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:41.884020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.809317+0000","last_change":"2024-04-25T00:35:07.806157+0000","last_active":"2024-04-25T00:35:07.809317+0000","last_peered":"2024-04-25T00:35:07.809317+0000","last_clean":"2024-04-25T00:35:07.809317+0000","last_became_active":"2024-04-25T00:35:07.805431+0000","last_became_peered":"2024-04-25T00:35:07.805431+0000","last_unstale":"2024-04-25T00:35:07.809317+0000","last_undegraded":"2024-04-25T00:35:07.809317+0000","last_fullsized":"2024-04-25T00:35:07.809317+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:54.704781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.774573+0000","last_change":"2024-04-25T00:35:06.770549+0000","last_active":"2024-04-25T00:35:07.774573+0000","last_peered":"2024-04-25T00:35:07.774573+0000","last_clean":"2024-04-25T00:35:07.774573+0000","last_became_active":"2024-04-25T00:35:06.769750+0000","last_became_peered":"2024-04-25T00:35:06.769750+0000","last_unstale":"2024-04-25T00:35:07.774573+0000","last_undegraded":"2024-04-25T00:35:07.774573+0000","last_fullsized":"2024-04-25T00:35:07.774573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:36.832974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822822+0000","last_change":"2024-04-25T00:35:07.810762+0000","last_active":"2024-04-25T00:35:07.822822+0000","last_peered":"2024-04-25T00:35:07.822822+0000","last_clean":"2024-04-25T00:35:07.822822+0000","last_became_active":"2024-04-25T00:35:07.809888+0000","last_became_peered":"2024-04-25T00:35:07.809888+0000","last_unstale":"2024-04-25T00:35:07.822822+0000","last_undegraded":"2024-04-25T00:35:07.822822+0000","last_fullsized":"2024-04-25T00:35:07.822822+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.364656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.781643+0000","last_change":"2024-04-25T00:35:06.773082+0000","last_active":"2024-04-25T00:35:07.781643+0000","last_peered":"2024-04-25T00:35:07.781643+0000","last_clean":"2024-04-25T00:35:07.781643+0000","last_became_active":"2024-04-25T00:35:06.771601+0000","last_became_peered":"2024-04-25T00:35:06.771601+0000","last_unstale":"2024-04-25T00:35:07.781643+0000","last_undegraded":"2024-04-25T00:35:07.781643+0000","last_fullsized":"2024-04-25T00:35:07.781643+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:01.283008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.816657+0000","last_change":"2024-04-25T00:35:07.812042+0000","last_active":"2024-04-25T00:35:07.816657+0000","last_peered":"2024-04-25T00:35:07.816657+0000","last_clean":"2024-04-25T00:35:07.816657+0000","last_became_active":"2024-04-25T00:35:07.811262+0000","last_became_peered":"2024-04-25T00:35:07.811262+0000","last_unstale":"2024-04-25T00:35:07.816657+0000","last_undegraded":"2024-04-25T00:35:07.816657+0000","last_fullsized":"2024-04-25T00:35:07.816657+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:42.894530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.800987+0000","last_change":"2024-04-25T00:35:06.779884+0000","last_active":"2024-04-25T00:35:07.800987+0000","last_peered":"2024-04-25T00:35:07.800987+0000","last_clean":"2024-04-25T00:35:07.800987+0000","last_became_active":"2024-04-25T00:35:06.779115+0000","last_became_peered":"2024-04-25T00:35:06.779115+0000","last_unstale":"2024-04-25T00:35:07.800987+0000","last_undegraded":"2024-04-25T00:35:07.800987+0000","last_fullsized":"2024-04-25T00:35:07.800987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:43.217314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824623+0000","last_change":"2024-04-25T00:35:07.818837+0000","last_active":"2024-04-25T00:35:07.824623+0000","last_peered":"2024-04-25T00:35:07.824623+0000","last_clean":"2024-04-25T00:35:07.824623+0000","last_became_active":"2024-04-25T00:35:07.817982+0000","last_became_peered":"2024-04-25T00:35:07.817982+0000","last_unstale":"2024-04-25T00:35:07.824623+0000","last_undegraded":"2024-04-25T00:35:07.824623+0000","last_fullsized":"2024-04-25T00:35:07.824623+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:41.925732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.800307+0000","last_change":"2024-04-25T00:35:06.775556+0000","last_active":"2024-04-25T00:35:07.800307+0000","last_peered":"2024-04-25T00:35:07.800307+0000","last_clean":"2024-04-25T00:35:07.800307+0000","last_became_active":"2024-04-25T00:35:06.775027+0000","last_became_peered":"2024-04-25T00:35:06.775027+0000","last_unstale":"2024-04-25T00:35:07.800307+0000","last_undegraded":"2024-04-25T00:35:07.800307+0000","last_fullsized":"2024-04-25T00:35:07.800307+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:36.477614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.808395+0000","last_change":"2024-04-25T00:35:07.802032+0000","last_active":"2024-04-25T00:35:07.808395+0000","last_peered":"2024-04-25T00:35:07.808395+0000","last_clean":"2024-04-25T00:35:07.808395+0000","last_became_active":"2024-04-25T00:35:07.797785+0000","last_became_peered":"2024-04-25T00:35:07.797785+0000","last_unstale":"2024-04-25T00:35:07.808395+0000","last_undegraded":"2024-04-25T00:35:07.808395+0000","last_fullsized":"2024-04-25T00:35:07.808395+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:16.382487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.841160+0000","last_change":"2024-04-25T00:35:06.775930+0000","last_active":"2024-04-25T00:35:07.841160+0000","last_peered":"2024-04-25T00:35:07.841160+0000","last_clean":"2024-04-25T00:35:07.841160+0000","last_became_active":"2024-04-25T00:35:06.775242+0000","last_became_peered":"2024-04-25T00:35:06.775242+0000","last_unstale":"2024-04-25T00:35:07.841160+0000","last_undegraded":"2024-04-25T00:35:07.841160+0000","last_fullsized":"2024-04-25T00:35:07.841160+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:32.245143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.825191+0000","last_change":"2024-04-25T00:35:07.823516+0000","last_active":"2024-04-25T00:35:07.825191+0000","last_peered":"2024-04-25T00:35:07.825191+0000","last_clean":"2024-04-25T00:35:07.825191+0000","last_became_active":"2024-04-25T00:35:07.819499+0000","last_became_peered":"2024-04-25T00:35:07.819499+0000","last_unstale":"2024-04-25T00:35:07.825191+0000","last_undegraded":"2024-04-25T00:35:07.825191+0000","last_fullsized":"2024-04-25T00:35:07.825191+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:39.963308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.841731+0000","last_change":"2024-04-25T00:35:06.778729+0000","last_active":"2024-04-25T00:35:07.841731+0000","last_peered":"2024-04-25T00:35:07.841731+0000","last_clean":"2024-04-25T00:35:07.841731+0000","last_became_active":"2024-04-25T00:35:06.778016+0000","last_became_peered":"2024-04-25T00:35:06.778016+0000","last_unstale":"2024-04-25T00:35:07.841731+0000","last_undegraded":"2024-04-25T00:35:07.841731+0000","last_fullsized":"2024-04-25T00:35:07.841731+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:15.371976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.797499+0000","last_change":"2024-04-25T00:35:07.792292+0000","last_active":"2024-04-25T00:35:07.797499+0000","last_peered":"2024-04-25T00:35:07.797499+0000","last_clean":"2024-04-25T00:35:07.797499+0000","last_became_active":"2024-04-25T00:35:07.791144+0000","last_became_peered":"2024-04-25T00:35:07.791144+0000","last_unstale":"2024-04-25T00:35:07.797499+0000","last_undegraded":"2024-04-25T00:35:07.797499+0000","last_fullsized":"2024-04-25T00:35:07.797499+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:33.255653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.835950+0000","last_change":"2024-04-25T00:35:06.767630+0000","last_active":"2024-04-25T00:35:07.835950+0000","last_peered":"2024-04-25T00:35:07.835950+0000","last_clean":"2024-04-25T00:35:07.835950+0000","last_became_active":"2024-04-25T00:35:06.767232+0000","last_became_peered":"2024-04-25T00:35:06.767232+0000","last_unstale":"2024-04-25T00:35:07.835950+0000","last_undegraded":"2024-04-25T00:35:07.835950+0000","last_fullsized":"2024-04-25T00:35:07.835950+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:40.915221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.814335+0000","last_change":"2024-04-25T00:35:07.811065+0000","last_active":"2024-04-25T00:35:07.814335+0000","last_peered":"2024-04-25T00:35:07.814335+0000","last_clean":"2024-04-25T00:35:07.814335+0000","last_became_active":"2024-04-25T00:35:07.809279+0000","last_became_peered":"2024-04-25T00:35:07.809279+0000","last_unstale":"2024-04-25T00:35:07.814335+0000","last_undegraded":"2024-04-25T00:35:07.814335+0000","last_fullsized":"2024-04-25T00:35:07.814335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:06.584791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.781787+0000","last_change":"2024-04-25T00:35:06.764142+0000","last_active":"2024-04-25T00:35:07.781787+0000","last_peered":"2024-04-25T00:35:07.781787+0000","last_clean":"2024-04-25T00:35:07.781787+0000","last_became_active":"2024-04-25T00:35:06.763345+0000","last_became_peered":"2024-04-25T00:35:06.763345+0000","last_unstale":"2024-04-25T00:35:07.781787+0000","last_undegraded":"2024-04-25T00:35:07.781787+0000","last_fullsized":"2024-04-25T00:35:07.781787+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:13.274709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.798435+0000","last_change":"2024-04-25T00:35:07.792491+0000","last_active":"2024-04-25T00:35:07.798435+0000","last_peered":"2024-04-25T00:35:07.798435+0000","last_clean":"2024-04-25T00:35:07.798435+0000","last_became_active":"2024-04-25T00:35:07.791836+0000","last_became_peered":"2024-04-25T00:35:07.791836+0000","last_unstale":"2024-04-25T00:35:07.798435+0000","last_undegraded":"2024-04-25T00:35:07.798435+0000","last_fullsized":"2024-04-25T00:35:07.798435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:22.500973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.774965+0000","last_change":"2024-04-25T00:35:06.775947+0000","last_active":"2024-04-25T00:35:07.774965+0000","last_peered":"2024-04-25T00:35:07.774965+0000","last_clean":"2024-04-25T00:35:07.774965+0000","last_became_active":"2024-04-25T00:35:06.775112+0000","last_became_peered":"2024-04-25T00:35:06.775112+0000","last_unstale":"2024-04-25T00:35:07.774965+0000","last_undegraded":"2024-04-25T00:35:07.774965+0000","last_fullsized":"2024-04-25T00:35:07.774965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:16.049571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824511+0000","last_change":"2024-04-25T00:35:07.817742+0000","last_active":"2024-04-25T00:35:07.824511+0000","last_peered":"2024-04-25T00:35:07.824511+0000","last_clean":"2024-04-25T00:35:07.824511+0000","last_became_active":"2024-04-25T00:35:07.817069+0000","last_became_peered":"2024-04-25T00:35:07.817069+0000","last_unstale":"2024-04-25T00:35:07.824511+0000","last_undegraded":"2024-04-25T00:35:07.824511+0000","last_fullsized":"2024-04-25T00:35:07.824511+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:34.831027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.843947+0000","last_change":"2024-04-25T00:35:06.782423+0000","last_active":"2024-04-25T00:35:07.843947+0000","last_peered":"2024-04-25T00:35:07.843947+0000","last_clean":"2024-04-25T00:35:07.843947+0000","last_became_active":"2024-04-25T00:35:06.781791+0000","last_became_peered":"2024-04-25T00:35:06.781791+0000","last_unstale":"2024-04-25T00:35:07.843947+0000","last_undegraded":"2024-04-25T00:35:07.843947+0000","last_fullsized":"2024-04-25T00:35:07.843947+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:59.723663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824689+0000","last_change":"2024-04-25T00:35:07.817996+0000","last_active":"2024-04-25T00:35:07.824689+0000","last_peered":"2024-04-25T00:35:07.824689+0000","last_clean":"2024-04-25T00:35:07.824689+0000","last_became_active":"2024-04-25T00:35:07.817328+0000","last_became_peered":"2024-04-25T00:35:07.817328+0000","last_unstale":"2024-04-25T00:35:07.824689+0000","last_undegraded":"2024-04-25T00:35:07.824689+0000","last_fullsized":"2024-04-25T00:35:07.824689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:31.098467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.834139+0000","last_change":"2024-04-25T00:35:06.772681+0000","last_active":"2024-04-25T00:35:07.834139+0000","last_peered":"2024-04-25T00:35:07.834139+0000","last_clean":"2024-04-25T00:35:07.834139+0000","last_became_active":"2024-04-25T00:35:06.772017+0000","last_became_peered":"2024-04-25T00:35:06.772017+0000","last_unstale":"2024-04-25T00:35:07.834139+0000","last_undegraded":"2024-04-25T00:35:07.834139+0000","last_fullsized":"2024-04-25T00:35:07.834139+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:30.087957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.813750+0000","last_change":"2024-04-25T00:35:07.801359+0000","last_active":"2024-04-25T00:35:07.813750+0000","last_peered":"2024-04-25T00:35:07.813750+0000","last_clean":"2024-04-25T00:35:07.813750+0000","last_became_active":"2024-04-25T00:35:07.800582+0000","last_became_peered":"2024-04-25T00:35:07.800582+0000","last_unstale":"2024-04-25T00:35:07.813750+0000","last_undegraded":"2024-04-25T00:35:07.813750+0000","last_fullsized":"2024-04-25T00:35:07.813750+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:16.351363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.834771+0000","last_change":"2024-04-25T00:35:06.776351+0000","last_active":"2024-04-25T00:35:07.834771+0000","last_peered":"2024-04-25T00:35:07.834771+0000","last_clean":"2024-04-25T00:35:07.834771+0000","last_became_active":"2024-04-25T00:35:06.775959+0000","last_became_peered":"2024-04-25T00:35:06.775959+0000","last_unstale":"2024-04-25T00:35:07.834771+0000","last_undegraded":"2024-04-25T00:35:07.834771+0000","last_fullsized":"2024-04-25T00:35:07.834771+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:00.087036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822224+0000","last_change":"2024-04-25T00:35:07.804006+0000","last_active":"2024-04-25T00:35:07.822224+0000","last_peered":"2024-04-25T00:35:07.822224+0000","last_clean":"2024-04-25T00:35:07.822224+0000","last_became_active":"2024-04-25T00:35:07.803267+0000","last_became_peered":"2024-04-25T00:35:07.803267+0000","last_unstale":"2024-04-25T00:35:07.822224+0000","last_undegraded":"2024-04-25T00:35:07.822224+0000","last_fullsized":"2024-04-25T00:35:07.822224+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:28.555554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.843186+0000","last_change":"2024-04-25T00:35:06.775580+0000","last_active":"2024-04-25T00:35:07.843186+0000","last_peered":"2024-04-25T00:35:07.843186+0000","last_clean":"2024-04-25T00:35:07.843186+0000","last_became_active":"2024-04-25T00:35:06.773804+0000","last_became_peered":"2024-04-25T00:35:06.773804+0000","last_unstale":"2024-04-25T00:35:07.843186+0000","last_undegraded":"2024-04-25T00:35:07.843186+0000","last_fullsized":"2024-04-25T00:35:07.843186+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:16.627989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.825167+0000","last_change":"2024-04-25T00:35:07.822995+0000","last_active":"2024-04-25T00:35:07.825167+0000","last_peered":"2024-04-25T00:35:07.825167+0000","last_clean":"2024-04-25T00:35:07.825167+0000","last_became_active":"2024-04-25T00:35:07.818496+0000","last_became_peered":"2024-04-25T00:35:07.818496+0000","last_unstale":"2024-04-25T00:35:07.825167+0000","last_undegraded":"2024-04-25T00:35:07.825167+0000","last_fullsized":"2024-04-25T00:35:07.825167+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:37.488124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.795441+0000","last_change":"2024-04-25T00:35:06.772616+0000","last_active":"2024-04-25T00:35:07.795441+0000","last_peered":"2024-04-25T00:35:07.795441+0000","last_clean":"2024-04-25T00:35:07.795441+0000","last_became_active":"2024-04-25T00:35:06.771660+0000","last_became_peered":"2024-04-25T00:35:06.771660+0000","last_unstale":"2024-04-25T00:35:07.795441+0000","last_undegraded":"2024-04-25T00:35:07.795441+0000","last_fullsized":"2024-04-25T00:35:07.795441+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:35.183700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.797564+0000","last_change":"2024-04-25T00:35:07.792613+0000","last_active":"2024-04-25T00:35:07.797564+0000","last_peered":"2024-04-25T00:35:07.797564+0000","last_clean":"2024-04-25T00:35:07.797564+0000","last_became_active":"2024-04-25T00:35:07.791912+0000","last_became_peered":"2024-04-25T00:35:07.791912+0000","last_unstale":"2024-04-25T00:35:07.797564+0000","last_undegraded":"2024-04-25T00:35:07.797564+0000","last_fullsized":"2024-04-25T00:35:07.797564+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:16.590960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.837909+0000","last_change":"2024-04-25T00:35:06.769911+0000","last_active":"2024-04-25T00:35:07.837909+0000","last_peered":"2024-04-25T00:35:07.837909+0000","last_clean":"2024-04-25T00:35:07.837909+0000","last_became_active":"2024-04-25T00:35:06.769533+0000","last_became_peered":"2024-04-25T00:35:06.769533+0000","last_unstale":"2024-04-25T00:35:07.837909+0000","last_undegraded":"2024-04-25T00:35:07.837909+0000","last_fullsized":"2024-04-25T00:35:07.837909+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:21.794152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.815374+0000","last_change":"2024-04-25T00:35:07.804754+0000","last_active":"2024-04-25T00:35:07.815374+0000","last_peered":"2024-04-25T00:35:07.815374+0000","last_clean":"2024-04-25T00:35:07.815374+0000","last_became_active":"2024-04-25T00:35:07.803758+0000","last_became_peered":"2024-04-25T00:35:07.803758+0000","last_unstale":"2024-04-25T00:35:07.815374+0000","last_undegraded":"2024-04-25T00:35:07.815374+0000","last_fullsized":"2024-04-25T00:35:07.815374+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:22.792165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.771357+0000","last_change":"2024-04-25T00:35:06.769240+0000","last_active":"2024-04-25T00:35:07.771357+0000","last_peered":"2024-04-25T00:35:07.771357+0000","last_clean":"2024-04-25T00:35:07.771357+0000","last_became_active":"2024-04-25T00:35:06.768597+0000","last_became_peered":"2024-04-25T00:35:06.768597+0000","last_unstale":"2024-04-25T00:35:07.771357+0000","last_undegraded":"2024-04-25T00:35:07.771357+0000","last_fullsized":"2024-04-25T00:35:07.771357+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:13.274709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822492+0000","last_change":"2024-04-25T00:35:07.813155+0000","last_active":"2024-04-25T00:35:07.822492+0000","last_peered":"2024-04-25T00:35:07.822492+0000","last_clean":"2024-04-25T00:35:07.822492+0000","last_became_active":"2024-04-25T00:35:07.811513+0000","last_became_peered":"2024-04-25T00:35:07.811513+0000","last_unstale":"2024-04-25T00:35:07.822492+0000","last_undegraded":"2024-04-25T00:35:07.822492+0000","last_fullsized":"2024-04-25T00:35:07.822492+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:16.590960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.773915+0000","last_change":"2024-04-25T00:35:06.772036+0000","last_active":"2024-04-25T00:35:07.773915+0000","last_peered":"2024-04-25T00:35:07.773915+0000","last_clean":"2024-04-25T00:35:07.773915+0000","last_became_active":"2024-04-25T00:35:06.771327+0000","last_became_peered":"2024-04-25T00:35:06.771327+0000","last_unstale":"2024-04-25T00:35:07.773915+0000","last_undegraded":"2024-04-25T00:35:07.773915+0000","last_fullsized":"2024-04-25T00:35:07.773915+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:00.087036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.812667+0000","last_change":"2024-04-25T00:35:07.802717+0000","last_active":"2024-04-25T00:35:07.812667+0000","last_peered":"2024-04-25T00:35:07.812667+0000","last_clean":"2024-04-25T00:35:07.812667+0000","last_became_active":"2024-04-25T00:35:07.801976+0000","last_became_peered":"2024-04-25T00:35:07.801976+0000","last_unstale":"2024-04-25T00:35:07.812667+0000","last_undegraded":"2024-04-25T00:35:07.812667+0000","last_fullsized":"2024-04-25T00:35:07.812667+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:06.584791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.835618+0000","last_change":"2024-04-25T00:35:06.762083+0000","last_active":"2024-04-25T00:35:07.835618+0000","last_peered":"2024-04-25T00:35:07.835618+0000","last_clean":"2024-04-25T00:35:07.835618+0000","last_became_active":"2024-04-25T00:35:06.761636+0000","last_became_peered":"2024-04-25T00:35:06.761636+0000","last_unstale":"2024-04-25T00:35:07.835618+0000","last_undegraded":"2024-04-25T00:35:07.835618+0000","last_fullsized":"2024-04-25T00:35:07.835618+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:53.694270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.800198+0000","last_change":"2024-04-25T00:35:07.788635+0000","last_active":"2024-04-25T00:35:07.800198+0000","last_peered":"2024-04-25T00:35:07.800198+0000","last_clean":"2024-04-25T00:35:07.800198+0000","last_became_active":"2024-04-25T00:35:07.787838+0000","last_became_peered":"2024-04-25T00:35:07.787838+0000","last_unstale":"2024-04-25T00:35:07.800198+0000","last_undegraded":"2024-04-25T00:35:07.800198+0000","last_fullsized":"2024-04-25T00:35:07.800198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.734173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.796015+0000","last_change":"2024-04-25T00:35:06.777786+0000","last_active":"2024-04-25T00:35:07.796015+0000","last_peered":"2024-04-25T00:35:07.796015+0000","last_clean":"2024-04-25T00:35:07.796015+0000","last_became_active":"2024-04-25T00:35:06.777358+0000","last_became_peered":"2024-04-25T00:35:06.777358+0000","last_unstale":"2024-04-25T00:35:07.796015+0000","last_undegraded":"2024-04-25T00:35:07.796015+0000","last_fullsized":"2024-04-25T00:35:07.796015+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:36.832974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.802603+0000","last_change":"2024-04-25T00:35:07.791535+0000","last_active":"2024-04-25T00:35:07.802603+0000","last_peered":"2024-04-25T00:35:07.802603+0000","last_clean":"2024-04-25T00:35:07.802603+0000","last_became_active":"2024-04-25T00:35:07.790779+0000","last_became_peered":"2024-04-25T00:35:07.790779+0000","last_unstale":"2024-04-25T00:35:07.802603+0000","last_undegraded":"2024-04-25T00:35:07.802603+0000","last_fullsized":"2024-04-25T00:35:07.802603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:02.293518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.781691+0000","last_change":"2024-04-25T00:35:06.771096+0000","last_active":"2024-04-25T00:35:07.781691+0000","last_peered":"2024-04-25T00:35:07.781691+0000","last_clean":"2024-04-25T00:35:07.781691+0000","last_became_active":"2024-04-25T00:35:06.770348+0000","last_became_peered":"2024-04-25T00:35:06.770348+0000","last_unstale":"2024-04-25T00:35:07.781691+0000","last_undegraded":"2024-04-25T00:35:07.781691+0000","last_fullsized":"2024-04-25T00:35:07.781691+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:24.200300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822733+0000","last_change":"2024-04-25T00:35:07.806367+0000","last_active":"2024-04-25T00:35:07.822733+0000","last_peered":"2024-04-25T00:35:07.822733+0000","last_clean":"2024-04-25T00:35:07.822733+0000","last_became_active":"2024-04-25T00:35:07.805335+0000","last_became_peered":"2024-04-25T00:35:07.805335+0000","last_unstale":"2024-04-25T00:35:07.822733+0000","last_undegraded":"2024-04-25T00:35:07.822733+0000","last_fullsized":"2024-04-25T00:35:07.822733+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:17.638499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.776587+0000","last_change":"2024-04-25T00:35:06.769923+0000","last_active":"2024-04-25T00:35:07.776587+0000","last_peered":"2024-04-25T00:35:07.776587+0000","last_clean":"2024-04-25T00:35:07.776587+0000","last_became_active":"2024-04-25T00:35:06.769156+0000","last_became_peered":"2024-04-25T00:35:06.769156+0000","last_unstale":"2024-04-25T00:35:07.776587+0000","last_undegraded":"2024-04-25T00:35:07.776587+0000","last_fullsized":"2024-04-25T00:35:07.776587+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:21.781654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824072+0000","last_change":"2024-04-25T00:35:07.809813+0000","last_active":"2024-04-25T00:35:07.824072+0000","last_peered":"2024-04-25T00:35:07.824072+0000","last_clean":"2024-04-25T00:35:07.824072+0000","last_became_active":"2024-04-25T00:35:07.807925+0000","last_became_peered":"2024-04-25T00:35:07.807925+0000","last_unstale":"2024-04-25T00:35:07.824072+0000","last_undegraded":"2024-04-25T00:35:07.824072+0000","last_fullsized":"2024-04-25T00:35:07.824072+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:37.901106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.771414+0000","last_change":"2024-04-25T00:35:06.762078+0000","last_active":"2024-04-25T00:35:07.771414+0000","last_peered":"2024-04-25T00:35:07.771414+0000","last_clean":"2024-04-25T00:35:07.771414+0000","last_became_active":"2024-04-25T00:35:06.761766+0000","last_became_peered":"2024-04-25T00:35:06.761766+0000","last_unstale":"2024-04-25T00:35:07.771414+0000","last_undegraded":"2024-04-25T00:35:07.771414+0000","last_fullsized":"2024-04-25T00:35:07.771414+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:06.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824548+0000","last_change":"2024-04-25T00:35:07.817773+0000","last_active":"2024-04-25T00:35:07.824548+0000","last_peered":"2024-04-25T00:35:07.824548+0000","last_clean":"2024-04-25T00:35:07.824548+0000","last_became_active":"2024-04-25T00:35:07.817007+0000","last_became_peered":"2024-04-25T00:35:07.817007+0000","last_unstale":"2024-04-25T00:35:07.824548+0000","last_undegraded":"2024-04-25T00:35:07.824548+0000","last_fullsized":"2024-04-25T00:35:07.824548+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:14.138865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.782599+0000","last_change":"2024-04-25T00:35:06.764548+0000","last_active":"2024-04-25T00:35:07.782599+0000","last_peered":"2024-04-25T00:35:07.782599+0000","last_clean":"2024-04-25T00:35:07.782599+0000","last_became_active":"2024-04-25T00:35:06.763862+0000","last_became_peered":"2024-04-25T00:35:06.763862+0000","last_unstale":"2024-04-25T00:35:07.782599+0000","last_undegraded":"2024-04-25T00:35:07.782599+0000","last_fullsized":"2024-04-25T00:35:07.782599+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:03.177703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.808190+0000","last_change":"2024-04-25T00:35:07.799638+0000","last_active":"2024-04-25T00:35:07.808190+0000","last_peered":"2024-04-25T00:35:07.808190+0000","last_clean":"2024-04-25T00:35:07.808190+0000","last_became_active":"2024-04-25T00:35:07.798881+0000","last_became_peered":"2024-04-25T00:35:07.798881+0000","last_unstale":"2024-04-25T00:35:07.808190+0000","last_undegraded":"2024-04-25T00:35:07.808190+0000","last_fullsized":"2024-04-25T00:35:07.808190+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:04.188213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.771308+0000","last_change":"2024-04-25T00:35:06.768474+0000","last_active":"2024-04-25T00:35:07.771308+0000","last_peered":"2024-04-25T00:35:07.771308+0000","last_clean":"2024-04-25T00:35:07.771308+0000","last_became_active":"2024-04-25T00:35:06.768123+0000","last_became_peered":"2024-04-25T00:35:06.768123+0000","last_unstale":"2024-04-25T00:35:07.771308+0000","last_undegraded":"2024-04-25T00:35:07.771308+0000","last_fullsized":"2024-04-25T00:35:07.771308+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:24.200300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822852+0000","last_change":"2024-04-25T00:35:07.812232+0000","last_active":"2024-04-25T00:35:07.822852+0000","last_peered":"2024-04-25T00:35:07.822852+0000","last_clean":"2024-04-25T00:35:07.822852+0000","last_became_active":"2024-04-25T00:35:07.811421+0000","last_became_peered":"2024-04-25T00:35:07.811421+0000","last_unstale":"2024-04-25T00:35:07.822852+0000","last_undegraded":"2024-04-25T00:35:07.822852+0000","last_fullsized":"2024-04-25T00:35:07.822852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:49.277843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.798256+0000","last_change":"2024-04-25T00:35:06.774317+0000","last_active":"2024-04-25T00:35:07.798256+0000","last_peered":"2024-04-25T00:35:07.798256+0000","last_clean":"2024-04-25T00:35:07.798256+0000","last_became_active":"2024-04-25T00:35:06.773806+0000","last_became_peered":"2024-04-25T00:35:06.773806+0000","last_unstale":"2024-04-25T00:35:07.798256+0000","last_undegraded":"2024-04-25T00:35:07.798256+0000","last_fullsized":"2024-04-25T00:35:07.798256+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:13.128354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822977+0000","last_change":"2024-04-25T00:35:07.814782+0000","last_active":"2024-04-25T00:35:07.822977+0000","last_peered":"2024-04-25T00:35:07.822977+0000","last_clean":"2024-04-25T00:35:07.822977+0000","last_became_active":"2024-04-25T00:35:07.813963+0000","last_became_peered":"2024-04-25T00:35:07.813963+0000","last_unstale":"2024-04-25T00:35:07.822977+0000","last_undegraded":"2024-04-25T00:35:07.822977+0000","last_fullsized":"2024-04-25T00:35:07.822977+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:25.408943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.782137+0000","last_change":"2024-04-25T00:35:06.765161+0000","last_active":"2024-04-25T00:35:07.782137+0000","last_peered":"2024-04-25T00:35:07.782137+0000","last_clean":"2024-04-25T00:35:07.782137+0000","last_became_active":"2024-04-25T00:35:06.764318+0000","last_became_peered":"2024-04-25T00:35:06.764318+0000","last_unstale":"2024-04-25T00:35:07.782137+0000","last_undegraded":"2024-04-25T00:35:07.782137+0000","last_fullsized":"2024-04-25T00:35:07.782137+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:06.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":240,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.791897+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:35:07.791897+0000","last_peered":"2024-04-25T00:35:07.791897+0000","last_clean":"2024-04-25T00:35:07.791897+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:35:07.791897+0000","last_undegraded":"2024-04-25T00:35:07.791897+0000","last_fullsized":"2024-04-25T00:35:07.791897+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":875041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":684032,"data_stored":284513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":684032,"data_stored":284513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":696320,"data_stored":286125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36016,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335824,"statfs":{"total":96636764160,"available":96599883776,"internally_reserved":0,"allocated":692224,"data_stored":284569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":284569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":688128,"data_stored":284603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36592,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335248,"statfs":{"total":96636764160,"available":96599293952,"internally_reserved":0,"allocated":1282048,"data_stored":875463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:35:11.854 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:35:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc5179f1640 1 Processor -- start 2024-04-25T00:35:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc5179f1640 1 -- start start 2024-04-25T00:35:12.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc5179f1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc510153030 0x7fc51006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc5179f1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5100cd200 0x7fc51006b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc5179f1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc51015d580 con 0x7fc51006b750 2024-04-25T00:35:12.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc5179f1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc51015d3d0 con 0x7fc510153030 2024-04-25T00:35:12.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc5179f1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc51015c3c0 con 0x7fc5100cd200 2024-04-25T00:35:12.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc514f8c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5100cd200 0x7fc51006b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc51578d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc510153030 0x7fc51006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc514f8c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5100cd200 0x7fc51006b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:37346/0 (socket says 172.21.15.29:37346) 2024-04-25T00:35:12.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc514f8c640 1 -- 172.21.15.29:0/2368120956 learned_addr learned my addr 172.21.15.29:0/2368120956 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:12.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc51578d640 1 -- 172.21.15.29:0/2368120956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5100cd200 msgr2=0x7fc51006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:12.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc51578d640 1 --2- 172.21.15.29:0/2368120956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5100cd200 0x7fc51006b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc51578d640 1 -- 172.21.15.29:0/2368120956 >> v1:172.21.15.157:6789/0 conn(0x7fc51006b750 legacy=0x7fc5100b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc51578d640 1 -- 172.21.15.29:0/2368120956 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51015c610 con 0x7fc510153030 2024-04-25T00:35:12.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc507fff640 1 -- 172.21.15.29:0/2368120956 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2472269995 0 0) 0x7fc51015d580 con 0x7fc51006b750 2024-04-25T00:35:12.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.017+0000 7fc514f8c640 1 --2- 172.21.15.29:0/2368120956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5100cd200 0x7fc51006b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:12.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc51578d640 1 --2- 172.21.15.29:0/2368120956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc510153030 0x7fc51006fe90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0097c0 tx=0x7fc50c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6542d074f5fdd0ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:12.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc507fff640 1 -- 172.21.15.29:0/2368120956 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0b0070 con 0x7fc510153030 2024-04-25T00:35:12.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc507fff640 1 -- 172.21.15.29:0/2368120956 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc50c0ab600 con 0x7fc510153030 2024-04-25T00:35:12.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc507fff640 1 -- 172.21.15.29:0/2368120956 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0ab9d0 con 0x7fc510153030 2024-04-25T00:35:12.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc5179f1640 1 -- 172.21.15.29:0/2368120956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc510153030 msgr2=0x7fc51006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc5179f1640 1 --2- 172.21.15.29:0/2368120956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc510153030 0x7fc51006fe90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0097c0 tx=0x7fc50c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc5179f1640 1 -- 172.21.15.29:0/2368120956 shutdown_connections 2024-04-25T00:35:12.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc5179f1640 1 --2- 172.21.15.29:0/2368120956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5100cd200 0x7fc51006b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc5179f1640 1 --2- 172.21.15.29:0/2368120956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc510153030 0x7fc51006fe90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc5179f1640 1 -- 172.21.15.29:0/2368120956 >> 172.21.15.29:0/2368120956 conn(0x7fc5100d0c10 msgr2=0x7fc5100d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:12.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc5179f1640 1 -- 172.21.15.29:0/2368120956 shutdown_connections 2024-04-25T00:35:12.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc5179f1640 1 -- 172.21.15.29:0/2368120956 wait complete. 2024-04-25T00:35:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc5179f1640 1 Processor -- start 2024-04-25T00:35:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.021+0000 7fc5179f1640 1 -- start start 2024-04-25T00:35:12.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc5179f1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51006b750 0x7fc510166ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc5179f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5100cd200 0x7fc5100c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc5179f1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc510153030 0x7fc5101f9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc5179f1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc510070550 con 0x7fc51006b750 2024-04-25T00:35:12.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc5179f1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5100703d0 con 0x7fc5100cd200 2024-04-25T00:35:12.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc5179f1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc510071d80 con 0x7fc510153030 2024-04-25T00:35:12.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc51578d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51006b750 0x7fc510166ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc51578d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51006b750 0x7fc510166ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59182/0 (socket says 172.21.15.29:59182) 2024-04-25T00:35:12.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc51578d640 1 -- 172.21.15.29:0/953438632 learned_addr learned my addr 172.21.15.29:0/953438632 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:12.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc515f8e640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc510153030 0x7fc5101f9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc514f8c640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5100cd200 0x7fc5100c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc51578d640 1 -- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc510153030 msgr2=0x7fc5101f9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc51578d640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc510153030 0x7fc5101f9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc51578d640 1 -- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5100cd200 msgr2=0x7fc5100c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc51578d640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5100cd200 0x7fc5100c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc51578d640 1 -- 172.21.15.29:0/953438632 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5101fa1a0 con 0x7fc51006b750 2024-04-25T00:35:12.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc51578d640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51006b750 0x7fc510166ea0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0ae040 tx=0x7fc50c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:12.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc505ffb640 1 -- 172.21.15.29:0/953438632 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0bc070 con 0x7fc51006b750 2024-04-25T00:35:12.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc505ffb640 1 -- 172.21.15.29:0/953438632 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc50c0b1040 con 0x7fc51006b750 2024-04-25T00:35:12.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc5179f1640 1 -- 172.21.15.29:0/953438632 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc510167530 con 0x7fc51006b750 2024-04-25T00:35:12.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc505ffb640 1 -- 172.21.15.29:0/953438632 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0b0050 con 0x7fc51006b750 2024-04-25T00:35:12.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc5179f1640 1 -- 172.21.15.29:0/953438632 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5100ceb70 con 0x7fc51006b750 2024-04-25T00:35:12.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc5179f1640 1 -- 172.21.15.29:0/953438632 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc510069270 con 0x7fc51006b750 2024-04-25T00:35:12.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc505ffb640 1 -- 172.21.15.29:0/953438632 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fc50c0bf070 con 0x7fc51006b750 2024-04-25T00:35:12.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc505ffb640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc4f4067a70 0x7fc4f4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc505ffb640 1 -- 172.21.15.29:0/953438632 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc50c11fb10 con 0x7fc51006b750 2024-04-25T00:35:12.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.025+0000 7fc514f8c640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc4f4067a70 0x7fc4f4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.029+0000 7fc514f8c640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc4f4067a70 0x7fc4f4069f30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc500009630 tx=0x7fc500008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:12.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.029+0000 7fc505ffb640 1 -- 172.21.15.29:0/953438632 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc50c016090 con 0x7fc51006b750 2024-04-25T00:35:12.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.185+0000 7fc5179f1640 1 -- 172.21.15.29:0/953438632 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc51015bb90 con 0x7fc4f4067a70 2024-04-25T00:35:12.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.201+0000 7fc505ffb640 1 -- 172.21.15.29:0/953438632 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251889 (secure 0 0 0) 0x7fc51015bb90 con 0x7fc4f4067a70 2024-04-25T00:35:12.206 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:12.207 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:35:12.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.205+0000 7fc5179f1640 1 -- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc4f4067a70 msgr2=0x7fc4f4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.205+0000 7fc5179f1640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc4f4067a70 0x7fc4f4069f30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc500009630 tx=0x7fc500008040 comp rx=0 tx=0).stop 2024-04-25T00:35:12.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.205+0000 7fc5179f1640 1 -- 172.21.15.29:0/953438632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51006b750 msgr2=0x7fc510166ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.205+0000 7fc5179f1640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51006b750 0x7fc510166ea0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0ae040 tx=0x7fc50c01ace0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.205+0000 7fc5179f1640 1 -- 172.21.15.29:0/953438632 shutdown_connections 2024-04-25T00:35:12.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.205+0000 7fc5179f1640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc510153030 0x7fc5101f9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.205+0000 7fc5179f1640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc4f4067a70 0x7fc4f4069f30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.205+0000 7fc5179f1640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5100cd200 0x7fc5100c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.205+0000 7fc5179f1640 1 --2- 172.21.15.29:0/953438632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc51006b750 0x7fc510166ea0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.205+0000 7fc5179f1640 1 -- 172.21.15.29:0/953438632 >> 172.21.15.29:0/953438632 conn(0x7fc5100d0c10 msgr2=0x7fc51015b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:12.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.205+0000 7fc5179f1640 1 -- 172.21.15.29:0/953438632 shutdown_connections 2024-04-25T00:35:12.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.205+0000 7fc5179f1640 1 -- 172.21.15.29:0/953438632 wait complete. 2024-04-25T00:35:12.223 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-04-25T00:35:11.522201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268468,"kb_used_data":6548,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754706252,"statfs":{"total":773094113280,"available":772819202048,"internally_reserved":0,"allocated":6705152,"data_stored":3459396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.789282"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.823842+0000","last_change":"2024-04-25T00:35:07.822239+0000","last_active":"2024-04-25T00:35:07.823842+0000","last_peered":"2024-04-25T00:35:07.823842+0000","last_clean":"2024-04-25T00:35:07.823842+0000","last_became_active":"2024-04-25T00:35:07.821746+0000","last_became_peered":"2024-04-25T00:35:07.821746+0000","last_unstale":"2024-04-25T00:35:07.823842+0000","last_undegraded":"2024-04-25T00:35:07.823842+0000","last_fullsized":"2024-04-25T00:35:07.823842+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:07.309543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822832+0000","last_change":"2024-04-25T00:35:07.809147+0000","last_active":"2024-04-25T00:35:07.822832+0000","last_peered":"2024-04-25T00:35:07.822832+0000","last_clean":"2024-04-25T00:35:07.822832+0000","last_became_active":"2024-04-25T00:35:07.804819+0000","last_became_peered":"2024-04-25T00:35:07.804819+0000","last_unstale":"2024-04-25T00:35:07.822832+0000","last_undegraded":"2024-04-25T00:35:07.822832+0000","last_fullsized":"2024-04-25T00:35:07.822832+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:09.219797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.825218+0000","last_change":"2024-04-25T00:35:07.824004+0000","last_active":"2024-04-25T00:35:07.825218+0000","last_peered":"2024-04-25T00:35:07.825218+0000","last_clean":"2024-04-25T00:35:07.825218+0000","last_became_active":"2024-04-25T00:35:07.822761+0000","last_became_peered":"2024-04-25T00:35:07.822761+0000","last_unstale":"2024-04-25T00:35:07.825218+0000","last_undegraded":"2024-04-25T00:35:07.825218+0000","last_fullsized":"2024-04-25T00:35:07.825218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:28.555554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.815188+0000","last_change":"2024-04-25T00:35:07.804678+0000","last_active":"2024-04-25T00:35:07.815188+0000","last_peered":"2024-04-25T00:35:07.815188+0000","last_clean":"2024-04-25T00:35:07.815188+0000","last_became_active":"2024-04-25T00:35:07.803986+0000","last_became_peered":"2024-04-25T00:35:07.803986+0000","last_unstale":"2024-04-25T00:35:07.815188+0000","last_undegraded":"2024-04-25T00:35:07.815188+0000","last_fullsized":"2024-04-25T00:35:07.815188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:41.263988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.821054+0000","last_change":"2024-04-25T00:35:07.803137+0000","last_active":"2024-04-25T00:35:07.821054+0000","last_peered":"2024-04-25T00:35:07.821054+0000","last_clean":"2024-04-25T00:35:07.821054+0000","last_became_active":"2024-04-25T00:35:07.801969+0000","last_became_peered":"2024-04-25T00:35:07.801969+0000","last_unstale":"2024-04-25T00:35:07.821054+0000","last_undegraded":"2024-04-25T00:35:07.821054+0000","last_fullsized":"2024-04-25T00:35:07.821054+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:54.231727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822027+0000","last_change":"2024-04-25T00:35:07.810933+0000","last_active":"2024-04-25T00:35:07.822027+0000","last_peered":"2024-04-25T00:35:07.822027+0000","last_clean":"2024-04-25T00:35:07.822027+0000","last_became_active":"2024-04-25T00:35:07.796146+0000","last_became_peered":"2024-04-25T00:35:07.796146+0000","last_unstale":"2024-04-25T00:35:07.822027+0000","last_undegraded":"2024-04-25T00:35:07.822027+0000","last_fullsized":"2024-04-25T00:35:07.822027+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.734173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.795189+0000","last_change":"2024-04-25T00:35:07.788292+0000","last_active":"2024-04-25T00:35:07.795189+0000","last_peered":"2024-04-25T00:35:07.795189+0000","last_clean":"2024-04-25T00:35:07.795189+0000","last_became_active":"2024-04-25T00:35:07.787495+0000","last_became_peered":"2024-04-25T00:35:07.787495+0000","last_unstale":"2024-04-25T00:35:07.795189+0000","last_undegraded":"2024-04-25T00:35:07.795189+0000","last_fullsized":"2024-04-25T00:35:07.795189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:35.987615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822267+0000","last_change":"2024-04-25T00:35:07.804218+0000","last_active":"2024-04-25T00:35:07.822267+0000","last_peered":"2024-04-25T00:35:07.822267+0000","last_clean":"2024-04-25T00:35:07.822267+0000","last_became_active":"2024-04-25T00:35:07.803430+0000","last_became_peered":"2024-04-25T00:35:07.803430+0000","last_unstale":"2024-04-25T00:35:07.822267+0000","last_undegraded":"2024-04-25T00:35:07.822267+0000","last_fullsized":"2024-04-25T00:35:07.822267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:47.139485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.814680+0000","last_change":"2024-04-25T00:35:07.804978+0000","last_active":"2024-04-25T00:35:07.814680+0000","last_peered":"2024-04-25T00:35:07.814680+0000","last_clean":"2024-04-25T00:35:07.814680+0000","last_became_active":"2024-04-25T00:35:07.804265+0000","last_became_peered":"2024-04-25T00:35:07.804265+0000","last_unstale":"2024-04-25T00:35:07.814680+0000","last_undegraded":"2024-04-25T00:35:07.814680+0000","last_fullsized":"2024-04-25T00:35:07.814680+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:02.770380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.800415+0000","last_change":"2024-04-25T00:35:07.794455+0000","last_active":"2024-04-25T00:35:07.800415+0000","last_peered":"2024-04-25T00:35:07.800415+0000","last_clean":"2024-04-25T00:35:07.800415+0000","last_became_active":"2024-04-25T00:35:07.793690+0000","last_became_peered":"2024-04-25T00:35:07.793690+0000","last_unstale":"2024-04-25T00:35:07.800415+0000","last_undegraded":"2024-04-25T00:35:07.800415+0000","last_fullsized":"2024-04-25T00:35:07.800415+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:23.889772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.818055+0000","last_change":"2024-04-25T00:35:07.813286+0000","last_active":"2024-04-25T00:35:07.818055+0000","last_peered":"2024-04-25T00:35:07.818055+0000","last_clean":"2024-04-25T00:35:07.818055+0000","last_became_active":"2024-04-25T00:35:07.812547+0000","last_became_peered":"2024-04-25T00:35:07.812547+0000","last_unstale":"2024-04-25T00:35:07.818055+0000","last_undegraded":"2024-04-25T00:35:07.818055+0000","last_fullsized":"2024-04-25T00:35:07.818055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:35.987615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.793287+0000","last_change":"2024-04-25T00:35:07.787035+0000","last_active":"2024-04-25T00:35:07.793287+0000","last_peered":"2024-04-25T00:35:07.793287+0000","last_clean":"2024-04-25T00:35:07.793287+0000","last_became_active":"2024-04-25T00:35:07.786332+0000","last_became_peered":"2024-04-25T00:35:07.786332+0000","last_unstale":"2024-04-25T00:35:07.793287+0000","last_undegraded":"2024-04-25T00:35:07.793287+0000","last_fullsized":"2024-04-25T00:35:07.793287+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:34.426589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.810050+0000","last_change":"2024-04-25T00:35:07.807593+0000","last_active":"2024-04-25T00:35:07.810050+0000","last_peered":"2024-04-25T00:35:07.810050+0000","last_clean":"2024-04-25T00:35:07.810050+0000","last_became_active":"2024-04-25T00:35:07.806877+0000","last_became_peered":"2024-04-25T00:35:07.806877+0000","last_unstale":"2024-04-25T00:35:07.810050+0000","last_undegraded":"2024-04-25T00:35:07.810050+0000","last_fullsized":"2024-04-25T00:35:07.810050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:23.690728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.800571+0000","last_change":"2024-04-25T00:35:07.798753+0000","last_active":"2024-04-25T00:35:07.800571+0000","last_peered":"2024-04-25T00:35:07.800571+0000","last_clean":"2024-04-25T00:35:07.800571+0000","last_became_active":"2024-04-25T00:35:07.797765+0000","last_became_peered":"2024-04-25T00:35:07.797765+0000","last_unstale":"2024-04-25T00:35:07.800571+0000","last_undegraded":"2024-04-25T00:35:07.800571+0000","last_fullsized":"2024-04-25T00:35:07.800571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:35.412434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.798543+0000","last_change":"2024-04-25T00:35:07.794567+0000","last_active":"2024-04-25T00:35:07.798543+0000","last_peered":"2024-04-25T00:35:07.798543+0000","last_clean":"2024-04-25T00:35:07.798543+0000","last_became_active":"2024-04-25T00:35:07.793650+0000","last_became_peered":"2024-04-25T00:35:07.793650+0000","last_unstale":"2024-04-25T00:35:07.798543+0000","last_undegraded":"2024-04-25T00:35:07.798543+0000","last_fullsized":"2024-04-25T00:35:07.798543+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:32.706100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.825339+0000","last_change":"2024-04-25T00:35:07.806476+0000","last_active":"2024-04-25T00:35:07.825339+0000","last_peered":"2024-04-25T00:35:07.825339+0000","last_clean":"2024-04-25T00:35:07.825339+0000","last_became_active":"2024-04-25T00:35:07.805284+0000","last_became_peered":"2024-04-25T00:35:07.805284+0000","last_unstale":"2024-04-25T00:35:07.825339+0000","last_undegraded":"2024-04-25T00:35:07.825339+0000","last_fullsized":"2024-04-25T00:35:07.825339+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:34.426589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.823713+0000","last_change":"2024-04-25T00:35:07.809793+0000","last_active":"2024-04-25T00:35:07.823713+0000","last_peered":"2024-04-25T00:35:07.823713+0000","last_clean":"2024-04-25T00:35:07.823713+0000","last_became_active":"2024-04-25T00:35:07.809006+0000","last_became_peered":"2024-04-25T00:35:07.809006+0000","last_unstale":"2024-04-25T00:35:07.823713+0000","last_undegraded":"2024-04-25T00:35:07.823713+0000","last_fullsized":"2024-04-25T00:35:07.823713+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:13.114459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824636+0000","last_change":"2024-04-25T00:35:07.818709+0000","last_active":"2024-04-25T00:35:07.824636+0000","last_peered":"2024-04-25T00:35:07.824636+0000","last_clean":"2024-04-25T00:35:07.824636+0000","last_became_active":"2024-04-25T00:35:07.817911+0000","last_became_peered":"2024-04-25T00:35:07.817911+0000","last_unstale":"2024-04-25T00:35:07.824636+0000","last_undegraded":"2024-04-25T00:35:07.824636+0000","last_fullsized":"2024-04-25T00:35:07.824636+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:22.804662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822489+0000","last_change":"2024-04-25T00:35:07.820860+0000","last_active":"2024-04-25T00:35:07.822489+0000","last_peered":"2024-04-25T00:35:07.822489+0000","last_clean":"2024-04-25T00:35:07.822489+0000","last_became_active":"2024-04-25T00:35:07.820496+0000","last_became_peered":"2024-04-25T00:35:07.820496+0000","last_unstale":"2024-04-25T00:35:07.822489+0000","last_undegraded":"2024-04-25T00:35:07.822489+0000","last_fullsized":"2024-04-25T00:35:07.822489+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:51.679806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822883+0000","last_change":"2024-04-25T00:35:07.809234+0000","last_active":"2024-04-25T00:35:07.822883+0000","last_peered":"2024-04-25T00:35:07.822883+0000","last_clean":"2024-04-25T00:35:07.822883+0000","last_became_active":"2024-04-25T00:35:07.808481+0000","last_became_peered":"2024-04-25T00:35:07.808481+0000","last_unstale":"2024-04-25T00:35:07.822883+0000","last_undegraded":"2024-04-25T00:35:07.822883+0000","last_fullsized":"2024-04-25T00:35:07.822883+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:44.325402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.812488+0000","last_change":"2024-04-25T00:35:07.809751+0000","last_active":"2024-04-25T00:35:07.812488+0000","last_peered":"2024-04-25T00:35:07.812488+0000","last_clean":"2024-04-25T00:35:07.812488+0000","last_became_active":"2024-04-25T00:35:07.806831+0000","last_became_peered":"2024-04-25T00:35:07.806831+0000","last_unstale":"2024-04-25T00:35:07.812488+0000","last_undegraded":"2024-04-25T00:35:07.812488+0000","last_fullsized":"2024-04-25T00:35:07.812488+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:16.351363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.827253+0000","last_change":"2024-04-25T00:35:07.825780+0000","last_active":"2024-04-25T00:35:07.827253+0000","last_peered":"2024-04-25T00:35:07.827253+0000","last_clean":"2024-04-25T00:35:07.827253+0000","last_became_active":"2024-04-25T00:35:07.824515+0000","last_became_peered":"2024-04-25T00:35:07.824515+0000","last_unstale":"2024-04-25T00:35:07.827253+0000","last_undegraded":"2024-04-25T00:35:07.827253+0000","last_fullsized":"2024-04-25T00:35:07.827253+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:20.156676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824441+0000","last_change":"2024-04-25T00:35:07.817376+0000","last_active":"2024-04-25T00:35:07.824441+0000","last_peered":"2024-04-25T00:35:07.824441+0000","last_clean":"2024-04-25T00:35:07.824441+0000","last_became_active":"2024-04-25T00:35:07.815097+0000","last_became_peered":"2024-04-25T00:35:07.815097+0000","last_unstale":"2024-04-25T00:35:07.824441+0000","last_undegraded":"2024-04-25T00:35:07.824441+0000","last_fullsized":"2024-04-25T00:35:07.824441+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:41.263988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.816741+0000","last_change":"2024-04-25T00:35:07.811010+0000","last_active":"2024-04-25T00:35:07.816741+0000","last_peered":"2024-04-25T00:35:07.816741+0000","last_clean":"2024-04-25T00:35:07.816741+0000","last_became_active":"2024-04-25T00:35:07.809910+0000","last_became_peered":"2024-04-25T00:35:07.809910+0000","last_unstale":"2024-04-25T00:35:07.816741+0000","last_undegraded":"2024-04-25T00:35:07.816741+0000","last_fullsized":"2024-04-25T00:35:07.816741+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:02.293518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.820802+0000","last_change":"2024-04-25T00:35:07.802933+0000","last_active":"2024-04-25T00:35:07.820802+0000","last_peered":"2024-04-25T00:35:07.820802+0000","last_clean":"2024-04-25T00:35:07.820802+0000","last_became_active":"2024-04-25T00:35:07.801022+0000","last_became_peered":"2024-04-25T00:35:07.801022+0000","last_unstale":"2024-04-25T00:35:07.820802+0000","last_undegraded":"2024-04-25T00:35:07.820802+0000","last_fullsized":"2024-04-25T00:35:07.820802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:47.597016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.794498+0000","last_change":"2024-04-25T00:35:07.791035+0000","last_active":"2024-04-25T00:35:07.794498+0000","last_peered":"2024-04-25T00:35:07.794498+0000","last_clean":"2024-04-25T00:35:07.794498+0000","last_became_active":"2024-04-25T00:35:07.790301+0000","last_became_peered":"2024-04-25T00:35:07.790301+0000","last_unstale":"2024-04-25T00:35:07.794498+0000","last_undegraded":"2024-04-25T00:35:07.794498+0000","last_fullsized":"2024-04-25T00:35:07.794498+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:44.227824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822778+0000","last_change":"2024-04-25T00:35:07.812129+0000","last_active":"2024-04-25T00:35:07.822778+0000","last_peered":"2024-04-25T00:35:07.822778+0000","last_clean":"2024-04-25T00:35:07.822778+0000","last_became_active":"2024-04-25T00:35:07.806228+0000","last_became_peered":"2024-04-25T00:35:07.806228+0000","last_unstale":"2024-04-25T00:35:07.822778+0000","last_undegraded":"2024-04-25T00:35:07.822778+0000","last_fullsized":"2024-04-25T00:35:07.822778+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:44.227824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.821098+0000","last_change":"2024-04-25T00:35:07.793709+0000","last_active":"2024-04-25T00:35:07.821098+0000","last_peered":"2024-04-25T00:35:07.821098+0000","last_clean":"2024-04-25T00:35:07.821098+0000","last_became_active":"2024-04-25T00:35:07.793060+0000","last_became_peered":"2024-04-25T00:35:07.793060+0000","last_unstale":"2024-04-25T00:35:07.821098+0000","last_undegraded":"2024-04-25T00:35:07.821098+0000","last_fullsized":"2024-04-25T00:35:07.821098+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:24.934142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824073+0000","last_change":"2024-04-25T00:35:07.819244+0000","last_active":"2024-04-25T00:35:07.824073+0000","last_peered":"2024-04-25T00:35:07.824073+0000","last_clean":"2024-04-25T00:35:07.824073+0000","last_became_active":"2024-04-25T00:35:07.818420+0000","last_became_peered":"2024-04-25T00:35:07.818420+0000","last_unstale":"2024-04-25T00:35:07.824073+0000","last_undegraded":"2024-04-25T00:35:07.824073+0000","last_fullsized":"2024-04-25T00:35:07.824073+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:43.644264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.811021+0000","last_change":"2024-04-25T00:35:07.808596+0000","last_active":"2024-04-25T00:35:07.811021+0000","last_peered":"2024-04-25T00:35:07.811021+0000","last_clean":"2024-04-25T00:35:07.811021+0000","last_became_active":"2024-04-25T00:35:07.806801+0000","last_became_peered":"2024-04-25T00:35:07.806801+0000","last_unstale":"2024-04-25T00:35:07.811021+0000","last_undegraded":"2024-04-25T00:35:07.811021+0000","last_fullsized":"2024-04-25T00:35:07.811021+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:16.590960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.820386+0000","last_change":"2024-04-25T00:35:07.794804+0000","last_active":"2024-04-25T00:35:07.820386+0000","last_peered":"2024-04-25T00:35:07.820386+0000","last_clean":"2024-04-25T00:35:07.820386+0000","last_became_active":"2024-04-25T00:35:07.793931+0000","last_became_peered":"2024-04-25T00:35:07.793931+0000","last_unstale":"2024-04-25T00:35:07.820386+0000","last_undegraded":"2024-04-25T00:35:07.820386+0000","last_fullsized":"2024-04-25T00:35:07.820386+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:50.335647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.825761+0000","last_change":"2024-04-25T00:35:07.824155+0000","last_active":"2024-04-25T00:35:07.825761+0000","last_peered":"2024-04-25T00:35:07.825761+0000","last_clean":"2024-04-25T00:35:07.825761+0000","last_became_active":"2024-04-25T00:35:07.823593+0000","last_became_peered":"2024-04-25T00:35:07.823593+0000","last_unstale":"2024-04-25T00:35:07.825761+0000","last_undegraded":"2024-04-25T00:35:07.825761+0000","last_fullsized":"2024-04-25T00:35:07.825761+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:06.584791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822639+0000","last_change":"2024-04-25T00:35:07.812833+0000","last_active":"2024-04-25T00:35:07.822639+0000","last_peered":"2024-04-25T00:35:07.822639+0000","last_clean":"2024-04-25T00:35:07.822639+0000","last_became_active":"2024-04-25T00:35:07.812060+0000","last_became_peered":"2024-04-25T00:35:07.812060+0000","last_unstale":"2024-04-25T00:35:07.822639+0000","last_undegraded":"2024-04-25T00:35:07.822639+0000","last_fullsized":"2024-04-25T00:35:07.822639+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:16.351363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.836837+0000","last_change":"2024-04-25T00:35:06.765335+0000","last_active":"2024-04-25T00:35:07.836837+0000","last_peered":"2024-04-25T00:35:07.836837+0000","last_clean":"2024-04-25T00:35:07.836837+0000","last_became_active":"2024-04-25T00:35:06.764727+0000","last_became_peered":"2024-04-25T00:35:06.764727+0000","last_unstale":"2024-04-25T00:35:07.836837+0000","last_undegraded":"2024-04-25T00:35:07.836837+0000","last_fullsized":"2024-04-25T00:35:07.836837+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:53.228452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.798497+0000","last_change":"2024-04-25T00:35:07.789834+0000","last_active":"2024-04-25T00:35:07.798497+0000","last_peered":"2024-04-25T00:35:07.798497+0000","last_clean":"2024-04-25T00:35:07.798497+0000","last_became_active":"2024-04-25T00:35:07.787542+0000","last_became_peered":"2024-04-25T00:35:07.787542+0000","last_unstale":"2024-04-25T00:35:07.798497+0000","last_undegraded":"2024-04-25T00:35:07.798497+0000","last_fullsized":"2024-04-25T00:35:07.798497+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:23.690728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.834281+0000","last_change":"2024-04-25T00:35:06.768866+0000","last_active":"2024-04-25T00:35:07.834281+0000","last_peered":"2024-04-25T00:35:07.834281+0000","last_clean":"2024-04-25T00:35:07.834281+0000","last_became_active":"2024-04-25T00:35:06.768452+0000","last_became_peered":"2024-04-25T00:35:06.768452+0000","last_unstale":"2024-04-25T00:35:07.834281+0000","last_undegraded":"2024-04-25T00:35:07.834281+0000","last_fullsized":"2024-04-25T00:35:07.834281+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:38.952798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.815281+0000","last_change":"2024-04-25T00:35:07.797375+0000","last_active":"2024-04-25T00:35:07.815281+0000","last_peered":"2024-04-25T00:35:07.815281+0000","last_clean":"2024-04-25T00:35:07.815281+0000","last_became_active":"2024-04-25T00:35:07.796362+0000","last_became_peered":"2024-04-25T00:35:07.796362+0000","last_unstale":"2024-04-25T00:35:07.815281+0000","last_undegraded":"2024-04-25T00:35:07.815281+0000","last_fullsized":"2024-04-25T00:35:07.815281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:48.598127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.835858+0000","last_change":"2024-04-25T00:35:06.772078+0000","last_active":"2024-04-25T00:35:07.835858+0000","last_peered":"2024-04-25T00:35:07.835858+0000","last_clean":"2024-04-25T00:35:07.835858+0000","last_became_active":"2024-04-25T00:35:06.767244+0000","last_became_peered":"2024-04-25T00:35:06.767244+0000","last_unstale":"2024-04-25T00:35:07.835858+0000","last_undegraded":"2024-04-25T00:35:07.835858+0000","last_fullsized":"2024-04-25T00:35:07.835858+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:16.049571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822822+0000","last_change":"2024-04-25T00:35:07.812414+0000","last_active":"2024-04-25T00:35:07.822822+0000","last_peered":"2024-04-25T00:35:07.822822+0000","last_clean":"2024-04-25T00:35:07.822822+0000","last_became_active":"2024-04-25T00:35:07.805561+0000","last_became_peered":"2024-04-25T00:35:07.805561+0000","last_unstale":"2024-04-25T00:35:07.822822+0000","last_undegraded":"2024-04-25T00:35:07.822822+0000","last_fullsized":"2024-04-25T00:35:07.822822+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:33.255653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.841549+0000","last_change":"2024-04-25T00:35:06.776301+0000","last_active":"2024-04-25T00:35:07.841549+0000","last_peered":"2024-04-25T00:35:07.841549+0000","last_clean":"2024-04-25T00:35:07.841549+0000","last_became_active":"2024-04-25T00:35:06.773770+0000","last_became_peered":"2024-04-25T00:35:06.773770+0000","last_unstale":"2024-04-25T00:35:07.841549+0000","last_undegraded":"2024-04-25T00:35:07.841549+0000","last_fullsized":"2024-04-25T00:35:07.841549+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:47.587617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.827198+0000","last_change":"2024-04-25T00:35:07.825128+0000","last_active":"2024-04-25T00:35:07.827198+0000","last_peered":"2024-04-25T00:35:07.827198+0000","last_clean":"2024-04-25T00:35:07.827198+0000","last_became_active":"2024-04-25T00:35:07.823608+0000","last_became_peered":"2024-04-25T00:35:07.823608+0000","last_unstale":"2024-04-25T00:35:07.827198+0000","last_undegraded":"2024-04-25T00:35:07.827198+0000","last_fullsized":"2024-04-25T00:35:07.827198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:56.521301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.837008+0000","last_change":"2024-04-25T00:35:06.773548+0000","last_active":"2024-04-25T00:35:07.837008+0000","last_peered":"2024-04-25T00:35:07.837008+0000","last_clean":"2024-04-25T00:35:07.837008+0000","last_became_active":"2024-04-25T00:35:06.772497+0000","last_became_peered":"2024-04-25T00:35:06.772497+0000","last_unstale":"2024-04-25T00:35:07.837008+0000","last_undegraded":"2024-04-25T00:35:07.837008+0000","last_fullsized":"2024-04-25T00:35:07.837008+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:22.879261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824041+0000","last_change":"2024-04-25T00:35:07.815053+0000","last_active":"2024-04-25T00:35:07.824041+0000","last_peered":"2024-04-25T00:35:07.824041+0000","last_clean":"2024-04-25T00:35:07.824041+0000","last_became_active":"2024-04-25T00:35:07.814314+0000","last_became_peered":"2024-04-25T00:35:07.814314+0000","last_unstale":"2024-04-25T00:35:07.824041+0000","last_undegraded":"2024-04-25T00:35:07.824041+0000","last_fullsized":"2024-04-25T00:35:07.824041+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:09.827741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.836595+0000","last_change":"2024-04-25T00:35:06.778251+0000","last_active":"2024-04-25T00:35:07.836595+0000","last_peered":"2024-04-25T00:35:07.836595+0000","last_clean":"2024-04-25T00:35:07.836595+0000","last_became_active":"2024-04-25T00:35:06.777809+0000","last_became_peered":"2024-04-25T00:35:06.777809+0000","last_unstale":"2024-04-25T00:35:07.836595+0000","last_undegraded":"2024-04-25T00:35:07.836595+0000","last_fullsized":"2024-04-25T00:35:07.836595+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:41.884020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.809317+0000","last_change":"2024-04-25T00:35:07.806157+0000","last_active":"2024-04-25T00:35:07.809317+0000","last_peered":"2024-04-25T00:35:07.809317+0000","last_clean":"2024-04-25T00:35:07.809317+0000","last_became_active":"2024-04-25T00:35:07.805431+0000","last_became_peered":"2024-04-25T00:35:07.805431+0000","last_unstale":"2024-04-25T00:35:07.809317+0000","last_undegraded":"2024-04-25T00:35:07.809317+0000","last_fullsized":"2024-04-25T00:35:07.809317+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:54.704781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.774573+0000","last_change":"2024-04-25T00:35:06.770549+0000","last_active":"2024-04-25T00:35:07.774573+0000","last_peered":"2024-04-25T00:35:07.774573+0000","last_clean":"2024-04-25T00:35:07.774573+0000","last_became_active":"2024-04-25T00:35:06.769750+0000","last_became_peered":"2024-04-25T00:35:06.769750+0000","last_unstale":"2024-04-25T00:35:07.774573+0000","last_undegraded":"2024-04-25T00:35:07.774573+0000","last_fullsized":"2024-04-25T00:35:07.774573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:36.832974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822822+0000","last_change":"2024-04-25T00:35:07.810762+0000","last_active":"2024-04-25T00:35:07.822822+0000","last_peered":"2024-04-25T00:35:07.822822+0000","last_clean":"2024-04-25T00:35:07.822822+0000","last_became_active":"2024-04-25T00:35:07.809888+0000","last_became_peered":"2024-04-25T00:35:07.809888+0000","last_unstale":"2024-04-25T00:35:07.822822+0000","last_undegraded":"2024-04-25T00:35:07.822822+0000","last_fullsized":"2024-04-25T00:35:07.822822+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:40.364656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.781643+0000","last_change":"2024-04-25T00:35:06.773082+0000","last_active":"2024-04-25T00:35:07.781643+0000","last_peered":"2024-04-25T00:35:07.781643+0000","last_clean":"2024-04-25T00:35:07.781643+0000","last_became_active":"2024-04-25T00:35:06.771601+0000","last_became_peered":"2024-04-25T00:35:06.771601+0000","last_unstale":"2024-04-25T00:35:07.781643+0000","last_undegraded":"2024-04-25T00:35:07.781643+0000","last_fullsized":"2024-04-25T00:35:07.781643+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:01.283008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.816657+0000","last_change":"2024-04-25T00:35:07.812042+0000","last_active":"2024-04-25T00:35:07.816657+0000","last_peered":"2024-04-25T00:35:07.816657+0000","last_clean":"2024-04-25T00:35:07.816657+0000","last_became_active":"2024-04-25T00:35:07.811262+0000","last_became_peered":"2024-04-25T00:35:07.811262+0000","last_unstale":"2024-04-25T00:35:07.816657+0000","last_undegraded":"2024-04-25T00:35:07.816657+0000","last_fullsized":"2024-04-25T00:35:07.816657+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:42.894530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.800987+0000","last_change":"2024-04-25T00:35:06.779884+0000","last_active":"2024-04-25T00:35:07.800987+0000","last_peered":"2024-04-25T00:35:07.800987+0000","last_clean":"2024-04-25T00:35:07.800987+0000","last_became_active":"2024-04-25T00:35:06.779115+0000","last_became_peered":"2024-04-25T00:35:06.779115+0000","last_unstale":"2024-04-25T00:35:07.800987+0000","last_undegraded":"2024-04-25T00:35:07.800987+0000","last_fullsized":"2024-04-25T00:35:07.800987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:43.217314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824623+0000","last_change":"2024-04-25T00:35:07.818837+0000","last_active":"2024-04-25T00:35:07.824623+0000","last_peered":"2024-04-25T00:35:07.824623+0000","last_clean":"2024-04-25T00:35:07.824623+0000","last_became_active":"2024-04-25T00:35:07.817982+0000","last_became_peered":"2024-04-25T00:35:07.817982+0000","last_unstale":"2024-04-25T00:35:07.824623+0000","last_undegraded":"2024-04-25T00:35:07.824623+0000","last_fullsized":"2024-04-25T00:35:07.824623+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:41.925732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.800307+0000","last_change":"2024-04-25T00:35:06.775556+0000","last_active":"2024-04-25T00:35:07.800307+0000","last_peered":"2024-04-25T00:35:07.800307+0000","last_clean":"2024-04-25T00:35:07.800307+0000","last_became_active":"2024-04-25T00:35:06.775027+0000","last_became_peered":"2024-04-25T00:35:06.775027+0000","last_unstale":"2024-04-25T00:35:07.800307+0000","last_undegraded":"2024-04-25T00:35:07.800307+0000","last_fullsized":"2024-04-25T00:35:07.800307+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:36.477614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.808395+0000","last_change":"2024-04-25T00:35:07.802032+0000","last_active":"2024-04-25T00:35:07.808395+0000","last_peered":"2024-04-25T00:35:07.808395+0000","last_clean":"2024-04-25T00:35:07.808395+0000","last_became_active":"2024-04-25T00:35:07.797785+0000","last_became_peered":"2024-04-25T00:35:07.797785+0000","last_unstale":"2024-04-25T00:35:07.808395+0000","last_undegraded":"2024-04-25T00:35:07.808395+0000","last_fullsized":"2024-04-25T00:35:07.808395+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:16.382487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.841160+0000","last_change":"2024-04-25T00:35:06.775930+0000","last_active":"2024-04-25T00:35:07.841160+0000","last_peered":"2024-04-25T00:35:07.841160+0000","last_clean":"2024-04-25T00:35:07.841160+0000","last_became_active":"2024-04-25T00:35:06.775242+0000","last_became_peered":"2024-04-25T00:35:06.775242+0000","last_unstale":"2024-04-25T00:35:07.841160+0000","last_undegraded":"2024-04-25T00:35:07.841160+0000","last_fullsized":"2024-04-25T00:35:07.841160+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:32.245143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.825191+0000","last_change":"2024-04-25T00:35:07.823516+0000","last_active":"2024-04-25T00:35:07.825191+0000","last_peered":"2024-04-25T00:35:07.825191+0000","last_clean":"2024-04-25T00:35:07.825191+0000","last_became_active":"2024-04-25T00:35:07.819499+0000","last_became_peered":"2024-04-25T00:35:07.819499+0000","last_unstale":"2024-04-25T00:35:07.825191+0000","last_undegraded":"2024-04-25T00:35:07.825191+0000","last_fullsized":"2024-04-25T00:35:07.825191+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:39.963308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.841731+0000","last_change":"2024-04-25T00:35:06.778729+0000","last_active":"2024-04-25T00:35:07.841731+0000","last_peered":"2024-04-25T00:35:07.841731+0000","last_clean":"2024-04-25T00:35:07.841731+0000","last_became_active":"2024-04-25T00:35:06.778016+0000","last_became_peered":"2024-04-25T00:35:06.778016+0000","last_unstale":"2024-04-25T00:35:07.841731+0000","last_undegraded":"2024-04-25T00:35:07.841731+0000","last_fullsized":"2024-04-25T00:35:07.841731+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:15.371976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.797499+0000","last_change":"2024-04-25T00:35:07.792292+0000","last_active":"2024-04-25T00:35:07.797499+0000","last_peered":"2024-04-25T00:35:07.797499+0000","last_clean":"2024-04-25T00:35:07.797499+0000","last_became_active":"2024-04-25T00:35:07.791144+0000","last_became_peered":"2024-04-25T00:35:07.791144+0000","last_unstale":"2024-04-25T00:35:07.797499+0000","last_undegraded":"2024-04-25T00:35:07.797499+0000","last_fullsized":"2024-04-25T00:35:07.797499+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:33.255653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.835950+0000","last_change":"2024-04-25T00:35:06.767630+0000","last_active":"2024-04-25T00:35:07.835950+0000","last_peered":"2024-04-25T00:35:07.835950+0000","last_clean":"2024-04-25T00:35:07.835950+0000","last_became_active":"2024-04-25T00:35:06.767232+0000","last_became_peered":"2024-04-25T00:35:06.767232+0000","last_unstale":"2024-04-25T00:35:07.835950+0000","last_undegraded":"2024-04-25T00:35:07.835950+0000","last_fullsized":"2024-04-25T00:35:07.835950+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:40.915221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.814335+0000","last_change":"2024-04-25T00:35:07.811065+0000","last_active":"2024-04-25T00:35:07.814335+0000","last_peered":"2024-04-25T00:35:07.814335+0000","last_clean":"2024-04-25T00:35:07.814335+0000","last_became_active":"2024-04-25T00:35:07.809279+0000","last_became_peered":"2024-04-25T00:35:07.809279+0000","last_unstale":"2024-04-25T00:35:07.814335+0000","last_undegraded":"2024-04-25T00:35:07.814335+0000","last_fullsized":"2024-04-25T00:35:07.814335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:06.584791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.781787+0000","last_change":"2024-04-25T00:35:06.764142+0000","last_active":"2024-04-25T00:35:07.781787+0000","last_peered":"2024-04-25T00:35:07.781787+0000","last_clean":"2024-04-25T00:35:07.781787+0000","last_became_active":"2024-04-25T00:35:06.763345+0000","last_became_peered":"2024-04-25T00:35:06.763345+0000","last_unstale":"2024-04-25T00:35:07.781787+0000","last_undegraded":"2024-04-25T00:35:07.781787+0000","last_fullsized":"2024-04-25T00:35:07.781787+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:13.274709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.798435+0000","last_change":"2024-04-25T00:35:07.792491+0000","last_active":"2024-04-25T00:35:07.798435+0000","last_peered":"2024-04-25T00:35:07.798435+0000","last_clean":"2024-04-25T00:35:07.798435+0000","last_became_active":"2024-04-25T00:35:07.791836+0000","last_became_peered":"2024-04-25T00:35:07.791836+0000","last_unstale":"2024-04-25T00:35:07.798435+0000","last_undegraded":"2024-04-25T00:35:07.798435+0000","last_fullsized":"2024-04-25T00:35:07.798435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:22.500973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.774965+0000","last_change":"2024-04-25T00:35:06.775947+0000","last_active":"2024-04-25T00:35:07.774965+0000","last_peered":"2024-04-25T00:35:07.774965+0000","last_clean":"2024-04-25T00:35:07.774965+0000","last_became_active":"2024-04-25T00:35:06.775112+0000","last_became_peered":"2024-04-25T00:35:06.775112+0000","last_unstale":"2024-04-25T00:35:07.774965+0000","last_undegraded":"2024-04-25T00:35:07.774965+0000","last_fullsized":"2024-04-25T00:35:07.774965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:16.049571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824511+0000","last_change":"2024-04-25T00:35:07.817742+0000","last_active":"2024-04-25T00:35:07.824511+0000","last_peered":"2024-04-25T00:35:07.824511+0000","last_clean":"2024-04-25T00:35:07.824511+0000","last_became_active":"2024-04-25T00:35:07.817069+0000","last_became_peered":"2024-04-25T00:35:07.817069+0000","last_unstale":"2024-04-25T00:35:07.824511+0000","last_undegraded":"2024-04-25T00:35:07.824511+0000","last_fullsized":"2024-04-25T00:35:07.824511+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:34.831027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.843947+0000","last_change":"2024-04-25T00:35:06.782423+0000","last_active":"2024-04-25T00:35:07.843947+0000","last_peered":"2024-04-25T00:35:07.843947+0000","last_clean":"2024-04-25T00:35:07.843947+0000","last_became_active":"2024-04-25T00:35:06.781791+0000","last_became_peered":"2024-04-25T00:35:06.781791+0000","last_unstale":"2024-04-25T00:35:07.843947+0000","last_undegraded":"2024-04-25T00:35:07.843947+0000","last_fullsized":"2024-04-25T00:35:07.843947+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:59.723663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824689+0000","last_change":"2024-04-25T00:35:07.817996+0000","last_active":"2024-04-25T00:35:07.824689+0000","last_peered":"2024-04-25T00:35:07.824689+0000","last_clean":"2024-04-25T00:35:07.824689+0000","last_became_active":"2024-04-25T00:35:07.817328+0000","last_became_peered":"2024-04-25T00:35:07.817328+0000","last_unstale":"2024-04-25T00:35:07.824689+0000","last_undegraded":"2024-04-25T00:35:07.824689+0000","last_fullsized":"2024-04-25T00:35:07.824689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:31.098467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.834139+0000","last_change":"2024-04-25T00:35:06.772681+0000","last_active":"2024-04-25T00:35:07.834139+0000","last_peered":"2024-04-25T00:35:07.834139+0000","last_clean":"2024-04-25T00:35:07.834139+0000","last_became_active":"2024-04-25T00:35:06.772017+0000","last_became_peered":"2024-04-25T00:35:06.772017+0000","last_unstale":"2024-04-25T00:35:07.834139+0000","last_undegraded":"2024-04-25T00:35:07.834139+0000","last_fullsized":"2024-04-25T00:35:07.834139+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:30.087957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.813750+0000","last_change":"2024-04-25T00:35:07.801359+0000","last_active":"2024-04-25T00:35:07.813750+0000","last_peered":"2024-04-25T00:35:07.813750+0000","last_clean":"2024-04-25T00:35:07.813750+0000","last_became_active":"2024-04-25T00:35:07.800582+0000","last_became_peered":"2024-04-25T00:35:07.800582+0000","last_unstale":"2024-04-25T00:35:07.813750+0000","last_undegraded":"2024-04-25T00:35:07.813750+0000","last_fullsized":"2024-04-25T00:35:07.813750+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:16.351363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.834771+0000","last_change":"2024-04-25T00:35:06.776351+0000","last_active":"2024-04-25T00:35:07.834771+0000","last_peered":"2024-04-25T00:35:07.834771+0000","last_clean":"2024-04-25T00:35:07.834771+0000","last_became_active":"2024-04-25T00:35:06.775959+0000","last_became_peered":"2024-04-25T00:35:06.775959+0000","last_unstale":"2024-04-25T00:35:07.834771+0000","last_undegraded":"2024-04-25T00:35:07.834771+0000","last_fullsized":"2024-04-25T00:35:07.834771+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:00.087036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822224+0000","last_change":"2024-04-25T00:35:07.804006+0000","last_active":"2024-04-25T00:35:07.822224+0000","last_peered":"2024-04-25T00:35:07.822224+0000","last_clean":"2024-04-25T00:35:07.822224+0000","last_became_active":"2024-04-25T00:35:07.803267+0000","last_became_peered":"2024-04-25T00:35:07.803267+0000","last_unstale":"2024-04-25T00:35:07.822224+0000","last_undegraded":"2024-04-25T00:35:07.822224+0000","last_fullsized":"2024-04-25T00:35:07.822224+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:28.555554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.843186+0000","last_change":"2024-04-25T00:35:06.775580+0000","last_active":"2024-04-25T00:35:07.843186+0000","last_peered":"2024-04-25T00:35:07.843186+0000","last_clean":"2024-04-25T00:35:07.843186+0000","last_became_active":"2024-04-25T00:35:06.773804+0000","last_became_peered":"2024-04-25T00:35:06.773804+0000","last_unstale":"2024-04-25T00:35:07.843186+0000","last_undegraded":"2024-04-25T00:35:07.843186+0000","last_fullsized":"2024-04-25T00:35:07.843186+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:16.627989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.825167+0000","last_change":"2024-04-25T00:35:07.822995+0000","last_active":"2024-04-25T00:35:07.825167+0000","last_peered":"2024-04-25T00:35:07.825167+0000","last_clean":"2024-04-25T00:35:07.825167+0000","last_became_active":"2024-04-25T00:35:07.818496+0000","last_became_peered":"2024-04-25T00:35:07.818496+0000","last_unstale":"2024-04-25T00:35:07.825167+0000","last_undegraded":"2024-04-25T00:35:07.825167+0000","last_fullsized":"2024-04-25T00:35:07.825167+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:37.488124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.795441+0000","last_change":"2024-04-25T00:35:06.772616+0000","last_active":"2024-04-25T00:35:07.795441+0000","last_peered":"2024-04-25T00:35:07.795441+0000","last_clean":"2024-04-25T00:35:07.795441+0000","last_became_active":"2024-04-25T00:35:06.771660+0000","last_became_peered":"2024-04-25T00:35:06.771660+0000","last_unstale":"2024-04-25T00:35:07.795441+0000","last_undegraded":"2024-04-25T00:35:07.795441+0000","last_fullsized":"2024-04-25T00:35:07.795441+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:35.183700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.797564+0000","last_change":"2024-04-25T00:35:07.792613+0000","last_active":"2024-04-25T00:35:07.797564+0000","last_peered":"2024-04-25T00:35:07.797564+0000","last_clean":"2024-04-25T00:35:07.797564+0000","last_became_active":"2024-04-25T00:35:07.791912+0000","last_became_peered":"2024-04-25T00:35:07.791912+0000","last_unstale":"2024-04-25T00:35:07.797564+0000","last_undegraded":"2024-04-25T00:35:07.797564+0000","last_fullsized":"2024-04-25T00:35:07.797564+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:16.590960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.837909+0000","last_change":"2024-04-25T00:35:06.769911+0000","last_active":"2024-04-25T00:35:07.837909+0000","last_peered":"2024-04-25T00:35:07.837909+0000","last_clean":"2024-04-25T00:35:07.837909+0000","last_became_active":"2024-04-25T00:35:06.769533+0000","last_became_peered":"2024-04-25T00:35:06.769533+0000","last_unstale":"2024-04-25T00:35:07.837909+0000","last_undegraded":"2024-04-25T00:35:07.837909+0000","last_fullsized":"2024-04-25T00:35:07.837909+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:21.794152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.815374+0000","last_change":"2024-04-25T00:35:07.804754+0000","last_active":"2024-04-25T00:35:07.815374+0000","last_peered":"2024-04-25T00:35:07.815374+0000","last_clean":"2024-04-25T00:35:07.815374+0000","last_became_active":"2024-04-25T00:35:07.803758+0000","last_became_peered":"2024-04-25T00:35:07.803758+0000","last_unstale":"2024-04-25T00:35:07.815374+0000","last_undegraded":"2024-04-25T00:35:07.815374+0000","last_fullsized":"2024-04-25T00:35:07.815374+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:22.792165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.771357+0000","last_change":"2024-04-25T00:35:06.769240+0000","last_active":"2024-04-25T00:35:07.771357+0000","last_peered":"2024-04-25T00:35:07.771357+0000","last_clean":"2024-04-25T00:35:07.771357+0000","last_became_active":"2024-04-25T00:35:06.768597+0000","last_became_peered":"2024-04-25T00:35:06.768597+0000","last_unstale":"2024-04-25T00:35:07.771357+0000","last_undegraded":"2024-04-25T00:35:07.771357+0000","last_fullsized":"2024-04-25T00:35:07.771357+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:13.274709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822492+0000","last_change":"2024-04-25T00:35:07.813155+0000","last_active":"2024-04-25T00:35:07.822492+0000","last_peered":"2024-04-25T00:35:07.822492+0000","last_clean":"2024-04-25T00:35:07.822492+0000","last_became_active":"2024-04-25T00:35:07.811513+0000","last_became_peered":"2024-04-25T00:35:07.811513+0000","last_unstale":"2024-04-25T00:35:07.822492+0000","last_undegraded":"2024-04-25T00:35:07.822492+0000","last_fullsized":"2024-04-25T00:35:07.822492+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:16.590960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.773915+0000","last_change":"2024-04-25T00:35:06.772036+0000","last_active":"2024-04-25T00:35:07.773915+0000","last_peered":"2024-04-25T00:35:07.773915+0000","last_clean":"2024-04-25T00:35:07.773915+0000","last_became_active":"2024-04-25T00:35:06.771327+0000","last_became_peered":"2024-04-25T00:35:06.771327+0000","last_unstale":"2024-04-25T00:35:07.773915+0000","last_undegraded":"2024-04-25T00:35:07.773915+0000","last_fullsized":"2024-04-25T00:35:07.773915+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:00.087036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.812667+0000","last_change":"2024-04-25T00:35:07.802717+0000","last_active":"2024-04-25T00:35:07.812667+0000","last_peered":"2024-04-25T00:35:07.812667+0000","last_clean":"2024-04-25T00:35:07.812667+0000","last_became_active":"2024-04-25T00:35:07.801976+0000","last_became_peered":"2024-04-25T00:35:07.801976+0000","last_unstale":"2024-04-25T00:35:07.812667+0000","last_undegraded":"2024-04-25T00:35:07.812667+0000","last_fullsized":"2024-04-25T00:35:07.812667+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:06.584791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.835618+0000","last_change":"2024-04-25T00:35:06.762083+0000","last_active":"2024-04-25T00:35:07.835618+0000","last_peered":"2024-04-25T00:35:07.835618+0000","last_clean":"2024-04-25T00:35:07.835618+0000","last_became_active":"2024-04-25T00:35:06.761636+0000","last_became_peered":"2024-04-25T00:35:06.761636+0000","last_unstale":"2024-04-25T00:35:07.835618+0000","last_undegraded":"2024-04-25T00:35:07.835618+0000","last_fullsized":"2024-04-25T00:35:07.835618+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:53.694270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.800198+0000","last_change":"2024-04-25T00:35:07.788635+0000","last_active":"2024-04-25T00:35:07.800198+0000","last_peered":"2024-04-25T00:35:07.800198+0000","last_clean":"2024-04-25T00:35:07.800198+0000","last_became_active":"2024-04-25T00:35:07.787838+0000","last_became_peered":"2024-04-25T00:35:07.787838+0000","last_unstale":"2024-04-25T00:35:07.800198+0000","last_undegraded":"2024-04-25T00:35:07.800198+0000","last_fullsized":"2024-04-25T00:35:07.800198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.734173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.796015+0000","last_change":"2024-04-25T00:35:06.777786+0000","last_active":"2024-04-25T00:35:07.796015+0000","last_peered":"2024-04-25T00:35:07.796015+0000","last_clean":"2024-04-25T00:35:07.796015+0000","last_became_active":"2024-04-25T00:35:06.777358+0000","last_became_peered":"2024-04-25T00:35:06.777358+0000","last_unstale":"2024-04-25T00:35:07.796015+0000","last_undegraded":"2024-04-25T00:35:07.796015+0000","last_fullsized":"2024-04-25T00:35:07.796015+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:36.832974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.802603+0000","last_change":"2024-04-25T00:35:07.791535+0000","last_active":"2024-04-25T00:35:07.802603+0000","last_peered":"2024-04-25T00:35:07.802603+0000","last_clean":"2024-04-25T00:35:07.802603+0000","last_became_active":"2024-04-25T00:35:07.790779+0000","last_became_peered":"2024-04-25T00:35:07.790779+0000","last_unstale":"2024-04-25T00:35:07.802603+0000","last_undegraded":"2024-04-25T00:35:07.802603+0000","last_fullsized":"2024-04-25T00:35:07.802603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:02.293518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.781691+0000","last_change":"2024-04-25T00:35:06.771096+0000","last_active":"2024-04-25T00:35:07.781691+0000","last_peered":"2024-04-25T00:35:07.781691+0000","last_clean":"2024-04-25T00:35:07.781691+0000","last_became_active":"2024-04-25T00:35:06.770348+0000","last_became_peered":"2024-04-25T00:35:06.770348+0000","last_unstale":"2024-04-25T00:35:07.781691+0000","last_undegraded":"2024-04-25T00:35:07.781691+0000","last_fullsized":"2024-04-25T00:35:07.781691+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:24.200300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822733+0000","last_change":"2024-04-25T00:35:07.806367+0000","last_active":"2024-04-25T00:35:07.822733+0000","last_peered":"2024-04-25T00:35:07.822733+0000","last_clean":"2024-04-25T00:35:07.822733+0000","last_became_active":"2024-04-25T00:35:07.805335+0000","last_became_peered":"2024-04-25T00:35:07.805335+0000","last_unstale":"2024-04-25T00:35:07.822733+0000","last_undegraded":"2024-04-25T00:35:07.822733+0000","last_fullsized":"2024-04-25T00:35:07.822733+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:17.638499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.776587+0000","last_change":"2024-04-25T00:35:06.769923+0000","last_active":"2024-04-25T00:35:07.776587+0000","last_peered":"2024-04-25T00:35:07.776587+0000","last_clean":"2024-04-25T00:35:07.776587+0000","last_became_active":"2024-04-25T00:35:06.769156+0000","last_became_peered":"2024-04-25T00:35:06.769156+0000","last_unstale":"2024-04-25T00:35:07.776587+0000","last_undegraded":"2024-04-25T00:35:07.776587+0000","last_fullsized":"2024-04-25T00:35:07.776587+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:21.781654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824072+0000","last_change":"2024-04-25T00:35:07.809813+0000","last_active":"2024-04-25T00:35:07.824072+0000","last_peered":"2024-04-25T00:35:07.824072+0000","last_clean":"2024-04-25T00:35:07.824072+0000","last_became_active":"2024-04-25T00:35:07.807925+0000","last_became_peered":"2024-04-25T00:35:07.807925+0000","last_unstale":"2024-04-25T00:35:07.824072+0000","last_undegraded":"2024-04-25T00:35:07.824072+0000","last_fullsized":"2024-04-25T00:35:07.824072+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:37.901106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.771414+0000","last_change":"2024-04-25T00:35:06.762078+0000","last_active":"2024-04-25T00:35:07.771414+0000","last_peered":"2024-04-25T00:35:07.771414+0000","last_clean":"2024-04-25T00:35:07.771414+0000","last_became_active":"2024-04-25T00:35:06.761766+0000","last_became_peered":"2024-04-25T00:35:06.761766+0000","last_unstale":"2024-04-25T00:35:07.771414+0000","last_undegraded":"2024-04-25T00:35:07.771414+0000","last_fullsized":"2024-04-25T00:35:07.771414+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:06.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.824548+0000","last_change":"2024-04-25T00:35:07.817773+0000","last_active":"2024-04-25T00:35:07.824548+0000","last_peered":"2024-04-25T00:35:07.824548+0000","last_clean":"2024-04-25T00:35:07.824548+0000","last_became_active":"2024-04-25T00:35:07.817007+0000","last_became_peered":"2024-04-25T00:35:07.817007+0000","last_unstale":"2024-04-25T00:35:07.824548+0000","last_undegraded":"2024-04-25T00:35:07.824548+0000","last_fullsized":"2024-04-25T00:35:07.824548+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:14.138865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.782599+0000","last_change":"2024-04-25T00:35:06.764548+0000","last_active":"2024-04-25T00:35:07.782599+0000","last_peered":"2024-04-25T00:35:07.782599+0000","last_clean":"2024-04-25T00:35:07.782599+0000","last_became_active":"2024-04-25T00:35:06.763862+0000","last_became_peered":"2024-04-25T00:35:06.763862+0000","last_unstale":"2024-04-25T00:35:07.782599+0000","last_undegraded":"2024-04-25T00:35:07.782599+0000","last_fullsized":"2024-04-25T00:35:07.782599+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:03.177703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.808190+0000","last_change":"2024-04-25T00:35:07.799638+0000","last_active":"2024-04-25T00:35:07.808190+0000","last_peered":"2024-04-25T00:35:07.808190+0000","last_clean":"2024-04-25T00:35:07.808190+0000","last_became_active":"2024-04-25T00:35:07.798881+0000","last_became_peered":"2024-04-25T00:35:07.798881+0000","last_unstale":"2024-04-25T00:35:07.808190+0000","last_undegraded":"2024-04-25T00:35:07.808190+0000","last_fullsized":"2024-04-25T00:35:07.808190+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:04.188213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.771308+0000","last_change":"2024-04-25T00:35:06.768474+0000","last_active":"2024-04-25T00:35:07.771308+0000","last_peered":"2024-04-25T00:35:07.771308+0000","last_clean":"2024-04-25T00:35:07.771308+0000","last_became_active":"2024-04-25T00:35:06.768123+0000","last_became_peered":"2024-04-25T00:35:06.768123+0000","last_unstale":"2024-04-25T00:35:07.771308+0000","last_undegraded":"2024-04-25T00:35:07.771308+0000","last_fullsized":"2024-04-25T00:35:07.771308+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:24.200300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822852+0000","last_change":"2024-04-25T00:35:07.812232+0000","last_active":"2024-04-25T00:35:07.822852+0000","last_peered":"2024-04-25T00:35:07.822852+0000","last_clean":"2024-04-25T00:35:07.822852+0000","last_became_active":"2024-04-25T00:35:07.811421+0000","last_became_peered":"2024-04-25T00:35:07.811421+0000","last_unstale":"2024-04-25T00:35:07.822852+0000","last_undegraded":"2024-04-25T00:35:07.822852+0000","last_fullsized":"2024-04-25T00:35:07.822852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:49.277843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.798256+0000","last_change":"2024-04-25T00:35:06.774317+0000","last_active":"2024-04-25T00:35:07.798256+0000","last_peered":"2024-04-25T00:35:07.798256+0000","last_clean":"2024-04-25T00:35:07.798256+0000","last_became_active":"2024-04-25T00:35:06.773806+0000","last_became_peered":"2024-04-25T00:35:06.773806+0000","last_unstale":"2024-04-25T00:35:07.798256+0000","last_undegraded":"2024-04-25T00:35:07.798256+0000","last_fullsized":"2024-04-25T00:35:07.798256+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:13.128354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.822977+0000","last_change":"2024-04-25T00:35:07.814782+0000","last_active":"2024-04-25T00:35:07.822977+0000","last_peered":"2024-04-25T00:35:07.822977+0000","last_clean":"2024-04-25T00:35:07.822977+0000","last_became_active":"2024-04-25T00:35:07.813963+0000","last_became_peered":"2024-04-25T00:35:07.813963+0000","last_unstale":"2024-04-25T00:35:07.822977+0000","last_undegraded":"2024-04-25T00:35:07.822977+0000","last_fullsized":"2024-04-25T00:35:07.822977+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:06.731126+0000","last_clean_scrub_stamp":"2024-04-25T00:35:06.731126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:25.408943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.782137+0000","last_change":"2024-04-25T00:35:06.765161+0000","last_active":"2024-04-25T00:35:07.782137+0000","last_peered":"2024-04-25T00:35:07.782137+0000","last_clean":"2024-04-25T00:35:07.782137+0000","last_became_active":"2024-04-25T00:35:06.764318+0000","last_became_peered":"2024-04-25T00:35:06.764318+0000","last_unstale":"2024-04-25T00:35:07.782137+0000","last_undegraded":"2024-04-25T00:35:07.782137+0000","last_fullsized":"2024-04-25T00:35:07.782137+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:05.720615+0000","last_clean_scrub_stamp":"2024-04-25T00:35:05.720615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:06.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":240,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T00:35:07.791897+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:35:07.791897+0000","last_peered":"2024-04-25T00:35:07.791897+0000","last_clean":"2024-04-25T00:35:07.791897+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:35:07.791897+0000","last_undegraded":"2024-04-25T00:35:07.791897+0000","last_fullsized":"2024-04-25T00:35:07.791897+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":875041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":684032,"data_stored":284513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":684032,"data_stored":284513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":696320,"data_stored":286125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36016,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335824,"statfs":{"total":96636764160,"available":96599883776,"internally_reserved":0,"allocated":692224,"data_stored":284569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":284569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":688128,"data_stored":284603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36592,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335248,"statfs":{"total":96636764160,"available":96599293952,"internally_reserved":0,"allocated":1282048,"data_stored":875463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:35:12.227 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:35:12.228 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:35:12.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13d1d6c640 1 Processor -- start 2024-04-25T00:35:12.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13d1d6c640 1 -- start start 2024-04-25T00:35:12.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13d1d6c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13cc0cd200 0x7f13cc06b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13d1d6c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f13cc06b750 0x7f13cc0b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13d1d6c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13cc0705a0 con 0x7f13cc153030 2024-04-25T00:35:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13d1d6c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f13cc15d3d0 con 0x7f13cc06b750 2024-04-25T00:35:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13d1d6c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f13cc15c3c0 con 0x7f13cc0cd200 2024-04-25T00:35:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13cbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f13cc06b750 0x7f13cc0b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13cbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f13cc06b750 0x7f13cc0b5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59198/0 (socket says 172.21.15.29:59198) 2024-04-25T00:35:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13cbfff640 1 -- 172.21.15.29:0/4196867326 learned_addr learned my addr 172.21.15.29:0/4196867326 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13caffd640 1 --2- 172.21.15.29:0/4196867326 >> v2:172.21.15.157:3300/0 conn(0x7f13cc0cd200 0x7f13cc06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13cbfff640 1 -- 172.21.15.29:0/4196867326 >> v2:172.21.15.157:3300/0 conn(0x7f13cc0cd200 msgr2=0x7f13cc06b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13cbfff640 1 --2- 172.21.15.29:0/4196867326 >> v2:172.21.15.157:3300/0 conn(0x7f13cc0cd200 0x7f13cc06b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13cbfff640 1 -- 172.21.15.29:0/4196867326 >> v1:172.21.15.157:6789/0 conn(0x7f13cc153030 legacy=0x7f13cc06fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.393+0000 7f13cbfff640 1 -- 172.21.15.29:0/4196867326 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc15d580 con 0x7f13cc06b750 2024-04-25T00:35:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.397+0000 7f13cbfff640 1 --2- 172.21.15.29:0/4196867326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13cc06b750 0x7f13cc0b5ac0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0d1a40 tx=0x7f13c401c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa37ae43b268ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.397+0000 7f13ca7fc640 1 -- 172.21.15.29:0/4196867326 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c40ac020 con 0x7f13cc06b750 2024-04-25T00:35:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.397+0000 7f13ca7fc640 1 -- 172.21.15.29:0/4196867326 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c40a64a0 con 0x7f13cc06b750 2024-04-25T00:35:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.397+0000 7f13ca7fc640 1 -- 172.21.15.29:0/4196867326 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c40a6870 con 0x7f13cc06b750 2024-04-25T00:35:12.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.397+0000 7f13d1d6c640 1 -- 172.21.15.29:0/4196867326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13cc06b750 msgr2=0x7f13cc0b5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.397+0000 7f13d1d6c640 1 --2- 172.21.15.29:0/4196867326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13cc06b750 0x7f13cc0b5ac0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0d1a40 tx=0x7f13c401c7d0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.397+0000 7f13d1d6c640 1 -- 172.21.15.29:0/4196867326 shutdown_connections 2024-04-25T00:35:12.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.397+0000 7f13d1d6c640 1 --2- 172.21.15.29:0/4196867326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13cc06b750 0x7f13cc0b5ac0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.397+0000 7f13d1d6c640 1 --2- 172.21.15.29:0/4196867326 >> v2:172.21.15.157:3300/0 conn(0x7f13cc0cd200 0x7f13cc06b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.397+0000 7f13d1d6c640 1 -- 172.21.15.29:0/4196867326 >> 172.21.15.29:0/4196867326 conn(0x7f13cc0d0c10 msgr2=0x7f13cc0d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:12.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.397+0000 7f13d1d6c640 1 -- 172.21.15.29:0/4196867326 shutdown_connections 2024-04-25T00:35:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.397+0000 7f13d1d6c640 1 -- 172.21.15.29:0/4196867326 wait complete. 2024-04-25T00:35:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13d1d6c640 1 Processor -- start 2024-04-25T00:35:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13d1d6c640 1 -- start start 2024-04-25T00:35:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13d1d6c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13cc06b750 0x7f13cc14e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13d1d6c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13cc0cd200 0x7f13cc167220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13d1d6c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13cc153030 0x7f13cc0b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13d1d6c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc071d80 con 0x7f13cc0cd200 2024-04-25T00:35:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13d1d6c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc15ceb0 con 0x7f13cc153030 2024-04-25T00:35:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13d1d6c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f13cc071f00 con 0x7f13cc06b750 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13caffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13cc0cd200 0x7f13cc167220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13caffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13cc0cd200 0x7f13cc167220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59204/0 (socket says 172.21.15.29:59204) 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13caffd640 1 -- 172.21.15.29:0/1584752133 learned_addr learned my addr 172.21.15.29:0/1584752133 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13cb7fe640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13cc06b750 0x7f13cc14e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13cbfff640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13cc153030 0x7f13cc0b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13caffd640 1 -- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13cc06b750 msgr2=0x7f13cc14e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13caffd640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13cc06b750 0x7f13cc14e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13caffd640 1 -- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13cc153030 msgr2=0x7f13cc0b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13caffd640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13cc153030 0x7f13cc0b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13caffd640 1 -- 172.21.15.29:0/1584752133 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc15be20 con 0x7f13cc0cd200 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13caffd640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13cc0cd200 0x7f13cc167220 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f13c000c820 tx=0x7f13c000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13b3fff640 1 -- 172.21.15.29:0/1584752133 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0004230 con 0x7f13cc0cd200 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13b3fff640 1 -- 172.21.15.29:0/1584752133 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c00043d0 con 0x7f13cc0cd200 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13b3fff640 1 -- 172.21.15.29:0/1584752133 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0007a70 con 0x7f13cc0cd200 2024-04-25T00:35:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13d1d6c640 1 -- 172.21.15.29:0/1584752133 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13cc14d2b0 con 0x7f13cc0cd200 2024-04-25T00:35:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13d1d6c640 1 -- 172.21.15.29:0/1584752133 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13cc14d870 con 0x7f13cc0cd200 2024-04-25T00:35:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13d1d6c640 1 -- 172.21.15.29:0/1584752133 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13cc14dfb0 con 0x7f13cc0cd200 2024-04-25T00:35:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.401+0000 7f13b3fff640 1 -- 172.21.15.29:0/1584752133 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f13c0007c10 con 0x7f13cc0cd200 2024-04-25T00:35:12.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.405+0000 7f13b3fff640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13b8067a70 0x7f13b8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.405+0000 7f13b3fff640 1 -- 172.21.15.29:0/1584752133 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f13c0086050 con 0x7f13cc0cd200 2024-04-25T00:35:12.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.405+0000 7f13cb7fe640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13b8067a70 0x7f13b8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.405+0000 7f13cb7fe640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13b8067a70 0x7f13b8069f30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0cc3a0 tx=0x7f13bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:12.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.409+0000 7f13b3fff640 1 -- 172.21.15.29:0/1584752133 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f13c0015020 con 0x7f13cc0cd200 2024-04-25T00:35:12.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.629+0000 7f13d1d6c640 1 -- 172.21.15.29:0/1584752133 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f13cc069270 con 0x7f13cc0cd200 2024-04-25T00:35:12.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.633+0000 7f13b3fff640 1 -- 172.21.15.29:0/1584752133 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f13c0056890 con 0x7f13cc0cd200 2024-04-25T00:35:12.634 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-04-25T00:35:12.634 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A== 2024-04-25T00:35:12.634 INFO:teuthology.orchestra.run.smithi029.stdout: caps mds = "allow" 2024-04-25T00:35:12.634 INFO:teuthology.orchestra.run.smithi029.stdout: caps mon = "allow r" 2024-04-25T00:35:12.634 INFO:teuthology.orchestra.run.smithi029.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:35:12.634 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-04-25T00:35:12.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.633+0000 7f13d1d6c640 1 -- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13b8067a70 msgr2=0x7f13b8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.633+0000 7f13d1d6c640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13b8067a70 0x7f13b8069f30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0cc3a0 tx=0x7f13bc008040 comp rx=0 tx=0).stop 2024-04-25T00:35:12.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.633+0000 7f13d1d6c640 1 -- 172.21.15.29:0/1584752133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13cc0cd200 msgr2=0x7f13cc167220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.633+0000 7f13d1d6c640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13cc0cd200 0x7f13cc167220 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f13c000c820 tx=0x7f13c000cce0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.633+0000 7f13d1d6c640 1 -- 172.21.15.29:0/1584752133 shutdown_connections 2024-04-25T00:35:12.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.633+0000 7f13d1d6c640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f13b8067a70 0x7f13b8069f30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.633+0000 7f13d1d6c640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13cc153030 0x7f13cc0b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.633+0000 7f13d1d6c640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13cc0cd200 0x7f13cc167220 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.633+0000 7f13d1d6c640 1 --2- 172.21.15.29:0/1584752133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13cc06b750 0x7f13cc14e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.633+0000 7f13d1d6c640 1 -- 172.21.15.29:0/1584752133 >> 172.21.15.29:0/1584752133 conn(0x7f13cc0d0c10 msgr2=0x7f13cc14ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:12.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.633+0000 7f13d1d6c640 1 -- 172.21.15.29:0/1584752133 shutdown_connections 2024-04-25T00:35:12.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.633+0000 7f13d1d6c640 1 -- 172.21.15.29:0/1584752133 wait complete. 2024-04-25T00:35:12.653 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:35:12.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.773+0000 7f8fc8b04640 1 Processor -- start 2024-04-25T00:35:12.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.773+0000 7f8fc8b04640 1 -- start start 2024-04-25T00:35:12.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc8b04640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fc40d0390 0x7f8fc415b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc8b04640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8fc415b640 0x7f8fc4071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc8b04640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fc415bc30 con 0x7f8fc415ad20 2024-04-25T00:35:12.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc8b04640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc40cb6a0 con 0x7f8fc415b640 2024-04-25T00:35:12.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc8b04640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc40cc240 con 0x7f8fc40d0390 2024-04-25T00:35:12.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fbbfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fc40d0390 0x7f8fc415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc2d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8fc415b640 0x7f8fc4071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fbbfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fc40d0390 0x7f8fc415b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:37380/0 (socket says 172.21.15.29:37380) 2024-04-25T00:35:12.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fbbfff640 1 -- 172.21.15.29:0/485098243 learned_addr learned my addr 172.21.15.29:0/485098243 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:12.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc2d9d640 1 -- 172.21.15.29:0/485098243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fc40d0390 msgr2=0x7f8fc415b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:12.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc2d9d640 1 --2- 172.21.15.29:0/485098243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fc40d0390 0x7f8fc415b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc2d9d640 1 -- 172.21.15.29:0/485098243 >> v1:172.21.15.157:6789/0 conn(0x7f8fc415ad20 legacy=0x7f8fc40cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc2d9d640 1 -- 172.21.15.29:0/485098243 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc415bf10 con 0x7f8fc415b640 2024-04-25T00:35:12.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc2d9d640 1 --2- 172.21.15.29:0/485098243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc415b640 0x7f8fc4071d80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00bfa0 tx=0x7f8fbc01eb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35ad95d3d7d985a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:12.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc1d9b640 1 -- 172.21.15.29:0/485098243 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00b2d0 con 0x7f8fc415b640 2024-04-25T00:35:12.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc1d9b640 1 -- 172.21.15.29:0/485098243 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0a7920 con 0x7f8fc415b640 2024-04-25T00:35:12.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.777+0000 7f8fc1d9b640 1 -- 172.21.15.29:0/485098243 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0a7cf0 con 0x7f8fc415b640 2024-04-25T00:35:12.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 -- 172.21.15.29:0/485098243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc415b640 msgr2=0x7f8fc4071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 --2- 172.21.15.29:0/485098243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc415b640 0x7f8fc4071d80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00bfa0 tx=0x7f8fbc01eb40 comp rx=0 tx=0).stop 2024-04-25T00:35:12.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 -- 172.21.15.29:0/485098243 shutdown_connections 2024-04-25T00:35:12.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 --2- 172.21.15.29:0/485098243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc415b640 0x7f8fc4071d80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 --2- 172.21.15.29:0/485098243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fc40d0390 0x7f8fc415b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 -- 172.21.15.29:0/485098243 >> 172.21.15.29:0/485098243 conn(0x7f8fc40d0c40 msgr2=0x7f8fc40d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:12.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 -- 172.21.15.29:0/485098243 shutdown_connections 2024-04-25T00:35:12.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 -- 172.21.15.29:0/485098243 wait complete. 2024-04-25T00:35:12.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 Processor -- start 2024-04-25T00:35:12.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 -- start start 2024-04-25T00:35:12.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fc40d0390 0x7f8fc4152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc415ad20 0x7f8fc4158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc415b640 0x7f8fc40b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc406b210 con 0x7f8fc415b640 2024-04-25T00:35:12.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc406f740 con 0x7f8fc415ad20 2024-04-25T00:35:12.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc8b04640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc406b390 con 0x7f8fc40d0390 2024-04-25T00:35:12.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc415b640 0x7f8fc40b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc415b640 0x7f8fc40b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59208/0 (socket says 172.21.15.29:59208) 2024-04-25T00:35:12.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc2d9d640 1 -- 172.21.15.29:0/3001777614 learned_addr learned my addr 172.21.15.29:0/3001777614 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:12.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fc259c640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fc40d0390 0x7f8fc4152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.781+0000 7f8fbbfff640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc415ad20 0x7f8fc4158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fc2d9d640 1 -- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fc40d0390 msgr2=0x7f8fc4152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fc2d9d640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fc40d0390 0x7f8fc4152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fc2d9d640 1 -- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc415ad20 msgr2=0x7f8fc4158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fc2d9d640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc415ad20 0x7f8fc4158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fc2d9d640 1 -- 172.21.15.29:0/3001777614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc415a3e0 con 0x7f8fc415b640 2024-04-25T00:35:12.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fc2d9d640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc415b640 0x7f8fc40b5ad0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0b1040 tx=0x7f8fbc01b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:12.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fbaffd640 1 -- 172.21.15.29:0/3001777614 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc009540 con 0x7f8fc415b640 2024-04-25T00:35:12.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fbaffd640 1 -- 172.21.15.29:0/3001777614 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fbc009700 con 0x7f8fc415b640 2024-04-25T00:35:12.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fc8b04640 1 -- 172.21.15.29:0/3001777614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc4159340 con 0x7f8fc415b640 2024-04-25T00:35:12.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fbaffd640 1 -- 172.21.15.29:0/3001777614 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc009af0 con 0x7f8fc415b640 2024-04-25T00:35:12.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fc8b04640 1 -- 172.21.15.29:0/3001777614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc4159900 con 0x7f8fc415b640 2024-04-25T00:35:12.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fc8b04640 1 -- 172.21.15.29:0/3001777614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc4069270 con 0x7f8fc415b640 2024-04-25T00:35:12.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fbaffd640 1 -- 172.21.15.29:0/3001777614 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f8fbc0c2030 con 0x7f8fc415b640 2024-04-25T00:35:12.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fbaffd640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fa4067a70 0x7f8fa4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:12.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fbaffd640 1 -- 172.21.15.29:0/3001777614 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8fbc122be0 con 0x7f8fc415b640 2024-04-25T00:35:12.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fc259c640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fa4067a70 0x7f8fa4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:12.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.785+0000 7f8fc259c640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fa4067a70 0x7f8fa4069f30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4001810 tx=0x7f8fb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:12.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.789+0000 7f8fbaffd640 1 -- 172.21.15.29:0/3001777614 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8fbc0c9020 con 0x7f8fc415b640 2024-04-25T00:35:12.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.989+0000 7f8fc8b04640 1 -- 172.21.15.29:0/3001777614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fc4156b40 con 0x7f8fc415b640 2024-04-25T00:35:12.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.989+0000 7f8fbaffd640 1 -- 172.21.15.29:0/3001777614 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5175 (secure 0 0 0) 0x7f8fbc018760 con 0x7f8fc415b640 2024-04-25T00:35:12.991 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:12.991 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/56623075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":56623075},{"type":"v1","addr":"172.21.15.157:6836","nonce":56623075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/748098725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":748098725},{"type":"v1","addr":"172.21.15.157:6837","nonce":748098725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2264413556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2264413556},{"type":"v1","addr":"172.21.15.29:6833","nonce":2264413556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:07.114520+0000","modified":"2024-04-25T00:35:10.185595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6372},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/802824676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":802824676},{"type":"v1","addr":"172.21.15.29:6835","nonce":802824676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372]},"id":4}]} 2024-04-25T00:35:12.992 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 34 2024-04-25T00:35:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.993+0000 7f8fc8b04640 1 -- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fa4067a70 msgr2=0x7f8fa4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.993+0000 7f8fc8b04640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fa4067a70 0x7f8fa4069f30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4001810 tx=0x7f8fb4009040 comp rx=0 tx=0).stop 2024-04-25T00:35:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.993+0000 7f8fc8b04640 1 -- 172.21.15.29:0/3001777614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc415b640 msgr2=0x7f8fc40b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.993+0000 7f8fc8b04640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc415b640 0x7f8fc40b5ad0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0b1040 tx=0x7f8fbc01b8e0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.993+0000 7f8fc8b04640 1 -- 172.21.15.29:0/3001777614 shutdown_connections 2024-04-25T00:35:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.993+0000 7f8fc8b04640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc415b640 0x7f8fc40b5ad0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.993+0000 7f8fc8b04640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fa4067a70 0x7f8fa4069f30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.993+0000 7f8fc8b04640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc415ad20 0x7f8fc4158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.993+0000 7f8fc8b04640 1 --2- 172.21.15.29:0/3001777614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fc40d0390 0x7f8fc4152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:12.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.993+0000 7f8fc8b04640 1 -- 172.21.15.29:0/3001777614 >> 172.21.15.29:0/3001777614 conn(0x7f8fc40d0c40 msgr2=0x7f8fc40c6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:12.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.993+0000 7f8fc8b04640 1 -- 172.21.15.29:0/3001777614 shutdown_connections 2024-04-25T00:35:12.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:12.993+0000 7f8fc8b04640 1 -- 172.21.15.29:0/3001777614 wait complete. 2024-04-25T00:35:13.414 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:35:07.114520+0000', 'modified': '2024-04-25T00:35:10.185595+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6372}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6372': {'gid': 6372, 'name': 'a', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/802824676', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 802824676}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 802824676}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6372, 'qdb_cluster': [6372]} 2024-04-25T00:35:13.414 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:35:13.414 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:35:13.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f8932819640 1 Processor -- start 2024-04-25T00:35:13.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f8932819640 1 -- start start 2024-04-25T00:35:13.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f8932819640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f892c15e8d0 0x7f892c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f8932819640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892c15f3c0 0x7f892c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f8932819640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f892c0b3720 con 0x7f892c0b4780 2024-04-25T00:35:13.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f8932819640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f892c0b42f0 con 0x7f892c15e8d0 2024-04-25T00:35:13.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f8932819640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f892c0b39c0 con 0x7f892c15f3c0 2024-04-25T00:35:13.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f892b7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f892c0b4780 0x7f892c15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46796/0 (socket says 172.21.15.29:46796) 2024-04-25T00:35:13.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f892b7fe640 1 -- 172.21.15.29:0/1869143699 learned_addr learned my addr 172.21.15.29:0/1869143699 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:13.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f892bfff640 1 --2- 172.21.15.29:0/1869143699 >> v2:172.21.15.29:3300/0 conn(0x7f892c15e8d0 0x7f892c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f8930db6640 1 --2- 172.21.15.29:0/1869143699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892c15f3c0 0x7f892c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f892affd640 1 -- 172.21.15.29:0/1869143699 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1427222884 0 0) 0x7f892c0b3720 con 0x7f892c0b4780 2024-04-25T00:35:13.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f892affd640 1 -- 172.21.15.29:0/1869143699 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8918003530 con 0x7f892c0b4780 2024-04-25T00:35:13.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f892bfff640 1 -- 172.21.15.29:0/1869143699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892c15f3c0 msgr2=0x7f892c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f892bfff640 1 --2- 172.21.15.29:0/1869143699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892c15f3c0 0x7f892c15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f892bfff640 1 -- 172.21.15.29:0/1869143699 >> v1:172.21.15.29:6789/0 conn(0x7f892c0b4780 legacy=0x7f892c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.549+0000 7f892bfff640 1 -- 172.21.15.29:0/1869143699 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c0b3ca0 con 0x7f892c15e8d0 2024-04-25T00:35:13.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f892bfff640 1 --2- 172.21.15.29:0/1869143699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f892c15e8d0 0x7f892c0cc240 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f891c009d90 tx=0x7f891c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbae0a078f7d283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f892affd640 1 -- 172.21.15.29:0/1869143699 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c0b2070 con 0x7f892c15e8d0 2024-04-25T00:35:13.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f892affd640 1 -- 172.21.15.29:0/1869143699 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f891c0a5910 con 0x7f892c15e8d0 2024-04-25T00:35:13.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f892affd640 1 -- 172.21.15.29:0/1869143699 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c0a5ce0 con 0x7f892c15e8d0 2024-04-25T00:35:13.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f8932819640 1 -- 172.21.15.29:0/1869143699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f892c15e8d0 msgr2=0x7f892c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f8932819640 1 --2- 172.21.15.29:0/1869143699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f892c15e8d0 0x7f892c0cc240 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f891c009d90 tx=0x7f891c01c940 comp rx=0 tx=0).stop 2024-04-25T00:35:13.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f8932819640 1 -- 172.21.15.29:0/1869143699 shutdown_connections 2024-04-25T00:35:13.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f8932819640 1 --2- 172.21.15.29:0/1869143699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892c15f3c0 0x7f892c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f8932819640 1 --2- 172.21.15.29:0/1869143699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f892c15e8d0 0x7f892c0cc240 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f8932819640 1 -- 172.21.15.29:0/1869143699 >> 172.21.15.29:0/1869143699 conn(0x7f892c0b1870 msgr2=0x7f892c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f8932819640 1 -- 172.21.15.29:0/1869143699 shutdown_connections 2024-04-25T00:35:13.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f8932819640 1 -- 172.21.15.29:0/1869143699 wait complete. 2024-04-25T00:35:13.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f8932819640 1 Processor -- start 2024-04-25T00:35:13.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.553+0000 7f8932819640 1 -- start start 2024-04-25T00:35:13.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8932819640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892c0b4780 0x7f892c1356d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8932819640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f892c15e8d0 0x7f892c0725a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8932819640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f892c15f3c0 0x7f892c1438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8932819640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f892c15d6a0 con 0x7f892c15e8d0 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8932819640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f892c15d520 con 0x7f892c15f3c0 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8932819640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f892c167c70 con 0x7f892c0b4780 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f892b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f892c15e8d0 0x7f892c0725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f892b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f892c15e8d0 0x7f892c0725a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59224/0 (socket says 172.21.15.29:59224) 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f892b7fe640 1 -- 172.21.15.29:0/1288603293 learned_addr learned my addr 172.21.15.29:0/1288603293 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f892bfff640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892c0b4780 0x7f892c1356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f892b7fe640 1 -- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892c0b4780 msgr2=0x7f892c1356d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f892b7fe640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892c0b4780 0x7f892c1356d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f892b7fe640 1 -- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f892c15f3c0 msgr2=0x7f892c1438c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8930db6640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f892c15f3c0 0x7f892c1438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f892b7fe640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f892c15f3c0 0x7f892c1438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f892b7fe640 1 -- 172.21.15.29:0/1288603293 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c072ae0 con 0x7f892c15e8d0 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f892b7fe640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f892c15e8d0 0x7f892c0725a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f891400c480 tx=0x7f891400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8928ff9640 1 -- 172.21.15.29:0/1288603293 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914017070 con 0x7f892c15e8d0 2024-04-25T00:35:13.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8928ff9640 1 -- 172.21.15.29:0/1288603293 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8914005cf0 con 0x7f892c15e8d0 2024-04-25T00:35:13.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8932819640 1 -- 172.21.15.29:0/1288603293 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892c0ce890 con 0x7f892c15e8d0 2024-04-25T00:35:13.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8928ff9640 1 -- 172.21.15.29:0/1288603293 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914007290 con 0x7f892c15e8d0 2024-04-25T00:35:13.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8932819640 1 -- 172.21.15.29:0/1288603293 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892c0ce2d0 con 0x7f892c15e8d0 2024-04-25T00:35:13.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8928ff9640 1 -- 172.21.15.29:0/1288603293 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f891401a070 con 0x7f892c15e8d0 2024-04-25T00:35:13.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8928ff9640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f890c067a70 0x7f890c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f8928ff9640 1 -- 172.21.15.29:0/1288603293 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f89140853b0 con 0x7f892c15e8d0 2024-04-25T00:35:13.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.557+0000 7f892bfff640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f890c067a70 0x7f890c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 Processor -- start 2024-04-25T00:35:13.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 -- start start 2024-04-25T00:35:13.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f892bfff640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f890c067a70 0x7f890c069f30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f891c0af000 tx=0x7f891c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 0x55ca7c9412d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 0x55ca7c8f7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c8f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ca7c7455e0 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ca7c6ccc50 con 0x55ca7c941810 2024-04-25T00:35:13.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55ca7c7415a0 con 0x55ca7c940ef0 2024-04-25T00:35:13.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8930db6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c8f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8930db6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c8f7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59232/0 (socket says 172.21.15.29:59232) 2024-04-25T00:35:13.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8930db6640 1 -- 172.21.15.29:0/2209494792 learned_addr learned my addr 172.21.15.29:0/2209494792 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:13.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f892bfff640 1 --2- 172.21.15.29:0/2209494792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 0x55ca7c9412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f892b7fe640 1 --2- 172.21.15.29:0/2209494792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 0x55ca7c8f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8930db6640 1 -- 172.21.15.29:0/2209494792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 msgr2=0x55ca7c9412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8930db6640 1 --2- 172.21.15.29:0/2209494792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 0x55ca7c9412d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8930db6640 1 -- 172.21.15.29:0/2209494792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 msgr2=0x55ca7c8f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8930db6640 1 --2- 172.21.15.29:0/2209494792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 0x55ca7c8f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8930db6640 1 -- 172.21.15.29:0/2209494792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca7c6cad10 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f892b7fe640 1 --2- 172.21.15.29:0/2209494792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 0x55ca7c8f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8930db6640 1 --2- 172.21.15.29:0/2209494792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c8f7fa0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f89240098c0 tx=0x7f8924009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c51f998c29736f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8921ffb640 1 -- 172.21.15.29:0/2209494792 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924017020 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8921ffb640 1 -- 172.21.15.29:0/2209494792 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892400e950 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8921ffb640 1 -- 172.21.15.29:0/2209494792 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892400f040 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 -- 172.21.15.29:0/2209494792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 msgr2=0x55ca7c8f7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 --2- 172.21.15.29:0/2209494792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c8f7fa0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f89240098c0 tx=0x7f8924009d80 comp rx=0 tx=0).stop 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 -- 172.21.15.29:0/2209494792 shutdown_connections 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 --2- 172.21.15.29:0/2209494792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c8f7fa0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 --2- 172.21.15.29:0/2209494792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 0x55ca7c8f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 --2- 172.21.15.29:0/2209494792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 0x55ca7c9412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 -- 172.21.15.29:0/2209494792 >> 172.21.15.29:0/2209494792 conn(0x55ca7c8f3b70 msgr2=0x55ca7c8f5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 -- 172.21.15.29:0/2209494792 shutdown_connections 2024-04-25T00:35:13.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 -- 172.21.15.29:0/2209494792 wait complete. 2024-04-25T00:35:13.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 -- start start 2024-04-25T00:35:13.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.561+0000 7f8935992000 1 Processor -- start 2024-04-25T00:35:13.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c925f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 0x55ca7c926460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c941810 0x55ca7c9269a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ca7c6ff520 con 0x55ca7c941810 2024-04-25T00:35:13.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ca7c8f39f0 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55ca7c6dddd0 con 0x55ca7c940ef0 2024-04-25T00:35:13.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 10 client.0 _ll_register_callbacks cb 0x55ca7c8e25c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:13.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8930db6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c941810 0x55ca7c9269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8930db6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c941810 0x55ca7c9269a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59236/0 (socket says 172.21.15.29:59236) 2024-04-25T00:35:13.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8930db6640 1 -- 172.21.15.29:0/3476633042 learned_addr learned my addr 172.21.15.29:0/3476633042 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:13.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8930db6640 1 -- 172.21.15.29:0/3476633042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 msgr2=0x55ca7c926460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8930db6640 1 --2- 172.21.15.29:0/3476633042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 0x55ca7c926460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8930db6640 1 -- 172.21.15.29:0/3476633042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c8f7bc0 msgr2=0x55ca7c925f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f892bfff640 1 --2- 172.21.15.29:0/3476633042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c925f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8930db6640 1 --2- 172.21.15.29:0/3476633042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c925f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8930db6640 1 -- 172.21.15.29:0/3476633042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca7c6dd7b0 con 0x55ca7c941810 2024-04-25T00:35:13.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8930db6640 1 --2- 172.21.15.29:0/3476633042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c941810 0x55ca7c9269a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8924016040 tx=0x7f8924004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f1bba15ee49dde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f890affd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:35:13.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f890affd640 1 -- 172.21.15.29:0/3476633042 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89240094c0 con 0x55ca7c941810 2024-04-25T00:35:13.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f890affd640 1 -- 172.21.15.29:0/3476633042 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f89240139d0 con 0x55ca7c941810 2024-04-25T00:35:13.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 -- 172.21.15.29:0/3476633042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f892400e950 con 0x55ca7c941810 2024-04-25T00:35:13.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f890affd640 1 -- 172.21.15.29:0/3476633042 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8924030050 con 0x55ca7c941810 2024-04-25T00:35:13.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 10 client.6492 fetch_fsmap learned FSMap version 34 2024-04-25T00:35:13.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 -- 172.21.15.29:0/3476633042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55ca7c6ccea0 con 0x55ca7c941810 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f890affd640 1 -- 172.21.15.29:0/3476633042 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892400e950 con 0x55ca7c941810 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f890affd640 1 -- 172.21.15.29:0/3476633042 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f8924033020 con 0x55ca7c941810 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 10 client.6492 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 10 client.6492 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 20 client.6492 populate_metadata read hostname 'smithi029' 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 --2- 172.21.15.29:0/3476633042 >> [v2:172.21.15.29:6834/802824676,v1:172.21.15.29:6835/802824676] conn(0x55ca7c950f90 0x55ca7c953380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 4 client.6492 mds_command: new command op to 6372 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 -- 172.21.15.29:0/3476633042 --> [v2:172.21.15.29:6834/802824676,v1:172.21.15.29:6835/802824676] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca7c6e0ab0 con 0x55ca7c950f90 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f892bfff640 1 --2- 172.21.15.29:0/3476633042 >> [v2:172.21.15.29:6834/802824676,v1:172.21.15.29:6835/802824676] conn(0x55ca7c950f90 0x55ca7c953380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f892bfff640 1 --2- 172.21.15.29:0/3476633042 >> [v2:172.21.15.29:6834/802824676,v1:172.21.15.29:6835/802824676] conn(0x55ca7c950f90 0x55ca7c953380 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b46c27e27faf353b server_cookie=5cf72ece5c689926 in_seq=0 out_seq=0 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f890affd640 10 client.6492 ms_handle_connect on v2:172.21.15.29:6834/802824676 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f890affd640 1 -- 172.21.15.29:0/3476633042 <== mds.0 v2:172.21.15.29:6834/802824676 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca7c6e0ab0 con 0x55ca7c950f90 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f890affd640 10 client.6492 handle_command_reply: tid=0 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 client.6492 shutdown 2024-04-25T00:35:13.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 -- 172.21.15.29:0/3476633042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c941810 msgr2=0x55ca7c9269a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 --2- 172.21.15.29:0/3476633042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c941810 0x55ca7c9269a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8924016040 tx=0x7f8924004740 comp rx=0 tx=0).stop 2024-04-25T00:35:13.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 -- 172.21.15.29:0/3476633042 shutdown_connections 2024-04-25T00:35:13.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 --2- 172.21.15.29:0/3476633042 >> [v2:172.21.15.29:6834/802824676,v1:172.21.15.29:6835/802824676] conn(0x55ca7c950f90 0x55ca7c953380 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 --2- 172.21.15.29:0/3476633042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c941810 0x55ca7c9269a0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 --2- 172.21.15.29:0/3476633042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 0x55ca7c926460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f890affd640 0 client.6492 ms_handle_reset on v2:172.21.15.29:6834/802824676 2024-04-25T00:35:13.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 --2- 172.21.15.29:0/3476633042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c925f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 -- 172.21.15.29:0/3476633042 >> 172.21.15.29:0/3476633042 conn(0x55ca7c8f3b70 msgr2=0x55ca7c8f67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 -- 172.21.15.29:0/3476633042 shutdown_connections 2024-04-25T00:35:13.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 1 -- 172.21.15.29:0/3476633042 wait complete. 2024-04-25T00:35:13.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.565+0000 7f8935992000 20 client.6492 trim_cache size 0 max 16384 2024-04-25T00:35:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 Processor -- start 2024-04-25T00:35:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 -- start start 2024-04-25T00:35:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c8f7bc0 0x55ca7c9ac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c940ef0 0x55ca7c9aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 0x55ca7c9acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f892c202240 con 0x55ca7c940ef0 2024-04-25T00:35:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f892c2015a0 con 0x55ca7c941810 2024-04-25T00:35:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f892c201bf0 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f892b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c940ef0 0x55ca7c9aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f892b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c940ef0 0x55ca7c9aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59244/0 (socket says 172.21.15.29:59244) 2024-04-25T00:35:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f892b7fe640 1 -- 172.21.15.29:0/3837649104 learned_addr learned my addr 172.21.15.29:0/3837649104 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:13.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f892b7fe640 1 -- 172.21.15.29:0/3837649104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c8f7bc0 msgr2=0x55ca7c9ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f892b7fe640 1 --2- 172.21.15.29:0/3837649104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c8f7bc0 0x55ca7c9ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f892b7fe640 1 -- 172.21.15.29:0/3837649104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 msgr2=0x55ca7c9acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f892b7fe640 1 --2- 172.21.15.29:0/3837649104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 0x55ca7c9acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f892b7fe640 1 -- 172.21.15.29:0/3837649104 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca7c6cde80 con 0x55ca7c940ef0 2024-04-25T00:35:13.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f892b7fe640 1 --2- 172.21.15.29:0/3837649104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c940ef0 0x55ca7c9aca40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8914055300 tx=0x7f8914055600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cb4efb6a2fe142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f89217fa640 1 -- 172.21.15.29:0/3837649104 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891405e1e0 con 0x55ca7c940ef0 2024-04-25T00:35:13.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f89217fa640 1 -- 172.21.15.29:0/3837649104 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8914051c70 con 0x55ca7c940ef0 2024-04-25T00:35:13.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f89217fa640 1 -- 172.21.15.29:0/3837649104 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891405dd50 con 0x55ca7c940ef0 2024-04-25T00:35:13.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 -- 172.21.15.29:0/3837649104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c940ef0 msgr2=0x55ca7c9aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 --2- 172.21.15.29:0/3837649104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c940ef0 0x55ca7c9aca40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8914055300 tx=0x7f8914055600 comp rx=0 tx=0).stop 2024-04-25T00:35:13.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 -- 172.21.15.29:0/3837649104 shutdown_connections 2024-04-25T00:35:13.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 --2- 172.21.15.29:0/3837649104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 0x55ca7c9acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 --2- 172.21.15.29:0/3837649104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c940ef0 0x55ca7c9aca40 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 --2- 172.21.15.29:0/3837649104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c8f7bc0 0x55ca7c9ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 -- 172.21.15.29:0/3837649104 >> 172.21.15.29:0/3837649104 conn(0x55ca7c8f3b70 msgr2=0x55ca7c94e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 -- 172.21.15.29:0/3837649104 shutdown_connections 2024-04-25T00:35:13.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.585+0000 7f8935992000 1 -- 172.21.15.29:0/3837649104 wait complete. 2024-04-25T00:35:13.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 -- start start 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 Processor -- start 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c9b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 0x55ca7c9b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 0x55ca7c9532d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ca7c6f9c40 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ca7c7f7e80 con 0x55ca7c941810 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55ca7c70e810 con 0x55ca7c940ef0 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 10 client.0 _ll_register_callbacks cb 0x7f8924009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f892bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c9b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f892bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c9b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59258/0 (socket says 172.21.15.29:59258) 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f892bfff640 1 -- 172.21.15.29:0/3028302681 learned_addr learned my addr 172.21.15.29:0/3028302681 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f892b7fe640 1 --2- 172.21.15.29:0/3028302681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 0x55ca7c9b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8930db6640 1 --2- 172.21.15.29:0/3028302681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 0x55ca7c9532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f892bfff640 1 -- 172.21.15.29:0/3028302681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 msgr2=0x55ca7c9b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f892bfff640 1 --2- 172.21.15.29:0/3028302681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 0x55ca7c9b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f892bfff640 1 -- 172.21.15.29:0/3028302681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 msgr2=0x55ca7c9532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f892bfff640 1 --2- 172.21.15.29:0/3028302681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 0x55ca7c9532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f892bfff640 1 -- 172.21.15.29:0/3028302681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca7c712760 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f892bfff640 1 --2- 172.21.15.29:0/3028302681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c9b0d20 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f891c000f90 tx=0x7f891c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb62b3908c6e9cda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f890affd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f890affd640 1 -- 172.21.15.29:0/3028302681 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c0cd070 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f890affd640 1 -- 172.21.15.29:0/3028302681 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f891c0c9410 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 -- 172.21.15.29:0/3028302681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8914051c70 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f890affd640 1 -- 172.21.15.29:0/3028302681 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f891c0d8050 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 10 client.6498 fetch_fsmap learned FSMap version 34 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 -- 172.21.15.29:0/3028302681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55ca7c79e9e0 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f890affd640 1 -- 172.21.15.29:0/3028302681 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8914051c70 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f890affd640 1 -- 172.21.15.29:0/3028302681 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f891c0db020 con 0x55ca7c8f7bc0 2024-04-25T00:35:13.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 10 client.6498 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:35:13.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 10 client.6498 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:35:13.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 20 client.6498 populate_metadata read hostname 'smithi029' 2024-04-25T00:35:13.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 --2- 172.21.15.29:0/3028302681 >> [v2:172.21.15.29:6834/802824676,v1:172.21.15.29:6835/802824676] conn(0x55ca7c953810 0x55ca7c953bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 4 client.6498 mds_command: new command op to 6372 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:35:13.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 -- 172.21.15.29:0/3028302681 --> [v2:172.21.15.29:6834/802824676,v1:172.21.15.29:6835/802824676] -- command(tid 0: {"prefix": "status"}) -- 0x55ca7c752b20 con 0x55ca7c953810 2024-04-25T00:35:13.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f892b7fe640 1 --2- 172.21.15.29:0/3028302681 >> [v2:172.21.15.29:6834/802824676,v1:172.21.15.29:6835/802824676] conn(0x55ca7c953810 0x55ca7c953bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f892b7fe640 1 --2- 172.21.15.29:0/3028302681 >> [v2:172.21.15.29:6834/802824676,v1:172.21.15.29:6835/802824676] conn(0x55ca7c953810 0x55ca7c953bf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c85e3a522318b31 server_cookie=716cbe246ebd60a4 in_seq=0 out_seq=0 2024-04-25T00:35:13.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f890affd640 10 client.6498 ms_handle_connect on v2:172.21.15.29:6834/802824676 2024-04-25T00:35:13.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f890affd640 1 -- 172.21.15.29:0/3028302681 <== mds.0 v2:172.21.15.29:6834/802824676 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55ca7c752b20 con 0x55ca7c953810 2024-04-25T00:35:13.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f890affd640 10 client.6498 handle_command_reply: tid=0 2024-04-25T00:35:13.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 client.6498 shutdown 2024-04-25T00:35:13.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 -- 172.21.15.29:0/3028302681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 msgr2=0x55ca7c9b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 --2- 172.21.15.29:0/3028302681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c9b0d20 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f891c000f90 tx=0x7f891c0196e0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 -- 172.21.15.29:0/3028302681 shutdown_connections 2024-04-25T00:35:13.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 --2- 172.21.15.29:0/3028302681 >> [v2:172.21.15.29:6834/802824676,v1:172.21.15.29:6835/802824676] conn(0x55ca7c953810 0x55ca7c953bf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 --2- 172.21.15.29:0/3028302681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca7c941810 0x55ca7c9532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 --2- 172.21.15.29:0/3028302681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55ca7c940ef0 0x55ca7c9b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 --2- 172.21.15.29:0/3028302681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca7c8f7bc0 0x55ca7c9b0d20 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f890affd640 0 client.6498 ms_handle_reset on v2:172.21.15.29:6834/802824676 2024-04-25T00:35:13.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 -- 172.21.15.29:0/3028302681 >> 172.21.15.29:0/3028302681 conn(0x55ca7c8f3b70 msgr2=0x55ca7c93f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 -- 172.21.15.29:0/3028302681 shutdown_connections 2024-04-25T00:35:13.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 1 -- 172.21.15.29:0/3028302681 wait complete. 2024-04-25T00:35:13.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.589+0000 7f8935992000 20 client.6498 trim_cache size 0 max 16384 2024-04-25T00:35:13.599 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:35:13.600 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "59afbe7d-aaa8-45e2-b468-69ee150cacea", 2024-04-25T00:35:13.600 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-04-25T00:35:13.600 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 6372, 2024-04-25T00:35:13.600 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-04-25T00:35:13.600 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-04-25T00:35:13.600 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-04-25T00:35:13.600 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 5.7745144369999997, 2024-04-25T00:35:13.600 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 34, 2024-04-25T00:35:13.600 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 58, 2024-04-25T00:35:13.600 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 58, 2024-04-25T00:35:13.600 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 10.806010449 2024-04-25T00:35:13.600 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:35:13.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.593+0000 7f8932819640 1 -- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f890c067a70 msgr2=0x7f890c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.593+0000 7f8932819640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f890c067a70 0x7f890c069f30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f891c0af000 tx=0x7f891c0a7070 comp rx=0 tx=0).stop 2024-04-25T00:35:13.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.593+0000 7f8932819640 1 -- 172.21.15.29:0/1288603293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f892c15e8d0 msgr2=0x7f892c0725a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.593+0000 7f8932819640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f892c15e8d0 0x7f892c0725a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f891400c480 tx=0x7f891400c940 comp rx=0 tx=0).stop 2024-04-25T00:35:13.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.593+0000 7f8932819640 1 -- 172.21.15.29:0/1288603293 shutdown_connections 2024-04-25T00:35:13.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.593+0000 7f8932819640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f890c067a70 0x7f890c069f30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.593+0000 7f8932819640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f892c15f3c0 0x7f892c1438c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.593+0000 7f8932819640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f892c15e8d0 0x7f892c0725a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.593+0000 7f8932819640 1 --2- 172.21.15.29:0/1288603293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892c0b4780 0x7f892c1356d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.593+0000 7f8932819640 1 -- 172.21.15.29:0/1288603293 >> 172.21.15.29:0/1288603293 conn(0x7f892c0b1870 msgr2=0x7f892c0b0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.593+0000 7f8932819640 1 -- 172.21.15.29:0/1288603293 shutdown_connections 2024-04-25T00:35:13.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.593+0000 7f8932819640 1 -- 172.21.15.29:0/1288603293 wait complete. 2024-04-25T00:35:13.610 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:35:13.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa156f50640 1 Processor -- start 2024-04-25T00:35:13.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa156f50640 1 -- start start 2024-04-25T00:35:13.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa156f50640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa1500cf690 0x7fa1500d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa156f50640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa1500d2a90 0x7fa1500b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa156f50640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1500d2ea0 con 0x7fa15015e200 2024-04-25T00:35:13.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa156f50640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa1500b38d0 con 0x7fa1500d2a90 2024-04-25T00:35:13.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa156f50640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa1500cfc10 con 0x7fa1500cf690 2024-04-25T00:35:13.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa154cec640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa15015e200 0x7fa1500d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46806/0 (socket says 172.21.15.29:46806) 2024-04-25T00:35:13.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa154cec640 1 -- 172.21.15.29:0/4048728285 learned_addr learned my addr 172.21.15.29:0/4048728285 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:13.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa1554ed640 1 --2- 172.21.15.29:0/4048728285 >> v2:172.21.15.29:3300/0 conn(0x7fa1500d2a90 0x7fa1500b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa14ffff640 1 --2- 172.21.15.29:0/4048728285 >> v2:172.21.15.157:3300/0 conn(0x7fa1500cf690 0x7fa1500d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa14f7fe640 1 -- 172.21.15.29:0/4048728285 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2276948635 0 0) 0x7fa140001120 con 0x7fa15015e200 2024-04-25T00:35:13.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa14f7fe640 1 -- 172.21.15.29:0/4048728285 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa138003530 con 0x7fa15015e200 2024-04-25T00:35:13.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa1554ed640 1 -- 172.21.15.29:0/4048728285 >> v2:172.21.15.157:3300/0 conn(0x7fa1500cf690 msgr2=0x7fa1500d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa1554ed640 1 --2- 172.21.15.29:0/4048728285 >> v2:172.21.15.157:3300/0 conn(0x7fa1500cf690 0x7fa1500d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa1554ed640 1 -- 172.21.15.29:0/4048728285 >> v1:172.21.15.29:6789/0 conn(0x7fa15015e200 legacy=0x7fa1500d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.721+0000 7fa1554ed640 1 -- 172.21.15.29:0/4048728285 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1500cfef0 con 0x7fa1500d2a90 2024-04-25T00:35:13.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa1554ed640 1 --2- 172.21.15.29:0/4048728285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1500d2a90 0x7fa1500b4320 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa1500d24a0 tx=0x7fa14801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa433114398d8045 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa14f7fe640 1 -- 172.21.15.29:0/4048728285 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1480b2070 con 0x7fa1500d2a90 2024-04-25T00:35:13.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa14f7fe640 1 -- 172.21.15.29:0/4048728285 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1480a58e0 con 0x7fa1500d2a90 2024-04-25T00:35:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa14f7fe640 1 -- 172.21.15.29:0/4048728285 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1480a5cb0 con 0x7fa1500d2a90 2024-04-25T00:35:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 -- 172.21.15.29:0/4048728285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1500d2a90 msgr2=0x7fa1500b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 --2- 172.21.15.29:0/4048728285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1500d2a90 0x7fa1500b4320 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa1500d24a0 tx=0x7fa14801c930 comp rx=0 tx=0).stop 2024-04-25T00:35:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 -- 172.21.15.29:0/4048728285 shutdown_connections 2024-04-25T00:35:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 --2- 172.21.15.29:0/4048728285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1500d2a90 0x7fa1500b4320 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 --2- 172.21.15.29:0/4048728285 >> v2:172.21.15.157:3300/0 conn(0x7fa1500cf690 0x7fa1500d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 -- 172.21.15.29:0/4048728285 >> 172.21.15.29:0/4048728285 conn(0x7fa15009d280 msgr2=0x7fa1500d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 -- 172.21.15.29:0/4048728285 shutdown_connections 2024-04-25T00:35:13.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 -- 172.21.15.29:0/4048728285 wait complete. 2024-04-25T00:35:13.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 Processor -- start 2024-04-25T00:35:13.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 -- start start 2024-04-25T00:35:13.730 INFO:tasks.ceph.mds.a.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fd586ccf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:35:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1500cf690 0x7fa15014e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1500d2a90 0x7fa150154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15015e200 0x7fa15015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa1500b51f0 con 0x7fa1500cf690 2024-04-25T00:35:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1500b5070 con 0x7fa1500d2a90 2024-04-25T00:35:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa156f50640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa1500b5370 con 0x7fa15015e200 2024-04-25T00:35:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa154cec640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1500cf690 0x7fa15014e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.725+0000 7fa154cec640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1500cf690 0x7fa15014e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59284/0 (socket says 172.21.15.29:59284) 2024-04-25T00:35:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa154cec640 1 -- 172.21.15.29:0/526451377 learned_addr learned my addr 172.21.15.29:0/526451377 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa1554ed640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15015e200 0x7fa15015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa14ffff640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1500d2a90 0x7fa150154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa154cec640 1 -- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15015e200 msgr2=0x7fa15015b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa154cec640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15015e200 0x7fa15015b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa154cec640 1 -- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1500d2a90 msgr2=0x7fa150154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa154cec640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1500d2a90 0x7fa150154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa154cec640 1 -- 172.21.15.29:0/526451377 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa150156430 con 0x7fa1500cf690 2024-04-25T00:35:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa1554ed640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15015e200 0x7fa15015b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa14ffff640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1500d2a90 0x7fa150154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa154cec640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1500cf690 0x7fa15014e4b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa14000c3c0 tx=0x7fa14000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa14d7fa640 1 -- 172.21.15.29:0/526451377 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140017070 con 0x7fa1500cf690 2024-04-25T00:35:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa14d7fa640 1 -- 172.21.15.29:0/526451377 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa140005cf0 con 0x7fa1500cf690 2024-04-25T00:35:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa156f50640 1 -- 172.21.15.29:0/526451377 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa150156710 con 0x7fa1500cf690 2024-04-25T00:35:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa14d7fa640 1 -- 172.21.15.29:0/526451377 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140007310 con 0x7fa1500cf690 2024-04-25T00:35:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa156f50640 1 -- 172.21.15.29:0/526451377 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa150156cd0 con 0x7fa1500cf690 2024-04-25T00:35:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa156f50640 1 -- 172.21.15.29:0/526451377 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa150069270 con 0x7fa1500cf690 2024-04-25T00:35:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa14d7fa640 1 -- 172.21.15.29:0/526451377 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fa14001a070 con 0x7fa1500cf690 2024-04-25T00:35:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa14d7fa640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa13c067aa0 0x7fa13c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa14d7fa640 1 -- 172.21.15.29:0/526451377 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa1400852e0 con 0x7fa1500cf690 2024-04-25T00:35:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa14ffff640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa13c067aa0 0x7fa13c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.729+0000 7fa14ffff640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa13c067aa0 0x7fa13c069f60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa1500ce240 tx=0x7fa144008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.733+0000 7fa14d7fa640 1 -- 172.21.15.29:0/526451377 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa1500d2ea0 con 0x7fa1500cf690 2024-04-25T00:35:13.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.929+0000 7fa156f50640 1 -- 172.21.15.29:0/526451377 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa1500d0460 con 0x7fa1500cf690 2024-04-25T00:35:13.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.929+0000 7fa14d7fa640 1 -- 172.21.15.29:0/526451377 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) ==== 53+0+73 (secure 0 0 0) 0x7fa140055c30 con 0x7fa1500cf690 2024-04-25T00:35:13.931 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:35:13.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.933+0000 7fa156f50640 1 -- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa13c067aa0 msgr2=0x7fa13c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.933+0000 7fa156f50640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa13c067aa0 0x7fa13c069f60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa1500ce240 tx=0x7fa144008040 comp rx=0 tx=0).stop 2024-04-25T00:35:13.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.933+0000 7fa156f50640 1 -- 172.21.15.29:0/526451377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1500cf690 msgr2=0x7fa15014e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.933+0000 7fa156f50640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1500cf690 0x7fa15014e4b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa14000c3c0 tx=0x7fa14000c880 comp rx=0 tx=0).stop 2024-04-25T00:35:13.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.933+0000 7fa156f50640 1 -- 172.21.15.29:0/526451377 shutdown_connections 2024-04-25T00:35:13.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.933+0000 7fa156f50640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15015e200 0x7fa15015b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.933+0000 7fa156f50640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa13c067aa0 0x7fa13c069f60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.933+0000 7fa156f50640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1500d2a90 0x7fa150154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.933+0000 7fa156f50640 1 --2- 172.21.15.29:0/526451377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1500cf690 0x7fa15014e4b0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.933+0000 7fa156f50640 1 -- 172.21.15.29:0/526451377 >> 172.21.15.29:0/526451377 conn(0x7fa15009d280 msgr2=0x7fa15006cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.933+0000 7fa156f50640 1 -- 172.21.15.29:0/526451377 shutdown_connections 2024-04-25T00:35:13.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:13.933+0000 7fa156f50640 1 -- 172.21.15.29:0/526451377 wait complete. 2024-04-25T00:35:13.948 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:35:13.949 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:35:13.949 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-25T00:35:13.949 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:35:13.949 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:35:13.949 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:35:13.949 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:35:13.949 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:35:13.949 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:35:13.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:13.950 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-25T00:35:13.956 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:35:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:35:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:35:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:35:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:13.956 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:35:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:35:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-25T00:35:13.957 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:35:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:35:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-25T00:35:13.957 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:35:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:35:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-25T00:35:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:35:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42246sec preferred_lft 42246sec 2024-04-25T00:35:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-04-25T00:35:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:13.957 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:35:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:35:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-25T00:35:13.958 INFO:teuthology.orchestra.run.smithi029.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:35:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 02:42:28:a5:36:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:35:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:35:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:13.958 INFO:teuthology.orchestra.run.smithi029.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:35:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:35:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:35:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:35:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:13.959 INFO:teuthology.orchestra.run.smithi029.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:35:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:35:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:35:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:13.959 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:35:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:35:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:35:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:13.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:13.959 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:35:14.012 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-25T00:35:14.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:14.013 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-25T00:35:14.063 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-25T00:35:14.063 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:35:14.063 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T00:35:14.063 INFO:teuthology.orchestra.run.smithi029.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:35:14.063 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T00:35:14.063 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:35:14.064 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:35:14.064 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:35:14.064 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:35:14.064 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:35:14.064 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:35:14.064 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:35:14.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:35:14.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:35:14.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:14.186 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:35:14.192 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:35:14.192 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:35:14.192 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:35:14.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:14.192 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-04-25T00:35:14.247 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:35:14.247 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:35:14.247 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:35:14.292 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:35:14.293 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:35:14.293 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:35:14.341 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:35:14.399 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:35:14.446 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:35:14.447 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:35:14.447 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:35:14.447 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:35:14.447 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:35:14.447 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:35:14.447 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:35:14.447 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:35:14.447 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:35:14.447 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:35:14.447 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:35:14.447 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:35:14.447 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:35:14.447 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:35:14.448 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:35:14.449 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:35:14.449 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:35:14.449 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:35:14.449 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:35:14.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:14.450 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:35:14.497 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:35:14.498 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:35:14.547 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:35:14.561 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:35:14.610 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:35:14.610 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:35:14.610 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:35:14.610 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:35:14.610 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:35:14.611 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:35:14.613 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:35:14.613 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:35:14.613 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:35:14.613 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:35:14.614 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:35:14.614 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:35:14.614 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:35:14.614 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:35:14.614 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:35:14.614 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:35:14.614 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:35:14.614 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:35:14.614 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:35:14.615 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:35:14.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:14.616 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:35:14.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:2024-04-25T00:35:14.625+0000 7f0b38c3d580 -1 init, newargv = 0x560e5e079ce0 newargc=15 2024-04-25T00:35:14.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[41893]: starting ceph client 2024-04-25T00:35:14.633 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[41893]: starting fuse 2024-04-25T00:35:15.732 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:35:15.750 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:35:15.801 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:35:15.801 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:35:15.801 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:35:15.801 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:35:15.801 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:35:15.801 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:35:15.801 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:35:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:35:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:35:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:35:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:35:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:35:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:35:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:35:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:35:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:35:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:35:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:35:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:35:15.802 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:35:15.803 INFO:teuthology.orchestra.run.smithi029.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:35:15.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:15.804 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:35:15.858 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:35:15.859 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:35:15.859 DEBUG:teuthology.orchestra.run.smithi029:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:35:15.859 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-04-25T00:35:15.859 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-04-25T00:35:15.859 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-04-25T00:35:15.859 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-04-25T00:35:15.859 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:35:15.859 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-04-25T00:35:15.859 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:35:15.859 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-04-25T00:35:15.859 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:35:15.859 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.0")) 2024-04-25T00:35:15.860 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-04-25T00:35:15.989 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.0.41893.asok 2024-04-25T00:35:15.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:35:16.001 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41893.asok 2024-04-25T00:35:16.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:16.001 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41893.asok status 2024-04-25T00:35:16.164 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:35:16.164 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-04-25T00:35:16.164 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:35:16.164 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:35:16.164 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "0", 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "41893", 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 6513, 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 6513 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:16.165 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 677449984 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 677449984 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.6513 192.168.0.1:0/677449984", 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.1:0/677449984", 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-04-25T00:35:16.166 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 34, 2024-04-25T00:35:16.167 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 58, 2024-04-25T00:35:16.167 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:35:16.167 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-04-25T00:35:16.167 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-04-25T00:35:16.167 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:35:16.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:16.179 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-04-25T00:35:16.179 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:35:16.185 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-04-25T00:35:16.186 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:35:16.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:16.186 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:35:16.242 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:35:16.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.409+0000 7f0c3064a640 1 Processor -- start 2024-04-25T00:35:16.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.409+0000 7f0c3064a640 1 -- start start 2024-04-25T00:35:16.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.409+0000 7f0c3064a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0c280b3c30 0x7f0c2806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:16.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.409+0000 7f0c3064a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0c280b4b10 0x7f0c280ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:16.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.409+0000 7f0c3064a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c280b41e0 con 0x7f0c280b3140 2024-04-25T00:35:16.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.409+0000 7f0c3064a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f0c2806d960 con 0x7f0c280b3c30 2024-04-25T00:35:16.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.409+0000 7f0c3064a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0c2806dc40 con 0x7f0c280b4b10 2024-04-25T00:35:16.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c2dbe5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0c280b3c30 0x7f0c2806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:16.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c2dbe5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0c280b3c30 0x7f0c2806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59298/0 (socket says 172.21.15.29:59298) 2024-04-25T00:35:16.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c2dbe5640 1 -- 172.21.15.29:0/3754167247 learned_addr learned my addr 172.21.15.29:0/3754167247 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:16.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c2ebe7640 1 --2- 172.21.15.29:0/3754167247 >> v2:172.21.15.157:3300/0 conn(0x7f0c280b4b10 0x7f0c280ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:16.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c2dbe5640 1 -- 172.21.15.29:0/3754167247 >> v2:172.21.15.157:3300/0 conn(0x7f0c280b4b10 msgr2=0x7f0c280ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:16.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c2dbe5640 1 --2- 172.21.15.29:0/3754167247 >> v2:172.21.15.157:3300/0 conn(0x7f0c280b4b10 0x7f0c280ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:16.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c2dbe5640 1 -- 172.21.15.29:0/3754167247 >> v1:172.21.15.157:6789/0 conn(0x7f0c280b3140 legacy=0x7f0c280b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:16.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c2dbe5640 1 -- 172.21.15.29:0/3754167247 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2806df20 con 0x7f0c280b3c30 2024-04-25T00:35:16.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c2ebe7640 1 --2- 172.21.15.29:0/3754167247 >> v2:172.21.15.157:3300/0 conn(0x7f0c280b4b10 0x7f0c280ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:16.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c2dbe5640 1 --2- 172.21.15.29:0/3754167247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c280b3c30 0x7f0c2806e440 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009870 tx=0x7f0c1c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ea6ccac370beb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:16.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c2d3e4640 1 -- 172.21.15.29:0/3754167247 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0b0070 con 0x7f0c280b3c30 2024-04-25T00:35:16.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c2d3e4640 1 -- 172.21.15.29:0/3754167247 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c0ab600 con 0x7f0c280b3c30 2024-04-25T00:35:16.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c2d3e4640 1 -- 172.21.15.29:0/3754167247 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0ab9d0 con 0x7f0c280b3c30 2024-04-25T00:35:16.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c3064a640 1 -- 172.21.15.29:0/3754167247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c280b3c30 msgr2=0x7f0c2806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:16.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c3064a640 1 --2- 172.21.15.29:0/3754167247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c280b3c30 0x7f0c2806e440 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009870 tx=0x7f0c1c01c4e0 comp rx=0 tx=0).stop 2024-04-25T00:35:16.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c3064a640 1 -- 172.21.15.29:0/3754167247 shutdown_connections 2024-04-25T00:35:16.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c3064a640 1 --2- 172.21.15.29:0/3754167247 >> v2:172.21.15.157:3300/0 conn(0x7f0c280b4b10 0x7f0c280ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:16.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c3064a640 1 --2- 172.21.15.29:0/3754167247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c280b3c30 0x7f0c2806e440 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:16.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c3064a640 1 -- 172.21.15.29:0/3754167247 >> 172.21.15.29:0/3754167247 conn(0x7f0c280b1870 msgr2=0x7f0c280b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:16.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c3064a640 1 -- 172.21.15.29:0/3754167247 shutdown_connections 2024-04-25T00:35:16.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.413+0000 7f0c3064a640 1 -- 172.21.15.29:0/3754167247 wait complete. 2024-04-25T00:35:16.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c3064a640 1 Processor -- start 2024-04-25T00:35:16.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c3064a640 1 -- start start 2024-04-25T00:35:16.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c3064a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c280b3140 0x7f0c281398a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:16.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c3064a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c280b3c30 0x7f0c281436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:16.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c3064a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c280b4b10 0x7f0c28153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:16.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c3064a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28168d80 con 0x7f0c280b3c30 2024-04-25T00:35:16.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c3064a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28168c00 con 0x7f0c280b4b10 2024-04-25T00:35:16.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c3064a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0c28168f00 con 0x7f0c280b3140 2024-04-25T00:35:16.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c2dbe5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c280b3c30 0x7f0c281436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:16.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c2dbe5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c280b3c30 0x7f0c281436f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59310/0 (socket says 172.21.15.29:59310) 2024-04-25T00:35:16.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c2dbe5640 1 -- 172.21.15.29:0/2049211872 learned_addr learned my addr 172.21.15.29:0/2049211872 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:16.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c2ebe7640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c280b4b10 0x7f0c28153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:16.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c2e3e6640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c280b3140 0x7f0c281398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:16.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c2dbe5640 1 -- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c280b3140 msgr2=0x7f0c281398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:16.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c2dbe5640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c280b3140 0x7f0c281398a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:16.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c2dbe5640 1 -- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c280b4b10 msgr2=0x7f0c28153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:16.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c2dbe5640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c280b4b10 0x7f0c28153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:16.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c2dbe5640 1 -- 172.21.15.29:0/2049211872 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2813d420 con 0x7f0c280b3c30 2024-04-25T00:35:16.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c2dbe5640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c280b3c30 0x7f0c281436f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0ae040 tx=0x7f0c1c01ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:16.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c1affd640 1 -- 172.21.15.29:0/2049211872 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0b0040 con 0x7f0c280b3c30 2024-04-25T00:35:16.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c1affd640 1 -- 172.21.15.29:0/2049211872 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c0bc070 con 0x7f0c280b3c30 2024-04-25T00:35:16.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c3064a640 1 -- 172.21.15.29:0/2049211872 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c28139000 con 0x7f0c280b3c30 2024-04-25T00:35:16.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c1affd640 1 -- 172.21.15.29:0/2049211872 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0b6880 con 0x7f0c280b3c30 2024-04-25T00:35:16.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c3064a640 1 -- 172.21.15.29:0/2049211872 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c281395c0 con 0x7f0c280b3c30 2024-04-25T00:35:16.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.417+0000 7f0c3064a640 1 -- 172.21.15.29:0/2049211872 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c2813d5f0 con 0x7f0c280b3c30 2024-04-25T00:35:16.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.421+0000 7f0c1affd640 1 -- 172.21.15.29:0/2049211872 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f0c1c0ab4c0 con 0x7f0c280b3c30 2024-04-25T00:35:16.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.421+0000 7f0c1affd640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0c0c067a70 0x7f0c0c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:16.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.421+0000 7f0c1affd640 1 -- 172.21.15.29:0/2049211872 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0c1c1204e0 con 0x7f0c280b3c30 2024-04-25T00:35:16.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.421+0000 7f0c2e3e6640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0c0c067a70 0x7f0c0c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:16.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.421+0000 7f0c2e3e6640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0c0c067a70 0x7f0c0c069f30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0c2813cb30 tx=0x7f0c24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:16.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.425+0000 7f0c1affd640 1 -- 172.21.15.29:0/2049211872 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0c1c0160d0 con 0x7f0c280b3c30 2024-04-25T00:35:16.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.620+0000 7f0c3064a640 1 -- 172.21.15.29:0/2049211872 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0c28069270 con 0x7f0c280b3c30 2024-04-25T00:35:16.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.620+0000 7f0c1affd640 1 -- 172.21.15.29:0/2049211872 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) ==== 53+0+73 (secure 0 0 0) 0x7f0c1c0f0c90 con 0x7f0c280b3c30 2024-04-25T00:35:16.624 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:35:16.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.624+0000 7f0c3064a640 1 -- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0c0c067a70 msgr2=0x7f0c0c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:16.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.624+0000 7f0c3064a640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0c0c067a70 0x7f0c0c069f30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0c2813cb30 tx=0x7f0c24008040 comp rx=0 tx=0).stop 2024-04-25T00:35:16.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.624+0000 7f0c3064a640 1 -- 172.21.15.29:0/2049211872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c280b3c30 msgr2=0x7f0c281436f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:16.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.624+0000 7f0c3064a640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c280b3c30 0x7f0c281436f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0ae040 tx=0x7f0c1c01ad20 comp rx=0 tx=0).stop 2024-04-25T00:35:16.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.624+0000 7f0c3064a640 1 -- 172.21.15.29:0/2049211872 shutdown_connections 2024-04-25T00:35:16.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.624+0000 7f0c3064a640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0c0c067a70 0x7f0c0c069f30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:16.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.624+0000 7f0c3064a640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c280b4b10 0x7f0c28153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:16.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.624+0000 7f0c3064a640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c280b3c30 0x7f0c281436f0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:16.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.624+0000 7f0c3064a640 1 --2- 172.21.15.29:0/2049211872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c280b3140 0x7f0c281398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:16.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.624+0000 7f0c3064a640 1 -- 172.21.15.29:0/2049211872 >> 172.21.15.29:0/2049211872 conn(0x7f0c280b1870 msgr2=0x7f0c28141480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:16.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.624+0000 7f0c3064a640 1 -- 172.21.15.29:0/2049211872 shutdown_connections 2024-04-25T00:35:16.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:16.624+0000 7f0c3064a640 1 -- 172.21.15.29:0/2049211872 wait complete. 2024-04-25T00:35:16.643 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:35:16.644 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:35:16.644 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-25T00:35:16.644 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:35:16.644 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:35:16.644 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:35:16.644 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:35:16.644 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:35:16.644 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:35:16.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:16.644 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-25T00:35:16.651 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:35:16.652 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:35:16.652 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:35:16.652 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:16.652 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:35:16.652 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:16.652 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:35:16.652 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:35:16.652 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-25T00:35:16.652 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:35:16.652 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:35:16.652 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-25T00:35:16.652 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42239sec preferred_lft 42239sec 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:9c:8d:4b:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:35:16.653 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:16.654 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:35:16.654 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:16.654 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:35:16.654 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:35:16.654 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:35:16.654 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:16.654 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:35:16.654 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:35:16.654 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:35:16.654 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:35:16.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:16.655 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:35:16.710 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:35:16.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:16.711 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-25T00:35:16.766 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-25T00:35:16.766 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:35:16.766 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:35:16.766 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:35:16.766 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:35:16.766 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:35:16.766 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:35:16.766 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:35:16.766 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:35:16.766 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:35:16.766 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:35:16.767 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:35:16.767 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:35:16.767 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:35:16.767 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:35:16.767 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:35:16.767 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:35:16.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:35:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:35:16.882 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:35:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:35:16.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:16.889 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:35:16.894 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:35:16.894 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:35:16.895 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:35:16.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:16.895 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-25T00:35:16.946 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T00:35:16.946 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:35:16.946 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:35:16.993 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:35:16.993 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:35:16.993 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:35:17.040 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:35:17.097 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:35:17.147 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:35:17.147 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:35:17.147 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:35:17.147 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:35:17.147 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:35:17.147 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:35:17.147 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:35:17.147 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:35:17.147 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:35:17.147 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:35:17.148 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:35:17.149 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:35:17.149 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:35:17.149 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:35:17.149 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:35:17.149 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:35:17.149 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:35:17.149 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:35:17.149 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:35:17.149 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:35:17.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:17.150 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:35:17.197 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:35:17.197 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T00:35:17.241 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:35:17.256 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:35:17.305 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:35:17.305 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:35:17.305 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:35:17.305 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:35:17.305 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:35:17.305 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:35:17.305 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:35:17.305 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:35:17.305 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:35:17.305 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:35:17.305 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:35:17.305 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:35:17.306 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:35:17.306 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:35:17.306 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:35:17.306 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:35:17.306 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:35:17.306 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:35:17.306 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:35:17.306 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:35:17.306 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:35:17.306 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:35:17.307 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:35:17.307 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:35:17.307 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:35:17.307 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:35:17.307 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:35:17.307 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:35:17.307 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:35:17.307 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:35:17.307 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:35:17.307 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:35:17.307 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:35:17.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:17.308 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:35:17.322 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-04-25T00:35:17.320+0000 7f487fd4a580 -1 init, newargv = 0x55f3408fe090 newargc=15 2024-04-25T00:35:17.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34041]: starting ceph client 2024-04-25T00:35:17.329 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34041]: starting fuse 2024-04-25T00:35:18.429 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:35:18.445 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:35:18.493 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:35:18.494 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:35:18.494 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:35:18.494 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:35:18.494 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:35:18.494 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:35:18.494 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:35:18.494 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:35:18.494 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:35:18.494 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:35:18.494 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:35:18.494 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:35:18.494 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:35:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:35:18.496 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:35:18.496 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:35:18.496 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:35:18.496 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:35:18.496 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:35:18.496 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:35:18.496 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:35:18.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:18.497 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:35:18.544 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:35:18.544 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-25T00:35:18.545 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-25T00:35:18.546 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:35:18.546 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-25T00:35:18.546 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:35:18.546 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:35:18.546 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:35:18.546 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-25T00:35:18.546 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-25T00:35:18.546 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-25T00:35:18.546 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:35:18.546 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:35:18.546 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-04-25T00:35:18.546 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-25T00:35:18.667 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.34041.asok 2024-04-25T00:35:18.670 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:35:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:35:18.679 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34041.asok 2024-04-25T00:35:18.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:18.680 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34041.asok status 2024-04-25T00:35:18.835 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:35:18.835 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-25T00:35:18.835 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:35:18.835 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:35:18.835 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-04-25T00:35:18.835 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-25T00:35:18.835 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T00:35:18.835 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "34041", 2024-04-25T00:35:18.835 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-25T00:35:18.835 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 5875, 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 5875 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 965583334 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-25T00:35:18.836 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 965583334 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.5875 192.168.0.1:0/965583334", 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/965583334", 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 34, 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 58, 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-25T00:35:18.837 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:35:18.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:18.848 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-25T00:35:18.849 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T00:35:18.854 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-25T00:35:18.854 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T00:35:18.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:18.855 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:35:18.908 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-25T00:35:19.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.048+0000 7fcc3dc76640 1 Processor -- start 2024-04-25T00:35:19.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.048+0000 7fcc3dc76640 1 -- start start 2024-04-25T00:35:19.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.048+0000 7fcc3dc76640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc380b8bf0 0x7fcc380d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.048+0000 7fcc3dc76640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc380cfed0 con 0x7fcc3815e560 2024-04-25T00:35:19.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.048+0000 7fcc3dc76640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc380b7100 con 0x7fcc3815e180 2024-04-25T00:35:19.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.048+0000 7fcc3dc76640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fcc380b73e0 con 0x7fcc380b8bf0 2024-04-25T00:35:19.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.052+0000 7fcc37fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc380b8bf0 0x7fcc380d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.052+0000 7fcc37fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc380b8bf0 0x7fcc380d20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46256/0 (socket says 172.21.15.29:46256) 2024-04-25T00:35:19.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.052+0000 7fcc37fff640 1 -- 172.21.15.29:0/4175385740 learned_addr learned my addr 172.21.15.29:0/4175385740 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:19.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.052+0000 7fcc37fff640 1 -- 172.21.15.29:0/4175385740 >> v1:172.21.15.157:6789/0 conn(0x7fcc3815e180 legacy=0x7fcc380cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:19.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.052+0000 7fcc37fff640 1 -- 172.21.15.29:0/4175385740 >> v1:172.21.15.29:6789/0 conn(0x7fcc3815e560 legacy=0x7fcc380d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.052+0000 7fcc37fff640 1 -- 172.21.15.29:0/4175385740 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc380b76c0 con 0x7fcc380b8bf0 2024-04-25T00:35:19.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.056+0000 7fcc37fff640 1 --2- 172.21.15.29:0/4175385740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380b8bf0 0x7fcc380d20e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009e50 tx=0x7fcc3001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacb0ca7462ee03f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.056+0000 7fcc367fc640 1 -- 172.21.15.29:0/4175385740 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc300b4070 con 0x7fcc380b8bf0 2024-04-25T00:35:19.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.056+0000 7fcc367fc640 1 -- 172.21.15.29:0/4175385740 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc300af400 con 0x7fcc380b8bf0 2024-04-25T00:35:19.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.056+0000 7fcc367fc640 1 -- 172.21.15.29:0/4175385740 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc300af7d0 con 0x7fcc380b8bf0 2024-04-25T00:35:19.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.056+0000 7fcc3dc76640 1 -- 172.21.15.29:0/4175385740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380b8bf0 msgr2=0x7fcc380d20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.056+0000 7fcc3dc76640 1 --2- 172.21.15.29:0/4175385740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380b8bf0 0x7fcc380d20e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009e50 tx=0x7fcc3001f1e0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.056+0000 7fcc3dc76640 1 -- 172.21.15.29:0/4175385740 shutdown_connections 2024-04-25T00:35:19.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.056+0000 7fcc3dc76640 1 --2- 172.21.15.29:0/4175385740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc380b8bf0 0x7fcc380d20e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.056+0000 7fcc3dc76640 1 -- 172.21.15.29:0/4175385740 >> 172.21.15.29:0/4175385740 conn(0x7fcc380d40a0 msgr2=0x7fcc380d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:19.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.056+0000 7fcc3dc76640 1 -- 172.21.15.29:0/4175385740 shutdown_connections 2024-04-25T00:35:19.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.056+0000 7fcc3dc76640 1 -- 172.21.15.29:0/4175385740 wait complete. 2024-04-25T00:35:19.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.056+0000 7fcc3dc76640 1 Processor -- start 2024-04-25T00:35:19.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.056+0000 7fcc3dc76640 1 -- start start 2024-04-25T00:35:19.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc3dc76640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380b8bf0 0x7fcc38152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc3dc76640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815e180 0x7fcc3813efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc3dc76640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815e560 0x7fcc3813c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc3dc76640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38168f50 con 0x7fcc3815e560 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc3dc76640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc380d2fe0 con 0x7fcc3815e180 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc3dc76640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcc381690d0 con 0x7fcc380b8bf0 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc37fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815e560 0x7fcc3813c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc37fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815e560 0x7fcc3813c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46264/0 (socket says 172.21.15.29:46264) 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc37fff640 1 -- 172.21.15.29:0/2826957478 learned_addr learned my addr 172.21.15.29:0/2826957478 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc377fe640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380b8bf0 0x7fcc38152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc36ffd640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815e180 0x7fcc3813efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc37fff640 1 -- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380b8bf0 msgr2=0x7fcc38152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc37fff640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380b8bf0 0x7fcc38152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc37fff640 1 -- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815e180 msgr2=0x7fcc3813efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc37fff640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815e180 0x7fcc3813efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc37fff640 1 -- 172.21.15.29:0/2826957478 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3813f8f0 con 0x7fcc3815e560 2024-04-25T00:35:19.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc37fff640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815e560 0x7fcc3813c370 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcc300b2040 tx=0x7fcc3001bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc1ffff640 1 -- 172.21.15.29:0/2826957478 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc300b4040 con 0x7fcc3815e560 2024-04-25T00:35:19.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc1ffff640 1 -- 172.21.15.29:0/2826957478 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc300c0070 con 0x7fcc3815e560 2024-04-25T00:35:19.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc3dc76640 1 -- 172.21.15.29:0/2826957478 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3813fbd0 con 0x7fcc3815e560 2024-04-25T00:35:19.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc1ffff640 1 -- 172.21.15.29:0/2826957478 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30009510 con 0x7fcc3815e560 2024-04-25T00:35:19.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc3dc76640 1 -- 172.21.15.29:0/2826957478 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3813bb70 con 0x7fcc3815e560 2024-04-25T00:35:19.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc3dc76640 1 -- 172.21.15.29:0/2826957478 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc380692e0 con 0x7fcc3815e560 2024-04-25T00:35:19.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc1ffff640 1 -- 172.21.15.29:0/2826957478 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fcc300c5460 con 0x7fcc3815e560 2024-04-25T00:35:19.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc1ffff640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc24067aa0 0x7fcc24069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.060+0000 7fcc1ffff640 1 -- 172.21.15.29:0/2826957478 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fcc30124440 con 0x7fcc3815e560 2024-04-25T00:35:19.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.064+0000 7fcc377fe640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc24067aa0 0x7fcc24069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.064+0000 7fcc377fe640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc24067aa0 0x7fcc24069f60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcc28004e30 tx=0x7fcc28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.068+0000 7fcc1ffff640 1 -- 172.21.15.29:0/2826957478 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcc30018d40 con 0x7fcc3815e560 2024-04-25T00:35:19.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.264+0000 7fcc3dc76640 1 -- 172.21.15.29:0/2826957478 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fcc380d2940 con 0x7fcc3815e560 2024-04-25T00:35:19.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.316+0000 7fcc1ffff640 1 -- 172.21.15.29:0/2826957478 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v35) ==== 136+0+0 (secure 0 0 0) 0x7fcc300f4bf0 con 0x7fcc3815e560 2024-04-25T00:35:19.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.320+0000 7fcc3dc76640 1 -- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc24067aa0 msgr2=0x7fcc24069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.320+0000 7fcc3dc76640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc24067aa0 0x7fcc24069f60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcc28004e30 tx=0x7fcc28009040 comp rx=0 tx=0).stop 2024-04-25T00:35:19.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.320+0000 7fcc3dc76640 1 -- 172.21.15.29:0/2826957478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815e560 msgr2=0x7fcc3813c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.320+0000 7fcc3dc76640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815e560 0x7fcc3813c370 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcc300b2040 tx=0x7fcc3001bf80 comp rx=0 tx=0).stop 2024-04-25T00:35:19.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.320+0000 7fcc3dc76640 1 -- 172.21.15.29:0/2826957478 shutdown_connections 2024-04-25T00:35:19.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.320+0000 7fcc3dc76640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc3815e560 0x7fcc3813c370 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.320+0000 7fcc3dc76640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcc24067aa0 0x7fcc24069f60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.320+0000 7fcc3dc76640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc3815e180 0x7fcc3813efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.320+0000 7fcc3dc76640 1 --2- 172.21.15.29:0/2826957478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc380b8bf0 0x7fcc38152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.320+0000 7fcc3dc76640 1 -- 172.21.15.29:0/2826957478 >> 172.21.15.29:0/2826957478 conn(0x7fcc380d40a0 msgr2=0x7fcc38159c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:19.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.320+0000 7fcc3dc76640 1 -- 172.21.15.29:0/2826957478 shutdown_connections 2024-04-25T00:35:19.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.320+0000 7fcc3dc76640 1 -- 172.21.15.29:0/2826957478 wait complete. 2024-04-25T00:35:19.339 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-25T00:35:19.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c39648640 1 Processor -- start 2024-04-25T00:35:19.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c39648640 1 -- start start 2024-04-25T00:35:19.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c39648640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c3415e8d0 0x7f7c340cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c39648640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3415f3c0 0x7f7c3415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c39648640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c340b3720 con 0x7f7c340b4780 2024-04-25T00:35:19.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c39648640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7c340b42f0 con 0x7f7c3415e8d0 2024-04-25T00:35:19.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c39648640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c340b39c0 con 0x7f7c3415f3c0 2024-04-25T00:35:19.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c32ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c3415e8d0 0x7f7c340cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c2a7fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7c340b4780 0x7f7c3415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:47812/0 (socket says 172.21.15.29:47812) 2024-04-25T00:35:19.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c2a7fc640 1 -- 172.21.15.29:0/939645066 learned_addr learned my addr 172.21.15.29:0/939645066 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:19.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c337fe640 1 --2- 172.21.15.29:0/939645066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3415f3c0 0x7f7c3415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c32ffd640 1 -- 172.21.15.29:0/939645066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3415f3c0 msgr2=0x7f7c3415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:19.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c32ffd640 1 --2- 172.21.15.29:0/939645066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3415f3c0 0x7f7c3415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c32ffd640 1 -- 172.21.15.29:0/939645066 >> v1:172.21.15.29:6789/0 conn(0x7f7c340b4780 legacy=0x7f7c3415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c32ffd640 1 -- 172.21.15.29:0/939645066 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c340b3ca0 con 0x7f7c3415e8d0 2024-04-25T00:35:19.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c327fc640 1 -- 172.21.15.29:0/939645066 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1628253444 0 0) 0x7f7c340b3720 con 0x7f7c340b4780 2024-04-25T00:35:19.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.460+0000 7f7c337fe640 1 --2- 172.21.15.29:0/939645066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3415f3c0 0x7f7c3415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:19.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c32ffd640 1 --2- 172.21.15.29:0/939645066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3415e8d0 0x7f7c340cc240 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7c240097c0 tx=0x7f7c2401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456403e13ce68fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c327fc640 1 -- 172.21.15.29:0/939645066 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c240b2070 con 0x7f7c3415e8d0 2024-04-25T00:35:19.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c327fc640 1 -- 172.21.15.29:0/939645066 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c240ad680 con 0x7f7c3415e8d0 2024-04-25T00:35:19.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c327fc640 1 -- 172.21.15.29:0/939645066 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c240ada50 con 0x7f7c3415e8d0 2024-04-25T00:35:19.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c39648640 1 -- 172.21.15.29:0/939645066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3415e8d0 msgr2=0x7f7c340cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c39648640 1 --2- 172.21.15.29:0/939645066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3415e8d0 0x7f7c340cc240 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7c240097c0 tx=0x7f7c2401c470 comp rx=0 tx=0).stop 2024-04-25T00:35:19.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c39648640 1 -- 172.21.15.29:0/939645066 shutdown_connections 2024-04-25T00:35:19.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c39648640 1 --2- 172.21.15.29:0/939645066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3415f3c0 0x7f7c3415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c39648640 1 --2- 172.21.15.29:0/939645066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3415e8d0 0x7f7c340cc240 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c39648640 1 -- 172.21.15.29:0/939645066 >> 172.21.15.29:0/939645066 conn(0x7f7c340b1870 msgr2=0x7f7c340b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:19.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c39648640 1 -- 172.21.15.29:0/939645066 shutdown_connections 2024-04-25T00:35:19.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c39648640 1 -- 172.21.15.29:0/939645066 wait complete. 2024-04-25T00:35:19.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c39648640 1 Processor -- start 2024-04-25T00:35:19.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.464+0000 7f7c39648640 1 -- start start 2024-04-25T00:35:19.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c39648640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c340b4780 0x7f7c340cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c39648640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3415e8d0 0x7f7c3413b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c39648640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3415f3c0 0x7f7c340ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c39648640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3415d6a0 con 0x7f7c3415f3c0 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c39648640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3415d520 con 0x7f7c340b4780 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c39648640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c34167c70 con 0x7f7c3415e8d0 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c337fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3415f3c0 0x7f7c340ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c337fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3415f3c0 0x7f7c340ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46268/0 (socket says 172.21.15.29:46268) 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c337fe640 1 -- 172.21.15.29:0/189618386 learned_addr learned my addr 172.21.15.29:0/189618386 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c2a7fc640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3415e8d0 0x7f7c3413b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c32ffd640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c340b4780 0x7f7c340cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c337fe640 1 -- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3415e8d0 msgr2=0x7f7c3413b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c337fe640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3415e8d0 0x7f7c3413b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c337fe640 1 -- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c340b4780 msgr2=0x7f7c340cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c337fe640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c340b4780 0x7f7c340cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c337fe640 1 -- 172.21.15.29:0/189618386 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3413bd50 con 0x7f7c3415f3c0 2024-04-25T00:35:19.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c2a7fc640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3415e8d0 0x7f7c3413b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:19.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c337fe640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3415f3c0 0x7f7c340ce180 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00cd80 tx=0x7f7c2c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c2bfff640 1 -- 172.21.15.29:0/189618386 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c00f680 con 0x7f7c3415f3c0 2024-04-25T00:35:19.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c2bfff640 1 -- 172.21.15.29:0/189618386 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2c00f840 con 0x7f7c3415f3c0 2024-04-25T00:35:19.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c39648640 1 -- 172.21.15.29:0/189618386 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3413c090 con 0x7f7c3415f3c0 2024-04-25T00:35:19.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c2bfff640 1 -- 172.21.15.29:0/189618386 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c00fc30 con 0x7f7c3415f3c0 2024-04-25T00:35:19.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c39648640 1 -- 172.21.15.29:0/189618386 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3413c8d0 con 0x7f7c3415f3c0 2024-04-25T00:35:19.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c39648640 1 -- 172.21.15.29:0/189618386 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c3413cad0 con 0x7f7c3415f3c0 2024-04-25T00:35:19.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c2bfff640 1 -- 172.21.15.29:0/189618386 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f7c2c0213f0 con 0x7f7c3415f3c0 2024-04-25T00:35:19.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c2bfff640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c14067a70 0x7f7c14069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c2bfff640 1 -- 172.21.15.29:0/189618386 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f7c2c086ea0 con 0x7f7c3415f3c0 2024-04-25T00:35:19.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.468+0000 7f7c32ffd640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c14067a70 0x7f7c14069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.472+0000 7f7c32ffd640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c14067a70 0x7f7c14069f30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7c240af000 tx=0x7f7c24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.472+0000 7f7c2bfff640 1 -- 172.21.15.29:0/189618386 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7c3413cad0 con 0x7f7c3415f3c0 2024-04-25T00:35:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.660+0000 7f7c39648640 1 -- 172.21.15.29:0/189618386 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7c3406e7b0 con 0x7f7c14067a70 2024-04-25T00:35:19.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.664+0000 7f7c2bfff640 1 -- 172.21.15.29:0/189618386 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f7c3406e7b0 con 0x7f7c14067a70 2024-04-25T00:35:19.665 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:35:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-04-25T00:35:19.665 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:19.665 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:35:19.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.664+0000 7f7c39648640 1 -- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c14067a70 msgr2=0x7f7c14069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.664+0000 7f7c39648640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c14067a70 0x7f7c14069f30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7c240af000 tx=0x7f7c24008040 comp rx=0 tx=0).stop 2024-04-25T00:35:19.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.664+0000 7f7c39648640 1 -- 172.21.15.29:0/189618386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3415f3c0 msgr2=0x7f7c340ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.664+0000 7f7c39648640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3415f3c0 0x7f7c340ce180 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00cd80 tx=0x7f7c2c00a650 comp rx=0 tx=0).stop 2024-04-25T00:35:19.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.664+0000 7f7c39648640 1 -- 172.21.15.29:0/189618386 shutdown_connections 2024-04-25T00:35:19.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.664+0000 7f7c39648640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c3415f3c0 0x7f7c340ce180 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.664+0000 7f7c39648640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3415e8d0 0x7f7c3413b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.664+0000 7f7c39648640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c14067a70 0x7f7c14069f30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.664+0000 7f7c39648640 1 --2- 172.21.15.29:0/189618386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c340b4780 0x7f7c340cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.664+0000 7f7c39648640 1 -- 172.21.15.29:0/189618386 >> 172.21.15.29:0/189618386 conn(0x7f7c340b1870 msgr2=0x7f7c340b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:19.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.664+0000 7f7c39648640 1 -- 172.21.15.29:0/189618386 shutdown_connections 2024-04-25T00:35:19.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.664+0000 7f7c39648640 1 -- 172.21.15.29:0/189618386 wait complete. 2024-04-25T00:35:19.684 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:35:19.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd953971640 1 Processor -- start 2024-04-25T00:35:19.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd953971640 1 -- start start 2024-04-25T00:35:19.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd953971640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd94c0b3c30 0x7fd94c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd953971640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd94c0b4b10 0x7fd94c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd953971640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd94c0b41e0 con 0x7fd94c0b3140 2024-04-25T00:35:19.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd953971640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd94c06d960 con 0x7fd94c0b3c30 2024-04-25T00:35:19.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd953971640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd94c06dc40 con 0x7fd94c0b4b10 2024-04-25T00:35:19.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd950f0c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd94c0b3c30 0x7fd94c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd950f0c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd94c0b3c30 0x7fd94c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46280/0 (socket says 172.21.15.29:46280) 2024-04-25T00:35:19.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd950f0c640 1 -- 172.21.15.29:0/2859093809 learned_addr learned my addr 172.21.15.29:0/2859093809 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:19.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd951f0e640 1 --2- 172.21.15.29:0/2859093809 >> v2:172.21.15.157:3300/0 conn(0x7fd94c0b4b10 0x7fd94c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd950f0c640 1 -- 172.21.15.29:0/2859093809 >> v2:172.21.15.157:3300/0 conn(0x7fd94c0b4b10 msgr2=0x7fd94c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:19.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd950f0c640 1 --2- 172.21.15.29:0/2859093809 >> v2:172.21.15.157:3300/0 conn(0x7fd94c0b4b10 0x7fd94c0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd950f0c640 1 -- 172.21.15.29:0/2859093809 >> v1:172.21.15.157:6789/0 conn(0x7fd94c0b3140 legacy=0x7fd94c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:19.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd950f0c640 1 -- 172.21.15.29:0/2859093809 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c06df20 con 0x7fd94c0b3c30 2024-04-25T00:35:19.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.812+0000 7fd951f0e640 1 --2- 172.21.15.29:0/2859093809 >> v2:172.21.15.157:3300/0 conn(0x7fd94c0b4b10 0x7fd94c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:19.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd950f0c640 1 --2- 172.21.15.29:0/2859093809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd94c0b3c30 0x7fd94c06e440 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009870 tx=0x7fd93c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806e72c8a4ac16bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd943fff640 1 -- 172.21.15.29:0/2859093809 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0b0070 con 0x7fd94c0b3c30 2024-04-25T00:35:19.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd943fff640 1 -- 172.21.15.29:0/2859093809 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd93c0ab600 con 0x7fd94c0b3c30 2024-04-25T00:35:19.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd943fff640 1 -- 172.21.15.29:0/2859093809 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0ab9d0 con 0x7fd94c0b3c30 2024-04-25T00:35:19.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 -- 172.21.15.29:0/2859093809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd94c0b3c30 msgr2=0x7fd94c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 --2- 172.21.15.29:0/2859093809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd94c0b3c30 0x7fd94c06e440 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009870 tx=0x7fd93c01c4e0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 -- 172.21.15.29:0/2859093809 shutdown_connections 2024-04-25T00:35:19.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 --2- 172.21.15.29:0/2859093809 >> v2:172.21.15.157:3300/0 conn(0x7fd94c0b4b10 0x7fd94c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 --2- 172.21.15.29:0/2859093809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd94c0b3c30 0x7fd94c06e440 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 -- 172.21.15.29:0/2859093809 >> 172.21.15.29:0/2859093809 conn(0x7fd94c0b1870 msgr2=0x7fd94c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:19.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 -- 172.21.15.29:0/2859093809 shutdown_connections 2024-04-25T00:35:19.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 -- 172.21.15.29:0/2859093809 wait complete. 2024-04-25T00:35:19.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 Processor -- start 2024-04-25T00:35:19.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 -- start start 2024-04-25T00:35:19.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c0b3140 0x7fd94c156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd94c0b3c30 0x7fd94c139170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0b4b10 0x7fd94c13a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c168d80 con 0x7fd94c0b3c30 2024-04-25T00:35:19.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c168c00 con 0x7fd94c0b3140 2024-04-25T00:35:19.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd953971640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd94c168f00 con 0x7fd94c0b4b10 2024-04-25T00:35:19.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd950f0c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd94c0b3c30 0x7fd94c139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd950f0c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd94c0b3c30 0x7fd94c139170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46282/0 (socket says 172.21.15.29:46282) 2024-04-25T00:35:19.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.816+0000 7fd950f0c640 1 -- 172.21.15.29:0/3574325211 learned_addr learned my addr 172.21.15.29:0/3574325211 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:19.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd951f0e640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0b4b10 0x7fd94c13a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd95170d640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c0b3140 0x7fd94c156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd950f0c640 1 -- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0b4b10 msgr2=0x7fd94c13a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd950f0c640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0b4b10 0x7fd94c13a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd950f0c640 1 -- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c0b3140 msgr2=0x7fd94c156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd950f0c640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c0b3140 0x7fd94c156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd950f0c640 1 -- 172.21.15.29:0/3574325211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c157020 con 0x7fd94c0b3c30 2024-04-25T00:35:19.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd950f0c640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd94c0b3c30 0x7fd94c139170 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0ae040 tx=0x7fd93c01ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd941ffb640 1 -- 172.21.15.29:0/3574325211 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0ba020 con 0x7fd94c0b3c30 2024-04-25T00:35:19.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd941ffb640 1 -- 172.21.15.29:0/3574325211 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd93c0b6640 con 0x7fd94c0b3c30 2024-04-25T00:35:19.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd953971640 1 -- 172.21.15.29:0/3574325211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c157300 con 0x7fd94c0b3c30 2024-04-25T00:35:19.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd941ffb640 1 -- 172.21.15.29:0/3574325211 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0b0040 con 0x7fd94c0b3c30 2024-04-25T00:35:19.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd953971640 1 -- 172.21.15.29:0/3574325211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c13af90 con 0x7fd94c0b3c30 2024-04-25T00:35:19.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd953971640 1 -- 172.21.15.29:0/3574325211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd94c069270 con 0x7fd94c0b3c30 2024-04-25T00:35:19.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd941ffb640 1 -- 172.21.15.29:0/3574325211 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fd93c0bd020 con 0x7fd94c0b3c30 2024-04-25T00:35:19.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd941ffb640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd930067aa0 0x7fd930069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:19.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd941ffb640 1 -- 172.21.15.29:0/3574325211 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fd93c120290 con 0x7fd94c0b3c30 2024-04-25T00:35:19.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd95170d640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd930067aa0 0x7fd930069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:19.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.820+0000 7fd95170d640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd930067aa0 0x7fd930069f60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd94800b810 tx=0x7fd948008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:19.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.824+0000 7fd941ffb640 1 -- 172.21.15.29:0/3574325211 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd93c0c4030 con 0x7fd94c0b3c30 2024-04-25T00:35:19.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.988+0000 7fd953971640 1 -- 172.21.15.29:0/3574325211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd94c159220 con 0x7fd94c0b3c30 2024-04-25T00:35:19.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.988+0000 7fd941ffb640 1 -- 172.21.15.29:0/3574325211 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fd93c0160d0 con 0x7fd94c0b3c30 2024-04-25T00:35:19.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.992+0000 7fd953971640 1 -- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd930067aa0 msgr2=0x7fd930069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.992+0000 7fd953971640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd930067aa0 0x7fd930069f60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd94800b810 tx=0x7fd948008040 comp rx=0 tx=0).stop 2024-04-25T00:35:19.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.992+0000 7fd953971640 1 -- 172.21.15.29:0/3574325211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd94c0b3c30 msgr2=0x7fd94c139170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:19.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.992+0000 7fd953971640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd94c0b3c30 0x7fd94c139170 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0ae040 tx=0x7fd93c01ad20 comp rx=0 tx=0).stop 2024-04-25T00:35:19.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.992+0000 7fd953971640 1 -- 172.21.15.29:0/3574325211 shutdown_connections 2024-04-25T00:35:19.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.992+0000 7fd953971640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0b4b10 0x7fd94c13a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.992+0000 7fd953971640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd94c0b3c30 0x7fd94c139170 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.992+0000 7fd953971640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd930067aa0 0x7fd930069f60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.992+0000 7fd953971640 1 --2- 172.21.15.29:0/3574325211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c0b3140 0x7fd94c156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:19.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.992+0000 7fd953971640 1 -- 172.21.15.29:0/3574325211 >> 172.21.15.29:0/3574325211 conn(0x7fd94c0b1870 msgr2=0x7fd94c1409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:19.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.992+0000 7fd953971640 1 -- 172.21.15.29:0/3574325211 shutdown_connections 2024-04-25T00:35:19.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:19.992+0000 7fd953971640 1 -- 172.21.15.29:0/3574325211 wait complete. 2024-04-25T00:35:20.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:20.011 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol62000) 2024-04-25T00:35:20.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:20.033 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-25T00:35:20.087 INFO:teuthology.orchestra.run.smithi029.stdout:cephfs_data 2024-04-25T00:35:20.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:20.088 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol62000) 2024-04-25T00:35:20.153 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol54001 2024-04-25T00:35:20.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.328+0000 7fb58599e640 1 Processor -- start 2024-04-25T00:35:20.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.328+0000 7fb58599e640 1 -- start start 2024-04-25T00:35:20.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.328+0000 7fb58599e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015e8d0 0x7fb5800cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:20.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.328+0000 7fb58599e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5800b3720 con 0x7fb58015f3c0 2024-04-25T00:35:20.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.328+0000 7fb58599e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5800b39c0 con 0x7fb5800b4780 2024-04-25T00:35:20.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.328+0000 7fb58599e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb5800b42f0 con 0x7fb58015e8d0 2024-04-25T00:35:20.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.328+0000 7fb57f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb58015f3c0 0x7fb58015c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:47818/0 (socket says 172.21.15.29:47818) 2024-04-25T00:35:20.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.328+0000 7fb57f7fe640 1 -- 172.21.15.29:0/1688756668 learned_addr learned my addr 172.21.15.29:0/1688756668 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:20.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.328+0000 7fb57effd640 1 --2- 172.21.15.29:0/1688756668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015e8d0 0x7fb5800cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:20.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.328+0000 7fb57effd640 1 -- 172.21.15.29:0/1688756668 >> v1:172.21.15.157:6789/0 conn(0x7fb5800b4780 legacy=0x7fb58015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:20.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.328+0000 7fb57effd640 1 -- 172.21.15.29:0/1688756668 >> v1:172.21.15.29:6789/0 conn(0x7fb58015f3c0 legacy=0x7fb58015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:20.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.328+0000 7fb57effd640 1 -- 172.21.15.29:0/1688756668 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5800b3ca0 con 0x7fb58015e8d0 2024-04-25T00:35:20.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.332+0000 7fb57effd640 1 --2- 172.21.15.29:0/1688756668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015e8d0 0x7fb5800cc240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb570009dc0 tx=0x7fb57001c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee6a583a55e7371a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:20.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.332+0000 7fb57dffb640 1 -- 172.21.15.29:0/1688756668 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5700b2070 con 0x7fb58015e8d0 2024-04-25T00:35:20.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.332+0000 7fb57dffb640 1 -- 172.21.15.29:0/1688756668 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5700a5930 con 0x7fb58015e8d0 2024-04-25T00:35:20.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.332+0000 7fb57dffb640 1 -- 172.21.15.29:0/1688756668 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5700a5d00 con 0x7fb58015e8d0 2024-04-25T00:35:20.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 -- 172.21.15.29:0/1688756668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015e8d0 msgr2=0x7fb5800cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:20.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 --2- 172.21.15.29:0/1688756668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015e8d0 0x7fb5800cc240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb570009dc0 tx=0x7fb57001c960 comp rx=0 tx=0).stop 2024-04-25T00:35:20.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 -- 172.21.15.29:0/1688756668 shutdown_connections 2024-04-25T00:35:20.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 --2- 172.21.15.29:0/1688756668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015e8d0 0x7fb5800cc240 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:20.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 -- 172.21.15.29:0/1688756668 >> 172.21.15.29:0/1688756668 conn(0x7fb5800b1870 msgr2=0x7fb5800b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:20.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 -- 172.21.15.29:0/1688756668 shutdown_connections 2024-04-25T00:35:20.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 -- 172.21.15.29:0/1688756668 wait complete. 2024-04-25T00:35:20.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 Processor -- start 2024-04-25T00:35:20.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 -- start start 2024-04-25T00:35:20.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5800b4780 0x7fb5801573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:20.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb58015e8d0 0x7fb580158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:20.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015f3c0 0x7fb5801593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:20.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb580168e00 con 0x7fb58015e8d0 2024-04-25T00:35:20.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb580168c80 con 0x7fb5800b4780 2024-04-25T00:35:20.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb580168f80 con 0x7fb58015f3c0 2024-04-25T00:35:20.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb57e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb58015e8d0 0x7fb580158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:20.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb57e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb58015e8d0 0x7fb580158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46296/0 (socket says 172.21.15.29:46296) 2024-04-25T00:35:20.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb57e7fc640 1 -- 172.21.15.29:0/1301816109 learned_addr learned my addr 172.21.15.29:0/1301816109 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:20.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb57effd640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5800b4780 0x7fb5801573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:20.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb57e7fc640 1 -- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015f3c0 msgr2=0x7fb5801593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:20.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb57e7fc640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015f3c0 0x7fb5801593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:20.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb57e7fc640 1 -- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5800b4780 msgr2=0x7fb5801573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:20.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb57e7fc640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5800b4780 0x7fb5801573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:20.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb57e7fc640 1 -- 172.21.15.29:0/1301816109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb580157cb0 con 0x7fb58015e8d0 2024-04-25T00:35:20.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb57effd640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5800b4780 0x7fb5801573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:20.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb57e7fc640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb58015e8d0 0x7fb580158e60 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb57400c500 tx=0x7fb57400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:20.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb567fff640 1 -- 172.21.15.29:0/1301816109 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574016020 con 0x7fb58015e8d0 2024-04-25T00:35:20.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb567fff640 1 -- 172.21.15.29:0/1301816109 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb574005220 con 0x7fb58015e8d0 2024-04-25T00:35:20.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 -- 172.21.15.29:0/1301816109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb580157f90 con 0x7fb58015e8d0 2024-04-25T00:35:20.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb567fff640 1 -- 172.21.15.29:0/1301816109 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574007030 con 0x7fb58015e8d0 2024-04-25T00:35:20.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.336+0000 7fb58599e640 1 -- 172.21.15.29:0/1301816109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb580158550 con 0x7fb58015e8d0 2024-04-25T00:35:20.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.340+0000 7fb58599e640 1 -- 172.21.15.29:0/1301816109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb580069270 con 0x7fb58015e8d0 2024-04-25T00:35:20.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.340+0000 7fb567fff640 1 -- 172.21.15.29:0/1301816109 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fb5740071f0 con 0x7fb58015e8d0 2024-04-25T00:35:20.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.340+0000 7fb567fff640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb56c067a70 0x7fb56c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:20.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.340+0000 7fb567fff640 1 -- 172.21.15.29:0/1301816109 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb574085a60 con 0x7fb58015e8d0 2024-04-25T00:35:20.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.340+0000 7fb57effd640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb56c067a70 0x7fb56c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:20.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.340+0000 7fb57effd640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb56c067a70 0x7fb56c069f30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb5700af000 tx=0x7fb5700a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:20.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.344+0000 7fb567fff640 1 -- 172.21.15.29:0/1301816109 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb574020020 con 0x7fb58015e8d0 2024-04-25T00:35:20.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.528+0000 7fb58599e640 1 -- 172.21.15.29:0/1301816109 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol54001", "target": ["mon-mgr", ""]}) -- 0x7fb5800cc170 con 0x7fb56c067a70 2024-04-25T00:35:20.540 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:20.540+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:20.540 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:20.540+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:20.540 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:20.540+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:20.541 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:20.540+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:20.541 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:20.540+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.600+0000 7fb567fff640 1 -- 172.21.15.29:0/1301816109 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb5800cc170 con 0x7fb56c067a70 2024-04-25T00:35:20.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.600+0000 7fb58599e640 1 -- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb56c067a70 msgr2=0x7fb56c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.600+0000 7fb58599e640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb56c067a70 0x7fb56c069f30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb5700af000 tx=0x7fb5700a7070 comp rx=0 tx=0).stop 2024-04-25T00:35:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.600+0000 7fb58599e640 1 -- 172.21.15.29:0/1301816109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb58015e8d0 msgr2=0x7fb580158e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.600+0000 7fb58599e640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb58015e8d0 0x7fb580158e60 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb57400c500 tx=0x7fb57400c9c0 comp rx=0 tx=0).stop 2024-04-25T00:35:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.600+0000 7fb58599e640 1 -- 172.21.15.29:0/1301816109 shutdown_connections 2024-04-25T00:35:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.600+0000 7fb58599e640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015f3c0 0x7fb5801593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.600+0000 7fb58599e640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb58015e8d0 0x7fb580158e60 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.600+0000 7fb58599e640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb56c067a70 0x7fb56c069f30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.600+0000 7fb58599e640 1 --2- 172.21.15.29:0/1301816109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5800b4780 0x7fb5801573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.600+0000 7fb58599e640 1 -- 172.21.15.29:0/1301816109 >> 172.21.15.29:0/1301816109 conn(0x7fb5800b1870 msgr2=0x7fb5800b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.604+0000 7fb58599e640 1 -- 172.21.15.29:0/1301816109 shutdown_connections 2024-04-25T00:35:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:20.604+0000 7fb58599e640 1 -- 172.21.15.29:0/1301816109 wait complete. 2024-04-25T00:35:20.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:20.624 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol54001/.meta ./volumes/_nogroup/subvol62000) 2024-04-25T00:35:22.811 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol62000 2024-04-25T00:35:22.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.948+0000 7fbf8aa46640 1 Processor -- start 2024-04-25T00:35:22.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.948+0000 7fbf8aa46640 1 -- start start 2024-04-25T00:35:22.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf8aa46640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf840b3140 0x7fbf8406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:22.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf8aa46640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf840b4b10 0x7fbf840b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:22.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf8aa46640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf840b3a60 0x7fbf8415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:22.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf8aa46640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbf840b41e0 con 0x7fbf840b3a60 2024-04-25T00:35:22.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf8aa46640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbf8406d960 con 0x7fbf840b3140 2024-04-25T00:35:22.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf8aa46640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8406dc40 con 0x7fbf840b4b10 2024-04-25T00:35:22.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf88fe3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf840b3a60 0x7fbf8415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:22.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf88fe3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf840b3a60 0x7fbf8415c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46308/0 (socket says 172.21.15.29:46308) 2024-04-25T00:35:22.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf88fe3640 1 -- 172.21.15.29:0/3466177162 learned_addr learned my addr 172.21.15.29:0/3466177162 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:22.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf83fff640 1 --2- 172.21.15.29:0/3466177162 >> v2:172.21.15.157:3300/0 conn(0x7fbf840b3140 0x7fbf8406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:22.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf837fe640 1 --2- 172.21.15.29:0/3466177162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf840b4b10 0x7fbf840b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:22.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf88fe3640 1 -- 172.21.15.29:0/3466177162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf840b4b10 msgr2=0x7fbf840b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:22.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf88fe3640 1 --2- 172.21.15.29:0/3466177162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf840b4b10 0x7fbf840b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:22.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf88fe3640 1 -- 172.21.15.29:0/3466177162 >> v2:172.21.15.157:3300/0 conn(0x7fbf840b3140 msgr2=0x7fbf8406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:22.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf88fe3640 1 --2- 172.21.15.29:0/3466177162 >> v2:172.21.15.157:3300/0 conn(0x7fbf840b3140 0x7fbf8406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:22.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf88fe3640 1 -- 172.21.15.29:0/3466177162 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8406df20 con 0x7fbf840b3a60 2024-04-25T00:35:22.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.952+0000 7fbf837fe640 1 --2- 172.21.15.29:0/3466177162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf840b4b10 0x7fbf840b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:22.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf88fe3640 1 --2- 172.21.15.29:0/3466177162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf840b3a60 0x7fbf8415c3e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fbf840d0bf0 tx=0x7fbf7c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31092c5b5e5a5a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:22.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf82ffd640 1 -- 172.21.15.29:0/3466177162 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0a5670 con 0x7fbf840b3a60 2024-04-25T00:35:22.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf82ffd640 1 -- 172.21.15.29:0/3466177162 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7c0a5810 con 0x7fbf840b3a60 2024-04-25T00:35:22.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf82ffd640 1 -- 172.21.15.29:0/3466177162 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0a5be0 con 0x7fbf840b3a60 2024-04-25T00:35:22.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf8aa46640 1 -- 172.21.15.29:0/3466177162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf840b3a60 msgr2=0x7fbf8415c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:22.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf8aa46640 1 --2- 172.21.15.29:0/3466177162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf840b3a60 0x7fbf8415c3e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fbf840d0bf0 tx=0x7fbf7c01c9f0 comp rx=0 tx=0).stop 2024-04-25T00:35:22.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf8aa46640 1 -- 172.21.15.29:0/3466177162 shutdown_connections 2024-04-25T00:35:22.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf8aa46640 1 --2- 172.21.15.29:0/3466177162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf840b3a60 0x7fbf8415c3e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:22.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf8aa46640 1 --2- 172.21.15.29:0/3466177162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf840b4b10 0x7fbf840b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:22.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf8aa46640 1 --2- 172.21.15.29:0/3466177162 >> v2:172.21.15.157:3300/0 conn(0x7fbf840b3140 0x7fbf8406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:22.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf8aa46640 1 -- 172.21.15.29:0/3466177162 >> 172.21.15.29:0/3466177162 conn(0x7fbf840b1870 msgr2=0x7fbf840b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:22.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf8aa46640 1 -- 172.21.15.29:0/3466177162 shutdown_connections 2024-04-25T00:35:22.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf8aa46640 1 -- 172.21.15.29:0/3466177162 wait complete. 2024-04-25T00:35:22.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf8aa46640 1 Processor -- start 2024-04-25T00:35:22.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.956+0000 7fbf8aa46640 1 -- start start 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf8aa46640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf840b3140 0x7fbf84158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf8aa46640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf840b3a60 0x7fbf84139470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf8aa46640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf840b4b10 0x7fbf841580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf8aa46640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbf840ce3c0 con 0x7fbf840b3140 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf8aa46640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf840ce240 con 0x7fbf840b3a60 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf8aa46640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbf840ce540 con 0x7fbf840b4b10 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf83fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf840b3140 0x7fbf84158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf83fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf840b3140 0x7fbf84158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46312/0 (socket says 172.21.15.29:46312) 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf83fff640 1 -- 172.21.15.29:0/1682101414 learned_addr learned my addr 172.21.15.29:0/1682101414 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf837fe640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf840b3a60 0x7fbf84139470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf88fe3640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf840b4b10 0x7fbf841580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf83fff640 1 -- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf840b4b10 msgr2=0x7fbf841580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf83fff640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf840b4b10 0x7fbf841580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:22.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf83fff640 1 -- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf840b3a60 msgr2=0x7fbf84139470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:22.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf83fff640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf840b3a60 0x7fbf84139470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:22.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf83fff640 1 -- 172.21.15.29:0/1682101414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf84158ee0 con 0x7fbf840b3140 2024-04-25T00:35:22.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf88fe3640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf840b4b10 0x7fbf841580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:22.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf83fff640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf840b3140 0x7fbf84158870 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400d830 tx=0x7fbf7400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:22.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf80ff9640 1 -- 172.21.15.29:0/1682101414 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74018020 con 0x7fbf840b3140 2024-04-25T00:35:22.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf80ff9640 1 -- 172.21.15.29:0/1682101414 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf740096a0 con 0x7fbf840b3140 2024-04-25T00:35:22.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf80ff9640 1 -- 172.21.15.29:0/1682101414 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74009a50 con 0x7fbf840b3140 2024-04-25T00:35:22.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf8aa46640 1 -- 172.21.15.29:0/1682101414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf841591c0 con 0x7fbf840b3140 2024-04-25T00:35:22.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf8aa46640 1 -- 172.21.15.29:0/1682101414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf84159780 con 0x7fbf840b3140 2024-04-25T00:35:22.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf8aa46640 1 -- 172.21.15.29:0/1682101414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf84069270 con 0x7fbf840b3140 2024-04-25T00:35:22.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.960+0000 7fbf80ff9640 1 -- 172.21.15.29:0/1682101414 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7fbf7401b020 con 0x7fbf840b3140 2024-04-25T00:35:22.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.964+0000 7fbf80ff9640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbf70067b00 0x7fbf70069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:22.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.964+0000 7fbf80ff9640 1 -- 172.21.15.29:0/1682101414 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fbf740868d0 con 0x7fbf840b3140 2024-04-25T00:35:22.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.964+0000 7fbf837fe640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbf70067b00 0x7fbf70069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:22.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.964+0000 7fbf837fe640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbf70067b00 0x7fbf70069fc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbf78006fd0 tx=0x7fbf78008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:22.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:22.968+0000 7fbf80ff9640 1 -- 172.21.15.29:0/1682101414 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbf74022020 con 0x7fbf840b3140 2024-04-25T00:35:23.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.156+0000 7fbf8aa46640 1 -- 172.21.15.29:0/1682101414 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol62000", "target": ["mon-mgr", ""]}) -- 0x7fbf8413bbe0 con 0x7fbf70067b00 2024-04-25T00:35:23.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.224+0000 7fbf80ff9640 1 -- 172.21.15.29:0/1682101414 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fbf8413bbe0 con 0x7fbf70067b00 2024-04-25T00:35:23.227 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/_nogroup/subvol62000 2024-04-25T00:35:23.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.224+0000 7fbf8aa46640 1 -- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbf70067b00 msgr2=0x7fbf70069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:23.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.224+0000 7fbf8aa46640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbf70067b00 0x7fbf70069fc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbf78006fd0 tx=0x7fbf78008040 comp rx=0 tx=0).stop 2024-04-25T00:35:23.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.224+0000 7fbf8aa46640 1 -- 172.21.15.29:0/1682101414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf840b3140 msgr2=0x7fbf84158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:23.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.224+0000 7fbf8aa46640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf840b3140 0x7fbf84158870 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400d830 tx=0x7fbf7400dcf0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.224+0000 7fbf8aa46640 1 -- 172.21.15.29:0/1682101414 shutdown_connections 2024-04-25T00:35:23.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.224+0000 7fbf8aa46640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf840b4b10 0x7fbf841580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.224+0000 7fbf8aa46640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbf70067b00 0x7fbf70069fc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.224+0000 7fbf8aa46640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf840b3a60 0x7fbf84139470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.224+0000 7fbf8aa46640 1 --2- 172.21.15.29:0/1682101414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf840b3140 0x7fbf84158870 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.224+0000 7fbf8aa46640 1 -- 172.21.15.29:0/1682101414 >> 172.21.15.29:0/1682101414 conn(0x7fbf840b1870 msgr2=0x7fbf84144bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:23.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.228+0000 7fbf8aa46640 1 -- 172.21.15.29:0/1682101414 shutdown_connections 2024-04-25T00:35:23.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.228+0000 7fbf8aa46640 1 -- 172.21.15.29:0/1682101414 wait complete. 2024-04-25T00:35:23.248 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol62000 2024-04-25T00:35:23.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.364+0000 7fd03ce5e640 1 Processor -- start 2024-04-25T00:35:23.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd03ce5e640 1 -- start start 2024-04-25T00:35:23.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd03ce5e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd03815d980 0x7fd03806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:23.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd03ce5e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd0380b2b70 0x7fd03806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:23.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd03ce5e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0380b3130 con 0x7fd038150df0 2024-04-25T00:35:23.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd03ce5e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd0380cf560 con 0x7fd0380b2b70 2024-04-25T00:35:23.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd03ce5e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd038151990 con 0x7fd03815d980 2024-04-25T00:35:23.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd035d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd0380b2b70 0x7fd03806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:23.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd035d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd0380b2b70 0x7fd03806db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46320/0 (socket says 172.21.15.29:46320) 2024-04-25T00:35:23.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd035d9b640 1 -- 172.21.15.29:0/275124454 learned_addr learned my addr 172.21.15.29:0/275124454 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:23.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd03659c640 1 --2- 172.21.15.29:0/275124454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd03815d980 0x7fd03806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:23.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd035d9b640 1 -- 172.21.15.29:0/275124454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd03815d980 msgr2=0x7fd03806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:23.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd035d9b640 1 --2- 172.21.15.29:0/275124454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd03815d980 0x7fd03806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd035d9b640 1 -- 172.21.15.29:0/275124454 >> v1:172.21.15.157:6789/0 conn(0x7fd038150df0 legacy=0x7fd038151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:23.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.368+0000 7fd035d9b640 1 -- 172.21.15.29:0/275124454 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0380b3410 con 0x7fd0380b2b70 2024-04-25T00:35:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd035d9b640 1 --2- 172.21.15.29:0/275124454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0380b2b70 0x7fd03806db10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00bb10 tx=0x7fd02c01ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c42a5fe8636fec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd03559a640 1 -- 172.21.15.29:0/275124454 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0b4070 con 0x7fd0380b2b70 2024-04-25T00:35:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd03559a640 1 -- 172.21.15.29:0/275124454 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd02c0a7c50 con 0x7fd0380b2b70 2024-04-25T00:35:23.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd03559a640 1 -- 172.21.15.29:0/275124454 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0af440 con 0x7fd0380b2b70 2024-04-25T00:35:23.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd03ce5e640 1 -- 172.21.15.29:0/275124454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0380b2b70 msgr2=0x7fd03806db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:23.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd03ce5e640 1 --2- 172.21.15.29:0/275124454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0380b2b70 0x7fd03806db10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00bb10 tx=0x7fd02c01ec00 comp rx=0 tx=0).stop 2024-04-25T00:35:23.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd03ce5e640 1 -- 172.21.15.29:0/275124454 shutdown_connections 2024-04-25T00:35:23.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd03ce5e640 1 --2- 172.21.15.29:0/275124454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0380b2b70 0x7fd03806db10 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd03ce5e640 1 --2- 172.21.15.29:0/275124454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd03815d980 0x7fd03806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd03ce5e640 1 -- 172.21.15.29:0/275124454 >> 172.21.15.29:0/275124454 conn(0x7fd0380b1870 msgr2=0x7fd0380b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:23.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd03ce5e640 1 -- 172.21.15.29:0/275124454 shutdown_connections 2024-04-25T00:35:23.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd03ce5e640 1 -- 172.21.15.29:0/275124454 wait complete. 2024-04-25T00:35:23.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd03ce5e640 1 Processor -- start 2024-04-25T00:35:23.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.372+0000 7fd03ce5e640 1 -- start start 2024-04-25T00:35:23.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd03ce5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0380b2b70 0x7fd03814baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:23.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd03ce5e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd038150df0 0x7fd03814db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:23.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd03ce5e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd03815d980 0x7fd038143f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:23.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd03ce5e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd03815b4b0 con 0x7fd03815d980 2024-04-25T00:35:23.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd03ce5e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd03815dd60 con 0x7fd0380b2b70 2024-04-25T00:35:23.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd03ce5e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd03815b630 con 0x7fd038150df0 2024-04-25T00:35:23.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd036d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd03815d980 0x7fd038143f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:23.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd036d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd03815d980 0x7fd038143f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46326/0 (socket says 172.21.15.29:46326) 2024-04-25T00:35:23.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd036d9d640 1 -- 172.21.15.29:0/375317155 learned_addr learned my addr 172.21.15.29:0/375317155 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:23.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd03659c640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0380b2b70 0x7fd03814baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:23.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd035d9b640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd038150df0 0x7fd03814db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:23.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd036d9d640 1 -- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd038150df0 msgr2=0x7fd03814db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:23.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd036d9d640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd038150df0 0x7fd03814db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd036d9d640 1 -- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0380b2b70 msgr2=0x7fd03814baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:23.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd036d9d640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0380b2b70 0x7fd03814baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd036d9d640 1 -- 172.21.15.29:0/375317155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd038070d80 con 0x7fd03815d980 2024-04-25T00:35:23.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd036d9d640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd03815d980 0x7fd038143f80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd03000c770 tx=0x7fd03000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:23.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd026ffd640 1 -- 172.21.15.29:0/375317155 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030003b80 con 0x7fd03815d980 2024-04-25T00:35:23.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd026ffd640 1 -- 172.21.15.29:0/375317155 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd030003d20 con 0x7fd03815d980 2024-04-25T00:35:23.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd03ce5e640 1 -- 172.21.15.29:0/375317155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03814c130 con 0x7fd03815d980 2024-04-25T00:35:23.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd026ffd640 1 -- 172.21.15.29:0/375317155 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03000aa90 con 0x7fd03815d980 2024-04-25T00:35:23.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd03ce5e640 1 -- 172.21.15.29:0/375317155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03814fa30 con 0x7fd03815d980 2024-04-25T00:35:23.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd03ce5e640 1 -- 172.21.15.29:0/375317155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd038069270 con 0x7fd03815d980 2024-04-25T00:35:23.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd026ffd640 1 -- 172.21.15.29:0/375317155 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7fd03000ac30 con 0x7fd03815d980 2024-04-25T00:35:23.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd026ffd640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd018067b00 0x7fd018069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:23.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd026ffd640 1 -- 172.21.15.29:0/375317155 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fd0300860e0 con 0x7fd03815d980 2024-04-25T00:35:23.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.376+0000 7fd03659c640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd018067b00 0x7fd018069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:23.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.380+0000 7fd03659c640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd018067b00 0x7fd018069fc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd0380aeb20 tx=0x7fd028008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:23.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.380+0000 7fd026ffd640 1 -- 172.21.15.29:0/375317155 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd030056820 con 0x7fd03815d980 2024-04-25T00:35:23.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.568+0000 7fd03ce5e640 1 -- 172.21.15.29:0/375317155 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol62000", "target": ["mon-mgr", ""]}) -- 0x7fd0380cf910 con 0x7fd018067b00 2024-04-25T00:35:23.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.584+0000 7fd026ffd640 1 -- 172.21.15.29:0/375317155 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fd0380cf910 con 0x7fd018067b00 2024-04-25T00:35:23.585 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/_nogroup/subvol62000 2024-04-25T00:35:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.584+0000 7fd03ce5e640 1 -- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd018067b00 msgr2=0x7fd018069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.584+0000 7fd03ce5e640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd018067b00 0x7fd018069fc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd0380aeb20 tx=0x7fd028008040 comp rx=0 tx=0).stop 2024-04-25T00:35:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.584+0000 7fd03ce5e640 1 -- 172.21.15.29:0/375317155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd03815d980 msgr2=0x7fd038143f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.584+0000 7fd03ce5e640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd03815d980 0x7fd038143f80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd03000c770 tx=0x7fd03000cc30 comp rx=0 tx=0).stop 2024-04-25T00:35:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.584+0000 7fd03ce5e640 1 -- 172.21.15.29:0/375317155 shutdown_connections 2024-04-25T00:35:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.584+0000 7fd03ce5e640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd03815d980 0x7fd038143f80 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.584+0000 7fd03ce5e640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd038150df0 0x7fd03814db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.584+0000 7fd03ce5e640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd018067b00 0x7fd018069fc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.584+0000 7fd03ce5e640 1 --2- 172.21.15.29:0/375317155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0380b2b70 0x7fd03814baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.584+0000 7fd03ce5e640 1 -- 172.21.15.29:0/375317155 >> 172.21.15.29:0/375317155 conn(0x7fd0380b1870 msgr2=0x7fd03814cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.584+0000 7fd03ce5e640 1 -- 172.21.15.29:0/375317155 shutdown_connections 2024-04-25T00:35:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:23.584+0000 7fd03ce5e640 1 -- 172.21.15.29:0/375317155 wait complete. 2024-04-25T00:35:23.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:23.604 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/e5bc512bc365e911c4300cd94f85bf79.meta) 2024-04-25T00:35:27.801 INFO:teuthology.orchestra.run.smithi029.stdout:[GLOBAL] 2024-04-25T00:35:27.801 INFO:teuthology.orchestra.run.smithi029.stdout:version = 1 2024-04-25T00:35:27.801 INFO:teuthology.orchestra.run.smithi029.stdout:type = subvolume 2024-04-25T00:35:27.802 INFO:teuthology.orchestra.run.smithi029.stdout:path = /volumes/_nogroup/subvol62000 2024-04-25T00:35:27.802 INFO:teuthology.orchestra.run.smithi029.stdout:state = complete 2024-04-25T00:35:27.802 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:27.803 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol62000 alice 2024-04-25T00:35:27.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892b4bf640 1 Processor -- start 2024-04-25T00:35:27.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892b4bf640 1 -- start start 2024-04-25T00:35:27.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892b4bf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89240b2b70 0x7f892415b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892b4bf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f892406dce0 0x7f892415b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892b4bf640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89240b40a0 con 0x7f8924151640 2024-04-25T00:35:27.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892b4bf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f89240b3000 con 0x7f892406dce0 2024-04-25T00:35:27.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892b4bf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f89240b32e0 con 0x7f89240b2b70 2024-04-25T00:35:27.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892925b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8924151640 0x7f892406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55068/0 (socket says 172.21.15.29:55068) 2024-04-25T00:35:27.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892925b640 1 -- 172.21.15.29:0/2850158147 learned_addr learned my addr 172.21.15.29:0/2850158147 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:27.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f8928a5a640 1 --2- 172.21.15.29:0/2850158147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89240b2b70 0x7f892415b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f8929a5c640 1 --2- 172.21.15.29:0/2850158147 >> v2:172.21.15.157:3300/0 conn(0x7f892406dce0 0x7f892415b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2663626123 0 0) 0x7f89240b40a0 con 0x7f8924151640 2024-04-25T00:35:27.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8910003530 con 0x7f8924151640 2024-04-25T00:35:27.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2522683213 0 0) 0x7f8910003530 con 0x7f8924151640 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89240b2b70 msgr2=0x7f892415b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 --2- 172.21.15.29:0/2850158147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89240b2b70 0x7f892415b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 >> v2:172.21.15.157:3300/0 conn(0x7f892406dce0 msgr2=0x7f892415b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 --2- 172.21.15.29:0/2850158147 >> v2:172.21.15.157:3300/0 conn(0x7f892406dce0 0x7f892415b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89240b35c0 con 0x7f8924151640 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8920001680 con 0x7f8924151640 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 >> v1:172.21.15.29:6789/0 conn(0x7f8924151640 legacy=0x7f892406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 --2- 172.21.15.29:0/2850158147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89100039c0 0x7f8910003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 --2- 172.21.15.29:0/2850158147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8910004810 0x7f8910004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 --2- 172.21.15.29:0/2850158147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89100051d0 0x7f891000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f89240b3000 con 0x7f89100039c0 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89240b32e0 con 0x7f8910004810 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f891000bf10 con 0x7f89100051d0 2024-04-25T00:35:27.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892925b640 1 --2- 172.21.15.29:0/2850158147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89100039c0 0x7f8910003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f8928a5a640 1 --2- 172.21.15.29:0/2850158147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8910004810 0x7f8910004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f8929a5c640 1 --2- 172.21.15.29:0/2850158147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89100051d0 0x7f891000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892925b640 1 -- 172.21.15.29:0/2850158147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89100051d0 msgr2=0x7f891000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:27.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892925b640 1 --2- 172.21.15.29:0/2850158147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89100051d0 0x7f891000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892925b640 1 -- 172.21.15.29:0/2850158147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8910004810 msgr2=0x7f8910004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:27.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892925b640 1 --2- 172.21.15.29:0/2850158147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8910004810 0x7f8910004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.944+0000 7f892925b640 1 -- 172.21.15.29:0/2850158147 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8910003530 con 0x7f89100039c0 2024-04-25T00:35:27.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.948+0000 7f892925b640 1 --2- 172.21.15.29:0/2850158147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89100039c0 0x7f8910003de0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f892000c210 tx=0x7f892001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4094717420f8705f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:27.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.948+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920007d60 con 0x7f89100039c0 2024-04-25T00:35:27.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.948+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89200a9020 con 0x7f89100039c0 2024-04-25T00:35:27.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.948+0000 7f891bfff640 1 -- 172.21.15.29:0/2850158147 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920005b20 con 0x7f89100039c0 2024-04-25T00:35:27.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.948+0000 7f892b4bf640 1 -- 172.21.15.29:0/2850158147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89100039c0 msgr2=0x7f8910003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:27.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.948+0000 7f892b4bf640 1 --2- 172.21.15.29:0/2850158147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89100039c0 0x7f8910003de0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f892000c210 tx=0x7f892001cbe0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.948+0000 7f8929a5c640 1 -- 172.21.15.29:0/2850158147 reap_dead start 2024-04-25T00:35:27.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.948+0000 7f892b4bf640 1 -- 172.21.15.29:0/2850158147 shutdown_connections 2024-04-25T00:35:27.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.948+0000 7f892b4bf640 1 -- 172.21.15.29:0/2850158147 >> 172.21.15.29:0/2850158147 conn(0x7f89240b1870 msgr2=0x7f89240b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:27.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.948+0000 7f892b4bf640 1 -- 172.21.15.29:0/2850158147 shutdown_connections 2024-04-25T00:35:27.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.948+0000 7f892b4bf640 1 -- 172.21.15.29:0/2850158147 wait complete. 2024-04-25T00:35:27.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.948+0000 7f892b4bf640 1 Processor -- start 2024-04-25T00:35:27.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f892b4bf640 1 -- start start 2024-04-25T00:35:27.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f892b4bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89100039c0 0x7f8924149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f892b4bf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89241f4ab0 0x7f892414a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f892b4bf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892414c430 0x7f892414c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f892b4bf640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f892415c210 con 0x7f89241f4ab0 2024-04-25T00:35:27.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f892b4bf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f892415c090 con 0x7f89100039c0 2024-04-25T00:35:27.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f892b4bf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f892415c390 con 0x7f892414c430 2024-04-25T00:35:27.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8928a5a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89241f4ab0 0x7f892414a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8928a5a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89241f4ab0 0x7f892414a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56048/0 (socket says 172.21.15.29:56048) 2024-04-25T00:35:27.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8928a5a640 1 -- 172.21.15.29:0/3364410214 learned_addr learned my addr 172.21.15.29:0/3364410214 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:27.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8929a5c640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892414c430 0x7f892414c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f892925b640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89100039c0 0x7f8924149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8928a5a640 1 -- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892414c430 msgr2=0x7f892414c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:27.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8928a5a640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892414c430 0x7f892414c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8928a5a640 1 -- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89100039c0 msgr2=0x7f8924149410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:27.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8928a5a640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89100039c0 0x7f8924149410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8928a5a640 1 -- 172.21.15.29:0/3364410214 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89241f5070 con 0x7f89241f4ab0 2024-04-25T00:35:27.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f892925b640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89100039c0 0x7f8924149410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:27.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8928a5a640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89241f4ab0 0x7f892414a8d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8914010750 tx=0x7f8914010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:27.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8919ffb640 1 -- 172.21.15.29:0/3364410214 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891400b040 con 0x7f89241f4ab0 2024-04-25T00:35:27.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8919ffb640 1 -- 172.21.15.29:0/3364410214 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f891400ecc0 con 0x7f89241f4ab0 2024-04-25T00:35:27.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f892b4bf640 1 -- 172.21.15.29:0/3364410214 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89241f5350 con 0x7f89241f4ab0 2024-04-25T00:35:27.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8919ffb640 1 -- 172.21.15.29:0/3364410214 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914012650 con 0x7f89241f4ab0 2024-04-25T00:35:27.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f892b4bf640 1 -- 172.21.15.29:0/3364410214 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89241f5910 con 0x7f89241f4ab0 2024-04-25T00:35:27.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f892b4bf640 1 -- 172.21.15.29:0/3364410214 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89240692e0 con 0x7f89241f4ab0 2024-04-25T00:35:27.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8919ffb640 1 -- 172.21.15.29:0/3364410214 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f89140127f0 con 0x7f89241f4ab0 2024-04-25T00:35:27.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8919ffb640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8908067b00 0x7f8908069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.952+0000 7f8919ffb640 1 -- 172.21.15.29:0/3364410214 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8914086b10 con 0x7f89241f4ab0 2024-04-25T00:35:27.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.956+0000 7f892925b640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8908067b00 0x7f8908069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.956+0000 7f892925b640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8908067b00 0x7f8908069fc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f89200aa000 tx=0x7f8920000ff0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:27.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:27.960+0000 7f8919ffb640 1 -- 172.21.15.29:0/3364410214 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f89140571e0 con 0x7f89241f4ab0 2024-04-25T00:35:28.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.144+0000 7f892b4bf640 1 -- 172.21.15.29:0/3364410214 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol62000", "auth_id": "alice", "target": ["mon-mgr", ""]}) -- 0x7f892414ce20 con 0x7f8908067b00 2024-04-25T00:35:28.171 INFO:tasks.ceph.mgr.x.smithi157.stderr:Creating meta for ID alice with tenant None 2024-04-25T00:35:28.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.236+0000 7f8919ffb640 1 -- 172.21.15.29:0/3364410214 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f892414ce20 con 0x7f8908067b00 2024-04-25T00:35:28.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.240+0000 7f892b4bf640 1 -- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8908067b00 msgr2=0x7f8908069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.240+0000 7f892b4bf640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8908067b00 0x7f8908069fc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f89200aa000 tx=0x7f8920000ff0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.240+0000 7f892b4bf640 1 -- 172.21.15.29:0/3364410214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89241f4ab0 msgr2=0x7f892414a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.240+0000 7f892b4bf640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89241f4ab0 0x7f892414a8d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8914010750 tx=0x7f8914010c10 comp rx=0 tx=0).stop 2024-04-25T00:35:28.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.240+0000 7f892b4bf640 1 -- 172.21.15.29:0/3364410214 shutdown_connections 2024-04-25T00:35:28.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.240+0000 7f892b4bf640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f892414c430 0x7f892414c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.240+0000 7f892b4bf640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89241f4ab0 0x7f892414a8d0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.240+0000 7f892b4bf640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8908067b00 0x7f8908069fc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.240+0000 7f892b4bf640 1 --2- 172.21.15.29:0/3364410214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89100039c0 0x7f8924149410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.240+0000 7f892b4bf640 1 -- 172.21.15.29:0/3364410214 >> 172.21.15.29:0/3364410214 conn(0x7f89240b1870 msgr2=0x7f892415c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:28.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.240+0000 7f892b4bf640 1 -- 172.21.15.29:0/3364410214 shutdown_connections 2024-04-25T00:35:28.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.240+0000 7f892b4bf640 1 -- 172.21.15.29:0/3364410214 wait complete. 2024-04-25T00:35:28.258 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2024-04-25T00:35:28.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa07a0fd640 1 Processor -- start 2024-04-25T00:35:28.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa07a0fd640 1 -- start start 2024-04-25T00:35:28.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa07a0fd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0740b67d0 0x7fa07406b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa07a0fd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa0740cca10 0x7fa07406e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa07a0fd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa07406bb30 con 0x7fa07406b210 2024-04-25T00:35:28.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa07a0fd640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa07406f5e0 con 0x7fa0740cca10 2024-04-25T00:35:28.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa07a0fd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa07406f8c0 con 0x7fa0740b67d0 2024-04-25T00:35:28.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa073fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa0740cca10 0x7fa07406e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa073fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa0740cca10 0x7fa07406e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56058/0 (socket says 172.21.15.29:56058) 2024-04-25T00:35:28.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa073fff640 1 -- 172.21.15.29:0/1963983939 learned_addr learned my addr 172.21.15.29:0/1963983939 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:28.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa072ffd640 1 --2- 172.21.15.29:0/1963983939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0740b67d0 0x7fa07406b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa073fff640 1 -- 172.21.15.29:0/1963983939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0740b67d0 msgr2=0x7fa07406b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:28.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa073fff640 1 --2- 172.21.15.29:0/1963983939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0740b67d0 0x7fa07406b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa073fff640 1 -- 172.21.15.29:0/1963983939 >> v1:172.21.15.157:6789/0 conn(0x7fa07406b210 legacy=0x7fa07415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:28.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.372+0000 7fa073fff640 1 -- 172.21.15.29:0/1963983939 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07406ddb0 con 0x7fa0740cca10 2024-04-25T00:35:28.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa073fff640 1 --2- 172.21.15.29:0/1963983939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0740cca10 0x7fa07406e5b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa064009e50 tx=0x7fa06401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f16d5fccc43489 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:28.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa0727fc640 1 -- 172.21.15.29:0/1963983939 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0640a57d0 con 0x7fa0740cca10 2024-04-25T00:35:28.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa0727fc640 1 -- 172.21.15.29:0/1963983939 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0640a5970 con 0x7fa0740cca10 2024-04-25T00:35:28.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa0727fc640 1 -- 172.21.15.29:0/1963983939 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0640a5d40 con 0x7fa0740cca10 2024-04-25T00:35:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 -- 172.21.15.29:0/1963983939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0740cca10 msgr2=0x7fa07406e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 --2- 172.21.15.29:0/1963983939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0740cca10 0x7fa07406e5b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa064009e50 tx=0x7fa06401c9c0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 -- 172.21.15.29:0/1963983939 shutdown_connections 2024-04-25T00:35:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 --2- 172.21.15.29:0/1963983939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0740cca10 0x7fa07406e5b0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 --2- 172.21.15.29:0/1963983939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0740b67d0 0x7fa07406b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 -- 172.21.15.29:0/1963983939 >> 172.21.15.29:0/1963983939 conn(0x7fa07406c320 msgr2=0x7fa07406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 -- 172.21.15.29:0/1963983939 shutdown_connections 2024-04-25T00:35:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 -- 172.21.15.29:0/1963983939 wait complete. 2024-04-25T00:35:28.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 Processor -- start 2024-04-25T00:35:28.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 -- start start 2024-04-25T00:35:28.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa07406b210 0x7fa0740ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0740b67d0 0x7fa0741511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0740cca10 0x7fa0741508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa07406f080 con 0x7fa0740b67d0 2024-04-25T00:35:28.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa07406ef00 con 0x7fa07406b210 2024-04-25T00:35:28.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa0741695b0 con 0x7fa0740cca10 2024-04-25T00:35:28.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa072ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0740b67d0 0x7fa0741511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa072ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0740b67d0 0x7fa0741511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56060/0 (socket says 172.21.15.29:56060) 2024-04-25T00:35:28.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa072ffd640 1 -- 172.21.15.29:0/3130842748 learned_addr learned my addr 172.21.15.29:0/3130842748 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:28.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa0737fe640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa07406b210 0x7fa0740ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa073fff640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0740cca10 0x7fa0741508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa072ffd640 1 -- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0740cca10 msgr2=0x7fa0741508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa072ffd640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0740cca10 0x7fa0741508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa072ffd640 1 -- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa07406b210 msgr2=0x7fa0740ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa072ffd640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa07406b210 0x7fa0740ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa072ffd640 1 -- 172.21.15.29:0/3130842748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa074151700 con 0x7fa0740b67d0 2024-04-25T00:35:28.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa072ffd640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0740b67d0 0x7fa0741511c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00d800 tx=0x7fa05c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:28.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa06bfff640 1 -- 172.21.15.29:0/3130842748 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00b6c0 con 0x7fa0740b67d0 2024-04-25T00:35:28.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa06bfff640 1 -- 172.21.15.29:0/3130842748 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05c019070 con 0x7fa0740b67d0 2024-04-25T00:35:28.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa06bfff640 1 -- 172.21.15.29:0/3130842748 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00baa0 con 0x7fa0740b67d0 2024-04-25T00:35:28.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 -- 172.21.15.29:0/3130842748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa074150e30 con 0x7fa0740b67d0 2024-04-25T00:35:28.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.376+0000 7fa07a0fd640 1 -- 172.21.15.29:0/3130842748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0740cab90 con 0x7fa0740b67d0 2024-04-25T00:35:28.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.380+0000 7fa07a0fd640 1 -- 172.21.15.29:0/3130842748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa074069270 con 0x7fa0740b67d0 2024-04-25T00:35:28.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.380+0000 7fa06bfff640 1 -- 172.21.15.29:0/3130842748 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7fa05c00bc40 con 0x7fa0740b67d0 2024-04-25T00:35:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.380+0000 7fa06bfff640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa054067b30 0x7fa054069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.380+0000 7fa06bfff640 1 -- 172.21.15.29:0/3130842748 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa05c086480 con 0x7fa0740b67d0 2024-04-25T00:35:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.380+0000 7fa0737fe640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa054067b30 0x7fa054069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.380+0000 7fa0737fe640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa054067b30 0x7fa054069ff0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00b810 tx=0x7fa06c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:28.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.384+0000 7fa06bfff640 1 -- 172.21.15.29:0/3130842748 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa05c056cf0 con 0x7fa0740b67d0 2024-04-25T00:35:28.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.596+0000 7fa07a0fd640 1 -- 172.21.15.29:0/3130842748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) -- 0x7fa0740727c0 con 0x7fa0740b67d0 2024-04-25T00:35:28.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.596+0000 7fa06bfff640 1 -- 172.21.15.29:0/3130842748 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v18) ==== 107+0+280 (secure 0 0 0) 0x7fa05c0565f0 con 0x7fa0740b67d0 2024-04-25T00:35:28.599 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:28.599 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:35:28.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:28.599 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.alice", 2024-04-25T00:35:28.599 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQpSlm+bmwCxAADTdjANQjpTtBIQzT9gU5Rw==", 2024-04-25T00:35:28.599 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:28.599 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol62000", 2024-04-25T00:35:28.599 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-04-25T00:35:28.599 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw pool=cephfs_data" 2024-04-25T00:35:28.599 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:28.600 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:28.600 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:35:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.600+0000 7fa07a0fd640 1 -- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa054067b30 msgr2=0x7fa054069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.600+0000 7fa07a0fd640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa054067b30 0x7fa054069ff0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00b810 tx=0x7fa06c008040 comp rx=0 tx=0).stop 2024-04-25T00:35:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.600+0000 7fa07a0fd640 1 -- 172.21.15.29:0/3130842748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0740b67d0 msgr2=0x7fa0741511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.600+0000 7fa07a0fd640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0740b67d0 0x7fa0741511c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00d800 tx=0x7fa05c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.600+0000 7fa07a0fd640 1 -- 172.21.15.29:0/3130842748 shutdown_connections 2024-04-25T00:35:28.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.600+0000 7fa07a0fd640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0740cca10 0x7fa0741508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.600+0000 7fa07a0fd640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0740b67d0 0x7fa0741511c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.600+0000 7fa07a0fd640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa054067b30 0x7fa054069ff0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.600+0000 7fa07a0fd640 1 --2- 172.21.15.29:0/3130842748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa07406b210 0x7fa0740ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.600+0000 7fa07a0fd640 1 -- 172.21.15.29:0/3130842748 >> 172.21.15.29:0/3130842748 conn(0x7fa07406c320 msgr2=0x7fa0741560e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:28.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.600+0000 7fa07a0fd640 1 -- 172.21.15.29:0/3130842748 shutdown_connections 2024-04-25T00:35:28.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.600+0000 7fa07a0fd640 1 -- 172.21.15.29:0/3130842748 wait complete. 2024-04-25T00:35:28.617 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol62000 2024-04-25T00:35:28.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4adbfe2640 1 Processor -- start 2024-04-25T00:35:28.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4adbfe2640 1 -- start start 2024-04-25T00:35:28.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4adbfe2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad40b3140 0x7f4ad406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4adbfe2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad40b41e0 con 0x7f4ad40b4b10 2024-04-25T00:35:28.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4adbfe2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad406d960 con 0x7f4ad40b3c30 2024-04-25T00:35:28.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4adbfe2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad406dc40 con 0x7f4ad40b3140 2024-04-25T00:35:28.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad957d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4ad40b4b10 0x7f4ad40b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55072/0 (socket says 172.21.15.29:55072) 2024-04-25T00:35:28.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad957d640 1 -- 172.21.15.29:0/2403962579 learned_addr learned my addr 172.21.15.29:0/2403962579 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:28.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad9d7e640 1 --2- 172.21.15.29:0/2403962579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad40b3140 0x7f4ad406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 188479821 0 0) 0x7f4ad40b41e0 con 0x7f4ad40b4b10 2024-04-25T00:35:28.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ac0003530 con 0x7f4ad40b4b10 2024-04-25T00:35:28.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 712574748 0 0) 0x7f4ac0003530 con 0x7f4ad40b4b10 2024-04-25T00:35:28.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad40b3140 msgr2=0x7f4ad406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:28.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 --2- 172.21.15.29:0/2403962579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad40b3140 0x7f4ad406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 >> v1:172.21.15.157:6789/0 conn(0x7f4ad40b3c30 legacy=0x7f4ad40ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad406df20 con 0x7f4ad40b4b10 2024-04-25T00:35:28.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4ad0001680 con 0x7f4ad40b4b10 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 >> v1:172.21.15.29:6789/0 conn(0x7f4ad40b4b10 legacy=0x7f4ad40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 --2- 172.21.15.29:0/2403962579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ac00039f0 0x7f4ac0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 --2- 172.21.15.29:0/2403962579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0004810 0x7f4ac0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 --2- 172.21.15.29:0/2403962579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad40b3140 0x7f4ac00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad406dc40 con 0x7f4ac00039f0 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac000ba80 con 0x7f4ac0004810 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4ac000bc80 con 0x7f4ad40b3140 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad9d7e640 1 --2- 172.21.15.29:0/2403962579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ac00039f0 0x7f4ac0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad957d640 1 --2- 172.21.15.29:0/2403962579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0004810 0x7f4ac0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ada57f640 1 --2- 172.21.15.29:0/2403962579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad40b3140 0x7f4ac00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad9d7e640 1 -- 172.21.15.29:0/2403962579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad40b3140 msgr2=0x7f4ac00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad9d7e640 1 --2- 172.21.15.29:0/2403962579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad40b3140 0x7f4ac00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad9d7e640 1 -- 172.21.15.29:0/2403962579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0004810 msgr2=0x7f4ac0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad9d7e640 1 --2- 172.21.15.29:0/2403962579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0004810 0x7f4ac0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad9d7e640 1 -- 172.21.15.29:0/2403962579 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac0003530 con 0x7f4ac00039f0 2024-04-25T00:35:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.740+0000 7f4ad957d640 1 --2- 172.21.15.29:0/2403962579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0004810 0x7f4ac0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.744+0000 7f4ad9d7e640 1 --2- 172.21.15.29:0/2403962579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ac00039f0 0x7f4ac0003e10 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009ab0 tx=0x7f4ac401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e84b7811c466c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.744+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac40096b0 con 0x7f4ac00039f0 2024-04-25T00:35:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.744+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac40a6410 con 0x7f4ac00039f0 2024-04-25T00:35:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.744+0000 7f4ad8d7c640 1 -- 172.21.15.29:0/2403962579 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac40a6800 con 0x7f4ac00039f0 2024-04-25T00:35:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.744+0000 7f4adbfe2640 1 -- 172.21.15.29:0/2403962579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ac00039f0 msgr2=0x7f4ac0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.744+0000 7f4adbfe2640 1 --2- 172.21.15.29:0/2403962579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ac00039f0 0x7f4ac0003e10 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009ab0 tx=0x7f4ac401ca30 comp rx=0 tx=0).stop 2024-04-25T00:35:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.744+0000 7f4ada57f640 1 -- 172.21.15.29:0/2403962579 reap_dead start 2024-04-25T00:35:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.744+0000 7f4adbfe2640 1 -- 172.21.15.29:0/2403962579 shutdown_connections 2024-04-25T00:35:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.744+0000 7f4adbfe2640 1 -- 172.21.15.29:0/2403962579 >> 172.21.15.29:0/2403962579 conn(0x7f4ad40b1870 msgr2=0x7f4ad40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.744+0000 7f4adbfe2640 1 -- 172.21.15.29:0/2403962579 shutdown_connections 2024-04-25T00:35:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.744+0000 7f4adbfe2640 1 -- 172.21.15.29:0/2403962579 wait complete. 2024-04-25T00:35:28.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4adbfe2640 1 Processor -- start 2024-04-25T00:35:28.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4adbfe2640 1 -- start start 2024-04-25T00:35:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4adbfe2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ac00039f0 0x7f4ad413a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4adbfe2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad41566b0 0x7f4ad415a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4adbfe2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad415ad60 0x7f4ad413d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4adbfe2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4167d80 con 0x7f4ac00039f0 2024-04-25T00:35:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4adbfe2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4167c00 con 0x7f4ad415ad60 2024-04-25T00:35:28.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4adbfe2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad4167f00 con 0x7f4ad41566b0 2024-04-25T00:35:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4ad9d7e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ac00039f0 0x7f4ad413a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4ad9d7e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ac00039f0 0x7f4ad413a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56068/0 (socket says 172.21.15.29:56068) 2024-04-25T00:35:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4ad9d7e640 1 -- 172.21.15.29:0/3829569045 learned_addr learned my addr 172.21.15.29:0/3829569045 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4ad957d640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad41566b0 0x7f4ad415a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4ada57f640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad415ad60 0x7f4ad413d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4ad9d7e640 1 -- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad41566b0 msgr2=0x7f4ad415a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4ad9d7e640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad41566b0 0x7f4ad415a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4ad9d7e640 1 -- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad415ad60 msgr2=0x7f4ad413d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4ad9d7e640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad415ad60 0x7f4ad413d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4ad9d7e640 1 -- 172.21.15.29:0/3829569045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad413bf30 con 0x7f4ac00039f0 2024-04-25T00:35:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4ada57f640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad415ad60 0x7f4ad413d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4ad9d7e640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ac00039f0 0x7f4ad413a880 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009ab0 tx=0x7f4ac0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:28.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4aca7fc640 1 -- 172.21.15.29:0/3829569045 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac400f000 con 0x7f4ac00039f0 2024-04-25T00:35:28.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4aca7fc640 1 -- 172.21.15.29:0/3829569045 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac40a6b40 con 0x7f4ac00039f0 2024-04-25T00:35:28.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4aca7fc640 1 -- 172.21.15.29:0/3829569045 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4004020 con 0x7f4ac00039f0 2024-04-25T00:35:28.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4adbfe2640 1 -- 172.21.15.29:0/3829569045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad413ae90 con 0x7f4ac00039f0 2024-04-25T00:35:28.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4adbfe2640 1 -- 172.21.15.29:0/3829569045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad413b450 con 0x7f4ac00039f0 2024-04-25T00:35:28.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4adbfe2640 1 -- 172.21.15.29:0/3829569045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad4069270 con 0x7f4ac00039f0 2024-04-25T00:35:28.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4aca7fc640 1 -- 172.21.15.29:0/3829569045 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f4ac40c2060 con 0x7f4ac00039f0 2024-04-25T00:35:28.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.748+0000 7f4aca7fc640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ab8067b30 0x7f4ab8069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:28.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.752+0000 7f4aca7fc640 1 -- 172.21.15.29:0/3829569045 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4ac411fdd0 con 0x7f4ac00039f0 2024-04-25T00:35:28.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.752+0000 7f4ad957d640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ab8067b30 0x7f4ab8069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:28.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.752+0000 7f4ad957d640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ab8067b30 0x7f4ab8069ff0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0001000 tx=0x7f4ad0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:28.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.756+0000 7f4aca7fc640 1 -- 172.21.15.29:0/3829569045 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4ac40ac020 con 0x7f4ac00039f0 2024-04-25T00:35:28.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.944+0000 7f4adbfe2640 1 -- 172.21.15.29:0/3829569045 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol62000", "target": ["mon-mgr", ""]}) -- 0x7f4ad415a3e0 con 0x7f4ab8067b30 2024-04-25T00:35:28.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.968+0000 7f4aca7fc640 1 -- 172.21.15.29:0/3829569045 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4ad415a3e0 con 0x7f4ab8067b30 2024-04-25T00:35:28.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.972+0000 7f4adbfe2640 1 -- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ab8067b30 msgr2=0x7f4ab8069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.972+0000 7f4adbfe2640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ab8067b30 0x7f4ab8069ff0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0001000 tx=0x7f4ad0009040 comp rx=0 tx=0).stop 2024-04-25T00:35:28.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.972+0000 7f4adbfe2640 1 -- 172.21.15.29:0/3829569045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ac00039f0 msgr2=0x7f4ad413a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:28.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.972+0000 7f4adbfe2640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ac00039f0 0x7f4ad413a880 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009ab0 tx=0x7f4ac0005840 comp rx=0 tx=0).stop 2024-04-25T00:35:28.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.972+0000 7f4adbfe2640 1 -- 172.21.15.29:0/3829569045 shutdown_connections 2024-04-25T00:35:28.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.972+0000 7f4adbfe2640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4ab8067b30 0x7f4ab8069ff0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.972+0000 7f4adbfe2640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad415ad60 0x7f4ad413d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.972+0000 7f4adbfe2640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ad41566b0 0x7f4ad415a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.972+0000 7f4adbfe2640 1 --2- 172.21.15.29:0/3829569045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ac00039f0 0x7f4ad413a880 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:28.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.972+0000 7f4adbfe2640 1 -- 172.21.15.29:0/3829569045 >> 172.21.15.29:0/3829569045 conn(0x7f4ad40b1870 msgr2=0x7f4ad40cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:28.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.972+0000 7f4adbfe2640 1 -- 172.21.15.29:0/3829569045 shutdown_connections 2024-04-25T00:35:28.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:28.972+0000 7f4adbfe2640 1 -- 172.21.15.29:0/3829569045 wait complete. 2024-04-25T00:35:28.978 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:28.976+0000 7f47fa863640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:28.978 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:28.976+0000 7f47fa863640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:28.978 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:28.976+0000 7f47fa863640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:28.979 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:28.976+0000 7f47fa863640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:28.979 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:28.976+0000 7f47fa863640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:28.991 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol54001 2024-04-25T00:35:28.999 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:28.996+0000 7f47f9060640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:28.999 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:28.996+0000 7f47f9060640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:28.999 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:28.996+0000 7f47f9060640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:28.999 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:28.996+0000 7f47f9060640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:28.999 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:35:28.996+0000 7f47f9060640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:35:29.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.112+0000 7efcb1822640 1 Processor -- start 2024-04-25T00:35:29.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.112+0000 7efcb1822640 1 -- start start 2024-04-25T00:35:29.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcb1822640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efcac06c950 0x7efcac06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcb1822640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcac0cd410 0x7efcac0cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcb1822640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcac0cc3b0 con 0x7efcac15e440 2024-04-25T00:35:29.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcb1822640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7efcac06d270 con 0x7efcac06c950 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcb1822640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efcac0b11d0 con 0x7efcac0cd410 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcaa7fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7efcac15e440 0x7efcac0d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55088/0 (socket says 172.21.15.29:55088) 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcaa7fc640 1 -- 172.21.15.29:0/1336717461 learned_addr learned my addr 172.21.15.29:0/1336717461 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcab7fe640 1 --2- 172.21.15.29:0/1336717461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcac0cd410 0x7efcac0cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcaaffd640 1 --2- 172.21.15.29:0/1336717461 >> v2:172.21.15.157:3300/0 conn(0x7efcac06c950 0x7efcac06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1861448106 0 0) 0x7efcac0cc3b0 con 0x7efcac15e440 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc94003530 con 0x7efcac15e440 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 557283327 0 0) 0x7efc94003530 con 0x7efcac15e440 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcac0cd410 msgr2=0x7efcac0cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 --2- 172.21.15.29:0/1336717461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcac0cd410 0x7efcac0cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 >> v2:172.21.15.157:3300/0 conn(0x7efcac06c950 msgr2=0x7efcac06cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 --2- 172.21.15.29:0/1336717461 >> v2:172.21.15.157:3300/0 conn(0x7efcac06c950 0x7efcac06cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcac0cc650 con 0x7efcac15e440 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7efca0001680 con 0x7efcac15e440 2024-04-25T00:35:29.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 >> v1:172.21.15.29:6789/0 conn(0x7efcac15e440 legacy=0x7efcac0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:29.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 --2- 172.21.15.29:0/1336717461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcac0cd410 0x7efc940039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 --2- 172.21.15.29:0/1336717461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc94003f40 0x7efc94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 --2- 172.21.15.29:0/1336717461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc94004d50 0x7efc940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efcac06d270 con 0x7efc94003f40 2024-04-25T00:35:29.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efcac0b11d0 con 0x7efc94004d50 2024-04-25T00:35:29.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc9400ba20 con 0x7efcac0cd410 2024-04-25T00:35:29.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcaa7fc640 1 --2- 172.21.15.29:0/1336717461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc94003f40 0x7efc94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:29.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcaa7fc640 1 -- 172.21.15.29:0/1336717461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcac0cd410 msgr2=0x7efc940039c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:35:29.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcaa7fc640 1 --2- 172.21.15.29:0/1336717461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcac0cd410 0x7efc940039c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcaa7fc640 1 -- 172.21.15.29:0/1336717461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc94004d50 msgr2=0x7efc940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:29.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcaa7fc640 1 --2- 172.21.15.29:0/1336717461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc94004d50 0x7efc940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.116+0000 7efcaa7fc640 1 -- 172.21.15.29:0/1336717461 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc94003530 con 0x7efc94003f40 2024-04-25T00:35:29.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcaa7fc640 1 --2- 172.21.15.29:0/1336717461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc94003f40 0x7efc94004810 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7efca0001060 tx=0x7efca001f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5482390f11803f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0007d80 con 0x7efc94003f40 2024-04-25T00:35:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca0005b00 con 0x7efc94003f40 2024-04-25T00:35:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efca9ffb640 1 -- 172.21.15.29:0/1336717461 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0007070 con 0x7efc94003f40 2024-04-25T00:35:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 -- 172.21.15.29:0/1336717461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc94003f40 msgr2=0x7efc94004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 --2- 172.21.15.29:0/1336717461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc94003f40 0x7efc94004810 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7efca0001060 tx=0x7efca001f410 comp rx=0 tx=0).stop 2024-04-25T00:35:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcab7fe640 1 -- 172.21.15.29:0/1336717461 reap_dead start 2024-04-25T00:35:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 -- 172.21.15.29:0/1336717461 shutdown_connections 2024-04-25T00:35:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 -- 172.21.15.29:0/1336717461 >> 172.21.15.29:0/1336717461 conn(0x7efcac0d1d60 msgr2=0x7efcac0d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 -- 172.21.15.29:0/1336717461 shutdown_connections 2024-04-25T00:35:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 -- 172.21.15.29:0/1336717461 wait complete. 2024-04-25T00:35:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 Processor -- start 2024-04-25T00:35:29.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 -- start start 2024-04-25T00:35:29.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc94003f40 0x7efcac156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcac0c7690 0x7efcac0c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcac15c0c0 0x7efcac15c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efcac0ccef0 con 0x7efcac15c0c0 2024-04-25T00:35:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efcac0ccc10 con 0x7efc94003f40 2024-04-25T00:35:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efcac06c520 con 0x7efcac0c7690 2024-04-25T00:35:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcab7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcac15c0c0 0x7efcac15c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcab7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcac15c0c0 0x7efcac15c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56078/0 (socket says 172.21.15.29:56078) 2024-04-25T00:35:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcab7fe640 1 -- 172.21.15.29:0/481374039 learned_addr learned my addr 172.21.15.29:0/481374039 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcaaffd640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc94003f40 0x7efcac156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcaa7fc640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcac0c7690 0x7efcac0c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcab7fe640 1 -- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcac0c7690 msgr2=0x7efcac0c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcab7fe640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcac0c7690 0x7efcac0c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcab7fe640 1 -- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc94003f40 msgr2=0x7efcac156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:29.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcab7fe640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc94003f40 0x7efcac156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcab7fe640 1 -- 172.21.15.29:0/481374039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcac0c8110 con 0x7efcac15c0c0 2024-04-25T00:35:29.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcab7fe640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcac15c0c0 0x7efcac15c540 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7efcac15e820 tx=0x7efca400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:29.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efc93fff640 1 -- 172.21.15.29:0/481374039 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca400bcf0 con 0x7efcac15c0c0 2024-04-25T00:35:29.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efc93fff640 1 -- 172.21.15.29:0/481374039 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca401a070 con 0x7efcac15c0c0 2024-04-25T00:35:29.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 -- 172.21.15.29:0/481374039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcac0c83f0 con 0x7efcac15c0c0 2024-04-25T00:35:29.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efc93fff640 1 -- 172.21.15.29:0/481374039 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4014640 con 0x7efcac15c0c0 2024-04-25T00:35:29.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.120+0000 7efcb1822640 1 -- 172.21.15.29:0/481374039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcac1f5350 con 0x7efcac15c0c0 2024-04-25T00:35:29.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.124+0000 7efcb1822640 1 -- 172.21.15.29:0/481374039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcac069270 con 0x7efcac15c0c0 2024-04-25T00:35:29.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.124+0000 7efc93fff640 1 -- 172.21.15.29:0/481374039 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7efca4013070 con 0x7efcac15c0c0 2024-04-25T00:35:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.124+0000 7efc93fff640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efc98067b00 0x7efc98069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.124+0000 7efc93fff640 1 -- 172.21.15.29:0/481374039 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7efca4086460 con 0x7efcac15c0c0 2024-04-25T00:35:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.124+0000 7efcaaffd640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efc98067b00 0x7efc98069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:29.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.124+0000 7efcaaffd640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efc98067b00 0x7efc98069fc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00b9b0 tx=0x7efc9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:29.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.128+0000 7efc93fff640 1 -- 172.21.15.29:0/481374039 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efca4056c00 con 0x7efcac15c0c0 2024-04-25T00:35:29.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.308+0000 7efcb1822640 1 -- 172.21.15.29:0/481374039 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol54001", "target": ["mon-mgr", ""]}) -- 0x7efcac1f5520 con 0x7efc98067b00 2024-04-25T00:35:29.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.344+0000 7efc93fff640 1 -- 172.21.15.29:0/481374039 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efcac1f5520 con 0x7efc98067b00 2024-04-25T00:35:29.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.344+0000 7efcb1822640 1 -- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efc98067b00 msgr2=0x7efc98069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:29.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.344+0000 7efcb1822640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efc98067b00 0x7efc98069fc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00b9b0 tx=0x7efc9c008040 comp rx=0 tx=0).stop 2024-04-25T00:35:29.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.344+0000 7efcb1822640 1 -- 172.21.15.29:0/481374039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcac15c0c0 msgr2=0x7efcac15c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:29.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.344+0000 7efcb1822640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcac15c0c0 0x7efcac15c540 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7efcac15e820 tx=0x7efca400b480 comp rx=0 tx=0).stop 2024-04-25T00:35:29.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.344+0000 7efcb1822640 1 -- 172.21.15.29:0/481374039 shutdown_connections 2024-04-25T00:35:29.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.344+0000 7efcb1822640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcac15c0c0 0x7efcac15c540 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.344+0000 7efcb1822640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcac0c7690 0x7efcac0c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.344+0000 7efcb1822640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7efc98067b00 0x7efc98069fc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.344+0000 7efcb1822640 1 --2- 172.21.15.29:0/481374039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc94003f40 0x7efcac156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.344+0000 7efcb1822640 1 -- 172.21.15.29:0/481374039 >> 172.21.15.29:0/481374039 conn(0x7efcac0d1d60 msgr2=0x7efcac0b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:29.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.344+0000 7efcb1822640 1 -- 172.21.15.29:0/481374039 shutdown_connections 2024-04-25T00:35:29.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.344+0000 7efcb1822640 1 -- 172.21.15.29:0/481374039 wait complete. 2024-04-25T00:35:29.365 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-25T00:35:29.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:35:29.416 INFO:teuthology.orchestra.run.smithi029.stdout:2 2024-04-25T00:35:29.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:35:29.422 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:35:29.422 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:35:29.502 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:35:29.562 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:35:29.562 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:35:29.562 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:35:29.562 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:35:29.570 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:29.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.736+0000 7f45c372e640 1 Processor -- start 2024-04-25T00:35:29.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.736+0000 7f45c372e640 1 -- start start 2024-04-25T00:35:29.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c372e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45bc0b3140 0x7f45bc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c372e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45bc0b4b10 0x7f45bc0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c372e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45bc0b41e0 con 0x7f45bc0b3a60 2024-04-25T00:35:29.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c372e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f45bc06d960 con 0x7f45bc0b3140 2024-04-25T00:35:29.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c372e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc06dc40 con 0x7f45bc0b4b10 2024-04-25T00:35:29.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c1ccb640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f45bc0b3a60 0x7f45bc15c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55092/0 (socket says 172.21.15.29:55092) 2024-04-25T00:35:29.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c1ccb640 1 -- 172.21.15.29:0/3681130226 learned_addr learned my addr 172.21.15.29:0/3681130226 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:29.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c14ca640 1 --2- 172.21.15.29:0/3681130226 >> v2:172.21.15.29:3300/0 conn(0x7f45bc0b3140 0x7f45bc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:29.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c0cc9640 1 --2- 172.21.15.29:0/3681130226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45bc0b4b10 0x7f45bc0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:29.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c14ca640 1 -- 172.21.15.29:0/3681130226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45bc0b4b10 msgr2=0x7f45bc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:29.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c14ca640 1 --2- 172.21.15.29:0/3681130226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45bc0b4b10 0x7f45bc0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c14ca640 1 -- 172.21.15.29:0/3681130226 >> v1:172.21.15.29:6789/0 conn(0x7f45bc0b3a60 legacy=0x7f45bc15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:29.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c14ca640 1 -- 172.21.15.29:0/3681130226 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc06df20 con 0x7f45bc0b3140 2024-04-25T00:35:29.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45c14ca640 1 --2- 172.21.15.29:0/3681130226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45bc0b3140 0x7f45bc06e440 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f45b00097c0 tx=0x7f45b001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe54fc8b81d039d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:29.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45affff640 1 -- 172.21.15.29:0/3681130226 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b00b0070 con 0x7f45bc0b3140 2024-04-25T00:35:29.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45affff640 1 -- 172.21.15.29:0/3681130226 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b00ab600 con 0x7f45bc0b3140 2024-04-25T00:35:29.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.740+0000 7f45affff640 1 -- 172.21.15.29:0/3681130226 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b00ab9d0 con 0x7f45bc0b3140 2024-04-25T00:35:29.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 -- 172.21.15.29:0/3681130226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45bc0b3140 msgr2=0x7f45bc06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:29.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 --2- 172.21.15.29:0/3681130226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45bc0b3140 0x7f45bc06e440 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f45b00097c0 tx=0x7f45b001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 -- 172.21.15.29:0/3681130226 shutdown_connections 2024-04-25T00:35:29.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 --2- 172.21.15.29:0/3681130226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45bc0b4b10 0x7f45bc0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 --2- 172.21.15.29:0/3681130226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45bc0b3140 0x7f45bc06e440 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 -- 172.21.15.29:0/3681130226 >> 172.21.15.29:0/3681130226 conn(0x7f45bc0b1870 msgr2=0x7f45bc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:29.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 -- 172.21.15.29:0/3681130226 shutdown_connections 2024-04-25T00:35:29.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 -- 172.21.15.29:0/3681130226 wait complete. 2024-04-25T00:35:29.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 Processor -- start 2024-04-25T00:35:29.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 -- start start 2024-04-25T00:35:29.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45bc0b3140 0x7f45bc13c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45bc0b3a60 0x7f45bc1404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45bc0b4b10 0x7f45bc1409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc168b70 con 0x7f45bc0b3140 2024-04-25T00:35:29.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1689f0 con 0x7f45bc0b4b10 2024-04-25T00:35:29.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc168cf0 con 0x7f45bc0b3a60 2024-04-25T00:35:29.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c14ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45bc0b3140 0x7f45bc13c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:29.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c14ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45bc0b3140 0x7f45bc13c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56088/0 (socket says 172.21.15.29:56088) 2024-04-25T00:35:29.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c14ca640 1 -- 172.21.15.29:0/2939734017 learned_addr learned my addr 172.21.15.29:0/2939734017 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:29.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c0cc9640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45bc0b3a60 0x7f45bc1404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c14ca640 1 -- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45bc0b3a60 msgr2=0x7f45bc1404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c14ca640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45bc0b3a60 0x7f45bc1404a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c14ca640 1 -- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45bc0b4b10 msgr2=0x7f45bc1409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c1ccb640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45bc0b4b10 0x7f45bc1409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c14ca640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45bc0b4b10 0x7f45bc1409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c14ca640 1 -- 172.21.15.29:0/2939734017 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc13c830 con 0x7f45bc0b3140 2024-04-25T00:35:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c14ca640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45bc0b3140 0x7f45bc13c2f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f45b00ae040 tx=0x7f45b001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45adffb640 1 -- 172.21.15.29:0/2939734017 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b00b0050 con 0x7f45bc0b3140 2024-04-25T00:35:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45adffb640 1 -- 172.21.15.29:0/2939734017 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b00bc070 con 0x7f45bc0b3140 2024-04-25T00:35:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 -- 172.21.15.29:0/2939734017 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc15b050 con 0x7f45bc0b3140 2024-04-25T00:35:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45adffb640 1 -- 172.21.15.29:0/2939734017 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b00b5850 con 0x7f45bc0b3140 2024-04-25T00:35:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.744+0000 7f45c372e640 1 -- 172.21.15.29:0/2939734017 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc15b610 con 0x7f45bc0b3140 2024-04-25T00:35:29.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.748+0000 7f45c372e640 1 -- 172.21.15.29:0/2939734017 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45bc069270 con 0x7f45bc0b3140 2024-04-25T00:35:29.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.748+0000 7f45adffb640 1 -- 172.21.15.29:0/2939734017 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f45b00b59f0 con 0x7f45bc0b3140 2024-04-25T00:35:29.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.748+0000 7f45adffb640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f45a0067b00 0x7f45a0069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:29.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.748+0000 7f45adffb640 1 -- 172.21.15.29:0/2939734017 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f45b0120ae0 con 0x7f45bc0b3140 2024-04-25T00:35:29.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.748+0000 7f45c0cc9640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f45a0067b00 0x7f45a0069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:29.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.748+0000 7f45c0cc9640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f45a0067b00 0x7f45a0069fc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0cee80 tx=0x7f45b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:29.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.752+0000 7f45adffb640 1 -- 172.21.15.29:0/2939734017 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f45b00a4e20 con 0x7f45bc0b3140 2024-04-25T00:35:29.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.920+0000 7f45c372e640 1 -- 172.21.15.29:0/2939734017 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45bc13d1e0 con 0x7f45bc0b3140 2024-04-25T00:35:29.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.920+0000 7f45adffb640 1 -- 172.21.15.29:0/2939734017 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16617 (secure 0 0 0) 0x7f45b0016090 con 0x7f45bc0b3140 2024-04-25T00:35:29.923 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:29.923 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":58,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:08.767613+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:05.458923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:06.104267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:29.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.924+0000 7f45c372e640 1 -- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f45a0067b00 msgr2=0x7f45a0069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.924+0000 7f45c372e640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f45a0067b00 0x7f45a0069fc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0cee80 tx=0x7f45b8008040 comp rx=0 tx=0).stop 2024-04-25T00:35:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.924+0000 7f45c372e640 1 -- 172.21.15.29:0/2939734017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45bc0b3140 msgr2=0x7f45bc13c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.924+0000 7f45c372e640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45bc0b3140 0x7f45bc13c2f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f45b00ae040 tx=0x7f45b001ace0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.924+0000 7f45c372e640 1 -- 172.21.15.29:0/2939734017 shutdown_connections 2024-04-25T00:35:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.924+0000 7f45c372e640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f45a0067b00 0x7f45a0069fc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.924+0000 7f45c372e640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45bc0b4b10 0x7f45bc1409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.924+0000 7f45c372e640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45bc0b3a60 0x7f45bc1404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.924+0000 7f45c372e640 1 --2- 172.21.15.29:0/2939734017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45bc0b3140 0x7f45bc13c2f0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.924+0000 7f45c372e640 1 -- 172.21.15.29:0/2939734017 >> 172.21.15.29:0/2939734017 conn(0x7f45bc0b1870 msgr2=0x7f45bc15e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.924+0000 7f45c372e640 1 -- 172.21.15.29:0/2939734017 shutdown_connections 2024-04-25T00:35:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:29.924+0000 7f45c372e640 1 -- 172.21.15.29:0/2939734017 wait complete. 2024-04-25T00:35:29.942 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:35:05.458923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:35:06.104267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:35:29.943 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:35:30.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27b4d31640 1 Processor -- start 2024-04-25T00:35:30.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27b4d31640 1 -- start start 2024-04-25T00:35:30.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27b4d31640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27b00b3c30 0x7f27b006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27b4d31640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27b00b4b10 0x7f27b00ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27b4d31640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27b00b41e0 con 0x7f27b00b3140 2024-04-25T00:35:30.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27b4d31640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f27b006d960 con 0x7f27b00b4b10 2024-04-25T00:35:30.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27b4d31640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f27b006dc40 con 0x7f27b00b3c30 2024-04-25T00:35:30.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27ae59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f27b00b3140 0x7f27b00b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55104/0 (socket says 172.21.15.29:55104) 2024-04-25T00:35:30.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27ae59c640 1 -- 172.21.15.29:0/1969873486 learned_addr learned my addr 172.21.15.29:0/1969873486 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:30.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27aed9d640 1 --2- 172.21.15.29:0/1969873486 >> v2:172.21.15.29:3300/0 conn(0x7f27b00b4b10 0x7f27b00ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27add9b640 1 --2- 172.21.15.29:0/1969873486 >> v2:172.21.15.157:3300/0 conn(0x7f27b00b3c30 0x7f27b006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27aed9d640 1 -- 172.21.15.29:0/1969873486 >> v2:172.21.15.157:3300/0 conn(0x7f27b00b3c30 msgr2=0x7f27b006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:30.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27aed9d640 1 --2- 172.21.15.29:0/1969873486 >> v2:172.21.15.157:3300/0 conn(0x7f27b00b3c30 0x7f27b006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27aed9d640 1 -- 172.21.15.29:0/1969873486 >> v1:172.21.15.29:6789/0 conn(0x7f27b00b3140 legacy=0x7f27b00b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27aed9d640 1 -- 172.21.15.29:0/1969873486 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b006df20 con 0x7f27b00b4b10 2024-04-25T00:35:30.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.072+0000 7f27ad59a640 1 -- 172.21.15.29:0/1969873486 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3712785540 0 0) 0x7f27b00b41e0 con 0x7f27b00b3140 2024-04-25T00:35:30.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27aed9d640 1 --2- 172.21.15.29:0/1969873486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b00b4b10 0x7f27b00ce240 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f27a80097c0 tx=0x7f27a801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc4cdeedc489810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:30.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27ad59a640 1 -- 172.21.15.29:0/1969873486 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a80b0070 con 0x7f27b00b4b10 2024-04-25T00:35:30.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27ad59a640 1 -- 172.21.15.29:0/1969873486 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a80ab600 con 0x7f27b00b4b10 2024-04-25T00:35:30.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27ad59a640 1 -- 172.21.15.29:0/1969873486 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a80ab9d0 con 0x7f27b00b4b10 2024-04-25T00:35:30.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 -- 172.21.15.29:0/1969873486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b00b4b10 msgr2=0x7f27b00ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 --2- 172.21.15.29:0/1969873486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b00b4b10 0x7f27b00ce240 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f27a80097c0 tx=0x7f27a801c460 comp rx=0 tx=0).stop 2024-04-25T00:35:30.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 -- 172.21.15.29:0/1969873486 shutdown_connections 2024-04-25T00:35:30.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 --2- 172.21.15.29:0/1969873486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b00b4b10 0x7f27b00ce240 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 --2- 172.21.15.29:0/1969873486 >> v2:172.21.15.157:3300/0 conn(0x7f27b00b3c30 0x7f27b006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 -- 172.21.15.29:0/1969873486 >> 172.21.15.29:0/1969873486 conn(0x7f27b00b1870 msgr2=0x7f27b00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:30.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 -- 172.21.15.29:0/1969873486 shutdown_connections 2024-04-25T00:35:30.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 -- 172.21.15.29:0/1969873486 wait complete. 2024-04-25T00:35:30.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 Processor -- start 2024-04-25T00:35:30.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 -- start start 2024-04-25T00:35:30.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27b00b3140 0x7f27b0164e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b00b3c30 0x7f27b015af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b00b4b10 0x7f27b0163d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0168d80 con 0x7f27b00b4b10 2024-04-25T00:35:30.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0168c00 con 0x7f27b00b3c30 2024-04-25T00:35:30.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27b4d31640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f27b0168f00 con 0x7f27b00b3140 2024-04-25T00:35:30.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27aed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b00b4b10 0x7f27b0163d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27aed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b00b4b10 0x7f27b0163d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56098/0 (socket says 172.21.15.29:56098) 2024-04-25T00:35:30.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27aed9d640 1 -- 172.21.15.29:0/1682008029 learned_addr learned my addr 172.21.15.29:0/1682008029 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:30.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27add9b640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b00b3c30 0x7f27b015af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27ae59c640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27b00b3140 0x7f27b0164e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27aed9d640 1 -- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27b00b3140 msgr2=0x7f27b0164e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27aed9d640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27b00b3140 0x7f27b0164e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27aed9d640 1 -- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b00b3c30 msgr2=0x7f27b015af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27aed9d640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b00b3c30 0x7f27b015af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27aed9d640 1 -- 172.21.15.29:0/1682008029 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b015b690 con 0x7f27b00b4b10 2024-04-25T00:35:30.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.076+0000 7f27ae59c640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27b00b3140 0x7f27b0164e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:30.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.080+0000 7f27aed9d640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b00b4b10 0x7f27b0163d40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f27a80ae040 tx=0x7f27a801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:30.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.080+0000 7f279effd640 1 -- 172.21.15.29:0/1682008029 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a80b0050 con 0x7f27b00b4b10 2024-04-25T00:35:30.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.080+0000 7f279effd640 1 -- 172.21.15.29:0/1682008029 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a80bc070 con 0x7f27b00b4b10 2024-04-25T00:35:30.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.080+0000 7f27b4d31640 1 -- 172.21.15.29:0/1682008029 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b015b970 con 0x7f27b00b4b10 2024-04-25T00:35:30.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.080+0000 7f279effd640 1 -- 172.21.15.29:0/1682008029 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8006e20 con 0x7f27b00b4b10 2024-04-25T00:35:30.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.080+0000 7f27b4d31640 1 -- 172.21.15.29:0/1682008029 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b0071130 con 0x7f27b00b4b10 2024-04-25T00:35:30.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.080+0000 7f27b4d31640 1 -- 172.21.15.29:0/1682008029 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b0069270 con 0x7f27b00b4b10 2024-04-25T00:35:30.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.080+0000 7f279effd640 1 -- 172.21.15.29:0/1682008029 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f27a80b5830 con 0x7f27b00b4b10 2024-04-25T00:35:30.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.080+0000 7f279effd640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2790067b00 0x7f2790069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.080+0000 7f279effd640 1 -- 172.21.15.29:0/1682008029 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f27a8121100 con 0x7f27b00b4b10 2024-04-25T00:35:30.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.080+0000 7f27ae59c640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2790067b00 0x7f2790069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.084+0000 7f27ae59c640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2790067b00 0x7f2790069fc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f27a0000f70 tx=0x7f27a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:30.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.084+0000 7f279effd640 1 -- 172.21.15.29:0/1682008029 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f27a8016050 con 0x7f27b00b4b10 2024-04-25T00:35:30.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.248+0000 7f27b4d31640 1 -- 172.21.15.29:0/1682008029 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f27b0071300 con 0x7f27b00b4b10 2024-04-25T00:35:30.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.248+0000 7f279effd640 1 -- 172.21.15.29:0/1682008029 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7f27a80f1b20 con 0x7f27b00b4b10 2024-04-25T00:35:30.250 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:35:30.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.252+0000 7f27b4d31640 1 -- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2790067b00 msgr2=0x7f2790069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.252+0000 7f27b4d31640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2790067b00 0x7f2790069fc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f27a0000f70 tx=0x7f27a0009040 comp rx=0 tx=0).stop 2024-04-25T00:35:30.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.252+0000 7f27b4d31640 1 -- 172.21.15.29:0/1682008029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b00b4b10 msgr2=0x7f27b0163d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.252+0000 7f27b4d31640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b00b4b10 0x7f27b0163d40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f27a80ae040 tx=0x7f27a801aca0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.252+0000 7f27b4d31640 1 -- 172.21.15.29:0/1682008029 shutdown_connections 2024-04-25T00:35:30.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.252+0000 7f27b4d31640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b00b4b10 0x7f27b0163d40 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.252+0000 7f27b4d31640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2790067b00 0x7f2790069fc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.252+0000 7f27b4d31640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b00b3c30 0x7f27b015af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.252+0000 7f27b4d31640 1 --2- 172.21.15.29:0/1682008029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27b00b3140 0x7f27b0164e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.252+0000 7f27b4d31640 1 -- 172.21.15.29:0/1682008029 >> 172.21.15.29:0/1682008029 conn(0x7f27b00b1870 msgr2=0x7f27b00cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:30.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.252+0000 7f27b4d31640 1 -- 172.21.15.29:0/1682008029 shutdown_connections 2024-04-25T00:35:30.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.252+0000 7f27b4d31640 1 -- 172.21.15.29:0/1682008029 wait complete. 2024-04-25T00:35:30.270 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:35:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5bb5a4f640 1 Processor -- start 2024-04-25T00:35:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5bb5a4f640 1 -- start start 2024-04-25T00:35:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5bb5a4f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5bb0158810 0x7f5bb015f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5bb5a4f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bb0071d80 0x7f5bb00721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5bb5a4f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bb015f830 con 0x7f5bb00cb260 2024-04-25T00:35:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5bb5a4f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5bb0072720 con 0x7f5bb0158810 2024-04-25T00:35:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5bb5a4f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb015b690 con 0x7f5bb0071d80 2024-04-25T00:35:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5baeffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5bb0158810 0x7f5bb015f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5baf7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5bb00cb260 0x7f5bb00cb6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55110/0 (socket says 172.21.15.29:55110) 2024-04-25T00:35:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5baf7fe640 1 -- 172.21.15.29:0/426066725 learned_addr learned my addr 172.21.15.29:0/426066725 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5baeffd640 1 -- 172.21.15.29:0/426066725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bb0071d80 msgr2=0x7f5bb00721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5baeffd640 1 --2- 172.21.15.29:0/426066725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bb0071d80 0x7f5bb00721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5baeffd640 1 -- 172.21.15.29:0/426066725 >> v1:172.21.15.29:6789/0 conn(0x7f5bb00cb260 legacy=0x7f5bb00cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5baeffd640 1 -- 172.21.15.29:0/426066725 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb015b970 con 0x7f5bb0158810 2024-04-25T00:35:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.392+0000 7f5badffb640 1 -- 172.21.15.29:0/426066725 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2449379065 0 0) 0x7f5ba8001120 con 0x7f5bb00cb260 2024-04-25T00:35:30.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5baeffd640 1 --2- 172.21.15.29:0/426066725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb0158810 0x7f5bb015f2f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009d90 tx=0x7f5ba001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9042fcc67ab90466 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:30.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5badffb640 1 -- 172.21.15.29:0/426066725 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba00b2070 con 0x7f5bb0158810 2024-04-25T00:35:30.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5badffb640 1 -- 172.21.15.29:0/426066725 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba00a5a00 con 0x7f5bb0158810 2024-04-25T00:35:30.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5badffb640 1 -- 172.21.15.29:0/426066725 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba00a5dd0 con 0x7f5bb0158810 2024-04-25T00:35:30.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/426066725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb0158810 msgr2=0x7f5bb015f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5bb5a4f640 1 --2- 172.21.15.29:0/426066725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb0158810 0x7f5bb015f2f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009d90 tx=0x7f5ba001c940 comp rx=0 tx=0).stop 2024-04-25T00:35:30.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/426066725 shutdown_connections 2024-04-25T00:35:30.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5bb5a4f640 1 --2- 172.21.15.29:0/426066725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bb0071d80 0x7f5bb00721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5bb5a4f640 1 --2- 172.21.15.29:0/426066725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb0158810 0x7f5bb015f2f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/426066725 >> 172.21.15.29:0/426066725 conn(0x7f5bb00d0c10 msgr2=0x7f5bb00d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:30.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/426066725 shutdown_connections 2024-04-25T00:35:30.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/426066725 wait complete. 2024-04-25T00:35:30.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5bb5a4f640 1 Processor -- start 2024-04-25T00:35:30.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.396+0000 7f5bb5a4f640 1 -- start start 2024-04-25T00:35:30.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5bb5a4f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb0071d80 0x7f5bb0147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5bb5a4f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bb00cb260 0x7f5bb015a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5bb5a4f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bb0158810 0x7f5bb0155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5bb5a4f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb015b470 con 0x7f5bb0071d80 2024-04-25T00:35:30.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5bb5a4f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb015b2f0 con 0x7f5bb0158810 2024-04-25T00:35:30.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5bb5a4f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb0158bf0 con 0x7f5bb00cb260 2024-04-25T00:35:30.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5baeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb0071d80 0x7f5bb0147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5baeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb0071d80 0x7f5bb0147e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56106/0 (socket says 172.21.15.29:56106) 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5baeffd640 1 -- 172.21.15.29:0/3218504993 learned_addr learned my addr 172.21.15.29:0/3218504993 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5baeffd640 1 -- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bb00cb260 msgr2=0x7f5bb015a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5bae7fc640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bb00cb260 0x7f5bb015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5baf7fe640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bb0158810 0x7f5bb0155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5baeffd640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bb00cb260 0x7f5bb015a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5baeffd640 1 -- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bb0158810 msgr2=0x7f5bb0155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5baeffd640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bb0158810 0x7f5bb0155690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5baeffd640 1 -- 172.21.15.29:0/3218504993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb0159010 con 0x7f5bb0071d80 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5baeffd640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb0071d80 0x7f5bb0147e50 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5ba001a570 tx=0x7f5ba00196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5ba7fff640 1 -- 172.21.15.29:0/3218504993 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba00b2040 con 0x7f5bb0071d80 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5ba7fff640 1 -- 172.21.15.29:0/3218504993 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba00bf070 con 0x7f5bb0071d80 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/3218504993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb0148390 con 0x7f5bb0071d80 2024-04-25T00:35:30.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5ba7fff640 1 -- 172.21.15.29:0/3218504993 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba00adb70 con 0x7f5bb0071d80 2024-04-25T00:35:30.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/3218504993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb0159590 con 0x7f5bb0071d80 2024-04-25T00:35:30.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/3218504993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb0069270 con 0x7f5bb0071d80 2024-04-25T00:35:30.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5ba7fff640 1 -- 172.21.15.29:0/3218504993 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f5ba00add10 con 0x7f5bb0071d80 2024-04-25T00:35:30.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5ba7fff640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5b90067b00 0x7f5b90069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5ba7fff640 1 -- 172.21.15.29:0/3218504993 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5ba0121500 con 0x7f5bb0071d80 2024-04-25T00:35:30.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.400+0000 7f5bae7fc640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5b90067b00 0x7f5b90069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.404+0000 7f5bae7fc640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5b90067b00 0x7f5b90069fc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5bb015e440 tx=0x7f5b98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:30.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.404+0000 7f5ba7fff640 1 -- 172.21.15.29:0/3218504993 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5ba0016560 con 0x7f5bb0071d80 2024-04-25T00:35:30.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.568+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/3218504993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5bb015a5b0 con 0x7f5bb0071d80 2024-04-25T00:35:30.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.568+0000 7f5ba7fff640 1 -- 172.21.15.29:0/3218504993 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) ==== 104+0+11 (secure 0 0 0) 0x7f5ba00ab340 con 0x7f5bb0071d80 2024-04-25T00:35:30.571 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:35:30.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.572+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5b90067b00 msgr2=0x7f5b90069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.572+0000 7f5bb5a4f640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5b90067b00 0x7f5b90069fc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5bb015e440 tx=0x7f5b98008040 comp rx=0 tx=0).stop 2024-04-25T00:35:30.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.572+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/3218504993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb0071d80 msgr2=0x7f5bb0147e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.572+0000 7f5bb5a4f640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb0071d80 0x7f5bb0147e50 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5ba001a570 tx=0x7f5ba00196e0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.572+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/3218504993 shutdown_connections 2024-04-25T00:35:30.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.572+0000 7f5bb5a4f640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5b90067b00 0x7f5b90069fc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.572+0000 7f5bb5a4f640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bb0158810 0x7f5bb0155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.572+0000 7f5bb5a4f640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bb00cb260 0x7f5bb015a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.572+0000 7f5bb5a4f640 1 --2- 172.21.15.29:0/3218504993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb0071d80 0x7f5bb0147e50 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.572+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/3218504993 >> 172.21.15.29:0/3218504993 conn(0x7f5bb00d0c10 msgr2=0x7f5bb00cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:30.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.572+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/3218504993 shutdown_connections 2024-04-25T00:35:30.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.572+0000 7f5bb5a4f640 1 -- 172.21.15.29:0/3218504993 wait complete. 2024-04-25T00:35:30.591 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:35:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb53323f640 1 Processor -- start 2024-04-25T00:35:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb53323f640 1 -- start start 2024-04-25T00:35:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb53323f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb52c0cf690 0x7fb52c0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb53323f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb52c0d2a90 0x7fb52c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb53323f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb52c0d2ea0 con 0x7fb52c15e200 2024-04-25T00:35:30.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb53323f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb52c0b38d0 con 0x7fb52c0cf690 2024-04-25T00:35:30.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb53323f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb52c0cfc10 con 0x7fb52c0d2a90 2024-04-25T00:35:30.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb52bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb52c0cf690 0x7fb52c0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb52bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb52c0cf690 0x7fb52c0d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56116/0 (socket says 172.21.15.29:56116) 2024-04-25T00:35:30.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb52bfff640 1 -- 172.21.15.29:0/245136181 learned_addr learned my addr 172.21.15.29:0/245136181 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:30.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb5317dc640 1 --2- 172.21.15.29:0/245136181 >> v2:172.21.15.157:3300/0 conn(0x7fb52c0d2a90 0x7fb52c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb52bfff640 1 -- 172.21.15.29:0/245136181 >> v2:172.21.15.157:3300/0 conn(0x7fb52c0d2a90 msgr2=0x7fb52c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:30.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb52bfff640 1 --2- 172.21.15.29:0/245136181 >> v2:172.21.15.157:3300/0 conn(0x7fb52c0d2a90 0x7fb52c0b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb52bfff640 1 -- 172.21.15.29:0/245136181 >> v1:172.21.15.157:6789/0 conn(0x7fb52c15e200 legacy=0x7fb52c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.720+0000 7fb52bfff640 1 -- 172.21.15.29:0/245136181 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c0cfef0 con 0x7fb52c0cf690 2024-04-25T00:35:30.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb52bfff640 1 --2- 172.21.15.29:0/245136181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb52c0cf690 0x7fb52c0d2550 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0cfb20 tx=0x7fb52001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fefafbaa25442d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:30.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb52b7fe640 1 -- 172.21.15.29:0/245136181 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5200ab610 con 0x7fb52c0cf690 2024-04-25T00:35:30.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb52b7fe640 1 -- 172.21.15.29:0/245136181 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5200b0070 con 0x7fb52c0cf690 2024-04-25T00:35:30.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb52b7fe640 1 -- 172.21.15.29:0/245136181 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5200ab9d0 con 0x7fb52c0cf690 2024-04-25T00:35:30.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb53323f640 1 -- 172.21.15.29:0/245136181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb52c0cf690 msgr2=0x7fb52c0d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb53323f640 1 --2- 172.21.15.29:0/245136181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb52c0cf690 0x7fb52c0d2550 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0cfb20 tx=0x7fb52001c520 comp rx=0 tx=0).stop 2024-04-25T00:35:30.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb53323f640 1 -- 172.21.15.29:0/245136181 shutdown_connections 2024-04-25T00:35:30.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb53323f640 1 --2- 172.21.15.29:0/245136181 >> v2:172.21.15.157:3300/0 conn(0x7fb52c0d2a90 0x7fb52c0b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb53323f640 1 --2- 172.21.15.29:0/245136181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb52c0cf690 0x7fb52c0d2550 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb53323f640 1 -- 172.21.15.29:0/245136181 >> 172.21.15.29:0/245136181 conn(0x7fb52c09d280 msgr2=0x7fb52c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:30.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb53323f640 1 -- 172.21.15.29:0/245136181 shutdown_connections 2024-04-25T00:35:30.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb53323f640 1 -- 172.21.15.29:0/245136181 wait complete. 2024-04-25T00:35:30.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb53323f640 1 Processor -- start 2024-04-25T00:35:30.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.724+0000 7fb53323f640 1 -- start start 2024-04-25T00:35:30.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb53323f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb52c0cf690 0x7fb52c15ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb53323f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c0d2a90 0x7fb52c15f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb53323f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb52c15e200 0x7fb52c155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb53323f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c0b51f0 con 0x7fb52c0cf690 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb53323f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c0b5070 con 0x7fb52c15e200 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb53323f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb52c0b5370 con 0x7fb52c0d2a90 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb530fdb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb52c0cf690 0x7fb52c15ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb530fdb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb52c0cf690 0x7fb52c15ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56130/0 (socket says 172.21.15.29:56130) 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb530fdb640 1 -- 172.21.15.29:0/3108891629 learned_addr learned my addr 172.21.15.29:0/3108891629 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb5317dc640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb52c15e200 0x7fb52c155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb52bfff640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c0d2a90 0x7fb52c15f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb530fdb640 1 -- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c0d2a90 msgr2=0x7fb52c15f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb530fdb640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c0d2a90 0x7fb52c15f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb530fdb640 1 -- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb52c15e200 msgr2=0x7fb52c155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb530fdb640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb52c15e200 0x7fb52c155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb530fdb640 1 -- 172.21.15.29:0/3108891629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c15f980 con 0x7fb52c0cf690 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb52bfff640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c0d2a90 0x7fb52c15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:30.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb530fdb640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb52c0cf690 0x7fb52c15ab30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00c3a0 tx=0x7fb51c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:30.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb5297fa640 1 -- 172.21.15.29:0/3108891629 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0052d0 con 0x7fb52c0cf690 2024-04-25T00:35:30.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb5297fa640 1 -- 172.21.15.29:0/3108891629 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51c005470 con 0x7fb52c0cf690 2024-04-25T00:35:30.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb53323f640 1 -- 172.21.15.29:0/3108891629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c15fc50 con 0x7fb52c0cf690 2024-04-25T00:35:30.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb5297fa640 1 -- 172.21.15.29:0/3108891629 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c007260 con 0x7fb52c0cf690 2024-04-25T00:35:30.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb53323f640 1 -- 172.21.15.29:0/3108891629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c160210 con 0x7fb52c0cf690 2024-04-25T00:35:30.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb53323f640 1 -- 172.21.15.29:0/3108891629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb52c069270 con 0x7fb52c0cf690 2024-04-25T00:35:30.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb5297fa640 1 -- 172.21.15.29:0/3108891629 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb51c007400 con 0x7fb52c0cf690 2024-04-25T00:35:30.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb5297fa640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb518067c50 0x7fb51806a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:30.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb5297fa640 1 -- 172.21.15.29:0/3108891629 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb51c0855f0 con 0x7fb52c0cf690 2024-04-25T00:35:30.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.728+0000 7fb52bfff640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb518067c50 0x7fb51806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:30.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.732+0000 7fb52bfff640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb518067c50 0x7fb51806a110 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb5200af000 tx=0x7fb520008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:30.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.732+0000 7fb5297fa640 1 -- 172.21.15.29:0/3108891629 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb52c0d2ea0 con 0x7fb52c0cf690 2024-04-25T00:35:30.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.900+0000 7fb53323f640 1 -- 172.21.15.29:0/3108891629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb52c156bc0 con 0x7fb52c0cf690 2024-04-25T00:35:30.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.900+0000 7fb5297fa640 1 -- 172.21.15.29:0/3108891629 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) ==== 100+0+11 (secure 0 0 0) 0x7fb51c055d70 con 0x7fb52c0cf690 2024-04-25T00:35:30.903 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:35:30.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.904+0000 7fb53323f640 1 -- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb518067c50 msgr2=0x7fb51806a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.904+0000 7fb53323f640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb518067c50 0x7fb51806a110 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb5200af000 tx=0x7fb520008040 comp rx=0 tx=0).stop 2024-04-25T00:35:30.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.904+0000 7fb53323f640 1 -- 172.21.15.29:0/3108891629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb52c0cf690 msgr2=0x7fb52c15ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:30.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.904+0000 7fb53323f640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb52c0cf690 0x7fb52c15ab30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00c3a0 tx=0x7fb51c00c860 comp rx=0 tx=0).stop 2024-04-25T00:35:30.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.904+0000 7fb53323f640 1 -- 172.21.15.29:0/3108891629 shutdown_connections 2024-04-25T00:35:30.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.904+0000 7fb53323f640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb518067c50 0x7fb51806a110 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.904+0000 7fb53323f640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb52c15e200 0x7fb52c155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.904+0000 7fb53323f640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c0d2a90 0x7fb52c15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.904+0000 7fb53323f640 1 --2- 172.21.15.29:0/3108891629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb52c0cf690 0x7fb52c15ab30 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:30.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.904+0000 7fb53323f640 1 -- 172.21.15.29:0/3108891629 >> 172.21.15.29:0/3108891629 conn(0x7fb52c09d280 msgr2=0x7fb52c15d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:30.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.904+0000 7fb53323f640 1 -- 172.21.15.29:0/3108891629 shutdown_connections 2024-04-25T00:35:30.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:30.904+0000 7fb53323f640 1 -- 172.21.15.29:0/3108891629 wait complete. 2024-04-25T00:35:30.922 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:35:31.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.040+0000 7faffff90640 1 Processor -- start 2024-04-25T00:35:31.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.040+0000 7faffff90640 1 -- start start 2024-04-25T00:35:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.040+0000 7faffff90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff80b3c30 0x7faff806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.040+0000 7faffff90640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff80b41e0 con 0x7faff80b4b10 2024-04-25T00:35:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.040+0000 7faffff90640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff806d960 con 0x7faff80b3140 2024-04-25T00:35:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.040+0000 7faffff90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faff806dc40 con 0x7faff80b3c30 2024-04-25T00:35:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.040+0000 7faffe52d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7faff80b4b10 0x7faff80ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55116/0 (socket says 172.21.15.29:55116) 2024-04-25T00:35:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.040+0000 7faffe52d640 1 -- 172.21.15.29:0/395595548 learned_addr learned my addr 172.21.15.29:0/395595548 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.040+0000 7faffd52b640 1 --2- 172.21.15.29:0/395595548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff80b3c30 0x7faff806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.040+0000 7faffcd2a640 1 -- 172.21.15.29:0/395595548 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 360423720 0 0) 0x7faff80b41e0 con 0x7faff80b4b10 2024-04-25T00:35:31.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.040+0000 7faffcd2a640 1 -- 172.21.15.29:0/395595548 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafe4003530 con 0x7faff80b4b10 2024-04-25T00:35:31.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffcd2a640 1 -- 172.21.15.29:0/395595548 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1667589894 0 0) 0x7faff806d960 con 0x7faff80b3140 2024-04-25T00:35:31.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffcd2a640 1 -- 172.21.15.29:0/395595548 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faff80b41e0 con 0x7faff80b3140 2024-04-25T00:35:31.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffd52b640 1 -- 172.21.15.29:0/395595548 >> v1:172.21.15.157:6789/0 conn(0x7faff80b3140 legacy=0x7faff80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffd52b640 1 -- 172.21.15.29:0/395595548 >> v1:172.21.15.29:6789/0 conn(0x7faff80b4b10 legacy=0x7faff80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffd52b640 1 -- 172.21.15.29:0/395595548 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff806df20 con 0x7faff80b3c30 2024-04-25T00:35:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffd52b640 1 --2- 172.21.15.29:0/395595548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff80b3c30 0x7faff806e440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fafe80097f0 tx=0x7fafe801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d096467431811ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffcd2a640 1 -- 172.21.15.29:0/395595548 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe80b0070 con 0x7faff80b3c30 2024-04-25T00:35:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffcd2a640 1 -- 172.21.15.29:0/395595548 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe80ab600 con 0x7faff80b3c30 2024-04-25T00:35:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffcd2a640 1 -- 172.21.15.29:0/395595548 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe80ab9b0 con 0x7faff80b3c30 2024-04-25T00:35:31.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffff90640 1 -- 172.21.15.29:0/395595548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff80b3c30 msgr2=0x7faff806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffff90640 1 --2- 172.21.15.29:0/395595548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff80b3c30 0x7faff806e440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fafe80097f0 tx=0x7fafe801c4e0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffff90640 1 -- 172.21.15.29:0/395595548 shutdown_connections 2024-04-25T00:35:31.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffff90640 1 --2- 172.21.15.29:0/395595548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff80b3c30 0x7faff806e440 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffff90640 1 -- 172.21.15.29:0/395595548 >> 172.21.15.29:0/395595548 conn(0x7faff80b1870 msgr2=0x7faff80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:31.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffff90640 1 -- 172.21.15.29:0/395595548 shutdown_connections 2024-04-25T00:35:31.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.044+0000 7faffff90640 1 -- 172.21.15.29:0/395595548 wait complete. 2024-04-25T00:35:31.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffff90640 1 Processor -- start 2024-04-25T00:35:31.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffff90640 1 -- start start 2024-04-25T00:35:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffff90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff80b3140 0x7faff8154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffff90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff80b3c30 0x7faff8138490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffff90640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faff80b4b10 0x7faff8139ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffff90640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faff8169d90 con 0x7faff80b4b10 2024-04-25T00:35:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffff90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faff8169c10 con 0x7faff80b3140 2024-04-25T00:35:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffff90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faff8169f10 con 0x7faff80b3c30 2024-04-25T00:35:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffe52d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faff80b4b10 0x7faff8139ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffe52d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faff80b4b10 0x7faff8139ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56138/0 (socket says 172.21.15.29:56138) 2024-04-25T00:35:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffe52d640 1 -- 172.21.15.29:0/3613337832 learned_addr learned my addr 172.21.15.29:0/3613337832 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffd52b640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff80b3c30 0x7faff8138490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffdd2c640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff80b3140 0x7faff8154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffe52d640 1 -- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff80b3c30 msgr2=0x7faff8138490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffe52d640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff80b3c30 0x7faff8138490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffe52d640 1 -- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff80b3140 msgr2=0x7faff8154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffe52d640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff80b3140 0x7faff8154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffe52d640 1 -- 172.21.15.29:0/3613337832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff813a3e0 con 0x7faff80b4b10 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffd52b640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff80b3c30 0x7faff8138490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffe52d640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faff80b4b10 0x7faff8139ea0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7faff000cbd0 tx=0x7faff000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7fafee7fc640 1 -- 172.21.15.29:0/3613337832 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0017070 con 0x7faff80b4b10 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7fafee7fc640 1 -- 172.21.15.29:0/3613337832 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff000ace0 con 0x7faff80b4b10 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7fafee7fc640 1 -- 172.21.15.29:0/3613337832 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0012620 con 0x7faff80b4b10 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffff90640 1 -- 172.21.15.29:0/3613337832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff8139770 con 0x7faff80b4b10 2024-04-25T00:35:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.048+0000 7faffff90640 1 -- 172.21.15.29:0/3613337832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff81319a0 con 0x7faff80b4b10 2024-04-25T00:35:31.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.052+0000 7faffff90640 1 -- 172.21.15.29:0/3613337832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff8139c90 con 0x7faff80b4b10 2024-04-25T00:35:31.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.052+0000 7fafee7fc640 1 -- 172.21.15.29:0/3613337832 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7faff001a070 con 0x7faff80b4b10 2024-04-25T00:35:31.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.052+0000 7fafee7fc640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fafdc067c20 0x7fafdc06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.052+0000 7fafee7fc640 1 -- 172.21.15.29:0/3613337832 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7faff00861d0 con 0x7faff80b4b10 2024-04-25T00:35:31.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.052+0000 7faffdd2c640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fafdc067c20 0x7fafdc06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.052+0000 7faffdd2c640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fafdc067c20 0x7fafdc06a0e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faff81f8180 tx=0x7faff4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:31.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.056+0000 7fafee7fc640 1 -- 172.21.15.29:0/3613337832 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faff0056950 con 0x7faff80b4b10 2024-04-25T00:35:31.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.232+0000 7faffff90640 1 -- 172.21.15.29:0/3613337832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faff8069270 con 0x7faff80b4b10 2024-04-25T00:35:31.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.232+0000 7fafee7fc640 1 -- 172.21.15.29:0/3613337832 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) ==== 80+0+0 (secure 0 0 0) 0x7faff0056250 con 0x7faff80b4b10 2024-04-25T00:35:31.234 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:35:31.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.236+0000 7faffff90640 1 -- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fafdc067c20 msgr2=0x7fafdc06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.236+0000 7faffff90640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fafdc067c20 0x7fafdc06a0e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faff81f8180 tx=0x7faff4009040 comp rx=0 tx=0).stop 2024-04-25T00:35:31.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.236+0000 7faffff90640 1 -- 172.21.15.29:0/3613337832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faff80b4b10 msgr2=0x7faff8139ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.236+0000 7faffff90640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faff80b4b10 0x7faff8139ea0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7faff000cbd0 tx=0x7faff000a430 comp rx=0 tx=0).stop 2024-04-25T00:35:31.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.236+0000 7faffff90640 1 -- 172.21.15.29:0/3613337832 shutdown_connections 2024-04-25T00:35:31.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.236+0000 7faffff90640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faff80b4b10 0x7faff8139ea0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.236+0000 7faffff90640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff80b3c30 0x7faff8138490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.236+0000 7faffff90640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fafdc067c20 0x7fafdc06a0e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.236+0000 7faffff90640 1 --2- 172.21.15.29:0/3613337832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff80b3140 0x7faff8154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.236+0000 7faffff90640 1 -- 172.21.15.29:0/3613337832 >> 172.21.15.29:0/3613337832 conn(0x7faff80b1870 msgr2=0x7faff8141890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:31.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.236+0000 7faffff90640 1 -- 172.21.15.29:0/3613337832 shutdown_connections 2024-04-25T00:35:31.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.236+0000 7faffff90640 1 -- 172.21.15.29:0/3613337832 wait complete. 2024-04-25T00:35:31.255 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-04-25T00:35:31.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:31.255 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:35:31.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[41893]: fuse finished with error 0 and tester_r 0 2024-04-25T00:35:31.269 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:35:31.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:31.269 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:35:31.437 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:35:31.437 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:35:31.437 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:35:31.444 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:31.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.612+0000 7f9e68f83640 1 Processor -- start 2024-04-25T00:35:31.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.612+0000 7f9e68f83640 1 -- start start 2024-04-25T00:35:31.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.612+0000 7f9e68f83640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6415e4e0 0x7f9e640d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.612+0000 7f9e68f83640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e640cfe30 con 0x7f9e640d4bf0 2024-04-25T00:35:31.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.612+0000 7f9e68f83640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e640b6f80 con 0x7f9e6415e8c0 2024-04-25T00:35:31.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.612+0000 7f9e68f83640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9e640b7260 con 0x7f9e6415e4e0 2024-04-25T00:35:31.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.612+0000 7f9e61d9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9e640d4bf0 0x7f9e640d23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55124/0 (socket says 172.21.15.29:55124) 2024-04-25T00:35:31.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.612+0000 7f9e61d9b640 1 -- 172.21.15.29:0/3189474168 learned_addr learned my addr 172.21.15.29:0/3189474168 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:31.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.612+0000 7f9e6259c640 1 --2- 172.21.15.29:0/3189474168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6415e4e0 0x7f9e640d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.612+0000 7f9e6259c640 1 -- 172.21.15.29:0/3189474168 >> v1:172.21.15.157:6789/0 conn(0x7f9e6415e8c0 legacy=0x7f9e640b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.612+0000 7f9e6259c640 1 -- 172.21.15.29:0/3189474168 >> v1:172.21.15.29:6789/0 conn(0x7f9e640d4bf0 legacy=0x7f9e640d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.612+0000 7f9e6259c640 1 -- 172.21.15.29:0/3189474168 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e640ced50 con 0x7f9e6415e4e0 2024-04-25T00:35:31.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.616+0000 7f9e6259c640 1 --2- 172.21.15.29:0/3189474168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6415e4e0 0x7f9e640d1eb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9e540097f0 tx=0x7f9e5401ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a4a7dfeb8034bed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:31.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e6159a640 1 -- 172.21.15.29:0/3189474168 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e540b5070 con 0x7f9e6415e4e0 2024-04-25T00:35:31.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e6159a640 1 -- 172.21.15.29:0/3189474168 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e540a89d0 con 0x7f9e6415e4e0 2024-04-25T00:35:31.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e6159a640 1 -- 172.21.15.29:0/3189474168 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e540a8da0 con 0x7f9e6415e4e0 2024-04-25T00:35:31.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 -- 172.21.15.29:0/3189474168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6415e4e0 msgr2=0x7f9e640d1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 --2- 172.21.15.29:0/3189474168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6415e4e0 0x7f9e640d1eb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9e540097f0 tx=0x7f9e5401ecd0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 -- 172.21.15.29:0/3189474168 shutdown_connections 2024-04-25T00:35:31.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 --2- 172.21.15.29:0/3189474168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6415e4e0 0x7f9e640d1eb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 -- 172.21.15.29:0/3189474168 >> 172.21.15.29:0/3189474168 conn(0x7f9e640d3fb0 msgr2=0x7f9e640d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:31.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 -- 172.21.15.29:0/3189474168 shutdown_connections 2024-04-25T00:35:31.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 -- 172.21.15.29:0/3189474168 wait complete. 2024-04-25T00:35:31.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 Processor -- start 2024-04-25T00:35:31.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 -- start start 2024-04-25T00:35:31.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e640d4bf0 0x7f9e640c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e6415e4e0 0x7f9e64158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6415e8c0 0x7f9e64158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9e640b8010 con 0x7f9e6415e4e0 2024-04-25T00:35:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9e640b7e90 con 0x7f9e640d4bf0 2024-04-25T00:35:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e68f83640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9e640b8190 con 0x7f9e6415e8c0 2024-04-25T00:35:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e61d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e6415e4e0 0x7f9e64158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e61d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e6415e4e0 0x7f9e64158220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56140/0 (socket says 172.21.15.29:56140) 2024-04-25T00:35:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e61d9b640 1 -- 172.21.15.29:0/3888542280 learned_addr learned my addr 172.21.15.29:0/3888542280 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e6259c640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e640d4bf0 0x7f9e640c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e62d9d640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6415e8c0 0x7f9e64158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e61d9b640 1 -- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6415e8c0 msgr2=0x7f9e64158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e61d9b640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6415e8c0 0x7f9e64158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e61d9b640 1 -- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e640d4bf0 msgr2=0x7f9e640c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e61d9b640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e640d4bf0 0x7f9e640c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e61d9b640 1 -- 172.21.15.29:0/3888542280 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e640cbbc0 con 0x7f9e6415e4e0 2024-04-25T00:35:31.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.620+0000 7f9e6259c640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e640d4bf0 0x7f9e640c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:31.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.624+0000 7f9e61d9b640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e6415e4e0 0x7f9e64158220 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00c530 tx=0x7f9e4c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:31.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.624+0000 7f9e5affd640 1 -- 172.21.15.29:0/3888542280 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c017070 con 0x7f9e6415e4e0 2024-04-25T00:35:31.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.624+0000 7f9e5affd640 1 -- 172.21.15.29:0/3888542280 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c005cf0 con 0x7f9e6415e4e0 2024-04-25T00:35:31.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.624+0000 7f9e5affd640 1 -- 172.21.15.29:0/3888542280 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c007270 con 0x7f9e6415e4e0 2024-04-25T00:35:31.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.624+0000 7f9e68f83640 1 -- 172.21.15.29:0/3888542280 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e640cbea0 con 0x7f9e6415e4e0 2024-04-25T00:35:31.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.624+0000 7f9e68f83640 1 -- 172.21.15.29:0/3888542280 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e640cc460 con 0x7f9e6415e4e0 2024-04-25T00:35:31.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.624+0000 7f9e68f83640 1 -- 172.21.15.29:0/3888542280 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e64069270 con 0x7f9e6415e4e0 2024-04-25T00:35:31.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.624+0000 7f9e5affd640 1 -- 172.21.15.29:0/3888542280 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f9e4c01a070 con 0x7f9e6415e4e0 2024-04-25T00:35:31.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.624+0000 7f9e5affd640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9e44067c20 0x7f9e4406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.624+0000 7f9e5affd640 1 -- 172.21.15.29:0/3888542280 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f9e4c0854d0 con 0x7f9e6415e4e0 2024-04-25T00:35:31.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.624+0000 7f9e6259c640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9e44067c20 0x7f9e4406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.624+0000 7f9e6259c640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9e44067c20 0x7f9e4406a0e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9e540b2000 tx=0x7f9e54008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:31.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.628+0000 7f9e5affd640 1 -- 172.21.15.29:0/3888542280 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9e4c055540 con 0x7f9e6415e4e0 2024-04-25T00:35:31.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.792+0000 7f9e68f83640 1 -- 172.21.15.29:0/3888542280 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e640751b0 con 0x7f9e6415e4e0 2024-04-25T00:35:31.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.796+0000 7f9e5affd640 1 -- 172.21.15.29:0/3888542280 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16617 (secure 0 0 0) 0x7f9e4c083e20 con 0x7f9e6415e4e0 2024-04-25T00:35:31.798 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:31.798 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":58,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:08.767613+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:05.458923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:06.104267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:31.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.796+0000 7f9e68f83640 1 -- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9e44067c20 msgr2=0x7f9e4406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.796+0000 7f9e68f83640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9e44067c20 0x7f9e4406a0e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9e540b2000 tx=0x7f9e54008040 comp rx=0 tx=0).stop 2024-04-25T00:35:31.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.796+0000 7f9e68f83640 1 -- 172.21.15.29:0/3888542280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e6415e4e0 msgr2=0x7f9e64158220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.796+0000 7f9e68f83640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e6415e4e0 0x7f9e64158220 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00c530 tx=0x7f9e4c00c9f0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.796+0000 7f9e68f83640 1 -- 172.21.15.29:0/3888542280 shutdown_connections 2024-04-25T00:35:31.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.796+0000 7f9e68f83640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e6415e8c0 0x7f9e64158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.796+0000 7f9e68f83640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e6415e4e0 0x7f9e64158220 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.796+0000 7f9e68f83640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9e44067c20 0x7f9e4406a0e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.796+0000 7f9e68f83640 1 --2- 172.21.15.29:0/3888542280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e640d4bf0 0x7f9e640c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.796+0000 7f9e68f83640 1 -- 172.21.15.29:0/3888542280 >> 172.21.15.29:0/3888542280 conn(0x7f9e640d3fb0 msgr2=0x7f9e640b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:31.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.796+0000 7f9e68f83640 1 -- 172.21.15.29:0/3888542280 shutdown_connections 2024-04-25T00:35:31.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.796+0000 7f9e68f83640 1 -- 172.21.15.29:0/3888542280 wait complete. 2024-04-25T00:35:31.819 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:35:05.458923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:35:06.104267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:35:31.819 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:35:31.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcfb52640 1 Processor -- start 2024-04-25T00:35:31.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcfb52640 1 -- start start 2024-04-25T00:35:31.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcfb52640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc80b3140 0x7fddc806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcfb52640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fddc80b4b10 0x7fddc80b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcfb52640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddc80b41e0 con 0x7fddc80b3a60 2024-04-25T00:35:31.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcfb52640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fddc806d960 con 0x7fddc80b4b10 2024-04-25T00:35:31.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcfb52640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fddc806dc40 con 0x7fddc80b3140 2024-04-25T00:35:31.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcd0ed640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fddc80b4b10 0x7fddc80b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcd0ed640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fddc80b4b10 0x7fddc80b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56150/0 (socket says 172.21.15.29:56150) 2024-04-25T00:35:31.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcd0ed640 1 -- 172.21.15.29:0/1705607312 learned_addr learned my addr 172.21.15.29:0/1705607312 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:31.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcd8ee640 1 --2- 172.21.15.29:0/1705607312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc80b3140 0x7fddc806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcd0ed640 1 -- 172.21.15.29:0/1705607312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc80b3140 msgr2=0x7fddc806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:31.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcd0ed640 1 --2- 172.21.15.29:0/1705607312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc80b3140 0x7fddc806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcd0ed640 1 -- 172.21.15.29:0/1705607312 >> v1:172.21.15.29:6789/0 conn(0x7fddc80b3a60 legacy=0x7fddc815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcd0ed640 1 -- 172.21.15.29:0/1705607312 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc806df20 con 0x7fddc80b4b10 2024-04-25T00:35:31.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.948+0000 7fddcc8ec640 1 -- 172.21.15.29:0/1705607312 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1452612693 0 0) 0x7fddc80b41e0 con 0x7fddc80b3a60 2024-04-25T00:35:31.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcd0ed640 1 --2- 172.21.15.29:0/1705607312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddc80b4b10 0x7fddc80b3520 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009870 tx=0x7fddbc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f148232928dbee7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:31.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcc8ec640 1 -- 172.21.15.29:0/1705607312 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0b0070 con 0x7fddc80b4b10 2024-04-25T00:35:31.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcc8ec640 1 -- 172.21.15.29:0/1705607312 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddbc0ab600 con 0x7fddc80b4b10 2024-04-25T00:35:31.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcc8ec640 1 -- 172.21.15.29:0/1705607312 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0ab9d0 con 0x7fddc80b4b10 2024-04-25T00:35:31.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 -- 172.21.15.29:0/1705607312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddc80b4b10 msgr2=0x7fddc80b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 --2- 172.21.15.29:0/1705607312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddc80b4b10 0x7fddc80b3520 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009870 tx=0x7fddbc01c560 comp rx=0 tx=0).stop 2024-04-25T00:35:31.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 -- 172.21.15.29:0/1705607312 shutdown_connections 2024-04-25T00:35:31.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 --2- 172.21.15.29:0/1705607312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddc80b4b10 0x7fddc80b3520 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 --2- 172.21.15.29:0/1705607312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc80b3140 0x7fddc806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 -- 172.21.15.29:0/1705607312 >> 172.21.15.29:0/1705607312 conn(0x7fddc80b1870 msgr2=0x7fddc80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:31.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 -- 172.21.15.29:0/1705607312 shutdown_connections 2024-04-25T00:35:31.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 -- 172.21.15.29:0/1705607312 wait complete. 2024-04-25T00:35:31.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 Processor -- start 2024-04-25T00:35:31.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 -- start start 2024-04-25T00:35:31.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc80b3140 0x7fddc815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddc80b3a60 0x7fddc8141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc80b4b10 0x7fddc8141b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8168b70 con 0x7fddc80b3a60 2024-04-25T00:35:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fddc81689f0 con 0x7fddc80b4b10 2024-04-25T00:35:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcfb52640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fddc8168cf0 con 0x7fddc80b3140 2024-04-25T00:35:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcd0ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddc80b3a60 0x7fddc8141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcd0ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddc80b3a60 0x7fddc8141630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56152/0 (socket says 172.21.15.29:56152) 2024-04-25T00:35:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.952+0000 7fddcd0ed640 1 -- 172.21.15.29:0/2026129467 learned_addr learned my addr 172.21.15.29:0/2026129467 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddcd8ee640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc80b3140 0x7fddc815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddce0ef640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc80b4b10 0x7fddc8141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddcd0ed640 1 -- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc80b3140 msgr2=0x7fddc815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddcd0ed640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc80b3140 0x7fddc815ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddcd0ed640 1 -- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc80b4b10 msgr2=0x7fddc8141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddcd0ed640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc80b4b10 0x7fddc8141b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddcd0ed640 1 -- 172.21.15.29:0/2026129467 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc8072a10 con 0x7fddc80b3a60 2024-04-25T00:35:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddcd8ee640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc80b3140 0x7fddc815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddcd0ed640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddc80b3a60 0x7fddc8141630 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0ae040 tx=0x7fddbc019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddc67fc640 1 -- 172.21.15.29:0/2026129467 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0b0050 con 0x7fddc80b3a60 2024-04-25T00:35:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddc67fc640 1 -- 172.21.15.29:0/2026129467 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddbc0bc070 con 0x7fddc80b3a60 2024-04-25T00:35:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddcfb52640 1 -- 172.21.15.29:0/2026129467 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc8141230 con 0x7fddc80b3a60 2024-04-25T00:35:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddc67fc640 1 -- 172.21.15.29:0/2026129467 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0b5a90 con 0x7fddc80b3a60 2024-04-25T00:35:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddcfb52640 1 -- 172.21.15.29:0/2026129467 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc8140f50 con 0x7fddc80b3a60 2024-04-25T00:35:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddcfb52640 1 -- 172.21.15.29:0/2026129467 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc8069270 con 0x7fddc80b3a60 2024-04-25T00:35:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddc67fc640 1 -- 172.21.15.29:0/2026129467 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fddbc0b5c30 con 0x7fddc80b3a60 2024-04-25T00:35:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddc67fc640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fddb4067c20 0x7fddb406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddc67fc640 1 -- 172.21.15.29:0/2026129467 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fddbc120a70 con 0x7fddc80b3a60 2024-04-25T00:35:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddcd8ee640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fddb4067c20 0x7fddb406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.956+0000 7fddcd8ee640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fddb4067c20 0x7fddb406a0e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009f50 tx=0x7fddc000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:31.960+0000 7fddc67fc640 1 -- 172.21.15.29:0/2026129467 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fddbc016150 con 0x7fddc80b3a60 2024-04-25T00:35:32.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.124+0000 7fddcfb52640 1 -- 172.21.15.29:0/2026129467 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fddc813f9a0 con 0x7fddc80b3a60 2024-04-25T00:35:32.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.124+0000 7fddc67fc640 1 -- 172.21.15.29:0/2026129467 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7fddbc0f11b0 con 0x7fddc80b3a60 2024-04-25T00:35:32.127 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:35:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.128+0000 7fddcfb52640 1 -- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fddb4067c20 msgr2=0x7fddb406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.128+0000 7fddcfb52640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fddb4067c20 0x7fddb406a0e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009f50 tx=0x7fddc000b040 comp rx=0 tx=0).stop 2024-04-25T00:35:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.128+0000 7fddcfb52640 1 -- 172.21.15.29:0/2026129467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddc80b3a60 msgr2=0x7fddc8141630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.128+0000 7fddcfb52640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddc80b3a60 0x7fddc8141630 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0ae040 tx=0x7fddbc019300 comp rx=0 tx=0).stop 2024-04-25T00:35:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.128+0000 7fddcfb52640 1 -- 172.21.15.29:0/2026129467 shutdown_connections 2024-04-25T00:35:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.128+0000 7fddcfb52640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fddb4067c20 0x7fddb406a0e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.128+0000 7fddcfb52640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc80b4b10 0x7fddc8141b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.128+0000 7fddcfb52640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddc80b3a60 0x7fddc8141630 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.128+0000 7fddcfb52640 1 --2- 172.21.15.29:0/2026129467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc80b3140 0x7fddc815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.128+0000 7fddcfb52640 1 -- 172.21.15.29:0/2026129467 >> 172.21.15.29:0/2026129467 conn(0x7fddc80b1870 msgr2=0x7fddc80ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.128+0000 7fddcfb52640 1 -- 172.21.15.29:0/2026129467 shutdown_connections 2024-04-25T00:35:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.128+0000 7fddcfb52640 1 -- 172.21.15.29:0/2026129467 wait complete. 2024-04-25T00:35:32.146 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:35:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f7108819640 1 Processor -- start 2024-04-25T00:35:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f7108819640 1 -- start start 2024-04-25T00:35:32.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f7108819640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f710415e440 0x7f71040d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f7108819640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f71040d0ff0 0x7f71040cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f7108819640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71040cc3b0 con 0x7f710406c950 2024-04-25T00:35:32.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f7108819640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f71040b11d0 con 0x7f710415e440 2024-04-25T00:35:32.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f7108819640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f71040cc650 con 0x7f71040d0ff0 2024-04-25T00:35:32.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f70fbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f710415e440 0x7f71040d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f70fbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f710415e440 0x7f71040d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56156/0 (socket says 172.21.15.29:56156) 2024-04-25T00:35:32.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f70fbfff640 1 -- 172.21.15.29:0/761351528 learned_addr learned my addr 172.21.15.29:0/761351528 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:32.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f7102d9d640 1 --2- 172.21.15.29:0/761351528 >> v2:172.21.15.157:3300/0 conn(0x7f71040d0ff0 0x7f71040cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f70fbfff640 1 -- 172.21.15.29:0/761351528 >> v2:172.21.15.157:3300/0 conn(0x7f71040d0ff0 msgr2=0x7f71040cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:32.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f70fbfff640 1 --2- 172.21.15.29:0/761351528 >> v2:172.21.15.157:3300/0 conn(0x7f71040d0ff0 0x7f71040cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f70fbfff640 1 -- 172.21.15.29:0/761351528 >> v1:172.21.15.29:6789/0 conn(0x7f710406c950 legacy=0x7f710406cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f7101d9b640 1 -- 172.21.15.29:0/761351528 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1613044526 0 0) 0x7f71040cc3b0 con 0x7f710406c950 2024-04-25T00:35:32.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.276+0000 7f70fbfff640 1 -- 172.21.15.29:0/761351528 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71040cc930 con 0x7f710415e440 2024-04-25T00:35:32.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f70fbfff640 1 --2- 172.21.15.29:0/761351528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f710415e440 0x7f71040d0ab0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f71040c8dc0 tx=0x7f70ec01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ca3b3d50c8b61b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7101d9b640 1 -- 172.21.15.29:0/761351528 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec0ab610 con 0x7f710415e440 2024-04-25T00:35:32.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7101d9b640 1 -- 172.21.15.29:0/761351528 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70ec0b0070 con 0x7f710415e440 2024-04-25T00:35:32.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7101d9b640 1 -- 172.21.15.29:0/761351528 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec0ab9d0 con 0x7f710415e440 2024-04-25T00:35:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 -- 172.21.15.29:0/761351528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f710415e440 msgr2=0x7f71040d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 --2- 172.21.15.29:0/761351528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f710415e440 0x7f71040d0ab0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f71040c8dc0 tx=0x7f70ec01c520 comp rx=0 tx=0).stop 2024-04-25T00:35:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 -- 172.21.15.29:0/761351528 shutdown_connections 2024-04-25T00:35:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 --2- 172.21.15.29:0/761351528 >> v2:172.21.15.157:3300/0 conn(0x7f71040d0ff0 0x7f71040cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 --2- 172.21.15.29:0/761351528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f710415e440 0x7f71040d0ab0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 -- 172.21.15.29:0/761351528 >> 172.21.15.29:0/761351528 conn(0x7f71040d1d60 msgr2=0x7f71040d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 -- 172.21.15.29:0/761351528 shutdown_connections 2024-04-25T00:35:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 -- 172.21.15.29:0/761351528 wait complete. 2024-04-25T00:35:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 Processor -- start 2024-04-25T00:35:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 -- start start 2024-04-25T00:35:32.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f710406c950 0x7f7104153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71040d0ff0 0x7f7104159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f710415e440 0x7f710414be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f71040ce2e0 con 0x7f710415e440 2024-04-25T00:35:32.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71040ce160 con 0x7f71040d0ff0 2024-04-25T00:35:32.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7108819640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f71040ce460 con 0x7f710406c950 2024-04-25T00:35:32.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7102d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f710415e440 0x7f710414be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7102d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f710415e440 0x7f710414be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56168/0 (socket says 172.21.15.29:56168) 2024-04-25T00:35:32.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7102d9d640 1 -- 172.21.15.29:0/2901905168 learned_addr learned my addr 172.21.15.29:0/2901905168 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:32.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f70fbfff640 1 --2- 172.21.15.29:0/2901905168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71040d0ff0 0x7f7104159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7102d9d640 1 -- 172.21.15.29:0/2901905168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f710406c950 msgr2=0x7f7104153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7102d9d640 1 --2- 172.21.15.29:0/2901905168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f710406c950 0x7f7104153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7102d9d640 1 -- 172.21.15.29:0/2901905168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71040d0ff0 msgr2=0x7f7104159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7102d9d640 1 --2- 172.21.15.29:0/2901905168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71040d0ff0 0x7f7104159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7102d9d640 1 -- 172.21.15.29:0/2901905168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710414c3d0 con 0x7f710415e440 2024-04-25T00:35:32.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.280+0000 7f7102d9d640 1 --2- 172.21.15.29:0/2901905168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f710415e440 0x7f710414be90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f70fc007d70 tx=0x7f70fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.284+0000 7f70faffd640 1 -- 172.21.15.29:0/2901905168 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc014420 con 0x7f710415e440 2024-04-25T00:35:32.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.284+0000 7f70faffd640 1 -- 172.21.15.29:0/2901905168 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70fc019070 con 0x7f710415e440 2024-04-25T00:35:32.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.284+0000 7f7108819640 1 -- 172.21.15.29:0/2901905168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710414c650 con 0x7f710415e440 2024-04-25T00:35:32.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.284+0000 7f70faffd640 1 -- 172.21.15.29:0/2901905168 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc014820 con 0x7f710415e440 2024-04-25T00:35:32.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.284+0000 7f7108819640 1 -- 172.21.15.29:0/2901905168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710414cc10 con 0x7f710415e440 2024-04-25T00:35:32.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.284+0000 7f7108819640 1 -- 172.21.15.29:0/2901905168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7104069270 con 0x7f710415e440 2024-04-25T00:35:32.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.284+0000 7f70faffd640 1 -- 172.21.15.29:0/2901905168 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f70fc0149c0 con 0x7f710415e440 2024-04-25T00:35:32.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.284+0000 7f70faffd640 1 --2- 172.21.15.29:0/2901905168 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f70e4067c20 0x7f70e406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.284+0000 7f70faffd640 1 -- 172.21.15.29:0/2901905168 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f70fc086a10 con 0x7f710415e440 2024-04-25T00:35:32.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.284+0000 7f710259c640 1 --2- 172.21.15.29:0/2901905168 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f70e4067c20 0x7f70e406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.284+0000 7f710259c640 1 --2- 172.21.15.29:0/2901905168 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f70e4067c20 0x7f70e406a0e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f70f4000f70 tx=0x7f70f40079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.288+0000 7f70faffd640 1 -- 172.21.15.29:0/2901905168 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f710414c3d0 con 0x7f710415e440 2024-04-25T00:35:32.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.448+0000 7f7108819640 1 -- 172.21.15.29:0/2901905168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f710415da80 con 0x7f710415e440 2024-04-25T00:35:32.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.452+0000 7f70faffd640 1 -- 172.21.15.29:0/2901905168 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) ==== 104+0+11 (secure 0 0 0) 0x7f70fc011260 con 0x7f710415e440 2024-04-25T00:35:32.453 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:35:32.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.452+0000 7f7108819640 1 -- 172.21.15.29:0/2901905168 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f70e4067c20 msgr2=0x7f70e406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.452+0000 7f7108819640 1 --2- 172.21.15.29:0/2901905168 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f70e4067c20 0x7f70e406a0e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f70f4000f70 tx=0x7f70f40079c0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.452+0000 7f7108819640 1 -- 172.21.15.29:0/2901905168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f710415e440 msgr2=0x7f710414be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.452+0000 7f7108819640 1 --2- 172.21.15.29:0/2901905168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f710415e440 0x7f710414be90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f70fc007d70 tx=0x7f70fc00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.452+0000 7f7108819640 1 -- 172.21.15.29:0/2901905168 shutdown_connections 2024-04-25T00:35:32.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.452+0000 7f7108819640 1 --2- 172.21.15.29:0/2901905168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f710415e440 0x7f710414be90 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.452+0000 7f7108819640 1 --2- 172.21.15.29:0/2901905168 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f70e4067c20 0x7f70e406a0e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.452+0000 7f7108819640 1 --2- 172.21.15.29:0/2901905168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71040d0ff0 0x7f7104159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.452+0000 7f7108819640 1 --2- 172.21.15.29:0/2901905168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f710406c950 0x7f7104153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.452+0000 7f7108819640 1 -- 172.21.15.29:0/2901905168 >> 172.21.15.29:0/2901905168 conn(0x7f71040d1d60 msgr2=0x7f71040d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:32.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.452+0000 7f7108819640 1 -- 172.21.15.29:0/2901905168 shutdown_connections 2024-04-25T00:35:32.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.452+0000 7f7108819640 1 -- 172.21.15.29:0/2901905168 wait complete. 2024-04-25T00:35:32.473 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:35:32.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3e1d35640 1 Processor -- start 2024-04-25T00:35:32.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3e1d35640 1 -- start start 2024-04-25T00:35:32.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3e1d35640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff3dc0b3140 0x7ff3dc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3e1d35640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3dc0b4b10 0x7ff3dc0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3e1d35640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3dc0b41e0 con 0x7ff3dc0b3a60 2024-04-25T00:35:32.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3e1d35640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff3dc06d960 con 0x7ff3dc0b3140 2024-04-25T00:35:32.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3e1d35640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff3dc06dc40 con 0x7ff3dc0b4b10 2024-04-25T00:35:32.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3db7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff3dc0b3140 0x7ff3dc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3db7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff3dc0b3140 0x7ff3dc06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56176/0 (socket says 172.21.15.29:56176) 2024-04-25T00:35:32.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3db7fe640 1 -- 172.21.15.29:0/3560994195 learned_addr learned my addr 172.21.15.29:0/3560994195 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3daffd640 1 --2- 172.21.15.29:0/3560994195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3dc0b4b10 0x7ff3dc0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3db7fe640 1 -- 172.21.15.29:0/3560994195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3dc0b4b10 msgr2=0x7ff3dc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3db7fe640 1 --2- 172.21.15.29:0/3560994195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3dc0b4b10 0x7ff3dc0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3db7fe640 1 -- 172.21.15.29:0/3560994195 >> v1:172.21.15.157:6789/0 conn(0x7ff3dc0b3a60 legacy=0x7ff3dc15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.604+0000 7ff3db7fe640 1 -- 172.21.15.29:0/3560994195 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3dc06df20 con 0x7ff3dc0b3140 2024-04-25T00:35:32.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.608+0000 7ff3db7fe640 1 --2- 172.21.15.29:0/3560994195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3dc0b3140 0x7ff3dc06e440 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009870 tx=0x7ff3d001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cf5fb973eff106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.608+0000 7ff3da7fc640 1 -- 172.21.15.29:0/3560994195 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00b0070 con 0x7ff3dc0b3140 2024-04-25T00:35:32.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.608+0000 7ff3da7fc640 1 -- 172.21.15.29:0/3560994195 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d00ab600 con 0x7ff3dc0b3140 2024-04-25T00:35:32.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.608+0000 7ff3da7fc640 1 -- 172.21.15.29:0/3560994195 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00a4e60 con 0x7ff3dc0b3140 2024-04-25T00:35:32.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.608+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3560994195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3dc0b3140 msgr2=0x7ff3dc06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.608+0000 7ff3e1d35640 1 --2- 172.21.15.29:0/3560994195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3dc0b3140 0x7ff3dc06e440 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009870 tx=0x7ff3d001c520 comp rx=0 tx=0).stop 2024-04-25T00:35:32.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.608+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3560994195 shutdown_connections 2024-04-25T00:35:32.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.608+0000 7ff3e1d35640 1 --2- 172.21.15.29:0/3560994195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3dc0b4b10 0x7ff3dc0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.608+0000 7ff3e1d35640 1 --2- 172.21.15.29:0/3560994195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3dc0b3140 0x7ff3dc06e440 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.608+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3560994195 >> 172.21.15.29:0/3560994195 conn(0x7ff3dc0b1870 msgr2=0x7ff3dc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:32.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.608+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3560994195 shutdown_connections 2024-04-25T00:35:32.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.608+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3560994195 wait complete. 2024-04-25T00:35:32.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3e1d35640 1 Processor -- start 2024-04-25T00:35:32.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3e1d35640 1 -- start start 2024-04-25T00:35:32.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3e1d35640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3dc0b3140 0x7ff3dc13cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3e1d35640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3dc0b3a60 0x7ff3dc070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3e1d35640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3dc0b4b10 0x7ff3dc13b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3e1d35640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc168b70 con 0x7ff3dc0b3140 2024-04-25T00:35:32.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3e1d35640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc1689f0 con 0x7ff3dc0b4b10 2024-04-25T00:35:32.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3e1d35640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff3dc168cf0 con 0x7ff3dc0b3a60 2024-04-25T00:35:32.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3db7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3dc0b3140 0x7ff3dc13cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3db7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3dc0b3140 0x7ff3dc13cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56186/0 (socket says 172.21.15.29:56186) 2024-04-25T00:35:32.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3db7fe640 1 -- 172.21.15.29:0/3849926305 learned_addr learned my addr 172.21.15.29:0/3849926305 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:32.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3daffd640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3dc0b3a60 0x7ff3dc070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3dbfff640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3dc0b4b10 0x7ff3dc13b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3db7fe640 1 -- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3dc0b3a60 msgr2=0x7ff3dc070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3db7fe640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3dc0b3a60 0x7ff3dc070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3db7fe640 1 -- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3dc0b4b10 msgr2=0x7ff3dc13b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3db7fe640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3dc0b4b10 0x7ff3dc13b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3db7fe640 1 -- 172.21.15.29:0/3849926305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3dc1f68b0 con 0x7ff3dc0b3140 2024-04-25T00:35:32.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3dbfff640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3dc0b4b10 0x7ff3dc13b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:32.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3db7fe640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3dc0b3140 0x7ff3dc13cd70 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00ae040 tx=0x7ff3d001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3c3fff640 1 -- 172.21.15.29:0/3849926305 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00ba020 con 0x7ff3dc0b3140 2024-04-25T00:35:32.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3c3fff640 1 -- 172.21.15.29:0/3849926305 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d00b6420 con 0x7ff3dc0b3140 2024-04-25T00:35:32.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3c3fff640 1 -- 172.21.15.29:0/3849926305 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00b0040 con 0x7ff3dc0b3140 2024-04-25T00:35:32.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3849926305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3dc1f6b50 con 0x7ff3dc0b3140 2024-04-25T00:35:32.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3849926305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3dc1f7110 con 0x7ff3dc0b3140 2024-04-25T00:35:32.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3849926305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3dc069270 con 0x7ff3dc0b3140 2024-04-25T00:35:32.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.612+0000 7ff3c3fff640 1 -- 172.21.15.29:0/3849926305 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7ff3d00bd020 con 0x7ff3dc0b3140 2024-04-25T00:35:32.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.616+0000 7ff3c3fff640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff3c8067c20 0x7ff3c806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.616+0000 7ff3c3fff640 1 -- 172.21.15.29:0/3849926305 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff3d0120420 con 0x7ff3dc0b3140 2024-04-25T00:35:32.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.616+0000 7ff3daffd640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff3c8067c20 0x7ff3c806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.616+0000 7ff3daffd640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff3c8067c20 0x7ff3c806a0e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc15bfc0 tx=0x7ff3cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.620+0000 7ff3c3fff640 1 -- 172.21.15.29:0/3849926305 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff3d00c4030 con 0x7ff3dc0b3140 2024-04-25T00:35:32.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.780+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3849926305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff3dc1f9e80 con 0x7ff3dc0b3140 2024-04-25T00:35:32.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.784+0000 7ff3c3fff640 1 -- 172.21.15.29:0/3849926305 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) ==== 100+0+11 (secure 0 0 0) 0x7ff3d0016110 con 0x7ff3dc0b3140 2024-04-25T00:35:32.785 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:35:32.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.784+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff3c8067c20 msgr2=0x7ff3c806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.784+0000 7ff3e1d35640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff3c8067c20 0x7ff3c806a0e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc15bfc0 tx=0x7ff3cc008040 comp rx=0 tx=0).stop 2024-04-25T00:35:32.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.784+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3849926305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3dc0b3140 msgr2=0x7ff3dc13cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.784+0000 7ff3e1d35640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3dc0b3140 0x7ff3dc13cd70 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00ae040 tx=0x7ff3d001ad60 comp rx=0 tx=0).stop 2024-04-25T00:35:32.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.784+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3849926305 shutdown_connections 2024-04-25T00:35:32.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.784+0000 7ff3e1d35640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff3c8067c20 0x7ff3c806a0e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.784+0000 7ff3e1d35640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3dc0b4b10 0x7ff3dc13b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.784+0000 7ff3e1d35640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3dc0b3a60 0x7ff3dc070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.784+0000 7ff3e1d35640 1 --2- 172.21.15.29:0/3849926305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3dc0b3140 0x7ff3dc13cd70 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.784+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3849926305 >> 172.21.15.29:0/3849926305 conn(0x7ff3dc0b1870 msgr2=0x7ff3dc154f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:32.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.784+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3849926305 shutdown_connections 2024-04-25T00:35:32.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.784+0000 7ff3e1d35640 1 -- 172.21.15.29:0/3849926305 wait complete. 2024-04-25T00:35:32.806 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:35:32.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.924+0000 7fa69fa84640 1 Processor -- start 2024-04-25T00:35:32.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.924+0000 7fa69fa84640 1 -- start start 2024-04-25T00:35:32.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.924+0000 7fa69fa84640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6980b3c30 0x7fa69806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.924+0000 7fa69fa84640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6980b41e0 con 0x7fa6980b3140 2024-04-25T00:35:32.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.924+0000 7fa69fa84640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa69806d960 con 0x7fa6980b4b10 2024-04-25T00:35:32.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.924+0000 7fa69fa84640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa69806dc40 con 0x7fa6980b3c30 2024-04-25T00:35:32.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.924+0000 7fa69d820640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa6980b3140 0x7fa6980b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55144/0 (socket says 172.21.15.29:55144) 2024-04-25T00:35:32.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.924+0000 7fa69d820640 1 -- 172.21.15.29:0/16153369 learned_addr learned my addr 172.21.15.29:0/16153369 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:32.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.924+0000 7fa69d01f640 1 --2- 172.21.15.29:0/16153369 >> v2:172.21.15.157:3300/0 conn(0x7fa6980b3c30 0x7fa69806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.924+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3656447357 0 0) 0x7fa6980b41e0 con 0x7fa6980b3140 2024-04-25T00:35:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.924+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa684003530 con 0x7fa6980b3140 2024-04-25T00:35:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.924+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2609893196 0 0) 0x7fa69806d960 con 0x7fa6980b4b10 2024-04-25T00:35:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.924+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6980b41e0 con 0x7fa6980b4b10 2024-04-25T00:35:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2605962763 0 0) 0x7fa684003530 con 0x7fa6980b3140 2024-04-25T00:35:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 >> v2:172.21.15.157:3300/0 conn(0x7fa6980b3c30 msgr2=0x7fa69806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 --2- 172.21.15.29:0/16153369 >> v2:172.21.15.157:3300/0 conn(0x7fa6980b3c30 0x7fa69806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 >> v1:172.21.15.157:6789/0 conn(0x7fa6980b4b10 legacy=0x7fa6980ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69806df20 con 0x7fa6980b3140 2024-04-25T00:35:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fa694001680 con 0x7fa6980b3140 2024-04-25T00:35:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 >> v1:172.21.15.29:6789/0 conn(0x7fa6980b3140 legacy=0x7fa6980b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 --2- 172.21.15.29:0/16153369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa684003bd0 0x7fa684003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 --2- 172.21.15.29:0/16153369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa684004810 0x7fa684004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 --2- 172.21.15.29:0/16153369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6840051d0 0x7fa68400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa69806dc40 con 0x7fa684004810 2024-04-25T00:35:32.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa68400c0d0 con 0x7fa684003bd0 2024-04-25T00:35:32.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa68400c2d0 con 0x7fa6840051d0 2024-04-25T00:35:32.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69d01f640 1 --2- 172.21.15.29:0/16153369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa684004810 0x7fa684004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69d820640 1 --2- 172.21.15.29:0/16153369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa684003bd0 0x7fa684003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69e021640 1 --2- 172.21.15.29:0/16153369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6840051d0 0x7fa68400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69d01f640 1 -- 172.21.15.29:0/16153369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6840051d0 msgr2=0x7fa68400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:32.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69d01f640 1 --2- 172.21.15.29:0/16153369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6840051d0 0x7fa68400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69d01f640 1 -- 172.21.15.29:0/16153369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa684003bd0 msgr2=0x7fa684003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:32.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69d01f640 1 --2- 172.21.15.29:0/16153369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa684003bd0 0x7fa684003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69d01f640 1 -- 172.21.15.29:0/16153369 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa684003530 con 0x7fa684004810 2024-04-25T00:35:32.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69e021640 1 -- 172.21.15.29:0/16153369 reap_dead start 2024-04-25T00:35:32.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69d820640 1 --2- 172.21.15.29:0/16153369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa684003bd0 0x7fa684003fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:32.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69d01f640 1 --2- 172.21.15.29:0/16153369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa684004810 0x7fa684004c90 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa688009ac0 tx=0x7fa68801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea799f0a838e1b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688009690 con 0x7fa684004810 2024-04-25T00:35:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6880a6470 con 0x7fa684004810 2024-04-25T00:35:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69c81e640 1 -- 172.21.15.29:0/16153369 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6880a6840 con 0x7fa684004810 2024-04-25T00:35:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69fa84640 1 -- 172.21.15.29:0/16153369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa684004810 msgr2=0x7fa684004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69fa84640 1 --2- 172.21.15.29:0/16153369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa684004810 0x7fa684004c90 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa688009ac0 tx=0x7fa68801c870 comp rx=0 tx=0).stop 2024-04-25T00:35:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69fa84640 1 -- 172.21.15.29:0/16153369 shutdown_connections 2024-04-25T00:35:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69fa84640 1 --2- 172.21.15.29:0/16153369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa684004810 0x7fa684004c90 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69fa84640 1 -- 172.21.15.29:0/16153369 >> 172.21.15.29:0/16153369 conn(0x7fa6980b1870 msgr2=0x7fa6980b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69fa84640 1 -- 172.21.15.29:0/16153369 shutdown_connections 2024-04-25T00:35:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.928+0000 7fa69fa84640 1 -- 172.21.15.29:0/16153369 wait complete. 2024-04-25T00:35:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69fa84640 1 Processor -- start 2024-04-25T00:35:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69fa84640 1 -- start start 2024-04-25T00:35:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69fa84640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa684004810 0x7fa6981f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69fa84640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa69813c710 0x7fa6980a9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69fa84640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa698070e00 0x7fa6980712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69fa84640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa698145a90 con 0x7fa684004810 2024-04-25T00:35:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69fa84640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa698145910 con 0x7fa69813c710 2024-04-25T00:35:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69fa84640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa69806e440 con 0x7fa698070e00 2024-04-25T00:35:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69d820640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa684004810 0x7fa6981f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69d820640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa684004810 0x7fa6981f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56212/0 (socket says 172.21.15.29:56212) 2024-04-25T00:35:32.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69d820640 1 -- 172.21.15.29:0/683518399 learned_addr learned my addr 172.21.15.29:0/683518399 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69d01f640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa69813c710 0x7fa6980a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69e021640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa698070e00 0x7fa6980712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69d820640 1 -- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa698070e00 msgr2=0x7fa6980712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69d820640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa698070e00 0x7fa6980712b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69d820640 1 -- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa69813c710 msgr2=0x7fa6980a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69d820640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa69813c710 0x7fa6980a9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69d820640 1 -- 172.21.15.29:0/683518399 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6981f4410 con 0x7fa684004810 2024-04-25T00:35:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69e021640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa698070e00 0x7fa6980712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69d820640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa684004810 0x7fa6981f3de0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa69400e7b0 tx=0x7fa69400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa68e7fc640 1 -- 172.21.15.29:0/683518399 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69400c6c0 con 0x7fa684004810 2024-04-25T00:35:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa68e7fc640 1 -- 172.21.15.29:0/683518399 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa694018070 con 0x7fa684004810 2024-04-25T00:35:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa68e7fc640 1 -- 172.21.15.29:0/683518399 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69400ca60 con 0x7fa684004810 2024-04-25T00:35:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69fa84640 1 -- 172.21.15.29:0/683518399 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6981f46f0 con 0x7fa684004810 2024-04-25T00:35:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69fa84640 1 -- 172.21.15.29:0/683518399 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6981f7470 con 0x7fa684004810 2024-04-25T00:35:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.932+0000 7fa69fa84640 1 -- 172.21.15.29:0/683518399 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa698069270 con 0x7fa684004810 2024-04-25T00:35:32.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.936+0000 7fa68e7fc640 1 -- 172.21.15.29:0/683518399 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fa69400cc00 con 0x7fa684004810 2024-04-25T00:35:32.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.936+0000 7fa68e7fc640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa67c067c20 0x7fa67c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:32.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.936+0000 7fa68e7fc640 1 -- 172.21.15.29:0/683518399 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa694085fa0 con 0x7fa684004810 2024-04-25T00:35:32.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.936+0000 7fa69d01f640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa67c067c20 0x7fa67c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:32.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.936+0000 7fa69d01f640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa67c067c20 0x7fa67c06a0e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa6880ad000 tx=0x7fa6880046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:32.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:32.940+0000 7fa68e7fc640 1 -- 172.21.15.29:0/683518399 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa694056760 con 0x7fa684004810 2024-04-25T00:35:33.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa69fa84640 1 -- 172.21.15.29:0/683518399 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa69813d1e0 con 0x7fa684004810 2024-04-25T00:35:33.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa68e7fc640 1 -- 172.21.15.29:0/683518399 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) ==== 80+0+0 (secure 0 0 0) 0x7fa694056060 con 0x7fa684004810 2024-04-25T00:35:33.110 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:35:33.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa69fa84640 1 -- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa67c067c20 msgr2=0x7fa67c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:33.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa69fa84640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa67c067c20 0x7fa67c06a0e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa6880ad000 tx=0x7fa6880046d0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa69fa84640 1 -- 172.21.15.29:0/683518399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa684004810 msgr2=0x7fa6981f3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:33.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa69fa84640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa684004810 0x7fa6981f3de0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa69400e7b0 tx=0x7fa69400ec70 comp rx=0 tx=0).stop 2024-04-25T00:35:33.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa69fa84640 1 -- 172.21.15.29:0/683518399 shutdown_connections 2024-04-25T00:35:33.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa69fa84640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa698070e00 0x7fa6980712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa69fa84640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fa67c067c20 0x7fa67c06a0e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa69fa84640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa69813c710 0x7fa6980a9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa69fa84640 1 --2- 172.21.15.29:0/683518399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa684004810 0x7fa6981f3de0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa69fa84640 1 -- 172.21.15.29:0/683518399 >> 172.21.15.29:0/683518399 conn(0x7fa6980b1870 msgr2=0x7fa69815ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:33.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa69fa84640 1 -- 172.21.15.29:0/683518399 shutdown_connections 2024-04-25T00:35:33.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.108+0000 7fa69fa84640 1 -- 172.21.15.29:0/683518399 wait complete. 2024-04-25T00:35:33.129 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-25T00:35:33.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:33.129 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T00:35:33.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34041]: fuse finished with error 0 and tester_r 0 2024-04-25T00:35:33.144 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:35:33.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:33.144 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:35:33.312 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:35:33.313 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:35:33.313 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:35:33.319 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:35:33.319 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:35:33.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:33.319 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:35:33.366 INFO:teuthology.orchestra.run.smithi029.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T00:35:33.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:35:33.367 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:35:33.367 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:35:33.367 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:35:33.373 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:35:33.373 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:35:33.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:35:33.374 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:35:33.420 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T00:35:33.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:35:33.421 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:35:33.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.548+0000 7fb94e519640 1 Processor -- start 2024-04-25T00:35:33.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.548+0000 7fb94e519640 1 -- start start 2024-04-25T00:35:33.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb94e519640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb94815e8d0 0x7fb9480cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:33.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb94e519640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9480b3720 con 0x7fb9480b4780 2024-04-25T00:35:33.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb94e519640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9480b39c0 con 0x7fb94815f3c0 2024-04-25T00:35:33.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb94e519640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb9480b42f0 con 0x7fb94815e8d0 2024-04-25T00:35:33.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb9477fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb9480b4780 0x7fb94815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55152/0 (socket says 172.21.15.29:55152) 2024-04-25T00:35:33.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb9477fe640 1 -- 172.21.15.29:0/2059229759 learned_addr learned my addr 172.21.15.29:0/2059229759 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:33.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb947fff640 1 --2- 172.21.15.29:0/2059229759 >> v2:172.21.15.29:3300/0 conn(0x7fb94815e8d0 0x7fb9480cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:33.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb946ffd640 1 -- 172.21.15.29:0/2059229759 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2539766567 0 0) 0x7fb9480b3720 con 0x7fb9480b4780 2024-04-25T00:35:33.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb946ffd640 1 -- 172.21.15.29:0/2059229759 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb930003530 con 0x7fb9480b4780 2024-04-25T00:35:33.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb946ffd640 1 -- 172.21.15.29:0/2059229759 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 219721765 0 0) 0x7fb930003530 con 0x7fb9480b4780 2024-04-25T00:35:33.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb947fff640 1 -- 172.21.15.29:0/2059229759 >> v1:172.21.15.157:6789/0 conn(0x7fb94815f3c0 legacy=0x7fb94815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:33.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb947fff640 1 -- 172.21.15.29:0/2059229759 >> v1:172.21.15.29:6789/0 conn(0x7fb9480b4780 legacy=0x7fb94815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:33.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb947fff640 1 -- 172.21.15.29:0/2059229759 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9480b3ca0 con 0x7fb94815e8d0 2024-04-25T00:35:33.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.552+0000 7fb947fff640 1 --2- 172.21.15.29:0/2059229759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb94815e8d0 0x7fb9480cc240 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb9380097c0 tx=0x7fb93801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5985655e23700b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:33.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb946ffd640 1 -- 172.21.15.29:0/2059229759 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9380b0070 con 0x7fb94815e8d0 2024-04-25T00:35:33.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb946ffd640 1 -- 172.21.15.29:0/2059229759 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9380ab400 con 0x7fb94815e8d0 2024-04-25T00:35:33.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb946ffd640 1 -- 172.21.15.29:0/2059229759 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c001680 con 0x7fb94815e8d0 2024-04-25T00:35:33.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 -- 172.21.15.29:0/2059229759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb94815e8d0 msgr2=0x7fb9480cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:33.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 --2- 172.21.15.29:0/2059229759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb94815e8d0 0x7fb9480cc240 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb9380097c0 tx=0x7fb93801c3c0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 -- 172.21.15.29:0/2059229759 shutdown_connections 2024-04-25T00:35:33.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 --2- 172.21.15.29:0/2059229759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb94815e8d0 0x7fb9480cc240 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 -- 172.21.15.29:0/2059229759 >> 172.21.15.29:0/2059229759 conn(0x7fb9480b1870 msgr2=0x7fb9480b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:33.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 -- 172.21.15.29:0/2059229759 shutdown_connections 2024-04-25T00:35:33.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 -- 172.21.15.29:0/2059229759 wait complete. 2024-04-25T00:35:33.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 Processor -- start 2024-04-25T00:35:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 -- start start 2024-04-25T00:35:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9480b4780 0x7fb948143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb94815e8d0 0x7fb94815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb94815f3c0 0x7fb9480716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb948168e00 con 0x7fb94815f3c0 2024-04-25T00:35:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb948168c80 con 0x7fb9480b4780 2024-04-25T00:35:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb948168f80 con 0x7fb94815e8d0 2024-04-25T00:35:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94cab6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb94815f3c0 0x7fb9480716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94cab6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb94815f3c0 0x7fb9480716f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56230/0 (socket says 172.21.15.29:56230) 2024-04-25T00:35:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94cab6640 1 -- 172.21.15.29:0/2076063501 learned_addr learned my addr 172.21.15.29:0/2076063501 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb947fff640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9480b4780 0x7fb948143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb9477fe640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb94815e8d0 0x7fb94815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:33.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94cab6640 1 -- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb94815e8d0 msgr2=0x7fb94815b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:33.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94cab6640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb94815e8d0 0x7fb94815b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94cab6640 1 -- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9480b4780 msgr2=0x7fb948143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:33.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94cab6640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9480b4780 0x7fb948143440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94cab6640 1 -- 172.21.15.29:0/2076063501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb948135570 con 0x7fb94815f3c0 2024-04-25T00:35:33.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94cab6640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb94815f3c0 0x7fb9480716f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb94000c730 tx=0x7fb94000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:33.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb944ff9640 1 -- 172.21.15.29:0/2076063501 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940003b80 con 0x7fb94815f3c0 2024-04-25T00:35:33.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb944ff9640 1 -- 172.21.15.29:0/2076063501 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb940003d20 con 0x7fb94815f3c0 2024-04-25T00:35:33.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 -- 172.21.15.29:0/2076063501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94815bc80 con 0x7fb94815f3c0 2024-04-25T00:35:33.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb944ff9640 1 -- 172.21.15.29:0/2076063501 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94000aa30 con 0x7fb94815f3c0 2024-04-25T00:35:33.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.556+0000 7fb94e519640 1 -- 172.21.15.29:0/2076063501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb948072220 con 0x7fb94815f3c0 2024-04-25T00:35:33.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.560+0000 7fb94e519640 1 -- 172.21.15.29:0/2076063501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb948069270 con 0x7fb94815f3c0 2024-04-25T00:35:33.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.560+0000 7fb944ff9640 1 -- 172.21.15.29:0/2076063501 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb94000abd0 con 0x7fb94815f3c0 2024-04-25T00:35:33.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.560+0000 7fb944ff9640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb934067c20 0x7fb93406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:33.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.560+0000 7fb944ff9640 1 -- 172.21.15.29:0/2076063501 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb9400860f0 con 0x7fb94815f3c0 2024-04-25T00:35:33.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.560+0000 7fb947fff640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb934067c20 0x7fb93406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:33.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.560+0000 7fb947fff640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb934067c20 0x7fb93406a0e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb9380ad000 tx=0x7fb9380a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:33.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.564+0000 7fb944ff9640 1 -- 172.21.15.29:0/2076063501 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb9400567b0 con 0x7fb94815f3c0 2024-04-25T00:35:33.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.756+0000 7fb94e519640 1 -- 172.21.15.29:0/2076063501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb94806e7b0 con 0x7fb94815f3c0 2024-04-25T00:35:33.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.756+0000 7fb944ff9640 1 -- 172.21.15.29:0/2076063501 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5175 (secure 0 0 0) 0x7fb9400560b0 con 0x7fb94815f3c0 2024-04-25T00:35:33.760 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:33.760 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/56623075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":56623075},{"type":"v1","addr":"172.21.15.157:6836","nonce":56623075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/748098725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":748098725},{"type":"v1","addr":"172.21.15.157:6837","nonce":748098725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2264413556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2264413556},{"type":"v1","addr":"172.21.15.29:6833","nonce":2264413556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:07.114520+0000","modified":"2024-04-25T00:35:10.185595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6372},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/802824676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":802824676},{"type":"v1","addr":"172.21.15.29:6835","nonce":802824676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372]},"id":4}]} 2024-04-25T00:35:33.760 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 35 2024-04-25T00:35:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.760+0000 7fb94e519640 1 -- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb934067c20 msgr2=0x7fb93406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.760+0000 7fb94e519640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb934067c20 0x7fb93406a0e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb9380ad000 tx=0x7fb9380a6000 comp rx=0 tx=0).stop 2024-04-25T00:35:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.760+0000 7fb94e519640 1 -- 172.21.15.29:0/2076063501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb94815f3c0 msgr2=0x7fb9480716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.760+0000 7fb94e519640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb94815f3c0 0x7fb9480716f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb94000c730 tx=0x7fb94000cbf0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.760+0000 7fb94e519640 1 -- 172.21.15.29:0/2076063501 shutdown_connections 2024-04-25T00:35:33.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.760+0000 7fb94e519640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb94815f3c0 0x7fb9480716f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.760+0000 7fb94e519640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb94815e8d0 0x7fb94815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.760+0000 7fb94e519640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb934067c20 0x7fb93406a0e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.760+0000 7fb94e519640 1 --2- 172.21.15.29:0/2076063501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9480b4780 0x7fb948143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.760+0000 7fb94e519640 1 -- 172.21.15.29:0/2076063501 >> 172.21.15.29:0/2076063501 conn(0x7fb9480b1870 msgr2=0x7fb9480b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:33.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.760+0000 7fb94e519640 1 -- 172.21.15.29:0/2076063501 shutdown_connections 2024-04-25T00:35:33.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.760+0000 7fb94e519640 1 -- 172.21.15.29:0/2076063501 wait complete. 2024-04-25T00:35:33.777 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:33.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.896+0000 7f1d93234640 1 Processor -- start 2024-04-25T00:35:33.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.896+0000 7f1d93234640 1 -- start start 2024-04-25T00:35:33.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.896+0000 7f1d93234640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1d8c15e200 0x7f1d8c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:33.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.896+0000 7f1d93234640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1d8c0d2c60 0x7f1d8c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:33.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.896+0000 7f1d93234640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d8c0cfc10 con 0x7f1d8c0cf690 2024-04-25T00:35:33.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.896+0000 7f1d93234640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1d8c0b38d0 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.896+0000 7f1d93234640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1d8c0cfef0 con 0x7f1d8c15e200 2024-04-25T00:35:33.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d917d1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1d8c0d2c60 0x7f1d8c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:33.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d917d1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1d8c0d2c60 0x7f1d8c0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56242/0 (socket says 172.21.15.29:56242) 2024-04-25T00:35:33.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d917d1640 1 -- 172.21.15.29:0/1162933983 learned_addr learned my addr 172.21.15.29:0/1162933983 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:33.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d90fd0640 1 --2- 172.21.15.29:0/1162933983 >> v2:172.21.15.157:3300/0 conn(0x7f1d8c15e200 0x7f1d8c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:33.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d917d1640 1 -- 172.21.15.29:0/1162933983 >> v2:172.21.15.157:3300/0 conn(0x7f1d8c15e200 msgr2=0x7f1d8c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:33.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d917d1640 1 --2- 172.21.15.29:0/1162933983 >> v2:172.21.15.157:3300/0 conn(0x7f1d8c15e200 0x7f1d8c0d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d917d1640 1 -- 172.21.15.29:0/1162933983 >> v1:172.21.15.157:6789/0 conn(0x7f1d8c0cf690 legacy=0x7f1d8c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:33.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d917d1640 1 -- 172.21.15.29:0/1162933983 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8c0b3b20 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d90fd0640 1 --2- 172.21.15.29:0/1162933983 >> v2:172.21.15.157:3300/0 conn(0x7f1d8c15e200 0x7f1d8c0d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:33.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d917d1640 1 --2- 172.21.15.29:0/1162933983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d8c0d2c60 0x7f1d8c0b4320 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c08d8f0 tx=0x7f1d8401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8387ce18845a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:33.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d8b7fe640 1 -- 172.21.15.29:0/1162933983 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d840a5860 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d8b7fe640 1 -- 172.21.15.29:0/1162933983 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d840a5a00 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d8b7fe640 1 -- 172.21.15.29:0/1162933983 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d840a5dd0 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d93234640 1 -- 172.21.15.29:0/1162933983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d8c0d2c60 msgr2=0x7f1d8c0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:33.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d93234640 1 --2- 172.21.15.29:0/1162933983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d8c0d2c60 0x7f1d8c0b4320 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c08d8f0 tx=0x7f1d8401c980 comp rx=0 tx=0).stop 2024-04-25T00:35:33.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d93234640 1 -- 172.21.15.29:0/1162933983 shutdown_connections 2024-04-25T00:35:33.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d93234640 1 --2- 172.21.15.29:0/1162933983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d8c0d2c60 0x7f1d8c0b4320 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d93234640 1 --2- 172.21.15.29:0/1162933983 >> v2:172.21.15.157:3300/0 conn(0x7f1d8c15e200 0x7f1d8c0d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d93234640 1 -- 172.21.15.29:0/1162933983 >> 172.21.15.29:0/1162933983 conn(0x7f1d8c09d280 msgr2=0x7f1d8c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:33.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d93234640 1 -- 172.21.15.29:0/1162933983 shutdown_connections 2024-04-25T00:35:33.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.900+0000 7f1d93234640 1 -- 172.21.15.29:0/1162933983 wait complete. 2024-04-25T00:35:33.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d93234640 1 Processor -- start 2024-04-25T00:35:33.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d93234640 1 -- start start 2024-04-25T00:35:33.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d93234640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d8c0cf690 0x7f1d8c0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:33.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d93234640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d8c0d2c60 0x7f1d8c154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:33.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d93234640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d8c15e200 0x7f1d8c155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:33.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d93234640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c0b51f0 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d93234640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c0b5070 con 0x7f1d8c15e200 2024-04-25T00:35:33.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d93234640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1d8c0b5370 con 0x7f1d8c0cf690 2024-04-25T00:35:33.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d8c0d2c60 0x7f1d8c154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:33.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d8c0d2c60 0x7f1d8c154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56254/0 (socket says 172.21.15.29:56254) 2024-04-25T00:35:33.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d8bfff640 1 -- 172.21.15.29:0/2899330761 learned_addr learned my addr 172.21.15.29:0/2899330761 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:33.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d917d1640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d8c15e200 0x7f1d8c155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:33.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d90fd0640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d8c0cf690 0x7f1d8c0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:33.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d8bfff640 1 -- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d8c0cf690 msgr2=0x7f1d8c0b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d8bfff640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d8c0cf690 0x7f1d8c0b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d8bfff640 1 -- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d8c15e200 msgr2=0x7f1d8c155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d8bfff640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d8c15e200 0x7f1d8c155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d8bfff640 1 -- 172.21.15.29:0/2899330761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8c155dd0 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d917d1640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d8c15e200 0x7f1d8c155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d8bfff640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d8c0d2c60 0x7f1d8c154fc0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000c490 tx=0x7f1d80001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d897fa640 1 -- 172.21.15.29:0/2899330761 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80017070 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d897fa640 1 -- 172.21.15.29:0/2899330761 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d80003600 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d93234640 1 -- 172.21.15.29:0/2899330761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8c1560b0 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d897fa640 1 -- 172.21.15.29:0/2899330761 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80003a30 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d93234640 1 -- 172.21.15.29:0/2899330761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d8c156670 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d93234640 1 -- 172.21.15.29:0/2899330761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d8c069270 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.904+0000 7f1d897fa640 1 -- 172.21.15.29:0/2899330761 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f1d8001a070 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.908+0000 7f1d897fa640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d78067c20 0x7f1d7806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:33.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.908+0000 7f1d897fa640 1 -- 172.21.15.29:0/2899330761 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1d800855e0 con 0x7f1d8c0d2c60 2024-04-25T00:35:33.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.908+0000 7f1d90fd0640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d78067c20 0x7f1d7806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:33.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.908+0000 7f1d90fd0640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d78067c20 0x7f1d7806a0e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c006fd0 tx=0x7f1d7c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:33.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:33.912+0000 7f1d897fa640 1 -- 172.21.15.29:0/2899330761 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1d80055d60 con 0x7f1d8c0d2c60 2024-04-25T00:35:34.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.072+0000 7f1d93234640 1 -- 172.21.15.29:0/2899330761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d8c156840 con 0x7f1d8c0d2c60 2024-04-25T00:35:34.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.076+0000 7f1d897fa640 1 -- 172.21.15.29:0/2899330761 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16617 (secure 0 0 0) 0x7f1d80055660 con 0x7f1d8c0d2c60 2024-04-25T00:35:34.079 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:34.080 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":58,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:08.767613+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:05.458923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:06.104267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:34.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.080+0000 7f1d93234640 1 -- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d78067c20 msgr2=0x7f1d7806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.080+0000 7f1d93234640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d78067c20 0x7f1d7806a0e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c006fd0 tx=0x7f1d7c008040 comp rx=0 tx=0).stop 2024-04-25T00:35:34.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.080+0000 7f1d93234640 1 -- 172.21.15.29:0/2899330761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d8c0d2c60 msgr2=0x7f1d8c154fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.080+0000 7f1d93234640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d8c0d2c60 0x7f1d8c154fc0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000c490 tx=0x7f1d80001e80 comp rx=0 tx=0).stop 2024-04-25T00:35:34.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.080+0000 7f1d93234640 1 -- 172.21.15.29:0/2899330761 shutdown_connections 2024-04-25T00:35:34.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.080+0000 7f1d93234640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f1d78067c20 0x7f1d7806a0e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.080+0000 7f1d93234640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d8c15e200 0x7f1d8c155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.080+0000 7f1d93234640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d8c0d2c60 0x7f1d8c154fc0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.080+0000 7f1d93234640 1 --2- 172.21.15.29:0/2899330761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d8c0cf690 0x7f1d8c0b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.080+0000 7f1d93234640 1 -- 172.21.15.29:0/2899330761 >> 172.21.15.29:0/2899330761 conn(0x7f1d8c09d280 msgr2=0x7f1d8c06c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:34.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.080+0000 7f1d93234640 1 -- 172.21.15.29:0/2899330761 shutdown_connections 2024-04-25T00:35:34.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.080+0000 7f1d93234640 1 -- 172.21.15.29:0/2899330761 wait complete. 2024-04-25T00:35:34.099 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:35:05.458923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:35:06.104267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:35:34.099 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:35:34.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf522c9640 1 Processor -- start 2024-04-25T00:35:34.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf522c9640 1 -- start start 2024-04-25T00:35:34.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf522c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c0b3140 0x7fcf4c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf522c9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcf4c0b3c30 0x7fcf4c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf522c9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf4c0b41e0 con 0x7fcf4c0b4b10 2024-04-25T00:35:34.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf522c9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcf4c06d960 con 0x7fcf4c0b3c30 2024-04-25T00:35:34.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf522c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c06dc40 con 0x7fcf4c0b3140 2024-04-25T00:35:34.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf4bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c0b3140 0x7fcf4c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf50866640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcf4c0b3c30 0x7fcf4c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf4bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c0b3140 0x7fcf4c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:47364/0 (socket says 172.21.15.29:47364) 2024-04-25T00:35:34.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf4bfff640 1 -- 172.21.15.29:0/2016977485 learned_addr learned my addr 172.21.15.29:0/2016977485 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:34.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf50866640 1 -- 172.21.15.29:0/2016977485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c0b3140 msgr2=0x7fcf4c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:34.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf50866640 1 --2- 172.21.15.29:0/2016977485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c0b3140 0x7fcf4c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf50866640 1 -- 172.21.15.29:0/2016977485 >> v1:172.21.15.157:6789/0 conn(0x7fcf4c0b4b10 legacy=0x7fcf4c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf50866640 1 -- 172.21.15.29:0/2016977485 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c06df20 con 0x7fcf4c0b3c30 2024-04-25T00:35:34.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.220+0000 7fcf4bfff640 1 --2- 172.21.15.29:0/2016977485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c0b3140 0x7fcf4c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:34.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.224+0000 7fcf50866640 1 --2- 172.21.15.29:0/2016977485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf4c0b3c30 0x7fcf4c0ce240 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009d90 tx=0x7fcf4401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74017a2649f4590 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:34.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.224+0000 7fcf4affd640 1 -- 172.21.15.29:0/2016977485 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf440b2070 con 0x7fcf4c0b3c30 2024-04-25T00:35:34.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.224+0000 7fcf4affd640 1 -- 172.21.15.29:0/2016977485 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf440a5830 con 0x7fcf4c0b3c30 2024-04-25T00:35:34.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.224+0000 7fcf4affd640 1 -- 172.21.15.29:0/2016977485 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf440a5c00 con 0x7fcf4c0b3c30 2024-04-25T00:35:34.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.224+0000 7fcf522c9640 1 -- 172.21.15.29:0/2016977485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf4c0b3c30 msgr2=0x7fcf4c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.224+0000 7fcf522c9640 1 --2- 172.21.15.29:0/2016977485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf4c0b3c30 0x7fcf4c0ce240 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009d90 tx=0x7fcf4401c970 comp rx=0 tx=0).stop 2024-04-25T00:35:34.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.224+0000 7fcf522c9640 1 -- 172.21.15.29:0/2016977485 shutdown_connections 2024-04-25T00:35:34.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.224+0000 7fcf522c9640 1 --2- 172.21.15.29:0/2016977485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf4c0b3c30 0x7fcf4c0ce240 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.224+0000 7fcf522c9640 1 --2- 172.21.15.29:0/2016977485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c0b3140 0x7fcf4c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.224+0000 7fcf522c9640 1 -- 172.21.15.29:0/2016977485 >> 172.21.15.29:0/2016977485 conn(0x7fcf4c0b1870 msgr2=0x7fcf4c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.224+0000 7fcf522c9640 1 -- 172.21.15.29:0/2016977485 shutdown_connections 2024-04-25T00:35:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.224+0000 7fcf522c9640 1 -- 172.21.15.29:0/2016977485 wait complete. 2024-04-25T00:35:34.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf522c9640 1 Processor -- start 2024-04-25T00:35:34.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf522c9640 1 -- start start 2024-04-25T00:35:34.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf522c9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf4c0b3140 0x7fcf4c15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf522c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf4c0b3c30 0x7fcf4c134210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf522c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c0b4b10 0x7fcf4c134750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf522c9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c168d80 con 0x7fcf4c0b3140 2024-04-25T00:35:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf522c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c168c00 con 0x7fcf4c0b3c30 2024-04-25T00:35:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf522c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c168f00 con 0x7fcf4c0b4b10 2024-04-25T00:35:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf4bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf4c0b3140 0x7fcf4c15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf4bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf4c0b3140 0x7fcf4c15a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56260/0 (socket says 172.21.15.29:56260) 2024-04-25T00:35:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf4bfff640 1 -- 172.21.15.29:0/3308113761 learned_addr learned my addr 172.21.15.29:0/3308113761 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf4b7fe640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf4c0b3c30 0x7fcf4c134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf50866640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c0b4b10 0x7fcf4c134750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf4bfff640 1 -- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c0b4b10 msgr2=0x7fcf4c134750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf4bfff640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c0b4b10 0x7fcf4c134750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf4bfff640 1 -- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf4c0b3c30 msgr2=0x7fcf4c134210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf4bfff640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf4c0b3c30 0x7fcf4c134210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf4bfff640 1 -- 172.21.15.29:0/3308113761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c141930 con 0x7fcf4c0b3140 2024-04-25T00:35:34.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf4bfff640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf4c0b3140 0x7fcf4c15a740 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00e930 tx=0x7fcf3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:34.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf48ff9640 1 -- 172.21.15.29:0/3308113761 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c017070 con 0x7fcf4c0b3140 2024-04-25T00:35:34.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf48ff9640 1 -- 172.21.15.29:0/3308113761 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf3c00ca00 con 0x7fcf4c0b3140 2024-04-25T00:35:34.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf48ff9640 1 -- 172.21.15.29:0/3308113761 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00cdd0 con 0x7fcf4c0b3140 2024-04-25T00:35:34.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf522c9640 1 -- 172.21.15.29:0/3308113761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4c141c10 con 0x7fcf4c0b3140 2024-04-25T00:35:34.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.228+0000 7fcf522c9640 1 -- 172.21.15.29:0/3308113761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4c1421d0 con 0x7fcf4c0b3140 2024-04-25T00:35:34.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.232+0000 7fcf522c9640 1 -- 172.21.15.29:0/3308113761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4c069270 con 0x7fcf4c0b3140 2024-04-25T00:35:34.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.232+0000 7fcf48ff9640 1 -- 172.21.15.29:0/3308113761 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fcf3c01a070 con 0x7fcf4c0b3140 2024-04-25T00:35:34.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.232+0000 7fcf48ff9640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcf38067c20 0x7fcf3806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.232+0000 7fcf48ff9640 1 -- 172.21.15.29:0/3308113761 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fcf3c0863d0 con 0x7fcf4c0b3140 2024-04-25T00:35:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.232+0000 7fcf4b7fe640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcf38067c20 0x7fcf3806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.232+0000 7fcf4b7fe640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcf38067c20 0x7fcf3806a0e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c139950 tx=0x7fcf40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:34.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.236+0000 7fcf48ff9640 1 -- 172.21.15.29:0/3308113761 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcf3c056b50 con 0x7fcf4c0b3140 2024-04-25T00:35:34.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.400+0000 7fcf522c9640 1 -- 172.21.15.29:0/3308113761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf4c0b0270 con 0x7fcf4c0b3140 2024-04-25T00:35:34.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.404+0000 7fcf48ff9640 1 -- 172.21.15.29:0/3308113761 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7fcf3c056450 con 0x7fcf4c0b3140 2024-04-25T00:35:34.405 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:35:34.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.404+0000 7fcf522c9640 1 -- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcf38067c20 msgr2=0x7fcf3806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.404+0000 7fcf522c9640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcf38067c20 0x7fcf3806a0e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c139950 tx=0x7fcf40009040 comp rx=0 tx=0).stop 2024-04-25T00:35:34.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.404+0000 7fcf522c9640 1 -- 172.21.15.29:0/3308113761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf4c0b3140 msgr2=0x7fcf4c15a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.404+0000 7fcf522c9640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf4c0b3140 0x7fcf4c15a740 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00e930 tx=0x7fcf3c00edf0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.404+0000 7fcf522c9640 1 -- 172.21.15.29:0/3308113761 shutdown_connections 2024-04-25T00:35:34.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.404+0000 7fcf522c9640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c0b4b10 0x7fcf4c134750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.404+0000 7fcf522c9640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcf38067c20 0x7fcf3806a0e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.404+0000 7fcf522c9640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf4c0b3c30 0x7fcf4c134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.404+0000 7fcf522c9640 1 --2- 172.21.15.29:0/3308113761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf4c0b3140 0x7fcf4c15a740 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.404+0000 7fcf522c9640 1 -- 172.21.15.29:0/3308113761 >> 172.21.15.29:0/3308113761 conn(0x7fcf4c0b1870 msgr2=0x7fcf4c144100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:34.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.404+0000 7fcf522c9640 1 -- 172.21.15.29:0/3308113761 shutdown_connections 2024-04-25T00:35:34.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.404+0000 7fcf522c9640 1 -- 172.21.15.29:0/3308113761 wait complete. 2024-04-25T00:35:34.424 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:35:34.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faaecc63640 1 Processor -- start 2024-04-25T00:35:34.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faaecc63640 1 -- start start 2024-04-25T00:35:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faaecc63640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faae806c950 0x7faae806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faaecc63640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faae815e440 0x7faae80d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faaecc63640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faae80cc3b0 con 0x7faae80d0ff0 2024-04-25T00:35:34.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faaecc63640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faae806d270 con 0x7faae815e440 2024-04-25T00:35:34.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faaecc63640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faae80b11d0 con 0x7faae806c950 2024-04-25T00:35:34.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faae659c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faae806c950 0x7faae806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faae5d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faae815e440 0x7faae80d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faae659c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faae806c950 0x7faae806cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:47374/0 (socket says 172.21.15.29:47374) 2024-04-25T00:35:34.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faae659c640 1 -- 172.21.15.29:0/3440256155 learned_addr learned my addr 172.21.15.29:0/3440256155 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:34.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faae5d9b640 1 --2- 172.21.15.29:0/3440256155 >> v2:172.21.15.157:3300/0 conn(0x7faae815e440 0x7faae80d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:59770/0 (socket says 172.21.15.29:59770) 2024-04-25T00:35:34.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faae5d9b640 1 -- 172.21.15.29:0/3440256155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faae806c950 msgr2=0x7faae806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:34.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faae5d9b640 1 --2- 172.21.15.29:0/3440256155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faae806c950 0x7faae806cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faae5d9b640 1 -- 172.21.15.29:0/3440256155 >> v1:172.21.15.157:6789/0 conn(0x7faae80d0ff0 legacy=0x7faae80ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faae5d9b640 1 -- 172.21.15.29:0/3440256155 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae80cc650 con 0x7faae815e440 2024-04-25T00:35:34.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.552+0000 7faae659c640 1 --2- 172.21.15.29:0/3440256155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faae806c950 0x7faae806cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:34.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faae5d9b640 1 --2- 172.21.15.29:0/3440256155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faae815e440 0x7faae80d0ab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7faadc0097c0 tx=0x7faadc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d882a47feff47b93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:34.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faae559a640 1 -- 172.21.15.29:0/3440256155 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc0b0070 con 0x7faae815e440 2024-04-25T00:35:34.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faae559a640 1 -- 172.21.15.29:0/3440256155 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faadc0ab600 con 0x7faae815e440 2024-04-25T00:35:34.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faae559a640 1 -- 172.21.15.29:0/3440256155 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc0ab9d0 con 0x7faae815e440 2024-04-25T00:35:34.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faaecc63640 1 -- 172.21.15.29:0/3440256155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faae815e440 msgr2=0x7faae80d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faaecc63640 1 --2- 172.21.15.29:0/3440256155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faae815e440 0x7faae80d0ab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7faadc0097c0 tx=0x7faadc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faaecc63640 1 -- 172.21.15.29:0/3440256155 shutdown_connections 2024-04-25T00:35:34.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faaecc63640 1 --2- 172.21.15.29:0/3440256155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faae815e440 0x7faae80d0ab0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faaecc63640 1 --2- 172.21.15.29:0/3440256155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faae806c950 0x7faae806cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faaecc63640 1 -- 172.21.15.29:0/3440256155 >> 172.21.15.29:0/3440256155 conn(0x7faae80d1d60 msgr2=0x7faae80d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:34.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faaecc63640 1 -- 172.21.15.29:0/3440256155 shutdown_connections 2024-04-25T00:35:34.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faaecc63640 1 -- 172.21.15.29:0/3440256155 wait complete. 2024-04-25T00:35:34.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faaecc63640 1 Processor -- start 2024-04-25T00:35:34.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.556+0000 7faaecc63640 1 -- start start 2024-04-25T00:35:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faaecc63640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faae806c950 0x7faae81523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faaecc63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae80d0ff0 0x7faae80cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faaecc63640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faae815e440 0x7faae80b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faaecc63640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faae806c520 con 0x7faae80d0ff0 2024-04-25T00:35:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faaecc63640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faae80ce420 con 0x7faae806c950 2024-04-25T00:35:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faaecc63640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faae816ae20 con 0x7faae815e440 2024-04-25T00:35:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faae5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae80d0ff0 0x7faae80cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faae5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae80d0ff0 0x7faae80cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56272/0 (socket says 172.21.15.29:56272) 2024-04-25T00:35:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faae5d9b640 1 -- 172.21.15.29:0/258906281 learned_addr learned my addr 172.21.15.29:0/258906281 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faae6d9d640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faae815e440 0x7faae80b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faae659c640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faae806c950 0x7faae81523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faae5d9b640 1 -- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faae815e440 msgr2=0x7faae80b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faae5d9b640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faae815e440 0x7faae80b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faae5d9b640 1 -- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faae806c950 msgr2=0x7faae81523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faae5d9b640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faae806c950 0x7faae81523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faae5d9b640 1 -- 172.21.15.29:0/258906281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae80cbc40 con 0x7faae80d0ff0 2024-04-25T00:35:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faae5d9b640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae80d0ff0 0x7faae80cb700 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7faadc0ae040 tx=0x7faadc01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:34.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faad6ffd640 1 -- 172.21.15.29:0/258906281 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc0b0050 con 0x7faae80d0ff0 2024-04-25T00:35:34.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faad6ffd640 1 -- 172.21.15.29:0/258906281 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faadc0bc070 con 0x7faae80d0ff0 2024-04-25T00:35:34.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faad6ffd640 1 -- 172.21.15.29:0/258906281 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc0b5870 con 0x7faae80d0ff0 2024-04-25T00:35:34.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faaecc63640 1 -- 172.21.15.29:0/258906281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae8156ae0 con 0x7faae80d0ff0 2024-04-25T00:35:34.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faaecc63640 1 -- 172.21.15.29:0/258906281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae815daa0 con 0x7faae80d0ff0 2024-04-25T00:35:34.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faaecc63640 1 -- 172.21.15.29:0/258906281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae8069270 con 0x7faae80d0ff0 2024-04-25T00:35:34.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faad6ffd640 1 -- 172.21.15.29:0/258906281 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7faadc0b5a10 con 0x7faae80d0ff0 2024-04-25T00:35:34.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faad6ffd640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faac8067c20 0x7faac806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faad6ffd640 1 -- 172.21.15.29:0/258906281 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7faadc120a60 con 0x7faae80d0ff0 2024-04-25T00:35:34.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.560+0000 7faae659c640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faac8067c20 0x7faac806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.564+0000 7faae659c640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faac8067c20 0x7faac806a0e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7faae81565e0 tx=0x7faad8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:34.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.564+0000 7faad6ffd640 1 -- 172.21.15.29:0/258906281 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faae80cbc40 con 0x7faae80d0ff0 2024-04-25T00:35:34.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faaecc63640 1 -- 172.21.15.29:0/258906281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faae815dc70 con 0x7faae80d0ff0 2024-04-25T00:35:34.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faad6ffd640 1 -- 172.21.15.29:0/258906281 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) ==== 104+0+11 (secure 0 0 0) 0x7faadc016090 con 0x7faae80d0ff0 2024-04-25T00:35:34.730 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:35:34.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faaecc63640 1 -- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faac8067c20 msgr2=0x7faac806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faaecc63640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faac8067c20 0x7faac806a0e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7faae81565e0 tx=0x7faad8008040 comp rx=0 tx=0).stop 2024-04-25T00:35:34.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faaecc63640 1 -- 172.21.15.29:0/258906281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae80d0ff0 msgr2=0x7faae80cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faaecc63640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae80d0ff0 0x7faae80cb700 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7faadc0ae040 tx=0x7faadc01ace0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faaecc63640 1 -- 172.21.15.29:0/258906281 shutdown_connections 2024-04-25T00:35:34.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faaecc63640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faae815e440 0x7faae80b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faaecc63640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae80d0ff0 0x7faae80cb700 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faaecc63640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7faac8067c20 0x7faac806a0e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faaecc63640 1 --2- 172.21.15.29:0/258906281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faae806c950 0x7faae81523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faaecc63640 1 -- 172.21.15.29:0/258906281 >> 172.21.15.29:0/258906281 conn(0x7faae80d1d60 msgr2=0x7faae80b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:34.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faaecc63640 1 -- 172.21.15.29:0/258906281 shutdown_connections 2024-04-25T00:35:34.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.728+0000 7faaecc63640 1 -- 172.21.15.29:0/258906281 wait complete. 2024-04-25T00:35:34.750 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:35:34.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4be0ab640 1 Processor -- start 2024-04-25T00:35:34.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4be0ab640 1 -- start start 2024-04-25T00:35:34.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4be0ab640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b80b3140 0x7fe4b806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4be0ab640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe4b80b4b10 0x7fe4b80b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4be0ab640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe4b80b3a60 0x7fe4b815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4be0ab640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b80b41e0 con 0x7fe4b80b4b10 2024-04-25T00:35:34.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4be0ab640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b806d960 con 0x7fe4b80b3a60 2024-04-25T00:35:34.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4be0ab640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b806dc40 con 0x7fe4b80b3140 2024-04-25T00:35:34.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4b6ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe4b80b4b10 0x7fe4b80b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4b6ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe4b80b4b10 0x7fe4b80b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56288/0 (socket says 172.21.15.29:56288) 2024-04-25T00:35:34.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4b6ffd640 1 -- 172.21.15.29:0/2509125372 learned_addr learned my addr 172.21.15.29:0/2509125372 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:34.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4b77fe640 1 --2- 172.21.15.29:0/2509125372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b80b3140 0x7fe4b806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4b7fff640 1 --2- 172.21.15.29:0/2509125372 >> v2:172.21.15.157:3300/0 conn(0x7fe4b80b3a60 0x7fe4b815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4b6ffd640 1 -- 172.21.15.29:0/2509125372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b80b3140 msgr2=0x7fe4b806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:34.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4b6ffd640 1 --2- 172.21.15.29:0/2509125372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b80b3140 0x7fe4b806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4b6ffd640 1 -- 172.21.15.29:0/2509125372 >> v2:172.21.15.157:3300/0 conn(0x7fe4b80b3a60 msgr2=0x7fe4b815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:34.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4b6ffd640 1 --2- 172.21.15.29:0/2509125372 >> v2:172.21.15.157:3300/0 conn(0x7fe4b80b3a60 0x7fe4b815c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.872+0000 7fe4b6ffd640 1 -- 172.21.15.29:0/2509125372 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b806df20 con 0x7fe4b80b4b10 2024-04-25T00:35:34.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4b6ffd640 1 --2- 172.21.15.29:0/2509125372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4b80b4b10 0x7fe4b80b3520 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80b4790 tx=0x7fe4ac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9ceeb5ff847f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:34.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4b67fc640 1 -- 172.21.15.29:0/2509125372 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac0b0070 con 0x7fe4b80b4b10 2024-04-25T00:35:34.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4b67fc640 1 -- 172.21.15.29:0/2509125372 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ac0ab600 con 0x7fe4b80b4b10 2024-04-25T00:35:34.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4b67fc640 1 -- 172.21.15.29:0/2509125372 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac0ab9d0 con 0x7fe4b80b4b10 2024-04-25T00:35:34.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4be0ab640 1 -- 172.21.15.29:0/2509125372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4b80b4b10 msgr2=0x7fe4b80b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4be0ab640 1 --2- 172.21.15.29:0/2509125372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4b80b4b10 0x7fe4b80b3520 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80b4790 tx=0x7fe4ac01c520 comp rx=0 tx=0).stop 2024-04-25T00:35:34.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4be0ab640 1 -- 172.21.15.29:0/2509125372 shutdown_connections 2024-04-25T00:35:34.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4be0ab640 1 --2- 172.21.15.29:0/2509125372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4b80b4b10 0x7fe4b80b3520 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4be0ab640 1 --2- 172.21.15.29:0/2509125372 >> v2:172.21.15.157:3300/0 conn(0x7fe4b80b3a60 0x7fe4b815c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4be0ab640 1 --2- 172.21.15.29:0/2509125372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b80b3140 0x7fe4b806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4be0ab640 1 -- 172.21.15.29:0/2509125372 >> 172.21.15.29:0/2509125372 conn(0x7fe4b80b1870 msgr2=0x7fe4b80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:34.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4be0ab640 1 -- 172.21.15.29:0/2509125372 shutdown_connections 2024-04-25T00:35:34.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4be0ab640 1 -- 172.21.15.29:0/2509125372 wait complete. 2024-04-25T00:35:34.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4be0ab640 1 Processor -- start 2024-04-25T00:35:34.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.876+0000 7fe4be0ab640 1 -- start start 2024-04-25T00:35:34.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4be0ab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4b80b3140 0x7fe4b813a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4be0ab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4b80b3a60 0x7fe4b813ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4be0ab640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b80b4b10 0x7fe4b813b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4be0ab640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b80ce3c0 con 0x7fe4b80b3a60 2024-04-25T00:35:34.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4be0ab640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b80ce240 con 0x7fe4b80b3140 2024-04-25T00:35:34.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4be0ab640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b80ce540 con 0x7fe4b80b4b10 2024-04-25T00:35:34.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4b6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4b80b3a60 0x7fe4b813ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4b6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4b80b3a60 0x7fe4b813ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56304/0 (socket says 172.21.15.29:56304) 2024-04-25T00:35:34.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4b6ffd640 1 -- 172.21.15.29:0/1201126459 learned_addr learned my addr 172.21.15.29:0/1201126459 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:34.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4b77fe640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4b80b3140 0x7fe4b813a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4b7fff640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b80b4b10 0x7fe4b813b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4b6ffd640 1 -- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b80b4b10 msgr2=0x7fe4b813b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4b6ffd640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b80b4b10 0x7fe4b813b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4b6ffd640 1 -- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4b80b3140 msgr2=0x7fe4b813a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:34.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4b6ffd640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4b80b3140 0x7fe4b813a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:34.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4b6ffd640 1 -- 172.21.15.29:0/1201126459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b8136070 con 0x7fe4b80b3a60 2024-04-25T00:35:34.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4b7fff640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b80b4b10 0x7fe4b813b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:34.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4b6ffd640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4b80b3a60 0x7fe4b813ea90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0ae040 tx=0x7fe4ac0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:34.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe49ffff640 1 -- 172.21.15.29:0/1201126459 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac0b0050 con 0x7fe4b80b3a60 2024-04-25T00:35:34.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe49ffff640 1 -- 172.21.15.29:0/1201126459 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ac0bc070 con 0x7fe4b80b3a60 2024-04-25T00:35:34.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4be0ab640 1 -- 172.21.15.29:0/1201126459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b8135550 con 0x7fe4b80b3a60 2024-04-25T00:35:34.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe49ffff640 1 -- 172.21.15.29:0/1201126459 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac0b5830 con 0x7fe4b80b3a60 2024-04-25T00:35:34.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4be0ab640 1 -- 172.21.15.29:0/1201126459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b8135d90 con 0x7fe4b80b3a60 2024-04-25T00:35:34.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe4be0ab640 1 -- 172.21.15.29:0/1201126459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b8069270 con 0x7fe4b80b3a60 2024-04-25T00:35:34.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe49ffff640 1 -- 172.21.15.29:0/1201126459 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fe4ac0b59d0 con 0x7fe4b80b3a60 2024-04-25T00:35:34.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe49ffff640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe4a4067c20 0x7fe4a406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:34.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.880+0000 7fe49ffff640 1 -- 172.21.15.29:0/1201126459 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe4ac120890 con 0x7fe4b80b3a60 2024-04-25T00:35:34.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.884+0000 7fe4b77fe640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe4a4067c20 0x7fe4a406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:34.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.884+0000 7fe4b77fe640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe4a4067c20 0x7fe4a406a0e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009a70 tx=0x7fe4a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:34.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:34.888+0000 7fe49ffff640 1 -- 172.21.15.29:0/1201126459 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe4b8135550 con 0x7fe4b80b3a60 2024-04-25T00:35:35.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.044+0000 7fe4be0ab640 1 -- 172.21.15.29:0/1201126459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe4b8144890 con 0x7fe4b80b3a60 2024-04-25T00:35:35.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.044+0000 7fe49ffff640 1 -- 172.21.15.29:0/1201126459 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) ==== 100+0+11 (secure 0 0 0) 0x7fe4ac016110 con 0x7fe4b80b3a60 2024-04-25T00:35:35.047 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:35:35.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.048+0000 7fe4be0ab640 1 -- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe4a4067c20 msgr2=0x7fe4a406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.048+0000 7fe4be0ab640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe4a4067c20 0x7fe4a406a0e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009a70 tx=0x7fe4a8008040 comp rx=0 tx=0).stop 2024-04-25T00:35:35.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.048+0000 7fe4be0ab640 1 -- 172.21.15.29:0/1201126459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4b80b3a60 msgr2=0x7fe4b813ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.048+0000 7fe4be0ab640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4b80b3a60 0x7fe4b813ea90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0ae040 tx=0x7fe4ac0192c0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.048+0000 7fe4be0ab640 1 -- 172.21.15.29:0/1201126459 shutdown_connections 2024-04-25T00:35:35.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.048+0000 7fe4be0ab640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b80b4b10 0x7fe4b813b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.048+0000 7fe4be0ab640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4b80b3a60 0x7fe4b813ea90 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.048+0000 7fe4be0ab640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe4a4067c20 0x7fe4a406a0e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.048+0000 7fe4be0ab640 1 --2- 172.21.15.29:0/1201126459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4b80b3140 0x7fe4b813a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.048+0000 7fe4be0ab640 1 -- 172.21.15.29:0/1201126459 >> 172.21.15.29:0/1201126459 conn(0x7fe4b80b1870 msgr2=0x7fe4b813f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:35.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.048+0000 7fe4be0ab640 1 -- 172.21.15.29:0/1201126459 shutdown_connections 2024-04-25T00:35:35.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.048+0000 7fe4be0ab640 1 -- 172.21.15.29:0/1201126459 wait complete. 2024-04-25T00:35:35.065 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:35:35.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43fdb9640 1 Processor -- start 2024-04-25T00:35:35.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43fdb9640 1 -- start start 2024-04-25T00:35:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43fdb9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff4380b4780 0x7ff43815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43fdb9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff4380cc240 0x7ff43815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43fdb9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4380b3720 con 0x7ff43815e8d0 2024-04-25T00:35:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43fdb9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff4380cc650 con 0x7ff4380cc240 2024-04-25T00:35:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43fdb9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff4380b42f0 con 0x7ff4380b4780 2024-04-25T00:35:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43e356640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff4380cc240 0x7ff43815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43e356640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff4380cc240 0x7ff43815c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56312/0 (socket says 172.21.15.29:56312) 2024-04-25T00:35:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43e356640 1 -- 172.21.15.29:0/3652734856 learned_addr learned my addr 172.21.15.29:0/3652734856 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43d354640 1 --2- 172.21.15.29:0/3652734856 >> v2:172.21.15.157:3300/0 conn(0x7ff4380b4780 0x7ff43815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43e356640 1 -- 172.21.15.29:0/3652734856 >> v2:172.21.15.157:3300/0 conn(0x7ff4380b4780 msgr2=0x7ff43815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43e356640 1 --2- 172.21.15.29:0/3652734856 >> v2:172.21.15.157:3300/0 conn(0x7ff4380b4780 0x7ff43815f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43e356640 1 -- 172.21.15.29:0/3652734856 >> v1:172.21.15.157:6789/0 conn(0x7ff43815e8d0 legacy=0x7ff43815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:35.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.188+0000 7ff43e356640 1 -- 172.21.15.29:0/3652734856 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4380b39c0 con 0x7ff4380cc240 2024-04-25T00:35:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43e356640 1 --2- 172.21.15.29:0/3652734856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4380cc240 0x7ff43815c7d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff438070910 tx=0x7ff43001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4794ea0ee4d2a55c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43cb53640 1 -- 172.21.15.29:0/3652734856 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4300aa610 con 0x7ff4380cc240 2024-04-25T00:35:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43cb53640 1 -- 172.21.15.29:0/3652734856 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4300af070 con 0x7ff4380cc240 2024-04-25T00:35:35.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43cb53640 1 -- 172.21.15.29:0/3652734856 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4300aa9d0 con 0x7ff4380cc240 2024-04-25T00:35:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3652734856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4380cc240 msgr2=0x7ff43815c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 --2- 172.21.15.29:0/3652734856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4380cc240 0x7ff43815c7d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff438070910 tx=0x7ff43001c4b0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3652734856 shutdown_connections 2024-04-25T00:35:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 --2- 172.21.15.29:0/3652734856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4380cc240 0x7ff43815c7d0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 --2- 172.21.15.29:0/3652734856 >> v2:172.21.15.157:3300/0 conn(0x7ff4380b4780 0x7ff43815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3652734856 >> 172.21.15.29:0/3652734856 conn(0x7ff4380b1870 msgr2=0x7ff4380b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3652734856 shutdown_connections 2024-04-25T00:35:35.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3652734856 wait complete. 2024-04-25T00:35:35.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 Processor -- start 2024-04-25T00:35:35.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 -- start start 2024-04-25T00:35:35.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4380b4780 0x7ff438157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4380cc240 0x7ff438136a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43815e8d0 0x7ff438139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff438144a20 con 0x7ff43815e8d0 2024-04-25T00:35:35.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff43815d520 con 0x7ff4380cc240 2024-04-25T00:35:35.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43fdb9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff43813a850 con 0x7ff4380b4780 2024-04-25T00:35:35.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43e356640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43815e8d0 0x7ff438139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43e356640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43815e8d0 0x7ff438139ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56328/0 (socket says 172.21.15.29:56328) 2024-04-25T00:35:35.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43e356640 1 -- 172.21.15.29:0/3685223088 learned_addr learned my addr 172.21.15.29:0/3685223088 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:35.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43d354640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4380cc240 0x7ff438136a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43e356640 1 -- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4380b4780 msgr2=0x7ff438157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43e356640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4380b4780 0x7ff438157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43e356640 1 -- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4380cc240 msgr2=0x7ff438136a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43e356640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4380cc240 0x7ff438136a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43e356640 1 -- 172.21.15.29:0/3685223088 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff438166660 con 0x7ff43815e8d0 2024-04-25T00:35:35.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.192+0000 7ff43d354640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4380cc240 0x7ff438136a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:35.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.196+0000 7ff43e356640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43815e8d0 0x7ff438139ec0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff43001a0e0 tx=0x7ff43001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:35.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.196+0000 7ff42a7fc640 1 -- 172.21.15.29:0/3685223088 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4300bd5f0 con 0x7ff43815e8d0 2024-04-25T00:35:35.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.196+0000 7ff42a7fc640 1 -- 172.21.15.29:0/3685223088 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4300bd790 con 0x7ff43815e8d0 2024-04-25T00:35:35.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.196+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3685223088 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff438136fb0 con 0x7ff43815e8d0 2024-04-25T00:35:35.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.196+0000 7ff42a7fc640 1 -- 172.21.15.29:0/3685223088 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4300bdb80 con 0x7ff43815e8d0 2024-04-25T00:35:35.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.196+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3685223088 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff438157000 con 0x7ff43815e8d0 2024-04-25T00:35:35.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.196+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3685223088 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff438069270 con 0x7ff43815e8d0 2024-04-25T00:35:35.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.196+0000 7ff42a7fc640 1 -- 172.21.15.29:0/3685223088 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7ff4300b33f0 con 0x7ff43815e8d0 2024-04-25T00:35:35.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.196+0000 7ff42a7fc640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff41c067c20 0x7ff41c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.196+0000 7ff42a7fc640 1 -- 172.21.15.29:0/3685223088 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff430121120 con 0x7ff43815e8d0 2024-04-25T00:35:35.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.196+0000 7ff43db55640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff41c067c20 0x7ff41c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.196+0000 7ff43db55640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff41c067c20 0x7ff41c06a0e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff438154c80 tx=0x7ff434009230 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:35.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.200+0000 7ff42a7fc640 1 -- 172.21.15.29:0/3685223088 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff4300c3060 con 0x7ff43815e8d0 2024-04-25T00:35:35.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3685223088 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4381532d0 con 0x7ff43815e8d0 2024-04-25T00:35:35.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff42a7fc640 1 -- 172.21.15.29:0/3685223088 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5175 (secure 0 0 0) 0x7ff4300a7070 con 0x7ff43815e8d0 2024-04-25T00:35:35.393 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:35.394 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/56623075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":56623075},{"type":"v1","addr":"172.21.15.157:6836","nonce":56623075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/748098725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":748098725},{"type":"v1","addr":"172.21.15.157:6837","nonce":748098725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2264413556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2264413556},{"type":"v1","addr":"172.21.15.29:6833","nonce":2264413556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:07.114520+0000","modified":"2024-04-25T00:35:10.185595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6372},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/802824676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":802824676},{"type":"v1","addr":"172.21.15.29:6835","nonce":802824676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372]},"id":4}]} 2024-04-25T00:35:35.394 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 35 2024-04-25T00:35:35.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff41c067c20 msgr2=0x7ff41c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff43fdb9640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff41c067c20 0x7ff41c06a0e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff438154c80 tx=0x7ff434009230 comp rx=0 tx=0).stop 2024-04-25T00:35:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3685223088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43815e8d0 msgr2=0x7ff438139ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff43fdb9640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43815e8d0 0x7ff438139ec0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff43001a0e0 tx=0x7ff43001acf0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3685223088 shutdown_connections 2024-04-25T00:35:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff43fdb9640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43815e8d0 0x7ff438139ec0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff43fdb9640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff41c067c20 0x7ff41c06a0e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff43fdb9640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4380cc240 0x7ff438136a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff43fdb9640 1 --2- 172.21.15.29:0/3685223088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4380b4780 0x7ff438157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3685223088 >> 172.21.15.29:0/3685223088 conn(0x7ff4380b1870 msgr2=0x7ff4380af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3685223088 shutdown_connections 2024-04-25T00:35:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.392+0000 7ff43fdb9640 1 -- 172.21.15.29:0/3685223088 wait complete. 2024-04-25T00:35:35.411 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:35.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f3201ee5640 1 Processor -- start 2024-04-25T00:35:35.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f3201ee5640 1 -- start start 2024-04-25T00:35:35.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f3201ee5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31fc156420 0x7f31fc15ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f3201ee5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f31fc15fa80 0x7f31fc06b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f3201ee5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31fc15c680 con 0x7f31fc15ff10 2024-04-25T00:35:35.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f3201ee5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f31fc15d3a0 con 0x7f31fc15fa80 2024-04-25T00:35:35.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f3201ee5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f31fc160440 con 0x7f31fc156420 2024-04-25T00:35:35.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f31faffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f31fc15fa80 0x7f31fc06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f31faffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f31fc15fa80 0x7f31fc06b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56332/0 (socket says 172.21.15.29:56332) 2024-04-25T00:35:35.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f31faffd640 1 -- 172.21.15.29:0/2862818143 learned_addr learned my addr 172.21.15.29:0/2862818143 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:35.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f31fb7fe640 1 --2- 172.21.15.29:0/2862818143 >> v2:172.21.15.157:3300/0 conn(0x7f31fc156420 0x7f31fc15ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f31faffd640 1 -- 172.21.15.29:0/2862818143 >> v2:172.21.15.157:3300/0 conn(0x7f31fc156420 msgr2=0x7f31fc15ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:35.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f31faffd640 1 --2- 172.21.15.29:0/2862818143 >> v2:172.21.15.157:3300/0 conn(0x7f31fc156420 0x7f31fc15ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f31faffd640 1 -- 172.21.15.29:0/2862818143 >> v1:172.21.15.157:6789/0 conn(0x7f31fc15ff10 legacy=0x7f31fc0cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:35.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.520+0000 7f31faffd640 1 -- 172.21.15.29:0/2862818143 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc160720 con 0x7f31fc15fa80 2024-04-25T00:35:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31faffd640 1 --2- 172.21.15.29:0/2862818143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc15fa80 0x7f31fc06b210 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f31fc073040 tx=0x7f31f001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54708e14017c36a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31fa7fc640 1 -- 172.21.15.29:0/2862818143 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f00ab5f0 con 0x7f31fc15fa80 2024-04-25T00:35:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31fa7fc640 1 -- 172.21.15.29:0/2862818143 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f00b0070 con 0x7f31fc15fa80 2024-04-25T00:35:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31fa7fc640 1 -- 172.21.15.29:0/2862818143 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f00ab9b0 con 0x7f31fc15fa80 2024-04-25T00:35:35.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 -- 172.21.15.29:0/2862818143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc15fa80 msgr2=0x7f31fc06b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 --2- 172.21.15.29:0/2862818143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc15fa80 0x7f31fc06b210 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f31fc073040 tx=0x7f31f001c4e0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 -- 172.21.15.29:0/2862818143 shutdown_connections 2024-04-25T00:35:35.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 --2- 172.21.15.29:0/2862818143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc15fa80 0x7f31fc06b210 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 --2- 172.21.15.29:0/2862818143 >> v2:172.21.15.157:3300/0 conn(0x7f31fc156420 0x7f31fc15ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 -- 172.21.15.29:0/2862818143 >> 172.21.15.29:0/2862818143 conn(0x7f31fc0d0c40 msgr2=0x7f31fc0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:35.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 -- 172.21.15.29:0/2862818143 shutdown_connections 2024-04-25T00:35:35.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 -- 172.21.15.29:0/2862818143 wait complete. 2024-04-25T00:35:35.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 Processor -- start 2024-04-25T00:35:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 -- start start 2024-04-25T00:35:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc156420 0x7f31fc153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc15fa80 0x7f31fc0b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31fc15ff10 0x7f31fc0b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc070160 con 0x7f31fc15fa80 2024-04-25T00:35:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc06bb60 con 0x7f31fc156420 2024-04-25T00:35:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f3201ee5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f31fc0702e0 con 0x7f31fc15ff10 2024-04-25T00:35:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31faffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc15fa80 0x7f31fc0b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31faffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc15fa80 0x7f31fc0b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56334/0 (socket says 172.21.15.29:56334) 2024-04-25T00:35:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31faffd640 1 -- 172.21.15.29:0/1420012193 learned_addr learned my addr 172.21.15.29:0/1420012193 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31fb7fe640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc156420 0x7f31fc153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31fbfff640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31fc15ff10 0x7f31fc0b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31faffd640 1 -- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31fc15ff10 msgr2=0x7f31fc0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31faffd640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31fc15ff10 0x7f31fc0b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31faffd640 1 -- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc156420 msgr2=0x7f31fc153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31faffd640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc156420 0x7f31fc153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31faffd640 1 -- 172.21.15.29:0/1420012193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc1536a0 con 0x7f31fc15fa80 2024-04-25T00:35:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31faffd640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc15fa80 0x7f31fc0b55b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f31f001cd80 tx=0x7f31f001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.524+0000 7f31e3fff640 1 -- 172.21.15.29:0/1420012193 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f00be5f0 con 0x7f31fc15fa80 2024-04-25T00:35:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.528+0000 7f31e3fff640 1 -- 172.21.15.29:0/1420012193 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f00be7b0 con 0x7f31fc15fa80 2024-04-25T00:35:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.528+0000 7f3201ee5640 1 -- 172.21.15.29:0/1420012193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31fc156f00 con 0x7f31fc15fa80 2024-04-25T00:35:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.528+0000 7f31e3fff640 1 -- 172.21.15.29:0/1420012193 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f00beba0 con 0x7f31fc15fa80 2024-04-25T00:35:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.528+0000 7f3201ee5640 1 -- 172.21.15.29:0/1420012193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31fc1574c0 con 0x7f31fc15fa80 2024-04-25T00:35:35.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.528+0000 7f3201ee5640 1 -- 172.21.15.29:0/1420012193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31fc069270 con 0x7f31fc15fa80 2024-04-25T00:35:35.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.528+0000 7f31e3fff640 1 -- 172.21.15.29:0/1420012193 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f31f00b43f0 con 0x7f31fc15fa80 2024-04-25T00:35:35.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.528+0000 7f31e3fff640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f31e8067c20 0x7f31e806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.528+0000 7f31e3fff640 1 -- 172.21.15.29:0/1420012193 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f31f0120620 con 0x7f31fc15fa80 2024-04-25T00:35:35.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.528+0000 7f31fb7fe640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f31e8067c20 0x7f31e806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.528+0000 7f31fb7fe640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f31e8067c20 0x7f31e806a0e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00b9d0 tx=0x7f31ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:35.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.532+0000 7f31e3fff640 1 -- 172.21.15.29:0/1420012193 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f31f00c4060 con 0x7f31fc15fa80 2024-04-25T00:35:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.692+0000 7f3201ee5640 1 -- 172.21.15.29:0/1420012193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31fc0d1980 con 0x7f31fc15fa80 2024-04-25T00:35:35.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.692+0000 7f31e3fff640 1 -- 172.21.15.29:0/1420012193 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16617 (secure 0 0 0) 0x7f31f00160d0 con 0x7f31fc15fa80 2024-04-25T00:35:35.695 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:35.695 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":58,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:08.767613+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:05.458923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:06.104267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.692+0000 7f3201ee5640 1 -- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f31e8067c20 msgr2=0x7f31e806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.692+0000 7f3201ee5640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f31e8067c20 0x7f31e806a0e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00b9d0 tx=0x7f31ec008040 comp rx=0 tx=0).stop 2024-04-25T00:35:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.692+0000 7f3201ee5640 1 -- 172.21.15.29:0/1420012193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc15fa80 msgr2=0x7f31fc0b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.692+0000 7f3201ee5640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc15fa80 0x7f31fc0b55b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f31f001cd80 tx=0x7f31f001ad20 comp rx=0 tx=0).stop 2024-04-25T00:35:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.696+0000 7f3201ee5640 1 -- 172.21.15.29:0/1420012193 shutdown_connections 2024-04-25T00:35:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.696+0000 7f3201ee5640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31fc15ff10 0x7f31fc0b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.696+0000 7f3201ee5640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc15fa80 0x7f31fc0b55b0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.696+0000 7f3201ee5640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f31e8067c20 0x7f31e806a0e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.696+0000 7f3201ee5640 1 --2- 172.21.15.29:0/1420012193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc156420 0x7f31fc153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.696+0000 7f3201ee5640 1 -- 172.21.15.29:0/1420012193 >> 172.21.15.29:0/1420012193 conn(0x7f31fc0d0c40 msgr2=0x7f31fc1558c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.696+0000 7f3201ee5640 1 -- 172.21.15.29:0/1420012193 shutdown_connections 2024-04-25T00:35:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.696+0000 7f3201ee5640 1 -- 172.21.15.29:0/1420012193 wait complete. 2024-04-25T00:35:35.713 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:35:35.713 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:35:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.832+0000 7fe44082b640 1 Processor -- start 2024-04-25T00:35:35.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.832+0000 7fe44082b640 1 -- start start 2024-04-25T00:35:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe44082b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe43c0b4780 0x7fe43c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe44082b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe43c0cc240 0x7fe43c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe44082b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe43c0b3720 con 0x7fe43c15e8d0 2024-04-25T00:35:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe44082b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe43c0b42f0 con 0x7fe43c0b4780 2024-04-25T00:35:35.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe44082b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe43c0b39c0 con 0x7fe43c0cc240 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43a59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe43c15e8d0 0x7fe43c15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55164/0 (socket says 172.21.15.29:55164) 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43a59c640 1 -- 172.21.15.29:0/1754285451 learned_addr learned my addr 172.21.15.29:0/1754285451 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe439d9b640 1 --2- 172.21.15.29:0/1754285451 >> v2:172.21.15.157:3300/0 conn(0x7fe43c0b4780 0x7fe43c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 981285726 0 0) 0x7fe43c0b3720 con 0x7fe43c15e8d0 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43ad9d640 1 --2- 172.21.15.29:0/1754285451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe43c0cc240 0x7fe43c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe424003530 con 0x7fe43c15e8d0 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2802083338 0 0) 0x7fe424003530 con 0x7fe43c15e8d0 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe43c0cc240 msgr2=0x7fe43c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 --2- 172.21.15.29:0/1754285451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe43c0cc240 0x7fe43c15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 >> v2:172.21.15.157:3300/0 conn(0x7fe43c0b4780 msgr2=0x7fe43c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 --2- 172.21.15.29:0/1754285451 >> v2:172.21.15.157:3300/0 conn(0x7fe43c0b4780 0x7fe43c15f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c0b3ca0 con 0x7fe43c15e8d0 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fe42c001680 con 0x7fe43c15e8d0 2024-04-25T00:35:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 >> v1:172.21.15.29:6789/0 conn(0x7fe43c15e8d0 legacy=0x7fe43c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 --2- 172.21.15.29:0/1754285451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4240039c0 0x7fe424003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 --2- 172.21.15.29:0/1754285451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe43c0cc240 0x7fe424004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 --2- 172.21.15.29:0/1754285451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe424004d50 0x7fe4240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c0b42f0 con 0x7fe4240039c0 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c0b39c0 con 0x7fe424004d50 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe42400b9d0 con 0x7fe43c0cc240 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43a59c640 1 --2- 172.21.15.29:0/1754285451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4240039c0 0x7fe424003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43ad9d640 1 --2- 172.21.15.29:0/1754285451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe424004d50 0x7fe4240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe439d9b640 1 --2- 172.21.15.29:0/1754285451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe43c0cc240 0x7fe424004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43a59c640 1 -- 172.21.15.29:0/1754285451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe43c0cc240 msgr2=0x7fe424004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43a59c640 1 --2- 172.21.15.29:0/1754285451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe43c0cc240 0x7fe424004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43a59c640 1 -- 172.21.15.29:0/1754285451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe424004d50 msgr2=0x7fe4240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43a59c640 1 --2- 172.21.15.29:0/1754285451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe424004d50 0x7fe4240051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.836+0000 7fe43a59c640 1 -- 172.21.15.29:0/1754285451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe424003530 con 0x7fe4240039c0 2024-04-25T00:35:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.840+0000 7fe43a59c640 1 --2- 172.21.15.29:0/1754285451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4240039c0 0x7fe424003e00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00c210 tx=0x7fe42c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75f1d0ead60e337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.840+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c007d60 con 0x7fe4240039c0 2024-04-25T00:35:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.840+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe42c0a9020 con 0x7fe4240039c0 2024-04-25T00:35:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.840+0000 7fe43959a640 1 -- 172.21.15.29:0/1754285451 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c005b20 con 0x7fe4240039c0 2024-04-25T00:35:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.840+0000 7fe44082b640 1 -- 172.21.15.29:0/1754285451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4240039c0 msgr2=0x7fe424003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.840+0000 7fe44082b640 1 --2- 172.21.15.29:0/1754285451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4240039c0 0x7fe424003e00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00c210 tx=0x7fe42c01cca0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.840+0000 7fe43ad9d640 1 -- 172.21.15.29:0/1754285451 reap_dead start 2024-04-25T00:35:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.840+0000 7fe44082b640 1 -- 172.21.15.29:0/1754285451 shutdown_connections 2024-04-25T00:35:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.840+0000 7fe44082b640 1 -- 172.21.15.29:0/1754285451 >> 172.21.15.29:0/1754285451 conn(0x7fe43c0b1870 msgr2=0x7fe43c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.840+0000 7fe44082b640 1 -- 172.21.15.29:0/1754285451 shutdown_connections 2024-04-25T00:35:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.840+0000 7fe44082b640 1 -- 172.21.15.29:0/1754285451 wait complete. 2024-04-25T00:35:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.840+0000 7fe44082b640 1 Processor -- start 2024-04-25T00:35:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.840+0000 7fe44082b640 1 -- start start 2024-04-25T00:35:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe44082b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4240039c0 0x7fe43c134940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe44082b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c13a850 0x7fe43c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe44082b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe43c15a520 0x7fe43c0722d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe44082b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c06df20 con 0x7fe43c15a520 2024-04-25T00:35:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe44082b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c06dda0 con 0x7fe43c13a850 2024-04-25T00:35:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe44082b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe43c06e0a0 con 0x7fe4240039c0 2024-04-25T00:35:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe43ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe43c15a520 0x7fe43c0722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe43ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe43c15a520 0x7fe43c0722d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56350/0 (socket says 172.21.15.29:56350) 2024-04-25T00:35:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe43ad9d640 1 -- 172.21.15.29:0/1962163401 learned_addr learned my addr 172.21.15.29:0/1962163401 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe43a59c640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4240039c0 0x7fe43c134940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe439d9b640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c13a850 0x7fe43c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe43ad9d640 1 -- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4240039c0 msgr2=0x7fe43c134940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe43ad9d640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4240039c0 0x7fe43c134940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe43ad9d640 1 -- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c13a850 msgr2=0x7fe43c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe43ad9d640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c13a850 0x7fe43c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe43ad9d640 1 -- 172.21.15.29:0/1962163401 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c071f90 con 0x7fe43c15a520 2024-04-25T00:35:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe43ad9d640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe43c15a520 0x7fe43c0722d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0b28b0 tx=0x7fe4340115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe42affd640 1 -- 172.21.15.29:0/1962163401 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434011e10 con 0x7fe43c15a520 2024-04-25T00:35:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe42affd640 1 -- 172.21.15.29:0/1962163401 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe43401b070 con 0x7fe43c15a520 2024-04-25T00:35:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe42affd640 1 -- 172.21.15.29:0/1962163401 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4340039a0 con 0x7fe43c15a520 2024-04-25T00:35:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe44082b640 1 -- 172.21.15.29:0/1962163401 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43c13acd0 con 0x7fe43c15a520 2024-04-25T00:35:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe44082b640 1 -- 172.21.15.29:0/1962163401 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43c133b80 con 0x7fe43c15a520 2024-04-25T00:35:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe44082b640 1 -- 172.21.15.29:0/1962163401 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe43c069270 con 0x7fe43c15a520 2024-04-25T00:35:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe42affd640 1 -- 172.21.15.29:0/1962163401 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fe434003b60 con 0x7fe43c15a520 2024-04-25T00:35:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe42affd640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe41c067c50 0x7fe41c06a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe42affd640 1 -- 172.21.15.29:0/1962163401 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe434086c00 con 0x7fe43c15a520 2024-04-25T00:35:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.844+0000 7fe43a59c640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe41c067c50 0x7fe41c06a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.848+0000 7fe43a59c640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe41c067c50 0x7fe41c06a110 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0aa000 tx=0x7fe42c000ff0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:35.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:35.848+0000 7fe42affd640 1 -- 172.21.15.29:0/1962163401 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe434025020 con 0x7fe43c15a520 2024-04-25T00:35:36.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.040+0000 7fe44082b640 1 -- 172.21.15.29:0/1962163401 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe43c1382d0 con 0x7fe43c15a520 2024-04-25T00:35:36.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.044+0000 7fe42affd640 1 -- 172.21.15.29:0/1962163401 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5175 (secure 0 0 0) 0x7fe43c13acd0 con 0x7fe43c15a520 2024-04-25T00:35:36.046 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:36.046 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/56623075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":56623075},{"type":"v1","addr":"172.21.15.157:6836","nonce":56623075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/748098725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":748098725},{"type":"v1","addr":"172.21.15.157:6837","nonce":748098725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2264413556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2264413556},{"type":"v1","addr":"172.21.15.29:6833","nonce":2264413556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:07.114520+0000","modified":"2024-04-25T00:35:10.185595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6372},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/802824676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":802824676},{"type":"v1","addr":"172.21.15.29:6835","nonce":802824676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372]},"id":4}]} 2024-04-25T00:35:36.046 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 35 2024-04-25T00:35:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.044+0000 7fe44082b640 1 -- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe41c067c50 msgr2=0x7fe41c06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.044+0000 7fe44082b640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe41c067c50 0x7fe41c06a110 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0aa000 tx=0x7fe42c000ff0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.044+0000 7fe44082b640 1 -- 172.21.15.29:0/1962163401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe43c15a520 msgr2=0x7fe43c0722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.044+0000 7fe44082b640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe43c15a520 0x7fe43c0722d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0b28b0 tx=0x7fe4340115a0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.044+0000 7fe44082b640 1 -- 172.21.15.29:0/1962163401 shutdown_connections 2024-04-25T00:35:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.044+0000 7fe44082b640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe43c15a520 0x7fe43c0722d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.044+0000 7fe44082b640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe41c067c50 0x7fe41c06a110 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.044+0000 7fe44082b640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c13a850 0x7fe43c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.044+0000 7fe44082b640 1 --2- 172.21.15.29:0/1962163401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4240039c0 0x7fe43c134940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.044+0000 7fe44082b640 1 -- 172.21.15.29:0/1962163401 >> 172.21.15.29:0/1962163401 conn(0x7fe43c0b1870 msgr2=0x7fe43c0af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.044+0000 7fe44082b640 1 -- 172.21.15.29:0/1962163401 shutdown_connections 2024-04-25T00:35:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.044+0000 7fe44082b640 1 -- 172.21.15.29:0/1962163401 wait complete. 2024-04-25T00:35:36.064 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:35:36.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101cc36640 1 Processor -- start 2024-04-25T00:35:36.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101cc36640 1 -- start start 2024-04-25T00:35:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101cc36640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f10180b3140 0x7f101806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101cc36640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f10180b4b10 0x7f10180b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101cc36640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10180b3a60 0x7f101815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101cc36640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f10180b41e0 con 0x7f10180b3140 2024-04-25T00:35:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101cc36640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f101806d960 con 0x7f10180b4b10 2024-04-25T00:35:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101cc36640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f101806dc40 con 0x7f10180b3a60 2024-04-25T00:35:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101659c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f10180b3140 0x7f101806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101659c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f10180b3140 0x7f101806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56352/0 (socket says 172.21.15.29:56352) 2024-04-25T00:35:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101659c640 1 -- 172.21.15.29:0/2130043814 learned_addr learned my addr 172.21.15.29:0/2130043814 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f1016d9d640 1 --2- 172.21.15.29:0/2130043814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10180b3a60 0x7f101815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f100dd9b640 1 --2- 172.21.15.29:0/2130043814 >> v2:172.21.15.157:3300/0 conn(0x7f10180b4b10 0x7f10180b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101659c640 1 -- 172.21.15.29:0/2130043814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10180b3a60 msgr2=0x7f101815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:36.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101659c640 1 --2- 172.21.15.29:0/2130043814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10180b3a60 0x7f101815c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101659c640 1 -- 172.21.15.29:0/2130043814 >> v2:172.21.15.157:3300/0 conn(0x7f10180b4b10 msgr2=0x7f10180b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:36.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101659c640 1 --2- 172.21.15.29:0/2130043814 >> v2:172.21.15.157:3300/0 conn(0x7f10180b4b10 0x7f10180b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101659c640 1 -- 172.21.15.29:0/2130043814 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101806df20 con 0x7f10180b3140 2024-04-25T00:35:36.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f1016d9d640 1 --2- 172.21.15.29:0/2130043814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10180b3a60 0x7f101815c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:36.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.192+0000 7f101659c640 1 --2- 172.21.15.29:0/2130043814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10180b3140 0x7f101806e440 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1008009870 tx=0x7f100801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308d270c43c96437 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:36.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f1015d9b640 1 -- 172.21.15.29:0/2130043814 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10080b0070 con 0x7f10180b3140 2024-04-25T00:35:36.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f1015d9b640 1 -- 172.21.15.29:0/2130043814 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10080ab600 con 0x7f10180b3140 2024-04-25T00:35:36.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f1015d9b640 1 -- 172.21.15.29:0/2130043814 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10080ab9d0 con 0x7f10180b3140 2024-04-25T00:35:36.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 -- 172.21.15.29:0/2130043814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10180b3140 msgr2=0x7f101806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 --2- 172.21.15.29:0/2130043814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10180b3140 0x7f101806e440 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1008009870 tx=0x7f100801c520 comp rx=0 tx=0).stop 2024-04-25T00:35:36.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 -- 172.21.15.29:0/2130043814 shutdown_connections 2024-04-25T00:35:36.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 --2- 172.21.15.29:0/2130043814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10180b3a60 0x7f101815c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 --2- 172.21.15.29:0/2130043814 >> v2:172.21.15.157:3300/0 conn(0x7f10180b4b10 0x7f10180b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 --2- 172.21.15.29:0/2130043814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10180b3140 0x7f101806e440 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 -- 172.21.15.29:0/2130043814 >> 172.21.15.29:0/2130043814 conn(0x7f10180b1870 msgr2=0x7f10180b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:36.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 -- 172.21.15.29:0/2130043814 shutdown_connections 2024-04-25T00:35:36.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 -- 172.21.15.29:0/2130043814 wait complete. 2024-04-25T00:35:36.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 Processor -- start 2024-04-25T00:35:36.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 -- start start 2024-04-25T00:35:36.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10180b3140 0x7f1018154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10180b3a60 0x7f10180ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10180b4b10 0x7f10180cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f10180ce3c0 con 0x7f10180b3140 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f10180ce240 con 0x7f10180b3a60 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f10180ce540 con 0x7f10180b4b10 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10180b3140 0x7f1018154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10180b3140 0x7f1018154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56360/0 (socket says 172.21.15.29:56360) 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101659c640 1 -- 172.21.15.29:0/902711848 learned_addr learned my addr 172.21.15.29:0/902711848 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f100dd9b640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10180b3a60 0x7f10180ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f1016d9d640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10180b4b10 0x7f10180cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101659c640 1 -- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10180b4b10 msgr2=0x7f10180cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101659c640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10180b4b10 0x7f10180cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101659c640 1 -- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10180b3a60 msgr2=0x7f10180ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101659c640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10180b3a60 0x7f10180ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101659c640 1 -- 172.21.15.29:0/902711848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10180cdd90 con 0x7f10180b3140 2024-04-25T00:35:36.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f1016d9d640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10180b4b10 0x7f10180cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:36.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f100dd9b640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10180b3a60 0x7f10180ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:36.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101659c640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10180b3140 0x7f1018154d50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f10080ae040 tx=0x7f10080192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:36.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f100f7fe640 1 -- 172.21.15.29:0/902711848 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10080b0040 con 0x7f10180b3140 2024-04-25T00:35:36.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f100f7fe640 1 -- 172.21.15.29:0/902711848 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10080b9070 con 0x7f10180b3140 2024-04-25T00:35:36.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 -- 172.21.15.29:0/902711848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1018156560 con 0x7f10180b3140 2024-04-25T00:35:36.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f100f7fe640 1 -- 172.21.15.29:0/902711848 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10080b4680 con 0x7f10180b3140 2024-04-25T00:35:36.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.196+0000 7f101cc36640 1 -- 172.21.15.29:0/902711848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1018156b20 con 0x7f10180b3140 2024-04-25T00:35:36.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.200+0000 7f101cc36640 1 -- 172.21.15.29:0/902711848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1018069270 con 0x7f10180b3140 2024-04-25T00:35:36.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.200+0000 7f100f7fe640 1 -- 172.21.15.29:0/902711848 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f10080b4820 con 0x7f10180b3140 2024-04-25T00:35:36.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.200+0000 7f100f7fe640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0ff8067c20 0x7f0ff806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.200+0000 7f100f7fe640 1 -- 172.21.15.29:0/902711848 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f100811ff30 con 0x7f10180b3140 2024-04-25T00:35:36.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.200+0000 7f100dd9b640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0ff8067c20 0x7f0ff806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.200+0000 7f100dd9b640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0ff8067c20 0x7f0ff806a0e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1018144780 tx=0x7f1000008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:36.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.204+0000 7f100f7fe640 1 -- 172.21.15.29:0/902711848 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1008016110 con 0x7f10180b3140 2024-04-25T00:35:36.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.400+0000 7f101cc36640 1 -- 172.21.15.29:0/902711848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f101815ad70 con 0x7f10180b3140 2024-04-25T00:35:36.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.400+0000 7f100f7fe640 1 -- 172.21.15.29:0/902711848 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5175 (secure 0 0 0) 0x7f10080f05e0 con 0x7f10180b3140 2024-04-25T00:35:36.403 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:36.403 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/56623075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":56623075},{"type":"v1","addr":"172.21.15.157:6836","nonce":56623075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/748098725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":748098725},{"type":"v1","addr":"172.21.15.157:6837","nonce":748098725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2264413556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2264413556},{"type":"v1","addr":"172.21.15.29:6833","nonce":2264413556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:07.114520+0000","modified":"2024-04-25T00:35:10.185595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6372},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/802824676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":802824676},{"type":"v1","addr":"172.21.15.29:6835","nonce":802824676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372]},"id":4}]} 2024-04-25T00:35:36.403 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 35 2024-04-25T00:35:36.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.404+0000 7f101cc36640 1 -- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0ff8067c20 msgr2=0x7f0ff806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.404+0000 7f101cc36640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0ff8067c20 0x7f0ff806a0e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1018144780 tx=0x7f1000008040 comp rx=0 tx=0).stop 2024-04-25T00:35:36.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.404+0000 7f101cc36640 1 -- 172.21.15.29:0/902711848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10180b3140 msgr2=0x7f1018154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.404+0000 7f101cc36640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10180b3140 0x7f1018154d50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f10080ae040 tx=0x7f10080192c0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.404+0000 7f101cc36640 1 -- 172.21.15.29:0/902711848 shutdown_connections 2024-04-25T00:35:36.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.404+0000 7f101cc36640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10180b4b10 0x7f10180cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.404+0000 7f101cc36640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0ff8067c20 0x7f0ff806a0e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.404+0000 7f101cc36640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10180b3a60 0x7f10180ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.404+0000 7f101cc36640 1 --2- 172.21.15.29:0/902711848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10180b3140 0x7f1018154d50 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.404+0000 7f101cc36640 1 -- 172.21.15.29:0/902711848 >> 172.21.15.29:0/902711848 conn(0x7f10180b1870 msgr2=0x7f1018159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:36.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.404+0000 7f101cc36640 1 -- 172.21.15.29:0/902711848 shutdown_connections 2024-04-25T00:35:36.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.404+0000 7f101cc36640 1 -- 172.21.15.29:0/902711848 wait complete. 2024-04-25T00:35:36.423 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:36.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.544+0000 7f17bea27640 1 Processor -- start 2024-04-25T00:35:36.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.544+0000 7f17bea27640 1 -- start start 2024-04-25T00:35:36.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17bea27640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17b806b210 0x7f17b815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17bea27640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17b815d3d0 con 0x7f17b80cca10 2024-04-25T00:35:36.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17bea27640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17b806f5e0 con 0x7f17b80b67d0 2024-04-25T00:35:36.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17bea27640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f17b806f8c0 con 0x7f17b806b210 2024-04-25T00:35:36.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17bcfc4640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f17b80cca10 0x7f17b806e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55170/0 (socket says 172.21.15.29:55170) 2024-04-25T00:35:36.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17bcfc4640 1 -- 172.21.15.29:0/2164091910 learned_addr learned my addr 172.21.15.29:0/2164091910 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:36.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b7fff640 1 --2- 172.21.15.29:0/2164091910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17b806b210 0x7f17b815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3069859476 0 0) 0x7f17b00014a0 con 0x7f17b80cca10 2024-04-25T00:35:36.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17a0003530 con 0x7f17b80cca10 2024-04-25T00:35:36.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2235356702 0 0) 0x7f17a0003530 con 0x7f17b80cca10 2024-04-25T00:35:36.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17b806b210 msgr2=0x7f17b815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 --2- 172.21.15.29:0/2164091910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17b806b210 0x7f17b815ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 >> v1:172.21.15.157:6789/0 conn(0x7f17b80b67d0 legacy=0x7f17b806b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b806ddb0 con 0x7f17b80cca10 2024-04-25T00:35:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f17b0001bd0 con 0x7f17b80cca10 2024-04-25T00:35:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 >> v1:172.21.15.29:6789/0 conn(0x7f17b80cca10 legacy=0x7f17b806e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 --2- 172.21.15.29:0/2164091910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17a00039f0 0x7f17a0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 --2- 172.21.15.29:0/2164091910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a0004810 0x7f17a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 --2- 172.21.15.29:0/2164091910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17b806b210 0x7f17a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f17b806f8c0 con 0x7f17a00039f0 2024-04-25T00:35:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17a000ba80 con 0x7f17a0004810 2024-04-25T00:35:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f17a000bc80 con 0x7f17b806b210 2024-04-25T00:35:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b7fff640 1 --2- 172.21.15.29:0/2164091910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17a00039f0 0x7f17a0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17bcfc4640 1 --2- 172.21.15.29:0/2164091910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17b806b210 0x7f17a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b77fe640 1 --2- 172.21.15.29:0/2164091910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a0004810 0x7f17a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b7fff640 1 -- 172.21.15.29:0/2164091910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17b806b210 msgr2=0x7f17a00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b7fff640 1 --2- 172.21.15.29:0/2164091910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17b806b210 0x7f17a00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b7fff640 1 -- 172.21.15.29:0/2164091910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a0004810 msgr2=0x7f17a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b7fff640 1 --2- 172.21.15.29:0/2164091910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a0004810 0x7f17a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b7fff640 1 -- 172.21.15.29:0/2164091910 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a0003530 con 0x7f17a00039f0 2024-04-25T00:35:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.548+0000 7f17b77fe640 1 --2- 172.21.15.29:0/2164091910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a0004810 0x7f17a0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b7fff640 1 --2- 172.21.15.29:0/2164091910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17a00039f0 0x7f17a0003e10 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f17a8009ab0 tx=0x7f17a801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7411b7cc9e5b24c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a80b2070 con 0x7f17a00039f0 2024-04-25T00:35:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17a80a5560 con 0x7f17a00039f0 2024-04-25T00:35:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b6ffd640 1 -- 172.21.15.29:0/2164091910 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a80096b0 con 0x7f17a00039f0 2024-04-25T00:35:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 -- 172.21.15.29:0/2164091910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17a00039f0 msgr2=0x7f17a0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 --2- 172.21.15.29:0/2164091910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17a00039f0 0x7f17a0003e10 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f17a8009ab0 tx=0x7f17a801ca00 comp rx=0 tx=0).stop 2024-04-25T00:35:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bcfc4640 1 -- 172.21.15.29:0/2164091910 reap_dead start 2024-04-25T00:35:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 -- 172.21.15.29:0/2164091910 shutdown_connections 2024-04-25T00:35:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 -- 172.21.15.29:0/2164091910 >> 172.21.15.29:0/2164091910 conn(0x7f17b806c320 msgr2=0x7f17b806c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 -- 172.21.15.29:0/2164091910 shutdown_connections 2024-04-25T00:35:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 -- 172.21.15.29:0/2164091910 wait complete. 2024-04-25T00:35:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 Processor -- start 2024-04-25T00:35:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 -- start start 2024-04-25T00:35:36.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a00039f0 0x7f17b8159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17b815dc00 0x7f17b815e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b80ceae0 0x7f17b80cef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f17b806ee40 con 0x7f17b815dc00 2024-04-25T00:35:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17b806ecc0 con 0x7f17b80ceae0 2024-04-25T00:35:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f17b806efc0 con 0x7f17a00039f0 2024-04-25T00:35:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17b815dc00 0x7f17b815e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17b815dc00 0x7f17b815e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56374/0 (socket says 172.21.15.29:56374) 2024-04-25T00:35:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b77fe640 1 -- 172.21.15.29:0/3539909659 learned_addr learned my addr 172.21.15.29:0/3539909659 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b7fff640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a00039f0 0x7f17b8159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bcfc4640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b80ceae0 0x7f17b80cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b77fe640 1 -- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a00039f0 msgr2=0x7f17b8159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b77fe640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a00039f0 0x7f17b8159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b77fe640 1 -- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b80ceae0 msgr2=0x7f17b80cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b77fe640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b80ceae0 0x7f17b80cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b77fe640 1 -- 172.21.15.29:0/3539909659 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b81fb290 con 0x7f17b815dc00 2024-04-25T00:35:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b77fe640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17b815dc00 0x7f17b815e080 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00d7a0 tx=0x7f17ac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b4ff9640 1 -- 172.21.15.29:0/3539909659 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac005270 con 0x7f17b815dc00 2024-04-25T00:35:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b4ff9640 1 -- 172.21.15.29:0/3539909659 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17ac017070 con 0x7f17b815dc00 2024-04-25T00:35:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17bea27640 1 -- 172.21.15.29:0/3539909659 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b81fb570 con 0x7f17b815dc00 2024-04-25T00:35:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.552+0000 7f17b4ff9640 1 -- 172.21.15.29:0/3539909659 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac00b650 con 0x7f17b815dc00 2024-04-25T00:35:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.556+0000 7f17bea27640 1 -- 172.21.15.29:0/3539909659 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b81fbb30 con 0x7f17b815dc00 2024-04-25T00:35:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.556+0000 7f17bea27640 1 -- 172.21.15.29:0/3539909659 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b815e5c0 con 0x7f17b815dc00 2024-04-25T00:35:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.556+0000 7f17b4ff9640 1 -- 172.21.15.29:0/3539909659 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f17ac00b810 con 0x7f17b815dc00 2024-04-25T00:35:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.556+0000 7f17b4ff9640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f17a4067c20 0x7f17a406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.556+0000 7f17b4ff9640 1 -- 172.21.15.29:0/3539909659 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f17ac085ee0 con 0x7f17b815dc00 2024-04-25T00:35:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.556+0000 7f17b7fff640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f17a4067c20 0x7f17a406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.556+0000 7f17b7fff640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f17a4067c20 0x7f17a406a0e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f17a80af000 tx=0x7f17a80046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:36.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.560+0000 7f17b4ff9640 1 -- 172.21.15.29:0/3539909659 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f17ac056590 con 0x7f17b815dc00 2024-04-25T00:35:36.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.728+0000 7f17bea27640 1 -- 172.21.15.29:0/3539909659 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17b8069270 con 0x7f17b815dc00 2024-04-25T00:35:36.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.728+0000 7f17b4ff9640 1 -- 172.21.15.29:0/3539909659 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16617 (secure 0 0 0) 0x7f17ac055e90 con 0x7f17b815dc00 2024-04-25T00:35:36.731 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:36.731 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":58,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:08.767613+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:05.458923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:06.104267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:36.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.732+0000 7f17bea27640 1 -- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f17a4067c20 msgr2=0x7f17a406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.732+0000 7f17bea27640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f17a4067c20 0x7f17a406a0e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f17a80af000 tx=0x7f17a80046d0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.732+0000 7f17bea27640 1 -- 172.21.15.29:0/3539909659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17b815dc00 msgr2=0x7f17b815e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.732+0000 7f17bea27640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17b815dc00 0x7f17b815e080 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00d7a0 tx=0x7f17ac00dc60 comp rx=0 tx=0).stop 2024-04-25T00:35:36.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.732+0000 7f17bea27640 1 -- 172.21.15.29:0/3539909659 shutdown_connections 2024-04-25T00:35:36.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.732+0000 7f17bea27640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f17a4067c20 0x7f17a406a0e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.732+0000 7f17bea27640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b80ceae0 0x7f17b80cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.732+0000 7f17bea27640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17b815dc00 0x7f17b815e080 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.732+0000 7f17bea27640 1 --2- 172.21.15.29:0/3539909659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a00039f0 0x7f17b8159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.732+0000 7f17bea27640 1 -- 172.21.15.29:0/3539909659 >> 172.21.15.29:0/3539909659 conn(0x7f17b806c320 msgr2=0x7f17b80b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:36.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.732+0000 7f17bea27640 1 -- 172.21.15.29:0/3539909659 shutdown_connections 2024-04-25T00:35:36.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.732+0000 7f17bea27640 1 -- 172.21.15.29:0/3539909659 wait complete. 2024-04-25T00:35:36.751 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:35:36.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.868+0000 7f038a7bf640 1 Processor -- start 2024-04-25T00:35:36.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.868+0000 7f038a7bf640 1 -- start start 2024-04-25T00:35:36.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.868+0000 7f038a7bf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f03840b3e40 0x7f038406d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.868+0000 7f038a7bf640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f038406d980 con 0x7f03840b4d30 2024-04-25T00:35:36.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.868+0000 7f038a7bf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f038406dc60 con 0x7f038415df70 2024-04-25T00:35:36.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.868+0000 7f038a7bf640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f038406d750 con 0x7f03840b3e40 2024-04-25T00:35:36.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.868+0000 7f0388d5c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f03840b3e40 0x7f038406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.868+0000 7f03837fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f03840b4d30 0x7f03840ad8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55176/0 (socket says 172.21.15.29:55176) 2024-04-25T00:35:36.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.868+0000 7f03837fe640 1 -- 172.21.15.29:0/4197120286 learned_addr learned my addr 172.21.15.29:0/4197120286 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:36.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.868+0000 7f0388d5c640 1 -- 172.21.15.29:0/4197120286 >> v1:172.21.15.157:6789/0 conn(0x7f038415df70 legacy=0x7f03840b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:36.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.868+0000 7f0388d5c640 1 -- 172.21.15.29:0/4197120286 >> v1:172.21.15.29:6789/0 conn(0x7f03840b4d30 legacy=0x7f03840ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.868+0000 7f0388d5c640 1 -- 172.21.15.29:0/4197120286 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038406df40 con 0x7f03840b3e40 2024-04-25T00:35:36.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.872+0000 7f0388d5c640 1 --2- 172.21.15.29:0/4197120286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03840b3e40 0x7f038406d210 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f037c0097c0 tx=0x7f037c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978f19da099bb65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:36.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.872+0000 7f0382ffd640 1 -- 172.21.15.29:0/4197120286 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c0ab430 con 0x7f03840b3e40 2024-04-25T00:35:36.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.872+0000 7f0382ffd640 1 -- 172.21.15.29:0/4197120286 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f037c0b0070 con 0x7f03840b3e40 2024-04-25T00:35:36.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.872+0000 7f0382ffd640 1 -- 172.21.15.29:0/4197120286 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c0ab7f0 con 0x7f03840b3e40 2024-04-25T00:35:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.872+0000 7f038a7bf640 1 -- 172.21.15.29:0/4197120286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03840b3e40 msgr2=0x7f038406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.872+0000 7f038a7bf640 1 --2- 172.21.15.29:0/4197120286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03840b3e40 0x7f038406d210 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f037c0097c0 tx=0x7f037c01c430 comp rx=0 tx=0).stop 2024-04-25T00:35:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.872+0000 7f038a7bf640 1 -- 172.21.15.29:0/4197120286 shutdown_connections 2024-04-25T00:35:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.872+0000 7f038a7bf640 1 --2- 172.21.15.29:0/4197120286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03840b3e40 0x7f038406d210 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.872+0000 7f038a7bf640 1 -- 172.21.15.29:0/4197120286 >> 172.21.15.29:0/4197120286 conn(0x7f03840b1870 msgr2=0x7f03840b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.872+0000 7f038a7bf640 1 -- 172.21.15.29:0/4197120286 shutdown_connections 2024-04-25T00:35:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.872+0000 7f038a7bf640 1 -- 172.21.15.29:0/4197120286 wait complete. 2024-04-25T00:35:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.872+0000 7f038a7bf640 1 Processor -- start 2024-04-25T00:35:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.872+0000 7f038a7bf640 1 -- start start 2024-04-25T00:35:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f038a7bf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03840b3e40 0x7f03841557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f038a7bf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03840b4d30 0x7f03840ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f038a7bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038415df70 0x7f0384150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f038a7bf640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f038406eba0 con 0x7f03840b3e40 2024-04-25T00:35:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f038a7bf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f038406ea20 con 0x7f038415df70 2024-04-25T00:35:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f038a7bf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f038416bbc0 con 0x7f03840b4d30 2024-04-25T00:35:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0383fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03840b3e40 0x7f03841557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0383fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03840b3e40 0x7f03841557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56380/0 (socket says 172.21.15.29:56380) 2024-04-25T00:35:36.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0383fff640 1 -- 172.21.15.29:0/2706885681 learned_addr learned my addr 172.21.15.29:0/2706885681 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0388d5c640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038415df70 0x7f0384150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f03837fe640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03840b4d30 0x7f03840ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0383fff640 1 -- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03840b4d30 msgr2=0x7f03840ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0383fff640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03840b4d30 0x7f03840ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0383fff640 1 -- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038415df70 msgr2=0x7f0384150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0383fff640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038415df70 0x7f0384150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0383fff640 1 -- 172.21.15.29:0/2706885681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0384158e00 con 0x7f03840b3e40 2024-04-25T00:35:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0383fff640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03840b3e40 0x7f03841557d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f037400ba50 tx=0x7f037400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0380ff9640 1 -- 172.21.15.29:0/2706885681 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0374019070 con 0x7f03840b3e40 2024-04-25T00:35:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0380ff9640 1 -- 172.21.15.29:0/2706885681 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f037400cc20 con 0x7f03840b3e40 2024-04-25T00:35:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f038a7bf640 1 -- 172.21.15.29:0/2706885681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0384157d60 con 0x7f03840b3e40 2024-04-25T00:35:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0380ff9640 1 -- 172.21.15.29:0/2706885681 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0374014680 con 0x7f03840b3e40 2024-04-25T00:35:36.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f038a7bf640 1 -- 172.21.15.29:0/2706885681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0384158320 con 0x7f03840b3e40 2024-04-25T00:35:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f038a7bf640 1 -- 172.21.15.29:0/2706885681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0384069270 con 0x7f03840b3e40 2024-04-25T00:35:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0380ff9640 1 -- 172.21.15.29:0/2706885681 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f037401c070 con 0x7f03840b3e40 2024-04-25T00:35:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0380ff9640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0370067c20 0x7f037006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f0380ff9640 1 -- 172.21.15.29:0/2706885681 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0374085f00 con 0x7f03840b3e40 2024-04-25T00:35:36.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.876+0000 7f03837fe640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0370067c20 0x7f037006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:36.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.880+0000 7f03837fe640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0370067c20 0x7f037006a0e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0378000f70 tx=0x7f0378009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:36.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:36.880+0000 7f0380ff9640 1 -- 172.21.15.29:0/2706885681 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0384158e00 con 0x7f03840b3e40 2024-04-25T00:35:37.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.076+0000 7f038a7bf640 1 -- 172.21.15.29:0/2706885681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0384151cd0 con 0x7f03840b3e40 2024-04-25T00:35:37.081 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-04-25T00:35:37.080+0000 7f3ac00b5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:35:37.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.080+0000 7f0380ff9640 1 -- 172.21.15.29:0/2706885681 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7f0374055ee0 con 0x7f03840b3e40 2024-04-25T00:35:37.084 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:35:37.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.084+0000 7f038a7bf640 1 -- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0370067c20 msgr2=0x7f037006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:37.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.084+0000 7f038a7bf640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0370067c20 0x7f037006a0e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0378000f70 tx=0x7f0378009040 comp rx=0 tx=0).stop 2024-04-25T00:35:37.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.084+0000 7f038a7bf640 1 -- 172.21.15.29:0/2706885681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03840b3e40 msgr2=0x7f03841557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:37.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.084+0000 7f038a7bf640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03840b3e40 0x7f03841557d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f037400ba50 tx=0x7f037400bf10 comp rx=0 tx=0).stop 2024-04-25T00:35:37.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.084+0000 7f038a7bf640 1 -- 172.21.15.29:0/2706885681 shutdown_connections 2024-04-25T00:35:37.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.084+0000 7f038a7bf640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0370067c20 0x7f037006a0e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:37.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.084+0000 7f038a7bf640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038415df70 0x7f0384150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:37.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.084+0000 7f038a7bf640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03840b4d30 0x7f03840ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:37.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.084+0000 7f038a7bf640 1 --2- 172.21.15.29:0/2706885681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03840b3e40 0x7f03841557d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:37.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.084+0000 7f038a7bf640 1 -- 172.21.15.29:0/2706885681 >> 172.21.15.29:0/2706885681 conn(0x7f03840b1870 msgr2=0x7f03840cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:37.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.084+0000 7f038a7bf640 1 -- 172.21.15.29:0/2706885681 shutdown_connections 2024-04-25T00:35:37.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.084+0000 7f038a7bf640 1 -- 172.21.15.29:0/2706885681 wait complete. 2024-04-25T00:35:37.105 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:35:37.124 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:35:37.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02cc33b640 1 Processor -- start 2024-04-25T00:35:37.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02cc33b640 1 -- start start 2024-04-25T00:35:37.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02cc33b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f02c415df70 0x7f02c40b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:37.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02cc33b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c406d210 0x7f02c40b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:37.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02cc33b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02c406d980 con 0x7f02c40b4b60 2024-04-25T00:35:37.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02cc33b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f02c40b4fa0 con 0x7f02c415df70 2024-04-25T00:35:37.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02cc33b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f02c406dc60 con 0x7f02c406d210 2024-04-25T00:35:37.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02ca0d7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f02c415df70 0x7f02c40b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:37.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02c98d6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c406d210 0x7f02c40b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:37.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02c98d6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c406d210 0x7f02c40b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:47516/0 (socket says 172.21.15.29:47516) 2024-04-25T00:35:37.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02c98d6640 1 -- 172.21.15.29:0/2259905656 learned_addr learned my addr 172.21.15.29:0/2259905656 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:37.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02c98d6640 1 -- 172.21.15.29:0/2259905656 >> v2:172.21.15.157:3300/0 conn(0x7f02c415df70 msgr2=0x7f02c40b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:37.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02c98d6640 1 --2- 172.21.15.29:0/2259905656 >> v2:172.21.15.157:3300/0 conn(0x7f02c415df70 0x7f02c40b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:37.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02c98d6640 1 -- 172.21.15.29:0/2259905656 >> v1:172.21.15.157:6789/0 conn(0x7f02c40b4b60 legacy=0x7f02c4070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:37.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.220+0000 7f02c98d6640 1 -- 172.21.15.29:0/2259905656 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c406df40 con 0x7f02c406d210 2024-04-25T00:35:37.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.224+0000 7f02c98d6640 1 --2- 172.21.15.29:0/2259905656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c406d210 0x7f02c40b4620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009870 tx=0x7f02b801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85236db69d646e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:37.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.224+0000 7f02c90d5640 1 -- 172.21.15.29:0/2259905656 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b80b0070 con 0x7f02c406d210 2024-04-25T00:35:37.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.224+0000 7f02c90d5640 1 -- 172.21.15.29:0/2259905656 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b80ab600 con 0x7f02c406d210 2024-04-25T00:35:37.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.224+0000 7f02c90d5640 1 -- 172.21.15.29:0/2259905656 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b80ab9d0 con 0x7f02c406d210 2024-04-25T00:35:37.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.224+0000 7f02cc33b640 1 -- 172.21.15.29:0/2259905656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c406d210 msgr2=0x7f02c40b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:37.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.224+0000 7f02cc33b640 1 --2- 172.21.15.29:0/2259905656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c406d210 0x7f02c40b4620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009870 tx=0x7f02b801c560 comp rx=0 tx=0).stop 2024-04-25T00:35:37.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.224+0000 7f02cc33b640 1 -- 172.21.15.29:0/2259905656 shutdown_connections 2024-04-25T00:35:37.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.224+0000 7f02cc33b640 1 --2- 172.21.15.29:0/2259905656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c406d210 0x7f02c40b4620 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:37.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.224+0000 7f02cc33b640 1 --2- 172.21.15.29:0/2259905656 >> v2:172.21.15.157:3300/0 conn(0x7f02c415df70 0x7f02c40b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:37.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.224+0000 7f02cc33b640 1 -- 172.21.15.29:0/2259905656 >> 172.21.15.29:0/2259905656 conn(0x7f02c40b1870 msgr2=0x7f02c40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:37.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.224+0000 7f02cc33b640 1 -- 172.21.15.29:0/2259905656 shutdown_connections 2024-04-25T00:35:37.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.224+0000 7f02cc33b640 1 -- 172.21.15.29:0/2259905656 wait complete. 2024-04-25T00:35:37.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02cc33b640 1 Processor -- start 2024-04-25T00:35:37.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02cc33b640 1 -- start start 2024-04-25T00:35:37.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02cc33b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c406d210 0x7f02c414db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:37.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02cc33b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c40b4b60 0x7f02c415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:37.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02cc33b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02c415df70 0x7f02c414ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:37.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02cc33b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f02c406ea20 con 0x7f02c415df70 2024-04-25T00:35:37.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02cc33b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f02c415e350 con 0x7f02c40b4b60 2024-04-25T00:35:37.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02cc33b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f02c406eba0 con 0x7f02c406d210 2024-04-25T00:35:37.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02ca8d8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02c415df70 0x7f02c414ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:37.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02ca8d8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02c415df70 0x7f02c414ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56414/0 (socket says 172.21.15.29:56414) 2024-04-25T00:35:37.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02ca8d8640 1 -- 172.21.15.29:0/44236835 learned_addr learned my addr 172.21.15.29:0/44236835 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:37.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02ca0d7640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c406d210 0x7f02c414db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:37.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02c98d6640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c40b4b60 0x7f02c415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:37.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02ca8d8640 1 -- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c406d210 msgr2=0x7f02c414db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:37.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02ca8d8640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c406d210 0x7f02c414db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:37.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02ca8d8640 1 -- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c40b4b60 msgr2=0x7f02c415a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:37.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02ca8d8640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c40b4b60 0x7f02c415a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:37.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02ca8d8640 1 -- 172.21.15.29:0/44236835 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c415aaa0 con 0x7f02c415df70 2024-04-25T00:35:37.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02c98d6640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c40b4b60 0x7f02c415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:37.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02ca8d8640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02c415df70 0x7f02c414ffa0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f02b400c390 tx=0x7f02b400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:37.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02c2ffd640 1 -- 172.21.15.29:0/44236835 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b40052d0 con 0x7f02c415df70 2024-04-25T00:35:37.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02c2ffd640 1 -- 172.21.15.29:0/44236835 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b4005470 con 0x7f02c415df70 2024-04-25T00:35:37.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02c2ffd640 1 -- 172.21.15.29:0/44236835 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4007240 con 0x7f02c415df70 2024-04-25T00:35:37.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02cc33b640 1 -- 172.21.15.29:0/44236835 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c415ad80 con 0x7f02c415df70 2024-04-25T00:35:37.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02cc33b640 1 -- 172.21.15.29:0/44236835 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c41534d0 con 0x7f02c415df70 2024-04-25T00:35:37.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02cc33b640 1 -- 172.21.15.29:0/44236835 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c41532f0 con 0x7f02c415df70 2024-04-25T00:35:37.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.228+0000 7f02c2ffd640 1 -- 172.21.15.29:0/44236835 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f02b40073e0 con 0x7f02c415df70 2024-04-25T00:35:37.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.232+0000 7f02c2ffd640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02b0067c50 0x7f02b006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:37.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.232+0000 7f02c2ffd640 1 -- 172.21.15.29:0/44236835 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7f02b4085590 con 0x7f02c415df70 2024-04-25T00:35:37.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.232+0000 7f02ca0d7640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02b0067c50 0x7f02b006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:37.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.232+0000 7f02ca0d7640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02b0067c50 0x7f02b006a110 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f02bc011340 tx=0x7f02bc00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:37.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.236+0000 7f02c2ffd640 1 -- 172.21.15.29:0/44236835 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f02c41532f0 con 0x7f02c415df70 2024-04-25T00:35:37.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:37.432+0000 7f02cc33b640 1 -- 172.21.15.29:0/44236835 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f02c4069270 con 0x7f02c415df70 2024-04-25T00:35:38.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.084+0000 7f02c2ffd640 1 -- 172.21.15.29:0/44236835 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7f02b4055660 con 0x7f02c415df70 2024-04-25T00:35:38.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.088+0000 7f02cc33b640 1 -- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02b0067c50 msgr2=0x7f02b006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:38.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.088+0000 7f02cc33b640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02b0067c50 0x7f02b006a110 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f02bc011340 tx=0x7f02bc00b040 comp rx=0 tx=0).stop 2024-04-25T00:35:38.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.088+0000 7f02cc33b640 1 -- 172.21.15.29:0/44236835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02c415df70 msgr2=0x7f02c414ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:38.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.088+0000 7f02cc33b640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02c415df70 0x7f02c414ffa0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f02b400c390 tx=0x7f02b400c850 comp rx=0 tx=0).stop 2024-04-25T00:35:38.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.088+0000 7f02cc33b640 1 -- 172.21.15.29:0/44236835 shutdown_connections 2024-04-25T00:35:38.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.088+0000 7f02cc33b640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02c415df70 0x7f02c414ffa0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:38.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.088+0000 7f02cc33b640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f02b0067c50 0x7f02b006a110 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:38.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.088+0000 7f02cc33b640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c40b4b60 0x7f02c415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:38.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.088+0000 7f02cc33b640 1 --2- 172.21.15.29:0/44236835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c406d210 0x7f02c414db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:38.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.088+0000 7f02cc33b640 1 -- 172.21.15.29:0/44236835 >> 172.21.15.29:0/44236835 conn(0x7f02c40b1870 msgr2=0x7f02c415d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:38.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.088+0000 7f02cc33b640 1 -- 172.21.15.29:0/44236835 shutdown_connections 2024-04-25T00:35:38.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.088+0000 7f02cc33b640 1 -- 172.21.15.29:0/44236835 wait complete. 2024-04-25T00:35:38.114 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:35:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.288+0000 7fb26fb64640 1 Processor -- start 2024-04-25T00:35:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.288+0000 7fb26fb64640 1 -- start start 2024-04-25T00:35:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26fb64640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2681615f0 0x7fb2680d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26fb64640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2680cfa90 0x7fb2680d1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26fb64640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2680d48a0 con 0x7fb2680d0cf0 2024-04-25T00:35:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26fb64640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb2680d2570 con 0x7fb2680cfa90 2024-04-25T00:35:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26fb64640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2680b6de0 con 0x7fb2681615f0 2024-04-25T00:35:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26d900640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2681615f0 0x7fb2680d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26e101640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2680cfa90 0x7fb2680d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26e101640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2680cfa90 0x7fb2680d1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:33674/0 (socket says 172.21.15.29:33674) 2024-04-25T00:35:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26e101640 1 -- 172.21.15.29:0/1995841922 learned_addr learned my addr 172.21.15.29:0/1995841922 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26d900640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2681615f0 0x7fb2680d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:34822/0 (socket says 172.21.15.29:34822) 2024-04-25T00:35:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26e101640 1 -- 172.21.15.29:0/1995841922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2681615f0 msgr2=0x7fb2680d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26e101640 1 --2- 172.21.15.29:0/1995841922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2681615f0 0x7fb2680d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26e101640 1 -- 172.21.15.29:0/1995841922 >> v1:172.21.15.157:6789/0 conn(0x7fb2680d0cf0 legacy=0x7fb2680b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26e101640 1 -- 172.21.15.29:0/1995841922 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2680d27c0 con 0x7fb2680cfa90 2024-04-25T00:35:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.292+0000 7fb26d900640 1 --2- 172.21.15.29:0/1995841922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2681615f0 0x7fb2680d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.296+0000 7fb26e101640 1 --2- 172.21.15.29:0/1995841922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2680cfa90 0x7fb2680d1150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb260009e50 tx=0x7fb26001ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4b9f01f83a2e1e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.296+0000 7fb26c8fe640 1 -- 172.21.15.29:0/1995841922 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2600a58d0 con 0x7fb2680cfa90 2024-04-25T00:35:38.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.296+0000 7fb26c8fe640 1 -- 172.21.15.29:0/1995841922 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2600a5a70 con 0x7fb2680cfa90 2024-04-25T00:35:38.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.296+0000 7fb26c8fe640 1 -- 172.21.15.29:0/1995841922 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2600a5e40 con 0x7fb2680cfa90 2024-04-25T00:35:38.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.296+0000 7fb26fb64640 1 -- 172.21.15.29:0/1995841922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2680cfa90 msgr2=0x7fb2680d1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:38.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.296+0000 7fb26fb64640 1 --2- 172.21.15.29:0/1995841922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2680cfa90 0x7fb2680d1150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb260009e50 tx=0x7fb26001ca00 comp rx=0 tx=0).stop 2024-04-25T00:35:38.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.296+0000 7fb26fb64640 1 -- 172.21.15.29:0/1995841922 shutdown_connections 2024-04-25T00:35:38.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.296+0000 7fb26fb64640 1 --2- 172.21.15.29:0/1995841922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2680cfa90 0x7fb2680d1150 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:38.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.296+0000 7fb26fb64640 1 --2- 172.21.15.29:0/1995841922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2681615f0 0x7fb2680d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:38.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.296+0000 7fb26fb64640 1 -- 172.21.15.29:0/1995841922 >> 172.21.15.29:0/1995841922 conn(0x7fb26809d300 msgr2=0x7fb2680d3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:38.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.296+0000 7fb26fb64640 1 -- 172.21.15.29:0/1995841922 shutdown_connections 2024-04-25T00:35:38.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.296+0000 7fb26fb64640 1 -- 172.21.15.29:0/1995841922 wait complete. 2024-04-25T00:35:38.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26fb64640 1 Processor -- start 2024-04-25T00:35:38.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26fb64640 1 -- start start 2024-04-25T00:35:38.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26fb64640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2680cfa90 0x7fb268157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:38.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26fb64640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2680d0cf0 0x7fb2681f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:38.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26fb64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2681615f0 0x7fb2680c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:38.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26fb64640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb268161b50 con 0x7fb2680cfa90 2024-04-25T00:35:38.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26fb64640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681619d0 con 0x7fb2681615f0 2024-04-25T00:35:38.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26fb64640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2680b84c0 con 0x7fb2680d0cf0 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26d900640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2680cfa90 0x7fb268157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26d900640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2680cfa90 0x7fb268157670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53338/0 (socket says 172.21.15.29:53338) 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26d900640 1 -- 172.21.15.29:0/4077493137 learned_addr learned my addr 172.21.15.29:0/4077493137 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26e101640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2681615f0 0x7fb2680c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26d0ff640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2680d0cf0 0x7fb2681f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26d900640 1 -- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2680d0cf0 msgr2=0x7fb2681f7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26d900640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2680d0cf0 0x7fb2681f7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26d900640 1 -- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2681615f0 msgr2=0x7fb2680c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26d900640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2681615f0 0x7fb2680c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26d900640 1 -- 172.21.15.29:0/4077493137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2680c8740 con 0x7fb2680cfa90 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26e101640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2681615f0 0x7fb2680c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26d900640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2680cfa90 0x7fb268157670 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb26400e980 tx=0x7fb26400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb25e7fc640 1 -- 172.21.15.29:0/4077493137 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb264017070 con 0x7fb2680cfa90 2024-04-25T00:35:38.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb25e7fc640 1 -- 172.21.15.29:0/4077493137 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb26400ca00 con 0x7fb2680cfa90 2024-04-25T00:35:38.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26fb64640 1 -- 172.21.15.29:0/4077493137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2680c8a20 con 0x7fb2680cfa90 2024-04-25T00:35:38.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb25e7fc640 1 -- 172.21.15.29:0/4077493137 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26400cdd0 con 0x7fb2680cfa90 2024-04-25T00:35:38.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26fb64640 1 -- 172.21.15.29:0/4077493137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb268159400 con 0x7fb2680cfa90 2024-04-25T00:35:38.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.300+0000 7fb26fb64640 1 -- 172.21.15.29:0/4077493137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb268157c60 con 0x7fb2680cfa90 2024-04-25T00:35:38.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.304+0000 7fb25e7fc640 1 -- 172.21.15.29:0/4077493137 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb26401a070 con 0x7fb2680cfa90 2024-04-25T00:35:38.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.304+0000 7fb25e7fc640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb24c067c20 0x7fb24c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:38.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.304+0000 7fb25e7fc640 1 -- 172.21.15.29:0/4077493137 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7fb264086440 con 0x7fb2680cfa90 2024-04-25T00:35:38.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.304+0000 7fb26d0ff640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb24c067c20 0x7fb24c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:38.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.304+0000 7fb26d0ff640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb24c067c20 0x7fb24c06a0e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb258007fd0 tx=0x7fb258009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:38.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.308+0000 7fb25e7fc640 1 -- 172.21.15.29:0/4077493137 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb264056ba0 con 0x7fb2680cfa90 2024-04-25T00:35:38.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:38.468+0000 7fb26fb64640 1 -- 172.21.15.29:0/4077493137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb2680692e0 con 0x7fb2680cfa90 2024-04-25T00:35:39.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.096+0000 7fb25e7fc640 1 -- 172.21.15.29:0/4077493137 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) ==== 181+0+0 (secure 0 0 0) 0x7fb2640564a0 con 0x7fb2680cfa90 2024-04-25T00:35:39.097 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:35:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.100+0000 7fb26fb64640 1 -- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb24c067c20 msgr2=0x7fb24c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.100+0000 7fb26fb64640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb24c067c20 0x7fb24c06a0e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb258007fd0 tx=0x7fb258009040 comp rx=0 tx=0).stop 2024-04-25T00:35:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.100+0000 7fb26fb64640 1 -- 172.21.15.29:0/4077493137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2680cfa90 msgr2=0x7fb268157670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.100+0000 7fb26fb64640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2680cfa90 0x7fb268157670 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb26400e980 tx=0x7fb26400ee40 comp rx=0 tx=0).stop 2024-04-25T00:35:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.100+0000 7fb26fb64640 1 -- 172.21.15.29:0/4077493137 shutdown_connections 2024-04-25T00:35:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.100+0000 7fb26fb64640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb24c067c20 0x7fb24c06a0e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.100+0000 7fb26fb64640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2681615f0 0x7fb2680c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.100+0000 7fb26fb64640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2680d0cf0 0x7fb2681f7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.100+0000 7fb26fb64640 1 --2- 172.21.15.29:0/4077493137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2680cfa90 0x7fb268157670 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.100+0000 7fb26fb64640 1 -- 172.21.15.29:0/4077493137 >> 172.21.15.29:0/4077493137 conn(0x7fb26809d300 msgr2=0x7fb26806bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.104+0000 7fb26fb64640 1 -- 172.21.15.29:0/4077493137 shutdown_connections 2024-04-25T00:35:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.104+0000 7fb26fb64640 1 -- 172.21.15.29:0/4077493137 wait complete. 2024-04-25T00:35:39.126 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:35:39.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.256+0000 7f95edfe0640 1 Processor -- start 2024-04-25T00:35:39.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.256+0000 7f95edfe0640 1 -- start start 2024-04-25T00:35:39.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.256+0000 7f95edfe0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f95e81456d0 0x7f95e80b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:39.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.256+0000 7f95edfe0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e80cf670 con 0x7f95e806cd40 2024-04-25T00:35:39.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.256+0000 7f95edfe0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e80b4380 con 0x7f95e80b3b50 2024-04-25T00:35:39.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.256+0000 7f95edfe0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f95e80b40a0 con 0x7f95e81456d0 2024-04-25T00:35:39.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.256+0000 7f95e77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f95e81456d0 0x7f95e80b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:39.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.256+0000 7f95e7fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f95e806cd40 0x7f95e806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42568/0 (socket says 172.21.15.29:42568) 2024-04-25T00:35:39.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.256+0000 7f95e7fff640 1 -- 172.21.15.29:0/2649088328 learned_addr learned my addr 172.21.15.29:0/2649088328 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:39.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.256+0000 7f95e77fe640 1 -- 172.21.15.29:0/2649088328 >> v1:172.21.15.157:6789/0 conn(0x7f95e80b3b50 legacy=0x7f95e80cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:39.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.256+0000 7f95e77fe640 1 -- 172.21.15.29:0/2649088328 >> v1:172.21.15.29:6789/0 conn(0x7f95e806cd40 legacy=0x7f95e806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:39.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.256+0000 7f95e67fc640 1 -- 172.21.15.29:0/2649088328 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3745382122 0 0) 0x7f95e80cf670 con 0x7f95e806cd40 2024-04-25T00:35:39.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.256+0000 7f95e77fe640 1 -- 172.21.15.29:0/2649088328 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e80ce480 con 0x7f95e81456d0 2024-04-25T00:35:39.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e77fe640 1 --2- 172.21.15.29:0/2649088328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e81456d0 0x7f95e80b2b70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f95d80097c0 tx=0x7f95d801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ff060b0fd00c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:39.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e67fc640 1 -- 172.21.15.29:0/2649088328 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d80b0070 con 0x7f95e81456d0 2024-04-25T00:35:39.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e67fc640 1 -- 172.21.15.29:0/2649088328 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d8004230 con 0x7f95e81456d0 2024-04-25T00:35:39.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e67fc640 1 -- 172.21.15.29:0/2649088328 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8003620 con 0x7f95e81456d0 2024-04-25T00:35:39.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 -- 172.21.15.29:0/2649088328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e81456d0 msgr2=0x7f95e80b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:39.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 --2- 172.21.15.29:0/2649088328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e81456d0 0x7f95e80b2b70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f95d80097c0 tx=0x7f95d801c460 comp rx=0 tx=0).stop 2024-04-25T00:35:39.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 -- 172.21.15.29:0/2649088328 shutdown_connections 2024-04-25T00:35:39.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 --2- 172.21.15.29:0/2649088328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e81456d0 0x7f95e80b2b70 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:39.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 -- 172.21.15.29:0/2649088328 >> 172.21.15.29:0/2649088328 conn(0x7f95e80b1870 msgr2=0x7f95e80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:39.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 -- 172.21.15.29:0/2649088328 shutdown_connections 2024-04-25T00:35:39.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 -- 172.21.15.29:0/2649088328 wait complete. 2024-04-25T00:35:39.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 Processor -- start 2024-04-25T00:35:39.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 -- start start 2024-04-25T00:35:39.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95e806cd40 0x7f95e81586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:39.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e80b3b50 0x7f95e815f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:39.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95e81456d0 0x7f95e815fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:39.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f95e8169b80 con 0x7f95e80b3b50 2024-04-25T00:35:39.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f95e80b5000 con 0x7f95e806cd40 2024-04-25T00:35:39.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95edfe0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f95e8169d00 con 0x7f95e81456d0 2024-04-25T00:35:39.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e80b3b50 0x7f95e815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:39.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e80b3b50 0x7f95e815f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53342/0 (socket says 172.21.15.29:53342) 2024-04-25T00:35:39.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e6ffd640 1 -- 172.21.15.29:0/97686120 learned_addr learned my addr 172.21.15.29:0/97686120 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:39.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e7fff640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95e81456d0 0x7f95e815fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:39.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e77fe640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95e806cd40 0x7f95e81586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:39.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e6ffd640 1 -- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95e81456d0 msgr2=0x7f95e815fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:39.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e6ffd640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95e81456d0 0x7f95e815fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:39.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e6ffd640 1 -- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95e806cd40 msgr2=0x7f95e81586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:39.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e6ffd640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95e806cd40 0x7f95e81586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:39.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e6ffd640 1 -- 172.21.15.29:0/97686120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e813d0c0 con 0x7f95e80b3b50 2024-04-25T00:35:39.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e7fff640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95e81456d0 0x7f95e815fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:39.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.260+0000 7f95e77fe640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95e806cd40 0x7f95e81586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:39.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.264+0000 7f95e6ffd640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e80b3b50 0x7f95e815f910 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00c480 tx=0x7f95dc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:39.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.264+0000 7f95cffff640 1 -- 172.21.15.29:0/97686120 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc012830 con 0x7f95e80b3b50 2024-04-25T00:35:39.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.264+0000 7f95cffff640 1 -- 172.21.15.29:0/97686120 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95dc017070 con 0x7f95e80b3b50 2024-04-25T00:35:39.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.264+0000 7f95edfe0640 1 -- 172.21.15.29:0/97686120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e813d3a0 con 0x7f95e80b3b50 2024-04-25T00:35:39.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.264+0000 7f95cffff640 1 -- 172.21.15.29:0/97686120 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc012c30 con 0x7f95e80b3b50 2024-04-25T00:35:39.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.264+0000 7f95edfe0640 1 -- 172.21.15.29:0/97686120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e815d5e0 con 0x7f95e80b3b50 2024-04-25T00:35:39.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.264+0000 7f95edfe0640 1 -- 172.21.15.29:0/97686120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e80692e0 con 0x7f95e80b3b50 2024-04-25T00:35:39.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.264+0000 7f95cffff640 1 -- 172.21.15.29:0/97686120 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f95dc01c460 con 0x7f95e80b3b50 2024-04-25T00:35:39.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.264+0000 7f95cffff640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f95d4067c20 0x7f95d406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:39.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.264+0000 7f95cffff640 1 -- 172.21.15.29:0/97686120 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5357+0+0 (secure 0 0 0) 0x7f95dc085e40 con 0x7f95e80b3b50 2024-04-25T00:35:39.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.264+0000 7f95e77fe640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f95d4067c20 0x7f95d406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:39.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.264+0000 7f95e77fe640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f95d4067c20 0x7f95d406a0e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f95d80ad000 tx=0x7f95d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:39.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.268+0000 7f95cffff640 1 -- 172.21.15.29:0/97686120 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f95dc056760 con 0x7f95e80b3b50 2024-04-25T00:35:39.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:39.432+0000 7f95edfe0640 1 -- 172.21.15.29:0/97686120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f95e80b4de0 con 0x7f95e80b3b50 2024-04-25T00:35:40.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.104+0000 7f95cffff640 1 -- 172.21.15.29:0/97686120 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) ==== 169+0+0 (secure 0 0 0) 0x7f95dc056060 con 0x7f95e80b3b50 2024-04-25T00:35:40.106 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-04-25T00:35:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.108+0000 7f95edfe0640 1 -- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f95d4067c20 msgr2=0x7f95d406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.108+0000 7f95edfe0640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f95d4067c20 0x7f95d406a0e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f95d80ad000 tx=0x7f95d8008040 comp rx=0 tx=0).stop 2024-04-25T00:35:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.108+0000 7f95edfe0640 1 -- 172.21.15.29:0/97686120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e80b3b50 msgr2=0x7f95e815f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.108+0000 7f95edfe0640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e80b3b50 0x7f95e815f910 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00c480 tx=0x7f95dc00c940 comp rx=0 tx=0).stop 2024-04-25T00:35:40.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.112+0000 7f95edfe0640 1 -- 172.21.15.29:0/97686120 shutdown_connections 2024-04-25T00:35:40.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.112+0000 7f95edfe0640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95e81456d0 0x7f95e815fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.112+0000 7f95edfe0640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e80b3b50 0x7f95e815f910 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.112+0000 7f95edfe0640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f95d4067c20 0x7f95d406a0e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.112+0000 7f95edfe0640 1 --2- 172.21.15.29:0/97686120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95e806cd40 0x7f95e81586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.112+0000 7f95edfe0640 1 -- 172.21.15.29:0/97686120 >> 172.21.15.29:0/97686120 conn(0x7f95e80b1870 msgr2=0x7f95e80b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:40.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.112+0000 7f95edfe0640 1 -- 172.21.15.29:0/97686120 shutdown_connections 2024-04-25T00:35:40.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.112+0000 7f95edfe0640 1 -- 172.21.15.29:0/97686120 wait complete. 2024-04-25T00:35:40.138 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:35:40.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.256+0000 7fec487b0640 1 Processor -- start 2024-04-25T00:35:40.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.256+0000 7fec487b0640 1 -- start start 2024-04-25T00:35:40.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec487b0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fec4006d320 0x7fec400d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec487b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec4006c530 0x7fec400cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec487b0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec4015e7a0 con 0x7fec4015e3c0 2024-04-25T00:35:40.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec487b0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fec400b4bc0 con 0x7fec4006d320 2024-04-25T00:35:40.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec487b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fec400d10f0 con 0x7fec4006c530 2024-04-25T00:35:40.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec4654c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fec4015e3c0 0x7fec4006cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42584/0 (socket says 172.21.15.29:42584) 2024-04-25T00:35:40.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec45d4b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fec4006d320 0x7fec400d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec4654c640 1 -- 172.21.15.29:0/1732924265 learned_addr learned my addr 172.21.15.29:0/1732924265 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:40.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec45d4b640 1 -- 172.21.15.29:0/1732924265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec4006c530 msgr2=0x7fec400cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:40.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec46d4d640 1 --2- 172.21.15.29:0/1732924265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec4006c530 0x7fec400cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec45d4b640 1 --2- 172.21.15.29:0/1732924265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec4006c530 0x7fec400cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec45d4b640 1 -- 172.21.15.29:0/1732924265 >> v1:172.21.15.29:6789/0 conn(0x7fec4015e3c0 legacy=0x7fec4006cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec4554a640 1 -- 172.21.15.29:0/1732924265 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3176203006 0 0) 0x7fec4015e7a0 con 0x7fec4015e3c0 2024-04-25T00:35:40.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec45d4b640 1 -- 172.21.15.29:0/1732924265 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec400cc560 con 0x7fec4006d320 2024-04-25T00:35:40.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec46d4d640 1 --2- 172.21.15.29:0/1732924265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec4006c530 0x7fec400cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:40.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec45d4b640 1 --2- 172.21.15.29:0/1732924265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec4006d320 0x7fec400d0a80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fec300097c0 tx=0x7fec3001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0e26dd19fd629e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:40.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec4554a640 1 -- 172.21.15.29:0/1732924265 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec300b2070 con 0x7fec4006d320 2024-04-25T00:35:40.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec4554a640 1 -- 172.21.15.29:0/1732924265 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec300ad640 con 0x7fec4006d320 2024-04-25T00:35:40.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec4554a640 1 -- 172.21.15.29:0/1732924265 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec300ada10 con 0x7fec4006d320 2024-04-25T00:35:40.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec487b0640 1 -- 172.21.15.29:0/1732924265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec4006d320 msgr2=0x7fec400d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.260+0000 7fec487b0640 1 --2- 172.21.15.29:0/1732924265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec4006d320 0x7fec400d0a80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fec300097c0 tx=0x7fec3001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 -- 172.21.15.29:0/1732924265 shutdown_connections 2024-04-25T00:35:40.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 --2- 172.21.15.29:0/1732924265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec4006c530 0x7fec400cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 --2- 172.21.15.29:0/1732924265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec4006d320 0x7fec400d0a80 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 -- 172.21.15.29:0/1732924265 >> 172.21.15.29:0/1732924265 conn(0x7fec400d1d70 msgr2=0x7fec400d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:40.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 -- 172.21.15.29:0/1732924265 shutdown_connections 2024-04-25T00:35:40.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 -- 172.21.15.29:0/1732924265 wait complete. 2024-04-25T00:35:40.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 Processor -- start 2024-04-25T00:35:40.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 -- start start 2024-04-25T00:35:40.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec4006c530 0x7fec400cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec4006d320 0x7fec40158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec4015e3c0 0x7fec401589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fec400cdf10 con 0x7fec4015e3c0 2024-04-25T00:35:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec400cdd90 con 0x7fec4006c530 2024-04-25T00:35:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec487b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fec400ce090 con 0x7fec4006d320 2024-04-25T00:35:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec46d4d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec4015e3c0 0x7fec401589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec46d4d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec4015e3c0 0x7fec401589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53354/0 (socket says 172.21.15.29:53354) 2024-04-25T00:35:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec46d4d640 1 -- 172.21.15.29:0/704589776 learned_addr learned my addr 172.21.15.29:0/704589776 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec45d4b640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec4006d320 0x7fec40158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec4654c640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec4006c530 0x7fec400cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec46d4d640 1 -- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec4006d320 msgr2=0x7fec40158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec46d4d640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec4006d320 0x7fec40158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec46d4d640 1 -- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec4006c530 msgr2=0x7fec400cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec46d4d640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec4006c530 0x7fec400cbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec46d4d640 1 -- 172.21.15.29:0/704589776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec40161160 con 0x7fec4015e3c0 2024-04-25T00:35:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec45d4b640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec4006d320 0x7fec40158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec46d4d640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec4015e3c0 0x7fec401589a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fec3800cd80 tx=0x7fec3800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec36ffd640 1 -- 172.21.15.29:0/704589776 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3800f680 con 0x7fec4015e3c0 2024-04-25T00:35:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec36ffd640 1 -- 172.21.15.29:0/704589776 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3800f840 con 0x7fec4015e3c0 2024-04-25T00:35:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.264+0000 7fec36ffd640 1 -- 172.21.15.29:0/704589776 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3800fc30 con 0x7fec4015e3c0 2024-04-25T00:35:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.268+0000 7fec487b0640 1 -- 172.21.15.29:0/704589776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec40161440 con 0x7fec4015e3c0 2024-04-25T00:35:40.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.268+0000 7fec487b0640 1 -- 172.21.15.29:0/704589776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec40161720 con 0x7fec4015e3c0 2024-04-25T00:35:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.268+0000 7fec487b0640 1 -- 172.21.15.29:0/704589776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec400c5ad0 con 0x7fec4015e3c0 2024-04-25T00:35:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.268+0000 7fec36ffd640 1 -- 172.21.15.29:0/704589776 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fec3800e3e0 con 0x7fec4015e3c0 2024-04-25T00:35:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.268+0000 7fec36ffd640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fec24067c20 0x7fec2406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.268+0000 7fec36ffd640 1 -- 172.21.15.29:0/704589776 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fec38086d90 con 0x7fec4015e3c0 2024-04-25T00:35:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.268+0000 7fec4654c640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fec24067c20 0x7fec2406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.268+0000 7fec4654c640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fec24067c20 0x7fec2406a0e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fec3c000f70 tx=0x7fec3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:40.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.272+0000 7fec36ffd640 1 -- 172.21.15.29:0/704589776 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fec38014030 con 0x7fec4015e3c0 2024-04-25T00:35:40.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.436+0000 7fec487b0640 1 -- 172.21.15.29:0/704589776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fec40069270 con 0x7fec4015e3c0 2024-04-25T00:35:40.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.440+0000 7fec36ffd640 1 -- 172.21.15.29:0/704589776 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fec38057700 con 0x7fec4015e3c0 2024-04-25T00:35:40.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.444+0000 7fec487b0640 1 -- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fec24067c20 msgr2=0x7fec2406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.444+0000 7fec487b0640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fec24067c20 0x7fec2406a0e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fec3c000f70 tx=0x7fec3c009040 comp rx=0 tx=0).stop 2024-04-25T00:35:40.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.444+0000 7fec487b0640 1 -- 172.21.15.29:0/704589776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec4015e3c0 msgr2=0x7fec401589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.444+0000 7fec487b0640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec4015e3c0 0x7fec401589a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fec3800cd80 tx=0x7fec3800a650 comp rx=0 tx=0).stop 2024-04-25T00:35:40.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.444+0000 7fec487b0640 1 -- 172.21.15.29:0/704589776 shutdown_connections 2024-04-25T00:35:40.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.444+0000 7fec487b0640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec4015e3c0 0x7fec401589a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.444+0000 7fec487b0640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec4006d320 0x7fec40158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.444+0000 7fec487b0640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fec24067c20 0x7fec2406a0e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.444+0000 7fec487b0640 1 --2- 172.21.15.29:0/704589776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec4006c530 0x7fec400cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.444+0000 7fec487b0640 1 -- 172.21.15.29:0/704589776 >> 172.21.15.29:0/704589776 conn(0x7fec400d1d70 msgr2=0x7fec4006ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.444+0000 7fec487b0640 1 -- 172.21.15.29:0/704589776 shutdown_connections 2024-04-25T00:35:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.444+0000 7fec487b0640 1 -- 172.21.15.29:0/704589776 wait complete. 2024-04-25T00:35:40.452 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:35:40.455 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:35:40.466 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-04-25T00:35:40.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.668+0000 7f07c6863640 1 Processor -- start 2024-04-25T00:35:40.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.668+0000 7f07c6863640 1 -- start start 2024-04-25T00:35:40.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.672+0000 7f07c6863640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f07c00aed50 0x7f07c015cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.672+0000 7f07c6863640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07c00cda50 0x7f07c00cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.672+0000 7f07c6863640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07c00afe50 con 0x7f07c00b4ab0 2024-04-25T00:35:40.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.672+0000 7f07c6863640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f07c015d430 con 0x7f07c00aed50 2024-04-25T00:35:40.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.672+0000 7f07c6863640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f07c00ce3d0 con 0x7f07c00cda50 2024-04-25T00:35:40.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.672+0000 7f07bffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f07c00aed50 0x7f07c015cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.672+0000 7f07bffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f07c00aed50 0x7f07c015cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53364/0 (socket says 172.21.15.29:53364) 2024-04-25T00:35:40.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.672+0000 7f07bffff640 1 -- 172.21.15.29:0/1800878983 learned_addr learned my addr 172.21.15.29:0/1800878983 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.672+0000 7f07c4e00640 1 --2- 172.21.15.29:0/1800878983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07c00cda50 0x7f07c00cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.672+0000 7f07bffff640 1 -- 172.21.15.29:0/1800878983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07c00cda50 msgr2=0x7f07c00cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.672+0000 7f07bffff640 1 --2- 172.21.15.29:0/1800878983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07c00cda50 0x7f07c00cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.672+0000 7f07bffff640 1 -- 172.21.15.29:0/1800878983 >> v1:172.21.15.157:6789/0 conn(0x7f07c00b4ab0 legacy=0x7f07c00b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.672+0000 7f07bffff640 1 -- 172.21.15.29:0/1800878983 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c00b3be0 con 0x7f07c00aed50 2024-04-25T00:35:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07bffff640 1 --2- 172.21.15.29:0/1800878983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c00aed50 0x7f07c015cef0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009870 tx=0x7f07b001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bc6d8fbe8b2442 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07beffd640 1 -- 172.21.15.29:0/1800878983 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00a95f0 con 0x7f07c00aed50 2024-04-25T00:35:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07beffd640 1 -- 172.21.15.29:0/1800878983 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b00a9790 con 0x7f07c00aed50 2024-04-25T00:35:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07beffd640 1 -- 172.21.15.29:0/1800878983 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00a9b60 con 0x7f07c00aed50 2024-04-25T00:35:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 -- 172.21.15.29:0/1800878983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c00aed50 msgr2=0x7f07c015cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 --2- 172.21.15.29:0/1800878983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c00aed50 0x7f07c015cef0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009870 tx=0x7f07b001c530 comp rx=0 tx=0).stop 2024-04-25T00:35:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 -- 172.21.15.29:0/1800878983 shutdown_connections 2024-04-25T00:35:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 --2- 172.21.15.29:0/1800878983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07c00cda50 0x7f07c00cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 --2- 172.21.15.29:0/1800878983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c00aed50 0x7f07c015cef0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 -- 172.21.15.29:0/1800878983 >> 172.21.15.29:0/1800878983 conn(0x7f07c0075030 msgr2=0x7f07c00b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 -- 172.21.15.29:0/1800878983 shutdown_connections 2024-04-25T00:35:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 -- 172.21.15.29:0/1800878983 wait complete. 2024-04-25T00:35:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 Processor -- start 2024-04-25T00:35:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 -- start start 2024-04-25T00:35:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c00aed50 0x7f07c014d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07c00b4ab0 0x7f07c014b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c00cda50 0x7f07c015f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f07c00af130 con 0x7f07c00cda50 2024-04-25T00:35:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0071650 con 0x7f07c00aed50 2024-04-25T00:35:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c6863640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f07c00b58a0 con 0x7f07c00b4ab0 2024-04-25T00:35:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c4e00640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c00cda50 0x7f07c015f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c4e00640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c00cda50 0x7f07c015f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53372/0 (socket says 172.21.15.29:53372) 2024-04-25T00:35:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07c4e00640 1 -- 172.21.15.29:0/3544092843 learned_addr learned my addr 172.21.15.29:0/3544092843 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07bffff640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c00aed50 0x7f07c014d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.676+0000 7f07bf7fe640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07c00b4ab0 0x7f07c014b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07c4e00640 1 -- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07c00b4ab0 msgr2=0x7f07c014b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07c4e00640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07c00b4ab0 0x7f07c014b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07c4e00640 1 -- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c00aed50 msgr2=0x7f07c014d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07c4e00640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c00aed50 0x7f07c014d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07c4e00640 1 -- 172.21.15.29:0/3544092843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c014dbc0 con 0x7f07c00cda50 2024-04-25T00:35:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07bffff640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c00aed50 0x7f07c014d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07bf7fe640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07c00b4ab0 0x7f07c014b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07c4e00640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c00cda50 0x7f07c015f960 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f07b800efc0 tx=0x7f07b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07bcff9640 1 -- 172.21.15.29:0/3544092843 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8018020 con 0x7f07c00cda50 2024-04-25T00:35:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07bcff9640 1 -- 172.21.15.29:0/3544092843 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b8014480 con 0x7f07c00cda50 2024-04-25T00:35:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07c6863640 1 -- 172.21.15.29:0/3544092843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c014dea0 con 0x7f07c00cda50 2024-04-25T00:35:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07bcff9640 1 -- 172.21.15.29:0/3544092843 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8014870 con 0x7f07c00cda50 2024-04-25T00:35:40.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07c6863640 1 -- 172.21.15.29:0/3544092843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c014b550 con 0x7f07c00cda50 2024-04-25T00:35:40.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07c6863640 1 -- 172.21.15.29:0/3544092843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c014e2b0 con 0x7f07c00cda50 2024-04-25T00:35:40.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07bcff9640 1 -- 172.21.15.29:0/3544092843 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f07b801b020 con 0x7f07c00cda50 2024-04-25T00:35:40.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07bcff9640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f07ac067c20 0x7f07ac06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07bcff9640 1 -- 172.21.15.29:0/3544092843 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f07b8086a30 con 0x7f07c00cda50 2024-04-25T00:35:40.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07bffff640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f07ac067c20 0x7f07ac06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.680+0000 7f07bffff640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f07ac067c20 0x7f07ac06a0e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f07b00ae000 tx=0x7f07b00b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:40.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.684+0000 7f07bcff9640 1 -- 172.21.15.29:0/3544092843 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f07b8022070 con 0x7f07c00cda50 2024-04-25T00:35:40.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:40.908+0000 7f07c6863640 1 -- 172.21.15.29:0/3544092843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f07c014e070 con 0x7f07c00cda50 2024-04-25T00:35:41.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.104+0000 7f07bcff9640 1 -- 172.21.15.29:0/3544092843 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v225) ==== 161+0+0 (secure 0 0 0) 0x7f07c014e070 con 0x7f07c00cda50 2024-04-25T00:35:41.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.108+0000 7f07c6863640 1 -- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f07ac067c20 msgr2=0x7f07ac06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.108+0000 7f07c6863640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f07ac067c20 0x7f07ac06a0e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f07b00ae000 tx=0x7f07b00b3040 comp rx=0 tx=0).stop 2024-04-25T00:35:41.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.108+0000 7f07c6863640 1 -- 172.21.15.29:0/3544092843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c00cda50 msgr2=0x7f07c015f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.108+0000 7f07c6863640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c00cda50 0x7f07c015f960 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f07b800efc0 tx=0x7f07b800c490 comp rx=0 tx=0).stop 2024-04-25T00:35:41.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.108+0000 7f07c6863640 1 -- 172.21.15.29:0/3544092843 shutdown_connections 2024-04-25T00:35:41.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.108+0000 7f07c6863640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c00cda50 0x7f07c015f960 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.108+0000 7f07c6863640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07c00b4ab0 0x7f07c014b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.108+0000 7f07c6863640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f07ac067c20 0x7f07ac06a0e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.108+0000 7f07c6863640 1 --2- 172.21.15.29:0/3544092843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c00aed50 0x7f07c014d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.108+0000 7f07c6863640 1 -- 172.21.15.29:0/3544092843 >> 172.21.15.29:0/3544092843 conn(0x7f07c0075030 msgr2=0x7f07c015d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:41.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.108+0000 7f07c6863640 1 -- 172.21.15.29:0/3544092843 shutdown_connections 2024-04-25T00:35:41.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.108+0000 7f07c6863640 1 -- 172.21.15.29:0/3544092843 wait complete. 2024-04-25T00:35:41.131 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2024-04-25T00:35:41.131 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2024-04-25T00:35:41.131 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:41.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f380cb98640 1 Processor -- start 2024-04-25T00:35:41.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f380cb98640 1 -- start start 2024-04-25T00:35:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f380cb98640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38080b3140 0x7f380806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f380cb98640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38080b4b10 0x7f38080b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f380cb98640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38080b41e0 con 0x7f38080b3a60 2024-04-25T00:35:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f380cb98640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f380806d960 con 0x7f38080b4b10 2024-04-25T00:35:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f380cb98640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f380806dc40 con 0x7f38080b3140 2024-04-25T00:35:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f3805d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38080b4b10 0x7f38080b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f3806d9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f38080b3a60 0x7f380815c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42586/0 (socket says 172.21.15.29:42586) 2024-04-25T00:35:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f3806d9d640 1 -- 172.21.15.29:0/158292033 learned_addr learned my addr 172.21.15.29:0/158292033 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f380659c640 1 --2- 172.21.15.29:0/158292033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38080b3140 0x7f380806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f3805d9b640 1 -- 172.21.15.29:0/158292033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38080b3140 msgr2=0x7f380806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f3805d9b640 1 --2- 172.21.15.29:0/158292033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38080b3140 0x7f380806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f3805d9b640 1 -- 172.21.15.29:0/158292033 >> v1:172.21.15.29:6789/0 conn(0x7f38080b3a60 legacy=0x7f380815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f3805d9b640 1 -- 172.21.15.29:0/158292033 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380806df20 con 0x7f38080b4b10 2024-04-25T00:35:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.252+0000 7f380559a640 1 -- 172.21.15.29:0/158292033 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3344836075 0 0) 0x7f38080b41e0 con 0x7f38080b3a60 2024-04-25T00:35:41.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f3805d9b640 1 --2- 172.21.15.29:0/158292033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38080b4b10 0x7f38080b3520 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f37fc0097c0 tx=0x7f37fc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5870641713b614f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:41.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380559a640 1 -- 172.21.15.29:0/158292033 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0b0070 con 0x7f38080b4b10 2024-04-25T00:35:41.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380559a640 1 -- 172.21.15.29:0/158292033 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37fc0ab600 con 0x7f38080b4b10 2024-04-25T00:35:41.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380559a640 1 -- 172.21.15.29:0/158292033 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0ab9d0 con 0x7f38080b4b10 2024-04-25T00:35:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 -- 172.21.15.29:0/158292033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38080b4b10 msgr2=0x7f38080b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 --2- 172.21.15.29:0/158292033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38080b4b10 0x7f38080b3520 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f37fc0097c0 tx=0x7f37fc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 -- 172.21.15.29:0/158292033 shutdown_connections 2024-04-25T00:35:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 --2- 172.21.15.29:0/158292033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38080b4b10 0x7f38080b3520 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 --2- 172.21.15.29:0/158292033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38080b3140 0x7f380806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 -- 172.21.15.29:0/158292033 >> 172.21.15.29:0/158292033 conn(0x7f38080b1870 msgr2=0x7f38080b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 -- 172.21.15.29:0/158292033 shutdown_connections 2024-04-25T00:35:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 -- 172.21.15.29:0/158292033 wait complete. 2024-04-25T00:35:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 Processor -- start 2024-04-25T00:35:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 -- start start 2024-04-25T00:35:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38080b3140 0x7f380813f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38080b3a60 0x7f3808164330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38080b4b10 0x7f3808164870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3808168b70 con 0x7f38080b4b10 2024-04-25T00:35:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38081689f0 con 0x7f38080b3a60 2024-04-25T00:35:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.256+0000 7f380cb98640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3808168cf0 con 0x7f38080b3140 2024-04-25T00:35:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f3806d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38080b4b10 0x7f3808164870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f3806d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38080b4b10 0x7f3808164870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53388/0 (socket says 172.21.15.29:53388) 2024-04-25T00:35:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f3806d9d640 1 -- 172.21.15.29:0/2329292177 learned_addr learned my addr 172.21.15.29:0/2329292177 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f380659c640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38080b3140 0x7f380813f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f3805d9b640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38080b3a60 0x7f3808164330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f3806d9d640 1 -- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38080b3140 msgr2=0x7f380813f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f3806d9d640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38080b3140 0x7f380813f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f3806d9d640 1 -- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38080b3a60 msgr2=0x7f3808164330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f3806d9d640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38080b3a60 0x7f3808164330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f3806d9d640 1 -- 172.21.15.29:0/2329292177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3808163b70 con 0x7f38080b4b10 2024-04-25T00:35:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f380659c640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38080b3140 0x7f380813f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f3806d9d640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38080b4b10 0x7f3808164870 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f380000c7a0 tx=0x7f380000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f37f6ffd640 1 -- 172.21.15.29:0/2329292177 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800018070 con 0x7f38080b4b10 2024-04-25T00:35:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f37f6ffd640 1 -- 172.21.15.29:0/2329292177 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f380000a640 con 0x7f38080b4b10 2024-04-25T00:35:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f37f6ffd640 1 -- 172.21.15.29:0/2329292177 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380000aab0 con 0x7f38080b4b10 2024-04-25T00:35:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f380cb98640 1 -- 172.21.15.29:0/2329292177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38081664a0 con 0x7f38080b4b10 2024-04-25T00:35:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f380cb98640 1 -- 172.21.15.29:0/2329292177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3808166780 con 0x7f38080b4b10 2024-04-25T00:35:41.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f380cb98640 1 -- 172.21.15.29:0/2329292177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ec0052a0 con 0x7f38080b4b10 2024-04-25T00:35:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f37f6ffd640 1 -- 172.21.15.29:0/2329292177 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f380001b070 con 0x7f38080b4b10 2024-04-25T00:35:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f37f6ffd640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37e8067c20 0x7f37e806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f37f6ffd640 1 -- 172.21.15.29:0/2329292177 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f38000855c0 con 0x7f38080b4b10 2024-04-25T00:35:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.260+0000 7f380659c640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37e8067c20 0x7f37e806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.264+0000 7f380659c640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37e8067c20 0x7f37e806a0e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3808166080 tx=0x7f37f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:41.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.264+0000 7f37f6ffd640 1 -- 172.21.15.29:0/2329292177 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3800055ef0 con 0x7f38080b4b10 2024-04-25T00:35:41.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.428+0000 7f380cb98640 1 -- 172.21.15.29:0/2329292177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37ec005860 con 0x7f38080b4b10 2024-04-25T00:35:41.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.428+0000 7f37f6ffd640 1 -- 172.21.15.29:0/2329292177 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+12970 (secure 0 0 0) 0x7f380005c0c0 con 0x7f38080b4b10 2024-04-25T00:35:41.430 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:41.431 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":61,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:40.098916+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6835/802824676":"2024-04-26T00:35:37.082542+0000","172.21.15.29:6834/802824676":"2024-04-26T00:35:37.082542+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.428+0000 7f380cb98640 1 -- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37e8067c20 msgr2=0x7f37e806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.428+0000 7f380cb98640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37e8067c20 0x7f37e806a0e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3808166080 tx=0x7f37f8008040 comp rx=0 tx=0).stop 2024-04-25T00:35:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.432+0000 7f380cb98640 1 -- 172.21.15.29:0/2329292177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38080b4b10 msgr2=0x7f3808164870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.432+0000 7f380cb98640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38080b4b10 0x7f3808164870 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f380000c7a0 tx=0x7f380000cc60 comp rx=0 tx=0).stop 2024-04-25T00:35:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.432+0000 7f380cb98640 1 -- 172.21.15.29:0/2329292177 shutdown_connections 2024-04-25T00:35:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.432+0000 7f380cb98640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38080b4b10 0x7f3808164870 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.432+0000 7f380cb98640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37e8067c20 0x7f37e806a0e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.432+0000 7f380cb98640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38080b3a60 0x7f3808164330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.432+0000 7f380cb98640 1 --2- 172.21.15.29:0/2329292177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38080b3140 0x7f380813f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.432+0000 7f380cb98640 1 -- 172.21.15.29:0/2329292177 >> 172.21.15.29:0/2329292177 conn(0x7f38080b1870 msgr2=0x7f380806cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.432+0000 7f380cb98640 1 -- 172.21.15.29:0/2329292177 shutdown_connections 2024-04-25T00:35:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.432+0000 7f380cb98640 1 -- 172.21.15.29:0/2329292177 wait complete. 2024-04-25T00:35:41.450 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:35:41.451 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:35:41.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7ebbd0640 1 Processor -- start 2024-04-25T00:35:41.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7ebbd0640 1 -- start start 2024-04-25T00:35:41.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7ebbd0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc7e40cf690 0x7fc7e40d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7ebbd0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7e40d2a90 0x7fc7e40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7ebbd0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e40d2ea0 con 0x7fc7e415e200 2024-04-25T00:35:41.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7ebbd0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e40b38d0 con 0x7fc7e40cf690 2024-04-25T00:35:41.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7ebbd0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e40cfc10 con 0x7fc7e40d2a90 2024-04-25T00:35:41.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7e916b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc7e40cf690 0x7fc7e40d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7e916b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc7e40cf690 0x7fc7e40d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53398/0 (socket says 172.21.15.29:53398) 2024-04-25T00:35:41.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7e916b640 1 -- 172.21.15.29:0/1687787712 learned_addr learned my addr 172.21.15.29:0/1687787712 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:41.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7ea16d640 1 --2- 172.21.15.29:0/1687787712 >> v2:172.21.15.157:3300/0 conn(0x7fc7e40d2a90 0x7fc7e40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7e916b640 1 -- 172.21.15.29:0/1687787712 >> v2:172.21.15.157:3300/0 conn(0x7fc7e40d2a90 msgr2=0x7fc7e40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:41.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7e916b640 1 --2- 172.21.15.29:0/1687787712 >> v2:172.21.15.157:3300/0 conn(0x7fc7e40d2a90 0x7fc7e40b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7e916b640 1 -- 172.21.15.29:0/1687787712 >> v1:172.21.15.157:6789/0 conn(0x7fc7e415e200 legacy=0x7fc7e40d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:41.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.576+0000 7fc7e916b640 1 -- 172.21.15.29:0/1687787712 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e40cfef0 con 0x7fc7e40cf690 2024-04-25T00:35:41.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7e916b640 1 --2- 172.21.15.29:0/1687787712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7e40cf690 0x7fc7e40d2550 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40cfb20 tx=0x7fc7d401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7217f96742fac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:41.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7e896a640 1 -- 172.21.15.29:0/1687787712 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40b0070 con 0x7fc7e40cf690 2024-04-25T00:35:41.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7e896a640 1 -- 172.21.15.29:0/1687787712 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d40ab600 con 0x7fc7e40cf690 2024-04-25T00:35:41.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7e896a640 1 -- 172.21.15.29:0/1687787712 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40ab9d0 con 0x7fc7e40cf690 2024-04-25T00:35:41.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/1687787712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7e40cf690 msgr2=0x7fc7e40d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 --2- 172.21.15.29:0/1687787712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7e40cf690 0x7fc7e40d2550 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40cfb20 tx=0x7fc7d401c300 comp rx=0 tx=0).stop 2024-04-25T00:35:41.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/1687787712 shutdown_connections 2024-04-25T00:35:41.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 --2- 172.21.15.29:0/1687787712 >> v2:172.21.15.157:3300/0 conn(0x7fc7e40d2a90 0x7fc7e40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 --2- 172.21.15.29:0/1687787712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7e40cf690 0x7fc7e40d2550 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/1687787712 >> 172.21.15.29:0/1687787712 conn(0x7fc7e409d280 msgr2=0x7fc7e40d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:41.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/1687787712 shutdown_connections 2024-04-25T00:35:41.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/1687787712 wait complete. 2024-04-25T00:35:41.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 Processor -- start 2024-04-25T00:35:41.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 -- start start 2024-04-25T00:35:41.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e40cf690 0x7fc7e41514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7e40d2a90 0x7fc7e40d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e415e200 0x7fc7e40d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e40b51f0 con 0x7fc7e40d2a90 2024-04-25T00:35:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e40b5070 con 0x7fc7e415e200 2024-04-25T00:35:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.580+0000 7fc7ebbd0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e40b5370 con 0x7fc7e40cf690 2024-04-25T00:35:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7e916b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7e40d2a90 0x7fc7e40d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7e916b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7e40d2a90 0x7fc7e40d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53402/0 (socket says 172.21.15.29:53402) 2024-04-25T00:35:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7e916b640 1 -- 172.21.15.29:0/787105870 learned_addr learned my addr 172.21.15.29:0/787105870 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7e996c640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e40cf690 0x7fc7e41514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7ea16d640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e415e200 0x7fc7e40d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7e916b640 1 -- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e40cf690 msgr2=0x7fc7e41514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7e916b640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e40cf690 0x7fc7e41514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7e916b640 1 -- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e415e200 msgr2=0x7fc7e40d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7e916b640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e415e200 0x7fc7e40d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7e916b640 1 -- 172.21.15.29:0/787105870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e415e890 con 0x7fc7e40d2a90 2024-04-25T00:35:41.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7e916b640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7e40d2a90 0x7fc7e40d0580 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40ae040 tx=0x7fc7d40190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:41.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7da7fc640 1 -- 172.21.15.29:0/787105870 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40b0040 con 0x7fc7e40d2a90 2024-04-25T00:35:41.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7da7fc640 1 -- 172.21.15.29:0/787105870 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d40b4420 con 0x7fc7e40d2a90 2024-04-25T00:35:41.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/787105870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e415eb70 con 0x7fc7e40d2a90 2024-04-25T00:35:41.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7da7fc640 1 -- 172.21.15.29:0/787105870 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40b4810 con 0x7fc7e40d2a90 2024-04-25T00:35:41.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/787105870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e415f130 con 0x7fc7e40d2a90 2024-04-25T00:35:41.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/787105870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e4069270 con 0x7fc7e40d2a90 2024-04-25T00:35:41.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7da7fc640 1 -- 172.21.15.29:0/787105870 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fc7d40b49b0 con 0x7fc7e40d2a90 2024-04-25T00:35:41.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7da7fc640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc7c8067c20 0x7fc7c806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7da7fc640 1 -- 172.21.15.29:0/787105870 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fc7d411fc90 con 0x7fc7e40d2a90 2024-04-25T00:35:41.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.584+0000 7fc7e996c640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc7c8067c20 0x7fc7c806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.588+0000 7fc7e996c640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc7c8067c20 0x7fc7c806a0e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40d1500 tx=0x7fc7e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:41.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.588+0000 7fc7da7fc640 1 -- 172.21.15.29:0/787105870 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc7e40d2ea0 con 0x7fc7e40d2a90 2024-04-25T00:35:41.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.752+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/787105870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc7e40d1fa0 con 0x7fc7e40d2a90 2024-04-25T00:35:41.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.752+0000 7fc7da7fc640 1 -- 172.21.15.29:0/787105870 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) ==== 93+0+10 (secure 0 0 0) 0x7fc7d40f0540 con 0x7fc7e40d2a90 2024-04-25T00:35:41.756 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:35:41.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.756+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc7c8067c20 msgr2=0x7fc7c806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.756+0000 7fc7ebbd0640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc7c8067c20 0x7fc7c806a0e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40d1500 tx=0x7fc7e0008040 comp rx=0 tx=0).stop 2024-04-25T00:35:41.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.756+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/787105870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7e40d2a90 msgr2=0x7fc7e40d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.756+0000 7fc7ebbd0640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7e40d2a90 0x7fc7e40d0580 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40ae040 tx=0x7fc7d40190a0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.756+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/787105870 shutdown_connections 2024-04-25T00:35:41.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.756+0000 7fc7ebbd0640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc7c8067c20 0x7fc7c806a0e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.756+0000 7fc7ebbd0640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e415e200 0x7fc7e40d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.756+0000 7fc7ebbd0640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7e40d2a90 0x7fc7e40d0580 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.756+0000 7fc7ebbd0640 1 --2- 172.21.15.29:0/787105870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e40cf690 0x7fc7e41514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.756+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/787105870 >> 172.21.15.29:0/787105870 conn(0x7fc7e409d280 msgr2=0x7fc7e415d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:41.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.756+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/787105870 shutdown_connections 2024-04-25T00:35:41.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.756+0000 7fc7ebbd0640 1 -- 172.21.15.29:0/787105870 wait complete. 2024-04-25T00:35:41.776 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-04-25T00:35:41.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0df6025640 1 Processor -- start 2024-04-25T00:35:41.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0df6025640 1 -- start start 2024-04-25T00:35:41.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0df6025640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df015e170 0x7f0df0161d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0df6025640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0df00b01d0 0x7f0df006b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0df6025640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df015f740 con 0x7f0df015e550 2024-04-25T00:35:41.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0df6025640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0df01622b0 con 0x7f0df00b01d0 2024-04-25T00:35:41.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0df6025640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0df0162580 con 0x7f0df015e170 2024-04-25T00:35:41.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0deffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0df00b01d0 0x7f0df006b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0def7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df015e170 0x7f0df0161d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0def7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df015e170 0x7f0df0161d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:34902/0 (socket says 172.21.15.29:34902) 2024-04-25T00:35:41.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0def7fe640 1 -- 172.21.15.29:0/4158704109 learned_addr learned my addr 172.21.15.29:0/4158704109 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:41.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0def7fe640 1 -- 172.21.15.29:0/4158704109 >> v2:172.21.15.157:3300/0 conn(0x7f0df00b01d0 msgr2=0x7f0df006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:41.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0def7fe640 1 --2- 172.21.15.29:0/4158704109 >> v2:172.21.15.157:3300/0 conn(0x7f0df00b01d0 0x7f0df006b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0def7fe640 1 -- 172.21.15.29:0/4158704109 >> v1:172.21.15.157:6789/0 conn(0x7f0df015e550 legacy=0x7f0df015f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.900+0000 7f0def7fe640 1 -- 172.21.15.29:0/4158704109 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df0162860 con 0x7f0df015e170 2024-04-25T00:35:41.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0def7fe640 1 --2- 172.21.15.29:0/4158704109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df015e170 0x7f0df0161d70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009870 tx=0x7f0de001ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47b385bd9755f16c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:41.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0dee7fc640 1 -- 172.21.15.29:0/4158704109 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de00b5070 con 0x7f0df015e170 2024-04-25T00:35:41.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0dee7fc640 1 -- 172.21.15.29:0/4158704109 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de00a8a50 con 0x7f0df015e170 2024-04-25T00:35:41.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0dee7fc640 1 -- 172.21.15.29:0/4158704109 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de00a8e20 con 0x7f0df015e170 2024-04-25T00:35:41.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0df6025640 1 -- 172.21.15.29:0/4158704109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df015e170 msgr2=0x7f0df0161d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0df6025640 1 --2- 172.21.15.29:0/4158704109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df015e170 0x7f0df0161d70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009870 tx=0x7f0de001ed50 comp rx=0 tx=0).stop 2024-04-25T00:35:41.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0df6025640 1 -- 172.21.15.29:0/4158704109 shutdown_connections 2024-04-25T00:35:41.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0df6025640 1 --2- 172.21.15.29:0/4158704109 >> v2:172.21.15.157:3300/0 conn(0x7f0df00b01d0 0x7f0df006b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0df6025640 1 --2- 172.21.15.29:0/4158704109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df015e170 0x7f0df0161d70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0df6025640 1 -- 172.21.15.29:0/4158704109 >> 172.21.15.29:0/4158704109 conn(0x7f0df00d0c30 msgr2=0x7f0df00d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:41.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0df6025640 1 -- 172.21.15.29:0/4158704109 shutdown_connections 2024-04-25T00:35:41.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0df6025640 1 -- 172.21.15.29:0/4158704109 wait complete. 2024-04-25T00:35:41.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0df6025640 1 Processor -- start 2024-04-25T00:35:41.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.904+0000 7f0df6025640 1 -- start start 2024-04-25T00:35:41.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0df6025640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0df00b01d0 0x7f0df00c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0df6025640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df015e170 0x7f0df01f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0df6025640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df015e550 0x7f0df00b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0df6025640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0df006b750 con 0x7f0df00b01d0 2024-04-25T00:35:41.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0df6025640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0df015eb10 con 0x7f0df015e170 2024-04-25T00:35:41.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0df6025640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0df006b8d0 con 0x7f0df015e550 2024-04-25T00:35:41.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0def7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0df00b01d0 0x7f0df00c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0def7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0df00b01d0 0x7f0df00c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53408/0 (socket says 172.21.15.29:53408) 2024-04-25T00:35:41.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0def7fe640 1 -- 172.21.15.29:0/1992730856 learned_addr learned my addr 172.21.15.29:0/1992730856 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:41.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0deeffd640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df015e170 0x7f0df01f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0deffff640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df015e550 0x7f0df00b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0def7fe640 1 -- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df015e550 msgr2=0x7f0df00b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0def7fe640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df015e550 0x7f0df00b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0def7fe640 1 -- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df015e170 msgr2=0x7f0df01f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0def7fe640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df015e170 0x7f0df01f4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0def7fe640 1 -- 172.21.15.29:0/1992730856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df01f5470 con 0x7f0df00b01d0 2024-04-25T00:35:41.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0def7fe640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0df00b01d0 0x7f0df00c7f30 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0de00b3040 tx=0x7f0de001baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:41.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0dd7fff640 1 -- 172.21.15.29:0/1992730856 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de00c0070 con 0x7f0df00b01d0 2024-04-25T00:35:41.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0dd7fff640 1 -- 172.21.15.29:0/1992730856 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de00ae550 con 0x7f0df00b01d0 2024-04-25T00:35:41.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0df6025640 1 -- 172.21.15.29:0/1992730856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df01f59c0 con 0x7f0df00b01d0 2024-04-25T00:35:41.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0dd7fff640 1 -- 172.21.15.29:0/1992730856 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de00b5070 con 0x7f0df00b01d0 2024-04-25T00:35:41.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0df6025640 1 -- 172.21.15.29:0/1992730856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df01f5f80 con 0x7f0df00b01d0 2024-04-25T00:35:41.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0df6025640 1 -- 172.21.15.29:0/1992730856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df0069270 con 0x7f0df00b01d0 2024-04-25T00:35:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0dd7fff640 1 -- 172.21.15.29:0/1992730856 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f0de00c3070 con 0x7f0df00b01d0 2024-04-25T00:35:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0dd7fff640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0ddc067c20 0x7f0ddc06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0dd7fff640 1 -- 172.21.15.29:0/1992730856 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f0de01237d0 con 0x7f0df00b01d0 2024-04-25T00:35:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.908+0000 7f0deeffd640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0ddc067c20 0x7f0ddc06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.912+0000 7f0deeffd640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0ddc067c20 0x7f0ddc06a0e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0df00cdcf0 tx=0x7f0de4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:41.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:41.912+0000 7f0dd7fff640 1 -- 172.21.15.29:0/1992730856 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0de00188b0 con 0x7f0df00b01d0 2024-04-25T00:35:42.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:42.128+0000 7f0df6025640 1 -- 172.21.15.29:0/1992730856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7f0df00671f0 con 0x7f0df00b01d0 2024-04-25T00:35:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.116+0000 7f0dd7fff640 1 -- 172.21.15.29:0/1992730856 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v227) ==== 134+0+0 (secure 0 0 0) 0x7f0de00f3ab0 con 0x7f0df00b01d0 2024-04-25T00:35:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.120+0000 7f0df6025640 1 -- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0ddc067c20 msgr2=0x7f0ddc06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.120+0000 7f0df6025640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0ddc067c20 0x7f0ddc06a0e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0df00cdcf0 tx=0x7f0de4009040 comp rx=0 tx=0).stop 2024-04-25T00:35:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.120+0000 7f0df6025640 1 -- 172.21.15.29:0/1992730856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0df00b01d0 msgr2=0x7f0df00c7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.120+0000 7f0df6025640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0df00b01d0 0x7f0df00c7f30 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0de00b3040 tx=0x7f0de001baf0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.120+0000 7f0df6025640 1 -- 172.21.15.29:0/1992730856 shutdown_connections 2024-04-25T00:35:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.120+0000 7f0df6025640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0df015e550 0x7f0df00b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.120+0000 7f0df6025640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0ddc067c20 0x7f0ddc06a0e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.120+0000 7f0df6025640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0df015e170 0x7f0df01f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.120+0000 7f0df6025640 1 --2- 172.21.15.29:0/1992730856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0df00b01d0 0x7f0df00c7f30 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.120+0000 7f0df6025640 1 -- 172.21.15.29:0/1992730856 >> 172.21.15.29:0/1992730856 conn(0x7f0df00d0c30 msgr2=0x7f0df00ad8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:43.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.120+0000 7f0df6025640 1 -- 172.21.15.29:0/1992730856 shutdown_connections 2024-04-25T00:35:43.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.120+0000 7f0df6025640 1 -- 172.21.15.29:0/1992730856 wait complete. 2024-04-25T00:35:43.142 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:35:43.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.272+0000 7fc1d1747640 1 Processor -- start 2024-04-25T00:35:43.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.272+0000 7fc1d1747640 1 -- start start 2024-04-25T00:35:43.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.272+0000 7fc1d1747640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc1cc06e440 0x7fc1cc0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1d1747640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1cc0b41e0 con 0x7fc1cc0b3140 2024-04-25T00:35:43.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1d1747640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1cc06d960 con 0x7fc1cc0b3c30 2024-04-25T00:35:43.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1d1747640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc1cc0b5050 con 0x7fc1cc06e440 2024-04-25T00:35:43.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1caffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc1cc0b3140 0x7fc1cc0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42600/0 (socket says 172.21.15.29:42600) 2024-04-25T00:35:43.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1caffd640 1 -- 172.21.15.29:0/1082901197 learned_addr learned my addr 172.21.15.29:0/1082901197 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1cb7fe640 1 --2- 172.21.15.29:0/1082901197 >> v2:172.21.15.157:3300/0 conn(0x7fc1cc06e440 0x7fc1cc0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 618845724 0 0) 0x7fc1cc0b41e0 con 0x7fc1cc0b3140 2024-04-25T00:35:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1b4003530 con 0x7fc1cc0b3140 2024-04-25T00:35:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2376903958 0 0) 0x7fc1b4003530 con 0x7fc1cc0b3140 2024-04-25T00:35:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 >> v2:172.21.15.157:3300/0 conn(0x7fc1cc06e440 msgr2=0x7fc1cc0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 --2- 172.21.15.29:0/1082901197 >> v2:172.21.15.157:3300/0 conn(0x7fc1cc06e440 0x7fc1cc0b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 >> v1:172.21.15.157:6789/0 conn(0x7fc1cc0b3c30 legacy=0x7fc1cc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc06dc40 con 0x7fc1cc0b3140 2024-04-25T00:35:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fc1bc001680 con 0x7fc1cc0b3140 2024-04-25T00:35:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 >> v1:172.21.15.29:6789/0 conn(0x7fc1cc0b3140 legacy=0x7fc1cc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 --2- 172.21.15.29:0/1082901197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1b40039c0 0x7fc1b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 --2- 172.21.15.29:0/1082901197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1b4004810 0x7fc1b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 --2- 172.21.15.29:0/1082901197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1b40051d0 0x7fc1b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc0b5050 con 0x7fc1b40051d0 2024-04-25T00:35:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b400c070 con 0x7fc1b4004810 2024-04-25T00:35:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b400c270 con 0x7fc1b40039c0 2024-04-25T00:35:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1cb7fe640 1 --2- 172.21.15.29:0/1082901197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1b40051d0 0x7fc1b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1ca7fc640 1 --2- 172.21.15.29:0/1082901197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1b4004810 0x7fc1b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1caffd640 1 --2- 172.21.15.29:0/1082901197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1b40039c0 0x7fc1b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1cb7fe640 1 -- 172.21.15.29:0/1082901197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1b40039c0 msgr2=0x7fc1b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1cb7fe640 1 --2- 172.21.15.29:0/1082901197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1b40039c0 0x7fc1b4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1cb7fe640 1 -- 172.21.15.29:0/1082901197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1b4004810 msgr2=0x7fc1b4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1cb7fe640 1 --2- 172.21.15.29:0/1082901197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1b4004810 0x7fc1b4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1cb7fe640 1 -- 172.21.15.29:0/1082901197 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b4003530 con 0x7fc1b40051d0 2024-04-25T00:35:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1caffd640 1 --2- 172.21.15.29:0/1082901197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1b40039c0 0x7fc1b4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:43.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1cb7fe640 1 -- 172.21.15.29:0/1082901197 reap_dead start 2024-04-25T00:35:43.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1cb7fe640 1 --2- 172.21.15.29:0/1082901197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1b40051d0 0x7fc1b400b9d0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009fc0 tx=0x7fc1c401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9982c4f34a97f1e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:43.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4009c10 con 0x7fc1b40051d0 2024-04-25T00:35:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c40a5d10 con 0x7fc1b40051d0 2024-04-25T00:35:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.276+0000 7fc1c9ffb640 1 -- 172.21.15.29:0/1082901197 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4003be0 con 0x7fc1b40051d0 2024-04-25T00:35:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 -- 172.21.15.29:0/1082901197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1b40051d0 msgr2=0x7fc1b400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 --2- 172.21.15.29:0/1082901197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1b40051d0 0x7fc1b400b9d0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009fc0 tx=0x7fc1c401ccc0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 -- 172.21.15.29:0/1082901197 shutdown_connections 2024-04-25T00:35:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 --2- 172.21.15.29:0/1082901197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1b40051d0 0x7fc1b400b9d0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 -- 172.21.15.29:0/1082901197 >> 172.21.15.29:0/1082901197 conn(0x7fc1cc0b1870 msgr2=0x7fc1cc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 -- 172.21.15.29:0/1082901197 shutdown_connections 2024-04-25T00:35:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 -- 172.21.15.29:0/1082901197 wait complete. 2024-04-25T00:35:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 Processor -- start 2024-04-25T00:35:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 -- start start 2024-04-25T00:35:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1b40051d0 0x7fc1cc1448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1cc1557d0 0x7fc1cc13c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1cc13c710 0x7fc1cc156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc145a90 con 0x7fc1b40051d0 2024-04-25T00:35:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc145910 con 0x7fc1cc13c710 2024-04-25T00:35:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc1cc0b4b10 con 0x7fc1cc1557d0 2024-04-25T00:35:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1caffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1b40051d0 0x7fc1cc1448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1caffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1b40051d0 0x7fc1cc1448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53428/0 (socket says 172.21.15.29:53428) 2024-04-25T00:35:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1caffd640 1 -- 172.21.15.29:0/2381488128 learned_addr learned my addr 172.21.15.29:0/2381488128 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1caffd640 1 -- 172.21.15.29:0/2381488128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1cc1557d0 msgr2=0x7fc1cc13c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1caffd640 1 --2- 172.21.15.29:0/2381488128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1cc1557d0 0x7fc1cc13c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1caffd640 1 -- 172.21.15.29:0/2381488128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1cc13c710 msgr2=0x7fc1cc156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1caffd640 1 --2- 172.21.15.29:0/2381488128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1cc13c710 0x7fc1cc156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1caffd640 1 -- 172.21.15.29:0/2381488128 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc153440 con 0x7fc1b40051d0 2024-04-25T00:35:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1caffd640 1 --2- 172.21.15.29:0/2381488128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1b40051d0 0x7fc1cc1448f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00d770 tx=0x7fc1bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1b37fe640 1 -- 172.21.15.29:0/2381488128 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc019070 con 0x7fc1b40051d0 2024-04-25T00:35:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1b37fe640 1 -- 172.21.15.29:0/2381488128 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1bc0074e0 con 0x7fc1b40051d0 2024-04-25T00:35:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1b37fe640 1 -- 172.21.15.29:0/2381488128 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0049f0 con 0x7fc1b40051d0 2024-04-25T00:35:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 -- 172.21.15.29:0/2381488128 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1cc13cb90 con 0x7fc1b40051d0 2024-04-25T00:35:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.280+0000 7fc1d1747640 1 -- 172.21.15.29:0/2381488128 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1cc1539c0 con 0x7fc1b40051d0 2024-04-25T00:35:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.284+0000 7fc1d1747640 1 -- 172.21.15.29:0/2381488128 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1cc069270 con 0x7fc1b40051d0 2024-04-25T00:35:43.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.284+0000 7fc1b37fe640 1 -- 172.21.15.29:0/2381488128 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fc1bc01c070 con 0x7fc1b40051d0 2024-04-25T00:35:43.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.284+0000 7fc1b37fe640 1 --2- 172.21.15.29:0/2381488128 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc1b8067c50 0x7fc1b806a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.284+0000 7fc1b37fe640 1 -- 172.21.15.29:0/2381488128 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fc1bc085a60 con 0x7fc1b40051d0 2024-04-25T00:35:43.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.284+0000 7fc1ca7fc640 1 --2- 172.21.15.29:0/2381488128 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc1b8067c50 0x7fc1b806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:43.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.284+0000 7fc1ca7fc640 1 --2- 172.21.15.29:0/2381488128 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc1b8067c50 0x7fc1b806a110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0cffa0 tx=0x7fc1c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:43.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.288+0000 7fc1b37fe640 1 -- 172.21.15.29:0/2381488128 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc1bc056400 con 0x7fc1b40051d0 2024-04-25T00:35:43.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.448+0000 7fc1d1747640 1 -- 172.21.15.29:0/2381488128 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc1cc0d0980 con 0x7fc1b40051d0 2024-04-25T00:35:43.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.452+0000 7fc1b37fe640 1 -- 172.21.15.29:0/2381488128 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fc1bc055d00 con 0x7fc1b40051d0 2024-04-25T00:35:43.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.456+0000 7fc1d1747640 1 -- 172.21.15.29:0/2381488128 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc1b8067c50 msgr2=0x7fc1b806a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:43.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.456+0000 7fc1d1747640 1 --2- 172.21.15.29:0/2381488128 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc1b8067c50 0x7fc1b806a110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0cffa0 tx=0x7fc1c0009040 comp rx=0 tx=0).stop 2024-04-25T00:35:43.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.456+0000 7fc1d1747640 1 -- 172.21.15.29:0/2381488128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1b40051d0 msgr2=0x7fc1cc1448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:43.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.456+0000 7fc1d1747640 1 --2- 172.21.15.29:0/2381488128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1b40051d0 0x7fc1cc1448f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00d770 tx=0x7fc1bc00dc30 comp rx=0 tx=0).stop 2024-04-25T00:35:43.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.456+0000 7fc1d1747640 1 -- 172.21.15.29:0/2381488128 shutdown_connections 2024-04-25T00:35:43.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.456+0000 7fc1d1747640 1 --2- 172.21.15.29:0/2381488128 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc1b8067c50 0x7fc1b806a110 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.456+0000 7fc1d1747640 1 --2- 172.21.15.29:0/2381488128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1cc13c710 0x7fc1cc156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.456+0000 7fc1d1747640 1 --2- 172.21.15.29:0/2381488128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1cc1557d0 0x7fc1cc13c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.456+0000 7fc1d1747640 1 --2- 172.21.15.29:0/2381488128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1b40051d0 0x7fc1cc1448f0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.456+0000 7fc1d1747640 1 -- 172.21.15.29:0/2381488128 >> 172.21.15.29:0/2381488128 conn(0x7fc1cc0b1870 msgr2=0x7fc1cc0ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:43.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.456+0000 7fc1d1747640 1 -- 172.21.15.29:0/2381488128 shutdown_connections 2024-04-25T00:35:43.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.456+0000 7fc1d1747640 1 -- 172.21.15.29:0/2381488128 wait complete. 2024-04-25T00:35:43.476 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:35:43.476 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:35:43.526 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:35:43.571 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:35:43.571 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:35:43.571 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:35:43.578 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:35:43.578 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:35:43.578 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:35:43.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:35:43.621 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:35:43.621 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:35:43.627 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:35:43.627 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:35:43.627 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:35:43.671 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:35:43.671 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:35:43.671 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:35:43.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.800+0000 7f29c5fe1640 1 Processor -- start 2024-04-25T00:35:43.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29c5fe1640 1 -- start start 2024-04-25T00:35:43.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29c5fe1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c00d1c10 0x7f29c00d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29c5fe1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29c015c290 0x7f29c00cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29c5fe1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29c00af770 con 0x7f29c00aea40 2024-04-25T00:35:43.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29c5fe1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f29c00b0380 con 0x7f29c015c290 2024-04-25T00:35:43.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29c5fe1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f29c00afa50 con 0x7f29c00d1c10 2024-04-25T00:35:43.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29bf7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c00d1c10 0x7f29c00d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:43.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29beffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29c015c290 0x7f29c00cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:43.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29bf7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c00d1c10 0x7f29c00d1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:34926/0 (socket says 172.21.15.29:34926) 2024-04-25T00:35:43.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29bf7fe640 1 -- 172.21.15.29:0/708370003 learned_addr learned my addr 172.21.15.29:0/708370003 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:43.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29bf7fe640 1 -- 172.21.15.29:0/708370003 >> v2:172.21.15.157:3300/0 conn(0x7f29c015c290 msgr2=0x7f29c00cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:43.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29bf7fe640 1 --2- 172.21.15.29:0/708370003 >> v2:172.21.15.157:3300/0 conn(0x7f29c015c290 0x7f29c00cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29bf7fe640 1 -- 172.21.15.29:0/708370003 >> v1:172.21.15.157:6789/0 conn(0x7f29c00aea40 legacy=0x7f29c00aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:43.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.804+0000 7f29bf7fe640 1 -- 172.21.15.29:0/708370003 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c00afd30 con 0x7f29c00d1c10 2024-04-25T00:35:43.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.808+0000 7f29bf7fe640 1 --2- 172.21.15.29:0/708370003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c00d1c10 0x7f29c00d1ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009870 tx=0x7f29b001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a66bf53b59db434 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:43.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.808+0000 7f29be7fc640 1 -- 172.21.15.29:0/708370003 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00a9610 con 0x7f29c00d1c10 2024-04-25T00:35:43.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.808+0000 7f29be7fc640 1 -- 172.21.15.29:0/708370003 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b00a97b0 con 0x7f29c00d1c10 2024-04-25T00:35:43.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.808+0000 7f29be7fc640 1 -- 172.21.15.29:0/708370003 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00a9b80 con 0x7f29c00d1c10 2024-04-25T00:35:43.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.808+0000 7f29c5fe1640 1 -- 172.21.15.29:0/708370003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c00d1c10 msgr2=0x7f29c00d1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:43.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.808+0000 7f29c5fe1640 1 --2- 172.21.15.29:0/708370003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c00d1c10 0x7f29c00d1ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009870 tx=0x7f29b001c530 comp rx=0 tx=0).stop 2024-04-25T00:35:43.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.808+0000 7f29c5fe1640 1 -- 172.21.15.29:0/708370003 shutdown_connections 2024-04-25T00:35:43.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.808+0000 7f29c5fe1640 1 --2- 172.21.15.29:0/708370003 >> v2:172.21.15.157:3300/0 conn(0x7f29c015c290 0x7f29c00cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.808+0000 7f29c5fe1640 1 --2- 172.21.15.29:0/708370003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c00d1c10 0x7f29c00d1ff0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.808+0000 7f29c5fe1640 1 -- 172.21.15.29:0/708370003 >> 172.21.15.29:0/708370003 conn(0x7f29c009aa30 msgr2=0x7f29c009ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:43.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.808+0000 7f29c5fe1640 1 -- 172.21.15.29:0/708370003 shutdown_connections 2024-04-25T00:35:43.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.808+0000 7f29c5fe1640 1 -- 172.21.15.29:0/708370003 wait complete. 2024-04-25T00:35:43.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29c5fe1640 1 Processor -- start 2024-04-25T00:35:43.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29c5fe1640 1 -- start start 2024-04-25T00:35:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29c5fe1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29c00aea40 0x7f29c01582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29c5fe1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c00d1c10 0x7f29c00c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29c5fe1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c015c290 0x7f29c00ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29c5fe1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f29c00b11a0 con 0x7f29c00aea40 2024-04-25T00:35:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29c5fe1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f29c00b1020 con 0x7f29c015c290 2024-04-25T00:35:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29c5fe1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f29c00b1320 con 0x7f29c00d1c10 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29bf7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29c00aea40 0x7f29c01582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29bf7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29c00aea40 0x7f29c01582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53430/0 (socket says 172.21.15.29:53430) 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29bf7fe640 1 -- 172.21.15.29:0/1801526205 learned_addr learned my addr 172.21.15.29:0/1801526205 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29beffd640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c00d1c10 0x7f29c00c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29bffff640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c015c290 0x7f29c00ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29bf7fe640 1 -- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c00d1c10 msgr2=0x7f29c00c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29bf7fe640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c00d1c10 0x7f29c00c7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29bf7fe640 1 -- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c015c290 msgr2=0x7f29c00ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29bf7fe640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c015c290 0x7f29c00ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29bf7fe640 1 -- 172.21.15.29:0/1801526205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c00c6970 con 0x7f29c00aea40 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29bf7fe640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29c00aea40 0x7f29c01582c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f29b00ab040 tx=0x7f29b001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29a7fff640 1 -- 172.21.15.29:0/1801526205 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00a9610 con 0x7f29c00aea40 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29a7fff640 1 -- 172.21.15.29:0/1801526205 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b00bc650 con 0x7f29c00aea40 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29c5fe1640 1 -- 172.21.15.29:0/1801526205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c00c6c50 con 0x7f29c00aea40 2024-04-25T00:35:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29a7fff640 1 -- 172.21.15.29:0/1801526205 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00bca20 con 0x7f29c00aea40 2024-04-25T00:35:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29c5fe1640 1 -- 172.21.15.29:0/1801526205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c00c7c70 con 0x7f29c00aea40 2024-04-25T00:35:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.812+0000 7f29c5fe1640 1 -- 172.21.15.29:0/1801526205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c0069270 con 0x7f29c00aea40 2024-04-25T00:35:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.816+0000 7f29a7fff640 1 -- 172.21.15.29:0/1801526205 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f29b00bcbc0 con 0x7f29c00aea40 2024-04-25T00:35:43.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.816+0000 7f29a7fff640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f29ac067c20 0x7f29ac06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:43.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.816+0000 7f29a7fff640 1 -- 172.21.15.29:0/1801526205 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f29b0120980 con 0x7f29c00aea40 2024-04-25T00:35:43.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.816+0000 7f29beffd640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f29ac067c20 0x7f29ac06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:43.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.816+0000 7f29beffd640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f29ac067c20 0x7f29ac06a0e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f29c0165f60 tx=0x7f29b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:43.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:43.820+0000 7f29a7fff640 1 -- 172.21.15.29:0/1801526205 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f29b00ad020 con 0x7f29c00aea40 2024-04-25T00:35:44.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.016+0000 7f29c5fe1640 1 -- 172.21.15.29:0/1801526205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29c00c7f60 con 0x7f29c00aea40 2024-04-25T00:35:44.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.020+0000 7f29a7fff640 1 -- 172.21.15.29:0/1801526205 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+3866 (secure 0 0 0) 0x7f29b0016150 con 0x7f29c00aea40 2024-04-25T00:35:44.021 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:44.021 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/56623075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":56623075},{"type":"v1","addr":"172.21.15.157:6836","nonce":56623075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/748098725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":748098725},{"type":"v1","addr":"172.21.15.157:6837","nonce":748098725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2264413556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2264413556},{"type":"v1","addr":"172.21.15.29:6833","nonce":2264413556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2472399335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":2472399335},{"type":"v1","addr":"172.21.15.29:6835","nonce":2472399335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[]} 2024-04-25T00:35:44.022 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 37 2024-04-25T00:35:44.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.020+0000 7f29c5fe1640 1 -- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f29ac067c20 msgr2=0x7f29ac06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:44.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.020+0000 7f29c5fe1640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f29ac067c20 0x7f29ac06a0e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f29c0165f60 tx=0x7f29b4008040 comp rx=0 tx=0).stop 2024-04-25T00:35:44.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.020+0000 7f29c5fe1640 1 -- 172.21.15.29:0/1801526205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29c00aea40 msgr2=0x7f29c01582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:44.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.020+0000 7f29c5fe1640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29c00aea40 0x7f29c01582c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f29b00ab040 tx=0x7f29b001ad70 comp rx=0 tx=0).stop 2024-04-25T00:35:44.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.020+0000 7f29c5fe1640 1 -- 172.21.15.29:0/1801526205 shutdown_connections 2024-04-25T00:35:44.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.020+0000 7f29c5fe1640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f29ac067c20 0x7f29ac06a0e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:44.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.020+0000 7f29c5fe1640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c015c290 0x7f29c00ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:44.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.020+0000 7f29c5fe1640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c00d1c10 0x7f29c00c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:44.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.020+0000 7f29c5fe1640 1 --2- 172.21.15.29:0/1801526205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29c00aea40 0x7f29c01582c0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:44.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.020+0000 7f29c5fe1640 1 -- 172.21.15.29:0/1801526205 >> 172.21.15.29:0/1801526205 conn(0x7f29c009aa30 msgr2=0x7f29c00d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:44.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.020+0000 7f29c5fe1640 1 -- 172.21.15.29:0/1801526205 shutdown_connections 2024-04-25T00:35:44.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.020+0000 7f29c5fe1640 1 -- 172.21.15.29:0/1801526205 wait complete. 2024-04-25T00:35:44.042 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:35:44.042 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:35:44.042 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:35:44.043 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:35:44.043 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:35:44.043 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:35:44.043 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:35:44.043 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:35:44.043 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:35:44.044 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:35:44.044 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:35:44.044 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:35:44.044 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:35:44.044 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:35:44.044 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:35:44.044 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:35:44.055 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:35:44.055 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:35:44.056 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:35:44.056 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:35:44.059 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:35:44.059 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:35:44.063 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:35:44.064 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:35:44.101 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:35:44.102 DEBUG:teuthology.parallel:result is None 2024-04-25T00:35:44.103 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:35:44.103 DEBUG:teuthology.parallel:result is None 2024-04-25T00:35:44.108 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:35:44.108 DEBUG:teuthology.parallel:result is None 2024-04-25T00:35:44.109 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:35:44.110 DEBUG:teuthology.parallel:result is None 2024-04-25T00:35:44.110 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:35:44.165 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:35:44.173 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:35:44.177 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-25T00:35:44.178 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-25T00:35:44.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f430d30e640 1 Processor -- start 2024-04-25T00:35:44.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f430d30e640 1 -- start start 2024-04-25T00:35:44.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f430d30e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f430815e200 0x7f43080d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:44.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f430d30e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43080cf690 0x7f43080d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:44.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f430d30e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080d2a90 0x7f43080d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:44.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f430d30e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f43080d2ee0 con 0x7f430815e200 2024-04-25T00:35:44.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f430d30e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f43080b38d0 con 0x7f43080cf690 2024-04-25T00:35:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f430d30e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f43080cfc10 con 0x7f43080d2a90 2024-04-25T00:35:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f4306ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f430815e200 0x7f43080d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f4306ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f430815e200 0x7f43080d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53488/0 (socket says 172.21.15.29:53488) 2024-04-25T00:35:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f4306ffd640 1 -- 172.21.15.29:0/4228080583 learned_addr learned my addr 172.21.15.29:0/4228080583 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f43067fc640 1 --2- 172.21.15.29:0/4228080583 >> v2:172.21.15.157:3300/0 conn(0x7f43080cf690 0x7f43080d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f43077fe640 1 --2- 172.21.15.29:0/4228080583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080d2a90 0x7f43080d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f4306ffd640 1 -- 172.21.15.29:0/4228080583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080d2a90 msgr2=0x7f43080d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f4306ffd640 1 --2- 172.21.15.29:0/4228080583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080d2a90 0x7f43080d05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f4306ffd640 1 -- 172.21.15.29:0/4228080583 >> v2:172.21.15.157:3300/0 conn(0x7f43080cf690 msgr2=0x7f43080d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f4306ffd640 1 --2- 172.21.15.29:0/4228080583 >> v2:172.21.15.157:3300/0 conn(0x7f43080cf690 0x7f43080d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f4306ffd640 1 -- 172.21.15.29:0/4228080583 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43080cfef0 con 0x7f430815e200 2024-04-25T00:35:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f43067fc640 1 --2- 172.21.15.29:0/4228080583 >> v2:172.21.15.157:3300/0 conn(0x7f43080cf690 0x7f43080d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:44.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.228+0000 7f4306ffd640 1 --2- 172.21.15.29:0/4228080583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f430815e200 0x7f43080d47f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f43080d24a0 tx=0x7f42f001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb91479d8fc1002 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f4305ffb640 1 -- 172.21.15.29:0/4228080583 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f00b0070 con 0x7f430815e200 2024-04-25T00:35:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f4305ffb640 1 -- 172.21.15.29:0/4228080583 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f00ab600 con 0x7f430815e200 2024-04-25T00:35:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f4305ffb640 1 -- 172.21.15.29:0/4228080583 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f00ab9d0 con 0x7f430815e200 2024-04-25T00:35:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 -- 172.21.15.29:0/4228080583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f430815e200 msgr2=0x7f43080d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 --2- 172.21.15.29:0/4228080583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f430815e200 0x7f43080d47f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f43080d24a0 tx=0x7f42f001c230 comp rx=0 tx=0).stop 2024-04-25T00:35:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 -- 172.21.15.29:0/4228080583 shutdown_connections 2024-04-25T00:35:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 --2- 172.21.15.29:0/4228080583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080d2a90 0x7f43080d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:44.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 --2- 172.21.15.29:0/4228080583 >> v2:172.21.15.157:3300/0 conn(0x7f43080cf690 0x7f43080d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:44.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 --2- 172.21.15.29:0/4228080583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f430815e200 0x7f43080d47f0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:44.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 -- 172.21.15.29:0/4228080583 >> 172.21.15.29:0/4228080583 conn(0x7f430809d280 msgr2=0x7f43080d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:44.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 -- 172.21.15.29:0/4228080583 shutdown_connections 2024-04-25T00:35:44.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 -- 172.21.15.29:0/4228080583 wait complete. 2024-04-25T00:35:44.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 Processor -- start 2024-04-25T00:35:44.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 -- start start 2024-04-25T00:35:44.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080cf690 0x7f4308156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080d2a90 0x7f43080ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f430815e200 0x7f430806c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f430815e760 con 0x7f43080d2a90 2024-04-25T00:35:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f430815e5e0 con 0x7f430815e200 2024-04-25T00:35:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f43080b4320 con 0x7f43080cf690 2024-04-25T00:35:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f43067fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080d2a90 0x7f43080ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f43067fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080d2a90 0x7f43080ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53490/0 (socket says 172.21.15.29:53490) 2024-04-25T00:35:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f43067fc640 1 -- 172.21.15.29:0/1445403141 learned_addr learned my addr 172.21.15.29:0/1445403141 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f4306ffd640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080cf690 0x7f4308156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f43077fe640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f430815e200 0x7f430806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f43067fc640 1 -- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080cf690 msgr2=0x7f4308156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f43067fc640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080cf690 0x7f4308156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f43067fc640 1 -- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f430815e200 msgr2=0x7f430806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f43067fc640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f430815e200 0x7f430806c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f43067fc640 1 -- 172.21.15.29:0/1445403141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4308156960 con 0x7f43080d2a90 2024-04-25T00:35:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f4306ffd640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080cf690 0x7f4308156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f43077fe640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f430815e200 0x7f430806c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f43067fc640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080d2a90 0x7f43080ca220 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f42f800e810 tx=0x7f42f800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f42ff7fe640 1 -- 172.21.15.29:0/1445403141 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f800f040 con 0x7f43080d2a90 2024-04-25T00:35:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f42ff7fe640 1 -- 172.21.15.29:0/1445403141 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f8004030 con 0x7f43080d2a90 2024-04-25T00:35:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 -- 172.21.15.29:0/1445403141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430815f6d0 con 0x7f43080d2a90 2024-04-25T00:35:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f42ff7fe640 1 -- 172.21.15.29:0/1445403141 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8004400 con 0x7f43080d2a90 2024-04-25T00:35:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.232+0000 7f430d30e640 1 -- 172.21.15.29:0/1445403141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f430815fc90 con 0x7f43080d2a90 2024-04-25T00:35:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.236+0000 7f430d30e640 1 -- 172.21.15.29:0/1445403141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4308069270 con 0x7f43080d2a90 2024-04-25T00:35:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.236+0000 7f42ff7fe640 1 -- 172.21.15.29:0/1445403141 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f42f800c6d0 con 0x7f43080d2a90 2024-04-25T00:35:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.236+0000 7f42ff7fe640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42e8067c20 0x7f42e806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.236+0000 7f42ff7fe640 1 -- 172.21.15.29:0/1445403141 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f42f8085c10 con 0x7f43080d2a90 2024-04-25T00:35:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.236+0000 7f4306ffd640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42e8067c20 0x7f42e806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.236+0000 7f4306ffd640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42e8067c20 0x7f42e806a0e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f42f00ad000 tx=0x7f42f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:44.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.240+0000 7f42ff7fe640 1 -- 172.21.15.29:0/1445403141 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f42f8017020 con 0x7f43080d2a90 2024-04-25T00:35:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:44.416+0000 7f430d30e640 1 -- 172.21.15.29:0/1445403141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f43080cedf0 con 0x7f43080d2a90 2024-04-25T00:35:45.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.476+0000 7f42ff7fe640 1 -- 172.21.15.29:0/1445403141 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v66) ==== 97+0+0 (secure 0 0 0) 0x7f42f8056580 con 0x7f43080d2a90 2024-04-25T00:35:45.480 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-04-25T00:35:45.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.480+0000 7f430d30e640 1 -- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42e8067c20 msgr2=0x7f42e806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.480+0000 7f430d30e640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42e8067c20 0x7f42e806a0e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f42f00ad000 tx=0x7f42f0008040 comp rx=0 tx=0).stop 2024-04-25T00:35:45.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.480+0000 7f430d30e640 1 -- 172.21.15.29:0/1445403141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080d2a90 msgr2=0x7f43080ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.480+0000 7f430d30e640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080d2a90 0x7f43080ca220 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f42f800e810 tx=0x7f42f800ecd0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.480+0000 7f430d30e640 1 -- 172.21.15.29:0/1445403141 shutdown_connections 2024-04-25T00:35:45.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.480+0000 7f430d30e640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f42e8067c20 0x7f42e806a0e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.480+0000 7f430d30e640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f430815e200 0x7f430806c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.480+0000 7f430d30e640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080d2a90 0x7f43080ca220 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.480+0000 7f430d30e640 1 --2- 172.21.15.29:0/1445403141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080cf690 0x7f4308156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.480+0000 7f430d30e640 1 -- 172.21.15.29:0/1445403141 >> 172.21.15.29:0/1445403141 conn(0x7f430809d280 msgr2=0x7f4308157430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:45.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.480+0000 7f430d30e640 1 -- 172.21.15.29:0/1445403141 shutdown_connections 2024-04-25T00:35:45.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.484+0000 7f430d30e640 1 -- 172.21.15.29:0/1445403141 wait complete. 2024-04-25T00:35:45.509 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:35:45.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.628+0000 7fcf82dfa640 1 Processor -- start 2024-04-25T00:35:45.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf82dfa640 1 -- start start 2024-04-25T00:35:45.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf82dfa640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcf7c0b3c30 0x7fcf7c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf82dfa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c0b4b10 0x7fcf7c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf82dfa640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf7c0b41e0 con 0x7fcf7c0b3140 2024-04-25T00:35:45.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf82dfa640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fcf7c06d960 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf82dfa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7c06dc40 con 0x7fcf7c0b4b10 2024-04-25T00:35:45.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf80b96640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcf7c0b3140 0x7fcf7c0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42626/0 (socket says 172.21.15.29:42626) 2024-04-25T00:35:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf80b96640 1 -- 172.21.15.29:0/2070515263 learned_addr learned my addr 172.21.15.29:0/2070515263 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf7bfff640 1 --2- 172.21.15.29:0/2070515263 >> v2:172.21.15.29:3300/0 conn(0x7fcf7c0b3c30 0x7fcf7c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf7bfff640 1 -- 172.21.15.29:0/2070515263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c0b4b10 msgr2=0x7fcf7c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf81397640 1 --2- 172.21.15.29:0/2070515263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c0b4b10 0x7fcf7c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf7b7fe640 1 -- 172.21.15.29:0/2070515263 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1651299243 0 0) 0x7fcf7c0b41e0 con 0x7fcf7c0b3140 2024-04-25T00:35:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf7bfff640 1 --2- 172.21.15.29:0/2070515263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c0b4b10 0x7fcf7c0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf7bfff640 1 -- 172.21.15.29:0/2070515263 >> v1:172.21.15.29:6789/0 conn(0x7fcf7c0b3140 legacy=0x7fcf7c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf7bfff640 1 -- 172.21.15.29:0/2070515263 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c06df20 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.632+0000 7fcf81397640 1 --2- 172.21.15.29:0/2070515263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c0b4b10 0x7fcf7c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:45.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf7bfff640 1 --2- 172.21.15.29:0/2070515263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf7c0b3c30 0x7fcf7c06e440 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009dc0 tx=0x7fcf7001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03ac06d81766a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:45.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf7b7fe640 1 -- 172.21.15.29:0/2070515263 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf700b2070 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf7b7fe640 1 -- 172.21.15.29:0/2070515263 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf700a5810 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf7b7fe640 1 -- 172.21.15.29:0/2070515263 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf700a5be0 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 -- 172.21.15.29:0/2070515263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf7c0b3c30 msgr2=0x7fcf7c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 --2- 172.21.15.29:0/2070515263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf7c0b3c30 0x7fcf7c06e440 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009dc0 tx=0x7fcf7001c950 comp rx=0 tx=0).stop 2024-04-25T00:35:45.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 -- 172.21.15.29:0/2070515263 shutdown_connections 2024-04-25T00:35:45.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 --2- 172.21.15.29:0/2070515263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c0b4b10 0x7fcf7c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 --2- 172.21.15.29:0/2070515263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf7c0b3c30 0x7fcf7c06e440 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 -- 172.21.15.29:0/2070515263 >> 172.21.15.29:0/2070515263 conn(0x7fcf7c0b1870 msgr2=0x7fcf7c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:45.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 -- 172.21.15.29:0/2070515263 shutdown_connections 2024-04-25T00:35:45.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 -- 172.21.15.29:0/2070515263 wait complete. 2024-04-25T00:35:45.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 Processor -- start 2024-04-25T00:35:45.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 -- start start 2024-04-25T00:35:45.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c0b3140 0x7fcf7c13de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf7c0b3c30 0x7fcf7c154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c0b4b10 0x7fcf7c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c168d80 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c168c00 con 0x7fcf7c0b4b10 2024-04-25T00:35:45.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf82dfa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7c168f00 con 0x7fcf7c0b3140 2024-04-25T00:35:45.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf7bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf7c0b3c30 0x7fcf7c154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf7bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf7c0b3c30 0x7fcf7c154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53518/0 (socket says 172.21.15.29:53518) 2024-04-25T00:35:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf7bfff640 1 -- 172.21.15.29:0/4026997903 learned_addr learned my addr 172.21.15.29:0/4026997903 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf80b96640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c0b3140 0x7fcf7c13de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf81397640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c0b4b10 0x7fcf7c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf7bfff640 1 -- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c0b3140 msgr2=0x7fcf7c13de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf7bfff640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c0b3140 0x7fcf7c13de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf7bfff640 1 -- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c0b4b10 msgr2=0x7fcf7c13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf7bfff640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c0b4b10 0x7fcf7c13b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf7bfff640 1 -- 172.21.15.29:0/4026997903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c15e890 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf81397640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c0b4b10 0x7fcf7c13b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf7bfff640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf7c0b3c30 0x7fcf7c154470 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fcf7001ce30 tx=0x7fcf700196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.636+0000 7fcf797fa640 1 -- 172.21.15.29:0/4026997903 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf700bd070 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.640+0000 7fcf797fa640 1 -- 172.21.15.29:0/4026997903 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf700b3040 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.640+0000 7fcf82dfa640 1 -- 172.21.15.29:0/4026997903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7c15d7f0 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.640+0000 7fcf797fa640 1 -- 172.21.15.29:0/4026997903 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf700b2050 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.640+0000 7fcf82dfa640 1 -- 172.21.15.29:0/4026997903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7c15ddb0 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.640+0000 7fcf82dfa640 1 -- 172.21.15.29:0/4026997903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7c069270 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.640+0000 7fcf797fa640 1 -- 172.21.15.29:0/4026997903 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fcf700a56d0 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.640+0000 7fcf797fa640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcf68067c50 0x7fcf6806a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:45.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.640+0000 7fcf797fa640 1 -- 172.21.15.29:0/4026997903 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7fcf701201f0 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.640+0000 7fcf80b96640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcf68067c50 0x7fcf6806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:45.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.640+0000 7fcf80b96640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcf68067c50 0x7fcf6806a110 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000f70 tx=0x7fcf6c007c10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:45.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.644+0000 7fcf797fa640 1 -- 172.21.15.29:0/4026997903 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcf70016540 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.860+0000 7fcf82dfa640 1 -- 172.21.15.29:0/4026997903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcf7c1447b0 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.860+0000 7fcf797fa640 1 -- 172.21.15.29:0/4026997903 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5617 (secure 0 0 0) 0x7fcf700f0d00 con 0x7fcf7c0b3c30 2024-04-25T00:35:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:45.864 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:35:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-04-25T00:35:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-04-25T00:35:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmyzgKAxAAU+nrUJgQhDX6xlARNFI/ng==", 2024-04-25T00:35:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:45.864 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:35:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:35:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-04-25T00:35:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmtyzPERAAxNUDANw1H/ZzOo+dhgzxlg==", 2024-04-25T00:35:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:35:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:35:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.865 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmYUl8CxAAj6pV1+s4a0rpWLq2EgZ+8w==", 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-04-25T00:35:45.866 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmuAhKGhAAOLIcWUodW5TqtVgHWZl12A==", 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAzpClmudk6MhAA6YFyd57jwtB2pdOaad7bxQ==", 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:45.867 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA2pClm76q0NRAAxWhEP2ZGoFnZcuiQyGjglw==", 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-04-25T00:35:45.868 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA6pClmcgULABAA2F5WBR9Df0v4Sn875AzF7g==", 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA9pClm9QuRBhAALFLruv4JU/uNZYMX0yNuXw==", 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBBpClmNj4VDBAAxaxMvyaCvjynPY3f81udnw==", 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBEpClmzBybEBAArry1lM4RIsM/txKfgBqDpw==", 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.870 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBHpClm5HfjFhAAbFwJCpbhegLWDu0mbS8FoA==", 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBKpClme/7oHBAAOR2iQ6CweexR7sC1PgesZw==", 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A==", 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:45.872 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm8W/EJxAA0HZW24Hhcwf+soSZn6jHJg==", 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmGuQJJRAAhVd4D0jc6Whgyf+B6xgFMw==", 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmn+ciLRAAg4tQuZ13WP0RnWJaJq5fUQ==", 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.874 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmPVihJBAA7PQge/teyRZKSkBnufZSZw==", 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.alice", 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQpSlm+bmwCxAADTdjANQjpTtBIQzT9gU5Rw==", 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol62000", 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw pool=cephfs_data" 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClm6WrdOhAAsY9FLCxG7PGpq2NCY4hGhQ==", 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.876 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:35:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:35:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:35:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-04-25T00:35:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmeR4UOBAANs9SRolOm6E8+/4ciupk5w==", 2024-04-25T00:35:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:35:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:35:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:35:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:35:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.877 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:45.878 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:35:45.878 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:35:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.864+0000 7fcf82dfa640 1 -- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcf68067c50 msgr2=0x7fcf6806a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.864+0000 7fcf82dfa640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcf68067c50 0x7fcf6806a110 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000f70 tx=0x7fcf6c007c10 comp rx=0 tx=0).stop 2024-04-25T00:35:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.864+0000 7fcf82dfa640 1 -- 172.21.15.29:0/4026997903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf7c0b3c30 msgr2=0x7fcf7c154470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.864+0000 7fcf82dfa640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf7c0b3c30 0x7fcf7c154470 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fcf7001ce30 tx=0x7fcf700196f0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.864+0000 7fcf82dfa640 1 -- 172.21.15.29:0/4026997903 shutdown_connections 2024-04-25T00:35:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.864+0000 7fcf82dfa640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcf68067c50 0x7fcf6806a110 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.864+0000 7fcf82dfa640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7c0b4b10 0x7fcf7c13b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.864+0000 7fcf82dfa640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf7c0b3c30 0x7fcf7c154470 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.864+0000 7fcf82dfa640 1 --2- 172.21.15.29:0/4026997903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf7c0b3140 0x7fcf7c13de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.864+0000 7fcf82dfa640 1 -- 172.21.15.29:0/4026997903 >> 172.21.15.29:0/4026997903 conn(0x7fcf7c0b1870 msgr2=0x7fcf7c0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.864+0000 7fcf82dfa640 1 -- 172.21.15.29:0/4026997903 shutdown_connections 2024-04-25T00:35:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:45.864+0000 7fcf82dfa640 1 -- 172.21.15.29:0/4026997903 wait complete. 2024-04-25T00:35:45.882 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2024-04-25T00:35:46.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.004+0000 7fe774e56640 1 Processor -- start 2024-04-25T00:35:46.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.004+0000 7fe774e56640 1 -- start start 2024-04-25T00:35:46.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.004+0000 7fe774e56640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7700b4780 0x7fe77015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.004+0000 7fe774e56640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe7700cc240 0x7fe77015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.004+0000 7fe774e56640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7700b3720 con 0x7fe77015e8d0 2024-04-25T00:35:46.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.004+0000 7fe774e56640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe7700b42f0 con 0x7fe7700cc240 2024-04-25T00:35:46.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.004+0000 7fe774e56640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe7700b39c0 con 0x7fe7700b4780 2024-04-25T00:35:46.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.004+0000 7fe76e59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe77015e8d0 0x7fe77015ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42640/0 (socket says 172.21.15.29:42640) 2024-04-25T00:35:46.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.004+0000 7fe76e59c640 1 -- 172.21.15.29:0/1003286476 learned_addr learned my addr 172.21.15.29:0/1003286476 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:46.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.004+0000 7fe76ed9d640 1 --2- 172.21.15.29:0/1003286476 >> v2:172.21.15.157:3300/0 conn(0x7fe7700cc240 0x7fe77015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.004+0000 7fe765d9b640 1 --2- 172.21.15.29:0/1003286476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7700b4780 0x7fe77015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.004+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3267359385 0 0) 0x7fe7700b3720 con 0x7fe77015e8d0 2024-04-25T00:35:46.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.004+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe75c005740 con 0x7fe77015e8d0 2024-04-25T00:35:46.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4190359436 0 0) 0x7fe75c005740 con 0x7fe77015e8d0 2024-04-25T00:35:46.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7700b4780 msgr2=0x7fe77015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:46.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 --2- 172.21.15.29:0/1003286476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7700b4780 0x7fe77015f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 >> v2:172.21.15.157:3300/0 conn(0x7fe7700cc240 msgr2=0x7fe77015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 --2- 172.21.15.29:0/1003286476 >> v2:172.21.15.157:3300/0 conn(0x7fe7700cc240 0x7fe77015c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7700b3ca0 con 0x7fe77015e8d0 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fe760001680 con 0x7fe77015e8d0 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 >> v1:172.21.15.29:6789/0 conn(0x7fe77015e8d0 legacy=0x7fe77015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 --2- 172.21.15.29:0/1003286476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe75c005bd0 0x7fe75c005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 --2- 172.21.15.29:0/1003286476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75c006a20 0x7fe75c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 --2- 172.21.15.29:0/1003286476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7700b4780 0x7fe75c0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe7700b42f0 con 0x7fe75c005bd0 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe7700b39c0 con 0x7fe75c006a20 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe75c00dbe0 con 0x7fe7700b4780 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76e59c640 1 --2- 172.21.15.29:0/1003286476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe75c005bd0 0x7fe75c005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76e59c640 1 -- 172.21.15.29:0/1003286476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7700b4780 msgr2=0x7fe75c0073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:46.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76e59c640 1 --2- 172.21.15.29:0/1003286476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7700b4780 0x7fe75c0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76e59c640 1 -- 172.21.15.29:0/1003286476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75c006a20 msgr2=0x7fe75c006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:46.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76e59c640 1 --2- 172.21.15.29:0/1003286476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75c006a20 0x7fe75c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76e59c640 1 -- 172.21.15.29:0/1003286476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c005740 con 0x7fe75c005bd0 2024-04-25T00:35:46.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76e59c640 1 --2- 172.21.15.29:0/1003286476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe75c005bd0 0x7fe75c005ff0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c2c0 tx=0x7fe76001cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d473ddd2340440c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:46.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760007d60 con 0x7fe75c005bd0 2024-04-25T00:35:46.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7600a9020 con 0x7fe75c005bd0 2024-04-25T00:35:46.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76dd9b640 1 -- 172.21.15.29:0/1003286476 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760005ae0 con 0x7fe75c005bd0 2024-04-25T00:35:46.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe774e56640 1 -- 172.21.15.29:0/1003286476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe75c005bd0 msgr2=0x7fe75c005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe774e56640 1 --2- 172.21.15.29:0/1003286476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe75c005bd0 0x7fe75c005ff0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c2c0 tx=0x7fe76001cd40 comp rx=0 tx=0).stop 2024-04-25T00:35:46.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe76ed9d640 1 -- 172.21.15.29:0/1003286476 reap_dead start 2024-04-25T00:35:46.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe774e56640 1 -- 172.21.15.29:0/1003286476 shutdown_connections 2024-04-25T00:35:46.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe774e56640 1 -- 172.21.15.29:0/1003286476 >> 172.21.15.29:0/1003286476 conn(0x7fe7700b1870 msgr2=0x7fe7700b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:46.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe774e56640 1 -- 172.21.15.29:0/1003286476 shutdown_connections 2024-04-25T00:35:46.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.008+0000 7fe774e56640 1 -- 172.21.15.29:0/1003286476 wait complete. 2024-04-25T00:35:46.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe774e56640 1 Processor -- start 2024-04-25T00:35:46.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe774e56640 1 -- start start 2024-04-25T00:35:46.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe774e56640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77015b440 0x7fe770137980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe774e56640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe770140400 0x7fe770072410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe774e56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77015b820 0x7fe770160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe774e56640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe77015d6a0 con 0x7fe77015b440 2024-04-25T00:35:46.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe774e56640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe77015d520 con 0x7fe77015b820 2024-04-25T00:35:46.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe774e56640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe77006d590 con 0x7fe770140400 2024-04-25T00:35:46.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe76e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77015b440 0x7fe770137980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe76e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77015b440 0x7fe770137980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53538/0 (socket says 172.21.15.29:53538) 2024-04-25T00:35:46.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe76e59c640 1 -- 172.21.15.29:0/3093855712 learned_addr learned my addr 172.21.15.29:0/3093855712 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:46.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe765d9b640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe770140400 0x7fe770072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe76ed9d640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77015b820 0x7fe770160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe76e59c640 1 -- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe770140400 msgr2=0x7fe770072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe76e59c640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe770140400 0x7fe770072410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe76e59c640 1 -- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77015b820 msgr2=0x7fe770160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe76e59c640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77015b820 0x7fe770160a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe76e59c640 1 -- 172.21.15.29:0/3093855712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe770072e80 con 0x7fe77015b440 2024-04-25T00:35:46.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe76ed9d640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77015b820 0x7fe770160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:46.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe76e59c640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77015b440 0x7fe770137980 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fe76001cb70 tx=0x7fe75c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:46.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe7677fe640 1 -- 172.21.15.29:0/3093855712 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760004ab0 con 0x7fe77015b440 2024-04-25T00:35:46.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe7677fe640 1 -- 172.21.15.29:0/3093855712 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7600b2030 con 0x7fe77015b440 2024-04-25T00:35:46.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe774e56640 1 -- 172.21.15.29:0/3093855712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7700ad940 con 0x7fe77015b440 2024-04-25T00:35:46.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe7677fe640 1 -- 172.21.15.29:0/3093855712 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7600070d0 con 0x7fe77015b440 2024-04-25T00:35:46.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe774e56640 1 -- 172.21.15.29:0/3093855712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7700adf00 con 0x7fe77015b440 2024-04-25T00:35:46.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe774e56640 1 -- 172.21.15.29:0/3093855712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe770069270 con 0x7fe77015b440 2024-04-25T00:35:46.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.012+0000 7fe7677fe640 1 -- 172.21.15.29:0/3093855712 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fe760007270 con 0x7fe77015b440 2024-04-25T00:35:46.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.016+0000 7fe7677fe640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe750067c50 0x7fe75006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.016+0000 7fe7677fe640 1 -- 172.21.15.29:0/3093855712 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7fe76011f080 con 0x7fe77015b440 2024-04-25T00:35:46.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.016+0000 7fe765d9b640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe750067c50 0x7fe75006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.016+0000 7fe765d9b640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe750067c50 0x7fe75006a110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe770143de0 tx=0x7fe758008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:46.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.020+0000 7fe7677fe640 1 -- 172.21.15.29:0/3093855712 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe760016a50 con 0x7fe77015b440 2024-04-25T00:35:46.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.240+0000 7fe774e56640 1 -- 172.21.15.29:0/3093855712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) -- 0x7fe77015c860 con 0x7fe77015b440 2024-04-25T00:35:46.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.244+0000 7fe7677fe640 1 -- 172.21.15.29:0/3093855712 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v19) ==== 82+0+0 (secure 0 0 0) 0x7fe7600efa90 con 0x7fe77015b440 2024-04-25T00:35:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.244+0000 7fe774e56640 1 -- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe750067c50 msgr2=0x7fe75006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.244+0000 7fe774e56640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe750067c50 0x7fe75006a110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe770143de0 tx=0x7fe758008040 comp rx=0 tx=0).stop 2024-04-25T00:35:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.244+0000 7fe774e56640 1 -- 172.21.15.29:0/3093855712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77015b440 msgr2=0x7fe770137980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.244+0000 7fe774e56640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77015b440 0x7fe770137980 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fe76001cb70 tx=0x7fe75c007a50 comp rx=0 tx=0).stop 2024-04-25T00:35:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.244+0000 7fe774e56640 1 -- 172.21.15.29:0/3093855712 shutdown_connections 2024-04-25T00:35:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.244+0000 7fe774e56640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fe750067c50 0x7fe75006a110 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.244+0000 7fe774e56640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77015b820 0x7fe770160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.244+0000 7fe774e56640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe770140400 0x7fe770072410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.244+0000 7fe774e56640 1 --2- 172.21.15.29:0/3093855712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77015b440 0x7fe770137980 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.244+0000 7fe774e56640 1 -- 172.21.15.29:0/3093855712 >> 172.21.15.29:0/3093855712 conn(0x7fe7700b1870 msgr2=0x7fe77006dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.244+0000 7fe774e56640 1 -- 172.21.15.29:0/3093855712 shutdown_connections 2024-04-25T00:35:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.244+0000 7fe774e56640 1 -- 172.21.15.29:0/3093855712 wait complete. 2024-04-25T00:35:46.266 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:46.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa9a84640 1 Processor -- start 2024-04-25T00:35:46.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa9a84640 1 -- start start 2024-04-25T00:35:46.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa9a84640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aa40b3280 0x7f9aa416d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa9a84640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aa4071690 con 0x7f9aa4162420 2024-04-25T00:35:46.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa9a84640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aa4071910 con 0x7f9aa40b4e00 2024-04-25T00:35:46.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa9a84640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa4071bf0 con 0x7f9aa40b3280 2024-04-25T00:35:46.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa37fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9aa4162420 0x7f9aa4070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42646/0 (socket says 172.21.15.29:42646) 2024-04-25T00:35:46.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa37fe640 1 -- 172.21.15.29:0/964698746 learned_addr learned my addr 172.21.15.29:0/964698746 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:46.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1915149463 0 0) 0x7f9a94001120 con 0x7f9aa4162420 2024-04-25T00:35:46.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a8c003530 con 0x7f9aa4162420 2024-04-25T00:35:46.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa3fff640 1 --2- 172.21.15.29:0/964698746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aa40b3280 0x7f9aa416d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1532837890 0 0) 0x7f9a8c003530 con 0x7f9aa4162420 2024-04-25T00:35:46.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aa40b3280 msgr2=0x7f9aa416d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:46.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 --2- 172.21.15.29:0/964698746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aa40b3280 0x7f9aa416d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 >> v1:172.21.15.157:6789/0 conn(0x7f9aa40b4e00 legacy=0x7f9aa40b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa4071ed0 con 0x7f9aa4162420 2024-04-25T00:35:46.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f9a94001850 con 0x7f9aa4162420 2024-04-25T00:35:46.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 >> v1:172.21.15.29:6789/0 conn(0x7f9aa4162420 legacy=0x7f9aa4070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 --2- 172.21.15.29:0/964698746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a8c0039f0 0x7f9a8c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 --2- 172.21.15.29:0/964698746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a8c004810 0x7f9a8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 --2- 172.21.15.29:0/964698746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a8c0051d0 0x7f9a8c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa4071bf0 con 0x7f9a8c004810 2024-04-25T00:35:46.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c00c0a0 con 0x7f9a8c0051d0 2024-04-25T00:35:46.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9a8c00c2a0 con 0x7f9a8c0039f0 2024-04-25T00:35:46.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa3fff640 1 --2- 172.21.15.29:0/964698746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aa40b3280 0x7f9aa416d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:46.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa2ffd640 1 --2- 172.21.15.29:0/964698746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a8c004810 0x7f9a8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.388+0000 7f9aa37fe640 1 --2- 172.21.15.29:0/964698746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a8c0039f0 0x7f9a8c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa3fff640 1 --2- 172.21.15.29:0/964698746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a8c0051d0 0x7f9a8c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa2ffd640 1 -- 172.21.15.29:0/964698746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a8c0039f0 msgr2=0x7f9a8c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:46.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa2ffd640 1 --2- 172.21.15.29:0/964698746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a8c0039f0 0x7f9a8c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa2ffd640 1 -- 172.21.15.29:0/964698746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a8c0051d0 msgr2=0x7f9a8c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:46.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa2ffd640 1 --2- 172.21.15.29:0/964698746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a8c0051d0 0x7f9a8c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa2ffd640 1 -- 172.21.15.29:0/964698746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c003530 con 0x7f9a8c004810 2024-04-25T00:35:46.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa2ffd640 1 --2- 172.21.15.29:0/964698746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a8c004810 0x7f9a8c004c90 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f9a98001780 tx=0x7f9a9801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e717aac0693b9c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:46.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a980013a0 con 0x7f9a8c004810 2024-04-25T00:35:46.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a980a9070 con 0x7f9a8c004810 2024-04-25T00:35:46.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa27fc640 1 -- 172.21.15.29:0/964698746 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98003600 con 0x7f9a8c004810 2024-04-25T00:35:46.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa9a84640 1 -- 172.21.15.29:0/964698746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a8c004810 msgr2=0x7f9a8c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa9a84640 1 --2- 172.21.15.29:0/964698746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a8c004810 0x7f9a8c004c90 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f9a98001780 tx=0x7f9a9801c420 comp rx=0 tx=0).stop 2024-04-25T00:35:46.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa3fff640 1 -- 172.21.15.29:0/964698746 reap_dead start 2024-04-25T00:35:46.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa9a84640 1 -- 172.21.15.29:0/964698746 shutdown_connections 2024-04-25T00:35:46.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa9a84640 1 -- 172.21.15.29:0/964698746 >> 172.21.15.29:0/964698746 conn(0x7f9aa40b1870 msgr2=0x7f9aa40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:46.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa9a84640 1 -- 172.21.15.29:0/964698746 shutdown_connections 2024-04-25T00:35:46.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.392+0000 7f9aa9a84640 1 -- 172.21.15.29:0/964698746 wait complete. 2024-04-25T00:35:46.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa9a84640 1 Processor -- start 2024-04-25T00:35:46.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa9a84640 1 -- start start 2024-04-25T00:35:46.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa9a84640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a8c004810 0x7f9aa415b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa9a84640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa414cd80 0x7f9aa41ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa9a84640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aa415c730 0x7f9aa415cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa9a84640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa406df90 con 0x7f9aa414cd80 2024-04-25T00:35:46.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa9a84640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa4162800 con 0x7f9aa415c730 2024-04-25T00:35:46.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa9a84640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa406e110 con 0x7f9a8c004810 2024-04-25T00:35:46.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa414cd80 0x7f9aa41ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa414cd80 0x7f9aa41ee4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53558/0 (socket says 172.21.15.29:53558) 2024-04-25T00:35:46.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa2ffd640 1 -- 172.21.15.29:0/2130360962 learned_addr learned my addr 172.21.15.29:0/2130360962 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:46.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa3fff640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aa415c730 0x7f9aa415cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa37fe640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a8c004810 0x7f9aa415b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa2ffd640 1 -- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a8c004810 msgr2=0x7f9aa415b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa2ffd640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a8c004810 0x7f9aa415b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa2ffd640 1 -- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aa415c730 msgr2=0x7f9aa415cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa2ffd640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aa415c730 0x7f9aa415cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa2ffd640 1 -- 172.21.15.29:0/2130360962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa415ba50 con 0x7f9aa414cd80 2024-04-25T00:35:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa3fff640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aa415c730 0x7f9aa415cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa2ffd640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa414cd80 0x7f9aa41ee4c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9a98005040 tx=0x7f9a8c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9a8bfff640 1 -- 172.21.15.29:0/2130360962 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98007a40 con 0x7f9aa414cd80 2024-04-25T00:35:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9a8bfff640 1 -- 172.21.15.29:0/2130360962 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a98007250 con 0x7f9aa414cd80 2024-04-25T00:35:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9a8bfff640 1 -- 172.21.15.29:0/2130360962 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98007400 con 0x7f9aa414cd80 2024-04-25T00:35:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa9a84640 1 -- 172.21.15.29:0/2130360962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa415bd30 con 0x7f9aa414cd80 2024-04-25T00:35:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa9a84640 1 -- 172.21.15.29:0/2130360962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa415c2f0 con 0x7f9aa414cd80 2024-04-25T00:35:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9aa9a84640 1 -- 172.21.15.29:0/2130360962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa4069270 con 0x7f9aa414cd80 2024-04-25T00:35:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9a8bfff640 1 -- 172.21.15.29:0/2130360962 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f9a980be020 con 0x7f9aa414cd80 2024-04-25T00:35:46.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9a8bfff640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9a90067c50 0x7f9a9006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.396+0000 7f9a8bfff640 1 -- 172.21.15.29:0/2130360962 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7f9a9811f850 con 0x7f9aa414cd80 2024-04-25T00:35:46.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.400+0000 7f9aa37fe640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9a90067c50 0x7f9a9006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.400+0000 7f9aa37fe640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9a90067c50 0x7f9a9006a110 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9a94001ec0 tx=0x7f9a94001e10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:46.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.404+0000 7f9a8bfff640 1 -- 172.21.15.29:0/2130360962 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9a98016130 con 0x7f9aa414cd80 2024-04-25T00:35:46.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.560+0000 7f9aa9a84640 1 -- 172.21.15.29:0/2130360962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9aa4072fa0 con 0x7f9aa414cd80 2024-04-25T00:35:46.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.560+0000 7f9a8bfff640 1 -- 172.21.15.29:0/2130360962 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+12843 (secure 0 0 0) 0x7f9a980f0140 con 0x7f9aa414cd80 2024-04-25T00:35:46.564 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:46.564 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":66,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:45.478677+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:46.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.564+0000 7f9aa9a84640 1 -- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9a90067c50 msgr2=0x7f9a9006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.564+0000 7f9aa9a84640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9a90067c50 0x7f9a9006a110 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9a94001ec0 tx=0x7f9a94001e10 comp rx=0 tx=0).stop 2024-04-25T00:35:46.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.564+0000 7f9aa9a84640 1 -- 172.21.15.29:0/2130360962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa414cd80 msgr2=0x7f9aa41ee4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.564+0000 7f9aa9a84640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa414cd80 0x7f9aa41ee4c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9a98005040 tx=0x7f9a8c005840 comp rx=0 tx=0).stop 2024-04-25T00:35:46.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.564+0000 7f9aa9a84640 1 -- 172.21.15.29:0/2130360962 shutdown_connections 2024-04-25T00:35:46.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.564+0000 7f9aa9a84640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9a90067c50 0x7f9a9006a110 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.564+0000 7f9aa9a84640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aa415c730 0x7f9aa415cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.564+0000 7f9aa9a84640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa414cd80 0x7f9aa41ee4c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.564+0000 7f9aa9a84640 1 --2- 172.21.15.29:0/2130360962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a8c004810 0x7f9aa415b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.564+0000 7f9aa9a84640 1 -- 172.21.15.29:0/2130360962 >> 172.21.15.29:0/2130360962 conn(0x7f9aa40b1870 msgr2=0x7f9aa40b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:46.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.564+0000 7f9aa9a84640 1 -- 172.21.15.29:0/2130360962 shutdown_connections 2024-04-25T00:35:46.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.564+0000 7f9aa9a84640 1 -- 172.21.15.29:0/2130360962 wait complete. 2024-04-25T00:35:46.582 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:35:46.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:35:46.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87fec0a640 1 Processor -- start 2024-04-25T00:35:46.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87fec0a640 1 -- start start 2024-04-25T00:35:46.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87fec0a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87f815c420 0x7f87f8071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87fec0a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f87f80722c0 0x7f87f80cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87fec0a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87f815e850 con 0x7f87f815e060 2024-04-25T00:35:46.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87fec0a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f87f815f700 con 0x7f87f80722c0 2024-04-25T00:35:46.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87fec0a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f87f815eaf0 con 0x7f87f815c420 2024-04-25T00:35:46.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87fc9a6640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f87f815e060 0x7f87f815bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42658/0 (socket says 172.21.15.29:42658) 2024-04-25T00:35:46.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87fc9a6640 1 -- 172.21.15.29:0/3650675273 learned_addr learned my addr 172.21.15.29:0/3650675273 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:46.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87fd1a7640 1 --2- 172.21.15.29:0/3650675273 >> v2:172.21.15.157:3300/0 conn(0x7f87f80722c0 0x7f87f80cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3175852486 0 0) 0x7f87f815e850 con 0x7f87f815e060 2024-04-25T00:35:46.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f7fff640 1 --2- 172.21.15.29:0/3650675273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87f815c420 0x7f87f8071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87d8003530 con 0x7f87f815e060 2024-04-25T00:35:46.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3850809051 0 0) 0x7f87d8003530 con 0x7f87f815e060 2024-04-25T00:35:46.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87f815c420 msgr2=0x7f87f8071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:46.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 --2- 172.21.15.29:0/3650675273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87f815c420 0x7f87f8071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 >> v2:172.21.15.157:3300/0 conn(0x7f87f80722c0 msgr2=0x7f87f80cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 --2- 172.21.15.29:0/3650675273 >> v2:172.21.15.157:3300/0 conn(0x7f87f80722c0 0x7f87f80cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f815edd0 con 0x7f87f815e060 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f87e8001a00 con 0x7f87f815e060 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 >> v1:172.21.15.29:6789/0 conn(0x7f87f815e060 legacy=0x7f87f815bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 --2- 172.21.15.29:0/3650675273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87f815c420 0x7f87d80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 --2- 172.21.15.29:0/3650675273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d8003f40 0x7f87d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 --2- 172.21.15.29:0/3650675273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87d8004d50 0x7f87d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f87f815f700 con 0x7f87d8003f40 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87f815eaf0 con 0x7f87d8004d50 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f87d800b9d0 con 0x7f87f815c420 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f7fff640 1 --2- 172.21.15.29:0/3650675273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d8003f40 0x7f87d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f7fff640 1 -- 172.21.15.29:0/3650675273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87f815c420 msgr2=0x7f87d80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:46.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f7fff640 1 --2- 172.21.15.29:0/3650675273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87f815c420 0x7f87d80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f7fff640 1 -- 172.21.15.29:0/3650675273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87d8004d50 msgr2=0x7f87d80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:46.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f7fff640 1 --2- 172.21.15.29:0/3650675273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87d8004d50 0x7f87d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.704+0000 7f87f7fff640 1 -- 172.21.15.29:0/3650675273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d8003530 con 0x7f87d8003f40 2024-04-25T00:35:46.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87f7fff640 1 --2- 172.21.15.29:0/3650675273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d8003f40 0x7f87d8004810 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009ae0 tx=0x7f87f001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73cdf76fdf53622 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:46.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00b2070 con 0x7f87d8003f40 2024-04-25T00:35:46.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f00a56a0 con 0x7f87d8003f40 2024-04-25T00:35:46.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87f77fe640 1 -- 172.21.15.29:0/3650675273 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00096b0 con 0x7f87d8003f40 2024-04-25T00:35:46.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 -- 172.21.15.29:0/3650675273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d8003f40 msgr2=0x7f87d8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 --2- 172.21.15.29:0/3650675273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d8003f40 0x7f87d8004810 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009ae0 tx=0x7f87f001ca50 comp rx=0 tx=0).stop 2024-04-25T00:35:46.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fd1a7640 1 -- 172.21.15.29:0/3650675273 reap_dead start 2024-04-25T00:35:46.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 -- 172.21.15.29:0/3650675273 shutdown_connections 2024-04-25T00:35:46.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 -- 172.21.15.29:0/3650675273 >> 172.21.15.29:0/3650675273 conn(0x7f87f806c320 msgr2=0x7f87f806c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:46.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 -- 172.21.15.29:0/3650675273 shutdown_connections 2024-04-25T00:35:46.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 -- 172.21.15.29:0/3650675273 wait complete. 2024-04-25T00:35:46.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 Processor -- start 2024-04-25T00:35:46.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 -- start start 2024-04-25T00:35:46.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d8003f40 0x7f87f80b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f80b12e0 0x7f87f8155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87f814b740 0x7f87f8155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f87f815e440 con 0x7f87d8003f40 2024-04-25T00:35:46.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87f815f390 con 0x7f87f80b12e0 2024-04-25T00:35:46.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fec0a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f87f815e5c0 con 0x7f87f814b740 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fc9a6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d8003f40 0x7f87f80b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fc9a6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d8003f40 0x7f87f80b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53574/0 (socket says 172.21.15.29:53574) 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fc9a6640 1 -- 172.21.15.29:0/3426028051 learned_addr learned my addr 172.21.15.29:0/3426028051 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87f7fff640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f80b12e0 0x7f87f8155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.708+0000 7f87fd1a7640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87f814b740 0x7f87f8155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87fc9a6640 1 -- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87f814b740 msgr2=0x7f87f8155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87fc9a6640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87f814b740 0x7f87f8155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87fc9a6640 1 -- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f80b12e0 msgr2=0x7f87f8155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87fc9a6640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f80b12e0 0x7f87f8155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87fc9a6640 1 -- 172.21.15.29:0/3426028051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f80b68a0 con 0x7f87d8003f40 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87f7fff640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f80b12e0 0x7f87f8155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87fc9a6640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d8003f40 0x7f87f80b0da0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f87e80012f0 tx=0x7f87e800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87f57fa640 1 -- 172.21.15.29:0/3426028051 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e801b070 con 0x7f87d8003f40 2024-04-25T00:35:46.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87f57fa640 1 -- 172.21.15.29:0/3426028051 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e800dd40 con 0x7f87d8003f40 2024-04-25T00:35:46.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87fec0a640 1 -- 172.21.15.29:0/3426028051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f80b5800 con 0x7f87d8003f40 2024-04-25T00:35:46.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87f57fa640 1 -- 172.21.15.29:0/3426028051 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e800a2f0 con 0x7f87d8003f40 2024-04-25T00:35:46.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87fec0a640 1 -- 172.21.15.29:0/3426028051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f80b5dc0 con 0x7f87d8003f40 2024-04-25T00:35:46.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87fec0a640 1 -- 172.21.15.29:0/3426028051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f8069270 con 0x7f87d8003f40 2024-04-25T00:35:46.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87f57fa640 1 -- 172.21.15.29:0/3426028051 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f87e801e070 con 0x7f87d8003f40 2024-04-25T00:35:46.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87f57fa640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f87dc067c50 0x7f87dc06a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:46.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87f57fa640 1 -- 172.21.15.29:0/3426028051 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7f87e8088cc0 con 0x7f87d8003f40 2024-04-25T00:35:46.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87f7fff640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f87dc067c50 0x7f87dc06a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:46.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.712+0000 7f87f7fff640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f87dc067c50 0x7f87dc06a110 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f87f00af000 tx=0x7f87f00046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:46.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.716+0000 7f87f57fa640 1 -- 172.21.15.29:0/3426028051 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f87e8059640 con 0x7f87d8003f40 2024-04-25T00:35:46.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87fec0a640 1 -- 172.21.15.29:0/3426028051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87f806e090 con 0x7f87d8003f40 2024-04-25T00:35:46.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87f57fa640 1 -- 172.21.15.29:0/3426028051 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f87e8058f40 con 0x7f87d8003f40 2024-04-25T00:35:46.883 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:35:46.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87fec0a640 1 -- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f87dc067c50 msgr2=0x7f87dc06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87fec0a640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f87dc067c50 0x7f87dc06a110 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f87f00af000 tx=0x7f87f00046d0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87fec0a640 1 -- 172.21.15.29:0/3426028051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d8003f40 msgr2=0x7f87f80b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:46.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87fec0a640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d8003f40 0x7f87f80b0da0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f87e80012f0 tx=0x7f87e800d430 comp rx=0 tx=0).stop 2024-04-25T00:35:46.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87fec0a640 1 -- 172.21.15.29:0/3426028051 shutdown_connections 2024-04-25T00:35:46.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87fec0a640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87f814b740 0x7f87f8155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87fec0a640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f87dc067c50 0x7f87dc06a110 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87fec0a640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f80b12e0 0x7f87f8155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87fec0a640 1 --2- 172.21.15.29:0/3426028051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d8003f40 0x7f87f80b0da0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:46.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87fec0a640 1 -- 172.21.15.29:0/3426028051 >> 172.21.15.29:0/3426028051 conn(0x7f87f806c320 msgr2=0x7f87f80cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:46.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87fec0a640 1 -- 172.21.15.29:0/3426028051 shutdown_connections 2024-04-25T00:35:46.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:46.880+0000 7f87fec0a640 1 -- 172.21.15.29:0/3426028051 wait complete. 2024-04-25T00:35:46.903 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:35:47.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.016+0000 7f4f060a7640 1 Processor -- start 2024-04-25T00:35:47.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.016+0000 7f4f060a7640 1 -- start start 2024-04-25T00:35:47.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4f060a7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4f0015e8d0 0x7f4f000cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:47.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4f060a7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4f000b4780 0x7f4f0015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:47.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4f060a7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f000b3720 con 0x7f4f0015f1f0 2024-04-25T00:35:47.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4f060a7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4f000b42f0 con 0x7f4f0015e8d0 2024-04-25T00:35:47.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4f060a7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4f0015d7e0 con 0x7f4f000b4780 2024-04-25T00:35:47.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4eff7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4f0015e8d0 0x7f4f000cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:47.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4efffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4f0015f1f0 0x7f4f0015b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42664/0 (socket says 172.21.15.29:42664) 2024-04-25T00:35:47.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4efffff640 1 -- 172.21.15.29:0/283534056 learned_addr learned my addr 172.21.15.29:0/283534056 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:47.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4eff7fe640 1 -- 172.21.15.29:0/283534056 >> v2:172.21.15.157:3300/0 conn(0x7f4f000b4780 msgr2=0x7f4f0015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:47.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4efeffd640 1 --2- 172.21.15.29:0/283534056 >> v2:172.21.15.157:3300/0 conn(0x7f4f000b4780 0x7f4f0015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:47.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4efe7fc640 1 -- 172.21.15.29:0/283534056 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2642910130 0 0) 0x7f4f000b3720 con 0x7f4f0015f1f0 2024-04-25T00:35:47.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4eff7fe640 1 --2- 172.21.15.29:0/283534056 >> v2:172.21.15.157:3300/0 conn(0x7f4f000b4780 0x7f4f0015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4eff7fe640 1 -- 172.21.15.29:0/283534056 >> v1:172.21.15.29:6789/0 conn(0x7f4f0015f1f0 legacy=0x7f4f0015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:47.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4eff7fe640 1 -- 172.21.15.29:0/283534056 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f000b39c0 con 0x7f4f0015e8d0 2024-04-25T00:35:47.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4efeffd640 1 --2- 172.21.15.29:0/283534056 >> v2:172.21.15.157:3300/0 conn(0x7f4f000b4780 0x7f4f0015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:47.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4eff7fe640 1 --2- 172.21.15.29:0/283534056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f0015e8d0 0x7f4f000cc240 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4f00070910 tx=0x7f4ef001eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5014a909b87c60e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:47.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4efe7fc640 1 -- 172.21.15.29:0/283534056 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0015c1c0 con 0x7f4f0015e8d0 2024-04-25T00:35:47.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4efe7fc640 1 -- 172.21.15.29:0/283534056 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef000b2d0 con 0x7f4f0015e8d0 2024-04-25T00:35:47.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.020+0000 7f4efe7fc640 1 -- 172.21.15.29:0/283534056 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00a7bf0 con 0x7f4f0015e8d0 2024-04-25T00:35:47.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 -- 172.21.15.29:0/283534056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f0015e8d0 msgr2=0x7f4f000cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:47.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 --2- 172.21.15.29:0/283534056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f0015e8d0 0x7f4f000cc240 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4f00070910 tx=0x7f4ef001eb00 comp rx=0 tx=0).stop 2024-04-25T00:35:47.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 -- 172.21.15.29:0/283534056 shutdown_connections 2024-04-25T00:35:47.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 --2- 172.21.15.29:0/283534056 >> v2:172.21.15.157:3300/0 conn(0x7f4f000b4780 0x7f4f0015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 --2- 172.21.15.29:0/283534056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f0015e8d0 0x7f4f000cc240 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 -- 172.21.15.29:0/283534056 >> 172.21.15.29:0/283534056 conn(0x7f4f000b1870 msgr2=0x7f4f000b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:47.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 -- 172.21.15.29:0/283534056 shutdown_connections 2024-04-25T00:35:47.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 -- 172.21.15.29:0/283534056 wait complete. 2024-04-25T00:35:47.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 Processor -- start 2024-04-25T00:35:47.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 -- start start 2024-04-25T00:35:47.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f000b4780 0x7f4f0013b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f0015e8d0 0x7f4f0015a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f0015f1f0 0x7f4f000af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00144a20 con 0x7f4f0015e8d0 2024-04-25T00:35:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0015f600 con 0x7f4f000b4780 2024-04-25T00:35:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4f0015aa70 con 0x7f4f0015f1f0 2024-04-25T00:35:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4efeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f0015e8d0 0x7f4f0015a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4efeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f0015e8d0 0x7f4f0015a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53590/0 (socket says 172.21.15.29:53590) 2024-04-25T00:35:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4efeffd640 1 -- 172.21.15.29:0/2584523764 learned_addr learned my addr 172.21.15.29:0/2584523764 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:47.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4eff7fe640 1 --2- 172.21.15.29:0/2584523764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f000b4780 0x7f4f0013b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4efeffd640 1 -- 172.21.15.29:0/2584523764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f0015f1f0 msgr2=0x7f4f000af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4efeffd640 1 --2- 172.21.15.29:0/2584523764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f0015f1f0 0x7f4f000af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4efeffd640 1 -- 172.21.15.29:0/2584523764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f000b4780 msgr2=0x7f4f0013b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4efeffd640 1 --2- 172.21.15.29:0/2584523764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f000b4780 0x7f4f0013b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4efeffd640 1 -- 172.21.15.29:0/2584523764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f00071cb0 con 0x7f4f0015e8d0 2024-04-25T00:35:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4efeffd640 1 --2- 172.21.15.29:0/2584523764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f0015e8d0 0x7f4f0015a530 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400c7c0 tx=0x7f4ef400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4ee7fff640 1 -- 172.21.15.29:0/2584523764 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4007a30 con 0x7f4f0015e8d0 2024-04-25T00:35:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4ee7fff640 1 -- 172.21.15.29:0/2584523764 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef4007bd0 con 0x7f4f0015e8d0 2024-04-25T00:35:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 -- 172.21.15.29:0/2584523764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f001fb210 con 0x7f4f0015e8d0 2024-04-25T00:35:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4ee7fff640 1 -- 172.21.15.29:0/2584523764 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40113f0 con 0x7f4f0015e8d0 2024-04-25T00:35:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.024+0000 7f4f060a7640 1 -- 172.21.15.29:0/2584523764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f001360a0 con 0x7f4f0015e8d0 2024-04-25T00:35:47.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.028+0000 7f4f060a7640 1 -- 172.21.15.29:0/2584523764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f00069270 con 0x7f4f0015e8d0 2024-04-25T00:35:47.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.028+0000 7f4ee7fff640 1 -- 172.21.15.29:0/2584523764 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f4ef4011590 con 0x7f4f0015e8d0 2024-04-25T00:35:47.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.028+0000 7f4ee7fff640 1 --2- 172.21.15.29:0/2584523764 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4eec067c20 0x7f4eec06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:47.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.028+0000 7f4ee7fff640 1 -- 172.21.15.29:0/2584523764 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7f4ef4086150 con 0x7f4f0015e8d0 2024-04-25T00:35:47.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.028+0000 7f4eff7fe640 1 --2- 172.21.15.29:0/2584523764 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4eec067c20 0x7f4eec06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:47.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.028+0000 7f4eff7fe640 1 --2- 172.21.15.29:0/2584523764 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4eec067c20 0x7f4eec06a0e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00b0000 tx=0x7f4ef00a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:47.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.032+0000 7f4ee7fff640 1 -- 172.21.15.29:0/2584523764 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4ef4015020 con 0x7f4f0015e8d0 2024-04-25T00:35:47.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.200+0000 7f4f060a7640 1 -- 172.21.15.29:0/2584523764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4f00144250 con 0x7f4f0015e8d0 2024-04-25T00:35:47.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.200+0000 7f4ee7fff640 1 -- 172.21.15.29:0/2584523764 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v66) ==== 84+0+65 (secure 0 0 0) 0x7f4ef4056ac0 con 0x7f4f0015e8d0 2024-04-25T00:35:47.203 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:47.203 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:35:47.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:35:47.204 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-04-25T00:35:47.204 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": ".mgr" 2024-04-25T00:35:47.204 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:35:47.204 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:35:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.204+0000 7f4f060a7640 1 -- 172.21.15.29:0/2584523764 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4eec067c20 msgr2=0x7f4eec06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.204+0000 7f4f060a7640 1 --2- 172.21.15.29:0/2584523764 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4eec067c20 0x7f4eec06a0e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00b0000 tx=0x7f4ef00a9070 comp rx=0 tx=0).stop 2024-04-25T00:35:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.204+0000 7f4f060a7640 1 -- 172.21.15.29:0/2584523764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f0015e8d0 msgr2=0x7f4f0015a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.204+0000 7f4f060a7640 1 --2- 172.21.15.29:0/2584523764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f0015e8d0 0x7f4f0015a530 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400c7c0 tx=0x7f4ef400cc80 comp rx=0 tx=0).stop 2024-04-25T00:35:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.204+0000 7f4f060a7640 1 -- 172.21.15.29:0/2584523764 shutdown_connections 2024-04-25T00:35:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.204+0000 7f4f060a7640 1 --2- 172.21.15.29:0/2584523764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f0015f1f0 0x7f4f000af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.204+0000 7f4f060a7640 1 --2- 172.21.15.29:0/2584523764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f0015e8d0 0x7f4f0015a530 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.204+0000 7f4f060a7640 1 --2- 172.21.15.29:0/2584523764 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4eec067c20 0x7f4eec06a0e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.204+0000 7f4f060a7640 1 --2- 172.21.15.29:0/2584523764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f000b4780 0x7f4f0013b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.204+0000 7f4f060a7640 1 -- 172.21.15.29:0/2584523764 >> 172.21.15.29:0/2584523764 conn(0x7f4f000b1870 msgr2=0x7f4f000cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.204+0000 7f4f060a7640 1 -- 172.21.15.29:0/2584523764 shutdown_connections 2024-04-25T00:35:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.204+0000 7f4f060a7640 1 -- 172.21.15.29:0/2584523764 wait complete. 2024-04-25T00:35:47.223 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:35:47.223 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:35:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba85293640 1 Processor -- start 2024-04-25T00:35:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba85293640 1 -- start start 2024-04-25T00:35:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba85293640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fba8015e1a0 0x7fba80071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba85293640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba80070ce0 0x7fba800b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba85293640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba800b3ac0 con 0x7fba800b4b30 2024-04-25T00:35:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba85293640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fba800714f0 con 0x7fba8015e1a0 2024-04-25T00:35:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba85293640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba800b4fa0 con 0x7fba80070ce0 2024-04-25T00:35:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba7f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fba800b4b30 0x7fba80161be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42678/0 (socket says 172.21.15.29:42678) 2024-04-25T00:35:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba7f7fe640 1 -- 172.21.15.29:0/1927584270 learned_addr learned my addr 172.21.15.29:0/1927584270 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba7effd640 1 --2- 172.21.15.29:0/1927584270 >> v2:172.21.15.29:3300/0 conn(0x7fba8015e1a0 0x7fba80071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba77fff640 1 --2- 172.21.15.29:0/1927584270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba80070ce0 0x7fba800b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba7e7fc640 1 -- 172.21.15.29:0/1927584270 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2246511172 0 0) 0x7fba800b3ac0 con 0x7fba800b4b30 2024-04-25T00:35:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba7e7fc640 1 -- 172.21.15.29:0/1927584270 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba6c003530 con 0x7fba800b4b30 2024-04-25T00:35:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba7effd640 1 -- 172.21.15.29:0/1927584270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba80070ce0 msgr2=0x7fba800b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba7effd640 1 --2- 172.21.15.29:0/1927584270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba80070ce0 0x7fba800b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba7effd640 1 -- 172.21.15.29:0/1927584270 >> v1:172.21.15.29:6789/0 conn(0x7fba800b4b30 legacy=0x7fba80161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.344+0000 7fba7effd640 1 -- 172.21.15.29:0/1927584270 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba800b3d60 con 0x7fba8015e1a0 2024-04-25T00:35:47.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.348+0000 7fba7effd640 1 --2- 172.21.15.29:0/1927584270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba8015e1a0 0x7fba80071b40 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fba700097c0 tx=0x7fba7001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee1a571a4e4f359 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:47.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.348+0000 7fba7e7fc640 1 -- 172.21.15.29:0/1927584270 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba700b0070 con 0x7fba8015e1a0 2024-04-25T00:35:47.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.348+0000 7fba7e7fc640 1 -- 172.21.15.29:0/1927584270 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba700ab600 con 0x7fba8015e1a0 2024-04-25T00:35:47.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.348+0000 7fba7e7fc640 1 -- 172.21.15.29:0/1927584270 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba700ab9b0 con 0x7fba8015e1a0 2024-04-25T00:35:47.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.348+0000 7fba85293640 1 -- 172.21.15.29:0/1927584270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba8015e1a0 msgr2=0x7fba80071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:47.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.348+0000 7fba85293640 1 --2- 172.21.15.29:0/1927584270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba8015e1a0 0x7fba80071b40 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fba700097c0 tx=0x7fba7001c410 comp rx=0 tx=0).stop 2024-04-25T00:35:47.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.348+0000 7fba85293640 1 -- 172.21.15.29:0/1927584270 shutdown_connections 2024-04-25T00:35:47.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.348+0000 7fba85293640 1 --2- 172.21.15.29:0/1927584270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba80070ce0 0x7fba800b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.348+0000 7fba85293640 1 --2- 172.21.15.29:0/1927584270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba8015e1a0 0x7fba80071b40 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.348+0000 7fba85293640 1 -- 172.21.15.29:0/1927584270 >> 172.21.15.29:0/1927584270 conn(0x7fba800b1870 msgr2=0x7fba800b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:47.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.348+0000 7fba85293640 1 -- 172.21.15.29:0/1927584270 shutdown_connections 2024-04-25T00:35:47.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.348+0000 7fba85293640 1 -- 172.21.15.29:0/1927584270 wait complete. 2024-04-25T00:35:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba85293640 1 Processor -- start 2024-04-25T00:35:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba85293640 1 -- start start 2024-04-25T00:35:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba85293640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba80070ce0 0x7fba801512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba85293640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba800b4b30 0x7fba8015f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba85293640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba8015e1a0 0x7fba800cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba85293640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fba8016af60 con 0x7fba8015e1a0 2024-04-25T00:35:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba85293640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba800712d0 con 0x7fba800b4b30 2024-04-25T00:35:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba85293640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba8016b0e0 con 0x7fba80070ce0 2024-04-25T00:35:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba7f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba8015e1a0 0x7fba800cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba7f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba8015e1a0 0x7fba800cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53604/0 (socket says 172.21.15.29:53604) 2024-04-25T00:35:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba7f7fe640 1 -- 172.21.15.29:0/2317952030 learned_addr learned my addr 172.21.15.29:0/2317952030 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba77fff640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba800b4b30 0x7fba8015f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba7effd640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba80070ce0 0x7fba801512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba7f7fe640 1 -- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba80070ce0 msgr2=0x7fba801512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba7f7fe640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba80070ce0 0x7fba801512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba7f7fe640 1 -- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba800b4b30 msgr2=0x7fba8015f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba7f7fe640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba800b4b30 0x7fba8015f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba7f7fe640 1 -- 172.21.15.29:0/2317952030 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8015fcf0 con 0x7fba8015e1a0 2024-04-25T00:35:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba7effd640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba80070ce0 0x7fba801512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba7f7fe640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba8015e1a0 0x7fba800cd690 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fba7800caa0 tx=0x7fba7800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba777fe640 1 -- 172.21.15.29:0/2317952030 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78017070 con 0x7fba8015e1a0 2024-04-25T00:35:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba777fe640 1 -- 172.21.15.29:0/2317952030 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba7800ac00 con 0x7fba8015e1a0 2024-04-25T00:35:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba85293640 1 -- 172.21.15.29:0/2317952030 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba800ccbb0 con 0x7fba8015e1a0 2024-04-25T00:35:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba777fe640 1 -- 172.21.15.29:0/2317952030 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78012400 con 0x7fba8015e1a0 2024-04-25T00:35:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba85293640 1 -- 172.21.15.29:0/2317952030 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba800cd170 con 0x7fba8015e1a0 2024-04-25T00:35:47.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba85293640 1 -- 172.21.15.29:0/2317952030 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba800692e0 con 0x7fba8015e1a0 2024-04-25T00:35:47.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba777fe640 1 -- 172.21.15.29:0/2317952030 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fba7801a070 con 0x7fba8015e1a0 2024-04-25T00:35:47.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba777fe640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba60067c20 0x7fba6006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:47.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.352+0000 7fba777fe640 1 -- 172.21.15.29:0/2317952030 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7fba78085bf0 con 0x7fba8015e1a0 2024-04-25T00:35:47.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.356+0000 7fba7effd640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba60067c20 0x7fba6006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:47.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.356+0000 7fba7effd640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba60067c20 0x7fba6006a0e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fba700ad000 tx=0x7fba700a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:47.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.360+0000 7fba777fe640 1 -- 172.21.15.29:0/2317952030 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fba78056700 con 0x7fba8015e1a0 2024-04-25T00:35:47.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:47.516+0000 7fba85293640 1 -- 172.21.15.29:0/2317952030 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fba801578c0 con 0x7fba8015e1a0 2024-04-25T00:35:48.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.140+0000 7fba777fe640 1 -- 172.21.15.29:0/2317952030 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v67) ==== 138+0+0 (secure 0 0 0) 0x7fba78056000 con 0x7fba8015e1a0 2024-04-25T00:35:48.141 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-04-25T00:35:48.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.144+0000 7fba85293640 1 -- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba60067c20 msgr2=0x7fba6006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:48.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.144+0000 7fba85293640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba60067c20 0x7fba6006a0e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fba700ad000 tx=0x7fba700a6000 comp rx=0 tx=0).stop 2024-04-25T00:35:48.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.144+0000 7fba85293640 1 -- 172.21.15.29:0/2317952030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba8015e1a0 msgr2=0x7fba800cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:48.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.144+0000 7fba85293640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba8015e1a0 0x7fba800cd690 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fba7800caa0 tx=0x7fba7800cf60 comp rx=0 tx=0).stop 2024-04-25T00:35:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.144+0000 7fba85293640 1 -- 172.21.15.29:0/2317952030 shutdown_connections 2024-04-25T00:35:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.144+0000 7fba85293640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba8015e1a0 0x7fba800cd690 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.144+0000 7fba85293640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fba60067c20 0x7fba6006a0e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.144+0000 7fba85293640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba800b4b30 0x7fba8015f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.144+0000 7fba85293640 1 --2- 172.21.15.29:0/2317952030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba80070ce0 0x7fba801512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.144+0000 7fba85293640 1 -- 172.21.15.29:0/2317952030 >> 172.21.15.29:0/2317952030 conn(0x7fba800b1870 msgr2=0x7fba800b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:48.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.144+0000 7fba85293640 1 -- 172.21.15.29:0/2317952030 shutdown_connections 2024-04-25T00:35:48.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.144+0000 7fba85293640 1 -- 172.21.15.29:0/2317952030 wait complete. 2024-04-25T00:35:48.173 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:35:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f246b18d640 1 Processor -- start 2024-04-25T00:35:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f246b18d640 1 -- start start 2024-04-25T00:35:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f246b18d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24640b0f80 0x7f24640b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f246b18d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24640b02f0 0x7f24640b0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f246b18d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24640cf740 con 0x7f246415e1b0 2024-04-25T00:35:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f246b18d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f24640ad8b0 con 0x7f24640b02f0 2024-04-25T00:35:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f246b18d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f24640b1920 con 0x7f24640b0f80 2024-04-25T00:35:48.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f246972a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24640b02f0 0x7f24640b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:48.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f2463fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24640b0f80 0x7f24640b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:48.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f2468f29640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f246415e1b0 0x7f24640ad1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:51298/0 (socket says 172.21.15.29:51298) 2024-04-25T00:35:48.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f2468f29640 1 -- 172.21.15.29:0/2337740668 learned_addr learned my addr 172.21.15.29:0/2337740668 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:48.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f246972a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24640b02f0 0x7f24640b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:34546/0 (socket says 172.21.15.29:34546) 2024-04-25T00:35:48.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f246972a640 1 -- 172.21.15.29:0/2337740668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24640b0f80 msgr2=0x7f24640b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:48.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f246972a640 1 --2- 172.21.15.29:0/2337740668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24640b0f80 0x7f24640b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:48.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f246972a640 1 -- 172.21.15.29:0/2337740668 >> v1:172.21.15.157:6789/0 conn(0x7f246415e1b0 legacy=0x7f24640ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:48.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.308+0000 7f246972a640 1 -- 172.21.15.29:0/2337740668 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24640cfa20 con 0x7f24640b02f0 2024-04-25T00:35:48.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.312+0000 7f246972a640 1 --2- 172.21.15.29:0/2337740668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24640b02f0 0x7f24640b0780 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f24540097c0 tx=0x7f245401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f96e5b9392c6c520 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.312+0000 7f24637fe640 1 -- 172.21.15.29:0/2337740668 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24540a7070 con 0x7f24640b02f0 2024-04-25T00:35:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.312+0000 7f24637fe640 1 -- 172.21.15.29:0/2337740668 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24540b0070 con 0x7f24640b02f0 2024-04-25T00:35:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.312+0000 7f24637fe640 1 -- 172.21.15.29:0/2337740668 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24540ab890 con 0x7f24640b02f0 2024-04-25T00:35:48.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.312+0000 7f246b18d640 1 -- 172.21.15.29:0/2337740668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24640b02f0 msgr2=0x7f24640b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:48.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.312+0000 7f246b18d640 1 --2- 172.21.15.29:0/2337740668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24640b02f0 0x7f24640b0780 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f24540097c0 tx=0x7f245401c470 comp rx=0 tx=0).stop 2024-04-25T00:35:48.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.312+0000 7f246b18d640 1 -- 172.21.15.29:0/2337740668 shutdown_connections 2024-04-25T00:35:48.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.312+0000 7f246b18d640 1 --2- 172.21.15.29:0/2337740668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24640b02f0 0x7f24640b0780 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:48.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.312+0000 7f246b18d640 1 --2- 172.21.15.29:0/2337740668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24640b0f80 0x7f24640b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:48.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.312+0000 7f246b18d640 1 -- 172.21.15.29:0/2337740668 >> 172.21.15.29:0/2337740668 conn(0x7f2464072930 msgr2=0x7f2464072d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:48.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.312+0000 7f246b18d640 1 -- 172.21.15.29:0/2337740668 shutdown_connections 2024-04-25T00:35:48.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.312+0000 7f246b18d640 1 -- 172.21.15.29:0/2337740668 wait complete. 2024-04-25T00:35:48.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246b18d640 1 Processor -- start 2024-04-25T00:35:48.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246b18d640 1 -- start start 2024-04-25T00:35:48.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246b18d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24640b02f0 0x7f2464161d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:48.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246b18d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24640b0f80 0x7f2464162240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:48.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246b18d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f246415e1b0 0x7f24641539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:48.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246b18d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f246415e590 con 0x7f246415e1b0 2024-04-25T00:35:48.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246b18d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f24640b0cc0 con 0x7f24640b02f0 2024-04-25T00:35:48.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246b18d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f24640d0ff0 con 0x7f24640b0f80 2024-04-25T00:35:48.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246972a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f246415e1b0 0x7f24641539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:48.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246972a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f246415e1b0 0x7f24641539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52274/0 (socket says 172.21.15.29:52274) 2024-04-25T00:35:48.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246972a640 1 -- 172.21.15.29:0/882410902 learned_addr learned my addr 172.21.15.29:0/882410902 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f2463fff640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24640b0f80 0x7f2464162240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f2468f29640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24640b02f0 0x7f2464161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246972a640 1 -- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24640b0f80 msgr2=0x7f2464162240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246972a640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24640b0f80 0x7f2464162240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246972a640 1 -- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24640b02f0 msgr2=0x7f2464161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246972a640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24640b02f0 0x7f2464161d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246972a640 1 -- 172.21.15.29:0/882410902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2464156050 con 0x7f246415e1b0 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f2463fff640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24640b0f80 0x7f2464162240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246972a640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f246415e1b0 0x7f24641539d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f245401a0a0 tx=0x7f245401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f24617fa640 1 -- 172.21.15.29:0/882410902 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2454006e20 con 0x7f246415e1b0 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f24617fa640 1 -- 172.21.15.29:0/882410902 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24540bf610 con 0x7f246415e1b0 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f24617fa640 1 -- 172.21.15.29:0/882410902 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24540bf9e0 con 0x7f246415e1b0 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246b18d640 1 -- 172.21.15.29:0/882410902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2464160cc0 con 0x7f246415e1b0 2024-04-25T00:35:48.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246b18d640 1 -- 172.21.15.29:0/882410902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2464157720 con 0x7f246415e1b0 2024-04-25T00:35:48.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f246b18d640 1 -- 172.21.15.29:0/882410902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2464075770 con 0x7f246415e1b0 2024-04-25T00:35:48.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.316+0000 7f24617fa640 1 -- 172.21.15.29:0/882410902 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f24540bd020 con 0x7f246415e1b0 2024-04-25T00:35:48.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.320+0000 7f24617fa640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2450067c50 0x7f245006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:48.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.320+0000 7f24617fa640 1 -- 172.21.15.29:0/882410902 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5227+0+0 (secure 0 0 0) 0x7f2454120640 con 0x7f246415e1b0 2024-04-25T00:35:48.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.320+0000 7f2468f29640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2450067c50 0x7f245006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:48.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.320+0000 7f2468f29640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2450067c50 0x7f245006a110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f245c0012f0 tx=0x7f245c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:48.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.324+0000 7f24617fa640 1 -- 172.21.15.29:0/882410902 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f24540c6070 con 0x7f246415e1b0 2024-04-25T00:35:48.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:48.484+0000 7f246b18d640 1 -- 172.21.15.29:0/882410902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2464160ab0 con 0x7f246415e1b0 2024-04-25T00:35:49.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.144+0000 7f24617fa640 1 -- 172.21.15.29:0/882410902 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v68) ==== 170+0+0 (secure 0 0 0) 0x7f2454016090 con 0x7f246415e1b0 2024-04-25T00:35:49.146 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-04-25T00:35:49.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.144+0000 7f246b18d640 1 -- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2450067c50 msgr2=0x7f245006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:49.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.144+0000 7f246b18d640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2450067c50 0x7f245006a110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f245c0012f0 tx=0x7f245c009040 comp rx=0 tx=0).stop 2024-04-25T00:35:49.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.144+0000 7f246b18d640 1 -- 172.21.15.29:0/882410902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f246415e1b0 msgr2=0x7f24641539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:49.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.144+0000 7f246b18d640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f246415e1b0 0x7f24641539d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f245401a0a0 tx=0x7f245401acb0 comp rx=0 tx=0).stop 2024-04-25T00:35:49.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.148+0000 7f246b18d640 1 -- 172.21.15.29:0/882410902 shutdown_connections 2024-04-25T00:35:49.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.148+0000 7f246b18d640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f246415e1b0 0x7f24641539d0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:49.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.148+0000 7f246b18d640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24640b0f80 0x7f2464162240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:49.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.148+0000 7f246b18d640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2450067c50 0x7f245006a110 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:49.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.148+0000 7f246b18d640 1 --2- 172.21.15.29:0/882410902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24640b02f0 0x7f2464161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:49.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.148+0000 7f246b18d640 1 -- 172.21.15.29:0/882410902 >> 172.21.15.29:0/882410902 conn(0x7f2464072930 msgr2=0x7f246406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:49.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.148+0000 7f246b18d640 1 -- 172.21.15.29:0/882410902 shutdown_connections 2024-04-25T00:35:49.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.148+0000 7f246b18d640 1 -- 172.21.15.29:0/882410902 wait complete. 2024-04-25T00:35:49.183 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:35:49.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd47742f640 1 Processor -- start 2024-04-25T00:35:49.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd47742f640 1 -- start start 2024-04-25T00:35:49.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd47742f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4700d0890 0x7fd47015fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:49.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd47742f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd4700d0c70 0x7fd4700ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:49.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd47742f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4700cd220 0x7fd47006ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:49.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd47742f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd4700723c0 con 0x7fd4700d0890 2024-04-25T00:35:49.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd47742f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd470071560 con 0x7fd4700d0c70 2024-04-25T00:35:49.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd47742f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd470071840 con 0x7fd4700cd220 2024-04-25T00:35:49.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd4751cb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4700d0890 0x7fd47015fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:49.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd4751cb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4700d0890 0x7fd47015fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52286/0 (socket says 172.21.15.29:52286) 2024-04-25T00:35:49.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd4751cb640 1 -- 172.21.15.29:0/4291076423 learned_addr learned my addr 172.21.15.29:0/4291076423 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:49.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd4759cc640 1 --2- 172.21.15.29:0/4291076423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4700cd220 0x7fd47006ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:49.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd4749ca640 1 --2- 172.21.15.29:0/4291076423 >> v2:172.21.15.157:3300/0 conn(0x7fd4700d0c70 0x7fd4700ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:49.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd4751cb640 1 -- 172.21.15.29:0/4291076423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4700cd220 msgr2=0x7fd47006ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:49.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd4751cb640 1 --2- 172.21.15.29:0/4291076423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4700cd220 0x7fd47006ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:49.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd4751cb640 1 -- 172.21.15.29:0/4291076423 >> v2:172.21.15.157:3300/0 conn(0x7fd4700d0c70 msgr2=0x7fd4700ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:49.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd4751cb640 1 --2- 172.21.15.29:0/4291076423 >> v2:172.21.15.157:3300/0 conn(0x7fd4700d0c70 0x7fd4700ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:49.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.312+0000 7fd4751cb640 1 -- 172.21.15.29:0/4291076423 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd470071b20 con 0x7fd4700d0890 2024-04-25T00:35:49.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd4751cb640 1 --2- 172.21.15.29:0/4291076423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4700d0890 0x7fd47015fb20 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd470072960 tx=0x7fd46001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8816d1d903fd0add server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:49.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd46ffff640 1 -- 172.21.15.29:0/4291076423 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4600b1070 con 0x7fd4700d0890 2024-04-25T00:35:49.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd46ffff640 1 -- 172.21.15.29:0/4291076423 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4600ac620 con 0x7fd4700d0890 2024-04-25T00:35:49.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd46ffff640 1 -- 172.21.15.29:0/4291076423 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4600ac9f0 con 0x7fd4700d0890 2024-04-25T00:35:49.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 -- 172.21.15.29:0/4291076423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4700d0890 msgr2=0x7fd47015fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:49.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 --2- 172.21.15.29:0/4291076423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4700d0890 0x7fd47015fb20 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd470072960 tx=0x7fd46001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:35:49.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 -- 172.21.15.29:0/4291076423 shutdown_connections 2024-04-25T00:35:49.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 --2- 172.21.15.29:0/4291076423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4700cd220 0x7fd47006ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:49.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 --2- 172.21.15.29:0/4291076423 >> v2:172.21.15.157:3300/0 conn(0x7fd4700d0c70 0x7fd4700ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:49.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 --2- 172.21.15.29:0/4291076423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4700d0890 0x7fd47015fb20 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:49.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 -- 172.21.15.29:0/4291076423 >> 172.21.15.29:0/4291076423 conn(0x7fd470074c00 msgr2=0x7fd47006b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:49.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 -- 172.21.15.29:0/4291076423 shutdown_connections 2024-04-25T00:35:49.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 -- 172.21.15.29:0/4291076423 wait complete. 2024-04-25T00:35:49.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 Processor -- start 2024-04-25T00:35:49.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 -- start start 2024-04-25T00:35:49.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4700cd220 0x7fd470150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:49.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4700d0890 0x7fd470152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:49.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4700d0c70 0x7fd470155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:49.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd47015d2a0 con 0x7fd4700d0890 2024-04-25T00:35:49.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd47015d120 con 0x7fd4700d0c70 2024-04-25T00:35:49.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd47742f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd47015d420 con 0x7fd4700cd220 2024-04-25T00:35:49.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd4749ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4700d0890 0x7fd470152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:49.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd4749ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4700d0890 0x7fd470152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52298/0 (socket says 172.21.15.29:52298) 2024-04-25T00:35:49.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd4749ca640 1 -- 172.21.15.29:0/3930637810 learned_addr learned my addr 172.21.15.29:0/3930637810 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd4751cb640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4700cd220 0x7fd470150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd4759cc640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4700d0c70 0x7fd470155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd4749ca640 1 -- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4700cd220 msgr2=0x7fd470150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd4749ca640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4700cd220 0x7fd470150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd4749ca640 1 -- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4700d0c70 msgr2=0x7fd470155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd4749ca640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4700d0c70 0x7fd470155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd4749ca640 1 -- 172.21.15.29:0/3930637810 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4701514d0 con 0x7fd4700d0890 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.316+0000 7fd4749ca640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4700d0890 0x7fd470152d20 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd464007e70 tx=0x7fd46400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.320+0000 7fd46dffb640 1 -- 172.21.15.29:0/3930637810 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464014620 con 0x7fd4700d0890 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.320+0000 7fd46dffb640 1 -- 172.21.15.29:0/3930637810 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd464019070 con 0x7fd4700d0890 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.320+0000 7fd47742f640 1 -- 172.21.15.29:0/3930637810 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4701596c0 con 0x7fd4700d0890 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.320+0000 7fd46dffb640 1 -- 172.21.15.29:0/3930637810 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464014a00 con 0x7fd4700d0890 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.320+0000 7fd47742f640 1 -- 172.21.15.29:0/3930637810 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd470159c80 con 0x7fd4700d0890 2024-04-25T00:35:49.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.320+0000 7fd47742f640 1 -- 172.21.15.29:0/3930637810 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd470159e80 con 0x7fd4700d0890 2024-04-25T00:35:49.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.320+0000 7fd46dffb640 1 -- 172.21.15.29:0/3930637810 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fd464014ba0 con 0x7fd4700d0890 2024-04-25T00:35:49.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.320+0000 7fd46dffb640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd45c067c50 0x7fd45c06a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:49.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.320+0000 7fd46dffb640 1 -- 172.21.15.29:0/3930637810 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5609+0+0 (secure 0 0 0) 0x7fd464087820 con 0x7fd4700d0890 2024-04-25T00:35:49.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.320+0000 7fd4751cb640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd45c067c50 0x7fd45c06a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:49.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.320+0000 7fd4751cb640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd45c067c50 0x7fd45c06a110 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd4600ae000 tx=0x7fd460008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:49.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.324+0000 7fd46dffb640 1 -- 172.21.15.29:0/3930637810 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd46401c360 con 0x7fd4700d0890 2024-04-25T00:35:49.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:49.524+0000 7fd47742f640 1 -- 172.21.15.29:0/3930637810 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd470069250 con 0x7fd4700d0890 2024-04-25T00:35:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 10 and data pool 11 2024-04-25T00:35:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.188+0000 7fd46dffb640 1 -- 172.21.15.29:0/3930637810 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v39) ==== 174+0+0 (secure 0 0 0) 0x7fd4640580b0 con 0x7fd4700d0890 2024-04-25T00:35:50.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.196+0000 7fd47742f640 1 -- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd45c067c50 msgr2=0x7fd45c06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.196+0000 7fd47742f640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd45c067c50 0x7fd45c06a110 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd4600ae000 tx=0x7fd460008040 comp rx=0 tx=0).stop 2024-04-25T00:35:50.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.196+0000 7fd47742f640 1 -- 172.21.15.29:0/3930637810 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4700d0890 msgr2=0x7fd470152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.196+0000 7fd47742f640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4700d0890 0x7fd470152d20 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd464007e70 tx=0x7fd46400e4b0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.196+0000 7fd47742f640 1 -- 172.21.15.29:0/3930637810 shutdown_connections 2024-04-25T00:35:50.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.196+0000 7fd47742f640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd45c067c50 0x7fd45c06a110 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.196+0000 7fd47742f640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4700d0c70 0x7fd470155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.196+0000 7fd47742f640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4700d0890 0x7fd470152d20 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.196+0000 7fd47742f640 1 --2- 172.21.15.29:0/3930637810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4700cd220 0x7fd470150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.196+0000 7fd47742f640 1 -- 172.21.15.29:0/3930637810 >> 172.21.15.29:0/3930637810 conn(0x7fd470074c00 msgr2=0x7fd4700b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:50.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.196+0000 7fd47742f640 1 -- 172.21.15.29:0/3930637810 shutdown_connections 2024-04-25T00:35:50.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.196+0000 7fd47742f640 1 -- 172.21.15.29:0/3930637810 wait complete. 2024-04-25T00:35:50.231 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:50.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.344+0000 7f08bceb5640 1 Processor -- start 2024-04-25T00:35:50.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.344+0000 7f08bceb5640 1 -- start start 2024-04-25T00:35:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08bceb5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08b80b3140 0x7f08b806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08bceb5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f08b80b3c30 0x7f08b80ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08bceb5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08b80b41e0 con 0x7f08b80b4b10 2024-04-25T00:35:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08bceb5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f08b806d960 con 0x7f08b80b3c30 2024-04-25T00:35:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08bceb5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f08b806dc40 con 0x7f08b80b3140 2024-04-25T00:35:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08b6d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f08b80b3c30 0x7f08b80ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08b5d9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f08b80b4b10 0x7f08b80b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35272/0 (socket says 172.21.15.29:35272) 2024-04-25T00:35:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08b5d9b640 1 -- 172.21.15.29:0/2319346241 learned_addr learned my addr 172.21.15.29:0/2319346241 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08b659c640 1 --2- 172.21.15.29:0/2319346241 >> v2:172.21.15.157:3300/0 conn(0x7f08b80b3140 0x7f08b806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08b6d9d640 1 -- 172.21.15.29:0/2319346241 >> v2:172.21.15.157:3300/0 conn(0x7f08b80b3140 msgr2=0x7f08b806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08b6d9d640 1 --2- 172.21.15.29:0/2319346241 >> v2:172.21.15.157:3300/0 conn(0x7f08b80b3140 0x7f08b806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08b6d9d640 1 -- 172.21.15.29:0/2319346241 >> v1:172.21.15.29:6789/0 conn(0x7f08b80b4b10 legacy=0x7f08b80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08b6d9d640 1 -- 172.21.15.29:0/2319346241 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b806df20 con 0x7f08b80b3c30 2024-04-25T00:35:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08b6d9d640 1 --2- 172.21.15.29:0/2319346241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08b80b3c30 0x7f08b80ce240 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009d90 tx=0x7f08b001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772332a8f31ae24d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08b559a640 1 -- 172.21.15.29:0/2319346241 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b00b2070 con 0x7f08b80b3c30 2024-04-25T00:35:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08b559a640 1 -- 172.21.15.29:0/2319346241 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08b00a57f0 con 0x7f08b80b3c30 2024-04-25T00:35:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08b559a640 1 -- 172.21.15.29:0/2319346241 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b00a5bc0 con 0x7f08b80b3c30 2024-04-25T00:35:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08bceb5640 1 -- 172.21.15.29:0/2319346241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08b80b3c30 msgr2=0x7f08b80ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08bceb5640 1 --2- 172.21.15.29:0/2319346241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08b80b3c30 0x7f08b80ce240 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009d90 tx=0x7f08b001c930 comp rx=0 tx=0).stop 2024-04-25T00:35:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08bceb5640 1 -- 172.21.15.29:0/2319346241 shutdown_connections 2024-04-25T00:35:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08bceb5640 1 --2- 172.21.15.29:0/2319346241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08b80b3c30 0x7f08b80ce240 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08bceb5640 1 --2- 172.21.15.29:0/2319346241 >> v2:172.21.15.157:3300/0 conn(0x7f08b80b3140 0x7f08b806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.348+0000 7f08bceb5640 1 -- 172.21.15.29:0/2319346241 >> 172.21.15.29:0/2319346241 conn(0x7f08b80b1870 msgr2=0x7f08b80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08bceb5640 1 -- 172.21.15.29:0/2319346241 shutdown_connections 2024-04-25T00:35:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08bceb5640 1 -- 172.21.15.29:0/2319346241 wait complete. 2024-04-25T00:35:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08bceb5640 1 Processor -- start 2024-04-25T00:35:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08bceb5640 1 -- start start 2024-04-25T00:35:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08bceb5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08b80b3140 0x7f08b815bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08bceb5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80b3c30 0x7f08b815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08bceb5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b80b4b10 0x7f08b813a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08bceb5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f08b8168d80 con 0x7f08b80b3140 2024-04-25T00:35:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08bceb5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08b8168c00 con 0x7f08b80b4b10 2024-04-25T00:35:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08bceb5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f08b8168f00 con 0x7f08b80b3c30 2024-04-25T00:35:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08b659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08b80b3140 0x7f08b815bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08b659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08b80b3140 0x7f08b815bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52308/0 (socket says 172.21.15.29:52308) 2024-04-25T00:35:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08b659c640 1 -- 172.21.15.29:0/288127193 learned_addr learned my addr 172.21.15.29:0/288127193 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08b6d9d640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b80b4b10 0x7f08b813a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08b5d9b640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80b3c30 0x7f08b815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08b659c640 1 -- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80b3c30 msgr2=0x7f08b815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08b659c640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80b3c30 0x7f08b815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08b659c640 1 -- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b80b4b10 msgr2=0x7f08b813a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08b659c640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b80b4b10 0x7f08b813a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08b659c640 1 -- 172.21.15.29:0/288127193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b813dcc0 con 0x7f08b80b3140 2024-04-25T00:35:50.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08b659c640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08b80b3140 0x7f08b815bab0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f08a800d620 tx=0x7f08a800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:50.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08a6ffd640 1 -- 172.21.15.29:0/288127193 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a800b620 con 0x7f08b80b3140 2024-04-25T00:35:50.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08a6ffd640 1 -- 172.21.15.29:0/288127193 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a8019070 con 0x7f08b80b3140 2024-04-25T00:35:50.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08bceb5640 1 -- 172.21.15.29:0/288127193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b815ca50 con 0x7f08b80b3140 2024-04-25T00:35:50.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08a6ffd640 1 -- 172.21.15.29:0/288127193 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a800b9e0 con 0x7f08b80b3140 2024-04-25T00:35:50.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08bceb5640 1 -- 172.21.15.29:0/288127193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b815d010 con 0x7f08b80b3140 2024-04-25T00:35:50.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.352+0000 7f08bceb5640 1 -- 172.21.15.29:0/288127193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b8069270 con 0x7f08b80b3140 2024-04-25T00:35:50.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.356+0000 7f08a6ffd640 1 -- 172.21.15.29:0/288127193 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f08a800bb80 con 0x7f08b80b3140 2024-04-25T00:35:50.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.356+0000 7f08a6ffd640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0898067c50 0x7f089806a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.356+0000 7f08a6ffd640 1 -- 172.21.15.29:0/288127193 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f08a80865b0 con 0x7f08b80b3140 2024-04-25T00:35:50.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.356+0000 7f08b5d9b640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0898067c50 0x7f089806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.356+0000 7f08b5d9b640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0898067c50 0x7f089806a110 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f08b815af40 tx=0x7f08ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:50.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.360+0000 7f08a6ffd640 1 -- 172.21.15.29:0/288127193 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f08a8056ce0 con 0x7f08b80b3140 2024-04-25T00:35:50.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.524+0000 7f08bceb5640 1 -- 172.21.15.29:0/288127193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08b8144610 con 0x7f08b80b3140 2024-04-25T00:35:50.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.528+0000 7f08a6ffd640 1 -- 172.21.15.29:0/288127193 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16633 (secure 0 0 0) 0x7f08a80565e0 con 0x7f08b80b3140 2024-04-25T00:35:50.530 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:50.530 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":69,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:49.527963+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:47.559043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:48.530288+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:50.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.528+0000 7f08bceb5640 1 -- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0898067c50 msgr2=0x7f089806a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.528+0000 7f08bceb5640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0898067c50 0x7f089806a110 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f08b815af40 tx=0x7f08ac009040 comp rx=0 tx=0).stop 2024-04-25T00:35:50.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.528+0000 7f08bceb5640 1 -- 172.21.15.29:0/288127193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08b80b3140 msgr2=0x7f08b815bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.528+0000 7f08bceb5640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08b80b3140 0x7f08b815bab0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f08a800d620 tx=0x7f08a800dae0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.528+0000 7f08bceb5640 1 -- 172.21.15.29:0/288127193 shutdown_connections 2024-04-25T00:35:50.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.528+0000 7f08bceb5640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0898067c50 0x7f089806a110 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.528+0000 7f08bceb5640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b80b4b10 0x7f08b813a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.528+0000 7f08bceb5640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80b3c30 0x7f08b815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.528+0000 7f08bceb5640 1 --2- 172.21.15.29:0/288127193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08b80b3140 0x7f08b815bab0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.528+0000 7f08bceb5640 1 -- 172.21.15.29:0/288127193 >> 172.21.15.29:0/288127193 conn(0x7f08b80b1870 msgr2=0x7f08b80b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:50.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.528+0000 7f08bceb5640 1 -- 172.21.15.29:0/288127193 shutdown_connections 2024-04-25T00:35:50.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.528+0000 7f08bceb5640 1 -- 172.21.15.29:0/288127193 wait complete. 2024-04-25T00:35:50.550 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:50.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.664+0000 7f7e3a017640 1 Processor -- start 2024-04-25T00:35:50.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.664+0000 7f7e3a017640 1 -- start start 2024-04-25T00:35:50.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e3a017640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e340b3140 0x7f7e3406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e3a017640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7e340b4b10 0x7f7e340b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e3a017640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e340b41e0 con 0x7f7e340b3a60 2024-04-25T00:35:50.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e3a017640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7e3406d960 con 0x7f7e340b4b10 2024-04-25T00:35:50.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e3a017640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7e3406dc40 con 0x7f7e340b3140 2024-04-25T00:35:50.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e32ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7e340b4b10 0x7f7e340b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e32ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7e340b4b10 0x7f7e340b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52314/0 (socket says 172.21.15.29:52314) 2024-04-25T00:35:50.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e32ffd640 1 -- 172.21.15.29:0/2644603634 learned_addr learned my addr 172.21.15.29:0/2644603634 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:50.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e337fe640 1 --2- 172.21.15.29:0/2644603634 >> v2:172.21.15.157:3300/0 conn(0x7f7e340b3140 0x7f7e3406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e32ffd640 1 -- 172.21.15.29:0/2644603634 >> v2:172.21.15.157:3300/0 conn(0x7f7e340b3140 msgr2=0x7f7e3406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:50.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e32ffd640 1 --2- 172.21.15.29:0/2644603634 >> v2:172.21.15.157:3300/0 conn(0x7f7e340b3140 0x7f7e3406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e32ffd640 1 -- 172.21.15.29:0/2644603634 >> v1:172.21.15.157:6789/0 conn(0x7f7e340b3a60 legacy=0x7f7e3415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:50.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e32ffd640 1 -- 172.21.15.29:0/2644603634 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3406df20 con 0x7f7e340b4b10 2024-04-25T00:35:50.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e337fe640 1 --2- 172.21.15.29:0/2644603634 >> v2:172.21.15.157:3300/0 conn(0x7f7e340b3140 0x7f7e3406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:50.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e32ffd640 1 --2- 172.21.15.29:0/2644603634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e340b4b10 0x7f7e340b3520 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009870 tx=0x7f7e2401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c56d216feb744f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:50.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e327fc640 1 -- 172.21.15.29:0/2644603634 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e240b0070 con 0x7f7e340b4b10 2024-04-25T00:35:50.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e327fc640 1 -- 172.21.15.29:0/2644603634 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e240ab600 con 0x7f7e340b4b10 2024-04-25T00:35:50.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.668+0000 7f7e327fc640 1 -- 172.21.15.29:0/2644603634 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e240a4e60 con 0x7f7e340b4b10 2024-04-25T00:35:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 -- 172.21.15.29:0/2644603634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e340b4b10 msgr2=0x7f7e340b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 --2- 172.21.15.29:0/2644603634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e340b4b10 0x7f7e340b3520 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009870 tx=0x7f7e2401c4e0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 -- 172.21.15.29:0/2644603634 shutdown_connections 2024-04-25T00:35:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 --2- 172.21.15.29:0/2644603634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e340b4b10 0x7f7e340b3520 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 --2- 172.21.15.29:0/2644603634 >> v2:172.21.15.157:3300/0 conn(0x7f7e340b3140 0x7f7e3406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 -- 172.21.15.29:0/2644603634 >> 172.21.15.29:0/2644603634 conn(0x7f7e340b1870 msgr2=0x7f7e340b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 -- 172.21.15.29:0/2644603634 shutdown_connections 2024-04-25T00:35:50.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 -- 172.21.15.29:0/2644603634 wait complete. 2024-04-25T00:35:50.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 Processor -- start 2024-04-25T00:35:50.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 -- start start 2024-04-25T00:35:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e340b3140 0x7f7e340ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e340b3a60 0x7f7e34136d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e340b4b10 0x7f7e34137280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7e34168b70 con 0x7f7e340b4b10 2024-04-25T00:35:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7e341689f0 con 0x7f7e340b3140 2024-04-25T00:35:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7e34168cf0 con 0x7f7e340b3a60 2024-04-25T00:35:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e33fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e340b4b10 0x7f7e34137280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e33fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e340b4b10 0x7f7e34137280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52322/0 (socket says 172.21.15.29:52322) 2024-04-25T00:35:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e33fff640 1 -- 172.21.15.29:0/3884134572 learned_addr learned my addr 172.21.15.29:0/3884134572 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e337fe640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e340b3140 0x7f7e340ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e32ffd640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e340b3a60 0x7f7e34136d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e33fff640 1 -- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e340b3a60 msgr2=0x7f7e34136d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e33fff640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e340b3a60 0x7f7e34136d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e33fff640 1 -- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e340b3140 msgr2=0x7f7e340ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e33fff640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e340b3140 0x7f7e340ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e33fff640 1 -- 172.21.15.29:0/3884134572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3413de30 con 0x7f7e340b4b10 2024-04-25T00:35:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e33fff640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e340b4b10 0x7f7e34137280 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00c780 tx=0x7f7e2c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e1bfff640 1 -- 172.21.15.29:0/3884134572 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c004b70 con 0x7f7e340b4b10 2024-04-25T00:35:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e1bfff640 1 -- 172.21.15.29:0/3884134572 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2c004d10 con 0x7f7e340b4b10 2024-04-25T00:35:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 -- 172.21.15.29:0/3884134572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3413e110 con 0x7f7e340b4b10 2024-04-25T00:35:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e1bfff640 1 -- 172.21.15.29:0/3884134572 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c00aa90 con 0x7f7e340b4b10 2024-04-25T00:35:50.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.672+0000 7f7e3a017640 1 -- 172.21.15.29:0/3884134572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3413e6d0 con 0x7f7e340b4b10 2024-04-25T00:35:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.676+0000 7f7e3a017640 1 -- 172.21.15.29:0/3884134572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e34069270 con 0x7f7e340b4b10 2024-04-25T00:35:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.676+0000 7f7e1bfff640 1 -- 172.21.15.29:0/3884134572 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f7e2c00ac30 con 0x7f7e340b4b10 2024-04-25T00:35:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.676+0000 7f7e1bfff640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7e20067c20 0x7f7e2006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.676+0000 7f7e1bfff640 1 -- 172.21.15.29:0/3884134572 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f7e2c086510 con 0x7f7e340b4b10 2024-04-25T00:35:50.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.676+0000 7f7e337fe640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7e20067c20 0x7f7e2006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.676+0000 7f7e337fe640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7e20067c20 0x7f7e2006a0e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7e28006fd0 tx=0x7f7e28008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:50.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.680+0000 7f7e1bfff640 1 -- 172.21.15.29:0/3884134572 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7e2c020070 con 0x7f7e340b4b10 2024-04-25T00:35:50.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.840+0000 7f7e3a017640 1 -- 172.21.15.29:0/3884134572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e3413ee10 con 0x7f7e340b4b10 2024-04-25T00:35:50.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.844+0000 7f7e1bfff640 1 -- 172.21.15.29:0/3884134572 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16633 (secure 0 0 0) 0x7f7e2c056bd0 con 0x7f7e340b4b10 2024-04-25T00:35:50.846 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:50.847 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":69,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:49.527963+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:47.559043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:48.530288+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:50.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.844+0000 7f7e3a017640 1 -- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7e20067c20 msgr2=0x7f7e2006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.844+0000 7f7e3a017640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7e20067c20 0x7f7e2006a0e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7e28006fd0 tx=0x7f7e28008040 comp rx=0 tx=0).stop 2024-04-25T00:35:50.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.844+0000 7f7e3a017640 1 -- 172.21.15.29:0/3884134572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e340b4b10 msgr2=0x7f7e34137280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.844+0000 7f7e3a017640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e340b4b10 0x7f7e34137280 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00c780 tx=0x7f7e2c00cc40 comp rx=0 tx=0).stop 2024-04-25T00:35:50.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.844+0000 7f7e3a017640 1 -- 172.21.15.29:0/3884134572 shutdown_connections 2024-04-25T00:35:50.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.844+0000 7f7e3a017640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e340b4b10 0x7f7e34137280 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.844+0000 7f7e3a017640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e340b3a60 0x7f7e34136d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.844+0000 7f7e3a017640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7e20067c20 0x7f7e2006a0e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.844+0000 7f7e3a017640 1 --2- 172.21.15.29:0/3884134572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e340b3140 0x7f7e340ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.848+0000 7f7e3a017640 1 -- 172.21.15.29:0/3884134572 >> 172.21.15.29:0/3884134572 conn(0x7f7e340b1870 msgr2=0x7f7e34144860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:50.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.848+0000 7f7e3a017640 1 -- 172.21.15.29:0/3884134572 shutdown_connections 2024-04-25T00:35:50.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.848+0000 7f7e3a017640 1 -- 172.21.15.29:0/3884134572 wait complete. 2024-04-25T00:35:50.865 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:35:50.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.988+0000 7fc890857640 1 Processor -- start 2024-04-25T00:35:50.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.988+0000 7fc890857640 1 -- start start 2024-04-25T00:35:50.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc890857640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc88c0b3c30 0x7fc88c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc890857640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc88c0b41e0 con 0x7fc88c0b4b10 2024-04-25T00:35:50.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc890857640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc88c06d960 con 0x7fc88c0b3140 2024-04-25T00:35:50.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc890857640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc88c06dc40 con 0x7fc88c0b3c30 2024-04-25T00:35:50.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88ad9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc88c0b4b10 0x7fc88c0ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35288/0 (socket says 172.21.15.29:35288) 2024-04-25T00:35:50.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88ad9d640 1 -- 172.21.15.29:0/2845119803 learned_addr learned my addr 172.21.15.29:0/2845119803 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:50.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 523223281 0 0) 0x7fc88c0b41e0 con 0x7fc88c0b4b10 2024-04-25T00:35:50.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc889d9b640 1 --2- 172.21.15.29:0/2845119803 >> v2:172.21.15.157:3300/0 conn(0x7fc88c0b3c30 0x7fc88c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc874003530 con 0x7fc88c0b4b10 2024-04-25T00:35:50.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 817746824 0 0) 0x7fc874003530 con 0x7fc88c0b4b10 2024-04-25T00:35:50.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 >> v2:172.21.15.157:3300/0 conn(0x7fc88c0b3c30 msgr2=0x7fc88c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:50.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 --2- 172.21.15.29:0/2845119803 >> v2:172.21.15.157:3300/0 conn(0x7fc88c0b3c30 0x7fc88c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 >> v1:172.21.15.157:6789/0 conn(0x7fc88c0b3140 legacy=0x7fc88c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c06df20 con 0x7fc88c0b4b10 2024-04-25T00:35:50.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fc884001a00 con 0x7fc88c0b4b10 2024-04-25T00:35:50.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 >> v1:172.21.15.29:6789/0 conn(0x7fc88c0b4b10 legacy=0x7fc88c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 --2- 172.21.15.29:0/2845119803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8740039f0 0x7fc874003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 --2- 172.21.15.29:0/2845119803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc874004810 0x7fc874004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 --2- 172.21.15.29:0/2845119803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8740051d0 0x7fc87400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:50.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c06dc40 con 0x7fc8740051d0 2024-04-25T00:35:50.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc87400c0a0 con 0x7fc874004810 2024-04-25T00:35:50.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc87400c2a0 con 0x7fc8740039f0 2024-04-25T00:35:50.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88ad9d640 1 --2- 172.21.15.29:0/2845119803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8740051d0 0x7fc87400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc889d9b640 1 --2- 172.21.15.29:0/2845119803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc874004810 0x7fc874004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88a59c640 1 --2- 172.21.15.29:0/2845119803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8740039f0 0x7fc874003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:50.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88ad9d640 1 -- 172.21.15.29:0/2845119803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8740039f0 msgr2=0x7fc874003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:50.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88ad9d640 1 --2- 172.21.15.29:0/2845119803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8740039f0 0x7fc874003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88ad9d640 1 -- 172.21.15.29:0/2845119803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc874004810 msgr2=0x7fc874004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:50.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88ad9d640 1 --2- 172.21.15.29:0/2845119803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc874004810 0x7fc874004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.992+0000 7fc88ad9d640 1 -- 172.21.15.29:0/2845119803 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc874003530 con 0x7fc8740051d0 2024-04-25T00:35:50.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc88ad9d640 1 -- 172.21.15.29:0/2845119803 reap_dead start 2024-04-25T00:35:50.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc88ad9d640 1 --2- 172.21.15.29:0/2845119803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8740051d0 0x7fc87400b9d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc88400a860 tx=0x7fc88401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2775f7b6c5dcaa07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:50.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88400a480 con 0x7fc8740051d0 2024-04-25T00:35:50.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8840ad630 con 0x7fc8740051d0 2024-04-25T00:35:50.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc88959a640 1 -- 172.21.15.29:0/2845119803 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8840ada20 con 0x7fc8740051d0 2024-04-25T00:35:50.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc890857640 1 -- 172.21.15.29:0/2845119803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8740051d0 msgr2=0x7fc87400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:50.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc890857640 1 --2- 172.21.15.29:0/2845119803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8740051d0 0x7fc87400b9d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc88400a860 tx=0x7fc88401d380 comp rx=0 tx=0).stop 2024-04-25T00:35:50.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc890857640 1 -- 172.21.15.29:0/2845119803 shutdown_connections 2024-04-25T00:35:50.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc890857640 1 --2- 172.21.15.29:0/2845119803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8740051d0 0x7fc87400b9d0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:50.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc890857640 1 -- 172.21.15.29:0/2845119803 >> 172.21.15.29:0/2845119803 conn(0x7fc88c0b1870 msgr2=0x7fc88c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:50.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc890857640 1 -- 172.21.15.29:0/2845119803 shutdown_connections 2024-04-25T00:35:50.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc890857640 1 -- 172.21.15.29:0/2845119803 wait complete. 2024-04-25T00:35:51.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc890857640 1 Processor -- start 2024-04-25T00:35:51.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:50.996+0000 7fc890857640 1 -- start start 2024-04-25T00:35:51.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc890857640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8740051d0 0x7fc88c154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:51.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc890857640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c12d1b0 0x7fc88c12d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:51.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc890857640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc88c13ef40 0x7fc88c13f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:51.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc890857640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c145a90 con 0x7fc8740051d0 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc890857640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c145910 con 0x7fc88c13ef40 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc890857640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc88c06e440 con 0x7fc88c12d1b0 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc88a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8740051d0 0x7fc88c154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc88a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8740051d0 0x7fc88c154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52344/0 (socket says 172.21.15.29:52344) 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc88a59c640 1 -- 172.21.15.29:0/2351934035 learned_addr learned my addr 172.21.15.29:0/2351934035 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc88ad9d640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc88c13ef40 0x7fc88c13f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc889d9b640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c12d1b0 0x7fc88c12d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc88a59c640 1 -- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c12d1b0 msgr2=0x7fc88c12d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc88a59c640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c12d1b0 0x7fc88c12d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc88a59c640 1 -- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc88c13ef40 msgr2=0x7fc88c13f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc88a59c640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc88c13ef40 0x7fc88c13f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc88a59c640 1 -- 172.21.15.29:0/2351934035 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c13d980 con 0x7fc8740051d0 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc88a59c640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8740051d0 0x7fc88c154f00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00e3c0 tx=0x7fc87c00e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:51.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc87affd640 1 -- 172.21.15.29:0/2351934035 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c019070 con 0x7fc8740051d0 2024-04-25T00:35:51.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc87affd640 1 -- 172.21.15.29:0/2351934035 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc87c0052e0 con 0x7fc8740051d0 2024-04-25T00:35:51.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc890857640 1 -- 172.21.15.29:0/2351934035 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88c13dee0 con 0x7fc8740051d0 2024-04-25T00:35:51.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc87affd640 1 -- 172.21.15.29:0/2351934035 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c003820 con 0x7fc8740051d0 2024-04-25T00:35:51.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc890857640 1 -- 172.21.15.29:0/2351934035 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc88c13e460 con 0x7fc8740051d0 2024-04-25T00:35:51.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc890857640 1 -- 172.21.15.29:0/2351934035 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc88c069270 con 0x7fc8740051d0 2024-04-25T00:35:51.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc87affd640 1 -- 172.21.15.29:0/2351934035 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fc87c01c070 con 0x7fc8740051d0 2024-04-25T00:35:51.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc87affd640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc86c067c20 0x7fc86c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:51.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc87affd640 1 -- 172.21.15.29:0/2351934035 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fc87c0856a0 con 0x7fc8740051d0 2024-04-25T00:35:51.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.000+0000 7fc889d9b640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc86c067c20 0x7fc86c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:51.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.004+0000 7fc889d9b640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc86c067c20 0x7fc86c06a0e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc880009a00 tx=0x7fc880008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:51.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.004+0000 7fc87affd640 1 -- 172.21.15.29:0/2351934035 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc87c055780 con 0x7fc8740051d0 2024-04-25T00:35:51.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:51.204+0000 7fc890857640 1 -- 172.21.15.29:0/2351934035 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc88c06ee00 con 0x7fc8740051d0 2024-04-25T00:35:52.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.192+0000 7fc87affd640 1 -- 172.21.15.29:0/2351934035 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) ==== 118+0+0 (secure 0 0 0) 0x7fc87c059a80 con 0x7fc8740051d0 2024-04-25T00:35:52.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.196+0000 7fc890857640 1 -- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc86c067c20 msgr2=0x7fc86c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:52.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.196+0000 7fc890857640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc86c067c20 0x7fc86c06a0e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc880009a00 tx=0x7fc880008040 comp rx=0 tx=0).stop 2024-04-25T00:35:52.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.196+0000 7fc890857640 1 -- 172.21.15.29:0/2351934035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8740051d0 msgr2=0x7fc88c154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:52.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.196+0000 7fc890857640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8740051d0 0x7fc88c154f00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00e3c0 tx=0x7fc87c00e880 comp rx=0 tx=0).stop 2024-04-25T00:35:52.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.196+0000 7fc890857640 1 -- 172.21.15.29:0/2351934035 shutdown_connections 2024-04-25T00:35:52.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.196+0000 7fc890857640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc86c067c20 0x7fc86c06a0e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:52.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.196+0000 7fc890857640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc88c13ef40 0x7fc88c13f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:52.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.196+0000 7fc890857640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc88c12d1b0 0x7fc88c12d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:52.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.196+0000 7fc890857640 1 --2- 172.21.15.29:0/2351934035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8740051d0 0x7fc88c154f00 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:52.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.196+0000 7fc890857640 1 -- 172.21.15.29:0/2351934035 >> 172.21.15.29:0/2351934035 conn(0x7fc88c0b1870 msgr2=0x7fc88c158d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:52.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.196+0000 7fc890857640 1 -- 172.21.15.29:0/2351934035 shutdown_connections 2024-04-25T00:35:52.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.196+0000 7fc890857640 1 -- 172.21.15.29:0/2351934035 wait complete. 2024-04-25T00:35:52.220 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:35:52.220 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:35:52.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fd1c9f640 1 Processor -- start 2024-04-25T00:35:52.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fd1c9f640 1 -- start start 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fd1c9f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8fcc0cf690 0x7f8fcc0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fd1c9f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fcc0d2ea0 con 0x7f8fcc0d2a90 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fd1c9f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fcc0cfc10 con 0x7f8fcc15e200 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fd1c9f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8fcc0b38d0 con 0x7f8fcc0cf690 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fcbfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8fcc0d2a90 0x7f8fcc0b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35298/0 (socket says 172.21.15.29:35298) 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fcbfff640 1 -- 172.21.15.29:0/636146311 learned_addr learned my addr 172.21.15.29:0/636146311 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fcaffd640 1 --2- 172.21.15.29:0/636146311 >> v2:172.21.15.157:3300/0 conn(0x7f8fcc0cf690 0x7f8fcc0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 218672062 0 0) 0x7f8fc40014a0 con 0x7f8fcc0d2a90 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fb4003530 con 0x7f8fcc0d2a90 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 58436374 0 0) 0x7f8fb4003530 con 0x7f8fcc0d2a90 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 >> v2:172.21.15.157:3300/0 conn(0x7f8fcc0cf690 msgr2=0x7f8fcc0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 --2- 172.21.15.29:0/636146311 >> v2:172.21.15.157:3300/0 conn(0x7f8fcc0cf690 0x7f8fcc0d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 >> v1:172.21.15.157:6789/0 conn(0x7f8fcc15e200 legacy=0x7f8fcc0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fcc0cfef0 con 0x7f8fcc0d2a90 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8fc4001bd0 con 0x7f8fcc0d2a90 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 >> v1:172.21.15.29:6789/0 conn(0x7f8fcc0d2a90 legacy=0x7f8fcc0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 --2- 172.21.15.29:0/636146311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fb40039c0 0x7f8fb4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 --2- 172.21.15.29:0/636146311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb4004810 0x7f8fb4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 --2- 172.21.15.29:0/636146311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb40051d0 0x7f8fb400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc0b38d0 con 0x7f8fb4004810 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb400c070 con 0x7f8fb40051d0 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb400c270 con 0x7f8fb40039c0 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fcaffd640 1 --2- 172.21.15.29:0/636146311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb4004810 0x7f8fb4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fcbfff640 1 --2- 172.21.15.29:0/636146311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb40051d0 0x7f8fb400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fcb7fe640 1 --2- 172.21.15.29:0/636146311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fb40039c0 0x7f8fb4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fcaffd640 1 -- 172.21.15.29:0/636146311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fb40039c0 msgr2=0x7f8fb4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fcaffd640 1 --2- 172.21.15.29:0/636146311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fb40039c0 0x7f8fb4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fcaffd640 1 -- 172.21.15.29:0/636146311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb40051d0 msgr2=0x7f8fb400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fcaffd640 1 --2- 172.21.15.29:0/636146311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb40051d0 0x7f8fb400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fcaffd640 1 -- 172.21.15.29:0/636146311 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb4003530 con 0x7f8fb4004810 2024-04-25T00:35:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.352+0000 7f8fcbfff640 1 -- 172.21.15.29:0/636146311 reap_dead start 2024-04-25T00:35:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.356+0000 7f8fcaffd640 1 --2- 172.21.15.29:0/636146311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb4004810 0x7f8fb4004c90 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009ae0 tx=0x7f8fc001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d55c323dde4c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.356+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00b2070 con 0x7f8fb4004810 2024-04-25T00:35:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.356+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc00a5660 con 0x7f8fb4004810 2024-04-25T00:35:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.356+0000 7f8fca7fc640 1 -- 172.21.15.29:0/636146311 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00096b0 con 0x7f8fb4004810 2024-04-25T00:35:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.356+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/636146311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb4004810 msgr2=0x7f8fb4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.356+0000 7f8fd1c9f640 1 --2- 172.21.15.29:0/636146311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb4004810 0x7f8fb4004c90 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009ae0 tx=0x7f8fc001ca70 comp rx=0 tx=0).stop 2024-04-25T00:35:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.356+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/636146311 shutdown_connections 2024-04-25T00:35:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.356+0000 7f8fd1c9f640 1 --2- 172.21.15.29:0/636146311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb4004810 0x7f8fb4004c90 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.356+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/636146311 >> 172.21.15.29:0/636146311 conn(0x7f8fcc09d280 msgr2=0x7f8fcc0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.356+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/636146311 shutdown_connections 2024-04-25T00:35:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.356+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/636146311 wait complete. 2024-04-25T00:35:52.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.356+0000 7f8fd1c9f640 1 Processor -- start 2024-04-25T00:35:52.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.356+0000 7f8fd1c9f640 1 -- start start 2024-04-25T00:35:52.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fd1c9f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fb4004810 0x7f8fcc0b8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:52.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fd1c9f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fcc0c92c0 0x7f8fcc0ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:52.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fd1c9f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fcc0ca5c0 0x7f8fcc1f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:52.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fd1c9f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc15e5e0 con 0x7f8fcc0c92c0 2024-04-25T00:35:52.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fd1c9f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc0b3e00 con 0x7f8fcc0ca5c0 2024-04-25T00:35:52.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fd1c9f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8fcc15e760 con 0x7f8fb4004810 2024-04-25T00:35:52.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fcaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fcc0c92c0 0x7f8fcc0ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:52.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fcaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fcc0c92c0 0x7f8fcc0ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52356/0 (socket says 172.21.15.29:52356) 2024-04-25T00:35:52.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fcaffd640 1 -- 172.21.15.29:0/1665241608 learned_addr learned my addr 172.21.15.29:0/1665241608 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:52.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fcb7fe640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fb4004810 0x7f8fcc0b8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fcaffd640 1 -- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fb4004810 msgr2=0x7f8fcc0b8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fcaffd640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fb4004810 0x7f8fcc0b8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fcbfff640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fcc0ca5c0 0x7f8fcc1f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fcaffd640 1 -- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fcc0ca5c0 msgr2=0x7f8fcc1f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fcaffd640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fcc0ca5c0 0x7f8fcc1f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fcaffd640 1 -- 172.21.15.29:0/1665241608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fcc0caa20 con 0x7f8fcc0c92c0 2024-04-25T00:35:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fcbfff640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fcc0ca5c0 0x7f8fcc1f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fcaffd640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fcc0c92c0 0x7f8fcc0ca080 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8fc001c8a0 tx=0x7f8fc0019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fb3fff640 1 -- 172.21.15.29:0/1665241608 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00b2040 con 0x7f8fcc0c92c0 2024-04-25T00:35:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fb3fff640 1 -- 172.21.15.29:0/1665241608 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc000f000 con 0x7f8fcc0c92c0 2024-04-25T00:35:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/1665241608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fcc0c9b10 con 0x7f8fcc0c92c0 2024-04-25T00:35:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fb3fff640 1 -- 172.21.15.29:0/1665241608 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0004280 con 0x7f8fcc0c92c0 2024-04-25T00:35:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/1665241608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fcc0b8670 con 0x7f8fcc0c92c0 2024-04-25T00:35:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/1665241608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fcc069270 con 0x7f8fcc0c92c0 2024-04-25T00:35:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fb3fff640 1 -- 172.21.15.29:0/1665241608 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f8fc00bf030 con 0x7f8fcc0c92c0 2024-04-25T00:35:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fb3fff640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fb8067c50 0x7f8fb806a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fb3fff640 1 -- 172.21.15.29:0/1665241608 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f8fc0120920 con 0x7f8fcc0c92c0 2024-04-25T00:35:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.360+0000 7f8fcb7fe640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fb8067c50 0x7f8fb806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:52.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.364+0000 7f8fcb7fe640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fb8067c50 0x7f8fb806a110 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc000c10 tx=0x7f8fbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:52.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.364+0000 7f8fb3fff640 1 -- 172.21.15.29:0/1665241608 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8fc00c6060 con 0x7f8fcc0c92c0 2024-04-25T00:35:52.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:52.564+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/1665241608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8fcc0c9e40 con 0x7f8fcc0c92c0 2024-04-25T00:35:53.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.204+0000 7f8fb3fff640 1 -- 172.21.15.29:0/1665241608 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7f8fc0016780 con 0x7f8fcc0c92c0 2024-04-25T00:35:53.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.208+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fb8067c50 msgr2=0x7f8fb806a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.208+0000 7f8fd1c9f640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fb8067c50 0x7f8fb806a110 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc000c10 tx=0x7f8fbc009040 comp rx=0 tx=0).stop 2024-04-25T00:35:53.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.208+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/1665241608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fcc0c92c0 msgr2=0x7f8fcc0ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.208+0000 7f8fd1c9f640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fcc0c92c0 0x7f8fcc0ca080 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8fc001c8a0 tx=0x7f8fc0019810 comp rx=0 tx=0).stop 2024-04-25T00:35:53.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.208+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/1665241608 shutdown_connections 2024-04-25T00:35:53.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.208+0000 7f8fd1c9f640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f8fb8067c50 0x7f8fb806a110 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.208+0000 7f8fd1c9f640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fcc0ca5c0 0x7f8fcc1f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.208+0000 7f8fd1c9f640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fcc0c92c0 0x7f8fcc0ca080 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.208+0000 7f8fd1c9f640 1 --2- 172.21.15.29:0/1665241608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8fb4004810 0x7f8fcc0b8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.208+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/1665241608 >> 172.21.15.29:0/1665241608 conn(0x7f8fcc09d280 msgr2=0x7f8fcc0b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:53.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.208+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/1665241608 shutdown_connections 2024-04-25T00:35:53.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.208+0000 7f8fd1c9f640 1 -- 172.21.15.29:0/1665241608 wait complete. 2024-04-25T00:35:53.234 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:35:53.234 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:35:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fedfd6640 1 Processor -- start 2024-04-25T00:35:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fedfd6640 1 -- start start 2024-04-25T00:35:53.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fedfd6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7fe80cf690 0x7f7fe80d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fedfd6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe80d2ea0 con 0x7f7fe815e200 2024-04-25T00:35:53.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fedfd6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe80cfc10 con 0x7f7fe80d2a90 2024-04-25T00:35:53.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fedfd6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7fe80b38d0 con 0x7f7fe80cf690 2024-04-25T00:35:53.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe77fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7fe815e200 0x7f7fe80d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35314/0 (socket says 172.21.15.29:35314) 2024-04-25T00:35:53.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe77fe640 1 -- 172.21.15.29:0/3624137723 learned_addr learned my addr 172.21.15.29:0/3624137723 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:53.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe6ffd640 1 --2- 172.21.15.29:0/3624137723 >> v2:172.21.15.157:3300/0 conn(0x7f7fe80cf690 0x7f7fe80d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 19033789 0 0) 0x7f7fd8001120 con 0x7f7fe815e200 2024-04-25T00:35:53.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fd4003530 con 0x7f7fe815e200 2024-04-25T00:35:53.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3312684214 0 0) 0x7f7fd4003530 con 0x7f7fe815e200 2024-04-25T00:35:53.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 >> v2:172.21.15.157:3300/0 conn(0x7f7fe80cf690 msgr2=0x7f7fe80d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:53.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 --2- 172.21.15.29:0/3624137723 >> v2:172.21.15.157:3300/0 conn(0x7f7fe80cf690 0x7f7fe80d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 >> v1:172.21.15.157:6789/0 conn(0x7f7fe80d2a90 legacy=0x7f7fe80b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe80cfef0 con 0x7f7fe815e200 2024-04-25T00:35:53.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f7fd8001850 con 0x7f7fe815e200 2024-04-25T00:35:53.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 >> v1:172.21.15.29:6789/0 conn(0x7f7fe815e200 legacy=0x7f7fe80d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 --2- 172.21.15.29:0/3624137723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40039c0 0x7f7fd4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 --2- 172.21.15.29:0/3624137723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fd4004810 0x7f7fd4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 --2- 172.21.15.29:0/3624137723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd40051d0 0x7f7fd400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe80b38d0 con 0x7f7fd40051d0 2024-04-25T00:35:53.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd400c070 con 0x7f7fd40039c0 2024-04-25T00:35:53.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd400c270 con 0x7f7fd4004810 2024-04-25T00:35:53.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe7fff640 1 --2- 172.21.15.29:0/3624137723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd40051d0 0x7f7fd400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe77fe640 1 --2- 172.21.15.29:0/3624137723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40039c0 0x7f7fd4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe6ffd640 1 --2- 172.21.15.29:0/3624137723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fd4004810 0x7f7fd4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe7fff640 1 -- 172.21.15.29:0/3624137723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fd4004810 msgr2=0x7f7fd4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:53.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe7fff640 1 --2- 172.21.15.29:0/3624137723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fd4004810 0x7f7fd4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe7fff640 1 -- 172.21.15.29:0/3624137723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40039c0 msgr2=0x7f7fd4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:53.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe7fff640 1 --2- 172.21.15.29:0/3624137723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40039c0 0x7f7fd4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.364+0000 7f7fe7fff640 1 -- 172.21.15.29:0/3624137723 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd4003530 con 0x7f7fd40051d0 2024-04-25T00:35:53.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fe7fff640 1 -- 172.21.15.29:0/3624137723 reap_dead start 2024-04-25T00:35:53.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fe7fff640 1 --2- 172.21.15.29:0/3624137723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd40051d0 0x7f7fd400b9d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0001780 tx=0x7f7fe001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3793792687ccd22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:53.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe00013a0 con 0x7f7fd40051d0 2024-04-25T00:35:53.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe00a9070 con 0x7f7fd40051d0 2024-04-25T00:35:53.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fe67fc640 1 -- 172.21.15.29:0/3624137723 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0003600 con 0x7f7fd40051d0 2024-04-25T00:35:53.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 -- 172.21.15.29:0/3624137723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd40051d0 msgr2=0x7f7fd400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 --2- 172.21.15.29:0/3624137723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd40051d0 0x7f7fd400b9d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0001780 tx=0x7f7fe001c420 comp rx=0 tx=0).stop 2024-04-25T00:35:53.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 -- 172.21.15.29:0/3624137723 shutdown_connections 2024-04-25T00:35:53.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 --2- 172.21.15.29:0/3624137723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd40051d0 0x7f7fd400b9d0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 -- 172.21.15.29:0/3624137723 >> 172.21.15.29:0/3624137723 conn(0x7f7fe809d280 msgr2=0x7f7fe80d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:53.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 -- 172.21.15.29:0/3624137723 shutdown_connections 2024-04-25T00:35:53.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 -- 172.21.15.29:0/3624137723 wait complete. 2024-04-25T00:35:53.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 Processor -- start 2024-04-25T00:35:53.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 -- start start 2024-04-25T00:35:53.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40051d0 0x7f7fe80b9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fe81f4e50 0x7f7fe81f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe81606f0 0x7f7fe80c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe815e5e0 con 0x7f7fe81f4e50 2024-04-25T00:35:53.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe80b3e00 con 0x7f7fd40051d0 2024-04-25T00:35:53.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fedfd6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe815e760 con 0x7f7fe81606f0 2024-04-25T00:35:53.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fe6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fe81f4e50 0x7f7fe81f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fe6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fe81f4e50 0x7f7fe81f74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52376/0 (socket says 172.21.15.29:52376) 2024-04-25T00:35:53.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.368+0000 7f7fe6ffd640 1 -- 172.21.15.29:0/1768452271 learned_addr learned my addr 172.21.15.29:0/1768452271 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:53.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fe7fff640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe81606f0 0x7f7fe80c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fe77fe640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40051d0 0x7f7fe80b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fe6ffd640 1 -- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe81606f0 msgr2=0x7f7fe80c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fe6ffd640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe81606f0 0x7f7fe80c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fe6ffd640 1 -- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40051d0 msgr2=0x7f7fe80b9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fe6ffd640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40051d0 0x7f7fe80b9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fe6ffd640 1 -- 172.21.15.29:0/1768452271 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe80b6b70 con 0x7f7fe81f4e50 2024-04-25T00:35:53.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fe6ffd640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fe81f4e50 0x7f7fe81f74e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009f10 tx=0x7f7fd00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:53.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fdffff640 1 -- 172.21.15.29:0/1768452271 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd001a020 con 0x7f7fe81f4e50 2024-04-25T00:35:53.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fdffff640 1 -- 172.21.15.29:0/1768452271 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd0016470 con 0x7f7fe81f4e50 2024-04-25T00:35:53.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fedfd6640 1 -- 172.21.15.29:0/1768452271 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe80b9a70 con 0x7f7fe81f4e50 2024-04-25T00:35:53.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fdffff640 1 -- 172.21.15.29:0/1768452271 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0016860 con 0x7f7fe81f4e50 2024-04-25T00:35:53.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fedfd6640 1 -- 172.21.15.29:0/1768452271 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe8160e90 con 0x7f7fe81f4e50 2024-04-25T00:35:53.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fedfd6640 1 -- 172.21.15.29:0/1768452271 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe8069270 con 0x7f7fe81f4e50 2024-04-25T00:35:53.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fdffff640 1 -- 172.21.15.29:0/1768452271 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f7fd001d020 con 0x7f7fe81f4e50 2024-04-25T00:35:53.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fdffff640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7fc8067c20 0x7f7fc806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fdffff640 1 -- 172.21.15.29:0/1768452271 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f7fd0087040 con 0x7f7fe81f4e50 2024-04-25T00:35:53.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fe77fe640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7fc8067c20 0x7f7fc806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.372+0000 7f7fe77fe640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7fc8067c20 0x7f7fc806a0e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80012c0 tx=0x7f7fd8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:53.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.376+0000 7f7fdffff640 1 -- 172.21.15.29:0/1768452271 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7fd0024030 con 0x7f7fe81f4e50 2024-04-25T00:35:53.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.576+0000 7f7fedfd6640 1 -- 172.21.15.29:0/1768452271 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fe815d6d0 con 0x7f7fe81f4e50 2024-04-25T00:35:53.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.576+0000 7f7fdffff640 1 -- 172.21.15.29:0/1768452271 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5186 (secure 0 0 0) 0x7f7fd00576f0 con 0x7f7fe81f4e50 2024-04-25T00:35:53.580 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:53.580 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/524224365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":524224365},{"type":"v1","addr":"172.21.15.157:6835","nonce":524224365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1007000548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1007000548},{"type":"v1","addr":"172.21.15.157:6837","nonce":1007000548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6810,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2091414875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2091414875},{"type":"v1","addr":"172.21.15.29:6833","nonce":2091414875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:49.528297+0000","modified":"2024-04-25T00:35:52.569008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1111146974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1111146974},{"type":"v1","addr":"172.21.15.29:6835","nonce":1111146974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6822,"qdb_cluster":[6822]},"id":5}]} 2024-04-25T00:35:53.580 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 43 2024-04-25T00:35:53.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.580+0000 7f7fedfd6640 1 -- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7fc8067c20 msgr2=0x7f7fc806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.580+0000 7f7fedfd6640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7fc8067c20 0x7f7fc806a0e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80012c0 tx=0x7f7fd8009040 comp rx=0 tx=0).stop 2024-04-25T00:35:53.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.580+0000 7f7fedfd6640 1 -- 172.21.15.29:0/1768452271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fe81f4e50 msgr2=0x7f7fe81f74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.580+0000 7f7fedfd6640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fe81f4e50 0x7f7fe81f74e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009f10 tx=0x7f7fd00115a0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.580+0000 7f7fedfd6640 1 -- 172.21.15.29:0/1768452271 shutdown_connections 2024-04-25T00:35:53.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.580+0000 7f7fedfd6640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fe81606f0 0x7f7fe80c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.580+0000 7f7fedfd6640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fe81f4e50 0x7f7fe81f74e0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.580+0000 7f7fedfd6640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7fc8067c20 0x7f7fc806a0e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.580+0000 7f7fedfd6640 1 --2- 172.21.15.29:0/1768452271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40051d0 0x7f7fe80b9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.580+0000 7f7fedfd6640 1 -- 172.21.15.29:0/1768452271 >> 172.21.15.29:0/1768452271 conn(0x7f7fe809d280 msgr2=0x7f7fe80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:53.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.580+0000 7f7fedfd6640 1 -- 172.21.15.29:0/1768452271 shutdown_connections 2024-04-25T00:35:53.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.580+0000 7f7fedfd6640 1 -- 172.21.15.29:0/1768452271 wait complete. 2024-04-25T00:35:53.597 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:35:53.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.704+0000 7f4fa17fb640 1 Processor -- start 2024-04-25T00:35:53.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.704+0000 7f4fa17fb640 1 -- start start 2024-04-25T00:35:53.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4fa17fb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4f9c0d4bf0 0x7f4f9c0d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4fa17fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c15e8c0 0x7f4f9c0b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4fa17fb640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f9c0cfe30 con 0x7f4f9c15e4e0 2024-04-25T00:35:53.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4fa17fb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4f9c0b6f80 con 0x7f4f9c0d4bf0 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4fa17fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4f9c0b7260 con 0x7f4f9c15e8c0 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9affd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4f9c15e4e0 0x7f4f9c0d1eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35318/0 (socket says 172.21.15.29:35318) 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9affd640 1 -- 172.21.15.29:0/3296049767 learned_addr learned my addr 172.21.15.29:0/3296049767 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9b7fe640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c15e8c0 0x7f4f9c0b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9a7fc640 1 --2- 172.21.15.29:0/3296049767 >> v2:172.21.15.157:3300/0 conn(0x7f4f9c0d4bf0 0x7f4f9c0d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3646284736 0 0) 0x7f4f9c0cfe30 con 0x7f4f9c15e4e0 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f84003530 con 0x7f4f9c15e4e0 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2312767929 0 0) 0x7f4f84003530 con 0x7f4f9c15e4e0 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c15e8c0 msgr2=0x7f4f9c0b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c15e8c0 0x7f4f9c0b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 >> v2:172.21.15.157:3300/0 conn(0x7f4f9c0d4bf0 msgr2=0x7f4f9c0d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 --2- 172.21.15.29:0/3296049767 >> v2:172.21.15.157:3300/0 conn(0x7f4f9c0d4bf0 0x7f4f9c0d25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9c0ced50 con 0x7f4f9c15e4e0 2024-04-25T00:35:53.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4f8c001680 con 0x7f4f9c15e4e0 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 >> v1:172.21.15.29:6789/0 conn(0x7f4f9c15e4e0 legacy=0x7f4f9c0d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f840039c0 0x7f4f84003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f84004810 0x7f4f84004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c15e8c0 0x7f4f840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c0b6f80 con 0x7f4f84004810 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c0b7260 con 0x7f4f840039c0 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4f8400b9d0 con 0x7f4f9c15e8c0 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9a7fc640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f84004810 0x7f4f84004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9affd640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f840039c0 0x7f4f84003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9b7fe640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c15e8c0 0x7f4f840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9a7fc640 1 -- 172.21.15.29:0/3296049767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c15e8c0 msgr2=0x7f4f840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9a7fc640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c15e8c0 0x7f4f840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9a7fc640 1 -- 172.21.15.29:0/3296049767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f840039c0 msgr2=0x7f4f84003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:53.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9a7fc640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f840039c0 0x7f4f84003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9a7fc640 1 -- 172.21.15.29:0/3296049767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f84003530 con 0x7f4f84004810 2024-04-25T00:35:53.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9affd640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f840039c0 0x7f4f84003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:53.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.708+0000 7f4f9b7fe640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c15e8c0 0x7f4f840051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:53.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4f9a7fc640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f84004810 0x7f4f84004c90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009ae0 tx=0x7f4f9001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329dc942df9a2d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:53.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f900b2070 con 0x7f4f84004810 2024-04-25T00:35:53.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f900a55c0 con 0x7f4f84004810 2024-04-25T00:35:53.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4f99ffb640 1 -- 172.21.15.29:0/3296049767 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f900096b0 con 0x7f4f84004810 2024-04-25T00:35:53.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3296049767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f84004810 msgr2=0x7f4f84004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 --2- 172.21.15.29:0/3296049767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f84004810 0x7f4f84004c90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009ae0 tx=0x7f4f9001cab0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4f9b7fe640 1 -- 172.21.15.29:0/3296049767 reap_dead start 2024-04-25T00:35:53.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3296049767 shutdown_connections 2024-04-25T00:35:53.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3296049767 >> 172.21.15.29:0/3296049767 conn(0x7f4f9c0d3fb0 msgr2=0x7f4f9c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:53.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3296049767 shutdown_connections 2024-04-25T00:35:53.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3296049767 wait complete. 2024-04-25T00:35:53.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 Processor -- start 2024-04-25T00:35:53.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 -- start start 2024-04-25T00:35:53.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c155670 0x7f4f9c0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c1617e0 0x7f4f9c15ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f9c160500 0x7f4f9c1f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c0b7900 con 0x7f4f9c160500 2024-04-25T00:35:53.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c0b7780 con 0x7f4f9c155670 2024-04-25T00:35:53.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4fa17fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4f9c0b7a80 con 0x7f4f9c1617e0 2024-04-25T00:35:53.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4f9b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f9c160500 0x7f4f9c1f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4f9b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f9c160500 0x7f4f9c1f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52400/0 (socket says 172.21.15.29:52400) 2024-04-25T00:35:53.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.712+0000 7f4f9b7fe640 1 -- 172.21.15.29:0/3503119775 learned_addr learned my addr 172.21.15.29:0/3503119775 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:53.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f9a7fc640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c1617e0 0x7f4f9c15ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f9affd640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c155670 0x7f4f9c0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f9b7fe640 1 -- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c1617e0 msgr2=0x7f4f9c15ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f9b7fe640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c1617e0 0x7f4f9c15ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f9b7fe640 1 -- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c155670 msgr2=0x7f4f9c0cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f9b7fe640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c155670 0x7f4f9c0cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f9b7fe640 1 -- 172.21.15.29:0/3503119775 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9c15ef20 con 0x7f4f9c160500 2024-04-25T00:35:53.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f9b7fe640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f9c160500 0x7f4f9c1f5080 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c1621e0 tx=0x7f4f9400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:53.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f837fe640 1 -- 172.21.15.29:0/3503119775 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9401c070 con 0x7f4f9c160500 2024-04-25T00:35:53.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f837fe640 1 -- 172.21.15.29:0/3503119775 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9400fe90 con 0x7f4f9c160500 2024-04-25T00:35:53.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3503119775 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9c15f200 con 0x7f4f9c160500 2024-04-25T00:35:53.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f837fe640 1 -- 172.21.15.29:0/3503119775 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94017620 con 0x7f4f9c160500 2024-04-25T00:35:53.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3503119775 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f9c15f7c0 con 0x7f4f9c160500 2024-04-25T00:35:53.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3503119775 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f9c069270 con 0x7f4f9c160500 2024-04-25T00:35:53.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f837fe640 1 -- 172.21.15.29:0/3503119775 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f4f9401f070 con 0x7f4f9c160500 2024-04-25T00:35:53.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f837fe640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f88067c20 0x7f4f8806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:53.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f837fe640 1 -- 172.21.15.29:0/3503119775 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f4f940862b0 con 0x7f4f9c160500 2024-04-25T00:35:53.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.716+0000 7f4f9affd640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f88067c20 0x7f4f8806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:53.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.720+0000 7f4f9affd640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f88067c20 0x7f4f8806a0e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c001930 tx=0x7f4f8c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:53.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.720+0000 7f4f837fe640 1 -- 172.21.15.29:0/3503119775 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4f94012070 con 0x7f4f9c160500 2024-04-25T00:35:53.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.884+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3503119775 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f9c0751b0 con 0x7f4f9c160500 2024-04-25T00:35:53.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.888+0000 7f4f837fe640 1 -- 172.21.15.29:0/3503119775 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16620 (secure 0 0 0) 0x7f4f940568d0 con 0x7f4f9c160500 2024-04-25T00:35:53.889 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:53.889 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":70,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:51.187302+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:47.559043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:48.530288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:35:53.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.888+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f88067c20 msgr2=0x7f4f8806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.888+0000 7f4fa17fb640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f88067c20 0x7f4f8806a0e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c001930 tx=0x7f4f8c009040 comp rx=0 tx=0).stop 2024-04-25T00:35:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.888+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3503119775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f9c160500 msgr2=0x7f4f9c1f5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.888+0000 7f4fa17fb640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f9c160500 0x7f4f9c1f5080 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c1621e0 tx=0x7f4f9400f4a0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.888+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3503119775 shutdown_connections 2024-04-25T00:35:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.888+0000 7f4fa17fb640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f9c160500 0x7f4f9c1f5080 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.888+0000 7f4fa17fb640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f9c1617e0 0x7f4f9c15ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.888+0000 7f4fa17fb640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f88067c20 0x7f4f8806a0e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.888+0000 7f4fa17fb640 1 --2- 172.21.15.29:0/3503119775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c155670 0x7f4f9c0cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.888+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3503119775 >> 172.21.15.29:0/3503119775 conn(0x7f4f9c0d3fb0 msgr2=0x7f4f9c15d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.888+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3503119775 shutdown_connections 2024-04-25T00:35:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:53.888+0000 7f4fa17fb640 1 -- 172.21.15.29:0/3503119775 wait complete. 2024-04-25T00:35:53.910 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:35:53.910 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:35:54.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1c4a9640 1 Processor -- start 2024-04-25T00:35:54.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1c4a9640 1 -- start start 2024-04-25T00:35:54.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1c4a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1406b210 0x7fac1406b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1c4a9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fac1415c470 0x7fac1415c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1c4a9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac1415d400 con 0x7fac141458c0 2024-04-25T00:35:54.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1c4a9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fac1406bbb0 con 0x7fac1415c470 2024-04-25T00:35:54.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1c4a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac1415d120 con 0x7fac1406b210 2024-04-25T00:35:54.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1a245640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fac141458c0 0x7fac140cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35322/0 (socket says 172.21.15.29:35322) 2024-04-25T00:35:54.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1a245640 1 -- 172.21.15.29:0/875958274 learned_addr learned my addr 172.21.15.29:0/875958274 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:54.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1aa46640 1 --2- 172.21.15.29:0/875958274 >> v2:172.21.15.29:3300/0 conn(0x7fac1415c470 0x7fac1415c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac19a44640 1 --2- 172.21.15.29:0/875958274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1406b210 0x7fac1406b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac19243640 1 -- 172.21.15.29:0/875958274 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3702773920 0 0) 0x7fac1415d400 con 0x7fac141458c0 2024-04-25T00:35:54.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac19243640 1 -- 172.21.15.29:0/875958274 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac00003530 con 0x7fac141458c0 2024-04-25T00:35:54.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1aa46640 1 -- 172.21.15.29:0/875958274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1406b210 msgr2=0x7fac1406b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1aa46640 1 --2- 172.21.15.29:0/875958274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1406b210 0x7fac1406b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1aa46640 1 -- 172.21.15.29:0/875958274 >> v1:172.21.15.29:6789/0 conn(0x7fac141458c0 legacy=0x7fac140cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.028+0000 7fac1aa46640 1 -- 172.21.15.29:0/875958274 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac140ce930 con 0x7fac1415c470 2024-04-25T00:35:54.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1aa46640 1 --2- 172.21.15.29:0/875958274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac1415c470 0x7fac1415c900 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009d90 tx=0x7fac0c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8b94721751fe28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac19243640 1 -- 172.21.15.29:0/875958274 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0b2070 con 0x7fac1415c470 2024-04-25T00:35:54.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac19243640 1 -- 172.21.15.29:0/875958274 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac0c0a5830 con 0x7fac1415c470 2024-04-25T00:35:54.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac19243640 1 -- 172.21.15.29:0/875958274 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0a5c00 con 0x7fac1415c470 2024-04-25T00:35:54.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 -- 172.21.15.29:0/875958274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac1415c470 msgr2=0x7fac1415c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 --2- 172.21.15.29:0/875958274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac1415c470 0x7fac1415c900 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009d90 tx=0x7fac0c01c970 comp rx=0 tx=0).stop 2024-04-25T00:35:54.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 -- 172.21.15.29:0/875958274 shutdown_connections 2024-04-25T00:35:54.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 --2- 172.21.15.29:0/875958274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac1415c470 0x7fac1415c900 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 --2- 172.21.15.29:0/875958274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1406b210 0x7fac1406b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 -- 172.21.15.29:0/875958274 >> 172.21.15.29:0/875958274 conn(0x7fac140b2920 msgr2=0x7fac140b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:54.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 -- 172.21.15.29:0/875958274 shutdown_connections 2024-04-25T00:35:54.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 -- 172.21.15.29:0/875958274 wait complete. 2024-04-25T00:35:54.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 Processor -- start 2024-04-25T00:35:54.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 -- start start 2024-04-25T00:35:54.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1406b210 0x7fac14159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac141458c0 0x7fac140add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac1415c470 0x7fac140b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac140ccf90 con 0x7fac141458c0 2024-04-25T00:35:54.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac140cfb20 con 0x7fac1415c470 2024-04-25T00:35:54.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1c4a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac140cd110 con 0x7fac1406b210 2024-04-25T00:35:54.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac19a44640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac141458c0 0x7fac140add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac19a44640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac141458c0 0x7fac140add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52420/0 (socket says 172.21.15.29:52420) 2024-04-25T00:35:54.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac19a44640 1 -- 172.21.15.29:0/2741693861 learned_addr learned my addr 172.21.15.29:0/2741693861 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:54.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1a245640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1406b210 0x7fac14159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1aa46640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac1415c470 0x7fac140b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac19a44640 1 -- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1406b210 msgr2=0x7fac14159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac19a44640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1406b210 0x7fac14159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac19a44640 1 -- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac1415c470 msgr2=0x7fac140b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac19a44640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac1415c470 0x7fac140b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac19a44640 1 -- 172.21.15.29:0/2741693861 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac140b1a40 con 0x7fac141458c0 2024-04-25T00:35:54.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac1aa46640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac1415c470 0x7fac140b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:54.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.032+0000 7fac19a44640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac141458c0 0x7fac140add70 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fac0400d620 tx=0x7fac0400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.036+0000 7fac0affd640 1 -- 172.21.15.29:0/2741693861 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0400b620 con 0x7fac141458c0 2024-04-25T00:35:54.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.036+0000 7fac0affd640 1 -- 172.21.15.29:0/2741693861 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac04019070 con 0x7fac141458c0 2024-04-25T00:35:54.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.036+0000 7fac1c4a9640 1 -- 172.21.15.29:0/2741693861 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac140b1d20 con 0x7fac141458c0 2024-04-25T00:35:54.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.036+0000 7fac0affd640 1 -- 172.21.15.29:0/2741693861 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0400b9e0 con 0x7fac141458c0 2024-04-25T00:35:54.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.036+0000 7fac1c4a9640 1 -- 172.21.15.29:0/2741693861 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac14159e90 con 0x7fac141458c0 2024-04-25T00:35:54.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.036+0000 7fac1c4a9640 1 -- 172.21.15.29:0/2741693861 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac1415a180 con 0x7fac141458c0 2024-04-25T00:35:54.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.036+0000 7fac0affd640 1 -- 172.21.15.29:0/2741693861 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fac0400bb80 con 0x7fac141458c0 2024-04-25T00:35:54.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.036+0000 7fac0affd640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fabf8067c50 0x7fabf806a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.036+0000 7fac0affd640 1 -- 172.21.15.29:0/2741693861 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fac04086540 con 0x7fac141458c0 2024-04-25T00:35:54.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.036+0000 7fac1a245640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fabf8067c50 0x7fabf806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.036+0000 7fac1a245640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fabf8067c50 0x7fabf806a110 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fac10007fd0 tx=0x7fac10009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.040+0000 7fac0affd640 1 -- 172.21.15.29:0/2741693861 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fac04056d00 con 0x7fac141458c0 2024-04-25T00:35:54.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.200+0000 7fac1c4a9640 1 -- 172.21.15.29:0/2741693861 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac14069270 con 0x7fabf8067c50 2024-04-25T00:35:54.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.216+0000 7fac0affd640 1 -- 172.21.15.29:0/2741693861 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240184 (secure 0 0 0) 0x7fac14069270 con 0x7fabf8067c50 2024-04-25T00:35:54.223 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:54.224 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:35:54.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.220+0000 7fac1c4a9640 1 -- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fabf8067c50 msgr2=0x7fabf806a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.220+0000 7fac1c4a9640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fabf8067c50 0x7fabf806a110 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fac10007fd0 tx=0x7fac10009040 comp rx=0 tx=0).stop 2024-04-25T00:35:54.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.220+0000 7fac1c4a9640 1 -- 172.21.15.29:0/2741693861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac141458c0 msgr2=0x7fac140add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.220+0000 7fac1c4a9640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac141458c0 0x7fac140add70 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fac0400d620 tx=0x7fac0400dae0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.220+0000 7fac1c4a9640 1 -- 172.21.15.29:0/2741693861 shutdown_connections 2024-04-25T00:35:54.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.220+0000 7fac1c4a9640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fabf8067c50 0x7fabf806a110 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.220+0000 7fac1c4a9640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac1415c470 0x7fac140b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.220+0000 7fac1c4a9640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac141458c0 0x7fac140add70 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.220+0000 7fac1c4a9640 1 --2- 172.21.15.29:0/2741693861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac1406b210 0x7fac14159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.220+0000 7fac1c4a9640 1 -- 172.21.15.29:0/2741693861 >> 172.21.15.29:0/2741693861 conn(0x7fac140b2920 msgr2=0x7fac1415e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:54.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.220+0000 7fac1c4a9640 1 -- 172.21.15.29:0/2741693861 shutdown_connections 2024-04-25T00:35:54.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.220+0000 7fac1c4a9640 1 -- 172.21.15.29:0/2741693861 wait complete. 2024-04-25T00:35:54.240 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-04-25T00:35:53.536245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277764,"kb_used_data":7468,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754696956,"statfs":{"total":773094113280,"available":772809682944,"internally_reserved":0,"allocated":7647232,"data_stored":3871475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.392058"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.154333+0000","last_change":"2024-04-25T00:35:48.154333+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.154333+0000","last_undegraded":"2024-04-25T00:35:48.154333+0000","last_fullsized":"2024-04-25T00:35:48.154333+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152222+0000","last_change":"2024-04-25T00:35:48.152222+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152222+0000","last_undegraded":"2024-04-25T00:35:48.152222+0000","last_fullsized":"2024-04-25T00:35:48.152222+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147865+0000","last_change":"2024-04-25T00:35:48.147865+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147865+0000","last_undegraded":"2024-04-25T00:35:48.147865+0000","last_fullsized":"2024-04-25T00:35:48.147865+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152879+0000","last_change":"2024-04-25T00:35:48.152879+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152879+0000","last_undegraded":"2024-04-25T00:35:48.152879+0000","last_fullsized":"2024-04-25T00:35:48.152879+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.151577+0000","last_change":"2024-04-25T00:35:48.151577+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.151577+0000","last_undegraded":"2024-04-25T00:35:48.151577+0000","last_fullsized":"2024-04-25T00:35:48.151577+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.155134+0000","last_change":"2024-04-25T00:35:48.155134+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.155134+0000","last_undegraded":"2024-04-25T00:35:48.155134+0000","last_fullsized":"2024-04-25T00:35:48.155134+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.148179+0000","last_change":"2024-04-25T00:35:48.148179+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.148179+0000","last_undegraded":"2024-04-25T00:35:48.148179+0000","last_fullsized":"2024-04-25T00:35:48.148179+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.150610+0000","last_change":"2024-04-25T00:35:48.150610+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.150610+0000","last_undegraded":"2024-04-25T00:35:48.150610+0000","last_fullsized":"2024-04-25T00:35:48.150610+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147914+0000","last_change":"2024-04-25T00:35:48.147914+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147914+0000","last_undegraded":"2024-04-25T00:35:48.147914+0000","last_fullsized":"2024-04-25T00:35:48.147914+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152776+0000","last_change":"2024-04-25T00:35:48.152776+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152776+0000","last_undegraded":"2024-04-25T00:35:48.152776+0000","last_fullsized":"2024-04-25T00:35:48.152776+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152911+0000","last_change":"2024-04-25T00:35:48.152911+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152911+0000","last_undegraded":"2024-04-25T00:35:48.152911+0000","last_fullsized":"2024-04-25T00:35:48.152911+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147856+0000","last_change":"2024-04-25T00:35:48.147856+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147856+0000","last_undegraded":"2024-04-25T00:35:48.147856+0000","last_fullsized":"2024-04-25T00:35:48.147856+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147891+0000","last_change":"2024-04-25T00:35:48.147891+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147891+0000","last_undegraded":"2024-04-25T00:35:48.147891+0000","last_fullsized":"2024-04-25T00:35:48.147891+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.154123+0000","last_change":"2024-04-25T00:35:48.154123+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.154123+0000","last_undegraded":"2024-04-25T00:35:48.154123+0000","last_fullsized":"2024-04-25T00:35:48.154123+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.149062+0000","last_change":"2024-04-25T00:35:48.149062+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.149062+0000","last_undegraded":"2024-04-25T00:35:48.149062+0000","last_fullsized":"2024-04-25T00:35:48.149062+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147937+0000","last_change":"2024-04-25T00:35:48.147937+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147937+0000","last_undegraded":"2024-04-25T00:35:48.147937+0000","last_fullsized":"2024-04-25T00:35:48.147937+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.159198+0000","last_change":"2024-04-25T00:35:48.159198+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.159198+0000","last_undegraded":"2024-04-25T00:35:48.159198+0000","last_fullsized":"2024-04-25T00:35:48.159198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.149376+0000","last_change":"2024-04-25T00:35:48.149376+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.149376+0000","last_undegraded":"2024-04-25T00:35:48.149376+0000","last_fullsized":"2024-04-25T00:35:48.149376+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.153859+0000","last_change":"2024-04-25T00:35:48.153859+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.153859+0000","last_undegraded":"2024-04-25T00:35:48.153859+0000","last_fullsized":"2024-04-25T00:35:48.153859+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.155771+0000","last_change":"2024-04-25T00:35:48.155771+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.155771+0000","last_undegraded":"2024-04-25T00:35:48.155771+0000","last_fullsized":"2024-04-25T00:35:48.155771+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152611+0000","last_change":"2024-04-25T00:35:48.152611+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152611+0000","last_undegraded":"2024-04-25T00:35:48.152611+0000","last_fullsized":"2024-04-25T00:35:48.152611+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147859+0000","last_change":"2024-04-25T00:35:48.147859+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147859+0000","last_undegraded":"2024-04-25T00:35:48.147859+0000","last_fullsized":"2024-04-25T00:35:48.147859+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152521+0000","last_change":"2024-04-25T00:35:48.152521+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152521+0000","last_undegraded":"2024-04-25T00:35:48.152521+0000","last_fullsized":"2024-04-25T00:35:48.152521+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.158091+0000","last_change":"2024-04-25T00:35:48.158091+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.158091+0000","last_undegraded":"2024-04-25T00:35:48.158091+0000","last_fullsized":"2024-04-25T00:35:48.158091+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.150268+0000","last_change":"2024-04-25T00:35:48.150268+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.150268+0000","last_undegraded":"2024-04-25T00:35:48.150268+0000","last_fullsized":"2024-04-25T00:35:48.150268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.151198+0000","last_change":"2024-04-25T00:35:48.151198+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.151198+0000","last_undegraded":"2024-04-25T00:35:48.151198+0000","last_fullsized":"2024-04-25T00:35:48.151198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152307+0000","last_change":"2024-04-25T00:35:48.152307+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152307+0000","last_undegraded":"2024-04-25T00:35:48.152307+0000","last_fullsized":"2024-04-25T00:35:48.152307+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.150478+0000","last_change":"2024-04-25T00:35:48.150478+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.150478+0000","last_undegraded":"2024-04-25T00:35:48.150478+0000","last_fullsized":"2024-04-25T00:35:48.150478+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147602+0000","last_change":"2024-04-25T00:35:48.147602+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147602+0000","last_undegraded":"2024-04-25T00:35:48.147602+0000","last_fullsized":"2024-04-25T00:35:48.147602+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.149920+0000","last_change":"2024-04-25T00:35:48.149920+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.149920+0000","last_undegraded":"2024-04-25T00:35:48.149920+0000","last_fullsized":"2024-04-25T00:35:48.149920+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.153712+0000","last_change":"2024-04-25T00:35:48.153712+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.153712+0000","last_undegraded":"2024-04-25T00:35:48.153712+0000","last_fullsized":"2024-04-25T00:35:48.153712+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152982+0000","last_change":"2024-04-25T00:35:48.152982+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152982+0000","last_undegraded":"2024-04-25T00:35:48.152982+0000","last_fullsized":"2024-04-25T00:35:48.152982+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":260,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T00:35:48.155850+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:35:48.155850+0000","last_peered":"2024-04-25T00:35:48.155850+0000","last_clean":"2024-04-25T00:35:48.155850+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:35:48.155850+0000","last_undegraded":"2024-04-25T00:35:48.155850+0000","last_fullsized":"2024-04-25T00:35:48.155850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1396736,"data_stored":924529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":339978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":802816,"data_stored":334161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":802816,"data_stored":334161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36140,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335700,"statfs":{"total":96636764160,"available":96599756800,"internally_reserved":0,"allocated":815104,"data_stored":339978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36128,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335712,"statfs":{"total":96636764160,"available":96599769088,"internally_reserved":0,"allocated":802816,"data_stored":334161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":802816,"data_stored":334161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36724,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335116,"statfs":{"total":96636764160,"available":96599158784,"internally_reserved":0,"allocated":1409024,"data_stored":930346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:35:54.245 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:35:54.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.396+0000 7fbff889a640 1 Processor -- start 2024-04-25T00:35:54.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.396+0000 7fbff889a640 1 -- start start 2024-04-25T00:35:54.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.396+0000 7fbff889a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff40b3140 0x7fbff406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.396+0000 7fbff889a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbff40b4b10 0x7fbff40b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.396+0000 7fbff889a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbff40b41e0 con 0x7fbff40b3a60 2024-04-25T00:35:54.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.396+0000 7fbff889a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbff406d960 con 0x7fbff40b4b10 2024-04-25T00:35:54.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.396+0000 7fbff889a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbff406dc40 con 0x7fbff40b3140 2024-04-25T00:35:54.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff1d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbff40b4b10 0x7fbff40b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff1d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbff40b4b10 0x7fbff40b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52422/0 (socket says 172.21.15.29:52422) 2024-04-25T00:35:54.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff1d9b640 1 -- 172.21.15.29:0/3640248700 learned_addr learned my addr 172.21.15.29:0/3640248700 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:54.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff259c640 1 --2- 172.21.15.29:0/3640248700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff40b3140 0x7fbff406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff1d9b640 1 -- 172.21.15.29:0/3640248700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff40b3140 msgr2=0x7fbff406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff1d9b640 1 --2- 172.21.15.29:0/3640248700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff40b3140 0x7fbff406e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff1d9b640 1 -- 172.21.15.29:0/3640248700 >> v1:172.21.15.157:6789/0 conn(0x7fbff40b3a60 legacy=0x7fbff415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff1d9b640 1 -- 172.21.15.29:0/3640248700 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff406df20 con 0x7fbff40b4b10 2024-04-25T00:35:54.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff1d9b640 1 --2- 172.21.15.29:0/3640248700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff40b4b10 0x7fbff40b3520 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009e50 tx=0x7fbfe401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f8e8ce7115ffd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff159a640 1 -- 172.21.15.29:0/3640248700 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40a5560 con 0x7fbff40b4b10 2024-04-25T00:35:54.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff159a640 1 -- 172.21.15.29:0/3640248700 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe40a5700 con 0x7fbff40b4b10 2024-04-25T00:35:54.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff159a640 1 -- 172.21.15.29:0/3640248700 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40a5ad0 con 0x7fbff40b4b10 2024-04-25T00:35:54.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff889a640 1 -- 172.21.15.29:0/3640248700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff40b4b10 msgr2=0x7fbff40b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff889a640 1 --2- 172.21.15.29:0/3640248700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff40b4b10 0x7fbff40b3520 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009e50 tx=0x7fbfe401c9f0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff889a640 1 -- 172.21.15.29:0/3640248700 shutdown_connections 2024-04-25T00:35:54.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff889a640 1 --2- 172.21.15.29:0/3640248700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff40b4b10 0x7fbff40b3520 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff889a640 1 --2- 172.21.15.29:0/3640248700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff40b3140 0x7fbff406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff889a640 1 -- 172.21.15.29:0/3640248700 >> 172.21.15.29:0/3640248700 conn(0x7fbff40b1870 msgr2=0x7fbff40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:54.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff889a640 1 -- 172.21.15.29:0/3640248700 shutdown_connections 2024-04-25T00:35:54.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.400+0000 7fbff889a640 1 -- 172.21.15.29:0/3640248700 wait complete. 2024-04-25T00:35:54.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff889a640 1 Processor -- start 2024-04-25T00:35:54.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff889a640 1 -- start start 2024-04-25T00:35:54.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff889a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff40b3140 0x7fbff41596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff889a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff40b3a60 0x7fbff4135410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff889a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff40b4b10 0x7fbff4135950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff889a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbff4168b70 con 0x7fbff40b4b10 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff889a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbff41689f0 con 0x7fbff40b3140 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff889a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbff4168cf0 con 0x7fbff40b3a60 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff40b4b10 0x7fbff4135950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff40b4b10 0x7fbff4135950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52434/0 (socket says 172.21.15.29:52434) 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff2d9d640 1 -- 172.21.15.29:0/3316388202 learned_addr learned my addr 172.21.15.29:0/3316388202 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff259c640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff40b3140 0x7fbff41596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff1d9b640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff40b3a60 0x7fbff4135410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff2d9d640 1 -- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff40b3a60 msgr2=0x7fbff4135410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff2d9d640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff40b3a60 0x7fbff4135410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff2d9d640 1 -- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff40b3140 msgr2=0x7fbff41596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff2d9d640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff40b3140 0x7fbff41596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff2d9d640 1 -- 172.21.15.29:0/3316388202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff4133a50 con 0x7fbff40b4b10 2024-04-25T00:35:54.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff2d9d640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff40b4b10 0x7fbff4135950 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00c3f0 tx=0x7fbfec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbfeaffd640 1 -- 172.21.15.29:0/3316388202 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec011790 con 0x7fbff40b4b10 2024-04-25T00:35:54.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbfeaffd640 1 -- 172.21.15.29:0/3316388202 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfec017070 con 0x7fbff40b4b10 2024-04-25T00:35:54.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff889a640 1 -- 172.21.15.29:0/3316388202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff4133d30 con 0x7fbff40b4b10 2024-04-25T00:35:54.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbfeaffd640 1 -- 172.21.15.29:0/3316388202 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec011b70 con 0x7fbff40b4b10 2024-04-25T00:35:54.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff889a640 1 -- 172.21.15.29:0/3316388202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff41342f0 con 0x7fbff40b4b10 2024-04-25T00:35:54.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.404+0000 7fbff889a640 1 -- 172.21.15.29:0/3316388202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff4069270 con 0x7fbff40b4b10 2024-04-25T00:35:54.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.408+0000 7fbfeaffd640 1 -- 172.21.15.29:0/3316388202 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fbfec011d10 con 0x7fbff40b4b10 2024-04-25T00:35:54.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.408+0000 7fbfeaffd640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfd4067c20 0x7fbfd406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.408+0000 7fbfeaffd640 1 -- 172.21.15.29:0/3316388202 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fbfec086000 con 0x7fbff40b4b10 2024-04-25T00:35:54.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.408+0000 7fbff259c640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfd4067c20 0x7fbfd406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.408+0000 7fbff259c640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfd4067c20 0x7fbfd406a0e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbff4142080 tx=0x7fbfdc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.412+0000 7fbfeaffd640 1 -- 172.21.15.29:0/3316388202 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbfec056730 con 0x7fbff40b4b10 2024-04-25T00:35:54.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.568+0000 7fbff889a640 1 -- 172.21.15.29:0/3316388202 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbff40753e0 con 0x7fbfd4067c20 2024-04-25T00:35:54.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.584+0000 7fbfeaffd640 1 -- 172.21.15.29:0/3316388202 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240184 (secure 0 0 0) 0x7fbff40753e0 con 0x7fbfd4067c20 2024-04-25T00:35:54.590 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:54.591 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:35:54.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.588+0000 7fbff889a640 1 -- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfd4067c20 msgr2=0x7fbfd406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.588+0000 7fbff889a640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfd4067c20 0x7fbfd406a0e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbff4142080 tx=0x7fbfdc008040 comp rx=0 tx=0).stop 2024-04-25T00:35:54.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.588+0000 7fbff889a640 1 -- 172.21.15.29:0/3316388202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff40b4b10 msgr2=0x7fbff4135950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.588+0000 7fbff889a640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff40b4b10 0x7fbff4135950 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00c3f0 tx=0x7fbfec00c8b0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.588+0000 7fbff889a640 1 -- 172.21.15.29:0/3316388202 shutdown_connections 2024-04-25T00:35:54.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.588+0000 7fbff889a640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff40b4b10 0x7fbff4135950 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.588+0000 7fbff889a640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbff40b3a60 0x7fbff4135410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.588+0000 7fbff889a640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbfd4067c20 0x7fbfd406a0e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.588+0000 7fbff889a640 1 --2- 172.21.15.29:0/3316388202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff40b3140 0x7fbff41596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.588+0000 7fbff889a640 1 -- 172.21.15.29:0/3316388202 >> 172.21.15.29:0/3316388202 conn(0x7fbff40b1870 msgr2=0x7fbff40ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:54.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.588+0000 7fbff889a640 1 -- 172.21.15.29:0/3316388202 shutdown_connections 2024-04-25T00:35:54.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.588+0000 7fbff889a640 1 -- 172.21.15.29:0/3316388202 wait complete. 2024-04-25T00:35:54.608 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-04-25T00:35:53.536245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277764,"kb_used_data":7468,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754696956,"statfs":{"total":773094113280,"available":772809682944,"internally_reserved":0,"allocated":7647232,"data_stored":3871475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.392058"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.154333+0000","last_change":"2024-04-25T00:35:48.154333+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.154333+0000","last_undegraded":"2024-04-25T00:35:48.154333+0000","last_fullsized":"2024-04-25T00:35:48.154333+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152222+0000","last_change":"2024-04-25T00:35:48.152222+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152222+0000","last_undegraded":"2024-04-25T00:35:48.152222+0000","last_fullsized":"2024-04-25T00:35:48.152222+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147865+0000","last_change":"2024-04-25T00:35:48.147865+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147865+0000","last_undegraded":"2024-04-25T00:35:48.147865+0000","last_fullsized":"2024-04-25T00:35:48.147865+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152879+0000","last_change":"2024-04-25T00:35:48.152879+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152879+0000","last_undegraded":"2024-04-25T00:35:48.152879+0000","last_fullsized":"2024-04-25T00:35:48.152879+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.151577+0000","last_change":"2024-04-25T00:35:48.151577+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.151577+0000","last_undegraded":"2024-04-25T00:35:48.151577+0000","last_fullsized":"2024-04-25T00:35:48.151577+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.155134+0000","last_change":"2024-04-25T00:35:48.155134+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.155134+0000","last_undegraded":"2024-04-25T00:35:48.155134+0000","last_fullsized":"2024-04-25T00:35:48.155134+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.148179+0000","last_change":"2024-04-25T00:35:48.148179+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.148179+0000","last_undegraded":"2024-04-25T00:35:48.148179+0000","last_fullsized":"2024-04-25T00:35:48.148179+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.150610+0000","last_change":"2024-04-25T00:35:48.150610+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.150610+0000","last_undegraded":"2024-04-25T00:35:48.150610+0000","last_fullsized":"2024-04-25T00:35:48.150610+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147914+0000","last_change":"2024-04-25T00:35:48.147914+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147914+0000","last_undegraded":"2024-04-25T00:35:48.147914+0000","last_fullsized":"2024-04-25T00:35:48.147914+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152776+0000","last_change":"2024-04-25T00:35:48.152776+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152776+0000","last_undegraded":"2024-04-25T00:35:48.152776+0000","last_fullsized":"2024-04-25T00:35:48.152776+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152911+0000","last_change":"2024-04-25T00:35:48.152911+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152911+0000","last_undegraded":"2024-04-25T00:35:48.152911+0000","last_fullsized":"2024-04-25T00:35:48.152911+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147856+0000","last_change":"2024-04-25T00:35:48.147856+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147856+0000","last_undegraded":"2024-04-25T00:35:48.147856+0000","last_fullsized":"2024-04-25T00:35:48.147856+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147891+0000","last_change":"2024-04-25T00:35:48.147891+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147891+0000","last_undegraded":"2024-04-25T00:35:48.147891+0000","last_fullsized":"2024-04-25T00:35:48.147891+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.154123+0000","last_change":"2024-04-25T00:35:48.154123+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.154123+0000","last_undegraded":"2024-04-25T00:35:48.154123+0000","last_fullsized":"2024-04-25T00:35:48.154123+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.149062+0000","last_change":"2024-04-25T00:35:48.149062+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.149062+0000","last_undegraded":"2024-04-25T00:35:48.149062+0000","last_fullsized":"2024-04-25T00:35:48.149062+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147937+0000","last_change":"2024-04-25T00:35:48.147937+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147937+0000","last_undegraded":"2024-04-25T00:35:48.147937+0000","last_fullsized":"2024-04-25T00:35:48.147937+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.159198+0000","last_change":"2024-04-25T00:35:48.159198+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.159198+0000","last_undegraded":"2024-04-25T00:35:48.159198+0000","last_fullsized":"2024-04-25T00:35:48.159198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.149376+0000","last_change":"2024-04-25T00:35:48.149376+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.149376+0000","last_undegraded":"2024-04-25T00:35:48.149376+0000","last_fullsized":"2024-04-25T00:35:48.149376+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.153859+0000","last_change":"2024-04-25T00:35:48.153859+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.153859+0000","last_undegraded":"2024-04-25T00:35:48.153859+0000","last_fullsized":"2024-04-25T00:35:48.153859+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.155771+0000","last_change":"2024-04-25T00:35:48.155771+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.155771+0000","last_undegraded":"2024-04-25T00:35:48.155771+0000","last_fullsized":"2024-04-25T00:35:48.155771+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152611+0000","last_change":"2024-04-25T00:35:48.152611+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152611+0000","last_undegraded":"2024-04-25T00:35:48.152611+0000","last_fullsized":"2024-04-25T00:35:48.152611+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147859+0000","last_change":"2024-04-25T00:35:48.147859+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147859+0000","last_undegraded":"2024-04-25T00:35:48.147859+0000","last_fullsized":"2024-04-25T00:35:48.147859+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152521+0000","last_change":"2024-04-25T00:35:48.152521+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152521+0000","last_undegraded":"2024-04-25T00:35:48.152521+0000","last_fullsized":"2024-04-25T00:35:48.152521+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.158091+0000","last_change":"2024-04-25T00:35:48.158091+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.158091+0000","last_undegraded":"2024-04-25T00:35:48.158091+0000","last_fullsized":"2024-04-25T00:35:48.158091+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.150268+0000","last_change":"2024-04-25T00:35:48.150268+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.150268+0000","last_undegraded":"2024-04-25T00:35:48.150268+0000","last_fullsized":"2024-04-25T00:35:48.150268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.151198+0000","last_change":"2024-04-25T00:35:48.151198+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.151198+0000","last_undegraded":"2024-04-25T00:35:48.151198+0000","last_fullsized":"2024-04-25T00:35:48.151198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152307+0000","last_change":"2024-04-25T00:35:48.152307+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152307+0000","last_undegraded":"2024-04-25T00:35:48.152307+0000","last_fullsized":"2024-04-25T00:35:48.152307+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.150478+0000","last_change":"2024-04-25T00:35:48.150478+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.150478+0000","last_undegraded":"2024-04-25T00:35:48.150478+0000","last_fullsized":"2024-04-25T00:35:48.150478+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147602+0000","last_change":"2024-04-25T00:35:48.147602+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147602+0000","last_undegraded":"2024-04-25T00:35:48.147602+0000","last_fullsized":"2024-04-25T00:35:48.147602+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.149920+0000","last_change":"2024-04-25T00:35:48.149920+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.149920+0000","last_undegraded":"2024-04-25T00:35:48.149920+0000","last_fullsized":"2024-04-25T00:35:48.149920+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.153712+0000","last_change":"2024-04-25T00:35:48.153712+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.153712+0000","last_undegraded":"2024-04-25T00:35:48.153712+0000","last_fullsized":"2024-04-25T00:35:48.153712+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152982+0000","last_change":"2024-04-25T00:35:48.152982+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152982+0000","last_undegraded":"2024-04-25T00:35:48.152982+0000","last_fullsized":"2024-04-25T00:35:48.152982+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":260,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T00:35:48.155850+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:35:48.155850+0000","last_peered":"2024-04-25T00:35:48.155850+0000","last_clean":"2024-04-25T00:35:48.155850+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:35:48.155850+0000","last_undegraded":"2024-04-25T00:35:48.155850+0000","last_fullsized":"2024-04-25T00:35:48.155850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1396736,"data_stored":924529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":339978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":802816,"data_stored":334161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":802816,"data_stored":334161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36140,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335700,"statfs":{"total":96636764160,"available":96599756800,"internally_reserved":0,"allocated":815104,"data_stored":339978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36128,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335712,"statfs":{"total":96636764160,"available":96599769088,"internally_reserved":0,"allocated":802816,"data_stored":334161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":802816,"data_stored":334161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36724,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335116,"statfs":{"total":96636764160,"available":96599158784,"internally_reserved":0,"allocated":1409024,"data_stored":930346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:35:54.655 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:35:54.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.776+0000 7f206ff1c640 1 Processor -- start 2024-04-25T00:35:54.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.776+0000 7f206ff1c640 1 -- start start 2024-04-25T00:35:54.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ff1c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f20680b3c30 0x7f206806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ff1c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20680b41e0 con 0x7f20680b4b10 2024-04-25T00:35:54.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ff1c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f206806d960 con 0x7f20680b3140 2024-04-25T00:35:54.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ff1c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f206806dc40 con 0x7f20680b3c30 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206e4b9640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f20680b4b10 0x7f20680ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35324/0 (socket says 172.21.15.29:35324) 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206e4b9640 1 -- 172.21.15.29:0/18689888 learned_addr learned my addr 172.21.15.29:0/18689888 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206d4b7640 1 --2- 172.21.15.29:0/18689888 >> v2:172.21.15.29:3300/0 conn(0x7f20680b3c30 0x7f206806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3038443319 0 0) 0x7f20680b41e0 con 0x7f20680b4b10 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2054003530 con 0x7f20680b4b10 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4092529366 0 0) 0x7f2054003530 con 0x7f20680b4b10 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 >> v2:172.21.15.29:3300/0 conn(0x7f20680b3c30 msgr2=0x7f206806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 --2- 172.21.15.29:0/18689888 >> v2:172.21.15.29:3300/0 conn(0x7f20680b3c30 0x7f206806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 >> v1:172.21.15.157:6789/0 conn(0x7f20680b3140 legacy=0x7f20680b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206806df20 con 0x7f20680b4b10 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2060001a00 con 0x7f20680b4b10 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206d4b7640 1 --2- 172.21.15.29:0/18689888 >> v2:172.21.15.29:3300/0 conn(0x7f20680b3c30 0x7f206806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 >> v1:172.21.15.29:6789/0 conn(0x7f20680b4b10 legacy=0x7f20680ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 --2- 172.21.15.29:0/18689888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2054003a40 0x7f2054003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 --2- 172.21.15.29:0/18689888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2054004810 0x7f2054004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 --2- 172.21.15.29:0/18689888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20540051d0 0x7f205400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f206806dc40 con 0x7f2054004810 2024-04-25T00:35:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f205400c100 con 0x7f20540051d0 2024-04-25T00:35:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f205400c300 con 0x7f2054003a40 2024-04-25T00:35:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206d4b7640 1 --2- 172.21.15.29:0/18689888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2054004810 0x7f2054004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206e4b9640 1 --2- 172.21.15.29:0/18689888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20540051d0 0x7f205400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206d4b7640 1 -- 172.21.15.29:0/18689888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2054003a40 msgr2=0x7f2054003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206d4b7640 1 --2- 172.21.15.29:0/18689888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2054003a40 0x7f2054003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206d4b7640 1 -- 172.21.15.29:0/18689888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20540051d0 msgr2=0x7f205400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206d4b7640 1 --2- 172.21.15.29:0/18689888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20540051d0 0x7f205400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206d4b7640 1 -- 172.21.15.29:0/18689888 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2054003530 con 0x7f2054004810 2024-04-25T00:35:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206e4b9640 1 -- 172.21.15.29:0/18689888 reap_dead start 2024-04-25T00:35:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206d4b7640 1 --2- 172.21.15.29:0/18689888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2054004810 0x7f2054004c90 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2058009cb0 tx=0x7f205801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61132280da33f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20580b2070 con 0x7f2054004810 2024-04-25T00:35:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20580a58d0 con 0x7f2054004810 2024-04-25T00:35:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.780+0000 7f206ccb6640 1 -- 172.21.15.29:0/18689888 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058009880 con 0x7f2054004810 2024-04-25T00:35:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 -- 172.21.15.29:0/18689888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2054004810 msgr2=0x7f2054004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 --2- 172.21.15.29:0/18689888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2054004810 0x7f2054004c90 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2058009cb0 tx=0x7f205801cc10 comp rx=0 tx=0).stop 2024-04-25T00:35:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 -- 172.21.15.29:0/18689888 shutdown_connections 2024-04-25T00:35:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 --2- 172.21.15.29:0/18689888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2054004810 0x7f2054004c90 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 -- 172.21.15.29:0/18689888 >> 172.21.15.29:0/18689888 conn(0x7f20680b1870 msgr2=0x7f20680b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 -- 172.21.15.29:0/18689888 shutdown_connections 2024-04-25T00:35:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 -- 172.21.15.29:0/18689888 wait complete. 2024-04-25T00:35:54.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 Processor -- start 2024-04-25T00:35:54.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 -- start start 2024-04-25T00:35:54.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2054004810 0x7f20681f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20680a9490 0x7f20680a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f206813ee00 0x7f206813f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2068145a90 con 0x7f206813ee00 2024-04-25T00:35:54.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2068145910 con 0x7f20680a9490 2024-04-25T00:35:54.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f206806e440 con 0x7f2054004810 2024-04-25T00:35:54.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206e4b9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f206813ee00 0x7f206813f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206e4b9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f206813ee00 0x7f206813f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52470/0 (socket says 172.21.15.29:52470) 2024-04-25T00:35:54.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206e4b9640 1 -- 172.21.15.29:0/2765907107 learned_addr learned my addr 172.21.15.29:0/2765907107 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:54.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206dcb8640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2054004810 0x7f20681f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206d4b7640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20680a9490 0x7f20680a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206e4b9640 1 -- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2054004810 msgr2=0x7f20681f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206e4b9640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2054004810 0x7f20681f1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206e4b9640 1 -- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20680a9490 msgr2=0x7f20680a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206e4b9640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20680a9490 0x7f20680a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206e4b9640 1 -- 172.21.15.29:0/2765907107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206813fb30 con 0x7f206813ee00 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206dcb8640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2054004810 0x7f20681f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206d4b7640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20680a9490 0x7f20680a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206e4b9640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f206813ee00 0x7f206813f280 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f20680b4790 tx=0x7f2060005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f205e7fc640 1 -- 172.21.15.29:0/2765907107 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206001a070 con 0x7f206813ee00 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f205e7fc640 1 -- 172.21.15.29:0/2765907107 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2060011040 con 0x7f206813ee00 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f205e7fc640 1 -- 172.21.15.29:0/2765907107 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206000eda0 con 0x7f206813ee00 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 -- 172.21.15.29:0/2765907107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2068134490 con 0x7f206813ee00 2024-04-25T00:35:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.784+0000 7f206ff1c640 1 -- 172.21.15.29:0/2765907107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2068134a50 con 0x7f206813ee00 2024-04-25T00:35:54.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.788+0000 7f206ff1c640 1 -- 172.21.15.29:0/2765907107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2068069270 con 0x7f206813ee00 2024-04-25T00:35:54.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.788+0000 7f205e7fc640 1 -- 172.21.15.29:0/2765907107 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f206001d070 con 0x7f206813ee00 2024-04-25T00:35:54.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.788+0000 7f205e7fc640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f204c067c20 0x7f204c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.788+0000 7f205e7fc640 1 -- 172.21.15.29:0/2765907107 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f2060088b20 con 0x7f206813ee00 2024-04-25T00:35:54.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.788+0000 7f206dcb8640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f204c067c20 0x7f204c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.788+0000 7f206dcb8640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f204c067c20 0x7f204c06a0e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2068069990 tx=0x7f2064009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.792+0000 7f205e7fc640 1 -- 172.21.15.29:0/2765907107 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2060058b00 con 0x7f206813ee00 2024-04-25T00:35:54.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.944+0000 7f206ff1c640 1 -- 172.21.15.29:0/2765907107 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2068134c20 con 0x7f204c067c20 2024-04-25T00:35:54.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.960+0000 7f205e7fc640 1 -- 172.21.15.29:0/2765907107 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240184 (secure 0 0 0) 0x7f2068134c20 con 0x7f204c067c20 2024-04-25T00:35:54.965 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:54.966 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:35:54.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.964+0000 7f206ff1c640 1 -- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f204c067c20 msgr2=0x7f204c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.964+0000 7f206ff1c640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f204c067c20 0x7f204c06a0e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2068069990 tx=0x7f2064009040 comp rx=0 tx=0).stop 2024-04-25T00:35:54.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.964+0000 7f206ff1c640 1 -- 172.21.15.29:0/2765907107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f206813ee00 msgr2=0x7f206813f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.964+0000 7f206ff1c640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f206813ee00 0x7f206813f280 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f20680b4790 tx=0x7f2060005b00 comp rx=0 tx=0).stop 2024-04-25T00:35:54.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.964+0000 7f206ff1c640 1 -- 172.21.15.29:0/2765907107 shutdown_connections 2024-04-25T00:35:54.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.964+0000 7f206ff1c640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f206813ee00 0x7f206813f280 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.964+0000 7f206ff1c640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f204c067c20 0x7f204c06a0e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.964+0000 7f206ff1c640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20680a9490 0x7f20680a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.964+0000 7f206ff1c640 1 --2- 172.21.15.29:0/2765907107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2054004810 0x7f20681f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.964+0000 7f206ff1c640 1 -- 172.21.15.29:0/2765907107 >> 172.21.15.29:0/2765907107 conn(0x7f20680b1870 msgr2=0x7f20680ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:54.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.964+0000 7f206ff1c640 1 -- 172.21.15.29:0/2765907107 shutdown_connections 2024-04-25T00:35:54.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:54.964+0000 7f206ff1c640 1 -- 172.21.15.29:0/2765907107 wait complete. 2024-04-25T00:35:54.981 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-04-25T00:35:53.536245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277764,"kb_used_data":7468,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754696956,"statfs":{"total":773094113280,"available":772809682944,"internally_reserved":0,"allocated":7647232,"data_stored":3871475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.392058"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.154333+0000","last_change":"2024-04-25T00:35:48.154333+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.154333+0000","last_undegraded":"2024-04-25T00:35:48.154333+0000","last_fullsized":"2024-04-25T00:35:48.154333+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152222+0000","last_change":"2024-04-25T00:35:48.152222+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152222+0000","last_undegraded":"2024-04-25T00:35:48.152222+0000","last_fullsized":"2024-04-25T00:35:48.152222+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147865+0000","last_change":"2024-04-25T00:35:48.147865+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147865+0000","last_undegraded":"2024-04-25T00:35:48.147865+0000","last_fullsized":"2024-04-25T00:35:48.147865+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152879+0000","last_change":"2024-04-25T00:35:48.152879+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152879+0000","last_undegraded":"2024-04-25T00:35:48.152879+0000","last_fullsized":"2024-04-25T00:35:48.152879+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.151577+0000","last_change":"2024-04-25T00:35:48.151577+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.151577+0000","last_undegraded":"2024-04-25T00:35:48.151577+0000","last_fullsized":"2024-04-25T00:35:48.151577+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.155134+0000","last_change":"2024-04-25T00:35:48.155134+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.155134+0000","last_undegraded":"2024-04-25T00:35:48.155134+0000","last_fullsized":"2024-04-25T00:35:48.155134+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.148179+0000","last_change":"2024-04-25T00:35:48.148179+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.148179+0000","last_undegraded":"2024-04-25T00:35:48.148179+0000","last_fullsized":"2024-04-25T00:35:48.148179+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.150610+0000","last_change":"2024-04-25T00:35:48.150610+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.150610+0000","last_undegraded":"2024-04-25T00:35:48.150610+0000","last_fullsized":"2024-04-25T00:35:48.150610+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147914+0000","last_change":"2024-04-25T00:35:48.147914+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147914+0000","last_undegraded":"2024-04-25T00:35:48.147914+0000","last_fullsized":"2024-04-25T00:35:48.147914+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152776+0000","last_change":"2024-04-25T00:35:48.152776+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152776+0000","last_undegraded":"2024-04-25T00:35:48.152776+0000","last_fullsized":"2024-04-25T00:35:48.152776+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152911+0000","last_change":"2024-04-25T00:35:48.152911+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152911+0000","last_undegraded":"2024-04-25T00:35:48.152911+0000","last_fullsized":"2024-04-25T00:35:48.152911+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147856+0000","last_change":"2024-04-25T00:35:48.147856+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147856+0000","last_undegraded":"2024-04-25T00:35:48.147856+0000","last_fullsized":"2024-04-25T00:35:48.147856+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147891+0000","last_change":"2024-04-25T00:35:48.147891+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147891+0000","last_undegraded":"2024-04-25T00:35:48.147891+0000","last_fullsized":"2024-04-25T00:35:48.147891+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.154123+0000","last_change":"2024-04-25T00:35:48.154123+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.154123+0000","last_undegraded":"2024-04-25T00:35:48.154123+0000","last_fullsized":"2024-04-25T00:35:48.154123+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.149062+0000","last_change":"2024-04-25T00:35:48.149062+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.149062+0000","last_undegraded":"2024-04-25T00:35:48.149062+0000","last_fullsized":"2024-04-25T00:35:48.149062+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147937+0000","last_change":"2024-04-25T00:35:48.147937+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147937+0000","last_undegraded":"2024-04-25T00:35:48.147937+0000","last_fullsized":"2024-04-25T00:35:48.147937+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.159198+0000","last_change":"2024-04-25T00:35:48.159198+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.159198+0000","last_undegraded":"2024-04-25T00:35:48.159198+0000","last_fullsized":"2024-04-25T00:35:48.159198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.149376+0000","last_change":"2024-04-25T00:35:48.149376+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.149376+0000","last_undegraded":"2024-04-25T00:35:48.149376+0000","last_fullsized":"2024-04-25T00:35:48.149376+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.153859+0000","last_change":"2024-04-25T00:35:48.153859+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.153859+0000","last_undegraded":"2024-04-25T00:35:48.153859+0000","last_fullsized":"2024-04-25T00:35:48.153859+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.155771+0000","last_change":"2024-04-25T00:35:48.155771+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.155771+0000","last_undegraded":"2024-04-25T00:35:48.155771+0000","last_fullsized":"2024-04-25T00:35:48.155771+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152611+0000","last_change":"2024-04-25T00:35:48.152611+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152611+0000","last_undegraded":"2024-04-25T00:35:48.152611+0000","last_fullsized":"2024-04-25T00:35:48.152611+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147859+0000","last_change":"2024-04-25T00:35:48.147859+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147859+0000","last_undegraded":"2024-04-25T00:35:48.147859+0000","last_fullsized":"2024-04-25T00:35:48.147859+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152521+0000","last_change":"2024-04-25T00:35:48.152521+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152521+0000","last_undegraded":"2024-04-25T00:35:48.152521+0000","last_fullsized":"2024-04-25T00:35:48.152521+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.158091+0000","last_change":"2024-04-25T00:35:48.158091+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.158091+0000","last_undegraded":"2024-04-25T00:35:48.158091+0000","last_fullsized":"2024-04-25T00:35:48.158091+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.150268+0000","last_change":"2024-04-25T00:35:48.150268+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.150268+0000","last_undegraded":"2024-04-25T00:35:48.150268+0000","last_fullsized":"2024-04-25T00:35:48.150268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.151198+0000","last_change":"2024-04-25T00:35:48.151198+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.151198+0000","last_undegraded":"2024-04-25T00:35:48.151198+0000","last_fullsized":"2024-04-25T00:35:48.151198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152307+0000","last_change":"2024-04-25T00:35:48.152307+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152307+0000","last_undegraded":"2024-04-25T00:35:48.152307+0000","last_fullsized":"2024-04-25T00:35:48.152307+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.150478+0000","last_change":"2024-04-25T00:35:48.150478+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.150478+0000","last_undegraded":"2024-04-25T00:35:48.150478+0000","last_fullsized":"2024-04-25T00:35:48.150478+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.147602+0000","last_change":"2024-04-25T00:35:48.147602+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.147602+0000","last_undegraded":"2024-04-25T00:35:48.147602+0000","last_fullsized":"2024-04-25T00:35:48.147602+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.149920+0000","last_change":"2024-04-25T00:35:48.149920+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.149920+0000","last_undegraded":"2024-04-25T00:35:48.149920+0000","last_fullsized":"2024-04-25T00:35:48.149920+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.153712+0000","last_change":"2024-04-25T00:35:48.153712+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.153712+0000","last_undegraded":"2024-04-25T00:35:48.153712+0000","last_fullsized":"2024-04-25T00:35:48.153712+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:35:49.141914+0000","last_change":"2024-04-25T00:35:49.141914+0000","last_active":"2024-04-25T00:35:49.141914+0000","last_peered":"2024-04-25T00:35:49.141914+0000","last_clean":"2024-04-25T00:35:49.141914+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:49.141914+0000","last_undegraded":"2024-04-25T00:35:49.141914+0000","last_fullsized":"2024-04-25T00:35:49.141914+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T00:35:48.152982+0000","last_change":"2024-04-25T00:35:48.152982+0000","last_active":"2024-04-25T00:35:48.133758+0000","last_peered":"2024-04-25T00:35:48.133758+0000","last_clean":"2024-04-25T00:35:48.133758+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:35:48.152982+0000","last_undegraded":"2024-04-25T00:35:48.152982+0000","last_fullsized":"2024-04-25T00:35:48.152982+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":260,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T00:35:48.155850+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:35:48.155850+0000","last_peered":"2024-04-25T00:35:48.155850+0000","last_clean":"2024-04-25T00:35:48.155850+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:35:48.155850+0000","last_undegraded":"2024-04-25T00:35:48.155850+0000","last_fullsized":"2024-04-25T00:35:48.155850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1396736,"data_stored":924529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":339978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":802816,"data_stored":334161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":802816,"data_stored":334161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36140,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335700,"statfs":{"total":96636764160,"available":96599756800,"internally_reserved":0,"allocated":815104,"data_stored":339978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36128,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335712,"statfs":{"total":96636764160,"available":96599769088,"internally_reserved":0,"allocated":802816,"data_stored":334161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":802816,"data_stored":334161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36724,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335116,"statfs":{"total":96636764160,"available":96599158784,"internally_reserved":0,"allocated":1409024,"data_stored":930346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:35:54.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2024-04-25T00:35:54.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2024-04-25T00:35:54.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2024-04-25T00:35:54.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2024-04-25T00:35:54.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2024-04-25T00:35:54.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2024-04-25T00:35:54.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2024-04-25T00:35:54.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2024-04-25T00:35:54.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2024-04-25T00:35:54.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2024-04-25T00:35:54.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2024-04-25T00:35:54.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2024-04-25T00:35:54.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2024-04-25T00:35:54.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2024-04-25T00:35:54.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2024-04-25T00:35:54.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2024-04-25T00:35:54.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2024-04-25T00:35:54.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2024-04-25T00:35:54.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2024-04-25T00:35:54.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2024-04-25T00:35:54.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2024-04-25T00:35:54.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2024-04-25T00:35:54.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2024-04-25T00:35:54.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2024-04-25T00:35:54.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2024-04-25T00:35:54.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2024-04-25T00:35:54.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2024-04-25T00:35:54.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2024-04-25T00:35:54.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2024-04-25T00:35:54.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2024-04-25T00:35:54.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2024-04-25T00:35:54.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2024-04-25T00:35:54.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-04-25T00:35:54.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2024-04-25T00:35:54.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.154333+0000', 'last_change': '2024-04-25T00:35:48.154333+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.154333+0000', 'last_undegraded': '2024-04-25T00:35:48.154333+0000', 'last_fullsized': '2024-04-25T00:35:48.154333+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:35:54.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2024-04-25T00:35:54.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2024-04-25T00:35:54.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.152222+0000', 'last_change': '2024-04-25T00:35:48.152222+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.152222+0000', 'last_undegraded': '2024-04-25T00:35:48.152222+0000', 'last_fullsized': '2024-04-25T00:35:48.152222+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:35:54.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2024-04-25T00:35:54.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2024-04-25T00:35:54.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.147865+0000', 'last_change': '2024-04-25T00:35:48.147865+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.147865+0000', 'last_undegraded': '2024-04-25T00:35:48.147865+0000', 'last_fullsized': '2024-04-25T00:35:48.147865+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:35:54.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-04-25T00:35:54.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2024-04-25T00:35:54.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.152879+0000', 'last_change': '2024-04-25T00:35:48.152879+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.152879+0000', 'last_undegraded': '2024-04-25T00:35:48.152879+0000', 'last_fullsized': '2024-04-25T00:35:48.152879+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:35:54.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2024-04-25T00:35:54.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2024-04-25T00:35:54.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.151577+0000', 'last_change': '2024-04-25T00:35:48.151577+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.151577+0000', 'last_undegraded': '2024-04-25T00:35:48.151577+0000', 'last_fullsized': '2024-04-25T00:35:48.151577+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:35:54.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2024-04-25T00:35:54.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2024-04-25T00:35:54.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.155134+0000', 'last_change': '2024-04-25T00:35:48.155134+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.155134+0000', 'last_undegraded': '2024-04-25T00:35:48.155134+0000', 'last_fullsized': '2024-04-25T00:35:48.155134+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:35:54.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2024-04-25T00:35:54.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2024-04-25T00:35:54.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.148179+0000', 'last_change': '2024-04-25T00:35:48.148179+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.148179+0000', 'last_undegraded': '2024-04-25T00:35:48.148179+0000', 'last_fullsized': '2024-04-25T00:35:48.148179+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:35:54.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-04-25T00:35:54.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2024-04-25T00:35:54.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.150610+0000', 'last_change': '2024-04-25T00:35:48.150610+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.150610+0000', 'last_undegraded': '2024-04-25T00:35:48.150610+0000', 'last_fullsized': '2024-04-25T00:35:48.150610+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:35:54.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2024-04-25T00:35:54.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2024-04-25T00:35:54.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.147914+0000', 'last_change': '2024-04-25T00:35:48.147914+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.147914+0000', 'last_undegraded': '2024-04-25T00:35:48.147914+0000', 'last_fullsized': '2024-04-25T00:35:48.147914+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:35:54.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-04-25T00:35:54.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2024-04-25T00:35:54.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.152776+0000', 'last_change': '2024-04-25T00:35:48.152776+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.152776+0000', 'last_undegraded': '2024-04-25T00:35:48.152776+0000', 'last_fullsized': '2024-04-25T00:35:48.152776+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:35:54.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2024-04-25T00:35:54.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2024-04-25T00:35:54.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.152911+0000', 'last_change': '2024-04-25T00:35:48.152911+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.152911+0000', 'last_undegraded': '2024-04-25T00:35:48.152911+0000', 'last_fullsized': '2024-04-25T00:35:48.152911+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:35:54.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2024-04-25T00:35:54.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2024-04-25T00:35:54.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.147856+0000', 'last_change': '2024-04-25T00:35:48.147856+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.147856+0000', 'last_undegraded': '2024-04-25T00:35:48.147856+0000', 'last_fullsized': '2024-04-25T00:35:48.147856+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:35:54.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-04-25T00:35:54.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2024-04-25T00:35:54.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.147891+0000', 'last_change': '2024-04-25T00:35:48.147891+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.147891+0000', 'last_undegraded': '2024-04-25T00:35:48.147891+0000', 'last_fullsized': '2024-04-25T00:35:48.147891+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:35:54.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2024-04-25T00:35:54.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2024-04-25T00:35:54.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.154123+0000', 'last_change': '2024-04-25T00:35:48.154123+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.154123+0000', 'last_undegraded': '2024-04-25T00:35:48.154123+0000', 'last_fullsized': '2024-04-25T00:35:48.154123+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:35:54.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-04-25T00:35:54.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2024-04-25T00:35:54.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.149062+0000', 'last_change': '2024-04-25T00:35:48.149062+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.149062+0000', 'last_undegraded': '2024-04-25T00:35:48.149062+0000', 'last_fullsized': '2024-04-25T00:35:48.149062+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:35:54.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2024-04-25T00:35:54.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2024-04-25T00:35:54.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.147937+0000', 'last_change': '2024-04-25T00:35:48.147937+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.147937+0000', 'last_undegraded': '2024-04-25T00:35:48.147937+0000', 'last_fullsized': '2024-04-25T00:35:48.147937+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:35:54.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2024-04-25T00:35:54.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2024-04-25T00:35:54.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.159198+0000', 'last_change': '2024-04-25T00:35:48.159198+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.159198+0000', 'last_undegraded': '2024-04-25T00:35:48.159198+0000', 'last_fullsized': '2024-04-25T00:35:48.159198+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:35:54.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-04-25T00:35:54.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2024-04-25T00:35:54.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.149376+0000', 'last_change': '2024-04-25T00:35:48.149376+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.149376+0000', 'last_undegraded': '2024-04-25T00:35:48.149376+0000', 'last_fullsized': '2024-04-25T00:35:48.149376+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:35:54.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2024-04-25T00:35:54.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2024-04-25T00:35:54.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.153859+0000', 'last_change': '2024-04-25T00:35:48.153859+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.153859+0000', 'last_undegraded': '2024-04-25T00:35:48.153859+0000', 'last_fullsized': '2024-04-25T00:35:48.153859+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:35:54.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2024-04-25T00:35:54.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2024-04-25T00:35:54.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.155771+0000', 'last_change': '2024-04-25T00:35:48.155771+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.155771+0000', 'last_undegraded': '2024-04-25T00:35:48.155771+0000', 'last_fullsized': '2024-04-25T00:35:48.155771+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:35:54.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2024-04-25T00:35:54.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2024-04-25T00:35:54.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.152611+0000', 'last_change': '2024-04-25T00:35:48.152611+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.152611+0000', 'last_undegraded': '2024-04-25T00:35:48.152611+0000', 'last_fullsized': '2024-04-25T00:35:48.152611+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:35:54.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2024-04-25T00:35:54.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2024-04-25T00:35:54.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.147859+0000', 'last_change': '2024-04-25T00:35:48.147859+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.147859+0000', 'last_undegraded': '2024-04-25T00:35:48.147859+0000', 'last_fullsized': '2024-04-25T00:35:48.147859+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:35:54.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2024-04-25T00:35:54.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:54.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2024-04-25T00:35:54.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.152521+0000', 'last_change': '2024-04-25T00:35:48.152521+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.152521+0000', 'last_undegraded': '2024-04-25T00:35:48.152521+0000', 'last_fullsized': '2024-04-25T00:35:48.152521+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:35:54.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-04-25T00:35:55.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:55.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2024-04-25T00:35:55.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.158091+0000', 'last_change': '2024-04-25T00:35:48.158091+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.158091+0000', 'last_undegraded': '2024-04-25T00:35:48.158091+0000', 'last_fullsized': '2024-04-25T00:35:48.158091+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:35:55.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2024-04-25T00:35:55.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:55.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2024-04-25T00:35:55.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.150268+0000', 'last_change': '2024-04-25T00:35:48.150268+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.150268+0000', 'last_undegraded': '2024-04-25T00:35:48.150268+0000', 'last_fullsized': '2024-04-25T00:35:48.150268+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:35:55.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-04-25T00:35:55.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:55.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2024-04-25T00:35:55.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.151198+0000', 'last_change': '2024-04-25T00:35:48.151198+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.151198+0000', 'last_undegraded': '2024-04-25T00:35:48.151198+0000', 'last_fullsized': '2024-04-25T00:35:48.151198+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:35:55.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-04-25T00:35:55.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:55.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2024-04-25T00:35:55.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.152307+0000', 'last_change': '2024-04-25T00:35:48.152307+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.152307+0000', 'last_undegraded': '2024-04-25T00:35:48.152307+0000', 'last_fullsized': '2024-04-25T00:35:48.152307+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:35:55.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2024-04-25T00:35:55.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:55.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2024-04-25T00:35:55.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.150478+0000', 'last_change': '2024-04-25T00:35:48.150478+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.150478+0000', 'last_undegraded': '2024-04-25T00:35:48.150478+0000', 'last_fullsized': '2024-04-25T00:35:48.150478+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:35:55.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2024-04-25T00:35:55.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:55.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2024-04-25T00:35:55.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.147602+0000', 'last_change': '2024-04-25T00:35:48.147602+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.147602+0000', 'last_undegraded': '2024-04-25T00:35:48.147602+0000', 'last_fullsized': '2024-04-25T00:35:48.147602+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:35:55.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2024-04-25T00:35:55.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:55.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2024-04-25T00:35:55.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.149920+0000', 'last_change': '2024-04-25T00:35:48.149920+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.149920+0000', 'last_undegraded': '2024-04-25T00:35:48.149920+0000', 'last_fullsized': '2024-04-25T00:35:48.149920+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:35:55.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2024-04-25T00:35:55.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:55.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2024-04-25T00:35:55.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.153712+0000', 'last_change': '2024-04-25T00:35:48.153712+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.153712+0000', 'last_undegraded': '2024-04-25T00:35:48.153712+0000', 'last_fullsized': '2024-04-25T00:35:48.153712+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:35:55.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2024-04-25T00:35:55.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T00:35:49.141914+0000', 'last_change': '2024-04-25T00:35:49.141914+0000', 'last_active': '2024-04-25T00:35:49.141914+0000', 'last_peered': '2024-04-25T00:35:49.141914+0000', 'last_clean': '2024-04-25T00:35:49.141914+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:49.141914+0000', 'last_undegraded': '2024-04-25T00:35:49.141914+0000', 'last_fullsized': '2024-04-25T00:35:49.141914+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:49.141914+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T00:35:55.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2024-04-25T00:35:55.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:35:48.152982+0000', 'last_change': '2024-04-25T00:35:48.152982+0000', 'last_active': '2024-04-25T00:35:48.133758+0000', 'last_peered': '2024-04-25T00:35:48.133758+0000', 'last_clean': '2024-04-25T00:35:48.133758+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:35:48.152982+0000', 'last_undegraded': '2024-04-25T00:35:48.152982+0000', 'last_fullsized': '2024-04-25T00:35:48.152982+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'last_clean_scrub_stamp': '2024-04-25T00:35:48.133758+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:35:55.003 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:35:55.137 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-04-25T00:35:55.136+0000 7ff4548bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:35:55.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.140+0000 7fee1eeff640 1 Processor -- start 2024-04-25T00:35:55.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.140+0000 7fee1eeff640 1 -- start start 2024-04-25T00:35:55.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1eeff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee1815e1b0 0x7fee18072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:55.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1eeff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fee180ccfd0 0x7fee1806b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:55.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1eeff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fee1806b750 0x7fee1806db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:55.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1eeff640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fee18161bb0 con 0x7fee1806b750 2024-04-25T00:35:55.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1eeff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fee1806bba0 con 0x7fee180ccfd0 2024-04-25T00:35:55.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1eeff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fee180d0a30 con 0x7fee1815e1b0 2024-04-25T00:35:55.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1d49c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fee1806b750 0x7fee1806db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:55.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1d49c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fee1806b750 0x7fee1806db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52484/0 (socket says 172.21.15.29:52484) 2024-04-25T00:35:55.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1d49c640 1 -- 172.21.15.29:0/4088323746 learned_addr learned my addr 172.21.15.29:0/4088323746 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:55.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee0ffff640 1 --2- 172.21.15.29:0/4088323746 >> v2:172.21.15.157:3300/0 conn(0x7fee180ccfd0 0x7fee1806b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:55.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1cc9b640 1 --2- 172.21.15.29:0/4088323746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee1815e1b0 0x7fee18072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:55.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1d49c640 1 -- 172.21.15.29:0/4088323746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee1815e1b0 msgr2=0x7fee18072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:55.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1d49c640 1 --2- 172.21.15.29:0/4088323746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee1815e1b0 0x7fee18072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1d49c640 1 -- 172.21.15.29:0/4088323746 >> v2:172.21.15.157:3300/0 conn(0x7fee180ccfd0 msgr2=0x7fee1806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:55.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1d49c640 1 --2- 172.21.15.29:0/4088323746 >> v2:172.21.15.157:3300/0 conn(0x7fee180ccfd0 0x7fee1806b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1d49c640 1 -- 172.21.15.29:0/4088323746 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee180cfcb0 con 0x7fee1806b750 2024-04-25T00:35:55.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee1d49c640 1 --2- 172.21.15.29:0/4088323746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee1806b750 0x7fee1806db90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fee180ce730 tx=0x7fee1001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968a14ff2cab3651 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:55.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee0f7fe640 1 -- 172.21.15.29:0/4088323746 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee100ac070 con 0x7fee1806b750 2024-04-25T00:35:55.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee0f7fe640 1 -- 172.21.15.29:0/4088323746 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee100a8420 con 0x7fee1806b750 2024-04-25T00:35:55.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.144+0000 7fee0f7fe640 1 -- 172.21.15.29:0/4088323746 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee100a87f0 con 0x7fee1806b750 2024-04-25T00:35:55.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 -- 172.21.15.29:0/4088323746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee1806b750 msgr2=0x7fee1806db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:55.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 --2- 172.21.15.29:0/4088323746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee1806b750 0x7fee1806db90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fee180ce730 tx=0x7fee1001c730 comp rx=0 tx=0).stop 2024-04-25T00:35:55.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 -- 172.21.15.29:0/4088323746 shutdown_connections 2024-04-25T00:35:55.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 --2- 172.21.15.29:0/4088323746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee1806b750 0x7fee1806db90 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 --2- 172.21.15.29:0/4088323746 >> v2:172.21.15.157:3300/0 conn(0x7fee180ccfd0 0x7fee1806b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 --2- 172.21.15.29:0/4088323746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee1815e1b0 0x7fee18072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 -- 172.21.15.29:0/4088323746 >> 172.21.15.29:0/4088323746 conn(0x7fee180b2920 msgr2=0x7fee180b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:55.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 -- 172.21.15.29:0/4088323746 shutdown_connections 2024-04-25T00:35:55.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 -- 172.21.15.29:0/4088323746 wait complete. 2024-04-25T00:35:55.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 Processor -- start 2024-04-25T00:35:55.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 -- start start 2024-04-25T00:35:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee1806b750 0x7fee18161460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee180ccfd0 0x7fee1815eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee1815e1b0 0x7fee18156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fee18072b50 con 0x7fee180ccfd0 2024-04-25T00:35:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee180d0510 con 0x7fee1815e1b0 2024-04-25T00:35:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fee180cd660 con 0x7fee1806b750 2024-04-25T00:35:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee0ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee180ccfd0 0x7fee1815eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee0ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee180ccfd0 0x7fee1815eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52494/0 (socket says 172.21.15.29:52494) 2024-04-25T00:35:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee0ffff640 1 -- 172.21.15.29:0/144137440 learned_addr learned my addr 172.21.15.29:0/144137440 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1cc9b640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee1806b750 0x7fee18161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1d49c640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee1815e1b0 0x7fee18156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee0ffff640 1 -- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee1806b750 msgr2=0x7fee18161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee0ffff640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee1806b750 0x7fee18161460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee0ffff640 1 -- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee1815e1b0 msgr2=0x7fee18156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee0ffff640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee1815e1b0 0x7fee18156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee0ffff640 1 -- 172.21.15.29:0/144137440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee181619a0 con 0x7fee180ccfd0 2024-04-25T00:35:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee0ffff640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee180ccfd0 0x7fee1815eae0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fee0800d800 tx=0x7fee0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee0d7fa640 1 -- 172.21.15.29:0/144137440 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08018020 con 0x7fee180ccfd0 2024-04-25T00:35:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee0d7fa640 1 -- 172.21.15.29:0/144137440 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee080096a0 con 0x7fee180ccfd0 2024-04-25T00:35:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 -- 172.21.15.29:0/144137440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1815f020 con 0x7fee180ccfd0 2024-04-25T00:35:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee0d7fa640 1 -- 172.21.15.29:0/144137440 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08009a50 con 0x7fee180ccfd0 2024-04-25T00:35:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.148+0000 7fee1eeff640 1 -- 172.21.15.29:0/144137440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee181538f0 con 0x7fee180ccfd0 2024-04-25T00:35:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.152+0000 7fee1eeff640 1 -- 172.21.15.29:0/144137440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee18069270 con 0x7fee180ccfd0 2024-04-25T00:35:55.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.152+0000 7fee0d7fa640 1 -- 172.21.15.29:0/144137440 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fee0801b020 con 0x7fee180ccfd0 2024-04-25T00:35:55.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.152+0000 7fee0d7fa640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fee04067c20 0x7fee0406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:55.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.152+0000 7fee0d7fa640 1 -- 172.21.15.29:0/144137440 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fee08086b80 con 0x7fee180ccfd0 2024-04-25T00:35:55.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.152+0000 7fee1cc9b640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fee04067c20 0x7fee0406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:55.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.152+0000 7fee1cc9b640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fee04067c20 0x7fee0406a0e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fee180b0ed0 tx=0x7fee14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:55.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.156+0000 7fee0d7fa640 1 -- 172.21.15.29:0/144137440 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fee08022020 con 0x7fee180ccfd0 2024-04-25T00:35:55.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.368+0000 7fee1eeff640 1 -- 172.21.15.29:0/144137440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fee180721c0 con 0x7fee180ccfd0 2024-04-25T00:35:55.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.372+0000 7fee0d7fa640 1 -- 172.21.15.29:0/144137440 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1528 (secure 0 0 0) 0x7fee08057440 con 0x7fee180ccfd0 2024-04-25T00:35:55.374 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:55.374 INFO:teuthology.orchestra.run.smithi029.stdout:{"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":282,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":70,"num_osds":8,"num_up_osds":8,"osd_up_since":1714005083,"num_in_osds":8,"osd_in_since":1714005074,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":284430336,"bytes_avail":772809682944,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":43,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6822}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-04-25T00:35:37.531519+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:35:55.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.372+0000 7fee1eeff640 1 -- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fee04067c20 msgr2=0x7fee0406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:55.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.372+0000 7fee1eeff640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fee04067c20 0x7fee0406a0e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fee180b0ed0 tx=0x7fee14008040 comp rx=0 tx=0).stop 2024-04-25T00:35:55.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.372+0000 7fee1eeff640 1 -- 172.21.15.29:0/144137440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee180ccfd0 msgr2=0x7fee1815eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:55.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.372+0000 7fee1eeff640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee180ccfd0 0x7fee1815eae0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fee0800d800 tx=0x7fee0800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.372+0000 7fee1eeff640 1 -- 172.21.15.29:0/144137440 shutdown_connections 2024-04-25T00:35:55.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.372+0000 7fee1eeff640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fee04067c20 0x7fee0406a0e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.372+0000 7fee1eeff640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee1815e1b0 0x7fee18156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.372+0000 7fee1eeff640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee180ccfd0 0x7fee1815eae0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.372+0000 7fee1eeff640 1 --2- 172.21.15.29:0/144137440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee1806b750 0x7fee18161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.372+0000 7fee1eeff640 1 -- 172.21.15.29:0/144137440 >> 172.21.15.29:0/144137440 conn(0x7fee180b2920 msgr2=0x7fee180b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:55.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.372+0000 7fee1eeff640 1 -- 172.21.15.29:0/144137440 shutdown_connections 2024-04-25T00:35:55.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.372+0000 7fee1eeff640 1 -- 172.21.15.29:0/144137440 wait complete. 2024-04-25T00:35:55.391 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T00:35:55.391 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:35:55.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b9350640 1 Processor -- start 2024-04-25T00:35:55.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b9350640 1 -- start start 2024-04-25T00:35:55.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b9350640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb5b415e200 0x7fb5b40d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:55.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b9350640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5b40cf690 0x7fb5b40d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:55.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b9350640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5b40cfc10 con 0x7fb5b40d2a90 2024-04-25T00:35:55.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b9350640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb5b40d2ee0 con 0x7fb5b415e200 2024-04-25T00:35:55.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b9350640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b40b38d0 con 0x7fb5b40cf690 2024-04-25T00:35:55.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb5b415e200 0x7fb5b40d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:55.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b37fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb5b40d2a90 0x7fb5b40d05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:51362/0 (socket says 172.21.15.29:51362) 2024-04-25T00:35:55.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b37fe640 1 -- 172.21.15.29:0/1315820915 learned_addr learned my addr 172.21.15.29:0/1315820915 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:55.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5abfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5b40cf690 0x7fb5b40d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:55.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b27fc640 1 -- 172.21.15.29:0/1315820915 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 398063269 0 0) 0x7fb5b40cfc10 con 0x7fb5b40d2a90 2024-04-25T00:35:55.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b27fc640 1 -- 172.21.15.29:0/1315820915 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5a0003530 con 0x7fb5b40d2a90 2024-04-25T00:35:55.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b2ffd640 1 -- 172.21.15.29:0/1315820915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5b40cf690 msgr2=0x7fb5b40d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:55.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b2ffd640 1 --2- 172.21.15.29:0/1315820915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5b40cf690 0x7fb5b40d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b2ffd640 1 -- 172.21.15.29:0/1315820915 >> v1:172.21.15.157:6789/0 conn(0x7fb5b40d2a90 legacy=0x7fb5b40d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:55.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.500+0000 7fb5b2ffd640 1 -- 172.21.15.29:0/1315820915 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b40cfef0 con 0x7fb5b415e200 2024-04-25T00:35:55.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b2ffd640 1 --2- 172.21.15.29:0/1315820915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b415e200 0x7fb5b40d47f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40097c0 tx=0x7fb5a401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f6de48751b6ce27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:55.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b27fc640 1 -- 172.21.15.29:0/1315820915 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40a7070 con 0x7fb5b415e200 2024-04-25T00:35:55.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b27fc640 1 -- 172.21.15.29:0/1315820915 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a40b0070 con 0x7fb5b415e200 2024-04-25T00:35:55.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b27fc640 1 -- 172.21.15.29:0/1315820915 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40ab890 con 0x7fb5b415e200 2024-04-25T00:35:55.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 -- 172.21.15.29:0/1315820915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b415e200 msgr2=0x7fb5b40d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:55.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 --2- 172.21.15.29:0/1315820915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b415e200 0x7fb5b40d47f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40097c0 tx=0x7fb5a401c470 comp rx=0 tx=0).stop 2024-04-25T00:35:55.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 -- 172.21.15.29:0/1315820915 shutdown_connections 2024-04-25T00:35:55.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 --2- 172.21.15.29:0/1315820915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5b40cf690 0x7fb5b40d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 --2- 172.21.15.29:0/1315820915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b415e200 0x7fb5b40d47f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 -- 172.21.15.29:0/1315820915 >> 172.21.15.29:0/1315820915 conn(0x7fb5b409d280 msgr2=0x7fb5b40d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:55.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 -- 172.21.15.29:0/1315820915 shutdown_connections 2024-04-25T00:35:55.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 -- 172.21.15.29:0/1315820915 wait complete. 2024-04-25T00:35:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 Processor -- start 2024-04-25T00:35:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 -- start start 2024-04-25T00:35:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5b40cf690 0x7fb5b4157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b40d2a90 0x7fb5b40ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5b415e200 0x7fb5b40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b415e760 con 0x7fb5b40cf690 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b415e5e0 con 0x7fb5b40d2a90 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.504+0000 7fb5b9350640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b416ae30 con 0x7fb5b415e200 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5b2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5b40cf690 0x7fb5b4157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5b2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5b40cf690 0x7fb5b4157ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52510/0 (socket says 172.21.15.29:52510) 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5b2ffd640 1 -- 172.21.15.29:0/2352479433 learned_addr learned my addr 172.21.15.29:0/2352479433 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5abfff640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b40d2a90 0x7fb5b40ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5b37fe640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5b415e200 0x7fb5b40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5b2ffd640 1 -- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5b415e200 msgr2=0x7fb5b40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5b2ffd640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5b415e200 0x7fb5b40b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5b2ffd640 1 -- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b40d2a90 msgr2=0x7fb5b40ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5b2ffd640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b40d2a90 0x7fb5b40ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5b2ffd640 1 -- 172.21.15.29:0/2352479433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b415f8b0 con 0x7fb5b40cf690 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5abfff640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b40d2a90 0x7fb5b40ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5b2ffd640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5b40cf690 0x7fb5b4157ed0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40ad040 tx=0x7fb5a401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5ab7fe640 1 -- 172.21.15.29:0/2352479433 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40bf610 con 0x7fb5b40cf690 2024-04-25T00:35:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5ab7fe640 1 -- 172.21.15.29:0/2352479433 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a40abe50 con 0x7fb5b40cf690 2024-04-25T00:35:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5b9350640 1 -- 172.21.15.29:0/2352479433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b415fb90 con 0x7fb5b40cf690 2024-04-25T00:35:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5ab7fe640 1 -- 172.21.15.29:0/2352479433 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40bfa10 con 0x7fb5b40cf690 2024-04-25T00:35:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5b9350640 1 -- 172.21.15.29:0/2352479433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b4160150 con 0x7fb5b40cf690 2024-04-25T00:35:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5b9350640 1 -- 172.21.15.29:0/2352479433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b4069270 con 0x7fb5b40cf690 2024-04-25T00:35:55.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5ab7fe640 1 -- 172.21.15.29:0/2352479433 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb5a40bd020 con 0x7fb5b40cf690 2024-04-25T00:35:55.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5ab7fe640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb594067c20 0x7fb59406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:55.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5ab7fe640 1 -- 172.21.15.29:0/2352479433 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fb5a41208e0 con 0x7fb5b40cf690 2024-04-25T00:35:55.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.508+0000 7fb5abfff640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb594067c20 0x7fb59406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:55.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.512+0000 7fb5abfff640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb594067c20 0x7fb59406a0e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40c78c0 tx=0x7fb59c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:55.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.512+0000 7fb5ab7fe640 1 -- 172.21.15.29:0/2352479433 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb5a4016090 con 0x7fb5b40cf690 2024-04-25T00:35:55.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.664+0000 7fb5b9350640 1 -- 172.21.15.29:0/2352479433 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5b4168c00 con 0x7fb594067c20 2024-04-25T00:35:55.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.684+0000 7fb5ab7fe640 1 -- 172.21.15.29:0/2352479433 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7fb5b4168c00 con 0x7fb594067c20 2024-04-25T00:35:55.688 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:55.689 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:35:55.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.688+0000 7fb5b9350640 1 -- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb594067c20 msgr2=0x7fb59406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:55.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.688+0000 7fb5b9350640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb594067c20 0x7fb59406a0e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40c78c0 tx=0x7fb59c008040 comp rx=0 tx=0).stop 2024-04-25T00:35:55.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.688+0000 7fb5b9350640 1 -- 172.21.15.29:0/2352479433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5b40cf690 msgr2=0x7fb5b4157ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:55.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.688+0000 7fb5b9350640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5b40cf690 0x7fb5b4157ed0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40ad040 tx=0x7fb5a401acb0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.688+0000 7fb5b9350640 1 -- 172.21.15.29:0/2352479433 shutdown_connections 2024-04-25T00:35:55.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.688+0000 7fb5b9350640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5b415e200 0x7fb5b40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.688+0000 7fb5b9350640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb594067c20 0x7fb59406a0e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.688+0000 7fb5b9350640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b40d2a90 0x7fb5b40ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.688+0000 7fb5b9350640 1 --2- 172.21.15.29:0/2352479433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5b40cf690 0x7fb5b4157ed0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:55.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.688+0000 7fb5b9350640 1 -- 172.21.15.29:0/2352479433 >> 172.21.15.29:0/2352479433 conn(0x7fb5b409d280 msgr2=0x7fb5b40b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:55.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.688+0000 7fb5b9350640 1 -- 172.21.15.29:0/2352479433 shutdown_connections 2024-04-25T00:35:55.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:55.688+0000 7fb5b9350640 1 -- 172.21.15.29:0/2352479433 wait complete. 2024-04-25T00:35:55.705 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-04-25T00:35:55.536843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282316,"kb_used_data":7880,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754692404,"statfs":{"total":773094113280,"available":772805021696,"internally_reserved":0,"allocated":8069120,"data_stored":4060180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.388268"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203249+0000","last_change":"2024-04-25T00:35:50.218836+0000","last_active":"2024-04-25T00:35:51.203249+0000","last_peered":"2024-04-25T00:35:51.203249+0000","last_clean":"2024-04-25T00:35:51.203249+0000","last_became_active":"2024-04-25T00:35:50.218565+0000","last_became_peered":"2024-04-25T00:35:50.218565+0000","last_unstale":"2024-04-25T00:35:51.203249+0000","last_undegraded":"2024-04-25T00:35:51.203249+0000","last_fullsized":"2024-04-25T00:35:51.203249+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:49.507678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.589210+0000","last_change":"2024-04-25T00:35:50.212533+0000","last_active":"2024-04-25T00:35:51.589210+0000","last_peered":"2024-04-25T00:35:51.589210+0000","last_clean":"2024-04-25T00:35:51.589210+0000","last_became_active":"2024-04-25T00:35:50.211826+0000","last_became_peered":"2024-04-25T00:35:50.211826+0000","last_unstale":"2024-04-25T00:35:51.589210+0000","last_undegraded":"2024-04-25T00:35:51.589210+0000","last_fullsized":"2024-04-25T00:35:51.589210+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:28.691635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091824+0000","last_change":"2024-04-25T00:35:50.206984+0000","last_active":"2024-04-25T00:35:52.091824+0000","last_peered":"2024-04-25T00:35:52.091824+0000","last_clean":"2024-04-25T00:35:52.091824+0000","last_became_active":"2024-04-25T00:35:50.206697+0000","last_became_peered":"2024-04-25T00:35:50.206697+0000","last_unstale":"2024-04-25T00:35:52.091824+0000","last_undegraded":"2024-04-25T00:35:52.091824+0000","last_fullsized":"2024-04-25T00:35:52.091824+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:59.790434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.090224+0000","last_change":"2024-04-25T00:35:50.233152+0000","last_active":"2024-04-25T00:35:52.090224+0000","last_peered":"2024-04-25T00:35:52.090224+0000","last_clean":"2024-04-25T00:35:52.090224+0000","last_became_active":"2024-04-25T00:35:50.232832+0000","last_became_peered":"2024-04-25T00:35:50.232832+0000","last_unstale":"2024-04-25T00:35:52.090224+0000","last_undegraded":"2024-04-25T00:35:52.090224+0000","last_fullsized":"2024-04-25T00:35:52.090224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:07.819731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.090177+0000","last_change":"2024-04-25T00:35:50.217003+0000","last_active":"2024-04-25T00:35:52.090177+0000","last_peered":"2024-04-25T00:35:52.090177+0000","last_clean":"2024-04-25T00:35:52.090177+0000","last_became_active":"2024-04-25T00:35:50.216705+0000","last_became_peered":"2024-04-25T00:35:50.216705+0000","last_unstale":"2024-04-25T00:35:52.090177+0000","last_undegraded":"2024-04-25T00:35:52.090177+0000","last_fullsized":"2024-04-25T00:35:52.090177+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:59.001749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588132+0000","last_change":"2024-04-25T00:35:50.228663+0000","last_active":"2024-04-25T00:35:51.588132+0000","last_peered":"2024-04-25T00:35:51.588132+0000","last_clean":"2024-04-25T00:35:51.588132+0000","last_became_active":"2024-04-25T00:35:50.228029+0000","last_became_peered":"2024-04-25T00:35:50.228029+0000","last_unstale":"2024-04-25T00:35:51.588132+0000","last_undegraded":"2024-04-25T00:35:51.588132+0000","last_fullsized":"2024-04-25T00:35:51.588132+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:50.971516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204105+0000","last_change":"2024-04-25T00:35:50.196683+0000","last_active":"2024-04-25T00:35:51.204105+0000","last_peered":"2024-04-25T00:35:51.204105+0000","last_clean":"2024-04-25T00:35:51.204105+0000","last_became_active":"2024-04-25T00:35:50.195978+0000","last_became_peered":"2024-04-25T00:35:50.195978+0000","last_unstale":"2024-04-25T00:35:51.204105+0000","last_undegraded":"2024-04-25T00:35:51.204105+0000","last_fullsized":"2024-04-25T00:35:51.204105+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:09.403470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588135+0000","last_change":"2024-04-25T00:35:50.228842+0000","last_active":"2024-04-25T00:35:51.588135+0000","last_peered":"2024-04-25T00:35:51.588135+0000","last_clean":"2024-04-25T00:35:51.588135+0000","last_became_active":"2024-04-25T00:35:50.228223+0000","last_became_peered":"2024-04-25T00:35:50.228223+0000","last_unstale":"2024-04-25T00:35:51.588135+0000","last_undegraded":"2024-04-25T00:35:51.588135+0000","last_fullsized":"2024-04-25T00:35:51.588135+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:48.996177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092028+0000","last_change":"2024-04-25T00:35:50.208954+0000","last_active":"2024-04-25T00:35:52.092028+0000","last_peered":"2024-04-25T00:35:52.092028+0000","last_clean":"2024-04-25T00:35:52.092028+0000","last_became_active":"2024-04-25T00:35:50.208667+0000","last_became_peered":"2024-04-25T00:35:50.208667+0000","last_unstale":"2024-04-25T00:35:52.092028+0000","last_undegraded":"2024-04-25T00:35:52.092028+0000","last_fullsized":"2024-04-25T00:35:52.092028+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:24.336521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202586+0000","last_change":"2024-04-25T00:35:50.200817+0000","last_active":"2024-04-25T00:35:51.202586+0000","last_peered":"2024-04-25T00:35:51.202586+0000","last_clean":"2024-04-25T00:35:51.202586+0000","last_became_active":"2024-04-25T00:35:50.199957+0000","last_became_peered":"2024-04-25T00:35:50.199957+0000","last_unstale":"2024-04-25T00:35:51.202586+0000","last_undegraded":"2024-04-25T00:35:51.202586+0000","last_fullsized":"2024-04-25T00:35:51.202586+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:50.971516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202696+0000","last_change":"2024-04-25T00:35:50.212930+0000","last_active":"2024-04-25T00:35:51.202696+0000","last_peered":"2024-04-25T00:35:51.202696+0000","last_clean":"2024-04-25T00:35:51.202696+0000","last_became_active":"2024-04-25T00:35:50.211866+0000","last_became_peered":"2024-04-25T00:35:50.211866+0000","last_unstale":"2024-04-25T00:35:51.202696+0000","last_undegraded":"2024-04-25T00:35:51.202696+0000","last_fullsized":"2024-04-25T00:35:51.202696+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:47.678076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.090627+0000","last_change":"2024-04-25T00:35:50.208972+0000","last_active":"2024-04-25T00:35:52.090627+0000","last_peered":"2024-04-25T00:35:52.090627+0000","last_clean":"2024-04-25T00:35:52.090627+0000","last_became_active":"2024-04-25T00:35:50.208343+0000","last_became_peered":"2024-04-25T00:35:50.208343+0000","last_unstale":"2024-04-25T00:35:52.090627+0000","last_undegraded":"2024-04-25T00:35:52.090627+0000","last_fullsized":"2024-04-25T00:35:52.090627+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:52.238530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089747+0000","last_change":"2024-04-25T00:35:50.228954+0000","last_active":"2024-04-25T00:35:52.089747+0000","last_peered":"2024-04-25T00:35:52.089747+0000","last_clean":"2024-04-25T00:35:52.089747+0000","last_became_active":"2024-04-25T00:35:50.228644+0000","last_became_peered":"2024-04-25T00:35:50.228644+0000","last_unstale":"2024-04-25T00:35:52.089747+0000","last_undegraded":"2024-04-25T00:35:52.089747+0000","last_fullsized":"2024-04-25T00:35:52.089747+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:17.997355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091268+0000","last_change":"2024-04-25T00:35:50.189250+0000","last_active":"2024-04-25T00:35:52.091268+0000","last_peered":"2024-04-25T00:35:52.091268+0000","last_clean":"2024-04-25T00:35:52.091268+0000","last_became_active":"2024-04-25T00:35:50.185920+0000","last_became_peered":"2024-04-25T00:35:50.185920+0000","last_unstale":"2024-04-25T00:35:52.091268+0000","last_undegraded":"2024-04-25T00:35:52.091268+0000","last_fullsized":"2024-04-25T00:35:52.091268+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:23.674777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.088788+0000","last_change":"2024-04-25T00:35:50.207445+0000","last_active":"2024-04-25T00:35:52.088788+0000","last_peered":"2024-04-25T00:35:52.088788+0000","last_clean":"2024-04-25T00:35:52.088788+0000","last_became_active":"2024-04-25T00:35:50.207142+0000","last_became_peered":"2024-04-25T00:35:50.207142+0000","last_unstale":"2024-04-25T00:35:52.088788+0000","last_undegraded":"2024-04-25T00:35:52.088788+0000","last_fullsized":"2024-04-25T00:35:52.088788+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:44.765172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.200972+0000","last_change":"2024-04-25T00:35:50.230662+0000","last_active":"2024-04-25T00:35:51.200972+0000","last_peered":"2024-04-25T00:35:51.200972+0000","last_clean":"2024-04-25T00:35:51.200972+0000","last_became_active":"2024-04-25T00:35:50.230020+0000","last_became_peered":"2024-04-25T00:35:50.230020+0000","last_unstale":"2024-04-25T00:35:51.200972+0000","last_undegraded":"2024-04-25T00:35:51.200972+0000","last_fullsized":"2024-04-25T00:35:51.200972+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:07.922034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092422+0000","last_change":"2024-04-25T00:35:50.208680+0000","last_active":"2024-04-25T00:35:52.092422+0000","last_peered":"2024-04-25T00:35:52.092422+0000","last_clean":"2024-04-25T00:35:52.092422+0000","last_became_active":"2024-04-25T00:35:50.208294+0000","last_became_peered":"2024-04-25T00:35:50.208294+0000","last_unstale":"2024-04-25T00:35:52.092422+0000","last_undegraded":"2024-04-25T00:35:52.092422+0000","last_fullsized":"2024-04-25T00:35:52.092422+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:05.215451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089648+0000","last_change":"2024-04-25T00:35:50.233176+0000","last_active":"2024-04-25T00:35:52.089648+0000","last_peered":"2024-04-25T00:35:52.089648+0000","last_clean":"2024-04-25T00:35:52.089648+0000","last_became_active":"2024-04-25T00:35:50.232887+0000","last_became_peered":"2024-04-25T00:35:50.232887+0000","last_unstale":"2024-04-25T00:35:52.089648+0000","last_undegraded":"2024-04-25T00:35:52.089648+0000","last_fullsized":"2024-04-25T00:35:52.089648+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:22.775445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588633+0000","last_change":"2024-04-25T00:35:50.228429+0000","last_active":"2024-04-25T00:35:51.588633+0000","last_peered":"2024-04-25T00:35:51.588633+0000","last_clean":"2024-04-25T00:35:51.588633+0000","last_became_active":"2024-04-25T00:35:50.225794+0000","last_became_peered":"2024-04-25T00:35:50.225794+0000","last_unstale":"2024-04-25T00:35:51.588633+0000","last_undegraded":"2024-04-25T00:35:51.588633+0000","last_fullsized":"2024-04-25T00:35:51.588633+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:26:00.591734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.088850+0000","last_change":"2024-04-25T00:35:50.199758+0000","last_active":"2024-04-25T00:35:52.088850+0000","last_peered":"2024-04-25T00:35:52.088850+0000","last_clean":"2024-04-25T00:35:52.088850+0000","last_became_active":"2024-04-25T00:35:50.199367+0000","last_became_peered":"2024-04-25T00:35:50.199367+0000","last_unstale":"2024-04-25T00:35:52.088850+0000","last_undegraded":"2024-04-25T00:35:52.088850+0000","last_fullsized":"2024-04-25T00:35:52.088850+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:33.770812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.201546+0000","last_change":"2024-04-25T00:35:50.200658+0000","last_active":"2024-04-25T00:35:51.201546+0000","last_peered":"2024-04-25T00:35:51.201546+0000","last_clean":"2024-04-25T00:35:51.201546+0000","last_became_active":"2024-04-25T00:35:50.199451+0000","last_became_peered":"2024-04-25T00:35:50.199451+0000","last_unstale":"2024-04-25T00:35:51.201546+0000","last_undegraded":"2024-04-25T00:35:51.201546+0000","last_fullsized":"2024-04-25T00:35:51.201546+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:54.993129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.201761+0000","last_change":"2024-04-25T00:35:50.200451+0000","last_active":"2024-04-25T00:35:51.201761+0000","last_peered":"2024-04-25T00:35:51.201761+0000","last_clean":"2024-04-25T00:35:51.201761+0000","last_became_active":"2024-04-25T00:35:50.199725+0000","last_became_peered":"2024-04-25T00:35:50.199725+0000","last_unstale":"2024-04-25T00:35:51.201761+0000","last_undegraded":"2024-04-25T00:35:51.201761+0000","last_fullsized":"2024-04-25T00:35:51.201761+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:35.617518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.200626+0000","last_change":"2024-04-25T00:35:50.208014+0000","last_active":"2024-04-25T00:35:51.200626+0000","last_peered":"2024-04-25T00:35:51.200626+0000","last_clean":"2024-04-25T00:35:51.200626+0000","last_became_active":"2024-04-25T00:35:50.206654+0000","last_became_peered":"2024-04-25T00:35:50.206654+0000","last_unstale":"2024-04-25T00:35:51.200626+0000","last_undegraded":"2024-04-25T00:35:51.200626+0000","last_fullsized":"2024-04-25T00:35:51.200626+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:04.372088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.199883+0000","last_change":"2024-04-25T00:35:50.242161+0000","last_active":"2024-04-25T00:35:51.199883+0000","last_peered":"2024-04-25T00:35:51.199883+0000","last_clean":"2024-04-25T00:35:51.199883+0000","last_became_active":"2024-04-25T00:35:50.241803+0000","last_became_peered":"2024-04-25T00:35:50.241803+0000","last_unstale":"2024-04-25T00:35:51.199883+0000","last_undegraded":"2024-04-25T00:35:51.199883+0000","last_fullsized":"2024-04-25T00:35:51.199883+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:00.123980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203043+0000","last_change":"2024-04-25T00:35:50.215247+0000","last_active":"2024-04-25T00:35:51.203043+0000","last_peered":"2024-04-25T00:35:51.203043+0000","last_clean":"2024-04-25T00:35:51.203043+0000","last_became_active":"2024-04-25T00:35:50.214354+0000","last_became_peered":"2024-04-25T00:35:50.214354+0000","last_unstale":"2024-04-25T00:35:51.203043+0000","last_undegraded":"2024-04-25T00:35:51.203043+0000","last_fullsized":"2024-04-25T00:35:51.203043+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:58.856012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.205024+0000","last_change":"2024-04-25T00:35:50.212628+0000","last_active":"2024-04-25T00:35:51.205024+0000","last_peered":"2024-04-25T00:35:51.205024+0000","last_clean":"2024-04-25T00:35:51.205024+0000","last_became_active":"2024-04-25T00:35:50.211748+0000","last_became_peered":"2024-04-25T00:35:50.211748+0000","last_unstale":"2024-04-25T00:35:51.205024+0000","last_undegraded":"2024-04-25T00:35:51.205024+0000","last_fullsized":"2024-04-25T00:35:51.205024+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:17.997355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092236+0000","last_change":"2024-04-25T00:35:50.216069+0000","last_active":"2024-04-25T00:35:52.092236+0000","last_peered":"2024-04-25T00:35:52.092236+0000","last_clean":"2024-04-25T00:35:52.092236+0000","last_became_active":"2024-04-25T00:35:50.215796+0000","last_became_peered":"2024-04-25T00:35:50.215796+0000","last_unstale":"2024-04-25T00:35:52.092236+0000","last_undegraded":"2024-04-25T00:35:52.092236+0000","last_fullsized":"2024-04-25T00:35:52.092236+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:36.963156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202365+0000","last_change":"2024-04-25T00:35:50.201636+0000","last_active":"2024-04-25T00:35:51.202365+0000","last_peered":"2024-04-25T00:35:51.202365+0000","last_clean":"2024-04-25T00:35:51.202365+0000","last_became_active":"2024-04-25T00:35:50.199849+0000","last_became_peered":"2024-04-25T00:35:50.199849+0000","last_unstale":"2024-04-25T00:35:51.202365+0000","last_undegraded":"2024-04-25T00:35:51.202365+0000","last_fullsized":"2024-04-25T00:35:51.202365+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:48.996177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092612+0000","last_change":"2024-04-25T00:35:50.206490+0000","last_active":"2024-04-25T00:35:52.092612+0000","last_peered":"2024-04-25T00:35:52.092612+0000","last_clean":"2024-04-25T00:35:52.092612+0000","last_became_active":"2024-04-25T00:35:50.206184+0000","last_became_peered":"2024-04-25T00:35:50.206184+0000","last_unstale":"2024-04-25T00:35:52.092612+0000","last_undegraded":"2024-04-25T00:35:52.092612+0000","last_fullsized":"2024-04-25T00:35:52.092612+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:36.130476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204448+0000","last_change":"2024-04-25T00:35:50.230056+0000","last_active":"2024-04-25T00:35:51.204448+0000","last_peered":"2024-04-25T00:35:51.204448+0000","last_clean":"2024-04-25T00:35:51.204448+0000","last_became_active":"2024-04-25T00:35:50.229697+0000","last_became_peered":"2024-04-25T00:35:50.229697+0000","last_unstale":"2024-04-25T00:35:51.204448+0000","last_undegraded":"2024-04-25T00:35:51.204448+0000","last_fullsized":"2024-04-25T00:35:51.204448+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:44.765172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204503+0000","last_change":"2024-04-25T00:35:50.185426+0000","last_active":"2024-04-25T00:35:51.204503+0000","last_peered":"2024-04-25T00:35:51.204503+0000","last_clean":"2024-04-25T00:35:51.204503+0000","last_became_active":"2024-04-25T00:35:50.184675+0000","last_became_peered":"2024-04-25T00:35:50.184675+0000","last_unstale":"2024-04-25T00:35:51.204503+0000","last_undegraded":"2024-04-25T00:35:51.204503+0000","last_fullsized":"2024-04-25T00:35:51.204503+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:33.770812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588231+0000","last_change":"2024-04-25T00:35:50.231446+0000","last_active":"2024-04-25T00:35:51.588231+0000","last_peered":"2024-04-25T00:35:51.588231+0000","last_clean":"2024-04-25T00:35:51.588231+0000","last_became_active":"2024-04-25T00:35:50.228644+0000","last_became_peered":"2024-04-25T00:35:50.228644+0000","last_unstale":"2024-04-25T00:35:51.588231+0000","last_undegraded":"2024-04-25T00:35:51.588231+0000","last_fullsized":"2024-04-25T00:35:51.588231+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:02.142046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202603+0000","last_change":"2024-04-25T00:35:50.239150+0000","last_active":"2024-04-25T00:35:51.202603+0000","last_peered":"2024-04-25T00:35:51.202603+0000","last_clean":"2024-04-25T00:35:51.202603+0000","last_became_active":"2024-04-25T00:35:50.238677+0000","last_became_peered":"2024-04-25T00:35:50.238677+0000","last_unstale":"2024-04-25T00:35:51.202603+0000","last_undegraded":"2024-04-25T00:35:51.202603+0000","last_fullsized":"2024-04-25T00:35:51.202603+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:05.975194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203770+0000","last_change":"2024-04-25T00:35:49.166557+0000","last_active":"2024-04-25T00:35:51.203770+0000","last_peered":"2024-04-25T00:35:51.203770+0000","last_clean":"2024-04-25T00:35:51.203770+0000","last_became_active":"2024-04-25T00:35:49.165245+0000","last_became_peered":"2024-04-25T00:35:49.165245+0000","last_unstale":"2024-04-25T00:35:51.203770+0000","last_undegraded":"2024-04-25T00:35:51.203770+0000","last_fullsized":"2024-04-25T00:35:51.203770+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:16.989199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092806+0000","last_change":"2024-04-25T00:35:50.208909+0000","last_active":"2024-04-25T00:35:52.092806+0000","last_peered":"2024-04-25T00:35:52.092806+0000","last_clean":"2024-04-25T00:35:52.092806+0000","last_became_active":"2024-04-25T00:35:50.208382+0000","last_became_peered":"2024-04-25T00:35:50.208382+0000","last_unstale":"2024-04-25T00:35:52.092806+0000","last_undegraded":"2024-04-25T00:35:52.092806+0000","last_fullsized":"2024-04-25T00:35:52.092806+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:13.509256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.093194+0000","last_change":"2024-04-25T00:35:49.172238+0000","last_active":"2024-04-25T00:35:52.093194+0000","last_peered":"2024-04-25T00:35:52.093194+0000","last_clean":"2024-04-25T00:35:52.093194+0000","last_became_active":"2024-04-25T00:35:49.171747+0000","last_became_peered":"2024-04-25T00:35:49.171747+0000","last_unstale":"2024-04-25T00:35:52.093194+0000","last_undegraded":"2024-04-25T00:35:52.093194+0000","last_fullsized":"2024-04-25T00:35:52.093194+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:49.963360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588592+0000","last_change":"2024-04-25T00:35:50.235945+0000","last_active":"2024-04-25T00:35:51.588592+0000","last_peered":"2024-04-25T00:35:51.588592+0000","last_clean":"2024-04-25T00:35:51.588592+0000","last_became_active":"2024-04-25T00:35:50.232455+0000","last_became_peered":"2024-04-25T00:35:50.232455+0000","last_unstale":"2024-04-25T00:35:51.588592+0000","last_undegraded":"2024-04-25T00:35:51.588592+0000","last_fullsized":"2024-04-25T00:35:51.588592+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:01.995731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202477+0000","last_change":"2024-04-25T00:35:49.187713+0000","last_active":"2024-04-25T00:35:51.202477+0000","last_peered":"2024-04-25T00:35:51.202477+0000","last_clean":"2024-04-25T00:35:51.202477+0000","last_became_active":"2024-04-25T00:35:49.186956+0000","last_became_peered":"2024-04-25T00:35:49.186956+0000","last_unstale":"2024-04-25T00:35:51.202477+0000","last_undegraded":"2024-04-25T00:35:51.202477+0000","last_fullsized":"2024-04-25T00:35:51.202477+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:25.728034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203082+0000","last_change":"2024-04-25T00:35:50.208041+0000","last_active":"2024-04-25T00:35:51.203082+0000","last_peered":"2024-04-25T00:35:51.203082+0000","last_clean":"2024-04-25T00:35:51.203082+0000","last_became_active":"2024-04-25T00:35:50.207367+0000","last_became_peered":"2024-04-25T00:35:50.207367+0000","last_unstale":"2024-04-25T00:35:51.203082+0000","last_undegraded":"2024-04-25T00:35:51.203082+0000","last_fullsized":"2024-04-25T00:35:51.203082+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:05.829437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588851+0000","last_change":"2024-04-25T00:35:49.188864+0000","last_active":"2024-04-25T00:35:51.588851+0000","last_peered":"2024-04-25T00:35:51.588851+0000","last_clean":"2024-04-25T00:35:51.588851+0000","last_became_active":"2024-04-25T00:35:49.183783+0000","last_became_peered":"2024-04-25T00:35:49.183783+0000","last_unstale":"2024-04-25T00:35:51.588851+0000","last_undegraded":"2024-04-25T00:35:51.588851+0000","last_fullsized":"2024-04-25T00:35:51.588851+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:21.365941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204684+0000","last_change":"2024-04-25T00:35:50.210087+0000","last_active":"2024-04-25T00:35:51.204684+0000","last_peered":"2024-04-25T00:35:51.204684+0000","last_clean":"2024-04-25T00:35:51.204684+0000","last_became_active":"2024-04-25T00:35:50.209334+0000","last_became_peered":"2024-04-25T00:35:50.209334+0000","last_unstale":"2024-04-25T00:35:51.204684+0000","last_undegraded":"2024-04-25T00:35:51.204684+0000","last_fullsized":"2024-04-25T00:35:51.204684+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:39.340439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092891+0000","last_change":"2024-04-25T00:35:49.176223+0000","last_active":"2024-04-25T00:35:52.092891+0000","last_peered":"2024-04-25T00:35:52.092891+0000","last_clean":"2024-04-25T00:35:52.092891+0000","last_became_active":"2024-04-25T00:35:49.175671+0000","last_became_peered":"2024-04-25T00:35:49.175671+0000","last_unstale":"2024-04-25T00:35:52.092891+0000","last_undegraded":"2024-04-25T00:35:52.092891+0000","last_fullsized":"2024-04-25T00:35:52.092891+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:04.967038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.201673+0000","last_change":"2024-04-25T00:35:50.223221+0000","last_active":"2024-04-25T00:35:51.201673+0000","last_peered":"2024-04-25T00:35:51.201673+0000","last_clean":"2024-04-25T00:35:51.201673+0000","last_became_active":"2024-04-25T00:35:50.222805+0000","last_became_peered":"2024-04-25T00:35:50.222805+0000","last_unstale":"2024-04-25T00:35:51.201673+0000","last_undegraded":"2024-04-25T00:35:51.201673+0000","last_fullsized":"2024-04-25T00:35:51.201673+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:13.474838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588312+0000","last_change":"2024-04-25T00:35:49.189623+0000","last_active":"2024-04-25T00:35:51.588312+0000","last_peered":"2024-04-25T00:35:51.588312+0000","last_clean":"2024-04-25T00:35:51.588312+0000","last_became_active":"2024-04-25T00:35:49.188902+0000","last_became_peered":"2024-04-25T00:35:49.188902+0000","last_unstale":"2024-04-25T00:35:51.588312+0000","last_undegraded":"2024-04-25T00:35:51.588312+0000","last_fullsized":"2024-04-25T00:35:51.588312+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:28.542118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089564+0000","last_change":"2024-04-25T00:35:50.201247+0000","last_active":"2024-04-25T00:35:52.089564+0000","last_peered":"2024-04-25T00:35:52.089564+0000","last_clean":"2024-04-25T00:35:52.089564+0000","last_became_active":"2024-04-25T00:35:50.200861+0000","last_became_peered":"2024-04-25T00:35:50.200861+0000","last_unstale":"2024-04-25T00:35:52.089564+0000","last_undegraded":"2024-04-25T00:35:52.089564+0000","last_fullsized":"2024-04-25T00:35:52.089564+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:15.666442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089568+0000","last_change":"2024-04-25T00:35:49.176647+0000","last_active":"2024-04-25T00:35:52.089568+0000","last_peered":"2024-04-25T00:35:52.089568+0000","last_clean":"2024-04-25T00:35:52.089568+0000","last_became_active":"2024-04-25T00:35:49.175953+0000","last_became_peered":"2024-04-25T00:35:49.175953+0000","last_unstale":"2024-04-25T00:35:52.089568+0000","last_undegraded":"2024-04-25T00:35:52.089568+0000","last_fullsized":"2024-04-25T00:35:52.089568+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:12.501099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.088906+0000","last_change":"2024-04-25T00:35:50.220004+0000","last_active":"2024-04-25T00:35:52.088906+0000","last_peered":"2024-04-25T00:35:52.088906+0000","last_clean":"2024-04-25T00:35:52.088906+0000","last_became_active":"2024-04-25T00:35:50.219656+0000","last_became_peered":"2024-04-25T00:35:50.219656+0000","last_unstale":"2024-04-25T00:35:52.088906+0000","last_undegraded":"2024-04-25T00:35:52.088906+0000","last_fullsized":"2024-04-25T00:35:52.088906+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:39.340439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089707+0000","last_change":"2024-04-25T00:35:49.168520+0000","last_active":"2024-04-25T00:35:52.089707+0000","last_peered":"2024-04-25T00:35:52.089707+0000","last_clean":"2024-04-25T00:35:52.089707+0000","last_became_active":"2024-04-25T00:35:49.168120+0000","last_became_peered":"2024-04-25T00:35:49.168120+0000","last_unstale":"2024-04-25T00:35:52.089707+0000","last_undegraded":"2024-04-25T00:35:52.089707+0000","last_fullsized":"2024-04-25T00:35:52.089707+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:44.173013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.205418+0000","last_change":"2024-04-25T00:35:50.193291+0000","last_active":"2024-04-25T00:35:51.205418+0000","last_peered":"2024-04-25T00:35:51.205418+0000","last_clean":"2024-04-25T00:35:51.205418+0000","last_became_active":"2024-04-25T00:35:50.192517+0000","last_became_peered":"2024-04-25T00:35:50.192517+0000","last_unstale":"2024-04-25T00:35:51.205418+0000","last_undegraded":"2024-04-25T00:35:51.205418+0000","last_fullsized":"2024-04-25T00:35:51.205418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:55.595220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203174+0000","last_change":"2024-04-25T00:35:49.174126+0000","last_active":"2024-04-25T00:35:51.203174+0000","last_peered":"2024-04-25T00:35:51.203174+0000","last_clean":"2024-04-25T00:35:51.203174+0000","last_became_active":"2024-04-25T00:35:49.173415+0000","last_became_peered":"2024-04-25T00:35:49.173415+0000","last_unstale":"2024-04-25T00:35:51.203174+0000","last_undegraded":"2024-04-25T00:35:51.203174+0000","last_fullsized":"2024-04-25T00:35:51.203174+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:50.622430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203627+0000","last_change":"2024-04-25T00:35:50.242836+0000","last_active":"2024-04-25T00:35:51.203627+0000","last_peered":"2024-04-25T00:35:51.203627+0000","last_clean":"2024-04-25T00:35:51.203627+0000","last_became_active":"2024-04-25T00:35:50.242031+0000","last_became_peered":"2024-04-25T00:35:50.242031+0000","last_unstale":"2024-04-25T00:35:51.203627+0000","last_undegraded":"2024-04-25T00:35:51.203627+0000","last_fullsized":"2024-04-25T00:35:51.203627+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:15.543514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588894+0000","last_change":"2024-04-25T00:35:49.193426+0000","last_active":"2024-04-25T00:35:51.588894+0000","last_peered":"2024-04-25T00:35:51.588894+0000","last_clean":"2024-04-25T00:35:51.588894+0000","last_became_active":"2024-04-25T00:35:49.192672+0000","last_became_peered":"2024-04-25T00:35:49.192672+0000","last_unstale":"2024-04-25T00:35:51.588894+0000","last_undegraded":"2024-04-25T00:35:51.588894+0000","last_fullsized":"2024-04-25T00:35:51.588894+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:21.767289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.589018+0000","last_change":"2024-04-25T00:35:50.231624+0000","last_active":"2024-04-25T00:35:51.589018+0000","last_peered":"2024-04-25T00:35:51.589018+0000","last_clean":"2024-04-25T00:35:51.589018+0000","last_became_active":"2024-04-25T00:35:50.228072+0000","last_became_peered":"2024-04-25T00:35:50.228072+0000","last_unstale":"2024-04-25T00:35:51.589018+0000","last_undegraded":"2024-04-25T00:35:51.589018+0000","last_fullsized":"2024-04-25T00:35:51.589018+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:33.388855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092388+0000","last_change":"2024-04-25T00:35:49.188084+0000","last_active":"2024-04-25T00:35:52.092388+0000","last_peered":"2024-04-25T00:35:52.092388+0000","last_clean":"2024-04-25T00:35:52.092388+0000","last_became_active":"2024-04-25T00:35:49.187449+0000","last_became_peered":"2024-04-25T00:35:49.187449+0000","last_unstale":"2024-04-25T00:35:52.092388+0000","last_undegraded":"2024-04-25T00:35:52.092388+0000","last_fullsized":"2024-04-25T00:35:52.092388+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:03.363932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.090981+0000","last_change":"2024-04-25T00:35:50.197729+0000","last_active":"2024-04-25T00:35:52.090981+0000","last_peered":"2024-04-25T00:35:52.090981+0000","last_clean":"2024-04-25T00:35:52.090981+0000","last_became_active":"2024-04-25T00:35:50.197368+0000","last_became_peered":"2024-04-25T00:35:50.197368+0000","last_unstale":"2024-04-25T00:35:52.090981+0000","last_undegraded":"2024-04-25T00:35:52.090981+0000","last_fullsized":"2024-04-25T00:35:52.090981+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:48.995580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"69'6","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091075+0000","last_change":"2024-04-25T00:35:49.173021+0000","last_active":"2024-04-25T00:35:52.091075+0000","last_peered":"2024-04-25T00:35:52.091075+0000","last_clean":"2024-04-25T00:35:52.091075+0000","last_became_active":"2024-04-25T00:35:49.172717+0000","last_became_peered":"2024-04-25T00:35:49.172717+0000","last_unstale":"2024-04-25T00:35:52.091075+0000","last_undegraded":"2024-04-25T00:35:52.091075+0000","last_fullsized":"2024-04-25T00:35:52.091075+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:21.365941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.200092+0000","last_change":"2024-04-25T00:35:50.219561+0000","last_active":"2024-04-25T00:35:51.200092+0000","last_peered":"2024-04-25T00:35:51.200092+0000","last_clean":"2024-04-25T00:35:51.200092+0000","last_became_active":"2024-04-25T00:35:50.218790+0000","last_became_peered":"2024-04-25T00:35:50.218790+0000","last_unstale":"2024-04-25T00:35:51.200092+0000","last_undegraded":"2024-04-25T00:35:51.200092+0000","last_fullsized":"2024-04-25T00:35:51.200092+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:23.345172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089603+0000","last_change":"2024-04-25T00:35:49.173508+0000","last_active":"2024-04-25T00:35:52.089603+0000","last_peered":"2024-04-25T00:35:52.089603+0000","last_clean":"2024-04-25T00:35:52.089603+0000","last_became_active":"2024-04-25T00:35:49.173087+0000","last_became_peered":"2024-04-25T00:35:49.173087+0000","last_unstale":"2024-04-25T00:35:52.089603+0000","last_undegraded":"2024-04-25T00:35:52.089603+0000","last_fullsized":"2024-04-25T00:35:52.089603+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:18.890757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204296+0000","last_change":"2024-04-25T00:35:50.202056+0000","last_active":"2024-04-25T00:35:51.204296+0000","last_peered":"2024-04-25T00:35:51.204296+0000","last_clean":"2024-04-25T00:35:51.204296+0000","last_became_active":"2024-04-25T00:35:50.201331+0000","last_became_peered":"2024-04-25T00:35:50.201331+0000","last_unstale":"2024-04-25T00:35:51.204296+0000","last_undegraded":"2024-04-25T00:35:51.204296+0000","last_fullsized":"2024-04-25T00:35:51.204296+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:08.583461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588595+0000","last_change":"2024-04-25T00:35:49.193702+0000","last_active":"2024-04-25T00:35:51.588595+0000","last_peered":"2024-04-25T00:35:51.588595+0000","last_clean":"2024-04-25T00:35:51.588595+0000","last_became_active":"2024-04-25T00:35:49.192211+0000","last_became_peered":"2024-04-25T00:35:49.192211+0000","last_unstale":"2024-04-25T00:35:51.588595+0000","last_undegraded":"2024-04-25T00:35:51.588595+0000","last_fullsized":"2024-04-25T00:35:51.588595+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:30.680476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.088411+0000","last_change":"2024-04-25T00:35:50.229674+0000","last_active":"2024-04-25T00:35:52.088411+0000","last_peered":"2024-04-25T00:35:52.088411+0000","last_clean":"2024-04-25T00:35:52.088411+0000","last_became_active":"2024-04-25T00:35:50.229376+0000","last_became_peered":"2024-04-25T00:35:50.229376+0000","last_unstale":"2024-04-25T00:35:52.088411+0000","last_undegraded":"2024-04-25T00:35:52.088411+0000","last_fullsized":"2024-04-25T00:35:52.088411+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:08.583461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204201+0000","last_change":"2024-04-25T00:35:49.184169+0000","last_active":"2024-04-25T00:35:51.204201+0000","last_peered":"2024-04-25T00:35:51.204201+0000","last_clean":"2024-04-25T00:35:51.204201+0000","last_became_active":"2024-04-25T00:35:49.183700+0000","last_became_peered":"2024-04-25T00:35:49.183700+0000","last_unstale":"2024-04-25T00:35:51.204201+0000","last_undegraded":"2024-04-25T00:35:51.204201+0000","last_fullsized":"2024-04-25T00:35:51.204201+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:53.984973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092108+0000","last_change":"2024-04-25T00:35:50.210231+0000","last_active":"2024-04-25T00:35:52.092108+0000","last_peered":"2024-04-25T00:35:52.092108+0000","last_clean":"2024-04-25T00:35:52.092108+0000","last_became_active":"2024-04-25T00:35:50.209951+0000","last_became_peered":"2024-04-25T00:35:50.209951+0000","last_unstale":"2024-04-25T00:35:52.092108+0000","last_undegraded":"2024-04-25T00:35:52.092108+0000","last_fullsized":"2024-04-25T00:35:52.092108+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:58.762152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":22,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.201123+0000","last_change":"2024-04-25T00:35:49.176243+0000","last_active":"2024-04-25T00:35:51.201123+0000","last_peered":"2024-04-25T00:35:51.201123+0000","last_clean":"2024-04-25T00:35:51.201123+0000","last_became_active":"2024-04-25T00:35:49.175498+0000","last_became_peered":"2024-04-25T00:35:49.175498+0000","last_unstale":"2024-04-25T00:35:51.201123+0000","last_undegraded":"2024-04-25T00:35:51.201123+0000","last_fullsized":"2024-04-25T00:35:51.201123+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:21.365941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202630+0000","last_change":"2024-04-25T00:35:50.200674+0000","last_active":"2024-04-25T00:35:51.202630+0000","last_peered":"2024-04-25T00:35:51.202630+0000","last_clean":"2024-04-25T00:35:51.202630+0000","last_became_active":"2024-04-25T00:35:50.200235+0000","last_became_peered":"2024-04-25T00:35:50.200235+0000","last_unstale":"2024-04-25T00:35:51.202630+0000","last_undegraded":"2024-04-25T00:35:51.202630+0000","last_fullsized":"2024-04-25T00:35:51.202630+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:02.142046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588669+0000","last_change":"2024-04-25T00:35:49.191787+0000","last_active":"2024-04-25T00:35:51.588669+0000","last_peered":"2024-04-25T00:35:51.588669+0000","last_clean":"2024-04-25T00:35:51.588669+0000","last_became_active":"2024-04-25T00:35:49.191116+0000","last_became_peered":"2024-04-25T00:35:49.191116+0000","last_unstale":"2024-04-25T00:35:51.588669+0000","last_undegraded":"2024-04-25T00:35:51.588669+0000","last_fullsized":"2024-04-25T00:35:51.588669+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:19.303738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203912+0000","last_change":"2024-04-25T00:35:50.210860+0000","last_active":"2024-04-25T00:35:51.203912+0000","last_peered":"2024-04-25T00:35:51.203912+0000","last_clean":"2024-04-25T00:35:51.203912+0000","last_became_active":"2024-04-25T00:35:50.210148+0000","last_became_peered":"2024-04-25T00:35:50.210148+0000","last_unstale":"2024-04-25T00:35:51.203912+0000","last_undegraded":"2024-04-25T00:35:51.203912+0000","last_fullsized":"2024-04-25T00:35:51.203912+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:00.957517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202737+0000","last_change":"2024-04-25T00:35:49.192932+0000","last_active":"2024-04-25T00:35:51.202737+0000","last_peered":"2024-04-25T00:35:51.202737+0000","last_clean":"2024-04-25T00:35:51.202737+0000","last_became_active":"2024-04-25T00:35:49.192527+0000","last_became_peered":"2024-04-25T00:35:49.192527+0000","last_unstale":"2024-04-25T00:35:51.202737+0000","last_undegraded":"2024-04-25T00:35:51.202737+0000","last_fullsized":"2024-04-25T00:35:51.202737+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:28.542118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091032+0000","last_change":"2024-04-25T00:35:50.203751+0000","last_active":"2024-04-25T00:35:52.091032+0000","last_peered":"2024-04-25T00:35:52.091032+0000","last_clean":"2024-04-25T00:35:52.091032+0000","last_became_active":"2024-04-25T00:35:50.203445+0000","last_became_peered":"2024-04-25T00:35:50.203445+0000","last_unstale":"2024-04-25T00:35:52.091032+0000","last_undegraded":"2024-04-25T00:35:52.091032+0000","last_fullsized":"2024-04-25T00:35:52.091032+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:26.055053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588506+0000","last_change":"2024-04-25T00:35:49.192591+0000","last_active":"2024-04-25T00:35:51.588506+0000","last_peered":"2024-04-25T00:35:51.588506+0000","last_clean":"2024-04-25T00:35:51.588506+0000","last_became_active":"2024-04-25T00:35:49.191599+0000","last_became_peered":"2024-04-25T00:35:49.191599+0000","last_unstale":"2024-04-25T00:35:51.588506+0000","last_undegraded":"2024-04-25T00:35:51.588506+0000","last_fullsized":"2024-04-25T00:35:51.588506+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:25.728034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091399+0000","last_change":"2024-04-25T00:35:50.214215+0000","last_active":"2024-04-25T00:35:52.091399+0000","last_peered":"2024-04-25T00:35:52.091399+0000","last_clean":"2024-04-25T00:35:52.091399+0000","last_became_active":"2024-04-25T00:35:50.213864+0000","last_became_peered":"2024-04-25T00:35:50.213864+0000","last_unstale":"2024-04-25T00:35:52.091399+0000","last_undegraded":"2024-04-25T00:35:52.091399+0000","last_fullsized":"2024-04-25T00:35:52.091399+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:17.241816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203160+0000","last_change":"2024-04-25T00:35:49.178876+0000","last_active":"2024-04-25T00:35:51.203160+0000","last_peered":"2024-04-25T00:35:51.203160+0000","last_clean":"2024-04-25T00:35:51.203160+0000","last_became_active":"2024-04-25T00:35:49.178162+0000","last_became_peered":"2024-04-25T00:35:49.178162+0000","last_unstale":"2024-04-25T00:35:51.203160+0000","last_undegraded":"2024-04-25T00:35:51.203160+0000","last_fullsized":"2024-04-25T00:35:51.203160+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:38.332283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092817+0000","last_change":"2024-04-25T00:35:50.206416+0000","last_active":"2024-04-25T00:35:52.092817+0000","last_peered":"2024-04-25T00:35:52.092817+0000","last_clean":"2024-04-25T00:35:52.092817+0000","last_became_active":"2024-04-25T00:35:50.205886+0000","last_became_peered":"2024-04-25T00:35:50.205886+0000","last_unstale":"2024-04-25T00:35:52.092817+0000","last_undegraded":"2024-04-25T00:35:52.092817+0000","last_fullsized":"2024-04-25T00:35:52.092817+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:21.592033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588851+0000","last_change":"2024-04-25T00:35:49.189004+0000","last_active":"2024-04-25T00:35:51.588851+0000","last_peered":"2024-04-25T00:35:51.588851+0000","last_clean":"2024-04-25T00:35:51.588851+0000","last_became_active":"2024-04-25T00:35:49.183196+0000","last_became_peered":"2024-04-25T00:35:49.183196+0000","last_unstale":"2024-04-25T00:35:51.588851+0000","last_undegraded":"2024-04-25T00:35:51.588851+0000","last_fullsized":"2024-04-25T00:35:51.588851+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:50.622430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091855+0000","last_change":"2024-04-25T00:35:50.229703+0000","last_active":"2024-04-25T00:35:52.091855+0000","last_peered":"2024-04-25T00:35:52.091855+0000","last_clean":"2024-04-25T00:35:52.091855+0000","last_became_active":"2024-04-25T00:35:50.229389+0000","last_became_peered":"2024-04-25T00:35:50.229389+0000","last_unstale":"2024-04-25T00:35:52.091855+0000","last_undegraded":"2024-04-25T00:35:52.091855+0000","last_fullsized":"2024-04-25T00:35:52.091855+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:40.507238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204580+0000","last_change":"2024-04-25T00:35:49.170971+0000","last_active":"2024-04-25T00:35:51.204580+0000","last_peered":"2024-04-25T00:35:51.204580+0000","last_clean":"2024-04-25T00:35:51.204580+0000","last_became_active":"2024-04-25T00:35:49.170132+0000","last_became_peered":"2024-04-25T00:35:49.170132+0000","last_unstale":"2024-04-25T00:35:51.204580+0000","last_undegraded":"2024-04-25T00:35:51.204580+0000","last_fullsized":"2024-04-25T00:35:51.204580+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:12.466682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092318+0000","last_change":"2024-04-25T00:35:50.229625+0000","last_active":"2024-04-25T00:35:52.092318+0000","last_peered":"2024-04-25T00:35:52.092318+0000","last_clean":"2024-04-25T00:35:52.092318+0000","last_became_active":"2024-04-25T00:35:50.229338+0000","last_became_peered":"2024-04-25T00:35:50.229338+0000","last_unstale":"2024-04-25T00:35:52.092318+0000","last_undegraded":"2024-04-25T00:35:52.092318+0000","last_fullsized":"2024-04-25T00:35:52.092318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:19.231960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.088078+0000","last_change":"2024-04-25T00:35:49.174136+0000","last_active":"2024-04-25T00:35:52.088078+0000","last_peered":"2024-04-25T00:35:52.088078+0000","last_clean":"2024-04-25T00:35:52.088078+0000","last_became_active":"2024-04-25T00:35:49.173623+0000","last_became_peered":"2024-04-25T00:35:49.173623+0000","last_unstale":"2024-04-25T00:35:52.088078+0000","last_undegraded":"2024-04-25T00:35:52.088078+0000","last_fullsized":"2024-04-25T00:35:52.088078+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:34.609362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.093044+0000","last_change":"2024-04-25T00:35:50.203655+0000","last_active":"2024-04-25T00:35:52.093044+0000","last_peered":"2024-04-25T00:35:52.093044+0000","last_clean":"2024-04-25T00:35:52.093044+0000","last_became_active":"2024-04-25T00:35:50.203357+0000","last_became_peered":"2024-04-25T00:35:50.203357+0000","last_unstale":"2024-04-25T00:35:52.093044+0000","last_undegraded":"2024-04-25T00:35:52.093044+0000","last_fullsized":"2024-04-25T00:35:52.093044+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:55.595220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588490+0000","last_change":"2024-04-25T00:35:49.190985+0000","last_active":"2024-04-25T00:35:51.588490+0000","last_peered":"2024-04-25T00:35:51.588490+0000","last_clean":"2024-04-25T00:35:51.588490+0000","last_became_active":"2024-04-25T00:35:49.190267+0000","last_became_peered":"2024-04-25T00:35:49.190267+0000","last_unstale":"2024-04-25T00:35:51.588490+0000","last_undegraded":"2024-04-25T00:35:51.588490+0000","last_fullsized":"2024-04-25T00:35:51.588490+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:54.517092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.205651+0000","last_change":"2024-04-25T00:35:50.203263+0000","last_active":"2024-04-25T00:35:51.205651+0000","last_peered":"2024-04-25T00:35:51.205651+0000","last_clean":"2024-04-25T00:35:51.205651+0000","last_became_active":"2024-04-25T00:35:50.202513+0000","last_became_peered":"2024-04-25T00:35:50.202513+0000","last_unstale":"2024-04-25T00:35:51.205651+0000","last_undegraded":"2024-04-25T00:35:51.205651+0000","last_fullsized":"2024-04-25T00:35:51.205651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:19.748799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.205449+0000","last_change":"2024-04-25T00:35:49.187128+0000","last_active":"2024-04-25T00:35:51.205449+0000","last_peered":"2024-04-25T00:35:51.205449+0000","last_clean":"2024-04-25T00:35:51.205449+0000","last_became_active":"2024-04-25T00:35:49.184464+0000","last_became_peered":"2024-04-25T00:35:49.184464+0000","last_unstale":"2024-04-25T00:35:51.205449+0000","last_undegraded":"2024-04-25T00:35:51.205449+0000","last_fullsized":"2024-04-25T00:35:51.205449+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:34.609362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203012+0000","last_change":"2024-04-25T00:35:50.239394+0000","last_active":"2024-04-25T00:35:51.203012+0000","last_peered":"2024-04-25T00:35:51.203012+0000","last_clean":"2024-04-25T00:35:51.203012+0000","last_became_active":"2024-04-25T00:35:50.238914+0000","last_became_peered":"2024-04-25T00:35:50.238914+0000","last_unstale":"2024-04-25T00:35:51.203012+0000","last_undegraded":"2024-04-25T00:35:51.203012+0000","last_fullsized":"2024-04-25T00:35:51.203012+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:50.847672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091453+0000","last_change":"2024-04-25T00:35:49.171187+0000","last_active":"2024-04-25T00:35:52.091453+0000","last_peered":"2024-04-25T00:35:52.091453+0000","last_clean":"2024-04-25T00:35:52.091453+0000","last_became_active":"2024-04-25T00:35:49.170719+0000","last_became_peered":"2024-04-25T00:35:49.170719+0000","last_unstale":"2024-04-25T00:35:52.091453+0000","last_undegraded":"2024-04-25T00:35:52.091453+0000","last_fullsized":"2024-04-25T00:35:52.091453+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:24.297162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203473+0000","last_change":"2024-04-25T00:35:50.207396+0000","last_active":"2024-04-25T00:35:51.203473+0000","last_peered":"2024-04-25T00:35:51.203473+0000","last_clean":"2024-04-25T00:35:51.203473+0000","last_became_active":"2024-04-25T00:35:50.206584+0000","last_became_peered":"2024-04-25T00:35:50.206584+0000","last_unstale":"2024-04-25T00:35:51.203473+0000","last_undegraded":"2024-04-25T00:35:51.203473+0000","last_fullsized":"2024-04-25T00:35:51.203473+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:03.847803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.088992+0000","last_change":"2024-04-25T00:35:49.176291+0000","last_active":"2024-04-25T00:35:52.088992+0000","last_peered":"2024-04-25T00:35:52.088992+0000","last_clean":"2024-04-25T00:35:52.088992+0000","last_became_active":"2024-04-25T00:35:49.175847+0000","last_became_peered":"2024-04-25T00:35:49.175847+0000","last_unstale":"2024-04-25T00:35:52.088992+0000","last_undegraded":"2024-04-25T00:35:52.088992+0000","last_fullsized":"2024-04-25T00:35:52.088992+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:47.988021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091705+0000","last_change":"2024-04-25T00:35:50.209546+0000","last_active":"2024-04-25T00:35:52.091705+0000","last_peered":"2024-04-25T00:35:52.091705+0000","last_clean":"2024-04-25T00:35:52.091705+0000","last_became_active":"2024-04-25T00:35:50.209273+0000","last_became_peered":"2024-04-25T00:35:50.209273+0000","last_unstale":"2024-04-25T00:35:52.091705+0000","last_undegraded":"2024-04-25T00:35:52.091705+0000","last_fullsized":"2024-04-25T00:35:52.091705+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:56.549654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.090336+0000","last_change":"2024-04-25T00:35:49.188246+0000","last_active":"2024-04-25T00:35:52.090336+0000","last_peered":"2024-04-25T00:35:52.090336+0000","last_clean":"2024-04-25T00:35:52.090336+0000","last_became_active":"2024-04-25T00:35:49.187766+0000","last_became_peered":"2024-04-25T00:35:49.187766+0000","last_unstale":"2024-04-25T00:35:52.090336+0000","last_undegraded":"2024-04-25T00:35:52.090336+0000","last_fullsized":"2024-04-25T00:35:52.090336+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:35.641595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588496+0000","last_change":"2024-04-25T00:35:50.235717+0000","last_active":"2024-04-25T00:35:51.588496+0000","last_peered":"2024-04-25T00:35:51.588496+0000","last_clean":"2024-04-25T00:35:51.588496+0000","last_became_active":"2024-04-25T00:35:50.232769+0000","last_became_peered":"2024-04-25T00:35:50.232769+0000","last_unstale":"2024-04-25T00:35:51.588496+0000","last_undegraded":"2024-04-25T00:35:51.588496+0000","last_fullsized":"2024-04-25T00:35:51.588496+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:58.856012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089404+0000","last_change":"2024-04-25T00:35:49.177523+0000","last_active":"2024-04-25T00:35:52.089404+0000","last_peered":"2024-04-25T00:35:52.089404+0000","last_clean":"2024-04-25T00:35:52.089404+0000","last_became_active":"2024-04-25T00:35:49.177122+0000","last_became_peered":"2024-04-25T00:35:49.177122+0000","last_unstale":"2024-04-25T00:35:52.089404+0000","last_undegraded":"2024-04-25T00:35:52.089404+0000","last_fullsized":"2024-04-25T00:35:52.089404+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:04.207295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089888+0000","last_change":"2024-04-25T00:35:50.209054+0000","last_active":"2024-04-25T00:35:52.089888+0000","last_peered":"2024-04-25T00:35:52.089888+0000","last_clean":"2024-04-25T00:35:52.089888+0000","last_became_active":"2024-04-25T00:35:50.208775+0000","last_became_peered":"2024-04-25T00:35:50.208775+0000","last_unstale":"2024-04-25T00:35:52.089888+0000","last_undegraded":"2024-04-25T00:35:52.089888+0000","last_fullsized":"2024-04-25T00:35:52.089888+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:31.688632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.201038+0000","last_change":"2024-04-25T00:35:49.177395+0000","last_active":"2024-04-25T00:35:51.201038+0000","last_peered":"2024-04-25T00:35:51.201038+0000","last_clean":"2024-04-25T00:35:51.201038+0000","last_became_active":"2024-04-25T00:35:49.175489+0000","last_became_peered":"2024-04-25T00:35:49.175489+0000","last_unstale":"2024-04-25T00:35:51.201038+0000","last_undegraded":"2024-04-25T00:35:51.201038+0000","last_fullsized":"2024-04-25T00:35:51.201038+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:09.958187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.587891+0000","last_change":"2024-04-25T00:35:50.214266+0000","last_active":"2024-04-25T00:35:51.587891+0000","last_peered":"2024-04-25T00:35:51.587891+0000","last_clean":"2024-04-25T00:35:51.587891+0000","last_became_active":"2024-04-25T00:35:50.213595+0000","last_became_peered":"2024-04-25T00:35:50.213595+0000","last_unstale":"2024-04-25T00:35:51.587891+0000","last_undegraded":"2024-04-25T00:35:51.587891+0000","last_fullsized":"2024-04-25T00:35:51.587891+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:35.617518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588389+0000","last_change":"2024-04-25T00:35:49.181943+0000","last_active":"2024-04-25T00:35:51.588389+0000","last_peered":"2024-04-25T00:35:51.588389+0000","last_clean":"2024-04-25T00:35:51.588389+0000","last_became_active":"2024-04-25T00:35:49.181556+0000","last_became_peered":"2024-04-25T00:35:49.181556+0000","last_unstale":"2024-04-25T00:35:51.588389+0000","last_undegraded":"2024-04-25T00:35:51.588389+0000","last_fullsized":"2024-04-25T00:35:51.588389+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:12.501099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091418+0000","last_change":"2024-04-25T00:35:50.236132+0000","last_active":"2024-04-25T00:35:52.091418+0000","last_peered":"2024-04-25T00:35:52.091418+0000","last_clean":"2024-04-25T00:35:52.091418+0000","last_became_active":"2024-04-25T00:35:50.235896+0000","last_became_peered":"2024-04-25T00:35:50.235896+0000","last_unstale":"2024-04-25T00:35:52.091418+0000","last_undegraded":"2024-04-25T00:35:52.091418+0000","last_fullsized":"2024-04-25T00:35:52.091418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:16.701600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588061+0000","last_change":"2024-04-25T00:35:49.182981+0000","last_active":"2024-04-25T00:35:51.588061+0000","last_peered":"2024-04-25T00:35:51.588061+0000","last_clean":"2024-04-25T00:35:51.588061+0000","last_became_active":"2024-04-25T00:35:49.182211+0000","last_became_peered":"2024-04-25T00:35:49.182211+0000","last_unstale":"2024-04-25T00:35:51.588061+0000","last_undegraded":"2024-04-25T00:35:51.588061+0000","last_fullsized":"2024-04-25T00:35:51.588061+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:04.207295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":266,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.199732+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:35:51.199732+0000","last_peered":"2024-04-25T00:35:51.199732+0000","last_clean":"2024-04-25T00:35:51.199732+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:35:51.199732+0000","last_undegraded":"2024-04-25T00:35:51.199732+0000","last_fullsized":"2024-04-25T00:35:51.199732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1454080,"data_stored":950312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":860160,"data_stored":359420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":856064,"data_stored":359418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36184,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335656,"statfs":{"total":96636764160,"available":96599711744,"internally_reserved":0,"allocated":856064,"data_stored":359398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335656,"statfs":{"total":96636764160,"available":96599711744,"internally_reserved":0,"allocated":860160,"data_stored":359440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36188,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335652,"statfs":{"total":96636764160,"available":96599707648,"internally_reserved":0,"allocated":864256,"data_stored":360940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335644,"statfs":{"total":96636764160,"available":96599699456,"internally_reserved":0,"allocated":872448,"data_stored":361556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36756,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335084,"statfs":{"total":96636764160,"available":96599126016,"internally_reserved":0,"allocated":1445888,"data_stored":949696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:35:58.710 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:35:58.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97ca0c640 1 Processor -- start 2024-04-25T00:35:58.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97ca0c640 1 -- start start 2024-04-25T00:35:58.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97ca0c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff9780b3140 0x7ff97806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97ca0c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9780b4b10 0x7ff9780b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97ca0c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9780b41e0 con 0x7ff9780b3a60 2024-04-25T00:35:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97ca0c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff97806d960 con 0x7ff9780b3140 2024-04-25T00:35:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97ca0c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff97806dc40 con 0x7ff9780b4b10 2024-04-25T00:35:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97659c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff9780b3140 0x7ff97806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97659c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff9780b3140 0x7ff97806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42054/0 (socket says 172.21.15.29:42054) 2024-04-25T00:35:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97659c640 1 -- 172.21.15.29:0/1663317086 learned_addr learned my addr 172.21.15.29:0/1663317086 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff976d9d640 1 --1- 172.21.15.29:0/1663317086 >> v1:172.21.15.29:6789/0 conn(0x7ff9780b3a60 0x7ff97815c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58492/0 (socket says 172.21.15.29:58492) 2024-04-25T00:35:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff975d9b640 1 --2- 172.21.15.29:0/1663317086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9780b4b10 0x7ff9780b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97659c640 1 -- 172.21.15.29:0/1663317086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9780b4b10 msgr2=0x7ff9780b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97659c640 1 --2- 172.21.15.29:0/1663317086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9780b4b10 0x7ff9780b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97659c640 1 -- 172.21.15.29:0/1663317086 >> v1:172.21.15.29:6789/0 conn(0x7ff9780b3a60 legacy=0x7ff97815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:58.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.844+0000 7ff97659c640 1 -- 172.21.15.29:0/1663317086 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97806df20 con 0x7ff9780b3140 2024-04-25T00:35:58.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97659c640 1 --2- 172.21.15.29:0/1663317086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9780b3140 0x7ff97806e440 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff968009870 tx=0x7ff96801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b966c6632f4898d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:58.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97559a640 1 -- 172.21.15.29:0/1663317086 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9680b0070 con 0x7ff9780b3140 2024-04-25T00:35:58.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97559a640 1 -- 172.21.15.29:0/1663317086 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9680ab600 con 0x7ff9780b3140 2024-04-25T00:35:58.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97559a640 1 -- 172.21.15.29:0/1663317086 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9680ab9d0 con 0x7ff9780b3140 2024-04-25T00:35:58.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 -- 172.21.15.29:0/1663317086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9780b3140 msgr2=0x7ff97806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:58.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 --2- 172.21.15.29:0/1663317086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9780b3140 0x7ff97806e440 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff968009870 tx=0x7ff96801c560 comp rx=0 tx=0).stop 2024-04-25T00:35:58.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 -- 172.21.15.29:0/1663317086 shutdown_connections 2024-04-25T00:35:58.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 --2- 172.21.15.29:0/1663317086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9780b4b10 0x7ff9780b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 --2- 172.21.15.29:0/1663317086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9780b3140 0x7ff97806e440 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 -- 172.21.15.29:0/1663317086 >> 172.21.15.29:0/1663317086 conn(0x7ff9780b1870 msgr2=0x7ff9780b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:58.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 -- 172.21.15.29:0/1663317086 shutdown_connections 2024-04-25T00:35:58.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 -- 172.21.15.29:0/1663317086 wait complete. 2024-04-25T00:35:58.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 Processor -- start 2024-04-25T00:35:58.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 -- start start 2024-04-25T00:35:58.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9780b3a60 0x7ff978154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9780b4b10 0x7ff978139340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff978139880 0x7ff97813da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff978168b70 con 0x7ff978139880 2024-04-25T00:35:58.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff9781689f0 con 0x7ff9780b4b10 2024-04-25T00:35:58.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff97ca0c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff978168cf0 con 0x7ff9780b3a60 2024-04-25T00:35:58.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff976d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff978139880 0x7ff97813da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff976d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff978139880 0x7ff97813da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42060/0 (socket says 172.21.15.29:42060) 2024-04-25T00:35:58.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.848+0000 7ff976d9d640 1 -- 172.21.15.29:0/256213765 learned_addr learned my addr 172.21.15.29:0/256213765 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:58.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff975d9b640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9780b4b10 0x7ff978139340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff976d9d640 1 -- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9780b3a60 msgr2=0x7ff978154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:58.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff97659c640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9780b3a60 0x7ff978154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff976d9d640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9780b3a60 0x7ff978154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff976d9d640 1 -- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9780b4b10 msgr2=0x7ff978139340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:58.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff976d9d640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9780b4b10 0x7ff978139340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:58.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff976d9d640 1 -- 172.21.15.29:0/256213765 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff978139eb0 con 0x7ff978139880 2024-04-25T00:35:58.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff975d9b640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9780b4b10 0x7ff978139340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:58.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff976d9d640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff978139880 0x7ff97813da80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff97000c7a0 tx=0x7ff97000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:58.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff966ffd640 1 -- 172.21.15.29:0/256213765 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970017070 con 0x7ff978139880 2024-04-25T00:35:58.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff966ffd640 1 -- 172.21.15.29:0/256213765 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff97000a640 con 0x7ff978139880 2024-04-25T00:35:58.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff966ffd640 1 -- 172.21.15.29:0/256213765 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97000a9f0 con 0x7ff978139880 2024-04-25T00:35:58.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff97ca0c640 1 -- 172.21.15.29:0/256213765 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97813a190 con 0x7ff978139880 2024-04-25T00:35:58.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff97ca0c640 1 -- 172.21.15.29:0/256213765 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97813ab40 con 0x7ff978139880 2024-04-25T00:35:58.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff97ca0c640 1 -- 172.21.15.29:0/256213765 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff978069270 con 0x7ff978139880 2024-04-25T00:35:58.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff966ffd640 1 -- 172.21.15.29:0/256213765 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7ff97001a070 con 0x7ff978139880 2024-04-25T00:35:58.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff966ffd640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff958067c20 0x7ff95806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:58.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff966ffd640 1 -- 172.21.15.29:0/256213765 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7ff9700858d0 con 0x7ff978139880 2024-04-25T00:35:58.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff97659c640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff958067c20 0x7ff95806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:58.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.852+0000 7ff97659c640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff958067c20 0x7ff95806a0e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff9680ad000 tx=0x7ff968008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:58.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:58.856+0000 7ff966ffd640 1 -- 172.21.15.29:0/256213765 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff970055870 con 0x7ff978139880 2024-04-25T00:35:59.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.012+0000 7ff97ca0c640 1 -- 172.21.15.29:0/256213765 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff97815f410 con 0x7ff958067c20 2024-04-25T00:35:59.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.032+0000 7ff966ffd640 1 -- 172.21.15.29:0/256213765 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7ff97815f410 con 0x7ff958067c20 2024-04-25T00:35:59.037 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:59.039 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:35:59.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.036+0000 7ff97ca0c640 1 -- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff958067c20 msgr2=0x7ff95806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.036+0000 7ff97ca0c640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff958067c20 0x7ff95806a0e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff9680ad000 tx=0x7ff968008040 comp rx=0 tx=0).stop 2024-04-25T00:35:59.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.036+0000 7ff97ca0c640 1 -- 172.21.15.29:0/256213765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff978139880 msgr2=0x7ff97813da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.036+0000 7ff97ca0c640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff978139880 0x7ff97813da80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff97000c7a0 tx=0x7ff97000cc60 comp rx=0 tx=0).stop 2024-04-25T00:35:59.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.036+0000 7ff97ca0c640 1 -- 172.21.15.29:0/256213765 shutdown_connections 2024-04-25T00:35:59.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.036+0000 7ff97ca0c640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff978139880 0x7ff97813da80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.036+0000 7ff97ca0c640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff958067c20 0x7ff95806a0e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.036+0000 7ff97ca0c640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9780b4b10 0x7ff978139340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.036+0000 7ff97ca0c640 1 --2- 172.21.15.29:0/256213765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9780b3a60 0x7ff978154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.036+0000 7ff97ca0c640 1 -- 172.21.15.29:0/256213765 >> 172.21.15.29:0/256213765 conn(0x7ff9780b1870 msgr2=0x7ff978142cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:59.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.036+0000 7ff97ca0c640 1 -- 172.21.15.29:0/256213765 shutdown_connections 2024-04-25T00:35:59.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.036+0000 7ff97ca0c640 1 -- 172.21.15.29:0/256213765 wait complete. 2024-04-25T00:35:59.056 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-04-25T00:35:57.537792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282316,"kb_used_data":7880,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754692404,"statfs":{"total":773094113280,"available":772805021696,"internally_reserved":0,"allocated":8069120,"data_stored":4060180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003004"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203249+0000","last_change":"2024-04-25T00:35:50.218836+0000","last_active":"2024-04-25T00:35:51.203249+0000","last_peered":"2024-04-25T00:35:51.203249+0000","last_clean":"2024-04-25T00:35:51.203249+0000","last_became_active":"2024-04-25T00:35:50.218565+0000","last_became_peered":"2024-04-25T00:35:50.218565+0000","last_unstale":"2024-04-25T00:35:51.203249+0000","last_undegraded":"2024-04-25T00:35:51.203249+0000","last_fullsized":"2024-04-25T00:35:51.203249+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:49.507678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.589210+0000","last_change":"2024-04-25T00:35:50.212533+0000","last_active":"2024-04-25T00:35:51.589210+0000","last_peered":"2024-04-25T00:35:51.589210+0000","last_clean":"2024-04-25T00:35:51.589210+0000","last_became_active":"2024-04-25T00:35:50.211826+0000","last_became_peered":"2024-04-25T00:35:50.211826+0000","last_unstale":"2024-04-25T00:35:51.589210+0000","last_undegraded":"2024-04-25T00:35:51.589210+0000","last_fullsized":"2024-04-25T00:35:51.589210+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:28.691635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091824+0000","last_change":"2024-04-25T00:35:50.206984+0000","last_active":"2024-04-25T00:35:52.091824+0000","last_peered":"2024-04-25T00:35:52.091824+0000","last_clean":"2024-04-25T00:35:52.091824+0000","last_became_active":"2024-04-25T00:35:50.206697+0000","last_became_peered":"2024-04-25T00:35:50.206697+0000","last_unstale":"2024-04-25T00:35:52.091824+0000","last_undegraded":"2024-04-25T00:35:52.091824+0000","last_fullsized":"2024-04-25T00:35:52.091824+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:59.790434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.090224+0000","last_change":"2024-04-25T00:35:50.233152+0000","last_active":"2024-04-25T00:35:52.090224+0000","last_peered":"2024-04-25T00:35:52.090224+0000","last_clean":"2024-04-25T00:35:52.090224+0000","last_became_active":"2024-04-25T00:35:50.232832+0000","last_became_peered":"2024-04-25T00:35:50.232832+0000","last_unstale":"2024-04-25T00:35:52.090224+0000","last_undegraded":"2024-04-25T00:35:52.090224+0000","last_fullsized":"2024-04-25T00:35:52.090224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:07.819731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.090177+0000","last_change":"2024-04-25T00:35:50.217003+0000","last_active":"2024-04-25T00:35:52.090177+0000","last_peered":"2024-04-25T00:35:52.090177+0000","last_clean":"2024-04-25T00:35:52.090177+0000","last_became_active":"2024-04-25T00:35:50.216705+0000","last_became_peered":"2024-04-25T00:35:50.216705+0000","last_unstale":"2024-04-25T00:35:52.090177+0000","last_undegraded":"2024-04-25T00:35:52.090177+0000","last_fullsized":"2024-04-25T00:35:52.090177+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:59.001749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588132+0000","last_change":"2024-04-25T00:35:50.228663+0000","last_active":"2024-04-25T00:35:51.588132+0000","last_peered":"2024-04-25T00:35:51.588132+0000","last_clean":"2024-04-25T00:35:51.588132+0000","last_became_active":"2024-04-25T00:35:50.228029+0000","last_became_peered":"2024-04-25T00:35:50.228029+0000","last_unstale":"2024-04-25T00:35:51.588132+0000","last_undegraded":"2024-04-25T00:35:51.588132+0000","last_fullsized":"2024-04-25T00:35:51.588132+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:50.971516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204105+0000","last_change":"2024-04-25T00:35:50.196683+0000","last_active":"2024-04-25T00:35:51.204105+0000","last_peered":"2024-04-25T00:35:51.204105+0000","last_clean":"2024-04-25T00:35:51.204105+0000","last_became_active":"2024-04-25T00:35:50.195978+0000","last_became_peered":"2024-04-25T00:35:50.195978+0000","last_unstale":"2024-04-25T00:35:51.204105+0000","last_undegraded":"2024-04-25T00:35:51.204105+0000","last_fullsized":"2024-04-25T00:35:51.204105+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:09.403470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588135+0000","last_change":"2024-04-25T00:35:50.228842+0000","last_active":"2024-04-25T00:35:51.588135+0000","last_peered":"2024-04-25T00:35:51.588135+0000","last_clean":"2024-04-25T00:35:51.588135+0000","last_became_active":"2024-04-25T00:35:50.228223+0000","last_became_peered":"2024-04-25T00:35:50.228223+0000","last_unstale":"2024-04-25T00:35:51.588135+0000","last_undegraded":"2024-04-25T00:35:51.588135+0000","last_fullsized":"2024-04-25T00:35:51.588135+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:48.996177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092028+0000","last_change":"2024-04-25T00:35:50.208954+0000","last_active":"2024-04-25T00:35:52.092028+0000","last_peered":"2024-04-25T00:35:52.092028+0000","last_clean":"2024-04-25T00:35:52.092028+0000","last_became_active":"2024-04-25T00:35:50.208667+0000","last_became_peered":"2024-04-25T00:35:50.208667+0000","last_unstale":"2024-04-25T00:35:52.092028+0000","last_undegraded":"2024-04-25T00:35:52.092028+0000","last_fullsized":"2024-04-25T00:35:52.092028+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:24.336521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202586+0000","last_change":"2024-04-25T00:35:50.200817+0000","last_active":"2024-04-25T00:35:51.202586+0000","last_peered":"2024-04-25T00:35:51.202586+0000","last_clean":"2024-04-25T00:35:51.202586+0000","last_became_active":"2024-04-25T00:35:50.199957+0000","last_became_peered":"2024-04-25T00:35:50.199957+0000","last_unstale":"2024-04-25T00:35:51.202586+0000","last_undegraded":"2024-04-25T00:35:51.202586+0000","last_fullsized":"2024-04-25T00:35:51.202586+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:50.971516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202696+0000","last_change":"2024-04-25T00:35:50.212930+0000","last_active":"2024-04-25T00:35:51.202696+0000","last_peered":"2024-04-25T00:35:51.202696+0000","last_clean":"2024-04-25T00:35:51.202696+0000","last_became_active":"2024-04-25T00:35:50.211866+0000","last_became_peered":"2024-04-25T00:35:50.211866+0000","last_unstale":"2024-04-25T00:35:51.202696+0000","last_undegraded":"2024-04-25T00:35:51.202696+0000","last_fullsized":"2024-04-25T00:35:51.202696+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:47.678076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.090627+0000","last_change":"2024-04-25T00:35:50.208972+0000","last_active":"2024-04-25T00:35:52.090627+0000","last_peered":"2024-04-25T00:35:52.090627+0000","last_clean":"2024-04-25T00:35:52.090627+0000","last_became_active":"2024-04-25T00:35:50.208343+0000","last_became_peered":"2024-04-25T00:35:50.208343+0000","last_unstale":"2024-04-25T00:35:52.090627+0000","last_undegraded":"2024-04-25T00:35:52.090627+0000","last_fullsized":"2024-04-25T00:35:52.090627+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:52.238530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089747+0000","last_change":"2024-04-25T00:35:50.228954+0000","last_active":"2024-04-25T00:35:52.089747+0000","last_peered":"2024-04-25T00:35:52.089747+0000","last_clean":"2024-04-25T00:35:52.089747+0000","last_became_active":"2024-04-25T00:35:50.228644+0000","last_became_peered":"2024-04-25T00:35:50.228644+0000","last_unstale":"2024-04-25T00:35:52.089747+0000","last_undegraded":"2024-04-25T00:35:52.089747+0000","last_fullsized":"2024-04-25T00:35:52.089747+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:17.997355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091268+0000","last_change":"2024-04-25T00:35:50.189250+0000","last_active":"2024-04-25T00:35:52.091268+0000","last_peered":"2024-04-25T00:35:52.091268+0000","last_clean":"2024-04-25T00:35:52.091268+0000","last_became_active":"2024-04-25T00:35:50.185920+0000","last_became_peered":"2024-04-25T00:35:50.185920+0000","last_unstale":"2024-04-25T00:35:52.091268+0000","last_undegraded":"2024-04-25T00:35:52.091268+0000","last_fullsized":"2024-04-25T00:35:52.091268+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:23.674777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.088788+0000","last_change":"2024-04-25T00:35:50.207445+0000","last_active":"2024-04-25T00:35:52.088788+0000","last_peered":"2024-04-25T00:35:52.088788+0000","last_clean":"2024-04-25T00:35:52.088788+0000","last_became_active":"2024-04-25T00:35:50.207142+0000","last_became_peered":"2024-04-25T00:35:50.207142+0000","last_unstale":"2024-04-25T00:35:52.088788+0000","last_undegraded":"2024-04-25T00:35:52.088788+0000","last_fullsized":"2024-04-25T00:35:52.088788+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:44.765172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.200972+0000","last_change":"2024-04-25T00:35:50.230662+0000","last_active":"2024-04-25T00:35:51.200972+0000","last_peered":"2024-04-25T00:35:51.200972+0000","last_clean":"2024-04-25T00:35:51.200972+0000","last_became_active":"2024-04-25T00:35:50.230020+0000","last_became_peered":"2024-04-25T00:35:50.230020+0000","last_unstale":"2024-04-25T00:35:51.200972+0000","last_undegraded":"2024-04-25T00:35:51.200972+0000","last_fullsized":"2024-04-25T00:35:51.200972+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:07.922034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092422+0000","last_change":"2024-04-25T00:35:50.208680+0000","last_active":"2024-04-25T00:35:52.092422+0000","last_peered":"2024-04-25T00:35:52.092422+0000","last_clean":"2024-04-25T00:35:52.092422+0000","last_became_active":"2024-04-25T00:35:50.208294+0000","last_became_peered":"2024-04-25T00:35:50.208294+0000","last_unstale":"2024-04-25T00:35:52.092422+0000","last_undegraded":"2024-04-25T00:35:52.092422+0000","last_fullsized":"2024-04-25T00:35:52.092422+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:05.215451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089648+0000","last_change":"2024-04-25T00:35:50.233176+0000","last_active":"2024-04-25T00:35:52.089648+0000","last_peered":"2024-04-25T00:35:52.089648+0000","last_clean":"2024-04-25T00:35:52.089648+0000","last_became_active":"2024-04-25T00:35:50.232887+0000","last_became_peered":"2024-04-25T00:35:50.232887+0000","last_unstale":"2024-04-25T00:35:52.089648+0000","last_undegraded":"2024-04-25T00:35:52.089648+0000","last_fullsized":"2024-04-25T00:35:52.089648+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:22.775445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588633+0000","last_change":"2024-04-25T00:35:50.228429+0000","last_active":"2024-04-25T00:35:51.588633+0000","last_peered":"2024-04-25T00:35:51.588633+0000","last_clean":"2024-04-25T00:35:51.588633+0000","last_became_active":"2024-04-25T00:35:50.225794+0000","last_became_peered":"2024-04-25T00:35:50.225794+0000","last_unstale":"2024-04-25T00:35:51.588633+0000","last_undegraded":"2024-04-25T00:35:51.588633+0000","last_fullsized":"2024-04-25T00:35:51.588633+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:26:00.591734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.088850+0000","last_change":"2024-04-25T00:35:50.199758+0000","last_active":"2024-04-25T00:35:52.088850+0000","last_peered":"2024-04-25T00:35:52.088850+0000","last_clean":"2024-04-25T00:35:52.088850+0000","last_became_active":"2024-04-25T00:35:50.199367+0000","last_became_peered":"2024-04-25T00:35:50.199367+0000","last_unstale":"2024-04-25T00:35:52.088850+0000","last_undegraded":"2024-04-25T00:35:52.088850+0000","last_fullsized":"2024-04-25T00:35:52.088850+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:33.770812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.201546+0000","last_change":"2024-04-25T00:35:50.200658+0000","last_active":"2024-04-25T00:35:51.201546+0000","last_peered":"2024-04-25T00:35:51.201546+0000","last_clean":"2024-04-25T00:35:51.201546+0000","last_became_active":"2024-04-25T00:35:50.199451+0000","last_became_peered":"2024-04-25T00:35:50.199451+0000","last_unstale":"2024-04-25T00:35:51.201546+0000","last_undegraded":"2024-04-25T00:35:51.201546+0000","last_fullsized":"2024-04-25T00:35:51.201546+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:54.993129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.201761+0000","last_change":"2024-04-25T00:35:50.200451+0000","last_active":"2024-04-25T00:35:51.201761+0000","last_peered":"2024-04-25T00:35:51.201761+0000","last_clean":"2024-04-25T00:35:51.201761+0000","last_became_active":"2024-04-25T00:35:50.199725+0000","last_became_peered":"2024-04-25T00:35:50.199725+0000","last_unstale":"2024-04-25T00:35:51.201761+0000","last_undegraded":"2024-04-25T00:35:51.201761+0000","last_fullsized":"2024-04-25T00:35:51.201761+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:35.617518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.200626+0000","last_change":"2024-04-25T00:35:50.208014+0000","last_active":"2024-04-25T00:35:51.200626+0000","last_peered":"2024-04-25T00:35:51.200626+0000","last_clean":"2024-04-25T00:35:51.200626+0000","last_became_active":"2024-04-25T00:35:50.206654+0000","last_became_peered":"2024-04-25T00:35:50.206654+0000","last_unstale":"2024-04-25T00:35:51.200626+0000","last_undegraded":"2024-04-25T00:35:51.200626+0000","last_fullsized":"2024-04-25T00:35:51.200626+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:04.372088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.199883+0000","last_change":"2024-04-25T00:35:50.242161+0000","last_active":"2024-04-25T00:35:51.199883+0000","last_peered":"2024-04-25T00:35:51.199883+0000","last_clean":"2024-04-25T00:35:51.199883+0000","last_became_active":"2024-04-25T00:35:50.241803+0000","last_became_peered":"2024-04-25T00:35:50.241803+0000","last_unstale":"2024-04-25T00:35:51.199883+0000","last_undegraded":"2024-04-25T00:35:51.199883+0000","last_fullsized":"2024-04-25T00:35:51.199883+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:00.123980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203043+0000","last_change":"2024-04-25T00:35:50.215247+0000","last_active":"2024-04-25T00:35:51.203043+0000","last_peered":"2024-04-25T00:35:51.203043+0000","last_clean":"2024-04-25T00:35:51.203043+0000","last_became_active":"2024-04-25T00:35:50.214354+0000","last_became_peered":"2024-04-25T00:35:50.214354+0000","last_unstale":"2024-04-25T00:35:51.203043+0000","last_undegraded":"2024-04-25T00:35:51.203043+0000","last_fullsized":"2024-04-25T00:35:51.203043+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:58.856012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.205024+0000","last_change":"2024-04-25T00:35:50.212628+0000","last_active":"2024-04-25T00:35:51.205024+0000","last_peered":"2024-04-25T00:35:51.205024+0000","last_clean":"2024-04-25T00:35:51.205024+0000","last_became_active":"2024-04-25T00:35:50.211748+0000","last_became_peered":"2024-04-25T00:35:50.211748+0000","last_unstale":"2024-04-25T00:35:51.205024+0000","last_undegraded":"2024-04-25T00:35:51.205024+0000","last_fullsized":"2024-04-25T00:35:51.205024+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:17.997355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092236+0000","last_change":"2024-04-25T00:35:50.216069+0000","last_active":"2024-04-25T00:35:52.092236+0000","last_peered":"2024-04-25T00:35:52.092236+0000","last_clean":"2024-04-25T00:35:52.092236+0000","last_became_active":"2024-04-25T00:35:50.215796+0000","last_became_peered":"2024-04-25T00:35:50.215796+0000","last_unstale":"2024-04-25T00:35:52.092236+0000","last_undegraded":"2024-04-25T00:35:52.092236+0000","last_fullsized":"2024-04-25T00:35:52.092236+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:36.963156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202365+0000","last_change":"2024-04-25T00:35:50.201636+0000","last_active":"2024-04-25T00:35:51.202365+0000","last_peered":"2024-04-25T00:35:51.202365+0000","last_clean":"2024-04-25T00:35:51.202365+0000","last_became_active":"2024-04-25T00:35:50.199849+0000","last_became_peered":"2024-04-25T00:35:50.199849+0000","last_unstale":"2024-04-25T00:35:51.202365+0000","last_undegraded":"2024-04-25T00:35:51.202365+0000","last_fullsized":"2024-04-25T00:35:51.202365+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:48.996177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092612+0000","last_change":"2024-04-25T00:35:50.206490+0000","last_active":"2024-04-25T00:35:52.092612+0000","last_peered":"2024-04-25T00:35:52.092612+0000","last_clean":"2024-04-25T00:35:52.092612+0000","last_became_active":"2024-04-25T00:35:50.206184+0000","last_became_peered":"2024-04-25T00:35:50.206184+0000","last_unstale":"2024-04-25T00:35:52.092612+0000","last_undegraded":"2024-04-25T00:35:52.092612+0000","last_fullsized":"2024-04-25T00:35:52.092612+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:36.130476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204448+0000","last_change":"2024-04-25T00:35:50.230056+0000","last_active":"2024-04-25T00:35:51.204448+0000","last_peered":"2024-04-25T00:35:51.204448+0000","last_clean":"2024-04-25T00:35:51.204448+0000","last_became_active":"2024-04-25T00:35:50.229697+0000","last_became_peered":"2024-04-25T00:35:50.229697+0000","last_unstale":"2024-04-25T00:35:51.204448+0000","last_undegraded":"2024-04-25T00:35:51.204448+0000","last_fullsized":"2024-04-25T00:35:51.204448+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:44.765172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204503+0000","last_change":"2024-04-25T00:35:50.185426+0000","last_active":"2024-04-25T00:35:51.204503+0000","last_peered":"2024-04-25T00:35:51.204503+0000","last_clean":"2024-04-25T00:35:51.204503+0000","last_became_active":"2024-04-25T00:35:50.184675+0000","last_became_peered":"2024-04-25T00:35:50.184675+0000","last_unstale":"2024-04-25T00:35:51.204503+0000","last_undegraded":"2024-04-25T00:35:51.204503+0000","last_fullsized":"2024-04-25T00:35:51.204503+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:33.770812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588231+0000","last_change":"2024-04-25T00:35:50.231446+0000","last_active":"2024-04-25T00:35:51.588231+0000","last_peered":"2024-04-25T00:35:51.588231+0000","last_clean":"2024-04-25T00:35:51.588231+0000","last_became_active":"2024-04-25T00:35:50.228644+0000","last_became_peered":"2024-04-25T00:35:50.228644+0000","last_unstale":"2024-04-25T00:35:51.588231+0000","last_undegraded":"2024-04-25T00:35:51.588231+0000","last_fullsized":"2024-04-25T00:35:51.588231+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:02.142046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202603+0000","last_change":"2024-04-25T00:35:50.239150+0000","last_active":"2024-04-25T00:35:51.202603+0000","last_peered":"2024-04-25T00:35:51.202603+0000","last_clean":"2024-04-25T00:35:51.202603+0000","last_became_active":"2024-04-25T00:35:50.238677+0000","last_became_peered":"2024-04-25T00:35:50.238677+0000","last_unstale":"2024-04-25T00:35:51.202603+0000","last_undegraded":"2024-04-25T00:35:51.202603+0000","last_fullsized":"2024-04-25T00:35:51.202603+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:05.975194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203770+0000","last_change":"2024-04-25T00:35:49.166557+0000","last_active":"2024-04-25T00:35:51.203770+0000","last_peered":"2024-04-25T00:35:51.203770+0000","last_clean":"2024-04-25T00:35:51.203770+0000","last_became_active":"2024-04-25T00:35:49.165245+0000","last_became_peered":"2024-04-25T00:35:49.165245+0000","last_unstale":"2024-04-25T00:35:51.203770+0000","last_undegraded":"2024-04-25T00:35:51.203770+0000","last_fullsized":"2024-04-25T00:35:51.203770+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:16.989199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092806+0000","last_change":"2024-04-25T00:35:50.208909+0000","last_active":"2024-04-25T00:35:52.092806+0000","last_peered":"2024-04-25T00:35:52.092806+0000","last_clean":"2024-04-25T00:35:52.092806+0000","last_became_active":"2024-04-25T00:35:50.208382+0000","last_became_peered":"2024-04-25T00:35:50.208382+0000","last_unstale":"2024-04-25T00:35:52.092806+0000","last_undegraded":"2024-04-25T00:35:52.092806+0000","last_fullsized":"2024-04-25T00:35:52.092806+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:13.509256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.093194+0000","last_change":"2024-04-25T00:35:49.172238+0000","last_active":"2024-04-25T00:35:52.093194+0000","last_peered":"2024-04-25T00:35:52.093194+0000","last_clean":"2024-04-25T00:35:52.093194+0000","last_became_active":"2024-04-25T00:35:49.171747+0000","last_became_peered":"2024-04-25T00:35:49.171747+0000","last_unstale":"2024-04-25T00:35:52.093194+0000","last_undegraded":"2024-04-25T00:35:52.093194+0000","last_fullsized":"2024-04-25T00:35:52.093194+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:49.963360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588592+0000","last_change":"2024-04-25T00:35:50.235945+0000","last_active":"2024-04-25T00:35:51.588592+0000","last_peered":"2024-04-25T00:35:51.588592+0000","last_clean":"2024-04-25T00:35:51.588592+0000","last_became_active":"2024-04-25T00:35:50.232455+0000","last_became_peered":"2024-04-25T00:35:50.232455+0000","last_unstale":"2024-04-25T00:35:51.588592+0000","last_undegraded":"2024-04-25T00:35:51.588592+0000","last_fullsized":"2024-04-25T00:35:51.588592+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:01.995731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202477+0000","last_change":"2024-04-25T00:35:49.187713+0000","last_active":"2024-04-25T00:35:51.202477+0000","last_peered":"2024-04-25T00:35:51.202477+0000","last_clean":"2024-04-25T00:35:51.202477+0000","last_became_active":"2024-04-25T00:35:49.186956+0000","last_became_peered":"2024-04-25T00:35:49.186956+0000","last_unstale":"2024-04-25T00:35:51.202477+0000","last_undegraded":"2024-04-25T00:35:51.202477+0000","last_fullsized":"2024-04-25T00:35:51.202477+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:25.728034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203082+0000","last_change":"2024-04-25T00:35:50.208041+0000","last_active":"2024-04-25T00:35:51.203082+0000","last_peered":"2024-04-25T00:35:51.203082+0000","last_clean":"2024-04-25T00:35:51.203082+0000","last_became_active":"2024-04-25T00:35:50.207367+0000","last_became_peered":"2024-04-25T00:35:50.207367+0000","last_unstale":"2024-04-25T00:35:51.203082+0000","last_undegraded":"2024-04-25T00:35:51.203082+0000","last_fullsized":"2024-04-25T00:35:51.203082+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:05.829437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588851+0000","last_change":"2024-04-25T00:35:49.188864+0000","last_active":"2024-04-25T00:35:51.588851+0000","last_peered":"2024-04-25T00:35:51.588851+0000","last_clean":"2024-04-25T00:35:51.588851+0000","last_became_active":"2024-04-25T00:35:49.183783+0000","last_became_peered":"2024-04-25T00:35:49.183783+0000","last_unstale":"2024-04-25T00:35:51.588851+0000","last_undegraded":"2024-04-25T00:35:51.588851+0000","last_fullsized":"2024-04-25T00:35:51.588851+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:21.365941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204684+0000","last_change":"2024-04-25T00:35:50.210087+0000","last_active":"2024-04-25T00:35:51.204684+0000","last_peered":"2024-04-25T00:35:51.204684+0000","last_clean":"2024-04-25T00:35:51.204684+0000","last_became_active":"2024-04-25T00:35:50.209334+0000","last_became_peered":"2024-04-25T00:35:50.209334+0000","last_unstale":"2024-04-25T00:35:51.204684+0000","last_undegraded":"2024-04-25T00:35:51.204684+0000","last_fullsized":"2024-04-25T00:35:51.204684+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:39.340439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092891+0000","last_change":"2024-04-25T00:35:49.176223+0000","last_active":"2024-04-25T00:35:52.092891+0000","last_peered":"2024-04-25T00:35:52.092891+0000","last_clean":"2024-04-25T00:35:52.092891+0000","last_became_active":"2024-04-25T00:35:49.175671+0000","last_became_peered":"2024-04-25T00:35:49.175671+0000","last_unstale":"2024-04-25T00:35:52.092891+0000","last_undegraded":"2024-04-25T00:35:52.092891+0000","last_fullsized":"2024-04-25T00:35:52.092891+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:04.967038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.201673+0000","last_change":"2024-04-25T00:35:50.223221+0000","last_active":"2024-04-25T00:35:51.201673+0000","last_peered":"2024-04-25T00:35:51.201673+0000","last_clean":"2024-04-25T00:35:51.201673+0000","last_became_active":"2024-04-25T00:35:50.222805+0000","last_became_peered":"2024-04-25T00:35:50.222805+0000","last_unstale":"2024-04-25T00:35:51.201673+0000","last_undegraded":"2024-04-25T00:35:51.201673+0000","last_fullsized":"2024-04-25T00:35:51.201673+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:13.474838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588312+0000","last_change":"2024-04-25T00:35:49.189623+0000","last_active":"2024-04-25T00:35:51.588312+0000","last_peered":"2024-04-25T00:35:51.588312+0000","last_clean":"2024-04-25T00:35:51.588312+0000","last_became_active":"2024-04-25T00:35:49.188902+0000","last_became_peered":"2024-04-25T00:35:49.188902+0000","last_unstale":"2024-04-25T00:35:51.588312+0000","last_undegraded":"2024-04-25T00:35:51.588312+0000","last_fullsized":"2024-04-25T00:35:51.588312+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:28.542118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089564+0000","last_change":"2024-04-25T00:35:50.201247+0000","last_active":"2024-04-25T00:35:52.089564+0000","last_peered":"2024-04-25T00:35:52.089564+0000","last_clean":"2024-04-25T00:35:52.089564+0000","last_became_active":"2024-04-25T00:35:50.200861+0000","last_became_peered":"2024-04-25T00:35:50.200861+0000","last_unstale":"2024-04-25T00:35:52.089564+0000","last_undegraded":"2024-04-25T00:35:52.089564+0000","last_fullsized":"2024-04-25T00:35:52.089564+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:15.666442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089568+0000","last_change":"2024-04-25T00:35:49.176647+0000","last_active":"2024-04-25T00:35:52.089568+0000","last_peered":"2024-04-25T00:35:52.089568+0000","last_clean":"2024-04-25T00:35:52.089568+0000","last_became_active":"2024-04-25T00:35:49.175953+0000","last_became_peered":"2024-04-25T00:35:49.175953+0000","last_unstale":"2024-04-25T00:35:52.089568+0000","last_undegraded":"2024-04-25T00:35:52.089568+0000","last_fullsized":"2024-04-25T00:35:52.089568+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:12.501099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.088906+0000","last_change":"2024-04-25T00:35:50.220004+0000","last_active":"2024-04-25T00:35:52.088906+0000","last_peered":"2024-04-25T00:35:52.088906+0000","last_clean":"2024-04-25T00:35:52.088906+0000","last_became_active":"2024-04-25T00:35:50.219656+0000","last_became_peered":"2024-04-25T00:35:50.219656+0000","last_unstale":"2024-04-25T00:35:52.088906+0000","last_undegraded":"2024-04-25T00:35:52.088906+0000","last_fullsized":"2024-04-25T00:35:52.088906+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:39.340439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089707+0000","last_change":"2024-04-25T00:35:49.168520+0000","last_active":"2024-04-25T00:35:52.089707+0000","last_peered":"2024-04-25T00:35:52.089707+0000","last_clean":"2024-04-25T00:35:52.089707+0000","last_became_active":"2024-04-25T00:35:49.168120+0000","last_became_peered":"2024-04-25T00:35:49.168120+0000","last_unstale":"2024-04-25T00:35:52.089707+0000","last_undegraded":"2024-04-25T00:35:52.089707+0000","last_fullsized":"2024-04-25T00:35:52.089707+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:44.173013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.205418+0000","last_change":"2024-04-25T00:35:50.193291+0000","last_active":"2024-04-25T00:35:51.205418+0000","last_peered":"2024-04-25T00:35:51.205418+0000","last_clean":"2024-04-25T00:35:51.205418+0000","last_became_active":"2024-04-25T00:35:50.192517+0000","last_became_peered":"2024-04-25T00:35:50.192517+0000","last_unstale":"2024-04-25T00:35:51.205418+0000","last_undegraded":"2024-04-25T00:35:51.205418+0000","last_fullsized":"2024-04-25T00:35:51.205418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:55.595220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203174+0000","last_change":"2024-04-25T00:35:49.174126+0000","last_active":"2024-04-25T00:35:51.203174+0000","last_peered":"2024-04-25T00:35:51.203174+0000","last_clean":"2024-04-25T00:35:51.203174+0000","last_became_active":"2024-04-25T00:35:49.173415+0000","last_became_peered":"2024-04-25T00:35:49.173415+0000","last_unstale":"2024-04-25T00:35:51.203174+0000","last_undegraded":"2024-04-25T00:35:51.203174+0000","last_fullsized":"2024-04-25T00:35:51.203174+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:50.622430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203627+0000","last_change":"2024-04-25T00:35:50.242836+0000","last_active":"2024-04-25T00:35:51.203627+0000","last_peered":"2024-04-25T00:35:51.203627+0000","last_clean":"2024-04-25T00:35:51.203627+0000","last_became_active":"2024-04-25T00:35:50.242031+0000","last_became_peered":"2024-04-25T00:35:50.242031+0000","last_unstale":"2024-04-25T00:35:51.203627+0000","last_undegraded":"2024-04-25T00:35:51.203627+0000","last_fullsized":"2024-04-25T00:35:51.203627+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:15.543514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588894+0000","last_change":"2024-04-25T00:35:49.193426+0000","last_active":"2024-04-25T00:35:51.588894+0000","last_peered":"2024-04-25T00:35:51.588894+0000","last_clean":"2024-04-25T00:35:51.588894+0000","last_became_active":"2024-04-25T00:35:49.192672+0000","last_became_peered":"2024-04-25T00:35:49.192672+0000","last_unstale":"2024-04-25T00:35:51.588894+0000","last_undegraded":"2024-04-25T00:35:51.588894+0000","last_fullsized":"2024-04-25T00:35:51.588894+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:21.767289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.589018+0000","last_change":"2024-04-25T00:35:50.231624+0000","last_active":"2024-04-25T00:35:51.589018+0000","last_peered":"2024-04-25T00:35:51.589018+0000","last_clean":"2024-04-25T00:35:51.589018+0000","last_became_active":"2024-04-25T00:35:50.228072+0000","last_became_peered":"2024-04-25T00:35:50.228072+0000","last_unstale":"2024-04-25T00:35:51.589018+0000","last_undegraded":"2024-04-25T00:35:51.589018+0000","last_fullsized":"2024-04-25T00:35:51.589018+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:33.388855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092388+0000","last_change":"2024-04-25T00:35:49.188084+0000","last_active":"2024-04-25T00:35:52.092388+0000","last_peered":"2024-04-25T00:35:52.092388+0000","last_clean":"2024-04-25T00:35:52.092388+0000","last_became_active":"2024-04-25T00:35:49.187449+0000","last_became_peered":"2024-04-25T00:35:49.187449+0000","last_unstale":"2024-04-25T00:35:52.092388+0000","last_undegraded":"2024-04-25T00:35:52.092388+0000","last_fullsized":"2024-04-25T00:35:52.092388+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:03.363932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.090981+0000","last_change":"2024-04-25T00:35:50.197729+0000","last_active":"2024-04-25T00:35:52.090981+0000","last_peered":"2024-04-25T00:35:52.090981+0000","last_clean":"2024-04-25T00:35:52.090981+0000","last_became_active":"2024-04-25T00:35:50.197368+0000","last_became_peered":"2024-04-25T00:35:50.197368+0000","last_unstale":"2024-04-25T00:35:52.090981+0000","last_undegraded":"2024-04-25T00:35:52.090981+0000","last_fullsized":"2024-04-25T00:35:52.090981+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:48.995580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"69'6","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091075+0000","last_change":"2024-04-25T00:35:49.173021+0000","last_active":"2024-04-25T00:35:52.091075+0000","last_peered":"2024-04-25T00:35:52.091075+0000","last_clean":"2024-04-25T00:35:52.091075+0000","last_became_active":"2024-04-25T00:35:49.172717+0000","last_became_peered":"2024-04-25T00:35:49.172717+0000","last_unstale":"2024-04-25T00:35:52.091075+0000","last_undegraded":"2024-04-25T00:35:52.091075+0000","last_fullsized":"2024-04-25T00:35:52.091075+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:21.365941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.200092+0000","last_change":"2024-04-25T00:35:50.219561+0000","last_active":"2024-04-25T00:35:51.200092+0000","last_peered":"2024-04-25T00:35:51.200092+0000","last_clean":"2024-04-25T00:35:51.200092+0000","last_became_active":"2024-04-25T00:35:50.218790+0000","last_became_peered":"2024-04-25T00:35:50.218790+0000","last_unstale":"2024-04-25T00:35:51.200092+0000","last_undegraded":"2024-04-25T00:35:51.200092+0000","last_fullsized":"2024-04-25T00:35:51.200092+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:23.345172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089603+0000","last_change":"2024-04-25T00:35:49.173508+0000","last_active":"2024-04-25T00:35:52.089603+0000","last_peered":"2024-04-25T00:35:52.089603+0000","last_clean":"2024-04-25T00:35:52.089603+0000","last_became_active":"2024-04-25T00:35:49.173087+0000","last_became_peered":"2024-04-25T00:35:49.173087+0000","last_unstale":"2024-04-25T00:35:52.089603+0000","last_undegraded":"2024-04-25T00:35:52.089603+0000","last_fullsized":"2024-04-25T00:35:52.089603+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:18.890757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204296+0000","last_change":"2024-04-25T00:35:50.202056+0000","last_active":"2024-04-25T00:35:51.204296+0000","last_peered":"2024-04-25T00:35:51.204296+0000","last_clean":"2024-04-25T00:35:51.204296+0000","last_became_active":"2024-04-25T00:35:50.201331+0000","last_became_peered":"2024-04-25T00:35:50.201331+0000","last_unstale":"2024-04-25T00:35:51.204296+0000","last_undegraded":"2024-04-25T00:35:51.204296+0000","last_fullsized":"2024-04-25T00:35:51.204296+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:08.583461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588595+0000","last_change":"2024-04-25T00:35:49.193702+0000","last_active":"2024-04-25T00:35:51.588595+0000","last_peered":"2024-04-25T00:35:51.588595+0000","last_clean":"2024-04-25T00:35:51.588595+0000","last_became_active":"2024-04-25T00:35:49.192211+0000","last_became_peered":"2024-04-25T00:35:49.192211+0000","last_unstale":"2024-04-25T00:35:51.588595+0000","last_undegraded":"2024-04-25T00:35:51.588595+0000","last_fullsized":"2024-04-25T00:35:51.588595+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:30.680476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.088411+0000","last_change":"2024-04-25T00:35:50.229674+0000","last_active":"2024-04-25T00:35:52.088411+0000","last_peered":"2024-04-25T00:35:52.088411+0000","last_clean":"2024-04-25T00:35:52.088411+0000","last_became_active":"2024-04-25T00:35:50.229376+0000","last_became_peered":"2024-04-25T00:35:50.229376+0000","last_unstale":"2024-04-25T00:35:52.088411+0000","last_undegraded":"2024-04-25T00:35:52.088411+0000","last_fullsized":"2024-04-25T00:35:52.088411+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:08.583461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204201+0000","last_change":"2024-04-25T00:35:49.184169+0000","last_active":"2024-04-25T00:35:51.204201+0000","last_peered":"2024-04-25T00:35:51.204201+0000","last_clean":"2024-04-25T00:35:51.204201+0000","last_became_active":"2024-04-25T00:35:49.183700+0000","last_became_peered":"2024-04-25T00:35:49.183700+0000","last_unstale":"2024-04-25T00:35:51.204201+0000","last_undegraded":"2024-04-25T00:35:51.204201+0000","last_fullsized":"2024-04-25T00:35:51.204201+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:53.984973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092108+0000","last_change":"2024-04-25T00:35:50.210231+0000","last_active":"2024-04-25T00:35:52.092108+0000","last_peered":"2024-04-25T00:35:52.092108+0000","last_clean":"2024-04-25T00:35:52.092108+0000","last_became_active":"2024-04-25T00:35:50.209951+0000","last_became_peered":"2024-04-25T00:35:50.209951+0000","last_unstale":"2024-04-25T00:35:52.092108+0000","last_undegraded":"2024-04-25T00:35:52.092108+0000","last_fullsized":"2024-04-25T00:35:52.092108+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:58.762152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":22,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.201123+0000","last_change":"2024-04-25T00:35:49.176243+0000","last_active":"2024-04-25T00:35:51.201123+0000","last_peered":"2024-04-25T00:35:51.201123+0000","last_clean":"2024-04-25T00:35:51.201123+0000","last_became_active":"2024-04-25T00:35:49.175498+0000","last_became_peered":"2024-04-25T00:35:49.175498+0000","last_unstale":"2024-04-25T00:35:51.201123+0000","last_undegraded":"2024-04-25T00:35:51.201123+0000","last_fullsized":"2024-04-25T00:35:51.201123+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:21.365941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202630+0000","last_change":"2024-04-25T00:35:50.200674+0000","last_active":"2024-04-25T00:35:51.202630+0000","last_peered":"2024-04-25T00:35:51.202630+0000","last_clean":"2024-04-25T00:35:51.202630+0000","last_became_active":"2024-04-25T00:35:50.200235+0000","last_became_peered":"2024-04-25T00:35:50.200235+0000","last_unstale":"2024-04-25T00:35:51.202630+0000","last_undegraded":"2024-04-25T00:35:51.202630+0000","last_fullsized":"2024-04-25T00:35:51.202630+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:02.142046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588669+0000","last_change":"2024-04-25T00:35:49.191787+0000","last_active":"2024-04-25T00:35:51.588669+0000","last_peered":"2024-04-25T00:35:51.588669+0000","last_clean":"2024-04-25T00:35:51.588669+0000","last_became_active":"2024-04-25T00:35:49.191116+0000","last_became_peered":"2024-04-25T00:35:49.191116+0000","last_unstale":"2024-04-25T00:35:51.588669+0000","last_undegraded":"2024-04-25T00:35:51.588669+0000","last_fullsized":"2024-04-25T00:35:51.588669+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:19.303738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203912+0000","last_change":"2024-04-25T00:35:50.210860+0000","last_active":"2024-04-25T00:35:51.203912+0000","last_peered":"2024-04-25T00:35:51.203912+0000","last_clean":"2024-04-25T00:35:51.203912+0000","last_became_active":"2024-04-25T00:35:50.210148+0000","last_became_peered":"2024-04-25T00:35:50.210148+0000","last_unstale":"2024-04-25T00:35:51.203912+0000","last_undegraded":"2024-04-25T00:35:51.203912+0000","last_fullsized":"2024-04-25T00:35:51.203912+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:00.957517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.202737+0000","last_change":"2024-04-25T00:35:49.192932+0000","last_active":"2024-04-25T00:35:51.202737+0000","last_peered":"2024-04-25T00:35:51.202737+0000","last_clean":"2024-04-25T00:35:51.202737+0000","last_became_active":"2024-04-25T00:35:49.192527+0000","last_became_peered":"2024-04-25T00:35:49.192527+0000","last_unstale":"2024-04-25T00:35:51.202737+0000","last_undegraded":"2024-04-25T00:35:51.202737+0000","last_fullsized":"2024-04-25T00:35:51.202737+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:28.542118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091032+0000","last_change":"2024-04-25T00:35:50.203751+0000","last_active":"2024-04-25T00:35:52.091032+0000","last_peered":"2024-04-25T00:35:52.091032+0000","last_clean":"2024-04-25T00:35:52.091032+0000","last_became_active":"2024-04-25T00:35:50.203445+0000","last_became_peered":"2024-04-25T00:35:50.203445+0000","last_unstale":"2024-04-25T00:35:52.091032+0000","last_undegraded":"2024-04-25T00:35:52.091032+0000","last_fullsized":"2024-04-25T00:35:52.091032+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:26.055053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588506+0000","last_change":"2024-04-25T00:35:49.192591+0000","last_active":"2024-04-25T00:35:51.588506+0000","last_peered":"2024-04-25T00:35:51.588506+0000","last_clean":"2024-04-25T00:35:51.588506+0000","last_became_active":"2024-04-25T00:35:49.191599+0000","last_became_peered":"2024-04-25T00:35:49.191599+0000","last_unstale":"2024-04-25T00:35:51.588506+0000","last_undegraded":"2024-04-25T00:35:51.588506+0000","last_fullsized":"2024-04-25T00:35:51.588506+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:25.728034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091399+0000","last_change":"2024-04-25T00:35:50.214215+0000","last_active":"2024-04-25T00:35:52.091399+0000","last_peered":"2024-04-25T00:35:52.091399+0000","last_clean":"2024-04-25T00:35:52.091399+0000","last_became_active":"2024-04-25T00:35:50.213864+0000","last_became_peered":"2024-04-25T00:35:50.213864+0000","last_unstale":"2024-04-25T00:35:52.091399+0000","last_undegraded":"2024-04-25T00:35:52.091399+0000","last_fullsized":"2024-04-25T00:35:52.091399+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:17.241816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203160+0000","last_change":"2024-04-25T00:35:49.178876+0000","last_active":"2024-04-25T00:35:51.203160+0000","last_peered":"2024-04-25T00:35:51.203160+0000","last_clean":"2024-04-25T00:35:51.203160+0000","last_became_active":"2024-04-25T00:35:49.178162+0000","last_became_peered":"2024-04-25T00:35:49.178162+0000","last_unstale":"2024-04-25T00:35:51.203160+0000","last_undegraded":"2024-04-25T00:35:51.203160+0000","last_fullsized":"2024-04-25T00:35:51.203160+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:38.332283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092817+0000","last_change":"2024-04-25T00:35:50.206416+0000","last_active":"2024-04-25T00:35:52.092817+0000","last_peered":"2024-04-25T00:35:52.092817+0000","last_clean":"2024-04-25T00:35:52.092817+0000","last_became_active":"2024-04-25T00:35:50.205886+0000","last_became_peered":"2024-04-25T00:35:50.205886+0000","last_unstale":"2024-04-25T00:35:52.092817+0000","last_undegraded":"2024-04-25T00:35:52.092817+0000","last_fullsized":"2024-04-25T00:35:52.092817+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:21.592033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588851+0000","last_change":"2024-04-25T00:35:49.189004+0000","last_active":"2024-04-25T00:35:51.588851+0000","last_peered":"2024-04-25T00:35:51.588851+0000","last_clean":"2024-04-25T00:35:51.588851+0000","last_became_active":"2024-04-25T00:35:49.183196+0000","last_became_peered":"2024-04-25T00:35:49.183196+0000","last_unstale":"2024-04-25T00:35:51.588851+0000","last_undegraded":"2024-04-25T00:35:51.588851+0000","last_fullsized":"2024-04-25T00:35:51.588851+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:50.622430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091855+0000","last_change":"2024-04-25T00:35:50.229703+0000","last_active":"2024-04-25T00:35:52.091855+0000","last_peered":"2024-04-25T00:35:52.091855+0000","last_clean":"2024-04-25T00:35:52.091855+0000","last_became_active":"2024-04-25T00:35:50.229389+0000","last_became_peered":"2024-04-25T00:35:50.229389+0000","last_unstale":"2024-04-25T00:35:52.091855+0000","last_undegraded":"2024-04-25T00:35:52.091855+0000","last_fullsized":"2024-04-25T00:35:52.091855+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:40.507238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.204580+0000","last_change":"2024-04-25T00:35:49.170971+0000","last_active":"2024-04-25T00:35:51.204580+0000","last_peered":"2024-04-25T00:35:51.204580+0000","last_clean":"2024-04-25T00:35:51.204580+0000","last_became_active":"2024-04-25T00:35:49.170132+0000","last_became_peered":"2024-04-25T00:35:49.170132+0000","last_unstale":"2024-04-25T00:35:51.204580+0000","last_undegraded":"2024-04-25T00:35:51.204580+0000","last_fullsized":"2024-04-25T00:35:51.204580+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:12.466682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.092318+0000","last_change":"2024-04-25T00:35:50.229625+0000","last_active":"2024-04-25T00:35:52.092318+0000","last_peered":"2024-04-25T00:35:52.092318+0000","last_clean":"2024-04-25T00:35:52.092318+0000","last_became_active":"2024-04-25T00:35:50.229338+0000","last_became_peered":"2024-04-25T00:35:50.229338+0000","last_unstale":"2024-04-25T00:35:52.092318+0000","last_undegraded":"2024-04-25T00:35:52.092318+0000","last_fullsized":"2024-04-25T00:35:52.092318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:19.231960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.088078+0000","last_change":"2024-04-25T00:35:49.174136+0000","last_active":"2024-04-25T00:35:52.088078+0000","last_peered":"2024-04-25T00:35:52.088078+0000","last_clean":"2024-04-25T00:35:52.088078+0000","last_became_active":"2024-04-25T00:35:49.173623+0000","last_became_peered":"2024-04-25T00:35:49.173623+0000","last_unstale":"2024-04-25T00:35:52.088078+0000","last_undegraded":"2024-04-25T00:35:52.088078+0000","last_fullsized":"2024-04-25T00:35:52.088078+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:34.609362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.093044+0000","last_change":"2024-04-25T00:35:50.203655+0000","last_active":"2024-04-25T00:35:52.093044+0000","last_peered":"2024-04-25T00:35:52.093044+0000","last_clean":"2024-04-25T00:35:52.093044+0000","last_became_active":"2024-04-25T00:35:50.203357+0000","last_became_peered":"2024-04-25T00:35:50.203357+0000","last_unstale":"2024-04-25T00:35:52.093044+0000","last_undegraded":"2024-04-25T00:35:52.093044+0000","last_fullsized":"2024-04-25T00:35:52.093044+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:55.595220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588490+0000","last_change":"2024-04-25T00:35:49.190985+0000","last_active":"2024-04-25T00:35:51.588490+0000","last_peered":"2024-04-25T00:35:51.588490+0000","last_clean":"2024-04-25T00:35:51.588490+0000","last_became_active":"2024-04-25T00:35:49.190267+0000","last_became_peered":"2024-04-25T00:35:49.190267+0000","last_unstale":"2024-04-25T00:35:51.588490+0000","last_undegraded":"2024-04-25T00:35:51.588490+0000","last_fullsized":"2024-04-25T00:35:51.588490+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:54.517092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.205651+0000","last_change":"2024-04-25T00:35:50.203263+0000","last_active":"2024-04-25T00:35:51.205651+0000","last_peered":"2024-04-25T00:35:51.205651+0000","last_clean":"2024-04-25T00:35:51.205651+0000","last_became_active":"2024-04-25T00:35:50.202513+0000","last_became_peered":"2024-04-25T00:35:50.202513+0000","last_unstale":"2024-04-25T00:35:51.205651+0000","last_undegraded":"2024-04-25T00:35:51.205651+0000","last_fullsized":"2024-04-25T00:35:51.205651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:19.748799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.205449+0000","last_change":"2024-04-25T00:35:49.187128+0000","last_active":"2024-04-25T00:35:51.205449+0000","last_peered":"2024-04-25T00:35:51.205449+0000","last_clean":"2024-04-25T00:35:51.205449+0000","last_became_active":"2024-04-25T00:35:49.184464+0000","last_became_peered":"2024-04-25T00:35:49.184464+0000","last_unstale":"2024-04-25T00:35:51.205449+0000","last_undegraded":"2024-04-25T00:35:51.205449+0000","last_fullsized":"2024-04-25T00:35:51.205449+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:34.609362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203012+0000","last_change":"2024-04-25T00:35:50.239394+0000","last_active":"2024-04-25T00:35:51.203012+0000","last_peered":"2024-04-25T00:35:51.203012+0000","last_clean":"2024-04-25T00:35:51.203012+0000","last_became_active":"2024-04-25T00:35:50.238914+0000","last_became_peered":"2024-04-25T00:35:50.238914+0000","last_unstale":"2024-04-25T00:35:51.203012+0000","last_undegraded":"2024-04-25T00:35:51.203012+0000","last_fullsized":"2024-04-25T00:35:51.203012+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:50.847672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091453+0000","last_change":"2024-04-25T00:35:49.171187+0000","last_active":"2024-04-25T00:35:52.091453+0000","last_peered":"2024-04-25T00:35:52.091453+0000","last_clean":"2024-04-25T00:35:52.091453+0000","last_became_active":"2024-04-25T00:35:49.170719+0000","last_became_peered":"2024-04-25T00:35:49.170719+0000","last_unstale":"2024-04-25T00:35:52.091453+0000","last_undegraded":"2024-04-25T00:35:52.091453+0000","last_fullsized":"2024-04-25T00:35:52.091453+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:24.297162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.203473+0000","last_change":"2024-04-25T00:35:50.207396+0000","last_active":"2024-04-25T00:35:51.203473+0000","last_peered":"2024-04-25T00:35:51.203473+0000","last_clean":"2024-04-25T00:35:51.203473+0000","last_became_active":"2024-04-25T00:35:50.206584+0000","last_became_peered":"2024-04-25T00:35:50.206584+0000","last_unstale":"2024-04-25T00:35:51.203473+0000","last_undegraded":"2024-04-25T00:35:51.203473+0000","last_fullsized":"2024-04-25T00:35:51.203473+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:03.847803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.088992+0000","last_change":"2024-04-25T00:35:49.176291+0000","last_active":"2024-04-25T00:35:52.088992+0000","last_peered":"2024-04-25T00:35:52.088992+0000","last_clean":"2024-04-25T00:35:52.088992+0000","last_became_active":"2024-04-25T00:35:49.175847+0000","last_became_peered":"2024-04-25T00:35:49.175847+0000","last_unstale":"2024-04-25T00:35:52.088992+0000","last_undegraded":"2024-04-25T00:35:52.088992+0000","last_fullsized":"2024-04-25T00:35:52.088992+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:47.988021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091705+0000","last_change":"2024-04-25T00:35:50.209546+0000","last_active":"2024-04-25T00:35:52.091705+0000","last_peered":"2024-04-25T00:35:52.091705+0000","last_clean":"2024-04-25T00:35:52.091705+0000","last_became_active":"2024-04-25T00:35:50.209273+0000","last_became_peered":"2024-04-25T00:35:50.209273+0000","last_unstale":"2024-04-25T00:35:52.091705+0000","last_undegraded":"2024-04-25T00:35:52.091705+0000","last_fullsized":"2024-04-25T00:35:52.091705+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:56.549654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.090336+0000","last_change":"2024-04-25T00:35:49.188246+0000","last_active":"2024-04-25T00:35:52.090336+0000","last_peered":"2024-04-25T00:35:52.090336+0000","last_clean":"2024-04-25T00:35:52.090336+0000","last_became_active":"2024-04-25T00:35:49.187766+0000","last_became_peered":"2024-04-25T00:35:49.187766+0000","last_unstale":"2024-04-25T00:35:52.090336+0000","last_undegraded":"2024-04-25T00:35:52.090336+0000","last_fullsized":"2024-04-25T00:35:52.090336+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:35.641595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588496+0000","last_change":"2024-04-25T00:35:50.235717+0000","last_active":"2024-04-25T00:35:51.588496+0000","last_peered":"2024-04-25T00:35:51.588496+0000","last_clean":"2024-04-25T00:35:51.588496+0000","last_became_active":"2024-04-25T00:35:50.232769+0000","last_became_peered":"2024-04-25T00:35:50.232769+0000","last_unstale":"2024-04-25T00:35:51.588496+0000","last_undegraded":"2024-04-25T00:35:51.588496+0000","last_fullsized":"2024-04-25T00:35:51.588496+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:58.856012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089404+0000","last_change":"2024-04-25T00:35:49.177523+0000","last_active":"2024-04-25T00:35:52.089404+0000","last_peered":"2024-04-25T00:35:52.089404+0000","last_clean":"2024-04-25T00:35:52.089404+0000","last_became_active":"2024-04-25T00:35:49.177122+0000","last_became_peered":"2024-04-25T00:35:49.177122+0000","last_unstale":"2024-04-25T00:35:52.089404+0000","last_undegraded":"2024-04-25T00:35:52.089404+0000","last_fullsized":"2024-04-25T00:35:52.089404+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:04.207295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.089888+0000","last_change":"2024-04-25T00:35:50.209054+0000","last_active":"2024-04-25T00:35:52.089888+0000","last_peered":"2024-04-25T00:35:52.089888+0000","last_clean":"2024-04-25T00:35:52.089888+0000","last_became_active":"2024-04-25T00:35:50.208775+0000","last_became_peered":"2024-04-25T00:35:50.208775+0000","last_unstale":"2024-04-25T00:35:52.089888+0000","last_undegraded":"2024-04-25T00:35:52.089888+0000","last_fullsized":"2024-04-25T00:35:52.089888+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:31.688632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.201038+0000","last_change":"2024-04-25T00:35:49.177395+0000","last_active":"2024-04-25T00:35:51.201038+0000","last_peered":"2024-04-25T00:35:51.201038+0000","last_clean":"2024-04-25T00:35:51.201038+0000","last_became_active":"2024-04-25T00:35:49.175489+0000","last_became_peered":"2024-04-25T00:35:49.175489+0000","last_unstale":"2024-04-25T00:35:51.201038+0000","last_undegraded":"2024-04-25T00:35:51.201038+0000","last_fullsized":"2024-04-25T00:35:51.201038+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:09.958187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.587891+0000","last_change":"2024-04-25T00:35:50.214266+0000","last_active":"2024-04-25T00:35:51.587891+0000","last_peered":"2024-04-25T00:35:51.587891+0000","last_clean":"2024-04-25T00:35:51.587891+0000","last_became_active":"2024-04-25T00:35:50.213595+0000","last_became_peered":"2024-04-25T00:35:50.213595+0000","last_unstale":"2024-04-25T00:35:51.587891+0000","last_undegraded":"2024-04-25T00:35:51.587891+0000","last_fullsized":"2024-04-25T00:35:51.587891+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:35.617518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588389+0000","last_change":"2024-04-25T00:35:49.181943+0000","last_active":"2024-04-25T00:35:51.588389+0000","last_peered":"2024-04-25T00:35:51.588389+0000","last_clean":"2024-04-25T00:35:51.588389+0000","last_became_active":"2024-04-25T00:35:49.181556+0000","last_became_peered":"2024-04-25T00:35:49.181556+0000","last_unstale":"2024-04-25T00:35:51.588389+0000","last_undegraded":"2024-04-25T00:35:51.588389+0000","last_fullsized":"2024-04-25T00:35:51.588389+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:12.501099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:52.091418+0000","last_change":"2024-04-25T00:35:50.236132+0000","last_active":"2024-04-25T00:35:52.091418+0000","last_peered":"2024-04-25T00:35:52.091418+0000","last_clean":"2024-04-25T00:35:52.091418+0000","last_became_active":"2024-04-25T00:35:50.235896+0000","last_became_peered":"2024-04-25T00:35:50.235896+0000","last_unstale":"2024-04-25T00:35:52.091418+0000","last_undegraded":"2024-04-25T00:35:52.091418+0000","last_fullsized":"2024-04-25T00:35:52.091418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:49.141914+0000","last_clean_scrub_stamp":"2024-04-25T00:35:49.141914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:16.701600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.588061+0000","last_change":"2024-04-25T00:35:49.182981+0000","last_active":"2024-04-25T00:35:51.588061+0000","last_peered":"2024-04-25T00:35:51.588061+0000","last_clean":"2024-04-25T00:35:51.588061+0000","last_became_active":"2024-04-25T00:35:49.182211+0000","last_became_peered":"2024-04-25T00:35:49.182211+0000","last_unstale":"2024-04-25T00:35:51.588061+0000","last_undegraded":"2024-04-25T00:35:51.588061+0000","last_fullsized":"2024-04-25T00:35:51.588061+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:35:48.133758+0000","last_clean_scrub_stamp":"2024-04-25T00:35:48.133758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:04.207295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":266,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-25T00:35:51.199732+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:35:51.199732+0000","last_peered":"2024-04-25T00:35:51.199732+0000","last_clean":"2024-04-25T00:35:51.199732+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:35:51.199732+0000","last_undegraded":"2024-04-25T00:35:51.199732+0000","last_fullsized":"2024-04-25T00:35:51.199732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1454080,"data_stored":950312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":860160,"data_stored":359420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":856064,"data_stored":359418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36184,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335656,"statfs":{"total":96636764160,"available":96599711744,"internally_reserved":0,"allocated":856064,"data_stored":359398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335656,"statfs":{"total":96636764160,"available":96599711744,"internally_reserved":0,"allocated":860160,"data_stored":359440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36188,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335652,"statfs":{"total":96636764160,"available":96599707648,"internally_reserved":0,"allocated":864256,"data_stored":360940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335644,"statfs":{"total":96636764160,"available":96599699456,"internally_reserved":0,"allocated":872448,"data_stored":361556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36756,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335084,"statfs":{"total":96636764160,"available":96599126016,"internally_reserved":0,"allocated":1445888,"data_stored":949696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:35:59.059 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:35:59.060 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:35:59.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.220+0000 7f82021a7640 1 Processor -- start 2024-04-25T00:35:59.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.220+0000 7f82021a7640 1 -- start start 2024-04-25T00:35:59.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f82021a7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81fc15c220 0x7f81fc0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f82021a7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f81fc0d0670 0x7f81fc0b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f82021a7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81fc0cdbd0 con 0x7f81fc0cfa10 2024-04-25T00:35:59.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f82021a7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f81fc15c600 con 0x7f81fc0d0670 2024-04-25T00:35:59.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f82021a7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc0b1a30 con 0x7f81fc15c220 2024-04-25T00:35:59.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81f3fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f81fc0cfa10 0x7f81fc0cff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58496/0 (socket says 172.21.15.29:58496) 2024-04-25T00:35:59.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81f3fff640 1 -- 172.21.15.29:0/3772485446 learned_addr learned my addr 172.21.15.29:0/3772485446 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:59.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81fb7fe640 1 --2- 172.21.15.29:0/3772485446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81fc15c220 0x7f81fc0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81fbfff640 1 --2- 172.21.15.29:0/3772485446 >> v2:172.21.15.157:3300/0 conn(0x7f81fc0d0670 0x7f81fc0b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 586347787 0 0) 0x7f81fc0cdbd0 con 0x7f81fc0cfa10 2024-04-25T00:35:59.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81e8003530 con 0x7f81fc0cfa10 2024-04-25T00:35:59.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 630465697 0 0) 0x7f81e8003530 con 0x7f81fc0cfa10 2024-04-25T00:35:59.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81fc15c220 msgr2=0x7f81fc0cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 --2- 172.21.15.29:0/3772485446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81fc15c220 0x7f81fc0cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 >> v2:172.21.15.157:3300/0 conn(0x7f81fc0d0670 msgr2=0x7f81fc0b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 --2- 172.21.15.29:0/3772485446 >> v2:172.21.15.157:3300/0 conn(0x7f81fc0d0670 0x7f81fc0b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc0b13e0 con 0x7f81fc0cfa10 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f81e4001680 con 0x7f81fc0cfa10 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 >> v1:172.21.15.29:6789/0 conn(0x7f81fc0cfa10 legacy=0x7f81fc0cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 --2- 172.21.15.29:0/3772485446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e80039c0 0x7f81e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 --2- 172.21.15.29:0/3772485446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81fc15c220 0x7f81e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 --2- 172.21.15.29:0/3772485446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81e8004d50 0x7f81e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc15c600 con 0x7f81e8004d50 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0b1a30 con 0x7f81e80039c0 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f81e800b9d0 con 0x7f81fc15c220 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81fbfff640 1 --2- 172.21.15.29:0/3772485446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81e8004d50 0x7f81e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81fb7fe640 1 --2- 172.21.15.29:0/3772485446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e80039c0 0x7f81e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81f3fff640 1 --2- 172.21.15.29:0/3772485446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81fc15c220 0x7f81e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81fbfff640 1 -- 172.21.15.29:0/3772485446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81fc15c220 msgr2=0x7f81e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:59.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81fbfff640 1 --2- 172.21.15.29:0/3772485446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81fc15c220 0x7f81e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81fbfff640 1 -- 172.21.15.29:0/3772485446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e80039c0 msgr2=0x7f81e8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:59.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81fbfff640 1 --2- 172.21.15.29:0/3772485446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e80039c0 0x7f81e8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.224+0000 7f81fbfff640 1 -- 172.21.15.29:0/3772485446 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e8003530 con 0x7f81e8004d50 2024-04-25T00:35:59.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81fbfff640 1 --2- 172.21.15.29:0/3772485446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81e8004d50 0x7f81e80051d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009fc0 tx=0x7f81f401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2e4b45b5e9b484 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:59.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f40b2070 con 0x7f81e8004d50 2024-04-25T00:35:59.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f40a5cd0 con 0x7f81e8004d50 2024-04-25T00:35:59.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81faffd640 1 -- 172.21.15.29:0/3772485446 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4009c10 con 0x7f81e8004d50 2024-04-25T00:35:59.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 -- 172.21.15.29:0/3772485446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81e8004d50 msgr2=0x7f81e80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 --2- 172.21.15.29:0/3772485446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81e8004d50 0x7f81e80051d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009fc0 tx=0x7f81f401ccc0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81fbfff640 1 -- 172.21.15.29:0/3772485446 reap_dead start 2024-04-25T00:35:59.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 -- 172.21.15.29:0/3772485446 shutdown_connections 2024-04-25T00:35:59.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 -- 172.21.15.29:0/3772485446 >> 172.21.15.29:0/3772485446 conn(0x7f81fc0d1d60 msgr2=0x7f81fc0d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:59.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 -- 172.21.15.29:0/3772485446 shutdown_connections 2024-04-25T00:35:59.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 -- 172.21.15.29:0/3772485446 wait complete. 2024-04-25T00:35:59.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 Processor -- start 2024-04-25T00:35:59.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 -- start start 2024-04-25T00:35:59.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e8004d50 0x7f81fc0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81fc0c8590 0x7f81fc0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81fc14b020 0x7f81fc14b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0cd4d0 con 0x7f81fc0c8590 2024-04-25T00:35:59.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0cd350 con 0x7f81e8004d50 2024-04-25T00:35:59.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc0cd650 con 0x7f81fc14b020 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81f3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81fc0c8590 0x7f81fc0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81f3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81fc0c8590 0x7f81fc0b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42070/0 (socket says 172.21.15.29:42070) 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81f3fff640 1 -- 172.21.15.29:0/3461027185 learned_addr learned my addr 172.21.15.29:0/3461027185 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81fb7fe640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e8004d50 0x7f81fc0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81fbfff640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81fc14b020 0x7f81fc14b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81f3fff640 1 -- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81fc14b020 msgr2=0x7f81fc14b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81f3fff640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81fc14b020 0x7f81fc14b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81f3fff640 1 -- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e8004d50 msgr2=0x7f81fc0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81f3fff640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e8004d50 0x7f81fc0b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81f3fff640 1 -- 172.21.15.29:0/3461027185 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc0b3d90 con 0x7f81fc0c8590 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81fb7fe640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e8004d50 0x7f81fc0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81f3fff640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81fc0c8590 0x7f81fc0b7090 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f81e400e830 tx=0x7f81e400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81f8ff9640 1 -- 172.21.15.29:0/3461027185 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e400c870 con 0x7f81fc0c8590 2024-04-25T00:35:59.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81f8ff9640 1 -- 172.21.15.29:0/3461027185 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e4018070 con 0x7f81fc0c8590 2024-04-25T00:35:59.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 -- 172.21.15.29:0/3461027185 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81fc0b5d30 con 0x7f81fc0c8590 2024-04-25T00:35:59.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f81f8ff9640 1 -- 172.21.15.29:0/3461027185 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e400cc10 con 0x7f81fc0c8590 2024-04-25T00:35:59.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.228+0000 7f82021a7640 1 -- 172.21.15.29:0/3461027185 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81fc0b62d0 con 0x7f81fc0c8590 2024-04-25T00:35:59.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.232+0000 7f82021a7640 1 -- 172.21.15.29:0/3461027185 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81fc069270 con 0x7f81fc0c8590 2024-04-25T00:35:59.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.232+0000 7f81f8ff9640 1 -- 172.21.15.29:0/3461027185 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f81e401d460 con 0x7f81fc0c8590 2024-04-25T00:35:59.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.232+0000 7f81f8ff9640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81dc067c20 0x7f81dc06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.232+0000 7f81f8ff9640 1 -- 172.21.15.29:0/3461027185 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f81e4086110 con 0x7f81fc0c8590 2024-04-25T00:35:59.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.232+0000 7f81fb7fe640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81dc067c20 0x7f81dc06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.232+0000 7f81fb7fe640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81dc067c20 0x7f81dc06a0e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009a20 tx=0x7f81ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:59.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.236+0000 7f81f8ff9640 1 -- 172.21.15.29:0/3461027185 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f81e4056800 con 0x7f81fc0c8590 2024-04-25T00:35:59.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.452+0000 7f82021a7640 1 -- 172.21.15.29:0/3461027185 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f81fc15ed60 con 0x7f81fc0c8590 2024-04-25T00:35:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.456+0000 7f81f8ff9640 1 -- 172.21.15.29:0/3461027185 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f81e4056100 con 0x7f81fc0c8590 2024-04-25T00:35:59.457 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-04-25T00:35:59.457 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A== 2024-04-25T00:35:59.457 INFO:teuthology.orchestra.run.smithi029.stdout: caps mds = "allow" 2024-04-25T00:35:59.458 INFO:teuthology.orchestra.run.smithi029.stdout: caps mon = "allow r" 2024-04-25T00:35:59.458 INFO:teuthology.orchestra.run.smithi029.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:35:59.458 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-04-25T00:35:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.456+0000 7f82021a7640 1 -- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81dc067c20 msgr2=0x7f81dc06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.456+0000 7f82021a7640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81dc067c20 0x7f81dc06a0e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009a20 tx=0x7f81ec008040 comp rx=0 tx=0).stop 2024-04-25T00:35:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.456+0000 7f82021a7640 1 -- 172.21.15.29:0/3461027185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81fc0c8590 msgr2=0x7f81fc0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.456+0000 7f82021a7640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81fc0c8590 0x7f81fc0b7090 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f81e400e830 tx=0x7f81e400ecf0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.456+0000 7f82021a7640 1 -- 172.21.15.29:0/3461027185 shutdown_connections 2024-04-25T00:35:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.456+0000 7f82021a7640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81fc14b020 0x7f81fc14b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.456+0000 7f82021a7640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81fc0c8590 0x7f81fc0b7090 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.456+0000 7f82021a7640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f81dc067c20 0x7f81dc06a0e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.456+0000 7f82021a7640 1 --2- 172.21.15.29:0/3461027185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81e8004d50 0x7f81fc0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.456+0000 7f82021a7640 1 -- 172.21.15.29:0/3461027185 >> 172.21.15.29:0/3461027185 conn(0x7f81fc0d1d60 msgr2=0x7f81fc070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.456+0000 7f82021a7640 1 -- 172.21.15.29:0/3461027185 shutdown_connections 2024-04-25T00:35:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.456+0000 7f82021a7640 1 -- 172.21.15.29:0/3461027185 wait complete. 2024-04-25T00:35:59.475 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:35:59.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd274012640 1 Processor -- start 2024-04-25T00:35:59.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd274012640 1 -- start start 2024-04-25T00:35:59.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd274012640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06b920 0x7fd26c0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd274012640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd26c160190 con 0x7fd26c15ce60 2024-04-25T00:35:59.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd274012640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd26c15c680 con 0x7fd26c156420 2024-04-25T00:35:59.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd274012640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd26c15d2f0 con 0x7fd26c06b920 2024-04-25T00:35:59.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd2715ad640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd26c15ce60 0x7fd26c06b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58512/0 (socket says 172.21.15.29:58512) 2024-04-25T00:35:59.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd2715ad640 1 -- 172.21.15.29:0/3023316362 learned_addr learned my addr 172.21.15.29:0/3023316362 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:35:59.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd2725af640 1 --2- 172.21.15.29:0/3023316362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06b920 0x7fd26c0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 896061874 0 0) 0x7fd26c160190 con 0x7fd26c15ce60 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd258003530 con 0x7fd26c15ce60 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3503782033 0 0) 0x7fd258003530 con 0x7fd26c15ce60 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06b920 msgr2=0x7fd26c0cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 --2- 172.21.15.29:0/3023316362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06b920 0x7fd26c0cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 >> v1:172.21.15.157:6789/0 conn(0x7fd26c156420 legacy=0x7fd26c15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26c160440 con 0x7fd26c15ce60 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd25c001680 con 0x7fd26c15ce60 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 >> v1:172.21.15.29:6789/0 conn(0x7fd26c15ce60 legacy=0x7fd26c06b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 --2- 172.21.15.29:0/3023316362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2580039f0 0x7fd258003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 --2- 172.21.15.29:0/3023316362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06b920 0x7fd258004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 --2- 172.21.15.29:0/3023316362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd258004d50 0x7fd2580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c15d2f0 con 0x7fd258004d50 2024-04-25T00:35:59.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd25800bae0 con 0x7fd2580039f0 2024-04-25T00:35:59.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd25800bce0 con 0x7fd26c06b920 2024-04-25T00:35:59.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd2725af640 1 --2- 172.21.15.29:0/3023316362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd258004d50 0x7fd2580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd271dae640 1 --2- 172.21.15.29:0/3023316362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2580039f0 0x7fd258003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd2715ad640 1 --2- 172.21.15.29:0/3023316362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06b920 0x7fd258004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd2725af640 1 -- 172.21.15.29:0/3023316362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06b920 msgr2=0x7fd258004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:59.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd2725af640 1 --2- 172.21.15.29:0/3023316362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06b920 0x7fd258004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd2725af640 1 -- 172.21.15.29:0/3023316362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2580039f0 msgr2=0x7fd258003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:59.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd2725af640 1 --2- 172.21.15.29:0/3023316362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2580039f0 0x7fd258003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.596+0000 7fd2725af640 1 -- 172.21.15.29:0/3023316362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd258003530 con 0x7fd258004d50 2024-04-25T00:35:59.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.600+0000 7fd2725af640 1 --2- 172.21.15.29:0/3023316362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd258004d50 0x7fd2580051d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd264009f90 tx=0x7fd26401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde88da065b4e1e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:59.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.600+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2640b2070 con 0x7fd258004d50 2024-04-25T00:35:59.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.600+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2640a5c50 con 0x7fd258004d50 2024-04-25T00:35:59.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.600+0000 7fd270dac640 1 -- 172.21.15.29:0/3023316362 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264009c10 con 0x7fd258004d50 2024-04-25T00:35:59.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.600+0000 7fd274012640 1 -- 172.21.15.29:0/3023316362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd258004d50 msgr2=0x7fd2580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.600+0000 7fd274012640 1 --2- 172.21.15.29:0/3023316362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd258004d50 0x7fd2580051d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd264009f90 tx=0x7fd26401cc40 comp rx=0 tx=0).stop 2024-04-25T00:35:59.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.600+0000 7fd2725af640 1 -- 172.21.15.29:0/3023316362 reap_dead start 2024-04-25T00:35:59.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.600+0000 7fd274012640 1 -- 172.21.15.29:0/3023316362 shutdown_connections 2024-04-25T00:35:59.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.600+0000 7fd274012640 1 -- 172.21.15.29:0/3023316362 >> 172.21.15.29:0/3023316362 conn(0x7fd26c0d0c40 msgr2=0x7fd26c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:59.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.600+0000 7fd274012640 1 -- 172.21.15.29:0/3023316362 shutdown_connections 2024-04-25T00:35:59.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.600+0000 7fd274012640 1 -- 172.21.15.29:0/3023316362 wait complete. 2024-04-25T00:35:59.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.600+0000 7fd274012640 1 Processor -- start 2024-04-25T00:35:59.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.600+0000 7fd274012640 1 -- start start 2024-04-25T00:35:59.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd274012640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd258004d50 0x7fd26c148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd274012640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd26c1486c0 0x7fd26c0c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd274012640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c0c4770 0x7fd26c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd274012640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c156800 con 0x7fd26c1486c0 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd274012640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c15c940 con 0x7fd258004d50 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd274012640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd26c156980 con 0x7fd26c0c4770 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2715ad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd26c1486c0 0x7fd26c0c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2715ad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd26c1486c0 0x7fd26c0c69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42088/0 (socket says 172.21.15.29:42088) 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2715ad640 1 -- 172.21.15.29:0/2637219876 learned_addr learned my addr 172.21.15.29:0/2637219876 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd271dae640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd258004d50 0x7fd26c148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2725af640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c0c4770 0x7fd26c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2715ad640 1 -- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c0c4770 msgr2=0x7fd26c0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2715ad640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c0c4770 0x7fd26c0b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2715ad640 1 -- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd258004d50 msgr2=0x7fd26c148040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2715ad640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd258004d50 0x7fd26c148040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2715ad640 1 -- 172.21.15.29:0/2637219876 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26c14a5b0 con 0x7fd26c1486c0 2024-04-25T00:35:59.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2725af640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c0c4770 0x7fd26c0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:59.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2715ad640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd26c1486c0 0x7fd26c0c69a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00e7c0 tx=0x7fd25c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:59.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2627fc640 1 -- 172.21.15.29:0/2637219876 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c00c870 con 0x7fd26c1486c0 2024-04-25T00:35:59.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2627fc640 1 -- 172.21.15.29:0/2637219876 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd25c018070 con 0x7fd26c1486c0 2024-04-25T00:35:59.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd274012640 1 -- 172.21.15.29:0/2637219876 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26c0c7170 con 0x7fd26c1486c0 2024-04-25T00:35:59.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2627fc640 1 -- 172.21.15.29:0/2637219876 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c00cc70 con 0x7fd26c1486c0 2024-04-25T00:35:59.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd274012640 1 -- 172.21.15.29:0/2637219876 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26c0c76f0 con 0x7fd26c1486c0 2024-04-25T00:35:59.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd274012640 1 -- 172.21.15.29:0/2637219876 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26c069270 con 0x7fd26c1486c0 2024-04-25T00:35:59.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2627fc640 1 -- 172.21.15.29:0/2637219876 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fd25c01d460 con 0x7fd26c1486c0 2024-04-25T00:35:59.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2627fc640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd250067c20 0x7fd25006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:59.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd2627fc640 1 -- 172.21.15.29:0/2637219876 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fd25c086080 con 0x7fd26c1486c0 2024-04-25T00:35:59.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.604+0000 7fd271dae640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd250067c20 0x7fd25006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:59.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.608+0000 7fd271dae640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd250067c20 0x7fd25006a0e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd268000c10 tx=0x7fd268009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:59.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.608+0000 7fd2627fc640 1 -- 172.21.15.29:0/2637219876 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd25c0567b0 con 0x7fd26c1486c0 2024-04-25T00:35:59.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.804+0000 7fd274012640 1 -- 172.21.15.29:0/2637219876 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd26c0ccdf0 con 0x7fd26c1486c0 2024-04-25T00:35:59.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.804+0000 7fd2627fc640 1 -- 172.21.15.29:0/2637219876 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5186 (secure 0 0 0) 0x7fd25c0560b0 con 0x7fd26c1486c0 2024-04-25T00:35:59.807 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:35:59.808 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/524224365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":524224365},{"type":"v1","addr":"172.21.15.157:6835","nonce":524224365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1007000548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1007000548},{"type":"v1","addr":"172.21.15.157:6837","nonce":1007000548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6810,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2091414875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2091414875},{"type":"v1","addr":"172.21.15.29:6833","nonce":2091414875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:49.528297+0000","modified":"2024-04-25T00:35:52.569008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1111146974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1111146974},{"type":"v1","addr":"172.21.15.29:6835","nonce":1111146974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6822,"qdb_cluster":[6822]},"id":5}]} 2024-04-25T00:35:59.808 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 43 2024-04-25T00:35:59.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.808+0000 7fd274012640 1 -- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd250067c20 msgr2=0x7fd25006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.808+0000 7fd274012640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd250067c20 0x7fd25006a0e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd268000c10 tx=0x7fd268009040 comp rx=0 tx=0).stop 2024-04-25T00:35:59.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.808+0000 7fd274012640 1 -- 172.21.15.29:0/2637219876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd26c1486c0 msgr2=0x7fd26c0c69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:59.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.808+0000 7fd274012640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd26c1486c0 0x7fd26c0c69a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00e7c0 tx=0x7fd25c00ec80 comp rx=0 tx=0).stop 2024-04-25T00:35:59.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.808+0000 7fd274012640 1 -- 172.21.15.29:0/2637219876 shutdown_connections 2024-04-25T00:35:59.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.808+0000 7fd274012640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c0c4770 0x7fd26c0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.808+0000 7fd274012640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd26c1486c0 0x7fd26c0c69a0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.808+0000 7fd274012640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd250067c20 0x7fd25006a0e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.808+0000 7fd274012640 1 --2- 172.21.15.29:0/2637219876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd258004d50 0x7fd26c148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:59.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.808+0000 7fd274012640 1 -- 172.21.15.29:0/2637219876 >> 172.21.15.29:0/2637219876 conn(0x7fd26c0d0c40 msgr2=0x7fd26c071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:59.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.808+0000 7fd274012640 1 -- 172.21.15.29:0/2637219876 shutdown_connections 2024-04-25T00:35:59.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:35:59.808+0000 7fd274012640 1 -- 172.21.15.29:0/2637219876 wait complete. 2024-04-25T00:36:00.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:35:49.528297+0000', 'modified': '2024-04-25T00:35:52.569008+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6822}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6822': {'gid': 6822, 'name': 'c', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/1111146974', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 1111146974}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 1111146974}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6822, 'qdb_cluster': [6822]} 2024-04-25T00:36:00.228 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:36:00.228 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:36:00.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6aca4e640 1 Processor -- start 2024-04-25T00:36:00.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6aca4e640 1 -- start start 2024-04-25T00:36:00.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6aca4e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb6a815be30 0x7fb6a80ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6aca4e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb6a80d0860 0x7fb6a815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6aca4e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6a80722e0 con 0x7fb6a80ceeb0 2024-04-25T00:36:00.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6aca4e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb6a815fa90 con 0x7fb6a80d0860 2024-04-25T00:36:00.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6aca4e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb6a80ccd80 con 0x7fb6a815be30 2024-04-25T00:36:00.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6a6d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb6a80d0860 0x7fb6a815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6a6d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb6a80d0860 0x7fb6a815ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42094/0 (socket says 172.21.15.29:42094) 2024-04-25T00:36:00.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6a5d9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb6a80ceeb0 0x7fb6a80d0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58516/0 (socket says 172.21.15.29:58516) 2024-04-25T00:36:00.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/2981785714 learned_addr learned my addr 172.21.15.29:0/2981785714 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:00.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6a659c640 1 --2- 172.21.15.29:0/2981785714 >> v2:172.21.15.157:3300/0 conn(0x7fb6a815be30 0x7fb6a80ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/2981785714 >> v2:172.21.15.157:3300/0 conn(0x7fb6a815be30 msgr2=0x7fb6a80ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:00.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6a6d9d640 1 --2- 172.21.15.29:0/2981785714 >> v2:172.21.15.157:3300/0 conn(0x7fb6a815be30 0x7fb6a80ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/2981785714 >> v1:172.21.15.29:6789/0 conn(0x7fb6a80ceeb0 legacy=0x7fb6a80d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.352+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/2981785714 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a80cd060 con 0x7fb6a80d0860 2024-04-25T00:36:00.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a6d9d640 1 --2- 172.21.15.29:0/2981785714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6a80d0860 0x7fb6a815ff20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8074400 tx=0x7fb69801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a82e01922bdf0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:00.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a559a640 1 -- 172.21.15.29:0/2981785714 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980aa610 con 0x7fb6a80d0860 2024-04-25T00:36:00.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a559a640 1 -- 172.21.15.29:0/2981785714 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6980af070 con 0x7fb6a80d0860 2024-04-25T00:36:00.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a559a640 1 -- 172.21.15.29:0/2981785714 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980aa9d0 con 0x7fb6a80d0860 2024-04-25T00:36:00.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 -- 172.21.15.29:0/2981785714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6a80d0860 msgr2=0x7fb6a815ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 --2- 172.21.15.29:0/2981785714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6a80d0860 0x7fb6a815ff20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8074400 tx=0x7fb69801c4f0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 -- 172.21.15.29:0/2981785714 shutdown_connections 2024-04-25T00:36:00.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 --2- 172.21.15.29:0/2981785714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6a80d0860 0x7fb6a815ff20 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 --2- 172.21.15.29:0/2981785714 >> v2:172.21.15.157:3300/0 conn(0x7fb6a815be30 0x7fb6a80ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 -- 172.21.15.29:0/2981785714 >> 172.21.15.29:0/2981785714 conn(0x7fb6a8074c00 msgr2=0x7fb6a806b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:00.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 -- 172.21.15.29:0/2981785714 shutdown_connections 2024-04-25T00:36:00.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 -- 172.21.15.29:0/2981785714 wait complete. 2024-04-25T00:36:00.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 Processor -- start 2024-04-25T00:36:00.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 -- start start 2024-04-25T00:36:00.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a80ceeb0 0x7fb6a8159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6a80d0860 0x7fb6a8161670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6a815be30 0x7fb6a8157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a81605e0 con 0x7fb6a815be30 2024-04-25T00:36:00.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a8160460 con 0x7fb6a80ceeb0 2024-04-25T00:36:00.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6aca4e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a8160760 con 0x7fb6a80d0860 2024-04-25T00:36:00.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6a815be30 0x7fb6a8157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6a815be30 0x7fb6a8157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42106/0 (socket says 172.21.15.29:42106) 2024-04-25T00:36:00.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/1786402867 learned_addr learned my addr 172.21.15.29:0/1786402867 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:00.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a659c640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a80ceeb0 0x7fb6a8159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a5d9b640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6a80d0860 0x7fb6a8161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6a80d0860 msgr2=0x7fb6a8161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a6d9d640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6a80d0860 0x7fb6a8161670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a80ceeb0 msgr2=0x7fb6a8159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a6d9d640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a80ceeb0 0x7fb6a8159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/1786402867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a8160bd0 con 0x7fb6a815be30 2024-04-25T00:36:00.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.356+0000 7fb6a659c640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a80ceeb0 0x7fb6a8159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:00.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6a6d9d640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6a815be30 0x7fb6a8157730 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb6980ac040 tx=0x7fb69801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:00.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb69effd640 1 -- 172.21.15.29:0/1786402867 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980bd5f0 con 0x7fb6a815be30 2024-04-25T00:36:00.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb69effd640 1 -- 172.21.15.29:0/1786402867 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6980bd7b0 con 0x7fb6a815be30 2024-04-25T00:36:00.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6aca4e640 1 -- 172.21.15.29:0/1786402867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a8158340 con 0x7fb6a815be30 2024-04-25T00:36:00.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb69effd640 1 -- 172.21.15.29:0/1786402867 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980bdba0 con 0x7fb6a815be30 2024-04-25T00:36:00.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6aca4e640 1 -- 172.21.15.29:0/1786402867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a8161150 con 0x7fb6a815be30 2024-04-25T00:36:00.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb69effd640 1 -- 172.21.15.29:0/1786402867 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb6980b33f0 con 0x7fb6a815be30 2024-04-25T00:36:00.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb69effd640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb688067c20 0x7fb68806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb69effd640 1 -- 172.21.15.29:0/1786402867 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fb6981208d0 con 0x7fb6a815be30 2024-04-25T00:36:00.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6a659c640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb688067c20 0x7fb68806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6afbc7000 1 Processor -- start 2024-04-25T00:36:00.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6afbc7000 1 -- start start 2024-04-25T00:36:00.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6a659c640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb688067c20 0x7fb68806a0e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8156c30 tx=0x7fb6a000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:00.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6afbc7000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56185128f350 0x56185128f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6afbc7000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 0x561851245a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6afbc7000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 0x5618512463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6afbc7000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56185108f5e0 con 0x561851245fd0 2024-04-25T00:36:00.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6afbc7000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561851016c50 con 0x56185128fc70 2024-04-25T00:36:00.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6afbc7000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x56185108b5a0 con 0x56185128f350 2024-04-25T00:36:00.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6a6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 0x5618512463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6a6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 0x5618512463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42122/0 (socket says 172.21.15.29:42122) 2024-04-25T00:36:00.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/897354032 learned_addr learned my addr 172.21.15.29:0/897354032 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:00.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6a659c640 1 --2- 172.21.15.29:0/897354032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56185128f350 0x56185128f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.360+0000 7fb6a5d9b640 1 --2- 172.21.15.29:0/897354032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 0x561851245a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/897354032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56185128f350 msgr2=0x56185128f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:00.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a6d9d640 1 --2- 172.21.15.29:0/897354032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56185128f350 0x56185128f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/897354032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 msgr2=0x561851245a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:00.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a6d9d640 1 --2- 172.21.15.29:0/897354032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 0x561851245a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/897354032 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561851014d10 con 0x561851245fd0 2024-04-25T00:36:00.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a6d9d640 1 --2- 172.21.15.29:0/897354032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 0x5618512463b0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb6980f0540 tx=0x7fb6980f01a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfc0bbbdaddfe52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:00.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb687fff640 1 -- 172.21.15.29:0/897354032 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980f4bf0 con 0x561851245fd0 2024-04-25T00:36:00.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb687fff640 1 -- 172.21.15.29:0/897354032 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6980af070 con 0x561851245fd0 2024-04-25T00:36:00.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb687fff640 1 -- 172.21.15.29:0/897354032 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980f9420 con 0x561851245fd0 2024-04-25T00:36:00.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 -- 172.21.15.29:0/897354032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 msgr2=0x5618512463b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/897354032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 0x5618512463b0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb6980f0540 tx=0x7fb6980f01a0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 -- 172.21.15.29:0/897354032 shutdown_connections 2024-04-25T00:36:00.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/897354032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 0x5618512463b0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/897354032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 0x561851245a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/897354032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56185128f350 0x56185128f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 -- 172.21.15.29:0/897354032 >> 172.21.15.29:0/897354032 conn(0x56185123db70 msgr2=0x56185123ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:00.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 -- 172.21.15.29:0/897354032 shutdown_connections 2024-04-25T00:36:00.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 -- 172.21.15.29:0/897354032 wait complete. 2024-04-25T00:36:00.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 -- start start 2024-04-25T00:36:00.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 Processor -- start 2024-04-25T00:36:00.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 0x56185128e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56185128f350 0x56185128ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 0x561851272840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56185108f5e0 con 0x561851245fd0 2024-04-25T00:36:00.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561851049520 con 0x56185128fc70 2024-04-25T00:36:00.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x56185105c340 con 0x56185128f350 2024-04-25T00:36:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 10 client.0 _ll_register_callbacks cb 0x56185122c5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 0x56185128e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 0x56185128e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42138/0 (socket says 172.21.15.29:42138) 2024-04-25T00:36:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a659c640 1 -- 172.21.15.29:0/2492613517 learned_addr learned my addr 172.21.15.29:0/2492613517 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a5d9b640 1 --2- 172.21.15.29:0/2492613517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56185128f350 0x56185128ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a6d9d640 1 --2- 172.21.15.29:0/2492613517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 0x561851272840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a659c640 1 -- 172.21.15.29:0/2492613517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56185128f350 msgr2=0x56185128ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a659c640 1 --2- 172.21.15.29:0/2492613517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56185128f350 0x56185128ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a659c640 1 -- 172.21.15.29:0/2492613517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 msgr2=0x561851272840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a659c640 1 --2- 172.21.15.29:0/2492613517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 0x561851272840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a659c640 1 -- 172.21.15.29:0/2492613517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618510277b0 con 0x561851245fd0 2024-04-25T00:36:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6a659c640 1 --2- 172.21.15.29:0/2492613517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 0x56185128e730 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009800 tx=0x7fb6a0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7f09badcb652ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6857fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:36:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6857fa640 1 -- 172.21.15.29:0/2492613517 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0014c20 con 0x561851245fd0 2024-04-25T00:36:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6857fa640 1 -- 172.21.15.29:0/2492613517 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fb6a0014dc0 con 0x561851245fd0 2024-04-25T00:36:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2492613517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6980af070 con 0x561851245fd0 2024-04-25T00:36:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6857fa640 1 -- 172.21.15.29:0/2492613517 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb6a0022420 con 0x561851245fd0 2024-04-25T00:36:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 10 client.6996 fetch_fsmap learned FSMap version 43 2024-04-25T00:36:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2492613517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x561851016ea0 con 0x561851245fd0 2024-04-25T00:36:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6857fa640 1 -- 172.21.15.29:0/2492613517 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6980af070 con 0x561851245fd0 2024-04-25T00:36:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6857fa640 1 -- 172.21.15.29:0/2492613517 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7fb6a0032080 con 0x561851245fd0 2024-04-25T00:36:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.364+0000 7fb6afbc7000 10 client.6996 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T00:36:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 10 client.6996 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:36:00.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 20 client.6996 populate_metadata read hostname 'smithi029' 2024-04-25T00:36:00.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2492613517 >> [v2:172.21.15.29:6834/1111146974,v1:172.21.15.29:6835/1111146974] conn(0x561851273250 0x5618512a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 4 client.6996 mds_command: new command op to 6822 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:36:00.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2492613517 --> [v2:172.21.15.29:6834/1111146974,v1:172.21.15.29:6835/1111146974] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56185102aab0 con 0x561851273250 2024-04-25T00:36:00.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6a5d9b640 1 --2- 172.21.15.29:0/2492613517 >> [v2:172.21.15.29:6834/1111146974,v1:172.21.15.29:6835/1111146974] conn(0x561851273250 0x5618512a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6a5d9b640 1 --2- 172.21.15.29:0/2492613517 >> [v2:172.21.15.29:6834/1111146974,v1:172.21.15.29:6835/1111146974] conn(0x561851273250 0x5618512a03f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7038b85776f1221 server_cookie=9e584f977643f642 in_seq=0 out_seq=0 2024-04-25T00:36:00.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6857fa640 10 client.6996 ms_handle_connect on v2:172.21.15.29:6834/1111146974 2024-04-25T00:36:00.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6857fa640 1 -- 172.21.15.29:0/2492613517 <== mds.0 v2:172.21.15.29:6834/1111146974 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56185102aab0 con 0x561851273250 2024-04-25T00:36:00.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6857fa640 10 client.6996 handle_command_reply: tid=0 2024-04-25T00:36:00.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 1 client.6996 shutdown 2024-04-25T00:36:00.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2492613517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 msgr2=0x56185128e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2492613517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 0x56185128e730 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009800 tx=0x7fb6a0004ac0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2492613517 shutdown_connections 2024-04-25T00:36:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2492613517 >> [v2:172.21.15.29:6834/1111146974,v1:172.21.15.29:6835/1111146974] conn(0x561851273250 0x5618512a03f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2492613517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 0x561851272840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2492613517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56185128f350 0x56185128ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2492613517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561851245fd0 0x56185128e730 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6857fa640 0 client.6996 ms_handle_reset on v2:172.21.15.29:6834/1111146974 2024-04-25T00:36:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2492613517 >> 172.21.15.29:0/2492613517 conn(0x56185123db70 msgr2=0x561851246e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2492613517 shutdown_connections 2024-04-25T00:36:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2492613517 wait complete. 2024-04-25T00:36:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.368+0000 7fb6afbc7000 20 client.6996 trim_cache size 0 max 16384 2024-04-25T00:36:00.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6afbc7000 1 Processor -- start 2024-04-25T00:36:00.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6afbc7000 1 -- start start 2024-04-25T00:36:00.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6afbc7000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561851245fd0 0x56185128e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6afbc7000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128f350 0x56185128ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6afbc7000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128fc70 0x56185126eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6afbc7000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a80b5330 con 0x56185128fc70 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6afbc7000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a80b4690 con 0x56185128f350 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6afbc7000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a80b4ce0 con 0x561851245fd0 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6a6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128fc70 0x56185126eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6a6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128fc70 0x56185126eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42148/0 (socket says 172.21.15.29:42148) 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/2762523191 learned_addr learned my addr 172.21.15.29:0/2762523191 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6a659c640 1 --2- 172.21.15.29:0/2762523191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561851245fd0 0x56185128e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6a5d9b640 1 --2- 172.21.15.29:0/2762523191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128f350 0x56185128ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/2762523191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561851245fd0 msgr2=0x56185128e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6a6d9d640 1 --2- 172.21.15.29:0/2762523191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561851245fd0 0x56185128e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/2762523191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128f350 msgr2=0x56185128ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6a6d9d640 1 --2- 172.21.15.29:0/2762523191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128f350 0x56185128ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6a6d9d640 1 -- 172.21.15.29:0/2762523191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56185128faa0 con 0x56185128fc70 2024-04-25T00:36:00.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.384+0000 7fb6a659c640 1 --2- 172.21.15.29:0/2762523191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561851245fd0 0x56185128e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:00.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6a6d9d640 1 --2- 172.21.15.29:0/2762523191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128fc70 0x56185126eed0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb698019100 tx=0x7fb6980f2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b915bdc5a1f5f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:00.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6877fe640 1 -- 172.21.15.29:0/2762523191 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69810e070 con 0x56185128fc70 2024-04-25T00:36:00.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6877fe640 1 -- 172.21.15.29:0/2762523191 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6980f4bf0 con 0x56185128fc70 2024-04-25T00:36:00.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6877fe640 1 -- 172.21.15.29:0/2762523191 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980ecae0 con 0x56185128fc70 2024-04-25T00:36:00.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2762523191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128fc70 msgr2=0x56185126eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2762523191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128fc70 0x56185126eed0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb698019100 tx=0x7fb6980f2000 comp rx=0 tx=0).stop 2024-04-25T00:36:00.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2762523191 shutdown_connections 2024-04-25T00:36:00.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2762523191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128fc70 0x56185126eed0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2762523191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128f350 0x56185128ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2762523191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561851245fd0 0x56185128e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2762523191 >> 172.21.15.29:0/2762523191 conn(0x56185123db70 msgr2=0x561851271c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:00.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2762523191 shutdown_connections 2024-04-25T00:36:00.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2762523191 wait complete. 2024-04-25T00:36:00.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 -- start start 2024-04-25T00:36:00.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 Processor -- start 2024-04-25T00:36:00.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561851245fd0 0x5618512fed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128f350 0x5618512ff290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 0x5618512a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56185123d9f0 con 0x56185128f350 2024-04-25T00:36:00.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561851093c10 con 0x56185128fc70 2024-04-25T00:36:00.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x56185108e8f0 con 0x561851245fd0 2024-04-25T00:36:00.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 10 client.0 _ll_register_callbacks cb 0x7fb6a0009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:00.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6a5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128f350 0x5618512ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6a5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128f350 0x5618512ff290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42162/0 (socket says 172.21.15.29:42162) 2024-04-25T00:36:00.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6a5d9b640 1 -- 172.21.15.29:0/2637822479 learned_addr learned my addr 172.21.15.29:0/2637822479 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:00.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6a659c640 1 --2- 172.21.15.29:0/2637822479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561851245fd0 0x5618512fed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6a6d9d640 1 --2- 172.21.15.29:0/2637822479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 0x5618512a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6a5d9b640 1 -- 172.21.15.29:0/2637822479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561851245fd0 msgr2=0x5618512fed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6a5d9b640 1 --2- 172.21.15.29:0/2637822479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561851245fd0 0x5618512fed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6a5d9b640 1 -- 172.21.15.29:0/2637822479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 msgr2=0x5618512a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6a5d9b640 1 --2- 172.21.15.29:0/2637822479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 0x5618512a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6a5d9b640 1 -- 172.21.15.29:0/2637822479 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561851017e80 con 0x56185128f350 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6a5d9b640 1 --2- 172.21.15.29:0/2637822479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128f350 0x5618512ff290 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fb690000fe0 tx=0x7fb6900074a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ec3f6062bc236b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6857fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6857fa640 1 -- 172.21.15.29:0/2637822479 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690026060 con 0x56185128f350 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2637822479 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6980f4bf0 con 0x56185128f350 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6857fa640 1 -- 172.21.15.29:0/2637822479 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fb6900075d0 con 0x56185128f350 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6857fa640 1 -- 172.21.15.29:0/2637822479 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb69002c050 con 0x56185128f350 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 10 client.7002 fetch_fsmap learned FSMap version 43 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2637822479 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x56185105c760 con 0x56185128f350 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6857fa640 1 -- 172.21.15.29:0/2637822479 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6980f4bf0 con 0x56185128f350 2024-04-25T00:36:00.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6857fa640 1 -- 172.21.15.29:0/2637822479 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7fb69002f020 con 0x56185128f350 2024-04-25T00:36:00.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 10 client.7002 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T00:36:00.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 10 client.7002 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:36:00.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 20 client.7002 populate_metadata read hostname 'smithi029' 2024-04-25T00:36:00.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2637822479 >> [v2:172.21.15.29:6834/1111146974,v1:172.21.15.29:6835/1111146974] conn(0x561851305c20 0x561851308010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 4 client.7002 mds_command: new command op to 6822 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:36:00.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2637822479 --> [v2:172.21.15.29:6834/1111146974,v1:172.21.15.29:6835/1111146974] -- command(tid 0: {"prefix": "status"}) -- 0x56185109cb20 con 0x561851305c20 2024-04-25T00:36:00.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.388+0000 7fb6a659c640 1 --2- 172.21.15.29:0/2637822479 >> [v2:172.21.15.29:6834/1111146974,v1:172.21.15.29:6835/1111146974] conn(0x561851305c20 0x561851308010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6857fa640 10 client.7002 ms_handle_connect on v2:172.21.15.29:6834/1111146974 2024-04-25T00:36:00.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6a659c640 1 --2- 172.21.15.29:0/2637822479 >> [v2:172.21.15.29:6834/1111146974,v1:172.21.15.29:6835/1111146974] conn(0x561851305c20 0x561851308010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e0f5a536bab33e6 server_cookie=8dca122863b65631 in_seq=0 out_seq=0 2024-04-25T00:36:00.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6857fa640 1 -- 172.21.15.29:0/2637822479 <== mds.0 v2:172.21.15.29:6834/1111146974 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x56185109cb20 con 0x561851305c20 2024-04-25T00:36:00.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6857fa640 10 client.7002 handle_command_reply: tid=0 2024-04-25T00:36:00.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6afbc7000 1 client.7002 shutdown 2024-04-25T00:36:00.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2637822479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128f350 msgr2=0x5618512ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2637822479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128f350 0x5618512ff290 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fb690000fe0 tx=0x7fb6900074a0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2637822479 shutdown_connections 2024-04-25T00:36:00.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2637822479 >> [v2:172.21.15.29:6834/1111146974,v1:172.21.15.29:6835/1111146974] conn(0x561851305c20 0x561851308010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2637822479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56185128fc70 0x5618512a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2637822479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56185128f350 0x5618512ff290 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6857fa640 0 client.7002 ms_handle_reset on v2:172.21.15.29:6834/1111146974 2024-04-25T00:36:00.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6afbc7000 1 --2- 172.21.15.29:0/2637822479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561851245fd0 0x5618512fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2637822479 >> 172.21.15.29:0/2637822479 conn(0x56185123db70 msgr2=0x5618512723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:00.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2637822479 shutdown_connections 2024-04-25T00:36:00.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6afbc7000 1 -- 172.21.15.29:0/2637822479 wait complete. 2024-04-25T00:36:00.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6afbc7000 20 client.7002 trim_cache size 0 max 16384 2024-04-25T00:36:00.400 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:36:00.400 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "59afbe7d-aaa8-45e2-b468-69ee150cacea", 2024-04-25T00:36:00.400 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-04-25T00:36:00.400 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 6822, 2024-04-25T00:36:00.400 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-04-25T00:36:00.400 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-04-25T00:36:00.400 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-04-25T00:36:00.400 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 10.205732217, 2024-04-25T00:36:00.400 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 43, 2024-04-25T00:36:00.400 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 70, 2024-04-25T00:36:00.400 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 70, 2024-04-25T00:36:00.401 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 16.217885826 2024-04-25T00:36:00.401 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:36:00.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6aca4e640 1 -- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb688067c20 msgr2=0x7fb68806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6aca4e640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb688067c20 0x7fb68806a0e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8156c30 tx=0x7fb6a000b040 comp rx=0 tx=0).stop 2024-04-25T00:36:00.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6aca4e640 1 -- 172.21.15.29:0/1786402867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6a815be30 msgr2=0x7fb6a8157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6aca4e640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6a815be30 0x7fb6a8157730 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb6980ac040 tx=0x7fb69801ad30 comp rx=0 tx=0).stop 2024-04-25T00:36:00.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6aca4e640 1 -- 172.21.15.29:0/1786402867 shutdown_connections 2024-04-25T00:36:00.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6aca4e640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6a815be30 0x7fb6a8157730 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6aca4e640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6a80d0860 0x7fb6a8161670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6aca4e640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb688067c20 0x7fb68806a0e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6aca4e640 1 --2- 172.21.15.29:0/1786402867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a80ceeb0 0x7fb6a8159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6aca4e640 1 -- 172.21.15.29:0/1786402867 >> 172.21.15.29:0/1786402867 conn(0x7fb6a8074c00 msgr2=0x7fb6a80b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:00.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6aca4e640 1 -- 172.21.15.29:0/1786402867 shutdown_connections 2024-04-25T00:36:00.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.392+0000 7fb6aca4e640 1 -- 172.21.15.29:0/1786402867 wait complete. 2024-04-25T00:36:00.409 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:36:00.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.512+0000 7fad909fd640 1 Processor -- start 2024-04-25T00:36:00.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.512+0000 7fad909fd640 1 -- start start 2024-04-25T00:36:00.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.512+0000 7fad909fd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad8c15e320 0x7fad8c06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.512+0000 7fad909fd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fad8c06d270 0x7fad8c0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.512+0000 7fad909fd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad8c0d0100 con 0x7fad8c15df40 2024-04-25T00:36:00.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.512+0000 7fad909fd640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fad8c0d03a0 con 0x7fad8c06d270 2024-04-25T00:36:00.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.512+0000 7fad909fd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fad8c0d0680 con 0x7fad8c15e320 2024-04-25T00:36:00.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.512+0000 7fad8ad9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fad8c06d270 0x7fad8c0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.512+0000 7fad8ad9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fad8c06d270 0x7fad8c0cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42168/0 (socket says 172.21.15.29:42168) 2024-04-25T00:36:00.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.512+0000 7fad8ad9d640 1 -- 172.21.15.29:0/4011820601 learned_addr learned my addr 172.21.15.29:0/4011820601 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:00.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.512+0000 7fad89d9b640 1 --2- 172.21.15.29:0/4011820601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad8c15e320 0x7fad8c06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad8ad9d640 1 -- 172.21.15.29:0/4011820601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad8c15e320 msgr2=0x7fad8c06cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:00.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad8ad9d640 1 --2- 172.21.15.29:0/4011820601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad8c15e320 0x7fad8c06cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad8ad9d640 1 -- 172.21.15.29:0/4011820601 >> v1:172.21.15.157:6789/0 conn(0x7fad8c15df40 legacy=0x7fad8c06c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:00.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad8ad9d640 1 -- 172.21.15.29:0/4011820601 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8c0d0960 con 0x7fad8c06d270 2024-04-25T00:36:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad8ad9d640 1 --2- 172.21.15.29:0/4011820601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad8c06d270 0x7fad8c0cc370 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fad84009e50 tx=0x7fad8401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd06c6a79bc97d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad8959a640 1 -- 172.21.15.29:0/4011820601 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad840a5780 con 0x7fad8c06d270 2024-04-25T00:36:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad8959a640 1 -- 172.21.15.29:0/4011820601 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad840a5920 con 0x7fad8c06d270 2024-04-25T00:36:00.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad8959a640 1 -- 172.21.15.29:0/4011820601 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad840a5cf0 con 0x7fad8c06d270 2024-04-25T00:36:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad909fd640 1 -- 172.21.15.29:0/4011820601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad8c06d270 msgr2=0x7fad8c0cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad909fd640 1 --2- 172.21.15.29:0/4011820601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad8c06d270 0x7fad8c0cc370 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fad84009e50 tx=0x7fad8401c9f0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad909fd640 1 -- 172.21.15.29:0/4011820601 shutdown_connections 2024-04-25T00:36:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad909fd640 1 --2- 172.21.15.29:0/4011820601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad8c06d270 0x7fad8c0cc370 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad909fd640 1 --2- 172.21.15.29:0/4011820601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad8c15e320 0x7fad8c06cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad909fd640 1 -- 172.21.15.29:0/4011820601 >> 172.21.15.29:0/4011820601 conn(0x7fad8c0d1d20 msgr2=0x7fad8c0d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad909fd640 1 -- 172.21.15.29:0/4011820601 shutdown_connections 2024-04-25T00:36:00.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.516+0000 7fad909fd640 1 -- 172.21.15.29:0/4011820601 wait complete. 2024-04-25T00:36:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad909fd640 1 Processor -- start 2024-04-25T00:36:00.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad909fd640 1 -- start start 2024-04-25T00:36:00.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad909fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad8c06d270 0x7fad8c15c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad909fd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad8c15df40 0x7fad8c0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad909fd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad8c15e320 0x7fad8c144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad909fd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c0ce5a0 con 0x7fad8c15df40 2024-04-25T00:36:00.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad909fd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c0d13b0 con 0x7fad8c06d270 2024-04-25T00:36:00.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad909fd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fad8c0ce720 con 0x7fad8c15e320 2024-04-25T00:36:00.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad89d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad8c15df40 0x7fad8c0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad89d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad8c15df40 0x7fad8c0cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42176/0 (socket says 172.21.15.29:42176) 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad89d9b640 1 -- 172.21.15.29:0/1685464464 learned_addr learned my addr 172.21.15.29:0/1685464464 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad8ad9d640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad8c15e320 0x7fad8c144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad8a59c640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad8c06d270 0x7fad8c15c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad89d9b640 1 -- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad8c15e320 msgr2=0x7fad8c144c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad89d9b640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad8c15e320 0x7fad8c144c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad89d9b640 1 -- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad8c06d270 msgr2=0x7fad8c15c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad89d9b640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad8c06d270 0x7fad8c15c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad89d9b640 1 -- 172.21.15.29:0/1685464464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8c0c8b90 con 0x7fad8c15df40 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad8ad9d640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad8c15e320 0x7fad8c144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad8a59c640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad8c06d270 0x7fad8c15c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad89d9b640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad8c15df40 0x7fad8c0cb8b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fad8000d800 tx=0x7fad8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad7affd640 1 -- 172.21.15.29:0/1685464464 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8000b6c0 con 0x7fad8c15df40 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad7affd640 1 -- 172.21.15.29:0/1685464464 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad80019070 con 0x7fad8c15df40 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad909fd640 1 -- 172.21.15.29:0/1685464464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8c1535f0 con 0x7fad8c15df40 2024-04-25T00:36:00.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad7affd640 1 -- 172.21.15.29:0/1685464464 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8000ba80 con 0x7fad8c15df40 2024-04-25T00:36:00.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad909fd640 1 -- 172.21.15.29:0/1685464464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad8c15cf30 con 0x7fad8c15df40 2024-04-25T00:36:00.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad909fd640 1 -- 172.21.15.29:0/1685464464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8c153850 con 0x7fad8c15df40 2024-04-25T00:36:00.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad7affd640 1 -- 172.21.15.29:0/1685464464 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fad8000bc20 con 0x7fad8c15df40 2024-04-25T00:36:00.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad7affd640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fad6c067c20 0x7fad6c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:00.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.520+0000 7fad7affd640 1 -- 172.21.15.29:0/1685464464 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fad800865f0 con 0x7fad8c15df40 2024-04-25T00:36:00.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.524+0000 7fad8a59c640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fad6c067c20 0x7fad6c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:00.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.524+0000 7fad8a59c640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fad6c067c20 0x7fad6c06a0e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fad7c003e40 tx=0x7fad7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:00.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.528+0000 7fad7affd640 1 -- 172.21.15.29:0/1685464464 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fad80056d20 con 0x7fad8c15df40 2024-04-25T00:36:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad909fd640 1 -- 172.21.15.29:0/1685464464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fad8c0d1860 con 0x7fad8c15df40 2024-04-25T00:36:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad7affd640 1 -- 172.21.15.29:0/1685464464 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) ==== 53+0+73 (secure 0 0 0) 0x7fad80056620 con 0x7fad8c15df40 2024-04-25T00:36:00.722 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:36:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad909fd640 1 -- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fad6c067c20 msgr2=0x7fad6c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad909fd640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fad6c067c20 0x7fad6c06a0e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fad7c003e40 tx=0x7fad7c009040 comp rx=0 tx=0).stop 2024-04-25T00:36:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad909fd640 1 -- 172.21.15.29:0/1685464464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad8c15df40 msgr2=0x7fad8c0cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad909fd640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad8c15df40 0x7fad8c0cb8b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fad8000d800 tx=0x7fad8000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad909fd640 1 -- 172.21.15.29:0/1685464464 shutdown_connections 2024-04-25T00:36:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad909fd640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad8c15e320 0x7fad8c144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad909fd640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad8c15df40 0x7fad8c0cb8b0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad909fd640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fad6c067c20 0x7fad6c06a0e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad909fd640 1 --2- 172.21.15.29:0/1685464464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad8c06d270 0x7fad8c15c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad909fd640 1 -- 172.21.15.29:0/1685464464 >> 172.21.15.29:0/1685464464 conn(0x7fad8c0d1d20 msgr2=0x7fad8c0aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:00.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad909fd640 1 -- 172.21.15.29:0/1685464464 shutdown_connections 2024-04-25T00:36:00.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00.720+0000 7fad909fd640 1 -- 172.21.15.29:0/1685464464 wait complete. 2024-04-25T00:36:00.739 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:36:00.739 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:36:00.739 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-25T00:36:00.739 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:36:00.739 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:00.740 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:36:00.740 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:36:00.740 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:36:00.740 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:36:00.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:00.740 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-25T00:36:00.745 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:36:00.746 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:36:00.746 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:36:00.746 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:00.746 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:36:00.746 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42199sec preferred_lft 42199sec 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:00.747 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:36:00.748 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:00.748 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-25T00:36:00.748 INFO:teuthology.orchestra.run.smithi029.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:36:00.748 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 02:42:28:a5:36:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:00.748 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:36:00.748 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:00.748 INFO:teuthology.orchestra.run.smithi029.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:36:00.748 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:00.748 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:36:00.748 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:00.748 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:36:00.748 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:00.748 INFO:teuthology.orchestra.run.smithi029.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:36:00.749 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:00.749 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:36:00.749 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:00.749 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:36:00.749 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:00.749 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:36:00.749 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:00.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:00.749 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:36:00.805 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-25T00:36:00.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:00.846 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-25T00:36:00.905 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-25T00:36:00.905 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:36:00.905 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T00:36:00.906 INFO:teuthology.orchestra.run.smithi029.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:36:00.906 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T00:36:00.906 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:36:00.906 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:36:00.906 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:36:00.906 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:36:00.906 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:36:00.906 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:36:00.907 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:36:00.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:36:01.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:36:01.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:01.035 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:36:01.042 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:36:01.042 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:36:01.046 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:36:01.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:01.046 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-04-25T00:36:01.102 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:36:01.102 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:36:01.102 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:01.155 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:36:01.155 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:36:01.155 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:01.206 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:36:01.264 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:36:01.314 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:36:01.314 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:36:01.314 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:36:01.314 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:36:01.315 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:01.316 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:01.317 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:01.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:01.317 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:36:01.365 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:36:01.365 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:36:01.410 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:36:01.425 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:36:01.473 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:36:01.474 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:36:01.474 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:36:01.474 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:36:01.474 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:36:01.474 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:36:01.474 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:36:01.474 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:36:01.474 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:36:01.474 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:36:01.474 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:36:01.474 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:36:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:36:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:36:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:36:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:36:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:36:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:36:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:36:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:36:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:36:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:36:01.475 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:36:01.476 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:36:01.476 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:36:01.476 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:36:01.476 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:36:01.476 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:36:01.476 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:36:01.476 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:01.476 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:01.476 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:01.476 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:01.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:01.477 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:36:01.491 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:2024-04-25T00:36:01.488+0000 7f4d55bc3580 -1 init, newargv = 0x5635adc300c0 newargc=15 2024-04-25T00:36:01.491 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[43867]: starting ceph client 2024-04-25T00:36:01.496 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[43867]: starting fuse 2024-04-25T00:36:02.598 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:36:02.614 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:36:02.664 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:36:02.664 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:36:02.664 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:36:02.664 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:36:02.664 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:36:02.664 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:36:02.664 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:36:02.665 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:36:02.666 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:36:02.666 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:36:02.666 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:36:02.666 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:36:02.666 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:36:02.666 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:36:02.666 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:36:02.666 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:36:02.666 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:02.667 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:02.667 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:02.667 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:02.667 INFO:teuthology.orchestra.run.smithi029.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:02.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:02.668 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:36:02.717 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:36:02.717 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:36:02.718 DEBUG:teuthology.orchestra.run.smithi029:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:36:02.718 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-04-25T00:36:02.718 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-04-25T00:36:02.718 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-04-25T00:36:02.718 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-04-25T00:36:02.718 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:36:02.718 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-04-25T00:36:02.718 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:36:02.718 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-04-25T00:36:02.718 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:36:02.718 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:36:02.718 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.0")) 2024-04-25T00:36:02.719 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-04-25T00:36:02.844 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.0.43867.asok 2024-04-25T00:36:02.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:36:02.856 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43867.asok 2024-04-25T00:36:02.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:02.857 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43867.asok status 2024-04-25T00:36:03.018 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:36:03.018 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-04-25T00:36:03.018 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:36:03.018 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:36:03.019 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "0", 2024-04-25T00:36:03.019 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-04-25T00:36:03.019 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:36:03.019 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "43867", 2024-04-25T00:36:03.019 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-04-25T00:36:03.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:03.019 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-04-25T00:36:03.019 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-04-25T00:36:03.019 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 7014, 2024-04-25T00:36:03.019 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-04-25T00:36:03.019 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-04-25T00:36:03.019 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-04-25T00:36:03.019 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 7014 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 284810620 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 284810620 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.7014 192.168.0.1:0/284810620", 2024-04-25T00:36:03.020 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.1:0/284810620", 2024-04-25T00:36:03.021 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-04-25T00:36:03.021 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 43, 2024-04-25T00:36:03.021 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 70, 2024-04-25T00:36:03.021 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:36:03.021 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-04-25T00:36:03.021 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-04-25T00:36:03.021 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:36:03.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:03.032 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-04-25T00:36:03.032 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:03.038 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-04-25T00:36:03.038 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:03.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:03.039 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:03.093 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:36:03.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.256+0000 7f6fb825a640 1 Processor -- start 2024-04-25T00:36:03.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.256+0000 7f6fb825a640 1 -- start start 2024-04-25T00:36:03.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb825a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6fb00b4780 0x7f6fb015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:03.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb825a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fb00b3720 con 0x7f6fb015e8d0 2024-04-25T00:36:03.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb825a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fb00b39c0 con 0x7f6fb00cc240 2024-04-25T00:36:03.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb825a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f6fb00cc650 con 0x7f6fb00b4780 2024-04-25T00:36:03.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb5ff6640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6fb015e8d0 0x7f6fb015ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58530/0 (socket says 172.21.15.29:58530) 2024-04-25T00:36:03.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb5ff6640 1 -- 172.21.15.29:0/3039979758 learned_addr learned my addr 172.21.15.29:0/3039979758 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb57f5640 1 --2- 172.21.15.29:0/3039979758 >> v2:172.21.15.157:3300/0 conn(0x7f6fb00b4780 0x7f6fb015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1899734974 0 0) 0x7f6fb00b3720 con 0x7f6fb015e8d0 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f9c003530 con 0x7f6fb015e8d0 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4119802749 0 0) 0x7f6f9c003530 con 0x7f6fb015e8d0 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 >> v2:172.21.15.157:3300/0 conn(0x7f6fb00b4780 msgr2=0x7f6fb015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 --2- 172.21.15.29:0/3039979758 >> v2:172.21.15.157:3300/0 conn(0x7f6fb00b4780 0x7f6fb015f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 >> v1:172.21.15.157:6789/0 conn(0x7f6fb00cc240 legacy=0x7f6fb015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb00b3ca0 con 0x7f6fb015e8d0 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f6fac001a00 con 0x7f6fb015e8d0 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 >> v1:172.21.15.29:6789/0 conn(0x7f6fb015e8d0 legacy=0x7f6fb015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 --2- 172.21.15.29:0/3039979758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f9c0039c0 0x7f6f9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 --2- 172.21.15.29:0/3039979758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f9c004810 0x7f6f9c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 --2- 172.21.15.29:0/3039979758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0051d0 0x7f6f9c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:03.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c00c040 con 0x7f6f9c0051d0 2024-04-25T00:36:03.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c00c1f0 con 0x7f6f9c004810 2024-04-25T00:36:03.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f9c00c3f0 con 0x7f6f9c0039c0 2024-04-25T00:36:03.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb67f7640 1 --2- 172.21.15.29:0/3039979758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0051d0 0x7f6f9c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:03.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb5ff6640 1 --2- 172.21.15.29:0/3039979758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f9c0039c0 0x7f6f9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:03.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb57f5640 1 --2- 172.21.15.29:0/3039979758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f9c004810 0x7f6f9c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:03.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb67f7640 1 -- 172.21.15.29:0/3039979758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f9c0039c0 msgr2=0x7f6f9c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:03.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb67f7640 1 --2- 172.21.15.29:0/3039979758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f9c0039c0 0x7f6f9c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:03.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb67f7640 1 -- 172.21.15.29:0/3039979758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f9c004810 msgr2=0x7f6f9c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:03.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb67f7640 1 --2- 172.21.15.29:0/3039979758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f9c004810 0x7f6f9c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:03.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb67f7640 1 -- 172.21.15.29:0/3039979758 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9c003530 con 0x7f6f9c0051d0 2024-04-25T00:36:03.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.260+0000 7f6fb57f5640 1 --2- 172.21.15.29:0/3039979758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f9c004810 0x7f6f9c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:03.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb67f7640 1 -- 172.21.15.29:0/3039979758 reap_dead start 2024-04-25T00:36:03.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb67f7640 1 --2- 172.21.15.29:0/3039979758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0051d0 0x7f6f9c00b9d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8001780 tx=0x7f6fa801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb8ce7793148edc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:03.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa80013a0 con 0x7f6f9c0051d0 2024-04-25T00:36:03.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa80a9070 con 0x7f6f9c0051d0 2024-04-25T00:36:03.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb4ff4640 1 -- 172.21.15.29:0/3039979758 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8003600 con 0x7f6f9c0051d0 2024-04-25T00:36:03.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 -- 172.21.15.29:0/3039979758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0051d0 msgr2=0x7f6f9c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:03.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 --2- 172.21.15.29:0/3039979758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0051d0 0x7f6f9c00b9d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8001780 tx=0x7f6fa801c420 comp rx=0 tx=0).stop 2024-04-25T00:36:03.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 -- 172.21.15.29:0/3039979758 shutdown_connections 2024-04-25T00:36:03.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 --2- 172.21.15.29:0/3039979758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0051d0 0x7f6f9c00b9d0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:03.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 -- 172.21.15.29:0/3039979758 >> 172.21.15.29:0/3039979758 conn(0x7f6fb00b1870 msgr2=0x7f6fb00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:03.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 -- 172.21.15.29:0/3039979758 shutdown_connections 2024-04-25T00:36:03.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 -- 172.21.15.29:0/3039979758 wait complete. 2024-04-25T00:36:03.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 Processor -- start 2024-04-25T00:36:03.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 -- start start 2024-04-25T00:36:03.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0051d0 0x7f6fb0168c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:03.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fb00af080 0x7f6fb0153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:03.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fb00cefd0 0x7f6fb00ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:03.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb00b42f0 con 0x7f6f9c0051d0 2024-04-25T00:36:03.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb00b3580 con 0x7f6fb00cefd0 2024-04-25T00:36:03.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb825a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb006cf10 con 0x7f6fb00af080 2024-04-25T00:36:03.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb5ff6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0051d0 0x7f6fb0168c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:03.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb5ff6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0051d0 0x7f6fb0168c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42200/0 (socket says 172.21.15.29:42200) 2024-04-25T00:36:03.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.264+0000 7f6fb5ff6640 1 -- 172.21.15.29:0/474018120 learned_addr learned my addr 172.21.15.29:0/474018120 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:03.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fb67f7640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fb00cefd0 0x7f6fb00ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:03.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fb57f5640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fb00af080 0x7f6fb0153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:03.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fb5ff6640 1 -- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fb00af080 msgr2=0x7f6fb0153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:03.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fb5ff6640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fb00af080 0x7f6fb0153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:03.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fb5ff6640 1 -- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fb00cefd0 msgr2=0x7f6fb00ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:03.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fb5ff6640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fb00cefd0 0x7f6fb00ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:03.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fb5ff6640 1 -- 172.21.15.29:0/474018120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb0133d10 con 0x7f6f9c0051d0 2024-04-25T00:36:03.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fb57f5640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fb00af080 0x7f6fb0153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:03.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fb5ff6640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0051d0 0x7f6fb0168c90 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f6fac007aa0 tx=0x7f6fac00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:03.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fa67fc640 1 -- 172.21.15.29:0/474018120 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00ee50 con 0x7f6f9c0051d0 2024-04-25T00:36:03.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fa67fc640 1 -- 172.21.15.29:0/474018120 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fac013420 con 0x7f6f9c0051d0 2024-04-25T00:36:03.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fb825a640 1 -- 172.21.15.29:0/474018120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb0134570 con 0x7f6f9c0051d0 2024-04-25T00:36:03.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fa67fc640 1 -- 172.21.15.29:0/474018120 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac013610 con 0x7f6f9c0051d0 2024-04-25T00:36:03.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fb825a640 1 -- 172.21.15.29:0/474018120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb0134290 con 0x7f6f9c0051d0 2024-04-25T00:36:03.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fb825a640 1 -- 172.21.15.29:0/474018120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb0069270 con 0x7f6f9c0051d0 2024-04-25T00:36:03.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fa67fc640 1 -- 172.21.15.29:0/474018120 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f6fac01a020 con 0x7f6f9c0051d0 2024-04-25T00:36:03.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fa67fc640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f94067c20 0x7f6f9406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:03.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fa67fc640 1 -- 172.21.15.29:0/474018120 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f6fac085f20 con 0x7f6f9c0051d0 2024-04-25T00:36:03.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.268+0000 7f6fb57f5640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f94067c20 0x7f6f9406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:03.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.272+0000 7f6fb57f5640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f94067c20 0x7f6f9406a0e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6fb015e750 tx=0x7f6fa0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:03.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.272+0000 7f6fa67fc640 1 -- 172.21.15.29:0/474018120 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6fac021020 con 0x7f6f9c0051d0 2024-04-25T00:36:03.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.472+0000 7f6fb825a640 1 -- 172.21.15.29:0/474018120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6fb0138f10 con 0x7f6f9c0051d0 2024-04-25T00:36:03.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.472+0000 7f6fa67fc640 1 -- 172.21.15.29:0/474018120 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) ==== 53+0+73 (secure 0 0 0) 0x7f6fac056650 con 0x7f6f9c0051d0 2024-04-25T00:36:03.476 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:36:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.476+0000 7f6fb825a640 1 -- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f94067c20 msgr2=0x7f6f9406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.476+0000 7f6fb825a640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f94067c20 0x7f6f9406a0e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6fb015e750 tx=0x7f6fa0008040 comp rx=0 tx=0).stop 2024-04-25T00:36:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.476+0000 7f6fb825a640 1 -- 172.21.15.29:0/474018120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0051d0 msgr2=0x7f6fb0168c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.476+0000 7f6fb825a640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0051d0 0x7f6fb0168c90 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f6fac007aa0 tx=0x7f6fac00e530 comp rx=0 tx=0).stop 2024-04-25T00:36:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.476+0000 7f6fb825a640 1 -- 172.21.15.29:0/474018120 shutdown_connections 2024-04-25T00:36:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.476+0000 7f6fb825a640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f6f94067c20 0x7f6f9406a0e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.476+0000 7f6fb825a640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fb00cefd0 0x7f6fb00ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.476+0000 7f6fb825a640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fb00af080 0x7f6fb0153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.476+0000 7f6fb825a640 1 --2- 172.21.15.29:0/474018120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0051d0 0x7f6fb0168c90 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.476+0000 7f6fb825a640 1 -- 172.21.15.29:0/474018120 >> 172.21.15.29:0/474018120 conn(0x7f6fb00b1870 msgr2=0x7f6fb00b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.476+0000 7f6fb825a640 1 -- 172.21.15.29:0/474018120 shutdown_connections 2024-04-25T00:36:03.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:03.476+0000 7f6fb825a640 1 -- 172.21.15.29:0/474018120 wait complete. 2024-04-25T00:36:03.495 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:36:03.495 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:36:03.495 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-25T00:36:03.495 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:36:03.495 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:03.495 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:36:03.495 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:36:03.495 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:36:03.495 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:36:03.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:03.496 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-25T00:36:03.503 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:36:03.503 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:36:03.503 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:36:03.503 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:03.503 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:36:03.503 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:03.503 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:36:03.503 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:03.503 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-25T00:36:03.503 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42192sec preferred_lft 42192sec 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:9c:8d:4b:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:36:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:36:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:03.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:03.506 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:36:03.559 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:36:03.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:03.561 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-25T00:36:03.612 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-25T00:36:03.613 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:36:03.613 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:36:03.613 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:36:03.613 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:36:03.613 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:36:03.613 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:36:03.613 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:36:03.613 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:36:03.613 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:36:03.613 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:36:03.614 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:36:03.614 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:36:03.614 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:36:03.614 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:36:03.614 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:36:03.614 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:36:03.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:36:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:36:03.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:36:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:36:03.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:03.743 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:36:03.748 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:36:03.748 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:36:03.748 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:36:03.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:03.748 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-25T00:36:03.802 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T00:36:03.803 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:36:03.803 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:03.847 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:36:03.848 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:36:03.848 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:03.896 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:36:03.953 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:36:04.001 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:36:04.001 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:36:04.001 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:36:04.001 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:36:04.001 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:36:04.001 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:36:04.001 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:36:04.001 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:36:04.002 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:36:04.002 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:36:04.002 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:36:04.002 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:36:04.002 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:36:04.002 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:36:04.002 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:36:04.002 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:36:04.002 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:36:04.002 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:36:04.002 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:36:04.002 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:36:04.002 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:36:04.003 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:36:04.003 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:36:04.003 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:36:04.003 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:36:04.003 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:36:04.003 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:36:04.003 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:36:04.003 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:36:04.003 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:04.003 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:04.003 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:04.003 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:04.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:04.004 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:36:04.052 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:36:04.052 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T00:36:04.097 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:36:04.112 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:36:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:36:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:36:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:36:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:36:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:36:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:36:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:36:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:36:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:36:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:36:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:36:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:36:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:36:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:36:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:36:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:36:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:36:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:36:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:36:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:36:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:36:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:36:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:36:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:36:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:36:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:36:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:36:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:36:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:36:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:04.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:04.164 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:36:04.174 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-04-25T00:36:04.172+0000 7f875b4d7580 -1 init, newargv = 0x562322f8f520 newargc=15 2024-04-25T00:36:04.174 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34232]: starting ceph client 2024-04-25T00:36:04.184 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34232]: starting fuse 2024-04-25T00:36:05.281 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:36:05.297 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:36:05.349 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:36:05.349 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:36:05.349 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:36:05.350 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:36:05.350 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:36:05.350 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:36:05.350 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:36:05.350 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:36:05.350 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:36:05.350 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:36:05.351 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:36:05.351 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:36:05.351 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:36:05.351 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:36:05.351 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:36:05.352 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:36:05.352 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:36:05.352 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:36:05.352 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:36:05.352 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:36:05.352 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:36:05.352 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:36:05.352 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:36:05.352 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:36:05.353 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:36:05.353 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:36:05.353 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:36:05.353 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:36:05.353 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:36:05.353 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:05.353 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:05.353 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:05.353 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:05.353 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:05.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:05.354 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:36:05.397 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:36:05.397 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:36:05.398 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:36:05.398 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-25T00:36:05.398 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-25T00:36:05.398 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-25T00:36:05.398 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-25T00:36:05.398 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:36:05.398 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-25T00:36:05.398 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T00:36:05.398 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-25T00:36:05.398 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T00:36:05.398 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:36:05.398 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-25T00:36:05.398 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-04-25T00:36:05.399 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-25T00:36:05.520 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.34232.asok 2024-04-25T00:36:05.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:36:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:36:05.533 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34232.asok 2024-04-25T00:36:05.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:05.534 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34232.asok status 2024-04-25T00:36:05.694 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:36:05.694 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-25T00:36:05.694 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:36:05.694 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:36:05.694 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-04-25T00:36:05.695 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-25T00:36:05.695 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T00:36:05.695 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "34232", 2024-04-25T00:36:05.695 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-25T00:36:05.695 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:36:05.695 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-25T00:36:05.695 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-25T00:36:05.695 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 5738, 2024-04-25T00:36:05.695 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-25T00:36:05.695 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-25T00:36:05.696 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-25T00:36:05.696 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 5738 2024-04-25T00:36:05.696 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:36:05.696 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:36:05.696 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:36:05.696 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:36:05.696 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 664361815 2024-04-25T00:36:05.696 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-25T00:36:05.696 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:36:05.696 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:36:05.696 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:36:05.697 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:36:05.697 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 664361815 2024-04-25T00:36:05.697 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:36:05.697 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.5738 192.168.0.1:0/664361815", 2024-04-25T00:36:05.697 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/664361815", 2024-04-25T00:36:05.697 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-25T00:36:05.697 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 43, 2024-04-25T00:36:05.697 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 70, 2024-04-25T00:36:05.697 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:36:05.697 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-25T00:36:05.697 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-25T00:36:05.697 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:36:05.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:05.708 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-25T00:36:05.708 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:05.714 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-25T00:36:05.715 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:05.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:05.715 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:05.770 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-25T00:36:05.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.908+0000 7fc4f5225640 1 Processor -- start 2024-04-25T00:36:05.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.908+0000 7fc4f5225640 1 -- start start 2024-04-25T00:36:05.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.908+0000 7fc4f5225640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f00b8bf0 0x7fc4f00d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:05.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.908+0000 7fc4f5225640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f00cfed0 con 0x7fc4f015e560 2024-04-25T00:36:05.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.908+0000 7fc4f5225640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f00b7100 con 0x7fc4f015e180 2024-04-25T00:36:05.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.908+0000 7fc4f5225640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f00b73e0 con 0x7fc4f00b8bf0 2024-04-25T00:36:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4ee59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc4f015e560 0x7fc4f00d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58542/0 (socket says 172.21.15.29:58542) 2024-04-25T00:36:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4ee59c640 1 -- 172.21.15.29:0/4061205596 learned_addr learned my addr 172.21.15.29:0/4061205596 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4ef59e640 1 --2- 172.21.15.29:0/4061205596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f00b8bf0 0x7fc4f00d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3359267300 0 0) 0x7fc4f00cfed0 con 0x7fc4f015e560 2024-04-25T00:36:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4d8003530 con 0x7fc4f015e560 2024-04-25T00:36:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1143233135 0 0) 0x7fc4d8003530 con 0x7fc4f015e560 2024-04-25T00:36:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f00b8bf0 msgr2=0x7fc4f00d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 --2- 172.21.15.29:0/4061205596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f00b8bf0 0x7fc4f00d20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 >> v1:172.21.15.157:6789/0 conn(0x7fc4f015e180 legacy=0x7fc4f00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f00b76c0 con 0x7fc4f015e560 2024-04-25T00:36:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fc4e4001680 con 0x7fc4f015e560 2024-04-25T00:36:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 >> v1:172.21.15.29:6789/0 conn(0x7fc4f015e560 legacy=0x7fc4f00d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 --2- 172.21.15.29:0/4061205596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f00b8bf0 0x7fc4d80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 --2- 172.21.15.29:0/4061205596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4d8003f70 0x7fc4d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 --2- 172.21.15.29:0/4061205596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d8004d50 0x7fc4d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f00b73e0 con 0x7fc4d8003f70 2024-04-25T00:36:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d800bb40 con 0x7fc4d8004d50 2024-04-25T00:36:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d800bd40 con 0x7fc4f00b8bf0 2024-04-25T00:36:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4ee59c640 1 --2- 172.21.15.29:0/4061205596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4d8003f70 0x7fc4d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4eed9d640 1 --2- 172.21.15.29:0/4061205596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f00b8bf0 0x7fc4d80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4ef59e640 1 --2- 172.21.15.29:0/4061205596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d8004d50 0x7fc4d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4ee59c640 1 -- 172.21.15.29:0/4061205596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f00b8bf0 msgr2=0x7fc4d80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4ee59c640 1 --2- 172.21.15.29:0/4061205596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f00b8bf0 0x7fc4d80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4ee59c640 1 -- 172.21.15.29:0/4061205596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d8004d50 msgr2=0x7fc4d80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4ee59c640 1 --2- 172.21.15.29:0/4061205596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d8004d50 0x7fc4d80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4ee59c640 1 -- 172.21.15.29:0/4061205596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d8003530 con 0x7fc4d8003f70 2024-04-25T00:36:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4ee59c640 1 --2- 172.21.15.29:0/4061205596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4d8003f70 0x7fc4d8004810 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c210 tx=0x7fc4e401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10500a79aca9946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4007d40 con 0x7fc4d8003f70 2024-04-25T00:36:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e40a9070 con 0x7fc4d8003f70 2024-04-25T00:36:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.912+0000 7fc4edd9b640 1 -- 172.21.15.29:0/4061205596 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40a5e40 con 0x7fc4d8003f70 2024-04-25T00:36:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 -- 172.21.15.29:0/4061205596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4d8003f70 msgr2=0x7fc4d8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 --2- 172.21.15.29:0/4061205596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4d8003f70 0x7fc4d8004810 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c210 tx=0x7fc4e401cca0 comp rx=0 tx=0).stop 2024-04-25T00:36:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4ef59e640 1 -- 172.21.15.29:0/4061205596 reap_dead start 2024-04-25T00:36:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 -- 172.21.15.29:0/4061205596 shutdown_connections 2024-04-25T00:36:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 -- 172.21.15.29:0/4061205596 >> 172.21.15.29:0/4061205596 conn(0x7fc4f00d40a0 msgr2=0x7fc4f00d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 -- 172.21.15.29:0/4061205596 shutdown_connections 2024-04-25T00:36:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 -- 172.21.15.29:0/4061205596 wait complete. 2024-04-25T00:36:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 Processor -- start 2024-04-25T00:36:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 -- start start 2024-04-25T00:36:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4d8003f70 0x7fc4f0160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f01f7390 0x7fc4f01f7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f0146030 0x7fc4f01464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f00d20e0 con 0x7fc4d8003f70 2024-04-25T00:36:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f00d2fa0 con 0x7fc4f01f7390 2024-04-25T00:36:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f00d2260 con 0x7fc4f0146030 2024-04-25T00:36:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4eed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4d8003f70 0x7fc4f0160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4eed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4d8003f70 0x7fc4f0160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42224/0 (socket says 172.21.15.29:42224) 2024-04-25T00:36:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4eed9d640 1 -- 172.21.15.29:0/3385754268 learned_addr learned my addr 172.21.15.29:0/3385754268 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4ee59c640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f01f7390 0x7fc4f01f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4ef59e640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f0146030 0x7fc4f01464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4eed9d640 1 -- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f0146030 msgr2=0x7fc4f01464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4eed9d640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f0146030 0x7fc4f01464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4eed9d640 1 -- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f01f7390 msgr2=0x7fc4f01f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4eed9d640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f01f7390 0x7fc4f01f7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4eed9d640 1 -- 172.21.15.29:0/3385754268 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f0160f60 con 0x7fc4d8003f70 2024-04-25T00:36:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4ee59c640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f01f7390 0x7fc4f01f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4ef59e640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f0146030 0x7fc4f01464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4eed9d640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4d8003f70 0x7fc4f0160680 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000e580 tx=0x7fc4e000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4d77fe640 1 -- 172.21.15.29:0/3385754268 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00052e0 con 0x7fc4d8003f70 2024-04-25T00:36:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4d77fe640 1 -- 172.21.15.29:0/3385754268 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e0018070 con 0x7fc4d8003f70 2024-04-25T00:36:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 -- 172.21.15.29:0/3385754268 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f0161240 con 0x7fc4d8003f70 2024-04-25T00:36:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4d77fe640 1 -- 172.21.15.29:0/3385754268 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0003820 con 0x7fc4d8003f70 2024-04-25T00:36:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.916+0000 7fc4f5225640 1 -- 172.21.15.29:0/3385754268 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f0161800 con 0x7fc4d8003f70 2024-04-25T00:36:05.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.920+0000 7fc4f5225640 1 -- 172.21.15.29:0/3385754268 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f00692e0 con 0x7fc4d8003f70 2024-04-25T00:36:05.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.920+0000 7fc4d77fe640 1 -- 172.21.15.29:0/3385754268 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fc4e00039c0 con 0x7fc4d8003f70 2024-04-25T00:36:05.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.920+0000 7fc4d77fe640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc4dc067c20 0x7fc4dc06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:05.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.920+0000 7fc4d77fe640 1 -- 172.21.15.29:0/3385754268 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fc4e00869c0 con 0x7fc4d8003f70 2024-04-25T00:36:05.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.920+0000 7fc4ee59c640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc4dc067c20 0x7fc4dc06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:05.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.920+0000 7fc4ee59c640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc4dc067c20 0x7fc4dc06a0e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40ab000 tx=0x7fc4e40b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:05.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:05.924+0000 7fc4d77fe640 1 -- 172.21.15.29:0/3385754268 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc4e0057180 con 0x7fc4d8003f70 2024-04-25T00:36:06.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.124+0000 7fc4f5225640 1 -- 172.21.15.29:0/3385754268 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fc4f01585b0 con 0x7fc4d8003f70 2024-04-25T00:36:06.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.620+0000 7fc4d77fe640 1 -- 172.21.15.29:0/3385754268 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v44) ==== 136+0+0 (secure 0 0 0) 0x7fc4e0005480 con 0x7fc4d8003f70 2024-04-25T00:36:06.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.624+0000 7fc4f5225640 1 -- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc4dc067c20 msgr2=0x7fc4dc06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:06.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.624+0000 7fc4f5225640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc4dc067c20 0x7fc4dc06a0e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40ab000 tx=0x7fc4e40b1040 comp rx=0 tx=0).stop 2024-04-25T00:36:06.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.624+0000 7fc4f5225640 1 -- 172.21.15.29:0/3385754268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4d8003f70 msgr2=0x7fc4f0160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:06.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.624+0000 7fc4f5225640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4d8003f70 0x7fc4f0160680 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000e580 tx=0x7fc4e000ea40 comp rx=0 tx=0).stop 2024-04-25T00:36:06.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.624+0000 7fc4f5225640 1 -- 172.21.15.29:0/3385754268 shutdown_connections 2024-04-25T00:36:06.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.624+0000 7fc4f5225640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4f0146030 0x7fc4f01464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:06.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.624+0000 7fc4f5225640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fc4dc067c20 0x7fc4dc06a0e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:06.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.624+0000 7fc4f5225640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f01f7390 0x7fc4f01f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:06.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.624+0000 7fc4f5225640 1 --2- 172.21.15.29:0/3385754268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4d8003f70 0x7fc4f0160680 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:06.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.624+0000 7fc4f5225640 1 -- 172.21.15.29:0/3385754268 >> 172.21.15.29:0/3385754268 conn(0x7fc4f00d40a0 msgr2=0x7fc4f00d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:06.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.624+0000 7fc4f5225640 1 -- 172.21.15.29:0/3385754268 shutdown_connections 2024-04-25T00:36:06.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.624+0000 7fc4f5225640 1 -- 172.21.15.29:0/3385754268 wait complete. 2024-04-25T00:36:06.648 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-25T00:36:06.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.776+0000 7f3a985bf640 1 Processor -- start 2024-04-25T00:36:06.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.776+0000 7f3a985bf640 1 -- start start 2024-04-25T00:36:06.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.776+0000 7f3a985bf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3a900b4780 0x7f3a9015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:06.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.776+0000 7f3a985bf640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a900b3720 con 0x7f3a900cc240 2024-04-25T00:36:06.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.776+0000 7f3a985bf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a900b39c0 con 0x7f3a9015e8d0 2024-04-25T00:36:06.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.776+0000 7f3a985bf640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f3a900cc650 con 0x7f3a900b4780 2024-04-25T00:36:06.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.776+0000 7f3a95b5a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3a900b4780 0x7f3a9015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:06.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.776+0000 7f3a96b5c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3a900cc240 0x7f3a9015c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58556/0 (socket says 172.21.15.29:58556) 2024-04-25T00:36:06.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.776+0000 7f3a96b5c640 1 -- 172.21.15.29:0/3555446530 learned_addr learned my addr 172.21.15.29:0/3555446530 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:06.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.776+0000 7f3a95b5a640 1 -- 172.21.15.29:0/3555446530 >> v1:172.21.15.157:6789/0 conn(0x7f3a9015e8d0 legacy=0x7f3a9015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:06.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.776+0000 7f3a95b5a640 1 -- 172.21.15.29:0/3555446530 >> v1:172.21.15.29:6789/0 conn(0x7f3a900cc240 legacy=0x7f3a9015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:06.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.776+0000 7f3a95b5a640 1 -- 172.21.15.29:0/3555446530 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a900b3ca0 con 0x7f3a900b4780 2024-04-25T00:36:06.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.776+0000 7f3a95359640 1 -- 172.21.15.29:0/3555446530 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3906635933 0 0) 0x7f3a900b3720 con 0x7f3a900cc240 2024-04-25T00:36:06.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.780+0000 7f3a95b5a640 1 --2- 172.21.15.29:0/3555446530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a900b4780 0x7f3a9015f3c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00ba60 tx=0x7f3a8c01eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c8a8505c2668f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:06.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.780+0000 7f3a95359640 1 -- 172.21.15.29:0/3555446530 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0b4070 con 0x7f3a900b4780 2024-04-25T00:36:06.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.780+0000 7f3a95359640 1 -- 172.21.15.29:0/3555446530 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a8c0a7ad0 con 0x7f3a900b4780 2024-04-25T00:36:06.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.780+0000 7f3a95359640 1 -- 172.21.15.29:0/3555446530 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0af440 con 0x7f3a900b4780 2024-04-25T00:36:06.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.780+0000 7f3a985bf640 1 -- 172.21.15.29:0/3555446530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a900b4780 msgr2=0x7f3a9015f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:06.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.780+0000 7f3a985bf640 1 --2- 172.21.15.29:0/3555446530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a900b4780 0x7f3a9015f3c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00ba60 tx=0x7f3a8c01eb30 comp rx=0 tx=0).stop 2024-04-25T00:36:06.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.780+0000 7f3a985bf640 1 -- 172.21.15.29:0/3555446530 shutdown_connections 2024-04-25T00:36:06.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.780+0000 7f3a985bf640 1 --2- 172.21.15.29:0/3555446530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a900b4780 0x7f3a9015f3c0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:06.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.780+0000 7f3a985bf640 1 -- 172.21.15.29:0/3555446530 >> 172.21.15.29:0/3555446530 conn(0x7f3a900b1870 msgr2=0x7f3a900b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:06.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.780+0000 7f3a985bf640 1 -- 172.21.15.29:0/3555446530 shutdown_connections 2024-04-25T00:36:06.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.780+0000 7f3a985bf640 1 -- 172.21.15.29:0/3555446530 wait complete. 2024-04-25T00:36:06.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a985bf640 1 Processor -- start 2024-04-25T00:36:06.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a985bf640 1 -- start start 2024-04-25T00:36:06.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a985bf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a900b4780 0x7f3a9014b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:06.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a985bf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a900cc240 0x7f3a9013eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:06.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a985bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9015e8d0 0x7f3a9013b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a985bf640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9006d750 con 0x7f3a900cc240 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a985bf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9006d5d0 con 0x7f3a9015e8d0 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a985bf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a9006d8d0 con 0x7f3a900b4780 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a95b5a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a900cc240 0x7f3a9013eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a95b5a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a900cc240 0x7f3a9013eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42244/0 (socket says 172.21.15.29:42244) 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a95b5a640 1 -- 172.21.15.29:0/4201512083 learned_addr learned my addr 172.21.15.29:0/4201512083 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a9635b640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a900b4780 0x7f3a9014b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a95b5a640 1 -- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a900b4780 msgr2=0x7f3a9014b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a96b5c640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9015e8d0 0x7f3a9013b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a95b5a640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a900b4780 0x7f3a9014b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a95b5a640 1 -- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9015e8d0 msgr2=0x7f3a9013b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a95b5a640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9015e8d0 0x7f3a9013b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a95b5a640 1 -- 172.21.15.29:0/4201512083 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a90139fa0 con 0x7f3a900cc240 2024-04-25T00:36:06.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a9635b640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a900b4780 0x7f3a9014b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:06.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a95b5a640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a900cc240 0x7f3a9013eb50 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0b2040 tx=0x7f3a8c01d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:06.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a86ffd640 1 -- 172.21.15.29:0/4201512083 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0bf070 con 0x7f3a900cc240 2024-04-25T00:36:06.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a86ffd640 1 -- 172.21.15.29:0/4201512083 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a8c0afca0 con 0x7f3a900cc240 2024-04-25T00:36:06.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a985bf640 1 -- 172.21.15.29:0/4201512083 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9013a280 con 0x7f3a900cc240 2024-04-25T00:36:06.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a86ffd640 1 -- 172.21.15.29:0/4201512083 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0b4040 con 0x7f3a900cc240 2024-04-25T00:36:06.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a985bf640 1 -- 172.21.15.29:0/4201512083 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a9013a840 con 0x7f3a900cc240 2024-04-25T00:36:06.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.784+0000 7f3a985bf640 1 -- 172.21.15.29:0/4201512083 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a90069270 con 0x7f3a900cc240 2024-04-25T00:36:06.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.788+0000 7f3a86ffd640 1 -- 172.21.15.29:0/4201512083 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f3a8c0c2070 con 0x7f3a900cc240 2024-04-25T00:36:06.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.788+0000 7f3a86ffd640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a74067c20 0x7f3a7406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:06.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.788+0000 7f3a86ffd640 1 -- 172.21.15.29:0/4201512083 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f3a8c120a00 con 0x7f3a900cc240 2024-04-25T00:36:06.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.788+0000 7f3a9635b640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a74067c20 0x7f3a7406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:06.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.788+0000 7f3a9635b640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a74067c20 0x7f3a7406a0e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000a850 tx=0x7f3a80008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:06.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.792+0000 7f3a86ffd640 1 -- 172.21.15.29:0/4201512083 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3a8c018720 con 0x7f3a900cc240 2024-04-25T00:36:06.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.972+0000 7f3a985bf640 1 -- 172.21.15.29:0/4201512083 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f3a9015c950 con 0x7f3a74067c20 2024-04-25T00:36:06.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.976+0000 7f3a86ffd640 1 -- 172.21.15.29:0/4201512083 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f3a9015c950 con 0x7f3a74067c20 2024-04-25T00:36:06.979 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:36:06.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:06.980 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-04-25T00:36:06.980 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:06.980 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:36:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.980+0000 7f3a985bf640 1 -- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a74067c20 msgr2=0x7f3a7406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.980+0000 7f3a985bf640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a74067c20 0x7f3a7406a0e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000a850 tx=0x7f3a80008040 comp rx=0 tx=0).stop 2024-04-25T00:36:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.980+0000 7f3a985bf640 1 -- 172.21.15.29:0/4201512083 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a900cc240 msgr2=0x7f3a9013eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.980+0000 7f3a985bf640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a900cc240 0x7f3a9013eb50 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0b2040 tx=0x7f3a8c01d370 comp rx=0 tx=0).stop 2024-04-25T00:36:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.980+0000 7f3a985bf640 1 -- 172.21.15.29:0/4201512083 shutdown_connections 2024-04-25T00:36:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.980+0000 7f3a985bf640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3a74067c20 0x7f3a7406a0e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.980+0000 7f3a985bf640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9015e8d0 0x7f3a9013b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.980+0000 7f3a985bf640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a900cc240 0x7f3a9013eb50 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.980+0000 7f3a985bf640 1 --2- 172.21.15.29:0/4201512083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a900b4780 0x7f3a9014b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.980+0000 7f3a985bf640 1 -- 172.21.15.29:0/4201512083 >> 172.21.15.29:0/4201512083 conn(0x7f3a900b1870 msgr2=0x7f3a9006dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:06.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.980+0000 7f3a985bf640 1 -- 172.21.15.29:0/4201512083 shutdown_connections 2024-04-25T00:36:06.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:06.980+0000 7f3a985bf640 1 -- 172.21.15.29:0/4201512083 wait complete. 2024-04-25T00:36:06.999 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:36:07.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb65ab640 1 Processor -- start 2024-04-25T00:36:07.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb65ab640 1 -- start start 2024-04-25T00:36:07.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb65ab640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb00b2b70 0x7f4fb015d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb65ab640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4fb015c5c0 0x7f4fb015ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb65ab640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4fb0150df0 0x7f4fb0151280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb65ab640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4fb00cf560 con 0x7f4fb0150df0 2024-04-25T00:36:07.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb65ab640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4fb00b3130 con 0x7f4fb015c5c0 2024-04-25T00:36:07.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb65ab640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb00b3410 con 0x7f4fb00b2b70 2024-04-25T00:36:07.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb4b48640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4fb0150df0 0x7f4fb0151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb4b48640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4fb0150df0 0x7f4fb0151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42258/0 (socket says 172.21.15.29:42258) 2024-04-25T00:36:07.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb4b48640 1 -- 172.21.15.29:0/3526433283 learned_addr learned my addr 172.21.15.29:0/3526433283 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:07.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4faffff640 1 --2- 172.21.15.29:0/3526433283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb00b2b70 0x7f4fb015d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4faf7fe640 1 --2- 172.21.15.29:0/3526433283 >> v2:172.21.15.157:3300/0 conn(0x7f4fb015c5c0 0x7f4fb015ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb4b48640 1 -- 172.21.15.29:0/3526433283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb00b2b70 msgr2=0x7f4fb015d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:07.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb4b48640 1 --2- 172.21.15.29:0/3526433283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb00b2b70 0x7f4fb015d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb4b48640 1 -- 172.21.15.29:0/3526433283 >> v2:172.21.15.157:3300/0 conn(0x7f4fb015c5c0 msgr2=0x7f4fb015ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:07.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb4b48640 1 --2- 172.21.15.29:0/3526433283 >> v2:172.21.15.157:3300/0 conn(0x7f4fb015c5c0 0x7f4fb015ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.120+0000 7f4fb4b48640 1 -- 172.21.15.29:0/3526433283 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb00b36f0 con 0x7f4fb0150df0 2024-04-25T00:36:07.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb4b48640 1 --2- 172.21.15.29:0/3526433283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fb0150df0 0x7f4fb0151280 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0074fb0 tx=0x7f4fa801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65904f32343f2ae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:07.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4faeffd640 1 -- 172.21.15.29:0/3526433283 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa80a5760 con 0x7f4fb0150df0 2024-04-25T00:36:07.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4faeffd640 1 -- 172.21.15.29:0/3526433283 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa80a5900 con 0x7f4fb0150df0 2024-04-25T00:36:07.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4faeffd640 1 -- 172.21.15.29:0/3526433283 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa80a5cd0 con 0x7f4fb0150df0 2024-04-25T00:36:07.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3526433283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fb0150df0 msgr2=0x7f4fb0151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 --2- 172.21.15.29:0/3526433283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fb0150df0 0x7f4fb0151280 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0074fb0 tx=0x7f4fa801c9f0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3526433283 shutdown_connections 2024-04-25T00:36:07.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 --2- 172.21.15.29:0/3526433283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fb0150df0 0x7f4fb0151280 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 --2- 172.21.15.29:0/3526433283 >> v2:172.21.15.157:3300/0 conn(0x7f4fb015c5c0 0x7f4fb015ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 --2- 172.21.15.29:0/3526433283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb00b2b70 0x7f4fb015d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3526433283 >> 172.21.15.29:0/3526433283 conn(0x7f4fb00b1870 msgr2=0x7f4fb00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:07.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3526433283 shutdown_connections 2024-04-25T00:36:07.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3526433283 wait complete. 2024-04-25T00:36:07.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 Processor -- start 2024-04-25T00:36:07.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 -- start start 2024-04-25T00:36:07.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fb00b2b70 0x7f4fb014e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fb0150df0 0x7f4fb013d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb015c5c0 0x7f4fb014c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb015e830 con 0x7f4fb00b2b70 2024-04-25T00:36:07.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb01519d0 con 0x7f4fb0150df0 2024-04-25T00:36:07.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4fb65ab640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb015e9b0 con 0x7f4fb015c5c0 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4faffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fb00b2b70 0x7f4fb014e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.124+0000 7f4faffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fb00b2b70 0x7f4fb014e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42268/0 (socket says 172.21.15.29:42268) 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4faffff640 1 -- 172.21.15.29:0/3578618211 learned_addr learned my addr 172.21.15.29:0/3578618211 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4fb4b48640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb015c5c0 0x7f4fb014c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4faf7fe640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fb0150df0 0x7f4fb013d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4faffff640 1 -- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb015c5c0 msgr2=0x7f4fb014c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4faffff640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb015c5c0 0x7f4fb014c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4faffff640 1 -- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fb0150df0 msgr2=0x7f4fb013d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4faffff640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fb0150df0 0x7f4fb013d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4faffff640 1 -- 172.21.15.29:0/3578618211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb0072d00 con 0x7f4fb00b2b70 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4faffff640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fb00b2b70 0x7f4fb014e510 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000e8b0 tx=0x7f4fa000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4facff9640 1 -- 172.21.15.29:0/3578618211 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0017070 con 0x7f4fb00b2b70 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4facff9640 1 -- 172.21.15.29:0/3578618211 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa000c940 con 0x7f4fb00b2b70 2024-04-25T00:36:07.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4facff9640 1 -- 172.21.15.29:0/3578618211 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000cd10 con 0x7f4fb00b2b70 2024-04-25T00:36:07.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3578618211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb0071e50 con 0x7f4fb00b2b70 2024-04-25T00:36:07.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3578618211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb00723d0 con 0x7f4fb00b2b70 2024-04-25T00:36:07.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3578618211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb0069270 con 0x7f4fb00b2b70 2024-04-25T00:36:07.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4facff9640 1 -- 172.21.15.29:0/3578618211 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f4fa001a070 con 0x7f4fb00b2b70 2024-04-25T00:36:07.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4facff9640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f9c067c50 0x7f4f9c06a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4facff9640 1 -- 172.21.15.29:0/3578618211 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f4fa0086890 con 0x7f4fb00b2b70 2024-04-25T00:36:07.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.128+0000 7f4faf7fe640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f9c067c50 0x7f4f9c06a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.132+0000 7f4faf7fe640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f9c067c50 0x7f4f9c06a110 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4fb015a440 tx=0x7f4fa4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:07.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.132+0000 7f4facff9640 1 -- 172.21.15.29:0/3578618211 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4fa0056a70 con 0x7f4fb00b2b70 2024-04-25T00:36:07.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3578618211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4fb00cf910 con 0x7f4fb00b2b70 2024-04-25T00:36:07.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4facff9640 1 -- 172.21.15.29:0/3578618211 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f4fa005ad70 con 0x7f4fb00b2b70 2024-04-25T00:36:07.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f9c067c50 msgr2=0x7f4f9c06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4fb65ab640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f9c067c50 0x7f4f9c06a110 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4fb015a440 tx=0x7f4fa4008040 comp rx=0 tx=0).stop 2024-04-25T00:36:07.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3578618211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fb00b2b70 msgr2=0x7f4fb014e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4fb65ab640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fb00b2b70 0x7f4fb014e510 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000e8b0 tx=0x7f4fa000ed70 comp rx=0 tx=0).stop 2024-04-25T00:36:07.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3578618211 shutdown_connections 2024-04-25T00:36:07.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4fb65ab640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb015c5c0 0x7f4fb014c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4fb65ab640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4f9c067c50 0x7f4f9c06a110 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4fb65ab640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fb0150df0 0x7f4fb013d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4fb65ab640 1 --2- 172.21.15.29:0/3578618211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fb00b2b70 0x7f4fb014e510 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3578618211 >> 172.21.15.29:0/3578618211 conn(0x7f4fb00b1870 msgr2=0x7f4fb00b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:07.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3578618211 shutdown_connections 2024-04-25T00:36:07.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.296+0000 7f4fb65ab640 1 -- 172.21.15.29:0/3578618211 wait complete. 2024-04-25T00:36:07.318 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:36:07.318 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:36:07.325 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:36:07.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.492+0000 7f3f423e7640 1 Processor -- start 2024-04-25T00:36:07.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.492+0000 7f3f423e7640 1 -- start start 2024-04-25T00:36:07.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f423e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3f3c15e8d0 0x7f3f3c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f423e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f3c15f3c0 0x7f3f3c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f423e7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3c0b3720 con 0x7f3f3c0b4780 2024-04-25T00:36:07.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f423e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3f3c0b42f0 con 0x7f3f3c15e8d0 2024-04-25T00:36:07.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f423e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3c0b39c0 con 0x7f3f3c15f3c0 2024-04-25T00:36:07.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f40984640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f3c15f3c0 0x7f3f3c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f40984640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f3c15f3c0 0x7f3f3c15c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:52194/0 (socket says 172.21.15.29:52194) 2024-04-25T00:36:07.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f40984640 1 -- 172.21.15.29:0/3667386460 learned_addr learned my addr 172.21.15.29:0/3667386460 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:07.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f3bfff640 1 --2- 172.21.15.29:0/3667386460 >> v2:172.21.15.157:3300/0 conn(0x7f3f3c15e8d0 0x7f3f3c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f40984640 1 -- 172.21.15.29:0/3667386460 >> v2:172.21.15.157:3300/0 conn(0x7f3f3c15e8d0 msgr2=0x7f3f3c0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:07.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f3b7fe640 1 -- 172.21.15.29:0/3667386460 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1072416222 0 0) 0x7f3f3c0b3720 con 0x7f3f3c0b4780 2024-04-25T00:36:07.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f40984640 1 --2- 172.21.15.29:0/3667386460 >> v2:172.21.15.157:3300/0 conn(0x7f3f3c15e8d0 0x7f3f3c0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f40984640 1 -- 172.21.15.29:0/3667386460 >> v1:172.21.15.157:6789/0 conn(0x7f3f3c0b4780 legacy=0x7f3f3c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f40984640 1 -- 172.21.15.29:0/3667386460 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c0b3ca0 con 0x7f3f3c15f3c0 2024-04-25T00:36:07.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f40984640 1 --2- 172.21.15.29:0/3667386460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f3c15f3c0 0x7f3f3c15c7d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009e50 tx=0x7f3f3401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1c6947be19bb22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:07.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.496+0000 7f3f3b7fe640 1 -- 172.21.15.29:0/3667386460 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f340a58b0 con 0x7f3f3c15f3c0 2024-04-25T00:36:07.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f3b7fe640 1 -- 172.21.15.29:0/3667386460 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f340a5a50 con 0x7f3f3c15f3c0 2024-04-25T00:36:07.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f3b7fe640 1 -- 172.21.15.29:0/3667386460 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f340a5e20 con 0x7f3f3c15f3c0 2024-04-25T00:36:07.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 -- 172.21.15.29:0/3667386460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f3c15f3c0 msgr2=0x7f3f3c15c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 --2- 172.21.15.29:0/3667386460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f3c15f3c0 0x7f3f3c15c7d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009e50 tx=0x7f3f3401ca00 comp rx=0 tx=0).stop 2024-04-25T00:36:07.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 -- 172.21.15.29:0/3667386460 shutdown_connections 2024-04-25T00:36:07.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 --2- 172.21.15.29:0/3667386460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f3c15f3c0 0x7f3f3c15c7d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 --2- 172.21.15.29:0/3667386460 >> v2:172.21.15.157:3300/0 conn(0x7f3f3c15e8d0 0x7f3f3c0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 -- 172.21.15.29:0/3667386460 >> 172.21.15.29:0/3667386460 conn(0x7f3f3c0b1870 msgr2=0x7f3f3c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 -- 172.21.15.29:0/3667386460 shutdown_connections 2024-04-25T00:36:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 -- 172.21.15.29:0/3667386460 wait complete. 2024-04-25T00:36:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 Processor -- start 2024-04-25T00:36:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 -- start start 2024-04-25T00:36:07.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f3c0b4780 0x7f3f3c157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f3c15e8d0 0x7f3f3c158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f3c15f3c0 0x7f3f3c158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c15d6a0 con 0x7f3f3c0b4780 2024-04-25T00:36:07.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c15d520 con 0x7f3f3c15e8d0 2024-04-25T00:36:07.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3c165c60 con 0x7f3f3c15f3c0 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f3bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f3c0b4780 0x7f3f3c157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f3bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f3c0b4780 0x7f3f3c157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42272/0 (socket says 172.21.15.29:42272) 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f3bfff640 1 -- 172.21.15.29:0/833057239 learned_addr learned my addr 172.21.15.29:0/833057239 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f40984640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f3c15f3c0 0x7f3f3c158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f33fff640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f3c15e8d0 0x7f3f3c158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f3bfff640 1 -- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f3c15f3c0 msgr2=0x7f3f3c158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f3bfff640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f3c15f3c0 0x7f3f3c158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f3bfff640 1 -- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f3c15e8d0 msgr2=0x7f3f3c158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f3bfff640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f3c15e8d0 0x7f3f3c158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f3bfff640 1 -- 172.21.15.29:0/833057239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c158180 con 0x7f3f3c0b4780 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f3bfff640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f3c0b4780 0x7f3f3c157c40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00e930 tx=0x7f3f2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f397fa640 1 -- 172.21.15.29:0/833057239 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c017070 con 0x7f3f3c0b4780 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f397fa640 1 -- 172.21.15.29:0/833057239 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2c00ca00 con 0x7f3f3c0b4780 2024-04-25T00:36:07.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f397fa640 1 -- 172.21.15.29:0/833057239 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c00cdb0 con 0x7f3f3c0b4780 2024-04-25T00:36:07.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 -- 172.21.15.29:0/833057239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3c152b30 con 0x7f3f3c0b4780 2024-04-25T00:36:07.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.500+0000 7f3f423e7640 1 -- 172.21.15.29:0/833057239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3c139b50 con 0x7f3f3c0b4780 2024-04-25T00:36:07.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.504+0000 7f3f423e7640 1 -- 172.21.15.29:0/833057239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f3c069270 con 0x7f3f3c0b4780 2024-04-25T00:36:07.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.504+0000 7f3f397fa640 1 -- 172.21.15.29:0/833057239 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f3f2c01a070 con 0x7f3f3c0b4780 2024-04-25T00:36:07.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.504+0000 7f3f397fa640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3f1c067c20 0x7f3f1c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.504+0000 7f3f397fa640 1 -- 172.21.15.29:0/833057239 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f3f2c086230 con 0x7f3f3c0b4780 2024-04-25T00:36:07.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.504+0000 7f3f33fff640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3f1c067c20 0x7f3f1c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.504+0000 7f3f33fff640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3f1c067c20 0x7f3f1c06a0e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c072d60 tx=0x7f3f24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:07.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.508+0000 7f3f397fa640 1 -- 172.21.15.29:0/833057239 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3f2c056a20 con 0x7f3f3c0b4780 2024-04-25T00:36:07.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.676+0000 7f3f423e7640 1 -- 172.21.15.29:0/833057239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f3c06e7b0 con 0x7f3f3c0b4780 2024-04-25T00:36:07.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.676+0000 7f3f397fa640 1 -- 172.21.15.29:0/833057239 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16620 (secure 0 0 0) 0x7f3f2c056320 con 0x7f3f3c0b4780 2024-04-25T00:36:07.679 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:07.679 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":70,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:51.187302+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:47.559043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:48.530288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:36:07.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.680+0000 7f3f423e7640 1 -- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3f1c067c20 msgr2=0x7f3f1c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.680+0000 7f3f423e7640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3f1c067c20 0x7f3f1c06a0e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c072d60 tx=0x7f3f24009040 comp rx=0 tx=0).stop 2024-04-25T00:36:07.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.680+0000 7f3f423e7640 1 -- 172.21.15.29:0/833057239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f3c0b4780 msgr2=0x7f3f3c157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.680+0000 7f3f423e7640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f3c0b4780 0x7f3f3c157c40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00e930 tx=0x7f3f2c00edf0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.680+0000 7f3f423e7640 1 -- 172.21.15.29:0/833057239 shutdown_connections 2024-04-25T00:36:07.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.680+0000 7f3f423e7640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f3c15f3c0 0x7f3f3c158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.680+0000 7f3f423e7640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f3f1c067c20 0x7f3f1c06a0e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.680+0000 7f3f423e7640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f3c15e8d0 0x7f3f3c158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.680+0000 7f3f423e7640 1 --2- 172.21.15.29:0/833057239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f3c0b4780 0x7f3f3c157c40 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.680+0000 7f3f423e7640 1 -- 172.21.15.29:0/833057239 >> 172.21.15.29:0/833057239 conn(0x7f3f3c0b1870 msgr2=0x7f3f3c13b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:07.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.680+0000 7f3f423e7640 1 -- 172.21.15.29:0/833057239 shutdown_connections 2024-04-25T00:36:07.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.680+0000 7f3f423e7640 1 -- 172.21.15.29:0/833057239 wait complete. 2024-04-25T00:36:07.699 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:35:47.559043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:35:48.530288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:36:07.700 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:36:07.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f444e11c640 1 Processor -- start 2024-04-25T00:36:07.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f444e11c640 1 -- start start 2024-04-25T00:36:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f444e11c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f444815e200 0x7f44480d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f444e11c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44480cf690 0x7f44480d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f444e11c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44480cfc10 con 0x7f44480d2a90 2024-04-25T00:36:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f444e11c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f44480d2ee0 con 0x7f44480cf690 2024-04-25T00:36:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f444e11c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44480b38d0 con 0x7f444815e200 2024-04-25T00:36:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f4447fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f44480d2a90 0x7f44480d05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59780/0 (socket says 172.21.15.29:59780) 2024-04-25T00:36:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f4447fff640 1 -- 172.21.15.29:0/4070239021 learned_addr learned my addr 172.21.15.29:0/4070239021 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f4446ffd640 1 --2- 172.21.15.29:0/4070239021 >> v2:172.21.15.157:3300/0 conn(0x7f44480cf690 0x7f44480d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44477fe640 1 --2- 172.21.15.29:0/4070239021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f444815e200 0x7f44480d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2163084231 0 0) 0x7f44480cfc10 con 0x7f44480d2a90 2024-04-25T00:36:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4430003530 con 0x7f44480d2a90 2024-04-25T00:36:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2910721954 0 0) 0x7f4430003530 con 0x7f44480d2a90 2024-04-25T00:36:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f444815e200 msgr2=0x7f44480d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 --2- 172.21.15.29:0/4070239021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f444815e200 0x7f44480d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 >> v2:172.21.15.157:3300/0 conn(0x7f44480cf690 msgr2=0x7f44480d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 --2- 172.21.15.29:0/4070239021 >> v2:172.21.15.157:3300/0 conn(0x7f44480cf690 0x7f44480d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44480cfef0 con 0x7f44480d2a90 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4440001a00 con 0x7f44480d2a90 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 >> v1:172.21.15.29:6789/0 conn(0x7f44480d2a90 legacy=0x7f44480d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 --2- 172.21.15.29:0/4070239021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44300039c0 0x7f4430003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 --2- 172.21.15.29:0/4070239021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f444815e200 0x7f4430004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 --2- 172.21.15.29:0/4070239021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4430004d50 0x7f44300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f44480d2ee0 con 0x7f4430004d50 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44480b38d0 con 0x7f44300039c0 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f443000b9d0 con 0x7f444815e200 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f4447fff640 1 --2- 172.21.15.29:0/4070239021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4430004d50 0x7f44300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f4446ffd640 1 --2- 172.21.15.29:0/4070239021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f444815e200 0x7f4430004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f44477fe640 1 --2- 172.21.15.29:0/4070239021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44300039c0 0x7f4430003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f4447fff640 1 -- 172.21.15.29:0/4070239021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f444815e200 msgr2=0x7f4430004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f4447fff640 1 --2- 172.21.15.29:0/4070239021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f444815e200 0x7f4430004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f4447fff640 1 -- 172.21.15.29:0/4070239021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44300039c0 msgr2=0x7f4430003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f4447fff640 1 --2- 172.21.15.29:0/4070239021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44300039c0 0x7f4430003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.820+0000 7f4447fff640 1 -- 172.21.15.29:0/4070239021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4430003530 con 0x7f4430004d50 2024-04-25T00:36:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.824+0000 7f4447fff640 1 --2- 172.21.15.29:0/4070239021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4430004d50 0x7f44300051d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f444000a7d0 tx=0x7f444001d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc0fa81a5fc0f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.824+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444000a3f0 con 0x7f4430004d50 2024-04-25T00:36:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.824+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44400ad630 con 0x7f4430004d50 2024-04-25T00:36:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.824+0000 7f44467fc640 1 -- 172.21.15.29:0/4070239021 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44400ada40 con 0x7f4430004d50 2024-04-25T00:36:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.824+0000 7f444e11c640 1 -- 172.21.15.29:0/4070239021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4430004d50 msgr2=0x7f44300051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.824+0000 7f444e11c640 1 --2- 172.21.15.29:0/4070239021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4430004d50 0x7f44300051d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f444000a7d0 tx=0x7f444001d300 comp rx=0 tx=0).stop 2024-04-25T00:36:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.824+0000 7f4447fff640 1 -- 172.21.15.29:0/4070239021 reap_dead start 2024-04-25T00:36:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.824+0000 7f444e11c640 1 -- 172.21.15.29:0/4070239021 shutdown_connections 2024-04-25T00:36:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.824+0000 7f444e11c640 1 -- 172.21.15.29:0/4070239021 >> 172.21.15.29:0/4070239021 conn(0x7f444809d280 msgr2=0x7f44480d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.824+0000 7f444e11c640 1 -- 172.21.15.29:0/4070239021 shutdown_connections 2024-04-25T00:36:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.824+0000 7f444e11c640 1 -- 172.21.15.29:0/4070239021 wait complete. 2024-04-25T00:36:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.824+0000 7f444e11c640 1 Processor -- start 2024-04-25T00:36:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.824+0000 7f444e11c640 1 -- start start 2024-04-25T00:36:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f444e11c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4430004d50 0x7f44481534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f444e11c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44480c8690 0x7f44480cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f444e11c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44480cd7d0 0x7f44480d0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f444e11c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f444815e5e0 con 0x7f44480cd7d0 2024-04-25T00:36:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f444e11c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44480b3e00 con 0x7f44480c8690 2024-04-25T00:36:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f444e11c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f444815e760 con 0x7f4430004d50 2024-04-25T00:36:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f4447fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44480cd7d0 0x7f44480d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f4447fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44480cd7d0 0x7f44480d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39564/0 (socket says 172.21.15.29:39564) 2024-04-25T00:36:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f4447fff640 1 -- 172.21.15.29:0/1206548819 learned_addr learned my addr 172.21.15.29:0/1206548819 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f44477fe640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4430004d50 0x7f44481534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f4446ffd640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44480c8690 0x7f44480cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f4447fff640 1 -- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4430004d50 msgr2=0x7f44481534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f4447fff640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4430004d50 0x7f44481534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f4447fff640 1 -- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44480c8690 msgr2=0x7f44480cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f4447fff640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44480c8690 0x7f44480cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f4447fff640 1 -- 172.21.15.29:0/1206548819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44481f7190 con 0x7f44480cd7d0 2024-04-25T00:36:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f4447fff640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44480cd7d0 0x7f44480d0900 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f444000a7d0 tx=0x7f4430005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f442ffff640 1 -- 172.21.15.29:0/1206548819 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44400b2040 con 0x7f44480cd7d0 2024-04-25T00:36:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f442ffff640 1 -- 172.21.15.29:0/1206548819 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44400bc070 con 0x7f44480cd7d0 2024-04-25T00:36:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f442ffff640 1 -- 172.21.15.29:0/1206548819 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440001f50 con 0x7f44480cd7d0 2024-04-25T00:36:07.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f444e11c640 1 -- 172.21.15.29:0/1206548819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44481f7470 con 0x7f44480cd7d0 2024-04-25T00:36:07.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f444e11c640 1 -- 172.21.15.29:0/1206548819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44481f7a30 con 0x7f44480cd7d0 2024-04-25T00:36:07.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f444e11c640 1 -- 172.21.15.29:0/1206548819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4448069270 con 0x7f44480cd7d0 2024-04-25T00:36:07.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f442ffff640 1 -- 172.21.15.29:0/1206548819 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f44400ad630 con 0x7f44480cd7d0 2024-04-25T00:36:07.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f442ffff640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4434067c50 0x7f443406a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:07.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f442ffff640 1 -- 172.21.15.29:0/1206548819 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f4440120cb0 con 0x7f44480cd7d0 2024-04-25T00:36:07.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.828+0000 7f44477fe640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4434067c50 0x7f443406a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.832+0000 7f44477fe640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4434067c50 0x7f443406a110 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f443c0099b0 tx=0x7f443c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:07.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.832+0000 7f442ffff640 1 -- 172.21.15.29:0/1206548819 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4440017010 con 0x7f44480cd7d0 2024-04-25T00:36:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.988+0000 7f444e11c640 1 -- 172.21.15.29:0/1206548819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f44480ca8b0 con 0x7f44480cd7d0 2024-04-25T00:36:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.988+0000 7f442ffff640 1 -- 172.21.15.29:0/1206548819 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f44400019c0 con 0x7f44480cd7d0 2024-04-25T00:36:07.995 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:36:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.992+0000 7f444e11c640 1 -- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4434067c50 msgr2=0x7f443406a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.992+0000 7f444e11c640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4434067c50 0x7f443406a110 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f443c0099b0 tx=0x7f443c008040 comp rx=0 tx=0).stop 2024-04-25T00:36:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.992+0000 7f444e11c640 1 -- 172.21.15.29:0/1206548819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44480cd7d0 msgr2=0x7f44480d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.992+0000 7f444e11c640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44480cd7d0 0x7f44480d0900 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f444000a7d0 tx=0x7f4430005840 comp rx=0 tx=0).stop 2024-04-25T00:36:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.992+0000 7f444e11c640 1 -- 172.21.15.29:0/1206548819 shutdown_connections 2024-04-25T00:36:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.992+0000 7f444e11c640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44480cd7d0 0x7f44480d0900 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.992+0000 7f444e11c640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f4434067c50 0x7f443406a110 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.992+0000 7f444e11c640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44480c8690 0x7f44480cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.992+0000 7f444e11c640 1 --2- 172.21.15.29:0/1206548819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4430004d50 0x7f44481534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.992+0000 7f444e11c640 1 -- 172.21.15.29:0/1206548819 >> 172.21.15.29:0/1206548819 conn(0x7f444809d280 msgr2=0x7f444806c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.992+0000 7f444e11c640 1 -- 172.21.15.29:0/1206548819 shutdown_connections 2024-04-25T00:36:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:07.992+0000 7f444e11c640 1 -- 172.21.15.29:0/1206548819 wait complete. 2024-04-25T00:36:08.009 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:36:08.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f9104f09640 1 Processor -- start 2024-04-25T00:36:08.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f9104f09640 1 -- start start 2024-04-25T00:36:08.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f9104f09640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91000ce100 0x7f91000cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f9104f09640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f910015c4c0 0x7f91000b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f9104f09640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f91000b3a10 0x7f91000715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f9104f09640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f91000afe20 con 0x7f91000b3a10 2024-04-25T00:36:08.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f9104f09640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f910015c9f0 con 0x7f91000ce100 2024-04-25T00:36:08.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f9104f09640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f910015ccd0 con 0x7f910015c4c0 2024-04-25T00:36:08.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f90fed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f91000b3a10 0x7f91000715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f90fed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f91000b3a10 0x7f91000715b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39580/0 (socket says 172.21.15.29:39580) 2024-04-25T00:36:08.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f90fed9d640 1 -- 172.21.15.29:0/3208396513 learned_addr learned my addr 172.21.15.29:0/3208396513 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:08.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f90fdd9b640 1 --2- 172.21.15.29:0/3208396513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f910015c4c0 0x7f91000b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f90fe59c640 1 --2- 172.21.15.29:0/3208396513 >> v2:172.21.15.157:3300/0 conn(0x7f91000ce100 0x7f91000cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f90fed9d640 1 -- 172.21.15.29:0/3208396513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f910015c4c0 msgr2=0x7f91000b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f90fed9d640 1 --2- 172.21.15.29:0/3208396513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f910015c4c0 0x7f91000b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f90fed9d640 1 -- 172.21.15.29:0/3208396513 >> v2:172.21.15.157:3300/0 conn(0x7f91000ce100 msgr2=0x7f91000cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f90fed9d640 1 --2- 172.21.15.29:0/3208396513 >> v2:172.21.15.157:3300/0 conn(0x7f91000ce100 0x7f91000cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.136+0000 7f90fed9d640 1 -- 172.21.15.29:0/3208396513 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910015cfb0 con 0x7f91000b3a10 2024-04-25T00:36:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f90fed9d640 1 --2- 172.21.15.29:0/3208396513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91000b3a10 0x7f91000715b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f91000b2d90 tx=0x7f90f001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2463447f542444 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f90fd59a640 1 -- 172.21.15.29:0/3208396513 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f00aa610 con 0x7f91000b3a10 2024-04-25T00:36:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f90fd59a640 1 -- 172.21.15.29:0/3208396513 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f00af070 con 0x7f91000b3a10 2024-04-25T00:36:08.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f90fd59a640 1 -- 172.21.15.29:0/3208396513 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f00aa9d0 con 0x7f91000b3a10 2024-04-25T00:36:08.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f9104f09640 1 -- 172.21.15.29:0/3208396513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91000b3a10 msgr2=0x7f91000715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f9104f09640 1 --2- 172.21.15.29:0/3208396513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91000b3a10 0x7f91000715b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f91000b2d90 tx=0x7f90f001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f9104f09640 1 -- 172.21.15.29:0/3208396513 shutdown_connections 2024-04-25T00:36:08.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f9104f09640 1 --2- 172.21.15.29:0/3208396513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91000b3a10 0x7f91000715b0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f9104f09640 1 --2- 172.21.15.29:0/3208396513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f910015c4c0 0x7f91000b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f9104f09640 1 --2- 172.21.15.29:0/3208396513 >> v2:172.21.15.157:3300/0 conn(0x7f91000ce100 0x7f91000cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f9104f09640 1 -- 172.21.15.29:0/3208396513 >> 172.21.15.29:0/3208396513 conn(0x7f9100075030 msgr2=0x7f91000b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:08.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f9104f09640 1 -- 172.21.15.29:0/3208396513 shutdown_connections 2024-04-25T00:36:08.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f9104f09640 1 -- 172.21.15.29:0/3208396513 wait complete. 2024-04-25T00:36:08.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f9104f09640 1 Processor -- start 2024-04-25T00:36:08.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.140+0000 7f9104f09640 1 -- start start 2024-04-25T00:36:08.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f9104f09640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91000ce100 0x7f9100146ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f9104f09640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f910015c4c0 0x7f910014d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f9104f09640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f910014b400 0x7f910014dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f9104f09640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f91000b57e0 con 0x7f91000ce100 2024-04-25T00:36:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f9104f09640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9100071f00 con 0x7f910015c4c0 2024-04-25T00:36:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f9104f09640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f91000b5960 con 0x7f910014b400 2024-04-25T00:36:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90fe59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91000ce100 0x7f9100146ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90fe59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91000ce100 0x7f9100146ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39584/0 (socket says 172.21.15.29:39584) 2024-04-25T00:36:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90fe59c640 1 -- 172.21.15.29:0/1338156269 learned_addr learned my addr 172.21.15.29:0/1338156269 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90fed9d640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f910014b400 0x7f910014dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90fdd9b640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f910015c4c0 0x7f910014d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90fe59c640 1 -- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f910014b400 msgr2=0x7f910014dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90fe59c640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f910014b400 0x7f910014dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90fe59c640 1 -- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f910015c4c0 msgr2=0x7f910014d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90fe59c640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f910015c4c0 0x7f910014d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90fe59c640 1 -- 172.21.15.29:0/1338156269 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910014c6f0 con 0x7f91000ce100 2024-04-25T00:36:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90fe59c640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91000ce100 0x7f9100146ba0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f90f8007e90 tx=0x7f90f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90eeffd640 1 -- 172.21.15.29:0/1338156269 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8014620 con 0x7f91000ce100 2024-04-25T00:36:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90eeffd640 1 -- 172.21.15.29:0/1338156269 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f8019070 con 0x7f91000ce100 2024-04-25T00:36:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f9104f09640 1 -- 172.21.15.29:0/1338156269 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910014c960 con 0x7f91000ce100 2024-04-25T00:36:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90eeffd640 1 -- 172.21.15.29:0/1338156269 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8014a00 con 0x7f91000ce100 2024-04-25T00:36:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f9104f09640 1 -- 172.21.15.29:0/1338156269 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9100150660 con 0x7f91000ce100 2024-04-25T00:36:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f9104f09640 1 -- 172.21.15.29:0/1338156269 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9100069270 con 0x7f91000ce100 2024-04-25T00:36:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90eeffd640 1 -- 172.21.15.29:0/1338156269 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f90f8014ba0 con 0x7f91000ce100 2024-04-25T00:36:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90eeffd640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f90e0067c20 0x7f90e006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90eeffd640 1 -- 172.21.15.29:0/1338156269 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f90f8086a60 con 0x7f91000ce100 2024-04-25T00:36:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.144+0000 7f90fdd9b640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f90e0067c20 0x7f90e006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.148+0000 7f90fdd9b640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f90e0067c20 0x7f90e006a0e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9100152250 tx=0x7f90f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.148+0000 7f90eeffd640 1 -- 172.21.15.29:0/1338156269 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f90f8011260 con 0x7f91000ce100 2024-04-25T00:36:08.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f9104f09640 1 -- 172.21.15.29:0/1338156269 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f91000af690 con 0x7f91000ce100 2024-04-25T00:36:08.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f90eeffd640 1 -- 172.21.15.29:0/1338156269 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) ==== 104+0+11 (secure 0 0 0) 0x7f90f8057190 con 0x7f91000ce100 2024-04-25T00:36:08.310 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:36:08.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f9104f09640 1 -- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f90e0067c20 msgr2=0x7f90e006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f9104f09640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f90e0067c20 0x7f90e006a0e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9100152250 tx=0x7f90f4008040 comp rx=0 tx=0).stop 2024-04-25T00:36:08.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f9104f09640 1 -- 172.21.15.29:0/1338156269 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91000ce100 msgr2=0x7f9100146ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f9104f09640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91000ce100 0x7f9100146ba0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f90f8007e90 tx=0x7f90f800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f9104f09640 1 -- 172.21.15.29:0/1338156269 shutdown_connections 2024-04-25T00:36:08.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f9104f09640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f910014b400 0x7f910014dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f9104f09640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f90e0067c20 0x7f90e006a0e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f9104f09640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f910015c4c0 0x7f910014d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f9104f09640 1 --2- 172.21.15.29:0/1338156269 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91000ce100 0x7f9100146ba0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f9104f09640 1 -- 172.21.15.29:0/1338156269 >> 172.21.15.29:0/1338156269 conn(0x7f9100075030 msgr2=0x7f910015ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:08.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f9104f09640 1 -- 172.21.15.29:0/1338156269 shutdown_connections 2024-04-25T00:36:08.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.308+0000 7f9104f09640 1 -- 172.21.15.29:0/1338156269 wait complete. 2024-04-25T00:36:08.328 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:36:08.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb0302e640 1 Processor -- start 2024-04-25T00:36:08.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb0302e640 1 -- start start 2024-04-25T00:36:08.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb0302e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feafc06b210 0x7feafc06db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb0302e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feafc0d18f0 0x7feafc15f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb0302e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feafc15f730 con 0x7feafc06b5f0 2024-04-25T00:36:08.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb0302e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7feafc0d0410 con 0x7feafc0d18f0 2024-04-25T00:36:08.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb0302e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7feafc0ce880 con 0x7feafc06b210 2024-04-25T00:36:08.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb015cb640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7feafc06b5f0 0x7feafc071fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59794/0 (socket says 172.21.15.29:59794) 2024-04-25T00:36:08.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb015cb640 1 -- 172.21.15.29:0/1098375424 learned_addr learned my addr 172.21.15.29:0/1098375424 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:08.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb00dca640 1 --2- 172.21.15.29:0/1098375424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feafc06b210 0x7feafc06db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feafbfff640 1 --2- 172.21.15.29:0/1098375424 >> v2:172.21.15.157:3300/0 conn(0x7feafc0d18f0 0x7feafc15f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feafb7fe640 1 -- 172.21.15.29:0/1098375424 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3052025412 0 0) 0x7feaf40014a0 con 0x7feafc06b5f0 2024-04-25T00:36:08.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feafb7fe640 1 -- 172.21.15.29:0/1098375424 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feae4003530 con 0x7feafc06b5f0 2024-04-25T00:36:08.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb00dca640 1 -- 172.21.15.29:0/1098375424 >> v2:172.21.15.157:3300/0 conn(0x7feafc0d18f0 msgr2=0x7feafc15f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb00dca640 1 --2- 172.21.15.29:0/1098375424 >> v2:172.21.15.157:3300/0 conn(0x7feafc0d18f0 0x7feafc15f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feafb7fe640 1 -- 172.21.15.29:0/1098375424 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 307033076 0 0) 0x7feae4003530 con 0x7feafc06b5f0 2024-04-25T00:36:08.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb00dca640 1 -- 172.21.15.29:0/1098375424 >> v1:172.21.15.29:6789/0 conn(0x7feafc06b5f0 legacy=0x7feafc071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.456+0000 7feb00dca640 1 -- 172.21.15.29:0/1098375424 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feafc0d06b0 con 0x7feafc06b210 2024-04-25T00:36:08.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feb00dca640 1 --2- 172.21.15.29:0/1098375424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feafc06b210 0x7feafc06db00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feaec0097c0 tx=0x7feaec01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47757e55479d5377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feafb7fe640 1 -- 172.21.15.29:0/1098375424 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec0b0070 con 0x7feafc06b210 2024-04-25T00:36:08.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feafb7fe640 1 -- 172.21.15.29:0/1098375424 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaec0ab400 con 0x7feafc06b210 2024-04-25T00:36:08.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feafb7fe640 1 -- 172.21.15.29:0/1098375424 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4001bd0 con 0x7feafc06b210 2024-04-25T00:36:08.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feb0302e640 1 -- 172.21.15.29:0/1098375424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feafc06b210 msgr2=0x7feafc06db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feb0302e640 1 --2- 172.21.15.29:0/1098375424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feafc06b210 0x7feafc06db00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feaec0097c0 tx=0x7feaec01c3c0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feb0302e640 1 -- 172.21.15.29:0/1098375424 shutdown_connections 2024-04-25T00:36:08.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feb0302e640 1 --2- 172.21.15.29:0/1098375424 >> v2:172.21.15.157:3300/0 conn(0x7feafc0d18f0 0x7feafc15f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feb0302e640 1 --2- 172.21.15.29:0/1098375424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feafc06b210 0x7feafc06db00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feb0302e640 1 -- 172.21.15.29:0/1098375424 >> 172.21.15.29:0/1098375424 conn(0x7feafc0d0cb0 msgr2=0x7feafc0d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:08.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feb0302e640 1 -- 172.21.15.29:0/1098375424 shutdown_connections 2024-04-25T00:36:08.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feb0302e640 1 -- 172.21.15.29:0/1098375424 wait complete. 2024-04-25T00:36:08.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feb0302e640 1 Processor -- start 2024-04-25T00:36:08.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.460+0000 7feb0302e640 1 -- start start 2024-04-25T00:36:08.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feb0302e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feafc06b210 0x7feafc0af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feb0302e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feafc06b5f0 0x7feafc0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feb0302e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feafc0d18f0 0x7feafc157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feb0302e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feafc158b40 con 0x7feafc06b5f0 2024-04-25T00:36:08.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feb0302e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feafc06ba70 con 0x7feafc06b210 2024-04-25T00:36:08.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feb0302e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7feafc16adb0 con 0x7feafc0d18f0 2024-04-25T00:36:08.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feafbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feafc06b5f0 0x7feafc0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feafbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feafc06b5f0 0x7feafc0cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39588/0 (socket says 172.21.15.29:39588) 2024-04-25T00:36:08.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feafbfff640 1 -- 172.21.15.29:0/1307805298 learned_addr learned my addr 172.21.15.29:0/1307805298 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:08.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feb00dca640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feafc06b210 0x7feafc0af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feb015cb640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feafc0d18f0 0x7feafc157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feafbfff640 1 -- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feafc0d18f0 msgr2=0x7feafc157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feafbfff640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feafc0d18f0 0x7feafc157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feafbfff640 1 -- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feafc06b210 msgr2=0x7feafc0af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feafbfff640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feafc06b210 0x7feafc0af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feafbfff640 1 -- 172.21.15.29:0/1307805298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feafc0cb050 con 0x7feafc06b5f0 2024-04-25T00:36:08.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feb00dca640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feafc06b210 0x7feafc0af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:08.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feafbfff640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feafc06b5f0 0x7feafc0cc2f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7feaf000e840 tx=0x7feaf000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feaf97fa640 1 -- 172.21.15.29:0/1307805298 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0017070 con 0x7feafc06b5f0 2024-04-25T00:36:08.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feaf97fa640 1 -- 172.21.15.29:0/1307805298 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf000ee90 con 0x7feafc06b5f0 2024-04-25T00:36:08.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feaf97fa640 1 -- 172.21.15.29:0/1307805298 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf000cb60 con 0x7feafc06b5f0 2024-04-25T00:36:08.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feb0302e640 1 -- 172.21.15.29:0/1307805298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feafc0cb330 con 0x7feafc06b5f0 2024-04-25T00:36:08.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feb0302e640 1 -- 172.21.15.29:0/1307805298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feafc0cb8f0 con 0x7feafc06b5f0 2024-04-25T00:36:08.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feb0302e640 1 -- 172.21.15.29:0/1307805298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feafc069270 con 0x7feafc06b5f0 2024-04-25T00:36:08.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feaf97fa640 1 -- 172.21.15.29:0/1307805298 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7feaf001a070 con 0x7feafc06b5f0 2024-04-25T00:36:08.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feaf97fa640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7feae8067c20 0x7feae806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feaf97fa640 1 -- 172.21.15.29:0/1307805298 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7feaf00864f0 con 0x7feafc06b5f0 2024-04-25T00:36:08.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.464+0000 7feb00dca640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7feae8067c20 0x7feae806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.468+0000 7feb00dca640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7feae8067c20 0x7feae806a0e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feaec0ad000 tx=0x7feaec0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.472+0000 7feaf97fa640 1 -- 172.21.15.29:0/1307805298 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feaf0056b50 con 0x7feafc06b5f0 2024-04-25T00:36:08.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.632+0000 7feb0302e640 1 -- 172.21.15.29:0/1307805298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feafc06e030 con 0x7feafc06b5f0 2024-04-25T00:36:08.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.632+0000 7feaf97fa640 1 -- 172.21.15.29:0/1307805298 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) ==== 100+0+11 (secure 0 0 0) 0x7feaf0056450 con 0x7feafc06b5f0 2024-04-25T00:36:08.635 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:36:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.636+0000 7feb0302e640 1 -- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7feae8067c20 msgr2=0x7feae806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.636+0000 7feb0302e640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7feae8067c20 0x7feae806a0e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feaec0ad000 tx=0x7feaec0a6000 comp rx=0 tx=0).stop 2024-04-25T00:36:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.636+0000 7feb0302e640 1 -- 172.21.15.29:0/1307805298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feafc06b5f0 msgr2=0x7feafc0cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.636+0000 7feb0302e640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feafc06b5f0 0x7feafc0cc2f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7feaf000e840 tx=0x7feaf000ec40 comp rx=0 tx=0).stop 2024-04-25T00:36:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.636+0000 7feb0302e640 1 -- 172.21.15.29:0/1307805298 shutdown_connections 2024-04-25T00:36:08.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.636+0000 7feb0302e640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feafc0d18f0 0x7feafc157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.636+0000 7feb0302e640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feafc06b5f0 0x7feafc0cc2f0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.636+0000 7feb0302e640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7feae8067c20 0x7feae806a0e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.636+0000 7feb0302e640 1 --2- 172.21.15.29:0/1307805298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feafc06b210 0x7feafc0af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.636+0000 7feb0302e640 1 -- 172.21.15.29:0/1307805298 >> 172.21.15.29:0/1307805298 conn(0x7feafc0d0cb0 msgr2=0x7feafc15c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:08.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.636+0000 7feb0302e640 1 -- 172.21.15.29:0/1307805298 shutdown_connections 2024-04-25T00:36:08.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.636+0000 7feb0302e640 1 -- 172.21.15.29:0/1307805298 wait complete. 2024-04-25T00:36:08.656 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:36:08.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cc6068640 1 Processor -- start 2024-04-25T00:36:08.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cc6068640 1 -- start start 2024-04-25T00:36:08.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cc6068640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cc00d2bb0 0x7f5cc00ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cc6068640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc00c8690 con 0x7f5cc015e630 2024-04-25T00:36:08.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cc6068640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc00c8970 con 0x7f5cc015e250 2024-04-25T00:36:08.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cc6068640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc00c8440 con 0x7f5cc00d2bb0 2024-04-25T00:36:08.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5cc015e630 0x7f5cc00cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59800/0 (socket says 172.21.15.29:59800) 2024-04-25T00:36:08.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbffff640 1 -- 172.21.15.29:0/1591707763 learned_addr learned my addr 172.21.15.29:0/1591707763 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:08.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbeffd640 1 --2- 172.21.15.29:0/1591707763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cc00d2bb0 0x7f5cc00ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3409872587 0 0) 0x7f5cc00c8690 con 0x7f5cc015e630 2024-04-25T00:36:08.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cac003530 con 0x7f5cc015e630 2024-04-25T00:36:08.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2924627146 0 0) 0x7f5cac003530 con 0x7f5cc015e630 2024-04-25T00:36:08.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cc00d2bb0 msgr2=0x7f5cc00ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 --2- 172.21.15.29:0/1591707763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cc00d2bb0 0x7f5cc00ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 >> v1:172.21.15.157:6789/0 conn(0x7f5cc015e250 legacy=0x7f5cc00d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc00c8c50 con 0x7f5cc015e630 2024-04-25T00:36:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f5cb80016a0 con 0x7f5cc015e630 2024-04-25T00:36:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 >> v1:172.21.15.29:6789/0 conn(0x7f5cc015e630 legacy=0x7f5cc00cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 --2- 172.21.15.29:0/1591707763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cac0039f0 0x7f5cac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 --2- 172.21.15.29:0/1591707763 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cac004810 0x7f5cac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 --2- 172.21.15.29:0/1591707763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cc00d2bb0 0x7f5cac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc00c8440 con 0x7f5cac004810 2024-04-25T00:36:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac00bb00 con 0x7f5cac0039f0 2024-04-25T00:36:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.780+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5cac00bd00 con 0x7f5cc00d2bb0 2024-04-25T00:36:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbeffd640 1 --2- 172.21.15.29:0/1591707763 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cac004810 0x7f5cac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbffff640 1 --2- 172.21.15.29:0/1591707763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cc00d2bb0 0x7f5cac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbf7fe640 1 --2- 172.21.15.29:0/1591707763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cac0039f0 0x7f5cac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbeffd640 1 -- 172.21.15.29:0/1591707763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cc00d2bb0 msgr2=0x7f5cac005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbeffd640 1 --2- 172.21.15.29:0/1591707763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cc00d2bb0 0x7f5cac005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbeffd640 1 -- 172.21.15.29:0/1591707763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cac0039f0 msgr2=0x7f5cac003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbeffd640 1 --2- 172.21.15.29:0/1591707763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cac0039f0 0x7f5cac003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbeffd640 1 -- 172.21.15.29:0/1591707763 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac003530 con 0x7f5cac004810 2024-04-25T00:36:08.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbf7fe640 1 --2- 172.21.15.29:0/1591707763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cac0039f0 0x7f5cac003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:08.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbeffd640 1 --2- 172.21.15.29:0/1591707763 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cac004810 0x7f5cac004c90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009ae0 tx=0x7f5ca801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6d7d4ddec7e09b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80b2070 con 0x7f5cac004810 2024-04-25T00:36:08.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca80a5550 con 0x7f5cac004810 2024-04-25T00:36:08.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbe7fc640 1 -- 172.21.15.29:0/1591707763 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80096b0 con 0x7f5cac004810 2024-04-25T00:36:08.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cc6068640 1 -- 172.21.15.29:0/1591707763 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cac004810 msgr2=0x7f5cac004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cc6068640 1 --2- 172.21.15.29:0/1591707763 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cac004810 0x7f5cac004c90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009ae0 tx=0x7f5ca801ca70 comp rx=0 tx=0).stop 2024-04-25T00:36:08.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cbffff640 1 -- 172.21.15.29:0/1591707763 reap_dead start 2024-04-25T00:36:08.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cc6068640 1 -- 172.21.15.29:0/1591707763 shutdown_connections 2024-04-25T00:36:08.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cc6068640 1 -- 172.21.15.29:0/1591707763 >> 172.21.15.29:0/1591707763 conn(0x7f5cc00d3f70 msgr2=0x7f5cc00d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:08.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cc6068640 1 -- 172.21.15.29:0/1591707763 shutdown_connections 2024-04-25T00:36:08.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.784+0000 7f5cc6068640 1 -- 172.21.15.29:0/1591707763 wait complete. 2024-04-25T00:36:08.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cc6068640 1 Processor -- start 2024-04-25T00:36:08.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cc6068640 1 -- start start 2024-04-25T00:36:08.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cc6068640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cac004810 0x7f5cc01572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cc6068640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cc0153010 0x7f5cc0157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cc6068640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cc01534a0 0x7f5cc0157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cc6068640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc0169ff0 con 0x7f5cc0153010 2024-04-25T00:36:08.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cc6068640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc00d1ed0 con 0x7f5cc01534a0 2024-04-25T00:36:08.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cc6068640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc016a170 con 0x7f5cac004810 2024-04-25T00:36:08.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cbeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cc0153010 0x7f5cc0157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cbeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cc0153010 0x7f5cc0157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39608/0 (socket says 172.21.15.29:39608) 2024-04-25T00:36:08.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cbeffd640 1 -- 172.21.15.29:0/3740624482 learned_addr learned my addr 172.21.15.29:0/3740624482 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:08.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cbffff640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cc01534a0 0x7f5cc0157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cbf7fe640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cac004810 0x7f5cc01572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cbeffd640 1 -- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cac004810 msgr2=0x7f5cc01572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cbeffd640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cac004810 0x7f5cc01572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cbeffd640 1 -- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cc01534a0 msgr2=0x7f5cc0157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cbeffd640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cc01534a0 0x7f5cc0157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cbeffd640 1 -- 172.21.15.29:0/3740624482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc00cdfb0 con 0x7f5cc0153010 2024-04-25T00:36:08.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cbeffd640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cc0153010 0x7f5cc0157800 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f5ca801cf50 tx=0x7f5cac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cb7fff640 1 -- 172.21.15.29:0/3740624482 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80bc030 con 0x7f5cc0153010 2024-04-25T00:36:08.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cb7fff640 1 -- 172.21.15.29:0/3740624482 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca800f000 con 0x7f5cc0153010 2024-04-25T00:36:08.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cc6068640 1 -- 172.21.15.29:0/3740624482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc00cd9f0 con 0x7f5cc0153010 2024-04-25T00:36:08.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cb7fff640 1 -- 172.21.15.29:0/3740624482 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80b2040 con 0x7f5cc0153010 2024-04-25T00:36:08.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cc6068640 1 -- 172.21.15.29:0/3740624482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc0161fd0 con 0x7f5cc0153010 2024-04-25T00:36:08.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cc6068640 1 -- 172.21.15.29:0/3740624482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc0069270 con 0x7f5cc0153010 2024-04-25T00:36:08.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.788+0000 7f5cb7fff640 1 -- 172.21.15.29:0/3740624482 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f5ca80bf070 con 0x7f5cc0153010 2024-04-25T00:36:08.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.792+0000 7f5cb7fff640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5ca0067c50 0x7f5ca006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.792+0000 7f5cb7fff640 1 -- 172.21.15.29:0/3740624482 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f5ca811ff70 con 0x7f5cc0153010 2024-04-25T00:36:08.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.792+0000 7f5cbf7fe640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5ca0067c50 0x7f5ca006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.792+0000 7f5cbf7fe640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5ca0067c50 0x7f5ca006a110 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0007fd0 tx=0x7f5cb0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.796+0000 7f5cb7fff640 1 -- 172.21.15.29:0/3740624482 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5ca80c7020 con 0x7f5cc0153010 2024-04-25T00:36:08.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.972+0000 7f5cc6068640 1 -- 172.21.15.29:0/3740624482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5cc00c7b90 con 0x7f5cc0153010 2024-04-25T00:36:08.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.972+0000 7f5cb7fff640 1 -- 172.21.15.29:0/3740624482 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7f5ca8016780 con 0x7f5cc0153010 2024-04-25T00:36:08.974 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:36:08.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.976+0000 7f5cc6068640 1 -- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5ca0067c50 msgr2=0x7f5ca006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.976+0000 7f5cc6068640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5ca0067c50 0x7f5ca006a110 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0007fd0 tx=0x7f5cb0009040 comp rx=0 tx=0).stop 2024-04-25T00:36:08.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.976+0000 7f5cc6068640 1 -- 172.21.15.29:0/3740624482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cc0153010 msgr2=0x7f5cc0157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.976+0000 7f5cc6068640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cc0153010 0x7f5cc0157800 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f5ca801cf50 tx=0x7f5cac005840 comp rx=0 tx=0).stop 2024-04-25T00:36:08.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.976+0000 7f5cc6068640 1 -- 172.21.15.29:0/3740624482 shutdown_connections 2024-04-25T00:36:08.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.976+0000 7f5cc6068640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5ca0067c50 0x7f5ca006a110 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.976+0000 7f5cc6068640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cc01534a0 0x7f5cc0157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.976+0000 7f5cc6068640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cc0153010 0x7f5cc0157800 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.976+0000 7f5cc6068640 1 --2- 172.21.15.29:0/3740624482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cac004810 0x7f5cc01572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.976+0000 7f5cc6068640 1 -- 172.21.15.29:0/3740624482 >> 172.21.15.29:0/3740624482 conn(0x7f5cc00d3f70 msgr2=0x7f5cc00d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:08.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.976+0000 7f5cc6068640 1 -- 172.21.15.29:0/3740624482 shutdown_connections 2024-04-25T00:36:08.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:08.976+0000 7f5cc6068640 1 -- 172.21.15.29:0/3740624482 wait complete. 2024-04-25T00:36:08.994 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:36:08.994 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:36:09.000 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:36:09.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.168+0000 7fb15ee9f640 1 Processor -- start 2024-04-25T00:36:09.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.168+0000 7fb15ee9f640 1 -- start start 2024-04-25T00:36:09.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb15ee9f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb1580cf690 0x7fb1580d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb15ee9f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1580d2ea0 con 0x7fb15815e200 2024-04-25T00:36:09.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb15ee9f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1580cfc10 con 0x7fb1580d2a90 2024-04-25T00:36:09.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb15ee9f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb1580b38d0 con 0x7fb1580cf690 2024-04-25T00:36:09.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb15cc3b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb15815e200 0x7fb1580d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59804/0 (socket says 172.21.15.29:59804) 2024-04-25T00:36:09.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb15cc3b640 1 -- 172.21.15.29:0/2505430848 learned_addr learned my addr 172.21.15.29:0/2505430848 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:09.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb157fff640 1 --2- 172.21.15.29:0/2505430848 >> v2:172.21.15.29:3300/0 conn(0x7fb1580cf690 0x7fb1580d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb157fff640 1 -- 172.21.15.29:0/2505430848 >> v1:172.21.15.157:6789/0 conn(0x7fb1580d2a90 legacy=0x7fb1580b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:09.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb157fff640 1 -- 172.21.15.29:0/2505430848 >> v1:172.21.15.29:6789/0 conn(0x7fb15815e200 legacy=0x7fb1580d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb157fff640 1 -- 172.21.15.29:0/2505430848 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1580cfef0 con 0x7fb1580cf690 2024-04-25T00:36:09.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb1577fe640 1 -- 172.21.15.29:0/2505430848 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2060975931 0 0) 0x7fb148001120 con 0x7fb15815e200 2024-04-25T00:36:09.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb157fff640 1 --2- 172.21.15.29:0/2505430848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1580cf690 0x7fb1580d2550 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0097c0 tx=0x7fb14c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc78e6f479254267 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:09.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb1577fe640 1 -- 172.21.15.29:0/2505430848 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0b0070 con 0x7fb1580cf690 2024-04-25T00:36:09.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb1577fe640 1 -- 172.21.15.29:0/2505430848 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb14c004230 con 0x7fb1580cf690 2024-04-25T00:36:09.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.172+0000 7fb1577fe640 1 -- 172.21.15.29:0/2505430848 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c003620 con 0x7fb1580cf690 2024-04-25T00:36:09.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 -- 172.21.15.29:0/2505430848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1580cf690 msgr2=0x7fb1580d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 --2- 172.21.15.29:0/2505430848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1580cf690 0x7fb1580d2550 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0097c0 tx=0x7fb14c01c460 comp rx=0 tx=0).stop 2024-04-25T00:36:09.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 -- 172.21.15.29:0/2505430848 shutdown_connections 2024-04-25T00:36:09.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 --2- 172.21.15.29:0/2505430848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1580cf690 0x7fb1580d2550 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 -- 172.21.15.29:0/2505430848 >> 172.21.15.29:0/2505430848 conn(0x7fb15809d280 msgr2=0x7fb1580d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:09.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 -- 172.21.15.29:0/2505430848 shutdown_connections 2024-04-25T00:36:09.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 -- 172.21.15.29:0/2505430848 wait complete. 2024-04-25T00:36:09.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 Processor -- start 2024-04-25T00:36:09.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 -- start start 2024-04-25T00:36:09.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1580cf690 0x7fb158156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1580d2a90 0x7fb1580b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb15815e200 0x7fb1580b9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb1580b53c0 con 0x7fb1580d2a90 2024-04-25T00:36:09.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb1580b5240 con 0x7fb1580cf690 2024-04-25T00:36:09.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15ee9f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb1580b5540 con 0x7fb15815e200 2024-04-25T00:36:09.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb157fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1580d2a90 0x7fb1580b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb157fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1580d2a90 0x7fb1580b7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39620/0 (socket says 172.21.15.29:39620) 2024-04-25T00:36:09.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb157fff640 1 -- 172.21.15.29:0/1756398410 learned_addr learned my addr 172.21.15.29:0/1756398410 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:09.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15cc3b640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1580cf690 0x7fb158156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb15d43c640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb15815e200 0x7fb1580b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb157fff640 1 -- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb15815e200 msgr2=0x7fb1580b9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb157fff640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb15815e200 0x7fb1580b9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb157fff640 1 -- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1580cf690 msgr2=0x7fb158156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb157fff640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1580cf690 0x7fb158156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb157fff640 1 -- 172.21.15.29:0/1756398410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1580cbeb0 con 0x7fb1580d2a90 2024-04-25T00:36:09.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb157fff640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1580d2a90 0x7fb1580b7580 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb14c019070 tx=0x7fb14c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:09.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb1557fa640 1 -- 172.21.15.29:0/1756398410 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0b0040 con 0x7fb1580d2a90 2024-04-25T00:36:09.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb1557fa640 1 -- 172.21.15.29:0/1756398410 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb14c0bc070 con 0x7fb1580d2a90 2024-04-25T00:36:09.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.176+0000 7fb1557fa640 1 -- 172.21.15.29:0/1756398410 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0b5480 con 0x7fb1580d2a90 2024-04-25T00:36:09.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.180+0000 7fb15ee9f640 1 -- 172.21.15.29:0/1756398410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1580cc190 con 0x7fb1580d2a90 2024-04-25T00:36:09.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.180+0000 7fb15ee9f640 1 -- 172.21.15.29:0/1756398410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1580cc750 con 0x7fb1580d2a90 2024-04-25T00:36:09.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.180+0000 7fb15ee9f640 1 -- 172.21.15.29:0/1756398410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb158069270 con 0x7fb1580d2a90 2024-04-25T00:36:09.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.180+0000 7fb1557fa640 1 -- 172.21.15.29:0/1756398410 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb14c0b3070 con 0x7fb1580d2a90 2024-04-25T00:36:09.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.180+0000 7fb1557fa640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb144067c20 0x7fb14406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.180+0000 7fb1557fa640 1 -- 172.21.15.29:0/1756398410 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fb14c120160 con 0x7fb1580d2a90 2024-04-25T00:36:09.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.180+0000 7fb15cc3b640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb144067c20 0x7fb14406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.180+0000 7fb15cc3b640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb144067c20 0x7fb14406a0e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb148000f70 tx=0x7fb148007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:09.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.184+0000 7fb1557fa640 1 -- 172.21.15.29:0/1756398410 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb14c016050 con 0x7fb1580d2a90 2024-04-25T00:36:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.340+0000 7fb15ee9f640 1 -- 172.21.15.29:0/1756398410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1580b43c0 con 0x7fb1580d2a90 2024-04-25T00:36:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.344+0000 7fb1557fa640 1 -- 172.21.15.29:0/1756398410 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16620 (secure 0 0 0) 0x7fb14c0f0740 con 0x7fb1580d2a90 2024-04-25T00:36:09.345 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:09.346 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":70,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:51.187302+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:47.559043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:48.530288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:36:09.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.344+0000 7fb15ee9f640 1 -- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb144067c20 msgr2=0x7fb14406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.344+0000 7fb15ee9f640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb144067c20 0x7fb14406a0e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb148000f70 tx=0x7fb148007b60 comp rx=0 tx=0).stop 2024-04-25T00:36:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.344+0000 7fb15ee9f640 1 -- 172.21.15.29:0/1756398410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1580d2a90 msgr2=0x7fb1580b7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.344+0000 7fb15ee9f640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1580d2a90 0x7fb1580b7580 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb14c019070 tx=0x7fb14c019200 comp rx=0 tx=0).stop 2024-04-25T00:36:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.344+0000 7fb15ee9f640 1 -- 172.21.15.29:0/1756398410 shutdown_connections 2024-04-25T00:36:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.344+0000 7fb15ee9f640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb15815e200 0x7fb1580b9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.344+0000 7fb15ee9f640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1580d2a90 0x7fb1580b7580 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.344+0000 7fb15ee9f640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb144067c20 0x7fb14406a0e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.344+0000 7fb15ee9f640 1 --2- 172.21.15.29:0/1756398410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1580cf690 0x7fb158156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.344+0000 7fb15ee9f640 1 -- 172.21.15.29:0/1756398410 >> 172.21.15.29:0/1756398410 conn(0x7fb15809d280 msgr2=0x7fb15806d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.344+0000 7fb15ee9f640 1 -- 172.21.15.29:0/1756398410 shutdown_connections 2024-04-25T00:36:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.344+0000 7fb15ee9f640 1 -- 172.21.15.29:0/1756398410 wait complete. 2024-04-25T00:36:09.362 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:35:47.559043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:35:48.530288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:36:09.363 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:36:09.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459fa20640 1 Processor -- start 2024-04-25T00:36:09.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459fa20640 1 -- start start 2024-04-25T00:36:09.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459fa20640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45980afb00 0x7f45980b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459fa20640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45980b0e60 0x7f45980afee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459fa20640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45980b1700 con 0x7f45980b0420 2024-04-25T00:36:09.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459fa20640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f45980b19e0 con 0x7f45980afb00 2024-04-25T00:36:09.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459fa20640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f45980b5090 con 0x7f45980b0e60 2024-04-25T00:36:09.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459d7bc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45980afb00 0x7f45980b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459d7bc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45980afb00 0x7f45980b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39624/0 (socket says 172.21.15.29:39624) 2024-04-25T00:36:09.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459d7bc640 1 -- 172.21.15.29:0/2097111498 learned_addr learned my addr 172.21.15.29:0/2097111498 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:09.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459cfbb640 1 --2- 172.21.15.29:0/2097111498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45980b0e60 0x7f45980afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459d7bc640 1 -- 172.21.15.29:0/2097111498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45980b0e60 msgr2=0x7f45980afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:09.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459d7bc640 1 --2- 172.21.15.29:0/2097111498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45980b0e60 0x7f45980afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459d7bc640 1 -- 172.21.15.29:0/2097111498 >> v1:172.21.15.157:6789/0 conn(0x7f45980b0420 legacy=0x7f45980ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:09.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459d7bc640 1 -- 172.21.15.29:0/2097111498 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4598067990 con 0x7f45980afb00 2024-04-25T00:36:09.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.476+0000 7f459cfbb640 1 --2- 172.21.15.29:0/2097111498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45980b0e60 0x7f45980afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:09.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459d7bc640 1 --2- 172.21.15.29:0/2097111498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45980afb00 0x7f45980b33b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4594009870 tx=0x7f459401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899d318bb7cd0d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:09.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f458ffff640 1 -- 172.21.15.29:0/2097111498 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45940b2070 con 0x7f45980afb00 2024-04-25T00:36:09.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f458ffff640 1 -- 172.21.15.29:0/2097111498 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45940a9070 con 0x7f45980afb00 2024-04-25T00:36:09.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f458ffff640 1 -- 172.21.15.29:0/2097111498 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45940ad880 con 0x7f45980afb00 2024-04-25T00:36:09.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 -- 172.21.15.29:0/2097111498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45980afb00 msgr2=0x7f45980b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 --2- 172.21.15.29:0/2097111498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45980afb00 0x7f45980b33b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4594009870 tx=0x7f459401c490 comp rx=0 tx=0).stop 2024-04-25T00:36:09.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 -- 172.21.15.29:0/2097111498 shutdown_connections 2024-04-25T00:36:09.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 --2- 172.21.15.29:0/2097111498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45980b0e60 0x7f45980afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 --2- 172.21.15.29:0/2097111498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45980afb00 0x7f45980b33b0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 -- 172.21.15.29:0/2097111498 >> 172.21.15.29:0/2097111498 conn(0x7f4598072650 msgr2=0x7f4598072a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:09.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 -- 172.21.15.29:0/2097111498 shutdown_connections 2024-04-25T00:36:09.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 -- 172.21.15.29:0/2097111498 wait complete. 2024-04-25T00:36:09.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 Processor -- start 2024-04-25T00:36:09.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 -- start start 2024-04-25T00:36:09.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45980afb00 0x7f4598153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45980b0420 0x7f45980cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45980b0e60 0x7f459814c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f459816b190 con 0x7f45980b0420 2024-04-25T00:36:09.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f459816b010 con 0x7f45980afb00 2024-04-25T00:36:09.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459fa20640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f459816b310 con 0x7f45980b0e60 2024-04-25T00:36:09.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459cfbb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45980b0420 0x7f45980cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459cfbb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45980b0420 0x7f45980cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39640/0 (socket says 172.21.15.29:39640) 2024-04-25T00:36:09.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459cfbb640 1 -- 172.21.15.29:0/1270380142 learned_addr learned my addr 172.21.15.29:0/1270380142 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:09.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459cfbb640 1 -- 172.21.15.29:0/1270380142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45980b0e60 msgr2=0x7f459814c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459cfbb640 1 --2- 172.21.15.29:0/1270380142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45980b0e60 0x7f459814c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459cfbb640 1 -- 172.21.15.29:0/1270380142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45980afb00 msgr2=0x7f4598153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459cfbb640 1 --2- 172.21.15.29:0/1270380142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45980afb00 0x7f4598153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459cfbb640 1 -- 172.21.15.29:0/1270380142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45981552b0 con 0x7f45980b0420 2024-04-25T00:36:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.480+0000 7f459cfbb640 1 --2- 172.21.15.29:0/1270380142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45980b0420 0x7f45980cb8c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f458800c7c0 tx=0x7f458800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.484+0000 7f458dffb640 1 -- 172.21.15.29:0/1270380142 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588007a30 con 0x7f45980b0420 2024-04-25T00:36:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.484+0000 7f458dffb640 1 -- 172.21.15.29:0/1270380142 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4588007bd0 con 0x7f45980b0420 2024-04-25T00:36:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.484+0000 7f459fa20640 1 -- 172.21.15.29:0/1270380142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4598154210 con 0x7f45980b0420 2024-04-25T00:36:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.484+0000 7f458dffb640 1 -- 172.21.15.29:0/1270380142 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45880113f0 con 0x7f45980b0420 2024-04-25T00:36:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.484+0000 7f459fa20640 1 -- 172.21.15.29:0/1270380142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45981547d0 con 0x7f45980b0420 2024-04-25T00:36:09.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.484+0000 7f459fa20640 1 -- 172.21.15.29:0/1270380142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4598069270 con 0x7f45980b0420 2024-04-25T00:36:09.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.484+0000 7f458dffb640 1 -- 172.21.15.29:0/1270380142 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f4588011590 con 0x7f45980b0420 2024-04-25T00:36:09.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.484+0000 7f458dffb640 1 --2- 172.21.15.29:0/1270380142 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f457c067c20 0x7f457c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.484+0000 7f458dffb640 1 -- 172.21.15.29:0/1270380142 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f45880864f0 con 0x7f45980b0420 2024-04-25T00:36:09.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.484+0000 7f459d7bc640 1 --2- 172.21.15.29:0/1270380142 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f457c067c20 0x7f457c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.484+0000 7f459d7bc640 1 --2- 172.21.15.29:0/1270380142 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f457c067c20 0x7f457c06a0e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f45940af000 tx=0x7f4594004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:09.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.488+0000 7f458dffb640 1 -- 172.21.15.29:0/1270380142 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4588015020 con 0x7f45980b0420 2024-04-25T00:36:09.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.648+0000 7f459fa20640 1 -- 172.21.15.29:0/1270380142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f45980b1440 con 0x7f45980b0420 2024-04-25T00:36:09.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.648+0000 7f458dffb640 1 -- 172.21.15.29:0/1270380142 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f4588056b10 con 0x7f45980b0420 2024-04-25T00:36:09.652 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:36:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.652+0000 7f459fa20640 1 -- 172.21.15.29:0/1270380142 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f457c067c20 msgr2=0x7f457c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.652+0000 7f459fa20640 1 --2- 172.21.15.29:0/1270380142 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f457c067c20 0x7f457c06a0e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f45940af000 tx=0x7f4594004740 comp rx=0 tx=0).stop 2024-04-25T00:36:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.652+0000 7f459fa20640 1 -- 172.21.15.29:0/1270380142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45980b0420 msgr2=0x7f45980cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.652+0000 7f459fa20640 1 --2- 172.21.15.29:0/1270380142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45980b0420 0x7f45980cb8c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f458800c7c0 tx=0x7f458800cc80 comp rx=0 tx=0).stop 2024-04-25T00:36:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.652+0000 7f459fa20640 1 -- 172.21.15.29:0/1270380142 shutdown_connections 2024-04-25T00:36:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.652+0000 7f459fa20640 1 --2- 172.21.15.29:0/1270380142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45980b0e60 0x7f459814c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.652+0000 7f459fa20640 1 --2- 172.21.15.29:0/1270380142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45980b0420 0x7f45980cb8c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.652+0000 7f459fa20640 1 --2- 172.21.15.29:0/1270380142 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f457c067c20 0x7f457c06a0e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.652+0000 7f459fa20640 1 --2- 172.21.15.29:0/1270380142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45980afb00 0x7f4598153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.652+0000 7f459fa20640 1 -- 172.21.15.29:0/1270380142 >> 172.21.15.29:0/1270380142 conn(0x7f4598072650 msgr2=0x7f45980acfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.652+0000 7f459fa20640 1 -- 172.21.15.29:0/1270380142 shutdown_connections 2024-04-25T00:36:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.652+0000 7f459fa20640 1 -- 172.21.15.29:0/1270380142 wait complete. 2024-04-25T00:36:09.671 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:36:09.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.784+0000 7f082e8d9640 1 Processor -- start 2024-04-25T00:36:09.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.784+0000 7f082e8d9640 1 -- start start 2024-04-25T00:36:09.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f082e8d9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f08280b3140 0x7f082806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f082e8d9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08280b3c30 0x7f08280ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f082e8d9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280b41e0 con 0x7f08280b4b10 2024-04-25T00:36:09.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f082e8d9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f082806d960 con 0x7f08280b3140 2024-04-25T00:36:09.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f082e8d9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f082806dc40 con 0x7f08280b3c30 2024-04-25T00:36:09.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f0827fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f08280b3140 0x7f082806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f08277fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f08280b4b10 0x7f08280b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59806/0 (socket says 172.21.15.29:59806) 2024-04-25T00:36:09.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f08277fe640 1 -- 172.21.15.29:0/7301036 learned_addr learned my addr 172.21.15.29:0/7301036 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:09.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f082ce76640 1 --2- 172.21.15.29:0/7301036 >> v2:172.21.15.157:3300/0 conn(0x7f08280b3c30 0x7f08280ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f0827fff640 1 -- 172.21.15.29:0/7301036 >> v2:172.21.15.157:3300/0 conn(0x7f08280b3c30 msgr2=0x7f08280ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:09.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f0827fff640 1 --2- 172.21.15.29:0/7301036 >> v2:172.21.15.157:3300/0 conn(0x7f08280b3c30 0x7f08280ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f0827fff640 1 -- 172.21.15.29:0/7301036 >> v1:172.21.15.29:6789/0 conn(0x7f08280b4b10 legacy=0x7f08280b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f0827fff640 1 -- 172.21.15.29:0/7301036 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082806df20 con 0x7f08280b3140 2024-04-25T00:36:09.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f0826ffd640 1 -- 172.21.15.29:0/7301036 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3307639144 0 0) 0x7f08280b41e0 con 0x7f08280b4b10 2024-04-25T00:36:09.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f0827fff640 1 --2- 172.21.15.29:0/7301036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08280b3140 0x7f082806e440 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f08180097c0 tx=0x7f081801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe24394f7ff8592 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:09.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f0826ffd640 1 -- 172.21.15.29:0/7301036 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08180b0070 con 0x7f08280b3140 2024-04-25T00:36:09.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f0826ffd640 1 -- 172.21.15.29:0/7301036 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08180ab600 con 0x7f08280b3140 2024-04-25T00:36:09.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.788+0000 7f0826ffd640 1 -- 172.21.15.29:0/7301036 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08180ab9d0 con 0x7f08280b3140 2024-04-25T00:36:09.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 -- 172.21.15.29:0/7301036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08280b3140 msgr2=0x7f082806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 --2- 172.21.15.29:0/7301036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08280b3140 0x7f082806e440 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f08180097c0 tx=0x7f081801c460 comp rx=0 tx=0).stop 2024-04-25T00:36:09.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 -- 172.21.15.29:0/7301036 shutdown_connections 2024-04-25T00:36:09.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 --2- 172.21.15.29:0/7301036 >> v2:172.21.15.157:3300/0 conn(0x7f08280b3c30 0x7f08280ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 --2- 172.21.15.29:0/7301036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08280b3140 0x7f082806e440 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 -- 172.21.15.29:0/7301036 >> 172.21.15.29:0/7301036 conn(0x7f08280b1870 msgr2=0x7f08280b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:09.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 -- 172.21.15.29:0/7301036 shutdown_connections 2024-04-25T00:36:09.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 -- 172.21.15.29:0/7301036 wait complete. 2024-04-25T00:36:09.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 Processor -- start 2024-04-25T00:36:09.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 -- start start 2024-04-25T00:36:09.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08280b3140 0x7f0828141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08280b3c30 0x7f082813a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08280b4b10 0x7f0828141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0828168d80 con 0x7f08280b3c30 2024-04-25T00:36:09.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0828168c00 con 0x7f08280b4b10 2024-04-25T00:36:09.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0828168f00 con 0x7f08280b3140 2024-04-25T00:36:09.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f08277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08280b3c30 0x7f082813a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f08277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08280b3c30 0x7f082813a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39668/0 (socket says 172.21.15.29:39668) 2024-04-25T00:36:09.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f08277fe640 1 -- 172.21.15.29:0/1795872608 learned_addr learned my addr 172.21.15.29:0/1795872608 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:09.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082ce76640 1 --2- 172.21.15.29:0/1795872608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08280b4b10 0x7f0828141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f08277fe640 1 -- 172.21.15.29:0/1795872608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08280b3140 msgr2=0x7f0828141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f08277fe640 1 --2- 172.21.15.29:0/1795872608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08280b3140 0x7f0828141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f08277fe640 1 -- 172.21.15.29:0/1795872608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08280b4b10 msgr2=0x7f0828141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f08277fe640 1 --2- 172.21.15.29:0/1795872608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08280b4b10 0x7f0828141d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f08277fe640 1 -- 172.21.15.29:0/1795872608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0828140220 con 0x7f08280b3c30 2024-04-25T00:36:09.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f08277fe640 1 --2- 172.21.15.29:0/1795872608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08280b3c30 0x7f082813a110 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f081c00c520 tx=0x7f081c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:09.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f0824ff9640 1 -- 172.21.15.29:0/1795872608 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c018070 con 0x7f08280b3c30 2024-04-25T00:36:09.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f0824ff9640 1 -- 172.21.15.29:0/1795872608 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f081c0074e0 con 0x7f08280b3c30 2024-04-25T00:36:09.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f0824ff9640 1 -- 172.21.15.29:0/1795872608 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c0049f0 con 0x7f08280b3c30 2024-04-25T00:36:09.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 -- 172.21.15.29:0/1795872608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08281404c0 con 0x7f08280b3c30 2024-04-25T00:36:09.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.792+0000 7f082e8d9640 1 -- 172.21.15.29:0/1795872608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0828140a80 con 0x7f08280b3c30 2024-04-25T00:36:09.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.796+0000 7f082e8d9640 1 -- 172.21.15.29:0/1795872608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0828069270 con 0x7f08280b3c30 2024-04-25T00:36:09.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.796+0000 7f0824ff9640 1 -- 172.21.15.29:0/1795872608 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f081c004b90 con 0x7f08280b3c30 2024-04-25T00:36:09.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.796+0000 7f0824ff9640 1 --2- 172.21.15.29:0/1795872608 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0814067c20 0x7f081406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:09.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.796+0000 7f0824ff9640 1 -- 172.21.15.29:0/1795872608 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f081c085490 con 0x7f08280b3c30 2024-04-25T00:36:09.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.796+0000 7f0827fff640 1 --2- 172.21.15.29:0/1795872608 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0814067c20 0x7f081406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:09.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.796+0000 7f0827fff640 1 --2- 172.21.15.29:0/1795872608 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0814067c20 0x7f081406a0e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f08180ad000 tx=0x7f0818008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:09.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.800+0000 7f0824ff9640 1 -- 172.21.15.29:0/1795872608 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f081c055500 con 0x7f08280b3c30 2024-04-25T00:36:09.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.956+0000 7f082e8d9640 1 -- 172.21.15.29:0/1795872608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0828072cd0 con 0x7f08280b3c30 2024-04-25T00:36:09.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.960+0000 7f0824ff9640 1 -- 172.21.15.29:0/1795872608 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) ==== 104+0+11 (secure 0 0 0) 0x7f081c059800 con 0x7f08280b3c30 2024-04-25T00:36:09.961 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:36:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.960+0000 7f082e8d9640 1 -- 172.21.15.29:0/1795872608 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0814067c20 msgr2=0x7f081406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.960+0000 7f082e8d9640 1 --2- 172.21.15.29:0/1795872608 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0814067c20 0x7f081406a0e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f08180ad000 tx=0x7f0818008040 comp rx=0 tx=0).stop 2024-04-25T00:36:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.960+0000 7f082e8d9640 1 -- 172.21.15.29:0/1795872608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08280b3c30 msgr2=0x7f082813a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.960+0000 7f082e8d9640 1 --2- 172.21.15.29:0/1795872608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08280b3c30 0x7f082813a110 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f081c00c520 tx=0x7f081c00c9e0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.960+0000 7f082e8d9640 1 -- 172.21.15.29:0/1795872608 shutdown_connections 2024-04-25T00:36:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.960+0000 7f082e8d9640 1 --2- 172.21.15.29:0/1795872608 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f0814067c20 0x7f081406a0e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.960+0000 7f082e8d9640 1 --2- 172.21.15.29:0/1795872608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08280b4b10 0x7f0828141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.960+0000 7f082e8d9640 1 --2- 172.21.15.29:0/1795872608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08280b3c30 0x7f082813a110 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.960+0000 7f082e8d9640 1 --2- 172.21.15.29:0/1795872608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08280b3140 0x7f0828141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:09.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.960+0000 7f082e8d9640 1 -- 172.21.15.29:0/1795872608 >> 172.21.15.29:0/1795872608 conn(0x7f08280b1870 msgr2=0x7f0828158c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.960+0000 7f082e8d9640 1 -- 172.21.15.29:0/1795872608 shutdown_connections 2024-04-25T00:36:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:09.960+0000 7f082e8d9640 1 -- 172.21.15.29:0/1795872608 wait complete. 2024-04-25T00:36:09.979 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:36:10.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd7b734640 1 Processor -- start 2024-04-25T00:36:10.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd7b734640 1 -- start start 2024-04-25T00:36:10.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd7b734640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd740d11f0 0x7fbd740cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd7b734640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd740ce1a0 con 0x7fbd74070060 2024-04-25T00:36:10.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd7b734640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd740d0150 con 0x7fbd740cddc0 2024-04-25T00:36:10.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd7b734640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbd740d0d60 con 0x7fbd740d11f0 2024-04-25T00:36:10.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd79cd1640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbd74070060 0x7fbd7415d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59820/0 (socket says 172.21.15.29:59820) 2024-04-25T00:36:10.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd79cd1640 1 -- 172.21.15.29:0/2045003070 learned_addr learned my addr 172.21.15.29:0/2045003070 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:10.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd78ccf640 1 --2- 172.21.15.29:0/2045003070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd740d11f0 0x7fbd740cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1251665380 0 0) 0x7fbd740ce1a0 con 0x7fbd74070060 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd60003530 con 0x7fbd74070060 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1706021146 0 0) 0x7fbd60003530 con 0x7fbd74070060 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd740d11f0 msgr2=0x7fbd740cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 --2- 172.21.15.29:0/2045003070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd740d11f0 0x7fbd740cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 >> v1:172.21.15.157:6789/0 conn(0x7fbd740cddc0 legacy=0x7fbd7406f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd740d0430 con 0x7fbd74070060 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fbd6c001a00 con 0x7fbd74070060 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 >> v1:172.21.15.29:6789/0 conn(0x7fbd74070060 legacy=0x7fbd7415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 --2- 172.21.15.29:0/2045003070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd740d11f0 0x7fbd600039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 --2- 172.21.15.29:0/2045003070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd60003f70 0x7fbd60004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 --2- 172.21.15.29:0/2045003070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd60004d50 0x7fbd600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbd740d0d60 con 0x7fbd60004d50 2024-04-25T00:36:10.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6000bb60 con 0x7fbd60003f70 2024-04-25T00:36:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6000bd60 con 0x7fbd740d11f0 2024-04-25T00:36:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd79cd1640 1 --2- 172.21.15.29:0/2045003070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd60004d50 0x7fbd600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd78ccf640 1 --2- 172.21.15.29:0/2045003070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd60003f70 0x7fbd60004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd79cd1640 1 -- 172.21.15.29:0/2045003070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd740d11f0 msgr2=0x7fbd600039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd79cd1640 1 --2- 172.21.15.29:0/2045003070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd740d11f0 0x7fbd600039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd79cd1640 1 -- 172.21.15.29:0/2045003070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd60003f70 msgr2=0x7fbd60004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd79cd1640 1 --2- 172.21.15.29:0/2045003070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd60003f70 0x7fbd60004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd79cd1640 1 -- 172.21.15.29:0/2045003070 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd60003530 con 0x7fbd60004d50 2024-04-25T00:36:10.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.096+0000 7fbd78ccf640 1 --2- 172.21.15.29:0/2045003070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd60003f70 0x7fbd60004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:10.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd79cd1640 1 --2- 172.21.15.29:0/2045003070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd60004d50 0x7fbd600051d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbd74072240 tx=0x7fbd6c01faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e1c27023c4f147 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:10.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00a4a0 con 0x7fbd60004d50 2024-04-25T00:36:10.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6c0aa6f0 con 0x7fbd60004d50 2024-04-25T00:36:10.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd6bfff640 1 -- 172.21.15.29:0/2045003070 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0aaae0 con 0x7fbd60004d50 2024-04-25T00:36:10.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd7b734640 1 -- 172.21.15.29:0/2045003070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd60004d50 msgr2=0x7fbd600051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd7b734640 1 --2- 172.21.15.29:0/2045003070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd60004d50 0x7fbd600051d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbd74072240 tx=0x7fbd6c01faa0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd79cd1640 1 -- 172.21.15.29:0/2045003070 reap_dead start 2024-04-25T00:36:10.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd7b734640 1 -- 172.21.15.29:0/2045003070 shutdown_connections 2024-04-25T00:36:10.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd7b734640 1 -- 172.21.15.29:0/2045003070 >> 172.21.15.29:0/2045003070 conn(0x7fbd740b3920 msgr2=0x7fbd740b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:10.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd7b734640 1 -- 172.21.15.29:0/2045003070 shutdown_connections 2024-04-25T00:36:10.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd7b734640 1 -- 172.21.15.29:0/2045003070 wait complete. 2024-04-25T00:36:10.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd7b734640 1 Processor -- start 2024-04-25T00:36:10.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd7b734640 1 -- start start 2024-04-25T00:36:10.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd7b734640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd60004d50 0x7fbd740b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.100+0000 7fbd7b734640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd7415b560 0x7fbd741f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd7b734640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd741f48a0 0x7fbd740b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd7b734640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7406f150 con 0x7fbd7415b560 2024-04-25T00:36:10.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd7b734640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd740d09f0 con 0x7fbd741f48a0 2024-04-25T00:36:10.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd7b734640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7406f2d0 con 0x7fbd60004d50 2024-04-25T00:36:10.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd78ccf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd7415b560 0x7fbd741f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd78ccf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd7415b560 0x7fbd741f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39694/0 (socket says 172.21.15.29:39694) 2024-04-25T00:36:10.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd78ccf640 1 -- 172.21.15.29:0/2283956572 learned_addr learned my addr 172.21.15.29:0/2283956572 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:10.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd79cd1640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd741f48a0 0x7fbd740b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd794d0640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd60004d50 0x7fbd740b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd78ccf640 1 -- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd60004d50 msgr2=0x7fbd740b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd78ccf640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd60004d50 0x7fbd740b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd78ccf640 1 -- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd741f48a0 msgr2=0x7fbd740b70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd78ccf640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd741f48a0 0x7fbd740b70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd78ccf640 1 -- 172.21.15.29:0/2283956572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd740b6090 con 0x7fbd7415b560 2024-04-25T00:36:10.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd78ccf640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd7415b560 0x7fbd741f4360 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbd640108c0 tx=0x7fbd64010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:10.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd69ffb640 1 -- 172.21.15.29:0/2283956572 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400f040 con 0x7fbd7415b560 2024-04-25T00:36:10.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd69ffb640 1 -- 172.21.15.29:0/2283956572 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd64004220 con 0x7fbd7415b560 2024-04-25T00:36:10.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd7b734640 1 -- 172.21.15.29:0/2283956572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd740c7040 con 0x7fbd7415b560 2024-04-25T00:36:10.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd69ffb640 1 -- 172.21.15.29:0/2283956572 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400e900 con 0x7fbd7415b560 2024-04-25T00:36:10.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd7b734640 1 -- 172.21.15.29:0/2283956572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd740b6610 con 0x7fbd7415b560 2024-04-25T00:36:10.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd7b734640 1 -- 172.21.15.29:0/2283956572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd74069270 con 0x7fbd7415b560 2024-04-25T00:36:10.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd69ffb640 1 -- 172.21.15.29:0/2283956572 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fbd6400eaa0 con 0x7fbd7415b560 2024-04-25T00:36:10.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd69ffb640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbd58067c20 0x7fbd5806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd69ffb640 1 -- 172.21.15.29:0/2283956572 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fbd64086100 con 0x7fbd7415b560 2024-04-25T00:36:10.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd794d0640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbd58067c20 0x7fbd5806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.104+0000 7fbd794d0640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbd58067c20 0x7fbd5806a0e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbd700015f0 tx=0x7fbd70009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:10.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.108+0000 7fbd69ffb640 1 -- 172.21.15.29:0/2283956572 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbd64018030 con 0x7fbd7415b560 2024-04-25T00:36:10.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd7b734640 1 -- 172.21.15.29:0/2283956572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbd740b4440 con 0x7fbd7415b560 2024-04-25T00:36:10.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd69ffb640 1 -- 172.21.15.29:0/2283956572 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) ==== 100+0+11 (secure 0 0 0) 0x7fbd640568c0 con 0x7fbd7415b560 2024-04-25T00:36:10.266 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:36:10.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd7b734640 1 -- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbd58067c20 msgr2=0x7fbd5806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd7b734640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbd58067c20 0x7fbd5806a0e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbd700015f0 tx=0x7fbd70009040 comp rx=0 tx=0).stop 2024-04-25T00:36:10.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd7b734640 1 -- 172.21.15.29:0/2283956572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd7415b560 msgr2=0x7fbd741f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd7b734640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd7415b560 0x7fbd741f4360 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbd640108c0 tx=0x7fbd64010d80 comp rx=0 tx=0).stop 2024-04-25T00:36:10.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd7b734640 1 -- 172.21.15.29:0/2283956572 shutdown_connections 2024-04-25T00:36:10.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd7b734640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fbd58067c20 0x7fbd5806a0e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd7b734640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd741f48a0 0x7fbd740b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd7b734640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd7415b560 0x7fbd741f4360 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd7b734640 1 --2- 172.21.15.29:0/2283956572 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd60004d50 0x7fbd740b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd7b734640 1 -- 172.21.15.29:0/2283956572 >> 172.21.15.29:0/2283956572 conn(0x7fbd740b3920 msgr2=0x7fbd7406dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:10.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd7b734640 1 -- 172.21.15.29:0/2283956572 shutdown_connections 2024-04-25T00:36:10.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.264+0000 7fbd7b734640 1 -- 172.21.15.29:0/2283956572 wait complete. 2024-04-25T00:36:10.283 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:36:10.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3eb56640 1 Processor -- start 2024-04-25T00:36:10.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3eb56640 1 -- start start 2024-04-25T00:36:10.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3eb56640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3815e200 0x7f7c380d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3eb56640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c380d2c60 0x7f7c380b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3eb56640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c380cfc10 con 0x7f7c380cf690 2024-04-25T00:36:10.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3eb56640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7c380b38d0 con 0x7f7c380d2c60 2024-04-25T00:36:10.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3eb56640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c380cfef0 con 0x7f7c3815e200 2024-04-25T00:36:10.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3d0f3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c380d2c60 0x7f7c380b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3d0f3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c380d2c60 0x7f7c380b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39700/0 (socket says 172.21.15.29:39700) 2024-04-25T00:36:10.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3d0f3640 1 -- 172.21.15.29:0/3177717186 learned_addr learned my addr 172.21.15.29:0/3177717186 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:10.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3c8f2640 1 --2- 172.21.15.29:0/3177717186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3815e200 0x7f7c380d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3d0f3640 1 -- 172.21.15.29:0/3177717186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3815e200 msgr2=0x7f7c380d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:10.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3d0f3640 1 --2- 172.21.15.29:0/3177717186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3815e200 0x7f7c380d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3d0f3640 1 -- 172.21.15.29:0/3177717186 >> v1:172.21.15.157:6789/0 conn(0x7f7c380cf690 legacy=0x7f7c380d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:10.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.396+0000 7f7c3d0f3640 1 -- 172.21.15.29:0/3177717186 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c380b3b20 con 0x7f7c380d2c60 2024-04-25T00:36:10.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3d0f3640 1 --2- 172.21.15.29:0/3177717186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c380d2c60 0x7f7c380b4320 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009e50 tx=0x7f7c3001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbadd04e31d92fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:10.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c377fe640 1 -- 172.21.15.29:0/3177717186 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c300a5860 con 0x7f7c380d2c60 2024-04-25T00:36:10.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c377fe640 1 -- 172.21.15.29:0/3177717186 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c300a5a00 con 0x7f7c380d2c60 2024-04-25T00:36:10.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c377fe640 1 -- 172.21.15.29:0/3177717186 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c300a5dd0 con 0x7f7c380d2c60 2024-04-25T00:36:10.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3177717186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c380d2c60 msgr2=0x7f7c380b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 --2- 172.21.15.29:0/3177717186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c380d2c60 0x7f7c380b4320 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009e50 tx=0x7f7c3001c9c0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3177717186 shutdown_connections 2024-04-25T00:36:10.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 --2- 172.21.15.29:0/3177717186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c380d2c60 0x7f7c380b4320 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 --2- 172.21.15.29:0/3177717186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3815e200 0x7f7c380d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3177717186 >> 172.21.15.29:0/3177717186 conn(0x7f7c3809d280 msgr2=0x7f7c380d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3177717186 shutdown_connections 2024-04-25T00:36:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3177717186 wait complete. 2024-04-25T00:36:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 Processor -- start 2024-04-25T00:36:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 -- start start 2024-04-25T00:36:10.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c380cf690 0x7f7c38155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c380d2c60 0x7f7c38156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3815e200 0x7f7c380b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7c380b51f0 con 0x7f7c380cf690 2024-04-25T00:36:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c380b5070 con 0x7f7c380d2c60 2024-04-25T00:36:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c380b5370 con 0x7f7c3815e200 2024-04-25T00:36:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3c8f2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c380cf690 0x7f7c38155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3c8f2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c380cf690 0x7f7c38155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39702/0 (socket says 172.21.15.29:39702) 2024-04-25T00:36:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3c8f2640 1 -- 172.21.15.29:0/3470701036 learned_addr learned my addr 172.21.15.29:0/3470701036 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3d0f3640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3815e200 0x7f7c380b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c37fff640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c380d2c60 0x7f7c38156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3c8f2640 1 -- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3815e200 msgr2=0x7f7c380b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3c8f2640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3815e200 0x7f7c380b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3c8f2640 1 -- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c380d2c60 msgr2=0x7f7c38156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3c8f2640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c380d2c60 0x7f7c38156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3c8f2640 1 -- 172.21.15.29:0/3470701036 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c38157a00 con 0x7f7c380cf690 2024-04-25T00:36:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3c8f2640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c380cf690 0x7f7c38155da0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00d800 tx=0x7f7c2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c357fa640 1 -- 172.21.15.29:0/3470701036 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c00b660 con 0x7f7c380cf690 2024-04-25T00:36:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c357fa640 1 -- 172.21.15.29:0/3470701036 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2c019070 con 0x7f7c380cf690 2024-04-25T00:36:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c357fa640 1 -- 172.21.15.29:0/3470701036 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c00ba00 con 0x7f7c380cf690 2024-04-25T00:36:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3470701036 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c380b8b80 con 0x7f7c380cf690 2024-04-25T00:36:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.400+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3470701036 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c380b9140 con 0x7f7c380cf690 2024-04-25T00:36:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.404+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3470701036 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c38069270 con 0x7f7c380cf690 2024-04-25T00:36:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.404+0000 7f7c357fa640 1 -- 172.21.15.29:0/3470701036 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f7c2c00bba0 con 0x7f7c380cf690 2024-04-25T00:36:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.404+0000 7f7c357fa640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c24067c20 0x7f7c2406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.404+0000 7f7c357fa640 1 -- 172.21.15.29:0/3470701036 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f7c2c086620 con 0x7f7c380cf690 2024-04-25T00:36:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.404+0000 7f7c37fff640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c24067c20 0x7f7c2406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.404+0000 7f7c37fff640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c24067c20 0x7f7c2406a0e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7c380b65c0 tx=0x7f7c28008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:10.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.408+0000 7f7c357fa640 1 -- 172.21.15.29:0/3470701036 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7c2c056d50 con 0x7f7c380cf690 2024-04-25T00:36:10.501 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:36:10.503 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:36:10.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.580+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3470701036 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7c380d1340 con 0x7f7c380cf690 2024-04-25T00:36:10.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.584+0000 7f7c357fa640 1 -- 172.21.15.29:0/3470701036 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7f7c2c056650 con 0x7f7c380cf690 2024-04-25T00:36:10.585 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:36:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.584+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c24067c20 msgr2=0x7f7c2406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.584+0000 7f7c3eb56640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c24067c20 0x7f7c2406a0e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7c380b65c0 tx=0x7f7c28008040 comp rx=0 tx=0).stop 2024-04-25T00:36:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.584+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3470701036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c380cf690 msgr2=0x7f7c38155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.584+0000 7f7c3eb56640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c380cf690 0x7f7c38155da0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00d800 tx=0x7f7c2c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.584+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3470701036 shutdown_connections 2024-04-25T00:36:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.584+0000 7f7c3eb56640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c3815e200 0x7f7c380b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.584+0000 7f7c3eb56640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7c24067c20 0x7f7c2406a0e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.584+0000 7f7c3eb56640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c380d2c60 0x7f7c38156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.584+0000 7f7c3eb56640 1 --2- 172.21.15.29:0/3470701036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c380cf690 0x7f7c38155da0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.584+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3470701036 >> 172.21.15.29:0/3470701036 conn(0x7f7c3809d280 msgr2=0x7f7c3806d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.584+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3470701036 shutdown_connections 2024-04-25T00:36:10.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.584+0000 7f7c3eb56640 1 -- 172.21.15.29:0/3470701036 wait complete. 2024-04-25T00:36:10.603 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-04-25T00:36:10.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:10.603 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:10.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[43867]: fuse finished with error 0 and tester_r 0 2024-04-25T00:36:10.658 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:36:10.691 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:36:10.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:10.692 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:36:10.737 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:36:10.737 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:36:10.744 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:36:10.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a82152640 1 Processor -- start 2024-04-25T00:36:10.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a82152640 1 -- start start 2024-04-25T00:36:10.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a82152640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9a7c15e8d0 0x7f9a7c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a82152640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a7c15f3c0 0x7f9a7c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a82152640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a7c0b3720 con 0x7f9a7c0b4780 2024-04-25T00:36:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a82152640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9a7c0b42f0 con 0x7f9a7c15e8d0 2024-04-25T00:36:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a82152640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9a7c0b39c0 con 0x7f9a7c15f3c0 2024-04-25T00:36:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a7b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9a7c15e8d0 0x7f9a7c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a73fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9a7c0b4780 0x7f9a7c15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59828/0 (socket says 172.21.15.29:59828) 2024-04-25T00:36:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a73fff640 1 -- 172.21.15.29:0/3910589168 learned_addr learned my addr 172.21.15.29:0/3910589168 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a7bfff640 1 --2- 172.21.15.29:0/3910589168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a7c15f3c0 0x7f9a7c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a7b7fe640 1 -- 172.21.15.29:0/3910589168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a7c15f3c0 msgr2=0x7f9a7c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a7b7fe640 1 --2- 172.21.15.29:0/3910589168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a7c15f3c0 0x7f9a7c15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a7b7fe640 1 -- 172.21.15.29:0/3910589168 >> v1:172.21.15.29:6789/0 conn(0x7f9a7c0b4780 legacy=0x7f9a7c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.908+0000 7f9a7b7fe640 1 -- 172.21.15.29:0/3910589168 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c0b3ca0 con 0x7f9a7c15e8d0 2024-04-25T00:36:10.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7b7fe640 1 --2- 172.21.15.29:0/3910589168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7c15e8d0 0x7f9a7c0cc240 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0097c0 tx=0x7f9a6c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51b1c0fb9361a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:10.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7affd640 1 -- 172.21.15.29:0/3910589168 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0b2070 con 0x7f9a7c15e8d0 2024-04-25T00:36:10.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7affd640 1 -- 172.21.15.29:0/3910589168 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a6c0ad680 con 0x7f9a7c15e8d0 2024-04-25T00:36:10.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7affd640 1 -- 172.21.15.29:0/3910589168 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0ada50 con 0x7f9a7c15e8d0 2024-04-25T00:36:10.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 -- 172.21.15.29:0/3910589168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7c15e8d0 msgr2=0x7f9a7c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 --2- 172.21.15.29:0/3910589168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7c15e8d0 0x7f9a7c0cc240 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0097c0 tx=0x7f9a6c01c470 comp rx=0 tx=0).stop 2024-04-25T00:36:10.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 -- 172.21.15.29:0/3910589168 shutdown_connections 2024-04-25T00:36:10.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 --2- 172.21.15.29:0/3910589168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a7c15f3c0 0x7f9a7c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 --2- 172.21.15.29:0/3910589168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7c15e8d0 0x7f9a7c0cc240 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 -- 172.21.15.29:0/3910589168 >> 172.21.15.29:0/3910589168 conn(0x7f9a7c0b1870 msgr2=0x7f9a7c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:10.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 -- 172.21.15.29:0/3910589168 shutdown_connections 2024-04-25T00:36:10.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 -- 172.21.15.29:0/3910589168 wait complete. 2024-04-25T00:36:10.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 Processor -- start 2024-04-25T00:36:10.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 -- start start 2024-04-25T00:36:10.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a7c0b4780 0x7f9a7c0ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a7c15e8d0 0x7f9a7c15b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7c15f3c0 0x7f9a7c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c15d6a0 con 0x7f9a7c15f3c0 2024-04-25T00:36:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c15d520 con 0x7f9a7c15e8d0 2024-04-25T00:36:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9a7c06d590 con 0x7f9a7c0b4780 2024-04-25T00:36:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7c15f3c0 0x7f9a7c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7c15f3c0 0x7f9a7c15bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39724/0 (socket says 172.21.15.29:39724) 2024-04-25T00:36:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7bfff640 1 -- 172.21.15.29:0/35543472 learned_addr learned my addr 172.21.15.29:0/35543472 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7b7fe640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a7c0b4780 0x7f9a7c0ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a73fff640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a7c15e8d0 0x7f9a7c15b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7bfff640 1 -- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a7c0b4780 msgr2=0x7f9a7c0ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7bfff640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a7c0b4780 0x7f9a7c0ae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7bfff640 1 -- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a7c15e8d0 msgr2=0x7f9a7c15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:10.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7bfff640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a7c15e8d0 0x7f9a7c15b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:10.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7bfff640 1 -- 172.21.15.29:0/35543472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c0723c0 con 0x7f9a7c15f3c0 2024-04-25T00:36:10.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a73fff640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a7c15e8d0 0x7f9a7c15b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:10.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a7bfff640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7c15f3c0 0x7f9a7c15bc90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9a74007e60 tx=0x7f9a7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:10.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a78ff9640 1 -- 172.21.15.29:0/35543472 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74014420 con 0x7f9a7c15f3c0 2024-04-25T00:36:10.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a78ff9640 1 -- 172.21.15.29:0/35543472 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a74019070 con 0x7f9a7c15f3c0 2024-04-25T00:36:10.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a82152640 1 -- 172.21.15.29:0/35543472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7c0aa180 con 0x7f9a7c15f3c0 2024-04-25T00:36:10.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.912+0000 7f9a78ff9640 1 -- 172.21.15.29:0/35543472 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74014820 con 0x7f9a7c15f3c0 2024-04-25T00:36:10.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.916+0000 7f9a82152640 1 -- 172.21.15.29:0/35543472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7c13b2c0 con 0x7f9a7c15f3c0 2024-04-25T00:36:10.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.916+0000 7f9a82152640 1 -- 172.21.15.29:0/35543472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a7c069270 con 0x7f9a7c15f3c0 2024-04-25T00:36:10.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.916+0000 7f9a78ff9640 1 -- 172.21.15.29:0/35543472 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f9a740149c0 con 0x7f9a7c15f3c0 2024-04-25T00:36:10.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.916+0000 7f9a78ff9640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9a60067c20 0x7f9a6006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:10.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.916+0000 7f9a78ff9640 1 -- 172.21.15.29:0/35543472 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f9a740869f0 con 0x7f9a7c15f3c0 2024-04-25T00:36:10.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.916+0000 7f9a7b7fe640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9a60067c20 0x7f9a6006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:10.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.916+0000 7f9a7b7fe640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9a60067c20 0x7f9a6006a0e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0af000 tx=0x7f9a6c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:10.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:10.920+0000 7f9a78ff9640 1 -- 172.21.15.29:0/35543472 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9a74057170 con 0x7f9a7c15f3c0 2024-04-25T00:36:11.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.084+0000 7f9a82152640 1 -- 172.21.15.29:0/35543472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a7c15c7d0 con 0x7f9a7c15f3c0 2024-04-25T00:36:11.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.088+0000 7f9a78ff9640 1 -- 172.21.15.29:0/35543472 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16620 (secure 0 0 0) 0x7f9a74056a70 con 0x7f9a7c15f3c0 2024-04-25T00:36:11.089 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:11.089 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":70,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:51.187302+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:47.559043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:48.530288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:36:11.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.088+0000 7f9a82152640 1 -- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9a60067c20 msgr2=0x7f9a6006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.088+0000 7f9a82152640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9a60067c20 0x7f9a6006a0e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0af000 tx=0x7f9a6c0a7000 comp rx=0 tx=0).stop 2024-04-25T00:36:11.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.088+0000 7f9a82152640 1 -- 172.21.15.29:0/35543472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7c15f3c0 msgr2=0x7f9a7c15bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.088+0000 7f9a82152640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7c15f3c0 0x7f9a7c15bc90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9a74007e60 tx=0x7f9a7400e4b0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.088+0000 7f9a82152640 1 -- 172.21.15.29:0/35543472 shutdown_connections 2024-04-25T00:36:11.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.088+0000 7f9a82152640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7c15f3c0 0x7f9a7c15bc90 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.088+0000 7f9a82152640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f9a60067c20 0x7f9a6006a0e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.088+0000 7f9a82152640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a7c15e8d0 0x7f9a7c15b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.088+0000 7f9a82152640 1 --2- 172.21.15.29:0/35543472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a7c0b4780 0x7f9a7c0ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.088+0000 7f9a82152640 1 -- 172.21.15.29:0/35543472 >> 172.21.15.29:0/35543472 conn(0x7f9a7c0b1870 msgr2=0x7f9a7c0b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:11.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.088+0000 7f9a82152640 1 -- 172.21.15.29:0/35543472 shutdown_connections 2024-04-25T00:36:11.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.088+0000 7f9a82152640 1 -- 172.21.15.29:0/35543472 wait complete. 2024-04-25T00:36:11.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:35:47.559043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:35:48.530288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:36:11.109 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:36:11.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37f3232640 1 Processor -- start 2024-04-25T00:36:11.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37f3232640 1 -- start start 2024-04-25T00:36:11.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37f3232640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37ec0b3140 0x7f37ec06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37f3232640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f37ec0b4b10 0x7f37ec0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37f3232640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37ec0b41e0 con 0x7f37ec0b3a60 2024-04-25T00:36:11.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37f3232640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f37ec06d960 con 0x7f37ec0b4b10 2024-04-25T00:36:11.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37f3232640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f37ec06dc40 con 0x7f37ec0b3140 2024-04-25T00:36:11.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37ebfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f37ec0b4b10 0x7f37ec0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37ebfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f37ec0b4b10 0x7f37ec0b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39740/0 (socket says 172.21.15.29:39740) 2024-04-25T00:36:11.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37ebfff640 1 -- 172.21.15.29:0/816361475 learned_addr learned my addr 172.21.15.29:0/816361475 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37f17cf640 1 --1- 172.21.15.29:0/816361475 >> v1:172.21.15.29:6789/0 conn(0x7f37ec0b3a60 0x7f37ec15c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59842/0 (socket says 172.21.15.29:59842) 2024-04-25T00:36:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37f0fce640 1 --2- 172.21.15.29:0/816361475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37ec0b3140 0x7f37ec06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37ebfff640 1 -- 172.21.15.29:0/816361475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37ec0b3140 msgr2=0x7f37ec06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37ebfff640 1 --2- 172.21.15.29:0/816361475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37ec0b3140 0x7f37ec06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37ebfff640 1 -- 172.21.15.29:0/816361475 >> v1:172.21.15.29:6789/0 conn(0x7f37ec0b3a60 legacy=0x7f37ec15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37ebfff640 1 -- 172.21.15.29:0/816361475 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec06df20 con 0x7f37ec0b4b10 2024-04-25T00:36:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37eb7fe640 1 -- 172.21.15.29:0/816361475 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 250921290 0 0) 0x7f37ec0b41e0 con 0x7f37ec0b3a60 2024-04-25T00:36:11.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.236+0000 7f37f0fce640 1 --2- 172.21.15.29:0/816361475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37ec0b3140 0x7f37ec06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:11.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37ebfff640 1 --2- 172.21.15.29:0/816361475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0b4b10 0x7f37ec0b3520 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009870 tx=0x7f37e001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac74a71b8b53eab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37eb7fe640 1 -- 172.21.15.29:0/816361475 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e00b0070 con 0x7f37ec0b4b10 2024-04-25T00:36:11.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37eb7fe640 1 -- 172.21.15.29:0/816361475 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e00ab600 con 0x7f37ec0b4b10 2024-04-25T00:36:11.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37eb7fe640 1 -- 172.21.15.29:0/816361475 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e00ab9d0 con 0x7f37ec0b4b10 2024-04-25T00:36:11.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37f3232640 1 -- 172.21.15.29:0/816361475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0b4b10 msgr2=0x7f37ec0b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37f3232640 1 --2- 172.21.15.29:0/816361475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0b4b10 0x7f37ec0b3520 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009870 tx=0x7f37e001c560 comp rx=0 tx=0).stop 2024-04-25T00:36:11.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37f3232640 1 -- 172.21.15.29:0/816361475 shutdown_connections 2024-04-25T00:36:11.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37f3232640 1 --2- 172.21.15.29:0/816361475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0b4b10 0x7f37ec0b3520 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37f3232640 1 --2- 172.21.15.29:0/816361475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37ec0b3140 0x7f37ec06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37f3232640 1 -- 172.21.15.29:0/816361475 >> 172.21.15.29:0/816361475 conn(0x7f37ec0b1870 msgr2=0x7f37ec0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:11.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37f3232640 1 -- 172.21.15.29:0/816361475 shutdown_connections 2024-04-25T00:36:11.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37f3232640 1 -- 172.21.15.29:0/816361475 wait complete. 2024-04-25T00:36:11.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37f3232640 1 Processor -- start 2024-04-25T00:36:11.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.240+0000 7f37f3232640 1 -- start start 2024-04-25T00:36:11.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37f3232640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37ec0b3140 0x7f37ec141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37f3232640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0b3a60 0x7f37ec15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37f3232640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37ec0b4b10 0x7f37ec1401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37f3232640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec168b70 con 0x7f37ec0b3a60 2024-04-25T00:36:11.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37f3232640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1689f0 con 0x7f37ec0b3140 2024-04-25T00:36:11.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37f3232640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f37ec168cf0 con 0x7f37ec0b4b10 2024-04-25T00:36:11.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37ebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0b3a60 0x7f37ec15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37ebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0b3a60 0x7f37ec15b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39742/0 (socket says 172.21.15.29:39742) 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37ebfff640 1 -- 172.21.15.29:0/1790522304 learned_addr learned my addr 172.21.15.29:0/1790522304 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37f17cf640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37ec0b4b10 0x7f37ec1401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37f0fce640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37ec0b3140 0x7f37ec141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37ebfff640 1 -- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37ec0b4b10 msgr2=0x7f37ec1401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37ebfff640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37ec0b4b10 0x7f37ec1401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37ebfff640 1 -- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37ec0b3140 msgr2=0x7f37ec141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37ebfff640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37ec0b3140 0x7f37ec141b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37ebfff640 1 -- 172.21.15.29:0/1790522304 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec142080 con 0x7f37ec0b3a60 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37f0fce640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37ec0b3140 0x7f37ec141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37ebfff640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0b3a60 0x7f37ec15b870 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f37e00ae040 tx=0x7f37e0019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37e97fa640 1 -- 172.21.15.29:0/1790522304 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e00b0050 con 0x7f37ec0b3a60 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37e97fa640 1 -- 172.21.15.29:0/1790522304 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e00bc070 con 0x7f37ec0b3a60 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37e97fa640 1 -- 172.21.15.29:0/1790522304 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e00b5a90 con 0x7f37ec0b3a60 2024-04-25T00:36:11.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37f3232640 1 -- 172.21.15.29:0/1790522304 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec140aa0 con 0x7f37ec0b3a60 2024-04-25T00:36:11.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37f3232640 1 -- 172.21.15.29:0/1790522304 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec141060 con 0x7f37ec0b3a60 2024-04-25T00:36:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37f3232640 1 -- 172.21.15.29:0/1790522304 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ec069270 con 0x7f37ec0b3a60 2024-04-25T00:36:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37e97fa640 1 -- 172.21.15.29:0/1790522304 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f37e00b5c30 con 0x7f37ec0b3a60 2024-04-25T00:36:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.244+0000 7f37e97fa640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37d8067c20 0x7f37d806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.248+0000 7f37e97fa640 1 -- 172.21.15.29:0/1790522304 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f37e0120ab0 con 0x7f37ec0b3a60 2024-04-25T00:36:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.248+0000 7f37f0fce640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37d8067c20 0x7f37d806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.248+0000 7f37f0fce640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37d8067c20 0x7f37d806a0e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0097a0 tx=0x7f37dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.252+0000 7f37e97fa640 1 -- 172.21.15.29:0/1790522304 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f37e0016150 con 0x7f37ec0b3a60 2024-04-25T00:36:11.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.412+0000 7f37f3232640 1 -- 172.21.15.29:0/1790522304 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f37ec06d230 con 0x7f37ec0b3a60 2024-04-25T00:36:11.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.412+0000 7f37e97fa640 1 -- 172.21.15.29:0/1790522304 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f37e00f11b0 con 0x7f37ec0b3a60 2024-04-25T00:36:11.415 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:36:11.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.416+0000 7f37f3232640 1 -- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37d8067c20 msgr2=0x7f37d806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.416+0000 7f37f3232640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37d8067c20 0x7f37d806a0e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0097a0 tx=0x7f37dc008040 comp rx=0 tx=0).stop 2024-04-25T00:36:11.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.416+0000 7f37f3232640 1 -- 172.21.15.29:0/1790522304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0b3a60 msgr2=0x7f37ec15b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.416+0000 7f37f3232640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0b3a60 0x7f37ec15b870 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f37e00ae040 tx=0x7f37e0019300 comp rx=0 tx=0).stop 2024-04-25T00:36:11.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.416+0000 7f37f3232640 1 -- 172.21.15.29:0/1790522304 shutdown_connections 2024-04-25T00:36:11.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.416+0000 7f37f3232640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37ec0b4b10 0x7f37ec1401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.416+0000 7f37f3232640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0b3a60 0x7f37ec15b870 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.416+0000 7f37f3232640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f37d8067c20 0x7f37d806a0e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.416+0000 7f37f3232640 1 --2- 172.21.15.29:0/1790522304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37ec0b3140 0x7f37ec141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.416+0000 7f37f3232640 1 -- 172.21.15.29:0/1790522304 >> 172.21.15.29:0/1790522304 conn(0x7f37ec0b1870 msgr2=0x7f37ec0cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:11.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.416+0000 7f37f3232640 1 -- 172.21.15.29:0/1790522304 shutdown_connections 2024-04-25T00:36:11.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.416+0000 7f37f3232640 1 -- 172.21.15.29:0/1790522304 wait complete. 2024-04-25T00:36:11.434 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:36:11.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9fbaf640 1 Processor -- start 2024-04-25T00:36:11.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9fbaf640 1 -- start start 2024-04-25T00:36:11.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9fbaf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d9815c420 0x7f7d98071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9fbaf640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9815e850 con 0x7f7d980722c0 2024-04-25T00:36:11.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9fbaf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9815eaf0 con 0x7f7d9815e060 2024-04-25T00:36:11.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9fbaf640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7d98072710 con 0x7f7d9815c420 2024-04-25T00:36:11.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9d14a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d9815c420 0x7f7d98071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9d14a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d9815c420 0x7f7d98071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39746/0 (socket says 172.21.15.29:39746) 2024-04-25T00:36:11.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9e14c640 1 --1- 172.21.15.29:0/2757567360 >> v1:172.21.15.29:6789/0 conn(0x7f7d980722c0 0x7f7d980cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59848/0 (socket says 172.21.15.29:59848) 2024-04-25T00:36:11.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9d14a640 1 -- 172.21.15.29:0/2757567360 learned_addr learned my addr 172.21.15.29:0/2757567360 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:11.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9d14a640 1 -- 172.21.15.29:0/2757567360 >> v1:172.21.15.157:6789/0 conn(0x7f7d9815e060 legacy=0x7f7d9815bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:11.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9d14a640 1 -- 172.21.15.29:0/2757567360 >> v1:172.21.15.29:6789/0 conn(0x7f7d980722c0 legacy=0x7f7d980cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9d14a640 1 -- 172.21.15.29:0/2757567360 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9815edd0 con 0x7f7d9815c420 2024-04-25T00:36:11.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.560+0000 7f7d9c949640 1 -- 172.21.15.29:0/2757567360 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3471653441 0 0) 0x7f7d9815e850 con 0x7f7d980722c0 2024-04-25T00:36:11.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9d14a640 1 --2- 172.21.15.29:0/2757567360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d9815c420 0x7f7d98071d80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7d98067e50 tx=0x7f7d9401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75f3da09e13aac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9c949640 1 -- 172.21.15.29:0/2757567360 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d940b1070 con 0x7f7d9815c420 2024-04-25T00:36:11.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9c949640 1 -- 172.21.15.29:0/2757567360 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d940ac600 con 0x7f7d9815c420 2024-04-25T00:36:11.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9c949640 1 -- 172.21.15.29:0/2757567360 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d940ac9d0 con 0x7f7d9815c420 2024-04-25T00:36:11.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2757567360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d9815c420 msgr2=0x7f7d98071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 --2- 172.21.15.29:0/2757567360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d9815c420 0x7f7d98071d80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7d98067e50 tx=0x7f7d9401c4b0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2757567360 shutdown_connections 2024-04-25T00:36:11.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 --2- 172.21.15.29:0/2757567360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d9815c420 0x7f7d98071d80 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2757567360 >> 172.21.15.29:0/2757567360 conn(0x7f7d9806c320 msgr2=0x7f7d9806c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:11.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2757567360 shutdown_connections 2024-04-25T00:36:11.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2757567360 wait complete. 2024-04-25T00:36:11.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 Processor -- start 2024-04-25T00:36:11.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 -- start start 2024-04-25T00:36:11.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d980722c0 0x7f7d98150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9815c420 0x7f7d980ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d9815e060 0x7f7d98159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9806f790 con 0x7f7d9815e060 2024-04-25T00:36:11.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9806f610 con 0x7f7d980722c0 2024-04-25T00:36:11.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9fbaf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7d9806b210 con 0x7f7d9815c420 2024-04-25T00:36:11.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9e14c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d9815e060 0x7f7d98159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9e14c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d9815e060 0x7f7d98159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39762/0 (socket says 172.21.15.29:39762) 2024-04-25T00:36:11.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9e14c640 1 -- 172.21.15.29:0/2706884283 learned_addr learned my addr 172.21.15.29:0/2706884283 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:11.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9d14a640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9815c420 0x7f7d980ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9d94b640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d980722c0 0x7f7d98150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9e14c640 1 -- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9815c420 msgr2=0x7f7d980ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9e14c640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9815c420 0x7f7d980ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9e14c640 1 -- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d980722c0 msgr2=0x7f7d98150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9e14c640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d980722c0 0x7f7d98150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9e14c640 1 -- 172.21.15.29:0/2706884283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d981509c0 con 0x7f7d9815e060 2024-04-25T00:36:11.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.564+0000 7f7d9d14a640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9815c420 0x7f7d980ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:11.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.568+0000 7f7d9e14c640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d9815e060 0x7f7d98159d30 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c7a0 tx=0x7f7d9000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.568+0000 7f7d8e7fc640 1 -- 172.21.15.29:0/2706884283 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90018070 con 0x7f7d9815e060 2024-04-25T00:36:11.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.568+0000 7f7d8e7fc640 1 -- 172.21.15.29:0/2706884283 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9000a640 con 0x7f7d9815e060 2024-04-25T00:36:11.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.568+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2706884283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d980b5b70 con 0x7f7d9815e060 2024-04-25T00:36:11.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.568+0000 7f7d8e7fc640 1 -- 172.21.15.29:0/2706884283 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000aab0 con 0x7f7d9815e060 2024-04-25T00:36:11.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.568+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2706884283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d980b6130 con 0x7f7d9815e060 2024-04-25T00:36:11.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.568+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2706884283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d98069270 con 0x7f7d9815e060 2024-04-25T00:36:11.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.568+0000 7f7d8e7fc640 1 -- 172.21.15.29:0/2706884283 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f7d9001b070 con 0x7f7d9815e060 2024-04-25T00:36:11.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.568+0000 7f7d8e7fc640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7d7c067c50 0x7f7d7c06a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.568+0000 7f7d8e7fc640 1 -- 172.21.15.29:0/2706884283 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f7d900857e0 con 0x7f7d9815e060 2024-04-25T00:36:11.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.568+0000 7f7d9d94b640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7d7c067c50 0x7f7d7c06a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.568+0000 7f7d9d94b640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7d7c067c50 0x7f7d7c06a110 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800a750 tx=0x7f7d88008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.572+0000 7f7d8e7fc640 1 -- 172.21.15.29:0/2706884283 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7d90055ea0 con 0x7f7d9815e060 2024-04-25T00:36:11.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.736+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2706884283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d9806e070 con 0x7f7d9815e060 2024-04-25T00:36:11.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.736+0000 7f7d8e7fc640 1 -- 172.21.15.29:0/2706884283 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) ==== 104+0+11 (secure 0 0 0) 0x7f7d900557a0 con 0x7f7d9815e060 2024-04-25T00:36:11.738 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:36:11.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.740+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7d7c067c50 msgr2=0x7f7d7c06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.740+0000 7f7d9fbaf640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7d7c067c50 0x7f7d7c06a110 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800a750 tx=0x7f7d88008040 comp rx=0 tx=0).stop 2024-04-25T00:36:11.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.740+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2706884283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d9815e060 msgr2=0x7f7d98159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.740+0000 7f7d9fbaf640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d9815e060 0x7f7d98159d30 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c7a0 tx=0x7f7d9000cc60 comp rx=0 tx=0).stop 2024-04-25T00:36:11.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.740+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2706884283 shutdown_connections 2024-04-25T00:36:11.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.740+0000 7f7d9fbaf640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d9815e060 0x7f7d98159d30 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.740+0000 7f7d9fbaf640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d9815c420 0x7f7d980ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.740+0000 7f7d9fbaf640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f7d7c067c50 0x7f7d7c06a110 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.740+0000 7f7d9fbaf640 1 --2- 172.21.15.29:0/2706884283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d980722c0 0x7f7d98150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.740+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2706884283 >> 172.21.15.29:0/2706884283 conn(0x7f7d9806c320 msgr2=0x7f7d9815c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:11.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.740+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2706884283 shutdown_connections 2024-04-25T00:36:11.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.740+0000 7f7d9fbaf640 1 -- 172.21.15.29:0/2706884283 wait complete. 2024-04-25T00:36:11.758 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:36:11.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed23e05640 1 Processor -- start 2024-04-25T00:36:11.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed23e05640 1 -- start start 2024-04-25T00:36:11.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed23e05640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed1c0b3140 0x7fed1c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed23e05640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fed1c0b3c30 0x7fed1c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed23e05640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed1c0b41e0 con 0x7fed1c0b4b10 2024-04-25T00:36:11.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed23e05640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fed1c06d960 con 0x7fed1c0b3c30 2024-04-25T00:36:11.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed23e05640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fed1c06dc40 con 0x7fed1c0b3140 2024-04-25T00:36:11.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed21ba1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed1c0b3140 0x7fed1c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed223a2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fed1c0b3c30 0x7fed1c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed21ba1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed1c0b3140 0x7fed1c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:41386/0 (socket says 172.21.15.29:41386) 2024-04-25T00:36:11.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed21ba1640 1 -- 172.21.15.29:0/3760652116 learned_addr learned my addr 172.21.15.29:0/3760652116 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:11.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed223a2640 1 -- 172.21.15.29:0/3760652116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed1c0b3140 msgr2=0x7fed1c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:11.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed223a2640 1 --2- 172.21.15.29:0/3760652116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed1c0b3140 0x7fed1c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed223a2640 1 -- 172.21.15.29:0/3760652116 >> v1:172.21.15.157:6789/0 conn(0x7fed1c0b4b10 legacy=0x7fed1c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed223a2640 1 -- 172.21.15.29:0/3760652116 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c06df20 con 0x7fed1c0b3c30 2024-04-25T00:36:11.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.888+0000 7fed21ba1640 1 --2- 172.21.15.29:0/3760652116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed1c0b3140 0x7fed1c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:36:11.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.892+0000 7fed223a2640 1 --2- 172.21.15.29:0/3760652116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed1c0b3c30 0x7fed1c0ce240 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fed14009d90 tx=0x7fed1401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1963a8be56289e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.892+0000 7fed20b9f640 1 -- 172.21.15.29:0/3760652116 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed140b2070 con 0x7fed1c0b3c30 2024-04-25T00:36:11.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.892+0000 7fed20b9f640 1 -- 172.21.15.29:0/3760652116 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed140a5830 con 0x7fed1c0b3c30 2024-04-25T00:36:11.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.892+0000 7fed20b9f640 1 -- 172.21.15.29:0/3760652116 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed140a5c00 con 0x7fed1c0b3c30 2024-04-25T00:36:11.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.892+0000 7fed23e05640 1 -- 172.21.15.29:0/3760652116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed1c0b3c30 msgr2=0x7fed1c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.892+0000 7fed23e05640 1 --2- 172.21.15.29:0/3760652116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed1c0b3c30 0x7fed1c0ce240 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fed14009d90 tx=0x7fed1401c970 comp rx=0 tx=0).stop 2024-04-25T00:36:11.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.892+0000 7fed23e05640 1 -- 172.21.15.29:0/3760652116 shutdown_connections 2024-04-25T00:36:11.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.892+0000 7fed23e05640 1 --2- 172.21.15.29:0/3760652116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed1c0b3c30 0x7fed1c0ce240 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.892+0000 7fed23e05640 1 --2- 172.21.15.29:0/3760652116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed1c0b3140 0x7fed1c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.892+0000 7fed23e05640 1 -- 172.21.15.29:0/3760652116 >> 172.21.15.29:0/3760652116 conn(0x7fed1c0b1870 msgr2=0x7fed1c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:11.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.892+0000 7fed23e05640 1 -- 172.21.15.29:0/3760652116 shutdown_connections 2024-04-25T00:36:11.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.892+0000 7fed23e05640 1 -- 172.21.15.29:0/3760652116 wait complete. 2024-04-25T00:36:11.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed23e05640 1 Processor -- start 2024-04-25T00:36:11.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed23e05640 1 -- start start 2024-04-25T00:36:11.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed23e05640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed1c0b3140 0x7fed1c13d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed23e05640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed1c0b3c30 0x7fed1c1426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed23e05640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed1c0b4b10 0x7fed1c0c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed23e05640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c168d80 con 0x7fed1c0b3140 2024-04-25T00:36:11.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed23e05640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c168c00 con 0x7fed1c0b4b10 2024-04-25T00:36:11.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed23e05640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fed1c168f00 con 0x7fed1c0b3c30 2024-04-25T00:36:11.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed21ba1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed1c0b3140 0x7fed1c13d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed21ba1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed1c0b3140 0x7fed1c13d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39774/0 (socket says 172.21.15.29:39774) 2024-04-25T00:36:11.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed21ba1640 1 -- 172.21.15.29:0/3712507230 learned_addr learned my addr 172.21.15.29:0/3712507230 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:11.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed223a2640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed1c0b4b10 0x7fed1c0c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed213a0640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed1c0b3c30 0x7fed1c1426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed21ba1640 1 -- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed1c0b3c30 msgr2=0x7fed1c1426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed21ba1640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed1c0b3c30 0x7fed1c1426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed21ba1640 1 -- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed1c0b4b10 msgr2=0x7fed1c0c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed21ba1640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed1c0b4b10 0x7fed1c0c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed21ba1640 1 -- 172.21.15.29:0/3712507230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c13dc60 con 0x7fed1c0b3140 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed213a0640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed1c0b3c30 0x7fed1c1426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed223a2640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed1c0b4b10 0x7fed1c0c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed21ba1640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed1c0b3140 0x7fed1c13d720 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fed1800e960 tx=0x7fed1800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed127fc640 1 -- 172.21.15.29:0/3712507230 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18017070 con 0x7fed1c0b3140 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed127fc640 1 -- 172.21.15.29:0/3712507230 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1800ca00 con 0x7fed1c0b3140 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed23e05640 1 -- 172.21.15.29:0/3712507230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1c142be0 con 0x7fed1c0b3140 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed127fc640 1 -- 172.21.15.29:0/3712507230 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1800cdb0 con 0x7fed1c0b3140 2024-04-25T00:36:11.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed23e05640 1 -- 172.21.15.29:0/3712507230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1c143160 con 0x7fed1c0b3140 2024-04-25T00:36:11.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed23e05640 1 -- 172.21.15.29:0/3712507230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed1c069270 con 0x7fed1c0b3140 2024-04-25T00:36:11.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.896+0000 7fed127fc640 1 -- 172.21.15.29:0/3712507230 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fed1801a070 con 0x7fed1c0b3140 2024-04-25T00:36:11.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.900+0000 7fed127fc640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fed00067c20 0x7fed0006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:11.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.900+0000 7fed127fc640 1 -- 172.21.15.29:0/3712507230 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fed180860f0 con 0x7fed1c0b3140 2024-04-25T00:36:11.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.900+0000 7fed213a0640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fed00067c20 0x7fed0006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:11.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.900+0000 7fed213a0640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fed00067c20 0x7fed0006a0e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fed1c15b6d0 tx=0x7fed0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:11.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:11.904+0000 7fed127fc640 1 -- 172.21.15.29:0/3712507230 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fed18056820 con 0x7fed1c0b3140 2024-04-25T00:36:12.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.064+0000 7fed23e05640 1 -- 172.21.15.29:0/3712507230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fed1c0af5a0 con 0x7fed1c0b3140 2024-04-25T00:36:12.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.064+0000 7fed127fc640 1 -- 172.21.15.29:0/3712507230 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) ==== 100+0+11 (secure 0 0 0) 0x7fed18056120 con 0x7fed1c0b3140 2024-04-25T00:36:12.066 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:36:12.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.068+0000 7fed23e05640 1 -- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fed00067c20 msgr2=0x7fed0006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.068+0000 7fed23e05640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fed00067c20 0x7fed0006a0e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fed1c15b6d0 tx=0x7fed0c009040 comp rx=0 tx=0).stop 2024-04-25T00:36:12.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.068+0000 7fed23e05640 1 -- 172.21.15.29:0/3712507230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed1c0b3140 msgr2=0x7fed1c13d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.068+0000 7fed23e05640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed1c0b3140 0x7fed1c13d720 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fed1800e960 tx=0x7fed1800ee20 comp rx=0 tx=0).stop 2024-04-25T00:36:12.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.068+0000 7fed23e05640 1 -- 172.21.15.29:0/3712507230 shutdown_connections 2024-04-25T00:36:12.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.068+0000 7fed23e05640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fed00067c20 0x7fed0006a0e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.068+0000 7fed23e05640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed1c0b4b10 0x7fed1c0c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.068+0000 7fed23e05640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed1c0b3c30 0x7fed1c1426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.068+0000 7fed23e05640 1 --2- 172.21.15.29:0/3712507230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed1c0b3140 0x7fed1c13d720 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.068+0000 7fed23e05640 1 -- 172.21.15.29:0/3712507230 >> 172.21.15.29:0/3712507230 conn(0x7fed1c0b1870 msgr2=0x7fed1c13dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:12.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.068+0000 7fed23e05640 1 -- 172.21.15.29:0/3712507230 shutdown_connections 2024-04-25T00:36:12.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.068+0000 7fed23e05640 1 -- 172.21.15.29:0/3712507230 wait complete. 2024-04-25T00:36:12.086 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:36:12.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.204+0000 7fcddcc0b640 1 Processor -- start 2024-04-25T00:36:12.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.204+0000 7fcddcc0b640 1 -- start start 2024-04-25T00:36:12.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.204+0000 7fcddcc0b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcdd80d0390 0x7fcdd815b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcddcc0b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd815bc30 con 0x7fcdd815b640 2024-04-25T00:36:12.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcddcc0b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd815bf10 con 0x7fcdd815ad20 2024-04-25T00:36:12.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcddcc0b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd80cb6a0 con 0x7fcdd80d0390 2024-04-25T00:36:12.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd6d9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcdd815b640 0x7fcdd8071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59852/0 (socket says 172.21.15.29:59852) 2024-04-25T00:36:12.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd6d9d640 1 -- 172.21.15.29:0/380016348 learned_addr learned my addr 172.21.15.29:0/380016348 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd5d9b640 1 --2- 172.21.15.29:0/380016348 >> v2:172.21.15.157:3300/0 conn(0x7fcdd80d0390 0x7fcdd815b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3336939373 0 0) 0x7fcdd815bc30 con 0x7fcdd815b640 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdc0003530 con 0x7fcdd815b640 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1228914394 0 0) 0x7fcdc0003530 con 0x7fcdd815b640 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 >> v2:172.21.15.157:3300/0 conn(0x7fcdd80d0390 msgr2=0x7fcdd815b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 --2- 172.21.15.29:0/380016348 >> v2:172.21.15.157:3300/0 conn(0x7fcdd80d0390 0x7fcdd815b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 >> v1:172.21.15.157:6789/0 conn(0x7fcdd815ad20 legacy=0x7fcdd80cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd815c1f0 con 0x7fcdd815b640 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fcdc8001680 con 0x7fcdd815b640 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 >> v1:172.21.15.29:6789/0 conn(0x7fcdd815b640 legacy=0x7fcdd8071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 --2- 172.21.15.29:0/380016348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdc00039c0 0x7fcdc0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 --2- 172.21.15.29:0/380016348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdc0004810 0x7fcdc0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 --2- 172.21.15.29:0/380016348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc00051d0 0x7fcdc000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd80cb6a0 con 0x7fcdc00039c0 2024-04-25T00:36:12.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc000c0a0 con 0x7fcdc00051d0 2024-04-25T00:36:12.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcdc000c2a0 con 0x7fcdc0004810 2024-04-25T00:36:12.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd659c640 1 --2- 172.21.15.29:0/380016348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdc00039c0 0x7fcdc0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd6d9d640 1 --2- 172.21.15.29:0/380016348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc00051d0 0x7fcdc000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd5d9b640 1 --2- 172.21.15.29:0/380016348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdc0004810 0x7fcdc0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd659c640 1 -- 172.21.15.29:0/380016348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdc0004810 msgr2=0x7fcdc0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:12.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd659c640 1 --2- 172.21.15.29:0/380016348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdc0004810 0x7fcdc0004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd659c640 1 -- 172.21.15.29:0/380016348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc00051d0 msgr2=0x7fcdc000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:12.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd659c640 1 --2- 172.21.15.29:0/380016348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc00051d0 0x7fcdc000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd659c640 1 -- 172.21.15.29:0/380016348 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc0003530 con 0x7fcdc00039c0 2024-04-25T00:36:12.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd6d9d640 1 -- 172.21.15.29:0/380016348 reap_dead start 2024-04-25T00:36:12.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd659c640 1 --2- 172.21.15.29:0/380016348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdc00039c0 0x7fcdc0003e00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0001e70 tx=0x7fcdd001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5df6b7ae698d0e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:12.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd00b0070 con 0x7fcdc00039c0 2024-04-25T00:36:12.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd00a7070 con 0x7fcdc00039c0 2024-04-25T00:36:12.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.208+0000 7fcdd559a640 1 -- 172.21.15.29:0/380016348 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0001a90 con 0x7fcdc00039c0 2024-04-25T00:36:12.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 -- 172.21.15.29:0/380016348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdc00039c0 msgr2=0x7fcdc0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 --2- 172.21.15.29:0/380016348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdc00039c0 0x7fcdc0003e00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0001e70 tx=0x7fcdd001c760 comp rx=0 tx=0).stop 2024-04-25T00:36:12.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 -- 172.21.15.29:0/380016348 shutdown_connections 2024-04-25T00:36:12.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 --2- 172.21.15.29:0/380016348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdc00039c0 0x7fcdc0003e00 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 -- 172.21.15.29:0/380016348 >> 172.21.15.29:0/380016348 conn(0x7fcdd80d0c40 msgr2=0x7fcdd80d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:12.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 -- 172.21.15.29:0/380016348 shutdown_connections 2024-04-25T00:36:12.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 -- 172.21.15.29:0/380016348 wait complete. 2024-04-25T00:36:12.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 Processor -- start 2024-04-25T00:36:12.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 -- start start 2024-04-25T00:36:12.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc00039c0 0x7fcdd80b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdd814dc50 0x7fcdd80c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdd80c7290 0x7fcdd8158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8072110 con 0x7fcdd80c7290 2024-04-25T00:36:12.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8071f90 con 0x7fcdc00039c0 2024-04-25T00:36:12.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd8072290 con 0x7fcdd814dc50 2024-04-25T00:36:12.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdd6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdd80c7290 0x7fcdd8158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdd6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdd80c7290 0x7fcdd8158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39784/0 (socket says 172.21.15.29:39784) 2024-04-25T00:36:12.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdd6d9d640 1 -- 172.21.15.29:0/2726170301 learned_addr learned my addr 172.21.15.29:0/2726170301 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:12.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdd5d9b640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdd814dc50 0x7fcdd80c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdd659c640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc00039c0 0x7fcdd80b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdd6d9d640 1 -- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdd814dc50 msgr2=0x7fcdd80c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdd6d9d640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdd814dc50 0x7fcdd80c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdd6d9d640 1 -- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc00039c0 msgr2=0x7fcdd80b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdd6d9d640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc00039c0 0x7fcdd80b6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdd6d9d640 1 -- 172.21.15.29:0/2726170301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd80d01c0 con 0x7fcdd80c7290 2024-04-25T00:36:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdd5d9b640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdd814dc50 0x7fcdd80c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdd6d9d640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdd80c7290 0x7fcdd8158e10 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80c4740 tx=0x7fcdc800d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdc6ffd640 1 -- 172.21.15.29:0/2726170301 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8007440 con 0x7fcdd80c7290 2024-04-25T00:36:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdc6ffd640 1 -- 172.21.15.29:0/2726170301 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc80075e0 con 0x7fcdd80c7290 2024-04-25T00:36:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 -- 172.21.15.29:0/2726170301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd80b52b0 con 0x7fcdd80c7290 2024-04-25T00:36:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcdc6ffd640 1 -- 172.21.15.29:0/2726170301 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8004a30 con 0x7fcdd80c7290 2024-04-25T00:36:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.212+0000 7fcddcc0b640 1 -- 172.21.15.29:0/2726170301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd80b5870 con 0x7fcdd80c7290 2024-04-25T00:36:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.216+0000 7fcddcc0b640 1 -- 172.21.15.29:0/2726170301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd8069270 con 0x7fcdd80c7290 2024-04-25T00:36:12.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.216+0000 7fcdc6ffd640 1 -- 172.21.15.29:0/2726170301 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fcdc8004bd0 con 0x7fcdd80c7290 2024-04-25T00:36:12.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.216+0000 7fcdc6ffd640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcdb8067c20 0x7fcdb806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.216+0000 7fcdc6ffd640 1 -- 172.21.15.29:0/2726170301 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fcdc8085ab0 con 0x7fcdd80c7290 2024-04-25T00:36:12.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.216+0000 7fcdd659c640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcdb8067c20 0x7fcdb806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.216+0000 7fcdd659c640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcdb8067c20 0x7fcdb806a0e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0001e70 tx=0x7fcdd0001520 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:12.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.220+0000 7fcdc6ffd640 1 -- 172.21.15.29:0/2726170301 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcdc801a060 con 0x7fcdd80c7290 2024-04-25T00:36:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.396+0000 7fcddcc0b640 1 -- 172.21.15.29:0/2726170301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcdd806fc60 con 0x7fcdd80c7290 2024-04-25T00:36:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.396+0000 7fcdc6ffd640 1 -- 172.21.15.29:0/2726170301 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7fcdc8056150 con 0x7fcdd80c7290 2024-04-25T00:36:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:36:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.400+0000 7fcddcc0b640 1 -- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcdb8067c20 msgr2=0x7fcdb806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.400+0000 7fcddcc0b640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcdb8067c20 0x7fcdb806a0e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0001e70 tx=0x7fcdd0001520 comp rx=0 tx=0).stop 2024-04-25T00:36:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.400+0000 7fcddcc0b640 1 -- 172.21.15.29:0/2726170301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdd80c7290 msgr2=0x7fcdd8158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.400+0000 7fcddcc0b640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdd80c7290 0x7fcdd8158e10 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80c4740 tx=0x7fcdc800d900 comp rx=0 tx=0).stop 2024-04-25T00:36:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.400+0000 7fcddcc0b640 1 -- 172.21.15.29:0/2726170301 shutdown_connections 2024-04-25T00:36:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.400+0000 7fcddcc0b640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdd80c7290 0x7fcdd8158e10 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.400+0000 7fcddcc0b640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdd814dc50 0x7fcdd80c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.400+0000 7fcddcc0b640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fcdb8067c20 0x7fcdb806a0e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.400+0000 7fcddcc0b640 1 --2- 172.21.15.29:0/2726170301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc00039c0 0x7fcdd80b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.400+0000 7fcddcc0b640 1 -- 172.21.15.29:0/2726170301 >> 172.21.15.29:0/2726170301 conn(0x7fcdd80d0c40 msgr2=0x7fcdd80b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.400+0000 7fcddcc0b640 1 -- 172.21.15.29:0/2726170301 shutdown_connections 2024-04-25T00:36:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.400+0000 7fcddcc0b640 1 -- 172.21.15.29:0/2726170301 wait complete. 2024-04-25T00:36:12.418 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:36:12.418 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:36:12.425 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:36:12.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4c0d5c640 1 Processor -- start 2024-04-25T00:36:12.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4c0d5c640 1 -- start start 2024-04-25T00:36:12.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4c0d5c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4bc15f5a0 0x7fb4bc0d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4c0d5c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc0cf9f0 0x7fb4bc0cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4c0d5c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4bc15f980 con 0x7fb4bc0d0300 2024-04-25T00:36:12.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4c0d5c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb4bc0cc8a0 con 0x7fb4bc15f5a0 2024-04-25T00:36:12.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4c0d5c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb4bc0cf3e0 con 0x7fb4bc0cf9f0 2024-04-25T00:36:12.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4bad9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb4bc0d0300 0x7fb4bc0c5870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59862/0 (socket says 172.21.15.29:59862) 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4bad9d640 1 -- 172.21.15.29:0/3394976717 learned_addr learned my addr 172.21.15.29:0/3394976717 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b9d9b640 1 --2- 172.21.15.29:0/3394976717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc0cf9f0 0x7fb4bc0cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3462701440 0 0) 0x7fb4bc15f980 con 0x7fb4bc0d0300 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4a4003530 con 0x7fb4bc0d0300 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4ba59c640 1 --2- 172.21.15.29:0/3394976717 >> v2:172.21.15.157:3300/0 conn(0x7fb4bc15f5a0 0x7fb4bc0d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 746505791 0 0) 0x7fb4a4003530 con 0x7fb4bc0d0300 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc0cf9f0 msgr2=0x7fb4bc0cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 --2- 172.21.15.29:0/3394976717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc0cf9f0 0x7fb4bc0cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 >> v2:172.21.15.157:3300/0 conn(0x7fb4bc15f5a0 msgr2=0x7fb4bc0d2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 --2- 172.21.15.29:0/3394976717 >> v2:172.21.15.157:3300/0 conn(0x7fb4bc15f5a0 0x7fb4bc0d2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc0ccad0 con 0x7fb4bc0d0300 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fb4b4001a00 con 0x7fb4bc0d0300 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 >> v1:172.21.15.29:6789/0 conn(0x7fb4bc0d0300 legacy=0x7fb4bc0c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 --2- 172.21.15.29:0/3394976717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc0cf9f0 0x7fb4a40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 --2- 172.21.15.29:0/3394976717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a4003f90 0x7fb4a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 --2- 172.21.15.29:0/3394976717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a4004d50 0x7fb4a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc0cf3e0 con 0x7fb4a4003f90 2024-04-25T00:36:12.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a400ba90 con 0x7fb4a4004d50 2024-04-25T00:36:12.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a400bc90 con 0x7fb4bc0cf9f0 2024-04-25T00:36:12.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b9d9b640 1 --2- 172.21.15.29:0/3394976717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a4003f90 0x7fb4a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4bad9d640 1 --2- 172.21.15.29:0/3394976717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a4004d50 0x7fb4a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b9d9b640 1 -- 172.21.15.29:0/3394976717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc0cf9f0 msgr2=0x7fb4a40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:12.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b9d9b640 1 --2- 172.21.15.29:0/3394976717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc0cf9f0 0x7fb4a40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b9d9b640 1 -- 172.21.15.29:0/3394976717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a4004d50 msgr2=0x7fb4a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:12.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b9d9b640 1 --2- 172.21.15.29:0/3394976717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a4004d50 0x7fb4a40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4b9d9b640 1 -- 172.21.15.29:0/3394976717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a4003530 con 0x7fb4a4003f90 2024-04-25T00:36:12.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.548+0000 7fb4bad9d640 1 --2- 172.21.15.29:0/3394976717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a4004d50 0x7fb4a40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:12.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.552+0000 7fb4b9d9b640 1 --2- 172.21.15.29:0/3394976717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a4003f90 0x7fb4a4004810 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000ba30 tx=0x7fb4b001c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2bacbf1c07b88c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:12.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.552+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b000b600 con 0x7fb4a4003f90 2024-04-25T00:36:12.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.552+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b00aa650 con 0x7fb4a4003f90 2024-04-25T00:36:12.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.552+0000 7fb4b959a640 1 -- 172.21.15.29:0/3394976717 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b00aaa40 con 0x7fb4a4003f90 2024-04-25T00:36:12.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.552+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/3394976717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a4003f90 msgr2=0x7fb4a4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.552+0000 7fb4c0d5c640 1 --2- 172.21.15.29:0/3394976717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a4003f90 0x7fb4a4004810 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000ba30 tx=0x7fb4b001c6e0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.552+0000 7fb4bad9d640 1 -- 172.21.15.29:0/3394976717 reap_dead start 2024-04-25T00:36:12.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.552+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/3394976717 shutdown_connections 2024-04-25T00:36:12.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.552+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/3394976717 >> 172.21.15.29:0/3394976717 conn(0x7fb4bc0d1d20 msgr2=0x7fb4bc0d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:12.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.552+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/3394976717 shutdown_connections 2024-04-25T00:36:12.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.552+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/3394976717 wait complete. 2024-04-25T00:36:12.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.552+0000 7fb4c0d5c640 1 Processor -- start 2024-04-25T00:36:12.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.552+0000 7fb4c0d5c640 1 -- start start 2024-04-25T00:36:12.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4c0d5c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a4003f90 0x7fb4bc1599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4c0d5c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4bc14cc20 0x7fb4bc14d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4c0d5c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc14aec0 0x7fb4bc14b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4c0d5c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc0cefb0 con 0x7fb4a4003f90 2024-04-25T00:36:12.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4c0d5c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc0cd090 con 0x7fb4bc14cc20 2024-04-25T00:36:12.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4c0d5c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb4bc0cf130 con 0x7fb4bc14aec0 2024-04-25T00:36:12.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4ba59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a4003f90 0x7fb4bc1599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4ba59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a4003f90 0x7fb4bc1599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39810/0 (socket says 172.21.15.29:39810) 2024-04-25T00:36:12.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4ba59c640 1 -- 172.21.15.29:0/1218154155 learned_addr learned my addr 172.21.15.29:0/1218154155 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:12.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4bad9d640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc14aec0 0x7fb4bc14b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4b9d9b640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4bc14cc20 0x7fb4bc14d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4ba59c640 1 -- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc14aec0 msgr2=0x7fb4bc14b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4ba59c640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc14aec0 0x7fb4bc14b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4ba59c640 1 -- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4bc14cc20 msgr2=0x7fb4bc14d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4ba59c640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4bc14cc20 0x7fb4bc14d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4ba59c640 1 -- 172.21.15.29:0/1218154155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc14bc80 con 0x7fb4a4003f90 2024-04-25T00:36:12.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4ba59c640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a4003f90 0x7fb4bc1599f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00e840 tx=0x7fb4ac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:12.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4aaffd640 1 -- 172.21.15.29:0/1218154155 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac017070 con 0x7fb4a4003f90 2024-04-25T00:36:12.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4aaffd640 1 -- 172.21.15.29:0/1218154155 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4ac00c940 con 0x7fb4a4003f90 2024-04-25T00:36:12.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/1218154155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4bc14bf60 con 0x7fb4a4003f90 2024-04-25T00:36:12.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4aaffd640 1 -- 172.21.15.29:0/1218154155 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac00cd30 con 0x7fb4a4003f90 2024-04-25T00:36:12.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/1218154155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4bc14c520 con 0x7fb4a4003f90 2024-04-25T00:36:12.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/1218154155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4bc069270 con 0x7fb4a4003f90 2024-04-25T00:36:12.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4aaffd640 1 -- 172.21.15.29:0/1218154155 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb4ac01a070 con 0x7fb4a4003f90 2024-04-25T00:36:12.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4aaffd640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb49c067c20 0x7fb49c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4aaffd640 1 -- 172.21.15.29:0/1218154155 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fb4ac086520 con 0x7fb4a4003f90 2024-04-25T00:36:12.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.556+0000 7fb4b9d9b640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb49c067c20 0x7fb49c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.560+0000 7fb4b9d9b640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb49c067c20 0x7fb49c06a0e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00af000 tx=0x7fb4b00b4040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:12.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.560+0000 7fb4aaffd640 1 -- 172.21.15.29:0/1218154155 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb4ac056c50 con 0x7fb4a4003f90 2024-04-25T00:36:12.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.724+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/1218154155 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4bc0d1010 con 0x7fb4a4003f90 2024-04-25T00:36:12.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.728+0000 7fb4aaffd640 1 -- 172.21.15.29:0/1218154155 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16620 (secure 0 0 0) 0x7fb4ac056550 con 0x7fb4a4003f90 2024-04-25T00:36:12.730 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:12.731 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":70,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:35:51.187302+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:47.559043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:48.530288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:36:12.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.732+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb49c067c20 msgr2=0x7fb49c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.732+0000 7fb4c0d5c640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb49c067c20 0x7fb49c06a0e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00af000 tx=0x7fb4b00b4040 comp rx=0 tx=0).stop 2024-04-25T00:36:12.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.732+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/1218154155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a4003f90 msgr2=0x7fb4bc1599f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.732+0000 7fb4c0d5c640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a4003f90 0x7fb4bc1599f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00e840 tx=0x7fb4ac00ed00 comp rx=0 tx=0).stop 2024-04-25T00:36:12.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.732+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/1218154155 shutdown_connections 2024-04-25T00:36:12.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.732+0000 7fb4c0d5c640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4bc14aec0 0x7fb4bc14b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.732+0000 7fb4c0d5c640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb49c067c20 0x7fb49c06a0e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.732+0000 7fb4c0d5c640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4bc14cc20 0x7fb4bc14d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.732+0000 7fb4c0d5c640 1 --2- 172.21.15.29:0/1218154155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4a4003f90 0x7fb4bc1599f0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.732+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/1218154155 >> 172.21.15.29:0/1218154155 conn(0x7fb4bc0d1d20 msgr2=0x7fb4bc0c7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:12.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.732+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/1218154155 shutdown_connections 2024-04-25T00:36:12.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.732+0000 7fb4c0d5c640 1 -- 172.21.15.29:0/1218154155 wait complete. 2024-04-25T00:36:12.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:35:47.559043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:35:48.530288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:36:12.751 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:36:12.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7dbb1f640 1 Processor -- start 2024-04-25T00:36:12.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7dbb1f640 1 -- start start 2024-04-25T00:36:12.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7dbb1f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff7d415e060 0x7ff7d415c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7dbb1f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415bce0 0x7ff7d4071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7dbb1f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff7d40722c0 0x7ff7d40cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7dbb1f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff7d415f700 con 0x7ff7d415e060 2024-04-25T00:36:12.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7dbb1f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff7d415e850 con 0x7ff7d40722c0 2024-04-25T00:36:12.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7dbb1f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d415eaf0 con 0x7ff7d415bce0 2024-04-25T00:36:12.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7d98bb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff7d415e060 0x7ff7d415c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7d98bb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff7d415e060 0x7ff7d415c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39818/0 (socket says 172.21.15.29:39818) 2024-04-25T00:36:12.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7d98bb640 1 -- 172.21.15.29:0/3254629062 learned_addr learned my addr 172.21.15.29:0/3254629062 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:12.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7d98bb640 1 -- 172.21.15.29:0/3254629062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415bce0 msgr2=0x7ff7d4071d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:36:12.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7d98bb640 1 --2- 172.21.15.29:0/3254629062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415bce0 0x7ff7d4071d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7d98bb640 1 -- 172.21.15.29:0/3254629062 >> v2:172.21.15.157:3300/0 conn(0x7ff7d40722c0 msgr2=0x7ff7d40cf100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:36:12.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7d98bb640 1 --2- 172.21.15.29:0/3254629062 >> v2:172.21.15.157:3300/0 conn(0x7ff7d40722c0 0x7ff7d40cf100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.876+0000 7ff7d98bb640 1 -- 172.21.15.29:0/3254629062 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d415edd0 con 0x7ff7d415e060 2024-04-25T00:36:12.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7d98bb640 1 --2- 172.21.15.29:0/3254629062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7d415e060 0x7ff7d415c420 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40b06c0 tx=0x7ff7cc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae83c73261092c4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:12.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7d88b9640 1 -- 172.21.15.29:0/3254629062 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc0ab410 con 0x7ff7d415e060 2024-04-25T00:36:12.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7d88b9640 1 -- 172.21.15.29:0/3254629062 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7cc0b0070 con 0x7ff7d415e060 2024-04-25T00:36:12.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7d88b9640 1 -- 172.21.15.29:0/3254629062 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc0ab7d0 con 0x7ff7d415e060 2024-04-25T00:36:12.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3254629062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7d415e060 msgr2=0x7ff7d415c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 --2- 172.21.15.29:0/3254629062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7d415e060 0x7ff7d415c420 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40b06c0 tx=0x7ff7cc01c560 comp rx=0 tx=0).stop 2024-04-25T00:36:12.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3254629062 shutdown_connections 2024-04-25T00:36:12.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 --2- 172.21.15.29:0/3254629062 >> v2:172.21.15.157:3300/0 conn(0x7ff7d40722c0 0x7ff7d40cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 --2- 172.21.15.29:0/3254629062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415bce0 0x7ff7d4071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 --2- 172.21.15.29:0/3254629062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7d415e060 0x7ff7d415c420 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3254629062 >> 172.21.15.29:0/3254629062 conn(0x7ff7d406c320 msgr2=0x7ff7d406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:12.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3254629062 shutdown_connections 2024-04-25T00:36:12.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3254629062 wait complete. 2024-04-25T00:36:12.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 Processor -- start 2024-04-25T00:36:12.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 -- start start 2024-04-25T00:36:12.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d40722c0 0x7ff7d40b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7d415bce0 0x7ff7d414d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d415e060 0x7ff7d4159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d415e5c0 con 0x7ff7d415bce0 2024-04-25T00:36:12.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d415e440 con 0x7ff7d415e060 2024-04-25T00:36:12.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7dbb1f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d406f610 con 0x7ff7d40722c0 2024-04-25T00:36:12.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7d90ba640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7d415bce0 0x7ff7d414d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7d90ba640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7d415bce0 0x7ff7d414d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39828/0 (socket says 172.21.15.29:39828) 2024-04-25T00:36:12.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7d90ba640 1 -- 172.21.15.29:0/3235437806 learned_addr learned my addr 172.21.15.29:0/3235437806 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:12.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7da0bc640 1 --2- 172.21.15.29:0/3235437806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d415e060 0x7ff7d4159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7d90ba640 1 -- 172.21.15.29:0/3235437806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d40722c0 msgr2=0x7ff7d40b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7d90ba640 1 --2- 172.21.15.29:0/3235437806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d40722c0 0x7ff7d40b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7d90ba640 1 -- 172.21.15.29:0/3235437806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d415e060 msgr2=0x7ff7d4159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:12.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7d90ba640 1 --2- 172.21.15.29:0/3235437806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d415e060 0x7ff7d4159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:12.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.880+0000 7ff7d90ba640 1 -- 172.21.15.29:0/3235437806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d40c8cc0 con 0x7ff7d415bce0 2024-04-25T00:36:12.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.884+0000 7ff7d90ba640 1 --2- 172.21.15.29:0/3235437806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7d415bce0 0x7ff7d414d830 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800ba80 tx=0x7ff7c800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:12.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.884+0000 7ff7d27fc640 1 -- 172.21.15.29:0/3235437806 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8019070 con 0x7ff7d415bce0 2024-04-25T00:36:12.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.884+0000 7ff7d27fc640 1 -- 172.21.15.29:0/3235437806 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7c800cc90 con 0x7ff7d415bce0 2024-04-25T00:36:12.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.884+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3235437806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d40c8fa0 con 0x7ff7d415bce0 2024-04-25T00:36:12.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.884+0000 7ff7d27fc640 1 -- 172.21.15.29:0/3235437806 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8014680 con 0x7ff7d415bce0 2024-04-25T00:36:12.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.884+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3235437806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d414dd70 con 0x7ff7d415bce0 2024-04-25T00:36:12.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.884+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3235437806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d4069270 con 0x7ff7d415bce0 2024-04-25T00:36:12.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.884+0000 7ff7d27fc640 1 -- 172.21.15.29:0/3235437806 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7ff7c801c070 con 0x7ff7d415bce0 2024-04-25T00:36:12.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.884+0000 7ff7d27fc640 1 --2- 172.21.15.29:0/3235437806 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7c0067c20 0x7ff7c006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:12.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.884+0000 7ff7d27fc640 1 -- 172.21.15.29:0/3235437806 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7ff7c8085ff0 con 0x7ff7d415bce0 2024-04-25T00:36:12.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.884+0000 7ff7d98bb640 1 --2- 172.21.15.29:0/3235437806 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7c0067c20 0x7ff7c006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:12.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.884+0000 7ff7d98bb640 1 --2- 172.21.15.29:0/3235437806 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7c0067c20 0x7ff7c006a0e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0af000 tx=0x7ff7cc004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:12.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:12.888+0000 7ff7d27fc640 1 -- 172.21.15.29:0/3235437806 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff7c80566e0 con 0x7ff7d415bce0 2024-04-25T00:36:13.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.052+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3235437806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff7d406e090 con 0x7ff7d415bce0 2024-04-25T00:36:13.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.052+0000 7ff7d27fc640 1 -- 172.21.15.29:0/3235437806 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7ff7c8055fe0 con 0x7ff7d415bce0 2024-04-25T00:36:13.054 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:36:13.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.052+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3235437806 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7c0067c20 msgr2=0x7ff7c006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.052+0000 7ff7dbb1f640 1 --2- 172.21.15.29:0/3235437806 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7c0067c20 0x7ff7c006a0e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0af000 tx=0x7ff7cc004740 comp rx=0 tx=0).stop 2024-04-25T00:36:13.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.052+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3235437806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7d415bce0 msgr2=0x7ff7d414d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.052+0000 7ff7dbb1f640 1 --2- 172.21.15.29:0/3235437806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7d415bce0 0x7ff7d414d830 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800ba80 tx=0x7ff7c800bf40 comp rx=0 tx=0).stop 2024-04-25T00:36:13.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.056+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3235437806 shutdown_connections 2024-04-25T00:36:13.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.056+0000 7ff7dbb1f640 1 --2- 172.21.15.29:0/3235437806 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7ff7c0067c20 0x7ff7c006a0e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.056+0000 7ff7dbb1f640 1 --2- 172.21.15.29:0/3235437806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d415e060 0x7ff7d4159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.056+0000 7ff7dbb1f640 1 --2- 172.21.15.29:0/3235437806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7d415bce0 0x7ff7d414d830 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.056+0000 7ff7dbb1f640 1 --2- 172.21.15.29:0/3235437806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d40722c0 0x7ff7d40b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.056+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3235437806 >> 172.21.15.29:0/3235437806 conn(0x7ff7d406c320 msgr2=0x7ff7d4153960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:13.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.056+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3235437806 shutdown_connections 2024-04-25T00:36:13.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.056+0000 7ff7dbb1f640 1 -- 172.21.15.29:0/3235437806 wait complete. 2024-04-25T00:36:13.075 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:36:13.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.204+0000 7f5d257f7640 1 Processor -- start 2024-04-25T00:36:13.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.204+0000 7f5d257f7640 1 -- start start 2024-04-25T00:36:13.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.204+0000 7f5d257f7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5d2006e440 0x7f5d200b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.204+0000 7f5d257f7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d200b41e0 con 0x7f5d200b3c30 2024-04-25T00:36:13.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.204+0000 7f5d257f7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d2006d960 con 0x7f5d200b3140 2024-04-25T00:36:13.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.204+0000 7f5d257f7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5d200b5050 con 0x7f5d2006e440 2024-04-25T00:36:13.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.204+0000 7f5d1f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5d2006e440 0x7f5d200b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.204+0000 7f5d1e7fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5d200b3c30 0x7f5d200ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59872/0 (socket says 172.21.15.29:59872) 2024-04-25T00:36:13.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.204+0000 7f5d1e7fc640 1 -- 172.21.15.29:0/2229109798 learned_addr learned my addr 172.21.15.29:0/2229109798 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:13.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.204+0000 7f5d1f7fe640 1 -- 172.21.15.29:0/2229109798 >> v1:172.21.15.157:6789/0 conn(0x7f5d200b3140 legacy=0x7f5d200b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:13.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.204+0000 7f5d1f7fe640 1 -- 172.21.15.29:0/2229109798 >> v1:172.21.15.29:6789/0 conn(0x7f5d200b3c30 legacy=0x7f5d200ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.204+0000 7f5d1dffb640 1 -- 172.21.15.29:0/2229109798 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 690520973 0 0) 0x7f5d200b41e0 con 0x7f5d200b3c30 2024-04-25T00:36:13.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.204+0000 7f5d1f7fe640 1 -- 172.21.15.29:0/2229109798 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2006dc40 con 0x7f5d2006e440 2024-04-25T00:36:13.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.208+0000 7f5d1f7fe640 1 --2- 172.21.15.29:0/2229109798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d2006e440 0x7f5d200b4b10 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5d100097f0 tx=0x7f5d1001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9884825be8d7ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:13.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.208+0000 7f5d1dffb640 1 -- 172.21.15.29:0/2229109798 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d100b0070 con 0x7f5d2006e440 2024-04-25T00:36:13.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.208+0000 7f5d1dffb640 1 -- 172.21.15.29:0/2229109798 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d100ab600 con 0x7f5d2006e440 2024-04-25T00:36:13.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.208+0000 7f5d1dffb640 1 -- 172.21.15.29:0/2229109798 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d100ab9d0 con 0x7f5d2006e440 2024-04-25T00:36:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.208+0000 7f5d257f7640 1 -- 172.21.15.29:0/2229109798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d2006e440 msgr2=0x7f5d200b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.208+0000 7f5d257f7640 1 --2- 172.21.15.29:0/2229109798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d2006e440 0x7f5d200b4b10 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5d100097f0 tx=0x7f5d1001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.208+0000 7f5d257f7640 1 -- 172.21.15.29:0/2229109798 shutdown_connections 2024-04-25T00:36:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.208+0000 7f5d257f7640 1 --2- 172.21.15.29:0/2229109798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d2006e440 0x7f5d200b4b10 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.208+0000 7f5d257f7640 1 -- 172.21.15.29:0/2229109798 >> 172.21.15.29:0/2229109798 conn(0x7f5d200b1870 msgr2=0x7f5d200b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.208+0000 7f5d257f7640 1 -- 172.21.15.29:0/2229109798 shutdown_connections 2024-04-25T00:36:13.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.208+0000 7f5d257f7640 1 -- 172.21.15.29:0/2229109798 wait complete. 2024-04-25T00:36:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.208+0000 7f5d257f7640 1 Processor -- start 2024-04-25T00:36:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.208+0000 7f5d257f7640 1 -- start start 2024-04-25T00:36:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d257f7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d2006e440 0x7f5d2013a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d257f7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d200b3140 0x7f5d2013f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d257f7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d200b3c30 0x7f5d20159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d257f7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20169d90 con 0x7f5d200b3140 2024-04-25T00:36:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d257f7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20169c10 con 0x7f5d200b3c30 2024-04-25T00:36:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d257f7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5d20169f10 con 0x7f5d2006e440 2024-04-25T00:36:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d1e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d200b3140 0x7f5d2013f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d1e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d200b3140 0x7f5d2013f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39850/0 (socket says 172.21.15.29:39850) 2024-04-25T00:36:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d1e7fc640 1 -- 172.21.15.29:0/1223726845 learned_addr learned my addr 172.21.15.29:0/1223726845 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d1e7fc640 1 -- 172.21.15.29:0/1223726845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d2006e440 msgr2=0x7f5d2013a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d1e7fc640 1 --2- 172.21.15.29:0/1223726845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d2006e440 0x7f5d2013a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d1e7fc640 1 -- 172.21.15.29:0/1223726845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d200b3c30 msgr2=0x7f5d20159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d1e7fc640 1 --2- 172.21.15.29:0/1223726845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d200b3c30 0x7f5d20159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d1e7fc640 1 -- 172.21.15.29:0/1223726845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d20164b20 con 0x7f5d200b3140 2024-04-25T00:36:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d1e7fc640 1 --2- 172.21.15.29:0/1223726845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d200b3140 0x7f5d2013f600 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800c8c0 tx=0x7f5d0800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d177fe640 1 -- 172.21.15.29:0/1223726845 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08018070 con 0x7f5d200b3140 2024-04-25T00:36:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d177fe640 1 -- 172.21.15.29:0/1223726845 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d0800a640 con 0x7f5d200b3140 2024-04-25T00:36:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d177fe640 1 -- 172.21.15.29:0/1223726845 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0800aa70 con 0x7f5d200b3140 2024-04-25T00:36:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d257f7640 1 -- 172.21.15.29:0/1223726845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d20164e00 con 0x7f5d200b3140 2024-04-25T00:36:13.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d257f7640 1 -- 172.21.15.29:0/1223726845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d201653c0 con 0x7f5d200b3140 2024-04-25T00:36:13.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d257f7640 1 -- 172.21.15.29:0/1223726845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d20069270 con 0x7f5d200b3140 2024-04-25T00:36:13.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d177fe640 1 -- 172.21.15.29:0/1223726845 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f5d0800ac10 con 0x7f5d200b3140 2024-04-25T00:36:13.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d177fe640 1 --2- 172.21.15.29:0/1223726845 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5d00067c20 0x7f5d0006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d177fe640 1 -- 172.21.15.29:0/1223726845 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f5d08085920 con 0x7f5d200b3140 2024-04-25T00:36:13.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.212+0000 7f5d1effd640 1 --2- 172.21.15.29:0/1223726845 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5d00067c20 0x7f5d0006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.216+0000 7f5d1effd640 1 --2- 172.21.15.29:0/1223726845 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5d00067c20 0x7f5d0006a0e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5d18000f70 tx=0x7f5d180079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:13.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.216+0000 7f5d177fe640 1 -- 172.21.15.29:0/1223726845 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5d08056050 con 0x7f5d200b3140 2024-04-25T00:36:13.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.380+0000 7f5d257f7640 1 -- 172.21.15.29:0/1223726845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5d200672f0 con 0x7f5d200b3140 2024-04-25T00:36:13.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.380+0000 7f5d177fe640 1 -- 172.21.15.29:0/1223726845 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) ==== 104+0+11 (secure 0 0 0) 0x7f5d08055950 con 0x7f5d200b3140 2024-04-25T00:36:13.385 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:36:13.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.384+0000 7f5d257f7640 1 -- 172.21.15.29:0/1223726845 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5d00067c20 msgr2=0x7f5d0006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.384+0000 7f5d257f7640 1 --2- 172.21.15.29:0/1223726845 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5d00067c20 0x7f5d0006a0e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5d18000f70 tx=0x7f5d180079c0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.384+0000 7f5d257f7640 1 -- 172.21.15.29:0/1223726845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d200b3140 msgr2=0x7f5d2013f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.384+0000 7f5d257f7640 1 --2- 172.21.15.29:0/1223726845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d200b3140 0x7f5d2013f600 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800c8c0 tx=0x7f5d0800cd80 comp rx=0 tx=0).stop 2024-04-25T00:36:13.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.384+0000 7f5d257f7640 1 -- 172.21.15.29:0/1223726845 shutdown_connections 2024-04-25T00:36:13.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.384+0000 7f5d257f7640 1 --2- 172.21.15.29:0/1223726845 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f5d00067c20 0x7f5d0006a0e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.384+0000 7f5d257f7640 1 --2- 172.21.15.29:0/1223726845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d200b3c30 0x7f5d20159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.384+0000 7f5d257f7640 1 --2- 172.21.15.29:0/1223726845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d200b3140 0x7f5d2013f600 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.384+0000 7f5d257f7640 1 --2- 172.21.15.29:0/1223726845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d2006e440 0x7f5d2013a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.384+0000 7f5d257f7640 1 -- 172.21.15.29:0/1223726845 >> 172.21.15.29:0/1223726845 conn(0x7f5d200b1870 msgr2=0x7f5d20167230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:13.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.384+0000 7f5d257f7640 1 -- 172.21.15.29:0/1223726845 shutdown_connections 2024-04-25T00:36:13.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.384+0000 7f5d257f7640 1 -- 172.21.15.29:0/1223726845 wait complete. 2024-04-25T00:36:13.404 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:36:13.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286adee640 1 Processor -- start 2024-04-25T00:36:13.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286adee640 1 -- start start 2024-04-25T00:36:13.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286adee640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28640d2bb0 0x7f28640ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286adee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f286415e630 0x7f28640cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286adee640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28640c8690 con 0x7f286415e250 2024-04-25T00:36:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286adee640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f28640c8440 con 0x7f286415e630 2024-04-25T00:36:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286adee640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f28640c8970 con 0x7f28640d2bb0 2024-04-25T00:36:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286938b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f286415e630 0x7f28640cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286938b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f286415e630 0x7f28640cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39860/0 (socket says 172.21.15.29:39860) 2024-04-25T00:36:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286938b640 1 -- 172.21.15.29:0/156473624 learned_addr learned my addr 172.21.15.29:0/156473624 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f2863fff640 1 --2- 172.21.15.29:0/156473624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28640d2bb0 0x7f28640ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286938b640 1 -- 172.21.15.29:0/156473624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28640d2bb0 msgr2=0x7f28640ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286938b640 1 --2- 172.21.15.29:0/156473624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28640d2bb0 0x7f28640ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286938b640 1 -- 172.21.15.29:0/156473624 >> v1:172.21.15.157:6789/0 conn(0x7f286415e250 legacy=0x7f28640d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:13.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.524+0000 7f286938b640 1 -- 172.21.15.29:0/156473624 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28640c8c50 con 0x7f286415e630 2024-04-25T00:36:13.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286938b640 1 --2- 172.21.15.29:0/156473624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286415e630 0x7f28640cf030 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2864161ee0 tx=0x7f285c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d6b09bd871a6c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:13.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f28637fe640 1 -- 172.21.15.29:0/156473624 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c0a56b0 con 0x7f286415e630 2024-04-25T00:36:13.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f28637fe640 1 -- 172.21.15.29:0/156473624 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285c0a5850 con 0x7f286415e630 2024-04-25T00:36:13.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f28637fe640 1 -- 172.21.15.29:0/156473624 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c0a5c20 con 0x7f286415e630 2024-04-25T00:36:13.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 -- 172.21.15.29:0/156473624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286415e630 msgr2=0x7f28640cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 --2- 172.21.15.29:0/156473624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286415e630 0x7f28640cf030 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2864161ee0 tx=0x7f285c01c920 comp rx=0 tx=0).stop 2024-04-25T00:36:13.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 -- 172.21.15.29:0/156473624 shutdown_connections 2024-04-25T00:36:13.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 --2- 172.21.15.29:0/156473624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286415e630 0x7f28640cf030 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 --2- 172.21.15.29:0/156473624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28640d2bb0 0x7f28640ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 -- 172.21.15.29:0/156473624 >> 172.21.15.29:0/156473624 conn(0x7f28640d3f70 msgr2=0x7f28640d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:13.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 -- 172.21.15.29:0/156473624 shutdown_connections 2024-04-25T00:36:13.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 -- 172.21.15.29:0/156473624 wait complete. 2024-04-25T00:36:13.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 Processor -- start 2024-04-25T00:36:13.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 -- start start 2024-04-25T00:36:13.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28640d2bb0 0x7f2864152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286415e250 0x7f28640b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286415e630 0x7f28640b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f286416aff0 con 0x7f286415e630 2024-04-25T00:36:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f28640d1ed0 con 0x7f286415e250 2024-04-25T00:36:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.528+0000 7f286adee640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f286416b170 con 0x7f28640d2bb0 2024-04-25T00:36:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f286938b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286415e630 0x7f28640b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f286938b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286415e630 0x7f28640b7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39868/0 (socket says 172.21.15.29:39868) 2024-04-25T00:36:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f286938b640 1 -- 172.21.15.29:0/1833056099 learned_addr learned my addr 172.21.15.29:0/1833056099 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f2868b8a640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28640d2bb0 0x7f2864152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f286938b640 1 -- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28640d2bb0 msgr2=0x7f2864152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f286938b640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28640d2bb0 0x7f2864152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f286938b640 1 -- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286415e250 msgr2=0x7f28640b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f286938b640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286415e250 0x7f28640b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f286938b640 1 -- 172.21.15.29:0/1833056099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28640c81e0 con 0x7f286415e630 2024-04-25T00:36:13.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f2868b8a640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28640d2bb0 0x7f2864152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:13.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f286938b640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286415e630 0x7f28640b7430 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f285c01afd0 tx=0x7f285c0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:13.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f28617fa640 1 -- 172.21.15.29:0/1833056099 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c0bc020 con 0x7f286415e630 2024-04-25T00:36:13.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f28617fa640 1 -- 172.21.15.29:0/1833056099 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285c0b8670 con 0x7f286415e630 2024-04-25T00:36:13.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f286adee640 1 -- 172.21.15.29:0/1833056099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2864153200 con 0x7f286415e630 2024-04-25T00:36:13.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f28617fa640 1 -- 172.21.15.29:0/1833056099 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c009510 con 0x7f286415e630 2024-04-25T00:36:13.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f286adee640 1 -- 172.21.15.29:0/1833056099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2864154580 con 0x7f286415e630 2024-04-25T00:36:13.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f286adee640 1 -- 172.21.15.29:0/1833056099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2864069270 con 0x7f286415e630 2024-04-25T00:36:13.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f28617fa640 1 -- 172.21.15.29:0/1833056099 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f285c0bf020 con 0x7f286415e630 2024-04-25T00:36:13.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f28617fa640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2850067c50 0x7f285006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.532+0000 7f28617fa640 1 -- 172.21.15.29:0/1833056099 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f285c1215f0 con 0x7f286415e630 2024-04-25T00:36:13.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.536+0000 7f2868b8a640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2850067c50 0x7f285006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.536+0000 7f2868b8a640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2850067c50 0x7f285006a110 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f28641fa7c0 tx=0x7f2854009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:13.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.536+0000 7f28617fa640 1 -- 172.21.15.29:0/1833056099 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f285c0c6070 con 0x7f286415e630 2024-04-25T00:36:13.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.692+0000 7f286adee640 1 -- 172.21.15.29:0/1833056099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f28640c7d10 con 0x7f286415e630 2024-04-25T00:36:13.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.696+0000 7f28617fa640 1 -- 172.21.15.29:0/1833056099 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) ==== 100+0+11 (secure 0 0 0) 0x7f285c0165a0 con 0x7f286415e630 2024-04-25T00:36:13.697 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:36:13.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.696+0000 7f286adee640 1 -- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2850067c50 msgr2=0x7f285006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.696+0000 7f286adee640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2850067c50 0x7f285006a110 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f28641fa7c0 tx=0x7f2854009040 comp rx=0 tx=0).stop 2024-04-25T00:36:13.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.696+0000 7f286adee640 1 -- 172.21.15.29:0/1833056099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286415e630 msgr2=0x7f28640b7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.696+0000 7f286adee640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286415e630 0x7f28640b7430 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f285c01afd0 tx=0x7f285c0196c0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.696+0000 7f286adee640 1 -- 172.21.15.29:0/1833056099 shutdown_connections 2024-04-25T00:36:13.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.696+0000 7f286adee640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286415e630 0x7f28640b7430 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.696+0000 7f286adee640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2850067c50 0x7f285006a110 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.696+0000 7f286adee640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286415e250 0x7f28640b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.696+0000 7f286adee640 1 --2- 172.21.15.29:0/1833056099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28640d2bb0 0x7f2864152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.696+0000 7f286adee640 1 -- 172.21.15.29:0/1833056099 >> 172.21.15.29:0/1833056099 conn(0x7f28640d3f70 msgr2=0x7f28640cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:13.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.696+0000 7f286adee640 1 -- 172.21.15.29:0/1833056099 shutdown_connections 2024-04-25T00:36:13.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.696+0000 7f286adee640 1 -- 172.21.15.29:0/1833056099 wait complete. 2024-04-25T00:36:13.715 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:36:13.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.836+0000 7fb6329cc640 1 Processor -- start 2024-04-25T00:36:13.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.836+0000 7fb6329cc640 1 -- start start 2024-04-25T00:36:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb6329cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0b1290 0x7fb62c0cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb6329cc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb62c0d0260 0x7fb62c0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb6329cc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb62c0d1340 con 0x7fb62c15e2f0 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb6329cc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb62c15e6d0 con 0x7fb62c0d0260 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb6329cc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb62c06c4a0 con 0x7fb62c0b1290 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb62c15e2f0 0x7fb62c06cf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59884/0 (socket says 172.21.15.29:59884) 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62bfff640 1 -- 172.21.15.29:0/4052955891 learned_addr learned my addr 172.21.15.29:0/4052955891 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62b7fe640 1 --2- 172.21.15.29:0/4052955891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0b1290 0x7fb62c0cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb630f69640 1 --2- 172.21.15.29:0/4052955891 >> v2:172.21.15.157:3300/0 conn(0x7fb62c0d0260 0x7fb62c0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 292117467 0 0) 0x7fb62c0d1340 con 0x7fb62c15e2f0 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb614003530 con 0x7fb62c15e2f0 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 88981816 0 0) 0x7fb614003530 con 0x7fb62c15e2f0 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0b1290 msgr2=0x7fb62c0cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 --2- 172.21.15.29:0/4052955891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0b1290 0x7fb62c0cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 >> v2:172.21.15.157:3300/0 conn(0x7fb62c0d0260 msgr2=0x7fb62c0cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 --2- 172.21.15.29:0/4052955891 >> v2:172.21.15.157:3300/0 conn(0x7fb62c0d0260 0x7fb62c0cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62c06c780 con 0x7fb62c15e2f0 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fb61c001a00 con 0x7fb62c15e2f0 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 >> v1:172.21.15.29:6789/0 conn(0x7fb62c15e2f0 legacy=0x7fb62c06cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 --2- 172.21.15.29:0/4052955891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0b1290 0x7fb6140039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 --2- 172.21.15.29:0/4052955891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614003f40 0x7fb614004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 --2- 172.21.15.29:0/4052955891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614004d50 0x7fb6140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c15e6d0 con 0x7fb614003f40 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c06c4a0 con 0x7fb614004d50 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb61400ba00 con 0x7fb62c0b1290 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62b7fe640 1 --2- 172.21.15.29:0/4052955891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614003f40 0x7fb614004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb630f69640 1 --2- 172.21.15.29:0/4052955891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614004d50 0x7fb6140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62bfff640 1 --2- 172.21.15.29:0/4052955891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0b1290 0x7fb6140039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62b7fe640 1 -- 172.21.15.29:0/4052955891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0b1290 msgr2=0x7fb6140039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62b7fe640 1 --2- 172.21.15.29:0/4052955891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0b1290 0x7fb6140039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62b7fe640 1 -- 172.21.15.29:0/4052955891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614004d50 msgr2=0x7fb6140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:13.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62b7fe640 1 --2- 172.21.15.29:0/4052955891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614004d50 0x7fb6140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62b7fe640 1 -- 172.21.15.29:0/4052955891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb614003530 con 0x7fb614003f40 2024-04-25T00:36:13.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.840+0000 7fb62bfff640 1 --2- 172.21.15.29:0/4052955891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0b1290 0x7fb6140039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:13.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.844+0000 7fb62b7fe640 1 --2- 172.21.15.29:0/4052955891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614003f40 0x7fb614004810 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb620009ac0 tx=0x7fb62001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6db72fba748f99a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:13.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.844+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620009690 con 0x7fb614003f40 2024-04-25T00:36:13.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.844+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6200059a0 con 0x7fb614003f40 2024-04-25T00:36:13.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.844+0000 7fb62affd640 1 -- 172.21.15.29:0/4052955891 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620005d90 con 0x7fb614003f40 2024-04-25T00:36:13.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.844+0000 7fb6329cc640 1 -- 172.21.15.29:0/4052955891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614003f40 msgr2=0x7fb614004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.844+0000 7fb6329cc640 1 --2- 172.21.15.29:0/4052955891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb614003f40 0x7fb614004810 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb620009ac0 tx=0x7fb62001ca50 comp rx=0 tx=0).stop 2024-04-25T00:36:13.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.844+0000 7fb630f69640 1 -- 172.21.15.29:0/4052955891 reap_dead start 2024-04-25T00:36:13.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.844+0000 7fb6329cc640 1 -- 172.21.15.29:0/4052955891 shutdown_connections 2024-04-25T00:36:13.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.844+0000 7fb6329cc640 1 -- 172.21.15.29:0/4052955891 >> 172.21.15.29:0/4052955891 conn(0x7fb62c0d1e10 msgr2=0x7fb62c0d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:13.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.844+0000 7fb6329cc640 1 -- 172.21.15.29:0/4052955891 shutdown_connections 2024-04-25T00:36:13.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.844+0000 7fb6329cc640 1 -- 172.21.15.29:0/4052955891 wait complete. 2024-04-25T00:36:13.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.844+0000 7fb6329cc640 1 Processor -- start 2024-04-25T00:36:13.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.844+0000 7fb6329cc640 1 -- start start 2024-04-25T00:36:13.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb6329cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614003f40 0x7fb62c153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb6329cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0c7040 0x7fb62c0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb6329cc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb62c0afdd0 0x7fb62c148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb6329cc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c06d690 con 0x7fb62c0afdd0 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb6329cc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c0cf6c0 con 0x7fb614003f40 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb6329cc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb62c06d810 con 0x7fb62c0c7040 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb630f69640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb62c0afdd0 0x7fb62c148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb630f69640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb62c0afdd0 0x7fb62c148c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39884/0 (socket says 172.21.15.29:39884) 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb630f69640 1 -- 172.21.15.29:0/3804541767 learned_addr learned my addr 172.21.15.29:0/3804541767 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb630f69640 1 -- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0c7040 msgr2=0x7fb62c0c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb62b7fe640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0c7040 0x7fb62c0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb62bfff640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614003f40 0x7fb62c153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb630f69640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0c7040 0x7fb62c0c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb630f69640 1 -- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614003f40 msgr2=0x7fb62c153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb630f69640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614003f40 0x7fb62c153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:13.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb630f69640 1 -- 172.21.15.29:0/3804541767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62c0c7520 con 0x7fb62c0afdd0 2024-04-25T00:36:13.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb630f69640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb62c0afdd0 0x7fb62c148c80 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb62c071e90 tx=0x7fb624011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:13.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb628ff9640 1 -- 172.21.15.29:0/3804541767 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624019020 con 0x7fb62c0afdd0 2024-04-25T00:36:13.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb628ff9640 1 -- 172.21.15.29:0/3804541767 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb624003620 con 0x7fb62c0afdd0 2024-04-25T00:36:13.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb6329cc640 1 -- 172.21.15.29:0/3804541767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62c0c7800 con 0x7fb62c0afdd0 2024-04-25T00:36:13.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb628ff9640 1 -- 172.21.15.29:0/3804541767 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624003a50 con 0x7fb62c0afdd0 2024-04-25T00:36:13.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb6329cc640 1 -- 172.21.15.29:0/3804541767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb62c1531f0 con 0x7fb62c0afdd0 2024-04-25T00:36:13.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb6329cc640 1 -- 172.21.15.29:0/3804541767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb62c069270 con 0x7fb62c0afdd0 2024-04-25T00:36:13.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb628ff9640 1 -- 172.21.15.29:0/3804541767 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb62401c020 con 0x7fb62c0afdd0 2024-04-25T00:36:13.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb628ff9640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb618067c20 0x7fb61806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:13.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb628ff9640 1 -- 172.21.15.29:0/3804541767 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fb6240863c0 con 0x7fb62c0afdd0 2024-04-25T00:36:13.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb62bfff640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb618067c20 0x7fb61806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:13.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.848+0000 7fb62bfff640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb618067c20 0x7fb61806a0e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb61c002240 tx=0x7fb61c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:13.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:13.852+0000 7fb628ff9640 1 -- 172.21.15.29:0/3804541767 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb624023030 con 0x7fb62c0afdd0 2024-04-25T00:36:14.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.028+0000 7fb6329cc640 1 -- 172.21.15.29:0/3804541767 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb62c0c8ba0 con 0x7fb62c0afdd0 2024-04-25T00:36:14.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.028+0000 7fb628ff9640 1 -- 172.21.15.29:0/3804541767 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7fb624056b80 con 0x7fb62c0afdd0 2024-04-25T00:36:14.030 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:36:14.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.032+0000 7fb6329cc640 1 -- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb618067c20 msgr2=0x7fb61806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:14.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.032+0000 7fb6329cc640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb618067c20 0x7fb61806a0e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb61c002240 tx=0x7fb61c009040 comp rx=0 tx=0).stop 2024-04-25T00:36:14.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.032+0000 7fb6329cc640 1 -- 172.21.15.29:0/3804541767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb62c0afdd0 msgr2=0x7fb62c148c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:14.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.032+0000 7fb6329cc640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb62c0afdd0 0x7fb62c148c80 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb62c071e90 tx=0x7fb624011920 comp rx=0 tx=0).stop 2024-04-25T00:36:14.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.032+0000 7fb6329cc640 1 -- 172.21.15.29:0/3804541767 shutdown_connections 2024-04-25T00:36:14.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.032+0000 7fb6329cc640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb62c0afdd0 0x7fb62c148c80 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:14.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.032+0000 7fb6329cc640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb62c0c7040 0x7fb62c0c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:14.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.032+0000 7fb6329cc640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fb618067c20 0x7fb61806a0e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:14.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.032+0000 7fb6329cc640 1 --2- 172.21.15.29:0/3804541767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb614003f40 0x7fb62c153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:14.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.032+0000 7fb6329cc640 1 -- 172.21.15.29:0/3804541767 >> 172.21.15.29:0/3804541767 conn(0x7fb62c0d1e10 msgr2=0x7fb62c0b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.032+0000 7fb6329cc640 1 -- 172.21.15.29:0/3804541767 shutdown_connections 2024-04-25T00:36:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:14.032+0000 7fb6329cc640 1 -- 172.21.15.29:0/3804541767 wait complete. 2024-04-25T00:36:14.050 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-25T00:36:14.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:14.050 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:14.064 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34232]: fuse finished with error 0 and tester_r 0 2024-04-25T00:36:14.065 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:36:20.167 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:36:20.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:20.168 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:36:20.176 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:36:20.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.308+0000 7f4817ad2640 1 Processor -- start 2024-04-25T00:36:20.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.308+0000 7f4817ad2640 1 -- start start 2024-04-25T00:36:20.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f4817ad2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48100b2b70 0x7f4810151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f4817ad2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48100b3130 con 0x7f481015d980 2024-04-25T00:36:20.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f4817ad2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48100b3410 con 0x7f481006dce0 2024-04-25T00:36:20.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f4817ad2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f48100cf560 con 0x7f48100b2b70 2024-04-25T00:36:20.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481586e640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f481015d980 0x7f481006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55064/0 (socket says 172.21.15.29:55064) 2024-04-25T00:36:20.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481586e640 1 -- 172.21.15.29:0/2220399869 learned_addr learned my addr 172.21.15.29:0/2220399869 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:20.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481606f640 1 --2- 172.21.15.29:0/2220399869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48100b2b70 0x7f4810151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:20.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1178490384 0 0) 0x7f48100b3130 con 0x7f481015d980 2024-04-25T00:36:20.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47fc004640 con 0x7f481015d980 2024-04-25T00:36:20.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3046312294 0 0) 0x7f47fc004640 con 0x7f481015d980 2024-04-25T00:36:20.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48100b2b70 msgr2=0x7f4810151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:20.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 --2- 172.21.15.29:0/2220399869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48100b2b70 0x7f4810151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 >> v1:172.21.15.157:6789/0 conn(0x7f481006dce0 legacy=0x7f4810150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:20.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48100b36f0 con 0x7f481015d980 2024-04-25T00:36:20.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f480c001680 con 0x7f481015d980 2024-04-25T00:36:20.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 >> v1:172.21.15.29:6789/0 conn(0x7f481015d980 legacy=0x7f481006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:20.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 --2- 172.21.15.29:0/2220399869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47fc004b00 0x7f47fc004f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 --2- 172.21.15.29:0/2220399869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47fc005920 0x7f47fc005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 --2- 172.21.15.29:0/2220399869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48100b2b70 0x7f47fc0062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f48100cf560 con 0x7f47fc004b00 2024-04-25T00:36:20.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc00cbc0 con 0x7f47fc005920 2024-04-25T00:36:20.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f47fc0034d0 con 0x7f48100b2b70 2024-04-25T00:36:20.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481586e640 1 --2- 172.21.15.29:0/2220399869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47fc004b00 0x7f47fc004f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:20.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481506d640 1 --2- 172.21.15.29:0/2220399869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47fc005920 0x7f47fc005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:20.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481586e640 1 -- 172.21.15.29:0/2220399869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48100b2b70 msgr2=0x7f47fc0062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:20.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481586e640 1 --2- 172.21.15.29:0/2220399869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48100b2b70 0x7f47fc0062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481586e640 1 -- 172.21.15.29:0/2220399869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47fc005920 msgr2=0x7f47fc005da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:20.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481586e640 1 --2- 172.21.15.29:0/2220399869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47fc005920 0x7f47fc005da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481586e640 1 -- 172.21.15.29:0/2220399869 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47fc004640 con 0x7f47fc004b00 2024-04-25T00:36:20.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.312+0000 7f481506d640 1 --2- 172.21.15.29:0/2220399869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47fc005920 0x7f47fc005da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f481586e640 1 --2- 172.21.15.29:0/2220399869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47fc004b00 0x7f47fc004f20 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c190 tx=0x7f480c01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58773856144bac33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c007d00 con 0x7f47fc004b00 2024-04-25T00:36:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f480c0a9020 con 0x7f47fc004b00 2024-04-25T00:36:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f481486c640 1 -- 172.21.15.29:0/2220399869 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c005b00 con 0x7f47fc004b00 2024-04-25T00:36:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 -- 172.21.15.29:0/2220399869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47fc004b00 msgr2=0x7f47fc004f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 --2- 172.21.15.29:0/2220399869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47fc004b00 0x7f47fc004f20 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c190 tx=0x7f480c01cbf0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f481606f640 1 -- 172.21.15.29:0/2220399869 reap_dead start 2024-04-25T00:36:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 -- 172.21.15.29:0/2220399869 shutdown_connections 2024-04-25T00:36:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 -- 172.21.15.29:0/2220399869 >> 172.21.15.29:0/2220399869 conn(0x7f48100b1870 msgr2=0x7f48100b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 -- 172.21.15.29:0/2220399869 shutdown_connections 2024-04-25T00:36:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 -- 172.21.15.29:0/2220399869 wait complete. 2024-04-25T00:36:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 Processor -- start 2024-04-25T00:36:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 -- start start 2024-04-25T00:36:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc004b00 0x7f481014a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4810071b60 0x7f4810070f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48100714a0 0x7f4810152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f481015c4c0 con 0x7f48100714a0 2024-04-25T00:36:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f481015dd60 con 0x7f4810071b60 2024-04-25T00:36:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f4817ad2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f481015c640 con 0x7f47fc004b00 2024-04-25T00:36:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f481606f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48100714a0 0x7f4810152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f481606f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48100714a0 0x7f4810152330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48544/0 (socket says 172.21.15.29:48544) 2024-04-25T00:36:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f481606f640 1 -- 172.21.15.29:0/3348293636 learned_addr learned my addr 172.21.15.29:0/3348293636 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.316+0000 7f481586e640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc004b00 0x7f481014a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f481506d640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4810071b60 0x7f4810070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f481606f640 1 -- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc004b00 msgr2=0x7f481014a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f481606f640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc004b00 0x7f481014a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f481606f640 1 -- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4810071b60 msgr2=0x7f4810070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f481606f640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4810071b60 0x7f4810070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f481606f640 1 -- 172.21.15.29:0/3348293636 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48101498f0 con 0x7f48100714a0 2024-04-25T00:36:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f481506d640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4810071b60 0x7f4810070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f481606f640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48100714a0 0x7f4810152330 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f481006f150 tx=0x7f480800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f48067fc640 1 -- 172.21.15.29:0/3348293636 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480800bae0 con 0x7f48100714a0 2024-04-25T00:36:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f48067fc640 1 -- 172.21.15.29:0/3348293636 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f480801a070 con 0x7f48100714a0 2024-04-25T00:36:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f48067fc640 1 -- 172.21.15.29:0/3348293636 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808014400 con 0x7f48100714a0 2024-04-25T00:36:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f4817ad2640 1 -- 172.21.15.29:0/3348293636 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4810149bd0 con 0x7f48100714a0 2024-04-25T00:36:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f4817ad2640 1 -- 172.21.15.29:0/3348293636 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481014a190 con 0x7f48100714a0 2024-04-25T00:36:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f4817ad2640 1 -- 172.21.15.29:0/3348293636 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4810069270 con 0x7f48100714a0 2024-04-25T00:36:20.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f48067fc640 1 -- 172.21.15.29:0/3348293636 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f4808013070 con 0x7f48100714a0 2024-04-25T00:36:20.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f48067fc640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f47f4067c20 0x7f47f406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f48067fc640 1 -- 172.21.15.29:0/3348293636 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f48080863c0 con 0x7f48100714a0 2024-04-25T00:36:20.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f481586e640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f47f4067c20 0x7f47f406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:20.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.320+0000 7f481586e640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f47f4067c20 0x7f47f406a0e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f480c0aa000 tx=0x7f480c0b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:20.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.324+0000 7f48067fc640 1 -- 172.21.15.29:0/3348293636 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4808024020 con 0x7f48100714a0 2024-04-25T00:36:20.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.520+0000 7f4817ad2640 1 -- 172.21.15.29:0/3348293636 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48100cf910 con 0x7f48100714a0 2024-04-25T00:36:20.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.520+0000 7f48067fc640 1 -- 172.21.15.29:0/3348293636 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7f48080569a0 con 0x7f48100714a0 2024-04-25T00:36:20.524 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:20.525 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/524224365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":524224365},{"type":"v1","addr":"172.21.15.157:6835","nonce":524224365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1007000548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1007000548},{"type":"v1","addr":"172.21.15.157:6837","nonce":1007000548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6810,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2091414875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2091414875},{"type":"v1","addr":"172.21.15.29:6833","nonce":2091414875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:49.528297+0000","modified":"2024-04-25T00:35:52.569008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1111146974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1111146974},{"type":"v1","addr":"172.21.15.29:6835","nonce":1111146974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6822,"qdb_cluster":[6822]},"id":5}]} 2024-04-25T00:36:20.525 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 44 2024-04-25T00:36:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.524+0000 7f4817ad2640 1 -- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f47f4067c20 msgr2=0x7f47f406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.524+0000 7f4817ad2640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f47f4067c20 0x7f47f406a0e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f480c0aa000 tx=0x7f480c0b1040 comp rx=0 tx=0).stop 2024-04-25T00:36:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.524+0000 7f4817ad2640 1 -- 172.21.15.29:0/3348293636 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48100714a0 msgr2=0x7f4810152330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.524+0000 7f4817ad2640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48100714a0 0x7f4810152330 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f481006f150 tx=0x7f480800b480 comp rx=0 tx=0).stop 2024-04-25T00:36:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.524+0000 7f4817ad2640 1 -- 172.21.15.29:0/3348293636 shutdown_connections 2024-04-25T00:36:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.524+0000 7f4817ad2640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48100714a0 0x7f4810152330 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.524+0000 7f4817ad2640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f47f4067c20 0x7f47f406a0e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.524+0000 7f4817ad2640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4810071b60 0x7f4810070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.524+0000 7f4817ad2640 1 --2- 172.21.15.29:0/3348293636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc004b00 0x7f481014a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.524+0000 7f4817ad2640 1 -- 172.21.15.29:0/3348293636 >> 172.21.15.29:0/3348293636 conn(0x7f48100b1870 msgr2=0x7f4810159180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.524+0000 7f4817ad2640 1 -- 172.21.15.29:0/3348293636 shutdown_connections 2024-04-25T00:36:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.524+0000 7f4817ad2640 1 -- 172.21.15.29:0/3348293636 wait complete. 2024-04-25T00:36:20.543 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:36:20.678 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-04-25T00:36:20.679 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-04-25T00:36:20.679 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol41740 2024-04-25T00:36:20.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe7979640 1 Processor -- start 2024-04-25T00:36:20.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe7979640 1 -- start start 2024-04-25T00:36:20.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe7979640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdbe0162420 0x7fdbe0070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe7979640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdbe00b3280 0x7fdbe006cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe7979640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbe0071910 con 0x7fdbe00b4e00 2024-04-25T00:36:20.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe7979640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdbe0071bf0 con 0x7fdbe00b3280 2024-04-25T00:36:20.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe7979640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdbe0071ed0 con 0x7fdbe0162420 2024-04-25T00:36:20.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe4f14640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdbe00b4e00 0x7fdbe00b2b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55074/0 (socket says 172.21.15.29:55074) 2024-04-25T00:36:20.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe4f14640 1 -- 172.21.15.29:0/4283313322 learned_addr learned my addr 172.21.15.29:0/4283313322 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:20.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe5f16640 1 --2- 172.21.15.29:0/4283313322 >> v2:172.21.15.29:3300/0 conn(0x7fdbe00b3280 0x7fdbe006cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:20.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbd3fff640 1 -- 172.21.15.29:0/4283313322 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1131848257 0 0) 0x7fdbe0071910 con 0x7fdbe00b4e00 2024-04-25T00:36:20.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbd3fff640 1 -- 172.21.15.29:0/4283313322 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbcc003530 con 0x7fdbe00b4e00 2024-04-25T00:36:20.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe5f16640 1 -- 172.21.15.29:0/4283313322 >> v2:172.21.15.157:3300/0 conn(0x7fdbe0162420 msgr2=0x7fdbe0070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:20.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe5f16640 1 --2- 172.21.15.29:0/4283313322 >> v2:172.21.15.157:3300/0 conn(0x7fdbe0162420 0x7fdbe0070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe5f16640 1 -- 172.21.15.29:0/4283313322 >> v1:172.21.15.29:6789/0 conn(0x7fdbe00b4e00 legacy=0x7fdbe00b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:20.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.812+0000 7fdbe5f16640 1 -- 172.21.15.29:0/4283313322 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe00721b0 con 0x7fdbe00b3280 2024-04-25T00:36:20.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe5f16640 1 --2- 172.21.15.29:0/4283313322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00b3280 0x7fdbe006cf80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0067e70 tx=0x7fdbd801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440d48d2781c10c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:20.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbd3fff640 1 -- 172.21.15.29:0/4283313322 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80b1070 con 0x7fdbe00b3280 2024-04-25T00:36:20.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbd3fff640 1 -- 172.21.15.29:0/4283313322 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd80ac5f0 con 0x7fdbe00b3280 2024-04-25T00:36:20.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbd3fff640 1 -- 172.21.15.29:0/4283313322 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe00716d0 con 0x7fdbe00b3280 2024-04-25T00:36:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 -- 172.21.15.29:0/4283313322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00b3280 msgr2=0x7fdbe006cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 --2- 172.21.15.29:0/4283313322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00b3280 0x7fdbe006cf80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0067e70 tx=0x7fdbd801c480 comp rx=0 tx=0).stop 2024-04-25T00:36:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 -- 172.21.15.29:0/4283313322 shutdown_connections 2024-04-25T00:36:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 --2- 172.21.15.29:0/4283313322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00b3280 0x7fdbe006cf80 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 --2- 172.21.15.29:0/4283313322 >> v2:172.21.15.157:3300/0 conn(0x7fdbe0162420 0x7fdbe0070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 -- 172.21.15.29:0/4283313322 >> 172.21.15.29:0/4283313322 conn(0x7fdbe00b1870 msgr2=0x7fdbe00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 -- 172.21.15.29:0/4283313322 shutdown_connections 2024-04-25T00:36:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 -- 172.21.15.29:0/4283313322 wait complete. 2024-04-25T00:36:20.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 Processor -- start 2024-04-25T00:36:20.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 -- start start 2024-04-25T00:36:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbe00b3280 0x7fdbe0160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00b4e00 0x7fdbe0145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe0162420 0x7fdbe0164bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe006e060 con 0x7fdbe00b4e00 2024-04-25T00:36:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe006dee0 con 0x7fdbe0162420 2024-04-25T00:36:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe7979640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe006e1e0 con 0x7fdbe00b3280 2024-04-25T00:36:20.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe4f14640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00b4e00 0x7fdbe0145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:20.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe4f14640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00b4e00 0x7fdbe0145860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48560/0 (socket says 172.21.15.29:48560) 2024-04-25T00:36:20.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe4f14640 1 -- 172.21.15.29:0/951014958 learned_addr learned my addr 172.21.15.29:0/951014958 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:20.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe5f16640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe0162420 0x7fdbe0164bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:20.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe4f14640 1 -- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbe00b3280 msgr2=0x7fdbe0160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:20.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe5715640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbe00b3280 0x7fdbe0160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:20.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe4f14640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbe00b3280 0x7fdbe0160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe4f14640 1 -- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe0162420 msgr2=0x7fdbe0164bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:20.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.816+0000 7fdbe4f14640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe0162420 0x7fdbe0164bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:20.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbe4f14640 1 -- 172.21.15.29:0/951014958 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe0158a60 con 0x7fdbe00b4e00 2024-04-25T00:36:20.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbe5f16640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe0162420 0x7fdbe0164bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:20.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbe4f14640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00b4e00 0x7fdbe0145860 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400c450 tx=0x7fdbd400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:20.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbd1ffb640 1 -- 172.21.15.29:0/951014958 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4017070 con 0x7fdbe00b4e00 2024-04-25T00:36:20.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbd1ffb640 1 -- 172.21.15.29:0/951014958 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd4005cf0 con 0x7fdbe00b4e00 2024-04-25T00:36:20.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbe7979640 1 -- 172.21.15.29:0/951014958 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe0158d40 con 0x7fdbe00b4e00 2024-04-25T00:36:20.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbd1ffb640 1 -- 172.21.15.29:0/951014958 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4007290 con 0x7fdbe00b4e00 2024-04-25T00:36:20.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbe7979640 1 -- 172.21.15.29:0/951014958 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe0159300 con 0x7fdbe00b4e00 2024-04-25T00:36:20.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbe7979640 1 -- 172.21.15.29:0/951014958 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe0069270 con 0x7fdbe00b4e00 2024-04-25T00:36:20.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbd1ffb640 1 -- 172.21.15.29:0/951014958 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fdbd401a070 con 0x7fdbe00b4e00 2024-04-25T00:36:20.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbd1ffb640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdbc4067c20 0x7fdbc406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:20.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbd1ffb640 1 -- 172.21.15.29:0/951014958 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7fdbd40855e0 con 0x7fdbe00b4e00 2024-04-25T00:36:20.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbe5715640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdbc4067c20 0x7fdbc406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:20.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.820+0000 7fdbe5715640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdbc4067c20 0x7fdbc406a0e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc007dc0 tx=0x7fdbdc0072f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:20.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:20.824+0000 7fdbd1ffb640 1 -- 172.21.15.29:0/951014958 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdbd4055cd0 con 0x7fdbe00b4e00 2024-04-25T00:36:21.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.012+0000 7fdbe7979640 1 -- 172.21.15.29:0/951014958 --> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol41740", "target": ["mon-mgr", ""]}) -- 0x7fdbe0072fa0 con 0x7fdbc4067c20 2024-04-25T00:36:21.025 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:21.024+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:21.025 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:21.024+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:21.025 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:21.024+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:21.025 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:21.024+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:21.025 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:21.024+0000 7f47f785d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:21.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.080+0000 7fdbd1ffb640 1 -- 172.21.15.29:0/951014958 <== mgr.4100 v2:172.21.15.157:6832/3474584311 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdbe0072fa0 con 0x7fdbc4067c20 2024-04-25T00:36:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.084+0000 7fdbe7979640 1 -- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdbc4067c20 msgr2=0x7fdbc406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.084+0000 7fdbe7979640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdbc4067c20 0x7fdbc406a0e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc007dc0 tx=0x7fdbdc0072f0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.084+0000 7fdbe7979640 1 -- 172.21.15.29:0/951014958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00b4e00 msgr2=0x7fdbe0145860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.084+0000 7fdbe7979640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00b4e00 0x7fdbe0145860 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400c450 tx=0x7fdbd400c910 comp rx=0 tx=0).stop 2024-04-25T00:36:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.084+0000 7fdbe7979640 1 -- 172.21.15.29:0/951014958 shutdown_connections 2024-04-25T00:36:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.084+0000 7fdbe7979640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fdbc4067c20 0x7fdbc406a0e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.084+0000 7fdbe7979640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe0162420 0x7fdbe0164bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.084+0000 7fdbe7979640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbe00b4e00 0x7fdbe0145860 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.084+0000 7fdbe7979640 1 --2- 172.21.15.29:0/951014958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbe00b3280 0x7fdbe0160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.084+0000 7fdbe7979640 1 -- 172.21.15.29:0/951014958 >> 172.21.15.29:0/951014958 conn(0x7fdbe00b1870 msgr2=0x7fdbe00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:21.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.084+0000 7fdbe7979640 1 -- 172.21.15.29:0/951014958 shutdown_connections 2024-04-25T00:36:21.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.084+0000 7fdbe7979640 1 -- 172.21.15.29:0/951014958 wait complete. 2024-04-25T00:36:21.104 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:36:21.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd737b93640 1 Processor -- start 2024-04-25T00:36:21.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd737b93640 1 -- start start 2024-04-25T00:36:21.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd737b93640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd73015e3c0 0x7fd7300d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd737b93640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd73006c530 0x7fd73006cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd737b93640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73006d150 0x7fd7300ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd737b93640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd7300b4bc0 con 0x7fd73006c530 2024-04-25T00:36:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd737b93640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd73006d5a0 con 0x7fd73015e3c0 2024-04-25T00:36:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd737b93640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd73015e7a0 con 0x7fd73006d150 2024-04-25T00:36:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd73512e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd73006c530 0x7fd73006cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd73512e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd73006c530 0x7fd73006cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48562/0 (socket says 172.21.15.29:48562) 2024-04-25T00:36:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd73512e640 1 -- 172.21.15.29:0/3492357581 learned_addr learned my addr 172.21.15.29:0/3492357581 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd73592f640 1 --2- 172.21.15.29:0/3492357581 >> v2:172.21.15.157:3300/0 conn(0x7fd73015e3c0 0x7fd7300d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd73512e640 1 -- 172.21.15.29:0/3492357581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73006d150 msgr2=0x7fd7300ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd736130640 1 --2- 172.21.15.29:0/3492357581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73006d150 0x7fd7300ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd73512e640 1 --2- 172.21.15.29:0/3492357581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73006d150 0x7fd7300ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd73512e640 1 -- 172.21.15.29:0/3492357581 >> v2:172.21.15.157:3300/0 conn(0x7fd73015e3c0 msgr2=0x7fd7300d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd73512e640 1 --2- 172.21.15.29:0/3492357581 >> v2:172.21.15.157:3300/0 conn(0x7fd73015e3c0 0x7fd7300d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd73512e640 1 -- 172.21.15.29:0/3492357581 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7300d10f0 con 0x7fd73006c530 2024-04-25T00:36:21.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.232+0000 7fd73592f640 1 --2- 172.21.15.29:0/3492357581 >> v2:172.21.15.157:3300/0 conn(0x7fd73015e3c0 0x7fd7300d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:21.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.236+0000 7fd73512e640 1 --2- 172.21.15.29:0/3492357581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd73006c530 0x7fd73006cc10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd7300718d0 tx=0x7fd72401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9a34a66d8633af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:21.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.236+0000 7fd73492d640 1 -- 172.21.15.29:0/3492357581 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240b0070 con 0x7fd73006c530 2024-04-25T00:36:21.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.236+0000 7fd73492d640 1 -- 172.21.15.29:0/3492357581 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7240ab600 con 0x7fd73006c530 2024-04-25T00:36:21.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.236+0000 7fd73492d640 1 -- 172.21.15.29:0/3492357581 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240ab9d0 con 0x7fd73006c530 2024-04-25T00:36:21.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.236+0000 7fd737b93640 1 -- 172.21.15.29:0/3492357581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd73006c530 msgr2=0x7fd73006cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.236+0000 7fd737b93640 1 --2- 172.21.15.29:0/3492357581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd73006c530 0x7fd73006cc10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd7300718d0 tx=0x7fd72401c260 comp rx=0 tx=0).stop 2024-04-25T00:36:21.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.236+0000 7fd737b93640 1 -- 172.21.15.29:0/3492357581 shutdown_connections 2024-04-25T00:36:21.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.236+0000 7fd737b93640 1 --2- 172.21.15.29:0/3492357581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd73006c530 0x7fd73006cc10 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.236+0000 7fd737b93640 1 --2- 172.21.15.29:0/3492357581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73006d150 0x7fd7300ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.236+0000 7fd737b93640 1 --2- 172.21.15.29:0/3492357581 >> v2:172.21.15.157:3300/0 conn(0x7fd73015e3c0 0x7fd7300d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.236+0000 7fd737b93640 1 -- 172.21.15.29:0/3492357581 >> 172.21.15.29:0/3492357581 conn(0x7fd7300d1d70 msgr2=0x7fd7300d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:21.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.236+0000 7fd737b93640 1 -- 172.21.15.29:0/3492357581 shutdown_connections 2024-04-25T00:36:21.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.236+0000 7fd737b93640 1 -- 172.21.15.29:0/3492357581 wait complete. 2024-04-25T00:36:21.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd737b93640 1 Processor -- start 2024-04-25T00:36:21.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd737b93640 1 -- start start 2024-04-25T00:36:21.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd737b93640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73006c530 0x7fd7301541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd737b93640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd73006d150 0x7fd73015b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd737b93640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd73015e3c0 0x7fd730155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd737b93640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7300cd040 con 0x7fd73006d150 2024-04-25T00:36:21.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd737b93640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd73006c9a0 con 0x7fd73015e3c0 2024-04-25T00:36:21.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd737b93640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd7300cd1c0 con 0x7fd73006c530 2024-04-25T00:36:21.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd73512e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd73006d150 0x7fd73015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd73512e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd73006d150 0x7fd73015b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48568/0 (socket says 172.21.15.29:48568) 2024-04-25T00:36:21.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd73512e640 1 -- 172.21.15.29:0/2409662842 learned_addr learned my addr 172.21.15.29:0/2409662842 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd736130640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd73015e3c0 0x7fd730155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd73592f640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73006c530 0x7fd7301541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd73512e640 1 -- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73006c530 msgr2=0x7fd7301541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd73512e640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73006c530 0x7fd7301541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd73512e640 1 -- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd73015e3c0 msgr2=0x7fd730155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd73512e640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd73015e3c0 0x7fd730155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd73512e640 1 -- 172.21.15.29:0/2409662842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73015bf00 con 0x7fd73006d150 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd73592f640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73006c530 0x7fd7301541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd73512e640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd73006d150 0x7fd73015b600 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd72401a910 tx=0x7fd724019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd72e7fc640 1 -- 172.21.15.29:0/2409662842 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240b9070 con 0x7fd73006d150 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd72e7fc640 1 -- 172.21.15.29:0/2409662842 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7240b4600 con 0x7fd73006d150 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd72e7fc640 1 -- 172.21.15.29:0/2409662842 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240b0040 con 0x7fd73006d150 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd737b93640 1 -- 172.21.15.29:0/2409662842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73015c1e0 con 0x7fd73006d150 2024-04-25T00:36:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd737b93640 1 -- 172.21.15.29:0/2409662842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73015c7a0 con 0x7fd73006d150 2024-04-25T00:36:21.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd737b93640 1 -- 172.21.15.29:0/2409662842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd730069270 con 0x7fd73006d150 2024-04-25T00:36:21.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd72e7fc640 1 -- 172.21.15.29:0/2409662842 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fd7240ab4c0 con 0x7fd73006d150 2024-04-25T00:36:21.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd72e7fc640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd71c067c20 0x7fd71c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.240+0000 7fd72e7fc640 1 -- 172.21.15.29:0/2409662842 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7fd72411f5d0 con 0x7fd73006d150 2024-04-25T00:36:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.244+0000 7fd73592f640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd71c067c20 0x7fd71c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.244+0000 7fd73592f640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd71c067c20 0x7fd71c06a0e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd7300c6c80 tx=0x7fd728007430 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:21.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.248+0000 7fd72e7fc640 1 -- 172.21.15.29:0/2409662842 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd724016110 con 0x7fd73006d150 2024-04-25T00:36:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd737b93640 1 -- 172.21.15.29:0/2409662842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7300c6e90 con 0x7fd73006d150 2024-04-25T00:36:21.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd72e7fc640 1 -- 172.21.15.29:0/2409662842 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7fd7240efbb0 con 0x7fd73006d150 2024-04-25T00:36:21.446 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:21.446 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/524224365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":524224365},{"type":"v1","addr":"172.21.15.157:6835","nonce":524224365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1007000548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1007000548},{"type":"v1","addr":"172.21.15.157:6837","nonce":1007000548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6810,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2091414875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2091414875},{"type":"v1","addr":"172.21.15.29:6833","nonce":2091414875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:49.528297+0000","modified":"2024-04-25T00:35:52.569008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1111146974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1111146974},{"type":"v1","addr":"172.21.15.29:6835","nonce":1111146974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6822,"qdb_cluster":[6822]},"id":5}]} 2024-04-25T00:36:21.446 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 44 2024-04-25T00:36:21.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd737b93640 1 -- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd71c067c20 msgr2=0x7fd71c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd737b93640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd71c067c20 0x7fd71c06a0e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd7300c6c80 tx=0x7fd728007430 comp rx=0 tx=0).stop 2024-04-25T00:36:21.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd737b93640 1 -- 172.21.15.29:0/2409662842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd73006d150 msgr2=0x7fd73015b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd737b93640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd73006d150 0x7fd73015b600 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd72401a910 tx=0x7fd724019000 comp rx=0 tx=0).stop 2024-04-25T00:36:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd737b93640 1 -- 172.21.15.29:0/2409662842 shutdown_connections 2024-04-25T00:36:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd737b93640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7fd71c067c20 0x7fd71c06a0e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd737b93640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd73015e3c0 0x7fd730155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd737b93640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd73006d150 0x7fd73015b600 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd737b93640 1 --2- 172.21.15.29:0/2409662842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73006c530 0x7fd7301541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd737b93640 1 -- 172.21.15.29:0/2409662842 >> 172.21.15.29:0/2409662842 conn(0x7fd7300d1d70 msgr2=0x7fd7300b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd737b93640 1 -- 172.21.15.29:0/2409662842 shutdown_connections 2024-04-25T00:36:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.444+0000 7fd737b93640 1 -- 172.21.15.29:0/2409662842 wait complete. 2024-04-25T00:36:21.466 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:36:21.597 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":6307,"entity":{"name":{"type":"client","num":6307},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4165584909}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.55199444399999997,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.972827227377451,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6307 172.21.15.157:0/4165584909","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"27599","root":"/"}}] 2024-04-25T00:36:21.598 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27599", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4165584909, "type": "any" }, "name": { "num": 6307, "type": "client" } }, "id": 6307, "inst": "client.6307 172.21.15.157:0/4165584909", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.97282722737745 }, "state": "open", "uptime": 0.551994444 } ] 2024-04-25T00:36:21.598 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T00:36:21.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f28734a7640 1 Processor -- start 2024-04-25T00:36:21.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f28734a7640 1 -- start start 2024-04-25T00:36:21.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f28734a7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f286c0b3140 0x7f286c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f28734a7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f286c0b41e0 con 0x7f286c0b3c30 2024-04-25T00:36:21.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f28734a7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f286c06d960 con 0x7f286c0b4b10 2024-04-25T00:36:21.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f28734a7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f286c06dc40 con 0x7f286c0b3140 2024-04-25T00:36:21.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2871a44640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f286c0b3c30 0x7f286c0ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55078/0 (socket says 172.21.15.29:55078) 2024-04-25T00:36:21.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2871a44640 1 -- 172.21.15.29:0/1988844322 learned_addr learned my addr 172.21.15.29:0/1988844322 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:21.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2871243640 1 --2- 172.21.15.29:0/1988844322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f286c0b3140 0x7f286c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4281814721 0 0) 0x7f286c0b41e0 con 0x7f286c0b3c30 2024-04-25T00:36:21.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2858003530 con 0x7f286c0b3c30 2024-04-25T00:36:21.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1337963454 0 0) 0x7f2858003530 con 0x7f286c0b3c30 2024-04-25T00:36:21.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f286c0b3140 msgr2=0x7f286c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 --2- 172.21.15.29:0/1988844322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f286c0b3140 0x7f286c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 >> v1:172.21.15.157:6789/0 conn(0x7f286c0b4b10 legacy=0x7f286c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c06df20 con 0x7f286c0b3c30 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2864001a00 con 0x7f286c0b3c30 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 >> v1:172.21.15.29:6789/0 conn(0x7f286c0b3c30 legacy=0x7f286c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 --2- 172.21.15.29:0/1988844322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f286c0b3140 0x7f28580039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 --2- 172.21.15.29:0/1988844322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2858003f70 0x7f2858004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 --2- 172.21.15.29:0/1988844322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2858004d50 0x7f28580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f286c06dc40 con 0x7f2858003f70 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f285800bab0 con 0x7f2858004d50 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f285800bcb0 con 0x7f286c0b3140 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2870a42640 1 --2- 172.21.15.29:0/1988844322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2858003f70 0x7f2858004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2871a44640 1 --2- 172.21.15.29:0/1988844322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2858004d50 0x7f28580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2871243640 1 --2- 172.21.15.29:0/1988844322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f286c0b3140 0x7f28580039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2870a42640 1 -- 172.21.15.29:0/1988844322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f286c0b3140 msgr2=0x7f28580039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:21.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2870a42640 1 --2- 172.21.15.29:0/1988844322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f286c0b3140 0x7f28580039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2870a42640 1 -- 172.21.15.29:0/1988844322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2858004d50 msgr2=0x7f28580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:21.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2870a42640 1 --2- 172.21.15.29:0/1988844322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2858004d50 0x7f28580051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2870a42640 1 -- 172.21.15.29:0/1988844322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2858003530 con 0x7f2858003f70 2024-04-25T00:36:21.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.708+0000 7f2871a44640 1 --2- 172.21.15.29:0/1988844322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2858004d50 0x7f28580051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:21.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.712+0000 7f2870a42640 1 --2- 172.21.15.29:0/1988844322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2858003f70 0x7f2858004810 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f2868001780 tx=0x7f286801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec3a7b9c364740a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:21.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.712+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28680013a0 con 0x7f2858003f70 2024-04-25T00:36:21.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.712+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28680a9070 con 0x7f2858003f70 2024-04-25T00:36:21.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.712+0000 7f2863fff640 1 -- 172.21.15.29:0/1988844322 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868003600 con 0x7f2858003f70 2024-04-25T00:36:21.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.712+0000 7f28734a7640 1 -- 172.21.15.29:0/1988844322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2858003f70 msgr2=0x7f2858004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.712+0000 7f28734a7640 1 --2- 172.21.15.29:0/1988844322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2858003f70 0x7f2858004810 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f2868001780 tx=0x7f286801c420 comp rx=0 tx=0).stop 2024-04-25T00:36:21.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.712+0000 7f2871a44640 1 -- 172.21.15.29:0/1988844322 reap_dead start 2024-04-25T00:36:21.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.712+0000 7f28734a7640 1 -- 172.21.15.29:0/1988844322 shutdown_connections 2024-04-25T00:36:21.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.712+0000 7f28734a7640 1 -- 172.21.15.29:0/1988844322 >> 172.21.15.29:0/1988844322 conn(0x7f286c0b1870 msgr2=0x7f286c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:21.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.712+0000 7f28734a7640 1 -- 172.21.15.29:0/1988844322 shutdown_connections 2024-04-25T00:36:21.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.712+0000 7f28734a7640 1 -- 172.21.15.29:0/1988844322 wait complete. 2024-04-25T00:36:21.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.712+0000 7f28734a7640 1 Processor -- start 2024-04-25T00:36:21.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.712+0000 7f28734a7640 1 -- start start 2024-04-25T00:36:21.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f28734a7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2858003f70 0x7f286c1f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f28734a7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286c071f20 0x7f286c13d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f28734a7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286c072380 0x7f286c13d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f28734a7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f286c145a90 con 0x7f286c072380 2024-04-25T00:36:21.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f28734a7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f286c145910 con 0x7f286c071f20 2024-04-25T00:36:21.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f28734a7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f286c06e440 con 0x7f2858003f70 2024-04-25T00:36:21.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2871a44640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286c072380 0x7f286c13d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2871a44640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286c072380 0x7f286c13d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48580/0 (socket says 172.21.15.29:48580) 2024-04-25T00:36:21.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2871a44640 1 -- 172.21.15.29:0/2548935273 learned_addr learned my addr 172.21.15.29:0/2548935273 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:21.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2871243640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2858003f70 0x7f286c1f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2870a42640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286c071f20 0x7f286c13d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2871a44640 1 -- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2858003f70 msgr2=0x7f286c1f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2871a44640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2858003f70 0x7f286c1f6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2871a44640 1 -- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286c071f20 msgr2=0x7f286c13d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2871a44640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286c071f20 0x7f286c13d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2871a44640 1 -- 172.21.15.29:0/2548935273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c13a710 con 0x7f286c072380 2024-04-25T00:36:21.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2871a44640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286c072380 0x7f286c13d5e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f2864002060 tx=0x7f2864011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:21.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2861ffb640 1 -- 172.21.15.29:0/2548935273 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864010040 con 0x7f286c072380 2024-04-25T00:36:21.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2861ffb640 1 -- 172.21.15.29:0/2548935273 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f286401a070 con 0x7f286c072380 2024-04-25T00:36:21.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f28734a7640 1 -- 172.21.15.29:0/2548935273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286c070e80 con 0x7f286c072380 2024-04-25T00:36:21.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2861ffb640 1 -- 172.21.15.29:0/2548935273 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286400faa0 con 0x7f286c072380 2024-04-25T00:36:21.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f28734a7640 1 -- 172.21.15.29:0/2548935273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286c071440 con 0x7f286c072380 2024-04-25T00:36:21.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f28734a7640 1 -- 172.21.15.29:0/2548935273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286c069270 con 0x7f286c072380 2024-04-25T00:36:21.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2861ffb640 1 -- 172.21.15.29:0/2548935273 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f286400fc40 con 0x7f286c072380 2024-04-25T00:36:21.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2861ffb640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2850067c20 0x7f285006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2861ffb640 1 -- 172.21.15.29:0/2548935273 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f2864089130 con 0x7f286c072380 2024-04-25T00:36:21.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.716+0000 7f2871243640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2850067c20 0x7f285006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.720+0000 7f2871243640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2850067c20 0x7f285006a0e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f285c009a30 tx=0x7f285c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:21.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.720+0000 7f2861ffb640 1 -- 172.21.15.29:0/2548935273 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2864059820 con 0x7f286c072380 2024-04-25T00:36:21.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.916+0000 7f28734a7640 1 -- 172.21.15.29:0/2548935273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f286c141940 con 0x7f286c072380 2024-04-25T00:36:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.928+0000 7f2861ffb640 1 -- 172.21.15.29:0/2548935273 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v11) ==== 81+0+453589 (secure 0 0 0) 0x7f2864059120 con 0x7f286c072380 2024-04-25T00:36:21.935 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:21.936 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:36:21.936 INFO:teuthology.orchestra.run.smithi029.stdout: "epoch": 11, 2024-04-25T00:36:21.936 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.936 INFO:teuthology.orchestra.run.smithi029.stdout: "active_gid": 4100, 2024-04-25T00:36:21.936 INFO:teuthology.orchestra.run.smithi029.stdout: "active_name": "x", 2024-04-25T00:36:21.936 INFO:teuthology.orchestra.run.smithi029.stdout: "active_addrs": { 2024-04-25T00:36:21.936 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-04-25T00:36:21.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:21.936 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-04-25T00:36:21.936 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.157:6832", 2024-04-25T00:36:21.936 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3474584311 2024-04-25T00:36:21.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:21.937 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:36:21.937 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.157:6833", 2024-04-25T00:36:21.937 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3474584311 2024-04-25T00:36:21.937 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:21.937 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:21.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.937 INFO:teuthology.orchestra.run.smithi029.stdout: "active_addr": "172.21.15.157:6833/3474584311", 2024-04-25T00:36:21.937 INFO:teuthology.orchestra.run.smithi029.stdout: "active_change": "2024-04-25T00:31:17.431955+0000", 2024-04-25T00:36:21.937 INFO:teuthology.orchestra.run.smithi029.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T00:36:21.938 INFO:teuthology.orchestra.run.smithi029.stdout: "available": true, 2024-04-25T00:36:21.938 INFO:teuthology.orchestra.run.smithi029.stdout: "standbys": [ 2024-04-25T00:36:21.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:21.938 INFO:teuthology.orchestra.run.smithi029.stdout: "gid": 4103, 2024-04-25T00:36:21.938 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "y", 2024-04-25T00:36:21.938 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_features": 4540701547738038271, 2024-04-25T00:36:21.938 INFO:teuthology.orchestra.run.smithi029.stdout: "available_modules": [ 2024-04-25T00:36:21.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:21.938 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "alerts", 2024-04-25T00:36:21.938 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:21.938 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:21.938 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:21.938 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-04-25T00:36:21.939 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-04-25T00:36:21.939 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:21.939 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.939 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.939 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-04-25T00:36:21.939 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.939 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.939 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.939 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:36:21.939 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.939 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.939 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:21.940 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:21.941 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:21.941 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:21.941 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:21.941 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.941 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.941 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.941 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:21.941 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:21.941 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:21.941 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.941 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.941 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:21.942 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.942 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.942 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.942 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:21.942 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.942 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.942 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.942 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:21.942 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:21.942 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.943 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.943 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.943 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:21.943 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.943 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.943 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:21.943 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:21.943 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:21.943 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:21.944 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:21.944 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:21.944 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:21.944 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:21.944 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:21.944 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.944 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.944 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.944 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:21.945 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:21.945 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:21.945 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.945 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.945 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:21.945 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.946 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.946 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.946 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:21.946 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.946 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.946 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.946 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_destination": { 2024-04-25T00:36:21.946 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_destination", 2024-04-25T00:36:21.946 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.946 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.947 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.947 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:21.947 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.947 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.947 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.947 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:36:21.947 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.947 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.947 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.947 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_from_name": { 2024-04-25T00:36:21.948 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_from_name", 2024-04-25T00:36:21.948 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.948 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.948 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.948 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "Ceph", 2024-04-25T00:36:21.948 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.948 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.948 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.948 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Email From: name", 2024-04-25T00:36:21.948 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.948 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.948 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.949 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_host": { 2024-04-25T00:36:21.949 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_host", 2024-04-25T00:36:21.949 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.949 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.949 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.949 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:21.949 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.949 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.949 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.949 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP server", 2024-04-25T00:36:21.949 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.949 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.950 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.950 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_password": { 2024-04-25T00:36:21.950 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_password", 2024-04-25T00:36:21.950 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.950 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.950 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.950 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:21.950 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.950 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.950 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.950 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Password to authenticate with", 2024-04-25T00:36:21.950 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.951 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.951 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.951 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_port": { 2024-04-25T00:36:21.951 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_port", 2024-04-25T00:36:21.951 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:21.951 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.951 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.951 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "465", 2024-04-25T00:36:21.951 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.951 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.951 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.951 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP port", 2024-04-25T00:36:21.952 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.952 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.952 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.952 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_sender": { 2024-04-25T00:36:21.952 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_sender", 2024-04-25T00:36:21.952 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.952 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.952 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.952 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:21.952 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.952 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.953 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.953 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:36:21.953 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.953 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.953 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.953 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_ssl": { 2024-04-25T00:36:21.953 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_ssl", 2024-04-25T00:36:21.953 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:21.953 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.954 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.954 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:21.954 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.954 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.954 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.954 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:36:21.954 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.954 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.954 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.954 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_user": { 2024-04-25T00:36:21.955 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_user", 2024-04-25T00:36:21.955 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.955 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.955 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.955 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:21.955 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.955 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.955 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.955 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "User to authenticate as", 2024-04-25T00:36:21.955 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.955 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.955 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.955 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:21.956 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:21.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:21.956 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "balancer", 2024-04-25T00:36:21.956 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:21.956 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:21.956 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:21.956 INFO:teuthology.orchestra.run.smithi029.stdout: "active": { 2024-04-25T00:36:21.956 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "active", 2024-04-25T00:36:21.956 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:21.956 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.956 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.956 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:21.957 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.957 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.957 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.957 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:36:21.957 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.957 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.957 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.957 INFO:teuthology.orchestra.run.smithi029.stdout: "begin_time": { 2024-04-25T00:36:21.957 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "begin_time", 2024-04-25T00:36:21.958 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.958 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.958 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.958 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0000", 2024-04-25T00:36:21.958 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.958 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.958 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.958 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:36:21.958 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:36:21.958 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.959 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.959 INFO:teuthology.orchestra.run.smithi029.stdout: "begin_weekday": { 2024-04-25T00:36:21.959 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "begin_weekday", 2024-04-25T00:36:21.959 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-04-25T00:36:21.959 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.959 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.959 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-04-25T00:36:21.959 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0", 2024-04-25T00:36:21.960 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "6", 2024-04-25T00:36:21.960 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.960 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:36:21.960 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:36:21.960 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.960 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.960 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_max_iterations": { 2024-04-25T00:36:21.960 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:36:21.960 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-04-25T00:36:21.961 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.961 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.961 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "25", 2024-04-25T00:36:21.961 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-04-25T00:36:21.961 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "250", 2024-04-25T00:36:21.961 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.961 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:36:21.961 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.961 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.961 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.962 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_metrics": { 2024-04-25T00:36:21.962 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_metrics", 2024-04-25T00:36:21.962 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.962 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.962 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.962 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:36:21.962 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.962 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.962 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.962 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:36:21.962 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:36:21.963 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.963 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.963 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_step": { 2024-04-25T00:36:21.963 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_step", 2024-04-25T00:36:21.963 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:21.963 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.963 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.963 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0.5", 2024-04-25T00:36:21.963 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0.001", 2024-04-25T00:36:21.963 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "0.999", 2024-04-25T00:36:21.963 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.964 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:36:21.964 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:36:21.964 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.964 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.964 INFO:teuthology.orchestra.run.smithi029.stdout: "end_time": { 2024-04-25T00:36:21.964 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "end_time", 2024-04-25T00:36:21.964 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.964 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.964 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.964 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "2359", 2024-04-25T00:36:21.965 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.965 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.965 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.965 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:36:21.965 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:36:21.965 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.965 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.965 INFO:teuthology.orchestra.run.smithi029.stdout: "end_weekday": { 2024-04-25T00:36:21.966 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "end_weekday", 2024-04-25T00:36:21.966 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-04-25T00:36:21.966 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.966 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.966 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-04-25T00:36:21.966 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0", 2024-04-25T00:36:21.966 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "6", 2024-04-25T00:36:21.966 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.966 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:36:21.966 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:36:21.966 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.967 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.967 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:21.967 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:21.967 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.967 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.967 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.967 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:21.967 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.967 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.967 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:21.967 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:21.968 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:21.968 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:21.968 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:21.968 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:21.968 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:21.968 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:21.968 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:21.968 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.968 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.968 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.969 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:21.969 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:21.969 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:21.969 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.969 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.969 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:21.969 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.969 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.969 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.970 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:21.970 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.970 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.970 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.970 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:21.970 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:21.970 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.970 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.970 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.970 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:21.970 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.971 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.971 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:21.971 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:21.971 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:21.971 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:21.971 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:21.971 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:21.971 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:21.971 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:21.971 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:21.971 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.971 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.971 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.972 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:21.972 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:21.972 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:21.972 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.972 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.972 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:21.972 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.972 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.972 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.972 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:21.973 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.973 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.973 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.973 INFO:teuthology.orchestra.run.smithi029.stdout: "min_score": { 2024-04-25T00:36:21.973 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "min_score", 2024-04-25T00:36:21.973 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:21.973 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.973 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.973 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-04-25T00:36:21.973 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.974 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.974 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.974 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:36:21.974 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.974 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.974 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.974 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": { 2024-04-25T00:36:21.974 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mode", 2024-04-25T00:36:21.975 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.975 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.975 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.975 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "upmap", 2024-04-25T00:36:21.975 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.975 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.975 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:21.975 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-compat", 2024-04-25T00:36:21.975 INFO:teuthology.orchestra.run.smithi029.stdout: "none", 2024-04-25T00:36:21.975 INFO:teuthology.orchestra.run.smithi029.stdout: "read", 2024-04-25T00:36:21.975 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap", 2024-04-25T00:36:21.976 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap-read" 2024-04-25T00:36:21.976 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:21.976 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Balancer mode", 2024-04-25T00:36:21.976 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.976 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.976 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.976 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_ids": { 2024-04-25T00:36:21.976 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pool_ids", 2024-04-25T00:36:21.976 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.976 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.976 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.977 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:21.977 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.977 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.977 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.977 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:36:21.977 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.977 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.977 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.977 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-04-25T00:36:21.977 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-04-25T00:36:21.977 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:21.978 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.978 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.978 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-04-25T00:36:21.978 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.978 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.978 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.978 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:36:21.978 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.978 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.978 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.978 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap_max_deviation": { 2024-04-25T00:36:21.979 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "upmap_max_deviation", 2024-04-25T00:36:21.979 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:21.979 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.979 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.979 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-04-25T00:36:21.979 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-04-25T00:36:21.979 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.979 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.979 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:36:21.979 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:36:21.979 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.980 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.980 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap_max_optimizations": { 2024-04-25T00:36:21.980 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:36:21.980 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-04-25T00:36:21.980 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.980 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.980 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:21.980 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.980 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.980 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.980 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:36:21.980 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.981 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.981 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.981 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:21.981 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:21.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:21.981 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephadm", 2024-04-25T00:36:21.981 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:21.981 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:21.981 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:21.981 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_down_multiplier": { 2024-04-25T00:36:21.981 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_down_multiplier", 2024-04-25T00:36:21.981 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:21.982 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.982 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.982 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3.0", 2024-04-25T00:36:21.982 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.982 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.982 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.982 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:36:21.982 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.982 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.982 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.983 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_refresh_rate": { 2024-04-25T00:36:21.983 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_refresh_rate", 2024-04-25T00:36:21.983 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:21.983 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.983 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.983 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "20", 2024-04-25T00:36:21.983 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.983 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.983 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.983 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:36:21.983 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.983 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.984 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.984 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_starting_port": { 2024-04-25T00:36:21.984 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_starting_port", 2024-04-25T00:36:21.984 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:21.984 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.984 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.984 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "4721", 2024-04-25T00:36:21.984 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.985 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.985 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.985 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:36:21.985 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.985 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.985 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.985 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_ptrace": { 2024-04-25T00:36:21.985 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_ptrace", 2024-04-25T00:36:21.985 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:21.985 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.986 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.986 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:21.986 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.986 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.986 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.986 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:36:21.986 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:36:21.986 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.986 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.986 INFO:teuthology.orchestra.run.smithi029.stdout: "autotune_interval": { 2024-04-25T00:36:21.986 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "autotune_interval", 2024-04-25T00:36:21.987 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:21.987 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.987 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.987 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-04-25T00:36:21.987 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.987 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.987 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.987 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:36:21.987 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.987 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.987 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.988 INFO:teuthology.orchestra.run.smithi029.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:36:21.988 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:36:21.988 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:21.988 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.988 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.988 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0.7", 2024-04-25T00:36:21.988 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.988 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.988 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.988 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:36:21.988 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.988 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.989 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.989 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm_log_destination": { 2024-04-25T00:36:21.989 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephadm_log_destination", 2024-04-25T00:36:21.989 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.989 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.989 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.989 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:21.989 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.989 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.989 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:21.989 INFO:teuthology.orchestra.run.smithi029.stdout: "file", 2024-04-25T00:36:21.990 INFO:teuthology.orchestra.run.smithi029.stdout: "file,syslog", 2024-04-25T00:36:21.990 INFO:teuthology.orchestra.run.smithi029.stdout: "syslog" 2024-04-25T00:36:21.990 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:21.990 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:36:21.990 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.990 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.990 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.990 INFO:teuthology.orchestra.run.smithi029.stdout: "cgroups_split": { 2024-04-25T00:36:21.990 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cgroups_split", 2024-04-25T00:36:21.990 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:21.990 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.991 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.991 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:21.991 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.991 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.991 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.991 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:36:21.991 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.991 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.991 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.991 INFO:teuthology.orchestra.run.smithi029.stdout: "config_checks_enabled": { 2024-04-25T00:36:21.991 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "config_checks_enabled", 2024-04-25T00:36:21.992 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:21.992 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.992 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.992 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:21.992 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.992 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.992 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.992 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:36:21.992 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.992 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.992 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.992 INFO:teuthology.orchestra.run.smithi029.stdout: "config_dashboard": { 2024-04-25T00:36:21.993 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "config_dashboard", 2024-04-25T00:36:21.993 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:21.993 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.993 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.993 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:21.993 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.993 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.993 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.993 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:36:21.993 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.993 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.993 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.994 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_alertmanager": { 2024-04-25T00:36:21.994 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_alertmanager", 2024-04-25T00:36:21.994 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.994 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.994 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.994 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:36:21.994 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.994 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.994 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.994 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-04-25T00:36:21.994 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.994 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.995 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.995 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_base": { 2024-04-25T00:36:21.995 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_base", 2024-04-25T00:36:21.995 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.995 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.995 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:21.995 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:36:21.995 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.995 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.995 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.995 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:36:21.995 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.996 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.996 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.996 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_elasticsearch": { 2024-04-25T00:36:21.996 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:36:21.996 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.996 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.996 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.996 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:36:21.996 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.996 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.996 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.997 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "elasticsearch container image", 2024-04-25T00:36:21.997 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.997 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.997 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.997 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_grafana": { 2024-04-25T00:36:21.997 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_grafana", 2024-04-25T00:36:21.997 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.997 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.997 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.997 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:36:21.997 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.997 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.998 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.998 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-04-25T00:36:21.998 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.998 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.998 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.998 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_haproxy": { 2024-04-25T00:36:21.998 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_haproxy", 2024-04-25T00:36:21.998 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:21.998 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:21.998 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:21.999 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:36:21.999 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:21.999 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:21.999 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:21.999 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "HAproxy container image", 2024-04-25T00:36:21.999 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:21.999 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:21.999 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:21.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:21.999 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_agent": { 2024-04-25T00:36:21.999 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:36:22.000 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.000 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.000 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.000 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:36:22.000 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.000 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.000 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.000 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:36:22.000 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.000 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.000 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.001 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_collector": { 2024-04-25T00:36:22.001 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:36:22.001 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.001 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.001 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.001 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:36:22.001 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.001 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.001 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.001 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:36:22.001 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.001 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.002 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.002 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_query": { 2024-04-25T00:36:22.002 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:36:22.002 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.002 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.002 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.002 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:36:22.002 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.002 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.003 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.003 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger query container image", 2024-04-25T00:36:22.003 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.003 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.003 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.003 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_keepalived": { 2024-04-25T00:36:22.003 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_keepalived", 2024-04-25T00:36:22.004 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.004 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.004 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.004 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:36:22.004 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.004 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.004 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.004 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Keepalived container image", 2024-04-25T00:36:22.004 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.004 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.004 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.005 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_loki": { 2024-04-25T00:36:22.005 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_loki", 2024-04-25T00:36:22.005 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.005 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.005 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.005 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:36:22.005 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.005 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.005 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.005 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Loki container image", 2024-04-25T00:36:22.005 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.005 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.006 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.006 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_node_exporter": { 2024-04-25T00:36:22.006 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_node_exporter", 2024-04-25T00:36:22.006 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.006 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.006 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.006 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:36:22.006 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.006 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.006 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_nvmeof": { 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_nvmeof", 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.007 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.008 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Nvme-of container image", 2024-04-25T00:36:22.008 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.008 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.008 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.008 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_prometheus": { 2024-04-25T00:36:22.008 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_prometheus", 2024-04-25T00:36:22.008 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.008 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.008 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.008 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:36:22.008 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.008 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.009 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.009 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-04-25T00:36:22.009 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.009 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.009 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.009 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_promtail": { 2024-04-25T00:36:22.009 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_promtail", 2024-04-25T00:36:22.009 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.009 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.009 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.009 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:36:22.009 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.010 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.010 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.010 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Promtail container image", 2024-04-25T00:36:22.010 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.010 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.010 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.010 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_samba": { 2024-04-25T00:36:22.010 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_samba", 2024-04-25T00:36:22.010 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.010 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.011 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.011 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:36:22.011 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.011 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.011 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.011 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:36:22.011 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.011 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.011 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.011 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_snmp_gateway": { 2024-04-25T00:36:22.011 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:36:22.011 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.012 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.012 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.012 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:36:22.012 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.012 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.012 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.012 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:36:22.012 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.012 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.012 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.012 INFO:teuthology.orchestra.run.smithi029.stdout: "container_init": { 2024-04-25T00:36:22.013 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_init", 2024-04-25T00:36:22.013 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.013 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.013 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.013 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.013 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.013 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.013 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.013 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:36:22.013 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.014 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.014 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.014 INFO:teuthology.orchestra.run.smithi029.stdout: "daemon_cache_timeout": { 2024-04-25T00:36:22.014 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:36:22.014 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.014 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.014 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.014 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-04-25T00:36:22.014 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.014 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.015 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.015 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:36:22.015 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.015 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.015 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.015 INFO:teuthology.orchestra.run.smithi029.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:36:22.015 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:36:22.015 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.015 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.015 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.016 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "900", 2024-04-25T00:36:22.016 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.016 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.016 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.016 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:36:22.016 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.016 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.016 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.016 INFO:teuthology.orchestra.run.smithi029.stdout: "default_registry": { 2024-04-25T00:36:22.016 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "default_registry", 2024-04-25T00:36:22.017 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.017 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.017 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.017 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io", 2024-04-25T00:36:22.017 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.017 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.017 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.017 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:36:22.017 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.017 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.017 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.018 INFO:teuthology.orchestra.run.smithi029.stdout: "device_cache_timeout": { 2024-04-25T00:36:22.018 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_cache_timeout", 2024-04-25T00:36:22.018 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.018 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.018 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.018 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1800", 2024-04-25T00:36:22.018 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.018 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.018 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.018 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:36:22.018 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.019 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.019 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.019 INFO:teuthology.orchestra.run.smithi029.stdout: "device_enhanced_scan": { 2024-04-25T00:36:22.019 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_enhanced_scan", 2024-04-25T00:36:22.019 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.019 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.019 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.019 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.019 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.019 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.020 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.020 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:36:22.020 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.020 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.020 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.020 INFO:teuthology.orchestra.run.smithi029.stdout: "facts_cache_timeout": { 2024-04-25T00:36:22.020 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "facts_cache_timeout", 2024-04-25T00:36:22.020 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.020 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.020 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.021 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-04-25T00:36:22.021 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.021 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.021 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.021 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:36:22.021 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.021 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.021 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.021 INFO:teuthology.orchestra.run.smithi029.stdout: "grafana_dashboards_path": { 2024-04-25T00:36:22.021 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:36:22.022 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.022 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.022 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.022 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:36:22.022 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.022 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.022 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.022 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:36:22.022 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.022 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.023 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.023 INFO:teuthology.orchestra.run.smithi029.stdout: "host_check_interval": { 2024-04-25T00:36:22.023 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "host_check_interval", 2024-04-25T00:36:22.023 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.023 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.023 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.023 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-04-25T00:36:22.023 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.023 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.023 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.023 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:36:22.024 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.024 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.024 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.024 INFO:teuthology.orchestra.run.smithi029.stdout: "hw_monitoring": { 2024-04-25T00:36:22.024 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "hw_monitoring", 2024-04-25T00:36:22.024 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.024 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.024 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.024 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.024 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.024 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.025 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.025 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:36:22.025 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.025 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.025 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.025 INFO:teuthology.orchestra.run.smithi029.stdout: "inventory_list_all": { 2024-04-25T00:36:22.025 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "inventory_list_all", 2024-04-25T00:36:22.025 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.025 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.025 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.026 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.026 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.026 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.026 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.026 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:36:22.026 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.026 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.026 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.026 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.026 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.027 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.027 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.027 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.027 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.027 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.027 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.027 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.027 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.027 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.027 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.027 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.027 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.028 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.028 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.028 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.028 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.028 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.028 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.028 INFO:teuthology.orchestra.run.smithi029.stdout: "log_refresh_metadata": { 2024-04-25T00:36:22.028 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_refresh_metadata", 2024-04-25T00:36:22.028 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.028 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.028 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.029 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.029 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.029 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.029 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.029 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:36:22.029 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.029 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.029 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.029 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.029 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.030 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.030 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.030 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.030 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.030 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.030 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.030 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.030 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:36:22.030 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.030 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.030 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.031 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.031 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.031 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.031 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.031 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.031 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.031 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.031 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.031 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.031 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.031 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.031 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.031 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.032 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.032 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.032 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.032 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.032 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.032 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.032 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.032 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.032 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.032 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.032 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.032 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.033 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.033 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.033 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.033 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.033 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.033 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.033 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.033 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.033 INFO:teuthology.orchestra.run.smithi029.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:36:22.033 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:36:22.033 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.033 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.034 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.034 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.034 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.034 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.034 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.034 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:36:22.034 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.034 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.034 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.034 INFO:teuthology.orchestra.run.smithi029.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:36:22.034 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:36:22.035 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.035 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.035 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.035 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "*", 2024-04-25T00:36:22.035 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.035 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.035 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.035 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:36:22.035 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.035 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.035 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.035 INFO:teuthology.orchestra.run.smithi029.stdout: "max_count_per_host": { 2024-04-25T00:36:22.036 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_count_per_host", 2024-04-25T00:36:22.036 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.036 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.036 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.036 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:22.036 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.036 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.036 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.036 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:36:22.036 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.036 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.036 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.037 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd_draining_count": { 2024-04-25T00:36:22.037 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_osd_draining_count", 2024-04-25T00:36:22.037 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.037 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.037 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.037 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:22.037 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.037 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.037 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.037 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:36:22.037 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.037 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.038 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.038 INFO:teuthology.orchestra.run.smithi029.stdout: "migration_current": { 2024-04-25T00:36:22.038 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "migration_current", 2024-04-25T00:36:22.038 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.038 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.038 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.038 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.038 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.038 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.038 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.038 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "internal - do not modify", 2024-04-25T00:36:22.039 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.039 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.039 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": { 2024-04-25T00:36:22.039 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mode", 2024-04-25T00:36:22.039 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.039 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.039 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.039 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "root", 2024-04-25T00:36:22.039 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.039 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.039 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.040 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm-package", 2024-04-25T00:36:22.040 INFO:teuthology.orchestra.run.smithi029.stdout: "root" 2024-04-25T00:36:22.040 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.040 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:36:22.040 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.040 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.040 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.040 INFO:teuthology.orchestra.run.smithi029.stdout: "oob_default_addr": { 2024-04-25T00:36:22.040 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "oob_default_addr", 2024-04-25T00:36:22.040 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.040 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.041 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.041 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "169.254.1.1", 2024-04-25T00:36:22.041 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.041 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.041 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.041 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:36:22.041 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.041 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.041 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.041 INFO:teuthology.orchestra.run.smithi029.stdout: "prometheus_alerts_path": { 2024-04-25T00:36:22.041 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:36:22.041 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.042 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.042 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.042 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:36:22.042 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.042 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.042 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.042 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:36:22.042 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.042 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.042 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.042 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_insecure": { 2024-04-25T00:36:22.043 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_insecure", 2024-04-25T00:36:22.043 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.043 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.043 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.043 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.043 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.043 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.043 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.043 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:36:22.043 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.043 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.044 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.044 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_password": { 2024-04-25T00:36:22.044 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_password", 2024-04-25T00:36:22.044 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.044 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.044 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.044 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.044 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.044 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.044 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.044 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:36:22.044 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.045 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.045 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.045 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_url": { 2024-04-25T00:36:22.045 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_url", 2024-04-25T00:36:22.045 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.045 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.045 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.045 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.045 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.045 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.045 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.045 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:36:22.046 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.046 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.046 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.046 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_username": { 2024-04-25T00:36:22.046 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_username", 2024-04-25T00:36:22.046 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.046 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.046 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.046 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.047 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.047 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.047 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.047 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:36:22.047 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.047 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.047 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.047 INFO:teuthology.orchestra.run.smithi029.stdout: "secure_monitoring_stack": { 2024-04-25T00:36:22.047 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:36:22.047 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.047 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.048 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.048 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.048 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.048 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.048 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.048 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:36:22.048 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.048 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.048 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.048 INFO:teuthology.orchestra.run.smithi029.stdout: "service_discovery_port": { 2024-04-25T00:36:22.049 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "service_discovery_port", 2024-04-25T00:36:22.049 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.049 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.049 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.049 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8765", 2024-04-25T00:36:22.049 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.049 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.049 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.049 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:36:22.050 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.050 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.050 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.050 INFO:teuthology.orchestra.run.smithi029.stdout: "ssh_config_file": { 2024-04-25T00:36:22.050 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssh_config_file", 2024-04-25T00:36:22.050 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.050 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.050 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.050 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.051 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.051 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.051 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.051 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:36:22.051 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.051 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.051 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.051 INFO:teuthology.orchestra.run.smithi029.stdout: "use_agent": { 2024-04-25T00:36:22.051 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "use_agent", 2024-04-25T00:36:22.052 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.052 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.052 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.052 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.052 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.052 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.052 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.052 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:36:22.052 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.052 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.052 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.053 INFO:teuthology.orchestra.run.smithi029.stdout: "use_repo_digest": { 2024-04-25T00:36:22.053 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "use_repo_digest", 2024-04-25T00:36:22.053 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.053 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.053 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.053 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.053 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.053 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.053 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.053 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:36:22.053 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.053 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.054 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.054 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_failed_host_check": { 2024-04-25T00:36:22.054 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:36:22.054 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.054 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.054 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.054 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.054 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.054 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.054 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.055 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:36:22.055 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.055 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.055 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.055 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_stray_daemons": { 2024-04-25T00:36:22.055 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:36:22.055 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.055 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.055 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.055 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.055 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.056 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.056 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.056 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:36:22.056 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.056 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.056 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.056 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_stray_hosts": { 2024-04-25T00:36:22.056 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:36:22.056 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.056 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.056 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.056 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.057 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.057 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.057 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.057 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:36:22.057 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.057 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.057 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.057 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crash", 2024-04-25T00:36:22.058 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.058 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.058 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.058 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.058 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.058 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.058 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.058 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.058 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.058 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.058 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.059 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.059 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.059 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.059 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.059 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.059 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.059 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.059 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.059 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.059 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.059 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.059 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.060 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.060 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.060 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.060 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.060 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.060 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.060 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.060 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.060 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.060 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.060 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.060 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.061 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.061 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.061 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.061 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.061 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.061 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.061 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.061 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.061 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.061 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.062 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.062 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.062 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.062 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.062 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.062 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.062 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.062 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.062 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.062 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.062 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.062 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.063 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.063 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.063 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.063 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.063 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.063 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.063 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.063 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.063 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.063 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.063 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.063 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.064 INFO:teuthology.orchestra.run.smithi029.stdout: "retain_interval": { 2024-04-25T00:36:22.064 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "retain_interval", 2024-04-25T00:36:22.064 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.064 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.064 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.064 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "31536000", 2024-04-25T00:36:22.064 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.064 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.064 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.064 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:36:22.064 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.065 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.065 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.065 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_recent_interval": { 2024-04-25T00:36:22.065 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_recent_interval", 2024-04-25T00:36:22.065 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.065 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.065 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.065 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1209600", 2024-04-25T00:36:22.065 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.065 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.065 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.066 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:36:22.066 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.066 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.066 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.066 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.066 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.066 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "dashboard", 2024-04-25T00:36:22.066 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.066 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.067 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.067 INFO:teuthology.orchestra.run.smithi029.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:36:22.067 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:36:22.067 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.067 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.067 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.067 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:22.067 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.067 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.067 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.067 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.068 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.068 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.068 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.068 INFO:teuthology.orchestra.run.smithi029.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:36:22.068 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:36:22.068 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.068 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.069 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.069 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.069 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.069 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.069 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.069 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.069 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.069 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.069 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.069 INFO:teuthology.orchestra.run.smithi029.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:36:22.069 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:36:22.070 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.070 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.070 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.070 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.070 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.070 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.070 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.070 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.070 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.070 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.070 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.071 INFO:teuthology.orchestra.run.smithi029.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:36:22.071 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:36:22.071 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.071 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.071 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.071 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.071 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.071 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.071 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.071 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.072 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.072 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.072 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.072 INFO:teuthology.orchestra.run.smithi029.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:36:22.072 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:36:22.072 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.072 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.072 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.072 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.072 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.072 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.072 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.073 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.073 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.073 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.073 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.073 INFO:teuthology.orchestra.run.smithi029.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:36:22.073 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:36:22.073 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.073 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.073 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.073 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.073 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.074 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.074 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.074 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.074 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.074 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.074 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.074 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:36:22.074 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:36:22.074 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.074 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.075 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.075 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.075 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.075 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.075 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.075 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.075 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.075 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.075 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.075 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:36:22.075 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:36:22.075 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.076 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.076 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.076 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.076 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.076 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.076 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.076 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.076 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.076 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.076 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.076 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:36:22.077 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:36:22.077 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.077 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.077 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.077 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.077 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.077 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.077 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.077 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.077 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.077 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.077 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.078 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:36:22.078 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:36:22.078 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.078 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.078 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.078 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.078 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.078 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.078 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.078 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.078 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.079 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.079 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.079 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:36:22.079 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:36:22.079 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.079 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.079 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.079 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.079 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.079 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.079 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.080 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.080 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.080 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.080 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.080 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:36:22.080 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:36:22.080 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.080 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.080 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.080 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.080 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.080 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.081 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.081 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.081 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.081 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.081 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.081 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:36:22.081 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:36:22.081 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.081 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.081 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.081 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.081 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.082 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.082 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.082 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.082 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.082 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.082 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.082 INFO:teuthology.orchestra.run.smithi029.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:36:22.082 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:36:22.082 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.082 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.082 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.083 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.083 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.083 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.083 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.083 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.083 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.083 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.083 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.083 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:36:22.083 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:36:22.083 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.083 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.084 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.084 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-04-25T00:36:22.084 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.084 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.084 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.084 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.084 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.084 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.084 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.084 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:36:22.084 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:36:22.085 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.085 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.085 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.085 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.085 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.085 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.085 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.085 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.085 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.085 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.085 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.086 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_URL": { 2024-04-25T00:36:22.086 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:36:22.086 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.086 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.086 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.086 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.086 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.086 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.086 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.086 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.086 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.087 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.087 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.087 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:36:22.087 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:36:22.087 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.087 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.087 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.087 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-04-25T00:36:22.087 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.087 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.087 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.088 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.088 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.088 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.088 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.088 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:36:22.088 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:36:22.088 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.088 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.088 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.088 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.088 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.089 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.089 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.089 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.089 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.089 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.089 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.089 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:36:22.089 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:36:22.089 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.089 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.089 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.090 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.090 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.090 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.090 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.090 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.090 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.090 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.090 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.090 INFO:teuthology.orchestra.run.smithi029.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:36:22.090 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:36:22.090 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:36:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:36:22.092 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.092 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.092 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.092 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.092 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.092 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.092 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.092 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.092 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.092 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.092 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.093 INFO:teuthology.orchestra.run.smithi029.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:36:22.093 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:36:22.093 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.093 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.093 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.093 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.093 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.093 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.093 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.093 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.093 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.093 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.094 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.094 INFO:teuthology.orchestra.run.smithi029.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:36:22.094 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:36:22.094 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.094 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.094 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.094 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.094 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.094 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.094 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.094 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.095 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.095 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.095 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.095 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:36:22.095 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:36:22.095 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.095 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.095 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.095 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.095 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.095 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.095 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.096 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.096 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.096 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.096 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.096 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:36:22.096 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:36:22.096 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.096 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.096 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.096 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.096 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.096 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.097 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.098 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.098 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.098 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.098 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.098 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.098 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:36:22.098 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:36:22.098 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.098 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.098 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.098 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.099 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.099 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.099 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.099 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.099 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.099 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.099 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.099 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:36:22.099 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:36:22.099 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.099 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.099 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.100 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.100 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.100 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.100 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.100 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.100 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.100 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.100 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.100 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:36:22.100 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:36:22.100 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.100 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.101 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.101 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.101 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.101 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.101 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.101 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.101 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.101 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.101 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.101 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:36:22.102 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:36:22.102 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.102 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.102 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.102 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.102 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.102 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.102 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.102 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.102 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.102 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.102 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.103 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:36:22.103 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:36:22.103 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.103 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.103 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.103 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.103 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.103 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.103 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.103 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.103 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.103 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.104 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.104 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:36:22.104 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:36:22.104 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.104 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.104 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.104 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:36:22.104 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.104 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.105 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.105 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.105 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.105 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.105 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.105 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:36:22.105 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:36:22.105 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.105 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.106 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.106 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:22.106 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.106 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.106 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.106 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.106 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.106 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.106 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.107 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:36:22.107 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:36:22.107 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.107 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.107 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.107 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8", 2024-04-25T00:36:22.107 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.107 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.107 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.107 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.108 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.108 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.108 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.108 INFO:teuthology.orchestra.run.smithi029.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:36:22.108 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:36:22.108 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.108 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.108 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.108 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "45", 2024-04-25T00:36:22.108 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.108 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.109 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.109 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.109 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.109 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.109 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.109 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:36:22.109 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:36:22.109 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.109 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.109 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.109 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.110 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.110 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.110 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.110 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.110 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.110 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.110 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.110 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:36:22.110 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:36:22.110 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.110 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.110 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.111 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-04-25T00:36:22.111 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.111 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.111 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.111 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.111 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.111 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.111 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.111 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:36:22.111 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:36:22.112 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.112 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.112 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.112 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.112 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.112 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.112 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.112 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.112 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.112 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.112 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.113 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:36:22.113 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:36:22.113 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.113 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.113 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.113 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.113 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.113 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.113 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.113 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.113 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.113 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.113 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.114 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:36:22.114 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:36:22.114 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.114 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.114 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.114 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-04-25T00:36:22.114 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.114 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.114 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.114 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.114 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.115 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.115 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.115 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:36:22.115 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:36:22.115 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.115 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.115 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.115 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:22.115 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.115 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.115 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.116 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.116 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.116 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.116 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.116 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:36:22.116 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:36:22.116 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.116 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.116 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.116 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-04-25T00:36:22.116 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.116 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.117 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.117 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.117 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.117 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.117 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.117 INFO:teuthology.orchestra.run.smithi029.stdout: "cross_origin_url": { 2024-04-25T00:36:22.117 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cross_origin_url", 2024-04-25T00:36:22.117 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.117 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.117 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.117 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.117 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.118 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.118 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.118 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.118 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.118 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.118 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.118 INFO:teuthology.orchestra.run.smithi029.stdout: "crt_file": { 2024-04-25T00:36:22.118 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crt_file", 2024-04-25T00:36:22.118 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.118 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.118 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.119 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.119 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.119 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.119 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.119 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.119 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.119 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.119 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.119 INFO:teuthology.orchestra.run.smithi029.stdout: "debug": { 2024-04-25T00:36:22.119 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "debug", 2024-04-25T00:36:22.119 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.120 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.120 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.120 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.120 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.120 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.120 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.120 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:36:22.120 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.120 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.120 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.120 INFO:teuthology.orchestra.run.smithi029.stdout: "jwt_token_ttl": { 2024-04-25T00:36:22.120 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "jwt_token_ttl", 2024-04-25T00:36:22.121 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.121 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.121 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.121 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "28800", 2024-04-25T00:36:22.121 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.121 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.121 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.121 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.121 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.121 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.121 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.122 INFO:teuthology.orchestra.run.smithi029.stdout: "key_file": { 2024-04-25T00:36:22.122 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "key_file", 2024-04-25T00:36:22.122 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.122 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.122 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.122 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.122 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.122 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.122 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.122 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.122 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.122 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.123 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.123 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.123 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.123 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.123 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.123 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.123 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.123 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.123 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.123 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.123 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.124 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.125 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.125 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.125 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.125 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.125 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.125 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.125 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.125 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.125 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.125 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.125 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.126 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.126 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.126 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.126 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.126 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.126 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.126 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.126 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.126 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.126 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.126 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.127 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.127 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.127 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.127 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.127 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.127 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.127 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.127 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.127 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.127 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.128 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.128 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.128 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.128 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.128 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.128 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.128 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.128 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.128 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.128 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.128 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.128 INFO:teuthology.orchestra.run.smithi029.stdout: "motd": { 2024-04-25T00:36:22.129 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "motd", 2024-04-25T00:36:22.129 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.129 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.129 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.129 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.129 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.129 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.129 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.129 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "The message of the day", 2024-04-25T00:36:22.129 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.129 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.129 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.130 INFO:teuthology.orchestra.run.smithi029.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:36:22.130 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:36:22.130 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.130 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.130 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.130 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.130 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.130 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.130 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.130 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.130 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.131 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.131 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.131 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-04-25T00:36:22.131 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-04-25T00:36:22.131 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.131 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.131 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.132 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "::", 2024-04-25T00:36:22.132 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.132 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.132 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.132 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.132 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.132 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.132 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.132 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-04-25T00:36:22.133 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-04-25T00:36:22.133 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.133 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.133 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.133 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8080", 2024-04-25T00:36:22.133 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.133 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.133 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.133 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.133 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.134 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.134 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.134 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl": { 2024-04-25T00:36:22.134 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl", 2024-04-25T00:36:22.134 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.134 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.134 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.134 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.134 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.134 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.134 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.135 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.135 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.135 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.135 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.135 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl_server_port": { 2024-04-25T00:36:22.135 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl_server_port", 2024-04-25T00:36:22.135 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.135 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.135 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.135 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8443", 2024-04-25T00:36:22.135 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.135 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.136 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.136 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.136 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.136 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.136 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.136 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_behaviour": { 2024-04-25T00:36:22.136 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_behaviour", 2024-04-25T00:36:22.136 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.136 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.136 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.136 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "redirect", 2024-04-25T00:36:22.137 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.137 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.137 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.137 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.137 INFO:teuthology.orchestra.run.smithi029.stdout: "redirect" 2024-04-25T00:36:22.137 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.137 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.137 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.137 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.137 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.137 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_error_status_code": { 2024-04-25T00:36:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_error_status_code", 2024-04-25T00:36:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "500", 2024-04-25T00:36:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "400", 2024-04-25T00:36:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "599", 2024-04-25T00:36:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.138 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: "url_prefix": { 2024-04-25T00:36:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "url_prefix", 2024-04-25T00:36:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.139 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "devicehealth", 2024-04-25T00:36:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: "enable_monitoring": { 2024-04-25T00:36:22.140 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enable_monitoring", 2024-04-25T00:36:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "monitor device health metrics", 2024-04-25T00:36:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.141 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.142 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.142 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.142 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.142 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.142 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.142 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.142 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.142 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.142 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.142 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.142 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.143 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.143 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.143 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.143 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.143 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.143 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.143 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.143 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.143 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.143 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.143 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.144 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.144 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.144 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.144 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.144 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.144 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.144 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.144 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.144 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.144 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.144 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.144 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.145 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.145 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.145 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.145 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.145 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.145 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.145 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.145 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.145 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.145 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.145 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.145 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.146 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.146 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.146 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.146 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.146 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.146 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.146 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.146 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.146 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.147 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.147 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.147 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.147 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.147 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.147 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.147 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.147 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.147 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.147 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.147 INFO:teuthology.orchestra.run.smithi029.stdout: "mark_out_threshold": { 2024-04-25T00:36:22.148 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mark_out_threshold", 2024-04-25T00:36:22.148 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.148 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.148 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.148 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "2419200", 2024-04-25T00:36:22.148 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.148 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.148 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.148 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:36:22.148 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.148 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.148 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.149 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": { 2024-04-25T00:36:22.149 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pool_name", 2024-04-25T00:36:22.149 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.149 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.149 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.149 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "device_health_metrics", 2024-04-25T00:36:22.149 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.149 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.149 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.149 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:36:22.149 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.149 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.150 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.150 INFO:teuthology.orchestra.run.smithi029.stdout: "retention_period": { 2024-04-25T00:36:22.150 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "retention_period", 2024-04-25T00:36:22.150 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.150 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.150 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.150 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15552000", 2024-04-25T00:36:22.150 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.150 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.150 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.150 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:36:22.151 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.151 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.151 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.151 INFO:teuthology.orchestra.run.smithi029.stdout: "scrape_frequency": { 2024-04-25T00:36:22.151 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "scrape_frequency", 2024-04-25T00:36:22.151 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.151 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.151 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.151 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "86400", 2024-04-25T00:36:22.151 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.151 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.152 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.152 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:36:22.152 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.152 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.152 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.152 INFO:teuthology.orchestra.run.smithi029.stdout: "self_heal": { 2024-04-25T00:36:22.152 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "self_heal", 2024-04-25T00:36:22.152 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.152 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.152 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.152 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.153 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.153 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.153 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.153 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:36:22.153 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.153 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.153 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.153 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-04-25T00:36:22.153 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-04-25T00:36:22.153 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.154 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.154 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.154 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-04-25T00:36:22.154 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.154 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.154 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.154 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:36:22.154 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.154 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.154 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.154 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_threshold": { 2024-04-25T00:36:22.154 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_threshold", 2024-04-25T00:36:22.155 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.155 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.155 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.155 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "7257600", 2024-04-25T00:36:22.155 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.155 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.155 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.155 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:36:22.155 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.155 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.155 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.155 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.156 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.156 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "diskprediction_local", 2024-04-25T00:36:22.156 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.156 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.156 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.156 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.156 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.156 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.156 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.156 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.157 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.157 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.157 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.157 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.157 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.157 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.157 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.157 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.157 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.157 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.157 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.157 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.158 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.158 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.158 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.158 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.158 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.158 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.158 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.158 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.158 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.158 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.158 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.159 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.159 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.159 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.159 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.159 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.159 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.159 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.159 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.159 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.159 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.159 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.160 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.160 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.160 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.160 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.160 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.160 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.160 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.160 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.160 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.160 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.160 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.160 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.160 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.161 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.161 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.161 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.161 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.161 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.161 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.161 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.161 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.161 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.161 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.162 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.162 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.162 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.162 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.162 INFO:teuthology.orchestra.run.smithi029.stdout: "predict_interval": { 2024-04-25T00:36:22.162 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "predict_interval", 2024-04-25T00:36:22.162 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.162 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.162 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.162 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "86400", 2024-04-25T00:36:22.162 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.163 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.163 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.163 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.163 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.163 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.163 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.163 INFO:teuthology.orchestra.run.smithi029.stdout: "predictor_model": { 2024-04-25T00:36:22.163 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "predictor_model", 2024-04-25T00:36:22.163 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.163 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.163 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.164 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "prophetstor", 2024-04-25T00:36:22.164 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.164 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.164 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.164 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.164 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.164 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.164 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.164 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-04-25T00:36:22.164 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-04-25T00:36:22.165 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.165 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.165 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.165 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-04-25T00:36:22.165 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.165 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.165 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.165 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.165 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.165 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.165 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.165 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.165 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.166 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "influx", 2024-04-25T00:36:22.166 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": false, 2024-04-25T00:36:22.166 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:36:22.166 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.166 INFO:teuthology.orchestra.run.smithi029.stdout: "batch_size": { 2024-04-25T00:36:22.166 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "batch_size", 2024-04-25T00:36:22.166 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.166 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.166 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.166 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5000", 2024-04-25T00:36:22.167 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.167 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.167 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.167 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:36:22.167 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.167 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.167 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.167 INFO:teuthology.orchestra.run.smithi029.stdout: "database": { 2024-04-25T00:36:22.167 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "database", 2024-04-25T00:36:22.167 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.167 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.167 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.168 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "ceph", 2024-04-25T00:36:22.168 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.168 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.168 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.168 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:36:22.168 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.168 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.168 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.168 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": { 2024-04-25T00:36:22.168 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "hostname", 2024-04-25T00:36:22.168 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.168 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.169 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.169 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.169 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.169 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.169 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.169 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:36:22.169 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.169 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.169 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.169 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-04-25T00:36:22.169 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-04-25T00:36:22.169 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.170 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.170 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.170 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "30", 2024-04-25T00:36:22.170 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "5", 2024-04-25T00:36:22.170 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.170 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.170 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:36:22.170 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.170 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.170 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.170 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.171 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.171 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.171 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.171 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.171 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.171 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.171 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.171 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.171 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.171 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.171 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.171 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.171 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.172 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.172 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.172 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.172 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.172 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.172 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.172 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.172 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.172 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.172 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.172 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.173 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.173 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.173 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.173 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.173 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.173 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.173 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.173 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.173 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.173 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.173 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.174 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.174 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.174 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.174 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.174 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.174 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.174 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.174 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.174 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.174 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.174 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.174 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.175 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.175 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.175 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.175 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.175 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.175 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.175 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.175 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.175 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.175 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.175 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.175 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.176 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.176 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.176 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.176 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.176 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.176 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.176 INFO:teuthology.orchestra.run.smithi029.stdout: "password": { 2024-04-25T00:36:22.176 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "password", 2024-04-25T00:36:22.176 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.177 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.177 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.177 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.177 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.177 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.177 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.177 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:36:22.177 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.177 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.177 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.177 INFO:teuthology.orchestra.run.smithi029.stdout: "port": { 2024-04-25T00:36:22.177 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "port", 2024-04-25T00:36:22.178 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.178 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.178 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.178 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8086", 2024-04-25T00:36:22.178 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.178 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.178 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.178 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB server port", 2024-04-25T00:36:22.178 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.178 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.178 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.179 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl": { 2024-04-25T00:36:22.179 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl", 2024-04-25T00:36:22.179 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.179 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.179 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.179 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "false", 2024-04-25T00:36:22.179 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.179 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.179 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.179 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:36:22.180 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.180 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.180 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.180 INFO:teuthology.orchestra.run.smithi029.stdout: "threads": { 2024-04-25T00:36:22.180 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "threads", 2024-04-25T00:36:22.180 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.180 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.181 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.181 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-04-25T00:36:22.181 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-04-25T00:36:22.181 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "32", 2024-04-25T00:36:22.181 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.181 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:36:22.181 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.181 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.181 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.181 INFO:teuthology.orchestra.run.smithi029.stdout: "username": { 2024-04-25T00:36:22.181 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "username", 2024-04-25T00:36:22.181 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.182 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.182 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.182 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.182 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.182 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.182 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.182 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:36:22.182 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.182 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.182 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.182 INFO:teuthology.orchestra.run.smithi029.stdout: "verify_ssl": { 2024-04-25T00:36:22.183 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "verify_ssl", 2024-04-25T00:36:22.183 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.183 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.183 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.183 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "true", 2024-04-25T00:36:22.183 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.183 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.183 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.183 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:36:22.183 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.184 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.184 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.184 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.184 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.184 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "insights", 2024-04-25T00:36:22.184 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.184 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.184 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.184 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.184 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.184 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.185 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.185 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.185 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.185 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.185 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.185 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.185 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.185 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.185 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.185 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.186 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.186 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.186 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.186 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.186 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.186 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.186 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.186 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.186 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.186 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.187 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.187 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.187 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.187 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.187 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.187 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.187 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.187 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.187 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.187 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.187 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.188 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.188 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.188 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.188 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.188 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.188 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.188 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.188 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.188 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.188 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.188 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.188 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.189 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.189 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.189 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.189 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.189 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.189 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.189 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.189 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.189 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.189 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.189 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.189 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.190 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.190 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.190 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.190 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.190 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.190 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.190 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.190 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.190 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.190 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.191 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "iostat", 2024-04-25T00:36:22.191 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.191 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.191 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.191 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.191 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.191 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.191 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.191 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.191 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.191 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.191 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.191 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.192 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.192 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.192 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.192 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.192 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.192 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.192 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.192 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.192 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.192 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.192 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.193 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.193 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.193 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.193 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.193 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.193 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.193 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.193 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.193 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.193 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.193 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.193 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.194 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.194 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.194 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.194 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.194 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.194 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.194 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.194 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.194 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.194 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.194 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.195 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.195 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.195 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.195 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.195 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.195 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.195 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.195 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.195 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.195 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.195 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.196 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.196 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.196 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.196 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.196 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.196 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.196 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.196 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.196 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.196 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.196 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.196 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.197 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.197 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.197 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "localpool", 2024-04-25T00:36:22.197 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.197 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.197 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.197 INFO:teuthology.orchestra.run.smithi029.stdout: "failure_domain": { 2024-04-25T00:36:22.197 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "failure_domain", 2024-04-25T00:36:22.197 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.198 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.198 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.198 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "host", 2024-04-25T00:36:22.198 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.198 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.198 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.198 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:36:22.198 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:36:22.198 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.198 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.198 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.199 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.199 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.199 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.199 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.199 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.199 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.199 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.199 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.199 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.199 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.199 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.199 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.199 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.200 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.200 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.200 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.200 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.200 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.200 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.200 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.200 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.200 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.200 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.200 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.200 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.201 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.202 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.202 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.202 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.202 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.202 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.202 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.202 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.202 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.202 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.202 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.202 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.202 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.203 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.203 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.203 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.203 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.203 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.203 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.203 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.203 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.203 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.203 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.203 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.204 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.204 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.204 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.204 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.204 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": { 2024-04-25T00:36:22.204 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "min_size", 2024-04-25T00:36:22.204 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.204 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.204 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.204 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.204 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.205 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.205 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.205 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:36:22.205 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:36:22.205 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.205 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.205 INFO:teuthology.orchestra.run.smithi029.stdout: "num_rep": { 2024-04-25T00:36:22.205 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "num_rep", 2024-04-25T00:36:22.205 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.205 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.205 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.206 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3", 2024-04-25T00:36:22.206 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.206 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.206 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.206 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:36:22.206 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.206 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.206 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.206 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": { 2024-04-25T00:36:22.206 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pg_num", 2024-04-25T00:36:22.206 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.207 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.207 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.207 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "128", 2024-04-25T00:36:22.207 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.207 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.207 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.207 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:36:22.207 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.207 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.207 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.208 INFO:teuthology.orchestra.run.smithi029.stdout: "prefix": { 2024-04-25T00:36:22.208 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prefix", 2024-04-25T00:36:22.208 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.208 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.208 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.208 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.208 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.208 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.208 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.208 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:36:22.209 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.209 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.209 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.209 INFO:teuthology.orchestra.run.smithi029.stdout: "subtree": { 2024-04-25T00:36:22.209 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "subtree", 2024-04-25T00:36:22.209 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.209 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.209 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.209 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "rack", 2024-04-25T00:36:22.209 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.209 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.210 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.210 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:36:22.210 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:36:22.210 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.210 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.210 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.210 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.210 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mirroring", 2024-04-25T00:36:22.210 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.211 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.211 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.211 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.211 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.211 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.211 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.211 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.211 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.211 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.211 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.211 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.211 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.212 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.212 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.212 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.212 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.212 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.212 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.212 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.212 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.212 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.212 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.212 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.213 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.213 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.213 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.213 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.213 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.213 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.213 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.213 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.213 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.213 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.213 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.214 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.214 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.214 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.214 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.214 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.214 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.214 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.215 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.215 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.215 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.215 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.215 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.215 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.215 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.215 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.215 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.215 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.216 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.216 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.216 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.216 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.216 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.216 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.216 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.216 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.216 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.216 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.216 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.216 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.217 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.217 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.217 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.217 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.217 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.217 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.217 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.217 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "nfs", 2024-04-25T00:36:22.217 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.217 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.217 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.218 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.218 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.218 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.218 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.218 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.218 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.218 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.218 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.218 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.218 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.218 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.218 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.219 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.219 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.219 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.219 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.219 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.219 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.219 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.219 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.219 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.219 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.219 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.220 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.220 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.220 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.220 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.220 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.220 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.220 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.220 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.220 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.220 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.221 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.221 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.221 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.221 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.221 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.221 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.221 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.221 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.221 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.221 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.221 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.221 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.222 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.222 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.222 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.222 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.222 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.222 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.222 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.222 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.222 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.222 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.222 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.223 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.223 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.223 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.223 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.223 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.223 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.223 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.223 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.223 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.223 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.223 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.224 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.224 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "orchestrator", 2024-04-25T00:36:22.224 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.224 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.224 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.224 INFO:teuthology.orchestra.run.smithi029.stdout: "fail_fs": { 2024-04-25T00:36:22.224 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "fail_fs", 2024-04-25T00:36:22.224 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.224 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.225 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.225 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.225 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.225 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.225 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.225 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:36:22.225 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.225 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.225 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.225 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.225 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.226 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.226 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.226 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.226 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.226 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.226 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.226 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.226 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.226 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.226 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.227 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.227 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.227 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.227 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.227 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.227 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.227 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.227 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.227 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.227 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.227 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.228 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.228 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.228 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.228 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.228 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.228 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.228 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.228 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.228 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.228 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.228 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.228 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.229 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.229 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.229 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.229 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.229 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.229 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.229 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.229 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.229 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.229 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.229 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.229 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.230 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.230 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.230 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.230 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.230 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.230 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.230 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.230 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.230 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.230 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.231 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.231 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.231 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.231 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.231 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.231 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.231 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.231 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.231 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator": { 2024-04-25T00:36:22.231 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "orchestrator", 2024-04-25T00:36:22.231 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm", 2024-04-25T00:36:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "rook", 2024-04-25T00:36:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "test_orchestrator" 2024-04-25T00:36:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.232 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Orchestrator backend", 2024-04-25T00:36:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "osd_perf_query", 2024-04-25T00:36:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.233 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.234 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.234 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.234 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.234 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.234 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.234 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.234 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.234 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.234 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.234 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.234 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.235 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.235 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.235 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.235 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.235 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.235 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.235 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.235 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.235 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.235 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.235 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.236 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.237 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.238 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.238 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.238 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.238 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.238 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.238 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.238 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.239 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.239 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.239 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.239 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.239 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.239 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.239 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.239 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.240 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.240 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.240 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.240 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.240 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.240 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.240 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "osd_support", 2024-04-25T00:36:22.240 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.240 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.240 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.241 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.241 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.241 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.241 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.241 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.241 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.241 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.241 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.241 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.241 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.241 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.241 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.241 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.242 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.242 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.242 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.242 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.242 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.242 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.242 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.242 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.242 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.242 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.243 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.243 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.243 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.243 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.243 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.243 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.243 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.243 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.243 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.243 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.243 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.244 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.244 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.244 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.244 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.244 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.244 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.244 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.244 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.244 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.244 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.244 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.244 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.245 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.245 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.245 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.245 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.245 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.245 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.245 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.245 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.245 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.245 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.245 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.246 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.246 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.246 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.246 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.246 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.246 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.246 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.246 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.246 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.246 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.246 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.247 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pg_autoscaler", 2024-04-25T00:36:22.247 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.247 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.247 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.247 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.247 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.247 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.247 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.247 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.247 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.249 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.249 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.249 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.249 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.249 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.249 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.249 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.249 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.249 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.249 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.249 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.250 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.250 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.250 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.250 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.250 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.250 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.250 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.250 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.250 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.250 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.250 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.251 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.251 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.251 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.251 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.251 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.251 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.251 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.251 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.251 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.251 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.251 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.251 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.251 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.252 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.252 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.252 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.252 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.252 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.252 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.252 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.252 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.252 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.252 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.252 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.253 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.253 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.253 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-04-25T00:36:22.253 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-04-25T00:36:22.253 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.253 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.253 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.253 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-04-25T00:36:22.253 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.253 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.253 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.253 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.254 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.254 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.254 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.254 INFO:teuthology.orchestra.run.smithi029.stdout: "threshold": { 2024-04-25T00:36:22.254 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "threshold", 2024-04-25T00:36:22.254 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:22.254 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.254 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.254 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3.0", 2024-04-25T00:36:22.254 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1.0", 2024-04-25T00:36:22.254 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.255 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.255 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "scaling threshold", 2024-04-25T00:36:22.255 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:36:22.255 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.255 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.255 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.255 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.255 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "progress", 2024-04-25T00:36:22.255 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.255 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.255 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.256 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_pg_recovery_event": { 2024-04-25T00:36:22.256 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:36:22.256 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.256 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.256 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.256 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.256 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.256 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.256 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.256 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:36:22.256 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.256 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.257 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.257 INFO:teuthology.orchestra.run.smithi029.stdout: "enabled": { 2024-04-25T00:36:22.257 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enabled", 2024-04-25T00:36:22.257 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.257 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.257 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.257 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.257 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.257 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.257 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.257 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.258 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.258 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.258 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.258 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.258 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.258 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.258 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.258 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.258 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.258 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.258 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.259 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.259 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.259 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.259 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.259 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.259 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.259 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.259 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.259 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.259 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.259 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.259 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.260 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.260 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.260 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.260 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.260 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.260 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.260 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.260 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.260 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.260 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.260 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.260 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.261 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.261 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.261 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.261 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.261 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.261 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.261 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.261 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.261 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.261 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.261 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.262 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.262 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.262 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.262 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.262 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.262 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.262 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.262 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.262 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.262 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.262 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.263 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.263 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.263 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.263 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.263 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.263 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.263 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.263 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.263 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.263 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.263 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: "max_completed_events": { 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_completed_events", 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "50", 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.264 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.265 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.265 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-04-25T00:36:22.265 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-04-25T00:36:22.265 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.265 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.265 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.265 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-04-25T00:36:22.265 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.265 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.266 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.266 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:36:22.266 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.266 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.266 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.266 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.266 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.266 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prometheus", 2024-04-25T00:36:22.266 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.266 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "cache": { 2024-04-25T00:36:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cache", 2024-04-25T00:36:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.267 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "exclude_perf_counters": { 2024-04-25T00:36:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "exclude_perf_counters", 2024-04-25T00:36:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.268 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:36:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:36:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.269 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.270 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.271 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.272 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.273 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.274 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.276 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_stats_pools": { 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_stats_pools", 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.277 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.278 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:36:22.278 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:36:22.278 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.278 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.278 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.278 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "300", 2024-04-25T00:36:22.278 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.278 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.278 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: "scrape_interval": { 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "scrape_interval", 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15.0", 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.279 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.280 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.280 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.280 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.280 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.280 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-04-25T00:36:22.280 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-04-25T00:36:22.280 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.280 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.280 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.280 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "::", 2024-04-25T00:36:22.280 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.280 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "9283", 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.281 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.282 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.282 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:36:22.282 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.282 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.282 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.282 INFO:teuthology.orchestra.run.smithi029.stdout: "stale_cache_strategy": { 2024-04-25T00:36:22.282 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "stale_cache_strategy", 2024-04-25T00:36:22.282 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.282 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.282 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.282 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "log", 2024-04-25T00:36:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_behaviour": { 2024-04-25T00:36:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_behaviour", 2024-04-25T00:36:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.283 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.284 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.284 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "default", 2024-04-25T00:36:22.284 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.284 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.284 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.284 INFO:teuthology.orchestra.run.smithi029.stdout: "default", 2024-04-25T00:36:22.284 INFO:teuthology.orchestra.run.smithi029.stdout: "error" 2024-04-25T00:36:22.284 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.284 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.284 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.284 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.285 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.285 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_error_status_code": { 2024-04-25T00:36:22.285 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_error_status_code", 2024-04-25T00:36:22.285 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.285 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.285 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.285 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "500", 2024-04-25T00:36:22.285 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "400", 2024-04-25T00:36:22.285 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "599", 2024-04-25T00:36:22.286 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.286 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.286 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.286 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.286 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.286 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.286 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.286 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_support", 2024-04-25T00:36:22.286 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.287 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.287 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.287 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.287 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.287 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.287 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.287 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.287 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.287 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.287 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.287 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.287 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.287 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.288 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.288 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.288 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.288 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.288 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.288 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.288 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.288 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.288 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.288 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.288 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.289 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.289 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.289 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.289 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.289 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.289 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.289 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.289 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.289 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.289 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.289 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.290 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.290 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.290 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.290 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.290 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.290 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.290 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.290 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.290 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.290 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.290 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.290 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.291 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.291 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.291 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.291 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.291 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.291 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.291 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.291 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.291 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.291 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.291 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.291 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.292 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.292 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.292 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.292 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.292 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.292 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.292 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.292 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.292 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.292 INFO:teuthology.orchestra.run.smithi029.stdout: "max_concurrent_snap_create": { 2024-04-25T00:36:22.292 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:36:22.293 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.293 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.293 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.293 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:22.293 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.293 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.293 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.293 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.293 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.293 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.293 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.294 INFO:teuthology.orchestra.run.smithi029.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:36:22.294 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:36:22.294 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.294 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.294 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.294 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.294 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.294 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.294 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.294 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.294 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.295 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.295 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.295 INFO:teuthology.orchestra.run.smithi029.stdout: "trash_purge_schedule": { 2024-04-25T00:36:22.295 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "trash_purge_schedule", 2024-04-25T00:36:22.295 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.295 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.295 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.295 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.295 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.295 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.295 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.295 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.296 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.296 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.296 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.296 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.296 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.296 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "restful", 2024-04-25T00:36:22.296 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.296 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.296 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.296 INFO:teuthology.orchestra.run.smithi029.stdout: "enable_auth": { 2024-04-25T00:36:22.297 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enable_auth", 2024-04-25T00:36:22.297 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.297 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.297 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.297 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.297 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.297 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.297 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.297 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.297 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.297 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.297 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.298 INFO:teuthology.orchestra.run.smithi029.stdout: "key_file": { 2024-04-25T00:36:22.298 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "key_file", 2024-04-25T00:36:22.298 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.298 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.298 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.298 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.298 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.298 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.299 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.299 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.299 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.299 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.299 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.299 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.299 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.299 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.299 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.299 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.300 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.300 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.300 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.300 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.300 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.300 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.300 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.300 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.301 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.301 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.301 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.301 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.301 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.301 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.301 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.301 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.301 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.302 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.302 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.302 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.302 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.302 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.302 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.302 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.302 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.302 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.302 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.302 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.303 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.304 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.305 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-04-25T00:36:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-04-25T00:36:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.306 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-04-25T00:36:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-04-25T00:36:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.307 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.308 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.308 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "selftest", 2024-04-25T00:36:22.308 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.308 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.308 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.308 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.308 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.308 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.309 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.309 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.309 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.309 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.309 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.309 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.309 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.309 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.309 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.309 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.309 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.309 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.309 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.310 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.310 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.310 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.310 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.310 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.310 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.310 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.310 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.310 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.310 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.310 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.311 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.311 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.311 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.311 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.311 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.311 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.311 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.311 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.311 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.311 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.311 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.311 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.312 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.312 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.312 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.312 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.312 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.312 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.312 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.312 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.312 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.312 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.312 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.312 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.313 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.313 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.313 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.313 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.313 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.313 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.313 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.313 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.313 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.313 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.313 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.313 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.314 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.314 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.314 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.314 INFO:teuthology.orchestra.run.smithi029.stdout: "roption1": { 2024-04-25T00:36:22.314 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "roption1", 2024-04-25T00:36:22.314 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.314 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.314 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.314 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.315 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.315 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.315 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.315 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.315 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.315 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.315 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.315 INFO:teuthology.orchestra.run.smithi029.stdout: "roption2": { 2024-04-25T00:36:22.315 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "roption2", 2024-04-25T00:36:22.315 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.315 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.316 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.316 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "xyz", 2024-04-25T00:36:22.316 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.316 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.316 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.316 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.316 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.316 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.316 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.316 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption1": { 2024-04-25T00:36:22.316 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption1", 2024-04-25T00:36:22.316 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.317 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.317 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.317 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.317 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.317 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.317 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.317 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.317 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.317 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.317 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.317 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption2": { 2024-04-25T00:36:22.317 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption2", 2024-04-25T00:36:22.318 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.318 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.318 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.318 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.318 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.318 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.318 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.318 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.318 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.318 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.318 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.318 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption3": { 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption3", 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.319 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption4": { 2024-04-25T00:36:22.320 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption4", 2024-04-25T00:36:22.320 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.320 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.320 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.320 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.320 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.320 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.320 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.320 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.320 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.320 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.320 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.321 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption5": { 2024-04-25T00:36:22.321 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption5", 2024-04-25T00:36:22.321 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.321 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.321 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.321 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.321 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.321 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.321 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.321 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.321 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.321 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.321 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.322 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption6": { 2024-04-25T00:36:22.322 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption6", 2024-04-25T00:36:22.322 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.322 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.322 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.322 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.322 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.322 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.322 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.322 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.323 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.323 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.323 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.323 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption7": { 2024-04-25T00:36:22.323 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption7", 2024-04-25T00:36:22.323 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.323 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.323 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.323 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.323 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-04-25T00:36:22.323 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "42", 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: "testkey": { 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testkey", 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.324 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.325 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.325 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.325 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.325 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.325 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.325 INFO:teuthology.orchestra.run.smithi029.stdout: "testlkey": { 2024-04-25T00:36:22.325 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testlkey", 2024-04-25T00:36:22.325 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.325 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.325 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.325 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.326 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.326 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.326 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.326 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.326 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.326 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.326 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.326 INFO:teuthology.orchestra.run.smithi029.stdout: "testnewline": { 2024-04-25T00:36:22.326 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testnewline", 2024-04-25T00:36:22.326 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.326 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.327 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.327 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.327 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.327 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.327 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.327 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.327 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.327 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.327 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.328 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snap_schedule", 2024-04-25T00:36:22.328 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.328 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.328 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.328 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_m_granularity": { 2024-04-25T00:36:22.328 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_m_granularity", 2024-04-25T00:36:22.328 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.328 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.328 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.328 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.328 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.328 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.328 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.329 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:36:22.329 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.329 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.329 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.329 INFO:teuthology.orchestra.run.smithi029.stdout: "dump_on_update": { 2024-04-25T00:36:22.329 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "dump_on_update", 2024-04-25T00:36:22.329 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.329 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.329 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.329 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.329 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.329 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.330 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.330 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:36:22.330 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.330 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.330 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.330 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.330 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.330 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.330 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.330 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.331 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.331 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.331 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.331 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.331 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.331 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.331 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.331 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.331 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.331 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.331 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.331 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.332 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.332 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.332 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.332 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.332 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.332 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.332 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.332 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.332 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.332 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.332 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.332 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.333 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.333 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.333 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.333 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.333 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.333 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.333 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.333 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.333 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.333 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.334 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.334 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.334 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.334 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.334 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.334 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.334 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.334 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.334 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.334 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.334 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.334 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.334 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.335 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.335 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.335 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.335 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.335 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.335 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.335 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.335 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.335 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.335 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.335 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.335 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "stats", 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.336 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.337 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.337 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.337 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.337 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.337 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.337 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.337 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.337 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.337 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.337 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.337 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.337 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.338 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.338 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.338 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.338 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.338 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.338 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.338 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.338 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.338 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.338 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.339 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.339 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.339 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.339 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.339 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.339 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.339 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.339 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.339 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.339 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.339 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.340 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.340 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.340 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.340 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.340 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.340 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.340 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.340 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.340 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.340 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.340 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.340 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.340 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.341 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.341 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.341 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.341 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.341 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.341 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.341 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.341 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.341 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.341 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.342 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.342 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.342 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.342 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.342 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.342 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.342 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.342 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.342 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "status", 2024-04-25T00:36:22.342 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.343 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.343 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.343 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.343 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.343 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.343 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.343 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.343 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.343 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.343 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.343 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.344 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.344 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.344 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.344 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.344 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.344 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.344 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.344 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.344 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.344 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.344 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.345 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.345 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.345 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.345 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.345 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.345 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.345 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.345 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.345 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.345 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.346 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.346 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.346 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.346 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.346 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.346 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.347 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.347 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.347 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.347 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.347 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.347 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.347 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.347 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.347 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.347 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.347 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.348 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.348 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.348 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.348 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.348 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.348 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.348 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.348 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.348 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.348 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.349 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.349 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.349 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.349 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.349 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.349 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.349 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.349 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.349 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.349 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.349 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.350 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "telegraf", 2024-04-25T00:36:22.350 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.350 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.350 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.350 INFO:teuthology.orchestra.run.smithi029.stdout: "address": { 2024-04-25T00:36:22.350 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "address", 2024-04-25T00:36:22.350 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.350 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.350 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.350 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:36:22.350 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.351 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.351 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.351 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.351 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.351 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.351 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.351 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-04-25T00:36:22.351 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-04-25T00:36:22.351 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.351 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.351 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.352 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15", 2024-04-25T00:36:22.352 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.352 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.352 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.352 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.352 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.352 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.352 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.352 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.352 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.352 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.353 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.353 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.353 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.353 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.353 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.353 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.353 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.353 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.353 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.353 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.353 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.353 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.354 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.354 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.354 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.354 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.354 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.354 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.354 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.354 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.354 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.354 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.355 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.355 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.355 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.355 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.355 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.355 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.355 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.355 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.355 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.355 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.355 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.356 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.356 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.356 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.356 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.356 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.356 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.356 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.356 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.356 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.356 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.356 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.356 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.356 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.357 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.357 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.357 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.357 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.357 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.357 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.357 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.357 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.357 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.357 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.357 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.358 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.358 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.358 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.358 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.358 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.358 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.358 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.358 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.358 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "telemetry", 2024-04-25T00:36:22.358 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.359 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.359 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.359 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_basic": { 2024-04-25T00:36:22.359 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_basic", 2024-04-25T00:36:22.359 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.359 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.359 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.359 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.359 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.359 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.359 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.359 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:36:22.360 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.360 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.360 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.360 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_crash": { 2024-04-25T00:36:22.360 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_crash", 2024-04-25T00:36:22.360 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.360 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.360 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.360 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.360 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.360 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.361 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.361 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:36:22.361 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.361 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.361 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.361 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_device": { 2024-04-25T00:36:22.361 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_device", 2024-04-25T00:36:22.361 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.361 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.361 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.362 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.362 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.362 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.362 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.362 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:36:22.362 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.362 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.362 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.362 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_ident": { 2024-04-25T00:36:22.362 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_ident", 2024-04-25T00:36:22.363 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.363 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.363 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.363 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.363 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.363 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.363 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.363 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:36:22.364 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.364 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.364 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.364 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_perf": { 2024-04-25T00:36:22.364 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_perf", 2024-04-25T00:36:22.364 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.364 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.364 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.365 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.365 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.365 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.365 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.365 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:36:22.365 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.365 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.365 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.365 INFO:teuthology.orchestra.run.smithi029.stdout: "contact": { 2024-04-25T00:36:22.365 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "contact", 2024-04-25T00:36:22.366 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.366 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.366 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.366 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.366 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.366 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.366 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.366 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.366 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.366 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.366 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.367 INFO:teuthology.orchestra.run.smithi029.stdout: "description": { 2024-04-25T00:36:22.367 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "description", 2024-04-25T00:36:22.367 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.367 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.367 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.367 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.367 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.367 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.367 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.367 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.367 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.368 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.368 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.368 INFO:teuthology.orchestra.run.smithi029.stdout: "device_url": { 2024-04-25T00:36:22.368 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_url", 2024-04-25T00:36:22.368 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.368 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.368 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.368 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:36:22.368 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.368 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.368 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.369 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.369 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.369 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.369 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.369 INFO:teuthology.orchestra.run.smithi029.stdout: "enabled": { 2024-04-25T00:36:22.369 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enabled", 2024-04-25T00:36:22.369 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.369 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.369 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.369 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.370 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.370 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.370 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.370 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.370 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.370 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.370 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.370 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-04-25T00:36:22.370 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-04-25T00:36:22.370 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.370 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.371 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.371 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "24", 2024-04-25T00:36:22.371 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "8", 2024-04-25T00:36:22.371 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.371 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.371 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.371 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.371 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.371 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.371 INFO:teuthology.orchestra.run.smithi029.stdout: "last_opt_revision": { 2024-04-25T00:36:22.372 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "last_opt_revision", 2024-04-25T00:36:22.372 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.372 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.372 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.372 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1", 2024-04-25T00:36:22.372 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.372 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.372 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.372 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.372 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.372 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.373 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.373 INFO:teuthology.orchestra.run.smithi029.stdout: "leaderboard": { 2024-04-25T00:36:22.373 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "leaderboard", 2024-04-25T00:36:22.373 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.373 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.373 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.373 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.373 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.373 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.373 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.373 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.373 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.374 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.374 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.374 INFO:teuthology.orchestra.run.smithi029.stdout: "leaderboard_description": { 2024-04-25T00:36:22.374 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "leaderboard_description", 2024-04-25T00:36:22.374 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.374 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.374 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.374 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.374 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.374 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.374 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.375 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.375 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.375 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.375 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.375 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.375 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.375 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.375 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.375 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.375 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.376 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.376 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.376 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.376 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.376 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.376 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.376 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.376 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.376 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.376 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.376 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.377 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.377 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.377 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.377 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.377 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.377 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.377 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.377 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.377 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.377 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.377 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.377 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.378 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.378 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.378 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.378 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.378 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.378 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.378 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.378 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.378 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.378 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.379 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.379 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.379 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.379 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.379 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.379 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.379 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.379 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.379 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.379 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.379 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.379 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.380 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.380 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.380 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.380 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.380 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.380 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.380 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.380 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.380 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.380 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.380 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.381 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.381 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.381 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.381 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.381 INFO:teuthology.orchestra.run.smithi029.stdout: "organization": { 2024-04-25T00:36:22.381 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "organization", 2024-04-25T00:36:22.381 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.381 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.381 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.381 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.381 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.382 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.382 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.382 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.382 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.382 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.382 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.382 INFO:teuthology.orchestra.run.smithi029.stdout: "proxy": { 2024-04-25T00:36:22.382 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "proxy", 2024-04-25T00:36:22.382 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.382 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.382 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.383 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.383 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.383 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.383 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.383 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.383 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.383 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.383 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.383 INFO:teuthology.orchestra.run.smithi029.stdout: "url": { 2024-04-25T00:36:22.383 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "url", 2024-04-25T00:36:22.383 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.384 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.384 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.384 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:36:22.384 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.384 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.384 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.384 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.384 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.384 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.384 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.384 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.385 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.385 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "test_orchestrator", 2024-04-25T00:36:22.385 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.385 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.385 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.385 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.385 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.385 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.385 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.385 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.386 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.386 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.386 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.386 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.386 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.386 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.386 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.386 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.386 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.386 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.386 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.386 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.387 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.387 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.387 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.387 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.387 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.387 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.387 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.387 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.387 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.387 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.387 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.388 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.388 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.388 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.388 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.388 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.388 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.388 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.388 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.388 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.388 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.388 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.388 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.389 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.389 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.389 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.389 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.389 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.389 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.389 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.389 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.389 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.389 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.389 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.389 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.390 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.390 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.390 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.390 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.390 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.390 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.390 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.390 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.390 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.390 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.391 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.391 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.391 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.391 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.391 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.391 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.391 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "volumes", 2024-04-25T00:36:22.391 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.391 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.391 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.391 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.392 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.392 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.392 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.392 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.392 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.392 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.392 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.392 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.392 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.392 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.392 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.392 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.393 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.393 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.393 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.393 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.393 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.393 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.393 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.393 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.393 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.393 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.393 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.394 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.394 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.394 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.394 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.394 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.394 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.394 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.394 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.394 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.395 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.395 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.395 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.395 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.395 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.395 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.395 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.395 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.395 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.396 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.396 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.396 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.396 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.396 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.396 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.396 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.396 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.396 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.396 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.396 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.397 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.397 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.397 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.397 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.397 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.397 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.397 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.397 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.397 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.397 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.397 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.398 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.398 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.398 INFO:teuthology.orchestra.run.smithi029.stdout: "max_concurrent_clones": { 2024-04-25T00:36:22.398 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_concurrent_clones", 2024-04-25T00:36:22.398 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.398 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.398 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.398 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "4", 2024-04-25T00:36:22.398 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.398 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.399 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.399 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:36:22.399 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.399 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.399 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.399 INFO:teuthology.orchestra.run.smithi029.stdout: "periodic_async_work": { 2024-04-25T00:36:22.399 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "periodic_async_work", 2024-04-25T00:36:22.399 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.399 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.399 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.399 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.399 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.400 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.400 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.400 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Periodically check for async work", 2024-04-25T00:36:22.400 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.400 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.400 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.400 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot_clone_delay": { 2024-04-25T00:36:22.400 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:36:22.400 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.400 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.400 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.401 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-04-25T00:36:22.401 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.401 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.401 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.401 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:36:22.401 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.401 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.401 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.401 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:36:22.401 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:36:22.402 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.402 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.402 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.402 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.402 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.402 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.402 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.402 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:36:22.402 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.402 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.402 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.402 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.403 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.403 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix", 2024-04-25T00:36:22.403 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.403 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.403 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.403 INFO:teuthology.orchestra.run.smithi029.stdout: "discovery_interval": { 2024-04-25T00:36:22.403 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "discovery_interval", 2024-04-25T00:36:22.403 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-04-25T00:36:22.403 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.403 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.404 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "100", 2024-04-25T00:36:22.404 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.404 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.404 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.404 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.404 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.404 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.404 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.404 INFO:teuthology.orchestra.run.smithi029.stdout: "identifier": { 2024-04-25T00:36:22.404 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "identifier", 2024-04-25T00:36:22.404 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.405 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.405 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.405 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.405 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.405 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.405 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.405 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.405 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.405 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.405 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.405 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-04-25T00:36:22.406 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-04-25T00:36:22.406 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.406 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.406 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.406 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-04-25T00:36:22.406 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.406 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.406 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.406 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.406 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.406 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.406 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.407 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.407 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.407 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.407 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.407 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.407 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.407 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.407 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.407 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.407 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.408 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.408 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.408 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.408 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.408 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.408 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.408 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.408 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.408 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.408 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.408 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.409 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.409 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.409 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.409 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.409 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.409 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.409 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.409 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.409 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.409 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.409 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.409 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.410 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.410 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.410 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.410 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.410 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.410 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.410 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.410 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.410 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.410 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.410 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.411 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.411 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.411 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.411 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.411 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.411 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.411 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.411 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.411 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.411 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.411 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.411 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.412 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.412 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.412 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.412 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.412 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.412 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.412 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.412 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.412 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.412 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.412 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_host": { 2024-04-25T00:36:22.413 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_host", 2024-04-25T00:36:22.413 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.413 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.413 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.413 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.413 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.413 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.413 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.413 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.413 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.413 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.413 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.414 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_port": { 2024-04-25T00:36:22.414 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_port", 2024-04-25T00:36:22.414 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.414 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.414 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.414 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10051", 2024-04-25T00:36:22.414 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.414 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.414 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.414 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.415 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.415 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.415 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.415 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_sender": { 2024-04-25T00:36:22.415 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_sender", 2024-04-25T00:36:22.415 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.415 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.415 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.415 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:36:22.415 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.416 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.416 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.416 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.416 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.416 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.416 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.416 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.416 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.416 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.416 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:22.416 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.416 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.416 INFO:teuthology.orchestra.run.smithi029.stdout: "modules": [ 2024-04-25T00:36:22.417 INFO:teuthology.orchestra.run.smithi029.stdout: "iostat", 2024-04-25T00:36:22.417 INFO:teuthology.orchestra.run.smithi029.stdout: "nfs", 2024-04-25T00:36:22.417 INFO:teuthology.orchestra.run.smithi029.stdout: "restful" 2024-04-25T00:36:22.417 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.417 INFO:teuthology.orchestra.run.smithi029.stdout: "available_modules": [ 2024-04-25T00:36:22.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.417 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "alerts", 2024-04-25T00:36:22.417 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.418 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.418 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.418 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-04-25T00:36:22.418 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-04-25T00:36:22.418 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.418 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.418 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.418 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-04-25T00:36:22.418 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.418 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.419 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.419 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:36:22.419 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.419 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.419 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.419 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.419 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.419 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.419 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.419 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.419 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.419 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.420 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.420 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.420 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.420 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.420 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.420 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.420 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.420 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.420 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.420 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.420 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.420 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.421 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.421 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.421 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.421 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.421 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.421 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.421 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.421 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.421 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.421 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.421 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.421 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.422 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.422 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.422 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.422 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.422 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.422 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.422 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.422 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.422 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.422 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.423 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.423 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.423 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.423 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.423 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.423 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.423 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.423 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.423 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.423 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.423 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.423 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.424 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.424 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.424 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.424 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.424 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.424 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.424 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.424 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.424 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.424 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.424 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.425 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.425 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.425 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_destination": { 2024-04-25T00:36:22.425 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_destination", 2024-04-25T00:36:22.425 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.425 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.425 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.425 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.425 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.425 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.425 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.425 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:36:22.426 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.426 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.426 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.426 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_from_name": { 2024-04-25T00:36:22.426 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_from_name", 2024-04-25T00:36:22.426 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.426 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.426 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.426 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "Ceph", 2024-04-25T00:36:22.426 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.426 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.427 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.427 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Email From: name", 2024-04-25T00:36:22.427 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.427 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.427 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.427 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_host": { 2024-04-25T00:36:22.427 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_host", 2024-04-25T00:36:22.427 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.427 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.427 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.427 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP server", 2024-04-25T00:36:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_password": { 2024-04-25T00:36:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_password", 2024-04-25T00:36:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.428 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.429 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.429 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.429 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.429 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.429 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Password to authenticate with", 2024-04-25T00:36:22.429 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.429 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.429 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.429 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_port": { 2024-04-25T00:36:22.429 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_port", 2024-04-25T00:36:22.429 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.430 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.430 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.430 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "465", 2024-04-25T00:36:22.430 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.430 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.430 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.430 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP port", 2024-04-25T00:36:22.430 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.430 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.430 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.431 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_sender": { 2024-04-25T00:36:22.431 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_sender", 2024-04-25T00:36:22.431 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.431 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.431 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.431 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.431 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.431 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.431 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.431 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:36:22.431 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.431 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.431 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.432 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_ssl": { 2024-04-25T00:36:22.432 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_ssl", 2024-04-25T00:36:22.432 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.432 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.432 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.432 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.432 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.432 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.432 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.432 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:36:22.432 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.433 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.433 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.433 INFO:teuthology.orchestra.run.smithi029.stdout: "smtp_user": { 2024-04-25T00:36:22.433 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "smtp_user", 2024-04-25T00:36:22.433 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.433 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.433 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.433 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.433 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.433 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.433 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.433 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "User to authenticate as", 2024-04-25T00:36:22.434 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.434 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.434 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.434 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.434 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.434 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "balancer", 2024-04-25T00:36:22.434 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.434 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.434 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.434 INFO:teuthology.orchestra.run.smithi029.stdout: "active": { 2024-04-25T00:36:22.435 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "active", 2024-04-25T00:36:22.435 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.435 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.435 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.435 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.435 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.435 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.435 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.435 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:36:22.435 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.435 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.435 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.436 INFO:teuthology.orchestra.run.smithi029.stdout: "begin_time": { 2024-04-25T00:36:22.436 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "begin_time", 2024-04-25T00:36:22.436 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.436 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.436 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.436 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0000", 2024-04-25T00:36:22.436 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.436 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.436 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.436 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:36:22.436 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:36:22.436 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.436 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.437 INFO:teuthology.orchestra.run.smithi029.stdout: "begin_weekday": { 2024-04-25T00:36:22.437 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "begin_weekday", 2024-04-25T00:36:22.437 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-04-25T00:36:22.437 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.437 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.437 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-04-25T00:36:22.437 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0", 2024-04-25T00:36:22.437 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "6", 2024-04-25T00:36:22.437 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.437 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:36:22.437 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:36:22.437 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.438 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.438 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_max_iterations": { 2024-04-25T00:36:22.438 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:36:22.438 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-04-25T00:36:22.438 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.438 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.438 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "25", 2024-04-25T00:36:22.438 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-04-25T00:36:22.438 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "250", 2024-04-25T00:36:22.438 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.439 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:36:22.439 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.439 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.439 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.439 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_metrics": { 2024-04-25T00:36:22.439 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_metrics", 2024-04-25T00:36:22.439 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.439 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.439 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.439 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:36:22.439 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.439 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.440 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.440 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:36:22.440 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:36:22.440 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.440 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.440 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_compat_step": { 2024-04-25T00:36:22.440 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crush_compat_step", 2024-04-25T00:36:22.440 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:22.440 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.440 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.440 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0.5", 2024-04-25T00:36:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0.001", 2024-04-25T00:36:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "0.999", 2024-04-25T00:36:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:36:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:36:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "end_time": { 2024-04-25T00:36:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "end_time", 2024-04-25T00:36:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.441 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "2359", 2024-04-25T00:36:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:36:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:36:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.442 INFO:teuthology.orchestra.run.smithi029.stdout: "end_weekday": { 2024-04-25T00:36:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "end_weekday", 2024-04-25T00:36:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-04-25T00:36:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-04-25T00:36:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "0", 2024-04-25T00:36:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "6", 2024-04-25T00:36:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:36:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:36:22.443 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.444 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.445 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.446 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.447 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.448 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.449 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "min_score": { 2024-04-25T00:36:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "min_score", 2024-04-25T00:36:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-04-25T00:36:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.450 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:36:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": { 2024-04-25T00:36:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mode", 2024-04-25T00:36:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "upmap", 2024-04-25T00:36:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.451 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-compat", 2024-04-25T00:36:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "none", 2024-04-25T00:36:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "read", 2024-04-25T00:36:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap", 2024-04-25T00:36:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap-read" 2024-04-25T00:36:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Balancer mode", 2024-04-25T00:36:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.452 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_ids": { 2024-04-25T00:36:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pool_ids", 2024-04-25T00:36:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:36:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.453 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-04-25T00:36:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-04-25T00:36:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-04-25T00:36:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.454 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap_max_deviation": { 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "upmap_max_deviation", 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.455 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:36:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:36:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "upmap_max_optimizations": { 2024-04-25T00:36:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:36:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-04-25T00:36:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.456 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:36:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephadm", 2024-04-25T00:36:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.457 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_down_multiplier": { 2024-04-25T00:36:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_down_multiplier", 2024-04-25T00:36:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3.0", 2024-04-25T00:36:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:36:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.458 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_refresh_rate": { 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_refresh_rate", 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "20", 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.459 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "agent_starting_port": { 2024-04-25T00:36:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "agent_starting_port", 2024-04-25T00:36:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "4721", 2024-04-25T00:36:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.460 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:36:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_ptrace": { 2024-04-25T00:36:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_ptrace", 2024-04-25T00:36:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.461 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:36:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:36:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "autotune_interval": { 2024-04-25T00:36:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "autotune_interval", 2024-04-25T00:36:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.462 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-04-25T00:36:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:36:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:36:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:36:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:22.463 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0.7", 2024-04-25T00:36:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:36:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.464 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm_log_destination": { 2024-04-25T00:36:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephadm_log_destination", 2024-04-25T00:36:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.465 INFO:teuthology.orchestra.run.smithi029.stdout: "file", 2024-04-25T00:36:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "file,syslog", 2024-04-25T00:36:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "syslog" 2024-04-25T00:36:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:36:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "cgroups_split": { 2024-04-25T00:36:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cgroups_split", 2024-04-25T00:36:22.466 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:36:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.467 INFO:teuthology.orchestra.run.smithi029.stdout: "config_checks_enabled": { 2024-04-25T00:36:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "config_checks_enabled", 2024-04-25T00:36:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:36:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.468 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "config_dashboard": { 2024-04-25T00:36:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "config_dashboard", 2024-04-25T00:36:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:36:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.469 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_alertmanager": { 2024-04-25T00:36:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_alertmanager", 2024-04-25T00:36:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:36:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.470 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-04-25T00:36:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_base": { 2024-04-25T00:36:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_base", 2024-04-25T00:36:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:36:22.471 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:36:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_elasticsearch": { 2024-04-25T00:36:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:36:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.472 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:36:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "elasticsearch container image", 2024-04-25T00:36:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_grafana": { 2024-04-25T00:36:22.473 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_grafana", 2024-04-25T00:36:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:36:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-04-25T00:36:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_haproxy": { 2024-04-25T00:36:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_haproxy", 2024-04-25T00:36:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:36:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "HAproxy container image", 2024-04-25T00:36:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.475 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_agent": { 2024-04-25T00:36:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:36:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:36:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:36:22.476 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_collector": { 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:36:22.477 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_jaeger_query": { 2024-04-25T00:36:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:36:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:36:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.478 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Jaeger query container image", 2024-04-25T00:36:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_keepalived": { 2024-04-25T00:36:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_keepalived", 2024-04-25T00:36:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:36:22.479 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Keepalived container image", 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_loki": { 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_loki", 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:36:22.480 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Loki container image", 2024-04-25T00:36:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_node_exporter": { 2024-04-25T00:36:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_node_exporter", 2024-04-25T00:36:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.481 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:36:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-04-25T00:36:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_nvmeof": { 2024-04-25T00:36:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_nvmeof", 2024-04-25T00:36:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.482 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:36:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Nvme-of container image", 2024-04-25T00:36:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_prometheus": { 2024-04-25T00:36:22.483 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_prometheus", 2024-04-25T00:36:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:36:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Prometheus container image", 2024-04-25T00:36:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.484 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_promtail": { 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_promtail", 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Promtail container image", 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.485 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_samba": { 2024-04-25T00:36:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_samba", 2024-04-25T00:36:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:36:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:36:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.486 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image_snmp_gateway": { 2024-04-25T00:36:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:36:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:36:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.487 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:36:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "container_init": { 2024-04-25T00:36:22.488 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "container_init", 2024-04-25T00:36:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:36:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.489 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "daemon_cache_timeout": { 2024-04-25T00:36:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:36:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-04-25T00:36:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:36:22.490 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:36:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:36:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "900", 2024-04-25T00:36:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.491 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:36:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "default_registry": { 2024-04-25T00:36:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "default_registry", 2024-04-25T00:36:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "docker.io", 2024-04-25T00:36:22.492 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:36:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "device_cache_timeout": { 2024-04-25T00:36:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_cache_timeout", 2024-04-25T00:36:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.493 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1800", 2024-04-25T00:36:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:36:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.494 INFO:teuthology.orchestra.run.smithi029.stdout: "device_enhanced_scan": { 2024-04-25T00:36:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_enhanced_scan", 2024-04-25T00:36:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:36:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.495 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "facts_cache_timeout": { 2024-04-25T00:36:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "facts_cache_timeout", 2024-04-25T00:36:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-04-25T00:36:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.496 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:36:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "grafana_dashboards_path": { 2024-04-25T00:36:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:36:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:36:22.497 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:36:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "host_check_interval": { 2024-04-25T00:36:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "host_check_interval", 2024-04-25T00:36:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.498 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-04-25T00:36:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:36:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "hw_monitoring": { 2024-04-25T00:36:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "hw_monitoring", 2024-04-25T00:36:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.499 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:36:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "inventory_list_all": { 2024-04-25T00:36:22.500 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "inventory_list_all", 2024-04-25T00:36:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:36:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.502 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "log_refresh_metadata": { 2024-04-25T00:36:22.503 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_refresh_metadata", 2024-04-25T00:36:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:36:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:36:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.505 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.506 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.507 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.508 INFO:teuthology.orchestra.run.smithi029.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:36:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:36:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:36:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.509 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:36:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:36:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "*", 2024-04-25T00:36:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:36:22.510 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "max_count_per_host": { 2024-04-25T00:36:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_count_per_host", 2024-04-25T00:36:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.511 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:36:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd_draining_count": { 2024-04-25T00:36:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_osd_draining_count", 2024-04-25T00:36:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.512 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:36:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "migration_current": { 2024-04-25T00:36:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "migration_current", 2024-04-25T00:36:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.513 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "internal - do not modify", 2024-04-25T00:36:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": { 2024-04-25T00:36:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mode", 2024-04-25T00:36:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.514 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "root", 2024-04-25T00:36:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm-package", 2024-04-25T00:36:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "root" 2024-04-25T00:36:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:36:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "oob_default_addr": { 2024-04-25T00:36:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "oob_default_addr", 2024-04-25T00:36:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "169.254.1.1", 2024-04-25T00:36:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:36:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.516 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "prometheus_alerts_path": { 2024-04-25T00:36:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:36:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:36:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:36:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_insecure": { 2024-04-25T00:36:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_insecure", 2024-04-25T00:36:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.518 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:36:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_password": { 2024-04-25T00:36:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_password", 2024-04-25T00:36:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.519 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:36:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_url": { 2024-04-25T00:36:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_url", 2024-04-25T00:36:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.520 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:36:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.521 INFO:teuthology.orchestra.run.smithi029.stdout: "registry_username": { 2024-04-25T00:36:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "registry_username", 2024-04-25T00:36:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:36:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.522 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "secure_monitoring_stack": { 2024-04-25T00:36:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:36:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.523 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:36:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "service_discovery_port": { 2024-04-25T00:36:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "service_discovery_port", 2024-04-25T00:36:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8765", 2024-04-25T00:36:22.524 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:36:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "ssh_config_file": { 2024-04-25T00:36:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssh_config_file", 2024-04-25T00:36:22.525 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:36:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "use_agent": { 2024-04-25T00:36:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "use_agent", 2024-04-25T00:36:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:36:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.527 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "use_repo_digest": { 2024-04-25T00:36:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "use_repo_digest", 2024-04-25T00:36:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.528 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:36:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_failed_host_check": { 2024-04-25T00:36:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:36:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.529 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:36:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_stray_daemons": { 2024-04-25T00:36:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:36:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.530 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:36:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_on_stray_hosts": { 2024-04-25T00:36:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:36:22.531 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:36:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crash", 2024-04-25T00:36:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.533 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.534 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.535 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.536 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.537 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "retain_interval": { 2024-04-25T00:36:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "retain_interval", 2024-04-25T00:36:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "31536000", 2024-04-25T00:36:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:36:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.539 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_recent_interval": { 2024-04-25T00:36:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_recent_interval", 2024-04-25T00:36:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1209600", 2024-04-25T00:36:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.540 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:36:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "dashboard", 2024-04-25T00:36:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.541 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:36:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:36:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.542 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:36:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:36:22.543 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.544 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:36:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:36:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.545 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:36:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:36:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.546 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:36:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:36:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.547 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:36:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:36:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.548 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:36:22.549 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:36:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:36:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:36:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.551 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:36:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:36:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.552 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:36:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:36:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.553 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:36:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:36:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.554 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:36:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:36:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.555 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.556 INFO:teuthology.orchestra.run.smithi029.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:36:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:36:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.557 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:36:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:36:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.558 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:36:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:36:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-04-25T00:36:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.559 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.560 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.561 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.561 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.561 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_URL": { 2024-04-25T00:36:22.561 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:36:22.561 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.561 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.561 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.561 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.561 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.561 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.561 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.562 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.562 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.562 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.562 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.562 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:36:22.562 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:36:22.562 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.562 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.562 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.562 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-04-25T00:36:22.562 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.563 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.563 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.563 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.563 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.563 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.563 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.563 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:36:22.563 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:36:22.563 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.563 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.564 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.564 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.564 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.564 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.564 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.564 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.564 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.564 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.564 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.564 INFO:teuthology.orchestra.run.smithi029.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:36:22.564 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:36:22.565 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.565 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.565 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.565 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.565 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.565 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.565 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.565 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.565 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.565 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.565 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.565 INFO:teuthology.orchestra.run.smithi029.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:36:22.566 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:36:22.566 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.566 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.566 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.566 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.566 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.566 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.566 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.566 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.566 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.567 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.567 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.567 INFO:teuthology.orchestra.run.smithi029.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:36:22.567 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:36:22.567 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.567 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.567 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.567 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.567 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.567 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.568 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.568 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.568 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.568 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.568 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.568 INFO:teuthology.orchestra.run.smithi029.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:36:22.568 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:36:22.568 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.568 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.568 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.568 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.568 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.569 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.569 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.569 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.569 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.569 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.569 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.569 INFO:teuthology.orchestra.run.smithi029.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:36:22.569 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:36:22.569 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.569 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.569 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.570 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.570 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.570 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.570 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.570 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.570 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.570 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.570 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.570 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:36:22.571 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:36:22.571 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.571 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.571 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.571 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.571 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.571 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.571 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.571 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.572 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.572 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.572 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.572 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:36:22.572 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:36:22.572 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.572 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.572 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.572 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.572 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.572 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.573 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.573 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.573 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.573 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.573 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.573 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:36:22.573 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:36:22.573 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.573 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.573 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.574 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.574 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.574 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.574 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.574 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.574 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.574 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.574 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.574 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:36:22.574 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:36:22.574 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.575 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.575 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.575 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.575 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.575 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.575 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.575 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.575 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.575 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.575 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.575 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:36:22.576 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:36:22.576 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.576 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.576 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.576 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.576 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.576 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.576 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.576 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.576 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.576 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.576 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.577 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:36:22.577 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:36:22.577 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.577 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.577 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.577 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.577 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.577 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.577 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.577 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.577 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.578 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.578 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.578 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:36:22.578 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:36:22.578 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.578 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.578 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.578 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.578 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.578 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.578 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:36:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:36:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.580 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.580 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.580 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.580 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.580 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.580 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:36:22.580 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:36:22.580 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.580 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.580 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.580 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:36:22.580 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:22.581 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.582 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.582 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.582 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.582 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.582 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.582 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.582 INFO:teuthology.orchestra.run.smithi029.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:36:22.582 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:36:22.582 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.582 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.582 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.583 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8", 2024-04-25T00:36:22.583 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.583 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.583 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.583 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.583 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.583 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.583 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.583 INFO:teuthology.orchestra.run.smithi029.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:36:22.583 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:36:22.583 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.583 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.584 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.584 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "45", 2024-04-25T00:36:22.584 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.584 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.584 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.584 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.584 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.584 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.584 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.584 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:36:22.584 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:36:22.585 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.585 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.585 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.585 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.585 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.585 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.585 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.585 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.585 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.585 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.585 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.586 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:36:22.586 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:36:22.586 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.586 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.586 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.586 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "admin", 2024-04-25T00:36:22.586 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.586 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.586 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.586 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.586 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.586 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.587 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.587 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:36:22.587 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:36:22.587 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.587 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.587 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.587 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.587 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.587 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.587 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.587 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.588 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.588 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.588 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.588 INFO:teuthology.orchestra.run.smithi029.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:36:22.588 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:36:22.588 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.588 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.588 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.588 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.588 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.588 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.589 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.589 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.589 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.589 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.589 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.589 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:36:22.589 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:36:22.589 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.589 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.589 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.589 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-04-25T00:36:22.589 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.590 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.590 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.590 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.590 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.590 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.590 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.590 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:36:22.590 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:36:22.590 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.590 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.590 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.591 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:22.591 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.591 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.591 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.591 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.591 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.591 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.591 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.591 INFO:teuthology.orchestra.run.smithi029.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:36:22.591 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:36:22.591 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.592 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.592 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.592 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-04-25T00:36:22.592 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.592 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.592 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.592 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.592 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.592 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.592 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.592 INFO:teuthology.orchestra.run.smithi029.stdout: "cross_origin_url": { 2024-04-25T00:36:22.593 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cross_origin_url", 2024-04-25T00:36:22.593 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.593 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.593 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.593 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.593 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.593 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.593 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.593 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.594 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.594 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.594 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.594 INFO:teuthology.orchestra.run.smithi029.stdout: "crt_file": { 2024-04-25T00:36:22.594 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "crt_file", 2024-04-25T00:36:22.594 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.594 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.594 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.595 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.595 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.595 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.595 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.595 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.595 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.595 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.595 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.595 INFO:teuthology.orchestra.run.smithi029.stdout: "debug": { 2024-04-25T00:36:22.595 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "debug", 2024-04-25T00:36:22.595 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.596 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.596 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.596 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.596 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.596 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.596 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.596 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:36:22.596 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.596 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.596 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.596 INFO:teuthology.orchestra.run.smithi029.stdout: "jwt_token_ttl": { 2024-04-25T00:36:22.596 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "jwt_token_ttl", 2024-04-25T00:36:22.597 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.597 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.597 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.597 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "28800", 2024-04-25T00:36:22.597 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.597 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.597 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.597 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.597 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.597 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.597 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.597 INFO:teuthology.orchestra.run.smithi029.stdout: "key_file": { 2024-04-25T00:36:22.598 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "key_file", 2024-04-25T00:36:22.598 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.598 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.598 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.598 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.598 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.598 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.598 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.598 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.598 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.599 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.599 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.599 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.599 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.599 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.599 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.599 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.599 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.599 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.599 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.599 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.600 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.600 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.600 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.600 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.600 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.600 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.600 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.600 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.600 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.600 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.600 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.601 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.602 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.603 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.604 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "motd": { 2024-04-25T00:36:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "motd", 2024-04-25T00:36:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "The message of the day", 2024-04-25T00:36:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:36:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:36:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.606 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-04-25T00:36:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-04-25T00:36:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "::", 2024-04-25T00:36:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.607 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-04-25T00:36:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-04-25T00:36:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8080", 2024-04-25T00:36:22.608 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl": { 2024-04-25T00:36:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl", 2024-04-25T00:36:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.609 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl_server_port": { 2024-04-25T00:36:22.610 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl_server_port", 2024-04-25T00:36:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8443", 2024-04-25T00:36:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_behaviour": { 2024-04-25T00:36:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_behaviour", 2024-04-25T00:36:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "redirect", 2024-04-25T00:36:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: "redirect" 2024-04-25T00:36:22.612 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_error_status_code": { 2024-04-25T00:36:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_error_status_code", 2024-04-25T00:36:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "500", 2024-04-25T00:36:22.613 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "400", 2024-04-25T00:36:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "599", 2024-04-25T00:36:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "url_prefix": { 2024-04-25T00:36:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "url_prefix", 2024-04-25T00:36:22.614 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.615 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "devicehealth", 2024-04-25T00:36:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "enable_monitoring": { 2024-04-25T00:36:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enable_monitoring", 2024-04-25T00:36:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.616 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "monitor device health metrics", 2024-04-25T00:36:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.617 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.618 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.619 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.620 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.621 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.622 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "mark_out_threshold": { 2024-04-25T00:36:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mark_out_threshold", 2024-04-25T00:36:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.623 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "2419200", 2024-04-25T00:36:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:36:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": { 2024-04-25T00:36:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pool_name", 2024-04-25T00:36:22.624 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "device_health_metrics", 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "retention_period": { 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "retention_period", 2024-04-25T00:36:22.625 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15552000", 2024-04-25T00:36:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:36:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.626 INFO:teuthology.orchestra.run.smithi029.stdout: "scrape_frequency": { 2024-04-25T00:36:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "scrape_frequency", 2024-04-25T00:36:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "86400", 2024-04-25T00:36:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:36:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "self_heal": { 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "self_heal", 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-04-25T00:36:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-04-25T00:36:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-04-25T00:36:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:36:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.629 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "warn_threshold": { 2024-04-25T00:36:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "warn_threshold", 2024-04-25T00:36:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "7257600", 2024-04-25T00:36:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.630 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "diskprediction_local", 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.631 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.632 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.633 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.634 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.635 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.636 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "predict_interval": { 2024-04-25T00:36:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "predict_interval", 2024-04-25T00:36:22.637 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "86400", 2024-04-25T00:36:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.638 INFO:teuthology.orchestra.run.smithi029.stdout: "predictor_model": { 2024-04-25T00:36:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "predictor_model", 2024-04-25T00:36:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "prophetstor", 2024-04-25T00:36:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.639 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-04-25T00:36:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-04-25T00:36:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "600", 2024-04-25T00:36:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.640 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "influx", 2024-04-25T00:36:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": false, 2024-04-25T00:36:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:36:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "batch_size": { 2024-04-25T00:36:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "batch_size", 2024-04-25T00:36:22.641 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5000", 2024-04-25T00:36:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:36:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.642 INFO:teuthology.orchestra.run.smithi029.stdout: "database": { 2024-04-25T00:36:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "database", 2024-04-25T00:36:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "ceph", 2024-04-25T00:36:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:36:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.643 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": { 2024-04-25T00:36:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "hostname", 2024-04-25T00:36:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.644 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:36:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-04-25T00:36:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-04-25T00:36:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "30", 2024-04-25T00:36:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "5", 2024-04-25T00:36:22.645 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:36:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.646 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.647 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.648 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.649 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.650 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.651 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "password": { 2024-04-25T00:36:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "password", 2024-04-25T00:36:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.652 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:36:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "port": { 2024-04-25T00:36:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "port", 2024-04-25T00:36:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.653 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "8086", 2024-04-25T00:36:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "InfluxDB server port", 2024-04-25T00:36:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "ssl": { 2024-04-25T00:36:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "ssl", 2024-04-25T00:36:22.654 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "false", 2024-04-25T00:36:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:36:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "threads": { 2024-04-25T00:36:22.655 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "threads", 2024-04-25T00:36:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-04-25T00:36:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-04-25T00:36:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "32", 2024-04-25T00:36:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:36:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "username": { 2024-04-25T00:36:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "username", 2024-04-25T00:36:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:36:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.657 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "verify_ssl": { 2024-04-25T00:36:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "verify_ssl", 2024-04-25T00:36:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "true", 2024-04-25T00:36:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.658 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:36:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "insights", 2024-04-25T00:36:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.659 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.660 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.661 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.662 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.663 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.664 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.665 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "iostat", 2024-04-25T00:36:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.666 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.667 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.668 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.669 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.670 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.671 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "localpool", 2024-04-25T00:36:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.672 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "failure_domain": { 2024-04-25T00:36:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "failure_domain", 2024-04-25T00:36:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "host", 2024-04-25T00:36:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:36:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:36:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.673 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.674 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.675 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.677 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.678 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.681 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.682 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": { 2024-04-25T00:36:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "min_size", 2024-04-25T00:36:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:36:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:36:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "num_rep": { 2024-04-25T00:36:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "num_rep", 2024-04-25T00:36:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3", 2024-04-25T00:36:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:36:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.684 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": { 2024-04-25T00:36:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pg_num", 2024-04-25T00:36:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "128", 2024-04-25T00:36:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:36:22.685 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "prefix": { 2024-04-25T00:36:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prefix", 2024-04-25T00:36:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.686 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:36:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "subtree": { 2024-04-25T00:36:22.687 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "subtree", 2024-04-25T00:36:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "rack", 2024-04-25T00:36:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:36:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:36:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.688 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mirroring", 2024-04-25T00:36:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.689 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.690 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.691 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.692 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.693 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.694 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "nfs", 2024-04-25T00:36:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.695 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.696 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.699 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.700 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.701 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "orchestrator", 2024-04-25T00:36:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "fail_fs": { 2024-04-25T00:36:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "fail_fs", 2024-04-25T00:36:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.702 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:36:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.703 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.704 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.706 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.707 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.708 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.708 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.708 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.708 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.708 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.708 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.708 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.708 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.708 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator": { 2024-04-25T00:36:22.708 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "orchestrator", 2024-04-25T00:36:22.708 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.709 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.709 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.709 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.709 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.709 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.709 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.709 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm", 2024-04-25T00:36:22.709 INFO:teuthology.orchestra.run.smithi029.stdout: "rook", 2024-04-25T00:36:22.709 INFO:teuthology.orchestra.run.smithi029.stdout: "test_orchestrator" 2024-04-25T00:36:22.709 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.709 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Orchestrator backend", 2024-04-25T00:36:22.710 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.710 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.710 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.710 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.710 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.710 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "osd_perf_query", 2024-04-25T00:36:22.711 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.711 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.711 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.711 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.711 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.711 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.711 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.711 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.711 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.711 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.711 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.711 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.712 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.712 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.712 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.712 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.712 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.712 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.712 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.712 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.712 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.712 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.712 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.712 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.713 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.713 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.713 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.713 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.713 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.713 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.713 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.713 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.713 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.713 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.713 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.714 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.714 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.714 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.714 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.714 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.714 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.714 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.714 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.714 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.714 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.715 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.715 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.715 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.715 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.715 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.715 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.715 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.715 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.715 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.715 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.715 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.716 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "osd_support", 2024-04-25T00:36:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.717 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.718 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.719 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.720 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.721 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.721 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.721 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.721 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.721 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.721 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.721 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.721 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.721 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.721 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.721 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.721 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.722 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.722 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.722 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.722 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.722 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.722 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.722 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.722 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.722 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.722 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.722 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.723 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.723 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.723 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.723 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.723 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.723 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.723 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.723 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.723 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.723 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.723 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "pg_autoscaler", 2024-04-25T00:36:22.723 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.724 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.724 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.724 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.724 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.724 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.724 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.724 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.724 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.724 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.724 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.724 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.724 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.724 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.725 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.725 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.725 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.725 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.725 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.725 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.725 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.725 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.725 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.725 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.725 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.726 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.726 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.726 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.726 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.726 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.726 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.726 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.726 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.726 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.726 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.726 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.726 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.726 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.727 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.727 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.727 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.727 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.727 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.727 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.727 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.727 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.727 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.727 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.727 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.727 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.728 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.728 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.728 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.728 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.728 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.728 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.728 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.728 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.728 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.728 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.728 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.728 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.729 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.729 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.729 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.729 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.729 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.729 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.729 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.729 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.729 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.729 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.729 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.729 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-04-25T00:36:22.729 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-04-25T00:36:22.730 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.730 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.730 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.730 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-04-25T00:36:22.730 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.730 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.730 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.730 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.730 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.730 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.730 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.730 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.731 INFO:teuthology.orchestra.run.smithi029.stdout: "threshold": { 2024-04-25T00:36:22.731 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "threshold", 2024-04-25T00:36:22.731 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:22.731 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.731 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.731 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "3.0", 2024-04-25T00:36:22.731 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1.0", 2024-04-25T00:36:22.731 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.731 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.731 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "scaling threshold", 2024-04-25T00:36:22.731 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:36:22.731 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.731 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.732 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.732 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.732 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.732 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.732 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "progress", 2024-04-25T00:36:22.732 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.732 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.732 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.732 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_pg_recovery_event": { 2024-04-25T00:36:22.732 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:36:22.732 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.732 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.733 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.733 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.733 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.733 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.733 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.733 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:36:22.733 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.733 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.733 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.733 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.733 INFO:teuthology.orchestra.run.smithi029.stdout: "enabled": { 2024-04-25T00:36:22.733 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enabled", 2024-04-25T00:36:22.733 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.734 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.734 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.734 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.734 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.734 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.734 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.734 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.734 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.734 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.734 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.734 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.735 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.735 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.735 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.735 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.735 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.735 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.735 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.735 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.735 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.735 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.735 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.735 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.735 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.736 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.736 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.736 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.736 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.736 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.736 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.736 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.736 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.736 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.736 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.736 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.736 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.736 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.737 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.737 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.737 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.737 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.737 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.737 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.737 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.737 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.737 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.737 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.737 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.737 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.737 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.738 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.738 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.738 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.738 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.738 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.738 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.738 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.738 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.738 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.738 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.738 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.738 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.739 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.739 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.739 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.739 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.739 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.739 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.739 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.739 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.739 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.739 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.739 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.739 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.739 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.740 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.740 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.740 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.740 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.740 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.740 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.740 INFO:teuthology.orchestra.run.smithi029.stdout: "max_completed_events": { 2024-04-25T00:36:22.740 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_completed_events", 2024-04-25T00:36:22.740 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.740 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.740 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.740 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "50", 2024-04-25T00:36:22.740 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.741 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.741 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.741 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:36:22.741 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.741 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.741 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.741 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.741 INFO:teuthology.orchestra.run.smithi029.stdout: "sleep_interval": { 2024-04-25T00:36:22.741 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "sleep_interval", 2024-04-25T00:36:22.741 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.741 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.741 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.742 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "5", 2024-04-25T00:36:22.742 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.742 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.742 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.742 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:36:22.742 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.742 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.742 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.742 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.742 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.742 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.743 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "prometheus", 2024-04-25T00:36:22.743 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.743 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.743 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.743 INFO:teuthology.orchestra.run.smithi029.stdout: "cache": { 2024-04-25T00:36:22.743 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cache", 2024-04-25T00:36:22.743 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.743 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.743 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.743 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.743 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.743 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.744 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.744 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.744 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.744 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.744 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.744 INFO:teuthology.orchestra.run.smithi029.stdout: "exclude_perf_counters": { 2024-04-25T00:36:22.744 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "exclude_perf_counters", 2024-04-25T00:36:22.744 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.744 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.744 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.744 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.745 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.745 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.745 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.745 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:36:22.745 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:36:22.745 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.745 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.745 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.745 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.745 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.746 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.746 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.746 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.746 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.746 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.746 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.746 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.746 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.747 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.747 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.747 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.747 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.747 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.747 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.747 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.747 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.747 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.747 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.747 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.748 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.748 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.748 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.748 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.748 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.748 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.748 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.748 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.748 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.748 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.748 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.748 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.749 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.749 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.749 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.749 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.749 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.749 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.749 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.749 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.749 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.749 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.749 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.749 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.750 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.750 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.750 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.750 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.750 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.750 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.750 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.750 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.750 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.750 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.750 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.751 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.751 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.751 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.751 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.751 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.751 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.751 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.751 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.751 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.751 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_stats_pools": { 2024-04-25T00:36:22.751 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_stats_pools", 2024-04-25T00:36:22.751 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.752 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.752 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.752 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.752 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.752 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.752 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.752 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.752 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.752 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.752 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.752 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:36:22.753 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:36:22.753 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.753 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.753 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.753 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "300", 2024-04-25T00:36:22.753 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.753 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.753 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.753 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.753 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.753 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.753 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.754 INFO:teuthology.orchestra.run.smithi029.stdout: "scrape_interval": { 2024-04-25T00:36:22.754 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "scrape_interval", 2024-04-25T00:36:22.754 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:22.754 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.754 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.754 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15.0", 2024-04-25T00:36:22.754 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.754 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.754 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.754 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.754 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.754 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.755 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.755 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-04-25T00:36:22.755 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-04-25T00:36:22.755 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.755 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.755 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.755 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "::", 2024-04-25T00:36:22.755 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.755 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.755 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.755 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:36:22.756 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.756 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.756 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.756 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-04-25T00:36:22.756 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-04-25T00:36:22.756 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.756 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.756 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.756 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "9283", 2024-04-25T00:36:22.756 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.756 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.757 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.757 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:36:22.757 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.757 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.757 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.757 INFO:teuthology.orchestra.run.smithi029.stdout: "stale_cache_strategy": { 2024-04-25T00:36:22.757 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "stale_cache_strategy", 2024-04-25T00:36:22.757 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.757 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.757 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.757 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "log", 2024-04-25T00:36:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_behaviour": { 2024-04-25T00:36:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_behaviour", 2024-04-25T00:36:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.758 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.759 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.759 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "default", 2024-04-25T00:36:22.759 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.759 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.759 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.759 INFO:teuthology.orchestra.run.smithi029.stdout: "default", 2024-04-25T00:36:22.759 INFO:teuthology.orchestra.run.smithi029.stdout: "error" 2024-04-25T00:36:22.759 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.759 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.759 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.759 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.759 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.760 INFO:teuthology.orchestra.run.smithi029.stdout: "standby_error_status_code": { 2024-04-25T00:36:22.760 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "standby_error_status_code", 2024-04-25T00:36:22.760 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.760 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.760 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.760 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "500", 2024-04-25T00:36:22.760 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "400", 2024-04-25T00:36:22.760 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "599", 2024-04-25T00:36:22.760 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.760 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.761 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.761 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.761 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.761 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.761 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.761 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_support", 2024-04-25T00:36:22.761 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.761 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.761 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.761 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.762 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.762 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.762 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.762 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.762 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.762 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.762 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.762 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.762 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.762 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.762 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.762 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.763 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.763 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.763 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.763 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.763 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.763 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.763 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.763 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.763 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.763 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.763 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.763 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.764 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.764 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.764 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.764 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.764 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.764 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.764 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.764 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.764 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.764 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.764 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.765 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.766 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.766 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.766 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.766 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.766 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.766 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.766 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.766 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.766 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.767 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.767 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.767 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.767 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.767 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.767 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.767 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.767 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.767 INFO:teuthology.orchestra.run.smithi029.stdout: "max_concurrent_snap_create": { 2024-04-25T00:36:22.768 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:36:22.768 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.768 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.768 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.768 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10", 2024-04-25T00:36:22.768 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.768 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.768 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.768 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.768 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.768 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.768 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.769 INFO:teuthology.orchestra.run.smithi029.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:36:22.769 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:36:22.769 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.769 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.769 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.769 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.769 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.769 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.769 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.769 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.769 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.770 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.770 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.770 INFO:teuthology.orchestra.run.smithi029.stdout: "trash_purge_schedule": { 2024-04-25T00:36:22.770 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "trash_purge_schedule", 2024-04-25T00:36:22.770 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.770 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.770 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.770 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.770 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.771 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.771 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.771 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.771 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.771 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.771 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.771 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.771 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.772 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "restful", 2024-04-25T00:36:22.772 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.772 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.772 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.772 INFO:teuthology.orchestra.run.smithi029.stdout: "enable_auth": { 2024-04-25T00:36:22.772 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enable_auth", 2024-04-25T00:36:22.772 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.772 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.772 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.772 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.772 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.773 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.773 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.773 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.773 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.773 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.773 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.773 INFO:teuthology.orchestra.run.smithi029.stdout: "key_file": { 2024-04-25T00:36:22.773 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "key_file", 2024-04-25T00:36:22.774 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.774 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.774 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.774 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.774 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.774 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.774 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.774 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.774 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.774 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.774 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.775 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.775 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.775 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.775 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.775 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.775 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.775 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.775 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.775 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.775 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.775 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.775 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.776 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.776 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.776 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.776 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.776 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.776 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.776 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.776 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.776 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.776 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.776 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.777 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.777 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.777 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.777 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.777 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.777 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.777 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.777 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.777 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.777 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.777 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.777 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.778 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.778 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.778 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.778 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.778 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.778 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.778 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.778 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.778 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.778 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.778 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.778 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.779 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.779 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.779 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.779 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.779 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.779 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.779 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.779 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.779 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.779 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.779 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.779 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.780 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.780 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.780 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.780 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.780 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.780 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.780 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.780 INFO:teuthology.orchestra.run.smithi029.stdout: "server_addr": { 2024-04-25T00:36:22.780 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_addr", 2024-04-25T00:36:22.780 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.780 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.780 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.781 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.781 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.781 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.781 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.781 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.781 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.781 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.781 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.781 INFO:teuthology.orchestra.run.smithi029.stdout: "server_port": { 2024-04-25T00:36:22.781 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "server_port", 2024-04-25T00:36:22.782 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.782 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.782 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.782 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.782 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.782 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.782 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.782 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.782 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.782 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.782 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.783 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.783 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.783 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "selftest", 2024-04-25T00:36:22.783 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.783 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.783 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.783 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.783 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.783 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.784 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.785 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.785 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.785 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.785 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.785 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.785 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.785 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.785 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.785 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.786 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.786 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.786 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.786 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.786 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.786 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.786 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.786 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.786 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.786 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.786 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.787 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.787 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.787 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.787 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.787 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.787 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.787 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.787 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.787 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.787 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.787 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.787 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.788 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.788 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.788 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.788 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.788 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.788 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.788 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.788 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.788 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.788 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.788 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.788 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.789 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.789 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.789 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.789 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.789 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.789 INFO:teuthology.orchestra.run.smithi029.stdout: "roption1": { 2024-04-25T00:36:22.789 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "roption1", 2024-04-25T00:36:22.789 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.789 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.789 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.789 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.789 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.790 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.790 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.790 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.790 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.790 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.790 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.790 INFO:teuthology.orchestra.run.smithi029.stdout: "roption2": { 2024-04-25T00:36:22.790 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "roption2", 2024-04-25T00:36:22.790 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.790 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.790 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.791 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "xyz", 2024-04-25T00:36:22.791 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.791 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.791 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.791 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.791 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.791 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.791 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.791 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption1": { 2024-04-25T00:36:22.791 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption1", 2024-04-25T00:36:22.791 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.792 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.792 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.792 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.792 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.792 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.792 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.792 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.792 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.792 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.792 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.792 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption2": { 2024-04-25T00:36:22.793 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption2", 2024-04-25T00:36:22.793 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.793 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.793 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.793 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.793 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.793 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.793 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.793 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.793 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.793 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.794 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.794 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption3": { 2024-04-25T00:36:22.794 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption3", 2024-04-25T00:36:22.794 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "float", 2024-04-25T00:36:22.794 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.794 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.794 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.794 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.794 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.795 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.795 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.795 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.795 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.795 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.795 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption4": { 2024-04-25T00:36:22.795 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption4", 2024-04-25T00:36:22.795 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.796 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.796 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.796 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.796 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.796 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.796 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.796 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.796 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.796 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.796 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.796 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption5": { 2024-04-25T00:36:22.796 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption5", 2024-04-25T00:36:22.797 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.797 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.797 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.797 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.797 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.797 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.797 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.797 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.797 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.797 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.797 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.798 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption6": { 2024-04-25T00:36:22.798 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption6", 2024-04-25T00:36:22.798 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.798 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.798 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.798 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.798 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.798 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.798 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.799 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.799 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.799 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.799 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.799 INFO:teuthology.orchestra.run.smithi029.stdout: "rwoption7": { 2024-04-25T00:36:22.799 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rwoption7", 2024-04-25T00:36:22.799 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.800 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.800 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.800 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.800 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "1", 2024-04-25T00:36:22.800 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "42", 2024-04-25T00:36:22.800 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.800 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.800 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.800 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.800 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.800 INFO:teuthology.orchestra.run.smithi029.stdout: "testkey": { 2024-04-25T00:36:22.800 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testkey", 2024-04-25T00:36:22.801 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.801 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.801 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.801 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.801 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.801 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.801 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.801 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.801 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.801 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.802 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.802 INFO:teuthology.orchestra.run.smithi029.stdout: "testlkey": { 2024-04-25T00:36:22.802 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testlkey", 2024-04-25T00:36:22.802 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.802 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.802 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.802 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.802 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.802 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.803 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.803 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.803 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.803 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.803 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.803 INFO:teuthology.orchestra.run.smithi029.stdout: "testnewline": { 2024-04-25T00:36:22.803 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "testnewline", 2024-04-25T00:36:22.803 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.803 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.803 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.803 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.804 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.804 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.804 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.804 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.804 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.804 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.804 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.804 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.804 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.805 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snap_schedule", 2024-04-25T00:36:22.805 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.805 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.805 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.805 INFO:teuthology.orchestra.run.smithi029.stdout: "allow_m_granularity": { 2024-04-25T00:36:22.805 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "allow_m_granularity", 2024-04-25T00:36:22.805 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.805 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.805 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.805 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.805 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.805 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.805 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.806 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:36:22.806 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.806 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.806 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.806 INFO:teuthology.orchestra.run.smithi029.stdout: "dump_on_update": { 2024-04-25T00:36:22.806 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "dump_on_update", 2024-04-25T00:36:22.806 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.806 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.806 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.806 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.807 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.808 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.808 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.808 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.808 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.808 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.808 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.808 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.808 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.808 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.808 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.808 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.808 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.808 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.809 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.809 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.809 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.809 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.809 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.809 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.809 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.809 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.809 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.809 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.809 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.810 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.810 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.810 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.810 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.810 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.810 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.810 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.810 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.810 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.810 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.810 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.811 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.811 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.811 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.811 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.811 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.811 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.811 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.811 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.811 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.811 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.811 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.811 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.812 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.812 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.812 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.812 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.812 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.812 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.812 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.812 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.812 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.812 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.812 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.813 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.813 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.813 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.813 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.813 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "stats", 2024-04-25T00:36:22.813 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.813 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.813 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.813 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.814 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.814 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.814 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.814 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.814 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.814 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.814 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.814 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.814 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.814 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.815 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.815 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.815 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.815 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.815 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.815 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.815 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.815 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.815 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.815 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.815 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.816 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.816 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.816 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.816 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.816 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.816 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.816 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.816 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.816 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.816 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.816 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.817 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.817 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.817 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.817 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.817 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.817 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.817 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.817 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.817 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.817 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.817 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.818 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.818 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.818 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.818 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.818 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.818 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.818 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.818 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.818 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.818 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.819 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.819 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.819 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.819 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.819 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.819 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.819 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.819 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.819 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.819 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.819 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.819 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.820 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.820 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.820 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "status", 2024-04-25T00:36:22.820 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.820 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.820 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.820 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.820 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.820 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.820 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.820 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.821 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.821 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.821 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.821 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.821 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.821 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.821 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.821 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.821 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.821 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.821 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.821 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.822 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.822 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.822 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.822 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.822 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.822 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.822 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.822 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.822 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.822 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.823 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.823 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.823 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.823 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.823 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.823 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.823 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.823 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.823 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.823 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.823 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.824 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.824 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.824 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.824 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.824 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.824 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.824 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.824 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.824 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.824 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.824 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.824 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.825 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.825 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.825 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.825 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.825 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.825 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.825 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.825 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.825 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.825 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.825 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.825 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.826 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.826 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.826 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.826 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.826 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.826 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.826 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "telegraf", 2024-04-25T00:36:22.826 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.826 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.826 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.827 INFO:teuthology.orchestra.run.smithi029.stdout: "address": { 2024-04-25T00:36:22.827 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "address", 2024-04-25T00:36:22.827 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.827 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.827 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.827 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:36:22.827 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.827 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.827 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.827 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.828 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.828 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.828 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.828 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-04-25T00:36:22.828 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-04-25T00:36:22.828 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.828 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.828 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "15", 2024-04-25T00:36:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.829 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.830 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.830 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.830 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.830 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.830 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.830 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.830 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.830 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.830 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.830 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.830 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.831 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.831 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.831 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.831 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.831 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.831 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.831 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.831 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.831 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.831 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.831 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.832 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.833 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.833 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.833 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.833 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.833 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.833 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.833 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.833 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.833 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.833 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.833 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.833 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.834 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.834 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.834 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.834 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.834 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.834 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.834 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.834 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.834 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.834 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.835 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.835 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.835 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.835 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.835 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.835 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.835 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.835 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.835 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.835 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.836 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "telemetry", 2024-04-25T00:36:22.836 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.836 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.836 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.836 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_basic": { 2024-04-25T00:36:22.836 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_basic", 2024-04-25T00:36:22.836 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.836 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.836 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.836 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.836 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.836 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.836 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.837 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:36:22.837 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.837 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.837 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.837 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_crash": { 2024-04-25T00:36:22.837 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_crash", 2024-04-25T00:36:22.837 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.837 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.837 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.837 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.837 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.838 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.838 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.838 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:36:22.838 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.838 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.838 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.838 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_device": { 2024-04-25T00:36:22.838 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_device", 2024-04-25T00:36:22.838 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.838 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.838 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.839 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.839 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.839 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.839 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.839 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:36:22.839 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.839 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.839 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.839 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_ident": { 2024-04-25T00:36:22.839 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_ident", 2024-04-25T00:36:22.839 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.839 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.840 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.840 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.840 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.840 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.840 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.840 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:36:22.840 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.840 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.840 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.840 INFO:teuthology.orchestra.run.smithi029.stdout: "channel_perf": { 2024-04-25T00:36:22.840 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "channel_perf", 2024-04-25T00:36:22.841 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.841 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.841 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.841 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.841 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.841 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.841 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.841 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:36:22.841 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.841 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.841 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.841 INFO:teuthology.orchestra.run.smithi029.stdout: "contact": { 2024-04-25T00:36:22.842 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "contact", 2024-04-25T00:36:22.842 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.842 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.842 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.842 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.842 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.842 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.842 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.842 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.842 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.842 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.842 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.843 INFO:teuthology.orchestra.run.smithi029.stdout: "description": { 2024-04-25T00:36:22.843 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "description", 2024-04-25T00:36:22.843 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.843 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.843 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.843 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.843 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.843 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.843 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.843 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.843 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.843 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.844 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.844 INFO:teuthology.orchestra.run.smithi029.stdout: "device_url": { 2024-04-25T00:36:22.844 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "device_url", 2024-04-25T00:36:22.844 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.844 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.844 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.844 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:36:22.844 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.844 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.844 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.845 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.845 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.845 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.845 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.845 INFO:teuthology.orchestra.run.smithi029.stdout: "enabled": { 2024-04-25T00:36:22.845 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "enabled", 2024-04-25T00:36:22.845 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.845 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.845 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.845 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.846 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.846 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.846 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.846 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.846 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.846 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.846 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.846 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-04-25T00:36:22.846 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-04-25T00:36:22.846 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.846 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.846 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.847 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "24", 2024-04-25T00:36:22.847 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "8", 2024-04-25T00:36:22.847 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.847 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.847 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.847 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.847 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.847 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.847 INFO:teuthology.orchestra.run.smithi029.stdout: "last_opt_revision": { 2024-04-25T00:36:22.847 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "last_opt_revision", 2024-04-25T00:36:22.847 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.848 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.848 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.848 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "1", 2024-04-25T00:36:22.848 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.848 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.848 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.848 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.848 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.848 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.848 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.848 INFO:teuthology.orchestra.run.smithi029.stdout: "leaderboard": { 2024-04-25T00:36:22.849 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "leaderboard", 2024-04-25T00:36:22.849 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.849 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.849 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.849 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.849 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.849 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.849 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.849 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.849 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.849 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.849 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.850 INFO:teuthology.orchestra.run.smithi029.stdout: "leaderboard_description": { 2024-04-25T00:36:22.850 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "leaderboard_description", 2024-04-25T00:36:22.850 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.850 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.850 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.850 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.850 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.850 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.850 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.850 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.850 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.851 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.851 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.851 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.851 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.851 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.851 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.851 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.851 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.851 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.851 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.851 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.852 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.852 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.852 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.852 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.852 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.852 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.852 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.852 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.852 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.852 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.852 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.852 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.853 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.853 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.853 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.853 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.853 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.853 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.853 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.853 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.853 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.853 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.853 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.853 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.854 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.854 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.854 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.854 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.854 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.854 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.854 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.854 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.854 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.854 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.854 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.855 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.855 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.855 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.855 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.855 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.855 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.855 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.855 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.855 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.855 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.855 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.855 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.856 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.856 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.856 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.856 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.856 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.856 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.856 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.856 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.856 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.856 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.856 INFO:teuthology.orchestra.run.smithi029.stdout: "organization": { 2024-04-25T00:36:22.856 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "organization", 2024-04-25T00:36:22.857 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.857 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.857 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.857 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.857 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.857 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.857 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.857 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.857 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.857 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.857 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.858 INFO:teuthology.orchestra.run.smithi029.stdout: "proxy": { 2024-04-25T00:36:22.858 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "proxy", 2024-04-25T00:36:22.858 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.858 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.858 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.858 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.858 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.858 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.858 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.858 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.858 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.859 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.859 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.859 INFO:teuthology.orchestra.run.smithi029.stdout: "url": { 2024-04-25T00:36:22.859 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "url", 2024-04-25T00:36:22.859 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.859 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.859 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.859 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:36:22.860 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.860 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.860 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.860 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.860 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.860 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.860 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.860 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.860 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.861 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "test_orchestrator", 2024-04-25T00:36:22.861 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.861 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.861 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.861 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.861 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.861 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.861 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.861 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.861 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.861 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.861 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.861 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.862 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.862 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.862 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.862 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.862 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.862 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.862 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.862 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.862 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.862 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.862 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.863 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.863 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.863 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.863 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.863 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.863 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.863 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.863 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.863 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.863 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.863 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.863 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.864 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.864 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.864 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.864 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.864 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.864 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.864 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.864 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.864 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.864 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.864 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.865 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.865 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.865 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.865 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.865 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.865 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.865 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.865 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.865 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.865 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.865 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.866 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.866 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.866 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.866 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.866 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.866 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.866 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.866 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.866 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.866 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.866 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.867 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.867 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.867 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.867 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "volumes", 2024-04-25T00:36:22.867 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.867 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.867 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.867 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.867 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.867 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.868 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.868 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.868 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.868 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.868 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.868 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.868 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.868 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.868 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.868 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.868 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.868 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.869 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.869 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.869 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.869 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.869 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.869 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.869 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.869 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.869 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.869 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.869 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.870 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.870 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.870 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.870 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.870 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.870 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.870 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.870 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.870 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.870 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.871 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.871 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.871 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.871 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.871 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.871 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.871 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.871 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.871 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.871 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.871 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.871 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.872 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.872 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.872 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.872 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.872 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.872 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.872 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.872 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.872 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.872 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.872 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.872 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.873 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.873 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.873 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.873 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.873 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.873 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.873 INFO:teuthology.orchestra.run.smithi029.stdout: "max_concurrent_clones": { 2024-04-25T00:36:22.873 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "max_concurrent_clones", 2024-04-25T00:36:22.873 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.873 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.873 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.873 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "4", 2024-04-25T00:36:22.874 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.874 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.874 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.874 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:36:22.874 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.874 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.874 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.874 INFO:teuthology.orchestra.run.smithi029.stdout: "periodic_async_work": { 2024-04-25T00:36:22.874 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "periodic_async_work", 2024-04-25T00:36:22.874 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.875 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.875 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.875 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.875 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.875 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.875 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.875 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Periodically check for async work", 2024-04-25T00:36:22.875 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.875 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.875 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.875 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot_clone_delay": { 2024-04-25T00:36:22.875 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:36:22.876 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.876 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.876 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.876 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "0", 2024-04-25T00:36:22.876 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.876 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.876 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.876 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:36:22.876 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.876 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.876 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.877 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:36:22.877 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:36:22.877 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.877 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.877 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.877 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "True", 2024-04-25T00:36:22.877 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.877 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.877 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.877 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:36:22.877 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.878 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.878 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.878 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.878 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.878 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix", 2024-04-25T00:36:22.878 INFO:teuthology.orchestra.run.smithi029.stdout: "can_run": true, 2024-04-25T00:36:22.878 INFO:teuthology.orchestra.run.smithi029.stdout: "error_string": "", 2024-04-25T00:36:22.878 INFO:teuthology.orchestra.run.smithi029.stdout: "module_options": { 2024-04-25T00:36:22.878 INFO:teuthology.orchestra.run.smithi029.stdout: "discovery_interval": { 2024-04-25T00:36:22.878 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "discovery_interval", 2024-04-25T00:36:22.879 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "uint", 2024-04-25T00:36:22.879 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.879 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.879 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "100", 2024-04-25T00:36:22.879 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.879 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.879 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.879 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.879 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.879 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.879 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.880 INFO:teuthology.orchestra.run.smithi029.stdout: "identifier": { 2024-04-25T00:36:22.880 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "identifier", 2024-04-25T00:36:22.880 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.880 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.880 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.880 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.880 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.880 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.880 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.880 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.880 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.880 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.881 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.881 INFO:teuthology.orchestra.run.smithi029.stdout: "interval": { 2024-04-25T00:36:22.881 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "interval", 2024-04-25T00:36:22.881 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "secs", 2024-04-25T00:36:22.881 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.881 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.881 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "60", 2024-04-25T00:36:22.881 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.881 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.881 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.881 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.881 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.882 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.882 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.882 INFO:teuthology.orchestra.run.smithi029.stdout: "log_level": { 2024-04-25T00:36:22.882 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_level", 2024-04-25T00:36:22.882 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.882 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.882 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.882 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.882 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.882 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.882 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.883 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.883 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.883 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.883 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.883 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.883 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.883 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.883 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.883 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.883 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.883 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.883 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster": { 2024-04-25T00:36:22.885 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster", 2024-04-25T00:36:22.886 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.886 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.886 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.886 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.886 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.886 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.886 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.886 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.886 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.886 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.886 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.887 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_cluster_level": { 2024-04-25T00:36:22.887 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_cluster_level", 2024-04-25T00:36:22.887 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.887 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.887 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.887 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "info", 2024-04-25T00:36:22.887 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.887 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.888 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [ 2024-04-25T00:36:22.888 INFO:teuthology.orchestra.run.smithi029.stdout: "", 2024-04-25T00:36:22.888 INFO:teuthology.orchestra.run.smithi029.stdout: "critical", 2024-04-25T00:36:22.888 INFO:teuthology.orchestra.run.smithi029.stdout: "debug", 2024-04-25T00:36:22.888 INFO:teuthology.orchestra.run.smithi029.stdout: "error", 2024-04-25T00:36:22.888 INFO:teuthology.orchestra.run.smithi029.stdout: "info", 2024-04-25T00:36:22.888 INFO:teuthology.orchestra.run.smithi029.stdout: "warning" 2024-04-25T00:36:22.888 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.888 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.888 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.889 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.889 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.889 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": { 2024-04-25T00:36:22.889 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "log_to_file", 2024-04-25T00:36:22.889 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "bool", 2024-04-25T00:36:22.889 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.889 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-04-25T00:36:22.889 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "False", 2024-04-25T00:36:22.889 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.889 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.889 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.889 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.890 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.890 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.890 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.890 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_host": { 2024-04-25T00:36:22.890 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_host", 2024-04-25T00:36:22.890 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.890 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.890 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.890 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "", 2024-04-25T00:36:22.890 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.890 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.891 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.891 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.891 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.891 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.891 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.891 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_port": { 2024-04-25T00:36:22.891 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_port", 2024-04-25T00:36:22.891 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "int", 2024-04-25T00:36:22.891 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.891 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.891 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "10051", 2024-04-25T00:36:22.892 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.892 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.892 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.892 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.892 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.892 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.892 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.892 INFO:teuthology.orchestra.run.smithi029.stdout: "zabbix_sender": { 2024-04-25T00:36:22.892 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "zabbix_sender", 2024-04-25T00:36:22.893 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "str", 2024-04-25T00:36:22.893 INFO:teuthology.orchestra.run.smithi029.stdout: "level": "advanced", 2024-04-25T00:36:22.893 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 0, 2024-04-25T00:36:22.893 INFO:teuthology.orchestra.run.smithi029.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:36:22.893 INFO:teuthology.orchestra.run.smithi029.stdout: "min": "", 2024-04-25T00:36:22.893 INFO:teuthology.orchestra.run.smithi029.stdout: "max": "", 2024-04-25T00:36:22.893 INFO:teuthology.orchestra.run.smithi029.stdout: "enum_allowed": [], 2024-04-25T00:36:22.893 INFO:teuthology.orchestra.run.smithi029.stdout: "desc": "", 2024-04-25T00:36:22.893 INFO:teuthology.orchestra.run.smithi029.stdout: "long_desc": "", 2024-04-25T00:36:22.893 INFO:teuthology.orchestra.run.smithi029.stdout: "tags": [], 2024-04-25T00:36:22.893 INFO:teuthology.orchestra.run.smithi029.stdout: "see_also": [] 2024-04-25T00:36:22.893 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.894 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.894 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.894 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.894 INFO:teuthology.orchestra.run.smithi029.stdout: "services": {}, 2024-04-25T00:36:22.894 INFO:teuthology.orchestra.run.smithi029.stdout: "always_on_modules": { 2024-04-25T00:36:22.894 INFO:teuthology.orchestra.run.smithi029.stdout: "octopus": [ 2024-04-25T00:36:22.894 INFO:teuthology.orchestra.run.smithi029.stdout: "balancer", 2024-04-25T00:36:22.894 INFO:teuthology.orchestra.run.smithi029.stdout: "crash", 2024-04-25T00:36:22.894 INFO:teuthology.orchestra.run.smithi029.stdout: "devicehealth", 2024-04-25T00:36:22.894 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator", 2024-04-25T00:36:22.894 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscaler", 2024-04-25T00:36:22.894 INFO:teuthology.orchestra.run.smithi029.stdout: "progress", 2024-04-25T00:36:22.895 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_support", 2024-04-25T00:36:22.895 INFO:teuthology.orchestra.run.smithi029.stdout: "status", 2024-04-25T00:36:22.895 INFO:teuthology.orchestra.run.smithi029.stdout: "telemetry", 2024-04-25T00:36:22.895 INFO:teuthology.orchestra.run.smithi029.stdout: "volumes" 2024-04-25T00:36:22.895 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.895 INFO:teuthology.orchestra.run.smithi029.stdout: "pacific": [ 2024-04-25T00:36:22.895 INFO:teuthology.orchestra.run.smithi029.stdout: "balancer", 2024-04-25T00:36:22.895 INFO:teuthology.orchestra.run.smithi029.stdout: "crash", 2024-04-25T00:36:22.895 INFO:teuthology.orchestra.run.smithi029.stdout: "devicehealth", 2024-04-25T00:36:22.895 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator", 2024-04-25T00:36:22.895 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscaler", 2024-04-25T00:36:22.895 INFO:teuthology.orchestra.run.smithi029.stdout: "progress", 2024-04-25T00:36:22.896 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_support", 2024-04-25T00:36:22.896 INFO:teuthology.orchestra.run.smithi029.stdout: "status", 2024-04-25T00:36:22.896 INFO:teuthology.orchestra.run.smithi029.stdout: "telemetry", 2024-04-25T00:36:22.896 INFO:teuthology.orchestra.run.smithi029.stdout: "volumes" 2024-04-25T00:36:22.896 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.896 INFO:teuthology.orchestra.run.smithi029.stdout: "quincy": [ 2024-04-25T00:36:22.896 INFO:teuthology.orchestra.run.smithi029.stdout: "balancer", 2024-04-25T00:36:22.896 INFO:teuthology.orchestra.run.smithi029.stdout: "crash", 2024-04-25T00:36:22.896 INFO:teuthology.orchestra.run.smithi029.stdout: "devicehealth", 2024-04-25T00:36:22.896 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator", 2024-04-25T00:36:22.896 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscaler", 2024-04-25T00:36:22.896 INFO:teuthology.orchestra.run.smithi029.stdout: "progress", 2024-04-25T00:36:22.897 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_support", 2024-04-25T00:36:22.897 INFO:teuthology.orchestra.run.smithi029.stdout: "status", 2024-04-25T00:36:22.897 INFO:teuthology.orchestra.run.smithi029.stdout: "telemetry", 2024-04-25T00:36:22.897 INFO:teuthology.orchestra.run.smithi029.stdout: "volumes" 2024-04-25T00:36:22.897 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.897 INFO:teuthology.orchestra.run.smithi029.stdout: "reef": [ 2024-04-25T00:36:22.897 INFO:teuthology.orchestra.run.smithi029.stdout: "balancer", 2024-04-25T00:36:22.897 INFO:teuthology.orchestra.run.smithi029.stdout: "crash", 2024-04-25T00:36:22.897 INFO:teuthology.orchestra.run.smithi029.stdout: "devicehealth", 2024-04-25T00:36:22.897 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator", 2024-04-25T00:36:22.897 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscaler", 2024-04-25T00:36:22.897 INFO:teuthology.orchestra.run.smithi029.stdout: "progress", 2024-04-25T00:36:22.898 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_support", 2024-04-25T00:36:22.898 INFO:teuthology.orchestra.run.smithi029.stdout: "status", 2024-04-25T00:36:22.898 INFO:teuthology.orchestra.run.smithi029.stdout: "telemetry", 2024-04-25T00:36:22.898 INFO:teuthology.orchestra.run.smithi029.stdout: "volumes" 2024-04-25T00:36:22.898 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:36:22.898 INFO:teuthology.orchestra.run.smithi029.stdout: "squid": [ 2024-04-25T00:36:22.898 INFO:teuthology.orchestra.run.smithi029.stdout: "balancer", 2024-04-25T00:36:22.898 INFO:teuthology.orchestra.run.smithi029.stdout: "crash", 2024-04-25T00:36:22.898 INFO:teuthology.orchestra.run.smithi029.stdout: "devicehealth", 2024-04-25T00:36:22.898 INFO:teuthology.orchestra.run.smithi029.stdout: "orchestrator", 2024-04-25T00:36:22.898 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscaler", 2024-04-25T00:36:22.898 INFO:teuthology.orchestra.run.smithi029.stdout: "progress", 2024-04-25T00:36:22.899 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_support", 2024-04-25T00:36:22.899 INFO:teuthology.orchestra.run.smithi029.stdout: "status", 2024-04-25T00:36:22.899 INFO:teuthology.orchestra.run.smithi029.stdout: "telemetry", 2024-04-25T00:36:22.899 INFO:teuthology.orchestra.run.smithi029.stdout: "volumes" 2024-04-25T00:36:22.899 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:22.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.899 INFO:teuthology.orchestra.run.smithi029.stdout: "last_failure_osd_epoch": 0, 2024-04-25T00:36:22.899 INFO:teuthology.orchestra.run.smithi029.stdout: "active_clients": [ 2024-04-25T00:36:22.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.899 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "devicehealth", 2024-04-25T00:36:22.899 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-04-25T00:36:22.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.900 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-04-25T00:36:22.900 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.157:0", 2024-04-25T00:36:22.900 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1408011519 2024-04-25T00:36:22.900 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.900 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:22.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.900 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "libcephsqlite", 2024-04-25T00:36:22.900 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-04-25T00:36:22.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.900 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-04-25T00:36:22.900 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.157:0", 2024-04-25T00:36:22.900 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2500505584 2024-04-25T00:36:22.901 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.901 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:22.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.901 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "rbd_support", 2024-04-25T00:36:22.901 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-04-25T00:36:22.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.901 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-04-25T00:36:22.901 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.157:0", 2024-04-25T00:36:22.901 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3796173968 2024-04-25T00:36:22.901 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.901 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:22.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.902 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "volumes", 2024-04-25T00:36:22.902 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-04-25T00:36:22.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.902 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-04-25T00:36:22.902 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.157:0", 2024-04-25T00:36:22.902 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3565775874 2024-04-25T00:36:22.902 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.902 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:22.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.902 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "volumes", 2024-04-25T00:36:22.903 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-04-25T00:36:22.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.903 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-04-25T00:36:22.903 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.157:0", 2024-04-25T00:36:22.903 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1922269872 2024-04-25T00:36:22.903 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.903 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:22.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.903 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "volumes", 2024-04-25T00:36:22.903 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-04-25T00:36:22.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.904 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-04-25T00:36:22.904 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.157:0", 2024-04-25T00:36:22.904 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1065963518 2024-04-25T00:36:22.904 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.904 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:22.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.904 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "volumes", 2024-04-25T00:36:22.904 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-04-25T00:36:22.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.904 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-04-25T00:36:22.904 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.157:0", 2024-04-25T00:36:22.904 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 37741218 2024-04-25T00:36:22.905 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.905 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:22.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.905 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "volumes", 2024-04-25T00:36:22.905 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-04-25T00:36:22.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.905 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-04-25T00:36:22.905 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.157:0", 2024-04-25T00:36:22.905 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 4070029771 2024-04-25T00:36:22.905 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.905 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:22.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.906 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "volumes", 2024-04-25T00:36:22.906 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-04-25T00:36:22.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.906 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-04-25T00:36:22.906 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.157:0", 2024-04-25T00:36:22.906 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3438031706 2024-04-25T00:36:22.906 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.906 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:22.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:22.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.906 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "volumes", 2024-04-25T00:36:22.907 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-04-25T00:36:22.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:22.907 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-04-25T00:36:22.907 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.157:0", 2024-04-25T00:36:22.907 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3212538336 2024-04-25T00:36:22.907 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.907 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:22.907 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:22.907 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:22.907 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:36:22.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.936+0000 7f28734a7640 1 -- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2850067c20 msgr2=0x7f285006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:22.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.936+0000 7f28734a7640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2850067c20 0x7f285006a0e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f285c009a30 tx=0x7f285c008040 comp rx=0 tx=0).stop 2024-04-25T00:36:22.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.936+0000 7f28734a7640 1 -- 172.21.15.29:0/2548935273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286c072380 msgr2=0x7f286c13d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:22.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.936+0000 7f28734a7640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286c072380 0x7f286c13d5e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f2864002060 tx=0x7f2864011c20 comp rx=0 tx=0).stop 2024-04-25T00:36:22.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.936+0000 7f28734a7640 1 -- 172.21.15.29:0/2548935273 shutdown_connections 2024-04-25T00:36:22.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.936+0000 7f28734a7640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f286c072380 0x7f286c13d5e0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:22.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.936+0000 7f28734a7640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f2850067c20 0x7f285006a0e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:22.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.936+0000 7f28734a7640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286c071f20 0x7f286c13d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:22.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.936+0000 7f28734a7640 1 --2- 172.21.15.29:0/2548935273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2858003f70 0x7f286c1f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:22.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.936+0000 7f28734a7640 1 -- 172.21.15.29:0/2548935273 >> 172.21.15.29:0/2548935273 conn(0x7f286c0b1870 msgr2=0x7f286c0ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:22.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.936+0000 7f28734a7640 1 -- 172.21.15.29:0/2548935273 shutdown_connections 2024-04-25T00:36:22.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:21.936+0000 7f28734a7640 1 -- 172.21.15.29:0/2548935273 wait complete. 2024-04-25T00:36:22.914 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail x 2024-04-25T00:36:23.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.048+0000 7f3305203640 1 Processor -- start 2024-04-25T00:36:23.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.048+0000 7f3305203640 1 -- start start 2024-04-25T00:36:23.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.048+0000 7f3305203640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33000b4780 0x7f330015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:23.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.048+0000 7f3305203640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33000cc240 0x7f330015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:23.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.048+0000 7f3305203640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33000b3720 con 0x7f330015e8d0 2024-04-25T00:36:23.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.048+0000 7f3305203640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f33000b42f0 con 0x7f33000b4780 2024-04-25T00:36:23.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.048+0000 7f3305203640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f33000b39c0 con 0x7f33000cc240 2024-04-25T00:36:23.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f32fe59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33000b4780 0x7f330015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:23.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f32fe59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33000b4780 0x7f330015f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48586/0 (socket says 172.21.15.29:48586) 2024-04-25T00:36:23.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f32fe59c640 1 -- 172.21.15.29:0/1004612541 learned_addr learned my addr 172.21.15.29:0/1004612541 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:23.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f32ff59e640 1 --2- 172.21.15.29:0/1004612541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33000cc240 0x7f330015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:23.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f32fe59c640 1 -- 172.21.15.29:0/1004612541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33000cc240 msgr2=0x7f330015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:23.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f32fe59c640 1 --2- 172.21.15.29:0/1004612541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33000cc240 0x7f330015c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f32fe59c640 1 -- 172.21.15.29:0/1004612541 >> v1:172.21.15.157:6789/0 conn(0x7f330015e8d0 legacy=0x7f330015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:23.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f32fe59c640 1 -- 172.21.15.29:0/1004612541 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33000b3ca0 con 0x7f33000b4780 2024-04-25T00:36:23.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f32fe59c640 1 --2- 172.21.15.29:0/1004612541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33000b4780 0x7f330015f3c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f33000b4470 tx=0x7f32f401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd6f0e849695d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:23.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f32fdd9b640 1 -- 172.21.15.29:0/1004612541 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40b4070 con 0x7f33000b4780 2024-04-25T00:36:23.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f32fdd9b640 1 -- 172.21.15.29:0/1004612541 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f40a7bb0 con 0x7f33000b4780 2024-04-25T00:36:23.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f32fdd9b640 1 -- 172.21.15.29:0/1004612541 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40af440 con 0x7f33000b4780 2024-04-25T00:36:23.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f3305203640 1 -- 172.21.15.29:0/1004612541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33000b4780 msgr2=0x7f330015f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:23.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f3305203640 1 --2- 172.21.15.29:0/1004612541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33000b4780 0x7f330015f3c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f33000b4470 tx=0x7f32f401ec10 comp rx=0 tx=0).stop 2024-04-25T00:36:23.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f3305203640 1 -- 172.21.15.29:0/1004612541 shutdown_connections 2024-04-25T00:36:23.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f3305203640 1 --2- 172.21.15.29:0/1004612541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33000cc240 0x7f330015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f3305203640 1 --2- 172.21.15.29:0/1004612541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33000b4780 0x7f330015f3c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f3305203640 1 -- 172.21.15.29:0/1004612541 >> 172.21.15.29:0/1004612541 conn(0x7f33000b1870 msgr2=0x7f33000b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:23.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.052+0000 7f3305203640 1 -- 172.21.15.29:0/1004612541 shutdown_connections 2024-04-25T00:36:23.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f3305203640 1 -- 172.21.15.29:0/1004612541 wait complete. 2024-04-25T00:36:23.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f3305203640 1 Processor -- start 2024-04-25T00:36:23.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f3305203640 1 -- start start 2024-04-25T00:36:23.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f3305203640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33000b4780 0x7f33000ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:23.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f3305203640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33000cc240 0x7f33001595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:23.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f3305203640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f330015e8d0 0x7f3300139270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:23.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f3305203640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f330015d6a0 con 0x7f33000cc240 2024-04-25T00:36:23.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f3305203640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f330015d520 con 0x7f330015e8d0 2024-04-25T00:36:23.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f3305203640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f330006d5d0 con 0x7f33000b4780 2024-04-25T00:36:23.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32fe59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33000cc240 0x7f33001595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:23.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32fe59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33000cc240 0x7f33001595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48590/0 (socket says 172.21.15.29:48590) 2024-04-25T00:36:23.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32fe59c640 1 -- 172.21.15.29:0/4129671466 learned_addr learned my addr 172.21.15.29:0/4129671466 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:23.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32fed9d640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33000b4780 0x7f33000ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32ff59e640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f330015e8d0 0x7f3300139270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32fe59c640 1 -- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33000b4780 msgr2=0x7f33000ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32fe59c640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33000b4780 0x7f33000ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32fe59c640 1 -- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f330015e8d0 msgr2=0x7f3300139270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32fe59c640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f330015e8d0 0x7f3300139270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32fe59c640 1 -- 172.21.15.29:0/4129671466 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3300159e90 con 0x7f33000cc240 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32ff59e640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f330015e8d0 0x7f3300139270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32fe59c640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33000cc240 0x7f33001595f0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f32f401c840 tx=0x7f32f401b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32e77fe640 1 -- 172.21.15.29:0/4129671466 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40b4040 con 0x7f33000cc240 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32e77fe640 1 -- 172.21.15.29:0/4129671466 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f40bf070 con 0x7f33000cc240 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f3305203640 1 -- 172.21.15.29:0/4129671466 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330015a170 con 0x7f33000cc240 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f32e77fe640 1 -- 172.21.15.29:0/4129671466 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40ba680 con 0x7f33000cc240 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f3305203640 1 -- 172.21.15.29:0/4129671466 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f330015a730 con 0x7f33000cc240 2024-04-25T00:36:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.056+0000 7f3305203640 1 -- 172.21.15.29:0/4129671466 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3300069270 con 0x7f33000cc240 2024-04-25T00:36:23.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.060+0000 7f32e77fe640 1 -- 172.21.15.29:0/4129671466 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 12) ==== 87802+0+0 (secure 0 0 0) 0x7f32f40ba820 con 0x7f33000cc240 2024-04-25T00:36:23.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.060+0000 7f32e77fe640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f32ec067cb0 0x7f32ec06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:23.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.060+0000 7f32e77fe640 1 -- 172.21.15.29:0/4129671466 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f32f4123a40 con 0x7f33000cc240 2024-04-25T00:36:23.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.060+0000 7f32fed9d640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f32ec067cb0 0x7f32ec06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:23.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.060+0000 7f32fed9d640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f32ec067cb0 0x7f32ec06a170 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f32f00015b0 tx=0x7f32f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:23.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.064+0000 7f32e77fe640 1 -- 172.21.15.29:0/4129671466 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f32f4018800 con 0x7f33000cc240 2024-04-25T00:36:23.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.256+0000 7f3305203640 1 -- 172.21.15.29:0/4129671466 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr fail", "who": "x"} v 0) -- 0x7f330006e7b0 con 0x7f33000cc240 2024-04-25T00:36:23.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.264+0000 7f32e77fe640 1 -- 172.21.15.29:0/4129671466 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "x"}]=0 v13) ==== 68+0+0 (secure 0 0 0) 0x7f32f40f3f60 con 0x7f33000cc240 2024-04-25T00:36:23.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.264+0000 7f3305203640 1 -- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f32ec067cb0 msgr2=0x7f32ec06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:23.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.264+0000 7f3305203640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f32ec067cb0 0x7f32ec06a170 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f32f00015b0 tx=0x7f32f0009040 comp rx=0 tx=0).stop 2024-04-25T00:36:23.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.264+0000 7f3305203640 1 -- 172.21.15.29:0/4129671466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33000cc240 msgr2=0x7f33001595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:23.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.268+0000 7f32e77fe640 1 -- 172.21.15.29:0/4129671466 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mgrmap(e 13) ==== 43764+0+0 (secure 0 0 0) 0x7f32f40efed0 con 0x7f33000cc240 2024-04-25T00:36:23.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.268+0000 7f3305203640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33000cc240 0x7f33001595f0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f32f401c840 tx=0x7f32f401b9b0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.268+0000 7f3305203640 1 -- 172.21.15.29:0/4129671466 shutdown_connections 2024-04-25T00:36:23.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.268+0000 7f3305203640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:6832/3474584311,v1:172.21.15.157:6833/3474584311] conn(0x7f32ec067cb0 0x7f32ec06a170 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.268+0000 7f3305203640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f330015e8d0 0x7f3300139270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.268+0000 7f3305203640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33000cc240 0x7f33001595f0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.268+0000 7f3305203640 1 --2- 172.21.15.29:0/4129671466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33000b4780 0x7f33000ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.268+0000 7f3305203640 1 -- 172.21.15.29:0/4129671466 >> 172.21.15.29:0/4129671466 conn(0x7f33000b1870 msgr2=0x7f330013c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:23.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.268+0000 7f3305203640 1 -- 172.21.15.29:0/4129671466 shutdown_connections 2024-04-25T00:36:23.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.268+0000 7f3305203640 1 -- 172.21.15.29:0/4129671466 wait complete. 2024-04-25T00:36:23.304 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:36:23.381 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:36:23.380+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:23.381 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:36:23.380+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:23.382 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:36:23.380+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:23.382 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:36:23.380+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:23.382 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:36:23.380+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:23.383 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:36:23.380+0000 7f987d589640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:23.383 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:36:23.380+0000 7f987d589640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:23.383 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:36:23.380+0000 7f987d589640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:23.383 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:36:23.380+0000 7f987d589640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:23.383 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:36:23.380+0000 7f987d589640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:36:23.430 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:23.428+0000 7fa1bebc0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:36:23.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4da4d81640 1 Processor -- start 2024-04-25T00:36:23.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4da4d81640 1 -- start start 2024-04-25T00:36:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4da4d81640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4da00c82d0 0x7f4da00c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4da4d81640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4da00c87e0 con 0x7f4da015e290 2024-04-25T00:36:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4da4d81640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4da00cfa40 con 0x7f4da006cf60 2024-04-25T00:36:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4da4d81640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4da015e670 con 0x7f4da00c82d0 2024-04-25T00:36:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9e59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4da015e290 0x7f4da006c850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55086/0 (socket says 172.21.15.29:55086) 2024-04-25T00:36:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9e59c640 1 -- 172.21.15.29:0/2932395764 learned_addr learned my addr 172.21.15.29:0/2932395764 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9ed9d640 1 --2- 172.21.15.29:0/2932395764 >> v2:172.21.15.157:3300/0 conn(0x7f4da00c82d0 0x7f4da00c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2696513774 0 0) 0x7f4da00c87e0 con 0x7f4da015e290 2024-04-25T00:36:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d88003530 con 0x7f4da015e290 2024-04-25T00:36:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4278792109 0 0) 0x7f4d88003530 con 0x7f4da015e290 2024-04-25T00:36:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 >> v2:172.21.15.157:3300/0 conn(0x7f4da00c82d0 msgr2=0x7f4da00c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 --2- 172.21.15.29:0/2932395764 >> v2:172.21.15.157:3300/0 conn(0x7f4da00c82d0 0x7f4da00c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 >> v1:172.21.15.157:6789/0 conn(0x7f4da006cf60 legacy=0x7f4da00d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da00cfd20 con 0x7f4da015e290 2024-04-25T00:36:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4d90001680 con 0x7f4da015e290 2024-04-25T00:36:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 >> v1:172.21.15.29:6789/0 conn(0x7f4da015e290 legacy=0x7f4da006c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 --2- 172.21.15.29:0/2932395764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d880039c0 0x7f4d88003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 --2- 172.21.15.29:0/2932395764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d88004810 0x7f4d88004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 --2- 172.21.15.29:0/2932395764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d880051d0 0x7f4d8800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4da015e670 con 0x7f4d880051d0 2024-04-25T00:36:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8800c170 con 0x7f4d88004810 2024-04-25T00:36:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8800c370 con 0x7f4d880039c0 2024-04-25T00:36:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9ed9d640 1 --2- 172.21.15.29:0/2932395764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d880051d0 0x7f4d8800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9dd9b640 1 --2- 172.21.15.29:0/2932395764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d88004810 0x7f4d88004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9e59c640 1 --2- 172.21.15.29:0/2932395764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d880039c0 0x7f4d88003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9ed9d640 1 -- 172.21.15.29:0/2932395764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d880039c0 msgr2=0x7f4d88003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9ed9d640 1 --2- 172.21.15.29:0/2932395764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d880039c0 0x7f4d88003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9ed9d640 1 -- 172.21.15.29:0/2932395764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d88004810 msgr2=0x7f4d88004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:23.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9ed9d640 1 --2- 172.21.15.29:0/2932395764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d88004810 0x7f4d88004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.432+0000 7f4d9ed9d640 1 -- 172.21.15.29:0/2932395764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d88003530 con 0x7f4d880051d0 2024-04-25T00:36:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9ed9d640 1 -- 172.21.15.29:0/2932395764 reap_dead start 2024-04-25T00:36:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9ed9d640 1 --2- 172.21.15.29:0/2932395764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d880051d0 0x7f4d8800b9d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009f90 tx=0x7f4d9801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e1bfc87667d44a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d980b2070 con 0x7f4d880051d0 2024-04-25T00:36:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d980a5b80 con 0x7f4d880051d0 2024-04-25T00:36:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9d59a640 1 -- 172.21.15.29:0/2932395764 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98009c10 con 0x7f4d880051d0 2024-04-25T00:36:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 -- 172.21.15.29:0/2932395764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d880051d0 msgr2=0x7f4d8800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 --2- 172.21.15.29:0/2932395764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d880051d0 0x7f4d8800b9d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009f90 tx=0x7f4d9801cc80 comp rx=0 tx=0).stop 2024-04-25T00:36:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 -- 172.21.15.29:0/2932395764 shutdown_connections 2024-04-25T00:36:23.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 --2- 172.21.15.29:0/2932395764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d880051d0 0x7f4d8800b9d0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 -- 172.21.15.29:0/2932395764 >> 172.21.15.29:0/2932395764 conn(0x7f4da00b3a00 msgr2=0x7f4da00b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 -- 172.21.15.29:0/2932395764 shutdown_connections 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 -- 172.21.15.29:0/2932395764 wait complete. 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 Processor -- start 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 -- start start 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d880051d0 0x7f4da009c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da0157db0 0x7f4da014d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da014dda0 0x7f4da015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4da006d4a0 con 0x7f4d880051d0 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4da00c8ac0 con 0x7f4da014dda0 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4da006d620 con 0x7f4da0157db0 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d880051d0 0x7f4da009c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d880051d0 0x7f4da009c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48740/0 (socket says 172.21.15.29:48740) 2024-04-25T00:36:23.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9e59c640 1 -- 172.21.15.29:0/1035293519 learned_addr learned my addr 172.21.15.29:0/1035293519 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:23.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9dd9b640 1 --2- 172.21.15.29:0/1035293519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da0157db0 0x7f4da014d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:23.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9e59c640 1 -- 172.21.15.29:0/1035293519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da0157db0 msgr2=0x7f4da014d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:23.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9ed9d640 1 --2- 172.21.15.29:0/1035293519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da014dda0 0x7f4da015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:23.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9e59c640 1 --2- 172.21.15.29:0/1035293519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da0157db0 0x7f4da014d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9e59c640 1 -- 172.21.15.29:0/1035293519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da014dda0 msgr2=0x7f4da015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:23.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9e59c640 1 --2- 172.21.15.29:0/1035293519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da014dda0 0x7f4da015f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9e59c640 1 -- 172.21.15.29:0/1035293519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da00b66f0 con 0x7f4d880051d0 2024-04-25T00:36:23.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9ed9d640 1 --2- 172.21.15.29:0/1035293519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da014dda0 0x7f4da015f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:23.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d9e59c640 1 --2- 172.21.15.29:0/1035293519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d880051d0 0x7f4da009c050 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f4d90011810 tx=0x7f4d90011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:23.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d8effd640 1 -- 172.21.15.29:0/1035293519 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d90010040 con 0x7f4d880051d0 2024-04-25T00:36:23.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d8effd640 1 -- 172.21.15.29:0/1035293519 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d9001a070 con 0x7f4d880051d0 2024-04-25T00:36:23.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4da4d81640 1 -- 172.21.15.29:0/1035293519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da00b69d0 con 0x7f4d880051d0 2024-04-25T00:36:23.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.436+0000 7f4d8effd640 1 -- 172.21.15.29:0/1035293519 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9000fde0 con 0x7f4d880051d0 2024-04-25T00:36:23.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.440+0000 7f4da4d81640 1 -- 172.21.15.29:0/1035293519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da00b6f90 con 0x7f4d880051d0 2024-04-25T00:36:23.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.440+0000 7f4d8effd640 1 -- 172.21.15.29:0/1035293519 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 13) ==== 43764+0+0 (secure 0 0 0) 0x7f4d9001f460 con 0x7f4d880051d0 2024-04-25T00:36:23.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.440+0000 7f4da4d81640 1 -- 172.21.15.29:0/1035293519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da0069270 con 0x7f4d880051d0 2024-04-25T00:36:23.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.440+0000 7f4d8effd640 1 -- 172.21.15.29:0/1035293519 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6326+0+0 (secure 0 0 0) 0x7f4d9004bf60 con 0x7f4d880051d0 2024-04-25T00:36:23.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.444+0000 7f4d8effd640 1 -- 172.21.15.29:0/1035293519 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4d90014530 con 0x7f4d880051d0 2024-04-25T00:36:23.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.632+0000 7f4da4d81640 1 -- 172.21.15.29:0/1035293519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4da00b7fe0 con 0x7f4d880051d0 2024-04-25T00:36:23.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.632+0000 7f4d8effd640 1 -- 172.21.15.29:0/1035293519 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7f4d90014790 con 0x7f4d880051d0 2024-04-25T00:36:23.636 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:23.636 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/524224365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":524224365},{"type":"v1","addr":"172.21.15.157:6835","nonce":524224365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1007000548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1007000548},{"type":"v1","addr":"172.21.15.157:6837","nonce":1007000548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6810,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2091414875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2091414875},{"type":"v1","addr":"172.21.15.29:6833","nonce":2091414875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:49.528297+0000","modified":"2024-04-25T00:35:52.569008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1111146974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1111146974},{"type":"v1","addr":"172.21.15.29:6835","nonce":1111146974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6822,"qdb_cluster":[6822]},"id":5}]} 2024-04-25T00:36:23.637 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 44 2024-04-25T00:36:23.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.636+0000 7f4da4d81640 1 -- 172.21.15.29:0/1035293519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d880051d0 msgr2=0x7f4da009c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:23.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.636+0000 7f4da4d81640 1 --2- 172.21.15.29:0/1035293519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d880051d0 0x7f4da009c050 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f4d90011810 tx=0x7f4d90011cd0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.636+0000 7f4da4d81640 1 -- 172.21.15.29:0/1035293519 shutdown_connections 2024-04-25T00:36:23.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.636+0000 7f4da4d81640 1 --2- 172.21.15.29:0/1035293519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da014dda0 0x7f4da015f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.636+0000 7f4da4d81640 1 --2- 172.21.15.29:0/1035293519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da0157db0 0x7f4da014d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.636+0000 7f4da4d81640 1 --2- 172.21.15.29:0/1035293519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d880051d0 0x7f4da009c050 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:23.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.636+0000 7f4da4d81640 1 -- 172.21.15.29:0/1035293519 >> 172.21.15.29:0/1035293519 conn(0x7f4da00b3a00 msgr2=0x7f4da00d2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:23.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.636+0000 7f4da4d81640 1 -- 172.21.15.29:0/1035293519 shutdown_connections 2024-04-25T00:36:23.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:23.636+0000 7f4da4d81640 1 -- 172.21.15.29:0/1035293519 wait complete. 2024-04-25T00:36:23.654 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:36:23.788 INFO:teuthology.orchestra.run.smithi029.stdout:[{"id":5873,"entity":{"name":{"type":"client","num":5873},"addr":{"type":"any","addr":"172.21.15.29:0","nonce":684573354}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.38799609800000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9982171143582488,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5873 172.21.15.29:0/684573354","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"y","hostname":"smithi029","pid":"27627","root":"/"}},{"id":6340,"entity":{"name":{"type":"client","num":6340},"addr":{"type":"any","addr":"172.21.15.29:0","nonce":3831800985}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.38799609800000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9982263479896378,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6340 172.21.15.29:0/3831800985","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"d2c45d977142e024a59b68a8f1dce8f0b9840551","ceph_version":"ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)","entity_id":"y","hostname":"smithi029","pid":"27627","root":"/"}}] 2024-04-25T00:36:23.789 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi029", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27627", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.29:0", "nonce": 684573354, "type": "any" }, "name": { "num": 5873, "type": "client" } }, "id": 5873, "inst": "client.5873 172.21.15.29:0/684573354", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9982171143582488 }, "state": "open", "uptime": 0.387996098 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi029", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "27627", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.29:0", "nonce": 3831800985, "type": "any" }, "name": { "num": 6340, "type": "client" } }, "id": 6340, "inst": "client.6340 172.21.15.29:0/3831800985", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9982263479896378 }, "state": "open", "uptime": 0.387996098 } ] 2024-04-25T00:36:23.789 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T00:36:23.789 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:36:23.789 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:36:23.798 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:23.796+0000 7fa1bebc0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:36:23.834 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:36:23.873 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:36:23.873 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:36:23.873 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:36:23.873 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:36:23.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:36:23.881 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:36:23.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:23.882 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:36:23.926 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:23.924+0000 7fa1bebc0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:36:23.929 INFO:teuthology.orchestra.run.smithi029.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T00:36:23.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:36:23.930 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:36:23.930 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:36:23.930 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:36:23.935 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:36:23.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:36:23.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:23.935 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:36:23.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:36:23.984 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T00:36:23.984 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:36:23.984 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:36:23.984 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:36:23.990 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:36:23.990 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:36:23.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:23.990 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:36:24.037 INFO:teuthology.orchestra.run.smithi029.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T00:36:24.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:36:24.038 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:36:24.038 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:36:24.038 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:36:24.044 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:36:24.044 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:36:24.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:24.044 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:36:24.091 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T00:36:24.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:36:24.093 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:36:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faef2e07640 1 Processor -- start 2024-04-25T00:36:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faef2e07640 1 -- start start 2024-04-25T00:36:24.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faef2e07640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faeec0b3140 0x7faeec06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faef2e07640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faeec0b4b10 0x7faeec0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faef2e07640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faeec0b3a60 0x7faeec15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faef2e07640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7faeec0b41e0 con 0x7faeec0b4b10 2024-04-25T00:36:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faef2e07640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faeec06d960 con 0x7faeec0b3a60 2024-04-25T00:36:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faef2e07640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faeec06dc40 con 0x7faeec0b3140 2024-04-25T00:36:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faeebfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faeec0b4b10 0x7faeec0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faeebfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faeec0b4b10 0x7faeec0b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48750/0 (socket says 172.21.15.29:48750) 2024-04-25T00:36:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faeebfff640 1 -- 172.21.15.29:0/3681184321 learned_addr learned my addr 172.21.15.29:0/3681184321 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faef0ba3640 1 --2- 172.21.15.29:0/3681184321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faeec0b3140 0x7faeec06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faef13a4640 1 --2- 172.21.15.29:0/3681184321 >> v2:172.21.15.157:3300/0 conn(0x7faeec0b3a60 0x7faeec15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faeebfff640 1 -- 172.21.15.29:0/3681184321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faeec0b3140 msgr2=0x7faeec06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faeebfff640 1 --2- 172.21.15.29:0/3681184321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faeec0b3140 0x7faeec06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faeebfff640 1 -- 172.21.15.29:0/3681184321 >> v2:172.21.15.157:3300/0 conn(0x7faeec0b3a60 msgr2=0x7faeec15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faeebfff640 1 --2- 172.21.15.29:0/3681184321 >> v2:172.21.15.157:3300/0 conn(0x7faeec0b3a60 0x7faeec15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faeebfff640 1 -- 172.21.15.29:0/3681184321 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeec06df20 con 0x7faeec0b4b10 2024-04-25T00:36:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.224+0000 7faef13a4640 1 --2- 172.21.15.29:0/3681184321 >> v2:172.21.15.157:3300/0 conn(0x7faeec0b3a60 0x7faeec15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:24.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faeebfff640 1 --2- 172.21.15.29:0/3681184321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faeec0b4b10 0x7faeec0b3520 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7faeec0b4790 tx=0x7faed801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1a5a4167d0fb3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faeeb7fe640 1 -- 172.21.15.29:0/3681184321 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed80b0070 con 0x7faeec0b4b10 2024-04-25T00:36:24.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faeeb7fe640 1 -- 172.21.15.29:0/3681184321 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed80ab600 con 0x7faeec0b4b10 2024-04-25T00:36:24.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faeeb7fe640 1 -- 172.21.15.29:0/3681184321 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed80ab9d0 con 0x7faeec0b4b10 2024-04-25T00:36:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 -- 172.21.15.29:0/3681184321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faeec0b4b10 msgr2=0x7faeec0b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 --2- 172.21.15.29:0/3681184321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faeec0b4b10 0x7faeec0b3520 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7faeec0b4790 tx=0x7faed801c520 comp rx=0 tx=0).stop 2024-04-25T00:36:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 -- 172.21.15.29:0/3681184321 shutdown_connections 2024-04-25T00:36:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 --2- 172.21.15.29:0/3681184321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faeec0b4b10 0x7faeec0b3520 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 --2- 172.21.15.29:0/3681184321 >> v2:172.21.15.157:3300/0 conn(0x7faeec0b3a60 0x7faeec15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 --2- 172.21.15.29:0/3681184321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faeec0b3140 0x7faeec06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 -- 172.21.15.29:0/3681184321 >> 172.21.15.29:0/3681184321 conn(0x7faeec0b1870 msgr2=0x7faeec0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 -- 172.21.15.29:0/3681184321 shutdown_connections 2024-04-25T00:36:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 -- 172.21.15.29:0/3681184321 wait complete. 2024-04-25T00:36:24.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 Processor -- start 2024-04-25T00:36:24.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 -- start start 2024-04-25T00:36:24.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faeec0b3140 0x7faeec070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faeec0b3a60 0x7faeec13ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faeec0b4b10 0x7faeec1594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faeec0ce3c0 con 0x7faeec0b3140 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faeec0ce240 con 0x7faeec0b3a60 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef2e07640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faeec0ce540 con 0x7faeec0b4b10 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef0ba3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faeec0b3140 0x7faeec070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef0ba3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faeec0b3140 0x7faeec070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48758/0 (socket says 172.21.15.29:48758) 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef0ba3640 1 -- 172.21.15.29:0/1945809877 learned_addr learned my addr 172.21.15.29:0/1945809877 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef13a4640 1 --2- 172.21.15.29:0/1945809877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faeec0b4b10 0x7faeec1594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef0ba3640 1 -- 172.21.15.29:0/1945809877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faeec0b4b10 msgr2=0x7faeec1594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef0ba3640 1 --2- 172.21.15.29:0/1945809877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faeec0b4b10 0x7faeec1594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef0ba3640 1 -- 172.21.15.29:0/1945809877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faeec0b3a60 msgr2=0x7faeec13ccc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef0ba3640 1 --2- 172.21.15.29:0/1945809877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faeec0b3a60 0x7faeec13ccc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.228+0000 7faef0ba3640 1 -- 172.21.15.29:0/1945809877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeec166700 con 0x7faeec0b3140 2024-04-25T00:36:24.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.232+0000 7faef0ba3640 1 --2- 172.21.15.29:0/1945809877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faeec0b3140 0x7faeec070ce0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7faee000d830 tx=0x7faee000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.232+0000 7faee97fa640 1 -- 172.21.15.29:0/1945809877 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0018020 con 0x7faeec0b3140 2024-04-25T00:36:24.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.232+0000 7faee97fa640 1 -- 172.21.15.29:0/1945809877 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee00096a0 con 0x7faeec0b3140 2024-04-25T00:36:24.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.232+0000 7faef2e07640 1 -- 172.21.15.29:0/1945809877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeec06cfc0 con 0x7faeec0b3140 2024-04-25T00:36:24.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.232+0000 7faee97fa640 1 -- 172.21.15.29:0/1945809877 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0009a50 con 0x7faeec0b3140 2024-04-25T00:36:24.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.232+0000 7faef2e07640 1 -- 172.21.15.29:0/1945809877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeec13d200 con 0x7faeec0b3140 2024-04-25T00:36:24.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.232+0000 7faee97fa640 1 -- 172.21.15.29:0/1945809877 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 13) ==== 43764+0+0 (secure 0 0 0) 0x7faee001b020 con 0x7faeec0b3140 2024-04-25T00:36:24.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.232+0000 7faee97fa640 1 -- 172.21.15.29:0/1945809877 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6326+0+0 (secure 0 0 0) 0x7faee00490b0 con 0x7faeec0b3140 2024-04-25T00:36:24.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.232+0000 7faef2e07640 1 -- 172.21.15.29:0/1945809877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeec069270 con 0x7faeec0b3140 2024-04-25T00:36:24.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.236+0000 7faee97fa640 1 -- 172.21.15.29:0/1945809877 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faee0022030 con 0x7faeec0b3140 2024-04-25T00:36:24.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.272+0000 7faee97fa640 1 -- 172.21.15.29:0/1945809877 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 14) ==== 43891+0+0 (secure 0 0 0) 0x7faee00146f0 con 0x7faeec0b3140 2024-04-25T00:36:24.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.272+0000 7faee97fa640 1 --2- 172.21.15.29:0/1945809877 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faed003ba10 0x7faed003de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.272+0000 7faeebfff640 1 --2- 172.21.15.29:0/1945809877 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faed003ba10 0x7faed003de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.276+0000 7faeebfff640 1 --2- 172.21.15.29:0/1945809877 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faed003ba10 0x7faed003de00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faed80ad000 tx=0x7faed80a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.347 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:24.348+0000 7fa1bebc0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:36:24.398 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:24.396+0000 7fa1bebc0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:36:24.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.436+0000 7faef2e07640 1 -- 172.21.15.29:0/1945809877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faeec164b30 con 0x7faeec0b3140 2024-04-25T00:36:24.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.440+0000 7faee97fa640 1 -- 172.21.15.29:0/1945809877 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7faee00102e0 con 0x7faeec0b3140 2024-04-25T00:36:24.441 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:24.441 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/524224365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":524224365},{"type":"v1","addr":"172.21.15.157:6835","nonce":524224365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1007000548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1007000548},{"type":"v1","addr":"172.21.15.157:6837","nonce":1007000548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6810,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2091414875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2091414875},{"type":"v1","addr":"172.21.15.29:6833","nonce":2091414875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:49.528297+0000","modified":"2024-04-25T00:35:52.569008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1111146974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1111146974},{"type":"v1","addr":"172.21.15.29:6835","nonce":1111146974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6822,"qdb_cluster":[6822]},"id":5}]} 2024-04-25T00:36:24.442 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 44 2024-04-25T00:36:24.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.440+0000 7faef2e07640 1 -- 172.21.15.29:0/1945809877 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faed003ba10 msgr2=0x7faed003de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.440+0000 7faef2e07640 1 --2- 172.21.15.29:0/1945809877 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faed003ba10 0x7faed003de00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faed80ad000 tx=0x7faed80a6000 comp rx=0 tx=0).stop 2024-04-25T00:36:24.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.440+0000 7faef2e07640 1 -- 172.21.15.29:0/1945809877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faeec0b3140 msgr2=0x7faeec070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.440+0000 7faef2e07640 1 --2- 172.21.15.29:0/1945809877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faeec0b3140 0x7faeec070ce0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7faee000d830 tx=0x7faee000dcf0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.440+0000 7faef2e07640 1 -- 172.21.15.29:0/1945809877 shutdown_connections 2024-04-25T00:36:24.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.440+0000 7faef2e07640 1 --2- 172.21.15.29:0/1945809877 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faed003ba10 0x7faed003de00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.440+0000 7faef2e07640 1 --2- 172.21.15.29:0/1945809877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faeec0b4b10 0x7faeec1594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.440+0000 7faef2e07640 1 --2- 172.21.15.29:0/1945809877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faeec0b3a60 0x7faeec13ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.440+0000 7faef2e07640 1 --2- 172.21.15.29:0/1945809877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faeec0b3140 0x7faeec070ce0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.440+0000 7faef2e07640 1 -- 172.21.15.29:0/1945809877 >> 172.21.15.29:0/1945809877 conn(0x7faeec0b1870 msgr2=0x7faeec0ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:24.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.440+0000 7faef2e07640 1 -- 172.21.15.29:0/1945809877 shutdown_connections 2024-04-25T00:36:24.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.440+0000 7faef2e07640 1 -- 172.21.15.29:0/1945809877 wait complete. 2024-04-25T00:36:24.445 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:24.444+0000 7fa1bebc0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:36:24.460 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:36:24.519 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:24.516+0000 7fa1bebc0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:36:24.567 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:24.568+0000 7fa1bebc0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:36:24.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7ad6aa9640 1 Processor -- start 2024-04-25T00:36:24.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7ad6aa9640 1 -- start start 2024-04-25T00:36:24.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7ad6aa9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ad006cd30 0x7f7ad0162450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7ad6aa9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad00b2380 0x7f7ad00b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7ad6aa9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ad006d4a0 con 0x7f7ad015e2b0 2024-04-25T00:36:24.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7ad6aa9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad006d780 con 0x7f7ad006cd30 2024-04-25T00:36:24.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7ad6aa9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad015e690 con 0x7f7ad00b2380 2024-04-25T00:36:24.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7acffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ad006cd30 0x7f7ad0162450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7acffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ad006cd30 0x7f7ad0162450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48768/0 (socket says 172.21.15.29:48768) 2024-04-25T00:36:24.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7acffff640 1 -- 172.21.15.29:0/3668940545 learned_addr learned my addr 172.21.15.29:0/3668940545 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:24.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7acffff640 1 -- 172.21.15.29:0/3668940545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad00b2380 msgr2=0x7f7ad00b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7ad5046640 1 --2- 172.21.15.29:0/3668940545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad00b2380 0x7f7ad00b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7acffff640 1 --2- 172.21.15.29:0/3668940545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad00b2380 0x7f7ad00b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7acffff640 1 -- 172.21.15.29:0/3668940545 >> v1:172.21.15.157:6789/0 conn(0x7f7ad015e2b0 legacy=0x7f7ad0161d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7acffff640 1 -- 172.21.15.29:0/3668940545 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad00b18a0 con 0x7f7ad006cd30 2024-04-25T00:36:24.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.584+0000 7f7ad5046640 1 --2- 172.21.15.29:0/3668940545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad00b2380 0x7f7ad00b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:24.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7acffff640 1 --2- 172.21.15.29:0/3668940545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad006cd30 0x7f7ad0162450 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009e80 tx=0x7f7ab801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82eb6fb45239f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7acf7fe640 1 -- 172.21.15.29:0/3668940545 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80a57d0 con 0x7f7ad006cd30 2024-04-25T00:36:24.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7acf7fe640 1 -- 172.21.15.29:0/3668940545 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab80a5970 con 0x7f7ad006cd30 2024-04-25T00:36:24.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7acf7fe640 1 -- 172.21.15.29:0/3668940545 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80a5d40 con 0x7f7ad006cd30 2024-04-25T00:36:24.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/3668940545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad006cd30 msgr2=0x7f7ad0162450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 --2- 172.21.15.29:0/3668940545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad006cd30 0x7f7ad0162450 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009e80 tx=0x7f7ab801ca10 comp rx=0 tx=0).stop 2024-04-25T00:36:24.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/3668940545 shutdown_connections 2024-04-25T00:36:24.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 --2- 172.21.15.29:0/3668940545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad00b2380 0x7f7ad00b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 --2- 172.21.15.29:0/3668940545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad006cd30 0x7f7ad0162450 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/3668940545 >> 172.21.15.29:0/3668940545 conn(0x7f7ad00b38b0 msgr2=0x7f7ad00b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:24.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/3668940545 shutdown_connections 2024-04-25T00:36:24.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/3668940545 wait complete. 2024-04-25T00:36:24.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 Processor -- start 2024-04-25T00:36:24.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 -- start start 2024-04-25T00:36:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad006cd30 0x7f7ad014f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad00b2380 0x7f7ad00b6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad015e2b0 0x7f7ad0151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad0162b10 con 0x7f7ad00b2380 2024-04-25T00:36:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad0162990 con 0x7f7ad006cd30 2024-04-25T00:36:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7ad6aa9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad016b080 con 0x7f7ad015e2b0 2024-04-25T00:36:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7acffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad00b2380 0x7f7ad00b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7acffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad00b2380 0x7f7ad00b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48784/0 (socket says 172.21.15.29:48784) 2024-04-25T00:36:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.588+0000 7f7acffff640 1 -- 172.21.15.29:0/2855956869 learned_addr learned my addr 172.21.15.29:0/2855956869 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7ad5046640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad015e2b0 0x7f7ad0151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7ad4845640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad006cd30 0x7f7ad014f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7acffff640 1 -- 172.21.15.29:0/2855956869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad015e2b0 msgr2=0x7f7ad0151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7acffff640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad015e2b0 0x7f7ad0151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7acffff640 1 -- 172.21.15.29:0/2855956869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad006cd30 msgr2=0x7f7ad014f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7acffff640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad006cd30 0x7f7ad014f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7acffff640 1 -- 172.21.15.29:0/2855956869 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad00c8bf0 con 0x7f7ad00b2380 2024-04-25T00:36:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7acffff640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad00b2380 0x7f7ad00b6920 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80af040 tx=0x7f7ab80197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7acd7fa640 1 -- 172.21.15.29:0/2855956869 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80b8670 con 0x7f7ad00b2380 2024-04-25T00:36:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7acd7fa640 1 -- 172.21.15.29:0/2855956869 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab80be070 con 0x7f7ad00b2380 2024-04-25T00:36:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/2855956869 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad014fc50 con 0x7f7ad00b2380 2024-04-25T00:36:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7acd7fa640 1 -- 172.21.15.29:0/2855956869 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80b8a70 con 0x7f7ad00b2380 2024-04-25T00:36:24.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/2855956869 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad015be00 con 0x7f7ad00b2380 2024-04-25T00:36:24.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7acd7fa640 1 -- 172.21.15.29:0/2855956869 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) ==== 43891+0+0 (secure 0 0 0) 0x7f7ab80b8c10 con 0x7f7ad00b2380 2024-04-25T00:36:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/2855956869 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad0069270 con 0x7f7ad00b2380 2024-04-25T00:36:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7acd7fa640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7ab0035d40 0x7f7ab0038200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7acd7fa640 1 -- 172.21.15.29:0/2855956869 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6326+0+0 (secure 0 0 0) 0x7f7ab80e3cc0 con 0x7f7ad00b2380 2024-04-25T00:36:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7ad4845640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7ab0035d40 0x7f7ab0038200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.592+0000 7f7ad4845640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7ab0035d40 0x7f7ab0038200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8001650 tx=0x7f7ac8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.596+0000 7f7acd7fa640 1 -- 172.21.15.29:0/2855956869 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7ab8016600 con 0x7f7ad00b2380 2024-04-25T00:36:24.641 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:24.640+0000 7fa1bebc0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:36:24.690 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:24.688+0000 7fa1bebc0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:36:24.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.764+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/2855956869 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ad00b7ce0 con 0x7f7ad00b2380 2024-04-25T00:36:24.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.764+0000 7f7acd7fa640 1 -- 172.21.15.29:0/2855956869 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+17540 (secure 0 0 0) 0x7f7ab80a5940 con 0x7f7ad00b2380 2024-04-25T00:36:24.767 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:24.767 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":73,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:36:23.261760+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:47.559043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:48.530288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6832/3474584311":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/1408011519":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/4070029771":"2024-04-26T00:36:23.261666+0000","172.21.15.157:6833/3474584311":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3212538336":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/1922269872":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/37741218":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3796173968":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3438031706":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/1065963518":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/4165584909":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/2500505584":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3565775874":"2024-04-26T00:36:23.261666+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:36:24.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.768+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/2855956869 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7ab0035d40 msgr2=0x7f7ab0038200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.768+0000 7f7ad6aa9640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7ab0035d40 0x7f7ab0038200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8001650 tx=0x7f7ac8009040 comp rx=0 tx=0).stop 2024-04-25T00:36:24.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.768+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/2855956869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad00b2380 msgr2=0x7f7ad00b6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.768+0000 7f7ad6aa9640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad00b2380 0x7f7ad00b6920 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80af040 tx=0x7f7ab80197b0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.768+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/2855956869 shutdown_connections 2024-04-25T00:36:24.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.768+0000 7f7ad6aa9640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7ab0035d40 0x7f7ab0038200 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.768+0000 7f7ad6aa9640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad015e2b0 0x7f7ad0151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.768+0000 7f7ad6aa9640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad00b2380 0x7f7ad00b6920 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.768+0000 7f7ad6aa9640 1 --2- 172.21.15.29:0/2855956869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad006cd30 0x7f7ad014f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.768+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/2855956869 >> 172.21.15.29:0/2855956869 conn(0x7f7ad00b38b0 msgr2=0x7f7ad0156050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:24.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.768+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/2855956869 shutdown_connections 2024-04-25T00:36:24.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.768+0000 7f7ad6aa9640 1 -- 172.21.15.29:0/2855956869 wait complete. 2024-04-25T00:36:24.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:35:47.559043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:35:48.530288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:36:24.787 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:36:24.827 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:24.828+0000 7fa1bebc0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:36:24.878 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:24.876+0000 7fa1bebc0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:36:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ff21640 1 Processor -- start 2024-04-25T00:36:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ff21640 1 -- start start 2024-04-25T00:36:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ff21640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe2380cf190 0x7fe2380b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ff21640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380d2360 0x7fe2380d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ff21640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2380cf570 con 0x7fe2380d06b0 2024-04-25T00:36:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ff21640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe2380d0b00 con 0x7fe2380cf190 2024-04-25T00:36:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ff21640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe2380b2bb0 con 0x7fe2380d2360 2024-04-25T00:36:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23e4be640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe2380d06b0 0x7fe23815fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55092/0 (socket says 172.21.15.29:55092) 2024-04-25T00:36:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23e4be640 1 -- 172.21.15.29:0/2528862705 learned_addr learned my addr 172.21.15.29:0/2528862705 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23d4bc640 1 --2- 172.21.15.29:0/2528862705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380d2360 0x7fe2380d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23dcbd640 1 --2- 172.21.15.29:0/2528862705 >> v2:172.21.15.157:3300/0 conn(0x7fe2380cf190 0x7fe2380b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1475617399 0 0) 0x7fe2380cf570 con 0x7fe2380d06b0 2024-04-25T00:36:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe224003530 con 0x7fe2380d06b0 2024-04-25T00:36:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 508995179 0 0) 0x7fe224003530 con 0x7fe2380d06b0 2024-04-25T00:36:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380d2360 msgr2=0x7fe2380d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 --2- 172.21.15.29:0/2528862705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380d2360 0x7fe2380d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 >> v2:172.21.15.157:3300/0 conn(0x7fe2380cf190 msgr2=0x7fe2380b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 --2- 172.21.15.29:0/2528862705 >> v2:172.21.15.157:3300/0 conn(0x7fe2380cf190 0x7fe2380b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2380b2e90 con 0x7fe2380d06b0 2024-04-25T00:36:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fe230001a00 con 0x7fe2380d06b0 2024-04-25T00:36:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23d4bc640 1 --2- 172.21.15.29:0/2528862705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380d2360 0x7fe2380d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23dcbd640 1 --2- 172.21.15.29:0/2528862705 >> v2:172.21.15.157:3300/0 conn(0x7fe2380cf190 0x7fe2380b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 >> v1:172.21.15.29:6789/0 conn(0x7fe2380d06b0 legacy=0x7fe23815fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 --2- 172.21.15.29:0/2528862705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2240039c0 0x7fe224003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 --2- 172.21.15.29:0/2528862705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe224004810 0x7fe224004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 --2- 172.21.15.29:0/2528862705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380d2360 0x7fe2240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe2380d0b00 con 0x7fe2240039c0 2024-04-25T00:36:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe2380b2bb0 con 0x7fe224004810 2024-04-25T00:36:24.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe22400bb30 con 0x7fe2380d2360 2024-04-25T00:36:24.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23dcbd640 1 --2- 172.21.15.29:0/2528862705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2240039c0 0x7fe224003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23d4bc640 1 --2- 172.21.15.29:0/2528862705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe224004810 0x7fe224004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23dcbd640 1 -- 172.21.15.29:0/2528862705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380d2360 msgr2=0x7fe2240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23e4be640 1 --2- 172.21.15.29:0/2528862705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380d2360 0x7fe2240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23dcbd640 1 --2- 172.21.15.29:0/2528862705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380d2360 0x7fe2240051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23dcbd640 1 -- 172.21.15.29:0/2528862705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe224004810 msgr2=0x7fe224004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:24.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23dcbd640 1 --2- 172.21.15.29:0/2528862705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe224004810 0x7fe224004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.924+0000 7fe23dcbd640 1 -- 172.21.15.29:0/2528862705 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe224003530 con 0x7fe2240039c0 2024-04-25T00:36:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.928+0000 7fe23dcbd640 1 --2- 172.21.15.29:0/2528862705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2240039c0 0x7fe224003de0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe2340098a0 tx=0x7fe23401c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184025add5d348d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.928+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23400f040 con 0x7fe2240039c0 2024-04-25T00:36:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.928+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2340a8610 con 0x7fe2240039c0 2024-04-25T00:36:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.928+0000 7fe23ccbb640 1 -- 172.21.15.29:0/2528862705 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2340a89c0 con 0x7fe2240039c0 2024-04-25T00:36:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.928+0000 7fe23ff21640 1 -- 172.21.15.29:0/2528862705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2240039c0 msgr2=0x7fe224003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.928+0000 7fe23ff21640 1 --2- 172.21.15.29:0/2528862705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2240039c0 0x7fe224003de0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe2340098a0 tx=0x7fe23401c5c0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.928+0000 7fe23e4be640 1 -- 172.21.15.29:0/2528862705 reap_dead start 2024-04-25T00:36:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.928+0000 7fe23ff21640 1 -- 172.21.15.29:0/2528862705 shutdown_connections 2024-04-25T00:36:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.928+0000 7fe23ff21640 1 -- 172.21.15.29:0/2528862705 >> 172.21.15.29:0/2528862705 conn(0x7fe23809aa30 msgr2=0x7fe23809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.928+0000 7fe23ff21640 1 -- 172.21.15.29:0/2528862705 shutdown_connections 2024-04-25T00:36:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.928+0000 7fe23ff21640 1 -- 172.21.15.29:0/2528862705 wait complete. 2024-04-25T00:36:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.928+0000 7fe23ff21640 1 Processor -- start 2024-04-25T00:36:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.928+0000 7fe23ff21640 1 -- start start 2024-04-25T00:36:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23ff21640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2240039c0 0x7fe238159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23ff21640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380ce4b0 0x7fe2380c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23ff21640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2380c7890 0x7fe23815cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23ff21640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe2380b2970 con 0x7fe2380c7890 2024-04-25T00:36:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23ff21640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe2380b27f0 con 0x7fe2240039c0 2024-04-25T00:36:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23ff21640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe238166bf0 con 0x7fe2380ce4b0 2024-04-25T00:36:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23e4be640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2380c7890 0x7fe23815cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23e4be640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2380c7890 0x7fe23815cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48804/0 (socket says 172.21.15.29:48804) 2024-04-25T00:36:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23e4be640 1 -- 172.21.15.29:0/1879858989 learned_addr learned my addr 172.21.15.29:0/1879858989 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23d4bc640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380ce4b0 0x7fe2380c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23dcbd640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2240039c0 0x7fe238159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23e4be640 1 -- 172.21.15.29:0/1879858989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380ce4b0 msgr2=0x7fe2380c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23e4be640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380ce4b0 0x7fe2380c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23e4be640 1 -- 172.21.15.29:0/1879858989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2240039c0 msgr2=0x7fe238159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23e4be640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2240039c0 0x7fe238159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23e4be640 1 -- 172.21.15.29:0/1879858989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2380c7d10 con 0x7fe2380c7890 2024-04-25T00:36:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23e4be640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2380c7890 0x7fe23815cf10 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe2300012f0 tx=0x7fe23000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe22e7fc640 1 -- 172.21.15.29:0/1879858989 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230004d70 con 0x7fe2380c7890 2024-04-25T00:36:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe22e7fc640 1 -- 172.21.15.29:0/1879858989 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe23000bb80 con 0x7fe2380c7890 2024-04-25T00:36:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe22e7fc640 1 -- 172.21.15.29:0/1879858989 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230007110 con 0x7fe2380c7890 2024-04-25T00:36:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23ff21640 1 -- 172.21.15.29:0/1879858989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2380c8f20 con 0x7fe2380c7890 2024-04-25T00:36:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23ff21640 1 -- 172.21.15.29:0/1879858989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23815a0f0 con 0x7fe2380c7890 2024-04-25T00:36:24.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe22e7fc640 1 -- 172.21.15.29:0/1879858989 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) ==== 43891+0+0 (secure 0 0 0) 0x7fe23001b060 con 0x7fe2380c7890 2024-04-25T00:36:24.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23ff21640 1 -- 172.21.15.29:0/1879858989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe238069270 con 0x7fe2380c7890 2024-04-25T00:36:24.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe22e7fc640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe21c035d10 0x7fe21c0381d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:24.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe22e7fc640 1 -- 172.21.15.29:0/1879858989 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6326+0+0 (secure 0 0 0) 0x7fe230048e90 con 0x7fe2380c7890 2024-04-25T00:36:24.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.932+0000 7fe23dcbd640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe21c035d10 0x7fe21c0381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:24.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.936+0000 7fe23dcbd640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe21c035d10 0x7fe21c0381d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe2340af000 tx=0x7fe2340046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:24.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:24.940+0000 7fe22e7fc640 1 -- 172.21.15.29:0/1879858989 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe230007490 con 0x7fe2380c7890 2024-04-25T00:36:25.094 INFO:tasks.ceph.mgr.x.smithi157.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T00:36:25.094 INFO:tasks.ceph.mgr.x.smithi157.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T00:36:25.094 INFO:tasks.ceph.mgr.x.smithi157.stderr: from numpy import show_config as show_numpy_config 2024-04-25T00:36:25.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.100+0000 7fe23ff21640 1 -- 172.21.15.29:0/1879858989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe23815f050 con 0x7fe2380c7890 2024-04-25T00:36:25.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.100+0000 7fe22e7fc640 1 -- 172.21.15.29:0/1879858989 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7fe23001d450 con 0x7fe2380c7890 2024-04-25T00:36:25.101 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:36:25.102 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:25.100+0000 7fa1bebc0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:36:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.100+0000 7fe23ff21640 1 -- 172.21.15.29:0/1879858989 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe21c035d10 msgr2=0x7fe21c0381d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.100+0000 7fe23ff21640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe21c035d10 0x7fe21c0381d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe2340af000 tx=0x7fe2340046d0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.100+0000 7fe23ff21640 1 -- 172.21.15.29:0/1879858989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2380c7890 msgr2=0x7fe23815cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.100+0000 7fe23ff21640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2380c7890 0x7fe23815cf10 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe2300012f0 tx=0x7fe23000b4a0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.100+0000 7fe23ff21640 1 -- 172.21.15.29:0/1879858989 shutdown_connections 2024-04-25T00:36:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.100+0000 7fe23ff21640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe21c035d10 0x7fe21c0381d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.100+0000 7fe23ff21640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2380c7890 0x7fe23815cf10 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.100+0000 7fe23ff21640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2380ce4b0 0x7fe2380c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.100+0000 7fe23ff21640 1 --2- 172.21.15.29:0/1879858989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2240039c0 0x7fe238159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.100+0000 7fe23ff21640 1 -- 172.21.15.29:0/1879858989 >> 172.21.15.29:0/1879858989 conn(0x7fe23809aa30 msgr2=0x7fe2381663e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.104+0000 7fe23ff21640 1 -- 172.21.15.29:0/1879858989 shutdown_connections 2024-04-25T00:36:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.104+0000 7fe23ff21640 1 -- 172.21.15.29:0/1879858989 wait complete. 2024-04-25T00:36:25.120 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:36:25.234 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:25.232+0000 7fa1bebc0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:36:25.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef0806a640 1 Processor -- start 2024-04-25T00:36:25.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef0806a640 1 -- start start 2024-04-25T00:36:25.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef0806a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef000b4780 0x7fef0015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef0806a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fef000cc240 0x7fef0015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef0806a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef000b3720 con 0x7fef0015e8d0 2024-04-25T00:36:25.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef0806a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fef000b42f0 con 0x7fef000cc240 2024-04-25T00:36:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef0806a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fef000b39c0 con 0x7fef000b4780 2024-04-25T00:36:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef06607640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fef000cc240 0x7fef0015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef05e06640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fef0015e8d0 0x7fef0015ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55098/0 (socket says 172.21.15.29:55098) 2024-04-25T00:36:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef05e06640 1 -- 172.21.15.29:0/647819601 learned_addr learned my addr 172.21.15.29:0/647819601 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef06607640 1 --2- 172.21.15.29:0/647819601 >> v2:172.21.15.29:3300/0 conn(0x7fef000cc240 0x7fef0015c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48820/0 (socket says 172.21.15.29:48820) 2024-04-25T00:36:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef05605640 1 --2- 172.21.15.29:0/647819601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef000b4780 0x7fef0015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef06607640 1 -- 172.21.15.29:0/647819601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef000b4780 msgr2=0x7fef0015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef06607640 1 --2- 172.21.15.29:0/647819601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef000b4780 0x7fef0015f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef06607640 1 -- 172.21.15.29:0/647819601 >> v1:172.21.15.29:6789/0 conn(0x7fef0015e8d0 legacy=0x7fef0015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef06607640 1 -- 172.21.15.29:0/647819601 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef000b3ca0 con 0x7fef000cc240 2024-04-25T00:36:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.236+0000 7fef04e04640 1 -- 172.21.15.29:0/647819601 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1264354172 0 0) 0x7fef000b3720 con 0x7fef0015e8d0 2024-04-25T00:36:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.240+0000 7fef06607640 1 --2- 172.21.15.29:0/647819601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef000cc240 0x7fef0015c7d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7feef8009d90 tx=0x7feef801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11fc3bd945be3dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.240+0000 7fef04e04640 1 -- 172.21.15.29:0/647819601 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef80b2070 con 0x7fef000cc240 2024-04-25T00:36:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.240+0000 7fef04e04640 1 -- 172.21.15.29:0/647819601 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef80a5900 con 0x7fef000cc240 2024-04-25T00:36:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.240+0000 7fef04e04640 1 -- 172.21.15.29:0/647819601 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef80a5cd0 con 0x7fef000cc240 2024-04-25T00:36:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.240+0000 7fef0806a640 1 -- 172.21.15.29:0/647819601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef000cc240 msgr2=0x7fef0015c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.240+0000 7fef0806a640 1 --2- 172.21.15.29:0/647819601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef000cc240 0x7fef0015c7d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7feef8009d90 tx=0x7feef801c970 comp rx=0 tx=0).stop 2024-04-25T00:36:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.240+0000 7fef0806a640 1 -- 172.21.15.29:0/647819601 shutdown_connections 2024-04-25T00:36:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.240+0000 7fef0806a640 1 --2- 172.21.15.29:0/647819601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef000cc240 0x7fef0015c7d0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.240+0000 7fef0806a640 1 --2- 172.21.15.29:0/647819601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef000b4780 0x7fef0015f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.240+0000 7fef0806a640 1 -- 172.21.15.29:0/647819601 >> 172.21.15.29:0/647819601 conn(0x7fef000b1870 msgr2=0x7fef000b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:25.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.240+0000 7fef0806a640 1 -- 172.21.15.29:0/647819601 shutdown_connections 2024-04-25T00:36:25.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.240+0000 7fef0806a640 1 -- 172.21.15.29:0/647819601 wait complete. 2024-04-25T00:36:25.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.240+0000 7fef0806a640 1 Processor -- start 2024-04-25T00:36:25.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef0806a640 1 -- start start 2024-04-25T00:36:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef0806a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef000b4780 0x7fef000af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef0806a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef000cc240 0x7fef00139c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef0806a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0015e8d0 0x7fef00154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef0806a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fef0015d6a0 con 0x7fef000b4780 2024-04-25T00:36:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef0806a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef0015d520 con 0x7fef0015e8d0 2024-04-25T00:36:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef0806a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fef00167c70 con 0x7fef000cc240 2024-04-25T00:36:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef05e06640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef000b4780 0x7fef000af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef05e06640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef000b4780 0x7fef000af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48824/0 (socket says 172.21.15.29:48824) 2024-04-25T00:36:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef05e06640 1 -- 172.21.15.29:0/1775222150 learned_addr learned my addr 172.21.15.29:0/1775222150 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef05605640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef000cc240 0x7fef00139c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef06607640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0015e8d0 0x7fef00154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef05e06640 1 -- 172.21.15.29:0/1775222150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef000cc240 msgr2=0x7fef00139c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef05e06640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef000cc240 0x7fef00139c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef05e06640 1 -- 172.21.15.29:0/1775222150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0015e8d0 msgr2=0x7fef00154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef05e06640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0015e8d0 0x7fef00154610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef05e06640 1 -- 172.21.15.29:0/1775222150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0013a170 con 0x7fef000b4780 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef05605640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef000cc240 0x7fef00139c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef05e06640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef000b4780 0x7fef000af020 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7feefc00c480 tx=0x7feefc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7feef67fc640 1 -- 172.21.15.29:0/1775222150 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc018070 con 0x7fef000b4780 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7feef67fc640 1 -- 172.21.15.29:0/1775222150 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feefc007500 con 0x7fef000b4780 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef0806a640 1 -- 172.21.15.29:0/1775222150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef00154f60 con 0x7fef000b4780 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7feef67fc640 1 -- 172.21.15.29:0/1775222150 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc004a30 con 0x7fef000b4780 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef0806a640 1 -- 172.21.15.29:0/1775222150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef000afd20 con 0x7fef000b4780 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef0806a640 1 -- 172.21.15.29:0/1775222150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef00069270 con 0x7fef000b4780 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7feef67fc640 1 -- 172.21.15.29:0/1775222150 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) ==== 43891+0+0 (secure 0 0 0) 0x7feefc01b070 con 0x7fef000b4780 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7feef67fc640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feee4035d10 0x7feee40381d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7feef67fc640 1 -- 172.21.15.29:0/1775222150 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6326+0+0 (secure 0 0 0) 0x7feefc0481b0 con 0x7fef000b4780 2024-04-25T00:36:25.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef05605640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feee4035d10 0x7feee40381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.244+0000 7fef05605640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feee4035d10 0x7feee40381d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fef00157c60 tx=0x7feef0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:25.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.248+0000 7feef67fc640 1 -- 172.21.15.29:0/1775222150 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feefc01ae10 con 0x7fef000b4780 2024-04-25T00:36:25.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.276+0000 7feef67fc640 1 -- 172.21.15.29:0/1775222150 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 15) ==== 44099+0+0 (secure 0 0 0) 0x7feefc01c8d0 con 0x7fef000b4780 2024-04-25T00:36:25.347 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:25.344+0000 7fa1bebc0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:36:25.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.408+0000 7fef0806a640 1 -- 172.21.15.29:0/1775222150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fef0006e7b0 con 0x7fef000b4780 2024-04-25T00:36:25.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.408+0000 7feef67fc640 1 -- 172.21.15.29:0/1775222150 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) ==== 104+0+11 (secure 0 0 0) 0x7feefc016120 con 0x7fef000b4780 2024-04-25T00:36:25.412 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:36:25.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.412+0000 7fef0806a640 1 -- 172.21.15.29:0/1775222150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feee4035d10 msgr2=0x7feee40381d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.412+0000 7fef0806a640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feee4035d10 0x7feee40381d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fef00157c60 tx=0x7feef0008040 comp rx=0 tx=0).stop 2024-04-25T00:36:25.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.412+0000 7fef0806a640 1 -- 172.21.15.29:0/1775222150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef000b4780 msgr2=0x7fef000af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.412+0000 7fef0806a640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef000b4780 0x7fef000af020 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7feefc00c480 tx=0x7feefc00c940 comp rx=0 tx=0).stop 2024-04-25T00:36:25.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.412+0000 7fef0806a640 1 -- 172.21.15.29:0/1775222150 shutdown_connections 2024-04-25T00:36:25.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.412+0000 7fef0806a640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feee4035d10 0x7feee40381d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.412+0000 7fef0806a640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef0015e8d0 0x7fef00154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.412+0000 7fef0806a640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef000cc240 0x7fef00139c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.412+0000 7fef0806a640 1 --2- 172.21.15.29:0/1775222150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef000b4780 0x7fef000af020 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.412+0000 7fef0806a640 1 -- 172.21.15.29:0/1775222150 >> 172.21.15.29:0/1775222150 conn(0x7fef000b1870 msgr2=0x7fef000cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:25.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.412+0000 7fef0806a640 1 -- 172.21.15.29:0/1775222150 shutdown_connections 2024-04-25T00:36:25.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.412+0000 7fef0806a640 1 -- 172.21.15.29:0/1775222150 wait complete. 2024-04-25T00:36:25.415 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:25.412+0000 7fa1bebc0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:36:25.429 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:36:25.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eefd42640 1 Processor -- start 2024-04-25T00:36:25.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eefd42640 1 -- start start 2024-04-25T00:36:25.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eefd42640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4ee815bd60 0x7f4ee806b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eefd42640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ee80cb330 0x7f4ee80cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eefd42640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ee806e1a0 con 0x7f4ee806dd10 2024-04-25T00:36:25.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eefd42640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee806edb0 con 0x7f4ee815bd60 2024-04-25T00:36:25.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eefd42640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee806e480 con 0x7f4ee80cb330 2024-04-25T00:36:25.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eedade640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4ee815bd60 0x7f4ee806b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eedade640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4ee815bd60 0x7f4ee806b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48860/0 (socket says 172.21.15.29:48860) 2024-04-25T00:36:25.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eedade640 1 -- 172.21.15.29:0/2928958203 learned_addr learned my addr 172.21.15.29:0/2928958203 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:25.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eedade640 1 -- 172.21.15.29:0/2928958203 >> v2:172.21.15.157:3300/0 conn(0x7f4ee80cb330 msgr2=0x7f4ee80cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:25.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eedade640 1 --2- 172.21.15.29:0/2928958203 >> v2:172.21.15.157:3300/0 conn(0x7f4ee80cb330 0x7f4ee80cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eedade640 1 -- 172.21.15.29:0/2928958203 >> v1:172.21.15.157:6789/0 conn(0x7f4ee806dd10 legacy=0x7f4ee815f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:25.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.544+0000 7f4eedade640 1 -- 172.21.15.29:0/2928958203 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee806e760 con 0x7f4ee815bd60 2024-04-25T00:36:25.548 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:25.548+0000 7fa1bebc0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:36:25.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eedade640 1 --2- 172.21.15.29:0/2928958203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee815bd60 0x7f4ee806b210 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8160a20 tx=0x7f4ee401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb42468d8b4a386 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:25.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eecadc640 1 -- 172.21.15.29:0/2928958203 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40a5860 con 0x7f4ee815bd60 2024-04-25T00:36:25.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eecadc640 1 -- 172.21.15.29:0/2928958203 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee40a5a00 con 0x7f4ee815bd60 2024-04-25T00:36:25.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eecadc640 1 -- 172.21.15.29:0/2928958203 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40a5dd0 con 0x7f4ee815bd60 2024-04-25T00:36:25.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eefd42640 1 -- 172.21.15.29:0/2928958203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee815bd60 msgr2=0x7f4ee806b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eefd42640 1 --2- 172.21.15.29:0/2928958203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee815bd60 0x7f4ee806b210 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8160a20 tx=0x7f4ee401c980 comp rx=0 tx=0).stop 2024-04-25T00:36:25.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eefd42640 1 -- 172.21.15.29:0/2928958203 shutdown_connections 2024-04-25T00:36:25.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eefd42640 1 --2- 172.21.15.29:0/2928958203 >> v2:172.21.15.157:3300/0 conn(0x7f4ee80cb330 0x7f4ee80cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eefd42640 1 --2- 172.21.15.29:0/2928958203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee815bd60 0x7f4ee806b210 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eefd42640 1 -- 172.21.15.29:0/2928958203 >> 172.21.15.29:0/2928958203 conn(0x7f4ee806c320 msgr2=0x7f4ee806c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:25.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eefd42640 1 -- 172.21.15.29:0/2928958203 shutdown_connections 2024-04-25T00:36:25.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eefd42640 1 -- 172.21.15.29:0/2928958203 wait complete. 2024-04-25T00:36:25.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eefd42640 1 Processor -- start 2024-04-25T00:36:25.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.548+0000 7f4eefd42640 1 -- start start 2024-04-25T00:36:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eefd42640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ee80cb330 0x7f4ee8152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eefd42640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ee815bd60 0x7f4ee80c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eefd42640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80c56f0 0x7f4ee80b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eefd42640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8168f30 con 0x7f4ee80c56f0 2024-04-25T00:36:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eefd42640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee806bb60 con 0x7f4ee80cb330 2024-04-25T00:36:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eefd42640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee81690b0 con 0x7f4ee815bd60 2024-04-25T00:36:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eee2df640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80c56f0 0x7f4ee80b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eee2df640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80c56f0 0x7f4ee80b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48868/0 (socket says 172.21.15.29:48868) 2024-04-25T00:36:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eee2df640 1 -- 172.21.15.29:0/3401432391 learned_addr learned my addr 172.21.15.29:0/3401432391 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eed2dd640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ee815bd60 0x7f4ee80c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eedade640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ee80cb330 0x7f4ee8152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eee2df640 1 -- 172.21.15.29:0/3401432391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ee815bd60 msgr2=0x7f4ee80c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eee2df640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ee815bd60 0x7f4ee80c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eee2df640 1 -- 172.21.15.29:0/3401432391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ee80cb330 msgr2=0x7f4ee8152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eee2df640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ee80cb330 0x7f4ee8152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eee2df640 1 -- 172.21.15.29:0/3401432391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee8152d80 con 0x7f4ee80c56f0 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eee2df640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80c56f0 0x7f4ee80b1090 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000b460 tx=0x7f4ee000b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4ede7fc640 1 -- 172.21.15.29:0/3401432391 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0015020 con 0x7f4ee80c56f0 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4ede7fc640 1 -- 172.21.15.29:0/3401432391 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee0011890 con 0x7f4ee80c56f0 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eefd42640 1 -- 172.21.15.29:0/3401432391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee80c5d20 con 0x7f4ee80c56f0 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4ede7fc640 1 -- 172.21.15.29:0/3401432391 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0011c80 con 0x7f4ee80c56f0 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eefd42640 1 -- 172.21.15.29:0/3401432391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee80c62e0 con 0x7f4ee80c56f0 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eefd42640 1 -- 172.21.15.29:0/3401432391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee8069270 con 0x7f4ee80c56f0 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4ede7fc640 1 -- 172.21.15.29:0/3401432391 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 44099+0+0 (secure 0 0 0) 0x7f4ee0018020 con 0x7f4ee80c56f0 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4ede7fc640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4ecc035f50 0x7f4ecc038410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4ede7fc640 1 -- 172.21.15.29:0/3401432391 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6326+0+0 (secure 0 0 0) 0x7f4ee0049330 con 0x7f4ee80c56f0 2024-04-25T00:36:25.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eedade640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4ecc035f50 0x7f4ecc038410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.552+0000 7f4eedade640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4ecc035f50 0x7f4ecc038410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40ae000 tx=0x7f4ee40a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:25.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.556+0000 7f4ede7fc640 1 -- 172.21.15.29:0/3401432391 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4ee001f020 con 0x7f4ee80c56f0 2024-04-25T00:36:25.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.716+0000 7f4eefd42640 1 -- 172.21.15.29:0/3401432391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4ee8072240 con 0x7f4ee80c56f0 2024-04-25T00:36:25.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.716+0000 7f4ede7fc640 1 -- 172.21.15.29:0/3401432391 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) ==== 100+0+11 (secure 0 0 0) 0x7f4ee000d040 con 0x7f4ee80c56f0 2024-04-25T00:36:25.720 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:36:25.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.720+0000 7f4eefd42640 1 -- 172.21.15.29:0/3401432391 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4ecc035f50 msgr2=0x7f4ecc038410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.720+0000 7f4eefd42640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4ecc035f50 0x7f4ecc038410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40ae000 tx=0x7f4ee40a7070 comp rx=0 tx=0).stop 2024-04-25T00:36:25.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.720+0000 7f4eefd42640 1 -- 172.21.15.29:0/3401432391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80c56f0 msgr2=0x7f4ee80b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.720+0000 7f4eefd42640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80c56f0 0x7f4ee80b1090 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000b460 tx=0x7f4ee000b920 comp rx=0 tx=0).stop 2024-04-25T00:36:25.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.720+0000 7f4eefd42640 1 -- 172.21.15.29:0/3401432391 shutdown_connections 2024-04-25T00:36:25.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.720+0000 7f4eefd42640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4ecc035f50 0x7f4ecc038410 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.720+0000 7f4eefd42640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80c56f0 0x7f4ee80b1090 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.720+0000 7f4eefd42640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ee815bd60 0x7f4ee80c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.720+0000 7f4eefd42640 1 --2- 172.21.15.29:0/3401432391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ee80cb330 0x7f4ee8152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.720+0000 7f4eefd42640 1 -- 172.21.15.29:0/3401432391 >> 172.21.15.29:0/3401432391 conn(0x7f4ee806c320 msgr2=0x7f4ee80b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:25.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.720+0000 7f4eefd42640 1 -- 172.21.15.29:0/3401432391 shutdown_connections 2024-04-25T00:36:25.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.720+0000 7f4eefd42640 1 -- 172.21.15.29:0/3401432391 wait complete. 2024-04-25T00:36:25.739 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:36:25.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc738c640 1 Processor -- start 2024-04-25T00:36:25.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc738c640 1 -- start start 2024-04-25T00:36:25.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc738c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcfc006b210 0x7fcfc006db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc738c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcfc006b5f0 0x7fcfc0071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc738c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc00d0410 con 0x7fcfc00d18f0 2024-04-25T00:36:25.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc738c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc00ce880 con 0x7fcfc006b5f0 2024-04-25T00:36:25.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc738c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc00d06b0 con 0x7fcfc006b210 2024-04-25T00:36:25.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc5929640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcfc006b5f0 0x7fcfc0071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc5929640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcfc006b5f0 0x7fcfc0071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48880/0 (socket says 172.21.15.29:48880) 2024-04-25T00:36:25.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc5929640 1 -- 172.21.15.29:0/4023091907 learned_addr learned my addr 172.21.15.29:0/4023091907 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:25.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc5128640 1 --2- 172.21.15.29:0/4023091907 >> v2:172.21.15.157:3300/0 conn(0x7fcfc006b210 0x7fcfc006db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc5929640 1 -- 172.21.15.29:0/4023091907 >> v2:172.21.15.157:3300/0 conn(0x7fcfc006b210 msgr2=0x7fcfc006db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:25.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc5929640 1 --2- 172.21.15.29:0/4023091907 >> v2:172.21.15.157:3300/0 conn(0x7fcfc006b210 0x7fcfc006db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc5929640 1 -- 172.21.15.29:0/4023091907 >> v1:172.21.15.157:6789/0 conn(0x7fcfc00d18f0 legacy=0x7fcfc015f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc5929640 1 -- 172.21.15.29:0/4023091907 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc00d0990 con 0x7fcfc006b5f0 2024-04-25T00:36:25.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.864+0000 7fcfc5128640 1 --2- 172.21.15.29:0/4023091907 >> v2:172.21.15.157:3300/0 conn(0x7fcfc006b210 0x7fcfc006db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:25.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc5929640 1 --2- 172.21.15.29:0/4023091907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcfc006b5f0 0x7fcfc0071fc0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0075730 tx=0x7fcfb801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b48104dc7276 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:25.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfb7fff640 1 -- 172.21.15.29:0/4023091907 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80a57a0 con 0x7fcfc006b5f0 2024-04-25T00:36:25.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfb7fff640 1 -- 172.21.15.29:0/4023091907 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb80a5940 con 0x7fcfc006b5f0 2024-04-25T00:36:25.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfb7fff640 1 -- 172.21.15.29:0/4023091907 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80a5d10 con 0x7fcfc006b5f0 2024-04-25T00:36:25.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 -- 172.21.15.29:0/4023091907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcfc006b5f0 msgr2=0x7fcfc0071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 --2- 172.21.15.29:0/4023091907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcfc006b5f0 0x7fcfc0071fc0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0075730 tx=0x7fcfb801c9f0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 -- 172.21.15.29:0/4023091907 shutdown_connections 2024-04-25T00:36:25.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 --2- 172.21.15.29:0/4023091907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcfc006b5f0 0x7fcfc0071fc0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 --2- 172.21.15.29:0/4023091907 >> v2:172.21.15.157:3300/0 conn(0x7fcfc006b210 0x7fcfc006db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 -- 172.21.15.29:0/4023091907 >> 172.21.15.29:0/4023091907 conn(0x7fcfc00d0cb0 msgr2=0x7fcfc00d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:25.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 -- 172.21.15.29:0/4023091907 shutdown_connections 2024-04-25T00:36:25.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 -- 172.21.15.29:0/4023091907 wait complete. 2024-04-25T00:36:25.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 Processor -- start 2024-04-25T00:36:25.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 -- start start 2024-04-25T00:36:25.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfc006b210 0x7fcfc01534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfc006b5f0 0x7fcfc014f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcfc00d18f0 0x7fcfc00b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc016adb0 con 0x7fcfc00d18f0 2024-04-25T00:36:25.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0158b40 con 0x7fcfc006b210 2024-04-25T00:36:25.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc738c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc016af30 con 0x7fcfc006b5f0 2024-04-25T00:36:25.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc5929640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcfc00d18f0 0x7fcfc00b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc5929640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcfc00d18f0 0x7fcfc00b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48886/0 (socket says 172.21.15.29:48886) 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc5929640 1 -- 172.21.15.29:0/3486160137 learned_addr learned my addr 172.21.15.29:0/3486160137 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc5128640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfc006b210 0x7fcfc01534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc4927640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfc006b5f0 0x7fcfc014f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc5929640 1 -- 172.21.15.29:0/3486160137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfc006b5f0 msgr2=0x7fcfc014f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc5929640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfc006b5f0 0x7fcfc014f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc5929640 1 -- 172.21.15.29:0/3486160137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfc006b210 msgr2=0x7fcfc01534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc5929640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfc006b210 0x7fcfc01534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc5929640 1 -- 172.21.15.29:0/3486160137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc0153a00 con 0x7fcfc00d18f0 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc5128640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfc006b210 0x7fcfc01534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.868+0000 7fcfc4927640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfc006b5f0 0x7fcfc014f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.872+0000 7fcfc5929640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcfc00d18f0 0x7fcfc00b1af0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80af040 tx=0x7fcfb8019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.872+0000 7fcfb5ffb640 1 -- 172.21.15.29:0/3486160137 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80bc020 con 0x7fcfc00d18f0 2024-04-25T00:36:25.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.872+0000 7fcfb5ffb640 1 -- 172.21.15.29:0/3486160137 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb80b8670 con 0x7fcfc00d18f0 2024-04-25T00:36:25.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.872+0000 7fcfc738c640 1 -- 172.21.15.29:0/3486160137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc00afe20 con 0x7fcfc00d18f0 2024-04-25T00:36:25.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.872+0000 7fcfb5ffb640 1 -- 172.21.15.29:0/3486160137 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80b8a60 con 0x7fcfc00d18f0 2024-04-25T00:36:25.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.872+0000 7fcfc738c640 1 -- 172.21.15.29:0/3486160137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc00b03e0 con 0x7fcfc00d18f0 2024-04-25T00:36:25.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.872+0000 7fcfc738c640 1 -- 172.21.15.29:0/3486160137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc00b0bb0 con 0x7fcfc00d18f0 2024-04-25T00:36:25.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.872+0000 7fcfb5ffb640 1 -- 172.21.15.29:0/3486160137 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 44099+0+0 (secure 0 0 0) 0x7fcfb80bf020 con 0x7fcfc00d18f0 2024-04-25T00:36:25.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.872+0000 7fcfb5ffb640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcfa4035f50 0x7fcfa4038410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:25.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.872+0000 7fcfb5ffb640 1 -- 172.21.15.29:0/3486160137 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6326+0+0 (secure 0 0 0) 0x7fcfb80e4600 con 0x7fcfc00d18f0 2024-04-25T00:36:25.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.872+0000 7fcfc5128640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcfa4035f50 0x7fcfa4038410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:25.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.872+0000 7fcfc5128640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcfa4035f50 0x7fcfa4038410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00c74e0 tx=0x7fcfbc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:25.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:25.876+0000 7fcfb5ffb640 1 -- 172.21.15.29:0/3486160137 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcfb80c6030 con 0x7fcfc00d18f0 2024-04-25T00:36:26.023 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:26.020+0000 7fa1bebc0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:36:26.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.076+0000 7fcfc738c640 1 -- 172.21.15.29:0/3486160137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfc0069270 con 0x7fcfc00d18f0 2024-04-25T00:36:26.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.080+0000 7fcfb5ffb640 1 -- 172.21.15.29:0/3486160137 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7fcfb80165e0 con 0x7fcfc00d18f0 2024-04-25T00:36:26.081 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:26.081 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/524224365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":524224365},{"type":"v1","addr":"172.21.15.157:6835","nonce":524224365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1007000548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1007000548},{"type":"v1","addr":"172.21.15.157:6837","nonce":1007000548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6810,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2091414875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2091414875},{"type":"v1","addr":"172.21.15.29:6833","nonce":2091414875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:49.528297+0000","modified":"2024-04-25T00:35:52.569008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1111146974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1111146974},{"type":"v1","addr":"172.21.15.29:6835","nonce":1111146974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6822,"qdb_cluster":[6822]},"id":5}]} 2024-04-25T00:36:26.082 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 44 2024-04-25T00:36:26.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.080+0000 7fcfc738c640 1 -- 172.21.15.29:0/3486160137 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcfa4035f50 msgr2=0x7fcfa4038410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.080+0000 7fcfc738c640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcfa4035f50 0x7fcfa4038410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00c74e0 tx=0x7fcfbc008040 comp rx=0 tx=0).stop 2024-04-25T00:36:26.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.080+0000 7fcfc738c640 1 -- 172.21.15.29:0/3486160137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcfc00d18f0 msgr2=0x7fcfc00b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.080+0000 7fcfc738c640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcfc00d18f0 0x7fcfc00b1af0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80af040 tx=0x7fcfb8019790 comp rx=0 tx=0).stop 2024-04-25T00:36:26.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.080+0000 7fcfc738c640 1 -- 172.21.15.29:0/3486160137 shutdown_connections 2024-04-25T00:36:26.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.080+0000 7fcfc738c640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcfa4035f50 0x7fcfa4038410 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.080+0000 7fcfc738c640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcfc00d18f0 0x7fcfc00b1af0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.080+0000 7fcfc738c640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfc006b5f0 0x7fcfc014f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.080+0000 7fcfc738c640 1 --2- 172.21.15.29:0/3486160137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfc006b210 0x7fcfc01534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.080+0000 7fcfc738c640 1 -- 172.21.15.29:0/3486160137 >> 172.21.15.29:0/3486160137 conn(0x7fcfc00d0cb0 msgr2=0x7fcfc015c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:26.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.080+0000 7fcfc738c640 1 -- 172.21.15.29:0/3486160137 shutdown_connections 2024-04-25T00:36:26.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.080+0000 7fcfc738c640 1 -- 172.21.15.29:0/3486160137 wait complete. 2024-04-25T00:36:26.094 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:26.092+0000 7fa1bebc0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:36:26.101 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:36:26.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec41a9e640 1 Processor -- start 2024-04-25T00:36:26.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec41a9e640 1 -- start start 2024-04-25T00:36:26.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec41a9e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fec3c15e200 0x7fec3c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec41a9e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec3c0cfc10 con 0x7fec3c0d2c60 2024-04-25T00:36:26.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec41a9e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec3c0cfef0 con 0x7fec3c0cf690 2024-04-25T00:36:26.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec41a9e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fec3c0b38d0 con 0x7fec3c15e200 2024-04-25T00:36:26.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fec3c0d2c60 0x7fec3c0b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55106/0 (socket says 172.21.15.29:55106) 2024-04-25T00:36:26.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3bfff640 1 -- 172.21.15.29:0/3578961169 learned_addr learned my addr 172.21.15.29:0/3578961169 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3b7fe640 1 --2- 172.21.15.29:0/3578961169 >> v2:172.21.15.157:3300/0 conn(0x7fec3c15e200 0x7fec3c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2869048123 0 0) 0x7fec3c0cfc10 con 0x7fec3c0d2c60 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec28003530 con 0x7fec3c0d2c60 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2985610686 0 0) 0x7fec28003530 con 0x7fec3c0d2c60 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 >> v2:172.21.15.157:3300/0 conn(0x7fec3c15e200 msgr2=0x7fec3c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 --2- 172.21.15.29:0/3578961169 >> v2:172.21.15.157:3300/0 conn(0x7fec3c15e200 0x7fec3c0d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 >> v1:172.21.15.157:6789/0 conn(0x7fec3c0cf690 legacy=0x7fec3c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c0b3b20 con 0x7fec3c0d2c60 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fec34001a00 con 0x7fec3c0d2c60 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 >> v1:172.21.15.29:6789/0 conn(0x7fec3c0d2c60 legacy=0x7fec3c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 --2- 172.21.15.29:0/3578961169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec280039c0 0x7fec28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 --2- 172.21.15.29:0/3578961169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec28004810 0x7fec28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 --2- 172.21.15.29:0/3578961169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec280051d0 0x7fec2800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c0b38d0 con 0x7fec28004810 2024-04-25T00:36:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec2800c070 con 0x7fec280039c0 2024-04-25T00:36:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fec2800c270 con 0x7fec280051d0 2024-04-25T00:36:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3affd640 1 --2- 172.21.15.29:0/3578961169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec28004810 0x7fec28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3b7fe640 1 --2- 172.21.15.29:0/3578961169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec280039c0 0x7fec28003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3affd640 1 -- 172.21.15.29:0/3578961169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec280051d0 msgr2=0x7fec2800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3affd640 1 --2- 172.21.15.29:0/3578961169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec280051d0 0x7fec2800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3affd640 1 -- 172.21.15.29:0/3578961169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec280039c0 msgr2=0x7fec28003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3affd640 1 --2- 172.21.15.29:0/3578961169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec280039c0 0x7fec28003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3affd640 1 -- 172.21.15.29:0/3578961169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec28003530 con 0x7fec28004810 2024-04-25T00:36:26.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.224+0000 7fec3bfff640 1 -- 172.21.15.29:0/3578961169 reap_dead start 2024-04-25T00:36:26.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.228+0000 7fec3affd640 1 --2- 172.21.15.29:0/3578961169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec28004810 0x7fec28004c90 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fec2c001780 tx=0x7fec2c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94436dc0dca9e06a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:26.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.228+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0013a0 con 0x7fec28004810 2024-04-25T00:36:26.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.228+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2c0052f0 con 0x7fec28004810 2024-04-25T00:36:26.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.228+0000 7fec3a7fc640 1 -- 172.21.15.29:0/3578961169 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c003840 con 0x7fec28004810 2024-04-25T00:36:26.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.228+0000 7fec41a9e640 1 -- 172.21.15.29:0/3578961169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec28004810 msgr2=0x7fec28004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.228+0000 7fec41a9e640 1 --2- 172.21.15.29:0/3578961169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec28004810 0x7fec28004c90 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fec2c001780 tx=0x7fec2c01c400 comp rx=0 tx=0).stop 2024-04-25T00:36:26.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.228+0000 7fec41a9e640 1 -- 172.21.15.29:0/3578961169 shutdown_connections 2024-04-25T00:36:26.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.228+0000 7fec41a9e640 1 --2- 172.21.15.29:0/3578961169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec28004810 0x7fec28004c90 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.228+0000 7fec41a9e640 1 -- 172.21.15.29:0/3578961169 >> 172.21.15.29:0/3578961169 conn(0x7fec3c09d280 msgr2=0x7fec3c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:26.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.228+0000 7fec41a9e640 1 -- 172.21.15.29:0/3578961169 shutdown_connections 2024-04-25T00:36:26.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.228+0000 7fec41a9e640 1 -- 172.21.15.29:0/3578961169 wait complete. 2024-04-25T00:36:26.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.228+0000 7fec41a9e640 1 Processor -- start 2024-04-25T00:36:26.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec41a9e640 1 -- start start 2024-04-25T00:36:26.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec41a9e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec28004810 0x7fec3c0b9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec41a9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec3c1f58f0 0x7fec3c0b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec41a9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec3c1f6d30 0x7fec3c1f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec41a9e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c15e760 con 0x7fec3c1f6d30 2024-04-25T00:36:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec41a9e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c15e5e0 con 0x7fec3c1f58f0 2024-04-25T00:36:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec41a9e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fec3c0d2550 con 0x7fec28004810 2024-04-25T00:36:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec3c1f6d30 0x7fec3c1f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec3c1f6d30 0x7fec3c1f71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48906/0 (socket says 172.21.15.29:48906) 2024-04-25T00:36:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3bfff640 1 -- 172.21.15.29:0/4206795493 learned_addr learned my addr 172.21.15.29:0/4206795493 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3affd640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec3c1f58f0 0x7fec3c0b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3b7fe640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec28004810 0x7fec3c0b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3bfff640 1 -- 172.21.15.29:0/4206795493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec28004810 msgr2=0x7fec3c0b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3bfff640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec28004810 0x7fec3c0b9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3bfff640 1 -- 172.21.15.29:0/4206795493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec3c1f58f0 msgr2=0x7fec3c0b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3bfff640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec3c1f58f0 0x7fec3c0b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3bfff640 1 -- 172.21.15.29:0/4206795493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c0b9a10 con 0x7fec3c1f6d30 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3b7fe640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec28004810 0x7fec3c0b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3bfff640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec3c1f6d30 0x7fec3c1f71e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fec34001360 tx=0x7fec3400fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec33fff640 1 -- 172.21.15.29:0/4206795493 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3401b070 con 0x7fec3c1f6d30 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec33fff640 1 -- 172.21.15.29:0/4206795493 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3400d820 con 0x7fec3c1f6d30 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec41a9e640 1 -- 172.21.15.29:0/4206795493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3c15f240 con 0x7fec3c1f6d30 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec33fff640 1 -- 172.21.15.29:0/4206795493 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3400dbf0 con 0x7fec3c1f6d30 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec41a9e640 1 -- 172.21.15.29:0/4206795493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3c15f800 con 0x7fec3c1f6d30 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec41a9e640 1 -- 172.21.15.29:0/4206795493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3c069270 con 0x7fec3c1f6d30 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec33fff640 1 -- 172.21.15.29:0/4206795493 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 44099+0+0 (secure 0 0 0) 0x7fec3401e070 con 0x7fec3c1f6d30 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec33fff640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fec1c035f50 0x7fec1c038410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec33fff640 1 -- 172.21.15.29:0/4206795493 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6326+0+0 (secure 0 0 0) 0x7fec3404b7f0 con 0x7fec3c1f6d30 2024-04-25T00:36:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3b7fe640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fec1c035f50 0x7fec1c038410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.232+0000 7fec3b7fe640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fec1c035f50 0x7fec1c038410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fec3c1618e0 tx=0x7fec24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:26.238 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:26.236+0000 7fa1bebc0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:36:26.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.236+0000 7fec33fff640 1 -- 172.21.15.29:0/4206795493 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fec34013300 con 0x7fec3c1f6d30 2024-04-25T00:36:26.311 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:26.312+0000 7fa1bebc0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:36:26.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.392+0000 7fec41a9e640 1 -- 172.21.15.29:0/4206795493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec3c06c320 con 0x7fec3c1f6d30 2024-04-25T00:36:26.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.396+0000 7fec33fff640 1 -- 172.21.15.29:0/4206795493 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+17540 (secure 0 0 0) 0x7fec34014360 con 0x7fec3c1f6d30 2024-04-25T00:36:26.398 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:26.398 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":73,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:36:23.261760+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:47.559043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:48.530288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6832/3474584311":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/1408011519":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/4070029771":"2024-04-26T00:36:23.261666+0000","172.21.15.157:6833/3474584311":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3212538336":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/1922269872":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/37741218":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3796173968":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3438031706":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/1065963518":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/4165584909":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/2500505584":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3565775874":"2024-04-26T00:36:23.261666+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:36:26.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.396+0000 7fec41a9e640 1 -- 172.21.15.29:0/4206795493 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fec1c035f50 msgr2=0x7fec1c038410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.396+0000 7fec41a9e640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fec1c035f50 0x7fec1c038410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fec3c1618e0 tx=0x7fec24008040 comp rx=0 tx=0).stop 2024-04-25T00:36:26.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.396+0000 7fec41a9e640 1 -- 172.21.15.29:0/4206795493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec3c1f6d30 msgr2=0x7fec3c1f71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.396+0000 7fec41a9e640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec3c1f6d30 0x7fec3c1f71e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fec34001360 tx=0x7fec3400fc50 comp rx=0 tx=0).stop 2024-04-25T00:36:26.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.396+0000 7fec41a9e640 1 -- 172.21.15.29:0/4206795493 shutdown_connections 2024-04-25T00:36:26.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.396+0000 7fec41a9e640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fec1c035f50 0x7fec1c038410 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.396+0000 7fec41a9e640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec3c1f6d30 0x7fec3c1f71e0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.396+0000 7fec41a9e640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec3c1f58f0 0x7fec3c0b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.396+0000 7fec41a9e640 1 --2- 172.21.15.29:0/4206795493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec28004810 0x7fec3c0b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.396+0000 7fec41a9e640 1 -- 172.21.15.29:0/4206795493 >> 172.21.15.29:0/4206795493 conn(0x7fec3c09d280 msgr2=0x7fec3c06cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:26.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.396+0000 7fec41a9e640 1 -- 172.21.15.29:0/4206795493 shutdown_connections 2024-04-25T00:36:26.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.396+0000 7fec41a9e640 1 -- 172.21.15.29:0/4206795493 wait complete. 2024-04-25T00:36:26.416 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:36:26.416 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:36:26.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7c4b16640 1 Processor -- start 2024-04-25T00:36:26.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7c4b16640 1 -- start start 2024-04-25T00:36:26.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7c4b16640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c00b3140 0x7ff7c006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7c4b16640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff7c00b3c30 0x7ff7c00ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7c4b16640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7c00b41e0 con 0x7ff7c00b4b10 2024-04-25T00:36:26.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7c4b16640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff7c006d960 con 0x7ff7c00b3c30 2024-04-25T00:36:26.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7c4b16640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c006dc40 con 0x7ff7c00b3140 2024-04-25T00:36:26.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7bed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff7c00b3c30 0x7ff7c00ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7be59c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c00b3140 0x7ff7c006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7bdd9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff7c00b4b10 0x7ff7c00b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:52326/0 (socket says 172.21.15.29:52326) 2024-04-25T00:36:26.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7bdd9b640 1 -- 172.21.15.29:0/2141815351 learned_addr learned my addr 172.21.15.29:0/2141815351 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:26.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7bed9d640 1 -- 172.21.15.29:0/2141815351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c00b3140 msgr2=0x7ff7c006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:26.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7bed9d640 1 --2- 172.21.15.29:0/2141815351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c00b3140 0x7ff7c006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7bed9d640 1 -- 172.21.15.29:0/2141815351 >> v1:172.21.15.157:6789/0 conn(0x7ff7c00b4b10 legacy=0x7ff7c00b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.536+0000 7ff7bed9d640 1 -- 172.21.15.29:0/2141815351 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c006df20 con 0x7ff7c00b3c30 2024-04-25T00:36:26.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.540+0000 7ff7bed9d640 1 --2- 172.21.15.29:0/2141815351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c00b3c30 0x7ff7c00ce240 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80097c0 tx=0x7ff7b801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba3c22d29b5413 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:26.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.540+0000 7ff7bd59a640 1 -- 172.21.15.29:0/2141815351 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80b0070 con 0x7ff7c00b3c30 2024-04-25T00:36:26.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.540+0000 7ff7bd59a640 1 -- 172.21.15.29:0/2141815351 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b80ab600 con 0x7ff7c00b3c30 2024-04-25T00:36:26.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.540+0000 7ff7bd59a640 1 -- 172.21.15.29:0/2141815351 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80ab9d0 con 0x7ff7c00b3c30 2024-04-25T00:36:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.540+0000 7ff7c4b16640 1 -- 172.21.15.29:0/2141815351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c00b3c30 msgr2=0x7ff7c00ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.540+0000 7ff7c4b16640 1 --2- 172.21.15.29:0/2141815351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c00b3c30 0x7ff7c00ce240 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80097c0 tx=0x7ff7b801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.540+0000 7ff7c4b16640 1 -- 172.21.15.29:0/2141815351 shutdown_connections 2024-04-25T00:36:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.540+0000 7ff7c4b16640 1 --2- 172.21.15.29:0/2141815351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c00b3c30 0x7ff7c00ce240 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.540+0000 7ff7c4b16640 1 --2- 172.21.15.29:0/2141815351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c00b3140 0x7ff7c006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.540+0000 7ff7c4b16640 1 -- 172.21.15.29:0/2141815351 >> 172.21.15.29:0/2141815351 conn(0x7ff7c00b1870 msgr2=0x7ff7c00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.540+0000 7ff7c4b16640 1 -- 172.21.15.29:0/2141815351 shutdown_connections 2024-04-25T00:36:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.540+0000 7ff7c4b16640 1 -- 172.21.15.29:0/2141815351 wait complete. 2024-04-25T00:36:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7c4b16640 1 Processor -- start 2024-04-25T00:36:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7c4b16640 1 -- start start 2024-04-25T00:36:26.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7c4b16640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c00b3140 0x7ff7c015a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7c4b16640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7c00b3c30 0x7ff7c01404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7c4b16640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c00b4b10 0x7ff7c01f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7c4b16640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0168d80 con 0x7ff7c00b3c30 2024-04-25T00:36:26.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7c4b16640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0168c00 con 0x7ff7c00b4b10 2024-04-25T00:36:26.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7c4b16640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c0168f00 con 0x7ff7c00b3140 2024-04-25T00:36:26.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7bdd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7c00b3c30 0x7ff7c01404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7bdd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7c00b3c30 0x7ff7c01404c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48912/0 (socket says 172.21.15.29:48912) 2024-04-25T00:36:26.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7bdd9b640 1 -- 172.21.15.29:0/3857199906 learned_addr learned my addr 172.21.15.29:0/3857199906 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:26.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7bed9d640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c00b4b10 0x7ff7c01f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7be59c640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c00b3140 0x7ff7c015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7bdd9b640 1 -- 172.21.15.29:0/3857199906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c00b3140 msgr2=0x7ff7c015a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7bdd9b640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c00b3140 0x7ff7c015a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7bdd9b640 1 -- 172.21.15.29:0/3857199906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c00b4b10 msgr2=0x7ff7c01f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7bdd9b640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c00b4b10 0x7ff7c01f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7bdd9b640 1 -- 172.21.15.29:0/3857199906 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c015ac10 con 0x7ff7c00b3c30 2024-04-25T00:36:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7be59c640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c00b3140 0x7ff7c015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7bdd9b640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7c00b3c30 0x7ff7c01404c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400cb40 tx=0x7ff7b400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7aeffd640 1 -- 172.21.15.29:0/3857199906 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4017070 con 0x7ff7c00b3c30 2024-04-25T00:36:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7aeffd640 1 -- 172.21.15.29:0/3857199906 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b400aca0 con 0x7ff7c00b3c30 2024-04-25T00:36:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7aeffd640 1 -- 172.21.15.29:0/3857199906 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4012620 con 0x7ff7c00b3c30 2024-04-25T00:36:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7c4b16640 1 -- 172.21.15.29:0/3857199906 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c01f57f0 con 0x7ff7c00b3c30 2024-04-25T00:36:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7c4b16640 1 -- 172.21.15.29:0/3857199906 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c01f5db0 con 0x7ff7c00b3c30 2024-04-25T00:36:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7c4b16640 1 -- 172.21.15.29:0/3857199906 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c0069270 con 0x7ff7c00b3c30 2024-04-25T00:36:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7aeffd640 1 -- 172.21.15.29:0/3857199906 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 44099+0+0 (secure 0 0 0) 0x7ff7b401a070 con 0x7ff7c00b3c30 2024-04-25T00:36:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7aeffd640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff7a0035f80 0x7ff7a0038440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.544+0000 7ff7aeffd640 1 -- 172.21.15.29:0/3857199906 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6326+0+0 (secure 0 0 0) 0x7ff7b4048f90 con 0x7ff7c00b3c30 2024-04-25T00:36:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.548+0000 7ff7be59c640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff7a0035f80 0x7ff7a0038440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.548+0000 7ff7be59c640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff7a0035f80 0x7ff7a0038440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009b10 tx=0x7ff7b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:26.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.552+0000 7ff7aeffd640 1 -- 172.21.15.29:0/3857199906 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff7c015ac10 con 0x7ff7c00b3c30 2024-04-25T00:36:26.591 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:26.588+0000 7fa1bebc0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:36:26.644 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-25T00:36:26.644+0000 7fa1bebc0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:36:26.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.748+0000 7ff7c4b16640 1 -- 172.21.15.29:0/3857199906 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7c013dba0 con 0x7ff7c00b3c30 2024-04-25T00:36:26.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.748+0000 7ff7aeffd640 1 -- 172.21.15.29:0/3857199906 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7ff7b4007a00 con 0x7ff7c00b3c30 2024-04-25T00:36:26.752 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:26.752 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/524224365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":524224365},{"type":"v1","addr":"172.21.15.157:6835","nonce":524224365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1007000548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1007000548},{"type":"v1","addr":"172.21.15.157:6837","nonce":1007000548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6810,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2091414875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2091414875},{"type":"v1","addr":"172.21.15.29:6833","nonce":2091414875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:49.528297+0000","modified":"2024-04-25T00:35:52.569008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1111146974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1111146974},{"type":"v1","addr":"172.21.15.29:6835","nonce":1111146974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6822,"qdb_cluster":[6822]},"id":5}]} 2024-04-25T00:36:26.752 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 44 2024-04-25T00:36:26.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.752+0000 7ff7c4b16640 1 -- 172.21.15.29:0/3857199906 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff7a0035f80 msgr2=0x7ff7a0038440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.752+0000 7ff7c4b16640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff7a0035f80 0x7ff7a0038440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009b10 tx=0x7ff7b0008040 comp rx=0 tx=0).stop 2024-04-25T00:36:26.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.752+0000 7ff7c4b16640 1 -- 172.21.15.29:0/3857199906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7c00b3c30 msgr2=0x7ff7c01404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.752+0000 7ff7c4b16640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7c00b3c30 0x7ff7c01404c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400cb40 tx=0x7ff7b400a060 comp rx=0 tx=0).stop 2024-04-25T00:36:26.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.752+0000 7ff7c4b16640 1 -- 172.21.15.29:0/3857199906 shutdown_connections 2024-04-25T00:36:26.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.752+0000 7ff7c4b16640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff7a0035f80 0x7ff7a0038440 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.752+0000 7ff7c4b16640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c00b4b10 0x7ff7c01f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.752+0000 7ff7c4b16640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7c00b3c30 0x7ff7c01404c0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.752+0000 7ff7c4b16640 1 --2- 172.21.15.29:0/3857199906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c00b3140 0x7ff7c015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.752+0000 7ff7c4b16640 1 -- 172.21.15.29:0/3857199906 >> 172.21.15.29:0/3857199906 conn(0x7ff7c00b1870 msgr2=0x7ff7c013e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:26.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.752+0000 7ff7c4b16640 1 -- 172.21.15.29:0/3857199906 shutdown_connections 2024-04-25T00:36:26.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.752+0000 7ff7c4b16640 1 -- 172.21.15.29:0/3857199906 wait complete. 2024-04-25T00:36:26.770 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:36:26.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.880+0000 7fa519d41640 1 Processor -- start 2024-04-25T00:36:26.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.880+0000 7fa519d41640 1 -- start start 2024-04-25T00:36:26.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.880+0000 7fa519d41640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5140b3140 0x7fa51406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.880+0000 7fa519d41640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5140b4b10 0x7fa5140b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.880+0000 7fa519d41640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5140b41e0 con 0x7fa5140b3a60 2024-04-25T00:36:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.880+0000 7fa519d41640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa51406d960 con 0x7fa5140b3140 2024-04-25T00:36:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.880+0000 7fa519d41640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa51406dc40 con 0x7fa5140b4b10 2024-04-25T00:36:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.880+0000 7fa512ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5140b4b10 0x7fa5140b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.880+0000 7fa5137fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5140b3140 0x7fa51406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.880+0000 7fa513fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa5140b3a60 0x7fa51415c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:52336/0 (socket says 172.21.15.29:52336) 2024-04-25T00:36:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.880+0000 7fa513fff640 1 -- 172.21.15.29:0/3532325115 learned_addr learned my addr 172.21.15.29:0/3532325115 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa5137fe640 1 -- 172.21.15.29:0/3532325115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5140b4b10 msgr2=0x7fa5140b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa5137fe640 1 --2- 172.21.15.29:0/3532325115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5140b4b10 0x7fa5140b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa5137fe640 1 -- 172.21.15.29:0/3532325115 >> v1:172.21.15.157:6789/0 conn(0x7fa5140b3a60 legacy=0x7fa51415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa5137fe640 1 -- 172.21.15.29:0/3532325115 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51406df20 con 0x7fa5140b3140 2024-04-25T00:36:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa512ffd640 1 --2- 172.21.15.29:0/3532325115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5140b4b10 0x7fa5140b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:36:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa5137fe640 1 --2- 172.21.15.29:0/3532325115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5140b3140 0x7fa51406e440 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa5040097c0 tx=0x7fa50401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f0d58d56e51a1c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa5127fc640 1 -- 172.21.15.29:0/3532325115 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5040b0070 con 0x7fa5140b3140 2024-04-25T00:36:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa5127fc640 1 -- 172.21.15.29:0/3532325115 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5040ab600 con 0x7fa5140b3140 2024-04-25T00:36:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa5127fc640 1 -- 172.21.15.29:0/3532325115 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5040ab9d0 con 0x7fa5140b3140 2024-04-25T00:36:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa519d41640 1 -- 172.21.15.29:0/3532325115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5140b3140 msgr2=0x7fa51406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa519d41640 1 --2- 172.21.15.29:0/3532325115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5140b3140 0x7fa51406e440 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa5040097c0 tx=0x7fa50401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa519d41640 1 -- 172.21.15.29:0/3532325115 shutdown_connections 2024-04-25T00:36:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa519d41640 1 --2- 172.21.15.29:0/3532325115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5140b4b10 0x7fa5140b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa519d41640 1 --2- 172.21.15.29:0/3532325115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5140b3140 0x7fa51406e440 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa519d41640 1 -- 172.21.15.29:0/3532325115 >> 172.21.15.29:0/3532325115 conn(0x7fa5140b1870 msgr2=0x7fa5140b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa519d41640 1 -- 172.21.15.29:0/3532325115 shutdown_connections 2024-04-25T00:36:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.884+0000 7fa519d41640 1 -- 172.21.15.29:0/3532325115 wait complete. 2024-04-25T00:36:26.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa519d41640 1 Processor -- start 2024-04-25T00:36:26.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa519d41640 1 -- start start 2024-04-25T00:36:26.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa519d41640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5140b3140 0x7fa51413c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa519d41640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5140b3a60 0x7fa5141404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa519d41640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5140b4b10 0x7fa5141409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa519d41640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa514168b70 con 0x7fa5140b4b10 2024-04-25T00:36:26.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa519d41640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5141689f0 con 0x7fa5140b3140 2024-04-25T00:36:26.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa519d41640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa514168cf0 con 0x7fa5140b3a60 2024-04-25T00:36:26.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa513fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5140b4b10 0x7fa5141409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa513fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5140b4b10 0x7fa5141409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48920/0 (socket says 172.21.15.29:48920) 2024-04-25T00:36:26.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa513fff640 1 -- 172.21.15.29:0/3328425559 learned_addr learned my addr 172.21.15.29:0/3328425559 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:26.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa512ffd640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5140b3a60 0x7fa5141404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa5137fe640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5140b3140 0x7fa51413c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa513fff640 1 -- 172.21.15.29:0/3328425559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5140b3a60 msgr2=0x7fa5141404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa513fff640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5140b3a60 0x7fa5141404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa513fff640 1 -- 172.21.15.29:0/3328425559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5140b3140 msgr2=0x7fa51413c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:26.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa513fff640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5140b3140 0x7fa51413c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:26.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa513fff640 1 -- 172.21.15.29:0/3328425559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51413c830 con 0x7fa5140b4b10 2024-04-25T00:36:26.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa513fff640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5140b4b10 0x7fa5141409e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00cbc0 tx=0x7fa50c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:26.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa4fbfff640 1 -- 172.21.15.29:0/3328425559 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c017070 con 0x7fa5140b4b10 2024-04-25T00:36:26.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa4fbfff640 1 -- 172.21.15.29:0/3328425559 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50c00ace0 con 0x7fa5140b4b10 2024-04-25T00:36:26.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa4fbfff640 1 -- 172.21.15.29:0/3328425559 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c012600 con 0x7fa5140b4b10 2024-04-25T00:36:26.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa519d41640 1 -- 172.21.15.29:0/3328425559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51415b050 con 0x7fa5140b4b10 2024-04-25T00:36:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa519d41640 1 -- 172.21.15.29:0/3328425559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51415b610 con 0x7fa5140b4b10 2024-04-25T00:36:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa519d41640 1 -- 172.21.15.29:0/3328425559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa514069270 con 0x7fa5140b4b10 2024-04-25T00:36:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa4fbfff640 1 -- 172.21.15.29:0/3328425559 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 44099+0+0 (secure 0 0 0) 0x7fa50c01a070 con 0x7fa5140b4b10 2024-04-25T00:36:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa4fbfff640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa500035f50 0x7fa500038410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa4fbfff640 1 -- 172.21.15.29:0/3328425559 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6326+0+0 (secure 0 0 0) 0x7fa50c048f20 con 0x7fa5140b4b10 2024-04-25T00:36:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.888+0000 7fa5137fe640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa500035f50 0x7fa500038410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.892+0000 7fa5137fe640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa500035f50 0x7fa500038410 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa5040ad000 tx=0x7fa5040a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:26.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:26.896+0000 7fa4fbfff640 1 -- 172.21.15.29:0/3328425559 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa50c01a520 con 0x7fa5140b4b10 2024-04-25T00:36:27.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.084+0000 7fa519d41640 1 -- 172.21.15.29:0/3328425559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa51413d1e0 con 0x7fa5140b4b10 2024-04-25T00:36:27.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.084+0000 7fa4fbfff640 1 -- 172.21.15.29:0/3328425559 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7fa50c007870 con 0x7fa5140b4b10 2024-04-25T00:36:27.087 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:27.087 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/524224365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":524224365},{"type":"v1","addr":"172.21.15.157:6835","nonce":524224365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1007000548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1007000548},{"type":"v1","addr":"172.21.15.157:6837","nonce":1007000548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6810,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2091414875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2091414875},{"type":"v1","addr":"172.21.15.29:6833","nonce":2091414875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:35:49.528297+0000","modified":"2024-04-25T00:35:52.569008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1111146974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1111146974},{"type":"v1","addr":"172.21.15.29:6835","nonce":1111146974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6822,"qdb_cluster":[6822]},"id":5}]} 2024-04-25T00:36:27.087 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 44 2024-04-25T00:36:27.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.088+0000 7fa519d41640 1 -- 172.21.15.29:0/3328425559 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa500035f50 msgr2=0x7fa500038410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.088+0000 7fa519d41640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa500035f50 0x7fa500038410 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa5040ad000 tx=0x7fa5040a6000 comp rx=0 tx=0).stop 2024-04-25T00:36:27.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.088+0000 7fa519d41640 1 -- 172.21.15.29:0/3328425559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5140b4b10 msgr2=0x7fa5141409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.088+0000 7fa519d41640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5140b4b10 0x7fa5141409e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00cbc0 tx=0x7fa50c00a430 comp rx=0 tx=0).stop 2024-04-25T00:36:27.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.088+0000 7fa519d41640 1 -- 172.21.15.29:0/3328425559 shutdown_connections 2024-04-25T00:36:27.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.088+0000 7fa519d41640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa500035f50 0x7fa500038410 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.088+0000 7fa519d41640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5140b4b10 0x7fa5141409e0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.088+0000 7fa519d41640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5140b3a60 0x7fa5141404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.088+0000 7fa519d41640 1 --2- 172.21.15.29:0/3328425559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5140b3140 0x7fa51413c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.088+0000 7fa519d41640 1 -- 172.21.15.29:0/3328425559 >> 172.21.15.29:0/3328425559 conn(0x7fa5140b1870 msgr2=0x7fa51415e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:27.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.088+0000 7fa519d41640 1 -- 172.21.15.29:0/3328425559 shutdown_connections 2024-04-25T00:36:27.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.088+0000 7fa519d41640 1 -- 172.21.15.29:0/3328425559 wait complete. 2024-04-25T00:36:27.104 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:36:27.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.212+0000 7f7018305640 1 Processor -- start 2024-04-25T00:36:27.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f7018305640 1 -- start start 2024-04-25T00:36:27.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f7018305640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f70100b3140 0x7f701006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f7018305640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70100b4b10 0x7f70100b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f7018305640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70100b3a60 0x7f701015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f7018305640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f70100b41e0 con 0x7f70100b4b10 2024-04-25T00:36:27.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f7018305640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f701006d960 con 0x7f70100b3140 2024-04-25T00:36:27.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f7018305640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f701006dc40 con 0x7f70100b3a60 2024-04-25T00:36:27.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f70158a0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70100b4b10 0x7f70100b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f70158a0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70100b4b10 0x7f70100b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48932/0 (socket says 172.21.15.29:48932) 2024-04-25T00:36:27.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f70158a0640 1 -- 172.21.15.29:0/2962937261 learned_addr learned my addr 172.21.15.29:0/2962937261 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:27.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f70158a0640 1 -- 172.21.15.29:0/2962937261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70100b3a60 msgr2=0x7f701015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:27.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f70168a2640 1 --2- 172.21.15.29:0/2962937261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70100b3a60 0x7f701015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f70160a1640 1 --2- 172.21.15.29:0/2962937261 >> v2:172.21.15.157:3300/0 conn(0x7f70100b3140 0x7f701006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f70158a0640 1 --2- 172.21.15.29:0/2962937261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70100b3a60 0x7f701015c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f70158a0640 1 -- 172.21.15.29:0/2962937261 >> v2:172.21.15.157:3300/0 conn(0x7f70100b3140 msgr2=0x7f701006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:27.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f70158a0640 1 --2- 172.21.15.29:0/2962937261 >> v2:172.21.15.157:3300/0 conn(0x7f70100b3140 0x7f701006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f70158a0640 1 -- 172.21.15.29:0/2962937261 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701006df20 con 0x7f70100b4b10 2024-04-25T00:36:27.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f70158a0640 1 --2- 172.21.15.29:0/2962937261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70100b4b10 0x7f70100b3520 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7000009870 tx=0x7f700001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df24655ea0685fd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:27.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f701509f640 1 -- 172.21.15.29:0/2962937261 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70000b0070 con 0x7f70100b4b10 2024-04-25T00:36:27.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f701509f640 1 -- 172.21.15.29:0/2962937261 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70000ab600 con 0x7f70100b4b10 2024-04-25T00:36:27.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.216+0000 7f701509f640 1 -- 172.21.15.29:0/2962937261 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70000ab9d0 con 0x7f70100b4b10 2024-04-25T00:36:27.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 -- 172.21.15.29:0/2962937261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70100b4b10 msgr2=0x7f70100b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 --2- 172.21.15.29:0/2962937261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70100b4b10 0x7f70100b3520 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7000009870 tx=0x7f700001c520 comp rx=0 tx=0).stop 2024-04-25T00:36:27.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 -- 172.21.15.29:0/2962937261 shutdown_connections 2024-04-25T00:36:27.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 --2- 172.21.15.29:0/2962937261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70100b4b10 0x7f70100b3520 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 --2- 172.21.15.29:0/2962937261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70100b3a60 0x7f701015c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 --2- 172.21.15.29:0/2962937261 >> v2:172.21.15.157:3300/0 conn(0x7f70100b3140 0x7f701006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 -- 172.21.15.29:0/2962937261 >> 172.21.15.29:0/2962937261 conn(0x7f70100b1870 msgr2=0x7f70100b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:27.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 -- 172.21.15.29:0/2962937261 shutdown_connections 2024-04-25T00:36:27.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 -- 172.21.15.29:0/2962937261 wait complete. 2024-04-25T00:36:27.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 Processor -- start 2024-04-25T00:36:27.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 -- start start 2024-04-25T00:36:27.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70100b3140 0x7f7010135f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70100b3a60 0x7f7010142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70100b4b10 0x7f70101390c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f70100ce3c0 con 0x7f70100b3140 2024-04-25T00:36:27.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f70100ce240 con 0x7f70100b4b10 2024-04-25T00:36:27.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f70100ce540 con 0x7f70100b3a60 2024-04-25T00:36:27.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f70160a1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70100b3140 0x7f7010135f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f70160a1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70100b3140 0x7f7010135f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48938/0 (socket says 172.21.15.29:48938) 2024-04-25T00:36:27.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f70160a1640 1 -- 172.21.15.29:0/350254042 learned_addr learned my addr 172.21.15.29:0/350254042 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:27.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f70158a0640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70100b3a60 0x7f7010142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f70160a1640 1 -- 172.21.15.29:0/350254042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70100b3a60 msgr2=0x7f7010142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f70160a1640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70100b3a60 0x7f7010142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f70160a1640 1 -- 172.21.15.29:0/350254042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70100b4b10 msgr2=0x7f70101390c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f70160a1640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70100b4b10 0x7f70101390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f70160a1640 1 -- 172.21.15.29:0/350254042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7010143990 con 0x7f70100b3140 2024-04-25T00:36:27.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f70158a0640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70100b3a60 0x7f7010142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:27.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f70160a1640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70100b3140 0x7f7010135f00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f700c00c7f0 tx=0x7f700c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:27.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7006ffd640 1 -- 172.21.15.29:0/350254042 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c015070 con 0x7f70100b3140 2024-04-25T00:36:27.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7006ffd640 1 -- 172.21.15.29:0/350254042 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700c004220 con 0x7f70100b3140 2024-04-25T00:36:27.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7006ffd640 1 -- 172.21.15.29:0/350254042 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c003620 con 0x7f70100b3140 2024-04-25T00:36:27.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 -- 172.21.15.29:0/350254042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7010143c70 con 0x7f70100b3140 2024-04-25T00:36:27.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 -- 172.21.15.29:0/350254042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7010153550 con 0x7f70100b3140 2024-04-25T00:36:27.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.220+0000 7f7018305640 1 -- 172.21.15.29:0/350254042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7010069270 con 0x7f70100b3140 2024-04-25T00:36:27.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.224+0000 7f7006ffd640 1 -- 172.21.15.29:0/350254042 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 44099+0+0 (secure 0 0 0) 0x7f700c0039d0 con 0x7f70100b3140 2024-04-25T00:36:27.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.224+0000 7f7006ffd640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6ff4035f50 0x7f6ff4038410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.224+0000 7f7006ffd640 1 -- 172.21.15.29:0/350254042 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6326+0+0 (secure 0 0 0) 0x7f700c049490 con 0x7f70100b3140 2024-04-25T00:36:27.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.224+0000 7f70158a0640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6ff4035f50 0x7f6ff4038410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.224+0000 7f70158a0640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6ff4035f50 0x7f6ff4038410 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f70000ad000 tx=0x7f70000a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:27.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.228+0000 7f7006ffd640 1 -- 172.21.15.29:0/350254042 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f700c003d70 con 0x7f70100b3140 2024-04-25T00:36:27.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.304+0000 7f7006ffd640 1 -- 172.21.15.29:0/350254042 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f700c007620 con 0x7f70100b3140 2024-04-25T00:36:27.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.388+0000 7f7018305640 1 -- 172.21.15.29:0/350254042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70100753e0 con 0x7f70100b3140 2024-04-25T00:36:27.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.392+0000 7f7006ffd640 1 -- 172.21.15.29:0/350254042 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+17540 (secure 0 0 0) 0x7f700c05a3f0 con 0x7f70100b3140 2024-04-25T00:36:27.393 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:27.393 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":73,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:36:23.261760+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:35:47.559043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:35:48.530288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6832/3474584311":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/1408011519":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/4070029771":"2024-04-26T00:36:23.261666+0000","172.21.15.157:6833/3474584311":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3212538336":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/1922269872":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/37741218":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3796173968":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3438031706":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/1065963518":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/4165584909":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/2500505584":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3565775874":"2024-04-26T00:36:23.261666+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:36:27.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.392+0000 7f7018305640 1 -- 172.21.15.29:0/350254042 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6ff4035f50 msgr2=0x7f6ff4038410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.392+0000 7f7018305640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6ff4035f50 0x7f6ff4038410 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f70000ad000 tx=0x7f70000a6000 comp rx=0 tx=0).stop 2024-04-25T00:36:27.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.392+0000 7f7018305640 1 -- 172.21.15.29:0/350254042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70100b3140 msgr2=0x7f7010135f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.392+0000 7f7018305640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70100b3140 0x7f7010135f00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f700c00c7f0 tx=0x7f700c00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.392+0000 7f7018305640 1 -- 172.21.15.29:0/350254042 shutdown_connections 2024-04-25T00:36:27.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.392+0000 7f7018305640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6ff4035f50 0x7f6ff4038410 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.392+0000 7f7018305640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70100b4b10 0x7f70101390c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.392+0000 7f7018305640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70100b3a60 0x7f7010142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.392+0000 7f7018305640 1 --2- 172.21.15.29:0/350254042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70100b3140 0x7f7010135f00 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.392+0000 7f7018305640 1 -- 172.21.15.29:0/350254042 >> 172.21.15.29:0/350254042 conn(0x7f70100b1870 msgr2=0x7f7010140dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:27.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.392+0000 7f7018305640 1 -- 172.21.15.29:0/350254042 shutdown_connections 2024-04-25T00:36:27.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.392+0000 7f7018305640 1 -- 172.21.15.29:0/350254042 wait complete. 2024-04-25T00:36:27.412 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:36:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff56dabc640 1 Processor -- start 2024-04-25T00:36:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff56dabc640 1 -- start start 2024-04-25T00:36:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff56dabc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff56815e630 0x7ff5680cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff56dabc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5680c8690 con 0x7ff56815e250 2024-04-25T00:36:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff56dabc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5680c8970 con 0x7ff5680d2bb0 2024-04-25T00:36:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff56dabc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff5680c8440 con 0x7ff56815e630 2024-04-25T00:36:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff5677fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff56815e250 0x7ff5680d24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55120/0 (socket says 172.21.15.29:55120) 2024-04-25T00:36:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff5677fe640 1 -- 172.21.15.29:0/299463831 learned_addr learned my addr 172.21.15.29:0/299463831 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:27.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 393056375 0 0) 0x7ff5680c8690 con 0x7ff56815e250 2024-04-25T00:36:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff567fff640 1 --2- 172.21.15.29:0/299463831 >> v2:172.21.15.157:3300/0 conn(0x7ff56815e630 0x7ff5680cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff554003530 con 0x7ff56815e250 2024-04-25T00:36:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1283293355 0 0) 0x7ff554003530 con 0x7ff56815e250 2024-04-25T00:36:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 >> v2:172.21.15.157:3300/0 conn(0x7ff56815e630 msgr2=0x7ff5680cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff566ffd640 1 --2- 172.21.15.29:0/299463831 >> v2:172.21.15.157:3300/0 conn(0x7ff56815e630 0x7ff5680cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 >> v1:172.21.15.157:6789/0 conn(0x7ff5680d2bb0 legacy=0x7ff5680ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5680c8c50 con 0x7ff56815e250 2024-04-25T00:36:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ff5580016a0 con 0x7ff56815e250 2024-04-25T00:36:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 >> v1:172.21.15.29:6789/0 conn(0x7ff56815e250 legacy=0x7ff5680d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff566ffd640 1 --2- 172.21.15.29:0/299463831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff554003a10 0x7ff554003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff566ffd640 1 --2- 172.21.15.29:0/299463831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff554004810 0x7ff554004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff566ffd640 1 --2- 172.21.15.29:0/299463831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff554005220 0x7ff55400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.524+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff5680c8440 con 0x7ff554004810 2024-04-25T00:36:27.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff55400c170 con 0x7ff554005220 2024-04-25T00:36:27.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff55400c370 con 0x7ff554003a10 2024-04-25T00:36:27.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff55effd640 1 --2- 172.21.15.29:0/299463831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff554004810 0x7ff554004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff5677fe640 1 --2- 172.21.15.29:0/299463831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff554003a10 0x7ff554003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff567fff640 1 --2- 172.21.15.29:0/299463831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff554005220 0x7ff55400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff55effd640 1 -- 172.21.15.29:0/299463831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff554003a10 msgr2=0x7ff554003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:27.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff55effd640 1 --2- 172.21.15.29:0/299463831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff554003a10 0x7ff554003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff55effd640 1 -- 172.21.15.29:0/299463831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff554005220 msgr2=0x7ff55400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:27.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff55effd640 1 --2- 172.21.15.29:0/299463831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff554005220 0x7ff55400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff55effd640 1 -- 172.21.15.29:0/299463831 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff554003530 con 0x7ff554004810 2024-04-25T00:36:27.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff567fff640 1 -- 172.21.15.29:0/299463831 reap_dead start 2024-04-25T00:36:27.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff55effd640 1 --2- 172.21.15.29:0/299463831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff554004810 0x7ff554004c90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff550001750 tx=0x7ff55001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4308b9485756b22b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:27.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5500013a0 con 0x7ff554004810 2024-04-25T00:36:27.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5500052f0 con 0x7ff554004810 2024-04-25T00:36:27.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff566ffd640 1 -- 172.21.15.29:0/299463831 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550003840 con 0x7ff554004810 2024-04-25T00:36:27.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff56dabc640 1 -- 172.21.15.29:0/299463831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff554004810 msgr2=0x7ff554004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff56dabc640 1 --2- 172.21.15.29:0/299463831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff554004810 0x7ff554004c90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff550001750 tx=0x7ff55001c3c0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff56dabc640 1 -- 172.21.15.29:0/299463831 shutdown_connections 2024-04-25T00:36:27.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff56dabc640 1 --2- 172.21.15.29:0/299463831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff554004810 0x7ff554004c90 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff56dabc640 1 -- 172.21.15.29:0/299463831 >> 172.21.15.29:0/299463831 conn(0x7ff5680d3f70 msgr2=0x7ff5680d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:27.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff56dabc640 1 -- 172.21.15.29:0/299463831 shutdown_connections 2024-04-25T00:36:27.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.528+0000 7ff56dabc640 1 -- 172.21.15.29:0/299463831 wait complete. 2024-04-25T00:36:27.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff56dabc640 1 Processor -- start 2024-04-25T00:36:27.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff56dabc640 1 -- start start 2024-04-25T00:36:27.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff56dabc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff554004810 0x7ff5681fab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff56dabc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff56814aa90 0x7ff56814d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff56dabc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff56806ced0 0x7ff56806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff56dabc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff568074fc0 con 0x7ff56814aa90 2024-04-25T00:36:27.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff56dabc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff568074e40 con 0x7ff56806ced0 2024-04-25T00:36:27.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff56dabc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff5680d1cc0 con 0x7ff554004810 2024-04-25T00:36:27.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff55effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff56814aa90 0x7ff56814d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff55effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff56814aa90 0x7ff56814d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48968/0 (socket says 172.21.15.29:48968) 2024-04-25T00:36:27.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff55effd640 1 -- 172.21.15.29:0/1568709685 learned_addr learned my addr 172.21.15.29:0/1568709685 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:27.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff5677fe640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff554004810 0x7ff5681fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff567fff640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff56806ced0 0x7ff56806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff55effd640 1 -- 172.21.15.29:0/1568709685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff554004810 msgr2=0x7ff5681fab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff55effd640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff554004810 0x7ff5681fab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff55effd640 1 -- 172.21.15.29:0/1568709685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff56806ced0 msgr2=0x7ff56806d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff55effd640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff56806ced0 0x7ff56806d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff55effd640 1 -- 172.21.15.29:0/1568709685 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56814db10 con 0x7ff56814aa90 2024-04-25T00:36:27.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff5677fe640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff554004810 0x7ff5681fab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:27.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff55effd640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff56814aa90 0x7ff56814d2d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff550005040 tx=0x7ff554005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:27.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff564ff9640 1 -- 172.21.15.29:0/1568709685 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5500b8020 con 0x7ff56814aa90 2024-04-25T00:36:27.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff564ff9640 1 -- 172.21.15.29:0/1568709685 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5500054a0 con 0x7ff56814aa90 2024-04-25T00:36:27.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff56dabc640 1 -- 172.21.15.29:0/1568709685 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff56814ddf0 con 0x7ff56814aa90 2024-04-25T00:36:27.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff564ff9640 1 -- 172.21.15.29:0/1568709685 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550007a40 con 0x7ff56814aa90 2024-04-25T00:36:27.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff56dabc640 1 -- 172.21.15.29:0/1568709685 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff56814b4b0 con 0x7ff56814aa90 2024-04-25T00:36:27.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.532+0000 7ff56dabc640 1 -- 172.21.15.29:0/1568709685 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff56814b2a0 con 0x7ff56814aa90 2024-04-25T00:36:27.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.536+0000 7ff564ff9640 1 -- 172.21.15.29:0/1568709685 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7ff5500bb020 con 0x7ff56814aa90 2024-04-25T00:36:27.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.536+0000 7ff564ff9640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5480679e0 0x7ff548069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.536+0000 7ff564ff9640 1 -- 172.21.15.29:0/1568709685 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6326+0+0 (secure 0 0 0) 0x7ff5501204e0 con 0x7ff56814aa90 2024-04-25T00:36:27.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.536+0000 7ff5677fe640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5480679e0 0x7ff548069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.536+0000 7ff5677fe640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5480679e0 0x7ff548069ea0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff558000c10 tx=0x7ff558009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:27.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.540+0000 7ff564ff9640 1 -- 172.21.15.29:0/1568709685 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff5500c2030 con 0x7ff56814aa90 2024-04-25T00:36:27.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.736+0000 7ff56dabc640 1 -- 172.21.15.29:0/1568709685 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff568069270 con 0x7ff56814aa90 2024-04-25T00:36:27.738 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-04-25T00:36:27.736+0000 7f3ac00b5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:36:27.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.736+0000 7ff564ff9640 1 -- 172.21.15.29:0/1568709685 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7ff5500160d0 con 0x7ff56814aa90 2024-04-25T00:36:27.740 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:36:27.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.740+0000 7ff56dabc640 1 -- 172.21.15.29:0/1568709685 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5480679e0 msgr2=0x7ff548069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.740+0000 7ff56dabc640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5480679e0 0x7ff548069ea0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff558000c10 tx=0x7ff558009040 comp rx=0 tx=0).stop 2024-04-25T00:36:27.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.740+0000 7ff56dabc640 1 -- 172.21.15.29:0/1568709685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff56814aa90 msgr2=0x7ff56814d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.740+0000 7ff56dabc640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff56814aa90 0x7ff56814d2d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff550005040 tx=0x7ff554005840 comp rx=0 tx=0).stop 2024-04-25T00:36:27.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.740+0000 7ff56dabc640 1 -- 172.21.15.29:0/1568709685 shutdown_connections 2024-04-25T00:36:27.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.740+0000 7ff56dabc640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5480679e0 0x7ff548069ea0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.740+0000 7ff56dabc640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff56806ced0 0x7ff56806d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.740+0000 7ff56dabc640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff56814aa90 0x7ff56814d2d0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.740+0000 7ff56dabc640 1 --2- 172.21.15.29:0/1568709685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff554004810 0x7ff5681fab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.740+0000 7ff56dabc640 1 -- 172.21.15.29:0/1568709685 >> 172.21.15.29:0/1568709685 conn(0x7ff5680d3f70 msgr2=0x7ff5681e9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:27.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.740+0000 7ff56dabc640 1 -- 172.21.15.29:0/1568709685 shutdown_connections 2024-04-25T00:36:27.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.740+0000 7ff56dabc640 1 -- 172.21.15.29:0/1568709685 wait complete. 2024-04-25T00:36:27.752 INFO:tasks.ceph.mds.c.smithi029.stderr: -6835> 2024-04-25T00:35:55.136+0000 7ff4548bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:36:27.760 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:36:27.799 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:36:27.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.868+0000 7fcc3a7ee640 1 Processor -- start 2024-04-25T00:36:27.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.868+0000 7fcc3a7ee640 1 -- start start 2024-04-25T00:36:27.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc3a7ee640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcc3406dd10 0x7fcc340cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc3a7ee640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc340cb330 0x7fcc340ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc3a7ee640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc3406e1a0 con 0x7fcc3415bd60 2024-04-25T00:36:27.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc3a7ee640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcc3406edb0 con 0x7fcc3406dd10 2024-04-25T00:36:27.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc3a7ee640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3406e480 con 0x7fcc340cb330 2024-04-25T00:36:27.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc33fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcc3415bd60 0x7fcc3406b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:37184/0 (socket says 172.21.15.29:37184) 2024-04-25T00:36:27.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc33fff640 1 -- 172.21.15.29:0/1165557745 learned_addr learned my addr 172.21.15.29:0/1165557745 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:27.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc38d8b640 1 --2- 172.21.15.29:0/1165557745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc340cb330 0x7fcc340ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc337fe640 1 --2- 172.21.15.29:0/1165557745 >> v2:172.21.15.157:3300/0 conn(0x7fcc3406dd10 0x7fcc340cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc32ffd640 1 -- 172.21.15.29:0/1165557745 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3889910272 0 0) 0x7fcc3406e1a0 con 0x7fcc3415bd60 2024-04-25T00:36:27.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc32ffd640 1 -- 172.21.15.29:0/1165557745 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc1c003530 con 0x7fcc3415bd60 2024-04-25T00:36:27.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc38d8b640 1 -- 172.21.15.29:0/1165557745 >> v2:172.21.15.157:3300/0 conn(0x7fcc3406dd10 msgr2=0x7fcc340cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:27.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc38d8b640 1 --2- 172.21.15.29:0/1165557745 >> v2:172.21.15.157:3300/0 conn(0x7fcc3406dd10 0x7fcc340cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc38d8b640 1 -- 172.21.15.29:0/1165557745 >> v1:172.21.15.157:6789/0 conn(0x7fcc3415bd60 legacy=0x7fcc3406b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.872+0000 7fcc38d8b640 1 -- 172.21.15.29:0/1165557745 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3406e760 con 0x7fcc340cb330 2024-04-25T00:36:27.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc38d8b640 1 --2- 172.21.15.29:0/1165557745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc340cb330 0x7fcc340ccfc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009d90 tx=0x7fcc2c01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7683d84e39ce7401 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:27.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc32ffd640 1 -- 172.21.15.29:0/1165557745 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0b2070 con 0x7fcc340cb330 2024-04-25T00:36:27.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc32ffd640 1 -- 172.21.15.29:0/1165557745 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc2c0a5a20 con 0x7fcc340cb330 2024-04-25T00:36:27.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc32ffd640 1 -- 172.21.15.29:0/1165557745 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0a5dd0 con 0x7fcc340cb330 2024-04-25T00:36:27.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/1165557745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc340cb330 msgr2=0x7fcc340ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 --2- 172.21.15.29:0/1165557745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc340cb330 0x7fcc340ccfc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009d90 tx=0x7fcc2c01c940 comp rx=0 tx=0).stop 2024-04-25T00:36:27.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/1165557745 shutdown_connections 2024-04-25T00:36:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 --2- 172.21.15.29:0/1165557745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc340cb330 0x7fcc340ccfc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 --2- 172.21.15.29:0/1165557745 >> v2:172.21.15.157:3300/0 conn(0x7fcc3406dd10 0x7fcc340cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/1165557745 >> 172.21.15.29:0/1165557745 conn(0x7fcc3406c320 msgr2=0x7fcc3406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/1165557745 shutdown_connections 2024-04-25T00:36:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/1165557745 wait complete. 2024-04-25T00:36:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 Processor -- start 2024-04-25T00:36:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 -- start start 2024-04-25T00:36:27.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc340cb330 0x7fcc341560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3415bd60 0x7fcc340b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc34151cd0 0x7fcc34154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3406baa0 con 0x7fcc34151cd0 2024-04-25T00:36:27.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3406b920 con 0x7fcc340cb330 2024-04-25T00:36:27.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcc34072bc0 con 0x7fcc3415bd60 2024-04-25T00:36:27.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc38d8b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc34151cd0 0x7fcc34154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc38d8b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc34151cd0 0x7fcc34154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38128/0 (socket says 172.21.15.29:38128) 2024-04-25T00:36:27.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc38d8b640 1 -- 172.21.15.29:0/2010969699 learned_addr learned my addr 172.21.15.29:0/2010969699 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:27.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc337fe640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3415bd60 0x7fcc340b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc33fff640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc340cb330 0x7fcc341560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc38d8b640 1 -- 172.21.15.29:0/2010969699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3415bd60 msgr2=0x7fcc340b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc38d8b640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3415bd60 0x7fcc340b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc38d8b640 1 -- 172.21.15.29:0/2010969699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc340cb330 msgr2=0x7fcc341560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:27.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc38d8b640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc340cb330 0x7fcc341560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:27.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc38d8b640 1 -- 172.21.15.29:0/2010969699 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc34155030 con 0x7fcc34151cd0 2024-04-25T00:36:27.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc38d8b640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc34151cd0 0x7fcc34154730 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c01a570 tx=0x7fcc2c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:27.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc30ff9640 1 -- 172.21.15.29:0/2010969699 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0b2050 con 0x7fcc34151cd0 2024-04-25T00:36:27.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc30ff9640 1 -- 172.21.15.29:0/2010969699 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc2c0be070 con 0x7fcc34151cd0 2024-04-25T00:36:27.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/2010969699 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc34155310 con 0x7fcc34151cd0 2024-04-25T00:36:27.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc30ff9640 1 -- 172.21.15.29:0/2010969699 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c004040 con 0x7fcc34151cd0 2024-04-25T00:36:27.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.876+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/2010969699 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc341558d0 con 0x7fcc34151cd0 2024-04-25T00:36:27.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.880+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/2010969699 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc34069270 con 0x7fcc34151cd0 2024-04-25T00:36:27.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.880+0000 7fcc30ff9640 1 -- 172.21.15.29:0/2010969699 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fcc2c0041e0 con 0x7fcc34151cd0 2024-04-25T00:36:27.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.880+0000 7fcc30ff9640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcc200679e0 0x7fcc20069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:27.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.880+0000 7fcc30ff9640 1 -- 172.21.15.29:0/2010969699 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6412+0+0 (secure 0 0 0) 0x7fcc2c121390 con 0x7fcc34151cd0 2024-04-25T00:36:27.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.880+0000 7fcc33fff640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcc200679e0 0x7fcc20069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:27.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.880+0000 7fcc33fff640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcc200679e0 0x7fcc20069ea0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcc24005890 tx=0x7fcc24009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:27.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:27.884+0000 7fcc30ff9640 1 -- 172.21.15.29:0/2010969699 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcc2c016560 con 0x7fcc34151cd0 2024-04-25T00:36:28.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.076+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/2010969699 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcc340b5a30 con 0x7fcc34151cd0 2024-04-25T00:36:28.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.740+0000 7fcc30ff9640 1 -- 172.21.15.29:0/2010969699 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) ==== 104+0+0 (secure 0 0 0) 0x7fcc2c0f19d0 con 0x7fcc34151cd0 2024-04-25T00:36:28.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.744+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/2010969699 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcc200679e0 msgr2=0x7fcc20069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.744+0000 7fcc3a7ee640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcc200679e0 0x7fcc20069ea0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcc24005890 tx=0x7fcc24009040 comp rx=0 tx=0).stop 2024-04-25T00:36:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.744+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/2010969699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc34151cd0 msgr2=0x7fcc34154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.744+0000 7fcc3a7ee640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc34151cd0 0x7fcc34154730 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c01a570 tx=0x7fcc2c0196e0 comp rx=0 tx=0).stop 2024-04-25T00:36:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.744+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/2010969699 shutdown_connections 2024-04-25T00:36:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.744+0000 7fcc3a7ee640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcc200679e0 0x7fcc20069ea0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.744+0000 7fcc3a7ee640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc34151cd0 0x7fcc34154730 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.744+0000 7fcc3a7ee640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc3415bd60 0x7fcc340b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.744+0000 7fcc3a7ee640 1 --2- 172.21.15.29:0/2010969699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc340cb330 0x7fcc341560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.744+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/2010969699 >> 172.21.15.29:0/2010969699 conn(0x7fcc3406c320 msgr2=0x7fcc340cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.744+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/2010969699 shutdown_connections 2024-04-25T00:36:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.744+0000 7fcc3a7ee640 1 -- 172.21.15.29:0/2010969699 wait complete. 2024-04-25T00:36:28.769 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:36:28.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.944+0000 7f28f66fa640 1 Processor -- start 2024-04-25T00:36:28.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.944+0000 7f28f66fa640 1 -- start start 2024-04-25T00:36:28.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.944+0000 7f28f66fa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f28f00d09c0 0x7f28f00cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:28.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.944+0000 7f28f66fa640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28f00d46c0 0x7f28f00d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:28.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.944+0000 7f28f66fa640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f00d12e0 con 0x7f28f00d1e10 2024-04-25T00:36:28.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.944+0000 7f28f66fa640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f28f00b3910 con 0x7f28f00d46c0 2024-04-25T00:36:28.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.944+0000 7f28f66fa640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f28f00b41b0 con 0x7f28f00d09c0 2024-04-25T00:36:28.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28ef7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28f00d46c0 0x7f28f00d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:28.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28ef7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28f00d46c0 0x7f28f00d0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38134/0 (socket says 172.21.15.29:38134) 2024-04-25T00:36:28.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28ef7fe640 1 -- 172.21.15.29:0/168545108 learned_addr learned my addr 172.21.15.29:0/168545108 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:28.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28effff640 1 --2- 172.21.15.29:0/168545108 >> v2:172.21.15.157:3300/0 conn(0x7f28f00d09c0 0x7f28f00cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:28.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28ef7fe640 1 -- 172.21.15.29:0/168545108 >> v2:172.21.15.157:3300/0 conn(0x7f28f00d09c0 msgr2=0x7f28f00cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:28.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28ef7fe640 1 --2- 172.21.15.29:0/168545108 >> v2:172.21.15.157:3300/0 conn(0x7f28f00d09c0 0x7f28f00cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:28.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28ef7fe640 1 -- 172.21.15.29:0/168545108 >> v1:172.21.15.157:6789/0 conn(0x7f28f00d1e10 legacy=0x7f28f00d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:28.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28ef7fe640 1 -- 172.21.15.29:0/168545108 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f00cfcd0 con 0x7f28f00d46c0 2024-04-25T00:36:28.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28ef7fe640 1 --2- 172.21.15.29:0/168545108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00d46c0 0x7f28f00d0da0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f28f008c890 tx=0x7f28e401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cf6deb2b365108 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:28.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28eeffd640 1 -- 172.21.15.29:0/168545108 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40a57e0 con 0x7f28f00d46c0 2024-04-25T00:36:28.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28eeffd640 1 -- 172.21.15.29:0/168545108 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e40a5980 con 0x7f28f00d46c0 2024-04-25T00:36:28.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28eeffd640 1 -- 172.21.15.29:0/168545108 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40a5d50 con 0x7f28f00d46c0 2024-04-25T00:36:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28f66fa640 1 -- 172.21.15.29:0/168545108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00d46c0 msgr2=0x7f28f00d0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28f66fa640 1 --2- 172.21.15.29:0/168545108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00d46c0 0x7f28f00d0da0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f28f008c890 tx=0x7f28e401c920 comp rx=0 tx=0).stop 2024-04-25T00:36:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28f66fa640 1 -- 172.21.15.29:0/168545108 shutdown_connections 2024-04-25T00:36:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28f66fa640 1 --2- 172.21.15.29:0/168545108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00d46c0 0x7f28f00d0da0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28f66fa640 1 --2- 172.21.15.29:0/168545108 >> v2:172.21.15.157:3300/0 conn(0x7f28f00d09c0 0x7f28f00cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28f66fa640 1 -- 172.21.15.29:0/168545108 >> 172.21.15.29:0/168545108 conn(0x7f28f009d280 msgr2=0x7f28f00d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28f66fa640 1 -- 172.21.15.29:0/168545108 shutdown_connections 2024-04-25T00:36:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.948+0000 7f28f66fa640 1 -- 172.21.15.29:0/168545108 wait complete. 2024-04-25T00:36:28.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28f66fa640 1 Processor -- start 2024-04-25T00:36:28.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28f66fa640 1 -- start start 2024-04-25T00:36:28.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28f66fa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28f00d09c0 0x7f28f0152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:28.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28f66fa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00d1e10 0x7f28f015f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:28.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28f66fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f00d46c0 0x7f28f015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:28.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28f66fa640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f28f015e5d0 con 0x7f28f00d1e10 2024-04-25T00:36:28.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28f66fa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f28f015e450 con 0x7f28f00d46c0 2024-04-25T00:36:28.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28f66fa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f28f016b4e0 con 0x7f28f00d09c0 2024-04-25T00:36:28.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28ef7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00d1e10 0x7f28f015f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:28.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28ef7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00d1e10 0x7f28f015f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38138/0 (socket says 172.21.15.29:38138) 2024-04-25T00:36:28.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28ef7fe640 1 -- 172.21.15.29:0/1887920201 learned_addr learned my addr 172.21.15.29:0/1887920201 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:28.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28f4c97640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f00d46c0 0x7f28f015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:28.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28effff640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28f00d09c0 0x7f28f0152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:28.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28ef7fe640 1 -- 172.21.15.29:0/1887920201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28f00d09c0 msgr2=0x7f28f0152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:28.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28ef7fe640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28f00d09c0 0x7f28f0152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:28.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28ef7fe640 1 -- 172.21.15.29:0/1887920201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f00d46c0 msgr2=0x7f28f015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:28.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28ef7fe640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f00d46c0 0x7f28f015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:28.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28ef7fe640 1 -- 172.21.15.29:0/1887920201 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f01f8990 con 0x7f28f00d1e10 2024-04-25T00:36:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28ef7fe640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00d1e10 0x7f28f015f060 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f28e40af040 tx=0x7f28e40196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28ecff9640 1 -- 172.21.15.29:0/1887920201 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40bc020 con 0x7f28f00d1e10 2024-04-25T00:36:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28ecff9640 1 -- 172.21.15.29:0/1887920201 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e40b8670 con 0x7f28f00d1e10 2024-04-25T00:36:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28f66fa640 1 -- 172.21.15.29:0/1887920201 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f01f83d0 con 0x7f28f00d1e10 2024-04-25T00:36:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28ecff9640 1 -- 172.21.15.29:0/1887920201 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40b8a60 con 0x7f28f00d1e10 2024-04-25T00:36:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28f66fa640 1 -- 172.21.15.29:0/1887920201 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f015e860 con 0x7f28f00d1e10 2024-04-25T00:36:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.952+0000 7f28f66fa640 1 -- 172.21.15.29:0/1887920201 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f00692e0 con 0x7f28f00d1e10 2024-04-25T00:36:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.956+0000 7f28ecff9640 1 -- 172.21.15.29:0/1887920201 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f28e40bf020 con 0x7f28f00d1e10 2024-04-25T00:36:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.956+0000 7f28ecff9640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f28dc0679e0 0x7f28dc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.956+0000 7f28ecff9640 1 -- 172.21.15.29:0/1887920201 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6412+0+0 (secure 0 0 0) 0x7f28e4120fd0 con 0x7f28f00d1e10 2024-04-25T00:36:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.956+0000 7f28effff640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f28dc0679e0 0x7f28dc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.956+0000 7f28effff640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f28dc0679e0 0x7f28dc069ea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f28e00079c0 tx=0x7f28e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:28.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:28.960+0000 7f28ecff9640 1 -- 172.21.15.29:0/1887920201 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f28e40c6030 con 0x7f28f00d1e10 2024-04-25T00:36:29.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.124+0000 7f28f66fa640 1 -- 172.21.15.29:0/1887920201 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f28f00d07a0 con 0x7f28f00d1e10 2024-04-25T00:36:29.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.744+0000 7f28ecff9640 1 -- 172.21.15.29:0/1887920201 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v75) ==== 181+0+0 (secure 0 0 0) 0x7f28e40165a0 con 0x7f28f00d1e10 2024-04-25T00:36:29.745 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:36:29.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.744+0000 7f28f66fa640 1 -- 172.21.15.29:0/1887920201 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f28dc0679e0 msgr2=0x7f28dc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.744+0000 7f28f66fa640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f28dc0679e0 0x7f28dc069ea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f28e00079c0 tx=0x7f28e0008040 comp rx=0 tx=0).stop 2024-04-25T00:36:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.744+0000 7f28f66fa640 1 -- 172.21.15.29:0/1887920201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00d1e10 msgr2=0x7f28f015f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.744+0000 7f28f66fa640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00d1e10 0x7f28f015f060 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f28e40af040 tx=0x7f28e40196c0 comp rx=0 tx=0).stop 2024-04-25T00:36:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.744+0000 7f28f66fa640 1 -- 172.21.15.29:0/1887920201 shutdown_connections 2024-04-25T00:36:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.744+0000 7f28f66fa640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f28dc0679e0 0x7f28dc069ea0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.744+0000 7f28f66fa640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f00d46c0 0x7f28f015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.744+0000 7f28f66fa640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00d1e10 0x7f28f015f060 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.744+0000 7f28f66fa640 1 --2- 172.21.15.29:0/1887920201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28f00d09c0 0x7f28f0152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.748+0000 7f28f66fa640 1 -- 172.21.15.29:0/1887920201 >> 172.21.15.29:0/1887920201 conn(0x7f28f009d280 msgr2=0x7f28f00b4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:29.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.748+0000 7f28f66fa640 1 -- 172.21.15.29:0/1887920201 shutdown_connections 2024-04-25T00:36:29.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.748+0000 7f28f66fa640 1 -- 172.21.15.29:0/1887920201 wait complete. 2024-04-25T00:36:29.769 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:36:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.900+0000 7f4982204640 1 Processor -- start 2024-04-25T00:36:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.900+0000 7f4982204640 1 -- start start 2024-04-25T00:36:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.904+0000 7f4982204640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c0c4d40 0x7f497c0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.904+0000 7f4982204640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f497c0ccb30 0x7f497c0ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.904+0000 7f4982204640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c0cf6a0 con 0x7f497c0b4440 2024-04-25T00:36:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.904+0000 7f4982204640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f497c0b15b0 con 0x7f497c0ccb30 2024-04-25T00:36:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.904+0000 7f4982204640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f497c0cc3c0 con 0x7f497c0c4d40 2024-04-25T00:36:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.904+0000 7f497bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f497c0ccb30 0x7f497c0ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.904+0000 7f497affd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f497c0b4440 0x7f497c0c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37948/0 (socket says 172.21.15.29:37948) 2024-04-25T00:36:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.904+0000 7f497affd640 1 -- 172.21.15.29:0/2893065025 learned_addr learned my addr 172.21.15.29:0/2893065025 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:29.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.904+0000 7f497bfff640 1 -- 172.21.15.29:0/2893065025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c0c4d40 msgr2=0x7f497c0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:29.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.904+0000 7f497bfff640 1 --2- 172.21.15.29:0/2893065025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c0c4d40 0x7f497c0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:29.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.904+0000 7f497bfff640 1 -- 172.21.15.29:0/2893065025 >> v1:172.21.15.29:6789/0 conn(0x7f497c0b4440 legacy=0x7f497c0c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:29.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.904+0000 7f497bfff640 1 -- 172.21.15.29:0/2893065025 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c1629e0 con 0x7f497c0ccb30 2024-04-25T00:36:29.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.912+0000 7f497bfff640 1 --2- 172.21.15.29:0/2893065025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f497c0ccb30 0x7f497c0ccfc0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f496c009d90 tx=0x7f496c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f198680f5351ccef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:29.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.912+0000 7f497a7fc640 1 -- 172.21.15.29:0/2893065025 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c0b2070 con 0x7f497c0ccb30 2024-04-25T00:36:29.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.912+0000 7f497a7fc640 1 -- 172.21.15.29:0/2893065025 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f496c0a5760 con 0x7f497c0ccb30 2024-04-25T00:36:29.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.912+0000 7f497a7fc640 1 -- 172.21.15.29:0/2893065025 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c0a5b70 con 0x7f497c0ccb30 2024-04-25T00:36:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.912+0000 7f4982204640 1 -- 172.21.15.29:0/2893065025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f497c0ccb30 msgr2=0x7f497c0ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.912+0000 7f4982204640 1 --2- 172.21.15.29:0/2893065025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f497c0ccb30 0x7f497c0ccfc0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f496c009d90 tx=0x7f496c01c760 comp rx=0 tx=0).stop 2024-04-25T00:36:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.912+0000 7f4982204640 1 -- 172.21.15.29:0/2893065025 shutdown_connections 2024-04-25T00:36:29.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.912+0000 7f4982204640 1 --2- 172.21.15.29:0/2893065025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f497c0ccb30 0x7f497c0ccfc0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.912+0000 7f4982204640 1 --2- 172.21.15.29:0/2893065025 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c0c4d40 0x7f497c0cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.912+0000 7f4982204640 1 -- 172.21.15.29:0/2893065025 >> 172.21.15.29:0/2893065025 conn(0x7f497c0b3920 msgr2=0x7f497c0b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.912+0000 7f4982204640 1 -- 172.21.15.29:0/2893065025 shutdown_connections 2024-04-25T00:36:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.912+0000 7f4982204640 1 -- 172.21.15.29:0/2893065025 wait complete. 2024-04-25T00:36:29.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4982204640 1 Processor -- start 2024-04-25T00:36:29.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4982204640 1 -- start start 2024-04-25T00:36:29.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4982204640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f497c0b4440 0x7f497c0b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:29.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4982204640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c0c4d40 0x7f497c15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:29.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4982204640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c0ccb30 0x7f497c15a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:29.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4982204640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f497c16f450 con 0x7f497c0b4440 2024-04-25T00:36:29.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4982204640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f497c0cc610 con 0x7f497c0c4d40 2024-04-25T00:36:29.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4982204640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f497c16f5d0 con 0x7f497c0ccb30 2024-04-25T00:36:29.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f497b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f497c0b4440 0x7f497c0b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:29.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f497b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f497c0b4440 0x7f497c0b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38164/0 (socket says 172.21.15.29:38164) 2024-04-25T00:36:29.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f497b7fe640 1 -- 172.21.15.29:0/2318189039 learned_addr learned my addr 172.21.15.29:0/2318189039 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:29.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f497affd640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c0c4d40 0x7f497c15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:29.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f497bfff640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c0ccb30 0x7f497c15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:29.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f497b7fe640 1 -- 172.21.15.29:0/2318189039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c0ccb30 msgr2=0x7f497c15a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:29.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f497b7fe640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c0ccb30 0x7f497c15a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:29.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f497b7fe640 1 -- 172.21.15.29:0/2318189039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c0c4d40 msgr2=0x7f497c15bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:29.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f497b7fe640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c0c4d40 0x7f497c15bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:29.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f497b7fe640 1 -- 172.21.15.29:0/2318189039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c0b0280 con 0x7f497c0b4440 2024-04-25T00:36:29.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f497affd640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c0c4d40 0x7f497c15bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:29.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f497b7fe640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f497c0b4440 0x7f497c0b0a40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f497000b410 tx=0x7f497000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:29.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4963fff640 1 -- 172.21.15.29:0/2318189039 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970015020 con 0x7f497c0b4440 2024-04-25T00:36:29.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4963fff640 1 -- 172.21.15.29:0/2318189039 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4970011830 con 0x7f497c0b4440 2024-04-25T00:36:29.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4963fff640 1 -- 172.21.15.29:0/2318189039 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970011c20 con 0x7f497c0b4440 2024-04-25T00:36:29.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4982204640 1 -- 172.21.15.29:0/2318189039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497c0cfe70 con 0x7f497c0b4440 2024-04-25T00:36:29.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4982204640 1 -- 172.21.15.29:0/2318189039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497c0b0520 con 0x7f497c0b4440 2024-04-25T00:36:29.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4982204640 1 -- 172.21.15.29:0/2318189039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f497c0692e0 con 0x7f497c0b4440 2024-04-25T00:36:29.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.916+0000 7f4963fff640 1 -- 172.21.15.29:0/2318189039 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f4970018020 con 0x7f497c0b4440 2024-04-25T00:36:29.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.920+0000 7f4963fff640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49680679e0 0x7f4968069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:29.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.920+0000 7f497affd640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49680679e0 0x7f4968069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:29.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.920+0000 7f4963fff640 1 -- 172.21.15.29:0/2318189039 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5974+0+0 (secure 0 0 0) 0x7f4970085e20 con 0x7f497c0b4440 2024-04-25T00:36:29.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.920+0000 7f497affd640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49680679e0 0x7f4968069ea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f497c0b3880 tx=0x7f4974009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:29.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:29.924+0000 7f4963fff640 1 -- 172.21.15.29:0/2318189039 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f497001f020 con 0x7f497c0b4440 2024-04-25T00:36:30.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.084+0000 7f4982204640 1 -- 172.21.15.29:0/2318189039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f497c0b0760 con 0x7f497c0b4440 2024-04-25T00:36:30.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.752+0000 7f4963fff640 1 -- 172.21.15.29:0/2318189039 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v76) ==== 169+0+0 (secure 0 0 0) 0x7f49700172c0 con 0x7f497c0b4440 2024-04-25T00:36:30.754 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-04-25T00:36:30.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.756+0000 7f4982204640 1 -- 172.21.15.29:0/2318189039 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49680679e0 msgr2=0x7f4968069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:30.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.756+0000 7f4982204640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49680679e0 0x7f4968069ea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f497c0b3880 tx=0x7f4974009040 comp rx=0 tx=0).stop 2024-04-25T00:36:30.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.756+0000 7f4982204640 1 -- 172.21.15.29:0/2318189039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f497c0b4440 msgr2=0x7f497c0b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:30.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.756+0000 7f4982204640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f497c0b4440 0x7f497c0b0a40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f497000b410 tx=0x7f497000b8d0 comp rx=0 tx=0).stop 2024-04-25T00:36:30.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.756+0000 7f4982204640 1 -- 172.21.15.29:0/2318189039 shutdown_connections 2024-04-25T00:36:30.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.756+0000 7f4982204640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49680679e0 0x7f4968069ea0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:30.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.756+0000 7f4982204640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f497c0ccb30 0x7f497c15a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:30.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.756+0000 7f4982204640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f497c0c4d40 0x7f497c15bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:30.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.756+0000 7f4982204640 1 --2- 172.21.15.29:0/2318189039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f497c0b4440 0x7f497c0b0a40 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:30.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.756+0000 7f4982204640 1 -- 172.21.15.29:0/2318189039 >> 172.21.15.29:0/2318189039 conn(0x7f497c0b3920 msgr2=0x7f497c0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:30.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.760+0000 7f4982204640 1 -- 172.21.15.29:0/2318189039 shutdown_connections 2024-04-25T00:36:30.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.760+0000 7f4982204640 1 -- 172.21.15.29:0/2318189039 wait complete. 2024-04-25T00:36:30.786 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:36:30.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.908+0000 7f61d1424640 1 Processor -- start 2024-04-25T00:36:30.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.908+0000 7f61d1424640 1 -- start start 2024-04-25T00:36:30.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61d1424640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61cc0cd350 0x7f61cc0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:30.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61d1424640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f61cc0b50f0 0x7f61cc15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:30.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61d1424640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61cc0cdaf0 con 0x7f61cc15bff0 2024-04-25T00:36:30.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61d1424640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f61cc0cf930 con 0x7f61cc0b50f0 2024-04-25T00:36:30.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61d1424640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f61cc0d0d70 con 0x7f61cc0cd350 2024-04-25T00:36:30.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61cb7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f61cc0b50f0 0x7f61cc15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:30.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61cb7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f61cc0b50f0 0x7f61cc15fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38172/0 (socket says 172.21.15.29:38172) 2024-04-25T00:36:30.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61cb7fe640 1 -- 172.21.15.29:0/634641565 learned_addr learned my addr 172.21.15.29:0/634641565 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:30.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61ca7fc640 1 --2- 172.21.15.29:0/634641565 >> v2:172.21.15.157:3300/0 conn(0x7f61cc0cd350 0x7f61cc0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:30.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61cb7fe640 1 -- 172.21.15.29:0/634641565 >> v2:172.21.15.157:3300/0 conn(0x7f61cc0cd350 msgr2=0x7f61cc0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:30.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61cb7fe640 1 --2- 172.21.15.29:0/634641565 >> v2:172.21.15.157:3300/0 conn(0x7f61cc0cd350 0x7f61cc0b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:30.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61cb7fe640 1 -- 172.21.15.29:0/634641565 >> v1:172.21.15.157:6789/0 conn(0x7f61cc15bff0 legacy=0x7f61cc0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:30.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61cb7fe640 1 -- 172.21.15.29:0/634641565 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc0cfb80 con 0x7f61cc0b50f0 2024-04-25T00:36:30.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61cb7fe640 1 --2- 172.21.15.29:0/634641565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61cc0b50f0 0x7f61cc15fb40 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0b4800 tx=0x7f61c401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ce876815001fa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:30.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61c9ffb640 1 -- 172.21.15.29:0/634641565 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c40a5840 con 0x7f61cc0b50f0 2024-04-25T00:36:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61c9ffb640 1 -- 172.21.15.29:0/634641565 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61c40a59e0 con 0x7f61cc0b50f0 2024-04-25T00:36:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61c9ffb640 1 -- 172.21.15.29:0/634641565 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c40a5db0 con 0x7f61cc0b50f0 2024-04-25T00:36:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61d1424640 1 -- 172.21.15.29:0/634641565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61cc0b50f0 msgr2=0x7f61cc15fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61d1424640 1 --2- 172.21.15.29:0/634641565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61cc0b50f0 0x7f61cc15fb40 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0b4800 tx=0x7f61c401c980 comp rx=0 tx=0).stop 2024-04-25T00:36:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61d1424640 1 -- 172.21.15.29:0/634641565 shutdown_connections 2024-04-25T00:36:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61d1424640 1 --2- 172.21.15.29:0/634641565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61cc0b50f0 0x7f61cc15fb40 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61d1424640 1 --2- 172.21.15.29:0/634641565 >> v2:172.21.15.157:3300/0 conn(0x7f61cc0cd350 0x7f61cc0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61d1424640 1 -- 172.21.15.29:0/634641565 >> 172.21.15.29:0/634641565 conn(0x7f61cc0b38b0 msgr2=0x7f61cc0b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61d1424640 1 -- 172.21.15.29:0/634641565 shutdown_connections 2024-04-25T00:36:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.912+0000 7f61d1424640 1 -- 172.21.15.29:0/634641565 wait complete. 2024-04-25T00:36:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61d1424640 1 Processor -- start 2024-04-25T00:36:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61d1424640 1 -- start start 2024-04-25T00:36:30.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61d1424640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61cc0b50f0 0x7f61cc0c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:30.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61d1424640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61cc0cd350 0x7f61cc0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:30.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61d1424640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61cc15bff0 0x7f61cc0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:30.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61d1424640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc0cd730 con 0x7f61cc0b50f0 2024-04-25T00:36:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61d1424640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc0b5540 con 0x7f61cc15bff0 2024-04-25T00:36:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61d1424640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f61cc0cd8b0 con 0x7f61cc0cd350 2024-04-25T00:36:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61caffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61cc0b50f0 0x7f61cc0c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61caffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61cc0b50f0 0x7f61cc0c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38184/0 (socket says 172.21.15.29:38184) 2024-04-25T00:36:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61caffd640 1 -- 172.21.15.29:0/3578895733 learned_addr learned my addr 172.21.15.29:0/3578895733 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61cb7fe640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61cc15bff0 0x7f61cc0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61ca7fc640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61cc0cd350 0x7f61cc0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61caffd640 1 -- 172.21.15.29:0/3578895733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61cc0cd350 msgr2=0x7f61cc0cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61caffd640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61cc0cd350 0x7f61cc0cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61caffd640 1 -- 172.21.15.29:0/3578895733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61cc15bff0 msgr2=0x7f61cc0cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61caffd640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61cc15bff0 0x7f61cc0cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61caffd640 1 -- 172.21.15.29:0/3578895733 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc0c9880 con 0x7f61cc0b50f0 2024-04-25T00:36:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61cb7fe640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61cc15bff0 0x7f61cc0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61caffd640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61cc0b50f0 0x7f61cc0c8e30 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c490 tx=0x7f61b4001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61c37fe640 1 -- 172.21.15.29:0/3578895733 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4017070 con 0x7f61cc0b50f0 2024-04-25T00:36:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61c37fe640 1 -- 172.21.15.29:0/3578895733 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b4003600 con 0x7f61cc0b50f0 2024-04-25T00:36:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61d1424640 1 -- 172.21.15.29:0/3578895733 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61cc0c9b60 con 0x7f61cc0b50f0 2024-04-25T00:36:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61c37fe640 1 -- 172.21.15.29:0/3578895733 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4003a30 con 0x7f61cc0b50f0 2024-04-25T00:36:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61d1424640 1 -- 172.21.15.29:0/3578895733 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61cc150200 con 0x7f61cc0b50f0 2024-04-25T00:36:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.916+0000 7f61d1424640 1 -- 172.21.15.29:0/3578895733 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61cc069270 con 0x7f61cc0b50f0 2024-04-25T00:36:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.920+0000 7f61c37fe640 1 -- 172.21.15.29:0/3578895733 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f61b401a070 con 0x7f61cc0b50f0 2024-04-25T00:36:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.920+0000 7f61c37fe640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f61ac0679e0 0x7f61ac069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.920+0000 7f61c37fe640 1 -- 172.21.15.29:0/3578895733 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7f61b4085190 con 0x7f61cc0b50f0 2024-04-25T00:36:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.920+0000 7f61ca7fc640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f61ac0679e0 0x7f61ac069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:30.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.920+0000 7f61ca7fc640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f61ac0679e0 0x7f61ac069ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61cc15b530 tx=0x7f61bc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:30.924+0000 7f61c37fe640 1 -- 172.21.15.29:0/3578895733 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f61b4055c90 con 0x7f61cc0b50f0 2024-04-25T00:36:31.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.104+0000 7f61d1424640 1 -- 172.21.15.29:0/3578895733 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f61cc0b7ce0 con 0x7f61cc0b50f0 2024-04-25T00:36:31.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.112+0000 7f61c37fe640 1 -- 172.21.15.29:0/3578895733 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f61b4055590 con 0x7f61cc0b50f0 2024-04-25T00:36:31.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.112+0000 7f61d1424640 1 -- 172.21.15.29:0/3578895733 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f61ac0679e0 msgr2=0x7f61ac069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:31.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.112+0000 7f61d1424640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f61ac0679e0 0x7f61ac069ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61cc15b530 tx=0x7f61bc008040 comp rx=0 tx=0).stop 2024-04-25T00:36:31.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.112+0000 7f61d1424640 1 -- 172.21.15.29:0/3578895733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61cc0b50f0 msgr2=0x7f61cc0c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:31.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.112+0000 7f61d1424640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61cc0b50f0 0x7f61cc0c8e30 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c490 tx=0x7f61b4001e80 comp rx=0 tx=0).stop 2024-04-25T00:36:31.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.112+0000 7f61d1424640 1 -- 172.21.15.29:0/3578895733 shutdown_connections 2024-04-25T00:36:31.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.112+0000 7f61d1424640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f61ac0679e0 0x7f61ac069ea0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.112+0000 7f61d1424640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61cc15bff0 0x7f61cc0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.112+0000 7f61d1424640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61cc0cd350 0x7f61cc0cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.112+0000 7f61d1424640 1 --2- 172.21.15.29:0/3578895733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61cc0b50f0 0x7f61cc0c8e30 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.112+0000 7f61d1424640 1 -- 172.21.15.29:0/3578895733 >> 172.21.15.29:0/3578895733 conn(0x7f61cc0b38b0 msgr2=0x7f61cc066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:31.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.112+0000 7f61d1424640 1 -- 172.21.15.29:0/3578895733 shutdown_connections 2024-04-25T00:36:31.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.112+0000 7f61d1424640 1 -- 172.21.15.29:0/3578895733 wait complete. 2024-04-25T00:36:31.132 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-04-25T00:36:31.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.252+0000 7f2040cbf640 1 Processor -- start 2024-04-25T00:36:31.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.252+0000 7f2040cbf640 1 -- start start 2024-04-25T00:36:31.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2040cbf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f203c0b3140 0x7f203c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2040cbf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f203c0b4b10 0x7f203c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2040cbf640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f203c0b41e0 con 0x7f203c0b3a60 2024-04-25T00:36:31.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2040cbf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f203c06d960 con 0x7f203c0b4b10 2024-04-25T00:36:31.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2040cbf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f203c06dc40 con 0x7f203c0b3140 2024-04-25T00:36:31.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f203ad9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f203c0b3a60 0x7f203c15c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37956/0 (socket says 172.21.15.29:37956) 2024-04-25T00:36:31.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f203ad9d640 1 -- 172.21.15.29:0/2623466149 learned_addr learned my addr 172.21.15.29:0/2623466149 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:31.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f203a59c640 1 --2- 172.21.15.29:0/2623466149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f203c0b3140 0x7f203c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2033fff640 1 --2- 172.21.15.29:0/2623466149 >> v2:172.21.15.157:3300/0 conn(0x7f203c0b4b10 0x7f203c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 389632003 0 0) 0x7f203c0b41e0 con 0x7f203c0b3a60 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2028003530 con 0x7f203c0b3a60 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 259578457 0 0) 0x7f2028003530 con 0x7f203c0b3a60 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f203c0b3140 msgr2=0x7f203c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 --2- 172.21.15.29:0/2623466149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f203c0b3140 0x7f203c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 >> v2:172.21.15.157:3300/0 conn(0x7f203c0b4b10 msgr2=0x7f203c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 --2- 172.21.15.29:0/2623466149 >> v2:172.21.15.157:3300/0 conn(0x7f203c0b4b10 0x7f203c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203c06df20 con 0x7f203c0b3a60 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2034001a00 con 0x7f203c0b3a60 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 >> v1:172.21.15.29:6789/0 conn(0x7f203c0b3a60 legacy=0x7f203c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 --2- 172.21.15.29:0/2623466149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20280039c0 0x7f2028003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 --2- 172.21.15.29:0/2623466149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2028004810 0x7f2028004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 --2- 172.21.15.29:0/2623466149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f203c0b3140 0x7f20280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f203c06d960 con 0x7f20280039c0 2024-04-25T00:36:31.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f203c06dc40 con 0x7f2028004810 2024-04-25T00:36:31.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f202800b9d0 con 0x7f203c0b3140 2024-04-25T00:36:31.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f203a59c640 1 --2- 172.21.15.29:0/2623466149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20280039c0 0x7f2028003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2033fff640 1 --2- 172.21.15.29:0/2623466149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2028004810 0x7f2028004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f203ad9d640 1 --2- 172.21.15.29:0/2623466149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f203c0b3140 0x7f20280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f203a59c640 1 -- 172.21.15.29:0/2623466149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f203c0b3140 msgr2=0x7f20280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:31.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f203a59c640 1 --2- 172.21.15.29:0/2623466149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f203c0b3140 0x7f20280051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f203a59c640 1 -- 172.21.15.29:0/2623466149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2028004810 msgr2=0x7f2028004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:31.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f203a59c640 1 --2- 172.21.15.29:0/2623466149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2028004810 0x7f2028004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f203a59c640 1 -- 172.21.15.29:0/2623466149 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2028003530 con 0x7f20280039c0 2024-04-25T00:36:31.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.256+0000 7f2033fff640 1 --2- 172.21.15.29:0/2623466149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2028004810 0x7f2028004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:31.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.260+0000 7f203a59c640 1 --2- 172.21.15.29:0/2623466149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20280039c0 0x7f2028003de0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f202c009ae0 tx=0x7f202c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de11107e6c62f457 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:31.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.260+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c0096b0 con 0x7f20280039c0 2024-04-25T00:36:31.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.260+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202c0af070 con 0x7f20280039c0 2024-04-25T00:36:31.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.260+0000 7f2039d9b640 1 -- 172.21.15.29:0/2623466149 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c0aa640 con 0x7f20280039c0 2024-04-25T00:36:31.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.260+0000 7f2040cbf640 1 -- 172.21.15.29:0/2623466149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20280039c0 msgr2=0x7f2028003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:31.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.260+0000 7f2040cbf640 1 --2- 172.21.15.29:0/2623466149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20280039c0 0x7f2028003de0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f202c009ae0 tx=0x7f202c01ca70 comp rx=0 tx=0).stop 2024-04-25T00:36:31.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.260+0000 7f203ad9d640 1 -- 172.21.15.29:0/2623466149 reap_dead start 2024-04-25T00:36:31.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.260+0000 7f2040cbf640 1 -- 172.21.15.29:0/2623466149 shutdown_connections 2024-04-25T00:36:31.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.260+0000 7f2040cbf640 1 -- 172.21.15.29:0/2623466149 >> 172.21.15.29:0/2623466149 conn(0x7f203c0b1870 msgr2=0x7f203c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:31.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.260+0000 7f2040cbf640 1 -- 172.21.15.29:0/2623466149 shutdown_connections 2024-04-25T00:36:31.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.260+0000 7f2040cbf640 1 -- 172.21.15.29:0/2623466149 wait complete. 2024-04-25T00:36:31.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.260+0000 7f2040cbf640 1 Processor -- start 2024-04-25T00:36:31.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.260+0000 7f2040cbf640 1 -- start start 2024-04-25T00:36:31.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2040cbf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20280039c0 0x7f203c15e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2040cbf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f203c15b360 0x7f203c15b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2040cbf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c162bf0 0x7f203c163070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2040cbf640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f203c166b60 con 0x7f20280039c0 2024-04-25T00:36:31.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2040cbf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f203c1669e0 con 0x7f203c162bf0 2024-04-25T00:36:31.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2040cbf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f203c166ce0 con 0x7f203c15b360 2024-04-25T00:36:31.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f203a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20280039c0 0x7f203c15e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f203a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20280039c0 0x7f203c15e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38208/0 (socket says 172.21.15.29:38208) 2024-04-25T00:36:31.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f203a59c640 1 -- 172.21.15.29:0/3207633616 learned_addr learned my addr 172.21.15.29:0/3207633616 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:31.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2033fff640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f203c15b360 0x7f203c15b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f203ad9d640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c162bf0 0x7f203c163070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f203a59c640 1 -- 172.21.15.29:0/3207633616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f203c15b360 msgr2=0x7f203c15b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:31.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f203a59c640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f203c15b360 0x7f203c15b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f203a59c640 1 -- 172.21.15.29:0/3207633616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c162bf0 msgr2=0x7f203c163070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:31.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f203a59c640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c162bf0 0x7f203c163070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f203a59c640 1 -- 172.21.15.29:0/3207633616 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203c1645e0 con 0x7f20280039c0 2024-04-25T00:36:31.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f203ad9d640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c162bf0 0x7f203c163070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:31.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f203a59c640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20280039c0 0x7f203c15e470 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f202c01c8a0 tx=0x7f2028005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:31.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2032ffd640 1 -- 172.21.15.29:0/3207633616 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c00f000 con 0x7f20280039c0 2024-04-25T00:36:31.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2032ffd640 1 -- 172.21.15.29:0/3207633616 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202c0aa620 con 0x7f20280039c0 2024-04-25T00:36:31.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2040cbf640 1 -- 172.21.15.29:0/3207633616 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203c163960 con 0x7f20280039c0 2024-04-25T00:36:31.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2032ffd640 1 -- 172.21.15.29:0/3207633616 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c0aaa10 con 0x7f20280039c0 2024-04-25T00:36:31.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2040cbf640 1 -- 172.21.15.29:0/3207633616 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f203c163f20 con 0x7f20280039c0 2024-04-25T00:36:31.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2040cbf640 1 -- 172.21.15.29:0/3207633616 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203c069270 con 0x7f20280039c0 2024-04-25T00:36:31.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2032ffd640 1 -- 172.21.15.29:0/3207633616 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f202c0a9070 con 0x7f20280039c0 2024-04-25T00:36:31.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2032ffd640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f201c067a10 0x7f201c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2032ffd640 1 -- 172.21.15.29:0/3207633616 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7f202c1202a0 con 0x7f20280039c0 2024-04-25T00:36:31.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2033fff640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f201c067a10 0x7f201c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.264+0000 7f2033fff640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f201c067a10 0x7f201c069ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f203c072720 tx=0x7f2024008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:31.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.268+0000 7f2032ffd640 1 -- 172.21.15.29:0/3207633616 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f202c016780 con 0x7f20280039c0 2024-04-25T00:36:31.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.492+0000 7f2040cbf640 1 -- 172.21.15.29:0/3207633616 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7f203c141230 con 0x7f20280039c0 2024-04-25T00:36:31.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.752+0000 7f2032ffd640 1 -- 172.21.15.29:0/3207633616 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v270) ==== 131+0+0 (secure 0 0 0) 0x7f202c0f0da0 con 0x7f20280039c0 2024-04-25T00:36:31.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.756+0000 7f2040cbf640 1 -- 172.21.15.29:0/3207633616 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f201c067a10 msgr2=0x7f201c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:31.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.756+0000 7f2040cbf640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f201c067a10 0x7f201c069ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f203c072720 tx=0x7f2024008040 comp rx=0 tx=0).stop 2024-04-25T00:36:31.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.756+0000 7f2040cbf640 1 -- 172.21.15.29:0/3207633616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20280039c0 msgr2=0x7f203c15e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:31.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.756+0000 7f2040cbf640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20280039c0 0x7f203c15e470 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f202c01c8a0 tx=0x7f2028005840 comp rx=0 tx=0).stop 2024-04-25T00:36:31.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.756+0000 7f2040cbf640 1 -- 172.21.15.29:0/3207633616 shutdown_connections 2024-04-25T00:36:31.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.756+0000 7f2040cbf640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f201c067a10 0x7f201c069ed0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.756+0000 7f2040cbf640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c162bf0 0x7f203c163070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.756+0000 7f2040cbf640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f203c15b360 0x7f203c15b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.756+0000 7f2040cbf640 1 --2- 172.21.15.29:0/3207633616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20280039c0 0x7f203c15e470 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.756+0000 7f2040cbf640 1 -- 172.21.15.29:0/3207633616 >> 172.21.15.29:0/3207633616 conn(0x7f203c0b1870 msgr2=0x7f203c144600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:31.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.756+0000 7f2040cbf640 1 -- 172.21.15.29:0/3207633616 shutdown_connections 2024-04-25T00:36:31.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.756+0000 7f2040cbf640 1 -- 172.21.15.29:0/3207633616 wait complete. 2024-04-25T00:36:31.779 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-04-25T00:36:31.779 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2024-04-25T00:36:31.779 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2024-04-25T00:36:31.780 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:36:31.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.904+0000 7fd079cf4640 1 Processor -- start 2024-04-25T00:36:31.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.904+0000 7fd079cf4640 1 -- start start 2024-04-25T00:36:31.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd079cf4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd07415df70 0x7fd0740b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd079cf4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd07406d210 0x7fd0740b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd079cf4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0740b4b60 0x7fd074070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd079cf4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd0740b4fd0 con 0x7fd07406d210 2024-04-25T00:36:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd079cf4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd07406d980 con 0x7fd07415df70 2024-04-25T00:36:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd079cf4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd07406dc60 con 0x7fd0740b4b60 2024-04-25T00:36:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd072ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd07406d210 0x7fd0740b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd072ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd07406d210 0x7fd0740b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38220/0 (socket says 172.21.15.29:38220) 2024-04-25T00:36:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd072ffd640 1 -- 172.21.15.29:0/2283063758 learned_addr learned my addr 172.21.15.29:0/2283063758 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd0737fe640 1 --2- 172.21.15.29:0/2283063758 >> v2:172.21.15.157:3300/0 conn(0x7fd07415df70 0x7fd0740b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd073fff640 1 --2- 172.21.15.29:0/2283063758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0740b4b60 0x7fd074070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd072ffd640 1 -- 172.21.15.29:0/2283063758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0740b4b60 msgr2=0x7fd074070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd072ffd640 1 --2- 172.21.15.29:0/2283063758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0740b4b60 0x7fd074070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd072ffd640 1 -- 172.21.15.29:0/2283063758 >> v2:172.21.15.157:3300/0 conn(0x7fd07415df70 msgr2=0x7fd0740b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd072ffd640 1 --2- 172.21.15.29:0/2283063758 >> v2:172.21.15.157:3300/0 conn(0x7fd07415df70 0x7fd0740b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd072ffd640 1 -- 172.21.15.29:0/2283063758 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07406df40 con 0x7fd07406d210 2024-04-25T00:36:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd073fff640 1 --2- 172.21.15.29:0/2283063758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0740b4b60 0x7fd074070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd072ffd640 1 --2- 172.21.15.29:0/2283063758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07406d210 0x7fd0740b4620 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd0740d0bf0 tx=0x7fd06401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966c88d50566b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:31.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd0727fc640 1 -- 172.21.15.29:0/2283063758 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0640b0070 con 0x7fd07406d210 2024-04-25T00:36:31.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd0727fc640 1 -- 172.21.15.29:0/2283063758 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0640ab600 con 0x7fd07406d210 2024-04-25T00:36:31.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.908+0000 7fd0727fc640 1 -- 172.21.15.29:0/2283063758 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0640ab9d0 con 0x7fd07406d210 2024-04-25T00:36:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 -- 172.21.15.29:0/2283063758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07406d210 msgr2=0x7fd0740b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 --2- 172.21.15.29:0/2283063758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07406d210 0x7fd0740b4620 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd0740d0bf0 tx=0x7fd06401c230 comp rx=0 tx=0).stop 2024-04-25T00:36:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 -- 172.21.15.29:0/2283063758 shutdown_connections 2024-04-25T00:36:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 --2- 172.21.15.29:0/2283063758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07406d210 0x7fd0740b4620 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 --2- 172.21.15.29:0/2283063758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0740b4b60 0x7fd074070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 --2- 172.21.15.29:0/2283063758 >> v2:172.21.15.157:3300/0 conn(0x7fd07415df70 0x7fd0740b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 -- 172.21.15.29:0/2283063758 >> 172.21.15.29:0/2283063758 conn(0x7fd0740b1870 msgr2=0x7fd0740b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 -- 172.21.15.29:0/2283063758 shutdown_connections 2024-04-25T00:36:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 -- 172.21.15.29:0/2283063758 wait complete. 2024-04-25T00:36:31.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 Processor -- start 2024-04-25T00:36:31.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 -- start start 2024-04-25T00:36:31.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07406d210 0x7fd07414f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0740b4b60 0x7fd07415ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd07415df70 0x7fd074155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd07406ea20 con 0x7fd07406d210 2024-04-25T00:36:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd07415e350 con 0x7fd0740b4b60 2024-04-25T00:36:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd07406eba0 con 0x7fd07415df70 2024-04-25T00:36:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd0737fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07406d210 0x7fd07414f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd0737fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07406d210 0x7fd07414f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38230/0 (socket says 172.21.15.29:38230) 2024-04-25T00:36:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd0737fe640 1 -- 172.21.15.29:0/929800463 learned_addr learned my addr 172.21.15.29:0/929800463 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd072ffd640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0740b4b60 0x7fd07415ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd073fff640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd07415df70 0x7fd074155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd0737fe640 1 -- 172.21.15.29:0/929800463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd07415df70 msgr2=0x7fd074155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd0737fe640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd07415df70 0x7fd074155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd0737fe640 1 -- 172.21.15.29:0/929800463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0740b4b60 msgr2=0x7fd07415ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:31.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd0737fe640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0740b4b60 0x7fd07415ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:31.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd0737fe640 1 -- 172.21.15.29:0/929800463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0740ceb80 con 0x7fd07406d210 2024-04-25T00:36:31.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd072ffd640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0740b4b60 0x7fd07415ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:31.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd0737fe640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07406d210 0x7fd07414f990 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fd06800d5f0 tx=0x7fd06800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:31.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd05bfff640 1 -- 172.21.15.29:0/929800463 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0680125f0 con 0x7fd07406d210 2024-04-25T00:36:31.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd05bfff640 1 -- 172.21.15.29:0/929800463 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd068019070 con 0x7fd07406d210 2024-04-25T00:36:31.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 -- 172.21.15.29:0/929800463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0740ce5c0 con 0x7fd07406d210 2024-04-25T00:36:31.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd05bfff640 1 -- 172.21.15.29:0/929800463 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0680129b0 con 0x7fd07406d210 2024-04-25T00:36:31.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.912+0000 7fd079cf4640 1 -- 172.21.15.29:0/929800463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07415a350 con 0x7fd07406d210 2024-04-25T00:36:31.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.916+0000 7fd079cf4640 1 -- 172.21.15.29:0/929800463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd074069270 con 0x7fd07406d210 2024-04-25T00:36:31.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.916+0000 7fd05bfff640 1 -- 172.21.15.29:0/929800463 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fd068012b50 con 0x7fd07406d210 2024-04-25T00:36:31.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.916+0000 7fd05bfff640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd0600679e0 0x7fd060069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:31.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.916+0000 7fd05bfff640 1 -- 172.21.15.29:0/929800463 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7fd068086080 con 0x7fd07406d210 2024-04-25T00:36:31.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.916+0000 7fd072ffd640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd0600679e0 0x7fd060069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:31.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.916+0000 7fd072ffd640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd0600679e0 0x7fd060069ea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd0640ad000 tx=0x7fd064008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:31.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:31.920+0000 7fd05bfff640 1 -- 172.21.15.29:0/929800463 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd068056c10 con 0x7fd07406d210 2024-04-25T00:36:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.084+0000 7fd079cf4640 1 -- 172.21.15.29:0/929800463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0740d0840 con 0x7fd07406d210 2024-04-25T00:36:32.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.084+0000 7fd05bfff640 1 -- 172.21.15.29:0/929800463 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+13796 (secure 0 0 0) 0x7fd068056510 con 0x7fd07406d210 2024-04-25T00:36:32.087 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:32.087 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:36:30.746945+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6834/1111146974":"2024-04-26T00:36:27.739342+0000","172.21.15.157:0/3565775874":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/4165584909":"2024-04-26T00:36:23.261666+0000","172.21.15.29:6835/1111146974":"2024-04-26T00:36:27.739342+0000","172.21.15.157:0/1065963518":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3438031706":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3796173968":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/37741218":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/3212538336":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/1922269872":"2024-04-26T00:36:23.261666+0000","172.21.15.157:6833/3474584311":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/2500505584":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/4070029771":"2024-04-26T00:36:23.261666+0000","172.21.15.157:0/1408011519":"2024-04-26T00:36:23.261666+0000","172.21.15.157:6832/3474584311":"2024-04-26T00:36:23.261666+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:36:32.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.088+0000 7fd079cf4640 1 -- 172.21.15.29:0/929800463 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd0600679e0 msgr2=0x7fd060069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:32.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.088+0000 7fd079cf4640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd0600679e0 0x7fd060069ea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd0640ad000 tx=0x7fd064008040 comp rx=0 tx=0).stop 2024-04-25T00:36:32.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.088+0000 7fd079cf4640 1 -- 172.21.15.29:0/929800463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07406d210 msgr2=0x7fd07414f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:32.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.088+0000 7fd079cf4640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07406d210 0x7fd07414f990 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fd06800d5f0 tx=0x7fd06800dab0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.088+0000 7fd079cf4640 1 -- 172.21.15.29:0/929800463 shutdown_connections 2024-04-25T00:36:32.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.088+0000 7fd079cf4640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd0600679e0 0x7fd060069ea0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.088+0000 7fd079cf4640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd07415df70 0x7fd074155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.088+0000 7fd079cf4640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0740b4b60 0x7fd07415ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.088+0000 7fd079cf4640 1 --2- 172.21.15.29:0/929800463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07406d210 0x7fd07414f990 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.088+0000 7fd079cf4640 1 -- 172.21.15.29:0/929800463 >> 172.21.15.29:0/929800463 conn(0x7fd0740b1870 msgr2=0x7fd0740ae2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:32.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.088+0000 7fd079cf4640 1 -- 172.21.15.29:0/929800463 shutdown_connections 2024-04-25T00:36:32.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.088+0000 7fd079cf4640 1 -- 172.21.15.29:0/929800463 wait complete. 2024-04-25T00:36:32.106 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:36:32.106 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:36:32.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.224+0000 7f592ca0b640 1 Processor -- start 2024-04-25T00:36:32.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.224+0000 7f592ca0b640 1 -- start start 2024-04-25T00:36:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.228+0000 7f592ca0b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f592815e8d0 0x7f59280cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.228+0000 7f592ca0b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59280b4780 0x7f592815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.228+0000 7f592ca0b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59280b3720 con 0x7f592815f1f0 2024-04-25T00:36:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.228+0000 7f592ca0b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f59280b42f0 con 0x7f592815e8d0 2024-04-25T00:36:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.228+0000 7f592ca0b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f592815d7e0 con 0x7f59280b4780 2024-04-25T00:36:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.228+0000 7f592659c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f592815e8d0 0x7f59280cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:32.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.228+0000 7f592659c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f592815e8d0 0x7f59280cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38240/0 (socket says 172.21.15.29:38240) 2024-04-25T00:36:32.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.228+0000 7f592659c640 1 -- 172.21.15.29:0/4077954250 learned_addr learned my addr 172.21.15.29:0/4077954250 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:32.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.228+0000 7f5925d9b640 1 --2- 172.21.15.29:0/4077954250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59280b4780 0x7f592815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:32.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.228+0000 7f592659c640 1 -- 172.21.15.29:0/4077954250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59280b4780 msgr2=0x7f592815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:32.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.228+0000 7f592659c640 1 --2- 172.21.15.29:0/4077954250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59280b4780 0x7f592815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.228+0000 7f592659c640 1 -- 172.21.15.29:0/4077954250 >> v1:172.21.15.157:6789/0 conn(0x7f592815f1f0 legacy=0x7f592815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:32.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.228+0000 7f592659c640 1 -- 172.21.15.29:0/4077954250 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59280b39c0 con 0x7f592815e8d0 2024-04-25T00:36:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592659c640 1 --2- 172.21.15.29:0/4077954250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592815e8d0 0x7f59280cc240 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5918009870 tx=0x7f591801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f19eba26112f816 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592559a640 1 -- 172.21.15.29:0/4077954250 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592815c1c0 con 0x7f592815e8d0 2024-04-25T00:36:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592559a640 1 -- 172.21.15.29:0/4077954250 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59180a95f0 con 0x7f592815e8d0 2024-04-25T00:36:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592559a640 1 -- 172.21.15.29:0/4077954250 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59180a99c0 con 0x7f592815e8d0 2024-04-25T00:36:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 -- 172.21.15.29:0/4077954250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592815e8d0 msgr2=0x7f59280cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 --2- 172.21.15.29:0/4077954250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592815e8d0 0x7f59280cc240 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5918009870 tx=0x7f591801c530 comp rx=0 tx=0).stop 2024-04-25T00:36:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 -- 172.21.15.29:0/4077954250 shutdown_connections 2024-04-25T00:36:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 --2- 172.21.15.29:0/4077954250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59280b4780 0x7f592815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 --2- 172.21.15.29:0/4077954250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592815e8d0 0x7f59280cc240 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 -- 172.21.15.29:0/4077954250 >> 172.21.15.29:0/4077954250 conn(0x7f59280b1870 msgr2=0x7f59280b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 -- 172.21.15.29:0/4077954250 shutdown_connections 2024-04-25T00:36:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 -- 172.21.15.29:0/4077954250 wait complete. 2024-04-25T00:36:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 Processor -- start 2024-04-25T00:36:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 -- start start 2024-04-25T00:36:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59280b4780 0x7f5928140f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592815e8d0 0x7f59280adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592815f1f0 0x7f592813c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5928144a20 con 0x7f592815f1f0 2024-04-25T00:36:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f592815f600 con 0x7f592815e8d0 2024-04-25T00:36:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592ca0b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f59280cd250 con 0x7f59280b4780 2024-04-25T00:36:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f5926d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592815f1f0 0x7f592813c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f5926d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592815f1f0 0x7f592813c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38242/0 (socket says 172.21.15.29:38242) 2024-04-25T00:36:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f5926d9d640 1 -- 172.21.15.29:0/4177787260 learned_addr learned my addr 172.21.15.29:0/4177787260 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f592659c640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59280b4780 0x7f5928140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f5925d9b640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592815e8d0 0x7f59280adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f5926d9d640 1 -- 172.21.15.29:0/4177787260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59280b4780 msgr2=0x7f5928140f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f5926d9d640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59280b4780 0x7f5928140f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f5926d9d640 1 -- 172.21.15.29:0/4177787260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592815e8d0 msgr2=0x7f59280adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f5926d9d640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592815e8d0 0x7f59280adee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.232+0000 7f5926d9d640 1 -- 172.21.15.29:0/4177787260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59280cd4f0 con 0x7f592815f1f0 2024-04-25T00:36:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.236+0000 7f5926d9d640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592815f1f0 0x7f592813c0f0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f592000c730 tx=0x7f592000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.236+0000 7f5916ffd640 1 -- 172.21.15.29:0/4177787260 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920016020 con 0x7f592815f1f0 2024-04-25T00:36:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.236+0000 7f5916ffd640 1 -- 172.21.15.29:0/4177787260 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5920006d60 con 0x7f592815f1f0 2024-04-25T00:36:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.236+0000 7f592ca0b640 1 -- 172.21.15.29:0/4177787260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59280cd7d0 con 0x7f592815f1f0 2024-04-25T00:36:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.236+0000 7f5916ffd640 1 -- 172.21.15.29:0/4177787260 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920003bc0 con 0x7f592815f1f0 2024-04-25T00:36:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.236+0000 7f592ca0b640 1 -- 172.21.15.29:0/4177787260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59280ce070 con 0x7f592815f1f0 2024-04-25T00:36:32.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.236+0000 7f592ca0b640 1 -- 172.21.15.29:0/4177787260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5928069270 con 0x7f592815f1f0 2024-04-25T00:36:32.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.236+0000 7f5916ffd640 1 -- 172.21.15.29:0/4177787260 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f59200050d0 con 0x7f592815f1f0 2024-04-25T00:36:32.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.236+0000 7f5916ffd640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5908067a10 0x7f5908069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:32.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.236+0000 7f5916ffd640 1 -- 172.21.15.29:0/4177787260 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7f59200858d0 con 0x7f592815f1f0 2024-04-25T00:36:32.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.236+0000 7f592659c640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5908067a10 0x7f5908069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:32.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.236+0000 7f592659c640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5908067a10 0x7f5908069ed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f59180ae000 tx=0x7f59180b3040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:32.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.240+0000 7f5916ffd640 1 -- 172.21.15.29:0/4177787260 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5920020020 con 0x7f592815f1f0 2024-04-25T00:36:32.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f592ca0b640 1 -- 172.21.15.29:0/4177787260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f592806e7b0 con 0x7f592815f1f0 2024-04-25T00:36:32.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f5916ffd640 1 -- 172.21.15.29:0/4177787260 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f5920056460 con 0x7f592815f1f0 2024-04-25T00:36:32.405 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:36:32.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f592ca0b640 1 -- 172.21.15.29:0/4177787260 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5908067a10 msgr2=0x7f5908069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:32.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f592ca0b640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5908067a10 0x7f5908069ed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f59180ae000 tx=0x7f59180b3040 comp rx=0 tx=0).stop 2024-04-25T00:36:32.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f592ca0b640 1 -- 172.21.15.29:0/4177787260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592815f1f0 msgr2=0x7f592813c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:32.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f592ca0b640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592815f1f0 0x7f592813c0f0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f592000c730 tx=0x7f592000cb30 comp rx=0 tx=0).stop 2024-04-25T00:36:32.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f592ca0b640 1 -- 172.21.15.29:0/4177787260 shutdown_connections 2024-04-25T00:36:32.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f592ca0b640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5908067a10 0x7f5908069ed0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f592ca0b640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592815f1f0 0x7f592813c0f0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f592ca0b640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592815e8d0 0x7f59280adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f592ca0b640 1 --2- 172.21.15.29:0/4177787260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59280b4780 0x7f5928140f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f592ca0b640 1 -- 172.21.15.29:0/4177787260 >> 172.21.15.29:0/4177787260 conn(0x7f59280b1870 msgr2=0x7f592813d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:32.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f592ca0b640 1 -- 172.21.15.29:0/4177787260 shutdown_connections 2024-04-25T00:36:32.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.404+0000 7f592ca0b640 1 -- 172.21.15.29:0/4177787260 wait complete. 2024-04-25T00:36:32.425 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-04-25T00:36:32.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.544+0000 7f19eeb6d640 1 Processor -- start 2024-04-25T00:36:32.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.544+0000 7f19eeb6d640 1 -- start start 2024-04-25T00:36:32.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.544+0000 7f19eeb6d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19e806e440 0x7f19e80b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:32.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.544+0000 7f19eeb6d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e80b41e0 con 0x7f19e80b3140 2024-04-25T00:36:32.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.544+0000 7f19eeb6d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e806d960 con 0x7f19e80b3c30 2024-04-25T00:36:32.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.544+0000 7f19eeb6d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f19e80b5050 con 0x7f19e806e440 2024-04-25T00:36:32.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.544+0000 7f19ed10a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19e806e440 0x7f19e80b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:32.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.544+0000 7f19ed10a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19e806e440 0x7f19e80b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38254/0 (socket says 172.21.15.29:38254) 2024-04-25T00:36:32.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.544+0000 7f19ec909640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f19e80b3140 0x7f19e80b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37966/0 (socket says 172.21.15.29:37966) 2024-04-25T00:36:32.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.544+0000 7f19ed10a640 1 -- 172.21.15.29:0/2210754152 learned_addr learned my addr 172.21.15.29:0/2210754152 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:32.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19ed10a640 1 -- 172.21.15.29:0/2210754152 >> v1:172.21.15.157:6789/0 conn(0x7f19e80b3c30 legacy=0x7f19e80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:32.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19ed10a640 1 -- 172.21.15.29:0/2210754152 >> v1:172.21.15.29:6789/0 conn(0x7f19e80b3140 legacy=0x7f19e80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:32.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19ed10a640 1 -- 172.21.15.29:0/2210754152 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e806dc40 con 0x7f19e806e440 2024-04-25T00:36:32.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19ed10a640 1 --2- 172.21.15.29:0/2210754152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19e806e440 0x7f19e80b4b10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009e80 tx=0x7f19e001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4bd980883b6969 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:32.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19e77fe640 1 -- 172.21.15.29:0/2210754152 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e00a5690 con 0x7f19e806e440 2024-04-25T00:36:32.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19e77fe640 1 -- 172.21.15.29:0/2210754152 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e00a5830 con 0x7f19e806e440 2024-04-25T00:36:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19e77fe640 1 -- 172.21.15.29:0/2210754152 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e00a5c00 con 0x7f19e806e440 2024-04-25T00:36:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19eeb6d640 1 -- 172.21.15.29:0/2210754152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19e806e440 msgr2=0x7f19e80b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19eeb6d640 1 --2- 172.21.15.29:0/2210754152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19e806e440 0x7f19e80b4b10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009e80 tx=0x7f19e001c9d0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19eeb6d640 1 -- 172.21.15.29:0/2210754152 shutdown_connections 2024-04-25T00:36:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19eeb6d640 1 --2- 172.21.15.29:0/2210754152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19e806e440 0x7f19e80b4b10 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19eeb6d640 1 -- 172.21.15.29:0/2210754152 >> 172.21.15.29:0/2210754152 conn(0x7f19e80b1870 msgr2=0x7f19e80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19eeb6d640 1 -- 172.21.15.29:0/2210754152 shutdown_connections 2024-04-25T00:36:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.548+0000 7f19eeb6d640 1 -- 172.21.15.29:0/2210754152 wait complete. 2024-04-25T00:36:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19eeb6d640 1 Processor -- start 2024-04-25T00:36:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19eeb6d640 1 -- start start 2024-04-25T00:36:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19eeb6d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e806e440 0x7f19e81661f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19eeb6d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19e80b3140 0x7f19e813f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19eeb6d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19e80b3c30 0x7f19e813f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19eeb6d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8169d90 con 0x7f19e80b3c30 2024-04-25T00:36:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19eeb6d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8169c10 con 0x7f19e806e440 2024-04-25T00:36:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19eeb6d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f19e8169f10 con 0x7f19e80b3140 2024-04-25T00:36:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19ed10a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19e80b3c30 0x7f19e813f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19ed10a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19e80b3c30 0x7f19e813f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38264/0 (socket says 172.21.15.29:38264) 2024-04-25T00:36:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19ed10a640 1 -- 172.21.15.29:0/1728202117 learned_addr learned my addr 172.21.15.29:0/1728202117 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19ec909640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e806e440 0x7f19e81661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19e7fff640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19e80b3140 0x7f19e813f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19ed10a640 1 -- 172.21.15.29:0/1728202117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19e80b3140 msgr2=0x7f19e813f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19ed10a640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19e80b3140 0x7f19e813f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19ed10a640 1 -- 172.21.15.29:0/1728202117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e806e440 msgr2=0x7f19e81661f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19ed10a640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e806e440 0x7f19e81661f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19ed10a640 1 -- 172.21.15.29:0/1728202117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e815eb20 con 0x7f19e80b3c30 2024-04-25T00:36:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19e7fff640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19e80b3140 0x7f19e813f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19ec909640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e806e440 0x7f19e81661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19ed10a640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19e80b3c30 0x7f19e813f6d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f19e001a600 tx=0x7f19e0019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19e57fa640 1 -- 172.21.15.29:0/1728202117 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e00b8470 con 0x7f19e80b3c30 2024-04-25T00:36:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19e57fa640 1 -- 172.21.15.29:0/1728202117 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e00b8610 con 0x7f19e80b3c30 2024-04-25T00:36:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19eeb6d640 1 -- 172.21.15.29:0/1728202117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e813a4a0 con 0x7f19e80b3c30 2024-04-25T00:36:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19e57fa640 1 -- 172.21.15.29:0/1728202117 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e00b8a00 con 0x7f19e80b3c30 2024-04-25T00:36:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19eeb6d640 1 -- 172.21.15.29:0/1728202117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e8166730 con 0x7f19e80b3c30 2024-04-25T00:36:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19eeb6d640 1 -- 172.21.15.29:0/1728202117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e8069270 con 0x7f19e80b3c30 2024-04-25T00:36:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.552+0000 7f19e57fa640 1 -- 172.21.15.29:0/1728202117 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f19e00c0020 con 0x7f19e80b3c30 2024-04-25T00:36:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.556+0000 7f19e57fa640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19d4067a10 0x7f19d4069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:32.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.556+0000 7f19e57fa640 1 -- 172.21.15.29:0/1728202117 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7f19e0120520 con 0x7f19e80b3c30 2024-04-25T00:36:32.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.556+0000 7f19ec909640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19d4067a10 0x7f19d4069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:32.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.556+0000 7f19ec909640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19d4067a10 0x7f19d4069ed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f19d8007fd0 tx=0x7f19d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:32.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.560+0000 7f19e57fa640 1 -- 172.21.15.29:0/1728202117 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f19e00c7020 con 0x7f19e80b3c30 2024-04-25T00:36:32.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:32.780+0000 7f19eeb6d640 1 -- 172.21.15.29:0/1728202117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7f19e80b47e0 con 0x7f19e80b3c30 2024-04-25T00:36:33.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.764+0000 7f19e57fa640 1 -- 172.21.15.29:0/1728202117 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v272) ==== 149+0+0 (secure 0 0 0) 0x7f19e00165c0 con 0x7f19e80b3c30 2024-04-25T00:36:33.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.768+0000 7f19eeb6d640 1 -- 172.21.15.29:0/1728202117 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19d4067a10 msgr2=0x7f19d4069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:33.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.768+0000 7f19eeb6d640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19d4067a10 0x7f19d4069ed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f19d8007fd0 tx=0x7f19d8009040 comp rx=0 tx=0).stop 2024-04-25T00:36:33.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.768+0000 7f19eeb6d640 1 -- 172.21.15.29:0/1728202117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19e80b3c30 msgr2=0x7f19e813f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:33.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.768+0000 7f19eeb6d640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19e80b3c30 0x7f19e813f6d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f19e001a600 tx=0x7f19e0019770 comp rx=0 tx=0).stop 2024-04-25T00:36:33.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.768+0000 7f19eeb6d640 1 -- 172.21.15.29:0/1728202117 shutdown_connections 2024-04-25T00:36:33.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.768+0000 7f19eeb6d640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19d4067a10 0x7f19d4069ed0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:33.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.768+0000 7f19eeb6d640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19e80b3c30 0x7f19e813f6d0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:33.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.768+0000 7f19eeb6d640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19e80b3140 0x7f19e813f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:33.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.768+0000 7f19eeb6d640 1 --2- 172.21.15.29:0/1728202117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19e806e440 0x7f19e81661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:33.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.768+0000 7f19eeb6d640 1 -- 172.21.15.29:0/1728202117 >> 172.21.15.29:0/1728202117 conn(0x7f19e80b1870 msgr2=0x7f19e80b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:33.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.768+0000 7f19eeb6d640 1 -- 172.21.15.29:0/1728202117 shutdown_connections 2024-04-25T00:36:33.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.768+0000 7f19eeb6d640 1 -- 172.21.15.29:0/1728202117 wait complete. 2024-04-25T00:36:33.794 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:36:33.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a1e096640 1 Processor -- start 2024-04-25T00:36:33.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a1e096640 1 -- start start 2024-04-25T00:36:33.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a1e096640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a180b3140 0x7f4a1806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:33.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a1e096640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b4b10 0x7f4a180b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:33.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a1e096640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4a180b3a60 0x7f4a1815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:33.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a1e096640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4a180b41e0 con 0x7f4a180b3a60 2024-04-25T00:36:33.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a1e096640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4a1806d960 con 0x7f4a180b3140 2024-04-25T00:36:33.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a1e096640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1806dc40 con 0x7f4a180b4b10 2024-04-25T00:36:33.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a17fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4a180b3a60 0x7f4a1815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:33.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a17fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4a180b3a60 0x7f4a1815c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38266/0 (socket says 172.21.15.29:38266) 2024-04-25T00:36:33.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a17fff640 1 -- 172.21.15.29:0/1496992235 learned_addr learned my addr 172.21.15.29:0/1496992235 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:33.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a177fe640 1 --2- 172.21.15.29:0/1496992235 >> v2:172.21.15.157:3300/0 conn(0x7f4a180b3140 0x7f4a1806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:33.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a16ffd640 1 --2- 172.21.15.29:0/1496992235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b4b10 0x7f4a180b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:33.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a17fff640 1 -- 172.21.15.29:0/1496992235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b4b10 msgr2=0x7f4a180b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:33.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a17fff640 1 --2- 172.21.15.29:0/1496992235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b4b10 0x7f4a180b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:33.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a17fff640 1 -- 172.21.15.29:0/1496992235 >> v2:172.21.15.157:3300/0 conn(0x7f4a180b3140 msgr2=0x7f4a1806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:33.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a17fff640 1 --2- 172.21.15.29:0/1496992235 >> v2:172.21.15.157:3300/0 conn(0x7f4a180b3140 0x7f4a1806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:33.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a17fff640 1 -- 172.21.15.29:0/1496992235 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1806df20 con 0x7f4a180b3a60 2024-04-25T00:36:33.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.924+0000 7f4a16ffd640 1 --2- 172.21.15.29:0/1496992235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b4b10 0x7f4a180b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a17fff640 1 --2- 172.21.15.29:0/1496992235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180b3a60 0x7f4a1815c3e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4a180d0bf0 tx=0x7f4a1001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a254cd28d181bae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a167fc640 1 -- 172.21.15.29:0/1496992235 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a100a5670 con 0x7f4a180b3a60 2024-04-25T00:36:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a167fc640 1 -- 172.21.15.29:0/1496992235 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a100a5810 con 0x7f4a180b3a60 2024-04-25T00:36:33.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a167fc640 1 -- 172.21.15.29:0/1496992235 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a100a5be0 con 0x7f4a180b3a60 2024-04-25T00:36:33.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 -- 172.21.15.29:0/1496992235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180b3a60 msgr2=0x7f4a1815c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:33.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 --2- 172.21.15.29:0/1496992235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180b3a60 0x7f4a1815c3e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4a180d0bf0 tx=0x7f4a1001c9f0 comp rx=0 tx=0).stop 2024-04-25T00:36:33.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 -- 172.21.15.29:0/1496992235 shutdown_connections 2024-04-25T00:36:33.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 --2- 172.21.15.29:0/1496992235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180b3a60 0x7f4a1815c3e0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:33.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 --2- 172.21.15.29:0/1496992235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b4b10 0x7f4a180b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:33.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 --2- 172.21.15.29:0/1496992235 >> v2:172.21.15.157:3300/0 conn(0x7f4a180b3140 0x7f4a1806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:33.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 -- 172.21.15.29:0/1496992235 >> 172.21.15.29:0/1496992235 conn(0x7f4a180b1870 msgr2=0x7f4a180b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:33.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 -- 172.21.15.29:0/1496992235 shutdown_connections 2024-04-25T00:36:33.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 -- 172.21.15.29:0/1496992235 wait complete. 2024-04-25T00:36:33.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 Processor -- start 2024-04-25T00:36:33.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 -- start start 2024-04-25T00:36:33.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b3140 0x7f4a18158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:33.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180b3a60 0x7f4a18139470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:33.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180b4b10 0x7f4a181580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:33.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4a180ce3c0 con 0x7f4a180b3a60 2024-04-25T00:36:33.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a180ce240 con 0x7f4a180b4b10 2024-04-25T00:36:33.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a1e096640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4a180ce540 con 0x7f4a180b3140 2024-04-25T00:36:33.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a16ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180b3a60 0x7f4a18139470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:33.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a16ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180b3a60 0x7f4a18139470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38270/0 (socket says 172.21.15.29:38270) 2024-04-25T00:36:33.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a16ffd640 1 -- 172.21.15.29:0/1538796727 learned_addr learned my addr 172.21.15.29:0/1538796727 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:33.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a177fe640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b3140 0x7f4a18158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a17fff640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180b4b10 0x7f4a181580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a16ffd640 1 -- 172.21.15.29:0/1538796727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b3140 msgr2=0x7f4a18158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a16ffd640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b3140 0x7f4a18158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a16ffd640 1 -- 172.21.15.29:0/1538796727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180b4b10 msgr2=0x7f4a181580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a16ffd640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180b4b10 0x7f4a181580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a16ffd640 1 -- 172.21.15.29:0/1538796727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a18158ee0 con 0x7f4a180b3a60 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a177fe640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b3140 0x7f4a18158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.928+0000 7f4a17fff640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180b4b10 0x7f4a181580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.932+0000 7f4a16ffd640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180b3a60 0x7f4a18139470 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00c820 tx=0x7f4a0c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.932+0000 7f49fffff640 1 -- 172.21.15.29:0/1538796727 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c015070 con 0x7f4a180b3a60 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.932+0000 7f49fffff640 1 -- 172.21.15.29:0/1538796727 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a0c007a30 con 0x7f4a180b3a60 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.932+0000 7f4a1e096640 1 -- 172.21.15.29:0/1538796727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a181591c0 con 0x7f4a180b3a60 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.932+0000 7f49fffff640 1 -- 172.21.15.29:0/1538796727 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c007e40 con 0x7f4a180b3a60 2024-04-25T00:36:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.932+0000 7f4a1e096640 1 -- 172.21.15.29:0/1538796727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a18159780 con 0x7f4a180b3a60 2024-04-25T00:36:33.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.932+0000 7f4a1e096640 1 -- 172.21.15.29:0/1538796727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a18069270 con 0x7f4a180b3a60 2024-04-25T00:36:33.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.932+0000 7f49fffff640 1 -- 172.21.15.29:0/1538796727 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f4a0c018070 con 0x7f4a180b3a60 2024-04-25T00:36:33.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.932+0000 7f49fffff640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4a040679e0 0x7f4a04069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:33.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.932+0000 7f49fffff640 1 -- 172.21.15.29:0/1538796727 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7f4a0c085ce0 con 0x7f4a180b3a60 2024-04-25T00:36:33.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.932+0000 7f4a177fe640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4a040679e0 0x7f4a04069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:33.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.932+0000 7f4a177fe640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4a040679e0 0x7f4a04069ea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4a181f8390 tx=0x7f4a08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:33.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:33.936+0000 7f49fffff640 1 -- 172.21.15.29:0/1538796727 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4a0c056760 con 0x7f4a180b3a60 2024-04-25T00:36:34.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.092+0000 7f4a1e096640 1 -- 172.21.15.29:0/1538796727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4a1813bbe0 con 0x7f4a180b3a60 2024-04-25T00:36:34.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.096+0000 7f49fffff640 1 -- 172.21.15.29:0/1538796727 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f4a0c056060 con 0x7f4a180b3a60 2024-04-25T00:36:34.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.100+0000 7f4a1e096640 1 -- 172.21.15.29:0/1538796727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4a040679e0 msgr2=0x7f4a04069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:34.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.100+0000 7f4a1e096640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4a040679e0 0x7f4a04069ea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4a181f8390 tx=0x7f4a08008040 comp rx=0 tx=0).stop 2024-04-25T00:36:34.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.100+0000 7f4a1e096640 1 -- 172.21.15.29:0/1538796727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180b3a60 msgr2=0x7f4a18139470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:34.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.100+0000 7f4a1e096640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180b3a60 0x7f4a18139470 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00c820 tx=0x7f4a0c00cce0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.100+0000 7f4a1e096640 1 -- 172.21.15.29:0/1538796727 shutdown_connections 2024-04-25T00:36:34.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.100+0000 7f4a1e096640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4a040679e0 0x7f4a04069ea0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.100+0000 7f4a1e096640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180b4b10 0x7f4a181580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.100+0000 7f4a1e096640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180b3a60 0x7f4a18139470 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.100+0000 7f4a1e096640 1 --2- 172.21.15.29:0/1538796727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b3140 0x7f4a18158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.100+0000 7f4a1e096640 1 -- 172.21.15.29:0/1538796727 >> 172.21.15.29:0/1538796727 conn(0x7f4a180b1870 msgr2=0x7f4a18144bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:34.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.100+0000 7f4a1e096640 1 -- 172.21.15.29:0/1538796727 shutdown_connections 2024-04-25T00:36:34.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.100+0000 7f4a1e096640 1 -- 172.21.15.29:0/1538796727 wait complete. 2024-04-25T00:36:34.117 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:36:34.117 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:36:34.170 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:36:34.213 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:36:34.213 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:36:34.213 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:36:34.219 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:36:34.219 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:36:34.219 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:36:34.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:36:34.260 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:36:34.260 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:36:34.265 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:36:34.265 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:36:34.266 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:36:34.308 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:36:34.308 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:36:34.308 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:36:34.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d3b00640 1 Processor -- start 2024-04-25T00:36:34.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d3b00640 1 -- start start 2024-04-25T00:36:34.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d3b00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1cc0b3140 0x7fb1cc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:34.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d3b00640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb1cc0b4b10 0x7fb1cc0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:34.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d3b00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1cc0b3a60 0x7fb1cc15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:34.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d3b00640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb1cc0b41e0 con 0x7fb1cc0b4b10 2024-04-25T00:36:34.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d3b00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb1cc06d960 con 0x7fb1cc0b3140 2024-04-25T00:36:34.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d3b00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb1cc06dc40 con 0x7fb1cc0b3a60 2024-04-25T00:36:34.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d109b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb1cc0b4b10 0x7fb1cc0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:34.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d109b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb1cc0b4b10 0x7fb1cc0b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38274/0 (socket says 172.21.15.29:38274) 2024-04-25T00:36:34.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d109b640 1 -- 172.21.15.29:0/3953413391 learned_addr learned my addr 172.21.15.29:0/3953413391 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:34.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d189c640 1 --2- 172.21.15.29:0/3953413391 >> v2:172.21.15.157:3300/0 conn(0x7fb1cc0b3140 0x7fb1cc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:34.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d209d640 1 --2- 172.21.15.29:0/3953413391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1cc0b3a60 0x7fb1cc15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:34.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d109b640 1 -- 172.21.15.29:0/3953413391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1cc0b3a60 msgr2=0x7fb1cc15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:34.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d109b640 1 --2- 172.21.15.29:0/3953413391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1cc0b3a60 0x7fb1cc15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d109b640 1 -- 172.21.15.29:0/3953413391 >> v2:172.21.15.157:3300/0 conn(0x7fb1cc0b3140 msgr2=0x7fb1cc06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:34.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d109b640 1 --2- 172.21.15.29:0/3953413391 >> v2:172.21.15.157:3300/0 conn(0x7fb1cc0b3140 0x7fb1cc06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.428+0000 7fb1d109b640 1 -- 172.21.15.29:0/3953413391 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1cc06df20 con 0x7fb1cc0b4b10 2024-04-25T00:36:34.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d109b640 1 --2- 172.21.15.29:0/3953413391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1cc0b4b10 0x7fb1cc0b3520 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009870 tx=0x7fb1c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e9630d7127f468 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:34.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d089a640 1 -- 172.21.15.29:0/3953413391 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80b0070 con 0x7fb1cc0b4b10 2024-04-25T00:36:34.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d089a640 1 -- 172.21.15.29:0/3953413391 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c80ab600 con 0x7fb1cc0b4b10 2024-04-25T00:36:34.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d089a640 1 -- 172.21.15.29:0/3953413391 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80ab9d0 con 0x7fb1cc0b4b10 2024-04-25T00:36:34.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d3b00640 1 -- 172.21.15.29:0/3953413391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1cc0b4b10 msgr2=0x7fb1cc0b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:34.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d3b00640 1 --2- 172.21.15.29:0/3953413391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1cc0b4b10 0x7fb1cc0b3520 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009870 tx=0x7fb1c801c520 comp rx=0 tx=0).stop 2024-04-25T00:36:34.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d3b00640 1 -- 172.21.15.29:0/3953413391 shutdown_connections 2024-04-25T00:36:34.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d3b00640 1 --2- 172.21.15.29:0/3953413391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1cc0b4b10 0x7fb1cc0b3520 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d3b00640 1 --2- 172.21.15.29:0/3953413391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1cc0b3a60 0x7fb1cc15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d3b00640 1 --2- 172.21.15.29:0/3953413391 >> v2:172.21.15.157:3300/0 conn(0x7fb1cc0b3140 0x7fb1cc06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d3b00640 1 -- 172.21.15.29:0/3953413391 >> 172.21.15.29:0/3953413391 conn(0x7fb1cc0b1870 msgr2=0x7fb1cc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:34.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d3b00640 1 -- 172.21.15.29:0/3953413391 shutdown_connections 2024-04-25T00:36:34.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d3b00640 1 -- 172.21.15.29:0/3953413391 wait complete. 2024-04-25T00:36:34.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d3b00640 1 Processor -- start 2024-04-25T00:36:34.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.432+0000 7fb1d3b00640 1 -- start start 2024-04-25T00:36:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d3b00640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1cc0b3140 0x7fb1cc137870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d3b00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1cc0b3a60 0x7fb1cc153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d3b00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1cc0b4b10 0x7fb1cc152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d3b00640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc0ce3c0 con 0x7fb1cc0b3140 2024-04-25T00:36:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d3b00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc0ce240 con 0x7fb1cc0b3a60 2024-04-25T00:36:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d3b00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb1cc0ce540 con 0x7fb1cc0b4b10 2024-04-25T00:36:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d189c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1cc0b3140 0x7fb1cc137870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d189c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1cc0b3140 0x7fb1cc137870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38278/0 (socket says 172.21.15.29:38278) 2024-04-25T00:36:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d189c640 1 -- 172.21.15.29:0/156861390 learned_addr learned my addr 172.21.15.29:0/156861390 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d209d640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1cc0b4b10 0x7fb1cc152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:34.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d109b640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1cc0b3a60 0x7fb1cc153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:34.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d189c640 1 -- 172.21.15.29:0/156861390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1cc0b4b10 msgr2=0x7fb1cc152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:34.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d189c640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1cc0b4b10 0x7fb1cc152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d189c640 1 -- 172.21.15.29:0/156861390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1cc0b3a60 msgr2=0x7fb1cc153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:34.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d189c640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1cc0b3a60 0x7fb1cc153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d189c640 1 -- 172.21.15.29:0/156861390 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1cc15bdc0 con 0x7fb1cc0b3140 2024-04-25T00:36:34.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d109b640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1cc0b3a60 0x7fb1cc153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:34.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d189c640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1cc0b3140 0x7fb1cc137870 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00e8b0 tx=0x7fb1bc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:34.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1c27fc640 1 -- 172.21.15.29:0/156861390 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc017070 con 0x7fb1cc0b3140 2024-04-25T00:36:34.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1c27fc640 1 -- 172.21.15.29:0/156861390 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1bc00c940 con 0x7fb1cc0b3140 2024-04-25T00:36:34.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d3b00640 1 -- 172.21.15.29:0/156861390 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1cc138f20 con 0x7fb1cc0b3140 2024-04-25T00:36:34.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1c27fc640 1 -- 172.21.15.29:0/156861390 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc00cd10 con 0x7fb1cc0b3140 2024-04-25T00:36:34.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d3b00640 1 -- 172.21.15.29:0/156861390 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1cc1394e0 con 0x7fb1cc0b3140 2024-04-25T00:36:34.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d3b00640 1 -- 172.21.15.29:0/156861390 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1cc069270 con 0x7fb1cc0b3140 2024-04-25T00:36:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1c27fc640 1 -- 172.21.15.29:0/156861390 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fb1bc01a070 con 0x7fb1cc0b3140 2024-04-25T00:36:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1c27fc640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb1b00679e0 0x7fb1b0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1c27fc640 1 -- 172.21.15.29:0/156861390 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7fb1bc0864b0 con 0x7fb1cc0b3140 2024-04-25T00:36:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.436+0000 7fb1d109b640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb1b00679e0 0x7fb1b0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.440+0000 7fb1d109b640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb1b00679e0 0x7fb1b0069ea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80ad000 tx=0x7fb1c80a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:34.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.440+0000 7fb1c27fc640 1 -- 172.21.15.29:0/156861390 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb1bc057090 con 0x7fb1cc0b3140 2024-04-25T00:36:34.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.640+0000 7fb1d3b00640 1 -- 172.21.15.29:0/156861390 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1cc06d390 con 0x7fb1cc0b3140 2024-04-25T00:36:34.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.640+0000 7fb1c27fc640 1 -- 172.21.15.29:0/156861390 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+3869 (secure 0 0 0) 0x7fb1bc056990 con 0x7fb1cc0b3140 2024-04-25T00:36:34.643 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:34.643 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/524224365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":524224365},{"type":"v1","addr":"172.21.15.157:6835","nonce":524224365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1007000548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1007000548},{"type":"v1","addr":"172.21.15.157:6837","nonce":1007000548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6810,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2091414875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2091414875},{"type":"v1","addr":"172.21.15.29:6833","nonce":2091414875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":7329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/863612148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":863612148},{"type":"v1","addr":"172.21.15.29:6835","nonce":863612148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[]} 2024-04-25T00:36:34.644 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 46 2024-04-25T00:36:34.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.644+0000 7fb1d3b00640 1 -- 172.21.15.29:0/156861390 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb1b00679e0 msgr2=0x7fb1b0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:34.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.644+0000 7fb1d3b00640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb1b00679e0 0x7fb1b0069ea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80ad000 tx=0x7fb1c80a6000 comp rx=0 tx=0).stop 2024-04-25T00:36:34.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.644+0000 7fb1d3b00640 1 -- 172.21.15.29:0/156861390 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1cc0b3140 msgr2=0x7fb1cc137870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:34.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.644+0000 7fb1d3b00640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1cc0b3140 0x7fb1cc137870 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00e8b0 tx=0x7fb1bc00ed70 comp rx=0 tx=0).stop 2024-04-25T00:36:34.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.644+0000 7fb1d3b00640 1 -- 172.21.15.29:0/156861390 shutdown_connections 2024-04-25T00:36:34.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.644+0000 7fb1d3b00640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb1b00679e0 0x7fb1b0069ea0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.644+0000 7fb1d3b00640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1cc0b4b10 0x7fb1cc152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.644+0000 7fb1d3b00640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1cc0b3a60 0x7fb1cc153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.644+0000 7fb1d3b00640 1 --2- 172.21.15.29:0/156861390 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1cc0b3140 0x7fb1cc137870 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.644+0000 7fb1d3b00640 1 -- 172.21.15.29:0/156861390 >> 172.21.15.29:0/156861390 conn(0x7fb1cc0b1870 msgr2=0x7fb1cc13c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:34.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.644+0000 7fb1d3b00640 1 -- 172.21.15.29:0/156861390 shutdown_connections 2024-04-25T00:36:34.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.644+0000 7fb1d3b00640 1 -- 172.21.15.29:0/156861390 wait complete. 2024-04-25T00:36:34.663 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:36:34.663 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:36:34.664 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:36:34.664 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:36:34.664 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:36:34.664 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:36:34.664 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:36:34.664 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:36:34.664 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:36:34.664 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:36:34.665 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:36:34.665 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:36:34.665 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:36:34.665 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:36:34.665 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:36:34.665 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:36:34.675 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:36:34.675 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:36:34.676 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:36:34.676 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:36:34.681 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:36:34.681 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:36:34.686 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:36:34.686 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:36:34.725 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:36:34.726 DEBUG:teuthology.parallel:result is None 2024-04-25T00:36:34.726 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:36:34.727 DEBUG:teuthology.parallel:result is None 2024-04-25T00:36:34.728 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:36:34.728 DEBUG:teuthology.parallel:result is None 2024-04-25T00:36:34.730 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:36:34.730 DEBUG:teuthology.parallel:result is None 2024-04-25T00:36:34.730 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:36:34.792 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:36:34.793 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:36:34.799 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-25T00:36:34.805 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-25T00:36:34.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.848+0000 7fab61be2640 1 Processor -- start 2024-04-25T00:36:34.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.848+0000 7fab61be2640 1 -- start start 2024-04-25T00:36:34.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab61be2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5c0721f0 0x7fab5c072650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:34.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab61be2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab5c06e580 con 0x7fab5c0ce820 2024-04-25T00:36:34.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab61be2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab5c06e820 con 0x7fab5c071e10 2024-04-25T00:36:34.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab61be2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fab5c06f5e0 con 0x7fab5c0721f0 2024-04-25T00:36:34.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fab5c0ce820 0x7fab5c0cec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37990/0 (socket says 172.21.15.29:37990) 2024-04-25T00:36:34.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5bfff640 1 -- 172.21.15.29:0/1866851088 learned_addr learned my addr 172.21.15.29:0/1866851088 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:34.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5affd640 1 --2- 172.21.15.29:0/1866851088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5c0721f0 0x7fab5c072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:34.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5a7fc640 1 -- 172.21.15.29:0/1866851088 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3181546281 0 0) 0x7fab5c06e820 con 0x7fab5c071e10 2024-04-25T00:36:34.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5a7fc640 1 -- 172.21.15.29:0/1866851088 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab44003530 con 0x7fab5c071e10 2024-04-25T00:36:34.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5a7fc640 1 -- 172.21.15.29:0/1866851088 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4040261911 0 0) 0x7fab5c06e580 con 0x7fab5c0ce820 2024-04-25T00:36:34.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5a7fc640 1 -- 172.21.15.29:0/1866851088 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab5c06e820 con 0x7fab5c0ce820 2024-04-25T00:36:34.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5affd640 1 -- 172.21.15.29:0/1866851088 >> v1:172.21.15.157:6789/0 conn(0x7fab5c071e10 legacy=0x7fab5c15ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:34.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5affd640 1 -- 172.21.15.29:0/1866851088 >> v1:172.21.15.29:6789/0 conn(0x7fab5c0ce820 legacy=0x7fab5c0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:34.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5affd640 1 -- 172.21.15.29:0/1866851088 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c06eb00 con 0x7fab5c0721f0 2024-04-25T00:36:34.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5affd640 1 --2- 172.21.15.29:0/1866851088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5c0721f0 0x7fab5c072650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab500097f0 tx=0x7fab5001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6e76605481b16e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:34.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5a7fc640 1 -- 172.21.15.29:0/1866851088 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab500b0070 con 0x7fab5c0721f0 2024-04-25T00:36:34.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5a7fc640 1 -- 172.21.15.29:0/1866851088 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab500ab600 con 0x7fab5c0721f0 2024-04-25T00:36:34.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab5a7fc640 1 -- 172.21.15.29:0/1866851088 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab500ab9d0 con 0x7fab5c0721f0 2024-04-25T00:36:34.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab61be2640 1 -- 172.21.15.29:0/1866851088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5c0721f0 msgr2=0x7fab5c072650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:34.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.852+0000 7fab61be2640 1 --2- 172.21.15.29:0/1866851088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5c0721f0 0x7fab5c072650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab500097f0 tx=0x7fab5001c4e0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 -- 172.21.15.29:0/1866851088 shutdown_connections 2024-04-25T00:36:34.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 --2- 172.21.15.29:0/1866851088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5c0721f0 0x7fab5c072650 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 -- 172.21.15.29:0/1866851088 >> 172.21.15.29:0/1866851088 conn(0x7fab5c0b2920 msgr2=0x7fab5c0b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:34.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 -- 172.21.15.29:0/1866851088 shutdown_connections 2024-04-25T00:36:34.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 -- 172.21.15.29:0/1866851088 wait complete. 2024-04-25T00:36:34.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 Processor -- start 2024-04-25T00:36:34.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 -- start start 2024-04-25T00:36:34.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5c071e10 0x7fab5c1471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:34.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab5c0721f0 0x7fab5c147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:34.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab5c0ce820 0x7fab5c147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:34.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c0cf930 con 0x7fab5c0ce820 2024-04-25T00:36:34.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c0cf7b0 con 0x7fab5c0721f0 2024-04-25T00:36:34.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fab5c169420 con 0x7fab5c071e10 2024-04-25T00:36:34.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab5bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab5c0ce820 0x7fab5c147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:34.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab5bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab5c0ce820 0x7fab5c147d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38314/0 (socket says 172.21.15.29:38314) 2024-04-25T00:36:34.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab5bfff640 1 -- 172.21.15.29:0/1606592522 learned_addr learned my addr 172.21.15.29:0/1606592522 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:34.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab5affd640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab5c0721f0 0x7fab5c147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:34.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab5b7fe640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5c071e10 0x7fab5c1471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:34.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab5bfff640 1 -- 172.21.15.29:0/1606592522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5c071e10 msgr2=0x7fab5c1471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:34.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab5bfff640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5c071e10 0x7fab5c1471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab5bfff640 1 -- 172.21.15.29:0/1606592522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab5c0721f0 msgr2=0x7fab5c147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:34.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab5bfff640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab5c0721f0 0x7fab5c147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:34.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab5bfff640 1 -- 172.21.15.29:0/1606592522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c152a20 con 0x7fab5c0ce820 2024-04-25T00:36:34.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab5bfff640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab5c0ce820 0x7fab5c147d50 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fab5400caf0 tx=0x7fab5400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:34.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab43fff640 1 -- 172.21.15.29:0/1606592522 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54017070 con 0x7fab5c0ce820 2024-04-25T00:36:34.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab43fff640 1 -- 172.21.15.29:0/1606592522 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5400aca0 con 0x7fab5c0ce820 2024-04-25T00:36:34.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 -- 172.21.15.29:0/1606592522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5c154e10 con 0x7fab5c0ce820 2024-04-25T00:36:34.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab43fff640 1 -- 172.21.15.29:0/1606592522 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54012640 con 0x7fab5c0ce820 2024-04-25T00:36:34.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.856+0000 7fab61be2640 1 -- 172.21.15.29:0/1606592522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5c1489e0 con 0x7fab5c0ce820 2024-04-25T00:36:34.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.860+0000 7fab61be2640 1 -- 172.21.15.29:0/1606592522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5c154c00 con 0x7fab5c0ce820 2024-04-25T00:36:34.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.860+0000 7fab43fff640 1 -- 172.21.15.29:0/1606592522 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fab5401a070 con 0x7fab5c0ce820 2024-04-25T00:36:34.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.860+0000 7fab43fff640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab480679e0 0x7fab48069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:34.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.860+0000 7fab43fff640 1 -- 172.21.15.29:0/1606592522 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7fab54085ae0 con 0x7fab5c0ce820 2024-04-25T00:36:34.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.860+0000 7fab5b7fe640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab480679e0 0x7fab48069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:34.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.860+0000 7fab5b7fe640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab480679e0 0x7fab48069ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab4c001020 tx=0x7fab4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:34.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:34.864+0000 7fab43fff640 1 -- 172.21.15.29:0/1606592522 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fab54055f10 con 0x7fab5c0ce820 2024-04-25T00:36:35.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.032+0000 7fab61be2640 1 -- 172.21.15.29:0/1606592522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fab5c069270 con 0x7fab5c0ce820 2024-04-25T00:36:35.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.784+0000 7fab43fff640 1 -- 172.21.15.29:0/1606592522 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v78) ==== 97+0+0 (secure 0 0 0) 0x7fab5405a210 con 0x7fab5c0ce820 2024-04-25T00:36:35.785 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-04-25T00:36:35.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.788+0000 7fab61be2640 1 -- 172.21.15.29:0/1606592522 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab480679e0 msgr2=0x7fab48069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.788+0000 7fab61be2640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab480679e0 0x7fab48069ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab4c001020 tx=0x7fab4c009040 comp rx=0 tx=0).stop 2024-04-25T00:36:35.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.788+0000 7fab61be2640 1 -- 172.21.15.29:0/1606592522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab5c0ce820 msgr2=0x7fab5c147d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.788+0000 7fab61be2640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab5c0ce820 0x7fab5c147d50 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fab5400caf0 tx=0x7fab5400a060 comp rx=0 tx=0).stop 2024-04-25T00:36:35.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.788+0000 7fab61be2640 1 -- 172.21.15.29:0/1606592522 shutdown_connections 2024-04-25T00:36:35.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.788+0000 7fab61be2640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab480679e0 0x7fab48069ea0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.788+0000 7fab61be2640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab5c0ce820 0x7fab5c147d50 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.788+0000 7fab61be2640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab5c0721f0 0x7fab5c147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.788+0000 7fab61be2640 1 --2- 172.21.15.29:0/1606592522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5c071e10 0x7fab5c1471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.788+0000 7fab61be2640 1 -- 172.21.15.29:0/1606592522 >> 172.21.15.29:0/1606592522 conn(0x7fab5c0b2920 msgr2=0x7fab5c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:35.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.788+0000 7fab61be2640 1 -- 172.21.15.29:0/1606592522 shutdown_connections 2024-04-25T00:36:35.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.788+0000 7fab61be2640 1 -- 172.21.15.29:0/1606592522 wait complete. 2024-04-25T00:36:35.811 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:36:35.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.932+0000 7f762eca5640 1 Processor -- start 2024-04-25T00:36:35.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.932+0000 7f762eca5640 1 -- start start 2024-04-25T00:36:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762eca5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f762815d980 0x7f762806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762eca5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76280b2b70 0x7f762806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762eca5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7628150df0 0x7f7628151280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762eca5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f76280cf560 con 0x7f762815d980 2024-04-25T00:36:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762eca5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f76280b3130 con 0x7f76280b2b70 2024-04-25T00:36:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762eca5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f76280b3410 con 0x7f7628150df0 2024-04-25T00:36:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762ca41640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f762815d980 0x7f762806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762ca41640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f762815d980 0x7f762806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38318/0 (socket says 172.21.15.29:38318) 2024-04-25T00:36:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762ca41640 1 -- 172.21.15.29:0/1501475214 learned_addr learned my addr 172.21.15.29:0/1501475214 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762ca41640 1 -- 172.21.15.29:0/1501475214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7628150df0 msgr2=0x7f7628151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762ca41640 1 --2- 172.21.15.29:0/1501475214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7628150df0 0x7f7628151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762ca41640 1 -- 172.21.15.29:0/1501475214 >> v2:172.21.15.157:3300/0 conn(0x7f76280b2b70 msgr2=0x7f762806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f7627fff640 1 --2- 172.21.15.29:0/1501475214 >> v2:172.21.15.157:3300/0 conn(0x7f76280b2b70 0x7f762806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762ca41640 1 --2- 172.21.15.29:0/1501475214 >> v2:172.21.15.157:3300/0 conn(0x7f76280b2b70 0x7f762806db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762ca41640 1 -- 172.21.15.29:0/1501475214 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76280b36f0 con 0x7f762815d980 2024-04-25T00:36:35.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f762ca41640 1 --2- 172.21.15.29:0/1501475214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f762815d980 0x7f762806d5d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f76280b2720 tx=0x7f761801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640a27fe9d6879ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:35.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f76277fe640 1 -- 172.21.15.29:0/1501475214 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76180ab430 con 0x7f762815d980 2024-04-25T00:36:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f76277fe640 1 -- 172.21.15.29:0/1501475214 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76180b0070 con 0x7f762815d980 2024-04-25T00:36:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.936+0000 7f76277fe640 1 -- 172.21.15.29:0/1501475214 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76180ab7f0 con 0x7f762815d980 2024-04-25T00:36:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 -- 172.21.15.29:0/1501475214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f762815d980 msgr2=0x7f762806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 --2- 172.21.15.29:0/1501475214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f762815d980 0x7f762806d5d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f76280b2720 tx=0x7f761801c4f0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 -- 172.21.15.29:0/1501475214 shutdown_connections 2024-04-25T00:36:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 --2- 172.21.15.29:0/1501475214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7628150df0 0x7f7628151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 --2- 172.21.15.29:0/1501475214 >> v2:172.21.15.157:3300/0 conn(0x7f76280b2b70 0x7f762806db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 --2- 172.21.15.29:0/1501475214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f762815d980 0x7f762806d5d0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 -- 172.21.15.29:0/1501475214 >> 172.21.15.29:0/1501475214 conn(0x7f76280b1870 msgr2=0x7f76280b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 -- 172.21.15.29:0/1501475214 shutdown_connections 2024-04-25T00:36:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 -- 172.21.15.29:0/1501475214 wait complete. 2024-04-25T00:36:35.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 Processor -- start 2024-04-25T00:36:35.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 -- start start 2024-04-25T00:36:35.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76280b2b70 0x7f76281459d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7628150df0 0x7f76281596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f762815d980 0x7f762813d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7628151940 con 0x7f7628150df0 2024-04-25T00:36:35.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f76281517c0 con 0x7f762815d980 2024-04-25T00:36:35.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762eca5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f762815e020 con 0x7f76280b2b70 2024-04-25T00:36:35.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f7627fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7628150df0 0x7f76281596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f7627fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7628150df0 0x7f76281596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38330/0 (socket says 172.21.15.29:38330) 2024-04-25T00:36:35.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f7627fff640 1 -- 172.21.15.29:0/2170505652 learned_addr learned my addr 172.21.15.29:0/2170505652 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:35.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762ca41640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76280b2b70 0x7f76281459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f762d242640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f762815d980 0x7f762813d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f7627fff640 1 -- 172.21.15.29:0/2170505652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76280b2b70 msgr2=0x7f76281459d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f7627fff640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76280b2b70 0x7f76281459d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f7627fff640 1 -- 172.21.15.29:0/2170505652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f762815d980 msgr2=0x7f762813d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f7627fff640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f762815d980 0x7f762813d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.940+0000 7f7627fff640 1 -- 172.21.15.29:0/2170505652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76280cc910 con 0x7f7628150df0 2024-04-25T00:36:35.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.944+0000 7f7627fff640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7628150df0 0x7f76281596b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f761c00c820 tx=0x7f761c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:35.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.944+0000 7f76257fa640 1 -- 172.21.15.29:0/2170505652 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c015070 con 0x7f7628150df0 2024-04-25T00:36:35.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.944+0000 7f76257fa640 1 -- 172.21.15.29:0/2170505652 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f761c004240 con 0x7f7628150df0 2024-04-25T00:36:35.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.944+0000 7f762eca5640 1 -- 172.21.15.29:0/2170505652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762815b430 con 0x7f7628150df0 2024-04-25T00:36:35.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.944+0000 7f76257fa640 1 -- 172.21.15.29:0/2170505652 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c00ce10 con 0x7f7628150df0 2024-04-25T00:36:35.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.944+0000 7f762eca5640 1 -- 172.21.15.29:0/2170505652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762813e890 con 0x7f7628150df0 2024-04-25T00:36:35.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.944+0000 7f762eca5640 1 -- 172.21.15.29:0/2170505652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7628069270 con 0x7f7628150df0 2024-04-25T00:36:35.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.944+0000 7f76257fa640 1 -- 172.21.15.29:0/2170505652 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f761c007830 con 0x7f7628150df0 2024-04-25T00:36:35.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.944+0000 7f76257fa640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7614067a10 0x7f7614069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.944+0000 7f76257fa640 1 -- 172.21.15.29:0/2170505652 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 4857+0+0 (secure 0 0 0) 0x7f761c085920 con 0x7f7628150df0 2024-04-25T00:36:35.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.944+0000 7f762ca41640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7614067a10 0x7f7614069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.944+0000 7f762ca41640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7614067a10 0x7f7614069ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f76180af000 tx=0x7f7618008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:35.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:35.948+0000 7f76257fa640 1 -- 172.21.15.29:0/2170505652 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f761c056620 con 0x7f7628150df0 2024-04-25T00:36:36.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.168+0000 7f762eca5640 1 -- 172.21.15.29:0/2170505652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f76280b0540 con 0x7f7628150df0 2024-04-25T00:36:36.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.168+0000 7f76257fa640 1 -- 172.21.15.29:0/2170505652 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f761c055f20 con 0x7f7628150df0 2024-04-25T00:36:36.173 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:36.173 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:36:36.173 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-04-25T00:36:36.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.173 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-04-25T00:36:36.173 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmyzgKAxAAU+nrUJgQhDX6xlARNFI/ng==", 2024-04-25T00:36:36.173 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.173 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:36:36.173 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:36:36.173 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:36:36.173 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.173 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmtyzPERAAxNUDANw1H/ZzOo+dhgzxlg==", 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmYUl8CxAAj6pV1+s4a0rpWLq2EgZ+8w==", 2024-04-25T00:36:36.174 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmuAhKGhAAOLIcWUodW5TqtVgHWZl12A==", 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.175 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAzpClmudk6MhAA6YFyd57jwtB2pdOaad7bxQ==", 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA2pClm76q0NRAAxWhEP2ZGoFnZcuiQyGjglw==", 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.176 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA6pClmcgULABAA2F5WBR9Df0v4Sn875AzF7g==", 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA9pClm9QuRBhAALFLruv4JU/uNZYMX0yNuXw==", 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBBpClmNj4VDBAAxaxMvyaCvjynPY3f81udnw==", 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:36:36.178 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBEpClmzBybEBAArry1lM4RIsM/txKfgBqDpw==", 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.179 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBHpClm5HfjFhAAbFwJCpbhegLWDu0mbS8FoA==", 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBKpClme/7oHBAAOR2iQ6CweexR7sC1PgesZw==", 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.180 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A==", 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm8W/EJxAA0HZW24Hhcwf+soSZn6jHJg==", 2024-04-25T00:36:36.181 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmGuQJJRAAhVd4D0jc6Whgyf+B6xgFMw==", 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:36:36.182 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmn+ciLRAAg4tQuZ13WP0RnWJaJq5fUQ==", 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.183 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmPVihJBAA7PQge/teyRZKSkBnufZSZw==", 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClm6WrdOhAAsY9FLCxG7PGpq2NCY4hGhQ==", 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:36:36.184 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-04-25T00:36:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmeR4UOBAANs9SRolOm6E8+/4ciupk5w==", 2024-04-25T00:36:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:36:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:36:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:36:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:36:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:36.185 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:36:36.185 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:36:36.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.172+0000 7f762eca5640 1 -- 172.21.15.29:0/2170505652 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7614067a10 msgr2=0x7f7614069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.172+0000 7f762eca5640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7614067a10 0x7f7614069ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f76180af000 tx=0x7f7618008040 comp rx=0 tx=0).stop 2024-04-25T00:36:36.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.172+0000 7f762eca5640 1 -- 172.21.15.29:0/2170505652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7628150df0 msgr2=0x7f76281596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.172+0000 7f762eca5640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7628150df0 0x7f76281596b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f761c00c820 tx=0x7f761c00cce0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.172+0000 7f762eca5640 1 -- 172.21.15.29:0/2170505652 shutdown_connections 2024-04-25T00:36:36.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.172+0000 7f762eca5640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7614067a10 0x7f7614069ed0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.172+0000 7f762eca5640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f762815d980 0x7f762813d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.172+0000 7f762eca5640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7628150df0 0x7f76281596b0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.172+0000 7f762eca5640 1 --2- 172.21.15.29:0/2170505652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76280b2b70 0x7f76281459d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.172+0000 7f762eca5640 1 -- 172.21.15.29:0/2170505652 >> 172.21.15.29:0/2170505652 conn(0x7f76280b1870 msgr2=0x7f76280afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:36.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.172+0000 7f762eca5640 1 -- 172.21.15.29:0/2170505652 shutdown_connections 2024-04-25T00:36:36.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.172+0000 7f762eca5640 1 -- 172.21.15.29:0/2170505652 wait complete. 2024-04-25T00:36:36.192 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:36:36.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f8eaee640 1 Processor -- start 2024-04-25T00:36:36.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f8eaee640 1 -- start start 2024-04-25T00:36:36.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f8eaee640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f8815f3c0 0x7f9f880cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f8eaee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f880b3720 con 0x7f9f880b4780 2024-04-25T00:36:36.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f8eaee640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f880b39c0 con 0x7f9f8815e8d0 2024-04-25T00:36:36.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f8eaee640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9f880b42f0 con 0x7f9f8815f3c0 2024-04-25T00:36:36.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f87fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9f880b4780 0x7f9f8815c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38002/0 (socket says 172.21.15.29:38002) 2024-04-25T00:36:36.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f87fff640 1 -- 172.21.15.29:0/3439923438 learned_addr learned my addr 172.21.15.29:0/3439923438 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:36.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f8d08b640 1 --2- 172.21.15.29:0/3439923438 >> v2:172.21.15.157:3300/0 conn(0x7f9f8815f3c0 0x7f9f880cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3040030449 0 0) 0x7f9f880b3720 con 0x7f9f880b4780 2024-04-25T00:36:36.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f78003530 con 0x7f9f880b4780 2024-04-25T00:36:36.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 284215968 0 0) 0x7f9f78003530 con 0x7f9f880b4780 2024-04-25T00:36:36.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 >> v2:172.21.15.157:3300/0 conn(0x7f9f8815f3c0 msgr2=0x7f9f880cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:36.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 --2- 172.21.15.29:0/3439923438 >> v2:172.21.15.157:3300/0 conn(0x7f9f8815f3c0 0x7f9f880cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 >> v1:172.21.15.157:6789/0 conn(0x7f9f8815e8d0 legacy=0x7f9f8815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f880b3ca0 con 0x7f9f880b4780 2024-04-25T00:36:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f9f74001680 con 0x7f9f880b4780 2024-04-25T00:36:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 >> v1:172.21.15.29:6789/0 conn(0x7f9f880b4780 legacy=0x7f9f8815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 --2- 172.21.15.29:0/3439923438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f780039c0 0x7f9f78003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 --2- 172.21.15.29:0/3439923438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f78004810 0x7f9f78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 --2- 172.21.15.29:0/3439923438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f780051d0 0x7f9f7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9f880b42f0 con 0x7f9f78004810 2024-04-25T00:36:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7800c070 con 0x7f9f780039c0 2024-04-25T00:36:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7800c270 con 0x7f9f780051d0 2024-04-25T00:36:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f87fff640 1 --2- 172.21.15.29:0/3439923438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f78004810 0x7f9f78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f8c88a640 1 --2- 172.21.15.29:0/3439923438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f780039c0 0x7f9f78003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f87fff640 1 -- 172.21.15.29:0/3439923438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f780051d0 msgr2=0x7f9f7800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f87fff640 1 --2- 172.21.15.29:0/3439923438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f780051d0 0x7f9f7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f87fff640 1 -- 172.21.15.29:0/3439923438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f780039c0 msgr2=0x7f9f78003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f87fff640 1 --2- 172.21.15.29:0/3439923438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f780039c0 0x7f9f78003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f87fff640 1 -- 172.21.15.29:0/3439923438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f78003530 con 0x7f9f78004810 2024-04-25T00:36:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f8d08b640 1 -- 172.21.15.29:0/3439923438 reap_dead start 2024-04-25T00:36:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.308+0000 7f9f8c88a640 1 --2- 172.21.15.29:0/3439923438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f780039c0 0x7f9f78003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f87fff640 1 --2- 172.21.15.29:0/3439923438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f78004810 0x7f9f78004c90 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400c210 tx=0x7f9f7401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f789070cac3b80ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74007d40 con 0x7f9f78004810 2024-04-25T00:36:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f74005b00 con 0x7f9f78004810 2024-04-25T00:36:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f877fe640 1 -- 172.21.15.29:0/3439923438 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74007070 con 0x7f9f78004810 2024-04-25T00:36:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 -- 172.21.15.29:0/3439923438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f78004810 msgr2=0x7f9f78004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 --2- 172.21.15.29:0/3439923438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f78004810 0x7f9f78004c90 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400c210 tx=0x7f9f7401f3b0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 -- 172.21.15.29:0/3439923438 shutdown_connections 2024-04-25T00:36:36.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 --2- 172.21.15.29:0/3439923438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f78004810 0x7f9f78004c90 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 -- 172.21.15.29:0/3439923438 >> 172.21.15.29:0/3439923438 conn(0x7f9f880b1870 msgr2=0x7f9f880b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:36.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 -- 172.21.15.29:0/3439923438 shutdown_connections 2024-04-25T00:36:36.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 -- 172.21.15.29:0/3439923438 wait complete. 2024-04-25T00:36:36.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 Processor -- start 2024-04-25T00:36:36.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 -- start start 2024-04-25T00:36:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f78004810 0x7f9f881357f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f88135d30 0x7f9f8815b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8815b8c0 0x7f9f88137620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88144a20 con 0x7f9f8815b8c0 2024-04-25T00:36:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8806cf10 con 0x7f9f88135d30 2024-04-25T00:36:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8eaee640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f88140980 con 0x7f9f78004810 2024-04-25T00:36:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8d08b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8815b8c0 0x7f9f88137620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8d08b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8815b8c0 0x7f9f88137620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38352/0 (socket says 172.21.15.29:38352) 2024-04-25T00:36:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.312+0000 7f9f8d08b640 1 -- 172.21.15.29:0/874024021 learned_addr learned my addr 172.21.15.29:0/874024021 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f8d08b640 1 -- 172.21.15.29:0/874024021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f78004810 msgr2=0x7f9f881357f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f8d08b640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f78004810 0x7f9f881357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f8d08b640 1 -- 172.21.15.29:0/874024021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f88135d30 msgr2=0x7f9f8815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f87fff640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f88135d30 0x7f9f8815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f8d08b640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f88135d30 0x7f9f8815b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f8d08b640 1 -- 172.21.15.29:0/874024021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f88070ce0 con 0x7f9f8815b8c0 2024-04-25T00:36:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f87fff640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f88135d30 0x7f9f8815b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f8d08b640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8815b8c0 0x7f9f88137620 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f9f88067d10 tx=0x7f9f8000d480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f857fa640 1 -- 172.21.15.29:0/874024021 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8000be60 con 0x7f9f8815b8c0 2024-04-25T00:36:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f857fa640 1 -- 172.21.15.29:0/874024021 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8001b070 con 0x7f9f8815b8c0 2024-04-25T00:36:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f8eaee640 1 -- 172.21.15.29:0/874024021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f88070f80 con 0x7f9f8815b8c0 2024-04-25T00:36:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f857fa640 1 -- 172.21.15.29:0/874024021 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8000dd50 con 0x7f9f8815b8c0 2024-04-25T00:36:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f8eaee640 1 -- 172.21.15.29:0/874024021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f88071540 con 0x7f9f8815b8c0 2024-04-25T00:36:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f8eaee640 1 -- 172.21.15.29:0/874024021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f88069270 con 0x7f9f8815b8c0 2024-04-25T00:36:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f857fa640 1 -- 172.21.15.29:0/874024021 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f9f80015070 con 0x7f9f8815b8c0 2024-04-25T00:36:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f857fa640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f6c067a10 0x7f9f6c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f857fa640 1 -- 172.21.15.29:0/874024021 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 4857+0+0 (secure 0 0 0) 0x7f9f80087be0 con 0x7f9f8815b8c0 2024-04-25T00:36:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f8c88a640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f6c067a10 0x7f9f6c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.316+0000 7f9f8c88a640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f6c067a10 0x7f9f6c069ed0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9f88069990 tx=0x7f9f7c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:36.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.320+0000 7f9f857fa640 1 -- 172.21.15.29:0/874024021 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9f800587c0 con 0x7f9f8815b8c0 2024-04-25T00:36:36.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.480+0000 7f9f8eaee640 1 -- 172.21.15.29:0/874024021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f8806e7b0 con 0x7f9f8815b8c0 2024-04-25T00:36:36.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.480+0000 7f9f857fa640 1 -- 172.21.15.29:0/874024021 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+12844 (secure 0 0 0) 0x7f9f800580c0 con 0x7f9f8815b8c0 2024-04-25T00:36:36.483 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:36.484 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":80,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:36:35.796851+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:36:36.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.484+0000 7f9f8eaee640 1 -- 172.21.15.29:0/874024021 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f6c067a10 msgr2=0x7f9f6c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.484+0000 7f9f8eaee640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f6c067a10 0x7f9f6c069ed0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9f88069990 tx=0x7f9f7c009040 comp rx=0 tx=0).stop 2024-04-25T00:36:36.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.484+0000 7f9f8eaee640 1 -- 172.21.15.29:0/874024021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8815b8c0 msgr2=0x7f9f88137620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.484+0000 7f9f8eaee640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8815b8c0 0x7f9f88137620 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f9f88067d10 tx=0x7f9f8000d480 comp rx=0 tx=0).stop 2024-04-25T00:36:36.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.484+0000 7f9f8eaee640 1 -- 172.21.15.29:0/874024021 shutdown_connections 2024-04-25T00:36:36.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.484+0000 7f9f8eaee640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f6c067a10 0x7f9f6c069ed0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.484+0000 7f9f8eaee640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8815b8c0 0x7f9f88137620 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.484+0000 7f9f8eaee640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f88135d30 0x7f9f8815b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.484+0000 7f9f8eaee640 1 --2- 172.21.15.29:0/874024021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f78004810 0x7f9f881357f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.484+0000 7f9f8eaee640 1 -- 172.21.15.29:0/874024021 >> 172.21.15.29:0/874024021 conn(0x7f9f880b1870 msgr2=0x7f9f8815cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:36.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.484+0000 7f9f8eaee640 1 -- 172.21.15.29:0/874024021 shutdown_connections 2024-04-25T00:36:36.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.484+0000 7f9f8eaee640 1 -- 172.21.15.29:0/874024021 wait complete. 2024-04-25T00:36:36.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:36:36.502 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:36:36.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd441d2f640 1 Processor -- start 2024-04-25T00:36:36.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd441d2f640 1 -- start start 2024-04-25T00:36:36.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd441d2f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd43c15e200 0x7fd43c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd441d2f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd43c0d2c60 0x7fd43c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd441d2f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd43c0cfc10 con 0x7fd43c0cf690 2024-04-25T00:36:36.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd441d2f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd43c0b38d0 con 0x7fd43c0d2c60 2024-04-25T00:36:36.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd441d2f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd43c0cfef0 con 0x7fd43c15e200 2024-04-25T00:36:36.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd43bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd43c0d2c60 0x7fd43c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd43affd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd43c0cf690 0x7fd43c0d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38012/0 (socket says 172.21.15.29:38012) 2024-04-25T00:36:36.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd43bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd43c0d2c60 0x7fd43c0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38362/0 (socket says 172.21.15.29:38362) 2024-04-25T00:36:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd43affd640 1 -- 172.21.15.29:0/344015614 learned_addr learned my addr 172.21.15.29:0/344015614 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd43b7fe640 1 --2- 172.21.15.29:0/344015614 >> v2:172.21.15.157:3300/0 conn(0x7fd43c15e200 0x7fd43c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd43bfff640 1 -- 172.21.15.29:0/344015614 >> v2:172.21.15.157:3300/0 conn(0x7fd43c15e200 msgr2=0x7fd43c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd43bfff640 1 --2- 172.21.15.29:0/344015614 >> v2:172.21.15.157:3300/0 conn(0x7fd43c15e200 0x7fd43c0d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd43bfff640 1 -- 172.21.15.29:0/344015614 >> v1:172.21.15.29:6789/0 conn(0x7fd43c0cf690 legacy=0x7fd43c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.632+0000 7fd43bfff640 1 -- 172.21.15.29:0/344015614 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c0b3b20 con 0x7fd43c0d2c60 2024-04-25T00:36:36.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd43bfff640 1 --2- 172.21.15.29:0/344015614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd43c0d2c60 0x7fd43c0b4320 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd43c08d8f0 tx=0x7fd43401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1f515131abfea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:36.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd43a7fc640 1 -- 172.21.15.29:0/344015614 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4340b2070 con 0x7fd43c0d2c60 2024-04-25T00:36:36.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd43a7fc640 1 -- 172.21.15.29:0/344015614 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4340a59e0 con 0x7fd43c0d2c60 2024-04-25T00:36:36.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd43a7fc640 1 -- 172.21.15.29:0/344015614 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4340a5db0 con 0x7fd43c0d2c60 2024-04-25T00:36:36.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd441d2f640 1 -- 172.21.15.29:0/344015614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd43c0d2c60 msgr2=0x7fd43c0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd441d2f640 1 --2- 172.21.15.29:0/344015614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd43c0d2c60 0x7fd43c0b4320 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd43c08d8f0 tx=0x7fd43401c900 comp rx=0 tx=0).stop 2024-04-25T00:36:36.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd441d2f640 1 -- 172.21.15.29:0/344015614 shutdown_connections 2024-04-25T00:36:36.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd441d2f640 1 --2- 172.21.15.29:0/344015614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd43c0d2c60 0x7fd43c0b4320 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd441d2f640 1 --2- 172.21.15.29:0/344015614 >> v2:172.21.15.157:3300/0 conn(0x7fd43c15e200 0x7fd43c0d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd441d2f640 1 -- 172.21.15.29:0/344015614 >> 172.21.15.29:0/344015614 conn(0x7fd43c09d280 msgr2=0x7fd43c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:36.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd441d2f640 1 -- 172.21.15.29:0/344015614 shutdown_connections 2024-04-25T00:36:36.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd441d2f640 1 -- 172.21.15.29:0/344015614 wait complete. 2024-04-25T00:36:36.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd441d2f640 1 Processor -- start 2024-04-25T00:36:36.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.636+0000 7fd441d2f640 1 -- start start 2024-04-25T00:36:36.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd441d2f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd43c0cf690 0x7fd43c0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd441d2f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c0d2c60 0x7fd43c14b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd441d2f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c15e200 0x7fd43c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd441d2f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c0b51f0 con 0x7fd43c0cf690 2024-04-25T00:36:36.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd441d2f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c0b5070 con 0x7fd43c15e200 2024-04-25T00:36:36.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd441d2f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd43c0b5370 con 0x7fd43c0d2c60 2024-04-25T00:36:36.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd43c0cf690 0x7fd43c0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd43c0cf690 0x7fd43c0cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38368/0 (socket says 172.21.15.29:38368) 2024-04-25T00:36:36.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43b7fe640 1 -- 172.21.15.29:0/2744736808 learned_addr learned my addr 172.21.15.29:0/2744736808 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:36.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43affd640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c0d2c60 0x7fd43c14b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43bfff640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c15e200 0x7fd43c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43b7fe640 1 -- 172.21.15.29:0/2744736808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c0d2c60 msgr2=0x7fd43c14b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43b7fe640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c0d2c60 0x7fd43c14b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43b7fe640 1 -- 172.21.15.29:0/2744736808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c15e200 msgr2=0x7fd43c0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43b7fe640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c15e200 0x7fd43c0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43b7fe640 1 -- 172.21.15.29:0/2744736808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c0ccb50 con 0x7fd43c0cf690 2024-04-25T00:36:36.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43affd640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c0d2c60 0x7fd43c14b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:36.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43b7fe640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd43c0cf690 0x7fd43c0cdbf0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00d5f0 tx=0x7fd42c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:36.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd423fff640 1 -- 172.21.15.29:0/2744736808 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0125f0 con 0x7fd43c0cf690 2024-04-25T00:36:36.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd423fff640 1 -- 172.21.15.29:0/2744736808 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42c019070 con 0x7fd43c0cf690 2024-04-25T00:36:36.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd441d2f640 1 -- 172.21.15.29:0/2744736808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c0cce30 con 0x7fd43c0cf690 2024-04-25T00:36:36.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd423fff640 1 -- 172.21.15.29:0/2744736808 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0129b0 con 0x7fd43c0cf690 2024-04-25T00:36:36.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd441d2f640 1 -- 172.21.15.29:0/2744736808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c0cd3f0 con 0x7fd43c0cf690 2024-04-25T00:36:36.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd441d2f640 1 -- 172.21.15.29:0/2744736808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43c069270 con 0x7fd43c0cf690 2024-04-25T00:36:36.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd423fff640 1 -- 172.21.15.29:0/2744736808 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fd42c012b50 con 0x7fd43c0cf690 2024-04-25T00:36:36.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd423fff640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd4280679e0 0x7fd428069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd423fff640 1 -- 172.21.15.29:0/2744736808 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 4857+0+0 (secure 0 0 0) 0x7fd42c085ed0 con 0x7fd43c0cf690 2024-04-25T00:36:36.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43affd640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd4280679e0 0x7fd428069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.640+0000 7fd43affd640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd4280679e0 0x7fd428069ea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0b9f40 tx=0x7fd430009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:36.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.644+0000 7fd423fff640 1 -- 172.21.15.29:0/2744736808 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd42c056c50 con 0x7fd43c0cf690 2024-04-25T00:36:36.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.808+0000 7fd441d2f640 1 -- 172.21.15.29:0/2744736808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd43c06c6c0 con 0x7fd43c0cf690 2024-04-25T00:36:36.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.808+0000 7fd423fff640 1 -- 172.21.15.29:0/2744736808 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7fd42c056550 con 0x7fd43c0cf690 2024-04-25T00:36:36.812 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:36:36.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.812+0000 7fd441d2f640 1 -- 172.21.15.29:0/2744736808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd4280679e0 msgr2=0x7fd428069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.812+0000 7fd441d2f640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd4280679e0 0x7fd428069ea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0b9f40 tx=0x7fd430009040 comp rx=0 tx=0).stop 2024-04-25T00:36:36.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.812+0000 7fd441d2f640 1 -- 172.21.15.29:0/2744736808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd43c0cf690 msgr2=0x7fd43c0cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.812+0000 7fd441d2f640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd43c0cf690 0x7fd43c0cdbf0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00d5f0 tx=0x7fd42c00dab0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.812+0000 7fd441d2f640 1 -- 172.21.15.29:0/2744736808 shutdown_connections 2024-04-25T00:36:36.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.812+0000 7fd441d2f640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd4280679e0 0x7fd428069ea0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.812+0000 7fd441d2f640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c15e200 0x7fd43c0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.812+0000 7fd441d2f640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c0d2c60 0x7fd43c14b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.812+0000 7fd441d2f640 1 --2- 172.21.15.29:0/2744736808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd43c0cf690 0x7fd43c0cdbf0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.812+0000 7fd441d2f640 1 -- 172.21.15.29:0/2744736808 >> 172.21.15.29:0/2744736808 conn(0x7fd43c09d280 msgr2=0x7fd43c0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:36.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.812+0000 7fd441d2f640 1 -- 172.21.15.29:0/2744736808 shutdown_connections 2024-04-25T00:36:36.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.812+0000 7fd441d2f640 1 -- 172.21.15.29:0/2744736808 wait complete. 2024-04-25T00:36:36.831 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:36:36.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.948+0000 7f77cf305640 1 Processor -- start 2024-04-25T00:36:36.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.948+0000 7f77cf305640 1 -- start start 2024-04-25T00:36:36.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.948+0000 7f77cf305640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f77c815e200 0x7f77c80d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.948+0000 7f77cf305640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c80cfc10 con 0x7f77c80d2c60 2024-04-25T00:36:36.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.948+0000 7f77cf305640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c80cfef0 con 0x7f77c80cf690 2024-04-25T00:36:36.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.948+0000 7f77cf305640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f77c80b38d0 con 0x7f77c815e200 2024-04-25T00:36:36.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.948+0000 7f77cd0a1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f77c815e200 0x7f77c80d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.948+0000 7f77cd0a1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f77c815e200 0x7f77c80d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38384/0 (socket says 172.21.15.29:38384) 2024-04-25T00:36:36.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.948+0000 7f77cd8a2640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f77c80d2c60 0x7f77c80b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38014/0 (socket says 172.21.15.29:38014) 2024-04-25T00:36:36.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.948+0000 7f77cd0a1640 1 -- 172.21.15.29:0/3941584987 learned_addr learned my addr 172.21.15.29:0/3941584987 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:36.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77cd0a1640 1 -- 172.21.15.29:0/3941584987 >> v1:172.21.15.157:6789/0 conn(0x7f77c80cf690 legacy=0x7f77c80d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:36.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77c7fff640 1 -- 172.21.15.29:0/3941584987 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2508828431 0 0) 0x7f77c80cfc10 con 0x7f77c80d2c60 2024-04-25T00:36:36.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77cd0a1640 1 -- 172.21.15.29:0/3941584987 >> v1:172.21.15.29:6789/0 conn(0x7f77c80d2c60 legacy=0x7f77c80b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77cd0a1640 1 -- 172.21.15.29:0/3941584987 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c80b3b20 con 0x7f77c815e200 2024-04-25T00:36:36.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77cd0a1640 1 --2- 172.21.15.29:0/3941584987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77c815e200 0x7f77c80d47f0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009e50 tx=0x7f77b801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579e92ab78e5888c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:36.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77c7fff640 1 -- 172.21.15.29:0/3941584987 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b80a58a0 con 0x7f77c815e200 2024-04-25T00:36:36.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77c7fff640 1 -- 172.21.15.29:0/3941584987 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b80a5a40 con 0x7f77c815e200 2024-04-25T00:36:36.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77c7fff640 1 -- 172.21.15.29:0/3941584987 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b80a5e10 con 0x7f77c815e200 2024-04-25T00:36:36.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77cf305640 1 -- 172.21.15.29:0/3941584987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77c815e200 msgr2=0x7f77c80d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77cf305640 1 --2- 172.21.15.29:0/3941584987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77c815e200 0x7f77c80d47f0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009e50 tx=0x7f77b801c9c0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77cf305640 1 -- 172.21.15.29:0/3941584987 shutdown_connections 2024-04-25T00:36:36.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77cf305640 1 --2- 172.21.15.29:0/3941584987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77c815e200 0x7f77c80d47f0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77cf305640 1 -- 172.21.15.29:0/3941584987 >> 172.21.15.29:0/3941584987 conn(0x7f77c809d280 msgr2=0x7f77c80d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:36.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77cf305640 1 -- 172.21.15.29:0/3941584987 shutdown_connections 2024-04-25T00:36:36.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.952+0000 7f77cf305640 1 -- 172.21.15.29:0/3941584987 wait complete. 2024-04-25T00:36:36.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cf305640 1 Processor -- start 2024-04-25T00:36:36.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cf305640 1 -- start start 2024-04-25T00:36:36.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cf305640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77c80cf690 0x7f77c81f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cf305640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77c80d2c60 0x7f77c80cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cf305640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77c80c9600 0x7f77c8159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cf305640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f77c80b53c0 con 0x7f77c80d2c60 2024-04-25T00:36:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cf305640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77c80b5240 con 0x7f77c80c9600 2024-04-25T00:36:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cf305640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f77c80b5540 con 0x7f77c80cf690 2024-04-25T00:36:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cc8a0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77c80d2c60 0x7f77c80cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cc8a0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77c80d2c60 0x7f77c80cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38390/0 (socket says 172.21.15.29:38390) 2024-04-25T00:36:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cc8a0640 1 -- 172.21.15.29:0/2201160292 learned_addr learned my addr 172.21.15.29:0/2201160292 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cc8a0640 1 -- 172.21.15.29:0/2201160292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77c80cf690 msgr2=0x7f77c81f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cc8a0640 1 --2- 172.21.15.29:0/2201160292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77c80cf690 0x7f77c81f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cc8a0640 1 -- 172.21.15.29:0/2201160292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77c80c9600 msgr2=0x7f77c8159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cc8a0640 1 --2- 172.21.15.29:0/2201160292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77c80c9600 0x7f77c8159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cc8a0640 1 -- 172.21.15.29:0/2201160292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c80ca050 con 0x7f77c80d2c60 2024-04-25T00:36:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cc8a0640 1 --2- 172.21.15.29:0/2201160292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77c80d2c60 0x7f77c80cb2a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00b410 tx=0x7f77bc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77c5ffb640 1 -- 172.21.15.29:0/2201160292 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc015030 con 0x7f77c80d2c60 2024-04-25T00:36:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77c5ffb640 1 -- 172.21.15.29:0/2201160292 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77bc004020 con 0x7f77c80d2c60 2024-04-25T00:36:36.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cf305640 1 -- 172.21.15.29:0/2201160292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c80b7d60 con 0x7f77c80d2c60 2024-04-25T00:36:36.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77c5ffb640 1 -- 172.21.15.29:0/2201160292 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc004450 con 0x7f77c80d2c60 2024-04-25T00:36:36.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cf305640 1 -- 172.21.15.29:0/2201160292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c80b8320 con 0x7f77c80d2c60 2024-04-25T00:36:36.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cf305640 1 -- 172.21.15.29:0/2201160292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c80b8520 con 0x7f77c80d2c60 2024-04-25T00:36:36.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77c5ffb640 1 -- 172.21.15.29:0/2201160292 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f77bc018070 con 0x7f77c80d2c60 2024-04-25T00:36:36.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77c5ffb640 1 --2- 172.21.15.29:0/2201160292 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f77b4067a10 0x7f77b4069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:36.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77c5ffb640 1 -- 172.21.15.29:0/2201160292 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 4857+0+0 (secure 0 0 0) 0x7f77bc0856b0 con 0x7f77c80d2c60 2024-04-25T00:36:36.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.956+0000 7f77cd0a1640 1 --2- 172.21.15.29:0/2201160292 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f77b4067a10 0x7f77b4069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:36.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.960+0000 7f77cd0a1640 1 --2- 172.21.15.29:0/2201160292 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f77b4067a10 0x7f77b4069ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f77b80ae000 tx=0x7f77b8004ac0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:36.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:36.964+0000 7f77c5ffb640 1 -- 172.21.15.29:0/2201160292 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f77c80b8520 con 0x7f77c80d2c60 2024-04-25T00:36:37.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.120+0000 7f77cf305640 1 -- 172.21.15.29:0/2201160292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f77c8069270 con 0x7f77c80d2c60 2024-04-25T00:36:37.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.120+0000 7f77c5ffb640 1 -- 172.21.15.29:0/2201160292 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) ==== 84+0+65 (secure 0 0 0) 0x7f77bc0563a0 con 0x7f77c80d2c60 2024-04-25T00:36:37.123 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:37.124 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:36:37.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:37.124 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-04-25T00:36:37.124 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": ".mgr" 2024-04-25T00:36:37.124 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:37.124 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:36:37.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.124+0000 7f77cf305640 1 -- 172.21.15.29:0/2201160292 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f77b4067a10 msgr2=0x7f77b4069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:37.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.124+0000 7f77cf305640 1 --2- 172.21.15.29:0/2201160292 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f77b4067a10 0x7f77b4069ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f77b80ae000 tx=0x7f77b8004ac0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.124+0000 7f77cf305640 1 -- 172.21.15.29:0/2201160292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77c80d2c60 msgr2=0x7f77c80cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:37.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.124+0000 7f77cf305640 1 --2- 172.21.15.29:0/2201160292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77c80d2c60 0x7f77c80cb2a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00b410 tx=0x7f77bc00b8d0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.124+0000 7f77cf305640 1 -- 172.21.15.29:0/2201160292 shutdown_connections 2024-04-25T00:36:37.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.124+0000 7f77cf305640 1 --2- 172.21.15.29:0/2201160292 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f77b4067a10 0x7f77b4069ed0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.124+0000 7f77cf305640 1 --2- 172.21.15.29:0/2201160292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77c80c9600 0x7f77c8159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.124+0000 7f77cf305640 1 --2- 172.21.15.29:0/2201160292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77c80d2c60 0x7f77c80cb2a0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.124+0000 7f77cf305640 1 --2- 172.21.15.29:0/2201160292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77c80cf690 0x7f77c81f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.124+0000 7f77cf305640 1 -- 172.21.15.29:0/2201160292 >> 172.21.15.29:0/2201160292 conn(0x7f77c809d280 msgr2=0x7f77c806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:37.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.124+0000 7f77cf305640 1 -- 172.21.15.29:0/2201160292 shutdown_connections 2024-04-25T00:36:37.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.124+0000 7f77cf305640 1 -- 172.21.15.29:0/2201160292 wait complete. 2024-04-25T00:36:37.141 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:36:37.141 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:36:37.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.264+0000 7fef6cfb3640 1 Processor -- start 2024-04-25T00:36:37.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.264+0000 7fef6cfb3640 1 -- start start 2024-04-25T00:36:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.264+0000 7fef6cfb3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fef6815e560 0x7fef680d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.264+0000 7fef6cfb3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef680d2680 0x7fef680d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.264+0000 7fef6cfb3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef680cfed0 con 0x7fef6815e180 2024-04-25T00:36:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.264+0000 7fef6cfb3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fef680b7100 con 0x7fef6815e560 2024-04-25T00:36:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.264+0000 7fef6cfb3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fef680b73e0 con 0x7fef680d2680 2024-04-25T00:36:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.264+0000 7fef6659c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fef6815e180 0x7fef680cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38016/0 (socket says 172.21.15.29:38016) 2024-04-25T00:36:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.264+0000 7fef65d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fef6815e560 0x7fef680d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.264+0000 7fef6659c640 1 -- 172.21.15.29:0/1067557104 learned_addr learned my addr 172.21.15.29:0/1067557104 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:37.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef66d9d640 1 --2- 172.21.15.29:0/1067557104 >> v2:172.21.15.157:3300/0 conn(0x7fef680d2680 0x7fef680d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:37.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef65d9b640 1 -- 172.21.15.29:0/1067557104 >> v2:172.21.15.157:3300/0 conn(0x7fef680d2680 msgr2=0x7fef680d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:37.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef65d9b640 1 --2- 172.21.15.29:0/1067557104 >> v2:172.21.15.157:3300/0 conn(0x7fef680d2680 0x7fef680d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef65d9b640 1 -- 172.21.15.29:0/1067557104 >> v1:172.21.15.29:6789/0 conn(0x7fef6815e180 legacy=0x7fef680cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:37.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef6559a640 1 -- 172.21.15.29:0/1067557104 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3831018462 0 0) 0x7fef680cfed0 con 0x7fef6815e180 2024-04-25T00:36:37.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef65d9b640 1 -- 172.21.15.29:0/1067557104 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef680b76c0 con 0x7fef6815e560 2024-04-25T00:36:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef66d9d640 1 --2- 172.21.15.29:0/1067557104 >> v2:172.21.15.157:3300/0 conn(0x7fef680d2680 0x7fef680d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef65d9b640 1 --2- 172.21.15.29:0/1067557104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6815e560 0x7fef680d20e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fef680ceac0 tx=0x7fef5801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c326a9733542ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef6559a640 1 -- 172.21.15.29:0/1067557104 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef580b5070 con 0x7fef6815e560 2024-04-25T00:36:37.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef6559a640 1 -- 172.21.15.29:0/1067557104 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef580a8970 con 0x7fef6815e560 2024-04-25T00:36:37.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef6559a640 1 -- 172.21.15.29:0/1067557104 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef580a8d40 con 0x7fef6815e560 2024-04-25T00:36:37.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef6cfb3640 1 -- 172.21.15.29:0/1067557104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6815e560 msgr2=0x7fef680d20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:37.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef6cfb3640 1 --2- 172.21.15.29:0/1067557104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6815e560 0x7fef680d20e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fef680ceac0 tx=0x7fef5801ec70 comp rx=0 tx=0).stop 2024-04-25T00:36:37.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef6cfb3640 1 -- 172.21.15.29:0/1067557104 shutdown_connections 2024-04-25T00:36:37.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef6cfb3640 1 --2- 172.21.15.29:0/1067557104 >> v2:172.21.15.157:3300/0 conn(0x7fef680d2680 0x7fef680d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef6cfb3640 1 --2- 172.21.15.29:0/1067557104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6815e560 0x7fef680d20e0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef6cfb3640 1 -- 172.21.15.29:0/1067557104 >> 172.21.15.29:0/1067557104 conn(0x7fef680d40a0 msgr2=0x7fef680d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:37.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef6cfb3640 1 -- 172.21.15.29:0/1067557104 shutdown_connections 2024-04-25T00:36:37.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.268+0000 7fef6cfb3640 1 -- 172.21.15.29:0/1067557104 wait complete. 2024-04-25T00:36:37.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef6cfb3640 1 Processor -- start 2024-04-25T00:36:37.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef6cfb3640 1 -- start start 2024-04-25T00:36:37.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef6cfb3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef680d2680 0x7fef68150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:37.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef6cfb3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6815e180 0x7fef6814d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:37.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef6cfb3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6815e560 0x7fef681f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:37.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef6cfb3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fef680b7d60 con 0x7fef6815e560 2024-04-25T00:36:37.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef6cfb3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef680b7be0 con 0x7fef680d2680 2024-04-25T00:36:37.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef6cfb3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fef680b7ee0 con 0x7fef6815e180 2024-04-25T00:36:37.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef66d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6815e560 0x7fef681f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:37.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef66d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6815e560 0x7fef681f67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38410/0 (socket says 172.21.15.29:38410) 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef66d9d640 1 -- 172.21.15.29:0/4286129689 learned_addr learned my addr 172.21.15.29:0/4286129689 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef65d9b640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6815e180 0x7fef6814d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef6659c640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef680d2680 0x7fef68150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef66d9d640 1 -- 172.21.15.29:0/4286129689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6815e180 msgr2=0x7fef6814d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef66d9d640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6815e180 0x7fef6814d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef66d9d640 1 -- 172.21.15.29:0/4286129689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef680d2680 msgr2=0x7fef68150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef66d9d640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef680d2680 0x7fef68150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef66d9d640 1 -- 172.21.15.29:0/4286129689 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6814dfd0 con 0x7fef6815e560 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef65d9b640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6815e180 0x7fef6814d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef6659c640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef680d2680 0x7fef68150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef66d9d640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6815e560 0x7fef681f67a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fef6000cef0 tx=0x7fef6000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef56ffd640 1 -- 172.21.15.29:0/4286129689 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6001a070 con 0x7fef6815e560 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef56ffd640 1 -- 172.21.15.29:0/4286129689 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef6000d860 con 0x7fef6815e560 2024-04-25T00:36:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef6cfb3640 1 -- 172.21.15.29:0/4286129689 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6814d730 con 0x7fef6815e560 2024-04-25T00:36:37.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef56ffd640 1 -- 172.21.15.29:0/4286129689 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6000dc50 con 0x7fef6815e560 2024-04-25T00:36:37.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef6cfb3640 1 -- 172.21.15.29:0/4286129689 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6814dcf0 con 0x7fef6815e560 2024-04-25T00:36:37.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef6cfb3640 1 -- 172.21.15.29:0/4286129689 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef680692e0 con 0x7fef6815e560 2024-04-25T00:36:37.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.272+0000 7fef56ffd640 1 -- 172.21.15.29:0/4286129689 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fef6001d070 con 0x7fef6815e560 2024-04-25T00:36:37.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.276+0000 7fef56ffd640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fef480679e0 0x7fef48069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:37.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.276+0000 7fef6659c640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fef480679e0 0x7fef48069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:37.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.276+0000 7fef56ffd640 1 -- 172.21.15.29:0/4286129689 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 4857+0+0 (secure 0 0 0) 0x7fef60086370 con 0x7fef6815e560 2024-04-25T00:36:37.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.276+0000 7fef6659c640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fef480679e0 0x7fef48069ea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef5c000f70 tx=0x7fef5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:37.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.280+0000 7fef56ffd640 1 -- 172.21.15.29:0/4286129689 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fef600571b0 con 0x7fef6815e560 2024-04-25T00:36:37.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.436+0000 7fef6cfb3640 1 -- 172.21.15.29:0/4286129689 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fef680caf10 con 0x7fef6815e560 2024-04-25T00:36:37.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.788+0000 7fef56ffd640 1 -- 172.21.15.29:0/4286129689 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) ==== 138+0+0 (secure 0 0 0) 0x7fef60056ab0 con 0x7fef6815e560 2024-04-25T00:36:37.791 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-04-25T00:36:37.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.792+0000 7fef6cfb3640 1 -- 172.21.15.29:0/4286129689 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fef480679e0 msgr2=0x7fef48069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:37.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.792+0000 7fef6cfb3640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fef480679e0 0x7fef48069ea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef5c000f70 tx=0x7fef5c009040 comp rx=0 tx=0).stop 2024-04-25T00:36:37.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.792+0000 7fef6cfb3640 1 -- 172.21.15.29:0/4286129689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6815e560 msgr2=0x7fef681f67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:37.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.792+0000 7fef6cfb3640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6815e560 0x7fef681f67a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fef6000cef0 tx=0x7fef6000a710 comp rx=0 tx=0).stop 2024-04-25T00:36:37.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.792+0000 7fef6cfb3640 1 -- 172.21.15.29:0/4286129689 shutdown_connections 2024-04-25T00:36:37.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.792+0000 7fef6cfb3640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fef480679e0 0x7fef48069ea0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.792+0000 7fef6cfb3640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef6815e560 0x7fef681f67a0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.792+0000 7fef6cfb3640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6815e180 0x7fef6814d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.792+0000 7fef6cfb3640 1 --2- 172.21.15.29:0/4286129689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef680d2680 0x7fef68150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.792+0000 7fef6cfb3640 1 -- 172.21.15.29:0/4286129689 >> 172.21.15.29:0/4286129689 conn(0x7fef680d40a0 msgr2=0x7fef6815d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:37.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.792+0000 7fef6cfb3640 1 -- 172.21.15.29:0/4286129689 shutdown_connections 2024-04-25T00:36:37.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.792+0000 7fef6cfb3640 1 -- 172.21.15.29:0/4286129689 wait complete. 2024-04-25T00:36:37.820 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:36:37.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe5375cb640 1 Processor -- start 2024-04-25T00:36:37.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe5375cb640 1 -- start start 2024-04-25T00:36:37.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe5375cb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe53015e330 0x7fe53006db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:37.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe5375cb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe53006b210 0x7fe53006b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:37.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe5375cb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5300cfe90 con 0x7fe53006f270 2024-04-25T00:36:37.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe5375cb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe5300d0170 con 0x7fe53015e330 2024-04-25T00:36:37.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe5375cb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe5300d0450 con 0x7fe53006b210 2024-04-25T00:36:37.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe534b66640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe53015e330 0x7fe53006db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:37.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe534b66640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe53015e330 0x7fe53006db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36838/0 (socket says 172.21.15.29:36838) 2024-04-25T00:36:37.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe534b66640 1 -- 172.21.15.29:0/2049166937 learned_addr learned my addr 172.21.15.29:0/2049166937 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:37.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe535b68640 1 --2- 172.21.15.29:0/2049166937 >> v2:172.21.15.157:3300/0 conn(0x7fe53006b210 0x7fe53006b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:37.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe534b66640 1 -- 172.21.15.29:0/2049166937 >> v2:172.21.15.157:3300/0 conn(0x7fe53006b210 msgr2=0x7fe53006b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:37.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe534b66640 1 --2- 172.21.15.29:0/2049166937 >> v2:172.21.15.157:3300/0 conn(0x7fe53006b210 0x7fe53006b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe534b66640 1 -- 172.21.15.29:0/2049166937 >> v1:172.21.15.157:6789/0 conn(0x7fe53006f270 legacy=0x7fe53006f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:37.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.956+0000 7fe534b66640 1 -- 172.21.15.29:0/2049166937 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5300d0730 con 0x7fe53015e330 2024-04-25T00:36:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.960+0000 7fe534b66640 1 --2- 172.21.15.29:0/2049166937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe53015e330 0x7fe53006db90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe5300cb220 tx=0x7fe52401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610f0d2742166e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.960+0000 7fe523fff640 1 -- 172.21.15.29:0/2049166937 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5240b0070 con 0x7fe53015e330 2024-04-25T00:36:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.960+0000 7fe523fff640 1 -- 172.21.15.29:0/2049166937 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5240ab600 con 0x7fe53015e330 2024-04-25T00:36:37.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.960+0000 7fe523fff640 1 -- 172.21.15.29:0/2049166937 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5240ab9d0 con 0x7fe53015e330 2024-04-25T00:36:37.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.960+0000 7fe5375cb640 1 -- 172.21.15.29:0/2049166937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe53015e330 msgr2=0x7fe53006db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:37.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.960+0000 7fe5375cb640 1 --2- 172.21.15.29:0/2049166937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe53015e330 0x7fe53006db90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe5300cb220 tx=0x7fe52401c450 comp rx=0 tx=0).stop 2024-04-25T00:36:37.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.960+0000 7fe5375cb640 1 -- 172.21.15.29:0/2049166937 shutdown_connections 2024-04-25T00:36:37.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.960+0000 7fe5375cb640 1 --2- 172.21.15.29:0/2049166937 >> v2:172.21.15.157:3300/0 conn(0x7fe53006b210 0x7fe53006b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.960+0000 7fe5375cb640 1 --2- 172.21.15.29:0/2049166937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe53015e330 0x7fe53006db90 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.960+0000 7fe5375cb640 1 -- 172.21.15.29:0/2049166937 >> 172.21.15.29:0/2049166937 conn(0x7fe5300b2920 msgr2=0x7fe5300b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.960+0000 7fe5375cb640 1 -- 172.21.15.29:0/2049166937 shutdown_connections 2024-04-25T00:36:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.960+0000 7fe5375cb640 1 -- 172.21.15.29:0/2049166937 wait complete. 2024-04-25T00:36:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe5375cb640 1 Processor -- start 2024-04-25T00:36:37.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe5375cb640 1 -- start start 2024-04-25T00:36:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe5375cb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe53006b210 0x7fe5300cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe5375cb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe53006f270 0x7fe5300af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe5375cb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5300cbf80 0x7fe5300af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe5375cb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe53016af40 con 0x7fe5300cbf80 2024-04-25T00:36:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe5375cb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe53015f460 con 0x7fe53006b210 2024-04-25T00:36:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe5375cb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe53016b0c0 con 0x7fe53006f270 2024-04-25T00:36:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe535b68640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5300cbf80 0x7fe5300af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:37.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe535b68640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5300cbf80 0x7fe5300af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36840/0 (socket says 172.21.15.29:36840) 2024-04-25T00:36:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe535b68640 1 -- 172.21.15.29:0/326597295 learned_addr learned my addr 172.21.15.29:0/326597295 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe534b66640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe53006f270 0x7fe5300af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe535367640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe53006b210 0x7fe5300cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe535b68640 1 -- 172.21.15.29:0/326597295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe53006f270 msgr2=0x7fe5300af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe535b68640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe53006f270 0x7fe5300af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe535b68640 1 -- 172.21.15.29:0/326597295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe53006b210 msgr2=0x7fe5300cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe535b68640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe53006b210 0x7fe5300cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe535b68640 1 -- 172.21.15.29:0/326597295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5300b5690 con 0x7fe5300cbf80 2024-04-25T00:36:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe535b68640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5300cbf80 0x7fe5300af730 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fe528007e60 tx=0x7fe52800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe521ffb640 1 -- 172.21.15.29:0/326597295 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528014420 con 0x7fe5300cbf80 2024-04-25T00:36:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe521ffb640 1 -- 172.21.15.29:0/326597295 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe528019070 con 0x7fe5300cbf80 2024-04-25T00:36:37.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe521ffb640 1 -- 172.21.15.29:0/326597295 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528014820 con 0x7fe5300cbf80 2024-04-25T00:36:37.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe5375cb640 1 -- 172.21.15.29:0/326597295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5300b5970 con 0x7fe5300cbf80 2024-04-25T00:36:37.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe5375cb640 1 -- 172.21.15.29:0/326597295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5300b5f30 con 0x7fe5300cbf80 2024-04-25T00:36:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.964+0000 7fe5375cb640 1 -- 172.21.15.29:0/326597295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe530069270 con 0x7fe5300cbf80 2024-04-25T00:36:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.968+0000 7fe521ffb640 1 -- 172.21.15.29:0/326597295 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fe5280149c0 con 0x7fe5300cbf80 2024-04-25T00:36:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.968+0000 7fe521ffb640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe5140679e0 0x7fe514069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.968+0000 7fe521ffb640 1 -- 172.21.15.29:0/326597295 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5227+0+0 (secure 0 0 0) 0x7fe5280873e0 con 0x7fe5300cbf80 2024-04-25T00:36:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.968+0000 7fe535367640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe5140679e0 0x7fe514069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:37.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.968+0000 7fe535367640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe5140679e0 0x7fe514069ea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe52c004e30 tx=0x7fe52c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:37.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:37.972+0000 7fe521ffb640 1 -- 172.21.15.29:0/326597295 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe528057fd0 con 0x7fe5300cbf80 2024-04-25T00:36:38.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.132+0000 7fe5375cb640 1 -- 172.21.15.29:0/326597295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe5300b0570 con 0x7fe5300cbf80 2024-04-25T00:36:38.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.800+0000 7fe521ffb640 1 -- 172.21.15.29:0/326597295 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) ==== 170+0+0 (secure 0 0 0) 0x7fe5280578d0 con 0x7fe5300cbf80 2024-04-25T00:36:38.802 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-04-25T00:36:38.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.804+0000 7fe5375cb640 1 -- 172.21.15.29:0/326597295 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe5140679e0 msgr2=0x7fe514069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.804+0000 7fe5375cb640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe5140679e0 0x7fe514069ea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe52c004e30 tx=0x7fe52c009040 comp rx=0 tx=0).stop 2024-04-25T00:36:38.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.804+0000 7fe5375cb640 1 -- 172.21.15.29:0/326597295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5300cbf80 msgr2=0x7fe5300af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.804+0000 7fe5375cb640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5300cbf80 0x7fe5300af730 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fe528007e60 tx=0x7fe52800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.804+0000 7fe5375cb640 1 -- 172.21.15.29:0/326597295 shutdown_connections 2024-04-25T00:36:38.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.804+0000 7fe5375cb640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe5140679e0 0x7fe514069ea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.804+0000 7fe5375cb640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5300cbf80 0x7fe5300af730 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.804+0000 7fe5375cb640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe53006f270 0x7fe5300af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.804+0000 7fe5375cb640 1 --2- 172.21.15.29:0/326597295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe53006b210 0x7fe5300cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.804+0000 7fe5375cb640 1 -- 172.21.15.29:0/326597295 >> 172.21.15.29:0/326597295 conn(0x7fe5300b2920 msgr2=0x7fe53015fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:38.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.808+0000 7fe5375cb640 1 -- 172.21.15.29:0/326597295 shutdown_connections 2024-04-25T00:36:38.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.816+0000 7fe5375cb640 1 -- 172.21.15.29:0/326597295 wait complete. 2024-04-25T00:36:38.850 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:36:38.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa783113640 1 Processor -- start 2024-04-25T00:36:38.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa783113640 1 -- start start 2024-04-25T00:36:38.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa783113640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa77c0b4780 0x7fa77c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa783113640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa77c0cc240 0x7fa77c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa783113640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa77c0b3720 con 0x7fa77c15e8d0 2024-04-25T00:36:38.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa783113640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa77c0b42f0 con 0x7fa77c0b4780 2024-04-25T00:36:38.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa783113640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa77c0b39c0 con 0x7fa77c0cc240 2024-04-25T00:36:38.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa77bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa77c0b4780 0x7fa77c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa77bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa77c0b4780 0x7fa77c15f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36856/0 (socket says 172.21.15.29:36856) 2024-04-25T00:36:38.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa77bfff640 1 -- 172.21.15.29:0/335418273 learned_addr learned my addr 172.21.15.29:0/335418273 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:38.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa7816b0640 1 --2- 172.21.15.29:0/335418273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa77c0cc240 0x7fa77c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa77bfff640 1 -- 172.21.15.29:0/335418273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa77c0cc240 msgr2=0x7fa77c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:38.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa77bfff640 1 --2- 172.21.15.29:0/335418273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa77c0cc240 0x7fa77c15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa77bfff640 1 -- 172.21.15.29:0/335418273 >> v1:172.21.15.157:6789/0 conn(0x7fa77c15e8d0 legacy=0x7fa77c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:38.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.976+0000 7fa77bfff640 1 -- 172.21.15.29:0/335418273 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c0b3ca0 con 0x7fa77c0b4780 2024-04-25T00:36:38.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa77bfff640 1 --2- 172.21.15.29:0/335418273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77c0b4780 0x7fa77c15f3c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0b4470 tx=0x7fa76401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf07ba4dac9495b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:38.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa77b7fe640 1 -- 172.21.15.29:0/335418273 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7640b2070 con 0x7fa77c0b4780 2024-04-25T00:36:38.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa77b7fe640 1 -- 172.21.15.29:0/335418273 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7640ad640 con 0x7fa77c0b4780 2024-04-25T00:36:38.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa77b7fe640 1 -- 172.21.15.29:0/335418273 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7640ada10 con 0x7fa77c0b4780 2024-04-25T00:36:38.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 -- 172.21.15.29:0/335418273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77c0b4780 msgr2=0x7fa77c15f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 --2- 172.21.15.29:0/335418273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77c0b4780 0x7fa77c15f3c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0b4470 tx=0x7fa76401c500 comp rx=0 tx=0).stop 2024-04-25T00:36:38.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 -- 172.21.15.29:0/335418273 shutdown_connections 2024-04-25T00:36:38.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 --2- 172.21.15.29:0/335418273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa77c0cc240 0x7fa77c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 --2- 172.21.15.29:0/335418273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77c0b4780 0x7fa77c15f3c0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 -- 172.21.15.29:0/335418273 >> 172.21.15.29:0/335418273 conn(0x7fa77c0b1870 msgr2=0x7fa77c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:38.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 -- 172.21.15.29:0/335418273 shutdown_connections 2024-04-25T00:36:38.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 -- 172.21.15.29:0/335418273 wait complete. 2024-04-25T00:36:38.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 Processor -- start 2024-04-25T00:36:38.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 -- start start 2024-04-25T00:36:38.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa77c0b4780 0x7fa77c15a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77c0cc240 0x7fa77c0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c15e8d0 0x7fa77c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c15d6a0 con 0x7fa77c0cc240 2024-04-25T00:36:38.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c15d520 con 0x7fa77c15e8d0 2024-04-25T00:36:38.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa783113640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa77c167c70 con 0x7fa77c0b4780 2024-04-25T00:36:38.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa77bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77c0cc240 0x7fa77c0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa77bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77c0cc240 0x7fa77c0cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36864/0 (socket says 172.21.15.29:36864) 2024-04-25T00:36:38.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa77bfff640 1 -- 172.21.15.29:0/2471761316 learned_addr learned my addr 172.21.15.29:0/2471761316 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:38.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa7816b0640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c15e8d0 0x7fa77c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa780eaf640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa77c0b4780 0x7fa77c15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa77bfff640 1 -- 172.21.15.29:0/2471761316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa77c0b4780 msgr2=0x7fa77c15a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa77bfff640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa77c0b4780 0x7fa77c15a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa77bfff640 1 -- 172.21.15.29:0/2471761316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c15e8d0 msgr2=0x7fa77c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:38.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa77bfff640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c15e8d0 0x7fa77c15b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:38.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.980+0000 7fa77bfff640 1 -- 172.21.15.29:0/2471761316 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c0ce2d0 con 0x7fa77c0cc240 2024-04-25T00:36:38.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.984+0000 7fa77bfff640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77c0cc240 0x7fa77c0cd880 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa76401cda0 tx=0x7fa7640192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:38.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.984+0000 7fa7797fa640 1 -- 172.21.15.29:0/2471761316 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7640b2040 con 0x7fa77c0cc240 2024-04-25T00:36:38.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.984+0000 7fa7797fa640 1 -- 172.21.15.29:0/2471761316 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7640bb070 con 0x7fa77c0cc240 2024-04-25T00:36:38.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.984+0000 7fa7797fa640 1 -- 172.21.15.29:0/2471761316 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7640b6680 con 0x7fa77c0cc240 2024-04-25T00:36:38.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.984+0000 7fa783113640 1 -- 172.21.15.29:0/2471761316 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77c0ce5b0 con 0x7fa77c0cc240 2024-04-25T00:36:38.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.984+0000 7fa783113640 1 -- 172.21.15.29:0/2471761316 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77c140cd0 con 0x7fa77c0cc240 2024-04-25T00:36:38.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.984+0000 7fa783113640 1 -- 172.21.15.29:0/2471761316 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa77c0cead0 con 0x7fa77c0cc240 2024-04-25T00:36:38.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.984+0000 7fa7797fa640 1 -- 172.21.15.29:0/2471761316 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fa7640b6820 con 0x7fa77c0cc240 2024-04-25T00:36:38.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.984+0000 7fa7797fa640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa75c067a10 0x7fa75c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:38.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.984+0000 7fa7797fa640 1 -- 172.21.15.29:0/2471761316 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5609+0+0 (secure 0 0 0) 0x7fa764120990 con 0x7fa77c0cc240 2024-04-25T00:36:38.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.984+0000 7fa780eaf640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa75c067a10 0x7fa75c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:38.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.984+0000 7fa780eaf640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa75c067a10 0x7fa75c069ed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00b810 tx=0x7fa76c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:38.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:38.988+0000 7fa7797fa640 1 -- 172.21.15.29:0/2471761316 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa77c0cead0 con 0x7fa77c0cc240 2024-04-25T00:36:39.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.188+0000 7fa783113640 1 -- 172.21.15.29:0/2471761316 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa77c069270 con 0x7fa77c0cc240 2024-04-25T00:36:39.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.824+0000 7fa7797fa640 1 -- 172.21.15.29:0/2471761316 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v49) ==== 174+0+0 (secure 0 0 0) 0x7fa7640160f0 con 0x7fa77c0cc240 2024-04-25T00:36:39.831 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 12 and data pool 13 2024-04-25T00:36:39.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.848+0000 7fa783113640 1 -- 172.21.15.29:0/2471761316 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa75c067a10 msgr2=0x7fa75c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:39.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.848+0000 7fa783113640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa75c067a10 0x7fa75c069ed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00b810 tx=0x7fa76c008040 comp rx=0 tx=0).stop 2024-04-25T00:36:39.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.848+0000 7fa783113640 1 -- 172.21.15.29:0/2471761316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77c0cc240 msgr2=0x7fa77c0cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:39.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.848+0000 7fa783113640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77c0cc240 0x7fa77c0cd880 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa76401cda0 tx=0x7fa7640192a0 comp rx=0 tx=0).stop 2024-04-25T00:36:39.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.848+0000 7fa783113640 1 -- 172.21.15.29:0/2471761316 shutdown_connections 2024-04-25T00:36:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.848+0000 7fa783113640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa75c067a10 0x7fa75c069ed0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.848+0000 7fa783113640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77c15e8d0 0x7fa77c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.848+0000 7fa783113640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77c0cc240 0x7fa77c0cd880 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.848+0000 7fa783113640 1 --2- 172.21.15.29:0/2471761316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa77c0b4780 0x7fa77c15a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.848+0000 7fa783113640 1 -- 172.21.15.29:0/2471761316 >> 172.21.15.29:0/2471761316 conn(0x7fa77c0b1870 msgr2=0x7fa77c0adf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.848+0000 7fa783113640 1 -- 172.21.15.29:0/2471761316 shutdown_connections 2024-04-25T00:36:39.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.848+0000 7fa783113640 1 -- 172.21.15.29:0/2471761316 wait complete. 2024-04-25T00:36:39.876 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:36:39.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.992+0000 7fa486df4640 1 Processor -- start 2024-04-25T00:36:39.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.992+0000 7fa486df4640 1 -- start start 2024-04-25T00:36:39.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.992+0000 7fa486df4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa480156420 0x7fa48015ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:39.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.992+0000 7fa486df4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48015fa80 0x7fa48006b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:39.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.992+0000 7fa486df4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa48015c680 con 0x7fa48015ff10 2024-04-25T00:36:39.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.992+0000 7fa486df4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa48015d3a0 con 0x7fa480156420 2024-04-25T00:36:39.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.992+0000 7fa486df4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa480160440 con 0x7fa48015fa80 2024-04-25T00:36:39.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa484b90640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa480156420 0x7fa48015ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:39.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa484b90640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa480156420 0x7fa48015ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36874/0 (socket says 172.21.15.29:36874) 2024-04-25T00:36:39.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa484b90640 1 -- 172.21.15.29:0/292754307 learned_addr learned my addr 172.21.15.29:0/292754307 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:39.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa477fff640 1 --2- 172.21.15.29:0/292754307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48015fa80 0x7fa48006b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:39.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa484b90640 1 -- 172.21.15.29:0/292754307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48015fa80 msgr2=0x7fa48006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:39.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa484b90640 1 --2- 172.21.15.29:0/292754307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48015fa80 0x7fa48006b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:39.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa484b90640 1 -- 172.21.15.29:0/292754307 >> v1:172.21.15.157:6789/0 conn(0x7fa48015ff10 legacy=0x7fa4800cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:39.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa484b90640 1 -- 172.21.15.29:0/292754307 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa480160720 con 0x7fa480156420 2024-04-25T00:36:39.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa484b90640 1 --2- 172.21.15.29:0/292754307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa480156420 0x7fa48015ce60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009870 tx=0x7fa47c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88718f6f0d3b9a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:39.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa4777fe640 1 -- 172.21.15.29:0/292754307 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0a7070 con 0x7fa480156420 2024-04-25T00:36:39.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa4777fe640 1 -- 172.21.15.29:0/292754307 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47c0b0070 con 0x7fa480156420 2024-04-25T00:36:39.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa4777fe640 1 -- 172.21.15.29:0/292754307 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0ab890 con 0x7fa480156420 2024-04-25T00:36:40.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa486df4640 1 -- 172.21.15.29:0/292754307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa480156420 msgr2=0x7fa48015ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa486df4640 1 --2- 172.21.15.29:0/292754307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa480156420 0x7fa48015ce60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009870 tx=0x7fa47c01c560 comp rx=0 tx=0).stop 2024-04-25T00:36:40.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa486df4640 1 -- 172.21.15.29:0/292754307 shutdown_connections 2024-04-25T00:36:40.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa486df4640 1 --2- 172.21.15.29:0/292754307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48015fa80 0x7fa48006b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa486df4640 1 --2- 172.21.15.29:0/292754307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa480156420 0x7fa48015ce60 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa486df4640 1 -- 172.21.15.29:0/292754307 >> 172.21.15.29:0/292754307 conn(0x7fa4800d0c40 msgr2=0x7fa4800d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:40.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa486df4640 1 -- 172.21.15.29:0/292754307 shutdown_connections 2024-04-25T00:36:40.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:39.996+0000 7fa486df4640 1 -- 172.21.15.29:0/292754307 wait complete. 2024-04-25T00:36:40.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa486df4640 1 Processor -- start 2024-04-25T00:36:40.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa486df4640 1 -- start start 2024-04-25T00:36:40.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa486df4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa480156420 0x7fa480149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa486df4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48015fa80 0x7fa48015e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa486df4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa48015ff10 0x7fa4800c5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa486df4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa480070160 con 0x7fa480156420 2024-04-25T00:36:40.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa486df4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa48006bb60 con 0x7fa48015ff10 2024-04-25T00:36:40.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa486df4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa4800702e0 con 0x7fa48015fa80 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa484b90640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa480156420 0x7fa480149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa484b90640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa480156420 0x7fa480149250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36880/0 (socket says 172.21.15.29:36880) 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa484b90640 1 -- 172.21.15.29:0/4127813075 learned_addr learned my addr 172.21.15.29:0/4127813075 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa477fff640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48015fa80 0x7fa48015e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa485391640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa48015ff10 0x7fa4800c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa484b90640 1 -- 172.21.15.29:0/4127813075 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48015fa80 msgr2=0x7fa48015e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa484b90640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48015fa80 0x7fa48015e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa484b90640 1 -- 172.21.15.29:0/4127813075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa48015ff10 msgr2=0x7fa4800c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa484b90640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa48015ff10 0x7fa4800c5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa484b90640 1 -- 172.21.15.29:0/4127813075 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48015efa0 con 0x7fa480156420 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa485391640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa48015ff10 0x7fa4800c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa477fff640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48015fa80 0x7fa48015e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa484b90640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa480156420 0x7fa480149250 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0ad040 tx=0x7fa47c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:40.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa4757fa640 1 -- 172.21.15.29:0/4127813075 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0be5f0 con 0x7fa480156420 2024-04-25T00:36:40.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa4757fa640 1 -- 172.21.15.29:0/4127813075 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47c0be790 con 0x7fa480156420 2024-04-25T00:36:40.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa486df4640 1 -- 172.21.15.29:0/4127813075 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48015f280 con 0x7fa480156420 2024-04-25T00:36:40.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa4757fa640 1 -- 172.21.15.29:0/4127813075 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0beb80 con 0x7fa480156420 2024-04-25T00:36:40.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa486df4640 1 -- 172.21.15.29:0/4127813075 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa480154670 con 0x7fa480156420 2024-04-25T00:36:40.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.000+0000 7fa486df4640 1 -- 172.21.15.29:0/4127813075 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa480069270 con 0x7fa480156420 2024-04-25T00:36:40.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.004+0000 7fa4757fa640 1 -- 172.21.15.29:0/4127813075 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fa47c0c9470 con 0x7fa480156420 2024-04-25T00:36:40.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.004+0000 7fa4757fa640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa46c0679e0 0x7fa46c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.004+0000 7fa4757fa640 1 -- 172.21.15.29:0/4127813075 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fa47c120bd0 con 0x7fa480156420 2024-04-25T00:36:40.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.004+0000 7fa477fff640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa46c0679e0 0x7fa46c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.004+0000 7fa477fff640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa46c0679e0 0x7fa46c069ea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa4801f8d50 tx=0x7fa470008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:40.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.008+0000 7fa4757fa640 1 -- 172.21.15.29:0/4127813075 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa47c0c4060 con 0x7fa480156420 2024-04-25T00:36:40.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.172+0000 7fa486df4640 1 -- 172.21.15.29:0/4127813075 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4800cb300 con 0x7fa480156420 2024-04-25T00:36:40.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.172+0000 7fa4757fa640 1 -- 172.21.15.29:0/4127813075 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16625 (secure 0 0 0) 0x7fa47c016150 con 0x7fa480156420 2024-04-25T00:36:40.177 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:40.177 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":83,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:36:39.191941+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:36:37.482565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:36:38.176961+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:36:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.176+0000 7fa486df4640 1 -- 172.21.15.29:0/4127813075 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa46c0679e0 msgr2=0x7fa46c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.176+0000 7fa486df4640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa46c0679e0 0x7fa46c069ea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa4801f8d50 tx=0x7fa470008040 comp rx=0 tx=0).stop 2024-04-25T00:36:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.176+0000 7fa486df4640 1 -- 172.21.15.29:0/4127813075 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa480156420 msgr2=0x7fa480149250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.176+0000 7fa486df4640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa480156420 0x7fa480149250 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0ad040 tx=0x7fa47c01ada0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.176+0000 7fa486df4640 1 -- 172.21.15.29:0/4127813075 shutdown_connections 2024-04-25T00:36:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.176+0000 7fa486df4640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa46c0679e0 0x7fa46c069ea0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.176+0000 7fa486df4640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa48015ff10 0x7fa4800c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.176+0000 7fa486df4640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa48015fa80 0x7fa48015e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.176+0000 7fa486df4640 1 --2- 172.21.15.29:0/4127813075 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa480156420 0x7fa480149250 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.176+0000 7fa486df4640 1 -- 172.21.15.29:0/4127813075 >> 172.21.15.29:0/4127813075 conn(0x7fa4800d0c40 msgr2=0x7fa4800b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.176+0000 7fa486df4640 1 -- 172.21.15.29:0/4127813075 shutdown_connections 2024-04-25T00:36:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.176+0000 7fa486df4640 1 -- 172.21.15.29:0/4127813075 wait complete. 2024-04-25T00:36:40.196 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:36:40.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f410d5a8640 1 Processor -- start 2024-04-25T00:36:40.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f410d5a8640 1 -- start start 2024-04-25T00:36:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f410d5a8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410815df70 0x7f41080b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f410d5a8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f410806d210 0x7f41080b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f410d5a8640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f410806d980 con 0x7f41080b4b60 2024-04-25T00:36:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f410d5a8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f41080b4fa0 con 0x7f410806d210 2024-04-25T00:36:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f410d5a8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f410806dc60 con 0x7f410815df70 2024-04-25T00:36:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f41077fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f41080b4b60 0x7f4108070ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53340/0 (socket says 172.21.15.29:53340) 2024-04-25T00:36:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f41077fe640 1 -- 172.21.15.29:0/1758335836 learned_addr learned my addr 172.21.15.29:0/1758335836 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f41067fc640 1 --2- 172.21.15.29:0/1758335836 >> v2:172.21.15.157:3300/0 conn(0x7f410806d210 0x7f41080b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4106ffd640 1 --2- 172.21.15.29:0/1758335836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410815df70 0x7f41080b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1253714740 0 0) 0x7f410806d980 con 0x7f41080b4b60 2024-04-25T00:36:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40f0003530 con 0x7f41080b4b60 2024-04-25T00:36:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1053723458 0 0) 0x7f40f0003530 con 0x7f41080b4b60 2024-04-25T00:36:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410815df70 msgr2=0x7f41080b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 --2- 172.21.15.29:0/1758335836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410815df70 0x7f41080b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 >> v2:172.21.15.157:3300/0 conn(0x7f410806d210 msgr2=0x7f41080b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 --2- 172.21.15.29:0/1758335836 >> v2:172.21.15.157:3300/0 conn(0x7f410806d210 0x7f41080b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410806df40 con 0x7f41080b4b60 2024-04-25T00:36:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4100001a00 con 0x7f41080b4b60 2024-04-25T00:36:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 >> v1:172.21.15.29:6789/0 conn(0x7f41080b4b60 legacy=0x7f4108070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 --2- 172.21.15.29:0/1758335836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410815df70 0x7f40f00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 --2- 172.21.15.29:0/1758335836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40f0003f40 0x7f40f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 --2- 172.21.15.29:0/1758335836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f0004d50 0x7f40f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f41080b4fa0 con 0x7f40f0004d50 2024-04-25T00:36:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f410806dc60 con 0x7f40f0003f40 2024-04-25T00:36:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f40f000b9d0 con 0x7f410815df70 2024-04-25T00:36:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f41077fe640 1 --2- 172.21.15.29:0/1758335836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f0004d50 0x7f40f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f4106ffd640 1 --2- 172.21.15.29:0/1758335836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410815df70 0x7f40f00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f41067fc640 1 --2- 172.21.15.29:0/1758335836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40f0003f40 0x7f40f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f41077fe640 1 -- 172.21.15.29:0/1758335836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410815df70 msgr2=0x7f40f00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f41077fe640 1 --2- 172.21.15.29:0/1758335836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410815df70 0x7f40f00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f41077fe640 1 -- 172.21.15.29:0/1758335836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40f0003f40 msgr2=0x7f40f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f41077fe640 1 --2- 172.21.15.29:0/1758335836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40f0003f40 0x7f40f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.320+0000 7f41077fe640 1 -- 172.21.15.29:0/1758335836 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f0003530 con 0x7f40f0004d50 2024-04-25T00:36:40.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.324+0000 7f41077fe640 1 --2- 172.21.15.29:0/1758335836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f0004d50 0x7f40f00051d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f410000a880 tx=0x7f410001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433f8c9cf55447f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.324+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410000a4a0 con 0x7f40f0004d50 2024-04-25T00:36:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.324+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41000ad630 con 0x7f40f0004d50 2024-04-25T00:36:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.324+0000 7f4105ffb640 1 -- 172.21.15.29:0/1758335836 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41000adaa0 con 0x7f40f0004d50 2024-04-25T00:36:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.324+0000 7f410d5a8640 1 -- 172.21.15.29:0/1758335836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f0004d50 msgr2=0x7f40f00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.324+0000 7f410d5a8640 1 --2- 172.21.15.29:0/1758335836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40f0004d50 0x7f40f00051d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f410000a880 tx=0x7f410001d340 comp rx=0 tx=0).stop 2024-04-25T00:36:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.324+0000 7f41077fe640 1 -- 172.21.15.29:0/1758335836 reap_dead start 2024-04-25T00:36:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.324+0000 7f410d5a8640 1 -- 172.21.15.29:0/1758335836 shutdown_connections 2024-04-25T00:36:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.324+0000 7f410d5a8640 1 -- 172.21.15.29:0/1758335836 >> 172.21.15.29:0/1758335836 conn(0x7f41080b1870 msgr2=0x7f41080b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.324+0000 7f410d5a8640 1 -- 172.21.15.29:0/1758335836 shutdown_connections 2024-04-25T00:36:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.324+0000 7f410d5a8640 1 -- 172.21.15.29:0/1758335836 wait complete. 2024-04-25T00:36:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.324+0000 7f410d5a8640 1 Processor -- start 2024-04-25T00:36:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.324+0000 7f410d5a8640 1 -- start start 2024-04-25T00:36:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f410d5a8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410814e6a0 0x7f41080c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f410d5a8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410814ea80 0x7f41080ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f410d5a8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41080ce6d0 0x7f41080c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f410d5a8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f410806d670 con 0x7f41080ce6d0 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f410d5a8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f410806e500 con 0x7f410814e6a0 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f410d5a8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f410815e350 con 0x7f410814ea80 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f41077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41080ce6d0 0x7f41080c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f41077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41080ce6d0 0x7f41080c4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36902/0 (socket says 172.21.15.29:36902) 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f41077fe640 1 -- 172.21.15.29:0/1271167771 learned_addr learned my addr 172.21.15.29:0/1271167771 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f4106ffd640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410814e6a0 0x7f41080c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f41067fc640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410814ea80 0x7f41080ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f41077fe640 1 -- 172.21.15.29:0/1271167771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410814ea80 msgr2=0x7f41080ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f41077fe640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410814ea80 0x7f41080ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f41077fe640 1 -- 172.21.15.29:0/1271167771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410814e6a0 msgr2=0x7f41080c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f41077fe640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410814e6a0 0x7f41080c6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f41077fe640 1 -- 172.21.15.29:0/1271167771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41080c5a40 con 0x7f41080ce6d0 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f41067fc640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410814ea80 0x7f41080ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f41077fe640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41080ce6d0 0x7f41080c4d10 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f410001ba50 tx=0x7f40f0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f40ef7fe640 1 -- 172.21.15.29:0/1271167771 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41000bb070 con 0x7f41080ce6d0 2024-04-25T00:36:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f40ef7fe640 1 -- 172.21.15.29:0/1271167771 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41000011d0 con 0x7f41080ce6d0 2024-04-25T00:36:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f40ef7fe640 1 -- 172.21.15.29:0/1271167771 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4100002000 con 0x7f41080ce6d0 2024-04-25T00:36:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f410d5a8640 1 -- 172.21.15.29:0/1271167771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41080c5d20 con 0x7f41080ce6d0 2024-04-25T00:36:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f410d5a8640 1 -- 172.21.15.29:0/1271167771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41080c62e0 con 0x7f41080ce6d0 2024-04-25T00:36:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f410d5a8640 1 -- 172.21.15.29:0/1271167771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4108069270 con 0x7f41080ce6d0 2024-04-25T00:36:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f40ef7fe640 1 -- 172.21.15.29:0/1271167771 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f41000ad630 con 0x7f41080ce6d0 2024-04-25T00:36:40.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f40ef7fe640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f40f4067a10 0x7f40f4069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f40ef7fe640 1 -- 172.21.15.29:0/1271167771 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f410011fc90 con 0x7f41080ce6d0 2024-04-25T00:36:40.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.328+0000 7f4106ffd640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f40f4067a10 0x7f40f4069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.332+0000 7f4106ffd640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f40f4067a10 0x7f40f4069ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009a30 tx=0x7f40fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:40.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.336+0000 7f40ef7fe640 1 -- 172.21.15.29:0/1271167771 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f41000170b0 con 0x7f41080ce6d0 2024-04-25T00:36:40.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.500+0000 7f410d5a8640 1 -- 172.21.15.29:0/1271167771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f410815b920 con 0x7f41080ce6d0 2024-04-25T00:36:40.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.500+0000 7f40ef7fe640 1 -- 172.21.15.29:0/1271167771 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16625 (secure 0 0 0) 0x7f41000f0510 con 0x7f41080ce6d0 2024-04-25T00:36:40.503 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:40.504 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":83,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:36:39.191941+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:36:37.482565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:36:38.176961+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:36:40.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.504+0000 7f410d5a8640 1 -- 172.21.15.29:0/1271167771 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f40f4067a10 msgr2=0x7f40f4069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.504+0000 7f410d5a8640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f40f4067a10 0x7f40f4069ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009a30 tx=0x7f40fc008040 comp rx=0 tx=0).stop 2024-04-25T00:36:40.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.504+0000 7f410d5a8640 1 -- 172.21.15.29:0/1271167771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41080ce6d0 msgr2=0x7f41080c4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.504+0000 7f410d5a8640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41080ce6d0 0x7f41080c4d10 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f410001ba50 tx=0x7f40f0005840 comp rx=0 tx=0).stop 2024-04-25T00:36:40.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.504+0000 7f410d5a8640 1 -- 172.21.15.29:0/1271167771 shutdown_connections 2024-04-25T00:36:40.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.504+0000 7f410d5a8640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f40f4067a10 0x7f40f4069ed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.504+0000 7f410d5a8640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41080ce6d0 0x7f41080c4d10 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.504+0000 7f410d5a8640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f410814ea80 0x7f41080ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.504+0000 7f410d5a8640 1 --2- 172.21.15.29:0/1271167771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410814e6a0 0x7f41080c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.504+0000 7f410d5a8640 1 -- 172.21.15.29:0/1271167771 >> 172.21.15.29:0/1271167771 conn(0x7f41080b1870 msgr2=0x7f410815c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:40.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.504+0000 7f410d5a8640 1 -- 172.21.15.29:0/1271167771 shutdown_connections 2024-04-25T00:36:40.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.504+0000 7f410d5a8640 1 -- 172.21.15.29:0/1271167771 wait complete. 2024-04-25T00:36:40.523 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:36:40.641 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:36:40.643 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:36:40.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f8934bec640 1 Processor -- start 2024-04-25T00:36:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f8934bec640 1 -- start start 2024-04-25T00:36:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f8934bec640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f89300b2200 0x7f893006c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f8934bec640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f893006c860 0x7f89300c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f8934bec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89300c5220 0x7f89300d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f8934bec640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f89300b25e0 con 0x7f89300b2200 2024-04-25T00:36:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f8934bec640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f89301663e0 con 0x7f893006c860 2024-04-25T00:36:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f8934bec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f89301666c0 con 0x7f89300c5220 2024-04-25T00:36:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f892e59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f89300b2200 0x7f893006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f892e59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f89300b2200 0x7f893006c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36914/0 (socket says 172.21.15.29:36914) 2024-04-25T00:36:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f892e59c640 1 -- 172.21.15.29:0/390116383 learned_addr learned my addr 172.21.15.29:0/390116383 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f892dd9b640 1 --2- 172.21.15.29:0/390116383 >> v2:172.21.15.157:3300/0 conn(0x7f893006c860 0x7f89300c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f892ed9d640 1 --2- 172.21.15.29:0/390116383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89300c5220 0x7f89300d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f892e59c640 1 -- 172.21.15.29:0/390116383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89300c5220 msgr2=0x7f89300d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f892e59c640 1 --2- 172.21.15.29:0/390116383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89300c5220 0x7f89300d1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f892e59c640 1 -- 172.21.15.29:0/390116383 >> v2:172.21.15.157:3300/0 conn(0x7f893006c860 msgr2=0x7f89300c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f892e59c640 1 --2- 172.21.15.29:0/390116383 >> v2:172.21.15.157:3300/0 conn(0x7f893006c860 0x7f89300c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.652+0000 7f892e59c640 1 -- 172.21.15.29:0/390116383 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89301669a0 con 0x7f89300b2200 2024-04-25T00:36:40.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f892e59c640 1 --2- 172.21.15.29:0/390116383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89300b2200 0x7f893006c320 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f893006c1f0 tx=0x7f892001c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55901c597a8c0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:40.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f892d59a640 1 -- 172.21.15.29:0/390116383 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89200b0070 con 0x7f89300b2200 2024-04-25T00:36:40.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f892d59a640 1 -- 172.21.15.29:0/390116383 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89200ab600 con 0x7f89300b2200 2024-04-25T00:36:40.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f892d59a640 1 -- 172.21.15.29:0/390116383 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89200ab9d0 con 0x7f89300b2200 2024-04-25T00:36:40.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 -- 172.21.15.29:0/390116383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89300b2200 msgr2=0x7f893006c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 --2- 172.21.15.29:0/390116383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89300b2200 0x7f893006c320 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f893006c1f0 tx=0x7f892001c1d0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 -- 172.21.15.29:0/390116383 shutdown_connections 2024-04-25T00:36:40.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 --2- 172.21.15.29:0/390116383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89300c5220 0x7f89300d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 --2- 172.21.15.29:0/390116383 >> v2:172.21.15.157:3300/0 conn(0x7f893006c860 0x7f89300c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 --2- 172.21.15.29:0/390116383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89300b2200 0x7f893006c320 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 -- 172.21.15.29:0/390116383 >> 172.21.15.29:0/390116383 conn(0x7f893009aa30 msgr2=0x7f893009ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:40.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 -- 172.21.15.29:0/390116383 shutdown_connections 2024-04-25T00:36:40.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 -- 172.21.15.29:0/390116383 wait complete. 2024-04-25T00:36:40.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 Processor -- start 2024-04-25T00:36:40.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 -- start start 2024-04-25T00:36:40.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f893006c860 0x7f89300c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89300b2200 0x7f893015af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89300c5220 0x7f89300afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f89300cc7c0 con 0x7f89300c5220 2024-04-25T00:36:40.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89300cc640 con 0x7f893006c860 2024-04-25T00:36:40.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.656+0000 7f8934bec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f89300cc940 con 0x7f89300b2200 2024-04-25T00:36:40.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89300c5220 0x7f89300afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89300c5220 0x7f89300afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36930/0 (socket says 172.21.15.29:36930) 2024-04-25T00:36:40.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892ed9d640 1 -- 172.21.15.29:0/3099966331 learned_addr learned my addr 172.21.15.29:0/3099966331 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:40.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892e59c640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f893006c860 0x7f89300c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892dd9b640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89300b2200 0x7f893015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892ed9d640 1 -- 172.21.15.29:0/3099966331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89300b2200 msgr2=0x7f893015af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892ed9d640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89300b2200 0x7f893015af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892ed9d640 1 -- 172.21.15.29:0/3099966331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f893006c860 msgr2=0x7f89300c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:40.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892ed9d640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f893006c860 0x7f89300c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:40.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892ed9d640 1 -- 172.21.15.29:0/3099966331 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89300c7250 con 0x7f89300c5220 2024-04-25T00:36:40.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892e59c640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f893006c860 0x7f89300c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:40.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892dd9b640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89300b2200 0x7f893015af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:40.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892ed9d640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89300c5220 0x7f89300afec0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f892800afd0 tx=0x7f892800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:40.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f891effd640 1 -- 172.21.15.29:0/3099966331 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928015070 con 0x7f89300c5220 2024-04-25T00:36:40.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f891effd640 1 -- 172.21.15.29:0/3099966331 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8928004220 con 0x7f89300c5220 2024-04-25T00:36:40.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f8934bec640 1 -- 172.21.15.29:0/3099966331 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89300c7530 con 0x7f89300c5220 2024-04-25T00:36:40.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f891effd640 1 -- 172.21.15.29:0/3099966331 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928003620 con 0x7f89300c5220 2024-04-25T00:36:40.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f8934bec640 1 -- 172.21.15.29:0/3099966331 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89300c7af0 con 0x7f89300c5220 2024-04-25T00:36:40.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f8934bec640 1 -- 172.21.15.29:0/3099966331 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8930069270 con 0x7f89300c5220 2024-04-25T00:36:40.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f891effd640 1 -- 172.21.15.29:0/3099966331 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f89280037c0 con 0x7f89300c5220 2024-04-25T00:36:40.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f891effd640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f89100679e0 0x7f8910069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:40.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f891effd640 1 -- 172.21.15.29:0/3099966331 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f8928085be0 con 0x7f89300c5220 2024-04-25T00:36:40.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892e59c640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f89100679e0 0x7f8910069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:40.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.660+0000 7f892e59c640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f89100679e0 0x7f8910069ea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89200ad000 tx=0x7f89200a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:40.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.664+0000 7f891effd640 1 -- 172.21.15.29:0/3099966331 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8928056590 con 0x7f89300c5220 2024-04-25T00:36:40.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:40.864+0000 7f8934bec640 1 -- 172.21.15.29:0/3099966331 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f89300ced80 con 0x7f89300c5220 2024-04-25T00:36:41.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:41.840+0000 7f891effd640 1 -- 172.21.15.29:0/3099966331 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) ==== 118+0+0 (secure 0 0 0) 0x7f8928055e90 con 0x7f89300c5220 2024-04-25T00:36:41.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:41.844+0000 7f8934bec640 1 -- 172.21.15.29:0/3099966331 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f89100679e0 msgr2=0x7f8910069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:41.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:41.844+0000 7f8934bec640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f89100679e0 0x7f8910069ea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89200ad000 tx=0x7f89200a6000 comp rx=0 tx=0).stop 2024-04-25T00:36:41.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:41.844+0000 7f8934bec640 1 -- 172.21.15.29:0/3099966331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89300c5220 msgr2=0x7f89300afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:41.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:41.844+0000 7f8934bec640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89300c5220 0x7f89300afec0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f892800afd0 tx=0x7f892800ca20 comp rx=0 tx=0).stop 2024-04-25T00:36:41.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:41.844+0000 7f8934bec640 1 -- 172.21.15.29:0/3099966331 shutdown_connections 2024-04-25T00:36:41.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:41.844+0000 7f8934bec640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f89100679e0 0x7f8910069ea0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:41.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:41.844+0000 7f8934bec640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89300c5220 0x7f89300afec0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:41.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:41.844+0000 7f8934bec640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89300b2200 0x7f893015af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:41.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:41.844+0000 7f8934bec640 1 --2- 172.21.15.29:0/3099966331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f893006c860 0x7f89300c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:41.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:41.844+0000 7f8934bec640 1 -- 172.21.15.29:0/3099966331 >> 172.21.15.29:0/3099966331 conn(0x7f893009aa30 msgr2=0x7f89300cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:41.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:41.844+0000 7f8934bec640 1 -- 172.21.15.29:0/3099966331 shutdown_connections 2024-04-25T00:36:41.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:41.844+0000 7f8934bec640 1 -- 172.21.15.29:0/3099966331 wait complete. 2024-04-25T00:36:41.868 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:36:41.868 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:36:42.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0ed0d3f640 1 Processor -- start 2024-04-25T00:36:42.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0ed0d3f640 1 -- start start 2024-04-25T00:36:42.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0ed0d3f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc15e8d0 0x7f0ecc0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:42.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0ed0d3f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0ecc15f3c0 0x7f0ecc15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:42.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0ed0d3f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ecc0b3720 con 0x7f0ecc0b4780 2024-04-25T00:36:42.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0ed0d3f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0ecc0b42f0 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0ed0d3f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0ecc0b39c0 con 0x7f0ecc15e8d0 2024-04-25T00:36:42.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0ecad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0ecc15f3c0 0x7f0ecc15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:42.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0eca59c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc15e8d0 0x7f0ecc0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:42.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0ec9d9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0ecc0b4780 0x7f0ecc15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:35270/0 (socket says 172.21.15.29:35270) 2024-04-25T00:36:42.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0ec9d9b640 1 -- 172.21.15.29:0/1115722893 learned_addr learned my addr 172.21.15.29:0/1115722893 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:42.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0eca59c640 1 -- 172.21.15.29:0/1115722893 >> v2:172.21.15.157:3300/0 conn(0x7f0ecc15f3c0 msgr2=0x7f0ecc15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:42.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0eca59c640 1 --2- 172.21.15.29:0/1115722893 >> v2:172.21.15.157:3300/0 conn(0x7f0ecc15f3c0 0x7f0ecc15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:42.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0eca59c640 1 -- 172.21.15.29:0/1115722893 >> v1:172.21.15.157:6789/0 conn(0x7f0ecc0b4780 legacy=0x7f0ecc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:42.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.000+0000 7f0eca59c640 1 -- 172.21.15.29:0/1115722893 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc0b3ca0 con 0x7f0ecc15e8d0 2024-04-25T00:36:42.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.004+0000 7f0eca59c640 1 --2- 172.21.15.29:0/1115722893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc15e8d0 0x7f0ecc0cc240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009d90 tx=0x7f0ebc01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38963c1dbc317804 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:42.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.004+0000 7f0ec959a640 1 -- 172.21.15.29:0/1115722893 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0b2070 con 0x7f0ecc15e8d0 2024-04-25T00:36:42.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.004+0000 7f0ec959a640 1 -- 172.21.15.29:0/1115722893 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ebc0a5910 con 0x7f0ecc15e8d0 2024-04-25T00:36:42.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.004+0000 7f0ec959a640 1 -- 172.21.15.29:0/1115722893 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0a5ce0 con 0x7f0ecc15e8d0 2024-04-25T00:36:42.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.004+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1115722893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc15e8d0 msgr2=0x7f0ecc0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:42.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.004+0000 7f0ed0d3f640 1 --2- 172.21.15.29:0/1115722893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc15e8d0 0x7f0ecc0cc240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009d90 tx=0x7f0ebc01c940 comp rx=0 tx=0).stop 2024-04-25T00:36:42.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.004+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1115722893 shutdown_connections 2024-04-25T00:36:42.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.004+0000 7f0ed0d3f640 1 --2- 172.21.15.29:0/1115722893 >> v2:172.21.15.157:3300/0 conn(0x7f0ecc15f3c0 0x7f0ecc15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:42.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.004+0000 7f0ed0d3f640 1 --2- 172.21.15.29:0/1115722893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc15e8d0 0x7f0ecc0cc240 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:42.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.004+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1115722893 >> 172.21.15.29:0/1115722893 conn(0x7f0ecc0b1870 msgr2=0x7f0ecc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:42.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.004+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1115722893 shutdown_connections 2024-04-25T00:36:42.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.004+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1115722893 wait complete. 2024-04-25T00:36:42.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ed0d3f640 1 Processor -- start 2024-04-25T00:36:42.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ed0d3f640 1 -- start start 2024-04-25T00:36:42.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ed0d3f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc0b4780 0x7f0ecc0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:42.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ed0d3f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ecc15e8d0 0x7f0ecc13c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:42.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ed0d3f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ecc15f3c0 0x7f0ecc158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:42.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ed0d3f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc15d6a0 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ed0d3f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc15d520 con 0x7f0ecc15e8d0 2024-04-25T00:36:42.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ed0d3f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0ecc167c70 con 0x7f0ecc0b4780 2024-04-25T00:36:42.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ecad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ecc15f3c0 0x7f0ecc158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:42.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ecad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ecc15f3c0 0x7f0ecc158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36944/0 (socket says 172.21.15.29:36944) 2024-04-25T00:36:42.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ecad9d640 1 -- 172.21.15.29:0/1099528057 learned_addr learned my addr 172.21.15.29:0/1099528057 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:42.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ec9d9b640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ecc15e8d0 0x7f0ecc13c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:42.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0eca59c640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc0b4780 0x7f0ecc0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:42.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ecad9d640 1 -- 172.21.15.29:0/1099528057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc0b4780 msgr2=0x7f0ecc0b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:42.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ecad9d640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc0b4780 0x7f0ecc0b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:42.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ecad9d640 1 -- 172.21.15.29:0/1099528057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ecc15e8d0 msgr2=0x7f0ecc13c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:42.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ecad9d640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ecc15e8d0 0x7f0ecc13c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:42.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ecad9d640 1 -- 172.21.15.29:0/1099528057 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc159aa0 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ecad9d640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ecc15f3c0 0x7f0ecc158dd0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400e7f0 tx=0x7f0ec400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:42.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ebaffd640 1 -- 172.21.15.29:0/1099528057 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4017070 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ebaffd640 1 -- 172.21.15.29:0/1099528057 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec400c940 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1099528057 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ecc159de0 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ebaffd640 1 -- 172.21.15.29:0/1099528057 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400eea0 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1099528057 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ecc15a620 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1099528057 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ecc15a820 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.008+0000 7f0ebaffd640 1 -- 172.21.15.29:0/1099528057 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f0ec401a070 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.012+0000 7f0ebaffd640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0eac0679e0 0x7f0eac069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:42.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.012+0000 7f0eca59c640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0eac0679e0 0x7f0eac069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:42.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.012+0000 7f0ebaffd640 1 -- 172.21.15.29:0/1099528057 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f0ec4085db0 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.012+0000 7f0eca59c640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0eac0679e0 0x7f0eac069ea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0af000 tx=0x7f0ebc0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:42.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.016+0000 7f0ebaffd640 1 -- 172.21.15.29:0/1099528057 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0ecc15a820 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.212+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1099528057 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0ecc06e7b0 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.848+0000 7f0ebaffd640 1 -- 172.21.15.29:0/1099528057 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) ==== 122+0+0 (secure 0 0 0) 0x7f0ecc159de0 con 0x7f0ecc15f3c0 2024-04-25T00:36:42.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.852+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1099528057 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0eac0679e0 msgr2=0x7f0eac069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:42.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.852+0000 7f0ed0d3f640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0eac0679e0 0x7f0eac069ea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0af000 tx=0x7f0ebc0a7070 comp rx=0 tx=0).stop 2024-04-25T00:36:42.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.852+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1099528057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ecc15f3c0 msgr2=0x7f0ecc158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:42.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.852+0000 7f0ed0d3f640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ecc15f3c0 0x7f0ecc158dd0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400e7f0 tx=0x7f0ec400ecb0 comp rx=0 tx=0).stop 2024-04-25T00:36:42.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.852+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1099528057 shutdown_connections 2024-04-25T00:36:42.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.852+0000 7f0ed0d3f640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0eac0679e0 0x7f0eac069ea0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:42.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.852+0000 7f0ed0d3f640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ecc15f3c0 0x7f0ecc158dd0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:42.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.852+0000 7f0ed0d3f640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ecc15e8d0 0x7f0ecc13c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:42.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.852+0000 7f0ed0d3f640 1 --2- 172.21.15.29:0/1099528057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ecc0b4780 0x7f0ecc0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:42.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.852+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1099528057 >> 172.21.15.29:0/1099528057 conn(0x7f0ecc0b1870 msgr2=0x7f0ecc13a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:42.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.852+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1099528057 shutdown_connections 2024-04-25T00:36:42.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:42.852+0000 7f0ed0d3f640 1 -- 172.21.15.29:0/1099528057 wait complete. 2024-04-25T00:36:42.875 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:36:42.875 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:36:43.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.000+0000 7f2c475a0640 1 Processor -- start 2024-04-25T00:36:43.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.000+0000 7f2c475a0640 1 -- start start 2024-04-25T00:36:43.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c475a0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c400b3140 0x7f2c4006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c475a0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2c400b4b10 0x7f2c400b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c475a0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c400b41e0 con 0x7f2c400b3a60 2024-04-25T00:36:43.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c475a0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2c4006d960 con 0x7f2c400b4b10 2024-04-25T00:36:43.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c475a0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4006dc40 con 0x7f2c400b3140 2024-04-25T00:36:43.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c4533c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c400b3140 0x7f2c4006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c4533c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c400b3140 0x7f2c4006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:37270/0 (socket says 172.21.15.29:37270) 2024-04-25T00:36:43.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c44b3b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2c400b4b10 0x7f2c400b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c4533c640 1 -- 172.21.15.29:0/1506456603 learned_addr learned my addr 172.21.15.29:0/1506456603 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:43.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c4533c640 1 -- 172.21.15.29:0/1506456603 >> v2:172.21.15.157:3300/0 conn(0x7f2c400b4b10 msgr2=0x7f2c400b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:43.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c4533c640 1 --2- 172.21.15.29:0/1506456603 >> v2:172.21.15.157:3300/0 conn(0x7f2c400b4b10 0x7f2c400b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c4533c640 1 -- 172.21.15.29:0/1506456603 >> v1:172.21.15.157:6789/0 conn(0x7f2c400b3a60 legacy=0x7f2c4015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c4533c640 1 -- 172.21.15.29:0/1506456603 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4006df20 con 0x7f2c400b3140 2024-04-25T00:36:43.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c4533c640 1 --2- 172.21.15.29:0/1506456603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c400b3140 0x7f2c4006e440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009870 tx=0x7f2c3c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b97559f5a8f7ae89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:43.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c37fff640 1 -- 172.21.15.29:0/1506456603 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c0b0070 con 0x7f2c400b3140 2024-04-25T00:36:43.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c37fff640 1 -- 172.21.15.29:0/1506456603 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c3c0ab600 con 0x7f2c400b3140 2024-04-25T00:36:43.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.004+0000 7f2c37fff640 1 -- 172.21.15.29:0/1506456603 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c0ab9d0 con 0x7f2c400b3140 2024-04-25T00:36:43.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 -- 172.21.15.29:0/1506456603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c400b3140 msgr2=0x7f2c4006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 --2- 172.21.15.29:0/1506456603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c400b3140 0x7f2c4006e440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009870 tx=0x7f2c3c01c560 comp rx=0 tx=0).stop 2024-04-25T00:36:43.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 -- 172.21.15.29:0/1506456603 shutdown_connections 2024-04-25T00:36:43.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 --2- 172.21.15.29:0/1506456603 >> v2:172.21.15.157:3300/0 conn(0x7f2c400b4b10 0x7f2c400b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 --2- 172.21.15.29:0/1506456603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c400b3140 0x7f2c4006e440 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 -- 172.21.15.29:0/1506456603 >> 172.21.15.29:0/1506456603 conn(0x7f2c400b1870 msgr2=0x7f2c400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:43.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 -- 172.21.15.29:0/1506456603 shutdown_connections 2024-04-25T00:36:43.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 -- 172.21.15.29:0/1506456603 wait complete. 2024-04-25T00:36:43.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 Processor -- start 2024-04-25T00:36:43.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 -- start start 2024-04-25T00:36:43.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c400b3140 0x7f2c40158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c400b3a60 0x7f2c401571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c400b4b10 0x7f2c401569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40168b70 con 0x7f2c400b3140 2024-04-25T00:36:43.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401689f0 con 0x7f2c400b3a60 2024-04-25T00:36:43.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2c40168cf0 con 0x7f2c400b4b10 2024-04-25T00:36:43.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c4533c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c400b3140 0x7f2c40158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c4533c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c400b3140 0x7f2c40158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36954/0 (socket says 172.21.15.29:36954) 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c4533c640 1 -- 172.21.15.29:0/653212450 learned_addr learned my addr 172.21.15.29:0/653212450 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c45b3d640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c400b4b10 0x7f2c401569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c44b3b640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c400b3a60 0x7f2c401571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c4533c640 1 -- 172.21.15.29:0/653212450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c400b4b10 msgr2=0x7f2c401569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c4533c640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c400b4b10 0x7f2c401569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c4533c640 1 -- 172.21.15.29:0/653212450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c400b3a60 msgr2=0x7f2c401571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c4533c640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c400b3a60 0x7f2c401571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c4533c640 1 -- 172.21.15.29:0/653212450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c40157a40 con 0x7f2c400b3140 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c45b3d640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c400b4b10 0x7f2c401569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c4533c640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c400b3140 0x7f2c40158ae0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0ae040 tx=0x7f2c3c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c35ffb640 1 -- 172.21.15.29:0/653212450 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c0ba020 con 0x7f2c400b3140 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c35ffb640 1 -- 172.21.15.29:0/653212450 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c3c0b6420 con 0x7f2c400b3140 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 -- 172.21.15.29:0/653212450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c40157d20 con 0x7f2c400b3140 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c35ffb640 1 -- 172.21.15.29:0/653212450 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c0b0040 con 0x7f2c400b3140 2024-04-25T00:36:43.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.008+0000 7f2c475a0640 1 -- 172.21.15.29:0/653212450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c401582e0 con 0x7f2c400b3140 2024-04-25T00:36:43.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.012+0000 7f2c475a0640 1 -- 172.21.15.29:0/653212450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c40069270 con 0x7f2c400b3140 2024-04-25T00:36:43.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.012+0000 7f2c35ffb640 1 -- 172.21.15.29:0/653212450 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f2c3c0bd020 con 0x7f2c400b3140 2024-04-25T00:36:43.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.012+0000 7f2c35ffb640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c240679e0 0x7f2c24069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.012+0000 7f2c35ffb640 1 -- 172.21.15.29:0/653212450 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f2c3c1202e0 con 0x7f2c400b3140 2024-04-25T00:36:43.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.012+0000 7f2c44b3b640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c240679e0 0x7f2c24069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.012+0000 7f2c44b3b640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c240679e0 0x7f2c24069ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c4015a0d0 tx=0x7f2c30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:43.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.016+0000 7f2c35ffb640 1 -- 172.21.15.29:0/653212450 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2c3c0c4030 con 0x7f2c400b3140 2024-04-25T00:36:43.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.208+0000 7f2c475a0640 1 -- 172.21.15.29:0/653212450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c4015b510 con 0x7f2c400b3140 2024-04-25T00:36:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.212+0000 7f2c35ffb640 1 -- 172.21.15.29:0/653212450 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5186 (secure 0 0 0) 0x7f2c3c016150 con 0x7f2c400b3140 2024-04-25T00:36:43.213 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:43.213 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2434273880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2434273880},{"type":"v1","addr":"172.21.15.157:6833","nonce":2434273880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/558648019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":558648019},{"type":"v1","addr":"172.21.15.157:6835","nonce":558648019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":7401,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1764147340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1764147340},{"type":"v1","addr":"172.21.15.29:6833","nonce":1764147340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:36:39.192693+0000","modified":"2024-04-25T00:36:42.218595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7404":{"gid":7404,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/3939243143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3939243143},{"type":"v1","addr":"172.21.15.29:6835","nonce":3939243143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7404,"qdb_cluster":[7404]},"id":6}]} 2024-04-25T00:36:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 53 2024-04-25T00:36:43.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.212+0000 7f2c475a0640 1 -- 172.21.15.29:0/653212450 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c240679e0 msgr2=0x7f2c24069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.212+0000 7f2c475a0640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c240679e0 0x7f2c24069ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c4015a0d0 tx=0x7f2c30008040 comp rx=0 tx=0).stop 2024-04-25T00:36:43.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.212+0000 7f2c475a0640 1 -- 172.21.15.29:0/653212450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c400b3140 msgr2=0x7f2c40158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.212+0000 7f2c475a0640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c400b3140 0x7f2c40158ae0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0ae040 tx=0x7f2c3c01ada0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.212+0000 7f2c475a0640 1 -- 172.21.15.29:0/653212450 shutdown_connections 2024-04-25T00:36:43.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.212+0000 7f2c475a0640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c240679e0 0x7f2c24069ea0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.212+0000 7f2c475a0640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c400b4b10 0x7f2c401569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.212+0000 7f2c475a0640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c400b3a60 0x7f2c401571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.212+0000 7f2c475a0640 1 --2- 172.21.15.29:0/653212450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c400b3140 0x7f2c40158ae0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.212+0000 7f2c475a0640 1 -- 172.21.15.29:0/653212450 >> 172.21.15.29:0/653212450 conn(0x7f2c400b1870 msgr2=0x7f2c4015c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:43.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.212+0000 7f2c475a0640 1 -- 172.21.15.29:0/653212450 shutdown_connections 2024-04-25T00:36:43.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.212+0000 7f2c475a0640 1 -- 172.21.15.29:0/653212450 wait complete. 2024-04-25T00:36:43.231 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:36:43.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.348+0000 7f1e4f2eb640 1 Processor -- start 2024-04-25T00:36:43.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.348+0000 7f1e4f2eb640 1 -- start start 2024-04-25T00:36:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.348+0000 7f1e4f2eb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e480b3c30 0x7f1e4806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.348+0000 7f1e4f2eb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1e480b4b10 0x7f1e480ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.348+0000 7f1e4f2eb640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e480b41e0 con 0x7f1e480b3140 2024-04-25T00:36:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.348+0000 7f1e4f2eb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1e4806d960 con 0x7f1e480b4b10 2024-04-25T00:36:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.348+0000 7f1e4f2eb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4806dc40 con 0x7f1e480b3c30 2024-04-25T00:36:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e4d888640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1e480b4b10 0x7f1e480ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e4d087640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1e480b3140 0x7f1e480b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53352/0 (socket says 172.21.15.29:53352) 2024-04-25T00:36:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e4d087640 1 -- 172.21.15.29:0/532604795 learned_addr learned my addr 172.21.15.29:0/532604795 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e4c886640 1 --2- 172.21.15.29:0/532604795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e480b3c30 0x7f1e4806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e3ffff640 1 -- 172.21.15.29:0/532604795 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3235768277 0 0) 0x7f1e480b41e0 con 0x7f1e480b3140 2024-04-25T00:36:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e3ffff640 1 -- 172.21.15.29:0/532604795 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e34003530 con 0x7f1e480b3140 2024-04-25T00:36:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e4d888640 1 -- 172.21.15.29:0/532604795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e480b3c30 msgr2=0x7f1e4806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e4d888640 1 --2- 172.21.15.29:0/532604795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e480b3c30 0x7f1e4806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e4d888640 1 -- 172.21.15.29:0/532604795 >> v1:172.21.15.29:6789/0 conn(0x7f1e480b3140 legacy=0x7f1e480b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e4d888640 1 -- 172.21.15.29:0/532604795 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4806df20 con 0x7f1e480b4b10 2024-04-25T00:36:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e4c886640 1 --2- 172.21.15.29:0/532604795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e480b3c30 0x7f1e4806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e4d888640 1 --2- 172.21.15.29:0/532604795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e480b4b10 0x7f1e480ce240 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009d90 tx=0x7f1e4001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daae5375eb1cf2e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e3ffff640 1 -- 172.21.15.29:0/532604795 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400b2070 con 0x7f1e480b4b10 2024-04-25T00:36:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e3ffff640 1 -- 172.21.15.29:0/532604795 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e400a5830 con 0x7f1e480b4b10 2024-04-25T00:36:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.352+0000 7f1e3ffff640 1 -- 172.21.15.29:0/532604795 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400a5c00 con 0x7f1e480b4b10 2024-04-25T00:36:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/532604795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e480b4b10 msgr2=0x7f1e480ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 --2- 172.21.15.29:0/532604795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e480b4b10 0x7f1e480ce240 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009d90 tx=0x7f1e4001c970 comp rx=0 tx=0).stop 2024-04-25T00:36:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/532604795 shutdown_connections 2024-04-25T00:36:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 --2- 172.21.15.29:0/532604795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e480b4b10 0x7f1e480ce240 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 --2- 172.21.15.29:0/532604795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e480b3c30 0x7f1e4806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/532604795 >> 172.21.15.29:0/532604795 conn(0x7f1e480b1870 msgr2=0x7f1e480b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/532604795 shutdown_connections 2024-04-25T00:36:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/532604795 wait complete. 2024-04-25T00:36:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 Processor -- start 2024-04-25T00:36:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 -- start start 2024-04-25T00:36:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e480b3140 0x7f1e4814b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e480b3c30 0x7f1e4813c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e480b4b10 0x7f1e48071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48168d80 con 0x7f1e480b3c30 2024-04-25T00:36:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48168c00 con 0x7f1e480b3140 2024-04-25T00:36:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1e48168f00 con 0x7f1e480b4b10 2024-04-25T00:36:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4c886640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e480b3c30 0x7f1e4813c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4c886640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e480b3c30 0x7f1e4813c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36962/0 (socket says 172.21.15.29:36962) 2024-04-25T00:36:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4c886640 1 -- 172.21.15.29:0/466828551 learned_addr learned my addr 172.21.15.29:0/466828551 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:43.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4c886640 1 -- 172.21.15.29:0/466828551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e480b4b10 msgr2=0x7f1e48071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4d888640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e480b4b10 0x7f1e48071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4d087640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e480b3140 0x7f1e4814b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4c886640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e480b4b10 0x7f1e48071ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4c886640 1 -- 172.21.15.29:0/466828551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e480b3140 msgr2=0x7f1e4814b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4c886640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e480b3140 0x7f1e4814b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4c886640 1 -- 172.21.15.29:0/466828551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4813cb50 con 0x7f1e480b3c30 2024-04-25T00:36:43.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4d087640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e480b3140 0x7f1e4814b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:43.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4c886640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e480b3c30 0x7f1e4813c610 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800e840 tx=0x7f1e3800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:43.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e3dffb640 1 -- 172.21.15.29:0/466828551 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38017070 con 0x7f1e480b3c30 2024-04-25T00:36:43.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e3dffb640 1 -- 172.21.15.29:0/466828551 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3800c8d0 con 0x7f1e480b3c30 2024-04-25T00:36:43.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/466828551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e48072530 con 0x7f1e480b3c30 2024-04-25T00:36:43.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e3dffb640 1 -- 172.21.15.29:0/466828551 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3800f040 con 0x7f1e480b3c30 2024-04-25T00:36:43.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/466828551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e48153990 con 0x7f1e480b3c30 2024-04-25T00:36:43.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.356+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/466828551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e48069270 con 0x7f1e480b3c30 2024-04-25T00:36:43.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.360+0000 7f1e3dffb640 1 -- 172.21.15.29:0/466828551 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f1e3801a070 con 0x7f1e480b3c30 2024-04-25T00:36:43.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.360+0000 7f1e3dffb640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1e2c067a10 0x7f1e2c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.360+0000 7f1e3dffb640 1 -- 172.21.15.29:0/466828551 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f1e38085aa0 con 0x7f1e480b3c30 2024-04-25T00:36:43.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.360+0000 7f1e4d087640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1e2c067a10 0x7f1e2c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.360+0000 7f1e4d087640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1e2c067a10 0x7f1e2c069ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005890 tx=0x7f1e44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.364+0000 7f1e3dffb640 1 -- 172.21.15.29:0/466828551 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1e380563b0 con 0x7f1e480b3c30 2024-04-25T00:36:43.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.528+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/466828551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e48154fd0 con 0x7f1e480b3c30 2024-04-25T00:36:43.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.532+0000 7f1e3dffb640 1 -- 172.21.15.29:0/466828551 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16612 (secure 0 0 0) 0x7f1e38055cb0 con 0x7f1e480b3c30 2024-04-25T00:36:43.534 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:43.534 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":84,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:36:40.822887+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:36:37.482565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:36:38.176961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:36:43.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.532+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/466828551 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1e2c067a10 msgr2=0x7f1e2c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.532+0000 7f1e4f2eb640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1e2c067a10 0x7f1e2c069ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005890 tx=0x7f1e44009040 comp rx=0 tx=0).stop 2024-04-25T00:36:43.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.532+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/466828551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e480b3c30 msgr2=0x7f1e4813c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.532+0000 7f1e4f2eb640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e480b3c30 0x7f1e4813c610 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800e840 tx=0x7f1e3800ed00 comp rx=0 tx=0).stop 2024-04-25T00:36:43.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.536+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/466828551 shutdown_connections 2024-04-25T00:36:43.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.536+0000 7f1e4f2eb640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1e2c067a10 0x7f1e2c069ed0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.536+0000 7f1e4f2eb640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e480b4b10 0x7f1e48071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.536+0000 7f1e4f2eb640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e480b3c30 0x7f1e4813c610 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.536+0000 7f1e4f2eb640 1 --2- 172.21.15.29:0/466828551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e480b3140 0x7f1e4814b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.536+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/466828551 >> 172.21.15.29:0/466828551 conn(0x7f1e480b1870 msgr2=0x7f1e481554f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:43.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.536+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/466828551 shutdown_connections 2024-04-25T00:36:43.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.536+0000 7f1e4f2eb640 1 -- 172.21.15.29:0/466828551 wait complete. 2024-04-25T00:36:43.554 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:36:43.554 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:36:43.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.668+0000 7f4965aa9640 1 Processor -- start 2024-04-25T00:36:43.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f4965aa9640 1 -- start start 2024-04-25T00:36:43.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f4965aa9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f496015e8d0 0x7f49600cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f4965aa9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f496015f3c0 0x7f496015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f4965aa9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49600b3720 con 0x7f49600b4780 2024-04-25T00:36:43.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f4965aa9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f49600b42f0 con 0x7f496015f3c0 2024-04-25T00:36:43.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f4965aa9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f49600b39c0 con 0x7f496015e8d0 2024-04-25T00:36:43.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f495ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f496015f3c0 0x7f496015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f495ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f496015f3c0 0x7f496015c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36974/0 (socket says 172.21.15.29:36974) 2024-04-25T00:36:43.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f495ffff640 1 -- 172.21.15.29:0/1659739955 learned_addr learned my addr 172.21.15.29:0/1659739955 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:43.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f495f7fe640 1 --2- 172.21.15.29:0/1659739955 >> v2:172.21.15.157:3300/0 conn(0x7f496015e8d0 0x7f49600cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f495ffff640 1 -- 172.21.15.29:0/1659739955 >> v2:172.21.15.157:3300/0 conn(0x7f496015e8d0 msgr2=0x7f49600cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:43.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f495ffff640 1 --2- 172.21.15.29:0/1659739955 >> v2:172.21.15.157:3300/0 conn(0x7f496015e8d0 0x7f49600cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f495ffff640 1 -- 172.21.15.29:0/1659739955 >> v1:172.21.15.157:6789/0 conn(0x7f49600b4780 legacy=0x7f496015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:43.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.672+0000 7f495ffff640 1 -- 172.21.15.29:0/1659739955 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49600b3ca0 con 0x7f496015f3c0 2024-04-25T00:36:43.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495ffff640 1 --2- 172.21.15.29:0/1659739955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f496015f3c0 0x7f496015c7d0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f49600b5260 tx=0x7f495801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80606d9773f641df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:43.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495e7fc640 1 -- 172.21.15.29:0/1659739955 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49580a5740 con 0x7f496015f3c0 2024-04-25T00:36:43.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495e7fc640 1 -- 172.21.15.29:0/1659739955 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49580a58e0 con 0x7f496015f3c0 2024-04-25T00:36:43.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495e7fc640 1 -- 172.21.15.29:0/1659739955 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49580a5cb0 con 0x7f496015f3c0 2024-04-25T00:36:43.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 -- 172.21.15.29:0/1659739955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f496015f3c0 msgr2=0x7f496015c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 --2- 172.21.15.29:0/1659739955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f496015f3c0 0x7f496015c7d0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f49600b5260 tx=0x7f495801c9b0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 -- 172.21.15.29:0/1659739955 shutdown_connections 2024-04-25T00:36:43.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 --2- 172.21.15.29:0/1659739955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f496015f3c0 0x7f496015c7d0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 --2- 172.21.15.29:0/1659739955 >> v2:172.21.15.157:3300/0 conn(0x7f496015e8d0 0x7f49600cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 -- 172.21.15.29:0/1659739955 >> 172.21.15.29:0/1659739955 conn(0x7f49600b1870 msgr2=0x7f49600b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:43.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 -- 172.21.15.29:0/1659739955 shutdown_connections 2024-04-25T00:36:43.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 -- 172.21.15.29:0/1659739955 wait complete. 2024-04-25T00:36:43.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 Processor -- start 2024-04-25T00:36:43.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 -- start start 2024-04-25T00:36:43.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49600b4780 0x7f4960143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f496015e8d0 0x7f49600aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f496015f3c0 0x7f496013f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f496015d6a0 con 0x7f496015f3c0 2024-04-25T00:36:43.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f496015d520 con 0x7f49600b4780 2024-04-25T00:36:43.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f4965aa9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4960167c70 con 0x7f496015e8d0 2024-04-25T00:36:43.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f496015f3c0 0x7f496013f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f496015f3c0 0x7f496013f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36990/0 (socket says 172.21.15.29:36990) 2024-04-25T00:36:43.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495ffff640 1 -- 172.21.15.29:0/2461201817 learned_addr learned my addr 172.21.15.29:0/2461201817 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:43.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495f7fe640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49600b4780 0x7f4960143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495ffff640 1 -- 172.21.15.29:0/2461201817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f496015e8d0 msgr2=0x7f49600aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495effd640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f496015e8d0 0x7f49600aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495ffff640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f496015e8d0 0x7f49600aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495ffff640 1 -- 172.21.15.29:0/2461201817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49600b4780 msgr2=0x7f4960143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495ffff640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49600b4780 0x7f4960143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495ffff640 1 -- 172.21.15.29:0/2461201817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4960071790 con 0x7f496015f3c0 2024-04-25T00:36:43.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495f7fe640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49600b4780 0x7f4960143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:43.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.676+0000 7f495effd640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f496015e8d0 0x7f49600aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:43.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.680+0000 7f495ffff640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f496015f3c0 0x7f496013f800 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f495801a5e0 tx=0x7f4958019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:43.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.680+0000 7f4957fff640 1 -- 172.21.15.29:0/2461201817 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958009510 con 0x7f496015f3c0 2024-04-25T00:36:43.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.680+0000 7f4957fff640 1 -- 172.21.15.29:0/2461201817 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49580be070 con 0x7f496015f3c0 2024-04-25T00:36:43.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.680+0000 7f4965aa9640 1 -- 172.21.15.29:0/2461201817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4960071a30 con 0x7f496015f3c0 2024-04-25T00:36:43.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.680+0000 7f4957fff640 1 -- 172.21.15.29:0/2461201817 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49580b88b0 con 0x7f496015f3c0 2024-04-25T00:36:43.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.680+0000 7f4965aa9640 1 -- 172.21.15.29:0/2461201817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49600af2e0 con 0x7f496015f3c0 2024-04-25T00:36:43.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.680+0000 7f4965aa9640 1 -- 172.21.15.29:0/2461201817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4960069270 con 0x7f496015f3c0 2024-04-25T00:36:43.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.680+0000 7f4957fff640 1 -- 172.21.15.29:0/2461201817 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f49580b8a50 con 0x7f496015f3c0 2024-04-25T00:36:43.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.680+0000 7f4957fff640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49400679e0 0x7f4940069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:43.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.680+0000 7f4957fff640 1 -- 172.21.15.29:0/2461201817 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f4958120f90 con 0x7f496015f3c0 2024-04-25T00:36:43.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.680+0000 7f495f7fe640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49400679e0 0x7f4940069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:43.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.680+0000 7f495f7fe640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49400679e0 0x7f4940069ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f49600ceca0 tx=0x7f4950008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:43.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.684+0000 7f4957fff640 1 -- 172.21.15.29:0/2461201817 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f49580165a0 con 0x7f496015f3c0 2024-04-25T00:36:43.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.840+0000 7f4965aa9640 1 -- 172.21.15.29:0/2461201817 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f496013ae90 con 0x7f49400679e0 2024-04-25T00:36:43.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.844+0000 7f4957fff640 1 -- 172.21.15.29:0/2461201817 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246756 (secure 0 0 0) 0x7f4950009e10 con 0x7f49400679e0 2024-04-25T00:36:43.851 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:43.852 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:36:43.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.848+0000 7f4965aa9640 1 -- 172.21.15.29:0/2461201817 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49400679e0 msgr2=0x7f4940069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.848+0000 7f4965aa9640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49400679e0 0x7f4940069ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f49600ceca0 tx=0x7f4950008040 comp rx=0 tx=0).stop 2024-04-25T00:36:43.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.848+0000 7f4965aa9640 1 -- 172.21.15.29:0/2461201817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f496015f3c0 msgr2=0x7f496013f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:43.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.848+0000 7f4965aa9640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f496015f3c0 0x7f496013f800 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f495801a5e0 tx=0x7f4958019750 comp rx=0 tx=0).stop 2024-04-25T00:36:43.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.848+0000 7f4965aa9640 1 -- 172.21.15.29:0/2461201817 shutdown_connections 2024-04-25T00:36:43.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.848+0000 7f4965aa9640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49400679e0 0x7f4940069ea0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.848+0000 7f4965aa9640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f496015f3c0 0x7f496013f800 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.848+0000 7f4965aa9640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f496015e8d0 0x7f49600aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.848+0000 7f4965aa9640 1 --2- 172.21.15.29:0/2461201817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49600b4780 0x7f4960143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.848+0000 7f4965aa9640 1 -- 172.21.15.29:0/2461201817 >> 172.21.15.29:0/2461201817 conn(0x7f49600b1870 msgr2=0x7f496006dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.848+0000 7f4965aa9640 1 -- 172.21.15.29:0/2461201817 shutdown_connections 2024-04-25T00:36:43.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:43.848+0000 7f4965aa9640 1 -- 172.21.15.29:0/2461201817 wait complete. 2024-04-25T00:36:43.869 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-25T00:36:43.312352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283400,"kb_used_data":8984,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754691320,"statfs":{"total":773094113280,"available":772803911680,"internally_reserved":0,"allocated":9199616,"data_stored":4606732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.519429"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825116+0000","last_change":"2024-04-25T00:36:38.825116+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825116+0000","last_undegraded":"2024-04-25T00:36:38.825116+0000","last_fullsized":"2024-04-25T00:36:38.825116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816578+0000","last_change":"2024-04-25T00:36:38.816578+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816578+0000","last_undegraded":"2024-04-25T00:36:38.816578+0000","last_fullsized":"2024-04-25T00:36:38.816578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.821596+0000","last_change":"2024-04-25T00:36:38.821596+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.821596+0000","last_undegraded":"2024-04-25T00:36:38.821596+0000","last_fullsized":"2024-04-25T00:36:38.821596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.829028+0000","last_change":"2024-04-25T00:36:38.829028+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.829028+0000","last_undegraded":"2024-04-25T00:36:38.829028+0000","last_fullsized":"2024-04-25T00:36:38.829028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.817867+0000","last_change":"2024-04-25T00:36:38.817867+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.817867+0000","last_undegraded":"2024-04-25T00:36:38.817867+0000","last_fullsized":"2024-04-25T00:36:38.817867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.827070+0000","last_change":"2024-04-25T00:36:38.827070+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.827070+0000","last_undegraded":"2024-04-25T00:36:38.827070+0000","last_fullsized":"2024-04-25T00:36:38.827070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825982+0000","last_change":"2024-04-25T00:36:38.825982+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825982+0000","last_undegraded":"2024-04-25T00:36:38.825982+0000","last_fullsized":"2024-04-25T00:36:38.825982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.824317+0000","last_change":"2024-04-25T00:36:38.824317+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.824317+0000","last_undegraded":"2024-04-25T00:36:38.824317+0000","last_fullsized":"2024-04-25T00:36:38.824317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.827607+0000","last_change":"2024-04-25T00:36:38.827607+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.827607+0000","last_undegraded":"2024-04-25T00:36:38.827607+0000","last_fullsized":"2024-04-25T00:36:38.827607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813471+0000","last_change":"2024-04-25T00:36:38.813471+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813471+0000","last_undegraded":"2024-04-25T00:36:38.813471+0000","last_fullsized":"2024-04-25T00:36:38.813471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814021+0000","last_change":"2024-04-25T00:36:38.814021+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814021+0000","last_undegraded":"2024-04-25T00:36:38.814021+0000","last_fullsized":"2024-04-25T00:36:38.814021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815493+0000","last_change":"2024-04-25T00:36:38.815493+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815493+0000","last_undegraded":"2024-04-25T00:36:38.815493+0000","last_fullsized":"2024-04-25T00:36:38.815493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.822046+0000","last_change":"2024-04-25T00:36:38.822046+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.822046+0000","last_undegraded":"2024-04-25T00:36:38.822046+0000","last_fullsized":"2024-04-25T00:36:38.822046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825738+0000","last_change":"2024-04-25T00:36:38.825738+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825738+0000","last_undegraded":"2024-04-25T00:36:38.825738+0000","last_fullsized":"2024-04-25T00:36:38.825738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815105+0000","last_change":"2024-04-25T00:36:38.815105+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815105+0000","last_undegraded":"2024-04-25T00:36:38.815105+0000","last_fullsized":"2024-04-25T00:36:38.815105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815145+0000","last_change":"2024-04-25T00:36:38.815145+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815145+0000","last_undegraded":"2024-04-25T00:36:38.815145+0000","last_fullsized":"2024-04-25T00:36:38.815145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.819692+0000","last_change":"2024-04-25T00:36:38.819692+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.819692+0000","last_undegraded":"2024-04-25T00:36:38.819692+0000","last_fullsized":"2024-04-25T00:36:38.819692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818362+0000","last_change":"2024-04-25T00:36:38.818362+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818362+0000","last_undegraded":"2024-04-25T00:36:38.818362+0000","last_fullsized":"2024-04-25T00:36:38.818362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.808911+0000","last_change":"2024-04-25T00:36:38.808911+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.808911+0000","last_undegraded":"2024-04-25T00:36:38.808911+0000","last_fullsized":"2024-04-25T00:36:38.808911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818455+0000","last_change":"2024-04-25T00:36:38.818455+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818455+0000","last_undegraded":"2024-04-25T00:36:38.818455+0000","last_fullsized":"2024-04-25T00:36:38.818455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818675+0000","last_change":"2024-04-25T00:36:38.818675+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818675+0000","last_undegraded":"2024-04-25T00:36:38.818675+0000","last_fullsized":"2024-04-25T00:36:38.818675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812757+0000","last_change":"2024-04-25T00:36:38.812757+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812757+0000","last_undegraded":"2024-04-25T00:36:38.812757+0000","last_fullsized":"2024-04-25T00:36:38.812757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812237+0000","last_change":"2024-04-25T00:36:38.812237+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812237+0000","last_undegraded":"2024-04-25T00:36:38.812237+0000","last_fullsized":"2024-04-25T00:36:38.812237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818137+0000","last_change":"2024-04-25T00:36:38.818137+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818137+0000","last_undegraded":"2024-04-25T00:36:38.818137+0000","last_fullsized":"2024-04-25T00:36:38.818137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813023+0000","last_change":"2024-04-25T00:36:38.813023+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813023+0000","last_undegraded":"2024-04-25T00:36:38.813023+0000","last_fullsized":"2024-04-25T00:36:38.813023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.820114+0000","last_change":"2024-04-25T00:36:38.820114+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.820114+0000","last_undegraded":"2024-04-25T00:36:38.820114+0000","last_fullsized":"2024-04-25T00:36:38.820114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818582+0000","last_change":"2024-04-25T00:36:38.818582+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818582+0000","last_undegraded":"2024-04-25T00:36:38.818582+0000","last_fullsized":"2024-04-25T00:36:38.818582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.822677+0000","last_change":"2024-04-25T00:36:38.822677+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.822677+0000","last_undegraded":"2024-04-25T00:36:38.822677+0000","last_fullsized":"2024-04-25T00:36:38.822677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812989+0000","last_change":"2024-04-25T00:36:38.812989+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812989+0000","last_undegraded":"2024-04-25T00:36:38.812989+0000","last_fullsized":"2024-04-25T00:36:38.812989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811755+0000","last_change":"2024-04-25T00:36:38.811755+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811755+0000","last_undegraded":"2024-04-25T00:36:38.811755+0000","last_fullsized":"2024-04-25T00:36:38.811755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814147+0000","last_change":"2024-04-25T00:36:38.814147+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814147+0000","last_undegraded":"2024-04-25T00:36:38.814147+0000","last_fullsized":"2024-04-25T00:36:38.814147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811325+0000","last_change":"2024-04-25T00:36:38.811325+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811325+0000","last_undegraded":"2024-04-25T00:36:38.811325+0000","last_fullsized":"2024-04-25T00:36:38.811325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.819706+0000","last_change":"2024-04-25T00:36:38.819706+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.819706+0000","last_undegraded":"2024-04-25T00:36:38.819706+0000","last_fullsized":"2024-04-25T00:36:38.819706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845710+0000","last_change":"2024-04-25T00:36:38.842146+0000","last_active":"2024-04-25T00:36:38.845710+0000","last_peered":"2024-04-25T00:36:38.845710+0000","last_clean":"2024-04-25T00:36:38.845710+0000","last_became_active":"2024-04-25T00:36:38.841379+0000","last_became_peered":"2024-04-25T00:36:38.841379+0000","last_unstale":"2024-04-25T00:36:38.845710+0000","last_undegraded":"2024-04-25T00:36:38.845710+0000","last_fullsized":"2024-04-25T00:36:38.845710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:54.615183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.824175+0000","last_change":"2024-04-25T00:36:38.824175+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.824175+0000","last_undegraded":"2024-04-25T00:36:38.824175+0000","last_fullsized":"2024-04-25T00:36:38.824175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846934+0000","last_change":"2024-04-25T00:36:38.836018+0000","last_active":"2024-04-25T00:36:38.846934+0000","last_peered":"2024-04-25T00:36:38.846934+0000","last_clean":"2024-04-25T00:36:38.846934+0000","last_became_active":"2024-04-25T00:36:38.835209+0000","last_became_peered":"2024-04-25T00:36:38.835209+0000","last_unstale":"2024-04-25T00:36:38.846934+0000","last_undegraded":"2024-04-25T00:36:38.846934+0000","last_fullsized":"2024-04-25T00:36:38.846934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:34.231936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.821618+0000","last_change":"2024-04-25T00:36:38.821618+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.821618+0000","last_undegraded":"2024-04-25T00:36:38.821618+0000","last_fullsized":"2024-04-25T00:36:38.821618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.829415+0000","last_change":"2024-04-25T00:36:38.828383+0000","last_active":"2024-04-25T00:36:38.829415+0000","last_peered":"2024-04-25T00:36:38.829415+0000","last_clean":"2024-04-25T00:36:38.829415+0000","last_became_active":"2024-04-25T00:36:38.827081+0000","last_became_peered":"2024-04-25T00:36:38.827081+0000","last_unstale":"2024-04-25T00:36:38.829415+0000","last_undegraded":"2024-04-25T00:36:38.829415+0000","last_fullsized":"2024-04-25T00:36:38.829415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:49.597505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.820427+0000","last_change":"2024-04-25T00:36:38.820427+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.820427+0000","last_undegraded":"2024-04-25T00:36:38.820427+0000","last_fullsized":"2024-04-25T00:36:38.820427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.827222+0000","last_change":"2024-04-25T00:36:38.820892+0000","last_active":"2024-04-25T00:36:38.827222+0000","last_peered":"2024-04-25T00:36:38.827222+0000","last_clean":"2024-04-25T00:36:38.827222+0000","last_became_active":"2024-04-25T00:36:38.820603+0000","last_became_peered":"2024-04-25T00:36:38.820603+0000","last_unstale":"2024-04-25T00:36:38.827222+0000","last_undegraded":"2024-04-25T00:36:38.827222+0000","last_fullsized":"2024-04-25T00:36:38.827222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:04.183502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811231+0000","last_change":"2024-04-25T00:36:38.811231+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811231+0000","last_undegraded":"2024-04-25T00:36:38.811231+0000","last_fullsized":"2024-04-25T00:36:38.811231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.847018+0000","last_change":"2024-04-25T00:36:38.837177+0000","last_active":"2024-04-25T00:36:38.847018+0000","last_peered":"2024-04-25T00:36:38.847018+0000","last_clean":"2024-04-25T00:36:38.847018+0000","last_became_active":"2024-04-25T00:36:38.836519+0000","last_became_peered":"2024-04-25T00:36:38.836519+0000","last_unstale":"2024-04-25T00:36:38.847018+0000","last_undegraded":"2024-04-25T00:36:38.847018+0000","last_fullsized":"2024-04-25T00:36:38.847018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:48.763968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816742+0000","last_change":"2024-04-25T00:36:38.816742+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816742+0000","last_undegraded":"2024-04-25T00:36:38.816742+0000","last_fullsized":"2024-04-25T00:36:38.816742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834797+0000","last_change":"2024-04-25T00:36:38.832260+0000","last_active":"2024-04-25T00:36:38.834797+0000","last_peered":"2024-04-25T00:36:38.834797+0000","last_clean":"2024-04-25T00:36:38.834797+0000","last_became_active":"2024-04-25T00:36:38.829378+0000","last_became_peered":"2024-04-25T00:36:38.829378+0000","last_unstale":"2024-04-25T00:36:38.834797+0000","last_undegraded":"2024-04-25T00:36:38.834797+0000","last_fullsized":"2024-04-25T00:36:38.834797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:58.973792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.820408+0000","last_change":"2024-04-25T00:36:38.820408+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.820408+0000","last_undegraded":"2024-04-25T00:36:38.820408+0000","last_fullsized":"2024-04-25T00:36:38.820408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.838358+0000","last_change":"2024-04-25T00:36:38.837498+0000","last_active":"2024-04-25T00:36:38.838358+0000","last_peered":"2024-04-25T00:36:38.838358+0000","last_clean":"2024-04-25T00:36:38.838358+0000","last_became_active":"2024-04-25T00:36:38.836899+0000","last_became_peered":"2024-04-25T00:36:38.836899+0000","last_unstale":"2024-04-25T00:36:38.838358+0000","last_undegraded":"2024-04-25T00:36:38.838358+0000","last_fullsized":"2024-04-25T00:36:38.838358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:25.603144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814334+0000","last_change":"2024-04-25T00:36:38.814334+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814334+0000","last_undegraded":"2024-04-25T00:36:38.814334+0000","last_fullsized":"2024-04-25T00:36:38.814334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.838637+0000","last_change":"2024-04-25T00:36:38.837750+0000","last_active":"2024-04-25T00:36:38.838637+0000","last_peered":"2024-04-25T00:36:38.838637+0000","last_clean":"2024-04-25T00:36:38.838637+0000","last_became_active":"2024-04-25T00:36:38.836970+0000","last_became_peered":"2024-04-25T00:36:38.836970+0000","last_unstale":"2024-04-25T00:36:38.838637+0000","last_undegraded":"2024-04-25T00:36:38.838637+0000","last_fullsized":"2024-04-25T00:36:38.838637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:27.980427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816222+0000","last_change":"2024-04-25T00:36:38.816222+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816222+0000","last_undegraded":"2024-04-25T00:36:38.816222+0000","last_fullsized":"2024-04-25T00:36:38.816222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846512+0000","last_change":"2024-04-25T00:36:38.842829+0000","last_active":"2024-04-25T00:36:38.846512+0000","last_peered":"2024-04-25T00:36:38.846512+0000","last_clean":"2024-04-25T00:36:38.846512+0000","last_became_active":"2024-04-25T00:36:38.838997+0000","last_became_peered":"2024-04-25T00:36:38.838997+0000","last_unstale":"2024-04-25T00:36:38.846512+0000","last_undegraded":"2024-04-25T00:36:38.846512+0000","last_fullsized":"2024-04-25T00:36:38.846512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:39.487660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.826636+0000","last_change":"2024-04-25T00:36:38.826636+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.826636+0000","last_undegraded":"2024-04-25T00:36:38.826636+0000","last_fullsized":"2024-04-25T00:36:38.826636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845605+0000","last_change":"2024-04-25T00:36:38.840680+0000","last_active":"2024-04-25T00:36:38.845605+0000","last_peered":"2024-04-25T00:36:38.845605+0000","last_clean":"2024-04-25T00:36:38.845605+0000","last_became_active":"2024-04-25T00:36:38.839742+0000","last_became_peered":"2024-04-25T00:36:38.839742+0000","last_unstale":"2024-04-25T00:36:38.845605+0000","last_undegraded":"2024-04-25T00:36:38.845605+0000","last_fullsized":"2024-04-25T00:36:38.845605+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:48.763968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.810011+0000","last_change":"2024-04-25T00:36:38.810011+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.810011+0000","last_undegraded":"2024-04-25T00:36:38.810011+0000","last_fullsized":"2024-04-25T00:36:38.810011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846500+0000","last_change":"2024-04-25T00:36:38.842786+0000","last_active":"2024-04-25T00:36:38.846500+0000","last_peered":"2024-04-25T00:36:38.846500+0000","last_clean":"2024-04-25T00:36:38.846500+0000","last_became_active":"2024-04-25T00:36:38.842159+0000","last_became_peered":"2024-04-25T00:36:38.842159+0000","last_unstale":"2024-04-25T00:36:38.846500+0000","last_undegraded":"2024-04-25T00:36:38.846500+0000","last_fullsized":"2024-04-25T00:36:38.846500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:36.318064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.817507+0000","last_change":"2024-04-25T00:36:38.817507+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.817507+0000","last_undegraded":"2024-04-25T00:36:38.817507+0000","last_fullsized":"2024-04-25T00:36:38.817507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.844370+0000","last_change":"2024-04-25T00:36:38.837497+0000","last_active":"2024-04-25T00:36:38.844370+0000","last_peered":"2024-04-25T00:36:38.844370+0000","last_clean":"2024-04-25T00:36:38.844370+0000","last_became_active":"2024-04-25T00:36:38.836746+0000","last_became_peered":"2024-04-25T00:36:38.836746+0000","last_unstale":"2024-04-25T00:36:38.844370+0000","last_undegraded":"2024-04-25T00:36:38.844370+0000","last_fullsized":"2024-04-25T00:36:38.844370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:49.597505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.808277+0000","last_change":"2024-04-25T00:36:38.808277+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.808277+0000","last_undegraded":"2024-04-25T00:36:38.808277+0000","last_fullsized":"2024-04-25T00:36:38.808277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.833351+0000","last_change":"2024-04-25T00:36:38.828622+0000","last_active":"2024-04-25T00:36:38.833351+0000","last_peered":"2024-04-25T00:36:38.833351+0000","last_clean":"2024-04-25T00:36:38.833351+0000","last_became_active":"2024-04-25T00:36:38.828270+0000","last_became_peered":"2024-04-25T00:36:38.828270+0000","last_unstale":"2024-04-25T00:36:38.833351+0000","last_undegraded":"2024-04-25T00:36:38.833351+0000","last_fullsized":"2024-04-25T00:36:38.833351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:33.405161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814505+0000","last_change":"2024-04-25T00:36:38.814505+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814505+0000","last_undegraded":"2024-04-25T00:36:38.814505+0000","last_fullsized":"2024-04-25T00:36:38.814505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.835128+0000","last_change":"2024-04-25T00:36:38.833289+0000","last_active":"2024-04-25T00:36:38.835128+0000","last_peered":"2024-04-25T00:36:38.835128+0000","last_clean":"2024-04-25T00:36:38.835128+0000","last_became_active":"2024-04-25T00:36:38.832940+0000","last_became_peered":"2024-04-25T00:36:38.832940+0000","last_unstale":"2024-04-25T00:36:38.835128+0000","last_undegraded":"2024-04-25T00:36:38.835128+0000","last_fullsized":"2024-04-25T00:36:38.835128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:15.477777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811606+0000","last_change":"2024-04-25T00:36:38.811606+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811606+0000","last_undegraded":"2024-04-25T00:36:38.811606+0000","last_fullsized":"2024-04-25T00:36:38.811606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.836292+0000","last_change":"2024-04-25T00:36:38.834888+0000","last_active":"2024-04-25T00:36:38.836292+0000","last_peered":"2024-04-25T00:36:38.836292+0000","last_clean":"2024-04-25T00:36:38.836292+0000","last_became_active":"2024-04-25T00:36:38.833935+0000","last_became_peered":"2024-04-25T00:36:38.833935+0000","last_unstale":"2024-04-25T00:36:38.836292+0000","last_undegraded":"2024-04-25T00:36:38.836292+0000","last_fullsized":"2024-04-25T00:36:38.836292+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:04.183502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811323+0000","last_change":"2024-04-25T00:36:38.811323+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811323+0000","last_undegraded":"2024-04-25T00:36:38.811323+0000","last_fullsized":"2024-04-25T00:36:38.811323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834500+0000","last_change":"2024-04-25T00:36:38.833368+0000","last_active":"2024-04-25T00:36:38.834500+0000","last_peered":"2024-04-25T00:36:38.834500+0000","last_clean":"2024-04-25T00:36:38.834500+0000","last_became_active":"2024-04-25T00:36:38.832870+0000","last_became_peered":"2024-04-25T00:36:38.832870+0000","last_unstale":"2024-04-25T00:36:38.834500+0000","last_undegraded":"2024-04-25T00:36:38.834500+0000","last_fullsized":"2024-04-25T00:36:38.834500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:05.341588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813368+0000","last_change":"2024-04-25T00:36:38.813368+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813368+0000","last_undegraded":"2024-04-25T00:36:38.813368+0000","last_fullsized":"2024-04-25T00:36:38.813368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.847195+0000","last_change":"2024-04-25T00:36:38.845296+0000","last_active":"2024-04-25T00:36:38.847195+0000","last_peered":"2024-04-25T00:36:38.847195+0000","last_clean":"2024-04-25T00:36:38.847195+0000","last_became_active":"2024-04-25T00:36:38.844447+0000","last_became_peered":"2024-04-25T00:36:38.844447+0000","last_unstale":"2024-04-25T00:36:38.847195+0000","last_undegraded":"2024-04-25T00:36:38.847195+0000","last_fullsized":"2024-04-25T00:36:38.847195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:38.147666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816628+0000","last_change":"2024-04-25T00:36:38.816628+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816628+0000","last_undegraded":"2024-04-25T00:36:38.816628+0000","last_fullsized":"2024-04-25T00:36:38.816628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.837861+0000","last_change":"2024-04-25T00:36:38.836218+0000","last_active":"2024-04-25T00:36:38.837861+0000","last_peered":"2024-04-25T00:36:38.837861+0000","last_clean":"2024-04-25T00:36:38.837861+0000","last_became_active":"2024-04-25T00:36:38.835946+0000","last_became_peered":"2024-04-25T00:36:38.835946+0000","last_unstale":"2024-04-25T00:36:38.837861+0000","last_undegraded":"2024-04-25T00:36:38.837861+0000","last_fullsized":"2024-04-25T00:36:38.837861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:48.430422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816425+0000","last_change":"2024-04-25T00:36:38.816425+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816425+0000","last_undegraded":"2024-04-25T00:36:38.816425+0000","last_fullsized":"2024-04-25T00:36:38.816425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.838313+0000","last_change":"2024-04-25T00:36:38.836980+0000","last_active":"2024-04-25T00:36:38.838313+0000","last_peered":"2024-04-25T00:36:38.838313+0000","last_clean":"2024-04-25T00:36:38.838313+0000","last_became_active":"2024-04-25T00:36:38.836722+0000","last_became_peered":"2024-04-25T00:36:38.836722+0000","last_unstale":"2024-04-25T00:36:38.838313+0000","last_undegraded":"2024-04-25T00:36:38.838313+0000","last_fullsized":"2024-04-25T00:36:38.838313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:29.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813225+0000","last_change":"2024-04-25T00:36:38.813225+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813225+0000","last_undegraded":"2024-04-25T00:36:38.813225+0000","last_fullsized":"2024-04-25T00:36:38.813225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.829873+0000","last_change":"2024-04-25T00:36:38.827338+0000","last_active":"2024-04-25T00:36:38.829873+0000","last_peered":"2024-04-25T00:36:38.829873+0000","last_clean":"2024-04-25T00:36:38.829873+0000","last_became_active":"2024-04-25T00:36:38.826884+0000","last_became_peered":"2024-04-25T00:36:38.826884+0000","last_unstale":"2024-04-25T00:36:38.829873+0000","last_undegraded":"2024-04-25T00:36:38.829873+0000","last_fullsized":"2024-04-25T00:36:38.829873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:10.228966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.828168+0000","last_change":"2024-04-25T00:36:38.828168+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.828168+0000","last_undegraded":"2024-04-25T00:36:38.828168+0000","last_fullsized":"2024-04-25T00:36:38.828168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834354+0000","last_change":"2024-04-25T00:36:38.829621+0000","last_active":"2024-04-25T00:36:38.834354+0000","last_peered":"2024-04-25T00:36:38.834354+0000","last_clean":"2024-04-25T00:36:38.834354+0000","last_became_active":"2024-04-25T00:36:38.827960+0000","last_became_peered":"2024-04-25T00:36:38.827960+0000","last_unstale":"2024-04-25T00:36:38.834354+0000","last_undegraded":"2024-04-25T00:36:38.834354+0000","last_fullsized":"2024-04-25T00:36:38.834354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:56.562023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.810472+0000","last_change":"2024-04-25T00:36:38.810472+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.810472+0000","last_undegraded":"2024-04-25T00:36:38.810472+0000","last_fullsized":"2024-04-25T00:36:38.810472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.827971+0000","last_change":"2024-04-25T00:36:38.823571+0000","last_active":"2024-04-25T00:36:38.827971+0000","last_peered":"2024-04-25T00:36:38.827971+0000","last_clean":"2024-04-25T00:36:38.827971+0000","last_became_active":"2024-04-25T00:36:38.823275+0000","last_became_peered":"2024-04-25T00:36:38.823275+0000","last_unstale":"2024-04-25T00:36:38.827971+0000","last_undegraded":"2024-04-25T00:36:38.827971+0000","last_fullsized":"2024-04-25T00:36:38.827971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:51.207453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.828686+0000","last_change":"2024-04-25T00:36:38.828686+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.828686+0000","last_undegraded":"2024-04-25T00:36:38.828686+0000","last_fullsized":"2024-04-25T00:36:38.828686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.829729+0000","last_change":"2024-04-25T00:36:38.827233+0000","last_active":"2024-04-25T00:36:38.829729+0000","last_peered":"2024-04-25T00:36:38.829729+0000","last_clean":"2024-04-25T00:36:38.829729+0000","last_became_active":"2024-04-25T00:36:38.826902+0000","last_became_peered":"2024-04-25T00:36:38.826902+0000","last_unstale":"2024-04-25T00:36:38.829729+0000","last_undegraded":"2024-04-25T00:36:38.829729+0000","last_fullsized":"2024-04-25T00:36:38.829729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:20.855968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818674+0000","last_change":"2024-04-25T00:36:38.818674+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818674+0000","last_undegraded":"2024-04-25T00:36:38.818674+0000","last_fullsized":"2024-04-25T00:36:38.818674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.839320+0000","last_change":"2024-04-25T00:36:38.838509+0000","last_active":"2024-04-25T00:36:38.839320+0000","last_peered":"2024-04-25T00:36:38.839320+0000","last_clean":"2024-04-25T00:36:38.839320+0000","last_became_active":"2024-04-25T00:36:38.837928+0000","last_became_peered":"2024-04-25T00:36:38.837928+0000","last_unstale":"2024-04-25T00:36:38.839320+0000","last_undegraded":"2024-04-25T00:36:38.839320+0000","last_fullsized":"2024-04-25T00:36:38.839320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:06.637343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812126+0000","last_change":"2024-04-25T00:36:38.812126+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812126+0000","last_undegraded":"2024-04-25T00:36:38.812126+0000","last_fullsized":"2024-04-25T00:36:38.812126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.833501+0000","last_change":"2024-04-25T00:36:38.828691+0000","last_active":"2024-04-25T00:36:38.833501+0000","last_peered":"2024-04-25T00:36:38.833501+0000","last_clean":"2024-04-25T00:36:38.833501+0000","last_became_active":"2024-04-25T00:36:38.828380+0000","last_became_peered":"2024-04-25T00:36:38.828380+0000","last_unstale":"2024-04-25T00:36:38.833501+0000","last_undegraded":"2024-04-25T00:36:38.833501+0000","last_fullsized":"2024-04-25T00:36:38.833501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:22.410800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816547+0000","last_change":"2024-04-25T00:36:38.816547+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816547+0000","last_undegraded":"2024-04-25T00:36:38.816547+0000","last_fullsized":"2024-04-25T00:36:38.816547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845767+0000","last_change":"2024-04-25T00:36:38.841467+0000","last_active":"2024-04-25T00:36:38.845767+0000","last_peered":"2024-04-25T00:36:38.845767+0000","last_clean":"2024-04-25T00:36:38.845767+0000","last_became_active":"2024-04-25T00:36:38.840182+0000","last_became_peered":"2024-04-25T00:36:38.840182+0000","last_unstale":"2024-04-25T00:36:38.845767+0000","last_undegraded":"2024-04-25T00:36:38.845767+0000","last_fullsized":"2024-04-25T00:36:38.845767+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:53.012076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815364+0000","last_change":"2024-04-25T00:36:38.815364+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815364+0000","last_undegraded":"2024-04-25T00:36:38.815364+0000","last_fullsized":"2024-04-25T00:36:38.815364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845638+0000","last_change":"2024-04-25T00:36:38.841047+0000","last_active":"2024-04-25T00:36:38.845638+0000","last_peered":"2024-04-25T00:36:38.845638+0000","last_clean":"2024-04-25T00:36:38.845638+0000","last_became_active":"2024-04-25T00:36:38.839481+0000","last_became_peered":"2024-04-25T00:36:38.839481+0000","last_unstale":"2024-04-25T00:36:38.845638+0000","last_undegraded":"2024-04-25T00:36:38.845638+0000","last_fullsized":"2024-04-25T00:36:38.845638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:11.985160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825951+0000","last_change":"2024-04-25T00:36:38.825951+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825951+0000","last_undegraded":"2024-04-25T00:36:38.825951+0000","last_fullsized":"2024-04-25T00:36:38.825951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.826182+0000","last_change":"2024-04-25T00:36:38.819664+0000","last_active":"2024-04-25T00:36:38.826182+0000","last_peered":"2024-04-25T00:36:38.826182+0000","last_clean":"2024-04-25T00:36:38.826182+0000","last_became_active":"2024-04-25T00:36:38.819378+0000","last_became_peered":"2024-04-25T00:36:38.819378+0000","last_unstale":"2024-04-25T00:36:38.826182+0000","last_undegraded":"2024-04-25T00:36:38.826182+0000","last_fullsized":"2024-04-25T00:36:38.826182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:52.487791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.807503+0000","last_change":"2024-04-25T00:36:38.807503+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.807503+0000","last_undegraded":"2024-04-25T00:36:38.807503+0000","last_fullsized":"2024-04-25T00:36:38.807503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834734+0000","last_change":"2024-04-25T00:36:38.830646+0000","last_active":"2024-04-25T00:36:38.834734+0000","last_peered":"2024-04-25T00:36:38.834734+0000","last_clean":"2024-04-25T00:36:38.834734+0000","last_became_active":"2024-04-25T00:36:38.829932+0000","last_became_peered":"2024-04-25T00:36:38.829932+0000","last_unstale":"2024-04-25T00:36:38.834734+0000","last_undegraded":"2024-04-25T00:36:38.834734+0000","last_fullsized":"2024-04-25T00:36:38.834734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:57.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818106+0000","last_change":"2024-04-25T00:36:38.818106+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818106+0000","last_undegraded":"2024-04-25T00:36:38.818106+0000","last_fullsized":"2024-04-25T00:36:38.818106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846554+0000","last_change":"2024-04-25T00:36:38.840716+0000","last_active":"2024-04-25T00:36:38.846554+0000","last_peered":"2024-04-25T00:36:38.846554+0000","last_clean":"2024-04-25T00:36:38.846554+0000","last_became_active":"2024-04-25T00:36:38.837997+0000","last_became_peered":"2024-04-25T00:36:38.837997+0000","last_unstale":"2024-04-25T00:36:38.846554+0000","last_undegraded":"2024-04-25T00:36:38.846554+0000","last_fullsized":"2024-04-25T00:36:38.846554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:54.469425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813359+0000","last_change":"2024-04-25T00:36:38.813359+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813359+0000","last_undegraded":"2024-04-25T00:36:38.813359+0000","last_fullsized":"2024-04-25T00:36:38.813359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846920+0000","last_change":"2024-04-25T00:36:38.834405+0000","last_active":"2024-04-25T00:36:38.846920+0000","last_peered":"2024-04-25T00:36:38.846920+0000","last_clean":"2024-04-25T00:36:38.846920+0000","last_became_active":"2024-04-25T00:36:38.833769+0000","last_became_peered":"2024-04-25T00:36:38.833769+0000","last_unstale":"2024-04-25T00:36:38.846920+0000","last_undegraded":"2024-04-25T00:36:38.846920+0000","last_fullsized":"2024-04-25T00:36:38.846920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:58.559610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814258+0000","last_change":"2024-04-25T00:36:38.814258+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814258+0000","last_undegraded":"2024-04-25T00:36:38.814258+0000","last_fullsized":"2024-04-25T00:36:38.814258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.842399+0000","last_change":"2024-04-25T00:36:38.835037+0000","last_active":"2024-04-25T00:36:38.842399+0000","last_peered":"2024-04-25T00:36:38.842399+0000","last_clean":"2024-04-25T00:36:38.842399+0000","last_became_active":"2024-04-25T00:36:38.833930+0000","last_became_peered":"2024-04-25T00:36:38.833930+0000","last_unstale":"2024-04-25T00:36:38.842399+0000","last_undegraded":"2024-04-25T00:36:38.842399+0000","last_fullsized":"2024-04-25T00:36:38.842399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:48.676020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":327,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.808362+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:36:38.808362+0000","last_peered":"2024-04-25T00:36:38.808362+0000","last_clean":"2024-04-25T00:36:38.808362+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:36:38.808362+0000","last_undegraded":"2024-04-25T00:36:38.808362+0000","last_fullsized":"2024-04-25T00:36:38.808362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36908,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334932,"statfs":{"total":96636764160,"available":96598970368,"internally_reserved":0,"allocated":1601536,"data_stored":1021755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":431387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":425112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1007616,"data_stored":431387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":995328,"data_stored":425112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":995328,"data_stored":425112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1007616,"data_stored":431387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36892,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334948,"statfs":{"total":96636764160,"available":96598986752,"internally_reserved":0,"allocated":1589248,"data_stored":1015480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:36:43.874 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:36:44.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe9ad11640 1 Processor -- start 2024-04-25T00:36:44.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe9ad11640 1 -- start start 2024-04-25T00:36:44.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe9ad11640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe9415e8d0 0x7fbe940cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe9ad11640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbe9415f3c0 0x7fbe9415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe9ad11640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe940b3720 con 0x7fbe940b4780 2024-04-25T00:36:44.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe9ad11640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbe940b42f0 con 0x7fbe9415f3c0 2024-04-25T00:36:44.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe9ad11640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbe940b39c0 con 0x7fbe9415e8d0 2024-04-25T00:36:44.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe992ae640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbe9415f3c0 0x7fbe9415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe992ae640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbe9415f3c0 0x7fbe9415c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37006/0 (socket says 172.21.15.29:37006) 2024-04-25T00:36:44.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe992ae640 1 -- 172.21.15.29:0/3672267779 learned_addr learned my addr 172.21.15.29:0/3672267779 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:44.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe992ae640 1 -- 172.21.15.29:0/3672267779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe9415e8d0 msgr2=0x7fbe940cc240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:36:44.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe992ae640 1 --2- 172.21.15.29:0/3672267779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe9415e8d0 0x7fbe940cc240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe992ae640 1 -- 172.21.15.29:0/3672267779 >> v1:172.21.15.29:6789/0 conn(0x7fbe940b4780 legacy=0x7fbe9415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.024+0000 7fbe992ae640 1 -- 172.21.15.29:0/3672267779 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe940b3ca0 con 0x7fbe9415f3c0 2024-04-25T00:36:44.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe992ae640 1 --2- 172.21.15.29:0/3672267779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe9415f3c0 0x7fbe9415c7d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009e50 tx=0x7fbe8401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef29ef1f177fc36b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:44.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe937fe640 1 -- 172.21.15.29:0/3672267779 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe840a58d0 con 0x7fbe9415f3c0 2024-04-25T00:36:44.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe937fe640 1 -- 172.21.15.29:0/3672267779 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe840a5a70 con 0x7fbe9415f3c0 2024-04-25T00:36:44.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe937fe640 1 -- 172.21.15.29:0/3672267779 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe840a5e40 con 0x7fbe9415f3c0 2024-04-25T00:36:44.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe9ad11640 1 -- 172.21.15.29:0/3672267779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe9415f3c0 msgr2=0x7fbe9415c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe9ad11640 1 --2- 172.21.15.29:0/3672267779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe9415f3c0 0x7fbe9415c7d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009e50 tx=0x7fbe8401ca20 comp rx=0 tx=0).stop 2024-04-25T00:36:44.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe9ad11640 1 -- 172.21.15.29:0/3672267779 shutdown_connections 2024-04-25T00:36:44.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe9ad11640 1 --2- 172.21.15.29:0/3672267779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe9415f3c0 0x7fbe9415c7d0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe9ad11640 1 --2- 172.21.15.29:0/3672267779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe9415e8d0 0x7fbe940cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe9ad11640 1 -- 172.21.15.29:0/3672267779 >> 172.21.15.29:0/3672267779 conn(0x7fbe940b1870 msgr2=0x7fbe940b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:44.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe9ad11640 1 -- 172.21.15.29:0/3672267779 shutdown_connections 2024-04-25T00:36:44.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe9ad11640 1 -- 172.21.15.29:0/3672267779 wait complete. 2024-04-25T00:36:44.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe9ad11640 1 Processor -- start 2024-04-25T00:36:44.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.028+0000 7fbe9ad11640 1 -- start start 2024-04-25T00:36:44.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe9ad11640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe940b4780 0x7fbe9415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe9ad11640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe9415e8d0 0x7fbe9413b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe9ad11640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe9415f3c0 0x7fbe9413b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe9ad11640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9415d6a0 con 0x7fbe9415f3c0 2024-04-25T00:36:44.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe9ad11640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9415d520 con 0x7fbe940b4780 2024-04-25T00:36:44.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe9ad11640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbe94167c70 con 0x7fbe9415e8d0 2024-04-25T00:36:44.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe992ae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe9415f3c0 0x7fbe9413b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe992ae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe9415f3c0 0x7fbe9413b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37008/0 (socket says 172.21.15.29:37008) 2024-04-25T00:36:44.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe992ae640 1 -- 172.21.15.29:0/2155331077 learned_addr learned my addr 172.21.15.29:0/2155331077 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:44.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe992ae640 1 -- 172.21.15.29:0/2155331077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe9415e8d0 msgr2=0x7fbe9413b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe93fff640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe9415e8d0 0x7fbe9413b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe98aad640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe940b4780 0x7fbe9415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe992ae640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe9415e8d0 0x7fbe9413b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe992ae640 1 -- 172.21.15.29:0/2155331077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe940b4780 msgr2=0x7fbe9415a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe992ae640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe940b4780 0x7fbe9415a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe992ae640 1 -- 172.21.15.29:0/2155331077 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9413eb70 con 0x7fbe9415f3c0 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe93fff640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe9415e8d0 0x7fbe9413b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe98aad640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe940b4780 0x7fbe9415a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe992ae640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe9415f3c0 0x7fbe9413b650 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009e50 tx=0x7fbe840197c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe917fa640 1 -- 172.21.15.29:0/2155331077 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84009510 con 0x7fbe9415f3c0 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe917fa640 1 -- 172.21.15.29:0/2155331077 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe840be070 con 0x7fbe9415f3c0 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe9ad11640 1 -- 172.21.15.29:0/2155331077 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9413ee50 con 0x7fbe9415f3c0 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe917fa640 1 -- 172.21.15.29:0/2155331077 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe840b86b0 con 0x7fbe9415f3c0 2024-04-25T00:36:44.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe9ad11640 1 -- 172.21.15.29:0/2155331077 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9413f410 con 0x7fbe9415f3c0 2024-04-25T00:36:44.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe9ad11640 1 -- 172.21.15.29:0/2155331077 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe94069270 con 0x7fbe9415f3c0 2024-04-25T00:36:44.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe917fa640 1 -- 172.21.15.29:0/2155331077 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fbe840b8850 con 0x7fbe9415f3c0 2024-04-25T00:36:44.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe917fa640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbe740679e0 0x7fbe74069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe917fa640 1 -- 172.21.15.29:0/2155331077 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7fbe84122080 con 0x7fbe9415f3c0 2024-04-25T00:36:44.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.032+0000 7fbe98aad640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbe740679e0 0x7fbe74069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.036+0000 7fbe98aad640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbe740679e0 0x7fbe74069ea0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe94143a50 tx=0x7fbe8c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:44.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.036+0000 7fbe917fa640 1 -- 172.21.15.29:0/2155331077 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbe84016640 con 0x7fbe9415f3c0 2024-04-25T00:36:44.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.192+0000 7fbe9ad11640 1 -- 172.21.15.29:0/2155331077 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe9406e7b0 con 0x7fbe740679e0 2024-04-25T00:36:44.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.204+0000 7fbe917fa640 1 -- 172.21.15.29:0/2155331077 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246756 (secure 0 0 0) 0x7fbe9406e7b0 con 0x7fbe740679e0 2024-04-25T00:36:44.208 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:44.209 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:36:44.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.208+0000 7fbe9ad11640 1 -- 172.21.15.29:0/2155331077 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbe740679e0 msgr2=0x7fbe74069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.208+0000 7fbe9ad11640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbe740679e0 0x7fbe74069ea0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe94143a50 tx=0x7fbe8c008040 comp rx=0 tx=0).stop 2024-04-25T00:36:44.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.208+0000 7fbe9ad11640 1 -- 172.21.15.29:0/2155331077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe9415f3c0 msgr2=0x7fbe9413b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.208+0000 7fbe9ad11640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe9415f3c0 0x7fbe9413b650 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009e50 tx=0x7fbe840197c0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.208+0000 7fbe9ad11640 1 -- 172.21.15.29:0/2155331077 shutdown_connections 2024-04-25T00:36:44.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.208+0000 7fbe9ad11640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbe740679e0 0x7fbe74069ea0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.208+0000 7fbe9ad11640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe9415f3c0 0x7fbe9413b650 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.208+0000 7fbe9ad11640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe9415e8d0 0x7fbe9413b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.208+0000 7fbe9ad11640 1 --2- 172.21.15.29:0/2155331077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe940b4780 0x7fbe9415a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.208+0000 7fbe9ad11640 1 -- 172.21.15.29:0/2155331077 >> 172.21.15.29:0/2155331077 conn(0x7fbe940b1870 msgr2=0x7fbe940ad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:44.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.208+0000 7fbe9ad11640 1 -- 172.21.15.29:0/2155331077 shutdown_connections 2024-04-25T00:36:44.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.208+0000 7fbe9ad11640 1 -- 172.21.15.29:0/2155331077 wait complete. 2024-04-25T00:36:44.225 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-25T00:36:43.312352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283400,"kb_used_data":8984,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754691320,"statfs":{"total":773094113280,"available":772803911680,"internally_reserved":0,"allocated":9199616,"data_stored":4606732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.519429"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825116+0000","last_change":"2024-04-25T00:36:38.825116+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825116+0000","last_undegraded":"2024-04-25T00:36:38.825116+0000","last_fullsized":"2024-04-25T00:36:38.825116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816578+0000","last_change":"2024-04-25T00:36:38.816578+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816578+0000","last_undegraded":"2024-04-25T00:36:38.816578+0000","last_fullsized":"2024-04-25T00:36:38.816578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.821596+0000","last_change":"2024-04-25T00:36:38.821596+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.821596+0000","last_undegraded":"2024-04-25T00:36:38.821596+0000","last_fullsized":"2024-04-25T00:36:38.821596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.829028+0000","last_change":"2024-04-25T00:36:38.829028+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.829028+0000","last_undegraded":"2024-04-25T00:36:38.829028+0000","last_fullsized":"2024-04-25T00:36:38.829028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.817867+0000","last_change":"2024-04-25T00:36:38.817867+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.817867+0000","last_undegraded":"2024-04-25T00:36:38.817867+0000","last_fullsized":"2024-04-25T00:36:38.817867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.827070+0000","last_change":"2024-04-25T00:36:38.827070+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.827070+0000","last_undegraded":"2024-04-25T00:36:38.827070+0000","last_fullsized":"2024-04-25T00:36:38.827070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825982+0000","last_change":"2024-04-25T00:36:38.825982+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825982+0000","last_undegraded":"2024-04-25T00:36:38.825982+0000","last_fullsized":"2024-04-25T00:36:38.825982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.824317+0000","last_change":"2024-04-25T00:36:38.824317+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.824317+0000","last_undegraded":"2024-04-25T00:36:38.824317+0000","last_fullsized":"2024-04-25T00:36:38.824317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.827607+0000","last_change":"2024-04-25T00:36:38.827607+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.827607+0000","last_undegraded":"2024-04-25T00:36:38.827607+0000","last_fullsized":"2024-04-25T00:36:38.827607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813471+0000","last_change":"2024-04-25T00:36:38.813471+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813471+0000","last_undegraded":"2024-04-25T00:36:38.813471+0000","last_fullsized":"2024-04-25T00:36:38.813471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814021+0000","last_change":"2024-04-25T00:36:38.814021+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814021+0000","last_undegraded":"2024-04-25T00:36:38.814021+0000","last_fullsized":"2024-04-25T00:36:38.814021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815493+0000","last_change":"2024-04-25T00:36:38.815493+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815493+0000","last_undegraded":"2024-04-25T00:36:38.815493+0000","last_fullsized":"2024-04-25T00:36:38.815493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.822046+0000","last_change":"2024-04-25T00:36:38.822046+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.822046+0000","last_undegraded":"2024-04-25T00:36:38.822046+0000","last_fullsized":"2024-04-25T00:36:38.822046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825738+0000","last_change":"2024-04-25T00:36:38.825738+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825738+0000","last_undegraded":"2024-04-25T00:36:38.825738+0000","last_fullsized":"2024-04-25T00:36:38.825738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815105+0000","last_change":"2024-04-25T00:36:38.815105+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815105+0000","last_undegraded":"2024-04-25T00:36:38.815105+0000","last_fullsized":"2024-04-25T00:36:38.815105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815145+0000","last_change":"2024-04-25T00:36:38.815145+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815145+0000","last_undegraded":"2024-04-25T00:36:38.815145+0000","last_fullsized":"2024-04-25T00:36:38.815145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.819692+0000","last_change":"2024-04-25T00:36:38.819692+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.819692+0000","last_undegraded":"2024-04-25T00:36:38.819692+0000","last_fullsized":"2024-04-25T00:36:38.819692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818362+0000","last_change":"2024-04-25T00:36:38.818362+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818362+0000","last_undegraded":"2024-04-25T00:36:38.818362+0000","last_fullsized":"2024-04-25T00:36:38.818362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.808911+0000","last_change":"2024-04-25T00:36:38.808911+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.808911+0000","last_undegraded":"2024-04-25T00:36:38.808911+0000","last_fullsized":"2024-04-25T00:36:38.808911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818455+0000","last_change":"2024-04-25T00:36:38.818455+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818455+0000","last_undegraded":"2024-04-25T00:36:38.818455+0000","last_fullsized":"2024-04-25T00:36:38.818455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818675+0000","last_change":"2024-04-25T00:36:38.818675+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818675+0000","last_undegraded":"2024-04-25T00:36:38.818675+0000","last_fullsized":"2024-04-25T00:36:38.818675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812757+0000","last_change":"2024-04-25T00:36:38.812757+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812757+0000","last_undegraded":"2024-04-25T00:36:38.812757+0000","last_fullsized":"2024-04-25T00:36:38.812757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812237+0000","last_change":"2024-04-25T00:36:38.812237+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812237+0000","last_undegraded":"2024-04-25T00:36:38.812237+0000","last_fullsized":"2024-04-25T00:36:38.812237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818137+0000","last_change":"2024-04-25T00:36:38.818137+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818137+0000","last_undegraded":"2024-04-25T00:36:38.818137+0000","last_fullsized":"2024-04-25T00:36:38.818137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813023+0000","last_change":"2024-04-25T00:36:38.813023+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813023+0000","last_undegraded":"2024-04-25T00:36:38.813023+0000","last_fullsized":"2024-04-25T00:36:38.813023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.820114+0000","last_change":"2024-04-25T00:36:38.820114+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.820114+0000","last_undegraded":"2024-04-25T00:36:38.820114+0000","last_fullsized":"2024-04-25T00:36:38.820114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818582+0000","last_change":"2024-04-25T00:36:38.818582+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818582+0000","last_undegraded":"2024-04-25T00:36:38.818582+0000","last_fullsized":"2024-04-25T00:36:38.818582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.822677+0000","last_change":"2024-04-25T00:36:38.822677+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.822677+0000","last_undegraded":"2024-04-25T00:36:38.822677+0000","last_fullsized":"2024-04-25T00:36:38.822677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812989+0000","last_change":"2024-04-25T00:36:38.812989+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812989+0000","last_undegraded":"2024-04-25T00:36:38.812989+0000","last_fullsized":"2024-04-25T00:36:38.812989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811755+0000","last_change":"2024-04-25T00:36:38.811755+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811755+0000","last_undegraded":"2024-04-25T00:36:38.811755+0000","last_fullsized":"2024-04-25T00:36:38.811755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814147+0000","last_change":"2024-04-25T00:36:38.814147+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814147+0000","last_undegraded":"2024-04-25T00:36:38.814147+0000","last_fullsized":"2024-04-25T00:36:38.814147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811325+0000","last_change":"2024-04-25T00:36:38.811325+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811325+0000","last_undegraded":"2024-04-25T00:36:38.811325+0000","last_fullsized":"2024-04-25T00:36:38.811325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.819706+0000","last_change":"2024-04-25T00:36:38.819706+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.819706+0000","last_undegraded":"2024-04-25T00:36:38.819706+0000","last_fullsized":"2024-04-25T00:36:38.819706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845710+0000","last_change":"2024-04-25T00:36:38.842146+0000","last_active":"2024-04-25T00:36:38.845710+0000","last_peered":"2024-04-25T00:36:38.845710+0000","last_clean":"2024-04-25T00:36:38.845710+0000","last_became_active":"2024-04-25T00:36:38.841379+0000","last_became_peered":"2024-04-25T00:36:38.841379+0000","last_unstale":"2024-04-25T00:36:38.845710+0000","last_undegraded":"2024-04-25T00:36:38.845710+0000","last_fullsized":"2024-04-25T00:36:38.845710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:54.615183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.824175+0000","last_change":"2024-04-25T00:36:38.824175+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.824175+0000","last_undegraded":"2024-04-25T00:36:38.824175+0000","last_fullsized":"2024-04-25T00:36:38.824175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846934+0000","last_change":"2024-04-25T00:36:38.836018+0000","last_active":"2024-04-25T00:36:38.846934+0000","last_peered":"2024-04-25T00:36:38.846934+0000","last_clean":"2024-04-25T00:36:38.846934+0000","last_became_active":"2024-04-25T00:36:38.835209+0000","last_became_peered":"2024-04-25T00:36:38.835209+0000","last_unstale":"2024-04-25T00:36:38.846934+0000","last_undegraded":"2024-04-25T00:36:38.846934+0000","last_fullsized":"2024-04-25T00:36:38.846934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:34.231936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.821618+0000","last_change":"2024-04-25T00:36:38.821618+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.821618+0000","last_undegraded":"2024-04-25T00:36:38.821618+0000","last_fullsized":"2024-04-25T00:36:38.821618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.829415+0000","last_change":"2024-04-25T00:36:38.828383+0000","last_active":"2024-04-25T00:36:38.829415+0000","last_peered":"2024-04-25T00:36:38.829415+0000","last_clean":"2024-04-25T00:36:38.829415+0000","last_became_active":"2024-04-25T00:36:38.827081+0000","last_became_peered":"2024-04-25T00:36:38.827081+0000","last_unstale":"2024-04-25T00:36:38.829415+0000","last_undegraded":"2024-04-25T00:36:38.829415+0000","last_fullsized":"2024-04-25T00:36:38.829415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:49.597505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.820427+0000","last_change":"2024-04-25T00:36:38.820427+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.820427+0000","last_undegraded":"2024-04-25T00:36:38.820427+0000","last_fullsized":"2024-04-25T00:36:38.820427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.827222+0000","last_change":"2024-04-25T00:36:38.820892+0000","last_active":"2024-04-25T00:36:38.827222+0000","last_peered":"2024-04-25T00:36:38.827222+0000","last_clean":"2024-04-25T00:36:38.827222+0000","last_became_active":"2024-04-25T00:36:38.820603+0000","last_became_peered":"2024-04-25T00:36:38.820603+0000","last_unstale":"2024-04-25T00:36:38.827222+0000","last_undegraded":"2024-04-25T00:36:38.827222+0000","last_fullsized":"2024-04-25T00:36:38.827222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:04.183502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811231+0000","last_change":"2024-04-25T00:36:38.811231+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811231+0000","last_undegraded":"2024-04-25T00:36:38.811231+0000","last_fullsized":"2024-04-25T00:36:38.811231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.847018+0000","last_change":"2024-04-25T00:36:38.837177+0000","last_active":"2024-04-25T00:36:38.847018+0000","last_peered":"2024-04-25T00:36:38.847018+0000","last_clean":"2024-04-25T00:36:38.847018+0000","last_became_active":"2024-04-25T00:36:38.836519+0000","last_became_peered":"2024-04-25T00:36:38.836519+0000","last_unstale":"2024-04-25T00:36:38.847018+0000","last_undegraded":"2024-04-25T00:36:38.847018+0000","last_fullsized":"2024-04-25T00:36:38.847018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:48.763968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816742+0000","last_change":"2024-04-25T00:36:38.816742+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816742+0000","last_undegraded":"2024-04-25T00:36:38.816742+0000","last_fullsized":"2024-04-25T00:36:38.816742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834797+0000","last_change":"2024-04-25T00:36:38.832260+0000","last_active":"2024-04-25T00:36:38.834797+0000","last_peered":"2024-04-25T00:36:38.834797+0000","last_clean":"2024-04-25T00:36:38.834797+0000","last_became_active":"2024-04-25T00:36:38.829378+0000","last_became_peered":"2024-04-25T00:36:38.829378+0000","last_unstale":"2024-04-25T00:36:38.834797+0000","last_undegraded":"2024-04-25T00:36:38.834797+0000","last_fullsized":"2024-04-25T00:36:38.834797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:58.973792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.820408+0000","last_change":"2024-04-25T00:36:38.820408+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.820408+0000","last_undegraded":"2024-04-25T00:36:38.820408+0000","last_fullsized":"2024-04-25T00:36:38.820408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.838358+0000","last_change":"2024-04-25T00:36:38.837498+0000","last_active":"2024-04-25T00:36:38.838358+0000","last_peered":"2024-04-25T00:36:38.838358+0000","last_clean":"2024-04-25T00:36:38.838358+0000","last_became_active":"2024-04-25T00:36:38.836899+0000","last_became_peered":"2024-04-25T00:36:38.836899+0000","last_unstale":"2024-04-25T00:36:38.838358+0000","last_undegraded":"2024-04-25T00:36:38.838358+0000","last_fullsized":"2024-04-25T00:36:38.838358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:25.603144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814334+0000","last_change":"2024-04-25T00:36:38.814334+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814334+0000","last_undegraded":"2024-04-25T00:36:38.814334+0000","last_fullsized":"2024-04-25T00:36:38.814334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.838637+0000","last_change":"2024-04-25T00:36:38.837750+0000","last_active":"2024-04-25T00:36:38.838637+0000","last_peered":"2024-04-25T00:36:38.838637+0000","last_clean":"2024-04-25T00:36:38.838637+0000","last_became_active":"2024-04-25T00:36:38.836970+0000","last_became_peered":"2024-04-25T00:36:38.836970+0000","last_unstale":"2024-04-25T00:36:38.838637+0000","last_undegraded":"2024-04-25T00:36:38.838637+0000","last_fullsized":"2024-04-25T00:36:38.838637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:27.980427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816222+0000","last_change":"2024-04-25T00:36:38.816222+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816222+0000","last_undegraded":"2024-04-25T00:36:38.816222+0000","last_fullsized":"2024-04-25T00:36:38.816222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846512+0000","last_change":"2024-04-25T00:36:38.842829+0000","last_active":"2024-04-25T00:36:38.846512+0000","last_peered":"2024-04-25T00:36:38.846512+0000","last_clean":"2024-04-25T00:36:38.846512+0000","last_became_active":"2024-04-25T00:36:38.838997+0000","last_became_peered":"2024-04-25T00:36:38.838997+0000","last_unstale":"2024-04-25T00:36:38.846512+0000","last_undegraded":"2024-04-25T00:36:38.846512+0000","last_fullsized":"2024-04-25T00:36:38.846512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:39.487660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.826636+0000","last_change":"2024-04-25T00:36:38.826636+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.826636+0000","last_undegraded":"2024-04-25T00:36:38.826636+0000","last_fullsized":"2024-04-25T00:36:38.826636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845605+0000","last_change":"2024-04-25T00:36:38.840680+0000","last_active":"2024-04-25T00:36:38.845605+0000","last_peered":"2024-04-25T00:36:38.845605+0000","last_clean":"2024-04-25T00:36:38.845605+0000","last_became_active":"2024-04-25T00:36:38.839742+0000","last_became_peered":"2024-04-25T00:36:38.839742+0000","last_unstale":"2024-04-25T00:36:38.845605+0000","last_undegraded":"2024-04-25T00:36:38.845605+0000","last_fullsized":"2024-04-25T00:36:38.845605+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:48.763968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.810011+0000","last_change":"2024-04-25T00:36:38.810011+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.810011+0000","last_undegraded":"2024-04-25T00:36:38.810011+0000","last_fullsized":"2024-04-25T00:36:38.810011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846500+0000","last_change":"2024-04-25T00:36:38.842786+0000","last_active":"2024-04-25T00:36:38.846500+0000","last_peered":"2024-04-25T00:36:38.846500+0000","last_clean":"2024-04-25T00:36:38.846500+0000","last_became_active":"2024-04-25T00:36:38.842159+0000","last_became_peered":"2024-04-25T00:36:38.842159+0000","last_unstale":"2024-04-25T00:36:38.846500+0000","last_undegraded":"2024-04-25T00:36:38.846500+0000","last_fullsized":"2024-04-25T00:36:38.846500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:36.318064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.817507+0000","last_change":"2024-04-25T00:36:38.817507+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.817507+0000","last_undegraded":"2024-04-25T00:36:38.817507+0000","last_fullsized":"2024-04-25T00:36:38.817507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.844370+0000","last_change":"2024-04-25T00:36:38.837497+0000","last_active":"2024-04-25T00:36:38.844370+0000","last_peered":"2024-04-25T00:36:38.844370+0000","last_clean":"2024-04-25T00:36:38.844370+0000","last_became_active":"2024-04-25T00:36:38.836746+0000","last_became_peered":"2024-04-25T00:36:38.836746+0000","last_unstale":"2024-04-25T00:36:38.844370+0000","last_undegraded":"2024-04-25T00:36:38.844370+0000","last_fullsized":"2024-04-25T00:36:38.844370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:49.597505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.808277+0000","last_change":"2024-04-25T00:36:38.808277+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.808277+0000","last_undegraded":"2024-04-25T00:36:38.808277+0000","last_fullsized":"2024-04-25T00:36:38.808277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.833351+0000","last_change":"2024-04-25T00:36:38.828622+0000","last_active":"2024-04-25T00:36:38.833351+0000","last_peered":"2024-04-25T00:36:38.833351+0000","last_clean":"2024-04-25T00:36:38.833351+0000","last_became_active":"2024-04-25T00:36:38.828270+0000","last_became_peered":"2024-04-25T00:36:38.828270+0000","last_unstale":"2024-04-25T00:36:38.833351+0000","last_undegraded":"2024-04-25T00:36:38.833351+0000","last_fullsized":"2024-04-25T00:36:38.833351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:33.405161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814505+0000","last_change":"2024-04-25T00:36:38.814505+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814505+0000","last_undegraded":"2024-04-25T00:36:38.814505+0000","last_fullsized":"2024-04-25T00:36:38.814505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.835128+0000","last_change":"2024-04-25T00:36:38.833289+0000","last_active":"2024-04-25T00:36:38.835128+0000","last_peered":"2024-04-25T00:36:38.835128+0000","last_clean":"2024-04-25T00:36:38.835128+0000","last_became_active":"2024-04-25T00:36:38.832940+0000","last_became_peered":"2024-04-25T00:36:38.832940+0000","last_unstale":"2024-04-25T00:36:38.835128+0000","last_undegraded":"2024-04-25T00:36:38.835128+0000","last_fullsized":"2024-04-25T00:36:38.835128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:15.477777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811606+0000","last_change":"2024-04-25T00:36:38.811606+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811606+0000","last_undegraded":"2024-04-25T00:36:38.811606+0000","last_fullsized":"2024-04-25T00:36:38.811606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.836292+0000","last_change":"2024-04-25T00:36:38.834888+0000","last_active":"2024-04-25T00:36:38.836292+0000","last_peered":"2024-04-25T00:36:38.836292+0000","last_clean":"2024-04-25T00:36:38.836292+0000","last_became_active":"2024-04-25T00:36:38.833935+0000","last_became_peered":"2024-04-25T00:36:38.833935+0000","last_unstale":"2024-04-25T00:36:38.836292+0000","last_undegraded":"2024-04-25T00:36:38.836292+0000","last_fullsized":"2024-04-25T00:36:38.836292+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:04.183502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811323+0000","last_change":"2024-04-25T00:36:38.811323+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811323+0000","last_undegraded":"2024-04-25T00:36:38.811323+0000","last_fullsized":"2024-04-25T00:36:38.811323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834500+0000","last_change":"2024-04-25T00:36:38.833368+0000","last_active":"2024-04-25T00:36:38.834500+0000","last_peered":"2024-04-25T00:36:38.834500+0000","last_clean":"2024-04-25T00:36:38.834500+0000","last_became_active":"2024-04-25T00:36:38.832870+0000","last_became_peered":"2024-04-25T00:36:38.832870+0000","last_unstale":"2024-04-25T00:36:38.834500+0000","last_undegraded":"2024-04-25T00:36:38.834500+0000","last_fullsized":"2024-04-25T00:36:38.834500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:05.341588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813368+0000","last_change":"2024-04-25T00:36:38.813368+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813368+0000","last_undegraded":"2024-04-25T00:36:38.813368+0000","last_fullsized":"2024-04-25T00:36:38.813368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.847195+0000","last_change":"2024-04-25T00:36:38.845296+0000","last_active":"2024-04-25T00:36:38.847195+0000","last_peered":"2024-04-25T00:36:38.847195+0000","last_clean":"2024-04-25T00:36:38.847195+0000","last_became_active":"2024-04-25T00:36:38.844447+0000","last_became_peered":"2024-04-25T00:36:38.844447+0000","last_unstale":"2024-04-25T00:36:38.847195+0000","last_undegraded":"2024-04-25T00:36:38.847195+0000","last_fullsized":"2024-04-25T00:36:38.847195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:38.147666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816628+0000","last_change":"2024-04-25T00:36:38.816628+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816628+0000","last_undegraded":"2024-04-25T00:36:38.816628+0000","last_fullsized":"2024-04-25T00:36:38.816628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.837861+0000","last_change":"2024-04-25T00:36:38.836218+0000","last_active":"2024-04-25T00:36:38.837861+0000","last_peered":"2024-04-25T00:36:38.837861+0000","last_clean":"2024-04-25T00:36:38.837861+0000","last_became_active":"2024-04-25T00:36:38.835946+0000","last_became_peered":"2024-04-25T00:36:38.835946+0000","last_unstale":"2024-04-25T00:36:38.837861+0000","last_undegraded":"2024-04-25T00:36:38.837861+0000","last_fullsized":"2024-04-25T00:36:38.837861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:48.430422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816425+0000","last_change":"2024-04-25T00:36:38.816425+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816425+0000","last_undegraded":"2024-04-25T00:36:38.816425+0000","last_fullsized":"2024-04-25T00:36:38.816425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.838313+0000","last_change":"2024-04-25T00:36:38.836980+0000","last_active":"2024-04-25T00:36:38.838313+0000","last_peered":"2024-04-25T00:36:38.838313+0000","last_clean":"2024-04-25T00:36:38.838313+0000","last_became_active":"2024-04-25T00:36:38.836722+0000","last_became_peered":"2024-04-25T00:36:38.836722+0000","last_unstale":"2024-04-25T00:36:38.838313+0000","last_undegraded":"2024-04-25T00:36:38.838313+0000","last_fullsized":"2024-04-25T00:36:38.838313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:29.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813225+0000","last_change":"2024-04-25T00:36:38.813225+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813225+0000","last_undegraded":"2024-04-25T00:36:38.813225+0000","last_fullsized":"2024-04-25T00:36:38.813225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.829873+0000","last_change":"2024-04-25T00:36:38.827338+0000","last_active":"2024-04-25T00:36:38.829873+0000","last_peered":"2024-04-25T00:36:38.829873+0000","last_clean":"2024-04-25T00:36:38.829873+0000","last_became_active":"2024-04-25T00:36:38.826884+0000","last_became_peered":"2024-04-25T00:36:38.826884+0000","last_unstale":"2024-04-25T00:36:38.829873+0000","last_undegraded":"2024-04-25T00:36:38.829873+0000","last_fullsized":"2024-04-25T00:36:38.829873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:10.228966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.828168+0000","last_change":"2024-04-25T00:36:38.828168+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.828168+0000","last_undegraded":"2024-04-25T00:36:38.828168+0000","last_fullsized":"2024-04-25T00:36:38.828168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834354+0000","last_change":"2024-04-25T00:36:38.829621+0000","last_active":"2024-04-25T00:36:38.834354+0000","last_peered":"2024-04-25T00:36:38.834354+0000","last_clean":"2024-04-25T00:36:38.834354+0000","last_became_active":"2024-04-25T00:36:38.827960+0000","last_became_peered":"2024-04-25T00:36:38.827960+0000","last_unstale":"2024-04-25T00:36:38.834354+0000","last_undegraded":"2024-04-25T00:36:38.834354+0000","last_fullsized":"2024-04-25T00:36:38.834354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:56.562023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.810472+0000","last_change":"2024-04-25T00:36:38.810472+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.810472+0000","last_undegraded":"2024-04-25T00:36:38.810472+0000","last_fullsized":"2024-04-25T00:36:38.810472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.827971+0000","last_change":"2024-04-25T00:36:38.823571+0000","last_active":"2024-04-25T00:36:38.827971+0000","last_peered":"2024-04-25T00:36:38.827971+0000","last_clean":"2024-04-25T00:36:38.827971+0000","last_became_active":"2024-04-25T00:36:38.823275+0000","last_became_peered":"2024-04-25T00:36:38.823275+0000","last_unstale":"2024-04-25T00:36:38.827971+0000","last_undegraded":"2024-04-25T00:36:38.827971+0000","last_fullsized":"2024-04-25T00:36:38.827971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:51.207453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.828686+0000","last_change":"2024-04-25T00:36:38.828686+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.828686+0000","last_undegraded":"2024-04-25T00:36:38.828686+0000","last_fullsized":"2024-04-25T00:36:38.828686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.829729+0000","last_change":"2024-04-25T00:36:38.827233+0000","last_active":"2024-04-25T00:36:38.829729+0000","last_peered":"2024-04-25T00:36:38.829729+0000","last_clean":"2024-04-25T00:36:38.829729+0000","last_became_active":"2024-04-25T00:36:38.826902+0000","last_became_peered":"2024-04-25T00:36:38.826902+0000","last_unstale":"2024-04-25T00:36:38.829729+0000","last_undegraded":"2024-04-25T00:36:38.829729+0000","last_fullsized":"2024-04-25T00:36:38.829729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:20.855968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818674+0000","last_change":"2024-04-25T00:36:38.818674+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818674+0000","last_undegraded":"2024-04-25T00:36:38.818674+0000","last_fullsized":"2024-04-25T00:36:38.818674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.839320+0000","last_change":"2024-04-25T00:36:38.838509+0000","last_active":"2024-04-25T00:36:38.839320+0000","last_peered":"2024-04-25T00:36:38.839320+0000","last_clean":"2024-04-25T00:36:38.839320+0000","last_became_active":"2024-04-25T00:36:38.837928+0000","last_became_peered":"2024-04-25T00:36:38.837928+0000","last_unstale":"2024-04-25T00:36:38.839320+0000","last_undegraded":"2024-04-25T00:36:38.839320+0000","last_fullsized":"2024-04-25T00:36:38.839320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:06.637343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812126+0000","last_change":"2024-04-25T00:36:38.812126+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812126+0000","last_undegraded":"2024-04-25T00:36:38.812126+0000","last_fullsized":"2024-04-25T00:36:38.812126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.833501+0000","last_change":"2024-04-25T00:36:38.828691+0000","last_active":"2024-04-25T00:36:38.833501+0000","last_peered":"2024-04-25T00:36:38.833501+0000","last_clean":"2024-04-25T00:36:38.833501+0000","last_became_active":"2024-04-25T00:36:38.828380+0000","last_became_peered":"2024-04-25T00:36:38.828380+0000","last_unstale":"2024-04-25T00:36:38.833501+0000","last_undegraded":"2024-04-25T00:36:38.833501+0000","last_fullsized":"2024-04-25T00:36:38.833501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:22.410800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816547+0000","last_change":"2024-04-25T00:36:38.816547+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816547+0000","last_undegraded":"2024-04-25T00:36:38.816547+0000","last_fullsized":"2024-04-25T00:36:38.816547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845767+0000","last_change":"2024-04-25T00:36:38.841467+0000","last_active":"2024-04-25T00:36:38.845767+0000","last_peered":"2024-04-25T00:36:38.845767+0000","last_clean":"2024-04-25T00:36:38.845767+0000","last_became_active":"2024-04-25T00:36:38.840182+0000","last_became_peered":"2024-04-25T00:36:38.840182+0000","last_unstale":"2024-04-25T00:36:38.845767+0000","last_undegraded":"2024-04-25T00:36:38.845767+0000","last_fullsized":"2024-04-25T00:36:38.845767+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:53.012076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815364+0000","last_change":"2024-04-25T00:36:38.815364+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815364+0000","last_undegraded":"2024-04-25T00:36:38.815364+0000","last_fullsized":"2024-04-25T00:36:38.815364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845638+0000","last_change":"2024-04-25T00:36:38.841047+0000","last_active":"2024-04-25T00:36:38.845638+0000","last_peered":"2024-04-25T00:36:38.845638+0000","last_clean":"2024-04-25T00:36:38.845638+0000","last_became_active":"2024-04-25T00:36:38.839481+0000","last_became_peered":"2024-04-25T00:36:38.839481+0000","last_unstale":"2024-04-25T00:36:38.845638+0000","last_undegraded":"2024-04-25T00:36:38.845638+0000","last_fullsized":"2024-04-25T00:36:38.845638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:11.985160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825951+0000","last_change":"2024-04-25T00:36:38.825951+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825951+0000","last_undegraded":"2024-04-25T00:36:38.825951+0000","last_fullsized":"2024-04-25T00:36:38.825951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.826182+0000","last_change":"2024-04-25T00:36:38.819664+0000","last_active":"2024-04-25T00:36:38.826182+0000","last_peered":"2024-04-25T00:36:38.826182+0000","last_clean":"2024-04-25T00:36:38.826182+0000","last_became_active":"2024-04-25T00:36:38.819378+0000","last_became_peered":"2024-04-25T00:36:38.819378+0000","last_unstale":"2024-04-25T00:36:38.826182+0000","last_undegraded":"2024-04-25T00:36:38.826182+0000","last_fullsized":"2024-04-25T00:36:38.826182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:52.487791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.807503+0000","last_change":"2024-04-25T00:36:38.807503+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.807503+0000","last_undegraded":"2024-04-25T00:36:38.807503+0000","last_fullsized":"2024-04-25T00:36:38.807503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834734+0000","last_change":"2024-04-25T00:36:38.830646+0000","last_active":"2024-04-25T00:36:38.834734+0000","last_peered":"2024-04-25T00:36:38.834734+0000","last_clean":"2024-04-25T00:36:38.834734+0000","last_became_active":"2024-04-25T00:36:38.829932+0000","last_became_peered":"2024-04-25T00:36:38.829932+0000","last_unstale":"2024-04-25T00:36:38.834734+0000","last_undegraded":"2024-04-25T00:36:38.834734+0000","last_fullsized":"2024-04-25T00:36:38.834734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:57.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818106+0000","last_change":"2024-04-25T00:36:38.818106+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818106+0000","last_undegraded":"2024-04-25T00:36:38.818106+0000","last_fullsized":"2024-04-25T00:36:38.818106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846554+0000","last_change":"2024-04-25T00:36:38.840716+0000","last_active":"2024-04-25T00:36:38.846554+0000","last_peered":"2024-04-25T00:36:38.846554+0000","last_clean":"2024-04-25T00:36:38.846554+0000","last_became_active":"2024-04-25T00:36:38.837997+0000","last_became_peered":"2024-04-25T00:36:38.837997+0000","last_unstale":"2024-04-25T00:36:38.846554+0000","last_undegraded":"2024-04-25T00:36:38.846554+0000","last_fullsized":"2024-04-25T00:36:38.846554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:54.469425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813359+0000","last_change":"2024-04-25T00:36:38.813359+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813359+0000","last_undegraded":"2024-04-25T00:36:38.813359+0000","last_fullsized":"2024-04-25T00:36:38.813359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846920+0000","last_change":"2024-04-25T00:36:38.834405+0000","last_active":"2024-04-25T00:36:38.846920+0000","last_peered":"2024-04-25T00:36:38.846920+0000","last_clean":"2024-04-25T00:36:38.846920+0000","last_became_active":"2024-04-25T00:36:38.833769+0000","last_became_peered":"2024-04-25T00:36:38.833769+0000","last_unstale":"2024-04-25T00:36:38.846920+0000","last_undegraded":"2024-04-25T00:36:38.846920+0000","last_fullsized":"2024-04-25T00:36:38.846920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:58.559610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814258+0000","last_change":"2024-04-25T00:36:38.814258+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814258+0000","last_undegraded":"2024-04-25T00:36:38.814258+0000","last_fullsized":"2024-04-25T00:36:38.814258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.842399+0000","last_change":"2024-04-25T00:36:38.835037+0000","last_active":"2024-04-25T00:36:38.842399+0000","last_peered":"2024-04-25T00:36:38.842399+0000","last_clean":"2024-04-25T00:36:38.842399+0000","last_became_active":"2024-04-25T00:36:38.833930+0000","last_became_peered":"2024-04-25T00:36:38.833930+0000","last_unstale":"2024-04-25T00:36:38.842399+0000","last_undegraded":"2024-04-25T00:36:38.842399+0000","last_fullsized":"2024-04-25T00:36:38.842399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:48.676020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":327,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.808362+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:36:38.808362+0000","last_peered":"2024-04-25T00:36:38.808362+0000","last_clean":"2024-04-25T00:36:38.808362+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:36:38.808362+0000","last_undegraded":"2024-04-25T00:36:38.808362+0000","last_fullsized":"2024-04-25T00:36:38.808362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36908,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334932,"statfs":{"total":96636764160,"available":96598970368,"internally_reserved":0,"allocated":1601536,"data_stored":1021755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":431387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":425112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1007616,"data_stored":431387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":995328,"data_stored":425112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":995328,"data_stored":425112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1007616,"data_stored":431387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36892,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334948,"statfs":{"total":96636764160,"available":96598986752,"internally_reserved":0,"allocated":1589248,"data_stored":1015480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:36:44.229 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:36:44.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.388+0000 7f1757023640 1 Processor -- start 2024-04-25T00:36:44.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.388+0000 7f1757023640 1 -- start start 2024-04-25T00:36:44.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f1757023640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17500b3140 0x7f175006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f1757023640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17500b3c30 0x7f17500ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f1757023640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17500b41e0 con 0x7f17500b4b10 2024-04-25T00:36:44.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f1757023640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f175006d960 con 0x7f17500b3c30 2024-04-25T00:36:44.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f1757023640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f175006dc40 con 0x7f17500b3140 2024-04-25T00:36:44.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f17555c0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17500b3c30 0x7f17500ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f17555c0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17500b3c30 0x7f17500ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37024/0 (socket says 172.21.15.29:37024) 2024-04-25T00:36:44.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f17555c0640 1 -- 172.21.15.29:0/2161003794 learned_addr learned my addr 172.21.15.29:0/2161003794 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:44.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f1754dbf640 1 --2- 172.21.15.29:0/2161003794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17500b3140 0x7f175006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f17555c0640 1 -- 172.21.15.29:0/2161003794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17500b3140 msgr2=0x7f175006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:44.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f17555c0640 1 --2- 172.21.15.29:0/2161003794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17500b3140 0x7f175006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f17555c0640 1 -- 172.21.15.29:0/2161003794 >> v1:172.21.15.157:6789/0 conn(0x7f17500b4b10 legacy=0x7f17500b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:44.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f17555c0640 1 -- 172.21.15.29:0/2161003794 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175006df20 con 0x7f17500b3c30 2024-04-25T00:36:44.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f1754dbf640 1 --2- 172.21.15.29:0/2161003794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17500b3140 0x7f175006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:44.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f17555c0640 1 --2- 172.21.15.29:0/2161003794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17500b3c30 0x7f17500ce240 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1748009e50 tx=0x7f174801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fa4474b5620f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:44.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f174f7fe640 1 -- 172.21.15.29:0/2161003794 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17480a5670 con 0x7f17500b3c30 2024-04-25T00:36:44.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f174f7fe640 1 -- 172.21.15.29:0/2161003794 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17480a5810 con 0x7f17500b3c30 2024-04-25T00:36:44.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.392+0000 7f174f7fe640 1 -- 172.21.15.29:0/2161003794 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17480a5be0 con 0x7f17500b3c30 2024-04-25T00:36:44.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 -- 172.21.15.29:0/2161003794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17500b3c30 msgr2=0x7f17500ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 --2- 172.21.15.29:0/2161003794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17500b3c30 0x7f17500ce240 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1748009e50 tx=0x7f174801c9f0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 -- 172.21.15.29:0/2161003794 shutdown_connections 2024-04-25T00:36:44.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 --2- 172.21.15.29:0/2161003794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17500b3c30 0x7f17500ce240 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 --2- 172.21.15.29:0/2161003794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17500b3140 0x7f175006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 -- 172.21.15.29:0/2161003794 >> 172.21.15.29:0/2161003794 conn(0x7f17500b1870 msgr2=0x7f17500b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:44.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 -- 172.21.15.29:0/2161003794 shutdown_connections 2024-04-25T00:36:44.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 -- 172.21.15.29:0/2161003794 wait complete. 2024-04-25T00:36:44.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 Processor -- start 2024-04-25T00:36:44.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 -- start start 2024-04-25T00:36:44.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17500b3140 0x7f1750132fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17500b3c30 0x7f175015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17500b4b10 0x7f175015a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1750168d80 con 0x7f17500b3c30 2024-04-25T00:36:44.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1750168c00 con 0x7f17500b3140 2024-04-25T00:36:44.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1757023640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1750168f00 con 0x7f17500b4b10 2024-04-25T00:36:44.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f174ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17500b3c30 0x7f175015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f174ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17500b3c30 0x7f175015a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37028/0 (socket says 172.21.15.29:37028) 2024-04-25T00:36:44.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f174ffff640 1 -- 172.21.15.29:0/3690237337 learned_addr learned my addr 172.21.15.29:0/3690237337 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:44.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f1754dbf640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17500b3140 0x7f1750132fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f17555c0640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17500b4b10 0x7f175015a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f174ffff640 1 -- 172.21.15.29:0/3690237337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17500b4b10 msgr2=0x7f175015a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f174ffff640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17500b4b10 0x7f175015a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f174ffff640 1 -- 172.21.15.29:0/3690237337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17500b3140 msgr2=0x7f1750132fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f174ffff640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17500b3140 0x7f1750132fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f174ffff640 1 -- 172.21.15.29:0/3690237337 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175015b340 con 0x7f17500b3c30 2024-04-25T00:36:44.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f17555c0640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17500b4b10 0x7f175015a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:44.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f174ffff640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17500b3c30 0x7f175015a310 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f173800c450 tx=0x7f173800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:44.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.396+0000 7f174d7fa640 1 -- 172.21.15.29:0/3690237337 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738012830 con 0x7f17500b3c30 2024-04-25T00:36:44.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.400+0000 7f174d7fa640 1 -- 172.21.15.29:0/3690237337 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1738017070 con 0x7f17500b3c30 2024-04-25T00:36:44.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.400+0000 7f1757023640 1 -- 172.21.15.29:0/3690237337 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1750138bc0 con 0x7f17500b3c30 2024-04-25T00:36:44.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.400+0000 7f174d7fa640 1 -- 172.21.15.29:0/3690237337 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738012c30 con 0x7f17500b3c30 2024-04-25T00:36:44.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.400+0000 7f1757023640 1 -- 172.21.15.29:0/3690237337 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1750139180 con 0x7f17500b3c30 2024-04-25T00:36:44.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.400+0000 7f1757023640 1 -- 172.21.15.29:0/3690237337 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1750069270 con 0x7f17500b3c30 2024-04-25T00:36:44.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.400+0000 7f174d7fa640 1 -- 172.21.15.29:0/3690237337 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f173801c460 con 0x7f17500b3c30 2024-04-25T00:36:44.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.400+0000 7f174d7fa640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17300679e0 0x7f1730069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.400+0000 7f174d7fa640 1 -- 172.21.15.29:0/3690237337 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f1738085cc0 con 0x7f17500b3c30 2024-04-25T00:36:44.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.400+0000 7f1754dbf640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17300679e0 0x7f1730069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.400+0000 7f1754dbf640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17300679e0 0x7f1730069ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1740006fd0 tx=0x7f1740008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:44.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.404+0000 7f174d7fa640 1 -- 172.21.15.29:0/3690237337 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1738056670 con 0x7f17500b3c30 2024-04-25T00:36:44.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.560+0000 7f1757023640 1 -- 172.21.15.29:0/3690237337 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f17500753e0 con 0x7f17300679e0 2024-04-25T00:36:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.580+0000 7f174d7fa640 1 -- 172.21.15.29:0/3690237337 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246756 (secure 0 0 0) 0x7f17500753e0 con 0x7f17300679e0 2024-04-25T00:36:44.583 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:44.585 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:36:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.584+0000 7f1757023640 1 -- 172.21.15.29:0/3690237337 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17300679e0 msgr2=0x7f1730069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.584+0000 7f1757023640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17300679e0 0x7f1730069ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1740006fd0 tx=0x7f1740008040 comp rx=0 tx=0).stop 2024-04-25T00:36:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.584+0000 7f1757023640 1 -- 172.21.15.29:0/3690237337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17500b3c30 msgr2=0x7f175015a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.584+0000 7f1757023640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17500b3c30 0x7f175015a310 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f173800c450 tx=0x7f173800c910 comp rx=0 tx=0).stop 2024-04-25T00:36:44.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.584+0000 7f1757023640 1 -- 172.21.15.29:0/3690237337 shutdown_connections 2024-04-25T00:36:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.584+0000 7f1757023640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17300679e0 0x7f1730069ea0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.584+0000 7f1757023640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17500b4b10 0x7f175015a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.584+0000 7f1757023640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17500b3c30 0x7f175015a310 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.584+0000 7f1757023640 1 --2- 172.21.15.29:0/3690237337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17500b3140 0x7f1750132fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.584+0000 7f1757023640 1 -- 172.21.15.29:0/3690237337 >> 172.21.15.29:0/3690237337 conn(0x7f17500b1870 msgr2=0x7f1750143810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.584+0000 7f1757023640 1 -- 172.21.15.29:0/3690237337 shutdown_connections 2024-04-25T00:36:44.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.584+0000 7f1757023640 1 -- 172.21.15.29:0/3690237337 wait complete. 2024-04-25T00:36:44.601 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-25T00:36:43.312352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283400,"kb_used_data":8984,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754691320,"statfs":{"total":773094113280,"available":772803911680,"internally_reserved":0,"allocated":9199616,"data_stored":4606732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.519429"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825116+0000","last_change":"2024-04-25T00:36:38.825116+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825116+0000","last_undegraded":"2024-04-25T00:36:38.825116+0000","last_fullsized":"2024-04-25T00:36:38.825116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816578+0000","last_change":"2024-04-25T00:36:38.816578+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816578+0000","last_undegraded":"2024-04-25T00:36:38.816578+0000","last_fullsized":"2024-04-25T00:36:38.816578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.821596+0000","last_change":"2024-04-25T00:36:38.821596+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.821596+0000","last_undegraded":"2024-04-25T00:36:38.821596+0000","last_fullsized":"2024-04-25T00:36:38.821596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.829028+0000","last_change":"2024-04-25T00:36:38.829028+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.829028+0000","last_undegraded":"2024-04-25T00:36:38.829028+0000","last_fullsized":"2024-04-25T00:36:38.829028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.817867+0000","last_change":"2024-04-25T00:36:38.817867+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.817867+0000","last_undegraded":"2024-04-25T00:36:38.817867+0000","last_fullsized":"2024-04-25T00:36:38.817867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.827070+0000","last_change":"2024-04-25T00:36:38.827070+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.827070+0000","last_undegraded":"2024-04-25T00:36:38.827070+0000","last_fullsized":"2024-04-25T00:36:38.827070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825982+0000","last_change":"2024-04-25T00:36:38.825982+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825982+0000","last_undegraded":"2024-04-25T00:36:38.825982+0000","last_fullsized":"2024-04-25T00:36:38.825982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.824317+0000","last_change":"2024-04-25T00:36:38.824317+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.824317+0000","last_undegraded":"2024-04-25T00:36:38.824317+0000","last_fullsized":"2024-04-25T00:36:38.824317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.827607+0000","last_change":"2024-04-25T00:36:38.827607+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.827607+0000","last_undegraded":"2024-04-25T00:36:38.827607+0000","last_fullsized":"2024-04-25T00:36:38.827607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813471+0000","last_change":"2024-04-25T00:36:38.813471+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813471+0000","last_undegraded":"2024-04-25T00:36:38.813471+0000","last_fullsized":"2024-04-25T00:36:38.813471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814021+0000","last_change":"2024-04-25T00:36:38.814021+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814021+0000","last_undegraded":"2024-04-25T00:36:38.814021+0000","last_fullsized":"2024-04-25T00:36:38.814021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815493+0000","last_change":"2024-04-25T00:36:38.815493+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815493+0000","last_undegraded":"2024-04-25T00:36:38.815493+0000","last_fullsized":"2024-04-25T00:36:38.815493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.822046+0000","last_change":"2024-04-25T00:36:38.822046+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.822046+0000","last_undegraded":"2024-04-25T00:36:38.822046+0000","last_fullsized":"2024-04-25T00:36:38.822046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825738+0000","last_change":"2024-04-25T00:36:38.825738+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825738+0000","last_undegraded":"2024-04-25T00:36:38.825738+0000","last_fullsized":"2024-04-25T00:36:38.825738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815105+0000","last_change":"2024-04-25T00:36:38.815105+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815105+0000","last_undegraded":"2024-04-25T00:36:38.815105+0000","last_fullsized":"2024-04-25T00:36:38.815105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815145+0000","last_change":"2024-04-25T00:36:38.815145+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815145+0000","last_undegraded":"2024-04-25T00:36:38.815145+0000","last_fullsized":"2024-04-25T00:36:38.815145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.819692+0000","last_change":"2024-04-25T00:36:38.819692+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.819692+0000","last_undegraded":"2024-04-25T00:36:38.819692+0000","last_fullsized":"2024-04-25T00:36:38.819692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818362+0000","last_change":"2024-04-25T00:36:38.818362+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818362+0000","last_undegraded":"2024-04-25T00:36:38.818362+0000","last_fullsized":"2024-04-25T00:36:38.818362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.808911+0000","last_change":"2024-04-25T00:36:38.808911+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.808911+0000","last_undegraded":"2024-04-25T00:36:38.808911+0000","last_fullsized":"2024-04-25T00:36:38.808911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818455+0000","last_change":"2024-04-25T00:36:38.818455+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818455+0000","last_undegraded":"2024-04-25T00:36:38.818455+0000","last_fullsized":"2024-04-25T00:36:38.818455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818675+0000","last_change":"2024-04-25T00:36:38.818675+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818675+0000","last_undegraded":"2024-04-25T00:36:38.818675+0000","last_fullsized":"2024-04-25T00:36:38.818675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812757+0000","last_change":"2024-04-25T00:36:38.812757+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812757+0000","last_undegraded":"2024-04-25T00:36:38.812757+0000","last_fullsized":"2024-04-25T00:36:38.812757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812237+0000","last_change":"2024-04-25T00:36:38.812237+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812237+0000","last_undegraded":"2024-04-25T00:36:38.812237+0000","last_fullsized":"2024-04-25T00:36:38.812237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818137+0000","last_change":"2024-04-25T00:36:38.818137+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818137+0000","last_undegraded":"2024-04-25T00:36:38.818137+0000","last_fullsized":"2024-04-25T00:36:38.818137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813023+0000","last_change":"2024-04-25T00:36:38.813023+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813023+0000","last_undegraded":"2024-04-25T00:36:38.813023+0000","last_fullsized":"2024-04-25T00:36:38.813023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.820114+0000","last_change":"2024-04-25T00:36:38.820114+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.820114+0000","last_undegraded":"2024-04-25T00:36:38.820114+0000","last_fullsized":"2024-04-25T00:36:38.820114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818582+0000","last_change":"2024-04-25T00:36:38.818582+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818582+0000","last_undegraded":"2024-04-25T00:36:38.818582+0000","last_fullsized":"2024-04-25T00:36:38.818582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.822677+0000","last_change":"2024-04-25T00:36:38.822677+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.822677+0000","last_undegraded":"2024-04-25T00:36:38.822677+0000","last_fullsized":"2024-04-25T00:36:38.822677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812989+0000","last_change":"2024-04-25T00:36:38.812989+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812989+0000","last_undegraded":"2024-04-25T00:36:38.812989+0000","last_fullsized":"2024-04-25T00:36:38.812989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811755+0000","last_change":"2024-04-25T00:36:38.811755+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811755+0000","last_undegraded":"2024-04-25T00:36:38.811755+0000","last_fullsized":"2024-04-25T00:36:38.811755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814147+0000","last_change":"2024-04-25T00:36:38.814147+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814147+0000","last_undegraded":"2024-04-25T00:36:38.814147+0000","last_fullsized":"2024-04-25T00:36:38.814147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811325+0000","last_change":"2024-04-25T00:36:38.811325+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811325+0000","last_undegraded":"2024-04-25T00:36:38.811325+0000","last_fullsized":"2024-04-25T00:36:38.811325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.819706+0000","last_change":"2024-04-25T00:36:38.819706+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.819706+0000","last_undegraded":"2024-04-25T00:36:38.819706+0000","last_fullsized":"2024-04-25T00:36:38.819706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845710+0000","last_change":"2024-04-25T00:36:38.842146+0000","last_active":"2024-04-25T00:36:38.845710+0000","last_peered":"2024-04-25T00:36:38.845710+0000","last_clean":"2024-04-25T00:36:38.845710+0000","last_became_active":"2024-04-25T00:36:38.841379+0000","last_became_peered":"2024-04-25T00:36:38.841379+0000","last_unstale":"2024-04-25T00:36:38.845710+0000","last_undegraded":"2024-04-25T00:36:38.845710+0000","last_fullsized":"2024-04-25T00:36:38.845710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:54.615183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.824175+0000","last_change":"2024-04-25T00:36:38.824175+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.824175+0000","last_undegraded":"2024-04-25T00:36:38.824175+0000","last_fullsized":"2024-04-25T00:36:38.824175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846934+0000","last_change":"2024-04-25T00:36:38.836018+0000","last_active":"2024-04-25T00:36:38.846934+0000","last_peered":"2024-04-25T00:36:38.846934+0000","last_clean":"2024-04-25T00:36:38.846934+0000","last_became_active":"2024-04-25T00:36:38.835209+0000","last_became_peered":"2024-04-25T00:36:38.835209+0000","last_unstale":"2024-04-25T00:36:38.846934+0000","last_undegraded":"2024-04-25T00:36:38.846934+0000","last_fullsized":"2024-04-25T00:36:38.846934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:34.231936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.821618+0000","last_change":"2024-04-25T00:36:38.821618+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.821618+0000","last_undegraded":"2024-04-25T00:36:38.821618+0000","last_fullsized":"2024-04-25T00:36:38.821618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.829415+0000","last_change":"2024-04-25T00:36:38.828383+0000","last_active":"2024-04-25T00:36:38.829415+0000","last_peered":"2024-04-25T00:36:38.829415+0000","last_clean":"2024-04-25T00:36:38.829415+0000","last_became_active":"2024-04-25T00:36:38.827081+0000","last_became_peered":"2024-04-25T00:36:38.827081+0000","last_unstale":"2024-04-25T00:36:38.829415+0000","last_undegraded":"2024-04-25T00:36:38.829415+0000","last_fullsized":"2024-04-25T00:36:38.829415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:49.597505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.820427+0000","last_change":"2024-04-25T00:36:38.820427+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.820427+0000","last_undegraded":"2024-04-25T00:36:38.820427+0000","last_fullsized":"2024-04-25T00:36:38.820427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.827222+0000","last_change":"2024-04-25T00:36:38.820892+0000","last_active":"2024-04-25T00:36:38.827222+0000","last_peered":"2024-04-25T00:36:38.827222+0000","last_clean":"2024-04-25T00:36:38.827222+0000","last_became_active":"2024-04-25T00:36:38.820603+0000","last_became_peered":"2024-04-25T00:36:38.820603+0000","last_unstale":"2024-04-25T00:36:38.827222+0000","last_undegraded":"2024-04-25T00:36:38.827222+0000","last_fullsized":"2024-04-25T00:36:38.827222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:04.183502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811231+0000","last_change":"2024-04-25T00:36:38.811231+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811231+0000","last_undegraded":"2024-04-25T00:36:38.811231+0000","last_fullsized":"2024-04-25T00:36:38.811231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.847018+0000","last_change":"2024-04-25T00:36:38.837177+0000","last_active":"2024-04-25T00:36:38.847018+0000","last_peered":"2024-04-25T00:36:38.847018+0000","last_clean":"2024-04-25T00:36:38.847018+0000","last_became_active":"2024-04-25T00:36:38.836519+0000","last_became_peered":"2024-04-25T00:36:38.836519+0000","last_unstale":"2024-04-25T00:36:38.847018+0000","last_undegraded":"2024-04-25T00:36:38.847018+0000","last_fullsized":"2024-04-25T00:36:38.847018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:48.763968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816742+0000","last_change":"2024-04-25T00:36:38.816742+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816742+0000","last_undegraded":"2024-04-25T00:36:38.816742+0000","last_fullsized":"2024-04-25T00:36:38.816742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834797+0000","last_change":"2024-04-25T00:36:38.832260+0000","last_active":"2024-04-25T00:36:38.834797+0000","last_peered":"2024-04-25T00:36:38.834797+0000","last_clean":"2024-04-25T00:36:38.834797+0000","last_became_active":"2024-04-25T00:36:38.829378+0000","last_became_peered":"2024-04-25T00:36:38.829378+0000","last_unstale":"2024-04-25T00:36:38.834797+0000","last_undegraded":"2024-04-25T00:36:38.834797+0000","last_fullsized":"2024-04-25T00:36:38.834797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:58.973792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.820408+0000","last_change":"2024-04-25T00:36:38.820408+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.820408+0000","last_undegraded":"2024-04-25T00:36:38.820408+0000","last_fullsized":"2024-04-25T00:36:38.820408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.838358+0000","last_change":"2024-04-25T00:36:38.837498+0000","last_active":"2024-04-25T00:36:38.838358+0000","last_peered":"2024-04-25T00:36:38.838358+0000","last_clean":"2024-04-25T00:36:38.838358+0000","last_became_active":"2024-04-25T00:36:38.836899+0000","last_became_peered":"2024-04-25T00:36:38.836899+0000","last_unstale":"2024-04-25T00:36:38.838358+0000","last_undegraded":"2024-04-25T00:36:38.838358+0000","last_fullsized":"2024-04-25T00:36:38.838358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:25.603144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814334+0000","last_change":"2024-04-25T00:36:38.814334+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814334+0000","last_undegraded":"2024-04-25T00:36:38.814334+0000","last_fullsized":"2024-04-25T00:36:38.814334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.838637+0000","last_change":"2024-04-25T00:36:38.837750+0000","last_active":"2024-04-25T00:36:38.838637+0000","last_peered":"2024-04-25T00:36:38.838637+0000","last_clean":"2024-04-25T00:36:38.838637+0000","last_became_active":"2024-04-25T00:36:38.836970+0000","last_became_peered":"2024-04-25T00:36:38.836970+0000","last_unstale":"2024-04-25T00:36:38.838637+0000","last_undegraded":"2024-04-25T00:36:38.838637+0000","last_fullsized":"2024-04-25T00:36:38.838637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:27.980427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816222+0000","last_change":"2024-04-25T00:36:38.816222+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816222+0000","last_undegraded":"2024-04-25T00:36:38.816222+0000","last_fullsized":"2024-04-25T00:36:38.816222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846512+0000","last_change":"2024-04-25T00:36:38.842829+0000","last_active":"2024-04-25T00:36:38.846512+0000","last_peered":"2024-04-25T00:36:38.846512+0000","last_clean":"2024-04-25T00:36:38.846512+0000","last_became_active":"2024-04-25T00:36:38.838997+0000","last_became_peered":"2024-04-25T00:36:38.838997+0000","last_unstale":"2024-04-25T00:36:38.846512+0000","last_undegraded":"2024-04-25T00:36:38.846512+0000","last_fullsized":"2024-04-25T00:36:38.846512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:39.487660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.826636+0000","last_change":"2024-04-25T00:36:38.826636+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.826636+0000","last_undegraded":"2024-04-25T00:36:38.826636+0000","last_fullsized":"2024-04-25T00:36:38.826636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845605+0000","last_change":"2024-04-25T00:36:38.840680+0000","last_active":"2024-04-25T00:36:38.845605+0000","last_peered":"2024-04-25T00:36:38.845605+0000","last_clean":"2024-04-25T00:36:38.845605+0000","last_became_active":"2024-04-25T00:36:38.839742+0000","last_became_peered":"2024-04-25T00:36:38.839742+0000","last_unstale":"2024-04-25T00:36:38.845605+0000","last_undegraded":"2024-04-25T00:36:38.845605+0000","last_fullsized":"2024-04-25T00:36:38.845605+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:48.763968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.810011+0000","last_change":"2024-04-25T00:36:38.810011+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.810011+0000","last_undegraded":"2024-04-25T00:36:38.810011+0000","last_fullsized":"2024-04-25T00:36:38.810011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846500+0000","last_change":"2024-04-25T00:36:38.842786+0000","last_active":"2024-04-25T00:36:38.846500+0000","last_peered":"2024-04-25T00:36:38.846500+0000","last_clean":"2024-04-25T00:36:38.846500+0000","last_became_active":"2024-04-25T00:36:38.842159+0000","last_became_peered":"2024-04-25T00:36:38.842159+0000","last_unstale":"2024-04-25T00:36:38.846500+0000","last_undegraded":"2024-04-25T00:36:38.846500+0000","last_fullsized":"2024-04-25T00:36:38.846500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:36.318064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.817507+0000","last_change":"2024-04-25T00:36:38.817507+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.817507+0000","last_undegraded":"2024-04-25T00:36:38.817507+0000","last_fullsized":"2024-04-25T00:36:38.817507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.844370+0000","last_change":"2024-04-25T00:36:38.837497+0000","last_active":"2024-04-25T00:36:38.844370+0000","last_peered":"2024-04-25T00:36:38.844370+0000","last_clean":"2024-04-25T00:36:38.844370+0000","last_became_active":"2024-04-25T00:36:38.836746+0000","last_became_peered":"2024-04-25T00:36:38.836746+0000","last_unstale":"2024-04-25T00:36:38.844370+0000","last_undegraded":"2024-04-25T00:36:38.844370+0000","last_fullsized":"2024-04-25T00:36:38.844370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:49.597505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.808277+0000","last_change":"2024-04-25T00:36:38.808277+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.808277+0000","last_undegraded":"2024-04-25T00:36:38.808277+0000","last_fullsized":"2024-04-25T00:36:38.808277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.833351+0000","last_change":"2024-04-25T00:36:38.828622+0000","last_active":"2024-04-25T00:36:38.833351+0000","last_peered":"2024-04-25T00:36:38.833351+0000","last_clean":"2024-04-25T00:36:38.833351+0000","last_became_active":"2024-04-25T00:36:38.828270+0000","last_became_peered":"2024-04-25T00:36:38.828270+0000","last_unstale":"2024-04-25T00:36:38.833351+0000","last_undegraded":"2024-04-25T00:36:38.833351+0000","last_fullsized":"2024-04-25T00:36:38.833351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:33.405161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814505+0000","last_change":"2024-04-25T00:36:38.814505+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814505+0000","last_undegraded":"2024-04-25T00:36:38.814505+0000","last_fullsized":"2024-04-25T00:36:38.814505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.835128+0000","last_change":"2024-04-25T00:36:38.833289+0000","last_active":"2024-04-25T00:36:38.835128+0000","last_peered":"2024-04-25T00:36:38.835128+0000","last_clean":"2024-04-25T00:36:38.835128+0000","last_became_active":"2024-04-25T00:36:38.832940+0000","last_became_peered":"2024-04-25T00:36:38.832940+0000","last_unstale":"2024-04-25T00:36:38.835128+0000","last_undegraded":"2024-04-25T00:36:38.835128+0000","last_fullsized":"2024-04-25T00:36:38.835128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:15.477777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811606+0000","last_change":"2024-04-25T00:36:38.811606+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811606+0000","last_undegraded":"2024-04-25T00:36:38.811606+0000","last_fullsized":"2024-04-25T00:36:38.811606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.836292+0000","last_change":"2024-04-25T00:36:38.834888+0000","last_active":"2024-04-25T00:36:38.836292+0000","last_peered":"2024-04-25T00:36:38.836292+0000","last_clean":"2024-04-25T00:36:38.836292+0000","last_became_active":"2024-04-25T00:36:38.833935+0000","last_became_peered":"2024-04-25T00:36:38.833935+0000","last_unstale":"2024-04-25T00:36:38.836292+0000","last_undegraded":"2024-04-25T00:36:38.836292+0000","last_fullsized":"2024-04-25T00:36:38.836292+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:04.183502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811323+0000","last_change":"2024-04-25T00:36:38.811323+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811323+0000","last_undegraded":"2024-04-25T00:36:38.811323+0000","last_fullsized":"2024-04-25T00:36:38.811323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834500+0000","last_change":"2024-04-25T00:36:38.833368+0000","last_active":"2024-04-25T00:36:38.834500+0000","last_peered":"2024-04-25T00:36:38.834500+0000","last_clean":"2024-04-25T00:36:38.834500+0000","last_became_active":"2024-04-25T00:36:38.832870+0000","last_became_peered":"2024-04-25T00:36:38.832870+0000","last_unstale":"2024-04-25T00:36:38.834500+0000","last_undegraded":"2024-04-25T00:36:38.834500+0000","last_fullsized":"2024-04-25T00:36:38.834500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:05.341588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813368+0000","last_change":"2024-04-25T00:36:38.813368+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813368+0000","last_undegraded":"2024-04-25T00:36:38.813368+0000","last_fullsized":"2024-04-25T00:36:38.813368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.847195+0000","last_change":"2024-04-25T00:36:38.845296+0000","last_active":"2024-04-25T00:36:38.847195+0000","last_peered":"2024-04-25T00:36:38.847195+0000","last_clean":"2024-04-25T00:36:38.847195+0000","last_became_active":"2024-04-25T00:36:38.844447+0000","last_became_peered":"2024-04-25T00:36:38.844447+0000","last_unstale":"2024-04-25T00:36:38.847195+0000","last_undegraded":"2024-04-25T00:36:38.847195+0000","last_fullsized":"2024-04-25T00:36:38.847195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:38.147666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816628+0000","last_change":"2024-04-25T00:36:38.816628+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816628+0000","last_undegraded":"2024-04-25T00:36:38.816628+0000","last_fullsized":"2024-04-25T00:36:38.816628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.837861+0000","last_change":"2024-04-25T00:36:38.836218+0000","last_active":"2024-04-25T00:36:38.837861+0000","last_peered":"2024-04-25T00:36:38.837861+0000","last_clean":"2024-04-25T00:36:38.837861+0000","last_became_active":"2024-04-25T00:36:38.835946+0000","last_became_peered":"2024-04-25T00:36:38.835946+0000","last_unstale":"2024-04-25T00:36:38.837861+0000","last_undegraded":"2024-04-25T00:36:38.837861+0000","last_fullsized":"2024-04-25T00:36:38.837861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:48.430422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816425+0000","last_change":"2024-04-25T00:36:38.816425+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816425+0000","last_undegraded":"2024-04-25T00:36:38.816425+0000","last_fullsized":"2024-04-25T00:36:38.816425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.838313+0000","last_change":"2024-04-25T00:36:38.836980+0000","last_active":"2024-04-25T00:36:38.838313+0000","last_peered":"2024-04-25T00:36:38.838313+0000","last_clean":"2024-04-25T00:36:38.838313+0000","last_became_active":"2024-04-25T00:36:38.836722+0000","last_became_peered":"2024-04-25T00:36:38.836722+0000","last_unstale":"2024-04-25T00:36:38.838313+0000","last_undegraded":"2024-04-25T00:36:38.838313+0000","last_fullsized":"2024-04-25T00:36:38.838313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:29.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813225+0000","last_change":"2024-04-25T00:36:38.813225+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813225+0000","last_undegraded":"2024-04-25T00:36:38.813225+0000","last_fullsized":"2024-04-25T00:36:38.813225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.829873+0000","last_change":"2024-04-25T00:36:38.827338+0000","last_active":"2024-04-25T00:36:38.829873+0000","last_peered":"2024-04-25T00:36:38.829873+0000","last_clean":"2024-04-25T00:36:38.829873+0000","last_became_active":"2024-04-25T00:36:38.826884+0000","last_became_peered":"2024-04-25T00:36:38.826884+0000","last_unstale":"2024-04-25T00:36:38.829873+0000","last_undegraded":"2024-04-25T00:36:38.829873+0000","last_fullsized":"2024-04-25T00:36:38.829873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:10.228966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.828168+0000","last_change":"2024-04-25T00:36:38.828168+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.828168+0000","last_undegraded":"2024-04-25T00:36:38.828168+0000","last_fullsized":"2024-04-25T00:36:38.828168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834354+0000","last_change":"2024-04-25T00:36:38.829621+0000","last_active":"2024-04-25T00:36:38.834354+0000","last_peered":"2024-04-25T00:36:38.834354+0000","last_clean":"2024-04-25T00:36:38.834354+0000","last_became_active":"2024-04-25T00:36:38.827960+0000","last_became_peered":"2024-04-25T00:36:38.827960+0000","last_unstale":"2024-04-25T00:36:38.834354+0000","last_undegraded":"2024-04-25T00:36:38.834354+0000","last_fullsized":"2024-04-25T00:36:38.834354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:56.562023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.810472+0000","last_change":"2024-04-25T00:36:38.810472+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.810472+0000","last_undegraded":"2024-04-25T00:36:38.810472+0000","last_fullsized":"2024-04-25T00:36:38.810472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.827971+0000","last_change":"2024-04-25T00:36:38.823571+0000","last_active":"2024-04-25T00:36:38.827971+0000","last_peered":"2024-04-25T00:36:38.827971+0000","last_clean":"2024-04-25T00:36:38.827971+0000","last_became_active":"2024-04-25T00:36:38.823275+0000","last_became_peered":"2024-04-25T00:36:38.823275+0000","last_unstale":"2024-04-25T00:36:38.827971+0000","last_undegraded":"2024-04-25T00:36:38.827971+0000","last_fullsized":"2024-04-25T00:36:38.827971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:51.207453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.828686+0000","last_change":"2024-04-25T00:36:38.828686+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.828686+0000","last_undegraded":"2024-04-25T00:36:38.828686+0000","last_fullsized":"2024-04-25T00:36:38.828686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.829729+0000","last_change":"2024-04-25T00:36:38.827233+0000","last_active":"2024-04-25T00:36:38.829729+0000","last_peered":"2024-04-25T00:36:38.829729+0000","last_clean":"2024-04-25T00:36:38.829729+0000","last_became_active":"2024-04-25T00:36:38.826902+0000","last_became_peered":"2024-04-25T00:36:38.826902+0000","last_unstale":"2024-04-25T00:36:38.829729+0000","last_undegraded":"2024-04-25T00:36:38.829729+0000","last_fullsized":"2024-04-25T00:36:38.829729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:20.855968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818674+0000","last_change":"2024-04-25T00:36:38.818674+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818674+0000","last_undegraded":"2024-04-25T00:36:38.818674+0000","last_fullsized":"2024-04-25T00:36:38.818674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.839320+0000","last_change":"2024-04-25T00:36:38.838509+0000","last_active":"2024-04-25T00:36:38.839320+0000","last_peered":"2024-04-25T00:36:38.839320+0000","last_clean":"2024-04-25T00:36:38.839320+0000","last_became_active":"2024-04-25T00:36:38.837928+0000","last_became_peered":"2024-04-25T00:36:38.837928+0000","last_unstale":"2024-04-25T00:36:38.839320+0000","last_undegraded":"2024-04-25T00:36:38.839320+0000","last_fullsized":"2024-04-25T00:36:38.839320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:06.637343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812126+0000","last_change":"2024-04-25T00:36:38.812126+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812126+0000","last_undegraded":"2024-04-25T00:36:38.812126+0000","last_fullsized":"2024-04-25T00:36:38.812126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.833501+0000","last_change":"2024-04-25T00:36:38.828691+0000","last_active":"2024-04-25T00:36:38.833501+0000","last_peered":"2024-04-25T00:36:38.833501+0000","last_clean":"2024-04-25T00:36:38.833501+0000","last_became_active":"2024-04-25T00:36:38.828380+0000","last_became_peered":"2024-04-25T00:36:38.828380+0000","last_unstale":"2024-04-25T00:36:38.833501+0000","last_undegraded":"2024-04-25T00:36:38.833501+0000","last_fullsized":"2024-04-25T00:36:38.833501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:22.410800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816547+0000","last_change":"2024-04-25T00:36:38.816547+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816547+0000","last_undegraded":"2024-04-25T00:36:38.816547+0000","last_fullsized":"2024-04-25T00:36:38.816547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845767+0000","last_change":"2024-04-25T00:36:38.841467+0000","last_active":"2024-04-25T00:36:38.845767+0000","last_peered":"2024-04-25T00:36:38.845767+0000","last_clean":"2024-04-25T00:36:38.845767+0000","last_became_active":"2024-04-25T00:36:38.840182+0000","last_became_peered":"2024-04-25T00:36:38.840182+0000","last_unstale":"2024-04-25T00:36:38.845767+0000","last_undegraded":"2024-04-25T00:36:38.845767+0000","last_fullsized":"2024-04-25T00:36:38.845767+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:53.012076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815364+0000","last_change":"2024-04-25T00:36:38.815364+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815364+0000","last_undegraded":"2024-04-25T00:36:38.815364+0000","last_fullsized":"2024-04-25T00:36:38.815364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845638+0000","last_change":"2024-04-25T00:36:38.841047+0000","last_active":"2024-04-25T00:36:38.845638+0000","last_peered":"2024-04-25T00:36:38.845638+0000","last_clean":"2024-04-25T00:36:38.845638+0000","last_became_active":"2024-04-25T00:36:38.839481+0000","last_became_peered":"2024-04-25T00:36:38.839481+0000","last_unstale":"2024-04-25T00:36:38.845638+0000","last_undegraded":"2024-04-25T00:36:38.845638+0000","last_fullsized":"2024-04-25T00:36:38.845638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:11.985160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825951+0000","last_change":"2024-04-25T00:36:38.825951+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825951+0000","last_undegraded":"2024-04-25T00:36:38.825951+0000","last_fullsized":"2024-04-25T00:36:38.825951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.826182+0000","last_change":"2024-04-25T00:36:38.819664+0000","last_active":"2024-04-25T00:36:38.826182+0000","last_peered":"2024-04-25T00:36:38.826182+0000","last_clean":"2024-04-25T00:36:38.826182+0000","last_became_active":"2024-04-25T00:36:38.819378+0000","last_became_peered":"2024-04-25T00:36:38.819378+0000","last_unstale":"2024-04-25T00:36:38.826182+0000","last_undegraded":"2024-04-25T00:36:38.826182+0000","last_fullsized":"2024-04-25T00:36:38.826182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:52.487791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.807503+0000","last_change":"2024-04-25T00:36:38.807503+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.807503+0000","last_undegraded":"2024-04-25T00:36:38.807503+0000","last_fullsized":"2024-04-25T00:36:38.807503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834734+0000","last_change":"2024-04-25T00:36:38.830646+0000","last_active":"2024-04-25T00:36:38.834734+0000","last_peered":"2024-04-25T00:36:38.834734+0000","last_clean":"2024-04-25T00:36:38.834734+0000","last_became_active":"2024-04-25T00:36:38.829932+0000","last_became_peered":"2024-04-25T00:36:38.829932+0000","last_unstale":"2024-04-25T00:36:38.834734+0000","last_undegraded":"2024-04-25T00:36:38.834734+0000","last_fullsized":"2024-04-25T00:36:38.834734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:57.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818106+0000","last_change":"2024-04-25T00:36:38.818106+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818106+0000","last_undegraded":"2024-04-25T00:36:38.818106+0000","last_fullsized":"2024-04-25T00:36:38.818106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846554+0000","last_change":"2024-04-25T00:36:38.840716+0000","last_active":"2024-04-25T00:36:38.846554+0000","last_peered":"2024-04-25T00:36:38.846554+0000","last_clean":"2024-04-25T00:36:38.846554+0000","last_became_active":"2024-04-25T00:36:38.837997+0000","last_became_peered":"2024-04-25T00:36:38.837997+0000","last_unstale":"2024-04-25T00:36:38.846554+0000","last_undegraded":"2024-04-25T00:36:38.846554+0000","last_fullsized":"2024-04-25T00:36:38.846554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:54.469425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813359+0000","last_change":"2024-04-25T00:36:38.813359+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813359+0000","last_undegraded":"2024-04-25T00:36:38.813359+0000","last_fullsized":"2024-04-25T00:36:38.813359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846920+0000","last_change":"2024-04-25T00:36:38.834405+0000","last_active":"2024-04-25T00:36:38.846920+0000","last_peered":"2024-04-25T00:36:38.846920+0000","last_clean":"2024-04-25T00:36:38.846920+0000","last_became_active":"2024-04-25T00:36:38.833769+0000","last_became_peered":"2024-04-25T00:36:38.833769+0000","last_unstale":"2024-04-25T00:36:38.846920+0000","last_undegraded":"2024-04-25T00:36:38.846920+0000","last_fullsized":"2024-04-25T00:36:38.846920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:58.559610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814258+0000","last_change":"2024-04-25T00:36:38.814258+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814258+0000","last_undegraded":"2024-04-25T00:36:38.814258+0000","last_fullsized":"2024-04-25T00:36:38.814258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.842399+0000","last_change":"2024-04-25T00:36:38.835037+0000","last_active":"2024-04-25T00:36:38.842399+0000","last_peered":"2024-04-25T00:36:38.842399+0000","last_clean":"2024-04-25T00:36:38.842399+0000","last_became_active":"2024-04-25T00:36:38.833930+0000","last_became_peered":"2024-04-25T00:36:38.833930+0000","last_unstale":"2024-04-25T00:36:38.842399+0000","last_undegraded":"2024-04-25T00:36:38.842399+0000","last_fullsized":"2024-04-25T00:36:38.842399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:48.676020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":327,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.808362+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:36:38.808362+0000","last_peered":"2024-04-25T00:36:38.808362+0000","last_clean":"2024-04-25T00:36:38.808362+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:36:38.808362+0000","last_undegraded":"2024-04-25T00:36:38.808362+0000","last_fullsized":"2024-04-25T00:36:38.808362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36908,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334932,"statfs":{"total":96636764160,"available":96598970368,"internally_reserved":0,"allocated":1601536,"data_stored":1021755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":431387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":425112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1007616,"data_stored":431387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":995328,"data_stored":425112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":995328,"data_stored":425112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1007616,"data_stored":431387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36892,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334948,"statfs":{"total":96636764160,"available":96598986752,"internally_reserved":0,"allocated":1589248,"data_stored":1015480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:36:44.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3f is not active+clean 2024-04-25T00:36:44.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.825116+0000', 'last_change': '2024-04-25T00:36:38.825116+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.825116+0000', 'last_undegraded': '2024-04-25T00:36:38.825116+0000', 'last_fullsized': '2024-04-25T00:36:38.825116+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:36:44.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2024-04-25T00:36:44.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.816578+0000', 'last_change': '2024-04-25T00:36:38.816578+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.816578+0000', 'last_undegraded': '2024-04-25T00:36:38.816578+0000', 'last_fullsized': '2024-04-25T00:36:38.816578+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:36:44.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2024-04-25T00:36:44.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.821596+0000', 'last_change': '2024-04-25T00:36:38.821596+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.821596+0000', 'last_undegraded': '2024-04-25T00:36:38.821596+0000', 'last_fullsized': '2024-04-25T00:36:38.821596+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:36:44.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2024-04-25T00:36:44.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.829028+0000', 'last_change': '2024-04-25T00:36:38.829028+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.829028+0000', 'last_undegraded': '2024-04-25T00:36:38.829028+0000', 'last_fullsized': '2024-04-25T00:36:38.829028+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:36:44.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3b is not active+clean 2024-04-25T00:36:44.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.817867+0000', 'last_change': '2024-04-25T00:36:38.817867+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.817867+0000', 'last_undegraded': '2024-04-25T00:36:38.817867+0000', 'last_fullsized': '2024-04-25T00:36:38.817867+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:36:44.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2024-04-25T00:36:44.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.827070+0000', 'last_change': '2024-04-25T00:36:38.827070+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.827070+0000', 'last_undegraded': '2024-04-25T00:36:38.827070+0000', 'last_fullsized': '2024-04-25T00:36:38.827070+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:36:44.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2024-04-25T00:36:44.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.825982+0000', 'last_change': '2024-04-25T00:36:38.825982+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.825982+0000', 'last_undegraded': '2024-04-25T00:36:38.825982+0000', 'last_fullsized': '2024-04-25T00:36:38.825982+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:36:44.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2024-04-25T00:36:44.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.824317+0000', 'last_change': '2024-04-25T00:36:38.824317+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.824317+0000', 'last_undegraded': '2024-04-25T00:36:38.824317+0000', 'last_fullsized': '2024-04-25T00:36:38.824317+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:36:44.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.37 is not active+clean 2024-04-25T00:36:44.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.827607+0000', 'last_change': '2024-04-25T00:36:38.827607+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.827607+0000', 'last_undegraded': '2024-04-25T00:36:38.827607+0000', 'last_fullsized': '2024-04-25T00:36:38.827607+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:36:44.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2024-04-25T00:36:44.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.813471+0000', 'last_change': '2024-04-25T00:36:38.813471+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.813471+0000', 'last_undegraded': '2024-04-25T00:36:38.813471+0000', 'last_fullsized': '2024-04-25T00:36:38.813471+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:36:44.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2024-04-25T00:36:44.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.814021+0000', 'last_change': '2024-04-25T00:36:38.814021+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.814021+0000', 'last_undegraded': '2024-04-25T00:36:38.814021+0000', 'last_fullsized': '2024-04-25T00:36:38.814021+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:36:44.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2024-04-25T00:36:44.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.815493+0000', 'last_change': '2024-04-25T00:36:38.815493+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.815493+0000', 'last_undegraded': '2024-04-25T00:36:38.815493+0000', 'last_fullsized': '2024-04-25T00:36:38.815493+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:36:44.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.33 is not active+clean 2024-04-25T00:36:44.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.822046+0000', 'last_change': '2024-04-25T00:36:38.822046+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.822046+0000', 'last_undegraded': '2024-04-25T00:36:38.822046+0000', 'last_fullsized': '2024-04-25T00:36:38.822046+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:36:44.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2024-04-25T00:36:44.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.825738+0000', 'last_change': '2024-04-25T00:36:38.825738+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.825738+0000', 'last_undegraded': '2024-04-25T00:36:38.825738+0000', 'last_fullsized': '2024-04-25T00:36:38.825738+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:36:44.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2024-04-25T00:36:44.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.815105+0000', 'last_change': '2024-04-25T00:36:38.815105+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.815105+0000', 'last_undegraded': '2024-04-25T00:36:38.815105+0000', 'last_fullsized': '2024-04-25T00:36:38.815105+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:36:44.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.30 is not active+clean 2024-04-25T00:36:44.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.815145+0000', 'last_change': '2024-04-25T00:36:38.815145+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.815145+0000', 'last_undegraded': '2024-04-25T00:36:38.815145+0000', 'last_fullsized': '2024-04-25T00:36:38.815145+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:36:44.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2f is not active+clean 2024-04-25T00:36:44.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.819692+0000', 'last_change': '2024-04-25T00:36:38.819692+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.819692+0000', 'last_undegraded': '2024-04-25T00:36:38.819692+0000', 'last_fullsized': '2024-04-25T00:36:38.819692+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:36:44.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2024-04-25T00:36:44.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.818362+0000', 'last_change': '2024-04-25T00:36:38.818362+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.818362+0000', 'last_undegraded': '2024-04-25T00:36:38.818362+0000', 'last_fullsized': '2024-04-25T00:36:38.818362+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:36:44.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2d is not active+clean 2024-04-25T00:36:44.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.808911+0000', 'last_change': '2024-04-25T00:36:38.808911+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.808911+0000', 'last_undegraded': '2024-04-25T00:36:38.808911+0000', 'last_fullsized': '2024-04-25T00:36:38.808911+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:36:44.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2024-04-25T00:36:44.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.818455+0000', 'last_change': '2024-04-25T00:36:38.818455+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.818455+0000', 'last_undegraded': '2024-04-25T00:36:38.818455+0000', 'last_fullsized': '2024-04-25T00:36:38.818455+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:36:44.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2024-04-25T00:36:44.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.818675+0000', 'last_change': '2024-04-25T00:36:38.818675+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.818675+0000', 'last_undegraded': '2024-04-25T00:36:38.818675+0000', 'last_fullsized': '2024-04-25T00:36:38.818675+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:36:44.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2024-04-25T00:36:44.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.812757+0000', 'last_change': '2024-04-25T00:36:38.812757+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.812757+0000', 'last_undegraded': '2024-04-25T00:36:38.812757+0000', 'last_fullsized': '2024-04-25T00:36:38.812757+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:36:44.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2024-04-25T00:36:44.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.812237+0000', 'last_change': '2024-04-25T00:36:38.812237+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.812237+0000', 'last_undegraded': '2024-04-25T00:36:38.812237+0000', 'last_fullsized': '2024-04-25T00:36:38.812237+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:36:44.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2024-04-25T00:36:44.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.818137+0000', 'last_change': '2024-04-25T00:36:38.818137+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.818137+0000', 'last_undegraded': '2024-04-25T00:36:38.818137+0000', 'last_fullsized': '2024-04-25T00:36:38.818137+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:36:44.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2024-04-25T00:36:44.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.813023+0000', 'last_change': '2024-04-25T00:36:38.813023+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.813023+0000', 'last_undegraded': '2024-04-25T00:36:38.813023+0000', 'last_fullsized': '2024-04-25T00:36:38.813023+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:36:44.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.26 is not active+clean 2024-04-25T00:36:44.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.820114+0000', 'last_change': '2024-04-25T00:36:38.820114+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.820114+0000', 'last_undegraded': '2024-04-25T00:36:38.820114+0000', 'last_fullsized': '2024-04-25T00:36:38.820114+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:36:44.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.25 is not active+clean 2024-04-25T00:36:44.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.818582+0000', 'last_change': '2024-04-25T00:36:38.818582+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.818582+0000', 'last_undegraded': '2024-04-25T00:36:38.818582+0000', 'last_fullsized': '2024-04-25T00:36:38.818582+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:36:44.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.24 is not active+clean 2024-04-25T00:36:44.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.822677+0000', 'last_change': '2024-04-25T00:36:38.822677+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.822677+0000', 'last_undegraded': '2024-04-25T00:36:38.822677+0000', 'last_fullsized': '2024-04-25T00:36:38.822677+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:36:44.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2024-04-25T00:36:44.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.812989+0000', 'last_change': '2024-04-25T00:36:38.812989+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.812989+0000', 'last_undegraded': '2024-04-25T00:36:38.812989+0000', 'last_fullsized': '2024-04-25T00:36:38.812989+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:36:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2024-04-25T00:36:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.811755+0000', 'last_change': '2024-04-25T00:36:38.811755+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.811755+0000', 'last_undegraded': '2024-04-25T00:36:38.811755+0000', 'last_fullsized': '2024-04-25T00:36:38.811755+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:36:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.21 is not active+clean 2024-04-25T00:36:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.814147+0000', 'last_change': '2024-04-25T00:36:38.814147+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.814147+0000', 'last_undegraded': '2024-04-25T00:36:38.814147+0000', 'last_fullsized': '2024-04-25T00:36:38.814147+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:36:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2024-04-25T00:36:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.811325+0000', 'last_change': '2024-04-25T00:36:38.811325+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.811325+0000', 'last_undegraded': '2024-04-25T00:36:38.811325+0000', 'last_fullsized': '2024-04-25T00:36:38.811325+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:36:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2024-04-25T00:36:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.819706+0000', 'last_change': '2024-04-25T00:36:38.819706+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.819706+0000', 'last_undegraded': '2024-04-25T00:36:38.819706+0000', 'last_fullsized': '2024-04-25T00:36:38.819706+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:36:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2024-04-25T00:36:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.824175+0000', 'last_change': '2024-04-25T00:36:38.824175+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.824175+0000', 'last_undegraded': '2024-04-25T00:36:38.824175+0000', 'last_fullsized': '2024-04-25T00:36:38.824175+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:36:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2024-04-25T00:36:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.821618+0000', 'last_change': '2024-04-25T00:36:38.821618+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.821618+0000', 'last_undegraded': '2024-04-25T00:36:38.821618+0000', 'last_fullsized': '2024-04-25T00:36:38.821618+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:36:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2024-04-25T00:36:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.820427+0000', 'last_change': '2024-04-25T00:36:38.820427+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.820427+0000', 'last_undegraded': '2024-04-25T00:36:38.820427+0000', 'last_fullsized': '2024-04-25T00:36:38.820427+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:36:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2024-04-25T00:36:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.811231+0000', 'last_change': '2024-04-25T00:36:38.811231+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.811231+0000', 'last_undegraded': '2024-04-25T00:36:38.811231+0000', 'last_fullsized': '2024-04-25T00:36:38.811231+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:36:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2024-04-25T00:36:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.816742+0000', 'last_change': '2024-04-25T00:36:38.816742+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.816742+0000', 'last_undegraded': '2024-04-25T00:36:38.816742+0000', 'last_fullsized': '2024-04-25T00:36:38.816742+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:36:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2024-04-25T00:36:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.820408+0000', 'last_change': '2024-04-25T00:36:38.820408+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.820408+0000', 'last_undegraded': '2024-04-25T00:36:38.820408+0000', 'last_fullsized': '2024-04-25T00:36:38.820408+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:36:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2024-04-25T00:36:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.814334+0000', 'last_change': '2024-04-25T00:36:38.814334+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.814334+0000', 'last_undegraded': '2024-04-25T00:36:38.814334+0000', 'last_fullsized': '2024-04-25T00:36:38.814334+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:36:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2024-04-25T00:36:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.816222+0000', 'last_change': '2024-04-25T00:36:38.816222+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.816222+0000', 'last_undegraded': '2024-04-25T00:36:38.816222+0000', 'last_fullsized': '2024-04-25T00:36:38.816222+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:36:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2024-04-25T00:36:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.826636+0000', 'last_change': '2024-04-25T00:36:38.826636+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.826636+0000', 'last_undegraded': '2024-04-25T00:36:38.826636+0000', 'last_fullsized': '2024-04-25T00:36:38.826636+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:36:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2024-04-25T00:36:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.810011+0000', 'last_change': '2024-04-25T00:36:38.810011+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.810011+0000', 'last_undegraded': '2024-04-25T00:36:38.810011+0000', 'last_fullsized': '2024-04-25T00:36:38.810011+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:36:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2024-04-25T00:36:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.817507+0000', 'last_change': '2024-04-25T00:36:38.817507+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.817507+0000', 'last_undegraded': '2024-04-25T00:36:38.817507+0000', 'last_fullsized': '2024-04-25T00:36:38.817507+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:36:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2024-04-25T00:36:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.808277+0000', 'last_change': '2024-04-25T00:36:38.808277+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.808277+0000', 'last_undegraded': '2024-04-25T00:36:38.808277+0000', 'last_fullsized': '2024-04-25T00:36:38.808277+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:36:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2024-04-25T00:36:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.814505+0000', 'last_change': '2024-04-25T00:36:38.814505+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.814505+0000', 'last_undegraded': '2024-04-25T00:36:38.814505+0000', 'last_fullsized': '2024-04-25T00:36:38.814505+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:36:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2024-04-25T00:36:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.811606+0000', 'last_change': '2024-04-25T00:36:38.811606+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.811606+0000', 'last_undegraded': '2024-04-25T00:36:38.811606+0000', 'last_fullsized': '2024-04-25T00:36:38.811606+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:36:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2024-04-25T00:36:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.811323+0000', 'last_change': '2024-04-25T00:36:38.811323+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.811323+0000', 'last_undegraded': '2024-04-25T00:36:38.811323+0000', 'last_fullsized': '2024-04-25T00:36:38.811323+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:36:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2024-04-25T00:36:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.813368+0000', 'last_change': '2024-04-25T00:36:38.813368+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.813368+0000', 'last_undegraded': '2024-04-25T00:36:38.813368+0000', 'last_fullsized': '2024-04-25T00:36:38.813368+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:36:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2024-04-25T00:36:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.816628+0000', 'last_change': '2024-04-25T00:36:38.816628+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.816628+0000', 'last_undegraded': '2024-04-25T00:36:38.816628+0000', 'last_fullsized': '2024-04-25T00:36:38.816628+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:36:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2024-04-25T00:36:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.816425+0000', 'last_change': '2024-04-25T00:36:38.816425+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.816425+0000', 'last_undegraded': '2024-04-25T00:36:38.816425+0000', 'last_fullsized': '2024-04-25T00:36:38.816425+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:36:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2024-04-25T00:36:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.813225+0000', 'last_change': '2024-04-25T00:36:38.813225+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.813225+0000', 'last_undegraded': '2024-04-25T00:36:38.813225+0000', 'last_fullsized': '2024-04-25T00:36:38.813225+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:36:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2024-04-25T00:36:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.828168+0000', 'last_change': '2024-04-25T00:36:38.828168+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.828168+0000', 'last_undegraded': '2024-04-25T00:36:38.828168+0000', 'last_fullsized': '2024-04-25T00:36:38.828168+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:36:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2024-04-25T00:36:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.810472+0000', 'last_change': '2024-04-25T00:36:38.810472+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.810472+0000', 'last_undegraded': '2024-04-25T00:36:38.810472+0000', 'last_fullsized': '2024-04-25T00:36:38.810472+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:36:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2024-04-25T00:36:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.828686+0000', 'last_change': '2024-04-25T00:36:38.828686+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.828686+0000', 'last_undegraded': '2024-04-25T00:36:38.828686+0000', 'last_fullsized': '2024-04-25T00:36:38.828686+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:36:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2024-04-25T00:36:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.818674+0000', 'last_change': '2024-04-25T00:36:38.818674+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.818674+0000', 'last_undegraded': '2024-04-25T00:36:38.818674+0000', 'last_fullsized': '2024-04-25T00:36:38.818674+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:36:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2024-04-25T00:36:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.812126+0000', 'last_change': '2024-04-25T00:36:38.812126+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.812126+0000', 'last_undegraded': '2024-04-25T00:36:38.812126+0000', 'last_fullsized': '2024-04-25T00:36:38.812126+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:36:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2024-04-25T00:36:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.816547+0000', 'last_change': '2024-04-25T00:36:38.816547+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.816547+0000', 'last_undegraded': '2024-04-25T00:36:38.816547+0000', 'last_fullsized': '2024-04-25T00:36:38.816547+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:36:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2024-04-25T00:36:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.815364+0000', 'last_change': '2024-04-25T00:36:38.815364+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.815364+0000', 'last_undegraded': '2024-04-25T00:36:38.815364+0000', 'last_fullsized': '2024-04-25T00:36:38.815364+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:36:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2024-04-25T00:36:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.825951+0000', 'last_change': '2024-04-25T00:36:38.825951+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.825951+0000', 'last_undegraded': '2024-04-25T00:36:38.825951+0000', 'last_fullsized': '2024-04-25T00:36:38.825951+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:36:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2024-04-25T00:36:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.807503+0000', 'last_change': '2024-04-25T00:36:38.807503+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.807503+0000', 'last_undegraded': '2024-04-25T00:36:38.807503+0000', 'last_fullsized': '2024-04-25T00:36:38.807503+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:36:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2024-04-25T00:36:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.818106+0000', 'last_change': '2024-04-25T00:36:38.818106+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.818106+0000', 'last_undegraded': '2024-04-25T00:36:38.818106+0000', 'last_fullsized': '2024-04-25T00:36:38.818106+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:36:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2024-04-25T00:36:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.813359+0000', 'last_change': '2024-04-25T00:36:38.813359+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.813359+0000', 'last_undegraded': '2024-04-25T00:36:38.813359+0000', 'last_fullsized': '2024-04-25T00:36:38.813359+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:36:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2024-04-25T00:36:44.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:36:38.814258+0000', 'last_change': '2024-04-25T00:36:38.814258+0000', 'last_active': '2024-04-25T00:36:38.792404+0000', 'last_peered': '2024-04-25T00:36:38.792404+0000', 'last_clean': '2024-04-25T00:36:38.792404+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:36:38.814258+0000', 'last_undegraded': '2024-04-25T00:36:38.814258+0000', 'last_fullsized': '2024-04-25T00:36:38.814258+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'last_clean_scrub_stamp': '2024-04-25T00:36:38.792404+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:36:44.618 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:36:44.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.764+0000 7f735d274640 1 Processor -- start 2024-04-25T00:36:44.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.764+0000 7f735d274640 1 -- start start 2024-04-25T00:36:44.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.764+0000 7f735d274640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f735815ad20 0x7f73580cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.764+0000 7f735d274640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7358071d80 0x7f73580721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.764+0000 7f735d274640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73580726e0 con 0x7f73580d0390 2024-04-25T00:36:44.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.764+0000 7f735d274640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f73580cb6a0 con 0x7f7358071d80 2024-04-25T00:36:44.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.764+0000 7f735d274640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f735815bc30 con 0x7f735815ad20 2024-04-25T00:36:44.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.764+0000 7f73577fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7358071d80 0x7f73580721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.764+0000 7f734ffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f73580d0390 0x7f735815b100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53376/0 (socket says 172.21.15.29:53376) 2024-04-25T00:36:44.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.764+0000 7f734ffff640 1 -- 172.21.15.29:0/845246242 learned_addr learned my addr 172.21.15.29:0/845246242 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:44.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f73577fe640 1 -- 172.21.15.29:0/845246242 >> v2:172.21.15.157:3300/0 conn(0x7f735815ad20 msgr2=0x7f73580cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:44.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f7356ffd640 1 --2- 172.21.15.29:0/845246242 >> v2:172.21.15.157:3300/0 conn(0x7f735815ad20 0x7f73580cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f73577fe640 1 --2- 172.21.15.29:0/845246242 >> v2:172.21.15.157:3300/0 conn(0x7f735815ad20 0x7f73580cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f73577fe640 1 -- 172.21.15.29:0/845246242 >> v1:172.21.15.29:6789/0 conn(0x7f73580d0390 legacy=0x7f735815b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f73567fc640 1 -- 172.21.15.29:0/845246242 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 873352911 0 0) 0x7f73580726e0 con 0x7f73580d0390 2024-04-25T00:36:44.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f73577fe640 1 -- 172.21.15.29:0/845246242 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735815bf10 con 0x7f7358071d80 2024-04-25T00:36:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f7356ffd640 1 --2- 172.21.15.29:0/845246242 >> v2:172.21.15.157:3300/0 conn(0x7f735815ad20 0x7f73580cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f73577fe640 1 --2- 172.21.15.29:0/845246242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7358071d80 0x7f73580721a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f73580cd620 tx=0x7f735001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5b61339528242f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f73567fc640 1 -- 172.21.15.29:0/845246242 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73500b2070 con 0x7f7358071d80 2024-04-25T00:36:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f73567fc640 1 -- 172.21.15.29:0/845246242 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73500a59c0 con 0x7f7358071d80 2024-04-25T00:36:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f73567fc640 1 -- 172.21.15.29:0/845246242 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73500a5d90 con 0x7f7358071d80 2024-04-25T00:36:44.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f735d274640 1 -- 172.21.15.29:0/845246242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7358071d80 msgr2=0x7f73580721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f735d274640 1 --2- 172.21.15.29:0/845246242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7358071d80 0x7f73580721a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f73580cd620 tx=0x7f735001c900 comp rx=0 tx=0).stop 2024-04-25T00:36:44.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f735d274640 1 -- 172.21.15.29:0/845246242 shutdown_connections 2024-04-25T00:36:44.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f735d274640 1 --2- 172.21.15.29:0/845246242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7358071d80 0x7f73580721a0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f735d274640 1 --2- 172.21.15.29:0/845246242 >> v2:172.21.15.157:3300/0 conn(0x7f735815ad20 0x7f73580cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f735d274640 1 -- 172.21.15.29:0/845246242 >> 172.21.15.29:0/845246242 conn(0x7f73580d0c40 msgr2=0x7f73580d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:44.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f735d274640 1 -- 172.21.15.29:0/845246242 shutdown_connections 2024-04-25T00:36:44.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.768+0000 7f735d274640 1 -- 172.21.15.29:0/845246242 wait complete. 2024-04-25T00:36:44.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f735d274640 1 Processor -- start 2024-04-25T00:36:44.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f735d274640 1 -- start start 2024-04-25T00:36:44.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f735d274640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7358071d80 0x7f73580c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f735d274640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73580d0390 0x7f735814c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f735d274640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f735815ad20 0x7f735814c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f735d274640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f735815b990 con 0x7f73580d0390 2024-04-25T00:36:44.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f735d274640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f735815b810 con 0x7f7358071d80 2024-04-25T00:36:44.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f735d274640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f735806f530 con 0x7f735815ad20 2024-04-25T00:36:44.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73580d0390 0x7f735814c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73580d0390 0x7f735814c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37038/0 (socket says 172.21.15.29:37038) 2024-04-25T00:36:44.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734ffff640 1 -- 172.21.15.29:0/1045650792 learned_addr learned my addr 172.21.15.29:0/1045650792 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:44.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734ffff640 1 -- 172.21.15.29:0/1045650792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f735815ad20 msgr2=0x7f735814c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734ffff640 1 --2- 172.21.15.29:0/1045650792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f735815ad20 0x7f735814c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734ffff640 1 -- 172.21.15.29:0/1045650792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7358071d80 msgr2=0x7f73580c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734ffff640 1 --2- 172.21.15.29:0/1045650792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7358071d80 0x7f73580c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734ffff640 1 -- 172.21.15.29:0/1045650792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73580b6020 con 0x7f73580d0390 2024-04-25T00:36:44.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734ffff640 1 --2- 172.21.15.29:0/1045650792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73580d0390 0x7f735814c240 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f734000c4a0 tx=0x7f734000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:44.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734f7fe640 1 -- 172.21.15.29:0/1045650792 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340018070 con 0x7f73580d0390 2024-04-25T00:36:44.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734f7fe640 1 -- 172.21.15.29:0/1045650792 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73400074e0 con 0x7f73580d0390 2024-04-25T00:36:44.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734f7fe640 1 -- 172.21.15.29:0/1045650792 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340004a30 con 0x7f73580d0390 2024-04-25T00:36:44.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f735d274640 1 -- 172.21.15.29:0/1045650792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73580b62c0 con 0x7f73580d0390 2024-04-25T00:36:44.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f735d274640 1 -- 172.21.15.29:0/1045650792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73580b44b0 con 0x7f73580d0390 2024-04-25T00:36:44.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f735d274640 1 -- 172.21.15.29:0/1045650792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7358069270 con 0x7f73580d0390 2024-04-25T00:36:44.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734f7fe640 1 -- 172.21.15.29:0/1045650792 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f734001b070 con 0x7f73580d0390 2024-04-25T00:36:44.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.772+0000 7f734f7fe640 1 --2- 172.21.15.29:0/1045650792 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f733c0679e0 0x7f733c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:44.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.776+0000 7f734f7fe640 1 -- 172.21.15.29:0/1045650792 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f73400850d0 con 0x7f73580d0390 2024-04-25T00:36:44.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.776+0000 7f7356ffd640 1 --2- 172.21.15.29:0/1045650792 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f733c0679e0 0x7f733c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:44.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.776+0000 7f7356ffd640 1 --2- 172.21.15.29:0/1045650792 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f733c0679e0 0x7f733c069ea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f734800a890 tx=0x7f7348008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:44.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.780+0000 7f734f7fe640 1 -- 172.21.15.29:0/1045650792 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7340055b00 con 0x7f73580d0390 2024-04-25T00:36:44.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.988+0000 7f735d274640 1 -- 172.21.15.29:0/1045650792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f735806b460 con 0x7f73580d0390 2024-04-25T00:36:44.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.988+0000 7f734f7fe640 1 -- 172.21.15.29:0/1045650792 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1453 (secure 0 0 0) 0x7f7340055400 con 0x7f73580d0390 2024-04-25T00:36:44.992 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:44.992 INFO:teuthology.orchestra.run.smithi029.stdout:{"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":332,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":84,"num_osds":8,"num_up_osds":8,"osd_up_since":1714005083,"num_in_osds":8,"osd_in_since":1714005074,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":290201600,"bytes_avail":772803911680,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":53,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7404}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-04-25T00:35:37.531519+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:36:44.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.992+0000 7f735d274640 1 -- 172.21.15.29:0/1045650792 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f733c0679e0 msgr2=0x7f733c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.992+0000 7f735d274640 1 --2- 172.21.15.29:0/1045650792 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f733c0679e0 0x7f733c069ea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f734800a890 tx=0x7f7348008040 comp rx=0 tx=0).stop 2024-04-25T00:36:44.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.992+0000 7f735d274640 1 -- 172.21.15.29:0/1045650792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73580d0390 msgr2=0x7f735814c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:44.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.992+0000 7f735d274640 1 --2- 172.21.15.29:0/1045650792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73580d0390 0x7f735814c240 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f734000c4a0 tx=0x7f734000c960 comp rx=0 tx=0).stop 2024-04-25T00:36:44.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.992+0000 7f735d274640 1 -- 172.21.15.29:0/1045650792 shutdown_connections 2024-04-25T00:36:44.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.992+0000 7f735d274640 1 --2- 172.21.15.29:0/1045650792 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f733c0679e0 0x7f733c069ea0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.992+0000 7f735d274640 1 --2- 172.21.15.29:0/1045650792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f735815ad20 0x7f735814c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.992+0000 7f735d274640 1 --2- 172.21.15.29:0/1045650792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73580d0390 0x7f735814c240 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.992+0000 7f735d274640 1 --2- 172.21.15.29:0/1045650792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7358071d80 0x7f73580c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:44.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.992+0000 7f735d274640 1 -- 172.21.15.29:0/1045650792 >> 172.21.15.29:0/1045650792 conn(0x7f73580d0c40 msgr2=0x7f73580b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:44.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.992+0000 7f735d274640 1 -- 172.21.15.29:0/1045650792 shutdown_connections 2024-04-25T00:36:44.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:44.992+0000 7f735d274640 1 -- 172.21.15.29:0/1045650792 wait complete. 2024-04-25T00:36:45.009 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T00:36:45.009 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:36:45.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00cbd2c640 1 Processor -- start 2024-04-25T00:36:45.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00cbd2c640 1 -- start start 2024-04-25T00:36:45.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00cbd2c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00c40d4d10 0x7f00c40d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:45.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00cbd2c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f00c40b4040 0x7f00c40b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:45.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00cbd2c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00c40cf8a0 con 0x7f00c415e2a0 2024-04-25T00:36:45.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00cbd2c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f00c40d19a0 con 0x7f00c40b4040 2024-04-25T00:36:45.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00cbd2c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f00c40cf650 con 0x7f00c40d4d10 2024-04-25T00:36:45.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00c9ac8640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f00c415e2a0 0x7f00c40d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53390/0 (socket says 172.21.15.29:53390) 2024-04-25T00:36:45.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00c9ac8640 1 -- 172.21.15.29:0/1970766068 learned_addr learned my addr 172.21.15.29:0/1970766068 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:45.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00ca2c9640 1 --2- 172.21.15.29:0/1970766068 >> v2:172.21.15.29:3300/0 conn(0x7f00c40b4040 0x7f00c40b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:45.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00c92c7640 1 --2- 172.21.15.29:0/1970766068 >> v2:172.21.15.157:3300/0 conn(0x7f00c40d4d10 0x7f00c40d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:45.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00c8ac6640 1 -- 172.21.15.29:0/1970766068 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4249821140 0 0) 0x7f00c40cf8a0 con 0x7f00c415e2a0 2024-04-25T00:36:45.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00c8ac6640 1 -- 172.21.15.29:0/1970766068 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00b0003530 con 0x7f00c415e2a0 2024-04-25T00:36:45.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00ca2c9640 1 -- 172.21.15.29:0/1970766068 >> v2:172.21.15.157:3300/0 conn(0x7f00c40d4d10 msgr2=0x7f00c40d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:45.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00ca2c9640 1 --2- 172.21.15.29:0/1970766068 >> v2:172.21.15.157:3300/0 conn(0x7f00c40d4d10 0x7f00c40d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:45.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00ca2c9640 1 -- 172.21.15.29:0/1970766068 >> v1:172.21.15.29:6789/0 conn(0x7f00c415e2a0 legacy=0x7f00c40d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:45.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.132+0000 7f00ca2c9640 1 -- 172.21.15.29:0/1970766068 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c40cfb80 con 0x7f00c40b4040 2024-04-25T00:36:45.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00ca2c9640 1 --2- 172.21.15.29:0/1970766068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00c40b4040 0x7f00c40b44b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f00c4075430 tx=0x7f00bc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3272e98fe2bceaac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:45.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00c8ac6640 1 -- 172.21.15.29:0/1970766068 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc0b2070 con 0x7f00c40b4040 2024-04-25T00:36:45.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00c8ac6640 1 -- 172.21.15.29:0/1970766068 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00bc0a5760 con 0x7f00c40b4040 2024-04-25T00:36:45.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00c8ac6640 1 -- 172.21.15.29:0/1970766068 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc0a5b30 con 0x7f00c40b4040 2024-04-25T00:36:45.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 -- 172.21.15.29:0/1970766068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00c40b4040 msgr2=0x7f00c40b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:45.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 --2- 172.21.15.29:0/1970766068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00c40b4040 0x7f00c40b44b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f00c4075430 tx=0x7f00bc01c8a0 comp rx=0 tx=0).stop 2024-04-25T00:36:45.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 -- 172.21.15.29:0/1970766068 shutdown_connections 2024-04-25T00:36:45.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 --2- 172.21.15.29:0/1970766068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00c40b4040 0x7f00c40b44b0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:45.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 --2- 172.21.15.29:0/1970766068 >> v2:172.21.15.157:3300/0 conn(0x7f00c40d4d10 0x7f00c40d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:45.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 -- 172.21.15.29:0/1970766068 >> 172.21.15.29:0/1970766068 conn(0x7f00c40d4030 msgr2=0x7f00c40d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:45.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 -- 172.21.15.29:0/1970766068 shutdown_connections 2024-04-25T00:36:45.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 -- 172.21.15.29:0/1970766068 wait complete. 2024-04-25T00:36:45.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 Processor -- start 2024-04-25T00:36:45.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 -- start start 2024-04-25T00:36:45.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00c40b4040 0x7f00c414cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:45.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00c40d4d10 0x7f00c414de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:45.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00c415e2a0 0x7f00c415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:45.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f00c416aef0 con 0x7f00c40b4040 2024-04-25T00:36:45.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f00c40d2a60 con 0x7f00c415e2a0 2024-04-25T00:36:45.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f00c416b070 con 0x7f00c40d4d10 2024-04-25T00:36:45.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00c9ac8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00c40b4040 0x7f00c414cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:45.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00c9ac8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00c40b4040 0x7f00c414cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37060/0 (socket says 172.21.15.29:37060) 2024-04-25T00:36:45.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00c9ac8640 1 -- 172.21.15.29:0/962285125 learned_addr learned my addr 172.21.15.29:0/962285125 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:45.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00ca2c9640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00c415e2a0 0x7f00c415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:45.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00c92c7640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00c40d4d10 0x7f00c414de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:45.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00c9ac8640 1 -- 172.21.15.29:0/962285125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00c40d4d10 msgr2=0x7f00c414de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:45.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00c9ac8640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00c40d4d10 0x7f00c414de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:45.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00c9ac8640 1 -- 172.21.15.29:0/962285125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00c415e2a0 msgr2=0x7f00c415a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:45.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00c9ac8640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00c415e2a0 0x7f00c415a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:45.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00c9ac8640 1 -- 172.21.15.29:0/962285125 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c414d4b0 con 0x7f00c40b4040 2024-04-25T00:36:45.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00c9ac8640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00c40b4040 0x7f00c414cf70 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c3f0 tx=0x7f00c000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:45.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00b67fc640 1 -- 172.21.15.29:0/962285125 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0017070 con 0x7f00c40b4040 2024-04-25T00:36:45.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00b67fc640 1 -- 172.21.15.29:0/962285125 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c0005cf0 con 0x7f00c40b4040 2024-04-25T00:36:45.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00b67fc640 1 -- 172.21.15.29:0/962285125 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c00072f0 con 0x7f00c40b4040 2024-04-25T00:36:45.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 -- 172.21.15.29:0/962285125 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c414e7d0 con 0x7f00c40b4040 2024-04-25T00:36:45.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.136+0000 7f00cbd2c640 1 -- 172.21.15.29:0/962285125 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c40cbc50 con 0x7f00c40b4040 2024-04-25T00:36:45.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.140+0000 7f00cbd2c640 1 -- 172.21.15.29:0/962285125 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c414ecf0 con 0x7f00c40b4040 2024-04-25T00:36:45.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.140+0000 7f00b67fc640 1 -- 172.21.15.29:0/962285125 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f00c001a070 con 0x7f00c40b4040 2024-04-25T00:36:45.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.140+0000 7f00b67fc640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f00a80679e0 0x7f00a8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:45.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.140+0000 7f00b67fc640 1 -- 172.21.15.29:0/962285125 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f00c00851e0 con 0x7f00c40b4040 2024-04-25T00:36:45.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.140+0000 7f00c92c7640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f00a80679e0 0x7f00a8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:45.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.140+0000 7f00c92c7640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f00a80679e0 0x7f00a8069ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f00c41fa5b0 tx=0x7f00b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:45.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.144+0000 7f00b67fc640 1 -- 172.21.15.29:0/962285125 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f00c0055c80 con 0x7f00c40b4040 2024-04-25T00:36:45.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.300+0000 7f00cbd2c640 1 -- 172.21.15.29:0/962285125 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00c40cba40 con 0x7f00a80679e0 2024-04-25T00:36:45.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.316+0000 7f00b67fc640 1 -- 172.21.15.29:0/962285125 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246756 (secure 0 0 0) 0x7f00c40cba40 con 0x7f00a80679e0 2024-04-25T00:36:45.322 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:45.324 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:36:45.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.320+0000 7f00cbd2c640 1 -- 172.21.15.29:0/962285125 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f00a80679e0 msgr2=0x7f00a8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:45.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.320+0000 7f00cbd2c640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f00a80679e0 0x7f00a8069ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f00c41fa5b0 tx=0x7f00b8008040 comp rx=0 tx=0).stop 2024-04-25T00:36:45.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.320+0000 7f00cbd2c640 1 -- 172.21.15.29:0/962285125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00c40b4040 msgr2=0x7f00c414cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:45.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.320+0000 7f00cbd2c640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00c40b4040 0x7f00c414cf70 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c3f0 tx=0x7f00c000c8b0 comp rx=0 tx=0).stop 2024-04-25T00:36:45.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.320+0000 7f00cbd2c640 1 -- 172.21.15.29:0/962285125 shutdown_connections 2024-04-25T00:36:45.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.320+0000 7f00cbd2c640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f00a80679e0 0x7f00a8069ea0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:45.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.320+0000 7f00cbd2c640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00c415e2a0 0x7f00c415a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:45.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.320+0000 7f00cbd2c640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00c40d4d10 0x7f00c414de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:45.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.320+0000 7f00cbd2c640 1 --2- 172.21.15.29:0/962285125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00c40b4040 0x7f00c414cf70 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:45.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.320+0000 7f00cbd2c640 1 -- 172.21.15.29:0/962285125 >> 172.21.15.29:0/962285125 conn(0x7f00c40d4030 msgr2=0x7f00c40b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:45.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.320+0000 7f00cbd2c640 1 -- 172.21.15.29:0/962285125 shutdown_connections 2024-04-25T00:36:45.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:45.320+0000 7f00cbd2c640 1 -- 172.21.15.29:0/962285125 wait complete. 2024-04-25T00:36:45.338 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-25T00:36:43.312352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283400,"kb_used_data":8984,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754691320,"statfs":{"total":773094113280,"available":772803911680,"internally_reserved":0,"allocated":9199616,"data_stored":4606732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.519429"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825116+0000","last_change":"2024-04-25T00:36:38.825116+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825116+0000","last_undegraded":"2024-04-25T00:36:38.825116+0000","last_fullsized":"2024-04-25T00:36:38.825116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816578+0000","last_change":"2024-04-25T00:36:38.816578+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816578+0000","last_undegraded":"2024-04-25T00:36:38.816578+0000","last_fullsized":"2024-04-25T00:36:38.816578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.821596+0000","last_change":"2024-04-25T00:36:38.821596+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.821596+0000","last_undegraded":"2024-04-25T00:36:38.821596+0000","last_fullsized":"2024-04-25T00:36:38.821596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.829028+0000","last_change":"2024-04-25T00:36:38.829028+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.829028+0000","last_undegraded":"2024-04-25T00:36:38.829028+0000","last_fullsized":"2024-04-25T00:36:38.829028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.817867+0000","last_change":"2024-04-25T00:36:38.817867+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.817867+0000","last_undegraded":"2024-04-25T00:36:38.817867+0000","last_fullsized":"2024-04-25T00:36:38.817867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.827070+0000","last_change":"2024-04-25T00:36:38.827070+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.827070+0000","last_undegraded":"2024-04-25T00:36:38.827070+0000","last_fullsized":"2024-04-25T00:36:38.827070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825982+0000","last_change":"2024-04-25T00:36:38.825982+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825982+0000","last_undegraded":"2024-04-25T00:36:38.825982+0000","last_fullsized":"2024-04-25T00:36:38.825982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.824317+0000","last_change":"2024-04-25T00:36:38.824317+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.824317+0000","last_undegraded":"2024-04-25T00:36:38.824317+0000","last_fullsized":"2024-04-25T00:36:38.824317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.827607+0000","last_change":"2024-04-25T00:36:38.827607+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.827607+0000","last_undegraded":"2024-04-25T00:36:38.827607+0000","last_fullsized":"2024-04-25T00:36:38.827607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813471+0000","last_change":"2024-04-25T00:36:38.813471+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813471+0000","last_undegraded":"2024-04-25T00:36:38.813471+0000","last_fullsized":"2024-04-25T00:36:38.813471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814021+0000","last_change":"2024-04-25T00:36:38.814021+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814021+0000","last_undegraded":"2024-04-25T00:36:38.814021+0000","last_fullsized":"2024-04-25T00:36:38.814021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815493+0000","last_change":"2024-04-25T00:36:38.815493+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815493+0000","last_undegraded":"2024-04-25T00:36:38.815493+0000","last_fullsized":"2024-04-25T00:36:38.815493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.822046+0000","last_change":"2024-04-25T00:36:38.822046+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.822046+0000","last_undegraded":"2024-04-25T00:36:38.822046+0000","last_fullsized":"2024-04-25T00:36:38.822046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825738+0000","last_change":"2024-04-25T00:36:38.825738+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825738+0000","last_undegraded":"2024-04-25T00:36:38.825738+0000","last_fullsized":"2024-04-25T00:36:38.825738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815105+0000","last_change":"2024-04-25T00:36:38.815105+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815105+0000","last_undegraded":"2024-04-25T00:36:38.815105+0000","last_fullsized":"2024-04-25T00:36:38.815105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815145+0000","last_change":"2024-04-25T00:36:38.815145+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815145+0000","last_undegraded":"2024-04-25T00:36:38.815145+0000","last_fullsized":"2024-04-25T00:36:38.815145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.819692+0000","last_change":"2024-04-25T00:36:38.819692+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.819692+0000","last_undegraded":"2024-04-25T00:36:38.819692+0000","last_fullsized":"2024-04-25T00:36:38.819692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818362+0000","last_change":"2024-04-25T00:36:38.818362+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818362+0000","last_undegraded":"2024-04-25T00:36:38.818362+0000","last_fullsized":"2024-04-25T00:36:38.818362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.808911+0000","last_change":"2024-04-25T00:36:38.808911+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.808911+0000","last_undegraded":"2024-04-25T00:36:38.808911+0000","last_fullsized":"2024-04-25T00:36:38.808911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818455+0000","last_change":"2024-04-25T00:36:38.818455+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818455+0000","last_undegraded":"2024-04-25T00:36:38.818455+0000","last_fullsized":"2024-04-25T00:36:38.818455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818675+0000","last_change":"2024-04-25T00:36:38.818675+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818675+0000","last_undegraded":"2024-04-25T00:36:38.818675+0000","last_fullsized":"2024-04-25T00:36:38.818675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812757+0000","last_change":"2024-04-25T00:36:38.812757+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812757+0000","last_undegraded":"2024-04-25T00:36:38.812757+0000","last_fullsized":"2024-04-25T00:36:38.812757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812237+0000","last_change":"2024-04-25T00:36:38.812237+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812237+0000","last_undegraded":"2024-04-25T00:36:38.812237+0000","last_fullsized":"2024-04-25T00:36:38.812237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818137+0000","last_change":"2024-04-25T00:36:38.818137+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818137+0000","last_undegraded":"2024-04-25T00:36:38.818137+0000","last_fullsized":"2024-04-25T00:36:38.818137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813023+0000","last_change":"2024-04-25T00:36:38.813023+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813023+0000","last_undegraded":"2024-04-25T00:36:38.813023+0000","last_fullsized":"2024-04-25T00:36:38.813023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.820114+0000","last_change":"2024-04-25T00:36:38.820114+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.820114+0000","last_undegraded":"2024-04-25T00:36:38.820114+0000","last_fullsized":"2024-04-25T00:36:38.820114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818582+0000","last_change":"2024-04-25T00:36:38.818582+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818582+0000","last_undegraded":"2024-04-25T00:36:38.818582+0000","last_fullsized":"2024-04-25T00:36:38.818582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.822677+0000","last_change":"2024-04-25T00:36:38.822677+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.822677+0000","last_undegraded":"2024-04-25T00:36:38.822677+0000","last_fullsized":"2024-04-25T00:36:38.822677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812989+0000","last_change":"2024-04-25T00:36:38.812989+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812989+0000","last_undegraded":"2024-04-25T00:36:38.812989+0000","last_fullsized":"2024-04-25T00:36:38.812989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811755+0000","last_change":"2024-04-25T00:36:38.811755+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811755+0000","last_undegraded":"2024-04-25T00:36:38.811755+0000","last_fullsized":"2024-04-25T00:36:38.811755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814147+0000","last_change":"2024-04-25T00:36:38.814147+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814147+0000","last_undegraded":"2024-04-25T00:36:38.814147+0000","last_fullsized":"2024-04-25T00:36:38.814147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811325+0000","last_change":"2024-04-25T00:36:38.811325+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811325+0000","last_undegraded":"2024-04-25T00:36:38.811325+0000","last_fullsized":"2024-04-25T00:36:38.811325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.819706+0000","last_change":"2024-04-25T00:36:38.819706+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.819706+0000","last_undegraded":"2024-04-25T00:36:38.819706+0000","last_fullsized":"2024-04-25T00:36:38.819706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845710+0000","last_change":"2024-04-25T00:36:38.842146+0000","last_active":"2024-04-25T00:36:38.845710+0000","last_peered":"2024-04-25T00:36:38.845710+0000","last_clean":"2024-04-25T00:36:38.845710+0000","last_became_active":"2024-04-25T00:36:38.841379+0000","last_became_peered":"2024-04-25T00:36:38.841379+0000","last_unstale":"2024-04-25T00:36:38.845710+0000","last_undegraded":"2024-04-25T00:36:38.845710+0000","last_fullsized":"2024-04-25T00:36:38.845710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:54.615183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.824175+0000","last_change":"2024-04-25T00:36:38.824175+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.824175+0000","last_undegraded":"2024-04-25T00:36:38.824175+0000","last_fullsized":"2024-04-25T00:36:38.824175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846934+0000","last_change":"2024-04-25T00:36:38.836018+0000","last_active":"2024-04-25T00:36:38.846934+0000","last_peered":"2024-04-25T00:36:38.846934+0000","last_clean":"2024-04-25T00:36:38.846934+0000","last_became_active":"2024-04-25T00:36:38.835209+0000","last_became_peered":"2024-04-25T00:36:38.835209+0000","last_unstale":"2024-04-25T00:36:38.846934+0000","last_undegraded":"2024-04-25T00:36:38.846934+0000","last_fullsized":"2024-04-25T00:36:38.846934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:34.231936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.821618+0000","last_change":"2024-04-25T00:36:38.821618+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.821618+0000","last_undegraded":"2024-04-25T00:36:38.821618+0000","last_fullsized":"2024-04-25T00:36:38.821618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.829415+0000","last_change":"2024-04-25T00:36:38.828383+0000","last_active":"2024-04-25T00:36:38.829415+0000","last_peered":"2024-04-25T00:36:38.829415+0000","last_clean":"2024-04-25T00:36:38.829415+0000","last_became_active":"2024-04-25T00:36:38.827081+0000","last_became_peered":"2024-04-25T00:36:38.827081+0000","last_unstale":"2024-04-25T00:36:38.829415+0000","last_undegraded":"2024-04-25T00:36:38.829415+0000","last_fullsized":"2024-04-25T00:36:38.829415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:49.597505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.820427+0000","last_change":"2024-04-25T00:36:38.820427+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.820427+0000","last_undegraded":"2024-04-25T00:36:38.820427+0000","last_fullsized":"2024-04-25T00:36:38.820427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.827222+0000","last_change":"2024-04-25T00:36:38.820892+0000","last_active":"2024-04-25T00:36:38.827222+0000","last_peered":"2024-04-25T00:36:38.827222+0000","last_clean":"2024-04-25T00:36:38.827222+0000","last_became_active":"2024-04-25T00:36:38.820603+0000","last_became_peered":"2024-04-25T00:36:38.820603+0000","last_unstale":"2024-04-25T00:36:38.827222+0000","last_undegraded":"2024-04-25T00:36:38.827222+0000","last_fullsized":"2024-04-25T00:36:38.827222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:04.183502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811231+0000","last_change":"2024-04-25T00:36:38.811231+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811231+0000","last_undegraded":"2024-04-25T00:36:38.811231+0000","last_fullsized":"2024-04-25T00:36:38.811231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.847018+0000","last_change":"2024-04-25T00:36:38.837177+0000","last_active":"2024-04-25T00:36:38.847018+0000","last_peered":"2024-04-25T00:36:38.847018+0000","last_clean":"2024-04-25T00:36:38.847018+0000","last_became_active":"2024-04-25T00:36:38.836519+0000","last_became_peered":"2024-04-25T00:36:38.836519+0000","last_unstale":"2024-04-25T00:36:38.847018+0000","last_undegraded":"2024-04-25T00:36:38.847018+0000","last_fullsized":"2024-04-25T00:36:38.847018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:48.763968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816742+0000","last_change":"2024-04-25T00:36:38.816742+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816742+0000","last_undegraded":"2024-04-25T00:36:38.816742+0000","last_fullsized":"2024-04-25T00:36:38.816742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834797+0000","last_change":"2024-04-25T00:36:38.832260+0000","last_active":"2024-04-25T00:36:38.834797+0000","last_peered":"2024-04-25T00:36:38.834797+0000","last_clean":"2024-04-25T00:36:38.834797+0000","last_became_active":"2024-04-25T00:36:38.829378+0000","last_became_peered":"2024-04-25T00:36:38.829378+0000","last_unstale":"2024-04-25T00:36:38.834797+0000","last_undegraded":"2024-04-25T00:36:38.834797+0000","last_fullsized":"2024-04-25T00:36:38.834797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:58.973792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.820408+0000","last_change":"2024-04-25T00:36:38.820408+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.820408+0000","last_undegraded":"2024-04-25T00:36:38.820408+0000","last_fullsized":"2024-04-25T00:36:38.820408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.838358+0000","last_change":"2024-04-25T00:36:38.837498+0000","last_active":"2024-04-25T00:36:38.838358+0000","last_peered":"2024-04-25T00:36:38.838358+0000","last_clean":"2024-04-25T00:36:38.838358+0000","last_became_active":"2024-04-25T00:36:38.836899+0000","last_became_peered":"2024-04-25T00:36:38.836899+0000","last_unstale":"2024-04-25T00:36:38.838358+0000","last_undegraded":"2024-04-25T00:36:38.838358+0000","last_fullsized":"2024-04-25T00:36:38.838358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:25.603144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814334+0000","last_change":"2024-04-25T00:36:38.814334+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814334+0000","last_undegraded":"2024-04-25T00:36:38.814334+0000","last_fullsized":"2024-04-25T00:36:38.814334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.838637+0000","last_change":"2024-04-25T00:36:38.837750+0000","last_active":"2024-04-25T00:36:38.838637+0000","last_peered":"2024-04-25T00:36:38.838637+0000","last_clean":"2024-04-25T00:36:38.838637+0000","last_became_active":"2024-04-25T00:36:38.836970+0000","last_became_peered":"2024-04-25T00:36:38.836970+0000","last_unstale":"2024-04-25T00:36:38.838637+0000","last_undegraded":"2024-04-25T00:36:38.838637+0000","last_fullsized":"2024-04-25T00:36:38.838637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:27.980427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816222+0000","last_change":"2024-04-25T00:36:38.816222+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816222+0000","last_undegraded":"2024-04-25T00:36:38.816222+0000","last_fullsized":"2024-04-25T00:36:38.816222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846512+0000","last_change":"2024-04-25T00:36:38.842829+0000","last_active":"2024-04-25T00:36:38.846512+0000","last_peered":"2024-04-25T00:36:38.846512+0000","last_clean":"2024-04-25T00:36:38.846512+0000","last_became_active":"2024-04-25T00:36:38.838997+0000","last_became_peered":"2024-04-25T00:36:38.838997+0000","last_unstale":"2024-04-25T00:36:38.846512+0000","last_undegraded":"2024-04-25T00:36:38.846512+0000","last_fullsized":"2024-04-25T00:36:38.846512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:39.487660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.826636+0000","last_change":"2024-04-25T00:36:38.826636+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.826636+0000","last_undegraded":"2024-04-25T00:36:38.826636+0000","last_fullsized":"2024-04-25T00:36:38.826636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845605+0000","last_change":"2024-04-25T00:36:38.840680+0000","last_active":"2024-04-25T00:36:38.845605+0000","last_peered":"2024-04-25T00:36:38.845605+0000","last_clean":"2024-04-25T00:36:38.845605+0000","last_became_active":"2024-04-25T00:36:38.839742+0000","last_became_peered":"2024-04-25T00:36:38.839742+0000","last_unstale":"2024-04-25T00:36:38.845605+0000","last_undegraded":"2024-04-25T00:36:38.845605+0000","last_fullsized":"2024-04-25T00:36:38.845605+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:48.763968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.810011+0000","last_change":"2024-04-25T00:36:38.810011+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.810011+0000","last_undegraded":"2024-04-25T00:36:38.810011+0000","last_fullsized":"2024-04-25T00:36:38.810011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846500+0000","last_change":"2024-04-25T00:36:38.842786+0000","last_active":"2024-04-25T00:36:38.846500+0000","last_peered":"2024-04-25T00:36:38.846500+0000","last_clean":"2024-04-25T00:36:38.846500+0000","last_became_active":"2024-04-25T00:36:38.842159+0000","last_became_peered":"2024-04-25T00:36:38.842159+0000","last_unstale":"2024-04-25T00:36:38.846500+0000","last_undegraded":"2024-04-25T00:36:38.846500+0000","last_fullsized":"2024-04-25T00:36:38.846500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:36.318064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.817507+0000","last_change":"2024-04-25T00:36:38.817507+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.817507+0000","last_undegraded":"2024-04-25T00:36:38.817507+0000","last_fullsized":"2024-04-25T00:36:38.817507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.844370+0000","last_change":"2024-04-25T00:36:38.837497+0000","last_active":"2024-04-25T00:36:38.844370+0000","last_peered":"2024-04-25T00:36:38.844370+0000","last_clean":"2024-04-25T00:36:38.844370+0000","last_became_active":"2024-04-25T00:36:38.836746+0000","last_became_peered":"2024-04-25T00:36:38.836746+0000","last_unstale":"2024-04-25T00:36:38.844370+0000","last_undegraded":"2024-04-25T00:36:38.844370+0000","last_fullsized":"2024-04-25T00:36:38.844370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:49.597505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.808277+0000","last_change":"2024-04-25T00:36:38.808277+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.808277+0000","last_undegraded":"2024-04-25T00:36:38.808277+0000","last_fullsized":"2024-04-25T00:36:38.808277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.833351+0000","last_change":"2024-04-25T00:36:38.828622+0000","last_active":"2024-04-25T00:36:38.833351+0000","last_peered":"2024-04-25T00:36:38.833351+0000","last_clean":"2024-04-25T00:36:38.833351+0000","last_became_active":"2024-04-25T00:36:38.828270+0000","last_became_peered":"2024-04-25T00:36:38.828270+0000","last_unstale":"2024-04-25T00:36:38.833351+0000","last_undegraded":"2024-04-25T00:36:38.833351+0000","last_fullsized":"2024-04-25T00:36:38.833351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:33.405161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814505+0000","last_change":"2024-04-25T00:36:38.814505+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814505+0000","last_undegraded":"2024-04-25T00:36:38.814505+0000","last_fullsized":"2024-04-25T00:36:38.814505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.835128+0000","last_change":"2024-04-25T00:36:38.833289+0000","last_active":"2024-04-25T00:36:38.835128+0000","last_peered":"2024-04-25T00:36:38.835128+0000","last_clean":"2024-04-25T00:36:38.835128+0000","last_became_active":"2024-04-25T00:36:38.832940+0000","last_became_peered":"2024-04-25T00:36:38.832940+0000","last_unstale":"2024-04-25T00:36:38.835128+0000","last_undegraded":"2024-04-25T00:36:38.835128+0000","last_fullsized":"2024-04-25T00:36:38.835128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:15.477777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811606+0000","last_change":"2024-04-25T00:36:38.811606+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811606+0000","last_undegraded":"2024-04-25T00:36:38.811606+0000","last_fullsized":"2024-04-25T00:36:38.811606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.836292+0000","last_change":"2024-04-25T00:36:38.834888+0000","last_active":"2024-04-25T00:36:38.836292+0000","last_peered":"2024-04-25T00:36:38.836292+0000","last_clean":"2024-04-25T00:36:38.836292+0000","last_became_active":"2024-04-25T00:36:38.833935+0000","last_became_peered":"2024-04-25T00:36:38.833935+0000","last_unstale":"2024-04-25T00:36:38.836292+0000","last_undegraded":"2024-04-25T00:36:38.836292+0000","last_fullsized":"2024-04-25T00:36:38.836292+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:04.183502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.811323+0000","last_change":"2024-04-25T00:36:38.811323+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.811323+0000","last_undegraded":"2024-04-25T00:36:38.811323+0000","last_fullsized":"2024-04-25T00:36:38.811323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834500+0000","last_change":"2024-04-25T00:36:38.833368+0000","last_active":"2024-04-25T00:36:38.834500+0000","last_peered":"2024-04-25T00:36:38.834500+0000","last_clean":"2024-04-25T00:36:38.834500+0000","last_became_active":"2024-04-25T00:36:38.832870+0000","last_became_peered":"2024-04-25T00:36:38.832870+0000","last_unstale":"2024-04-25T00:36:38.834500+0000","last_undegraded":"2024-04-25T00:36:38.834500+0000","last_fullsized":"2024-04-25T00:36:38.834500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:05.341588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813368+0000","last_change":"2024-04-25T00:36:38.813368+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813368+0000","last_undegraded":"2024-04-25T00:36:38.813368+0000","last_fullsized":"2024-04-25T00:36:38.813368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.847195+0000","last_change":"2024-04-25T00:36:38.845296+0000","last_active":"2024-04-25T00:36:38.847195+0000","last_peered":"2024-04-25T00:36:38.847195+0000","last_clean":"2024-04-25T00:36:38.847195+0000","last_became_active":"2024-04-25T00:36:38.844447+0000","last_became_peered":"2024-04-25T00:36:38.844447+0000","last_unstale":"2024-04-25T00:36:38.847195+0000","last_undegraded":"2024-04-25T00:36:38.847195+0000","last_fullsized":"2024-04-25T00:36:38.847195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:38.147666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816628+0000","last_change":"2024-04-25T00:36:38.816628+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816628+0000","last_undegraded":"2024-04-25T00:36:38.816628+0000","last_fullsized":"2024-04-25T00:36:38.816628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.837861+0000","last_change":"2024-04-25T00:36:38.836218+0000","last_active":"2024-04-25T00:36:38.837861+0000","last_peered":"2024-04-25T00:36:38.837861+0000","last_clean":"2024-04-25T00:36:38.837861+0000","last_became_active":"2024-04-25T00:36:38.835946+0000","last_became_peered":"2024-04-25T00:36:38.835946+0000","last_unstale":"2024-04-25T00:36:38.837861+0000","last_undegraded":"2024-04-25T00:36:38.837861+0000","last_fullsized":"2024-04-25T00:36:38.837861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:48.430422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816425+0000","last_change":"2024-04-25T00:36:38.816425+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816425+0000","last_undegraded":"2024-04-25T00:36:38.816425+0000","last_fullsized":"2024-04-25T00:36:38.816425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.838313+0000","last_change":"2024-04-25T00:36:38.836980+0000","last_active":"2024-04-25T00:36:38.838313+0000","last_peered":"2024-04-25T00:36:38.838313+0000","last_clean":"2024-04-25T00:36:38.838313+0000","last_became_active":"2024-04-25T00:36:38.836722+0000","last_became_peered":"2024-04-25T00:36:38.836722+0000","last_unstale":"2024-04-25T00:36:38.838313+0000","last_undegraded":"2024-04-25T00:36:38.838313+0000","last_fullsized":"2024-04-25T00:36:38.838313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:29.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813225+0000","last_change":"2024-04-25T00:36:38.813225+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813225+0000","last_undegraded":"2024-04-25T00:36:38.813225+0000","last_fullsized":"2024-04-25T00:36:38.813225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.829873+0000","last_change":"2024-04-25T00:36:38.827338+0000","last_active":"2024-04-25T00:36:38.829873+0000","last_peered":"2024-04-25T00:36:38.829873+0000","last_clean":"2024-04-25T00:36:38.829873+0000","last_became_active":"2024-04-25T00:36:38.826884+0000","last_became_peered":"2024-04-25T00:36:38.826884+0000","last_unstale":"2024-04-25T00:36:38.829873+0000","last_undegraded":"2024-04-25T00:36:38.829873+0000","last_fullsized":"2024-04-25T00:36:38.829873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:10.228966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.828168+0000","last_change":"2024-04-25T00:36:38.828168+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.828168+0000","last_undegraded":"2024-04-25T00:36:38.828168+0000","last_fullsized":"2024-04-25T00:36:38.828168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834354+0000","last_change":"2024-04-25T00:36:38.829621+0000","last_active":"2024-04-25T00:36:38.834354+0000","last_peered":"2024-04-25T00:36:38.834354+0000","last_clean":"2024-04-25T00:36:38.834354+0000","last_became_active":"2024-04-25T00:36:38.827960+0000","last_became_peered":"2024-04-25T00:36:38.827960+0000","last_unstale":"2024-04-25T00:36:38.834354+0000","last_undegraded":"2024-04-25T00:36:38.834354+0000","last_fullsized":"2024-04-25T00:36:38.834354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:56.562023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.810472+0000","last_change":"2024-04-25T00:36:38.810472+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.810472+0000","last_undegraded":"2024-04-25T00:36:38.810472+0000","last_fullsized":"2024-04-25T00:36:38.810472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.827971+0000","last_change":"2024-04-25T00:36:38.823571+0000","last_active":"2024-04-25T00:36:38.827971+0000","last_peered":"2024-04-25T00:36:38.827971+0000","last_clean":"2024-04-25T00:36:38.827971+0000","last_became_active":"2024-04-25T00:36:38.823275+0000","last_became_peered":"2024-04-25T00:36:38.823275+0000","last_unstale":"2024-04-25T00:36:38.827971+0000","last_undegraded":"2024-04-25T00:36:38.827971+0000","last_fullsized":"2024-04-25T00:36:38.827971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:51.207453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.828686+0000","last_change":"2024-04-25T00:36:38.828686+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.828686+0000","last_undegraded":"2024-04-25T00:36:38.828686+0000","last_fullsized":"2024-04-25T00:36:38.828686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.829729+0000","last_change":"2024-04-25T00:36:38.827233+0000","last_active":"2024-04-25T00:36:38.829729+0000","last_peered":"2024-04-25T00:36:38.829729+0000","last_clean":"2024-04-25T00:36:38.829729+0000","last_became_active":"2024-04-25T00:36:38.826902+0000","last_became_peered":"2024-04-25T00:36:38.826902+0000","last_unstale":"2024-04-25T00:36:38.829729+0000","last_undegraded":"2024-04-25T00:36:38.829729+0000","last_fullsized":"2024-04-25T00:36:38.829729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:20.855968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818674+0000","last_change":"2024-04-25T00:36:38.818674+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818674+0000","last_undegraded":"2024-04-25T00:36:38.818674+0000","last_fullsized":"2024-04-25T00:36:38.818674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.839320+0000","last_change":"2024-04-25T00:36:38.838509+0000","last_active":"2024-04-25T00:36:38.839320+0000","last_peered":"2024-04-25T00:36:38.839320+0000","last_clean":"2024-04-25T00:36:38.839320+0000","last_became_active":"2024-04-25T00:36:38.837928+0000","last_became_peered":"2024-04-25T00:36:38.837928+0000","last_unstale":"2024-04-25T00:36:38.839320+0000","last_undegraded":"2024-04-25T00:36:38.839320+0000","last_fullsized":"2024-04-25T00:36:38.839320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:06.637343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.812126+0000","last_change":"2024-04-25T00:36:38.812126+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.812126+0000","last_undegraded":"2024-04-25T00:36:38.812126+0000","last_fullsized":"2024-04-25T00:36:38.812126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.833501+0000","last_change":"2024-04-25T00:36:38.828691+0000","last_active":"2024-04-25T00:36:38.833501+0000","last_peered":"2024-04-25T00:36:38.833501+0000","last_clean":"2024-04-25T00:36:38.833501+0000","last_became_active":"2024-04-25T00:36:38.828380+0000","last_became_peered":"2024-04-25T00:36:38.828380+0000","last_unstale":"2024-04-25T00:36:38.833501+0000","last_undegraded":"2024-04-25T00:36:38.833501+0000","last_fullsized":"2024-04-25T00:36:38.833501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:22.410800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.816547+0000","last_change":"2024-04-25T00:36:38.816547+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.816547+0000","last_undegraded":"2024-04-25T00:36:38.816547+0000","last_fullsized":"2024-04-25T00:36:38.816547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845767+0000","last_change":"2024-04-25T00:36:38.841467+0000","last_active":"2024-04-25T00:36:38.845767+0000","last_peered":"2024-04-25T00:36:38.845767+0000","last_clean":"2024-04-25T00:36:38.845767+0000","last_became_active":"2024-04-25T00:36:38.840182+0000","last_became_peered":"2024-04-25T00:36:38.840182+0000","last_unstale":"2024-04-25T00:36:38.845767+0000","last_undegraded":"2024-04-25T00:36:38.845767+0000","last_fullsized":"2024-04-25T00:36:38.845767+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:53.012076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.815364+0000","last_change":"2024-04-25T00:36:38.815364+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.815364+0000","last_undegraded":"2024-04-25T00:36:38.815364+0000","last_fullsized":"2024-04-25T00:36:38.815364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.845638+0000","last_change":"2024-04-25T00:36:38.841047+0000","last_active":"2024-04-25T00:36:38.845638+0000","last_peered":"2024-04-25T00:36:38.845638+0000","last_clean":"2024-04-25T00:36:38.845638+0000","last_became_active":"2024-04-25T00:36:38.839481+0000","last_became_peered":"2024-04-25T00:36:38.839481+0000","last_unstale":"2024-04-25T00:36:38.845638+0000","last_undegraded":"2024-04-25T00:36:38.845638+0000","last_fullsized":"2024-04-25T00:36:38.845638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:11.985160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.825951+0000","last_change":"2024-04-25T00:36:38.825951+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.825951+0000","last_undegraded":"2024-04-25T00:36:38.825951+0000","last_fullsized":"2024-04-25T00:36:38.825951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.826182+0000","last_change":"2024-04-25T00:36:38.819664+0000","last_active":"2024-04-25T00:36:38.826182+0000","last_peered":"2024-04-25T00:36:38.826182+0000","last_clean":"2024-04-25T00:36:38.826182+0000","last_became_active":"2024-04-25T00:36:38.819378+0000","last_became_peered":"2024-04-25T00:36:38.819378+0000","last_unstale":"2024-04-25T00:36:38.826182+0000","last_undegraded":"2024-04-25T00:36:38.826182+0000","last_fullsized":"2024-04-25T00:36:38.826182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:52.487791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.807503+0000","last_change":"2024-04-25T00:36:38.807503+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.807503+0000","last_undegraded":"2024-04-25T00:36:38.807503+0000","last_fullsized":"2024-04-25T00:36:38.807503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.834734+0000","last_change":"2024-04-25T00:36:38.830646+0000","last_active":"2024-04-25T00:36:38.834734+0000","last_peered":"2024-04-25T00:36:38.834734+0000","last_clean":"2024-04-25T00:36:38.834734+0000","last_became_active":"2024-04-25T00:36:38.829932+0000","last_became_peered":"2024-04-25T00:36:38.829932+0000","last_unstale":"2024-04-25T00:36:38.834734+0000","last_undegraded":"2024-04-25T00:36:38.834734+0000","last_fullsized":"2024-04-25T00:36:38.834734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:57.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.818106+0000","last_change":"2024-04-25T00:36:38.818106+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.818106+0000","last_undegraded":"2024-04-25T00:36:38.818106+0000","last_fullsized":"2024-04-25T00:36:38.818106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846554+0000","last_change":"2024-04-25T00:36:38.840716+0000","last_active":"2024-04-25T00:36:38.846554+0000","last_peered":"2024-04-25T00:36:38.846554+0000","last_clean":"2024-04-25T00:36:38.846554+0000","last_became_active":"2024-04-25T00:36:38.837997+0000","last_became_peered":"2024-04-25T00:36:38.837997+0000","last_unstale":"2024-04-25T00:36:38.846554+0000","last_undegraded":"2024-04-25T00:36:38.846554+0000","last_fullsized":"2024-04-25T00:36:38.846554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:54.469425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.813359+0000","last_change":"2024-04-25T00:36:38.813359+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.813359+0000","last_undegraded":"2024-04-25T00:36:38.813359+0000","last_fullsized":"2024-04-25T00:36:38.813359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.846920+0000","last_change":"2024-04-25T00:36:38.834405+0000","last_active":"2024-04-25T00:36:38.846920+0000","last_peered":"2024-04-25T00:36:38.846920+0000","last_clean":"2024-04-25T00:36:38.846920+0000","last_became_active":"2024-04-25T00:36:38.833769+0000","last_became_peered":"2024-04-25T00:36:38.833769+0000","last_unstale":"2024-04-25T00:36:38.846920+0000","last_undegraded":"2024-04-25T00:36:38.846920+0000","last_fullsized":"2024-04-25T00:36:38.846920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:58.559610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-04-25T00:36:38.814258+0000","last_change":"2024-04-25T00:36:38.814258+0000","last_active":"2024-04-25T00:36:38.792404+0000","last_peered":"2024-04-25T00:36:38.792404+0000","last_clean":"2024-04-25T00:36:38.792404+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:36:38.814258+0000","last_undegraded":"2024-04-25T00:36:38.814258+0000","last_fullsized":"2024-04-25T00:36:38.814258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.842399+0000","last_change":"2024-04-25T00:36:38.835037+0000","last_active":"2024-04-25T00:36:38.842399+0000","last_peered":"2024-04-25T00:36:38.842399+0000","last_clean":"2024-04-25T00:36:38.842399+0000","last_became_active":"2024-04-25T00:36:38.833930+0000","last_became_peered":"2024-04-25T00:36:38.833930+0000","last_unstale":"2024-04-25T00:36:38.842399+0000","last_undegraded":"2024-04-25T00:36:38.842399+0000","last_fullsized":"2024-04-25T00:36:38.842399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:48.676020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":327,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:36:38.808362+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:36:38.808362+0000","last_peered":"2024-04-25T00:36:38.808362+0000","last_clean":"2024-04-25T00:36:38.808362+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:36:38.808362+0000","last_undegraded":"2024-04-25T00:36:38.808362+0000","last_fullsized":"2024-04-25T00:36:38.808362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36908,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334932,"statfs":{"total":96636764160,"available":96598970368,"internally_reserved":0,"allocated":1601536,"data_stored":1021755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":431387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":425112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1007616,"data_stored":431387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":995328,"data_stored":425112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":995328,"data_stored":425112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1007616,"data_stored":431387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36892,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334948,"statfs":{"total":96636764160,"available":96598986752,"internally_reserved":0,"allocated":1589248,"data_stored":1015480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:36:45.791 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-04-25T00:36:45.788+0000 7fcf16a05640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:36:48.343 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:36:48.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb842d06640 1 Processor -- start 2024-04-25T00:36:48.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb842d06640 1 -- start start 2024-04-25T00:36:48.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb842d06640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb83c15ad20 0x7fb83c0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb842d06640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb83c0d0390 0x7fb83c15b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb842d06640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb83c15bc30 con 0x7fb83c15b640 2024-04-25T00:36:48.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb842d06640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb83c0cb6a0 con 0x7fb83c0d0390 2024-04-25T00:36:48.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb842d06640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb83c15bf10 con 0x7fb83c15ad20 2024-04-25T00:36:48.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb83bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb83c0d0390 0x7fb83c15b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb83bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb83c0d0390 0x7fb83c15b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36540/0 (socket says 172.21.15.29:36540) 2024-04-25T00:36:48.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb83bfff640 1 -- 172.21.15.29:0/1750949776 learned_addr learned my addr 172.21.15.29:0/1750949776 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:48.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb840aa2640 1 --2- 172.21.15.29:0/1750949776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb83c15ad20 0x7fb83c0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb83bfff640 1 -- 172.21.15.29:0/1750949776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb83c15ad20 msgr2=0x7fb83c0cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:48.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb83bfff640 1 --2- 172.21.15.29:0/1750949776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb83c15ad20 0x7fb83c0cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb83bfff640 1 -- 172.21.15.29:0/1750949776 >> v1:172.21.15.157:6789/0 conn(0x7fb83c15b640 legacy=0x7fb83c071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:48.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.476+0000 7fb83bfff640 1 -- 172.21.15.29:0/1750949776 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83c15c1f0 con 0x7fb83c0d0390 2024-04-25T00:36:48.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb83bfff640 1 --2- 172.21.15.29:0/1750949776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb83c0d0390 0x7fb83c15b100 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb830009870 tx=0x7fb83001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21a57af767eeb1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:48.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb83b7fe640 1 -- 172.21.15.29:0/1750949776 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8300aa430 con 0x7fb83c0d0390 2024-04-25T00:36:48.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb83b7fe640 1 -- 172.21.15.29:0/1750949776 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8300af070 con 0x7fb83c0d0390 2024-04-25T00:36:48.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb83b7fe640 1 -- 172.21.15.29:0/1750949776 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8300aa7f0 con 0x7fb83c0d0390 2024-04-25T00:36:48.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb842d06640 1 -- 172.21.15.29:0/1750949776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb83c0d0390 msgr2=0x7fb83c15b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb842d06640 1 --2- 172.21.15.29:0/1750949776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb83c0d0390 0x7fb83c15b100 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb830009870 tx=0x7fb83001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb842d06640 1 -- 172.21.15.29:0/1750949776 shutdown_connections 2024-04-25T00:36:48.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb842d06640 1 --2- 172.21.15.29:0/1750949776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb83c0d0390 0x7fb83c15b100 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb842d06640 1 --2- 172.21.15.29:0/1750949776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb83c15ad20 0x7fb83c0cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb842d06640 1 -- 172.21.15.29:0/1750949776 >> 172.21.15.29:0/1750949776 conn(0x7fb83c0d0c40 msgr2=0x7fb83c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:48.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb842d06640 1 -- 172.21.15.29:0/1750949776 shutdown_connections 2024-04-25T00:36:48.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb842d06640 1 -- 172.21.15.29:0/1750949776 wait complete. 2024-04-25T00:36:48.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb842d06640 1 Processor -- start 2024-04-25T00:36:48.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.480+0000 7fb842d06640 1 -- start start 2024-04-25T00:36:48.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb842d06640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb83c0d0390 0x7fb83c151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb842d06640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb83c15ad20 0x7fb83c0c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb842d06640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb83c15b640 0x7fb83c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb842d06640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c06f530 con 0x7fb83c0d0390 2024-04-25T00:36:48.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb842d06640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c0726a0 con 0x7fb83c15ad20 2024-04-25T00:36:48.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb842d06640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb83c06f6b0 con 0x7fb83c15b640 2024-04-25T00:36:48.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb840aa2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb83c0d0390 0x7fb83c151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb840aa2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb83c0d0390 0x7fb83c151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36554/0 (socket says 172.21.15.29:36554) 2024-04-25T00:36:48.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb840aa2640 1 -- 172.21.15.29:0/660826740 learned_addr learned my addr 172.21.15.29:0/660826740 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:48.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb83bfff640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb83c15ad20 0x7fb83c0c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb8412a3640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb83c15b640 0x7fb83c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb840aa2640 1 -- 172.21.15.29:0/660826740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb83c15b640 msgr2=0x7fb83c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb840aa2640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb83c15b640 0x7fb83c0c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb840aa2640 1 -- 172.21.15.29:0/660826740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb83c15ad20 msgr2=0x7fb83c0c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb840aa2640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb83c15ad20 0x7fb83c0c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb840aa2640 1 -- 172.21.15.29:0/660826740 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83c1f8ba0 con 0x7fb83c0d0390 2024-04-25T00:36:48.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb840aa2640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb83c0d0390 0x7fb83c151120 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00d800 tx=0x7fb82c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:48.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb8397fa640 1 -- 172.21.15.29:0/660826740 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c00b6c0 con 0x7fb83c0d0390 2024-04-25T00:36:48.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb8397fa640 1 -- 172.21.15.29:0/660826740 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb82c019070 con 0x7fb83c0d0390 2024-04-25T00:36:48.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb8397fa640 1 -- 172.21.15.29:0/660826740 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c00ba60 con 0x7fb83c0d0390 2024-04-25T00:36:48.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb842d06640 1 -- 172.21.15.29:0/660826740 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83c1f88c0 con 0x7fb83c0d0390 2024-04-25T00:36:48.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb842d06640 1 -- 172.21.15.29:0/660826740 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83c153390 con 0x7fb83c0d0390 2024-04-25T00:36:48.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb842d06640 1 -- 172.21.15.29:0/660826740 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb83c069270 con 0x7fb83c0d0390 2024-04-25T00:36:48.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb8397fa640 1 -- 172.21.15.29:0/660826740 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fb82c00bc00 con 0x7fb83c0d0390 2024-04-25T00:36:48.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb8397fa640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb828067a10 0x7fb828069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb8397fa640 1 -- 172.21.15.29:0/660826740 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7fb82c0862a0 con 0x7fb83c0d0390 2024-04-25T00:36:48.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.484+0000 7fb83bfff640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb828067a10 0x7fb828069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.488+0000 7fb83bfff640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb828067a10 0x7fb828069ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb8300ae000 tx=0x7fb8300a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:48.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.492+0000 7fb8397fa640 1 -- 172.21.15.29:0/660826740 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb82c056cd0 con 0x7fb83c0d0390 2024-04-25T00:36:48.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.648+0000 7fb842d06640 1 -- 172.21.15.29:0/660826740 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb83c155a30 con 0x7fb828067a10 2024-04-25T00:36:48.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.664+0000 7fb8397fa640 1 -- 172.21.15.29:0/660826740 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252085 (secure 0 0 0) 0x7fb83c155a30 con 0x7fb828067a10 2024-04-25T00:36:48.671 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:48.672 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:36:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.672+0000 7fb842d06640 1 -- 172.21.15.29:0/660826740 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb828067a10 msgr2=0x7fb828069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.672+0000 7fb842d06640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb828067a10 0x7fb828069ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb8300ae000 tx=0x7fb8300a7000 comp rx=0 tx=0).stop 2024-04-25T00:36:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.672+0000 7fb842d06640 1 -- 172.21.15.29:0/660826740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb83c0d0390 msgr2=0x7fb83c151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.672+0000 7fb842d06640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb83c0d0390 0x7fb83c151120 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00d800 tx=0x7fb82c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.672+0000 7fb842d06640 1 -- 172.21.15.29:0/660826740 shutdown_connections 2024-04-25T00:36:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.672+0000 7fb842d06640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb828067a10 0x7fb828069ed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.672+0000 7fb842d06640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb83c15b640 0x7fb83c0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.672+0000 7fb842d06640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb83c15ad20 0x7fb83c0c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.672+0000 7fb842d06640 1 --2- 172.21.15.29:0/660826740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb83c0d0390 0x7fb83c151120 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.672+0000 7fb842d06640 1 -- 172.21.15.29:0/660826740 >> 172.21.15.29:0/660826740 conn(0x7fb83c0d0c40 msgr2=0x7fb83c0c4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.672+0000 7fb842d06640 1 -- 172.21.15.29:0/660826740 shutdown_connections 2024-04-25T00:36:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.672+0000 7fb842d06640 1 -- 172.21.15.29:0/660826740 wait complete. 2024-04-25T00:36:48.691 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-25T00:36:47.319919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283692,"kb_used_data":9288,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754691028,"statfs":{"total":773094113280,"available":772803612672,"internally_reserved":0,"allocated":9510912,"data_stored":4741252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.009151"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.879457+0000","last_change":"2024-04-25T00:36:39.870624+0000","last_active":"2024-04-25T00:36:40.879457+0000","last_peered":"2024-04-25T00:36:40.879457+0000","last_clean":"2024-04-25T00:36:40.879457+0000","last_became_active":"2024-04-25T00:36:39.869668+0000","last_became_peered":"2024-04-25T00:36:39.869668+0000","last_unstale":"2024-04-25T00:36:40.879457+0000","last_undegraded":"2024-04-25T00:36:40.879457+0000","last_fullsized":"2024-04-25T00:36:40.879457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:17.501846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.878259+0000","last_change":"2024-04-25T00:36:39.867180+0000","last_active":"2024-04-25T00:36:40.878259+0000","last_peered":"2024-04-25T00:36:40.878259+0000","last_clean":"2024-04-25T00:36:40.878259+0000","last_became_active":"2024-04-25T00:36:39.866757+0000","last_became_peered":"2024-04-25T00:36:39.866757+0000","last_unstale":"2024-04-25T00:36:40.878259+0000","last_undegraded":"2024-04-25T00:36:40.878259+0000","last_fullsized":"2024-04-25T00:36:40.878259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:49.440924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830391+0000","last_change":"2024-04-25T00:36:39.833676+0000","last_active":"2024-04-25T00:36:40.830391+0000","last_peered":"2024-04-25T00:36:40.830391+0000","last_clean":"2024-04-25T00:36:40.830391+0000","last_became_active":"2024-04-25T00:36:39.833050+0000","last_became_peered":"2024-04-25T00:36:39.833050+0000","last_unstale":"2024-04-25T00:36:40.830391+0000","last_undegraded":"2024-04-25T00:36:40.830391+0000","last_fullsized":"2024-04-25T00:36:40.830391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:11.239468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.878334+0000","last_change":"2024-04-25T00:36:39.840767+0000","last_active":"2024-04-25T00:36:40.878334+0000","last_peered":"2024-04-25T00:36:40.878334+0000","last_clean":"2024-04-25T00:36:40.878334+0000","last_became_active":"2024-04-25T00:36:39.837852+0000","last_became_peered":"2024-04-25T00:36:39.837852+0000","last_unstale":"2024-04-25T00:36:40.878334+0000","last_undegraded":"2024-04-25T00:36:40.878334+0000","last_fullsized":"2024-04-25T00:36:40.878334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:10.136537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.832241+0000","last_change":"2024-04-25T00:36:39.841026+0000","last_active":"2024-04-25T00:36:40.832241+0000","last_peered":"2024-04-25T00:36:40.832241+0000","last_clean":"2024-04-25T00:36:40.832241+0000","last_became_active":"2024-04-25T00:36:39.840475+0000","last_became_peered":"2024-04-25T00:36:39.840475+0000","last_unstale":"2024-04-25T00:36:40.832241+0000","last_undegraded":"2024-04-25T00:36:40.832241+0000","last_fullsized":"2024-04-25T00:36:40.832241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:29.687859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.878071+0000","last_change":"2024-04-25T00:36:39.839712+0000","last_active":"2024-04-25T00:36:40.878071+0000","last_peered":"2024-04-25T00:36:40.878071+0000","last_clean":"2024-04-25T00:36:40.878071+0000","last_became_active":"2024-04-25T00:36:39.839023+0000","last_became_peered":"2024-04-25T00:36:39.839023+0000","last_unstale":"2024-04-25T00:36:40.878071+0000","last_undegraded":"2024-04-25T00:36:40.878071+0000","last_fullsized":"2024-04-25T00:36:40.878071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:01.743410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.876727+0000","last_change":"2024-04-25T00:36:39.840080+0000","last_active":"2024-04-25T00:36:40.876727+0000","last_peered":"2024-04-25T00:36:40.876727+0000","last_clean":"2024-04-25T00:36:40.876727+0000","last_became_active":"2024-04-25T00:36:39.839794+0000","last_became_peered":"2024-04-25T00:36:39.839794+0000","last_unstale":"2024-04-25T00:36:40.876727+0000","last_undegraded":"2024-04-25T00:36:40.876727+0000","last_fullsized":"2024-04-25T00:36:40.876727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:58.233951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877905+0000","last_change":"2024-04-25T00:36:39.841327+0000","last_active":"2024-04-25T00:36:40.877905+0000","last_peered":"2024-04-25T00:36:40.877905+0000","last_clean":"2024-04-25T00:36:40.877905+0000","last_became_active":"2024-04-25T00:36:39.840861+0000","last_became_peered":"2024-04-25T00:36:39.840861+0000","last_unstale":"2024-04-25T00:36:40.877905+0000","last_undegraded":"2024-04-25T00:36:40.877905+0000","last_fullsized":"2024-04-25T00:36:40.877905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:29.687859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.879004+0000","last_change":"2024-04-25T00:36:39.838030+0000","last_active":"2024-04-25T00:36:40.879004+0000","last_peered":"2024-04-25T00:36:40.879004+0000","last_clean":"2024-04-25T00:36:40.879004+0000","last_became_active":"2024-04-25T00:36:39.837411+0000","last_became_peered":"2024-04-25T00:36:39.837411+0000","last_unstale":"2024-04-25T00:36:40.879004+0000","last_undegraded":"2024-04-25T00:36:40.879004+0000","last_fullsized":"2024-04-25T00:36:40.879004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:19.424502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.880264+0000","last_change":"2024-04-25T00:36:39.851262+0000","last_active":"2024-04-25T00:36:40.880264+0000","last_peered":"2024-04-25T00:36:40.880264+0000","last_clean":"2024-04-25T00:36:40.880264+0000","last_became_active":"2024-04-25T00:36:39.850531+0000","last_became_peered":"2024-04-25T00:36:39.850531+0000","last_unstale":"2024-04-25T00:36:40.880264+0000","last_undegraded":"2024-04-25T00:36:40.880264+0000","last_fullsized":"2024-04-25T00:36:40.880264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:13.307372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.878355+0000","last_change":"2024-04-25T00:36:39.829200+0000","last_active":"2024-04-25T00:36:40.878355+0000","last_peered":"2024-04-25T00:36:40.878355+0000","last_clean":"2024-04-25T00:36:40.878355+0000","last_became_active":"2024-04-25T00:36:39.827160+0000","last_became_peered":"2024-04-25T00:36:39.827160+0000","last_unstale":"2024-04-25T00:36:40.878355+0000","last_undegraded":"2024-04-25T00:36:40.878355+0000","last_fullsized":"2024-04-25T00:36:40.878355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:30.157728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.879646+0000","last_change":"2024-04-25T00:36:39.856816+0000","last_active":"2024-04-25T00:36:40.879646+0000","last_peered":"2024-04-25T00:36:40.879646+0000","last_clean":"2024-04-25T00:36:40.879646+0000","last_became_active":"2024-04-25T00:36:39.849414+0000","last_became_peered":"2024-04-25T00:36:39.849414+0000","last_unstale":"2024-04-25T00:36:40.879646+0000","last_undegraded":"2024-04-25T00:36:40.879646+0000","last_fullsized":"2024-04-25T00:36:40.879646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:12.941629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830168+0000","last_change":"2024-04-25T00:36:39.835226+0000","last_active":"2024-04-25T00:36:40.830168+0000","last_peered":"2024-04-25T00:36:40.830168+0000","last_clean":"2024-04-25T00:36:40.830168+0000","last_became_active":"2024-04-25T00:36:39.834217+0000","last_became_peered":"2024-04-25T00:36:39.834217+0000","last_unstale":"2024-04-25T00:36:40.830168+0000","last_undegraded":"2024-04-25T00:36:40.830168+0000","last_fullsized":"2024-04-25T00:36:40.830168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:21.866470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877011+0000","last_change":"2024-04-25T00:36:39.843046+0000","last_active":"2024-04-25T00:36:40.877011+0000","last_peered":"2024-04-25T00:36:40.877011+0000","last_clean":"2024-04-25T00:36:40.877011+0000","last_became_active":"2024-04-25T00:36:39.839904+0000","last_became_peered":"2024-04-25T00:36:39.839904+0000","last_unstale":"2024-04-25T00:36:40.877011+0000","last_undegraded":"2024-04-25T00:36:40.877011+0000","last_fullsized":"2024-04-25T00:36:40.877011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:23.421302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.876174+0000","last_change":"2024-04-25T00:36:39.828007+0000","last_active":"2024-04-25T00:36:40.876174+0000","last_peered":"2024-04-25T00:36:40.876174+0000","last_clean":"2024-04-25T00:36:40.876174+0000","last_became_active":"2024-04-25T00:36:39.827573+0000","last_became_peered":"2024-04-25T00:36:39.827573+0000","last_unstale":"2024-04-25T00:36:40.876174+0000","last_undegraded":"2024-04-25T00:36:40.876174+0000","last_fullsized":"2024-04-25T00:36:40.876174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:52.513559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.831148+0000","last_change":"2024-04-25T00:36:39.863527+0000","last_active":"2024-04-25T00:36:40.831148+0000","last_peered":"2024-04-25T00:36:40.831148+0000","last_clean":"2024-04-25T00:36:40.831148+0000","last_became_active":"2024-04-25T00:36:39.862794+0000","last_became_peered":"2024-04-25T00:36:39.862794+0000","last_unstale":"2024-04-25T00:36:40.831148+0000","last_undegraded":"2024-04-25T00:36:40.831148+0000","last_fullsized":"2024-04-25T00:36:40.831148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:11.242523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.832496+0000","last_change":"2024-04-25T00:36:39.839533+0000","last_active":"2024-04-25T00:36:40.832496+0000","last_peered":"2024-04-25T00:36:40.832496+0000","last_clean":"2024-04-25T00:36:40.832496+0000","last_became_active":"2024-04-25T00:36:39.839014+0000","last_became_peered":"2024-04-25T00:36:39.839014+0000","last_unstale":"2024-04-25T00:36:40.832496+0000","last_undegraded":"2024-04-25T00:36:40.832496+0000","last_fullsized":"2024-04-25T00:36:40.832496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:01.743410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.878063+0000","last_change":"2024-04-25T00:36:39.857840+0000","last_active":"2024-04-25T00:36:40.878063+0000","last_peered":"2024-04-25T00:36:40.878063+0000","last_clean":"2024-04-25T00:36:40.878063+0000","last_became_active":"2024-04-25T00:36:39.857371+0000","last_became_peered":"2024-04-25T00:36:39.857371+0000","last_unstale":"2024-04-25T00:36:40.878063+0000","last_undegraded":"2024-04-25T00:36:40.878063+0000","last_fullsized":"2024-04-25T00:36:40.878063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:09.399289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.831864+0000","last_change":"2024-04-25T00:36:39.847450+0000","last_active":"2024-04-25T00:36:40.831864+0000","last_peered":"2024-04-25T00:36:40.831864+0000","last_clean":"2024-04-25T00:36:40.831864+0000","last_became_active":"2024-04-25T00:36:39.847012+0000","last_became_peered":"2024-04-25T00:36:39.847012+0000","last_unstale":"2024-04-25T00:36:40.831864+0000","last_undegraded":"2024-04-25T00:36:40.831864+0000","last_fullsized":"2024-04-25T00:36:40.831864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:22.664986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877433+0000","last_change":"2024-04-25T00:36:39.848406+0000","last_active":"2024-04-25T00:36:40.877433+0000","last_peered":"2024-04-25T00:36:40.877433+0000","last_clean":"2024-04-25T00:36:40.877433+0000","last_became_active":"2024-04-25T00:36:39.848102+0000","last_became_peered":"2024-04-25T00:36:39.848102+0000","last_unstale":"2024-04-25T00:36:40.877433+0000","last_undegraded":"2024-04-25T00:36:40.877433+0000","last_fullsized":"2024-04-25T00:36:40.877433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:45.245710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877285+0000","last_change":"2024-04-25T00:36:39.848364+0000","last_active":"2024-04-25T00:36:40.877285+0000","last_peered":"2024-04-25T00:36:40.877285+0000","last_clean":"2024-04-25T00:36:40.877285+0000","last_became_active":"2024-04-25T00:36:39.848011+0000","last_became_peered":"2024-04-25T00:36:39.848011+0000","last_unstale":"2024-04-25T00:36:40.877285+0000","last_undegraded":"2024-04-25T00:36:40.877285+0000","last_fullsized":"2024-04-25T00:36:40.877285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:07.647845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877727+0000","last_change":"2024-04-25T00:36:39.845162+0000","last_active":"2024-04-25T00:36:40.877727+0000","last_peered":"2024-04-25T00:36:40.877727+0000","last_clean":"2024-04-25T00:36:40.877727+0000","last_became_active":"2024-04-25T00:36:39.844054+0000","last_became_peered":"2024-04-25T00:36:39.844054+0000","last_unstale":"2024-04-25T00:36:40.877727+0000","last_undegraded":"2024-04-25T00:36:40.877727+0000","last_fullsized":"2024-04-25T00:36:40.877727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:39.158168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.879169+0000","last_change":"2024-04-25T00:36:39.857174+0000","last_active":"2024-04-25T00:36:40.879169+0000","last_peered":"2024-04-25T00:36:40.879169+0000","last_clean":"2024-04-25T00:36:40.879169+0000","last_became_active":"2024-04-25T00:36:39.856209+0000","last_became_peered":"2024-04-25T00:36:39.856209+0000","last_unstale":"2024-04-25T00:36:40.879169+0000","last_undegraded":"2024-04-25T00:36:40.879169+0000","last_fullsized":"2024-04-25T00:36:40.879169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:00.668586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.876691+0000","last_change":"2024-04-25T00:36:39.850191+0000","last_active":"2024-04-25T00:36:40.876691+0000","last_peered":"2024-04-25T00:36:40.876691+0000","last_clean":"2024-04-25T00:36:40.876691+0000","last_became_active":"2024-04-25T00:36:39.849853+0000","last_became_peered":"2024-04-25T00:36:39.849853+0000","last_unstale":"2024-04-25T00:36:40.876691+0000","last_undegraded":"2024-04-25T00:36:40.876691+0000","last_fullsized":"2024-04-25T00:36:40.876691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:59.053960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.878111+0000","last_change":"2024-04-25T00:36:39.868874+0000","last_active":"2024-04-25T00:36:40.878111+0000","last_peered":"2024-04-25T00:36:40.878111+0000","last_clean":"2024-04-25T00:36:40.878111+0000","last_became_active":"2024-04-25T00:36:39.868515+0000","last_became_peered":"2024-04-25T00:36:39.868515+0000","last_unstale":"2024-04-25T00:36:40.878111+0000","last_undegraded":"2024-04-25T00:36:40.878111+0000","last_fullsized":"2024-04-25T00:36:40.878111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:06.352090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.832215+0000","last_change":"2024-04-25T00:36:39.826767+0000","last_active":"2024-04-25T00:36:40.832215+0000","last_peered":"2024-04-25T00:36:40.832215+0000","last_clean":"2024-04-25T00:36:40.832215+0000","last_became_active":"2024-04-25T00:36:39.825843+0000","last_became_peered":"2024-04-25T00:36:39.825843+0000","last_unstale":"2024-04-25T00:36:40.832215+0000","last_undegraded":"2024-04-25T00:36:40.832215+0000","last_fullsized":"2024-04-25T00:36:40.832215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:42.137830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.831985+0000","last_change":"2024-04-25T00:36:39.837720+0000","last_active":"2024-04-25T00:36:40.831985+0000","last_peered":"2024-04-25T00:36:40.831985+0000","last_clean":"2024-04-25T00:36:40.831985+0000","last_became_active":"2024-04-25T00:36:39.837054+0000","last_became_peered":"2024-04-25T00:36:39.837054+0000","last_unstale":"2024-04-25T00:36:40.831985+0000","last_undegraded":"2024-04-25T00:36:40.831985+0000","last_fullsized":"2024-04-25T00:36:40.831985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:54.636427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.878868+0000","last_change":"2024-04-25T00:36:39.856259+0000","last_active":"2024-04-25T00:36:40.878868+0000","last_peered":"2024-04-25T00:36:40.878868+0000","last_clean":"2024-04-25T00:36:40.878868+0000","last_became_active":"2024-04-25T00:36:39.844819+0000","last_became_peered":"2024-04-25T00:36:39.844819+0000","last_unstale":"2024-04-25T00:36:40.878868+0000","last_undegraded":"2024-04-25T00:36:40.878868+0000","last_fullsized":"2024-04-25T00:36:40.878868+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:40.202210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877983+0000","last_change":"2024-04-25T00:36:39.830569+0000","last_active":"2024-04-25T00:36:40.877983+0000","last_peered":"2024-04-25T00:36:40.877983+0000","last_clean":"2024-04-25T00:36:40.877983+0000","last_became_active":"2024-04-25T00:36:39.830142+0000","last_became_peered":"2024-04-25T00:36:39.830142+0000","last_unstale":"2024-04-25T00:36:40.877983+0000","last_undegraded":"2024-04-25T00:36:40.877983+0000","last_fullsized":"2024-04-25T00:36:40.877983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:50.608007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877719+0000","last_change":"2024-04-25T00:36:39.841142+0000","last_active":"2024-04-25T00:36:40.877719+0000","last_peered":"2024-04-25T00:36:40.877719+0000","last_clean":"2024-04-25T00:36:40.877719+0000","last_became_active":"2024-04-25T00:36:39.840405+0000","last_became_peered":"2024-04-25T00:36:39.840405+0000","last_unstale":"2024-04-25T00:36:40.877719+0000","last_undegraded":"2024-04-25T00:36:40.877719+0000","last_fullsized":"2024-04-25T00:36:40.877719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:55.479927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.831704+0000","last_change":"2024-04-25T00:36:39.851159+0000","last_active":"2024-04-25T00:36:40.831704+0000","last_peered":"2024-04-25T00:36:40.831704+0000","last_clean":"2024-04-25T00:36:40.831704+0000","last_became_active":"2024-04-25T00:36:39.850436+0000","last_became_peered":"2024-04-25T00:36:39.850436+0000","last_unstale":"2024-04-25T00:36:40.831704+0000","last_undegraded":"2024-04-25T00:36:40.831704+0000","last_fullsized":"2024-04-25T00:36:40.831704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:59.080995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.878720+0000","last_change":"2024-04-25T00:36:39.849881+0000","last_active":"2024-04-25T00:36:40.878720+0000","last_peered":"2024-04-25T00:36:40.878720+0000","last_clean":"2024-04-25T00:36:40.878720+0000","last_became_active":"2024-04-25T00:36:39.849050+0000","last_became_peered":"2024-04-25T00:36:39.849050+0000","last_unstale":"2024-04-25T00:36:40.878720+0000","last_undegraded":"2024-04-25T00:36:40.878720+0000","last_fullsized":"2024-04-25T00:36:40.878720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:49.686521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.832975+0000","last_change":"2024-04-25T00:36:39.858476+0000","last_active":"2024-04-25T00:36:40.832975+0000","last_peered":"2024-04-25T00:36:40.832975+0000","last_clean":"2024-04-25T00:36:40.832975+0000","last_became_active":"2024-04-25T00:36:39.857865+0000","last_became_peered":"2024-04-25T00:36:39.857865+0000","last_unstale":"2024-04-25T00:36:40.832975+0000","last_undegraded":"2024-04-25T00:36:40.832975+0000","last_fullsized":"2024-04-25T00:36:40.832975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:26.214972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.880099+0000","last_change":"2024-04-25T00:36:38.842146+0000","last_active":"2024-04-25T00:36:40.880099+0000","last_peered":"2024-04-25T00:36:40.880099+0000","last_clean":"2024-04-25T00:36:40.880099+0000","last_became_active":"2024-04-25T00:36:38.841379+0000","last_became_peered":"2024-04-25T00:36:38.841379+0000","last_unstale":"2024-04-25T00:36:40.880099+0000","last_undegraded":"2024-04-25T00:36:40.880099+0000","last_fullsized":"2024-04-25T00:36:40.880099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:16.685327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.879730+0000","last_change":"2024-04-25T00:36:39.869995+0000","last_active":"2024-04-25T00:36:40.879730+0000","last_peered":"2024-04-25T00:36:40.879730+0000","last_clean":"2024-04-25T00:36:40.879730+0000","last_became_active":"2024-04-25T00:36:39.868975+0000","last_became_peered":"2024-04-25T00:36:39.868975+0000","last_unstale":"2024-04-25T00:36:40.879730+0000","last_undegraded":"2024-04-25T00:36:40.879730+0000","last_fullsized":"2024-04-25T00:36:40.879730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:25:39.600896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.832889+0000","last_change":"2024-04-25T00:36:38.836018+0000","last_active":"2024-04-25T00:36:40.832889+0000","last_peered":"2024-04-25T00:36:40.832889+0000","last_clean":"2024-04-25T00:36:40.832889+0000","last_became_active":"2024-04-25T00:36:38.835209+0000","last_became_peered":"2024-04-25T00:36:38.835209+0000","last_unstale":"2024-04-25T00:36:40.832889+0000","last_undegraded":"2024-04-25T00:36:40.832889+0000","last_fullsized":"2024-04-25T00:36:40.832889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:54.469425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.879264+0000","last_change":"2024-04-25T00:36:39.845143+0000","last_active":"2024-04-25T00:36:40.879264+0000","last_peered":"2024-04-25T00:36:40.879264+0000","last_clean":"2024-04-25T00:36:40.879264+0000","last_became_active":"2024-04-25T00:36:39.844403+0000","last_became_peered":"2024-04-25T00:36:39.844403+0000","last_unstale":"2024-04-25T00:36:40.879264+0000","last_undegraded":"2024-04-25T00:36:40.879264+0000","last_fullsized":"2024-04-25T00:36:40.879264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:14.405447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.832110+0000","last_change":"2024-04-25T00:36:38.828383+0000","last_active":"2024-04-25T00:36:40.832110+0000","last_peered":"2024-04-25T00:36:40.832110+0000","last_clean":"2024-04-25T00:36:40.832110+0000","last_became_active":"2024-04-25T00:36:38.827081+0000","last_became_peered":"2024-04-25T00:36:38.827081+0000","last_unstale":"2024-04-25T00:36:40.832110+0000","last_undegraded":"2024-04-25T00:36:40.832110+0000","last_fullsized":"2024-04-25T00:36:40.832110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:28.677357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830762+0000","last_change":"2024-04-25T00:36:39.838456+0000","last_active":"2024-04-25T00:36:40.830762+0000","last_peered":"2024-04-25T00:36:40.830762+0000","last_clean":"2024-04-25T00:36:40.830762+0000","last_became_active":"2024-04-25T00:36:39.837975+0000","last_became_peered":"2024-04-25T00:36:39.837975+0000","last_unstale":"2024-04-25T00:36:40.830762+0000","last_undegraded":"2024-04-25T00:36:40.830762+0000","last_fullsized":"2024-04-25T00:36:40.830762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:41.329534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830396+0000","last_change":"2024-04-25T00:36:38.820892+0000","last_active":"2024-04-25T00:36:40.830396+0000","last_peered":"2024-04-25T00:36:40.830396+0000","last_clean":"2024-04-25T00:36:40.830396+0000","last_became_active":"2024-04-25T00:36:38.820603+0000","last_became_peered":"2024-04-25T00:36:38.820603+0000","last_unstale":"2024-04-25T00:36:40.830396+0000","last_undegraded":"2024-04-25T00:36:40.830396+0000","last_fullsized":"2024-04-25T00:36:40.830396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:53.625925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877844+0000","last_change":"2024-04-25T00:36:39.840774+0000","last_active":"2024-04-25T00:36:40.877844+0000","last_peered":"2024-04-25T00:36:40.877844+0000","last_clean":"2024-04-25T00:36:40.877844+0000","last_became_active":"2024-04-25T00:36:39.839914+0000","last_became_peered":"2024-04-25T00:36:39.839914+0000","last_unstale":"2024-04-25T00:36:40.877844+0000","last_undegraded":"2024-04-25T00:36:40.877844+0000","last_fullsized":"2024-04-25T00:36:40.877844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:53.498293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.832819+0000","last_change":"2024-04-25T00:36:38.837177+0000","last_active":"2024-04-25T00:36:40.832819+0000","last_peered":"2024-04-25T00:36:40.832819+0000","last_clean":"2024-04-25T00:36:40.832819+0000","last_became_active":"2024-04-25T00:36:38.836519+0000","last_became_peered":"2024-04-25T00:36:38.836519+0000","last_unstale":"2024-04-25T00:36:40.832819+0000","last_undegraded":"2024-04-25T00:36:40.832819+0000","last_fullsized":"2024-04-25T00:36:40.832819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:36.318064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.832582+0000","last_change":"2024-04-25T00:36:39.839663+0000","last_active":"2024-04-25T00:36:40.832582+0000","last_peered":"2024-04-25T00:36:40.832582+0000","last_clean":"2024-04-25T00:36:40.832582+0000","last_became_active":"2024-04-25T00:36:39.839037+0000","last_became_peered":"2024-04-25T00:36:39.839037+0000","last_unstale":"2024-04-25T00:36:40.832582+0000","last_undegraded":"2024-04-25T00:36:40.832582+0000","last_fullsized":"2024-04-25T00:36:40.832582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:06.678057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.831594+0000","last_change":"2024-04-25T00:36:38.832260+0000","last_active":"2024-04-25T00:36:40.831594+0000","last_peered":"2024-04-25T00:36:40.831594+0000","last_clean":"2024-04-25T00:36:40.831594+0000","last_became_active":"2024-04-25T00:36:38.829378+0000","last_became_peered":"2024-04-25T00:36:38.829378+0000","last_unstale":"2024-04-25T00:36:40.831594+0000","last_undegraded":"2024-04-25T00:36:40.831594+0000","last_fullsized":"2024-04-25T00:36:40.831594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:53.012076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.829895+0000","last_change":"2024-04-25T00:36:39.834922+0000","last_active":"2024-04-25T00:36:40.829895+0000","last_peered":"2024-04-25T00:36:40.829895+0000","last_clean":"2024-04-25T00:36:40.829895+0000","last_became_active":"2024-04-25T00:36:39.834290+0000","last_became_peered":"2024-04-25T00:36:39.834290+0000","last_unstale":"2024-04-25T00:36:40.829895+0000","last_undegraded":"2024-04-25T00:36:40.829895+0000","last_fullsized":"2024-04-25T00:36:40.829895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:03.476860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830120+0000","last_change":"2024-04-25T00:36:38.837498+0000","last_active":"2024-04-25T00:36:40.830120+0000","last_peered":"2024-04-25T00:36:40.830120+0000","last_clean":"2024-04-25T00:36:40.830120+0000","last_became_active":"2024-04-25T00:36:38.836899+0000","last_became_peered":"2024-04-25T00:36:38.836899+0000","last_unstale":"2024-04-25T00:36:40.830120+0000","last_undegraded":"2024-04-25T00:36:40.830120+0000","last_fullsized":"2024-04-25T00:36:40.830120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:24.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.876333+0000","last_change":"2024-04-25T00:36:39.821290+0000","last_active":"2024-04-25T00:36:40.876333+0000","last_peered":"2024-04-25T00:36:40.876333+0000","last_clean":"2024-04-25T00:36:40.876333+0000","last_became_active":"2024-04-25T00:36:39.820510+0000","last_became_peered":"2024-04-25T00:36:39.820510+0000","last_unstale":"2024-04-25T00:36:40.876333+0000","last_undegraded":"2024-04-25T00:36:40.876333+0000","last_fullsized":"2024-04-25T00:36:40.876333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:06.508997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877402+0000","last_change":"2024-04-25T00:36:38.837750+0000","last_active":"2024-04-25T00:36:40.877402+0000","last_peered":"2024-04-25T00:36:40.877402+0000","last_clean":"2024-04-25T00:36:40.877402+0000","last_became_active":"2024-04-25T00:36:38.836970+0000","last_became_peered":"2024-04-25T00:36:38.836970+0000","last_unstale":"2024-04-25T00:36:40.877402+0000","last_undegraded":"2024-04-25T00:36:40.877402+0000","last_fullsized":"2024-04-25T00:36:40.877402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:21.829484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877787+0000","last_change":"2024-04-25T00:36:39.839319+0000","last_active":"2024-04-25T00:36:40.877787+0000","last_peered":"2024-04-25T00:36:40.877787+0000","last_clean":"2024-04-25T00:36:40.877787+0000","last_became_active":"2024-04-25T00:36:39.838956+0000","last_became_peered":"2024-04-25T00:36:39.838956+0000","last_unstale":"2024-04-25T00:36:40.877787+0000","last_undegraded":"2024-04-25T00:36:40.877787+0000","last_fullsized":"2024-04-25T00:36:40.877787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:43.992239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.880268+0000","last_change":"2024-04-25T00:36:38.842829+0000","last_active":"2024-04-25T00:36:40.880268+0000","last_peered":"2024-04-25T00:36:40.880268+0000","last_clean":"2024-04-25T00:36:40.880268+0000","last_became_active":"2024-04-25T00:36:38.838997+0000","last_became_peered":"2024-04-25T00:36:38.838997+0000","last_unstale":"2024-04-25T00:36:40.880268+0000","last_undegraded":"2024-04-25T00:36:40.880268+0000","last_fullsized":"2024-04-25T00:36:40.880268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:28.677357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.880841+0000","last_change":"2024-04-25T00:36:39.836689+0000","last_active":"2024-04-25T00:36:40.880841+0000","last_peered":"2024-04-25T00:36:40.880841+0000","last_clean":"2024-04-25T00:36:40.880841+0000","last_became_active":"2024-04-25T00:36:39.836040+0000","last_became_peered":"2024-04-25T00:36:39.836040+0000","last_unstale":"2024-04-25T00:36:40.880841+0000","last_undegraded":"2024-04-25T00:36:40.880841+0000","last_fullsized":"2024-04-25T00:36:40.880841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:26.214972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.880399+0000","last_change":"2024-04-25T00:36:38.840680+0000","last_active":"2024-04-25T00:36:40.880399+0000","last_peered":"2024-04-25T00:36:40.880399+0000","last_clean":"2024-04-25T00:36:40.880399+0000","last_became_active":"2024-04-25T00:36:38.839742+0000","last_became_peered":"2024-04-25T00:36:38.839742+0000","last_unstale":"2024-04-25T00:36:40.880399+0000","last_undegraded":"2024-04-25T00:36:40.880399+0000","last_fullsized":"2024-04-25T00:36:40.880399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:05.341588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830480+0000","last_change":"2024-04-25T00:36:39.862489+0000","last_active":"2024-04-25T00:36:40.830480+0000","last_peered":"2024-04-25T00:36:40.830480+0000","last_clean":"2024-04-25T00:36:40.830480+0000","last_became_active":"2024-04-25T00:36:39.862145+0000","last_became_peered":"2024-04-25T00:36:39.862145+0000","last_unstale":"2024-04-25T00:36:40.830480+0000","last_undegraded":"2024-04-25T00:36:40.830480+0000","last_fullsized":"2024-04-25T00:36:40.830480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:40.202210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.879832+0000","last_change":"2024-04-25T00:36:38.842786+0000","last_active":"2024-04-25T00:36:40.879832+0000","last_peered":"2024-04-25T00:36:40.879832+0000","last_clean":"2024-04-25T00:36:40.879832+0000","last_became_active":"2024-04-25T00:36:38.842159+0000","last_became_peered":"2024-04-25T00:36:38.842159+0000","last_unstale":"2024-04-25T00:36:40.879832+0000","last_undegraded":"2024-04-25T00:36:40.879832+0000","last_fullsized":"2024-04-25T00:36:40.879832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:52.487791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.832080+0000","last_change":"2024-04-25T00:36:39.852992+0000","last_active":"2024-04-25T00:36:40.832080+0000","last_peered":"2024-04-25T00:36:40.832080+0000","last_clean":"2024-04-25T00:36:40.832080+0000","last_became_active":"2024-04-25T00:36:39.852073+0000","last_became_peered":"2024-04-25T00:36:39.852073+0000","last_unstale":"2024-04-25T00:36:40.832080+0000","last_undegraded":"2024-04-25T00:36:40.832080+0000","last_fullsized":"2024-04-25T00:36:40.832080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:17.695828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"83'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.879092+0000","last_change":"2024-04-25T00:36:38.837497+0000","last_active":"2024-04-25T00:36:40.879092+0000","last_peered":"2024-04-25T00:36:40.879092+0000","last_clean":"2024-04-25T00:36:40.879092+0000","last_became_active":"2024-04-25T00:36:38.836746+0000","last_became_peered":"2024-04-25T00:36:38.836746+0000","last_unstale":"2024-04-25T00:36:40.879092+0000","last_undegraded":"2024-04-25T00:36:40.879092+0000","last_fullsized":"2024-04-25T00:36:40.879092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:48.430422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.829934+0000","last_change":"2024-04-25T00:36:39.847521+0000","last_active":"2024-04-25T00:36:40.829934+0000","last_peered":"2024-04-25T00:36:40.829934+0000","last_clean":"2024-04-25T00:36:40.829934+0000","last_became_active":"2024-04-25T00:36:39.846806+0000","last_became_peered":"2024-04-25T00:36:39.846806+0000","last_unstale":"2024-04-25T00:36:40.829934+0000","last_undegraded":"2024-04-25T00:36:40.829934+0000","last_fullsized":"2024-04-25T00:36:40.829934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:26.214972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877919+0000","last_change":"2024-04-25T00:36:38.828622+0000","last_active":"2024-04-25T00:36:40.877919+0000","last_peered":"2024-04-25T00:36:40.877919+0000","last_clean":"2024-04-25T00:36:40.877919+0000","last_became_active":"2024-04-25T00:36:38.828270+0000","last_became_peered":"2024-04-25T00:36:38.828270+0000","last_unstale":"2024-04-25T00:36:40.877919+0000","last_undegraded":"2024-04-25T00:36:40.877919+0000","last_fullsized":"2024-04-25T00:36:40.877919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:08.388787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830557+0000","last_change":"2024-04-25T00:36:39.848863+0000","last_active":"2024-04-25T00:36:40.830557+0000","last_peered":"2024-04-25T00:36:40.830557+0000","last_clean":"2024-04-25T00:36:40.830557+0000","last_became_active":"2024-04-25T00:36:39.848449+0000","last_became_peered":"2024-04-25T00:36:39.848449+0000","last_unstale":"2024-04-25T00:36:40.830557+0000","last_undegraded":"2024-04-25T00:36:40.830557+0000","last_fullsized":"2024-04-25T00:36:40.830557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:16.488279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.876473+0000","last_change":"2024-04-25T00:36:38.833289+0000","last_active":"2024-04-25T00:36:40.876473+0000","last_peered":"2024-04-25T00:36:40.876473+0000","last_clean":"2024-04-25T00:36:40.876473+0000","last_became_active":"2024-04-25T00:36:38.832940+0000","last_became_peered":"2024-04-25T00:36:38.832940+0000","last_unstale":"2024-04-25T00:36:40.876473+0000","last_undegraded":"2024-04-25T00:36:40.876473+0000","last_fullsized":"2024-04-25T00:36:40.876473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:02.466359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877474+0000","last_change":"2024-04-25T00:36:39.840269+0000","last_active":"2024-04-25T00:36:40.877474+0000","last_peered":"2024-04-25T00:36:40.877474+0000","last_clean":"2024-04-25T00:36:40.877474+0000","last_became_active":"2024-04-25T00:36:39.839566+0000","last_became_peered":"2024-04-25T00:36:39.839566+0000","last_unstale":"2024-04-25T00:36:40.877474+0000","last_undegraded":"2024-04-25T00:36:40.877474+0000","last_fullsized":"2024-04-25T00:36:40.877474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:37.328566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.878408+0000","last_change":"2024-04-25T00:36:38.834888+0000","last_active":"2024-04-25T00:36:40.878408+0000","last_peered":"2024-04-25T00:36:40.878408+0000","last_clean":"2024-04-25T00:36:40.878408+0000","last_became_active":"2024-04-25T00:36:38.833935+0000","last_became_peered":"2024-04-25T00:36:38.833935+0000","last_unstale":"2024-04-25T00:36:40.878408+0000","last_undegraded":"2024-04-25T00:36:40.878408+0000","last_fullsized":"2024-04-25T00:36:40.878408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:29.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877552+0000","last_change":"2024-04-25T00:36:39.853926+0000","last_active":"2024-04-25T00:36:40.877552+0000","last_peered":"2024-04-25T00:36:40.877552+0000","last_clean":"2024-04-25T00:36:40.877552+0000","last_became_active":"2024-04-25T00:36:39.853592+0000","last_became_peered":"2024-04-25T00:36:39.853592+0000","last_unstale":"2024-04-25T00:36:40.877552+0000","last_undegraded":"2024-04-25T00:36:40.877552+0000","last_fullsized":"2024-04-25T00:36:40.877552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:40.498162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"83'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.829956+0000","last_change":"2024-04-25T00:36:38.833368+0000","last_active":"2024-04-25T00:36:40.829956+0000","last_peered":"2024-04-25T00:36:40.829956+0000","last_clean":"2024-04-25T00:36:40.829956+0000","last_became_active":"2024-04-25T00:36:38.832870+0000","last_became_peered":"2024-04-25T00:36:38.832870+0000","last_unstale":"2024-04-25T00:36:40.829956+0000","last_undegraded":"2024-04-25T00:36:40.829956+0000","last_fullsized":"2024-04-25T00:36:40.829956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:24.770464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.832610+0000","last_change":"2024-04-25T00:36:39.825403+0000","last_active":"2024-04-25T00:36:40.832610+0000","last_peered":"2024-04-25T00:36:40.832610+0000","last_clean":"2024-04-25T00:36:40.832610+0000","last_became_active":"2024-04-25T00:36:39.824589+0000","last_became_peered":"2024-04-25T00:36:39.824589+0000","last_unstale":"2024-04-25T00:36:40.832610+0000","last_undegraded":"2024-04-25T00:36:40.832610+0000","last_fullsized":"2024-04-25T00:36:40.832610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:01.618124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.880517+0000","last_change":"2024-04-25T00:36:38.845296+0000","last_active":"2024-04-25T00:36:40.880517+0000","last_peered":"2024-04-25T00:36:40.880517+0000","last_clean":"2024-04-25T00:36:40.880517+0000","last_became_active":"2024-04-25T00:36:38.844447+0000","last_became_peered":"2024-04-25T00:36:38.844447+0000","last_unstale":"2024-04-25T00:36:40.880517+0000","last_undegraded":"2024-04-25T00:36:40.880517+0000","last_fullsized":"2024-04-25T00:36:40.880517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:24.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.879954+0000","last_change":"2024-04-25T00:36:39.870274+0000","last_active":"2024-04-25T00:36:40.879954+0000","last_peered":"2024-04-25T00:36:40.879954+0000","last_clean":"2024-04-25T00:36:40.879954+0000","last_became_active":"2024-04-25T00:36:39.869018+0000","last_became_peered":"2024-04-25T00:36:39.869018+0000","last_unstale":"2024-04-25T00:36:40.879954+0000","last_undegraded":"2024-04-25T00:36:40.879954+0000","last_fullsized":"2024-04-25T00:36:40.879954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:34.189348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830275+0000","last_change":"2024-04-25T00:36:38.836218+0000","last_active":"2024-04-25T00:36:40.830275+0000","last_peered":"2024-04-25T00:36:40.830275+0000","last_clean":"2024-04-25T00:36:40.830275+0000","last_became_active":"2024-04-25T00:36:38.835946+0000","last_became_peered":"2024-04-25T00:36:38.835946+0000","last_unstale":"2024-04-25T00:36:40.830275+0000","last_undegraded":"2024-04-25T00:36:40.830275+0000","last_fullsized":"2024-04-25T00:36:40.830275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:10.232021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.832480+0000","last_change":"2024-04-25T00:36:39.840330+0000","last_active":"2024-04-25T00:36:40.832480+0000","last_peered":"2024-04-25T00:36:40.832480+0000","last_clean":"2024-04-25T00:36:40.832480+0000","last_became_active":"2024-04-25T00:36:39.839539+0000","last_became_peered":"2024-04-25T00:36:39.839539+0000","last_unstale":"2024-04-25T00:36:40.832480+0000","last_undegraded":"2024-04-25T00:36:40.832480+0000","last_fullsized":"2024-04-25T00:36:40.832480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:25.980539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830724+0000","last_change":"2024-04-25T00:36:38.836980+0000","last_active":"2024-04-25T00:36:40.830724+0000","last_peered":"2024-04-25T00:36:40.830724+0000","last_clean":"2024-04-25T00:36:40.830724+0000","last_became_active":"2024-04-25T00:36:38.836722+0000","last_became_peered":"2024-04-25T00:36:38.836722+0000","last_unstale":"2024-04-25T00:36:40.830724+0000","last_undegraded":"2024-04-25T00:36:40.830724+0000","last_fullsized":"2024-04-25T00:36:40.830724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:16.685327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877245+0000","last_change":"2024-04-25T00:36:39.836988+0000","last_active":"2024-04-25T00:36:40.877245+0000","last_peered":"2024-04-25T00:36:40.877245+0000","last_clean":"2024-04-25T00:36:40.877245+0000","last_became_active":"2024-04-25T00:36:39.836662+0000","last_became_peered":"2024-04-25T00:36:39.836662+0000","last_unstale":"2024-04-25T00:36:40.877245+0000","last_undegraded":"2024-04-25T00:36:40.877245+0000","last_fullsized":"2024-04-25T00:36:40.877245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:28.990929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.876766+0000","last_change":"2024-04-25T00:36:38.827338+0000","last_active":"2024-04-25T00:36:40.876766+0000","last_peered":"2024-04-25T00:36:40.876766+0000","last_clean":"2024-04-25T00:36:40.876766+0000","last_became_active":"2024-04-25T00:36:38.826884+0000","last_became_peered":"2024-04-25T00:36:38.826884+0000","last_unstale":"2024-04-25T00:36:40.876766+0000","last_undegraded":"2024-04-25T00:36:40.876766+0000","last_fullsized":"2024-04-25T00:36:40.876766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:40.319032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.878120+0000","last_change":"2024-04-25T00:36:39.836432+0000","last_active":"2024-04-25T00:36:40.878120+0000","last_peered":"2024-04-25T00:36:40.878120+0000","last_clean":"2024-04-25T00:36:40.878120+0000","last_became_active":"2024-04-25T00:36:39.835682+0000","last_became_peered":"2024-04-25T00:36:39.835682+0000","last_unstale":"2024-04-25T00:36:40.878120+0000","last_undegraded":"2024-04-25T00:36:40.878120+0000","last_fullsized":"2024-04-25T00:36:40.878120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:06.678057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877389+0000","last_change":"2024-04-25T00:36:38.829621+0000","last_active":"2024-04-25T00:36:40.877389+0000","last_peered":"2024-04-25T00:36:40.877389+0000","last_clean":"2024-04-25T00:36:40.877389+0000","last_became_active":"2024-04-25T00:36:38.827960+0000","last_became_peered":"2024-04-25T00:36:38.827960+0000","last_unstale":"2024-04-25T00:36:40.877389+0000","last_undegraded":"2024-04-25T00:36:40.877389+0000","last_fullsized":"2024-04-25T00:36:40.877389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:02.114826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.876869+0000","last_change":"2024-04-25T00:36:39.838034+0000","last_active":"2024-04-25T00:36:40.876869+0000","last_peered":"2024-04-25T00:36:40.876869+0000","last_clean":"2024-04-25T00:36:40.876869+0000","last_became_active":"2024-04-25T00:36:39.837675+0000","last_became_peered":"2024-04-25T00:36:39.837675+0000","last_unstale":"2024-04-25T00:36:40.876869+0000","last_undegraded":"2024-04-25T00:36:40.876869+0000","last_fullsized":"2024-04-25T00:36:40.876869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:34.415662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.876881+0000","last_change":"2024-04-25T00:36:38.823571+0000","last_active":"2024-04-25T00:36:40.876881+0000","last_peered":"2024-04-25T00:36:40.876881+0000","last_clean":"2024-04-25T00:36:40.876881+0000","last_became_active":"2024-04-25T00:36:38.823275+0000","last_became_peered":"2024-04-25T00:36:38.823275+0000","last_unstale":"2024-04-25T00:36:40.876881+0000","last_undegraded":"2024-04-25T00:36:40.876881+0000","last_fullsized":"2024-04-25T00:36:40.876881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:43.633117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877915+0000","last_change":"2024-04-25T00:36:39.854426+0000","last_active":"2024-04-25T00:36:40.877915+0000","last_peered":"2024-04-25T00:36:40.877915+0000","last_clean":"2024-04-25T00:36:40.877915+0000","last_became_active":"2024-04-25T00:36:39.853138+0000","last_became_peered":"2024-04-25T00:36:39.853138+0000","last_unstale":"2024-04-25T00:36:40.877915+0000","last_undegraded":"2024-04-25T00:36:40.877915+0000","last_fullsized":"2024-04-25T00:36:40.877915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:25.980539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.876094+0000","last_change":"2024-04-25T00:36:38.827233+0000","last_active":"2024-04-25T00:36:40.876094+0000","last_peered":"2024-04-25T00:36:40.876094+0000","last_clean":"2024-04-25T00:36:40.876094+0000","last_became_active":"2024-04-25T00:36:38.826902+0000","last_became_peered":"2024-04-25T00:36:38.826902+0000","last_unstale":"2024-04-25T00:36:40.876094+0000","last_undegraded":"2024-04-25T00:36:40.876094+0000","last_fullsized":"2024-04-25T00:36:40.876094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:41.127328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.832710+0000","last_change":"2024-04-25T00:36:39.828492+0000","last_active":"2024-04-25T00:36:40.832710+0000","last_peered":"2024-04-25T00:36:40.832710+0000","last_clean":"2024-04-25T00:36:40.832710+0000","last_became_active":"2024-04-25T00:36:39.824670+0000","last_became_peered":"2024-04-25T00:36:39.824670+0000","last_unstale":"2024-04-25T00:36:40.832710+0000","last_undegraded":"2024-04-25T00:36:40.832710+0000","last_fullsized":"2024-04-25T00:36:40.832710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:19.424502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.878119+0000","last_change":"2024-04-25T00:36:38.838509+0000","last_active":"2024-04-25T00:36:40.878119+0000","last_peered":"2024-04-25T00:36:40.878119+0000","last_clean":"2024-04-25T00:36:40.878119+0000","last_became_active":"2024-04-25T00:36:38.837928+0000","last_became_peered":"2024-04-25T00:36:38.837928+0000","last_unstale":"2024-04-25T00:36:40.878119+0000","last_undegraded":"2024-04-25T00:36:40.878119+0000","last_fullsized":"2024-04-25T00:36:40.878119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:44.235208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830597+0000","last_change":"2024-04-25T00:36:39.870487+0000","last_active":"2024-04-25T00:36:40.830597+0000","last_peered":"2024-04-25T00:36:40.830597+0000","last_clean":"2024-04-25T00:36:40.830597+0000","last_became_active":"2024-04-25T00:36:39.869982+0000","last_became_peered":"2024-04-25T00:36:39.869982+0000","last_unstale":"2024-04-25T00:36:40.830597+0000","last_undegraded":"2024-04-25T00:36:40.830597+0000","last_fullsized":"2024-04-25T00:36:40.830597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:09.687711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.878266+0000","last_change":"2024-04-25T00:36:38.828691+0000","last_active":"2024-04-25T00:36:40.878266+0000","last_peered":"2024-04-25T00:36:40.878266+0000","last_clean":"2024-04-25T00:36:40.878266+0000","last_became_active":"2024-04-25T00:36:38.828380+0000","last_became_peered":"2024-04-25T00:36:38.828380+0000","last_unstale":"2024-04-25T00:36:40.878266+0000","last_undegraded":"2024-04-25T00:36:40.878266+0000","last_fullsized":"2024-04-25T00:36:40.878266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:09.126035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.879437+0000","last_change":"2024-04-25T00:36:39.863555+0000","last_active":"2024-04-25T00:36:40.879437+0000","last_peered":"2024-04-25T00:36:40.879437+0000","last_clean":"2024-04-25T00:36:40.879437+0000","last_became_active":"2024-04-25T00:36:39.861855+0000","last_became_peered":"2024-04-25T00:36:39.861855+0000","last_unstale":"2024-04-25T00:36:40.879437+0000","last_undegraded":"2024-04-25T00:36:40.879437+0000","last_fullsized":"2024-04-25T00:36:40.879437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:37.895159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.880347+0000","last_change":"2024-04-25T00:36:38.841467+0000","last_active":"2024-04-25T00:36:40.880347+0000","last_peered":"2024-04-25T00:36:40.880347+0000","last_clean":"2024-04-25T00:36:40.880347+0000","last_became_active":"2024-04-25T00:36:38.840182+0000","last_became_peered":"2024-04-25T00:36:38.840182+0000","last_unstale":"2024-04-25T00:36:40.880347+0000","last_undegraded":"2024-04-25T00:36:40.880347+0000","last_fullsized":"2024-04-25T00:36:40.880347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:04.183502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.832627+0000","last_change":"2024-04-25T00:36:39.847982+0000","last_active":"2024-04-25T00:36:40.832627+0000","last_peered":"2024-04-25T00:36:40.832627+0000","last_clean":"2024-04-25T00:36:40.832627+0000","last_became_active":"2024-04-25T00:36:39.847361+0000","last_became_peered":"2024-04-25T00:36:39.847361+0000","last_unstale":"2024-04-25T00:36:40.832627+0000","last_undegraded":"2024-04-25T00:36:40.832627+0000","last_fullsized":"2024-04-25T00:36:40.832627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:24:28.494612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.879671+0000","last_change":"2024-04-25T00:36:38.841047+0000","last_active":"2024-04-25T00:36:40.879671+0000","last_peered":"2024-04-25T00:36:40.879671+0000","last_clean":"2024-04-25T00:36:40.879671+0000","last_became_active":"2024-04-25T00:36:38.839481+0000","last_became_peered":"2024-04-25T00:36:38.839481+0000","last_unstale":"2024-04-25T00:36:40.879671+0000","last_undegraded":"2024-04-25T00:36:40.879671+0000","last_fullsized":"2024-04-25T00:36:40.879671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:49.597505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.878434+0000","last_change":"2024-04-25T00:36:39.834327+0000","last_active":"2024-04-25T00:36:40.878434+0000","last_peered":"2024-04-25T00:36:40.878434+0000","last_clean":"2024-04-25T00:36:40.878434+0000","last_became_active":"2024-04-25T00:36:39.833563+0000","last_became_peered":"2024-04-25T00:36:39.833563+0000","last_unstale":"2024-04-25T00:36:40.878434+0000","last_undegraded":"2024-04-25T00:36:40.878434+0000","last_fullsized":"2024-04-25T00:36:40.878434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:24:28.494612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830827+0000","last_change":"2024-04-25T00:36:38.819664+0000","last_active":"2024-04-25T00:36:40.830827+0000","last_peered":"2024-04-25T00:36:40.830827+0000","last_clean":"2024-04-25T00:36:40.830827+0000","last_became_active":"2024-04-25T00:36:38.819378+0000","last_became_peered":"2024-04-25T00:36:38.819378+0000","last_unstale":"2024-04-25T00:36:40.830827+0000","last_undegraded":"2024-04-25T00:36:40.830827+0000","last_fullsized":"2024-04-25T00:36:40.830827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:58.070493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830424+0000","last_change":"2024-04-25T00:36:39.847373+0000","last_active":"2024-04-25T00:36:40.830424+0000","last_peered":"2024-04-25T00:36:40.830424+0000","last_clean":"2024-04-25T00:36:40.830424+0000","last_became_active":"2024-04-25T00:36:39.846890+0000","last_became_peered":"2024-04-25T00:36:39.846890+0000","last_unstale":"2024-04-25T00:36:40.830424+0000","last_undegraded":"2024-04-25T00:36:40.830424+0000","last_fullsized":"2024-04-25T00:36:40.830424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:49.686521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.831521+0000","last_change":"2024-04-25T00:36:38.830646+0000","last_active":"2024-04-25T00:36:40.831521+0000","last_peered":"2024-04-25T00:36:40.831521+0000","last_clean":"2024-04-25T00:36:40.831521+0000","last_became_active":"2024-04-25T00:36:38.829932+0000","last_became_peered":"2024-04-25T00:36:38.829932+0000","last_unstale":"2024-04-25T00:36:40.831521+0000","last_undegraded":"2024-04-25T00:36:40.831521+0000","last_fullsized":"2024-04-25T00:36:40.831521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:54.615183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.829894+0000","last_change":"2024-04-25T00:36:39.828472+0000","last_active":"2024-04-25T00:36:40.829894+0000","last_peered":"2024-04-25T00:36:40.829894+0000","last_clean":"2024-04-25T00:36:40.829894+0000","last_became_active":"2024-04-25T00:36:39.827739+0000","last_became_peered":"2024-04-25T00:36:39.827739+0000","last_unstale":"2024-04-25T00:36:40.829894+0000","last_undegraded":"2024-04-25T00:36:40.829894+0000","last_fullsized":"2024-04-25T00:36:40.829894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:25.122975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.879353+0000","last_change":"2024-04-25T00:36:38.840716+0000","last_active":"2024-04-25T00:36:40.879353+0000","last_peered":"2024-04-25T00:36:40.879353+0000","last_clean":"2024-04-25T00:36:40.879353+0000","last_became_active":"2024-04-25T00:36:38.837997+0000","last_became_peered":"2024-04-25T00:36:38.837997+0000","last_unstale":"2024-04-25T00:36:40.879353+0000","last_undegraded":"2024-04-25T00:36:40.879353+0000","last_fullsized":"2024-04-25T00:36:40.879353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:29.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.879067+0000","last_change":"2024-04-25T00:36:39.857740+0000","last_active":"2024-04-25T00:36:40.879067+0000","last_peered":"2024-04-25T00:36:40.879067+0000","last_clean":"2024-04-25T00:36:40.879067+0000","last_became_active":"2024-04-25T00:36:39.857358+0000","last_became_peered":"2024-04-25T00:36:39.857358+0000","last_unstale":"2024-04-25T00:36:40.879067+0000","last_undegraded":"2024-04-25T00:36:40.879067+0000","last_fullsized":"2024-04-25T00:36:40.879067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:22.664986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.831083+0000","last_change":"2024-04-25T00:36:38.834405+0000","last_active":"2024-04-25T00:36:40.831083+0000","last_peered":"2024-04-25T00:36:40.831083+0000","last_clean":"2024-04-25T00:36:40.831083+0000","last_became_active":"2024-04-25T00:36:38.833769+0000","last_became_peered":"2024-04-25T00:36:38.833769+0000","last_unstale":"2024-04-25T00:36:40.831083+0000","last_undegraded":"2024-04-25T00:36:40.831083+0000","last_fullsized":"2024-04-25T00:36:40.831083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:11.985160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.877889+0000","last_change":"2024-04-25T00:36:39.826490+0000","last_active":"2024-04-25T00:36:40.877889+0000","last_peered":"2024-04-25T00:36:40.877889+0000","last_clean":"2024-04-25T00:36:40.877889+0000","last_became_active":"2024-04-25T00:36:39.825650+0000","last_became_peered":"2024-04-25T00:36:39.825650+0000","last_unstale":"2024-04-25T00:36:40.877889+0000","last_undegraded":"2024-04-25T00:36:40.877889+0000","last_fullsized":"2024-04-25T00:36:40.877889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:38.792404+0000","last_clean_scrub_stamp":"2024-04-25T00:36:38.792404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:05.194004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830991+0000","last_change":"2024-04-25T00:36:38.835037+0000","last_active":"2024-04-25T00:36:40.830991+0000","last_peered":"2024-04-25T00:36:40.830991+0000","last_clean":"2024-04-25T00:36:40.830991+0000","last_became_active":"2024-04-25T00:36:38.833930+0000","last_became_peered":"2024-04-25T00:36:38.833930+0000","last_unstale":"2024-04-25T00:36:40.830991+0000","last_undegraded":"2024-04-25T00:36:40.830991+0000","last_fullsized":"2024-04-25T00:36:40.830991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:36:37.781903+0000","last_clean_scrub_stamp":"2024-04-25T00:36:37.781903+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:21.654484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":331,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:36:40.830321+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:36:40.830321+0000","last_peered":"2024-04-25T00:36:40.830321+0000","last_clean":"2024-04-25T00:36:40.830321+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:36:40.830321+0000","last_undegraded":"2024-04-25T00:36:40.830321+0000","last_fullsized":"2024-04-25T00:36:40.830321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36932,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334908,"statfs":{"total":96636764160,"available":96598945792,"internally_reserved":0,"allocated":1630208,"data_stored":1034900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":445156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574916,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":446096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335488,"statfs":{"total":96636764160,"available":96599539712,"internally_reserved":0,"allocated":1032192,"data_stored":444462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36360,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335480,"statfs":{"total":96636764160,"available":96599531520,"internally_reserved":0,"allocated":1044480,"data_stored":446074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36352,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335488,"statfs":{"total":96636764160,"available":96599539712,"internally_reserved":0,"allocated":1036288,"data_stored":445044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36360,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335480,"statfs":{"total":96636764160,"available":96599531520,"internally_reserved":0,"allocated":1044480,"data_stored":444656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36932,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334908,"statfs":{"total":96636764160,"available":96598945792,"internally_reserved":0,"allocated":1630208,"data_stored":1034864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:36:48.695 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:36:48.695 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:36:48.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.860+0000 7f1fc3c32640 1 Processor -- start 2024-04-25T00:36:48.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.860+0000 7f1fc3c32640 1 -- start start 2024-04-25T00:36:48.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc3c32640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1fbc15e3c0 0x7f1fbc0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc3c32640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fbc06c530 0x7f1fbc06cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc3c32640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fbc15e7a0 con 0x7f1fbc06d150 2024-04-25T00:36:48.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc3c32640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1fbc06d5c0 con 0x7f1fbc15e3c0 2024-04-25T00:36:48.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc3c32640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1fbc0b4bc0 con 0x7f1fbc06c530 2024-04-25T00:36:48.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc19ce640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1fbc15e3c0 0x7f1fbc0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc11cd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fbc06c530 0x7f1fbc06cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc21cf640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1fbc06d150 0x7f1fbc0ce5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:54012/0 (socket says 172.21.15.29:54012) 2024-04-25T00:36:48.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc21cf640 1 -- 172.21.15.29:0/1634198209 learned_addr learned my addr 172.21.15.29:0/1634198209 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:48.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc11cd640 1 -- 172.21.15.29:0/1634198209 >> v2:172.21.15.157:3300/0 conn(0x7f1fbc15e3c0 msgr2=0x7f1fbc0d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:48.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc11cd640 1 --2- 172.21.15.29:0/1634198209 >> v2:172.21.15.157:3300/0 conn(0x7f1fbc15e3c0 0x7f1fbc0d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc11cd640 1 -- 172.21.15.29:0/1634198209 >> v1:172.21.15.157:6789/0 conn(0x7f1fbc06d150 legacy=0x7f1fbc0ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc11cd640 1 -- 172.21.15.29:0/1634198209 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc0d10f0 con 0x7f1fbc06c530 2024-04-25T00:36:48.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc19ce640 1 --2- 172.21.15.29:0/1634198209 >> v2:172.21.15.157:3300/0 conn(0x7f1fbc15e3c0 0x7f1fbc0d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:36:48.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.864+0000 7f1fc11cd640 1 --2- 172.21.15.29:0/1634198209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fbc06c530 0x7f1fbc06cc10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80097c0 tx=0x7f1fb801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71b4442e1d00fc2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:48.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc09cc640 1 -- 172.21.15.29:0/1634198209 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80b0070 con 0x7f1fbc06c530 2024-04-25T00:36:48.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc09cc640 1 -- 172.21.15.29:0/1634198209 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb80ab600 con 0x7f1fbc06c530 2024-04-25T00:36:48.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc09cc640 1 -- 172.21.15.29:0/1634198209 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80ab9d0 con 0x7f1fbc06c530 2024-04-25T00:36:48.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 -- 172.21.15.29:0/1634198209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fbc06c530 msgr2=0x7f1fbc06cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 --2- 172.21.15.29:0/1634198209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fbc06c530 0x7f1fbc06cc10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80097c0 tx=0x7f1fb801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 -- 172.21.15.29:0/1634198209 shutdown_connections 2024-04-25T00:36:48.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 --2- 172.21.15.29:0/1634198209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fbc06c530 0x7f1fbc06cc10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 --2- 172.21.15.29:0/1634198209 >> v2:172.21.15.157:3300/0 conn(0x7f1fbc15e3c0 0x7f1fbc0d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 -- 172.21.15.29:0/1634198209 >> 172.21.15.29:0/1634198209 conn(0x7f1fbc0d1d70 msgr2=0x7f1fbc0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:48.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 -- 172.21.15.29:0/1634198209 shutdown_connections 2024-04-25T00:36:48.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 -- 172.21.15.29:0/1634198209 wait complete. 2024-04-25T00:36:48.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 Processor -- start 2024-04-25T00:36:48.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 -- start start 2024-04-25T00:36:48.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fbc06c530 0x7f1fbc0c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fbc06d150 0x7f1fbc0b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fbc15e3c0 0x7f1fbc151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc0ce050 con 0x7f1fbc06c530 2024-04-25T00:36:48.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc06c970 con 0x7f1fbc15e3c0 2024-04-25T00:36:48.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc3c32640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1fbc0ce1d0 con 0x7f1fbc06d150 2024-04-25T00:36:48.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc19ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fbc06c530 0x7f1fbc0c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc19ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fbc06c530 0x7f1fbc0c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36566/0 (socket says 172.21.15.29:36566) 2024-04-25T00:36:48.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc19ce640 1 -- 172.21.15.29:0/3620524354 learned_addr learned my addr 172.21.15.29:0/3620524354 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:48.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc11cd640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fbc06d150 0x7f1fbc0b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc21cf640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fbc15e3c0 0x7f1fbc151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc19ce640 1 -- 172.21.15.29:0/3620524354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fbc06d150 msgr2=0x7f1fbc0b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc19ce640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fbc06d150 0x7f1fbc0b55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc19ce640 1 -- 172.21.15.29:0/3620524354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fbc15e3c0 msgr2=0x7f1fbc151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc19ce640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fbc15e3c0 0x7f1fbc151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc19ce640 1 -- 172.21.15.29:0/3620524354 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc0b5ce0 con 0x7f1fbc06c530 2024-04-25T00:36:48.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.868+0000 7f1fc19ce640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fbc06c530 0x7f1fbc0c8200 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000ebe0 tx=0x7f1fb000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:48.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.872+0000 7f1fae7fc640 1 -- 172.21.15.29:0/3620524354 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0017070 con 0x7f1fbc06c530 2024-04-25T00:36:48.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.872+0000 7f1fae7fc640 1 -- 172.21.15.29:0/3620524354 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb000cc30 con 0x7f1fbc06c530 2024-04-25T00:36:48.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.872+0000 7f1fae7fc640 1 -- 172.21.15.29:0/3620524354 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0013470 con 0x7f1fbc06c530 2024-04-25T00:36:48.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.872+0000 7f1fc3c32640 1 -- 172.21.15.29:0/3620524354 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fbc0b5fc0 con 0x7f1fbc06c530 2024-04-25T00:36:48.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.872+0000 7f1fc3c32640 1 -- 172.21.15.29:0/3620524354 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fbc0b6580 con 0x7f1fbc06c530 2024-04-25T00:36:48.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.872+0000 7f1fc3c32640 1 -- 172.21.15.29:0/3620524354 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fbc069270 con 0x7f1fbc06c530 2024-04-25T00:36:48.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.872+0000 7f1fae7fc640 1 -- 172.21.15.29:0/3620524354 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f1fb001a070 con 0x7f1fbc06c530 2024-04-25T00:36:48.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.872+0000 7f1fae7fc640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fa0067a10 0x7f1fa0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.872+0000 7f1fae7fc640 1 -- 172.21.15.29:0/3620524354 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f1fb00860f0 con 0x7f1fbc06c530 2024-04-25T00:36:48.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.872+0000 7f1fc11cd640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fa0067a10 0x7f1fa0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.872+0000 7f1fc11cd640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fa0067a10 0x7f1fa0069ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80ad000 tx=0x7f1fb8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:48.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:48.876+0000 7f1fae7fc640 1 -- 172.21.15.29:0/3620524354 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1fb0056b20 con 0x7f1fbc06c530 2024-04-25T00:36:49.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.096+0000 7f1fc3c32640 1 -- 172.21.15.29:0/3620524354 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1fbc06ccb0 con 0x7f1fbc06c530 2024-04-25T00:36:49.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.100+0000 7f1fae7fc640 1 -- 172.21.15.29:0/3620524354 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f1fb0056420 con 0x7f1fbc06c530 2024-04-25T00:36:49.100 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-04-25T00:36:49.101 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A== 2024-04-25T00:36:49.101 INFO:teuthology.orchestra.run.smithi029.stdout: caps mds = "allow" 2024-04-25T00:36:49.101 INFO:teuthology.orchestra.run.smithi029.stdout: caps mon = "allow r" 2024-04-25T00:36:49.101 INFO:teuthology.orchestra.run.smithi029.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:36:49.101 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-04-25T00:36:49.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.100+0000 7f1fc3c32640 1 -- 172.21.15.29:0/3620524354 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fa0067a10 msgr2=0x7f1fa0069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:49.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.100+0000 7f1fc3c32640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fa0067a10 0x7f1fa0069ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80ad000 tx=0x7f1fb8008040 comp rx=0 tx=0).stop 2024-04-25T00:36:49.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.100+0000 7f1fc3c32640 1 -- 172.21.15.29:0/3620524354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fbc06c530 msgr2=0x7f1fbc0c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:49.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.100+0000 7f1fc3c32640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fbc06c530 0x7f1fbc0c8200 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000ebe0 tx=0x7f1fb000c520 comp rx=0 tx=0).stop 2024-04-25T00:36:49.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.100+0000 7f1fc3c32640 1 -- 172.21.15.29:0/3620524354 shutdown_connections 2024-04-25T00:36:49.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.100+0000 7f1fc3c32640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fa0067a10 0x7f1fa0069ed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.100+0000 7f1fc3c32640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fbc15e3c0 0x7f1fbc151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.100+0000 7f1fc3c32640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fbc06d150 0x7f1fbc0b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.100+0000 7f1fc3c32640 1 --2- 172.21.15.29:0/3620524354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fbc06c530 0x7f1fbc0c8200 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.100+0000 7f1fc3c32640 1 -- 172.21.15.29:0/3620524354 >> 172.21.15.29:0/3620524354 conn(0x7f1fbc0d1d70 msgr2=0x7f1fbc0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:49.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.100+0000 7f1fc3c32640 1 -- 172.21.15.29:0/3620524354 shutdown_connections 2024-04-25T00:36:49.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.100+0000 7f1fc3c32640 1 -- 172.21.15.29:0/3620524354 wait complete. 2024-04-25T00:36:49.118 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:36:49.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff76b9640 1 Processor -- start 2024-04-25T00:36:49.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff76b9640 1 -- start start 2024-04-25T00:36:49.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff76b9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ff015e8d0 0x7f5ff00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:49.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff76b9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ff015f3c0 0x7f5ff015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:49.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff76b9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ff00b3720 con 0x7f5ff00b4780 2024-04-25T00:36:49.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff76b9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5ff00b42f0 con 0x7f5ff015e8d0 2024-04-25T00:36:49.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff76b9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff00b39c0 con 0x7f5ff015f3c0 2024-04-25T00:36:49.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff5455640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ff015e8d0 0x7f5ff00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:49.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff5455640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ff015e8d0 0x7f5ff00cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36582/0 (socket says 172.21.15.29:36582) 2024-04-25T00:36:49.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff5455640 1 -- 172.21.15.29:0/2401500379 learned_addr learned my addr 172.21.15.29:0/2401500379 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:49.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff5c56640 1 --2- 172.21.15.29:0/2401500379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ff015f3c0 0x7f5ff015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:49.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff5455640 1 -- 172.21.15.29:0/2401500379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ff015f3c0 msgr2=0x7f5ff015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:49.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff5455640 1 --2- 172.21.15.29:0/2401500379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ff015f3c0 0x7f5ff015c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff5455640 1 -- 172.21.15.29:0/2401500379 >> v1:172.21.15.157:6789/0 conn(0x7f5ff00b4780 legacy=0x7f5ff015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:49.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.240+0000 7f5ff5455640 1 -- 172.21.15.29:0/2401500379 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff00b3ca0 con 0x7f5ff015e8d0 2024-04-25T00:36:49.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff5455640 1 --2- 172.21.15.29:0/2401500379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff015e8d0 0x7f5ff00cc240 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009870 tx=0x7f5fe001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6950b1503c10e565 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:49.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5feffff640 1 -- 172.21.15.29:0/2401500379 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00ab640 con 0x7f5ff015e8d0 2024-04-25T00:36:49.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5feffff640 1 -- 172.21.15.29:0/2401500379 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe00b0070 con 0x7f5ff015e8d0 2024-04-25T00:36:49.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5feffff640 1 -- 172.21.15.29:0/2401500379 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00aba00 con 0x7f5ff015e8d0 2024-04-25T00:36:49.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2401500379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff015e8d0 msgr2=0x7f5ff00cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:49.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 --2- 172.21.15.29:0/2401500379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff015e8d0 0x7f5ff00cc240 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009870 tx=0x7f5fe001c530 comp rx=0 tx=0).stop 2024-04-25T00:36:49.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2401500379 shutdown_connections 2024-04-25T00:36:49.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 --2- 172.21.15.29:0/2401500379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ff015f3c0 0x7f5ff015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 --2- 172.21.15.29:0/2401500379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff015e8d0 0x7f5ff00cc240 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2401500379 >> 172.21.15.29:0/2401500379 conn(0x7f5ff00b1870 msgr2=0x7f5ff00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:49.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2401500379 shutdown_connections 2024-04-25T00:36:49.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2401500379 wait complete. 2024-04-25T00:36:49.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 Processor -- start 2024-04-25T00:36:49.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 -- start start 2024-04-25T00:36:49.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff00b4780 0x7f5ff015b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:49.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff015e8d0 0x7f5ff013b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:49.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ff015f3c0 0x7f5ff013b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:49.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff015d6a0 con 0x7f5ff015e8d0 2024-04-25T00:36:49.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff015d520 con 0x7f5ff00b4780 2024-04-25T00:36:49.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff76b9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff0167c70 con 0x7f5ff015f3c0 2024-04-25T00:36:49.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff4c54640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff015e8d0 0x7f5ff013b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:49.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff4c54640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff015e8d0 0x7f5ff013b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36592/0 (socket says 172.21.15.29:36592) 2024-04-25T00:36:49.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff4c54640 1 -- 172.21.15.29:0/2870223899 learned_addr learned my addr 172.21.15.29:0/2870223899 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:49.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff5455640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff00b4780 0x7f5ff015b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:49.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff5c56640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ff015f3c0 0x7f5ff013b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:49.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff4c54640 1 -- 172.21.15.29:0/2870223899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ff015f3c0 msgr2=0x7f5ff013b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:49.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff4c54640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ff015f3c0 0x7f5ff013b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff4c54640 1 -- 172.21.15.29:0/2870223899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff00b4780 msgr2=0x7f5ff015b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:49.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff4c54640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff00b4780 0x7f5ff015b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff4c54640 1 -- 172.21.15.29:0/2870223899 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff015b7d0 con 0x7f5ff015e8d0 2024-04-25T00:36:49.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff5455640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff00b4780 0x7f5ff015b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:49.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.244+0000 7f5ff5c56640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ff015f3c0 0x7f5ff013b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:49.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.248+0000 7f5ff4c54640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff015e8d0 0x7f5ff013b2f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c500 tx=0x7f5fe400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:49.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.248+0000 7f5fedffb640 1 -- 172.21.15.29:0/2870223899 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4016020 con 0x7f5ff015e8d0 2024-04-25T00:36:49.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.248+0000 7f5fedffb640 1 -- 172.21.15.29:0/2870223899 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe4005220 con 0x7f5ff015e8d0 2024-04-25T00:36:49.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.248+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2870223899 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff0136240 con 0x7f5ff015e8d0 2024-04-25T00:36:49.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.248+0000 7f5fedffb640 1 -- 172.21.15.29:0/2870223899 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4007030 con 0x7f5ff015e8d0 2024-04-25T00:36:49.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.248+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2870223899 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff0136800 con 0x7f5ff015e8d0 2024-04-25T00:36:49.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.248+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2870223899 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff0069270 con 0x7f5ff015e8d0 2024-04-25T00:36:49.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.248+0000 7f5fedffb640 1 -- 172.21.15.29:0/2870223899 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f5fe40071f0 con 0x7f5ff015e8d0 2024-04-25T00:36:49.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.248+0000 7f5fedffb640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fdc0679e0 0x7f5fdc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:49.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.248+0000 7f5fedffb640 1 -- 172.21.15.29:0/2870223899 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f5fe40865a0 con 0x7f5ff015e8d0 2024-04-25T00:36:49.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.248+0000 7f5ff5455640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fdc0679e0 0x7f5fdc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:49.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.248+0000 7f5ff5455640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fdc0679e0 0x7f5fdc069ea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00af000 tx=0x7f5fe0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:49.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.252+0000 7f5fedffb640 1 -- 172.21.15.29:0/2870223899 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5fe4020020 con 0x7f5ff015e8d0 2024-04-25T00:36:49.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.452+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2870223899 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ff006e7b0 con 0x7f5ff015e8d0 2024-04-25T00:36:49.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.452+0000 7f5fedffb640 1 -- 172.21.15.29:0/2870223899 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5186 (secure 0 0 0) 0x7f5fe4057060 con 0x7f5ff015e8d0 2024-04-25T00:36:49.458 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:36:49.458 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2434273880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2434273880},{"type":"v1","addr":"172.21.15.157:6833","nonce":2434273880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/558648019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":558648019},{"type":"v1","addr":"172.21.15.157:6835","nonce":558648019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":7401,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1764147340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1764147340},{"type":"v1","addr":"172.21.15.29:6833","nonce":1764147340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:36:39.192693+0000","modified":"2024-04-25T00:36:42.218595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7404":{"gid":7404,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/3939243143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3939243143},{"type":"v1","addr":"172.21.15.29:6835","nonce":3939243143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7404,"qdb_cluster":[7404]},"id":6}]} 2024-04-25T00:36:49.459 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 53 2024-04-25T00:36:49.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.456+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2870223899 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fdc0679e0 msgr2=0x7f5fdc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:49.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.456+0000 7f5ff76b9640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fdc0679e0 0x7f5fdc069ea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00af000 tx=0x7f5fe0008040 comp rx=0 tx=0).stop 2024-04-25T00:36:49.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.456+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2870223899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff015e8d0 msgr2=0x7f5ff013b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:49.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.456+0000 7f5ff76b9640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff015e8d0 0x7f5ff013b2f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c500 tx=0x7f5fe400c9c0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.456+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2870223899 shutdown_connections 2024-04-25T00:36:49.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.456+0000 7f5ff76b9640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fdc0679e0 0x7f5fdc069ea0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.456+0000 7f5ff76b9640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ff015f3c0 0x7f5ff013b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.456+0000 7f5ff76b9640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff015e8d0 0x7f5ff013b2f0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.456+0000 7f5ff76b9640 1 --2- 172.21.15.29:0/2870223899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff00b4780 0x7f5ff015b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:49.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.456+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2870223899 >> 172.21.15.29:0/2870223899 conn(0x7f5ff00b1870 msgr2=0x7f5ff006dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:49.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.456+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2870223899 shutdown_connections 2024-04-25T00:36:49.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:49.456+0000 7f5ff76b9640 1 -- 172.21.15.29:0/2870223899 wait complete. 2024-04-25T00:36:49.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:36:39.192693+0000', 'modified': '2024-04-25T00:36:42.218595+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7404}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7404': {'gid': 7404, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/3939243143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 3939243143}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 3939243143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7404, 'qdb_cluster': [7404]} 2024-04-25T00:36:49.878 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:36:49.878 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:36:50.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f95746e5640 1 Processor -- start 2024-04-25T00:36:50.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f95746e5640 1 -- start start 2024-04-25T00:36:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f95746e5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c0d0dc0 0x7f956c0b97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f95746e5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f956c0d1ec0 con 0x7f956c0d0940 2024-04-25T00:36:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f95746e5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f956c0d21a0 con 0x7f956c15e250 2024-04-25T00:36:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f95746e5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f956c0d2ad0 con 0x7f956c0d0dc0 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f9571c80640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f956c0d0940 0x7f956c0b90c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39986/0 (socket says 172.21.15.29:39986) 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f9571c80640 1 -- 172.21.15.29:0/3298046073 learned_addr learned my addr 172.21.15.29:0/3298046073 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f9572c82640 1 --2- 172.21.15.29:0/3298046073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c0d0dc0 0x7f956c0b97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252175234 0 0) 0x7f956c0d1ec0 con 0x7f956c0d0940 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9558003530 con 0x7f956c0d0940 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3219373390 0 0) 0x7f9558003530 con 0x7f956c0d0940 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c0d0dc0 msgr2=0x7f956c0b97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 --2- 172.21.15.29:0/3298046073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c0d0dc0 0x7f956c0b97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 >> v1:172.21.15.157:6789/0 conn(0x7f956c15e250 legacy=0x7f956c0cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956c0d2480 con 0x7f956c0d0940 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f9560001a00 con 0x7f956c0d0940 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 >> v1:172.21.15.29:6789/0 conn(0x7f956c0d0940 legacy=0x7f956c0b90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 --2- 172.21.15.29:0/3298046073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95580039f0 0x7f9558003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 --2- 172.21.15.29:0/3298046073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9558004810 0x7f9558004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 --2- 172.21.15.29:0/3298046073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c0d0dc0 0x7f95580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f956c0d2ad0 con 0x7f9558004810 2024-04-25T00:36:50.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f955800ba80 con 0x7f95580039f0 2024-04-25T00:36:50.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f955800bc80 con 0x7f956c0d0dc0 2024-04-25T00:36:50.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f9571c80640 1 --2- 172.21.15.29:0/3298046073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9558004810 0x7f9558004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f9572c82640 1 --2- 172.21.15.29:0/3298046073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c0d0dc0 0x7f95580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f9572481640 1 --2- 172.21.15.29:0/3298046073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95580039f0 0x7f9558003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f9571c80640 1 -- 172.21.15.29:0/3298046073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c0d0dc0 msgr2=0x7f95580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:50.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f9571c80640 1 --2- 172.21.15.29:0/3298046073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c0d0dc0 0x7f95580051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f9571c80640 1 -- 172.21.15.29:0/3298046073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95580039f0 msgr2=0x7f9558003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:50.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f9571c80640 1 --2- 172.21.15.29:0/3298046073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95580039f0 0x7f9558003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.016+0000 7f9571c80640 1 -- 172.21.15.29:0/3298046073 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9558003530 con 0x7f9558004810 2024-04-25T00:36:50.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.020+0000 7f9571c80640 1 --2- 172.21.15.29:0/3298046073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9558004810 0x7f9558004c90 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9560002240 tx=0x7f956001faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649f7b91b256dc04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:50.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.020+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956000a4a0 con 0x7f9558004810 2024-04-25T00:36:50.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.020+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95600aa6f0 con 0x7f9558004810 2024-04-25T00:36:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.020+0000 7f957147f640 1 -- 172.21.15.29:0/3298046073 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95600aaae0 con 0x7f9558004810 2024-04-25T00:36:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.020+0000 7f95746e5640 1 -- 172.21.15.29:0/3298046073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9558004810 msgr2=0x7f9558004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.020+0000 7f95746e5640 1 --2- 172.21.15.29:0/3298046073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9558004810 0x7f9558004c90 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9560002240 tx=0x7f956001faa0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.020+0000 7f9572c82640 1 -- 172.21.15.29:0/3298046073 reap_dead start 2024-04-25T00:36:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.020+0000 7f95746e5640 1 -- 172.21.15.29:0/3298046073 shutdown_connections 2024-04-25T00:36:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.020+0000 7f95746e5640 1 -- 172.21.15.29:0/3298046073 >> 172.21.15.29:0/3298046073 conn(0x7f956c09d2a0 msgr2=0x7f956c0d3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.020+0000 7f95746e5640 1 -- 172.21.15.29:0/3298046073 shutdown_connections 2024-04-25T00:36:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.020+0000 7f95746e5640 1 -- 172.21.15.29:0/3298046073 wait complete. 2024-04-25T00:36:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f95746e5640 1 Processor -- start 2024-04-25T00:36:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f95746e5640 1 -- start start 2024-04-25T00:36:50.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f95746e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9558004810 0x7f956c15b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f95746e5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f956c1516d0 0x7f956c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f95746e5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c0cb020 0x7f956c0cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f95746e5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f956c0d2f60 con 0x7f956c1516d0 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f95746e5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f956c0d1250 con 0x7f9558004810 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f95746e5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f956c0cfe80 con 0x7f956c0cb020 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f9571c80640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f956c1516d0 0x7f956c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f9571c80640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f956c1516d0 0x7f956c0ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36600/0 (socket says 172.21.15.29:36600) 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f9571c80640 1 -- 172.21.15.29:0/674292309 learned_addr learned my addr 172.21.15.29:0/674292309 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f9572c82640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c0cb020 0x7f956c0cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f9572481640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9558004810 0x7f956c15b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f9571c80640 1 -- 172.21.15.29:0/674292309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c0cb020 msgr2=0x7f956c0cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f9571c80640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c0cb020 0x7f956c0cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f9571c80640 1 -- 172.21.15.29:0/674292309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9558004810 msgr2=0x7f956c15b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f9571c80640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9558004810 0x7f956c15b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f9571c80640 1 -- 172.21.15.29:0/674292309 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956c0c91a0 con 0x7f956c1516d0 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f9571c80640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f956c1516d0 0x7f956c0ca240 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f956001f8d0 tx=0x7f9558005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f955effd640 1 -- 172.21.15.29:0/674292309 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956000a480 con 0x7f956c1516d0 2024-04-25T00:36:50.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f955effd640 1 -- 172.21.15.29:0/674292309 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95600c2070 con 0x7f956c1516d0 2024-04-25T00:36:50.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f95746e5640 1 -- 172.21.15.29:0/674292309 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956c0c9480 con 0x7f956c1516d0 2024-04-25T00:36:50.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f955effd640 1 -- 172.21.15.29:0/674292309 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95600aaaa0 con 0x7f956c1516d0 2024-04-25T00:36:50.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f95746e5640 1 -- 172.21.15.29:0/674292309 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956c0c9a40 con 0x7f956c1516d0 2024-04-25T00:36:50.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.024+0000 7f955effd640 1 -- 172.21.15.29:0/674292309 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f95600aac40 con 0x7f956c1516d0 2024-04-25T00:36:50.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f955effd640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9550067a10 0x7f9550069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9572481640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9550067a10 0x7f9550069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f955effd640 1 -- 172.21.15.29:0/674292309 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f9560124350 con 0x7f956c1516d0 2024-04-25T00:36:50.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9572481640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9550067a10 0x7f9550069ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f956c1f7790 tx=0x7f9568009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:50.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 Processor -- start 2024-04-25T00:36:50.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 -- start start 2024-04-25T00:36:50.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4def0 0x5569f4f4e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4e810 0x5569f4f04680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 0x5569f4f04fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5569f4d525e0 con 0x5569f4f04bc0 2024-04-25T00:36:50.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5569f4cd9c50 con 0x5569f4f4e810 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5569f4d4e5a0 con 0x5569f4f4def0 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9572c82640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 0x5569f4f04fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9572c82640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 0x5569f4f04fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36602/0 (socket says 172.21.15.29:36602) 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9572c82640 1 -- 172.21.15.29:0/49028224 learned_addr learned my addr 172.21.15.29:0/49028224 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9572481640 1 --2- 172.21.15.29:0/49028224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4def0 0x5569f4f4e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9571c80640 1 --2- 172.21.15.29:0/49028224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4e810 0x5569f4f04680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9572c82640 1 -- 172.21.15.29:0/49028224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4def0 msgr2=0x5569f4f4e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9572c82640 1 --2- 172.21.15.29:0/49028224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4def0 0x5569f4f4e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9572c82640 1 -- 172.21.15.29:0/49028224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4e810 msgr2=0x5569f4f04680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9572c82640 1 --2- 172.21.15.29:0/49028224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4e810 0x5569f4f04680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9572c82640 1 -- 172.21.15.29:0/49028224 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f4cd7d10 con 0x5569f4f04bc0 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9572481640 1 --2- 172.21.15.29:0/49028224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4def0 0x5569f4f4e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f9572c82640 1 --2- 172.21.15.29:0/49028224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 0x5569f4f04fa0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f956c0b6b00 tx=0x7f9564009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d3a12f62a8946f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f954ffff640 1 -- 172.21.15.29:0/49028224 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956401a030 con 0x5569f4f04bc0 2024-04-25T00:36:50.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f954ffff640 1 -- 172.21.15.29:0/49028224 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9564004020 con 0x5569f4f04bc0 2024-04-25T00:36:50.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f954ffff640 1 -- 172.21.15.29:0/49028224 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95640043d0 con 0x5569f4f04bc0 2024-04-25T00:36:50.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 -- 172.21.15.29:0/49028224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 msgr2=0x5569f4f04fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 --2- 172.21.15.29:0/49028224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 0x5569f4f04fa0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f956c0b6b00 tx=0x7f9564009190 comp rx=0 tx=0).stop 2024-04-25T00:36:50.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 -- 172.21.15.29:0/49028224 shutdown_connections 2024-04-25T00:36:50.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 --2- 172.21.15.29:0/49028224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 0x5569f4f04fa0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 --2- 172.21.15.29:0/49028224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4e810 0x5569f4f04680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 --2- 172.21.15.29:0/49028224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4def0 0x5569f4f4e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 -- 172.21.15.29:0/49028224 >> 172.21.15.29:0/49028224 conn(0x5569f4f00b70 msgr2=0x5569f4f02f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:50.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 -- 172.21.15.29:0/49028224 shutdown_connections 2024-04-25T00:36:50.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 -- 172.21.15.29:0/49028224 wait complete. 2024-04-25T00:36:50.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.028+0000 7f957785e000 1 -- start start 2024-04-25T00:36:50.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 Processor -- start 2024-04-25T00:36:50.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f04bc0 0x5569f4f32f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4def0 0x5569f4f33460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 0x5569f4f339a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5569f4d0c520 con 0x5569f4f4e810 2024-04-25T00:36:50.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5569f4f009f0 con 0x5569f4f04bc0 2024-04-25T00:36:50.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5569f4ceadd0 con 0x5569f4f4def0 2024-04-25T00:36:50.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 10 client.0 _ll_register_callbacks cb 0x5569f4eef5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:50.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9572c82640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 0x5569f4f339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9572c82640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 0x5569f4f339a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36618/0 (socket says 172.21.15.29:36618) 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9572c82640 1 -- 172.21.15.29:0/794950887 learned_addr learned my addr 172.21.15.29:0/794950887 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9571c80640 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4def0 0x5569f4f33460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9572481640 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f04bc0 0x5569f4f32f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9572c82640 1 -- 172.21.15.29:0/794950887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4def0 msgr2=0x5569f4f33460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9572c82640 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4def0 0x5569f4f33460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9572c82640 1 -- 172.21.15.29:0/794950887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f04bc0 msgr2=0x5569f4f32f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9572c82640 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f04bc0 0x5569f4f32f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9572c82640 1 -- 172.21.15.29:0/794950887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f4cea7b0 con 0x5569f4f4e810 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9572481640 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f04bc0 0x5569f4f32f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9572c82640 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 0x5569f4f339a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x5569f4ee4460 tx=0x7f95640046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e4c38de6333a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f954d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f954d7fa640 1 -- 172.21.15.29:0/794950887 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564005bb0 con 0x5569f4f4e810 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f954d7fa640 1 -- 172.21.15.29:0/794950887 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f9564005d50 con 0x5569f4f4e810 2024-04-25T00:36:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 -- 172.21.15.29:0/794950887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9564004020 con 0x5569f4f4e810 2024-04-25T00:36:50.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f954d7fa640 1 -- 172.21.15.29:0/794950887 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f9564031050 con 0x5569f4f4e810 2024-04-25T00:36:50.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 10 client.7575 fetch_fsmap learned FSMap version 53 2024-04-25T00:36:50.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 -- 172.21.15.29:0/794950887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x5569f4cd9ea0 con 0x5569f4f4e810 2024-04-25T00:36:50.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f954d7fa640 1 -- 172.21.15.29:0/794950887 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9564004020 con 0x5569f4f4e810 2024-04-25T00:36:50.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f954d7fa640 1 -- 172.21.15.29:0/794950887 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f9564034020 con 0x5569f4f4e810 2024-04-25T00:36:50.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 10 client.7575 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T00:36:50.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 10 client.7575 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:36:50.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 20 client.7575 populate_metadata read hostname 'smithi029' 2024-04-25T00:36:50.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.29:6834/3939243143,v1:172.21.15.29:6835/3939243143] conn(0x5569f4f5df90 0x5569f4f60380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 4 client.7575 mds_command: new command op to 7404 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:36:50.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 -- 172.21.15.29:0/794950887 --> [v2:172.21.15.29:6834/3939243143,v1:172.21.15.29:6835/3939243143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569f4cedab0 con 0x5569f4f5df90 2024-04-25T00:36:50.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9572481640 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.29:6834/3939243143,v1:172.21.15.29:6835/3939243143] conn(0x5569f4f5df90 0x5569f4f60380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f9572481640 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.29:6834/3939243143,v1:172.21.15.29:6835/3939243143] conn(0x5569f4f5df90 0x5569f4f60380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d508fbc72e9b7a server_cookie=5afcc16ba40b9126 in_seq=0 out_seq=0 2024-04-25T00:36:50.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f954d7fa640 10 client.7575 ms_handle_connect on v2:172.21.15.29:6834/3939243143 2024-04-25T00:36:50.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f954d7fa640 1 -- 172.21.15.29:0/794950887 <== mds.0 v2:172.21.15.29:6834/3939243143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5569f4cedab0 con 0x5569f4f5df90 2024-04-25T00:36:50.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f954d7fa640 10 client.7575 handle_command_reply: tid=0 2024-04-25T00:36:50.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 client.7575 shutdown 2024-04-25T00:36:50.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 -- 172.21.15.29:0/794950887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 msgr2=0x5569f4f339a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 0x5569f4f339a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x5569f4ee4460 tx=0x7f95640046d0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 -- 172.21.15.29:0/794950887 shutdown_connections 2024-04-25T00:36:50.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 0x5569f4f339a0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.29:6834/3939243143,v1:172.21.15.29:6835/3939243143] conn(0x5569f4f5df90 0x5569f4f60380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4def0 0x5569f4f33460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 --2- 172.21.15.29:0/794950887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f04bc0 0x5569f4f32f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f954d7fa640 0 client.7575 ms_handle_reset on v2:172.21.15.29:6834/3939243143 2024-04-25T00:36:50.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 -- 172.21.15.29:0/794950887 >> 172.21.15.29:0/794950887 conn(0x5569f4f00b70 msgr2=0x5569f4f037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:50.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 -- 172.21.15.29:0/794950887 shutdown_connections 2024-04-25T00:36:50.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.032+0000 7f957785e000 1 -- 172.21.15.29:0/794950887 wait complete. 2024-04-25T00:36:50.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.036+0000 7f957785e000 20 client.7575 trim_cache size 0 max 16384 2024-04-25T00:36:50.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f957785e000 1 Processor -- start 2024-04-25T00:36:50.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f957785e000 1 -- start start 2024-04-25T00:36:50.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f957785e000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 0x5569f4fb9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f957785e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4def0 0x5569f4fb9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f957785e000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4e810 0x5569f4fb9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f957785e000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f956c1ffca0 con 0x5569f4f04bc0 2024-04-25T00:36:50.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f957785e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f956c0c83a0 con 0x5569f4f4def0 2024-04-25T00:36:50.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f957785e000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f956c1f4d70 con 0x5569f4f4e810 2024-04-25T00:36:50.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f9572481640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 0x5569f4fb9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f9572481640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 0x5569f4fb9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36622/0 (socket says 172.21.15.29:36622) 2024-04-25T00:36:50.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f9572481640 1 -- 172.21.15.29:0/1121819574 learned_addr learned my addr 172.21.15.29:0/1121819574 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:50.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f9572481640 1 -- 172.21.15.29:0/1121819574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4e810 msgr2=0x5569f4fb9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:50.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f9572481640 1 --2- 172.21.15.29:0/1121819574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4e810 0x5569f4fb9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f9572481640 1 -- 172.21.15.29:0/1121819574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4def0 msgr2=0x5569f4fb9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:50.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f9572481640 1 --2- 172.21.15.29:0/1121819574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4def0 0x5569f4fb9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f9572481640 1 -- 172.21.15.29:0/1121819574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f4cdae80 con 0x5569f4f04bc0 2024-04-25T00:36:50.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f9572481640 1 --2- 172.21.15.29:0/1121819574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 0x5569f4fb9500 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9568005620 tx=0x7f9568013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e736ffc5facc008a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:50.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f954f7fe640 1 -- 172.21.15.29:0/1121819574 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956801a040 con 0x5569f4f04bc0 2024-04-25T00:36:50.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f954f7fe640 1 -- 172.21.15.29:0/1121819574 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9568024900 con 0x5569f4f04bc0 2024-04-25T00:36:50.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.052+0000 7f954f7fe640 1 -- 172.21.15.29:0/1121819574 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568024cb0 con 0x5569f4f04bc0 2024-04-25T00:36:50.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 -- 172.21.15.29:0/1121819574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 msgr2=0x5569f4fb9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 --2- 172.21.15.29:0/1121819574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 0x5569f4fb9500 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9568005620 tx=0x7f9568013000 comp rx=0 tx=0).stop 2024-04-25T00:36:50.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 -- 172.21.15.29:0/1121819574 shutdown_connections 2024-04-25T00:36:50.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 --2- 172.21.15.29:0/1121819574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f4e810 0x5569f4fb9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 --2- 172.21.15.29:0/1121819574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4def0 0x5569f4fb9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 --2- 172.21.15.29:0/1121819574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f04bc0 0x5569f4fb9500 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 -- 172.21.15.29:0/1121819574 >> 172.21.15.29:0/1121819574 conn(0x5569f4f00b70 msgr2=0x5569f4f5b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:50.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 -- 172.21.15.29:0/1121819574 shutdown_connections 2024-04-25T00:36:50.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 -- 172.21.15.29:0/1121819574 wait complete. 2024-04-25T00:36:50.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 -- start start 2024-04-25T00:36:50.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 Processor -- start 2024-04-25T00:36:50.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f04bc0 0x5569f4fbce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4def0 0x5569f4f602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 0x5569f4f60810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5569f4d06c40 con 0x5569f4f4e810 2024-04-25T00:36:50.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5569f4e04e80 con 0x5569f4f4def0 2024-04-25T00:36:50.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5569f4d1b810 con 0x5569f4f04bc0 2024-04-25T00:36:50.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 10 client.0 _ll_register_callbacks cb 0x7f9564009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:50.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9572c82640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 0x5569f4f60810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9572c82640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 0x5569f4f60810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36638/0 (socket says 172.21.15.29:36638) 2024-04-25T00:36:50.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9572c82640 1 -- 172.21.15.29:0/2841911102 learned_addr learned my addr 172.21.15.29:0/2841911102 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:50.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9572481640 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f04bc0 0x5569f4fbce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9571c80640 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4def0 0x5569f4f602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9572c82640 1 -- 172.21.15.29:0/2841911102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f04bc0 msgr2=0x5569f4fbce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:50.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9572c82640 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f04bc0 0x5569f4fbce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9572c82640 1 -- 172.21.15.29:0/2841911102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4def0 msgr2=0x5569f4f602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:50.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9572c82640 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4def0 0x5569f4f602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9572c82640 1 -- 172.21.15.29:0/2841911102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f4d1f760 con 0x5569f4f4e810 2024-04-25T00:36:50.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9571c80640 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4def0 0x5569f4f602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:50.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9572c82640 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 0x5569f4f60810 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9564013ac0 tx=0x7f9564004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7439aeb71d509ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:50.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f954d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:36:50.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f954d7fa640 1 -- 172.21.15.29:0/2841911102 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956401a070 con 0x5569f4f4e810 2024-04-25T00:36:50.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f954d7fa640 1 -- 172.21.15.29:0/2841911102 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f9564036aa0 con 0x5569f4f4e810 2024-04-25T00:36:50.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 -- 172.21.15.29:0/2841911102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9568024900 con 0x5569f4f4e810 2024-04-25T00:36:50.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f954d7fa640 1 -- 172.21.15.29:0/2841911102 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f956403d050 con 0x5569f4f4e810 2024-04-25T00:36:50.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 10 client.7581 fetch_fsmap learned FSMap version 53 2024-04-25T00:36:50.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 -- 172.21.15.29:0/2841911102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x5569f4dab9e0 con 0x5569f4f4e810 2024-04-25T00:36:50.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f954d7fa640 1 -- 172.21.15.29:0/2841911102 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9568024900 con 0x5569f4f4e810 2024-04-25T00:36:50.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f954d7fa640 1 -- 172.21.15.29:0/2841911102 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f9564040020 con 0x5569f4f4e810 2024-04-25T00:36:50.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 10 client.7581 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T00:36:50.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 10 client.7581 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:36:50.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 20 client.7581 populate_metadata read hostname 'smithi029' 2024-04-25T00:36:50.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.29:6834/3939243143,v1:172.21.15.29:6835/3939243143] conn(0x5569f4f60d50 0x5569f4f61130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 4 client.7581 mds_command: new command op to 7404 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:36:50.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 -- 172.21.15.29:0/2841911102 --> [v2:172.21.15.29:6834/3939243143,v1:172.21.15.29:6835/3939243143] -- command(tid 0: {"prefix": "status"}) -- 0x5569f4d5fb20 con 0x5569f4f60d50 2024-04-25T00:36:50.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9572481640 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.29:6834/3939243143,v1:172.21.15.29:6835/3939243143] conn(0x5569f4f60d50 0x5569f4f61130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f9572481640 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.29:6834/3939243143,v1:172.21.15.29:6835/3939243143] conn(0x5569f4f60d50 0x5569f4f61130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa916349b3d9543 server_cookie=3c97ff89cbb43d58 in_seq=0 out_seq=0 2024-04-25T00:36:50.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f954d7fa640 10 client.7581 ms_handle_connect on v2:172.21.15.29:6834/3939243143 2024-04-25T00:36:50.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f954d7fa640 1 -- 172.21.15.29:0/2841911102 <== mds.0 v2:172.21.15.29:6834/3939243143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x5569f4d5fb20 con 0x5569f4f60d50 2024-04-25T00:36:50.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f954d7fa640 10 client.7581 handle_command_reply: tid=0 2024-04-25T00:36:50.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.056+0000 7f957785e000 1 client.7581 shutdown 2024-04-25T00:36:50.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f957785e000 1 -- 172.21.15.29:0/2841911102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 msgr2=0x5569f4f60810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f957785e000 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 0x5569f4f60810 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9564013ac0 tx=0x7f9564004740 comp rx=0 tx=0).stop 2024-04-25T00:36:50.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f957785e000 1 -- 172.21.15.29:0/2841911102 shutdown_connections 2024-04-25T00:36:50.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f957785e000 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5569f4f4e810 0x5569f4f60810 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f957785e000 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5569f4f4def0 0x5569f4f602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f957785e000 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.29:6834/3939243143,v1:172.21.15.29:6835/3939243143] conn(0x5569f4f60d50 0x5569f4f61130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f957785e000 1 --2- 172.21.15.29:0/2841911102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5569f4f04bc0 0x5569f4fbce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f954d7fa640 0 client.7581 ms_handle_reset on v2:172.21.15.29:6834/3939243143 2024-04-25T00:36:50.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f957785e000 1 -- 172.21.15.29:0/2841911102 >> 172.21.15.29:0/2841911102 conn(0x5569f4f00b70 msgr2=0x5569f4f4c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:50.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f957785e000 1 -- 172.21.15.29:0/2841911102 shutdown_connections 2024-04-25T00:36:50.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f957785e000 1 -- 172.21.15.29:0/2841911102 wait complete. 2024-04-25T00:36:50.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f957785e000 20 client.7581 trim_cache size 0 max 16384 2024-04-25T00:36:50.067 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:36:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "59afbe7d-aaa8-45e2-b468-69ee150cacea", 2024-04-25T00:36:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-04-25T00:36:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 7404, 2024-04-25T00:36:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-04-25T00:36:50.070 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-04-25T00:36:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-04-25T00:36:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 10.234925118, 2024-04-25T00:36:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 53, 2024-04-25T00:36:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 84, 2024-04-25T00:36:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 84, 2024-04-25T00:36:50.071 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 15.265237098 2024-04-25T00:36:50.071 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:36:50.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f95746e5640 1 -- 172.21.15.29:0/674292309 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9550067a10 msgr2=0x7f9550069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f95746e5640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9550067a10 0x7f9550069ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f956c1f7790 tx=0x7f9568009040 comp rx=0 tx=0).stop 2024-04-25T00:36:50.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f95746e5640 1 -- 172.21.15.29:0/674292309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f956c1516d0 msgr2=0x7f956c0ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f95746e5640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f956c1516d0 0x7f956c0ca240 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f956001f8d0 tx=0x7f9558005840 comp rx=0 tx=0).stop 2024-04-25T00:36:50.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f95746e5640 1 -- 172.21.15.29:0/674292309 shutdown_connections 2024-04-25T00:36:50.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f95746e5640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9550067a10 0x7f9550069ed0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f95746e5640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c0cb020 0x7f956c0cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f95746e5640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f956c1516d0 0x7f956c0ca240 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f95746e5640 1 --2- 172.21.15.29:0/674292309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9558004810 0x7f956c15b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f95746e5640 1 -- 172.21.15.29:0/674292309 >> 172.21.15.29:0/674292309 conn(0x7f956c09d2a0 msgr2=0x7f956c06d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:50.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f95746e5640 1 -- 172.21.15.29:0/674292309 shutdown_connections 2024-04-25T00:36:50.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.060+0000 7f95746e5640 1 -- 172.21.15.29:0/674292309 wait complete. 2024-04-25T00:36:50.077 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:36:50.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.184+0000 7f5f3b41f640 1 Processor -- start 2024-04-25T00:36:50.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.184+0000 7f5f3b41f640 1 -- start start 2024-04-25T00:36:50.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f3b41f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5f340b4780 0x7f5f3415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f3b41f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f340b3720 con 0x7f5f340cc240 2024-04-25T00:36:50.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f3b41f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f340b39c0 con 0x7f5f3415e8d0 2024-04-25T00:36:50.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f3b41f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5f340cc650 con 0x7f5f340b4780 2024-04-25T00:36:50.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f389ba640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5f340b4780 0x7f5f3415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f389ba640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5f340b4780 0x7f5f3415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36650/0 (socket says 172.21.15.29:36650) 2024-04-25T00:36:50.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f399bc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5f340cc240 0x7f5f3415c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39994/0 (socket says 172.21.15.29:39994) 2024-04-25T00:36:50.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f389ba640 1 -- 172.21.15.29:0/2714200321 learned_addr learned my addr 172.21.15.29:0/2714200321 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:50.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f389ba640 1 -- 172.21.15.29:0/2714200321 >> v1:172.21.15.157:6789/0 conn(0x7f5f3415e8d0 legacy=0x7f5f3415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:50.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f389ba640 1 -- 172.21.15.29:0/2714200321 >> v1:172.21.15.29:6789/0 conn(0x7f5f340cc240 legacy=0x7f5f3415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f389ba640 1 -- 172.21.15.29:0/2714200321 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f340b3ca0 con 0x7f5f340b4780 2024-04-25T00:36:50.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f389ba640 1 --2- 172.21.15.29:0/2714200321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f340b4780 0x7f5f3415f3c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009870 tx=0x7f5f2401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b779bc44549028a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:50.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f2bfff640 1 -- 172.21.15.29:0/2714200321 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f240ab5f0 con 0x7f5f340b4780 2024-04-25T00:36:50.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f2bfff640 1 -- 172.21.15.29:0/2714200321 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f240b0070 con 0x7f5f340b4780 2024-04-25T00:36:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.188+0000 7f5f2bfff640 1 -- 172.21.15.29:0/2714200321 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f240ab9b0 con 0x7f5f340b4780 2024-04-25T00:36:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 -- 172.21.15.29:0/2714200321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f340b4780 msgr2=0x7f5f3415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 --2- 172.21.15.29:0/2714200321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f340b4780 0x7f5f3415f3c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009870 tx=0x7f5f2401c520 comp rx=0 tx=0).stop 2024-04-25T00:36:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 -- 172.21.15.29:0/2714200321 shutdown_connections 2024-04-25T00:36:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 --2- 172.21.15.29:0/2714200321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f340b4780 0x7f5f3415f3c0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 -- 172.21.15.29:0/2714200321 >> 172.21.15.29:0/2714200321 conn(0x7f5f340b1870 msgr2=0x7f5f340b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 -- 172.21.15.29:0/2714200321 shutdown_connections 2024-04-25T00:36:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 -- 172.21.15.29:0/2714200321 wait complete. 2024-04-25T00:36:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 Processor -- start 2024-04-25T00:36:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 -- start start 2024-04-25T00:36:50.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f340b4780 0x7f5f3415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f340cc240 0x7f5f34142be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f3415e8d0 0x7f5f34143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34168e00 con 0x7f5f3415e8d0 2024-04-25T00:36:50.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34168c80 con 0x7f5f340cc240 2024-04-25T00:36:50.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5f34168f80 con 0x7f5f340b4780 2024-04-25T00:36:50.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f399bc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f3415e8d0 0x7f5f34143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f399bc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f3415e8d0 0x7f5f34143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36660/0 (socket says 172.21.15.29:36660) 2024-04-25T00:36:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f399bc640 1 -- 172.21.15.29:0/3448406663 learned_addr learned my addr 172.21.15.29:0/3448406663 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f389ba640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f340cc240 0x7f5f34142be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f399bc640 1 -- 172.21.15.29:0/3448406663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f340b4780 msgr2=0x7f5f3415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f391bb640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f340b4780 0x7f5f3415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f399bc640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f340b4780 0x7f5f3415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f399bc640 1 -- 172.21.15.29:0/3448406663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f340cc240 msgr2=0x7f5f34142be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f399bc640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f340cc240 0x7f5f34142be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f399bc640 1 -- 172.21.15.29:0/3448406663 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f341313f0 con 0x7f5f3415e8d0 2024-04-25T00:36:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f399bc640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f3415e8d0 0x7f5f34143120 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000c440 tx=0x7f5f3000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f29ffb640 1 -- 172.21.15.29:0/3448406663 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30018070 con 0x7f5f3415e8d0 2024-04-25T00:36:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f29ffb640 1 -- 172.21.15.29:0/3448406663 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f30007480 con 0x7f5f3415e8d0 2024-04-25T00:36:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 -- 172.21.15.29:0/3448406663 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f340722d0 con 0x7f5f3415e8d0 2024-04-25T00:36:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f29ffb640 1 -- 172.21.15.29:0/3448406663 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300047f0 con 0x7f5f3415e8d0 2024-04-25T00:36:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.192+0000 7f5f3b41f640 1 -- 172.21.15.29:0/3448406663 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f34072890 con 0x7f5f3415e8d0 2024-04-25T00:36:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.196+0000 7f5f3b41f640 1 -- 172.21.15.29:0/3448406663 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f34069270 con 0x7f5f3415e8d0 2024-04-25T00:36:50.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.196+0000 7f5f29ffb640 1 -- 172.21.15.29:0/3448406663 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f5f3001b070 con 0x7f5f3415e8d0 2024-04-25T00:36:50.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.196+0000 7f5f29ffb640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5f180679e0 0x7f5f18069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:50.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.196+0000 7f5f29ffb640 1 -- 172.21.15.29:0/3448406663 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f5f30084ee0 con 0x7f5f3415e8d0 2024-04-25T00:36:50.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.196+0000 7f5f391bb640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5f180679e0 0x7f5f18069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:50.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.196+0000 7f5f391bb640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5f180679e0 0x7f5f18069ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c007bf0 tx=0x7f5f2c007240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:50.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.200+0000 7f5f29ffb640 1 -- 172.21.15.29:0/3448406663 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5f300552c0 con 0x7f5f3415e8d0 2024-04-25T00:36:50.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.388+0000 7f5f3b41f640 1 -- 172.21.15.29:0/3448406663 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5f3406e7b0 con 0x7f5f3415e8d0 2024-04-25T00:36:50.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.388+0000 7f5f29ffb640 1 -- 172.21.15.29:0/3448406663 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7f5f300595c0 con 0x7f5f3415e8d0 2024-04-25T00:36:50.392 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:36:50.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.392+0000 7f5f3b41f640 1 -- 172.21.15.29:0/3448406663 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5f180679e0 msgr2=0x7f5f18069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.392+0000 7f5f3b41f640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5f180679e0 0x7f5f18069ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c007bf0 tx=0x7f5f2c007240 comp rx=0 tx=0).stop 2024-04-25T00:36:50.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.392+0000 7f5f3b41f640 1 -- 172.21.15.29:0/3448406663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f3415e8d0 msgr2=0x7f5f34143120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:50.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.392+0000 7f5f3b41f640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f3415e8d0 0x7f5f34143120 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000c440 tx=0x7f5f3000c900 comp rx=0 tx=0).stop 2024-04-25T00:36:50.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.392+0000 7f5f3b41f640 1 -- 172.21.15.29:0/3448406663 shutdown_connections 2024-04-25T00:36:50.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.392+0000 7f5f3b41f640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5f180679e0 0x7f5f18069ea0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.392+0000 7f5f3b41f640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f3415e8d0 0x7f5f34143120 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.392+0000 7f5f3b41f640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f340cc240 0x7f5f34142be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.392+0000 7f5f3b41f640 1 --2- 172.21.15.29:0/3448406663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f340b4780 0x7f5f3415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:50.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.392+0000 7f5f3b41f640 1 -- 172.21.15.29:0/3448406663 >> 172.21.15.29:0/3448406663 conn(0x7f5f340b1870 msgr2=0x7f5f340afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:50.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.392+0000 7f5f3b41f640 1 -- 172.21.15.29:0/3448406663 shutdown_connections 2024-04-25T00:36:50.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50.392+0000 7f5f3b41f640 1 -- 172.21.15.29:0/3448406663 wait complete. 2024-04-25T00:36:50.408 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:36:50.409 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:36:50.409 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-25T00:36:50.409 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:36:50.409 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:50.409 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:36:50.409 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:36:50.409 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:36:50.409 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:36:50.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:50.410 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-25T00:36:50.415 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:36:50.415 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:36:50.415 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:36:50.416 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:50.416 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:36:50.416 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:50.416 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:36:50.416 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:50.416 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-25T00:36:50.416 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:36:50.416 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:50.416 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-25T00:36:50.416 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:36:50.416 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:50.416 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-25T00:36:50.417 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:36:50.417 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42149sec preferred_lft 42149sec 2024-04-25T00:36:50.417 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-04-25T00:36:50.417 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:50.417 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:36:50.417 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:50.417 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-25T00:36:50.417 INFO:teuthology.orchestra.run.smithi029.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:36:50.417 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 02:42:28:a5:36:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:50.417 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:36:50.417 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:50.417 INFO:teuthology.orchestra.run.smithi029.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:36:50.417 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:50.418 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:36:50.418 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:50.418 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:36:50.418 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:50.418 INFO:teuthology.orchestra.run.smithi029.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:36:50.418 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:50.418 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:36:50.418 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:50.418 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:36:50.418 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:50.418 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:36:50.418 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:50.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:50.419 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:36:50.466 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-25T00:36:50.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:50.468 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-25T00:36:50.522 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-25T00:36:50.522 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:36:50.522 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T00:36:50.522 INFO:teuthology.orchestra.run.smithi029.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:36:50.522 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T00:36:50.522 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:36:50.523 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:36:50.523 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:36:50.523 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:36:50.523 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:36:50.523 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:36:50.523 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:36:50.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:36:50.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:36:50.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:50.641 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:36:50.691 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:36:50.691 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:36:50.691 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:36:50.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:50.691 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-04-25T00:36:50.745 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:36:50.746 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:36:50.746 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:50.793 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:36:50.793 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:36:50.793 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:50.840 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:36:50.901 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:36:50.950 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:36:50.950 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:36:50.950 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:36:50.950 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:36:50.950 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:36:50.950 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:36:50.950 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:36:50.950 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:36:50.950 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:36:50.951 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:36:50.952 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:36:50.952 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:36:50.952 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:36:50.952 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:36:50.952 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:36:50.952 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:36:50.952 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:50.952 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:50.952 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:50.952 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:50.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:50.953 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:36:51.001 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:36:51.001 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:36:51.046 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:36:51.060 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:36:51.109 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:36:51.109 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:36:51.109 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:36:51.109 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:36:51.109 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:36:51.109 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:36:51.109 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:36:51.109 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:36:51.109 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:36:51.110 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:36:51.110 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:36:51.110 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:36:51.110 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:36:51.110 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:36:51.110 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:36:51.110 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:36:51.110 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:36:51.110 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:36:51.110 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:36:51.110 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:36:51.111 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:36:51.111 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:36:51.111 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:36:51.111 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:36:51.111 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:36:51.111 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:36:51.111 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:36:51.111 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:36:51.111 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:36:51.111 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:51.111 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:51.111 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:51.111 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:51.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:51.112 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:36:51.131 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:2024-04-25T00:36:51.128+0000 7f86d1446580 -1 init, newargv = 0x560dec8d50e0 newargc=15 2024-04-25T00:36:51.131 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[46165]: starting ceph client 2024-04-25T00:36:51.137 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[46165]: starting fuse 2024-04-25T00:36:52.237 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:36:52.254 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:36:52.303 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:36:52.303 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:36:52.303 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:36:52.303 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:36:52.304 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:52.305 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:36:52.306 INFO:teuthology.orchestra.run.smithi029.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:52.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:52.307 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:36:52.353 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:36:52.354 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:36:52.354 DEBUG:teuthology.orchestra.run.smithi029:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:36:52.354 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-04-25T00:36:52.354 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-04-25T00:36:52.354 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-04-25T00:36:52.354 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-04-25T00:36:52.354 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:36:52.354 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-04-25T00:36:52.355 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:36:52.356 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:36:52.356 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.0")) 2024-04-25T00:36:52.356 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-04-25T00:36:52.485 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.0.46165.asok 2024-04-25T00:36:52.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:36:52.497 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46165.asok 2024-04-25T00:36:52.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:52.497 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46165.asok status 2024-04-25T00:36:52.657 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:36:52.657 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-04-25T00:36:52.657 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:36:52.658 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:36:52.658 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "0", 2024-04-25T00:36:52.658 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-04-25T00:36:52.658 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:36:52.658 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "46165", 2024-04-25T00:36:52.658 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-04-25T00:36:52.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:52.658 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-04-25T00:36:52.658 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-04-25T00:36:52.658 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 7596, 2024-04-25T00:36:52.658 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-04-25T00:36:52.658 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-04-25T00:36:52.659 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-04-25T00:36:52.659 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 7596 2024-04-25T00:36:52.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:52.659 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:36:52.659 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:36:52.659 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:36:52.659 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 467822111 2024-04-25T00:36:52.659 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:52.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:52.659 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:36:52.659 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:36:52.659 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:36:52.659 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 467822111 2024-04-25T00:36:52.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:36:52.660 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.7596 192.168.0.1:0/467822111", 2024-04-25T00:36:52.660 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.1:0/467822111", 2024-04-25T00:36:52.660 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-04-25T00:36:52.660 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 53, 2024-04-25T00:36:52.660 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 84, 2024-04-25T00:36:52.660 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:36:52.660 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-04-25T00:36:52.660 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-04-25T00:36:52.660 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:36:52.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:52.671 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-04-25T00:36:52.671 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:52.678 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-04-25T00:36:52.678 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:52.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:52.678 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:36:52.733 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:36:52.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56b6a5c640 1 Processor -- start 2024-04-25T00:36:52.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56b6a5c640 1 -- start start 2024-04-25T00:36:52.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56b6a5c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f56b015e200 0x7f56b00d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:52.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56b6a5c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b00cfc10 con 0x7f56b00cf690 2024-04-25T00:36:52.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56b6a5c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b00cfef0 con 0x7f56b00d2c60 2024-04-25T00:36:52.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56b6a5c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f56b00b38d0 con 0x7f56b015e200 2024-04-25T00:36:52.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56affff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f56b015e200 0x7f56b00d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:52.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56affff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f56b015e200 0x7f56b00d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36668/0 (socket says 172.21.15.29:36668) 2024-04-25T00:36:52.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56affff640 1 -- 172.21.15.29:0/1142913137 learned_addr learned my addr 172.21.15.29:0/1142913137 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:52.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56aeffd640 1 -- 172.21.15.29:0/1142913137 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3933761667 0 0) 0x7f56b00cfc10 con 0x7f56b00cf690 2024-04-25T00:36:52.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56aeffd640 1 -- 172.21.15.29:0/1142913137 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f569c003530 con 0x7f56b00cf690 2024-04-25T00:36:52.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56affff640 1 -- 172.21.15.29:0/1142913137 >> v1:172.21.15.157:6789/0 conn(0x7f56b00d2c60 legacy=0x7f56b00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:52.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56affff640 1 -- 172.21.15.29:0/1142913137 >> v1:172.21.15.29:6789/0 conn(0x7f56b00cf690 legacy=0x7f56b00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:52.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.904+0000 7f56affff640 1 -- 172.21.15.29:0/1142913137 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b00b3b20 con 0x7f56b015e200 2024-04-25T00:36:52.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56affff640 1 --2- 172.21.15.29:0/1142913137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56b015e200 0x7f56b00d47f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009870 tx=0x7f56a001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b8d055ae0b0481 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:52.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56aeffd640 1 -- 172.21.15.29:0/1142913137 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a00a9610 con 0x7f56b015e200 2024-04-25T00:36:52.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56aeffd640 1 -- 172.21.15.29:0/1142913137 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a00a97b0 con 0x7f56b015e200 2024-04-25T00:36:52.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56aeffd640 1 -- 172.21.15.29:0/1142913137 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a00a9b80 con 0x7f56b015e200 2024-04-25T00:36:52.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 -- 172.21.15.29:0/1142913137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56b015e200 msgr2=0x7f56b00d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:52.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 --2- 172.21.15.29:0/1142913137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56b015e200 0x7f56b00d47f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009870 tx=0x7f56a001c530 comp rx=0 tx=0).stop 2024-04-25T00:36:52.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 -- 172.21.15.29:0/1142913137 shutdown_connections 2024-04-25T00:36:52.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 --2- 172.21.15.29:0/1142913137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56b015e200 0x7f56b00d47f0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:52.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 -- 172.21.15.29:0/1142913137 >> 172.21.15.29:0/1142913137 conn(0x7f56b009d280 msgr2=0x7f56b00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:52.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 -- 172.21.15.29:0/1142913137 shutdown_connections 2024-04-25T00:36:52.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 -- 172.21.15.29:0/1142913137 wait complete. 2024-04-25T00:36:52.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 Processor -- start 2024-04-25T00:36:52.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 -- start start 2024-04-25T00:36:52.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56b00cf690 0x7f56b00ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:52.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b00d2c60 0x7f56b0151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:52.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56b015e200 0x7f56b015f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:52.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f56b00b53c0 con 0x7f56b00cf690 2024-04-25T00:36:52.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56b00b5240 con 0x7f56b00d2c60 2024-04-25T00:36:52.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b6a5c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f56b00b5540 con 0x7f56b015e200 2024-04-25T00:36:52.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56affff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56b00cf690 0x7f56b00ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:52.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56affff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56b00cf690 0x7f56b00ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36684/0 (socket says 172.21.15.29:36684) 2024-04-25T00:36:52.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56affff640 1 -- 172.21.15.29:0/2938221121 learned_addr learned my addr 172.21.15.29:0/2938221121 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:52.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56af7fe640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b00d2c60 0x7f56b0151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:52.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56b4ff9640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56b015e200 0x7f56b015f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:52.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56affff640 1 -- 172.21.15.29:0/2938221121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56b015e200 msgr2=0x7f56b015f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:52.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56affff640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56b015e200 0x7f56b015f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:52.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56affff640 1 -- 172.21.15.29:0/2938221121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b00d2c60 msgr2=0x7f56b0151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:52.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56affff640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b00d2c60 0x7f56b0151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:52.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56affff640 1 -- 172.21.15.29:0/2938221121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b01616c0 con 0x7f56b00cf690 2024-04-25T00:36:52.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56af7fe640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b00d2c60 0x7f56b0151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:52.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56affff640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56b00cf690 0x7f56b00ca590 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f56a0019140 tx=0x7f56a00192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:52.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56acff9640 1 -- 172.21.15.29:0/2938221121 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a00a9610 con 0x7f56b00cf690 2024-04-25T00:36:52.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.908+0000 7f56acff9640 1 -- 172.21.15.29:0/2938221121 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a00ba680 con 0x7f56b00cf690 2024-04-25T00:36:52.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.912+0000 7f56acff9640 1 -- 172.21.15.29:0/2938221121 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a00baa70 con 0x7f56b00cf690 2024-04-25T00:36:52.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.912+0000 7f56b6a5c640 1 -- 172.21.15.29:0/2938221121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b015e9f0 con 0x7f56b00cf690 2024-04-25T00:36:52.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.912+0000 7f56b6a5c640 1 -- 172.21.15.29:0/2938221121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b015efb0 con 0x7f56b00cf690 2024-04-25T00:36:52.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.912+0000 7f56b6a5c640 1 -- 172.21.15.29:0/2938221121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b0069270 con 0x7f56b00cf690 2024-04-25T00:36:52.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.912+0000 7f56acff9640 1 -- 172.21.15.29:0/2938221121 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f56a00bac10 con 0x7f56b00cf690 2024-04-25T00:36:52.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.912+0000 7f56acff9640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f56900679e0 0x7f5690069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:52.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.912+0000 7f56af7fe640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f56900679e0 0x7f5690069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:52.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.912+0000 7f56acff9640 1 -- 172.21.15.29:0/2938221121 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f56a0120410 con 0x7f56b00cf690 2024-04-25T00:36:52.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.912+0000 7f56af7fe640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f56900679e0 0x7f5690069ea0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f56b01f8530 tx=0x7f5698009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:52.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:52.916+0000 7f56acff9640 1 -- 172.21.15.29:0/2938221121 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f56a00ad020 con 0x7f56b00cf690 2024-04-25T00:36:53.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.112+0000 7f56b6a5c640 1 -- 172.21.15.29:0/2938221121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f56b00d2a00 con 0x7f56b00cf690 2024-04-25T00:36:53.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.116+0000 7f56acff9640 1 -- 172.21.15.29:0/2938221121 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7f56a0016150 con 0x7f56b00cf690 2024-04-25T00:36:53.117 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:36:53.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.116+0000 7f56b6a5c640 1 -- 172.21.15.29:0/2938221121 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f56900679e0 msgr2=0x7f5690069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:53.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.116+0000 7f56b6a5c640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f56900679e0 0x7f5690069ea0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f56b01f8530 tx=0x7f5698009040 comp rx=0 tx=0).stop 2024-04-25T00:36:53.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.116+0000 7f56b6a5c640 1 -- 172.21.15.29:0/2938221121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56b00cf690 msgr2=0x7f56b00ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:53.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.116+0000 7f56b6a5c640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56b00cf690 0x7f56b00ca590 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f56a0019140 tx=0x7f56a00192d0 comp rx=0 tx=0).stop 2024-04-25T00:36:53.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.116+0000 7f56b6a5c640 1 -- 172.21.15.29:0/2938221121 shutdown_connections 2024-04-25T00:36:53.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.116+0000 7f56b6a5c640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f56900679e0 0x7f5690069ea0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:53.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.116+0000 7f56b6a5c640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56b015e200 0x7f56b015f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:53.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.116+0000 7f56b6a5c640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b00d2c60 0x7f56b0151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:53.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.116+0000 7f56b6a5c640 1 --2- 172.21.15.29:0/2938221121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56b00cf690 0x7f56b00ca590 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:53.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.116+0000 7f56b6a5c640 1 -- 172.21.15.29:0/2938221121 >> 172.21.15.29:0/2938221121 conn(0x7f56b009d280 msgr2=0x7f56b00d4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:53.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.116+0000 7f56b6a5c640 1 -- 172.21.15.29:0/2938221121 shutdown_connections 2024-04-25T00:36:53.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:53.116+0000 7f56b6a5c640 1 -- 172.21.15.29:0/2938221121 wait complete. 2024-04-25T00:36:53.135 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:36:53.135 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:36:53.135 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-25T00:36:53.135 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:36:53.135 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:53.135 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:36:53.135 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:36:53.136 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:36:53.136 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:36:53.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:53.136 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-25T00:36:53.143 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:36:53.144 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:36:53.144 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:36:53.144 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:53.144 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:36:53.144 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:53.144 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:36:53.144 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:53.145 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-25T00:36:53.145 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:36:53.145 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:53.145 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-25T00:36:53.145 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:36:53.145 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:53.146 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:36:53.146 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:53.146 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:36:53.146 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42142sec preferred_lft 42142sec 2024-04-25T00:36:53.146 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-25T00:36:53.150 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:53.150 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:36:53.151 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:9c:8d:4b:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:53.151 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:36:53.151 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:53.151 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:36:53.151 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:36:53.151 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:36:53.152 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:53.152 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:36:53.152 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:53.152 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:36:53.152 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:53.152 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:36:53.152 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:53.152 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:36:53.152 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:53.152 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:36:53.152 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:36:53.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:53.153 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:36:53.199 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:36:53.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:53.200 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-25T00:36:53.253 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-25T00:36:53.253 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:36:53.253 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:36:53.253 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:36:53.253 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:36:53.253 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:36:53.253 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:36:53.253 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:36:53.253 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:36:53.253 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:36:53.253 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:36:53.254 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:36:53.254 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:36:53.254 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:36:53.254 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:36:53.254 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:36:53.254 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:36:53.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:36:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:36:53.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:36:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:36:53.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:53.384 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:36:53.389 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:36:53.389 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:36:53.389 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:36:53.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:53.389 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-25T00:36:53.442 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T00:36:53.442 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:36:53.443 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:53.492 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:36:53.492 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:36:53.492 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:53.540 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:36:53.598 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:36:53.644 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:36:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:36:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:36:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:36:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:36:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:36:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:36:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:36:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:36:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:36:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:36:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:36:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:36:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:36:53.647 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:36:53.647 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:36:53.647 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:53.647 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:53.647 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:53.647 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:53.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:53.648 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:36:53.696 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:36:53.696 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T00:36:53.741 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:36:53.755 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:36:53.804 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:36:53.804 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:36:53.804 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:36:53.804 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:36:53.804 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:36:53.804 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:36:53.804 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:36:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:36:53.806 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:36:53.806 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:36:53.806 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:36:53.806 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:36:53.806 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:36:53.806 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:36:53.806 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:36:53.806 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:53.806 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:53.806 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:53.806 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:53.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:53.807 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:36:53.822 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-04-25T00:36:53.819+0000 7f1377212580 -1 init, newargv = 0x56367cf87ab0 newargc=15 2024-04-25T00:36:53.822 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34437]: starting ceph client 2024-04-25T00:36:53.829 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34437]: starting fuse 2024-04-25T00:36:54.928 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:36:54.944 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:36:54.994 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:36:54.994 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:36:54.994 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:36:54.994 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:36:54.995 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:36:54.995 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:36:54.995 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:36:54.995 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:36:54.995 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:36:54.995 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:36:54.995 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:36:54.995 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:36:54.995 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:36:54.995 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:36:54.995 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:36:54.995 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:36:54.995 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:54.996 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:36:54.997 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:54.997 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:36:54.997 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:54.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:54.998 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:36:55.043 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:36:55.044 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:36:55.044 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:36:55.044 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-25T00:36:55.044 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-25T00:36:55.044 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-25T00:36:55.044 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-25T00:36:55.044 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:36:55.044 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-25T00:36:55.044 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-25T00:36:55.045 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-25T00:36:55.046 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:36:55.046 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:36:55.046 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-04-25T00:36:55.046 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-25T00:36:55.166 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.34437.asok 2024-04-25T00:36:55.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:36:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:36:55.177 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34437.asok 2024-04-25T00:36:55.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:55.177 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34437.asok status 2024-04-25T00:36:55.333 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:36:55.333 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-25T00:36:55.333 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:36:55.333 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:36:55.333 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-04-25T00:36:55.333 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-25T00:36:55.333 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T00:36:55.333 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "34437", 2024-04-25T00:36:55.333 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-25T00:36:55.333 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:36:55.333 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-25T00:36:55.334 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-25T00:36:55.334 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 6044, 2024-04-25T00:36:55.334 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-25T00:36:55.334 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-25T00:36:55.334 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-25T00:36:55.334 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 6044 2024-04-25T00:36:55.334 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:36:55.334 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:36:55.334 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:36:55.334 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:36:55.334 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1735542167 2024-04-25T00:36:55.334 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-25T00:36:55.334 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1735542167 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.6044 192.168.0.1:0/1735542167", 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/1735542167", 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 53, 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 84, 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-25T00:36:55.335 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:36:55.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:55.347 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-25T00:36:55.347 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:55.353 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-25T00:36:55.353 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:55.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:36:55.354 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:36:55.408 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-25T00:36:55.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f65a68fb640 1 Processor -- start 2024-04-25T00:36:55.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f65a68fb640 1 -- start start 2024-04-25T00:36:55.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f65a68fb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f65a0151640 0x7f65a006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:55.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f65a68fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65a00b2b70 0x7f65a006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:55.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f65a68fb640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a00b40a0 con 0x7f65a015dfd0 2024-04-25T00:36:55.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f65a68fb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f65a00b3000 con 0x7f65a0151640 2024-04-25T00:36:55.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f65a68fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65a00b32e0 con 0x7f65a00b2b70 2024-04-25T00:36:55.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f659ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f65a0151640 0x7f65a006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:55.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f65a4e98640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f65a015dfd0 0x7f65a015e400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40010/0 (socket says 172.21.15.29:40010) 2024-04-25T00:36:55.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f65a4e98640 1 -- 172.21.15.29:0/485708903 learned_addr learned my addr 172.21.15.29:0/485708903 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:55.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f659ffff640 1 -- 172.21.15.29:0/485708903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65a00b2b70 msgr2=0x7f65a006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:55.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f6597fff640 1 --2- 172.21.15.29:0/485708903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65a00b2b70 0x7f65a006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:55.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f659ffff640 1 --2- 172.21.15.29:0/485708903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65a00b2b70 0x7f65a006db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:55.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f659ffff640 1 -- 172.21.15.29:0/485708903 >> v1:172.21.15.29:6789/0 conn(0x7f65a015dfd0 legacy=0x7f65a015e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:55.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f659ffff640 1 -- 172.21.15.29:0/485708903 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a00b35c0 con 0x7f65a0151640 2024-04-25T00:36:55.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.547+0000 7f6597fff640 1 --2- 172.21.15.29:0/485708903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65a00b2b70 0x7f65a006db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:55.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.551+0000 7f659ffff640 1 --2- 172.21.15.29:0/485708903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a0151640 0x7f65a006d5d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f65880097c0 tx=0x7f658801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a340b107cb4cb750 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:55.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.551+0000 7f659f7fe640 1 -- 172.21.15.29:0/485708903 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65880b0070 con 0x7f65a0151640 2024-04-25T00:36:55.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.551+0000 7f659f7fe640 1 -- 172.21.15.29:0/485708903 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65880ab600 con 0x7f65a0151640 2024-04-25T00:36:55.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.551+0000 7f659f7fe640 1 -- 172.21.15.29:0/485708903 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65880ab9d0 con 0x7f65a0151640 2024-04-25T00:36:55.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.551+0000 7f65a68fb640 1 -- 172.21.15.29:0/485708903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a0151640 msgr2=0x7f65a006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:55.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.551+0000 7f65a68fb640 1 --2- 172.21.15.29:0/485708903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a0151640 0x7f65a006d5d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f65880097c0 tx=0x7f658801c410 comp rx=0 tx=0).stop 2024-04-25T00:36:55.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.551+0000 7f65a68fb640 1 -- 172.21.15.29:0/485708903 shutdown_connections 2024-04-25T00:36:55.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.551+0000 7f65a68fb640 1 --2- 172.21.15.29:0/485708903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65a00b2b70 0x7f65a006db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:55.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.551+0000 7f65a68fb640 1 --2- 172.21.15.29:0/485708903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a0151640 0x7f65a006d5d0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:55.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.551+0000 7f65a68fb640 1 -- 172.21.15.29:0/485708903 >> 172.21.15.29:0/485708903 conn(0x7f65a00b1870 msgr2=0x7f65a00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:55.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.551+0000 7f65a68fb640 1 -- 172.21.15.29:0/485708903 shutdown_connections 2024-04-25T00:36:55.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.551+0000 7f65a68fb640 1 -- 172.21.15.29:0/485708903 wait complete. 2024-04-25T00:36:55.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f65a68fb640 1 Processor -- start 2024-04-25T00:36:55.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f65a68fb640 1 -- start start 2024-04-25T00:36:55.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f65a68fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65a0151640 0x7f65a014f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:55.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f65a68fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a015dfd0 0x7f65a0151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:55.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f65a68fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65a00cdb20 0x7f65a0148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:55.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f65a68fb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f65a006d0d0 con 0x7f65a015dfd0 2024-04-25T00:36:55.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f65a68fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f65a006cf50 con 0x7f65a00cdb20 2024-04-25T00:36:55.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f65a68fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65a006d250 con 0x7f65a0151640 2024-04-25T00:36:55.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f6597fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a015dfd0 0x7f65a0151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:55.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f6597fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a015dfd0 0x7f65a0151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36694/0 (socket says 172.21.15.29:36694) 2024-04-25T00:36:55.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f6597fff640 1 -- 172.21.15.29:0/2837774993 learned_addr learned my addr 172.21.15.29:0/2837774993 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:55.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f659ffff640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65a0151640 0x7f65a014f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:55.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f65a4e98640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65a00cdb20 0x7f65a0148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:55.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f6597fff640 1 -- 172.21.15.29:0/2837774993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65a0151640 msgr2=0x7f65a014f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:55.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f6597fff640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65a0151640 0x7f65a014f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:55.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f6597fff640 1 -- 172.21.15.29:0/2837774993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65a00cdb20 msgr2=0x7f65a0148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:55.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f6597fff640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65a00cdb20 0x7f65a0148ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:55.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f6597fff640 1 -- 172.21.15.29:0/2837774993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a01482e0 con 0x7f65a015dfd0 2024-04-25T00:36:55.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f659ffff640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65a0151640 0x7f65a014f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:55.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f65a4e98640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65a00cdb20 0x7f65a0148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:55.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f6597fff640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a015dfd0 0x7f65a0151a20 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f659000c7f0 tx=0x7f659000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:55.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f659d7fa640 1 -- 172.21.15.29:0/2837774993 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590015070 con 0x7f65a015dfd0 2024-04-25T00:36:55.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f659d7fa640 1 -- 172.21.15.29:0/2837774993 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6590007820 con 0x7f65a015dfd0 2024-04-25T00:36:55.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f659d7fa640 1 -- 172.21.15.29:0/2837774993 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590007c10 con 0x7f65a015dfd0 2024-04-25T00:36:55.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f65a68fb640 1 -- 172.21.15.29:0/2837774993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a00ce570 con 0x7f65a015dfd0 2024-04-25T00:36:55.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f65a68fb640 1 -- 172.21.15.29:0/2837774993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a0149880 con 0x7f65a015dfd0 2024-04-25T00:36:55.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f65a68fb640 1 -- 172.21.15.29:0/2837774993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a00692e0 con 0x7f65a015dfd0 2024-04-25T00:36:55.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f659d7fa640 1 -- 172.21.15.29:0/2837774993 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f6590018070 con 0x7f65a015dfd0 2024-04-25T00:36:55.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.555+0000 7f659d7fa640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f65800679e0 0x7f6580069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:55.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.559+0000 7f659d7fa640 1 -- 172.21.15.29:0/2837774993 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f6590085ff0 con 0x7f65a015dfd0 2024-04-25T00:36:55.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.559+0000 7f659ffff640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f65800679e0 0x7f6580069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:55.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.559+0000 7f659ffff640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f65800679e0 0x7f6580069ea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f65880ad000 tx=0x7f6588008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:55.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.563+0000 7f659d7fa640 1 -- 172.21.15.29:0/2837774993 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6590056960 con 0x7f65a015dfd0 2024-04-25T00:36:55.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:55.763+0000 7f65a68fb640 1 -- 172.21.15.29:0/2837774993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f65a006e1d0 con 0x7f65a015dfd0 2024-04-25T00:36:56.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.403+0000 7f659d7fa640 1 -- 172.21.15.29:0/2837774993 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v54) ==== 136+0+0 (secure 0 0 0) 0x7f6590056260 con 0x7f65a015dfd0 2024-04-25T00:36:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.407+0000 7f65a68fb640 1 -- 172.21.15.29:0/2837774993 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f65800679e0 msgr2=0x7f6580069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.407+0000 7f65a68fb640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f65800679e0 0x7f6580069ea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f65880ad000 tx=0x7f6588008040 comp rx=0 tx=0).stop 2024-04-25T00:36:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.407+0000 7f65a68fb640 1 -- 172.21.15.29:0/2837774993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a015dfd0 msgr2=0x7f65a0151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.407+0000 7f65a68fb640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a015dfd0 0x7f65a0151a20 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f659000c7f0 tx=0x7f659000ccb0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.407+0000 7f65a68fb640 1 -- 172.21.15.29:0/2837774993 shutdown_connections 2024-04-25T00:36:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.407+0000 7f65a68fb640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f65800679e0 0x7f6580069ea0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.407+0000 7f65a68fb640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65a00cdb20 0x7f65a0148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.407+0000 7f65a68fb640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a015dfd0 0x7f65a0151a20 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.407+0000 7f65a68fb640 1 --2- 172.21.15.29:0/2837774993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65a0151640 0x7f65a014f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.407+0000 7f65a68fb640 1 -- 172.21.15.29:0/2837774993 >> 172.21.15.29:0/2837774993 conn(0x7f65a00b1870 msgr2=0x7f65a00714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.407+0000 7f65a68fb640 1 -- 172.21.15.29:0/2837774993 shutdown_connections 2024-04-25T00:36:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.407+0000 7f65a68fb640 1 -- 172.21.15.29:0/2837774993 wait complete. 2024-04-25T00:36:56.432 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-25T00:36:56.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effe63c9640 1 Processor -- start 2024-04-25T00:36:56.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effe63c9640 1 -- start start 2024-04-25T00:36:56.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effe63c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effe01588a0 0x7effe00b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effe63c9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7effe0071cc0 0x7effe00b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effe63c9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe015e930 con 0x7effe01584c0 2024-04-25T00:36:56.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effe63c9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7effe006d330 con 0x7effe0071cc0 2024-04-25T00:36:56.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effe63c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7effe00ae3b0 con 0x7effe01588a0 2024-04-25T00:36:56.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7effe01584c0 0x7effe00715b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40024/0 (socket says 172.21.15.29:40024) 2024-04-25T00:36:56.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdffff640 1 -- 172.21.15.29:0/3710236154 learned_addr learned my addr 172.21.15.29:0/3710236154 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:56.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effe4966640 1 --2- 172.21.15.29:0/3710236154 >> v2:172.21.15.157:3300/0 conn(0x7effe0071cc0 0x7effe00b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdf7fe640 1 --2- 172.21.15.29:0/3710236154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effe01588a0 0x7effe00b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1925456864 0 0) 0x7effe015e930 con 0x7effe01584c0 2024-04-25T00:36:56.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effcc003530 con 0x7effe01584c0 2024-04-25T00:36:56.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 388348807 0 0) 0x7effcc003530 con 0x7effe01584c0 2024-04-25T00:36:56.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effe01588a0 msgr2=0x7effe00b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:56.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 --2- 172.21.15.29:0/3710236154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effe01588a0 0x7effe00b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 >> v2:172.21.15.157:3300/0 conn(0x7effe0071cc0 msgr2=0x7effe00b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:56.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 --2- 172.21.15.29:0/3710236154 >> v2:172.21.15.157:3300/0 conn(0x7effe0071cc0 0x7effe00b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe00b4c40 con 0x7effe01584c0 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7effd8001a00 con 0x7effe01584c0 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 >> v1:172.21.15.29:6789/0 conn(0x7effe01584c0 legacy=0x7effe00715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 --2- 172.21.15.29:0/3710236154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effcc0039c0 0x7effcc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 --2- 172.21.15.29:0/3710236154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effcc004810 0x7effcc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 --2- 172.21.15.29:0/3710236154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effe01588a0 0x7effcc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7effcc00b9d0 con 0x7effcc0039c0 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7effcc00bb50 con 0x7effcc004810 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7effcc00bcd0 con 0x7effe01588a0 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdffff640 1 --2- 172.21.15.29:0/3710236154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effcc0039c0 0x7effcc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdf7fe640 1 --2- 172.21.15.29:0/3710236154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effcc004810 0x7effcc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdffff640 1 -- 172.21.15.29:0/3710236154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effe01588a0 msgr2=0x7effcc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effe4966640 1 --2- 172.21.15.29:0/3710236154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effe01588a0 0x7effcc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdffff640 1 --2- 172.21.15.29:0/3710236154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effe01588a0 0x7effcc0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdffff640 1 -- 172.21.15.29:0/3710236154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effcc004810 msgr2=0x7effcc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:56.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdffff640 1 --2- 172.21.15.29:0/3710236154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effcc004810 0x7effcc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdffff640 1 -- 172.21.15.29:0/3710236154 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effcc003530 con 0x7effcc0039c0 2024-04-25T00:36:56.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.559+0000 7effdf7fe640 1 --2- 172.21.15.29:0/3710236154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effcc004810 0x7effcc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:56.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.563+0000 7effdffff640 1 --2- 172.21.15.29:0/3710236154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effcc0039c0 0x7effcc003de0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7effd800a870 tx=0x7effd801d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a586f3f467db1782 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:56.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.563+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd800a4c0 con 0x7effcc0039c0 2024-04-25T00:36:56.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.563+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd80ad630 con 0x7effcc0039c0 2024-04-25T00:36:56.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.563+0000 7effdeffd640 1 -- 172.21.15.29:0/3710236154 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd80ada20 con 0x7effcc0039c0 2024-04-25T00:36:56.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.563+0000 7effe63c9640 1 -- 172.21.15.29:0/3710236154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effcc0039c0 msgr2=0x7effcc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:56.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.563+0000 7effe63c9640 1 --2- 172.21.15.29:0/3710236154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effcc0039c0 0x7effcc003de0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7effd800a870 tx=0x7effd801d3a0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.563+0000 7effe4966640 1 -- 172.21.15.29:0/3710236154 reap_dead start 2024-04-25T00:36:56.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.563+0000 7effe63c9640 1 -- 172.21.15.29:0/3710236154 shutdown_connections 2024-04-25T00:36:56.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.563+0000 7effe63c9640 1 -- 172.21.15.29:0/3710236154 >> 172.21.15.29:0/3710236154 conn(0x7effe0075030 msgr2=0x7effe00b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:56.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.563+0000 7effe63c9640 1 -- 172.21.15.29:0/3710236154 shutdown_connections 2024-04-25T00:36:56.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.563+0000 7effe63c9640 1 -- 172.21.15.29:0/3710236154 wait complete. 2024-04-25T00:36:56.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe63c9640 1 Processor -- start 2024-04-25T00:36:56.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe63c9640 1 -- start start 2024-04-25T00:36:56.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe63c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effcc0039c0 0x7effe014dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe63c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effe0162580 0x7effe0162a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe63c9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effe006e380 0x7effe006e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe63c9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7effe00af770 con 0x7effe006e380 2024-04-25T00:36:56.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe63c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7effe00af5f0 con 0x7effcc0039c0 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe63c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7effe00af8f0 con 0x7effe0162580 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe4966640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effe006e380 0x7effe006e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe4966640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effe006e380 0x7effe006e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36718/0 (socket says 172.21.15.29:36718) 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe4966640 1 -- 172.21.15.29:0/2210188002 learned_addr learned my addr 172.21.15.29:0/2210188002 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effdffff640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effcc0039c0 0x7effe014dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effdf7fe640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effe0162580 0x7effe0162a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe4966640 1 -- 172.21.15.29:0/2210188002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effe0162580 msgr2=0x7effe0162a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe4966640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effe0162580 0x7effe0162a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe4966640 1 -- 172.21.15.29:0/2210188002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effcc0039c0 msgr2=0x7effe014dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe4966640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effcc0039c0 0x7effe014dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe4966640 1 -- 172.21.15.29:0/2210188002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe014e170 con 0x7effe006e380 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe4966640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effe006e380 0x7effe006e800 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7effe00b5bc0 tx=0x7effd0010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effdcff9640 1 -- 172.21.15.29:0/2210188002 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd000f040 con 0x7effe006e380 2024-04-25T00:36:56.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effdcff9640 1 -- 172.21.15.29:0/2210188002 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd001c070 con 0x7effe006e380 2024-04-25T00:36:56.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe63c9640 1 -- 172.21.15.29:0/2210188002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe01636f0 con 0x7effe006e380 2024-04-25T00:36:56.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effdcff9640 1 -- 172.21.15.29:0/2210188002 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd000eb30 con 0x7effe006e380 2024-04-25T00:36:56.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe63c9640 1 -- 172.21.15.29:0/2210188002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe0163cb0 con 0x7effe006e380 2024-04-25T00:36:56.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effe63c9640 1 -- 172.21.15.29:0/2210188002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe0069270 con 0x7effe006e380 2024-04-25T00:36:56.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effdcff9640 1 -- 172.21.15.29:0/2210188002 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7effd000ecd0 con 0x7effe006e380 2024-04-25T00:36:56.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.567+0000 7effdcff9640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7effc00679e0 0x7effc0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.571+0000 7effdcff9640 1 -- 172.21.15.29:0/2210188002 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7effd0085590 con 0x7effe006e380 2024-04-25T00:36:56.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.571+0000 7effdffff640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7effc00679e0 0x7effc0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.571+0000 7effdffff640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7effc00679e0 0x7effc0069ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7effd80ad000 tx=0x7effd8001370 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:56.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.575+0000 7effdcff9640 1 -- 172.21.15.29:0/2210188002 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7effd0025020 con 0x7effe006e380 2024-04-25T00:36:56.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.759+0000 7effe63c9640 1 -- 172.21.15.29:0/2210188002 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7effe00719e0 con 0x7effc00679e0 2024-04-25T00:36:56.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.763+0000 7effdcff9640 1 -- 172.21.15.29:0/2210188002 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7effe00719e0 con 0x7effc00679e0 2024-04-25T00:36:56.765 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:36:56.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:36:56.765 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-04-25T00:36:56.765 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:36:56.765 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:36:56.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.763+0000 7effe63c9640 1 -- 172.21.15.29:0/2210188002 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7effc00679e0 msgr2=0x7effc0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:56.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.763+0000 7effe63c9640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7effc00679e0 0x7effc0069ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7effd80ad000 tx=0x7effd8001370 comp rx=0 tx=0).stop 2024-04-25T00:36:56.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.763+0000 7effe63c9640 1 -- 172.21.15.29:0/2210188002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effe006e380 msgr2=0x7effe006e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:56.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.763+0000 7effe63c9640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effe006e380 0x7effe006e800 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7effe00b5bc0 tx=0x7effd0010c20 comp rx=0 tx=0).stop 2024-04-25T00:36:56.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.763+0000 7effe63c9640 1 -- 172.21.15.29:0/2210188002 shutdown_connections 2024-04-25T00:36:56.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.763+0000 7effe63c9640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7effc00679e0 0x7effc0069ea0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.763+0000 7effe63c9640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effe006e380 0x7effe006e800 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.763+0000 7effe63c9640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effe0162580 0x7effe0162a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.763+0000 7effe63c9640 1 --2- 172.21.15.29:0/2210188002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effcc0039c0 0x7effe014dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.763+0000 7effe63c9640 1 -- 172.21.15.29:0/2210188002 >> 172.21.15.29:0/2210188002 conn(0x7effe0075030 msgr2=0x7effe0156f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:56.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.763+0000 7effe63c9640 1 -- 172.21.15.29:0/2210188002 shutdown_connections 2024-04-25T00:36:56.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.763+0000 7effe63c9640 1 -- 172.21.15.29:0/2210188002 wait complete. 2024-04-25T00:36:56.784 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:36:56.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf3117640 1 Processor -- start 2024-04-25T00:36:56.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf3117640 1 -- start start 2024-04-25T00:36:56.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf3117640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efdec0b3140 0x7efdec06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf3117640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdec0b41e0 con 0x7efdec0b3c30 2024-04-25T00:36:56.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf3117640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdec06d960 con 0x7efdec0b4b10 2024-04-25T00:36:56.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf3117640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7efdec06dc40 con 0x7efdec0b3140 2024-04-25T00:36:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf16b4640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7efdec0b3c30 0x7efdec0ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40040/0 (socket says 172.21.15.29:40040) 2024-04-25T00:36:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf16b4640 1 -- 172.21.15.29:0/1089818682 learned_addr learned my addr 172.21.15.29:0/1089818682 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf0eb3640 1 --2- 172.21.15.29:0/1089818682 >> v2:172.21.15.157:3300/0 conn(0x7efdec0b3140 0x7efdec06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3856820704 0 0) 0x7efdec0b41e0 con 0x7efdec0b3c30 2024-04-25T00:36:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdd4003530 con 0x7efdec0b3c30 2024-04-25T00:36:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2862283791 0 0) 0x7efdd4003530 con 0x7efdec0b3c30 2024-04-25T00:36:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 >> v2:172.21.15.157:3300/0 conn(0x7efdec0b3140 msgr2=0x7efdec06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 --2- 172.21.15.29:0/1089818682 >> v2:172.21.15.157:3300/0 conn(0x7efdec0b3140 0x7efdec06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 >> v1:172.21.15.157:6789/0 conn(0x7efdec0b4b10 legacy=0x7efdec0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec06df20 con 0x7efdec0b3c30 2024-04-25T00:36:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7efde4001a00 con 0x7efdec0b3c30 2024-04-25T00:36:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 >> v1:172.21.15.29:6789/0 conn(0x7efdec0b3c30 legacy=0x7efdec0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 --2- 172.21.15.29:0/1089818682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd40039c0 0x7efdd4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 --2- 172.21.15.29:0/1089818682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd4004810 0x7efdd4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 --2- 172.21.15.29:0/1089818682 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdd40051d0 0x7efdd400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efdec06dc40 con 0x7efdd40039c0 2024-04-25T00:36:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efdd4010480 con 0x7efdd4004810 2024-04-25T00:36:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efdd4010680 con 0x7efdd40051d0 2024-04-25T00:36:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf0eb3640 1 --2- 172.21.15.29:0/1089818682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd40039c0 0x7efdd4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf16b4640 1 --2- 172.21.15.29:0/1089818682 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdd40051d0 0x7efdd400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf0eb3640 1 -- 172.21.15.29:0/1089818682 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdd40051d0 msgr2=0x7efdd400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdebfff640 1 --2- 172.21.15.29:0/1089818682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd4004810 0x7efdd4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf0eb3640 1 --2- 172.21.15.29:0/1089818682 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdd40051d0 0x7efdd400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf0eb3640 1 -- 172.21.15.29:0/1089818682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd4004810 msgr2=0x7efdd4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf0eb3640 1 --2- 172.21.15.29:0/1089818682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd4004810 0x7efdd4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf0eb3640 1 -- 172.21.15.29:0/1089818682 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd4003530 con 0x7efdd40039c0 2024-04-25T00:36:56.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdf16b4640 1 -- 172.21.15.29:0/1089818682 reap_dead start 2024-04-25T00:36:56.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.915+0000 7efdebfff640 1 --2- 172.21.15.29:0/1089818682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd4004810 0x7efdd4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:56.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.919+0000 7efdf0eb3640 1 --2- 172.21.15.29:0/1089818682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd40039c0 0x7efdd4003de0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7efddc009ae0 tx=0x7efddc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c75b44fb54bd071 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:56.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.919+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc0096b0 con 0x7efdd40039c0 2024-04-25T00:36:56.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.919+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efddc0a6430 con 0x7efdd40039c0 2024-04-25T00:36:56.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.919+0000 7efdeb7fe640 1 -- 172.21.15.29:0/1089818682 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc0a6820 con 0x7efdd40039c0 2024-04-25T00:36:56.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.919+0000 7efdf3117640 1 -- 172.21.15.29:0/1089818682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd40039c0 msgr2=0x7efdd4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:56.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.919+0000 7efdf3117640 1 --2- 172.21.15.29:0/1089818682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd40039c0 0x7efdd4003de0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7efddc009ae0 tx=0x7efddc01ca50 comp rx=0 tx=0).stop 2024-04-25T00:36:56.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.919+0000 7efdf3117640 1 -- 172.21.15.29:0/1089818682 shutdown_connections 2024-04-25T00:36:56.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.919+0000 7efdf3117640 1 --2- 172.21.15.29:0/1089818682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd40039c0 0x7efdd4003de0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.919+0000 7efdf3117640 1 -- 172.21.15.29:0/1089818682 >> 172.21.15.29:0/1089818682 conn(0x7efdec0b1870 msgr2=0x7efdec0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:56.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.919+0000 7efdf3117640 1 -- 172.21.15.29:0/1089818682 shutdown_connections 2024-04-25T00:36:56.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.919+0000 7efdf3117640 1 -- 172.21.15.29:0/1089818682 wait complete. 2024-04-25T00:36:56.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.919+0000 7efdf3117640 1 Processor -- start 2024-04-25T00:36:56.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.919+0000 7efdf3117640 1 -- start start 2024-04-25T00:36:56.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf3117640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd40039c0 0x7efdec1f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf3117640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdec1568b0 0x7efdec1f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf3117640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdec1f6a30 0x7efdec0b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf3117640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efdec15cbf0 con 0x7efdd40039c0 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf3117640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efdec072cf0 con 0x7efdec1f6a30 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf3117640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efdec15cd70 con 0x7efdec1568b0 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf0eb3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd40039c0 0x7efdec1f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf0eb3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd40039c0 0x7efdec1f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36742/0 (socket says 172.21.15.29:36742) 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf0eb3640 1 -- 172.21.15.29:0/1794363923 learned_addr learned my addr 172.21.15.29:0/1794363923 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdebfff640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdec1568b0 0x7efdec1f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf16b4640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdec1f6a30 0x7efdec0b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf0eb3640 1 -- 172.21.15.29:0/1794363923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdec1568b0 msgr2=0x7efdec1f64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf0eb3640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdec1568b0 0x7efdec1f64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf0eb3640 1 -- 172.21.15.29:0/1794363923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdec1f6a30 msgr2=0x7efdec0b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf0eb3640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdec1f6a30 0x7efdec0b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf0eb3640 1 -- 172.21.15.29:0/1794363923 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec1f7340 con 0x7efdd40039c0 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf0eb3640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd40039c0 0x7efdec1f3af0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7efddc009ae0 tx=0x7efdd4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efde97fa640 1 -- 172.21.15.29:0/1794363923 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc0be070 con 0x7efdd40039c0 2024-04-25T00:36:56.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efde97fa640 1 -- 172.21.15.29:0/1794363923 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efddc00f000 con 0x7efdd40039c0 2024-04-25T00:36:56.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf3117640 1 -- 172.21.15.29:0/1794363923 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdec1f7060 con 0x7efdd40039c0 2024-04-25T00:36:56.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efde97fa640 1 -- 172.21.15.29:0/1794363923 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc0b9410 con 0x7efdd40039c0 2024-04-25T00:36:56.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf3117640 1 -- 172.21.15.29:0/1794363923 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdec1f32f0 con 0x7efdd40039c0 2024-04-25T00:36:56.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdf3117640 1 -- 172.21.15.29:0/1794363923 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdec069270 con 0x7efdd40039c0 2024-04-25T00:36:56.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efde97fa640 1 -- 172.21.15.29:0/1794363923 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7efddc0c1070 con 0x7efdd40039c0 2024-04-25T00:36:56.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efde97fa640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efdd80679e0 0x7efdd8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:56.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efde97fa640 1 -- 172.21.15.29:0/1794363923 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7efddc11f830 con 0x7efdd40039c0 2024-04-25T00:36:56.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdebfff640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efdd80679e0 0x7efdd8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:56.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.923+0000 7efdebfff640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efdd80679e0 0x7efdd8069ea0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efdec13a640 tx=0x7efde0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:56.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:56.927+0000 7efde97fa640 1 -- 172.21.15.29:0/1794363923 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efddc0ac020 con 0x7efdd40039c0 2024-04-25T00:36:57.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.091+0000 7efdf3117640 1 -- 172.21.15.29:0/1794363923 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efdec15b3f0 con 0x7efdd40039c0 2024-04-25T00:36:57.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.095+0000 7efde97fa640 1 -- 172.21.15.29:0/1794363923 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7efddc016760 con 0x7efdd40039c0 2024-04-25T00:36:57.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.095+0000 7efdf3117640 1 -- 172.21.15.29:0/1794363923 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efdd80679e0 msgr2=0x7efdd8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:57.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.095+0000 7efdf3117640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efdd80679e0 0x7efdd8069ea0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efdec13a640 tx=0x7efde0009040 comp rx=0 tx=0).stop 2024-04-25T00:36:57.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.095+0000 7efdf3117640 1 -- 172.21.15.29:0/1794363923 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd40039c0 msgr2=0x7efdec1f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:57.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.095+0000 7efdf3117640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd40039c0 0x7efdec1f3af0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7efddc009ae0 tx=0x7efdd4005840 comp rx=0 tx=0).stop 2024-04-25T00:36:57.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.095+0000 7efdf3117640 1 -- 172.21.15.29:0/1794363923 shutdown_connections 2024-04-25T00:36:57.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.095+0000 7efdf3117640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efdd80679e0 0x7efdd8069ea0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.095+0000 7efdf3117640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdec1f6a30 0x7efdec0b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.095+0000 7efdf3117640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdec1568b0 0x7efdec1f64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.095+0000 7efdf3117640 1 --2- 172.21.15.29:0/1794363923 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd40039c0 0x7efdec1f3af0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.095+0000 7efdf3117640 1 -- 172.21.15.29:0/1794363923 >> 172.21.15.29:0/1794363923 conn(0x7efdec0b1870 msgr2=0x7efdec15a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:57.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.095+0000 7efdf3117640 1 -- 172.21.15.29:0/1794363923 shutdown_connections 2024-04-25T00:36:57.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.095+0000 7efdf3117640 1 -- 172.21.15.29:0/1794363923 wait complete. 2024-04-25T00:36:57.116 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2024-04-25T00:36:57.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.243+0000 7f4483a70640 1 Processor -- start 2024-04-25T00:36:57.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.243+0000 7f4483a70640 1 -- start start 2024-04-25T00:36:57.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f4483a70640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f447c0b3140 0x7f447c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f4483a70640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f447c0b4b10 0x7f447c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f4483a70640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f447c0b41e0 con 0x7f447c0b3a60 2024-04-25T00:36:57.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f4483a70640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f447c06d960 con 0x7f447c0b3140 2024-04-25T00:36:57.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f4483a70640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f447c06dc40 con 0x7f447c0b4b10 2024-04-25T00:36:57.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448200d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f447c0b3a60 0x7f447c15c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40054/0 (socket says 172.21.15.29:40054) 2024-04-25T00:36:57.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448200d640 1 -- 172.21.15.29:0/1222009715 learned_addr learned my addr 172.21.15.29:0/1222009715 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:36:57.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448100b640 1 --2- 172.21.15.29:0/1222009715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f447c0b4b10 0x7f447c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448180c640 1 --2- 172.21.15.29:0/1222009715 >> v2:172.21.15.157:3300/0 conn(0x7f447c0b3140 0x7f447c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2602455212 0 0) 0x7f447c0b41e0 con 0x7f447c0b3a60 2024-04-25T00:36:57.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4468003530 con 0x7f447c0b3a60 2024-04-25T00:36:57.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4132621164 0 0) 0x7f4468003530 con 0x7f447c0b3a60 2024-04-25T00:36:57.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f447c0b4b10 msgr2=0x7f447c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:57.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 --2- 172.21.15.29:0/1222009715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f447c0b4b10 0x7f447c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 >> v2:172.21.15.157:3300/0 conn(0x7f447c0b3140 msgr2=0x7f447c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:57.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 --2- 172.21.15.29:0/1222009715 >> v2:172.21.15.157:3300/0 conn(0x7f447c0b3140 0x7f447c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c06df20 con 0x7f447c0b3a60 2024-04-25T00:36:57.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4474001680 con 0x7f447c0b3a60 2024-04-25T00:36:57.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 >> v1:172.21.15.29:6789/0 conn(0x7f447c0b3a60 legacy=0x7f447c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:57.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 --2- 172.21.15.29:0/1222009715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44680039c0 0x7f4468003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 --2- 172.21.15.29:0/1222009715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f447c0b4b10 0x7f4468004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 --2- 172.21.15.29:0/1222009715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4468004d50 0x7f44680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f447c06d960 con 0x7f44680039c0 2024-04-25T00:36:57.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f447c06dc40 con 0x7f4468004d50 2024-04-25T00:36:57.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f446800b9d0 con 0x7f447c0b4b10 2024-04-25T00:36:57.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448180c640 1 --2- 172.21.15.29:0/1222009715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44680039c0 0x7f4468003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448200d640 1 --2- 172.21.15.29:0/1222009715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4468004d50 0x7f44680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448180c640 1 -- 172.21.15.29:0/1222009715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f447c0b4b10 msgr2=0x7f4468004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:57.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448180c640 1 --2- 172.21.15.29:0/1222009715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f447c0b4b10 0x7f4468004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448180c640 1 -- 172.21.15.29:0/1222009715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4468004d50 msgr2=0x7f44680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:57.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448180c640 1 --2- 172.21.15.29:0/1222009715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4468004d50 0x7f44680051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448180c640 1 -- 172.21.15.29:0/1222009715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4468003530 con 0x7f44680039c0 2024-04-25T00:36:57.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.247+0000 7f448200d640 1 --2- 172.21.15.29:0/1222009715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4468004d50 0x7f44680051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:57.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f448180c640 1 --2- 172.21.15.29:0/1222009715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44680039c0 0x7f4468003de0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f447800e170 tx=0x7f447801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d919d664fa4bc2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:57.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44780b2070 con 0x7f44680039c0 2024-04-25T00:36:57.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44780a5980 con 0x7f44680039c0 2024-04-25T00:36:57.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f448080a640 1 -- 172.21.15.29:0/1222009715 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478009c10 con 0x7f44680039c0 2024-04-25T00:36:57.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 -- 172.21.15.29:0/1222009715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44680039c0 msgr2=0x7f4468003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:57.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 --2- 172.21.15.29:0/1222009715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44680039c0 0x7f4468003de0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f447800e170 tx=0x7f447801cca0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f448200d640 1 -- 172.21.15.29:0/1222009715 reap_dead start 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 -- 172.21.15.29:0/1222009715 shutdown_connections 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 -- 172.21.15.29:0/1222009715 >> 172.21.15.29:0/1222009715 conn(0x7f447c0b1870 msgr2=0x7f447c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 -- 172.21.15.29:0/1222009715 shutdown_connections 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 -- 172.21.15.29:0/1222009715 wait complete. 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 Processor -- start 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 -- start start 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44680039c0 0x7f447c15aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f447c13ff00 0x7f447c140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f447c13ae40 0x7f447c13b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f447c166b60 con 0x7f447c13ae40 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1669e0 con 0x7f447c13ff00 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f4483a70640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f447c166ce0 con 0x7f44680039c0 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f448200d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f447c13ae40 0x7f447c13b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f448200d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f447c13ae40 0x7f447c13b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36764/0 (socket says 172.21.15.29:36764) 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f448200d640 1 -- 172.21.15.29:0/3486890414 learned_addr learned my addr 172.21.15.29:0/3486890414 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.251+0000 7f448100b640 1 --2- 172.21.15.29:0/3486890414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f447c13ff00 0x7f447c140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f448200d640 1 -- 172.21.15.29:0/3486890414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44680039c0 msgr2=0x7f447c15aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f448180c640 1 --2- 172.21.15.29:0/3486890414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44680039c0 0x7f447c15aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f448200d640 1 --2- 172.21.15.29:0/3486890414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44680039c0 0x7f447c15aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f448200d640 1 -- 172.21.15.29:0/3486890414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f447c13ff00 msgr2=0x7f447c140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f448200d640 1 --2- 172.21.15.29:0/3486890414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f447c13ff00 0x7f447c140380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f448200d640 1 -- 172.21.15.29:0/3486890414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c15b9c0 con 0x7f447c13ae40 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f448100b640 1 --2- 172.21.15.29:0/3486890414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f447c13ff00 0x7f447c140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f448200d640 1 --2- 172.21.15.29:0/3486890414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f447c13ae40 0x7f447c13b2c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f447c070c70 tx=0x7f447400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f44727fc640 1 -- 172.21.15.29:0/3486890414 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44740072d0 con 0x7f447c13ae40 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f44727fc640 1 -- 172.21.15.29:0/3486890414 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4474007490 con 0x7f447c13ae40 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f4483a70640 1 -- 172.21.15.29:0/3486890414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c135220 con 0x7f447c13ae40 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f44727fc640 1 -- 172.21.15.29:0/3486890414 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474004a30 con 0x7f447c13ae40 2024-04-25T00:36:57.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f4483a70640 1 -- 172.21.15.29:0/3486890414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c1357e0 con 0x7f447c13ae40 2024-04-25T00:36:57.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f4483a70640 1 -- 172.21.15.29:0/3486890414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447c069270 con 0x7f447c13ae40 2024-04-25T00:36:57.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f44727fc640 1 -- 172.21.15.29:0/3486890414 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f447401b020 con 0x7f447c13ae40 2024-04-25T00:36:57.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f44727fc640 1 --2- 172.21.15.29:0/3486890414 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4460067a10 0x7f4460069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f448180c640 1 --2- 172.21.15.29:0/3486890414 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4460067a10 0x7f4460069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f44727fc640 1 -- 172.21.15.29:0/3486890414 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f4474085a70 con 0x7f447c13ae40 2024-04-25T00:36:57.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.255+0000 7f448180c640 1 --2- 172.21.15.29:0/3486890414 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4460067a10 0x7f4460069ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f44780af000 tx=0x7f44780a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:57.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.259+0000 7f44727fc640 1 -- 172.21.15.29:0/3486890414 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f447c15b9c0 con 0x7f447c13ae40 2024-04-25T00:36:57.450 INFO:teuthology.orchestra.run.smithi029.stderr:Invalid command: invalid chars @# in abcd@# 2024-04-25T00:36:57.450 INFO:teuthology.orchestra.run.smithi029.stderr:fs volume create [] : Create a CephFS volume 2024-04-25T00:36:57.450 INFO:teuthology.orchestra.run.smithi029.stderr:Error EINVAL: invalid command 2024-04-25T00:36:57.454 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T00:36:57.455 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2024-04-25T00:36:57.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f065132a640 1 Processor -- start 2024-04-25T00:36:57.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f065132a640 1 -- start start 2024-04-25T00:36:57.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f065132a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f064c0b4d30 0x7f064c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f065132a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f064c06d210 0x7f064c0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f065132a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f064c06d6c0 con 0x7f064c15df70 2024-04-25T00:36:57.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f065132a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f064c06d980 con 0x7f064c0b4d30 2024-04-25T00:36:57.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f065132a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f064c06dc60 con 0x7f064c06d210 2024-04-25T00:36:57.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f06427fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f064c0b4d30 0x7f064c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f06427fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f064c0b4d30 0x7f064c0b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36768/0 (socket says 172.21.15.29:36768) 2024-04-25T00:36:57.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f06427fc640 1 -- 172.21.15.29:0/2948746182 learned_addr learned my addr 172.21.15.29:0/2948746182 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:57.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f064affd640 1 --1- 172.21.15.29:0/2948746182 >> v1:172.21.15.29:6789/0 conn(0x7f064c15df70 0x7f064c0b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40062/0 (socket says 172.21.15.29:40062) 2024-04-25T00:36:57.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f064b7fe640 1 --2- 172.21.15.29:0/2948746182 >> v2:172.21.15.157:3300/0 conn(0x7f064c06d210 0x7f064c0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f06427fc640 1 -- 172.21.15.29:0/2948746182 >> v2:172.21.15.157:3300/0 conn(0x7f064c06d210 msgr2=0x7f064c0ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:57.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f06427fc640 1 --2- 172.21.15.29:0/2948746182 >> v2:172.21.15.157:3300/0 conn(0x7f064c06d210 0x7f064c0ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f06427fc640 1 -- 172.21.15.29:0/2948746182 >> v1:172.21.15.29:6789/0 conn(0x7f064c15df70 legacy=0x7f064c0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:57.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f06427fc640 1 -- 172.21.15.29:0/2948746182 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c06df40 con 0x7f064c0b4d30 2024-04-25T00:36:57.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.583+0000 7f064a7fc640 1 -- 172.21.15.29:0/2948746182 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3280495408 0 0) 0x7f064c06d6c0 con 0x7f064c15df70 2024-04-25T00:36:57.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f06427fc640 1 --2- 172.21.15.29:0/2948746182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c0b4d30 0x7f064c0b3e40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f064c071d10 tx=0x7f063401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e354425fd455c96c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:57.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f064a7fc640 1 -- 172.21.15.29:0/2948746182 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06340aa650 con 0x7f064c0b4d30 2024-04-25T00:36:57.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f064a7fc640 1 -- 172.21.15.29:0/2948746182 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06340af070 con 0x7f064c0b4d30 2024-04-25T00:36:57.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f064a7fc640 1 -- 172.21.15.29:0/2948746182 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06340aaa10 con 0x7f064c0b4d30 2024-04-25T00:36:57.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f065132a640 1 -- 172.21.15.29:0/2948746182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c0b4d30 msgr2=0x7f064c0b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:57.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f065132a640 1 --2- 172.21.15.29:0/2948746182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c0b4d30 0x7f064c0b3e40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f064c071d10 tx=0x7f063401c4f0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f065132a640 1 -- 172.21.15.29:0/2948746182 shutdown_connections 2024-04-25T00:36:57.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f065132a640 1 --2- 172.21.15.29:0/2948746182 >> v2:172.21.15.157:3300/0 conn(0x7f064c06d210 0x7f064c0ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f065132a640 1 --2- 172.21.15.29:0/2948746182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c0b4d30 0x7f064c0b3e40 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f065132a640 1 -- 172.21.15.29:0/2948746182 >> 172.21.15.29:0/2948746182 conn(0x7f064c0b1870 msgr2=0x7f064c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:57.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f065132a640 1 -- 172.21.15.29:0/2948746182 shutdown_connections 2024-04-25T00:36:57.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f065132a640 1 -- 172.21.15.29:0/2948746182 wait complete. 2024-04-25T00:36:57.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f065132a640 1 Processor -- start 2024-04-25T00:36:57.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.587+0000 7f065132a640 1 -- start start 2024-04-25T00:36:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f065132a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f064c06d210 0x7f064c147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f065132a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f064c0b4d30 0x7f064c156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f065132a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c15df70 0x7f064c156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f065132a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f064c06eba0 con 0x7f064c15df70 2024-04-25T00:36:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f065132a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f064c06ea20 con 0x7f064c06d210 2024-04-25T00:36:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f065132a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f064c0ae5f0 con 0x7f064c0b4d30 2024-04-25T00:36:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f064b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c15df70 0x7f064c156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f064b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c15df70 0x7f064c156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36778/0 (socket says 172.21.15.29:36778) 2024-04-25T00:36:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f064b7fe640 1 -- 172.21.15.29:0/477413522 learned_addr learned my addr 172.21.15.29:0/477413522 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:36:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f064affd640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f064c06d210 0x7f064c147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f06427fc640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f064c0b4d30 0x7f064c156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f064b7fe640 1 -- 172.21.15.29:0/477413522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f064c0b4d30 msgr2=0x7f064c156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f064b7fe640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f064c0b4d30 0x7f064c156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f064b7fe640 1 -- 172.21.15.29:0/477413522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f064c06d210 msgr2=0x7f064c147950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f064b7fe640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f064c06d210 0x7f064c147950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f064b7fe640 1 -- 172.21.15.29:0/477413522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c14a5e0 con 0x7f064c15df70 2024-04-25T00:36:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f064b7fe640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c15df70 0x7f064c156ea0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f0644007e40 tx=0x7f064400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f0643fff640 1 -- 172.21.15.29:0/477413522 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644014620 con 0x7f064c15df70 2024-04-25T00:36:57.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f0643fff640 1 -- 172.21.15.29:0/477413522 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0644019070 con 0x7f064c15df70 2024-04-25T00:36:57.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f065132a640 1 -- 172.21.15.29:0/477413522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064c14a8c0 con 0x7f064c15df70 2024-04-25T00:36:57.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f0643fff640 1 -- 172.21.15.29:0/477413522 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644014a20 con 0x7f064c15df70 2024-04-25T00:36:57.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f065132a640 1 -- 172.21.15.29:0/477413522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064c148de0 con 0x7f064c15df70 2024-04-25T00:36:57.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f065132a640 1 -- 172.21.15.29:0/477413522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f064c069270 con 0x7f064c15df70 2024-04-25T00:36:57.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f0643fff640 1 -- 172.21.15.29:0/477413522 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f0644014bc0 con 0x7f064c15df70 2024-04-25T00:36:57.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f0643fff640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f062c0679e0 0x7f062c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:57.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f0643fff640 1 -- 172.21.15.29:0/477413522 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f0644086680 con 0x7f064c15df70 2024-04-25T00:36:57.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f064affd640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f062c0679e0 0x7f062c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:57.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.591+0000 7f064affd640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f062c0679e0 0x7f062c069ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f063c000f70 tx=0x7f063c007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:57.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.595+0000 7f0643fff640 1 -- 172.21.15.29:0/477413522 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0644057140 con 0x7f064c15df70 2024-04-25T00:36:57.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:36:57.783+0000 7f065132a640 1 -- 172.21.15.29:0/477413522 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) -- 0x7f064c0afaa0 con 0x7f062c0679e0 2024-04-25T00:37:00.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.431+0000 7f0643fff640 1 -- 172.21.15.29:0/477413522 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) ==== 60+0+0 (secure 0 0 0) 0x7f064c0afaa0 con 0x7f062c0679e0 2024-04-25T00:37:00.433 INFO:teuthology.orchestra.run.smithi029.stderr:Volume created successfully (no MDS daemons created) 2024-04-25T00:37:00.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.435+0000 7f065132a640 1 -- 172.21.15.29:0/477413522 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f062c0679e0 msgr2=0x7f062c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:00.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.435+0000 7f065132a640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f062c0679e0 0x7f062c069ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f063c000f70 tx=0x7f063c007c10 comp rx=0 tx=0).stop 2024-04-25T00:37:00.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.435+0000 7f065132a640 1 -- 172.21.15.29:0/477413522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c15df70 msgr2=0x7f064c156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:00.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.435+0000 7f065132a640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c15df70 0x7f064c156ea0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f0644007e40 tx=0x7f064400e4b0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.439+0000 7f065132a640 1 -- 172.21.15.29:0/477413522 shutdown_connections 2024-04-25T00:37:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.439+0000 7f065132a640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f062c0679e0 0x7f062c069ea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.439+0000 7f065132a640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c15df70 0x7f064c156ea0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.439+0000 7f065132a640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f064c0b4d30 0x7f064c156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.439+0000 7f065132a640 1 --2- 172.21.15.29:0/477413522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f064c06d210 0x7f064c147950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.439+0000 7f065132a640 1 -- 172.21.15.29:0/477413522 >> 172.21.15.29:0/477413522 conn(0x7f064c0b1870 msgr2=0x7f064c0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.439+0000 7f065132a640 1 -- 172.21.15.29:0/477413522 shutdown_connections 2024-04-25T00:37:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.439+0000 7f065132a640 1 -- 172.21.15.29:0/477413522 wait complete. 2024-04-25T00:37:00.461 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2024-04-25T00:37:00.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.587+0000 7fc76f666640 1 Processor -- start 2024-04-25T00:37:00.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.587+0000 7fc76f666640 1 -- start start 2024-04-25T00:37:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76f666640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7680b3140 0x7fc76806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76f666640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7680b4b10 0x7fc7680b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76f666640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7680b41e0 con 0x7fc7680b3a60 2024-04-25T00:37:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76f666640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc76806d960 con 0x7fc7680b4b10 2024-04-25T00:37:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76f666640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc76806dc40 con 0x7fc7680b3140 2024-04-25T00:37:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76dc03640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc7680b3a60 0x7fc76815c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53572/0 (socket says 172.21.15.29:53572) 2024-04-25T00:37:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76dc03640 1 -- 172.21.15.29:0/4216737620 learned_addr learned my addr 172.21.15.29:0/4216737620 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76cc01640 1 --2- 172.21.15.29:0/4216737620 >> v2:172.21.15.157:3300/0 conn(0x7fc7680b4b10 0x7fc7680b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2412840498 0 0) 0x7fc7680b41e0 con 0x7fc7680b3a60 2024-04-25T00:37:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76d402640 1 --2- 172.21.15.29:0/4216737620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7680b3140 0x7fc76806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc754003530 con 0x7fc7680b3a60 2024-04-25T00:37:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4149607959 0 0) 0x7fc754003530 con 0x7fc7680b3a60 2024-04-25T00:37:00.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7680b3140 msgr2=0x7fc76806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 --2- 172.21.15.29:0/4216737620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7680b3140 0x7fc76806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 >> v2:172.21.15.157:3300/0 conn(0x7fc7680b4b10 msgr2=0x7fc7680b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 --2- 172.21.15.29:0/4216737620 >> v2:172.21.15.157:3300/0 conn(0x7fc7680b4b10 0x7fc7680b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76806df20 con 0x7fc7680b3a60 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fc760001a00 con 0x7fc7680b3a60 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 >> v1:172.21.15.29:6789/0 conn(0x7fc7680b3a60 legacy=0x7fc76815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 --2- 172.21.15.29:0/4216737620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7540039c0 0x7fc754003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 --2- 172.21.15.29:0/4216737620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc754004810 0x7fc754004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 --2- 172.21.15.29:0/4216737620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7680b3140 0x7fc7540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc76806d960 con 0x7fc754004810 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc76806dc40 con 0x7fc7540039c0 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc75400b9d0 con 0x7fc7680b3140 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76cc01640 1 --2- 172.21.15.29:0/4216737620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc754004810 0x7fc754004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76d402640 1 --2- 172.21.15.29:0/4216737620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7540039c0 0x7fc754003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76dc03640 1 --2- 172.21.15.29:0/4216737620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7680b3140 0x7fc7540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76cc01640 1 -- 172.21.15.29:0/4216737620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7680b3140 msgr2=0x7fc7540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:00.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76cc01640 1 --2- 172.21.15.29:0/4216737620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7680b3140 0x7fc7540051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76cc01640 1 -- 172.21.15.29:0/4216737620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7540039c0 msgr2=0x7fc754003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:00.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76cc01640 1 --2- 172.21.15.29:0/4216737620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7540039c0 0x7fc754003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.591+0000 7fc76cc01640 1 -- 172.21.15.29:0/4216737620 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc754003530 con 0x7fc754004810 2024-04-25T00:37:00.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76cc01640 1 --2- 172.21.15.29:0/4216737620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc754004810 0x7fc754004c90 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fc764009ae0 tx=0x7fc76401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7aa11dfbe85f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:00.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640096b0 con 0x7fc754004810 2024-04-25T00:37:00.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7640a64c0 con 0x7fc754004810 2024-04-25T00:37:00.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc75ffff640 1 -- 172.21.15.29:0/4216737620 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640a68b0 con 0x7fc754004810 2024-04-25T00:37:00.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 -- 172.21.15.29:0/4216737620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc754004810 msgr2=0x7fc754004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:00.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 --2- 172.21.15.29:0/4216737620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc754004810 0x7fc754004c90 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fc764009ae0 tx=0x7fc76401cab0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76dc03640 1 -- 172.21.15.29:0/4216737620 reap_dead start 2024-04-25T00:37:00.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 -- 172.21.15.29:0/4216737620 shutdown_connections 2024-04-25T00:37:00.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 -- 172.21.15.29:0/4216737620 >> 172.21.15.29:0/4216737620 conn(0x7fc7680b1870 msgr2=0x7fc7680b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:00.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 -- 172.21.15.29:0/4216737620 shutdown_connections 2024-04-25T00:37:00.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 -- 172.21.15.29:0/4216737620 wait complete. 2024-04-25T00:37:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 Processor -- start 2024-04-25T00:37:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 -- start start 2024-04-25T00:37:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754004810 0x7fc76813c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc76813f030 0x7fc7680cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76813ce90 0x7fc76813d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc768145910 con 0x7fc76813f030 2024-04-25T00:37:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7680b4f70 con 0x7fc754004810 2024-04-25T00:37:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76f666640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc768145a90 con 0x7fc76813ce90 2024-04-25T00:37:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76cc01640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc76813f030 0x7fc7680cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76cc01640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc76813f030 0x7fc7680cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47980/0 (socket says 172.21.15.29:47980) 2024-04-25T00:37:00.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76cc01640 1 -- 172.21.15.29:0/3949063059 learned_addr learned my addr 172.21.15.29:0/3949063059 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76d402640 1 --2- 172.21.15.29:0/3949063059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754004810 0x7fc76813c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.595+0000 7fc76dc03640 1 --2- 172.21.15.29:0/3949063059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76813ce90 0x7fc76813d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc76cc01640 1 -- 172.21.15.29:0/3949063059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76813ce90 msgr2=0x7fc76813d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc76cc01640 1 --2- 172.21.15.29:0/3949063059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76813ce90 0x7fc76813d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc76cc01640 1 -- 172.21.15.29:0/3949063059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754004810 msgr2=0x7fc76813c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc76cc01640 1 --2- 172.21.15.29:0/3949063059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754004810 0x7fc76813c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc76cc01640 1 -- 172.21.15.29:0/3949063059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76813f8d0 con 0x7fc76813f030 2024-04-25T00:37:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc76d402640 1 --2- 172.21.15.29:0/3949063059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754004810 0x7fc76813c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc76dc03640 1 --2- 172.21.15.29:0/3949063059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76813ce90 0x7fc76813d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc76cc01640 1 --2- 172.21.15.29:0/3949063059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc76813f030 0x7fc7680cc3b0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fc7640b0040 tx=0x7fc754005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc75dffb640 1 -- 172.21.15.29:0/3949063059 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640a6670 con 0x7fc76813f030 2024-04-25T00:37:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc75dffb640 1 -- 172.21.15.29:0/3949063059 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7640bf070 con 0x7fc76813f030 2024-04-25T00:37:00.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc76f666640 1 -- 172.21.15.29:0/3949063059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76813cac0 con 0x7fc76813f030 2024-04-25T00:37:00.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc75dffb640 1 -- 172.21.15.29:0/3949063059 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640a6ce0 con 0x7fc76813f030 2024-04-25T00:37:00.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc76f666640 1 -- 172.21.15.29:0/3949063059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc768071290 con 0x7fc76813f030 2024-04-25T00:37:00.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc76f666640 1 -- 172.21.15.29:0/3949063059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc768069270 con 0x7fc76813f030 2024-04-25T00:37:00.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc75dffb640 1 -- 172.21.15.29:0/3949063059 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fc764006040 con 0x7fc76813f030 2024-04-25T00:37:00.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc75dffb640 1 --2- 172.21.15.29:0/3949063059 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc74c067a10 0x7fc74c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc75dffb640 1 -- 172.21.15.29:0/3949063059 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6543+0+0 (secure 0 0 0) 0x7fc764120ff0 con 0x7fc76813f030 2024-04-25T00:37:00.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc76d402640 1 --2- 172.21.15.29:0/3949063059 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc74c067a10 0x7fc74c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.599+0000 7fc76d402640 1 --2- 172.21.15.29:0/3949063059 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc74c067a10 0x7fc74c069ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc758009a00 tx=0x7fc758008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:00.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.603+0000 7fc75dffb640 1 -- 172.21.15.29:0/3949063059 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc7640167c0 con 0x7fc76813f030 2024-04-25T00:37:00.793 INFO:teuthology.orchestra.run.smithi029.stderr:Invalid command: invalid chars @# in abcd@# 2024-04-25T00:37:00.793 INFO:teuthology.orchestra.run.smithi029.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2024-04-25T00:37:00.793 INFO:teuthology.orchestra.run.smithi029.stderr:Error EINVAL: invalid command 2024-04-25T00:37:00.798 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T00:37:00.798 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2024-04-25T00:37:00.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.923+0000 7f564d56d640 1 Processor -- start 2024-04-25T00:37:00.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.923+0000 7f564d56d640 1 -- start start 2024-04-25T00:37:00.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f564d56d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f56480b4d30 0x7f56480b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f564d56d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564806d210 0x7f56480ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f564d56d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f564806d6c0 con 0x7f564815df70 2024-04-25T00:37:00.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f564d56d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f564806d980 con 0x7f56480b4d30 2024-04-25T00:37:00.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f564d56d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f564806dc60 con 0x7f564806d210 2024-04-25T00:37:00.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f5646ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f564815df70 0x7f56480b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53588/0 (socket says 172.21.15.29:53588) 2024-04-25T00:37:00.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f5646ffd640 1 -- 172.21.15.29:0/1349105414 learned_addr learned my addr 172.21.15.29:0/1349105414 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:00.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f56467fc640 1 --2- 172.21.15.29:0/1349105414 >> v2:172.21.15.157:3300/0 conn(0x7f56480b4d30 0x7f56480b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f56477fe640 1 --2- 172.21.15.29:0/1349105414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564806d210 0x7f56480ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f5645ffb640 1 -- 172.21.15.29:0/1349105414 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1556809469 0 0) 0x7f564806d6c0 con 0x7f564815df70 2024-04-25T00:37:00.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f5645ffb640 1 -- 172.21.15.29:0/1349105414 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5634003530 con 0x7f564815df70 2024-04-25T00:37:00.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f56477fe640 1 -- 172.21.15.29:0/1349105414 >> v2:172.21.15.157:3300/0 conn(0x7f56480b4d30 msgr2=0x7f56480b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:00.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f56477fe640 1 --2- 172.21.15.29:0/1349105414 >> v2:172.21.15.157:3300/0 conn(0x7f56480b4d30 0x7f56480b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f5645ffb640 1 -- 172.21.15.29:0/1349105414 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1621013384 0 0) 0x7f5634003530 con 0x7f564815df70 2024-04-25T00:37:00.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f56477fe640 1 -- 172.21.15.29:0/1349105414 >> v1:172.21.15.29:6789/0 conn(0x7f564815df70 legacy=0x7f56480b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:00.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f56477fe640 1 -- 172.21.15.29:0/1349105414 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564806df40 con 0x7f564806d210 2024-04-25T00:37:00.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.927+0000 7f56477fe640 1 --2- 172.21.15.29:0/1349105414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564806d210 0x7f56480ad8a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5640009d90 tx=0x7f564001c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35acb1fee301ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f5645ffb640 1 -- 172.21.15.29:0/1349105414 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56400b2070 con 0x7f564806d210 2024-04-25T00:37:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f5645ffb640 1 -- 172.21.15.29:0/1349105414 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56400a5660 con 0x7f564806d210 2024-04-25T00:37:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f5645ffb640 1 -- 172.21.15.29:0/1349105414 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638001680 con 0x7f564806d210 2024-04-25T00:37:00.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 -- 172.21.15.29:0/1349105414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564806d210 msgr2=0x7f56480ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:00.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 --2- 172.21.15.29:0/1349105414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564806d210 0x7f56480ad8a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5640009d90 tx=0x7f564001c860 comp rx=0 tx=0).stop 2024-04-25T00:37:00.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 -- 172.21.15.29:0/1349105414 shutdown_connections 2024-04-25T00:37:00.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 --2- 172.21.15.29:0/1349105414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564806d210 0x7f56480ad8a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 --2- 172.21.15.29:0/1349105414 >> v2:172.21.15.157:3300/0 conn(0x7f56480b4d30 0x7f56480b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 -- 172.21.15.29:0/1349105414 >> 172.21.15.29:0/1349105414 conn(0x7f56480b1870 msgr2=0x7f56480b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:00.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 -- 172.21.15.29:0/1349105414 shutdown_connections 2024-04-25T00:37:00.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 -- 172.21.15.29:0/1349105414 wait complete. 2024-04-25T00:37:00.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 Processor -- start 2024-04-25T00:37:00.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 -- start start 2024-04-25T00:37:00.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564806d210 0x7f5648166d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56480b4d30 0x7f564814f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564815df70 0x7f564814f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f564806eba0 con 0x7f56480b4d30 2024-04-25T00:37:00.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f564806ea20 con 0x7f564815df70 2024-04-25T00:37:00.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f56480ae5f0 con 0x7f564806d210 2024-04-25T00:37:00.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f56467fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56480b4d30 0x7f564814f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f56467fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56480b4d30 0x7f564814f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47996/0 (socket says 172.21.15.29:47996) 2024-04-25T00:37:00.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f56467fc640 1 -- 172.21.15.29:0/3717930794 learned_addr learned my addr 172.21.15.29:0/3717930794 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:00.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f5646ffd640 1 --2- 172.21.15.29:0/3717930794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564806d210 0x7f5648166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f56477fe640 1 --2- 172.21.15.29:0/3717930794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564815df70 0x7f564814f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f56467fc640 1 -- 172.21.15.29:0/3717930794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564806d210 msgr2=0x7f5648166d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:00.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f56467fc640 1 --2- 172.21.15.29:0/3717930794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564806d210 0x7f5648166d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f56467fc640 1 -- 172.21.15.29:0/3717930794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564815df70 msgr2=0x7f564814f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:00.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f56467fc640 1 --2- 172.21.15.29:0/3717930794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564815df70 0x7f564814f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:00.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f56467fc640 1 -- 172.21.15.29:0/3717930794 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564814ffa0 con 0x7f56480b4d30 2024-04-25T00:37:00.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f56477fe640 1 --2- 172.21.15.29:0/3717930794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564815df70 0x7f564814f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:00.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f56467fc640 1 --2- 172.21.15.29:0/3717930794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56480b4d30 0x7f564814f3e0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f563000e930 tx=0x7f563000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:00.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f563f7fe640 1 -- 172.21.15.29:0/3717930794 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630017070 con 0x7f56480b4d30 2024-04-25T00:37:00.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f563f7fe640 1 -- 172.21.15.29:0/3717930794 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f563000ca00 con 0x7f56480b4d30 2024-04-25T00:37:00.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f563f7fe640 1 -- 172.21.15.29:0/3717930794 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563000cdb0 con 0x7f56480b4d30 2024-04-25T00:37:00.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.931+0000 7f564d56d640 1 -- 172.21.15.29:0/3717930794 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5648150280 con 0x7f56480b4d30 2024-04-25T00:37:00.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.935+0000 7f564d56d640 1 -- 172.21.15.29:0/3717930794 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5648159c70 con 0x7f56480b4d30 2024-04-25T00:37:00.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.935+0000 7f564d56d640 1 -- 172.21.15.29:0/3717930794 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5648069270 con 0x7f56480b4d30 2024-04-25T00:37:00.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.935+0000 7f563f7fe640 1 -- 172.21.15.29:0/3717930794 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f563001a070 con 0x7f56480b4d30 2024-04-25T00:37:00.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.935+0000 7f563f7fe640 1 --2- 172.21.15.29:0/3717930794 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f56280679e0 0x7f5628069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:00.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.935+0000 7f563f7fe640 1 -- 172.21.15.29:0/3717930794 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6543+0+0 (secure 0 0 0) 0x7f5630086550 con 0x7f56480b4d30 2024-04-25T00:37:00.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.935+0000 7f5646ffd640 1 --2- 172.21.15.29:0/3717930794 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f56280679e0 0x7f5628069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:00.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.935+0000 7f5646ffd640 1 --2- 172.21.15.29:0/3717930794 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f56280679e0 0x7f5628069ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f56481fe310 tx=0x7f5638009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:00.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:00.939+0000 7f563f7fe640 1 -- 172.21.15.29:0/3717930794 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5630056d00 con 0x7f56480b4d30 2024-04-25T00:37:01.132 INFO:teuthology.orchestra.run.smithi029.stderr:Invalid command: invalid chars #@ in abcd@# 2024-04-25T00:37:01.132 INFO:teuthology.orchestra.run.smithi029.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2024-04-25T00:37:01.132 INFO:teuthology.orchestra.run.smithi029.stderr:Error EINVAL: invalid command 2024-04-25T00:37:01.136 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T00:37:01.137 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2024-04-25T00:37:01.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f172e898640 1 Processor -- start 2024-04-25T00:37:01.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f172e898640 1 -- start start 2024-04-25T00:37:01.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f172e898640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172806c530 0x7f17280b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:01.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f172e898640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f17280b1ad0 0x7f17280b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:01.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f172e898640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17280d12a0 con 0x7f17280d1a10 2024-04-25T00:37:01.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f172e898640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f172806ced0 con 0x7f17280b1ad0 2024-04-25T00:37:01.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f172e898640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f172806d1b0 con 0x7f172806c530 2024-04-25T00:37:01.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f1727fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f17280d1a10 0x7f17280d1df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53596/0 (socket says 172.21.15.29:53596) 2024-04-25T00:37:01.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f1727fff640 1 -- 172.21.15.29:0/2092802102 learned_addr learned my addr 172.21.15.29:0/2092802102 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:01.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f172ce35640 1 --2- 172.21.15.29:0/2092802102 >> v2:172.21.15.157:3300/0 conn(0x7f17280b1ad0 0x7f17280b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:01.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f17277fe640 1 --2- 172.21.15.29:0/2092802102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172806c530 0x7f17280b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:01.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f1726ffd640 1 -- 172.21.15.29:0/2092802102 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844056716 0 0) 0x7f17280d12a0 con 0x7f17280d1a10 2024-04-25T00:37:01.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f1726ffd640 1 -- 172.21.15.29:0/2092802102 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1714003530 con 0x7f17280d1a10 2024-04-25T00:37:01.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f172ce35640 1 -- 172.21.15.29:0/2092802102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172806c530 msgr2=0x7f17280b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:01.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f172ce35640 1 --2- 172.21.15.29:0/2092802102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172806c530 0x7f17280b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:01.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f172ce35640 1 -- 172.21.15.29:0/2092802102 >> v1:172.21.15.29:6789/0 conn(0x7f17280d1a10 legacy=0x7f17280d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:01.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f1726ffd640 1 -- 172.21.15.29:0/2092802102 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3481056041 0 0) 0x7f1714003530 con 0x7f17280d1a10 2024-04-25T00:37:01.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f172ce35640 1 -- 172.21.15.29:0/2092802102 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17280d0aa0 con 0x7f17280b1ad0 2024-04-25T00:37:01.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.267+0000 7f17277fe640 1 --2- 172.21.15.29:0/2092802102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172806c530 0x7f17280b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:37:01.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f172ce35640 1 --2- 172.21.15.29:0/2092802102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17280b1ad0 0x7f17280b1f60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1720009d90 tx=0x7f172001c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94ed3ee773c709b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:01.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f1726ffd640 1 -- 172.21.15.29:0/2092802102 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718001680 con 0x7f17280b1ad0 2024-04-25T00:37:01.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f1726ffd640 1 -- 172.21.15.29:0/2092802102 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17200a8460 con 0x7f17280b1ad0 2024-04-25T00:37:01.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f1726ffd640 1 -- 172.21.15.29:0/2092802102 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17200a8810 con 0x7f17280b1ad0 2024-04-25T00:37:01.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f172e898640 1 -- 172.21.15.29:0/2092802102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17280b1ad0 msgr2=0x7f17280b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:01.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f172e898640 1 --2- 172.21.15.29:0/2092802102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17280b1ad0 0x7f17280b1f60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1720009d90 tx=0x7f172001c7d0 comp rx=0 tx=0).stop 2024-04-25T00:37:01.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f172e898640 1 -- 172.21.15.29:0/2092802102 shutdown_connections 2024-04-25T00:37:01.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f172e898640 1 --2- 172.21.15.29:0/2092802102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17280b1ad0 0x7f17280b1f60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:01.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f172e898640 1 --2- 172.21.15.29:0/2092802102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172806c530 0x7f17280b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:01.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f172e898640 1 -- 172.21.15.29:0/2092802102 >> 172.21.15.29:0/2092802102 conn(0x7f172809aa30 msgr2=0x7f172809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:01.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f172e898640 1 -- 172.21.15.29:0/2092802102 shutdown_connections 2024-04-25T00:37:01.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f172e898640 1 -- 172.21.15.29:0/2092802102 wait complete. 2024-04-25T00:37:01.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f172e898640 1 Processor -- start 2024-04-25T00:37:01.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.271+0000 7f172e898640 1 -- start start 2024-04-25T00:37:01.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f172e898640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172806c530 0x7f17280c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:01.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f172e898640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17280b1ad0 0x7f1728154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:01.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f172e898640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17280d1a10 0x7f172814ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:01.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f172e898640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f172816af80 con 0x7f17280b1ad0 2024-04-25T00:37:01.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f172e898640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f172816ae00 con 0x7f172806c530 2024-04-25T00:37:01.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f172e898640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f172816b100 con 0x7f17280d1a10 2024-04-25T00:37:01.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f17277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17280b1ad0 0x7f1728154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:01.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f17277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17280b1ad0 0x7f1728154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48008/0 (socket says 172.21.15.29:48008) 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f17277fe640 1 -- 172.21.15.29:0/2440222956 learned_addr learned my addr 172.21.15.29:0/2440222956 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f172ce35640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17280d1a10 0x7f172814ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f1727fff640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172806c530 0x7f17280c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f17277fe640 1 -- 172.21.15.29:0/2440222956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17280d1a10 msgr2=0x7f172814ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f17277fe640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17280d1a10 0x7f172814ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f17277fe640 1 -- 172.21.15.29:0/2440222956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172806c530 msgr2=0x7f17280c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f17277fe640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172806c530 0x7f17280c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f17277fe640 1 -- 172.21.15.29:0/2440222956 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17280c8bd0 con 0x7f17280b1ad0 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f1727fff640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172806c530 0x7f17280c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f17277fe640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17280b1ad0 0x7f1728154470 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f171000e980 tx=0x7f171000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f1724ff9640 1 -- 172.21.15.29:0/2440222956 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710017070 con 0x7f17280b1ad0 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f1724ff9640 1 -- 172.21.15.29:0/2440222956 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f171000ca00 con 0x7f17280b1ad0 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f172e898640 1 -- 172.21.15.29:0/2440222956 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17281533d0 con 0x7f17280b1ad0 2024-04-25T00:37:01.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f1724ff9640 1 -- 172.21.15.29:0/2440222956 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171000cdd0 con 0x7f17280b1ad0 2024-04-25T00:37:01.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f172e898640 1 -- 172.21.15.29:0/2440222956 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1728153990 con 0x7f17280b1ad0 2024-04-25T00:37:01.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f172e898640 1 -- 172.21.15.29:0/2440222956 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1728069270 con 0x7f17280b1ad0 2024-04-25T00:37:01.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f1724ff9640 1 -- 172.21.15.29:0/2440222956 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f171001a070 con 0x7f17280b1ad0 2024-04-25T00:37:01.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f1724ff9640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17080679e0 0x7f1708069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:01.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f1724ff9640 1 -- 172.21.15.29:0/2440222956 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6543+0+0 (secure 0 0 0) 0x7f1710086030 con 0x7f17280b1ad0 2024-04-25T00:37:01.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.275+0000 7f1727fff640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17080679e0 0x7f1708069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:01.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.279+0000 7f1727fff640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17080679e0 0x7f1708069ea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1718007af0 tx=0x7f1718009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:01.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.283+0000 7f1724ff9640 1 -- 172.21.15.29:0/2440222956 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1710056730 con 0x7f17280b1ad0 2024-04-25T00:37:01.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:01.467+0000 7f172e898640 1 -- 172.21.15.29:0/2440222956 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7f172806cb10 con 0x7f17080679e0 2024-04-25T00:37:01.477 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-04-25T00:37:01.475+0000 7f3ac00b5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:37:01.521 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:37:04.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.495+0000 7f1724ff9640 1 -- 172.21.15.29:0/2440222956 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+78 (secure 0 0 0) 0x7f172806cb10 con 0x7f17080679e0 2024-04-25T00:37:04.499 INFO:teuthology.orchestra.run.smithi029.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2024-04-25T00:37:04.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.499+0000 7f172e898640 1 -- 172.21.15.29:0/2440222956 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17080679e0 msgr2=0x7f1708069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.499+0000 7f172e898640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17080679e0 0x7f1708069ea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1718007af0 tx=0x7f1718009040 comp rx=0 tx=0).stop 2024-04-25T00:37:04.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.499+0000 7f172e898640 1 -- 172.21.15.29:0/2440222956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17280b1ad0 msgr2=0x7f1728154470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.499+0000 7f172e898640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17280b1ad0 0x7f1728154470 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f171000e980 tx=0x7f171000ee40 comp rx=0 tx=0).stop 2024-04-25T00:37:04.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.499+0000 7f172e898640 1 -- 172.21.15.29:0/2440222956 shutdown_connections 2024-04-25T00:37:04.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.499+0000 7f172e898640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17080679e0 0x7f1708069ea0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.499+0000 7f172e898640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17280d1a10 0x7f172814ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.499+0000 7f172e898640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17280b1ad0 0x7f1728154470 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.499+0000 7f172e898640 1 --2- 172.21.15.29:0/2440222956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172806c530 0x7f17280c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.499+0000 7f172e898640 1 -- 172.21.15.29:0/2440222956 >> 172.21.15.29:0/2440222956 conn(0x7f172809aa30 msgr2=0x7f1728070640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:04.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.499+0000 7f172e898640 1 -- 172.21.15.29:0/2440222956 shutdown_connections 2024-04-25T00:37:04.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.499+0000 7f172e898640 1 -- 172.21.15.29:0/2440222956 wait complete. 2024-04-25T00:37:04.522 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:37:04.522 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:37:04.574 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:37:04.633 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:37:04.633 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:37:04.633 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:37:04.633 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:04.640 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:37:04.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.807+0000 7fab304ed640 1 Processor -- start 2024-04-25T00:37:04.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.807+0000 7fab304ed640 1 -- start start 2024-04-25T00:37:04.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab304ed640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab280b4780 0x7fab2815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab304ed640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab280b3720 con 0x7fab280cc240 2024-04-25T00:37:04.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab304ed640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab280b39c0 con 0x7fab2815e8d0 2024-04-25T00:37:04.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab304ed640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fab280b42f0 con 0x7fab280b4780 2024-04-25T00:37:04.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2ea8a640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fab280cc240 0x7fab2815c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53608/0 (socket says 172.21.15.29:53608) 2024-04-25T00:37:04.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2ea8a640 1 -- 172.21.15.29:0/1549482209 learned_addr learned my addr 172.21.15.29:0/1549482209 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:04.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3886260927 0 0) 0x7fab280b3720 con 0x7fab280cc240 2024-04-25T00:37:04.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2da88640 1 --2- 172.21.15.29:0/1549482209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab280b4780 0x7fab2815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab14003530 con 0x7fab280cc240 2024-04-25T00:37:04.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 978528920 0 0) 0x7fab14003530 con 0x7fab280cc240 2024-04-25T00:37:04.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab280b4780 msgr2=0x7fab2815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:04.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 --2- 172.21.15.29:0/1549482209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab280b4780 0x7fab2815f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 >> v1:172.21.15.157:6789/0 conn(0x7fab2815e8d0 legacy=0x7fab2815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab280b3ca0 con 0x7fab280cc240 2024-04-25T00:37:04.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fab20001680 con 0x7fab280cc240 2024-04-25T00:37:04.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 >> v1:172.21.15.29:6789/0 conn(0x7fab280cc240 legacy=0x7fab2815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 --2- 172.21.15.29:0/1549482209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab140039f0 0x7fab14003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 --2- 172.21.15.29:0/1549482209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab280b4780 0x7fab14004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 --2- 172.21.15.29:0/1549482209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab14004d50 0x7fab140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fab280b42f0 con 0x7fab140039f0 2024-04-25T00:37:04.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab1400bb60 con 0x7fab14004d50 2024-04-25T00:37:04.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fab1400bd30 con 0x7fab280b4780 2024-04-25T00:37:04.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2e289640 1 --2- 172.21.15.29:0/1549482209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab140039f0 0x7fab14003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2da88640 1 --2- 172.21.15.29:0/1549482209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab280b4780 0x7fab14004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2ea8a640 1 --2- 172.21.15.29:0/1549482209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab14004d50 0x7fab140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2e289640 1 -- 172.21.15.29:0/1549482209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab280b4780 msgr2=0x7fab14004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:04.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2e289640 1 --2- 172.21.15.29:0/1549482209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab280b4780 0x7fab14004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2e289640 1 -- 172.21.15.29:0/1549482209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab14004d50 msgr2=0x7fab140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:04.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2e289640 1 --2- 172.21.15.29:0/1549482209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab14004d50 0x7fab140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.811+0000 7fab2e289640 1 -- 172.21.15.29:0/1549482209 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab14003530 con 0x7fab140039f0 2024-04-25T00:37:04.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.815+0000 7fab2e289640 1 --2- 172.21.15.29:0/1549482209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab140039f0 0x7fab14003e10 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fab24001ea0 tx=0x7fab2401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5404b772b3772b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:04.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.815+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab240af070 con 0x7fab140039f0 2024-04-25T00:37:04.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.815+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab24003b80 con 0x7fab140039f0 2024-04-25T00:37:04.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.815+0000 7fab2d287640 1 -- 172.21.15.29:0/1549482209 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24001a90 con 0x7fab140039f0 2024-04-25T00:37:04.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.815+0000 7fab304ed640 1 -- 172.21.15.29:0/1549482209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab140039f0 msgr2=0x7fab14003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.815+0000 7fab304ed640 1 --2- 172.21.15.29:0/1549482209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab140039f0 0x7fab14003e10 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fab24001ea0 tx=0x7fab2401c780 comp rx=0 tx=0).stop 2024-04-25T00:37:04.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.815+0000 7fab2ea8a640 1 -- 172.21.15.29:0/1549482209 reap_dead start 2024-04-25T00:37:04.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.815+0000 7fab304ed640 1 -- 172.21.15.29:0/1549482209 shutdown_connections 2024-04-25T00:37:04.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.815+0000 7fab304ed640 1 -- 172.21.15.29:0/1549482209 >> 172.21.15.29:0/1549482209 conn(0x7fab280b1870 msgr2=0x7fab280b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:04.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.815+0000 7fab304ed640 1 -- 172.21.15.29:0/1549482209 shutdown_connections 2024-04-25T00:37:04.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.815+0000 7fab304ed640 1 -- 172.21.15.29:0/1549482209 wait complete. 2024-04-25T00:37:04.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.815+0000 7fab304ed640 1 Processor -- start 2024-04-25T00:37:04.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.815+0000 7fab304ed640 1 -- start start 2024-04-25T00:37:04.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab304ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab140039f0 0x7fab281654a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab304ed640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab28143310 0x7fab28160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab304ed640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab28157210 0x7fab28157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab304ed640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fab28166df0 con 0x7fab140039f0 2024-04-25T00:37:04.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab304ed640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab28166c70 con 0x7fab28143310 2024-04-25T00:37:04.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab304ed640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fab28166f70 con 0x7fab28157210 2024-04-25T00:37:04.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab2e289640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab140039f0 0x7fab281654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab2e289640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab140039f0 0x7fab281654a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48034/0 (socket says 172.21.15.29:48034) 2024-04-25T00:37:04.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab2e289640 1 -- 172.21.15.29:0/977659703 learned_addr learned my addr 172.21.15.29:0/977659703 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:04.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab2ea8a640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab28157210 0x7fab28157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab2da88640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab28143310 0x7fab28160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab2e289640 1 -- 172.21.15.29:0/977659703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab28157210 msgr2=0x7fab28157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab2e289640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab28157210 0x7fab28157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab2e289640 1 -- 172.21.15.29:0/977659703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab28143310 msgr2=0x7fab28160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:04.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab2e289640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab28143310 0x7fab28160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:04.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab2e289640 1 -- 172.21.15.29:0/977659703 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab28142270 con 0x7fab140039f0 2024-04-25T00:37:04.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab2e289640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab140039f0 0x7fab281654a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fab24005040 tx=0x7fab14005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab1effd640 1 -- 172.21.15.29:0/977659703 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab240af070 con 0x7fab140039f0 2024-04-25T00:37:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab1effd640 1 -- 172.21.15.29:0/977659703 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab240bc070 con 0x7fab140039f0 2024-04-25T00:37:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab304ed640 1 -- 172.21.15.29:0/977659703 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab28142550 con 0x7fab140039f0 2024-04-25T00:37:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab1effd640 1 -- 172.21.15.29:0/977659703 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab240052f0 con 0x7fab140039f0 2024-04-25T00:37:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab304ed640 1 -- 172.21.15.29:0/977659703 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab28142b10 con 0x7fab140039f0 2024-04-25T00:37:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab304ed640 1 -- 172.21.15.29:0/977659703 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab28069270 con 0x7fab140039f0 2024-04-25T00:37:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab1effd640 1 -- 172.21.15.29:0/977659703 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fab240a9070 con 0x7fab140039f0 2024-04-25T00:37:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab1effd640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab0c0679e0 0x7fab0c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab1effd640 1 -- 172.21.15.29:0/977659703 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fab2411ff60 con 0x7fab140039f0 2024-04-25T00:37:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab2da88640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab0c0679e0 0x7fab0c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.819+0000 7fab2da88640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab0c0679e0 0x7fab0c069ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fab2815ae60 tx=0x7fab18008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:04.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.823+0000 7fab1effd640 1 -- 172.21.15.29:0/977659703 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fab24016490 con 0x7fab140039f0 2024-04-25T00:37:04.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.991+0000 7fab304ed640 1 -- 172.21.15.29:0/977659703 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab28140ba0 con 0x7fab140039f0 2024-04-25T00:37:04.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.995+0000 7fab1effd640 1 -- 172.21.15.29:0/977659703 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16741 (secure 0 0 0) 0x7fab240f0940 con 0x7fab140039f0 2024-04-25T00:37:04.998 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:04.998 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":91,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:37:04.487226+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:36:37.482565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:36:38.176961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6833/1764147340":"2024-04-26T00:37:01.479125+0000","172.21.15.29:6832/1764147340":"2024-04-26T00:37:01.479125+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:37:05.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.999+0000 7fab304ed640 1 -- 172.21.15.29:0/977659703 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab0c0679e0 msgr2=0x7fab0c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.999+0000 7fab304ed640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab0c0679e0 0x7fab0c069ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fab2815ae60 tx=0x7fab18008040 comp rx=0 tx=0).stop 2024-04-25T00:37:05.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.999+0000 7fab304ed640 1 -- 172.21.15.29:0/977659703 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab140039f0 msgr2=0x7fab281654a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.999+0000 7fab304ed640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab140039f0 0x7fab281654a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fab24005040 tx=0x7fab14005840 comp rx=0 tx=0).stop 2024-04-25T00:37:05.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.999+0000 7fab304ed640 1 -- 172.21.15.29:0/977659703 shutdown_connections 2024-04-25T00:37:05.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.999+0000 7fab304ed640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab0c0679e0 0x7fab0c069ea0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.999+0000 7fab304ed640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab28157210 0x7fab28157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.999+0000 7fab304ed640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab28143310 0x7fab28160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.999+0000 7fab304ed640 1 --2- 172.21.15.29:0/977659703 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab140039f0 0x7fab281654a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.999+0000 7fab304ed640 1 -- 172.21.15.29:0/977659703 >> 172.21.15.29:0/977659703 conn(0x7fab280b1870 msgr2=0x7fab2815ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:05.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.999+0000 7fab304ed640 1 -- 172.21.15.29:0/977659703 shutdown_connections 2024-04-25T00:37:05.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:04.999+0000 7fab304ed640 1 -- 172.21.15.29:0/977659703 wait complete. 2024-04-25T00:37:05.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:36:37.482565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:36:38.176961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:37:05.018 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:37:05.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.139+0000 7f6e8e873640 1 Processor -- start 2024-04-25T00:37:05.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.139+0000 7f6e8e873640 1 -- start start 2024-04-25T00:37:05.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e8e873640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e8815e1b0 0x7f6e880ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e8e873640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6e880b0a00 0x7f6e880b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e8e873640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e880cf740 con 0x7f6e880b0f80 2024-04-25T00:37:05.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e8e873640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6e880b1920 con 0x7f6e880b0a00 2024-04-25T00:37:05.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e8e873640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6e880cfa20 con 0x7f6e8815e1b0 2024-04-25T00:37:05.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e8ce10640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6e880b0a00 0x7f6e880b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e7f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6e880b0f80 0x7f6e880b02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53618/0 (socket says 172.21.15.29:53618) 2024-04-25T00:37:05.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e8ce10640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6e880b0a00 0x7f6e880b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48044/0 (socket says 172.21.15.29:48044) 2024-04-25T00:37:05.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e7f7fe640 1 -- 172.21.15.29:0/66755311 learned_addr learned my addr 172.21.15.29:0/66755311 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:05.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e87fff640 1 --2- 172.21.15.29:0/66755311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e8815e1b0 0x7f6e880ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e8ce10640 1 -- 172.21.15.29:0/66755311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e8815e1b0 msgr2=0x7f6e880ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:05.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e8ce10640 1 --2- 172.21.15.29:0/66755311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e8815e1b0 0x7f6e880ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e8ce10640 1 -- 172.21.15.29:0/66755311 >> v1:172.21.15.29:6789/0 conn(0x7f6e880b0f80 legacy=0x7f6e880b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e877fe640 1 -- 172.21.15.29:0/66755311 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3330058433 0 0) 0x7f6e880cf740 con 0x7f6e880b0f80 2024-04-25T00:37:05.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e8ce10640 1 -- 172.21.15.29:0/66755311 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e880cfd00 con 0x7f6e880b0a00 2024-04-25T00:37:05.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e8ce10640 1 --2- 172.21.15.29:0/66755311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e880b0a00 0x7f6e880b13e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009d90 tx=0x7f6e8001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4cbb775c1d08cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:05.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e877fe640 1 -- 172.21.15.29:0/66755311 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800b2070 con 0x7f6e880b0a00 2024-04-25T00:37:05.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e877fe640 1 -- 172.21.15.29:0/66755311 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e800a5a00 con 0x7f6e880b0a00 2024-04-25T00:37:05.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.143+0000 7f6e877fe640 1 -- 172.21.15.29:0/66755311 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800a5dd0 con 0x7f6e880b0a00 2024-04-25T00:37:05.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 -- 172.21.15.29:0/66755311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e880b0a00 msgr2=0x7f6e880b13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 --2- 172.21.15.29:0/66755311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e880b0a00 0x7f6e880b13e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009d90 tx=0x7f6e8001c940 comp rx=0 tx=0).stop 2024-04-25T00:37:05.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 -- 172.21.15.29:0/66755311 shutdown_connections 2024-04-25T00:37:05.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 --2- 172.21.15.29:0/66755311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e880b0a00 0x7f6e880b13e0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 --2- 172.21.15.29:0/66755311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e8815e1b0 0x7f6e880ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 -- 172.21.15.29:0/66755311 >> 172.21.15.29:0/66755311 conn(0x7f6e88072930 msgr2=0x7f6e88072d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:05.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 -- 172.21.15.29:0/66755311 shutdown_connections 2024-04-25T00:37:05.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 -- 172.21.15.29:0/66755311 wait complete. 2024-04-25T00:37:05.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 Processor -- start 2024-04-25T00:37:05.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 -- start start 2024-04-25T00:37:05.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e880b0a00 0x7f6e88099520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e880b0f80 0x7f6e880b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8815e1b0 0x7f6e880ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880ad860 con 0x7f6e8815e1b0 2024-04-25T00:37:05.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880ad6e0 con 0x7f6e880b0a00 2024-04-25T00:37:05.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6e880d0ff0 con 0x7f6e880b0f80 2024-04-25T00:37:05.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8ce10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8815e1b0 0x7f6e880ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8ce10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8815e1b0 0x7f6e880ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48054/0 (socket says 172.21.15.29:48054) 2024-04-25T00:37:05.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8ce10640 1 -- 172.21.15.29:0/3217699380 learned_addr learned my addr 172.21.15.29:0/3217699380 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:05.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e7f7fe640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e880b0f80 0x7f6e880b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e87fff640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e880b0a00 0x7f6e88099520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8ce10640 1 -- 172.21.15.29:0/3217699380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e880b0f80 msgr2=0x7f6e880b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8ce10640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e880b0f80 0x7f6e880b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8ce10640 1 -- 172.21.15.29:0/3217699380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e880b0a00 msgr2=0x7f6e88099520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8ce10640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e880b0a00 0x7f6e88099520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8ce10640 1 -- 172.21.15.29:0/3217699380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e88099a60 con 0x7f6e8815e1b0 2024-04-25T00:37:05.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e87fff640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e880b0a00 0x7f6e88099520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:05.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8ce10640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8815e1b0 0x7f6e880ada30 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009d90 tx=0x7f6e800196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:05.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e857fa640 1 -- 172.21.15.29:0/3217699380 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800b2040 con 0x7f6e8815e1b0 2024-04-25T00:37:05.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e857fa640 1 -- 172.21.15.29:0/3217699380 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e800ad540 con 0x7f6e8815e1b0 2024-04-25T00:37:05.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e857fa640 1 -- 172.21.15.29:0/3217699380 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800ad910 con 0x7f6e8815e1b0 2024-04-25T00:37:05.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 -- 172.21.15.29:0/3217699380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e880c4250 con 0x7f6e8815e1b0 2024-04-25T00:37:05.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.147+0000 7f6e8e873640 1 -- 172.21.15.29:0/3217699380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e880c4810 con 0x7f6e8815e1b0 2024-04-25T00:37:05.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.151+0000 7f6e8e873640 1 -- 172.21.15.29:0/3217699380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e88075770 con 0x7f6e8815e1b0 2024-04-25T00:37:05.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.151+0000 7f6e857fa640 1 -- 172.21.15.29:0/3217699380 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f6e800c1020 con 0x7f6e8815e1b0 2024-04-25T00:37:05.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.151+0000 7f6e857fa640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e680679e0 0x7f6e68069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.151+0000 7f6e857fa640 1 -- 172.21.15.29:0/3217699380 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f6e801209d0 con 0x7f6e8815e1b0 2024-04-25T00:37:05.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.151+0000 7f6e87fff640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e680679e0 0x7f6e68069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.151+0000 7f6e87fff640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e680679e0 0x7f6e68069ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e8815b140 tx=0x7f6e78008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:05.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.155+0000 7f6e857fa640 1 -- 172.21.15.29:0/3217699380 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6e800c8020 con 0x7f6e8815e1b0 2024-04-25T00:37:05.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.315+0000 7f6e8e873640 1 -- 172.21.15.29:0/3217699380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6e88069270 con 0x7f6e8815e1b0 2024-04-25T00:37:05.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.315+0000 7f6e857fa640 1 -- 172.21.15.29:0/3217699380 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f6e80016560 con 0x7f6e8815e1b0 2024-04-25T00:37:05.319 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:37:05.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.319+0000 7f6e8e873640 1 -- 172.21.15.29:0/3217699380 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e680679e0 msgr2=0x7f6e68069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.319+0000 7f6e8e873640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e680679e0 0x7f6e68069ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e8815b140 tx=0x7f6e78008040 comp rx=0 tx=0).stop 2024-04-25T00:37:05.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.319+0000 7f6e8e873640 1 -- 172.21.15.29:0/3217699380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8815e1b0 msgr2=0x7f6e880ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.319+0000 7f6e8e873640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8815e1b0 0x7f6e880ada30 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009d90 tx=0x7f6e800196e0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.319+0000 7f6e8e873640 1 -- 172.21.15.29:0/3217699380 shutdown_connections 2024-04-25T00:37:05.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.319+0000 7f6e8e873640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e680679e0 0x7f6e68069ea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.319+0000 7f6e8e873640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8815e1b0 0x7f6e880ada30 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.319+0000 7f6e8e873640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e880b0f80 0x7f6e880b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.319+0000 7f6e8e873640 1 --2- 172.21.15.29:0/3217699380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e880b0a00 0x7f6e88099520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.319+0000 7f6e8e873640 1 -- 172.21.15.29:0/3217699380 >> 172.21.15.29:0/3217699380 conn(0x7f6e88072930 msgr2=0x7f6e8806d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:05.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.319+0000 7f6e8e873640 1 -- 172.21.15.29:0/3217699380 shutdown_connections 2024-04-25T00:37:05.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.319+0000 7f6e8e873640 1 -- 172.21.15.29:0/3217699380 wait complete. 2024-04-25T00:37:05.339 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:37:05.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.467+0000 7f253da44640 1 Processor -- start 2024-04-25T00:37:05.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.467+0000 7f253da44640 1 -- start start 2024-04-25T00:37:05.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f253da44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25380b3c30 0x7f253806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f253da44640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f25380b4b10 0x7f25380ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f253da44640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25380b41e0 con 0x7f25380b3140 2024-04-25T00:37:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f253da44640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f253806d960 con 0x7f25380b4b10 2024-04-25T00:37:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f253da44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f253806dc40 con 0x7f25380b3c30 2024-04-25T00:37:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f25367fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25380b3c30 0x7f253806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f25377fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f25380b4b10 0x7f25380ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f25367fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25380b3c30 0x7f253806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:45712/0 (socket says 172.21.15.29:45712) 2024-04-25T00:37:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f25367fc640 1 -- 172.21.15.29:0/72537514 learned_addr learned my addr 172.21.15.29:0/72537514 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f2536ffd640 1 --1- 172.21.15.29:0/72537514 >> v1:172.21.15.157:6789/0 conn(0x7f25380b3140 0x7f25380b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:49952/0 (socket says 172.21.15.29:49952) 2024-04-25T00:37:05.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f25367fc640 1 -- 172.21.15.29:0/72537514 >> v2:172.21.15.157:3300/0 conn(0x7f25380b4b10 msgr2=0x7f25380ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:05.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f25367fc640 1 --2- 172.21.15.29:0/72537514 >> v2:172.21.15.157:3300/0 conn(0x7f25380b4b10 0x7f25380ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f25367fc640 1 -- 172.21.15.29:0/72537514 >> v1:172.21.15.157:6789/0 conn(0x7f25380b3140 legacy=0x7f25380b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f25367fc640 1 -- 172.21.15.29:0/72537514 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253806df20 con 0x7f25380b3c30 2024-04-25T00:37:05.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f25367fc640 1 --2- 172.21.15.29:0/72537514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25380b3c30 0x7f253806e440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f252c009870 tx=0x7f252c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6b90a9141e8ef4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:05.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f2535ffb640 1 -- 172.21.15.29:0/72537514 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c0b2070 con 0x7f25380b3c30 2024-04-25T00:37:05.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f2535ffb640 1 -- 172.21.15.29:0/72537514 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f252c0a9070 con 0x7f25380b3c30 2024-04-25T00:37:05.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.471+0000 7f2535ffb640 1 -- 172.21.15.29:0/72537514 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c0ad880 con 0x7f25380b3c30 2024-04-25T00:37:05.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 -- 172.21.15.29:0/72537514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25380b3c30 msgr2=0x7f253806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 --2- 172.21.15.29:0/72537514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25380b3c30 0x7f253806e440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f252c009870 tx=0x7f252c01c560 comp rx=0 tx=0).stop 2024-04-25T00:37:05.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 -- 172.21.15.29:0/72537514 shutdown_connections 2024-04-25T00:37:05.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 --2- 172.21.15.29:0/72537514 >> v2:172.21.15.157:3300/0 conn(0x7f25380b4b10 0x7f25380ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 --2- 172.21.15.29:0/72537514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25380b3c30 0x7f253806e440 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 -- 172.21.15.29:0/72537514 >> 172.21.15.29:0/72537514 conn(0x7f25380b1870 msgr2=0x7f25380b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:05.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 -- 172.21.15.29:0/72537514 shutdown_connections 2024-04-25T00:37:05.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 -- 172.21.15.29:0/72537514 wait complete. 2024-04-25T00:37:05.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 Processor -- start 2024-04-25T00:37:05.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 -- start start 2024-04-25T00:37:05.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25380b3140 0x7f2538138d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25380b3c30 0x7f25380b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25380b4b10 0x7f25380b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2538168d80 con 0x7f25380b4b10 2024-04-25T00:37:05.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2538168c00 con 0x7f25380b3c30 2024-04-25T00:37:05.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f253da44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2538168f00 con 0x7f25380b3140 2024-04-25T00:37:05.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f25377fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25380b4b10 0x7f25380b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f25377fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25380b4b10 0x7f25380b0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48064/0 (socket says 172.21.15.29:48064) 2024-04-25T00:37:05.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f25377fe640 1 -- 172.21.15.29:0/4030809800 learned_addr learned my addr 172.21.15.29:0/4030809800 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:05.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f2536ffd640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25380b3140 0x7f2538138d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f25367fc640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25380b3c30 0x7f25380b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f25377fe640 1 -- 172.21.15.29:0/4030809800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25380b3140 msgr2=0x7f2538138d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f25377fe640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25380b3140 0x7f2538138d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f25377fe640 1 -- 172.21.15.29:0/4030809800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25380b3c30 msgr2=0x7f25380b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f25377fe640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25380b3c30 0x7f25380b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f25377fe640 1 -- 172.21.15.29:0/4030809800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253815b0e0 con 0x7f25380b4b10 2024-04-25T00:37:05.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f2536ffd640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25380b3140 0x7f2538138d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:05.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.475+0000 7f25367fc640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25380b3c30 0x7f25380b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:05.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.479+0000 7f25377fe640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25380b4b10 0x7f25380b0930 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f253000efc0 tx=0x7f253000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:05.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.479+0000 7f251ffff640 1 -- 172.21.15.29:0/4030809800 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530018020 con 0x7f25380b4b10 2024-04-25T00:37:05.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.479+0000 7f251ffff640 1 -- 172.21.15.29:0/4030809800 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2530014480 con 0x7f25380b4b10 2024-04-25T00:37:05.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.479+0000 7f253da44640 1 -- 172.21.15.29:0/4030809800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253815b3c0 con 0x7f25380b4b10 2024-04-25T00:37:05.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.479+0000 7f251ffff640 1 -- 172.21.15.29:0/4030809800 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530014870 con 0x7f25380b4b10 2024-04-25T00:37:05.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.479+0000 7f253da44640 1 -- 172.21.15.29:0/4030809800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253815b980 con 0x7f25380b4b10 2024-04-25T00:37:05.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.479+0000 7f253da44640 1 -- 172.21.15.29:0/4030809800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2538069270 con 0x7f25380b4b10 2024-04-25T00:37:05.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.479+0000 7f251ffff640 1 -- 172.21.15.29:0/4030809800 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f253001b020 con 0x7f25380b4b10 2024-04-25T00:37:05.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.479+0000 7f251ffff640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f25240679e0 0x7f2524069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.479+0000 7f251ffff640 1 -- 172.21.15.29:0/4030809800 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f2530086ab0 con 0x7f25380b4b10 2024-04-25T00:37:05.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.479+0000 7f2536ffd640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f25240679e0 0x7f2524069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.479+0000 7f2536ffd640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f25240679e0 0x7f2524069ea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2528000f70 tx=0x7f2528009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:05.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.483+0000 7f251ffff640 1 -- 172.21.15.29:0/4030809800 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2530022070 con 0x7f25380b4b10 2024-04-25T00:37:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.643+0000 7f253da44640 1 -- 172.21.15.29:0/4030809800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f25381429b0 con 0x7f25380b4b10 2024-04-25T00:37:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.643+0000 7f251ffff640 1 -- 172.21.15.29:0/4030809800 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f253005d070 con 0x7f25380b4b10 2024-04-25T00:37:05.646 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:37:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.647+0000 7f253da44640 1 -- 172.21.15.29:0/4030809800 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f25240679e0 msgr2=0x7f2524069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.647+0000 7f253da44640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f25240679e0 0x7f2524069ea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2528000f70 tx=0x7f2528009040 comp rx=0 tx=0).stop 2024-04-25T00:37:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.647+0000 7f253da44640 1 -- 172.21.15.29:0/4030809800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25380b4b10 msgr2=0x7f25380b0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.647+0000 7f253da44640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25380b4b10 0x7f25380b0930 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f253000efc0 tx=0x7f253000c490 comp rx=0 tx=0).stop 2024-04-25T00:37:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.647+0000 7f253da44640 1 -- 172.21.15.29:0/4030809800 shutdown_connections 2024-04-25T00:37:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.647+0000 7f253da44640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f25240679e0 0x7f2524069ea0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.647+0000 7f253da44640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25380b4b10 0x7f25380b0930 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.647+0000 7f253da44640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25380b3c30 0x7f25380b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.647+0000 7f253da44640 1 --2- 172.21.15.29:0/4030809800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25380b3140 0x7f2538138d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.647+0000 7f253da44640 1 -- 172.21.15.29:0/4030809800 >> 172.21.15.29:0/4030809800 conn(0x7f25380b1870 msgr2=0x7f25380cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.647+0000 7f253da44640 1 -- 172.21.15.29:0/4030809800 shutdown_connections 2024-04-25T00:37:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.647+0000 7f253da44640 1 -- 172.21.15.29:0/4030809800 wait complete. 2024-04-25T00:37:05.664 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:37:05.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4afba8640 1 Processor -- start 2024-04-25T00:37:05.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4afba8640 1 -- start start 2024-04-25T00:37:05.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4afba8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4a8162420 0x7fc4a8070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4afba8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a80b4e00 0x7fc4a80b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4afba8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4a8071910 con 0x7fc4a80b30b0 2024-04-25T00:37:05.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4afba8640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc4a80b3500 con 0x7fc4a8162420 2024-04-25T00:37:05.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4afba8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a8071bf0 con 0x7fc4a80b4e00 2024-04-25T00:37:05.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4ad944640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4a8162420 0x7fc4a8070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4ad944640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4a8162420 0x7fc4a8070f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48072/0 (socket says 172.21.15.29:48072) 2024-04-25T00:37:05.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4ad944640 1 -- 172.21.15.29:0/1163889315 learned_addr learned my addr 172.21.15.29:0/1163889315 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:05.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4ad143640 1 --2- 172.21.15.29:0/1163889315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a80b4e00 0x7fc4a80b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4ad944640 1 -- 172.21.15.29:0/1163889315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a80b4e00 msgr2=0x7fc4a80b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:05.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4ad944640 1 --2- 172.21.15.29:0/1163889315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a80b4e00 0x7fc4a80b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4ad944640 1 -- 172.21.15.29:0/1163889315 >> v1:172.21.15.157:6789/0 conn(0x7fc4a80b30b0 legacy=0x7fc4a8166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:05.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4ad944640 1 -- 172.21.15.29:0/1163889315 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a8071ed0 con 0x7fc4a8162420 2024-04-25T00:37:05.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.783+0000 7fc4ad143640 1 --2- 172.21.15.29:0/1163889315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a80b4e00 0x7fc4a80b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:05.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4ad944640 1 --2- 172.21.15.29:0/1163889315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a8162420 0x7fc4a8070f80 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009870 tx=0x7fc49c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9364f29f435ab3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:05.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4ac942640 1 -- 172.21.15.29:0/1163889315 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0b2070 con 0x7fc4a8162420 2024-04-25T00:37:05.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4ac942640 1 -- 172.21.15.29:0/1163889315 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc49c0a9070 con 0x7fc4a8162420 2024-04-25T00:37:05.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4ac942640 1 -- 172.21.15.29:0/1163889315 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0ad880 con 0x7fc4a8162420 2024-04-25T00:37:05.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 -- 172.21.15.29:0/1163889315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a8162420 msgr2=0x7fc4a8070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 --2- 172.21.15.29:0/1163889315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a8162420 0x7fc4a8070f80 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009870 tx=0x7fc49c01c520 comp rx=0 tx=0).stop 2024-04-25T00:37:05.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 -- 172.21.15.29:0/1163889315 shutdown_connections 2024-04-25T00:37:05.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 --2- 172.21.15.29:0/1163889315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a80b4e00 0x7fc4a80b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 --2- 172.21.15.29:0/1163889315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a8162420 0x7fc4a8070f80 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 -- 172.21.15.29:0/1163889315 >> 172.21.15.29:0/1163889315 conn(0x7fc4a80b1870 msgr2=0x7fc4a80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:05.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 -- 172.21.15.29:0/1163889315 shutdown_connections 2024-04-25T00:37:05.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 -- 172.21.15.29:0/1163889315 wait complete. 2024-04-25T00:37:05.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 Processor -- start 2024-04-25T00:37:05.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 -- start start 2024-04-25T00:37:05.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a80b30b0 0x7fc4a81616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80b4e00 0x7fc4a815ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a8162420 0x7fc4a8148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a816f540 con 0x7fc4a80b4e00 2024-04-25T00:37:05.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a816f3c0 con 0x7fc4a8162420 2024-04-25T00:37:05.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4afba8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a816f6c0 con 0x7fc4a80b30b0 2024-04-25T00:37:05.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4ad143640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80b4e00 0x7fc4a815ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4ad143640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80b4e00 0x7fc4a815ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48078/0 (socket says 172.21.15.29:48078) 2024-04-25T00:37:05.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.787+0000 7fc4ad143640 1 -- 172.21.15.29:0/1456895081 learned_addr learned my addr 172.21.15.29:0/1456895081 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:05.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc4ad143640 1 -- 172.21.15.29:0/1456895081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a80b30b0 msgr2=0x7fc4a81616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc4ad143640 1 --2- 172.21.15.29:0/1456895081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a80b30b0 0x7fc4a81616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc4ad143640 1 -- 172.21.15.29:0/1456895081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a8162420 msgr2=0x7fc4a8148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc4ad143640 1 --2- 172.21.15.29:0/1456895081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a8162420 0x7fc4a8148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc4ad143640 1 -- 172.21.15.29:0/1456895081 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a8161bf0 con 0x7fc4a80b4e00 2024-04-25T00:37:05.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc4ad143640 1 --2- 172.21.15.29:0/1456895081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80b4e00 0x7fc4a815ae20 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400cf30 tx=0x7fc4a400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:05.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc49a7fc640 1 -- 172.21.15.29:0/1456895081 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a401a070 con 0x7fc4a80b4e00 2024-04-25T00:37:05.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc49a7fc640 1 -- 172.21.15.29:0/1456895081 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a400d840 con 0x7fc4a80b4e00 2024-04-25T00:37:05.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc49a7fc640 1 -- 172.21.15.29:0/1456895081 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a400dc10 con 0x7fc4a80b4e00 2024-04-25T00:37:05.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc4afba8640 1 -- 172.21.15.29:0/1456895081 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a81fd000 con 0x7fc4a80b4e00 2024-04-25T00:37:05.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc4afba8640 1 -- 172.21.15.29:0/1456895081 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a81fd5c0 con 0x7fc4a80b4e00 2024-04-25T00:37:05.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc4afba8640 1 -- 172.21.15.29:0/1456895081 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a81fdd90 con 0x7fc4a80b4e00 2024-04-25T00:37:05.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc49a7fc640 1 -- 172.21.15.29:0/1456895081 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fc4a401d070 con 0x7fc4a80b4e00 2024-04-25T00:37:05.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc49a7fc640 1 --2- 172.21.15.29:0/1456895081 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc48c0679e0 0x7fc48c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:05.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc4ad944640 1 --2- 172.21.15.29:0/1456895081 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc48c0679e0 0x7fc48c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:05.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc49a7fc640 1 -- 172.21.15.29:0/1456895081 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fc4a40865b0 con 0x7fc4a80b4e00 2024-04-25T00:37:05.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.791+0000 7fc4ad944640 1 --2- 172.21.15.29:0/1456895081 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc48c0679e0 0x7fc48c069ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0af000 tx=0x7fc49c004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:05.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.795+0000 7fc49a7fc640 1 -- 172.21.15.29:0/1456895081 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc4a4056930 con 0x7fc4a80b4e00 2024-04-25T00:37:05.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.951+0000 7fc4afba8640 1 -- 172.21.15.29:0/1456895081 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc4a8069270 con 0x7fc4a80b4e00 2024-04-25T00:37:05.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.951+0000 7fc49a7fc640 1 -- 172.21.15.29:0/1456895081 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fc4a405ac30 con 0x7fc4a80b4e00 2024-04-25T00:37:05.954 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:37:05.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.955+0000 7fc4afba8640 1 -- 172.21.15.29:0/1456895081 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc48c0679e0 msgr2=0x7fc48c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.955+0000 7fc4afba8640 1 --2- 172.21.15.29:0/1456895081 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc48c0679e0 0x7fc48c069ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0af000 tx=0x7fc49c004740 comp rx=0 tx=0).stop 2024-04-25T00:37:05.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.955+0000 7fc4afba8640 1 -- 172.21.15.29:0/1456895081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80b4e00 msgr2=0x7fc4a815ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:05.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.955+0000 7fc4afba8640 1 --2- 172.21.15.29:0/1456895081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80b4e00 0x7fc4a815ae20 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400cf30 tx=0x7fc4a400a7d0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.955+0000 7fc4afba8640 1 -- 172.21.15.29:0/1456895081 shutdown_connections 2024-04-25T00:37:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.955+0000 7fc4afba8640 1 --2- 172.21.15.29:0/1456895081 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc48c0679e0 0x7fc48c069ea0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.955+0000 7fc4afba8640 1 --2- 172.21.15.29:0/1456895081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a8162420 0x7fc4a8148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.955+0000 7fc4afba8640 1 --2- 172.21.15.29:0/1456895081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80b4e00 0x7fc4a815ae20 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.955+0000 7fc4afba8640 1 --2- 172.21.15.29:0/1456895081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a80b30b0 0x7fc4a81616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.955+0000 7fc4afba8640 1 -- 172.21.15.29:0/1456895081 >> 172.21.15.29:0/1456895081 conn(0x7fc4a80b1870 msgr2=0x7fc4a8162aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.955+0000 7fc4afba8640 1 -- 172.21.15.29:0/1456895081 shutdown_connections 2024-04-25T00:37:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:05.955+0000 7fc4afba8640 1 -- 172.21.15.29:0/1456895081 wait complete. 2024-04-25T00:37:05.973 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:37:06.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8affe5640 1 Processor -- start 2024-04-25T00:37:06.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8affe5640 1 -- start start 2024-04-25T00:37:06.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8affe5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd8a80b3140 0x7fd8a806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8affe5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80b4b10 0x7fd8a80b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8affe5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8a80b41e0 con 0x7fd8a80b3a60 2024-04-25T00:37:06.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8affe5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd8a806d960 con 0x7fd8a80b3140 2024-04-25T00:37:06.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8affe5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a806dc40 con 0x7fd8a80b4b10 2024-04-25T00:37:06.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8ae582640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd8a80b3a60 0x7fd8a815c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53620/0 (socket says 172.21.15.29:53620) 2024-04-25T00:37:06.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8ae582640 1 -- 172.21.15.29:0/1096036522 learned_addr learned my addr 172.21.15.29:0/1096036522 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:06.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8ad580640 1 --2- 172.21.15.29:0/1096036522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80b4b10 0x7fd8a80b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8add81640 1 --2- 172.21.15.29:0/1096036522 >> v2:172.21.15.157:3300/0 conn(0x7fd8a80b3140 0x7fd8a806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1241378017 0 0) 0x7fd8a80b41e0 con 0x7fd8a80b3a60 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd894003530 con 0x7fd8a80b3a60 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3264645765 0 0) 0x7fd894003530 con 0x7fd8a80b3a60 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80b4b10 msgr2=0x7fd8a80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 --2- 172.21.15.29:0/1096036522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80b4b10 0x7fd8a80b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 >> v2:172.21.15.157:3300/0 conn(0x7fd8a80b3140 msgr2=0x7fd8a806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 --2- 172.21.15.29:0/1096036522 >> v2:172.21.15.157:3300/0 conn(0x7fd8a80b3140 0x7fd8a806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a806df20 con 0x7fd8a80b3a60 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd8a0001a00 con 0x7fd8a80b3a60 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 >> v1:172.21.15.29:6789/0 conn(0x7fd8a80b3a60 legacy=0x7fd8a815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 --2- 172.21.15.29:0/1096036522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8940039c0 0x7fd894003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 --2- 172.21.15.29:0/1096036522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80b4b10 0x7fd894004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 --2- 172.21.15.29:0/1096036522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd894004d50 0x7fd8940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a806d960 con 0x7fd894004d50 2024-04-25T00:37:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a806dc40 con 0x7fd8940039c0 2024-04-25T00:37:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd89400b9d0 con 0x7fd8a80b4b10 2024-04-25T00:37:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8ae582640 1 --2- 172.21.15.29:0/1096036522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd894004d50 0x7fd8940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8ad580640 1 --2- 172.21.15.29:0/1096036522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80b4b10 0x7fd894004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8add81640 1 --2- 172.21.15.29:0/1096036522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8940039c0 0x7fd894003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8ae582640 1 -- 172.21.15.29:0/1096036522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80b4b10 msgr2=0x7fd894004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8ae582640 1 --2- 172.21.15.29:0/1096036522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80b4b10 0x7fd894004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8ae582640 1 -- 172.21.15.29:0/1096036522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8940039c0 msgr2=0x7fd894003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8ae582640 1 --2- 172.21.15.29:0/1096036522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8940039c0 0x7fd894003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.095+0000 7fd8ae582640 1 -- 172.21.15.29:0/1096036522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd894003530 con 0x7fd894004d50 2024-04-25T00:37:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.099+0000 7fd8ae582640 1 --2- 172.21.15.29:0/1096036522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd894004d50 0x7fd8940051d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000a800 tx=0x7fd8a001d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc668d67b0048aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.099+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000a3f0 con 0x7fd894004d50 2024-04-25T00:37:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.099+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a00ad630 con 0x7fd894004d50 2024-04-25T00:37:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.099+0000 7fd8acd7f640 1 -- 172.21.15.29:0/1096036522 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00ada20 con 0x7fd894004d50 2024-04-25T00:37:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.099+0000 7fd8affe5640 1 -- 172.21.15.29:0/1096036522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd894004d50 msgr2=0x7fd8940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.099+0000 7fd8affe5640 1 --2- 172.21.15.29:0/1096036522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd894004d50 0x7fd8940051d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000a800 tx=0x7fd8a001d2e0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.099+0000 7fd8ae582640 1 -- 172.21.15.29:0/1096036522 reap_dead start 2024-04-25T00:37:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.099+0000 7fd8affe5640 1 -- 172.21.15.29:0/1096036522 shutdown_connections 2024-04-25T00:37:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.099+0000 7fd8affe5640 1 -- 172.21.15.29:0/1096036522 >> 172.21.15.29:0/1096036522 conn(0x7fd8a80b1870 msgr2=0x7fd8a80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.099+0000 7fd8affe5640 1 -- 172.21.15.29:0/1096036522 shutdown_connections 2024-04-25T00:37:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.099+0000 7fd8affe5640 1 -- 172.21.15.29:0/1096036522 wait complete. 2024-04-25T00:37:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.099+0000 7fd8affe5640 1 Processor -- start 2024-04-25T00:37:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.099+0000 7fd8affe5640 1 -- start start 2024-04-25T00:37:06.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8affe5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8a813ed80 0x7fd8a813cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8affe5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a813f160 0x7fd8a81f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8affe5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a81f4000 0x7fd8a8154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8affe5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a8145910 con 0x7fd8a813ed80 2024-04-25T00:37:06.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8affe5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a80b4f70 con 0x7fd8a813f160 2024-04-25T00:37:06.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8affe5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a8145a90 con 0x7fd8a81f4000 2024-04-25T00:37:06.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8add81640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8a813ed80 0x7fd8a813cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8add81640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8a813ed80 0x7fd8a813cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48088/0 (socket says 172.21.15.29:48088) 2024-04-25T00:37:06.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8add81640 1 -- 172.21.15.29:0/3790006097 learned_addr learned my addr 172.21.15.29:0/3790006097 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:06.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8ad580640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a813f160 0x7fd8a81f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8ae582640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a81f4000 0x7fd8a8154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8add81640 1 -- 172.21.15.29:0/3790006097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a81f4000 msgr2=0x7fd8a8154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8add81640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a81f4000 0x7fd8a8154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8add81640 1 -- 172.21.15.29:0/3790006097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a813f160 msgr2=0x7fd8a81f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8add81640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a813f160 0x7fd8a81f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8add81640 1 -- 172.21.15.29:0/3790006097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a80712a0 con 0x7fd8a813ed80 2024-04-25T00:37:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8add81640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8a813ed80 0x7fd8a813cfe0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4011540 tx=0x7fd8a4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd89e7fc640 1 -- 172.21.15.29:0/3790006097 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4015870 con 0x7fd8a813ed80 2024-04-25T00:37:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd89e7fc640 1 -- 172.21.15.29:0/3790006097 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a4015a10 con 0x7fd8a813ed80 2024-04-25T00:37:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8affe5640 1 -- 172.21.15.29:0/3790006097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a8071580 con 0x7fd8a813ed80 2024-04-25T00:37:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd89e7fc640 1 -- 172.21.15.29:0/3790006097 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4015de0 con 0x7fd8a813ed80 2024-04-25T00:37:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8affe5640 1 -- 172.21.15.29:0/3790006097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a8071b40 con 0x7fd8a813ed80 2024-04-25T00:37:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd8affe5640 1 -- 172.21.15.29:0/3790006097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a8069270 con 0x7fd8a813ed80 2024-04-25T00:37:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd89e7fc640 1 -- 172.21.15.29:0/3790006097 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fd8a400f460 con 0x7fd8a813ed80 2024-04-25T00:37:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.103+0000 7fd89e7fc640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd88c0679e0 0x7fd88c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.107+0000 7fd89e7fc640 1 -- 172.21.15.29:0/3790006097 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fd8a4086900 con 0x7fd8a813ed80 2024-04-25T00:37:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.107+0000 7fd8ad580640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd88c0679e0 0x7fd88c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.107+0000 7fd8ad580640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd88c0679e0 0x7fd88c069ea0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd8a81448f0 tx=0x7fd898008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.107+0000 7fd89e7fc640 1 -- 172.21.15.29:0/3790006097 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd8a4019030 con 0x7fd8a813ed80 2024-04-25T00:37:06.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.287+0000 7fd8affe5640 1 -- 172.21.15.29:0/3790006097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd8a806e720 con 0x7fd8a813ed80 2024-04-25T00:37:06.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.291+0000 7fd89e7fc640 1 -- 172.21.15.29:0/3790006097 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v91) ==== 80+0+122 (secure 0 0 0) 0x7fd8a4056df0 con 0x7fd8a813ed80 2024-04-25T00:37:06.293 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:6833/1764147340 2024-04-26T00:37:01.479125+0000 2024-04-25T00:37:06.293 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:6832/1764147340 2024-04-26T00:37:01.479125+0000 2024-04-25T00:37:06.293 INFO:teuthology.orchestra.run.smithi029.stderr:listed 2 entries 2024-04-25T00:37:06.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.291+0000 7fd8affe5640 1 -- 172.21.15.29:0/3790006097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd88c0679e0 msgr2=0x7fd88c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:06.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.291+0000 7fd8affe5640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd88c0679e0 0x7fd88c069ea0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd8a81448f0 tx=0x7fd898008040 comp rx=0 tx=0).stop 2024-04-25T00:37:06.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.291+0000 7fd8affe5640 1 -- 172.21.15.29:0/3790006097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8a813ed80 msgr2=0x7fd8a813cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:06.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.291+0000 7fd8affe5640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8a813ed80 0x7fd8a813cfe0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4011540 tx=0x7fd8a4011a00 comp rx=0 tx=0).stop 2024-04-25T00:37:06.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.291+0000 7fd8affe5640 1 -- 172.21.15.29:0/3790006097 shutdown_connections 2024-04-25T00:37:06.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.291+0000 7fd8affe5640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd88c0679e0 0x7fd88c069ea0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.291+0000 7fd8affe5640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a81f4000 0x7fd8a8154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.291+0000 7fd8affe5640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a813f160 0x7fd8a81f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.291+0000 7fd8affe5640 1 --2- 172.21.15.29:0/3790006097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8a813ed80 0x7fd8a813cfe0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.291+0000 7fd8affe5640 1 -- 172.21.15.29:0/3790006097 >> 172.21.15.29:0/3790006097 conn(0x7fd8a80b1870 msgr2=0x7fd8a815f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:06.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.291+0000 7fd8affe5640 1 -- 172.21.15.29:0/3790006097 shutdown_connections 2024-04-25T00:37:06.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.291+0000 7fd8affe5640 1 -- 172.21.15.29:0/3790006097 wait complete. 2024-04-25T00:37:06.312 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-04-25T00:37:06.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:06.312 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:37:06.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[46165]: fuse finished with error 0 and tester_r 0 2024-04-25T00:37:06.325 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:37:06.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:06.326 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:37:06.495 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:37:06.495 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:37:06.495 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:06.502 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:37:06.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.663+0000 7f962f12e640 1 Processor -- start 2024-04-25T00:37:06.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962f12e640 1 -- start start 2024-04-25T00:37:06.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962f12e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f96280b4780 0x7f962815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962f12e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f96280cc240 0x7f962815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962f12e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96280b3720 con 0x7f962815e8d0 2024-04-25T00:37:06.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962f12e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f96280cc650 con 0x7f96280cc240 2024-04-25T00:37:06.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962f12e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f96280b42f0 con 0x7f96280b4780 2024-04-25T00:37:06.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962d6cb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f96280cc240 0x7f962815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962d6cb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f96280cc240 0x7f962815c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48098/0 (socket says 172.21.15.29:48098) 2024-04-25T00:37:06.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962ceca640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f962815e8d0 0x7f962815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53622/0 (socket says 172.21.15.29:53622) 2024-04-25T00:37:06.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962d6cb640 1 -- 172.21.15.29:0/306187885 learned_addr learned my addr 172.21.15.29:0/306187885 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:06.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f9627fff640 1 --2- 172.21.15.29:0/306187885 >> v2:172.21.15.157:3300/0 conn(0x7f96280b4780 0x7f962815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962d6cb640 1 -- 172.21.15.29:0/306187885 >> v2:172.21.15.157:3300/0 conn(0x7f96280b4780 msgr2=0x7f962815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:06.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962d6cb640 1 --2- 172.21.15.29:0/306187885 >> v2:172.21.15.157:3300/0 conn(0x7f96280b4780 0x7f962815f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962d6cb640 1 -- 172.21.15.29:0/306187885 >> v1:172.21.15.29:6789/0 conn(0x7f962815e8d0 legacy=0x7f962815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:06.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f962d6cb640 1 -- 172.21.15.29:0/306187885 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96280b39c0 con 0x7f96280cc240 2024-04-25T00:37:06.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f96277fe640 1 -- 172.21.15.29:0/306187885 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1487265907 0 0) 0x7f96280b3720 con 0x7f962815e8d0 2024-04-25T00:37:06.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.667+0000 7f9627fff640 1 --2- 172.21.15.29:0/306187885 >> v2:172.21.15.157:3300/0 conn(0x7f96280b4780 0x7f962815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:06.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962d6cb640 1 --2- 172.21.15.29:0/306187885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96280cc240 0x7f962815c7d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9628070910 tx=0x7f962001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445eb31439044c82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:06.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f96277fe640 1 -- 172.21.15.29:0/306187885 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96200a5880 con 0x7f96280cc240 2024-04-25T00:37:06.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f96277fe640 1 -- 172.21.15.29:0/306187885 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96200a5a20 con 0x7f96280cc240 2024-04-25T00:37:06.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f96277fe640 1 -- 172.21.15.29:0/306187885 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96200a5df0 con 0x7f96280cc240 2024-04-25T00:37:06.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 -- 172.21.15.29:0/306187885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96280cc240 msgr2=0x7f962815c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:06.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 --2- 172.21.15.29:0/306187885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96280cc240 0x7f962815c7d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9628070910 tx=0x7f962001c9c0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 -- 172.21.15.29:0/306187885 shutdown_connections 2024-04-25T00:37:06.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 --2- 172.21.15.29:0/306187885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96280cc240 0x7f962815c7d0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 --2- 172.21.15.29:0/306187885 >> v2:172.21.15.157:3300/0 conn(0x7f96280b4780 0x7f962815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 -- 172.21.15.29:0/306187885 >> 172.21.15.29:0/306187885 conn(0x7f96280b1870 msgr2=0x7f96280b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:06.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 -- 172.21.15.29:0/306187885 shutdown_connections 2024-04-25T00:37:06.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 -- 172.21.15.29:0/306187885 wait complete. 2024-04-25T00:37:06.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 Processor -- start 2024-04-25T00:37:06.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 -- start start 2024-04-25T00:37:06.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96280b4780 0x7f9628159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96280cc240 0x7f96281597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962815e8d0 0x7f962813ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9628144a20 con 0x7f962815e8d0 2024-04-25T00:37:06.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f962815d520 con 0x7f96280b4780 2024-04-25T00:37:06.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962f12e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9628134c40 con 0x7f96280cc240 2024-04-25T00:37:06.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962d6cb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962815e8d0 0x7f962813ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962d6cb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962815e8d0 0x7f962813ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48112/0 (socket says 172.21.15.29:48112) 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962d6cb640 1 -- 172.21.15.29:0/3639020307 learned_addr learned my addr 172.21.15.29:0/3639020307 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f9627fff640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96280cc240 0x7f96281597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.671+0000 7f962ceca640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96280b4780 0x7f9628159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f962d6cb640 1 -- 172.21.15.29:0/3639020307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96280cc240 msgr2=0x7f96281597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f962d6cb640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96280cc240 0x7f96281597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f962d6cb640 1 -- 172.21.15.29:0/3639020307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96280b4780 msgr2=0x7f9628159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f962d6cb640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96280b4780 0x7f9628159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f962d6cb640 1 -- 172.21.15.29:0/3639020307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9628159d00 con 0x7f962815e8d0 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f962d6cb640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962815e8d0 0x7f962813ce80 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f962001cea0 tx=0x7f9620019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f96257fa640 1 -- 172.21.15.29:0/3639020307 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96200039a0 con 0x7f962815e8d0 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f96257fa640 1 -- 172.21.15.29:0/3639020307 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9620003b40 con 0x7f962815e8d0 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f962f12e640 1 -- 172.21.15.29:0/3639020307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9628134310 con 0x7f962815e8d0 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f96257fa640 1 -- 172.21.15.29:0/3639020307 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620004020 con 0x7f962815e8d0 2024-04-25T00:37:06.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f962f12e640 1 -- 172.21.15.29:0/3639020307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96281348d0 con 0x7f962815e8d0 2024-04-25T00:37:06.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f962f12e640 1 -- 172.21.15.29:0/3639020307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9628069270 con 0x7f962815e8d0 2024-04-25T00:37:06.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f96257fa640 1 -- 172.21.15.29:0/3639020307 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f96200c0030 con 0x7f962815e8d0 2024-04-25T00:37:06.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f96257fa640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f960c067a10 0x7f960c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:06.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f96257fa640 1 -- 172.21.15.29:0/3639020307 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f96201209d0 con 0x7f962815e8d0 2024-04-25T00:37:06.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f962ceca640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f960c067a10 0x7f960c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:06.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.675+0000 7f962ceca640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f960c067a10 0x7f960c069ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f961c000f70 tx=0x7f961c007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:06.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.683+0000 7f96257fa640 1 -- 172.21.15.29:0/3639020307 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f96200c7020 con 0x7f962815e8d0 2024-04-25T00:37:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.843+0000 7f962f12e640 1 -- 172.21.15.29:0/3639020307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9628071160 con 0x7f962815e8d0 2024-04-25T00:37:07.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.847+0000 7f96257fa640 1 -- 172.21.15.29:0/3639020307 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16741 (secure 0 0 0) 0x7f96200165e0 con 0x7f962815e8d0 2024-04-25T00:37:07.044 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:07.045 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":91,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:37:04.487226+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:36:37.482565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:36:38.176961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6833/1764147340":"2024-04-26T00:37:01.479125+0000","172.21.15.29:6832/1764147340":"2024-04-26T00:37:01.479125+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:37:07.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.847+0000 7f962f12e640 1 -- 172.21.15.29:0/3639020307 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f960c067a10 msgr2=0x7f960c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.847+0000 7f962f12e640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f960c067a10 0x7f960c069ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f961c000f70 tx=0x7f961c007c10 comp rx=0 tx=0).stop 2024-04-25T00:37:07.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.847+0000 7f962f12e640 1 -- 172.21.15.29:0/3639020307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962815e8d0 msgr2=0x7f962813ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.847+0000 7f962f12e640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962815e8d0 0x7f962813ce80 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f962001cea0 tx=0x7f9620019760 comp rx=0 tx=0).stop 2024-04-25T00:37:07.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.847+0000 7f962f12e640 1 -- 172.21.15.29:0/3639020307 shutdown_connections 2024-04-25T00:37:07.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.847+0000 7f962f12e640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f960c067a10 0x7f960c069ed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.847+0000 7f962f12e640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962815e8d0 0x7f962813ce80 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.847+0000 7f962f12e640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96280cc240 0x7f96281597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.847+0000 7f962f12e640 1 --2- 172.21.15.29:0/3639020307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96280b4780 0x7f9628159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.847+0000 7f962f12e640 1 -- 172.21.15.29:0/3639020307 >> 172.21.15.29:0/3639020307 conn(0x7f96280b1870 msgr2=0x7f9628072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:07.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.847+0000 7f962f12e640 1 -- 172.21.15.29:0/3639020307 shutdown_connections 2024-04-25T00:37:07.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:06.847+0000 7f962f12e640 1 -- 172.21.15.29:0/3639020307 wait complete. 2024-04-25T00:37:07.048 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:36:37.482565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:36:38.176961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:37:07.048 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:37:07.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.179+0000 7f423b32e640 1 Processor -- start 2024-04-25T00:37:07.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.179+0000 7f423b32e640 1 -- start start 2024-04-25T00:37:07.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.179+0000 7f423b32e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f42340b3c30 0x7f423406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.179+0000 7f423b32e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42340b41e0 con 0x7f42340b3140 2024-04-25T00:37:07.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.179+0000 7f423b32e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f423406d960 con 0x7f42340b4b10 2024-04-25T00:37:07.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.179+0000 7f423b32e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f423406dc40 con 0x7f42340b3c30 2024-04-25T00:37:07.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.179+0000 7f42390ca640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f42340b3140 0x7f42340b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53638/0 (socket says 172.21.15.29:53638) 2024-04-25T00:37:07.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.179+0000 7f42388c9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f42340b3c30 0x7f423406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.179+0000 7f42390ca640 1 -- 172.21.15.29:0/2270786913 learned_addr learned my addr 172.21.15.29:0/2270786913 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:07.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.179+0000 7f42388c9640 1 -- 172.21.15.29:0/2270786913 >> v1:172.21.15.157:6789/0 conn(0x7f42340b4b10 legacy=0x7f42340ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.179+0000 7f42388c9640 1 -- 172.21.15.29:0/2270786913 >> v1:172.21.15.29:6789/0 conn(0x7f42340b3140 legacy=0x7f42340b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.179+0000 7f42388c9640 1 -- 172.21.15.29:0/2270786913 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423406df20 con 0x7f42340b3c30 2024-04-25T00:37:07.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f42388c9640 1 --2- 172.21.15.29:0/2270786913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42340b3c30 0x7f423406e440 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f42240097f0 tx=0x7f422401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44379448291d0fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:07.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f422bfff640 1 -- 172.21.15.29:0/2270786913 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42240b0070 con 0x7f42340b3c30 2024-04-25T00:37:07.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f422bfff640 1 -- 172.21.15.29:0/2270786913 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42240ab600 con 0x7f42340b3c30 2024-04-25T00:37:07.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f422bfff640 1 -- 172.21.15.29:0/2270786913 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42240ab9d0 con 0x7f42340b3c30 2024-04-25T00:37:07.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 -- 172.21.15.29:0/2270786913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42340b3c30 msgr2=0x7f423406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 --2- 172.21.15.29:0/2270786913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42340b3c30 0x7f423406e440 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f42240097f0 tx=0x7f422401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 -- 172.21.15.29:0/2270786913 shutdown_connections 2024-04-25T00:37:07.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 --2- 172.21.15.29:0/2270786913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42340b3c30 0x7f423406e440 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 -- 172.21.15.29:0/2270786913 >> 172.21.15.29:0/2270786913 conn(0x7f42340b1870 msgr2=0x7f42340b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:07.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 -- 172.21.15.29:0/2270786913 shutdown_connections 2024-04-25T00:37:07.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 -- 172.21.15.29:0/2270786913 wait complete. 2024-04-25T00:37:07.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 Processor -- start 2024-04-25T00:37:07.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 -- start start 2024-04-25T00:37:07.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42340b3140 0x7f423415d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42340b3c30 0x7f423415da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42340b4b10 0x7f4234139070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4234169d90 con 0x7f42340b3c30 2024-04-25T00:37:07.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4234169c10 con 0x7f42340b4b10 2024-04-25T00:37:07.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.183+0000 7f423b32e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4234169f10 con 0x7f42340b3140 2024-04-25T00:37:07.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f42388c9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42340b3c30 0x7f423415da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f42388c9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42340b3c30 0x7f423415da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48124/0 (socket says 172.21.15.29:48124) 2024-04-25T00:37:07.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f42388c9640 1 -- 172.21.15.29:0/3794204002 learned_addr learned my addr 172.21.15.29:0/3794204002 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:07.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f42390ca640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42340b3140 0x7f423415d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f42398cb640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42340b4b10 0x7f4234139070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f42388c9640 1 -- 172.21.15.29:0/3794204002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42340b3140 msgr2=0x7f423415d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f42388c9640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42340b3140 0x7f423415d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f42388c9640 1 -- 172.21.15.29:0/3794204002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42340b4b10 msgr2=0x7f4234139070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f42388c9640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42340b4b10 0x7f4234139070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f42388c9640 1 -- 172.21.15.29:0/3794204002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423413d1a0 con 0x7f42340b3c30 2024-04-25T00:37:07.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f42388c9640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42340b3c30 0x7f423415da70 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f42240097f0 tx=0x7f4224019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:07.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f4229ffb640 1 -- 172.21.15.29:0/3794204002 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42240b0040 con 0x7f42340b3c30 2024-04-25T00:37:07.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f4229ffb640 1 -- 172.21.15.29:0/3794204002 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42240ba070 con 0x7f42340b3c30 2024-04-25T00:37:07.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f423b32e640 1 -- 172.21.15.29:0/3794204002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423413bc00 con 0x7f42340b3c30 2024-04-25T00:37:07.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f4229ffb640 1 -- 172.21.15.29:0/3794204002 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42240b4850 con 0x7f42340b3c30 2024-04-25T00:37:07.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f423b32e640 1 -- 172.21.15.29:0/3794204002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423413c180 con 0x7f42340b3c30 2024-04-25T00:37:07.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f423b32e640 1 -- 172.21.15.29:0/3794204002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4234069270 con 0x7f42340b3c30 2024-04-25T00:37:07.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f4229ffb640 1 -- 172.21.15.29:0/3794204002 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f42240ab4c0 con 0x7f42340b3c30 2024-04-25T00:37:07.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f4229ffb640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f42180679e0 0x7f4218069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f4229ffb640 1 -- 172.21.15.29:0/3794204002 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f422411fc90 con 0x7f42340b3c30 2024-04-25T00:37:07.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f42390ca640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f42180679e0 0x7f4218069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.187+0000 7f42390ca640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f42180679e0 0x7f4218069ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4230005c10 tx=0x7f4230009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:07.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.191+0000 7f4229ffb640 1 -- 172.21.15.29:0/3794204002 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4224016090 con 0x7f42340b3c30 2024-04-25T00:37:07.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.351+0000 7f423b32e640 1 -- 172.21.15.29:0/3794204002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4234154160 con 0x7f42340b3c30 2024-04-25T00:37:07.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.355+0000 7f4229ffb640 1 -- 172.21.15.29:0/3794204002 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f42240f0530 con 0x7f42340b3c30 2024-04-25T00:37:07.357 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:37:07.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.355+0000 7f423b32e640 1 -- 172.21.15.29:0/3794204002 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f42180679e0 msgr2=0x7f4218069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.355+0000 7f423b32e640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f42180679e0 0x7f4218069ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4230005c10 tx=0x7f4230009040 comp rx=0 tx=0).stop 2024-04-25T00:37:07.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.355+0000 7f423b32e640 1 -- 172.21.15.29:0/3794204002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42340b3c30 msgr2=0x7f423415da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.355+0000 7f423b32e640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42340b3c30 0x7f423415da70 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f42240097f0 tx=0x7f4224019240 comp rx=0 tx=0).stop 2024-04-25T00:37:07.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.355+0000 7f423b32e640 1 -- 172.21.15.29:0/3794204002 shutdown_connections 2024-04-25T00:37:07.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.355+0000 7f423b32e640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f42180679e0 0x7f4218069ea0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.355+0000 7f423b32e640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42340b4b10 0x7f4234139070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.355+0000 7f423b32e640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42340b3c30 0x7f423415da70 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.355+0000 7f423b32e640 1 --2- 172.21.15.29:0/3794204002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42340b3140 0x7f423415d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.355+0000 7f423b32e640 1 -- 172.21.15.29:0/3794204002 >> 172.21.15.29:0/3794204002 conn(0x7f42340b1870 msgr2=0x7f42341436f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.355+0000 7f423b32e640 1 -- 172.21.15.29:0/3794204002 shutdown_connections 2024-04-25T00:37:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.355+0000 7f423b32e640 1 -- 172.21.15.29:0/3794204002 wait complete. 2024-04-25T00:37:07.375 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:37:07.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.503+0000 7f03f088f640 1 Processor -- start 2024-04-25T00:37:07.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.503+0000 7f03f088f640 1 -- start start 2024-04-25T00:37:07.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.503+0000 7f03f088f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03ec0d2250 0x7f03ec0d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.503+0000 7f03f088f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03ec163020 con 0x7f03ec0b4b50 2024-04-25T00:37:07.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.503+0000 7f03f088f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03ec0b5050 con 0x7f03ec0d2b70 2024-04-25T00:37:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.503+0000 7f03f088f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f03ec0b0de0 con 0x7f03ec0d2250 2024-04-25T00:37:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.503+0000 7f03ead9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f03ec0b4b50 0x7f03ec162910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53652/0 (socket says 172.21.15.29:53652) 2024-04-25T00:37:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.503+0000 7f03ead9d640 1 -- 172.21.15.29:0/2103087485 learned_addr learned my addr 172.21.15.29:0/2103087485 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.503+0000 7f03ea59c640 1 --2- 172.21.15.29:0/2103087485 >> v2:172.21.15.157:3300/0 conn(0x7f03ec0d2250 0x7f03ec0d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.503+0000 7f03ea59c640 1 -- 172.21.15.29:0/2103087485 >> v1:172.21.15.157:6789/0 conn(0x7f03ec0d2b70 legacy=0x7f03ec162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.503+0000 7f03ea59c640 1 -- 172.21.15.29:0/2103087485 >> v1:172.21.15.29:6789/0 conn(0x7f03ec0b4b50 legacy=0x7f03ec162910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.503+0000 7f03ea59c640 1 -- 172.21.15.29:0/2103087485 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ec0b5330 con 0x7f03ec0d2250 2024-04-25T00:37:07.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.507+0000 7f03ea59c640 1 --2- 172.21.15.29:0/2103087485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03ec0d2250 0x7f03ec0d2630 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0b0f60 tx=0x7f03dc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43a6bdb1e145f5cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:07.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.507+0000 7f03e959a640 1 -- 172.21.15.29:0/2103087485 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0b1070 con 0x7f03ec0d2250 2024-04-25T00:37:07.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.507+0000 7f03e959a640 1 -- 172.21.15.29:0/2103087485 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03dc0ac5f0 con 0x7f03ec0d2250 2024-04-25T00:37:07.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.507+0000 7f03e959a640 1 -- 172.21.15.29:0/2103087485 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0ac9c0 con 0x7f03ec0d2250 2024-04-25T00:37:07.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.507+0000 7f03f088f640 1 -- 172.21.15.29:0/2103087485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03ec0d2250 msgr2=0x7f03ec0d2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.507+0000 7f03f088f640 1 --2- 172.21.15.29:0/2103087485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03ec0d2250 0x7f03ec0d2630 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0b0f60 tx=0x7f03dc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.507+0000 7f03f088f640 1 -- 172.21.15.29:0/2103087485 shutdown_connections 2024-04-25T00:37:07.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.507+0000 7f03f088f640 1 --2- 172.21.15.29:0/2103087485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03ec0d2250 0x7f03ec0d2630 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.507+0000 7f03f088f640 1 -- 172.21.15.29:0/2103087485 >> 172.21.15.29:0/2103087485 conn(0x7f03ec09d580 msgr2=0x7f03ec0d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:07.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.507+0000 7f03f088f640 1 -- 172.21.15.29:0/2103087485 shutdown_connections 2024-04-25T00:37:07.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.507+0000 7f03f088f640 1 -- 172.21.15.29:0/2103087485 wait complete. 2024-04-25T00:37:07.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03f088f640 1 Processor -- start 2024-04-25T00:37:07.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03f088f640 1 -- start start 2024-04-25T00:37:07.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03f088f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03ec0b4b50 0x7f03ec0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03f088f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03ec0d2250 0x7f03ec153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03f088f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03ec0d2b70 0x7f03ec1542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03f088f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec16be60 con 0x7f03ec0d2250 2024-04-25T00:37:07.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03f088f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec16bce0 con 0x7f03ec0d2b70 2024-04-25T00:37:07.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03f088f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f03ec16bfe0 con 0x7f03ec0b4b50 2024-04-25T00:37:07.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03e9d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03ec0d2250 0x7f03ec153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03e9d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03ec0d2250 0x7f03ec153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48138/0 (socket says 172.21.15.29:48138) 2024-04-25T00:37:07.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03e9d9b640 1 -- 172.21.15.29:0/946935378 learned_addr learned my addr 172.21.15.29:0/946935378 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:07.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03ead9d640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03ec0d2b70 0x7f03ec1542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03ea59c640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03ec0b4b50 0x7f03ec0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03e9d9b640 1 -- 172.21.15.29:0/946935378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03ec0b4b50 msgr2=0x7f03ec0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03e9d9b640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03ec0b4b50 0x7f03ec0cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03e9d9b640 1 -- 172.21.15.29:0/946935378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03ec0d2b70 msgr2=0x7f03ec1542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03e9d9b640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03ec0d2b70 0x7f03ec1542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03e9d9b640 1 -- 172.21.15.29:0/946935378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ec1547f0 con 0x7f03ec0d2250 2024-04-25T00:37:07.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03e9d9b640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03ec0d2250 0x7f03ec153d70 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f03e000c3a0 tx=0x7f03e000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:07.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03daffd640 1 -- 172.21.15.29:0/946935378 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e00052d0 con 0x7f03ec0d2250 2024-04-25T00:37:07.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03daffd640 1 -- 172.21.15.29:0/946935378 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e0005470 con 0x7f03ec0d2250 2024-04-25T00:37:07.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03f088f640 1 -- 172.21.15.29:0/946935378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ec15b600 con 0x7f03ec0d2250 2024-04-25T00:37:07.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03daffd640 1 -- 172.21.15.29:0/946935378 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0007260 con 0x7f03ec0d2250 2024-04-25T00:37:07.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03f088f640 1 -- 172.21.15.29:0/946935378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ec15bbc0 con 0x7f03ec0d2250 2024-04-25T00:37:07.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03f088f640 1 -- 172.21.15.29:0/946935378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ec069270 con 0x7f03ec0d2250 2024-04-25T00:37:07.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.511+0000 7f03daffd640 1 -- 172.21.15.29:0/946935378 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f03e0007400 con 0x7f03ec0d2250 2024-04-25T00:37:07.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.515+0000 7f03daffd640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f03cc0679e0 0x7f03cc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.515+0000 7f03daffd640 1 -- 172.21.15.29:0/946935378 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f03e00852d0 con 0x7f03ec0d2250 2024-04-25T00:37:07.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.515+0000 7f03ea59c640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f03cc0679e0 0x7f03cc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.515+0000 7f03ea59c640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f03cc0679e0 0x7f03cc069ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03dc0ae000 tx=0x7f03dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:07.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.519+0000 7f03daffd640 1 -- 172.21.15.29:0/946935378 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f03e0055ca0 con 0x7f03ec0d2250 2024-04-25T00:37:07.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.675+0000 7f03f088f640 1 -- 172.21.15.29:0/946935378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f03ec0d4ce0 con 0x7f03ec0d2250 2024-04-25T00:37:07.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.679+0000 7f03daffd640 1 -- 172.21.15.29:0/946935378 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f03e00555a0 con 0x7f03ec0d2250 2024-04-25T00:37:07.681 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:37:07.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.679+0000 7f03f088f640 1 -- 172.21.15.29:0/946935378 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f03cc0679e0 msgr2=0x7f03cc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.679+0000 7f03f088f640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f03cc0679e0 0x7f03cc069ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03dc0ae000 tx=0x7f03dc008040 comp rx=0 tx=0).stop 2024-04-25T00:37:07.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.679+0000 7f03f088f640 1 -- 172.21.15.29:0/946935378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03ec0d2250 msgr2=0x7f03ec153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.679+0000 7f03f088f640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03ec0d2250 0x7f03ec153d70 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f03e000c3a0 tx=0x7f03e000c860 comp rx=0 tx=0).stop 2024-04-25T00:37:07.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.679+0000 7f03f088f640 1 -- 172.21.15.29:0/946935378 shutdown_connections 2024-04-25T00:37:07.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.679+0000 7f03f088f640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f03cc0679e0 0x7f03cc069ea0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.679+0000 7f03f088f640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03ec0d2b70 0x7f03ec1542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.679+0000 7f03f088f640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03ec0d2250 0x7f03ec153d70 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.679+0000 7f03f088f640 1 --2- 172.21.15.29:0/946935378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03ec0b4b50 0x7f03ec0cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.679+0000 7f03f088f640 1 -- 172.21.15.29:0/946935378 >> 172.21.15.29:0/946935378 conn(0x7f03ec09d580 msgr2=0x7f03ec0b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:07.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.679+0000 7f03f088f640 1 -- 172.21.15.29:0/946935378 shutdown_connections 2024-04-25T00:37:07.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.679+0000 7f03f088f640 1 -- 172.21.15.29:0/946935378 wait complete. 2024-04-25T00:37:07.699 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:37:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd2df1640 1 Processor -- start 2024-04-25T00:37:07.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd2df1640 1 -- start start 2024-04-25T00:37:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd2df1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3fcc0b3c30 0x7f3fcc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd2df1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fcc0b41e0 con 0x7f3fcc0b3140 2024-04-25T00:37:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd2df1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fcc06d960 con 0x7f3fcc0b4b10 2024-04-25T00:37:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd2df1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3fcc06dc40 con 0x7f3fcc0b3c30 2024-04-25T00:37:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd0b8d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3fcc0b3140 0x7f3fcc0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41406/0 (socket says 172.21.15.29:41406) 2024-04-25T00:37:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd0b8d640 1 -- 172.21.15.29:0/2430987094 learned_addr learned my addr 172.21.15.29:0/2430987094 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcbfff640 1 --2- 172.21.15.29:0/2430987094 >> v2:172.21.15.157:3300/0 conn(0x7f3fcc0b3c30 0x7f3fcc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 315301959 0 0) 0x7f3fcc0b41e0 con 0x7f3fcc0b3140 2024-04-25T00:37:07.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fb4003530 con 0x7f3fcc0b3140 2024-04-25T00:37:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2041808804 0 0) 0x7f3fb4003530 con 0x7f3fcc0b3140 2024-04-25T00:37:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 >> v2:172.21.15.157:3300/0 conn(0x7f3fcc0b3c30 msgr2=0x7f3fcc06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 --2- 172.21.15.29:0/2430987094 >> v2:172.21.15.157:3300/0 conn(0x7f3fcc0b3c30 0x7f3fcc06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 >> v1:172.21.15.157:6789/0 conn(0x7f3fcc0b4b10 legacy=0x7f3fcc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fcc06df20 con 0x7f3fcc0b3140 2024-04-25T00:37:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f3fbc001680 con 0x7f3fcc0b3140 2024-04-25T00:37:07.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 >> v1:172.21.15.29:6789/0 conn(0x7f3fcc0b3140 legacy=0x7f3fcc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 --2- 172.21.15.29:0/2430987094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fb40039f0 0x7f3fb4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 --2- 172.21.15.29:0/2430987094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb4004810 0x7f3fb4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 --2- 172.21.15.29:0/2430987094 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fb40051d0 0x7f3fb400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc06dc40 con 0x7f3fb40051d0 2024-04-25T00:37:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb400c070 con 0x7f3fb4004810 2024-04-25T00:37:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb400c270 con 0x7f3fb40039f0 2024-04-25T00:37:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd138e640 1 --2- 172.21.15.29:0/2430987094 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fb40051d0 0x7f3fb400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd138e640 1 -- 172.21.15.29:0/2430987094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fb40039f0 msgr2=0x7f3fb4003df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:37:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd138e640 1 --2- 172.21.15.29:0/2430987094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fb40039f0 0x7f3fb4003df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd138e640 1 -- 172.21.15.29:0/2430987094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb4004810 msgr2=0x7f3fb4004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:37:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd138e640 1 --2- 172.21.15.29:0/2430987094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb4004810 0x7f3fb4004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.823+0000 7f3fd138e640 1 -- 172.21.15.29:0/2430987094 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb4003530 con 0x7f3fb40051d0 2024-04-25T00:37:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fd138e640 1 -- 172.21.15.29:0/2430987094 reap_dead start 2024-04-25T00:37:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fd138e640 1 --2- 172.21.15.29:0/2430987094 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fb40051d0 0x7f3fb400b9d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4001d50 tx=0x7f3fc401c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0adf9eab935f61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4001950 con 0x7f3fb40051d0 2024-04-25T00:37:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc4003ba0 con 0x7f3fb40051d0 2024-04-25T00:37:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fcb7fe640 1 -- 172.21.15.29:0/2430987094 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4005070 con 0x7f3fb40051d0 2024-04-25T00:37:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fd2df1640 1 -- 172.21.15.29:0/2430987094 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fb40051d0 msgr2=0x7f3fb400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fd2df1640 1 --2- 172.21.15.29:0/2430987094 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fb40051d0 0x7f3fb400b9d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4001d50 tx=0x7f3fc401c680 comp rx=0 tx=0).stop 2024-04-25T00:37:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fd2df1640 1 -- 172.21.15.29:0/2430987094 shutdown_connections 2024-04-25T00:37:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fd2df1640 1 --2- 172.21.15.29:0/2430987094 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fb40051d0 0x7f3fb400b9d0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fd2df1640 1 -- 172.21.15.29:0/2430987094 >> 172.21.15.29:0/2430987094 conn(0x7f3fcc0b1870 msgr2=0x7f3fcc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fd2df1640 1 -- 172.21.15.29:0/2430987094 shutdown_connections 2024-04-25T00:37:07.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fd2df1640 1 -- 172.21.15.29:0/2430987094 wait complete. 2024-04-25T00:37:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fd2df1640 1 Processor -- start 2024-04-25T00:37:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.827+0000 7f3fd2df1640 1 -- start start 2024-04-25T00:37:07.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd2df1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb40051d0 0x7f3fcc13c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd2df1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fcc0c4ae0 0x7f3fcc0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd2df1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fcc0c4f60 0x7f3fcc0c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd2df1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc145a90 con 0x7f3fcc0c4f60 2024-04-25T00:37:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd2df1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc145910 con 0x7f3fb40051d0 2024-04-25T00:37:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd2df1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3fcc06e440 con 0x7f3fcc0c4ae0 2024-04-25T00:37:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd138e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fcc0c4f60 0x7f3fcc0c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd138e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fcc0c4f60 0x7f3fcc0c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58912/0 (socket says 172.21.15.29:58912) 2024-04-25T00:37:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd138e640 1 -- 172.21.15.29:0/315726404 learned_addr learned my addr 172.21.15.29:0/315726404 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd138e640 1 -- 172.21.15.29:0/315726404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fcc0c4ae0 msgr2=0x7f3fcc0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd138e640 1 --2- 172.21.15.29:0/315726404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fcc0c4ae0 0x7f3fcc0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd138e640 1 -- 172.21.15.29:0/315726404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb40051d0 msgr2=0x7f3fcc13c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd138e640 1 --2- 172.21.15.29:0/315726404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb40051d0 0x7f3fcc13c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:07.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd138e640 1 -- 172.21.15.29:0/315726404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fcc13cac0 con 0x7f3fcc0c4f60 2024-04-25T00:37:07.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd138e640 1 --2- 172.21.15.29:0/315726404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fcc0c4f60 0x7f3fcc0c7310 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4007fd0 tx=0x7f3fb4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:07.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fc97fa640 1 -- 172.21.15.29:0/315726404 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4001930 con 0x7f3fcc0c4f60 2024-04-25T00:37:07.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fc97fa640 1 -- 172.21.15.29:0/315726404 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc40a8c10 con 0x7f3fcc0c4f60 2024-04-25T00:37:07.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd2df1640 1 -- 172.21.15.29:0/315726404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fcc0c54c0 con 0x7f3fcc0c4f60 2024-04-25T00:37:07.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fc97fa640 1 -- 172.21.15.29:0/315726404 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4006b40 con 0x7f3fcc0c4f60 2024-04-25T00:37:07.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd2df1640 1 -- 172.21.15.29:0/315726404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fcc0c5a80 con 0x7f3fcc0c4f60 2024-04-25T00:37:07.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd2df1640 1 -- 172.21.15.29:0/315726404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fcc069270 con 0x7f3fcc0c4f60 2024-04-25T00:37:07.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fc97fa640 1 -- 172.21.15.29:0/315726404 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f3fc40a7070 con 0x7f3fcc0c4f60 2024-04-25T00:37:07.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fc97fa640 1 --2- 172.21.15.29:0/315726404 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3fb8067a10 0x7f3fb8069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:07.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fc97fa640 1 -- 172.21.15.29:0/315726404 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f3fc411f6e0 con 0x7f3fcc0c4f60 2024-04-25T00:37:07.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd0b8d640 1 --2- 172.21.15.29:0/315726404 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3fb8067a10 0x7f3fb8069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:07.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.831+0000 7f3fd0b8d640 1 --2- 172.21.15.29:0/315726404 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3fb8067a10 0x7f3fb8069ed0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0078e0 tx=0x7f3fbc0014f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:07.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.835+0000 7f3fc97fa640 1 -- 172.21.15.29:0/315726404 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3fc4016390 con 0x7f3fcc0c4f60 2024-04-25T00:37:07.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.995+0000 7f3fd2df1640 1 -- 172.21.15.29:0/315726404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3fcc0b39a0 con 0x7f3fcc0c4f60 2024-04-25T00:37:08.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.995+0000 7f3fc97fa640 1 -- 172.21.15.29:0/315726404 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f3fc40f0030 con 0x7f3fcc0c4f60 2024-04-25T00:37:08.002 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:37:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.999+0000 7f3fd2df1640 1 -- 172.21.15.29:0/315726404 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3fb8067a10 msgr2=0x7f3fb8069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.999+0000 7f3fd2df1640 1 --2- 172.21.15.29:0/315726404 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3fb8067a10 0x7f3fb8069ed0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0078e0 tx=0x7f3fbc0014f0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.999+0000 7f3fd2df1640 1 -- 172.21.15.29:0/315726404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fcc0c4f60 msgr2=0x7f3fcc0c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.999+0000 7f3fd2df1640 1 --2- 172.21.15.29:0/315726404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fcc0c4f60 0x7f3fcc0c7310 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4007fd0 tx=0x7f3fb4005840 comp rx=0 tx=0).stop 2024-04-25T00:37:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.999+0000 7f3fd2df1640 1 -- 172.21.15.29:0/315726404 shutdown_connections 2024-04-25T00:37:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.999+0000 7f3fd2df1640 1 --2- 172.21.15.29:0/315726404 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3fb8067a10 0x7f3fb8069ed0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.999+0000 7f3fd2df1640 1 --2- 172.21.15.29:0/315726404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fcc0c4f60 0x7f3fcc0c7310 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.999+0000 7f3fd2df1640 1 --2- 172.21.15.29:0/315726404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fcc0c4ae0 0x7f3fcc0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.999+0000 7f3fd2df1640 1 --2- 172.21.15.29:0/315726404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb40051d0 0x7f3fcc13c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.999+0000 7f3fd2df1640 1 -- 172.21.15.29:0/315726404 >> 172.21.15.29:0/315726404 conn(0x7f3fcc0b1870 msgr2=0x7f3fcc15b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.999+0000 7f3fd2df1640 1 -- 172.21.15.29:0/315726404 shutdown_connections 2024-04-25T00:37:08.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:07.999+0000 7f3fd2df1640 1 -- 172.21.15.29:0/315726404 wait complete. 2024-04-25T00:37:08.019 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:37:08.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cbbd33640 1 Processor -- start 2024-04-25T00:37:08.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cbbd33640 1 -- start start 2024-04-25T00:37:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cbbd33640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2cb40cc970 0x7f2cb40b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cbbd33640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb40b17e0 con 0x7f2cb40d0380 2024-04-25T00:37:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cbbd33640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb40b1ac0 con 0x7f2cb415bf00 2024-04-25T00:37:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cbbd33640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb41600b0 con 0x7f2cb40cc970 2024-04-25T00:37:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cba2d0640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2cb40d0380 0x7f2cb40b3220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41414/0 (socket says 172.21.15.29:41414) 2024-04-25T00:37:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cba2d0640 1 -- 172.21.15.29:0/1816346778 learned_addr learned my addr 172.21.15.29:0/1816346778 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb92ce640 1 --2- 172.21.15.29:0/1816346778 >> v2:172.21.15.157:3300/0 conn(0x7f2cb40cc970 0x7f2cb40b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 8639359 0 0) 0x7f2cb40b17e0 con 0x7f2cb40d0380 2024-04-25T00:37:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ca0003530 con 0x7f2cb40d0380 2024-04-25T00:37:08.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 752184041 0 0) 0x7f2ca0003530 con 0x7f2cb40d0380 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 >> v2:172.21.15.157:3300/0 conn(0x7f2cb40cc970 msgr2=0x7f2cb40b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 --2- 172.21.15.29:0/1816346778 >> v2:172.21.15.157:3300/0 conn(0x7f2cb40cc970 0x7f2cb40b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 >> v1:172.21.15.157:6789/0 conn(0x7f2cb415bf00 legacy=0x7f2cb4160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb415fd40 con 0x7f2cb40d0380 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2cac001a00 con 0x7f2cb40d0380 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 >> v1:172.21.15.29:6789/0 conn(0x7f2cb40d0380 legacy=0x7f2cb40b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 --2- 172.21.15.29:0/1816346778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca00039c0 0x7f2ca0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 --2- 172.21.15.29:0/1816346778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca0004810 0x7f2ca0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 --2- 172.21.15.29:0/1816346778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ca00051d0 0x7f2ca000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41600b0 con 0x7f2ca00039c0 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca000c170 con 0x7f2ca00051d0 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca000c370 con 0x7f2ca0004810 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb9acf640 1 --2- 172.21.15.29:0/1816346778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca00039c0 0x7f2ca0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:08.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cba2d0640 1 --2- 172.21.15.29:0/1816346778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ca00051d0 0x7f2ca000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb92ce640 1 --2- 172.21.15.29:0/1816346778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca0004810 0x7f2ca0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb9acf640 1 -- 172.21.15.29:0/1816346778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca0004810 msgr2=0x7f2ca0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb9acf640 1 --2- 172.21.15.29:0/1816346778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca0004810 0x7f2ca0004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb9acf640 1 -- 172.21.15.29:0/1816346778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ca00051d0 msgr2=0x7f2ca000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb9acf640 1 --2- 172.21.15.29:0/1816346778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ca00051d0 0x7f2ca000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cb9acf640 1 -- 172.21.15.29:0/1816346778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca0003530 con 0x7f2ca00039c0 2024-04-25T00:37:08.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.143+0000 7f2cba2d0640 1 -- 172.21.15.29:0/1816346778 reap_dead start 2024-04-25T00:37:08.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.147+0000 7f2cb9acf640 1 --2- 172.21.15.29:0/1816346778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca00039c0 0x7f2ca0003de0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0001780 tx=0x7f2cb001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2230cda0def7d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.147+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00013a0 con 0x7f2ca00039c0 2024-04-25T00:37:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.147+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb00052f0 con 0x7f2ca00039c0 2024-04-25T00:37:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.147+0000 7f2cb8acd640 1 -- 172.21.15.29:0/1816346778 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0003840 con 0x7f2ca00039c0 2024-04-25T00:37:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.147+0000 7f2cbbd33640 1 -- 172.21.15.29:0/1816346778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca00039c0 msgr2=0x7f2ca0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.147+0000 7f2cbbd33640 1 --2- 172.21.15.29:0/1816346778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca00039c0 0x7f2ca0003de0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0001780 tx=0x7f2cb001c400 comp rx=0 tx=0).stop 2024-04-25T00:37:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.147+0000 7f2cbbd33640 1 -- 172.21.15.29:0/1816346778 shutdown_connections 2024-04-25T00:37:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.147+0000 7f2cbbd33640 1 --2- 172.21.15.29:0/1816346778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca00039c0 0x7f2ca0003de0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.147+0000 7f2cbbd33640 1 -- 172.21.15.29:0/1816346778 >> 172.21.15.29:0/1816346778 conn(0x7f2cb4072930 msgr2=0x7f2cb4072d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.147+0000 7f2cbbd33640 1 -- 172.21.15.29:0/1816346778 shutdown_connections 2024-04-25T00:37:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.147+0000 7f2cbbd33640 1 -- 172.21.15.29:0/1816346778 wait complete. 2024-04-25T00:37:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.147+0000 7f2cbbd33640 1 Processor -- start 2024-04-25T00:37:08.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.147+0000 7f2cbbd33640 1 -- start start 2024-04-25T00:37:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cbbd33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca00039c0 0x7f2cb415ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cbbd33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cb406d5e0 0x7f2cb406da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cbbd33640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cb415c4f0 0x7f2cb415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cbbd33640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb40b3bb0 con 0x7f2cb415c4f0 2024-04-25T00:37:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cbbd33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb40b3a30 con 0x7f2cb406d5e0 2024-04-25T00:37:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cbbd33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb40b3d30 con 0x7f2ca00039c0 2024-04-25T00:37:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cba2d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cb415c4f0 0x7f2cb415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cba2d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cb415c4f0 0x7f2cb415c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58932/0 (socket says 172.21.15.29:58932) 2024-04-25T00:37:08.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cba2d0640 1 -- 172.21.15.29:0/208181257 learned_addr learned my addr 172.21.15.29:0/208181257 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cb9acf640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca00039c0 0x7f2cb415ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cb92ce640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cb406d5e0 0x7f2cb406da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cba2d0640 1 -- 172.21.15.29:0/208181257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca00039c0 msgr2=0x7f2cb415ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cba2d0640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca00039c0 0x7f2cb415ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cba2d0640 1 -- 172.21.15.29:0/208181257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cb406d5e0 msgr2=0x7f2cb406da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cba2d0640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cb406d5e0 0x7f2cb406da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cba2d0640 1 -- 172.21.15.29:0/208181257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb41f7160 con 0x7f2cb415c4f0 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cb9acf640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca00039c0 0x7f2cb415ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cb92ce640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cb406d5e0 0x7f2cb406da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cba2d0640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cb415c4f0 0x7f2cb415c9a0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f2cb406d3f0 tx=0x7f2cac007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2caa7fc640 1 -- 172.21.15.29:0/208181257 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00ea80 con 0x7f2cb415c4f0 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2caa7fc640 1 -- 172.21.15.29:0/208181257 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cac018070 con 0x7f2cb415c4f0 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cbbd33640 1 -- 172.21.15.29:0/208181257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb415d220 con 0x7f2cb415c4f0 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2caa7fc640 1 -- 172.21.15.29:0/208181257 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00ee60 con 0x7f2cb415c4f0 2024-04-25T00:37:08.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cbbd33640 1 -- 172.21.15.29:0/208181257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb414d6a0 con 0x7f2cb415c4f0 2024-04-25T00:37:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cbbd33640 1 -- 172.21.15.29:0/208181257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb4075770 con 0x7f2cb415c4f0 2024-04-25T00:37:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2caa7fc640 1 -- 172.21.15.29:0/208181257 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f2cac01d460 con 0x7f2cb415c4f0 2024-04-25T00:37:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2caa7fc640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c98067a10 0x7f2c98069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2caa7fc640 1 -- 172.21.15.29:0/208181257 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f2cac086240 con 0x7f2cb415c4f0 2024-04-25T00:37:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cb9acf640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c98067a10 0x7f2c98069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:08.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.151+0000 7f2cb9acf640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c98067a10 0x7f2c98069ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0001750 tx=0x7f2cb00011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:08.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.155+0000 7f2caa7fc640 1 -- 172.21.15.29:0/208181257 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2cac056ae0 con 0x7f2cb415c4f0 2024-04-25T00:37:08.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2cbbd33640 1 -- 172.21.15.29:0/208181257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2cb4069270 con 0x7f2cb415c4f0 2024-04-25T00:37:08.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2caa7fc640 1 -- 172.21.15.29:0/208181257 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v91) ==== 80+0+122 (secure 0 0 0) 0x7f2cac0563e0 con 0x7f2cb415c4f0 2024-04-25T00:37:08.326 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:6833/1764147340 2024-04-26T00:37:01.479125+0000 2024-04-25T00:37:08.327 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29:6832/1764147340 2024-04-26T00:37:01.479125+0000 2024-04-25T00:37:08.327 INFO:teuthology.orchestra.run.smithi029.stderr:listed 2 entries 2024-04-25T00:37:08.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2cbbd33640 1 -- 172.21.15.29:0/208181257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c98067a10 msgr2=0x7f2c98069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2cbbd33640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c98067a10 0x7f2c98069ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0001750 tx=0x7f2cb00011a0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2cbbd33640 1 -- 172.21.15.29:0/208181257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cb415c4f0 msgr2=0x7f2cb415c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2cbbd33640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cb415c4f0 0x7f2cb415c9a0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f2cb406d3f0 tx=0x7f2cac007bc0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2cbbd33640 1 -- 172.21.15.29:0/208181257 shutdown_connections 2024-04-25T00:37:08.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2cbbd33640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c98067a10 0x7f2c98069ed0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2cbbd33640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cb415c4f0 0x7f2cb415c9a0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2cbbd33640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cb406d5e0 0x7f2cb406da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2cbbd33640 1 --2- 172.21.15.29:0/208181257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca00039c0 0x7f2cb415ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2cbbd33640 1 -- 172.21.15.29:0/208181257 >> 172.21.15.29:0/208181257 conn(0x7f2cb4072930 msgr2=0x7f2cb406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:08.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2cbbd33640 1 -- 172.21.15.29:0/208181257 shutdown_connections 2024-04-25T00:37:08.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.323+0000 7f2cbbd33640 1 -- 172.21.15.29:0/208181257 wait complete. 2024-04-25T00:37:08.343 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-25T00:37:08.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:08.343 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T00:37:08.356 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34437]: fuse finished with error 0 and tester_r 0 2024-04-25T00:37:08.357 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:37:08.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:08.357 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:37:08.428 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:37:08.428 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:37:08.429 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:08.434 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:37:08.434 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:37:08.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:08.435 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:37:08.480 INFO:teuthology.orchestra.run.smithi029.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T00:37:08.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:37:08.481 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:37:08.482 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:37:08.482 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:08.487 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:37:08.487 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:37:08.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:08.488 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:37:08.539 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T00:37:08.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:37:08.541 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:37:08.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.667+0000 7f9d691cf640 1 Processor -- start 2024-04-25T00:37:08.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.667+0000 7f9d691cf640 1 -- start start 2024-04-25T00:37:08.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d691cf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9d6415e8d0 0x7f9d640cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d691cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d6415f3c0 0x7f9d6415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d691cf640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d640b3720 con 0x7f9d640b4780 2024-04-25T00:37:08.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d691cf640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9d640b42f0 con 0x7f9d6415e8d0 2024-04-25T00:37:08.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d691cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9d640b39c0 con 0x7f9d6415f3c0 2024-04-25T00:37:08.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d62d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9d6415e8d0 0x7f9d640cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:08.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d62d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9d6415e8d0 0x7f9d640cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58942/0 (socket says 172.21.15.29:58942) 2024-04-25T00:37:08.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d5bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9d640b4780 0x7f9d6415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41420/0 (socket says 172.21.15.29:41420) 2024-04-25T00:37:08.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d62d9d640 1 -- 172.21.15.29:0/568801429 learned_addr learned my addr 172.21.15.29:0/568801429 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:08.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d6359e640 1 --2- 172.21.15.29:0/568801429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d6415f3c0 0x7f9d6415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:08.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d62d9d640 1 -- 172.21.15.29:0/568801429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d6415f3c0 msgr2=0x7f9d6415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:08.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d62d9d640 1 --2- 172.21.15.29:0/568801429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d6415f3c0 0x7f9d6415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d62d9d640 1 -- 172.21.15.29:0/568801429 >> v1:172.21.15.29:6789/0 conn(0x7f9d640b4780 legacy=0x7f9d6415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d62d9d640 1 -- 172.21.15.29:0/568801429 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d640b3ca0 con 0x7f9d6415e8d0 2024-04-25T00:37:08.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d6259c640 1 -- 172.21.15.29:0/568801429 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 739036476 0 0) 0x7f9d640b3720 con 0x7f9d640b4780 2024-04-25T00:37:08.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.671+0000 7f9d6359e640 1 --2- 172.21.15.29:0/568801429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d6415f3c0 0x7f9d6415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:08.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d62d9d640 1 --2- 172.21.15.29:0/568801429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d6415e8d0 0x7f9d640cc240 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400bb10 tx=0x7f9d5401ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44baed5b057f7a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:08.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d6259c640 1 -- 172.21.15.29:0/568801429 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d540b4070 con 0x7f9d6415e8d0 2024-04-25T00:37:08.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d6259c640 1 -- 172.21.15.29:0/568801429 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d540a7be0 con 0x7f9d6415e8d0 2024-04-25T00:37:08.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d6259c640 1 -- 172.21.15.29:0/568801429 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d540af440 con 0x7f9d6415e8d0 2024-04-25T00:37:08.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 -- 172.21.15.29:0/568801429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d6415e8d0 msgr2=0x7f9d640cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 --2- 172.21.15.29:0/568801429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d6415e8d0 0x7f9d640cc240 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400bb10 tx=0x7f9d5401ec00 comp rx=0 tx=0).stop 2024-04-25T00:37:08.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 -- 172.21.15.29:0/568801429 shutdown_connections 2024-04-25T00:37:08.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 --2- 172.21.15.29:0/568801429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d6415f3c0 0x7f9d6415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 --2- 172.21.15.29:0/568801429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d6415e8d0 0x7f9d640cc240 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 -- 172.21.15.29:0/568801429 >> 172.21.15.29:0/568801429 conn(0x7f9d640b1870 msgr2=0x7f9d640b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:08.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 -- 172.21.15.29:0/568801429 shutdown_connections 2024-04-25T00:37:08.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 -- 172.21.15.29:0/568801429 wait complete. 2024-04-25T00:37:08.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 Processor -- start 2024-04-25T00:37:08.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 -- start start 2024-04-25T00:37:08.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d640b4780 0x7f9d641594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d6415e8d0 0x7f9d64158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d6415f3c0 0x7f9d64139340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6415d6a0 con 0x7f9d6415e8d0 2024-04-25T00:37:08.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6415d520 con 0x7f9d640b4780 2024-04-25T00:37:08.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9d6406d5d0 con 0x7f9d6415f3c0 2024-04-25T00:37:08.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d5bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d6415e8d0 0x7f9d64158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:08.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d5bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d6415e8d0 0x7f9d64158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58944/0 (socket says 172.21.15.29:58944) 2024-04-25T00:37:08.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d5bfff640 1 -- 172.21.15.29:0/2676867808 learned_addr learned my addr 172.21.15.29:0/2676867808 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:08.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d5bfff640 1 -- 172.21.15.29:0/2676867808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d6415f3c0 msgr2=0x7f9d64139340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:37:08.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d5bfff640 1 --2- 172.21.15.29:0/2676867808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d6415f3c0 0x7f9d64139340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d5bfff640 1 -- 172.21.15.29:0/2676867808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d640b4780 msgr2=0x7f9d641594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d5bfff640 1 --2- 172.21.15.29:0/2676867808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d640b4780 0x7f9d641594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d5bfff640 1 -- 172.21.15.29:0/2676867808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d64159ae0 con 0x7f9d6415e8d0 2024-04-25T00:37:08.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d5bfff640 1 --2- 172.21.15.29:0/2676867808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d6415e8d0 0x7f9d64158e00 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0011f0 tx=0x7f9d4c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:08.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d5b7fe640 1 -- 172.21.15.29:0/2676867808 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c018070 con 0x7f9d6415e8d0 2024-04-25T00:37:08.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d5b7fe640 1 -- 172.21.15.29:0/2676867808 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d4c0074e0 con 0x7f9d6415e8d0 2024-04-25T00:37:08.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 -- 172.21.15.29:0/2676867808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d640ce640 con 0x7f9d6415e8d0 2024-04-25T00:37:08.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d5b7fe640 1 -- 172.21.15.29:0/2676867808 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c004a10 con 0x7f9d6415e8d0 2024-04-25T00:37:08.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.675+0000 7f9d691cf640 1 -- 172.21.15.29:0/2676867808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d640cec00 con 0x7f9d6415e8d0 2024-04-25T00:37:08.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.679+0000 7f9d691cf640 1 -- 172.21.15.29:0/2676867808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d64069270 con 0x7f9d6415e8d0 2024-04-25T00:37:08.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.679+0000 7f9d5b7fe640 1 -- 172.21.15.29:0/2676867808 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f9d4c01b070 con 0x7f9d6415e8d0 2024-04-25T00:37:08.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.679+0000 7f9d5b7fe640 1 --2- 172.21.15.29:0/2676867808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d440679e0 0x7f9d44069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:08.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.679+0000 7f9d5b7fe640 1 -- 172.21.15.29:0/2676867808 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f9d4c085110 con 0x7f9d6415e8d0 2024-04-25T00:37:08.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.679+0000 7f9d62d9d640 1 --2- 172.21.15.29:0/2676867808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d440679e0 0x7f9d44069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:08.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.679+0000 7f9d62d9d640 1 --2- 172.21.15.29:0/2676867808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d440679e0 0x7f9d44069ea0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9d540b1000 tx=0x7f9d54004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:08.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.683+0000 7f9d5b7fe640 1 -- 172.21.15.29:0/2676867808 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9d4c055b70 con 0x7f9d6415e8d0 2024-04-25T00:37:08.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.879+0000 7f9d691cf640 1 -- 172.21.15.29:0/2676867808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d6406e7b0 con 0x7f9d6415e8d0 2024-04-25T00:37:08.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.879+0000 7f9d5b7fe640 1 -- 172.21.15.29:0/2676867808 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5186 (secure 0 0 0) 0x7f9d4c055470 con 0x7f9d6415e8d0 2024-04-25T00:37:08.882 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:08.883 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2434273880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2434273880},{"type":"v1","addr":"172.21.15.157:6833","nonce":2434273880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/558648019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":558648019},{"type":"v1","addr":"172.21.15.157:6835","nonce":558648019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":7665,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/4178766542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4178766542},{"type":"v1","addr":"172.21.15.29:6833","nonce":4178766542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:36:39.192693+0000","modified":"2024-04-25T00:36:42.218595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7404":{"gid":7404,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/3939243143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3939243143},{"type":"v1","addr":"172.21.15.29:6835","nonce":3939243143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7404,"qdb_cluster":[7404]},"id":6}]} 2024-04-25T00:37:08.883 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 59 2024-04-25T00:37:08.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.883+0000 7f9d691cf640 1 -- 172.21.15.29:0/2676867808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d440679e0 msgr2=0x7f9d44069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.883+0000 7f9d691cf640 1 --2- 172.21.15.29:0/2676867808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d440679e0 0x7f9d44069ea0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9d540b1000 tx=0x7f9d54004740 comp rx=0 tx=0).stop 2024-04-25T00:37:08.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.883+0000 7f9d691cf640 1 -- 172.21.15.29:0/2676867808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d6415e8d0 msgr2=0x7f9d64158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:08.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.883+0000 7f9d691cf640 1 --2- 172.21.15.29:0/2676867808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d6415e8d0 0x7f9d64158e00 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0011f0 tx=0x7f9d4c00c9b0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.883+0000 7f9d691cf640 1 -- 172.21.15.29:0/2676867808 shutdown_connections 2024-04-25T00:37:08.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.883+0000 7f9d691cf640 1 --2- 172.21.15.29:0/2676867808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d440679e0 0x7f9d44069ea0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.883+0000 7f9d691cf640 1 --2- 172.21.15.29:0/2676867808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d6415f3c0 0x7f9d64139340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.883+0000 7f9d691cf640 1 --2- 172.21.15.29:0/2676867808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d6415e8d0 0x7f9d64158e00 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.883+0000 7f9d691cf640 1 --2- 172.21.15.29:0/2676867808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d640b4780 0x7f9d641594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:08.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.883+0000 7f9d691cf640 1 -- 172.21.15.29:0/2676867808 >> 172.21.15.29:0/2676867808 conn(0x7f9d640b1870 msgr2=0x7f9d640b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:08.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.883+0000 7f9d691cf640 1 -- 172.21.15.29:0/2676867808 shutdown_connections 2024-04-25T00:37:08.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:08.883+0000 7f9d691cf640 1 -- 172.21.15.29:0/2676867808 wait complete. 2024-04-25T00:37:08.903 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:37:09.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa81b630640 1 Processor -- start 2024-04-25T00:37:09.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa81b630640 1 -- start start 2024-04-25T00:37:09.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa81b630640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81415e200 0x7fa8140d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa81b630640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8140d2c60 0x7fa8140b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa81b630640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8140cfc10 con 0x7fa8140cf690 2024-04-25T00:37:09.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa81b630640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa8140b38d0 con 0x7fa8140d2c60 2024-04-25T00:37:09.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa81b630640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa8140cfef0 con 0x7fa81415e200 2024-04-25T00:37:09.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa818bcb640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa8140cf690 0x7fa8140d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41422/0 (socket says 172.21.15.29:41422) 2024-04-25T00:37:09.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa818bcb640 1 -- 172.21.15.29:0/2319101003 learned_addr learned my addr 172.21.15.29:0/2319101003 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:09.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa819bcd640 1 --2- 172.21.15.29:0/2319101003 >> v2:172.21.15.157:3300/0 conn(0x7fa8140d2c60 0x7fa8140b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa8193cc640 1 --2- 172.21.15.29:0/2319101003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81415e200 0x7fa8140d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3701729229 0 0) 0x7fa8140cfc10 con 0x7fa8140cf690 2024-04-25T00:37:09.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa800003530 con 0x7fa8140cf690 2024-04-25T00:37:09.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2169237844 0 0) 0x7fa800003530 con 0x7fa8140cf690 2024-04-25T00:37:09.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81415e200 msgr2=0x7fa8140d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:09.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 --2- 172.21.15.29:0/2319101003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81415e200 0x7fa8140d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 >> v2:172.21.15.157:3300/0 conn(0x7fa8140d2c60 msgr2=0x7fa8140b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 --2- 172.21.15.29:0/2319101003 >> v2:172.21.15.157:3300/0 conn(0x7fa8140d2c60 0x7fa8140b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8140b3b20 con 0x7fa8140cf690 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fa810001680 con 0x7fa8140cf690 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 >> v1:172.21.15.29:6789/0 conn(0x7fa8140cf690 legacy=0x7fa8140d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 --2- 172.21.15.29:0/2319101003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81415e200 0x7fa8000039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 --2- 172.21.15.29:0/2319101003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa800003f40 0x7fa800004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 --2- 172.21.15.29:0/2319101003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa800004d50 0x7fa8000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa8140b38d0 con 0x7fa800004d50 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8140cfef0 con 0x7fa800003f40 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa80000b9d0 con 0x7fa81415e200 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa819bcd640 1 --2- 172.21.15.29:0/2319101003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa800004d50 0x7fa8000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa8193cc640 1 --2- 172.21.15.29:0/2319101003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81415e200 0x7fa8000039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa818bcb640 1 --2- 172.21.15.29:0/2319101003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa800003f40 0x7fa800004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa819bcd640 1 -- 172.21.15.29:0/2319101003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81415e200 msgr2=0x7fa8000039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:09.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa819bcd640 1 --2- 172.21.15.29:0/2319101003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81415e200 0x7fa8000039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa819bcd640 1 -- 172.21.15.29:0/2319101003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa800003f40 msgr2=0x7fa800004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:09.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa819bcd640 1 --2- 172.21.15.29:0/2319101003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa800003f40 0x7fa800004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa819bcd640 1 -- 172.21.15.29:0/2319101003 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa800003530 con 0x7fa800004d50 2024-04-25T00:37:09.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.019+0000 7fa818bcb640 1 --2- 172.21.15.29:0/2319101003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa800003f40 0x7fa800004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:09.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.023+0000 7fa819bcd640 1 --2- 172.21.15.29:0/2319101003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa800004d50 0x7fa8000051d0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009f90 tx=0x7fa80c01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0fa833ab9cbb9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:09.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.023+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c0b2070 con 0x7fa800004d50 2024-04-25T00:37:09.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.023+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80c0a5c90 con 0x7fa800004d50 2024-04-25T00:37:09.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.023+0000 7fa80bfff640 1 -- 172.21.15.29:0/2319101003 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c009c10 con 0x7fa800004d50 2024-04-25T00:37:09.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.023+0000 7fa81b630640 1 -- 172.21.15.29:0/2319101003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa800004d50 msgr2=0x7fa8000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.023+0000 7fa81b630640 1 --2- 172.21.15.29:0/2319101003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa800004d50 0x7fa8000051d0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009f90 tx=0x7fa80c01cc20 comp rx=0 tx=0).stop 2024-04-25T00:37:09.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.023+0000 7fa819bcd640 1 -- 172.21.15.29:0/2319101003 reap_dead start 2024-04-25T00:37:09.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.023+0000 7fa81b630640 1 -- 172.21.15.29:0/2319101003 shutdown_connections 2024-04-25T00:37:09.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.023+0000 7fa81b630640 1 -- 172.21.15.29:0/2319101003 >> 172.21.15.29:0/2319101003 conn(0x7fa81409d280 msgr2=0x7fa8140d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:09.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.023+0000 7fa81b630640 1 -- 172.21.15.29:0/2319101003 shutdown_connections 2024-04-25T00:37:09.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.023+0000 7fa81b630640 1 -- 172.21.15.29:0/2319101003 wait complete. 2024-04-25T00:37:09.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.023+0000 7fa81b630640 1 Processor -- start 2024-04-25T00:37:09.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.023+0000 7fa81b630640 1 -- start start 2024-04-25T00:37:09.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa81b630640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8140d0470 0x7fa8140b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa81b630640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8140b92e0 0x7fa8141f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa81b630640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8140d0850 0x7fa8141f7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa81b630640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa81415e760 con 0x7fa8140d0470 2024-04-25T00:37:09.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa81b630640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa81415e5e0 con 0x7fa8140b92e0 2024-04-25T00:37:09.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa81b630640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa8140d47f0 con 0x7fa8140d0850 2024-04-25T00:37:09.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa8193cc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8140d0470 0x7fa8140b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa8193cc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8140d0470 0x7fa8140b8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58958/0 (socket says 172.21.15.29:58958) 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa8193cc640 1 -- 172.21.15.29:0/2139248497 learned_addr learned my addr 172.21.15.29:0/2139248497 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa819bcd640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8140d0850 0x7fa8141f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa818bcb640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8140b92e0 0x7fa8141f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa8193cc640 1 -- 172.21.15.29:0/2139248497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8140d0850 msgr2=0x7fa8141f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa8193cc640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8140d0850 0x7fa8141f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa8193cc640 1 -- 172.21.15.29:0/2139248497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8140b92e0 msgr2=0x7fa8141f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa8193cc640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8140b92e0 0x7fa8141f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa8193cc640 1 -- 172.21.15.29:0/2139248497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8140b9860 con 0x7fa8140d0470 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa818bcb640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8140b92e0 0x7fa8141f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa8193cc640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8140d0470 0x7fa8140b8da0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fa8040115e0 tx=0x7fa804011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa809ffb640 1 -- 172.21.15.29:0/2139248497 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80400f8f0 con 0x7fa8140d0470 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa809ffb640 1 -- 172.21.15.29:0/2139248497 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80400fa90 con 0x7fa8140d0470 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa81b630640 1 -- 172.21.15.29:0/2139248497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8141f3b90 con 0x7fa8140d0470 2024-04-25T00:37:09.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa809ffb640 1 -- 172.21.15.29:0/2139248497 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80400fe60 con 0x7fa8140d0470 2024-04-25T00:37:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa81b630640 1 -- 172.21.15.29:0/2139248497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8141f4150 con 0x7fa8140d0470 2024-04-25T00:37:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa81b630640 1 -- 172.21.15.29:0/2139248497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa814069270 con 0x7fa8140d0470 2024-04-25T00:37:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa809ffb640 1 -- 172.21.15.29:0/2139248497 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fa804015460 con 0x7fa8140d0470 2024-04-25T00:37:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa809ffb640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa7f80679e0 0x7fa7f8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa809ffb640 1 -- 172.21.15.29:0/2139248497 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fa804086810 con 0x7fa8140d0470 2024-04-25T00:37:09.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa818bcb640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa7f80679e0 0x7fa7f8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.027+0000 7fa818bcb640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa7f80679e0 0x7fa7f8069ea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa810001ec0 tx=0x7fa810009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:09.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.031+0000 7fa809ffb640 1 -- 172.21.15.29:0/2139248497 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa804019030 con 0x7fa8140d0470 2024-04-25T00:37:09.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.195+0000 7fa81b630640 1 -- 172.21.15.29:0/2139248497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa814067900 con 0x7fa8140d0470 2024-04-25T00:37:09.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.199+0000 7fa809ffb640 1 -- 172.21.15.29:0/2139248497 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16741 (secure 0 0 0) 0x7fa804057270 con 0x7fa8140d0470 2024-04-25T00:37:09.202 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:09.202 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":91,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:37:04.487226+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:36:37.482565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:36:38.176961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6833/1764147340":"2024-04-26T00:37:01.479125+0000","172.21.15.29:6832/1764147340":"2024-04-26T00:37:01.479125+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:37:09.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.199+0000 7fa81b630640 1 -- 172.21.15.29:0/2139248497 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa7f80679e0 msgr2=0x7fa7f8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.199+0000 7fa81b630640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa7f80679e0 0x7fa7f8069ea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa810001ec0 tx=0x7fa810009040 comp rx=0 tx=0).stop 2024-04-25T00:37:09.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.199+0000 7fa81b630640 1 -- 172.21.15.29:0/2139248497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8140d0470 msgr2=0x7fa8140b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.199+0000 7fa81b630640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8140d0470 0x7fa8140b8da0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fa8040115e0 tx=0x7fa804011aa0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.203+0000 7fa81b630640 1 -- 172.21.15.29:0/2139248497 shutdown_connections 2024-04-25T00:37:09.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.203+0000 7fa81b630640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa7f80679e0 0x7fa7f8069ea0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.203+0000 7fa81b630640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8140d0850 0x7fa8141f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.203+0000 7fa81b630640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8140b92e0 0x7fa8141f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.203+0000 7fa81b630640 1 --2- 172.21.15.29:0/2139248497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8140d0470 0x7fa8140b8da0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.203+0000 7fa81b630640 1 -- 172.21.15.29:0/2139248497 >> 172.21.15.29:0/2139248497 conn(0x7fa81409d280 msgr2=0x7fa8140d2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:09.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.203+0000 7fa81b630640 1 -- 172.21.15.29:0/2139248497 shutdown_connections 2024-04-25T00:37:09.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.203+0000 7fa81b630640 1 -- 172.21.15.29:0/2139248497 wait complete. 2024-04-25T00:37:09.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:36:37.482565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:36:38.176961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:37:09.221 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:37:09.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff03d8df640 1 Processor -- start 2024-04-25T00:37:09.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff03d8df640 1 -- start start 2024-04-25T00:37:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff03d8df640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff03815c290 0x7ff0380cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff03d8df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0380aea40 0x7ff0380aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff03d8df640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0380af770 con 0x7ff0380d1c10 2024-04-25T00:37:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff03d8df640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff0380b0380 con 0x7ff03815c290 2024-04-25T00:37:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff03d8df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff0380afa50 con 0x7ff0380aea40 2024-04-25T00:37:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff0367fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff03815c290 0x7ff0380cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff036ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff0380d1c10 0x7ff0380d1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41432/0 (socket says 172.21.15.29:41432) 2024-04-25T00:37:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff036ffd640 1 -- 172.21.15.29:0/3640900129 learned_addr learned my addr 172.21.15.29:0/3640900129 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff0377fe640 1 --2- 172.21.15.29:0/3640900129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0380aea40 0x7ff0380aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff0367fc640 1 -- 172.21.15.29:0/3640900129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0380aea40 msgr2=0x7ff0380aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff035ffb640 1 -- 172.21.15.29:0/3640900129 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2238463146 0 0) 0x7ff0380af770 con 0x7ff0380d1c10 2024-04-25T00:37:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff0367fc640 1 --2- 172.21.15.29:0/3640900129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0380aea40 0x7ff0380aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff0367fc640 1 -- 172.21.15.29:0/3640900129 >> v1:172.21.15.29:6789/0 conn(0x7ff0380d1c10 legacy=0x7ff0380d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff0367fc640 1 -- 172.21.15.29:0/3640900129 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0380afd30 con 0x7ff03815c290 2024-04-25T00:37:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.339+0000 7ff0377fe640 1 --2- 172.21.15.29:0/3640900129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0380aea40 0x7ff0380aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff0367fc640 1 --2- 172.21.15.29:0/3640900129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff03815c290 0x7ff0380cc770 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0097c0 tx=0x7ff02c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f879afe998352701 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff035ffb640 1 -- 172.21.15.29:0/3640900129 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0aa5f0 con 0x7ff03815c290 2024-04-25T00:37:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff035ffb640 1 -- 172.21.15.29:0/3640900129 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02c0af070 con 0x7ff03815c290 2024-04-25T00:37:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff035ffb640 1 -- 172.21.15.29:0/3640900129 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0aa9b0 con 0x7ff03815c290 2024-04-25T00:37:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 -- 172.21.15.29:0/3640900129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff03815c290 msgr2=0x7ff0380cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 --2- 172.21.15.29:0/3640900129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff03815c290 0x7ff0380cc770 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0097c0 tx=0x7ff02c01c470 comp rx=0 tx=0).stop 2024-04-25T00:37:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 -- 172.21.15.29:0/3640900129 shutdown_connections 2024-04-25T00:37:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 --2- 172.21.15.29:0/3640900129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0380aea40 0x7ff0380aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 --2- 172.21.15.29:0/3640900129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff03815c290 0x7ff0380cc770 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 -- 172.21.15.29:0/3640900129 >> 172.21.15.29:0/3640900129 conn(0x7ff03809aa30 msgr2=0x7ff03809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 -- 172.21.15.29:0/3640900129 shutdown_connections 2024-04-25T00:37:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 -- 172.21.15.29:0/3640900129 wait complete. 2024-04-25T00:37:09.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 Processor -- start 2024-04-25T00:37:09.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 -- start start 2024-04-25T00:37:09.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0380aea40 0x7ff0381544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0380d1c10 0x7ff03815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff03815c290 0x7ff038166340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff0380b11a0 con 0x7ff03815c290 2024-04-25T00:37:09.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff0380b1020 con 0x7ff0380d1c10 2024-04-25T00:37:09.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff03d8df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff0380b1320 con 0x7ff0380aea40 2024-04-25T00:37:09.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff0377fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff03815c290 0x7ff038166340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.343+0000 7ff0377fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff03815c290 0x7ff038166340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58976/0 (socket says 172.21.15.29:58976) 2024-04-25T00:37:09.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff0377fe640 1 -- 172.21.15.29:0/4264299456 learned_addr learned my addr 172.21.15.29:0/4264299456 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:09.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff036ffd640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0380aea40 0x7ff0381544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff0367fc640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0380d1c10 0x7ff03815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff0377fe640 1 -- 172.21.15.29:0/4264299456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0380aea40 msgr2=0x7ff0381544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff0377fe640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0380aea40 0x7ff0381544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff0377fe640 1 -- 172.21.15.29:0/4264299456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0380d1c10 msgr2=0x7ff03815b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff0377fe640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0380d1c10 0x7ff03815b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff0377fe640 1 -- 172.21.15.29:0/4264299456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0381431e0 con 0x7ff03815c290 2024-04-25T00:37:09.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff0377fe640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff03815c290 0x7ff038166340 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7ff02800c7f0 tx=0x7ff02800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:09.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff01ffff640 1 -- 172.21.15.29:0/4264299456 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff028015020 con 0x7ff03815c290 2024-04-25T00:37:09.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff01ffff640 1 -- 172.21.15.29:0/4264299456 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff028004250 con 0x7ff03815c290 2024-04-25T00:37:09.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff03d8df640 1 -- 172.21.15.29:0/4264299456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0381434c0 con 0x7ff03815c290 2024-04-25T00:37:09.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff01ffff640 1 -- 172.21.15.29:0/4264299456 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff028003620 con 0x7ff03815c290 2024-04-25T00:37:09.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff03d8df640 1 -- 172.21.15.29:0/4264299456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03815b8e0 con 0x7ff03815c290 2024-04-25T00:37:09.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff03d8df640 1 -- 172.21.15.29:0/4264299456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff038069270 con 0x7ff03815c290 2024-04-25T00:37:09.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff01ffff640 1 -- 172.21.15.29:0/4264299456 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7ff028018020 con 0x7ff03815c290 2024-04-25T00:37:09.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff01ffff640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff0240679e0 0x7ff024069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff01ffff640 1 -- 172.21.15.29:0/4264299456 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7ff028085f60 con 0x7ff03815c290 2024-04-25T00:37:09.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff036ffd640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff0240679e0 0x7ff024069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.347+0000 7ff036ffd640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff0240679e0 0x7ff024069ea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff030007fd0 tx=0x7ff030007ee0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:09.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.351+0000 7ff01ffff640 1 -- 172.21.15.29:0/4264299456 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff02801f030 con 0x7ff03815c290 2024-04-25T00:37:09.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.507+0000 7ff03d8df640 1 -- 172.21.15.29:0/4264299456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff038143850 con 0x7ff03815c290 2024-04-25T00:37:09.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.507+0000 7ff01ffff640 1 -- 172.21.15.29:0/4264299456 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7ff028056820 con 0x7ff03815c290 2024-04-25T00:37:09.511 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:37:09.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.511+0000 7ff03d8df640 1 -- 172.21.15.29:0/4264299456 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff0240679e0 msgr2=0x7ff024069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.511+0000 7ff03d8df640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff0240679e0 0x7ff024069ea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff030007fd0 tx=0x7ff030007ee0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.511+0000 7ff03d8df640 1 -- 172.21.15.29:0/4264299456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff03815c290 msgr2=0x7ff038166340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.511+0000 7ff03d8df640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff03815c290 0x7ff038166340 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7ff02800c7f0 tx=0x7ff02800ccb0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.511+0000 7ff03d8df640 1 -- 172.21.15.29:0/4264299456 shutdown_connections 2024-04-25T00:37:09.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.511+0000 7ff03d8df640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff0240679e0 0x7ff024069ea0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.511+0000 7ff03d8df640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff03815c290 0x7ff038166340 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.511+0000 7ff03d8df640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0380d1c10 0x7ff03815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.511+0000 7ff03d8df640 1 --2- 172.21.15.29:0/4264299456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0380aea40 0x7ff0381544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.511+0000 7ff03d8df640 1 -- 172.21.15.29:0/4264299456 >> 172.21.15.29:0/4264299456 conn(0x7ff03809aa30 msgr2=0x7ff0380ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:09.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.511+0000 7ff03d8df640 1 -- 172.21.15.29:0/4264299456 shutdown_connections 2024-04-25T00:37:09.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.511+0000 7ff03d8df640 1 -- 172.21.15.29:0/4264299456 wait complete. 2024-04-25T00:37:09.529 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:37:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.651+0000 7fd1b3427640 1 Processor -- start 2024-04-25T00:37:09.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.651+0000 7fd1b3427640 1 -- start start 2024-04-25T00:37:09.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.651+0000 7fd1b3427640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd1ac0b3c30 0x7fd1ac06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.651+0000 7fd1b3427640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1ac0b41e0 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.651+0000 7fd1b3427640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1ac06d960 con 0x7fd1ac0b3140 2024-04-25T00:37:09.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.651+0000 7fd1b3427640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd1ac06dc40 con 0x7fd1ac0b3c30 2024-04-25T00:37:09.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b09c2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd1ac0b3c30 0x7fd1ac06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b19c4640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd1ac0b4b10 0x7fd1ac0ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41440/0 (socket says 172.21.15.29:41440) 2024-04-25T00:37:09.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b09c2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd1ac0b3c30 0x7fd1ac06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58988/0 (socket says 172.21.15.29:58988) 2024-04-25T00:37:09.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b19c4640 1 -- 172.21.15.29:0/1746593139 learned_addr learned my addr 172.21.15.29:0/1746593139 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:09.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b09c2640 1 -- 172.21.15.29:0/1746593139 >> v1:172.21.15.157:6789/0 conn(0x7fd1ac0b3140 legacy=0x7fd1ac0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:09.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b09c2640 1 -- 172.21.15.29:0/1746593139 >> v1:172.21.15.29:6789/0 conn(0x7fd1ac0b4b10 legacy=0x7fd1ac0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b09c2640 1 -- 172.21.15.29:0/1746593139 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac06df20 con 0x7fd1ac0b3c30 2024-04-25T00:37:09.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b09c2640 1 --2- 172.21.15.29:0/1746593139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1ac0b3c30 0x7fd1ac06e440 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0097f0 tx=0x7fd19c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16658fd0722e979 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:09.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1a3fff640 1 -- 172.21.15.29:0/1746593139 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0b0070 con 0x7fd1ac0b3c30 2024-04-25T00:37:09.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1a3fff640 1 -- 172.21.15.29:0/1746593139 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd19c0ab600 con 0x7fd1ac0b3c30 2024-04-25T00:37:09.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1a3fff640 1 -- 172.21.15.29:0/1746593139 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0ab9d0 con 0x7fd1ac0b3c30 2024-04-25T00:37:09.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b3427640 1 -- 172.21.15.29:0/1746593139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1ac0b3c30 msgr2=0x7fd1ac06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b3427640 1 --2- 172.21.15.29:0/1746593139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1ac0b3c30 0x7fd1ac06e440 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0097f0 tx=0x7fd19c01c460 comp rx=0 tx=0).stop 2024-04-25T00:37:09.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b3427640 1 -- 172.21.15.29:0/1746593139 shutdown_connections 2024-04-25T00:37:09.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b3427640 1 --2- 172.21.15.29:0/1746593139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1ac0b3c30 0x7fd1ac06e440 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b3427640 1 -- 172.21.15.29:0/1746593139 >> 172.21.15.29:0/1746593139 conn(0x7fd1ac0b1870 msgr2=0x7fd1ac0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:09.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b3427640 1 -- 172.21.15.29:0/1746593139 shutdown_connections 2024-04-25T00:37:09.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.655+0000 7fd1b3427640 1 -- 172.21.15.29:0/1746593139 wait complete. 2024-04-25T00:37:09.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b3427640 1 Processor -- start 2024-04-25T00:37:09.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b3427640 1 -- start start 2024-04-25T00:37:09.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b3427640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1ac0b3140 0x7fd1ac143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b3427640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1ac0b3c30 0x7fd1ac13d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b3427640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1ac0b4b10 0x7fd1ac139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b3427640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac169d90 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b3427640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac169c10 con 0x7fd1ac0b3c30 2024-04-25T00:37:09.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b3427640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd1ac169f10 con 0x7fd1ac0b3140 2024-04-25T00:37:09.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b19c4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1ac0b4b10 0x7fd1ac139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b19c4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1ac0b4b10 0x7fd1ac139670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58990/0 (socket says 172.21.15.29:58990) 2024-04-25T00:37:09.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b19c4640 1 -- 172.21.15.29:0/3183739864 learned_addr learned my addr 172.21.15.29:0/3183739864 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:09.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b09c2640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1ac0b3c30 0x7fd1ac13d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b11c3640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1ac0b3140 0x7fd1ac143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b19c4640 1 -- 172.21.15.29:0/3183739864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1ac0b3140 msgr2=0x7fd1ac143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b19c4640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1ac0b3140 0x7fd1ac143450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b19c4640 1 -- 172.21.15.29:0/3183739864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1ac0b3c30 msgr2=0x7fd1ac13d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b19c4640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1ac0b3c30 0x7fd1ac13d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b19c4640 1 -- 172.21.15.29:0/3183739864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac071890 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b09c2640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1ac0b3c30 0x7fd1ac13d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:09.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b11c3640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1ac0b3140 0x7fd1ac143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:09.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b19c4640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1ac0b4b10 0x7fd1ac139670 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400cad0 tx=0x7fd1a400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:09.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1a1ffb640 1 -- 172.21.15.29:0/3183739864 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4017070 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1a1ffb640 1 -- 172.21.15.29:0/3183739864 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a400ac30 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b3427640 1 -- 172.21.15.29:0/3183739864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ac071b30 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1a1ffb640 1 -- 172.21.15.29:0/3183739864 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4012620 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b3427640 1 -- 172.21.15.29:0/3183739864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ac072610 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.659+0000 7fd1b3427640 1 -- 172.21.15.29:0/3183739864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1ac143990 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.663+0000 7fd1a1ffb640 1 -- 172.21.15.29:0/3183739864 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fd1a401a070 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.663+0000 7fd1a1ffb640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd190067a10 0x7fd190069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.663+0000 7fd1a1ffb640 1 -- 172.21.15.29:0/3183739864 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fd1a4085c10 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.663+0000 7fd1b11c3640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd190067a10 0x7fd190069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.663+0000 7fd1b11c3640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd190067a10 0x7fd190069ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800a850 tx=0x7fd1a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:09.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.667+0000 7fd1a1ffb640 1 -- 172.21.15.29:0/3183739864 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd1a400ae20 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.831+0000 7fd1b3427640 1 -- 172.21.15.29:0/3183739864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd1ac069270 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.831+0000 7fd1a1ffb640 1 -- 172.21.15.29:0/3183739864 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fd1a4056550 con 0x7fd1ac0b4b10 2024-04-25T00:37:09.835 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:37:09.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.835+0000 7fd1b3427640 1 -- 172.21.15.29:0/3183739864 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd190067a10 msgr2=0x7fd190069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.835+0000 7fd1b3427640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd190067a10 0x7fd190069ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800a850 tx=0x7fd1a8008040 comp rx=0 tx=0).stop 2024-04-25T00:37:09.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.835+0000 7fd1b3427640 1 -- 172.21.15.29:0/3183739864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1ac0b4b10 msgr2=0x7fd1ac139670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.835+0000 7fd1b3427640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1ac0b4b10 0x7fd1ac139670 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400cad0 tx=0x7fd1a400cf90 comp rx=0 tx=0).stop 2024-04-25T00:37:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.835+0000 7fd1b3427640 1 -- 172.21.15.29:0/3183739864 shutdown_connections 2024-04-25T00:37:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.835+0000 7fd1b3427640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd190067a10 0x7fd190069ed0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.835+0000 7fd1b3427640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1ac0b4b10 0x7fd1ac139670 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.835+0000 7fd1b3427640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1ac0b3c30 0x7fd1ac13d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.835+0000 7fd1b3427640 1 --2- 172.21.15.29:0/3183739864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1ac0b3140 0x7fd1ac143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.835+0000 7fd1b3427640 1 -- 172.21.15.29:0/3183739864 >> 172.21.15.29:0/3183739864 conn(0x7fd1ac0b1870 msgr2=0x7fd1ac0cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.835+0000 7fd1b3427640 1 -- 172.21.15.29:0/3183739864 shutdown_connections 2024-04-25T00:37:09.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.835+0000 7fd1b3427640 1 -- 172.21.15.29:0/3183739864 wait complete. 2024-04-25T00:37:09.854 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:37:09.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.967+0000 7f52d030b640 1 Processor -- start 2024-04-25T00:37:09.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.967+0000 7f52d030b640 1 -- start start 2024-04-25T00:37:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52d030b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f52c806e440 0x7f52c80b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52d030b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c80b41e0 con 0x7f52c80b3c30 2024-04-25T00:37:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52d030b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c806d960 con 0x7f52c80b3140 2024-04-25T00:37:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52d030b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f52c80b5050 con 0x7f52c806e440 2024-04-25T00:37:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd8a6640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f52c80b3c30 0x7f52c80ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41448/0 (socket says 172.21.15.29:41448) 2024-04-25T00:37:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd8a6640 1 -- 172.21.15.29:0/2761225538 learned_addr learned my addr 172.21.15.29:0/2761225538 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 197525662 0 0) 0x7f52c80b41e0 con 0x7f52c80b3c30 2024-04-25T00:37:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52ce8a8640 1 --2- 172.21.15.29:0/2761225538 >> v2:172.21.15.157:3300/0 conn(0x7f52c806e440 0x7f52c80b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52b4003530 con 0x7f52c80b3c30 2024-04-25T00:37:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1835118702 0 0) 0x7f52b4003530 con 0x7f52c80b3c30 2024-04-25T00:37:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 >> v2:172.21.15.157:3300/0 conn(0x7f52c806e440 msgr2=0x7f52c80b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 --2- 172.21.15.29:0/2761225538 >> v2:172.21.15.157:3300/0 conn(0x7f52c806e440 0x7f52c80b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 >> v1:172.21.15.157:6789/0 conn(0x7f52c80b3140 legacy=0x7f52c80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c806dc40 con 0x7f52c80b3c30 2024-04-25T00:37:09.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f52b8001680 con 0x7f52c80b3c30 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 >> v1:172.21.15.29:6789/0 conn(0x7f52c80b3c30 legacy=0x7f52c80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 --2- 172.21.15.29:0/2761225538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52b40039c0 0x7f52b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 --2- 172.21.15.29:0/2761225538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b4004810 0x7f52b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 --2- 172.21.15.29:0/2761225538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b40051d0 0x7f52b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f52c80b5050 con 0x7f52b40039c0 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52b400c070 con 0x7f52b40051d0 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f52b400c270 con 0x7f52b4004810 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52ce0a7640 1 --2- 172.21.15.29:0/2761225538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52b40039c0 0x7f52b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52cd8a6640 1 --2- 172.21.15.29:0/2761225538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b4004810 0x7f52b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52ce8a8640 1 --2- 172.21.15.29:0/2761225538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b40051d0 0x7f52b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52ce0a7640 1 -- 172.21.15.29:0/2761225538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b4004810 msgr2=0x7f52b4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52ce0a7640 1 --2- 172.21.15.29:0/2761225538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b4004810 0x7f52b4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52ce0a7640 1 -- 172.21.15.29:0/2761225538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b40051d0 msgr2=0x7f52b400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52ce0a7640 1 --2- 172.21.15.29:0/2761225538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b40051d0 0x7f52b400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52ce0a7640 1 -- 172.21.15.29:0/2761225538 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b4003530 con 0x7f52b40039c0 2024-04-25T00:37:09.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52ce8a8640 1 --2- 172.21.15.29:0/2761225538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b40051d0 0x7f52b400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:09.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.971+0000 7f52ce8a8640 1 -- 172.21.15.29:0/2761225538 reap_dead start 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52ce0a7640 1 --2- 172.21.15.29:0/2761225538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52b40039c0 0x7f52b4003de0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f52c4001780 tx=0x7f52c401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a89fc1427e8947 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c40013a0 con 0x7f52b40039c0 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c40052f0 con 0x7f52b40039c0 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52cd0a5640 1 -- 172.21.15.29:0/2761225538 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4003840 con 0x7f52b40039c0 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 -- 172.21.15.29:0/2761225538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52b40039c0 msgr2=0x7f52b4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 --2- 172.21.15.29:0/2761225538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52b40039c0 0x7f52b4003de0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f52c4001780 tx=0x7f52c401c400 comp rx=0 tx=0).stop 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 -- 172.21.15.29:0/2761225538 shutdown_connections 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 --2- 172.21.15.29:0/2761225538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52b40039c0 0x7f52b4003de0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 -- 172.21.15.29:0/2761225538 >> 172.21.15.29:0/2761225538 conn(0x7f52c80b1870 msgr2=0x7f52c80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 -- 172.21.15.29:0/2761225538 shutdown_connections 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 -- 172.21.15.29:0/2761225538 wait complete. 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 Processor -- start 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 -- start start 2024-04-25T00:37:09.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b40039c0 0x7f52c81f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c81387d0 0x7f52c813c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52c813df90 0x7f52c813e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f52c80b3520 con 0x7f52c813df90 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52c806e820 con 0x7f52c81387d0 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f52c80b36a0 con 0x7f52b40039c0 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52ce8a8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52c813df90 0x7f52c813e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52ce8a8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52c813df90 0x7f52c813e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59012/0 (socket says 172.21.15.29:59012) 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52ce8a8640 1 -- 172.21.15.29:0/2681780569 learned_addr learned my addr 172.21.15.29:0/2681780569 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52cd8a6640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c81387d0 0x7f52c813c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52ce8a8640 1 -- 172.21.15.29:0/2681780569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b40039c0 msgr2=0x7f52c81f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52ce8a8640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b40039c0 0x7f52c81f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52ce8a8640 1 -- 172.21.15.29:0/2681780569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c81387d0 msgr2=0x7f52c813c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52ce8a8640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c81387d0 0x7f52c813c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52ce8a8640 1 -- 172.21.15.29:0/2681780569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c81f5610 con 0x7f52c813df90 2024-04-25T00:37:09.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52cd8a6640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c81387d0 0x7f52c813c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52ce8a8640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52c813df90 0x7f52c813e410 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009ea0 tx=0x7f52c0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52beffd640 1 -- 172.21.15.29:0/2681780569 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0016420 con 0x7f52c813df90 2024-04-25T00:37:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52beffd640 1 -- 172.21.15.29:0/2681780569 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c001b070 con 0x7f52c813df90 2024-04-25T00:37:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 -- 172.21.15.29:0/2681780569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c81f4d70 con 0x7f52c813df90 2024-04-25T00:37:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52beffd640 1 -- 172.21.15.29:0/2681780569 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c00167c0 con 0x7f52c813df90 2024-04-25T00:37:09.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.975+0000 7f52d030b640 1 -- 172.21.15.29:0/2681780569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c81f5330 con 0x7f52c813df90 2024-04-25T00:37:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.979+0000 7f52d030b640 1 -- 172.21.15.29:0/2681780569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c8069270 con 0x7f52c813df90 2024-04-25T00:37:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.979+0000 7f52beffd640 1 -- 172.21.15.29:0/2681780569 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f52c0016960 con 0x7f52c813df90 2024-04-25T00:37:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.979+0000 7f52beffd640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f52ac0679e0 0x7f52ac069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.979+0000 7f52beffd640 1 -- 172.21.15.29:0/2681780569 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f52c0086aa0 con 0x7f52c813df90 2024-04-25T00:37:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.979+0000 7f52ce0a7640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f52ac0679e0 0x7f52ac069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:09.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.979+0000 7f52ce0a7640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f52ac0679e0 0x7f52ac069ea0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52c4001750 tx=0x7f52c4001130 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:09.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:09.983+0000 7f52beffd640 1 -- 172.21.15.29:0/2681780569 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f52c81f5610 con 0x7f52c813df90 2024-04-25T00:37:10.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.147+0000 7f52d030b640 1 -- 172.21.15.29:0/2681780569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f52c8139570 con 0x7f52c813df90 2024-04-25T00:37:10.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.147+0000 7f52beffd640 1 -- 172.21.15.29:0/2681780569 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f52c00573f0 con 0x7f52c813df90 2024-04-25T00:37:10.152 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:37:10.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.151+0000 7f52d030b640 1 -- 172.21.15.29:0/2681780569 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f52ac0679e0 msgr2=0x7f52ac069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.151+0000 7f52d030b640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f52ac0679e0 0x7f52ac069ea0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52c4001750 tx=0x7f52c4001130 comp rx=0 tx=0).stop 2024-04-25T00:37:10.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.151+0000 7f52d030b640 1 -- 172.21.15.29:0/2681780569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52c813df90 msgr2=0x7f52c813e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.151+0000 7f52d030b640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52c813df90 0x7f52c813e410 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009ea0 tx=0x7f52c0009f90 comp rx=0 tx=0).stop 2024-04-25T00:37:10.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.151+0000 7f52d030b640 1 -- 172.21.15.29:0/2681780569 shutdown_connections 2024-04-25T00:37:10.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.151+0000 7f52d030b640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f52ac0679e0 0x7f52ac069ea0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.151+0000 7f52d030b640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52c813df90 0x7f52c813e410 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.151+0000 7f52d030b640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c81387d0 0x7f52c813c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.151+0000 7f52d030b640 1 --2- 172.21.15.29:0/2681780569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b40039c0 0x7f52c81f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.151+0000 7f52d030b640 1 -- 172.21.15.29:0/2681780569 >> 172.21.15.29:0/2681780569 conn(0x7f52c80b1870 msgr2=0x7f52c8143f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:10.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.151+0000 7f52d030b640 1 -- 172.21.15.29:0/2681780569 shutdown_connections 2024-04-25T00:37:10.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.151+0000 7f52d030b640 1 -- 172.21.15.29:0/2681780569 wait complete. 2024-04-25T00:37:10.171 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:37:10.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f7176cef640 1 Processor -- start 2024-04-25T00:37:10.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f7176cef640 1 -- start start 2024-04-25T00:37:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f7176cef640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f717015c220 0x7f71700cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f7176cef640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700d0670 0x7f71700b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f7176cef640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71700cdbd0 con 0x7f71700cfa10 2024-04-25T00:37:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f7176cef640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f717015c600 con 0x7f717015c220 2024-04-25T00:37:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f7176cef640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f71700b1a30 con 0x7f71700d0670 2024-04-25T00:37:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f7174a8b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f717015c220 0x7f71700cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f716ffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f71700cfa10 0x7f71700cff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41464/0 (socket says 172.21.15.29:41464) 2024-04-25T00:37:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f716ffff640 1 -- 172.21.15.29:0/594994226 learned_addr learned my addr 172.21.15.29:0/594994226 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f717528c640 1 --2- 172.21.15.29:0/594994226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700d0670 0x7f71700b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f7174a8b640 1 -- 172.21.15.29:0/594994226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700d0670 msgr2=0x7f71700b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f7174a8b640 1 --2- 172.21.15.29:0/594994226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700d0670 0x7f71700b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f7174a8b640 1 -- 172.21.15.29:0/594994226 >> v1:172.21.15.29:6789/0 conn(0x7f71700cfa10 legacy=0x7f71700cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f7174a8b640 1 -- 172.21.15.29:0/594994226 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71700b13e0 con 0x7f717015c220 2024-04-25T00:37:10.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f716f7fe640 1 -- 172.21.15.29:0/594994226 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1522375648 0 0) 0x7f71700cdbd0 con 0x7f71700cfa10 2024-04-25T00:37:10.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.291+0000 7f7174a8b640 1 --2- 172.21.15.29:0/594994226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f717015c220 0x7f71700cd350 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f71640097c0 tx=0x7f716401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed41785395834823 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:10.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f716f7fe640 1 -- 172.21.15.29:0/594994226 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71640b0070 con 0x7f717015c220 2024-04-25T00:37:10.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f716f7fe640 1 -- 172.21.15.29:0/594994226 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71640ab600 con 0x7f717015c220 2024-04-25T00:37:10.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f716f7fe640 1 -- 172.21.15.29:0/594994226 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71640ab9d0 con 0x7f717015c220 2024-04-25T00:37:10.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 -- 172.21.15.29:0/594994226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f717015c220 msgr2=0x7f71700cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 --2- 172.21.15.29:0/594994226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f717015c220 0x7f71700cd350 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f71640097c0 tx=0x7f716401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 -- 172.21.15.29:0/594994226 shutdown_connections 2024-04-25T00:37:10.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 --2- 172.21.15.29:0/594994226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700d0670 0x7f71700b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 --2- 172.21.15.29:0/594994226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f717015c220 0x7f71700cd350 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 -- 172.21.15.29:0/594994226 >> 172.21.15.29:0/594994226 conn(0x7f71700d1d60 msgr2=0x7f71700d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:10.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 -- 172.21.15.29:0/594994226 shutdown_connections 2024-04-25T00:37:10.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 -- 172.21.15.29:0/594994226 wait complete. 2024-04-25T00:37:10.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 Processor -- start 2024-04-25T00:37:10.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 -- start start 2024-04-25T00:37:10.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71700cfa10 0x7f717006ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71700d0670 0x7f717015c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717015c220 0x7f71701537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f71700b2fe0 con 0x7f71700cfa10 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71700b2e60 con 0x7f71700d0670 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f71700b3160 con 0x7f717015c220 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7174a8b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71700cfa10 0x7f717006ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7174a8b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71700cfa10 0x7f717006ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59034/0 (socket says 172.21.15.29:59034) 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7174a8b640 1 -- 172.21.15.29:0/3235902642 learned_addr learned my addr 172.21.15.29:0/3235902642 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f717528c640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717015c220 0x7f71701537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7174a8b640 1 -- 172.21.15.29:0/3235902642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717015c220 msgr2=0x7f71701537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7174a8b640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717015c220 0x7f71701537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f716ffff640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71700d0670 0x7f717015c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7174a8b640 1 -- 172.21.15.29:0/3235902642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71700d0670 msgr2=0x7f717015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7174a8b640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71700d0670 0x7f717015c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7174a8b640 1 -- 172.21.15.29:0/3235902642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71701642a0 con 0x7f71700cfa10 2024-04-25T00:37:10.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7174a8b640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71700cfa10 0x7f717006ca30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f71640ae040 tx=0x7f716401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:10.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f716d7fa640 1 -- 172.21.15.29:0/3235902642 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71640b0050 con 0x7f71700cfa10 2024-04-25T00:37:10.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f716d7fa640 1 -- 172.21.15.29:0/3235902642 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71640bc070 con 0x7f71700cfa10 2024-04-25T00:37:10.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 -- 172.21.15.29:0/3235902642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7170164580 con 0x7f71700cfa10 2024-04-25T00:37:10.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f716d7fa640 1 -- 172.21.15.29:0/3235902642 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71640b5a50 con 0x7f71700cfa10 2024-04-25T00:37:10.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.295+0000 7f7176cef640 1 -- 172.21.15.29:0/3235902642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717015ebb0 con 0x7f71700cfa10 2024-04-25T00:37:10.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.299+0000 7f7176cef640 1 -- 172.21.15.29:0/3235902642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7170069270 con 0x7f71700cfa10 2024-04-25T00:37:10.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.299+0000 7f716d7fa640 1 -- 172.21.15.29:0/3235902642 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f71640ab4a0 con 0x7f71700cfa10 2024-04-25T00:37:10.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.299+0000 7f716d7fa640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7154067a10 0x7f7154069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.299+0000 7f716d7fa640 1 -- 172.21.15.29:0/3235902642 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f7164120900 con 0x7f71700cfa10 2024-04-25T00:37:10.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.299+0000 7f716ffff640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7154067a10 0x7f7154069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.299+0000 7f716ffff640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7154067a10 0x7f7154069ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7170166c80 tx=0x7f715c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:10.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.303+0000 7f716d7fa640 1 -- 172.21.15.29:0/3235902642 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7164016090 con 0x7f71700cfa10 2024-04-25T00:37:10.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.495+0000 7f7176cef640 1 -- 172.21.15.29:0/3235902642 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f717014a780 con 0x7f71700cfa10 2024-04-25T00:37:10.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.495+0000 7f716d7fa640 1 -- 172.21.15.29:0/3235902642 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5186 (secure 0 0 0) 0x7f71640f11b0 con 0x7f71700cfa10 2024-04-25T00:37:10.499 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:10.499 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2434273880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2434273880},{"type":"v1","addr":"172.21.15.157:6833","nonce":2434273880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/558648019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":558648019},{"type":"v1","addr":"172.21.15.157:6835","nonce":558648019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":7665,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/4178766542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4178766542},{"type":"v1","addr":"172.21.15.29:6833","nonce":4178766542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:36:39.192693+0000","modified":"2024-04-25T00:36:42.218595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7404":{"gid":7404,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/3939243143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3939243143},{"type":"v1","addr":"172.21.15.29:6835","nonce":3939243143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7404,"qdb_cluster":[7404]},"id":6}]} 2024-04-25T00:37:10.499 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 59 2024-04-25T00:37:10.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.499+0000 7f7176cef640 1 -- 172.21.15.29:0/3235902642 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7154067a10 msgr2=0x7f7154069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.499+0000 7f7176cef640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7154067a10 0x7f7154069ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7170166c80 tx=0x7f715c009040 comp rx=0 tx=0).stop 2024-04-25T00:37:10.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.499+0000 7f7176cef640 1 -- 172.21.15.29:0/3235902642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71700cfa10 msgr2=0x7f717006ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.499+0000 7f7176cef640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71700cfa10 0x7f717006ca30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f71640ae040 tx=0x7f716401ace0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.499+0000 7f7176cef640 1 -- 172.21.15.29:0/3235902642 shutdown_connections 2024-04-25T00:37:10.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.499+0000 7f7176cef640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7154067a10 0x7f7154069ed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.499+0000 7f7176cef640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717015c220 0x7f71701537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.499+0000 7f7176cef640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71700d0670 0x7f717015c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.499+0000 7f7176cef640 1 --2- 172.21.15.29:0/3235902642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71700cfa10 0x7f717006ca30 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.499+0000 7f7176cef640 1 -- 172.21.15.29:0/3235902642 >> 172.21.15.29:0/3235902642 conn(0x7f71700d1d60 msgr2=0x7f7170155430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:10.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.499+0000 7f7176cef640 1 -- 172.21.15.29:0/3235902642 shutdown_connections 2024-04-25T00:37:10.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.499+0000 7f7176cef640 1 -- 172.21.15.29:0/3235902642 wait complete. 2024-04-25T00:37:10.517 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:37:10.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.635+0000 7f851c8be640 1 Processor -- start 2024-04-25T00:37:10.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.635+0000 7f851c8be640 1 -- start start 2024-04-25T00:37:10.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851c8be640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f85180b3140 0x7f851806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851c8be640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85180b3c30 0x7f85180ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851c8be640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85180b41e0 con 0x7f85180b4b10 2024-04-25T00:37:10.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851c8be640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f851806d960 con 0x7f85180b3140 2024-04-25T00:37:10.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851c8be640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f851806dc40 con 0x7f85180b3c30 2024-04-25T00:37:10.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851659c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f85180b3140 0x7f851806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851659c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f85180b3140 0x7f851806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59044/0 (socket says 172.21.15.29:59044) 2024-04-25T00:37:10.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851659c640 1 -- 172.21.15.29:0/3766974444 learned_addr learned my addr 172.21.15.29:0/3766974444 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:10.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851659c640 1 -- 172.21.15.29:0/3766974444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85180b3c30 msgr2=0x7f85180ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:10.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f8516d9d640 1 --2- 172.21.15.29:0/3766974444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85180b3c30 0x7f85180ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851659c640 1 --2- 172.21.15.29:0/3766974444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85180b3c30 0x7f85180ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851659c640 1 -- 172.21.15.29:0/3766974444 >> v1:172.21.15.157:6789/0 conn(0x7f85180b4b10 legacy=0x7f85180b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:10.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851659c640 1 -- 172.21.15.29:0/3766974444 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851806df20 con 0x7f85180b3140 2024-04-25T00:37:10.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851659c640 1 --2- 172.21.15.29:0/3766974444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180b3140 0x7f851806e440 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f85080098a0 tx=0x7f850801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efc17e5551ab075 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:10.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851559a640 1 -- 172.21.15.29:0/3766974444 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85080b0070 con 0x7f85180b3140 2024-04-25T00:37:10.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851559a640 1 -- 172.21.15.29:0/3766974444 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85080ab600 con 0x7f85180b3140 2024-04-25T00:37:10.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851559a640 1 -- 172.21.15.29:0/3766974444 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85080ab9d0 con 0x7f85180b3140 2024-04-25T00:37:10.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851c8be640 1 -- 172.21.15.29:0/3766974444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180b3140 msgr2=0x7f851806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851c8be640 1 --2- 172.21.15.29:0/3766974444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180b3140 0x7f851806e440 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f85080098a0 tx=0x7f850801c500 comp rx=0 tx=0).stop 2024-04-25T00:37:10.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851c8be640 1 -- 172.21.15.29:0/3766974444 shutdown_connections 2024-04-25T00:37:10.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.639+0000 7f851c8be640 1 --2- 172.21.15.29:0/3766974444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85180b3c30 0x7f85180ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 --2- 172.21.15.29:0/3766974444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180b3140 0x7f851806e440 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 -- 172.21.15.29:0/3766974444 >> 172.21.15.29:0/3766974444 conn(0x7f85180b1870 msgr2=0x7f85180b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:10.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 -- 172.21.15.29:0/3766974444 shutdown_connections 2024-04-25T00:37:10.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 -- 172.21.15.29:0/3766974444 wait complete. 2024-04-25T00:37:10.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 Processor -- start 2024-04-25T00:37:10.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 -- start start 2024-04-25T00:37:10.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85180b3140 0x7f85180729a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180b3c30 0x7f85181574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85180b4b10 0x7f8518139d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8518168d80 con 0x7f85180b3c30 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8518168c00 con 0x7f85180b3140 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8518168f00 con 0x7f85180b4b10 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f8515d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180b3c30 0x7f85181574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f8515d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180b3c30 0x7f85181574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59048/0 (socket says 172.21.15.29:59048) 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f8515d9b640 1 -- 172.21.15.29:0/316576912 learned_addr learned my addr 172.21.15.29:0/316576912 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f8516d9d640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85180b4b10 0x7f8518139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851659c640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85180b3140 0x7f85180729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f8515d9b640 1 -- 172.21.15.29:0/316576912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85180b4b10 msgr2=0x7f8518139d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f8515d9b640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85180b4b10 0x7f8518139d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f8515d9b640 1 -- 172.21.15.29:0/316576912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85180b3140 msgr2=0x7f85180729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f8515d9b640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85180b3140 0x7f85180729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f8515d9b640 1 -- 172.21.15.29:0/316576912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8518157b80 con 0x7f85180b3c30 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f8515d9b640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180b3c30 0x7f85181574f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f850c00c450 tx=0x7f850c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:10.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f8506ffd640 1 -- 172.21.15.29:0/316576912 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c012830 con 0x7f85180b3c30 2024-04-25T00:37:10.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f8506ffd640 1 -- 172.21.15.29:0/316576912 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f850c017070 con 0x7f85180b3c30 2024-04-25T00:37:10.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f8506ffd640 1 -- 172.21.15.29:0/316576912 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c012c30 con 0x7f85180b3c30 2024-04-25T00:37:10.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 -- 172.21.15.29:0/316576912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8518158360 con 0x7f85180b3c30 2024-04-25T00:37:10.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.643+0000 7f851c8be640 1 -- 172.21.15.29:0/316576912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8518158920 con 0x7f85180b3c30 2024-04-25T00:37:10.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.647+0000 7f851c8be640 1 -- 172.21.15.29:0/316576912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8518158b20 con 0x7f85180b3c30 2024-04-25T00:37:10.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.647+0000 7f8506ffd640 1 -- 172.21.15.29:0/316576912 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f850c01c460 con 0x7f85180b3c30 2024-04-25T00:37:10.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.647+0000 7f8506ffd640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f84f80679e0 0x7f84f8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.647+0000 7f8506ffd640 1 -- 172.21.15.29:0/316576912 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f850c085c80 con 0x7f85180b3c30 2024-04-25T00:37:10.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.647+0000 7f851659c640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f84f80679e0 0x7f84f8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.647+0000 7f851659c640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f84f80679e0 0x7f84f8069ea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f85080ad000 tx=0x7f8508008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:10.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.651+0000 7f8506ffd640 1 -- 172.21.15.29:0/316576912 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8518158b20 con 0x7f85180b3c30 2024-04-25T00:37:10.708 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:37:10.710 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:37:10.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.815+0000 7f851c8be640 1 -- 172.21.15.29:0/316576912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8518069270 con 0x7f85180b3c30 2024-04-25T00:37:10.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.815+0000 7f8506ffd640 1 -- 172.21.15.29:0/316576912 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16741 (secure 0 0 0) 0x7f850c056640 con 0x7f85180b3c30 2024-04-25T00:37:10.820 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:10.820 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":91,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:37:04.487226+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:36:37.482565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:36:38.176961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6833/1764147340":"2024-04-26T00:37:01.479125+0000","172.21.15.29:6832/1764147340":"2024-04-26T00:37:01.479125+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:37:10.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.819+0000 7f851c8be640 1 -- 172.21.15.29:0/316576912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f84f80679e0 msgr2=0x7f84f8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.819+0000 7f851c8be640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f84f80679e0 0x7f84f8069ea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f85080ad000 tx=0x7f8508008040 comp rx=0 tx=0).stop 2024-04-25T00:37:10.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.819+0000 7f851c8be640 1 -- 172.21.15.29:0/316576912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180b3c30 msgr2=0x7f85181574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.819+0000 7f851c8be640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180b3c30 0x7f85181574f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f850c00c450 tx=0x7f850c00c910 comp rx=0 tx=0).stop 2024-04-25T00:37:10.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.819+0000 7f851c8be640 1 -- 172.21.15.29:0/316576912 shutdown_connections 2024-04-25T00:37:10.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.819+0000 7f851c8be640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f84f80679e0 0x7f84f8069ea0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.819+0000 7f851c8be640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85180b4b10 0x7f8518139d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.819+0000 7f851c8be640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85180b3c30 0x7f85181574f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.819+0000 7f851c8be640 1 --2- 172.21.15.29:0/316576912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85180b3140 0x7f85180729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.819+0000 7f851c8be640 1 -- 172.21.15.29:0/316576912 >> 172.21.15.29:0/316576912 conn(0x7f85180b1870 msgr2=0x7f8518142e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.819+0000 7f851c8be640 1 -- 172.21.15.29:0/316576912 shutdown_connections 2024-04-25T00:37:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.819+0000 7f851c8be640 1 -- 172.21.15.29:0/316576912 wait complete. 2024-04-25T00:37:10.840 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:37:10.840 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:37:10.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.963+0000 7f167033e640 1 Processor -- start 2024-04-25T00:37:10.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.963+0000 7f167033e640 1 -- start start 2024-04-25T00:37:10.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.963+0000 7f167033e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166806e440 0x7f16680b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.963+0000 7f167033e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16680b41e0 con 0x7f16680b3140 2024-04-25T00:37:10.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.963+0000 7f167033e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f166806d960 con 0x7f16680b3c30 2024-04-25T00:37:10.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.963+0000 7f167033e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f16680b5050 con 0x7f166806e440 2024-04-25T00:37:10.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.963+0000 7f166e0da640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f16680b3140 0x7f16680b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41474/0 (socket says 172.21.15.29:41474) 2024-04-25T00:37:10.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.963+0000 7f166e0da640 1 -- 172.21.15.29:0/3749141389 learned_addr learned my addr 172.21.15.29:0/3749141389 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:10.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.963+0000 7f166e8db640 1 --2- 172.21.15.29:0/3749141389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166806e440 0x7f16680b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2916293740 0 0) 0x7f16680b41e0 con 0x7f16680b3140 2024-04-25T00:37:10.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1654003530 con 0x7f16680b3140 2024-04-25T00:37:10.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1870809202 0 0) 0x7f1654003530 con 0x7f16680b3140 2024-04-25T00:37:10.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166806e440 msgr2=0x7f16680b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:10.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 --2- 172.21.15.29:0/3749141389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166806e440 0x7f16680b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 >> v1:172.21.15.157:6789/0 conn(0x7f16680b3c30 legacy=0x7f16680ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166806dc40 con 0x7f16680b3140 2024-04-25T00:37:10.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f1664001680 con 0x7f16680b3140 2024-04-25T00:37:10.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 >> v1:172.21.15.29:6789/0 conn(0x7f16680b3140 legacy=0x7f16680b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 --2- 172.21.15.29:0/3749141389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16540039f0 0x7f1654003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 --2- 172.21.15.29:0/3749141389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166806e440 0x7f1654004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 --2- 172.21.15.29:0/3749141389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1654004d50 0x7f16540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f16680b5050 con 0x7f16540039f0 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f165400ba80 con 0x7f1654004d50 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f165400bc30 con 0x7f166806e440 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166e0da640 1 --2- 172.21.15.29:0/3749141389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16540039f0 0x7f1654003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d8d9640 1 --2- 172.21.15.29:0/3749141389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166806e440 0x7f1654004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166e8db640 1 --2- 172.21.15.29:0/3749141389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1654004d50 0x7f16540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166e0da640 1 -- 172.21.15.29:0/3749141389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166806e440 msgr2=0x7f1654004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166e0da640 1 --2- 172.21.15.29:0/3749141389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166806e440 0x7f1654004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166e0da640 1 -- 172.21.15.29:0/3749141389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1654004d50 msgr2=0x7f16540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166e0da640 1 --2- 172.21.15.29:0/3749141389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1654004d50 0x7f16540051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166e0da640 1 -- 172.21.15.29:0/3749141389 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1654003530 con 0x7f16540039f0 2024-04-25T00:37:10.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166e0da640 1 --2- 172.21.15.29:0/3749141389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16540039f0 0x7f1654003dd0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f166400c160 tx=0x7f166401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c12b5c7706f69ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:10.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664007c90 con 0x7f16540039f0 2024-04-25T00:37:10.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16640a9070 con 0x7f16540039f0 2024-04-25T00:37:10.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.967+0000 7f166d0d8640 1 -- 172.21.15.29:0/3749141389 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664005d40 con 0x7f16540039f0 2024-04-25T00:37:10.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 -- 172.21.15.29:0/3749141389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16540039f0 msgr2=0x7f1654003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 --2- 172.21.15.29:0/3749141389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16540039f0 0x7f1654003dd0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f166400c160 tx=0x7f166401cbe0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f166e8db640 1 -- 172.21.15.29:0/3749141389 reap_dead start 2024-04-25T00:37:10.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 -- 172.21.15.29:0/3749141389 shutdown_connections 2024-04-25T00:37:10.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 -- 172.21.15.29:0/3749141389 >> 172.21.15.29:0/3749141389 conn(0x7f16680b1870 msgr2=0x7f16680b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:10.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 -- 172.21.15.29:0/3749141389 shutdown_connections 2024-04-25T00:37:10.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 -- 172.21.15.29:0/3749141389 wait complete. 2024-04-25T00:37:10.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 Processor -- start 2024-04-25T00:37:10.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 -- start start 2024-04-25T00:37:10.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16540039f0 0x7f16680c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166813aa20 0x7f16680c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166813ae90 0x7f166813dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1668167d80 con 0x7f16540039f0 2024-04-25T00:37:10.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1668167c00 con 0x7f166813aa20 2024-04-25T00:37:10.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1668167f00 con 0x7f166813ae90 2024-04-25T00:37:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f166e0da640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16540039f0 0x7f16680c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f166e0da640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16540039f0 0x7f16680c4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59064/0 (socket says 172.21.15.29:59064) 2024-04-25T00:37:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f166e0da640 1 -- 172.21.15.29:0/2411228012 learned_addr learned my addr 172.21.15.29:0/2411228012 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f166d8d9640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166813aa20 0x7f16680c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f166e8db640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166813ae90 0x7f166813dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f166e0da640 1 -- 172.21.15.29:0/2411228012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166813ae90 msgr2=0x7f166813dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f166e0da640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166813ae90 0x7f166813dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f166e0da640 1 -- 172.21.15.29:0/2411228012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166813aa20 msgr2=0x7f16680c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f166e0da640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166813aa20 0x7f16680c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f166e0da640 1 -- 172.21.15.29:0/2411228012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166815bbb0 con 0x7f16540039f0 2024-04-25T00:37:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f166e8db640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166813ae90 0x7f166813dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f166e0da640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16540039f0 0x7f16680c4ae0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f166401ca10 tx=0x7f1654005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f165effd640 1 -- 172.21.15.29:0/2411228012 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664005d20 con 0x7f16540039f0 2024-04-25T00:37:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f165effd640 1 -- 172.21.15.29:0/2411228012 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16640047f0 con 0x7f16540039f0 2024-04-25T00:37:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 -- 172.21.15.29:0/2411228012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166813f7f0 con 0x7f16540039f0 2024-04-25T00:37:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f165effd640 1 -- 172.21.15.29:0/2411228012 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664004bc0 con 0x7f16540039f0 2024-04-25T00:37:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.971+0000 7f167033e640 1 -- 172.21.15.29:0/2411228012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166813fdb0 con 0x7f16540039f0 2024-04-25T00:37:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.975+0000 7f167033e640 1 -- 172.21.15.29:0/2411228012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1668069270 con 0x7f16540039f0 2024-04-25T00:37:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.975+0000 7f165effd640 1 -- 172.21.15.29:0/2411228012 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f1664007070 con 0x7f16540039f0 2024-04-25T00:37:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.975+0000 7f165effd640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f164c0679e0 0x7f164c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.975+0000 7f165effd640 1 -- 172.21.15.29:0/2411228012 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f166411ede0 con 0x7f16540039f0 2024-04-25T00:37:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.975+0000 7f166d8d9640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f164c0679e0 0x7f164c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:10.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.975+0000 7f166d8d9640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f164c0679e0 0x7f164c069ea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f16681f8200 tx=0x7f1658009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:10.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:10.979+0000 7f165effd640 1 -- 172.21.15.29:0/2411228012 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f16640be070 con 0x7f16540039f0 2024-04-25T00:37:11.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.175+0000 7f167033e640 1 -- 172.21.15.29:0/2411228012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1668159e70 con 0x7f16540039f0 2024-04-25T00:37:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.179+0000 7f165effd640 1 -- 172.21.15.29:0/2411228012 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5186 (secure 0 0 0) 0x7f16640168f0 con 0x7f16540039f0 2024-04-25T00:37:11.181 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:11.182 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2434273880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2434273880},{"type":"v1","addr":"172.21.15.157:6833","nonce":2434273880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/558648019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":558648019},{"type":"v1","addr":"172.21.15.157:6835","nonce":558648019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":7665,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/4178766542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4178766542},{"type":"v1","addr":"172.21.15.29:6833","nonce":4178766542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:36:39.192693+0000","modified":"2024-04-25T00:36:42.218595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7404":{"gid":7404,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/3939243143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3939243143},{"type":"v1","addr":"172.21.15.29:6835","nonce":3939243143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7404,"qdb_cluster":[7404]},"id":6}]} 2024-04-25T00:37:11.182 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 59 2024-04-25T00:37:11.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.179+0000 7f167033e640 1 -- 172.21.15.29:0/2411228012 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f164c0679e0 msgr2=0x7f164c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.179+0000 7f167033e640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f164c0679e0 0x7f164c069ea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f16681f8200 tx=0x7f1658009040 comp rx=0 tx=0).stop 2024-04-25T00:37:11.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.179+0000 7f167033e640 1 -- 172.21.15.29:0/2411228012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16540039f0 msgr2=0x7f16680c4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.179+0000 7f167033e640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16540039f0 0x7f16680c4ae0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f166401ca10 tx=0x7f1654005840 comp rx=0 tx=0).stop 2024-04-25T00:37:11.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.179+0000 7f167033e640 1 -- 172.21.15.29:0/2411228012 shutdown_connections 2024-04-25T00:37:11.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.179+0000 7f167033e640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f164c0679e0 0x7f164c069ea0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.179+0000 7f167033e640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f166813ae90 0x7f166813dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.179+0000 7f167033e640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166813aa20 0x7f16680c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.179+0000 7f167033e640 1 --2- 172.21.15.29:0/2411228012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16540039f0 0x7f16680c4ae0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.179+0000 7f167033e640 1 -- 172.21.15.29:0/2411228012 >> 172.21.15.29:0/2411228012 conn(0x7f16680b1870 msgr2=0x7f1668144800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:11.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.179+0000 7f167033e640 1 -- 172.21.15.29:0/2411228012 shutdown_connections 2024-04-25T00:37:11.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.179+0000 7f167033e640 1 -- 172.21.15.29:0/2411228012 wait complete. 2024-04-25T00:37:11.200 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:37:11.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.315+0000 7fd64b378640 1 Processor -- start 2024-04-25T00:37:11.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.315+0000 7fd64b378640 1 -- start start 2024-04-25T00:37:11.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd64b378640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd644162420 0x7fd644070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd64b378640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd6440b4e00 0x7fd6440b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd64b378640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd644071910 con 0x7fd6440b30b0 2024-04-25T00:37:11.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd64b378640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd6440b3500 con 0x7fd644162420 2024-04-25T00:37:11.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd64b378640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd644071bf0 con 0x7fd6440b4e00 2024-04-25T00:37:11.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd649915640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd6440b30b0 0x7fd644166020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41484/0 (socket says 172.21.15.29:41484) 2024-04-25T00:37:11.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd649114640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd644162420 0x7fd644070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd649915640 1 -- 172.21.15.29:0/3375815395 learned_addr learned my addr 172.21.15.29:0/3375815395 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:11.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd649114640 1 -- 172.21.15.29:0/3375815395 >> v2:172.21.15.157:3300/0 conn(0x7fd6440b4e00 msgr2=0x7fd6440b2b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:37:11.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd649114640 1 --2- 172.21.15.29:0/3375815395 >> v2:172.21.15.157:3300/0 conn(0x7fd6440b4e00 0x7fd6440b2b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd649114640 1 -- 172.21.15.29:0/3375815395 >> v1:172.21.15.29:6789/0 conn(0x7fd6440b30b0 legacy=0x7fd644166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd649114640 1 -- 172.21.15.29:0/3375815395 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd644071ed0 con 0x7fd644162420 2024-04-25T00:37:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd649114640 1 --2- 172.21.15.29:0/3375815395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd644162420 0x7fd644070f80 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fd6440af790 tx=0x7fd63c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a9c21355659241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd63bfff640 1 -- 172.21.15.29:0/3375815395 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0b2070 con 0x7fd644162420 2024-04-25T00:37:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd63bfff640 1 -- 172.21.15.29:0/3375815395 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd63c0a5890 con 0x7fd644162420 2024-04-25T00:37:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.319+0000 7fd63bfff640 1 -- 172.21.15.29:0/3375815395 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0a5c60 con 0x7fd644162420 2024-04-25T00:37:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 -- 172.21.15.29:0/3375815395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd644162420 msgr2=0x7fd644070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 --2- 172.21.15.29:0/3375815395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd644162420 0x7fd644070f80 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fd6440af790 tx=0x7fd63c01c950 comp rx=0 tx=0).stop 2024-04-25T00:37:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 -- 172.21.15.29:0/3375815395 shutdown_connections 2024-04-25T00:37:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 --2- 172.21.15.29:0/3375815395 >> v2:172.21.15.157:3300/0 conn(0x7fd6440b4e00 0x7fd6440b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 --2- 172.21.15.29:0/3375815395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd644162420 0x7fd644070f80 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 -- 172.21.15.29:0/3375815395 >> 172.21.15.29:0/3375815395 conn(0x7fd6440b1870 msgr2=0x7fd6440b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 -- 172.21.15.29:0/3375815395 shutdown_connections 2024-04-25T00:37:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 -- 172.21.15.29:0/3375815395 wait complete. 2024-04-25T00:37:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 Processor -- start 2024-04-25T00:37:11.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 -- start start 2024-04-25T00:37:11.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6440b30b0 0x7fd644155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6440b4e00 0x7fd64415b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd644162420 0x7fd644159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd64416f3c0 con 0x7fd6440b30b0 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd6440716d0 con 0x7fd6440b4e00 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd64416f540 con 0x7fd644162420 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd649114640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6440b30b0 0x7fd644155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd649114640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6440b30b0 0x7fd644155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59076/0 (socket says 172.21.15.29:59076) 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd649114640 1 -- 172.21.15.29:0/4124652756 learned_addr learned my addr 172.21.15.29:0/4124652756 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd649114640 1 -- 172.21.15.29:0/4124652756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd644162420 msgr2=0x7fd644159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd649114640 1 --2- 172.21.15.29:0/4124652756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd644162420 0x7fd644159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd649114640 1 -- 172.21.15.29:0/4124652756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6440b4e00 msgr2=0x7fd64415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd649114640 1 --2- 172.21.15.29:0/4124652756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6440b4e00 0x7fd64415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd649114640 1 -- 172.21.15.29:0/4124652756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6441563e0 con 0x7fd6440b30b0 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd649114640 1 --2- 172.21.15.29:0/4124652756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6440b30b0 0x7fd644155ea0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fd63c019560 tx=0x7fd63c0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd639ffb640 1 -- 172.21.15.29:0/4124652756 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0b2050 con 0x7fd6440b30b0 2024-04-25T00:37:11.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd639ffb640 1 -- 172.21.15.29:0/4124652756 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd63c0be070 con 0x7fd6440b30b0 2024-04-25T00:37:11.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 -- 172.21.15.29:0/4124652756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64415b7c0 con 0x7fd6440b30b0 2024-04-25T00:37:11.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd639ffb640 1 -- 172.21.15.29:0/4124652756 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0ada50 con 0x7fd6440b30b0 2024-04-25T00:37:11.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.323+0000 7fd64b378640 1 -- 172.21.15.29:0/4124652756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd64415a7a0 con 0x7fd6440b30b0 2024-04-25T00:37:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.327+0000 7fd64b378640 1 -- 172.21.15.29:0/4124652756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd644069270 con 0x7fd6440b30b0 2024-04-25T00:37:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.327+0000 7fd639ffb640 1 -- 172.21.15.29:0/4124652756 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fd63c0adbf0 con 0x7fd6440b30b0 2024-04-25T00:37:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.327+0000 7fd639ffb640 1 --2- 172.21.15.29:0/4124652756 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd6280679e0 0x7fd628069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.327+0000 7fd639ffb640 1 -- 172.21.15.29:0/4124652756 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fd63c121130 con 0x7fd6440b30b0 2024-04-25T00:37:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.327+0000 7fd648913640 1 --2- 172.21.15.29:0/4124652756 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd6280679e0 0x7fd628069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.327+0000 7fd648913640 1 --2- 172.21.15.29:0/4124652756 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd6280679e0 0x7fd628069ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd64416cb80 tx=0x7fd634006c50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:11.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.331+0000 7fd639ffb640 1 -- 172.21.15.29:0/4124652756 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd63c016540 con 0x7fd6440b30b0 2024-04-25T00:37:11.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.527+0000 7fd64b378640 1 -- 172.21.15.29:0/4124652756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd644072fa0 con 0x7fd6440b30b0 2024-04-25T00:37:11.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.531+0000 7fd639ffb640 1 -- 172.21.15.29:0/4124652756 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5186 (secure 0 0 0) 0x7fd63c0f1af0 con 0x7fd6440b30b0 2024-04-25T00:37:11.534 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:11.534 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2434273880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2434273880},{"type":"v1","addr":"172.21.15.157:6833","nonce":2434273880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/558648019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":558648019},{"type":"v1","addr":"172.21.15.157:6835","nonce":558648019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":7665,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/4178766542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4178766542},{"type":"v1","addr":"172.21.15.29:6833","nonce":4178766542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:36:39.192693+0000","modified":"2024-04-25T00:36:42.218595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7404":{"gid":7404,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/3939243143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3939243143},{"type":"v1","addr":"172.21.15.29:6835","nonce":3939243143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7404,"qdb_cluster":[7404]},"id":6}]} 2024-04-25T00:37:11.534 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 59 2024-04-25T00:37:11.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.535+0000 7fd64b378640 1 -- 172.21.15.29:0/4124652756 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd6280679e0 msgr2=0x7fd628069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.535+0000 7fd64b378640 1 --2- 172.21.15.29:0/4124652756 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd6280679e0 0x7fd628069ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd64416cb80 tx=0x7fd634006c50 comp rx=0 tx=0).stop 2024-04-25T00:37:11.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.535+0000 7fd64b378640 1 -- 172.21.15.29:0/4124652756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6440b30b0 msgr2=0x7fd644155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.535+0000 7fd64b378640 1 --2- 172.21.15.29:0/4124652756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6440b30b0 0x7fd644155ea0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fd63c019560 tx=0x7fd63c0196f0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.535+0000 7fd64b378640 1 -- 172.21.15.29:0/4124652756 shutdown_connections 2024-04-25T00:37:11.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.535+0000 7fd64b378640 1 --2- 172.21.15.29:0/4124652756 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd6280679e0 0x7fd628069ea0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.535+0000 7fd64b378640 1 --2- 172.21.15.29:0/4124652756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd644162420 0x7fd644159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.535+0000 7fd64b378640 1 --2- 172.21.15.29:0/4124652756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6440b4e00 0x7fd64415b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.535+0000 7fd64b378640 1 --2- 172.21.15.29:0/4124652756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6440b30b0 0x7fd644155ea0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.535+0000 7fd64b378640 1 -- 172.21.15.29:0/4124652756 >> 172.21.15.29:0/4124652756 conn(0x7fd6440b1870 msgr2=0x7fd644165240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:11.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.535+0000 7fd64b378640 1 -- 172.21.15.29:0/4124652756 shutdown_connections 2024-04-25T00:37:11.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.535+0000 7fd64b378640 1 -- 172.21.15.29:0/4124652756 wait complete. 2024-04-25T00:37:11.554 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:37:11.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efda4fa1640 1 Processor -- start 2024-04-25T00:37:11.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efda4fa1640 1 -- start start 2024-04-25T00:37:11.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efda4fa1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efda00cf690 0x7efda00d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efda4fa1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda00d2a90 0x7efda00b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efda4fa1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efda00cfc10 con 0x7efda015e200 2024-04-25T00:37:11.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efda4fa1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7efda00d2ee0 con 0x7efda00cf690 2024-04-25T00:37:11.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efda4fa1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efda00b38d0 con 0x7efda00d2a90 2024-04-25T00:37:11.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efd95d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efda00cf690 0x7efda00d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efd95d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efda00cf690 0x7efda00d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59086/0 (socket says 172.21.15.29:59086) 2024-04-25T00:37:11.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efd95d9b640 1 -- 172.21.15.29:0/141027303 learned_addr learned my addr 172.21.15.29:0/141027303 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:11.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efd9ed9d640 1 --2- 172.21.15.29:0/141027303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda00d2a90 0x7efda00b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efd95d9b640 1 -- 172.21.15.29:0/141027303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda00d2a90 msgr2=0x7efda00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:11.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efd95d9b640 1 --2- 172.21.15.29:0/141027303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda00d2a90 0x7efda00b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efd95d9b640 1 -- 172.21.15.29:0/141027303 >> v1:172.21.15.157:6789/0 conn(0x7efda015e200 legacy=0x7efda00d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:11.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.671+0000 7efd95d9b640 1 -- 172.21.15.29:0/141027303 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda00cfef0 con 0x7efda00cf690 2024-04-25T00:37:11.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.675+0000 7efd95d9b640 1 --2- 172.21.15.29:0/141027303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda00cf690 0x7efda00d2550 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7efd88009870 tx=0x7efd8801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c52d6d1b41a75b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:11.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.675+0000 7efd9dd9b640 1 -- 172.21.15.29:0/141027303 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd880b1070 con 0x7efda00cf690 2024-04-25T00:37:11.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.675+0000 7efd9dd9b640 1 -- 172.21.15.29:0/141027303 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd880ac630 con 0x7efda00cf690 2024-04-25T00:37:11.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.675+0000 7efd9dd9b640 1 -- 172.21.15.29:0/141027303 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd880a8050 con 0x7efda00cf690 2024-04-25T00:37:11.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.675+0000 7efda4fa1640 1 -- 172.21.15.29:0/141027303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda00cf690 msgr2=0x7efda00d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.675+0000 7efda4fa1640 1 --2- 172.21.15.29:0/141027303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda00cf690 0x7efda00d2550 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7efd88009870 tx=0x7efd8801c520 comp rx=0 tx=0).stop 2024-04-25T00:37:11.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.675+0000 7efda4fa1640 1 -- 172.21.15.29:0/141027303 shutdown_connections 2024-04-25T00:37:11.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.675+0000 7efda4fa1640 1 --2- 172.21.15.29:0/141027303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda00d2a90 0x7efda00b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.675+0000 7efda4fa1640 1 --2- 172.21.15.29:0/141027303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda00cf690 0x7efda00d2550 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.675+0000 7efda4fa1640 1 -- 172.21.15.29:0/141027303 >> 172.21.15.29:0/141027303 conn(0x7efda009d280 msgr2=0x7efda00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:11.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.675+0000 7efda4fa1640 1 -- 172.21.15.29:0/141027303 shutdown_connections 2024-04-25T00:37:11.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.675+0000 7efda4fa1640 1 -- 172.21.15.29:0/141027303 wait complete. 2024-04-25T00:37:11.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efda4fa1640 1 Processor -- start 2024-04-25T00:37:11.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efda4fa1640 1 -- start start 2024-04-25T00:37:11.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efda4fa1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda00cf690 0x7efda01543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efda4fa1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda00d2a90 0x7efda015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efda4fa1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda015e200 0x7efda00ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efda4fa1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efda00b51f0 con 0x7efda015e200 2024-04-25T00:37:11.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efda4fa1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efda00b5070 con 0x7efda00cf690 2024-04-25T00:37:11.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efda4fa1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efda00b5370 con 0x7efda00d2a90 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd9ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda015e200 0x7efda00ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd9ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda015e200 0x7efda00ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59092/0 (socket says 172.21.15.29:59092) 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd9ed9d640 1 -- 172.21.15.29:0/2051196370 learned_addr learned my addr 172.21.15.29:0/2051196370 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd95d9b640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda00d2a90 0x7efda015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd9e59c640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda00cf690 0x7efda01543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd9ed9d640 1 -- 172.21.15.29:0/2051196370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda00d2a90 msgr2=0x7efda015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd9ed9d640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda00d2a90 0x7efda015fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd9ed9d640 1 -- 172.21.15.29:0/2051196370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda00cf690 msgr2=0x7efda01543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd9ed9d640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda00cf690 0x7efda01543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd9ed9d640 1 -- 172.21.15.29:0/2051196370 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda015eb90 con 0x7efda015e200 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd9ed9d640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda015e200 0x7efda00ca120 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7efd98007e40 tx=0x7efd9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd977fe640 1 -- 172.21.15.29:0/2051196370 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98014620 con 0x7efda015e200 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd977fe640 1 -- 172.21.15.29:0/2051196370 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd98019070 con 0x7efda015e200 2024-04-25T00:37:11.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efda4fa1640 1 -- 172.21.15.29:0/2051196370 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda015ee70 con 0x7efda015e200 2024-04-25T00:37:11.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd977fe640 1 -- 172.21.15.29:0/2051196370 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98014a20 con 0x7efda015e200 2024-04-25T00:37:11.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efda4fa1640 1 -- 172.21.15.29:0/2051196370 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda015f430 con 0x7efda015e200 2024-04-25T00:37:11.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efda4fa1640 1 -- 172.21.15.29:0/2051196370 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda0069270 con 0x7efda015e200 2024-04-25T00:37:11.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd977fe640 1 -- 172.21.15.29:0/2051196370 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7efd98014bc0 con 0x7efda015e200 2024-04-25T00:37:11.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd977fe640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd800679e0 0x7efd80069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd977fe640 1 -- 172.21.15.29:0/2051196370 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7efd98086720 con 0x7efda015e200 2024-04-25T00:37:11.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.679+0000 7efd9e59c640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd800679e0 0x7efd80069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.683+0000 7efd9e59c640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd800679e0 0x7efd80069ea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efd9000b810 tx=0x7efd90008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:11.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.687+0000 7efd977fe640 1 -- 172.21.15.29:0/2051196370 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efd98011260 con 0x7efda015e200 2024-04-25T00:37:11.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.851+0000 7efda4fa1640 1 -- 172.21.15.29:0/2051196370 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efda00d1d50 con 0x7efda015e200 2024-04-25T00:37:11.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.851+0000 7efd977fe640 1 -- 172.21.15.29:0/2051196370 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16741 (secure 0 0 0) 0x7efd980570f0 con 0x7efda015e200 2024-04-25T00:37:11.855 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:11.855 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":91,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:37:04.487226+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:36:37.482565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:36:38.176961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6833/1764147340":"2024-04-26T00:37:01.479125+0000","172.21.15.29:6832/1764147340":"2024-04-26T00:37:01.479125+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:37:11.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.855+0000 7efda4fa1640 1 -- 172.21.15.29:0/2051196370 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd800679e0 msgr2=0x7efd80069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.855+0000 7efda4fa1640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd800679e0 0x7efd80069ea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efd9000b810 tx=0x7efd90008040 comp rx=0 tx=0).stop 2024-04-25T00:37:11.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.855+0000 7efda4fa1640 1 -- 172.21.15.29:0/2051196370 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda015e200 msgr2=0x7efda00ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.855+0000 7efda4fa1640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda015e200 0x7efda00ca120 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7efd98007e40 tx=0x7efd9800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.855+0000 7efda4fa1640 1 -- 172.21.15.29:0/2051196370 shutdown_connections 2024-04-25T00:37:11.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.855+0000 7efda4fa1640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd800679e0 0x7efd80069ea0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.855+0000 7efda4fa1640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda015e200 0x7efda00ca120 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.855+0000 7efda4fa1640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda00d2a90 0x7efda015fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.855+0000 7efda4fa1640 1 --2- 172.21.15.29:0/2051196370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda00cf690 0x7efda01543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.855+0000 7efda4fa1640 1 -- 172.21.15.29:0/2051196370 >> 172.21.15.29:0/2051196370 conn(0x7efda009d280 msgr2=0x7efda015d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:11.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.855+0000 7efda4fa1640 1 -- 172.21.15.29:0/2051196370 shutdown_connections 2024-04-25T00:37:11.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.855+0000 7efda4fa1640 1 -- 172.21.15.29:0/2051196370 wait complete. 2024-04-25T00:37:11.873 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:37:11.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.983+0000 7fe645267640 1 Processor -- start 2024-04-25T00:37:11.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.983+0000 7fe645267640 1 -- start start 2024-04-25T00:37:11.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.983+0000 7fe645267640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400b4780 0x7fe64015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.983+0000 7fe645267640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe6400cc240 0x7fe64015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.983+0000 7fe645267640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6400b3720 con 0x7fe64015e8d0 2024-04-25T00:37:11.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.983+0000 7fe645267640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe6400b42f0 con 0x7fe6400cc240 2024-04-25T00:37:11.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.983+0000 7fe645267640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe6400b39c0 con 0x7fe6400b4780 2024-04-25T00:37:11.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.983+0000 7fe63effd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe64015e8d0 0x7fe64015ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41490/0 (socket says 172.21.15.29:41490) 2024-04-25T00:37:11.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.983+0000 7fe63effd640 1 -- 172.21.15.29:0/1450462257 learned_addr learned my addr 172.21.15.29:0/1450462257 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:11.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.983+0000 7fe63f7fe640 1 --2- 172.21.15.29:0/1450462257 >> v2:172.21.15.157:3300/0 conn(0x7fe6400cc240 0x7fe64015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.983+0000 7fe637fff640 1 --2- 172.21.15.29:0/1450462257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400b4780 0x7fe64015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.983+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4084924447 0 0) 0x7fe6400b3720 con 0x7fe64015e8d0 2024-04-25T00:37:11.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.983+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe62c003530 con 0x7fe64015e8d0 2024-04-25T00:37:11.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 266963819 0 0) 0x7fe62c003530 con 0x7fe64015e8d0 2024-04-25T00:37:11.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400b4780 msgr2=0x7fe64015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:11.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 --2- 172.21.15.29:0/1450462257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400b4780 0x7fe64015f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 >> v2:172.21.15.157:3300/0 conn(0x7fe6400cc240 msgr2=0x7fe64015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:11.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 --2- 172.21.15.29:0/1450462257 >> v2:172.21.15.157:3300/0 conn(0x7fe6400cc240 0x7fe64015c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6400b3ca0 con 0x7fe64015e8d0 2024-04-25T00:37:11.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fe630001680 con 0x7fe64015e8d0 2024-04-25T00:37:11.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 >> v1:172.21.15.29:6789/0 conn(0x7fe64015e8d0 legacy=0x7fe64015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 --2- 172.21.15.29:0/1450462257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe62c0039c0 0x7fe62c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 --2- 172.21.15.29:0/1450462257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400b4780 0x7fe62c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 --2- 172.21.15.29:0/1450462257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c004d50 0x7fe62c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe6400b42f0 con 0x7fe62c004d50 2024-04-25T00:37:11.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe6400b39c0 con 0x7fe62c0039c0 2024-04-25T00:37:11.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe62c00dbe0 con 0x7fe6400b4780 2024-04-25T00:37:11.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63f7fe640 1 --2- 172.21.15.29:0/1450462257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c004d50 0x7fe62c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63effd640 1 --2- 172.21.15.29:0/1450462257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe62c0039c0 0x7fe62c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe637fff640 1 --2- 172.21.15.29:0/1450462257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400b4780 0x7fe62c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63f7fe640 1 -- 172.21.15.29:0/1450462257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400b4780 msgr2=0x7fe62c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:11.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63f7fe640 1 --2- 172.21.15.29:0/1450462257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400b4780 0x7fe62c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63f7fe640 1 -- 172.21.15.29:0/1450462257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe62c0039c0 msgr2=0x7fe62c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:11.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63f7fe640 1 --2- 172.21.15.29:0/1450462257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe62c0039c0 0x7fe62c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63f7fe640 1 -- 172.21.15.29:0/1450462257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c003530 con 0x7fe62c004d50 2024-04-25T00:37:11.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe637fff640 1 --2- 172.21.15.29:0/1450462257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400b4780 0x7fe62c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:11.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63f7fe640 1 --2- 172.21.15.29:0/1450462257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c004d50 0x7fe62c0051d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe638009fc0 tx=0x7fe63801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c065dba7bf7c2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:11.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6380b2070 con 0x7fe62c004d50 2024-04-25T00:37:11.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6380a5cd0 con 0x7fe62c004d50 2024-04-25T00:37:11.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63e7fc640 1 -- 172.21.15.29:0/1450462257 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638009c10 con 0x7fe62c004d50 2024-04-25T00:37:11.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe645267640 1 -- 172.21.15.29:0/1450462257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c004d50 msgr2=0x7fe62c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe645267640 1 --2- 172.21.15.29:0/1450462257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c004d50 0x7fe62c0051d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe638009fc0 tx=0x7fe63801ccc0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe63f7fe640 1 -- 172.21.15.29:0/1450462257 reap_dead start 2024-04-25T00:37:11.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe645267640 1 -- 172.21.15.29:0/1450462257 shutdown_connections 2024-04-25T00:37:11.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe645267640 1 -- 172.21.15.29:0/1450462257 >> 172.21.15.29:0/1450462257 conn(0x7fe6400b1870 msgr2=0x7fe6400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:11.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe645267640 1 -- 172.21.15.29:0/1450462257 shutdown_connections 2024-04-25T00:37:11.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.987+0000 7fe645267640 1 -- 172.21.15.29:0/1450462257 wait complete. 2024-04-25T00:37:11.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe645267640 1 Processor -- start 2024-04-25T00:37:11.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe645267640 1 -- start start 2024-04-25T00:37:11.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe645267640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c004d50 0x7fe640137530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe645267640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400cddd0 0x7fe6400ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe645267640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6400aebd0 0x7fe6400b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe645267640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe640144a20 con 0x7fe62c004d50 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe645267640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe64006cf10 con 0x7fe6400aebd0 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe645267640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe6400af0e0 con 0x7fe6400cddd0 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe63effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c004d50 0x7fe640137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe63effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c004d50 0x7fe640137530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59104/0 (socket says 172.21.15.29:59104) 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe63effd640 1 -- 172.21.15.29:0/3170044211 learned_addr learned my addr 172.21.15.29:0/3170044211 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe63effd640 1 -- 172.21.15.29:0/3170044211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400cddd0 msgr2=0x7fe6400ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe63f7fe640 1 --2- 172.21.15.29:0/3170044211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6400aebd0 0x7fe6400b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe63effd640 1 --2- 172.21.15.29:0/3170044211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400cddd0 0x7fe6400ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe63effd640 1 -- 172.21.15.29:0/3170044211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6400aebd0 msgr2=0x7fe6400b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe63effd640 1 --2- 172.21.15.29:0/3170044211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6400aebd0 0x7fe6400b0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe63effd640 1 -- 172.21.15.29:0/3170044211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6400af3c0 con 0x7fe62c004d50 2024-04-25T00:37:11.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe63effd640 1 --2- 172.21.15.29:0/3170044211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c004d50 0x7fe640137530 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe63000e7b0 tx=0x7fe63000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:11.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe6377fe640 1 -- 172.21.15.29:0/3170044211 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63000c870 con 0x7fe62c004d50 2024-04-25T00:37:11.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe6377fe640 1 -- 172.21.15.29:0/3170044211 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe630018070 con 0x7fe62c004d50 2024-04-25T00:37:11.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe6377fe640 1 -- 172.21.15.29:0/3170044211 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63000cc50 con 0x7fe62c004d50 2024-04-25T00:37:11.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe645267640 1 -- 172.21.15.29:0/3170044211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6400cd2f0 con 0x7fe62c004d50 2024-04-25T00:37:11.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe645267640 1 -- 172.21.15.29:0/3170044211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6400cd8b0 con 0x7fe62c004d50 2024-04-25T00:37:11.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe645267640 1 -- 172.21.15.29:0/3170044211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe640068520 con 0x7fe62c004d50 2024-04-25T00:37:11.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.991+0000 7fe6377fe640 1 -- 172.21.15.29:0/3170044211 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fe63001d460 con 0x7fe62c004d50 2024-04-25T00:37:11.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.995+0000 7fe6377fe640 1 --2- 172.21.15.29:0/3170044211 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe6200679e0 0x7fe620069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:11.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.995+0000 7fe637fff640 1 --2- 172.21.15.29:0/3170044211 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe6200679e0 0x7fe620069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:11.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.995+0000 7fe6377fe640 1 -- 172.21.15.29:0/3170044211 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fe630085dc0 con 0x7fe62c004d50 2024-04-25T00:37:11.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.995+0000 7fe637fff640 1 --2- 172.21.15.29:0/3170044211 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe6200679e0 0x7fe620069ea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe6400ae5e0 tx=0x7fe628008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:12.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:11.999+0000 7fe6377fe640 1 -- 172.21.15.29:0/3170044211 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe630056790 con 0x7fe62c004d50 2024-04-25T00:37:12.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.195+0000 7fe645267640 1 -- 172.21.15.29:0/3170044211 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe640069250 con 0x7fe62c004d50 2024-04-25T00:37:12.197 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-04-25T00:37:12.195+0000 7f3ac00b5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:37:12.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.195+0000 7fe6377fe640 1 -- 172.21.15.29:0/3170044211 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) ==== 161+0+0 (secure 0 0 0) 0x7fe630056090 con 0x7fe62c004d50 2024-04-25T00:37:12.200 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:37:12.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.199+0000 7fe645267640 1 -- 172.21.15.29:0/3170044211 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe6200679e0 msgr2=0x7fe620069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:12.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.199+0000 7fe645267640 1 --2- 172.21.15.29:0/3170044211 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe6200679e0 0x7fe620069ea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe6400ae5e0 tx=0x7fe628008040 comp rx=0 tx=0).stop 2024-04-25T00:37:12.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.199+0000 7fe645267640 1 -- 172.21.15.29:0/3170044211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c004d50 msgr2=0x7fe640137530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:12.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.199+0000 7fe645267640 1 --2- 172.21.15.29:0/3170044211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c004d50 0x7fe640137530 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe63000e7b0 tx=0x7fe63000ec70 comp rx=0 tx=0).stop 2024-04-25T00:37:12.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.199+0000 7fe645267640 1 -- 172.21.15.29:0/3170044211 shutdown_connections 2024-04-25T00:37:12.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.199+0000 7fe645267640 1 --2- 172.21.15.29:0/3170044211 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe6200679e0 0x7fe620069ea0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:12.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.199+0000 7fe645267640 1 --2- 172.21.15.29:0/3170044211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6400aebd0 0x7fe6400b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:12.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.199+0000 7fe645267640 1 --2- 172.21.15.29:0/3170044211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400cddd0 0x7fe6400ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:12.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.199+0000 7fe645267640 1 --2- 172.21.15.29:0/3170044211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c004d50 0x7fe640137530 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:12.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.199+0000 7fe645267640 1 -- 172.21.15.29:0/3170044211 >> 172.21.15.29:0/3170044211 conn(0x7fe6400b1870 msgr2=0x7fe6400af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:12.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.199+0000 7fe645267640 1 -- 172.21.15.29:0/3170044211 shutdown_connections 2024-04-25T00:37:12.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.199+0000 7fe645267640 1 -- 172.21.15.29:0/3170044211 wait complete. 2024-04-25T00:37:12.204 INFO:tasks.ceph.mds.c.smithi029.stderr: -1678> 2024-04-25T00:36:45.788+0000 7fcf16a05640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:37:12.220 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:37:12.230 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:37:12.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f059c034640 1 Processor -- start 2024-04-25T00:37:12.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f059c034640 1 -- start start 2024-04-25T00:37:12.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f059c034640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f05940cf690 0x7f05940d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f059c034640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05940d2a90 0x7f05940b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f059c034640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05940d2ea0 con 0x7f059415e200 2024-04-25T00:37:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f059c034640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f05940b38d0 con 0x7f05940cf690 2024-04-25T00:37:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f059c034640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f05940cfc10 con 0x7f05940d2a90 2024-04-25T00:37:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f05995cf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f05940cf690 0x7f05940d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f05995cf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f05940cf690 0x7f05940d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59120/0 (socket says 172.21.15.29:59120) 2024-04-25T00:37:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f05995cf640 1 -- 172.21.15.29:0/1457296175 learned_addr learned my addr 172.21.15.29:0/1457296175 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f059a5d1640 1 --2- 172.21.15.29:0/1457296175 >> v2:172.21.15.157:3300/0 conn(0x7f05940d2a90 0x7f05940b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f05995cf640 1 -- 172.21.15.29:0/1457296175 >> v2:172.21.15.157:3300/0 conn(0x7f05940d2a90 msgr2=0x7f05940b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f05995cf640 1 --2- 172.21.15.29:0/1457296175 >> v2:172.21.15.157:3300/0 conn(0x7f05940d2a90 0x7f05940b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f05995cf640 1 -- 172.21.15.29:0/1457296175 >> v1:172.21.15.157:6789/0 conn(0x7f059415e200 legacy=0x7f05940d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f05995cf640 1 -- 172.21.15.29:0/1457296175 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05940cfef0 con 0x7f05940cf690 2024-04-25T00:37:12.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.327+0000 7f05995cf640 1 --2- 172.21.15.29:0/1457296175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05940cf690 0x7f05940d2550 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f05940cfb20 tx=0x7f058401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baef1084de91a262 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:12.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f0598dce640 1 -- 172.21.15.29:0/1457296175 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05840b0070 con 0x7f05940cf690 2024-04-25T00:37:12.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f0598dce640 1 -- 172.21.15.29:0/1457296175 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05840ab600 con 0x7f05940cf690 2024-04-25T00:37:12.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f0598dce640 1 -- 172.21.15.29:0/1457296175 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05840ab9d0 con 0x7f05940cf690 2024-04-25T00:37:12.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 -- 172.21.15.29:0/1457296175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05940cf690 msgr2=0x7f05940d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:12.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 --2- 172.21.15.29:0/1457296175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05940cf690 0x7f05940d2550 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f05940cfb20 tx=0x7f058401c300 comp rx=0 tx=0).stop 2024-04-25T00:37:12.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 -- 172.21.15.29:0/1457296175 shutdown_connections 2024-04-25T00:37:12.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 --2- 172.21.15.29:0/1457296175 >> v2:172.21.15.157:3300/0 conn(0x7f05940d2a90 0x7f05940b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:12.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 --2- 172.21.15.29:0/1457296175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05940cf690 0x7f05940d2550 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:12.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 -- 172.21.15.29:0/1457296175 >> 172.21.15.29:0/1457296175 conn(0x7f059409d280 msgr2=0x7f05940d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:12.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 -- 172.21.15.29:0/1457296175 shutdown_connections 2024-04-25T00:37:12.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 -- 172.21.15.29:0/1457296175 wait complete. 2024-04-25T00:37:12.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 Processor -- start 2024-04-25T00:37:12.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 -- start start 2024-04-25T00:37:12.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05940cf690 0x7f0594160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:12.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05940d2a90 0x7f05940b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:12.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f059415e200 0x7f0594155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:12.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f05940b51f0 con 0x7f05940d2a90 2024-04-25T00:37:12.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05940b5070 con 0x7f059415e200 2024-04-25T00:37:12.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f05940b5370 con 0x7f05940cf690 2024-04-25T00:37:12.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f05995cf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05940d2a90 0x7f05940b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f05995cf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05940d2a90 0x7f05940b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59132/0 (socket says 172.21.15.29:59132) 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f05995cf640 1 -- 172.21.15.29:0/667507916 learned_addr learned my addr 172.21.15.29:0/667507916 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059a5d1640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f059415e200 0x7f0594155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f05995cf640 1 -- 172.21.15.29:0/667507916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05940cf690 msgr2=0x7f0594160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f0599dd0640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05940cf690 0x7f0594160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f05995cf640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05940cf690 0x7f0594160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f05995cf640 1 -- 172.21.15.29:0/667507916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f059415e200 msgr2=0x7f0594155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f05995cf640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f059415e200 0x7f0594155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f05995cf640 1 -- 172.21.15.29:0/667507916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05940c7aa0 con 0x7f05940d2a90 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059a5d1640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f059415e200 0x7f0594155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f05995cf640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05940d2a90 0x7f05940b6dd0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f05840ae040 tx=0x7f05840190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f058a7fc640 1 -- 172.21.15.29:0/667507916 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05840b0040 con 0x7f05940d2a90 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f058a7fc640 1 -- 172.21.15.29:0/667507916 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05840b9070 con 0x7f05940d2a90 2024-04-25T00:37:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 -- 172.21.15.29:0/667507916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05940c7de0 con 0x7f05940d2a90 2024-04-25T00:37:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f058a7fc640 1 -- 172.21.15.29:0/667507916 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05840b4680 con 0x7f05940d2a90 2024-04-25T00:37:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.331+0000 7f059c034640 1 -- 172.21.15.29:0/667507916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05940c8620 con 0x7f05940d2a90 2024-04-25T00:37:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.335+0000 7f059c034640 1 -- 172.21.15.29:0/667507916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0594069250 con 0x7f05940d2a90 2024-04-25T00:37:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.335+0000 7f058a7fc640 1 -- 172.21.15.29:0/667507916 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f05840b4820 con 0x7f05940d2a90 2024-04-25T00:37:12.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.335+0000 7f058a7fc640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f05780679e0 0x7f0578069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:12.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.335+0000 7f058a7fc640 1 -- 172.21.15.29:0/667507916 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5881+0+0 (secure 0 0 0) 0x7f058411fdc0 con 0x7f05940d2a90 2024-04-25T00:37:12.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.335+0000 7f0599dd0640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f05780679e0 0x7f0578069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:12.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.335+0000 7f0599dd0640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f05780679e0 0x7f0578069ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f059000a750 tx=0x7f0590008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:12.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.339+0000 7f058a7fc640 1 -- 172.21.15.29:0/667507916 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f05940c7de0 con 0x7f05940d2a90 2024-04-25T00:37:12.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:12.527+0000 7f059c034640 1 -- 172.21.15.29:0/667507916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0594157390 con 0x7f05940d2a90 2024-04-25T00:37:13.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.199+0000 7f058a7fc640 1 -- 172.21.15.29:0/667507916 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) ==== 104+0+0 (secure 0 0 0) 0x7f05940d2ea0 con 0x7f05940d2a90 2024-04-25T00:37:13.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.203+0000 7f059c034640 1 -- 172.21.15.29:0/667507916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f05780679e0 msgr2=0x7f0578069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:13.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.203+0000 7f059c034640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f05780679e0 0x7f0578069ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f059000a750 tx=0x7f0590008040 comp rx=0 tx=0).stop 2024-04-25T00:37:13.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.203+0000 7f059c034640 1 -- 172.21.15.29:0/667507916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05940d2a90 msgr2=0x7f05940b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:13.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.203+0000 7f059c034640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05940d2a90 0x7f05940b6dd0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f05840ae040 tx=0x7f05840190a0 comp rx=0 tx=0).stop 2024-04-25T00:37:13.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.203+0000 7f059c034640 1 -- 172.21.15.29:0/667507916 shutdown_connections 2024-04-25T00:37:13.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.203+0000 7f059c034640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f05780679e0 0x7f0578069ea0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:13.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.203+0000 7f059c034640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f059415e200 0x7f0594155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:13.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.203+0000 7f059c034640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05940d2a90 0x7f05940b6dd0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:13.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.203+0000 7f059c034640 1 --2- 172.21.15.29:0/667507916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05940cf690 0x7f0594160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:13.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.203+0000 7f059c034640 1 -- 172.21.15.29:0/667507916 >> 172.21.15.29:0/667507916 conn(0x7f059409d280 msgr2=0x7f059406cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:13.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.203+0000 7f059c034640 1 -- 172.21.15.29:0/667507916 shutdown_connections 2024-04-25T00:37:13.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.203+0000 7f059c034640 1 -- 172.21.15.29:0/667507916 wait complete. 2024-04-25T00:37:13.230 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:37:13.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa58b411640 1 Processor -- start 2024-04-25T00:37:13.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa58b411640 1 -- start start 2024-04-25T00:37:13.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa58b411640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5840d09c0 0x7fa5840cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:13.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa58b411640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5840b4640 0x7fa5840b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:13.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa58b411640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5840d2e20 con 0x7fa5840d46c0 2024-04-25T00:37:13.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa58b411640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa5840b3910 con 0x7fa5840b4640 2024-04-25T00:37:13.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa58b411640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa5840b41b0 con 0x7fa5840d09c0 2024-04-25T00:37:13.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa5899ae640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5840b4640 0x7fa5840b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:13.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa5899ae640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5840b4640 0x7fa5840b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:51056/0 (socket says 172.21.15.29:51056) 2024-04-25T00:37:13.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa5899ae640 1 -- 172.21.15.29:0/1687216124 learned_addr learned my addr 172.21.15.29:0/1687216124 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:13.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa5889ac640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa5840d46c0 0x7fa5840d0da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:43920/0 (socket says 172.21.15.29:43920) 2024-04-25T00:37:13.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa5891ad640 1 --2- 172.21.15.29:0/1687216124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5840d09c0 0x7fa5840cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:13.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa5899ae640 1 -- 172.21.15.29:0/1687216124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5840d09c0 msgr2=0x7fa5840cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:13.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa5899ae640 1 --2- 172.21.15.29:0/1687216124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5840d09c0 0x7fa5840cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:13.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa5899ae640 1 -- 172.21.15.29:0/1687216124 >> v1:172.21.15.157:6789/0 conn(0x7fa5840d46c0 legacy=0x7fa5840d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:13.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa5899ae640 1 -- 172.21.15.29:0/1687216124 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5840cfcd0 con 0x7fa5840b4640 2024-04-25T00:37:13.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.407+0000 7fa57bfff640 1 -- 172.21.15.29:0/1687216124 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1643978661 0 0) 0x7fa5840d2e20 con 0x7fa5840d46c0 2024-04-25T00:37:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.411+0000 7fa5899ae640 1 --2- 172.21.15.29:0/1687216124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5840b4640 0x7fa5840b4ad0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009e50 tx=0x7fa57c01c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=313ce2ce63a9f0ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.411+0000 7fa57bfff640 1 -- 172.21.15.29:0/1687216124 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0a5860 con 0x7fa5840b4640 2024-04-25T00:37:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.411+0000 7fa57bfff640 1 -- 172.21.15.29:0/1687216124 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa57c0a5a00 con 0x7fa5840b4640 2024-04-25T00:37:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.411+0000 7fa57bfff640 1 -- 172.21.15.29:0/1687216124 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0a5dd0 con 0x7fa5840b4640 2024-04-25T00:37:13.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.411+0000 7fa58b411640 1 -- 172.21.15.29:0/1687216124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5840b4640 msgr2=0x7fa5840b4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:13.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.411+0000 7fa58b411640 1 --2- 172.21.15.29:0/1687216124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5840b4640 0x7fa5840b4ad0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009e50 tx=0x7fa57c01c9a0 comp rx=0 tx=0).stop 2024-04-25T00:37:13.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.411+0000 7fa58b411640 1 -- 172.21.15.29:0/1687216124 shutdown_connections 2024-04-25T00:37:13.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.411+0000 7fa58b411640 1 --2- 172.21.15.29:0/1687216124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5840b4640 0x7fa5840b4ad0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:13.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.411+0000 7fa58b411640 1 --2- 172.21.15.29:0/1687216124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5840d09c0 0x7fa5840cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:13.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.411+0000 7fa58b411640 1 -- 172.21.15.29:0/1687216124 >> 172.21.15.29:0/1687216124 conn(0x7fa58409d280 msgr2=0x7fa5840d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:13.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.411+0000 7fa58b411640 1 -- 172.21.15.29:0/1687216124 shutdown_connections 2024-04-25T00:37:13.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.411+0000 7fa58b411640 1 -- 172.21.15.29:0/1687216124 wait complete. 2024-04-25T00:37:13.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa58b411640 1 Processor -- start 2024-04-25T00:37:13.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa58b411640 1 -- start start 2024-04-25T00:37:13.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa58b411640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5840b4640 0x7fa5841439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa58b411640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5840d09c0 0x7fa584155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa58b411640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5840d46c0 0x7fa58415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa58b411640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa58416ae50 con 0x7fa5840b4640 2024-04-25T00:37:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa58b411640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa58416acd0 con 0x7fa5840d46c0 2024-04-25T00:37:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa58b411640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa58416afd0 con 0x7fa5840d09c0 2024-04-25T00:37:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa5891ad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5840b4640 0x7fa5841439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa5891ad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5840b4640 0x7fa5841439d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59140/0 (socket says 172.21.15.29:59140) 2024-04-25T00:37:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa5891ad640 1 -- 172.21.15.29:0/3741357045 learned_addr learned my addr 172.21.15.29:0/3741357045 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa5899ae640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5840d46c0 0x7fa58415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa5889ac640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5840d09c0 0x7fa584155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa5891ad640 1 -- 172.21.15.29:0/3741357045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5840d09c0 msgr2=0x7fa584155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa5891ad640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5840d09c0 0x7fa584155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa5891ad640 1 -- 172.21.15.29:0/3741357045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5840d46c0 msgr2=0x7fa58415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:13.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa5891ad640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5840d46c0 0x7fa58415c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:13.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa5891ad640 1 -- 172.21.15.29:0/3741357045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa584143f10 con 0x7fa5840b4640 2024-04-25T00:37:13.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa5891ad640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5840b4640 0x7fa5841439d0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fa57400e8a0 tx=0x7fa57400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:13.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa579ffb640 1 -- 172.21.15.29:0/3741357045 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574017070 con 0x7fa5840b4640 2024-04-25T00:37:13.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa579ffb640 1 -- 172.21.15.29:0/3741357045 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa57400c940 con 0x7fa5840b4640 2024-04-25T00:37:13.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa579ffb640 1 -- 172.21.15.29:0/3741357045 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57400cd10 con 0x7fa5840b4640 2024-04-25T00:37:13.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa58b411640 1 -- 172.21.15.29:0/3741357045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5840cd0b0 con 0x7fa5840b4640 2024-04-25T00:37:13.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa58b411640 1 -- 172.21.15.29:0/3741357045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5840cd670 con 0x7fa5840b4640 2024-04-25T00:37:13.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa58b411640 1 -- 172.21.15.29:0/3741357045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5840692e0 con 0x7fa5840b4640 2024-04-25T00:37:13.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.415+0000 7fa579ffb640 1 -- 172.21.15.29:0/3741357045 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fa57401a070 con 0x7fa5840b4640 2024-04-25T00:37:13.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.419+0000 7fa579ffb640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5680679e0 0x7fa568069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:13.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.419+0000 7fa579ffb640 1 -- 172.21.15.29:0/3741357045 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5881+0+0 (secure 0 0 0) 0x7fa574085d00 con 0x7fa5840b4640 2024-04-25T00:37:13.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.419+0000 7fa5889ac640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5680679e0 0x7fa568069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:13.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.419+0000 7fa5889ac640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5680679e0 0x7fa568069ea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa584156aa0 tx=0x7fa580007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:13.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.423+0000 7fa579ffb640 1 -- 172.21.15.29:0/3741357045 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa584143f10 con 0x7fa5840b4640 2024-04-25T00:37:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:13.587+0000 7fa58b411640 1 -- 172.21.15.29:0/3741357045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa5840d07a0 con 0x7fa5840b4640 2024-04-25T00:37:14.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.539+0000 7fa579ffb640 1 -- 172.21.15.29:0/3741357045 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7fa574056030 con 0x7fa5840b4640 2024-04-25T00:37:14.543 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:37:14.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.543+0000 7fa58b411640 1 -- 172.21.15.29:0/3741357045 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5680679e0 msgr2=0x7fa568069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:14.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.543+0000 7fa58b411640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5680679e0 0x7fa568069ea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa584156aa0 tx=0x7fa580007c10 comp rx=0 tx=0).stop 2024-04-25T00:37:14.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.543+0000 7fa58b411640 1 -- 172.21.15.29:0/3741357045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5840b4640 msgr2=0x7fa5841439d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:14.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.543+0000 7fa58b411640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5840b4640 0x7fa5841439d0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fa57400e8a0 tx=0x7fa57400ed60 comp rx=0 tx=0).stop 2024-04-25T00:37:14.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.547+0000 7fa58b411640 1 -- 172.21.15.29:0/3741357045 shutdown_connections 2024-04-25T00:37:14.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.547+0000 7fa58b411640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5680679e0 0x7fa568069ea0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:14.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.547+0000 7fa58b411640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5840d46c0 0x7fa58415c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:14.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.547+0000 7fa58b411640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5840d09c0 0x7fa584155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:14.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.547+0000 7fa58b411640 1 --2- 172.21.15.29:0/3741357045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5840b4640 0x7fa5841439d0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:14.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.547+0000 7fa58b411640 1 -- 172.21.15.29:0/3741357045 >> 172.21.15.29:0/3741357045 conn(0x7fa58409d280 msgr2=0x7fa5840d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:14.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.547+0000 7fa58b411640 1 -- 172.21.15.29:0/3741357045 shutdown_connections 2024-04-25T00:37:14.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.547+0000 7fa58b411640 1 -- 172.21.15.29:0/3741357045 wait complete. 2024-04-25T00:37:14.571 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:37:14.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d2c16640 1 Processor -- start 2024-04-25T00:37:14.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d2c16640 1 -- start start 2024-04-25T00:37:14.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d2c16640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58cc0b4590 0x7f58cc070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:14.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d2c16640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f58cc0cd1c0 0x7f58cc072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:14.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d2c16640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58cc0712c0 con 0x7f58cc15e9a0 2024-04-25T00:37:14.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d2c16640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f58cc071ed0 con 0x7f58cc0cd1c0 2024-04-25T00:37:14.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d2c16640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f58cc0715a0 con 0x7f58cc0b4590 2024-04-25T00:37:14.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d09b2640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f58cc15e9a0 0x7f58cc15ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41498/0 (socket says 172.21.15.29:41498) 2024-04-25T00:37:14.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d09b2640 1 -- 172.21.15.29:0/749460651 learned_addr learned my addr 172.21.15.29:0/749460651 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:14.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d11b3640 1 --2- 172.21.15.29:0/749460651 >> v2:172.21.15.29:3300/0 conn(0x7f58cc0cd1c0 0x7f58cc072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:14.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58cbfff640 1 --2- 172.21.15.29:0/749460651 >> v2:172.21.15.157:3300/0 conn(0x7f58cc0b4590 0x7f58cc070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:14.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d11b3640 1 -- 172.21.15.29:0/749460651 >> v2:172.21.15.157:3300/0 conn(0x7f58cc0b4590 msgr2=0x7f58cc070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:14.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d11b3640 1 --2- 172.21.15.29:0/749460651 >> v2:172.21.15.157:3300/0 conn(0x7f58cc0b4590 0x7f58cc070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:14.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d11b3640 1 -- 172.21.15.29:0/749460651 >> v1:172.21.15.29:6789/0 conn(0x7f58cc15e9a0 legacy=0x7f58cc15ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:14.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58d11b3640 1 -- 172.21.15.29:0/749460651 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc071880 con 0x7f58cc0cd1c0 2024-04-25T00:37:14.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.691+0000 7f58cbfff640 1 --2- 172.21.15.29:0/749460651 >> v2:172.21.15.157:3300/0 conn(0x7f58cc0b4590 0x7f58cc070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:14.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.695+0000 7f58d11b3640 1 --2- 172.21.15.29:0/749460651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0cd1c0 0x7f58cc072360 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f58cc067fa0 tx=0x7f58c401c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3df4dd1c361ba66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:14.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.695+0000 7f58cb7fe640 1 -- 172.21.15.29:0/749460651 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c40b2070 con 0x7f58cc0cd1c0 2024-04-25T00:37:14.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.695+0000 7f58cb7fe640 1 -- 172.21.15.29:0/749460651 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c40a5760 con 0x7f58cc0cd1c0 2024-04-25T00:37:14.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.695+0000 7f58cb7fe640 1 -- 172.21.15.29:0/749460651 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c40a5b30 con 0x7f58cc0cd1c0 2024-04-25T00:37:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.695+0000 7f58d2c16640 1 -- 172.21.15.29:0/749460651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0cd1c0 msgr2=0x7f58cc072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.695+0000 7f58d2c16640 1 --2- 172.21.15.29:0/749460651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0cd1c0 0x7f58cc072360 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f58cc067fa0 tx=0x7f58c401c700 comp rx=0 tx=0).stop 2024-04-25T00:37:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.695+0000 7f58d2c16640 1 -- 172.21.15.29:0/749460651 shutdown_connections 2024-04-25T00:37:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.695+0000 7f58d2c16640 1 --2- 172.21.15.29:0/749460651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0cd1c0 0x7f58cc072360 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.695+0000 7f58d2c16640 1 --2- 172.21.15.29:0/749460651 >> v2:172.21.15.157:3300/0 conn(0x7f58cc0b4590 0x7f58cc070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.695+0000 7f58d2c16640 1 -- 172.21.15.29:0/749460651 >> 172.21.15.29:0/749460651 conn(0x7f58cc0b1870 msgr2=0x7f58cc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.695+0000 7f58d2c16640 1 -- 172.21.15.29:0/749460651 shutdown_connections 2024-04-25T00:37:14.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.695+0000 7f58d2c16640 1 -- 172.21.15.29:0/749460651 wait complete. 2024-04-25T00:37:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.695+0000 7f58d2c16640 1 Processor -- start 2024-04-25T00:37:14.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d2c16640 1 -- start start 2024-04-25T00:37:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d2c16640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0b4590 0x7f58cc0b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d2c16640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58cc0cd1c0 0x7f58cc136010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d2c16640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58cc15e9a0 0x7f58cc139a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d2c16640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc1449f0 con 0x7f58cc0b4590 2024-04-25T00:37:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d2c16640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc144870 con 0x7f58cc15e9a0 2024-04-25T00:37:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d2c16640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc167d40 con 0x7f58cc0cd1c0 2024-04-25T00:37:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d09b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0b4590 0x7f58cc0b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d09b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0b4590 0x7f58cc0b1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59160/0 (socket says 172.21.15.29:59160) 2024-04-25T00:37:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d09b2640 1 -- 172.21.15.29:0/2443240302 learned_addr learned my addr 172.21.15.29:0/2443240302 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d09b2640 1 -- 172.21.15.29:0/2443240302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58cc0cd1c0 msgr2=0x7f58cc136010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d09b2640 1 --2- 172.21.15.29:0/2443240302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58cc0cd1c0 0x7f58cc136010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d09b2640 1 -- 172.21.15.29:0/2443240302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58cc15e9a0 msgr2=0x7f58cc139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d09b2640 1 --2- 172.21.15.29:0/2443240302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58cc15e9a0 0x7f58cc139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d09b2640 1 -- 172.21.15.29:0/2443240302 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc0b15b0 con 0x7f58cc0b4590 2024-04-25T00:37:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d09b2640 1 --2- 172.21.15.29:0/2443240302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0b4590 0x7f58cc0b1070 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00c780 tx=0x7f58bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58c97fa640 1 -- 172.21.15.29:0/2443240302 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc018070 con 0x7f58cc0b4590 2024-04-25T00:37:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58c97fa640 1 -- 172.21.15.29:0/2443240302 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58bc0072d0 con 0x7f58cc0b4590 2024-04-25T00:37:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d2c16640 1 -- 172.21.15.29:0/2443240302 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc138080 con 0x7f58cc0b4590 2024-04-25T00:37:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58c97fa640 1 -- 172.21.15.29:0/2443240302 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc0047f0 con 0x7f58cc0b4590 2024-04-25T00:37:14.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d2c16640 1 -- 172.21.15.29:0/2443240302 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc138640 con 0x7f58cc0b4590 2024-04-25T00:37:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58d2c16640 1 -- 172.21.15.29:0/2443240302 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58cc0692e0 con 0x7f58cc0b4590 2024-04-25T00:37:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58c97fa640 1 -- 172.21.15.29:0/2443240302 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f58bc01b070 con 0x7f58cc0b4590 2024-04-25T00:37:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58c97fa640 1 --2- 172.21.15.29:0/2443240302 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f58b80679e0 0x7f58b8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58c97fa640 1 -- 172.21.15.29:0/2443240302 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5443+0+0 (secure 0 0 0) 0x7f58bc0850b0 con 0x7f58cc0b4590 2024-04-25T00:37:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.699+0000 7f58cbfff640 1 --2- 172.21.15.29:0/2443240302 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f58b80679e0 0x7f58b8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.703+0000 7f58cbfff640 1 --2- 172.21.15.29:0/2443240302 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f58b80679e0 0x7f58b8069ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0752b0 tx=0x7f58c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:14.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.707+0000 7f58c97fa640 1 -- 172.21.15.29:0/2443240302 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f58bc055d30 con 0x7f58cc0b4590 2024-04-25T00:37:14.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:14.871+0000 7f58d2c16640 1 -- 172.21.15.29:0/2443240302 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f58cc068710 con 0x7f58cc0b4590 2024-04-25T00:37:15.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.547+0000 7f58c97fa640 1 -- 172.21.15.29:0/2443240302 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f58bc055630 con 0x7f58cc0b4590 2024-04-25T00:37:15.551 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-04-25T00:37:15.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.555+0000 7f58d2c16640 1 -- 172.21.15.29:0/2443240302 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f58b80679e0 msgr2=0x7f58b8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.555+0000 7f58d2c16640 1 --2- 172.21.15.29:0/2443240302 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f58b80679e0 0x7f58b8069ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0752b0 tx=0x7f58c0008040 comp rx=0 tx=0).stop 2024-04-25T00:37:15.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.555+0000 7f58d2c16640 1 -- 172.21.15.29:0/2443240302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0b4590 msgr2=0x7f58cc0b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.555+0000 7f58d2c16640 1 --2- 172.21.15.29:0/2443240302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0b4590 0x7f58cc0b1070 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00c780 tx=0x7f58bc00cc40 comp rx=0 tx=0).stop 2024-04-25T00:37:15.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.555+0000 7f58d2c16640 1 -- 172.21.15.29:0/2443240302 shutdown_connections 2024-04-25T00:37:15.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.555+0000 7f58d2c16640 1 --2- 172.21.15.29:0/2443240302 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f58b80679e0 0x7f58b8069ea0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.555+0000 7f58d2c16640 1 --2- 172.21.15.29:0/2443240302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58cc15e9a0 0x7f58cc139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.555+0000 7f58d2c16640 1 --2- 172.21.15.29:0/2443240302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58cc0cd1c0 0x7f58cc136010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.555+0000 7f58d2c16640 1 --2- 172.21.15.29:0/2443240302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0b4590 0x7f58cc0b1070 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.555+0000 7f58d2c16640 1 -- 172.21.15.29:0/2443240302 >> 172.21.15.29:0/2443240302 conn(0x7f58cc0b1870 msgr2=0x7f58cc13eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:15.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.555+0000 7f58d2c16640 1 -- 172.21.15.29:0/2443240302 shutdown_connections 2024-04-25T00:37:15.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.555+0000 7f58d2c16640 1 -- 172.21.15.29:0/2443240302 wait complete. 2024-04-25T00:37:15.581 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:37:15.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbed3794640 1 Processor -- start 2024-04-25T00:37:15.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbed3794640 1 -- start start 2024-04-25T00:37:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbed3794640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc15e8d0 0x7fbecc0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbed3794640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbecc0b3720 con 0x7fbecc0b4780 2024-04-25T00:37:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbed3794640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbecc0b39c0 con 0x7fbecc15f3c0 2024-04-25T00:37:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbed3794640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbecc0b42f0 con 0x7fbecc15e8d0 2024-04-25T00:37:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbed0d2f640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbecc0b4780 0x7fbecc15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41506/0 (socket says 172.21.15.29:41506) 2024-04-25T00:37:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbed0d2f640 1 -- 172.21.15.29:0/4019132219 learned_addr learned my addr 172.21.15.29:0/4019132219 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:15.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbed1530640 1 --2- 172.21.15.29:0/4019132219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc15e8d0 0x7fbecc0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3453683510 0 0) 0x7fbecc0b3720 con 0x7fbecc0b4780 2024-04-25T00:37:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbeb8003530 con 0x7fbecc0b4780 2024-04-25T00:37:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4153657276 0 0) 0x7fbeb8003530 con 0x7fbecc0b4780 2024-04-25T00:37:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc15e8d0 msgr2=0x7fbecc0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 --2- 172.21.15.29:0/4019132219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc15e8d0 0x7fbecc0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 >> v1:172.21.15.157:6789/0 conn(0x7fbecc15f3c0 legacy=0x7fbecc15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc0b3ca0 con 0x7fbecc0b4780 2024-04-25T00:37:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fbebc001680 con 0x7fbecc0b4780 2024-04-25T00:37:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 >> v1:172.21.15.29:6789/0 conn(0x7fbecc0b4780 legacy=0x7fbecc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 --2- 172.21.15.29:0/4019132219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb80039f0 0x7fbeb8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 --2- 172.21.15.29:0/4019132219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc15e8d0 0x7fbeb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 --2- 172.21.15.29:0/4019132219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeb8004d50 0x7fbeb80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc0b42f0 con 0x7fbeb80039f0 2024-04-25T00:37:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb800ba80 con 0x7fbeb8004d50 2024-04-25T00:37:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb800bc80 con 0x7fbecc15e8d0 2024-04-25T00:37:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbed1530640 1 --2- 172.21.15.29:0/4019132219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb80039f0 0x7fbeb8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.707+0000 7fbed0d2f640 1 --2- 172.21.15.29:0/4019132219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc15e8d0 0x7fbeb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed1530640 1 -- 172.21.15.29:0/4019132219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc15e8d0 msgr2=0x7fbeb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed1d31640 1 --2- 172.21.15.29:0/4019132219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeb8004d50 0x7fbeb80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed1530640 1 --2- 172.21.15.29:0/4019132219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc15e8d0 0x7fbeb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed1530640 1 -- 172.21.15.29:0/4019132219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeb8004d50 msgr2=0x7fbeb80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed1530640 1 --2- 172.21.15.29:0/4019132219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeb8004d50 0x7fbeb80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed1530640 1 -- 172.21.15.29:0/4019132219 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb8003530 con 0x7fbeb80039f0 2024-04-25T00:37:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed1530640 1 --2- 172.21.15.29:0/4019132219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb80039f0 0x7fbeb8003e10 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009ab0 tx=0x7fbec801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ac18014c081920 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:15.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec80096b0 con 0x7fbeb80039f0 2024-04-25T00:37:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbec80a6510 con 0x7fbeb80039f0 2024-04-25T00:37:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbec3fff640 1 -- 172.21.15.29:0/4019132219 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec80a6900 con 0x7fbeb80039f0 2024-04-25T00:37:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed3794640 1 -- 172.21.15.29:0/4019132219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb80039f0 msgr2=0x7fbeb8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed3794640 1 --2- 172.21.15.29:0/4019132219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb80039f0 0x7fbeb8003e10 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009ab0 tx=0x7fbec801ca20 comp rx=0 tx=0).stop 2024-04-25T00:37:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed1d31640 1 -- 172.21.15.29:0/4019132219 reap_dead start 2024-04-25T00:37:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed3794640 1 -- 172.21.15.29:0/4019132219 shutdown_connections 2024-04-25T00:37:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed3794640 1 -- 172.21.15.29:0/4019132219 >> 172.21.15.29:0/4019132219 conn(0x7fbecc0b1870 msgr2=0x7fbecc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed3794640 1 -- 172.21.15.29:0/4019132219 shutdown_connections 2024-04-25T00:37:15.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.711+0000 7fbed3794640 1 -- 172.21.15.29:0/4019132219 wait complete. 2024-04-25T00:37:15.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed3794640 1 Processor -- start 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed3794640 1 -- start start 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed3794640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeb80039f0 0x7fbecc143e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed3794640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbecc15ac40 0x7fbecc13b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed3794640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc15a330 0x7fbecc13f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed3794640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc166df0 con 0x7fbecc15ac40 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed3794640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc166c70 con 0x7fbeb80039f0 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed3794640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbecc166f70 con 0x7fbecc15a330 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed0d2f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbecc15ac40 0x7fbecc13b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed0d2f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbecc15ac40 0x7fbecc13b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59178/0 (socket says 172.21.15.29:59178) 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed0d2f640 1 -- 172.21.15.29:0/509149626 learned_addr learned my addr 172.21.15.29:0/509149626 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed1d31640 1 --2- 172.21.15.29:0/509149626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc15a330 0x7fbecc13f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed0d2f640 1 -- 172.21.15.29:0/509149626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc15a330 msgr2=0x7fbecc13f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed0d2f640 1 --2- 172.21.15.29:0/509149626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc15a330 0x7fbecc13f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed0d2f640 1 -- 172.21.15.29:0/509149626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeb80039f0 msgr2=0x7fbecc143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed0d2f640 1 --2- 172.21.15.29:0/509149626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeb80039f0 0x7fbecc143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed0d2f640 1 -- 172.21.15.29:0/509149626 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc13bbd0 con 0x7fbecc15ac40 2024-04-25T00:37:15.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed0d2f640 1 --2- 172.21.15.29:0/509149626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbecc15ac40 0x7fbecc13b1c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00e7b0 tx=0x7fbebc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:15.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbec1ffb640 1 -- 172.21.15.29:0/509149626 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00c8d0 con 0x7fbecc15ac40 2024-04-25T00:37:15.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbec1ffb640 1 -- 172.21.15.29:0/509149626 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbebc018070 con 0x7fbecc15ac40 2024-04-25T00:37:15.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed3794640 1 -- 172.21.15.29:0/509149626 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbecc13beb0 con 0x7fbecc15ac40 2024-04-25T00:37:15.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbec1ffb640 1 -- 172.21.15.29:0/509149626 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00ccd0 con 0x7fbecc15ac40 2024-04-25T00:37:15.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed3794640 1 -- 172.21.15.29:0/509149626 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbecc1f8520 con 0x7fbecc15ac40 2024-04-25T00:37:15.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed3794640 1 -- 172.21.15.29:0/509149626 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbecc069270 con 0x7fbecc15ac40 2024-04-25T00:37:15.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbec1ffb640 1 -- 172.21.15.29:0/509149626 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fbebc01d460 con 0x7fbecc15ac40 2024-04-25T00:37:15.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbec1ffb640 1 --2- 172.21.15.29:0/509149626 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbeb00679e0 0x7fbeb0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbec1ffb640 1 -- 172.21.15.29:0/509149626 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fbebc0859c0 con 0x7fbecc15ac40 2024-04-25T00:37:15.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.715+0000 7fbed1530640 1 --2- 172.21.15.29:0/509149626 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbeb00679e0 0x7fbeb0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.719+0000 7fbed1530640 1 --2- 172.21.15.29:0/509149626 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbeb00679e0 0x7fbeb0069ea0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbec80af000 tx=0x7fbec8004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:15.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.723+0000 7fbec1ffb640 1 -- 172.21.15.29:0/509149626 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbebc056700 con 0x7fbecc15ac40 2024-04-25T00:37:15.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.875+0000 7fbed3794640 1 -- 172.21.15.29:0/509149626 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fbecc06e7b0 con 0x7fbecc15ac40 2024-04-25T00:37:15.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.879+0000 7fbec1ffb640 1 -- 172.21.15.29:0/509149626 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fbebc056000 con 0x7fbecc15ac40 2024-04-25T00:37:15.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.883+0000 7fbed3794640 1 -- 172.21.15.29:0/509149626 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbeb00679e0 msgr2=0x7fbeb0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.883+0000 7fbed3794640 1 --2- 172.21.15.29:0/509149626 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbeb00679e0 0x7fbeb0069ea0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbec80af000 tx=0x7fbec8004740 comp rx=0 tx=0).stop 2024-04-25T00:37:15.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.883+0000 7fbed3794640 1 -- 172.21.15.29:0/509149626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbecc15ac40 msgr2=0x7fbecc13b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.883+0000 7fbed3794640 1 --2- 172.21.15.29:0/509149626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbecc15ac40 0x7fbecc13b1c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00e7b0 tx=0x7fbebc00ec70 comp rx=0 tx=0).stop 2024-04-25T00:37:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.883+0000 7fbed3794640 1 -- 172.21.15.29:0/509149626 shutdown_connections 2024-04-25T00:37:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.883+0000 7fbed3794640 1 --2- 172.21.15.29:0/509149626 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbeb00679e0 0x7fbeb0069ea0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.883+0000 7fbed3794640 1 --2- 172.21.15.29:0/509149626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc15a330 0x7fbecc13f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.883+0000 7fbed3794640 1 --2- 172.21.15.29:0/509149626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbecc15ac40 0x7fbecc13b1c0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.883+0000 7fbed3794640 1 --2- 172.21.15.29:0/509149626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeb80039f0 0x7fbecc143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.883+0000 7fbed3794640 1 -- 172.21.15.29:0/509149626 >> 172.21.15.29:0/509149626 conn(0x7fbecc0b1870 msgr2=0x7fbecc15ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.883+0000 7fbed3794640 1 -- 172.21.15.29:0/509149626 shutdown_connections 2024-04-25T00:37:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:15.883+0000 7fbed3794640 1 -- 172.21.15.29:0/509149626 wait complete. 2024-04-25T00:37:15.904 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-04-25T00:37:16.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f684c9b4640 1 Processor -- start 2024-04-25T00:37:16.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f684c9b4640 1 -- start start 2024-04-25T00:37:16.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f684c9b4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f68480b4d30 0x7f68480b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f684c9b4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f684806d210 0x7f68480ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f684c9b4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f684806d6c0 con 0x7f684815df70 2024-04-25T00:37:16.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f684c9b4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f684806d980 con 0x7f68480b4d30 2024-04-25T00:37:16.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f684c9b4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f684806dc60 con 0x7f684806d210 2024-04-25T00:37:16.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f6845d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f68480b4d30 0x7f68480b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:16.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f6845d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f68480b4d30 0x7f68480b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59188/0 (socket says 172.21.15.29:59188) 2024-04-25T00:37:16.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f6845d9b640 1 -- 172.21.15.29:0/3416218420 learned_addr learned my addr 172.21.15.29:0/3416218420 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:16.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f6846d9d640 1 --2- 172.21.15.29:0/3416218420 >> v2:172.21.15.157:3300/0 conn(0x7f684806d210 0x7f68480ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:16.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f6845d9b640 1 -- 172.21.15.29:0/3416218420 >> v2:172.21.15.157:3300/0 conn(0x7f684806d210 msgr2=0x7f68480ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:16.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f6845d9b640 1 --2- 172.21.15.29:0/3416218420 >> v2:172.21.15.157:3300/0 conn(0x7f684806d210 0x7f68480ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f6845d9b640 1 -- 172.21.15.29:0/3416218420 >> v1:172.21.15.157:6789/0 conn(0x7f684815df70 legacy=0x7f68480b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:16.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.015+0000 7f6845d9b640 1 -- 172.21.15.29:0/3416218420 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684806df40 con 0x7f68480b4d30 2024-04-25T00:37:16.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.019+0000 7f6845d9b640 1 --2- 172.21.15.29:0/3416218420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68480b4d30 0x7f68480b3e40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6848071d10 tx=0x7f683801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c155022a62b174f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:16.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.019+0000 7f684559a640 1 -- 172.21.15.29:0/3416218420 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68380a5880 con 0x7f68480b4d30 2024-04-25T00:37:16.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.019+0000 7f684559a640 1 -- 172.21.15.29:0/3416218420 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68380a5a20 con 0x7f68480b4d30 2024-04-25T00:37:16.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.019+0000 7f684559a640 1 -- 172.21.15.29:0/3416218420 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68380a5df0 con 0x7f68480b4d30 2024-04-25T00:37:16.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.019+0000 7f684c9b4640 1 -- 172.21.15.29:0/3416218420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68480b4d30 msgr2=0x7f68480b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:16.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.019+0000 7f684c9b4640 1 --2- 172.21.15.29:0/3416218420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68480b4d30 0x7f68480b3e40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6848071d10 tx=0x7f683801c9c0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.019+0000 7f684c9b4640 1 -- 172.21.15.29:0/3416218420 shutdown_connections 2024-04-25T00:37:16.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.019+0000 7f684c9b4640 1 --2- 172.21.15.29:0/3416218420 >> v2:172.21.15.157:3300/0 conn(0x7f684806d210 0x7f68480ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.019+0000 7f684c9b4640 1 --2- 172.21.15.29:0/3416218420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68480b4d30 0x7f68480b3e40 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.019+0000 7f684c9b4640 1 -- 172.21.15.29:0/3416218420 >> 172.21.15.29:0/3416218420 conn(0x7f68480b1870 msgr2=0x7f68480b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:16.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.019+0000 7f684c9b4640 1 -- 172.21.15.29:0/3416218420 shutdown_connections 2024-04-25T00:37:16.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.019+0000 7f684c9b4640 1 -- 172.21.15.29:0/3416218420 wait complete. 2024-04-25T00:37:16.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.019+0000 7f684c9b4640 1 Processor -- start 2024-04-25T00:37:16.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f684c9b4640 1 -- start start 2024-04-25T00:37:16.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f684c9b4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f684806d210 0x7f684814d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f684c9b4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68480b4d30 0x7f6848156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f684c9b4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f684815df70 0x7f684814c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f684c9b4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f684806eba0 con 0x7f68480b4d30 2024-04-25T00:37:16.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f684c9b4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f684806ea20 con 0x7f684815df70 2024-04-25T00:37:16.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f684c9b4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f68480ae5f0 con 0x7f684806d210 2024-04-25T00:37:16.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f6845d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68480b4d30 0x7f6848156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:16.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f6845d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68480b4d30 0x7f6848156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59204/0 (socket says 172.21.15.29:59204) 2024-04-25T00:37:16.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f6845d9b640 1 -- 172.21.15.29:0/2208594738 learned_addr learned my addr 172.21.15.29:0/2208594738 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f684659c640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f684806d210 0x7f684814d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f6846d9d640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f684815df70 0x7f684814c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f6845d9b640 1 -- 172.21.15.29:0/2208594738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f684806d210 msgr2=0x7f684814d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f6845d9b640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f684806d210 0x7f684814d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f6845d9b640 1 -- 172.21.15.29:0/2208594738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f684815df70 msgr2=0x7f684814c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f6845d9b640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f684815df70 0x7f684814c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f6845d9b640 1 -- 172.21.15.29:0/2208594738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684815af50 con 0x7f68480b4d30 2024-04-25T00:37:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f684659c640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f684806d210 0x7f684814d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f6845d9b640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68480b4d30 0x7f6848156620 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f683801a5f0 tx=0x7f6838019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f683effd640 1 -- 172.21.15.29:0/2208594738 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68380b8470 con 0x7f68480b4d30 2024-04-25T00:37:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f683effd640 1 -- 172.21.15.29:0/2208594738 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68380b8670 con 0x7f68480b4d30 2024-04-25T00:37:16.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f683effd640 1 -- 172.21.15.29:0/2208594738 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68380b8a60 con 0x7f68480b4d30 2024-04-25T00:37:16.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f684c9b4640 1 -- 172.21.15.29:0/2208594738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684814f490 con 0x7f68480b4d30 2024-04-25T00:37:16.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f684c9b4640 1 -- 172.21.15.29:0/2208594738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f684814dc20 con 0x7f68480b4d30 2024-04-25T00:37:16.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f684c9b4640 1 -- 172.21.15.29:0/2208594738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6848069270 con 0x7f68480b4d30 2024-04-25T00:37:16.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f683effd640 1 -- 172.21.15.29:0/2208594738 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f68380c0030 con 0x7f68480b4d30 2024-04-25T00:37:16.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f683effd640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6828067a10 0x7f6828069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f683effd640 1 -- 172.21.15.29:0/2208594738 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f68381215d0 con 0x7f68480b4d30 2024-04-25T00:37:16.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.023+0000 7f684659c640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6828067a10 0x7f6828069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:16.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.027+0000 7f684659c640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6828067a10 0x7f6828069ed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6830001650 tx=0x7f6830009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:16.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.027+0000 7f683effd640 1 -- 172.21.15.29:0/2208594738 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f68380c7020 con 0x7f68480b4d30 2024-04-25T00:37:16.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.259+0000 7f684c9b4640 1 -- 172.21.15.29:0/2208594738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7f68481561a0 con 0x7f68480b4d30 2024-04-25T00:37:16.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.551+0000 7f683effd640 1 -- 172.21.15.29:0/2208594738 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v311) ==== 146+0+0 (secure 0 0 0) 0x7f68380165e0 con 0x7f68480b4d30 2024-04-25T00:37:16.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.551+0000 7f684c9b4640 1 -- 172.21.15.29:0/2208594738 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6828067a10 msgr2=0x7f6828069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:16.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.551+0000 7f684c9b4640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6828067a10 0x7f6828069ed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6830001650 tx=0x7f6830009040 comp rx=0 tx=0).stop 2024-04-25T00:37:16.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.551+0000 7f684c9b4640 1 -- 172.21.15.29:0/2208594738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68480b4d30 msgr2=0x7f6848156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:16.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.551+0000 7f684c9b4640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68480b4d30 0x7f6848156620 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f683801a5f0 tx=0x7f6838019760 comp rx=0 tx=0).stop 2024-04-25T00:37:16.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.555+0000 7f684c9b4640 1 -- 172.21.15.29:0/2208594738 shutdown_connections 2024-04-25T00:37:16.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.555+0000 7f684c9b4640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6828067a10 0x7f6828069ed0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.555+0000 7f684c9b4640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f684815df70 0x7f684814c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.555+0000 7f684c9b4640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68480b4d30 0x7f6848156620 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.555+0000 7f684c9b4640 1 --2- 172.21.15.29:0/2208594738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f684806d210 0x7f684814d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.555+0000 7f684c9b4640 1 -- 172.21.15.29:0/2208594738 >> 172.21.15.29:0/2208594738 conn(0x7f68480b1870 msgr2=0x7f68480cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:16.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.555+0000 7f684c9b4640 1 -- 172.21.15.29:0/2208594738 shutdown_connections 2024-04-25T00:37:16.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.555+0000 7f684c9b4640 1 -- 172.21.15.29:0/2208594738 wait complete. 2024-04-25T00:37:16.576 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2024-04-25T00:37:16.577 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-04-25T00:37:16.577 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2024-04-25T00:37:16.577 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:37:16.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.695+0000 7f2876cdc640 1 Processor -- start 2024-04-25T00:37:16.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.695+0000 7f2876cdc640 1 -- start start 2024-04-25T00:37:16.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2876cdc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f28700b3280 0x7f287016d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2876cdc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2870071690 con 0x7f2870162420 2024-04-25T00:37:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2876cdc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2870071910 con 0x7f28700b4e00 2024-04-25T00:37:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2876cdc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2870071bf0 con 0x7f28700b3280 2024-04-25T00:37:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2874a78640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2870162420 0x7f2870070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41518/0 (socket says 172.21.15.29:41518) 2024-04-25T00:37:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2874a78640 1 -- 172.21.15.29:0/2225253793 learned_addr learned my addr 172.21.15.29:0/2225253793 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2875279640 1 --2- 172.21.15.29:0/2225253793 >> v2:172.21.15.157:3300/0 conn(0x7f28700b3280 0x7f287016d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2365034026 0 0) 0x7f2860001120 con 0x7f2870162420 2024-04-25T00:37:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2858003530 con 0x7f2870162420 2024-04-25T00:37:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3999700575 0 0) 0x7f2858003530 con 0x7f2870162420 2024-04-25T00:37:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 >> v2:172.21.15.157:3300/0 conn(0x7f28700b3280 msgr2=0x7f287016d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 --2- 172.21.15.29:0/2225253793 >> v2:172.21.15.157:3300/0 conn(0x7f28700b3280 0x7f287016d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 >> v1:172.21.15.157:6789/0 conn(0x7f28700b4e00 legacy=0x7f28700b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2870071ed0 con 0x7f2870162420 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2860001850 con 0x7f2870162420 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 >> v1:172.21.15.29:6789/0 conn(0x7f2870162420 legacy=0x7f2870070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 --2- 172.21.15.29:0/2225253793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2858003a10 0x7f2858003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 --2- 172.21.15.29:0/2225253793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2858004810 0x7f2858004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 --2- 172.21.15.29:0/2225253793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28580051d0 0x7f285800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2870071bf0 con 0x7f2858003a10 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f285800c0d0 con 0x7f28580051d0 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f285800c2d0 con 0x7f2858004810 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2874a78640 1 --2- 172.21.15.29:0/2225253793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2858003a10 0x7f2858003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2874a78640 1 -- 172.21.15.29:0/2225253793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2858004810 msgr2=0x7f2858004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2874a78640 1 --2- 172.21.15.29:0/2225253793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2858004810 0x7f2858004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2874a78640 1 -- 172.21.15.29:0/2225253793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28580051d0 msgr2=0x7f285800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2874a78640 1 --2- 172.21.15.29:0/2225253793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28580051d0 0x7f285800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2874a78640 1 -- 172.21.15.29:0/2225253793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2858003530 con 0x7f2858003a10 2024-04-25T00:37:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.699+0000 7f2875279640 1 -- 172.21.15.29:0/2225253793 reap_dead start 2024-04-25T00:37:16.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.703+0000 7f2874a78640 1 --2- 172.21.15.29:0/2225253793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2858003a10 0x7f2858003e30 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f286000c320 tx=0x7f286001cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7ea1616733006c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:16.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.703+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860007e20 con 0x7f2858003a10 2024-04-25T00:37:16.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.703+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28600a9070 con 0x7f2858003a10 2024-04-25T00:37:16.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.703+0000 7f286f7fe640 1 -- 172.21.15.29:0/2225253793 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860005d80 con 0x7f2858003a10 2024-04-25T00:37:16.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.703+0000 7f2876cdc640 1 -- 172.21.15.29:0/2225253793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2858003a10 msgr2=0x7f2858003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:16.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.703+0000 7f2876cdc640 1 --2- 172.21.15.29:0/2225253793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2858003a10 0x7f2858003e30 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f286000c320 tx=0x7f286001cba0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.703+0000 7f2876cdc640 1 -- 172.21.15.29:0/2225253793 shutdown_connections 2024-04-25T00:37:16.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.703+0000 7f2876cdc640 1 --2- 172.21.15.29:0/2225253793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2858003a10 0x7f2858003e30 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.703+0000 7f2876cdc640 1 -- 172.21.15.29:0/2225253793 >> 172.21.15.29:0/2225253793 conn(0x7f28700b1870 msgr2=0x7f28700b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:16.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.703+0000 7f2876cdc640 1 -- 172.21.15.29:0/2225253793 shutdown_connections 2024-04-25T00:37:16.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.703+0000 7f2876cdc640 1 -- 172.21.15.29:0/2225253793 wait complete. 2024-04-25T00:37:16.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.703+0000 7f2876cdc640 1 Processor -- start 2024-04-25T00:37:16.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.703+0000 7f2876cdc640 1 -- start start 2024-04-25T00:37:16.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2876cdc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2858003a10 0x7f2870154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2876cdc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2870152ee0 0x7f28700cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2876cdc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2870156500 0x7f2870158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2876cdc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2870071510 con 0x7f2870156500 2024-04-25T00:37:16.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2876cdc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2870071390 con 0x7f2858003a10 2024-04-25T00:37:16.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2876cdc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2870166830 con 0x7f2870152ee0 2024-04-25T00:37:16.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2875279640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2870156500 0x7f2870158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:16.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2875279640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2870156500 0x7f2870158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59222/0 (socket says 172.21.15.29:59222) 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2875279640 1 -- 172.21.15.29:0/4061761194 learned_addr learned my addr 172.21.15.29:0/4061761194 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2874a78640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2858003a10 0x7f2870154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f286ffff640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2870152ee0 0x7f28700cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2875279640 1 -- 172.21.15.29:0/4061761194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2870152ee0 msgr2=0x7f28700cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2875279640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2870152ee0 0x7f28700cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2875279640 1 -- 172.21.15.29:0/4061761194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2858003a10 msgr2=0x7f2870154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2875279640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2858003a10 0x7f2870154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2875279640 1 -- 172.21.15.29:0/4061761194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2870153360 con 0x7f2870156500 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f286ffff640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2870152ee0 0x7f28700cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2875279640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2870156500 0x7f2870158880 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f28700b4360 tx=0x7f286800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f286d7fa640 1 -- 172.21.15.29:0/4061761194 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286800bb10 con 0x7f2870156500 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f286d7fa640 1 -- 172.21.15.29:0/4061761194 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f286801a070 con 0x7f2870156500 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2876cdc640 1 -- 172.21.15.29:0/4061761194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2870155460 con 0x7f2870156500 2024-04-25T00:37:16.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f286d7fa640 1 -- 172.21.15.29:0/4061761194 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868014400 con 0x7f2870156500 2024-04-25T00:37:16.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2876cdc640 1 -- 172.21.15.29:0/4061761194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2870155a20 con 0x7f2870156500 2024-04-25T00:37:16.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2876cdc640 1 -- 172.21.15.29:0/4061761194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2870069270 con 0x7f2870156500 2024-04-25T00:37:16.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f286d7fa640 1 -- 172.21.15.29:0/4061761194 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f2868013070 con 0x7f2870156500 2024-04-25T00:37:16.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f286d7fa640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f285c0679e0 0x7f285c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:16.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2874a78640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f285c0679e0 0x7f285c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:16.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f286d7fa640 1 -- 172.21.15.29:0/4061761194 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f2868085d10 con 0x7f2870156500 2024-04-25T00:37:16.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.707+0000 7f2874a78640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f285c0679e0 0x7f285c069ea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f28600ab000 tx=0x7f28600a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:16.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.711+0000 7f286d7fa640 1 -- 172.21.15.29:0/4061761194 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f286808b020 con 0x7f2870156500 2024-04-25T00:37:16.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.875+0000 7f2876cdc640 1 -- 172.21.15.29:0/4061761194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2870072fa0 con 0x7f2870156500 2024-04-25T00:37:16.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.875+0000 7f286d7fa640 1 -- 172.21.15.29:0/4061761194 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+13103 (secure 0 0 0) 0x7f2870153360 con 0x7f2870156500 2024-04-25T00:37:16.880 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:16.880 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":94,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:37:15.544885+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6835/3939243143":"2024-04-26T00:37:12.198831+0000","172.21.15.29:6834/3939243143":"2024-04-26T00:37:12.198831+0000","172.21.15.29:6833/1764147340":"2024-04-26T00:37:01.479125+0000","172.21.15.29:6832/1764147340":"2024-04-26T00:37:01.479125+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:37:16.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.879+0000 7f2876cdc640 1 -- 172.21.15.29:0/4061761194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f285c0679e0 msgr2=0x7f285c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:16.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.879+0000 7f2876cdc640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f285c0679e0 0x7f285c069ea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f28600ab000 tx=0x7f28600a7070 comp rx=0 tx=0).stop 2024-04-25T00:37:16.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.879+0000 7f2876cdc640 1 -- 172.21.15.29:0/4061761194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2870156500 msgr2=0x7f2870158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:16.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.879+0000 7f2876cdc640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2870156500 0x7f2870158880 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f28700b4360 tx=0x7f286800eee0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.879+0000 7f2876cdc640 1 -- 172.21.15.29:0/4061761194 shutdown_connections 2024-04-25T00:37:16.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.879+0000 7f2876cdc640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f285c0679e0 0x7f285c069ea0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.879+0000 7f2876cdc640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2870156500 0x7f2870158880 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.879+0000 7f2876cdc640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2870152ee0 0x7f28700cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.879+0000 7f2876cdc640 1 --2- 172.21.15.29:0/4061761194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2858003a10 0x7f2870154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:16.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.879+0000 7f2876cdc640 1 -- 172.21.15.29:0/4061761194 >> 172.21.15.29:0/4061761194 conn(0x7f28700b1870 msgr2=0x7f28700b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:16.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.879+0000 7f2876cdc640 1 -- 172.21.15.29:0/4061761194 shutdown_connections 2024-04-25T00:37:16.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:16.879+0000 7f2876cdc640 1 -- 172.21.15.29:0/4061761194 wait complete. 2024-04-25T00:37:16.899 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:37:16.901 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:37:17.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.027+0000 7f1a7caac640 1 Processor -- start 2024-04-25T00:37:17.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.027+0000 7f1a7caac640 1 -- start start 2024-04-25T00:37:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.027+0000 7f1a7caac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7815f3c0 0x7f1a780cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.027+0000 7f1a7caac640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a780b3720 con 0x7f1a7815e8d0 2024-04-25T00:37:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.027+0000 7f1a7caac640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a780b39c0 con 0x7f1a780b4780 2024-04-25T00:37:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.027+0000 7f1a7caac640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1a780b42f0 con 0x7f1a7815f3c0 2024-04-25T00:37:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.027+0000 7f1a7659c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1a7815e8d0 0x7f1a7815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41530/0 (socket says 172.21.15.29:41530) 2024-04-25T00:37:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.027+0000 7f1a7659c640 1 -- 172.21.15.29:0/2115058878 learned_addr learned my addr 172.21.15.29:0/2115058878 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.027+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4266320571 0 0) 0x7f1a780b3720 con 0x7f1a7815e8d0 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.027+0000 7f1a76d9d640 1 --2- 172.21.15.29:0/2115058878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7815f3c0 0x7f1a780cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.027+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a60003530 con 0x7f1a7815e8d0 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1861230155 0 0) 0x7f1a60003530 con 0x7f1a7815e8d0 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7815f3c0 msgr2=0x7f1a780cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 --2- 172.21.15.29:0/2115058878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7815f3c0 0x7f1a780cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 >> v1:172.21.15.157:6789/0 conn(0x7f1a780b4780 legacy=0x7f1a7815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a780b3ca0 con 0x7f1a7815e8d0 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f1a68001680 con 0x7f1a7815e8d0 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 >> v1:172.21.15.29:6789/0 conn(0x7f1a7815e8d0 legacy=0x7f1a7815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 --2- 172.21.15.29:0/2115058878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a600039f0 0x7f1a60003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 --2- 172.21.15.29:0/2115058878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a60004810 0x7f1a60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 --2- 172.21.15.29:0/2115058878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7815f3c0 0x7f1a60005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1a780b42f0 con 0x7f1a60004810 2024-04-25T00:37:17.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6000bb00 con 0x7f1a600039f0 2024-04-25T00:37:17.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6000bd00 con 0x7f1a7815f3c0 2024-04-25T00:37:17.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a75d9b640 1 --2- 172.21.15.29:0/2115058878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a60004810 0x7f1a60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7659c640 1 --2- 172.21.15.29:0/2115058878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a600039f0 0x7f1a60003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a75d9b640 1 -- 172.21.15.29:0/2115058878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7815f3c0 msgr2=0x7f1a60005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a75d9b640 1 --2- 172.21.15.29:0/2115058878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a7815f3c0 0x7f1a60005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a75d9b640 1 -- 172.21.15.29:0/2115058878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a600039f0 msgr2=0x7f1a60003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a75d9b640 1 --2- 172.21.15.29:0/2115058878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a600039f0 0x7f1a60003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a75d9b640 1 -- 172.21.15.29:0/2115058878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a60003530 con 0x7f1a60004810 2024-04-25T00:37:17.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a75d9b640 1 --2- 172.21.15.29:0/2115058878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a60004810 0x7f1a60004c90 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c001780 tx=0x7f1a6c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8f4881bd889d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:17.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c0013a0 con 0x7f1a60004810 2024-04-25T00:37:17.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a6c0052f0 con 0x7f1a60004810 2024-04-25T00:37:17.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7559a640 1 -- 172.21.15.29:0/2115058878 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c003840 con 0x7f1a60004810 2024-04-25T00:37:17.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7caac640 1 -- 172.21.15.29:0/2115058878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a60004810 msgr2=0x7f1a60004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7caac640 1 --2- 172.21.15.29:0/2115058878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a60004810 0x7f1a60004c90 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c001780 tx=0x7f1a6c01c420 comp rx=0 tx=0).stop 2024-04-25T00:37:17.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a76d9d640 1 -- 172.21.15.29:0/2115058878 reap_dead start 2024-04-25T00:37:17.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7caac640 1 -- 172.21.15.29:0/2115058878 shutdown_connections 2024-04-25T00:37:17.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7caac640 1 -- 172.21.15.29:0/2115058878 >> 172.21.15.29:0/2115058878 conn(0x7f1a780b1870 msgr2=0x7f1a780b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:17.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7caac640 1 -- 172.21.15.29:0/2115058878 shutdown_connections 2024-04-25T00:37:17.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.031+0000 7f1a7caac640 1 -- 172.21.15.29:0/2115058878 wait complete. 2024-04-25T00:37:17.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7caac640 1 Processor -- start 2024-04-25T00:37:17.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7caac640 1 -- start start 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7caac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a78159d30 0x7f1a78143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7caac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a78136220 0x7f1a78138e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7caac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a78150620 0x7f1a78150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7caac640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1a78144a20 con 0x7f1a78159d30 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7caac640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7806cf10 con 0x7f1a78136220 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7caac640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1a781389b0 con 0x7f1a78150620 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a78159d30 0x7f1a78143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a78159d30 0x7f1a78143330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59246/0 (socket says 172.21.15.29:59246) 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7659c640 1 -- 172.21.15.29:0/3042130182 learned_addr learned my addr 172.21.15.29:0/3042130182 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a75d9b640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a78136220 0x7f1a78138e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a76d9d640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a78150620 0x7f1a78150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7659c640 1 -- 172.21.15.29:0/3042130182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a78150620 msgr2=0x7f1a78150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7659c640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a78150620 0x7f1a78150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7659c640 1 -- 172.21.15.29:0/3042130182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a78136220 msgr2=0x7f1a78138e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7659c640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a78136220 0x7f1a78138e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7659c640 1 -- 172.21.15.29:0/3042130182 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a781383a0 con 0x7f1a78159d30 2024-04-25T00:37:17.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a76d9d640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a78150620 0x7f1a78150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:17.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7659c640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a78159d30 0x7f1a78143330 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f1a680118b0 tx=0x7f1a68011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:17.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a66ffd640 1 -- 172.21.15.29:0/3042130182 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68010040 con 0x7f1a78159d30 2024-04-25T00:37:17.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a66ffd640 1 -- 172.21.15.29:0/3042130182 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a6801a070 con 0x7f1a78159d30 2024-04-25T00:37:17.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7caac640 1 -- 172.21.15.29:0/3042130182 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a781366a0 con 0x7f1a78159d30 2024-04-25T00:37:17.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a66ffd640 1 -- 172.21.15.29:0/3042130182 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6800fda0 con 0x7f1a78159d30 2024-04-25T00:37:17.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7caac640 1 -- 172.21.15.29:0/3042130182 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a78136c30 con 0x7f1a78159d30 2024-04-25T00:37:17.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.035+0000 7f1a7caac640 1 -- 172.21.15.29:0/3042130182 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a78150fe0 con 0x7f1a78159d30 2024-04-25T00:37:17.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.039+0000 7f1a66ffd640 1 -- 172.21.15.29:0/3042130182 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f1a6801f460 con 0x7f1a78159d30 2024-04-25T00:37:17.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.039+0000 7f1a66ffd640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1a580679e0 0x7f1a58069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.039+0000 7f1a66ffd640 1 -- 172.21.15.29:0/3042130182 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f1a68088e40 con 0x7f1a78159d30 2024-04-25T00:37:17.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.039+0000 7f1a75d9b640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1a580679e0 0x7f1a58069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.039+0000 7f1a75d9b640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1a580679e0 0x7f1a58069ea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c001750 tx=0x7f1a6c0011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:17.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.043+0000 7f1a66ffd640 1 -- 172.21.15.29:0/3042130182 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1a68059b10 con 0x7f1a78159d30 2024-04-25T00:37:17.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.207+0000 7f1a7caac640 1 -- 172.21.15.29:0/3042130182 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a78136e70 con 0x7f1a78159d30 2024-04-25T00:37:17.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.207+0000 7f1a66ffd640 1 -- 172.21.15.29:0/3042130182 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f1a78136e70 con 0x7f1a78159d30 2024-04-25T00:37:17.211 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:37:17.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.211+0000 7f1a7caac640 1 -- 172.21.15.29:0/3042130182 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1a580679e0 msgr2=0x7f1a58069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.211+0000 7f1a7caac640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1a580679e0 0x7f1a58069ea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c001750 tx=0x7f1a6c0011a0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.211+0000 7f1a7caac640 1 -- 172.21.15.29:0/3042130182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a78159d30 msgr2=0x7f1a78143330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.211+0000 7f1a7caac640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a78159d30 0x7f1a78143330 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f1a680118b0 tx=0x7f1a68011d70 comp rx=0 tx=0).stop 2024-04-25T00:37:17.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.211+0000 7f1a7caac640 1 -- 172.21.15.29:0/3042130182 shutdown_connections 2024-04-25T00:37:17.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.211+0000 7f1a7caac640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1a580679e0 0x7f1a58069ea0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.211+0000 7f1a7caac640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a78150620 0x7f1a78150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.211+0000 7f1a7caac640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a78136220 0x7f1a78138e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.211+0000 7f1a7caac640 1 --2- 172.21.15.29:0/3042130182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a78159d30 0x7f1a78143330 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.211+0000 7f1a7caac640 1 -- 172.21.15.29:0/3042130182 >> 172.21.15.29:0/3042130182 conn(0x7f1a780b1870 msgr2=0x7f1a7815dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:17.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.211+0000 7f1a7caac640 1 -- 172.21.15.29:0/3042130182 shutdown_connections 2024-04-25T00:37:17.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.211+0000 7f1a7caac640 1 -- 172.21.15.29:0/3042130182 wait complete. 2024-04-25T00:37:17.231 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-04-25T00:37:17.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc35e8e640 1 Processor -- start 2024-04-25T00:37:17.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc35e8e640 1 -- start start 2024-04-25T00:37:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc35e8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc300cf690 0x7fdc300d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc35e8e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc300cfc10 con 0x7fdc300d2a90 2024-04-25T00:37:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc35e8e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc300cfef0 con 0x7fdc3015e200 2024-04-25T00:37:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc35e8e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc300d2ee0 con 0x7fdc300cf690 2024-04-25T00:37:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2ffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdc300d2a90 0x7fdc300b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41536/0 (socket says 172.21.15.29:41536) 2024-04-25T00:37:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2ffff640 1 -- 172.21.15.29:0/62429900 learned_addr learned my addr 172.21.15.29:0/62429900 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc27fff640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc300cf690 0x7fdc300d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2370711672 0 0) 0x7fdc300cfc10 con 0x7fdc300d2a90 2024-04-25T00:37:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc1c003530 con 0x7fdc300d2a90 2024-04-25T00:37:17.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2932316118 0 0) 0x7fdc1c003530 con 0x7fdc300d2a90 2024-04-25T00:37:17.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc300cf690 msgr2=0x7fdc300d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc300cf690 0x7fdc300d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 >> v1:172.21.15.157:6789/0 conn(0x7fdc3015e200 legacy=0x7fdc300d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc300b3b20 con 0x7fdc300d2a90 2024-04-25T00:37:17.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fdc28001a00 con 0x7fdc300d2a90 2024-04-25T00:37:17.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 >> v1:172.21.15.29:6789/0 conn(0x7fdc300d2a90 legacy=0x7fdc300b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c0039f0 0x7fdc1c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c004810 0x7fdc1c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc300cf690 0x7fdc1c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdc300d2ee0 con 0x7fdc1c0039f0 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c00ba80 con 0x7fdc1c004810 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1c00bc80 con 0x7fdc300cf690 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2f7fe640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c0039f0 0x7fdc1c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc27fff640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c004810 0x7fdc1c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2f7fe640 1 -- 172.21.15.29:0/62429900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc300cf690 msgr2=0x7fdc1c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2ffff640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc300cf690 0x7fdc1c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2f7fe640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc300cf690 0x7fdc1c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2f7fe640 1 -- 172.21.15.29:0/62429900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c004810 msgr2=0x7fdc1c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2f7fe640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c004810 0x7fdc1c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2f7fe640 1 -- 172.21.15.29:0/62429900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c003530 con 0x7fdc1c0039f0 2024-04-25T00:37:17.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc2ffff640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc300cf690 0x7fdc1c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:17.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.351+0000 7fdc27fff640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c004810 0x7fdc1c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:17.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc2f7fe640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c0039f0 0x7fdc1c003e10 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdc20001750 tx=0x7fdc2001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1fc9208f1e2132 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:17.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc200013a0 con 0x7fdc1c0039f0 2024-04-25T00:37:17.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc200052f0 con 0x7fdc1c0039f0 2024-04-25T00:37:17.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc2effd640 1 -- 172.21.15.29:0/62429900 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20003840 con 0x7fdc1c0039f0 2024-04-25T00:37:17.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 -- 172.21.15.29:0/62429900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c0039f0 msgr2=0x7fdc1c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 --2- 172.21.15.29:0/62429900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c0039f0 0x7fdc1c003e10 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdc20001750 tx=0x7fdc2001c3c0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc2ffff640 1 -- 172.21.15.29:0/62429900 reap_dead start 2024-04-25T00:37:17.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 -- 172.21.15.29:0/62429900 shutdown_connections 2024-04-25T00:37:17.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 -- 172.21.15.29:0/62429900 >> 172.21.15.29:0/62429900 conn(0x7fdc3009d280 msgr2=0x7fdc300d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:17.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 -- 172.21.15.29:0/62429900 shutdown_connections 2024-04-25T00:37:17.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 -- 172.21.15.29:0/62429900 wait complete. 2024-04-25T00:37:17.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 Processor -- start 2024-04-25T00:37:17.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 -- start start 2024-04-25T00:37:17.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c0039f0 0x7fdc30149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc30160c80 0x7fdc300b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc300b8d80 0x7fdc300b9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdc300b55d0 con 0x7fdc300b8d80 2024-04-25T00:37:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc300b5450 con 0x7fdc1c0039f0 2024-04-25T00:37:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc35e8e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc300b38d0 con 0x7fdc30160c80 2024-04-25T00:37:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc2ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc300b8d80 0x7fdc300b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc2ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc300b8d80 0x7fdc300b9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59264/0 (socket says 172.21.15.29:59264) 2024-04-25T00:37:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.355+0000 7fdc2ffff640 1 -- 172.21.15.29:0/1838000848 learned_addr learned my addr 172.21.15.29:0/1838000848 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2f7fe640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c0039f0 0x7fdc30149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc27fff640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc30160c80 0x7fdc300b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2ffff640 1 -- 172.21.15.29:0/1838000848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc30160c80 msgr2=0x7fdc300b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2ffff640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc30160c80 0x7fdc300b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2ffff640 1 -- 172.21.15.29:0/1838000848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c0039f0 msgr2=0x7fdc30149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:17.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2ffff640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c0039f0 0x7fdc30149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:17.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2ffff640 1 -- 172.21.15.29:0/1838000848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc300b7470 con 0x7fdc300b8d80 2024-04-25T00:37:17.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2ffff640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc300b8d80 0x7fdc300b9200 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fdc300b6a70 tx=0x7fdc2800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:17.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2cff9640 1 -- 172.21.15.29:0/1838000848 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28004d70 con 0x7fdc300b8d80 2024-04-25T00:37:17.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2cff9640 1 -- 172.21.15.29:0/1838000848 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc2800bb80 con 0x7fdc300b8d80 2024-04-25T00:37:17.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc35e8e640 1 -- 172.21.15.29:0/1838000848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc300b7750 con 0x7fdc300b8d80 2024-04-25T00:37:17.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2cff9640 1 -- 172.21.15.29:0/1838000848 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28007110 con 0x7fdc300b8d80 2024-04-25T00:37:17.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc35e8e640 1 -- 172.21.15.29:0/1838000848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc300b7d10 con 0x7fdc300b8d80 2024-04-25T00:37:17.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc35e8e640 1 -- 172.21.15.29:0/1838000848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc30069270 con 0x7fdc300b8d80 2024-04-25T00:37:17.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2cff9640 1 -- 172.21.15.29:0/1838000848 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fdc2801b020 con 0x7fdc300b8d80 2024-04-25T00:37:17.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2cff9640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc10067a10 0x7fdc10069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:17.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2cff9640 1 -- 172.21.15.29:0/1838000848 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fdc28085db0 con 0x7fdc300b8d80 2024-04-25T00:37:17.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2f7fe640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc10067a10 0x7fdc10069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:17.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.359+0000 7fdc2f7fe640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc10067a10 0x7fdc10069ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0057f0 tx=0x7fdc200011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:17.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.363+0000 7fdc2cff9640 1 -- 172.21.15.29:0/1838000848 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdc28056a00 con 0x7fdc300b8d80 2024-04-25T00:37:17.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:17.583+0000 7fdc35e8e640 1 -- 172.21.15.29:0/1838000848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7fdc300b2e20 con 0x7fdc300b8d80 2024-04-25T00:37:18.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.559+0000 7fdc2cff9640 1 -- 172.21.15.29:0/1838000848 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v313) ==== 152+0+0 (secure 0 0 0) 0x7fdc28056300 con 0x7fdc300b8d80 2024-04-25T00:37:18.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.563+0000 7fdc35e8e640 1 -- 172.21.15.29:0/1838000848 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc10067a10 msgr2=0x7fdc10069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:18.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.563+0000 7fdc35e8e640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc10067a10 0x7fdc10069ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0057f0 tx=0x7fdc200011a0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.563+0000 7fdc35e8e640 1 -- 172.21.15.29:0/1838000848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc300b8d80 msgr2=0x7fdc300b9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:18.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.563+0000 7fdc35e8e640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc300b8d80 0x7fdc300b9200 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fdc300b6a70 tx=0x7fdc2800b4a0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.563+0000 7fdc35e8e640 1 -- 172.21.15.29:0/1838000848 shutdown_connections 2024-04-25T00:37:18.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.563+0000 7fdc35e8e640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc10067a10 0x7fdc10069ed0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.563+0000 7fdc35e8e640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc300b8d80 0x7fdc300b9200 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.563+0000 7fdc35e8e640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc30160c80 0x7fdc300b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.563+0000 7fdc35e8e640 1 --2- 172.21.15.29:0/1838000848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c0039f0 0x7fdc30149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.563+0000 7fdc35e8e640 1 -- 172.21.15.29:0/1838000848 >> 172.21.15.29:0/1838000848 conn(0x7fdc3009d280 msgr2=0x7fdc300b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:18.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.563+0000 7fdc35e8e640 1 -- 172.21.15.29:0/1838000848 shutdown_connections 2024-04-25T00:37:18.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.563+0000 7fdc35e8e640 1 -- 172.21.15.29:0/1838000848 wait complete. 2024-04-25T00:37:18.588 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:37:18.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff3a7f640 1 Processor -- start 2024-04-25T00:37:18.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff3a7f640 1 -- start start 2024-04-25T00:37:18.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff3a7f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2fec0b3c30 0x7f2fec06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:18.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff3a7f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2fec0b4b10 0x7f2fec0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:18.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff3a7f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fec0b41e0 con 0x7f2fec0b3140 2024-04-25T00:37:18.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff3a7f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2fec06d960 con 0x7f2fec0b4b10 2024-04-25T00:37:18.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff3a7f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2fec06dc40 con 0x7f2fec0b3c30 2024-04-25T00:37:18.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff181b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2fec0b3140 0x7f2fec0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42694/0 (socket says 172.21.15.29:42694) 2024-04-25T00:37:18.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff181b640 1 -- 172.21.15.29:0/1547757902 learned_addr learned my addr 172.21.15.29:0/1547757902 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:18.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff201c640 1 --2- 172.21.15.29:0/1547757902 >> v2:172.21.15.29:3300/0 conn(0x7f2fec0b4b10 0x7f2fec0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:18.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff101a640 1 --2- 172.21.15.29:0/1547757902 >> v2:172.21.15.157:3300/0 conn(0x7f2fec0b3c30 0x7f2fec06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:18.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff201c640 1 -- 172.21.15.29:0/1547757902 >> v2:172.21.15.157:3300/0 conn(0x7f2fec0b3c30 msgr2=0x7f2fec06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:18.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff201c640 1 --2- 172.21.15.29:0/1547757902 >> v2:172.21.15.157:3300/0 conn(0x7f2fec0b3c30 0x7f2fec06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff201c640 1 -- 172.21.15.29:0/1547757902 >> v1:172.21.15.29:6789/0 conn(0x7f2fec0b3140 legacy=0x7f2fec0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:18.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff201c640 1 -- 172.21.15.29:0/1547757902 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fec06df20 con 0x7f2fec0b4b10 2024-04-25T00:37:18.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff101a640 1 --2- 172.21.15.29:0/1547757902 >> v2:172.21.15.157:3300/0 conn(0x7f2fec0b3c30 0x7f2fec06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:18.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff201c640 1 --2- 172.21.15.29:0/1547757902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fec0b4b10 0x7f2fec0ce240 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009d90 tx=0x7f2fe401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731d467bc189922b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:18.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff0819640 1 -- 172.21.15.29:0/1547757902 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40b2070 con 0x7f2fec0b4b10 2024-04-25T00:37:18.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff0819640 1 -- 172.21.15.29:0/1547757902 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe40a57f0 con 0x7f2fec0b4b10 2024-04-25T00:37:18.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.719+0000 7f2ff0819640 1 -- 172.21.15.29:0/1547757902 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40a5bc0 con 0x7f2fec0b4b10 2024-04-25T00:37:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/1547757902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fec0b4b10 msgr2=0x7f2fec0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 --2- 172.21.15.29:0/1547757902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fec0b4b10 0x7f2fec0ce240 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009d90 tx=0x7f2fe401c930 comp rx=0 tx=0).stop 2024-04-25T00:37:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/1547757902 shutdown_connections 2024-04-25T00:37:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 --2- 172.21.15.29:0/1547757902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fec0b4b10 0x7f2fec0ce240 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 --2- 172.21.15.29:0/1547757902 >> v2:172.21.15.157:3300/0 conn(0x7f2fec0b3c30 0x7f2fec06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/1547757902 >> 172.21.15.29:0/1547757902 conn(0x7f2fec0b1870 msgr2=0x7f2fec0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/1547757902 shutdown_connections 2024-04-25T00:37:18.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/1547757902 wait complete. 2024-04-25T00:37:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 Processor -- start 2024-04-25T00:37:18.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 -- start start 2024-04-25T00:37:18.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fec0b3140 0x7f2fec1393b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec0b3c30 0x7f2fec154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fec0b4b10 0x7f2fec0716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec168d80 con 0x7f2fec0b4b10 2024-04-25T00:37:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec168c00 con 0x7f2fec0b3140 2024-04-25T00:37:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2fec168f00 con 0x7f2fec0b3c30 2024-04-25T00:37:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff201c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fec0b4b10 0x7f2fec0716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff201c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fec0b4b10 0x7f2fec0716f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38140/0 (socket says 172.21.15.29:38140) 2024-04-25T00:37:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff201c640 1 -- 172.21.15.29:0/2000126355 learned_addr learned my addr 172.21.15.29:0/2000126355 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff101a640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec0b3c30 0x7f2fec154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff181b640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fec0b3140 0x7f2fec1393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff201c640 1 -- 172.21.15.29:0/2000126355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec0b3c30 msgr2=0x7f2fec154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff201c640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec0b3c30 0x7f2fec154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff201c640 1 -- 172.21.15.29:0/2000126355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fec0b3140 msgr2=0x7f2fec1393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:18.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff201c640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fec0b3140 0x7f2fec1393b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff201c640 1 -- 172.21.15.29:0/2000126355 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fec071dd0 con 0x7f2fec0b4b10 2024-04-25T00:37:18.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff201c640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fec0b4b10 0x7f2fec0716f0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40b0040 tx=0x7f2fe40196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:18.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2fe27fc640 1 -- 172.21.15.29:0/2000126355 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40b2050 con 0x7f2fec0b4b10 2024-04-25T00:37:18.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2fe27fc640 1 -- 172.21.15.29:0/2000126355 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe40be070 con 0x7f2fec0b4b10 2024-04-25T00:37:18.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2fe27fc640 1 -- 172.21.15.29:0/2000126355 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40ada70 con 0x7f2fec0b4b10 2024-04-25T00:37:18.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/2000126355 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fec0720b0 con 0x7f2fec0b4b10 2024-04-25T00:37:18.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.723+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/2000126355 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fec0cfaf0 con 0x7f2fec0b4b10 2024-04-25T00:37:18.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.727+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/2000126355 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fec069270 con 0x7f2fec0b4b10 2024-04-25T00:37:18.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.727+0000 7f2fe27fc640 1 -- 172.21.15.29:0/2000126355 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f2fe40adc10 con 0x7f2fec0b4b10 2024-04-25T00:37:18.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.727+0000 7f2fe27fc640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2fd00679e0 0x7f2fd0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:18.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.727+0000 7f2fe27fc640 1 -- 172.21.15.29:0/2000126355 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f2fe4120d20 con 0x7f2fec0b4b10 2024-04-25T00:37:18.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.727+0000 7f2ff181b640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2fd00679e0 0x7f2fd0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:18.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.727+0000 7f2ff181b640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2fd00679e0 0x7f2fd0069ea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fec1f84e0 tx=0x7f2fe8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:18.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.731+0000 7f2fe27fc640 1 -- 172.21.15.29:0/2000126355 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2fe4016520 con 0x7f2fec0b4b10 2024-04-25T00:37:18.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.895+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/2000126355 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2fec1f8610 con 0x7f2fec0b4b10 2024-04-25T00:37:18.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.899+0000 7f2fe27fc640 1 -- 172.21.15.29:0/2000126355 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f2fe40f1430 con 0x7f2fec0b4b10 2024-04-25T00:37:18.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.903+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/2000126355 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2fd00679e0 msgr2=0x7f2fd0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:18.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.903+0000 7f2ff3a7f640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2fd00679e0 0x7f2fd0069ea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fec1f84e0 tx=0x7f2fe8009040 comp rx=0 tx=0).stop 2024-04-25T00:37:18.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.903+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/2000126355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fec0b4b10 msgr2=0x7f2fec0716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:18.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.903+0000 7f2ff3a7f640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fec0b4b10 0x7f2fec0716f0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40b0040 tx=0x7f2fe40196d0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.903+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/2000126355 shutdown_connections 2024-04-25T00:37:18.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.903+0000 7f2ff3a7f640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2fd00679e0 0x7f2fd0069ea0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.903+0000 7f2ff3a7f640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fec0b4b10 0x7f2fec0716f0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.903+0000 7f2ff3a7f640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec0b3c30 0x7f2fec154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.903+0000 7f2ff3a7f640 1 --2- 172.21.15.29:0/2000126355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fec0b3140 0x7f2fec1393b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:18.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.903+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/2000126355 >> 172.21.15.29:0/2000126355 conn(0x7f2fec0b1870 msgr2=0x7f2fec0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:18.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.903+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/2000126355 shutdown_connections 2024-04-25T00:37:18.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:18.903+0000 7f2ff3a7f640 1 -- 172.21.15.29:0/2000126355 wait complete. 2024-04-25T00:37:18.923 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:37:18.923 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:37:18.969 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:37:19.005 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:37:19.005 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:37:19.005 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:19.011 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:37:19.011 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:37:19.011 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:19.051 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:37:19.052 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:37:19.052 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:19.058 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:37:19.058 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:37:19.058 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:19.100 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:37:19.100 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:37:19.101 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:37:19.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.231+0000 7f9702d44640 1 Processor -- start 2024-04-25T00:37:19.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.231+0000 7f9702d44640 1 -- start start 2024-04-25T00:37:19.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.231+0000 7f9702d44640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f96fc06dce0 0x7f96fc150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:19.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.231+0000 7f9702d44640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96fc0b3130 con 0x7f96fc0b2b70 2024-04-25T00:37:19.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.231+0000 7f9702d44640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96fc0b3410 con 0x7f96fc15d980 2024-04-25T00:37:19.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.231+0000 7f9702d44640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f96fc0cf560 con 0x7f96fc06dce0 2024-04-25T00:37:19.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.231+0000 7f96fbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f96fc06dce0 0x7f96fc150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:19.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.231+0000 7f97012e1640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f96fc0b2b70 0x7f96fc151330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42708/0 (socket says 172.21.15.29:42708) 2024-04-25T00:37:19.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.231+0000 7f97012e1640 1 -- 172.21.15.29:0/4191991062 learned_addr learned my addr 172.21.15.29:0/4191991062 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:19.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.231+0000 7f96fbfff640 1 -- 172.21.15.29:0/4191991062 >> v1:172.21.15.157:6789/0 conn(0x7f96fc15d980 legacy=0x7f96fc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:19.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.231+0000 7f96fbfff640 1 -- 172.21.15.29:0/4191991062 >> v1:172.21.15.29:6789/0 conn(0x7f96fc0b2b70 legacy=0x7f96fc151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:19.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.231+0000 7f96fbfff640 1 -- 172.21.15.29:0/4191991062 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc0b36f0 con 0x7f96fc06dce0 2024-04-25T00:37:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.231+0000 7f96fbfff640 1 --2- 172.21.15.29:0/4191991062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96fc06dce0 0x7f96fc150df0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f96e400a960 tx=0x7f96e401d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafdb719420b74cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96fb7fe640 1 -- 172.21.15.29:0/4191991062 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e400b040 con 0x7f96fc06dce0 2024-04-25T00:37:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96fb7fe640 1 -- 172.21.15.29:0/4191991062 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e40a7690 con 0x7f96fc06dce0 2024-04-25T00:37:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96fb7fe640 1 -- 172.21.15.29:0/4191991062 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e40a7a60 con 0x7f96fc06dce0 2024-04-25T00:37:19.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 -- 172.21.15.29:0/4191991062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96fc06dce0 msgr2=0x7f96fc150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:19.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 --2- 172.21.15.29:0/4191991062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96fc06dce0 0x7f96fc150df0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f96e400a960 tx=0x7f96e401d830 comp rx=0 tx=0).stop 2024-04-25T00:37:19.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 -- 172.21.15.29:0/4191991062 shutdown_connections 2024-04-25T00:37:19.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 --2- 172.21.15.29:0/4191991062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96fc06dce0 0x7f96fc150df0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 -- 172.21.15.29:0/4191991062 >> 172.21.15.29:0/4191991062 conn(0x7f96fc0b1870 msgr2=0x7f96fc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:19.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 -- 172.21.15.29:0/4191991062 shutdown_connections 2024-04-25T00:37:19.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 -- 172.21.15.29:0/4191991062 wait complete. 2024-04-25T00:37:19.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 Processor -- start 2024-04-25T00:37:19.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 -- start start 2024-04-25T00:37:19.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96fc06dce0 0x7f96fc1548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:19.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96fc0b2b70 0x7f96fc0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:19.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96fc15d980 0x7f96fc157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:19.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc15c640 con 0x7f96fc0b2b70 2024-04-25T00:37:19.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc15c4c0 con 0x7f96fc15d980 2024-04-25T00:37:19.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96fc15c7c0 con 0x7f96fc06dce0 2024-04-25T00:37:19.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96fbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96fc0b2b70 0x7f96fc0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:19.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96fbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96fc0b2b70 0x7f96fc0cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38156/0 (socket says 172.21.15.29:38156) 2024-04-25T00:37:19.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96fbfff640 1 -- 172.21.15.29:0/2139681702 learned_addr learned my addr 172.21.15.29:0/2139681702 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:19.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96fbfff640 1 -- 172.21.15.29:0/2139681702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96fc06dce0 msgr2=0x7f96fc1548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:19.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96fbfff640 1 --2- 172.21.15.29:0/2139681702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96fc06dce0 0x7f96fc1548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96fbfff640 1 -- 172.21.15.29:0/2139681702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96fc15d980 msgr2=0x7f96fc157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:19.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f97012e1640 1 --2- 172.21.15.29:0/2139681702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96fc15d980 0x7f96fc157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:19.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96fbfff640 1 --2- 172.21.15.29:0/2139681702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96fc15d980 0x7f96fc157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96fbfff640 1 -- 172.21.15.29:0/2139681702 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc0cc9f0 con 0x7f96fc0b2b70 2024-04-25T00:37:19.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96fbfff640 1 --2- 172.21.15.29:0/2139681702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96fc0b2b70 0x7f96fc0cda90 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f96e401a440 tx=0x7f96e401c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:19.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96f97fa640 1 -- 172.21.15.29:0/2139681702 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e40a8880 con 0x7f96fc0b2b70 2024-04-25T00:37:19.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96f97fa640 1 -- 172.21.15.29:0/2139681702 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e40a8a20 con 0x7f96fc0b2b70 2024-04-25T00:37:19.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 -- 172.21.15.29:0/2139681702 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96fc0cccd0 con 0x7f96fc0b2b70 2024-04-25T00:37:19.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f96f97fa640 1 -- 172.21.15.29:0/2139681702 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e40a8e10 con 0x7f96fc0b2b70 2024-04-25T00:37:19.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.235+0000 7f9702d44640 1 -- 172.21.15.29:0/2139681702 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96fc0cd290 con 0x7f96fc0b2b70 2024-04-25T00:37:19.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.239+0000 7f9702d44640 1 -- 172.21.15.29:0/2139681702 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96fc069270 con 0x7f96fc0b2b70 2024-04-25T00:37:19.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.239+0000 7f96f97fa640 1 -- 172.21.15.29:0/2139681702 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f96e40a7510 con 0x7f96fc0b2b70 2024-04-25T00:37:19.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.239+0000 7f96f97fa640 1 --2- 172.21.15.29:0/2139681702 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f96dc0679e0 0x7f96dc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:19.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.239+0000 7f96f97fa640 1 -- 172.21.15.29:0/2139681702 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f96e41206d0 con 0x7f96fc0b2b70 2024-04-25T00:37:19.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.239+0000 7f9700ae0640 1 --2- 172.21.15.29:0/2139681702 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f96dc0679e0 0x7f96dc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:19.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.239+0000 7f9700ae0640 1 --2- 172.21.15.29:0/2139681702 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f96dc0679e0 0x7f96dc069ea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f96fc14de00 tx=0x7f96f4006bd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:19.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.243+0000 7f96f97fa640 1 -- 172.21.15.29:0/2139681702 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f96e40ad030 con 0x7f96fc0b2b70 2024-04-25T00:37:19.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f9702d44640 1 -- 172.21.15.29:0/2139681702 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96fc0cf910 con 0x7f96fc0b2b70 2024-04-25T00:37:19.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f96f97fa640 1 -- 172.21.15.29:0/2139681702 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+3872 (secure 0 0 0) 0x7f96e4017420 con 0x7f96fc0b2b70 2024-04-25T00:37:19.438 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:19.438 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":61,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2434273880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2434273880},{"type":"v1","addr":"172.21.15.157:6833","nonce":2434273880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/558648019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":558648019},{"type":"v1","addr":"172.21.15.157:6835","nonce":558648019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":7665,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/4178766542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4178766542},{"type":"v1","addr":"172.21.15.29:6833","nonce":4178766542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":7812,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/1434463534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1434463534},{"type":"v1","addr":"172.21.15.29:6835","nonce":1434463534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[]} 2024-04-25T00:37:19.438 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 61 2024-04-25T00:37:19.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f9702d44640 1 -- 172.21.15.29:0/2139681702 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f96dc0679e0 msgr2=0x7f96dc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:19.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f9702d44640 1 --2- 172.21.15.29:0/2139681702 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f96dc0679e0 0x7f96dc069ea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f96fc14de00 tx=0x7f96f4006bd0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f9702d44640 1 -- 172.21.15.29:0/2139681702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96fc0b2b70 msgr2=0x7f96fc0cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:19.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f9702d44640 1 --2- 172.21.15.29:0/2139681702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96fc0b2b70 0x7f96fc0cda90 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f96e401a440 tx=0x7f96e401c070 comp rx=0 tx=0).stop 2024-04-25T00:37:19.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f9702d44640 1 -- 172.21.15.29:0/2139681702 shutdown_connections 2024-04-25T00:37:19.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f9702d44640 1 --2- 172.21.15.29:0/2139681702 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f96dc0679e0 0x7f96dc069ea0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f9702d44640 1 --2- 172.21.15.29:0/2139681702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96fc15d980 0x7f96fc157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f9702d44640 1 --2- 172.21.15.29:0/2139681702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96fc0b2b70 0x7f96fc0cda90 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f9702d44640 1 --2- 172.21.15.29:0/2139681702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96fc06dce0 0x7f96fc1548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f9702d44640 1 -- 172.21.15.29:0/2139681702 >> 172.21.15.29:0/2139681702 conn(0x7f96fc0b1870 msgr2=0x7f96fc0b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:19.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f9702d44640 1 -- 172.21.15.29:0/2139681702 shutdown_connections 2024-04-25T00:37:19.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.435+0000 7f9702d44640 1 -- 172.21.15.29:0/2139681702 wait complete. 2024-04-25T00:37:19.456 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:37:19.456 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:37:19.456 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:37:19.456 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:37:19.456 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:37:19.456 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:37:19.457 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:37:19.457 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:37:19.457 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:37:19.457 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:37:19.457 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:37:19.457 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:37:19.457 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:37:19.457 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:37:19.458 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:37:19.458 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:37:19.467 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:37:19.468 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:37:19.468 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:37:19.468 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:37:19.472 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:37:19.472 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:37:19.475 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:37:19.475 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:37:19.517 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:37:19.518 DEBUG:teuthology.parallel:result is None 2024-04-25T00:37:19.518 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:37:19.518 DEBUG:teuthology.parallel:result is None 2024-04-25T00:37:19.520 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:37:19.520 DEBUG:teuthology.parallel:result is None 2024-04-25T00:37:19.521 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:37:19.522 DEBUG:teuthology.parallel:result is None 2024-04-25T00:37:19.522 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:37:19.578 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:37:19.579 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:37:19.587 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-25T00:37:19.591 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-25T00:37:19.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb274cd3640 1 Processor -- start 2024-04-25T00:37:19.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb274cd3640 1 -- start start 2024-04-25T00:37:19.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb274cd3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2701584c0 0x7fb2700715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:19.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb274cd3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb2701588a0 0x7fb270071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:19.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb274cd3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2700b4060 0x7fb2700b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:19.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb274cd3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb27006d330 con 0x7fb2701588a0 2024-04-25T00:37:19.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb274cd3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb2700ae3b0 con 0x7fb2700b4060 2024-04-25T00:37:19.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb274cd3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2700ae950 con 0x7fb2701584c0 2024-04-25T00:37:19.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb26dd9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb2701588a0 0x7fb270071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:19.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb26dd9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb2701588a0 0x7fb270071af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38186/0 (socket says 172.21.15.29:38186) 2024-04-25T00:37:19.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb26dd9b640 1 -- 172.21.15.29:0/3094249356 learned_addr learned my addr 172.21.15.29:0/3094249356 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:19.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb26ed9d640 1 --2- 172.21.15.29:0/3094249356 >> v2:172.21.15.157:3300/0 conn(0x7fb2700b4060 0x7fb2700b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:19.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb26dd9b640 1 -- 172.21.15.29:0/3094249356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2701584c0 msgr2=0x7fb2700715b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:37:19.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb26dd9b640 1 --2- 172.21.15.29:0/3094249356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2701584c0 0x7fb2700715b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb26dd9b640 1 -- 172.21.15.29:0/3094249356 >> v2:172.21.15.157:3300/0 conn(0x7fb2700b4060 msgr2=0x7fb2700b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:19.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb26dd9b640 1 --2- 172.21.15.29:0/3094249356 >> v2:172.21.15.157:3300/0 conn(0x7fb2700b4060 0x7fb2700b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb26dd9b640 1 -- 172.21.15.29:0/3094249356 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27015e930 con 0x7fb2701588a0 2024-04-25T00:37:19.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.627+0000 7fb26dd9b640 1 --2- 172.21.15.29:0/3094249356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2701588a0 0x7fb270071af0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fb2700749d0 tx=0x7fb26001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7fb70b4b988c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:19.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26d59a640 1 -- 172.21.15.29:0/3094249356 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2600ab5f0 con 0x7fb2701588a0 2024-04-25T00:37:19.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26d59a640 1 -- 172.21.15.29:0/3094249356 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2600b0070 con 0x7fb2701588a0 2024-04-25T00:37:19.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26d59a640 1 -- 172.21.15.29:0/3094249356 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2600ab9b0 con 0x7fb2701588a0 2024-04-25T00:37:19.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 -- 172.21.15.29:0/3094249356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2701588a0 msgr2=0x7fb270071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:19.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 --2- 172.21.15.29:0/3094249356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2701588a0 0x7fb270071af0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fb2700749d0 tx=0x7fb26001c540 comp rx=0 tx=0).stop 2024-04-25T00:37:19.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 -- 172.21.15.29:0/3094249356 shutdown_connections 2024-04-25T00:37:19.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 --2- 172.21.15.29:0/3094249356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2701588a0 0x7fb270071af0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 --2- 172.21.15.29:0/3094249356 >> v2:172.21.15.157:3300/0 conn(0x7fb2700b4060 0x7fb2700b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 --2- 172.21.15.29:0/3094249356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2701584c0 0x7fb2700715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 -- 172.21.15.29:0/3094249356 >> 172.21.15.29:0/3094249356 conn(0x7fb270075030 msgr2=0x7fb2700b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:19.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 -- 172.21.15.29:0/3094249356 shutdown_connections 2024-04-25T00:37:19.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 -- 172.21.15.29:0/3094249356 wait complete. 2024-04-25T00:37:19.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 Processor -- start 2024-04-25T00:37:19.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 -- start start 2024-04-25T00:37:19.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2700b4060 0x7fb2700b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:19.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2701584c0 0x7fb270154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:19.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2701588a0 0x7fb2700ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:19.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701623f0 con 0x7fb2701588a0 2024-04-25T00:37:19.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2700b5200 con 0x7fb2700b4060 2024-04-25T00:37:19.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb270162570 con 0x7fb2701584c0 2024-04-25T00:37:19.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2701588a0 0x7fb2700ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2701588a0 0x7fb2700ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38190/0 (socket says 172.21.15.29:38190) 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26ed9d640 1 -- 172.21.15.29:0/2701693853 learned_addr learned my addr 172.21.15.29:0/2701693853 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26dd9b640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2701584c0 0x7fb270154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26e59c640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2700b4060 0x7fb2700b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26ed9d640 1 -- 172.21.15.29:0/2701693853 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2701584c0 msgr2=0x7fb270154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26ed9d640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2701584c0 0x7fb270154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26ed9d640 1 -- 172.21.15.29:0/2701693853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2700b4060 msgr2=0x7fb2700b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26ed9d640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2700b4060 0x7fb2700b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26ed9d640 1 -- 172.21.15.29:0/2701693853 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2700b1e00 con 0x7fb2701588a0 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb26ed9d640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2701588a0 0x7fb2700ce5b0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fb268007e40 tx=0x7fb26800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb266ffd640 1 -- 172.21.15.29:0/2701693853 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268014620 con 0x7fb2701588a0 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb266ffd640 1 -- 172.21.15.29:0/2701693853 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb268019070 con 0x7fb2701588a0 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb266ffd640 1 -- 172.21.15.29:0/2701693853 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268014a20 con 0x7fb2701588a0 2024-04-25T00:37:19.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 -- 172.21.15.29:0/2701693853 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27014a420 con 0x7fb2701588a0 2024-04-25T00:37:19.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.631+0000 7fb274cd3640 1 -- 172.21.15.29:0/2701693853 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27014a9e0 con 0x7fb2701588a0 2024-04-25T00:37:19.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.635+0000 7fb274cd3640 1 -- 172.21.15.29:0/2701693853 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb270069270 con 0x7fb2701588a0 2024-04-25T00:37:19.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.635+0000 7fb266ffd640 1 -- 172.21.15.29:0/2701693853 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fb268014bc0 con 0x7fb2701588a0 2024-04-25T00:37:19.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.635+0000 7fb266ffd640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb2500679e0 0x7fb250069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:19.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.635+0000 7fb266ffd640 1 -- 172.21.15.29:0/2701693853 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fb268086260 con 0x7fb2701588a0 2024-04-25T00:37:19.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.635+0000 7fb26e59c640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb2500679e0 0x7fb250069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:19.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.635+0000 7fb26e59c640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb2500679e0 0x7fb250069ea0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb2700b1680 tx=0x7fb2580090e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:19.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.639+0000 7fb266ffd640 1 -- 172.21.15.29:0/2701693853 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb2700b1e00 con 0x7fb2701588a0 2024-04-25T00:37:19.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:19.803+0000 7fb274cd3640 1 -- 172.21.15.29:0/2701693853 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb27014b320 con 0x7fb2701588a0 2024-04-25T00:37:20.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:20.923+0000 7fb266ffd640 1 -- 172.21.15.29:0/2701693853 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) ==== 97+0+0 (secure 0 0 0) 0x7fb268056970 con 0x7fb2701588a0 2024-04-25T00:37:20.927 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-04-25T00:37:20.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:20.927+0000 7fb274cd3640 1 -- 172.21.15.29:0/2701693853 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb2500679e0 msgr2=0x7fb250069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:20.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:20.927+0000 7fb274cd3640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb2500679e0 0x7fb250069ea0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb2700b1680 tx=0x7fb2580090e0 comp rx=0 tx=0).stop 2024-04-25T00:37:20.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:20.927+0000 7fb274cd3640 1 -- 172.21.15.29:0/2701693853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2701588a0 msgr2=0x7fb2700ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:20.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:20.927+0000 7fb274cd3640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2701588a0 0x7fb2700ce5b0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fb268007e40 tx=0x7fb26800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:37:20.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:20.927+0000 7fb274cd3640 1 -- 172.21.15.29:0/2701693853 shutdown_connections 2024-04-25T00:37:20.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:20.927+0000 7fb274cd3640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb2500679e0 0x7fb250069ea0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:20.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:20.927+0000 7fb274cd3640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2701588a0 0x7fb2700ce5b0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:20.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:20.927+0000 7fb274cd3640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2701584c0 0x7fb270154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:20.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:20.927+0000 7fb274cd3640 1 --2- 172.21.15.29:0/2701693853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2700b4060 0x7fb2700b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:20.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:20.927+0000 7fb274cd3640 1 -- 172.21.15.29:0/2701693853 >> 172.21.15.29:0/2701693853 conn(0x7fb270075030 msgr2=0x7fb27006e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:20.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:20.927+0000 7fb274cd3640 1 -- 172.21.15.29:0/2701693853 shutdown_connections 2024-04-25T00:37:20.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:20.927+0000 7fb274cd3640 1 -- 172.21.15.29:0/2701693853 wait complete. 2024-04-25T00:37:20.952 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:37:21.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.079+0000 7efc3ca53640 1 Processor -- start 2024-04-25T00:37:21.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc3ca53640 1 -- start start 2024-04-25T00:37:21.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc3ca53640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efc380cf8d0 0x7efc3815def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc3ca53640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc380ae590 0x7efc3815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc3ca53640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc380af620 con 0x7efc380cfcb0 2024-04-25T00:37:21.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc3ca53640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7efc38074320 con 0x7efc380cf8d0 2024-04-25T00:37:21.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc3ca53640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc38074600 con 0x7efc380ae590 2024-04-25T00:37:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc3659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efc380cf8d0 0x7efc3815def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc36d9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc380ae590 0x7efc3815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc3659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efc380cf8d0 0x7efc3815def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:34120/0 (socket says 172.21.15.29:34120) 2024-04-25T00:37:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc3659c640 1 -- 172.21.15.29:0/3208102618 learned_addr learned my addr 172.21.15.29:0/3208102618 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc35d9b640 1 --1- 172.21.15.29:0/3208102618 >> v1:172.21.15.157:6789/0 conn(0x7efc380cfcb0 0x7efc38161f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:54830/0 (socket says 172.21.15.29:54830) 2024-04-25T00:37:21.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc36d9d640 1 -- 172.21.15.29:0/3208102618 >> v2:172.21.15.157:3300/0 conn(0x7efc380cf8d0 msgr2=0x7efc3815def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:21.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc36d9d640 1 --2- 172.21.15.29:0/3208102618 >> v2:172.21.15.157:3300/0 conn(0x7efc380cf8d0 0x7efc3815def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc36d9d640 1 -- 172.21.15.29:0/3208102618 >> v1:172.21.15.157:6789/0 conn(0x7efc380cfcb0 legacy=0x7efc38161f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc36d9d640 1 -- 172.21.15.29:0/3208102618 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3806bf80 con 0x7efc380ae590 2024-04-25T00:37:21.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.083+0000 7efc3659c640 1 --2- 172.21.15.29:0/3208102618 >> v2:172.21.15.157:3300/0 conn(0x7efc380cf8d0 0x7efc3815def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:37:21.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.087+0000 7efc36d9d640 1 --2- 172.21.15.29:0/3208102618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc380ae590 0x7efc3815e430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efc300097c0 tx=0x7efc3001c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f43c27fbe1dd4f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:21.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.087+0000 7efc3559a640 1 -- 172.21.15.29:0/3208102618 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc300b0070 con 0x7efc380ae590 2024-04-25T00:37:21.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.087+0000 7efc3559a640 1 -- 172.21.15.29:0/3208102618 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc300ab600 con 0x7efc380ae590 2024-04-25T00:37:21.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.087+0000 7efc3559a640 1 -- 172.21.15.29:0/3208102618 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc300ab9d0 con 0x7efc380ae590 2024-04-25T00:37:21.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.087+0000 7efc3ca53640 1 -- 172.21.15.29:0/3208102618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc380ae590 msgr2=0x7efc3815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.087+0000 7efc3ca53640 1 --2- 172.21.15.29:0/3208102618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc380ae590 0x7efc3815e430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efc300097c0 tx=0x7efc3001c380 comp rx=0 tx=0).stop 2024-04-25T00:37:21.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.087+0000 7efc3ca53640 1 -- 172.21.15.29:0/3208102618 shutdown_connections 2024-04-25T00:37:21.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.087+0000 7efc3ca53640 1 --2- 172.21.15.29:0/3208102618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc380ae590 0x7efc3815e430 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.087+0000 7efc3ca53640 1 --2- 172.21.15.29:0/3208102618 >> v2:172.21.15.157:3300/0 conn(0x7efc380cf8d0 0x7efc3815def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.087+0000 7efc3ca53640 1 -- 172.21.15.29:0/3208102618 >> 172.21.15.29:0/3208102618 conn(0x7efc380b1ec0 msgr2=0x7efc380b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:21.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.087+0000 7efc3ca53640 1 -- 172.21.15.29:0/3208102618 shutdown_connections 2024-04-25T00:37:21.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.087+0000 7efc3ca53640 1 -- 172.21.15.29:0/3208102618 wait complete. 2024-04-25T00:37:21.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc3ca53640 1 Processor -- start 2024-04-25T00:37:21.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc3ca53640 1 -- start start 2024-04-25T00:37:21.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc3ca53640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc380ae590 0x7efc38099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc3ca53640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc380cf8d0 0x7efc3815d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc3ca53640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc380cfcb0 0x7efc38154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc3ca53640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc3816b450 con 0x7efc380cfcb0 2024-04-25T00:37:21.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc3ca53640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc3816b2d0 con 0x7efc380ae590 2024-04-25T00:37:21.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc3ca53640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc3816b5d0 con 0x7efc380cf8d0 2024-04-25T00:37:21.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc36d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc380cfcb0 0x7efc38154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc36d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc380cfcb0 0x7efc38154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38196/0 (socket says 172.21.15.29:38196) 2024-04-25T00:37:21.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc36d9d640 1 -- 172.21.15.29:0/89772639 learned_addr learned my addr 172.21.15.29:0/89772639 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:21.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc35d9b640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc380cf8d0 0x7efc3815d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc3659c640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc380ae590 0x7efc38099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc36d9d640 1 -- 172.21.15.29:0/89772639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc380cf8d0 msgr2=0x7efc3815d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc36d9d640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc380cf8d0 0x7efc3815d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc36d9d640 1 -- 172.21.15.29:0/89772639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc380ae590 msgr2=0x7efc38099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc36d9d640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc380ae590 0x7efc38099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc36d9d640 1 -- 172.21.15.29:0/89772639 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc380d0430 con 0x7efc380cfcb0 2024-04-25T00:37:21.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc36d9d640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc380cfcb0 0x7efc38154b10 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7efc30019fb0 tx=0x7efc30019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:21.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc26ffd640 1 -- 172.21.15.29:0/89772639 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc300abe60 con 0x7efc380cfcb0 2024-04-25T00:37:21.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc26ffd640 1 -- 172.21.15.29:0/89772639 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc300b9070 con 0x7efc380cfcb0 2024-04-25T00:37:21.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc3ca53640 1 -- 172.21.15.29:0/89772639 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3815efb0 con 0x7efc380cfcb0 2024-04-25T00:37:21.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc26ffd640 1 -- 172.21.15.29:0/89772639 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc300b0040 con 0x7efc380cfcb0 2024-04-25T00:37:21.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc3ca53640 1 -- 172.21.15.29:0/89772639 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc380c7100 con 0x7efc380cfcb0 2024-04-25T00:37:21.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc3ca53640 1 -- 172.21.15.29:0/89772639 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc38069270 con 0x7efc380cfcb0 2024-04-25T00:37:21.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.091+0000 7efc26ffd640 1 -- 172.21.15.29:0/89772639 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7efc300b4650 con 0x7efc380cfcb0 2024-04-25T00:37:21.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.095+0000 7efc26ffd640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efc180679e0 0x7efc18069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.095+0000 7efc26ffd640 1 -- 172.21.15.29:0/89772639 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7efc3011f800 con 0x7efc380cfcb0 2024-04-25T00:37:21.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.095+0000 7efc3659c640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efc180679e0 0x7efc18069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.095+0000 7efc3659c640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efc180679e0 0x7efc18069ea0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc380b62a0 tx=0x7efc2800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:21.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.099+0000 7efc26ffd640 1 -- 172.21.15.29:0/89772639 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efc30016090 con 0x7efc380cfcb0 2024-04-25T00:37:21.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.307+0000 7efc3ca53640 1 -- 172.21.15.29:0/89772639 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7efc380b6310 con 0x7efc380cfcb0 2024-04-25T00:37:21.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.307+0000 7efc26ffd640 1 -- 172.21.15.29:0/89772639 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7efc300f03e0 con 0x7efc380cfcb0 2024-04-25T00:37:21.311 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:21.311 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmyzgKAxAAU+nrUJgQhDX6xlARNFI/ng==", 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-04-25T00:37:21.312 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmtyzPERAAxNUDANw1H/ZzOo+dhgzxlg==", 2024-04-25T00:37:21.313 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.313 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:37:21.313 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:37:21.313 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:37:21.313 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.313 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.313 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-04-25T00:37:21.313 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmYUl8CxAAj6pV1+s4a0rpWLq2EgZ+8w==", 2024-04-25T00:37:21.313 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.313 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:37:21.313 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmuAhKGhAAOLIcWUodW5TqtVgHWZl12A==", 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.315 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-04-25T00:37:21.315 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAzpClmudk6MhAA6YFyd57jwtB2pdOaad7bxQ==", 2024-04-25T00:37:21.315 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.315 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:37:21.315 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:37:21.315 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.315 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.315 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-04-25T00:37:21.315 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA2pClm76q0NRAAxWhEP2ZGoFnZcuiQyGjglw==", 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA6pClmcgULABAA2F5WBR9Df0v4Sn875AzF7g==", 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA9pClm9QuRBhAALFLruv4JU/uNZYMX0yNuXw==", 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBBpClmNj4VDBAAxaxMvyaCvjynPY3f81udnw==", 2024-04-25T00:37:21.317 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBEpClmzBybEBAArry1lM4RIsM/txKfgBqDpw==", 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.319 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-04-25T00:37:21.319 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBHpClm5HfjFhAAbFwJCpbhegLWDu0mbS8FoA==", 2024-04-25T00:37:21.319 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.319 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:37:21.319 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:37:21.319 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.319 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.319 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-04-25T00:37:21.319 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBKpClme/7oHBAAOR2iQ6CweexR7sC1PgesZw==", 2024-04-25T00:37:21.319 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A==", 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm8W/EJxAA0HZW24Hhcwf+soSZn6jHJg==", 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmGuQJJRAAhVd4D0jc6Whgyf+B6xgFMw==", 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.321 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:37:21.322 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:37:21.322 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:37:21.322 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:37:21.322 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.322 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-04-25T00:37:21.322 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmn+ciLRAAg4tQuZ13WP0RnWJaJq5fUQ==", 2024-04-25T00:37:21.322 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.322 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:37:21.322 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:37:21.322 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:37:21.322 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmPVihJBAA7PQge/teyRZKSkBnufZSZw==", 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-04-25T00:37:21.323 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClm6WrdOhAAsY9FLCxG7PGpq2NCY4hGhQ==", 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmeR4UOBAANs9SRolOm6E8+/4ciupk5w==", 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:37:21.324 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.325 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:21.325 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:37:21.325 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:37:21.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.311+0000 7efc3ca53640 1 -- 172.21.15.29:0/89772639 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efc180679e0 msgr2=0x7efc18069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.311+0000 7efc3ca53640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efc180679e0 0x7efc18069ea0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc380b62a0 tx=0x7efc2800b040 comp rx=0 tx=0).stop 2024-04-25T00:37:21.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.311+0000 7efc3ca53640 1 -- 172.21.15.29:0/89772639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc380cfcb0 msgr2=0x7efc38154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.311+0000 7efc3ca53640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc380cfcb0 0x7efc38154b10 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7efc30019fb0 tx=0x7efc30019120 comp rx=0 tx=0).stop 2024-04-25T00:37:21.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.311+0000 7efc3ca53640 1 -- 172.21.15.29:0/89772639 shutdown_connections 2024-04-25T00:37:21.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.311+0000 7efc3ca53640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efc180679e0 0x7efc18069ea0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.311+0000 7efc3ca53640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc380cfcb0 0x7efc38154b10 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.311+0000 7efc3ca53640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc380cf8d0 0x7efc3815d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.311+0000 7efc3ca53640 1 --2- 172.21.15.29:0/89772639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc380ae590 0x7efc38099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.311+0000 7efc3ca53640 1 -- 172.21.15.29:0/89772639 >> 172.21.15.29:0/89772639 conn(0x7efc380b1ec0 msgr2=0x7efc38161bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:21.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.311+0000 7efc3ca53640 1 -- 172.21.15.29:0/89772639 shutdown_connections 2024-04-25T00:37:21.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.311+0000 7efc3ca53640 1 -- 172.21.15.29:0/89772639 wait complete. 2024-04-25T00:37:21.330 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:37:21.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7ebac640 1 Processor -- start 2024-04-25T00:37:21.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7ebac640 1 -- start start 2024-04-25T00:37:21.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7ebac640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2c7815e3c0 0x7f2c780d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7ebac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7806d320 0x7f2c780cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7ebac640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c7815e7a0 con 0x7f2c7806c530 2024-04-25T00:37:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7ebac640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2c780b4bc0 con 0x7f2c7815e3c0 2024-04-25T00:37:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7ebac640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2c780d10f0 con 0x7f2c7806d320 2024-04-25T00:37:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7c948640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2c7815e3c0 0x7f2c780d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c77fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2c7806c530 0x7f2c7806cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42738/0 (socket says 172.21.15.29:42738) 2024-04-25T00:37:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c77fff640 1 -- 172.21.15.29:0/3138968125 learned_addr learned my addr 172.21.15.29:0/3138968125 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7d149640 1 --2- 172.21.15.29:0/3138968125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7806d320 0x7f2c780cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7c948640 1 -- 172.21.15.29:0/3138968125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7806d320 msgr2=0x7f2c780cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7c948640 1 --2- 172.21.15.29:0/3138968125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7806d320 0x7f2c780cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7c948640 1 -- 172.21.15.29:0/3138968125 >> v1:172.21.15.29:6789/0 conn(0x7f2c7806c530 legacy=0x7f2c7806cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7c948640 1 -- 172.21.15.29:0/3138968125 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c780cc560 con 0x7f2c7815e3c0 2024-04-25T00:37:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.439+0000 7f2c7d149640 1 --2- 172.21.15.29:0/3138968125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7806d320 0x7f2c780cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7c948640 1 --2- 172.21.15.29:0/3138968125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7815e3c0 0x7f2c780d0a80 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f2c680097c0 tx=0x7f2c6801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b0edf8bcee4979 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c777fe640 1 -- 172.21.15.29:0/3138968125 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c680b2070 con 0x7f2c7815e3c0 2024-04-25T00:37:21.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c777fe640 1 -- 172.21.15.29:0/3138968125 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c680ad640 con 0x7f2c7815e3c0 2024-04-25T00:37:21.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c777fe640 1 -- 172.21.15.29:0/3138968125 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c680ada10 con 0x7f2c7815e3c0 2024-04-25T00:37:21.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 -- 172.21.15.29:0/3138968125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7815e3c0 msgr2=0x7f2c780d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 --2- 172.21.15.29:0/3138968125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7815e3c0 0x7f2c780d0a80 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f2c680097c0 tx=0x7f2c6801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 -- 172.21.15.29:0/3138968125 shutdown_connections 2024-04-25T00:37:21.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 --2- 172.21.15.29:0/3138968125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7806d320 0x7f2c780cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 --2- 172.21.15.29:0/3138968125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7815e3c0 0x7f2c780d0a80 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 -- 172.21.15.29:0/3138968125 >> 172.21.15.29:0/3138968125 conn(0x7f2c780d1d70 msgr2=0x7f2c780d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:21.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 -- 172.21.15.29:0/3138968125 shutdown_connections 2024-04-25T00:37:21.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 -- 172.21.15.29:0/3138968125 wait complete. 2024-04-25T00:37:21.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 Processor -- start 2024-04-25T00:37:21.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 -- start start 2024-04-25T00:37:21.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7806c530 0x7f2c78153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7806d320 0x7f2c7815bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7815e3c0 0x7f2c7815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2c780cdf10 con 0x7f2c7815e3c0 2024-04-25T00:37:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2c780cdd90 con 0x7f2c7806d320 2024-04-25T00:37:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7ebac640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2c780ce090 con 0x7f2c7806c530 2024-04-25T00:37:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7d149640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7815e3c0 0x7f2c7815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7d149640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7815e3c0 0x7f2c7815c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38204/0 (socket says 172.21.15.29:38204) 2024-04-25T00:37:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7d149640 1 -- 172.21.15.29:0/2493930912 learned_addr learned my addr 172.21.15.29:0/2493930912 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7d149640 1 -- 172.21.15.29:0/2493930912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7806c530 msgr2=0x7f2c78153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7d149640 1 --2- 172.21.15.29:0/2493930912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7806c530 0x7f2c78153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7d149640 1 -- 172.21.15.29:0/2493930912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7806d320 msgr2=0x7f2c7815bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7d149640 1 --2- 172.21.15.29:0/2493930912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7806d320 0x7f2c7815bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.443+0000 7f2c7d149640 1 -- 172.21.15.29:0/2493930912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c78155880 con 0x7f2c7815e3c0 2024-04-25T00:37:21.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.447+0000 7f2c7d149640 1 --2- 172.21.15.29:0/2493930912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7815e3c0 0x7f2c7815c2a0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000cdb0 tx=0x7f2c7000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:21.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.447+0000 7f2c757fa640 1 -- 172.21.15.29:0/2493930912 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7000f680 con 0x7f2c7815e3c0 2024-04-25T00:37:21.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.447+0000 7f2c757fa640 1 -- 172.21.15.29:0/2493930912 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c7000f840 con 0x7f2c7815e3c0 2024-04-25T00:37:21.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.447+0000 7f2c757fa640 1 -- 172.21.15.29:0/2493930912 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7000fc30 con 0x7f2c7815e3c0 2024-04-25T00:37:21.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.447+0000 7f2c7ebac640 1 -- 172.21.15.29:0/2493930912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c78155b60 con 0x7f2c7815e3c0 2024-04-25T00:37:21.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.447+0000 7f2c7ebac640 1 -- 172.21.15.29:0/2493930912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c78156120 con 0x7f2c7815e3c0 2024-04-25T00:37:21.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.447+0000 7f2c7ebac640 1 -- 172.21.15.29:0/2493930912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c78069270 con 0x7f2c7815e3c0 2024-04-25T00:37:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.447+0000 7f2c757fa640 1 -- 172.21.15.29:0/2493930912 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f2c7000e3e0 con 0x7f2c7815e3c0 2024-04-25T00:37:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.447+0000 7f2c757fa640 1 --2- 172.21.15.29:0/2493930912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c640679e0 0x7f2c64069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.447+0000 7f2c757fa640 1 -- 172.21.15.29:0/2493930912 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f2c700869c0 con 0x7f2c7815e3c0 2024-04-25T00:37:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.447+0000 7f2c7c948640 1 --2- 172.21.15.29:0/2493930912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c640679e0 0x7f2c64069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.447+0000 7f2c7c948640 1 --2- 172.21.15.29:0/2493930912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c640679e0 0x7f2c64069ea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2c680af000 tx=0x7f2c68004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:21.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.451+0000 7f2c757fa640 1 -- 172.21.15.29:0/2493930912 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2c70014030 con 0x7f2c7815e3c0 2024-04-25T00:37:21.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c7ebac640 1 -- 172.21.15.29:0/2493930912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c780b78c0 con 0x7f2c7815e3c0 2024-04-25T00:37:21.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c757fa640 1 -- 172.21.15.29:0/2493930912 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+12844 (secure 0 0 0) 0x7f2c70057620 con 0x7f2c7815e3c0 2024-04-25T00:37:21.613 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:21.613 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":99,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:37:20.925861+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:37:21.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c7ebac640 1 -- 172.21.15.29:0/2493930912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c640679e0 msgr2=0x7f2c64069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c7ebac640 1 --2- 172.21.15.29:0/2493930912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c640679e0 0x7f2c64069ea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2c680af000 tx=0x7f2c68004740 comp rx=0 tx=0).stop 2024-04-25T00:37:21.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c7ebac640 1 -- 172.21.15.29:0/2493930912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7815e3c0 msgr2=0x7f2c7815c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c7ebac640 1 --2- 172.21.15.29:0/2493930912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7815e3c0 0x7f2c7815c2a0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000cdb0 tx=0x7f2c7000a650 comp rx=0 tx=0).stop 2024-04-25T00:37:21.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c7ebac640 1 -- 172.21.15.29:0/2493930912 shutdown_connections 2024-04-25T00:37:21.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c7ebac640 1 --2- 172.21.15.29:0/2493930912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c640679e0 0x7f2c64069ea0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c7ebac640 1 --2- 172.21.15.29:0/2493930912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c7815e3c0 0x7f2c7815c2a0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c7ebac640 1 --2- 172.21.15.29:0/2493930912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7806d320 0x7f2c7815bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c7ebac640 1 --2- 172.21.15.29:0/2493930912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7806c530 0x7f2c78153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c7ebac640 1 -- 172.21.15.29:0/2493930912 >> 172.21.15.29:0/2493930912 conn(0x7f2c780d1d70 msgr2=0x7f2c7806f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:21.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c7ebac640 1 -- 172.21.15.29:0/2493930912 shutdown_connections 2024-04-25T00:37:21.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.611+0000 7f2c7ebac640 1 -- 172.21.15.29:0/2493930912 wait complete. 2024-04-25T00:37:21.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:37:21.631 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:37:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.743+0000 7f9ddcb55640 1 Processor -- start 2024-04-25T00:37:21.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.743+0000 7f9ddcb55640 1 -- start start 2024-04-25T00:37:21.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9ddcb55640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd806c950 0x7f9dd806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9ddcb55640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9dd815e440 0x7f9dd80d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9ddcb55640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dd80cc3b0 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9ddcb55640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd806d270 con 0x7f9dd815e440 2024-04-25T00:37:21.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9ddcb55640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd80b11d0 con 0x7f9dd806c950 2024-04-25T00:37:21.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd5d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9dd815e440 0x7f9dd80d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd5d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9dd815e440 0x7f9dd80d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38206/0 (socket says 172.21.15.29:38206) 2024-04-25T00:37:21.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd5d9b640 1 -- 172.21.15.29:0/2026483148 learned_addr learned my addr 172.21.15.29:0/2026483148 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:21.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd6d9d640 1 --1- 172.21.15.29:0/2026483148 >> v1:172.21.15.29:6789/0 conn(0x7f9dd80d0ff0 0x7f9dd80ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42746/0 (socket says 172.21.15.29:42746) 2024-04-25T00:37:21.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd659c640 1 --2- 172.21.15.29:0/2026483148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd806c950 0x7f9dd806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd5d9b640 1 -- 172.21.15.29:0/2026483148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd806c950 msgr2=0x7f9dd806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:21.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd5d9b640 1 --2- 172.21.15.29:0/2026483148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd806c950 0x7f9dd806cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd5d9b640 1 -- 172.21.15.29:0/2026483148 >> v1:172.21.15.29:6789/0 conn(0x7f9dd80d0ff0 legacy=0x7f9dd80ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd5d9b640 1 -- 172.21.15.29:0/2026483148 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd80cc650 con 0x7f9dd815e440 2024-04-25T00:37:21.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd559a640 1 -- 172.21.15.29:0/2026483148 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2402609585 0 0) 0x7f9dd80cc3b0 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd659c640 1 --2- 172.21.15.29:0/2026483148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd806c950 0x7f9dd806cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:21.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd5d9b640 1 --2- 172.21.15.29:0/2026483148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd815e440 0x7f9dd80d0ab0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009870 tx=0x7f9dc801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff79c622a99b074f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:21.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd559a640 1 -- 172.21.15.29:0/2026483148 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80b0070 con 0x7f9dd815e440 2024-04-25T00:37:21.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd559a640 1 -- 172.21.15.29:0/2026483148 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc80ab600 con 0x7f9dd815e440 2024-04-25T00:37:21.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.747+0000 7f9dd559a640 1 -- 172.21.15.29:0/2026483148 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80ab9d0 con 0x7f9dd815e440 2024-04-25T00:37:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 -- 172.21.15.29:0/2026483148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd815e440 msgr2=0x7f9dd80d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 --2- 172.21.15.29:0/2026483148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd815e440 0x7f9dd80d0ab0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009870 tx=0x7f9dc801c560 comp rx=0 tx=0).stop 2024-04-25T00:37:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 -- 172.21.15.29:0/2026483148 shutdown_connections 2024-04-25T00:37:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 --2- 172.21.15.29:0/2026483148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd815e440 0x7f9dd80d0ab0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 --2- 172.21.15.29:0/2026483148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd806c950 0x7f9dd806cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 -- 172.21.15.29:0/2026483148 >> 172.21.15.29:0/2026483148 conn(0x7f9dd80d1d60 msgr2=0x7f9dd80d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 -- 172.21.15.29:0/2026483148 shutdown_connections 2024-04-25T00:37:21.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 -- 172.21.15.29:0/2026483148 wait complete. 2024-04-25T00:37:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 Processor -- start 2024-04-25T00:37:21.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 -- start start 2024-04-25T00:37:21.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd806c950 0x7f9dd80cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd80d0ff0 0x7f9dd81465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd815e440 0x7f9dd8145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd806c520 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd80ce420 con 0x7f9dd815e440 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9ddcb55640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd816ae20 con 0x7f9dd806c950 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9dd5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd80d0ff0 0x7f9dd81465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9dd5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd80d0ff0 0x7f9dd81465a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38210/0 (socket says 172.21.15.29:38210) 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9dd5d9b640 1 -- 172.21.15.29:0/1359927031 learned_addr learned my addr 172.21.15.29:0/1359927031 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9dd5d9b640 1 -- 172.21.15.29:0/1359927031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd806c950 msgr2=0x7f9dd80cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9dd659c640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd806c950 0x7f9dd80cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9dd5d9b640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd806c950 0x7f9dd80cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9dd5d9b640 1 -- 172.21.15.29:0/1359927031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd815e440 msgr2=0x7f9dd8145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9dd6d9d640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd815e440 0x7f9dd8145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9dd5d9b640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd815e440 0x7f9dd8145d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9dd5d9b640 1 -- 172.21.15.29:0/1359927031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd815b390 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.751+0000 7f9dd5d9b640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd80d0ff0 0x7f9dd81465a0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80ae040 tx=0x7f9dc801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:21.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.755+0000 7f9dc6ffd640 1 -- 172.21.15.29:0/1359927031 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80b0040 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.755+0000 7f9dc6ffd640 1 -- 172.21.15.29:0/1359927031 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc80bc070 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.755+0000 7f9ddcb55640 1 -- 172.21.15.29:0/1359927031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd815b670 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.755+0000 7f9dc6ffd640 1 -- 172.21.15.29:0/1359927031 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80b6880 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.755+0000 7f9ddcb55640 1 -- 172.21.15.29:0/1359927031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd81462d0 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.755+0000 7f9ddcb55640 1 -- 172.21.15.29:0/1359927031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd8069270 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.755+0000 7f9dc6ffd640 1 -- 172.21.15.29:0/1359927031 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f9dc80b6a20 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.755+0000 7f9dc6ffd640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9db8067a10 0x7f9db8069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:21.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.755+0000 7f9dc6ffd640 1 -- 172.21.15.29:0/1359927031 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f9dc8120b40 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.755+0000 7f9dd659c640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9db8067a10 0x7f9db8069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:21.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.755+0000 7f9dd659c640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9db8067a10 0x7f9db8069ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc006fd0 tx=0x7f9dcc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:21.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.759+0000 7f9dc6ffd640 1 -- 172.21.15.29:0/1359927031 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9dc8016150 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.923+0000 7f9ddcb55640 1 -- 172.21.15.29:0/1359927031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9dd8075130 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.927+0000 7f9dc6ffd640 1 -- 172.21.15.29:0/1359927031 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f9dc80f1790 con 0x7f9dd80d0ff0 2024-04-25T00:37:21.928 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:37:21.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.927+0000 7f9ddcb55640 1 -- 172.21.15.29:0/1359927031 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9db8067a10 msgr2=0x7f9db8069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.927+0000 7f9ddcb55640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9db8067a10 0x7f9db8069ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc006fd0 tx=0x7f9dcc008040 comp rx=0 tx=0).stop 2024-04-25T00:37:21.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.927+0000 7f9ddcb55640 1 -- 172.21.15.29:0/1359927031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd80d0ff0 msgr2=0x7f9dd81465a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:21.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.927+0000 7f9ddcb55640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd80d0ff0 0x7f9dd81465a0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80ae040 tx=0x7f9dc801ada0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.927+0000 7f9ddcb55640 1 -- 172.21.15.29:0/1359927031 shutdown_connections 2024-04-25T00:37:21.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.927+0000 7f9ddcb55640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9db8067a10 0x7f9db8069ed0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.927+0000 7f9ddcb55640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd815e440 0x7f9dd8145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.927+0000 7f9ddcb55640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dd80d0ff0 0x7f9dd81465a0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.927+0000 7f9ddcb55640 1 --2- 172.21.15.29:0/1359927031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd806c950 0x7f9dd80cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:21.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.927+0000 7f9ddcb55640 1 -- 172.21.15.29:0/1359927031 >> 172.21.15.29:0/1359927031 conn(0x7f9dd80d1d60 msgr2=0x7f9dd80b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:21.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.927+0000 7f9ddcb55640 1 -- 172.21.15.29:0/1359927031 shutdown_connections 2024-04-25T00:37:21.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:21.927+0000 7f9ddcb55640 1 -- 172.21.15.29:0/1359927031 wait complete. 2024-04-25T00:37:21.947 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:37:22.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee72c1640 1 Processor -- start 2024-04-25T00:37:22.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee72c1640 1 -- start start 2024-04-25T00:37:22.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee72c1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ee015e1b0 0x7f5ee00cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee72c1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee00cb7e0 0x7f5ee006f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee72c1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ee00d01d0 con 0x7f5ee0071fa0 2024-04-25T00:37:22.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee72c1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5ee00d0a70 con 0x7f5ee015e1b0 2024-04-25T00:37:22.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee72c1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee015e830 con 0x7f5ee00cb7e0 2024-04-25T00:37:22.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee485c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5ee0071fa0 0x7f5ee015ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42762/0 (socket says 172.21.15.29:42762) 2024-04-25T00:37:22.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee485c640 1 -- 172.21.15.29:0/3762252847 learned_addr learned my addr 172.21.15.29:0/3762252847 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:22.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee505d640 1 --2- 172.21.15.29:0/3762252847 >> v2:172.21.15.157:3300/0 conn(0x7f5ee015e1b0 0x7f5ee00cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee585e640 1 --2- 172.21.15.29:0/3762252847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee00cb7e0 0x7f5ee006f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2919273260 0 0) 0x7f5ee00d01d0 con 0x7f5ee0071fa0 2024-04-25T00:37:22.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ec8003530 con 0x7f5ee0071fa0 2024-04-25T00:37:22.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4066720977 0 0) 0x7f5ec8003530 con 0x7f5ee0071fa0 2024-04-25T00:37:22.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee00cb7e0 msgr2=0x7f5ee006f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:22.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 --2- 172.21.15.29:0/3762252847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee00cb7e0 0x7f5ee006f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 >> v2:172.21.15.157:3300/0 conn(0x7f5ee015e1b0 msgr2=0x7f5ee00cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:22.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 --2- 172.21.15.29:0/3762252847 >> v2:172.21.15.157:3300/0 conn(0x7f5ee015e1b0 0x7f5ee00cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee00d04b0 con 0x7f5ee0071fa0 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f5ed4001680 con 0x7f5ee0071fa0 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 >> v1:172.21.15.29:6789/0 conn(0x7f5ee0071fa0 legacy=0x7f5ee015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 --2- 172.21.15.29:0/3762252847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ec80039c0 0x7f5ec8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 --2- 172.21.15.29:0/3762252847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec8004810 0x7f5ec8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 --2- 172.21.15.29:0/3762252847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee00cb7e0 0x7f5ec80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee00d0a70 con 0x7f5ec80039c0 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee015e830 con 0x7f5ec8004810 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec800b9d0 con 0x7f5ee00cb7e0 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee505d640 1 --2- 172.21.15.29:0/3762252847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ec80039c0 0x7f5ec8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee585e640 1 --2- 172.21.15.29:0/3762252847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee00cb7e0 0x7f5ec80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee505d640 1 -- 172.21.15.29:0/3762252847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee00cb7e0 msgr2=0x7f5ec80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee505d640 1 --2- 172.21.15.29:0/3762252847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ee00cb7e0 0x7f5ec80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee505d640 1 -- 172.21.15.29:0/3762252847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec8004810 msgr2=0x7f5ec8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:22.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee505d640 1 --2- 172.21.15.29:0/3762252847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec8004810 0x7f5ec8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.071+0000 7f5ee505d640 1 -- 172.21.15.29:0/3762252847 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec8003530 con 0x7f5ec80039c0 2024-04-25T00:37:22.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.075+0000 7f5ee505d640 1 --2- 172.21.15.29:0/3762252847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ec80039c0 0x7f5ec8003de0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009ae0 tx=0x7f5ed001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de26d61c16cdee1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.075+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed00096b0 con 0x7f5ec80039c0 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.075+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed00a64c0 con 0x7f5ec80039c0 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.075+0000 7f5edffff640 1 -- 172.21.15.29:0/3762252847 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed00a68b0 con 0x7f5ec80039c0 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.075+0000 7f5ee72c1640 1 -- 172.21.15.29:0/3762252847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ec80039c0 msgr2=0x7f5ec8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.075+0000 7f5ee72c1640 1 --2- 172.21.15.29:0/3762252847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ec80039c0 0x7f5ec8003de0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009ae0 tx=0x7f5ed001cab0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.075+0000 7f5ee585e640 1 -- 172.21.15.29:0/3762252847 reap_dead start 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.075+0000 7f5ee72c1640 1 -- 172.21.15.29:0/3762252847 shutdown_connections 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.075+0000 7f5ee72c1640 1 -- 172.21.15.29:0/3762252847 >> 172.21.15.29:0/3762252847 conn(0x7f5ee00b2920 msgr2=0x7f5ee00b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.075+0000 7f5ee72c1640 1 -- 172.21.15.29:0/3762252847 shutdown_connections 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.075+0000 7f5ee72c1640 1 -- 172.21.15.29:0/3762252847 wait complete. 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.075+0000 7f5ee72c1640 1 Processor -- start 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.075+0000 7f5ee72c1640 1 -- start start 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee72c1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80039c0 0x7f5ee00b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee72c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee0157530 0x7f5ee0155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee72c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ee0155e60 0x7f5ee01522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee72c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee0168db0 con 0x7f5ee0155e60 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee72c1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee0168c30 con 0x7f5ee0157530 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee72c1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee0168f30 con 0x7f5ec80039c0 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee585e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ee0155e60 0x7f5ee01522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee585e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ee0155e60 0x7f5ee01522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38236/0 (socket says 172.21.15.29:38236) 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee585e640 1 -- 172.21.15.29:0/1893691780 learned_addr learned my addr 172.21.15.29:0/1893691780 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee585e640 1 -- 172.21.15.29:0/1893691780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80039c0 msgr2=0x7f5ee00b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee585e640 1 --2- 172.21.15.29:0/1893691780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80039c0 0x7f5ee00b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee585e640 1 -- 172.21.15.29:0/1893691780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee0157530 msgr2=0x7f5ee0155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee585e640 1 --2- 172.21.15.29:0/1893691780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee0157530 0x7f5ee0155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee585e640 1 -- 172.21.15.29:0/1893691780 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee01556d0 con 0x7f5ee0155e60 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee585e640 1 --2- 172.21.15.29:0/1893691780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ee0155e60 0x7f5ee01522a0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0071c80 tx=0x7f5ed80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:22.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5eddffb640 1 -- 172.21.15.29:0/1893691780 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8011e10 con 0x7f5ee0155e60 2024-04-25T00:37:22.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5eddffb640 1 -- 172.21.15.29:0/1893691780 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed801b070 con 0x7f5ee0155e60 2024-04-25T00:37:22.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee72c1640 1 -- 172.21.15.29:0/1893691780 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee0156490 con 0x7f5ee0155e60 2024-04-25T00:37:22.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5eddffb640 1 -- 172.21.15.29:0/1893691780 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed80039a0 con 0x7f5ee0155e60 2024-04-25T00:37:22.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee72c1640 1 -- 172.21.15.29:0/1893691780 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee0156a50 con 0x7f5ee0155e60 2024-04-25T00:37:22.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee72c1640 1 -- 172.21.15.29:0/1893691780 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee0069270 con 0x7f5ee0155e60 2024-04-25T00:37:22.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5eddffb640 1 -- 172.21.15.29:0/1893691780 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f5ed8003b60 con 0x7f5ee0155e60 2024-04-25T00:37:22.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5eddffb640 1 --2- 172.21.15.29:0/1893691780 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5ecc067a10 0x7f5ecc069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5eddffb640 1 -- 172.21.15.29:0/1893691780 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f5ed80862f0 con 0x7f5ee0155e60 2024-04-25T00:37:22.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee505d640 1 --2- 172.21.15.29:0/1893691780 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5ecc067a10 0x7f5ecc069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.079+0000 7f5ee505d640 1 --2- 172.21.15.29:0/1893691780 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5ecc067a10 0x7f5ecc069ed0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00af000 tx=0x7f5ed0004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:22.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.083+0000 7f5eddffb640 1 -- 172.21.15.29:0/1893691780 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5ed8056ee0 con 0x7f5ee0155e60 2024-04-25T00:37:22.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.243+0000 7f5ee72c1640 1 -- 172.21.15.29:0/1893691780 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5ee015dc60 con 0x7f5ee0155e60 2024-04-25T00:37:22.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.243+0000 7f5eddffb640 1 -- 172.21.15.29:0/1893691780 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) ==== 84+0+65 (secure 0 0 0) 0x7f5ed80567e0 con 0x7f5ee0155e60 2024-04-25T00:37:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:22.248 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:37:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-04-25T00:37:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": ".mgr" 2024-04-25T00:37:22.248 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:22.248 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:37:22.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.247+0000 7f5ee72c1640 1 -- 172.21.15.29:0/1893691780 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5ecc067a10 msgr2=0x7f5ecc069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:22.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.247+0000 7f5ee72c1640 1 --2- 172.21.15.29:0/1893691780 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5ecc067a10 0x7f5ecc069ed0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00af000 tx=0x7f5ed0004740 comp rx=0 tx=0).stop 2024-04-25T00:37:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.247+0000 7f5ee72c1640 1 -- 172.21.15.29:0/1893691780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ee0155e60 msgr2=0x7f5ee01522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.247+0000 7f5ee72c1640 1 --2- 172.21.15.29:0/1893691780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ee0155e60 0x7f5ee01522a0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0071c80 tx=0x7f5ed80115a0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.247+0000 7f5ee72c1640 1 -- 172.21.15.29:0/1893691780 shutdown_connections 2024-04-25T00:37:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.247+0000 7f5ee72c1640 1 --2- 172.21.15.29:0/1893691780 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5ecc067a10 0x7f5ecc069ed0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.247+0000 7f5ee72c1640 1 --2- 172.21.15.29:0/1893691780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ee0155e60 0x7f5ee01522a0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.247+0000 7f5ee72c1640 1 --2- 172.21.15.29:0/1893691780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee0157530 0x7f5ee0155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.247+0000 7f5ee72c1640 1 --2- 172.21.15.29:0/1893691780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80039c0 0x7f5ee00b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.247+0000 7f5ee72c1640 1 -- 172.21.15.29:0/1893691780 >> 172.21.15.29:0/1893691780 conn(0x7f5ee00b2920 msgr2=0x7f5ee0160310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:22.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.247+0000 7f5ee72c1640 1 -- 172.21.15.29:0/1893691780 shutdown_connections 2024-04-25T00:37:22.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.247+0000 7f5ee72c1640 1 -- 172.21.15.29:0/1893691780 wait complete. 2024-04-25T00:37:22.264 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:37:22.265 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:37:22.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f4342373640 1 Processor -- start 2024-04-25T00:37:22.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f4342373640 1 -- start start 2024-04-25T00:37:22.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f4342373640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f433c0b4590 0x7f433c070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f4342373640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433c0712c0 con 0x7f433c15e9a0 2024-04-25T00:37:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f4342373640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433c0715a0 con 0x7f433c0cd1c0 2024-04-25T00:37:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f4342373640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f433c071ed0 con 0x7f433c0b4590 2024-04-25T00:37:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f433c15e9a0 0x7f433c15ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42766/0 (socket says 172.21.15.29:42766) 2024-04-25T00:37:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433bfff640 1 -- 172.21.15.29:0/1397116064 learned_addr learned my addr 172.21.15.29:0/1397116064 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f43337fe640 1 --2- 172.21.15.29:0/1397116064 >> v2:172.21.15.157:3300/0 conn(0x7f433c0b4590 0x7f433c070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4231839906 0 0) 0x7f433c0712c0 con 0x7f433c15e9a0 2024-04-25T00:37:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4328003530 con 0x7f433c15e9a0 2024-04-25T00:37:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3079991729 0 0) 0x7f4328003530 con 0x7f433c15e9a0 2024-04-25T00:37:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 >> v2:172.21.15.157:3300/0 conn(0x7f433c0b4590 msgr2=0x7f433c070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 --2- 172.21.15.29:0/1397116064 >> v2:172.21.15.157:3300/0 conn(0x7f433c0b4590 0x7f433c070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 >> v1:172.21.15.157:6789/0 conn(0x7f433c0cd1c0 legacy=0x7f433c072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c071880 con 0x7f433c15e9a0 2024-04-25T00:37:22.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f432c001680 con 0x7f433c15e9a0 2024-04-25T00:37:22.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f43337fe640 1 --2- 172.21.15.29:0/1397116064 >> v2:172.21.15.157:3300/0 conn(0x7f433c0b4590 0x7f433c070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:22.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 >> v1:172.21.15.29:6789/0 conn(0x7f433c15e9a0 legacy=0x7f433c15ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:22.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 --2- 172.21.15.29:0/1397116064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43280039c0 0x7f4328003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 --2- 172.21.15.29:0/1397116064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4328004810 0x7f4328004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 --2- 172.21.15.29:0/1397116064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43280051d0 0x7f432800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f433c071ed0 con 0x7f4328004810 2024-04-25T00:37:22.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f432800c120 con 0x7f43280051d0 2024-04-25T00:37:22.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f432800c320 con 0x7f43280039c0 2024-04-25T00:37:22.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.383+0000 7f43337fe640 1 --2- 172.21.15.29:0/1397116064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4328004810 0x7f4328004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f433bfff640 1 --2- 172.21.15.29:0/1397116064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43280039c0 0x7f4328003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f4340910640 1 --2- 172.21.15.29:0/1397116064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43280051d0 0x7f432800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f43337fe640 1 -- 172.21.15.29:0/1397116064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43280039c0 msgr2=0x7f4328003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:22.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f43337fe640 1 --2- 172.21.15.29:0/1397116064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43280039c0 0x7f4328003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f43337fe640 1 -- 172.21.15.29:0/1397116064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43280051d0 msgr2=0x7f432800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:22.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f43337fe640 1 --2- 172.21.15.29:0/1397116064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43280051d0 0x7f432800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f4340910640 1 -- 172.21.15.29:0/1397116064 reap_dead start 2024-04-25T00:37:22.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f43337fe640 1 -- 172.21.15.29:0/1397116064 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4328003530 con 0x7f4328004810 2024-04-25T00:37:22.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f43337fe640 1 --2- 172.21.15.29:0/1397116064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4328004810 0x7f4328004c90 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f43240098d0 tx=0x7f432401c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b632bf5aa29f4db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:22.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432400f040 con 0x7f4328004810 2024-04-25T00:37:22.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43240af070 con 0x7f4328004810 2024-04-25T00:37:22.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f433b7fe640 1 -- 172.21.15.29:0/1397116064 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43240aa870 con 0x7f4328004810 2024-04-25T00:37:22.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f4342373640 1 -- 172.21.15.29:0/1397116064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4328004810 msgr2=0x7f4328004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:22.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f4342373640 1 --2- 172.21.15.29:0/1397116064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4328004810 0x7f4328004c90 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f43240098d0 tx=0x7f432401c5d0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f4342373640 1 -- 172.21.15.29:0/1397116064 shutdown_connections 2024-04-25T00:37:22.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f4342373640 1 --2- 172.21.15.29:0/1397116064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4328004810 0x7f4328004c90 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f4342373640 1 -- 172.21.15.29:0/1397116064 >> 172.21.15.29:0/1397116064 conn(0x7f433c0b1870 msgr2=0x7f433c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:22.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f4342373640 1 -- 172.21.15.29:0/1397116064 shutdown_connections 2024-04-25T00:37:22.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.387+0000 7f4342373640 1 -- 172.21.15.29:0/1397116064 wait complete. 2024-04-25T00:37:22.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f4342373640 1 Processor -- start 2024-04-25T00:37:22.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f4342373640 1 -- start start 2024-04-25T00:37:22.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f4342373640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4328004810 0x7f433c15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f4342373640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c138950 0x7f433c0ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f4342373640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433c138dd0 0x7f433c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f4342373640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f433c0cd630 con 0x7f4328004810 2024-04-25T00:37:22.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f4342373640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f433c15f700 con 0x7f433c138dd0 2024-04-25T00:37:22.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f4342373640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f433c0cd7b0 con 0x7f433c138950 2024-04-25T00:37:22.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f433bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4328004810 0x7f433c15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f433bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4328004810 0x7f433c15a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38268/0 (socket says 172.21.15.29:38268) 2024-04-25T00:37:22.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f433bfff640 1 -- 172.21.15.29:0/271707030 learned_addr learned my addr 172.21.15.29:0/271707030 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f43337fe640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c138950 0x7f433c0ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f4340910640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433c138dd0 0x7f433c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f433bfff640 1 -- 172.21.15.29:0/271707030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c138950 msgr2=0x7f433c0ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f433bfff640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c138950 0x7f433c0ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f433bfff640 1 -- 172.21.15.29:0/271707030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433c138dd0 msgr2=0x7f433c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f433bfff640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433c138dd0 0x7f433c0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f433bfff640 1 -- 172.21.15.29:0/271707030 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c136940 con 0x7f4328004810 2024-04-25T00:37:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f43337fe640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c138950 0x7f433c0ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f433bfff640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4328004810 0x7f433c15a5a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f432c011980 tx=0x7f432c011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f43397fa640 1 -- 172.21.15.29:0/271707030 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c010040 con 0x7f4328004810 2024-04-25T00:37:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f43397fa640 1 -- 172.21.15.29:0/271707030 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f432c01a070 con 0x7f4328004810 2024-04-25T00:37:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f4342373640 1 -- 172.21.15.29:0/271707030 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433c136c20 con 0x7f4328004810 2024-04-25T00:37:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f43397fa640 1 -- 172.21.15.29:0/271707030 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c00fde0 con 0x7f4328004810 2024-04-25T00:37:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f4342373640 1 -- 172.21.15.29:0/271707030 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433c1371e0 con 0x7f4328004810 2024-04-25T00:37:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f4342373640 1 -- 172.21.15.29:0/271707030 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433c0692e0 con 0x7f4328004810 2024-04-25T00:37:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.391+0000 7f43397fa640 1 -- 172.21.15.29:0/271707030 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f432c01f460 con 0x7f4328004810 2024-04-25T00:37:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.395+0000 7f43397fa640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f431c0679e0 0x7f431c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:22.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.395+0000 7f43397fa640 1 -- 172.21.15.29:0/271707030 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f432c088e00 con 0x7f4328004810 2024-04-25T00:37:22.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.395+0000 7f43337fe640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f431c0679e0 0x7f431c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:22.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.395+0000 7f43337fe640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f431c0679e0 0x7f431c069ea0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f43240ae000 tx=0x7f43240046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:22.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.399+0000 7f43397fa640 1 -- 172.21.15.29:0/271707030 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f432c059af0 con 0x7f4328004810 2024-04-25T00:37:22.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.555+0000 7f4342373640 1 -- 172.21.15.29:0/271707030 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f433c0b3260 con 0x7f4328004810 2024-04-25T00:37:22.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.931+0000 7f43397fa640 1 -- 172.21.15.29:0/271707030 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) ==== 138+0+0 (secure 0 0 0) 0x7f432c0593f0 con 0x7f4328004810 2024-04-25T00:37:22.932 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-04-25T00:37:22.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.931+0000 7f4342373640 1 -- 172.21.15.29:0/271707030 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f431c0679e0 msgr2=0x7f431c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:22.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.931+0000 7f4342373640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f431c0679e0 0x7f431c069ea0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f43240ae000 tx=0x7f43240046d0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.931+0000 7f4342373640 1 -- 172.21.15.29:0/271707030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4328004810 msgr2=0x7f433c15a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:22.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.931+0000 7f4342373640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4328004810 0x7f433c15a5a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f432c011980 tx=0x7f432c011e40 comp rx=0 tx=0).stop 2024-04-25T00:37:22.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.931+0000 7f4342373640 1 -- 172.21.15.29:0/271707030 shutdown_connections 2024-04-25T00:37:22.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.931+0000 7f4342373640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f431c0679e0 0x7f431c069ea0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.931+0000 7f4342373640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433c138dd0 0x7f433c0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.931+0000 7f4342373640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f433c138950 0x7f433c0ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.931+0000 7f4342373640 1 --2- 172.21.15.29:0/271707030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4328004810 0x7f433c15a5a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:22.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.931+0000 7f4342373640 1 -- 172.21.15.29:0/271707030 >> 172.21.15.29:0/271707030 conn(0x7f433c0b1870 msgr2=0x7f433c142980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:22.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.931+0000 7f4342373640 1 -- 172.21.15.29:0/271707030 shutdown_connections 2024-04-25T00:37:22.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:22.931+0000 7f4342373640 1 -- 172.21.15.29:0/271707030 wait complete. 2024-04-25T00:37:22.961 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:37:23.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb71a833640 1 Processor -- start 2024-04-25T00:37:23.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb71a833640 1 -- start start 2024-04-25T00:37:23.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb71a833640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7140b3c30 0x7fb71406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:23.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb71a833640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7140b41e0 con 0x7fb7140b4b10 2024-04-25T00:37:23.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb71a833640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb71406d960 con 0x7fb7140b3140 2024-04-25T00:37:23.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb71a833640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb71406dc40 con 0x7fb7140b3c30 2024-04-25T00:37:23.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb718dd0640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb7140b4b10 0x7fb7140ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42780/0 (socket says 172.21.15.29:42780) 2024-04-25T00:37:23.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb718dd0640 1 -- 172.21.15.29:0/4268847805 learned_addr learned my addr 172.21.15.29:0/4268847805 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:23.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb7137fe640 1 --2- 172.21.15.29:0/4268847805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7140b3c30 0x7fb71406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:23.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399367303 0 0) 0x7fb7140b41e0 con 0x7fb7140b4b10 2024-04-25T00:37:23.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6fc003530 con 0x7fb7140b4b10 2024-04-25T00:37:23.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2099144543 0 0) 0x7fb6fc003530 con 0x7fb7140b4b10 2024-04-25T00:37:23.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7140b3c30 msgr2=0x7fb71406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb712ffd640 1 --2- 172.21.15.29:0/4268847805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7140b3c30 0x7fb71406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 >> v1:172.21.15.157:6789/0 conn(0x7fb7140b3140 legacy=0x7fb7140b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71406df20 con 0x7fb7140b4b10 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fb704001680 con 0x7fb7140b4b10 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.091+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 >> v1:172.21.15.29:6789/0 conn(0x7fb7140b4b10 legacy=0x7fb7140ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb712ffd640 1 --2- 172.21.15.29:0/4268847805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6fc0039f0 0x7fb6fc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb712ffd640 1 --2- 172.21.15.29:0/4268847805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc004810 0x7fb6fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb712ffd640 1 --2- 172.21.15.29:0/4268847805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7140b3c30 0x7fb6fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb71406dc40 con 0x7fb6fc0039f0 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc00bab0 con 0x7fb6fc004810 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb6fc00bcb0 con 0x7fb7140b3c30 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb713fff640 1 --2- 172.21.15.29:0/4268847805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6fc0039f0 0x7fb6fc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb7137fe640 1 --2- 172.21.15.29:0/4268847805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc004810 0x7fb6fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb718dd0640 1 --2- 172.21.15.29:0/4268847805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7140b3c30 0x7fb6fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:23.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb713fff640 1 -- 172.21.15.29:0/4268847805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7140b3c30 msgr2=0x7fb6fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:23.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb713fff640 1 --2- 172.21.15.29:0/4268847805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7140b3c30 0x7fb6fc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:23.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb713fff640 1 -- 172.21.15.29:0/4268847805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc004810 msgr2=0x7fb6fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:23.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb713fff640 1 --2- 172.21.15.29:0/4268847805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc004810 0x7fb6fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:23.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.095+0000 7fb713fff640 1 -- 172.21.15.29:0/4268847805 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc003530 con 0x7fb6fc0039f0 2024-04-25T00:37:23.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb713fff640 1 --2- 172.21.15.29:0/4268847805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6fc0039f0 0x7fb6fc003e10 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fb70c001e40 tx=0x7fb70c01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f717e2b2a9e328 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:23.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c0b0070 con 0x7fb6fc0039f0 2024-04-25T00:37:23.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb70c003b80 con 0x7fb6fc0039f0 2024-04-25T00:37:23.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb712ffd640 1 -- 172.21.15.29:0/4268847805 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c001a90 con 0x7fb6fc0039f0 2024-04-25T00:37:23.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 -- 172.21.15.29:0/4268847805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6fc0039f0 msgr2=0x7fb6fc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:23.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 --2- 172.21.15.29:0/4268847805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6fc0039f0 0x7fb6fc003e10 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fb70c001e40 tx=0x7fb70c01c740 comp rx=0 tx=0).stop 2024-04-25T00:37:23.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb718dd0640 1 -- 172.21.15.29:0/4268847805 reap_dead start 2024-04-25T00:37:23.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 -- 172.21.15.29:0/4268847805 shutdown_connections 2024-04-25T00:37:23.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 -- 172.21.15.29:0/4268847805 >> 172.21.15.29:0/4268847805 conn(0x7fb7140b1870 msgr2=0x7fb7140b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:23.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 -- 172.21.15.29:0/4268847805 shutdown_connections 2024-04-25T00:37:23.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 -- 172.21.15.29:0/4268847805 wait complete. 2024-04-25T00:37:23.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 Processor -- start 2024-04-25T00:37:23.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 -- start start 2024-04-25T00:37:23.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc0039f0 0x7fb7141558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:23.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7141393c0 0x7fb714138c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:23.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7141375c0 0x7fb714156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:23.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb714167d80 con 0x7fb7141375c0 2024-04-25T00:37:23.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb714167c00 con 0x7fb6fc0039f0 2024-04-25T00:37:23.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb71a833640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb714167f00 con 0x7fb7141393c0 2024-04-25T00:37:23.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb718dd0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7141375c0 0x7fb714156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:23.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb718dd0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7141375c0 0x7fb714156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38294/0 (socket says 172.21.15.29:38294) 2024-04-25T00:37:23.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb718dd0640 1 -- 172.21.15.29:0/868766123 learned_addr learned my addr 172.21.15.29:0/868766123 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:23.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb713fff640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc0039f0 0x7fb7141558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:23.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb7137fe640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7141393c0 0x7fb714138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:23.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb718dd0640 1 -- 172.21.15.29:0/868766123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7141393c0 msgr2=0x7fb714138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:23.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb718dd0640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7141393c0 0x7fb714138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:23.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb718dd0640 1 -- 172.21.15.29:0/868766123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc0039f0 msgr2=0x7fb7141558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:23.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb718dd0640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc0039f0 0x7fb7141558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:23.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.099+0000 7fb718dd0640 1 -- 172.21.15.29:0/868766123 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb714137eb0 con 0x7fb7141375c0 2024-04-25T00:37:23.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.103+0000 7fb718dd0640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7141375c0 0x7fb714156650 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fb7140708a0 tx=0x7fb704007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:23.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.103+0000 7fb710ff9640 1 -- 172.21.15.29:0/868766123 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70400f8b0 con 0x7fb7141375c0 2024-04-25T00:37:23.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.103+0000 7fb710ff9640 1 -- 172.21.15.29:0/868766123 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb704018070 con 0x7fb7141375c0 2024-04-25T00:37:23.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.103+0000 7fb71a833640 1 -- 172.21.15.29:0/868766123 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb714138150 con 0x7fb7141375c0 2024-04-25T00:37:23.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.103+0000 7fb710ff9640 1 -- 172.21.15.29:0/868766123 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70400fcb0 con 0x7fb7141375c0 2024-04-25T00:37:23.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.103+0000 7fb71a833640 1 -- 172.21.15.29:0/868766123 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71413f0a0 con 0x7fb7141375c0 2024-04-25T00:37:23.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.103+0000 7fb71a833640 1 -- 172.21.15.29:0/868766123 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb714069270 con 0x7fb7141375c0 2024-04-25T00:37:23.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.103+0000 7fb710ff9640 1 -- 172.21.15.29:0/868766123 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fb70401d460 con 0x7fb7141375c0 2024-04-25T00:37:23.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.103+0000 7fb710ff9640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb700067a10 0x7fb700069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:23.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.103+0000 7fb710ff9640 1 -- 172.21.15.29:0/868766123 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5227+0+0 (secure 0 0 0) 0x7fb704086550 con 0x7fb7141375c0 2024-04-25T00:37:23.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.103+0000 7fb713fff640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb700067a10 0x7fb700069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:23.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.103+0000 7fb713fff640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb700067a10 0x7fb700069ed0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0ad000 tx=0x7fb70c001520 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:23.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.107+0000 7fb710ff9640 1 -- 172.21.15.29:0/868766123 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb7040571d0 con 0x7fb7141375c0 2024-04-25T00:37:23.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.263+0000 7fb71a833640 1 -- 172.21.15.29:0/868766123 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb714067650 con 0x7fb7141375c0 2024-04-25T00:37:23.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.939+0000 7fb710ff9640 1 -- 172.21.15.29:0/868766123 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) ==== 170+0+0 (secure 0 0 0) 0x7fb704056ad0 con 0x7fb7141375c0 2024-04-25T00:37:23.941 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-04-25T00:37:23.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.943+0000 7fb71a833640 1 -- 172.21.15.29:0/868766123 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb700067a10 msgr2=0x7fb700069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:23.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.943+0000 7fb71a833640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb700067a10 0x7fb700069ed0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0ad000 tx=0x7fb70c001520 comp rx=0 tx=0).stop 2024-04-25T00:37:23.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.943+0000 7fb71a833640 1 -- 172.21.15.29:0/868766123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7141375c0 msgr2=0x7fb714156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:23.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.943+0000 7fb71a833640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7141375c0 0x7fb714156650 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fb7140708a0 tx=0x7fb704007720 comp rx=0 tx=0).stop 2024-04-25T00:37:23.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.943+0000 7fb71a833640 1 -- 172.21.15.29:0/868766123 shutdown_connections 2024-04-25T00:37:23.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.943+0000 7fb71a833640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb700067a10 0x7fb700069ed0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:23.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.943+0000 7fb71a833640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7141375c0 0x7fb714156650 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:23.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.943+0000 7fb71a833640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7141393c0 0x7fb714138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:23.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.943+0000 7fb71a833640 1 --2- 172.21.15.29:0/868766123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc0039f0 0x7fb7141558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:23.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.943+0000 7fb71a833640 1 -- 172.21.15.29:0/868766123 >> 172.21.15.29:0/868766123 conn(0x7fb7140b1870 msgr2=0x7fb714141b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:23.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.947+0000 7fb71a833640 1 -- 172.21.15.29:0/868766123 shutdown_connections 2024-04-25T00:37:23.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:23.947+0000 7fb71a833640 1 -- 172.21.15.29:0/868766123 wait complete. 2024-04-25T00:37:23.989 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:37:24.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.107+0000 7f4127806640 1 Processor -- start 2024-04-25T00:37:24.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.111+0000 7f4127806640 1 -- start start 2024-04-25T00:37:24.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.111+0000 7f4127806640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f412006e440 0x7f41200b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.111+0000 7f4127806640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41200b41e0 con 0x7f41200b3c30 2024-04-25T00:37:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.111+0000 7f4127806640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f412006d960 con 0x7f41200b3140 2024-04-25T00:37:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.111+0000 7f4127806640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f41200b5050 con 0x7f412006e440 2024-04-25T00:37:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.111+0000 7f4125da3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f412006e440 0x7f41200b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.111+0000 7f4124da1640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f41200b3c30 0x7f41200ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42796/0 (socket says 172.21.15.29:42796) 2024-04-25T00:37:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.111+0000 7f4124da1640 1 -- 172.21.15.29:0/1561142571 learned_addr learned my addr 172.21.15.29:0/1561142571 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.111+0000 7f4117fff640 1 -- 172.21.15.29:0/1561142571 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2440328217 0 0) 0x7f41200b41e0 con 0x7f41200b3c30 2024-04-25T00:37:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.111+0000 7f4117fff640 1 -- 172.21.15.29:0/1561142571 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f410c003530 con 0x7f41200b3c30 2024-04-25T00:37:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.111+0000 7f4125da3640 1 -- 172.21.15.29:0/1561142571 >> v1:172.21.15.157:6789/0 conn(0x7f41200b3140 legacy=0x7f41200b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.111+0000 7f4125da3640 1 -- 172.21.15.29:0/1561142571 >> v1:172.21.15.29:6789/0 conn(0x7f41200b3c30 legacy=0x7f41200ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.111+0000 7f4125da3640 1 -- 172.21.15.29:0/1561142571 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412006dc40 con 0x7f412006e440 2024-04-25T00:37:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4125da3640 1 --2- 172.21.15.29:0/1561142571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f412006e440 0x7f41200b4b10 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4118009dc0 tx=0x7f411801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7c09b0b2d0d286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4117fff640 1 -- 172.21.15.29:0/1561142571 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41180b2070 con 0x7f412006e440 2024-04-25T00:37:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4117fff640 1 -- 172.21.15.29:0/1561142571 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41180a57d0 con 0x7f412006e440 2024-04-25T00:37:24.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4117fff640 1 -- 172.21.15.29:0/1561142571 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41180a5ba0 con 0x7f412006e440 2024-04-25T00:37:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 -- 172.21.15.29:0/1561142571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f412006e440 msgr2=0x7f41200b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 --2- 172.21.15.29:0/1561142571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f412006e440 0x7f41200b4b10 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4118009dc0 tx=0x7f411801c910 comp rx=0 tx=0).stop 2024-04-25T00:37:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 -- 172.21.15.29:0/1561142571 shutdown_connections 2024-04-25T00:37:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 --2- 172.21.15.29:0/1561142571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f412006e440 0x7f41200b4b10 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 -- 172.21.15.29:0/1561142571 >> 172.21.15.29:0/1561142571 conn(0x7f41200b1870 msgr2=0x7f41200b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 -- 172.21.15.29:0/1561142571 shutdown_connections 2024-04-25T00:37:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 -- 172.21.15.29:0/1561142571 wait complete. 2024-04-25T00:37:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 Processor -- start 2024-04-25T00:37:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 -- start start 2024-04-25T00:37:24.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f412006e440 0x7f4120142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:24.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41200b3140 0x7f4120142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:24.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41200b3c30 0x7f41201388c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:24.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4120169d90 con 0x7f412006e440 2024-04-25T00:37:24.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4120169c10 con 0x7f41200b3140 2024-04-25T00:37:24.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4127806640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4120169f10 con 0x7f41200b3c30 2024-04-25T00:37:24.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f41255a2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f412006e440 0x7f4120142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:24.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f41255a2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f412006e440 0x7f4120142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38320/0 (socket says 172.21.15.29:38320) 2024-04-25T00:37:24.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f41255a2640 1 -- 172.21.15.29:0/184729915 learned_addr learned my addr 172.21.15.29:0/184729915 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:24.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4124da1640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41200b3140 0x7f4120142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:24.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f41255a2640 1 -- 172.21.15.29:0/184729915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41200b3c30 msgr2=0x7f41201388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:24.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f41255a2640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41200b3c30 0x7f41201388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:24.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f41255a2640 1 -- 172.21.15.29:0/184729915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41200b3140 msgr2=0x7f4120142fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:24.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f41255a2640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41200b3140 0x7f4120142fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:24.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f41255a2640 1 -- 172.21.15.29:0/184729915 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41201428b0 con 0x7f412006e440 2024-04-25T00:37:24.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.115+0000 7f4124da1640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41200b3140 0x7f4120142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:24.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.119+0000 7f41255a2640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f412006e440 0x7f4120142060 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f411c00c480 tx=0x7f411c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:24.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.119+0000 7f4115ffb640 1 -- 172.21.15.29:0/184729915 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c012830 con 0x7f412006e440 2024-04-25T00:37:24.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.119+0000 7f4115ffb640 1 -- 172.21.15.29:0/184729915 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f411c017070 con 0x7f412006e440 2024-04-25T00:37:24.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.119+0000 7f4127806640 1 -- 172.21.15.29:0/184729915 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412015a020 con 0x7f412006e440 2024-04-25T00:37:24.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.119+0000 7f4115ffb640 1 -- 172.21.15.29:0/184729915 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c012c30 con 0x7f412006e440 2024-04-25T00:37:24.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.119+0000 7f4127806640 1 -- 172.21.15.29:0/184729915 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412015a5e0 con 0x7f412006e440 2024-04-25T00:37:24.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.119+0000 7f4127806640 1 -- 172.21.15.29:0/184729915 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f412015a7e0 con 0x7f412006e440 2024-04-25T00:37:24.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.119+0000 7f4115ffb640 1 -- 172.21.15.29:0/184729915 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f411c01c460 con 0x7f412006e440 2024-04-25T00:37:24.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.119+0000 7f4115ffb640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f41040679e0 0x7f4104069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:24.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.119+0000 7f4124da1640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f41040679e0 0x7f4104069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:24.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.119+0000 7f4115ffb640 1 -- 172.21.15.29:0/184729915 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5609+0+0 (secure 0 0 0) 0x7f411c0866f0 con 0x7f412006e440 2024-04-25T00:37:24.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.119+0000 7f4124da1640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f41040679e0 0x7f4104069ea0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f41201fa210 tx=0x7f4110009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:24.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.123+0000 7f4115ffb640 1 -- 172.21.15.29:0/184729915 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f412015a7e0 con 0x7f412006e440 2024-04-25T00:37:24.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.323+0000 7f4127806640 1 -- 172.21.15.29:0/184729915 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4120069270 con 0x7f412006e440 2024-04-25T00:37:24.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.939+0000 7f4115ffb640 1 -- 172.21.15.29:0/184729915 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v63) ==== 174+0+0 (secure 0 0 0) 0x7f411c0571f0 con 0x7f412006e440 2024-04-25T00:37:24.943 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 16 and data pool 17 2024-04-25T00:37:24.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.943+0000 7f4127806640 1 -- 172.21.15.29:0/184729915 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f41040679e0 msgr2=0x7f4104069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:24.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.943+0000 7f4127806640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f41040679e0 0x7f4104069ea0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f41201fa210 tx=0x7f4110009040 comp rx=0 tx=0).stop 2024-04-25T00:37:24.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.943+0000 7f4127806640 1 -- 172.21.15.29:0/184729915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f412006e440 msgr2=0x7f4120142060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:24.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.943+0000 7f4127806640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f412006e440 0x7f4120142060 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f411c00c480 tx=0x7f411c00c940 comp rx=0 tx=0).stop 2024-04-25T00:37:24.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.943+0000 7f4127806640 1 -- 172.21.15.29:0/184729915 shutdown_connections 2024-04-25T00:37:24.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.943+0000 7f4127806640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f41040679e0 0x7f4104069ea0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:24.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.943+0000 7f4127806640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41200b3c30 0x7f41201388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:24.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.943+0000 7f4127806640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41200b3140 0x7f4120142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:24.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.943+0000 7f4127806640 1 --2- 172.21.15.29:0/184729915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f412006e440 0x7f4120142060 secure :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f411c00c480 tx=0x7f411c00c940 comp rx=0 tx=0).stop 2024-04-25T00:37:24.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.943+0000 7f4127806640 1 -- 172.21.15.29:0/184729915 >> 172.21.15.29:0/184729915 conn(0x7f41200b1870 msgr2=0x7f41200b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:24.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.943+0000 7f4127806640 1 -- 172.21.15.29:0/184729915 shutdown_connections 2024-04-25T00:37:24.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:24.943+0000 7f4127806640 1 -- 172.21.15.29:0/184729915 wait complete. 2024-04-25T00:37:24.977 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:37:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d838bc640 1 Processor -- start 2024-04-25T00:37:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d838bc640 1 -- start start 2024-04-25T00:37:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d838bc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d7c15c0e0 0x7f7d7c0c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d838bc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0cf540 0x7f7d7c0c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d838bc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d7c0d04f0 con 0x7f7d7c0d0010 2024-04-25T00:37:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d838bc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7d7c15c4c0 con 0x7f7d7c15c0e0 2024-04-25T00:37:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d838bc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7c0d07d0 con 0x7f7d7c0cf540 2024-04-25T00:37:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d81658640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d7c15c0e0 0x7f7d7c0c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d81658640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d7c15c0e0 0x7f7d7c0c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38334/0 (socket says 172.21.15.29:38334) 2024-04-25T00:37:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d81658640 1 -- 172.21.15.29:0/3206127258 learned_addr learned my addr 172.21.15.29:0/3206127258 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d81e59640 1 --2- 172.21.15.29:0/3206127258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0cf540 0x7f7d7c0c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d81658640 1 -- 172.21.15.29:0/3206127258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0cf540 msgr2=0x7f7d7c0c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d81658640 1 --2- 172.21.15.29:0/3206127258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0cf540 0x7f7d7c0c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d81658640 1 -- 172.21.15.29:0/3206127258 >> v1:172.21.15.157:6789/0 conn(0x7f7d7c0d0010 legacy=0x7f7d7c0cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.099+0000 7f7d81658640 1 -- 172.21.15.29:0/3206127258 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c0d0ab0 con 0x7f7d7c15c0e0 2024-04-25T00:37:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d81658640 1 --2- 172.21.15.29:0/3206127258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d7c15c0e0 0x7f7d7c0c7680 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009870 tx=0x7f7d7801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d111c1628530811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d73fff640 1 -- 172.21.15.29:0/3206127258 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d780a95f0 con 0x7f7d7c15c0e0 2024-04-25T00:37:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d73fff640 1 -- 172.21.15.29:0/3206127258 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d780a9790 con 0x7f7d7c15c0e0 2024-04-25T00:37:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d73fff640 1 -- 172.21.15.29:0/3206127258 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d780a9b60 con 0x7f7d7c15c0e0 2024-04-25T00:37:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 -- 172.21.15.29:0/3206127258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d7c15c0e0 msgr2=0x7f7d7c0c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 --2- 172.21.15.29:0/3206127258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d7c15c0e0 0x7f7d7c0c7680 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009870 tx=0x7f7d7801c530 comp rx=0 tx=0).stop 2024-04-25T00:37:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 -- 172.21.15.29:0/3206127258 shutdown_connections 2024-04-25T00:37:25.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 --2- 172.21.15.29:0/3206127258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0cf540 0x7f7d7c0c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 --2- 172.21.15.29:0/3206127258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d7c15c0e0 0x7f7d7c0c7680 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 -- 172.21.15.29:0/3206127258 >> 172.21.15.29:0/3206127258 conn(0x7f7d7c0b3a00 msgr2=0x7f7d7c0b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:25.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 -- 172.21.15.29:0/3206127258 shutdown_connections 2024-04-25T00:37:25.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 -- 172.21.15.29:0/3206127258 wait complete. 2024-04-25T00:37:25.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 Processor -- start 2024-04-25T00:37:25.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 -- start start 2024-04-25T00:37:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0cf540 0x7f7d7c06c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d7c0d0010 0x7f7d7c1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c15c0e0 0x7f7d7c165090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c169050 con 0x7f7d7c0d0010 2024-04-25T00:37:25.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c168ed0 con 0x7f7d7c15c0e0 2024-04-25T00:37:25.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d838bc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7c1691d0 con 0x7f7d7c0cf540 2024-04-25T00:37:25.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d80e57640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d7c0d0010 0x7f7d7c1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d80e57640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d7c0d0010 0x7f7d7c1564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38338/0 (socket says 172.21.15.29:38338) 2024-04-25T00:37:25.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d80e57640 1 -- 172.21.15.29:0/3088535488 learned_addr learned my addr 172.21.15.29:0/3088535488 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:25.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d81658640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0cf540 0x7f7d7c06c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d81e59640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c15c0e0 0x7f7d7c165090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d80e57640 1 -- 172.21.15.29:0/3088535488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0cf540 msgr2=0x7f7d7c06c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d80e57640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0cf540 0x7f7d7c06c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d80e57640 1 -- 172.21.15.29:0/3088535488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c15c0e0 msgr2=0x7f7d7c165090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d80e57640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c15c0e0 0x7f7d7c165090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.103+0000 7f7d80e57640 1 -- 172.21.15.29:0/3088535488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c156a30 con 0x7f7d7c0d0010 2024-04-25T00:37:25.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.107+0000 7f7d80e57640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d7c0d0010 0x7f7d7c1564f0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00c400 tx=0x7f7d6c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:25.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.107+0000 7f7d71ffb640 1 -- 172.21.15.29:0/3088535488 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c0052d0 con 0x7f7d7c0d0010 2024-04-25T00:37:25.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.107+0000 7f7d71ffb640 1 -- 172.21.15.29:0/3088535488 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6c005470 con 0x7f7d7c0d0010 2024-04-25T00:37:25.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.107+0000 7f7d838bc640 1 -- 172.21.15.29:0/3088535488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c06c9c0 con 0x7f7d7c0d0010 2024-04-25T00:37:25.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.107+0000 7f7d71ffb640 1 -- 172.21.15.29:0/3088535488 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c007260 con 0x7f7d7c0d0010 2024-04-25T00:37:25.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.107+0000 7f7d838bc640 1 -- 172.21.15.29:0/3088535488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c06cf80 con 0x7f7d7c0d0010 2024-04-25T00:37:25.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.107+0000 7f7d838bc640 1 -- 172.21.15.29:0/3088535488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d7c069270 con 0x7f7d7c0d0010 2024-04-25T00:37:25.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.107+0000 7f7d71ffb640 1 -- 172.21.15.29:0/3088535488 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f7d6c007400 con 0x7f7d7c0d0010 2024-04-25T00:37:25.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.107+0000 7f7d71ffb640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d600679e0 0x7f7d60069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.107+0000 7f7d71ffb640 1 -- 172.21.15.29:0/3088535488 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f7d6c0852d0 con 0x7f7d7c0d0010 2024-04-25T00:37:25.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.107+0000 7f7d81658640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d600679e0 0x7f7d60069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.107+0000 7f7d81658640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d600679e0 0x7f7d60069ea0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7d780ae000 tx=0x7f7d780b3040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:25.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.111+0000 7f7d71ffb640 1 -- 172.21.15.29:0/3088535488 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7d6c055d00 con 0x7f7d7c0d0010 2024-04-25T00:37:25.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.275+0000 7f7d838bc640 1 -- 172.21.15.29:0/3088535488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d7c0b7ba0 con 0x7f7d7c0d0010 2024-04-25T00:37:25.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.275+0000 7f7d71ffb640 1 -- 172.21.15.29:0/3088535488 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16592 (secure 0 0 0) 0x7f7d6c055600 con 0x7f7d7c0d0010 2024-04-25T00:37:25.280 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:25.280 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":102,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:37:24.327058+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:37:22.602795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:37:23.310674+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:37:25.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.279+0000 7f7d838bc640 1 -- 172.21.15.29:0/3088535488 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d600679e0 msgr2=0x7f7d60069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.279+0000 7f7d838bc640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d600679e0 0x7f7d60069ea0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7d780ae000 tx=0x7f7d780b3040 comp rx=0 tx=0).stop 2024-04-25T00:37:25.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.279+0000 7f7d838bc640 1 -- 172.21.15.29:0/3088535488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d7c0d0010 msgr2=0x7f7d7c1564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.279+0000 7f7d838bc640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d7c0d0010 0x7f7d7c1564f0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00c400 tx=0x7f7d6c00c8c0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.279+0000 7f7d838bc640 1 -- 172.21.15.29:0/3088535488 shutdown_connections 2024-04-25T00:37:25.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.279+0000 7f7d838bc640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d600679e0 0x7f7d60069ea0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.279+0000 7f7d838bc640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c15c0e0 0x7f7d7c165090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.279+0000 7f7d838bc640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d7c0d0010 0x7f7d7c1564f0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.279+0000 7f7d838bc640 1 --2- 172.21.15.29:0/3088535488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0cf540 0x7f7d7c06c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.279+0000 7f7d838bc640 1 -- 172.21.15.29:0/3088535488 >> 172.21.15.29:0/3088535488 conn(0x7f7d7c0b3a00 msgr2=0x7f7d7c06fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:25.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.279+0000 7f7d838bc640 1 -- 172.21.15.29:0/3088535488 shutdown_connections 2024-04-25T00:37:25.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.279+0000 7f7d838bc640 1 -- 172.21.15.29:0/3088535488 wait complete. 2024-04-25T00:37:25.299 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:37:25.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6d0e1640 1 Processor -- start 2024-04-25T00:37:25.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6d0e1640 1 -- start start 2024-04-25T00:37:25.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6d0e1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbc680b4780 0x7fbc6815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6d0e1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc680cc240 0x7fbc6815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6d0e1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc680b3720 con 0x7fbc6815e8d0 2024-04-25T00:37:25.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6d0e1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbc680b42f0 con 0x7fbc680b4780 2024-04-25T00:37:25.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6d0e1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbc680b39c0 con 0x7fbc680cc240 2024-04-25T00:37:25.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc66d9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbc6815e8d0 0x7fbc6815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42804/0 (socket says 172.21.15.29:42804) 2024-04-25T00:37:25.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc66d9d640 1 -- 172.21.15.29:0/3536651859 learned_addr learned my addr 172.21.15.29:0/3536651859 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:25.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 138694807 0 0) 0x7fbc680b3720 con 0x7fbc6815e8d0 2024-04-25T00:37:25.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc50003530 con 0x7fbc6815e8d0 2024-04-25T00:37:25.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6759e640 1 --2- 172.21.15.29:0/3536651859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc680cc240 0x7fbc6815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1112886771 0 0) 0x7fbc50003530 con 0x7fbc6815e8d0 2024-04-25T00:37:25.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc680cc240 msgr2=0x7fbc6815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:25.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 --2- 172.21.15.29:0/3536651859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc680cc240 0x7fbc6815c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6659c640 1 --2- 172.21.15.29:0/3536651859 >> v2:172.21.15.157:3300/0 conn(0x7fbc680b4780 0x7fbc6815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 >> v2:172.21.15.157:3300/0 conn(0x7fbc680b4780 msgr2=0x7fbc6815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 --2- 172.21.15.29:0/3536651859 >> v2:172.21.15.157:3300/0 conn(0x7fbc680b4780 0x7fbc6815f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc680b3ca0 con 0x7fbc6815e8d0 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fbc58001680 con 0x7fbc6815e8d0 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 >> v1:172.21.15.29:6789/0 conn(0x7fbc6815e8d0 legacy=0x7fbc6815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 --2- 172.21.15.29:0/3536651859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc680cc240 0x7fbc500039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 --2- 172.21.15.29:0/3536651859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc50004010 0x7fbc50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 --2- 172.21.15.29:0/3536651859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc50004d50 0x7fbc500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbc680b42f0 con 0x7fbc50004010 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbc680b39c0 con 0x7fbc50004d50 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5000bbe0 con 0x7fbc680cc240 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6659c640 1 --2- 172.21.15.29:0/3536651859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc50004010 0x7fbc50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc66d9d640 1 --2- 172.21.15.29:0/3536651859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc680cc240 0x7fbc500039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6659c640 1 -- 172.21.15.29:0/3536651859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc680cc240 msgr2=0x7fbc500039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:25.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6659c640 1 --2- 172.21.15.29:0/3536651859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc680cc240 0x7fbc500039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6659c640 1 -- 172.21.15.29:0/3536651859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc50004d50 msgr2=0x7fbc500051d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:37:25.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6659c640 1 --2- 172.21.15.29:0/3536651859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc50004d50 0x7fbc500051d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc6659c640 1 -- 172.21.15.29:0/3536651859 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc50003530 con 0x7fbc50004010 2024-04-25T00:37:25.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.423+0000 7fbc66d9d640 1 --2- 172.21.15.29:0/3536651859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc680cc240 0x7fbc500039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:25.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.427+0000 7fbc6659c640 1 --2- 172.21.15.29:0/3536651859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc50004010 0x7fbc50004810 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00a990 tx=0x7fbc5c01c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644e67919dab13b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:25.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.427+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c00a590 con 0x7fbc50004010 2024-04-25T00:37:25.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.427+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc5c0af070 con 0x7fbc50004010 2024-04-25T00:37:25.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.427+0000 7fbc65d9b640 1 -- 172.21.15.29:0/3536651859 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0aa880 con 0x7fbc50004010 2024-04-25T00:37:25.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.427+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/3536651859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc50004010 msgr2=0x7fbc50004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.427+0000 7fbc6d0e1640 1 --2- 172.21.15.29:0/3536651859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc50004010 0x7fbc50004810 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00a990 tx=0x7fbc5c01c5d0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.427+0000 7fbc6759e640 1 -- 172.21.15.29:0/3536651859 reap_dead start 2024-04-25T00:37:25.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.427+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/3536651859 shutdown_connections 2024-04-25T00:37:25.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.427+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/3536651859 >> 172.21.15.29:0/3536651859 conn(0x7fbc680b1870 msgr2=0x7fbc680b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:25.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.427+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/3536651859 shutdown_connections 2024-04-25T00:37:25.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.427+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/3536651859 wait complete. 2024-04-25T00:37:25.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.427+0000 7fbc6d0e1640 1 Processor -- start 2024-04-25T00:37:25.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.427+0000 7fbc6d0e1640 1 -- start start 2024-04-25T00:37:25.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6d0e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc681589d0 0x7fbc68143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6d0e1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc68158db0 0x7fbc6813adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6d0e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6815d850 0x7fbc6815dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6d0e1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbc68144a20 con 0x7fbc681589d0 2024-04-25T00:37:25.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6d0e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6806cf10 con 0x7fbc6815d850 2024-04-25T00:37:25.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6d0e1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbc68070e00 con 0x7fbc68158db0 2024-04-25T00:37:25.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc66d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc681589d0 0x7fbc68143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc66d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc681589d0 0x7fbc68143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38364/0 (socket says 172.21.15.29:38364) 2024-04-25T00:37:25.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc66d9d640 1 -- 172.21.15.29:0/2457885791 learned_addr learned my addr 172.21.15.29:0/2457885791 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:25.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6659c640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc68158db0 0x7fbc6813adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6759e640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6815d850 0x7fbc6815dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc66d9d640 1 -- 172.21.15.29:0/2457885791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc68158db0 msgr2=0x7fbc6813adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc66d9d640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc68158db0 0x7fbc6813adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc66d9d640 1 -- 172.21.15.29:0/2457885791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6815d850 msgr2=0x7fbc6815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc66d9d640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6815d850 0x7fbc6815dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc66d9d640 1 -- 172.21.15.29:0/2457885791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6806e5e0 con 0x7fbc681589d0 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6759e640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6815d850 0x7fbc6815dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc66d9d640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc681589d0 0x7fbc68143e80 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fbc58010c30 tx=0x7fbc5800e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc4f7fe640 1 -- 172.21.15.29:0/2457885791 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5801b070 con 0x7fbc681589d0 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc4f7fe640 1 -- 172.21.15.29:0/2457885791 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc58011040 con 0x7fbc681589d0 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc4f7fe640 1 -- 172.21.15.29:0/2457885791 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc580145a0 con 0x7fbc681589d0 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/2457885791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc681f43f0 con 0x7fbc681589d0 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/2457885791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc68071050 con 0x7fbc681589d0 2024-04-25T00:37:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/2457885791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc68069270 con 0x7fbc681589d0 2024-04-25T00:37:25.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc4f7fe640 1 -- 172.21.15.29:0/2457885791 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fbc5801e070 con 0x7fbc681589d0 2024-04-25T00:37:25.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc4f7fe640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc540679e0 0x7fbc54069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc4f7fe640 1 -- 172.21.15.29:0/2457885791 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fbc58088a00 con 0x7fbc681589d0 2024-04-25T00:37:25.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6659c640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc540679e0 0x7fbc54069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.431+0000 7fbc6659c640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc540679e0 0x7fbc54069ea0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00a990 tx=0x7fbc5c0b5040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:25.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.435+0000 7fbc4f7fe640 1 -- 172.21.15.29:0/2457885791 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbc580594c0 con 0x7fbc681589d0 2024-04-25T00:37:25.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.599+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/2457885791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc6806e7b0 con 0x7fbc681589d0 2024-04-25T00:37:25.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.603+0000 7fbc4f7fe640 1 -- 172.21.15.29:0/2457885791 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16592 (secure 0 0 0) 0x7fbc58058dc0 con 0x7fbc681589d0 2024-04-25T00:37:25.604 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:25.604 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":102,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:37:24.327058+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:37:22.602795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:37:23.310674+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:37:25.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.603+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/2457885791 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc540679e0 msgr2=0x7fbc54069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.603+0000 7fbc6d0e1640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc540679e0 0x7fbc54069ea0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00a990 tx=0x7fbc5c0b5040 comp rx=0 tx=0).stop 2024-04-25T00:37:25.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.603+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/2457885791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc681589d0 msgr2=0x7fbc68143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.603+0000 7fbc6d0e1640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc681589d0 0x7fbc68143e80 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fbc58010c30 tx=0x7fbc5800e4d0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.603+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/2457885791 shutdown_connections 2024-04-25T00:37:25.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.603+0000 7fbc6d0e1640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc540679e0 0x7fbc54069ea0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.603+0000 7fbc6d0e1640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6815d850 0x7fbc6815dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.603+0000 7fbc6d0e1640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc68158db0 0x7fbc6813adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.603+0000 7fbc6d0e1640 1 --2- 172.21.15.29:0/2457885791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc681589d0 0x7fbc68143e80 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.603+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/2457885791 >> 172.21.15.29:0/2457885791 conn(0x7fbc680b1870 msgr2=0x7fbc680ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:25.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.603+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/2457885791 shutdown_connections 2024-04-25T00:37:25.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.603+0000 7fbc6d0e1640 1 -- 172.21.15.29:0/2457885791 wait complete. 2024-04-25T00:37:25.625 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:37:25.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.747+0000 7f79651f3640 1 Processor -- start 2024-04-25T00:37:25.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.747+0000 7f79651f3640 1 -- start start 2024-04-25T00:37:25.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f79651f3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f796006dce0 0x7f7960150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f79651f3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f79600b2b70 0x7f7960151330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f79651f3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79600b3130 con 0x7f796015d980 2024-04-25T00:37:25.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f79651f3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f79600cf560 con 0x7f79600b2b70 2024-04-25T00:37:25.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f79651f3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f79600b3410 con 0x7f796006dce0 2024-04-25T00:37:25.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f795ed9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f796015d980 0x7f796006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42810/0 (socket says 172.21.15.29:42810) 2024-04-25T00:37:25.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f795ed9d640 1 -- 172.21.15.29:0/2648404317 learned_addr learned my addr 172.21.15.29:0/2648404317 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:25.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f795f59e640 1 --2- 172.21.15.29:0/2648404317 >> v2:172.21.15.29:3300/0 conn(0x7f79600b2b70 0x7f7960151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f795dd9b640 1 -- 172.21.15.29:0/2648404317 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1069390571 0 0) 0x7f79600b3130 con 0x7f796015d980 2024-04-25T00:37:25.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f795dd9b640 1 -- 172.21.15.29:0/2648404317 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7948003530 con 0x7f796015d980 2024-04-25T00:37:25.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f795f59e640 1 -- 172.21.15.29:0/2648404317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f796006dce0 msgr2=0x7f7960150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:25.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f795f59e640 1 --2- 172.21.15.29:0/2648404317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f796006dce0 0x7f7960150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f795f59e640 1 -- 172.21.15.29:0/2648404317 >> v1:172.21.15.29:6789/0 conn(0x7f796015d980 legacy=0x7f796006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f795f59e640 1 -- 172.21.15.29:0/2648404317 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79600b36f0 con 0x7f79600b2b70 2024-04-25T00:37:25.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f795f59e640 1 --2- 172.21.15.29:0/2648404317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79600b2b70 0x7f7960151330 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f7958009d90 tx=0x7f795801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f3bf09113780e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:25.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f795dd9b640 1 -- 172.21.15.29:0/2648404317 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79580b2070 con 0x7f79600b2b70 2024-04-25T00:37:25.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f795dd9b640 1 -- 172.21.15.29:0/2648404317 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79580a58f0 con 0x7f79600b2b70 2024-04-25T00:37:25.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.751+0000 7f795dd9b640 1 -- 172.21.15.29:0/2648404317 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79580a5cc0 con 0x7f79600b2b70 2024-04-25T00:37:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 -- 172.21.15.29:0/2648404317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79600b2b70 msgr2=0x7f7960151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 --2- 172.21.15.29:0/2648404317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79600b2b70 0x7f7960151330 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f7958009d90 tx=0x7f795801c8b0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 -- 172.21.15.29:0/2648404317 shutdown_connections 2024-04-25T00:37:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 --2- 172.21.15.29:0/2648404317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79600b2b70 0x7f7960151330 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 --2- 172.21.15.29:0/2648404317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f796006dce0 0x7f7960150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 -- 172.21.15.29:0/2648404317 >> 172.21.15.29:0/2648404317 conn(0x7f79600b1870 msgr2=0x7f79600b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 -- 172.21.15.29:0/2648404317 shutdown_connections 2024-04-25T00:37:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 -- 172.21.15.29:0/2648404317 wait complete. 2024-04-25T00:37:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 Processor -- start 2024-04-25T00:37:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 -- start start 2024-04-25T00:37:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f796006dce0 0x7f7960145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79600b2b70 0x7f79601566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f796015d980 0x7f7960156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f796015b4b0 con 0x7f796015d980 2024-04-25T00:37:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7960151870 con 0x7f79600b2b70 2024-04-25T00:37:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f796015b630 con 0x7f796006dce0 2024-04-25T00:37:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f795f59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f796015d980 0x7f7960156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f795f59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f796015d980 0x7f7960156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38378/0 (socket says 172.21.15.29:38378) 2024-04-25T00:37:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f795f59e640 1 -- 172.21.15.29:0/128444336 learned_addr learned my addr 172.21.15.29:0/128444336 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f795f59e640 1 -- 172.21.15.29:0/128444336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f796006dce0 msgr2=0x7f7960145830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:37:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f795f59e640 1 --2- 172.21.15.29:0/128444336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f796006dce0 0x7f7960145830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f795f59e640 1 -- 172.21.15.29:0/128444336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79600b2b70 msgr2=0x7f79601566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f795f59e640 1 --2- 172.21.15.29:0/128444336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79600b2b70 0x7f79601566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f795f59e640 1 -- 172.21.15.29:0/128444336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796015ef10 con 0x7f796015d980 2024-04-25T00:37:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f795f59e640 1 --2- 172.21.15.29:0/128444336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f796015d980 0x7f7960156c10 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f79580b0040 tx=0x7f7958019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79477fe640 1 -- 172.21.15.29:0/128444336 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79580b2050 con 0x7f796015d980 2024-04-25T00:37:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79477fe640 1 -- 172.21.15.29:0/128444336 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79580be070 con 0x7f796015d980 2024-04-25T00:37:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79477fe640 1 -- 172.21.15.29:0/128444336 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79580ada90 con 0x7f796015d980 2024-04-25T00:37:25.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.755+0000 7f79651f3640 1 -- 172.21.15.29:0/128444336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79600ce900 con 0x7f796015d980 2024-04-25T00:37:25.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.759+0000 7f79651f3640 1 -- 172.21.15.29:0/128444336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79600ce060 con 0x7f796015d980 2024-04-25T00:37:25.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.759+0000 7f79651f3640 1 -- 172.21.15.29:0/128444336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796014a050 con 0x7f796015d980 2024-04-25T00:37:25.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.759+0000 7f79477fe640 1 -- 172.21.15.29:0/128444336 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f79580adc30 con 0x7f796015d980 2024-04-25T00:37:25.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.759+0000 7f79477fe640 1 --2- 172.21.15.29:0/128444336 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f794c067a10 0x7f794c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:25.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.759+0000 7f79477fe640 1 -- 172.21.15.29:0/128444336 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f7958120bf0 con 0x7f796015d980 2024-04-25T00:37:25.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.759+0000 7f795ed9d640 1 --2- 172.21.15.29:0/128444336 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f794c067a10 0x7f794c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:25.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.759+0000 7f795ed9d640 1 --2- 172.21.15.29:0/128444336 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f794c067a10 0x7f794c069ed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7950000f70 tx=0x7f79500010b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:25.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.763+0000 7f79477fe640 1 -- 172.21.15.29:0/128444336 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7958016560 con 0x7f796015d980 2024-04-25T00:37:25.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:25.963+0000 7f79651f3640 1 -- 172.21.15.29:0/128444336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f79600cf910 con 0x7f796015d980 2024-04-25T00:37:26.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:26.955+0000 7f79477fe640 1 -- 172.21.15.29:0/128444336 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) ==== 118+0+0 (secure 0 0 0) 0x7f79580f1610 con 0x7f796015d980 2024-04-25T00:37:26.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:26.959+0000 7f79651f3640 1 -- 172.21.15.29:0/128444336 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f794c067a10 msgr2=0x7f794c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:26.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:26.959+0000 7f79651f3640 1 --2- 172.21.15.29:0/128444336 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f794c067a10 0x7f794c069ed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7950000f70 tx=0x7f79500010b0 comp rx=0 tx=0).stop 2024-04-25T00:37:26.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:26.959+0000 7f79651f3640 1 -- 172.21.15.29:0/128444336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f796015d980 msgr2=0x7f7960156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:26.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:26.959+0000 7f79651f3640 1 --2- 172.21.15.29:0/128444336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f796015d980 0x7f7960156c10 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f79580b0040 tx=0x7f7958019650 comp rx=0 tx=0).stop 2024-04-25T00:37:26.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:26.959+0000 7f79651f3640 1 -- 172.21.15.29:0/128444336 shutdown_connections 2024-04-25T00:37:26.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:26.959+0000 7f79651f3640 1 --2- 172.21.15.29:0/128444336 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f794c067a10 0x7f794c069ed0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:26.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:26.959+0000 7f79651f3640 1 --2- 172.21.15.29:0/128444336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f796015d980 0x7f7960156c10 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:26.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:26.959+0000 7f79651f3640 1 --2- 172.21.15.29:0/128444336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79600b2b70 0x7f79601566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:26.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:26.959+0000 7f79651f3640 1 --2- 172.21.15.29:0/128444336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f796006dce0 0x7f7960145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:26.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:26.959+0000 7f79651f3640 1 -- 172.21.15.29:0/128444336 >> 172.21.15.29:0/128444336 conn(0x7f79600b1870 msgr2=0x7f796015e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:26.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:26.959+0000 7f79651f3640 1 -- 172.21.15.29:0/128444336 shutdown_connections 2024-04-25T00:37:26.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:26.959+0000 7f79651f3640 1 -- 172.21.15.29:0/128444336 wait complete. 2024-04-25T00:37:26.983 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:37:26.984 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:37:27.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefe493e640 1 Processor -- start 2024-04-25T00:37:27.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefe493e640 1 -- start start 2024-04-25T00:37:27.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefe493e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe00b3c30 0x7fefe006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:27.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefe493e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe00b41e0 con 0x7fefe00b4b10 2024-04-25T00:37:27.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefe493e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe006d960 con 0x7fefe00b3140 2024-04-25T00:37:27.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefe493e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fefe006dc40 con 0x7fefe00b3c30 2024-04-25T00:37:27.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefded9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fefe00b4b10 0x7fefe00ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42824/0 (socket says 172.21.15.29:42824) 2024-04-25T00:37:27.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefded9d640 1 -- 172.21.15.29:0/684233721 learned_addr learned my addr 172.21.15.29:0/684233721 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:27.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefd7fff640 1 --2- 172.21.15.29:0/684233721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe00b3c30 0x7fefe006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:27.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3917695762 0 0) 0x7fefe00b41e0 con 0x7fefe00b4b10 2024-04-25T00:37:27.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefcc003530 con 0x7fefe00b4b10 2024-04-25T00:37:27.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3014392175 0 0) 0x7fefcc003530 con 0x7fefe00b4b10 2024-04-25T00:37:27.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe00b3c30 msgr2=0x7fefe006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:27.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 --2- 172.21.15.29:0/684233721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe00b3c30 0x7fefe006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:27.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 >> v1:172.21.15.157:6789/0 conn(0x7fefe00b3140 legacy=0x7fefe00b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:27.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe006df20 con 0x7fefe00b4b10 2024-04-25T00:37:27.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fefd8001a00 con 0x7fefe00b4b10 2024-04-25T00:37:27.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 >> v1:172.21.15.29:6789/0 conn(0x7fefe00b4b10 legacy=0x7fefe00ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:27.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 --2- 172.21.15.29:0/684233721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc0039f0 0x7fefcc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:27.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 --2- 172.21.15.29:0/684233721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefcc004810 0x7fefcc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:27.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 --2- 172.21.15.29:0/684233721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe00b3c30 0x7fefcc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:27.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fefe006dc40 con 0x7fefcc004810 2024-04-25T00:37:27.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc00bab0 con 0x7fefcc0039f0 2024-04-25T00:37:27.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fefcc00bcb0 con 0x7fefe00b3c30 2024-04-25T00:37:27.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefd7fff640 1 --2- 172.21.15.29:0/684233721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefcc004810 0x7fefcc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:27.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefde59c640 1 --2- 172.21.15.29:0/684233721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc0039f0 0x7fefcc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:27.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefded9d640 1 --2- 172.21.15.29:0/684233721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe00b3c30 0x7fefcc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:27.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefd7fff640 1 -- 172.21.15.29:0/684233721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe00b3c30 msgr2=0x7fefcc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:27.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefd7fff640 1 --2- 172.21.15.29:0/684233721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe00b3c30 0x7fefcc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:27.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefd7fff640 1 -- 172.21.15.29:0/684233721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc0039f0 msgr2=0x7fefcc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:27.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefd7fff640 1 --2- 172.21.15.29:0/684233721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc0039f0 0x7fefcc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:27.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.111+0000 7fefd7fff640 1 -- 172.21.15.29:0/684233721 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefcc003530 con 0x7fefcc004810 2024-04-25T00:37:27.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefd7fff640 1 --2- 172.21.15.29:0/684233721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefcc004810 0x7fefcc004c90 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009ab0 tx=0x7fefc801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6958d4386847f623 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:27.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc80b2070 con 0x7fefcc004810 2024-04-25T00:37:27.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc80a5510 con 0x7fefcc004810 2024-04-25T00:37:27.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefddd9b640 1 -- 172.21.15.29:0/684233721 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc80096b0 con 0x7fefcc004810 2024-04-25T00:37:27.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 -- 172.21.15.29:0/684233721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefcc004810 msgr2=0x7fefcc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:27.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 --2- 172.21.15.29:0/684233721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefcc004810 0x7fefcc004c90 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009ab0 tx=0x7fefc801ca30 comp rx=0 tx=0).stop 2024-04-25T00:37:27.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefded9d640 1 -- 172.21.15.29:0/684233721 reap_dead start 2024-04-25T00:37:27.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 -- 172.21.15.29:0/684233721 shutdown_connections 2024-04-25T00:37:27.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 -- 172.21.15.29:0/684233721 >> 172.21.15.29:0/684233721 conn(0x7fefe00b1870 msgr2=0x7fefe00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:27.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 -- 172.21.15.29:0/684233721 shutdown_connections 2024-04-25T00:37:27.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 -- 172.21.15.29:0/684233721 wait complete. 2024-04-25T00:37:27.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 Processor -- start 2024-04-25T00:37:27.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 -- start start 2024-04-25T00:37:27.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc004810 0x7fefe015b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:27.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefe013f070 0x7fefe013f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:27.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe00c4ae0 0x7fefe00c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:27.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0145a90 con 0x7fefe013f070 2024-04-25T00:37:27.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0145910 con 0x7fefcc004810 2024-04-25T00:37:27.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefe493e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fefe006e440 con 0x7fefe00c4ae0 2024-04-25T00:37:27.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.115+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefe013f070 0x7fefe013f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:27.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefe013f070 0x7fefe013f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38406/0 (socket says 172.21.15.29:38406) 2024-04-25T00:37:27.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd7fff640 1 -- 172.21.15.29:0/1410592358 learned_addr learned my addr 172.21.15.29:0/1410592358 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:27.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefded9d640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe00c4ae0 0x7fefe00c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:27.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefde59c640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc004810 0x7fefe015b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:27.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd7fff640 1 -- 172.21.15.29:0/1410592358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe00c4ae0 msgr2=0x7fefe00c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:27.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd7fff640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe00c4ae0 0x7fefe00c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:27.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd7fff640 1 -- 172.21.15.29:0/1410592358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc004810 msgr2=0x7fefe015b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:27.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd7fff640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc004810 0x7fefe015b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:27.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd7fff640 1 -- 172.21.15.29:0/1410592358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe015bd20 con 0x7fefe013f070 2024-04-25T00:37:27.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd7fff640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefe013f070 0x7fefe013f4f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fefc801c860 tx=0x7fefcc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:27.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd6ffd640 1 -- 172.21.15.29:0/1410592358 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc80bc020 con 0x7fefe013f070 2024-04-25T00:37:27.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd6ffd640 1 -- 172.21.15.29:0/1410592358 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc8004020 con 0x7fefe013f070 2024-04-25T00:37:27.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefe493e640 1 -- 172.21.15.29:0/1410592358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe013fb10 con 0x7fefe013f070 2024-04-25T00:37:27.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd6ffd640 1 -- 172.21.15.29:0/1410592358 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc80b2040 con 0x7fefe013f070 2024-04-25T00:37:27.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefe493e640 1 -- 172.21.15.29:0/1410592358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe013ab00 con 0x7fefe013f070 2024-04-25T00:37:27.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefe493e640 1 -- 172.21.15.29:0/1410592358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe0069270 con 0x7fefe013f070 2024-04-25T00:37:27.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd6ffd640 1 -- 172.21.15.29:0/1410592358 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fefc8003620 con 0x7fefe013f070 2024-04-25T00:37:27.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd6ffd640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fefc0067a10 0x7fefc0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:27.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefd6ffd640 1 -- 172.21.15.29:0/1410592358 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fefc8120220 con 0x7fefe013f070 2024-04-25T00:37:27.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefde59c640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fefc0067a10 0x7fefc0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:27.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.119+0000 7fefde59c640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fefc0067a10 0x7fefc0069ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fefd0005510 tx=0x7fefd0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:27.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.123+0000 7fefd6ffd640 1 -- 172.21.15.29:0/1410592358 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fefc80c6020 con 0x7fefe013f070 2024-04-25T00:37:27.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.315+0000 7fefe493e640 1 -- 172.21.15.29:0/1410592358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fefe01446a0 con 0x7fefe013f070 2024-04-25T00:37:27.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.963+0000 7fefd6ffd640 1 -- 172.21.15.29:0/1410592358 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) ==== 122+0+0 (secure 0 0 0) 0x7fefc8016740 con 0x7fefe013f070 2024-04-25T00:37:27.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.967+0000 7fefe493e640 1 -- 172.21.15.29:0/1410592358 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fefc0067a10 msgr2=0x7fefc0069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:27.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.967+0000 7fefe493e640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fefc0067a10 0x7fefc0069ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fefd0005510 tx=0x7fefd0009040 comp rx=0 tx=0).stop 2024-04-25T00:37:27.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.967+0000 7fefe493e640 1 -- 172.21.15.29:0/1410592358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefe013f070 msgr2=0x7fefe013f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:27.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.967+0000 7fefe493e640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefe013f070 0x7fefe013f4f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fefc801c860 tx=0x7fefcc005840 comp rx=0 tx=0).stop 2024-04-25T00:37:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.967+0000 7fefe493e640 1 -- 172.21.15.29:0/1410592358 shutdown_connections 2024-04-25T00:37:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.967+0000 7fefe493e640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fefc0067a10 0x7fefc0069ed0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.967+0000 7fefe493e640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe00c4ae0 0x7fefe00c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.967+0000 7fefe493e640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefe013f070 0x7fefe013f4f0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.967+0000 7fefe493e640 1 --2- 172.21.15.29:0/1410592358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefcc004810 0x7fefe015b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.967+0000 7fefe493e640 1 -- 172.21.15.29:0/1410592358 >> 172.21.15.29:0/1410592358 conn(0x7fefe00b1870 msgr2=0x7fefe0143bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.967+0000 7fefe493e640 1 -- 172.21.15.29:0/1410592358 shutdown_connections 2024-04-25T00:37:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:27.967+0000 7fefe493e640 1 -- 172.21.15.29:0/1410592358 wait complete. 2024-04-25T00:37:27.993 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:37:27.993 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:37:28.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f1479d79640 1 Processor -- start 2024-04-25T00:37:28.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f1479d79640 1 -- start start 2024-04-25T00:37:28.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f1479d79640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f147415df70 0x7f14740b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f1479d79640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f147406d210 0x7f14740b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f1479d79640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14740b4b60 0x7f1474070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f1479d79640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f14740b4fd0 con 0x7f147415df70 2024-04-25T00:37:28.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f1479d79640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f147406d980 con 0x7f147406d210 2024-04-25T00:37:28.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f1479d79640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f147406dc60 con 0x7f14740b4b60 2024-04-25T00:37:28.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f14737fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f147415df70 0x7f14740b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f14737fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f147415df70 0x7f14740b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43556/0 (socket says 172.21.15.29:43556) 2024-04-25T00:37:28.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f14737fe640 1 -- 172.21.15.29:0/3396199691 learned_addr learned my addr 172.21.15.29:0/3396199691 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:28.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f1473fff640 1 --2- 172.21.15.29:0/3396199691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14740b4b60 0x7f1474070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f14737fe640 1 -- 172.21.15.29:0/3396199691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14740b4b60 msgr2=0x7f1474070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:28.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f14737fe640 1 --2- 172.21.15.29:0/3396199691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14740b4b60 0x7f1474070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f14737fe640 1 -- 172.21.15.29:0/3396199691 >> v2:172.21.15.157:3300/0 conn(0x7f147406d210 msgr2=0x7f14740b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:28.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f14737fe640 1 --2- 172.21.15.29:0/3396199691 >> v2:172.21.15.157:3300/0 conn(0x7f147406d210 0x7f14740b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.119+0000 7f14737fe640 1 -- 172.21.15.29:0/3396199691 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147406df40 con 0x7f147415df70 2024-04-25T00:37:28.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f14737fe640 1 --2- 172.21.15.29:0/3396199691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f147415df70 0x7f14740b3e40 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f14740cfb70 tx=0x7f146801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3942291b6868031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:28.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f14727fc640 1 -- 172.21.15.29:0/3396199691 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14680b0070 con 0x7f147415df70 2024-04-25T00:37:28.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f14727fc640 1 -- 172.21.15.29:0/3396199691 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14680ab600 con 0x7f147415df70 2024-04-25T00:37:28.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f14727fc640 1 -- 172.21.15.29:0/3396199691 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14680ab9d0 con 0x7f147415df70 2024-04-25T00:37:28.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 -- 172.21.15.29:0/3396199691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f147415df70 msgr2=0x7f14740b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 --2- 172.21.15.29:0/3396199691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f147415df70 0x7f14740b3e40 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f14740cfb70 tx=0x7f146801c230 comp rx=0 tx=0).stop 2024-04-25T00:37:28.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 -- 172.21.15.29:0/3396199691 shutdown_connections 2024-04-25T00:37:28.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 --2- 172.21.15.29:0/3396199691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14740b4b60 0x7f1474070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 --2- 172.21.15.29:0/3396199691 >> v2:172.21.15.157:3300/0 conn(0x7f147406d210 0x7f14740b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 --2- 172.21.15.29:0/3396199691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f147415df70 0x7f14740b3e40 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 -- 172.21.15.29:0/3396199691 >> 172.21.15.29:0/3396199691 conn(0x7f14740b1870 msgr2=0x7f14740b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:28.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 -- 172.21.15.29:0/3396199691 shutdown_connections 2024-04-25T00:37:28.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 -- 172.21.15.29:0/3396199691 wait complete. 2024-04-25T00:37:28.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 Processor -- start 2024-04-25T00:37:28.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 -- start start 2024-04-25T00:37:28.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f147406d210 0x7f14741404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14740b4b60 0x7f14741512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f147415df70 0x7f14741517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f147406ea20 con 0x7f147406d210 2024-04-25T00:37:28.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f147415e350 con 0x7f147415df70 2024-04-25T00:37:28.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1479d79640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f147406eba0 con 0x7f14740b4b60 2024-04-25T00:37:28.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f14737fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f147406d210 0x7f14741404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f14737fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f147406d210 0x7f14741404c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43568/0 (socket says 172.21.15.29:43568) 2024-04-25T00:37:28.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f14737fe640 1 -- 172.21.15.29:0/3103261498 learned_addr learned my addr 172.21.15.29:0/3103261498 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:28.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.123+0000 7f1472ffd640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14740b4b60 0x7f14741512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f1473fff640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f147415df70 0x7f14741517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f14737fe640 1 -- 172.21.15.29:0/3103261498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14740b4b60 msgr2=0x7f14741512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f14737fe640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14740b4b60 0x7f14741512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f14737fe640 1 -- 172.21.15.29:0/3103261498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f147415df70 msgr2=0x7f14741517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f14737fe640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f147415df70 0x7f14741517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f14737fe640 1 -- 172.21.15.29:0/3103261498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1474155a20 con 0x7f147406d210 2024-04-25T00:37:28.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f14737fe640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f147406d210 0x7f14741404c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f146801cad0 tx=0x7f146801aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:28.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f145bfff640 1 -- 172.21.15.29:0/3103261498 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14680b0040 con 0x7f147406d210 2024-04-25T00:37:28.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f145bfff640 1 -- 172.21.15.29:0/3103261498 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14680bc070 con 0x7f147406d210 2024-04-25T00:37:28.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f1479d79640 1 -- 172.21.15.29:0/3103261498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1474155d00 con 0x7f147406d210 2024-04-25T00:37:28.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f145bfff640 1 -- 172.21.15.29:0/3103261498 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14680b6880 con 0x7f147406d210 2024-04-25T00:37:28.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f1479d79640 1 -- 172.21.15.29:0/3103261498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14741562c0 con 0x7f147406d210 2024-04-25T00:37:28.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f1479d79640 1 -- 172.21.15.29:0/3103261498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1474069270 con 0x7f147406d210 2024-04-25T00:37:28.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f145bfff640 1 -- 172.21.15.29:0/3103261498 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f14680b6a20 con 0x7f147406d210 2024-04-25T00:37:28.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f145bfff640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f14600679e0 0x7f1460069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f145bfff640 1 -- 172.21.15.29:0/3103261498 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f14681203f0 con 0x7f147406d210 2024-04-25T00:37:28.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f1472ffd640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f14600679e0 0x7f1460069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.127+0000 7f1472ffd640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f14600679e0 0x7f1460069ea0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1464006fd0 tx=0x7f1464008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:28.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.131+0000 7f145bfff640 1 -- 172.21.15.29:0/3103261498 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1468016110 con 0x7f147406d210 2024-04-25T00:37:28.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.327+0000 7f1479d79640 1 -- 172.21.15.29:0/3103261498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1474155370 con 0x7f147406d210 2024-04-25T00:37:28.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.327+0000 7f145bfff640 1 -- 172.21.15.29:0/3103261498 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5189 (secure 0 0 0) 0x7f14680f0da0 con 0x7f147406d210 2024-04-25T00:37:28.332 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:28.332 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1884979435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1884979435},{"type":"v1","addr":"172.21.15.157:6833","nonce":1884979435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1394307674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1394307674},{"type":"v1","addr":"172.21.15.157:6835","nonce":1394307674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/3915943091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3915943091},{"type":"v1","addr":"172.21.15.29:6834","nonce":3915943091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:37:24.327819+0000","modified":"2024-04-25T00:37:27.319963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1444171533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1444171533},{"type":"v1","addr":"172.21.15.29:6835","nonce":1444171533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7896,"qdb_cluster":[7896]},"id":8}]} 2024-04-25T00:37:28.332 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 67 2024-04-25T00:37:28.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.331+0000 7f1479d79640 1 -- 172.21.15.29:0/3103261498 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f14600679e0 msgr2=0x7f1460069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.331+0000 7f1479d79640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f14600679e0 0x7f1460069ea0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1464006fd0 tx=0x7f1464008040 comp rx=0 tx=0).stop 2024-04-25T00:37:28.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.331+0000 7f1479d79640 1 -- 172.21.15.29:0/3103261498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f147406d210 msgr2=0x7f14741404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.331+0000 7f1479d79640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f147406d210 0x7f14741404c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f146801cad0 tx=0x7f146801aa70 comp rx=0 tx=0).stop 2024-04-25T00:37:28.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.331+0000 7f1479d79640 1 -- 172.21.15.29:0/3103261498 shutdown_connections 2024-04-25T00:37:28.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.331+0000 7f1479d79640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f14600679e0 0x7f1460069ea0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.331+0000 7f1479d79640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f147415df70 0x7f14741517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.331+0000 7f1479d79640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14740b4b60 0x7f14741512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.331+0000 7f1479d79640 1 --2- 172.21.15.29:0/3103261498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f147406d210 0x7f14741404c0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.331+0000 7f1479d79640 1 -- 172.21.15.29:0/3103261498 >> 172.21.15.29:0/3103261498 conn(0x7f14740b1870 msgr2=0x7f147415c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:28.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.331+0000 7f1479d79640 1 -- 172.21.15.29:0/3103261498 shutdown_connections 2024-04-25T00:37:28.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.331+0000 7f1479d79640 1 -- 172.21.15.29:0/3103261498 wait complete. 2024-04-25T00:37:28.350 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:37:28.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7fa13640 1 Processor -- start 2024-04-25T00:37:28.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7fa13640 1 -- start start 2024-04-25T00:37:28.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7fa13640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9a7815d980 0x7f9a7806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7fa13640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9a780b2b70 0x7f9a7806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7fa13640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a78151950 con 0x7f9a78150df0 2024-04-25T00:37:28.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7fa13640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9a780cf560 con 0x7f9a7815d980 2024-04-25T00:37:28.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7fa13640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9a780b3130 con 0x7f9a780b2b70 2024-04-25T00:37:28.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7d7af640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9a7815d980 0x7f9a7806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7d7af640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9a7815d980 0x7f9a7806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43574/0 (socket says 172.21.15.29:43574) 2024-04-25T00:37:28.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7dfb0640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9a78150df0 0x7f9a78151240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57994/0 (socket says 172.21.15.29:57994) 2024-04-25T00:37:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7d7af640 1 -- 172.21.15.29:0/875666897 learned_addr learned my addr 172.21.15.29:0/875666897 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7cfae640 1 --2- 172.21.15.29:0/875666897 >> v2:172.21.15.157:3300/0 conn(0x7f9a780b2b70 0x7f9a7806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7d7af640 1 -- 172.21.15.29:0/875666897 >> v2:172.21.15.157:3300/0 conn(0x7f9a780b2b70 msgr2=0x7f9a7806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7d7af640 1 --2- 172.21.15.29:0/875666897 >> v2:172.21.15.157:3300/0 conn(0x7f9a780b2b70 0x7f9a7806db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7d7af640 1 -- 172.21.15.29:0/875666897 >> v1:172.21.15.29:6789/0 conn(0x7f9a78150df0 legacy=0x7f9a78151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7d7af640 1 -- 172.21.15.29:0/875666897 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a780b3410 con 0x7f9a7815d980 2024-04-25T00:37:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a77fff640 1 -- 172.21.15.29:0/875666897 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1800954522 0 0) 0x7f9a78151950 con 0x7f9a78150df0 2024-04-25T00:37:28.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.459+0000 7f9a7cfae640 1 --2- 172.21.15.29:0/875666897 >> v2:172.21.15.157:3300/0 conn(0x7f9a780b2b70 0x7f9a7806db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7d7af640 1 --2- 172.21.15.29:0/875666897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7815d980 0x7f9a7806d5d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9a780b2720 tx=0x7f9a6801ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d343d568c7d3d48c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:28.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a77fff640 1 -- 172.21.15.29:0/875666897 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800b2d0 con 0x7f9a7815d980 2024-04-25T00:37:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a77fff640 1 -- 172.21.15.29:0/875666897 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a6800b470 con 0x7f9a7815d980 2024-04-25T00:37:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a77fff640 1 -- 172.21.15.29:0/875666897 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a680a7c30 con 0x7f9a7815d980 2024-04-25T00:37:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 -- 172.21.15.29:0/875666897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7815d980 msgr2=0x7f9a7806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 --2- 172.21.15.29:0/875666897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7815d980 0x7f9a7806d5d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9a780b2720 tx=0x7f9a6801ebc0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 -- 172.21.15.29:0/875666897 shutdown_connections 2024-04-25T00:37:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 --2- 172.21.15.29:0/875666897 >> v2:172.21.15.157:3300/0 conn(0x7f9a780b2b70 0x7f9a7806db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 --2- 172.21.15.29:0/875666897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7815d980 0x7f9a7806d5d0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 -- 172.21.15.29:0/875666897 >> 172.21.15.29:0/875666897 conn(0x7f9a780b1870 msgr2=0x7f9a780b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 -- 172.21.15.29:0/875666897 shutdown_connections 2024-04-25T00:37:28.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 -- 172.21.15.29:0/875666897 wait complete. 2024-04-25T00:37:28.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 Processor -- start 2024-04-25T00:37:28.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 -- start start 2024-04-25T00:37:28.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a780b2b70 0x7f9a781faa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a78150df0 0x7f9a781406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7815d980 0x7f9a780ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7815b4b0 con 0x7f9a7815d980 2024-04-25T00:37:28.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7815dd60 con 0x7f9a780b2b70 2024-04-25T00:37:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9a7815b630 con 0x7f9a78150df0 2024-04-25T00:37:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7dfb0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7815d980 0x7f9a780ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7dfb0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7815d980 0x7f9a780ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43586/0 (socket says 172.21.15.29:43586) 2024-04-25T00:37:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7dfb0640 1 -- 172.21.15.29:0/568515086 learned_addr learned my addr 172.21.15.29:0/568515086 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7d7af640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a780b2b70 0x7f9a781faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7cfae640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a78150df0 0x7f9a781406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7dfb0640 1 -- 172.21.15.29:0/568515086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a78150df0 msgr2=0x7f9a781406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7dfb0640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a78150df0 0x7f9a781406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7dfb0640 1 -- 172.21.15.29:0/568515086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a780b2b70 msgr2=0x7f9a781faa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7dfb0640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a780b2b70 0x7f9a781faa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7dfb0640 1 -- 172.21.15.29:0/568515086 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a78140c00 con 0x7f9a7815d980 2024-04-25T00:37:28.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7cfae640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a78150df0 0x7f9a781406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7dfb0640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7815d980 0x7f9a780ae790 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000c450 tx=0x7f9a7000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a75ffb640 1 -- 172.21.15.29:0/568515086 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a700074e0 con 0x7f9a7815d980 2024-04-25T00:37:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a75ffb640 1 -- 172.21.15.29:0/568515086 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a700047f0 con 0x7f9a7815d980 2024-04-25T00:37:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 -- 172.21.15.29:0/568515086 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a781f9fd0 con 0x7f9a7815d980 2024-04-25T00:37:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a75ffb640 1 -- 172.21.15.29:0/568515086 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a700076a0 con 0x7f9a7815d980 2024-04-25T00:37:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.463+0000 7f9a7fa13640 1 -- 172.21.15.29:0/568515086 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a781fa270 con 0x7f9a7815d980 2024-04-25T00:37:28.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.467+0000 7f9a7fa13640 1 -- 172.21.15.29:0/568515086 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a78069270 con 0x7f9a7815d980 2024-04-25T00:37:28.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.467+0000 7f9a75ffb640 1 -- 172.21.15.29:0/568515086 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f9a7001a020 con 0x7f9a7815d980 2024-04-25T00:37:28.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.467+0000 7f9a75ffb640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a64067a10 0x7f9a64069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.467+0000 7f9a7d7af640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a64067a10 0x7f9a64069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.467+0000 7f9a75ffb640 1 -- 172.21.15.29:0/568515086 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f9a700855e0 con 0x7f9a7815d980 2024-04-25T00:37:28.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.467+0000 7f9a7d7af640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a64067a10 0x7f9a64069ed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9a680b0000 tx=0x7f9a680a9070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:28.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.471+0000 7f9a75ffb640 1 -- 172.21.15.29:0/568515086 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9a70056010 con 0x7f9a7815d980 2024-04-25T00:37:28.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.627+0000 7f9a7fa13640 1 -- 172.21.15.29:0/568515086 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a7806e1d0 con 0x7f9a7815d980 2024-04-25T00:37:28.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.627+0000 7f9a75ffb640 1 -- 172.21.15.29:0/568515086 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16579 (secure 0 0 0) 0x7f9a70055910 con 0x7f9a7815d980 2024-04-25T00:37:28.632 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:28.632 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":103,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:37:25.609914+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:37:22.602795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:37:23.310674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:37:28.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.631+0000 7f9a7fa13640 1 -- 172.21.15.29:0/568515086 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a64067a10 msgr2=0x7f9a64069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.631+0000 7f9a7fa13640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a64067a10 0x7f9a64069ed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9a680b0000 tx=0x7f9a680a9070 comp rx=0 tx=0).stop 2024-04-25T00:37:28.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.631+0000 7f9a7fa13640 1 -- 172.21.15.29:0/568515086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7815d980 msgr2=0x7f9a780ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.631+0000 7f9a7fa13640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7815d980 0x7f9a780ae790 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000c450 tx=0x7f9a7000c910 comp rx=0 tx=0).stop 2024-04-25T00:37:28.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.631+0000 7f9a7fa13640 1 -- 172.21.15.29:0/568515086 shutdown_connections 2024-04-25T00:37:28.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.631+0000 7f9a7fa13640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a64067a10 0x7f9a64069ed0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.631+0000 7f9a7fa13640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a7815d980 0x7f9a780ae790 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.631+0000 7f9a7fa13640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a78150df0 0x7f9a781406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.631+0000 7f9a7fa13640 1 --2- 172.21.15.29:0/568515086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a780b2b70 0x7f9a781faa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.631+0000 7f9a7fa13640 1 -- 172.21.15.29:0/568515086 >> 172.21.15.29:0/568515086 conn(0x7f9a780b1870 msgr2=0x7f9a780b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:28.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.631+0000 7f9a7fa13640 1 -- 172.21.15.29:0/568515086 shutdown_connections 2024-04-25T00:37:28.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.631+0000 7f9a7fa13640 1 -- 172.21.15.29:0/568515086 wait complete. 2024-04-25T00:37:28.649 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:37:28.649 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:37:28.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac28c1640 1 Processor -- start 2024-04-25T00:37:28.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac28c1640 1 -- start start 2024-04-25T00:37:28.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac28c1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feabc0b4d30 0x7feabc0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac28c1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feabc06d210 0x7feabc0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac28c1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feabc06d980 con 0x7feabc15df70 2024-04-25T00:37:28.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac28c1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7feabc06dc60 con 0x7feabc06d210 2024-04-25T00:37:28.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac28c1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7feabc06df40 con 0x7feabc0b4d30 2024-04-25T00:37:28.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabbfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7feabc15df70 0x7feabc0b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58008/0 (socket says 172.21.15.29:58008) 2024-04-25T00:37:28.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabbfff640 1 -- 172.21.15.29:0/639516940 learned_addr learned my addr 172.21.15.29:0/639516940 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac0e5e640 1 --2- 172.21.15.29:0/639516940 >> v2:172.21.15.157:3300/0 conn(0x7feabc06d210 0x7feabc0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 977353486 0 0) 0x7feabc06d980 con 0x7feabc15df70 2024-04-25T00:37:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabb7fe640 1 --2- 172.21.15.29:0/639516940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feabc0b4d30 0x7feabc0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feaa8003530 con 0x7feabc15df70 2024-04-25T00:37:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1943107194 0 0) 0x7feaa8003530 con 0x7feabc15df70 2024-04-25T00:37:28.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feabc0b4d30 msgr2=0x7feabc0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 --2- 172.21.15.29:0/639516940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feabc0b4d30 0x7feabc0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 >> v2:172.21.15.157:3300/0 conn(0x7feabc06d210 msgr2=0x7feabc0ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 --2- 172.21.15.29:0/639516940 >> v2:172.21.15.157:3300/0 conn(0x7feabc06d210 0x7feabc0ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc06e220 con 0x7feabc15df70 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7feaa4001680 con 0x7feabc15df70 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 >> v1:172.21.15.29:6789/0 conn(0x7feabc15df70 legacy=0x7feabc0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 --2- 172.21.15.29:0/639516940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feabc0b4d30 0x7feaa80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 --2- 172.21.15.29:0/639516940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaa8003f40 0x7feaa8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 --2- 172.21.15.29:0/639516940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa8004d50 0x7feaa80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feabc06dc60 con 0x7feaa8004d50 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feabc06df40 con 0x7feaa8003f40 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7feaa800b9d0 con 0x7feabc0b4d30 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac0e5e640 1 --2- 172.21.15.29:0/639516940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa8004d50 0x7feaa80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabbfff640 1 --2- 172.21.15.29:0/639516940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feabc0b4d30 0x7feaa80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feabb7fe640 1 --2- 172.21.15.29:0/639516940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaa8003f40 0x7feaa8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac0e5e640 1 -- 172.21.15.29:0/639516940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feabc0b4d30 msgr2=0x7feaa80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:28.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac0e5e640 1 --2- 172.21.15.29:0/639516940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feabc0b4d30 0x7feaa80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac0e5e640 1 -- 172.21.15.29:0/639516940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaa8003f40 msgr2=0x7feaa8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:28.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac0e5e640 1 --2- 172.21.15.29:0/639516940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaa8003f40 0x7feaa8004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.771+0000 7feac0e5e640 1 -- 172.21.15.29:0/639516940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa8003530 con 0x7feaa8004d50 2024-04-25T00:37:28.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.775+0000 7feac0e5e640 1 --2- 172.21.15.29:0/639516940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa8004d50 0x7feaa80051d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7feab4009ae0 tx=0x7feab401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5847fd125b79c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:28.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.775+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab40b2070 con 0x7feaa8004d50 2024-04-25T00:37:28.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.775+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab40a56a0 con 0x7feaa8004d50 2024-04-25T00:37:28.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.775+0000 7feabaffd640 1 -- 172.21.15.29:0/639516940 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab40096b0 con 0x7feaa8004d50 2024-04-25T00:37:28.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.775+0000 7feac28c1640 1 -- 172.21.15.29:0/639516940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa8004d50 msgr2=0x7feaa80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.775+0000 7feac28c1640 1 --2- 172.21.15.29:0/639516940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa8004d50 0x7feaa80051d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7feab4009ae0 tx=0x7feab401ca50 comp rx=0 tx=0).stop 2024-04-25T00:37:28.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.775+0000 7feac0e5e640 1 -- 172.21.15.29:0/639516940 reap_dead start 2024-04-25T00:37:28.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.775+0000 7feac28c1640 1 -- 172.21.15.29:0/639516940 shutdown_connections 2024-04-25T00:37:28.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.775+0000 7feac28c1640 1 -- 172.21.15.29:0/639516940 >> 172.21.15.29:0/639516940 conn(0x7feabc0b1870 msgr2=0x7feabc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:28.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.775+0000 7feac28c1640 1 -- 172.21.15.29:0/639516940 shutdown_connections 2024-04-25T00:37:28.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.775+0000 7feac28c1640 1 -- 172.21.15.29:0/639516940 wait complete. 2024-04-25T00:37:28.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.775+0000 7feac28c1640 1 Processor -- start 2024-04-25T00:37:28.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.775+0000 7feac28c1640 1 -- start start 2024-04-25T00:37:28.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feac28c1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feaa8004d50 0x7feabc0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feac28c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feabc14e6a0 0x7feabc0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feac28c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feabc0c7310 0x7feabc0c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feac28c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feabc15e350 con 0x7feabc14e6a0 2024-04-25T00:37:28.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feac28c1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feabc06d680 con 0x7feabc0c7310 2024-04-25T00:37:28.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feac28c1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7feabc06ea20 con 0x7feaa8004d50 2024-04-25T00:37:28.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feabb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feabc14e6a0 0x7feabc0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feabb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feabc14e6a0 0x7feabc0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43598/0 (socket says 172.21.15.29:43598) 2024-04-25T00:37:28.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feabb7fe640 1 -- 172.21.15.29:0/4218194326 learned_addr learned my addr 172.21.15.29:0/4218194326 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:28.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feabbfff640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feaa8004d50 0x7feabc0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feac0e5e640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feabc0c7310 0x7feabc0c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feabb7fe640 1 -- 172.21.15.29:0/4218194326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feaa8004d50 msgr2=0x7feabc0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feabb7fe640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feaa8004d50 0x7feabc0ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feabb7fe640 1 -- 172.21.15.29:0/4218194326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feabc0c7310 msgr2=0x7feabc0c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feabb7fe640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feabc0c7310 0x7feabc0c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feabb7fe640 1 -- 172.21.15.29:0/4218194326 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc0cc830 con 0x7feabc14e6a0 2024-04-25T00:37:28.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feabb7fe640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feabc14e6a0 0x7feabc0c6dd0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7feaac00f350 tx=0x7feaac00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:28.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feab8ff9640 1 -- 172.21.15.29:0/4218194326 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac01c070 con 0x7feabc14e6a0 2024-04-25T00:37:28.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feab8ff9640 1 -- 172.21.15.29:0/4218194326 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaac017620 con 0x7feabc14e6a0 2024-04-25T00:37:28.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feac28c1640 1 -- 172.21.15.29:0/4218194326 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feabc0ccb10 con 0x7feabc14e6a0 2024-04-25T00:37:28.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feab8ff9640 1 -- 172.21.15.29:0/4218194326 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac017a10 con 0x7feabc14e6a0 2024-04-25T00:37:28.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feac28c1640 1 -- 172.21.15.29:0/4218194326 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feabc0cd0d0 con 0x7feabc14e6a0 2024-04-25T00:37:28.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feac28c1640 1 -- 172.21.15.29:0/4218194326 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feabc069270 con 0x7feabc14e6a0 2024-04-25T00:37:28.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feab8ff9640 1 -- 172.21.15.29:0/4218194326 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7feaac01f070 con 0x7feabc14e6a0 2024-04-25T00:37:28.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feab8ff9640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fea9c0679e0 0x7fea9c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:28.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feab8ff9640 1 -- 172.21.15.29:0/4218194326 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7feaac086230 con 0x7feabc14e6a0 2024-04-25T00:37:28.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.779+0000 7feabbfff640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fea9c0679e0 0x7fea9c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:28.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.783+0000 7feabbfff640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fea9c0679e0 0x7fea9c069ea0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feaa4000fc0 tx=0x7feaa4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:28.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.783+0000 7feab8ff9640 1 -- 172.21.15.29:0/4218194326 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feaac012070 con 0x7feabc14e6a0 2024-04-25T00:37:28.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.943+0000 7feac28c1640 1 -- 172.21.15.29:0/4218194326 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feabc0d0780 con 0x7fea9c0679e0 2024-04-25T00:37:28.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.963+0000 7feab8ff9640 1 -- 172.21.15.29:0/4218194326 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247087 (secure 0 0 0) 0x7feabc0d0780 con 0x7fea9c0679e0 2024-04-25T00:37:28.967 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:28.969 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:37:28.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.967+0000 7feac28c1640 1 -- 172.21.15.29:0/4218194326 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fea9c0679e0 msgr2=0x7fea9c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.967+0000 7feac28c1640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fea9c0679e0 0x7fea9c069ea0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feaa4000fc0 tx=0x7feaa4009040 comp rx=0 tx=0).stop 2024-04-25T00:37:28.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.967+0000 7feac28c1640 1 -- 172.21.15.29:0/4218194326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feabc14e6a0 msgr2=0x7feabc0c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:28.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.967+0000 7feac28c1640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feabc14e6a0 0x7feabc0c6dd0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7feaac00f350 tx=0x7feaac00f810 comp rx=0 tx=0).stop 2024-04-25T00:37:28.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.967+0000 7feac28c1640 1 -- 172.21.15.29:0/4218194326 shutdown_connections 2024-04-25T00:37:28.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.967+0000 7feac28c1640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fea9c0679e0 0x7fea9c069ea0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.967+0000 7feac28c1640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feabc0c7310 0x7feabc0c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.967+0000 7feac28c1640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feabc14e6a0 0x7feabc0c6dd0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.967+0000 7feac28c1640 1 --2- 172.21.15.29:0/4218194326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feaa8004d50 0x7feabc0ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:28.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.967+0000 7feac28c1640 1 -- 172.21.15.29:0/4218194326 >> 172.21.15.29:0/4218194326 conn(0x7feabc0b1870 msgr2=0x7feabc0adc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:28.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.967+0000 7feac28c1640 1 -- 172.21.15.29:0/4218194326 shutdown_connections 2024-04-25T00:37:28.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:28.967+0000 7feac28c1640 1 -- 172.21.15.29:0/4218194326 wait complete. 2024-04-25T00:37:28.986 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-04-25T00:37:27.331634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289496,"kb_used_data":10924,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754685224,"statfs":{"total":773094113280,"available":772797669376,"internally_reserved":0,"allocated":11186176,"data_stored":5545569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.396659"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.963576+0000","last_change":"2024-04-25T00:37:23.963576+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.963576+0000","last_undegraded":"2024-04-25T00:37:23.963576+0000","last_fullsized":"2024-04-25T00:37:23.963576+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.950282+0000","last_change":"2024-04-25T00:37:23.950282+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.950282+0000","last_undegraded":"2024-04-25T00:37:23.950282+0000","last_fullsized":"2024-04-25T00:37:23.950282+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.966291+0000","last_change":"2024-04-25T00:37:23.966291+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.966291+0000","last_undegraded":"2024-04-25T00:37:23.966291+0000","last_fullsized":"2024-04-25T00:37:23.966291+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.963777+0000","last_change":"2024-04-25T00:37:23.963777+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.963777+0000","last_undegraded":"2024-04-25T00:37:23.963777+0000","last_fullsized":"2024-04-25T00:37:23.963777+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951734+0000","last_change":"2024-04-25T00:37:23.951734+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951734+0000","last_undegraded":"2024-04-25T00:37:23.951734+0000","last_fullsized":"2024-04-25T00:37:23.951734+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.954303+0000","last_change":"2024-04-25T00:37:23.954303+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.954303+0000","last_undegraded":"2024-04-25T00:37:23.954303+0000","last_fullsized":"2024-04-25T00:37:23.954303+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957857+0000","last_change":"2024-04-25T00:37:23.957857+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957857+0000","last_undegraded":"2024-04-25T00:37:23.957857+0000","last_fullsized":"2024-04-25T00:37:23.957857+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.964892+0000","last_change":"2024-04-25T00:37:23.964892+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.964892+0000","last_undegraded":"2024-04-25T00:37:23.964892+0000","last_fullsized":"2024-04-25T00:37:23.964892+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.963074+0000","last_change":"2024-04-25T00:37:23.963074+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.963074+0000","last_undegraded":"2024-04-25T00:37:23.963074+0000","last_fullsized":"2024-04-25T00:37:23.963074+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.962966+0000","last_change":"2024-04-25T00:37:23.962966+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.962966+0000","last_undegraded":"2024-04-25T00:37:23.962966+0000","last_fullsized":"2024-04-25T00:37:23.962966+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.949178+0000","last_change":"2024-04-25T00:37:23.949178+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.949178+0000","last_undegraded":"2024-04-25T00:37:23.949178+0000","last_fullsized":"2024-04-25T00:37:23.949178+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.963247+0000","last_change":"2024-04-25T00:37:23.963247+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.963247+0000","last_undegraded":"2024-04-25T00:37:23.963247+0000","last_fullsized":"2024-04-25T00:37:23.963247+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.964817+0000","last_change":"2024-04-25T00:37:23.964817+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.964817+0000","last_undegraded":"2024-04-25T00:37:23.964817+0000","last_fullsized":"2024-04-25T00:37:23.964817+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.965283+0000","last_change":"2024-04-25T00:37:23.965283+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.965283+0000","last_undegraded":"2024-04-25T00:37:23.965283+0000","last_fullsized":"2024-04-25T00:37:23.965283+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957419+0000","last_change":"2024-04-25T00:37:23.957419+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957419+0000","last_undegraded":"2024-04-25T00:37:23.957419+0000","last_fullsized":"2024-04-25T00:37:23.957419+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.955558+0000","last_change":"2024-04-25T00:37:23.955558+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.955558+0000","last_undegraded":"2024-04-25T00:37:23.955558+0000","last_fullsized":"2024-04-25T00:37:23.955558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.950919+0000","last_change":"2024-04-25T00:37:23.950919+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.950919+0000","last_undegraded":"2024-04-25T00:37:23.950919+0000","last_fullsized":"2024-04-25T00:37:23.950919+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.960887+0000","last_change":"2024-04-25T00:37:23.960887+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.960887+0000","last_undegraded":"2024-04-25T00:37:23.960887+0000","last_fullsized":"2024-04-25T00:37:23.960887+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.953618+0000","last_change":"2024-04-25T00:37:23.953618+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.953618+0000","last_undegraded":"2024-04-25T00:37:23.953618+0000","last_fullsized":"2024-04-25T00:37:23.953618+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.949114+0000","last_change":"2024-04-25T00:37:23.949114+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.949114+0000","last_undegraded":"2024-04-25T00:37:23.949114+0000","last_fullsized":"2024-04-25T00:37:23.949114+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.949966+0000","last_change":"2024-04-25T00:37:23.949966+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.949966+0000","last_undegraded":"2024-04-25T00:37:23.949966+0000","last_fullsized":"2024-04-25T00:37:23.949966+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952217+0000","last_change":"2024-04-25T00:37:23.952217+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952217+0000","last_undegraded":"2024-04-25T00:37:23.952217+0000","last_fullsized":"2024-04-25T00:37:23.952217+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.953829+0000","last_change":"2024-04-25T00:37:23.953829+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.953829+0000","last_undegraded":"2024-04-25T00:37:23.953829+0000","last_fullsized":"2024-04-25T00:37:23.953829+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952069+0000","last_change":"2024-04-25T00:37:23.952069+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952069+0000","last_undegraded":"2024-04-25T00:37:23.952069+0000","last_fullsized":"2024-04-25T00:37:23.952069+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.949141+0000","last_change":"2024-04-25T00:37:23.949141+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.949141+0000","last_undegraded":"2024-04-25T00:37:23.949141+0000","last_fullsized":"2024-04-25T00:37:23.949141+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957875+0000","last_change":"2024-04-25T00:37:23.957875+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957875+0000","last_undegraded":"2024-04-25T00:37:23.957875+0000","last_fullsized":"2024-04-25T00:37:23.957875+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.956611+0000","last_change":"2024-04-25T00:37:23.956611+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.956611+0000","last_undegraded":"2024-04-25T00:37:23.956611+0000","last_fullsized":"2024-04-25T00:37:23.956611+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.955228+0000","last_change":"2024-04-25T00:37:23.955228+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.955228+0000","last_undegraded":"2024-04-25T00:37:23.955228+0000","last_fullsized":"2024-04-25T00:37:23.955228+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.956067+0000","last_change":"2024-04-25T00:37:23.956067+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.956067+0000","last_undegraded":"2024-04-25T00:37:23.956067+0000","last_fullsized":"2024-04-25T00:37:23.956067+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.953026+0000","last_change":"2024-04-25T00:37:23.953026+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.953026+0000","last_undegraded":"2024-04-25T00:37:23.953026+0000","last_fullsized":"2024-04-25T00:37:23.953026+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957263+0000","last_change":"2024-04-25T00:37:23.957263+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957263+0000","last_undegraded":"2024-04-25T00:37:23.957263+0000","last_fullsized":"2024-04-25T00:37:23.957263+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952836+0000","last_change":"2024-04-25T00:37:23.952836+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952836+0000","last_undegraded":"2024-04-25T00:37:23.952836+0000","last_fullsized":"2024-04-25T00:37:23.952836+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.954441+0000","last_change":"2024-04-25T00:37:23.954441+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.954441+0000","last_undegraded":"2024-04-25T00:37:23.954441+0000","last_fullsized":"2024-04-25T00:37:23.954441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.984445+0000","last_change":"2024-04-25T00:37:23.981233+0000","last_active":"2024-04-25T00:37:23.984445+0000","last_peered":"2024-04-25T00:37:23.984445+0000","last_clean":"2024-04-25T00:37:23.984445+0000","last_became_active":"2024-04-25T00:37:23.980150+0000","last_became_peered":"2024-04-25T00:37:23.980150+0000","last_unstale":"2024-04-25T00:37:23.984445+0000","last_undegraded":"2024-04-25T00:37:23.984445+0000","last_fullsized":"2024-04-25T00:37:23.984445+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:38:19.934694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.956938+0000","last_change":"2024-04-25T00:37:23.956938+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.956938+0000","last_undegraded":"2024-04-25T00:37:23.956938+0000","last_fullsized":"2024-04-25T00:37:23.956938+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.979620+0000","last_change":"2024-04-25T00:37:23.974780+0000","last_active":"2024-04-25T00:37:23.979620+0000","last_peered":"2024-04-25T00:37:23.979620+0000","last_clean":"2024-04-25T00:37:23.979620+0000","last_became_active":"2024-04-25T00:37:23.973844+0000","last_became_peered":"2024-04-25T00:37:23.973844+0000","last_unstale":"2024-04-25T00:37:23.979620+0000","last_undegraded":"2024-04-25T00:37:23.979620+0000","last_fullsized":"2024-04-25T00:37:23.979620+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:23.689041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.958270+0000","last_change":"2024-04-25T00:37:23.958270+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.958270+0000","last_undegraded":"2024-04-25T00:37:23.958270+0000","last_fullsized":"2024-04-25T00:37:23.958270+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.964726+0000","last_change":"2024-04-25T00:37:23.962945+0000","last_active":"2024-04-25T00:37:23.964726+0000","last_peered":"2024-04-25T00:37:23.964726+0000","last_clean":"2024-04-25T00:37:23.964726+0000","last_became_active":"2024-04-25T00:37:23.962460+0000","last_became_peered":"2024-04-25T00:37:23.962460+0000","last_unstale":"2024-04-25T00:37:23.964726+0000","last_undegraded":"2024-04-25T00:37:23.964726+0000","last_fullsized":"2024-04-25T00:37:23.964726+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:09.261299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952456+0000","last_change":"2024-04-25T00:37:23.952456+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952456+0000","last_undegraded":"2024-04-25T00:37:23.952456+0000","last_fullsized":"2024-04-25T00:37:23.952456+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.968954+0000","last_change":"2024-04-25T00:37:23.967844+0000","last_active":"2024-04-25T00:37:23.968954+0000","last_peered":"2024-04-25T00:37:23.968954+0000","last_clean":"2024-04-25T00:37:23.968954+0000","last_became_active":"2024-04-25T00:37:23.967545+0000","last_became_peered":"2024-04-25T00:37:23.967545+0000","last_unstale":"2024-04-25T00:37:23.968954+0000","last_undegraded":"2024-04-25T00:37:23.968954+0000","last_fullsized":"2024-04-25T00:37:23.968954+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:55.377792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952448+0000","last_change":"2024-04-25T00:37:23.952448+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952448+0000","last_undegraded":"2024-04-25T00:37:23.952448+0000","last_fullsized":"2024-04-25T00:37:23.952448+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.978848+0000","last_change":"2024-04-25T00:37:23.971620+0000","last_active":"2024-04-25T00:37:23.978848+0000","last_peered":"2024-04-25T00:37:23.978848+0000","last_clean":"2024-04-25T00:37:23.978848+0000","last_became_active":"2024-04-25T00:37:23.970901+0000","last_became_peered":"2024-04-25T00:37:23.970901+0000","last_unstale":"2024-04-25T00:37:23.978848+0000","last_undegraded":"2024-04-25T00:37:23.978848+0000","last_fullsized":"2024-04-25T00:37:23.978848+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:13.119808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.955049+0000","last_change":"2024-04-25T00:37:23.955049+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.955049+0000","last_undegraded":"2024-04-25T00:37:23.955049+0000","last_fullsized":"2024-04-25T00:37:23.955049+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.976090+0000","last_change":"2024-04-25T00:37:23.974470+0000","last_active":"2024-04-25T00:37:23.976090+0000","last_peered":"2024-04-25T00:37:23.976090+0000","last_clean":"2024-04-25T00:37:23.976090+0000","last_became_active":"2024-04-25T00:37:23.974141+0000","last_became_peered":"2024-04-25T00:37:23.974141+0000","last_unstale":"2024-04-25T00:37:23.976090+0000","last_undegraded":"2024-04-25T00:37:23.976090+0000","last_fullsized":"2024-04-25T00:37:23.976090+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:23:20.555785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.955063+0000","last_change":"2024-04-25T00:37:23.955063+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.955063+0000","last_undegraded":"2024-04-25T00:37:23.955063+0000","last_fullsized":"2024-04-25T00:37:23.955063+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.962331+0000","last_change":"2024-04-25T00:37:23.960317+0000","last_active":"2024-04-25T00:37:23.962331+0000","last_peered":"2024-04-25T00:37:23.962331+0000","last_clean":"2024-04-25T00:37:23.962331+0000","last_became_active":"2024-04-25T00:37:23.959614+0000","last_became_peered":"2024-04-25T00:37:23.959614+0000","last_unstale":"2024-04-25T00:37:23.962331+0000","last_undegraded":"2024-04-25T00:37:23.962331+0000","last_fullsized":"2024-04-25T00:37:23.962331+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:09.024737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.958913+0000","last_change":"2024-04-25T00:37:23.958913+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.958913+0000","last_undegraded":"2024-04-25T00:37:23.958913+0000","last_fullsized":"2024-04-25T00:37:23.958913+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.974447+0000","last_change":"2024-04-25T00:37:23.971464+0000","last_active":"2024-04-25T00:37:23.974447+0000","last_peered":"2024-04-25T00:37:23.974447+0000","last_clean":"2024-04-25T00:37:23.974447+0000","last_became_active":"2024-04-25T00:37:23.970908+0000","last_became_peered":"2024-04-25T00:37:23.970908+0000","last_unstale":"2024-04-25T00:37:23.974447+0000","last_undegraded":"2024-04-25T00:37:23.974447+0000","last_fullsized":"2024-04-25T00:37:23.974447+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:56.021774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.956732+0000","last_change":"2024-04-25T00:37:23.956732+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.956732+0000","last_undegraded":"2024-04-25T00:37:23.956732+0000","last_fullsized":"2024-04-25T00:37:23.956732+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.964410+0000","last_change":"2024-04-25T00:37:23.962738+0000","last_active":"2024-04-25T00:37:23.964410+0000","last_peered":"2024-04-25T00:37:23.964410+0000","last_clean":"2024-04-25T00:37:23.964410+0000","last_became_active":"2024-04-25T00:37:23.962330+0000","last_became_peered":"2024-04-25T00:37:23.962330+0000","last_unstale":"2024-04-25T00:37:23.964410+0000","last_undegraded":"2024-04-25T00:37:23.964410+0000","last_fullsized":"2024-04-25T00:37:23.964410+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:44.729272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.955830+0000","last_change":"2024-04-25T00:37:23.955830+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.955830+0000","last_undegraded":"2024-04-25T00:37:23.955830+0000","last_fullsized":"2024-04-25T00:37:23.955830+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.976070+0000","last_change":"2024-04-25T00:37:23.973733+0000","last_active":"2024-04-25T00:37:23.976070+0000","last_peered":"2024-04-25T00:37:23.976070+0000","last_clean":"2024-04-25T00:37:23.976070+0000","last_became_active":"2024-04-25T00:37:23.973362+0000","last_became_peered":"2024-04-25T00:37:23.973362+0000","last_unstale":"2024-04-25T00:37:23.976070+0000","last_undegraded":"2024-04-25T00:37:23.976070+0000","last_fullsized":"2024-04-25T00:37:23.976070+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:50.821959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.954708+0000","last_change":"2024-04-25T00:37:23.954708+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.954708+0000","last_undegraded":"2024-04-25T00:37:23.954708+0000","last_fullsized":"2024-04-25T00:37:23.954708+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.972416+0000","last_change":"2024-04-25T00:37:23.966485+0000","last_active":"2024-04-25T00:37:23.972416+0000","last_peered":"2024-04-25T00:37:23.972416+0000","last_clean":"2024-04-25T00:37:23.972416+0000","last_became_active":"2024-04-25T00:37:23.966125+0000","last_became_peered":"2024-04-25T00:37:23.966125+0000","last_unstale":"2024-04-25T00:37:23.972416+0000","last_undegraded":"2024-04-25T00:37:23.972416+0000","last_fullsized":"2024-04-25T00:37:23.972416+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:51.405634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.959932+0000","last_change":"2024-04-25T00:37:23.959932+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.959932+0000","last_undegraded":"2024-04-25T00:37:23.959932+0000","last_fullsized":"2024-04-25T00:37:23.959932+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.974336+0000","last_change":"2024-04-25T00:37:23.971682+0000","last_active":"2024-04-25T00:37:23.974336+0000","last_peered":"2024-04-25T00:37:23.974336+0000","last_clean":"2024-04-25T00:37:23.974336+0000","last_became_active":"2024-04-25T00:37:23.970797+0000","last_became_peered":"2024-04-25T00:37:23.970797+0000","last_unstale":"2024-04-25T00:37:23.974336+0000","last_undegraded":"2024-04-25T00:37:23.974336+0000","last_fullsized":"2024-04-25T00:37:23.974336+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:02.162592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951746+0000","last_change":"2024-04-25T00:37:23.951746+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951746+0000","last_undegraded":"2024-04-25T00:37:23.951746+0000","last_fullsized":"2024-04-25T00:37:23.951746+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.980578+0000","last_change":"2024-04-25T00:37:23.980023+0000","last_active":"2024-04-25T00:37:23.980578+0000","last_peered":"2024-04-25T00:37:23.980578+0000","last_clean":"2024-04-25T00:37:23.980578+0000","last_became_active":"2024-04-25T00:37:23.979702+0000","last_became_peered":"2024-04-25T00:37:23.979702+0000","last_unstale":"2024-04-25T00:37:23.980578+0000","last_undegraded":"2024-04-25T00:37:23.980578+0000","last_fullsized":"2024-04-25T00:37:23.980578+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:03.459207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951412+0000","last_change":"2024-04-25T00:37:23.951412+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951412+0000","last_undegraded":"2024-04-25T00:37:23.951412+0000","last_fullsized":"2024-04-25T00:37:23.951412+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.981951+0000","last_change":"2024-04-25T00:37:23.974737+0000","last_active":"2024-04-25T00:37:23.981951+0000","last_peered":"2024-04-25T00:37:23.981951+0000","last_clean":"2024-04-25T00:37:23.981951+0000","last_became_active":"2024-04-25T00:37:23.974006+0000","last_became_peered":"2024-04-25T00:37:23.974006+0000","last_unstale":"2024-04-25T00:37:23.981951+0000","last_undegraded":"2024-04-25T00:37:23.981951+0000","last_fullsized":"2024-04-25T00:37:23.981951+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:43.160583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.955228+0000","last_change":"2024-04-25T00:37:23.955228+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.955228+0000","last_undegraded":"2024-04-25T00:37:23.955228+0000","last_fullsized":"2024-04-25T00:37:23.955228+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.973622+0000","last_change":"2024-04-25T00:37:23.972616+0000","last_active":"2024-04-25T00:37:23.973622+0000","last_peered":"2024-04-25T00:37:23.973622+0000","last_clean":"2024-04-25T00:37:23.973622+0000","last_became_active":"2024-04-25T00:37:23.972302+0000","last_became_peered":"2024-04-25T00:37:23.972302+0000","last_unstale":"2024-04-25T00:37:23.973622+0000","last_undegraded":"2024-04-25T00:37:23.973622+0000","last_fullsized":"2024-04-25T00:37:23.973622+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:53.826036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952853+0000","last_change":"2024-04-25T00:37:23.952853+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952853+0000","last_undegraded":"2024-04-25T00:37:23.952853+0000","last_fullsized":"2024-04-25T00:37:23.952853+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.975580+0000","last_change":"2024-04-25T00:37:23.973856+0000","last_active":"2024-04-25T00:37:23.975580+0000","last_peered":"2024-04-25T00:37:23.975580+0000","last_clean":"2024-04-25T00:37:23.975580+0000","last_became_active":"2024-04-25T00:37:23.973576+0000","last_became_peered":"2024-04-25T00:37:23.973576+0000","last_unstale":"2024-04-25T00:37:23.975580+0000","last_undegraded":"2024-04-25T00:37:23.975580+0000","last_fullsized":"2024-04-25T00:37:23.975580+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:08.824434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.953910+0000","last_change":"2024-04-25T00:37:23.953910+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.953910+0000","last_undegraded":"2024-04-25T00:37:23.953910+0000","last_fullsized":"2024-04-25T00:37:23.953910+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.964463+0000","last_change":"2024-04-25T00:37:23.963264+0000","last_active":"2024-04-25T00:37:23.964463+0000","last_peered":"2024-04-25T00:37:23.964463+0000","last_clean":"2024-04-25T00:37:23.964463+0000","last_became_active":"2024-04-25T00:37:23.962746+0000","last_became_peered":"2024-04-25T00:37:23.962746+0000","last_unstale":"2024-04-25T00:37:23.964463+0000","last_undegraded":"2024-04-25T00:37:23.964463+0000","last_fullsized":"2024-04-25T00:37:23.964463+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:06.004795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951573+0000","last_change":"2024-04-25T00:37:23.951573+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951573+0000","last_undegraded":"2024-04-25T00:37:23.951573+0000","last_fullsized":"2024-04-25T00:37:23.951573+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.973831+0000","last_change":"2024-04-25T00:37:23.969113+0000","last_active":"2024-04-25T00:37:23.973831+0000","last_peered":"2024-04-25T00:37:23.973831+0000","last_clean":"2024-04-25T00:37:23.973831+0000","last_became_active":"2024-04-25T00:37:23.968373+0000","last_became_peered":"2024-04-25T00:37:23.968373+0000","last_unstale":"2024-04-25T00:37:23.973831+0000","last_undegraded":"2024-04-25T00:37:23.973831+0000","last_fullsized":"2024-04-25T00:37:23.973831+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:44.008910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951193+0000","last_change":"2024-04-25T00:37:23.951193+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951193+0000","last_undegraded":"2024-04-25T00:37:23.951193+0000","last_fullsized":"2024-04-25T00:37:23.951193+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.983736+0000","last_change":"2024-04-25T00:37:23.975090+0000","last_active":"2024-04-25T00:37:23.983736+0000","last_peered":"2024-04-25T00:37:23.983736+0000","last_clean":"2024-04-25T00:37:23.983736+0000","last_became_active":"2024-04-25T00:37:23.973684+0000","last_became_peered":"2024-04-25T00:37:23.973684+0000","last_unstale":"2024-04-25T00:37:23.983736+0000","last_undegraded":"2024-04-25T00:37:23.983736+0000","last_fullsized":"2024-04-25T00:37:23.983736+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:34.935440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.953635+0000","last_change":"2024-04-25T00:37:23.953635+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.953635+0000","last_undegraded":"2024-04-25T00:37:23.953635+0000","last_fullsized":"2024-04-25T00:37:23.953635+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.967381+0000","last_change":"2024-04-25T00:37:23.962101+0000","last_active":"2024-04-25T00:37:23.967381+0000","last_peered":"2024-04-25T00:37:23.967381+0000","last_clean":"2024-04-25T00:37:23.967381+0000","last_became_active":"2024-04-25T00:37:23.961767+0000","last_became_peered":"2024-04-25T00:37:23.961767+0000","last_unstale":"2024-04-25T00:37:23.967381+0000","last_undegraded":"2024-04-25T00:37:23.967381+0000","last_fullsized":"2024-04-25T00:37:23.967381+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:45.756449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957228+0000","last_change":"2024-04-25T00:37:23.957228+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957228+0000","last_undegraded":"2024-04-25T00:37:23.957228+0000","last_fullsized":"2024-04-25T00:37:23.957228+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.964300+0000","last_change":"2024-04-25T00:37:23.962517+0000","last_active":"2024-04-25T00:37:23.964300+0000","last_peered":"2024-04-25T00:37:23.964300+0000","last_clean":"2024-04-25T00:37:23.964300+0000","last_became_active":"2024-04-25T00:37:23.962068+0000","last_became_peered":"2024-04-25T00:37:23.962068+0000","last_unstale":"2024-04-25T00:37:23.964300+0000","last_undegraded":"2024-04-25T00:37:23.964300+0000","last_fullsized":"2024-04-25T00:37:23.964300+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:38.445130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952486+0000","last_change":"2024-04-25T00:37:23.952486+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952486+0000","last_undegraded":"2024-04-25T00:37:23.952486+0000","last_fullsized":"2024-04-25T00:37:23.952486+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.963895+0000","last_change":"2024-04-25T00:37:23.962073+0000","last_active":"2024-04-25T00:37:23.963895+0000","last_peered":"2024-04-25T00:37:23.963895+0000","last_clean":"2024-04-25T00:37:23.963895+0000","last_became_active":"2024-04-25T00:37:23.961617+0000","last_became_peered":"2024-04-25T00:37:23.961617+0000","last_unstale":"2024-04-25T00:37:23.963895+0000","last_undegraded":"2024-04-25T00:37:23.963895+0000","last_fullsized":"2024-04-25T00:37:23.963895+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:26.276154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.950948+0000","last_change":"2024-04-25T00:37:23.950948+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.950948+0000","last_undegraded":"2024-04-25T00:37:23.950948+0000","last_fullsized":"2024-04-25T00:37:23.950948+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.975764+0000","last_change":"2024-04-25T00:37:23.974633+0000","last_active":"2024-04-25T00:37:23.975764+0000","last_peered":"2024-04-25T00:37:23.975764+0000","last_clean":"2024-04-25T00:37:23.975764+0000","last_became_active":"2024-04-25T00:37:23.974067+0000","last_became_peered":"2024-04-25T00:37:23.974067+0000","last_unstale":"2024-04-25T00:37:23.975764+0000","last_undegraded":"2024-04-25T00:37:23.975764+0000","last_fullsized":"2024-04-25T00:37:23.975764+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:31.191986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.959904+0000","last_change":"2024-04-25T00:37:23.959904+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.959904+0000","last_undegraded":"2024-04-25T00:37:23.959904+0000","last_fullsized":"2024-04-25T00:37:23.959904+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.976046+0000","last_change":"2024-04-25T00:37:23.973640+0000","last_active":"2024-04-25T00:37:23.976046+0000","last_peered":"2024-04-25T00:37:23.976046+0000","last_clean":"2024-04-25T00:37:23.976046+0000","last_became_active":"2024-04-25T00:37:23.973210+0000","last_became_peered":"2024-04-25T00:37:23.973210+0000","last_unstale":"2024-04-25T00:37:23.976046+0000","last_undegraded":"2024-04-25T00:37:23.976046+0000","last_fullsized":"2024-04-25T00:37:23.976046+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:23.689041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957953+0000","last_change":"2024-04-25T00:37:23.957953+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957953+0000","last_undegraded":"2024-04-25T00:37:23.957953+0000","last_fullsized":"2024-04-25T00:37:23.957953+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.982782+0000","last_change":"2024-04-25T00:37:23.976356+0000","last_active":"2024-04-25T00:37:23.982782+0000","last_peered":"2024-04-25T00:37:23.982782+0000","last_clean":"2024-04-25T00:37:23.982782+0000","last_became_active":"2024-04-25T00:37:23.975148+0000","last_became_peered":"2024-04-25T00:37:23.975148+0000","last_unstale":"2024-04-25T00:37:23.982782+0000","last_undegraded":"2024-04-25T00:37:23.982782+0000","last_fullsized":"2024-04-25T00:37:23.982782+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:08.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957683+0000","last_change":"2024-04-25T00:37:23.957683+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957683+0000","last_undegraded":"2024-04-25T00:37:23.957683+0000","last_fullsized":"2024-04-25T00:37:23.957683+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.975969+0000","last_change":"2024-04-25T00:37:23.975126+0000","last_active":"2024-04-25T00:37:23.975969+0000","last_peered":"2024-04-25T00:37:23.975969+0000","last_clean":"2024-04-25T00:37:23.975969+0000","last_became_active":"2024-04-25T00:37:23.974571+0000","last_became_peered":"2024-04-25T00:37:23.974571+0000","last_unstale":"2024-04-25T00:37:23.975969+0000","last_undegraded":"2024-04-25T00:37:23.975969+0000","last_fullsized":"2024-04-25T00:37:23.975969+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:37.137351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.956677+0000","last_change":"2024-04-25T00:37:23.956677+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.956677+0000","last_undegraded":"2024-04-25T00:37:23.956677+0000","last_fullsized":"2024-04-25T00:37:23.956677+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.964294+0000","last_change":"2024-04-25T00:37:23.962739+0000","last_active":"2024-04-25T00:37:23.964294+0000","last_peered":"2024-04-25T00:37:23.964294+0000","last_clean":"2024-04-25T00:37:23.964294+0000","last_became_active":"2024-04-25T00:37:23.962425+0000","last_became_peered":"2024-04-25T00:37:23.962425+0000","last_unstale":"2024-04-25T00:37:23.964294+0000","last_undegraded":"2024-04-25T00:37:23.964294+0000","last_fullsized":"2024-04-25T00:37:23.964294+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:06.834610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.949912+0000","last_change":"2024-04-25T00:37:23.949912+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.949912+0000","last_undegraded":"2024-04-25T00:37:23.949912+0000","last_fullsized":"2024-04-25T00:37:23.949912+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.983791+0000","last_change":"2024-04-25T00:37:23.976960+0000","last_active":"2024-04-25T00:37:23.983791+0000","last_peered":"2024-04-25T00:37:23.983791+0000","last_clean":"2024-04-25T00:37:23.983791+0000","last_became_active":"2024-04-25T00:37:23.971621+0000","last_became_peered":"2024-04-25T00:37:23.971621+0000","last_unstale":"2024-04-25T00:37:23.983791+0000","last_undegraded":"2024-04-25T00:37:23.983791+0000","last_fullsized":"2024-04-25T00:37:23.983791+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:51.829250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.953895+0000","last_change":"2024-04-25T00:37:23.953895+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.953895+0000","last_undegraded":"2024-04-25T00:37:23.953895+0000","last_fullsized":"2024-04-25T00:37:23.953895+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.971517+0000","last_change":"2024-04-25T00:37:23.963951+0000","last_active":"2024-04-25T00:37:23.971517+0000","last_peered":"2024-04-25T00:37:23.971517+0000","last_clean":"2024-04-25T00:37:23.971517+0000","last_became_active":"2024-04-25T00:37:23.963276+0000","last_became_peered":"2024-04-25T00:37:23.963276+0000","last_unstale":"2024-04-25T00:37:23.971517+0000","last_undegraded":"2024-04-25T00:37:23.971517+0000","last_fullsized":"2024-04-25T00:37:23.971517+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:40.780233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951430+0000","last_change":"2024-04-25T00:37:23.951430+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951430+0000","last_undegraded":"2024-04-25T00:37:23.951430+0000","last_fullsized":"2024-04-25T00:37:23.951430+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.968956+0000","last_change":"2024-04-25T00:37:23.968009+0000","last_active":"2024-04-25T00:37:23.968956+0000","last_peered":"2024-04-25T00:37:23.968956+0000","last_clean":"2024-04-25T00:37:23.968956+0000","last_became_active":"2024-04-25T00:37:23.967586+0000","last_became_peered":"2024-04-25T00:37:23.967586+0000","last_unstale":"2024-04-25T00:37:23.968956+0000","last_undegraded":"2024-04-25T00:37:23.968956+0000","last_fullsized":"2024-04-25T00:37:23.968956+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:00.626603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.956755+0000","last_change":"2024-04-25T00:37:23.956755+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.956755+0000","last_undegraded":"2024-04-25T00:37:23.956755+0000","last_fullsized":"2024-04-25T00:37:23.956755+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.975697+0000","last_change":"2024-04-25T00:37:23.974640+0000","last_active":"2024-04-25T00:37:23.975697+0000","last_peered":"2024-04-25T00:37:23.975697+0000","last_clean":"2024-04-25T00:37:23.975697+0000","last_became_active":"2024-04-25T00:37:23.974224+0000","last_became_peered":"2024-04-25T00:37:23.974224+0000","last_unstale":"2024-04-25T00:37:23.975697+0000","last_undegraded":"2024-04-25T00:37:23.975697+0000","last_fullsized":"2024-04-25T00:37:23.975697+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:41.853687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951624+0000","last_change":"2024-04-25T00:37:23.951624+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951624+0000","last_undegraded":"2024-04-25T00:37:23.951624+0000","last_fullsized":"2024-04-25T00:37:23.951624+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.983706+0000","last_change":"2024-04-25T00:37:23.974436+0000","last_active":"2024-04-25T00:37:23.983706+0000","last_peered":"2024-04-25T00:37:23.983706+0000","last_clean":"2024-04-25T00:37:23.983706+0000","last_became_active":"2024-04-25T00:37:23.973775+0000","last_became_peered":"2024-04-25T00:37:23.973775+0000","last_unstale":"2024-04-25T00:37:23.983706+0000","last_undegraded":"2024-04-25T00:37:23.983706+0000","last_fullsized":"2024-04-25T00:37:23.983706+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:45.785490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":363,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.947499+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:37:23.947499+0000","last_peered":"2024-04-25T00:37:23.947499+0000","last_clean":"2024-04-25T00:37:23.947499+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:37:23.947499+0000","last_undegraded":"2024-04-25T00:37:23.947499+0000","last_fullsized":"2024-04-25T00:37:23.947499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37152,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334688,"statfs":{"total":96636764160,"available":96598720512,"internally_reserved":0,"allocated":1851392,"data_stored":1139894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1245184,"data_stored":543251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":549526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36572,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335268,"statfs":{"total":96636764160,"available":96599314432,"internally_reserved":0,"allocated":1257472,"data_stored":549526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1245184,"data_stored":543251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1245184,"data_stored":543251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1245184,"data_stored":543251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37136,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334704,"statfs":{"total":96636764160,"available":96598736896,"internally_reserved":0,"allocated":1839104,"data_stored":1133619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:37:28.991 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:37:29.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.151+0000 7f68726db640 1 Processor -- start 2024-04-25T00:37:29.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.151+0000 7f68726db640 1 -- start start 2024-04-25T00:37:29.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.151+0000 7f68726db640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f686c0b3140 0x7f686c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.151+0000 7f68726db640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f686c0b4b10 0x7f686c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.151+0000 7f68726db640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f686c0b41e0 con 0x7f686c0b3a60 2024-04-25T00:37:29.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.151+0000 7f68726db640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f686c06d960 con 0x7f686c0b4b10 2024-04-25T00:37:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.151+0000 7f68726db640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f686c06dc40 con 0x7f686c0b3140 2024-04-25T00:37:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.155+0000 7f686b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f686c0b4b10 0x7f686c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.155+0000 7f686b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f686c0b4b10 0x7f686c0b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43610/0 (socket says 172.21.15.29:43610) 2024-04-25T00:37:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.155+0000 7f686b7fe640 1 -- 172.21.15.29:0/137255036 learned_addr learned my addr 172.21.15.29:0/137255036 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.155+0000 7f686bfff640 1 --2- 172.21.15.29:0/137255036 >> v2:172.21.15.157:3300/0 conn(0x7f686c0b3140 0x7f686c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.155+0000 7f686b7fe640 1 -- 172.21.15.29:0/137255036 >> v2:172.21.15.157:3300/0 conn(0x7f686c0b3140 msgr2=0x7f686c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.155+0000 7f686b7fe640 1 --2- 172.21.15.29:0/137255036 >> v2:172.21.15.157:3300/0 conn(0x7f686c0b3140 0x7f686c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.155+0000 7f686b7fe640 1 -- 172.21.15.29:0/137255036 >> v1:172.21.15.157:6789/0 conn(0x7f686c0b3a60 legacy=0x7f686c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.155+0000 7f686b7fe640 1 -- 172.21.15.29:0/137255036 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c06df20 con 0x7f686c0b4b10 2024-04-25T00:37:29.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f686b7fe640 1 --2- 172.21.15.29:0/137255036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b4b10 0x7f686c0b3520 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6860009870 tx=0x7f686001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7d8f0a867f8180 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f686affd640 1 -- 172.21.15.29:0/137255036 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600b0070 con 0x7f686c0b4b10 2024-04-25T00:37:29.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f686affd640 1 -- 172.21.15.29:0/137255036 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68600ab600 con 0x7f686c0b4b10 2024-04-25T00:37:29.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f686affd640 1 -- 172.21.15.29:0/137255036 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600a4e60 con 0x7f686c0b4b10 2024-04-25T00:37:29.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f68726db640 1 -- 172.21.15.29:0/137255036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b4b10 msgr2=0x7f686c0b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f68726db640 1 --2- 172.21.15.29:0/137255036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b4b10 0x7f686c0b3520 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6860009870 tx=0x7f686001c4e0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f68726db640 1 -- 172.21.15.29:0/137255036 shutdown_connections 2024-04-25T00:37:29.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f68726db640 1 --2- 172.21.15.29:0/137255036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b4b10 0x7f686c0b3520 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f68726db640 1 --2- 172.21.15.29:0/137255036 >> v2:172.21.15.157:3300/0 conn(0x7f686c0b3140 0x7f686c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f68726db640 1 -- 172.21.15.29:0/137255036 >> 172.21.15.29:0/137255036 conn(0x7f686c0b1870 msgr2=0x7f686c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:29.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f68726db640 1 -- 172.21.15.29:0/137255036 shutdown_connections 2024-04-25T00:37:29.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f68726db640 1 -- 172.21.15.29:0/137255036 wait complete. 2024-04-25T00:37:29.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f68726db640 1 Processor -- start 2024-04-25T00:37:29.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f68726db640 1 -- start start 2024-04-25T00:37:29.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.159+0000 7f68726db640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686c0b3140 0x7f686c0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f68726db640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b3a60 0x7f686c166040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f68726db640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0b4b10 0x7f686c138040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f68726db640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f686c168b70 con 0x7f686c0b3a60 2024-04-25T00:37:29.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f68726db640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f686c1689f0 con 0x7f686c0b3140 2024-04-25T00:37:29.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f68726db640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f686c168cf0 con 0x7f686c0b4b10 2024-04-25T00:37:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f686b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b3a60 0x7f686c166040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f686b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b3a60 0x7f686c166040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43624/0 (socket says 172.21.15.29:43624) 2024-04-25T00:37:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f686b7fe640 1 -- 172.21.15.29:0/2504049587 learned_addr learned my addr 172.21.15.29:0/2504049587 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f686b7fe640 1 -- 172.21.15.29:0/2504049587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0b4b10 msgr2=0x7f686c138040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f6870c78640 1 --2- 172.21.15.29:0/2504049587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0b4b10 0x7f686c138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f686b7fe640 1 --2- 172.21.15.29:0/2504049587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0b4b10 0x7f686c138040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f686b7fe640 1 -- 172.21.15.29:0/2504049587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686c0b3140 msgr2=0x7f686c0cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f686b7fe640 1 --2- 172.21.15.29:0/2504049587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686c0b3140 0x7f686c0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f686b7fe640 1 -- 172.21.15.29:0/2504049587 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c15c900 con 0x7f686c0b3a60 2024-04-25T00:37:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f686b7fe640 1 --2- 172.21.15.29:0/2504049587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b3a60 0x7f686c166040 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f68600ae040 tx=0x7f6860019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f6868ff9640 1 -- 172.21.15.29:0/2504049587 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600ba020 con 0x7f686c0b3a60 2024-04-25T00:37:29.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f6868ff9640 1 -- 172.21.15.29:0/2504049587 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68600b6420 con 0x7f686c0b3a60 2024-04-25T00:37:29.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f68726db640 1 -- 172.21.15.29:0/2504049587 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686c138580 con 0x7f686c0b3a60 2024-04-25T00:37:29.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f6868ff9640 1 -- 172.21.15.29:0/2504049587 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600b0040 con 0x7f686c0b3a60 2024-04-25T00:37:29.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f68726db640 1 -- 172.21.15.29:0/2504049587 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686c071820 con 0x7f686c0b3a60 2024-04-25T00:37:29.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f68726db640 1 -- 172.21.15.29:0/2504049587 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f686c069270 con 0x7f686c0b3a60 2024-04-25T00:37:29.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f6868ff9640 1 -- 172.21.15.29:0/2504049587 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f68600bd020 con 0x7f686c0b3a60 2024-04-25T00:37:29.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f6868ff9640 1 --2- 172.21.15.29:0/2504049587 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6858067a10 0x7f6858069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f6868ff9640 1 -- 172.21.15.29:0/2504049587 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f686011ffc0 con 0x7f686c0b3a60 2024-04-25T00:37:29.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f686bfff640 1 --2- 172.21.15.29:0/2504049587 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6858067a10 0x7f6858069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.163+0000 7f686bfff640 1 --2- 172.21.15.29:0/2504049587 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6858067a10 0x7f6858069ed0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f685c00b850 tx=0x7f685c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.167+0000 7f6868ff9640 1 -- 172.21.15.29:0/2504049587 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f68600c4030 con 0x7f686c0b3a60 2024-04-25T00:37:29.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.327+0000 7f68726db640 1 -- 172.21.15.29:0/2504049587 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f686c166580 con 0x7f6858067a10 2024-04-25T00:37:29.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.343+0000 7f6868ff9640 1 -- 172.21.15.29:0/2504049587 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247087 (secure 0 0 0) 0x7f686c166580 con 0x7f6858067a10 2024-04-25T00:37:29.349 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:29.351 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:37:29.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.347+0000 7f68726db640 1 -- 172.21.15.29:0/2504049587 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6858067a10 msgr2=0x7f6858069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.347+0000 7f68726db640 1 --2- 172.21.15.29:0/2504049587 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6858067a10 0x7f6858069ed0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f685c00b850 tx=0x7f685c008040 comp rx=0 tx=0).stop 2024-04-25T00:37:29.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.347+0000 7f68726db640 1 -- 172.21.15.29:0/2504049587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b3a60 msgr2=0x7f686c166040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.347+0000 7f68726db640 1 --2- 172.21.15.29:0/2504049587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b3a60 0x7f686c166040 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f68600ae040 tx=0x7f6860019280 comp rx=0 tx=0).stop 2024-04-25T00:37:29.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.347+0000 7f68726db640 1 -- 172.21.15.29:0/2504049587 shutdown_connections 2024-04-25T00:37:29.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.347+0000 7f68726db640 1 --2- 172.21.15.29:0/2504049587 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6858067a10 0x7f6858069ed0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.347+0000 7f68726db640 1 --2- 172.21.15.29:0/2504049587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686c0b4b10 0x7f686c138040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.347+0000 7f68726db640 1 --2- 172.21.15.29:0/2504049587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f686c0b3a60 0x7f686c166040 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.347+0000 7f68726db640 1 --2- 172.21.15.29:0/2504049587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686c0b3140 0x7f686c0cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.347+0000 7f68726db640 1 -- 172.21.15.29:0/2504049587 >> 172.21.15.29:0/2504049587 conn(0x7f686c0b1870 msgr2=0x7f686c1409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:29.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.347+0000 7f68726db640 1 -- 172.21.15.29:0/2504049587 shutdown_connections 2024-04-25T00:37:29.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.347+0000 7f68726db640 1 -- 172.21.15.29:0/2504049587 wait complete. 2024-04-25T00:37:29.368 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-04-25T00:37:27.331634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289496,"kb_used_data":10924,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754685224,"statfs":{"total":773094113280,"available":772797669376,"internally_reserved":0,"allocated":11186176,"data_stored":5545569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.396659"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.963576+0000","last_change":"2024-04-25T00:37:23.963576+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.963576+0000","last_undegraded":"2024-04-25T00:37:23.963576+0000","last_fullsized":"2024-04-25T00:37:23.963576+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.950282+0000","last_change":"2024-04-25T00:37:23.950282+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.950282+0000","last_undegraded":"2024-04-25T00:37:23.950282+0000","last_fullsized":"2024-04-25T00:37:23.950282+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.966291+0000","last_change":"2024-04-25T00:37:23.966291+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.966291+0000","last_undegraded":"2024-04-25T00:37:23.966291+0000","last_fullsized":"2024-04-25T00:37:23.966291+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.963777+0000","last_change":"2024-04-25T00:37:23.963777+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.963777+0000","last_undegraded":"2024-04-25T00:37:23.963777+0000","last_fullsized":"2024-04-25T00:37:23.963777+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951734+0000","last_change":"2024-04-25T00:37:23.951734+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951734+0000","last_undegraded":"2024-04-25T00:37:23.951734+0000","last_fullsized":"2024-04-25T00:37:23.951734+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.954303+0000","last_change":"2024-04-25T00:37:23.954303+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.954303+0000","last_undegraded":"2024-04-25T00:37:23.954303+0000","last_fullsized":"2024-04-25T00:37:23.954303+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957857+0000","last_change":"2024-04-25T00:37:23.957857+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957857+0000","last_undegraded":"2024-04-25T00:37:23.957857+0000","last_fullsized":"2024-04-25T00:37:23.957857+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.964892+0000","last_change":"2024-04-25T00:37:23.964892+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.964892+0000","last_undegraded":"2024-04-25T00:37:23.964892+0000","last_fullsized":"2024-04-25T00:37:23.964892+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.963074+0000","last_change":"2024-04-25T00:37:23.963074+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.963074+0000","last_undegraded":"2024-04-25T00:37:23.963074+0000","last_fullsized":"2024-04-25T00:37:23.963074+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.962966+0000","last_change":"2024-04-25T00:37:23.962966+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.962966+0000","last_undegraded":"2024-04-25T00:37:23.962966+0000","last_fullsized":"2024-04-25T00:37:23.962966+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.949178+0000","last_change":"2024-04-25T00:37:23.949178+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.949178+0000","last_undegraded":"2024-04-25T00:37:23.949178+0000","last_fullsized":"2024-04-25T00:37:23.949178+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.963247+0000","last_change":"2024-04-25T00:37:23.963247+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.963247+0000","last_undegraded":"2024-04-25T00:37:23.963247+0000","last_fullsized":"2024-04-25T00:37:23.963247+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.964817+0000","last_change":"2024-04-25T00:37:23.964817+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.964817+0000","last_undegraded":"2024-04-25T00:37:23.964817+0000","last_fullsized":"2024-04-25T00:37:23.964817+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.965283+0000","last_change":"2024-04-25T00:37:23.965283+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.965283+0000","last_undegraded":"2024-04-25T00:37:23.965283+0000","last_fullsized":"2024-04-25T00:37:23.965283+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957419+0000","last_change":"2024-04-25T00:37:23.957419+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957419+0000","last_undegraded":"2024-04-25T00:37:23.957419+0000","last_fullsized":"2024-04-25T00:37:23.957419+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.955558+0000","last_change":"2024-04-25T00:37:23.955558+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.955558+0000","last_undegraded":"2024-04-25T00:37:23.955558+0000","last_fullsized":"2024-04-25T00:37:23.955558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.950919+0000","last_change":"2024-04-25T00:37:23.950919+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.950919+0000","last_undegraded":"2024-04-25T00:37:23.950919+0000","last_fullsized":"2024-04-25T00:37:23.950919+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.960887+0000","last_change":"2024-04-25T00:37:23.960887+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.960887+0000","last_undegraded":"2024-04-25T00:37:23.960887+0000","last_fullsized":"2024-04-25T00:37:23.960887+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.953618+0000","last_change":"2024-04-25T00:37:23.953618+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.953618+0000","last_undegraded":"2024-04-25T00:37:23.953618+0000","last_fullsized":"2024-04-25T00:37:23.953618+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.949114+0000","last_change":"2024-04-25T00:37:23.949114+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.949114+0000","last_undegraded":"2024-04-25T00:37:23.949114+0000","last_fullsized":"2024-04-25T00:37:23.949114+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.949966+0000","last_change":"2024-04-25T00:37:23.949966+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.949966+0000","last_undegraded":"2024-04-25T00:37:23.949966+0000","last_fullsized":"2024-04-25T00:37:23.949966+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952217+0000","last_change":"2024-04-25T00:37:23.952217+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952217+0000","last_undegraded":"2024-04-25T00:37:23.952217+0000","last_fullsized":"2024-04-25T00:37:23.952217+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.953829+0000","last_change":"2024-04-25T00:37:23.953829+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.953829+0000","last_undegraded":"2024-04-25T00:37:23.953829+0000","last_fullsized":"2024-04-25T00:37:23.953829+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952069+0000","last_change":"2024-04-25T00:37:23.952069+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952069+0000","last_undegraded":"2024-04-25T00:37:23.952069+0000","last_fullsized":"2024-04-25T00:37:23.952069+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.949141+0000","last_change":"2024-04-25T00:37:23.949141+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.949141+0000","last_undegraded":"2024-04-25T00:37:23.949141+0000","last_fullsized":"2024-04-25T00:37:23.949141+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957875+0000","last_change":"2024-04-25T00:37:23.957875+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957875+0000","last_undegraded":"2024-04-25T00:37:23.957875+0000","last_fullsized":"2024-04-25T00:37:23.957875+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.956611+0000","last_change":"2024-04-25T00:37:23.956611+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.956611+0000","last_undegraded":"2024-04-25T00:37:23.956611+0000","last_fullsized":"2024-04-25T00:37:23.956611+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.955228+0000","last_change":"2024-04-25T00:37:23.955228+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.955228+0000","last_undegraded":"2024-04-25T00:37:23.955228+0000","last_fullsized":"2024-04-25T00:37:23.955228+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.956067+0000","last_change":"2024-04-25T00:37:23.956067+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.956067+0000","last_undegraded":"2024-04-25T00:37:23.956067+0000","last_fullsized":"2024-04-25T00:37:23.956067+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.953026+0000","last_change":"2024-04-25T00:37:23.953026+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.953026+0000","last_undegraded":"2024-04-25T00:37:23.953026+0000","last_fullsized":"2024-04-25T00:37:23.953026+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957263+0000","last_change":"2024-04-25T00:37:23.957263+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957263+0000","last_undegraded":"2024-04-25T00:37:23.957263+0000","last_fullsized":"2024-04-25T00:37:23.957263+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952836+0000","last_change":"2024-04-25T00:37:23.952836+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952836+0000","last_undegraded":"2024-04-25T00:37:23.952836+0000","last_fullsized":"2024-04-25T00:37:23.952836+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.954441+0000","last_change":"2024-04-25T00:37:23.954441+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.954441+0000","last_undegraded":"2024-04-25T00:37:23.954441+0000","last_fullsized":"2024-04-25T00:37:23.954441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.984445+0000","last_change":"2024-04-25T00:37:23.981233+0000","last_active":"2024-04-25T00:37:23.984445+0000","last_peered":"2024-04-25T00:37:23.984445+0000","last_clean":"2024-04-25T00:37:23.984445+0000","last_became_active":"2024-04-25T00:37:23.980150+0000","last_became_peered":"2024-04-25T00:37:23.980150+0000","last_unstale":"2024-04-25T00:37:23.984445+0000","last_undegraded":"2024-04-25T00:37:23.984445+0000","last_fullsized":"2024-04-25T00:37:23.984445+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:38:19.934694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.956938+0000","last_change":"2024-04-25T00:37:23.956938+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.956938+0000","last_undegraded":"2024-04-25T00:37:23.956938+0000","last_fullsized":"2024-04-25T00:37:23.956938+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.979620+0000","last_change":"2024-04-25T00:37:23.974780+0000","last_active":"2024-04-25T00:37:23.979620+0000","last_peered":"2024-04-25T00:37:23.979620+0000","last_clean":"2024-04-25T00:37:23.979620+0000","last_became_active":"2024-04-25T00:37:23.973844+0000","last_became_peered":"2024-04-25T00:37:23.973844+0000","last_unstale":"2024-04-25T00:37:23.979620+0000","last_undegraded":"2024-04-25T00:37:23.979620+0000","last_fullsized":"2024-04-25T00:37:23.979620+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:23.689041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.958270+0000","last_change":"2024-04-25T00:37:23.958270+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.958270+0000","last_undegraded":"2024-04-25T00:37:23.958270+0000","last_fullsized":"2024-04-25T00:37:23.958270+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.964726+0000","last_change":"2024-04-25T00:37:23.962945+0000","last_active":"2024-04-25T00:37:23.964726+0000","last_peered":"2024-04-25T00:37:23.964726+0000","last_clean":"2024-04-25T00:37:23.964726+0000","last_became_active":"2024-04-25T00:37:23.962460+0000","last_became_peered":"2024-04-25T00:37:23.962460+0000","last_unstale":"2024-04-25T00:37:23.964726+0000","last_undegraded":"2024-04-25T00:37:23.964726+0000","last_fullsized":"2024-04-25T00:37:23.964726+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:09.261299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952456+0000","last_change":"2024-04-25T00:37:23.952456+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952456+0000","last_undegraded":"2024-04-25T00:37:23.952456+0000","last_fullsized":"2024-04-25T00:37:23.952456+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.968954+0000","last_change":"2024-04-25T00:37:23.967844+0000","last_active":"2024-04-25T00:37:23.968954+0000","last_peered":"2024-04-25T00:37:23.968954+0000","last_clean":"2024-04-25T00:37:23.968954+0000","last_became_active":"2024-04-25T00:37:23.967545+0000","last_became_peered":"2024-04-25T00:37:23.967545+0000","last_unstale":"2024-04-25T00:37:23.968954+0000","last_undegraded":"2024-04-25T00:37:23.968954+0000","last_fullsized":"2024-04-25T00:37:23.968954+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:55.377792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952448+0000","last_change":"2024-04-25T00:37:23.952448+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952448+0000","last_undegraded":"2024-04-25T00:37:23.952448+0000","last_fullsized":"2024-04-25T00:37:23.952448+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.978848+0000","last_change":"2024-04-25T00:37:23.971620+0000","last_active":"2024-04-25T00:37:23.978848+0000","last_peered":"2024-04-25T00:37:23.978848+0000","last_clean":"2024-04-25T00:37:23.978848+0000","last_became_active":"2024-04-25T00:37:23.970901+0000","last_became_peered":"2024-04-25T00:37:23.970901+0000","last_unstale":"2024-04-25T00:37:23.978848+0000","last_undegraded":"2024-04-25T00:37:23.978848+0000","last_fullsized":"2024-04-25T00:37:23.978848+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:13.119808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.955049+0000","last_change":"2024-04-25T00:37:23.955049+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.955049+0000","last_undegraded":"2024-04-25T00:37:23.955049+0000","last_fullsized":"2024-04-25T00:37:23.955049+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.976090+0000","last_change":"2024-04-25T00:37:23.974470+0000","last_active":"2024-04-25T00:37:23.976090+0000","last_peered":"2024-04-25T00:37:23.976090+0000","last_clean":"2024-04-25T00:37:23.976090+0000","last_became_active":"2024-04-25T00:37:23.974141+0000","last_became_peered":"2024-04-25T00:37:23.974141+0000","last_unstale":"2024-04-25T00:37:23.976090+0000","last_undegraded":"2024-04-25T00:37:23.976090+0000","last_fullsized":"2024-04-25T00:37:23.976090+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:23:20.555785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.955063+0000","last_change":"2024-04-25T00:37:23.955063+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.955063+0000","last_undegraded":"2024-04-25T00:37:23.955063+0000","last_fullsized":"2024-04-25T00:37:23.955063+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.962331+0000","last_change":"2024-04-25T00:37:23.960317+0000","last_active":"2024-04-25T00:37:23.962331+0000","last_peered":"2024-04-25T00:37:23.962331+0000","last_clean":"2024-04-25T00:37:23.962331+0000","last_became_active":"2024-04-25T00:37:23.959614+0000","last_became_peered":"2024-04-25T00:37:23.959614+0000","last_unstale":"2024-04-25T00:37:23.962331+0000","last_undegraded":"2024-04-25T00:37:23.962331+0000","last_fullsized":"2024-04-25T00:37:23.962331+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:09.024737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.958913+0000","last_change":"2024-04-25T00:37:23.958913+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.958913+0000","last_undegraded":"2024-04-25T00:37:23.958913+0000","last_fullsized":"2024-04-25T00:37:23.958913+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.974447+0000","last_change":"2024-04-25T00:37:23.971464+0000","last_active":"2024-04-25T00:37:23.974447+0000","last_peered":"2024-04-25T00:37:23.974447+0000","last_clean":"2024-04-25T00:37:23.974447+0000","last_became_active":"2024-04-25T00:37:23.970908+0000","last_became_peered":"2024-04-25T00:37:23.970908+0000","last_unstale":"2024-04-25T00:37:23.974447+0000","last_undegraded":"2024-04-25T00:37:23.974447+0000","last_fullsized":"2024-04-25T00:37:23.974447+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:56.021774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.956732+0000","last_change":"2024-04-25T00:37:23.956732+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.956732+0000","last_undegraded":"2024-04-25T00:37:23.956732+0000","last_fullsized":"2024-04-25T00:37:23.956732+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.964410+0000","last_change":"2024-04-25T00:37:23.962738+0000","last_active":"2024-04-25T00:37:23.964410+0000","last_peered":"2024-04-25T00:37:23.964410+0000","last_clean":"2024-04-25T00:37:23.964410+0000","last_became_active":"2024-04-25T00:37:23.962330+0000","last_became_peered":"2024-04-25T00:37:23.962330+0000","last_unstale":"2024-04-25T00:37:23.964410+0000","last_undegraded":"2024-04-25T00:37:23.964410+0000","last_fullsized":"2024-04-25T00:37:23.964410+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:44.729272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.955830+0000","last_change":"2024-04-25T00:37:23.955830+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.955830+0000","last_undegraded":"2024-04-25T00:37:23.955830+0000","last_fullsized":"2024-04-25T00:37:23.955830+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.976070+0000","last_change":"2024-04-25T00:37:23.973733+0000","last_active":"2024-04-25T00:37:23.976070+0000","last_peered":"2024-04-25T00:37:23.976070+0000","last_clean":"2024-04-25T00:37:23.976070+0000","last_became_active":"2024-04-25T00:37:23.973362+0000","last_became_peered":"2024-04-25T00:37:23.973362+0000","last_unstale":"2024-04-25T00:37:23.976070+0000","last_undegraded":"2024-04-25T00:37:23.976070+0000","last_fullsized":"2024-04-25T00:37:23.976070+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:50.821959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.954708+0000","last_change":"2024-04-25T00:37:23.954708+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.954708+0000","last_undegraded":"2024-04-25T00:37:23.954708+0000","last_fullsized":"2024-04-25T00:37:23.954708+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.972416+0000","last_change":"2024-04-25T00:37:23.966485+0000","last_active":"2024-04-25T00:37:23.972416+0000","last_peered":"2024-04-25T00:37:23.972416+0000","last_clean":"2024-04-25T00:37:23.972416+0000","last_became_active":"2024-04-25T00:37:23.966125+0000","last_became_peered":"2024-04-25T00:37:23.966125+0000","last_unstale":"2024-04-25T00:37:23.972416+0000","last_undegraded":"2024-04-25T00:37:23.972416+0000","last_fullsized":"2024-04-25T00:37:23.972416+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:51.405634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.959932+0000","last_change":"2024-04-25T00:37:23.959932+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.959932+0000","last_undegraded":"2024-04-25T00:37:23.959932+0000","last_fullsized":"2024-04-25T00:37:23.959932+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.974336+0000","last_change":"2024-04-25T00:37:23.971682+0000","last_active":"2024-04-25T00:37:23.974336+0000","last_peered":"2024-04-25T00:37:23.974336+0000","last_clean":"2024-04-25T00:37:23.974336+0000","last_became_active":"2024-04-25T00:37:23.970797+0000","last_became_peered":"2024-04-25T00:37:23.970797+0000","last_unstale":"2024-04-25T00:37:23.974336+0000","last_undegraded":"2024-04-25T00:37:23.974336+0000","last_fullsized":"2024-04-25T00:37:23.974336+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:02.162592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951746+0000","last_change":"2024-04-25T00:37:23.951746+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951746+0000","last_undegraded":"2024-04-25T00:37:23.951746+0000","last_fullsized":"2024-04-25T00:37:23.951746+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.980578+0000","last_change":"2024-04-25T00:37:23.980023+0000","last_active":"2024-04-25T00:37:23.980578+0000","last_peered":"2024-04-25T00:37:23.980578+0000","last_clean":"2024-04-25T00:37:23.980578+0000","last_became_active":"2024-04-25T00:37:23.979702+0000","last_became_peered":"2024-04-25T00:37:23.979702+0000","last_unstale":"2024-04-25T00:37:23.980578+0000","last_undegraded":"2024-04-25T00:37:23.980578+0000","last_fullsized":"2024-04-25T00:37:23.980578+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:03.459207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951412+0000","last_change":"2024-04-25T00:37:23.951412+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951412+0000","last_undegraded":"2024-04-25T00:37:23.951412+0000","last_fullsized":"2024-04-25T00:37:23.951412+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.981951+0000","last_change":"2024-04-25T00:37:23.974737+0000","last_active":"2024-04-25T00:37:23.981951+0000","last_peered":"2024-04-25T00:37:23.981951+0000","last_clean":"2024-04-25T00:37:23.981951+0000","last_became_active":"2024-04-25T00:37:23.974006+0000","last_became_peered":"2024-04-25T00:37:23.974006+0000","last_unstale":"2024-04-25T00:37:23.981951+0000","last_undegraded":"2024-04-25T00:37:23.981951+0000","last_fullsized":"2024-04-25T00:37:23.981951+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:43.160583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.955228+0000","last_change":"2024-04-25T00:37:23.955228+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.955228+0000","last_undegraded":"2024-04-25T00:37:23.955228+0000","last_fullsized":"2024-04-25T00:37:23.955228+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.973622+0000","last_change":"2024-04-25T00:37:23.972616+0000","last_active":"2024-04-25T00:37:23.973622+0000","last_peered":"2024-04-25T00:37:23.973622+0000","last_clean":"2024-04-25T00:37:23.973622+0000","last_became_active":"2024-04-25T00:37:23.972302+0000","last_became_peered":"2024-04-25T00:37:23.972302+0000","last_unstale":"2024-04-25T00:37:23.973622+0000","last_undegraded":"2024-04-25T00:37:23.973622+0000","last_fullsized":"2024-04-25T00:37:23.973622+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:53.826036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952853+0000","last_change":"2024-04-25T00:37:23.952853+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952853+0000","last_undegraded":"2024-04-25T00:37:23.952853+0000","last_fullsized":"2024-04-25T00:37:23.952853+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.975580+0000","last_change":"2024-04-25T00:37:23.973856+0000","last_active":"2024-04-25T00:37:23.975580+0000","last_peered":"2024-04-25T00:37:23.975580+0000","last_clean":"2024-04-25T00:37:23.975580+0000","last_became_active":"2024-04-25T00:37:23.973576+0000","last_became_peered":"2024-04-25T00:37:23.973576+0000","last_unstale":"2024-04-25T00:37:23.975580+0000","last_undegraded":"2024-04-25T00:37:23.975580+0000","last_fullsized":"2024-04-25T00:37:23.975580+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:08.824434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.953910+0000","last_change":"2024-04-25T00:37:23.953910+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.953910+0000","last_undegraded":"2024-04-25T00:37:23.953910+0000","last_fullsized":"2024-04-25T00:37:23.953910+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.964463+0000","last_change":"2024-04-25T00:37:23.963264+0000","last_active":"2024-04-25T00:37:23.964463+0000","last_peered":"2024-04-25T00:37:23.964463+0000","last_clean":"2024-04-25T00:37:23.964463+0000","last_became_active":"2024-04-25T00:37:23.962746+0000","last_became_peered":"2024-04-25T00:37:23.962746+0000","last_unstale":"2024-04-25T00:37:23.964463+0000","last_undegraded":"2024-04-25T00:37:23.964463+0000","last_fullsized":"2024-04-25T00:37:23.964463+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:06.004795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951573+0000","last_change":"2024-04-25T00:37:23.951573+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951573+0000","last_undegraded":"2024-04-25T00:37:23.951573+0000","last_fullsized":"2024-04-25T00:37:23.951573+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.973831+0000","last_change":"2024-04-25T00:37:23.969113+0000","last_active":"2024-04-25T00:37:23.973831+0000","last_peered":"2024-04-25T00:37:23.973831+0000","last_clean":"2024-04-25T00:37:23.973831+0000","last_became_active":"2024-04-25T00:37:23.968373+0000","last_became_peered":"2024-04-25T00:37:23.968373+0000","last_unstale":"2024-04-25T00:37:23.973831+0000","last_undegraded":"2024-04-25T00:37:23.973831+0000","last_fullsized":"2024-04-25T00:37:23.973831+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:44.008910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951193+0000","last_change":"2024-04-25T00:37:23.951193+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951193+0000","last_undegraded":"2024-04-25T00:37:23.951193+0000","last_fullsized":"2024-04-25T00:37:23.951193+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.983736+0000","last_change":"2024-04-25T00:37:23.975090+0000","last_active":"2024-04-25T00:37:23.983736+0000","last_peered":"2024-04-25T00:37:23.983736+0000","last_clean":"2024-04-25T00:37:23.983736+0000","last_became_active":"2024-04-25T00:37:23.973684+0000","last_became_peered":"2024-04-25T00:37:23.973684+0000","last_unstale":"2024-04-25T00:37:23.983736+0000","last_undegraded":"2024-04-25T00:37:23.983736+0000","last_fullsized":"2024-04-25T00:37:23.983736+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:34.935440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.953635+0000","last_change":"2024-04-25T00:37:23.953635+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.953635+0000","last_undegraded":"2024-04-25T00:37:23.953635+0000","last_fullsized":"2024-04-25T00:37:23.953635+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.967381+0000","last_change":"2024-04-25T00:37:23.962101+0000","last_active":"2024-04-25T00:37:23.967381+0000","last_peered":"2024-04-25T00:37:23.967381+0000","last_clean":"2024-04-25T00:37:23.967381+0000","last_became_active":"2024-04-25T00:37:23.961767+0000","last_became_peered":"2024-04-25T00:37:23.961767+0000","last_unstale":"2024-04-25T00:37:23.967381+0000","last_undegraded":"2024-04-25T00:37:23.967381+0000","last_fullsized":"2024-04-25T00:37:23.967381+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:45.756449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957228+0000","last_change":"2024-04-25T00:37:23.957228+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957228+0000","last_undegraded":"2024-04-25T00:37:23.957228+0000","last_fullsized":"2024-04-25T00:37:23.957228+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.964300+0000","last_change":"2024-04-25T00:37:23.962517+0000","last_active":"2024-04-25T00:37:23.964300+0000","last_peered":"2024-04-25T00:37:23.964300+0000","last_clean":"2024-04-25T00:37:23.964300+0000","last_became_active":"2024-04-25T00:37:23.962068+0000","last_became_peered":"2024-04-25T00:37:23.962068+0000","last_unstale":"2024-04-25T00:37:23.964300+0000","last_undegraded":"2024-04-25T00:37:23.964300+0000","last_fullsized":"2024-04-25T00:37:23.964300+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:38.445130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.952486+0000","last_change":"2024-04-25T00:37:23.952486+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.952486+0000","last_undegraded":"2024-04-25T00:37:23.952486+0000","last_fullsized":"2024-04-25T00:37:23.952486+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.963895+0000","last_change":"2024-04-25T00:37:23.962073+0000","last_active":"2024-04-25T00:37:23.963895+0000","last_peered":"2024-04-25T00:37:23.963895+0000","last_clean":"2024-04-25T00:37:23.963895+0000","last_became_active":"2024-04-25T00:37:23.961617+0000","last_became_peered":"2024-04-25T00:37:23.961617+0000","last_unstale":"2024-04-25T00:37:23.963895+0000","last_undegraded":"2024-04-25T00:37:23.963895+0000","last_fullsized":"2024-04-25T00:37:23.963895+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:26.276154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.950948+0000","last_change":"2024-04-25T00:37:23.950948+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.950948+0000","last_undegraded":"2024-04-25T00:37:23.950948+0000","last_fullsized":"2024-04-25T00:37:23.950948+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.975764+0000","last_change":"2024-04-25T00:37:23.974633+0000","last_active":"2024-04-25T00:37:23.975764+0000","last_peered":"2024-04-25T00:37:23.975764+0000","last_clean":"2024-04-25T00:37:23.975764+0000","last_became_active":"2024-04-25T00:37:23.974067+0000","last_became_peered":"2024-04-25T00:37:23.974067+0000","last_unstale":"2024-04-25T00:37:23.975764+0000","last_undegraded":"2024-04-25T00:37:23.975764+0000","last_fullsized":"2024-04-25T00:37:23.975764+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:31.191986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.959904+0000","last_change":"2024-04-25T00:37:23.959904+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.959904+0000","last_undegraded":"2024-04-25T00:37:23.959904+0000","last_fullsized":"2024-04-25T00:37:23.959904+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.976046+0000","last_change":"2024-04-25T00:37:23.973640+0000","last_active":"2024-04-25T00:37:23.976046+0000","last_peered":"2024-04-25T00:37:23.976046+0000","last_clean":"2024-04-25T00:37:23.976046+0000","last_became_active":"2024-04-25T00:37:23.973210+0000","last_became_peered":"2024-04-25T00:37:23.973210+0000","last_unstale":"2024-04-25T00:37:23.976046+0000","last_undegraded":"2024-04-25T00:37:23.976046+0000","last_fullsized":"2024-04-25T00:37:23.976046+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:23.689041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957953+0000","last_change":"2024-04-25T00:37:23.957953+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957953+0000","last_undegraded":"2024-04-25T00:37:23.957953+0000","last_fullsized":"2024-04-25T00:37:23.957953+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.982782+0000","last_change":"2024-04-25T00:37:23.976356+0000","last_active":"2024-04-25T00:37:23.982782+0000","last_peered":"2024-04-25T00:37:23.982782+0000","last_clean":"2024-04-25T00:37:23.982782+0000","last_became_active":"2024-04-25T00:37:23.975148+0000","last_became_peered":"2024-04-25T00:37:23.975148+0000","last_unstale":"2024-04-25T00:37:23.982782+0000","last_undegraded":"2024-04-25T00:37:23.982782+0000","last_fullsized":"2024-04-25T00:37:23.982782+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:08.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.957683+0000","last_change":"2024-04-25T00:37:23.957683+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.957683+0000","last_undegraded":"2024-04-25T00:37:23.957683+0000","last_fullsized":"2024-04-25T00:37:23.957683+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.975969+0000","last_change":"2024-04-25T00:37:23.975126+0000","last_active":"2024-04-25T00:37:23.975969+0000","last_peered":"2024-04-25T00:37:23.975969+0000","last_clean":"2024-04-25T00:37:23.975969+0000","last_became_active":"2024-04-25T00:37:23.974571+0000","last_became_peered":"2024-04-25T00:37:23.974571+0000","last_unstale":"2024-04-25T00:37:23.975969+0000","last_undegraded":"2024-04-25T00:37:23.975969+0000","last_fullsized":"2024-04-25T00:37:23.975969+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:37.137351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.956677+0000","last_change":"2024-04-25T00:37:23.956677+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.956677+0000","last_undegraded":"2024-04-25T00:37:23.956677+0000","last_fullsized":"2024-04-25T00:37:23.956677+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.964294+0000","last_change":"2024-04-25T00:37:23.962739+0000","last_active":"2024-04-25T00:37:23.964294+0000","last_peered":"2024-04-25T00:37:23.964294+0000","last_clean":"2024-04-25T00:37:23.964294+0000","last_became_active":"2024-04-25T00:37:23.962425+0000","last_became_peered":"2024-04-25T00:37:23.962425+0000","last_unstale":"2024-04-25T00:37:23.964294+0000","last_undegraded":"2024-04-25T00:37:23.964294+0000","last_fullsized":"2024-04-25T00:37:23.964294+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:06.834610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.949912+0000","last_change":"2024-04-25T00:37:23.949912+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.949912+0000","last_undegraded":"2024-04-25T00:37:23.949912+0000","last_fullsized":"2024-04-25T00:37:23.949912+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.983791+0000","last_change":"2024-04-25T00:37:23.976960+0000","last_active":"2024-04-25T00:37:23.983791+0000","last_peered":"2024-04-25T00:37:23.983791+0000","last_clean":"2024-04-25T00:37:23.983791+0000","last_became_active":"2024-04-25T00:37:23.971621+0000","last_became_peered":"2024-04-25T00:37:23.971621+0000","last_unstale":"2024-04-25T00:37:23.983791+0000","last_undegraded":"2024-04-25T00:37:23.983791+0000","last_fullsized":"2024-04-25T00:37:23.983791+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:51.829250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.953895+0000","last_change":"2024-04-25T00:37:23.953895+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.953895+0000","last_undegraded":"2024-04-25T00:37:23.953895+0000","last_fullsized":"2024-04-25T00:37:23.953895+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.971517+0000","last_change":"2024-04-25T00:37:23.963951+0000","last_active":"2024-04-25T00:37:23.971517+0000","last_peered":"2024-04-25T00:37:23.971517+0000","last_clean":"2024-04-25T00:37:23.971517+0000","last_became_active":"2024-04-25T00:37:23.963276+0000","last_became_peered":"2024-04-25T00:37:23.963276+0000","last_unstale":"2024-04-25T00:37:23.971517+0000","last_undegraded":"2024-04-25T00:37:23.971517+0000","last_fullsized":"2024-04-25T00:37:23.971517+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:40.780233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951430+0000","last_change":"2024-04-25T00:37:23.951430+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951430+0000","last_undegraded":"2024-04-25T00:37:23.951430+0000","last_fullsized":"2024-04-25T00:37:23.951430+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.968956+0000","last_change":"2024-04-25T00:37:23.968009+0000","last_active":"2024-04-25T00:37:23.968956+0000","last_peered":"2024-04-25T00:37:23.968956+0000","last_clean":"2024-04-25T00:37:23.968956+0000","last_became_active":"2024-04-25T00:37:23.967586+0000","last_became_peered":"2024-04-25T00:37:23.967586+0000","last_unstale":"2024-04-25T00:37:23.968956+0000","last_undegraded":"2024-04-25T00:37:23.968956+0000","last_fullsized":"2024-04-25T00:37:23.968956+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:00.626603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.956755+0000","last_change":"2024-04-25T00:37:23.956755+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.956755+0000","last_undegraded":"2024-04-25T00:37:23.956755+0000","last_fullsized":"2024-04-25T00:37:23.956755+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.975697+0000","last_change":"2024-04-25T00:37:23.974640+0000","last_active":"2024-04-25T00:37:23.975697+0000","last_peered":"2024-04-25T00:37:23.975697+0000","last_clean":"2024-04-25T00:37:23.975697+0000","last_became_active":"2024-04-25T00:37:23.974224+0000","last_became_peered":"2024-04-25T00:37:23.974224+0000","last_unstale":"2024-04-25T00:37:23.975697+0000","last_undegraded":"2024-04-25T00:37:23.975697+0000","last_fullsized":"2024-04-25T00:37:23.975697+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:41.853687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-25T00:37:23.951624+0000","last_change":"2024-04-25T00:37:23.951624+0000","last_active":"2024-04-25T00:37:23.934372+0000","last_peered":"2024-04-25T00:37:23.934372+0000","last_clean":"2024-04-25T00:37:23.934372+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:37:23.951624+0000","last_undegraded":"2024-04-25T00:37:23.951624+0000","last_fullsized":"2024-04-25T00:37:23.951624+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.983706+0000","last_change":"2024-04-25T00:37:23.974436+0000","last_active":"2024-04-25T00:37:23.983706+0000","last_peered":"2024-04-25T00:37:23.983706+0000","last_clean":"2024-04-25T00:37:23.983706+0000","last_became_active":"2024-04-25T00:37:23.973775+0000","last_became_peered":"2024-04-25T00:37:23.973775+0000","last_unstale":"2024-04-25T00:37:23.983706+0000","last_undegraded":"2024-04-25T00:37:23.983706+0000","last_fullsized":"2024-04-25T00:37:23.983706+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:45.785490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":363,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-25T00:37:23.947499+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:37:23.947499+0000","last_peered":"2024-04-25T00:37:23.947499+0000","last_clean":"2024-04-25T00:37:23.947499+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:37:23.947499+0000","last_undegraded":"2024-04-25T00:37:23.947499+0000","last_fullsized":"2024-04-25T00:37:23.947499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37152,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334688,"statfs":{"total":96636764160,"available":96598720512,"internally_reserved":0,"allocated":1851392,"data_stored":1139894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1245184,"data_stored":543251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":549526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36572,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335268,"statfs":{"total":96636764160,"available":96599314432,"internally_reserved":0,"allocated":1257472,"data_stored":549526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1245184,"data_stored":543251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1245184,"data_stored":543251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1245184,"data_stored":543251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37136,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334704,"statfs":{"total":96636764160,"available":96598736896,"internally_reserved":0,"allocated":1839104,"data_stored":1133619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:37:29.372 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:37:29.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66e2e49640 1 Processor -- start 2024-04-25T00:37:29.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66e2e49640 1 -- start start 2024-04-25T00:37:29.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66e2e49640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f66dc15f3c0 0x7f66dc0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66e2e49640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66dc0b3720 con 0x7f66dc15e8d0 2024-04-25T00:37:29.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66e2e49640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66dc0b39c0 con 0x7f66dc0b4780 2024-04-25T00:37:29.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66e2e49640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f66dc0b42f0 con 0x7f66dc15f3c0 2024-04-25T00:37:29.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66e0be5640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f66dc15e8d0 0x7f66dc15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58010/0 (socket says 172.21.15.29:58010) 2024-04-25T00:37:29.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66e0be5640 1 -- 172.21.15.29:0/466177905 learned_addr learned my addr 172.21.15.29:0/466177905 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:29.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66e13e6640 1 --2- 172.21.15.29:0/466177905 >> v2:172.21.15.29:3300/0 conn(0x7f66dc15f3c0 0x7f66dc0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66db7fe640 1 -- 172.21.15.29:0/466177905 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3114765881 0 0) 0x7f66dc0b3720 con 0x7f66dc15e8d0 2024-04-25T00:37:29.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66db7fe640 1 -- 172.21.15.29:0/466177905 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66cc003530 con 0x7f66dc15e8d0 2024-04-25T00:37:29.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66e13e6640 1 -- 172.21.15.29:0/466177905 >> v1:172.21.15.157:6789/0 conn(0x7f66dc0b4780 legacy=0x7f66dc15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66e13e6640 1 -- 172.21.15.29:0/466177905 >> v1:172.21.15.29:6789/0 conn(0x7f66dc15e8d0 legacy=0x7f66dc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.535+0000 7f66e13e6640 1 -- 172.21.15.29:0/466177905 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc0b3ca0 con 0x7f66dc15f3c0 2024-04-25T00:37:29.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e13e6640 1 --2- 172.21.15.29:0/466177905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66dc15f3c0 0x7f66dc0cc240 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009d90 tx=0x7f66d401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528bfa6afd9c7f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66db7fe640 1 -- 172.21.15.29:0/466177905 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d40b2070 con 0x7f66dc15f3c0 2024-04-25T00:37:29.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66db7fe640 1 -- 172.21.15.29:0/466177905 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d40a58c0 con 0x7f66dc15f3c0 2024-04-25T00:37:29.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66db7fe640 1 -- 172.21.15.29:0/466177905 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d40a5c90 con 0x7f66dc15f3c0 2024-04-25T00:37:29.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 -- 172.21.15.29:0/466177905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66dc15f3c0 msgr2=0x7f66dc0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 --2- 172.21.15.29:0/466177905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66dc15f3c0 0x7f66dc0cc240 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009d90 tx=0x7f66d401c930 comp rx=0 tx=0).stop 2024-04-25T00:37:29.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 -- 172.21.15.29:0/466177905 shutdown_connections 2024-04-25T00:37:29.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 --2- 172.21.15.29:0/466177905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66dc15f3c0 0x7f66dc0cc240 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 -- 172.21.15.29:0/466177905 >> 172.21.15.29:0/466177905 conn(0x7f66dc0b1870 msgr2=0x7f66dc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 -- 172.21.15.29:0/466177905 shutdown_connections 2024-04-25T00:37:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 -- 172.21.15.29:0/466177905 wait complete. 2024-04-25T00:37:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 Processor -- start 2024-04-25T00:37:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 -- start start 2024-04-25T00:37:29.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66dc0b4780 0x7f66dc1596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66dc15e8d0 0x7f66dc142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66dc15f3c0 0x7f66dc0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc168e00 con 0x7f66dc15f3c0 2024-04-25T00:37:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc168c80 con 0x7f66dc15e8d0 2024-04-25T00:37:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e2e49640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f66dc168f80 con 0x7f66dc0b4780 2024-04-25T00:37:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e13e6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66dc15f3c0 0x7f66dc0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e13e6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66dc15f3c0 0x7f66dc0b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43646/0 (socket says 172.21.15.29:43646) 2024-04-25T00:37:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e13e6640 1 -- 172.21.15.29:0/3428500488 learned_addr learned my addr 172.21.15.29:0/3428500488 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e0be5640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66dc0b4780 0x7f66dc1596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66dbfff640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66dc15e8d0 0x7f66dc142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e13e6640 1 -- 172.21.15.29:0/3428500488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66dc0b4780 msgr2=0x7f66dc1596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e13e6640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66dc0b4780 0x7f66dc1596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e13e6640 1 -- 172.21.15.29:0/3428500488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66dc15e8d0 msgr2=0x7f66dc142cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e13e6640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66dc15e8d0 0x7f66dc142cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e13e6640 1 -- 172.21.15.29:0/3428500488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc143590 con 0x7f66dc15f3c0 2024-04-25T00:37:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66e0be5640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66dc0b4780 0x7f66dc1596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.539+0000 7f66dbfff640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66dc15e8d0 0x7f66dc142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.543+0000 7f66e13e6640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66dc15f3c0 0x7f66dc0b0990 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f66d401a560 tx=0x7f66d40196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.543+0000 7f66d97fa640 1 -- 172.21.15.29:0/3428500488 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d40bd020 con 0x7f66dc15f3c0 2024-04-25T00:37:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.543+0000 7f66d97fa640 1 -- 172.21.15.29:0/3428500488 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d40b8470 con 0x7f66dc15f3c0 2024-04-25T00:37:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.543+0000 7f66d97fa640 1 -- 172.21.15.29:0/3428500488 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d40b2040 con 0x7f66dc15f3c0 2024-04-25T00:37:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.543+0000 7f66e2e49640 1 -- 172.21.15.29:0/3428500488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66dc143870 con 0x7f66dc15f3c0 2024-04-25T00:37:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.543+0000 7f66e2e49640 1 -- 172.21.15.29:0/3428500488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66dc13eba0 con 0x7f66dc15f3c0 2024-04-25T00:37:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.543+0000 7f66e2e49640 1 -- 172.21.15.29:0/3428500488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66dc069270 con 0x7f66dc15f3c0 2024-04-25T00:37:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.543+0000 7f66d97fa640 1 -- 172.21.15.29:0/3428500488 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f66d40c0020 con 0x7f66dc15f3c0 2024-04-25T00:37:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.543+0000 7f66d97fa640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66c0067a10 0x7f66c0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.543+0000 7f66d97fa640 1 -- 172.21.15.29:0/3428500488 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f66d4120a10 con 0x7f66dc15f3c0 2024-04-25T00:37:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.543+0000 7f66e0be5640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66c0067a10 0x7f66c0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.543+0000 7f66e0be5640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66c0067a10 0x7f66c0069ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f66d0005890 tx=0x7f66d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.547+0000 7f66d97fa640 1 -- 172.21.15.29:0/3428500488 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f66d40c7070 con 0x7f66dc15f3c0 2024-04-25T00:37:29.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.699+0000 7f66e2e49640 1 -- 172.21.15.29:0/3428500488 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66dc06e7b0 con 0x7f66c0067a10 2024-04-25T00:37:29.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.715+0000 7f66d97fa640 1 -- 172.21.15.29:0/3428500488 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7f66dc06e7b0 con 0x7f66c0067a10 2024-04-25T00:37:29.721 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:29.722 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:37:29.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.719+0000 7f66e2e49640 1 -- 172.21.15.29:0/3428500488 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66c0067a10 msgr2=0x7f66c0069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.719+0000 7f66e2e49640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66c0067a10 0x7f66c0069ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f66d0005890 tx=0x7f66d0009040 comp rx=0 tx=0).stop 2024-04-25T00:37:29.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.719+0000 7f66e2e49640 1 -- 172.21.15.29:0/3428500488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66dc15f3c0 msgr2=0x7f66dc0b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.719+0000 7f66e2e49640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66dc15f3c0 0x7f66dc0b0990 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f66d401a560 tx=0x7f66d40196d0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.719+0000 7f66e2e49640 1 -- 172.21.15.29:0/3428500488 shutdown_connections 2024-04-25T00:37:29.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.719+0000 7f66e2e49640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66c0067a10 0x7f66c0069ed0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.719+0000 7f66e2e49640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66dc15f3c0 0x7f66dc0b0990 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.719+0000 7f66e2e49640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66dc15e8d0 0x7f66dc142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.719+0000 7f66e2e49640 1 --2- 172.21.15.29:0/3428500488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66dc0b4780 0x7f66dc1596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.719+0000 7f66e2e49640 1 -- 172.21.15.29:0/3428500488 >> 172.21.15.29:0/3428500488 conn(0x7f66dc0b1870 msgr2=0x7f66dc0af880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:29.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.719+0000 7f66e2e49640 1 -- 172.21.15.29:0/3428500488 shutdown_connections 2024-04-25T00:37:29.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.719+0000 7f66e2e49640 1 -- 172.21.15.29:0/3428500488 wait complete. 2024-04-25T00:37:29.738 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-25T00:37:29.347985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289816,"kb_used_data":11240,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754684904,"statfs":{"total":773094113280,"available":772797341696,"internally_reserved":0,"allocated":11509760,"data_stored":5686364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.017469"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617055+0000","last_change":"2024-04-25T00:37:24.657606+0000","last_active":"2024-04-25T00:37:25.617055+0000","last_peered":"2024-04-25T00:37:25.617055+0000","last_clean":"2024-04-25T00:37:25.617055+0000","last_became_active":"2024-04-25T00:37:24.656859+0000","last_became_peered":"2024-04-25T00:37:24.656859+0000","last_unstale":"2024-04-25T00:37:25.617055+0000","last_undegraded":"2024-04-25T00:37:25.617055+0000","last_fullsized":"2024-04-25T00:37:25.617055+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:16.259398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619452+0000","last_change":"2024-04-25T00:37:24.648164+0000","last_active":"2024-04-25T00:37:25.619452+0000","last_peered":"2024-04-25T00:37:25.619452+0000","last_clean":"2024-04-25T00:37:25.619452+0000","last_became_active":"2024-04-25T00:37:24.647383+0000","last_became_peered":"2024-04-25T00:37:24.647383+0000","last_unstale":"2024-04-25T00:37:25.619452+0000","last_undegraded":"2024-04-25T00:37:25.619452+0000","last_fullsized":"2024-04-25T00:37:25.619452+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:57.102276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616600+0000","last_change":"2024-04-25T00:37:24.644232+0000","last_active":"2024-04-25T00:37:25.616600+0000","last_peered":"2024-04-25T00:37:25.616600+0000","last_clean":"2024-04-25T00:37:25.616600+0000","last_became_active":"2024-04-25T00:37:24.642886+0000","last_became_peered":"2024-04-25T00:37:24.642886+0000","last_unstale":"2024-04-25T00:37:25.616600+0000","last_undegraded":"2024-04-25T00:37:25.616600+0000","last_fullsized":"2024-04-25T00:37:25.616600+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:21.524079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617224+0000","last_change":"2024-04-25T00:37:24.657429+0000","last_active":"2024-04-25T00:37:25.617224+0000","last_peered":"2024-04-25T00:37:25.617224+0000","last_clean":"2024-04-25T00:37:25.617224+0000","last_became_active":"2024-04-25T00:37:24.656738+0000","last_became_peered":"2024-04-25T00:37:24.656738+0000","last_unstale":"2024-04-25T00:37:25.617224+0000","last_undegraded":"2024-04-25T00:37:25.617224+0000","last_fullsized":"2024-04-25T00:37:25.617224+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:37.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890462+0000","last_change":"2024-04-25T00:37:24.650575+0000","last_active":"2024-04-25T00:37:25.890462+0000","last_peered":"2024-04-25T00:37:25.890462+0000","last_clean":"2024-04-25T00:37:25.890462+0000","last_became_active":"2024-04-25T00:37:24.649239+0000","last_became_peered":"2024-04-25T00:37:24.649239+0000","last_unstale":"2024-04-25T00:37:25.890462+0000","last_undegraded":"2024-04-25T00:37:25.890462+0000","last_fullsized":"2024-04-25T00:37:25.890462+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:09.766305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892915+0000","last_change":"2024-04-25T00:37:24.655067+0000","last_active":"2024-04-25T00:37:25.892915+0000","last_peered":"2024-04-25T00:37:25.892915+0000","last_clean":"2024-04-25T00:37:25.892915+0000","last_became_active":"2024-04-25T00:37:24.654401+0000","last_became_peered":"2024-04-25T00:37:24.654401+0000","last_unstale":"2024-04-25T00:37:25.892915+0000","last_undegraded":"2024-04-25T00:37:25.892915+0000","last_fullsized":"2024-04-25T00:37:25.892915+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:36.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890044+0000","last_change":"2024-04-25T00:37:24.649776+0000","last_active":"2024-04-25T00:37:25.890044+0000","last_peered":"2024-04-25T00:37:25.890044+0000","last_clean":"2024-04-25T00:37:25.890044+0000","last_became_active":"2024-04-25T00:37:24.648979+0000","last_became_peered":"2024-04-25T00:37:24.648979+0000","last_unstale":"2024-04-25T00:37:25.890044+0000","last_undegraded":"2024-04-25T00:37:25.890044+0000","last_fullsized":"2024-04-25T00:37:25.890044+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:44.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616146+0000","last_change":"2024-04-25T00:37:24.655900+0000","last_active":"2024-04-25T00:37:25.616146+0000","last_peered":"2024-04-25T00:37:25.616146+0000","last_clean":"2024-04-25T00:37:25.616146+0000","last_became_active":"2024-04-25T00:37:24.655169+0000","last_became_peered":"2024-04-25T00:37:24.655169+0000","last_unstale":"2024-04-25T00:37:25.616146+0000","last_undegraded":"2024-04-25T00:37:25.616146+0000","last_fullsized":"2024-04-25T00:37:25.616146+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:57.102276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617147+0000","last_change":"2024-04-25T00:37:24.644695+0000","last_active":"2024-04-25T00:37:25.617147+0000","last_peered":"2024-04-25T00:37:25.617147+0000","last_clean":"2024-04-25T00:37:25.617147+0000","last_became_active":"2024-04-25T00:37:24.643886+0000","last_became_peered":"2024-04-25T00:37:24.643886+0000","last_unstale":"2024-04-25T00:37:25.617147+0000","last_undegraded":"2024-04-25T00:37:25.617147+0000","last_fullsized":"2024-04-25T00:37:25.617147+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:24.434950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616914+0000","last_change":"2024-04-25T00:37:24.668000+0000","last_active":"2024-04-25T00:37:25.616914+0000","last_peered":"2024-04-25T00:37:25.616914+0000","last_clean":"2024-04-25T00:37:25.616914+0000","last_became_active":"2024-04-25T00:37:24.663574+0000","last_became_peered":"2024-04-25T00:37:24.663574+0000","last_unstale":"2024-04-25T00:37:25.616914+0000","last_undegraded":"2024-04-25T00:37:25.616914+0000","last_fullsized":"2024-04-25T00:37:25.616914+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:37.489323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619004+0000","last_change":"2024-04-25T00:37:24.649566+0000","last_active":"2024-04-25T00:37:25.619004+0000","last_peered":"2024-04-25T00:37:25.619004+0000","last_clean":"2024-04-25T00:37:25.619004+0000","last_became_active":"2024-04-25T00:37:24.648905+0000","last_became_peered":"2024-04-25T00:37:24.648905+0000","last_unstale":"2024-04-25T00:37:25.619004+0000","last_undegraded":"2024-04-25T00:37:25.619004+0000","last_fullsized":"2024-04-25T00:37:25.619004+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:30.099048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617355+0000","last_change":"2024-04-25T00:37:24.659945+0000","last_active":"2024-04-25T00:37:25.617355+0000","last_peered":"2024-04-25T00:37:25.617355+0000","last_clean":"2024-04-25T00:37:25.617355+0000","last_became_active":"2024-04-25T00:37:24.659273+0000","last_became_peered":"2024-04-25T00:37:24.659273+0000","last_unstale":"2024-04-25T00:37:25.617355+0000","last_undegraded":"2024-04-25T00:37:25.617355+0000","last_fullsized":"2024-04-25T00:37:25.617355+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:12.673417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617382+0000","last_change":"2024-04-25T00:37:24.656446+0000","last_active":"2024-04-25T00:37:25.617382+0000","last_peered":"2024-04-25T00:37:25.617382+0000","last_clean":"2024-04-25T00:37:25.617382+0000","last_became_active":"2024-04-25T00:37:24.655626+0000","last_became_peered":"2024-04-25T00:37:24.655626+0000","last_unstale":"2024-04-25T00:37:25.617382+0000","last_undegraded":"2024-04-25T00:37:25.617382+0000","last_fullsized":"2024-04-25T00:37:25.617382+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:07.618067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616119+0000","last_change":"2024-04-25T00:37:24.625020+0000","last_active":"2024-04-25T00:37:25.616119+0000","last_peered":"2024-04-25T00:37:25.616119+0000","last_clean":"2024-04-25T00:37:25.616119+0000","last_became_active":"2024-04-25T00:37:24.624274+0000","last_became_peered":"2024-04-25T00:37:24.624274+0000","last_unstale":"2024-04-25T00:37:25.616119+0000","last_undegraded":"2024-04-25T00:37:25.616119+0000","last_fullsized":"2024-04-25T00:37:25.616119+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:52.953810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890889+0000","last_change":"2024-04-25T00:37:24.653503+0000","last_active":"2024-04-25T00:37:25.890889+0000","last_peered":"2024-04-25T00:37:25.890889+0000","last_clean":"2024-04-25T00:37:25.890889+0000","last_became_active":"2024-04-25T00:37:24.652726+0000","last_became_peered":"2024-04-25T00:37:24.652726+0000","last_unstale":"2024-04-25T00:37:25.890889+0000","last_undegraded":"2024-04-25T00:37:25.890889+0000","last_fullsized":"2024-04-25T00:37:25.890889+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:36.836347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616099+0000","last_change":"2024-04-25T00:37:24.665542+0000","last_active":"2024-04-25T00:37:25.616099+0000","last_peered":"2024-04-25T00:37:25.616099+0000","last_clean":"2024-04-25T00:37:25.616099+0000","last_became_active":"2024-04-25T00:37:24.664831+0000","last_became_peered":"2024-04-25T00:37:24.664831+0000","last_unstale":"2024-04-25T00:37:25.616099+0000","last_undegraded":"2024-04-25T00:37:25.616099+0000","last_fullsized":"2024-04-25T00:37:25.616099+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:28.766277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619433+0000","last_change":"2024-04-25T00:37:24.650948+0000","last_active":"2024-04-25T00:37:25.619433+0000","last_peered":"2024-04-25T00:37:25.619433+0000","last_clean":"2024-04-25T00:37:25.619433+0000","last_became_active":"2024-04-25T00:37:24.649607+0000","last_became_peered":"2024-04-25T00:37:24.649607+0000","last_unstale":"2024-04-25T00:37:25.619433+0000","last_undegraded":"2024-04-25T00:37:25.619433+0000","last_fullsized":"2024-04-25T00:37:25.619433+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:52.953810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891102+0000","last_change":"2024-04-25T00:37:24.641490+0000","last_active":"2024-04-25T00:37:25.891102+0000","last_peered":"2024-04-25T00:37:25.891102+0000","last_clean":"2024-04-25T00:37:25.891102+0000","last_became_active":"2024-04-25T00:37:24.640694+0000","last_became_peered":"2024-04-25T00:37:24.640694+0000","last_unstale":"2024-04-25T00:37:25.891102+0000","last_undegraded":"2024-04-25T00:37:25.891102+0000","last_fullsized":"2024-04-25T00:37:25.891102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:42.857330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617010+0000","last_change":"2024-04-25T00:37:24.656560+0000","last_active":"2024-04-25T00:37:25.617010+0000","last_peered":"2024-04-25T00:37:25.617010+0000","last_clean":"2024-04-25T00:37:25.617010+0000","last_became_active":"2024-04-25T00:37:24.655791+0000","last_became_peered":"2024-04-25T00:37:24.655791+0000","last_unstale":"2024-04-25T00:37:25.617010+0000","last_undegraded":"2024-04-25T00:37:25.617010+0000","last_fullsized":"2024-04-25T00:37:25.617010+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:21.476709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.618771+0000","last_change":"2024-04-25T00:37:24.659487+0000","last_active":"2024-04-25T00:37:25.618771+0000","last_peered":"2024-04-25T00:37:25.618771+0000","last_clean":"2024-04-25T00:37:25.618771+0000","last_became_active":"2024-04-25T00:37:24.658548+0000","last_became_peered":"2024-04-25T00:37:24.658548+0000","last_unstale":"2024-04-25T00:37:25.618771+0000","last_undegraded":"2024-04-25T00:37:25.618771+0000","last_fullsized":"2024-04-25T00:37:25.618771+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:38.140994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619151+0000","last_change":"2024-04-25T00:37:24.648480+0000","last_active":"2024-04-25T00:37:25.619151+0000","last_peered":"2024-04-25T00:37:25.619151+0000","last_clean":"2024-04-25T00:37:25.619151+0000","last_became_active":"2024-04-25T00:37:24.647516+0000","last_became_peered":"2024-04-25T00:37:24.647516+0000","last_unstale":"2024-04-25T00:37:25.619151+0000","last_undegraded":"2024-04-25T00:37:25.619151+0000","last_fullsized":"2024-04-25T00:37:25.619151+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:45.097585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892096+0000","last_change":"2024-04-25T00:37:24.663784+0000","last_active":"2024-04-25T00:37:25.892096+0000","last_peered":"2024-04-25T00:37:25.892096+0000","last_clean":"2024-04-25T00:37:25.892096+0000","last_became_active":"2024-04-25T00:37:24.662549+0000","last_became_peered":"2024-04-25T00:37:24.662549+0000","last_unstale":"2024-04-25T00:37:25.892096+0000","last_undegraded":"2024-04-25T00:37:25.892096+0000","last_fullsized":"2024-04-25T00:37:25.892096+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:09.915478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617924+0000","last_change":"2024-04-25T00:37:24.658880+0000","last_active":"2024-04-25T00:37:25.617924+0000","last_peered":"2024-04-25T00:37:25.617924+0000","last_clean":"2024-04-25T00:37:25.617924+0000","last_became_active":"2024-04-25T00:37:24.658120+0000","last_became_peered":"2024-04-25T00:37:24.658120+0000","last_unstale":"2024-04-25T00:37:25.617924+0000","last_undegraded":"2024-04-25T00:37:25.617924+0000","last_fullsized":"2024-04-25T00:37:25.617924+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:03.166234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892331+0000","last_change":"2024-04-25T00:37:24.667001+0000","last_active":"2024-04-25T00:37:25.892331+0000","last_peered":"2024-04-25T00:37:25.892331+0000","last_clean":"2024-04-25T00:37:25.892331+0000","last_became_active":"2024-04-25T00:37:24.664458+0000","last_became_peered":"2024-04-25T00:37:24.664458+0000","last_unstale":"2024-04-25T00:37:25.892331+0000","last_undegraded":"2024-04-25T00:37:25.892331+0000","last_fullsized":"2024-04-25T00:37:25.892331+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:21.476709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619962+0000","last_change":"2024-04-25T00:37:24.650335+0000","last_active":"2024-04-25T00:37:25.619962+0000","last_peered":"2024-04-25T00:37:25.619962+0000","last_clean":"2024-04-25T00:37:25.619962+0000","last_became_active":"2024-04-25T00:37:24.648644+0000","last_became_peered":"2024-04-25T00:37:24.648644+0000","last_unstale":"2024-04-25T00:37:25.619962+0000","last_undegraded":"2024-04-25T00:37:25.619962+0000","last_fullsized":"2024-04-25T00:37:25.619962+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:57.228594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617137+0000","last_change":"2024-04-25T00:37:24.638520+0000","last_active":"2024-04-25T00:37:25.617137+0000","last_peered":"2024-04-25T00:37:25.617137+0000","last_clean":"2024-04-25T00:37:25.617137+0000","last_became_active":"2024-04-25T00:37:24.637640+0000","last_became_peered":"2024-04-25T00:37:24.637640+0000","last_unstale":"2024-04-25T00:37:25.617137+0000","last_undegraded":"2024-04-25T00:37:25.617137+0000","last_fullsized":"2024-04-25T00:37:25.617137+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:43.653979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891294+0000","last_change":"2024-04-25T00:37:24.659187+0000","last_active":"2024-04-25T00:37:25.891294+0000","last_peered":"2024-04-25T00:37:25.891294+0000","last_clean":"2024-04-25T00:37:25.891294+0000","last_became_active":"2024-04-25T00:37:24.658411+0000","last_became_peered":"2024-04-25T00:37:24.658411+0000","last_unstale":"2024-04-25T00:37:25.891294+0000","last_undegraded":"2024-04-25T00:37:25.891294+0000","last_fullsized":"2024-04-25T00:37:25.891294+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:46.789133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892883+0000","last_change":"2024-04-25T00:37:24.645549+0000","last_active":"2024-04-25T00:37:25.892883+0000","last_peered":"2024-04-25T00:37:25.892883+0000","last_clean":"2024-04-25T00:37:25.892883+0000","last_became_active":"2024-04-25T00:37:24.644769+0000","last_became_peered":"2024-04-25T00:37:24.644769+0000","last_unstale":"2024-04-25T00:37:25.892883+0000","last_undegraded":"2024-04-25T00:37:25.892883+0000","last_fullsized":"2024-04-25T00:37:25.892883+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:07.618067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.618138+0000","last_change":"2024-04-25T00:37:24.664583+0000","last_active":"2024-04-25T00:37:25.618138+0000","last_peered":"2024-04-25T00:37:25.618138+0000","last_clean":"2024-04-25T00:37:25.618138+0000","last_became_active":"2024-04-25T00:37:24.663713+0000","last_became_peered":"2024-04-25T00:37:24.663713+0000","last_unstale":"2024-04-25T00:37:25.618138+0000","last_undegraded":"2024-04-25T00:37:25.618138+0000","last_fullsized":"2024-04-25T00:37:25.618138+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:42.857330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890809+0000","last_change":"2024-04-25T00:37:24.650632+0000","last_active":"2024-04-25T00:37:25.890809+0000","last_peered":"2024-04-25T00:37:25.890809+0000","last_clean":"2024-04-25T00:37:25.890809+0000","last_became_active":"2024-04-25T00:37:24.649835+0000","last_became_peered":"2024-04-25T00:37:24.649835+0000","last_unstale":"2024-04-25T00:37:25.890809+0000","last_undegraded":"2024-04-25T00:37:25.890809+0000","last_fullsized":"2024-04-25T00:37:25.890809+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:24.692684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616247+0000","last_change":"2024-04-25T00:37:24.638702+0000","last_active":"2024-04-25T00:37:25.616247+0000","last_peered":"2024-04-25T00:37:25.616247+0000","last_clean":"2024-04-25T00:37:25.616247+0000","last_became_active":"2024-04-25T00:37:24.637878+0000","last_became_peered":"2024-04-25T00:37:24.637878+0000","last_unstale":"2024-04-25T00:37:25.616247+0000","last_undegraded":"2024-04-25T00:37:25.616247+0000","last_fullsized":"2024-04-25T00:37:25.616247+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:18.566344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891202+0000","last_change":"2024-04-25T00:37:24.647978+0000","last_active":"2024-04-25T00:37:25.891202+0000","last_peered":"2024-04-25T00:37:25.891202+0000","last_clean":"2024-04-25T00:37:25.891202+0000","last_became_active":"2024-04-25T00:37:24.647202+0000","last_became_peered":"2024-04-25T00:37:24.647202+0000","last_unstale":"2024-04-25T00:37:25.891202+0000","last_undegraded":"2024-04-25T00:37:25.891202+0000","last_fullsized":"2024-04-25T00:37:25.891202+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:14.123451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891406+0000","last_change":"2024-04-25T00:37:24.651972+0000","last_active":"2024-04-25T00:37:25.891406+0000","last_peered":"2024-04-25T00:37:25.891406+0000","last_clean":"2024-04-25T00:37:25.891406+0000","last_became_active":"2024-04-25T00:37:24.650939+0000","last_became_peered":"2024-04-25T00:37:24.650939+0000","last_unstale":"2024-04-25T00:37:25.891406+0000","last_undegraded":"2024-04-25T00:37:25.891406+0000","last_fullsized":"2024-04-25T00:37:25.891406+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:35.939082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"102'4","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891440+0000","last_change":"2024-04-25T00:37:23.981233+0000","last_active":"2024-04-25T00:37:25.891440+0000","last_peered":"2024-04-25T00:37:25.891440+0000","last_clean":"2024-04-25T00:37:25.891440+0000","last_became_active":"2024-04-25T00:37:23.980150+0000","last_became_peered":"2024-04-25T00:37:23.980150+0000","last_unstale":"2024-04-25T00:37:25.891440+0000","last_undegraded":"2024-04-25T00:37:25.891440+0000","last_fullsized":"2024-04-25T00:37:25.891440+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:47.226214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891957+0000","last_change":"2024-04-25T00:37:24.634734+0000","last_active":"2024-04-25T00:37:25.891957+0000","last_peered":"2024-04-25T00:37:25.891957+0000","last_clean":"2024-04-25T00:37:25.891957+0000","last_became_active":"2024-04-25T00:37:24.628248+0000","last_became_peered":"2024-04-25T00:37:24.628248+0000","last_unstale":"2024-04-25T00:37:25.891957+0000","last_undegraded":"2024-04-25T00:37:25.891957+0000","last_fullsized":"2024-04-25T00:37:25.891957+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:45.097585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617426+0000","last_change":"2024-04-25T00:37:23.974780+0000","last_active":"2024-04-25T00:37:25.617426+0000","last_peered":"2024-04-25T00:37:25.617426+0000","last_clean":"2024-04-25T00:37:25.617426+0000","last_became_active":"2024-04-25T00:37:23.973844+0000","last_became_peered":"2024-04-25T00:37:23.973844+0000","last_unstale":"2024-04-25T00:37:25.617426+0000","last_undegraded":"2024-04-25T00:37:25.617426+0000","last_fullsized":"2024-04-25T00:37:25.617426+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:50.821959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891301+0000","last_change":"2024-04-25T00:37:24.654395+0000","last_active":"2024-04-25T00:37:25.891301+0000","last_peered":"2024-04-25T00:37:25.891301+0000","last_clean":"2024-04-25T00:37:25.891301+0000","last_became_active":"2024-04-25T00:37:24.652977+0000","last_became_peered":"2024-04-25T00:37:24.652977+0000","last_unstale":"2024-04-25T00:37:25.891301+0000","last_undegraded":"2024-04-25T00:37:25.891301+0000","last_fullsized":"2024-04-25T00:37:25.891301+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:38:20.938337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889564+0000","last_change":"2024-04-25T00:37:23.962945+0000","last_active":"2024-04-25T00:37:25.889564+0000","last_peered":"2024-04-25T00:37:25.889564+0000","last_clean":"2024-04-25T00:37:25.889564+0000","last_became_active":"2024-04-25T00:37:23.962460+0000","last_became_peered":"2024-04-25T00:37:23.962460+0000","last_unstale":"2024-04-25T00:37:25.889564+0000","last_undegraded":"2024-04-25T00:37:25.889564+0000","last_fullsized":"2024-04-25T00:37:25.889564+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:05.179578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617374+0000","last_change":"2024-04-25T00:37:24.656227+0000","last_active":"2024-04-25T00:37:25.617374+0000","last_peered":"2024-04-25T00:37:25.617374+0000","last_clean":"2024-04-25T00:37:25.617374+0000","last_became_active":"2024-04-25T00:37:24.655453+0000","last_became_peered":"2024-04-25T00:37:24.655453+0000","last_unstale":"2024-04-25T00:37:25.617374+0000","last_undegraded":"2024-04-25T00:37:25.617374+0000","last_fullsized":"2024-04-25T00:37:25.617374+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:38.243062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890255+0000","last_change":"2024-04-25T00:37:23.967844+0000","last_active":"2024-04-25T00:37:25.890255+0000","last_peered":"2024-04-25T00:37:25.890255+0000","last_clean":"2024-04-25T00:37:25.890255+0000","last_became_active":"2024-04-25T00:37:23.967545+0000","last_became_peered":"2024-04-25T00:37:23.967545+0000","last_unstale":"2024-04-25T00:37:25.890255+0000","last_undegraded":"2024-04-25T00:37:25.890255+0000","last_fullsized":"2024-04-25T00:37:25.890255+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:03.459207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616154+0000","last_change":"2024-04-25T00:37:24.665789+0000","last_active":"2024-04-25T00:37:25.616154+0000","last_peered":"2024-04-25T00:37:25.616154+0000","last_clean":"2024-04-25T00:37:25.616154+0000","last_became_active":"2024-04-25T00:37:24.665110+0000","last_became_peered":"2024-04-25T00:37:24.665110+0000","last_unstale":"2024-04-25T00:37:25.616154+0000","last_undegraded":"2024-04-25T00:37:25.616154+0000","last_fullsized":"2024-04-25T00:37:25.616154+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:24.334663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889798+0000","last_change":"2024-04-25T00:37:23.971620+0000","last_active":"2024-04-25T00:37:25.889798+0000","last_peered":"2024-04-25T00:37:25.889798+0000","last_clean":"2024-04-25T00:37:25.889798+0000","last_became_active":"2024-04-25T00:37:23.970901+0000","last_became_peered":"2024-04-25T00:37:23.970901+0000","last_unstale":"2024-04-25T00:37:25.889798+0000","last_undegraded":"2024-04-25T00:37:25.889798+0000","last_fullsized":"2024-04-25T00:37:25.889798+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:54.449682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.618554+0000","last_change":"2024-04-25T00:37:24.654380+0000","last_active":"2024-04-25T00:37:25.618554+0000","last_peered":"2024-04-25T00:37:25.618554+0000","last_clean":"2024-04-25T00:37:25.618554+0000","last_became_active":"2024-04-25T00:37:24.653655+0000","last_became_peered":"2024-04-25T00:37:24.653655+0000","last_unstale":"2024-04-25T00:37:25.618554+0000","last_undegraded":"2024-04-25T00:37:25.618554+0000","last_fullsized":"2024-04-25T00:37:25.618554+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.195629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892229+0000","last_change":"2024-04-25T00:37:23.974470+0000","last_active":"2024-04-25T00:37:25.892229+0000","last_peered":"2024-04-25T00:37:25.892229+0000","last_clean":"2024-04-25T00:37:25.892229+0000","last_became_active":"2024-04-25T00:37:23.974141+0000","last_became_peered":"2024-04-25T00:37:23.974141+0000","last_unstale":"2024-04-25T00:37:25.892229+0000","last_undegraded":"2024-04-25T00:37:25.892229+0000","last_fullsized":"2024-04-25T00:37:25.892229+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:08.824434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616040+0000","last_change":"2024-04-25T00:37:24.659767+0000","last_active":"2024-04-25T00:37:25.616040+0000","last_peered":"2024-04-25T00:37:25.616040+0000","last_clean":"2024-04-25T00:37:25.616040+0000","last_became_active":"2024-04-25T00:37:24.659084+0000","last_became_peered":"2024-04-25T00:37:24.659084+0000","last_unstale":"2024-04-25T00:37:25.616040+0000","last_undegraded":"2024-04-25T00:37:25.616040+0000","last_fullsized":"2024-04-25T00:37:25.616040+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:45.475299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617781+0000","last_change":"2024-04-25T00:37:23.960317+0000","last_active":"2024-04-25T00:37:25.617781+0000","last_peered":"2024-04-25T00:37:25.617781+0000","last_clean":"2024-04-25T00:37:25.617781+0000","last_became_active":"2024-04-25T00:37:23.959614+0000","last_became_peered":"2024-04-25T00:37:23.959614+0000","last_unstale":"2024-04-25T00:37:25.617781+0000","last_undegraded":"2024-04-25T00:37:25.617781+0000","last_fullsized":"2024-04-25T00:37:25.617781+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:34.332398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889853+0000","last_change":"2024-04-25T00:37:24.651641+0000","last_active":"2024-04-25T00:37:25.889853+0000","last_peered":"2024-04-25T00:37:25.889853+0000","last_clean":"2024-04-25T00:37:25.889853+0000","last_became_active":"2024-04-25T00:37:24.649207+0000","last_became_peered":"2024-04-25T00:37:24.649207+0000","last_unstale":"2024-04-25T00:37:25.889853+0000","last_undegraded":"2024-04-25T00:37:25.889853+0000","last_fullsized":"2024-04-25T00:37:25.889853+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:45.012553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616204+0000","last_change":"2024-04-25T00:37:23.971464+0000","last_active":"2024-04-25T00:37:25.616204+0000","last_peered":"2024-04-25T00:37:25.616204+0000","last_clean":"2024-04-25T00:37:25.616204+0000","last_became_active":"2024-04-25T00:37:23.970908+0000","last_became_peered":"2024-04-25T00:37:23.970908+0000","last_unstale":"2024-04-25T00:37:25.616204+0000","last_undegraded":"2024-04-25T00:37:25.616204+0000","last_fullsized":"2024-04-25T00:37:25.616204+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:56.224951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890699+0000","last_change":"2024-04-25T00:37:24.651933+0000","last_active":"2024-04-25T00:37:25.890699+0000","last_peered":"2024-04-25T00:37:25.890699+0000","last_clean":"2024-04-25T00:37:25.890699+0000","last_became_active":"2024-04-25T00:37:24.651128+0000","last_became_peered":"2024-04-25T00:37:24.651128+0000","last_unstale":"2024-04-25T00:37:25.890699+0000","last_undegraded":"2024-04-25T00:37:25.890699+0000","last_fullsized":"2024-04-25T00:37:25.890699+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:07.838253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889662+0000","last_change":"2024-04-25T00:37:23.962738+0000","last_active":"2024-04-25T00:37:25.889662+0000","last_peered":"2024-04-25T00:37:25.889662+0000","last_clean":"2024-04-25T00:37:25.889662+0000","last_became_active":"2024-04-25T00:37:23.962330+0000","last_became_peered":"2024-04-25T00:37:23.962330+0000","last_unstale":"2024-04-25T00:37:25.889662+0000","last_undegraded":"2024-04-25T00:37:25.889662+0000","last_fullsized":"2024-04-25T00:37:25.889662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:50.647322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891917+0000","last_change":"2024-04-25T00:37:24.639458+0000","last_active":"2024-04-25T00:37:25.891917+0000","last_peered":"2024-04-25T00:37:25.891917+0000","last_clean":"2024-04-25T00:37:25.891917+0000","last_became_active":"2024-04-25T00:37:24.637555+0000","last_became_peered":"2024-04-25T00:37:24.637555+0000","last_unstale":"2024-04-25T00:37:25.891917+0000","last_undegraded":"2024-04-25T00:37:25.891917+0000","last_fullsized":"2024-04-25T00:37:25.891917+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:30.099048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891682+0000","last_change":"2024-04-25T00:37:23.973733+0000","last_active":"2024-04-25T00:37:25.891682+0000","last_peered":"2024-04-25T00:37:25.891682+0000","last_clean":"2024-04-25T00:37:25.891682+0000","last_became_active":"2024-04-25T00:37:23.973362+0000","last_became_peered":"2024-04-25T00:37:23.973362+0000","last_unstale":"2024-04-25T00:37:25.891682+0000","last_undegraded":"2024-04-25T00:37:25.891682+0000","last_fullsized":"2024-04-25T00:37:25.891682+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:51.405634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.893367+0000","last_change":"2024-04-25T00:37:24.649357+0000","last_active":"2024-04-25T00:37:25.893367+0000","last_peered":"2024-04-25T00:37:25.893367+0000","last_clean":"2024-04-25T00:37:25.893367+0000","last_became_active":"2024-04-25T00:37:24.647321+0000","last_became_peered":"2024-04-25T00:37:24.647321+0000","last_unstale":"2024-04-25T00:37:25.893367+0000","last_undegraded":"2024-04-25T00:37:25.893367+0000","last_fullsized":"2024-04-25T00:37:25.893367+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:26.159139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"102'3","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616207+0000","last_change":"2024-04-25T00:37:23.966485+0000","last_active":"2024-04-25T00:37:25.616207+0000","last_peered":"2024-04-25T00:37:25.616207+0000","last_clean":"2024-04-25T00:37:25.616207+0000","last_became_active":"2024-04-25T00:37:23.966125+0000","last_became_peered":"2024-04-25T00:37:23.966125+0000","last_unstale":"2024-04-25T00:37:25.616207+0000","last_undegraded":"2024-04-25T00:37:25.616207+0000","last_fullsized":"2024-04-25T00:37:25.616207+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:44.093942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890417+0000","last_change":"2024-04-25T00:37:24.659214+0000","last_active":"2024-04-25T00:37:25.890417+0000","last_peered":"2024-04-25T00:37:25.890417+0000","last_clean":"2024-04-25T00:37:25.890417+0000","last_became_active":"2024-04-25T00:37:24.658485+0000","last_became_peered":"2024-04-25T00:37:24.658485+0000","last_unstale":"2024-04-25T00:37:25.890417+0000","last_undegraded":"2024-04-25T00:37:25.890417+0000","last_fullsized":"2024-04-25T00:37:25.890417+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:09.576753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"102'6","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617542+0000","last_change":"2024-04-25T00:37:23.971682+0000","last_active":"2024-04-25T00:37:25.617542+0000","last_peered":"2024-04-25T00:37:25.617542+0000","last_clean":"2024-04-25T00:37:25.617542+0000","last_became_active":"2024-04-25T00:37:23.970797+0000","last_became_peered":"2024-04-25T00:37:23.970797+0000","last_unstale":"2024-04-25T00:37:25.617542+0000","last_undegraded":"2024-04-25T00:37:25.617542+0000","last_fullsized":"2024-04-25T00:37:25.617542+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:27.326430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.893051+0000","last_change":"2024-04-25T00:37:24.649497+0000","last_active":"2024-04-25T00:37:25.893051+0000","last_peered":"2024-04-25T00:37:25.893051+0000","last_clean":"2024-04-25T00:37:25.893051+0000","last_became_active":"2024-04-25T00:37:24.648738+0000","last_became_peered":"2024-04-25T00:37:24.648738+0000","last_unstale":"2024-04-25T00:37:25.893051+0000","last_undegraded":"2024-04-25T00:37:25.893051+0000","last_fullsized":"2024-04-25T00:37:25.893051+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:39.326233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891628+0000","last_change":"2024-04-25T00:37:23.980023+0000","last_active":"2024-04-25T00:37:25.891628+0000","last_peered":"2024-04-25T00:37:25.891628+0000","last_clean":"2024-04-25T00:37:25.891628+0000","last_became_active":"2024-04-25T00:37:23.979702+0000","last_became_peered":"2024-04-25T00:37:23.979702+0000","last_unstale":"2024-04-25T00:37:25.891628+0000","last_undegraded":"2024-04-25T00:37:25.891628+0000","last_fullsized":"2024-04-25T00:37:25.891628+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:36.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.893084+0000","last_change":"2024-04-25T00:37:24.663106+0000","last_active":"2024-04-25T00:37:25.893084+0000","last_peered":"2024-04-25T00:37:25.893084+0000","last_clean":"2024-04-25T00:37:25.893084+0000","last_became_active":"2024-04-25T00:37:24.662275+0000","last_became_peered":"2024-04-25T00:37:24.662275+0000","last_unstale":"2024-04-25T00:37:25.893084+0000","last_undegraded":"2024-04-25T00:37:25.893084+0000","last_fullsized":"2024-04-25T00:37:25.893084+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:49.644731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891008+0000","last_change":"2024-04-25T00:37:23.974737+0000","last_active":"2024-04-25T00:37:25.891008+0000","last_peered":"2024-04-25T00:37:25.891008+0000","last_clean":"2024-04-25T00:37:25.891008+0000","last_became_active":"2024-04-25T00:37:23.974006+0000","last_became_peered":"2024-04-25T00:37:23.974006+0000","last_unstale":"2024-04-25T00:37:25.891008+0000","last_undegraded":"2024-04-25T00:37:25.891008+0000","last_fullsized":"2024-04-25T00:37:25.891008+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:06.834610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891850+0000","last_change":"2024-04-25T00:37:24.648169+0000","last_active":"2024-04-25T00:37:25.891850+0000","last_peered":"2024-04-25T00:37:25.891850+0000","last_clean":"2024-04-25T00:37:25.891850+0000","last_became_active":"2024-04-25T00:37:24.646922+0000","last_became_peered":"2024-04-25T00:37:24.646922+0000","last_unstale":"2024-04-25T00:37:25.891850+0000","last_undegraded":"2024-04-25T00:37:25.891850+0000","last_fullsized":"2024-04-25T00:37:25.891850+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:49.957775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"102'8","reported_seq":22,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889187+0000","last_change":"2024-04-25T00:37:23.972616+0000","last_active":"2024-04-25T00:37:25.889187+0000","last_peered":"2024-04-25T00:37:25.889187+0000","last_clean":"2024-04-25T00:37:25.889187+0000","last_became_active":"2024-04-25T00:37:23.972302+0000","last_became_peered":"2024-04-25T00:37:23.972302+0000","last_unstale":"2024-04-25T00:37:25.889187+0000","last_undegraded":"2024-04-25T00:37:25.889187+0000","last_fullsized":"2024-04-25T00:37:25.889187+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:26.276154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890017+0000","last_change":"2024-04-25T00:37:24.649798+0000","last_active":"2024-04-25T00:37:25.890017+0000","last_peered":"2024-04-25T00:37:25.890017+0000","last_clean":"2024-04-25T00:37:25.890017+0000","last_became_active":"2024-04-25T00:37:24.648984+0000","last_became_peered":"2024-04-25T00:37:24.648984+0000","last_unstale":"2024-04-25T00:37:25.890017+0000","last_undegraded":"2024-04-25T00:37:25.890017+0000","last_fullsized":"2024-04-25T00:37:25.890017+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:37:41.353172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617304+0000","last_change":"2024-04-25T00:37:23.973856+0000","last_active":"2024-04-25T00:37:25.617304+0000","last_peered":"2024-04-25T00:37:25.617304+0000","last_clean":"2024-04-25T00:37:25.617304+0000","last_became_active":"2024-04-25T00:37:23.973576+0000","last_became_peered":"2024-04-25T00:37:23.973576+0000","last_unstale":"2024-04-25T00:37:25.617304+0000","last_undegraded":"2024-04-25T00:37:25.617304+0000","last_fullsized":"2024-04-25T00:37:25.617304+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:56.098633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891635+0000","last_change":"2024-04-25T00:37:24.659185+0000","last_active":"2024-04-25T00:37:25.891635+0000","last_peered":"2024-04-25T00:37:25.891635+0000","last_clean":"2024-04-25T00:37:25.891635+0000","last_became_active":"2024-04-25T00:37:24.658389+0000","last_became_peered":"2024-04-25T00:37:24.658389+0000","last_unstale":"2024-04-25T00:37:25.891635+0000","last_undegraded":"2024-04-25T00:37:25.891635+0000","last_fullsized":"2024-04-25T00:37:25.891635+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:52.832892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889893+0000","last_change":"2024-04-25T00:37:23.963264+0000","last_active":"2024-04-25T00:37:25.889893+0000","last_peered":"2024-04-25T00:37:25.889893+0000","last_clean":"2024-04-25T00:37:25.889893+0000","last_became_active":"2024-04-25T00:37:23.962746+0000","last_became_peered":"2024-04-25T00:37:23.962746+0000","last_unstale":"2024-04-25T00:37:25.889893+0000","last_undegraded":"2024-04-25T00:37:25.889893+0000","last_fullsized":"2024-04-25T00:37:25.889893+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:36.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616475+0000","last_change":"2024-04-25T00:37:24.632170+0000","last_active":"2024-04-25T00:37:25.616475+0000","last_peered":"2024-04-25T00:37:25.616475+0000","last_clean":"2024-04-25T00:37:25.616475+0000","last_became_active":"2024-04-25T00:37:24.631515+0000","last_became_peered":"2024-04-25T00:37:24.631515+0000","last_unstale":"2024-04-25T00:37:25.616475+0000","last_undegraded":"2024-04-25T00:37:25.616475+0000","last_fullsized":"2024-04-25T00:37:25.616475+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:49.957775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891186+0000","last_change":"2024-04-25T00:37:23.969113+0000","last_active":"2024-04-25T00:37:25.891186+0000","last_peered":"2024-04-25T00:37:25.891186+0000","last_clean":"2024-04-25T00:37:25.891186+0000","last_became_active":"2024-04-25T00:37:23.968373+0000","last_became_peered":"2024-04-25T00:37:23.968373+0000","last_unstale":"2024-04-25T00:37:25.891186+0000","last_undegraded":"2024-04-25T00:37:25.891186+0000","last_fullsized":"2024-04-25T00:37:25.891186+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:23.689041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617733+0000","last_change":"2024-04-25T00:37:24.643621+0000","last_active":"2024-04-25T00:37:25.617733+0000","last_peered":"2024-04-25T00:37:25.617733+0000","last_clean":"2024-04-25T00:37:25.617733+0000","last_became_active":"2024-04-25T00:37:24.641963+0000","last_became_peered":"2024-04-25T00:37:24.641963+0000","last_unstale":"2024-04-25T00:37:25.617733+0000","last_undegraded":"2024-04-25T00:37:25.617733+0000","last_fullsized":"2024-04-25T00:37:25.617733+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:49.644731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889848+0000","last_change":"2024-04-25T00:37:23.975090+0000","last_active":"2024-04-25T00:37:25.889848+0000","last_peered":"2024-04-25T00:37:25.889848+0000","last_clean":"2024-04-25T00:37:25.889848+0000","last_became_active":"2024-04-25T00:37:23.973684+0000","last_became_peered":"2024-04-25T00:37:23.973684+0000","last_unstale":"2024-04-25T00:37:25.889848+0000","last_undegraded":"2024-04-25T00:37:25.889848+0000","last_fullsized":"2024-04-25T00:37:25.889848+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:35.743952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617677+0000","last_change":"2024-04-25T00:37:24.641044+0000","last_active":"2024-04-25T00:37:25.617677+0000","last_peered":"2024-04-25T00:37:25.617677+0000","last_clean":"2024-04-25T00:37:25.617677+0000","last_became_active":"2024-04-25T00:37:24.635401+0000","last_became_peered":"2024-04-25T00:37:24.635401+0000","last_unstale":"2024-04-25T00:37:25.617677+0000","last_undegraded":"2024-04-25T00:37:25.617677+0000","last_fullsized":"2024-04-25T00:37:25.617677+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:39.326233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891670+0000","last_change":"2024-04-25T00:37:23.962101+0000","last_active":"2024-04-25T00:37:25.891670+0000","last_peered":"2024-04-25T00:37:25.891670+0000","last_clean":"2024-04-25T00:37:25.891670+0000","last_became_active":"2024-04-25T00:37:23.961767+0000","last_became_peered":"2024-04-25T00:37:23.961767+0000","last_unstale":"2024-04-25T00:37:25.891670+0000","last_undegraded":"2024-04-25T00:37:25.891670+0000","last_fullsized":"2024-04-25T00:37:25.891670+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:05.179578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616518+0000","last_change":"2024-04-25T00:37:24.624859+0000","last_active":"2024-04-25T00:37:25.616518+0000","last_peered":"2024-04-25T00:37:25.616518+0000","last_clean":"2024-04-25T00:37:25.616518+0000","last_became_active":"2024-04-25T00:37:24.624061+0000","last_became_peered":"2024-04-25T00:37:24.624061+0000","last_unstale":"2024-04-25T00:37:25.616518+0000","last_undegraded":"2024-04-25T00:37:25.616518+0000","last_fullsized":"2024-04-25T00:37:25.616518+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:10.228348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889401+0000","last_change":"2024-04-25T00:37:23.962517+0000","last_active":"2024-04-25T00:37:25.889401+0000","last_peered":"2024-04-25T00:37:25.889401+0000","last_clean":"2024-04-25T00:37:25.889401+0000","last_became_active":"2024-04-25T00:37:23.962068+0000","last_became_peered":"2024-04-25T00:37:23.962068+0000","last_unstale":"2024-04-25T00:37:25.889401+0000","last_undegraded":"2024-04-25T00:37:25.889401+0000","last_fullsized":"2024-04-25T00:37:25.889401+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:45.756449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616212+0000","last_change":"2024-04-25T00:37:24.656268+0000","last_active":"2024-04-25T00:37:25.616212+0000","last_peered":"2024-04-25T00:37:25.616212+0000","last_clean":"2024-04-25T00:37:25.616212+0000","last_became_active":"2024-04-25T00:37:24.655483+0000","last_became_peered":"2024-04-25T00:37:24.655483+0000","last_unstale":"2024-04-25T00:37:25.616212+0000","last_undegraded":"2024-04-25T00:37:25.616212+0000","last_fullsized":"2024-04-25T00:37:25.616212+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:57.228594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891056+0000","last_change":"2024-04-25T00:37:23.962073+0000","last_active":"2024-04-25T00:37:25.891056+0000","last_peered":"2024-04-25T00:37:25.891056+0000","last_clean":"2024-04-25T00:37:25.891056+0000","last_became_active":"2024-04-25T00:37:23.961617+0000","last_became_peered":"2024-04-25T00:37:23.961617+0000","last_unstale":"2024-04-25T00:37:25.891056+0000","last_undegraded":"2024-04-25T00:37:25.891056+0000","last_fullsized":"2024-04-25T00:37:25.891056+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:50.647322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892296+0000","last_change":"2024-04-25T00:37:24.657468+0000","last_active":"2024-04-25T00:37:25.892296+0000","last_peered":"2024-04-25T00:37:25.892296+0000","last_clean":"2024-04-25T00:37:25.892296+0000","last_became_active":"2024-04-25T00:37:24.656654+0000","last_became_peered":"2024-04-25T00:37:24.656654+0000","last_unstale":"2024-04-25T00:37:25.892296+0000","last_undegraded":"2024-04-25T00:37:25.892296+0000","last_fullsized":"2024-04-25T00:37:25.892296+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:21.524079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616345+0000","last_change":"2024-04-25T00:37:23.974633+0000","last_active":"2024-04-25T00:37:25.616345+0000","last_peered":"2024-04-25T00:37:25.616345+0000","last_clean":"2024-04-25T00:37:25.616345+0000","last_became_active":"2024-04-25T00:37:23.974067+0000","last_became_peered":"2024-04-25T00:37:23.974067+0000","last_unstale":"2024-04-25T00:37:25.616345+0000","last_undegraded":"2024-04-25T00:37:25.616345+0000","last_fullsized":"2024-04-25T00:37:25.616345+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:08.911836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891425+0000","last_change":"2024-04-25T00:37:24.639298+0000","last_active":"2024-04-25T00:37:25.891425+0000","last_peered":"2024-04-25T00:37:25.891425+0000","last_clean":"2024-04-25T00:37:25.891425+0000","last_became_active":"2024-04-25T00:37:24.637729+0000","last_became_peered":"2024-04-25T00:37:24.637729+0000","last_unstale":"2024-04-25T00:37:25.891425+0000","last_undegraded":"2024-04-25T00:37:25.891425+0000","last_fullsized":"2024-04-25T00:37:25.891425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:38.140994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892247+0000","last_change":"2024-04-25T00:37:23.973640+0000","last_active":"2024-04-25T00:37:25.892247+0000","last_peered":"2024-04-25T00:37:25.892247+0000","last_clean":"2024-04-25T00:37:25.892247+0000","last_became_active":"2024-04-25T00:37:23.973210+0000","last_became_peered":"2024-04-25T00:37:23.973210+0000","last_unstale":"2024-04-25T00:37:25.892247+0000","last_undegraded":"2024-04-25T00:37:25.892247+0000","last_fullsized":"2024-04-25T00:37:25.892247+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:56.021774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617275+0000","last_change":"2024-04-25T00:37:24.643054+0000","last_active":"2024-04-25T00:37:25.617275+0000","last_peered":"2024-04-25T00:37:25.617275+0000","last_clean":"2024-04-25T00:37:25.617275+0000","last_became_active":"2024-04-25T00:37:24.642305+0000","last_became_peered":"2024-04-25T00:37:24.642305+0000","last_unstale":"2024-04-25T00:37:25.617275+0000","last_undegraded":"2024-04-25T00:37:25.617275+0000","last_fullsized":"2024-04-25T00:37:25.617275+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:28.830671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"103'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891625+0000","last_change":"2024-04-25T00:37:23.976356+0000","last_active":"2024-04-25T00:37:25.891625+0000","last_peered":"2024-04-25T00:37:25.891625+0000","last_clean":"2024-04-25T00:37:25.891625+0000","last_became_active":"2024-04-25T00:37:23.975148+0000","last_became_peered":"2024-04-25T00:37:23.975148+0000","last_unstale":"2024-04-25T00:37:25.891625+0000","last_undegraded":"2024-04-25T00:37:25.891625+0000","last_fullsized":"2024-04-25T00:37:25.891625+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:50.821959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617474+0000","last_change":"2024-04-25T00:37:24.642433+0000","last_active":"2024-04-25T00:37:25.617474+0000","last_peered":"2024-04-25T00:37:25.617474+0000","last_clean":"2024-04-25T00:37:25.617474+0000","last_became_active":"2024-04-25T00:37:24.641630+0000","last_became_peered":"2024-04-25T00:37:24.641630+0000","last_unstale":"2024-04-25T00:37:25.617474+0000","last_undegraded":"2024-04-25T00:37:25.617474+0000","last_fullsized":"2024-04-25T00:37:25.617474+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:10.590475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616674+0000","last_change":"2024-04-25T00:37:23.975126+0000","last_active":"2024-04-25T00:37:25.616674+0000","last_peered":"2024-04-25T00:37:25.616674+0000","last_clean":"2024-04-25T00:37:25.616674+0000","last_became_active":"2024-04-25T00:37:23.974571+0000","last_became_peered":"2024-04-25T00:37:23.974571+0000","last_unstale":"2024-04-25T00:37:25.616674+0000","last_undegraded":"2024-04-25T00:37:25.616674+0000","last_fullsized":"2024-04-25T00:37:25.616674+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:51.950167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891114+0000","last_change":"2024-04-25T00:37:24.637810+0000","last_active":"2024-04-25T00:37:25.891114+0000","last_peered":"2024-04-25T00:37:25.891114+0000","last_clean":"2024-04-25T00:37:25.891114+0000","last_became_active":"2024-04-25T00:37:24.636719+0000","last_became_peered":"2024-04-25T00:37:24.636719+0000","last_unstale":"2024-04-25T00:37:25.891114+0000","last_undegraded":"2024-04-25T00:37:25.891114+0000","last_fullsized":"2024-04-25T00:37:25.891114+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.195629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.893433+0000","last_change":"2024-04-25T00:37:23.962739+0000","last_active":"2024-04-25T00:37:25.893433+0000","last_peered":"2024-04-25T00:37:25.893433+0000","last_clean":"2024-04-25T00:37:25.893433+0000","last_became_active":"2024-04-25T00:37:23.962425+0000","last_became_peered":"2024-04-25T00:37:23.962425+0000","last_unstale":"2024-04-25T00:37:25.893433+0000","last_undegraded":"2024-04-25T00:37:25.893433+0000","last_fullsized":"2024-04-25T00:37:25.893433+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:02.162592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617946+0000","last_change":"2024-04-25T00:37:24.640862+0000","last_active":"2024-04-25T00:37:25.617946+0000","last_peered":"2024-04-25T00:37:25.617946+0000","last_clean":"2024-04-25T00:37:25.617946+0000","last_became_active":"2024-04-25T00:37:24.640060+0000","last_became_peered":"2024-04-25T00:37:24.640060+0000","last_unstale":"2024-04-25T00:37:25.617946+0000","last_undegraded":"2024-04-25T00:37:25.617946+0000","last_fullsized":"2024-04-25T00:37:25.617946+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:09.828077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889925+0000","last_change":"2024-04-25T00:37:23.976960+0000","last_active":"2024-04-25T00:37:25.889925+0000","last_peered":"2024-04-25T00:37:25.889925+0000","last_clean":"2024-04-25T00:37:25.889925+0000","last_became_active":"2024-04-25T00:37:23.971621+0000","last_became_peered":"2024-04-25T00:37:23.971621+0000","last_unstale":"2024-04-25T00:37:25.889925+0000","last_undegraded":"2024-04-25T00:37:25.889925+0000","last_fullsized":"2024-04-25T00:37:25.889925+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:51.258332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.618498+0000","last_change":"2024-04-25T00:37:24.653749+0000","last_active":"2024-04-25T00:37:25.618498+0000","last_peered":"2024-04-25T00:37:25.618498+0000","last_clean":"2024-04-25T00:37:25.618498+0000","last_became_active":"2024-04-25T00:37:24.653059+0000","last_became_peered":"2024-04-25T00:37:24.653059+0000","last_unstale":"2024-04-25T00:37:25.618498+0000","last_undegraded":"2024-04-25T00:37:25.618498+0000","last_fullsized":"2024-04-25T00:37:25.618498+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:28.330073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617928+0000","last_change":"2024-04-25T00:37:23.963951+0000","last_active":"2024-04-25T00:37:25.617928+0000","last_peered":"2024-04-25T00:37:25.617928+0000","last_clean":"2024-04-25T00:37:25.617928+0000","last_became_active":"2024-04-25T00:37:23.963276+0000","last_became_peered":"2024-04-25T00:37:23.963276+0000","last_unstale":"2024-04-25T00:37:25.617928+0000","last_undegraded":"2024-04-25T00:37:25.617928+0000","last_fullsized":"2024-04-25T00:37:25.617928+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:41.471801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616544+0000","last_change":"2024-04-25T00:37:24.625317+0000","last_active":"2024-04-25T00:37:25.616544+0000","last_peered":"2024-04-25T00:37:25.616544+0000","last_clean":"2024-04-25T00:37:25.616544+0000","last_became_active":"2024-04-25T00:37:24.624551+0000","last_became_peered":"2024-04-25T00:37:24.624551+0000","last_unstale":"2024-04-25T00:37:25.616544+0000","last_undegraded":"2024-04-25T00:37:25.616544+0000","last_fullsized":"2024-04-25T00:37:25.616544+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:09.915478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889262+0000","last_change":"2024-04-25T00:37:23.968009+0000","last_active":"2024-04-25T00:37:25.889262+0000","last_peered":"2024-04-25T00:37:25.889262+0000","last_clean":"2024-04-25T00:37:25.889262+0000","last_became_active":"2024-04-25T00:37:23.967586+0000","last_became_peered":"2024-04-25T00:37:23.967586+0000","last_unstale":"2024-04-25T00:37:25.889262+0000","last_undegraded":"2024-04-25T00:37:25.889262+0000","last_fullsized":"2024-04-25T00:37:25.889262+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:47.615185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616472+0000","last_change":"2024-04-25T00:37:24.658718+0000","last_active":"2024-04-25T00:37:25.616472+0000","last_peered":"2024-04-25T00:37:25.616472+0000","last_clean":"2024-04-25T00:37:25.616472+0000","last_became_active":"2024-04-25T00:37:24.657945+0000","last_became_peered":"2024-04-25T00:37:24.657945+0000","last_unstale":"2024-04-25T00:37:25.616472+0000","last_undegraded":"2024-04-25T00:37:25.616472+0000","last_fullsized":"2024-04-25T00:37:25.616472+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:07.998140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616393+0000","last_change":"2024-04-25T00:37:23.974640+0000","last_active":"2024-04-25T00:37:25.616393+0000","last_peered":"2024-04-25T00:37:25.616393+0000","last_clean":"2024-04-25T00:37:25.616393+0000","last_became_active":"2024-04-25T00:37:23.974224+0000","last_became_peered":"2024-04-25T00:37:23.974224+0000","last_unstale":"2024-04-25T00:37:25.616393+0000","last_undegraded":"2024-04-25T00:37:25.616393+0000","last_fullsized":"2024-04-25T00:37:25.616393+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:20.520436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892767+0000","last_change":"2024-04-25T00:37:24.665407+0000","last_active":"2024-04-25T00:37:25.892767+0000","last_peered":"2024-04-25T00:37:25.892767+0000","last_clean":"2024-04-25T00:37:25.892767+0000","last_became_active":"2024-04-25T00:37:24.664257+0000","last_became_peered":"2024-04-25T00:37:24.664257+0000","last_unstale":"2024-04-25T00:37:25.892767+0000","last_undegraded":"2024-04-25T00:37:25.892767+0000","last_fullsized":"2024-04-25T00:37:25.892767+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:38.243062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890107+0000","last_change":"2024-04-25T00:37:23.974436+0000","last_active":"2024-04-25T00:37:25.890107+0000","last_peered":"2024-04-25T00:37:25.890107+0000","last_clean":"2024-04-25T00:37:25.890107+0000","last_became_active":"2024-04-25T00:37:23.973775+0000","last_became_peered":"2024-04-25T00:37:23.973775+0000","last_unstale":"2024-04-25T00:37:25.890107+0000","last_undegraded":"2024-04-25T00:37:25.890107+0000","last_fullsized":"2024-04-25T00:37:25.890107+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:09.024737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":368,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616842+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:37:25.616842+0000","last_peered":"2024-04-25T00:37:25.616842+0000","last_clean":"2024-04-25T00:37:25.616842+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:37:25.616842+0000","last_undegraded":"2024-04-25T00:37:25.616842+0000","last_fullsized":"2024-04-25T00:37:25.616842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37192,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1884160,"data_stored":1153585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1286144,"data_stored":562671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":563205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1282048,"data_stored":562601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36608,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335232,"statfs":{"total":96636764160,"available":96599277568,"internally_reserved":0,"allocated":1298432,"data_stored":564283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36592,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335248,"statfs":{"total":96636764160,"available":96599293952,"internally_reserved":0,"allocated":1282048,"data_stored":562601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36600,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335240,"statfs":{"total":96636764160,"available":96599285760,"internally_reserved":0,"allocated":1290240,"data_stored":562725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":1154693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:37:29.742 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:37:29.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.895+0000 7f686ba8b640 1 Processor -- start 2024-04-25T00:37:29.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.895+0000 7f686ba8b640 1 -- start start 2024-04-25T00:37:29.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f686ba8b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f686415e8d0 0x7f68640cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f686ba8b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686415f3c0 0x7f686415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f686ba8b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68640b3720 con 0x7f68640b4780 2024-04-25T00:37:29.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f686ba8b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f68640b42f0 con 0x7f686415e8d0 2024-04-25T00:37:29.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f686ba8b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f68640b39c0 con 0x7f686415f3c0 2024-04-25T00:37:29.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6869026640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f68640b4780 0x7f686415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58026/0 (socket says 172.21.15.29:58026) 2024-04-25T00:37:29.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6869026640 1 -- 172.21.15.29:0/4056270136 learned_addr learned my addr 172.21.15.29:0/4056270136 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:29.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6869827640 1 --2- 172.21.15.29:0/4056270136 >> v2:172.21.15.157:3300/0 conn(0x7f686415e8d0 0x7f68640cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f686a028640 1 --2- 172.21.15.29:0/4056270136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686415f3c0 0x7f686415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3713783373 0 0) 0x7f68640b3720 con 0x7f68640b4780 2024-04-25T00:37:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6850003530 con 0x7f68640b4780 2024-04-25T00:37:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 673965399 0 0) 0x7f6850003530 con 0x7f68640b4780 2024-04-25T00:37:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686415f3c0 msgr2=0x7f686415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 --2- 172.21.15.29:0/4056270136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686415f3c0 0x7f686415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 >> v2:172.21.15.157:3300/0 conn(0x7f686415e8d0 msgr2=0x7f68640cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 --2- 172.21.15.29:0/4056270136 >> v2:172.21.15.157:3300/0 conn(0x7f686415e8d0 0x7f68640cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68640b3ca0 con 0x7f68640b4780 2024-04-25T00:37:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f6854001680 con 0x7f68640b4780 2024-04-25T00:37:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 >> v1:172.21.15.29:6789/0 conn(0x7f68640b4780 legacy=0x7f686415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 --2- 172.21.15.29:0/4056270136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68500039c0 0x7f6850003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 --2- 172.21.15.29:0/4056270136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6850004810 0x7f6850004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 --2- 172.21.15.29:0/4056270136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686415f3c0 0x7f68500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f68640b42f0 con 0x7f68500039c0 2024-04-25T00:37:29.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f68640b39c0 con 0x7f6850004810 2024-04-25T00:37:29.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f685000b9d0 con 0x7f686415f3c0 2024-04-25T00:37:29.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6869827640 1 --2- 172.21.15.29:0/4056270136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68500039c0 0x7f6850003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6869026640 1 --2- 172.21.15.29:0/4056270136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6850004810 0x7f6850004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f686a028640 1 --2- 172.21.15.29:0/4056270136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686415f3c0 0x7f68500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6869827640 1 -- 172.21.15.29:0/4056270136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686415f3c0 msgr2=0x7f68500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6869827640 1 --2- 172.21.15.29:0/4056270136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686415f3c0 0x7f68500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6869827640 1 -- 172.21.15.29:0/4056270136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6850004810 msgr2=0x7f6850004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6869827640 1 --2- 172.21.15.29:0/4056270136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6850004810 0x7f6850004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6869827640 1 -- 172.21.15.29:0/4056270136 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6850003530 con 0x7f68500039c0 2024-04-25T00:37:29.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f686a028640 1 --2- 172.21.15.29:0/4056270136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f686415f3c0 0x7f68500051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:29.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6869827640 1 --2- 172.21.15.29:0/4056270136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68500039c0 0x7f6850003de0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6860009ae0 tx=0x7f686001ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea30788c7fcb046 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600096b0 con 0x7f68500039c0 2024-04-25T00:37:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686415d7e0 con 0x7f68500039c0 2024-04-25T00:37:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.899+0000 7f6868825640 1 -- 172.21.15.29:0/4056270136 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600a66d0 con 0x7f68500039c0 2024-04-25T00:37:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 -- 172.21.15.29:0/4056270136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68500039c0 msgr2=0x7f6850003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 --2- 172.21.15.29:0/4056270136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68500039c0 0x7f6850003de0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6860009ae0 tx=0x7f686001ca80 comp rx=0 tx=0).stop 2024-04-25T00:37:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686a028640 1 -- 172.21.15.29:0/4056270136 reap_dead start 2024-04-25T00:37:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 -- 172.21.15.29:0/4056270136 shutdown_connections 2024-04-25T00:37:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 -- 172.21.15.29:0/4056270136 >> 172.21.15.29:0/4056270136 conn(0x7f68640b1870 msgr2=0x7f68640b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 -- 172.21.15.29:0/4056270136 shutdown_connections 2024-04-25T00:37:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 -- 172.21.15.29:0/4056270136 wait complete. 2024-04-25T00:37:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 Processor -- start 2024-04-25T00:37:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 -- start start 2024-04-25T00:37:29.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68500039c0 0x7f6864159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6864142ab0 0x7f6864162b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686415a320 0x7f6864163040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f686415d160 con 0x7f68500039c0 2024-04-25T00:37:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f686415cfe0 con 0x7f686415a320 2024-04-25T00:37:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f686415d2e0 con 0x7f6864142ab0 2024-04-25T00:37:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f6869827640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68500039c0 0x7f6864159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f6869827640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68500039c0 0x7f6864159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43660/0 (socket says 172.21.15.29:43660) 2024-04-25T00:37:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f6869827640 1 -- 172.21.15.29:0/1208379110 learned_addr learned my addr 172.21.15.29:0/1208379110 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f6869026640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6864142ab0 0x7f6864162b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686a028640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686415a320 0x7f6864163040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f6869827640 1 -- 172.21.15.29:0/1208379110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6864142ab0 msgr2=0x7f6864162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f6869827640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6864142ab0 0x7f6864162b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f6869827640 1 -- 172.21.15.29:0/1208379110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686415a320 msgr2=0x7f6864163040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f6869827640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686415a320 0x7f6864163040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f6869827640 1 -- 172.21.15.29:0/1208379110 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686415a130 con 0x7f68500039c0 2024-04-25T00:37:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686a028640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686415a320 0x7f6864163040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f6869827640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68500039c0 0x7f6864159bf0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f686001c8b0 tx=0x7f6850005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f685a7fc640 1 -- 172.21.15.29:0/1208379110 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600a6a30 con 0x7f68500039c0 2024-04-25T00:37:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f685a7fc640 1 -- 172.21.15.29:0/1208379110 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68600a6c10 con 0x7f68500039c0 2024-04-25T00:37:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 -- 172.21.15.29:0/1208379110 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686415fa90 con 0x7f68500039c0 2024-04-25T00:37:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f685a7fc640 1 -- 172.21.15.29:0/1208379110 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600a7470 con 0x7f68500039c0 2024-04-25T00:37:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.903+0000 7f686ba8b640 1 -- 172.21.15.29:0/1208379110 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6864143300 con 0x7f68500039c0 2024-04-25T00:37:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.907+0000 7f686ba8b640 1 -- 172.21.15.29:0/1208379110 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6864069270 con 0x7f68500039c0 2024-04-25T00:37:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.907+0000 7f685a7fc640 1 -- 172.21.15.29:0/1208379110 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f68600a7610 con 0x7f68500039c0 2024-04-25T00:37:29.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.907+0000 7f685a7fc640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6848067a10 0x7f6848069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.907+0000 7f685a7fc640 1 -- 172.21.15.29:0/1208379110 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f6860120310 con 0x7f68500039c0 2024-04-25T00:37:29.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.907+0000 7f6869026640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6848067a10 0x7f6848069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.907+0000 7f6869026640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6848067a10 0x7f6848069ed0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6854001ec0 tx=0x7f6854000f00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:29.911+0000 7f685a7fc640 1 -- 172.21.15.29:0/1208379110 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f68600ac020 con 0x7f68500039c0 2024-04-25T00:37:30.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.131+0000 7f686ba8b640 1 -- 172.21.15.29:0/1208379110 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f686406e7b0 con 0x7f68500039c0 2024-04-25T00:37:30.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.131+0000 7f685a7fc640 1 -- 172.21.15.29:0/1208379110 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1562 (secure 0 0 0) 0x7f68600167c0 con 0x7f68500039c0 2024-04-25T00:37:30.135 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:30.135 INFO:teuthology.orchestra.run.smithi029.stdout:{"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":377,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":103,"num_osds":8,"num_up_osds":8,"osd_up_since":1714005083,"num_in_osds":8,"osd_in_since":1714005074,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":296771584,"bytes_avail":772797341696,"bytes_total":773094113280,"write_bytes_sec":2382,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":67,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":7896}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-04-25T00:37:29.348310+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:37:30.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.135+0000 7f686ba8b640 1 -- 172.21.15.29:0/1208379110 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6848067a10 msgr2=0x7f6848069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.135+0000 7f686ba8b640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6848067a10 0x7f6848069ed0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6854001ec0 tx=0x7f6854000f00 comp rx=0 tx=0).stop 2024-04-25T00:37:30.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.135+0000 7f686ba8b640 1 -- 172.21.15.29:0/1208379110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68500039c0 msgr2=0x7f6864159bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.135+0000 7f686ba8b640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68500039c0 0x7f6864159bf0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f686001c8b0 tx=0x7f6850005840 comp rx=0 tx=0).stop 2024-04-25T00:37:30.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.135+0000 7f686ba8b640 1 -- 172.21.15.29:0/1208379110 shutdown_connections 2024-04-25T00:37:30.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.135+0000 7f686ba8b640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6848067a10 0x7f6848069ed0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.135+0000 7f686ba8b640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f686415a320 0x7f6864163040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.135+0000 7f686ba8b640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6864142ab0 0x7f6864162b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.135+0000 7f686ba8b640 1 --2- 172.21.15.29:0/1208379110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68500039c0 0x7f6864159bf0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.135+0000 7f686ba8b640 1 -- 172.21.15.29:0/1208379110 >> 172.21.15.29:0/1208379110 conn(0x7f68640b1870 msgr2=0x7f68640b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:30.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.135+0000 7f686ba8b640 1 -- 172.21.15.29:0/1208379110 shutdown_connections 2024-04-25T00:37:30.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.135+0000 7f686ba8b640 1 -- 172.21.15.29:0/1208379110 wait complete. 2024-04-25T00:37:30.154 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T00:37:30.154 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:37:30.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.275+0000 7f702513d640 1 Processor -- start 2024-04-25T00:37:30.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.275+0000 7f702513d640 1 -- start start 2024-04-25T00:37:30.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.275+0000 7f702513d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70200b3c30 0x7f702006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.275+0000 7f702513d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70200b41e0 con 0x7f70200b4b10 2024-04-25T00:37:30.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.275+0000 7f702513d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f702006d960 con 0x7f70200b3140 2024-04-25T00:37:30.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.275+0000 7f702513d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f702006dc40 con 0x7f70200b3c30 2024-04-25T00:37:30.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.275+0000 7f701e59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70200b3c30 0x7f702006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.275+0000 7f701e59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70200b3c30 0x7f702006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43670/0 (socket says 172.21.15.29:43670) 2024-04-25T00:37:30.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.275+0000 7f701e59c640 1 -- 172.21.15.29:0/1735142197 learned_addr learned my addr 172.21.15.29:0/1735142197 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:30.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.275+0000 7f701e59c640 1 -- 172.21.15.29:0/1735142197 >> v1:172.21.15.157:6789/0 conn(0x7f70200b3140 legacy=0x7f70200b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:30.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.275+0000 7f701e59c640 1 -- 172.21.15.29:0/1735142197 >> v1:172.21.15.29:6789/0 conn(0x7f70200b4b10 legacy=0x7f70200ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.275+0000 7f701e59c640 1 -- 172.21.15.29:0/1735142197 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702006df20 con 0x7f70200b3c30 2024-04-25T00:37:30.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.275+0000 7f701dd9b640 1 -- 172.21.15.29:0/1735142197 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 50774522 0 0) 0x7f70200b41e0 con 0x7f70200b4b10 2024-04-25T00:37:30.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f701e59c640 1 --2- 172.21.15.29:0/1735142197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70200b3c30 0x7f702006e440 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f70140098a0 tx=0x7f701401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d6fb928ae1f9fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:30.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f701dd9b640 1 -- 172.21.15.29:0/1735142197 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70140b0070 con 0x7f70200b3c30 2024-04-25T00:37:30.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f701dd9b640 1 -- 172.21.15.29:0/1735142197 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70140ab600 con 0x7f70200b3c30 2024-04-25T00:37:30.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f701dd9b640 1 -- 172.21.15.29:0/1735142197 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70140ab9d0 con 0x7f70200b3c30 2024-04-25T00:37:30.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 -- 172.21.15.29:0/1735142197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70200b3c30 msgr2=0x7f702006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 --2- 172.21.15.29:0/1735142197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70200b3c30 0x7f702006e440 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f70140098a0 tx=0x7f701401c500 comp rx=0 tx=0).stop 2024-04-25T00:37:30.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 -- 172.21.15.29:0/1735142197 shutdown_connections 2024-04-25T00:37:30.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 --2- 172.21.15.29:0/1735142197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70200b3c30 0x7f702006e440 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 -- 172.21.15.29:0/1735142197 >> 172.21.15.29:0/1735142197 conn(0x7f70200b1870 msgr2=0x7f70200b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:30.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 -- 172.21.15.29:0/1735142197 shutdown_connections 2024-04-25T00:37:30.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 -- 172.21.15.29:0/1735142197 wait complete. 2024-04-25T00:37:30.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 Processor -- start 2024-04-25T00:37:30.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 -- start start 2024-04-25T00:37:30.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70200b3140 0x7f702013fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70200b3c30 0x7f7020138b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70200b4b10 0x7f7020142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7020169d90 con 0x7f70200b3c30 2024-04-25T00:37:30.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7020169c10 con 0x7f70200b4b10 2024-04-25T00:37:30.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f702513d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7020169f10 con 0x7f70200b3140 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f701e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70200b3c30 0x7f7020138b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f701e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70200b3c30 0x7f7020138b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43676/0 (socket says 172.21.15.29:43676) 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.279+0000 7f701e59c640 1 -- 172.21.15.29:0/1184480792 learned_addr learned my addr 172.21.15.29:0/1184480792 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f701f59e640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70200b4b10 0x7f7020142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f701ed9d640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70200b3140 0x7f702013fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f701e59c640 1 -- 172.21.15.29:0/1184480792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70200b3140 msgr2=0x7f702013fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f701e59c640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70200b3140 0x7f702013fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f701e59c640 1 -- 172.21.15.29:0/1184480792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70200b4b10 msgr2=0x7f7020142b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f701e59c640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70200b4b10 0x7f7020142b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f701e59c640 1 -- 172.21.15.29:0/1184480792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702013d890 con 0x7f70200b3c30 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f701ed9d640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70200b3140 0x7f702013fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f701e59c640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70200b3c30 0x7f7020138b90 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f7014019110 tx=0x7f701401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f70077fe640 1 -- 172.21.15.29:0/1184480792 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70140b0040 con 0x7f70200b3c30 2024-04-25T00:37:30.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f70077fe640 1 -- 172.21.15.29:0/1184480792 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70140bc070 con 0x7f70200b3c30 2024-04-25T00:37:30.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f70077fe640 1 -- 172.21.15.29:0/1184480792 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70140b6880 con 0x7f70200b3c30 2024-04-25T00:37:30.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f702513d640 1 -- 172.21.15.29:0/1184480792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702013db70 con 0x7f70200b3c30 2024-04-25T00:37:30.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f702513d640 1 -- 172.21.15.29:0/1184480792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702013e130 con 0x7f70200b3c30 2024-04-25T00:37:30.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f702513d640 1 -- 172.21.15.29:0/1184480792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7020069270 con 0x7f70200b3c30 2024-04-25T00:37:30.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f70077fe640 1 -- 172.21.15.29:0/1184480792 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f70140b6a20 con 0x7f70200b3c30 2024-04-25T00:37:30.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f70077fe640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f700c0679e0 0x7f700c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:30.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f70077fe640 1 -- 172.21.15.29:0/1184480792 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f70141202b0 con 0x7f70200b3c30 2024-04-25T00:37:30.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f701ed9d640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f700c0679e0 0x7f700c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:30.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.283+0000 7f701ed9d640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f700c0679e0 0x7f700c069ea0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70201f81f0 tx=0x7f7010008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:30.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.287+0000 7f70077fe640 1 -- 172.21.15.29:0/1184480792 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f70140160f0 con 0x7f70200b3c30 2024-04-25T00:37:30.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.447+0000 7f702513d640 1 -- 172.21.15.29:0/1184480792 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70200753e0 con 0x7f700c0679e0 2024-04-25T00:37:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.455+0000 7f70077fe640 1 -- 172.21.15.29:0/1184480792 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7f70200753e0 con 0x7f700c0679e0 2024-04-25T00:37:30.459 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:30.460 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:37:30.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.459+0000 7f702513d640 1 -- 172.21.15.29:0/1184480792 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f700c0679e0 msgr2=0x7f700c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.459+0000 7f702513d640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f700c0679e0 0x7f700c069ea0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70201f81f0 tx=0x7f7010008040 comp rx=0 tx=0).stop 2024-04-25T00:37:30.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.459+0000 7f702513d640 1 -- 172.21.15.29:0/1184480792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70200b3c30 msgr2=0x7f7020138b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:30.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.459+0000 7f702513d640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70200b3c30 0x7f7020138b90 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f7014019110 tx=0x7f701401ad40 comp rx=0 tx=0).stop 2024-04-25T00:37:30.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.459+0000 7f702513d640 1 -- 172.21.15.29:0/1184480792 shutdown_connections 2024-04-25T00:37:30.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.459+0000 7f702513d640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f700c0679e0 0x7f700c069ea0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.459+0000 7f702513d640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70200b4b10 0x7f7020142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.459+0000 7f702513d640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70200b3c30 0x7f7020138b90 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.459+0000 7f702513d640 1 --2- 172.21.15.29:0/1184480792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70200b3140 0x7f702013fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:30.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.459+0000 7f702513d640 1 -- 172.21.15.29:0/1184480792 >> 172.21.15.29:0/1184480792 conn(0x7f70200b1870 msgr2=0x7f7020154030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:30.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.459+0000 7f702513d640 1 -- 172.21.15.29:0/1184480792 shutdown_connections 2024-04-25T00:37:30.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:30.459+0000 7f702513d640 1 -- 172.21.15.29:0/1184480792 wait complete. 2024-04-25T00:37:30.478 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-25T00:37:29.347985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289816,"kb_used_data":11240,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754684904,"statfs":{"total":773094113280,"available":772797341696,"internally_reserved":0,"allocated":11509760,"data_stored":5686364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.017469"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617055+0000","last_change":"2024-04-25T00:37:24.657606+0000","last_active":"2024-04-25T00:37:25.617055+0000","last_peered":"2024-04-25T00:37:25.617055+0000","last_clean":"2024-04-25T00:37:25.617055+0000","last_became_active":"2024-04-25T00:37:24.656859+0000","last_became_peered":"2024-04-25T00:37:24.656859+0000","last_unstale":"2024-04-25T00:37:25.617055+0000","last_undegraded":"2024-04-25T00:37:25.617055+0000","last_fullsized":"2024-04-25T00:37:25.617055+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:16.259398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619452+0000","last_change":"2024-04-25T00:37:24.648164+0000","last_active":"2024-04-25T00:37:25.619452+0000","last_peered":"2024-04-25T00:37:25.619452+0000","last_clean":"2024-04-25T00:37:25.619452+0000","last_became_active":"2024-04-25T00:37:24.647383+0000","last_became_peered":"2024-04-25T00:37:24.647383+0000","last_unstale":"2024-04-25T00:37:25.619452+0000","last_undegraded":"2024-04-25T00:37:25.619452+0000","last_fullsized":"2024-04-25T00:37:25.619452+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:57.102276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616600+0000","last_change":"2024-04-25T00:37:24.644232+0000","last_active":"2024-04-25T00:37:25.616600+0000","last_peered":"2024-04-25T00:37:25.616600+0000","last_clean":"2024-04-25T00:37:25.616600+0000","last_became_active":"2024-04-25T00:37:24.642886+0000","last_became_peered":"2024-04-25T00:37:24.642886+0000","last_unstale":"2024-04-25T00:37:25.616600+0000","last_undegraded":"2024-04-25T00:37:25.616600+0000","last_fullsized":"2024-04-25T00:37:25.616600+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:21.524079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617224+0000","last_change":"2024-04-25T00:37:24.657429+0000","last_active":"2024-04-25T00:37:25.617224+0000","last_peered":"2024-04-25T00:37:25.617224+0000","last_clean":"2024-04-25T00:37:25.617224+0000","last_became_active":"2024-04-25T00:37:24.656738+0000","last_became_peered":"2024-04-25T00:37:24.656738+0000","last_unstale":"2024-04-25T00:37:25.617224+0000","last_undegraded":"2024-04-25T00:37:25.617224+0000","last_fullsized":"2024-04-25T00:37:25.617224+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:37.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890462+0000","last_change":"2024-04-25T00:37:24.650575+0000","last_active":"2024-04-25T00:37:25.890462+0000","last_peered":"2024-04-25T00:37:25.890462+0000","last_clean":"2024-04-25T00:37:25.890462+0000","last_became_active":"2024-04-25T00:37:24.649239+0000","last_became_peered":"2024-04-25T00:37:24.649239+0000","last_unstale":"2024-04-25T00:37:25.890462+0000","last_undegraded":"2024-04-25T00:37:25.890462+0000","last_fullsized":"2024-04-25T00:37:25.890462+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:09.766305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892915+0000","last_change":"2024-04-25T00:37:24.655067+0000","last_active":"2024-04-25T00:37:25.892915+0000","last_peered":"2024-04-25T00:37:25.892915+0000","last_clean":"2024-04-25T00:37:25.892915+0000","last_became_active":"2024-04-25T00:37:24.654401+0000","last_became_peered":"2024-04-25T00:37:24.654401+0000","last_unstale":"2024-04-25T00:37:25.892915+0000","last_undegraded":"2024-04-25T00:37:25.892915+0000","last_fullsized":"2024-04-25T00:37:25.892915+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:36.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890044+0000","last_change":"2024-04-25T00:37:24.649776+0000","last_active":"2024-04-25T00:37:25.890044+0000","last_peered":"2024-04-25T00:37:25.890044+0000","last_clean":"2024-04-25T00:37:25.890044+0000","last_became_active":"2024-04-25T00:37:24.648979+0000","last_became_peered":"2024-04-25T00:37:24.648979+0000","last_unstale":"2024-04-25T00:37:25.890044+0000","last_undegraded":"2024-04-25T00:37:25.890044+0000","last_fullsized":"2024-04-25T00:37:25.890044+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:44.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616146+0000","last_change":"2024-04-25T00:37:24.655900+0000","last_active":"2024-04-25T00:37:25.616146+0000","last_peered":"2024-04-25T00:37:25.616146+0000","last_clean":"2024-04-25T00:37:25.616146+0000","last_became_active":"2024-04-25T00:37:24.655169+0000","last_became_peered":"2024-04-25T00:37:24.655169+0000","last_unstale":"2024-04-25T00:37:25.616146+0000","last_undegraded":"2024-04-25T00:37:25.616146+0000","last_fullsized":"2024-04-25T00:37:25.616146+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:57.102276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617147+0000","last_change":"2024-04-25T00:37:24.644695+0000","last_active":"2024-04-25T00:37:25.617147+0000","last_peered":"2024-04-25T00:37:25.617147+0000","last_clean":"2024-04-25T00:37:25.617147+0000","last_became_active":"2024-04-25T00:37:24.643886+0000","last_became_peered":"2024-04-25T00:37:24.643886+0000","last_unstale":"2024-04-25T00:37:25.617147+0000","last_undegraded":"2024-04-25T00:37:25.617147+0000","last_fullsized":"2024-04-25T00:37:25.617147+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:24.434950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616914+0000","last_change":"2024-04-25T00:37:24.668000+0000","last_active":"2024-04-25T00:37:25.616914+0000","last_peered":"2024-04-25T00:37:25.616914+0000","last_clean":"2024-04-25T00:37:25.616914+0000","last_became_active":"2024-04-25T00:37:24.663574+0000","last_became_peered":"2024-04-25T00:37:24.663574+0000","last_unstale":"2024-04-25T00:37:25.616914+0000","last_undegraded":"2024-04-25T00:37:25.616914+0000","last_fullsized":"2024-04-25T00:37:25.616914+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:37.489323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619004+0000","last_change":"2024-04-25T00:37:24.649566+0000","last_active":"2024-04-25T00:37:25.619004+0000","last_peered":"2024-04-25T00:37:25.619004+0000","last_clean":"2024-04-25T00:37:25.619004+0000","last_became_active":"2024-04-25T00:37:24.648905+0000","last_became_peered":"2024-04-25T00:37:24.648905+0000","last_unstale":"2024-04-25T00:37:25.619004+0000","last_undegraded":"2024-04-25T00:37:25.619004+0000","last_fullsized":"2024-04-25T00:37:25.619004+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:30.099048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617355+0000","last_change":"2024-04-25T00:37:24.659945+0000","last_active":"2024-04-25T00:37:25.617355+0000","last_peered":"2024-04-25T00:37:25.617355+0000","last_clean":"2024-04-25T00:37:25.617355+0000","last_became_active":"2024-04-25T00:37:24.659273+0000","last_became_peered":"2024-04-25T00:37:24.659273+0000","last_unstale":"2024-04-25T00:37:25.617355+0000","last_undegraded":"2024-04-25T00:37:25.617355+0000","last_fullsized":"2024-04-25T00:37:25.617355+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:12.673417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617382+0000","last_change":"2024-04-25T00:37:24.656446+0000","last_active":"2024-04-25T00:37:25.617382+0000","last_peered":"2024-04-25T00:37:25.617382+0000","last_clean":"2024-04-25T00:37:25.617382+0000","last_became_active":"2024-04-25T00:37:24.655626+0000","last_became_peered":"2024-04-25T00:37:24.655626+0000","last_unstale":"2024-04-25T00:37:25.617382+0000","last_undegraded":"2024-04-25T00:37:25.617382+0000","last_fullsized":"2024-04-25T00:37:25.617382+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:07.618067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616119+0000","last_change":"2024-04-25T00:37:24.625020+0000","last_active":"2024-04-25T00:37:25.616119+0000","last_peered":"2024-04-25T00:37:25.616119+0000","last_clean":"2024-04-25T00:37:25.616119+0000","last_became_active":"2024-04-25T00:37:24.624274+0000","last_became_peered":"2024-04-25T00:37:24.624274+0000","last_unstale":"2024-04-25T00:37:25.616119+0000","last_undegraded":"2024-04-25T00:37:25.616119+0000","last_fullsized":"2024-04-25T00:37:25.616119+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:52.953810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890889+0000","last_change":"2024-04-25T00:37:24.653503+0000","last_active":"2024-04-25T00:37:25.890889+0000","last_peered":"2024-04-25T00:37:25.890889+0000","last_clean":"2024-04-25T00:37:25.890889+0000","last_became_active":"2024-04-25T00:37:24.652726+0000","last_became_peered":"2024-04-25T00:37:24.652726+0000","last_unstale":"2024-04-25T00:37:25.890889+0000","last_undegraded":"2024-04-25T00:37:25.890889+0000","last_fullsized":"2024-04-25T00:37:25.890889+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:36.836347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616099+0000","last_change":"2024-04-25T00:37:24.665542+0000","last_active":"2024-04-25T00:37:25.616099+0000","last_peered":"2024-04-25T00:37:25.616099+0000","last_clean":"2024-04-25T00:37:25.616099+0000","last_became_active":"2024-04-25T00:37:24.664831+0000","last_became_peered":"2024-04-25T00:37:24.664831+0000","last_unstale":"2024-04-25T00:37:25.616099+0000","last_undegraded":"2024-04-25T00:37:25.616099+0000","last_fullsized":"2024-04-25T00:37:25.616099+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:28.766277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619433+0000","last_change":"2024-04-25T00:37:24.650948+0000","last_active":"2024-04-25T00:37:25.619433+0000","last_peered":"2024-04-25T00:37:25.619433+0000","last_clean":"2024-04-25T00:37:25.619433+0000","last_became_active":"2024-04-25T00:37:24.649607+0000","last_became_peered":"2024-04-25T00:37:24.649607+0000","last_unstale":"2024-04-25T00:37:25.619433+0000","last_undegraded":"2024-04-25T00:37:25.619433+0000","last_fullsized":"2024-04-25T00:37:25.619433+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:52.953810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891102+0000","last_change":"2024-04-25T00:37:24.641490+0000","last_active":"2024-04-25T00:37:25.891102+0000","last_peered":"2024-04-25T00:37:25.891102+0000","last_clean":"2024-04-25T00:37:25.891102+0000","last_became_active":"2024-04-25T00:37:24.640694+0000","last_became_peered":"2024-04-25T00:37:24.640694+0000","last_unstale":"2024-04-25T00:37:25.891102+0000","last_undegraded":"2024-04-25T00:37:25.891102+0000","last_fullsized":"2024-04-25T00:37:25.891102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:42.857330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617010+0000","last_change":"2024-04-25T00:37:24.656560+0000","last_active":"2024-04-25T00:37:25.617010+0000","last_peered":"2024-04-25T00:37:25.617010+0000","last_clean":"2024-04-25T00:37:25.617010+0000","last_became_active":"2024-04-25T00:37:24.655791+0000","last_became_peered":"2024-04-25T00:37:24.655791+0000","last_unstale":"2024-04-25T00:37:25.617010+0000","last_undegraded":"2024-04-25T00:37:25.617010+0000","last_fullsized":"2024-04-25T00:37:25.617010+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:21.476709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.618771+0000","last_change":"2024-04-25T00:37:24.659487+0000","last_active":"2024-04-25T00:37:25.618771+0000","last_peered":"2024-04-25T00:37:25.618771+0000","last_clean":"2024-04-25T00:37:25.618771+0000","last_became_active":"2024-04-25T00:37:24.658548+0000","last_became_peered":"2024-04-25T00:37:24.658548+0000","last_unstale":"2024-04-25T00:37:25.618771+0000","last_undegraded":"2024-04-25T00:37:25.618771+0000","last_fullsized":"2024-04-25T00:37:25.618771+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:38.140994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619151+0000","last_change":"2024-04-25T00:37:24.648480+0000","last_active":"2024-04-25T00:37:25.619151+0000","last_peered":"2024-04-25T00:37:25.619151+0000","last_clean":"2024-04-25T00:37:25.619151+0000","last_became_active":"2024-04-25T00:37:24.647516+0000","last_became_peered":"2024-04-25T00:37:24.647516+0000","last_unstale":"2024-04-25T00:37:25.619151+0000","last_undegraded":"2024-04-25T00:37:25.619151+0000","last_fullsized":"2024-04-25T00:37:25.619151+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:45.097585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892096+0000","last_change":"2024-04-25T00:37:24.663784+0000","last_active":"2024-04-25T00:37:25.892096+0000","last_peered":"2024-04-25T00:37:25.892096+0000","last_clean":"2024-04-25T00:37:25.892096+0000","last_became_active":"2024-04-25T00:37:24.662549+0000","last_became_peered":"2024-04-25T00:37:24.662549+0000","last_unstale":"2024-04-25T00:37:25.892096+0000","last_undegraded":"2024-04-25T00:37:25.892096+0000","last_fullsized":"2024-04-25T00:37:25.892096+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:09.915478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617924+0000","last_change":"2024-04-25T00:37:24.658880+0000","last_active":"2024-04-25T00:37:25.617924+0000","last_peered":"2024-04-25T00:37:25.617924+0000","last_clean":"2024-04-25T00:37:25.617924+0000","last_became_active":"2024-04-25T00:37:24.658120+0000","last_became_peered":"2024-04-25T00:37:24.658120+0000","last_unstale":"2024-04-25T00:37:25.617924+0000","last_undegraded":"2024-04-25T00:37:25.617924+0000","last_fullsized":"2024-04-25T00:37:25.617924+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:03.166234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892331+0000","last_change":"2024-04-25T00:37:24.667001+0000","last_active":"2024-04-25T00:37:25.892331+0000","last_peered":"2024-04-25T00:37:25.892331+0000","last_clean":"2024-04-25T00:37:25.892331+0000","last_became_active":"2024-04-25T00:37:24.664458+0000","last_became_peered":"2024-04-25T00:37:24.664458+0000","last_unstale":"2024-04-25T00:37:25.892331+0000","last_undegraded":"2024-04-25T00:37:25.892331+0000","last_fullsized":"2024-04-25T00:37:25.892331+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:21.476709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619962+0000","last_change":"2024-04-25T00:37:24.650335+0000","last_active":"2024-04-25T00:37:25.619962+0000","last_peered":"2024-04-25T00:37:25.619962+0000","last_clean":"2024-04-25T00:37:25.619962+0000","last_became_active":"2024-04-25T00:37:24.648644+0000","last_became_peered":"2024-04-25T00:37:24.648644+0000","last_unstale":"2024-04-25T00:37:25.619962+0000","last_undegraded":"2024-04-25T00:37:25.619962+0000","last_fullsized":"2024-04-25T00:37:25.619962+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:57.228594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617137+0000","last_change":"2024-04-25T00:37:24.638520+0000","last_active":"2024-04-25T00:37:25.617137+0000","last_peered":"2024-04-25T00:37:25.617137+0000","last_clean":"2024-04-25T00:37:25.617137+0000","last_became_active":"2024-04-25T00:37:24.637640+0000","last_became_peered":"2024-04-25T00:37:24.637640+0000","last_unstale":"2024-04-25T00:37:25.617137+0000","last_undegraded":"2024-04-25T00:37:25.617137+0000","last_fullsized":"2024-04-25T00:37:25.617137+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:43.653979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891294+0000","last_change":"2024-04-25T00:37:24.659187+0000","last_active":"2024-04-25T00:37:25.891294+0000","last_peered":"2024-04-25T00:37:25.891294+0000","last_clean":"2024-04-25T00:37:25.891294+0000","last_became_active":"2024-04-25T00:37:24.658411+0000","last_became_peered":"2024-04-25T00:37:24.658411+0000","last_unstale":"2024-04-25T00:37:25.891294+0000","last_undegraded":"2024-04-25T00:37:25.891294+0000","last_fullsized":"2024-04-25T00:37:25.891294+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:46.789133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892883+0000","last_change":"2024-04-25T00:37:24.645549+0000","last_active":"2024-04-25T00:37:25.892883+0000","last_peered":"2024-04-25T00:37:25.892883+0000","last_clean":"2024-04-25T00:37:25.892883+0000","last_became_active":"2024-04-25T00:37:24.644769+0000","last_became_peered":"2024-04-25T00:37:24.644769+0000","last_unstale":"2024-04-25T00:37:25.892883+0000","last_undegraded":"2024-04-25T00:37:25.892883+0000","last_fullsized":"2024-04-25T00:37:25.892883+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:07.618067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.618138+0000","last_change":"2024-04-25T00:37:24.664583+0000","last_active":"2024-04-25T00:37:25.618138+0000","last_peered":"2024-04-25T00:37:25.618138+0000","last_clean":"2024-04-25T00:37:25.618138+0000","last_became_active":"2024-04-25T00:37:24.663713+0000","last_became_peered":"2024-04-25T00:37:24.663713+0000","last_unstale":"2024-04-25T00:37:25.618138+0000","last_undegraded":"2024-04-25T00:37:25.618138+0000","last_fullsized":"2024-04-25T00:37:25.618138+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:42.857330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890809+0000","last_change":"2024-04-25T00:37:24.650632+0000","last_active":"2024-04-25T00:37:25.890809+0000","last_peered":"2024-04-25T00:37:25.890809+0000","last_clean":"2024-04-25T00:37:25.890809+0000","last_became_active":"2024-04-25T00:37:24.649835+0000","last_became_peered":"2024-04-25T00:37:24.649835+0000","last_unstale":"2024-04-25T00:37:25.890809+0000","last_undegraded":"2024-04-25T00:37:25.890809+0000","last_fullsized":"2024-04-25T00:37:25.890809+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:24.692684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616247+0000","last_change":"2024-04-25T00:37:24.638702+0000","last_active":"2024-04-25T00:37:25.616247+0000","last_peered":"2024-04-25T00:37:25.616247+0000","last_clean":"2024-04-25T00:37:25.616247+0000","last_became_active":"2024-04-25T00:37:24.637878+0000","last_became_peered":"2024-04-25T00:37:24.637878+0000","last_unstale":"2024-04-25T00:37:25.616247+0000","last_undegraded":"2024-04-25T00:37:25.616247+0000","last_fullsized":"2024-04-25T00:37:25.616247+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:18.566344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891202+0000","last_change":"2024-04-25T00:37:24.647978+0000","last_active":"2024-04-25T00:37:25.891202+0000","last_peered":"2024-04-25T00:37:25.891202+0000","last_clean":"2024-04-25T00:37:25.891202+0000","last_became_active":"2024-04-25T00:37:24.647202+0000","last_became_peered":"2024-04-25T00:37:24.647202+0000","last_unstale":"2024-04-25T00:37:25.891202+0000","last_undegraded":"2024-04-25T00:37:25.891202+0000","last_fullsized":"2024-04-25T00:37:25.891202+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:14.123451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891406+0000","last_change":"2024-04-25T00:37:24.651972+0000","last_active":"2024-04-25T00:37:25.891406+0000","last_peered":"2024-04-25T00:37:25.891406+0000","last_clean":"2024-04-25T00:37:25.891406+0000","last_became_active":"2024-04-25T00:37:24.650939+0000","last_became_peered":"2024-04-25T00:37:24.650939+0000","last_unstale":"2024-04-25T00:37:25.891406+0000","last_undegraded":"2024-04-25T00:37:25.891406+0000","last_fullsized":"2024-04-25T00:37:25.891406+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:35.939082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"102'4","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891440+0000","last_change":"2024-04-25T00:37:23.981233+0000","last_active":"2024-04-25T00:37:25.891440+0000","last_peered":"2024-04-25T00:37:25.891440+0000","last_clean":"2024-04-25T00:37:25.891440+0000","last_became_active":"2024-04-25T00:37:23.980150+0000","last_became_peered":"2024-04-25T00:37:23.980150+0000","last_unstale":"2024-04-25T00:37:25.891440+0000","last_undegraded":"2024-04-25T00:37:25.891440+0000","last_fullsized":"2024-04-25T00:37:25.891440+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:47.226214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891957+0000","last_change":"2024-04-25T00:37:24.634734+0000","last_active":"2024-04-25T00:37:25.891957+0000","last_peered":"2024-04-25T00:37:25.891957+0000","last_clean":"2024-04-25T00:37:25.891957+0000","last_became_active":"2024-04-25T00:37:24.628248+0000","last_became_peered":"2024-04-25T00:37:24.628248+0000","last_unstale":"2024-04-25T00:37:25.891957+0000","last_undegraded":"2024-04-25T00:37:25.891957+0000","last_fullsized":"2024-04-25T00:37:25.891957+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:45.097585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617426+0000","last_change":"2024-04-25T00:37:23.974780+0000","last_active":"2024-04-25T00:37:25.617426+0000","last_peered":"2024-04-25T00:37:25.617426+0000","last_clean":"2024-04-25T00:37:25.617426+0000","last_became_active":"2024-04-25T00:37:23.973844+0000","last_became_peered":"2024-04-25T00:37:23.973844+0000","last_unstale":"2024-04-25T00:37:25.617426+0000","last_undegraded":"2024-04-25T00:37:25.617426+0000","last_fullsized":"2024-04-25T00:37:25.617426+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:50.821959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891301+0000","last_change":"2024-04-25T00:37:24.654395+0000","last_active":"2024-04-25T00:37:25.891301+0000","last_peered":"2024-04-25T00:37:25.891301+0000","last_clean":"2024-04-25T00:37:25.891301+0000","last_became_active":"2024-04-25T00:37:24.652977+0000","last_became_peered":"2024-04-25T00:37:24.652977+0000","last_unstale":"2024-04-25T00:37:25.891301+0000","last_undegraded":"2024-04-25T00:37:25.891301+0000","last_fullsized":"2024-04-25T00:37:25.891301+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:38:20.938337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889564+0000","last_change":"2024-04-25T00:37:23.962945+0000","last_active":"2024-04-25T00:37:25.889564+0000","last_peered":"2024-04-25T00:37:25.889564+0000","last_clean":"2024-04-25T00:37:25.889564+0000","last_became_active":"2024-04-25T00:37:23.962460+0000","last_became_peered":"2024-04-25T00:37:23.962460+0000","last_unstale":"2024-04-25T00:37:25.889564+0000","last_undegraded":"2024-04-25T00:37:25.889564+0000","last_fullsized":"2024-04-25T00:37:25.889564+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:05.179578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617374+0000","last_change":"2024-04-25T00:37:24.656227+0000","last_active":"2024-04-25T00:37:25.617374+0000","last_peered":"2024-04-25T00:37:25.617374+0000","last_clean":"2024-04-25T00:37:25.617374+0000","last_became_active":"2024-04-25T00:37:24.655453+0000","last_became_peered":"2024-04-25T00:37:24.655453+0000","last_unstale":"2024-04-25T00:37:25.617374+0000","last_undegraded":"2024-04-25T00:37:25.617374+0000","last_fullsized":"2024-04-25T00:37:25.617374+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:38.243062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890255+0000","last_change":"2024-04-25T00:37:23.967844+0000","last_active":"2024-04-25T00:37:25.890255+0000","last_peered":"2024-04-25T00:37:25.890255+0000","last_clean":"2024-04-25T00:37:25.890255+0000","last_became_active":"2024-04-25T00:37:23.967545+0000","last_became_peered":"2024-04-25T00:37:23.967545+0000","last_unstale":"2024-04-25T00:37:25.890255+0000","last_undegraded":"2024-04-25T00:37:25.890255+0000","last_fullsized":"2024-04-25T00:37:25.890255+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:03.459207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616154+0000","last_change":"2024-04-25T00:37:24.665789+0000","last_active":"2024-04-25T00:37:25.616154+0000","last_peered":"2024-04-25T00:37:25.616154+0000","last_clean":"2024-04-25T00:37:25.616154+0000","last_became_active":"2024-04-25T00:37:24.665110+0000","last_became_peered":"2024-04-25T00:37:24.665110+0000","last_unstale":"2024-04-25T00:37:25.616154+0000","last_undegraded":"2024-04-25T00:37:25.616154+0000","last_fullsized":"2024-04-25T00:37:25.616154+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:24.334663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889798+0000","last_change":"2024-04-25T00:37:23.971620+0000","last_active":"2024-04-25T00:37:25.889798+0000","last_peered":"2024-04-25T00:37:25.889798+0000","last_clean":"2024-04-25T00:37:25.889798+0000","last_became_active":"2024-04-25T00:37:23.970901+0000","last_became_peered":"2024-04-25T00:37:23.970901+0000","last_unstale":"2024-04-25T00:37:25.889798+0000","last_undegraded":"2024-04-25T00:37:25.889798+0000","last_fullsized":"2024-04-25T00:37:25.889798+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:54.449682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.618554+0000","last_change":"2024-04-25T00:37:24.654380+0000","last_active":"2024-04-25T00:37:25.618554+0000","last_peered":"2024-04-25T00:37:25.618554+0000","last_clean":"2024-04-25T00:37:25.618554+0000","last_became_active":"2024-04-25T00:37:24.653655+0000","last_became_peered":"2024-04-25T00:37:24.653655+0000","last_unstale":"2024-04-25T00:37:25.618554+0000","last_undegraded":"2024-04-25T00:37:25.618554+0000","last_fullsized":"2024-04-25T00:37:25.618554+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.195629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892229+0000","last_change":"2024-04-25T00:37:23.974470+0000","last_active":"2024-04-25T00:37:25.892229+0000","last_peered":"2024-04-25T00:37:25.892229+0000","last_clean":"2024-04-25T00:37:25.892229+0000","last_became_active":"2024-04-25T00:37:23.974141+0000","last_became_peered":"2024-04-25T00:37:23.974141+0000","last_unstale":"2024-04-25T00:37:25.892229+0000","last_undegraded":"2024-04-25T00:37:25.892229+0000","last_fullsized":"2024-04-25T00:37:25.892229+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:08.824434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616040+0000","last_change":"2024-04-25T00:37:24.659767+0000","last_active":"2024-04-25T00:37:25.616040+0000","last_peered":"2024-04-25T00:37:25.616040+0000","last_clean":"2024-04-25T00:37:25.616040+0000","last_became_active":"2024-04-25T00:37:24.659084+0000","last_became_peered":"2024-04-25T00:37:24.659084+0000","last_unstale":"2024-04-25T00:37:25.616040+0000","last_undegraded":"2024-04-25T00:37:25.616040+0000","last_fullsized":"2024-04-25T00:37:25.616040+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:45.475299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617781+0000","last_change":"2024-04-25T00:37:23.960317+0000","last_active":"2024-04-25T00:37:25.617781+0000","last_peered":"2024-04-25T00:37:25.617781+0000","last_clean":"2024-04-25T00:37:25.617781+0000","last_became_active":"2024-04-25T00:37:23.959614+0000","last_became_peered":"2024-04-25T00:37:23.959614+0000","last_unstale":"2024-04-25T00:37:25.617781+0000","last_undegraded":"2024-04-25T00:37:25.617781+0000","last_fullsized":"2024-04-25T00:37:25.617781+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:34.332398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889853+0000","last_change":"2024-04-25T00:37:24.651641+0000","last_active":"2024-04-25T00:37:25.889853+0000","last_peered":"2024-04-25T00:37:25.889853+0000","last_clean":"2024-04-25T00:37:25.889853+0000","last_became_active":"2024-04-25T00:37:24.649207+0000","last_became_peered":"2024-04-25T00:37:24.649207+0000","last_unstale":"2024-04-25T00:37:25.889853+0000","last_undegraded":"2024-04-25T00:37:25.889853+0000","last_fullsized":"2024-04-25T00:37:25.889853+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:45.012553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616204+0000","last_change":"2024-04-25T00:37:23.971464+0000","last_active":"2024-04-25T00:37:25.616204+0000","last_peered":"2024-04-25T00:37:25.616204+0000","last_clean":"2024-04-25T00:37:25.616204+0000","last_became_active":"2024-04-25T00:37:23.970908+0000","last_became_peered":"2024-04-25T00:37:23.970908+0000","last_unstale":"2024-04-25T00:37:25.616204+0000","last_undegraded":"2024-04-25T00:37:25.616204+0000","last_fullsized":"2024-04-25T00:37:25.616204+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:56.224951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890699+0000","last_change":"2024-04-25T00:37:24.651933+0000","last_active":"2024-04-25T00:37:25.890699+0000","last_peered":"2024-04-25T00:37:25.890699+0000","last_clean":"2024-04-25T00:37:25.890699+0000","last_became_active":"2024-04-25T00:37:24.651128+0000","last_became_peered":"2024-04-25T00:37:24.651128+0000","last_unstale":"2024-04-25T00:37:25.890699+0000","last_undegraded":"2024-04-25T00:37:25.890699+0000","last_fullsized":"2024-04-25T00:37:25.890699+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:07.838253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889662+0000","last_change":"2024-04-25T00:37:23.962738+0000","last_active":"2024-04-25T00:37:25.889662+0000","last_peered":"2024-04-25T00:37:25.889662+0000","last_clean":"2024-04-25T00:37:25.889662+0000","last_became_active":"2024-04-25T00:37:23.962330+0000","last_became_peered":"2024-04-25T00:37:23.962330+0000","last_unstale":"2024-04-25T00:37:25.889662+0000","last_undegraded":"2024-04-25T00:37:25.889662+0000","last_fullsized":"2024-04-25T00:37:25.889662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:50.647322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891917+0000","last_change":"2024-04-25T00:37:24.639458+0000","last_active":"2024-04-25T00:37:25.891917+0000","last_peered":"2024-04-25T00:37:25.891917+0000","last_clean":"2024-04-25T00:37:25.891917+0000","last_became_active":"2024-04-25T00:37:24.637555+0000","last_became_peered":"2024-04-25T00:37:24.637555+0000","last_unstale":"2024-04-25T00:37:25.891917+0000","last_undegraded":"2024-04-25T00:37:25.891917+0000","last_fullsized":"2024-04-25T00:37:25.891917+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:30.099048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891682+0000","last_change":"2024-04-25T00:37:23.973733+0000","last_active":"2024-04-25T00:37:25.891682+0000","last_peered":"2024-04-25T00:37:25.891682+0000","last_clean":"2024-04-25T00:37:25.891682+0000","last_became_active":"2024-04-25T00:37:23.973362+0000","last_became_peered":"2024-04-25T00:37:23.973362+0000","last_unstale":"2024-04-25T00:37:25.891682+0000","last_undegraded":"2024-04-25T00:37:25.891682+0000","last_fullsized":"2024-04-25T00:37:25.891682+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:51.405634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.893367+0000","last_change":"2024-04-25T00:37:24.649357+0000","last_active":"2024-04-25T00:37:25.893367+0000","last_peered":"2024-04-25T00:37:25.893367+0000","last_clean":"2024-04-25T00:37:25.893367+0000","last_became_active":"2024-04-25T00:37:24.647321+0000","last_became_peered":"2024-04-25T00:37:24.647321+0000","last_unstale":"2024-04-25T00:37:25.893367+0000","last_undegraded":"2024-04-25T00:37:25.893367+0000","last_fullsized":"2024-04-25T00:37:25.893367+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:26.159139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"102'3","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616207+0000","last_change":"2024-04-25T00:37:23.966485+0000","last_active":"2024-04-25T00:37:25.616207+0000","last_peered":"2024-04-25T00:37:25.616207+0000","last_clean":"2024-04-25T00:37:25.616207+0000","last_became_active":"2024-04-25T00:37:23.966125+0000","last_became_peered":"2024-04-25T00:37:23.966125+0000","last_unstale":"2024-04-25T00:37:25.616207+0000","last_undegraded":"2024-04-25T00:37:25.616207+0000","last_fullsized":"2024-04-25T00:37:25.616207+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:44.093942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890417+0000","last_change":"2024-04-25T00:37:24.659214+0000","last_active":"2024-04-25T00:37:25.890417+0000","last_peered":"2024-04-25T00:37:25.890417+0000","last_clean":"2024-04-25T00:37:25.890417+0000","last_became_active":"2024-04-25T00:37:24.658485+0000","last_became_peered":"2024-04-25T00:37:24.658485+0000","last_unstale":"2024-04-25T00:37:25.890417+0000","last_undegraded":"2024-04-25T00:37:25.890417+0000","last_fullsized":"2024-04-25T00:37:25.890417+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:09.576753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"102'6","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617542+0000","last_change":"2024-04-25T00:37:23.971682+0000","last_active":"2024-04-25T00:37:25.617542+0000","last_peered":"2024-04-25T00:37:25.617542+0000","last_clean":"2024-04-25T00:37:25.617542+0000","last_became_active":"2024-04-25T00:37:23.970797+0000","last_became_peered":"2024-04-25T00:37:23.970797+0000","last_unstale":"2024-04-25T00:37:25.617542+0000","last_undegraded":"2024-04-25T00:37:25.617542+0000","last_fullsized":"2024-04-25T00:37:25.617542+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:27.326430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.893051+0000","last_change":"2024-04-25T00:37:24.649497+0000","last_active":"2024-04-25T00:37:25.893051+0000","last_peered":"2024-04-25T00:37:25.893051+0000","last_clean":"2024-04-25T00:37:25.893051+0000","last_became_active":"2024-04-25T00:37:24.648738+0000","last_became_peered":"2024-04-25T00:37:24.648738+0000","last_unstale":"2024-04-25T00:37:25.893051+0000","last_undegraded":"2024-04-25T00:37:25.893051+0000","last_fullsized":"2024-04-25T00:37:25.893051+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:39.326233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891628+0000","last_change":"2024-04-25T00:37:23.980023+0000","last_active":"2024-04-25T00:37:25.891628+0000","last_peered":"2024-04-25T00:37:25.891628+0000","last_clean":"2024-04-25T00:37:25.891628+0000","last_became_active":"2024-04-25T00:37:23.979702+0000","last_became_peered":"2024-04-25T00:37:23.979702+0000","last_unstale":"2024-04-25T00:37:25.891628+0000","last_undegraded":"2024-04-25T00:37:25.891628+0000","last_fullsized":"2024-04-25T00:37:25.891628+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:36.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.893084+0000","last_change":"2024-04-25T00:37:24.663106+0000","last_active":"2024-04-25T00:37:25.893084+0000","last_peered":"2024-04-25T00:37:25.893084+0000","last_clean":"2024-04-25T00:37:25.893084+0000","last_became_active":"2024-04-25T00:37:24.662275+0000","last_became_peered":"2024-04-25T00:37:24.662275+0000","last_unstale":"2024-04-25T00:37:25.893084+0000","last_undegraded":"2024-04-25T00:37:25.893084+0000","last_fullsized":"2024-04-25T00:37:25.893084+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:49.644731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891008+0000","last_change":"2024-04-25T00:37:23.974737+0000","last_active":"2024-04-25T00:37:25.891008+0000","last_peered":"2024-04-25T00:37:25.891008+0000","last_clean":"2024-04-25T00:37:25.891008+0000","last_became_active":"2024-04-25T00:37:23.974006+0000","last_became_peered":"2024-04-25T00:37:23.974006+0000","last_unstale":"2024-04-25T00:37:25.891008+0000","last_undegraded":"2024-04-25T00:37:25.891008+0000","last_fullsized":"2024-04-25T00:37:25.891008+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:06.834610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891850+0000","last_change":"2024-04-25T00:37:24.648169+0000","last_active":"2024-04-25T00:37:25.891850+0000","last_peered":"2024-04-25T00:37:25.891850+0000","last_clean":"2024-04-25T00:37:25.891850+0000","last_became_active":"2024-04-25T00:37:24.646922+0000","last_became_peered":"2024-04-25T00:37:24.646922+0000","last_unstale":"2024-04-25T00:37:25.891850+0000","last_undegraded":"2024-04-25T00:37:25.891850+0000","last_fullsized":"2024-04-25T00:37:25.891850+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:49.957775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"102'8","reported_seq":22,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889187+0000","last_change":"2024-04-25T00:37:23.972616+0000","last_active":"2024-04-25T00:37:25.889187+0000","last_peered":"2024-04-25T00:37:25.889187+0000","last_clean":"2024-04-25T00:37:25.889187+0000","last_became_active":"2024-04-25T00:37:23.972302+0000","last_became_peered":"2024-04-25T00:37:23.972302+0000","last_unstale":"2024-04-25T00:37:25.889187+0000","last_undegraded":"2024-04-25T00:37:25.889187+0000","last_fullsized":"2024-04-25T00:37:25.889187+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:26.276154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890017+0000","last_change":"2024-04-25T00:37:24.649798+0000","last_active":"2024-04-25T00:37:25.890017+0000","last_peered":"2024-04-25T00:37:25.890017+0000","last_clean":"2024-04-25T00:37:25.890017+0000","last_became_active":"2024-04-25T00:37:24.648984+0000","last_became_peered":"2024-04-25T00:37:24.648984+0000","last_unstale":"2024-04-25T00:37:25.890017+0000","last_undegraded":"2024-04-25T00:37:25.890017+0000","last_fullsized":"2024-04-25T00:37:25.890017+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:37:41.353172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617304+0000","last_change":"2024-04-25T00:37:23.973856+0000","last_active":"2024-04-25T00:37:25.617304+0000","last_peered":"2024-04-25T00:37:25.617304+0000","last_clean":"2024-04-25T00:37:25.617304+0000","last_became_active":"2024-04-25T00:37:23.973576+0000","last_became_peered":"2024-04-25T00:37:23.973576+0000","last_unstale":"2024-04-25T00:37:25.617304+0000","last_undegraded":"2024-04-25T00:37:25.617304+0000","last_fullsized":"2024-04-25T00:37:25.617304+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:56.098633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891635+0000","last_change":"2024-04-25T00:37:24.659185+0000","last_active":"2024-04-25T00:37:25.891635+0000","last_peered":"2024-04-25T00:37:25.891635+0000","last_clean":"2024-04-25T00:37:25.891635+0000","last_became_active":"2024-04-25T00:37:24.658389+0000","last_became_peered":"2024-04-25T00:37:24.658389+0000","last_unstale":"2024-04-25T00:37:25.891635+0000","last_undegraded":"2024-04-25T00:37:25.891635+0000","last_fullsized":"2024-04-25T00:37:25.891635+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:52.832892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889893+0000","last_change":"2024-04-25T00:37:23.963264+0000","last_active":"2024-04-25T00:37:25.889893+0000","last_peered":"2024-04-25T00:37:25.889893+0000","last_clean":"2024-04-25T00:37:25.889893+0000","last_became_active":"2024-04-25T00:37:23.962746+0000","last_became_peered":"2024-04-25T00:37:23.962746+0000","last_unstale":"2024-04-25T00:37:25.889893+0000","last_undegraded":"2024-04-25T00:37:25.889893+0000","last_fullsized":"2024-04-25T00:37:25.889893+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:36.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616475+0000","last_change":"2024-04-25T00:37:24.632170+0000","last_active":"2024-04-25T00:37:25.616475+0000","last_peered":"2024-04-25T00:37:25.616475+0000","last_clean":"2024-04-25T00:37:25.616475+0000","last_became_active":"2024-04-25T00:37:24.631515+0000","last_became_peered":"2024-04-25T00:37:24.631515+0000","last_unstale":"2024-04-25T00:37:25.616475+0000","last_undegraded":"2024-04-25T00:37:25.616475+0000","last_fullsized":"2024-04-25T00:37:25.616475+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:49.957775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891186+0000","last_change":"2024-04-25T00:37:23.969113+0000","last_active":"2024-04-25T00:37:25.891186+0000","last_peered":"2024-04-25T00:37:25.891186+0000","last_clean":"2024-04-25T00:37:25.891186+0000","last_became_active":"2024-04-25T00:37:23.968373+0000","last_became_peered":"2024-04-25T00:37:23.968373+0000","last_unstale":"2024-04-25T00:37:25.891186+0000","last_undegraded":"2024-04-25T00:37:25.891186+0000","last_fullsized":"2024-04-25T00:37:25.891186+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:23.689041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617733+0000","last_change":"2024-04-25T00:37:24.643621+0000","last_active":"2024-04-25T00:37:25.617733+0000","last_peered":"2024-04-25T00:37:25.617733+0000","last_clean":"2024-04-25T00:37:25.617733+0000","last_became_active":"2024-04-25T00:37:24.641963+0000","last_became_peered":"2024-04-25T00:37:24.641963+0000","last_unstale":"2024-04-25T00:37:25.617733+0000","last_undegraded":"2024-04-25T00:37:25.617733+0000","last_fullsized":"2024-04-25T00:37:25.617733+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:49.644731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889848+0000","last_change":"2024-04-25T00:37:23.975090+0000","last_active":"2024-04-25T00:37:25.889848+0000","last_peered":"2024-04-25T00:37:25.889848+0000","last_clean":"2024-04-25T00:37:25.889848+0000","last_became_active":"2024-04-25T00:37:23.973684+0000","last_became_peered":"2024-04-25T00:37:23.973684+0000","last_unstale":"2024-04-25T00:37:25.889848+0000","last_undegraded":"2024-04-25T00:37:25.889848+0000","last_fullsized":"2024-04-25T00:37:25.889848+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:35.743952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617677+0000","last_change":"2024-04-25T00:37:24.641044+0000","last_active":"2024-04-25T00:37:25.617677+0000","last_peered":"2024-04-25T00:37:25.617677+0000","last_clean":"2024-04-25T00:37:25.617677+0000","last_became_active":"2024-04-25T00:37:24.635401+0000","last_became_peered":"2024-04-25T00:37:24.635401+0000","last_unstale":"2024-04-25T00:37:25.617677+0000","last_undegraded":"2024-04-25T00:37:25.617677+0000","last_fullsized":"2024-04-25T00:37:25.617677+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:39.326233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891670+0000","last_change":"2024-04-25T00:37:23.962101+0000","last_active":"2024-04-25T00:37:25.891670+0000","last_peered":"2024-04-25T00:37:25.891670+0000","last_clean":"2024-04-25T00:37:25.891670+0000","last_became_active":"2024-04-25T00:37:23.961767+0000","last_became_peered":"2024-04-25T00:37:23.961767+0000","last_unstale":"2024-04-25T00:37:25.891670+0000","last_undegraded":"2024-04-25T00:37:25.891670+0000","last_fullsized":"2024-04-25T00:37:25.891670+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:05.179578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616518+0000","last_change":"2024-04-25T00:37:24.624859+0000","last_active":"2024-04-25T00:37:25.616518+0000","last_peered":"2024-04-25T00:37:25.616518+0000","last_clean":"2024-04-25T00:37:25.616518+0000","last_became_active":"2024-04-25T00:37:24.624061+0000","last_became_peered":"2024-04-25T00:37:24.624061+0000","last_unstale":"2024-04-25T00:37:25.616518+0000","last_undegraded":"2024-04-25T00:37:25.616518+0000","last_fullsized":"2024-04-25T00:37:25.616518+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:10.228348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889401+0000","last_change":"2024-04-25T00:37:23.962517+0000","last_active":"2024-04-25T00:37:25.889401+0000","last_peered":"2024-04-25T00:37:25.889401+0000","last_clean":"2024-04-25T00:37:25.889401+0000","last_became_active":"2024-04-25T00:37:23.962068+0000","last_became_peered":"2024-04-25T00:37:23.962068+0000","last_unstale":"2024-04-25T00:37:25.889401+0000","last_undegraded":"2024-04-25T00:37:25.889401+0000","last_fullsized":"2024-04-25T00:37:25.889401+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:45.756449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616212+0000","last_change":"2024-04-25T00:37:24.656268+0000","last_active":"2024-04-25T00:37:25.616212+0000","last_peered":"2024-04-25T00:37:25.616212+0000","last_clean":"2024-04-25T00:37:25.616212+0000","last_became_active":"2024-04-25T00:37:24.655483+0000","last_became_peered":"2024-04-25T00:37:24.655483+0000","last_unstale":"2024-04-25T00:37:25.616212+0000","last_undegraded":"2024-04-25T00:37:25.616212+0000","last_fullsized":"2024-04-25T00:37:25.616212+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:57.228594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891056+0000","last_change":"2024-04-25T00:37:23.962073+0000","last_active":"2024-04-25T00:37:25.891056+0000","last_peered":"2024-04-25T00:37:25.891056+0000","last_clean":"2024-04-25T00:37:25.891056+0000","last_became_active":"2024-04-25T00:37:23.961617+0000","last_became_peered":"2024-04-25T00:37:23.961617+0000","last_unstale":"2024-04-25T00:37:25.891056+0000","last_undegraded":"2024-04-25T00:37:25.891056+0000","last_fullsized":"2024-04-25T00:37:25.891056+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:50.647322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892296+0000","last_change":"2024-04-25T00:37:24.657468+0000","last_active":"2024-04-25T00:37:25.892296+0000","last_peered":"2024-04-25T00:37:25.892296+0000","last_clean":"2024-04-25T00:37:25.892296+0000","last_became_active":"2024-04-25T00:37:24.656654+0000","last_became_peered":"2024-04-25T00:37:24.656654+0000","last_unstale":"2024-04-25T00:37:25.892296+0000","last_undegraded":"2024-04-25T00:37:25.892296+0000","last_fullsized":"2024-04-25T00:37:25.892296+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:21.524079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616345+0000","last_change":"2024-04-25T00:37:23.974633+0000","last_active":"2024-04-25T00:37:25.616345+0000","last_peered":"2024-04-25T00:37:25.616345+0000","last_clean":"2024-04-25T00:37:25.616345+0000","last_became_active":"2024-04-25T00:37:23.974067+0000","last_became_peered":"2024-04-25T00:37:23.974067+0000","last_unstale":"2024-04-25T00:37:25.616345+0000","last_undegraded":"2024-04-25T00:37:25.616345+0000","last_fullsized":"2024-04-25T00:37:25.616345+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:08.911836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891425+0000","last_change":"2024-04-25T00:37:24.639298+0000","last_active":"2024-04-25T00:37:25.891425+0000","last_peered":"2024-04-25T00:37:25.891425+0000","last_clean":"2024-04-25T00:37:25.891425+0000","last_became_active":"2024-04-25T00:37:24.637729+0000","last_became_peered":"2024-04-25T00:37:24.637729+0000","last_unstale":"2024-04-25T00:37:25.891425+0000","last_undegraded":"2024-04-25T00:37:25.891425+0000","last_fullsized":"2024-04-25T00:37:25.891425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:38.140994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892247+0000","last_change":"2024-04-25T00:37:23.973640+0000","last_active":"2024-04-25T00:37:25.892247+0000","last_peered":"2024-04-25T00:37:25.892247+0000","last_clean":"2024-04-25T00:37:25.892247+0000","last_became_active":"2024-04-25T00:37:23.973210+0000","last_became_peered":"2024-04-25T00:37:23.973210+0000","last_unstale":"2024-04-25T00:37:25.892247+0000","last_undegraded":"2024-04-25T00:37:25.892247+0000","last_fullsized":"2024-04-25T00:37:25.892247+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:56.021774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617275+0000","last_change":"2024-04-25T00:37:24.643054+0000","last_active":"2024-04-25T00:37:25.617275+0000","last_peered":"2024-04-25T00:37:25.617275+0000","last_clean":"2024-04-25T00:37:25.617275+0000","last_became_active":"2024-04-25T00:37:24.642305+0000","last_became_peered":"2024-04-25T00:37:24.642305+0000","last_unstale":"2024-04-25T00:37:25.617275+0000","last_undegraded":"2024-04-25T00:37:25.617275+0000","last_fullsized":"2024-04-25T00:37:25.617275+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:28.830671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"103'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891625+0000","last_change":"2024-04-25T00:37:23.976356+0000","last_active":"2024-04-25T00:37:25.891625+0000","last_peered":"2024-04-25T00:37:25.891625+0000","last_clean":"2024-04-25T00:37:25.891625+0000","last_became_active":"2024-04-25T00:37:23.975148+0000","last_became_peered":"2024-04-25T00:37:23.975148+0000","last_unstale":"2024-04-25T00:37:25.891625+0000","last_undegraded":"2024-04-25T00:37:25.891625+0000","last_fullsized":"2024-04-25T00:37:25.891625+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:50.821959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617474+0000","last_change":"2024-04-25T00:37:24.642433+0000","last_active":"2024-04-25T00:37:25.617474+0000","last_peered":"2024-04-25T00:37:25.617474+0000","last_clean":"2024-04-25T00:37:25.617474+0000","last_became_active":"2024-04-25T00:37:24.641630+0000","last_became_peered":"2024-04-25T00:37:24.641630+0000","last_unstale":"2024-04-25T00:37:25.617474+0000","last_undegraded":"2024-04-25T00:37:25.617474+0000","last_fullsized":"2024-04-25T00:37:25.617474+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:10.590475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616674+0000","last_change":"2024-04-25T00:37:23.975126+0000","last_active":"2024-04-25T00:37:25.616674+0000","last_peered":"2024-04-25T00:37:25.616674+0000","last_clean":"2024-04-25T00:37:25.616674+0000","last_became_active":"2024-04-25T00:37:23.974571+0000","last_became_peered":"2024-04-25T00:37:23.974571+0000","last_unstale":"2024-04-25T00:37:25.616674+0000","last_undegraded":"2024-04-25T00:37:25.616674+0000","last_fullsized":"2024-04-25T00:37:25.616674+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:51.950167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891114+0000","last_change":"2024-04-25T00:37:24.637810+0000","last_active":"2024-04-25T00:37:25.891114+0000","last_peered":"2024-04-25T00:37:25.891114+0000","last_clean":"2024-04-25T00:37:25.891114+0000","last_became_active":"2024-04-25T00:37:24.636719+0000","last_became_peered":"2024-04-25T00:37:24.636719+0000","last_unstale":"2024-04-25T00:37:25.891114+0000","last_undegraded":"2024-04-25T00:37:25.891114+0000","last_fullsized":"2024-04-25T00:37:25.891114+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.195629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.893433+0000","last_change":"2024-04-25T00:37:23.962739+0000","last_active":"2024-04-25T00:37:25.893433+0000","last_peered":"2024-04-25T00:37:25.893433+0000","last_clean":"2024-04-25T00:37:25.893433+0000","last_became_active":"2024-04-25T00:37:23.962425+0000","last_became_peered":"2024-04-25T00:37:23.962425+0000","last_unstale":"2024-04-25T00:37:25.893433+0000","last_undegraded":"2024-04-25T00:37:25.893433+0000","last_fullsized":"2024-04-25T00:37:25.893433+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:02.162592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617946+0000","last_change":"2024-04-25T00:37:24.640862+0000","last_active":"2024-04-25T00:37:25.617946+0000","last_peered":"2024-04-25T00:37:25.617946+0000","last_clean":"2024-04-25T00:37:25.617946+0000","last_became_active":"2024-04-25T00:37:24.640060+0000","last_became_peered":"2024-04-25T00:37:24.640060+0000","last_unstale":"2024-04-25T00:37:25.617946+0000","last_undegraded":"2024-04-25T00:37:25.617946+0000","last_fullsized":"2024-04-25T00:37:25.617946+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:09.828077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889925+0000","last_change":"2024-04-25T00:37:23.976960+0000","last_active":"2024-04-25T00:37:25.889925+0000","last_peered":"2024-04-25T00:37:25.889925+0000","last_clean":"2024-04-25T00:37:25.889925+0000","last_became_active":"2024-04-25T00:37:23.971621+0000","last_became_peered":"2024-04-25T00:37:23.971621+0000","last_unstale":"2024-04-25T00:37:25.889925+0000","last_undegraded":"2024-04-25T00:37:25.889925+0000","last_fullsized":"2024-04-25T00:37:25.889925+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:51.258332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.618498+0000","last_change":"2024-04-25T00:37:24.653749+0000","last_active":"2024-04-25T00:37:25.618498+0000","last_peered":"2024-04-25T00:37:25.618498+0000","last_clean":"2024-04-25T00:37:25.618498+0000","last_became_active":"2024-04-25T00:37:24.653059+0000","last_became_peered":"2024-04-25T00:37:24.653059+0000","last_unstale":"2024-04-25T00:37:25.618498+0000","last_undegraded":"2024-04-25T00:37:25.618498+0000","last_fullsized":"2024-04-25T00:37:25.618498+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:28.330073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617928+0000","last_change":"2024-04-25T00:37:23.963951+0000","last_active":"2024-04-25T00:37:25.617928+0000","last_peered":"2024-04-25T00:37:25.617928+0000","last_clean":"2024-04-25T00:37:25.617928+0000","last_became_active":"2024-04-25T00:37:23.963276+0000","last_became_peered":"2024-04-25T00:37:23.963276+0000","last_unstale":"2024-04-25T00:37:25.617928+0000","last_undegraded":"2024-04-25T00:37:25.617928+0000","last_fullsized":"2024-04-25T00:37:25.617928+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:41.471801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616544+0000","last_change":"2024-04-25T00:37:24.625317+0000","last_active":"2024-04-25T00:37:25.616544+0000","last_peered":"2024-04-25T00:37:25.616544+0000","last_clean":"2024-04-25T00:37:25.616544+0000","last_became_active":"2024-04-25T00:37:24.624551+0000","last_became_peered":"2024-04-25T00:37:24.624551+0000","last_unstale":"2024-04-25T00:37:25.616544+0000","last_undegraded":"2024-04-25T00:37:25.616544+0000","last_fullsized":"2024-04-25T00:37:25.616544+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:09.915478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889262+0000","last_change":"2024-04-25T00:37:23.968009+0000","last_active":"2024-04-25T00:37:25.889262+0000","last_peered":"2024-04-25T00:37:25.889262+0000","last_clean":"2024-04-25T00:37:25.889262+0000","last_became_active":"2024-04-25T00:37:23.967586+0000","last_became_peered":"2024-04-25T00:37:23.967586+0000","last_unstale":"2024-04-25T00:37:25.889262+0000","last_undegraded":"2024-04-25T00:37:25.889262+0000","last_fullsized":"2024-04-25T00:37:25.889262+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:47.615185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616472+0000","last_change":"2024-04-25T00:37:24.658718+0000","last_active":"2024-04-25T00:37:25.616472+0000","last_peered":"2024-04-25T00:37:25.616472+0000","last_clean":"2024-04-25T00:37:25.616472+0000","last_became_active":"2024-04-25T00:37:24.657945+0000","last_became_peered":"2024-04-25T00:37:24.657945+0000","last_unstale":"2024-04-25T00:37:25.616472+0000","last_undegraded":"2024-04-25T00:37:25.616472+0000","last_fullsized":"2024-04-25T00:37:25.616472+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:07.998140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616393+0000","last_change":"2024-04-25T00:37:23.974640+0000","last_active":"2024-04-25T00:37:25.616393+0000","last_peered":"2024-04-25T00:37:25.616393+0000","last_clean":"2024-04-25T00:37:25.616393+0000","last_became_active":"2024-04-25T00:37:23.974224+0000","last_became_peered":"2024-04-25T00:37:23.974224+0000","last_unstale":"2024-04-25T00:37:25.616393+0000","last_undegraded":"2024-04-25T00:37:25.616393+0000","last_fullsized":"2024-04-25T00:37:25.616393+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:20.520436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892767+0000","last_change":"2024-04-25T00:37:24.665407+0000","last_active":"2024-04-25T00:37:25.892767+0000","last_peered":"2024-04-25T00:37:25.892767+0000","last_clean":"2024-04-25T00:37:25.892767+0000","last_became_active":"2024-04-25T00:37:24.664257+0000","last_became_peered":"2024-04-25T00:37:24.664257+0000","last_unstale":"2024-04-25T00:37:25.892767+0000","last_undegraded":"2024-04-25T00:37:25.892767+0000","last_fullsized":"2024-04-25T00:37:25.892767+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:38.243062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890107+0000","last_change":"2024-04-25T00:37:23.974436+0000","last_active":"2024-04-25T00:37:25.890107+0000","last_peered":"2024-04-25T00:37:25.890107+0000","last_clean":"2024-04-25T00:37:25.890107+0000","last_became_active":"2024-04-25T00:37:23.973775+0000","last_became_peered":"2024-04-25T00:37:23.973775+0000","last_unstale":"2024-04-25T00:37:25.890107+0000","last_undegraded":"2024-04-25T00:37:25.890107+0000","last_fullsized":"2024-04-25T00:37:25.890107+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:09.024737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":368,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616842+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:37:25.616842+0000","last_peered":"2024-04-25T00:37:25.616842+0000","last_clean":"2024-04-25T00:37:25.616842+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:37:25.616842+0000","last_undegraded":"2024-04-25T00:37:25.616842+0000","last_fullsized":"2024-04-25T00:37:25.616842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37192,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1884160,"data_stored":1153585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1286144,"data_stored":562671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":563205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1282048,"data_stored":562601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36608,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335232,"statfs":{"total":96636764160,"available":96599277568,"internally_reserved":0,"allocated":1298432,"data_stored":564283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36592,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335248,"statfs":{"total":96636764160,"available":96599293952,"internally_reserved":0,"allocated":1282048,"data_stored":562601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36600,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335240,"statfs":{"total":96636764160,"available":96599285760,"internally_reserved":0,"allocated":1290240,"data_stored":562725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":1154693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:37:30.585 INFO:tasks.ceph.mds.a.smithi029.stderr:2024-04-25T00:37:30.583+0000 7fb852d69640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:37:33.483 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:37:33.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.615+0000 7fd7cabba640 1 Processor -- start 2024-04-25T00:37:33.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.615+0000 7fd7cabba640 1 -- start start 2024-04-25T00:37:33.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7cabba640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7c415ad20 0x7fd7c40cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:33.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7cabba640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7c415bc30 con 0x7fd7c4071d80 2024-04-25T00:37:33.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7cabba640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7c415bf10 con 0x7fd7c40d0390 2024-04-25T00:37:33.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7cabba640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd7c40cb6a0 con 0x7fd7c415ad20 2024-04-25T00:37:33.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c9157640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd7c4071d80 0x7fd7c40721a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58052/0 (socket says 172.21.15.29:58052) 2024-04-25T00:37:33.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c9157640 1 -- 172.21.15.29:0/2284283114 learned_addr learned my addr 172.21.15.29:0/2284283114 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:33.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c8956640 1 --2- 172.21.15.29:0/2284283114 >> v2:172.21.15.157:3300/0 conn(0x7fd7c415ad20 0x7fd7c40cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:33.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3334827522 0 0) 0x7fd7c415bc30 con 0x7fd7c4071d80 2024-04-25T00:37:33.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7b4003530 con 0x7fd7c4071d80 2024-04-25T00:37:33.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 13261062 0 0) 0x7fd7b4003530 con 0x7fd7c4071d80 2024-04-25T00:37:33.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 >> v2:172.21.15.157:3300/0 conn(0x7fd7c415ad20 msgr2=0x7fd7c40cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:33.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 --2- 172.21.15.29:0/2284283114 >> v2:172.21.15.157:3300/0 conn(0x7fd7c415ad20 0x7fd7c40cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 >> v1:172.21.15.157:6789/0 conn(0x7fd7c40d0390 legacy=0x7fd7c415b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:33.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c415c1f0 con 0x7fd7c4071d80 2024-04-25T00:37:33.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd7bc001a00 con 0x7fd7c4071d80 2024-04-25T00:37:33.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 >> v1:172.21.15.29:6789/0 conn(0x7fd7c4071d80 legacy=0x7fd7c40721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:33.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 --2- 172.21.15.29:0/2284283114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7b40039c0 0x7fd7b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:33.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 --2- 172.21.15.29:0/2284283114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7b4004810 0x7fd7b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:33.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 --2- 172.21.15.29:0/2284283114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b40051d0 0x7fd7b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:33.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c40cb6a0 con 0x7fd7b40039c0 2024-04-25T00:37:33.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b400c0a0 con 0x7fd7b40051d0 2024-04-25T00:37:33.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b400c2a0 con 0x7fd7b4004810 2024-04-25T00:37:33.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c8956640 1 --2- 172.21.15.29:0/2284283114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7b40039c0 0x7fd7b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:33.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c3fff640 1 --2- 172.21.15.29:0/2284283114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7b4004810 0x7fd7b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:33.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c8956640 1 -- 172.21.15.29:0/2284283114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7b4004810 msgr2=0x7fd7b4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:33.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c9157640 1 --2- 172.21.15.29:0/2284283114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b40051d0 0x7fd7b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:33.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c8956640 1 --2- 172.21.15.29:0/2284283114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7b4004810 0x7fd7b4004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c8956640 1 -- 172.21.15.29:0/2284283114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b40051d0 msgr2=0x7fd7b400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:33.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c8956640 1 --2- 172.21.15.29:0/2284283114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b40051d0 0x7fd7b400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c8956640 1 -- 172.21.15.29:0/2284283114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b4003530 con 0x7fd7b40039c0 2024-04-25T00:37:33.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c9157640 1 --2- 172.21.15.29:0/2284283114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b40051d0 0x7fd7b400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:33.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.619+0000 7fd7c9157640 1 -- 172.21.15.29:0/2284283114 reap_dead start 2024-04-25T00:37:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c8956640 1 --2- 172.21.15.29:0/2284283114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7b40039c0 0x7fd7b4003e00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009670 tx=0x7fd7b801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ec878af7bf1978 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80b4070 con 0x7fd7b40039c0 2024-04-25T00:37:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b80a7900 con 0x7fd7b40039c0 2024-04-25T00:37:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c37fe640 1 -- 172.21.15.29:0/2284283114 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80096b0 con 0x7fd7b40039c0 2024-04-25T00:37:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 -- 172.21.15.29:0/2284283114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7b40039c0 msgr2=0x7fd7b4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 --2- 172.21.15.29:0/2284283114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7b40039c0 0x7fd7b4003e00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009670 tx=0x7fd7b801ec70 comp rx=0 tx=0).stop 2024-04-25T00:37:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 -- 172.21.15.29:0/2284283114 shutdown_connections 2024-04-25T00:37:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 --2- 172.21.15.29:0/2284283114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7b40039c0 0x7fd7b4003e00 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 -- 172.21.15.29:0/2284283114 >> 172.21.15.29:0/2284283114 conn(0x7fd7c40d0c40 msgr2=0x7fd7c40d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 -- 172.21.15.29:0/2284283114 shutdown_connections 2024-04-25T00:37:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 -- 172.21.15.29:0/2284283114 wait complete. 2024-04-25T00:37:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 Processor -- start 2024-04-25T00:37:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 -- start start 2024-04-25T00:37:33.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7b40039c0 0x7fd7c414e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:33.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c41592d0 0x7fd7c4159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:33.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c40ca1e0 0x7fd7c40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:33.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c406f5e0 con 0x7fd7c41592d0 2024-04-25T00:37:33.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c406f460 con 0x7fd7c40ca1e0 2024-04-25T00:37:33.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7cabba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c406f760 con 0x7fd7b40039c0 2024-04-25T00:37:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c41592d0 0x7fd7c4159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c41592d0 0x7fd7c4159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43694/0 (socket says 172.21.15.29:43694) 2024-04-25T00:37:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c3fff640 1 -- 172.21.15.29:0/2205434082 learned_addr learned my addr 172.21.15.29:0/2205434082 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c8956640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7b40039c0 0x7fd7c414e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c3fff640 1 -- 172.21.15.29:0/2205434082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7b40039c0 msgr2=0x7fd7c414e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c3fff640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7b40039c0 0x7fd7c414e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c9157640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c40ca1e0 0x7fd7c40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c3fff640 1 -- 172.21.15.29:0/2205434082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c40ca1e0 msgr2=0x7fd7c40ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c3fff640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c40ca1e0 0x7fd7c40ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c3fff640 1 -- 172.21.15.29:0/2205434082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c414c970 con 0x7fd7c41592d0 2024-04-25T00:37:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c8956640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7b40039c0 0x7fd7c414e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.623+0000 7fd7c9157640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c40ca1e0 0x7fd7c40ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.627+0000 7fd7c3fff640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c41592d0 0x7fd7c4159750 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000d5a0 tx=0x7fd7b000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.627+0000 7fd7c17fa640 1 -- 172.21.15.29:0/2205434082 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0005480 con 0x7fd7c41592d0 2024-04-25T00:37:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.627+0000 7fd7c17fa640 1 -- 172.21.15.29:0/2205434082 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b0017070 con 0x7fd7c41592d0 2024-04-25T00:37:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.627+0000 7fd7cabba640 1 -- 172.21.15.29:0/2205434082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c414f220 con 0x7fd7c41592d0 2024-04-25T00:37:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.627+0000 7fd7c17fa640 1 -- 172.21.15.29:0/2205434082 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0003820 con 0x7fd7c41592d0 2024-04-25T00:37:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.627+0000 7fd7cabba640 1 -- 172.21.15.29:0/2205434082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c414cbe0 con 0x7fd7c41592d0 2024-04-25T00:37:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.627+0000 7fd7cabba640 1 -- 172.21.15.29:0/2205434082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c4069270 con 0x7fd7c41592d0 2024-04-25T00:37:33.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.627+0000 7fd7c17fa640 1 -- 172.21.15.29:0/2205434082 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fd7b00039c0 con 0x7fd7c41592d0 2024-04-25T00:37:33.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.627+0000 7fd7c17fa640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd7a80679e0 0x7fd7a8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:33.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.627+0000 7fd7c17fa640 1 -- 172.21.15.29:0/2205434082 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fd7b0085cd0 con 0x7fd7c41592d0 2024-04-25T00:37:33.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.627+0000 7fd7c8956640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd7a80679e0 0x7fd7a8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:33.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.627+0000 7fd7c8956640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd7a80679e0 0x7fd7a8069ea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80b2040 tx=0x7fd7b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:33.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.631+0000 7fd7c17fa640 1 -- 172.21.15.29:0/2205434082 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd7b0019310 con 0x7fd7c41592d0 2024-04-25T00:37:33.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.787+0000 7fd7cabba640 1 -- 172.21.15.29:0/2205434082 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd7c415a7e0 con 0x7fd7a80679e0 2024-04-25T00:37:33.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.795+0000 7fd7c17fa640 1 -- 172.21.15.29:0/2205434082 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7fd7c415a7e0 con 0x7fd7a80679e0 2024-04-25T00:37:33.801 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:33.802 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:37:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.799+0000 7fd7cabba640 1 -- 172.21.15.29:0/2205434082 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd7a80679e0 msgr2=0x7fd7a8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.799+0000 7fd7cabba640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd7a80679e0 0x7fd7a8069ea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80b2040 tx=0x7fd7b8008040 comp rx=0 tx=0).stop 2024-04-25T00:37:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.799+0000 7fd7cabba640 1 -- 172.21.15.29:0/2205434082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c41592d0 msgr2=0x7fd7c4159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.799+0000 7fd7cabba640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c41592d0 0x7fd7c4159750 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000d5a0 tx=0x7fd7b000da60 comp rx=0 tx=0).stop 2024-04-25T00:37:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.799+0000 7fd7cabba640 1 -- 172.21.15.29:0/2205434082 shutdown_connections 2024-04-25T00:37:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.799+0000 7fd7cabba640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd7a80679e0 0x7fd7a8069ea0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.799+0000 7fd7cabba640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c40ca1e0 0x7fd7c40ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.799+0000 7fd7cabba640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c41592d0 0x7fd7c4159750 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.799+0000 7fd7cabba640 1 --2- 172.21.15.29:0/2205434082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7b40039c0 0x7fd7c414e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.799+0000 7fd7cabba640 1 -- 172.21.15.29:0/2205434082 >> 172.21.15.29:0/2205434082 conn(0x7fd7c40d0c40 msgr2=0x7fd7c40ce7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.799+0000 7fd7cabba640 1 -- 172.21.15.29:0/2205434082 shutdown_connections 2024-04-25T00:37:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.799+0000 7fd7cabba640 1 -- 172.21.15.29:0/2205434082 wait complete. 2024-04-25T00:37:33.819 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-25T00:37:33.349306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289816,"kb_used_data":11240,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754684904,"statfs":{"total":773094113280,"available":772797341696,"internally_reserved":0,"allocated":11509760,"data_stored":5686364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.738253"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617055+0000","last_change":"2024-04-25T00:37:24.657606+0000","last_active":"2024-04-25T00:37:25.617055+0000","last_peered":"2024-04-25T00:37:25.617055+0000","last_clean":"2024-04-25T00:37:25.617055+0000","last_became_active":"2024-04-25T00:37:24.656859+0000","last_became_peered":"2024-04-25T00:37:24.656859+0000","last_unstale":"2024-04-25T00:37:25.617055+0000","last_undegraded":"2024-04-25T00:37:25.617055+0000","last_fullsized":"2024-04-25T00:37:25.617055+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:16.259398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619452+0000","last_change":"2024-04-25T00:37:24.648164+0000","last_active":"2024-04-25T00:37:25.619452+0000","last_peered":"2024-04-25T00:37:25.619452+0000","last_clean":"2024-04-25T00:37:25.619452+0000","last_became_active":"2024-04-25T00:37:24.647383+0000","last_became_peered":"2024-04-25T00:37:24.647383+0000","last_unstale":"2024-04-25T00:37:25.619452+0000","last_undegraded":"2024-04-25T00:37:25.619452+0000","last_fullsized":"2024-04-25T00:37:25.619452+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:57.102276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616600+0000","last_change":"2024-04-25T00:37:24.644232+0000","last_active":"2024-04-25T00:37:25.616600+0000","last_peered":"2024-04-25T00:37:25.616600+0000","last_clean":"2024-04-25T00:37:25.616600+0000","last_became_active":"2024-04-25T00:37:24.642886+0000","last_became_peered":"2024-04-25T00:37:24.642886+0000","last_unstale":"2024-04-25T00:37:25.616600+0000","last_undegraded":"2024-04-25T00:37:25.616600+0000","last_fullsized":"2024-04-25T00:37:25.616600+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:21.524079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617224+0000","last_change":"2024-04-25T00:37:24.657429+0000","last_active":"2024-04-25T00:37:25.617224+0000","last_peered":"2024-04-25T00:37:25.617224+0000","last_clean":"2024-04-25T00:37:25.617224+0000","last_became_active":"2024-04-25T00:37:24.656738+0000","last_became_peered":"2024-04-25T00:37:24.656738+0000","last_unstale":"2024-04-25T00:37:25.617224+0000","last_undegraded":"2024-04-25T00:37:25.617224+0000","last_fullsized":"2024-04-25T00:37:25.617224+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:37.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890462+0000","last_change":"2024-04-25T00:37:24.650575+0000","last_active":"2024-04-25T00:37:25.890462+0000","last_peered":"2024-04-25T00:37:25.890462+0000","last_clean":"2024-04-25T00:37:25.890462+0000","last_became_active":"2024-04-25T00:37:24.649239+0000","last_became_peered":"2024-04-25T00:37:24.649239+0000","last_unstale":"2024-04-25T00:37:25.890462+0000","last_undegraded":"2024-04-25T00:37:25.890462+0000","last_fullsized":"2024-04-25T00:37:25.890462+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:09.766305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892915+0000","last_change":"2024-04-25T00:37:24.655067+0000","last_active":"2024-04-25T00:37:25.892915+0000","last_peered":"2024-04-25T00:37:25.892915+0000","last_clean":"2024-04-25T00:37:25.892915+0000","last_became_active":"2024-04-25T00:37:24.654401+0000","last_became_peered":"2024-04-25T00:37:24.654401+0000","last_unstale":"2024-04-25T00:37:25.892915+0000","last_undegraded":"2024-04-25T00:37:25.892915+0000","last_fullsized":"2024-04-25T00:37:25.892915+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:36.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890044+0000","last_change":"2024-04-25T00:37:24.649776+0000","last_active":"2024-04-25T00:37:25.890044+0000","last_peered":"2024-04-25T00:37:25.890044+0000","last_clean":"2024-04-25T00:37:25.890044+0000","last_became_active":"2024-04-25T00:37:24.648979+0000","last_became_peered":"2024-04-25T00:37:24.648979+0000","last_unstale":"2024-04-25T00:37:25.890044+0000","last_undegraded":"2024-04-25T00:37:25.890044+0000","last_fullsized":"2024-04-25T00:37:25.890044+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:44.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616146+0000","last_change":"2024-04-25T00:37:24.655900+0000","last_active":"2024-04-25T00:37:25.616146+0000","last_peered":"2024-04-25T00:37:25.616146+0000","last_clean":"2024-04-25T00:37:25.616146+0000","last_became_active":"2024-04-25T00:37:24.655169+0000","last_became_peered":"2024-04-25T00:37:24.655169+0000","last_unstale":"2024-04-25T00:37:25.616146+0000","last_undegraded":"2024-04-25T00:37:25.616146+0000","last_fullsized":"2024-04-25T00:37:25.616146+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:57.102276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617147+0000","last_change":"2024-04-25T00:37:24.644695+0000","last_active":"2024-04-25T00:37:25.617147+0000","last_peered":"2024-04-25T00:37:25.617147+0000","last_clean":"2024-04-25T00:37:25.617147+0000","last_became_active":"2024-04-25T00:37:24.643886+0000","last_became_peered":"2024-04-25T00:37:24.643886+0000","last_unstale":"2024-04-25T00:37:25.617147+0000","last_undegraded":"2024-04-25T00:37:25.617147+0000","last_fullsized":"2024-04-25T00:37:25.617147+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:24.434950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616914+0000","last_change":"2024-04-25T00:37:24.668000+0000","last_active":"2024-04-25T00:37:25.616914+0000","last_peered":"2024-04-25T00:37:25.616914+0000","last_clean":"2024-04-25T00:37:25.616914+0000","last_became_active":"2024-04-25T00:37:24.663574+0000","last_became_peered":"2024-04-25T00:37:24.663574+0000","last_unstale":"2024-04-25T00:37:25.616914+0000","last_undegraded":"2024-04-25T00:37:25.616914+0000","last_fullsized":"2024-04-25T00:37:25.616914+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:37.489323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619004+0000","last_change":"2024-04-25T00:37:24.649566+0000","last_active":"2024-04-25T00:37:25.619004+0000","last_peered":"2024-04-25T00:37:25.619004+0000","last_clean":"2024-04-25T00:37:25.619004+0000","last_became_active":"2024-04-25T00:37:24.648905+0000","last_became_peered":"2024-04-25T00:37:24.648905+0000","last_unstale":"2024-04-25T00:37:25.619004+0000","last_undegraded":"2024-04-25T00:37:25.619004+0000","last_fullsized":"2024-04-25T00:37:25.619004+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:30.099048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617355+0000","last_change":"2024-04-25T00:37:24.659945+0000","last_active":"2024-04-25T00:37:25.617355+0000","last_peered":"2024-04-25T00:37:25.617355+0000","last_clean":"2024-04-25T00:37:25.617355+0000","last_became_active":"2024-04-25T00:37:24.659273+0000","last_became_peered":"2024-04-25T00:37:24.659273+0000","last_unstale":"2024-04-25T00:37:25.617355+0000","last_undegraded":"2024-04-25T00:37:25.617355+0000","last_fullsized":"2024-04-25T00:37:25.617355+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:12.673417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617382+0000","last_change":"2024-04-25T00:37:24.656446+0000","last_active":"2024-04-25T00:37:25.617382+0000","last_peered":"2024-04-25T00:37:25.617382+0000","last_clean":"2024-04-25T00:37:25.617382+0000","last_became_active":"2024-04-25T00:37:24.655626+0000","last_became_peered":"2024-04-25T00:37:24.655626+0000","last_unstale":"2024-04-25T00:37:25.617382+0000","last_undegraded":"2024-04-25T00:37:25.617382+0000","last_fullsized":"2024-04-25T00:37:25.617382+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:07.618067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616119+0000","last_change":"2024-04-25T00:37:24.625020+0000","last_active":"2024-04-25T00:37:25.616119+0000","last_peered":"2024-04-25T00:37:25.616119+0000","last_clean":"2024-04-25T00:37:25.616119+0000","last_became_active":"2024-04-25T00:37:24.624274+0000","last_became_peered":"2024-04-25T00:37:24.624274+0000","last_unstale":"2024-04-25T00:37:25.616119+0000","last_undegraded":"2024-04-25T00:37:25.616119+0000","last_fullsized":"2024-04-25T00:37:25.616119+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:52.953810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890889+0000","last_change":"2024-04-25T00:37:24.653503+0000","last_active":"2024-04-25T00:37:25.890889+0000","last_peered":"2024-04-25T00:37:25.890889+0000","last_clean":"2024-04-25T00:37:25.890889+0000","last_became_active":"2024-04-25T00:37:24.652726+0000","last_became_peered":"2024-04-25T00:37:24.652726+0000","last_unstale":"2024-04-25T00:37:25.890889+0000","last_undegraded":"2024-04-25T00:37:25.890889+0000","last_fullsized":"2024-04-25T00:37:25.890889+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:36.836347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616099+0000","last_change":"2024-04-25T00:37:24.665542+0000","last_active":"2024-04-25T00:37:25.616099+0000","last_peered":"2024-04-25T00:37:25.616099+0000","last_clean":"2024-04-25T00:37:25.616099+0000","last_became_active":"2024-04-25T00:37:24.664831+0000","last_became_peered":"2024-04-25T00:37:24.664831+0000","last_unstale":"2024-04-25T00:37:25.616099+0000","last_undegraded":"2024-04-25T00:37:25.616099+0000","last_fullsized":"2024-04-25T00:37:25.616099+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:28.766277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619433+0000","last_change":"2024-04-25T00:37:24.650948+0000","last_active":"2024-04-25T00:37:25.619433+0000","last_peered":"2024-04-25T00:37:25.619433+0000","last_clean":"2024-04-25T00:37:25.619433+0000","last_became_active":"2024-04-25T00:37:24.649607+0000","last_became_peered":"2024-04-25T00:37:24.649607+0000","last_unstale":"2024-04-25T00:37:25.619433+0000","last_undegraded":"2024-04-25T00:37:25.619433+0000","last_fullsized":"2024-04-25T00:37:25.619433+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:52.953810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891102+0000","last_change":"2024-04-25T00:37:24.641490+0000","last_active":"2024-04-25T00:37:25.891102+0000","last_peered":"2024-04-25T00:37:25.891102+0000","last_clean":"2024-04-25T00:37:25.891102+0000","last_became_active":"2024-04-25T00:37:24.640694+0000","last_became_peered":"2024-04-25T00:37:24.640694+0000","last_unstale":"2024-04-25T00:37:25.891102+0000","last_undegraded":"2024-04-25T00:37:25.891102+0000","last_fullsized":"2024-04-25T00:37:25.891102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:42.857330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617010+0000","last_change":"2024-04-25T00:37:24.656560+0000","last_active":"2024-04-25T00:37:25.617010+0000","last_peered":"2024-04-25T00:37:25.617010+0000","last_clean":"2024-04-25T00:37:25.617010+0000","last_became_active":"2024-04-25T00:37:24.655791+0000","last_became_peered":"2024-04-25T00:37:24.655791+0000","last_unstale":"2024-04-25T00:37:25.617010+0000","last_undegraded":"2024-04-25T00:37:25.617010+0000","last_fullsized":"2024-04-25T00:37:25.617010+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:21.476709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.618771+0000","last_change":"2024-04-25T00:37:24.659487+0000","last_active":"2024-04-25T00:37:25.618771+0000","last_peered":"2024-04-25T00:37:25.618771+0000","last_clean":"2024-04-25T00:37:25.618771+0000","last_became_active":"2024-04-25T00:37:24.658548+0000","last_became_peered":"2024-04-25T00:37:24.658548+0000","last_unstale":"2024-04-25T00:37:25.618771+0000","last_undegraded":"2024-04-25T00:37:25.618771+0000","last_fullsized":"2024-04-25T00:37:25.618771+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:38.140994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619151+0000","last_change":"2024-04-25T00:37:24.648480+0000","last_active":"2024-04-25T00:37:25.619151+0000","last_peered":"2024-04-25T00:37:25.619151+0000","last_clean":"2024-04-25T00:37:25.619151+0000","last_became_active":"2024-04-25T00:37:24.647516+0000","last_became_peered":"2024-04-25T00:37:24.647516+0000","last_unstale":"2024-04-25T00:37:25.619151+0000","last_undegraded":"2024-04-25T00:37:25.619151+0000","last_fullsized":"2024-04-25T00:37:25.619151+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:45.097585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892096+0000","last_change":"2024-04-25T00:37:24.663784+0000","last_active":"2024-04-25T00:37:25.892096+0000","last_peered":"2024-04-25T00:37:25.892096+0000","last_clean":"2024-04-25T00:37:25.892096+0000","last_became_active":"2024-04-25T00:37:24.662549+0000","last_became_peered":"2024-04-25T00:37:24.662549+0000","last_unstale":"2024-04-25T00:37:25.892096+0000","last_undegraded":"2024-04-25T00:37:25.892096+0000","last_fullsized":"2024-04-25T00:37:25.892096+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:09.915478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617924+0000","last_change":"2024-04-25T00:37:24.658880+0000","last_active":"2024-04-25T00:37:25.617924+0000","last_peered":"2024-04-25T00:37:25.617924+0000","last_clean":"2024-04-25T00:37:25.617924+0000","last_became_active":"2024-04-25T00:37:24.658120+0000","last_became_peered":"2024-04-25T00:37:24.658120+0000","last_unstale":"2024-04-25T00:37:25.617924+0000","last_undegraded":"2024-04-25T00:37:25.617924+0000","last_fullsized":"2024-04-25T00:37:25.617924+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:03.166234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892331+0000","last_change":"2024-04-25T00:37:24.667001+0000","last_active":"2024-04-25T00:37:25.892331+0000","last_peered":"2024-04-25T00:37:25.892331+0000","last_clean":"2024-04-25T00:37:25.892331+0000","last_became_active":"2024-04-25T00:37:24.664458+0000","last_became_peered":"2024-04-25T00:37:24.664458+0000","last_unstale":"2024-04-25T00:37:25.892331+0000","last_undegraded":"2024-04-25T00:37:25.892331+0000","last_fullsized":"2024-04-25T00:37:25.892331+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:21.476709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.619962+0000","last_change":"2024-04-25T00:37:24.650335+0000","last_active":"2024-04-25T00:37:25.619962+0000","last_peered":"2024-04-25T00:37:25.619962+0000","last_clean":"2024-04-25T00:37:25.619962+0000","last_became_active":"2024-04-25T00:37:24.648644+0000","last_became_peered":"2024-04-25T00:37:24.648644+0000","last_unstale":"2024-04-25T00:37:25.619962+0000","last_undegraded":"2024-04-25T00:37:25.619962+0000","last_fullsized":"2024-04-25T00:37:25.619962+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:57.228594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617137+0000","last_change":"2024-04-25T00:37:24.638520+0000","last_active":"2024-04-25T00:37:25.617137+0000","last_peered":"2024-04-25T00:37:25.617137+0000","last_clean":"2024-04-25T00:37:25.617137+0000","last_became_active":"2024-04-25T00:37:24.637640+0000","last_became_peered":"2024-04-25T00:37:24.637640+0000","last_unstale":"2024-04-25T00:37:25.617137+0000","last_undegraded":"2024-04-25T00:37:25.617137+0000","last_fullsized":"2024-04-25T00:37:25.617137+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:43.653979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891294+0000","last_change":"2024-04-25T00:37:24.659187+0000","last_active":"2024-04-25T00:37:25.891294+0000","last_peered":"2024-04-25T00:37:25.891294+0000","last_clean":"2024-04-25T00:37:25.891294+0000","last_became_active":"2024-04-25T00:37:24.658411+0000","last_became_peered":"2024-04-25T00:37:24.658411+0000","last_unstale":"2024-04-25T00:37:25.891294+0000","last_undegraded":"2024-04-25T00:37:25.891294+0000","last_fullsized":"2024-04-25T00:37:25.891294+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:46.789133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892883+0000","last_change":"2024-04-25T00:37:24.645549+0000","last_active":"2024-04-25T00:37:25.892883+0000","last_peered":"2024-04-25T00:37:25.892883+0000","last_clean":"2024-04-25T00:37:25.892883+0000","last_became_active":"2024-04-25T00:37:24.644769+0000","last_became_peered":"2024-04-25T00:37:24.644769+0000","last_unstale":"2024-04-25T00:37:25.892883+0000","last_undegraded":"2024-04-25T00:37:25.892883+0000","last_fullsized":"2024-04-25T00:37:25.892883+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:07.618067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.618138+0000","last_change":"2024-04-25T00:37:24.664583+0000","last_active":"2024-04-25T00:37:25.618138+0000","last_peered":"2024-04-25T00:37:25.618138+0000","last_clean":"2024-04-25T00:37:25.618138+0000","last_became_active":"2024-04-25T00:37:24.663713+0000","last_became_peered":"2024-04-25T00:37:24.663713+0000","last_unstale":"2024-04-25T00:37:25.618138+0000","last_undegraded":"2024-04-25T00:37:25.618138+0000","last_fullsized":"2024-04-25T00:37:25.618138+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:42.857330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890809+0000","last_change":"2024-04-25T00:37:24.650632+0000","last_active":"2024-04-25T00:37:25.890809+0000","last_peered":"2024-04-25T00:37:25.890809+0000","last_clean":"2024-04-25T00:37:25.890809+0000","last_became_active":"2024-04-25T00:37:24.649835+0000","last_became_peered":"2024-04-25T00:37:24.649835+0000","last_unstale":"2024-04-25T00:37:25.890809+0000","last_undegraded":"2024-04-25T00:37:25.890809+0000","last_fullsized":"2024-04-25T00:37:25.890809+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:24.692684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616247+0000","last_change":"2024-04-25T00:37:24.638702+0000","last_active":"2024-04-25T00:37:25.616247+0000","last_peered":"2024-04-25T00:37:25.616247+0000","last_clean":"2024-04-25T00:37:25.616247+0000","last_became_active":"2024-04-25T00:37:24.637878+0000","last_became_peered":"2024-04-25T00:37:24.637878+0000","last_unstale":"2024-04-25T00:37:25.616247+0000","last_undegraded":"2024-04-25T00:37:25.616247+0000","last_fullsized":"2024-04-25T00:37:25.616247+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:18.566344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891202+0000","last_change":"2024-04-25T00:37:24.647978+0000","last_active":"2024-04-25T00:37:25.891202+0000","last_peered":"2024-04-25T00:37:25.891202+0000","last_clean":"2024-04-25T00:37:25.891202+0000","last_became_active":"2024-04-25T00:37:24.647202+0000","last_became_peered":"2024-04-25T00:37:24.647202+0000","last_unstale":"2024-04-25T00:37:25.891202+0000","last_undegraded":"2024-04-25T00:37:25.891202+0000","last_fullsized":"2024-04-25T00:37:25.891202+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:14.123451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891406+0000","last_change":"2024-04-25T00:37:24.651972+0000","last_active":"2024-04-25T00:37:25.891406+0000","last_peered":"2024-04-25T00:37:25.891406+0000","last_clean":"2024-04-25T00:37:25.891406+0000","last_became_active":"2024-04-25T00:37:24.650939+0000","last_became_peered":"2024-04-25T00:37:24.650939+0000","last_unstale":"2024-04-25T00:37:25.891406+0000","last_undegraded":"2024-04-25T00:37:25.891406+0000","last_fullsized":"2024-04-25T00:37:25.891406+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:35.939082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"102'4","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891440+0000","last_change":"2024-04-25T00:37:23.981233+0000","last_active":"2024-04-25T00:37:25.891440+0000","last_peered":"2024-04-25T00:37:25.891440+0000","last_clean":"2024-04-25T00:37:25.891440+0000","last_became_active":"2024-04-25T00:37:23.980150+0000","last_became_peered":"2024-04-25T00:37:23.980150+0000","last_unstale":"2024-04-25T00:37:25.891440+0000","last_undegraded":"2024-04-25T00:37:25.891440+0000","last_fullsized":"2024-04-25T00:37:25.891440+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:47.226214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891957+0000","last_change":"2024-04-25T00:37:24.634734+0000","last_active":"2024-04-25T00:37:25.891957+0000","last_peered":"2024-04-25T00:37:25.891957+0000","last_clean":"2024-04-25T00:37:25.891957+0000","last_became_active":"2024-04-25T00:37:24.628248+0000","last_became_peered":"2024-04-25T00:37:24.628248+0000","last_unstale":"2024-04-25T00:37:25.891957+0000","last_undegraded":"2024-04-25T00:37:25.891957+0000","last_fullsized":"2024-04-25T00:37:25.891957+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:45.097585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617426+0000","last_change":"2024-04-25T00:37:23.974780+0000","last_active":"2024-04-25T00:37:25.617426+0000","last_peered":"2024-04-25T00:37:25.617426+0000","last_clean":"2024-04-25T00:37:25.617426+0000","last_became_active":"2024-04-25T00:37:23.973844+0000","last_became_peered":"2024-04-25T00:37:23.973844+0000","last_unstale":"2024-04-25T00:37:25.617426+0000","last_undegraded":"2024-04-25T00:37:25.617426+0000","last_fullsized":"2024-04-25T00:37:25.617426+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:50.821959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891301+0000","last_change":"2024-04-25T00:37:24.654395+0000","last_active":"2024-04-25T00:37:25.891301+0000","last_peered":"2024-04-25T00:37:25.891301+0000","last_clean":"2024-04-25T00:37:25.891301+0000","last_became_active":"2024-04-25T00:37:24.652977+0000","last_became_peered":"2024-04-25T00:37:24.652977+0000","last_unstale":"2024-04-25T00:37:25.891301+0000","last_undegraded":"2024-04-25T00:37:25.891301+0000","last_fullsized":"2024-04-25T00:37:25.891301+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:38:20.938337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889564+0000","last_change":"2024-04-25T00:37:23.962945+0000","last_active":"2024-04-25T00:37:25.889564+0000","last_peered":"2024-04-25T00:37:25.889564+0000","last_clean":"2024-04-25T00:37:25.889564+0000","last_became_active":"2024-04-25T00:37:23.962460+0000","last_became_peered":"2024-04-25T00:37:23.962460+0000","last_unstale":"2024-04-25T00:37:25.889564+0000","last_undegraded":"2024-04-25T00:37:25.889564+0000","last_fullsized":"2024-04-25T00:37:25.889564+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:05.179578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617374+0000","last_change":"2024-04-25T00:37:24.656227+0000","last_active":"2024-04-25T00:37:25.617374+0000","last_peered":"2024-04-25T00:37:25.617374+0000","last_clean":"2024-04-25T00:37:25.617374+0000","last_became_active":"2024-04-25T00:37:24.655453+0000","last_became_peered":"2024-04-25T00:37:24.655453+0000","last_unstale":"2024-04-25T00:37:25.617374+0000","last_undegraded":"2024-04-25T00:37:25.617374+0000","last_fullsized":"2024-04-25T00:37:25.617374+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:38.243062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890255+0000","last_change":"2024-04-25T00:37:23.967844+0000","last_active":"2024-04-25T00:37:25.890255+0000","last_peered":"2024-04-25T00:37:25.890255+0000","last_clean":"2024-04-25T00:37:25.890255+0000","last_became_active":"2024-04-25T00:37:23.967545+0000","last_became_peered":"2024-04-25T00:37:23.967545+0000","last_unstale":"2024-04-25T00:37:25.890255+0000","last_undegraded":"2024-04-25T00:37:25.890255+0000","last_fullsized":"2024-04-25T00:37:25.890255+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:03.459207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616154+0000","last_change":"2024-04-25T00:37:24.665789+0000","last_active":"2024-04-25T00:37:25.616154+0000","last_peered":"2024-04-25T00:37:25.616154+0000","last_clean":"2024-04-25T00:37:25.616154+0000","last_became_active":"2024-04-25T00:37:24.665110+0000","last_became_peered":"2024-04-25T00:37:24.665110+0000","last_unstale":"2024-04-25T00:37:25.616154+0000","last_undegraded":"2024-04-25T00:37:25.616154+0000","last_fullsized":"2024-04-25T00:37:25.616154+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:24.334663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889798+0000","last_change":"2024-04-25T00:37:23.971620+0000","last_active":"2024-04-25T00:37:25.889798+0000","last_peered":"2024-04-25T00:37:25.889798+0000","last_clean":"2024-04-25T00:37:25.889798+0000","last_became_active":"2024-04-25T00:37:23.970901+0000","last_became_peered":"2024-04-25T00:37:23.970901+0000","last_unstale":"2024-04-25T00:37:25.889798+0000","last_undegraded":"2024-04-25T00:37:25.889798+0000","last_fullsized":"2024-04-25T00:37:25.889798+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:54.449682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.618554+0000","last_change":"2024-04-25T00:37:24.654380+0000","last_active":"2024-04-25T00:37:25.618554+0000","last_peered":"2024-04-25T00:37:25.618554+0000","last_clean":"2024-04-25T00:37:25.618554+0000","last_became_active":"2024-04-25T00:37:24.653655+0000","last_became_peered":"2024-04-25T00:37:24.653655+0000","last_unstale":"2024-04-25T00:37:25.618554+0000","last_undegraded":"2024-04-25T00:37:25.618554+0000","last_fullsized":"2024-04-25T00:37:25.618554+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.195629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892229+0000","last_change":"2024-04-25T00:37:23.974470+0000","last_active":"2024-04-25T00:37:25.892229+0000","last_peered":"2024-04-25T00:37:25.892229+0000","last_clean":"2024-04-25T00:37:25.892229+0000","last_became_active":"2024-04-25T00:37:23.974141+0000","last_became_peered":"2024-04-25T00:37:23.974141+0000","last_unstale":"2024-04-25T00:37:25.892229+0000","last_undegraded":"2024-04-25T00:37:25.892229+0000","last_fullsized":"2024-04-25T00:37:25.892229+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:08.824434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616040+0000","last_change":"2024-04-25T00:37:24.659767+0000","last_active":"2024-04-25T00:37:25.616040+0000","last_peered":"2024-04-25T00:37:25.616040+0000","last_clean":"2024-04-25T00:37:25.616040+0000","last_became_active":"2024-04-25T00:37:24.659084+0000","last_became_peered":"2024-04-25T00:37:24.659084+0000","last_unstale":"2024-04-25T00:37:25.616040+0000","last_undegraded":"2024-04-25T00:37:25.616040+0000","last_fullsized":"2024-04-25T00:37:25.616040+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:45.475299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617781+0000","last_change":"2024-04-25T00:37:23.960317+0000","last_active":"2024-04-25T00:37:25.617781+0000","last_peered":"2024-04-25T00:37:25.617781+0000","last_clean":"2024-04-25T00:37:25.617781+0000","last_became_active":"2024-04-25T00:37:23.959614+0000","last_became_peered":"2024-04-25T00:37:23.959614+0000","last_unstale":"2024-04-25T00:37:25.617781+0000","last_undegraded":"2024-04-25T00:37:25.617781+0000","last_fullsized":"2024-04-25T00:37:25.617781+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:34.332398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889853+0000","last_change":"2024-04-25T00:37:24.651641+0000","last_active":"2024-04-25T00:37:25.889853+0000","last_peered":"2024-04-25T00:37:25.889853+0000","last_clean":"2024-04-25T00:37:25.889853+0000","last_became_active":"2024-04-25T00:37:24.649207+0000","last_became_peered":"2024-04-25T00:37:24.649207+0000","last_unstale":"2024-04-25T00:37:25.889853+0000","last_undegraded":"2024-04-25T00:37:25.889853+0000","last_fullsized":"2024-04-25T00:37:25.889853+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:45.012553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616204+0000","last_change":"2024-04-25T00:37:23.971464+0000","last_active":"2024-04-25T00:37:25.616204+0000","last_peered":"2024-04-25T00:37:25.616204+0000","last_clean":"2024-04-25T00:37:25.616204+0000","last_became_active":"2024-04-25T00:37:23.970908+0000","last_became_peered":"2024-04-25T00:37:23.970908+0000","last_unstale":"2024-04-25T00:37:25.616204+0000","last_undegraded":"2024-04-25T00:37:25.616204+0000","last_fullsized":"2024-04-25T00:37:25.616204+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:56.224951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890699+0000","last_change":"2024-04-25T00:37:24.651933+0000","last_active":"2024-04-25T00:37:25.890699+0000","last_peered":"2024-04-25T00:37:25.890699+0000","last_clean":"2024-04-25T00:37:25.890699+0000","last_became_active":"2024-04-25T00:37:24.651128+0000","last_became_peered":"2024-04-25T00:37:24.651128+0000","last_unstale":"2024-04-25T00:37:25.890699+0000","last_undegraded":"2024-04-25T00:37:25.890699+0000","last_fullsized":"2024-04-25T00:37:25.890699+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:07.838253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889662+0000","last_change":"2024-04-25T00:37:23.962738+0000","last_active":"2024-04-25T00:37:25.889662+0000","last_peered":"2024-04-25T00:37:25.889662+0000","last_clean":"2024-04-25T00:37:25.889662+0000","last_became_active":"2024-04-25T00:37:23.962330+0000","last_became_peered":"2024-04-25T00:37:23.962330+0000","last_unstale":"2024-04-25T00:37:25.889662+0000","last_undegraded":"2024-04-25T00:37:25.889662+0000","last_fullsized":"2024-04-25T00:37:25.889662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:50.647322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891917+0000","last_change":"2024-04-25T00:37:24.639458+0000","last_active":"2024-04-25T00:37:25.891917+0000","last_peered":"2024-04-25T00:37:25.891917+0000","last_clean":"2024-04-25T00:37:25.891917+0000","last_became_active":"2024-04-25T00:37:24.637555+0000","last_became_peered":"2024-04-25T00:37:24.637555+0000","last_unstale":"2024-04-25T00:37:25.891917+0000","last_undegraded":"2024-04-25T00:37:25.891917+0000","last_fullsized":"2024-04-25T00:37:25.891917+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:30.099048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891682+0000","last_change":"2024-04-25T00:37:23.973733+0000","last_active":"2024-04-25T00:37:25.891682+0000","last_peered":"2024-04-25T00:37:25.891682+0000","last_clean":"2024-04-25T00:37:25.891682+0000","last_became_active":"2024-04-25T00:37:23.973362+0000","last_became_peered":"2024-04-25T00:37:23.973362+0000","last_unstale":"2024-04-25T00:37:25.891682+0000","last_undegraded":"2024-04-25T00:37:25.891682+0000","last_fullsized":"2024-04-25T00:37:25.891682+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:51.405634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.893367+0000","last_change":"2024-04-25T00:37:24.649357+0000","last_active":"2024-04-25T00:37:25.893367+0000","last_peered":"2024-04-25T00:37:25.893367+0000","last_clean":"2024-04-25T00:37:25.893367+0000","last_became_active":"2024-04-25T00:37:24.647321+0000","last_became_peered":"2024-04-25T00:37:24.647321+0000","last_unstale":"2024-04-25T00:37:25.893367+0000","last_undegraded":"2024-04-25T00:37:25.893367+0000","last_fullsized":"2024-04-25T00:37:25.893367+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:26.159139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"102'3","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616207+0000","last_change":"2024-04-25T00:37:23.966485+0000","last_active":"2024-04-25T00:37:25.616207+0000","last_peered":"2024-04-25T00:37:25.616207+0000","last_clean":"2024-04-25T00:37:25.616207+0000","last_became_active":"2024-04-25T00:37:23.966125+0000","last_became_peered":"2024-04-25T00:37:23.966125+0000","last_unstale":"2024-04-25T00:37:25.616207+0000","last_undegraded":"2024-04-25T00:37:25.616207+0000","last_fullsized":"2024-04-25T00:37:25.616207+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:44.093942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890417+0000","last_change":"2024-04-25T00:37:24.659214+0000","last_active":"2024-04-25T00:37:25.890417+0000","last_peered":"2024-04-25T00:37:25.890417+0000","last_clean":"2024-04-25T00:37:25.890417+0000","last_became_active":"2024-04-25T00:37:24.658485+0000","last_became_peered":"2024-04-25T00:37:24.658485+0000","last_unstale":"2024-04-25T00:37:25.890417+0000","last_undegraded":"2024-04-25T00:37:25.890417+0000","last_fullsized":"2024-04-25T00:37:25.890417+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:09.576753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"102'6","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617542+0000","last_change":"2024-04-25T00:37:23.971682+0000","last_active":"2024-04-25T00:37:25.617542+0000","last_peered":"2024-04-25T00:37:25.617542+0000","last_clean":"2024-04-25T00:37:25.617542+0000","last_became_active":"2024-04-25T00:37:23.970797+0000","last_became_peered":"2024-04-25T00:37:23.970797+0000","last_unstale":"2024-04-25T00:37:25.617542+0000","last_undegraded":"2024-04-25T00:37:25.617542+0000","last_fullsized":"2024-04-25T00:37:25.617542+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:27.326430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.893051+0000","last_change":"2024-04-25T00:37:24.649497+0000","last_active":"2024-04-25T00:37:25.893051+0000","last_peered":"2024-04-25T00:37:25.893051+0000","last_clean":"2024-04-25T00:37:25.893051+0000","last_became_active":"2024-04-25T00:37:24.648738+0000","last_became_peered":"2024-04-25T00:37:24.648738+0000","last_unstale":"2024-04-25T00:37:25.893051+0000","last_undegraded":"2024-04-25T00:37:25.893051+0000","last_fullsized":"2024-04-25T00:37:25.893051+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:39.326233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891628+0000","last_change":"2024-04-25T00:37:23.980023+0000","last_active":"2024-04-25T00:37:25.891628+0000","last_peered":"2024-04-25T00:37:25.891628+0000","last_clean":"2024-04-25T00:37:25.891628+0000","last_became_active":"2024-04-25T00:37:23.979702+0000","last_became_peered":"2024-04-25T00:37:23.979702+0000","last_unstale":"2024-04-25T00:37:25.891628+0000","last_undegraded":"2024-04-25T00:37:25.891628+0000","last_fullsized":"2024-04-25T00:37:25.891628+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:36.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.893084+0000","last_change":"2024-04-25T00:37:24.663106+0000","last_active":"2024-04-25T00:37:25.893084+0000","last_peered":"2024-04-25T00:37:25.893084+0000","last_clean":"2024-04-25T00:37:25.893084+0000","last_became_active":"2024-04-25T00:37:24.662275+0000","last_became_peered":"2024-04-25T00:37:24.662275+0000","last_unstale":"2024-04-25T00:37:25.893084+0000","last_undegraded":"2024-04-25T00:37:25.893084+0000","last_fullsized":"2024-04-25T00:37:25.893084+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:49.644731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891008+0000","last_change":"2024-04-25T00:37:23.974737+0000","last_active":"2024-04-25T00:37:25.891008+0000","last_peered":"2024-04-25T00:37:25.891008+0000","last_clean":"2024-04-25T00:37:25.891008+0000","last_became_active":"2024-04-25T00:37:23.974006+0000","last_became_peered":"2024-04-25T00:37:23.974006+0000","last_unstale":"2024-04-25T00:37:25.891008+0000","last_undegraded":"2024-04-25T00:37:25.891008+0000","last_fullsized":"2024-04-25T00:37:25.891008+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:06.834610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891850+0000","last_change":"2024-04-25T00:37:24.648169+0000","last_active":"2024-04-25T00:37:25.891850+0000","last_peered":"2024-04-25T00:37:25.891850+0000","last_clean":"2024-04-25T00:37:25.891850+0000","last_became_active":"2024-04-25T00:37:24.646922+0000","last_became_peered":"2024-04-25T00:37:24.646922+0000","last_unstale":"2024-04-25T00:37:25.891850+0000","last_undegraded":"2024-04-25T00:37:25.891850+0000","last_fullsized":"2024-04-25T00:37:25.891850+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:49.957775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"102'8","reported_seq":22,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889187+0000","last_change":"2024-04-25T00:37:23.972616+0000","last_active":"2024-04-25T00:37:25.889187+0000","last_peered":"2024-04-25T00:37:25.889187+0000","last_clean":"2024-04-25T00:37:25.889187+0000","last_became_active":"2024-04-25T00:37:23.972302+0000","last_became_peered":"2024-04-25T00:37:23.972302+0000","last_unstale":"2024-04-25T00:37:25.889187+0000","last_undegraded":"2024-04-25T00:37:25.889187+0000","last_fullsized":"2024-04-25T00:37:25.889187+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:26.276154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890017+0000","last_change":"2024-04-25T00:37:24.649798+0000","last_active":"2024-04-25T00:37:25.890017+0000","last_peered":"2024-04-25T00:37:25.890017+0000","last_clean":"2024-04-25T00:37:25.890017+0000","last_became_active":"2024-04-25T00:37:24.648984+0000","last_became_peered":"2024-04-25T00:37:24.648984+0000","last_unstale":"2024-04-25T00:37:25.890017+0000","last_undegraded":"2024-04-25T00:37:25.890017+0000","last_fullsized":"2024-04-25T00:37:25.890017+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:37:41.353172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617304+0000","last_change":"2024-04-25T00:37:23.973856+0000","last_active":"2024-04-25T00:37:25.617304+0000","last_peered":"2024-04-25T00:37:25.617304+0000","last_clean":"2024-04-25T00:37:25.617304+0000","last_became_active":"2024-04-25T00:37:23.973576+0000","last_became_peered":"2024-04-25T00:37:23.973576+0000","last_unstale":"2024-04-25T00:37:25.617304+0000","last_undegraded":"2024-04-25T00:37:25.617304+0000","last_fullsized":"2024-04-25T00:37:25.617304+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:56.098633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891635+0000","last_change":"2024-04-25T00:37:24.659185+0000","last_active":"2024-04-25T00:37:25.891635+0000","last_peered":"2024-04-25T00:37:25.891635+0000","last_clean":"2024-04-25T00:37:25.891635+0000","last_became_active":"2024-04-25T00:37:24.658389+0000","last_became_peered":"2024-04-25T00:37:24.658389+0000","last_unstale":"2024-04-25T00:37:25.891635+0000","last_undegraded":"2024-04-25T00:37:25.891635+0000","last_fullsized":"2024-04-25T00:37:25.891635+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:52.832892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889893+0000","last_change":"2024-04-25T00:37:23.963264+0000","last_active":"2024-04-25T00:37:25.889893+0000","last_peered":"2024-04-25T00:37:25.889893+0000","last_clean":"2024-04-25T00:37:25.889893+0000","last_became_active":"2024-04-25T00:37:23.962746+0000","last_became_peered":"2024-04-25T00:37:23.962746+0000","last_unstale":"2024-04-25T00:37:25.889893+0000","last_undegraded":"2024-04-25T00:37:25.889893+0000","last_fullsized":"2024-04-25T00:37:25.889893+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:36.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616475+0000","last_change":"2024-04-25T00:37:24.632170+0000","last_active":"2024-04-25T00:37:25.616475+0000","last_peered":"2024-04-25T00:37:25.616475+0000","last_clean":"2024-04-25T00:37:25.616475+0000","last_became_active":"2024-04-25T00:37:24.631515+0000","last_became_peered":"2024-04-25T00:37:24.631515+0000","last_unstale":"2024-04-25T00:37:25.616475+0000","last_undegraded":"2024-04-25T00:37:25.616475+0000","last_fullsized":"2024-04-25T00:37:25.616475+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:49.957775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891186+0000","last_change":"2024-04-25T00:37:23.969113+0000","last_active":"2024-04-25T00:37:25.891186+0000","last_peered":"2024-04-25T00:37:25.891186+0000","last_clean":"2024-04-25T00:37:25.891186+0000","last_became_active":"2024-04-25T00:37:23.968373+0000","last_became_peered":"2024-04-25T00:37:23.968373+0000","last_unstale":"2024-04-25T00:37:25.891186+0000","last_undegraded":"2024-04-25T00:37:25.891186+0000","last_fullsized":"2024-04-25T00:37:25.891186+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:23.689041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617733+0000","last_change":"2024-04-25T00:37:24.643621+0000","last_active":"2024-04-25T00:37:25.617733+0000","last_peered":"2024-04-25T00:37:25.617733+0000","last_clean":"2024-04-25T00:37:25.617733+0000","last_became_active":"2024-04-25T00:37:24.641963+0000","last_became_peered":"2024-04-25T00:37:24.641963+0000","last_unstale":"2024-04-25T00:37:25.617733+0000","last_undegraded":"2024-04-25T00:37:25.617733+0000","last_fullsized":"2024-04-25T00:37:25.617733+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:49.644731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889848+0000","last_change":"2024-04-25T00:37:23.975090+0000","last_active":"2024-04-25T00:37:25.889848+0000","last_peered":"2024-04-25T00:37:25.889848+0000","last_clean":"2024-04-25T00:37:25.889848+0000","last_became_active":"2024-04-25T00:37:23.973684+0000","last_became_peered":"2024-04-25T00:37:23.973684+0000","last_unstale":"2024-04-25T00:37:25.889848+0000","last_undegraded":"2024-04-25T00:37:25.889848+0000","last_fullsized":"2024-04-25T00:37:25.889848+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:35.743952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617677+0000","last_change":"2024-04-25T00:37:24.641044+0000","last_active":"2024-04-25T00:37:25.617677+0000","last_peered":"2024-04-25T00:37:25.617677+0000","last_clean":"2024-04-25T00:37:25.617677+0000","last_became_active":"2024-04-25T00:37:24.635401+0000","last_became_peered":"2024-04-25T00:37:24.635401+0000","last_unstale":"2024-04-25T00:37:25.617677+0000","last_undegraded":"2024-04-25T00:37:25.617677+0000","last_fullsized":"2024-04-25T00:37:25.617677+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:39.326233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891670+0000","last_change":"2024-04-25T00:37:23.962101+0000","last_active":"2024-04-25T00:37:25.891670+0000","last_peered":"2024-04-25T00:37:25.891670+0000","last_clean":"2024-04-25T00:37:25.891670+0000","last_became_active":"2024-04-25T00:37:23.961767+0000","last_became_peered":"2024-04-25T00:37:23.961767+0000","last_unstale":"2024-04-25T00:37:25.891670+0000","last_undegraded":"2024-04-25T00:37:25.891670+0000","last_fullsized":"2024-04-25T00:37:25.891670+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:05.179578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616518+0000","last_change":"2024-04-25T00:37:24.624859+0000","last_active":"2024-04-25T00:37:25.616518+0000","last_peered":"2024-04-25T00:37:25.616518+0000","last_clean":"2024-04-25T00:37:25.616518+0000","last_became_active":"2024-04-25T00:37:24.624061+0000","last_became_peered":"2024-04-25T00:37:24.624061+0000","last_unstale":"2024-04-25T00:37:25.616518+0000","last_undegraded":"2024-04-25T00:37:25.616518+0000","last_fullsized":"2024-04-25T00:37:25.616518+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:10.228348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889401+0000","last_change":"2024-04-25T00:37:23.962517+0000","last_active":"2024-04-25T00:37:25.889401+0000","last_peered":"2024-04-25T00:37:25.889401+0000","last_clean":"2024-04-25T00:37:25.889401+0000","last_became_active":"2024-04-25T00:37:23.962068+0000","last_became_peered":"2024-04-25T00:37:23.962068+0000","last_unstale":"2024-04-25T00:37:25.889401+0000","last_undegraded":"2024-04-25T00:37:25.889401+0000","last_fullsized":"2024-04-25T00:37:25.889401+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:45.756449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616212+0000","last_change":"2024-04-25T00:37:24.656268+0000","last_active":"2024-04-25T00:37:25.616212+0000","last_peered":"2024-04-25T00:37:25.616212+0000","last_clean":"2024-04-25T00:37:25.616212+0000","last_became_active":"2024-04-25T00:37:24.655483+0000","last_became_peered":"2024-04-25T00:37:24.655483+0000","last_unstale":"2024-04-25T00:37:25.616212+0000","last_undegraded":"2024-04-25T00:37:25.616212+0000","last_fullsized":"2024-04-25T00:37:25.616212+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:57.228594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891056+0000","last_change":"2024-04-25T00:37:23.962073+0000","last_active":"2024-04-25T00:37:25.891056+0000","last_peered":"2024-04-25T00:37:25.891056+0000","last_clean":"2024-04-25T00:37:25.891056+0000","last_became_active":"2024-04-25T00:37:23.961617+0000","last_became_peered":"2024-04-25T00:37:23.961617+0000","last_unstale":"2024-04-25T00:37:25.891056+0000","last_undegraded":"2024-04-25T00:37:25.891056+0000","last_fullsized":"2024-04-25T00:37:25.891056+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:50.647322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892296+0000","last_change":"2024-04-25T00:37:24.657468+0000","last_active":"2024-04-25T00:37:25.892296+0000","last_peered":"2024-04-25T00:37:25.892296+0000","last_clean":"2024-04-25T00:37:25.892296+0000","last_became_active":"2024-04-25T00:37:24.656654+0000","last_became_peered":"2024-04-25T00:37:24.656654+0000","last_unstale":"2024-04-25T00:37:25.892296+0000","last_undegraded":"2024-04-25T00:37:25.892296+0000","last_fullsized":"2024-04-25T00:37:25.892296+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:21.524079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616345+0000","last_change":"2024-04-25T00:37:23.974633+0000","last_active":"2024-04-25T00:37:25.616345+0000","last_peered":"2024-04-25T00:37:25.616345+0000","last_clean":"2024-04-25T00:37:25.616345+0000","last_became_active":"2024-04-25T00:37:23.974067+0000","last_became_peered":"2024-04-25T00:37:23.974067+0000","last_unstale":"2024-04-25T00:37:25.616345+0000","last_undegraded":"2024-04-25T00:37:25.616345+0000","last_fullsized":"2024-04-25T00:37:25.616345+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:08.911836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891425+0000","last_change":"2024-04-25T00:37:24.639298+0000","last_active":"2024-04-25T00:37:25.891425+0000","last_peered":"2024-04-25T00:37:25.891425+0000","last_clean":"2024-04-25T00:37:25.891425+0000","last_became_active":"2024-04-25T00:37:24.637729+0000","last_became_peered":"2024-04-25T00:37:24.637729+0000","last_unstale":"2024-04-25T00:37:25.891425+0000","last_undegraded":"2024-04-25T00:37:25.891425+0000","last_fullsized":"2024-04-25T00:37:25.891425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:38.140994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892247+0000","last_change":"2024-04-25T00:37:23.973640+0000","last_active":"2024-04-25T00:37:25.892247+0000","last_peered":"2024-04-25T00:37:25.892247+0000","last_clean":"2024-04-25T00:37:25.892247+0000","last_became_active":"2024-04-25T00:37:23.973210+0000","last_became_peered":"2024-04-25T00:37:23.973210+0000","last_unstale":"2024-04-25T00:37:25.892247+0000","last_undegraded":"2024-04-25T00:37:25.892247+0000","last_fullsized":"2024-04-25T00:37:25.892247+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:56.021774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617275+0000","last_change":"2024-04-25T00:37:24.643054+0000","last_active":"2024-04-25T00:37:25.617275+0000","last_peered":"2024-04-25T00:37:25.617275+0000","last_clean":"2024-04-25T00:37:25.617275+0000","last_became_active":"2024-04-25T00:37:24.642305+0000","last_became_peered":"2024-04-25T00:37:24.642305+0000","last_unstale":"2024-04-25T00:37:25.617275+0000","last_undegraded":"2024-04-25T00:37:25.617275+0000","last_fullsized":"2024-04-25T00:37:25.617275+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:28.830671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"103'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891625+0000","last_change":"2024-04-25T00:37:23.976356+0000","last_active":"2024-04-25T00:37:25.891625+0000","last_peered":"2024-04-25T00:37:25.891625+0000","last_clean":"2024-04-25T00:37:25.891625+0000","last_became_active":"2024-04-25T00:37:23.975148+0000","last_became_peered":"2024-04-25T00:37:23.975148+0000","last_unstale":"2024-04-25T00:37:25.891625+0000","last_undegraded":"2024-04-25T00:37:25.891625+0000","last_fullsized":"2024-04-25T00:37:25.891625+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:50.821959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617474+0000","last_change":"2024-04-25T00:37:24.642433+0000","last_active":"2024-04-25T00:37:25.617474+0000","last_peered":"2024-04-25T00:37:25.617474+0000","last_clean":"2024-04-25T00:37:25.617474+0000","last_became_active":"2024-04-25T00:37:24.641630+0000","last_became_peered":"2024-04-25T00:37:24.641630+0000","last_unstale":"2024-04-25T00:37:25.617474+0000","last_undegraded":"2024-04-25T00:37:25.617474+0000","last_fullsized":"2024-04-25T00:37:25.617474+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:10.590475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616674+0000","last_change":"2024-04-25T00:37:23.975126+0000","last_active":"2024-04-25T00:37:25.616674+0000","last_peered":"2024-04-25T00:37:25.616674+0000","last_clean":"2024-04-25T00:37:25.616674+0000","last_became_active":"2024-04-25T00:37:23.974571+0000","last_became_peered":"2024-04-25T00:37:23.974571+0000","last_unstale":"2024-04-25T00:37:25.616674+0000","last_undegraded":"2024-04-25T00:37:25.616674+0000","last_fullsized":"2024-04-25T00:37:25.616674+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:51.950167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.891114+0000","last_change":"2024-04-25T00:37:24.637810+0000","last_active":"2024-04-25T00:37:25.891114+0000","last_peered":"2024-04-25T00:37:25.891114+0000","last_clean":"2024-04-25T00:37:25.891114+0000","last_became_active":"2024-04-25T00:37:24.636719+0000","last_became_peered":"2024-04-25T00:37:24.636719+0000","last_unstale":"2024-04-25T00:37:25.891114+0000","last_undegraded":"2024-04-25T00:37:25.891114+0000","last_fullsized":"2024-04-25T00:37:25.891114+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.195629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.893433+0000","last_change":"2024-04-25T00:37:23.962739+0000","last_active":"2024-04-25T00:37:25.893433+0000","last_peered":"2024-04-25T00:37:25.893433+0000","last_clean":"2024-04-25T00:37:25.893433+0000","last_became_active":"2024-04-25T00:37:23.962425+0000","last_became_peered":"2024-04-25T00:37:23.962425+0000","last_unstale":"2024-04-25T00:37:25.893433+0000","last_undegraded":"2024-04-25T00:37:25.893433+0000","last_fullsized":"2024-04-25T00:37:25.893433+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:02.162592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617946+0000","last_change":"2024-04-25T00:37:24.640862+0000","last_active":"2024-04-25T00:37:25.617946+0000","last_peered":"2024-04-25T00:37:25.617946+0000","last_clean":"2024-04-25T00:37:25.617946+0000","last_became_active":"2024-04-25T00:37:24.640060+0000","last_became_peered":"2024-04-25T00:37:24.640060+0000","last_unstale":"2024-04-25T00:37:25.617946+0000","last_undegraded":"2024-04-25T00:37:25.617946+0000","last_fullsized":"2024-04-25T00:37:25.617946+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:09.828077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889925+0000","last_change":"2024-04-25T00:37:23.976960+0000","last_active":"2024-04-25T00:37:25.889925+0000","last_peered":"2024-04-25T00:37:25.889925+0000","last_clean":"2024-04-25T00:37:25.889925+0000","last_became_active":"2024-04-25T00:37:23.971621+0000","last_became_peered":"2024-04-25T00:37:23.971621+0000","last_unstale":"2024-04-25T00:37:25.889925+0000","last_undegraded":"2024-04-25T00:37:25.889925+0000","last_fullsized":"2024-04-25T00:37:25.889925+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:51.258332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.618498+0000","last_change":"2024-04-25T00:37:24.653749+0000","last_active":"2024-04-25T00:37:25.618498+0000","last_peered":"2024-04-25T00:37:25.618498+0000","last_clean":"2024-04-25T00:37:25.618498+0000","last_became_active":"2024-04-25T00:37:24.653059+0000","last_became_peered":"2024-04-25T00:37:24.653059+0000","last_unstale":"2024-04-25T00:37:25.618498+0000","last_undegraded":"2024-04-25T00:37:25.618498+0000","last_fullsized":"2024-04-25T00:37:25.618498+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:28.330073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.617928+0000","last_change":"2024-04-25T00:37:23.963951+0000","last_active":"2024-04-25T00:37:25.617928+0000","last_peered":"2024-04-25T00:37:25.617928+0000","last_clean":"2024-04-25T00:37:25.617928+0000","last_became_active":"2024-04-25T00:37:23.963276+0000","last_became_peered":"2024-04-25T00:37:23.963276+0000","last_unstale":"2024-04-25T00:37:25.617928+0000","last_undegraded":"2024-04-25T00:37:25.617928+0000","last_fullsized":"2024-04-25T00:37:25.617928+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:41.471801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616544+0000","last_change":"2024-04-25T00:37:24.625317+0000","last_active":"2024-04-25T00:37:25.616544+0000","last_peered":"2024-04-25T00:37:25.616544+0000","last_clean":"2024-04-25T00:37:25.616544+0000","last_became_active":"2024-04-25T00:37:24.624551+0000","last_became_peered":"2024-04-25T00:37:24.624551+0000","last_unstale":"2024-04-25T00:37:25.616544+0000","last_undegraded":"2024-04-25T00:37:25.616544+0000","last_fullsized":"2024-04-25T00:37:25.616544+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:09.915478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.889262+0000","last_change":"2024-04-25T00:37:23.968009+0000","last_active":"2024-04-25T00:37:25.889262+0000","last_peered":"2024-04-25T00:37:25.889262+0000","last_clean":"2024-04-25T00:37:25.889262+0000","last_became_active":"2024-04-25T00:37:23.967586+0000","last_became_peered":"2024-04-25T00:37:23.967586+0000","last_unstale":"2024-04-25T00:37:25.889262+0000","last_undegraded":"2024-04-25T00:37:25.889262+0000","last_fullsized":"2024-04-25T00:37:25.889262+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:47.615185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616472+0000","last_change":"2024-04-25T00:37:24.658718+0000","last_active":"2024-04-25T00:37:25.616472+0000","last_peered":"2024-04-25T00:37:25.616472+0000","last_clean":"2024-04-25T00:37:25.616472+0000","last_became_active":"2024-04-25T00:37:24.657945+0000","last_became_peered":"2024-04-25T00:37:24.657945+0000","last_unstale":"2024-04-25T00:37:25.616472+0000","last_undegraded":"2024-04-25T00:37:25.616472+0000","last_fullsized":"2024-04-25T00:37:25.616472+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:07.998140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616393+0000","last_change":"2024-04-25T00:37:23.974640+0000","last_active":"2024-04-25T00:37:25.616393+0000","last_peered":"2024-04-25T00:37:25.616393+0000","last_clean":"2024-04-25T00:37:25.616393+0000","last_became_active":"2024-04-25T00:37:23.974224+0000","last_became_peered":"2024-04-25T00:37:23.974224+0000","last_unstale":"2024-04-25T00:37:25.616393+0000","last_undegraded":"2024-04-25T00:37:25.616393+0000","last_fullsized":"2024-04-25T00:37:25.616393+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:20.520436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.892767+0000","last_change":"2024-04-25T00:37:24.665407+0000","last_active":"2024-04-25T00:37:25.892767+0000","last_peered":"2024-04-25T00:37:25.892767+0000","last_clean":"2024-04-25T00:37:25.892767+0000","last_became_active":"2024-04-25T00:37:24.664257+0000","last_became_peered":"2024-04-25T00:37:24.664257+0000","last_unstale":"2024-04-25T00:37:25.892767+0000","last_undegraded":"2024-04-25T00:37:25.892767+0000","last_fullsized":"2024-04-25T00:37:25.892767+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:23.934372+0000","last_clean_scrub_stamp":"2024-04-25T00:37:23.934372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:38.243062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.890107+0000","last_change":"2024-04-25T00:37:23.974436+0000","last_active":"2024-04-25T00:37:25.890107+0000","last_peered":"2024-04-25T00:37:25.890107+0000","last_clean":"2024-04-25T00:37:25.890107+0000","last_became_active":"2024-04-25T00:37:23.973775+0000","last_became_peered":"2024-04-25T00:37:23.973775+0000","last_unstale":"2024-04-25T00:37:25.890107+0000","last_undegraded":"2024-04-25T00:37:25.890107+0000","last_fullsized":"2024-04-25T00:37:25.890107+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:37:22.930729+0000","last_clean_scrub_stamp":"2024-04-25T00:37:22.930729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:09.024737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":368,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:37:25.616842+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:37:25.616842+0000","last_peered":"2024-04-25T00:37:25.616842+0000","last_clean":"2024-04-25T00:37:25.616842+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:37:25.616842+0000","last_undegraded":"2024-04-25T00:37:25.616842+0000","last_fullsized":"2024-04-25T00:37:25.616842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37192,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1884160,"data_stored":1153585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1286144,"data_stored":562671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":563205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1282048,"data_stored":562601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36608,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335232,"statfs":{"total":96636764160,"available":96599277568,"internally_reserved":0,"allocated":1298432,"data_stored":564283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36592,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335248,"statfs":{"total":96636764160,"available":96599293952,"internally_reserved":0,"allocated":1282048,"data_stored":562601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36600,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335240,"statfs":{"total":96636764160,"available":96599285760,"internally_reserved":0,"allocated":1290240,"data_stored":562725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":1154693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:37:33.823 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:37:33.823 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:37:33.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8bf42640 1 Processor -- start 2024-04-25T00:37:33.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8bf42640 1 -- start start 2024-04-25T00:37:33.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8bf42640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda840b4780 0x7fda8415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:33.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8bf42640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fda840cc240 0x7fda8415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:33.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8bf42640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda840b3720 con 0x7fda8415e8d0 2024-04-25T00:37:33.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8bf42640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fda840b42f0 con 0x7fda840cc240 2024-04-25T00:37:33.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8bf42640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fda840b39c0 con 0x7fda840b4780 2024-04-25T00:37:33.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8a4df640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fda840cc240 0x7fda8415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:33.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8a4df640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fda840cc240 0x7fda8415c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43708/0 (socket says 172.21.15.29:43708) 2024-04-25T00:37:33.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8a4df640 1 -- 172.21.15.29:0/26516393 learned_addr learned my addr 172.21.15.29:0/26516393 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:33.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda894dd640 1 --2- 172.21.15.29:0/26516393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda840b4780 0x7fda8415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:33.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8a4df640 1 -- 172.21.15.29:0/26516393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda840b4780 msgr2=0x7fda8415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:33.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8a4df640 1 --2- 172.21.15.29:0/26516393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda840b4780 0x7fda8415f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8a4df640 1 -- 172.21.15.29:0/26516393 >> v1:172.21.15.29:6789/0 conn(0x7fda8415e8d0 legacy=0x7fda8415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:33.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda88cdc640 1 -- 172.21.15.29:0/26516393 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 743303368 0 0) 0x7fda840b3720 con 0x7fda8415e8d0 2024-04-25T00:37:33.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.987+0000 7fda8a4df640 1 -- 172.21.15.29:0/26516393 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda840b3ca0 con 0x7fda840cc240 2024-04-25T00:37:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.991+0000 7fda8a4df640 1 --2- 172.21.15.29:0/26516393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda840cc240 0x7fda8415c7d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009e50 tx=0x7fda7c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df594b19415dae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.991+0000 7fda88cdc640 1 -- 172.21.15.29:0/26516393 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c0a57d0 con 0x7fda840cc240 2024-04-25T00:37:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.991+0000 7fda88cdc640 1 -- 172.21.15.29:0/26516393 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7c0a5970 con 0x7fda840cc240 2024-04-25T00:37:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.991+0000 7fda88cdc640 1 -- 172.21.15.29:0/26516393 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c0a5d40 con 0x7fda840cc240 2024-04-25T00:37:33.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.991+0000 7fda8bf42640 1 -- 172.21.15.29:0/26516393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda840cc240 msgr2=0x7fda8415c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:33.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.991+0000 7fda8bf42640 1 --2- 172.21.15.29:0/26516393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda840cc240 0x7fda8415c7d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009e50 tx=0x7fda7c01ca30 comp rx=0 tx=0).stop 2024-04-25T00:37:33.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.991+0000 7fda8bf42640 1 -- 172.21.15.29:0/26516393 shutdown_connections 2024-04-25T00:37:33.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.991+0000 7fda8bf42640 1 --2- 172.21.15.29:0/26516393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda840cc240 0x7fda8415c7d0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.991+0000 7fda8bf42640 1 --2- 172.21.15.29:0/26516393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda840b4780 0x7fda8415f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.991+0000 7fda8bf42640 1 -- 172.21.15.29:0/26516393 >> 172.21.15.29:0/26516393 conn(0x7fda840b1870 msgr2=0x7fda840b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:33.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.991+0000 7fda8bf42640 1 -- 172.21.15.29:0/26516393 shutdown_connections 2024-04-25T00:37:33.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.991+0000 7fda8bf42640 1 -- 172.21.15.29:0/26516393 wait complete. 2024-04-25T00:37:33.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda8bf42640 1 Processor -- start 2024-04-25T00:37:33.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda8bf42640 1 -- start start 2024-04-25T00:37:33.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda8bf42640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda840b4780 0x7fda8413d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:33.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda8bf42640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda840cc240 0x7fda840abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:33.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda8bf42640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda8415e8d0 0x7fda840ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:33.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda8bf42640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fda8415d6a0 con 0x7fda840b4780 2024-04-25T00:37:33.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda8bf42640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda8415d520 con 0x7fda840cc240 2024-04-25T00:37:33.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda8bf42640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fda84167c70 con 0x7fda8415e8d0 2024-04-25T00:37:33.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda89cde640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda840b4780 0x7fda8413d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:33.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda89cde640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda840b4780 0x7fda8413d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43716/0 (socket says 172.21.15.29:43716) 2024-04-25T00:37:33.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda89cde640 1 -- 172.21.15.29:0/1928845157 learned_addr learned my addr 172.21.15.29:0/1928845157 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:33.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda89cde640 1 -- 172.21.15.29:0/1928845157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda8415e8d0 msgr2=0x7fda840ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:33.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda89cde640 1 --2- 172.21.15.29:0/1928845157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda8415e8d0 0x7fda840ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda89cde640 1 -- 172.21.15.29:0/1928845157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda840cc240 msgr2=0x7fda840abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:33.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda89cde640 1 --2- 172.21.15.29:0/1928845157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda840cc240 0x7fda840abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:33.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda89cde640 1 -- 172.21.15.29:0/1928845157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda840af150 con 0x7fda840b4780 2024-04-25T00:37:33.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda89cde640 1 --2- 172.21.15.29:0/1928845157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda840b4780 0x7fda8413d9b0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fda8000c450 tx=0x7fda8000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:33.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda7a7fc640 1 -- 172.21.15.29:0/1928845157 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80017070 con 0x7fda840b4780 2024-04-25T00:37:33.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda7a7fc640 1 -- 172.21.15.29:0/1928845157 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda80003600 con 0x7fda840b4780 2024-04-25T00:37:33.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda7a7fc640 1 -- 172.21.15.29:0/1928845157 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda800039f0 con 0x7fda840b4780 2024-04-25T00:37:33.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda8bf42640 1 -- 172.21.15.29:0/1928845157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda84142110 con 0x7fda840b4780 2024-04-25T00:37:33.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda8bf42640 1 -- 172.21.15.29:0/1928845157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8413def0 con 0x7fda840b4780 2024-04-25T00:37:34.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda8bf42640 1 -- 172.21.15.29:0/1928845157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda84069270 con 0x7fda840b4780 2024-04-25T00:37:34.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda7a7fc640 1 -- 172.21.15.29:0/1928845157 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fda8001a070 con 0x7fda840b4780 2024-04-25T00:37:34.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda7a7fc640 1 --2- 172.21.15.29:0/1928845157 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fda680679e0 0x7fda68069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:34.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda894dd640 1 --2- 172.21.15.29:0/1928845157 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fda680679e0 0x7fda68069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:34.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.995+0000 7fda7a7fc640 1 -- 172.21.15.29:0/1928845157 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fda80085180 con 0x7fda840b4780 2024-04-25T00:37:34.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:33.999+0000 7fda894dd640 1 --2- 172.21.15.29:0/1928845157 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fda680679e0 0x7fda68069ea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fda84068aa0 tx=0x7fda74008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:34.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.003+0000 7fda7a7fc640 1 -- 172.21.15.29:0/1928845157 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fda80055c40 con 0x7fda840b4780 2024-04-25T00:37:34.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.223+0000 7fda8bf42640 1 -- 172.21.15.29:0/1928845157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fda8406e7b0 con 0x7fda840b4780 2024-04-25T00:37:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.227+0000 7fda7a7fc640 1 -- 172.21.15.29:0/1928845157 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7fda80055540 con 0x7fda840b4780 2024-04-25T00:37:34.229 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-04-25T00:37:34.230 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A== 2024-04-25T00:37:34.230 INFO:teuthology.orchestra.run.smithi029.stdout: caps mds = "allow" 2024-04-25T00:37:34.230 INFO:teuthology.orchestra.run.smithi029.stdout: caps mon = "allow r" 2024-04-25T00:37:34.230 INFO:teuthology.orchestra.run.smithi029.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:37:34.230 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-04-25T00:37:34.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.227+0000 7fda8bf42640 1 -- 172.21.15.29:0/1928845157 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fda680679e0 msgr2=0x7fda68069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:34.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.227+0000 7fda8bf42640 1 --2- 172.21.15.29:0/1928845157 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fda680679e0 0x7fda68069ea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fda84068aa0 tx=0x7fda74008040 comp rx=0 tx=0).stop 2024-04-25T00:37:34.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.227+0000 7fda8bf42640 1 -- 172.21.15.29:0/1928845157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda840b4780 msgr2=0x7fda8413d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:34.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.227+0000 7fda8bf42640 1 --2- 172.21.15.29:0/1928845157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda840b4780 0x7fda8413d9b0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fda8000c450 tx=0x7fda8000c910 comp rx=0 tx=0).stop 2024-04-25T00:37:34.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.227+0000 7fda8bf42640 1 -- 172.21.15.29:0/1928845157 shutdown_connections 2024-04-25T00:37:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.227+0000 7fda8bf42640 1 --2- 172.21.15.29:0/1928845157 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fda680679e0 0x7fda68069ea0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.227+0000 7fda8bf42640 1 --2- 172.21.15.29:0/1928845157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda8415e8d0 0x7fda840ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.227+0000 7fda8bf42640 1 --2- 172.21.15.29:0/1928845157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda840cc240 0x7fda840abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.227+0000 7fda8bf42640 1 --2- 172.21.15.29:0/1928845157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda840b4780 0x7fda8413d9b0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.231+0000 7fda8bf42640 1 -- 172.21.15.29:0/1928845157 >> 172.21.15.29:0/1928845157 conn(0x7fda840b1870 msgr2=0x7fda840af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.231+0000 7fda8bf42640 1 -- 172.21.15.29:0/1928845157 shutdown_connections 2024-04-25T00:37:34.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.231+0000 7fda8bf42640 1 -- 172.21.15.29:0/1928845157 wait complete. 2024-04-25T00:37:34.249 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:37:34.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.371+0000 7f498a5cd640 1 Processor -- start 2024-04-25T00:37:34.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.371+0000 7f498a5cd640 1 -- start start 2024-04-25T00:37:34.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f498a5cd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f498415e8d0 0x7f49840cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:34.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f498a5cd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49840b4780 0x7f498415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:34.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f498a5cd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f498415f1f0 0x7f498415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:34.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f498a5cd640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f49840b42f0 con 0x7f498415e8d0 2024-04-25T00:37:34.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f498a5cd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f49840b3720 con 0x7f498415f1f0 2024-04-25T00:37:34.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f498a5cd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f49840b39c0 con 0x7f49840b4780 2024-04-25T00:37:34.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4983fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f498415e8d0 0x7f49840cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:34.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4983fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f498415e8d0 0x7f49840cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43730/0 (socket says 172.21.15.29:43730) 2024-04-25T00:37:34.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4983fff640 1 -- 172.21.15.29:0/2264531921 learned_addr learned my addr 172.21.15.29:0/2264531921 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:34.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4988b6a640 1 --2- 172.21.15.29:0/2264531921 >> v2:172.21.15.157:3300/0 conn(0x7f498415f1f0 0x7f498415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:34.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f49837fe640 1 --2- 172.21.15.29:0/2264531921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49840b4780 0x7f498415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:34.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4983fff640 1 -- 172.21.15.29:0/2264531921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49840b4780 msgr2=0x7f498415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:34.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4983fff640 1 --2- 172.21.15.29:0/2264531921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49840b4780 0x7f498415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4983fff640 1 -- 172.21.15.29:0/2264531921 >> v2:172.21.15.157:3300/0 conn(0x7f498415f1f0 msgr2=0x7f498415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:34.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4983fff640 1 --2- 172.21.15.29:0/2264531921 >> v2:172.21.15.157:3300/0 conn(0x7f498415f1f0 0x7f498415b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4983fff640 1 -- 172.21.15.29:0/2264531921 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49840b3ca0 con 0x7f498415e8d0 2024-04-25T00:37:34.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4988b6a640 1 --2- 172.21.15.29:0/2264531921 >> v2:172.21.15.157:3300/0 conn(0x7f498415f1f0 0x7f498415b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:34.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f49837fe640 1 --2- 172.21.15.29:0/2264531921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49840b4780 0x7f498415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:34.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4983fff640 1 --2- 172.21.15.29:0/2264531921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f498415e8d0 0x7f49840cc240 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f49840708a0 tx=0x7f497401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8e563d214a3c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:34.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4982ffd640 1 -- 172.21.15.29:0/2264531921 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740aa5f0 con 0x7f498415e8d0 2024-04-25T00:37:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4982ffd640 1 -- 172.21.15.29:0/2264531921 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49740af070 con 0x7f498415e8d0 2024-04-25T00:37:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.375+0000 7f4982ffd640 1 -- 172.21.15.29:0/2264531921 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740aa9b0 con 0x7f498415e8d0 2024-04-25T00:37:34.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 -- 172.21.15.29:0/2264531921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f498415e8d0 msgr2=0x7f49840cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:34.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 --2- 172.21.15.29:0/2264531921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f498415e8d0 0x7f49840cc240 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f49840708a0 tx=0x7f497401c4f0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 -- 172.21.15.29:0/2264531921 shutdown_connections 2024-04-25T00:37:34.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 --2- 172.21.15.29:0/2264531921 >> v2:172.21.15.157:3300/0 conn(0x7f498415f1f0 0x7f498415b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 --2- 172.21.15.29:0/2264531921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49840b4780 0x7f498415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 --2- 172.21.15.29:0/2264531921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f498415e8d0 0x7f49840cc240 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 -- 172.21.15.29:0/2264531921 >> 172.21.15.29:0/2264531921 conn(0x7f49840b1870 msgr2=0x7f49840b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:34.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 -- 172.21.15.29:0/2264531921 shutdown_connections 2024-04-25T00:37:34.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 -- 172.21.15.29:0/2264531921 wait complete. 2024-04-25T00:37:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 Processor -- start 2024-04-25T00:37:34.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 -- start start 2024-04-25T00:37:34.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49840b4780 0x7f498413e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f498415f1f0 0x7f49841f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f498413b7e0 0x7f498406d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f498415f600 con 0x7f498413b7e0 2024-04-25T00:37:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f498415c7d0 con 0x7f498415f1f0 2024-04-25T00:37:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f498413a300 con 0x7f49840b4780 2024-04-25T00:37:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f4988b6a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f498413b7e0 0x7f498406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f4988b6a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f498413b7e0 0x7f498406d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43740/0 (socket says 172.21.15.29:43740) 2024-04-25T00:37:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f4988b6a640 1 -- 172.21.15.29:0/498407998 learned_addr learned my addr 172.21.15.29:0/498407998 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f49837fe640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f498415f1f0 0x7f49841f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f4983fff640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49840b4780 0x7f498413e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f4988b6a640 1 -- 172.21.15.29:0/498407998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49840b4780 msgr2=0x7f498413e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:34.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f4988b6a640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49840b4780 0x7f498413e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f4988b6a640 1 -- 172.21.15.29:0/498407998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f498415f1f0 msgr2=0x7f49841f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f4988b6a640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f498415f1f0 0x7f49841f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f4988b6a640 1 -- 172.21.15.29:0/498407998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4984139ef0 con 0x7f498413b7e0 2024-04-25T00:37:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f4988b6a640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f498413b7e0 0x7f498406d830 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f497c00cd50 tx=0x7f497c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f4980ff9640 1 -- 172.21.15.29:0/498407998 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c00f680 con 0x7f498413b7e0 2024-04-25T00:37:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f4980ff9640 1 -- 172.21.15.29:0/498407998 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497c00f840 con 0x7f498413b7e0 2024-04-25T00:37:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f4980ff9640 1 -- 172.21.15.29:0/498407998 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c00fc30 con 0x7f498413b7e0 2024-04-25T00:37:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 -- 172.21.15.29:0/498407998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49841f5830 con 0x7f498413b7e0 2024-04-25T00:37:34.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.379+0000 7f498a5cd640 1 -- 172.21.15.29:0/498407998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49841f5df0 con 0x7f498413b7e0 2024-04-25T00:37:34.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.383+0000 7f498a5cd640 1 -- 172.21.15.29:0/498407998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4984069270 con 0x7f498413b7e0 2024-04-25T00:37:34.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.383+0000 7f4980ff9640 1 -- 172.21.15.29:0/498407998 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f497c00e3e0 con 0x7f498413b7e0 2024-04-25T00:37:34.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.383+0000 7f4980ff9640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49700679e0 0x7f4970069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:34.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.383+0000 7f4980ff9640 1 -- 172.21.15.29:0/498407998 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f497c086d20 con 0x7f498413b7e0 2024-04-25T00:37:34.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.383+0000 7f4983fff640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49700679e0 0x7f4970069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:34.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.383+0000 7f4983fff640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49700679e0 0x7f4970069ea0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f49740ae000 tx=0x7f49740a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:34.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.387+0000 7f4980ff9640 1 -- 172.21.15.29:0/498407998 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f497c08b070 con 0x7f498413b7e0 2024-04-25T00:37:34.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.575+0000 7f498a5cd640 1 -- 172.21.15.29:0/498407998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f498415e100 con 0x7f498413b7e0 2024-04-25T00:37:34.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.575+0000 7f4980ff9640 1 -- 172.21.15.29:0/498407998 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5189 (secure 0 0 0) 0x7f497c014030 con 0x7f498413b7e0 2024-04-25T00:37:34.579 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:34.579 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1884979435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1884979435},{"type":"v1","addr":"172.21.15.157:6833","nonce":1884979435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1394307674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1394307674},{"type":"v1","addr":"172.21.15.157:6835","nonce":1394307674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/3915943091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3915943091},{"type":"v1","addr":"172.21.15.29:6834","nonce":3915943091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:37:24.327819+0000","modified":"2024-04-25T00:37:27.319963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1444171533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1444171533},{"type":"v1","addr":"172.21.15.29:6835","nonce":1444171533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7896,"qdb_cluster":[7896]},"id":8}]} 2024-04-25T00:37:34.580 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 67 2024-04-25T00:37:34.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.579+0000 7f498a5cd640 1 -- 172.21.15.29:0/498407998 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49700679e0 msgr2=0x7f4970069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:34.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.579+0000 7f498a5cd640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49700679e0 0x7f4970069ea0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f49740ae000 tx=0x7f49740a7000 comp rx=0 tx=0).stop 2024-04-25T00:37:34.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.579+0000 7f498a5cd640 1 -- 172.21.15.29:0/498407998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f498413b7e0 msgr2=0x7f498406d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:34.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.579+0000 7f498a5cd640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f498413b7e0 0x7f498406d830 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f497c00cd50 tx=0x7f497c00a650 comp rx=0 tx=0).stop 2024-04-25T00:37:34.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.579+0000 7f498a5cd640 1 -- 172.21.15.29:0/498407998 shutdown_connections 2024-04-25T00:37:34.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.579+0000 7f498a5cd640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f49700679e0 0x7f4970069ea0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.579+0000 7f498a5cd640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f498413b7e0 0x7f498406d830 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.579+0000 7f498a5cd640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f498415f1f0 0x7f49841f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.579+0000 7f498a5cd640 1 --2- 172.21.15.29:0/498407998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49840b4780 0x7f498413e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:34.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.579+0000 7f498a5cd640 1 -- 172.21.15.29:0/498407998 >> 172.21.15.29:0/498407998 conn(0x7f49840b1870 msgr2=0x7f49840b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:34.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.579+0000 7f498a5cd640 1 -- 172.21.15.29:0/498407998 shutdown_connections 2024-04-25T00:37:34.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:34.579+0000 7f498a5cd640 1 -- 172.21.15.29:0/498407998 wait complete. 2024-04-25T00:37:34.999 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:37:24.327819+0000', 'modified': '2024-04-25T00:37:27.319963+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7896}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7896': {'gid': 7896, 'name': 'a', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/1444171533', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 1444171533}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 1444171533}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7896, 'qdb_cluster': [7896]} 2024-04-25T00:37:35.000 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:37:35.000 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:37:35.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.131+0000 7ff3943bc640 1 Processor -- start 2024-04-25T00:37:35.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff3943bc640 1 -- start start 2024-04-25T00:37:35.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff3943bc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff38c15f3c0 0x7ff38c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff3943bc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff38c0b3720 con 0x7ff38c0b4780 2024-04-25T00:37:35.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff3943bc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff38c0b39c0 con 0x7ff38c15e8d0 2024-04-25T00:37:35.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff3943bc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff38c0b42f0 con 0x7ff38c15f3c0 2024-04-25T00:37:35.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391957640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff38c0b4780 0x7ff38c15c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58064/0 (socket says 172.21.15.29:58064) 2024-04-25T00:37:35.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391957640 1 -- 172.21.15.29:0/1671979393 learned_addr learned my addr 172.21.15.29:0/1671979393 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:35.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff392959640 1 --2- 172.21.15.29:0/1671979393 >> v2:172.21.15.157:3300/0 conn(0x7ff38c15f3c0 0x7ff38c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1158248583 0 0) 0x7ff38c0b3720 con 0x7ff38c0b4780 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff378003530 con 0x7ff38c0b4780 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 491940808 0 0) 0x7ff378003530 con 0x7ff38c0b4780 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 >> v2:172.21.15.157:3300/0 conn(0x7ff38c15f3c0 msgr2=0x7ff38c0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 --2- 172.21.15.29:0/1671979393 >> v2:172.21.15.157:3300/0 conn(0x7ff38c15f3c0 0x7ff38c0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 >> v1:172.21.15.157:6789/0 conn(0x7ff38c15e8d0 legacy=0x7ff38c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c0b3ca0 con 0x7ff38c0b4780 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ff37c001680 con 0x7ff38c0b4780 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 >> v1:172.21.15.29:6789/0 conn(0x7ff38c0b4780 legacy=0x7ff38c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 --2- 172.21.15.29:0/1671979393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3780039c0 0x7ff378003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 --2- 172.21.15.29:0/1671979393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff378004810 0x7ff378004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 --2- 172.21.15.29:0/1671979393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3780051d0 0x7ff37800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c0b42f0 con 0x7ff3780051d0 2024-04-25T00:37:35.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff37800c070 con 0x7ff378004810 2024-04-25T00:37:35.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff37800c270 con 0x7ff3780039c0 2024-04-25T00:37:35.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff392959640 1 --2- 172.21.15.29:0/1671979393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3780051d0 0x7ff37800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391957640 1 --2- 172.21.15.29:0/1671979393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff378004810 0x7ff378004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff392158640 1 --2- 172.21.15.29:0/1671979393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3780039c0 0x7ff378003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff392959640 1 -- 172.21.15.29:0/1671979393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3780039c0 msgr2=0x7ff378003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:35.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff392959640 1 --2- 172.21.15.29:0/1671979393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3780039c0 0x7ff378003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff392959640 1 -- 172.21.15.29:0/1671979393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff378004810 msgr2=0x7ff378004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:35.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff392959640 1 --2- 172.21.15.29:0/1671979393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff378004810 0x7ff378004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff392959640 1 -- 172.21.15.29:0/1671979393 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff378003530 con 0x7ff3780051d0 2024-04-25T00:37:35.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.135+0000 7ff391957640 1 --2- 172.21.15.29:0/1671979393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff378004810 0x7ff378004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:35.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff392959640 1 -- 172.21.15.29:0/1671979393 reap_dead start 2024-04-25T00:37:35.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff392959640 1 --2- 172.21.15.29:0/1671979393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3780051d0 0x7ff37800b9d0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff384009fc0 tx=0x7ff38401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d810042ea86926 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:35.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384009c10 con 0x7ff3780051d0 2024-04-25T00:37:35.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3840a5cd0 con 0x7ff3780051d0 2024-04-25T00:37:35.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff391156640 1 -- 172.21.15.29:0/1671979393 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384003be0 con 0x7ff3780051d0 2024-04-25T00:37:35.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 -- 172.21.15.29:0/1671979393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3780051d0 msgr2=0x7ff37800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 --2- 172.21.15.29:0/1671979393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3780051d0 0x7ff37800b9d0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff384009fc0 tx=0x7ff38401ccc0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 -- 172.21.15.29:0/1671979393 shutdown_connections 2024-04-25T00:37:35.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 --2- 172.21.15.29:0/1671979393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3780051d0 0x7ff37800b9d0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 -- 172.21.15.29:0/1671979393 >> 172.21.15.29:0/1671979393 conn(0x7ff38c0b1870 msgr2=0x7ff38c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:35.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 -- 172.21.15.29:0/1671979393 shutdown_connections 2024-04-25T00:37:35.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 -- 172.21.15.29:0/1671979393 wait complete. 2024-04-25T00:37:35.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 Processor -- start 2024-04-25T00:37:35.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 -- start start 2024-04-25T00:37:35.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3780051d0 0x7ff38c143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff38c0ac0c0 0x7ff38c0aa230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38c0a9490 0x7ff38c134d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c06cf10 con 0x7ff3780051d0 2024-04-25T00:37:35.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c0b3580 con 0x7ff38c0ac0c0 2024-04-25T00:37:35.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff3943bc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff38c144a20 con 0x7ff38c0a9490 2024-04-25T00:37:35.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff392158640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3780051d0 0x7ff38c143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff392158640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3780051d0 0x7ff38c143970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43760/0 (socket says 172.21.15.29:43760) 2024-04-25T00:37:35.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.139+0000 7ff392158640 1 -- 172.21.15.29:0/1325300749 learned_addr learned my addr 172.21.15.29:0/1325300749 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:35.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff392959640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38c0a9490 0x7ff38c134d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff391957640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff38c0ac0c0 0x7ff38c0aa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff392158640 1 -- 172.21.15.29:0/1325300749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38c0a9490 msgr2=0x7ff38c134d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff392158640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38c0a9490 0x7ff38c134d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff392158640 1 -- 172.21.15.29:0/1325300749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff38c0ac0c0 msgr2=0x7ff38c0aa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff392158640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff38c0ac0c0 0x7ff38c0aa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff392158640 1 -- 172.21.15.29:0/1325300749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c143f50 con 0x7ff3780051d0 2024-04-25T00:37:35.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff391957640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff38c0ac0c0 0x7ff38c0aa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:35.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff392158640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3780051d0 0x7ff38c143970 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7ff38800e4d0 tx=0x7ff38800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:35.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff382ffd640 1 -- 172.21.15.29:0/1325300749 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3880052e0 con 0x7ff3780051d0 2024-04-25T00:37:35.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff382ffd640 1 -- 172.21.15.29:0/1325300749 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff388018070 con 0x7ff3780051d0 2024-04-25T00:37:35.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff3943bc640 1 -- 172.21.15.29:0/1325300749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38c1371e0 con 0x7ff3780051d0 2024-04-25T00:37:35.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff382ffd640 1 -- 172.21.15.29:0/1325300749 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388003820 con 0x7ff3780051d0 2024-04-25T00:37:35.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff3943bc640 1 -- 172.21.15.29:0/1325300749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38c1377a0 con 0x7ff3780051d0 2024-04-25T00:37:35.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff382ffd640 1 -- 172.21.15.29:0/1325300749 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7ff3880039c0 con 0x7ff3780051d0 2024-04-25T00:37:35.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff382ffd640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff3700679e0 0x7ff370069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff382ffd640 1 -- 172.21.15.29:0/1325300749 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7ff388085b40 con 0x7ff3780051d0 2024-04-25T00:37:35.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff391957640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff3700679e0 0x7ff370069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff391957640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff3700679e0 0x7ff370069ea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff37c001210 tx=0x7ff37c000ee0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff397535000 1 Processor -- start 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.143+0000 7ff397535000 1 -- start start 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723f2350 0x55d2723f2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723a8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723a8fd0 0x55d2723a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d2721f25e0 con 0x55d2723f2c70 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d272179c50 con 0x55d2723f2350 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55d2721ee5a0 con 0x55d2723a8fd0 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff391957640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723a8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff391957640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723a8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43772/0 (socket says 172.21.15.29:43772) 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff391957640 1 -- 172.21.15.29:0/16687382 learned_addr learned my addr 172.21.15.29:0/16687382 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392959640 1 --2- 172.21.15.29:0/16687382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723a8fd0 0x55d2723a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392158640 1 --2- 172.21.15.29:0/16687382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723f2350 0x55d2723f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff391957640 1 -- 172.21.15.29:0/16687382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723a8fd0 msgr2=0x55d2723a93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff391957640 1 --2- 172.21.15.29:0/16687382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723a8fd0 0x55d2723a93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff391957640 1 -- 172.21.15.29:0/16687382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723f2350 msgr2=0x55d2723f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff391957640 1 --2- 172.21.15.29:0/16687382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723f2350 0x55d2723f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff391957640 1 -- 172.21.15.29:0/16687382 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d272177d10 con 0x55d2723f2c70 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392158640 1 --2- 172.21.15.29:0/16687382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723f2350 0x55d2723f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392959640 1 --2- 172.21.15.29:0/16687382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723a8fd0 0x55d2723a93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff391957640 1 --2- 172.21.15.29:0/16687382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723a8a90 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00fa00 tx=0x7ff37c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cb144fee19261b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff36ffff640 1 -- 172.21.15.29:0/16687382 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c010040 con 0x55d2723f2c70 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff36ffff640 1 -- 172.21.15.29:0/16687382 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff37c0169d0 con 0x55d2723f2c70 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff36ffff640 1 -- 172.21.15.29:0/16687382 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c016d80 con 0x55d2723f2c70 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 -- 172.21.15.29:0/16687382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 msgr2=0x55d2723a8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 --2- 172.21.15.29:0/16687382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723a8a90 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00fa00 tx=0x7ff37c012000 comp rx=0 tx=0).stop 2024-04-25T00:37:35.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 -- 172.21.15.29:0/16687382 shutdown_connections 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 --2- 172.21.15.29:0/16687382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723a8fd0 0x55d2723a93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 --2- 172.21.15.29:0/16687382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723a8a90 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 --2- 172.21.15.29:0/16687382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723f2350 0x55d2723f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 -- 172.21.15.29:0/16687382 >> 172.21.15.29:0/16687382 conn(0x55d2723a0b70 msgr2=0x55d2723a2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 -- 172.21.15.29:0/16687382 shutdown_connections 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 -- 172.21.15.29:0/16687382 wait complete. 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 -- start start 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 Processor -- start 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 0x55d2723bbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 0x55d2723bc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723bc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d2721ac520 con 0x55d2723f2c70 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d2721bb3c0 con 0x55d2723a8fd0 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55d2723a09f0 con 0x55d2723f2350 2024-04-25T00:37:35.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 10 client.0 _ll_register_callbacks cb 0x55d27238f5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:35.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392959640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723bc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392959640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723bc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43776/0 (socket says 172.21.15.29:43776) 2024-04-25T00:37:35.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392959640 1 -- 172.21.15.29:0/207550813 learned_addr learned my addr 172.21.15.29:0/207550813 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:35.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff391957640 1 --2- 172.21.15.29:0/207550813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 0x55d2723bc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392158640 1 --2- 172.21.15.29:0/207550813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 0x55d2723bbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392959640 1 -- 172.21.15.29:0/207550813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 msgr2=0x55d2723bc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:35.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392959640 1 --2- 172.21.15.29:0/207550813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 0x55d2723bc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392959640 1 -- 172.21.15.29:0/207550813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 msgr2=0x55d2723bbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:35.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392959640 1 --2- 172.21.15.29:0/207550813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 0x55d2723bbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392959640 1 -- 172.21.15.29:0/207550813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27218a7b0 con 0x55d2723f2c70 2024-04-25T00:37:35.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff392959640 1 --2- 172.21.15.29:0/207550813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723bc7f0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7ff38401c730 tx=0x7ff378005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154e5c95c207dcde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:35.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff36d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:37:35.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff36d7fa640 1 -- 172.21.15.29:0/207550813 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384009ca0 con 0x55d2723f2c70 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff397535000 1 -- 172.21.15.29:0/207550813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff37c0169d0 con 0x55d2723f2c70 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.147+0000 7ff36d7fa640 1 -- 172.21.15.29:0/207550813 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7ff3840b3040 con 0x55d2723f2c70 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff36d7fa640 1 -- 172.21.15.29:0/207550813 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7ff3840c3050 con 0x55d2723f2c70 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 10 client.8070 fetch_fsmap learned FSMap version 67 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 -- 172.21.15.29:0/207550813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55d272179ea0 con 0x55d2723f2c70 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff36d7fa640 1 -- 172.21.15.29:0/207550813 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff37c0169d0 con 0x55d2723f2c70 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff36d7fa640 1 -- 172.21.15.29:0/207550813 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7ff384003bb0 con 0x55d2723f2c70 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 10 client.8070 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 10 client.8070 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 20 client.8070 populate_metadata read hostname 'smithi029' 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 --2- 172.21.15.29:0/207550813 >> [v2:172.21.15.29:6833/1444171533,v1:172.21.15.29:6835/1444171533] conn(0x55d2724015f0 0x55d2724039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 4 client.8070 mds_command: new command op to 7896 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 -- 172.21.15.29:0/207550813 --> [v2:172.21.15.29:6833/1444171533,v1:172.21.15.29:6835/1444171533] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d27218dab0 con 0x55d2724015f0 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff392158640 1 --2- 172.21.15.29:0/207550813 >> [v2:172.21.15.29:6833/1444171533,v1:172.21.15.29:6835/1444171533] conn(0x55d2724015f0 0x55d2724039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff392158640 1 --2- 172.21.15.29:0/207550813 >> [v2:172.21.15.29:6833/1444171533,v1:172.21.15.29:6835/1444171533] conn(0x55d2724015f0 0x55d2724039e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fad3c865e0c7f2e8 server_cookie=7fcff10d5a7761c0 in_seq=0 out_seq=0 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff36d7fa640 10 client.8070 ms_handle_connect on v2:172.21.15.29:6833/1444171533 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff36d7fa640 1 -- 172.21.15.29:0/207550813 <== mds.0 v2:172.21.15.29:6833/1444171533 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d27218dab0 con 0x55d2724015f0 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff36d7fa640 10 client.8070 handle_command_reply: tid=0 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 client.8070 shutdown 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 -- 172.21.15.29:0/207550813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 msgr2=0x55d2723bc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 --2- 172.21.15.29:0/207550813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723bc7f0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7ff38401c730 tx=0x7ff378005840 comp rx=0 tx=0).stop 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 -- 172.21.15.29:0/207550813 shutdown_connections 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 --2- 172.21.15.29:0/207550813 >> [v2:172.21.15.29:6833/1444171533,v1:172.21.15.29:6835/1444171533] conn(0x55d2724015f0 0x55d2724039e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 --2- 172.21.15.29:0/207550813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723bc7f0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 --2- 172.21.15.29:0/207550813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 0x55d2723bc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff36d7fa640 0 client.8070 ms_handle_reset on v2:172.21.15.29:6833/1444171533 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 --2- 172.21.15.29:0/207550813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 0x55d2723bbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 -- 172.21.15.29:0/207550813 >> 172.21.15.29:0/207550813 conn(0x55d2723a0b70 msgr2=0x55d2723a9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 -- 172.21.15.29:0/207550813 shutdown_connections 2024-04-25T00:37:35.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 1 -- 172.21.15.29:0/207550813 wait complete. 2024-04-25T00:37:35.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.151+0000 7ff397535000 20 client.8070 trim_cache size 0 max 16384 2024-04-25T00:37:35.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 Processor -- start 2024-04-25T00:37:35.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 -- start start 2024-04-25T00:37:35.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 0x55d2723f0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 0x55d2723f06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723f0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1fc630 con 0x55d2723f2c70 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c16a4f0 con 0x55d2723a8fd0 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff38c1387f0 con 0x55d2723f2350 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff392959640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723f0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff392959640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723f0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43792/0 (socket says 172.21.15.29:43792) 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff392959640 1 -- 172.21.15.29:0/2995898943 learned_addr learned my addr 172.21.15.29:0/2995898943 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff392959640 1 -- 172.21.15.29:0/2995898943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 msgr2=0x55d2723f06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff392959640 1 --2- 172.21.15.29:0/2995898943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 0x55d2723f06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff392959640 1 -- 172.21.15.29:0/2995898943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 msgr2=0x55d2723f0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff392959640 1 --2- 172.21.15.29:0/2995898943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 0x55d2723f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff392959640 1 -- 172.21.15.29:0/2995898943 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2723f2aa0 con 0x55d2723f2c70 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff392959640 1 --2- 172.21.15.29:0/2995898943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723f0bf0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7ff3840c1040 tx=0x7ff3840b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4641c6e977ce4aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff36f7fe640 1 -- 172.21.15.29:0/2995898943 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384009c10 con 0x55d2723f2c70 2024-04-25T00:37:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff36f7fe640 1 -- 172.21.15.29:0/2995898943 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3840cb020 con 0x55d2723f2c70 2024-04-25T00:37:35.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff36f7fe640 1 -- 172.21.15.29:0/2995898943 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384003bb0 con 0x55d2723f2c70 2024-04-25T00:37:35.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 -- 172.21.15.29:0/2995898943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 msgr2=0x55d2723f0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 --2- 172.21.15.29:0/2995898943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723f0bf0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7ff3840c1040 tx=0x7ff3840b7000 comp rx=0 tx=0).stop 2024-04-25T00:37:35.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 -- 172.21.15.29:0/2995898943 shutdown_connections 2024-04-25T00:37:35.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 --2- 172.21.15.29:0/2995898943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d2723f0bf0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 --2- 172.21.15.29:0/2995898943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 0x55d2723f06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 --2- 172.21.15.29:0/2995898943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 0x55d2723f0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 -- 172.21.15.29:0/2995898943 >> 172.21.15.29:0/2995898943 conn(0x55d2723a0b70 msgr2=0x55d2723a2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:35.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 -- 172.21.15.29:0/2995898943 shutdown_connections 2024-04-25T00:37:35.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 -- 172.21.15.29:0/2995898943 wait complete. 2024-04-25T00:37:35.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 -- start start 2024-04-25T00:37:35.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 Processor -- start 2024-04-25T00:37:35.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 0x55d2724619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 0x55d272404a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d272404f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d2721f18f0 con 0x55d2723f2c70 2024-04-25T00:37:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c169ea0 con 0x55d2723a8fd0 2024-04-25T00:37:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55d272198810 con 0x55d2723f2350 2024-04-25T00:37:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff397535000 10 client.0 _ll_register_callbacks cb 0x7ff378004710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff392959640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d272404f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff392959640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d272404f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43808/0 (socket says 172.21.15.29:43808) 2024-04-25T00:37:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.171+0000 7ff392959640 1 -- 172.21.15.29:0/3913699878 learned_addr learned my addr 172.21.15.29:0/3913699878 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff392158640 1 --2- 172.21.15.29:0/3913699878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 0x55d2724619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff391957640 1 --2- 172.21.15.29:0/3913699878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 0x55d272404a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff392959640 1 -- 172.21.15.29:0/3913699878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 msgr2=0x55d272404a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff392959640 1 --2- 172.21.15.29:0/3913699878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 0x55d272404a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff392959640 1 -- 172.21.15.29:0/3913699878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 msgr2=0x55d2724619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff392959640 1 --2- 172.21.15.29:0/3913699878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 0x55d2724619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff392959640 1 -- 172.21.15.29:0/3913699878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27217ae80 con 0x55d2723f2c70 2024-04-25T00:37:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff392959640 1 --2- 172.21.15.29:0/3913699878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d272404f70 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7ff38401c730 tx=0x7ff38400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29d0df31f8cb95b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff36d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:37:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff36d7fa640 1 -- 172.21.15.29:0/3913699878 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3840d4070 con 0x55d2723f2c70 2024-04-25T00:37:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff36d7fa640 1 -- 172.21.15.29:0/3913699878 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7ff3840cc040 con 0x55d2723f2c70 2024-04-25T00:37:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 -- 172.21.15.29:0/3913699878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3840cb020 con 0x55d2723f2c70 2024-04-25T00:37:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff36d7fa640 1 -- 172.21.15.29:0/3913699878 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7ff3840de050 con 0x55d2723f2c70 2024-04-25T00:37:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 10 client.8076 fetch_fsmap learned FSMap version 67 2024-04-25T00:37:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 -- 172.21.15.29:0/3913699878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55d2721bf760 con 0x55d2723f2c70 2024-04-25T00:37:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff36d7fa640 1 -- 172.21.15.29:0/3913699878 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3840cb020 con 0x55d2723f2c70 2024-04-25T00:37:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff36d7fa640 1 -- 172.21.15.29:0/3913699878 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7ff3840041d0 con 0x55d2723f2c70 2024-04-25T00:37:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 10 client.8076 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T00:37:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 10 client.8076 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:37:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 20 client.8076 populate_metadata read hostname 'smithi029' 2024-04-25T00:37:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 --2- 172.21.15.29:0/3913699878 >> [v2:172.21.15.29:6833/1444171533,v1:172.21.15.29:6835/1444171533] conn(0x55d2724054b0 0x55d272405890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 4 client.8076 mds_command: new command op to 7896 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:37:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 -- 172.21.15.29:0/3913699878 --> [v2:172.21.15.29:6833/1444171533,v1:172.21.15.29:6835/1444171533] -- command(tid 0: {"prefix": "status"}) -- 0x55d2721ffb20 con 0x55d2724054b0 2024-04-25T00:37:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff392158640 1 --2- 172.21.15.29:0/3913699878 >> [v2:172.21.15.29:6833/1444171533,v1:172.21.15.29:6835/1444171533] conn(0x55d2724054b0 0x55d272405890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff36d7fa640 10 client.8076 ms_handle_connect on v2:172.21.15.29:6833/1444171533 2024-04-25T00:37:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff392158640 1 --2- 172.21.15.29:0/3913699878 >> [v2:172.21.15.29:6833/1444171533,v1:172.21.15.29:6835/1444171533] conn(0x55d2724054b0 0x55d272405890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74060e3a23098a05 server_cookie=3d733f03bccf1c in_seq=0 out_seq=0 2024-04-25T00:37:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff36d7fa640 1 -- 172.21.15.29:0/3913699878 <== mds.0 v2:172.21.15.29:6833/1444171533 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d2721ffb20 con 0x55d2724054b0 2024-04-25T00:37:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff36d7fa640 10 client.8076 handle_command_reply: tid=0 2024-04-25T00:37:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 client.8076 shutdown 2024-04-25T00:37:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 -- 172.21.15.29:0/3913699878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 msgr2=0x55d272404f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 --2- 172.21.15.29:0/3913699878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d272404f70 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7ff38401c730 tx=0x7ff38400b040 comp rx=0 tx=0).stop 2024-04-25T00:37:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 -- 172.21.15.29:0/3913699878 shutdown_connections 2024-04-25T00:37:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 --2- 172.21.15.29:0/3913699878 >> [v2:172.21.15.29:6833/1444171533,v1:172.21.15.29:6835/1444171533] conn(0x55d2724054b0 0x55d272405890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 --2- 172.21.15.29:0/3913699878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d2723f2c70 0x55d272404f70 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 --2- 172.21.15.29:0/3913699878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d2723f2350 0x55d272404a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 --2- 172.21.15.29:0/3913699878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2723a8fd0 0x55d2724619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff36d7fa640 0 client.8076 ms_handle_reset on v2:172.21.15.29:6833/1444171533 2024-04-25T00:37:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 -- 172.21.15.29:0/3913699878 >> 172.21.15.29:0/3913699878 conn(0x55d2723a0b70 msgr2=0x55d2723a37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 -- 172.21.15.29:0/3913699878 shutdown_connections 2024-04-25T00:37:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 1 -- 172.21.15.29:0/3913699878 wait complete. 2024-04-25T00:37:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff397535000 20 client.8076 trim_cache size 0 max 16384 2024-04-25T00:37:35.185 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:37:35.185 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "59afbe7d-aaa8-45e2-b468-69ee150cacea", 2024-04-25T00:37:35.185 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-04-25T00:37:35.185 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 7896, 2024-04-25T00:37:35.185 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-04-25T00:37:35.185 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-04-25T00:37:35.185 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-04-25T00:37:35.185 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 10.233708651000001, 2024-04-25T00:37:35.185 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 67, 2024-04-25T00:37:35.185 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 103, 2024-04-25T00:37:35.185 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 103, 2024-04-25T00:37:35.185 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 15.597823384 2024-04-25T00:37:35.185 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:37:35.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff3943bc640 1 -- 172.21.15.29:0/1325300749 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff3700679e0 msgr2=0x7ff370069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff3943bc640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff3700679e0 0x7ff370069ea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff37c001210 tx=0x7ff37c000ee0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff3943bc640 1 -- 172.21.15.29:0/1325300749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3780051d0 msgr2=0x7ff38c143970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff3943bc640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3780051d0 0x7ff38c143970 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7ff38800e4d0 tx=0x7ff38800e990 comp rx=0 tx=0).stop 2024-04-25T00:37:35.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff3943bc640 1 -- 172.21.15.29:0/1325300749 shutdown_connections 2024-04-25T00:37:35.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff3943bc640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff3700679e0 0x7ff370069ea0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff3943bc640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff38c0a9490 0x7ff38c134d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff3943bc640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff38c0ac0c0 0x7ff38c0aa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff3943bc640 1 --2- 172.21.15.29:0/1325300749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3780051d0 0x7ff38c143970 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff3943bc640 1 -- 172.21.15.29:0/1325300749 >> 172.21.15.29:0/1325300749 conn(0x7ff38c0b1870 msgr2=0x7ff38c15dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:35.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff3943bc640 1 -- 172.21.15.29:0/1325300749 shutdown_connections 2024-04-25T00:37:35.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.175+0000 7ff3943bc640 1 -- 172.21.15.29:0/1325300749 wait complete. 2024-04-25T00:37:35.195 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:37:35.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd9a55c5640 1 Processor -- start 2024-04-25T00:37:35.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd9a55c5640 1 -- start start 2024-04-25T00:37:35.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd9a55c5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd9a006c950 0x7fd9a006cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd9a55c5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a015e440 0x7fd9a00d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd9a55c5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9a00d0ff0 0x7fd9a00ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd9a55c5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a00cc3b0 con 0x7fd9a006c950 2024-04-25T00:37:35.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd9a55c5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a006d270 con 0x7fd9a00d0ff0 2024-04-25T00:37:35.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd9a55c5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a00b11d0 con 0x7fd9a015e440 2024-04-25T00:37:35.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd99effd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd9a006c950 0x7fd9a006cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd99effd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd9a006c950 0x7fd9a006cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43816/0 (socket says 172.21.15.29:43816) 2024-04-25T00:37:35.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd99effd640 1 -- 172.21.15.29:0/506001705 learned_addr learned my addr 172.21.15.29:0/506001705 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:35.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd99e7fc640 1 --2- 172.21.15.29:0/506001705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a015e440 0x7fd9a00d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd99f7fe640 1 --2- 172.21.15.29:0/506001705 >> v2:172.21.15.157:3300/0 conn(0x7fd9a00d0ff0 0x7fd9a00ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd99effd640 1 -- 172.21.15.29:0/506001705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a015e440 msgr2=0x7fd9a00d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:35.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd99effd640 1 --2- 172.21.15.29:0/506001705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a015e440 0x7fd9a00d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd99effd640 1 -- 172.21.15.29:0/506001705 >> v2:172.21.15.157:3300/0 conn(0x7fd9a00d0ff0 msgr2=0x7fd9a00ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:35.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd99effd640 1 --2- 172.21.15.29:0/506001705 >> v2:172.21.15.157:3300/0 conn(0x7fd9a00d0ff0 0x7fd9a00ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.307+0000 7fd99effd640 1 -- 172.21.15.29:0/506001705 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a00cc650 con 0x7fd9a006c950 2024-04-25T00:37:35.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd99effd640 1 --2- 172.21.15.29:0/506001705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a006c950 0x7fd9a006cd30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00d2e40 tx=0x7fd99001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425dc53e2b0b0898 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:35.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd99dffb640 1 -- 172.21.15.29:0/506001705 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9900b0070 con 0x7fd9a006c950 2024-04-25T00:37:35.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd99dffb640 1 -- 172.21.15.29:0/506001705 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9900ab600 con 0x7fd9a006c950 2024-04-25T00:37:35.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd99dffb640 1 -- 172.21.15.29:0/506001705 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9900ab9d0 con 0x7fd9a006c950 2024-04-25T00:37:35.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd9a55c5640 1 -- 172.21.15.29:0/506001705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a006c950 msgr2=0x7fd9a006cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd9a55c5640 1 --2- 172.21.15.29:0/506001705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a006c950 0x7fd9a006cd30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00d2e40 tx=0x7fd99001c260 comp rx=0 tx=0).stop 2024-04-25T00:37:35.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd9a55c5640 1 -- 172.21.15.29:0/506001705 shutdown_connections 2024-04-25T00:37:35.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd9a55c5640 1 --2- 172.21.15.29:0/506001705 >> v2:172.21.15.157:3300/0 conn(0x7fd9a00d0ff0 0x7fd9a00ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd9a55c5640 1 --2- 172.21.15.29:0/506001705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a015e440 0x7fd9a00d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd9a55c5640 1 --2- 172.21.15.29:0/506001705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a006c950 0x7fd9a006cd30 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd9a55c5640 1 -- 172.21.15.29:0/506001705 >> 172.21.15.29:0/506001705 conn(0x7fd9a00d1d60 msgr2=0x7fd9a00d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:35.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd9a55c5640 1 -- 172.21.15.29:0/506001705 shutdown_connections 2024-04-25T00:37:35.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd9a55c5640 1 -- 172.21.15.29:0/506001705 wait complete. 2024-04-25T00:37:35.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd9a55c5640 1 Processor -- start 2024-04-25T00:37:35.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.311+0000 7fd9a55c5640 1 -- start start 2024-04-25T00:37:35.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9a55c5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a006c950 0x7fd9a01582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9a55c5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a00d0ff0 0x7fd9a01522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9a55c5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a015e440 0x7fd9a00c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9a55c5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a00cd410 con 0x7fd9a015e440 2024-04-25T00:37:35.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9a55c5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a006c520 con 0x7fd9a006c950 2024-04-25T00:37:35.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9a55c5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a00cd590 con 0x7fd9a00d0ff0 2024-04-25T00:37:35.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd99f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a015e440 0x7fd9a00c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd99f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a015e440 0x7fd9a00c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43824/0 (socket says 172.21.15.29:43824) 2024-04-25T00:37:35.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd99f7fe640 1 -- 172.21.15.29:0/1250170146 learned_addr learned my addr 172.21.15.29:0/1250170146 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:35.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd99effd640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a006c950 0x7fd9a01582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd99e7fc640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a00d0ff0 0x7fd9a01522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd99f7fe640 1 -- 172.21.15.29:0/1250170146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a00d0ff0 msgr2=0x7fd9a01522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd99f7fe640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a00d0ff0 0x7fd9a01522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd99f7fe640 1 -- 172.21.15.29:0/1250170146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a006c950 msgr2=0x7fd9a01582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd99f7fe640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a006c950 0x7fd9a01582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd99f7fe640 1 -- 172.21.15.29:0/1250170146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a0158ad0 con 0x7fd9a015e440 2024-04-25T00:37:35.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd99f7fe640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a015e440 0x7fd9a00c8e30 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd99800d5f0 tx=0x7fd99800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:35.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9977fe640 1 -- 172.21.15.29:0/1250170146 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980125f0 con 0x7fd9a015e440 2024-04-25T00:37:35.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9977fe640 1 -- 172.21.15.29:0/1250170146 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd998019070 con 0x7fd9a015e440 2024-04-25T00:37:35.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9a55c5640 1 -- 172.21.15.29:0/1250170146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a0158db0 con 0x7fd9a015e440 2024-04-25T00:37:35.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9977fe640 1 -- 172.21.15.29:0/1250170146 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980129f0 con 0x7fd9a015e440 2024-04-25T00:37:35.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9a55c5640 1 -- 172.21.15.29:0/1250170146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a0159370 con 0x7fd9a015e440 2024-04-25T00:37:35.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9a55c5640 1 -- 172.21.15.29:0/1250170146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a0069270 con 0x7fd9a015e440 2024-04-25T00:37:35.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9977fe640 1 -- 172.21.15.29:0/1250170146 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fd998012b90 con 0x7fd9a015e440 2024-04-25T00:37:35.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9977fe640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9800679e0 0x7fd980069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:35.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd9977fe640 1 -- 172.21.15.29:0/1250170146 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fd998086bf0 con 0x7fd9a015e440 2024-04-25T00:37:35.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.315+0000 7fd99effd640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9800679e0 0x7fd980069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:35.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.319+0000 7fd99effd640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9800679e0 0x7fd980069ea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd9900ad000 tx=0x7fd9900a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:35.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.319+0000 7fd9977fe640 1 -- 172.21.15.29:0/1250170146 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd998057700 con 0x7fd9a015e440 2024-04-25T00:37:35.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.519+0000 7fd9a55c5640 1 -- 172.21.15.29:0/1250170146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd9a015ae30 con 0x7fd9a015e440 2024-04-25T00:37:35.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.519+0000 7fd9977fe640 1 -- 172.21.15.29:0/1250170146 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7fd998057000 con 0x7fd9a015e440 2024-04-25T00:37:35.522 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:37:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.519+0000 7fd9a55c5640 1 -- 172.21.15.29:0/1250170146 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9800679e0 msgr2=0x7fd980069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.519+0000 7fd9a55c5640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9800679e0 0x7fd980069ea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd9900ad000 tx=0x7fd9900a6000 comp rx=0 tx=0).stop 2024-04-25T00:37:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.519+0000 7fd9a55c5640 1 -- 172.21.15.29:0/1250170146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a015e440 msgr2=0x7fd9a00c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:35.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.519+0000 7fd9a55c5640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a015e440 0x7fd9a00c8e30 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd99800d5f0 tx=0x7fd99800dab0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.519+0000 7fd9a55c5640 1 -- 172.21.15.29:0/1250170146 shutdown_connections 2024-04-25T00:37:35.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.519+0000 7fd9a55c5640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9800679e0 0x7fd980069ea0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.519+0000 7fd9a55c5640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9a015e440 0x7fd9a00c8e30 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.519+0000 7fd9a55c5640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9a00d0ff0 0x7fd9a01522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.519+0000 7fd9a55c5640 1 --2- 172.21.15.29:0/1250170146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a006c950 0x7fd9a01582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:35.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.519+0000 7fd9a55c5640 1 -- 172.21.15.29:0/1250170146 >> 172.21.15.29:0/1250170146 conn(0x7fd9a00d1d60 msgr2=0x7fd9a00b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:35.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.523+0000 7fd9a55c5640 1 -- 172.21.15.29:0/1250170146 shutdown_connections 2024-04-25T00:37:35.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35.523+0000 7fd9a55c5640 1 -- 172.21.15.29:0/1250170146 wait complete. 2024-04-25T00:37:35.542 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:37:35.542 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:37:35.542 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-25T00:37:35.542 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:37:35.542 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:37:35.542 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:37:35.543 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:37:35.543 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:37:35.543 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:37:35.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:35.544 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-25T00:37:35.551 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:37:35.551 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:37:35.551 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:37:35.551 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:35.551 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:37:35.552 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:35.552 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:37:35.552 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:37:35.552 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-25T00:37:35.552 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:37:35.552 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:37:35.553 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-25T00:37:35.553 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:37:35.553 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:37:35.553 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-25T00:37:35.553 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:37:35.553 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42104sec preferred_lft 42104sec 2024-04-25T00:37:35.554 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-04-25T00:37:35.554 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:35.554 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:37:35.554 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:37:35.554 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-25T00:37:35.554 INFO:teuthology.orchestra.run.smithi029.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:37:35.555 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 02:42:28:a5:36:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:37:35.555 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:37:35.555 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:35.555 INFO:teuthology.orchestra.run.smithi029.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:37:35.555 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:37:35.555 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:37:35.556 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:35.556 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:37:35.556 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:35.556 INFO:teuthology.orchestra.run.smithi029.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:37:35.556 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:37:35.556 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:37:35.557 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:35.557 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:37:35.557 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:37:35.557 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:37:35.557 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:35.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:35.558 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:37:35.605 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-25T00:37:35.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:35.607 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-25T00:37:35.656 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-25T00:37:35.656 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:37:35.656 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T00:37:35.656 INFO:teuthology.orchestra.run.smithi029.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:37:35.657 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T00:37:35.657 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:37:35.658 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:37:35.658 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:37:35.658 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:37:35.658 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:37:35.659 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:37:35.659 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:37:35.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:37:35.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:37:35.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:35.786 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:37:35.791 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:37:35.791 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:37:35.792 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:37:35.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:35.792 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-04-25T00:37:35.848 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:37:35.848 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:37:35.848 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:37:35.897 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:37:35.897 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:37:35.897 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:37:35.944 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:37:36.002 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:37:36.049 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:37:36.049 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:37:36.050 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:37:36.050 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:37:36.050 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:37:36.050 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:37:36.050 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:37:36.050 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:37:36.050 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:37:36.050 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:37:36.050 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:37:36.050 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:37:36.050 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:37:36.050 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:37:36.050 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:37:36.051 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:37:36.052 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:37:36.052 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:37:36.052 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:37:36.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:36.053 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:37:36.101 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:37:36.101 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:37:36.145 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:37:36.161 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:37:36.209 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:37:36.209 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:37:36.209 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:37:36.209 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:37:36.209 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:37:36.209 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:37:36.210 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:37:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:37:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:37:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:37:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:37:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:37:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:37:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:37:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:37:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:37:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:37:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:37:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:37:36.211 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:37:36.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:36.212 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:37:36.231 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:2024-04-25T00:37:36.227+0000 7fb5f76c8580 -1 init, newargv = 0x55c95a683d20 newargc=15 2024-04-25T00:37:36.231 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[48075]: starting ceph client 2024-04-25T00:37:36.236 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[48075]: starting fuse 2024-04-25T00:37:37.337 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:37:37.354 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:37:37.403 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:37:37.403 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:37:37.404 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:37:37.404 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:37:37.404 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:37:37.404 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:37:37.404 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:37:37.404 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:37:37.404 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:37:37.404 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:37:37.404 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:37:37.404 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:37:37.404 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:37:37.404 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:37:37.405 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:37:37.406 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:37:37.406 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:37:37.406 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:37:37.406 INFO:teuthology.orchestra.run.smithi029.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:37:37.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:37.407 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:37:37.454 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:37:37.590 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:37:37.590 DEBUG:teuthology.orchestra.run.smithi029:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:37:37.590 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-04-25T00:37:37.590 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-04-25T00:37:37.590 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:37:37.591 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:37:37.592 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-04-25T00:37:37.592 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-04-25T00:37:37.592 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-04-25T00:37:37.592 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:37:37.592 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:37:37.592 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.0")) 2024-04-25T00:37:37.592 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-04-25T00:37:37.680 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.0.48075.asok 2024-04-25T00:37:37.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:37:37.692 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48075.asok 2024-04-25T00:37:37.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:37.693 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48075.asok status 2024-04-25T00:37:37.853 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:37:37.853 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-04-25T00:37:37.853 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:37:37.853 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:37:37.853 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "0", 2024-04-25T00:37:37.853 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-04-25T00:37:37.853 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:37:37.853 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "48075", 2024-04-25T00:37:37.853 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-04-25T00:37:37.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:37.854 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-04-25T00:37:37.854 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-04-25T00:37:37.854 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 8088, 2024-04-25T00:37:37.854 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-04-25T00:37:37.854 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-04-25T00:37:37.854 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-04-25T00:37:37.854 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 8088 2024-04-25T00:37:37.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:37.854 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:37:37.854 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:37:37.854 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:37:37.854 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 759121938 2024-04-25T00:37:37.854 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:37.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:37.855 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:37:37.855 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:37:37.855 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:37:37.855 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 759121938 2024-04-25T00:37:37.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:37.855 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.8088 192.168.0.1:0/759121938", 2024-04-25T00:37:37.855 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.1:0/759121938", 2024-04-25T00:37:37.855 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-04-25T00:37:37.855 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 67, 2024-04-25T00:37:37.855 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 103, 2024-04-25T00:37:37.855 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:37:37.855 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-04-25T00:37:37.856 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-04-25T00:37:37.856 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:37:37.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:37.867 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-04-25T00:37:37.867 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:37:37.873 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-04-25T00:37:37.873 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:37:37.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:37.873 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:37:37.929 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:37:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c57935640 1 Processor -- start 2024-04-25T00:37:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c57935640 1 -- start start 2024-04-25T00:37:38.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c57935640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c500b3140 0x7f4c5006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:38.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c57935640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4c500b4b10 0x7f4c500b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:38.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c57935640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c500b3a60 0x7f4c5015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:38.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c57935640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4c500b41e0 con 0x7f4c500b4b10 2024-04-25T00:37:38.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c57935640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4c5006d960 con 0x7f4c500b3140 2024-04-25T00:37:38.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c57935640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5006dc40 con 0x7f4c500b3a60 2024-04-25T00:37:38.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c54ed0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4c500b4b10 0x7f4c500b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:38.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c54ed0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4c500b4b10 0x7f4c500b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34788/0 (socket says 172.21.15.29:34788) 2024-04-25T00:37:38.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c54ed0640 1 -- 172.21.15.29:0/2930447296 learned_addr learned my addr 172.21.15.29:0/2930447296 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:38.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c55ed2640 1 --2- 172.21.15.29:0/2930447296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c500b3a60 0x7f4c5015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:38.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c556d1640 1 --2- 172.21.15.29:0/2930447296 >> v2:172.21.15.157:3300/0 conn(0x7f4c500b3140 0x7f4c5006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:38.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c54ed0640 1 -- 172.21.15.29:0/2930447296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c500b3a60 msgr2=0x7f4c5015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:38.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c54ed0640 1 --2- 172.21.15.29:0/2930447296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c500b3a60 0x7f4c5015c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:38.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c54ed0640 1 -- 172.21.15.29:0/2930447296 >> v2:172.21.15.157:3300/0 conn(0x7f4c500b3140 msgr2=0x7f4c5006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:38.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c54ed0640 1 --2- 172.21.15.29:0/2930447296 >> v2:172.21.15.157:3300/0 conn(0x7f4c500b3140 0x7f4c5006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:38.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.103+0000 7f4c54ed0640 1 -- 172.21.15.29:0/2930447296 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5006df20 con 0x7f4c500b4b10 2024-04-25T00:37:38.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c54ed0640 1 --2- 172.21.15.29:0/2930447296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c500b4b10 0x7f4c500b3520 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009870 tx=0x7f4c4001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea1b91efc7bb737 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:38.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c47fff640 1 -- 172.21.15.29:0/2930447296 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c400b0070 con 0x7f4c500b4b10 2024-04-25T00:37:38.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c47fff640 1 -- 172.21.15.29:0/2930447296 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c400ab600 con 0x7f4c500b4b10 2024-04-25T00:37:38.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c47fff640 1 -- 172.21.15.29:0/2930447296 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c400ab9d0 con 0x7f4c500b4b10 2024-04-25T00:37:38.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 -- 172.21.15.29:0/2930447296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c500b4b10 msgr2=0x7f4c500b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:38.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 --2- 172.21.15.29:0/2930447296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c500b4b10 0x7f4c500b3520 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009870 tx=0x7f4c4001c520 comp rx=0 tx=0).stop 2024-04-25T00:37:38.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 -- 172.21.15.29:0/2930447296 shutdown_connections 2024-04-25T00:37:38.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 --2- 172.21.15.29:0/2930447296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c500b4b10 0x7f4c500b3520 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:38.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 --2- 172.21.15.29:0/2930447296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c500b3a60 0x7f4c5015c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:38.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 --2- 172.21.15.29:0/2930447296 >> v2:172.21.15.157:3300/0 conn(0x7f4c500b3140 0x7f4c5006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:38.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 -- 172.21.15.29:0/2930447296 >> 172.21.15.29:0/2930447296 conn(0x7f4c500b1870 msgr2=0x7f4c500b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:38.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 -- 172.21.15.29:0/2930447296 shutdown_connections 2024-04-25T00:37:38.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 -- 172.21.15.29:0/2930447296 wait complete. 2024-04-25T00:37:38.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 Processor -- start 2024-04-25T00:37:38.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 -- start start 2024-04-25T00:37:38.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c500b3140 0x7f4c5013f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:38.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c500b3a60 0x7f4c501418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:38.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c500b4b10 0x7f4c50141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:38.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4c500ce3c0 con 0x7f4c500b3140 2024-04-25T00:37:38.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4c500ce240 con 0x7f4c500b3a60 2024-04-25T00:37:38.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.107+0000 7f4c57935640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4c500ce540 con 0x7f4c500b4b10 2024-04-25T00:37:38.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c556d1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c500b3140 0x7f4c5013f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:38.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c556d1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c500b3140 0x7f4c5013f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34792/0 (socket says 172.21.15.29:34792) 2024-04-25T00:37:38.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c556d1640 1 -- 172.21.15.29:0/4252890911 learned_addr learned my addr 172.21.15.29:0/4252890911 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:38.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c55ed2640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c500b4b10 0x7f4c50141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:38.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c54ed0640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c500b3a60 0x7f4c501418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:38.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c556d1640 1 -- 172.21.15.29:0/4252890911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c500b4b10 msgr2=0x7f4c50141e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:38.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c556d1640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c500b4b10 0x7f4c50141e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:38.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c556d1640 1 -- 172.21.15.29:0/4252890911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c500b3a60 msgr2=0x7f4c501418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:38.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c556d1640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c500b3a60 0x7f4c501418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:38.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c556d1640 1 -- 172.21.15.29:0/4252890911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c50154a80 con 0x7f4c500b3140 2024-04-25T00:37:38.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c556d1640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c500b3140 0x7f4c5013f000 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00d830 tx=0x7f4c4c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:38.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c45ffb640 1 -- 172.21.15.29:0/4252890911 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c018020 con 0x7f4c500b3140 2024-04-25T00:37:38.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c45ffb640 1 -- 172.21.15.29:0/4252890911 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c4c0096a0 con 0x7f4c500b3140 2024-04-25T00:37:38.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c57935640 1 -- 172.21.15.29:0/4252890911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c50135c50 con 0x7f4c500b3140 2024-04-25T00:37:38.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c45ffb640 1 -- 172.21.15.29:0/4252890911 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c009a50 con 0x7f4c500b3140 2024-04-25T00:37:38.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c57935640 1 -- 172.21.15.29:0/4252890911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5013d420 con 0x7f4c500b3140 2024-04-25T00:37:38.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c57935640 1 -- 172.21.15.29:0/4252890911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c50069270 con 0x7f4c500b3140 2024-04-25T00:37:38.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c45ffb640 1 -- 172.21.15.29:0/4252890911 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f4c4c01b020 con 0x7f4c500b3140 2024-04-25T00:37:38.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c45ffb640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4c34067a10 0x7f4c34069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:38.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c54ed0640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4c34067a10 0x7f4c34069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:38.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c45ffb640 1 -- 172.21.15.29:0/4252890911 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f4c4c086620 con 0x7f4c500b3140 2024-04-25T00:37:38.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.111+0000 7f4c54ed0640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4c34067a10 0x7f4c34069ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4c400ad000 tx=0x7f4c400a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:38.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.115+0000 7f4c45ffb640 1 -- 172.21.15.29:0/4252890911 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4c4c022020 con 0x7f4c500b3140 2024-04-25T00:37:38.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c57935640 1 -- 172.21.15.29:0/4252890911 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4c50155420 con 0x7f4c500b3140 2024-04-25T00:37:38.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c45ffb640 1 -- 172.21.15.29:0/4252890911 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7f4c4c011340 con 0x7f4c500b3140 2024-04-25T00:37:38.317 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:37:38.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c57935640 1 -- 172.21.15.29:0/4252890911 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4c34067a10 msgr2=0x7f4c34069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:38.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c57935640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4c34067a10 0x7f4c34069ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4c400ad000 tx=0x7f4c400a6000 comp rx=0 tx=0).stop 2024-04-25T00:37:38.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c57935640 1 -- 172.21.15.29:0/4252890911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c500b3140 msgr2=0x7f4c5013f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:38.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c57935640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c500b3140 0x7f4c5013f000 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00d830 tx=0x7f4c4c00dcf0 comp rx=0 tx=0).stop 2024-04-25T00:37:38.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c57935640 1 -- 172.21.15.29:0/4252890911 shutdown_connections 2024-04-25T00:37:38.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c57935640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4c34067a10 0x7f4c34069ed0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:38.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c57935640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c500b4b10 0x7f4c50141e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:38.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c57935640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c500b3a60 0x7f4c501418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:38.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c57935640 1 --2- 172.21.15.29:0/4252890911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c500b3140 0x7f4c5013f000 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:38.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c57935640 1 -- 172.21.15.29:0/4252890911 >> 172.21.15.29:0/4252890911 conn(0x7f4c500b1870 msgr2=0x7f4c50140ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:38.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c57935640 1 -- 172.21.15.29:0/4252890911 shutdown_connections 2024-04-25T00:37:38.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:38.315+0000 7f4c57935640 1 -- 172.21.15.29:0/4252890911 wait complete. 2024-04-25T00:37:38.336 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:37:38.336 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:37:38.336 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-25T00:37:38.336 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:37:38.336 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:37:38.336 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:37:38.336 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:37:38.336 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:37:38.336 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:37:38.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:38.337 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-25T00:37:38.344 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:37:38.344 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:37:38.344 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:37:38.344 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:38.344 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:37:38.344 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:38.344 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:37:38.344 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:37:38.344 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-25T00:37:38.344 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:37:38.344 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42097sec preferred_lft 42097sec 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:9c:8d:4b:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:38.345 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:37:38.346 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:37:38.346 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:37:38.346 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:38.346 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:37:38.346 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:38.346 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:37:38.346 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:37:38.346 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:37:38.346 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:38.346 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:37:38.346 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:37:38.346 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:37:38.346 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:37:38.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:38.347 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:37:38.399 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:37:38.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:38.400 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-25T00:37:38.452 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-25T00:37:38.453 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:37:38.453 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:37:38.453 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:37:38.453 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:37:38.453 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:37:38.453 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:37:38.453 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:37:38.453 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:37:38.453 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:37:38.453 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:37:38.454 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:37:38.454 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:37:38.454 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:37:38.454 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:37:38.454 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:37:38.454 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:37:38.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:37:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:37:38.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:37:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:37:38.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:38.583 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:37:38.589 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:37:38.589 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:37:38.589 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:37:38.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:38.589 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-25T00:37:38.643 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T00:37:38.643 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:37:38.643 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:37:38.692 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:37:38.692 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:37:38.692 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:37:38.739 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:37:38.797 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:37:38.844 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:37:38.845 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:37:38.845 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:37:38.845 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:37:38.845 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:37:38.845 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:37:38.845 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:37:38.845 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:37:38.845 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:37:38.845 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:37:38.845 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:37:38.845 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:37:38.845 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:37:38.845 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:37:38.846 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:37:38.846 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:37:38.846 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:37:38.846 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:37:38.846 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:37:38.846 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:37:38.846 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:37:38.846 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:37:38.846 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:37:38.846 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:37:38.846 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:37:38.846 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:37:38.847 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:37:38.847 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:37:38.847 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:37:38.847 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:37:38.847 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:37:38.847 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:37:38.847 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:37:38.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:38.848 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:37:38.895 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:37:38.895 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T00:37:38.940 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:37:38.956 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:37:39.003 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:37:39.004 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:37:39.004 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:37:39.004 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:37:39.004 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:37:39.004 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:37:39.004 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:37:39.004 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:37:39.004 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:37:39.004 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:37:39.004 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:37:39.004 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:37:39.004 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:37:39.005 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:37:39.006 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:37:39.006 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:37:39.006 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:37:39.006 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:37:39.006 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:37:39.006 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:37:39.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:39.007 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:37:39.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-04-25T00:37:39.015+0000 7f7c0e976580 -1 init, newargv = 0x55bb41246140 newargc=15 2024-04-25T00:37:39.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34591]: starting ceph client 2024-04-25T00:37:39.025 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34591]: starting fuse 2024-04-25T00:37:40.121 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:37:40.137 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:37:40.184 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:37:40.184 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:37:40.185 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:37:40.186 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:37:40.186 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:37:40.186 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:37:40.186 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:37:40.186 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:37:40.186 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:37:40.186 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:37:40.186 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:37:40.186 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:37:40.186 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:37:40.186 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:37:40.186 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:37:40.186 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:37:40.187 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:37:40.187 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:37:40.187 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:37:40.187 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:37:40.187 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:37:40.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:40.188 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:37:40.235 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:37:40.236 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:37:40.236 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:37:40.236 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-25T00:37:40.236 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-25T00:37:40.236 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-25T00:37:40.236 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-25T00:37:40.236 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:37:40.236 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-25T00:37:40.236 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T00:37:40.236 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:37:40.237 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:37:40.238 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-04-25T00:37:40.238 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-25T00:37:40.360 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.34591.asok 2024-04-25T00:37:40.364 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:37:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:37:40.371 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34591.asok 2024-04-25T00:37:40.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:40.372 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34591.asok status 2024-04-25T00:37:40.523 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:37:40.523 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "34591", 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 6254, 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-25T00:37:40.524 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 6254 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 4115174468 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 4115174468 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.6254 192.168.0.1:0/4115174468", 2024-04-25T00:37:40.525 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/4115174468", 2024-04-25T00:37:40.526 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-25T00:37:40.526 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 67, 2024-04-25T00:37:40.526 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 103, 2024-04-25T00:37:40.526 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:37:40.526 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-25T00:37:40.526 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-25T00:37:40.526 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:37:40.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:40.538 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-25T00:37:40.538 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T00:37:40.545 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-25T00:37:40.545 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T00:37:40.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:40.545 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:37:40.597 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-25T00:37:40.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.735+0000 7f96c90cb640 1 Processor -- start 2024-04-25T00:37:40.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.735+0000 7f96c90cb640 1 -- start start 2024-04-25T00:37:40.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c90cb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c40b4590 0x7f96c4070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:40.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c90cb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f96c40cd1c0 0x7f96c4072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:40.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c90cb640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c40712c0 con 0x7f96c415e9a0 2024-04-25T00:37:40.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c90cb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f96c4071ed0 con 0x7f96c40cd1c0 2024-04-25T00:37:40.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c90cb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96c40715a0 con 0x7f96c40b4590 2024-04-25T00:37:40.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c2d9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f96c415e9a0 0x7f96c415ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53988/0 (socket says 172.21.15.29:53988) 2024-04-25T00:37:40.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c2d9d640 1 -- 172.21.15.29:0/1760722614 learned_addr learned my addr 172.21.15.29:0/1760722614 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:40.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c259c640 1 --2- 172.21.15.29:0/1760722614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c40b4590 0x7f96c4070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:40.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1916267839 0 0) 0x7f96c40712c0 con 0x7f96c415e9a0 2024-04-25T00:37:40.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c359e640 1 --2- 172.21.15.29:0/1760722614 >> v2:172.21.15.157:3300/0 conn(0x7f96c40cd1c0 0x7f96c4072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:40.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96ac003530 con 0x7f96c415e9a0 2024-04-25T00:37:40.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4423574 0 0) 0x7f96ac003530 con 0x7f96c415e9a0 2024-04-25T00:37:40.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c40b4590 msgr2=0x7f96c4070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:40.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 --2- 172.21.15.29:0/1760722614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c40b4590 0x7f96c4070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:40.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 >> v2:172.21.15.157:3300/0 conn(0x7f96c40cd1c0 msgr2=0x7f96c4072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:40.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 --2- 172.21.15.29:0/1760722614 >> v2:172.21.15.157:3300/0 conn(0x7f96c40cd1c0 0x7f96c4072360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:40.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c4071880 con 0x7f96c415e9a0 2024-04-25T00:37:40.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f96b4001680 con 0x7f96c415e9a0 2024-04-25T00:37:40.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 >> v1:172.21.15.29:6789/0 conn(0x7f96c415e9a0 legacy=0x7f96c415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:40.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 --2- 172.21.15.29:0/1760722614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96ac0039c0 0x7f96ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:40.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 --2- 172.21.15.29:0/1760722614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c40b4590 0x7f96ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:40.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 --2- 172.21.15.29:0/1760722614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96ac004d50 0x7f96ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:40.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f96c4071ed0 con 0x7f96ac004d50 2024-04-25T00:37:40.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96c40715a0 con 0x7f96ac0039c0 2024-04-25T00:37:40.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96ac00b9d0 con 0x7f96c40b4590 2024-04-25T00:37:40.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c359e640 1 --2- 172.21.15.29:0/1760722614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96ac004d50 0x7f96ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:40.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c259c640 1 --2- 172.21.15.29:0/1760722614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c40b4590 0x7f96ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:40.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c359e640 1 -- 172.21.15.29:0/1760722614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c40b4590 msgr2=0x7f96ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:40.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c359e640 1 --2- 172.21.15.29:0/1760722614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c40b4590 0x7f96ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:40.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c359e640 1 -- 172.21.15.29:0/1760722614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96ac0039c0 msgr2=0x7f96ac003de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:37:40.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c359e640 1 --2- 172.21.15.29:0/1760722614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96ac0039c0 0x7f96ac003de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:40.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.739+0000 7f96c359e640 1 -- 172.21.15.29:0/1760722614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac003530 con 0x7f96ac004d50 2024-04-25T00:37:40.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c359e640 1 --2- 172.21.15.29:0/1760722614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96ac004d50 0x7f96ac0051d0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009fc0 tx=0x7f96bc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e47069a39239f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:40.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc0b2070 con 0x7f96ac004d50 2024-04-25T00:37:40.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96bc0a5cd0 con 0x7f96ac004d50 2024-04-25T00:37:40.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c1d9b640 1 -- 172.21.15.29:0/1760722614 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc009c10 con 0x7f96ac004d50 2024-04-25T00:37:40.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 -- 172.21.15.29:0/1760722614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96ac004d50 msgr2=0x7f96ac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:40.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 --2- 172.21.15.29:0/1760722614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96ac004d50 0x7f96ac0051d0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009fc0 tx=0x7f96bc01ccc0 comp rx=0 tx=0).stop 2024-04-25T00:37:40.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c359e640 1 -- 172.21.15.29:0/1760722614 reap_dead start 2024-04-25T00:37:40.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 -- 172.21.15.29:0/1760722614 shutdown_connections 2024-04-25T00:37:40.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 -- 172.21.15.29:0/1760722614 >> 172.21.15.29:0/1760722614 conn(0x7f96c40b1870 msgr2=0x7f96c40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:40.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 -- 172.21.15.29:0/1760722614 shutdown_connections 2024-04-25T00:37:40.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 -- 172.21.15.29:0/1760722614 wait complete. 2024-04-25T00:37:40.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 Processor -- start 2024-04-25T00:37:40.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 -- start start 2024-04-25T00:37:40.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c4072410 0x7f96c4139930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:40.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96c41667f0 0x7f96c41304e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:40.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c40727f0 0x7f96c4130a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:40.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f96c40cd650 con 0x7f96c40727f0 2024-04-25T00:37:40.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96c415f700 con 0x7f96c41667f0 2024-04-25T00:37:40.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c90cb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96c40cd7d0 con 0x7f96c4072410 2024-04-25T00:37:40.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c359e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c40727f0 0x7f96c4130a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:40.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c359e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c40727f0 0x7f96c4130a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34822/0 (socket says 172.21.15.29:34822) 2024-04-25T00:37:40.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c359e640 1 -- 172.21.15.29:0/3679651582 learned_addr learned my addr 172.21.15.29:0/3679651582 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:40.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c259c640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96c41667f0 0x7f96c41304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:40.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c2d9d640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c4072410 0x7f96c4139930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:40.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c359e640 1 -- 172.21.15.29:0/3679651582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c4072410 msgr2=0x7f96c4139930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:40.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c359e640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c4072410 0x7f96c4139930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:40.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c359e640 1 -- 172.21.15.29:0/3679651582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96c41667f0 msgr2=0x7f96c41304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:40.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c359e640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96c41667f0 0x7f96c41304e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:40.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c359e640 1 -- 172.21.15.29:0/3679651582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c412fd20 con 0x7f96c40727f0 2024-04-25T00:37:40.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.743+0000 7f96c359e640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c40727f0 0x7f96c4130a20 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0987f0 tx=0x7f96bc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:40.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.747+0000 7f96ab7fe640 1 -- 172.21.15.29:0/3679651582 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc0b2040 con 0x7f96c40727f0 2024-04-25T00:37:40.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.747+0000 7f96ab7fe640 1 -- 172.21.15.29:0/3679651582 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96bc0a5ad0 con 0x7f96c40727f0 2024-04-25T00:37:40.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.747+0000 7f96c90cb640 1 -- 172.21.15.29:0/3679651582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c412dff0 con 0x7f96c40727f0 2024-04-25T00:37:40.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.747+0000 7f96ab7fe640 1 -- 172.21.15.29:0/3679651582 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc009c10 con 0x7f96c40727f0 2024-04-25T00:37:40.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.747+0000 7f96c90cb640 1 -- 172.21.15.29:0/3679651582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c412e5b0 con 0x7f96c40727f0 2024-04-25T00:37:40.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.747+0000 7f96c90cb640 1 -- 172.21.15.29:0/3679651582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c412e7f0 con 0x7f96c40727f0 2024-04-25T00:37:40.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.747+0000 7f96ab7fe640 1 -- 172.21.15.29:0/3679651582 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f96bc0039a0 con 0x7f96c40727f0 2024-04-25T00:37:40.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.747+0000 7f96ab7fe640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f96b0067a10 0x7f96b0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:40.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.747+0000 7f96ab7fe640 1 -- 172.21.15.29:0/3679651582 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f96bc120be0 con 0x7f96c40727f0 2024-04-25T00:37:40.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.747+0000 7f96c2d9d640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f96b0067a10 0x7f96b0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:40.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.747+0000 7f96c2d9d640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f96b0067a10 0x7f96b0069ed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f96b4001870 tx=0x7f96b4001640 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:40.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.751+0000 7f96ab7fe640 1 -- 172.21.15.29:0/3679651582 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f96c412e7f0 con 0x7f96c40727f0 2024-04-25T00:37:40.755 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:37:40.757 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:37:40.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:40.947+0000 7f96c90cb640 1 -- 172.21.15.29:0/3679651582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f96c40692e0 con 0x7f96c40727f0 2024-04-25T00:37:41.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.427+0000 7f96ab7fe640 1 -- 172.21.15.29:0/3679651582 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v68) ==== 136+0+0 (secure 0 0 0) 0x7f96bc0169d0 con 0x7f96c40727f0 2024-04-25T00:37:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.431+0000 7f96c90cb640 1 -- 172.21.15.29:0/3679651582 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f96b0067a10 msgr2=0x7f96b0069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.431+0000 7f96c90cb640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f96b0067a10 0x7f96b0069ed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f96b4001870 tx=0x7f96b4001640 comp rx=0 tx=0).stop 2024-04-25T00:37:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.431+0000 7f96c90cb640 1 -- 172.21.15.29:0/3679651582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c40727f0 msgr2=0x7f96c4130a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.431+0000 7f96c90cb640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c40727f0 0x7f96c4130a20 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0987f0 tx=0x7f96bc009750 comp rx=0 tx=0).stop 2024-04-25T00:37:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.431+0000 7f96c90cb640 1 -- 172.21.15.29:0/3679651582 shutdown_connections 2024-04-25T00:37:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.431+0000 7f96c90cb640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f96b0067a10 0x7f96b0069ed0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.431+0000 7f96c90cb640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96c40727f0 0x7f96c4130a20 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.431+0000 7f96c90cb640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96c41667f0 0x7f96c41304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.431+0000 7f96c90cb640 1 --2- 172.21.15.29:0/3679651582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96c4072410 0x7f96c4139930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.431+0000 7f96c90cb640 1 -- 172.21.15.29:0/3679651582 >> 172.21.15.29:0/3679651582 conn(0x7f96c40b1870 msgr2=0x7f96c4136da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:41.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.431+0000 7f96c90cb640 1 -- 172.21.15.29:0/3679651582 shutdown_connections 2024-04-25T00:37:41.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.431+0000 7f96c90cb640 1 -- 172.21.15.29:0/3679651582 wait complete. 2024-04-25T00:37:41.456 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-25T00:37:41.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57d2d6d640 1 Processor -- start 2024-04-25T00:37:41.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57d2d6d640 1 -- start start 2024-04-25T00:37:41.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57d2d6d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57cc15e8d0 0x7f57cc0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:41.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57d2d6d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57cc0b4780 0x7f57cc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:41.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57d2d6d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57cc15f1f0 0x7f57cc15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:41.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57d2d6d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f57cc0b42f0 con 0x7f57cc0b4780 2024-04-25T00:37:41.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57d2d6d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f57cc0b3720 con 0x7f57cc15f1f0 2024-04-25T00:37:41.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57d2d6d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc0b39c0 con 0x7f57cc15e8d0 2024-04-25T00:37:41.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57cbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57cc0b4780 0x7f57cc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:41.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57cbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57cc0b4780 0x7f57cc15ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34824/0 (socket says 172.21.15.29:34824) 2024-04-25T00:37:41.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57cbfff640 1 -- 172.21.15.29:0/3871094976 learned_addr learned my addr 172.21.15.29:0/3871094976 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:41.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57d0b09640 1 --2- 172.21.15.29:0/3871094976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57cc15e8d0 0x7f57cc0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:41.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57d130a640 1 --2- 172.21.15.29:0/3871094976 >> v2:172.21.15.157:3300/0 conn(0x7f57cc15f1f0 0x7f57cc15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:41.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57cbfff640 1 -- 172.21.15.29:0/3871094976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57cc15e8d0 msgr2=0x7f57cc0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:41.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57cbfff640 1 --2- 172.21.15.29:0/3871094976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57cc15e8d0 0x7f57cc0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57cbfff640 1 -- 172.21.15.29:0/3871094976 >> v2:172.21.15.157:3300/0 conn(0x7f57cc15f1f0 msgr2=0x7f57cc15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:41.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57cbfff640 1 --2- 172.21.15.29:0/3871094976 >> v2:172.21.15.157:3300/0 conn(0x7f57cc15f1f0 0x7f57cc15b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.587+0000 7f57cbfff640 1 -- 172.21.15.29:0/3871094976 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc0b3ca0 con 0x7f57cc0b4780 2024-04-25T00:37:41.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57cbfff640 1 --2- 172.21.15.29:0/3871094976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57cc0b4780 0x7f57cc15ecb0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0708a0 tx=0x7f57c001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29df16cc4327a937 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57cb7fe640 1 -- 172.21.15.29:0/3871094976 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c00aa5f0 con 0x7f57cc0b4780 2024-04-25T00:37:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57cb7fe640 1 -- 172.21.15.29:0/3871094976 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c00af070 con 0x7f57cc0b4780 2024-04-25T00:37:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57cb7fe640 1 -- 172.21.15.29:0/3871094976 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c00aa9b0 con 0x7f57cc0b4780 2024-04-25T00:37:41.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 -- 172.21.15.29:0/3871094976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57cc0b4780 msgr2=0x7f57cc15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:41.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 --2- 172.21.15.29:0/3871094976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57cc0b4780 0x7f57cc15ecb0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0708a0 tx=0x7f57c001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 -- 172.21.15.29:0/3871094976 shutdown_connections 2024-04-25T00:37:41.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 --2- 172.21.15.29:0/3871094976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57cc0b4780 0x7f57cc15ecb0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 --2- 172.21.15.29:0/3871094976 >> v2:172.21.15.157:3300/0 conn(0x7f57cc15f1f0 0x7f57cc15b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 --2- 172.21.15.29:0/3871094976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57cc15e8d0 0x7f57cc0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 -- 172.21.15.29:0/3871094976 >> 172.21.15.29:0/3871094976 conn(0x7f57cc0b1870 msgr2=0x7f57cc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:41.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 -- 172.21.15.29:0/3871094976 shutdown_connections 2024-04-25T00:37:41.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 -- 172.21.15.29:0/3871094976 wait complete. 2024-04-25T00:37:41.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 Processor -- start 2024-04-25T00:37:41.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 -- start start 2024-04-25T00:37:41.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57cc0b4780 0x7f57cc15a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:41.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57cc15e8d0 0x7f57cc133570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:41.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57cc15f1f0 0x7f57cc133ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:41.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc15f600 con 0x7f57cc0b4780 2024-04-25T00:37:41.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc15c7d0 con 0x7f57cc15f1f0 2024-04-25T00:37:41.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.591+0000 7f57d2d6d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc13d0f0 con 0x7f57cc15e8d0 2024-04-25T00:37:41.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57d0b09640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57cc0b4780 0x7f57cc15a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:41.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57d0b09640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57cc0b4780 0x7f57cc15a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34834/0 (socket says 172.21.15.29:34834) 2024-04-25T00:37:41.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57d0b09640 1 -- 172.21.15.29:0/2741039277 learned_addr learned my addr 172.21.15.29:0/2741039277 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:41.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57d130a640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57cc15f1f0 0x7f57cc133ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:41.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57cbfff640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57cc15e8d0 0x7f57cc133570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:41.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57d0b09640 1 -- 172.21.15.29:0/2741039277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57cc15e8d0 msgr2=0x7f57cc133570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:41.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57d0b09640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57cc15e8d0 0x7f57cc133570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57d0b09640 1 -- 172.21.15.29:0/2741039277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57cc15f1f0 msgr2=0x7f57cc133ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:41.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57d0b09640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57cc15f1f0 0x7f57cc133ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57d0b09640 1 -- 172.21.15.29:0/2741039277 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc13d3d0 con 0x7f57cc0b4780 2024-04-25T00:37:41.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57cbfff640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57cc15e8d0 0x7f57cc133570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:41.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57d0b09640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57cc0b4780 0x7f57cc15a150 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00d800 tx=0x7f57bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:41.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57c97fa640 1 -- 172.21.15.29:0/2741039277 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc018020 con 0x7f57cc0b4780 2024-04-25T00:37:41.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57c97fa640 1 -- 172.21.15.29:0/2741039277 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57bc0096a0 con 0x7f57cc0b4780 2024-04-25T00:37:41.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57d2d6d640 1 -- 172.21.15.29:0/2741039277 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc13d6b0 con 0x7f57cc0b4780 2024-04-25T00:37:41.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57c97fa640 1 -- 172.21.15.29:0/2741039277 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc009a50 con 0x7f57cc0b4780 2024-04-25T00:37:41.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57d2d6d640 1 -- 172.21.15.29:0/2741039277 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc13dc70 con 0x7f57cc0b4780 2024-04-25T00:37:41.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57d2d6d640 1 -- 172.21.15.29:0/2741039277 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57cc069270 con 0x7f57cc0b4780 2024-04-25T00:37:41.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57c97fa640 1 -- 172.21.15.29:0/2741039277 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f57bc01b020 con 0x7f57cc0b4780 2024-04-25T00:37:41.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57c97fa640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f57b80679e0 0x7f57b8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:41.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57cbfff640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f57b80679e0 0x7f57b8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:41.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57c97fa640 1 -- 172.21.15.29:0/2741039277 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f57bc0863e0 con 0x7f57cc0b4780 2024-04-25T00:37:41.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.595+0000 7f57cbfff640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f57b80679e0 0x7f57b8069ea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f57c00ae000 tx=0x7f57c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:41.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.599+0000 7f57c97fa640 1 -- 172.21.15.29:0/2741039277 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f57bc022020 con 0x7f57cc0b4780 2024-04-25T00:37:41.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.787+0000 7f57d2d6d640 1 -- 172.21.15.29:0/2741039277 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f57cc06e7b0 con 0x7f57b80679e0 2024-04-25T00:37:41.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.787+0000 7f57c97fa640 1 -- 172.21.15.29:0/2741039277 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f57cc06e7b0 con 0x7f57b80679e0 2024-04-25T00:37:41.791 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:37:41.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:41.792 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-04-25T00:37:41.792 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:41.792 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:37:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.791+0000 7f57d2d6d640 1 -- 172.21.15.29:0/2741039277 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f57b80679e0 msgr2=0x7f57b8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.791+0000 7f57d2d6d640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f57b80679e0 0x7f57b8069ea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f57c00ae000 tx=0x7f57c0008040 comp rx=0 tx=0).stop 2024-04-25T00:37:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.791+0000 7f57d2d6d640 1 -- 172.21.15.29:0/2741039277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57cc0b4780 msgr2=0x7f57cc15a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.791+0000 7f57d2d6d640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57cc0b4780 0x7f57cc15a150 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00d800 tx=0x7f57bc00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.791+0000 7f57d2d6d640 1 -- 172.21.15.29:0/2741039277 shutdown_connections 2024-04-25T00:37:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.791+0000 7f57d2d6d640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f57b80679e0 0x7f57b8069ea0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.791+0000 7f57d2d6d640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57cc15f1f0 0x7f57cc133ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.791+0000 7f57d2d6d640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57cc15e8d0 0x7f57cc133570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.791+0000 7f57d2d6d640 1 --2- 172.21.15.29:0/2741039277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57cc0b4780 0x7f57cc15a150 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.791+0000 7f57d2d6d640 1 -- 172.21.15.29:0/2741039277 >> 172.21.15.29:0/2741039277 conn(0x7f57cc0b1870 msgr2=0x7f57cc0cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:41.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.791+0000 7f57d2d6d640 1 -- 172.21.15.29:0/2741039277 shutdown_connections 2024-04-25T00:37:41.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.791+0000 7f57d2d6d640 1 -- 172.21.15.29:0/2741039277 wait complete. 2024-04-25T00:37:41.811 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:37:41.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430efb9640 1 Processor -- start 2024-04-25T00:37:41.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430efb9640 1 -- start start 2024-04-25T00:37:41.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430efb9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f430815e200 0x7f43080d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:41.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430efb9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080d2c60 0x7f43080b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:41.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430efb9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43080cfc10 con 0x7f43080cf690 2024-04-25T00:37:41.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430efb9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f43080b38d0 con 0x7f430815e200 2024-04-25T00:37:41.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430efb9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f43080cfef0 con 0x7f43080d2c60 2024-04-25T00:37:41.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430cd55640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f430815e200 0x7f43080d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:41.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430d556640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080d2c60 0x7f43080b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:41.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f4307fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f43080cf690 0x7f43080d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:38244/0 (socket says 172.21.15.29:38244) 2024-04-25T00:37:41.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f4307fff640 1 -- 172.21.15.29:0/1142079519 learned_addr learned my addr 172.21.15.29:0/1142079519 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:41.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430d556640 1 -- 172.21.15.29:0/1142079519 >> v2:172.21.15.157:3300/0 conn(0x7f430815e200 msgr2=0x7f43080d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:41.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430d556640 1 --2- 172.21.15.29:0/1142079519 >> v2:172.21.15.157:3300/0 conn(0x7f430815e200 0x7f43080d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430d556640 1 -- 172.21.15.29:0/1142079519 >> v1:172.21.15.157:6789/0 conn(0x7f43080cf690 legacy=0x7f43080d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:41.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430d556640 1 -- 172.21.15.29:0/1142079519 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43080b3b20 con 0x7f43080d2c60 2024-04-25T00:37:41.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.939+0000 7f430cd55640 1 --2- 172.21.15.29:0/1142079519 >> v2:172.21.15.157:3300/0 conn(0x7f430815e200 0x7f43080d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:37:41.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.943+0000 7f430d556640 1 --2- 172.21.15.29:0/1142079519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080d2c60 0x7f43080b4320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4300009d90 tx=0x7f430001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e988ad94727cd2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:41.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.943+0000 7f43077fe640 1 -- 172.21.15.29:0/1142079519 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43000b2070 con 0x7f43080d2c60 2024-04-25T00:37:41.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.943+0000 7f43077fe640 1 -- 172.21.15.29:0/1142079519 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43000a5a20 con 0x7f43080d2c60 2024-04-25T00:37:41.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.943+0000 7f43077fe640 1 -- 172.21.15.29:0/1142079519 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43000a5df0 con 0x7f43080d2c60 2024-04-25T00:37:41.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.943+0000 7f430efb9640 1 -- 172.21.15.29:0/1142079519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080d2c60 msgr2=0x7f43080b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:41.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.943+0000 7f430efb9640 1 --2- 172.21.15.29:0/1142079519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080d2c60 0x7f43080b4320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4300009d90 tx=0x7f430001c940 comp rx=0 tx=0).stop 2024-04-25T00:37:41.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.943+0000 7f430efb9640 1 -- 172.21.15.29:0/1142079519 shutdown_connections 2024-04-25T00:37:41.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.943+0000 7f430efb9640 1 --2- 172.21.15.29:0/1142079519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43080d2c60 0x7f43080b4320 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.943+0000 7f430efb9640 1 --2- 172.21.15.29:0/1142079519 >> v2:172.21.15.157:3300/0 conn(0x7f430815e200 0x7f43080d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.943+0000 7f430efb9640 1 -- 172.21.15.29:0/1142079519 >> 172.21.15.29:0/1142079519 conn(0x7f430809d280 msgr2=0x7f43080d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:41.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.943+0000 7f430efb9640 1 -- 172.21.15.29:0/1142079519 shutdown_connections 2024-04-25T00:37:41.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.943+0000 7f430efb9640 1 -- 172.21.15.29:0/1142079519 wait complete. 2024-04-25T00:37:41.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430efb9640 1 Processor -- start 2024-04-25T00:37:41.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430efb9640 1 -- start start 2024-04-25T00:37:41.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430efb9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080cf690 0x7f43080cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:41.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430efb9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43080d2c60 0x7f43080ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:41.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430efb9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f430815e200 0x7f4308160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:41.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430efb9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f43080b51f0 con 0x7f43080cf690 2024-04-25T00:37:41.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430efb9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f43080b5070 con 0x7f43080d2c60 2024-04-25T00:37:41.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430efb9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f43080b5370 con 0x7f430815e200 2024-04-25T00:37:41.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430cd55640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080cf690 0x7f43080cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:41.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430cd55640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080cf690 0x7f43080cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34844/0 (socket says 172.21.15.29:34844) 2024-04-25T00:37:41.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430cd55640 1 -- 172.21.15.29:0/3467509704 learned_addr learned my addr 172.21.15.29:0/3467509704 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:41.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430d556640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f430815e200 0x7f4308160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f4307fff640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43080d2c60 0x7f43080ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430cd55640 1 -- 172.21.15.29:0/3467509704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f430815e200 msgr2=0x7f4308160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430cd55640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f430815e200 0x7f4308160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430cd55640 1 -- 172.21.15.29:0/3467509704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43080d2c60 msgr2=0x7f43080ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430cd55640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43080d2c60 0x7f43080ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430cd55640 1 -- 172.21.15.29:0/3467509704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43080b8110 con 0x7f43080cf690 2024-04-25T00:37:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430cd55640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080cf690 0x7f43080cb040 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f42f800e980 tx=0x7f42f800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f43057fa640 1 -- 172.21.15.29:0/3467509704 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8017070 con 0x7f43080cf690 2024-04-25T00:37:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f43057fa640 1 -- 172.21.15.29:0/3467509704 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f800ca00 con 0x7f43080cf690 2024-04-25T00:37:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f43057fa640 1 -- 172.21.15.29:0/3467509704 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f800cdd0 con 0x7f43080cf690 2024-04-25T00:37:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430efb9640 1 -- 172.21.15.29:0/3467509704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43080b83f0 con 0x7f43080cf690 2024-04-25T00:37:41.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430efb9640 1 -- 172.21.15.29:0/3467509704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43080cab00 con 0x7f43080cf690 2024-04-25T00:37:41.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.947+0000 7f430efb9640 1 -- 172.21.15.29:0/3467509704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4308069270 con 0x7f43080cf690 2024-04-25T00:37:41.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.951+0000 7f43057fa640 1 -- 172.21.15.29:0/3467509704 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f42f801a070 con 0x7f43080cf690 2024-04-25T00:37:41.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.951+0000 7f43057fa640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f42f40679e0 0x7f42f4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:41.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.951+0000 7f43057fa640 1 -- 172.21.15.29:0/3467509704 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f42f8086050 con 0x7f43080cf690 2024-04-25T00:37:41.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.951+0000 7f4307fff640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f42f40679e0 0x7f42f4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:41.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.951+0000 7f4307fff640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f42f40679e0 0x7f42f4069ea0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43081618e0 tx=0x7f42fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:41.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:41.955+0000 7f43057fa640 1 -- 172.21.15.29:0/3467509704 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f42f8056430 con 0x7f43080cf690 2024-04-25T00:37:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.115+0000 7f430efb9640 1 -- 172.21.15.29:0/3467509704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f430815da10 con 0x7f43080cf690 2024-04-25T00:37:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.119+0000 7f43057fa640 1 -- 172.21.15.29:0/3467509704 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f42f805a730 con 0x7f43080cf690 2024-04-25T00:37:42.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.119+0000 7f430efb9640 1 -- 172.21.15.29:0/3467509704 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f42f40679e0 msgr2=0x7f42f4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.119+0000 7f430efb9640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f42f40679e0 0x7f42f4069ea0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43081618e0 tx=0x7f42fc009040 comp rx=0 tx=0).stop 2024-04-25T00:37:42.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.119+0000 7f430efb9640 1 -- 172.21.15.29:0/3467509704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080cf690 msgr2=0x7f43080cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.119+0000 7f430efb9640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080cf690 0x7f43080cb040 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f42f800e980 tx=0x7f42f800ee40 comp rx=0 tx=0).stop 2024-04-25T00:37:42.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.119+0000 7f430efb9640 1 -- 172.21.15.29:0/3467509704 shutdown_connections 2024-04-25T00:37:42.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.119+0000 7f430efb9640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f42f40679e0 0x7f42f4069ea0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.119+0000 7f430efb9640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f430815e200 0x7f4308160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.119+0000 7f430efb9640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43080d2c60 0x7f43080ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.119+0000 7f430efb9640 1 --2- 172.21.15.29:0/3467509704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43080cf690 0x7f43080cb040 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.119+0000 7f430efb9640 1 -- 172.21.15.29:0/3467509704 >> 172.21.15.29:0/3467509704 conn(0x7f430809d280 msgr2=0x7f430806cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:42.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.119+0000 7f430efb9640 1 -- 172.21.15.29:0/3467509704 shutdown_connections 2024-04-25T00:37:42.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.119+0000 7f430efb9640 1 -- 172.21.15.29:0/3467509704 wait complete. 2024-04-25T00:37:42.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:42.140 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1) 2024-04-25T00:37:42.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:42.163 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/fake) 2024-04-25T00:37:42.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:42.227 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-25T00:37:42.275 INFO:teuthology.orchestra.run.smithi029.stdout:cephfs_data 2024-04-25T00:37:42.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:42.275 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1) 2024-04-25T00:37:42.333 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:37:42.334 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol6790/.meta 2024-04-25T00:37:42.396 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol6790 2024-04-25T00:37:42.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.575+0000 7f15050c2640 1 Processor -- start 2024-04-25T00:37:42.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.575+0000 7f15050c2640 1 -- start start 2024-04-25T00:37:42.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.575+0000 7f15050c2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f150015df70 0x7f15000b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.575+0000 7f15050c2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f150006d210 0x7f15000b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.575+0000 7f15050c2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f150006d980 con 0x7f15000b4b60 2024-04-25T00:37:42.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.575+0000 7f15050c2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f15000b4fa0 con 0x7f150015df70 2024-04-25T00:37:42.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.575+0000 7f15050c2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f150006dc60 con 0x7f150006d210 2024-04-25T00:37:42.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.575+0000 7f14fed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f150015df70 0x7f15000b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.575+0000 7f14fed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f150015df70 0x7f15000b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34852/0 (socket says 172.21.15.29:34852) 2024-04-25T00:37:42.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.575+0000 7f14fed9d640 1 -- 172.21.15.29:0/2048375964 learned_addr learned my addr 172.21.15.29:0/2048375964 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:42.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.575+0000 7f14fe59c640 1 --2- 172.21.15.29:0/2048375964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f150006d210 0x7f15000b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f14fed9d640 1 -- 172.21.15.29:0/2048375964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f150006d210 msgr2=0x7f15000b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:42.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f14fed9d640 1 --2- 172.21.15.29:0/2048375964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f150006d210 0x7f15000b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f14fed9d640 1 -- 172.21.15.29:0/2048375964 >> v1:172.21.15.29:6789/0 conn(0x7f15000b4b60 legacy=0x7f1500070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f14fed9d640 1 -- 172.21.15.29:0/2048375964 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150006df40 con 0x7f150015df70 2024-04-25T00:37:42.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f14fe59c640 1 --2- 172.21.15.29:0/2048375964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f150006d210 0x7f15000b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:42.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f14fed9d640 1 --2- 172.21.15.29:0/2048375964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f150015df70 0x7f15000b3e40 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009e50 tx=0x7f14f801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260d56dc534e59ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:42.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f14fdd9b640 1 -- 172.21.15.29:0/2048375964 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f80a56e0 con 0x7f150015df70 2024-04-25T00:37:42.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f14fdd9b640 1 -- 172.21.15.29:0/2048375964 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f80a5880 con 0x7f150015df70 2024-04-25T00:37:42.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f14fdd9b640 1 -- 172.21.15.29:0/2048375964 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f80a5c50 con 0x7f150015df70 2024-04-25T00:37:42.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f15050c2640 1 -- 172.21.15.29:0/2048375964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f150015df70 msgr2=0x7f15000b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f15050c2640 1 --2- 172.21.15.29:0/2048375964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f150015df70 0x7f15000b3e40 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009e50 tx=0x7f14f801c8c0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f15050c2640 1 -- 172.21.15.29:0/2048375964 shutdown_connections 2024-04-25T00:37:42.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f15050c2640 1 --2- 172.21.15.29:0/2048375964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f150006d210 0x7f15000b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f15050c2640 1 --2- 172.21.15.29:0/2048375964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f150015df70 0x7f15000b3e40 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f15050c2640 1 -- 172.21.15.29:0/2048375964 >> 172.21.15.29:0/2048375964 conn(0x7f15000b1870 msgr2=0x7f15000b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:42.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f15050c2640 1 -- 172.21.15.29:0/2048375964 shutdown_connections 2024-04-25T00:37:42.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.579+0000 7f15050c2640 1 -- 172.21.15.29:0/2048375964 wait complete. 2024-04-25T00:37:42.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f15050c2640 1 Processor -- start 2024-04-25T00:37:42.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f15050c2640 1 -- start start 2024-04-25T00:37:42.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f15050c2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f150006d210 0x7f15001576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f15050c2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15000b4b60 0x7f15000cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f15050c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150015df70 0x7f150014a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f15050c2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f150006d670 con 0x7f15000b4b60 2024-04-25T00:37:42.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f15050c2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f150006e500 con 0x7f150015df70 2024-04-25T00:37:42.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f15050c2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f150015e350 con 0x7f150006d210 2024-04-25T00:37:42.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f14fe59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15000b4b60 0x7f15000cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f14fe59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15000b4b60 0x7f15000cfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34858/0 (socket says 172.21.15.29:34858) 2024-04-25T00:37:42.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f14fe59c640 1 -- 172.21.15.29:0/3375559261 learned_addr learned my addr 172.21.15.29:0/3375559261 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:42.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f14ff59e640 1 --2- 172.21.15.29:0/3375559261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150015df70 0x7f150014a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f14fe59c640 1 -- 172.21.15.29:0/3375559261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f150006d210 msgr2=0x7f15001576b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:37:42.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f14fe59c640 1 --2- 172.21.15.29:0/3375559261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f150006d210 0x7f15001576b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f14fe59c640 1 -- 172.21.15.29:0/3375559261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150015df70 msgr2=0x7f150014a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f14fe59c640 1 --2- 172.21.15.29:0/3375559261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150015df70 0x7f150014a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f14fe59c640 1 -- 172.21.15.29:0/3375559261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15000cc210 con 0x7f15000b4b60 2024-04-25T00:37:42.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f14fe59c640 1 --2- 172.21.15.29:0/3375559261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15000b4b60 0x7f15000cfdd0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f14e800e810 tx=0x7f14e800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:42.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f14f77fe640 1 -- 172.21.15.29:0/3375559261 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8017070 con 0x7f15000b4b60 2024-04-25T00:37:42.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f14f77fe640 1 -- 172.21.15.29:0/3375559261 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e800c8d0 con 0x7f15000b4b60 2024-04-25T00:37:42.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f15050c2640 1 -- 172.21.15.29:0/3375559261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15000cc4f0 con 0x7f15000b4b60 2024-04-25T00:37:42.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f14f77fe640 1 -- 172.21.15.29:0/3375559261 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e800f040 con 0x7f15000b4b60 2024-04-25T00:37:42.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f15050c2640 1 -- 172.21.15.29:0/3375559261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15000ccab0 con 0x7f15000b4b60 2024-04-25T00:37:42.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.583+0000 7f15050c2640 1 -- 172.21.15.29:0/3375559261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1500069270 con 0x7f15000b4b60 2024-04-25T00:37:42.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.587+0000 7f14f77fe640 1 -- 172.21.15.29:0/3375559261 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f14e801a070 con 0x7f15000b4b60 2024-04-25T00:37:42.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.587+0000 7f14f77fe640 1 --2- 172.21.15.29:0/3375559261 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f14e0067a30 0x7f14e0069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.587+0000 7f14f77fe640 1 -- 172.21.15.29:0/3375559261 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f14e8085a70 con 0x7f15000b4b60 2024-04-25T00:37:42.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.587+0000 7f14fed9d640 1 --2- 172.21.15.29:0/3375559261 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f14e0067a30 0x7f14e0069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.587+0000 7f14fed9d640 1 --2- 172.21.15.29:0/3375559261 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f14e0067a30 0x7f14e0069ef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f14f80ae000 tx=0x7f14f80a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:42.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.591+0000 7f14f77fe640 1 -- 172.21.15.29:0/3375559261 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f14e8056470 con 0x7f15000b4b60 2024-04-25T00:37:42.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.779+0000 7f15050c2640 1 -- 172.21.15.29:0/3375559261 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol6790", "target": ["mon-mgr", ""]}) -- 0x7f15000753e0 con 0x7f14e0067a30 2024-04-25T00:37:42.790 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:42.787+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:42.790 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:42.787+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:42.790 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:42.787+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:42.791 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:42.787+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:42.791 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:42.787+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:42.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.879+0000 7f14f77fe640 1 -- 172.21.15.29:0/3375559261 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+66 (secure 0 0 0) 0x7f15000753e0 con 0x7f14e0067a30 2024-04-25T00:37:42.881 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1 2024-04-25T00:37:42.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.879+0000 7f15050c2640 1 -- 172.21.15.29:0/3375559261 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f14e0067a30 msgr2=0x7f14e0069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.879+0000 7f15050c2640 1 --2- 172.21.15.29:0/3375559261 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f14e0067a30 0x7f14e0069ef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f14f80ae000 tx=0x7f14f80a7070 comp rx=0 tx=0).stop 2024-04-25T00:37:42.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.879+0000 7f15050c2640 1 -- 172.21.15.29:0/3375559261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15000b4b60 msgr2=0x7f15000cfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.879+0000 7f15050c2640 1 --2- 172.21.15.29:0/3375559261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15000b4b60 0x7f15000cfdd0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f14e800e810 tx=0x7f14e800ecd0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.879+0000 7f15050c2640 1 -- 172.21.15.29:0/3375559261 shutdown_connections 2024-04-25T00:37:42.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.879+0000 7f15050c2640 1 --2- 172.21.15.29:0/3375559261 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f14e0067a30 0x7f14e0069ef0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.879+0000 7f15050c2640 1 --2- 172.21.15.29:0/3375559261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150015df70 0x7f150014a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.879+0000 7f15050c2640 1 --2- 172.21.15.29:0/3375559261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15000b4b60 0x7f15000cfdd0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.879+0000 7f15050c2640 1 --2- 172.21.15.29:0/3375559261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f150006d210 0x7f15001576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.879+0000 7f15050c2640 1 -- 172.21.15.29:0/3375559261 >> 172.21.15.29:0/3375559261 conn(0x7f15000b1870 msgr2=0x7f150015c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:42.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.879+0000 7f15050c2640 1 -- 172.21.15.29:0/3375559261 shutdown_connections 2024-04-25T00:37:42.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:42.879+0000 7f15050c2640 1 -- 172.21.15.29:0/3375559261 wait complete. 2024-04-25T00:37:42.899 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-04-25T00:37:43.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.011+0000 7f7be0821640 1 Processor -- start 2024-04-25T00:37:43.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.011+0000 7f7be0821640 1 -- start start 2024-04-25T00:37:43.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7be0821640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7bdc0cf690 0x7f7bdc0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7be0821640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bdc0d2ea0 con 0x7f7bdc0d2a90 2024-04-25T00:37:43.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7be0821640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bdc0cfc10 con 0x7f7bdc15e200 2024-04-25T00:37:43.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7be0821640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7bdc0b38d0 con 0x7f7bdc0cf690 2024-04-25T00:37:43.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7bd9d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7bdc0cf690 0x7f7bdc0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7bdad9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7bdc0d2a90 0x7f7bdc0b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54010/0 (socket says 172.21.15.29:54010) 2024-04-25T00:37:43.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7bdad9d640 1 -- 172.21.15.29:0/1103790888 learned_addr learned my addr 172.21.15.29:0/1103790888 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:43.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7bd9d9b640 1 -- 172.21.15.29:0/1103790888 >> v1:172.21.15.157:6789/0 conn(0x7f7bdc15e200 legacy=0x7f7bdc0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:43.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7bd9d9b640 1 -- 172.21.15.29:0/1103790888 >> v1:172.21.15.29:6789/0 conn(0x7f7bdc0d2a90 legacy=0x7f7bdc0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7bd9d9b640 1 -- 172.21.15.29:0/1103790888 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc0cfef0 con 0x7f7bdc0cf690 2024-04-25T00:37:43.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7bd959a640 1 -- 172.21.15.29:0/1103790888 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 316738184 0 0) 0x7f7bd40014a0 con 0x7f7bdc0d2a90 2024-04-25T00:37:43.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7bd9d9b640 1 --2- 172.21.15.29:0/1103790888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bdc0cf690 0x7f7bdc0d2550 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00097c0 tx=0x7f7bd001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2408b506df5ef56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7bd959a640 1 -- 172.21.15.29:0/1103790888 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd00b0070 con 0x7f7bdc0cf690 2024-04-25T00:37:43.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7bd959a640 1 -- 172.21.15.29:0/1103790888 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd0004230 con 0x7f7bdc0cf690 2024-04-25T00:37:43.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.015+0000 7f7bd959a640 1 -- 172.21.15.29:0/1103790888 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0003620 con 0x7f7bdc0cf690 2024-04-25T00:37:43.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 -- 172.21.15.29:0/1103790888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bdc0cf690 msgr2=0x7f7bdc0d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 --2- 172.21.15.29:0/1103790888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bdc0cf690 0x7f7bdc0d2550 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00097c0 tx=0x7f7bd001c460 comp rx=0 tx=0).stop 2024-04-25T00:37:43.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 -- 172.21.15.29:0/1103790888 shutdown_connections 2024-04-25T00:37:43.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 --2- 172.21.15.29:0/1103790888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bdc0cf690 0x7f7bdc0d2550 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 -- 172.21.15.29:0/1103790888 >> 172.21.15.29:0/1103790888 conn(0x7f7bdc09d280 msgr2=0x7f7bdc0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:43.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 -- 172.21.15.29:0/1103790888 shutdown_connections 2024-04-25T00:37:43.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 -- 172.21.15.29:0/1103790888 wait complete. 2024-04-25T00:37:43.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 Processor -- start 2024-04-25T00:37:43.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 -- start start 2024-04-25T00:37:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bdc0cf690 0x7f7bdc1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bdc0d2a90 0x7f7bdc155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bdc15e200 0x7f7bdc0b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc0b53c0 con 0x7f7bdc15e200 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc0b5240 con 0x7f7bdc0d2a90 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7bdc0b5540 con 0x7f7bdc0cf690 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bdad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bdc15e200 0x7f7bdc0b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bdad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bdc15e200 0x7f7bdc0b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34908/0 (socket says 172.21.15.29:34908) 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bdad9d640 1 -- 172.21.15.29:0/3737292952 learned_addr learned my addr 172.21.15.29:0/3737292952 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bda59c640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bdc0cf690 0x7f7bdc1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bd9d9b640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bdc0d2a90 0x7f7bdc155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bdad9d640 1 -- 172.21.15.29:0/3737292952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bdc0cf690 msgr2=0x7f7bdc1597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bdad9d640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bdc0cf690 0x7f7bdc1597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bdad9d640 1 -- 172.21.15.29:0/3737292952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bdc0d2a90 msgr2=0x7f7bdc155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bdad9d640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bdc0d2a90 0x7f7bdc155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bdad9d640 1 -- 172.21.15.29:0/3737292952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc1603b0 con 0x7f7bdc15e200 2024-04-25T00:37:43.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bdad9d640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bdc15e200 0x7f7bdc0b1850 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400c700 tx=0x7f7bd400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bcaffd640 1 -- 172.21.15.29:0/3737292952 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4018070 con 0x7f7bdc15e200 2024-04-25T00:37:43.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bcaffd640 1 -- 172.21.15.29:0/3737292952 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd400ce10 con 0x7f7bdc15e200 2024-04-25T00:37:43.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 -- 172.21.15.29:0/3737292952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bdc160690 con 0x7f7bdc15e200 2024-04-25T00:37:43.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7bcaffd640 1 -- 172.21.15.29:0/3737292952 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400a860 con 0x7f7bdc15e200 2024-04-25T00:37:43.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.019+0000 7f7be0821640 1 -- 172.21.15.29:0/3737292952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bdc160c50 con 0x7f7bdc15e200 2024-04-25T00:37:43.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.023+0000 7f7be0821640 1 -- 172.21.15.29:0/3737292952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bdc069270 con 0x7f7bdc15e200 2024-04-25T00:37:43.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.023+0000 7f7bcaffd640 1 -- 172.21.15.29:0/3737292952 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f7bd401b070 con 0x7f7bdc15e200 2024-04-25T00:37:43.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.023+0000 7f7bcaffd640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7bbc0679e0 0x7f7bbc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.023+0000 7f7bda59c640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7bbc0679e0 0x7f7bbc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.023+0000 7f7bcaffd640 1 -- 172.21.15.29:0/3737292952 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f7bd4085730 con 0x7f7bdc15e200 2024-04-25T00:37:43.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.023+0000 7f7bda59c640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7bbc0679e0 0x7f7bbc069ea0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc004e30 tx=0x7f7bcc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.027+0000 7f7bcaffd640 1 -- 172.21.15.29:0/3737292952 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7bdc0d2ea0 con 0x7f7bdc15e200 2024-04-25T00:37:43.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.211+0000 7f7be0821640 1 -- 172.21.15.29:0/3737292952 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f7bdc0b4570 con 0x7f7bbc0679e0 2024-04-25T00:37:43.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.219+0000 7f7bcaffd640 1 -- 172.21.15.29:0/3737292952 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+45 (secure 0 0 0) 0x7f7bdc0b4570 con 0x7f7bbc0679e0 2024-04-25T00:37:43.222 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:37:43.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:43.222 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "subvol6790" 2024-04-25T00:37:43.222 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:43.222 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:37:43.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.219+0000 7f7be0821640 1 -- 172.21.15.29:0/3737292952 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7bbc0679e0 msgr2=0x7f7bbc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.219+0000 7f7be0821640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7bbc0679e0 0x7f7bbc069ea0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc004e30 tx=0x7f7bcc009040 comp rx=0 tx=0).stop 2024-04-25T00:37:43.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.219+0000 7f7be0821640 1 -- 172.21.15.29:0/3737292952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bdc15e200 msgr2=0x7f7bdc0b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.219+0000 7f7be0821640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bdc15e200 0x7f7bdc0b1850 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400c700 tx=0x7f7bd400cbc0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.219+0000 7f7be0821640 1 -- 172.21.15.29:0/3737292952 shutdown_connections 2024-04-25T00:37:43.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.223+0000 7f7be0821640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7bbc0679e0 0x7f7bbc069ea0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.223+0000 7f7be0821640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bdc15e200 0x7f7bdc0b1850 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.223+0000 7f7be0821640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bdc0d2a90 0x7f7bdc155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.223+0000 7f7be0821640 1 --2- 172.21.15.29:0/3737292952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bdc0cf690 0x7f7bdc1597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.223+0000 7f7be0821640 1 -- 172.21.15.29:0/3737292952 >> 172.21.15.29:0/3737292952 conn(0x7f7bdc09d280 msgr2=0x7f7bdc0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:43.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.223+0000 7f7be0821640 1 -- 172.21.15.29:0/3737292952 shutdown_connections 2024-04-25T00:37:43.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.223+0000 7f7be0821640 1 -- 172.21.15.29:0/3737292952 wait complete. 2024-04-25T00:37:43.241 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol6790 2024-04-25T00:37:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f51939ed640 1 Processor -- start 2024-04-25T00:37:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f51939ed640 1 -- start start 2024-04-25T00:37:43.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f51939ed640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f518c15e200 0x7f518c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f51939ed640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518c0cf690 0x7f518c0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f51939ed640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f518c0cfc10 con 0x7f518c0d2a90 2024-04-25T00:37:43.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f51939ed640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f518c0d2ee0 con 0x7f518c15e200 2024-04-25T00:37:43.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f51939ed640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f518c0b38d0 con 0x7f518c0cf690 2024-04-25T00:37:43.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f5190f88640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518c0cf690 0x7f518c0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f5190f88640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518c0cf690 0x7f518c0d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:59844/0 (socket says 172.21.15.29:59844) 2024-04-25T00:37:43.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f5190f88640 1 -- 172.21.15.29:0/3873578857 learned_addr learned my addr 172.21.15.29:0/3873578857 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:43.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f5191789640 1 --2- 172.21.15.29:0/3873578857 >> v2:172.21.15.157:3300/0 conn(0x7f518c15e200 0x7f518c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f5190f88640 1 -- 172.21.15.29:0/3873578857 >> v2:172.21.15.157:3300/0 conn(0x7f518c15e200 msgr2=0x7f518c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:43.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f5190f88640 1 --2- 172.21.15.29:0/3873578857 >> v2:172.21.15.157:3300/0 conn(0x7f518c15e200 0x7f518c0d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f5190f88640 1 -- 172.21.15.29:0/3873578857 >> v1:172.21.15.157:6789/0 conn(0x7f518c0d2a90 legacy=0x7f518c0d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.363+0000 7f5190f88640 1 -- 172.21.15.29:0/3873578857 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c0cfef0 con 0x7f518c0cf690 2024-04-25T00:37:43.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.367+0000 7f5190f88640 1 --2- 172.21.15.29:0/3873578857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518c0cf690 0x7f518c0d2550 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f517c009870 tx=0x7f517c01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e34850b434372b4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.367+0000 7f5183fff640 1 -- 172.21.15.29:0/3873578857 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c0a95f0 con 0x7f518c0cf690 2024-04-25T00:37:43.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.367+0000 7f5183fff640 1 -- 172.21.15.29:0/3873578857 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f517c0a9790 con 0x7f518c0cf690 2024-04-25T00:37:43.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.367+0000 7f5183fff640 1 -- 172.21.15.29:0/3873578857 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c0a9b60 con 0x7f518c0cf690 2024-04-25T00:37:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.367+0000 7f51939ed640 1 -- 172.21.15.29:0/3873578857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518c0cf690 msgr2=0x7f518c0d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.367+0000 7f51939ed640 1 --2- 172.21.15.29:0/3873578857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518c0cf690 0x7f518c0d2550 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f517c009870 tx=0x7f517c01c530 comp rx=0 tx=0).stop 2024-04-25T00:37:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.367+0000 7f51939ed640 1 -- 172.21.15.29:0/3873578857 shutdown_connections 2024-04-25T00:37:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.367+0000 7f51939ed640 1 --2- 172.21.15.29:0/3873578857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518c0cf690 0x7f518c0d2550 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.367+0000 7f51939ed640 1 --2- 172.21.15.29:0/3873578857 >> v2:172.21.15.157:3300/0 conn(0x7f518c15e200 0x7f518c0d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.367+0000 7f51939ed640 1 -- 172.21.15.29:0/3873578857 >> 172.21.15.29:0/3873578857 conn(0x7f518c09d280 msgr2=0x7f518c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.367+0000 7f51939ed640 1 -- 172.21.15.29:0/3873578857 shutdown_connections 2024-04-25T00:37:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.367+0000 7f51939ed640 1 -- 172.21.15.29:0/3873578857 wait complete. 2024-04-25T00:37:43.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.367+0000 7f51939ed640 1 Processor -- start 2024-04-25T00:37:43.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f51939ed640 1 -- start start 2024-04-25T00:37:43.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f51939ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f518c0cf690 0x7f518c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f51939ed640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c0d2a90 0x7f518c154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f51939ed640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518c15e200 0x7f518c154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f51939ed640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f518c15e760 con 0x7f518c0cf690 2024-04-25T00:37:43.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f51939ed640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f518c15e5e0 con 0x7f518c0d2a90 2024-04-25T00:37:43.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f51939ed640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f518c16ae30 con 0x7f518c15e200 2024-04-25T00:37:43.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5191789640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f518c0cf690 0x7f518c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5191789640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f518c0cf690 0x7f518c06ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34914/0 (socket says 172.21.15.29:34914) 2024-04-25T00:37:43.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5191789640 1 -- 172.21.15.29:0/4544691 learned_addr learned my addr 172.21.15.29:0/4544691 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:43.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5190f88640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c0d2a90 0x7f518c154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5191f8a640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518c15e200 0x7f518c154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5191789640 1 -- 172.21.15.29:0/4544691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518c15e200 msgr2=0x7f518c154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5191789640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518c15e200 0x7f518c154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5191789640 1 -- 172.21.15.29:0/4544691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c0d2a90 msgr2=0x7f518c154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5191789640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c0d2a90 0x7f518c154980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5191789640 1 -- 172.21.15.29:0/4544691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c1580d0 con 0x7f518c0cf690 2024-04-25T00:37:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5191789640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f518c0cf690 0x7f518c06ccd0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f518800e930 tx=0x7f518800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5181ffb640 1 -- 172.21.15.29:0/4544691 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188017070 con 0x7f518c0cf690 2024-04-25T00:37:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5181ffb640 1 -- 172.21.15.29:0/4544691 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f518800ca00 con 0x7f518c0cf690 2024-04-25T00:37:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f51939ed640 1 -- 172.21.15.29:0/4544691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c1583b0 con 0x7f518c0cf690 2024-04-25T00:37:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5181ffb640 1 -- 172.21.15.29:0/4544691 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518800cdb0 con 0x7f518c0cf690 2024-04-25T00:37:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f51939ed640 1 -- 172.21.15.29:0/4544691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c06d210 con 0x7f518c0cf690 2024-04-25T00:37:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f51939ed640 1 -- 172.21.15.29:0/4544691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f518c069270 con 0x7f518c0cf690 2024-04-25T00:37:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5181ffb640 1 -- 172.21.15.29:0/4544691 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f518801a070 con 0x7f518c0cf690 2024-04-25T00:37:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5181ffb640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f51700679e0 0x7f5170069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5181ffb640 1 -- 172.21.15.29:0/4544691 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f5188086180 con 0x7f518c0cf690 2024-04-25T00:37:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.371+0000 7f5190f88640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f51700679e0 0x7f5170069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.375+0000 7f5190f88640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f51700679e0 0x7f5170069ea0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f517c0ae000 tx=0x7f517c0b3040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.379+0000 7f5181ffb640 1 -- 172.21.15.29:0/4544691 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5188056d00 con 0x7f518c0cf690 2024-04-25T00:37:43.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.567+0000 7f51939ed640 1 -- 172.21.15.29:0/4544691 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol6790", "target": ["mon-mgr", ""]}) -- 0x7f518c157e80 con 0x7f51700679e0 2024-04-25T00:37:43.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.587+0000 7f5181ffb640 1 -- 172.21.15.29:0/4544691 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+685 (secure 0 0 0) 0x7f518c157e80 con 0x7f51700679e0 2024-04-25T00:37:43.592 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:37:43.592 INFO:teuthology.orchestra.run.smithi029.stdout: "atime": "2024-04-25 00:37:42", 2024-04-25T00:37:43.592 INFO:teuthology.orchestra.run.smithi029.stdout: "bytes_pcent": "undefined", 2024-04-25T00:37:43.592 INFO:teuthology.orchestra.run.smithi029.stdout: "bytes_quota": "infinite", 2024-04-25T00:37:43.592 INFO:teuthology.orchestra.run.smithi029.stdout: "bytes_used": 0, 2024-04-25T00:37:43.592 INFO:teuthology.orchestra.run.smithi029.stdout: "created_at": "2024-04-25 00:37:42", 2024-04-25T00:37:43.592 INFO:teuthology.orchestra.run.smithi029.stdout: "ctime": "2024-04-25 00:37:42", 2024-04-25T00:37:43.592 INFO:teuthology.orchestra.run.smithi029.stdout: "data_pool": "cephfs_data", 2024-04-25T00:37:43.592 INFO:teuthology.orchestra.run.smithi029.stdout: "features": [ 2024-04-25T00:37:43.592 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot-clone", 2024-04-25T00:37:43.592 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot-autoprotect" 2024-04-25T00:37:43.592 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:37:43.592 INFO:teuthology.orchestra.run.smithi029.stdout: "flavor": 1, 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout: "gid": 0, 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": 16877, 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_addrs": [ 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout: "172.21.15.29:6789", 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout: "172.21.15.157:6789", 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout: "172.21.15.157:6790" 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout: "mtime": "2024-04-25 00:37:42", 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1", 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_namespace": "", 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "complete", 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "subvolume", 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout: "uid": 0 2024-04-25T00:37:43.593 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:37:43.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.591+0000 7f51939ed640 1 -- 172.21.15.29:0/4544691 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f51700679e0 msgr2=0x7f5170069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.591+0000 7f51939ed640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f51700679e0 0x7f5170069ea0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f517c0ae000 tx=0x7f517c0b3040 comp rx=0 tx=0).stop 2024-04-25T00:37:43.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.591+0000 7f51939ed640 1 -- 172.21.15.29:0/4544691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f518c0cf690 msgr2=0x7f518c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.591+0000 7f51939ed640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f518c0cf690 0x7f518c06ccd0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f518800e930 tx=0x7f518800edf0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.591+0000 7f51939ed640 1 -- 172.21.15.29:0/4544691 shutdown_connections 2024-04-25T00:37:43.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.591+0000 7f51939ed640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f51700679e0 0x7f5170069ea0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.591+0000 7f51939ed640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518c15e200 0x7f518c154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.591+0000 7f51939ed640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c0d2a90 0x7f518c154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.591+0000 7f51939ed640 1 --2- 172.21.15.29:0/4544691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f518c0cf690 0x7f518c06ccd0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.591+0000 7f51939ed640 1 -- 172.21.15.29:0/4544691 >> 172.21.15.29:0/4544691 conn(0x7f518c09d280 msgr2=0x7f518c157680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:43.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.591+0000 7f51939ed640 1 -- 172.21.15.29:0/4544691 shutdown_connections 2024-04-25T00:37:43.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.591+0000 7f51939ed640 1 -- 172.21.15.29:0/4544691 wait complete. 2024-04-25T00:37:43.611 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol6790 10485760 2024-04-25T00:37:43.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.727+0000 7f7590700640 1 Processor -- start 2024-04-25T00:37:43.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.727+0000 7f7590700640 1 -- start start 2024-04-25T00:37:43.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.727+0000 7f7590700640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758815e560 0x7f75880d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.727+0000 7f7590700640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f75880d2680 0x7f75880d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.727+0000 7f7590700640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75880d3050 con 0x7f758815e180 2024-04-25T00:37:43.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.727+0000 7f7590700640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f75880cfed0 con 0x7f75880d2680 2024-04-25T00:37:43.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.727+0000 7f7590700640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f75880b7100 con 0x7f758815e560 2024-04-25T00:37:43.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.727+0000 7f758ec9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f75880d2680 0x7f75880d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.727+0000 7f758ec9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f75880d2680 0x7f75880d2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34918/0 (socket says 172.21.15.29:34918) 2024-04-25T00:37:43.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.727+0000 7f758ec9d640 1 -- 172.21.15.29:0/3128611150 learned_addr learned my addr 172.21.15.29:0/3128611150 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:43.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f758dc9b640 1 --2- 172.21.15.29:0/3128611150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758815e560 0x7f75880d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f758ec9d640 1 -- 172.21.15.29:0/3128611150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758815e560 msgr2=0x7f75880d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:43.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f758ec9d640 1 --2- 172.21.15.29:0/3128611150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758815e560 0x7f75880d20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f758ec9d640 1 -- 172.21.15.29:0/3128611150 >> v1:172.21.15.157:6789/0 conn(0x7f758815e180 legacy=0x7f75880cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:43.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f758ec9d640 1 -- 172.21.15.29:0/3128611150 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75880b73e0 con 0x7f75880d2680 2024-04-25T00:37:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f758ec9d640 1 --2- 172.21.15.29:0/3128611150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75880d2680 0x7f75880d2b10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f7580009e80 tx=0x7f758001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645e04229f4c88ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f758d49a640 1 -- 172.21.15.29:0/3128611150 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75800b4070 con 0x7f75880d2680 2024-04-25T00:37:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f758d49a640 1 -- 172.21.15.29:0/3128611150 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75800af400 con 0x7f75880d2680 2024-04-25T00:37:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f758d49a640 1 -- 172.21.15.29:0/3128611150 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75800af7d0 con 0x7f75880d2680 2024-04-25T00:37:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f7590700640 1 -- 172.21.15.29:0/3128611150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75880d2680 msgr2=0x7f75880d2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f7590700640 1 --2- 172.21.15.29:0/3128611150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75880d2680 0x7f75880d2b10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f7580009e80 tx=0x7f758001f1c0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f7590700640 1 -- 172.21.15.29:0/3128611150 shutdown_connections 2024-04-25T00:37:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f7590700640 1 --2- 172.21.15.29:0/3128611150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75880d2680 0x7f75880d2b10 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f7590700640 1 --2- 172.21.15.29:0/3128611150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f758815e560 0x7f75880d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f7590700640 1 -- 172.21.15.29:0/3128611150 >> 172.21.15.29:0/3128611150 conn(0x7f75880d40a0 msgr2=0x7f75880d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f7590700640 1 -- 172.21.15.29:0/3128611150 shutdown_connections 2024-04-25T00:37:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.731+0000 7f7590700640 1 -- 172.21.15.29:0/3128611150 wait complete. 2024-04-25T00:37:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f7590700640 1 Processor -- start 2024-04-25T00:37:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f7590700640 1 -- start start 2024-04-25T00:37:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f7590700640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75880d2680 0x7f75881f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f7590700640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f758815e180 0x7f75881f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f7590700640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f758815e560 0x7f7588160030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f7590700640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f75880b7d60 con 0x7f758815e180 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f7590700640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f75880b7be0 con 0x7f758815e560 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f7590700640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f75880b7ee0 con 0x7f75880d2680 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758dc9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f758815e180 0x7f75881f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758dc9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f758815e180 0x7f75881f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34920/0 (socket says 172.21.15.29:34920) 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758dc9b640 1 -- 172.21.15.29:0/1614708990 learned_addr learned my addr 172.21.15.29:0/1614708990 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758ec9d640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f758815e560 0x7f7588160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758e49c640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75880d2680 0x7f75881f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758dc9b640 1 -- 172.21.15.29:0/1614708990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75880d2680 msgr2=0x7f75881f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758dc9b640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75880d2680 0x7f75881f4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758dc9b640 1 -- 172.21.15.29:0/1614708990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f758815e560 msgr2=0x7f7588160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758dc9b640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f758815e560 0x7f7588160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758dc9b640 1 -- 172.21.15.29:0/1614708990 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75880c7730 con 0x7f758815e180 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758ec9d640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f758815e560 0x7f7588160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758e49c640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75880d2680 0x7f75881f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758dc9b640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f758815e180 0x7f75881f3420 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f757c00c7f0 tx=0x7f757c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f757affd640 1 -- 172.21.15.29:0/1614708990 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c015070 con 0x7f758815e180 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f757affd640 1 -- 172.21.15.29:0/1614708990 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c007820 con 0x7f758815e180 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f7590700640 1 -- 172.21.15.29:0/1614708990 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75880734b0 con 0x7f758815e180 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f757affd640 1 -- 172.21.15.29:0/1614708990 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c007c10 con 0x7f758815e180 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f7590700640 1 -- 172.21.15.29:0/1614708990 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75881f4b70 con 0x7f758815e180 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f7590700640 1 -- 172.21.15.29:0/1614708990 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75880692e0 con 0x7f758815e180 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f757affd640 1 -- 172.21.15.29:0/1614708990 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f757c018070 con 0x7f758815e180 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f757affd640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f756c0679e0 0x7f756c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f758e49c640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f756c0679e0 0x7f756c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.735+0000 7f757affd640 1 -- 172.21.15.29:0/1614708990 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f757c085f60 con 0x7f758815e180 2024-04-25T00:37:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.739+0000 7f758e49c640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f756c0679e0 0x7f756c069ea0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f75880d4bb0 tx=0x7f7584009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.743+0000 7f757affd640 1 -- 172.21.15.29:0/1614708990 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f757c056910 con 0x7f758815e180 2024-04-25T00:37:43.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.931+0000 7f7590700640 1 -- 172.21.15.29:0/1614708990 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol6790", "new_size": "10485760", "target": ["mon-mgr", ""]}) -- 0x7f7588067a90 con 0x7f756c0679e0 2024-04-25T00:37:43.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.959+0000 7f757affd640 1 -- 172.21.15.29:0/1614708990 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+128 (secure 0 0 0) 0x7f7588067a90 con 0x7f756c0679e0 2024-04-25T00:37:43.963 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:37:43.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:43.964 INFO:teuthology.orchestra.run.smithi029.stdout: "bytes_used": 0 2024-04-25T00:37:43.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:43.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:43.964 INFO:teuthology.orchestra.run.smithi029.stdout: "bytes_quota": 10485760 2024-04-25T00:37:43.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:37:43.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:37:43.964 INFO:teuthology.orchestra.run.smithi029.stdout: "bytes_pcent": "0.00" 2024-04-25T00:37:43.964 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:43.964 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:37:43.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.963+0000 7f7590700640 1 -- 172.21.15.29:0/1614708990 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f756c0679e0 msgr2=0x7f756c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.963+0000 7f7590700640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f756c0679e0 0x7f756c069ea0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f75880d4bb0 tx=0x7f7584009040 comp rx=0 tx=0).stop 2024-04-25T00:37:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.963+0000 7f7590700640 1 -- 172.21.15.29:0/1614708990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f758815e180 msgr2=0x7f75881f3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.963+0000 7f7590700640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f758815e180 0x7f75881f3420 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f757c00c7f0 tx=0x7f757c00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.963+0000 7f7590700640 1 -- 172.21.15.29:0/1614708990 shutdown_connections 2024-04-25T00:37:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.963+0000 7f7590700640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f756c0679e0 0x7f756c069ea0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.963+0000 7f7590700640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f758815e560 0x7f7588160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.963+0000 7f7590700640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f758815e180 0x7f75881f3420 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.963+0000 7f7590700640 1 --2- 172.21.15.29:0/1614708990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75880d2680 0x7f75881f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.963+0000 7f7590700640 1 -- 172.21.15.29:0/1614708990 >> 172.21.15.29:0/1614708990 conn(0x7f75880d40a0 msgr2=0x7f75880d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.963+0000 7f7590700640 1 -- 172.21.15.29:0/1614708990 shutdown_connections 2024-04-25T00:37:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:43.963+0000 7f7590700640 1 -- 172.21.15.29:0/1614708990 wait complete. 2024-04-25T00:37:43.983 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol6790 2024-04-25T00:37:44.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.095+0000 7f76b3016640 1 Processor -- start 2024-04-25T00:37:44.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76b3016640 1 -- start start 2024-04-25T00:37:44.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76b3016640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76ac1584c0 0x7f76ac0715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76b3016640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f76ac1588a0 0x7f76ac071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76b3016640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76ac15e930 con 0x7f76ac0b4060 2024-04-25T00:37:44.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76b3016640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f76ac06d330 con 0x7f76ac1588a0 2024-04-25T00:37:44.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76b3016640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac0ae3b0 con 0x7f76ac1584c0 2024-04-25T00:37:44.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76b15b3640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f76ac0b4060 0x7f76ac161be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54016/0 (socket says 172.21.15.29:54016) 2024-04-25T00:37:44.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76b15b3640 1 -- 172.21.15.29:0/3962344211 learned_addr learned my addr 172.21.15.29:0/3962344211 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:44.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76abfff640 1 --2- 172.21.15.29:0/3962344211 >> v2:172.21.15.29:3300/0 conn(0x7f76ac1588a0 0x7f76ac071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76b0db2640 1 --2- 172.21.15.29:0/3962344211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76ac1584c0 0x7f76ac0715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76ab7fe640 1 -- 172.21.15.29:0/3962344211 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2642156822 0 0) 0x7f76ac15e930 con 0x7f76ac0b4060 2024-04-25T00:37:44.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76ab7fe640 1 -- 172.21.15.29:0/3962344211 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7694003530 con 0x7f76ac0b4060 2024-04-25T00:37:44.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76abfff640 1 -- 172.21.15.29:0/3962344211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76ac1584c0 msgr2=0x7f76ac0715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:44.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76abfff640 1 --2- 172.21.15.29:0/3962344211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76ac1584c0 0x7f76ac0715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76abfff640 1 -- 172.21.15.29:0/3962344211 >> v1:172.21.15.29:6789/0 conn(0x7f76ac0b4060 legacy=0x7f76ac161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76abfff640 1 -- 172.21.15.29:0/3962344211 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac0b4c40 con 0x7f76ac1588a0 2024-04-25T00:37:44.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76abfff640 1 --2- 172.21.15.29:0/3962344211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76ac1588a0 0x7f76ac071af0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f76a00097c0 tx=0x7f76a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431423aeb215a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:44.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.099+0000 7f76ab7fe640 1 -- 172.21.15.29:0/3962344211 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00b2070 con 0x7f76ac1588a0 2024-04-25T00:37:44.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76ab7fe640 1 -- 172.21.15.29:0/3962344211 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a00ad640 con 0x7f76ac1588a0 2024-04-25T00:37:44.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76ab7fe640 1 -- 172.21.15.29:0/3962344211 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00ad9f0 con 0x7f76ac1588a0 2024-04-25T00:37:44.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 -- 172.21.15.29:0/3962344211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76ac1588a0 msgr2=0x7f76ac071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 --2- 172.21.15.29:0/3962344211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76ac1588a0 0x7f76ac071af0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f76a00097c0 tx=0x7f76a001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 -- 172.21.15.29:0/3962344211 shutdown_connections 2024-04-25T00:37:44.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 --2- 172.21.15.29:0/3962344211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76ac1588a0 0x7f76ac071af0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 --2- 172.21.15.29:0/3962344211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76ac1584c0 0x7f76ac0715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 -- 172.21.15.29:0/3962344211 >> 172.21.15.29:0/3962344211 conn(0x7f76ac075030 msgr2=0x7f76ac0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:44.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 -- 172.21.15.29:0/3962344211 shutdown_connections 2024-04-25T00:37:44.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 -- 172.21.15.29:0/3962344211 wait complete. 2024-04-25T00:37:44.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 Processor -- start 2024-04-25T00:37:44.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 -- start start 2024-04-25T00:37:44.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76ac0b4060 0x7f76ac140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76ac1584c0 0x7f76ac0b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76ac1588a0 0x7f76ac0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac0aff70 con 0x7f76ac1584c0 2024-04-25T00:37:44.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac0afdf0 con 0x7f76ac1588a0 2024-04-25T00:37:44.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac0b4850 con 0x7f76ac0b4060 2024-04-25T00:37:44.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76abfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76ac1584c0 0x7f76ac0b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76abfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76ac1584c0 0x7f76ac0b1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34930/0 (socket says 172.21.15.29:34930) 2024-04-25T00:37:44.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76abfff640 1 -- 172.21.15.29:0/517974498 learned_addr learned my addr 172.21.15.29:0/517974498 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:44.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76abfff640 1 -- 172.21.15.29:0/517974498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76ac0b4060 msgr2=0x7f76ac140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76abfff640 1 --2- 172.21.15.29:0/517974498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76ac0b4060 0x7f76ac140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76abfff640 1 -- 172.21.15.29:0/517974498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76ac1588a0 msgr2=0x7f76ac0b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76abfff640 1 --2- 172.21.15.29:0/517974498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76ac1588a0 0x7f76ac0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76abfff640 1 -- 172.21.15.29:0/517974498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac151980 con 0x7f76ac1584c0 2024-04-25T00:37:44.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76abfff640 1 --2- 172.21.15.29:0/517974498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76ac1584c0 0x7f76ac0b1140 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f76a001a0d0 tx=0x7f76a0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:44.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76a97fa640 1 -- 172.21.15.29:0/517974498 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00b2040 con 0x7f76ac1584c0 2024-04-25T00:37:44.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76a97fa640 1 -- 172.21.15.29:0/517974498 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a00ad440 con 0x7f76ac1584c0 2024-04-25T00:37:44.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 -- 172.21.15.29:0/517974498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ac151c60 con 0x7f76ac1584c0 2024-04-25T00:37:44.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76a97fa640 1 -- 172.21.15.29:0/517974498 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0006e20 con 0x7f76ac1584c0 2024-04-25T00:37:44.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.103+0000 7f76b3016640 1 -- 172.21.15.29:0/517974498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ac152220 con 0x7f76ac1584c0 2024-04-25T00:37:44.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.107+0000 7f76b3016640 1 -- 172.21.15.29:0/517974498 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ac069270 con 0x7f76ac1584c0 2024-04-25T00:37:44.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.107+0000 7f76a97fa640 1 -- 172.21.15.29:0/517974498 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f76a00be020 con 0x7f76ac1584c0 2024-04-25T00:37:44.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.107+0000 7f76a97fa640 1 --2- 172.21.15.29:0/517974498 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f76980679e0 0x7f7698069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.107+0000 7f76a97fa640 1 -- 172.21.15.29:0/517974498 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f76a011f650 con 0x7f76ac1584c0 2024-04-25T00:37:44.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.107+0000 7f76b0db2640 1 --2- 172.21.15.29:0/517974498 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f76980679e0 0x7f7698069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.107+0000 7f76b0db2640 1 --2- 172.21.15.29:0/517974498 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f76980679e0 0x7f7698069ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f769c006aa0 tx=0x7f769c006a30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:44.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.111+0000 7f76a97fa640 1 -- 172.21.15.29:0/517974498 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f76a0016090 con 0x7f76ac1584c0 2024-04-25T00:37:44.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.291+0000 7f76b3016640 1 -- 172.21.15.29:0/517974498 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol6790", "target": ["mon-mgr", ""]}) -- 0x7f76ac152a00 con 0x7f76980679e0 2024-04-25T00:37:44.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.307+0000 7f76a97fa640 1 -- 172.21.15.29:0/517974498 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+678 (secure 0 0 0) 0x7f769c0096e0 con 0x7f76980679e0 2024-04-25T00:37:44.310 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:37:44.310 INFO:teuthology.orchestra.run.smithi029.stdout: "atime": "2024-04-25 00:37:42", 2024-04-25T00:37:44.310 INFO:teuthology.orchestra.run.smithi029.stdout: "bytes_pcent": "0.00", 2024-04-25T00:37:44.310 INFO:teuthology.orchestra.run.smithi029.stdout: "bytes_quota": 10485760, 2024-04-25T00:37:44.310 INFO:teuthology.orchestra.run.smithi029.stdout: "bytes_used": 0, 2024-04-25T00:37:44.310 INFO:teuthology.orchestra.run.smithi029.stdout: "created_at": "2024-04-25 00:37:42", 2024-04-25T00:37:44.310 INFO:teuthology.orchestra.run.smithi029.stdout: "ctime": "2024-04-25 00:37:43", 2024-04-25T00:37:44.310 INFO:teuthology.orchestra.run.smithi029.stdout: "data_pool": "cephfs_data", 2024-04-25T00:37:44.310 INFO:teuthology.orchestra.run.smithi029.stdout: "features": [ 2024-04-25T00:37:44.310 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot-clone", 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot-autoprotect" 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "flavor": 1, 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "gid": 0, 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": 16877, 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_addrs": [ 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "172.21.15.29:6789", 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "172.21.15.157:6789", 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "172.21.15.157:6790" 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "mtime": "2024-04-25 00:37:42", 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1", 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_namespace": "", 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "complete", 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "subvolume", 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout: "uid": 0 2024-04-25T00:37:44.311 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:37:44.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.311+0000 7f76b3016640 1 -- 172.21.15.29:0/517974498 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f76980679e0 msgr2=0x7f7698069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.311+0000 7f76b3016640 1 --2- 172.21.15.29:0/517974498 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f76980679e0 0x7f7698069ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f769c006aa0 tx=0x7f769c006a30 comp rx=0 tx=0).stop 2024-04-25T00:37:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.311+0000 7f76b3016640 1 -- 172.21.15.29:0/517974498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76ac1584c0 msgr2=0x7f76ac0b1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.311+0000 7f76b3016640 1 --2- 172.21.15.29:0/517974498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76ac1584c0 0x7f76ac0b1140 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f76a001a0d0 tx=0x7f76a0019240 comp rx=0 tx=0).stop 2024-04-25T00:37:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.311+0000 7f76b3016640 1 -- 172.21.15.29:0/517974498 shutdown_connections 2024-04-25T00:37:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.311+0000 7f76b3016640 1 --2- 172.21.15.29:0/517974498 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f76980679e0 0x7f7698069ea0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.311+0000 7f76b3016640 1 --2- 172.21.15.29:0/517974498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76ac1588a0 0x7f76ac0b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.311+0000 7f76b3016640 1 --2- 172.21.15.29:0/517974498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76ac1584c0 0x7f76ac0b1140 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.311+0000 7f76b3016640 1 --2- 172.21.15.29:0/517974498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76ac0b4060 0x7f76ac140bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.311+0000 7f76b3016640 1 -- 172.21.15.29:0/517974498 >> 172.21.15.29:0/517974498 conn(0x7f76ac075030 msgr2=0x7f76ac06b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.311+0000 7f76b3016640 1 -- 172.21.15.29:0/517974498 shutdown_connections 2024-04-25T00:37:44.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.311+0000 7f76b3016640 1 -- 172.21.15.29:0/517974498 wait complete. 2024-04-25T00:37:44.327 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol6790 --mode 777 --uid 1000 --gid 1000 2024-04-25T00:37:44.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.459+0000 7fadd0c14640 1 Processor -- start 2024-04-25T00:37:44.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.459+0000 7fadd0c14640 1 -- start start 2024-04-25T00:37:44.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadd0c14640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fadcc15e240 0x7fadcc06b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadd0c14640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadcc0cb280 0x7fadcc06b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadd0c14640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadcc071df0 con 0x7fadcc0cca60 2024-04-25T00:37:44.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadd0c14640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fadcc0720d0 con 0x7fadcc15e240 2024-04-25T00:37:44.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadd0c14640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fadcc0723b0 con 0x7fadcc0cb280 2024-04-25T00:37:44.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadca59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fadcc15e240 0x7fadcc06b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadca59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fadcc15e240 0x7fadcc06b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34932/0 (socket says 172.21.15.29:34932) 2024-04-25T00:37:44.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadca59c640 1 -- 172.21.15.29:0/3881177010 learned_addr learned my addr 172.21.15.29:0/3881177010 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:44.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadc9d9b640 1 --2- 172.21.15.29:0/3881177010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadcc0cb280 0x7fadcc06b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadca59c640 1 -- 172.21.15.29:0/3881177010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadcc0cb280 msgr2=0x7fadcc06b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:44.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadca59c640 1 --2- 172.21.15.29:0/3881177010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadcc0cb280 0x7fadcc06b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadca59c640 1 -- 172.21.15.29:0/3881177010 >> v1:172.21.15.157:6789/0 conn(0x7fadcc0cca60 legacy=0x7fadcc0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadca59c640 1 -- 172.21.15.29:0/3881177010 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadcc072690 con 0x7fadcc15e240 2024-04-25T00:37:44.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadca59c640 1 --2- 172.21.15.29:0/3881177010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadcc15e240 0x7fadcc06b3f0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009870 tx=0x7fadb401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7aa90775f472fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:44.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadc959a640 1 -- 172.21.15.29:0/3881177010 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb40b5070 con 0x7fadcc15e240 2024-04-25T00:37:44.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadc959a640 1 -- 172.21.15.29:0/3881177010 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb40a8a50 con 0x7fadcc15e240 2024-04-25T00:37:44.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.463+0000 7fadc959a640 1 -- 172.21.15.29:0/3881177010 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb40a8e20 con 0x7fadcc15e240 2024-04-25T00:37:44.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 -- 172.21.15.29:0/3881177010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadcc15e240 msgr2=0x7fadcc06b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 --2- 172.21.15.29:0/3881177010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadcc15e240 0x7fadcc06b3f0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009870 tx=0x7fadb401ed50 comp rx=0 tx=0).stop 2024-04-25T00:37:44.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 -- 172.21.15.29:0/3881177010 shutdown_connections 2024-04-25T00:37:44.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 --2- 172.21.15.29:0/3881177010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadcc0cb280 0x7fadcc06b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 --2- 172.21.15.29:0/3881177010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadcc15e240 0x7fadcc06b3f0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 -- 172.21.15.29:0/3881177010 >> 172.21.15.29:0/3881177010 conn(0x7fadcc0d0c30 msgr2=0x7fadcc0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:44.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 -- 172.21.15.29:0/3881177010 shutdown_connections 2024-04-25T00:37:44.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 -- 172.21.15.29:0/3881177010 wait complete. 2024-04-25T00:37:44.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 Processor -- start 2024-04-25T00:37:44.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 -- start start 2024-04-25T00:37:44.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadcc0cb280 0x7fadcc0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadcc0cca60 0x7fadcc0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadcc15e240 0x7fadcc0aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc15e620 con 0x7fadcc0cb280 2024-04-25T00:37:44.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc072c50 con 0x7fadcc0cca60 2024-04-25T00:37:44.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fadcc0ccef0 con 0x7fadcc15e240 2024-04-25T00:37:44.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadca59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadcc0cb280 0x7fadcc0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadca59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadcc0cb280 0x7fadcc0c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34948/0 (socket says 172.21.15.29:34948) 2024-04-25T00:37:44.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadca59c640 1 -- 172.21.15.29:0/101620105 learned_addr learned my addr 172.21.15.29:0/101620105 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:44.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadcad9d640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadcc15e240 0x7fadcc0aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadc9d9b640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadcc0cca60 0x7fadcc0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadca59c640 1 -- 172.21.15.29:0/101620105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadcc15e240 msgr2=0x7fadcc0aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadca59c640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadcc15e240 0x7fadcc0aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadca59c640 1 -- 172.21.15.29:0/101620105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadcc0cca60 msgr2=0x7fadcc0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadca59c640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadcc0cca60 0x7fadcc0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadca59c640 1 -- 172.21.15.29:0/101620105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadcc15a0a0 con 0x7fadcc0cb280 2024-04-25T00:37:44.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadca59c640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadcc0cb280 0x7fadcc0c7e10 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fadb40b3040 tx=0x7fadb401baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:44.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadc2ffd640 1 -- 172.21.15.29:0/101620105 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb40b5070 con 0x7fadcc0cb280 2024-04-25T00:37:44.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadc2ffd640 1 -- 172.21.15.29:0/101620105 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb40b0de0 con 0x7fadcc0cb280 2024-04-25T00:37:44.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 -- 172.21.15.29:0/101620105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadcc152990 con 0x7fadcc0cb280 2024-04-25T00:37:44.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadc2ffd640 1 -- 172.21.15.29:0/101620105 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb40bb670 con 0x7fadcc0cb280 2024-04-25T00:37:44.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.467+0000 7fadd0c14640 1 -- 172.21.15.29:0/101620105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadcc15a620 con 0x7fadcc0cb280 2024-04-25T00:37:44.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.471+0000 7fadd0c14640 1 -- 172.21.15.29:0/101620105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadcc0692e0 con 0x7fadcc0cb280 2024-04-25T00:37:44.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.471+0000 7fadc2ffd640 1 -- 172.21.15.29:0/101620105 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 87598+0+0 (secure 0 0 0) 0x7fadb40c2060 con 0x7fadcc0cb280 2024-04-25T00:37:44.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.471+0000 7fadc2ffd640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fadac067a70 0x7fadac069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.471+0000 7fadc2ffd640 1 -- 172.21.15.29:0/101620105 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fadb4123460 con 0x7fadcc0cb280 2024-04-25T00:37:44.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.471+0000 7fadc9d9b640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fadac067a70 0x7fadac069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.471+0000 7fadc9d9b640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fadac067a70 0x7fadac069f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fadcc159b80 tx=0x7fadbc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:44.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.475+0000 7fadc2ffd640 1 -- 172.21.15.29:0/101620105 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fadb40188b0 con 0x7fadcc0cb280 2024-04-25T00:37:44.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.667+0000 7fadd0c14640 1 -- 172.21.15.29:0/101620105 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol6790", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) -- 0x7fadcc0d1a60 con 0x7fadac067a70 2024-04-25T00:37:44.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.687+0000 7fadc2ffd640 1 -- 172.21.15.29:0/101620105 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fadcc0d1a60 con 0x7fadac067a70 2024-04-25T00:37:44.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.691+0000 7fadd0c14640 1 -- 172.21.15.29:0/101620105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fadac067a70 msgr2=0x7fadac069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.691+0000 7fadd0c14640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fadac067a70 0x7fadac069f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fadcc159b80 tx=0x7fadbc008040 comp rx=0 tx=0).stop 2024-04-25T00:37:44.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.691+0000 7fadd0c14640 1 -- 172.21.15.29:0/101620105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadcc0cb280 msgr2=0x7fadcc0c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.691+0000 7fadd0c14640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadcc0cb280 0x7fadcc0c7e10 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fadb40b3040 tx=0x7fadb401baf0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.691+0000 7fadd0c14640 1 -- 172.21.15.29:0/101620105 shutdown_connections 2024-04-25T00:37:44.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.691+0000 7fadd0c14640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fadac067a70 0x7fadac069f30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.691+0000 7fadd0c14640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadcc15e240 0x7fadcc0aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.691+0000 7fadd0c14640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadcc0cca60 0x7fadcc0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.691+0000 7fadd0c14640 1 --2- 172.21.15.29:0/101620105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadcc0cb280 0x7fadcc0c7e10 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.691+0000 7fadd0c14640 1 -- 172.21.15.29:0/101620105 >> 172.21.15.29:0/101620105 conn(0x7fadcc0d0c30 msgr2=0x7fadcc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:44.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.691+0000 7fadd0c14640 1 -- 172.21.15.29:0/101620105 shutdown_connections 2024-04-25T00:37:44.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.691+0000 7fadd0c14640 1 -- 172.21.15.29:0/101620105 wait complete. 2024-04-25T00:37:44.710 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol6790 2024-04-25T00:37:44.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.831+0000 7fdda6fa9640 1 Processor -- start 2024-04-25T00:37:44.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.831+0000 7fdda6fa9640 1 -- start start 2024-04-25T00:37:44.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.831+0000 7fdda6fa9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdda00b4780 0x7fdda015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.831+0000 7fdda6fa9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdda00cc240 0x7fdda015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.831+0000 7fdda6fa9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdda00b3720 con 0x7fdda015e8d0 2024-04-25T00:37:44.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.831+0000 7fdda6fa9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdda00cc650 con 0x7fdda00cc240 2024-04-25T00:37:44.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.831+0000 7fdda6fa9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdda00b42f0 con 0x7fdda00b4780 2024-04-25T00:37:44.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.831+0000 7fdda5546640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdda00cc240 0x7fdda015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.831+0000 7fdda5546640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdda00cc240 0x7fdda015c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34958/0 (socket says 172.21.15.29:34958) 2024-04-25T00:37:44.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.831+0000 7fdda5546640 1 -- 172.21.15.29:0/2276413132 learned_addr learned my addr 172.21.15.29:0/2276413132 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:44.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.831+0000 7fdd9ffff640 1 --2- 172.21.15.29:0/2276413132 >> v2:172.21.15.157:3300/0 conn(0x7fdda00b4780 0x7fdda015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdda5546640 1 -- 172.21.15.29:0/2276413132 >> v2:172.21.15.157:3300/0 conn(0x7fdda00b4780 msgr2=0x7fdda015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:44.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdda5546640 1 --2- 172.21.15.29:0/2276413132 >> v2:172.21.15.157:3300/0 conn(0x7fdda00b4780 0x7fdda015f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdda5546640 1 -- 172.21.15.29:0/2276413132 >> v1:172.21.15.157:6789/0 conn(0x7fdda015e8d0 legacy=0x7fdda015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdda5546640 1 -- 172.21.15.29:0/2276413132 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda00b39c0 con 0x7fdda00cc240 2024-04-25T00:37:44.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdda5546640 1 --2- 172.21.15.29:0/2276413132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00cc240 0x7fdda015c7d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fdda0070910 tx=0x7fdd9801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fec2ac794659d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:44.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdd9f7fe640 1 -- 172.21.15.29:0/2276413132 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd980a5880 con 0x7fdda00cc240 2024-04-25T00:37:44.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdd9f7fe640 1 -- 172.21.15.29:0/2276413132 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd980a5a20 con 0x7fdda00cc240 2024-04-25T00:37:44.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdd9f7fe640 1 -- 172.21.15.29:0/2276413132 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd980a5df0 con 0x7fdda00cc240 2024-04-25T00:37:44.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdda6fa9640 1 -- 172.21.15.29:0/2276413132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00cc240 msgr2=0x7fdda015c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdda6fa9640 1 --2- 172.21.15.29:0/2276413132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00cc240 0x7fdda015c7d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fdda0070910 tx=0x7fdd9801c9c0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdda6fa9640 1 -- 172.21.15.29:0/2276413132 shutdown_connections 2024-04-25T00:37:44.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdda6fa9640 1 --2- 172.21.15.29:0/2276413132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00cc240 0x7fdda015c7d0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdda6fa9640 1 --2- 172.21.15.29:0/2276413132 >> v2:172.21.15.157:3300/0 conn(0x7fdda00b4780 0x7fdda015f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdda6fa9640 1 -- 172.21.15.29:0/2276413132 >> 172.21.15.29:0/2276413132 conn(0x7fdda00b1870 msgr2=0x7fdda00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:44.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdda6fa9640 1 -- 172.21.15.29:0/2276413132 shutdown_connections 2024-04-25T00:37:44.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.835+0000 7fdda6fa9640 1 -- 172.21.15.29:0/2276413132 wait complete. 2024-04-25T00:37:44.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda6fa9640 1 Processor -- start 2024-04-25T00:37:44.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda6fa9640 1 -- start start 2024-04-25T00:37:44.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda6fa9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda00b4780 0x7fdda00cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda6fa9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00cc240 0x7fdda00cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda6fa9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdda015e8d0 0x7fdda00cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda6fa9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdda0144a20 con 0x7fdda00cc240 2024-04-25T00:37:44.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda6fa9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdda015d520 con 0x7fdda00b4780 2024-04-25T00:37:44.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda6fa9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdda013e2a0 con 0x7fdda015e8d0 2024-04-25T00:37:44.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdd9ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00cc240 0x7fdda00cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdd9ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00cc240 0x7fdda00cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34974/0 (socket says 172.21.15.29:34974) 2024-04-25T00:37:44.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdd9ffff640 1 -- 172.21.15.29:0/1165340727 learned_addr learned my addr 172.21.15.29:0/1165340727 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:44.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda5546640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdda015e8d0 0x7fdda00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdd9ffff640 1 -- 172.21.15.29:0/1165340727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdda015e8d0 msgr2=0x7fdda00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdd9ffff640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdda015e8d0 0x7fdda00cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdd9ffff640 1 -- 172.21.15.29:0/1165340727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda00b4780 msgr2=0x7fdda00cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:44.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda4d45640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda00b4780 0x7fdda00cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdd9ffff640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda00b4780 0x7fdda00cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:44.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdd9ffff640 1 -- 172.21.15.29:0/1165340727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda0071a30 con 0x7fdda00cc240 2024-04-25T00:37:44.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda4d45640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda00b4780 0x7fdda00cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:44.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdd9ffff640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00cc240 0x7fdda00cd6c0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400d800 tx=0x7fdd9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:44.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdd9d7fa640 1 -- 172.21.15.29:0/1165340727 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400b6c0 con 0x7fdda00cc240 2024-04-25T00:37:44.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdd9d7fa640 1 -- 172.21.15.29:0/1165340727 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd94019070 con 0x7fdda00cc240 2024-04-25T00:37:44.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdd9d7fa640 1 -- 172.21.15.29:0/1165340727 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400bae0 con 0x7fdda00cc240 2024-04-25T00:37:44.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda6fa9640 1 -- 172.21.15.29:0/1165340727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda0071d10 con 0x7fdda00cc240 2024-04-25T00:37:44.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda6fa9640 1 -- 172.21.15.29:0/1165340727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda013e7d0 con 0x7fdda00cc240 2024-04-25T00:37:44.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.839+0000 7fdda6fa9640 1 -- 172.21.15.29:0/1165340727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda013e9d0 con 0x7fdda00cc240 2024-04-25T00:37:44.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.843+0000 7fdd9d7fa640 1 -- 172.21.15.29:0/1165340727 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 87598+0+0 (secure 0 0 0) 0x7fdd9400bc80 con 0x7fdda00cc240 2024-04-25T00:37:44.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.843+0000 7fdd9d7fa640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdd8c067a70 0x7fdd8c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:44.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.843+0000 7fdd9d7fa640 1 -- 172.21.15.29:0/1165340727 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fdd94086460 con 0x7fdda00cc240 2024-04-25T00:37:44.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.843+0000 7fdda4d45640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdd8c067a70 0x7fdd8c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:44.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.843+0000 7fdda4d45640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdd8c067a70 0x7fdd8c069f30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdd90001650 tx=0x7fdd90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:44.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:44.847+0000 7fdd9d7fa640 1 -- 172.21.15.29:0/1165340727 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdda013e9d0 con 0x7fdda00cc240 2024-04-25T00:37:45.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.035+0000 7fdda6fa9640 1 -- 172.21.15.29:0/1165340727 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol6790", "target": ["mon-mgr", ""]}) -- 0x7fdda0069270 con 0x7fdd8c067a70 2024-04-25T00:37:45.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.063+0000 7fdd9d7fa640 1 -- 172.21.15.29:0/1165340727 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+66 (secure 0 0 0) 0x7fdda0069270 con 0x7fdd8c067a70 2024-04-25T00:37:45.066 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1 2024-04-25T00:37:45.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.063+0000 7fdda6fa9640 1 -- 172.21.15.29:0/1165340727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdd8c067a70 msgr2=0x7fdd8c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:45.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.063+0000 7fdda6fa9640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdd8c067a70 0x7fdd8c069f30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdd90001650 tx=0x7fdd90009040 comp rx=0 tx=0).stop 2024-04-25T00:37:45.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.063+0000 7fdda6fa9640 1 -- 172.21.15.29:0/1165340727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00cc240 msgr2=0x7fdda00cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:45.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.063+0000 7fdda6fa9640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00cc240 0x7fdda00cd6c0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400d800 tx=0x7fdd9400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:37:45.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.063+0000 7fdda6fa9640 1 -- 172.21.15.29:0/1165340727 shutdown_connections 2024-04-25T00:37:45.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.063+0000 7fdda6fa9640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdd8c067a70 0x7fdd8c069f30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:45.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.063+0000 7fdda6fa9640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdda015e8d0 0x7fdda00cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:45.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.063+0000 7fdda6fa9640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00cc240 0x7fdda00cd6c0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:45.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.063+0000 7fdda6fa9640 1 --2- 172.21.15.29:0/1165340727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda00b4780 0x7fdda00cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:45.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.063+0000 7fdda6fa9640 1 -- 172.21.15.29:0/1165340727 >> 172.21.15.29:0/1165340727 conn(0x7fdda00b1870 msgr2=0x7fdda00ad8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:45.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.063+0000 7fdda6fa9640 1 -- 172.21.15.29:0/1165340727 shutdown_connections 2024-04-25T00:37:45.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:45.067+0000 7fdda6fa9640 1 -- 172.21.15.29:0/1165340727 wait complete. 2024-04-25T00:37:45.083 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol6790 with 8 files each 1MB size under directory volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1 2024-04-25T00:37:45.083 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:37:45.083 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:45.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:45.089 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T00:37:45.154 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-25T00:37:45.155 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-25T00:37:45.155 INFO:teuthology.orchestra.run.smithi029.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118009 s, 88.9 MB/s 2024-04-25T00:37:45.156 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:37:45.156 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:45.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:45.205 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T00:37:45.281 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-25T00:37:45.281 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-25T00:37:45.281 INFO:teuthology.orchestra.run.smithi029.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165268 s, 63.4 MB/s 2024-04-25T00:37:45.282 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:37:45.282 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:45.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:45.329 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T00:37:45.407 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-25T00:37:45.408 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-25T00:37:45.408 INFO:teuthology.orchestra.run.smithi029.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184839 s, 56.7 MB/s 2024-04-25T00:37:45.409 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:37:45.409 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:45.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:45.458 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T00:37:45.533 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-25T00:37:45.533 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-25T00:37:45.533 INFO:teuthology.orchestra.run.smithi029.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189153 s, 55.4 MB/s 2024-04-25T00:37:45.534 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:37:45.534 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:45.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:45.584 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T00:37:45.664 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-25T00:37:45.664 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-25T00:37:45.664 INFO:teuthology.orchestra.run.smithi029.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161954 s, 64.7 MB/s 2024-04-25T00:37:45.665 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:37:45.665 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:45.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:45.714 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T00:37:45.789 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-25T00:37:45.789 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-25T00:37:45.789 INFO:teuthology.orchestra.run.smithi029.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162327 s, 64.6 MB/s 2024-04-25T00:37:45.791 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:37:45.791 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:45.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:45.837 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T00:37:45.912 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-25T00:37:45.912 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-25T00:37:45.912 INFO:teuthology.orchestra.run.smithi029.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01592 s, 65.9 MB/s 2024-04-25T00:37:45.913 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:37:45.913 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:37:45.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:45.962 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T00:37:46.037 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-25T00:37:46.037 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-25T00:37:46.037 INFO:teuthology.orchestra.run.smithi029.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201867 s, 51.9 MB/s 2024-04-25T00:37:46.038 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol6790 subvol_snap26070 2024-04-25T00:37:46.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0f4ca640 1 Processor -- start 2024-04-25T00:37:46.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0f4ca640 1 -- start start 2024-04-25T00:37:46.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0f4ca640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa0815e560 0x7ffa080d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0f4ca640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa080d2680 0x7ffa080d2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0f4ca640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa080d3030 con 0x7ffa0815e180 2024-04-25T00:37:46.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0f4ca640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ffa080cfed0 con 0x7ffa0815e560 2024-04-25T00:37:46.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0f4ca640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffa080b7100 con 0x7ffa080d2680 2024-04-25T00:37:46.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0ca65640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa0815e560 0x7ffa080d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0ca65640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa0815e560 0x7ffa080d20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34990/0 (socket says 172.21.15.29:34990) 2024-04-25T00:37:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0ca65640 1 -- 172.21.15.29:0/2573502579 learned_addr learned my addr 172.21.15.29:0/2573502579 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0da67640 1 --2- 172.21.15.29:0/2573502579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa080d2680 0x7ffa080d2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0ca65640 1 -- 172.21.15.29:0/2573502579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa080d2680 msgr2=0x7ffa080d2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0ca65640 1 --2- 172.21.15.29:0/2573502579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa080d2680 0x7ffa080d2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0ca65640 1 -- 172.21.15.29:0/2573502579 >> v1:172.21.15.157:6789/0 conn(0x7ffa0815e180 legacy=0x7ffa080cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:46.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.219+0000 7ffa0ca65640 1 -- 172.21.15.29:0/2573502579 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa080b73e0 con 0x7ffa0815e560 2024-04-25T00:37:46.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0ca65640 1 --2- 172.21.15.29:0/2573502579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa0815e560 0x7ffa080d20e0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009870 tx=0x7ff9f801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51a803e494c890f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:46.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ff9fffff640 1 -- 172.21.15.29:0/2573502579 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80b5070 con 0x7ffa0815e560 2024-04-25T00:37:46.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ff9fffff640 1 -- 172.21.15.29:0/2573502579 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f80a8a50 con 0x7ffa0815e560 2024-04-25T00:37:46.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ff9fffff640 1 -- 172.21.15.29:0/2573502579 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80a8e20 con 0x7ffa0815e560 2024-04-25T00:37:46.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/2573502579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa0815e560 msgr2=0x7ffa080d20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:46.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 --2- 172.21.15.29:0/2573502579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa0815e560 0x7ffa080d20e0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009870 tx=0x7ff9f801ed50 comp rx=0 tx=0).stop 2024-04-25T00:37:46.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/2573502579 shutdown_connections 2024-04-25T00:37:46.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 --2- 172.21.15.29:0/2573502579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa080d2680 0x7ffa080d2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 --2- 172.21.15.29:0/2573502579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa0815e560 0x7ffa080d20e0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/2573502579 >> 172.21.15.29:0/2573502579 conn(0x7ffa080d40a0 msgr2=0x7ffa080d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:46.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/2573502579 shutdown_connections 2024-04-25T00:37:46.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/2573502579 wait complete. 2024-04-25T00:37:46.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 Processor -- start 2024-04-25T00:37:46.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 -- start start 2024-04-25T00:37:46.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa080d2680 0x7ffa081f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa0815e180 0x7ffa08152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa0815e560 0x7ffa080c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffa080b7d60 con 0x7ffa0815e180 2024-04-25T00:37:46.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa080b7be0 con 0x7ffa080d2680 2024-04-25T00:37:46.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0f4ca640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffa080b7ee0 con 0x7ffa0815e560 2024-04-25T00:37:46.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0ca65640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa0815e180 0x7ffa08152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:46.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0ca65640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa0815e180 0x7ffa08152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35002/0 (socket says 172.21.15.29:35002) 2024-04-25T00:37:46.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0ca65640 1 -- 172.21.15.29:0/1758769212 learned_addr learned my addr 172.21.15.29:0/1758769212 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:46.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0da67640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa0815e560 0x7ffa080c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:46.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0d266640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa080d2680 0x7ffa081f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:46.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0ca65640 1 -- 172.21.15.29:0/1758769212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa0815e560 msgr2=0x7ffa080c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:46.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0ca65640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa0815e560 0x7ffa080c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0ca65640 1 -- 172.21.15.29:0/1758769212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa080d2680 msgr2=0x7ffa081f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:46.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0ca65640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa080d2680 0x7ffa081f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0ca65640 1 -- 172.21.15.29:0/1758769212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa080c8d20 con 0x7ffa0815e180 2024-04-25T00:37:46.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.223+0000 7ffa0d266640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa080d2680 0x7ffa081f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:46.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.227+0000 7ffa0ca65640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa0815e180 0x7ffa08152610 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7ff9f801d400 tx=0x7ff9f801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:46.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.227+0000 7ff9fdffb640 1 -- 172.21.15.29:0/1758769212 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80b5070 con 0x7ffa0815e180 2024-04-25T00:37:46.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.227+0000 7ff9fdffb640 1 -- 172.21.15.29:0/1758769212 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f80b0de0 con 0x7ffa0815e180 2024-04-25T00:37:46.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.227+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/1758769212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa080cb210 con 0x7ffa0815e180 2024-04-25T00:37:46.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.227+0000 7ff9fdffb640 1 -- 172.21.15.29:0/1758769212 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80bb670 con 0x7ffa0815e180 2024-04-25T00:37:46.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.227+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/1758769212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa080cb7d0 con 0x7ffa0815e180 2024-04-25T00:37:46.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.227+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/1758769212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa080692e0 con 0x7ffa0815e180 2024-04-25T00:37:46.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.227+0000 7ff9fdffb640 1 -- 172.21.15.29:0/1758769212 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 87598+0+0 (secure 0 0 0) 0x7ff9f80c2060 con 0x7ffa0815e180 2024-04-25T00:37:46.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.227+0000 7ff9fdffb640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff9ec067a70 0x7ff9ec069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.227+0000 7ff9fdffb640 1 -- 172.21.15.29:0/1758769212 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7ff9f81233b0 con 0x7ffa0815e180 2024-04-25T00:37:46.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.227+0000 7ffa0d266640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff9ec067a70 0x7ff9ec069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:46.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.227+0000 7ffa0d266640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff9ec067a70 0x7ff9ec069f30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa040016f0 tx=0x7ffa04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:46.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.231+0000 7ff9fdffb640 1 -- 172.21.15.29:0/1758769212 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff9f80188b0 con 0x7ffa0815e180 2024-04-25T00:37:46.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.423+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/1758769212 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol6790", "snap_name": "subvol_snap26070", "target": ["mon-mgr", ""]}) -- 0x7ffa080cdaa0 con 0x7ff9ec067a70 2024-04-25T00:37:46.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.447+0000 7ff9fdffb640 1 -- 172.21.15.29:0/1758769212 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffa04001350 con 0x7ff9ec067a70 2024-04-25T00:37:46.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.451+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/1758769212 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff9ec067a70 msgr2=0x7ff9ec069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:46.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.451+0000 7ffa0f4ca640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff9ec067a70 0x7ff9ec069f30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa040016f0 tx=0x7ffa04009040 comp rx=0 tx=0).stop 2024-04-25T00:37:46.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.451+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/1758769212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa0815e180 msgr2=0x7ffa08152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:46.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.451+0000 7ffa0f4ca640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa0815e180 0x7ffa08152610 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7ff9f801d400 tx=0x7ff9f801baf0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.451+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/1758769212 shutdown_connections 2024-04-25T00:37:46.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.451+0000 7ffa0f4ca640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff9ec067a70 0x7ff9ec069f30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.451+0000 7ffa0f4ca640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa0815e560 0x7ffa080c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.451+0000 7ffa0f4ca640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa0815e180 0x7ffa08152610 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.451+0000 7ffa0f4ca640 1 --2- 172.21.15.29:0/1758769212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa080d2680 0x7ffa081f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.451+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/1758769212 >> 172.21.15.29:0/1758769212 conn(0x7ffa080d40a0 msgr2=0x7ffa081609b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:46.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.451+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/1758769212 shutdown_connections 2024-04-25T00:37:46.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.451+0000 7ffa0f4ca640 1 -- 172.21.15.29:0/1758769212 wait complete. 2024-04-25T00:37:46.471 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol6790 subvol_snap26070 subvol_clone39330 2024-04-25T00:37:46.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.595+0000 7f5bf9148640 1 Processor -- start 2024-04-25T00:37:46.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.595+0000 7f5bf9148640 1 -- start start 2024-04-25T00:37:46.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf9148640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5bf415e8d0 0x7f5bf40cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf9148640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf40b3720 con 0x7f5bf415f3c0 2024-04-25T00:37:46.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf9148640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf40b39c0 con 0x7f5bf40b4780 2024-04-25T00:37:46.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf9148640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5bf40b42f0 con 0x7f5bf415e8d0 2024-04-25T00:37:46.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf359e640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5bf415f3c0 0x7f5bf415c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54032/0 (socket says 172.21.15.29:54032) 2024-04-25T00:37:46.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf359e640 1 -- 172.21.15.29:0/2203448543 learned_addr learned my addr 172.21.15.29:0/2203448543 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:46.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf2d9d640 1 --2- 172.21.15.29:0/2203448543 >> v2:172.21.15.157:3300/0 conn(0x7f5bf415e8d0 0x7f5bf40cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:46.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2026858910 0 0) 0x7f5bf40b3720 con 0x7f5bf415f3c0 2024-04-25T00:37:46.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bdc003530 con 0x7f5bf415f3c0 2024-04-25T00:37:46.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1912355134 0 0) 0x7f5bdc003530 con 0x7f5bf415f3c0 2024-04-25T00:37:46.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 >> v2:172.21.15.157:3300/0 conn(0x7f5bf415e8d0 msgr2=0x7f5bf40cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:46.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 --2- 172.21.15.29:0/2203448543 >> v2:172.21.15.157:3300/0 conn(0x7f5bf415e8d0 0x7f5bf40cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 >> v1:172.21.15.157:6789/0 conn(0x7f5bf40b4780 legacy=0x7f5bf415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:46.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf40b3ca0 con 0x7f5bf415f3c0 2024-04-25T00:37:46.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f5bec001a00 con 0x7f5bf415f3c0 2024-04-25T00:37:46.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 >> v1:172.21.15.29:6789/0 conn(0x7f5bf415f3c0 legacy=0x7f5bf415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:46.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 --2- 172.21.15.29:0/2203448543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bdc0039c0 0x7f5bdc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 --2- 172.21.15.29:0/2203448543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bdc004810 0x7f5bdc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 --2- 172.21.15.29:0/2203448543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bdc0051d0 0x7f5bdc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf40b42f0 con 0x7f5bdc0051d0 2024-04-25T00:37:46.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc00c070 con 0x7f5bdc0039c0 2024-04-25T00:37:46.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5bdc00e480 con 0x7f5bdc004810 2024-04-25T00:37:46.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf359e640 1 --2- 172.21.15.29:0/2203448543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bdc0051d0 0x7f5bdc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:46.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf2d9d640 1 --2- 172.21.15.29:0/2203448543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bdc0039c0 0x7f5bdc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:46.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf259c640 1 --2- 172.21.15.29:0/2203448543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bdc004810 0x7f5bdc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:46.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf359e640 1 -- 172.21.15.29:0/2203448543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bdc004810 msgr2=0x7f5bdc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:46.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf359e640 1 --2- 172.21.15.29:0/2203448543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bdc004810 0x7f5bdc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf359e640 1 -- 172.21.15.29:0/2203448543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bdc0039c0 msgr2=0x7f5bdc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:46.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf359e640 1 --2- 172.21.15.29:0/2203448543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bdc0039c0 0x7f5bdc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.599+0000 7f5bf359e640 1 -- 172.21.15.29:0/2203448543 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc003530 con 0x7f5bdc0051d0 2024-04-25T00:37:46.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf359e640 1 -- 172.21.15.29:0/2203448543 reap_dead start 2024-04-25T00:37:46.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf359e640 1 --2- 172.21.15.29:0/2203448543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bdc0051d0 0x7f5bdc00b9d0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00a8d0 tx=0x7f5bec01d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a800d5c4fda9f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:46.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec00a4a0 con 0x7f5bdc0051d0 2024-04-25T00:37:46.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bec0ac640 con 0x7f5bdc0051d0 2024-04-25T00:37:46.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf1d9b640 1 -- 172.21.15.29:0/2203448543 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0aca30 con 0x7f5bdc0051d0 2024-04-25T00:37:46.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 -- 172.21.15.29:0/2203448543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bdc0051d0 msgr2=0x7f5bdc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:46.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 --2- 172.21.15.29:0/2203448543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bdc0051d0 0x7f5bdc00b9d0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00a8d0 tx=0x7f5bec01d400 comp rx=0 tx=0).stop 2024-04-25T00:37:46.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 -- 172.21.15.29:0/2203448543 shutdown_connections 2024-04-25T00:37:46.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 --2- 172.21.15.29:0/2203448543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bdc0051d0 0x7f5bdc00b9d0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 -- 172.21.15.29:0/2203448543 >> 172.21.15.29:0/2203448543 conn(0x7f5bf40b1870 msgr2=0x7f5bf40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:46.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 -- 172.21.15.29:0/2203448543 shutdown_connections 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 -- 172.21.15.29:0/2203448543 wait complete. 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 Processor -- start 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 -- start start 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bdc0051d0 0x7f5bf4138ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf413cb20 0x7f5bf413cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bf4071c00 0x7f5bf4072090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf406cf10 con 0x7f5bf4071c00 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf40b3580 con 0x7f5bf413cb20 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf9148640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf4144a20 con 0x7f5bdc0051d0 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf359e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bf4071c00 0x7f5bf4072090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf359e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bf4071c00 0x7f5bf4072090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35020/0 (socket says 172.21.15.29:35020) 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf359e640 1 -- 172.21.15.29:0/3688600173 learned_addr learned my addr 172.21.15.29:0/3688600173 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.603+0000 7f5bf259c640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf413cb20 0x7f5bf413cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bf359e640 1 -- 172.21.15.29:0/3688600173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bdc0051d0 msgr2=0x7f5bf4138ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bf359e640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bdc0051d0 0x7f5bf4138ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bf359e640 1 -- 172.21.15.29:0/3688600173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf413cb20 msgr2=0x7f5bf413cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bf359e640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf413cb20 0x7f5bf413cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bf359e640 1 -- 172.21.15.29:0/3688600173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf413a390 con 0x7f5bf4071c00 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bf259c640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf413cb20 0x7f5bf413cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bf359e640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bf4071c00 0x7f5bf4072090 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f5bec01dca0 tx=0x7f5bdc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bdb7fe640 1 -- 172.21.15.29:0/3688600173 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0b0030 con 0x7f5bf4071c00 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bdb7fe640 1 -- 172.21.15.29:0/3688600173 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bec0019c0 con 0x7f5bf4071c00 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bf9148640 1 -- 172.21.15.29:0/3688600173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf413a630 con 0x7f5bf4071c00 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bdb7fe640 1 -- 172.21.15.29:0/3688600173 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0aca10 con 0x7f5bf4071c00 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bf9148640 1 -- 172.21.15.29:0/3688600173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf413abf0 con 0x7f5bf4071c00 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bf9148640 1 -- 172.21.15.29:0/3688600173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf4069270 con 0x7f5bf4071c00 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bdb7fe640 1 -- 172.21.15.29:0/3688600173 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 87598+0+0 (secure 0 0 0) 0x7f5bec0b7420 con 0x7f5bf4071c00 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bdb7fe640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5be0067a70 0x7f5be0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:46.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bf2d9d640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5be0067a70 0x7f5be0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:46.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bdb7fe640 1 -- 172.21.15.29:0/3688600173 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f5bec1201e0 con 0x7f5bf4071c00 2024-04-25T00:37:46.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.607+0000 7f5bf2d9d640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5be0067a70 0x7f5be0069f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40729f0 tx=0x7f5be4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:46.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.611+0000 7f5bdb7fe640 1 -- 172.21.15.29:0/3688600173 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5bec017110 con 0x7f5bf4071c00 2024-04-25T00:37:46.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.807+0000 7f5bf9148640 1 -- 172.21.15.29:0/3688600173 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol6790", "snap_name": "subvol_snap26070", "target_sub_name": "subvol_clone39330", "target": ["mon-mgr", ""]}) -- 0x7f5bf4071030 con 0x7f5be0067a70 2024-04-25T00:37:46.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.955+0000 7f5bdb7fe640 1 -- 172.21.15.29:0/3688600173 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5be4011860 con 0x7f5be0067a70 2024-04-25T00:37:46.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.959+0000 7f5bf9148640 1 -- 172.21.15.29:0/3688600173 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5be0067a70 msgr2=0x7f5be0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:46.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.959+0000 7f5bf9148640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5be0067a70 0x7f5be0069f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40729f0 tx=0x7f5be4008040 comp rx=0 tx=0).stop 2024-04-25T00:37:46.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.959+0000 7f5bf9148640 1 -- 172.21.15.29:0/3688600173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bf4071c00 msgr2=0x7f5bf4072090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:46.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.959+0000 7f5bf9148640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bf4071c00 0x7f5bf4072090 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f5bec01dca0 tx=0x7f5bdc005840 comp rx=0 tx=0).stop 2024-04-25T00:37:46.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.959+0000 7f5bf9148640 1 -- 172.21.15.29:0/3688600173 shutdown_connections 2024-04-25T00:37:46.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.959+0000 7f5bf9148640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5be0067a70 0x7f5be0069f30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.959+0000 7f5bf9148640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bf4071c00 0x7f5bf4072090 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.959+0000 7f5bf9148640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf413cb20 0x7f5bf413cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.959+0000 7f5bf9148640 1 --2- 172.21.15.29:0/3688600173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bdc0051d0 0x7f5bf4138ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:46.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.959+0000 7f5bf9148640 1 -- 172.21.15.29:0/3688600173 >> 172.21.15.29:0/3688600173 conn(0x7f5bf40b1870 msgr2=0x7f5bf406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:46.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.959+0000 7f5bf9148640 1 -- 172.21.15.29:0/3688600173 shutdown_connections 2024-04-25T00:37:46.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:46.959+0000 7f5bf9148640 1 -- 172.21.15.29:0/3688600173 wait complete. 2024-04-25T00:37:46.966 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:46.963+0000 7f987d589640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:46.967 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:46.963+0000 7f987d589640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:46.967 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:46.963+0000 7f987d589640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:46.967 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:46.963+0000 7f987d589640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:46.967 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:46.963+0000 7f987d589640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:46.983 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone39330 2024-04-25T00:37:46.986 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:46.983+0000 7f987c587640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:46.986 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:46.983+0000 7f987c587640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:46.986 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:46.983+0000 7f987c587640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:46.986 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:46.983+0000 7f987c587640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:46.986 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:37:46.983+0000 7f987c587640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:37:47.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc7251f640 1 Processor -- start 2024-04-25T00:37:47.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc7251f640 1 -- start start 2024-04-25T00:37:47.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc7251f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbc6c0d2c60 0x7fbc6c168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:47.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc7251f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc6c0cfc10 con 0x7fbc6c15e200 2024-04-25T00:37:47.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc7251f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc6c0cfef0 con 0x7fbc6c0cf690 2024-04-25T00:37:47.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc7251f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbc6c0b38d0 con 0x7fbc6c0d2c60 2024-04-25T00:37:47.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc70abc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbc6c0d2c60 0x7fbc6c168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:47.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc70abc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbc6c0d2c60 0x7fbc6c168e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35060/0 (socket says 172.21.15.29:35060) 2024-04-25T00:37:47.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc6bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbc6c15e200 0x7fbc6c0d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54046/0 (socket says 172.21.15.29:54046) 2024-04-25T00:37:47.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc70abc640 1 -- 172.21.15.29:0/1874274959 learned_addr learned my addr 172.21.15.29:0/1874274959 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:47.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc70abc640 1 -- 172.21.15.29:0/1874274959 >> v1:172.21.15.157:6789/0 conn(0x7fbc6c0cf690 legacy=0x7fbc6c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:47.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc6affd640 1 -- 172.21.15.29:0/1874274959 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 842982880 0 0) 0x7fbc6c0cfc10 con 0x7fbc6c15e200 2024-04-25T00:37:47.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc70abc640 1 -- 172.21.15.29:0/1874274959 >> v1:172.21.15.29:6789/0 conn(0x7fbc6c15e200 legacy=0x7fbc6c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:47.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.115+0000 7fbc70abc640 1 -- 172.21.15.29:0/1874274959 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c0b3b20 con 0x7fbc6c0d2c60 2024-04-25T00:37:47.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.119+0000 7fbc70abc640 1 --2- 172.21.15.29:0/1874274959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc6c0d2c60 0x7fbc6c168e20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009e50 tx=0x7fbc6401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2c07a44ca3d203 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:47.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.119+0000 7fbc6affd640 1 -- 172.21.15.29:0/1874274959 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc640a56e0 con 0x7fbc6c0d2c60 2024-04-25T00:37:47.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.119+0000 7fbc6affd640 1 -- 172.21.15.29:0/1874274959 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc640a5880 con 0x7fbc6c0d2c60 2024-04-25T00:37:47.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.119+0000 7fbc6affd640 1 -- 172.21.15.29:0/1874274959 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc640a5c50 con 0x7fbc6c0d2c60 2024-04-25T00:37:47.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.119+0000 7fbc7251f640 1 -- 172.21.15.29:0/1874274959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc6c0d2c60 msgr2=0x7fbc6c168e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:47.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.119+0000 7fbc7251f640 1 --2- 172.21.15.29:0/1874274959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc6c0d2c60 0x7fbc6c168e20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009e50 tx=0x7fbc6401c9f0 comp rx=0 tx=0).stop 2024-04-25T00:37:47.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.119+0000 7fbc7251f640 1 -- 172.21.15.29:0/1874274959 shutdown_connections 2024-04-25T00:37:47.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.119+0000 7fbc7251f640 1 --2- 172.21.15.29:0/1874274959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc6c0d2c60 0x7fbc6c168e20 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:47.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.119+0000 7fbc7251f640 1 -- 172.21.15.29:0/1874274959 >> 172.21.15.29:0/1874274959 conn(0x7fbc6c09d280 msgr2=0x7fbc6c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:47.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.119+0000 7fbc7251f640 1 -- 172.21.15.29:0/1874274959 shutdown_connections 2024-04-25T00:37:47.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.119+0000 7fbc7251f640 1 -- 172.21.15.29:0/1874274959 wait complete. 2024-04-25T00:37:47.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.119+0000 7fbc7251f640 1 Processor -- start 2024-04-25T00:37:47.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.119+0000 7fbc7251f640 1 -- start start 2024-04-25T00:37:47.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc7251f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0cf690 0x7fbc6c0ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:47.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc7251f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc6c0d2c60 0x7fbc6c156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:47.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc7251f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc6c15e200 0x7fbc6c158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:47.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc7251f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c16bf10 con 0x7fbc6c15e200 2024-04-25T00:37:47.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc7251f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c16bd90 con 0x7fbc6c0cf690 2024-04-25T00:37:47.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc7251f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6c16c090 con 0x7fbc6c0d2c60 2024-04-25T00:37:47.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc6bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0cf690 0x7fbc6c0ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:47.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc6bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0cf690 0x7fbc6c0ca570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:46544/0 (socket says 172.21.15.29:46544) 2024-04-25T00:37:47.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc6b7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc6c0d2c60 0x7fbc6c156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:47.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc6bfff640 1 -- 172.21.15.29:0/2073651336 learned_addr learned my addr 172.21.15.29:0/2073651336 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:47.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc70abc640 1 --2- 172.21.15.29:0/2073651336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc6c15e200 0x7fbc6c158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:47.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc6bfff640 1 -- 172.21.15.29:0/2073651336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc6c0d2c60 msgr2=0x7fbc6c156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:47.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc6bfff640 1 --2- 172.21.15.29:0/2073651336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc6c0d2c60 0x7fbc6c156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:47.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc6bfff640 1 -- 172.21.15.29:0/2073651336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc6c15e200 msgr2=0x7fbc6c158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:47.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc6bfff640 1 --2- 172.21.15.29:0/2073651336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc6c15e200 0x7fbc6c158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:47.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc6bfff640 1 -- 172.21.15.29:0/2073651336 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c1594d0 con 0x7fbc6c0cf690 2024-04-25T00:37:47.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc6bfff640 1 --2- 172.21.15.29:0/2073651336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0cf690 0x7fbc6c0ca570 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00c4a0 tx=0x7fbc5c00c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:47.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc68ff9640 1 -- 172.21.15.29:0/2073651336 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c018070 con 0x7fbc6c0cf690 2024-04-25T00:37:47.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc68ff9640 1 -- 172.21.15.29:0/2073651336 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc5c0074e0 con 0x7fbc6c0cf690 2024-04-25T00:37:47.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc7251f640 1 -- 172.21.15.29:0/2073651336 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6c1597b0 con 0x7fbc6c0cf690 2024-04-25T00:37:47.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc68ff9640 1 -- 172.21.15.29:0/2073651336 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c004a30 con 0x7fbc6c0cf690 2024-04-25T00:37:47.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc7251f640 1 -- 172.21.15.29:0/2073651336 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6c159d70 con 0x7fbc6c0cf690 2024-04-25T00:37:47.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.123+0000 7fbc7251f640 1 -- 172.21.15.29:0/2073651336 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6c069270 con 0x7fbc6c0cf690 2024-04-25T00:37:47.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.127+0000 7fbc68ff9640 1 -- 172.21.15.29:0/2073651336 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 87598+0+0 (secure 0 0 0) 0x7fbc5c004bd0 con 0x7fbc6c0cf690 2024-04-25T00:37:47.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.127+0000 7fbc68ff9640 1 --2- 172.21.15.29:0/2073651336 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc4c067a70 0x7fbc4c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:47.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.127+0000 7fbc68ff9640 1 -- 172.21.15.29:0/2073651336 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fbc5c085690 con 0x7fbc6c0cf690 2024-04-25T00:37:47.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.127+0000 7fbc6b7fe640 1 --2- 172.21.15.29:0/2073651336 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc4c067a70 0x7fbc4c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:47.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.127+0000 7fbc6b7fe640 1 --2- 172.21.15.29:0/2073651336 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc4c067a70 0x7fbc4c069f30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c1fa6f0 tx=0x7fbc54009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:47.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.139+0000 7fbc68ff9640 1 -- 172.21.15.29:0/2073651336 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbc5c021020 con 0x7fbc6c0cf690 2024-04-25T00:37:47.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:47.363+0000 7fbc7251f640 1 -- 172.21.15.29:0/2073651336 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone39330", "target": ["mon-mgr", ""]}) -- 0x7fbc6c0d4ad0 con 0x7fbc4c067a70 2024-04-25T00:37:48.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:48.491+0000 7fbc68ff9640 1 -- 172.21.15.29:0/2073651336 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7fbc5c052020 con 0x7fbc6c0cf690 2024-04-25T00:37:49.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.603+0000 7fbc68ff9640 1 -- 172.21.15.29:0/2073651336 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fbc6c0d4ad0 con 0x7fbc4c067a70 2024-04-25T00:37:49.607 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:37:49.607 INFO:teuthology.orchestra.run.smithi029.stdout: "status": { 2024-04-25T00:37:49.607 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "complete" 2024-04-25T00:37:49.607 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:49.607 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:37:49.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.607+0000 7fbc7251f640 1 -- 172.21.15.29:0/2073651336 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc4c067a70 msgr2=0x7fbc4c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:49.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.607+0000 7fbc7251f640 1 --2- 172.21.15.29:0/2073651336 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc4c067a70 0x7fbc4c069f30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c1fa6f0 tx=0x7fbc54009060 comp rx=0 tx=0).stop 2024-04-25T00:37:49.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.607+0000 7fbc7251f640 1 -- 172.21.15.29:0/2073651336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0cf690 msgr2=0x7fbc6c0ca570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:49.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.607+0000 7fbc7251f640 1 --2- 172.21.15.29:0/2073651336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0cf690 0x7fbc6c0ca570 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00c4a0 tx=0x7fbc5c00c960 comp rx=0 tx=0).stop 2024-04-25T00:37:49.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.607+0000 7fbc7251f640 1 -- 172.21.15.29:0/2073651336 shutdown_connections 2024-04-25T00:37:49.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.607+0000 7fbc7251f640 1 --2- 172.21.15.29:0/2073651336 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc4c067a70 0x7fbc4c069f30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:49.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.607+0000 7fbc7251f640 1 --2- 172.21.15.29:0/2073651336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc6c15e200 0x7fbc6c158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:49.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.607+0000 7fbc7251f640 1 --2- 172.21.15.29:0/2073651336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc6c0d2c60 0x7fbc6c156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:49.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.607+0000 7fbc7251f640 1 --2- 172.21.15.29:0/2073651336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0cf690 0x7fbc6c0ca570 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:49.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.607+0000 7fbc7251f640 1 -- 172.21.15.29:0/2073651336 >> 172.21.15.29:0/2073651336 conn(0x7fbc6c09d280 msgr2=0x7fbc6c0d3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:49.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.607+0000 7fbc7251f640 1 -- 172.21.15.29:0/2073651336 shutdown_connections 2024-04-25T00:37:49.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.607+0000 7fbc7251f640 1 -- 172.21.15.29:0/2073651336 wait complete. 2024-04-25T00:37:49.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:49.632 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone39330/.meta) 2024-04-25T00:37:49.649 INFO:teuthology.orchestra.run.smithi029.stdout:[GLOBAL] 2024-04-25T00:37:49.649 INFO:teuthology.orchestra.run.smithi029.stdout:version = 2 2024-04-25T00:37:49.649 INFO:teuthology.orchestra.run.smithi029.stdout:type = clone 2024-04-25T00:37:49.649 INFO:teuthology.orchestra.run.smithi029.stdout:path = /volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9 2024-04-25T00:37:49.650 INFO:teuthology.orchestra.run.smithi029.stdout:state = complete 2024-04-25T00:37:49.650 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:37:49.651 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol6790 2024-04-25T00:37:49.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd456fd3640 1 Processor -- start 2024-04-25T00:37:49.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd456fd3640 1 -- start start 2024-04-25T00:37:49.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd456fd3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd45015e200 0x7fd4500d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:49.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd456fd3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4500cf690 0x7fd4500d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:49.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd456fd3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4500d2a90 0x7fd4500d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:49.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd456fd3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd4500d2ee0 con 0x7fd4500d2a90 2024-04-25T00:37:49.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd456fd3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd4500b38d0 con 0x7fd45015e200 2024-04-25T00:37:49.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd456fd3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd4500cfc10 con 0x7fd4500cf690 2024-04-25T00:37:49.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd455570640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4500d2a90 0x7fd4500d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:49.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd455570640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4500d2a90 0x7fd4500d05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42374/0 (socket says 172.21.15.29:42374) 2024-04-25T00:37:49.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd455570640 1 -- 172.21.15.29:0/4208124989 learned_addr learned my addr 172.21.15.29:0/4208124989 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd44ffff640 1 --2- 172.21.15.29:0/4208124989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4500cf690 0x7fd4500d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd455570640 1 -- 172.21.15.29:0/4208124989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4500cf690 msgr2=0x7fd4500d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd455570640 1 --2- 172.21.15.29:0/4208124989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4500cf690 0x7fd4500d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd455570640 1 -- 172.21.15.29:0/4208124989 >> v2:172.21.15.157:3300/0 conn(0x7fd45015e200 msgr2=0x7fd4500d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd455570640 1 --2- 172.21.15.29:0/4208124989 >> v2:172.21.15.157:3300/0 conn(0x7fd45015e200 0x7fd4500d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.823+0000 7fd455570640 1 -- 172.21.15.29:0/4208124989 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4500cfef0 con 0x7fd4500d2a90 2024-04-25T00:37:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd455570640 1 --2- 172.21.15.29:0/4208124989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4500d2a90 0x7fd4500d05a0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fd45008d8f0 tx=0x7fd44801c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828fc57f4fa1652d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd44f7fe640 1 -- 172.21.15.29:0/4208124989 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480a5580 con 0x7fd4500d2a90 2024-04-25T00:37:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd44f7fe640 1 -- 172.21.15.29:0/4208124989 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4480a5720 con 0x7fd4500d2a90 2024-04-25T00:37:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd44f7fe640 1 -- 172.21.15.29:0/4208124989 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480a5af0 con 0x7fd4500d2a90 2024-04-25T00:37:49.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 -- 172.21.15.29:0/4208124989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4500d2a90 msgr2=0x7fd4500d05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:49.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 --2- 172.21.15.29:0/4208124989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4500d2a90 0x7fd4500d05a0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fd45008d8f0 tx=0x7fd44801c700 comp rx=0 tx=0).stop 2024-04-25T00:37:49.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 -- 172.21.15.29:0/4208124989 shutdown_connections 2024-04-25T00:37:49.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 --2- 172.21.15.29:0/4208124989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4500d2a90 0x7fd4500d05a0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:49.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 --2- 172.21.15.29:0/4208124989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4500cf690 0x7fd4500d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:49.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 --2- 172.21.15.29:0/4208124989 >> v2:172.21.15.157:3300/0 conn(0x7fd45015e200 0x7fd4500d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:49.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 -- 172.21.15.29:0/4208124989 >> 172.21.15.29:0/4208124989 conn(0x7fd45009d280 msgr2=0x7fd4500d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:49.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 -- 172.21.15.29:0/4208124989 shutdown_connections 2024-04-25T00:37:49.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 -- 172.21.15.29:0/4208124989 wait complete. 2024-04-25T00:37:49.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 Processor -- start 2024-04-25T00:37:49.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 -- start start 2024-04-25T00:37:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4500cf690 0x7fd450158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4500d2a90 0x7fd450160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45015e200 0x7fd450160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd45015e760 con 0x7fd4500cf690 2024-04-25T00:37:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd45015e5e0 con 0x7fd45015e200 2024-04-25T00:37:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd456fd3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd4500b4320 con 0x7fd4500d2a90 2024-04-25T00:37:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd454d6f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4500cf690 0x7fd450158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd454d6f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4500cf690 0x7fd450158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42390/0 (socket says 172.21.15.29:42390) 2024-04-25T00:37:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.827+0000 7fd454d6f640 1 -- 172.21.15.29:0/2373985366 learned_addr learned my addr 172.21.15.29:0/2373985366 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd455570640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45015e200 0x7fd450160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd44ffff640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4500d2a90 0x7fd450160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd454d6f640 1 -- 172.21.15.29:0/2373985366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4500d2a90 msgr2=0x7fd450160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd454d6f640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4500d2a90 0x7fd450160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd454d6f640 1 -- 172.21.15.29:0/2373985366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45015e200 msgr2=0x7fd450160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd454d6f640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45015e200 0x7fd450160fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd454d6f640 1 -- 172.21.15.29:0/2373985366 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4500b2cb0 con 0x7fd4500cf690 2024-04-25T00:37:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd454d6f640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4500cf690 0x7fd450158520 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fd44000b410 tx=0x7fd44000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd44d7fa640 1 -- 172.21.15.29:0/2373985366 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440015020 con 0x7fd4500cf690 2024-04-25T00:37:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd44d7fa640 1 -- 172.21.15.29:0/2373985366 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd440011830 con 0x7fd4500cf690 2024-04-25T00:37:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd456fd3640 1 -- 172.21.15.29:0/2373985366 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4500cb920 con 0x7fd4500cf690 2024-04-25T00:37:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd44d7fa640 1 -- 172.21.15.29:0/2373985366 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440011c20 con 0x7fd4500cf690 2024-04-25T00:37:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd456fd3640 1 -- 172.21.15.29:0/2373985366 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd45015fbd0 con 0x7fd4500cf690 2024-04-25T00:37:49.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd456fd3640 1 -- 172.21.15.29:0/2373985366 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd450069270 con 0x7fd4500cf690 2024-04-25T00:37:49.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd44d7fa640 1 -- 172.21.15.29:0/2373985366 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7fd440018020 con 0x7fd4500cf690 2024-04-25T00:37:49.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd44d7fa640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd43c067b90 0x7fd43c06a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:49.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd44d7fa640 1 -- 172.21.15.29:0/2373985366 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fd440085ec0 con 0x7fd4500cf690 2024-04-25T00:37:49.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd44ffff640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd43c067b90 0x7fd43c06a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:49.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.831+0000 7fd44ffff640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd43c067b90 0x7fd43c06a050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd444007a60 tx=0x7fd444008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:49.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:49.835+0000 7fd44d7fa640 1 -- 172.21.15.29:0/2373985366 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd44001f020 con 0x7fd4500cf690 2024-04-25T00:37:50.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.027+0000 7fd456fd3640 1 -- 172.21.15.29:0/2373985366 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol6790", "target": ["mon-mgr", ""]}) -- 0x7fd4500d3970 con 0x7fd43c067b90 2024-04-25T00:37:50.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.047+0000 7fd44d7fa640 1 -- 172.21.15.29:0/2373985366 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+66 (secure 0 0 0) 0x7fd4500d3970 con 0x7fd43c067b90 2024-04-25T00:37:50.051 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1 2024-04-25T00:37:50.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.051+0000 7fd456fd3640 1 -- 172.21.15.29:0/2373985366 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd43c067b90 msgr2=0x7fd43c06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:50.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.051+0000 7fd456fd3640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd43c067b90 0x7fd43c06a050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd444007a60 tx=0x7fd444008040 comp rx=0 tx=0).stop 2024-04-25T00:37:50.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.051+0000 7fd456fd3640 1 -- 172.21.15.29:0/2373985366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4500cf690 msgr2=0x7fd450158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:50.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.051+0000 7fd456fd3640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4500cf690 0x7fd450158520 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fd44000b410 tx=0x7fd44000b8d0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.051+0000 7fd456fd3640 1 -- 172.21.15.29:0/2373985366 shutdown_connections 2024-04-25T00:37:50.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.051+0000 7fd456fd3640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd43c067b90 0x7fd43c06a050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.051+0000 7fd456fd3640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45015e200 0x7fd450160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.051+0000 7fd456fd3640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4500d2a90 0x7fd450160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.051+0000 7fd456fd3640 1 --2- 172.21.15.29:0/2373985366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4500cf690 0x7fd450158520 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.051+0000 7fd456fd3640 1 -- 172.21.15.29:0/2373985366 >> 172.21.15.29:0/2373985366 conn(0x7fd45009d280 msgr2=0x7fd4500d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:50.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.051+0000 7fd456fd3640 1 -- 172.21.15.29:0/2373985366 shutdown_connections 2024-04-25T00:37:50.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.051+0000 7fd456fd3640 1 -- 172.21.15.29:0/2373985366 wait complete. 2024-04-25T00:37:50.071 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone39330 2024-04-25T00:37:50.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.199+0000 7f8ec1c34640 1 Processor -- start 2024-04-25T00:37:50.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.199+0000 7f8ec1c34640 1 -- start start 2024-04-25T00:37:50.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ec1c34640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8ebc15e160 0x7f8ebc06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:50.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ec1c34640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ebc0b4ce0 0x7f8ebc0715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:50.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ec1c34640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ebc06e3d0 con 0x7f8ebc071af0 2024-04-25T00:37:50.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ec1c34640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8ebc071f50 con 0x7f8ebc15e160 2024-04-25T00:37:50.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ec1c34640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8ebc06e6b0 con 0x7f8ebc0b4ce0 2024-04-25T00:37:50.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ebb7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8ebc15e160 0x7f8ebc06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:50.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ebbfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8ebc071af0 0x7f8ebc0b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46824/0 (socket says 172.21.15.29:46824) 2024-04-25T00:37:50.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ebbfff640 1 -- 172.21.15.29:0/3188074838 learned_addr learned my addr 172.21.15.29:0/3188074838 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:37:50.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ebaffd640 1 --2- 172.21.15.29:0/3188074838 >> v2:172.21.15.157:3300/0 conn(0x7f8ebc0b4ce0 0x7f8ebc0715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ebb7fe640 1 -- 172.21.15.29:0/3188074838 >> v2:172.21.15.157:3300/0 conn(0x7f8ebc0b4ce0 msgr2=0x7f8ebc0715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ebb7fe640 1 --2- 172.21.15.29:0/3188074838 >> v2:172.21.15.157:3300/0 conn(0x7f8ebc0b4ce0 0x7f8ebc0715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ebb7fe640 1 -- 172.21.15.29:0/3188074838 >> v1:172.21.15.29:6789/0 conn(0x7f8ebc071af0 legacy=0x7f8ebc0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ebb7fe640 1 -- 172.21.15.29:0/3188074838 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc0ae700 con 0x7f8ebc15e160 2024-04-25T00:37:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8ebb7fe640 1 --2- 172.21.15.29:0/3188074838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc15e160 0x7f8ebc06dd40 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0b2fc0 tx=0x7f8eac01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6179f91e17be08e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8eba7fc640 1 -- 172.21.15.29:0/3188074838 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0b0070 con 0x7f8ebc15e160 2024-04-25T00:37:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8eba7fc640 1 -- 172.21.15.29:0/3188074838 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac0ab600 con 0x7f8ebc15e160 2024-04-25T00:37:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.203+0000 7f8eba7fc640 1 -- 172.21.15.29:0/3188074838 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0ab9d0 con 0x7f8ebc15e160 2024-04-25T00:37:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 -- 172.21.15.29:0/3188074838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc15e160 msgr2=0x7f8ebc06dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 --2- 172.21.15.29:0/3188074838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc15e160 0x7f8ebc06dd40 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0b2fc0 tx=0x7f8eac01c3c0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 -- 172.21.15.29:0/3188074838 shutdown_connections 2024-04-25T00:37:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 --2- 172.21.15.29:0/3188074838 >> v2:172.21.15.157:3300/0 conn(0x7f8ebc0b4ce0 0x7f8ebc0715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 --2- 172.21.15.29:0/3188074838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc15e160 0x7f8ebc06dd40 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 -- 172.21.15.29:0/3188074838 >> 172.21.15.29:0/3188074838 conn(0x7f8ebc075030 msgr2=0x7f8ebc0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:50.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 -- 172.21.15.29:0/3188074838 shutdown_connections 2024-04-25T00:37:50.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 -- 172.21.15.29:0/3188074838 wait complete. 2024-04-25T00:37:50.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 Processor -- start 2024-04-25T00:37:50.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 -- start start 2024-04-25T00:37:50.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc071af0 0x7f8ebc15be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:50.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ebc0b4ce0 0x7f8ebc14f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:50.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc15e160 0x7f8ebc14f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:50.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc0ccfe0 con 0x7f8ebc071af0 2024-04-25T00:37:50.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc0cce60 con 0x7f8ebc15e160 2024-04-25T00:37:50.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8ebc0b50c0 con 0x7f8ebc0b4ce0 2024-04-25T00:37:50.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ebb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc071af0 0x7f8ebc15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:50.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ebb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc071af0 0x7f8ebc15be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42416/0 (socket says 172.21.15.29:42416) 2024-04-25T00:37:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ebbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc15e160 0x7f8ebc14f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ebb7fe640 1 -- 172.21.15.29:0/524134877 learned_addr learned my addr 172.21.15.29:0/524134877 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ebaffd640 1 --2- 172.21.15.29:0/524134877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ebc0b4ce0 0x7f8ebc14f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ebb7fe640 1 -- 172.21.15.29:0/524134877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ebc0b4ce0 msgr2=0x7f8ebc14f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ebb7fe640 1 --2- 172.21.15.29:0/524134877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ebc0b4ce0 0x7f8ebc14f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ebb7fe640 1 -- 172.21.15.29:0/524134877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc15e160 msgr2=0x7f8ebc14f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ebb7fe640 1 --2- 172.21.15.29:0/524134877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc15e160 0x7f8ebc14f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ebb7fe640 1 -- 172.21.15.29:0/524134877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc160820 con 0x7f8ebc071af0 2024-04-25T00:37:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ebb7fe640 1 --2- 172.21.15.29:0/524134877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc071af0 0x7f8ebc15be70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0ae040 tx=0x7f8eac01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ea3fff640 1 -- 172.21.15.29:0/524134877 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0b0050 con 0x7f8ebc071af0 2024-04-25T00:37:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ea3fff640 1 -- 172.21.15.29:0/524134877 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac0bc070 con 0x7f8ebc071af0 2024-04-25T00:37:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ea3fff640 1 -- 172.21.15.29:0/524134877 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0b5870 con 0x7f8ebc071af0 2024-04-25T00:37:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.207+0000 7f8ec1c34640 1 -- 172.21.15.29:0/524134877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ebc160b00 con 0x7f8ebc071af0 2024-04-25T00:37:50.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.211+0000 7f8ec1c34640 1 -- 172.21.15.29:0/524134877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ebc1610c0 con 0x7f8ebc071af0 2024-04-25T00:37:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.211+0000 7f8ec1c34640 1 -- 172.21.15.29:0/524134877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ebc069270 con 0x7f8ebc071af0 2024-04-25T00:37:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.211+0000 7f8ea3fff640 1 -- 172.21.15.29:0/524134877 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f8eac0b5a10 con 0x7f8ebc071af0 2024-04-25T00:37:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.211+0000 7f8ea3fff640 1 --2- 172.21.15.29:0/524134877 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ea8067b90 0x7f8ea806a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.211+0000 7f8ea3fff640 1 -- 172.21.15.29:0/524134877 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f8eac1208f0 con 0x7f8ebc071af0 2024-04-25T00:37:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.211+0000 7f8ebaffd640 1 --2- 172.21.15.29:0/524134877 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ea8067b90 0x7f8ea806a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.211+0000 7f8ebaffd640 1 --2- 172.21.15.29:0/524134877 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ea8067b90 0x7f8ea806a050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc151d80 tx=0x7f8eb0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:50.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.215+0000 7f8ea3fff640 1 -- 172.21.15.29:0/524134877 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8eac016070 con 0x7f8ebc071af0 2024-04-25T00:37:50.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.403+0000 7f8ec1c34640 1 -- 172.21.15.29:0/524134877 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone39330", "target": ["mon-mgr", ""]}) -- 0x7f8ebc15caa0 con 0x7f8ea8067b90 2024-04-25T00:37:50.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.411+0000 7f8ea3fff640 1 -- 172.21.15.29:0/524134877 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+73 (secure 0 0 0) 0x7f8ebc15caa0 con 0x7f8ea8067b90 2024-04-25T00:37:50.413 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9 2024-04-25T00:37:50.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.411+0000 7f8ec1c34640 1 -- 172.21.15.29:0/524134877 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ea8067b90 msgr2=0x7f8ea806a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:50.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.411+0000 7f8ec1c34640 1 --2- 172.21.15.29:0/524134877 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ea8067b90 0x7f8ea806a050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc151d80 tx=0x7f8eb0008040 comp rx=0 tx=0).stop 2024-04-25T00:37:50.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.411+0000 7f8ec1c34640 1 -- 172.21.15.29:0/524134877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc071af0 msgr2=0x7f8ebc15be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:50.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.411+0000 7f8ec1c34640 1 --2- 172.21.15.29:0/524134877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc071af0 0x7f8ebc15be70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0ae040 tx=0x7f8eac01ac00 comp rx=0 tx=0).stop 2024-04-25T00:37:50.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.415+0000 7f8ec1c34640 1 -- 172.21.15.29:0/524134877 shutdown_connections 2024-04-25T00:37:50.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.415+0000 7f8ec1c34640 1 --2- 172.21.15.29:0/524134877 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ea8067b90 0x7f8ea806a050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.415+0000 7f8ec1c34640 1 --2- 172.21.15.29:0/524134877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc15e160 0x7f8ebc14f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.415+0000 7f8ec1c34640 1 --2- 172.21.15.29:0/524134877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ebc0b4ce0 0x7f8ebc14f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.415+0000 7f8ec1c34640 1 --2- 172.21.15.29:0/524134877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc071af0 0x7f8ebc15be70 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.415+0000 7f8ec1c34640 1 -- 172.21.15.29:0/524134877 >> 172.21.15.29:0/524134877 conn(0x7f8ebc075030 msgr2=0x7f8ebc066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:50.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.415+0000 7f8ec1c34640 1 -- 172.21.15.29:0/524134877 shutdown_connections 2024-04-25T00:37:50.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.415+0000 7f8ec1c34640 1 -- 172.21.15.29:0/524134877 wait complete. 2024-04-25T00:37:50.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:50.433 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070) 2024-04-25T00:37:50.449 INFO:teuthology.orchestra.run.smithi029.stdout:9 2024-04-25T00:37:50.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:50.449 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9) 2024-04-25T00:37:50.503 INFO:teuthology.orchestra.run.smithi029.stdout:9 2024-04-25T00:37:50.504 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone39330 2024-04-25T00:37:50.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.679+0000 7f04106eb640 1 Processor -- start 2024-04-25T00:37:50.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.679+0000 7f04106eb640 1 -- start start 2024-04-25T00:37:50.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f04106eb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f04080b3c30 0x7f040806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:50.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f04106eb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f04080b4b10 0x7f04080ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:50.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f04106eb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04080b41e0 con 0x7f04080b3140 2024-04-25T00:37:50.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f04106eb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f040806d960 con 0x7f04080b3c30 2024-04-25T00:37:50.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f04106eb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f040806dc40 con 0x7f04080b4b10 2024-04-25T00:37:50.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f040dc86640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f04080b3c30 0x7f040806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:50.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f040dc86640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f04080b3c30 0x7f040806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42424/0 (socket says 172.21.15.29:42424) 2024-04-25T00:37:50.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f040dc86640 1 -- 172.21.15.29:0/1079120463 learned_addr learned my addr 172.21.15.29:0/1079120463 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:50.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f040ec88640 1 --2- 172.21.15.29:0/1079120463 >> v2:172.21.15.157:3300/0 conn(0x7f04080b4b10 0x7f04080ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:50.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f040dc86640 1 -- 172.21.15.29:0/1079120463 >> v2:172.21.15.157:3300/0 conn(0x7f04080b4b10 msgr2=0x7f04080ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:50.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f040dc86640 1 --2- 172.21.15.29:0/1079120463 >> v2:172.21.15.157:3300/0 conn(0x7f04080b4b10 0x7f04080ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f040dc86640 1 -- 172.21.15.29:0/1079120463 >> v1:172.21.15.157:6789/0 conn(0x7f04080b3140 legacy=0x7f04080b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:50.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f040dc86640 1 -- 172.21.15.29:0/1079120463 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040806df20 con 0x7f04080b3c30 2024-04-25T00:37:50.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f040ec88640 1 --2- 172.21.15.29:0/1079120463 >> v2:172.21.15.157:3300/0 conn(0x7f04080b4b10 0x7f04080ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:50.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f040dc86640 1 --2- 172.21.15.29:0/1079120463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04080b3c30 0x7f040806e440 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009e50 tx=0x7f03fc01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556710dbed6cd1cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:50.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f040d485640 1 -- 172.21.15.29:0/1079120463 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0a5670 con 0x7f04080b3c30 2024-04-25T00:37:50.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f040d485640 1 -- 172.21.15.29:0/1079120463 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03fc0a5810 con 0x7f04080b3c30 2024-04-25T00:37:50.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.683+0000 7f040d485640 1 -- 172.21.15.29:0/1079120463 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0a5be0 con 0x7f04080b3c30 2024-04-25T00:37:50.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 -- 172.21.15.29:0/1079120463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04080b3c30 msgr2=0x7f040806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:50.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 --2- 172.21.15.29:0/1079120463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04080b3c30 0x7f040806e440 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009e50 tx=0x7f03fc01c9b0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 -- 172.21.15.29:0/1079120463 shutdown_connections 2024-04-25T00:37:50.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 --2- 172.21.15.29:0/1079120463 >> v2:172.21.15.157:3300/0 conn(0x7f04080b4b10 0x7f04080ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 --2- 172.21.15.29:0/1079120463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04080b3c30 0x7f040806e440 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 -- 172.21.15.29:0/1079120463 >> 172.21.15.29:0/1079120463 conn(0x7f04080b1870 msgr2=0x7f04080b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:50.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 -- 172.21.15.29:0/1079120463 shutdown_connections 2024-04-25T00:37:50.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 -- 172.21.15.29:0/1079120463 wait complete. 2024-04-25T00:37:50.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 Processor -- start 2024-04-25T00:37:50.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 -- start start 2024-04-25T00:37:50.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04080b3140 0x7f040813d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:50.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04080b3c30 0x7f040815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:50.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04080b4b10 0x7f04081414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:50.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0408168d80 con 0x7f04080b3140 2024-04-25T00:37:50.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0408168c00 con 0x7f04080b3c30 2024-04-25T00:37:50.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0408168f00 con 0x7f04080b4b10 2024-04-25T00:37:50.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f040e487640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04080b3140 0x7f040813d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:50.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f040e487640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04080b3140 0x7f040813d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42430/0 (socket says 172.21.15.29:42430) 2024-04-25T00:37:50.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f040e487640 1 -- 172.21.15.29:0/3336961801 learned_addr learned my addr 172.21.15.29:0/3336961801 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:50.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f040ec88640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04080b4b10 0x7f04081414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:50.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f040dc86640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04080b3c30 0x7f040815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:50.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f040e487640 1 -- 172.21.15.29:0/3336961801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04080b4b10 msgr2=0x7f04081414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:50.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f040e487640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04080b4b10 0x7f04081414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f040e487640 1 -- 172.21.15.29:0/3336961801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04080b3c30 msgr2=0x7f040815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:50.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f040e487640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04080b3c30 0x7f040815a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f040e487640 1 -- 172.21.15.29:0/3336961801 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040815b640 con 0x7f04080b3140 2024-04-25T00:37:50.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f040dc86640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04080b3c30 0x7f040815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:50.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f040e487640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04080b3140 0x7f040813d420 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f040400b410 tx=0x7f040400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:50.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f03faffd640 1 -- 172.21.15.29:0/3336961801 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404015030 con 0x7f04080b3140 2024-04-25T00:37:50.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f03faffd640 1 -- 172.21.15.29:0/3336961801 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0404004020 con 0x7f04080b3140 2024-04-25T00:37:50.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 -- 172.21.15.29:0/3336961801 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040815b920 con 0x7f04080b3140 2024-04-25T00:37:50.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f03faffd640 1 -- 172.21.15.29:0/3336961801 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404004450 con 0x7f04080b3140 2024-04-25T00:37:50.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.687+0000 7f04106eb640 1 -- 172.21.15.29:0/3336961801 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0408154a70 con 0x7f04080b3140 2024-04-25T00:37:50.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.691+0000 7f04106eb640 1 -- 172.21.15.29:0/3336961801 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0408154580 con 0x7f04080b3140 2024-04-25T00:37:50.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.691+0000 7f03faffd640 1 -- 172.21.15.29:0/3336961801 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f0404018070 con 0x7f04080b3140 2024-04-25T00:37:50.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.691+0000 7f03faffd640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f03ec067b90 0x7f03ec06a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:50.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.691+0000 7f03faffd640 1 -- 172.21.15.29:0/3336961801 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f0404085c80 con 0x7f04080b3140 2024-04-25T00:37:50.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.691+0000 7f040dc86640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f03ec067b90 0x7f03ec06a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:50.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.691+0000 7f040dc86640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f03ec067b90 0x7f03ec06a050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0ae000 tx=0x7f03fc0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:50.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.695+0000 7f03faffd640 1 -- 172.21.15.29:0/3336961801 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0404056440 con 0x7f04080b3140 2024-04-25T00:37:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.883+0000 7f04106eb640 1 -- 172.21.15.29:0/3336961801 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone39330", "target": ["mon-mgr", ""]}) -- 0x7f040815b3f0 con 0x7f03ec067b90 2024-04-25T00:37:50.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.899+0000 7f03faffd640 1 -- 172.21.15.29:0/3336961801 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+724 (secure 0 0 0) 0x7f040815b3f0 con 0x7f03ec067b90 2024-04-25T00:37:50.902 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:37:50.902 INFO:teuthology.orchestra.run.smithi029.stdout: "atime": "2024-04-25 00:37:42", 2024-04-25T00:37:50.902 INFO:teuthology.orchestra.run.smithi029.stdout: "bytes_pcent": "80.00", 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "bytes_quota": 10485760, 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "bytes_used": 8388608, 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "created_at": "2024-04-25 00:37:46", 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "ctime": "2024-04-25 00:37:47", 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "data_pool": "cephfs_data", 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "features": [ 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot-clone", 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot-autoprotect", 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot-retention" 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "flavor": 2, 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "gid": 1000, 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "mode": 16895, 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_addrs": [ 2024-04-25T00:37:50.903 INFO:teuthology.orchestra.run.smithi029.stdout: "172.21.15.29:6789", 2024-04-25T00:37:50.904 INFO:teuthology.orchestra.run.smithi029.stdout: "172.21.15.157:6789", 2024-04-25T00:37:50.904 INFO:teuthology.orchestra.run.smithi029.stdout: "172.21.15.157:6790" 2024-04-25T00:37:50.904 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-04-25T00:37:50.904 INFO:teuthology.orchestra.run.smithi029.stdout: "mtime": "2024-04-25 00:37:46", 2024-04-25T00:37:50.904 INFO:teuthology.orchestra.run.smithi029.stdout: "path": "/volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9", 2024-04-25T00:37:50.904 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_namespace": "", 2024-04-25T00:37:50.904 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "complete", 2024-04-25T00:37:50.904 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "clone", 2024-04-25T00:37:50.904 INFO:teuthology.orchestra.run.smithi029.stdout: "uid": 1000 2024-04-25T00:37:50.904 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:37:50.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.903+0000 7f04106eb640 1 -- 172.21.15.29:0/3336961801 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f03ec067b90 msgr2=0x7f03ec06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:50.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.903+0000 7f04106eb640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f03ec067b90 0x7f03ec06a050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0ae000 tx=0x7f03fc0a7070 comp rx=0 tx=0).stop 2024-04-25T00:37:50.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.903+0000 7f04106eb640 1 -- 172.21.15.29:0/3336961801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04080b3140 msgr2=0x7f040813d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:50.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.903+0000 7f04106eb640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04080b3140 0x7f040813d420 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f040400b410 tx=0x7f040400b8d0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.903+0000 7f04106eb640 1 -- 172.21.15.29:0/3336961801 shutdown_connections 2024-04-25T00:37:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.903+0000 7f04106eb640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f03ec067b90 0x7f03ec06a050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.903+0000 7f04106eb640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04080b4b10 0x7f04081414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.903+0000 7f04106eb640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04080b3c30 0x7f040815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.903+0000 7f04106eb640 1 --2- 172.21.15.29:0/3336961801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04080b3140 0x7f040813d420 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.903+0000 7f04106eb640 1 -- 172.21.15.29:0/3336961801 >> 172.21.15.29:0/3336961801 conn(0x7f04080b1870 msgr2=0x7f04080b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.903+0000 7f04106eb640 1 -- 172.21.15.29:0/3336961801 shutdown_connections 2024-04-25T00:37:50.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:50.903+0000 7f04106eb640 1 -- 172.21.15.29:0/3336961801 wait complete. 2024-04-25T00:37:50.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:50.922 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070) 2024-04-25T00:37:50.936 INFO:teuthology.orchestra.run.smithi029.stdout:10485760 2024-04-25T00:37:50.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:50.936 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070) 2024-04-25T00:37:50.992 INFO:teuthology.orchestra.run.smithi029.stdout:cephfs_data 2024-04-25T00:37:50.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:50.992 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070) 2024-04-25T00:37:51.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.053 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070) 2024-04-25T00:37:51.119 INFO:teuthology.orchestra.run.smithi029.stdout:volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070 2024-04-25T00:37:51.119 INFO:teuthology.orchestra.run.smithi029.stdout:volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.2 2024-04-25T00:37:51.119 INFO:teuthology.orchestra.run.smithi029.stdout:volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.5 2024-04-25T00:37:51.119 INFO:teuthology.orchestra.run.smithi029.stdout:volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.4 2024-04-25T00:37:51.119 INFO:teuthology.orchestra.run.smithi029.stdout:volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.7 2024-04-25T00:37:51.119 INFO:teuthology.orchestra.run.smithi029.stdout:volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.6 2024-04-25T00:37:51.119 INFO:teuthology.orchestra.run.smithi029.stdout:volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.0 2024-04-25T00:37:51.119 INFO:teuthology.orchestra.run.smithi029.stdout:volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.3 2024-04-25T00:37:51.119 INFO:teuthology.orchestra.run.smithi029.stdout:volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.1 2024-04-25T00:37:51.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.120 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070) 2024-04-25T00:37:51.168 INFO:teuthology.orchestra.run.smithi029.stdout:41ff 2024-04-25T00:37:51.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.169 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/) 2024-04-25T00:37:51.217 INFO:teuthology.orchestra.run.smithi029.stdout:41ff 2024-04-25T00:37:51.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.218 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070) 2024-04-25T00:37:51.269 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:51.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.270 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/) 2024-04-25T00:37:51.318 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:51.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.323 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070) 2024-04-25T00:37:51.368 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:51.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.369 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/) 2024-04-25T00:37:51.418 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:51.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.419 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070) 2024-04-25T00:37:51.469 INFO:teuthology.orchestra.run.smithi029.stdout:1714005466 2024-04-25T00:37:51.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.470 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/) 2024-04-25T00:37:51.518 INFO:teuthology.orchestra.run.smithi029.stdout:1714005466 2024-04-25T00:37:51.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.519 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.2) 2024-04-25T00:37:51.568 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:51.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.570 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.2) 2024-04-25T00:37:51.630 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:51.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.631 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.2) 2024-04-25T00:37:51.685 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:51.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.686 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.2) 2024-04-25T00:37:51.735 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:51.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.736 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.2) 2024-04-25T00:37:51.790 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:51.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.791 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.2) 2024-04-25T00:37:51.843 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:51.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.844 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.2) 2024-04-25T00:37:51.898 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:51.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.899 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.2) 2024-04-25T00:37:51.949 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:51.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:51.951 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.5) 2024-04-25T00:37:52.006 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:52.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.007 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.5) 2024-04-25T00:37:52.066 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:52.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.068 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.5) 2024-04-25T00:37:52.122 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:52.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.123 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.5) 2024-04-25T00:37:52.170 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:52.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.171 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.5) 2024-04-25T00:37:52.221 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:52.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.222 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.5) 2024-04-25T00:37:52.270 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:52.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.271 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.5) 2024-04-25T00:37:52.325 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:52.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.326 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.5) 2024-04-25T00:37:52.374 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:52.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.375 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.4) 2024-04-25T00:37:52.430 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:52.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.431 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.4) 2024-04-25T00:37:52.490 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:52.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.491 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.4) 2024-04-25T00:37:52.546 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:52.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.547 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.4) 2024-04-25T00:37:52.594 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:52.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.595 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.4) 2024-04-25T00:37:52.649 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:52.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.649 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.4) 2024-04-25T00:37:52.698 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:52.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.699 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.4) 2024-04-25T00:37:52.753 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:52.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.754 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.4) 2024-04-25T00:37:52.807 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:52.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.808 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.7) 2024-04-25T00:37:52.867 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:52.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.867 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.7) 2024-04-25T00:37:52.943 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:52.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.943 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.7) 2024-04-25T00:37:52.997 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:52.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:52.998 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.7) 2024-04-25T00:37:53.060 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:53.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.061 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.7) 2024-04-25T00:37:53.120 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:53.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.122 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.7) 2024-04-25T00:37:53.174 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:53.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.175 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.7) 2024-04-25T00:37:53.233 INFO:teuthology.orchestra.run.smithi029.stdout:1714005466 2024-04-25T00:37:53.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.234 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.7) 2024-04-25T00:37:53.284 INFO:teuthology.orchestra.run.smithi029.stdout:1714005466 2024-04-25T00:37:53.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.286 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.6) 2024-04-25T00:37:53.343 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:53.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.344 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.6) 2024-04-25T00:37:53.401 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:53.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.402 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.6) 2024-04-25T00:37:53.454 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:53.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.454 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.6) 2024-04-25T00:37:53.505 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:53.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.506 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.6) 2024-04-25T00:37:53.557 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:53.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.558 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.6) 2024-04-25T00:37:53.613 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:53.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.614 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.6) 2024-04-25T00:37:53.672 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:53.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.673 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.6) 2024-04-25T00:37:53.730 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:53.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.731 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.0) 2024-04-25T00:37:53.785 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:53.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.790 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.0) 2024-04-25T00:37:53.843 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:53.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.844 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.0) 2024-04-25T00:37:53.898 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:53.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.900 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.0) 2024-04-25T00:37:53.952 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:53.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:53.953 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.0) 2024-04-25T00:37:54.011 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:54.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.012 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.0) 2024-04-25T00:37:54.063 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:54.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.064 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.0) 2024-04-25T00:37:54.118 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:54.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.119 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.0) 2024-04-25T00:37:54.167 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:54.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.168 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.3) 2024-04-25T00:37:54.221 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:54.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.222 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.3) 2024-04-25T00:37:54.276 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:54.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.277 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.3) 2024-04-25T00:37:54.329 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:54.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.331 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.3) 2024-04-25T00:37:54.381 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:54.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.382 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.3) 2024-04-25T00:37:54.431 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:54.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.432 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.3) 2024-04-25T00:37:54.483 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:54.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.484 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.3) 2024-04-25T00:37:54.537 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:54.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.538 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.3) 2024-04-25T00:37:54.587 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:54.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.588 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.1) 2024-04-25T00:37:54.641 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:54.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.643 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.1) 2024-04-25T00:37:54.691 INFO:teuthology.orchestra.run.smithi029.stdout:81b4 2024-04-25T00:37:54.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.692 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.1) 2024-04-25T00:37:54.744 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:54.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.745 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.1) 2024-04-25T00:37:54.793 INFO:teuthology.orchestra.run.smithi029.stdout:1000 2024-04-25T00:37:54.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.794 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.1) 2024-04-25T00:37:54.845 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:54.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.846 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.1) 2024-04-25T00:37:54.894 INFO:teuthology.orchestra.run.smithi029.stdout:1271 2024-04-25T00:37:54.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.895 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1/.snap/subvol_snap26070/subvolume_file.1) 2024-04-25T00:37:54.945 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:54.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:37:54.947 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39330/29f118b2-e5e9-4fac-95d0-59006a1964c9/subvolume_file.1) 2024-04-25T00:37:54.995 INFO:teuthology.orchestra.run.smithi029.stdout:1714005465 2024-04-25T00:37:54.996 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol6790 fake subvol_clone3071 2024-04-25T00:37:55.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d496b5640 1 Processor -- start 2024-04-25T00:37:55.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d496b5640 1 -- start start 2024-04-25T00:37:55.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d496b5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d44162420 0x7f7d44070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:55.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d496b5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d440b4e00 0x7f7d440b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:55.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d496b5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d440b30b0 0x7f7d44166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:55.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d496b5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7d440b3500 con 0x7f7d440b4e00 2024-04-25T00:37:55.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d496b5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7d44071910 con 0x7f7d44162420 2024-04-25T00:37:55.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d496b5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7d44071bf0 con 0x7f7d440b30b0 2024-04-25T00:37:55.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d3bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d440b4e00 0x7f7d440b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:55.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d3bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d440b4e00 0x7f7d440b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42438/0 (socket says 172.21.15.29:42438) 2024-04-25T00:37:55.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d3bfff640 1 -- 172.21.15.29:0/2082668807 learned_addr learned my addr 172.21.15.29:0/2082668807 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:55.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d437fe640 1 --2- 172.21.15.29:0/2082668807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d440b30b0 0x7f7d44166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:55.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d42ffd640 1 --2- 172.21.15.29:0/2082668807 >> v2:172.21.15.157:3300/0 conn(0x7f7d44162420 0x7f7d44070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:55.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d3bfff640 1 -- 172.21.15.29:0/2082668807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d440b30b0 msgr2=0x7f7d44166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:55.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d3bfff640 1 --2- 172.21.15.29:0/2082668807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d440b30b0 0x7f7d44166020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d3bfff640 1 -- 172.21.15.29:0/2082668807 >> v2:172.21.15.157:3300/0 conn(0x7f7d44162420 msgr2=0x7f7d44070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:55.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d3bfff640 1 --2- 172.21.15.29:0/2082668807 >> v2:172.21.15.157:3300/0 conn(0x7f7d44162420 0x7f7d44070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d3bfff640 1 -- 172.21.15.29:0/2082668807 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d44071ed0 con 0x7f7d440b4e00 2024-04-25T00:37:55.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.179+0000 7f7d437fe640 1 --2- 172.21.15.29:0/2082668807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d440b30b0 0x7f7d44166020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:55.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d3bfff640 1 --2- 172.21.15.29:0/2082668807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d440b4e00 0x7f7d440b2b70 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f7d44067e70 tx=0x7f7d2c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03b83de05b20dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:55.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d427fc640 1 -- 172.21.15.29:0/2082668807 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0b0070 con 0x7f7d440b4e00 2024-04-25T00:37:55.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d427fc640 1 -- 172.21.15.29:0/2082668807 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d2c0ab600 con 0x7f7d440b4e00 2024-04-25T00:37:55.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d427fc640 1 -- 172.21.15.29:0/2082668807 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0ab9d0 con 0x7f7d440b4e00 2024-04-25T00:37:55.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 -- 172.21.15.29:0/2082668807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d440b4e00 msgr2=0x7f7d440b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:55.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 --2- 172.21.15.29:0/2082668807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d440b4e00 0x7f7d440b2b70 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f7d44067e70 tx=0x7f7d2c01c260 comp rx=0 tx=0).stop 2024-04-25T00:37:55.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 -- 172.21.15.29:0/2082668807 shutdown_connections 2024-04-25T00:37:55.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 --2- 172.21.15.29:0/2082668807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d440b4e00 0x7f7d440b2b70 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 --2- 172.21.15.29:0/2082668807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d440b30b0 0x7f7d44166020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 --2- 172.21.15.29:0/2082668807 >> v2:172.21.15.157:3300/0 conn(0x7f7d44162420 0x7f7d44070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 -- 172.21.15.29:0/2082668807 >> 172.21.15.29:0/2082668807 conn(0x7f7d440b1870 msgr2=0x7f7d440b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:55.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 -- 172.21.15.29:0/2082668807 shutdown_connections 2024-04-25T00:37:55.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 -- 172.21.15.29:0/2082668807 wait complete. 2024-04-25T00:37:55.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 Processor -- start 2024-04-25T00:37:55.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 -- start start 2024-04-25T00:37:55.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d440b30b0 0x7f7d44143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:55.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b4e00 0x7f7d441602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:55.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d44162420 0x7f7d44160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:55.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4406cf60 con 0x7f7d440b30b0 2024-04-25T00:37:55.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d440716d0 con 0x7f7d440b4e00 2024-04-25T00:37:55.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.183+0000 7f7d496b5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4406d0e0 con 0x7f7d44162420 2024-04-25T00:37:55.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d42ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d440b30b0 0x7f7d44143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:55.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d42ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d440b30b0 0x7f7d44143c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42450/0 (socket says 172.21.15.29:42450) 2024-04-25T00:37:55.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d42ffd640 1 -- 172.21.15.29:0/1298584750 learned_addr learned my addr 172.21.15.29:0/1298584750 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:55.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d3bfff640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b4e00 0x7f7d441602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:55.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d437fe640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d44162420 0x7f7d44160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:55.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d42ffd640 1 -- 172.21.15.29:0/1298584750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d44162420 msgr2=0x7f7d44160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:55.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d42ffd640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d44162420 0x7f7d44160800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d42ffd640 1 -- 172.21.15.29:0/1298584750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b4e00 msgr2=0x7f7d441602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:55.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d42ffd640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b4e00 0x7f7d441602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d42ffd640 1 -- 172.21.15.29:0/1298584750 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d44164b20 con 0x7f7d440b30b0 2024-04-25T00:37:55.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d42ffd640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d440b30b0 0x7f7d44143c50 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400d5f0 tx=0x7f7d3400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:55.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d3b7fe640 1 -- 172.21.15.29:0/1298584750 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d340125f0 con 0x7f7d440b30b0 2024-04-25T00:37:55.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d3b7fe640 1 -- 172.21.15.29:0/1298584750 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d34019070 con 0x7f7d440b30b0 2024-04-25T00:37:55.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d496b5640 1 -- 172.21.15.29:0/1298584750 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d44164e00 con 0x7f7d440b30b0 2024-04-25T00:37:55.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d3b7fe640 1 -- 172.21.15.29:0/1298584750 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d340129b0 con 0x7f7d440b30b0 2024-04-25T00:37:55.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d496b5640 1 -- 172.21.15.29:0/1298584750 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d441653c0 con 0x7f7d440b30b0 2024-04-25T00:37:55.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d496b5640 1 -- 172.21.15.29:0/1298584750 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d44165730 con 0x7f7d440b30b0 2024-04-25T00:37:55.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d3b7fe640 1 -- 172.21.15.29:0/1298584750 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f7d34012b50 con 0x7f7d440b30b0 2024-04-25T00:37:55.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d3b7fe640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d24067b90 0x7f7d2406a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:55.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d3b7fe640 1 -- 172.21.15.29:0/1298584750 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f7d34086460 con 0x7f7d440b30b0 2024-04-25T00:37:55.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d3bfff640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d24067b90 0x7f7d2406a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:55.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.187+0000 7f7d3bfff640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d24067b90 0x7f7d2406a050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0ad000 tx=0x7f7d2c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:55.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.191+0000 7f7d3b7fe640 1 -- 172.21.15.29:0/1298584750 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7d44165730 con 0x7f7d440b30b0 2024-04-25T00:37:55.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.387+0000 7f7d496b5640 1 -- 172.21.15.29:0/1298584750 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol6790", "snap_name": "fake", "target_sub_name": "subvol_clone3071", "target": ["mon-mgr", ""]}) -- 0x7f7d44071160 con 0x7f7d24067b90 2024-04-25T00:37:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.511+0000 7f7d3b7fe640 1 -- 172.21.15.29:0/1298584750 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7d44071160 con 0x7f7d24067b90 2024-04-25T00:37:55.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.511+0000 7f7d496b5640 1 -- 172.21.15.29:0/1298584750 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d24067b90 msgr2=0x7f7d2406a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.511+0000 7f7d496b5640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d24067b90 0x7f7d2406a050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0ad000 tx=0x7f7d2c008040 comp rx=0 tx=0).stop 2024-04-25T00:37:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.511+0000 7f7d496b5640 1 -- 172.21.15.29:0/1298584750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d440b30b0 msgr2=0x7f7d44143c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.511+0000 7f7d496b5640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d440b30b0 0x7f7d44143c50 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400d5f0 tx=0x7f7d3400dab0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.511+0000 7f7d496b5640 1 -- 172.21.15.29:0/1298584750 shutdown_connections 2024-04-25T00:37:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.511+0000 7f7d496b5640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d24067b90 0x7f7d2406a050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.511+0000 7f7d496b5640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d44162420 0x7f7d44160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.511+0000 7f7d496b5640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b4e00 0x7f7d441602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.511+0000 7f7d496b5640 1 --2- 172.21.15.29:0/1298584750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d440b30b0 0x7f7d44143c50 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.511+0000 7f7d496b5640 1 -- 172.21.15.29:0/1298584750 >> 172.21.15.29:0/1298584750 conn(0x7f7d440b1870 msgr2=0x7f7d440af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.511+0000 7f7d496b5640 1 -- 172.21.15.29:0/1298584750 shutdown_connections 2024-04-25T00:37:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.511+0000 7f7d496b5640 1 -- 172.21.15.29:0/1298584750 wait complete. 2024-04-25T00:37:55.536 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone3071 2024-04-25T00:37:55.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18b0f29640 1 Processor -- start 2024-04-25T00:37:55.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18b0f29640 1 -- start start 2024-04-25T00:37:55.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18b0f29640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f18ac0aec60 0x7f18ac0cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:55.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18b0f29640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18ac0af040 0x7f18ac15c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:55.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18b0f29640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18ac15ccd0 con 0x7f18ac15d720 2024-04-25T00:37:55.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18b0f29640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f18ac15caa0 con 0x7f18ac0aec60 2024-04-25T00:37:55.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18b0f29640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f18ac15cfb0 con 0x7f18ac0af040 2024-04-25T00:37:55.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18aa59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f18ac0aec60 0x7f18ac0cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:55.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18aa59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f18ac0aec60 0x7f18ac0cfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42456/0 (socket says 172.21.15.29:42456) 2024-04-25T00:37:55.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18aa59c640 1 -- 172.21.15.29:0/2954237172 learned_addr learned my addr 172.21.15.29:0/2954237172 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:55.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18a9d9b640 1 --2- 172.21.15.29:0/2954237172 >> v2:172.21.15.157:3300/0 conn(0x7f18ac0af040 0x7f18ac15c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:55.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18aa59c640 1 -- 172.21.15.29:0/2954237172 >> v2:172.21.15.157:3300/0 conn(0x7f18ac0af040 msgr2=0x7f18ac15c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:55.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18aa59c640 1 --2- 172.21.15.29:0/2954237172 >> v2:172.21.15.157:3300/0 conn(0x7f18ac0af040 0x7f18ac15c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18aa59c640 1 -- 172.21.15.29:0/2954237172 >> v1:172.21.15.157:6789/0 conn(0x7f18ac15d720 legacy=0x7f18ac0b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:55.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.667+0000 7f18aa59c640 1 -- 172.21.15.29:0/2954237172 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ac15d290 con 0x7f18ac0aec60 2024-04-25T00:37:55.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18aa59c640 1 --2- 172.21.15.29:0/2954237172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac0aec60 0x7f18ac0cfe40 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0cf4b0 tx=0x7f189401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f647f07c00e94a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:55.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18a959a640 1 -- 172.21.15.29:0/2954237172 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18940b0070 con 0x7f18ac0aec60 2024-04-25T00:37:55.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18a959a640 1 -- 172.21.15.29:0/2954237172 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18940ab600 con 0x7f18ac0aec60 2024-04-25T00:37:55.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18a959a640 1 -- 172.21.15.29:0/2954237172 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18940ab9d0 con 0x7f18ac0aec60 2024-04-25T00:37:55.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 -- 172.21.15.29:0/2954237172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac0aec60 msgr2=0x7f18ac0cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:55.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 --2- 172.21.15.29:0/2954237172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac0aec60 0x7f18ac0cfe40 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0cf4b0 tx=0x7f189401c450 comp rx=0 tx=0).stop 2024-04-25T00:37:55.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 -- 172.21.15.29:0/2954237172 shutdown_connections 2024-04-25T00:37:55.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 --2- 172.21.15.29:0/2954237172 >> v2:172.21.15.157:3300/0 conn(0x7f18ac0af040 0x7f18ac15c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 --2- 172.21.15.29:0/2954237172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac0aec60 0x7f18ac0cfe40 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 -- 172.21.15.29:0/2954237172 >> 172.21.15.29:0/2954237172 conn(0x7f18ac075030 msgr2=0x7f18ac0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:55.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 -- 172.21.15.29:0/2954237172 shutdown_connections 2024-04-25T00:37:55.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 -- 172.21.15.29:0/2954237172 wait complete. 2024-04-25T00:37:55.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 Processor -- start 2024-04-25T00:37:55.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 -- start start 2024-04-25T00:37:55.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ac0aec60 0x7f18ac0ab610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:55.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac0af040 0x7f18ac14f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:55.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18ac15d720 0x7f18ac1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:55.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac168d20 con 0x7f18ac0af040 2024-04-25T00:37:55.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac168ba0 con 0x7f18ac15d720 2024-04-25T00:37:55.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18b0f29640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f18ac168ea0 con 0x7f18ac0aec60 2024-04-25T00:37:55.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18a9d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac0af040 0x7f18ac14f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:55.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18a9d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac0af040 0x7f18ac14f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42464/0 (socket says 172.21.15.29:42464) 2024-04-25T00:37:55.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18a9d9b640 1 -- 172.21.15.29:0/310872353 learned_addr learned my addr 172.21.15.29:0/310872353 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:37:55.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18aad9d640 1 --2- 172.21.15.29:0/310872353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18ac15d720 0x7f18ac1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:55.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18a9d9b640 1 -- 172.21.15.29:0/310872353 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ac0aec60 msgr2=0x7f18ac0ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:55.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18a9d9b640 1 --2- 172.21.15.29:0/310872353 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ac0aec60 0x7f18ac0ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18a9d9b640 1 -- 172.21.15.29:0/310872353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18ac15d720 msgr2=0x7f18ac1556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:55.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18a9d9b640 1 --2- 172.21.15.29:0/310872353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18ac15d720 0x7f18ac1556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:55.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18a9d9b640 1 -- 172.21.15.29:0/310872353 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ac14fc50 con 0x7f18ac0af040 2024-04-25T00:37:55.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.671+0000 7f18a9d9b640 1 --2- 172.21.15.29:0/310872353 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac0af040 0x7f18ac14f710 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f189c007e70 tx=0x7f189c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:55.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.675+0000 7f18a2ffd640 1 -- 172.21.15.29:0/310872353 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c01a070 con 0x7f18ac0af040 2024-04-25T00:37:55.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.675+0000 7f18a2ffd640 1 -- 172.21.15.29:0/310872353 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189c003ba0 con 0x7f18ac0af040 2024-04-25T00:37:55.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.675+0000 7f18b0f29640 1 -- 172.21.15.29:0/310872353 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ac0acd40 con 0x7f18ac0af040 2024-04-25T00:37:55.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.675+0000 7f18a2ffd640 1 -- 172.21.15.29:0/310872353 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c004030 con 0x7f18ac0af040 2024-04-25T00:37:55.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.675+0000 7f18b0f29640 1 -- 172.21.15.29:0/310872353 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ac0abf80 con 0x7f18ac0af040 2024-04-25T00:37:55.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.675+0000 7f18b0f29640 1 -- 172.21.15.29:0/310872353 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ac0ac980 con 0x7f18ac0af040 2024-04-25T00:37:55.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.675+0000 7f18a2ffd640 1 -- 172.21.15.29:0/310872353 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f189c01d070 con 0x7f18ac0af040 2024-04-25T00:37:55.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.675+0000 7f18a2ffd640 1 --2- 172.21.15.29:0/310872353 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f188c067bc0 0x7f188c06a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:55.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.675+0000 7f18a2ffd640 1 -- 172.21.15.29:0/310872353 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f189c086280 con 0x7f18ac0af040 2024-04-25T00:37:55.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.675+0000 7f18aa59c640 1 --2- 172.21.15.29:0/310872353 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f188c067bc0 0x7f188c06a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:55.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.675+0000 7f18aa59c640 1 --2- 172.21.15.29:0/310872353 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f188c067bc0 0x7f188c06a080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f18940ad000 tx=0x7f1894008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:55.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.679+0000 7f18a2ffd640 1 -- 172.21.15.29:0/310872353 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f189c056420 con 0x7f18ac0af040 2024-04-25T00:37:55.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:55.875+0000 7f18b0f29640 1 -- 172.21.15.29:0/310872353 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone3071", "target": ["mon-mgr", ""]}) -- 0x7f18ac069270 con 0x7f188c067bc0 2024-04-25T00:37:59.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:59.619+0000 7f18a2ffd640 1 -- 172.21.15.29:0/310872353 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+152 (secure 0 0 0) 0x7f18ac069270 con 0x7f188c067bc0 2024-04-25T00:37:59.623 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:37:59.623 INFO:teuthology.orchestra.run.smithi029.stdout: "status": { 2024-04-25T00:37:59.623 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "pending", 2024-04-25T00:37:59.623 INFO:teuthology.orchestra.run.smithi029.stdout: "source": { 2024-04-25T00:37:59.623 INFO:teuthology.orchestra.run.smithi029.stdout: "volume": "cephfs", 2024-04-25T00:37:59.623 INFO:teuthology.orchestra.run.smithi029.stdout: "subvolume": "subvol6790", 2024-04-25T00:37:59.623 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot": "fake" 2024-04-25T00:37:59.624 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:59.624 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:37:59.624 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:37:59.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:59.623+0000 7f18b0f29640 1 -- 172.21.15.29:0/310872353 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f188c067bc0 msgr2=0x7f188c06a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:59.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:59.623+0000 7f18b0f29640 1 --2- 172.21.15.29:0/310872353 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f188c067bc0 0x7f188c06a080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f18940ad000 tx=0x7f1894008040 comp rx=0 tx=0).stop 2024-04-25T00:37:59.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:59.623+0000 7f18b0f29640 1 -- 172.21.15.29:0/310872353 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac0af040 msgr2=0x7f18ac14f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:59.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:59.623+0000 7f18b0f29640 1 --2- 172.21.15.29:0/310872353 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac0af040 0x7f18ac14f710 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f189c007e70 tx=0x7f189c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:37:59.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:59.623+0000 7f18b0f29640 1 -- 172.21.15.29:0/310872353 shutdown_connections 2024-04-25T00:37:59.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:59.623+0000 7f18b0f29640 1 --2- 172.21.15.29:0/310872353 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f188c067bc0 0x7f188c06a080 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:59.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:59.623+0000 7f18b0f29640 1 --2- 172.21.15.29:0/310872353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18ac15d720 0x7f18ac1556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:59.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:59.623+0000 7f18b0f29640 1 --2- 172.21.15.29:0/310872353 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac0af040 0x7f18ac14f710 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:59.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:59.623+0000 7f18b0f29640 1 --2- 172.21.15.29:0/310872353 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ac0aec60 0x7f18ac0ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:59.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:59.623+0000 7f18b0f29640 1 -- 172.21.15.29:0/310872353 >> 172.21.15.29:0/310872353 conn(0x7f18ac075030 msgr2=0x7f18ac0b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:59.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:59.623+0000 7f18b0f29640 1 -- 172.21.15.29:0/310872353 shutdown_connections 2024-04-25T00:37:59.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:37:59.623+0000 7f18b0f29640 1 -- 172.21.15.29:0/310872353 wait complete. 2024-04-25T00:38:00.654 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone3071 2024-04-25T00:38:00.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f9311ac2640 1 Processor -- start 2024-04-25T00:38:00.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f9311ac2640 1 -- start start 2024-04-25T00:38:00.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f9311ac2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f930c0b3140 0x7f930c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:00.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f9311ac2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f930c0b4b10 0x7f930c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:00.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f9311ac2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f930c0b41e0 con 0x7f930c0b3a60 2024-04-25T00:38:00.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f9311ac2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f930c06d960 con 0x7f930c0b4b10 2024-04-25T00:38:00.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f9311ac2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f930c06dc40 con 0x7f930c0b3140 2024-04-25T00:38:00.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f930b7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f930c0b3140 0x7f930c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:00.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f930affd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f930c0b4b10 0x7f930c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:00.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f930b7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f930c0b3140 0x7f930c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:53588/0 (socket says 172.21.15.29:53588) 2024-04-25T00:38:00.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f930b7fe640 1 -- 172.21.15.29:0/2386858890 learned_addr learned my addr 172.21.15.29:0/2386858890 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:00.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f930b7fe640 1 -- 172.21.15.29:0/2386858890 >> v2:172.21.15.157:3300/0 conn(0x7f930c0b4b10 msgr2=0x7f930c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:00.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f930b7fe640 1 --2- 172.21.15.29:0/2386858890 >> v2:172.21.15.157:3300/0 conn(0x7f930c0b4b10 0x7f930c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:00.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f930b7fe640 1 -- 172.21.15.29:0/2386858890 >> v1:172.21.15.157:6789/0 conn(0x7f930c0b3a60 legacy=0x7f930c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:00.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.795+0000 7f930b7fe640 1 -- 172.21.15.29:0/2386858890 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c06df20 con 0x7f930c0b3140 2024-04-25T00:38:00.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f930b7fe640 1 --2- 172.21.15.29:0/2386858890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f930c0b3140 0x7f930c06e440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009870 tx=0x7f92fc01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48676b3759c373c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:00.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f930a7fc640 1 -- 172.21.15.29:0/2386858890 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc0b0070 con 0x7f930c0b3140 2024-04-25T00:38:00.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f930a7fc640 1 -- 172.21.15.29:0/2386858890 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92fc0ab600 con 0x7f930c0b3140 2024-04-25T00:38:00.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f930a7fc640 1 -- 172.21.15.29:0/2386858890 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc0ab9d0 con 0x7f930c0b3140 2024-04-25T00:38:00.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f9311ac2640 1 -- 172.21.15.29:0/2386858890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f930c0b3140 msgr2=0x7f930c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:00.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f9311ac2640 1 --2- 172.21.15.29:0/2386858890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f930c0b3140 0x7f930c06e440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009870 tx=0x7f92fc01c560 comp rx=0 tx=0).stop 2024-04-25T00:38:00.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f9311ac2640 1 -- 172.21.15.29:0/2386858890 shutdown_connections 2024-04-25T00:38:00.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f9311ac2640 1 --2- 172.21.15.29:0/2386858890 >> v2:172.21.15.157:3300/0 conn(0x7f930c0b4b10 0x7f930c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:00.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f9311ac2640 1 --2- 172.21.15.29:0/2386858890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f930c0b3140 0x7f930c06e440 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:00.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f9311ac2640 1 -- 172.21.15.29:0/2386858890 >> 172.21.15.29:0/2386858890 conn(0x7f930c0b1870 msgr2=0x7f930c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:00.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f9311ac2640 1 -- 172.21.15.29:0/2386858890 shutdown_connections 2024-04-25T00:38:00.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f9311ac2640 1 -- 172.21.15.29:0/2386858890 wait complete. 2024-04-25T00:38:00.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f9311ac2640 1 Processor -- start 2024-04-25T00:38:00.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.799+0000 7f9311ac2640 1 -- start start 2024-04-25T00:38:00.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f9311ac2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f930c0b3140 0x7f930c156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:00.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f9311ac2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f930c0b3a60 0x7f930c0c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:00.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f9311ac2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f930c0b4b10 0x7f930c0c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f9311ac2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f930c168b70 con 0x7f930c0b3140 2024-04-25T00:38:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f9311ac2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1689f0 con 0x7f930c0b4b10 2024-04-25T00:38:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f9311ac2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f930c168cf0 con 0x7f930c0b3a60 2024-04-25T00:38:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f930b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f930c0b3140 0x7f930c156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f930b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f930c0b3140 0x7f930c156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38804/0 (socket says 172.21.15.29:38804) 2024-04-25T00:38:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f930b7fe640 1 -- 172.21.15.29:0/694264535 learned_addr learned my addr 172.21.15.29:0/694264535 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f930affd640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f930c0b3a60 0x7f930c0c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f930bfff640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f930c0b4b10 0x7f930c0c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f930b7fe640 1 -- 172.21.15.29:0/694264535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f930c0b3a60 msgr2=0x7f930c0c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f930b7fe640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f930c0b3a60 0x7f930c0c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f930b7fe640 1 -- 172.21.15.29:0/694264535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f930c0b4b10 msgr2=0x7f930c0c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f930b7fe640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f930c0b4b10 0x7f930c0c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:00.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f930b7fe640 1 -- 172.21.15.29:0/694264535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c0c59e0 con 0x7f930c0b3140 2024-04-25T00:38:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f930bfff640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f930c0b4b10 0x7f930c0c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f930b7fe640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f930c0b3140 0x7f930c156820 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0ae040 tx=0x7f92fc01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f92f3fff640 1 -- 172.21.15.29:0/694264535 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc0b0040 con 0x7f930c0b3140 2024-04-25T00:38:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f92f3fff640 1 -- 172.21.15.29:0/694264535 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92fc0ab400 con 0x7f930c0b3140 2024-04-25T00:38:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f9311ac2640 1 -- 172.21.15.29:0/694264535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930c0c5cc0 con 0x7f930c0b3140 2024-04-25T00:38:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f92f3fff640 1 -- 172.21.15.29:0/694264535 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc0b6680 con 0x7f930c0b3140 2024-04-25T00:38:00.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f9311ac2640 1 -- 172.21.15.29:0/694264535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f930c0c6280 con 0x7f930c0b3140 2024-04-25T00:38:00.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f9311ac2640 1 -- 172.21.15.29:0/694264535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f930c069270 con 0x7f930c0b3140 2024-04-25T00:38:00.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f92f3fff640 1 -- 172.21.15.29:0/694264535 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f92fc0be030 con 0x7f930c0b3140 2024-04-25T00:38:00.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f92f3fff640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f92f8067b90 0x7f92f806a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:00.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f930affd640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f92f8067b90 0x7f92f806a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:00.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.803+0000 7f92f3fff640 1 -- 172.21.15.29:0/694264535 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f92fc11fa60 con 0x7f930c0b3140 2024-04-25T00:38:00.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.807+0000 7f930affd640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f92f8067b90 0x7f92f806a050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f930c0cec80 tx=0x7f9300008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:00.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:00.811+0000 7f92f3fff640 1 -- 172.21.15.29:0/694264535 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f92fc0c5060 con 0x7f930c0b3140 2024-04-25T00:38:01.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:01.007+0000 7f9311ac2640 1 -- 172.21.15.29:0/694264535 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone3071", "target": ["mon-mgr", ""]}) -- 0x7f930c06d020 con 0x7f92f8067b90 2024-04-25T00:38:04.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.599+0000 7f92f3fff640 1 -- 172.21.15.29:0/694264535 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f930c06d020 con 0x7f92f8067b90 2024-04-25T00:38:04.603 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:38:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: "status": { 2024-04-25T00:38:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "complete" 2024-04-25T00:38:04.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:04.604 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:38:04.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.603+0000 7f9311ac2640 1 -- 172.21.15.29:0/694264535 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f92f8067b90 msgr2=0x7f92f806a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:04.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.603+0000 7f9311ac2640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f92f8067b90 0x7f92f806a050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f930c0cec80 tx=0x7f9300008040 comp rx=0 tx=0).stop 2024-04-25T00:38:04.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.603+0000 7f9311ac2640 1 -- 172.21.15.29:0/694264535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f930c0b3140 msgr2=0x7f930c156820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:04.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.603+0000 7f9311ac2640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f930c0b3140 0x7f930c156820 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0ae040 tx=0x7f92fc01ada0 comp rx=0 tx=0).stop 2024-04-25T00:38:04.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.603+0000 7f9311ac2640 1 -- 172.21.15.29:0/694264535 shutdown_connections 2024-04-25T00:38:04.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.603+0000 7f9311ac2640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f92f8067b90 0x7f92f806a050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:04.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.603+0000 7f9311ac2640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f930c0b4b10 0x7f930c0c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:04.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.603+0000 7f9311ac2640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f930c0b3a60 0x7f930c0c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:04.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.603+0000 7f9311ac2640 1 --2- 172.21.15.29:0/694264535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f930c0b3140 0x7f930c156820 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:04.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.603+0000 7f9311ac2640 1 -- 172.21.15.29:0/694264535 >> 172.21.15.29:0/694264535 conn(0x7f930c0b1870 msgr2=0x7f930c070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:04.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.603+0000 7f9311ac2640 1 -- 172.21.15.29:0/694264535 shutdown_connections 2024-04-25T00:38:04.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.603+0000 7f9311ac2640 1 -- 172.21.15.29:0/694264535 wait complete. 2024-04-25T00:38:04.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:04.626 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone3071/.meta) 2024-04-25T00:38:04.643 INFO:teuthology.orchestra.run.smithi029.stdout:[GLOBAL] 2024-04-25T00:38:04.643 INFO:teuthology.orchestra.run.smithi029.stdout:version = 2 2024-04-25T00:38:04.643 INFO:teuthology.orchestra.run.smithi029.stdout:type = clone 2024-04-25T00:38:04.643 INFO:teuthology.orchestra.run.smithi029.stdout:path = /volumes/_nogroup/subvol_clone3071/f7333468-98d3-462a-935e-b2784fdfadd0 2024-04-25T00:38:04.643 INFO:teuthology.orchestra.run.smithi029.stdout:state = complete 2024-04-25T00:38:04.643 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:04.644 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol6790 subvol_snap26070 2024-04-25T00:38:04.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.823+0000 7fbb7e9a2640 1 Processor -- start 2024-04-25T00:38:04.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.823+0000 7fbb7e9a2640 1 -- start start 2024-04-25T00:38:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb7e9a2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbb78151640 0x7fbb7806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb7e9a2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbb780b2b70 0x7fbb7806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb7e9a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7806cd40 0x7fbb7815c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb7e9a2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbb780b40a0 con 0x7fbb78151640 2024-04-25T00:38:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb7e9a2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbb780b3000 con 0x7fbb780b2b70 2024-04-25T00:38:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb7e9a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbb780b32e0 con 0x7fbb7806cd40 2024-04-25T00:38:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb77fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbb78151640 0x7fbb7806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb77fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbb78151640 0x7fbb7806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38814/0 (socket says 172.21.15.29:38814) 2024-04-25T00:38:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb77fff640 1 -- 172.21.15.29:0/1862406382 learned_addr learned my addr 172.21.15.29:0/1862406382 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb7cf3f640 1 --2- 172.21.15.29:0/1862406382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7806cd40 0x7fbb7815c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:04.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb777fe640 1 --2- 172.21.15.29:0/1862406382 >> v2:172.21.15.157:3300/0 conn(0x7fbb780b2b70 0x7fbb7806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:04.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb77fff640 1 -- 172.21.15.29:0/1862406382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7806cd40 msgr2=0x7fbb7815c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:04.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb77fff640 1 --2- 172.21.15.29:0/1862406382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7806cd40 0x7fbb7815c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:04.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb77fff640 1 -- 172.21.15.29:0/1862406382 >> v2:172.21.15.157:3300/0 conn(0x7fbb780b2b70 msgr2=0x7fbb7806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:04.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb77fff640 1 --2- 172.21.15.29:0/1862406382 >> v2:172.21.15.157:3300/0 conn(0x7fbb780b2b70 0x7fbb7806db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:04.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb77fff640 1 -- 172.21.15.29:0/1862406382 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb780b35c0 con 0x7fbb78151640 2024-04-25T00:38:04.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.827+0000 7fbb7cf3f640 1 --2- 172.21.15.29:0/1862406382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7806cd40 0x7fbb7815c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:04.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.831+0000 7fbb77fff640 1 --2- 172.21.15.29:0/1862406382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb78151640 0x7fbb7806d5d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fbb78151a20 tx=0x7fbb7001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d29f788e7e6eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:04.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.831+0000 7fbb76ffd640 1 -- 172.21.15.29:0/1862406382 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb700b0070 con 0x7fbb78151640 2024-04-25T00:38:04.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.831+0000 7fbb76ffd640 1 -- 172.21.15.29:0/1862406382 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb700ab600 con 0x7fbb78151640 2024-04-25T00:38:04.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.831+0000 7fbb76ffd640 1 -- 172.21.15.29:0/1862406382 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb700ab9d0 con 0x7fbb78151640 2024-04-25T00:38:04.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.831+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/1862406382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb78151640 msgr2=0x7fbb7806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:04.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.831+0000 7fbb7e9a2640 1 --2- 172.21.15.29:0/1862406382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb78151640 0x7fbb7806d5d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fbb78151a20 tx=0x7fbb7001c490 comp rx=0 tx=0).stop 2024-04-25T00:38:04.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.831+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/1862406382 shutdown_connections 2024-04-25T00:38:04.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.831+0000 7fbb7e9a2640 1 --2- 172.21.15.29:0/1862406382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7806cd40 0x7fbb7815c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:04.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.831+0000 7fbb7e9a2640 1 --2- 172.21.15.29:0/1862406382 >> v2:172.21.15.157:3300/0 conn(0x7fbb780b2b70 0x7fbb7806db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:04.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.831+0000 7fbb7e9a2640 1 --2- 172.21.15.29:0/1862406382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb78151640 0x7fbb7806d5d0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:04.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.831+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/1862406382 >> 172.21.15.29:0/1862406382 conn(0x7fbb780b1870 msgr2=0x7fbb780b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:04.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.831+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/1862406382 shutdown_connections 2024-04-25T00:38:04.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.831+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/1862406382 wait complete. 2024-04-25T00:38:04.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7e9a2640 1 Processor -- start 2024-04-25T00:38:04.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7e9a2640 1 -- start start 2024-04-25T00:38:04.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7e9a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7806cd40 0x7fbb7814fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:04.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7e9a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb78151640 0x7fbb78150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:04.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7e9a2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb78162780 0x7fbb781507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:04.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7e9a2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7815eb70 con 0x7fbb78162780 2024-04-25T00:38:04.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7e9a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7815e9f0 con 0x7fbb78151640 2024-04-25T00:38:04.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7e9a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbb7815ecf0 con 0x7fbb7806cd40 2024-04-25T00:38:04.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7cf3f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb78162780 0x7fbb781507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:04.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7cf3f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb78162780 0x7fbb781507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38818/0 (socket says 172.21.15.29:38818) 2024-04-25T00:38:04.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7cf3f640 1 -- 172.21.15.29:0/2281444044 learned_addr learned my addr 172.21.15.29:0/2281444044 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:04.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb77fff640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7806cd40 0x7fbb7814fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:04.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb777fe640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb78151640 0x7fbb78150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:04.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7cf3f640 1 -- 172.21.15.29:0/2281444044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7806cd40 msgr2=0x7fbb7814fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:04.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7cf3f640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7806cd40 0x7fbb7814fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:04.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7cf3f640 1 -- 172.21.15.29:0/2281444044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb78151640 msgr2=0x7fbb78150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:04.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7cf3f640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb78151640 0x7fbb78150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:04.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7cf3f640 1 -- 172.21.15.29:0/2281444044 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb78164370 con 0x7fbb78162780 2024-04-25T00:38:04.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7cf3f640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb78162780 0x7fbb781507c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800cd40 tx=0x7fbb6800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:04.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb74ff9640 1 -- 172.21.15.29:0/2281444044 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6800f680 con 0x7fbb78162780 2024-04-25T00:38:04.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb74ff9640 1 -- 172.21.15.29:0/2281444044 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb6800f840 con 0x7fbb78162780 2024-04-25T00:38:04.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/2281444044 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb781616e0 con 0x7fbb78162780 2024-04-25T00:38:04.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb74ff9640 1 -- 172.21.15.29:0/2281444044 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6800fc30 con 0x7fbb78162780 2024-04-25T00:38:04.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/2281444044 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb78161ca0 con 0x7fbb78162780 2024-04-25T00:38:04.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.835+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/2281444044 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb780692e0 con 0x7fbb78162780 2024-04-25T00:38:04.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.839+0000 7fbb74ff9640 1 -- 172.21.15.29:0/2281444044 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7fbb680213f0 con 0x7fbb78162780 2024-04-25T00:38:04.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.839+0000 7fbb74ff9640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbb58067b90 0x7fbb5806a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:04.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.839+0000 7fbb74ff9640 1 -- 172.21.15.29:0/2281444044 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fbb68086e90 con 0x7fbb78162780 2024-04-25T00:38:04.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.839+0000 7fbb77fff640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbb58067b90 0x7fbb5806a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:04.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.839+0000 7fbb77fff640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbb58067b90 0x7fbb5806a050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbb700ad000 tx=0x7fbb70008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:04.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:04.843+0000 7fbb74ff9640 1 -- 172.21.15.29:0/2281444044 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbb68014030 con 0x7fbb78162780 2024-04-25T00:38:05.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.035+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/2281444044 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol6790", "snap_name": "subvol_snap26070", "target": ["mon-mgr", ""]}) -- 0x7fbb7806e1d0 con 0x7fbb58067b90 2024-04-25T00:38:05.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.051+0000 7fbb74ff9640 1 -- 172.21.15.29:0/2281444044 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+115 (secure 0 0 0) 0x7fbb7806e1d0 con 0x7fbb58067b90 2024-04-25T00:38:05.055 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:38:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: "created_at": "2024-04-25 00:37:46.447186", 2024-04-25T00:38:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: "data_pool": "cephfs_data", 2024-04-25T00:38:05.056 INFO:teuthology.orchestra.run.smithi029.stdout: "has_pending_clones": "no" 2024-04-25T00:38:05.056 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:38:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.055+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/2281444044 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbb58067b90 msgr2=0x7fbb5806a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.055+0000 7fbb7e9a2640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbb58067b90 0x7fbb5806a050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbb700ad000 tx=0x7fbb70008040 comp rx=0 tx=0).stop 2024-04-25T00:38:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.055+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/2281444044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb78162780 msgr2=0x7fbb781507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.055+0000 7fbb7e9a2640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb78162780 0x7fbb781507c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800cd40 tx=0x7fbb6800a650 comp rx=0 tx=0).stop 2024-04-25T00:38:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.055+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/2281444044 shutdown_connections 2024-04-25T00:38:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.055+0000 7fbb7e9a2640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbb58067b90 0x7fbb5806a050 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.055+0000 7fbb7e9a2640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb78162780 0x7fbb781507c0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.055+0000 7fbb7e9a2640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb78151640 0x7fbb78150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.055+0000 7fbb7e9a2640 1 --2- 172.21.15.29:0/2281444044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7806cd40 0x7fbb7814fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.055+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/2281444044 >> 172.21.15.29:0/2281444044 conn(0x7fbb780b1870 msgr2=0x7fbb780b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:05.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.055+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/2281444044 shutdown_connections 2024-04-25T00:38:05.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.055+0000 7fbb7e9a2640 1 -- 172.21.15.29:0/2281444044 wait complete. 2024-04-25T00:38:05.077 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvol6790 2024-04-25T00:38:05.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.215+0000 7fb2d65e3640 1 Processor -- start 2024-04-25T00:38:05.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.215+0000 7fb2d65e3640 1 -- start start 2024-04-25T00:38:05.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2d65e3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2d00ce1a0 0x7fb2d0166d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2d65e3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2d00cc9d0 con 0x7fb2d015c100 2024-04-25T00:38:05.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2d65e3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2d00b3050 con 0x7fb2d00b29c0 2024-04-25T00:38:05.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2d65e3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb2d00b2e20 con 0x7fb2d00ce1a0 2024-04-25T00:38:05.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2cffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb2d015c100 0x7fb2d00d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59118/0 (socket says 172.21.15.29:59118) 2024-04-25T00:38:05.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2cffff640 1 -- 172.21.15.29:0/4264099850 learned_addr learned my addr 172.21.15.29:0/4264099850 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:05.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2467489082 0 0) 0x7fb2d00cc9d0 con 0x7fb2d015c100 2024-04-25T00:38:05.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2b8003530 con 0x7fb2d015c100 2024-04-25T00:38:05.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2d4b80640 1 --2- 172.21.15.29:0/4264099850 >> v2:172.21.15.157:3300/0 conn(0x7fb2d00ce1a0 0x7fb2d0166d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3908496749 0 0) 0x7fb2b8003530 con 0x7fb2d015c100 2024-04-25T00:38:05.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 >> v2:172.21.15.157:3300/0 conn(0x7fb2d00ce1a0 msgr2=0x7fb2d0166d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:05.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 --2- 172.21.15.29:0/4264099850 >> v2:172.21.15.157:3300/0 conn(0x7fb2d00ce1a0 0x7fb2d0166d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 >> v1:172.21.15.157:6789/0 conn(0x7fb2d00b29c0 legacy=0x7fb2d00d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:05.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d00b3330 con 0x7fb2d015c100 2024-04-25T00:38:05.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fb2c00016a0 con 0x7fb2d015c100 2024-04-25T00:38:05.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 >> v1:172.21.15.29:6789/0 conn(0x7fb2d015c100 legacy=0x7fb2d00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 --2- 172.21.15.29:0/4264099850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2b80039c0 0x7fb2b8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 --2- 172.21.15.29:0/4264099850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b8004810 0x7fb2b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 --2- 172.21.15.29:0/4264099850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b80053d0 0x7fb2b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b800c240 con 0x7fb2b8004810 2024-04-25T00:38:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b800c3f0 con 0x7fb2b80053d0 2024-04-25T00:38:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b800c5f0 con 0x7fb2b80039c0 2024-04-25T00:38:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2d4b80640 1 --2- 172.21.15.29:0/4264099850 >> v2:172.21.15.157:3300/0 conn(0x7fb2d00ce1a0 0x7fb2d0166d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2cf7fe640 1 --2- 172.21.15.29:0/4264099850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b8004810 0x7fb2b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2d4b80640 1 --2- 172.21.15.29:0/4264099850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b80053d0 0x7fb2b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2cffff640 1 --2- 172.21.15.29:0/4264099850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2b80039c0 0x7fb2b8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2cf7fe640 1 -- 172.21.15.29:0/4264099850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2b80039c0 msgr2=0x7fb2b8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2cf7fe640 1 --2- 172.21.15.29:0/4264099850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2b80039c0 0x7fb2b8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2cf7fe640 1 -- 172.21.15.29:0/4264099850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b80053d0 msgr2=0x7fb2b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2cf7fe640 1 --2- 172.21.15.29:0/4264099850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b80053d0 0x7fb2b800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2cf7fe640 1 -- 172.21.15.29:0/4264099850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b8003530 con 0x7fb2b8004810 2024-04-25T00:38:05.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2cffff640 1 --2- 172.21.15.29:0/4264099850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2b80039c0 0x7fb2b8003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:05.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2d4b80640 1 --2- 172.21.15.29:0/4264099850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b80053d0 0x7fb2b800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:05.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.219+0000 7fb2d4b80640 1 -- 172.21.15.29:0/4264099850 reap_dead start 2024-04-25T00:38:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2cf7fe640 1 --2- 172.21.15.29:0/4264099850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b8004810 0x7fb2b8004c90 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4003db0 tx=0x7fb2c401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db10a41456c95c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40b0070 con 0x7fb2b8004810 2024-04-25T00:38:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c40ab620 con 0x7fb2b8004810 2024-04-25T00:38:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2ceffd640 1 -- 172.21.15.29:0/4264099850 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40039b0 con 0x7fb2b8004810 2024-04-25T00:38:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 -- 172.21.15.29:0/4264099850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b8004810 msgr2=0x7fb2b8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 --2- 172.21.15.29:0/4264099850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b8004810 0x7fb2b8004c90 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4003db0 tx=0x7fb2c401c300 comp rx=0 tx=0).stop 2024-04-25T00:38:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 -- 172.21.15.29:0/4264099850 shutdown_connections 2024-04-25T00:38:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 --2- 172.21.15.29:0/4264099850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b8004810 0x7fb2b8004c90 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 -- 172.21.15.29:0/4264099850 >> 172.21.15.29:0/4264099850 conn(0x7fb2d009aa30 msgr2=0x7fb2d009ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 -- 172.21.15.29:0/4264099850 shutdown_connections 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 -- 172.21.15.29:0/4264099850 wait complete. 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 Processor -- start 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 -- start start 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b8004810 0x7fb2d0154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d00c59c0 0x7fb2d00c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d00c6380 0x7fb2d00c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d00d2020 con 0x7fb2d00c59c0 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d00d1ea0 con 0x7fb2b8004810 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d65e3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d00d21a0 con 0x7fb2d00c6380 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2cf7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d00c59c0 0x7fb2d00c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2cf7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d00c59c0 0x7fb2d00c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38840/0 (socket says 172.21.15.29:38840) 2024-04-25T00:38:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2cf7fe640 1 -- 172.21.15.29:0/3788648398 learned_addr learned my addr 172.21.15.29:0/3788648398 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:05.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2d4b80640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d00c6380 0x7fb2d00c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2cf7fe640 1 -- 172.21.15.29:0/3788648398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d00c6380 msgr2=0x7fb2d00c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2cffff640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b8004810 0x7fb2d0154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2cf7fe640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d00c6380 0x7fb2d00c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2cf7fe640 1 -- 172.21.15.29:0/3788648398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b8004810 msgr2=0x7fb2d0154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2cf7fe640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b8004810 0x7fb2d0154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2cf7fe640 1 -- 172.21.15.29:0/3788648398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d00c8960 con 0x7fb2d00c59c0 2024-04-25T00:38:05.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2cffff640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b8004810 0x7fb2d0154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:05.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.223+0000 7fb2cf7fe640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d00c59c0 0x7fb2d00c5e40 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb2c401c7e0 tx=0x7fb2c40190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:05.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.227+0000 7fb2ccff9640 1 -- 172.21.15.29:0/3788648398 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40b0050 con 0x7fb2d00c59c0 2024-04-25T00:38:05.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.227+0000 7fb2ccff9640 1 -- 172.21.15.29:0/3788648398 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c40bc070 con 0x7fb2d00c59c0 2024-04-25T00:38:05.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.227+0000 7fb2d65e3640 1 -- 172.21.15.29:0/3788648398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d00c6800 con 0x7fb2d00c59c0 2024-04-25T00:38:05.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.227+0000 7fb2ccff9640 1 -- 172.21.15.29:0/3788648398 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40b7670 con 0x7fb2d00c59c0 2024-04-25T00:38:05.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.227+0000 7fb2d65e3640 1 -- 172.21.15.29:0/3788648398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d00c7de0 con 0x7fb2d00c59c0 2024-04-25T00:38:05.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.227+0000 7fb2d65e3640 1 -- 172.21.15.29:0/3788648398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d00692e0 con 0x7fb2d00c59c0 2024-04-25T00:38:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.227+0000 7fb2ccff9640 1 -- 172.21.15.29:0/3788648398 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7fb2c40b7810 con 0x7fb2d00c59c0 2024-04-25T00:38:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.227+0000 7fb2ccff9640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb2bc067bc0 0x7fb2bc06a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.227+0000 7fb2ccff9640 1 -- 172.21.15.29:0/3788648398 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fb2c4120fc0 con 0x7fb2d00c59c0 2024-04-25T00:38:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.227+0000 7fb2cffff640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb2bc067bc0 0x7fb2bc06a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.227+0000 7fb2cffff640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb2bc067bc0 0x7fb2bc06a080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0001810 tx=0x7fb2c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:05.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.231+0000 7fb2ccff9640 1 -- 172.21.15.29:0/3788648398 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb2c4016010 con 0x7fb2d00c59c0 2024-04-25T00:38:05.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.423+0000 7fb2d65e3640 1 -- 172.21.15.29:0/3788648398 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvol6790", "target": ["mon-mgr", ""]}) -- 0x7fb2d00afbf0 con 0x7fb2bc067bc0 2024-04-25T00:38:05.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.455+0000 7fb2ccff9640 1 -- 172.21.15.29:0/3788648398 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+87 (secure 0 0 0) 0x7fb2d00afbf0 con 0x7fb2bc067bc0 2024-04-25T00:38:05.459 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:38:05.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:05.459 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "fake" 2024-04-25T00:38:05.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:05.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:05.459 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "subvol_snap26070" 2024-04-25T00:38:05.459 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:05.459 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:38:05.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.459+0000 7fb2d65e3640 1 -- 172.21.15.29:0/3788648398 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb2bc067bc0 msgr2=0x7fb2bc06a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.459+0000 7fb2d65e3640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb2bc067bc0 0x7fb2bc06a080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0001810 tx=0x7fb2c0009040 comp rx=0 tx=0).stop 2024-04-25T00:38:05.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.459+0000 7fb2d65e3640 1 -- 172.21.15.29:0/3788648398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d00c59c0 msgr2=0x7fb2d00c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.459+0000 7fb2d65e3640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d00c59c0 0x7fb2d00c5e40 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb2c401c7e0 tx=0x7fb2c40190a0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.459+0000 7fb2d65e3640 1 -- 172.21.15.29:0/3788648398 shutdown_connections 2024-04-25T00:38:05.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.459+0000 7fb2d65e3640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb2bc067bc0 0x7fb2bc06a080 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.459+0000 7fb2d65e3640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d00c6380 0x7fb2d00c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.459+0000 7fb2d65e3640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2d00c59c0 0x7fb2d00c5e40 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.459+0000 7fb2d65e3640 1 --2- 172.21.15.29:0/3788648398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b8004810 0x7fb2d0154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.459+0000 7fb2d65e3640 1 -- 172.21.15.29:0/3788648398 >> 172.21.15.29:0/3788648398 conn(0x7fb2d009aa30 msgr2=0x7fb2d00aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:05.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.459+0000 7fb2d65e3640 1 -- 172.21.15.29:0/3788648398 shutdown_connections 2024-04-25T00:38:05.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.459+0000 7fb2d65e3640 1 -- 172.21.15.29:0/3788648398 wait complete. 2024-04-25T00:38:05.479 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol6790 subvol_snap26070 2024-04-25T00:38:05.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.599+0000 7f8b3b117640 1 Processor -- start 2024-04-25T00:38:05.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b3b117640 1 -- start start 2024-04-25T00:38:05.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b3b117640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b340ce6c0 0x7f8b340ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b3b117640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b340d1000 0x7f8b340b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b3b117640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b340af470 con 0x7f8b340b0dd0 2024-04-25T00:38:05.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b3b117640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8b3415e750 con 0x7f8b340ce6c0 2024-04-25T00:38:05.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b3b117640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8b340d1430 con 0x7f8b340d1000 2024-04-25T00:38:05.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b38eb3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b340ce6c0 0x7f8b340ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b38eb3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b340ce6c0 0x7f8b340ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38856/0 (socket says 172.21.15.29:38856) 2024-04-25T00:38:05.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b38eb3640 1 -- 172.21.15.29:0/3737411488 learned_addr learned my addr 172.21.15.29:0/3737411488 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:05.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b396b4640 1 --2- 172.21.15.29:0/3737411488 >> v2:172.21.15.157:3300/0 conn(0x7f8b340d1000 0x7f8b340b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b38eb3640 1 -- 172.21.15.29:0/3737411488 >> v2:172.21.15.157:3300/0 conn(0x7f8b340d1000 msgr2=0x7f8b340b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:05.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b38eb3640 1 --2- 172.21.15.29:0/3737411488 >> v2:172.21.15.157:3300/0 conn(0x7f8b340d1000 0x7f8b340b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b38eb3640 1 -- 172.21.15.29:0/3737411488 >> v1:172.21.15.157:6789/0 conn(0x7f8b340b0dd0 legacy=0x7f8b340af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:05.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.603+0000 7f8b38eb3640 1 -- 172.21.15.29:0/3737411488 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3406c530 con 0x7f8b340ce6c0 2024-04-25T00:38:05.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b38eb3640 1 --2- 172.21.15.29:0/3737411488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b340ce6c0 0x7f8b340ceaa0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f8b34072be0 tx=0x7f8b2401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed63c242ab41f45c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:05.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b337fe640 1 -- 172.21.15.29:0/3737411488 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b240b0070 con 0x7f8b340ce6c0 2024-04-25T00:38:05.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b337fe640 1 -- 172.21.15.29:0/3737411488 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b240ab600 con 0x7f8b340ce6c0 2024-04-25T00:38:05.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b337fe640 1 -- 172.21.15.29:0/3737411488 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b240ab9d0 con 0x7f8b340ce6c0 2024-04-25T00:38:05.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 -- 172.21.15.29:0/3737411488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b340ce6c0 msgr2=0x7f8b340ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 --2- 172.21.15.29:0/3737411488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b340ce6c0 0x7f8b340ceaa0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f8b34072be0 tx=0x7f8b2401c300 comp rx=0 tx=0).stop 2024-04-25T00:38:05.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 -- 172.21.15.29:0/3737411488 shutdown_connections 2024-04-25T00:38:05.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 --2- 172.21.15.29:0/3737411488 >> v2:172.21.15.157:3300/0 conn(0x7f8b340d1000 0x7f8b340b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 --2- 172.21.15.29:0/3737411488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b340ce6c0 0x7f8b340ceaa0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 -- 172.21.15.29:0/3737411488 >> 172.21.15.29:0/3737411488 conn(0x7f8b340d1d20 msgr2=0x7f8b340d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:05.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 -- 172.21.15.29:0/3737411488 shutdown_connections 2024-04-25T00:38:05.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 -- 172.21.15.29:0/3737411488 wait complete. 2024-04-25T00:38:05.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 Processor -- start 2024-04-25T00:38:05.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 -- start start 2024-04-25T00:38:05.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b340b0dd0 0x7f8b34150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b340ce6c0 0x7f8b341513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b340d1000 0x7f8b34151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8b340cefe0 con 0x7f8b340d1000 2024-04-25T00:38:05.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3406d2f0 con 0x7f8b340b0dd0 2024-04-25T00:38:05.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b3b117640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3416ae30 con 0x7f8b340ce6c0 2024-04-25T00:38:05.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b396b4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b340d1000 0x7f8b34151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b396b4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b340d1000 0x7f8b34151930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38864/0 (socket says 172.21.15.29:38864) 2024-04-25T00:38:05.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b396b4640 1 -- 172.21.15.29:0/2741093882 learned_addr learned my addr 172.21.15.29:0/2741093882 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:05.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b38eb3640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b340b0dd0 0x7f8b34150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b33fff640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b340ce6c0 0x7f8b341513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b396b4640 1 -- 172.21.15.29:0/2741093882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b340ce6c0 msgr2=0x7f8b341513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b396b4640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b340ce6c0 0x7f8b341513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b396b4640 1 -- 172.21.15.29:0/2741093882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b340b0dd0 msgr2=0x7f8b34150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b396b4640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b340b0dd0 0x7f8b34150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b396b4640 1 -- 172.21.15.29:0/2741093882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b34151fd0 con 0x7f8b340d1000 2024-04-25T00:38:05.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b33fff640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b340ce6c0 0x7f8b341513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:05.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.607+0000 7f8b396b4640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b340d1000 0x7f8b34151930 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c007e90 tx=0x7f8b2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:05.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.611+0000 7f8b317fa640 1 -- 172.21.15.29:0/2741093882 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c014420 con 0x7f8b340d1000 2024-04-25T00:38:05.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.611+0000 7f8b317fa640 1 -- 172.21.15.29:0/2741093882 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b2c019070 con 0x7f8b340d1000 2024-04-25T00:38:05.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.611+0000 7f8b3b117640 1 -- 172.21.15.29:0/2741093882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b341522b0 con 0x7f8b340d1000 2024-04-25T00:38:05.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.611+0000 7f8b317fa640 1 -- 172.21.15.29:0/2741093882 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c014800 con 0x7f8b340d1000 2024-04-25T00:38:05.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.611+0000 7f8b3b117640 1 -- 172.21.15.29:0/2741093882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b340b4b90 con 0x7f8b340d1000 2024-04-25T00:38:05.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.611+0000 7f8b3b117640 1 -- 172.21.15.29:0/2741093882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b340692e0 con 0x7f8b340d1000 2024-04-25T00:38:05.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.611+0000 7f8b317fa640 1 -- 172.21.15.29:0/2741093882 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f8b2c0149a0 con 0x7f8b340d1000 2024-04-25T00:38:05.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.611+0000 7f8b317fa640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b20067b90 0x7f8b2006a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.611+0000 7f8b317fa640 1 -- 172.21.15.29:0/2741093882 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f8b2c0868e0 con 0x7f8b340d1000 2024-04-25T00:38:05.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.611+0000 7f8b38eb3640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b20067b90 0x7f8b2006a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.611+0000 7f8b38eb3640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b20067b90 0x7f8b2006a050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8b240ad000 tx=0x7f8b24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:05.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.615+0000 7f8b317fa640 1 -- 172.21.15.29:0/2741093882 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8b2c011260 con 0x7f8b340d1000 2024-04-25T00:38:05.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.807+0000 7f8b3b117640 1 -- 172.21.15.29:0/2741093882 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol6790", "snap_name": "subvol_snap26070", "target": ["mon-mgr", ""]}) -- 0x7f8b340b35f0 con 0x7f8b20067b90 2024-04-25T00:38:05.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.843+0000 7f8b317fa640 1 -- 172.21.15.29:0/2741093882 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8b340b35f0 con 0x7f8b20067b90 2024-04-25T00:38:05.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.847+0000 7f8b3b117640 1 -- 172.21.15.29:0/2741093882 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b20067b90 msgr2=0x7f8b2006a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.847+0000 7f8b3b117640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b20067b90 0x7f8b2006a050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8b240ad000 tx=0x7f8b24008040 comp rx=0 tx=0).stop 2024-04-25T00:38:05.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.847+0000 7f8b3b117640 1 -- 172.21.15.29:0/2741093882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b340d1000 msgr2=0x7f8b34151930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.847+0000 7f8b3b117640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b340d1000 0x7f8b34151930 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c007e90 tx=0x7f8b2c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.847+0000 7f8b3b117640 1 -- 172.21.15.29:0/2741093882 shutdown_connections 2024-04-25T00:38:05.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.847+0000 7f8b3b117640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b20067b90 0x7f8b2006a050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.847+0000 7f8b3b117640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b340d1000 0x7f8b34151930 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.847+0000 7f8b3b117640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b340ce6c0 0x7f8b341513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.847+0000 7f8b3b117640 1 --2- 172.21.15.29:0/2741093882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b340b0dd0 0x7f8b34150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.847+0000 7f8b3b117640 1 -- 172.21.15.29:0/2741093882 >> 172.21.15.29:0/2741093882 conn(0x7f8b340d1d20 msgr2=0x7f8b3406f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:05.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.847+0000 7f8b3b117640 1 -- 172.21.15.29:0/2741093882 shutdown_connections 2024-04-25T00:38:05.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.847+0000 7f8b3b117640 1 -- 172.21.15.29:0/2741093882 wait complete. 2024-04-25T00:38:05.865 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol6790 fake 2024-04-25T00:38:05.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.979+0000 7f305cbf6640 1 Processor -- start 2024-04-25T00:38:05.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.979+0000 7f305cbf6640 1 -- start start 2024-04-25T00:38:05.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305cbf6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f30581456d0 0x7f30580b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305cbf6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30580cf670 con 0x7f30580b3b50 2024-04-25T00:38:05.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305cbf6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30580b4380 con 0x7f305806cd40 2024-04-25T00:38:05.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305cbf6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f30580b40a0 con 0x7f30581456d0 2024-04-25T00:38:05.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f3055d9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f30580b3b50 0x7f30580cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59124/0 (socket says 172.21.15.29:59124) 2024-04-25T00:38:05.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f3055d9b640 1 -- 172.21.15.29:0/2326616371 learned_addr learned my addr 172.21.15.29:0/2326616371 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:05.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1008528420 0 0) 0x7f30580cf670 con 0x7f30580b3b50 2024-04-25T00:38:05.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3040003530 con 0x7f30580b3b50 2024-04-25T00:38:05.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305659c640 1 --2- 172.21.15.29:0/2326616371 >> v2:172.21.15.157:3300/0 conn(0x7f30581456d0 0x7f30580b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 107048838 0 0) 0x7f3040003530 con 0x7f30580b3b50 2024-04-25T00:38:05.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 >> v2:172.21.15.157:3300/0 conn(0x7f30581456d0 msgr2=0x7f30580b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:05.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 --2- 172.21.15.29:0/2326616371 >> v2:172.21.15.157:3300/0 conn(0x7f30581456d0 0x7f30580b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 >> v1:172.21.15.157:6789/0 conn(0x7f305806cd40 legacy=0x7f305806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:05.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30580ce480 con 0x7f30580b3b50 2024-04-25T00:38:05.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f304c001680 con 0x7f30580b3b50 2024-04-25T00:38:05.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305659c640 1 --2- 172.21.15.29:0/2326616371 >> v2:172.21.15.157:3300/0 conn(0x7f30581456d0 0x7f30580b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:05.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 >> v1:172.21.15.29:6789/0 conn(0x7f30580b3b50 legacy=0x7f30580cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 --2- 172.21.15.29:0/2326616371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30400039c0 0x7f3040003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 --2- 172.21.15.29:0/2326616371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3040004810 0x7f3040004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 --2- 172.21.15.29:0/2326616371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400051d0 0x7f304000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30580b40a0 con 0x7f30400051d0 2024-04-25T00:38:05.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f304000c100 con 0x7f30400039c0 2024-04-25T00:38:05.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f304000c300 con 0x7f3040004810 2024-04-25T00:38:05.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f3056d9d640 1 --2- 172.21.15.29:0/2326616371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400051d0 0x7f304000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305659c640 1 --2- 172.21.15.29:0/2326616371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30400039c0 0x7f3040003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f3055d9b640 1 --2- 172.21.15.29:0/2326616371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3040004810 0x7f3040004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f3056d9d640 1 -- 172.21.15.29:0/2326616371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3040004810 msgr2=0x7f3040004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:05.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f3056d9d640 1 --2- 172.21.15.29:0/2326616371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3040004810 0x7f3040004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f3056d9d640 1 -- 172.21.15.29:0/2326616371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30400039c0 msgr2=0x7f3040003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:05.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f3056d9d640 1 --2- 172.21.15.29:0/2326616371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30400039c0 0x7f3040003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f3056d9d640 1 -- 172.21.15.29:0/2326616371 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3040003530 con 0x7f30400051d0 2024-04-25T00:38:05.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f3055d9b640 1 --2- 172.21.15.29:0/2326616371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3040004810 0x7f3040004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:05.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f3056d9d640 1 -- 172.21.15.29:0/2326616371 reap_dead start 2024-04-25T00:38:05.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f3056d9d640 1 --2- 172.21.15.29:0/2326616371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400051d0 0x7f304000b9d0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f3048007fd0 tx=0x7f304801c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0bc50a213e438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:05.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30480039b0 con 0x7f30400051d0 2024-04-25T00:38:05.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30480ab660 con 0x7f30400051d0 2024-04-25T00:38:05.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.983+0000 7f305559a640 1 -- 172.21.15.29:0/2326616371 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30480aba50 con 0x7f30400051d0 2024-04-25T00:38:05.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 -- 172.21.15.29:0/2326616371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400051d0 msgr2=0x7f304000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 --2- 172.21.15.29:0/2326616371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400051d0 0x7f304000b9d0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f3048007fd0 tx=0x7f304801c300 comp rx=0 tx=0).stop 2024-04-25T00:38:05.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 -- 172.21.15.29:0/2326616371 shutdown_connections 2024-04-25T00:38:05.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 --2- 172.21.15.29:0/2326616371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400051d0 0x7f304000b9d0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 -- 172.21.15.29:0/2326616371 >> 172.21.15.29:0/2326616371 conn(0x7f30580b1870 msgr2=0x7f30580b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 -- 172.21.15.29:0/2326616371 shutdown_connections 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 -- 172.21.15.29:0/2326616371 wait complete. 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 Processor -- start 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 -- start start 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400051d0 0x7f30581544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f305813a090 0x7f305813adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f305813b330 0x7f30581f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3058145ab0 con 0x7f30400051d0 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f30580cea40 con 0x7f305813a090 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f30580b5000 con 0x7f305813b330 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400051d0 0x7f30581544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400051d0 0x7f30581544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38882/0 (socket says 172.21.15.29:38882) 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305659c640 1 -- 172.21.15.29:0/787864495 learned_addr learned my addr 172.21.15.29:0/787864495 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f3055d9b640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f305813a090 0x7f305813adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f3056d9d640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f305813b330 0x7f30581f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305659c640 1 -- 172.21.15.29:0/787864495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f305813b330 msgr2=0x7f30581f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305659c640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f305813b330 0x7f30581f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305659c640 1 -- 172.21.15.29:0/787864495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f305813a090 msgr2=0x7f305813adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:05.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305659c640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f305813a090 0x7f305813adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:05.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305659c640 1 -- 172.21.15.29:0/787864495 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3058072cc0 con 0x7f30400051d0 2024-04-25T00:38:05.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305659c640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400051d0 0x7f30581544c0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f30500046e0 tx=0x7f3050007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:05.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f3046ffd640 1 -- 172.21.15.29:0/787864495 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050014410 con 0x7f30400051d0 2024-04-25T00:38:05.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f3046ffd640 1 -- 172.21.15.29:0/787864495 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3050019070 con 0x7f30400051d0 2024-04-25T00:38:05.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f3046ffd640 1 -- 172.21.15.29:0/787864495 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500147b0 con 0x7f30400051d0 2024-04-25T00:38:05.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 -- 172.21.15.29:0/787864495 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30580ae3e0 con 0x7f30400051d0 2024-04-25T00:38:05.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.987+0000 7f305cbf6640 1 -- 172.21.15.29:0/787864495 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30580ae680 con 0x7f30400051d0 2024-04-25T00:38:05.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.991+0000 7f305cbf6640 1 -- 172.21.15.29:0/787864495 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30580692e0 con 0x7f30400051d0 2024-04-25T00:38:05.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.991+0000 7f3046ffd640 1 -- 172.21.15.29:0/787864495 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f3050014950 con 0x7f30400051d0 2024-04-25T00:38:05.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.991+0000 7f3046ffd640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3038067bc0 0x7f303806a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:05.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.991+0000 7f3055d9b640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3038067bc0 0x7f303806a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:05.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.991+0000 7f3046ffd640 1 -- 172.21.15.29:0/787864495 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f3050086a60 con 0x7f30400051d0 2024-04-25T00:38:05.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.991+0000 7f3055d9b640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3038067bc0 0x7f303806a080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f304c000fe0 tx=0x7f304c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:05.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:05.995+0000 7f3046ffd640 1 -- 172.21.15.29:0/787864495 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3050057190 con 0x7f30400051d0 2024-04-25T00:38:06.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.183+0000 7f305cbf6640 1 -- 172.21.15.29:0/787864495 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol6790", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f305813b710 con 0x7f3038067bc0 2024-04-25T00:38:06.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.227+0000 7f3046ffd640 1 -- 172.21.15.29:0/787864495 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f305813b710 con 0x7f3038067bc0 2024-04-25T00:38:06.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.227+0000 7f305cbf6640 1 -- 172.21.15.29:0/787864495 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3038067bc0 msgr2=0x7f303806a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:06.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.227+0000 7f305cbf6640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3038067bc0 0x7f303806a080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f304c000fe0 tx=0x7f304c009040 comp rx=0 tx=0).stop 2024-04-25T00:38:06.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.227+0000 7f305cbf6640 1 -- 172.21.15.29:0/787864495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400051d0 msgr2=0x7f30581544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:06.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.227+0000 7f305cbf6640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400051d0 0x7f30581544c0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f30500046e0 tx=0x7f3050007700 comp rx=0 tx=0).stop 2024-04-25T00:38:06.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.227+0000 7f305cbf6640 1 -- 172.21.15.29:0/787864495 shutdown_connections 2024-04-25T00:38:06.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.227+0000 7f305cbf6640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3038067bc0 0x7f303806a080 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:06.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.227+0000 7f305cbf6640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f305813b330 0x7f30581f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:06.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.227+0000 7f305cbf6640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f305813a090 0x7f305813adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:06.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.227+0000 7f305cbf6640 1 --2- 172.21.15.29:0/787864495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400051d0 0x7f30581544c0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:06.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.227+0000 7f305cbf6640 1 -- 172.21.15.29:0/787864495 >> 172.21.15.29:0/787864495 conn(0x7f30580b1870 msgr2=0x7f305806d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:06.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.227+0000 7f305cbf6640 1 -- 172.21.15.29:0/787864495 shutdown_connections 2024-04-25T00:38:06.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:06.227+0000 7f305cbf6640 1 -- 172.21.15.29:0/787864495 wait complete. 2024-04-25T00:38:06.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:06.248 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol6790/.meta) 2024-04-25T00:38:09.593 INFO:teuthology.orchestra.run.smithi029.stdout:[GLOBAL] 2024-04-25T00:38:09.593 INFO:teuthology.orchestra.run.smithi029.stdout:version = 1 2024-04-25T00:38:09.593 INFO:teuthology.orchestra.run.smithi029.stdout:type = subvolume 2024-04-25T00:38:09.593 INFO:teuthology.orchestra.run.smithi029.stdout:path = /volumes/_nogroup/subvol6790/de5fe82d-b246-466e-9d46-2d128beb9fd1 2024-04-25T00:38:09.593 INFO:teuthology.orchestra.run.smithi029.stdout:state = complete 2024-04-25T00:38:09.593 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:09.596 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol6790 2024-04-25T00:38:09.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f726aa6f640 1 Processor -- start 2024-04-25T00:38:09.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f726aa6f640 1 -- start start 2024-04-25T00:38:09.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f726aa6f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72640677e0 0x7f72640b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f726aa6f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72640b3fd0 con 0x7f72640cc3e0 2024-04-25T00:38:09.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f726aa6f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72640d0fb0 con 0x7f726415a880 2024-04-25T00:38:09.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f726aa6f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f726415ac60 con 0x7f72640677e0 2024-04-25T00:38:09.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f726900c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f72640cc3e0 0x7f72640acfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54340/0 (socket says 172.21.15.29:54340) 2024-04-25T00:38:09.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f726900c640 1 -- 172.21.15.29:0/838818325 learned_addr learned my addr 172.21.15.29:0/838818325 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:09.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f7263fff640 1 --2- 172.21.15.29:0/838818325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72640677e0 0x7f72640b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 17954811 0 0) 0x7f72640b3fd0 con 0x7f72640cc3e0 2024-04-25T00:38:09.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f724c003530 con 0x7f72640cc3e0 2024-04-25T00:38:09.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1160483313 0 0) 0x7f724c003530 con 0x7f72640cc3e0 2024-04-25T00:38:09.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72640677e0 msgr2=0x7f72640b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 --2- 172.21.15.29:0/838818325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72640677e0 0x7f72640b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 >> v1:172.21.15.157:6789/0 conn(0x7f726415a880 legacy=0x7f72640b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72640d0cd0 con 0x7f72640cc3e0 2024-04-25T00:38:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f725c001a00 con 0x7f72640cc3e0 2024-04-25T00:38:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 >> v1:172.21.15.29:6789/0 conn(0x7f72640cc3e0 legacy=0x7f72640acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 --2- 172.21.15.29:0/838818325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72640677e0 0x7f724c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 --2- 172.21.15.29:0/838818325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c003f70 0x7f724c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 --2- 172.21.15.29:0/838818325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f724c004d50 0x7f724c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f726415ac60 con 0x7f724c003f70 2024-04-25T00:38:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f724c00bb40 con 0x7f724c004d50 2024-04-25T00:38:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f724c00df50 con 0x7f72640677e0 2024-04-25T00:38:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f7263fff640 1 --2- 172.21.15.29:0/838818325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c003f70 0x7f724c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f726880b640 1 --2- 172.21.15.29:0/838818325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72640677e0 0x7f724c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f726900c640 1 --2- 172.21.15.29:0/838818325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f724c004d50 0x7f724c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f7263fff640 1 -- 172.21.15.29:0/838818325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72640677e0 msgr2=0x7f724c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:09.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f7263fff640 1 --2- 172.21.15.29:0/838818325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72640677e0 0x7f724c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f7263fff640 1 -- 172.21.15.29:0/838818325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f724c004d50 msgr2=0x7f724c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:09.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f7263fff640 1 --2- 172.21.15.29:0/838818325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f724c004d50 0x7f724c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.735+0000 7f7263fff640 1 -- 172.21.15.29:0/838818325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c003530 con 0x7f724c003f70 2024-04-25T00:38:09.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.739+0000 7f7263fff640 1 --2- 172.21.15.29:0/838818325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c003f70 0x7f724c004810 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f725800aa60 tx=0x7f725801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaf3b51cc059df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:09.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.739+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72580b2070 con 0x7f724c003f70 2024-04-25T00:38:09.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.739+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72580a5c70 con 0x7f724c003f70 2024-04-25T00:38:09.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.739+0000 7f72637fe640 1 -- 172.21.15.29:0/838818325 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725800b040 con 0x7f724c003f70 2024-04-25T00:38:09.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.739+0000 7f726aa6f640 1 -- 172.21.15.29:0/838818325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c003f70 msgr2=0x7f724c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:09.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.739+0000 7f726aa6f640 1 --2- 172.21.15.29:0/838818325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c003f70 0x7f724c004810 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f725800aa60 tx=0x7f725801cc30 comp rx=0 tx=0).stop 2024-04-25T00:38:09.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.739+0000 7f726900c640 1 -- 172.21.15.29:0/838818325 reap_dead start 2024-04-25T00:38:09.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.739+0000 7f726aa6f640 1 -- 172.21.15.29:0/838818325 shutdown_connections 2024-04-25T00:38:09.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.739+0000 7f726aa6f640 1 -- 172.21.15.29:0/838818325 >> 172.21.15.29:0/838818325 conn(0x7f7264072650 msgr2=0x7f7264072a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:09.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.739+0000 7f726aa6f640 1 -- 172.21.15.29:0/838818325 shutdown_connections 2024-04-25T00:38:09.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.739+0000 7f726aa6f640 1 -- 172.21.15.29:0/838818325 wait complete. 2024-04-25T00:38:09.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726aa6f640 1 Processor -- start 2024-04-25T00:38:09.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726aa6f640 1 -- start start 2024-04-25T00:38:09.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726aa6f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c003f70 0x7f72640cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726aa6f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72641f1380 0x7f72641f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726aa6f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72641f4690 0x7f72641f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726aa6f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f72640b08d0 con 0x7f724c003f70 2024-04-25T00:38:09.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726aa6f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f72640d0a80 con 0x7f72641f4690 2024-04-25T00:38:09.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726aa6f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f72640b0bb0 con 0x7f72641f1380 2024-04-25T00:38:09.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726880b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c003f70 0x7f72640cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726880b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c003f70 0x7f72640cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55174/0 (socket says 172.21.15.29:55174) 2024-04-25T00:38:09.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726880b640 1 -- 172.21.15.29:0/410365159 learned_addr learned my addr 172.21.15.29:0/410365159 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:09.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f7263fff640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72641f1380 0x7f72641f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726900c640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72641f4690 0x7f72641f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726880b640 1 -- 172.21.15.29:0/410365159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72641f1380 msgr2=0x7f72641f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:09.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726880b640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72641f1380 0x7f72641f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726880b640 1 -- 172.21.15.29:0/410365159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72641f4690 msgr2=0x7f72641f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:09.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726880b640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72641f4690 0x7f72641f4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726880b640 1 -- 172.21.15.29:0/410365159 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7264150f10 con 0x7f724c003f70 2024-04-25T00:38:09.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726880b640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c003f70 0x7f72640cb5e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f725400e550 tx=0x7f725400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:09.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f72617fa640 1 -- 172.21.15.29:0/410365159 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72540052e0 con 0x7f724c003f70 2024-04-25T00:38:09.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f72617fa640 1 -- 172.21.15.29:0/410365159 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7254018070 con 0x7f724c003f70 2024-04-25T00:38:09.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726aa6f640 1 -- 172.21.15.29:0/410365159 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72641f1d40 con 0x7f724c003f70 2024-04-25T00:38:09.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f72617fa640 1 -- 172.21.15.29:0/410365159 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254003820 con 0x7f724c003f70 2024-04-25T00:38:09.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726aa6f640 1 -- 172.21.15.29:0/410365159 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7264150430 con 0x7f724c003f70 2024-04-25T00:38:09.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.743+0000 7f726aa6f640 1 -- 172.21.15.29:0/410365159 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7264069270 con 0x7f724c003f70 2024-04-25T00:38:09.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.747+0000 7f72617fa640 1 -- 172.21.15.29:0/410365159 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f72540039c0 con 0x7f724c003f70 2024-04-25T00:38:09.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.747+0000 7f72617fa640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7250067b90 0x7f725006a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.747+0000 7f72617fa640 1 -- 172.21.15.29:0/410365159 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f72540866c0 con 0x7f724c003f70 2024-04-25T00:38:09.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.747+0000 7f7263fff640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7250067b90 0x7f725006a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.747+0000 7f7263fff640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7250067b90 0x7f725006a050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f72580af000 tx=0x7f72580046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:09.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.751+0000 7f72617fa640 1 -- 172.21.15.29:0/410365159 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7254005480 con 0x7f724c003f70 2024-04-25T00:38:09.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.939+0000 7f726aa6f640 1 -- 172.21.15.29:0/410365159 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol6790", "target": ["mon-mgr", ""]}) -- 0x7f726414ff00 con 0x7f7250067b90 2024-04-25T00:38:10.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:09.999+0000 7f72617fa640 1 -- 172.21.15.29:0/410365159 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f726414ff00 con 0x7f7250067b90 2024-04-25T00:38:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.003+0000 7f726aa6f640 1 -- 172.21.15.29:0/410365159 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7250067b90 msgr2=0x7f725006a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.003+0000 7f726aa6f640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7250067b90 0x7f725006a050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f72580af000 tx=0x7f72580046d0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.003+0000 7f726aa6f640 1 -- 172.21.15.29:0/410365159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c003f70 msgr2=0x7f72640cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.003+0000 7f726aa6f640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c003f70 0x7f72640cb5e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f725400e550 tx=0x7f725400ea10 comp rx=0 tx=0).stop 2024-04-25T00:38:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.003+0000 7f726aa6f640 1 -- 172.21.15.29:0/410365159 shutdown_connections 2024-04-25T00:38:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.003+0000 7f726aa6f640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7250067b90 0x7f725006a050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.003+0000 7f726aa6f640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72641f4690 0x7f72641f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.003+0000 7f726aa6f640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72641f1380 0x7f72641f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.003+0000 7f726aa6f640 1 --2- 172.21.15.29:0/410365159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c003f70 0x7f72640cb5e0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.003+0000 7f726aa6f640 1 -- 172.21.15.29:0/410365159 >> 172.21.15.29:0/410365159 conn(0x7f7264072650 msgr2=0x7f72641e1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.003+0000 7f726aa6f640 1 -- 172.21.15.29:0/410365159 shutdown_connections 2024-04-25T00:38:10.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.003+0000 7f726aa6f640 1 -- 172.21.15.29:0/410365159 wait complete. 2024-04-25T00:38:10.008 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:10.003+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:10.008 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:10.007+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:10.008 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:10.007+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:10.008 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:10.007+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:10.008 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:10.007+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:10.023 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:10.019+0000 7f9879581640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:10.023 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:10.019+0000 7f9879581640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:10.023 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:10.019+0000 7f9879581640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:10.023 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:10.019+0000 7f9879581640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:10.024 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:10.019+0000 7f9879581640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:10.024 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone39330 2024-04-25T00:38:10.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe459208640 1 Processor -- start 2024-04-25T00:38:10.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe459208640 1 -- start start 2024-04-25T00:38:10.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe459208640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe45415e8d0 0x7fe4540cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe459208640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe4540b4780 0x7fe45415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe459208640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe45415f1f0 0x7fe45415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe459208640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe4540b42f0 con 0x7fe45415e8d0 2024-04-25T00:38:10.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe459208640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe4540b3720 con 0x7fe4540b4780 2024-04-25T00:38:10.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe459208640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe4540b39c0 con 0x7fe45415f1f0 2024-04-25T00:38:10.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe452d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe45415e8d0 0x7fe4540cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe452d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe45415e8d0 0x7fe4540cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55228/0 (socket says 172.21.15.29:55228) 2024-04-25T00:38:10.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe452d9d640 1 -- 172.21.15.29:0/412917183 learned_addr learned my addr 172.21.15.29:0/412917183 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:10.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe45359e640 1 --2- 172.21.15.29:0/412917183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe45415f1f0 0x7fe45415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe45259c640 1 --2- 172.21.15.29:0/412917183 >> v2:172.21.15.157:3300/0 conn(0x7fe4540b4780 0x7fe45415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe452d9d640 1 -- 172.21.15.29:0/412917183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe45415f1f0 msgr2=0x7fe45415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe452d9d640 1 --2- 172.21.15.29:0/412917183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe45415f1f0 0x7fe45415b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe452d9d640 1 -- 172.21.15.29:0/412917183 >> v2:172.21.15.157:3300/0 conn(0x7fe4540b4780 msgr2=0x7fe45415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe452d9d640 1 --2- 172.21.15.29:0/412917183 >> v2:172.21.15.157:3300/0 conn(0x7fe4540b4780 0x7fe45415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.147+0000 7fe452d9d640 1 -- 172.21.15.29:0/412917183 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4540b3ca0 con 0x7fe45415e8d0 2024-04-25T00:38:10.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe452d9d640 1 --2- 172.21.15.29:0/412917183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe45415e8d0 0x7fe4540cc240 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fe454070910 tx=0x7fe44401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac1f1d75ad28213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe451d9b640 1 -- 172.21.15.29:0/412917183 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4440aa5f0 con 0x7fe45415e8d0 2024-04-25T00:38:10.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe451d9b640 1 -- 172.21.15.29:0/412917183 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4440af070 con 0x7fe45415e8d0 2024-04-25T00:38:10.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe451d9b640 1 -- 172.21.15.29:0/412917183 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4440aa9b0 con 0x7fe45415e8d0 2024-04-25T00:38:10.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe459208640 1 -- 172.21.15.29:0/412917183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe45415e8d0 msgr2=0x7fe4540cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe459208640 1 --2- 172.21.15.29:0/412917183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe45415e8d0 0x7fe4540cc240 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fe454070910 tx=0x7fe44401c4f0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe459208640 1 -- 172.21.15.29:0/412917183 shutdown_connections 2024-04-25T00:38:10.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe459208640 1 --2- 172.21.15.29:0/412917183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe45415f1f0 0x7fe45415b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe459208640 1 --2- 172.21.15.29:0/412917183 >> v2:172.21.15.157:3300/0 conn(0x7fe4540b4780 0x7fe45415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe459208640 1 --2- 172.21.15.29:0/412917183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe45415e8d0 0x7fe4540cc240 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe459208640 1 -- 172.21.15.29:0/412917183 >> 172.21.15.29:0/412917183 conn(0x7fe4540b1870 msgr2=0x7fe4540b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe459208640 1 -- 172.21.15.29:0/412917183 shutdown_connections 2024-04-25T00:38:10.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe459208640 1 -- 172.21.15.29:0/412917183 wait complete. 2024-04-25T00:38:10.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe459208640 1 Processor -- start 2024-04-25T00:38:10.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.151+0000 7fe459208640 1 -- start start 2024-04-25T00:38:10.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe459208640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4540b4780 0x7fe45415a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe459208640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe45415e8d0 0x7fe4541359a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe459208640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe45415f1f0 0x7fe4540cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe459208640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe45415f600 con 0x7fe4540b4780 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe459208640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe45415c7d0 con 0x7fe45415f1f0 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe459208640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe454132300 con 0x7fe45415e8d0 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe452d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4540b4780 0x7fe45415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe452d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4540b4780 0x7fe45415a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55238/0 (socket says 172.21.15.29:55238) 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe452d9d640 1 -- 172.21.15.29:0/3364996419 learned_addr learned my addr 172.21.15.29:0/3364996419 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe45259c640 1 --2- 172.21.15.29:0/3364996419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe45415e8d0 0x7fe4541359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe452d9d640 1 -- 172.21.15.29:0/3364996419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe45415e8d0 msgr2=0x7fe4541359a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe452d9d640 1 --2- 172.21.15.29:0/3364996419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe45415e8d0 0x7fe4541359a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe452d9d640 1 -- 172.21.15.29:0/3364996419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe45415f1f0 msgr2=0x7fe4540cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe452d9d640 1 --2- 172.21.15.29:0/3364996419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe45415f1f0 0x7fe4540cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe452d9d640 1 -- 172.21.15.29:0/3364996419 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45415a840 con 0x7fe4540b4780 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe452d9d640 1 --2- 172.21.15.29:0/3364996419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4540b4780 0x7fe45415a300 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fe44401aba0 tx=0x7fe44401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe43b7fe640 1 -- 172.21.15.29:0/3364996419 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4440bd5f0 con 0x7fe4540b4780 2024-04-25T00:38:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe43b7fe640 1 -- 172.21.15.29:0/3364996419 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4440bd790 con 0x7fe4540b4780 2024-04-25T00:38:10.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe43b7fe640 1 -- 172.21.15.29:0/3364996419 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4440bdb60 con 0x7fe4540b4780 2024-04-25T00:38:10.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe459208640 1 -- 172.21.15.29:0/3364996419 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe454132550 con 0x7fe4540b4780 2024-04-25T00:38:10.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe459208640 1 -- 172.21.15.29:0/3364996419 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe454132b10 con 0x7fe4540b4780 2024-04-25T00:38:10.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe459208640 1 -- 172.21.15.29:0/3364996419 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45406e7b0 con 0x7fe4540b4780 2024-04-25T00:38:10.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.155+0000 7fe43b7fe640 1 -- 172.21.15.29:0/3364996419 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7fe4440b33f0 con 0x7fe4540b4780 2024-04-25T00:38:10.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.159+0000 7fe43b7fe640 1 --2- 172.21.15.29:0/3364996419 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe440067b90 0x7fe44006a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.159+0000 7fe43b7fe640 1 -- 172.21.15.29:0/3364996419 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fe444120370 con 0x7fe4540b4780 2024-04-25T00:38:10.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.159+0000 7fe45259c640 1 --2- 172.21.15.29:0/3364996419 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe440067b90 0x7fe44006a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.159+0000 7fe45259c640 1 --2- 172.21.15.29:0/3364996419 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe440067b90 0x7fe44006a050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe4541f83a0 tx=0x7fe448008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.163+0000 7fe43b7fe640 1 -- 172.21.15.29:0/3364996419 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe45415a840 con 0x7fe4540b4780 2024-04-25T00:38:10.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.347+0000 7fe459208640 1 -- 172.21.15.29:0/3364996419 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone39330", "target": ["mon-mgr", ""]}) -- 0x7fe454069250 con 0x7fe440067b90 2024-04-25T00:38:10.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.375+0000 7fe43b7fe640 1 -- 172.21.15.29:0/3364996419 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe454069250 con 0x7fe440067b90 2024-04-25T00:38:10.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.379+0000 7fe459208640 1 -- 172.21.15.29:0/3364996419 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe440067b90 msgr2=0x7fe44006a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.379+0000 7fe459208640 1 --2- 172.21.15.29:0/3364996419 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe440067b90 0x7fe44006a050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe4541f83a0 tx=0x7fe448008040 comp rx=0 tx=0).stop 2024-04-25T00:38:10.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.379+0000 7fe459208640 1 -- 172.21.15.29:0/3364996419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4540b4780 msgr2=0x7fe45415a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.379+0000 7fe459208640 1 --2- 172.21.15.29:0/3364996419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4540b4780 0x7fe45415a300 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fe44401aba0 tx=0x7fe44401ad30 comp rx=0 tx=0).stop 2024-04-25T00:38:10.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.379+0000 7fe459208640 1 -- 172.21.15.29:0/3364996419 shutdown_connections 2024-04-25T00:38:10.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.379+0000 7fe459208640 1 --2- 172.21.15.29:0/3364996419 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe440067b90 0x7fe44006a050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.379+0000 7fe459208640 1 --2- 172.21.15.29:0/3364996419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe45415f1f0 0x7fe4540cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.379+0000 7fe459208640 1 --2- 172.21.15.29:0/3364996419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe45415e8d0 0x7fe4541359a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.379+0000 7fe459208640 1 --2- 172.21.15.29:0/3364996419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4540b4780 0x7fe45415a300 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.379+0000 7fe459208640 1 -- 172.21.15.29:0/3364996419 >> 172.21.15.29:0/3364996419 conn(0x7fe4540b1870 msgr2=0x7fe4540afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.379+0000 7fe459208640 1 -- 172.21.15.29:0/3364996419 shutdown_connections 2024-04-25T00:38:10.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.379+0000 7fe459208640 1 -- 172.21.15.29:0/3364996419 wait complete. 2024-04-25T00:38:10.399 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone3071 2024-04-25T00:38:10.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd729b8e640 1 Processor -- start 2024-04-25T00:38:10.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd729b8e640 1 -- start start 2024-04-25T00:38:10.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd729b8e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7240b3140 0x7fd72406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd729b8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7240b4b10 0x7fd7240b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd729b8e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7240b41e0 con 0x7fd7240b3a60 2024-04-25T00:38:10.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd729b8e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd72406d960 con 0x7fd7240b3140 2024-04-25T00:38:10.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd729b8e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd72406dc40 con 0x7fd7240b4b10 2024-04-25T00:38:10.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd722ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7240b4b10 0x7fd7240b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd7237fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7240b3140 0x7fd72406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd722ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7240b4b10 0x7fd7240b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:60416/0 (socket says 172.21.15.29:60416) 2024-04-25T00:38:10.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd722ffd640 1 -- 172.21.15.29:0/1477295056 learned_addr learned my addr 172.21.15.29:0/1477295056 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd722ffd640 1 -- 172.21.15.29:0/1477295056 >> v2:172.21.15.157:3300/0 conn(0x7fd7240b3140 msgr2=0x7fd72406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd722ffd640 1 --2- 172.21.15.29:0/1477295056 >> v2:172.21.15.157:3300/0 conn(0x7fd7240b3140 0x7fd72406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd722ffd640 1 -- 172.21.15.29:0/1477295056 >> v1:172.21.15.157:6789/0 conn(0x7fd7240b3a60 legacy=0x7fd72415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.515+0000 7fd722ffd640 1 -- 172.21.15.29:0/1477295056 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72406df20 con 0x7fd7240b4b10 2024-04-25T00:38:10.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd722ffd640 1 --2- 172.21.15.29:0/1477295056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7240b4b10 0x7fd7240b3520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd718009870 tx=0x7fd71801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ee46297ac795dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd7227fc640 1 -- 172.21.15.29:0/1477295056 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7180b0070 con 0x7fd7240b4b10 2024-04-25T00:38:10.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd7227fc640 1 -- 172.21.15.29:0/1477295056 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7180ab600 con 0x7fd7240b4b10 2024-04-25T00:38:10.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd7227fc640 1 -- 172.21.15.29:0/1477295056 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7180ab9d0 con 0x7fd7240b4b10 2024-04-25T00:38:10.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd729b8e640 1 -- 172.21.15.29:0/1477295056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7240b4b10 msgr2=0x7fd7240b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd729b8e640 1 --2- 172.21.15.29:0/1477295056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7240b4b10 0x7fd7240b3520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd718009870 tx=0x7fd71801c560 comp rx=0 tx=0).stop 2024-04-25T00:38:10.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd729b8e640 1 -- 172.21.15.29:0/1477295056 shutdown_connections 2024-04-25T00:38:10.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd729b8e640 1 --2- 172.21.15.29:0/1477295056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7240b4b10 0x7fd7240b3520 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd729b8e640 1 --2- 172.21.15.29:0/1477295056 >> v2:172.21.15.157:3300/0 conn(0x7fd7240b3140 0x7fd72406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd729b8e640 1 -- 172.21.15.29:0/1477295056 >> 172.21.15.29:0/1477295056 conn(0x7fd7240b1870 msgr2=0x7fd7240b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd729b8e640 1 -- 172.21.15.29:0/1477295056 shutdown_connections 2024-04-25T00:38:10.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd729b8e640 1 -- 172.21.15.29:0/1477295056 wait complete. 2024-04-25T00:38:10.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd729b8e640 1 Processor -- start 2024-04-25T00:38:10.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.519+0000 7fd729b8e640 1 -- start start 2024-04-25T00:38:10.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd729b8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7240b3140 0x7fd724070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd729b8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7240b3a60 0x7fd72415ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd729b8e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7240b4b10 0x7fd7240c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd729b8e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd724168b70 con 0x7fd7240b4b10 2024-04-25T00:38:10.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd729b8e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd7241689f0 con 0x7fd7240b3140 2024-04-25T00:38:10.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd729b8e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd724168cf0 con 0x7fd7240b3a60 2024-04-25T00:38:10.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd723fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7240b4b10 0x7fd7240c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd723fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7240b4b10 0x7fd7240c6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55242/0 (socket says 172.21.15.29:55242) 2024-04-25T00:38:10.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd723fff640 1 -- 172.21.15.29:0/975711592 learned_addr learned my addr 172.21.15.29:0/975711592 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:10.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd722ffd640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7240b3a60 0x7fd72415ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd7237fe640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7240b3140 0x7fd724070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd723fff640 1 -- 172.21.15.29:0/975711592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7240b3a60 msgr2=0x7fd72415ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd723fff640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7240b3a60 0x7fd72415ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd723fff640 1 -- 172.21.15.29:0/975711592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7240b3140 msgr2=0x7fd724070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd723fff640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7240b3140 0x7fd724070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd723fff640 1 -- 172.21.15.29:0/975711592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd724071370 con 0x7fd7240b4b10 2024-04-25T00:38:10.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd723fff640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7240b4b10 0x7fd7240c6570 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00c760 tx=0x7fd71c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd70bfff640 1 -- 172.21.15.29:0/975711592 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c003b80 con 0x7fd7240b4b10 2024-04-25T00:38:10.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd70bfff640 1 -- 172.21.15.29:0/975711592 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd71c003d20 con 0x7fd7240b4b10 2024-04-25T00:38:10.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd729b8e640 1 -- 172.21.15.29:0/975711592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd724071650 con 0x7fd7240b4b10 2024-04-25T00:38:10.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd70bfff640 1 -- 172.21.15.29:0/975711592 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c00aa30 con 0x7fd7240b4b10 2024-04-25T00:38:10.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd729b8e640 1 -- 172.21.15.29:0/975711592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd724071c10 con 0x7fd7240b4b10 2024-04-25T00:38:10.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd729b8e640 1 -- 172.21.15.29:0/975711592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd724069270 con 0x7fd7240b4b10 2024-04-25T00:38:10.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd70bfff640 1 -- 172.21.15.29:0/975711592 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7fd71c00abd0 con 0x7fd7240b4b10 2024-04-25T00:38:10.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd70bfff640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd710067b90 0x7fd71006a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd70bfff640 1 -- 172.21.15.29:0/975711592 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fd71c085fa0 con 0x7fd7240b4b10 2024-04-25T00:38:10.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd7237fe640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd710067b90 0x7fd71006a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.523+0000 7fd7237fe640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd710067b90 0x7fd71006a050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd724165f70 tx=0x7fd714008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.531+0000 7fd70bfff640 1 -- 172.21.15.29:0/975711592 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd71c056790 con 0x7fd7240b4b10 2024-04-25T00:38:10.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.731+0000 7fd729b8e640 1 -- 172.21.15.29:0/975711592 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone3071", "target": ["mon-mgr", ""]}) -- 0x7fd7240c6ab0 con 0x7fd710067b90 2024-04-25T00:38:10.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.759+0000 7fd70bfff640 1 -- 172.21.15.29:0/975711592 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd7240c6ab0 con 0x7fd710067b90 2024-04-25T00:38:10.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.763+0000 7fd729b8e640 1 -- 172.21.15.29:0/975711592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd710067b90 msgr2=0x7fd71006a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.763+0000 7fd729b8e640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd710067b90 0x7fd71006a050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd724165f70 tx=0x7fd714008040 comp rx=0 tx=0).stop 2024-04-25T00:38:10.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.763+0000 7fd729b8e640 1 -- 172.21.15.29:0/975711592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7240b4b10 msgr2=0x7fd7240c6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.763+0000 7fd729b8e640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7240b4b10 0x7fd7240c6570 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00c760 tx=0x7fd71c00cc20 comp rx=0 tx=0).stop 2024-04-25T00:38:10.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.763+0000 7fd729b8e640 1 -- 172.21.15.29:0/975711592 shutdown_connections 2024-04-25T00:38:10.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.763+0000 7fd729b8e640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd710067b90 0x7fd71006a050 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.763+0000 7fd729b8e640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7240b4b10 0x7fd7240c6570 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.763+0000 7fd729b8e640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7240b3a60 0x7fd72415ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.763+0000 7fd729b8e640 1 --2- 172.21.15.29:0/975711592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7240b3140 0x7fd724070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.763+0000 7fd729b8e640 1 -- 172.21.15.29:0/975711592 >> 172.21.15.29:0/975711592 conn(0x7fd7240b1870 msgr2=0x7fd72415e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.763+0000 7fd729b8e640 1 -- 172.21.15.29:0/975711592 shutdown_connections 2024-04-25T00:38:10.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10.763+0000 7fd729b8e640 1 -- 172.21.15.29:0/975711592 wait complete. 2024-04-25T00:38:10.784 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-25T00:38:10.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:38:10.828 INFO:teuthology.orchestra.run.smithi029.stdout:2 2024-04-25T00:38:10.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:38:10.834 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:38:10.834 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:38:10.877 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:38:10.920 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:38:10.922 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:38:10.952 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:38:10.952 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:38:10.952 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:38:10.953 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:38:11.000 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:11.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.167+0000 7f7796004640 1 Processor -- start 2024-04-25T00:38:11.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f7796004640 1 -- start start 2024-04-25T00:38:11.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f7796004640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77900b4780 0x7f779015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f7796004640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f77900cc240 0x7f779015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f7796004640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77900b3720 con 0x7f779015e8d0 2024-04-25T00:38:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f7796004640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f77900b42f0 con 0x7f77900cc240 2024-04-25T00:38:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f7796004640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f77900b39c0 con 0x7f77900b4780 2024-04-25T00:38:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f778f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f779015e8d0 0x7f779015ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54342/0 (socket says 172.21.15.29:54342) 2024-04-25T00:38:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f778f7fe640 1 -- 172.21.15.29:0/720523166 learned_addr learned my addr 172.21.15.29:0/720523166 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f778ffff640 1 --2- 172.21.15.29:0/720523166 >> v2:172.21.15.29:3300/0 conn(0x7f77900cc240 0x7f779015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f778e7fc640 1 -- 172.21.15.29:0/720523166 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2888197798 0 0) 0x7f77900b3720 con 0x7f779015e8d0 2024-04-25T00:38:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f778e7fc640 1 -- 172.21.15.29:0/720523166 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7778003530 con 0x7f779015e8d0 2024-04-25T00:38:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f778effd640 1 --2- 172.21.15.29:0/720523166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77900b4780 0x7f779015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f778ffff640 1 -- 172.21.15.29:0/720523166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77900b4780 msgr2=0x7f779015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f778ffff640 1 --2- 172.21.15.29:0/720523166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77900b4780 0x7f779015f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f778ffff640 1 -- 172.21.15.29:0/720523166 >> v1:172.21.15.29:6789/0 conn(0x7f779015e8d0 legacy=0x7f779015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f778ffff640 1 -- 172.21.15.29:0/720523166 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77900b3ca0 con 0x7f77900cc240 2024-04-25T00:38:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f778ffff640 1 --2- 172.21.15.29:0/720523166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77900cc240 0x7f779015c7d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f7788009d90 tx=0x7f778801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a98b55e34f5e0e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.171+0000 7f778e7fc640 1 -- 172.21.15.29:0/720523166 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77880b2070 con 0x7f77900cc240 2024-04-25T00:38:11.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f778e7fc640 1 -- 172.21.15.29:0/720523166 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77880a5900 con 0x7f77900cc240 2024-04-25T00:38:11.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f778e7fc640 1 -- 172.21.15.29:0/720523166 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77880a5cd0 con 0x7f77900cc240 2024-04-25T00:38:11.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 -- 172.21.15.29:0/720523166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77900cc240 msgr2=0x7f779015c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 --2- 172.21.15.29:0/720523166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77900cc240 0x7f779015c7d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f7788009d90 tx=0x7f778801c970 comp rx=0 tx=0).stop 2024-04-25T00:38:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 -- 172.21.15.29:0/720523166 shutdown_connections 2024-04-25T00:38:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 --2- 172.21.15.29:0/720523166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77900cc240 0x7f779015c7d0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 --2- 172.21.15.29:0/720523166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77900b4780 0x7f779015f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 -- 172.21.15.29:0/720523166 >> 172.21.15.29:0/720523166 conn(0x7f77900b1870 msgr2=0x7f77900b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 -- 172.21.15.29:0/720523166 shutdown_connections 2024-04-25T00:38:11.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 -- 172.21.15.29:0/720523166 wait complete. 2024-04-25T00:38:11.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 Processor -- start 2024-04-25T00:38:11.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 -- start start 2024-04-25T00:38:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77900b4780 0x7f779015a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77900cc240 0x7f77900ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f779015e8d0 0x7f7790071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f779015d6a0 con 0x7f779015e8d0 2024-04-25T00:38:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f779015d520 con 0x7f77900cc240 2024-04-25T00:38:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f779006d590 con 0x7f77900b4780 2024-04-25T00:38:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f778ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f779015e8d0 0x7f7790071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f778ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f779015e8d0 0x7f7790071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55260/0 (socket says 172.21.15.29:55260) 2024-04-25T00:38:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f778ffff640 1 -- 172.21.15.29:0/893226283 learned_addr learned my addr 172.21.15.29:0/893226283 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f778f7fe640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77900b4780 0x7f779015a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f778ffff640 1 -- 172.21.15.29:0/893226283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77900b4780 msgr2=0x7f779015a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f778ffff640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77900b4780 0x7f779015a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f778effd640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77900cc240 0x7f77900ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f778ffff640 1 -- 172.21.15.29:0/893226283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77900cc240 msgr2=0x7f77900ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f778ffff640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77900cc240 0x7f77900ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f778ffff640 1 -- 172.21.15.29:0/893226283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77900ae1a0 con 0x7f779015e8d0 2024-04-25T00:38:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f778ffff640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f779015e8d0 0x7f7790071e60 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f77880b0040 tx=0x7f7788019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7777fff640 1 -- 172.21.15.29:0/893226283 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77880b2050 con 0x7f779015e8d0 2024-04-25T00:38:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7777fff640 1 -- 172.21.15.29:0/893226283 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77880be070 con 0x7f779015e8d0 2024-04-25T00:38:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7777fff640 1 -- 172.21.15.29:0/893226283 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77880ada90 con 0x7f779015e8d0 2024-04-25T00:38:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 -- 172.21.15.29:0/893226283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77900ae480 con 0x7f779015e8d0 2024-04-25T00:38:11.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.175+0000 7f7796004640 1 -- 172.21.15.29:0/893226283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77900aea40 con 0x7f779015e8d0 2024-04-25T00:38:11.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.179+0000 7f7796004640 1 -- 172.21.15.29:0/893226283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7790069270 con 0x7f779015e8d0 2024-04-25T00:38:11.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.179+0000 7f7777fff640 1 -- 172.21.15.29:0/893226283 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f77880adc30 con 0x7f779015e8d0 2024-04-25T00:38:11.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.179+0000 7f7777fff640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f777c067b90 0x7f777c06a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.179+0000 7f7777fff640 1 -- 172.21.15.29:0/893226283 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f7788121280 con 0x7f779015e8d0 2024-04-25T00:38:11.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.179+0000 7f778f7fe640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f777c067b90 0x7f777c06a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.179+0000 7f778f7fe640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f777c067b90 0x7f777c06a050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7780007fd0 tx=0x7f7780009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:11.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.183+0000 7f7777fff640 1 -- 172.21.15.29:0/893226283 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7788016560 con 0x7f779015e8d0 2024-04-25T00:38:11.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.347+0000 7f7796004640 1 -- 172.21.15.29:0/893226283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f779006e7b0 con 0x7f779015e8d0 2024-04-25T00:38:11.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.351+0000 7f7777fff640 1 -- 172.21.15.29:0/893226283 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16690 (secure 0 0 0) 0x7f77880ab340 con 0x7f779015e8d0 2024-04-25T00:38:11.353 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:11.353 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":104,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:10.553442+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:37:22.602795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:37:23.310674+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":4,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":3,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":3,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:11.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.351+0000 7f7796004640 1 -- 172.21.15.29:0/893226283 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f777c067b90 msgr2=0x7f777c06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.351+0000 7f7796004640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f777c067b90 0x7f777c06a050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7780007fd0 tx=0x7f7780009040 comp rx=0 tx=0).stop 2024-04-25T00:38:11.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.351+0000 7f7796004640 1 -- 172.21.15.29:0/893226283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f779015e8d0 msgr2=0x7f7790071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.351+0000 7f7796004640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f779015e8d0 0x7f7790071e60 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f77880b0040 tx=0x7f7788019710 comp rx=0 tx=0).stop 2024-04-25T00:38:11.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.351+0000 7f7796004640 1 -- 172.21.15.29:0/893226283 shutdown_connections 2024-04-25T00:38:11.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.351+0000 7f7796004640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f777c067b90 0x7f777c06a050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.351+0000 7f7796004640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f779015e8d0 0x7f7790071e60 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.351+0000 7f7796004640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77900cc240 0x7f77900ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.351+0000 7f7796004640 1 --2- 172.21.15.29:0/893226283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77900b4780 0x7f779015a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.351+0000 7f7796004640 1 -- 172.21.15.29:0/893226283 >> 172.21.15.29:0/893226283 conn(0x7f77900b1870 msgr2=0x7f77900b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:11.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.351+0000 7f7796004640 1 -- 172.21.15.29:0/893226283 shutdown_connections 2024-04-25T00:38:11.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.351+0000 7f7796004640 1 -- 172.21.15.29:0/893226283 wait complete. 2024-04-25T00:38:11.373 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:37:22.602795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:37:23.310674+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 4, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:38:11.373 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:38:11.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb46f9c640 1 Processor -- start 2024-04-25T00:38:11.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb46f9c640 1 -- start start 2024-04-25T00:38:11.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb46f9c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feb400b3140 0x7feb4006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb46f9c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb400b41e0 con 0x7feb400b3c30 2024-04-25T00:38:11.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb46f9c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb4006d960 con 0x7feb400b4b10 2024-04-25T00:38:11.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb46f9c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7feb4006dc40 con 0x7feb400b3140 2024-04-25T00:38:11.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb45539640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7feb400b3c30 0x7feb400ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54358/0 (socket says 172.21.15.29:54358) 2024-04-25T00:38:11.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb44d38640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feb400b3140 0x7feb4006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb45539640 1 -- 172.21.15.29:0/4278416915 learned_addr learned my addr 172.21.15.29:0/4278416915 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:11.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb3f7fe640 1 -- 172.21.15.29:0/4278416915 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1996536760 0 0) 0x7feb400b41e0 con 0x7feb400b3c30 2024-04-25T00:38:11.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb3f7fe640 1 -- 172.21.15.29:0/4278416915 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb28003530 con 0x7feb400b3c30 2024-04-25T00:38:11.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb44d38640 1 -- 172.21.15.29:0/4278416915 >> v1:172.21.15.157:6789/0 conn(0x7feb400b4b10 legacy=0x7feb400b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb44d38640 1 -- 172.21.15.29:0/4278416915 >> v1:172.21.15.29:6789/0 conn(0x7feb400b3c30 legacy=0x7feb400ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.503+0000 7feb44d38640 1 -- 172.21.15.29:0/4278416915 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4006df20 con 0x7feb400b3140 2024-04-25T00:38:11.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.507+0000 7feb44d38640 1 --2- 172.21.15.29:0/4278416915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b3140 0x7feb4006e440 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7feb300097f0 tx=0x7feb3001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f89c2efdc94f471 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:11.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.507+0000 7feb3f7fe640 1 -- 172.21.15.29:0/4278416915 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300b0070 con 0x7feb400b3140 2024-04-25T00:38:11.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.507+0000 7feb3f7fe640 1 -- 172.21.15.29:0/4278416915 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb300ab600 con 0x7feb400b3140 2024-04-25T00:38:11.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.507+0000 7feb3f7fe640 1 -- 172.21.15.29:0/4278416915 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300ab9d0 con 0x7feb400b3140 2024-04-25T00:38:11.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.507+0000 7feb46f9c640 1 -- 172.21.15.29:0/4278416915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b3140 msgr2=0x7feb4006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.507+0000 7feb46f9c640 1 --2- 172.21.15.29:0/4278416915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b3140 0x7feb4006e440 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7feb300097f0 tx=0x7feb3001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.507+0000 7feb46f9c640 1 -- 172.21.15.29:0/4278416915 shutdown_connections 2024-04-25T00:38:11.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.507+0000 7feb46f9c640 1 --2- 172.21.15.29:0/4278416915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b3140 0x7feb4006e440 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.507+0000 7feb46f9c640 1 -- 172.21.15.29:0/4278416915 >> 172.21.15.29:0/4278416915 conn(0x7feb400b1870 msgr2=0x7feb400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:11.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.507+0000 7feb46f9c640 1 -- 172.21.15.29:0/4278416915 shutdown_connections 2024-04-25T00:38:11.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.507+0000 7feb46f9c640 1 -- 172.21.15.29:0/4278416915 wait complete. 2024-04-25T00:38:11.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.507+0000 7feb46f9c640 1 Processor -- start 2024-04-25T00:38:11.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.507+0000 7feb46f9c640 1 -- start start 2024-04-25T00:38:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb46f9c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb400b3140 0x7feb4013d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb46f9c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb400b3c30 0x7feb4014e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb46f9c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b4b10 0x7feb4013bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb46f9c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feb40169d90 con 0x7feb400b4b10 2024-04-25T00:38:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb46f9c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb40169c10 con 0x7feb400b3c30 2024-04-25T00:38:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb46f9c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7feb40169f10 con 0x7feb400b3140 2024-04-25T00:38:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb45539640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b4b10 0x7feb4013bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb45539640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b4b10 0x7feb4013bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55268/0 (socket says 172.21.15.29:55268) 2024-04-25T00:38:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb45539640 1 -- 172.21.15.29:0/3265965722 learned_addr learned my addr 172.21.15.29:0/3265965722 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb44d38640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb400b3140 0x7feb4013d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb3ffff640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb400b3c30 0x7feb4014e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb45539640 1 -- 172.21.15.29:0/3265965722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb400b3140 msgr2=0x7feb4013d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb45539640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb400b3140 0x7feb4013d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb45539640 1 -- 172.21.15.29:0/3265965722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb400b3c30 msgr2=0x7feb4014e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb45539640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb400b3c30 0x7feb4014e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb45539640 1 -- 172.21.15.29:0/3265965722 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4014ee50 con 0x7feb400b4b10 2024-04-25T00:38:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb44d38640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb400b3140 0x7feb4013d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb45539640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b4b10 0x7feb4013bf00 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7feb3800cad0 tx=0x7feb3800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb3d7fa640 1 -- 172.21.15.29:0/3265965722 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38017070 con 0x7feb400b4b10 2024-04-25T00:38:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb3d7fa640 1 -- 172.21.15.29:0/3265965722 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb3800ac30 con 0x7feb400b4b10 2024-04-25T00:38:11.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb46f9c640 1 -- 172.21.15.29:0/3265965722 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb4014f130 con 0x7feb400b4b10 2024-04-25T00:38:11.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb3d7fa640 1 -- 172.21.15.29:0/3265965722 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38012400 con 0x7feb400b4b10 2024-04-25T00:38:11.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb46f9c640 1 -- 172.21.15.29:0/3265965722 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb4014f6f0 con 0x7feb400b4b10 2024-04-25T00:38:11.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb46f9c640 1 -- 172.21.15.29:0/3265965722 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb40069270 con 0x7feb400b4b10 2024-04-25T00:38:11.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb3d7fa640 1 -- 172.21.15.29:0/3265965722 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7feb3801a070 con 0x7feb400b4b10 2024-04-25T00:38:11.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb3d7fa640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feb2c067b90 0x7feb2c06a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb3d7fa640 1 -- 172.21.15.29:0/3265965722 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7feb38086010 con 0x7feb400b4b10 2024-04-25T00:38:11.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb44d38640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feb2c067b90 0x7feb2c06a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.511+0000 7feb44d38640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feb2c067b90 0x7feb2c06a050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feb300ad000 tx=0x7feb30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:11.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.515+0000 7feb3d7fa640 1 -- 172.21.15.29:0/3265965722 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feb380566e0 con 0x7feb400b4b10 2024-04-25T00:38:11.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.687+0000 7feb46f9c640 1 -- 172.21.15.29:0/3265965722 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feb4006efe0 con 0x7feb400b4b10 2024-04-25T00:38:11.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.691+0000 7feb3d7fa640 1 -- 172.21.15.29:0/3265965722 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7feb38055fe0 con 0x7feb400b4b10 2024-04-25T00:38:11.692 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:38:11.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.691+0000 7feb46f9c640 1 -- 172.21.15.29:0/3265965722 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feb2c067b90 msgr2=0x7feb2c06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.691+0000 7feb46f9c640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feb2c067b90 0x7feb2c06a050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feb300ad000 tx=0x7feb30008040 comp rx=0 tx=0).stop 2024-04-25T00:38:11.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.691+0000 7feb46f9c640 1 -- 172.21.15.29:0/3265965722 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b4b10 msgr2=0x7feb4013bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.691+0000 7feb46f9c640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b4b10 0x7feb4013bf00 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7feb3800cad0 tx=0x7feb3800cf90 comp rx=0 tx=0).stop 2024-04-25T00:38:11.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.691+0000 7feb46f9c640 1 -- 172.21.15.29:0/3265965722 shutdown_connections 2024-04-25T00:38:11.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.691+0000 7feb46f9c640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feb2c067b90 0x7feb2c06a050 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.691+0000 7feb46f9c640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b4b10 0x7feb4013bf00 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.691+0000 7feb46f9c640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb400b3c30 0x7feb4014e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.691+0000 7feb46f9c640 1 --2- 172.21.15.29:0/3265965722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb400b3140 0x7feb4013d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.691+0000 7feb46f9c640 1 -- 172.21.15.29:0/3265965722 >> 172.21.15.29:0/3265965722 conn(0x7feb400b1870 msgr2=0x7feb40144b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:11.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.691+0000 7feb46f9c640 1 -- 172.21.15.29:0/3265965722 shutdown_connections 2024-04-25T00:38:11.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.691+0000 7feb46f9c640 1 -- 172.21.15.29:0/3265965722 wait complete. 2024-04-25T00:38:11.714 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:38:11.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.839+0000 7f719c5c7640 1 Processor -- start 2024-04-25T00:38:11.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.843+0000 7f719c5c7640 1 -- start start 2024-04-25T00:38:11.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.843+0000 7f719c5c7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f71940b4d30 0x7f71940b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.843+0000 7f719c5c7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f719406d6c0 con 0x7f719406d210 2024-04-25T00:38:11.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.843+0000 7f719c5c7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f719406d980 con 0x7f719415df70 2024-04-25T00:38:11.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.843+0000 7f719c5c7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f719406dc60 con 0x7f71940b4d30 2024-04-25T00:38:11.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.843+0000 7f7199b62640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f71940b4d30 0x7f71940b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.843+0000 7f7199b62640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f71940b4d30 0x7f71940b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55270/0 (socket says 172.21.15.29:55270) 2024-04-25T00:38:11.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.843+0000 7f7199b62640 1 -- 172.21.15.29:0/2788964060 learned_addr learned my addr 172.21.15.29:0/2788964060 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:11.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.843+0000 7f719ab64640 1 --1- 172.21.15.29:0/2788964060 >> v1:172.21.15.29:6789/0 conn(0x7f719406d210 0x7f71940ad8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54368/0 (socket says 172.21.15.29:54368) 2024-04-25T00:38:11.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.843+0000 7f7199b62640 1 -- 172.21.15.29:0/2788964060 >> v1:172.21.15.157:6789/0 conn(0x7f719415df70 legacy=0x7f71940b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.843+0000 7f7199b62640 1 -- 172.21.15.29:0/2788964060 >> v1:172.21.15.29:6789/0 conn(0x7f719406d210 legacy=0x7f71940ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.843+0000 7f7199361640 1 -- 172.21.15.29:0/2788964060 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 911418414 0 0) 0x7f719406d6c0 con 0x7f719406d210 2024-04-25T00:38:11.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.843+0000 7f7199b62640 1 -- 172.21.15.29:0/2788964060 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719406df40 con 0x7f71940b4d30 2024-04-25T00:38:11.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f7199b62640 1 --2- 172.21.15.29:0/2788964060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71940b4d30 0x7f71940b3e40 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7184009870 tx=0x7f718401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04fed0a3d6a5b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:11.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f7199361640 1 -- 172.21.15.29:0/2788964060 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71840a95f0 con 0x7f71940b4d30 2024-04-25T00:38:11.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f7199361640 1 -- 172.21.15.29:0/2788964060 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71840a9790 con 0x7f71940b4d30 2024-04-25T00:38:11.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f7199361640 1 -- 172.21.15.29:0/2788964060 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71840a9b60 con 0x7f71940b4d30 2024-04-25T00:38:11.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 -- 172.21.15.29:0/2788964060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71940b4d30 msgr2=0x7f71940b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 --2- 172.21.15.29:0/2788964060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71940b4d30 0x7f71940b3e40 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7184009870 tx=0x7f718401c530 comp rx=0 tx=0).stop 2024-04-25T00:38:11.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 -- 172.21.15.29:0/2788964060 shutdown_connections 2024-04-25T00:38:11.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 --2- 172.21.15.29:0/2788964060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71940b4d30 0x7f71940b3e40 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 -- 172.21.15.29:0/2788964060 >> 172.21.15.29:0/2788964060 conn(0x7f71940b1870 msgr2=0x7f71940b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:11.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 -- 172.21.15.29:0/2788964060 shutdown_connections 2024-04-25T00:38:11.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 -- 172.21.15.29:0/2788964060 wait complete. 2024-04-25T00:38:11.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 Processor -- start 2024-04-25T00:38:11.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 -- start start 2024-04-25T00:38:11.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719406d210 0x7f7194158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71940b4d30 0x7f719415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f719415df70 0x7f719415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f719406eba0 con 0x7f719415df70 2024-04-25T00:38:11.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f719406ea20 con 0x7f71940b4d30 2024-04-25T00:38:11.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719c5c7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f719416bbc0 con 0x7f719406d210 2024-04-25T00:38:11.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719ab64640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f719415df70 0x7f719415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719ab64640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f719415df70 0x7f719415b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55282/0 (socket says 172.21.15.29:55282) 2024-04-25T00:38:11.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719ab64640 1 -- 172.21.15.29:0/1316484889 learned_addr learned my addr 172.21.15.29:0/1316484889 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:11.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f7199b62640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71940b4d30 0x7f719415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719a363640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719406d210 0x7f7194158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719ab64640 1 -- 172.21.15.29:0/1316484889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719406d210 msgr2=0x7f7194158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719ab64640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719406d210 0x7f7194158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719ab64640 1 -- 172.21.15.29:0/1316484889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71940b4d30 msgr2=0x7f719415b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:11.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719ab64640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71940b4d30 0x7f719415b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:11.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719ab64640 1 -- 172.21.15.29:0/1316484889 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7194157760 con 0x7f719415df70 2024-04-25T00:38:11.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f719ab64640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f719415df70 0x7f719415b5f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c7a0 tx=0x7f718c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:11.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.847+0000 7f718affd640 1 -- 172.21.15.29:0/1316484889 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c018070 con 0x7f719415df70 2024-04-25T00:38:11.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.851+0000 7f718affd640 1 -- 172.21.15.29:0/1316484889 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f718c00a640 con 0x7f719415df70 2024-04-25T00:38:11.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.851+0000 7f719c5c7640 1 -- 172.21.15.29:0/1316484889 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7194157a40 con 0x7f719415df70 2024-04-25T00:38:11.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.851+0000 7f718affd640 1 -- 172.21.15.29:0/1316484889 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c00aab0 con 0x7f719415df70 2024-04-25T00:38:11.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.851+0000 7f719c5c7640 1 -- 172.21.15.29:0/1316484889 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7194158000 con 0x7f719415df70 2024-04-25T00:38:11.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.851+0000 7f719c5c7640 1 -- 172.21.15.29:0/1316484889 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7194069270 con 0x7f719415df70 2024-04-25T00:38:11.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.851+0000 7f718affd640 1 -- 172.21.15.29:0/1316484889 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f718c01b070 con 0x7f719415df70 2024-04-25T00:38:11.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.851+0000 7f718affd640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7178067b90 0x7f717806a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:11.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.851+0000 7f718affd640 1 -- 172.21.15.29:0/1316484889 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f718c085720 con 0x7f719415df70 2024-04-25T00:38:11.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.851+0000 7f719a363640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7178067b90 0x7f717806a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:11.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.851+0000 7f719a363640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7178067b90 0x7f717806a050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7190000f70 tx=0x7f7190009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:11.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:11.855+0000 7f718affd640 1 -- 172.21.15.29:0/1316484889 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f718c055f50 con 0x7f719415df70 2024-04-25T00:38:12.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.027+0000 7f719c5c7640 1 -- 172.21.15.29:0/1316484889 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f71940b46f0 con 0x7f719415df70 2024-04-25T00:38:12.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.027+0000 7f718affd640 1 -- 172.21.15.29:0/1316484889 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f718c055850 con 0x7f719415df70 2024-04-25T00:38:12.031 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:38:12.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.031+0000 7f719c5c7640 1 -- 172.21.15.29:0/1316484889 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7178067b90 msgr2=0x7f717806a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.031+0000 7f719c5c7640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7178067b90 0x7f717806a050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7190000f70 tx=0x7f7190009040 comp rx=0 tx=0).stop 2024-04-25T00:38:12.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.031+0000 7f719c5c7640 1 -- 172.21.15.29:0/1316484889 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f719415df70 msgr2=0x7f719415b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.031+0000 7f719c5c7640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f719415df70 0x7f719415b5f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c7a0 tx=0x7f718c00cc60 comp rx=0 tx=0).stop 2024-04-25T00:38:12.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.031+0000 7f719c5c7640 1 -- 172.21.15.29:0/1316484889 shutdown_connections 2024-04-25T00:38:12.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.031+0000 7f719c5c7640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7178067b90 0x7f717806a050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.031+0000 7f719c5c7640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f719415df70 0x7f719415b5f0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.031+0000 7f719c5c7640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71940b4d30 0x7f719415b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.031+0000 7f719c5c7640 1 --2- 172.21.15.29:0/1316484889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f719406d210 0x7f7194158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.031+0000 7f719c5c7640 1 -- 172.21.15.29:0/1316484889 >> 172.21.15.29:0/1316484889 conn(0x7f71940b1870 msgr2=0x7f719415d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:12.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.031+0000 7f719c5c7640 1 -- 172.21.15.29:0/1316484889 shutdown_connections 2024-04-25T00:38:12.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.031+0000 7f719c5c7640 1 -- 172.21.15.29:0/1316484889 wait complete. 2024-04-25T00:38:12.050 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:38:12.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f52423c1640 1 Processor -- start 2024-04-25T00:38:12.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f52423c1640 1 -- start start 2024-04-25T00:38:12.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f52423c1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f523c0b3c30 0x7f523c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f52423c1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f523c0b4b10 0x7f523c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f52423c1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f523c0b41e0 con 0x7f523c0b3140 2024-04-25T00:38:12.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f52423c1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f523c06d960 con 0x7f523c0b4b10 2024-04-25T00:38:12.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f52423c1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f523c06dc40 con 0x7f523c0b3c30 2024-04-25T00:38:12.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f524095e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f523c0b4b10 0x7f523c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f524095e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f523c0b4b10 0x7f523c0ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55294/0 (socket says 172.21.15.29:55294) 2024-04-25T00:38:12.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f524095e640 1 -- 172.21.15.29:0/3550973346 learned_addr learned my addr 172.21.15.29:0/3550973346 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:12.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f523b7fe640 1 --2- 172.21.15.29:0/3550973346 >> v2:172.21.15.157:3300/0 conn(0x7f523c0b3c30 0x7f523c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f524095e640 1 -- 172.21.15.29:0/3550973346 >> v2:172.21.15.157:3300/0 conn(0x7f523c0b3c30 msgr2=0x7f523c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:12.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f524095e640 1 --2- 172.21.15.29:0/3550973346 >> v2:172.21.15.157:3300/0 conn(0x7f523c0b3c30 0x7f523c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f524095e640 1 -- 172.21.15.29:0/3550973346 >> v1:172.21.15.157:6789/0 conn(0x7f523c0b3140 legacy=0x7f523c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:12.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.171+0000 7f524095e640 1 -- 172.21.15.29:0/3550973346 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c06df20 con 0x7f523c0b4b10 2024-04-25T00:38:12.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f524095e640 1 --2- 172.21.15.29:0/3550973346 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523c0b4b10 0x7f523c0ce240 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5234009e50 tx=0x7f523401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54bcb6391007299 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:12.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f523affd640 1 -- 172.21.15.29:0/3550973346 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52340a5670 con 0x7f523c0b4b10 2024-04-25T00:38:12.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f523affd640 1 -- 172.21.15.29:0/3550973346 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52340a5810 con 0x7f523c0b4b10 2024-04-25T00:38:12.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f523affd640 1 -- 172.21.15.29:0/3550973346 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52340a5be0 con 0x7f523c0b4b10 2024-04-25T00:38:12.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 -- 172.21.15.29:0/3550973346 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523c0b4b10 msgr2=0x7f523c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 --2- 172.21.15.29:0/3550973346 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523c0b4b10 0x7f523c0ce240 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5234009e50 tx=0x7f523401c9b0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 -- 172.21.15.29:0/3550973346 shutdown_connections 2024-04-25T00:38:12.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 --2- 172.21.15.29:0/3550973346 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523c0b4b10 0x7f523c0ce240 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 --2- 172.21.15.29:0/3550973346 >> v2:172.21.15.157:3300/0 conn(0x7f523c0b3c30 0x7f523c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 -- 172.21.15.29:0/3550973346 >> 172.21.15.29:0/3550973346 conn(0x7f523c0b1870 msgr2=0x7f523c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:12.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 -- 172.21.15.29:0/3550973346 shutdown_connections 2024-04-25T00:38:12.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 -- 172.21.15.29:0/3550973346 wait complete. 2024-04-25T00:38:12.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 Processor -- start 2024-04-25T00:38:12.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 -- start start 2024-04-25T00:38:12.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523c0b3140 0x7f523c15e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f523c0b3c30 0x7f523c157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f523c0b4b10 0x7f523c1392f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f523c168d80 con 0x7f523c0b3140 2024-04-25T00:38:12.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f523c168c00 con 0x7f523c0b3c30 2024-04-25T00:38:12.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f52423c1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f523c168f00 con 0x7f523c0b4b10 2024-04-25T00:38:12.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f523bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523c0b3140 0x7f523c15e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.175+0000 7f523bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523c0b3140 0x7f523c15e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55296/0 (socket says 172.21.15.29:55296) 2024-04-25T00:38:12.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f523bfff640 1 -- 172.21.15.29:0/318667509 learned_addr learned my addr 172.21.15.29:0/318667509 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:12.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f523b7fe640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f523c0b3c30 0x7f523c157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f524095e640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f523c0b4b10 0x7f523c1392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f523bfff640 1 -- 172.21.15.29:0/318667509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f523c0b4b10 msgr2=0x7f523c1392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f523bfff640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f523c0b4b10 0x7f523c1392f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f523bfff640 1 -- 172.21.15.29:0/318667509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f523c0b3c30 msgr2=0x7f523c157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f523bfff640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f523c0b3c30 0x7f523c157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f523bfff640 1 -- 172.21.15.29:0/318667509 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c159880 con 0x7f523c0b3140 2024-04-25T00:38:12.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f523b7fe640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f523c0b3c30 0x7f523c157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:12.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f523bfff640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523c0b3140 0x7f523c15e6c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c450 tx=0x7f522c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:12.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f5238ff9640 1 -- 172.21.15.29:0/318667509 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c012830 con 0x7f523c0b3140 2024-04-25T00:38:12.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f5238ff9640 1 -- 172.21.15.29:0/318667509 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f522c017070 con 0x7f523c0b3140 2024-04-25T00:38:12.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f52423c1640 1 -- 172.21.15.29:0/318667509 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c159b40 con 0x7f523c0b3140 2024-04-25T00:38:12.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f5238ff9640 1 -- 172.21.15.29:0/318667509 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c012c30 con 0x7f523c0b3140 2024-04-25T00:38:12.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f52423c1640 1 -- 172.21.15.29:0/318667509 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c15a100 con 0x7f523c0b3140 2024-04-25T00:38:12.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f52423c1640 1 -- 172.21.15.29:0/318667509 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523c15a540 con 0x7f523c0b3140 2024-04-25T00:38:12.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f5238ff9640 1 -- 172.21.15.29:0/318667509 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f522c01c460 con 0x7f523c0b3140 2024-04-25T00:38:12.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f5238ff9640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f521c067b90 0x7f521c06a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f5238ff9640 1 -- 172.21.15.29:0/318667509 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f522c085f10 con 0x7f523c0b3140 2024-04-25T00:38:12.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f523b7fe640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f521c067b90 0x7f521c06a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.179+0000 7f523b7fe640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f521c067b90 0x7f521c06a050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f523c152ce0 tx=0x7f5224008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:12.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.183+0000 7f5238ff9640 1 -- 172.21.15.29:0/318667509 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f522c0566b0 con 0x7f523c0b3140 2024-04-25T00:38:12.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.343+0000 7f52423c1640 1 -- 172.21.15.29:0/318667509 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f523c069270 con 0x7f523c0b3140 2024-04-25T00:38:12.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.343+0000 7f5238ff9640 1 -- 172.21.15.29:0/318667509 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f522c055fb0 con 0x7f523c0b3140 2024-04-25T00:38:12.347 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:38:12.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.347+0000 7f52423c1640 1 -- 172.21.15.29:0/318667509 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f521c067b90 msgr2=0x7f521c06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.347+0000 7f52423c1640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f521c067b90 0x7f521c06a050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f523c152ce0 tx=0x7f5224008040 comp rx=0 tx=0).stop 2024-04-25T00:38:12.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.347+0000 7f52423c1640 1 -- 172.21.15.29:0/318667509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523c0b3140 msgr2=0x7f523c15e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.347+0000 7f52423c1640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523c0b3140 0x7f523c15e6c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c450 tx=0x7f522c00c910 comp rx=0 tx=0).stop 2024-04-25T00:38:12.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.347+0000 7f52423c1640 1 -- 172.21.15.29:0/318667509 shutdown_connections 2024-04-25T00:38:12.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.347+0000 7f52423c1640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f521c067b90 0x7f521c06a050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.347+0000 7f52423c1640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f523c0b4b10 0x7f523c1392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.347+0000 7f52423c1640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f523c0b3c30 0x7f523c157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.347+0000 7f52423c1640 1 --2- 172.21.15.29:0/318667509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523c0b3140 0x7f523c15e6c0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.347+0000 7f52423c1640 1 -- 172.21.15.29:0/318667509 >> 172.21.15.29:0/318667509 conn(0x7f523c0b1870 msgr2=0x7f523c0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:12.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.347+0000 7f52423c1640 1 -- 172.21.15.29:0/318667509 shutdown_connections 2024-04-25T00:38:12.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.347+0000 7f52423c1640 1 -- 172.21.15.29:0/318667509 wait complete. 2024-04-25T00:38:12.367 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:38:12.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f69481e4640 1 Processor -- start 2024-04-25T00:38:12.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f69481e4640 1 -- start start 2024-04-25T00:38:12.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f69481e4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69400b3c30 0x7f694006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f69481e4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69400b41e0 con 0x7f69400b3140 2024-04-25T00:38:12.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f69481e4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f694006d960 con 0x7f69400b4b10 2024-04-25T00:38:12.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f69481e4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f694006dc40 con 0x7f69400b3c30 2024-04-25T00:38:12.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f6945f80640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f69400b3140 0x7f69400b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54380/0 (socket says 172.21.15.29:54380) 2024-04-25T00:38:12.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f6945f80640 1 -- 172.21.15.29:0/2704710968 learned_addr learned my addr 172.21.15.29:0/2704710968 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:12.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f694577f640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69400b3c30 0x7f694006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3854863171 0 0) 0x7f69400b41e0 con 0x7f69400b3140 2024-04-25T00:38:12.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6928003530 con 0x7f69400b3140 2024-04-25T00:38:12.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 626019699 0 0) 0x7f6928003530 con 0x7f69400b3140 2024-04-25T00:38:12.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69400b3c30 msgr2=0x7f694006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:12.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f6944f7e640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69400b3c30 0x7f694006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 >> v1:172.21.15.157:6789/0 conn(0x7f69400b4b10 legacy=0x7f69400ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694006df20 con 0x7f69400b3140 2024-04-25T00:38:12.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.491+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f693c001a00 con 0x7f69400b3140 2024-04-25T00:38:12.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 >> v1:172.21.15.29:6789/0 conn(0x7f69400b3140 legacy=0x7f69400b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6944f7e640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69280039f0 0x7f6928003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6944f7e640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6928004810 0x7f6928004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6944f7e640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69400b3c30 0x7f69280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f694006dc40 con 0x7f6928004810 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f692800bab0 con 0x7f69280039f0 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f692800bcb0 con 0x7f69400b3c30 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f694577f640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6928004810 0x7f6928004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6945f80640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69280039f0 0x7f6928003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6946781640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69400b3c30 0x7f69280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f694577f640 1 -- 172.21.15.29:0/2704710968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69400b3c30 msgr2=0x7f69280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f694577f640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69400b3c30 0x7f69280051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f694577f640 1 -- 172.21.15.29:0/2704710968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69280039f0 msgr2=0x7f6928003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f694577f640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69280039f0 0x7f6928003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f694577f640 1 -- 172.21.15.29:0/2704710968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6928003530 con 0x7f6928004810 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6946781640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69400b3c30 0x7f69280051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:12.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6945f80640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69280039f0 0x7f6928003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:12.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f694577f640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6928004810 0x7f6928004c90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6930009a90 tx=0x7f693001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1515e2c04b5ffb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:12.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69300b2070 con 0x7f6928004810 2024-04-25T00:38:12.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69300a5600 con 0x7f6928004810 2024-04-25T00:38:12.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6944f7e640 1 -- 172.21.15.29:0/2704710968 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930009690 con 0x7f6928004810 2024-04-25T00:38:12.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f69481e4640 1 -- 172.21.15.29:0/2704710968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6928004810 msgr2=0x7f6928004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f69481e4640 1 --2- 172.21.15.29:0/2704710968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6928004810 0x7f6928004c90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6930009a90 tx=0x7f693001ca10 comp rx=0 tx=0).stop 2024-04-25T00:38:12.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f6946781640 1 -- 172.21.15.29:0/2704710968 reap_dead start 2024-04-25T00:38:12.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f69481e4640 1 -- 172.21.15.29:0/2704710968 shutdown_connections 2024-04-25T00:38:12.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f69481e4640 1 -- 172.21.15.29:0/2704710968 >> 172.21.15.29:0/2704710968 conn(0x7f69400b1870 msgr2=0x7f69400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:12.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f69481e4640 1 -- 172.21.15.29:0/2704710968 shutdown_connections 2024-04-25T00:38:12.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.495+0000 7f69481e4640 1 -- 172.21.15.29:0/2704710968 wait complete. 2024-04-25T00:38:12.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69481e4640 1 Processor -- start 2024-04-25T00:38:12.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69481e4640 1 -- start start 2024-04-25T00:38:12.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69481e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6928004810 0x7f69401effe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69481e4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69401394c0 0x7f6940138c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69481e4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6940156050 0x7f69400cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69481e4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6940145a90 con 0x7f6940156050 2024-04-25T00:38:12.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69481e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6940145910 con 0x7f6928004810 2024-04-25T00:38:12.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69481e4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f694006e440 con 0x7f69401394c0 2024-04-25T00:38:12.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f6946781640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6940156050 0x7f69400cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f6946781640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6940156050 0x7f69400cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55322/0 (socket says 172.21.15.29:55322) 2024-04-25T00:38:12.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f6946781640 1 -- 172.21.15.29:0/919650139 learned_addr learned my addr 172.21.15.29:0/919650139 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:12.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f694577f640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69401394c0 0x7f6940138c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f6945f80640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6928004810 0x7f69401effe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f6946781640 1 -- 172.21.15.29:0/919650139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69401394c0 msgr2=0x7f6940138c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f6946781640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69401394c0 0x7f6940138c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f6946781640 1 -- 172.21.15.29:0/919650139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6928004810 msgr2=0x7f69401effe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f6946781640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6928004810 0x7f69401effe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f6946781640 1 -- 172.21.15.29:0/919650139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69401f0520 con 0x7f6940156050 2024-04-25T00:38:12.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f6945f80640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6928004810 0x7f69401effe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:12.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f6946781640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6940156050 0x7f69400cfd40 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f69400b4ef0 tx=0x7f693800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:12.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69367fc640 1 -- 172.21.15.29:0/919650139 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938007030 con 0x7f6940156050 2024-04-25T00:38:12.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69367fc640 1 -- 172.21.15.29:0/919650139 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69380071f0 con 0x7f6940156050 2024-04-25T00:38:12.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69367fc640 1 -- 172.21.15.29:0/919650139 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69380075e0 con 0x7f6940156050 2024-04-25T00:38:12.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69481e4640 1 -- 172.21.15.29:0/919650139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694013bd70 con 0x7f6940156050 2024-04-25T00:38:12.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69481e4640 1 -- 172.21.15.29:0/919650139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f694013c330 con 0x7f6940156050 2024-04-25T00:38:12.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69481e4640 1 -- 172.21.15.29:0/919650139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6940069270 con 0x7f6940156050 2024-04-25T00:38:12.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.499+0000 7f69367fc640 1 -- 172.21.15.29:0/919650139 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f693801b020 con 0x7f6940156050 2024-04-25T00:38:12.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.503+0000 7f69367fc640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f692c067b90 0x7f692c06a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:12.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.503+0000 7f69367fc640 1 -- 172.21.15.29:0/919650139 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f6938085a70 con 0x7f6940156050 2024-04-25T00:38:12.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.503+0000 7f6945f80640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f692c067b90 0x7f692c06a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:12.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.503+0000 7f6945f80640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f692c067b90 0x7f692c06a050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f693c000f90 tx=0x7f693c001860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:12.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.507+0000 7f69367fc640 1 -- 172.21.15.29:0/919650139 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6938056190 con 0x7f6940156050 2024-04-25T00:38:12.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.583+0000 7f69367fc640 1 -- 172.21.15.29:0/919650139 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f6938052100 con 0x7f6940156050 2024-04-25T00:38:12.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69481e4640 1 -- 172.21.15.29:0/919650139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f694006ea30 con 0x7f6940156050 2024-04-25T00:38:12.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69367fc640 1 -- 172.21.15.29:0/919650139 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7f693805c5a0 con 0x7f6940156050 2024-04-25T00:38:12.692 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:38:12.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69481e4640 1 -- 172.21.15.29:0/919650139 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f692c067b90 msgr2=0x7f692c06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69481e4640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f692c067b90 0x7f692c06a050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f693c000f90 tx=0x7f693c001860 comp rx=0 tx=0).stop 2024-04-25T00:38:12.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69481e4640 1 -- 172.21.15.29:0/919650139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6940156050 msgr2=0x7f69400cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:12.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69481e4640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6940156050 0x7f69400cfd40 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f69400b4ef0 tx=0x7f693800d890 comp rx=0 tx=0).stop 2024-04-25T00:38:12.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69481e4640 1 -- 172.21.15.29:0/919650139 shutdown_connections 2024-04-25T00:38:12.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69481e4640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f692c067b90 0x7f692c06a050 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69481e4640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6940156050 0x7f69400cfd40 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69481e4640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69401394c0 0x7f6940138c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69481e4640 1 --2- 172.21.15.29:0/919650139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6928004810 0x7f69401effe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69481e4640 1 -- 172.21.15.29:0/919650139 >> 172.21.15.29:0/919650139 conn(0x7f69400b1870 msgr2=0x7f6940071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69481e4640 1 -- 172.21.15.29:0/919650139 shutdown_connections 2024-04-25T00:38:12.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:12.691+0000 7f69481e4640 1 -- 172.21.15.29:0/919650139 wait complete. 2024-04-25T00:38:12.712 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-04-25T00:38:12.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:12.712 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:38:12.723 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[48075]: fuse finished with error 0 and tester_r 0 2024-04-25T00:38:12.725 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:38:12.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:12.725 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:38:12.875 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:38:12.875 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:38:12.875 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:38:12.882 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:13.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4ef200c640 1 Processor -- start 2024-04-25T00:38:13.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4ef200c640 1 -- start start 2024-04-25T00:38:13.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4ef200c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4eec06dd10 0x7f4eec0cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4ef200c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eec0cb330 0x7f4eec0ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4ef200c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eec06e1a0 con 0x7f4eec15bd60 2024-04-25T00:38:13.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4ef200c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4eec06edb0 con 0x7f4eec06dd10 2024-04-25T00:38:13.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4ef200c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4eec06e480 con 0x7f4eec0cb330 2024-04-25T00:38:13.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4eeb7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4eec15bd60 0x7f4eec06b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54392/0 (socket says 172.21.15.29:54392) 2024-04-25T00:38:13.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4eeb7fe640 1 -- 172.21.15.29:0/2291886424 learned_addr learned my addr 172.21.15.29:0/2291886424 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:13.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4eeaffd640 1 --2- 172.21.15.29:0/2291886424 >> v2:172.21.15.29:3300/0 conn(0x7f4eec06dd10 0x7f4eec0cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4eebfff640 1 --2- 172.21.15.29:0/2291886424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eec0cb330 0x7f4eec0ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4eeaffd640 1 -- 172.21.15.29:0/2291886424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eec0cb330 msgr2=0x7f4eec0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:13.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4eeaffd640 1 --2- 172.21.15.29:0/2291886424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eec0cb330 0x7f4eec0ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4eeaffd640 1 -- 172.21.15.29:0/2291886424 >> v1:172.21.15.29:6789/0 conn(0x7f4eec15bd60 legacy=0x7f4eec06b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4eeaffd640 1 -- 172.21.15.29:0/2291886424 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec06e760 con 0x7f4eec06dd10 2024-04-25T00:38:13.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4eea7fc640 1 -- 172.21.15.29:0/2291886424 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3549265761 0 0) 0x7f4eec06e1a0 con 0x7f4eec15bd60 2024-04-25T00:38:13.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4eeaffd640 1 --2- 172.21.15.29:0/2291886424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eec06dd10 0x7f4eec0cca80 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00097c0 tx=0x7f4ee001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a84b2f231ef4575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:13.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.011+0000 7f4eea7fc640 1 -- 172.21.15.29:0/2291886424 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00aa5f0 con 0x7f4eec06dd10 2024-04-25T00:38:13.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4eea7fc640 1 -- 172.21.15.29:0/2291886424 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee00af070 con 0x7f4eec06dd10 2024-04-25T00:38:13.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4eea7fc640 1 -- 172.21.15.29:0/2291886424 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00aa9b0 con 0x7f4eec06dd10 2024-04-25T00:38:13.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 -- 172.21.15.29:0/2291886424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eec06dd10 msgr2=0x7f4eec0cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 --2- 172.21.15.29:0/2291886424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eec06dd10 0x7f4eec0cca80 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00097c0 tx=0x7f4ee001c470 comp rx=0 tx=0).stop 2024-04-25T00:38:13.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 -- 172.21.15.29:0/2291886424 shutdown_connections 2024-04-25T00:38:13.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 --2- 172.21.15.29:0/2291886424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eec0cb330 0x7f4eec0ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 --2- 172.21.15.29:0/2291886424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eec06dd10 0x7f4eec0cca80 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 -- 172.21.15.29:0/2291886424 >> 172.21.15.29:0/2291886424 conn(0x7f4eec06c320 msgr2=0x7f4eec06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:13.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 -- 172.21.15.29:0/2291886424 shutdown_connections 2024-04-25T00:38:13.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 -- 172.21.15.29:0/2291886424 wait complete. 2024-04-25T00:38:13.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 Processor -- start 2024-04-25T00:38:13.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 -- start start 2024-04-25T00:38:13.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eec0cb330 0x7f4eec1584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eec15bd60 0x7f4eec156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eec157080 0x7f4eec0c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec06baa0 con 0x7f4eec15bd60 2024-04-25T00:38:13.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec06b920 con 0x7f4eec0cb330 2024-04-25T00:38:13.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4eec072bc0 con 0x7f4eec157080 2024-04-25T00:38:13.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4eeaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eec15bd60 0x7f4eec156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4eeaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eec15bd60 0x7f4eec156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55338/0 (socket says 172.21.15.29:55338) 2024-04-25T00:38:13.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4eeaffd640 1 -- 172.21.15.29:0/1232927424 learned_addr learned my addr 172.21.15.29:0/1232927424 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:13.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4eebfff640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eec157080 0x7f4eec0c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4eeb7fe640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eec0cb330 0x7f4eec1584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4eeaffd640 1 -- 172.21.15.29:0/1232927424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eec157080 msgr2=0x7f4eec0c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4eeaffd640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eec157080 0x7f4eec0c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4eeaffd640 1 -- 172.21.15.29:0/1232927424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eec0cb330 msgr2=0x7f4eec1584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4eeaffd640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eec0cb330 0x7f4eec1584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4eeaffd640 1 -- 172.21.15.29:0/1232927424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec1f8a60 con 0x7f4eec15bd60 2024-04-25T00:38:13.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4eeaffd640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eec15bd60 0x7f4eec156b40 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00ac040 tx=0x7f4ee001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:13.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ed3fff640 1 -- 172.21.15.29:0/1232927424 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00be610 con 0x7f4eec15bd60 2024-04-25T00:38:13.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ed3fff640 1 -- 172.21.15.29:0/1232927424 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee00be7f0 con 0x7f4eec15bd60 2024-04-25T00:38:13.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ed3fff640 1 -- 172.21.15.29:0/1232927424 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00bebe0 con 0x7f4eec15bd60 2024-04-25T00:38:13.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 -- 172.21.15.29:0/1232927424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eec1f8d40 con 0x7f4eec15bd60 2024-04-25T00:38:13.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.015+0000 7f4ef200c640 1 -- 172.21.15.29:0/1232927424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eec153eb0 con 0x7f4eec15bd60 2024-04-25T00:38:13.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.019+0000 7f4ef200c640 1 -- 172.21.15.29:0/1232927424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eec069270 con 0x7f4eec15bd60 2024-04-25T00:38:13.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.019+0000 7f4ed3fff640 1 -- 172.21.15.29:0/1232927424 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f4ee00bc020 con 0x7f4eec15bd60 2024-04-25T00:38:13.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.019+0000 7f4ed3fff640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4ed8067cb0 0x7f4ed806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.019+0000 7f4ed3fff640 1 -- 172.21.15.29:0/1232927424 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f4ee0120860 con 0x7f4eec15bd60 2024-04-25T00:38:13.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.019+0000 7f4eeb7fe640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4ed8067cb0 0x7f4ed806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.019+0000 7f4eeb7fe640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4ed8067cb0 0x7f4ed806a170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4edc000f70 tx=0x7f4edc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:13.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.023+0000 7f4ed3fff640 1 -- 172.21.15.29:0/1232927424 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4ee00c5030 con 0x7f4eec15bd60 2024-04-25T00:38:13.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.187+0000 7f4ef200c640 1 -- 172.21.15.29:0/1232927424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4eec160800 con 0x7f4eec15bd60 2024-04-25T00:38:13.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.187+0000 7f4ed3fff640 1 -- 172.21.15.29:0/1232927424 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16713 (secure 0 0 0) 0x7f4ee0016090 con 0x7f4eec15bd60 2024-04-25T00:38:13.191 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:13.191 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":105,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:11.560718+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:37:22.602795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:37:23.310674+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:13.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.191+0000 7f4ef200c640 1 -- 172.21.15.29:0/1232927424 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4ed8067cb0 msgr2=0x7f4ed806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.191+0000 7f4ef200c640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4ed8067cb0 0x7f4ed806a170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4edc000f70 tx=0x7f4edc009040 comp rx=0 tx=0).stop 2024-04-25T00:38:13.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.191+0000 7f4ef200c640 1 -- 172.21.15.29:0/1232927424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eec15bd60 msgr2=0x7f4eec156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.191+0000 7f4ef200c640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eec15bd60 0x7f4eec156b40 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00ac040 tx=0x7f4ee001acb0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.191+0000 7f4ef200c640 1 -- 172.21.15.29:0/1232927424 shutdown_connections 2024-04-25T00:38:13.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.191+0000 7f4ef200c640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4ed8067cb0 0x7f4ed806a170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.191+0000 7f4ef200c640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eec157080 0x7f4eec0c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.191+0000 7f4ef200c640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eec15bd60 0x7f4eec156b40 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.191+0000 7f4ef200c640 1 --2- 172.21.15.29:0/1232927424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eec0cb330 0x7f4eec1584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.191+0000 7f4ef200c640 1 -- 172.21.15.29:0/1232927424 >> 172.21.15.29:0/1232927424 conn(0x7f4eec06c320 msgr2=0x7f4eec06b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:13.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.191+0000 7f4ef200c640 1 -- 172.21.15.29:0/1232927424 shutdown_connections 2024-04-25T00:38:13.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.191+0000 7f4ef200c640 1 -- 172.21.15.29:0/1232927424 wait complete. 2024-04-25T00:38:13.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:37:22.602795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:37:23.310674+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:38:13.211 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:38:13.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f8820e27640 1 Processor -- start 2024-04-25T00:38:13.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f8820e27640 1 -- start start 2024-04-25T00:38:13.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f8820e27640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c15e8d0 0x7f881c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f8820e27640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f881c0b3720 con 0x7f881c15f3c0 2024-04-25T00:38:13.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f8820e27640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f881c0b39c0 con 0x7f881c0b4780 2024-04-25T00:38:13.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f8820e27640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f881c0b42f0 con 0x7f881c15e8d0 2024-04-25T00:38:13.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881ad9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f881c15f3c0 0x7f881c15c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54396/0 (socket says 172.21.15.29:54396) 2024-04-25T00:38:13.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881ad9d640 1 -- 172.21.15.29:0/1117380493 learned_addr learned my addr 172.21.15.29:0/1117380493 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:13.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881a59c640 1 --2- 172.21.15.29:0/1117380493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c15e8d0 0x7f881c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955908317 0 0) 0x7f881c0b3720 con 0x7f881c15f3c0 2024-04-25T00:38:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8804003530 con 0x7f881c15f3c0 2024-04-25T00:38:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1525145484 0 0) 0x7f881c0b39c0 con 0x7f881c0b4780 2024-04-25T00:38:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f881c0b3720 con 0x7f881c0b4780 2024-04-25T00:38:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 686909154 0 0) 0x7f8804003530 con 0x7f881c15f3c0 2024-04-25T00:38:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c15e8d0 msgr2=0x7f881c0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 --2- 172.21.15.29:0/1117380493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c15e8d0 0x7f881c0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 >> v1:172.21.15.157:6789/0 conn(0x7f881c0b4780 legacy=0x7f881c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c0b3ca0 con 0x7f881c15f3c0 2024-04-25T00:38:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f880c001680 con 0x7f881c15f3c0 2024-04-25T00:38:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881a59c640 1 --2- 172.21.15.29:0/1117380493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c15e8d0 0x7f881c0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:38:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 >> v1:172.21.15.29:6789/0 conn(0x7f881c15f3c0 legacy=0x7f881c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 --2- 172.21.15.29:0/1117380493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8804003b80 0x7f8804003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 --2- 172.21.15.29:0/1117380493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c15e8d0 0x7f8804004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 --2- 172.21.15.29:0/1117380493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804004d50 0x7f88040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f881c0b42f0 con 0x7f8804003b80 2024-04-25T00:38:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f880400ba80 con 0x7f8804004d50 2024-04-25T00:38:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f880400bc00 con 0x7f881c15e8d0 2024-04-25T00:38:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881a59c640 1 --2- 172.21.15.29:0/1117380493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8804003b80 0x7f8804003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f8819d9b640 1 --2- 172.21.15.29:0/1117380493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c15e8d0 0x7f8804004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881a59c640 1 -- 172.21.15.29:0/1117380493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c15e8d0 msgr2=0x7f8804004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881a59c640 1 --2- 172.21.15.29:0/1117380493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c15e8d0 0x7f8804004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881a59c640 1 -- 172.21.15.29:0/1117380493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804004d50 msgr2=0x7f88040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.331+0000 7f881ad9d640 1 --2- 172.21.15.29:0/1117380493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804004d50 0x7f88040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.335+0000 7f881a59c640 1 --2- 172.21.15.29:0/1117380493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804004d50 0x7f88040051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.335+0000 7f881a59c640 1 -- 172.21.15.29:0/1117380493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8804003530 con 0x7f8804003b80 2024-04-25T00:38:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.335+0000 7f881a59c640 1 --2- 172.21.15.29:0/1117380493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8804003b80 0x7f8804003f80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f881400e380 tx=0x7f881401cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe425ac873a6ae66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.335+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88140b2070 con 0x7f8804003b80 2024-04-25T00:38:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.335+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88140a5cb0 con 0x7f8804003b80 2024-04-25T00:38:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.335+0000 7f881959a640 1 -- 172.21.15.29:0/1117380493 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814009de0 con 0x7f8804003b80 2024-04-25T00:38:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.335+0000 7f8820e27640 1 -- 172.21.15.29:0/1117380493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8804003b80 msgr2=0x7f8804003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.335+0000 7f8820e27640 1 --2- 172.21.15.29:0/1117380493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8804003b80 0x7f8804003f80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f881400e380 tx=0x7f881401cdb0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.335+0000 7f881ad9d640 1 -- 172.21.15.29:0/1117380493 reap_dead start 2024-04-25T00:38:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.335+0000 7f8820e27640 1 -- 172.21.15.29:0/1117380493 shutdown_connections 2024-04-25T00:38:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.335+0000 7f8820e27640 1 -- 172.21.15.29:0/1117380493 >> 172.21.15.29:0/1117380493 conn(0x7f881c0b1870 msgr2=0x7f881c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.335+0000 7f8820e27640 1 -- 172.21.15.29:0/1117380493 shutdown_connections 2024-04-25T00:38:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.335+0000 7f8820e27640 1 -- 172.21.15.29:0/1117380493 wait complete. 2024-04-25T00:38:13.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8820e27640 1 Processor -- start 2024-04-25T00:38:13.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8820e27640 1 -- start start 2024-04-25T00:38:13.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8820e27640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804003b80 0x7f881c13c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8820e27640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f881c13f920 0x7f881c13fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8820e27640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c162900 0x7f881c162d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8820e27640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f881c166df0 con 0x7f881c13f920 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8820e27640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f881c166c70 con 0x7f8804003b80 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8820e27640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f881c166f70 con 0x7f881c162900 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8819d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f881c13f920 0x7f881c13fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8819d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f881c13f920 0x7f881c13fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55350/0 (socket says 172.21.15.29:55350) 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8819d9b640 1 -- 172.21.15.29:0/345517464 learned_addr learned my addr 172.21.15.29:0/345517464 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f881ad9d640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c162900 0x7f881c162d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f881a59c640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804003b80 0x7f881c13c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8819d9b640 1 -- 172.21.15.29:0/345517464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c162900 msgr2=0x7f881c162d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8819d9b640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c162900 0x7f881c162d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8819d9b640 1 -- 172.21.15.29:0/345517464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804003b80 msgr2=0x7f881c13c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8819d9b640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804003b80 0x7f881c13c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8819d9b640 1 -- 172.21.15.29:0/345517464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c164ac0 con 0x7f881c13f920 2024-04-25T00:38:13.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8819d9b640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f881c13f920 0x7f881c13fda0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f881000e7b0 tx=0x7f881000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:13.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f880affd640 1 -- 172.21.15.29:0/345517464 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881000ee60 con 0x7f881c13f920 2024-04-25T00:38:13.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f880affd640 1 -- 172.21.15.29:0/345517464 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8810018070 con 0x7f881c13f920 2024-04-25T00:38:13.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f880affd640 1 -- 172.21.15.29:0/345517464 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881000c660 con 0x7f881c13f920 2024-04-25T00:38:13.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8820e27640 1 -- 172.21.15.29:0/345517464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881c164da0 con 0x7f881c13f920 2024-04-25T00:38:13.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8820e27640 1 -- 172.21.15.29:0/345517464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881c165360 con 0x7f881c13f920 2024-04-25T00:38:13.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.339+0000 7f8820e27640 1 -- 172.21.15.29:0/345517464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881c069270 con 0x7f881c13f920 2024-04-25T00:38:13.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.343+0000 7f880affd640 1 -- 172.21.15.29:0/345517464 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f881000c800 con 0x7f881c13f920 2024-04-25T00:38:13.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.343+0000 7f880affd640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f87fc067cb0 0x7f87fc06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.343+0000 7f880affd640 1 -- 172.21.15.29:0/345517464 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f8810086240 con 0x7f881c13f920 2024-04-25T00:38:13.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.343+0000 7f881a59c640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f87fc067cb0 0x7f87fc06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.343+0000 7f881a59c640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f87fc067cb0 0x7f87fc06a170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f88140af000 tx=0x7f8814004ac0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:13.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.347+0000 7f880affd640 1 -- 172.21.15.29:0/345517464 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8810056750 con 0x7f881c13f920 2024-04-25T00:38:13.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f8820e27640 1 -- 172.21.15.29:0/345517464 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f881c06e7b0 con 0x7f881c13f920 2024-04-25T00:38:13.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f880affd640 1 -- 172.21.15.29:0/345517464 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f8810056050 con 0x7f881c13f920 2024-04-25T00:38:13.505 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:38:13.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f8820e27640 1 -- 172.21.15.29:0/345517464 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f87fc067cb0 msgr2=0x7f87fc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f8820e27640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f87fc067cb0 0x7f87fc06a170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f88140af000 tx=0x7f8814004ac0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f8820e27640 1 -- 172.21.15.29:0/345517464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f881c13f920 msgr2=0x7f881c13fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f8820e27640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f881c13f920 0x7f881c13fda0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f881000e7b0 tx=0x7f881000ec70 comp rx=0 tx=0).stop 2024-04-25T00:38:13.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f8820e27640 1 -- 172.21.15.29:0/345517464 shutdown_connections 2024-04-25T00:38:13.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f8820e27640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f87fc067cb0 0x7f87fc06a170 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f8820e27640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f881c162900 0x7f881c162d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f8820e27640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f881c13f920 0x7f881c13fda0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f8820e27640 1 --2- 172.21.15.29:0/345517464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804003b80 0x7f881c13c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f8820e27640 1 -- 172.21.15.29:0/345517464 >> 172.21.15.29:0/345517464 conn(0x7f881c0b1870 msgr2=0x7f881c15c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:13.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f8820e27640 1 -- 172.21.15.29:0/345517464 shutdown_connections 2024-04-25T00:38:13.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.503+0000 7f8820e27640 1 -- 172.21.15.29:0/345517464 wait complete. 2024-04-25T00:38:13.523 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:38:13.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.639+0000 7fd1b5912640 1 Processor -- start 2024-04-25T00:38:13.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.639+0000 7fd1b5912640 1 -- start start 2024-04-25T00:38:13.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.639+0000 7fd1b5912640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b00d4d10 0x7fd1b00d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.639+0000 7fd1b5912640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd1b00b4040 0x7fd1b00b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.639+0000 7fd1b5912640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1b00cf8a0 con 0x7fd1b015e2a0 2024-04-25T00:38:13.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.639+0000 7fd1b5912640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd1b00d19a0 con 0x7fd1b00b4040 2024-04-25T00:38:13.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.639+0000 7fd1b5912640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b00cf650 con 0x7fd1b00d4d10 2024-04-25T00:38:13.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1af7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd1b00b4040 0x7fd1b00b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1af7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd1b00b4040 0x7fd1b00b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55362/0 (socket says 172.21.15.29:55362) 2024-04-25T00:38:13.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1af7fe640 1 -- 172.21.15.29:0/2581889018 learned_addr learned my addr 172.21.15.29:0/2581889018 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:13.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1ae7fc640 1 --2- 172.21.15.29:0/2581889018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b00d4d10 0x7fd1b00d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1af7fe640 1 -- 172.21.15.29:0/2581889018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b00d4d10 msgr2=0x7fd1b00d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:13.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1af7fe640 1 --2- 172.21.15.29:0/2581889018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b00d4d10 0x7fd1b00d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1af7fe640 1 -- 172.21.15.29:0/2581889018 >> v1:172.21.15.29:6789/0 conn(0x7fd1b015e2a0 legacy=0x7fd1b00d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1af7fe640 1 -- 172.21.15.29:0/2581889018 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b00cfb80 con 0x7fd1b00b4040 2024-04-25T00:38:13.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1af7fe640 1 --2- 172.21.15.29:0/2581889018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1b00b4040 0x7fd1b00b44f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009e50 tx=0x7fd1a801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f843008c100fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:13.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1adffb640 1 -- 172.21.15.29:0/2581889018 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80a5660 con 0x7fd1b00b4040 2024-04-25T00:38:13.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1adffb640 1 -- 172.21.15.29:0/2581889018 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a80a5800 con 0x7fd1b00b4040 2024-04-25T00:38:13.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1adffb640 1 -- 172.21.15.29:0/2581889018 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80a5bd0 con 0x7fd1b00b4040 2024-04-25T00:38:13.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1b5912640 1 -- 172.21.15.29:0/2581889018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1b00b4040 msgr2=0x7fd1b00b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1b5912640 1 --2- 172.21.15.29:0/2581889018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1b00b4040 0x7fd1b00b44f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009e50 tx=0x7fd1a801c9a0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1b5912640 1 -- 172.21.15.29:0/2581889018 shutdown_connections 2024-04-25T00:38:13.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1b5912640 1 --2- 172.21.15.29:0/2581889018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1b00b4040 0x7fd1b00b44f0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1b5912640 1 --2- 172.21.15.29:0/2581889018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b00d4d10 0x7fd1b00d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1b5912640 1 -- 172.21.15.29:0/2581889018 >> 172.21.15.29:0/2581889018 conn(0x7fd1b00d4030 msgr2=0x7fd1b00d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:13.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1b5912640 1 -- 172.21.15.29:0/2581889018 shutdown_connections 2024-04-25T00:38:13.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.643+0000 7fd1b5912640 1 -- 172.21.15.29:0/2581889018 wait complete. 2024-04-25T00:38:13.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1b5912640 1 Processor -- start 2024-04-25T00:38:13.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1b5912640 1 -- start start 2024-04-25T00:38:13.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1b5912640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b00b4040 0x7fd1b01549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1b5912640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1b00d4d10 0x7fd1b01570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1b5912640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1b015e2a0 0x7fd1b01575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1b5912640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b016aef0 con 0x7fd1b00d4d10 2024-04-25T00:38:13.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1b5912640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b00d2a60 con 0x7fd1b015e2a0 2024-04-25T00:38:13.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1b5912640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b016b070 con 0x7fd1b00b4040 2024-04-25T00:38:13.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1ae7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1b00d4d10 0x7fd1b01570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1ae7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1b00d4d10 0x7fd1b01570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55366/0 (socket says 172.21.15.29:55366) 2024-04-25T00:38:13.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1ae7fc640 1 -- 172.21.15.29:0/4259808945 learned_addr learned my addr 172.21.15.29:0/4259808945 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:13.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1af7fe640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1b015e2a0 0x7fd1b01575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1aeffd640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b00b4040 0x7fd1b01549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1ae7fc640 1 -- 172.21.15.29:0/4259808945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b00b4040 msgr2=0x7fd1b01549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1ae7fc640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b00b4040 0x7fd1b01549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1ae7fc640 1 -- 172.21.15.29:0/4259808945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1b015e2a0 msgr2=0x7fd1b01575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1ae7fc640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1b015e2a0 0x7fd1b01575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1ae7fc640 1 -- 172.21.15.29:0/4259808945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b015e9c0 con 0x7fd1b00d4d10 2024-04-25T00:38:13.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1ae7fc640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1b00d4d10 0x7fd1b01570b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4006e90 tx=0x7fd1a400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:13.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd197fff640 1 -- 172.21.15.29:0/4259808945 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400b6c0 con 0x7fd1b00d4d10 2024-04-25T00:38:13.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd197fff640 1 -- 172.21.15.29:0/4259808945 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a4019070 con 0x7fd1b00d4d10 2024-04-25T00:38:13.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1b5912640 1 -- 172.21.15.29:0/4259808945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b015eca0 con 0x7fd1b00d4d10 2024-04-25T00:38:13.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd197fff640 1 -- 172.21.15.29:0/4259808945 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400bac0 con 0x7fd1b00d4d10 2024-04-25T00:38:13.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1b5912640 1 -- 172.21.15.29:0/4259808945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b0160cc0 con 0x7fd1b00d4d10 2024-04-25T00:38:13.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd1b5912640 1 -- 172.21.15.29:0/4259808945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b0069270 con 0x7fd1b00d4d10 2024-04-25T00:38:13.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd197fff640 1 -- 172.21.15.29:0/4259808945 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fd1a400bc60 con 0x7fd1b00d4d10 2024-04-25T00:38:13.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.647+0000 7fd197fff640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd19c067cb0 0x7fd19c06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.651+0000 7fd1aeffd640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd19c067cb0 0x7fd19c06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.651+0000 7fd197fff640 1 -- 172.21.15.29:0/4259808945 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7fd1a4086760 con 0x7fd1b00d4d10 2024-04-25T00:38:13.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.651+0000 7fd1aeffd640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd19c067cb0 0x7fd19c06a170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0007fd0 tx=0x7fd1a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:13.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.655+0000 7fd197fff640 1 -- 172.21.15.29:0/4259808945 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd1a4056d80 con 0x7fd1b00d4d10 2024-04-25T00:38:13.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.815+0000 7fd1b5912640 1 -- 172.21.15.29:0/4259808945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd1b00d3e10 con 0x7fd1b00d4d10 2024-04-25T00:38:13.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.815+0000 7fd197fff640 1 -- 172.21.15.29:0/4259808945 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7fd1a4056680 con 0x7fd1b00d4d10 2024-04-25T00:38:13.819 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:38:13.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.819+0000 7fd1b5912640 1 -- 172.21.15.29:0/4259808945 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd19c067cb0 msgr2=0x7fd19c06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.819+0000 7fd1b5912640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd19c067cb0 0x7fd19c06a170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0007fd0 tx=0x7fd1a0009040 comp rx=0 tx=0).stop 2024-04-25T00:38:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.819+0000 7fd1b5912640 1 -- 172.21.15.29:0/4259808945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1b00d4d10 msgr2=0x7fd1b01570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.819+0000 7fd1b5912640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1b00d4d10 0x7fd1b01570b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4006e90 tx=0x7fd1a400dc60 comp rx=0 tx=0).stop 2024-04-25T00:38:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.819+0000 7fd1b5912640 1 -- 172.21.15.29:0/4259808945 shutdown_connections 2024-04-25T00:38:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.819+0000 7fd1b5912640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd19c067cb0 0x7fd19c06a170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.819+0000 7fd1b5912640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1b015e2a0 0x7fd1b01575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.819+0000 7fd1b5912640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1b00d4d10 0x7fd1b01570b0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.819+0000 7fd1b5912640 1 --2- 172.21.15.29:0/4259808945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b00b4040 0x7fd1b01549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.819+0000 7fd1b5912640 1 -- 172.21.15.29:0/4259808945 >> 172.21.15.29:0/4259808945 conn(0x7fd1b00d4030 msgr2=0x7fd1b006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.819+0000 7fd1b5912640 1 -- 172.21.15.29:0/4259808945 shutdown_connections 2024-04-25T00:38:13.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.819+0000 7fd1b5912640 1 -- 172.21.15.29:0/4259808945 wait complete. 2024-04-25T00:38:13.839 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:38:13.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e3c01640 1 Processor -- start 2024-04-25T00:38:13.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e3c01640 1 -- start start 2024-04-25T00:38:13.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e3c01640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9dc0b3140 0x7fd9dc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e3c01640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9dc0b3c30 0x7fd9dc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e3c01640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc0b41e0 con 0x7fd9dc0b4b10 2024-04-25T00:38:13.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e3c01640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd9dc06d960 con 0x7fd9dc0b3c30 2024-04-25T00:38:13.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e3c01640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9dc06dc40 con 0x7fd9dc0b3140 2024-04-25T00:38:13.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e119c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd9dc0b4b10 0x7fd9dc0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54416/0 (socket says 172.21.15.29:54416) 2024-04-25T00:38:13.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e119c640 1 -- 172.21.15.29:0/2308483601 learned_addr learned my addr 172.21.15.29:0/2308483601 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:13.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e219e640 1 --2- 172.21.15.29:0/2308483601 >> v2:172.21.15.157:3300/0 conn(0x7fd9dc0b3c30 0x7fd9dc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e199d640 1 --2- 172.21.15.29:0/2308483601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9dc0b3140 0x7fd9dc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3579874285 0 0) 0x7fd9dc0b41e0 con 0x7fd9dc0b4b10 2024-04-25T00:38:13.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9c8003530 con 0x7fd9dc0b4b10 2024-04-25T00:38:13.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2655607003 0 0) 0x7fd9c8003530 con 0x7fd9dc0b4b10 2024-04-25T00:38:13.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9dc0b3140 msgr2=0x7fd9dc06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:13.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 --2- 172.21.15.29:0/2308483601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9dc0b3140 0x7fd9dc06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 >> v2:172.21.15.157:3300/0 conn(0x7fd9dc0b3c30 msgr2=0x7fd9dc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 --2- 172.21.15.29:0/2308483601 >> v2:172.21.15.157:3300/0 conn(0x7fd9dc0b3c30 0x7fd9dc0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc06df20 con 0x7fd9dc0b4b10 2024-04-25T00:38:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd9cc001680 con 0x7fd9dc0b4b10 2024-04-25T00:38:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 >> v1:172.21.15.29:6789/0 conn(0x7fd9dc0b4b10 legacy=0x7fd9dc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 --2- 172.21.15.29:0/2308483601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9c80039c0 0x7fd9c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 --2- 172.21.15.29:0/2308483601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9dc0b3140 0x7fd9c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 --2- 172.21.15.29:0/2308483601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c8004d50 0x7fd9c80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc06d960 con 0x7fd9c80039c0 2024-04-25T00:38:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc06dc40 con 0x7fd9c8004d50 2024-04-25T00:38:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9c800b9d0 con 0x7fd9dc0b3140 2024-04-25T00:38:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e199d640 1 --2- 172.21.15.29:0/2308483601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9c80039c0 0x7fd9c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e219e640 1 --2- 172.21.15.29:0/2308483601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c8004d50 0x7fd9c80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e199d640 1 -- 172.21.15.29:0/2308483601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9dc0b3140 msgr2=0x7fd9c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e199d640 1 --2- 172.21.15.29:0/2308483601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9dc0b3140 0x7fd9c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e199d640 1 -- 172.21.15.29:0/2308483601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c8004d50 msgr2=0x7fd9c80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e199d640 1 --2- 172.21.15.29:0/2308483601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c8004d50 0x7fd9c80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.963+0000 7fd9e199d640 1 -- 172.21.15.29:0/2308483601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c8003530 con 0x7fd9c80039c0 2024-04-25T00:38:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.967+0000 7fd9e199d640 1 --2- 172.21.15.29:0/2308483601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9c80039c0 0x7fd9c8003de0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009ac0 tx=0x7fd9d801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be85b2967d0b739f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.967+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8009690 con 0x7fd9c80039c0 2024-04-25T00:38:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.967+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d80a6470 con 0x7fd9c80039c0 2024-04-25T00:38:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.967+0000 7fd9e099b640 1 -- 172.21.15.29:0/2308483601 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80a6840 con 0x7fd9c80039c0 2024-04-25T00:38:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.967+0000 7fd9e3c01640 1 -- 172.21.15.29:0/2308483601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9c80039c0 msgr2=0x7fd9c8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.967+0000 7fd9e3c01640 1 --2- 172.21.15.29:0/2308483601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9c80039c0 0x7fd9c8003de0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009ac0 tx=0x7fd9d801ca90 comp rx=0 tx=0).stop 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.967+0000 7fd9e219e640 1 -- 172.21.15.29:0/2308483601 reap_dead start 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.967+0000 7fd9e3c01640 1 -- 172.21.15.29:0/2308483601 shutdown_connections 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.967+0000 7fd9e3c01640 1 -- 172.21.15.29:0/2308483601 >> 172.21.15.29:0/2308483601 conn(0x7fd9dc0b1870 msgr2=0x7fd9dc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.967+0000 7fd9e3c01640 1 -- 172.21.15.29:0/2308483601 shutdown_connections 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.967+0000 7fd9e3c01640 1 -- 172.21.15.29:0/2308483601 wait complete. 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.967+0000 7fd9e3c01640 1 Processor -- start 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.967+0000 7fd9e3c01640 1 -- start start 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e3c01640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9c80039c0 0x7fd9dc15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e3c01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9dc132fd0 0x7fd9dc0ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e3c01640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9dc133450 0x7fd9dc165940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e3c01640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc166d70 con 0x7fd9dc133450 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e3c01640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc166bf0 con 0x7fd9dc132fd0 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e3c01640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9dc166ef0 con 0x7fd9c80039c0 2024-04-25T00:38:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e219e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9dc133450 0x7fd9dc165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e219e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9dc133450 0x7fd9dc165940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55386/0 (socket says 172.21.15.29:55386) 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e219e640 1 -- 172.21.15.29:0/1678206288 learned_addr learned my addr 172.21.15.29:0/1678206288 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e119c640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9dc132fd0 0x7fd9dc0ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e199d640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9c80039c0 0x7fd9dc15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e219e640 1 -- 172.21.15.29:0/1678206288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9c80039c0 msgr2=0x7fd9dc15b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e219e640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9c80039c0 0x7fd9dc15b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e219e640 1 -- 172.21.15.29:0/1678206288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9dc132fd0 msgr2=0x7fd9dc0ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e219e640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9dc132fd0 0x7fd9dc0ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e219e640 1 -- 172.21.15.29:0/1678206288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc1648a0 con 0x7fd9dc133450 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e219e640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9dc133450 0x7fd9dc165940 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc074fe0 tx=0x7fd9d400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9d27fc640 1 -- 172.21.15.29:0/1678206288 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4016620 con 0x7fd9dc133450 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9d27fc640 1 -- 172.21.15.29:0/1678206288 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d401b070 con 0x7fd9dc133450 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e3c01640 1 -- 172.21.15.29:0/1678206288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9dc164b80 con 0x7fd9dc133450 2024-04-25T00:38:13.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9d27fc640 1 -- 172.21.15.29:0/1678206288 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4016a20 con 0x7fd9dc133450 2024-04-25T00:38:13.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e3c01640 1 -- 172.21.15.29:0/1678206288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9dc165140 con 0x7fd9dc133450 2024-04-25T00:38:13.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e3c01640 1 -- 172.21.15.29:0/1678206288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9dc069270 con 0x7fd9dc133450 2024-04-25T00:38:13.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9d27fc640 1 -- 172.21.15.29:0/1678206288 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fd9d4016bc0 con 0x7fd9dc133450 2024-04-25T00:38:13.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9d27fc640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9c0067ce0 0x7fd9c006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:13.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9d27fc640 1 -- 172.21.15.29:0/1678206288 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7fd9d4087260 con 0x7fd9dc133450 2024-04-25T00:38:13.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e199d640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9c0067ce0 0x7fd9c006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:13.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.971+0000 7fd9e199d640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9c0067ce0 0x7fd9c006a1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80ad000 tx=0x7fd9d80b2040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:13.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:13.975+0000 7fd9d27fc640 1 -- 172.21.15.29:0/1678206288 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd9d4025020 con 0x7fd9dc133450 2024-04-25T00:38:14.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.139+0000 7fd9e3c01640 1 -- 172.21.15.29:0/1678206288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd9dc143960 con 0x7fd9dc133450 2024-04-25T00:38:14.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.139+0000 7fd9d27fc640 1 -- 172.21.15.29:0/1678206288 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7fd9d401d310 con 0x7fd9dc133450 2024-04-25T00:38:14.142 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:38:14.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.143+0000 7fd9e3c01640 1 -- 172.21.15.29:0/1678206288 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9c0067ce0 msgr2=0x7fd9c006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:14.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.143+0000 7fd9e3c01640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9c0067ce0 0x7fd9c006a1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80ad000 tx=0x7fd9d80b2040 comp rx=0 tx=0).stop 2024-04-25T00:38:14.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.143+0000 7fd9e3c01640 1 -- 172.21.15.29:0/1678206288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9dc133450 msgr2=0x7fd9dc165940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:14.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.143+0000 7fd9e3c01640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9dc133450 0x7fd9dc165940 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc074fe0 tx=0x7fd9d400f520 comp rx=0 tx=0).stop 2024-04-25T00:38:14.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.143+0000 7fd9e3c01640 1 -- 172.21.15.29:0/1678206288 shutdown_connections 2024-04-25T00:38:14.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.143+0000 7fd9e3c01640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9c0067ce0 0x7fd9c006a1a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.143+0000 7fd9e3c01640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9dc133450 0x7fd9dc165940 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.143+0000 7fd9e3c01640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9dc132fd0 0x7fd9dc0ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.143+0000 7fd9e3c01640 1 --2- 172.21.15.29:0/1678206288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9c80039c0 0x7fd9dc15b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.143+0000 7fd9e3c01640 1 -- 172.21.15.29:0/1678206288 >> 172.21.15.29:0/1678206288 conn(0x7fd9dc0b1870 msgr2=0x7fd9dc15f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:14.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.143+0000 7fd9e3c01640 1 -- 172.21.15.29:0/1678206288 shutdown_connections 2024-04-25T00:38:14.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.143+0000 7fd9e3c01640 1 -- 172.21.15.29:0/1678206288 wait complete. 2024-04-25T00:38:14.161 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:38:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f855d3b0640 1 Processor -- start 2024-04-25T00:38:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f855d3b0640 1 -- start start 2024-04-25T00:38:14.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f855d3b0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f85580cf690 0x7f85580d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f855d3b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85580d2a90 0x7f85580b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f855d3b0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85580cfc10 con 0x7f855815e200 2024-04-25T00:38:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f855d3b0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f85580d2ee0 con 0x7f85580cf690 2024-04-25T00:38:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f855d3b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f85580b38d0 con 0x7f85580d2a90 2024-04-25T00:38:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f85567fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f85580cf690 0x7f85580d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f85567fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f85580cf690 0x7f85580d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55402/0 (socket says 172.21.15.29:55402) 2024-04-25T00:38:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f85567fc640 1 -- 172.21.15.29:0/432562868 learned_addr learned my addr 172.21.15.29:0/432562868 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f85577fe640 1 --2- 172.21.15.29:0/432562868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85580d2a90 0x7f85580b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f85567fc640 1 -- 172.21.15.29:0/432562868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85580d2a90 msgr2=0x7f85580b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:14.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f85567fc640 1 --2- 172.21.15.29:0/432562868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85580d2a90 0x7f85580b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f85567fc640 1 -- 172.21.15.29:0/432562868 >> v1:172.21.15.157:6789/0 conn(0x7f855815e200 legacy=0x7f85580d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:14.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.287+0000 7f85567fc640 1 -- 172.21.15.29:0/432562868 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85580cfef0 con 0x7f85580cf690 2024-04-25T00:38:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f85567fc640 1 --2- 172.21.15.29:0/432562868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85580cf690 0x7f85580d2550 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f854c009870 tx=0x7f854c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c04b1c9c06a83e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f8555ffb640 1 -- 172.21.15.29:0/432562868 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c0b1070 con 0x7f85580cf690 2024-04-25T00:38:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f8555ffb640 1 -- 172.21.15.29:0/432562868 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f854c0ac630 con 0x7f85580cf690 2024-04-25T00:38:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f8555ffb640 1 -- 172.21.15.29:0/432562868 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c0a8050 con 0x7f85580cf690 2024-04-25T00:38:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f855d3b0640 1 -- 172.21.15.29:0/432562868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85580cf690 msgr2=0x7f85580d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f855d3b0640 1 --2- 172.21.15.29:0/432562868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85580cf690 0x7f85580d2550 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f854c009870 tx=0x7f854c01c520 comp rx=0 tx=0).stop 2024-04-25T00:38:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f855d3b0640 1 -- 172.21.15.29:0/432562868 shutdown_connections 2024-04-25T00:38:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f855d3b0640 1 --2- 172.21.15.29:0/432562868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85580d2a90 0x7f85580b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f855d3b0640 1 --2- 172.21.15.29:0/432562868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85580cf690 0x7f85580d2550 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f855d3b0640 1 -- 172.21.15.29:0/432562868 >> 172.21.15.29:0/432562868 conn(0x7f855809d280 msgr2=0x7f85580d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:14.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f855d3b0640 1 -- 172.21.15.29:0/432562868 shutdown_connections 2024-04-25T00:38:14.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f855d3b0640 1 -- 172.21.15.29:0/432562868 wait complete. 2024-04-25T00:38:14.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f855d3b0640 1 Processor -- start 2024-04-25T00:38:14.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.291+0000 7f855d3b0640 1 -- start start 2024-04-25T00:38:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f855d3b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85580cf690 0x7f85581573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f855d3b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85580d2a90 0x7f8558155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f855d3b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855815e200 0x7f8558160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f855d3b0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f85580b51f0 con 0x7f855815e200 2024-04-25T00:38:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f855d3b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85580b5070 con 0x7f85580cf690 2024-04-25T00:38:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f855d3b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f85580b5370 con 0x7f85580d2a90 2024-04-25T00:38:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f85577fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855815e200 0x7f8558160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:14.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f85577fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855815e200 0x7f8558160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55412/0 (socket says 172.21.15.29:55412) 2024-04-25T00:38:14.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f85577fe640 1 -- 172.21.15.29:0/954126406 learned_addr learned my addr 172.21.15.29:0/954126406 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:14.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f85567fc640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85580d2a90 0x7f8558155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:14.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f8556ffd640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85580cf690 0x7f85581573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:14.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f85577fe640 1 -- 172.21.15.29:0/954126406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85580d2a90 msgr2=0x7f8558155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:14.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f85577fe640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85580d2a90 0x7f8558155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f85577fe640 1 -- 172.21.15.29:0/954126406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85580cf690 msgr2=0x7f85581573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:14.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f85577fe640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85580cf690 0x7f85581573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f85577fe640 1 -- 172.21.15.29:0/954126406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85580c8940 con 0x7f855815e200 2024-04-25T00:38:14.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f85567fc640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85580d2a90 0x7f8558155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:14.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f85577fe640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855815e200 0x7f8558160bb0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f8550007e40 tx=0x7f855000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:14.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f853f7fe640 1 -- 172.21.15.29:0/954126406 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550014620 con 0x7f855815e200 2024-04-25T00:38:14.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f853f7fe640 1 -- 172.21.15.29:0/954126406 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8550019070 con 0x7f855815e200 2024-04-25T00:38:14.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f855d3b0640 1 -- 172.21.15.29:0/954126406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85581565f0 con 0x7f855815e200 2024-04-25T00:38:14.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f853f7fe640 1 -- 172.21.15.29:0/954126406 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550014a20 con 0x7f855815e200 2024-04-25T00:38:14.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f855d3b0640 1 -- 172.21.15.29:0/954126406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8558156bb0 con 0x7f855815e200 2024-04-25T00:38:14.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f855d3b0640 1 -- 172.21.15.29:0/954126406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8558069270 con 0x7f855815e200 2024-04-25T00:38:14.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f853f7fe640 1 -- 172.21.15.29:0/954126406 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f8550014bc0 con 0x7f855815e200 2024-04-25T00:38:14.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f853f7fe640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8544067cb0 0x7f854406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f853f7fe640 1 -- 172.21.15.29:0/954126406 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f8550086d30 con 0x7f855815e200 2024-04-25T00:38:14.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f8556ffd640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8544067cb0 0x7f854406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:14.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.295+0000 7f8556ffd640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8544067cb0 0x7f854406a170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8558157c40 tx=0x7f8548008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:14.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.299+0000 7f853f7fe640 1 -- 172.21.15.29:0/954126406 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8550057240 con 0x7f855815e200 2024-04-25T00:38:14.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.471+0000 7f855d3b0640 1 -- 172.21.15.29:0/954126406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f855806c570 con 0x7f855815e200 2024-04-25T00:38:14.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.471+0000 7f853f7fe640 1 -- 172.21.15.29:0/954126406 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7f8550056b40 con 0x7f855815e200 2024-04-25T00:38:14.475 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:38:14.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.471+0000 7f855d3b0640 1 -- 172.21.15.29:0/954126406 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8544067cb0 msgr2=0x7f854406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:14.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.475+0000 7f855d3b0640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8544067cb0 0x7f854406a170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8558157c40 tx=0x7f8548008040 comp rx=0 tx=0).stop 2024-04-25T00:38:14.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.475+0000 7f855d3b0640 1 -- 172.21.15.29:0/954126406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855815e200 msgr2=0x7f8558160bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:14.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.475+0000 7f855d3b0640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855815e200 0x7f8558160bb0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f8550007e40 tx=0x7f855000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.475+0000 7f855d3b0640 1 -- 172.21.15.29:0/954126406 shutdown_connections 2024-04-25T00:38:14.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.475+0000 7f855d3b0640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8544067cb0 0x7f854406a170 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.475+0000 7f855d3b0640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855815e200 0x7f8558160bb0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.475+0000 7f855d3b0640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85580d2a90 0x7f8558155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.475+0000 7f855d3b0640 1 --2- 172.21.15.29:0/954126406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85580cf690 0x7f85581573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.475+0000 7f855d3b0640 1 -- 172.21.15.29:0/954126406 >> 172.21.15.29:0/954126406 conn(0x7f855809d280 msgr2=0x7f85580d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:14.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.475+0000 7f855d3b0640 1 -- 172.21.15.29:0/954126406 shutdown_connections 2024-04-25T00:38:14.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.475+0000 7f855d3b0640 1 -- 172.21.15.29:0/954126406 wait complete. 2024-04-25T00:38:14.491 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-25T00:38:14.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:14.492 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T00:38:14.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34591]: fuse finished with error 0 and tester_r 0 2024-04-25T00:38:14.507 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:38:14.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:14.507 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:38:14.659 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:38:14.660 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:38:14.660 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:38:14.666 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:38:14.666 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:38:14.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:14.666 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:38:14.712 INFO:teuthology.orchestra.run.smithi029.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T00:38:14.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:38:14.713 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:38:14.713 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:38:14.713 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:38:14.719 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:38:14.720 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:38:14.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:14.720 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:38:14.767 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T00:38:14.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:38:14.768 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:38:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f3569640 1 Processor -- start 2024-04-25T00:38:14.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f3569640 1 -- start start 2024-04-25T00:38:14.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f3569640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec0d0670 0x7f01ec166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f3569640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ec0cdbd0 con 0x7f01ec0cfa10 2024-04-25T00:38:14.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f3569640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ec0b13e0 con 0x7f01ec15c220 2024-04-25T00:38:14.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f3569640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f01ec15c600 con 0x7f01ec0d0670 2024-04-25T00:38:14.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f0b04640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f01ec0cfa10 0x7f01ec0cff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54426/0 (socket says 172.21.15.29:54426) 2024-04-25T00:38:14.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f0b04640 1 -- 172.21.15.29:0/908989602 learned_addr learned my addr 172.21.15.29:0/908989602 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:14.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f1b06640 1 --2- 172.21.15.29:0/908989602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec0d0670 0x7f01ec166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:14.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2376384439 0 0) 0x7f01ec0cdbd0 con 0x7f01ec0cfa10 2024-04-25T00:38:14.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01d4003530 con 0x7f01ec0cfa10 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 414250706 0 0) 0x7f01d4003530 con 0x7f01ec0cfa10 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec0d0670 msgr2=0x7f01ec166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 --2- 172.21.15.29:0/908989602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec0d0670 0x7f01ec166cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 >> v1:172.21.15.157:6789/0 conn(0x7f01ec15c220 legacy=0x7f01ec0cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec0b16c0 con 0x7f01ec0cfa10 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f01e0001680 con 0x7f01ec0cfa10 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 >> v1:172.21.15.29:6789/0 conn(0x7f01ec0cfa10 legacy=0x7f01ec0cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 --2- 172.21.15.29:0/908989602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d40039f0 0x7f01d4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 --2- 172.21.15.29:0/908989602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec0d0670 0x7f01d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 --2- 172.21.15.29:0/908989602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01d4004d50 0x7f01d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec15c600 con 0x7f01d4004d50 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01d400fec0 con 0x7f01d40039f0 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f01d40100c0 con 0x7f01ec0d0670 2024-04-25T00:38:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f1b06640 1 --2- 172.21.15.29:0/908989602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01d4004d50 0x7f01d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f1b06640 1 -- 172.21.15.29:0/908989602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec0d0670 msgr2=0x7f01d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f1b06640 1 --2- 172.21.15.29:0/908989602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec0d0670 0x7f01d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f1b06640 1 -- 172.21.15.29:0/908989602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d40039f0 msgr2=0x7f01d4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f1b06640 1 --2- 172.21.15.29:0/908989602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d40039f0 0x7f01d4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.895+0000 7f01f1b06640 1 -- 172.21.15.29:0/908989602 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d4003530 con 0x7f01d4004d50 2024-04-25T00:38:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.899+0000 7f01f1b06640 1 --2- 172.21.15.29:0/908989602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01d4004d50 0x7f01d40051d0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f01ec06bc40 tx=0x7f01e401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34361bf2c82b976 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.899+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e40b2070 con 0x7f01d4004d50 2024-04-25T00:38:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.899+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e40a5640 con 0x7f01d4004d50 2024-04-25T00:38:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.899+0000 7f01ebfff640 1 -- 172.21.15.29:0/908989602 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e40096b0 con 0x7f01d4004d50 2024-04-25T00:38:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.899+0000 7f01f3569640 1 -- 172.21.15.29:0/908989602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01d4004d50 msgr2=0x7f01d40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.899+0000 7f01f3569640 1 --2- 172.21.15.29:0/908989602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01d4004d50 0x7f01d40051d0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f01ec06bc40 tx=0x7f01e401ca50 comp rx=0 tx=0).stop 2024-04-25T00:38:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.899+0000 7f01f1b06640 1 -- 172.21.15.29:0/908989602 reap_dead start 2024-04-25T00:38:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.899+0000 7f01f3569640 1 -- 172.21.15.29:0/908989602 shutdown_connections 2024-04-25T00:38:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.899+0000 7f01f3569640 1 -- 172.21.15.29:0/908989602 >> 172.21.15.29:0/908989602 conn(0x7f01ec0d1d60 msgr2=0x7f01ec0d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.899+0000 7f01f3569640 1 -- 172.21.15.29:0/908989602 shutdown_connections 2024-04-25T00:38:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.899+0000 7f01f3569640 1 -- 172.21.15.29:0/908989602 wait complete. 2024-04-25T00:38:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.899+0000 7f01f3569640 1 Processor -- start 2024-04-25T00:38:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.899+0000 7f01f3569640 1 -- start start 2024-04-25T00:38:14.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f3569640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d4004d50 0x7f01ec09f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f3569640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01ec157b40 0x7f01ec15e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f3569640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec157fa0 0x7f01ec15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f3569640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec0b1a30 con 0x7f01ec157b40 2024-04-25T00:38:14.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f3569640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec0b3160 con 0x7f01d4004d50 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f3569640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f01ec06c320 con 0x7f01ec157fa0 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f0b04640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01ec157b40 0x7f01ec15e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f0b04640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01ec157b40 0x7f01ec15e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55428/0 (socket says 172.21.15.29:55428) 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f0b04640 1 -- 172.21.15.29:0/1197250247 learned_addr learned my addr 172.21.15.29:0/1197250247 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f1b06640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec157fa0 0x7f01ec15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f1305640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d4004d50 0x7f01ec09f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f0b04640 1 -- 172.21.15.29:0/1197250247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec157fa0 msgr2=0x7f01ec15ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f0b04640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec157fa0 0x7f01ec15ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f0b04640 1 -- 172.21.15.29:0/1197250247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d4004d50 msgr2=0x7f01ec09f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f0b04640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d4004d50 0x7f01ec09f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f0b04640 1 -- 172.21.15.29:0/1197250247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec09e5d0 con 0x7f01ec157b40 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f0b04640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01ec157b40 0x7f01ec15e7c0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f01e0000f70 tx=0x7f01e000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01e9ffb640 1 -- 172.21.15.29:0/1197250247 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0018070 con 0x7f01ec157b40 2024-04-25T00:38:14.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01e9ffb640 1 -- 172.21.15.29:0/1197250247 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e00074e0 con 0x7f01ec157b40 2024-04-25T00:38:14.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f3569640 1 -- 172.21.15.29:0/1197250247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ec09e8b0 con 0x7f01ec157b40 2024-04-25T00:38:14.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01e9ffb640 1 -- 172.21.15.29:0/1197250247 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e00049f0 con 0x7f01ec157b40 2024-04-25T00:38:14.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f3569640 1 -- 172.21.15.29:0/1197250247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ec09ee70 con 0x7f01ec157b40 2024-04-25T00:38:14.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f3569640 1 -- 172.21.15.29:0/1197250247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ec069270 con 0x7f01ec157b40 2024-04-25T00:38:14.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01e9ffb640 1 -- 172.21.15.29:0/1197250247 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f01e001b070 con 0x7f01ec157b40 2024-04-25T00:38:14.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01e9ffb640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f01d8067cb0 0x7f01d806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:14.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f1305640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f01d8067cb0 0x7f01d806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:14.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01e9ffb640 1 -- 172.21.15.29:0/1197250247 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f01e0085b10 con 0x7f01ec157b40 2024-04-25T00:38:14.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.903+0000 7f01f1305640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f01d8067cb0 0x7f01d806a170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f01dc007fd0 tx=0x7f01dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:14.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:14.907+0000 7f01e9ffb640 1 -- 172.21.15.29:0/1197250247 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f01e0055f90 con 0x7f01ec157b40 2024-04-25T00:38:15.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.103+0000 7f01f3569640 1 -- 172.21.15.29:0/1197250247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01ec15a6b0 con 0x7f01ec157b40 2024-04-25T00:38:15.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.103+0000 7f01e9ffb640 1 -- 172.21.15.29:0/1197250247 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5189 (secure 0 0 0) 0x7f01e0055890 con 0x7f01ec157b40 2024-04-25T00:38:15.108 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:15.108 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1884979435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1884979435},{"type":"v1","addr":"172.21.15.157:6833","nonce":1884979435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1394307674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1394307674},{"type":"v1","addr":"172.21.15.157:6835","nonce":1394307674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/3915943091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3915943091},{"type":"v1","addr":"172.21.15.29:6834","nonce":3915943091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:37:24.327819+0000","modified":"2024-04-25T00:37:27.319963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1444171533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1444171533},{"type":"v1","addr":"172.21.15.29:6835","nonce":1444171533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7896,"qdb_cluster":[7896]},"id":8}]} 2024-04-25T00:38:15.108 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 68 2024-04-25T00:38:15.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.107+0000 7f01f3569640 1 -- 172.21.15.29:0/1197250247 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f01d8067cb0 msgr2=0x7f01d806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.107+0000 7f01f3569640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f01d8067cb0 0x7f01d806a170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f01dc007fd0 tx=0x7f01dc009040 comp rx=0 tx=0).stop 2024-04-25T00:38:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.107+0000 7f01f3569640 1 -- 172.21.15.29:0/1197250247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01ec157b40 msgr2=0x7f01ec15e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.107+0000 7f01f3569640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01ec157b40 0x7f01ec15e7c0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f01e0000f70 tx=0x7f01e000cc20 comp rx=0 tx=0).stop 2024-04-25T00:38:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.107+0000 7f01f3569640 1 -- 172.21.15.29:0/1197250247 shutdown_connections 2024-04-25T00:38:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.107+0000 7f01f3569640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f01d8067cb0 0x7f01d806a170 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.107+0000 7f01f3569640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01ec157fa0 0x7f01ec15ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.107+0000 7f01f3569640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01ec157b40 0x7f01ec15e7c0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.107+0000 7f01f3569640 1 --2- 172.21.15.29:0/1197250247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d4004d50 0x7f01ec09f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.107+0000 7f01f3569640 1 -- 172.21.15.29:0/1197250247 >> 172.21.15.29:0/1197250247 conn(0x7f01ec0d1d60 msgr2=0x7f01ec0707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.107+0000 7f01f3569640 1 -- 172.21.15.29:0/1197250247 shutdown_connections 2024-04-25T00:38:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.107+0000 7f01f3569640 1 -- 172.21.15.29:0/1197250247 wait complete. 2024-04-25T00:38:15.125 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:15.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa83d984640 1 Processor -- start 2024-04-25T00:38:15.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa83d984640 1 -- start start 2024-04-25T00:38:15.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa83d984640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa83806b210 0x7fa83806db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa83d984640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa83806b5f0 0x7fa838071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa83d984640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8380d0410 con 0x7fa8380d18f0 2024-04-25T00:38:15.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa83d984640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa8380ce880 con 0x7fa83806b210 2024-04-25T00:38:15.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa83d984640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa8380d06b0 con 0x7fa83806b5f0 2024-04-25T00:38:15.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa836ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa83806b210 0x7fa83806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:15.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa836ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa83806b210 0x7fa83806db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55430/0 (socket says 172.21.15.29:55430) 2024-04-25T00:38:15.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa836ffd640 1 -- 172.21.15.29:0/285740162 learned_addr learned my addr 172.21.15.29:0/285740162 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:15.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa836ffd640 1 -- 172.21.15.29:0/285740162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa83806b5f0 msgr2=0x7fa838071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:15.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa836ffd640 1 --2- 172.21.15.29:0/285740162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa83806b5f0 0x7fa838071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa836ffd640 1 -- 172.21.15.29:0/285740162 >> v1:172.21.15.157:6789/0 conn(0x7fa8380d18f0 legacy=0x7fa83815f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:15.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.247+0000 7fa836ffd640 1 -- 172.21.15.29:0/285740162 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8380d0990 con 0x7fa83806b210 2024-04-25T00:38:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa836ffd640 1 --2- 172.21.15.29:0/285740162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa83806b210 0x7fa83806db00 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fa828009870 tx=0x7fa82801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fbe00fcc7747b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa835ffb640 1 -- 172.21.15.29:0/285740162 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8280b1070 con 0x7fa83806b210 2024-04-25T00:38:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa835ffb640 1 -- 172.21.15.29:0/285740162 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8280ac630 con 0x7fa83806b210 2024-04-25T00:38:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa835ffb640 1 -- 172.21.15.29:0/285740162 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8280a8050 con 0x7fa83806b210 2024-04-25T00:38:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 -- 172.21.15.29:0/285740162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa83806b210 msgr2=0x7fa83806db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 --2- 172.21.15.29:0/285740162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa83806b210 0x7fa83806db00 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fa828009870 tx=0x7fa82801c520 comp rx=0 tx=0).stop 2024-04-25T00:38:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 -- 172.21.15.29:0/285740162 shutdown_connections 2024-04-25T00:38:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 --2- 172.21.15.29:0/285740162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa83806b5f0 0x7fa838071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 --2- 172.21.15.29:0/285740162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa83806b210 0x7fa83806db00 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 -- 172.21.15.29:0/285740162 >> 172.21.15.29:0/285740162 conn(0x7fa8380d0cb0 msgr2=0x7fa8380d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 -- 172.21.15.29:0/285740162 shutdown_connections 2024-04-25T00:38:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 -- 172.21.15.29:0/285740162 wait complete. 2024-04-25T00:38:15.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 Processor -- start 2024-04-25T00:38:15.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 -- start start 2024-04-25T00:38:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa83806b210 0x7fa838147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa83806b5f0 0x7fa8380b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8380d18f0 0x7fa8380b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa83816adb0 con 0x7fa8380d18f0 2024-04-25T00:38:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa83806ba50 con 0x7fa83806b5f0 2024-04-25T00:38:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.251+0000 7fa83d984640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa83816af30 con 0x7fa83806b210 2024-04-25T00:38:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa8377fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8380d18f0 0x7fa8380b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa8377fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8380d18f0 0x7fa8380b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55440/0 (socket says 172.21.15.29:55440) 2024-04-25T00:38:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa8377fe640 1 -- 172.21.15.29:0/3518521096 learned_addr learned my addr 172.21.15.29:0/3518521096 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa836ffd640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa83806b210 0x7fa838147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa8367fc640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa83806b5f0 0x7fa8380b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa8377fe640 1 -- 172.21.15.29:0/3518521096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa83806b210 msgr2=0x7fa838147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa8377fe640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa83806b210 0x7fa838147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa8377fe640 1 -- 172.21.15.29:0/3518521096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa83806b5f0 msgr2=0x7fa8380b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa8377fe640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa83806b5f0 0x7fa8380b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa8377fe640 1 -- 172.21.15.29:0/3518521096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa838139370 con 0x7fa8380d18f0 2024-04-25T00:38:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa836ffd640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa83806b210 0x7fa838147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa8367fc640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa83806b5f0 0x7fa8380b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa8377fe640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8380d18f0 0x7fa8380b1870 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b970 tx=0x7fa83000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa81ffff640 1 -- 172.21.15.29:0/3518521096 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830019070 con 0x7fa8380d18f0 2024-04-25T00:38:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa81ffff640 1 -- 172.21.15.29:0/3518521096 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83000cb00 con 0x7fa8380d18f0 2024-04-25T00:38:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa81ffff640 1 -- 172.21.15.29:0/3518521096 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830014440 con 0x7fa8380d18f0 2024-04-25T00:38:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa83d984640 1 -- 172.21.15.29:0/3518521096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8381387f0 con 0x7fa8380d18f0 2024-04-25T00:38:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa83d984640 1 -- 172.21.15.29:0/3518521096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa838138db0 con 0x7fa8380d18f0 2024-04-25T00:38:15.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa83d984640 1 -- 172.21.15.29:0/3518521096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa838069270 con 0x7fa8380d18f0 2024-04-25T00:38:15.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa81ffff640 1 -- 172.21.15.29:0/3518521096 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fa83001c070 con 0x7fa8380d18f0 2024-04-25T00:38:15.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa81ffff640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa824067cb0 0x7fa82406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa836ffd640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa824067cb0 0x7fa82406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:15.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.255+0000 7fa81ffff640 1 -- 172.21.15.29:0/3518521096 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7fa8300864b0 con 0x7fa8380d18f0 2024-04-25T00:38:15.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.259+0000 7fa836ffd640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa824067cb0 0x7fa82406a170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8280ae000 tx=0x7fa828008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:15.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.259+0000 7fa81ffff640 1 -- 172.21.15.29:0/3518521096 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa830023020 con 0x7fa8380d18f0 2024-04-25T00:38:15.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.423+0000 7fa83d984640 1 -- 172.21.15.29:0/3518521096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8381514d0 con 0x7fa8380d18f0 2024-04-25T00:38:15.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.427+0000 7fa81ffff640 1 -- 172.21.15.29:0/3518521096 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16713 (secure 0 0 0) 0x7fa830056a90 con 0x7fa8380d18f0 2024-04-25T00:38:15.429 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:15.429 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":105,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:11.560718+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:37:22.602795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:37:23.310674+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:15.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.427+0000 7fa83d984640 1 -- 172.21.15.29:0/3518521096 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa824067cb0 msgr2=0x7fa82406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.427+0000 7fa83d984640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa824067cb0 0x7fa82406a170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8280ae000 tx=0x7fa828008040 comp rx=0 tx=0).stop 2024-04-25T00:38:15.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.427+0000 7fa83d984640 1 -- 172.21.15.29:0/3518521096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8380d18f0 msgr2=0x7fa8380b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.427+0000 7fa83d984640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8380d18f0 0x7fa8380b1870 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b970 tx=0x7fa83000be30 comp rx=0 tx=0).stop 2024-04-25T00:38:15.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.427+0000 7fa83d984640 1 -- 172.21.15.29:0/3518521096 shutdown_connections 2024-04-25T00:38:15.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.427+0000 7fa83d984640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa824067cb0 0x7fa82406a170 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.427+0000 7fa83d984640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8380d18f0 0x7fa8380b1870 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.427+0000 7fa83d984640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa83806b5f0 0x7fa8380b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.427+0000 7fa83d984640 1 --2- 172.21.15.29:0/3518521096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa83806b210 0x7fa838147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.427+0000 7fa83d984640 1 -- 172.21.15.29:0/3518521096 >> 172.21.15.29:0/3518521096 conn(0x7fa8380d0cb0 msgr2=0x7fa8380c4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:15.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.427+0000 7fa83d984640 1 -- 172.21.15.29:0/3518521096 shutdown_connections 2024-04-25T00:38:15.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.427+0000 7fa83d984640 1 -- 172.21.15.29:0/3518521096 wait complete. 2024-04-25T00:38:15.448 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:37:22.602795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:37:23.310674+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:38:15.449 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:38:15.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c68a3640 1 Processor -- start 2024-04-25T00:38:15.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c68a3640 1 -- start start 2024-04-25T00:38:15.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c68a3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44c006c950 0x7f44c006cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c68a3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f44c00cd410 0x7f44c00cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c68a3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44c00cc3b0 con 0x7f44c015e440 2024-04-25T00:38:15.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c68a3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f44c006d270 con 0x7f44c00cd410 2024-04-25T00:38:15.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c68a3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44c00b11d0 con 0x7f44c006c950 2024-04-25T00:38:15.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c4e40640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f44c00cd410 0x7f44c00cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:15.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c4e40640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f44c00cd410 0x7f44c00cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55442/0 (socket says 172.21.15.29:55442) 2024-04-25T00:38:15.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c4e40640 1 -- 172.21.15.29:0/982864628 learned_addr learned my addr 172.21.15.29:0/982864628 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:15.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c4e40640 1 -- 172.21.15.29:0/982864628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44c006c950 msgr2=0x7f44c006cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:15.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c4e40640 1 --2- 172.21.15.29:0/982864628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44c006c950 0x7f44c006cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c4e40640 1 -- 172.21.15.29:0/982864628 >> v1:172.21.15.29:6789/0 conn(0x7f44c015e440 legacy=0x7f44c00d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.563+0000 7f44c4e40640 1 -- 172.21.15.29:0/982864628 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c00cc650 con 0x7f44c00cd410 2024-04-25T00:38:15.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c4e40640 1 --2- 172.21.15.29:0/982864628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44c00cd410 0x7f44c00cd8a0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009870 tx=0x7f44b801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d198702d9a6cca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:15.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44beffd640 1 -- 172.21.15.29:0/982864628 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b80ab650 con 0x7f44c00cd410 2024-04-25T00:38:15.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44beffd640 1 -- 172.21.15.29:0/982864628 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44b80b0070 con 0x7f44c00cd410 2024-04-25T00:38:15.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44beffd640 1 -- 172.21.15.29:0/982864628 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b80aba10 con 0x7f44c00cd410 2024-04-25T00:38:15.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 -- 172.21.15.29:0/982864628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44c00cd410 msgr2=0x7f44c00cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 --2- 172.21.15.29:0/982864628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44c00cd410 0x7f44c00cd8a0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009870 tx=0x7f44b801c560 comp rx=0 tx=0).stop 2024-04-25T00:38:15.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 -- 172.21.15.29:0/982864628 shutdown_connections 2024-04-25T00:38:15.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 --2- 172.21.15.29:0/982864628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44c00cd410 0x7f44c00cd8a0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 --2- 172.21.15.29:0/982864628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44c006c950 0x7f44c006cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 -- 172.21.15.29:0/982864628 >> 172.21.15.29:0/982864628 conn(0x7f44c00d1d60 msgr2=0x7f44c00d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:15.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 -- 172.21.15.29:0/982864628 shutdown_connections 2024-04-25T00:38:15.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 -- 172.21.15.29:0/982864628 wait complete. 2024-04-25T00:38:15.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 Processor -- start 2024-04-25T00:38:15.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 -- start start 2024-04-25T00:38:15.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44c006c950 0x7f44c014eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44c00cd410 0x7f44c00c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c015e440 0x7f44c00c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f44c00d11c0 con 0x7f44c006c950 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44c006c520 con 0x7f44c015e440 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44c016ae20 con 0x7f44c00cd410 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44bffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44c006c950 0x7f44c014eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44bffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44c006c950 0x7f44c014eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55448/0 (socket says 172.21.15.29:55448) 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44bffff640 1 -- 172.21.15.29:0/816923288 learned_addr learned my addr 172.21.15.29:0/816923288 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44bf7fe640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44c00cd410 0x7f44c00c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44bffff640 1 -- 172.21.15.29:0/816923288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44c00cd410 msgr2=0x7f44c00c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44bffff640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44c00cd410 0x7f44c00c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44bffff640 1 -- 172.21.15.29:0/816923288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c015e440 msgr2=0x7f44c00c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c4e40640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c015e440 0x7f44c00c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44bffff640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c015e440 0x7f44c00c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44bffff640 1 -- 172.21.15.29:0/816923288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c015a8a0 con 0x7f44c006c950 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44bf7fe640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44c00cd410 0x7f44c00c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44bffff640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44c006c950 0x7f44c014eb20 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f44b000ba80 tx=0x7f44b000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44bcff9640 1 -- 172.21.15.29:0/816923288 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0019070 con 0x7f44c006c950 2024-04-25T00:38:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44bcff9640 1 -- 172.21.15.29:0/816923288 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44b000cc20 con 0x7f44c006c950 2024-04-25T00:38:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.567+0000 7f44c68a3640 1 -- 172.21.15.29:0/816923288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c00b0e90 con 0x7f44c006c950 2024-04-25T00:38:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.571+0000 7f44bcff9640 1 -- 172.21.15.29:0/816923288 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0014440 con 0x7f44c006c950 2024-04-25T00:38:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.571+0000 7f44c68a3640 1 -- 172.21.15.29:0/816923288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c00c8520 con 0x7f44c006c950 2024-04-25T00:38:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.571+0000 7f44c68a3640 1 -- 172.21.15.29:0/816923288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c0069270 con 0x7f44c006c950 2024-04-25T00:38:15.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.571+0000 7f44bcff9640 1 -- 172.21.15.29:0/816923288 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f44b001c070 con 0x7f44c006c950 2024-04-25T00:38:15.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.571+0000 7f44bcff9640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f44a0067cb0 0x7f44a006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.571+0000 7f44bf7fe640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f44a0067cb0 0x7f44a006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:15.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.571+0000 7f44bcff9640 1 -- 172.21.15.29:0/816923288 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f44b0085fb0 con 0x7f44c006c950 2024-04-25T00:38:15.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.571+0000 7f44bf7fe640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f44a0067cb0 0x7f44a006a170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f44c00b7c00 tx=0x7f44a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:15.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.575+0000 7f44bcff9640 1 -- 172.21.15.29:0/816923288 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f44c015a8a0 con 0x7f44c006c950 2024-04-25T00:38:15.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44c68a3640 1 -- 172.21.15.29:0/816923288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f44c00d0100 con 0x7f44c006c950 2024-04-25T00:38:15.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44bcff9640 1 -- 172.21.15.29:0/816923288 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f44b0055f00 con 0x7f44c006c950 2024-04-25T00:38:15.741 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:38:15.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44c68a3640 1 -- 172.21.15.29:0/816923288 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f44a0067cb0 msgr2=0x7f44a006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44c68a3640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f44a0067cb0 0x7f44a006a170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f44c00b7c00 tx=0x7f44a8009040 comp rx=0 tx=0).stop 2024-04-25T00:38:15.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44c68a3640 1 -- 172.21.15.29:0/816923288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44c006c950 msgr2=0x7f44c014eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44c68a3640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44c006c950 0x7f44c014eb20 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f44b000ba80 tx=0x7f44b000bf40 comp rx=0 tx=0).stop 2024-04-25T00:38:15.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44c68a3640 1 -- 172.21.15.29:0/816923288 shutdown_connections 2024-04-25T00:38:15.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44c68a3640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f44a0067cb0 0x7f44a006a170 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44c68a3640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c015e440 0x7f44c00c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44c68a3640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44c00cd410 0x7f44c00c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44c68a3640 1 --2- 172.21.15.29:0/816923288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44c006c950 0x7f44c014eb20 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44c68a3640 1 -- 172.21.15.29:0/816923288 >> 172.21.15.29:0/816923288 conn(0x7f44c00d1d60 msgr2=0x7f44c015d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:15.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44c68a3640 1 -- 172.21.15.29:0/816923288 shutdown_connections 2024-04-25T00:38:15.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.739+0000 7f44c68a3640 1 -- 172.21.15.29:0/816923288 wait complete. 2024-04-25T00:38:15.761 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:38:15.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fcd362640 1 Processor -- start 2024-04-25T00:38:15.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fcd362640 1 -- start start 2024-04-25T00:38:15.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fcd362640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fc815e250 0x7f1fc80d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fcd362640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1fc815e630 0x7f1fc80d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fcd362640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fc80c8690 con 0x7f1fc80c9450 2024-04-25T00:38:15.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fcd362640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1fc80c8440 con 0x7f1fc815e630 2024-04-25T00:38:15.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fcd362640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc80c8970 con 0x7f1fc815e250 2024-04-25T00:38:15.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fc67fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1fc815e630 0x7f1fc80d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:15.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fc67fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1fc815e630 0x7f1fc80d29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55458/0 (socket says 172.21.15.29:55458) 2024-04-25T00:38:15.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fc67fc640 1 -- 172.21.15.29:0/1313887589 learned_addr learned my addr 172.21.15.29:0/1313887589 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:15.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fc67fc640 1 -- 172.21.15.29:0/1313887589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fc815e250 msgr2=0x7f1fc80d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:15.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fc67fc640 1 --2- 172.21.15.29:0/1313887589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fc815e250 0x7f1fc80d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fc67fc640 1 -- 172.21.15.29:0/1313887589 >> v1:172.21.15.157:6789/0 conn(0x7f1fc80c9450 legacy=0x7f1fc80c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:15.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.871+0000 7f1fc67fc640 1 -- 172.21.15.29:0/1313887589 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc80c8c50 con 0x7f1fc815e630 2024-04-25T00:38:15.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fc67fc640 1 --2- 172.21.15.29:0/1313887589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fc815e630 0x7f1fc80d29e0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80c85c0 tx=0x7f1fbc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19b14c9b279e21d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:15.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fc5ffb640 1 -- 172.21.15.29:0/1313887589 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0b0070 con 0x7f1fc815e630 2024-04-25T00:38:15.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fc5ffb640 1 -- 172.21.15.29:0/1313887589 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fbc0ab600 con 0x7f1fc815e630 2024-04-25T00:38:15.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fc5ffb640 1 -- 172.21.15.29:0/1313887589 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0ab9d0 con 0x7f1fc815e630 2024-04-25T00:38:15.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fcd362640 1 -- 172.21.15.29:0/1313887589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fc815e630 msgr2=0x7f1fc80d29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:15.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fcd362640 1 --2- 172.21.15.29:0/1313887589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fc815e630 0x7f1fc80d29e0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80c85c0 tx=0x7f1fbc01c470 comp rx=0 tx=0).stop 2024-04-25T00:38:15.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fcd362640 1 -- 172.21.15.29:0/1313887589 shutdown_connections 2024-04-25T00:38:15.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fcd362640 1 --2- 172.21.15.29:0/1313887589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fc815e630 0x7f1fc80d29e0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fcd362640 1 --2- 172.21.15.29:0/1313887589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fc815e250 0x7f1fc80d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fcd362640 1 -- 172.21.15.29:0/1313887589 >> 172.21.15.29:0/1313887589 conn(0x7f1fc80d3f70 msgr2=0x7f1fc80d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:15.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fcd362640 1 -- 172.21.15.29:0/1313887589 shutdown_connections 2024-04-25T00:38:15.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fcd362640 1 -- 172.21.15.29:0/1313887589 wait complete. 2024-04-25T00:38:15.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fcd362640 1 Processor -- start 2024-04-25T00:38:15.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.875+0000 7f1fcd362640 1 -- start start 2024-04-25T00:38:15.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fcd362640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fc80c9450 0x7f1fc8154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fcd362640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc815e250 0x7f1fc8158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fcd362640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fc815e630 0x7f1fc815a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fcd362640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc816aff0 con 0x7f1fc80c9450 2024-04-25T00:38:15.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fcd362640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc80d1ed0 con 0x7f1fc815e250 2024-04-25T00:38:15.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fcd362640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc816b170 con 0x7f1fc815e630 2024-04-25T00:38:15.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fc6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fc80c9450 0x7f1fc8154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fc6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fc80c9450 0x7f1fc8154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55466/0 (socket says 172.21.15.29:55466) 2024-04-25T00:38:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fc6ffd640 1 -- 172.21.15.29:0/507608850 learned_addr learned my addr 172.21.15.29:0/507608850 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fc6ffd640 1 -- 172.21.15.29:0/507608850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fc815e630 msgr2=0x7f1fc815a360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:38:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fc6ffd640 1 --2- 172.21.15.29:0/507608850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fc815e630 0x7f1fc815a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fc6ffd640 1 -- 172.21.15.29:0/507608850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc815e250 msgr2=0x7f1fc8158970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:38:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fc6ffd640 1 --2- 172.21.15.29:0/507608850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc815e250 0x7f1fc8158970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fc6ffd640 1 -- 172.21.15.29:0/507608850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc81592c0 con 0x7f1fc80c9450 2024-04-25T00:38:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fc6ffd640 1 --2- 172.21.15.29:0/507608850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fc80c9450 0x7f1fc8154470 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800ba50 tx=0x7f1fb800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1faf7fe640 1 -- 172.21.15.29:0/507608850 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8019070 con 0x7f1fc80c9450 2024-04-25T00:38:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1faf7fe640 1 -- 172.21.15.29:0/507608850 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb800cc00 con 0x7f1fc80c9450 2024-04-25T00:38:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1faf7fe640 1 -- 172.21.15.29:0/507608850 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8014440 con 0x7f1fc80c9450 2024-04-25T00:38:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fcd362640 1 -- 172.21.15.29:0/507608850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc81595a0 con 0x7f1fc80c9450 2024-04-25T00:38:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fcd362640 1 -- 172.21.15.29:0/507608850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc8159b60 con 0x7f1fc80c9450 2024-04-25T00:38:15.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fcd362640 1 -- 172.21.15.29:0/507608850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc8069270 con 0x7f1fc80c9450 2024-04-25T00:38:15.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1faf7fe640 1 -- 172.21.15.29:0/507608850 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f1fb801c070 con 0x7f1fc80c9450 2024-04-25T00:38:15.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1faf7fe640 1 --2- 172.21.15.29:0/507608850 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fb4067cb0 0x7f1fb406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:15.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1faf7fe640 1 -- 172.21.15.29:0/507608850 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f1fb8086860 con 0x7f1fc80c9450 2024-04-25T00:38:15.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.879+0000 7f1fc67fc640 1 --2- 172.21.15.29:0/507608850 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fb4067cb0 0x7f1fb406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:15.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.883+0000 7f1fc67fc640 1 --2- 172.21.15.29:0/507608850 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fb4067cb0 0x7f1fb406a170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0ad000 tx=0x7f1fbc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:15.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:15.883+0000 7f1faf7fe640 1 -- 172.21.15.29:0/507608850 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1fb8023070 con 0x7f1fc80c9450 2024-04-25T00:38:16.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.039+0000 7f1fcd362640 1 -- 172.21.15.29:0/507608850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1fc815ee90 con 0x7f1fc80c9450 2024-04-25T00:38:16.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.039+0000 7f1faf7fe640 1 -- 172.21.15.29:0/507608850 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f1fb8056e70 con 0x7f1fc80c9450 2024-04-25T00:38:16.042 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:38:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.043+0000 7f1fcd362640 1 -- 172.21.15.29:0/507608850 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fb4067cb0 msgr2=0x7f1fb406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.043+0000 7f1fcd362640 1 --2- 172.21.15.29:0/507608850 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fb4067cb0 0x7f1fb406a170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0ad000 tx=0x7f1fbc008040 comp rx=0 tx=0).stop 2024-04-25T00:38:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.043+0000 7f1fcd362640 1 -- 172.21.15.29:0/507608850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fc80c9450 msgr2=0x7f1fc8154470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.043+0000 7f1fcd362640 1 --2- 172.21.15.29:0/507608850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fc80c9450 0x7f1fc8154470 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800ba50 tx=0x7f1fb800bf10 comp rx=0 tx=0).stop 2024-04-25T00:38:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.043+0000 7f1fcd362640 1 -- 172.21.15.29:0/507608850 shutdown_connections 2024-04-25T00:38:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.043+0000 7f1fcd362640 1 --2- 172.21.15.29:0/507608850 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fb4067cb0 0x7f1fb406a170 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.043+0000 7f1fcd362640 1 --2- 172.21.15.29:0/507608850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fc815e630 0x7f1fc815a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.043+0000 7f1fcd362640 1 --2- 172.21.15.29:0/507608850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc815e250 0x7f1fc8158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.043+0000 7f1fcd362640 1 --2- 172.21.15.29:0/507608850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fc80c9450 0x7f1fc8154470 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.043+0000 7f1fcd362640 1 -- 172.21.15.29:0/507608850 >> 172.21.15.29:0/507608850 conn(0x7f1fc80d3f70 msgr2=0x7f1fc80d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.043+0000 7f1fcd362640 1 -- 172.21.15.29:0/507608850 shutdown_connections 2024-04-25T00:38:16.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.043+0000 7f1fcd362640 1 -- 172.21.15.29:0/507608850 wait complete. 2024-04-25T00:38:16.060 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:38:16.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177e533640 1 Processor -- start 2024-04-25T00:38:16.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177e533640 1 -- start start 2024-04-25T00:38:16.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177e533640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f17780cf690 0x7f17780d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177e533640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17780d2ea0 con 0x7f17780d2a90 2024-04-25T00:38:16.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177e533640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17780cfc10 con 0x7f177815e200 2024-04-25T00:38:16.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177e533640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f17780b38d0 con 0x7f17780cf690 2024-04-25T00:38:16.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177cad0640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f17780d2a90 0x7f17780b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54432/0 (socket says 172.21.15.29:54432) 2024-04-25T00:38:16.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177cad0640 1 -- 172.21.15.29:0/1627136229 learned_addr learned my addr 172.21.15.29:0/1627136229 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:16.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f17777fe640 1 --2- 172.21.15.29:0/1627136229 >> v2:172.21.15.157:3300/0 conn(0x7f17780cf690 0x7f17780d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2556499003 0 0) 0x7f17700014a0 con 0x7f17780d2a90 2024-04-25T00:38:16.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1760003530 con 0x7f17780d2a90 2024-04-25T00:38:16.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2202055317 0 0) 0x7f1760003530 con 0x7f17780d2a90 2024-04-25T00:38:16.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 >> v2:172.21.15.157:3300/0 conn(0x7f17780cf690 msgr2=0x7f17780d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:16.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 --2- 172.21.15.29:0/1627136229 >> v2:172.21.15.157:3300/0 conn(0x7f17780cf690 0x7f17780d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 >> v1:172.21.15.157:6789/0 conn(0x7f177815e200 legacy=0x7f17780d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:16.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17780cfef0 con 0x7f17780d2a90 2024-04-25T00:38:16.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f1770001bd0 con 0x7f17780d2a90 2024-04-25T00:38:16.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 >> v1:172.21.15.29:6789/0 conn(0x7f17780d2a90 legacy=0x7f17780b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 --2- 172.21.15.29:0/1627136229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17600039c0 0x7f1760003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 --2- 172.21.15.29:0/1627136229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1760004810 0x7f1760004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 --2- 172.21.15.29:0/1627136229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17600051d0 0x7f176000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f17780b38d0 con 0x7f17600051d0 2024-04-25T00:38:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f176000c050 con 0x7f1760004810 2024-04-25T00:38:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f176000c250 con 0x7f17600039c0 2024-04-25T00:38:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177cad0640 1 --2- 172.21.15.29:0/1627136229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17600051d0 0x7f176000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f17777fe640 1 --2- 172.21.15.29:0/1627136229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1760004810 0x7f1760004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1777fff640 1 --2- 172.21.15.29:0/1627136229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17600039c0 0x7f1760003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177cad0640 1 -- 172.21.15.29:0/1627136229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17600039c0 msgr2=0x7f1760003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177cad0640 1 --2- 172.21.15.29:0/1627136229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17600039c0 0x7f1760003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177cad0640 1 -- 172.21.15.29:0/1627136229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1760004810 msgr2=0x7f1760004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177cad0640 1 --2- 172.21.15.29:0/1627136229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1760004810 0x7f1760004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f177cad0640 1 -- 172.21.15.29:0/1627136229 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1760003530 con 0x7f17600051d0 2024-04-25T00:38:16.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.179+0000 7f1777fff640 1 --2- 172.21.15.29:0/1627136229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17600039c0 0x7f1760003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f177cad0640 1 -- 172.21.15.29:0/1627136229 reap_dead start 2024-04-25T00:38:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f177cad0640 1 --2- 172.21.15.29:0/1627136229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17600051d0 0x7f176000b9d0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f177000aa70 tx=0x7f177001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902d51255da204c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177000a690 con 0x7f17600051d0 2024-04-25T00:38:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1770007310 con 0x7f17600051d0 2024-04-25T00:38:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f1776ffd640 1 -- 172.21.15.29:0/1627136229 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770007700 con 0x7f17600051d0 2024-04-25T00:38:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f177e533640 1 -- 172.21.15.29:0/1627136229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17600051d0 msgr2=0x7f176000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f177e533640 1 --2- 172.21.15.29:0/1627136229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17600051d0 0x7f176000b9d0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f177000aa70 tx=0x7f177001d3a0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f177e533640 1 -- 172.21.15.29:0/1627136229 shutdown_connections 2024-04-25T00:38:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f177e533640 1 --2- 172.21.15.29:0/1627136229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17600051d0 0x7f176000b9d0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f177e533640 1 -- 172.21.15.29:0/1627136229 >> 172.21.15.29:0/1627136229 conn(0x7f177809d280 msgr2=0x7f17780d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f177e533640 1 -- 172.21.15.29:0/1627136229 shutdown_connections 2024-04-25T00:38:16.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f177e533640 1 -- 172.21.15.29:0/1627136229 wait complete. 2024-04-25T00:38:16.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f177e533640 1 Processor -- start 2024-04-25T00:38:16.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.183+0000 7f177e533640 1 -- start start 2024-04-25T00:38:16.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177e533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17600051d0 0x7f1778156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177e533640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1778157530 0x7f17781600a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177e533640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17781605e0 0x7f17780c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177e533640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f177815e5e0 con 0x7f17781605e0 2024-04-25T00:38:16.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177e533640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17780b3e00 con 0x7f1778157530 2024-04-25T00:38:16.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177e533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f177815e760 con 0x7f17600051d0 2024-04-25T00:38:16.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177cad0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17781605e0 0x7f17780c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177cad0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17781605e0 0x7f17780c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55480/0 (socket says 172.21.15.29:55480) 2024-04-25T00:38:16.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177cad0640 1 -- 172.21.15.29:0/3893712227 learned_addr learned my addr 172.21.15.29:0/3893712227 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:16.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f17777fe640 1 --2- 172.21.15.29:0/3893712227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1778157530 0x7f17781600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177cad0640 1 -- 172.21.15.29:0/3893712227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17600051d0 msgr2=0x7f1778156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177cad0640 1 --2- 172.21.15.29:0/3893712227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17600051d0 0x7f1778156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177cad0640 1 -- 172.21.15.29:0/3893712227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1778157530 msgr2=0x7f17781600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177cad0640 1 --2- 172.21.15.29:0/3893712227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1778157530 0x7f17781600a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177cad0640 1 -- 172.21.15.29:0/3893712227 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17780c8650 con 0x7f17781605e0 2024-04-25T00:38:16.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177cad0640 1 --2- 172.21.15.29:0/3893712227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17781605e0 0x7f17780c8b30 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f1770007040 tx=0x7f1760005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:16.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f1774ff9640 1 -- 172.21.15.29:0/3893712227 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177000a690 con 0x7f17781605e0 2024-04-25T00:38:16.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f1774ff9640 1 -- 172.21.15.29:0/3893712227 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17700070d0 con 0x7f17781605e0 2024-04-25T00:38:16.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177e533640 1 -- 172.21.15.29:0/3893712227 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17781579b0 con 0x7f17781605e0 2024-04-25T00:38:16.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f1774ff9640 1 -- 172.21.15.29:0/3893712227 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770001bb0 con 0x7f17781605e0 2024-04-25T00:38:16.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177e533640 1 -- 172.21.15.29:0/3893712227 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17780ca670 con 0x7f17781605e0 2024-04-25T00:38:16.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f177e533640 1 -- 172.21.15.29:0/3893712227 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17780c8820 con 0x7f17781605e0 2024-04-25T00:38:16.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f1774ff9640 1 -- 172.21.15.29:0/3893712227 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f1770004b70 con 0x7f17781605e0 2024-04-25T00:38:16.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f1774ff9640 1 --2- 172.21.15.29:0/3893712227 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1764067ce0 0x7f176406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f1777fff640 1 --2- 172.21.15.29:0/3893712227 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1764067ce0 0x7f176406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f1774ff9640 1 -- 172.21.15.29:0/3893712227 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f177011fda0 con 0x7f17781605e0 2024-04-25T00:38:16.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.187+0000 7f1777fff640 1 --2- 172.21.15.29:0/3893712227 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1764067ce0 0x7f176406a1a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1768003de0 tx=0x7f1768009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:16.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.191+0000 7f1774ff9640 1 -- 172.21.15.29:0/3893712227 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f17780d2ea0 con 0x7f17781605e0 2024-04-25T00:38:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.355+0000 7f177e533640 1 -- 172.21.15.29:0/3893712227 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1778069270 con 0x7f17781605e0 2024-04-25T00:38:16.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.355+0000 7f1774ff9640 1 -- 172.21.15.29:0/3893712227 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f17700f01a0 con 0x7f17781605e0 2024-04-25T00:38:16.357 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:38:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.355+0000 7f177e533640 1 -- 172.21.15.29:0/3893712227 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1764067ce0 msgr2=0x7f176406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.355+0000 7f177e533640 1 --2- 172.21.15.29:0/3893712227 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1764067ce0 0x7f176406a1a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1768003de0 tx=0x7f1768009040 comp rx=0 tx=0).stop 2024-04-25T00:38:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.355+0000 7f177e533640 1 -- 172.21.15.29:0/3893712227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17781605e0 msgr2=0x7f17780c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.355+0000 7f177e533640 1 --2- 172.21.15.29:0/3893712227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17781605e0 0x7f17780c8b30 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f1770007040 tx=0x7f1760005840 comp rx=0 tx=0).stop 2024-04-25T00:38:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.355+0000 7f177e533640 1 -- 172.21.15.29:0/3893712227 shutdown_connections 2024-04-25T00:38:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.355+0000 7f177e533640 1 --2- 172.21.15.29:0/3893712227 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1764067ce0 0x7f176406a1a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.355+0000 7f177e533640 1 --2- 172.21.15.29:0/3893712227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17781605e0 0x7f17780c8b30 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.355+0000 7f177e533640 1 --2- 172.21.15.29:0/3893712227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1778157530 0x7f17781600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.355+0000 7f177e533640 1 --2- 172.21.15.29:0/3893712227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17600051d0 0x7f1778156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.355+0000 7f177e533640 1 -- 172.21.15.29:0/3893712227 >> 172.21.15.29:0/3893712227 conn(0x7f177809d280 msgr2=0x7f177806d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:16.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.359+0000 7f177e533640 1 -- 172.21.15.29:0/3893712227 shutdown_connections 2024-04-25T00:38:16.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.359+0000 7f177e533640 1 -- 172.21.15.29:0/3893712227 wait complete. 2024-04-25T00:38:16.377 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:38:16.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.491+0000 7f1be7dd0640 1 Processor -- start 2024-04-25T00:38:16.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.491+0000 7f1be7dd0640 1 -- start start 2024-04-25T00:38:16.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.495+0000 7f1be7dd0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1be006b920 0x7f1be0071d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.495+0000 7f1be7dd0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1be006f7d0 con 0x7f1be00d01d0 2024-04-25T00:38:16.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.495+0000 7f1be7dd0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1be006f350 con 0x7f1be00cc9d0 2024-04-25T00:38:16.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.495+0000 7f1be7dd0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1be01627a0 con 0x7f1be006b920 2024-04-25T00:38:16.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.495+0000 7f1be636d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1be006b920 0x7f1be0071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.495+0000 7f1be636d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1be006b920 0x7f1be0071d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55490/0 (socket says 172.21.15.29:55490) 2024-04-25T00:38:16.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.495+0000 7f1be636d640 1 -- 172.21.15.29:0/1368385675 learned_addr learned my addr 172.21.15.29:0/1368385675 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:16.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.495+0000 7f1be536b640 1 --1- 172.21.15.29:0/1368385675 >> v1:172.21.15.29:6789/0 conn(0x7f1be00d01d0 0x7f1be006b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54434/0 (socket says 172.21.15.29:54434) 2024-04-25T00:38:16.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.495+0000 7f1be636d640 1 -- 172.21.15.29:0/1368385675 >> v1:172.21.15.157:6789/0 conn(0x7f1be00cc9d0 legacy=0x7f1be00ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:16.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.495+0000 7f1be636d640 1 -- 172.21.15.29:0/1368385675 >> v1:172.21.15.29:6789/0 conn(0x7f1be00d01d0 legacy=0x7f1be006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.495+0000 7f1be4b6a640 1 -- 172.21.15.29:0/1368385675 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 479272606 0 0) 0x7f1be006f7d0 con 0x7f1be00d01d0 2024-04-25T00:38:16.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.495+0000 7f1be636d640 1 -- 172.21.15.29:0/1368385675 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be0162a80 con 0x7f1be006b920 2024-04-25T00:38:16.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be636d640 1 --2- 172.21.15.29:0/1368385675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be006b920 0x7f1be0071d10 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009e80 tx=0x7f1bd801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f6055ea5eed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:16.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be4b6a640 1 -- 172.21.15.29:0/1368385675 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80a57f0 con 0x7f1be006b920 2024-04-25T00:38:16.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be4b6a640 1 -- 172.21.15.29:0/1368385675 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd80a5990 con 0x7f1be006b920 2024-04-25T00:38:16.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be4b6a640 1 -- 172.21.15.29:0/1368385675 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80a5d60 con 0x7f1be006b920 2024-04-25T00:38:16.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1368385675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be006b920 msgr2=0x7f1be0071d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 --2- 172.21.15.29:0/1368385675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be006b920 0x7f1be0071d10 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009e80 tx=0x7f1bd801ca10 comp rx=0 tx=0).stop 2024-04-25T00:38:16.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1368385675 shutdown_connections 2024-04-25T00:38:16.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 --2- 172.21.15.29:0/1368385675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be006b920 0x7f1be0071d10 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1368385675 >> 172.21.15.29:0/1368385675 conn(0x7f1be00d0c30 msgr2=0x7f1be00d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:16.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1368385675 shutdown_connections 2024-04-25T00:38:16.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1368385675 wait complete. 2024-04-25T00:38:16.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 Processor -- start 2024-04-25T00:38:16.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 -- start start 2024-04-25T00:38:16.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1be006b920 0x7f1be0155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1be00cc9d0 0x7f1be00adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be00d01d0 0x7f1be0152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1be016ca70 con 0x7f1be00d01d0 2024-04-25T00:38:16.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1be016c8f0 con 0x7f1be006b920 2024-04-25T00:38:16.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be7dd0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1be016cbf0 con 0x7f1be00cc9d0 2024-04-25T00:38:16.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be636d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be00d01d0 0x7f1be0152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be636d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be00d01d0 0x7f1be0152d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55500/0 (socket says 172.21.15.29:55500) 2024-04-25T00:38:16.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be636d640 1 -- 172.21.15.29:0/1098144105 learned_addr learned my addr 172.21.15.29:0/1098144105 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:16.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be5b6c640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1be006b920 0x7f1be0155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be536b640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1be00cc9d0 0x7f1be00adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be636d640 1 -- 172.21.15.29:0/1098144105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1be00cc9d0 msgr2=0x7f1be00adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be636d640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1be00cc9d0 0x7f1be00adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be636d640 1 -- 172.21.15.29:0/1098144105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1be006b920 msgr2=0x7f1be0155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.499+0000 7f1be636d640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1be006b920 0x7f1be0155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.503+0000 7f1be636d640 1 -- 172.21.15.29:0/1098144105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be0154890 con 0x7f1be00d01d0 2024-04-25T00:38:16.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.503+0000 7f1be636d640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be00d01d0 0x7f1be0152d10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8019620 tx=0x7f1bd80197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:16.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.503+0000 7f1bd67fc640 1 -- 172.21.15.29:0/1098144105 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80b8670 con 0x7f1be00d01d0 2024-04-25T00:38:16.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.503+0000 7f1bd67fc640 1 -- 172.21.15.29:0/1098144105 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd80be070 con 0x7f1be00d01d0 2024-04-25T00:38:16.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.503+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1098144105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be0154b70 con 0x7f1be00d01d0 2024-04-25T00:38:16.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.503+0000 7f1bd67fc640 1 -- 172.21.15.29:0/1098144105 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80b8a90 con 0x7f1be00d01d0 2024-04-25T00:38:16.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.503+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1098144105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be0155130 con 0x7f1be00d01d0 2024-04-25T00:38:16.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.503+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1098144105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be0069270 con 0x7f1be00d01d0 2024-04-25T00:38:16.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.503+0000 7f1bd67fc640 1 -- 172.21.15.29:0/1098144105 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f1bd80b8c30 con 0x7f1be00d01d0 2024-04-25T00:38:16.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.503+0000 7f1bd67fc640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1bc4067cb0 0x7f1bc406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.503+0000 7f1be5b6c640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1bc4067cb0 0x7f1bc406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.503+0000 7f1bd67fc640 1 -- 172.21.15.29:0/1098144105 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f1bd8121590 con 0x7f1be00d01d0 2024-04-25T00:38:16.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.503+0000 7f1be5b6c640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1bc4067cb0 0x7f1bc406a170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1be0159fc0 tx=0x7f1bdc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:16.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.507+0000 7f1bd67fc640 1 -- 172.21.15.29:0/1098144105 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1bd8016600 con 0x7f1be00d01d0 2024-04-25T00:38:16.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.699+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1098144105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1be00b0820 con 0x7f1be00d01d0 2024-04-25T00:38:16.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.703+0000 7f1bd67fc640 1 -- 172.21.15.29:0/1098144105 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5189 (secure 0 0 0) 0x7f1bd80f1aa0 con 0x7f1be00d01d0 2024-04-25T00:38:16.704 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:16.704 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1884979435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1884979435},{"type":"v1","addr":"172.21.15.157:6833","nonce":1884979435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1394307674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1394307674},{"type":"v1","addr":"172.21.15.157:6835","nonce":1394307674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/3915943091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3915943091},{"type":"v1","addr":"172.21.15.29:6834","nonce":3915943091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:37:24.327819+0000","modified":"2024-04-25T00:37:27.319963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1444171533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1444171533},{"type":"v1","addr":"172.21.15.29:6835","nonce":1444171533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7896,"qdb_cluster":[7896]},"id":8}]} 2024-04-25T00:38:16.705 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 68 2024-04-25T00:38:16.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.703+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1098144105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1bc4067cb0 msgr2=0x7f1bc406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.703+0000 7f1be7dd0640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1bc4067cb0 0x7f1bc406a170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1be0159fc0 tx=0x7f1bdc009040 comp rx=0 tx=0).stop 2024-04-25T00:38:16.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.703+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1098144105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be00d01d0 msgr2=0x7f1be0152d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.703+0000 7f1be7dd0640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be00d01d0 0x7f1be0152d10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8019620 tx=0x7f1bd80197b0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.703+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1098144105 shutdown_connections 2024-04-25T00:38:16.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.703+0000 7f1be7dd0640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1bc4067cb0 0x7f1bc406a170 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.703+0000 7f1be7dd0640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be00d01d0 0x7f1be0152d10 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.703+0000 7f1be7dd0640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1be00cc9d0 0x7f1be00adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.703+0000 7f1be7dd0640 1 --2- 172.21.15.29:0/1098144105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1be006b920 0x7f1be0155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.703+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1098144105 >> 172.21.15.29:0/1098144105 conn(0x7f1be00d0c30 msgr2=0x7f1be00c70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:16.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.703+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1098144105 shutdown_connections 2024-04-25T00:38:16.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.703+0000 7f1be7dd0640 1 -- 172.21.15.29:0/1098144105 wait complete. 2024-04-25T00:38:16.722 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:16.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a47955640 1 Processor -- start 2024-04-25T00:38:16.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a47955640 1 -- start start 2024-04-25T00:38:16.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a47955640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a4006dce0 0x7f3a40150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a47955640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400b2b70 0x7f3a40151330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a47955640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a400b3130 con 0x7f3a4015d980 2024-04-25T00:38:16.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a47955640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3a400cf560 con 0x7f3a4006dce0 2024-04-25T00:38:16.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a47955640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a400b3410 con 0x7f3a400b2b70 2024-04-25T00:38:16.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a456f1640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3a4015d980 0x7f3a4006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54440/0 (socket says 172.21.15.29:54440) 2024-04-25T00:38:16.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a456f1640 1 -- 172.21.15.29:0/3784753809 learned_addr learned my addr 172.21.15.29:0/3784753809 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:16.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3767749080 0 0) 0x7f3a400b3130 con 0x7f3a4015d980 2024-04-25T00:38:16.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a44ef0640 1 --2- 172.21.15.29:0/3784753809 >> v2:172.21.15.157:3300/0 conn(0x7f3a4006dce0 0x7f3a40150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a2c003450 con 0x7f3a4015d980 2024-04-25T00:38:16.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a45ef2640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400b2b70 0x7f3a40151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1047398477 0 0) 0x7f3a2c003450 con 0x7f3a4015d980 2024-04-25T00:38:16.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400b2b70 msgr2=0x7f3a40151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:16.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400b2b70 0x7f3a40151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 >> v2:172.21.15.157:3300/0 conn(0x7f3a4006dce0 msgr2=0x7f3a40150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:16.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 --2- 172.21.15.29:0/3784753809 >> v2:172.21.15.157:3300/0 conn(0x7f3a4006dce0 0x7f3a40150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a400b36f0 con 0x7f3a4015d980 2024-04-25T00:38:16.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f3a3c001680 con 0x7f3a4015d980 2024-04-25T00:38:16.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 >> v1:172.21.15.29:6789/0 conn(0x7f3a4015d980 legacy=0x7f3a4006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a2c003ab0 0x7f3a2c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a2c008860 0x7f3a2c00acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400b2b70 0x7f3a2c00d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a400cf560 con 0x7f3a2c008860 2024-04-25T00:38:16.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3a400b3410 con 0x7f3a2c003ab0 2024-04-25T00:38:16.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a2c00dc50 con 0x7f3a400b2b70 2024-04-25T00:38:16.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a44ef0640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a2c008860 0x7f3a2c00acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a456f1640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a2c003ab0 0x7f3a2c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a45ef2640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400b2b70 0x7f3a2c00d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a44ef0640 1 -- 172.21.15.29:0/3784753809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400b2b70 msgr2=0x7f3a2c00d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:16.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a44ef0640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400b2b70 0x7f3a2c00d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a44ef0640 1 -- 172.21.15.29:0/3784753809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a2c003ab0 msgr2=0x7f3a2c003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:16.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a44ef0640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a2c003ab0 0x7f3a2c003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a44ef0640 1 -- 172.21.15.29:0/3784753809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2c003450 con 0x7f3a2c008860 2024-04-25T00:38:16.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a456f1640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a2c003ab0 0x7f3a2c003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:16.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.831+0000 7f3a45ef2640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400b2b70 0x7f3a2c00d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:16.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a44ef0640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a2c008860 0x7f3a2c00acd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009ae0 tx=0x7f3a3001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e3e9e412930c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:16.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a300b2070 con 0x7f3a2c008860 2024-04-25T00:38:16.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a300a56a0 con 0x7f3a2c008860 2024-04-25T00:38:16.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a37fff640 1 -- 172.21.15.29:0/3784753809 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a300096b0 con 0x7f3a2c008860 2024-04-25T00:38:16.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 -- 172.21.15.29:0/3784753809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a2c008860 msgr2=0x7f3a2c00acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 --2- 172.21.15.29:0/3784753809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a2c008860 0x7f3a2c00acd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009ae0 tx=0x7f3a3001cab0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a45ef2640 1 -- 172.21.15.29:0/3784753809 reap_dead start 2024-04-25T00:38:16.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 -- 172.21.15.29:0/3784753809 shutdown_connections 2024-04-25T00:38:16.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 -- 172.21.15.29:0/3784753809 >> 172.21.15.29:0/3784753809 conn(0x7f3a400b1870 msgr2=0x7f3a400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:16.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 -- 172.21.15.29:0/3784753809 shutdown_connections 2024-04-25T00:38:16.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 -- 172.21.15.29:0/3784753809 wait complete. 2024-04-25T00:38:16.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 Processor -- start 2024-04-25T00:38:16.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 -- start start 2024-04-25T00:38:16.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a2c008860 0x7f3a4014e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a4015c470 0x7f3a4015c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400ccb80 0x7f3a400cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4015dd60 con 0x7f3a4015c470 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3a400b3cb0 con 0x7f3a2c008860 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a47955640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a40151330 con 0x7f3a400ccb80 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a44ef0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a4015c470 0x7f3a4015c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a44ef0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a4015c470 0x7f3a4015c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55520/0 (socket says 172.21.15.29:55520) 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a44ef0640 1 -- 172.21.15.29:0/24713854 learned_addr learned my addr 172.21.15.29:0/24713854 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a456f1640 1 --2- 172.21.15.29:0/24713854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a2c008860 0x7f3a4014e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a44ef0640 1 -- 172.21.15.29:0/24713854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400ccb80 msgr2=0x7f3a400cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a44ef0640 1 --2- 172.21.15.29:0/24713854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400ccb80 0x7f3a400cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a44ef0640 1 -- 172.21.15.29:0/24713854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a2c008860 msgr2=0x7f3a4014e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a44ef0640 1 --2- 172.21.15.29:0/24713854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a2c008860 0x7f3a4014e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.835+0000 7f3a44ef0640 1 -- 172.21.15.29:0/24713854 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a401551b0 con 0x7f3a4015c470 2024-04-25T00:38:16.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.839+0000 7f3a44ef0640 1 --2- 172.21.15.29:0/24713854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a4015c470 0x7f3a4015c8f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009ae0 tx=0x7f3a2c003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:16.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.839+0000 7f3a35ffb640 1 -- 172.21.15.29:0/24713854 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a300b2040 con 0x7f3a4015c470 2024-04-25T00:38:16.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.839+0000 7f3a35ffb640 1 -- 172.21.15.29:0/24713854 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a3000f000 con 0x7f3a4015c470 2024-04-25T00:38:16.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.839+0000 7f3a47955640 1 -- 172.21.15.29:0/24713854 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a40155490 con 0x7f3a4015c470 2024-04-25T00:38:16.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.839+0000 7f3a35ffb640 1 -- 172.21.15.29:0/24713854 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a300096b0 con 0x7f3a4015c470 2024-04-25T00:38:16.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.839+0000 7f3a47955640 1 -- 172.21.15.29:0/24713854 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a400cd540 con 0x7f3a4015c470 2024-04-25T00:38:16.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.839+0000 7f3a47955640 1 -- 172.21.15.29:0/24713854 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4015ce30 con 0x7f3a4015c470 2024-04-25T00:38:16.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.839+0000 7f3a35ffb640 1 -- 172.21.15.29:0/24713854 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f3a300c0030 con 0x7f3a4015c470 2024-04-25T00:38:16.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.839+0000 7f3a35ffb640 1 --2- 172.21.15.29:0/24713854 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3a24067cb0 0x7f3a2406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:16.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.839+0000 7f3a35ffb640 1 -- 172.21.15.29:0/24713854 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f3a30120120 con 0x7f3a4015c470 2024-04-25T00:38:16.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.839+0000 7f3a456f1640 1 --2- 172.21.15.29:0/24713854 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3a24067cb0 0x7f3a2406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:16.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.839+0000 7f3a456f1640 1 --2- 172.21.15.29:0/24713854 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3a24067cb0 0x7f3a2406a170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3a4015d050 tx=0x7f3a3c001440 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:16.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.843+0000 7f3a35ffb640 1 -- 172.21.15.29:0/24713854 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3a300c7060 con 0x7f3a4015c470 2024-04-25T00:38:17.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:16.999+0000 7f3a47955640 1 -- 172.21.15.29:0/24713854 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a40069270 con 0x7f3a4015c470 2024-04-25T00:38:17.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.003+0000 7f3a35ffb640 1 -- 172.21.15.29:0/24713854 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16713 (secure 0 0 0) 0x7f3a300167c0 con 0x7f3a4015c470 2024-04-25T00:38:17.004 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:17.004 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":105,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:11.560718+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:37:22.602795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:37:23.310674+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:17.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.003+0000 7f3a47955640 1 -- 172.21.15.29:0/24713854 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3a24067cb0 msgr2=0x7f3a2406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.003+0000 7f3a47955640 1 --2- 172.21.15.29:0/24713854 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3a24067cb0 0x7f3a2406a170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3a4015d050 tx=0x7f3a3c001440 comp rx=0 tx=0).stop 2024-04-25T00:38:17.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.003+0000 7f3a47955640 1 -- 172.21.15.29:0/24713854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a4015c470 msgr2=0x7f3a4015c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.003+0000 7f3a47955640 1 --2- 172.21.15.29:0/24713854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a4015c470 0x7f3a4015c8f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009ae0 tx=0x7f3a2c003740 comp rx=0 tx=0).stop 2024-04-25T00:38:17.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.003+0000 7f3a47955640 1 -- 172.21.15.29:0/24713854 shutdown_connections 2024-04-25T00:38:17.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.003+0000 7f3a47955640 1 --2- 172.21.15.29:0/24713854 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3a24067cb0 0x7f3a2406a170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.003+0000 7f3a47955640 1 --2- 172.21.15.29:0/24713854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400ccb80 0x7f3a400cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.003+0000 7f3a47955640 1 --2- 172.21.15.29:0/24713854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a4015c470 0x7f3a4015c8f0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.003+0000 7f3a47955640 1 --2- 172.21.15.29:0/24713854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a2c008860 0x7f3a4014e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.003+0000 7f3a47955640 1 -- 172.21.15.29:0/24713854 >> 172.21.15.29:0/24713854 conn(0x7f3a400b1870 msgr2=0x7f3a400cfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:17.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.003+0000 7f3a47955640 1 -- 172.21.15.29:0/24713854 shutdown_connections 2024-04-25T00:38:17.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.003+0000 7f3a47955640 1 -- 172.21.15.29:0/24713854 wait complete. 2024-04-25T00:38:17.022 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:38:17.022 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:38:17.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.139+0000 7f0c61234640 1 Processor -- start 2024-04-25T00:38:17.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.139+0000 7f0c61234640 1 -- start start 2024-04-25T00:38:17.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c61234640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0c5c0b4d30 0x7f0c5c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c61234640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c5c06d210 0x7f0c5c0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c61234640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c5c06d6c0 con 0x7f0c5c15df70 2024-04-25T00:38:17.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c61234640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f0c5c06d980 con 0x7f0c5c0b4d30 2024-04-25T00:38:17.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c61234640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0c5c06dc60 con 0x7f0c5c06d210 2024-04-25T00:38:17.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c5a59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0c5c0b4d30 0x7f0c5c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c5a59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0c5c0b4d30 0x7f0c5c0b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55534/0 (socket says 172.21.15.29:55534) 2024-04-25T00:38:17.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c5a59c640 1 -- 172.21.15.29:0/3395469034 learned_addr learned my addr 172.21.15.29:0/3395469034 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:17.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c5a59c640 1 -- 172.21.15.29:0/3395469034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c5c06d210 msgr2=0x7f0c5c0ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:17.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c5b59e640 1 --2- 172.21.15.29:0/3395469034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c5c06d210 0x7f0c5c0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c5a59c640 1 --2- 172.21.15.29:0/3395469034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c5c06d210 0x7f0c5c0ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c5a59c640 1 -- 172.21.15.29:0/3395469034 >> v1:172.21.15.157:6789/0 conn(0x7f0c5c15df70 legacy=0x7f0c5c0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:17.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c5a59c640 1 -- 172.21.15.29:0/3395469034 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5c06df40 con 0x7f0c5c0b4d30 2024-04-25T00:38:17.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c5b59e640 1 --2- 172.21.15.29:0/3395469034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c5c06d210 0x7f0c5c0ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:17.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c5a59c640 1 --2- 172.21.15.29:0/3395469034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c5c0b4d30 0x7f0c5c0b3e40 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009870 tx=0x7f0c5001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c177097456426e77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:17.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c59d9b640 1 -- 172.21.15.29:0/3395469034 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c500aa610 con 0x7f0c5c0b4d30 2024-04-25T00:38:17.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c59d9b640 1 -- 172.21.15.29:0/3395469034 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c500af070 con 0x7f0c5c0b4d30 2024-04-25T00:38:17.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.143+0000 7f0c59d9b640 1 -- 172.21.15.29:0/3395469034 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c500aa9d0 con 0x7f0c5c0b4d30 2024-04-25T00:38:17.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 -- 172.21.15.29:0/3395469034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c5c0b4d30 msgr2=0x7f0c5c0b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 --2- 172.21.15.29:0/3395469034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c5c0b4d30 0x7f0c5c0b3e40 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009870 tx=0x7f0c5001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 -- 172.21.15.29:0/3395469034 shutdown_connections 2024-04-25T00:38:17.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 --2- 172.21.15.29:0/3395469034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c5c06d210 0x7f0c5c0ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 --2- 172.21.15.29:0/3395469034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c5c0b4d30 0x7f0c5c0b3e40 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 -- 172.21.15.29:0/3395469034 >> 172.21.15.29:0/3395469034 conn(0x7f0c5c0b1870 msgr2=0x7f0c5c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:17.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 -- 172.21.15.29:0/3395469034 shutdown_connections 2024-04-25T00:38:17.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 -- 172.21.15.29:0/3395469034 wait complete. 2024-04-25T00:38:17.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 Processor -- start 2024-04-25T00:38:17.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 -- start start 2024-04-25T00:38:17.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c5c06d210 0x7f0c5c14e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c5c0b4d30 0x7f0c5c152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c5c15df70 0x7f0c5c14d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c06eba0 con 0x7f0c5c06d210 2024-04-25T00:38:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c06ea20 con 0x7f0c5c0b4d30 2024-04-25T00:38:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c61234640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0c5c0ae5f0 con 0x7f0c5c15df70 2024-04-25T00:38:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c5ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c5c06d210 0x7f0c5c14e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c5ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c5c06d210 0x7f0c5c14e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55538/0 (socket says 172.21.15.29:55538) 2024-04-25T00:38:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c5ad9d640 1 -- 172.21.15.29:0/2516877180 learned_addr learned my addr 172.21.15.29:0/2516877180 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c5a59c640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c5c0b4d30 0x7f0c5c152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c5b59e640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c5c15df70 0x7f0c5c14d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c5ad9d640 1 -- 172.21.15.29:0/2516877180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c5c15df70 msgr2=0x7f0c5c14d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c5ad9d640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c5c15df70 0x7f0c5c14d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c5ad9d640 1 -- 172.21.15.29:0/2516877180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c5c0b4d30 msgr2=0x7f0c5c152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c5ad9d640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c5c0b4d30 0x7f0c5c152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c5ad9d640 1 -- 172.21.15.29:0/2516877180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5c14eb90 con 0x7f0c5c06d210 2024-04-25T00:38:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c5b59e640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c5c15df70 0x7f0c5c14d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.147+0000 7f0c5ad9d640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c5c06d210 0x7f0c5c14e2d0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00b410 tx=0x7f0c4c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.151+0000 7f0c437fe640 1 -- 172.21.15.29:0/2516877180 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c015030 con 0x7f0c5c06d210 2024-04-25T00:38:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.151+0000 7f0c437fe640 1 -- 172.21.15.29:0/2516877180 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c4c004020 con 0x7f0c5c06d210 2024-04-25T00:38:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.151+0000 7f0c437fe640 1 -- 172.21.15.29:0/2516877180 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c004450 con 0x7f0c5c06d210 2024-04-25T00:38:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.151+0000 7f0c61234640 1 -- 172.21.15.29:0/2516877180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5c14ee70 con 0x7f0c5c06d210 2024-04-25T00:38:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.151+0000 7f0c61234640 1 -- 172.21.15.29:0/2516877180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c5c14f430 con 0x7f0c5c06d210 2024-04-25T00:38:17.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.151+0000 7f0c61234640 1 -- 172.21.15.29:0/2516877180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c5c069270 con 0x7f0c5c06d210 2024-04-25T00:38:17.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.151+0000 7f0c437fe640 1 -- 172.21.15.29:0/2516877180 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f0c4c018070 con 0x7f0c5c06d210 2024-04-25T00:38:17.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.151+0000 7f0c437fe640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0c48067cb0 0x7f0c4806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.151+0000 7f0c437fe640 1 -- 172.21.15.29:0/2516877180 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5781+0+0 (secure 0 0 0) 0x7f0c4c086740 con 0x7f0c5c06d210 2024-04-25T00:38:17.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.151+0000 7f0c5a59c640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0c48067cb0 0x7f0c4806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.151+0000 7f0c5a59c640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0c48067cb0 0x7f0c4806a170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0c500ae000 tx=0x7f0c50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:17.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.155+0000 7f0c437fe640 1 -- 172.21.15.29:0/2516877180 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0c4c056d80 con 0x7f0c5c06d210 2024-04-25T00:38:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.355+0000 7f0c61234640 1 -- 172.21.15.29:0/2516877180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c5c0d0760 con 0x7f0c5c06d210 2024-04-25T00:38:17.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.355+0000 7f0c437fe640 1 -- 172.21.15.29:0/2516877180 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5189 (secure 0 0 0) 0x7f0c4c056680 con 0x7f0c5c06d210 2024-04-25T00:38:17.358 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:17.358 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1884979435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1884979435},{"type":"v1","addr":"172.21.15.157:6833","nonce":1884979435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1394307674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1394307674},{"type":"v1","addr":"172.21.15.157:6835","nonce":1394307674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/3915943091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3915943091},{"type":"v1","addr":"172.21.15.29:6834","nonce":3915943091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:37:24.327819+0000","modified":"2024-04-25T00:37:27.319963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1444171533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1444171533},{"type":"v1","addr":"172.21.15.29:6835","nonce":1444171533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7896,"qdb_cluster":[7896]},"id":8}]} 2024-04-25T00:38:17.358 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 68 2024-04-25T00:38:17.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.359+0000 7f0c61234640 1 -- 172.21.15.29:0/2516877180 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0c48067cb0 msgr2=0x7f0c4806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.359+0000 7f0c61234640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0c48067cb0 0x7f0c4806a170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0c500ae000 tx=0x7f0c50008040 comp rx=0 tx=0).stop 2024-04-25T00:38:17.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.359+0000 7f0c61234640 1 -- 172.21.15.29:0/2516877180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c5c06d210 msgr2=0x7f0c5c14e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.359+0000 7f0c61234640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c5c06d210 0x7f0c5c14e2d0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00b410 tx=0x7f0c4c00b8d0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.359+0000 7f0c61234640 1 -- 172.21.15.29:0/2516877180 shutdown_connections 2024-04-25T00:38:17.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.359+0000 7f0c61234640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0c48067cb0 0x7f0c4806a170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.359+0000 7f0c61234640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c5c15df70 0x7f0c5c14d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.359+0000 7f0c61234640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c5c0b4d30 0x7f0c5c152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.359+0000 7f0c61234640 1 --2- 172.21.15.29:0/2516877180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c5c06d210 0x7f0c5c14e2d0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.359+0000 7f0c61234640 1 -- 172.21.15.29:0/2516877180 >> 172.21.15.29:0/2516877180 conn(0x7f0c5c0b1870 msgr2=0x7f0c5c15ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:17.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.359+0000 7f0c61234640 1 -- 172.21.15.29:0/2516877180 shutdown_connections 2024-04-25T00:38:17.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.359+0000 7f0c61234640 1 -- 172.21.15.29:0/2516877180 wait complete. 2024-04-25T00:38:17.377 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:38:17.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f2352c52640 1 Processor -- start 2024-04-25T00:38:17.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f2352c52640 1 -- start start 2024-04-25T00:38:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f2352c52640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f234c0ce4b0 0x7f234c0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f2352c52640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c0cdf60 0x7f234c0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f2352c52640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f234c0cf3c0 con 0x7f234c15ed50 2024-04-25T00:38:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f2352c52640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f234c06c110 con 0x7f234c0ce4b0 2024-04-25T00:38:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f2352c52640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f234c0af480 con 0x7f234c0cdf60 2024-04-25T00:38:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f23511ef640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f234c15ed50 0x7f234c15f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54454/0 (socket says 172.21.15.29:54454) 2024-04-25T00:38:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f23511ef640 1 -- 172.21.15.29:0/50476955 learned_addr learned my addr 172.21.15.29:0/50476955 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f23509ee640 1 --2- 172.21.15.29:0/50476955 >> v2:172.21.15.157:3300/0 conn(0x7f234c0ce4b0 0x7f234c0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f234bfff640 1 --2- 172.21.15.29:0/50476955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c0cdf60 0x7f234c0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f234b7fe640 1 -- 172.21.15.29:0/50476955 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 561827933 0 0) 0x7f234c0cf3c0 con 0x7f234c15ed50 2024-04-25T00:38:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f234b7fe640 1 -- 172.21.15.29:0/50476955 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f233c003530 con 0x7f234c15ed50 2024-04-25T00:38:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f234b7fe640 1 -- 172.21.15.29:0/50476955 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2058224427 0 0) 0x7f233c003530 con 0x7f234c15ed50 2024-04-25T00:38:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f234bfff640 1 -- 172.21.15.29:0/50476955 >> v2:172.21.15.157:3300/0 conn(0x7f234c0ce4b0 msgr2=0x7f234c0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f234bfff640 1 --2- 172.21.15.29:0/50476955 >> v2:172.21.15.157:3300/0 conn(0x7f234c0ce4b0 0x7f234c0ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f234bfff640 1 -- 172.21.15.29:0/50476955 >> v1:172.21.15.29:6789/0 conn(0x7f234c15ed50 legacy=0x7f234c15f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f234bfff640 1 -- 172.21.15.29:0/50476955 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c074340 con 0x7f234c0cdf60 2024-04-25T00:38:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.499+0000 7f23509ee640 1 --2- 172.21.15.29:0/50476955 >> v2:172.21.15.157:3300/0 conn(0x7f234c0ce4b0 0x7f234c0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:38:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f234bfff640 1 --2- 172.21.15.29:0/50476955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c0cdf60 0x7f234c0cedd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f23380097c0 tx=0x7f233801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a6c7530235314df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f234b7fe640 1 -- 172.21.15.29:0/50476955 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23380b0070 con 0x7f234c0cdf60 2024-04-25T00:38:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f234b7fe640 1 -- 172.21.15.29:0/50476955 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23380ab400 con 0x7f234c0cdf60 2024-04-25T00:38:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f234b7fe640 1 -- 172.21.15.29:0/50476955 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344001a00 con 0x7f234c0cdf60 2024-04-25T00:38:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 -- 172.21.15.29:0/50476955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c0cdf60 msgr2=0x7f234c0cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 --2- 172.21.15.29:0/50476955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c0cdf60 0x7f234c0cedd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f23380097c0 tx=0x7f233801c330 comp rx=0 tx=0).stop 2024-04-25T00:38:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 -- 172.21.15.29:0/50476955 shutdown_connections 2024-04-25T00:38:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 --2- 172.21.15.29:0/50476955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c0cdf60 0x7f234c0cedd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 --2- 172.21.15.29:0/50476955 >> v2:172.21.15.157:3300/0 conn(0x7f234c0ce4b0 0x7f234c0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 -- 172.21.15.29:0/50476955 >> 172.21.15.29:0/50476955 conn(0x7f234c0b1ec0 msgr2=0x7f234c0b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 -- 172.21.15.29:0/50476955 shutdown_connections 2024-04-25T00:38:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 -- 172.21.15.29:0/50476955 wait complete. 2024-04-25T00:38:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 Processor -- start 2024-04-25T00:38:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 -- start start 2024-04-25T00:38:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f234c0cdf60 0x7f234c165600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c0ce4b0 0x7f234c0992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c15ed50 0x7f234c156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f234c15c680 con 0x7f234c0cdf60 2024-04-25T00:38:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0b2c00 con 0x7f234c15ed50 2024-04-25T00:38:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f2352c52640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f234c164560 con 0x7f234c0ce4b0 2024-04-25T00:38:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f23509ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f234c0cdf60 0x7f234c165600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f23509ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f234c0cdf60 0x7f234c165600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55546/0 (socket says 172.21.15.29:55546) 2024-04-25T00:38:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.503+0000 7f23509ee640 1 -- 172.21.15.29:0/2722418040 learned_addr learned my addr 172.21.15.29:0/2722418040 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23511ef640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c15ed50 0x7f234c156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f234bfff640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c0ce4b0 0x7f234c0992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23509ee640 1 -- 172.21.15.29:0/2722418040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c0ce4b0 msgr2=0x7f234c0992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23509ee640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c0ce4b0 0x7f234c0992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23509ee640 1 -- 172.21.15.29:0/2722418040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c15ed50 msgr2=0x7f234c156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23509ee640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c15ed50 0x7f234c156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23509ee640 1 -- 172.21.15.29:0/2722418040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c164840 con 0x7f234c0cdf60 2024-04-25T00:38:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23511ef640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c15ed50 0x7f234c156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:17.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f234bfff640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c0ce4b0 0x7f234c0992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:17.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23509ee640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f234c0cdf60 0x7f234c165600 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f234000ebe0 tx=0x7f234000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:17.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23497fa640 1 -- 172.21.15.29:0/2722418040 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340017070 con 0x7f234c0cdf60 2024-04-25T00:38:17.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23497fa640 1 -- 172.21.15.29:0/2722418040 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234000cc30 con 0x7f234c0cdf60 2024-04-25T00:38:17.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f2352c52640 1 -- 172.21.15.29:0/2722418040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234c164b20 con 0x7f234c0cdf60 2024-04-25T00:38:17.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23497fa640 1 -- 172.21.15.29:0/2722418040 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340013470 con 0x7f234c0cdf60 2024-04-25T00:38:17.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f2352c52640 1 -- 172.21.15.29:0/2722418040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234c1650e0 con 0x7f234c0cdf60 2024-04-25T00:38:17.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f2352c52640 1 -- 172.21.15.29:0/2722418040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f234c069270 con 0x7f234c0cdf60 2024-04-25T00:38:17.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23497fa640 1 -- 172.21.15.29:0/2722418040 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f234001a070 con 0x7f234c0cdf60 2024-04-25T00:38:17.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23497fa640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2330067cb0 0x7f233006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f23497fa640 1 -- 172.21.15.29:0/2722418040 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5737+0+0 (secure 0 0 0) 0x7f2340086620 con 0x7f234c0cdf60 2024-04-25T00:38:17.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f234bfff640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2330067cb0 0x7f233006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.507+0000 7f234bfff640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2330067cb0 0x7f233006a170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f23380ad000 tx=0x7f23380a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:17.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.511+0000 7f23497fa640 1 -- 172.21.15.29:0/2722418040 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2340056c60 con 0x7f234c0cdf60 2024-04-25T00:38:17.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f2352c52640 1 -- 172.21.15.29:0/2722418040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f234c0af130 con 0x7f234c0cdf60 2024-04-25T00:38:17.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f23497fa640 1 -- 172.21.15.29:0/2722418040 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5189 (secure 0 0 0) 0x7f2340056560 con 0x7f234c0cdf60 2024-04-25T00:38:17.709 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:17.709 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1884979435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1884979435},{"type":"v1","addr":"172.21.15.157:6833","nonce":1884979435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1394307674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1394307674},{"type":"v1","addr":"172.21.15.157:6835","nonce":1394307674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/3915943091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3915943091},{"type":"v1","addr":"172.21.15.29:6834","nonce":3915943091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:37:24.327819+0000","modified":"2024-04-25T00:37:27.319963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1444171533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1444171533},{"type":"v1","addr":"172.21.15.29:6835","nonce":1444171533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7896,"qdb_cluster":[7896]},"id":8}]} 2024-04-25T00:38:17.710 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 68 2024-04-25T00:38:17.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f2352c52640 1 -- 172.21.15.29:0/2722418040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2330067cb0 msgr2=0x7f233006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f2352c52640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2330067cb0 0x7f233006a170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f23380ad000 tx=0x7f23380a6000 comp rx=0 tx=0).stop 2024-04-25T00:38:17.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f2352c52640 1 -- 172.21.15.29:0/2722418040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f234c0cdf60 msgr2=0x7f234c165600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f2352c52640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f234c0cdf60 0x7f234c165600 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f234000ebe0 tx=0x7f234000c520 comp rx=0 tx=0).stop 2024-04-25T00:38:17.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f2352c52640 1 -- 172.21.15.29:0/2722418040 shutdown_connections 2024-04-25T00:38:17.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f2352c52640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2330067cb0 0x7f233006a170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f2352c52640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c15ed50 0x7f234c156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f2352c52640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c0ce4b0 0x7f234c0992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f2352c52640 1 --2- 172.21.15.29:0/2722418040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f234c0cdf60 0x7f234c165600 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f2352c52640 1 -- 172.21.15.29:0/2722418040 >> 172.21.15.29:0/2722418040 conn(0x7f234c0b1ec0 msgr2=0x7f234c06e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:17.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f2352c52640 1 -- 172.21.15.29:0/2722418040 shutdown_connections 2024-04-25T00:38:17.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.707+0000 7f2352c52640 1 -- 172.21.15.29:0/2722418040 wait complete. 2024-04-25T00:38:17.727 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.839+0000 7f20f1d22640 1 Processor -- start 2024-04-25T00:38:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.839+0000 7f20f1d22640 1 -- start start 2024-04-25T00:38:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.839+0000 7f20f1d22640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f20ec0b3c30 0x7f20ec06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.839+0000 7f20f1d22640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20ec0b41e0 con 0x7f20ec0b4b10 2024-04-25T00:38:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.839+0000 7f20f1d22640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20ec06d960 con 0x7f20ec0b3140 2024-04-25T00:38:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.839+0000 7f20f1d22640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f20ec06dc40 con 0x7f20ec0b3c30 2024-04-25T00:38:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.839+0000 7f20ebfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f20ec0b4b10 0x7f20ec0ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55968/0 (socket says 172.21.15.29:55968) 2024-04-25T00:38:17.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.839+0000 7f20ebfff640 1 -- 172.21.15.29:0/4067486082 learned_addr learned my addr 172.21.15.29:0/4067486082 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:17.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3769094838 0 0) 0x7f20ec0b41e0 con 0x7f20ec0b4b10 2024-04-25T00:38:17.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20d8003530 con 0x7f20ec0b4b10 2024-04-25T00:38:17.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20e3fff640 1 --2- 172.21.15.29:0/4067486082 >> v2:172.21.15.157:3300/0 conn(0x7f20ec0b3c30 0x7f20ec06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 390346068 0 0) 0x7f20d8003530 con 0x7f20ec0b4b10 2024-04-25T00:38:17.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 >> v2:172.21.15.157:3300/0 conn(0x7f20ec0b3c30 msgr2=0x7f20ec06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 --2- 172.21.15.29:0/4067486082 >> v2:172.21.15.157:3300/0 conn(0x7f20ec0b3c30 0x7f20ec06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 >> v1:172.21.15.157:6789/0 conn(0x7f20ec0b3140 legacy=0x7f20ec0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec06df20 con 0x7f20ec0b4b10 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f20e4001a00 con 0x7f20ec0b4b10 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 >> v1:172.21.15.29:6789/0 conn(0x7f20ec0b4b10 legacy=0x7f20ec0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 --2- 172.21.15.29:0/4067486082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20d80039f0 0x7f20d8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 --2- 172.21.15.29:0/4067486082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d8004810 0x7f20d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 --2- 172.21.15.29:0/4067486082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d80051d0 0x7f20d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec06dc40 con 0x7f20d8004810 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20d800c0a0 con 0x7f20d80051d0 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f20d800c2a0 con 0x7f20d80039f0 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20e3fff640 1 --2- 172.21.15.29:0/4067486082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d8004810 0x7f20d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20e3fff640 1 -- 172.21.15.29:0/4067486082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20d80039f0 msgr2=0x7f20d8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20e3fff640 1 --2- 172.21.15.29:0/4067486082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20d80039f0 0x7f20d8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20e3fff640 1 -- 172.21.15.29:0/4067486082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d80051d0 msgr2=0x7f20d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:17.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20e3fff640 1 --2- 172.21.15.29:0/4067486082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d80051d0 0x7f20d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20e3fff640 1 -- 172.21.15.29:0/4067486082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d8003530 con 0x7f20d8004810 2024-04-25T00:38:17.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20ebfff640 1 -- 172.21.15.29:0/4067486082 reap_dead start 2024-04-25T00:38:17.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20e3fff640 1 --2- 172.21.15.29:0/4067486082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d8004810 0x7f20d8004c90 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009ae0 tx=0x7f20d401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d67dd6c065c87c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:17.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d40b2070 con 0x7f20d8004810 2024-04-25T00:38:17.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d40a5680 con 0x7f20d8004810 2024-04-25T00:38:17.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20eaffd640 1 -- 172.21.15.29:0/4067486082 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d40096b0 con 0x7f20d8004810 2024-04-25T00:38:17.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20f1d22640 1 -- 172.21.15.29:0/4067486082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d8004810 msgr2=0x7f20d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20f1d22640 1 --2- 172.21.15.29:0/4067486082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d8004810 0x7f20d8004c90 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009ae0 tx=0x7f20d401ca90 comp rx=0 tx=0).stop 2024-04-25T00:38:17.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20f1d22640 1 -- 172.21.15.29:0/4067486082 shutdown_connections 2024-04-25T00:38:17.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20f1d22640 1 --2- 172.21.15.29:0/4067486082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d8004810 0x7f20d8004c90 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20f1d22640 1 -- 172.21.15.29:0/4067486082 >> 172.21.15.29:0/4067486082 conn(0x7f20ec0b1870 msgr2=0x7f20ec0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:17.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20f1d22640 1 -- 172.21.15.29:0/4067486082 shutdown_connections 2024-04-25T00:38:17.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.843+0000 7f20f1d22640 1 -- 172.21.15.29:0/4067486082 wait complete. 2024-04-25T00:38:17.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20f1d22640 1 Processor -- start 2024-04-25T00:38:17.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20f1d22640 1 -- start start 2024-04-25T00:38:17.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20f1d22640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d8004810 0x7f20ec158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20f1d22640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20ec13c0f0 0x7f20ec1e4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20f1d22640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20ec1e4b80 0x7f20ec0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20f1d22640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec145a90 con 0x7f20d8004810 2024-04-25T00:38:17.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20f1d22640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec145910 con 0x7f20ec1e4b80 2024-04-25T00:38:17.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20f1d22640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec06e440 con 0x7f20ec13c0f0 2024-04-25T00:38:17.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20eb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d8004810 0x7f20ec158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20eb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d8004810 0x7f20ec158820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51276/0 (socket says 172.21.15.29:51276) 2024-04-25T00:38:17.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20eb7fe640 1 -- 172.21.15.29:0/958120177 learned_addr learned my addr 172.21.15.29:0/958120177 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:17.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20e3fff640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20ec13c0f0 0x7f20ec1e4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20eb7fe640 1 -- 172.21.15.29:0/958120177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20ec13c0f0 msgr2=0x7f20ec1e4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:17.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20eb7fe640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20ec13c0f0 0x7f20ec1e4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20eb7fe640 1 -- 172.21.15.29:0/958120177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20ec1e4b80 msgr2=0x7f20ec0ce7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:38:17.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20eb7fe640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20ec1e4b80 0x7f20ec0ce7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:17.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20eb7fe640 1 -- 172.21.15.29:0/958120177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec157780 con 0x7f20d8004810 2024-04-25T00:38:17.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20e3fff640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20ec13c0f0 0x7f20ec1e4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:17.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20eb7fe640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d8004810 0x7f20ec158820 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00c3c0 tx=0x7f20dc001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:17.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20e8ff9640 1 -- 172.21.15.29:0/958120177 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc017070 con 0x7f20d8004810 2024-04-25T00:38:17.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20e8ff9640 1 -- 172.21.15.29:0/958120177 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20dc0052f0 con 0x7f20d8004810 2024-04-25T00:38:17.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20e8ff9640 1 -- 172.21.15.29:0/958120177 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc003840 con 0x7f20d8004810 2024-04-25T00:38:17.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20f1d22640 1 -- 172.21.15.29:0/958120177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ec157a60 con 0x7f20d8004810 2024-04-25T00:38:17.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20f1d22640 1 -- 172.21.15.29:0/958120177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ec158020 con 0x7f20d8004810 2024-04-25T00:38:17.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20f1d22640 1 -- 172.21.15.29:0/958120177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20ec069270 con 0x7f20d8004810 2024-04-25T00:38:17.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.847+0000 7f20e8ff9640 1 -- 172.21.15.29:0/958120177 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f20dc01a070 con 0x7f20d8004810 2024-04-25T00:38:17.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.851+0000 7f20e8ff9640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f20cc067cb0 0x7f20cc06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:17.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.851+0000 7f20e8ff9640 1 -- 172.21.15.29:0/958120177 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5737+0+0 (secure 0 0 0) 0x7f20dc085600 con 0x7f20d8004810 2024-04-25T00:38:17.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.851+0000 7f20e3fff640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f20cc067cb0 0x7f20cc06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:17.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.851+0000 7f20e3fff640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f20cc067cb0 0x7f20cc06a170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f20d40af000 tx=0x7f20d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:17.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:17.855+0000 7f20e8ff9640 1 -- 172.21.15.29:0/958120177 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f20dc055cd0 con 0x7f20d8004810 2024-04-25T00:38:18.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.015+0000 7f20f1d22640 1 -- 172.21.15.29:0/958120177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20ec0753e0 con 0x7f20d8004810 2024-04-25T00:38:18.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.015+0000 7f20e8ff9640 1 -- 172.21.15.29:0/958120177 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16646 (secure 0 0 0) 0x7f20dc0555d0 con 0x7f20d8004810 2024-04-25T00:38:18.019 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:18.019 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":106,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:17.454194+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:37:22.602795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:37:23.310674+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:18.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.019+0000 7f20f1d22640 1 -- 172.21.15.29:0/958120177 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f20cc067cb0 msgr2=0x7f20cc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.019+0000 7f20f1d22640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f20cc067cb0 0x7f20cc06a170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f20d40af000 tx=0x7f20d4008040 comp rx=0 tx=0).stop 2024-04-25T00:38:18.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.019+0000 7f20f1d22640 1 -- 172.21.15.29:0/958120177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d8004810 msgr2=0x7f20ec158820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.019+0000 7f20f1d22640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d8004810 0x7f20ec158820 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00c3c0 tx=0x7f20dc001e80 comp rx=0 tx=0).stop 2024-04-25T00:38:18.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.019+0000 7f20f1d22640 1 -- 172.21.15.29:0/958120177 shutdown_connections 2024-04-25T00:38:18.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.019+0000 7f20f1d22640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f20cc067cb0 0x7f20cc06a170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.019+0000 7f20f1d22640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20ec1e4b80 0x7f20ec0ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.019+0000 7f20f1d22640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20ec13c0f0 0x7f20ec1e4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.019+0000 7f20f1d22640 1 --2- 172.21.15.29:0/958120177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d8004810 0x7f20ec158820 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.019+0000 7f20f1d22640 1 -- 172.21.15.29:0/958120177 >> 172.21.15.29:0/958120177 conn(0x7f20ec0b1870 msgr2=0x7f20ec144b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:18.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.019+0000 7f20f1d22640 1 -- 172.21.15.29:0/958120177 shutdown_connections 2024-04-25T00:38:18.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.019+0000 7f20f1d22640 1 -- 172.21.15.29:0/958120177 wait complete. 2024-04-25T00:38:18.039 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:38:18.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a77e0640 1 Processor -- start 2024-04-25T00:38:18.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a77e0640 1 -- start start 2024-04-25T00:38:18.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a77e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a00cca20 0x7ff9a0162720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:18.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a77e0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a00d06c0 con 0x7ff9a006b920 2024-04-25T00:38:18.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a77e0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a0161c40 con 0x7ff9a00d01a0 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a77e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a0161f20 con 0x7ff9a00cca20 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a5d7d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff9a006b920 0x7ff9a006dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55976/0 (socket says 172.21.15.29:55976) 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a5d7d640 1 -- 172.21.15.29:0/1215724788 learned_addr learned my addr 172.21.15.29:0/1215724788 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a557c640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a00cca20 0x7ff9a0162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1228306299 0 0) 0x7ff9a00d06c0 con 0x7ff9a006b920 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff98c003530 con 0x7ff9a006b920 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 512698699 0 0) 0x7ff98c003530 con 0x7ff9a006b920 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a00cca20 msgr2=0x7ff9a0162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a00cca20 0x7ff9a0162720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 >> v1:172.21.15.157:6789/0 conn(0x7ff9a00d01a0 legacy=0x7ff9a006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a0162200 con 0x7ff9a006b920 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ff99c001a00 con 0x7ff9a006b920 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 >> v1:172.21.15.29:6789/0 conn(0x7ff9a006b920 legacy=0x7ff9a006dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff98c0039f0 0x7ff98c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:18.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a00cca20 0x7ff98c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:18.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff98c004d50 0x7ff98c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:18.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0161f20 con 0x7ff98c0039f0 2024-04-25T00:38:18.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c00bb00 con 0x7ff98c004d50 2024-04-25T00:38:18.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff98c00bd00 con 0x7ff9a00cca20 2024-04-25T00:38:18.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a557c640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff98c0039f0 0x7ff98c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:18.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a5d7d640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff98c004d50 0x7ff98c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:18.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a4d7b640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a00cca20 0x7ff98c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:18.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a557c640 1 -- 172.21.15.29:0/1215724788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a00cca20 msgr2=0x7ff98c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:18.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a557c640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a00cca20 0x7ff98c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a557c640 1 -- 172.21.15.29:0/1215724788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff98c004d50 msgr2=0x7ff98c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:18.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a557c640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff98c004d50 0x7ff98c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a557c640 1 -- 172.21.15.29:0/1215724788 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98c003530 con 0x7ff98c0039f0 2024-04-25T00:38:18.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a4d7b640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a00cca20 0x7ff98c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:18.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.163+0000 7ff9a5d7d640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff98c004d50 0x7ff98c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:18.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.167+0000 7ff9a557c640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff98c0039f0 0x7ff98c003e10 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff998009ac0 tx=0x7ff99801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e4aca890777995 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:18.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.167+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998009690 con 0x7ff98c0039f0 2024-04-25T00:38:18.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.167+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9980a6490 con 0x7ff98c0039f0 2024-04-25T00:38:18.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.167+0000 7ff997fff640 1 -- 172.21.15.29:0/1215724788 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9980a6860 con 0x7ff98c0039f0 2024-04-25T00:38:18.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.167+0000 7ff9a77e0640 1 -- 172.21.15.29:0/1215724788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff98c0039f0 msgr2=0x7ff98c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.167+0000 7ff9a77e0640 1 --2- 172.21.15.29:0/1215724788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff98c0039f0 0x7ff98c003e10 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff998009ac0 tx=0x7ff99801c940 comp rx=0 tx=0).stop 2024-04-25T00:38:18.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.167+0000 7ff9a5d7d640 1 -- 172.21.15.29:0/1215724788 reap_dead start 2024-04-25T00:38:18.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.167+0000 7ff9a77e0640 1 -- 172.21.15.29:0/1215724788 shutdown_connections 2024-04-25T00:38:18.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.167+0000 7ff9a77e0640 1 -- 172.21.15.29:0/1215724788 >> 172.21.15.29:0/1215724788 conn(0x7ff9a006c320 msgr2=0x7ff9a006c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:18.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.167+0000 7ff9a77e0640 1 -- 172.21.15.29:0/1215724788 shutdown_connections 2024-04-25T00:38:18.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.167+0000 7ff9a77e0640 1 -- 172.21.15.29:0/1215724788 wait complete. 2024-04-25T00:38:18.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.167+0000 7ff9a77e0640 1 Processor -- start 2024-04-25T00:38:18.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.167+0000 7ff9a77e0640 1 -- start start 2024-04-25T00:38:18.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a77e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff98c0039f0 0x7ff9a015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:18.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a77e0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9a00b18c0 0x7ff9a00b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:18.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a77e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9a00c9d70 0x7ff9a00ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:18.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a77e0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a006e5a0 con 0x7ff9a00b18c0 2024-04-25T00:38:18.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a77e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a006e420 con 0x7ff9a00c9d70 2024-04-25T00:38:18.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a77e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a006e720 con 0x7ff98c0039f0 2024-04-25T00:38:18.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a4d7b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9a00b18c0 0x7ff9a00b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:18.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a4d7b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9a00b18c0 0x7ff9a00b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51294/0 (socket says 172.21.15.29:51294) 2024-04-25T00:38:18.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a4d7b640 1 -- 172.21.15.29:0/3347398638 learned_addr learned my addr 172.21.15.29:0/3347398638 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:18.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a5d7d640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9a00c9d70 0x7ff9a00ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:18.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a4d7b640 1 -- 172.21.15.29:0/3347398638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff98c0039f0 msgr2=0x7ff9a015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a4d7b640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff98c0039f0 0x7ff9a015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a4d7b640 1 -- 172.21.15.29:0/3347398638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9a00c9d70 msgr2=0x7ff9a00ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a4d7b640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9a00c9d70 0x7ff9a00ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a4d7b640 1 -- 172.21.15.29:0/3347398638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a00ca980 con 0x7ff9a00b18c0 2024-04-25T00:38:18.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a5d7d640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9a00c9d70 0x7ff9a00ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:18.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a4d7b640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9a00b18c0 0x7ff9a00b1d20 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff990000c10 tx=0x7ff99000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:18.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff995ffb640 1 -- 172.21.15.29:0/3347398638 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990019070 con 0x7ff9a00b18c0 2024-04-25T00:38:18.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff995ffb640 1 -- 172.21.15.29:0/3347398638 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff990005270 con 0x7ff9a00b18c0 2024-04-25T00:38:18.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a77e0640 1 -- 172.21.15.29:0/3347398638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a00c7ee0 con 0x7ff9a00b18c0 2024-04-25T00:38:18.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff995ffb640 1 -- 172.21.15.29:0/3347398638 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990007030 con 0x7ff9a00b18c0 2024-04-25T00:38:18.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a77e0640 1 -- 172.21.15.29:0/3347398638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a00c84a0 con 0x7ff9a00b18c0 2024-04-25T00:38:18.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a77e0640 1 -- 172.21.15.29:0/3347398638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a00b2260 con 0x7ff9a00b18c0 2024-04-25T00:38:18.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff995ffb640 1 -- 172.21.15.29:0/3347398638 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7ff99001c070 con 0x7ff9a00b18c0 2024-04-25T00:38:18.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff995ffb640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff984067ce0 0x7ff98406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:18.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a557c640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff984067ce0 0x7ff98406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:18.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff995ffb640 1 -- 172.21.15.29:0/3347398638 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5737+0+0 (secure 0 0 0) 0x7ff990085d50 con 0x7ff9a00b18c0 2024-04-25T00:38:18.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.171+0000 7ff9a557c640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff984067ce0 0x7ff98406a1a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff9980ad000 tx=0x7ff998004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:18.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.175+0000 7ff995ffb640 1 -- 172.21.15.29:0/3347398638 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff9900560f0 con 0x7ff9a00b18c0 2024-04-25T00:38:18.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.375+0000 7ff9a77e0640 1 -- 172.21.15.29:0/3347398638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff9a0069270 con 0x7ff9a00b18c0 2024-04-25T00:38:18.377 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-04-25T00:38:18.375+0000 7f3ac00b5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:38:18.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.379+0000 7ff995ffb640 1 -- 172.21.15.29:0/3347398638 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) ==== 161+0+0 (secure 0 0 0) 0x7ff9900559f0 con 0x7ff9a00b18c0 2024-04-25T00:38:18.380 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:38:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.379+0000 7ff9a77e0640 1 -- 172.21.15.29:0/3347398638 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff984067ce0 msgr2=0x7ff98406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.379+0000 7ff9a77e0640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff984067ce0 0x7ff98406a1a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff9980ad000 tx=0x7ff998004740 comp rx=0 tx=0).stop 2024-04-25T00:38:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.379+0000 7ff9a77e0640 1 -- 172.21.15.29:0/3347398638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9a00b18c0 msgr2=0x7ff9a00b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.379+0000 7ff9a77e0640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9a00b18c0 0x7ff9a00b1d20 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff990000c10 tx=0x7ff99000dc00 comp rx=0 tx=0).stop 2024-04-25T00:38:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.383+0000 7ff9a77e0640 1 -- 172.21.15.29:0/3347398638 shutdown_connections 2024-04-25T00:38:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.383+0000 7ff9a77e0640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff984067ce0 0x7ff98406a1a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.383+0000 7ff9a77e0640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9a00c9d70 0x7ff9a00ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.383+0000 7ff9a77e0640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9a00b18c0 0x7ff9a00b1d20 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.383+0000 7ff9a77e0640 1 --2- 172.21.15.29:0/3347398638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff98c0039f0 0x7ff9a015a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.383+0000 7ff9a77e0640 1 -- 172.21.15.29:0/3347398638 >> 172.21.15.29:0/3347398638 conn(0x7ff9a006c320 msgr2=0x7ff9a006c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:18.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.383+0000 7ff9a77e0640 1 -- 172.21.15.29:0/3347398638 shutdown_connections 2024-04-25T00:38:18.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.383+0000 7ff9a77e0640 1 -- 172.21.15.29:0/3347398638 wait complete. 2024-04-25T00:38:18.407 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:38:18.431 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:38:18.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.531+0000 7f10501d6640 1 Processor -- start 2024-04-25T00:38:18.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.531+0000 7f10501d6640 1 -- start start 2024-04-25T00:38:18.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.531+0000 7f10501d6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f104815f3c0 0x7f10480cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:18.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.531+0000 7f10501d6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10480b3720 con 0x7f10480b4780 2024-04-25T00:38:18.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.531+0000 7f10501d6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10480b39c0 con 0x7f104815e8d0 2024-04-25T00:38:18.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.531+0000 7f10501d6640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f10480b42f0 con 0x7f104815f3c0 2024-04-25T00:38:18.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.531+0000 7f104d771640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f10480b4780 0x7f104815c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55998/0 (socket says 172.21.15.29:55998) 2024-04-25T00:38:18.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.531+0000 7f104d771640 1 -- 172.21.15.29:0/4080002513 learned_addr learned my addr 172.21.15.29:0/4080002513 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:18.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.531+0000 7f104e773640 1 --2- 172.21.15.29:0/4080002513 >> v2:172.21.15.29:3300/0 conn(0x7f104815f3c0 0x7f10480cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.531+0000 7f104e773640 1 -- 172.21.15.29:0/4080002513 >> v1:172.21.15.157:6789/0 conn(0x7f104815e8d0 legacy=0x7f104815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.531+0000 7f104e773640 1 -- 172.21.15.29:0/4080002513 >> v1:172.21.15.29:6789/0 conn(0x7f10480b4780 legacy=0x7f104815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.531+0000 7f104cf70640 1 -- 172.21.15.29:0/4080002513 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1806108986 0 0) 0x7f10480b3720 con 0x7f10480b4780 2024-04-25T00:38:18.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.531+0000 7f104e773640 1 -- 172.21.15.29:0/4080002513 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10480b3ca0 con 0x7f104815f3c0 2024-04-25T00:38:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f104e773640 1 --2- 172.21.15.29:0/4080002513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f104815f3c0 0x7f10480cc240 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f1040009d90 tx=0x7f104001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e38174f503a87e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f104cf70640 1 -- 172.21.15.29:0/4080002513 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10400b2070 con 0x7f104815f3c0 2024-04-25T00:38:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f104cf70640 1 -- 172.21.15.29:0/4080002513 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10400a5900 con 0x7f104815f3c0 2024-04-25T00:38:18.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f104cf70640 1 -- 172.21.15.29:0/4080002513 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10400a5cd0 con 0x7f104815f3c0 2024-04-25T00:38:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 -- 172.21.15.29:0/4080002513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f104815f3c0 msgr2=0x7f10480cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 --2- 172.21.15.29:0/4080002513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f104815f3c0 0x7f10480cc240 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f1040009d90 tx=0x7f104001c970 comp rx=0 tx=0).stop 2024-04-25T00:38:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 -- 172.21.15.29:0/4080002513 shutdown_connections 2024-04-25T00:38:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 --2- 172.21.15.29:0/4080002513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f104815f3c0 0x7f10480cc240 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 -- 172.21.15.29:0/4080002513 >> 172.21.15.29:0/4080002513 conn(0x7f10480b1870 msgr2=0x7f10480b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:18.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 -- 172.21.15.29:0/4080002513 shutdown_connections 2024-04-25T00:38:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 -- 172.21.15.29:0/4080002513 wait complete. 2024-04-25T00:38:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 Processor -- start 2024-04-25T00:38:18.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 -- start start 2024-04-25T00:38:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10480b4780 0x7f10481394e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104815e8d0 0x7f10480aa070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f104815f3c0 0x7f10480aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1048168e00 con 0x7f104815f3c0 2024-04-25T00:38:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1048168c80 con 0x7f104815e8d0 2024-04-25T00:38:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f10501d6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1048168f80 con 0x7f10480b4780 2024-04-25T00:38:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f104e773640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f104815f3c0 0x7f10480aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f104e773640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f104815f3c0 0x7f10480aa5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51340/0 (socket says 172.21.15.29:51340) 2024-04-25T00:38:18.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f104e773640 1 -- 172.21.15.29:0/2416121563 learned_addr learned my addr 172.21.15.29:0/2416121563 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f104e773640 1 -- 172.21.15.29:0/2416121563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10480b4780 msgr2=0x7f10481394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f104e773640 1 --2- 172.21.15.29:0/2416121563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10480b4780 0x7f10481394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f104e773640 1 -- 172.21.15.29:0/2416121563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104815e8d0 msgr2=0x7f10480aa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f104e773640 1 --2- 172.21.15.29:0/2416121563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104815e8d0 0x7f10480aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.535+0000 7f104e773640 1 -- 172.21.15.29:0/2416121563 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1048071b60 con 0x7f104815f3c0 2024-04-25T00:38:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.539+0000 7f104e773640 1 --2- 172.21.15.29:0/2416121563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f104815f3c0 0x7f10480aa5b0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f1040009d90 tx=0x7f1040019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.539+0000 7f103e7fc640 1 -- 172.21.15.29:0/2416121563 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10400b2040 con 0x7f104815f3c0 2024-04-25T00:38:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.539+0000 7f103e7fc640 1 -- 172.21.15.29:0/2416121563 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1040003ba0 con 0x7f104815f3c0 2024-04-25T00:38:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.539+0000 7f10501d6640 1 -- 172.21.15.29:0/2416121563 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1048071e40 con 0x7f104815f3c0 2024-04-25T00:38:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.539+0000 7f103e7fc640 1 -- 172.21.15.29:0/2416121563 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040004030 con 0x7f104815f3c0 2024-04-25T00:38:18.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.539+0000 7f10501d6640 1 -- 172.21.15.29:0/2416121563 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1048072400 con 0x7f104815f3c0 2024-04-25T00:38:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.539+0000 7f10501d6640 1 -- 172.21.15.29:0/2416121563 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1048072830 con 0x7f104815f3c0 2024-04-25T00:38:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.539+0000 7f103e7fc640 1 -- 172.21.15.29:0/2416121563 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f10400041d0 con 0x7f104815f3c0 2024-04-25T00:38:18.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.539+0000 7f103e7fc640 1 --2- 172.21.15.29:0/2416121563 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f102c067cb0 0x7f102c06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.539+0000 7f103e7fc640 1 -- 172.21.15.29:0/2416121563 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5795+0+0 (secure 0 0 0) 0x7f1040120e70 con 0x7f104815f3c0 2024-04-25T00:38:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.539+0000 7f104df72640 1 --2- 172.21.15.29:0/2416121563 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f102c067cb0 0x7f102c06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:18.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.539+0000 7f104df72640 1 --2- 172.21.15.29:0/2416121563 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f102c067cb0 0x7f102c06a170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1044000f70 tx=0x7f10440079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:18.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.543+0000 7f103e7fc640 1 -- 172.21.15.29:0/2416121563 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1040016560 con 0x7f104815f3c0 2024-04-25T00:38:18.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:18.739+0000 7f10501d6640 1 -- 172.21.15.29:0/2416121563 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1048069250 con 0x7f104815f3c0 2024-04-25T00:38:19.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.379+0000 7f103e7fc640 1 -- 172.21.15.29:0/2416121563 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) ==== 104+0+0 (secure 0 0 0) 0x7f10400f7030 con 0x7f104815f3c0 2024-04-25T00:38:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.383+0000 7f10501d6640 1 -- 172.21.15.29:0/2416121563 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f102c067cb0 msgr2=0x7f102c06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.383+0000 7f10501d6640 1 --2- 172.21.15.29:0/2416121563 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f102c067cb0 0x7f102c06a170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1044000f70 tx=0x7f10440079c0 comp rx=0 tx=0).stop 2024-04-25T00:38:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.383+0000 7f10501d6640 1 -- 172.21.15.29:0/2416121563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f104815f3c0 msgr2=0x7f10480aa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.383+0000 7f10501d6640 1 --2- 172.21.15.29:0/2416121563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f104815f3c0 0x7f10480aa5b0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f1040009d90 tx=0x7f1040019710 comp rx=0 tx=0).stop 2024-04-25T00:38:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.383+0000 7f10501d6640 1 -- 172.21.15.29:0/2416121563 shutdown_connections 2024-04-25T00:38:19.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.383+0000 7f10501d6640 1 --2- 172.21.15.29:0/2416121563 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f102c067cb0 0x7f102c06a170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:19.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.383+0000 7f10501d6640 1 --2- 172.21.15.29:0/2416121563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f104815f3c0 0x7f10480aa5b0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:19.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.383+0000 7f10501d6640 1 --2- 172.21.15.29:0/2416121563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104815e8d0 0x7f10480aa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:19.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.383+0000 7f10501d6640 1 --2- 172.21.15.29:0/2416121563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10480b4780 0x7f10481394e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:19.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.383+0000 7f10501d6640 1 -- 172.21.15.29:0/2416121563 >> 172.21.15.29:0/2416121563 conn(0x7f10480b1870 msgr2=0x7f104815c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:19.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.383+0000 7f10501d6640 1 -- 172.21.15.29:0/2416121563 shutdown_connections 2024-04-25T00:38:19.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.383+0000 7f10501d6640 1 -- 172.21.15.29:0/2416121563 wait complete. 2024-04-25T00:38:19.411 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:38:19.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.583+0000 7f90b94dd640 1 Processor -- start 2024-04-25T00:38:19.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.583+0000 7f90b94dd640 1 -- start start 2024-04-25T00:38:19.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b94dd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b40d0aa0 0x7f90b40b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:19.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b94dd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90b40741a0 0x7f90b40b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:19.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b94dd640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b4161bb0 con 0x7f90b41472a0 2024-04-25T00:38:19.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b94dd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f90b40d0e80 con 0x7f90b40741a0 2024-04-25T00:38:19.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b94dd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f90b4161e90 con 0x7f90b40d0aa0 2024-04-25T00:38:19.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b37fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f90b41472a0 0x7f90b40715f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56008/0 (socket says 172.21.15.29:56008) 2024-04-25T00:38:19.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b37fe640 1 -- 172.21.15.29:0/1622735484 learned_addr learned my addr 172.21.15.29:0/1622735484 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:19.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b2ffd640 1 --2- 172.21.15.29:0/1622735484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b40d0aa0 0x7f90b40b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:19.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b27fc640 1 --2- 172.21.15.29:0/1622735484 >> v2:172.21.15.157:3300/0 conn(0x7f90b40741a0 0x7f90b40b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:19.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1762910344 0 0) 0x7f90b4161bb0 con 0x7f90b41472a0 2024-04-25T00:38:19.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f909c003530 con 0x7f90b41472a0 2024-04-25T00:38:19.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3959059796 0 0) 0x7f909c003530 con 0x7f90b41472a0 2024-04-25T00:38:19.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b40d0aa0 msgr2=0x7f90b40b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:19.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 --2- 172.21.15.29:0/1622735484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b40d0aa0 0x7f90b40b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:19.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 >> v2:172.21.15.157:3300/0 conn(0x7f90b40741a0 msgr2=0x7f90b40b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:19.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 --2- 172.21.15.29:0/1622735484 >> v2:172.21.15.157:3300/0 conn(0x7f90b40741a0 0x7f90b40b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:19.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b4162170 con 0x7f90b41472a0 2024-04-25T00:38:19.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f90ac001a00 con 0x7f90b41472a0 2024-04-25T00:38:19.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 >> v1:172.21.15.29:6789/0 conn(0x7f90b41472a0 legacy=0x7f90b40715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:19.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 --2- 172.21.15.29:0/1622735484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c0039c0 0x7f909c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:19.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 --2- 172.21.15.29:0/1622735484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f909c004810 0x7f909c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:19.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 --2- 172.21.15.29:0/1622735484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b40d0aa0 0x7f909c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:19.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f90b40d0e80 con 0x7f909c0039c0 2024-04-25T00:38:19.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90b4161e90 con 0x7f909c004810 2024-04-25T00:38:19.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f909c00b9d0 con 0x7f90b40d0aa0 2024-04-25T00:38:19.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b2ffd640 1 --2- 172.21.15.29:0/1622735484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c0039c0 0x7f909c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:19.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b27fc640 1 --2- 172.21.15.29:0/1622735484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f909c004810 0x7f909c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:19.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b2ffd640 1 -- 172.21.15.29:0/1622735484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b40d0aa0 msgr2=0x7f909c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:19.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b2ffd640 1 --2- 172.21.15.29:0/1622735484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b40d0aa0 0x7f909c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:19.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b2ffd640 1 -- 172.21.15.29:0/1622735484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f909c004810 msgr2=0x7f909c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:19.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b2ffd640 1 --2- 172.21.15.29:0/1622735484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f909c004810 0x7f909c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:19.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b2ffd640 1 -- 172.21.15.29:0/1622735484 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909c003530 con 0x7f909c0039c0 2024-04-25T00:38:19.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.587+0000 7f90b27fc640 1 --2- 172.21.15.29:0/1622735484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f909c004810 0x7f909c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:19.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b2ffd640 1 --2- 172.21.15.29:0/1622735484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c0039c0 0x7f909c003de0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009b10 tx=0x7f90a401ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c23d6a319656e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:19.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a40b2070 con 0x7f909c0039c0 2024-04-25T00:38:19.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a40a5590 con 0x7f909c0039c0 2024-04-25T00:38:19.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b1ffb640 1 -- 172.21.15.29:0/1622735484 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a40096b0 con 0x7f909c0039c0 2024-04-25T00:38:19.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 -- 172.21.15.29:0/1622735484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c0039c0 msgr2=0x7f909c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:19.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 --2- 172.21.15.29:0/1622735484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c0039c0 0x7f909c003de0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009b10 tx=0x7f90a401ca80 comp rx=0 tx=0).stop 2024-04-25T00:38:19.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b37fe640 1 -- 172.21.15.29:0/1622735484 reap_dead start 2024-04-25T00:38:19.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 -- 172.21.15.29:0/1622735484 shutdown_connections 2024-04-25T00:38:19.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 -- 172.21.15.29:0/1622735484 >> 172.21.15.29:0/1622735484 conn(0x7f90b40b20f0 msgr2=0x7f90b40b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:19.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 -- 172.21.15.29:0/1622735484 shutdown_connections 2024-04-25T00:38:19.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 -- 172.21.15.29:0/1622735484 wait complete. 2024-04-25T00:38:19.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 Processor -- start 2024-04-25T00:38:19.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 -- start start 2024-04-25T00:38:19.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f909c0039c0 0x7f90b4166180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:19.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b415d410 0x7f90b413b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:19.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b406b470 0x7f90b415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:19.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f90b4168eb0 con 0x7f90b406b470 2024-04-25T00:38:19.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90b4168d30 con 0x7f90b415d410 2024-04-25T00:38:19.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b94dd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f90b4169030 con 0x7f909c0039c0 2024-04-25T00:38:19.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b406b470 0x7f90b415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:19.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b406b470 0x7f90b415c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51360/0 (socket says 172.21.15.29:51360) 2024-04-25T00:38:19.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b37fe640 1 -- 172.21.15.29:0/1436838296 learned_addr learned my addr 172.21.15.29:0/1436838296 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:19.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b2ffd640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f909c0039c0 0x7f90b4166180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:19.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b27fc640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b415d410 0x7f90b413b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:19.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b37fe640 1 -- 172.21.15.29:0/1436838296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f909c0039c0 msgr2=0x7f90b4166180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:19.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b37fe640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f909c0039c0 0x7f90b4166180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:19.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b37fe640 1 -- 172.21.15.29:0/1436838296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b415d410 msgr2=0x7f90b413b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:19.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b37fe640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b415d410 0x7f90b413b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:19.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.591+0000 7f90b37fe640 1 -- 172.21.15.29:0/1436838296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b4165720 con 0x7f90b406b470 2024-04-25T00:38:19.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.595+0000 7f90b37fe640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b406b470 0x7f90b415c7c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f90b4141710 tx=0x7f90ac00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:19.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.595+0000 7f909b7fe640 1 -- 172.21.15.29:0/1436838296 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac00a5e0 con 0x7f90b406b470 2024-04-25T00:38:19.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.595+0000 7f909b7fe640 1 -- 172.21.15.29:0/1436838296 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ac00a7a0 con 0x7f90b406b470 2024-04-25T00:38:19.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.595+0000 7f909b7fe640 1 -- 172.21.15.29:0/1436838296 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac00ab90 con 0x7f90b406b470 2024-04-25T00:38:19.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.595+0000 7f90b94dd640 1 -- 172.21.15.29:0/1436838296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b4165a00 con 0x7f90b406b470 2024-04-25T00:38:19.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.595+0000 7f90b94dd640 1 -- 172.21.15.29:0/1436838296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b4138840 con 0x7f90b406b470 2024-04-25T00:38:19.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.595+0000 7f90b94dd640 1 -- 172.21.15.29:0/1436838296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b40692e0 con 0x7f90b406b470 2024-04-25T00:38:19.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.595+0000 7f909b7fe640 1 -- 172.21.15.29:0/1436838296 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f90ac00ad30 con 0x7f90b406b470 2024-04-25T00:38:19.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.595+0000 7f909b7fe640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90a0067ce0 0x7f90a006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:19.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.595+0000 7f90b2ffd640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90a0067ce0 0x7f90a006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:19.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.595+0000 7f909b7fe640 1 -- 172.21.15.29:0/1436838296 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5795+0+0 (secure 0 0 0) 0x7f90ac085cc0 con 0x7f90b406b470 2024-04-25T00:38:19.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.595+0000 7f90b2ffd640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90a0067ce0 0x7f90a006a1a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f90a40af000 tx=0x7f90a40046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:19.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.599+0000 7f909b7fe640 1 -- 172.21.15.29:0/1436838296 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f90ac08c080 con 0x7f90b406b470 2024-04-25T00:38:19.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:19.755+0000 7f90b94dd640 1 -- 172.21.15.29:0/1436838296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f90b406bee0 con 0x7f90b406b470 2024-04-25T00:38:20.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.619+0000 7f909b7fe640 1 -- 172.21.15.29:0/1436838296 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7f90ac0561c0 con 0x7f90b406b470 2024-04-25T00:38:20.622 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:38:20.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.623+0000 7f90b94dd640 1 -- 172.21.15.29:0/1436838296 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90a0067ce0 msgr2=0x7f90a006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:20.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.623+0000 7f90b94dd640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90a0067ce0 0x7f90a006a1a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f90a40af000 tx=0x7f90a40046d0 comp rx=0 tx=0).stop 2024-04-25T00:38:20.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.623+0000 7f90b94dd640 1 -- 172.21.15.29:0/1436838296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b406b470 msgr2=0x7f90b415c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:20.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.623+0000 7f90b94dd640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b406b470 0x7f90b415c7c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f90b4141710 tx=0x7f90ac00ccd0 comp rx=0 tx=0).stop 2024-04-25T00:38:20.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.623+0000 7f90b94dd640 1 -- 172.21.15.29:0/1436838296 shutdown_connections 2024-04-25T00:38:20.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.623+0000 7f90b94dd640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90a0067ce0 0x7f90a006a1a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:20.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.623+0000 7f90b94dd640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b406b470 0x7f90b415c7c0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:20.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.623+0000 7f90b94dd640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b415d410 0x7f90b413b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:20.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.623+0000 7f90b94dd640 1 --2- 172.21.15.29:0/1436838296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f909c0039c0 0x7f90b4166180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:20.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.623+0000 7f90b94dd640 1 -- 172.21.15.29:0/1436838296 >> 172.21.15.29:0/1436838296 conn(0x7f90b40b20f0 msgr2=0x7f90b40b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:20.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.627+0000 7f90b94dd640 1 -- 172.21.15.29:0/1436838296 shutdown_connections 2024-04-25T00:38:20.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.627+0000 7f90b94dd640 1 -- 172.21.15.29:0/1436838296 wait complete. 2024-04-25T00:38:20.651 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:38:20.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.779+0000 7fa62a0e9640 1 Processor -- start 2024-04-25T00:38:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.779+0000 7fa62a0e9640 1 -- start start 2024-04-25T00:38:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa62a0e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6240b3b50 0x7fa6240b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa62a0e9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa62406cd40 0x7fa62406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa62a0e9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6240cf670 con 0x7fa6241456d0 2024-04-25T00:38:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa62a0e9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa6240b4380 con 0x7fa62406cd40 2024-04-25T00:38:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa62a0e9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa6240b40a0 con 0x7fa6240b3b50 2024-04-25T00:38:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa623fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa62406cd40 0x7fa62406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa6237fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa6241456d0 0x7fa6240cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56010/0 (socket says 172.21.15.29:56010) 2024-04-25T00:38:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa6237fe640 1 -- 172.21.15.29:0/2518390938 learned_addr learned my addr 172.21.15.29:0/2518390938 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa623fff640 1 -- 172.21.15.29:0/2518390938 >> v2:172.21.15.157:3300/0 conn(0x7fa6240b3b50 msgr2=0x7fa6240b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa623fff640 1 --2- 172.21.15.29:0/2518390938 >> v2:172.21.15.157:3300/0 conn(0x7fa6240b3b50 0x7fa6240b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa623fff640 1 -- 172.21.15.29:0/2518390938 >> v1:172.21.15.29:6789/0 conn(0x7fa6241456d0 legacy=0x7fa6240cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa623fff640 1 -- 172.21.15.29:0/2518390938 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6240ce480 con 0x7fa62406cd40 2024-04-25T00:38:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa623fff640 1 --2- 172.21.15.29:0/2518390938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa62406cd40 0x7fa62406d150 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fa6240cfb50 tx=0x7fa61c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425eaa72a0972746 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa6227fc640 1 -- 172.21.15.29:0/2518390938 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0b2070 con 0x7fa62406cd40 2024-04-25T00:38:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa6227fc640 1 -- 172.21.15.29:0/2518390938 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c0a58e0 con 0x7fa62406cd40 2024-04-25T00:38:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa6227fc640 1 -- 172.21.15.29:0/2518390938 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0a5cb0 con 0x7fa62406cd40 2024-04-25T00:38:20.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa62a0e9640 1 -- 172.21.15.29:0/2518390938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa62406cd40 msgr2=0x7fa62406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:20.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa62a0e9640 1 --2- 172.21.15.29:0/2518390938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa62406cd40 0x7fa62406d150 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fa6240cfb50 tx=0x7fa61c01c930 comp rx=0 tx=0).stop 2024-04-25T00:38:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa62a0e9640 1 -- 172.21.15.29:0/2518390938 shutdown_connections 2024-04-25T00:38:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa62a0e9640 1 --2- 172.21.15.29:0/2518390938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa62406cd40 0x7fa62406d150 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa62a0e9640 1 --2- 172.21.15.29:0/2518390938 >> v2:172.21.15.157:3300/0 conn(0x7fa6240b3b50 0x7fa6240b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa62a0e9640 1 -- 172.21.15.29:0/2518390938 >> 172.21.15.29:0/2518390938 conn(0x7fa6240b1870 msgr2=0x7fa6240b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa62a0e9640 1 -- 172.21.15.29:0/2518390938 shutdown_connections 2024-04-25T00:38:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.783+0000 7fa62a0e9640 1 -- 172.21.15.29:0/2518390938 wait complete. 2024-04-25T00:38:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa62a0e9640 1 Processor -- start 2024-04-25T00:38:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa62a0e9640 1 -- start start 2024-04-25T00:38:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa62a0e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa62406cd40 0x7fa624159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa62a0e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6240b3b50 0x7fa62413b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa62a0e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6241456d0 0x7fa62415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa62a0e9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa62406d690 con 0x7fa6241456d0 2024-04-25T00:38:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa62a0e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6240b5000 con 0x7fa6240b3b50 2024-04-25T00:38:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa62a0e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa624168b70 con 0x7fa62406cd40 2024-04-25T00:38:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa623fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6241456d0 0x7fa62415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa623fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6241456d0 0x7fa62415b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51376/0 (socket says 172.21.15.29:51376) 2024-04-25T00:38:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa623fff640 1 -- 172.21.15.29:0/1833362586 learned_addr learned my addr 172.21.15.29:0/1833362586 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa623fff640 1 -- 172.21.15.29:0/1833362586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa62406cd40 msgr2=0x7fa624159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa623fff640 1 --2- 172.21.15.29:0/1833362586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa62406cd40 0x7fa624159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:20.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa623fff640 1 -- 172.21.15.29:0/1833362586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6240b3b50 msgr2=0x7fa62413b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:20.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa622ffd640 1 --2- 172.21.15.29:0/1833362586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6240b3b50 0x7fa62413b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:20.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa623fff640 1 --2- 172.21.15.29:0/1833362586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6240b3b50 0x7fa62413b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:20.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa623fff640 1 -- 172.21.15.29:0/1833362586 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62413f310 con 0x7fa6241456d0 2024-04-25T00:38:20.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa623fff640 1 --2- 172.21.15.29:0/1833362586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6241456d0 0x7fa62415b3f0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fa61c01a560 tx=0x7fa61c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:20.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa60bfff640 1 -- 172.21.15.29:0/1833362586 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0b2050 con 0x7fa6241456d0 2024-04-25T00:38:20.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa60bfff640 1 -- 172.21.15.29:0/1833362586 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c0be070 con 0x7fa6241456d0 2024-04-25T00:38:20.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa62a0e9640 1 -- 172.21.15.29:0/1833362586 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62413da70 con 0x7fa6241456d0 2024-04-25T00:38:20.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa60bfff640 1 -- 172.21.15.29:0/1833362586 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0ada50 con 0x7fa6241456d0 2024-04-25T00:38:20.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa62a0e9640 1 -- 172.21.15.29:0/1833362586 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa624158d80 con 0x7fa6241456d0 2024-04-25T00:38:20.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.787+0000 7fa62a0e9640 1 -- 172.21.15.29:0/1833362586 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6240692e0 con 0x7fa6241456d0 2024-04-25T00:38:20.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.791+0000 7fa60bfff640 1 -- 172.21.15.29:0/1833362586 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fa61c0adbf0 con 0x7fa6241456d0 2024-04-25T00:38:20.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.791+0000 7fa60bfff640 1 --2- 172.21.15.29:0/1833362586 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa610067ce0 0x7fa61006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:20.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.791+0000 7fa60bfff640 1 -- 172.21.15.29:0/1833362586 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5357+0+0 (secure 0 0 0) 0x7fa61c121830 con 0x7fa6241456d0 2024-04-25T00:38:20.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.791+0000 7fa6237fe640 1 --2- 172.21.15.29:0/1833362586 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa610067ce0 0x7fa61006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:20.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.791+0000 7fa6237fe640 1 --2- 172.21.15.29:0/1833362586 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa610067ce0 0x7fa61006a1a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa614000f70 tx=0x7fa614007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:20.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.795+0000 7fa60bfff640 1 -- 172.21.15.29:0/1833362586 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa61c0c7020 con 0x7fa6241456d0 2024-04-25T00:38:20.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:20.955+0000 7fa62a0e9640 1 -- 172.21.15.29:0/1833362586 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa6240b1f70 con 0x7fa6241456d0 2024-04-25T00:38:21.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.627+0000 7fa60bfff640 1 -- 172.21.15.29:0/1833362586 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7fa62413f310 con 0x7fa6241456d0 2024-04-25T00:38:21.632 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-04-25T00:38:21.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.631+0000 7fa62a0e9640 1 -- 172.21.15.29:0/1833362586 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa610067ce0 msgr2=0x7fa61006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:21.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.631+0000 7fa62a0e9640 1 --2- 172.21.15.29:0/1833362586 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa610067ce0 0x7fa61006a1a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa614000f70 tx=0x7fa614007a90 comp rx=0 tx=0).stop 2024-04-25T00:38:21.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.631+0000 7fa62a0e9640 1 -- 172.21.15.29:0/1833362586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6241456d0 msgr2=0x7fa62415b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:21.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.631+0000 7fa62a0e9640 1 --2- 172.21.15.29:0/1833362586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6241456d0 0x7fa62415b3f0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fa61c01a560 tx=0x7fa61c0196d0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.631+0000 7fa62a0e9640 1 -- 172.21.15.29:0/1833362586 shutdown_connections 2024-04-25T00:38:21.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.631+0000 7fa62a0e9640 1 --2- 172.21.15.29:0/1833362586 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa610067ce0 0x7fa61006a1a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.631+0000 7fa62a0e9640 1 --2- 172.21.15.29:0/1833362586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6241456d0 0x7fa62415b3f0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.631+0000 7fa62a0e9640 1 --2- 172.21.15.29:0/1833362586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6240b3b50 0x7fa62413b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.631+0000 7fa62a0e9640 1 --2- 172.21.15.29:0/1833362586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa62406cd40 0x7fa624159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.631+0000 7fa62a0e9640 1 -- 172.21.15.29:0/1833362586 >> 172.21.15.29:0/1833362586 conn(0x7fa6240b1870 msgr2=0x7fa6240af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:21.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.635+0000 7fa62a0e9640 1 -- 172.21.15.29:0/1833362586 shutdown_connections 2024-04-25T00:38:21.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.635+0000 7fa62a0e9640 1 -- 172.21.15.29:0/1833362586 wait complete. 2024-04-25T00:38:21.665 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:38:21.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.783+0000 7efc95a0f640 1 Processor -- start 2024-04-25T00:38:21.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.783+0000 7efc95a0f640 1 -- start start 2024-04-25T00:38:21.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc95a0f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efc9015e2f0 0x7efc9006cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:21.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc95a0f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efc900b1290 0x7efc900cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:21.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc95a0f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc900d1340 con 0x7efc9006d4c0 2024-04-25T00:38:21.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc95a0f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7efc9015e6d0 con 0x7efc9015e2f0 2024-04-25T00:38:21.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc95a0f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7efc9006c4a0 con 0x7efc900b1290 2024-04-25T00:38:21.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc8effd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efc9015e2f0 0x7efc9006cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:21.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc8f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7efc9006d4c0 0x7efc900ce690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56022/0 (socket says 172.21.15.29:56022) 2024-04-25T00:38:21.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc8f7fe640 1 -- 172.21.15.29:0/3013443392 learned_addr learned my addr 172.21.15.29:0/3013443392 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:21.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc8effd640 1 -- 172.21.15.29:0/3013443392 >> v2:172.21.15.157:3300/0 conn(0x7efc900b1290 msgr2=0x7efc900cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:21.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc8effd640 1 --2- 172.21.15.29:0/3013443392 >> v2:172.21.15.157:3300/0 conn(0x7efc900b1290 0x7efc900cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc8effd640 1 -- 172.21.15.29:0/3013443392 >> v1:172.21.15.29:6789/0 conn(0x7efc9006d4c0 legacy=0x7efc900ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:21.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc8effd640 1 -- 172.21.15.29:0/3013443392 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9006c780 con 0x7efc9015e2f0 2024-04-25T00:38:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc8effd640 1 --2- 172.21.15.29:0/3013443392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc9015e2f0 0x7efc9006cf80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7efc90071b90 tx=0x7efc8001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1538765486dfc958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc8dffb640 1 -- 172.21.15.29:0/3013443392 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc800b5070 con 0x7efc9015e2f0 2024-04-25T00:38:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc8dffb640 1 -- 172.21.15.29:0/3013443392 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc800a8970 con 0x7efc9015e2f0 2024-04-25T00:38:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc8dffb640 1 -- 172.21.15.29:0/3013443392 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc800a8d40 con 0x7efc9015e2f0 2024-04-25T00:38:21.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc95a0f640 1 -- 172.21.15.29:0/3013443392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc9015e2f0 msgr2=0x7efc9006cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:21.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc95a0f640 1 --2- 172.21.15.29:0/3013443392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc9015e2f0 0x7efc9006cf80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7efc90071b90 tx=0x7efc8001ec70 comp rx=0 tx=0).stop 2024-04-25T00:38:21.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc95a0f640 1 -- 172.21.15.29:0/3013443392 shutdown_connections 2024-04-25T00:38:21.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc95a0f640 1 --2- 172.21.15.29:0/3013443392 >> v2:172.21.15.157:3300/0 conn(0x7efc900b1290 0x7efc900cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc95a0f640 1 --2- 172.21.15.29:0/3013443392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc9015e2f0 0x7efc9006cf80 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc95a0f640 1 -- 172.21.15.29:0/3013443392 >> 172.21.15.29:0/3013443392 conn(0x7efc900d1e10 msgr2=0x7efc900d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:21.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc95a0f640 1 -- 172.21.15.29:0/3013443392 shutdown_connections 2024-04-25T00:38:21.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.787+0000 7efc95a0f640 1 -- 172.21.15.29:0/3013443392 wait complete. 2024-04-25T00:38:21.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc95a0f640 1 Processor -- start 2024-04-25T00:38:21.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc95a0f640 1 -- start start 2024-04-25T00:38:21.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc95a0f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc9006d4c0 0x7efc900b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:21.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc95a0f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b1290 0x7efc900c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:21.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc95a0f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9015e2f0 0x7efc9015a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:21.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc95a0f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc900ceda0 con 0x7efc9006d4c0 2024-04-25T00:38:21.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc95a0f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc900cf6c0 con 0x7efc9015e2f0 2024-04-25T00:38:21.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc95a0f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc900cef20 con 0x7efc900b1290 2024-04-25T00:38:21.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc8effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc9006d4c0 0x7efc900b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:21.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc8effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc9006d4c0 0x7efc900b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51396/0 (socket says 172.21.15.29:51396) 2024-04-25T00:38:21.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc8effd640 1 -- 172.21.15.29:0/179983273 learned_addr learned my addr 172.21.15.29:0/179983273 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:21.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc8effd640 1 -- 172.21.15.29:0/179983273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b1290 msgr2=0x7efc900c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:21.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc8effd640 1 --2- 172.21.15.29:0/179983273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b1290 0x7efc900c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc8effd640 1 -- 172.21.15.29:0/179983273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9015e2f0 msgr2=0x7efc9015a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:21.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc8effd640 1 --2- 172.21.15.29:0/179983273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9015e2f0 0x7efc9015a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc8effd640 1 -- 172.21.15.29:0/179983273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9015b540 con 0x7efc9006d4c0 2024-04-25T00:38:21.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc8effd640 1 --2- 172.21.15.29:0/179983273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc9006d4c0 0x7efc900b00b0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7efc8001b880 tx=0x7efc8001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:21.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc77fff640 1 -- 172.21.15.29:0/179983273 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc800b5040 con 0x7efc9006d4c0 2024-04-25T00:38:21.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc77fff640 1 -- 172.21.15.29:0/179983273 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc800bf070 con 0x7efc9006d4c0 2024-04-25T00:38:21.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc95a0f640 1 -- 172.21.15.29:0/179983273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc900c9060 con 0x7efc9006d4c0 2024-04-25T00:38:21.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc77fff640 1 -- 172.21.15.29:0/179983273 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc800b0a40 con 0x7efc9006d4c0 2024-04-25T00:38:21.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc95a0f640 1 -- 172.21.15.29:0/179983273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc900c9620 con 0x7efc9006d4c0 2024-04-25T00:38:21.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc95a0f640 1 -- 172.21.15.29:0/179983273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc900cf3c0 con 0x7efc9006d4c0 2024-04-25T00:38:21.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.791+0000 7efc77fff640 1 -- 172.21.15.29:0/179983273 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7efc800b0be0 con 0x7efc9006d4c0 2024-04-25T00:38:21.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.795+0000 7efc77fff640 1 --2- 172.21.15.29:0/179983273 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efc7c067ce0 0x7efc7c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:21.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.795+0000 7efc77fff640 1 -- 172.21.15.29:0/179983273 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4943+0+0 (secure 0 0 0) 0x7efc80123660 con 0x7efc9006d4c0 2024-04-25T00:38:21.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.795+0000 7efc8e7fc640 1 --2- 172.21.15.29:0/179983273 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efc7c067ce0 0x7efc7c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:21.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.795+0000 7efc8e7fc640 1 --2- 172.21.15.29:0/179983273 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efc7c067ce0 0x7efc7c06a1a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efc900b0020 tx=0x7efc84006bf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:21.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.799+0000 7efc77fff640 1 -- 172.21.15.29:0/179983273 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efc800187d0 con 0x7efc9006d4c0 2024-04-25T00:38:21.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.967+0000 7efc95a0f640 1 -- 172.21.15.29:0/179983273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7efc90069250 con 0x7efc9006d4c0 2024-04-25T00:38:21.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.971+0000 7efc77fff640 1 -- 172.21.15.29:0/179983273 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7efc800f3f50 con 0x7efc9006d4c0 2024-04-25T00:38:21.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.975+0000 7efc95a0f640 1 -- 172.21.15.29:0/179983273 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efc7c067ce0 msgr2=0x7efc7c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:21.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.975+0000 7efc95a0f640 1 --2- 172.21.15.29:0/179983273 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efc7c067ce0 0x7efc7c06a1a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efc900b0020 tx=0x7efc84006bf0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.975+0000 7efc95a0f640 1 -- 172.21.15.29:0/179983273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc9006d4c0 msgr2=0x7efc900b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:21.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.975+0000 7efc95a0f640 1 --2- 172.21.15.29:0/179983273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc9006d4c0 0x7efc900b00b0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7efc8001b880 tx=0x7efc8001ba10 comp rx=0 tx=0).stop 2024-04-25T00:38:21.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.975+0000 7efc95a0f640 1 -- 172.21.15.29:0/179983273 shutdown_connections 2024-04-25T00:38:21.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.975+0000 7efc95a0f640 1 --2- 172.21.15.29:0/179983273 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efc7c067ce0 0x7efc7c06a1a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.975+0000 7efc95a0f640 1 --2- 172.21.15.29:0/179983273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9015e2f0 0x7efc9015a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.975+0000 7efc95a0f640 1 --2- 172.21.15.29:0/179983273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b1290 0x7efc900c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.975+0000 7efc95a0f640 1 --2- 172.21.15.29:0/179983273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc9006d4c0 0x7efc900b00b0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:21.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.975+0000 7efc95a0f640 1 -- 172.21.15.29:0/179983273 >> 172.21.15.29:0/179983273 conn(0x7efc900d1e10 msgr2=0x7efc900b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:21.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.975+0000 7efc95a0f640 1 -- 172.21.15.29:0/179983273 shutdown_connections 2024-04-25T00:38:21.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:21.975+0000 7efc95a0f640 1 -- 172.21.15.29:0/179983273 wait complete. 2024-04-25T00:38:21.995 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-04-25T00:38:22.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d3444640 1 Processor -- start 2024-04-25T00:38:22.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d3444640 1 -- start start 2024-04-25T00:38:22.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d3444640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9cc0b3140 0x7fc9cc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d3444640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc9cc0b3c30 0x7fc9cc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d3444640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9cc0b41e0 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d3444640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc9cc06d960 con 0x7fc9cc0b3c30 2024-04-25T00:38:22.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d3444640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc9cc06dc40 con 0x7fc9cc0b3140 2024-04-25T00:38:22.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d09df640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc9cc0b4b10 0x7fc9cc0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56030/0 (socket says 172.21.15.29:56030) 2024-04-25T00:38:22.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d09df640 1 -- 172.21.15.29:0/3224724929 learned_addr learned my addr 172.21.15.29:0/3224724929 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:22.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d19e1640 1 --2- 172.21.15.29:0/3224724929 >> v2:172.21.15.29:3300/0 conn(0x7fc9cc0b3c30 0x7fc9cc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:22.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d11e0640 1 --2- 172.21.15.29:0/3224724929 >> v2:172.21.15.157:3300/0 conn(0x7fc9cc0b3140 0x7fc9cc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:22.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9c3fff640 1 -- 172.21.15.29:0/3224724929 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1451122614 0 0) 0x7fc9cc0b41e0 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9c3fff640 1 -- 172.21.15.29:0/3224724929 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9b8003530 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d19e1640 1 -- 172.21.15.29:0/3224724929 >> v2:172.21.15.157:3300/0 conn(0x7fc9cc0b3140 msgr2=0x7fc9cc06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:22.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d19e1640 1 --2- 172.21.15.29:0/3224724929 >> v2:172.21.15.157:3300/0 conn(0x7fc9cc0b3140 0x7fc9cc06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d19e1640 1 -- 172.21.15.29:0/3224724929 >> v1:172.21.15.29:6789/0 conn(0x7fc9cc0b4b10 legacy=0x7fc9cc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:22.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d19e1640 1 -- 172.21.15.29:0/3224724929 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9cc06df20 con 0x7fc9cc0b3c30 2024-04-25T00:38:22.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.103+0000 7fc9d11e0640 1 --2- 172.21.15.29:0/3224724929 >> v2:172.21.15.157:3300/0 conn(0x7fc9cc0b3140 0x7fc9cc06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:22.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d19e1640 1 --2- 172.21.15.29:0/3224724929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9cc0b3c30 0x7fc9cc0ce240 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009d90 tx=0x7fc9c801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fffa34e708cc01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:22.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9c3fff640 1 -- 172.21.15.29:0/3224724929 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80b2070 con 0x7fc9cc0b3c30 2024-04-25T00:38:22.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9c3fff640 1 -- 172.21.15.29:0/3224724929 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c80a57f0 con 0x7fc9cc0b3c30 2024-04-25T00:38:22.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9c3fff640 1 -- 172.21.15.29:0/3224724929 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80a5bc0 con 0x7fc9cc0b3c30 2024-04-25T00:38:22.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 -- 172.21.15.29:0/3224724929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9cc0b3c30 msgr2=0x7fc9cc0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:22.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 --2- 172.21.15.29:0/3224724929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9cc0b3c30 0x7fc9cc0ce240 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009d90 tx=0x7fc9c801c930 comp rx=0 tx=0).stop 2024-04-25T00:38:22.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 -- 172.21.15.29:0/3224724929 shutdown_connections 2024-04-25T00:38:22.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 --2- 172.21.15.29:0/3224724929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9cc0b3c30 0x7fc9cc0ce240 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 --2- 172.21.15.29:0/3224724929 >> v2:172.21.15.157:3300/0 conn(0x7fc9cc0b3140 0x7fc9cc06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 -- 172.21.15.29:0/3224724929 >> 172.21.15.29:0/3224724929 conn(0x7fc9cc0b1870 msgr2=0x7fc9cc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:22.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 -- 172.21.15.29:0/3224724929 shutdown_connections 2024-04-25T00:38:22.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 -- 172.21.15.29:0/3224724929 wait complete. 2024-04-25T00:38:22.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 Processor -- start 2024-04-25T00:38:22.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 -- start start 2024-04-25T00:38:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9cc0b3140 0x7fc9cc0cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc0b3c30 0x7fc9cc13ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9cc0b4b10 0x7fc9cc13f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc168d80 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc168c00 con 0x7fc9cc0b3140 2024-04-25T00:38:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d3444640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc9cc168f00 con 0x7fc9cc0b3c30 2024-04-25T00:38:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d19e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9cc0b4b10 0x7fc9cc13f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d19e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9cc0b4b10 0x7fc9cc13f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51416/0 (socket says 172.21.15.29:51416) 2024-04-25T00:38:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d19e1640 1 -- 172.21.15.29:0/1456344798 learned_addr learned my addr 172.21.15.29:0/1456344798 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d11e0640 1 --2- 172.21.15.29:0/1456344798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9cc0b3140 0x7fc9cc0cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d19e1640 1 -- 172.21.15.29:0/1456344798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc0b3c30 msgr2=0x7fc9cc13ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d19e1640 1 --2- 172.21.15.29:0/1456344798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc0b3c30 0x7fc9cc13ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d19e1640 1 -- 172.21.15.29:0/1456344798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9cc0b3140 msgr2=0x7fc9cc0cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d19e1640 1 --2- 172.21.15.29:0/1456344798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9cc0b3140 0x7fc9cc0cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.107+0000 7fc9d19e1640 1 -- 172.21.15.29:0/1456344798 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9cc13ffb0 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.111+0000 7fc9d19e1640 1 --2- 172.21.15.29:0/1456344798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9cc0b4b10 0x7fc9cc13f3c0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80b0040 tx=0x7fc9c80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:22.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.111+0000 7fc9c1ffb640 1 -- 172.21.15.29:0/1456344798 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80b2050 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.111+0000 7fc9c1ffb640 1 -- 172.21.15.29:0/1456344798 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c80be070 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.111+0000 7fc9d3444640 1 -- 172.21.15.29:0/1456344798 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9cc13f9f0 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.111+0000 7fc9c1ffb640 1 -- 172.21.15.29:0/1456344798 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80ad850 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.111+0000 7fc9d3444640 1 -- 172.21.15.29:0/1456344798 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9cc1480c0 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.111+0000 7fc9d3444640 1 -- 172.21.15.29:0/1456344798 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9cc069270 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.111+0000 7fc9c1ffb640 1 -- 172.21.15.29:0/1456344798 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fc9c80ad9f0 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.111+0000 7fc9c1ffb640 1 --2- 172.21.15.29:0/1456344798 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc9b0067cb0 0x7fc9b006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.111+0000 7fc9d11e0640 1 --2- 172.21.15.29:0/1456344798 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc9b0067cb0 0x7fc9b006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:22.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.111+0000 7fc9c1ffb640 1 -- 172.21.15.29:0/1456344798 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4943+0+0 (secure 0 0 0) 0x7fc9c81211d0 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.111+0000 7fc9d11e0640 1 --2- 172.21.15.29:0/1456344798 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc9b0067cb0 0x7fc9b006a170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40096d0 tx=0x7fc9c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:22.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.115+0000 7fc9c1ffb640 1 -- 172.21.15.29:0/1456344798 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc9c8016520 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.335+0000 7fc9d3444640 1 -- 172.21.15.29:0/1456344798 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7fc9cc0ce450 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.631+0000 7fc9c1ffb640 1 -- 172.21.15.29:0/1456344798 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v366) ==== 149+0+0 (secure 0 0 0) 0x7fc9c811fe20 con 0x7fc9cc0b4b10 2024-04-25T00:38:22.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.635+0000 7fc9d3444640 1 -- 172.21.15.29:0/1456344798 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc9b0067cb0 msgr2=0x7fc9b006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:22.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.635+0000 7fc9d3444640 1 --2- 172.21.15.29:0/1456344798 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc9b0067cb0 0x7fc9b006a170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40096d0 tx=0x7fc9c4008040 comp rx=0 tx=0).stop 2024-04-25T00:38:22.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.635+0000 7fc9d3444640 1 -- 172.21.15.29:0/1456344798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9cc0b4b10 msgr2=0x7fc9cc13f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:22.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.635+0000 7fc9d3444640 1 --2- 172.21.15.29:0/1456344798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9cc0b4b10 0x7fc9cc13f3c0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80b0040 tx=0x7fc9c80196d0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.635+0000 7fc9d3444640 1 -- 172.21.15.29:0/1456344798 shutdown_connections 2024-04-25T00:38:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.635+0000 7fc9d3444640 1 --2- 172.21.15.29:0/1456344798 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc9b0067cb0 0x7fc9b006a170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.635+0000 7fc9d3444640 1 --2- 172.21.15.29:0/1456344798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9cc0b4b10 0x7fc9cc13f3c0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.635+0000 7fc9d3444640 1 --2- 172.21.15.29:0/1456344798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9cc0b3c30 0x7fc9cc13ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.635+0000 7fc9d3444640 1 --2- 172.21.15.29:0/1456344798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9cc0b3140 0x7fc9cc0cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.635+0000 7fc9d3444640 1 -- 172.21.15.29:0/1456344798 >> 172.21.15.29:0/1456344798 conn(0x7fc9cc0b1870 msgr2=0x7fc9cc0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.635+0000 7fc9d3444640 1 -- 172.21.15.29:0/1456344798 shutdown_connections 2024-04-25T00:38:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.635+0000 7fc9d3444640 1 -- 172.21.15.29:0/1456344798 wait complete. 2024-04-25T00:38:22.658 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-04-25T00:38:22.658 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-04-25T00:38:22.658 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-04-25T00:38:22.658 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:22.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.775+0000 7f59ddd6c640 1 Processor -- start 2024-04-25T00:38:22.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.775+0000 7f59ddd6c640 1 -- start start 2024-04-25T00:38:22.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59ddd6c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59d815e8d0 0x7f59d80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59ddd6c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59d80b3720 con 0x7f59d815f3c0 2024-04-25T00:38:22.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59ddd6c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59d80b39c0 con 0x7f59d80b4780 2024-04-25T00:38:22.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59ddd6c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f59d80b42f0 con 0x7f59d815e8d0 2024-04-25T00:38:22.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d7fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f59d815f3c0 0x7f59d815c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56034/0 (socket says 172.21.15.29:56034) 2024-04-25T00:38:22.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d7fff640 1 -- 172.21.15.29:0/3856626137 learned_addr learned my addr 172.21.15.29:0/3856626137 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:22.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3634868484 0 0) 0x7f59d80b3720 con 0x7f59d815f3c0 2024-04-25T00:38:22.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d77fe640 1 --2- 172.21.15.29:0/3856626137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59d815e8d0 0x7f59d80cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:22.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59c0005740 con 0x7f59d815f3c0 2024-04-25T00:38:22.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 608476652 0 0) 0x7f59c0005740 con 0x7f59d815f3c0 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59d815e8d0 msgr2=0x7f59d80cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 --2- 172.21.15.29:0/3856626137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59d815e8d0 0x7f59d80cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 >> v1:172.21.15.157:6789/0 conn(0x7f59d80b4780 legacy=0x7f59d815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d80b3ca0 con 0x7f59d815f3c0 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f59d0001a00 con 0x7f59d815f3c0 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 >> v1:172.21.15.29:6789/0 conn(0x7f59d815f3c0 legacy=0x7f59d815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 --2- 172.21.15.29:0/3856626137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59c0005c00 0x7f59c0006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 --2- 172.21.15.29:0/3856626137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59c0006a20 0x7f59c0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 --2- 172.21.15.29:0/3856626137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59d815e8d0 0x7f59c00073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59d80b42f0 con 0x7f59c0005c00 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f59c000dc90 con 0x7f59c0006a20 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f59c000de90 con 0x7f59d815e8d0 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d77fe640 1 --2- 172.21.15.29:0/3856626137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59c0005c00 0x7f59c0006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:22.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d6ffd640 1 --2- 172.21.15.29:0/3856626137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59c0006a20 0x7f59c0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:22.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d7fff640 1 --2- 172.21.15.29:0/3856626137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59d815e8d0 0x7f59c00073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:22.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d77fe640 1 -- 172.21.15.29:0/3856626137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59d815e8d0 msgr2=0x7f59c00073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:22.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d77fe640 1 --2- 172.21.15.29:0/3856626137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59d815e8d0 0x7f59c00073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d77fe640 1 -- 172.21.15.29:0/3856626137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59c0006a20 msgr2=0x7f59c0006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:22.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d77fe640 1 --2- 172.21.15.29:0/3856626137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59c0006a20 0x7f59c0006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d77fe640 1 -- 172.21.15.29:0/3856626137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c0005740 con 0x7f59c0005c00 2024-04-25T00:38:22.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.779+0000 7f59d77fe640 1 --2- 172.21.15.29:0/3856626137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59c0005c00 0x7f59c0006020 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009ab0 tx=0x7f59c801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc1ce431a96027e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:22.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80096b0 con 0x7f59c0005c00 2024-04-25T00:38:22.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c80a64f0 con 0x7f59c0005c00 2024-04-25T00:38:22.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d67fc640 1 -- 172.21.15.29:0/3856626137 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80a68e0 con 0x7f59c0005c00 2024-04-25T00:38:22.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3856626137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59c0005c00 msgr2=0x7f59c0006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:22.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 --2- 172.21.15.29:0/3856626137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59c0005c00 0x7f59c0006020 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009ab0 tx=0x7f59c801ca00 comp rx=0 tx=0).stop 2024-04-25T00:38:22.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d7fff640 1 -- 172.21.15.29:0/3856626137 reap_dead start 2024-04-25T00:38:22.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3856626137 shutdown_connections 2024-04-25T00:38:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3856626137 >> 172.21.15.29:0/3856626137 conn(0x7f59d80b1870 msgr2=0x7f59d80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3856626137 shutdown_connections 2024-04-25T00:38:22.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3856626137 wait complete. 2024-04-25T00:38:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 Processor -- start 2024-04-25T00:38:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 -- start start 2024-04-25T00:38:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59c0005c00 0x7f59d81e8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59d81e6950 0x7f59d815a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59d80722c0 0x7f59d8072740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59d806d710 con 0x7f59c0005c00 2024-04-25T00:38:22.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f59d806d590 con 0x7f59d80722c0 2024-04-25T00:38:22.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f59d806d890 con 0x7f59d81e6950 2024-04-25T00:38:22.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59c0005c00 0x7f59d81e8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:22.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59c0005c00 0x7f59d81e8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51440/0 (socket says 172.21.15.29:51440) 2024-04-25T00:38:22.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d77fe640 1 -- 172.21.15.29:0/3220348278 learned_addr learned my addr 172.21.15.29:0/3220348278 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:22.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d7fff640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59d80722c0 0x7f59d8072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:22.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d6ffd640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59d81e6950 0x7f59d815a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:22.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d77fe640 1 -- 172.21.15.29:0/3220348278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59d81e6950 msgr2=0x7f59d815a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:22.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d77fe640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59d81e6950 0x7f59d815a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d77fe640 1 -- 172.21.15.29:0/3220348278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59d80722c0 msgr2=0x7f59d8072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:22.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d77fe640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59d80722c0 0x7f59d8072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d77fe640 1 -- 172.21.15.29:0/3220348278 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d813f180 con 0x7f59c0005c00 2024-04-25T00:38:22.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59d77fe640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59c0005c00 0x7f59d81e8aa0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f59c80b0040 tx=0x7f59c0007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:22.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59bffff640 1 -- 172.21.15.29:0/3220348278 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c800f000 con 0x7f59c0005c00 2024-04-25T00:38:22.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59bffff640 1 -- 172.21.15.29:0/3220348278 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c80bf070 con 0x7f59c0005c00 2024-04-25T00:38:22.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.783+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3220348278 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d81e6ec0 con 0x7f59c0005c00 2024-04-25T00:38:22.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.787+0000 7f59bffff640 1 -- 172.21.15.29:0/3220348278 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80a68e0 con 0x7f59c0005c00 2024-04-25T00:38:22.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.787+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3220348278 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d81e7480 con 0x7f59c0005c00 2024-04-25T00:38:22.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.787+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3220348278 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d8069270 con 0x7f59c0005c00 2024-04-25T00:38:22.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.787+0000 7f59bffff640 1 -- 172.21.15.29:0/3220348278 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f59c80a6a80 con 0x7f59c0005c00 2024-04-25T00:38:22.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.787+0000 7f59bffff640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f59c4067cb0 0x7f59c406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:22.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.787+0000 7f59bffff640 1 -- 172.21.15.29:0/3220348278 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4943+0+0 (secure 0 0 0) 0x7f59c8120a20 con 0x7f59c0005c00 2024-04-25T00:38:22.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.787+0000 7f59d6ffd640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f59c4067cb0 0x7f59c406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:22.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.787+0000 7f59d6ffd640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f59c4067cb0 0x7f59c406a170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f59d81f6660 tx=0x7f59cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:22.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.791+0000 7f59bffff640 1 -- 172.21.15.29:0/3220348278 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f59c8016740 con 0x7f59c0005c00 2024-04-25T00:38:22.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.951+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3220348278 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59d806e7b0 con 0x7f59c0005c00 2024-04-25T00:38:22.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.955+0000 7f59bffff640 1 -- 172.21.15.29:0/3220348278 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+12982 (secure 0 0 0) 0x7f59c80f1180 con 0x7f59c0005c00 2024-04-25T00:38:22.957 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:22.957 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":109,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:21.624482+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6835/1444171533":"2024-04-26T00:38:18.379549+0000","172.21.15.29:6833/1444171533":"2024-04-26T00:38:18.379549+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:22.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.959+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3220348278 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f59c4067cb0 msgr2=0x7f59c406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:22.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.959+0000 7f59ddd6c640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f59c4067cb0 0x7f59c406a170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f59d81f6660 tx=0x7f59cc009040 comp rx=0 tx=0).stop 2024-04-25T00:38:22.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.959+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3220348278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59c0005c00 msgr2=0x7f59d81e8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:22.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.959+0000 7f59ddd6c640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59c0005c00 0x7f59d81e8aa0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f59c80b0040 tx=0x7f59c0007a50 comp rx=0 tx=0).stop 2024-04-25T00:38:22.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.959+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3220348278 shutdown_connections 2024-04-25T00:38:22.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.959+0000 7f59ddd6c640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f59c4067cb0 0x7f59c406a170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.959+0000 7f59ddd6c640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59d80722c0 0x7f59d8072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.959+0000 7f59ddd6c640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59d81e6950 0x7f59d815a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.959+0000 7f59ddd6c640 1 --2- 172.21.15.29:0/3220348278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59c0005c00 0x7f59d81e8aa0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:22.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.959+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3220348278 >> 172.21.15.29:0/3220348278 conn(0x7f59d80b1870 msgr2=0x7f59d80b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:22.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.959+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3220348278 shutdown_connections 2024-04-25T00:38:22.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:22.959+0000 7f59ddd6c640 1 -- 172.21.15.29:0/3220348278 wait complete. 2024-04-25T00:38:22.977 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:38:22.977 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:38:23.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e3065f640 1 Processor -- start 2024-04-25T00:38:23.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e3065f640 1 -- start start 2024-04-25T00:38:23.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e3065f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5e2815df70 0x7f5e280b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e3065f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e2806d210 0x7f5e280b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e3065f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e2806d980 con 0x7f5e280b4b60 2024-04-25T00:38:23.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e3065f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5e280b4fa0 con 0x7f5e2815df70 2024-04-25T00:38:23.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e3065f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2806dc60 con 0x7f5e2806d210 2024-04-25T00:38:23.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e2e3fb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5e2815df70 0x7f5e280b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e2e3fb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5e2815df70 0x7f5e280b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51450/0 (socket says 172.21.15.29:51450) 2024-04-25T00:38:23.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e2e3fb640 1 -- 172.21.15.29:0/2720492166 learned_addr learned my addr 172.21.15.29:0/2720492166 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:23.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e2dbfa640 1 --2- 172.21.15.29:0/2720492166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e2806d210 0x7f5e280b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e2e3fb640 1 -- 172.21.15.29:0/2720492166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e2806d210 msgr2=0x7f5e280b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:23.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e2e3fb640 1 --2- 172.21.15.29:0/2720492166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e2806d210 0x7f5e280b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e2e3fb640 1 -- 172.21.15.29:0/2720492166 >> v1:172.21.15.157:6789/0 conn(0x7f5e280b4b60 legacy=0x7f5e28070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.103+0000 7f5e2e3fb640 1 -- 172.21.15.29:0/2720492166 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2806df40 con 0x7f5e2815df70 2024-04-25T00:38:23.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e2e3fb640 1 --2- 172.21.15.29:0/2720492166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e2815df70 0x7f5e280b3e40 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009870 tx=0x7f5e2401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764fe9821798c72e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:23.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e2d3f9640 1 -- 172.21.15.29:0/2720492166 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e240b0070 con 0x7f5e2815df70 2024-04-25T00:38:23.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e2d3f9640 1 -- 172.21.15.29:0/2720492166 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e240ab600 con 0x7f5e2815df70 2024-04-25T00:38:23.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e2d3f9640 1 -- 172.21.15.29:0/2720492166 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e240ab9d0 con 0x7f5e2815df70 2024-04-25T00:38:23.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 -- 172.21.15.29:0/2720492166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e2815df70 msgr2=0x7f5e280b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 --2- 172.21.15.29:0/2720492166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e2815df70 0x7f5e280b3e40 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009870 tx=0x7f5e2401c560 comp rx=0 tx=0).stop 2024-04-25T00:38:23.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 -- 172.21.15.29:0/2720492166 shutdown_connections 2024-04-25T00:38:23.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 --2- 172.21.15.29:0/2720492166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e2806d210 0x7f5e280b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 --2- 172.21.15.29:0/2720492166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e2815df70 0x7f5e280b3e40 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 -- 172.21.15.29:0/2720492166 >> 172.21.15.29:0/2720492166 conn(0x7f5e280b1870 msgr2=0x7f5e280b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:23.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 -- 172.21.15.29:0/2720492166 shutdown_connections 2024-04-25T00:38:23.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 -- 172.21.15.29:0/2720492166 wait complete. 2024-04-25T00:38:23.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 Processor -- start 2024-04-25T00:38:23.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 -- start start 2024-04-25T00:38:23.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e2806d210 0x7f5e281599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e280b4b60 0x7f5e280b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2815df70 0x7f5e2815ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2806ea20 con 0x7f5e280b4b60 2024-04-25T00:38:23.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2806d5f0 con 0x7f5e2815df70 2024-04-25T00:38:23.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e3065f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2806eba0 con 0x7f5e2806d210 2024-04-25T00:38:23.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e2dbfa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e280b4b60 0x7f5e280b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e2dbfa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e280b4b60 0x7f5e280b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51466/0 (socket says 172.21.15.29:51466) 2024-04-25T00:38:23.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e2dbfa640 1 -- 172.21.15.29:0/2760405517 learned_addr learned my addr 172.21.15.29:0/2760405517 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:23.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e2e3fb640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e2806d210 0x7f5e281599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.107+0000 7f5e2ebfc640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2815df70 0x7f5e2815ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e2dbfa640 1 -- 172.21.15.29:0/2760405517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e2806d210 msgr2=0x7f5e281599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e2dbfa640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e2806d210 0x7f5e281599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e2dbfa640 1 -- 172.21.15.29:0/2760405517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2815df70 msgr2=0x7f5e2815ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e2dbfa640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2815df70 0x7f5e2815ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e2dbfa640 1 -- 172.21.15.29:0/2760405517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e281569d0 con 0x7f5e280b4b60 2024-04-25T00:38:23.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e2dbfa640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e280b4b60 0x7f5e280b0590 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f5e1800d800 tx=0x7f5e1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:23.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e1effd640 1 -- 172.21.15.29:0/2760405517 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1800b6c0 con 0x7f5e280b4b60 2024-04-25T00:38:23.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e1effd640 1 -- 172.21.15.29:0/2760405517 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e18019070 con 0x7f5e280b4b60 2024-04-25T00:38:23.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e3065f640 1 -- 172.21.15.29:0/2760405517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e28156cb0 con 0x7f5e280b4b60 2024-04-25T00:38:23.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e1effd640 1 -- 172.21.15.29:0/2760405517 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1800bae0 con 0x7f5e280b4b60 2024-04-25T00:38:23.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e3065f640 1 -- 172.21.15.29:0/2760405517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e28157270 con 0x7f5e280b4b60 2024-04-25T00:38:23.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e3065f640 1 -- 172.21.15.29:0/2760405517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e280b0b30 con 0x7f5e280b4b60 2024-04-25T00:38:23.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e1effd640 1 -- 172.21.15.29:0/2760405517 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f5e1800bc80 con 0x7f5e280b4b60 2024-04-25T00:38:23.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e1effd640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5e0c067cb0 0x7f5e0c06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e1effd640 1 -- 172.21.15.29:0/2760405517 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4943+0+0 (secure 0 0 0) 0x7f5e180864c0 con 0x7f5e280b4b60 2024-04-25T00:38:23.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e2e3fb640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5e0c067cb0 0x7f5e0c06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.111+0000 7f5e2e3fb640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5e0c067cb0 0x7f5e0c06a170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5e240ad000 tx=0x7f5e240a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:23.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.115+0000 7f5e1effd640 1 -- 172.21.15.29:0/2760405517 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5e18056db0 con 0x7f5e280b4b60 2024-04-25T00:38:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e3065f640 1 -- 172.21.15.29:0/2760405517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5e28069270 con 0x7f5e280b4b60 2024-04-25T00:38:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e1effd640 1 -- 172.21.15.29:0/2760405517 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7f5e180566b0 con 0x7f5e280b4b60 2024-04-25T00:38:23.273 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:38:23.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e3065f640 1 -- 172.21.15.29:0/2760405517 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5e0c067cb0 msgr2=0x7f5e0c06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e3065f640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5e0c067cb0 0x7f5e0c06a170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5e240ad000 tx=0x7f5e240a6000 comp rx=0 tx=0).stop 2024-04-25T00:38:23.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e3065f640 1 -- 172.21.15.29:0/2760405517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e280b4b60 msgr2=0x7f5e280b0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e3065f640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e280b4b60 0x7f5e280b0590 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f5e1800d800 tx=0x7f5e1800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e3065f640 1 -- 172.21.15.29:0/2760405517 shutdown_connections 2024-04-25T00:38:23.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e3065f640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5e0c067cb0 0x7f5e0c06a170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e3065f640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e2815df70 0x7f5e2815ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e3065f640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e280b4b60 0x7f5e280b0590 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e3065f640 1 --2- 172.21.15.29:0/2760405517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e2806d210 0x7f5e281599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e3065f640 1 -- 172.21.15.29:0/2760405517 >> 172.21.15.29:0/2760405517 conn(0x7f5e280b1870 msgr2=0x7f5e2815be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:23.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e3065f640 1 -- 172.21.15.29:0/2760405517 shutdown_connections 2024-04-25T00:38:23.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.271+0000 7f5e3065f640 1 -- 172.21.15.29:0/2760405517 wait complete. 2024-04-25T00:38:23.291 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-04-25T00:38:23.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff669ab9640 1 Processor -- start 2024-04-25T00:38:23.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff669ab9640 1 -- start start 2024-04-25T00:38:23.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff669ab9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6640aec60 0x7ff6640cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff669ab9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff6640af040 0x7ff66415c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff669ab9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff66415ccd0 con 0x7ff66415d720 2024-04-25T00:38:23.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff669ab9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff66415caa0 con 0x7ff6640af040 2024-04-25T00:38:23.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff669ab9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff66415cfb0 con 0x7ff6640aec60 2024-04-25T00:38:23.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff662ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff6640af040 0x7ff66415c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff6637fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6640aec60 0x7ff6640cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff662ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff6640af040 0x7ff66415c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:46618/0 (socket says 172.21.15.29:46618) 2024-04-25T00:38:23.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff662ffd640 1 -- 172.21.15.29:0/73743920 learned_addr learned my addr 172.21.15.29:0/73743920 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:23.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff662ffd640 1 -- 172.21.15.29:0/73743920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6640aec60 msgr2=0x7ff6640cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:23.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff662ffd640 1 --2- 172.21.15.29:0/73743920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6640aec60 0x7ff6640cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff662ffd640 1 -- 172.21.15.29:0/73743920 >> v1:172.21.15.157:6789/0 conn(0x7ff66415d720 legacy=0x7ff6640b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.415+0000 7ff662ffd640 1 -- 172.21.15.29:0/73743920 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66415d290 con 0x7ff6640af040 2024-04-25T00:38:23.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff662ffd640 1 --2- 172.21.15.29:0/73743920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6640af040 0x7ff66415c560 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff654009870 tx=0x7ff65401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c3737c5a981324d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:23.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff6627fc640 1 -- 172.21.15.29:0/73743920 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6540b0070 con 0x7ff6640af040 2024-04-25T00:38:23.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff6627fc640 1 -- 172.21.15.29:0/73743920 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6540ab600 con 0x7ff6640af040 2024-04-25T00:38:23.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff6627fc640 1 -- 172.21.15.29:0/73743920 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6540ab9d0 con 0x7ff6640af040 2024-04-25T00:38:23.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff669ab9640 1 -- 172.21.15.29:0/73743920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6640af040 msgr2=0x7ff66415c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:23.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff669ab9640 1 --2- 172.21.15.29:0/73743920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6640af040 0x7ff66415c560 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff654009870 tx=0x7ff65401c4d0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff669ab9640 1 -- 172.21.15.29:0/73743920 shutdown_connections 2024-04-25T00:38:23.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff669ab9640 1 --2- 172.21.15.29:0/73743920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6640af040 0x7ff66415c560 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff669ab9640 1 --2- 172.21.15.29:0/73743920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6640aec60 0x7ff6640cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff669ab9640 1 -- 172.21.15.29:0/73743920 >> 172.21.15.29:0/73743920 conn(0x7ff664075030 msgr2=0x7ff6640b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:23.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff669ab9640 1 -- 172.21.15.29:0/73743920 shutdown_connections 2024-04-25T00:38:23.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff669ab9640 1 -- 172.21.15.29:0/73743920 wait complete. 2024-04-25T00:38:23.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff669ab9640 1 Processor -- start 2024-04-25T00:38:23.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff669ab9640 1 -- start start 2024-04-25T00:38:23.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff669ab9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6640aec60 0x7ff6640b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff669ab9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6640af040 0x7ff66414f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.419+0000 7ff669ab9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff66415d720 0x7ff66414f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff669ab9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff664168d20 con 0x7ff66415d720 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff669ab9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff664168ba0 con 0x7ff6640af040 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff669ab9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff664168ea0 con 0x7ff6640aec60 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff663fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff66415d720 0x7ff66414f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff663fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff66415d720 0x7ff66414f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51474/0 (socket says 172.21.15.29:51474) 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff663fff640 1 -- 172.21.15.29:0/4075693814 learned_addr learned my addr 172.21.15.29:0/4075693814 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff663fff640 1 -- 172.21.15.29:0/4075693814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6640aec60 msgr2=0x7ff6640b0340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff663fff640 1 --2- 172.21.15.29:0/4075693814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6640aec60 0x7ff6640b0340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff663fff640 1 -- 172.21.15.29:0/4075693814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6640af040 msgr2=0x7ff66414f030 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff663fff640 1 --2- 172.21.15.29:0/4075693814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6640af040 0x7ff66414f030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff663fff640 1 -- 172.21.15.29:0/4075693814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66414fc40 con 0x7ff66415d720 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff663fff640 1 --2- 172.21.15.29:0/4075693814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff66415d720 0x7ff66414f570 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00c7a0 tx=0x7ff65c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff65bfff640 1 -- 172.21.15.29:0/4075693814 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c003b80 con 0x7ff66415d720 2024-04-25T00:38:23.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff65bfff640 1 -- 172.21.15.29:0/4075693814 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65c003d20 con 0x7ff66415d720 2024-04-25T00:38:23.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff669ab9640 1 -- 172.21.15.29:0/4075693814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6640aa500 con 0x7ff66415d720 2024-04-25T00:38:23.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff65bfff640 1 -- 172.21.15.29:0/4075693814 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c00aa90 con 0x7ff66415d720 2024-04-25T00:38:23.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff669ab9640 1 -- 172.21.15.29:0/4075693814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6640aaac0 con 0x7ff66415d720 2024-04-25T00:38:23.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff669ab9640 1 -- 172.21.15.29:0/4075693814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff664069270 con 0x7ff66415d720 2024-04-25T00:38:23.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff65bfff640 1 -- 172.21.15.29:0/4075693814 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7ff65c00ac30 con 0x7ff66415d720 2024-04-25T00:38:23.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff65bfff640 1 --2- 172.21.15.29:0/4075693814 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff644067ce0 0x7ff64406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:23.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff65bfff640 1 -- 172.21.15.29:0/4075693814 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4943+0+0 (secure 0 0 0) 0x7ff65c086030 con 0x7ff66415d720 2024-04-25T00:38:23.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff6637fe640 1 --2- 172.21.15.29:0/4075693814 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff644067ce0 0x7ff64406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:23.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.423+0000 7ff6637fe640 1 --2- 172.21.15.29:0/4075693814 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff644067ce0 0x7ff64406a1a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009950 tx=0x7ff64c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:23.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.427+0000 7ff65bfff640 1 -- 172.21.15.29:0/4075693814 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff65c056890 con 0x7ff66415d720 2024-04-25T00:38:23.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:23.647+0000 7ff669ab9640 1 -- 172.21.15.29:0/4075693814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7ff66415c4f0 con 0x7ff66415d720 2024-04-25T00:38:24.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.639+0000 7ff65bfff640 1 -- 172.21.15.29:0/4075693814 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v368) ==== 151+0+0 (secure 0 0 0) 0x7ff65c056190 con 0x7ff66415d720 2024-04-25T00:38:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.643+0000 7ff669ab9640 1 -- 172.21.15.29:0/4075693814 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff644067ce0 msgr2=0x7ff64406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.643+0000 7ff669ab9640 1 --2- 172.21.15.29:0/4075693814 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff644067ce0 0x7ff64406a1a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009950 tx=0x7ff64c009190 comp rx=0 tx=0).stop 2024-04-25T00:38:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.643+0000 7ff669ab9640 1 -- 172.21.15.29:0/4075693814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff66415d720 msgr2=0x7ff66414f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.643+0000 7ff669ab9640 1 --2- 172.21.15.29:0/4075693814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff66415d720 0x7ff66414f570 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00c7a0 tx=0x7ff65c00cc60 comp rx=0 tx=0).stop 2024-04-25T00:38:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.643+0000 7ff669ab9640 1 -- 172.21.15.29:0/4075693814 shutdown_connections 2024-04-25T00:38:24.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.643+0000 7ff669ab9640 1 --2- 172.21.15.29:0/4075693814 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff644067ce0 0x7ff64406a1a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.643+0000 7ff669ab9640 1 --2- 172.21.15.29:0/4075693814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff66415d720 0x7ff66414f570 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.643+0000 7ff669ab9640 1 --2- 172.21.15.29:0/4075693814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6640af040 0x7ff66414f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.643+0000 7ff669ab9640 1 --2- 172.21.15.29:0/4075693814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6640aec60 0x7ff6640b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.643+0000 7ff669ab9640 1 -- 172.21.15.29:0/4075693814 >> 172.21.15.29:0/4075693814 conn(0x7ff664075030 msgr2=0x7ff6640ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:24.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.643+0000 7ff669ab9640 1 -- 172.21.15.29:0/4075693814 shutdown_connections 2024-04-25T00:38:24.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.643+0000 7ff669ab9640 1 -- 172.21.15.29:0/4075693814 wait complete. 2024-04-25T00:38:24.666 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:38:24.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.795+0000 7f38338fe640 1 Processor -- start 2024-04-25T00:38:24.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.795+0000 7f38338fe640 1 -- start start 2024-04-25T00:38:24.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f38338fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f382c15e8d0 0x7f382c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:24.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f38338fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f382c0b4780 0x7f382c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:24.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f38338fe640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f382c0b3720 con 0x7f382c15f1f0 2024-04-25T00:38:24.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f38338fe640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f382c0b42f0 con 0x7f382c0b4780 2024-04-25T00:38:24.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f38338fe640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f382c15d7e0 con 0x7f382c15e8d0 2024-04-25T00:38:24.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f3831e9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f382c15f1f0 0x7f382c15b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56046/0 (socket says 172.21.15.29:56046) 2024-04-25T00:38:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f3831e9b640 1 -- 172.21.15.29:0/2023439950 learned_addr learned my addr 172.21.15.29:0/2023439950 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f3830e99640 1 --2- 172.21.15.29:0/2023439950 >> v2:172.21.15.29:3300/0 conn(0x7f382c0b4780 0x7f382c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f383169a640 1 --2- 172.21.15.29:0/2023439950 >> v2:172.21.15.157:3300/0 conn(0x7f382c15e8d0 0x7f382c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f3823fff640 1 -- 172.21.15.29:0/2023439950 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2599120966 0 0) 0x7f382c0b3720 con 0x7f382c15f1f0 2024-04-25T00:38:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f3823fff640 1 -- 172.21.15.29:0/2023439950 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3818003530 con 0x7f382c15f1f0 2024-04-25T00:38:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f3830e99640 1 -- 172.21.15.29:0/2023439950 >> v2:172.21.15.157:3300/0 conn(0x7f382c15e8d0 msgr2=0x7f382c0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f3830e99640 1 --2- 172.21.15.29:0/2023439950 >> v2:172.21.15.157:3300/0 conn(0x7f382c15e8d0 0x7f382c0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f3830e99640 1 -- 172.21.15.29:0/2023439950 >> v1:172.21.15.29:6789/0 conn(0x7f382c15f1f0 legacy=0x7f382c15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f3830e99640 1 -- 172.21.15.29:0/2023439950 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382c0b39c0 con 0x7f382c0b4780 2024-04-25T00:38:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f3830e99640 1 --2- 172.21.15.29:0/2023439950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f382c0b4780 0x7f382c15ecb0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f382c0b5260 tx=0x7f381c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6839596979568826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f3823fff640 1 -- 172.21.15.29:0/2023439950 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c15c1c0 con 0x7f382c0b4780 2024-04-25T00:38:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.799+0000 7f3823fff640 1 -- 172.21.15.29:0/2023439950 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f381c0b0070 con 0x7f382c0b4780 2024-04-25T00:38:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f3823fff640 1 -- 172.21.15.29:0/2023439950 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c0ab850 con 0x7f382c0b4780 2024-04-25T00:38:24.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 -- 172.21.15.29:0/2023439950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f382c0b4780 msgr2=0x7f382c15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:24.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 --2- 172.21.15.29:0/2023439950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f382c0b4780 0x7f382c15ecb0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f382c0b5260 tx=0x7f381c01c450 comp rx=0 tx=0).stop 2024-04-25T00:38:24.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 -- 172.21.15.29:0/2023439950 shutdown_connections 2024-04-25T00:38:24.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 --2- 172.21.15.29:0/2023439950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f382c0b4780 0x7f382c15ecb0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 --2- 172.21.15.29:0/2023439950 >> v2:172.21.15.157:3300/0 conn(0x7f382c15e8d0 0x7f382c0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 -- 172.21.15.29:0/2023439950 >> 172.21.15.29:0/2023439950 conn(0x7f382c0b1870 msgr2=0x7f382c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:24.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 -- 172.21.15.29:0/2023439950 shutdown_connections 2024-04-25T00:38:24.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 -- 172.21.15.29:0/2023439950 wait complete. 2024-04-25T00:38:24.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 Processor -- start 2024-04-25T00:38:24.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 -- start start 2024-04-25T00:38:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f382c0b4780 0x7f382c15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c15e8d0 0x7f382c0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f382c15f1f0 0x7f382c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f382c144a20 con 0x7f382c0b4780 2024-04-25T00:38:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f382c15f600 con 0x7f382c15e8d0 2024-04-25T00:38:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f382c0cf0a0 con 0x7f382c15f1f0 2024-04-25T00:38:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f383169a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f382c0b4780 0x7f382c15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f383169a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f382c0b4780 0x7f382c15a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51490/0 (socket says 172.21.15.29:51490) 2024-04-25T00:38:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f383169a640 1 -- 172.21.15.29:0/3926502040 learned_addr learned my addr 172.21.15.29:0/3926502040 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f383169a640 1 -- 172.21.15.29:0/3926502040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f382c15f1f0 msgr2=0x7f382c159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f3831e9b640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f382c15f1f0 0x7f382c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f383169a640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f382c15f1f0 0x7f382c159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f3830e99640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c15e8d0 0x7f382c0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f383169a640 1 -- 172.21.15.29:0/3926502040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c15e8d0 msgr2=0x7f382c0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f383169a640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c15e8d0 0x7f382c0ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f383169a640 1 -- 172.21.15.29:0/3926502040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382c0677c0 con 0x7f382c0b4780 2024-04-25T00:38:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f383169a640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f382c0b4780 0x7f382c15a740 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f382800d800 tx=0x7f382800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f3821ffb640 1 -- 172.21.15.29:0/3926502040 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382800b660 con 0x7f382c0b4780 2024-04-25T00:38:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f3821ffb640 1 -- 172.21.15.29:0/3926502040 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3828019070 con 0x7f382c0b4780 2024-04-25T00:38:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f3821ffb640 1 -- 172.21.15.29:0/3926502040 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382800ba00 con 0x7f382c0b4780 2024-04-25T00:38:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 -- 172.21.15.29:0/3926502040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382c135510 con 0x7f382c0b4780 2024-04-25T00:38:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.803+0000 7f38338fe640 1 -- 172.21.15.29:0/3926502040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f382c135ad0 con 0x7f382c0b4780 2024-04-25T00:38:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.807+0000 7f38338fe640 1 -- 172.21.15.29:0/3926502040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f382c069270 con 0x7f382c0b4780 2024-04-25T00:38:24.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.807+0000 7f3821ffb640 1 -- 172.21.15.29:0/3926502040 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f382800bba0 con 0x7f382c0b4780 2024-04-25T00:38:24.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.807+0000 7f3821ffb640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3810067ce0 0x7f381006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:24.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.807+0000 7f3821ffb640 1 -- 172.21.15.29:0/3926502040 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4943+0+0 (secure 0 0 0) 0x7f3828086350 con 0x7f382c0b4780 2024-04-25T00:38:24.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.807+0000 7f3830e99640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3810067ce0 0x7f381006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:24.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.807+0000 7f3830e99640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3810067ce0 0x7f381006a1a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f381c0af000 tx=0x7f381c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:24.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.811+0000 7f3821ffb640 1 -- 172.21.15.29:0/3926502040 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3828056d40 con 0x7f382c0b4780 2024-04-25T00:38:24.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.967+0000 7f38338fe640 1 -- 172.21.15.29:0/3926502040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f382c06e7b0 con 0x7f382c0b4780 2024-04-25T00:38:24.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.971+0000 7f3821ffb640 1 -- 172.21.15.29:0/3926502040 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f3828056640 con 0x7f382c0b4780 2024-04-25T00:38:24.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.975+0000 7f38338fe640 1 -- 172.21.15.29:0/3926502040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3810067ce0 msgr2=0x7f381006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:24.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.975+0000 7f38338fe640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3810067ce0 0x7f381006a1a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f381c0af000 tx=0x7f381c008040 comp rx=0 tx=0).stop 2024-04-25T00:38:24.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.975+0000 7f38338fe640 1 -- 172.21.15.29:0/3926502040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f382c0b4780 msgr2=0x7f382c15a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:24.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.975+0000 7f38338fe640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f382c0b4780 0x7f382c15a740 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f382800d800 tx=0x7f382800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.975+0000 7f38338fe640 1 -- 172.21.15.29:0/3926502040 shutdown_connections 2024-04-25T00:38:24.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.975+0000 7f38338fe640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3810067ce0 0x7f381006a1a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.975+0000 7f38338fe640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f382c15f1f0 0x7f382c159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.975+0000 7f38338fe640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c15e8d0 0x7f382c0ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.975+0000 7f38338fe640 1 --2- 172.21.15.29:0/3926502040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f382c0b4780 0x7f382c15a740 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:24.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.975+0000 7f38338fe640 1 -- 172.21.15.29:0/3926502040 >> 172.21.15.29:0/3926502040 conn(0x7f382c0b1870 msgr2=0x7f382c13e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:24.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.975+0000 7f38338fe640 1 -- 172.21.15.29:0/3926502040 shutdown_connections 2024-04-25T00:38:24.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:24.975+0000 7f38338fe640 1 -- 172.21.15.29:0/3926502040 wait complete. 2024-04-25T00:38:24.994 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:38:24.994 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:38:25.049 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:38:25.100 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:38:25.100 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:38:25.101 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:38:25.106 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:38:25.106 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:38:25.106 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:38:25.112 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:38:25.112 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:38:25.112 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:38:25.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:38:25.152 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:38:25.152 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:38:25.162 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:38:25.163 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:38:25.163 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:38:25.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9b3cf640 1 Processor -- start 2024-04-25T00:38:25.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9b3cf640 1 -- start start 2024-04-25T00:38:25.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9b3cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a940b3c30 0x7f2a9406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9b3cf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2a940b4b10 0x7f2a940ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9b3cf640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a940b41e0 con 0x7f2a940b3140 2024-04-25T00:38:25.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9b3cf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9406d960 con 0x7f2a940b4b10 2024-04-25T00:38:25.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9b3cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9406dc40 con 0x7f2a940b3c30 2024-04-25T00:38:25.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9916b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2a940b3140 0x7f2a940b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56058/0 (socket says 172.21.15.29:56058) 2024-04-25T00:38:25.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9916b640 1 -- 172.21.15.29:0/1173197744 learned_addr learned my addr 172.21.15.29:0/1173197744 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9996c640 1 --2- 172.21.15.29:0/1173197744 >> v2:172.21.15.157:3300/0 conn(0x7f2a940b4b10 0x7f2a940ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9896a640 1 --2- 172.21.15.29:0/1173197744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a940b3c30 0x7f2a9406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 843113448 0 0) 0x7f2a940b41e0 con 0x7f2a940b3140 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a80003530 con 0x7f2a940b3140 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4137590367 0 0) 0x7f2a80003530 con 0x7f2a940b3140 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a940b3c30 msgr2=0x7f2a9406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 --2- 172.21.15.29:0/1173197744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a940b3c30 0x7f2a9406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 >> v2:172.21.15.157:3300/0 conn(0x7f2a940b4b10 msgr2=0x7f2a940ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 --2- 172.21.15.29:0/1173197744 >> v2:172.21.15.157:3300/0 conn(0x7f2a940b4b10 0x7f2a940ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9406df20 con 0x7f2a940b3140 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2a90001680 con 0x7f2a940b3140 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9996c640 1 --2- 172.21.15.29:0/1173197744 >> v2:172.21.15.157:3300/0 conn(0x7f2a940b4b10 0x7f2a940ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 >> v1:172.21.15.29:6789/0 conn(0x7f2a940b3140 legacy=0x7f2a940b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 --2- 172.21.15.29:0/1173197744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a800039c0 0x7f2a80003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 --2- 172.21.15.29:0/1173197744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a940b3c30 0x7f2a80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 --2- 172.21.15.29:0/1173197744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a80004d50 0x7f2a800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9406d960 con 0x7f2a800039c0 2024-04-25T00:38:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9406dc40 con 0x7f2a80004d50 2024-04-25T00:38:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2a8000ba00 con 0x7f2a940b3c30 2024-04-25T00:38:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9916b640 1 --2- 172.21.15.29:0/1173197744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a800039c0 0x7f2a80003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9896a640 1 --2- 172.21.15.29:0/1173197744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a940b3c30 0x7f2a80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9996c640 1 --2- 172.21.15.29:0/1173197744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a80004d50 0x7f2a800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9916b640 1 -- 172.21.15.29:0/1173197744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a940b3c30 msgr2=0x7f2a80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9916b640 1 --2- 172.21.15.29:0/1173197744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a940b3c30 0x7f2a80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9916b640 1 -- 172.21.15.29:0/1173197744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a80004d50 msgr2=0x7f2a800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:25.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9916b640 1 --2- 172.21.15.29:0/1173197744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a80004d50 0x7f2a800051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9916b640 1 -- 172.21.15.29:0/1173197744 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a80003530 con 0x7f2a800039c0 2024-04-25T00:38:25.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.323+0000 7f2a9996c640 1 --2- 172.21.15.29:0/1173197744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a80004d50 0x7f2a800051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:25.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9916b640 1 --2- 172.21.15.29:0/1173197744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a800039c0 0x7f2a80003de0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000c240 tx=0x7f2a9001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1faf80b7f79d982 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:25.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90007d60 con 0x7f2a800039c0 2024-04-25T00:38:25.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a900a9020 con 0x7f2a800039c0 2024-04-25T00:38:25.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a8bfff640 1 -- 172.21.15.29:0/1173197744 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90005ae0 con 0x7f2a800039c0 2024-04-25T00:38:25.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1173197744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a800039c0 msgr2=0x7f2a80003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:25.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 --2- 172.21.15.29:0/1173197744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a800039c0 0x7f2a80003de0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000c240 tx=0x7f2a9001ccc0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9996c640 1 -- 172.21.15.29:0/1173197744 reap_dead start 2024-04-25T00:38:25.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1173197744 shutdown_connections 2024-04-25T00:38:25.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1173197744 >> 172.21.15.29:0/1173197744 conn(0x7f2a940b1870 msgr2=0x7f2a940b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:25.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1173197744 shutdown_connections 2024-04-25T00:38:25.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1173197744 wait complete. 2024-04-25T00:38:25.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 Processor -- start 2024-04-25T00:38:25.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 -- start start 2024-04-25T00:38:25.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a800039c0 0x7f2a9415d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a9415dd30 0x7f2a9413c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a9406cd40 0x7f2a9406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2a94166d70 con 0x7f2a9406cd40 2024-04-25T00:38:25.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a94166bf0 con 0x7f2a800039c0 2024-04-25T00:38:25.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9b3cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2a94166ef0 con 0x7f2a9415dd30 2024-04-25T00:38:25.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.327+0000 7f2a9996c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a9406cd40 0x7f2a9406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9996c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a9406cd40 0x7f2a9406d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51498/0 (socket says 172.21.15.29:51498) 2024-04-25T00:38:25.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9996c640 1 -- 172.21.15.29:0/1236047567 learned_addr learned my addr 172.21.15.29:0/1236047567 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:25.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9896a640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a9415dd30 0x7f2a9413c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9916b640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a800039c0 0x7f2a9415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9996c640 1 -- 172.21.15.29:0/1236047567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a9415dd30 msgr2=0x7f2a9413c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:25.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9996c640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a9415dd30 0x7f2a9413c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9996c640 1 -- 172.21.15.29:0/1236047567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a800039c0 msgr2=0x7f2a9415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:25.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9996c640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a800039c0 0x7f2a9415d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9996c640 1 -- 172.21.15.29:0/1236047567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9415e510 con 0x7f2a9406cd40 2024-04-25T00:38:25.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9916b640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a800039c0 0x7f2a9415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:25.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9896a640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a9415dd30 0x7f2a9413c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:25.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9996c640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a9406cd40 0x7f2a9406d1f0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f2a940708a0 tx=0x7f2a8c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:25.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a89ffb640 1 -- 172.21.15.29:0/1236047567 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c019070 con 0x7f2a9406cd40 2024-04-25T00:38:25.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a89ffb640 1 -- 172.21.15.29:0/1236047567 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a8c007cb0 con 0x7f2a9406cd40 2024-04-25T00:38:25.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1236047567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9415e7f0 con 0x7f2a9406cd40 2024-04-25T00:38:25.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a89ffb640 1 -- 172.21.15.29:0/1236047567 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c007e60 con 0x7f2a9406cd40 2024-04-25T00:38:25.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1236047567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9415edb0 con 0x7f2a9406cd40 2024-04-25T00:38:25.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1236047567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a94069270 con 0x7f2a9406cd40 2024-04-25T00:38:25.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a89ffb640 1 -- 172.21.15.29:0/1236047567 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f2a8c01c070 con 0x7f2a9406cd40 2024-04-25T00:38:25.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a89ffb640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2a78067cb0 0x7f2a7806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a89ffb640 1 -- 172.21.15.29:0/1236047567 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4943+0+0 (secure 0 0 0) 0x7f2a8c085bb0 con 0x7f2a9406cd40 2024-04-25T00:38:25.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9916b640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2a78067cb0 0x7f2a7806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.331+0000 7f2a9916b640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2a78067cb0 0x7f2a7806a170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000c240 tx=0x7f2a90001ac0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:25.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.335+0000 7f2a89ffb640 1 -- 172.21.15.29:0/1236047567 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2a8c056410 con 0x7f2a9406cd40 2024-04-25T00:38:25.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.531+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1236047567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a9406ef80 con 0x7f2a9406cd40 2024-04-25T00:38:25.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.531+0000 7f2a89ffb640 1 -- 172.21.15.29:0/1236047567 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+3875 (secure 0 0 0) 0x7f2a8c055d10 con 0x7f2a9406cd40 2024-04-25T00:38:25.535 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:25.535 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":70,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1884979435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1884979435},{"type":"v1","addr":"172.21.15.157:6833","nonce":1884979435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1394307674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1394307674},{"type":"v1","addr":"172.21.15.157:6835","nonce":1394307674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/3915943091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3915943091},{"type":"v1","addr":"172.21.15.29:6834","nonce":3915943091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":8436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/1257339321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1257339321},{"type":"v1","addr":"172.21.15.29:6835","nonce":1257339321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[]} 2024-04-25T00:38:25.536 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 70 2024-04-25T00:38:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.535+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1236047567 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2a78067cb0 msgr2=0x7f2a7806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.535+0000 7f2a9b3cf640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2a78067cb0 0x7f2a7806a170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000c240 tx=0x7f2a90001ac0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.535+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1236047567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a9406cd40 msgr2=0x7f2a9406d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.535+0000 7f2a9b3cf640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a9406cd40 0x7f2a9406d1f0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f2a940708a0 tx=0x7f2a8c004840 comp rx=0 tx=0).stop 2024-04-25T00:38:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.535+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1236047567 shutdown_connections 2024-04-25T00:38:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.535+0000 7f2a9b3cf640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2a78067cb0 0x7f2a7806a170 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.535+0000 7f2a9b3cf640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a9406cd40 0x7f2a9406d1f0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.535+0000 7f2a9b3cf640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a9415dd30 0x7f2a9413c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.535+0000 7f2a9b3cf640 1 --2- 172.21.15.29:0/1236047567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a800039c0 0x7f2a9415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.535+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1236047567 >> 172.21.15.29:0/1236047567 conn(0x7f2a940b1870 msgr2=0x7f2a9415b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.535+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1236047567 shutdown_connections 2024-04-25T00:38:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.535+0000 7f2a9b3cf640 1 -- 172.21.15.29:0/1236047567 wait complete. 2024-04-25T00:38:25.554 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:38:25.555 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:38:25.555 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:38:25.555 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:38:25.555 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:38:25.555 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:38:25.555 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:38:25.555 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:38:25.556 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:38:25.556 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:38:25.556 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:38:25.556 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:38:25.556 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:38:25.556 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:38:25.556 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:38:25.556 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:38:25.564 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:38:25.565 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:38:25.566 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:38:25.566 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:38:25.569 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:38:25.569 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:38:25.574 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:38:25.574 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:38:25.608 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:38:25.609 DEBUG:teuthology.parallel:result is None 2024-04-25T00:38:25.610 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:38:25.610 DEBUG:teuthology.parallel:result is None 2024-04-25T00:38:25.619 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:38:25.620 DEBUG:teuthology.parallel:result is None 2024-04-25T00:38:25.620 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:38:25.621 DEBUG:teuthology.parallel:result is None 2024-04-25T00:38:25.621 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:38:25.667 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:38:25.673 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:38:25.691 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-25T00:38:25.691 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-25T00:38:25.692 INFO:tasks.ceph.mds.d.smithi157.stderr:2024-04-25T00:38:25.690+0000 7f251f666640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T00:38:25.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21e25b2640 1 Processor -- start 2024-04-25T00:38:25.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21e25b2640 1 -- start start 2024-04-25T00:38:25.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21e25b2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f21dc15d980 0x7f21dc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21e25b2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21dc0b3130 con 0x7f21dc0b2b70 2024-04-25T00:38:25.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21e25b2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21dc0b3410 con 0x7f21dc06db10 2024-04-25T00:38:25.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21e25b2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f21dc0cf560 con 0x7f21dc15d980 2024-04-25T00:38:25.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21db7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f21dc0b2b70 0x7f21dc150df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56092/0 (socket says 172.21.15.29:56092) 2024-04-25T00:38:25.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21db7fe640 1 -- 172.21.15.29:0/2481089230 learned_addr learned my addr 172.21.15.29:0/2481089230 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:25.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21dbfff640 1 --2- 172.21.15.29:0/2481089230 >> v2:172.21.15.157:3300/0 conn(0x7f21dc15d980 0x7f21dc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21daffd640 1 -- 172.21.15.29:0/2481089230 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4235295642 0 0) 0x7f21dc0b3130 con 0x7f21dc0b2b70 2024-04-25T00:38:25.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21daffd640 1 -- 172.21.15.29:0/2481089230 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21c4003530 con 0x7f21dc0b2b70 2024-04-25T00:38:25.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21dbfff640 1 -- 172.21.15.29:0/2481089230 >> v1:172.21.15.157:6789/0 conn(0x7f21dc06db10 legacy=0x7f21dc15b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:25.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21dbfff640 1 -- 172.21.15.29:0/2481089230 >> v1:172.21.15.29:6789/0 conn(0x7f21dc0b2b70 legacy=0x7f21dc150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:25.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.735+0000 7f21dbfff640 1 -- 172.21.15.29:0/2481089230 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc0b36f0 con 0x7f21dc15d980 2024-04-25T00:38:25.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21dbfff640 1 --2- 172.21.15.29:0/2481089230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21dc15d980 0x7f21dc06d5d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0097c0 tx=0x7f21cc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c06976799637178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:25.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21daffd640 1 -- 172.21.15.29:0/2481089230 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0b1070 con 0x7f21dc15d980 2024-04-25T00:38:25.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21daffd640 1 -- 172.21.15.29:0/2481089230 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21cc0ac5f0 con 0x7f21dc15d980 2024-04-25T00:38:25.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21daffd640 1 -- 172.21.15.29:0/2481089230 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0ac9c0 con 0x7f21dc15d980 2024-04-25T00:38:25.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 -- 172.21.15.29:0/2481089230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21dc15d980 msgr2=0x7f21dc06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:25.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 --2- 172.21.15.29:0/2481089230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21dc15d980 0x7f21dc06d5d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0097c0 tx=0x7f21cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 -- 172.21.15.29:0/2481089230 shutdown_connections 2024-04-25T00:38:25.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 --2- 172.21.15.29:0/2481089230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21dc15d980 0x7f21dc06d5d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 -- 172.21.15.29:0/2481089230 >> 172.21.15.29:0/2481089230 conn(0x7f21dc0b1870 msgr2=0x7f21dc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:25.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 -- 172.21.15.29:0/2481089230 shutdown_connections 2024-04-25T00:38:25.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 -- 172.21.15.29:0/2481089230 wait complete. 2024-04-25T00:38:25.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 Processor -- start 2024-04-25T00:38:25.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 -- start start 2024-04-25T00:38:25.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21dc06db10 0x7f21dc150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21dc0b2b70 0x7f21dc150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15d980 0x7f21dc14b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1694f0 con 0x7f21dc0b2b70 2024-04-25T00:38:25.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc15dd60 con 0x7f21dc06db10 2024-04-25T00:38:25.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.739+0000 7f21e25b2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f21dc169670 con 0x7f21dc15d980 2024-04-25T00:38:25.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21db7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21dc0b2b70 0x7f21dc150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21db7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21dc0b2b70 0x7f21dc150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51530/0 (socket says 172.21.15.29:51530) 2024-04-25T00:38:25.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21db7fe640 1 -- 172.21.15.29:0/1015166373 learned_addr learned my addr 172.21.15.29:0/1015166373 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:25.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21dbfff640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21dc06db10 0x7f21dc150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21e0b4f640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15d980 0x7f21dc14b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21db7fe640 1 -- 172.21.15.29:0/1015166373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15d980 msgr2=0x7f21dc14b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:25.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21db7fe640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15d980 0x7f21dc14b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21db7fe640 1 -- 172.21.15.29:0/1015166373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21dc06db10 msgr2=0x7f21dc150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:25.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21db7fe640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21dc06db10 0x7f21dc150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:25.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21db7fe640 1 -- 172.21.15.29:0/1015166373 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc151f10 con 0x7f21dc0b2b70 2024-04-25T00:38:25.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21db7fe640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21dc0b2b70 0x7f21dc150180 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c420 tx=0x7f21d000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:25.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21d8ff9640 1 -- 172.21.15.29:0/1015166373 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0017070 con 0x7f21dc0b2b70 2024-04-25T00:38:25.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21d8ff9640 1 -- 172.21.15.29:0/1015166373 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d0005cf0 con 0x7f21dc0b2b70 2024-04-25T00:38:25.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21e25b2640 1 -- 172.21.15.29:0/1015166373 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21dc14e570 con 0x7f21dc0b2b70 2024-04-25T00:38:25.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21d8ff9640 1 -- 172.21.15.29:0/1015166373 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d00072d0 con 0x7f21dc0b2b70 2024-04-25T00:38:25.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21e25b2640 1 -- 172.21.15.29:0/1015166373 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21dc14eb30 con 0x7f21dc0b2b70 2024-04-25T00:38:25.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21e25b2640 1 -- 172.21.15.29:0/1015166373 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21dc069270 con 0x7f21dc0b2b70 2024-04-25T00:38:25.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21d8ff9640 1 -- 172.21.15.29:0/1015166373 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f21d001a070 con 0x7f21dc0b2b70 2024-04-25T00:38:25.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21d8ff9640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f21c8067cb0 0x7f21c806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:25.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21d8ff9640 1 -- 172.21.15.29:0/1015166373 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4943+0+0 (secure 0 0 0) 0x7f21d0085420 con 0x7f21dc0b2b70 2024-04-25T00:38:25.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21dbfff640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f21c8067cb0 0x7f21c806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:25.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.743+0000 7f21dbfff640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f21c8067cb0 0x7f21c806a170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0ae000 tx=0x7f21cc0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:25.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.747+0000 7f21d8ff9640 1 -- 172.21.15.29:0/1015166373 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f21d0055650 con 0x7f21dc0b2b70 2024-04-25T00:38:25.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:25.919+0000 7f21e25b2640 1 -- 172.21.15.29:0/1015166373 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f21dc0cf910 con 0x7f21dc0b2b70 2024-04-25T00:38:26.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:26.995+0000 7f21d8ff9640 1 -- 172.21.15.29:0/1015166373 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v114) ==== 97+0+0 (secure 0 0 0) 0x7f21d0059950 con 0x7f21dc0b2b70 2024-04-25T00:38:26.999 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-04-25T00:38:27.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:26.999+0000 7f21e25b2640 1 -- 172.21.15.29:0/1015166373 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f21c8067cb0 msgr2=0x7f21c806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:26.999+0000 7f21e25b2640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f21c8067cb0 0x7f21c806a170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0ae000 tx=0x7f21cc0a7000 comp rx=0 tx=0).stop 2024-04-25T00:38:27.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:26.999+0000 7f21e25b2640 1 -- 172.21.15.29:0/1015166373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21dc0b2b70 msgr2=0x7f21dc150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:26.999+0000 7f21e25b2640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21dc0b2b70 0x7f21dc150180 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c420 tx=0x7f21d000c8e0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:26.999+0000 7f21e25b2640 1 -- 172.21.15.29:0/1015166373 shutdown_connections 2024-04-25T00:38:27.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:26.999+0000 7f21e25b2640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f21c8067cb0 0x7f21c806a170 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:26.999+0000 7f21e25b2640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21dc15d980 0x7f21dc14b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:26.999+0000 7f21e25b2640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21dc0b2b70 0x7f21dc150180 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:26.999+0000 7f21e25b2640 1 --2- 172.21.15.29:0/1015166373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21dc06db10 0x7f21dc150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:26.999+0000 7f21e25b2640 1 -- 172.21.15.29:0/1015166373 >> 172.21.15.29:0/1015166373 conn(0x7f21dc0b1870 msgr2=0x7f21dc0b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:27.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:26.999+0000 7f21e25b2640 1 -- 172.21.15.29:0/1015166373 shutdown_connections 2024-04-25T00:38:27.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:26.999+0000 7f21e25b2640 1 -- 172.21.15.29:0/1015166373 wait complete. 2024-04-25T00:38:27.026 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:38:27.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f62982b6640 1 Processor -- start 2024-04-25T00:38:27.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f62982b6640 1 -- start start 2024-04-25T00:38:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f62982b6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f629015e440 0x7f62900d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f62982b6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f62900d0ff0 0x7f62900cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f62982b6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62900cc3b0 con 0x7f629006c950 2024-04-25T00:38:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f62982b6640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f62900b11d0 con 0x7f629015e440 2024-04-25T00:38:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f62982b6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f62900cc650 con 0x7f62900d0ff0 2024-04-25T00:38:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f6295851640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f629015e440 0x7f62900d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f6295851640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f629015e440 0x7f62900d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51536/0 (socket says 172.21.15.29:51536) 2024-04-25T00:38:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f6295851640 1 -- 172.21.15.29:0/3545617137 learned_addr learned my addr 172.21.15.29:0/3545617137 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f6296052640 1 --1- 172.21.15.29:0/3545617137 >> v1:172.21.15.29:6789/0 conn(0x7f629006c950 0x7f629006cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56104/0 (socket says 172.21.15.29:56104) 2024-04-25T00:38:27.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f6296853640 1 --2- 172.21.15.29:0/3545617137 >> v2:172.21.15.157:3300/0 conn(0x7f62900d0ff0 0x7f62900cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f6295851640 1 -- 172.21.15.29:0/3545617137 >> v2:172.21.15.157:3300/0 conn(0x7f62900d0ff0 msgr2=0x7f62900cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:27.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f6295851640 1 --2- 172.21.15.29:0/3545617137 >> v2:172.21.15.157:3300/0 conn(0x7f62900d0ff0 0x7f62900cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f6295851640 1 -- 172.21.15.29:0/3545617137 >> v1:172.21.15.29:6789/0 conn(0x7f629006c950 legacy=0x7f629006cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f6295851640 1 -- 172.21.15.29:0/3545617137 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62900cc930 con 0x7f629015e440 2024-04-25T00:38:27.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.199+0000 7f6295050640 1 -- 172.21.15.29:0/3545617137 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1559602281 0 0) 0x7f62900cc3b0 con 0x7f629006c950 2024-04-25T00:38:27.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f6295851640 1 --2- 172.21.15.29:0/3545617137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f629015e440 0x7f62900d0ab0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f62900c8dc0 tx=0x7f628001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e0aba7c2cc0ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:27.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f6295050640 1 -- 172.21.15.29:0/3545617137 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800ab610 con 0x7f629015e440 2024-04-25T00:38:27.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f6295050640 1 -- 172.21.15.29:0/3545617137 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62800b0070 con 0x7f629015e440 2024-04-25T00:38:27.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f6295050640 1 -- 172.21.15.29:0/3545617137 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800ab9d0 con 0x7f629015e440 2024-04-25T00:38:27.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 -- 172.21.15.29:0/3545617137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f629015e440 msgr2=0x7f62900d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 --2- 172.21.15.29:0/3545617137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f629015e440 0x7f62900d0ab0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f62900c8dc0 tx=0x7f628001c520 comp rx=0 tx=0).stop 2024-04-25T00:38:27.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 -- 172.21.15.29:0/3545617137 shutdown_connections 2024-04-25T00:38:27.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 --2- 172.21.15.29:0/3545617137 >> v2:172.21.15.157:3300/0 conn(0x7f62900d0ff0 0x7f62900cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 --2- 172.21.15.29:0/3545617137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f629015e440 0x7f62900d0ab0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 -- 172.21.15.29:0/3545617137 >> 172.21.15.29:0/3545617137 conn(0x7f62900d1d60 msgr2=0x7f62900d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:27.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 -- 172.21.15.29:0/3545617137 shutdown_connections 2024-04-25T00:38:27.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 -- 172.21.15.29:0/3545617137 wait complete. 2024-04-25T00:38:27.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 Processor -- start 2024-04-25T00:38:27.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 -- start start 2024-04-25T00:38:27.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629006c950 0x7f6290153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62900d0ff0 0x7f6290155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f629015e440 0x7f6290156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f62900ce2e0 con 0x7f62900d0ff0 2024-04-25T00:38:27.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f62900ce160 con 0x7f629006c950 2024-04-25T00:38:27.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f62982b6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f62900ce460 con 0x7f629015e440 2024-04-25T00:38:27.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f6295851640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62900d0ff0 0x7f6290155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f6295851640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62900d0ff0 0x7f6290155ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51544/0 (socket says 172.21.15.29:51544) 2024-04-25T00:38:27.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f6295851640 1 -- 172.21.15.29:0/1871441223 learned_addr learned my addr 172.21.15.29:0/1871441223 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:27.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f6296052640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629006c950 0x7f6290153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.203+0000 7f6296853640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f629015e440 0x7f6290156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6295851640 1 -- 172.21.15.29:0/1871441223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f629015e440 msgr2=0x7f6290156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6295851640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f629015e440 0x7f6290156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6295851640 1 -- 172.21.15.29:0/1871441223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629006c950 msgr2=0x7f6290153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6295851640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629006c950 0x7f6290153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6295851640 1 -- 172.21.15.29:0/1871441223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62900b13f0 con 0x7f62900d0ff0 2024-04-25T00:38:27.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6295851640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62900d0ff0 0x7f6290155ac0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f62800ad040 tx=0x7f62800192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:27.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6286ffd640 1 -- 172.21.15.29:0/1871441223 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800bf610 con 0x7f62900d0ff0 2024-04-25T00:38:27.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6286ffd640 1 -- 172.21.15.29:0/1871441223 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62800bf7b0 con 0x7f62900d0ff0 2024-04-25T00:38:27.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f62982b6640 1 -- 172.21.15.29:0/1871441223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629015c730 con 0x7f62900d0ff0 2024-04-25T00:38:27.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6286ffd640 1 -- 172.21.15.29:0/1871441223 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800bfba0 con 0x7f62900d0ff0 2024-04-25T00:38:27.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f62982b6640 1 -- 172.21.15.29:0/1871441223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62900b1990 con 0x7f62900d0ff0 2024-04-25T00:38:27.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f62982b6640 1 -- 172.21.15.29:0/1871441223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6290069270 con 0x7f62900d0ff0 2024-04-25T00:38:27.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6286ffd640 1 -- 172.21.15.29:0/1871441223 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f62800bd020 con 0x7f62900d0ff0 2024-04-25T00:38:27.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6286ffd640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6274067cb0 0x7f627406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6286ffd640 1 -- 172.21.15.29:0/1871441223 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4857+0+0 (secure 0 0 0) 0x7f6280120750 con 0x7f62900d0ff0 2024-04-25T00:38:27.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6296052640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6274067cb0 0x7f627406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.207+0000 7f6296052640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6274067cb0 0x7f627406a170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f628c0012f0 tx=0x7f628c005ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:27.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.211+0000 7f6286ffd640 1 -- 172.21.15.29:0/1871441223 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f62800a8050 con 0x7f62900d0ff0 2024-04-25T00:38:27.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.431+0000 7f62982b6640 1 -- 172.21.15.29:0/1871441223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6290156690 con 0x7f62900d0ff0 2024-04-25T00:38:27.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.435+0000 7f6286ffd640 1 -- 172.21.15.29:0/1871441223 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7f6280016110 con 0x7f62900d0ff0 2024-04-25T00:38:27.436 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:27.436 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:38:27.436 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-04-25T00:38:27.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.436 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-04-25T00:38:27.437 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmyzgKAxAAU+nrUJgQhDX6xlARNFI/ng==", 2024-04-25T00:38:27.437 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.437 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:38:27.437 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:38:27.437 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:38:27.437 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.437 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.437 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-04-25T00:38:27.437 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmtyzPERAAxNUDANw1H/ZzOo+dhgzxlg==", 2024-04-25T00:38:27.437 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.438 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:38:27.438 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:38:27.438 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:38:27.438 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.438 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.438 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-04-25T00:38:27.438 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmYUl8CxAAj6pV1+s4a0rpWLq2EgZ+8w==", 2024-04-25T00:38:27.438 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.438 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:38:27.438 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmuAhKGhAAOLIcWUodW5TqtVgHWZl12A==", 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAzpClmudk6MhAA6YFyd57jwtB2pdOaad7bxQ==", 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA2pClm76q0NRAAxWhEP2ZGoFnZcuiQyGjglw==", 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:38:27.440 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:38:27.441 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.441 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.441 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-04-25T00:38:27.441 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA6pClmcgULABAA2F5WBR9Df0v4Sn875AzF7g==", 2024-04-25T00:38:27.441 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.441 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:38:27.441 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:38:27.441 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.441 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.442 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-04-25T00:38:27.442 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA9pClm9QuRBhAALFLruv4JU/uNZYMX0yNuXw==", 2024-04-25T00:38:27.442 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.442 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:38:27.442 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:38:27.442 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.442 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.442 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-04-25T00:38:27.442 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBBpClmNj4VDBAAxaxMvyaCvjynPY3f81udnw==", 2024-04-25T00:38:27.442 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.443 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:38:27.443 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:38:27.443 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.443 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.443 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-04-25T00:38:27.443 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBEpClmzBybEBAArry1lM4RIsM/txKfgBqDpw==", 2024-04-25T00:38:27.443 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.443 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:38:27.443 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBHpClm5HfjFhAAbFwJCpbhegLWDu0mbS8FoA==", 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-04-25T00:38:27.444 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBKpClme/7oHBAAOR2iQ6CweexR7sC1PgesZw==", 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A==", 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.446 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-04-25T00:38:27.446 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm8W/EJxAA0HZW24Hhcwf+soSZn6jHJg==", 2024-04-25T00:38:27.446 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.446 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:38:27.446 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:38:27.446 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:38:27.446 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:38:27.446 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.446 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmGuQJJRAAhVd4D0jc6Whgyf+B6xgFMw==", 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmn+ciLRAAg4tQuZ13WP0RnWJaJq5fUQ==", 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:38:27.447 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmPVihJBAA7PQge/teyRZKSkBnufZSZw==", 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.448 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-04-25T00:38:27.449 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClm6WrdOhAAsY9FLCxG7PGpq2NCY4hGhQ==", 2024-04-25T00:38:27.449 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.449 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:38:27.449 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:38:27.449 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.449 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:27.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:27.449 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-04-25T00:38:27.449 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmeR4UOBAANs9SRolOm6E8+/4ciupk5w==", 2024-04-25T00:38:27.449 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:38:27.449 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:38:27.450 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:38:27.450 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:38:27.450 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.450 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:27.450 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:38:27.450 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:38:27.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.435+0000 7f62982b6640 1 -- 172.21.15.29:0/1871441223 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6274067cb0 msgr2=0x7f627406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.435+0000 7f62982b6640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6274067cb0 0x7f627406a170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f628c0012f0 tx=0x7f628c005ec0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.435+0000 7f62982b6640 1 -- 172.21.15.29:0/1871441223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62900d0ff0 msgr2=0x7f6290155ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.435+0000 7f62982b6640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62900d0ff0 0x7f6290155ac0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f62800ad040 tx=0x7f62800192c0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.435+0000 7f62982b6640 1 -- 172.21.15.29:0/1871441223 shutdown_connections 2024-04-25T00:38:27.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.435+0000 7f62982b6640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6274067cb0 0x7f627406a170 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.435+0000 7f62982b6640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f629015e440 0x7f6290156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.435+0000 7f62982b6640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62900d0ff0 0x7f6290155ac0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.435+0000 7f62982b6640 1 --2- 172.21.15.29:0/1871441223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629006c950 0x7f6290153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.435+0000 7f62982b6640 1 -- 172.21.15.29:0/1871441223 >> 172.21.15.29:0/1871441223 conn(0x7f62900d1d60 msgr2=0x7f62900d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:27.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.435+0000 7f62982b6640 1 -- 172.21.15.29:0/1871441223 shutdown_connections 2024-04-25T00:38:27.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.435+0000 7f62982b6640 1 -- 172.21.15.29:0/1871441223 wait complete. 2024-04-25T00:38:27.455 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:27.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439fe52640 1 Processor -- start 2024-04-25T00:38:27.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439fe52640 1 -- start start 2024-04-25T00:38:27.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439fe52640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f439815c220 0x7f43980cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439fe52640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f43980cfa10 0x7f43980cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439fe52640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43980cdbd0 con 0x7f43980d04a0 2024-04-25T00:38:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439fe52640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f43980d0900 con 0x7f43980cfa10 2024-04-25T00:38:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439fe52640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f439815c600 con 0x7f439815c220 2024-04-25T00:38:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439e3ef640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f43980d04a0 0x7f43980ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56114/0 (socket says 172.21.15.29:56114) 2024-04-25T00:38:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439e3ef640 1 -- 172.21.15.29:0/4214366352 learned_addr learned my addr 172.21.15.29:0/4214366352 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439d3ed640 1 --2- 172.21.15.29:0/4214366352 >> v2:172.21.15.29:3300/0 conn(0x7f43980cfa10 0x7f43980cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439dbee640 1 --2- 172.21.15.29:0/4214366352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f439815c220 0x7f43980cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439d3ed640 1 -- 172.21.15.29:0/4214366352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f439815c220 msgr2=0x7f43980cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439d3ed640 1 --2- 172.21.15.29:0/4214366352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f439815c220 0x7f43980cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439d3ed640 1 -- 172.21.15.29:0/4214366352 >> v1:172.21.15.29:6789/0 conn(0x7f43980d04a0 legacy=0x7f43980ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.567+0000 7f439d3ed640 1 -- 172.21.15.29:0/4214366352 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43980b13e0 con 0x7f43980cfa10 2024-04-25T00:38:27.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439d3ed640 1 --2- 172.21.15.29:0/4214366352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43980cfa10 0x7f43980cff60 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f43940097c0 tx=0x7f439401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cb9e45a6daf4f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:27.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439cbec640 1 -- 172.21.15.29:0/4214366352 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43940b0070 con 0x7f43980cfa10 2024-04-25T00:38:27.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439cbec640 1 -- 172.21.15.29:0/4214366352 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43940ab600 con 0x7f43980cfa10 2024-04-25T00:38:27.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439cbec640 1 -- 172.21.15.29:0/4214366352 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43940ab9d0 con 0x7f43980cfa10 2024-04-25T00:38:27.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 -- 172.21.15.29:0/4214366352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43980cfa10 msgr2=0x7f43980cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 --2- 172.21.15.29:0/4214366352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43980cfa10 0x7f43980cff60 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f43940097c0 tx=0x7f439401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 -- 172.21.15.29:0/4214366352 shutdown_connections 2024-04-25T00:38:27.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 --2- 172.21.15.29:0/4214366352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43980cfa10 0x7f43980cff60 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 --2- 172.21.15.29:0/4214366352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f439815c220 0x7f43980cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 -- 172.21.15.29:0/4214366352 >> 172.21.15.29:0/4214366352 conn(0x7f43980d1d60 msgr2=0x7f43980d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:27.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 -- 172.21.15.29:0/4214366352 shutdown_connections 2024-04-25T00:38:27.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 -- 172.21.15.29:0/4214366352 wait complete. 2024-04-25T00:38:27.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 Processor -- start 2024-04-25T00:38:27.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 -- start start 2024-04-25T00:38:27.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43980cfa10 0x7f439815d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43980d04a0 0x7f4398159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f439815c220 0x7f43980c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4398169090 con 0x7f43980d04a0 2024-04-25T00:38:27.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4398168f10 con 0x7f43980cfa10 2024-04-25T00:38:27.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439fe52640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4398169210 con 0x7f439815c220 2024-04-25T00:38:27.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439d3ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43980d04a0 0x7f4398159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.571+0000 7f439d3ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43980d04a0 0x7f4398159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51570/0 (socket says 172.21.15.29:51570) 2024-04-25T00:38:27.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f439d3ed640 1 -- 172.21.15.29:0/394284333 learned_addr learned my addr 172.21.15.29:0/394284333 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:27.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f439d3ed640 1 -- 172.21.15.29:0/394284333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f439815c220 msgr2=0x7f43980c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f439e3ef640 1 --2- 172.21.15.29:0/394284333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f439815c220 0x7f43980c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f439d3ed640 1 --2- 172.21.15.29:0/394284333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f439815c220 0x7f43980c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f439d3ed640 1 -- 172.21.15.29:0/394284333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43980cfa10 msgr2=0x7f439815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f439d3ed640 1 --2- 172.21.15.29:0/394284333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43980cfa10 0x7f439815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f439d3ed640 1 -- 172.21.15.29:0/394284333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439806c500 con 0x7f43980d04a0 2024-04-25T00:38:27.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f439d3ed640 1 --2- 172.21.15.29:0/394284333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43980d04a0 0x7f4398159bc0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f43940ae040 tx=0x7f4394019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:27.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f438e7fc640 1 -- 172.21.15.29:0/394284333 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43940b0040 con 0x7f43980d04a0 2024-04-25T00:38:27.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f438e7fc640 1 -- 172.21.15.29:0/394284333 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43940ab400 con 0x7f43980d04a0 2024-04-25T00:38:27.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f439fe52640 1 -- 172.21.15.29:0/394284333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439806c7e0 con 0x7f43980d04a0 2024-04-25T00:38:27.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f438e7fc640 1 -- 172.21.15.29:0/394284333 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43940b4610 con 0x7f43980d04a0 2024-04-25T00:38:27.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f439fe52640 1 -- 172.21.15.29:0/394284333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439806cda0 con 0x7f43980d04a0 2024-04-25T00:38:27.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f439fe52640 1 -- 172.21.15.29:0/394284333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f439806d0d0 con 0x7f43980d04a0 2024-04-25T00:38:27.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f438e7fc640 1 -- 172.21.15.29:0/394284333 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f43940bb030 con 0x7f43980d04a0 2024-04-25T00:38:27.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f438e7fc640 1 --2- 172.21.15.29:0/394284333 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f437c067cb0 0x7f437c06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f438e7fc640 1 -- 172.21.15.29:0/394284333 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4857+0+0 (secure 0 0 0) 0x7f4394120010 con 0x7f43980d04a0 2024-04-25T00:38:27.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f439dbee640 1 --2- 172.21.15.29:0/394284333 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f437c067cb0 0x7f437c06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.575+0000 7f439dbee640 1 --2- 172.21.15.29:0/394284333 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f437c067cb0 0x7f437c06a170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f438800b850 tx=0x7f4388008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.579+0000 7f438e7fc640 1 -- 172.21.15.29:0/394284333 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f43940c2020 con 0x7f43980d04a0 2024-04-25T00:38:27.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.739+0000 7f439fe52640 1 -- 172.21.15.29:0/394284333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4398069270 con 0x7f43980d04a0 2024-04-25T00:38:27.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.743+0000 7f438e7fc640 1 -- 172.21.15.29:0/394284333 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+12853 (secure 0 0 0) 0x7f4394016090 con 0x7f43980d04a0 2024-04-25T00:38:27.745 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:27.746 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":114,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:26.998119+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:27.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.743+0000 7f439fe52640 1 -- 172.21.15.29:0/394284333 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f437c067cb0 msgr2=0x7f437c06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.743+0000 7f439fe52640 1 --2- 172.21.15.29:0/394284333 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f437c067cb0 0x7f437c06a170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f438800b850 tx=0x7f4388008040 comp rx=0 tx=0).stop 2024-04-25T00:38:27.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.743+0000 7f439fe52640 1 -- 172.21.15.29:0/394284333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43980d04a0 msgr2=0x7f4398159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.743+0000 7f439fe52640 1 --2- 172.21.15.29:0/394284333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43980d04a0 0x7f4398159bc0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f43940ae040 tx=0x7f4394019240 comp rx=0 tx=0).stop 2024-04-25T00:38:27.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.747+0000 7f439fe52640 1 -- 172.21.15.29:0/394284333 shutdown_connections 2024-04-25T00:38:27.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.747+0000 7f439fe52640 1 --2- 172.21.15.29:0/394284333 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f437c067cb0 0x7f437c06a170 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.747+0000 7f439fe52640 1 --2- 172.21.15.29:0/394284333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f439815c220 0x7f43980c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.747+0000 7f439fe52640 1 --2- 172.21.15.29:0/394284333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43980d04a0 0x7f4398159bc0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.747+0000 7f439fe52640 1 --2- 172.21.15.29:0/394284333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43980cfa10 0x7f439815d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.747+0000 7f439fe52640 1 -- 172.21.15.29:0/394284333 >> 172.21.15.29:0/394284333 conn(0x7f43980d1d60 msgr2=0x7f4398154a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:27.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.747+0000 7f439fe52640 1 -- 172.21.15.29:0/394284333 shutdown_connections 2024-04-25T00:38:27.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.747+0000 7f439fe52640 1 -- 172.21.15.29:0/394284333 wait complete. 2024-04-25T00:38:27.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:38:27.764 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:38:27.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf8329b640 1 Processor -- start 2024-04-25T00:38:27.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf8329b640 1 -- start start 2024-04-25T00:38:27.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf8329b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf7c0b4780 0x7faf7c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf8329b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faf7c0cc240 0x7faf7c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf8329b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf7c0b3720 con 0x7faf7c15e8d0 2024-04-25T00:38:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf8329b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7faf7c0cc650 con 0x7faf7c0cc240 2024-04-25T00:38:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf8329b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faf7c0b42f0 con 0x7faf7c0b4780 2024-04-25T00:38:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf81838640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faf7c0cc240 0x7faf7c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf81037640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7faf7c15e8d0 0x7faf7c15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40908/0 (socket says 172.21.15.29:40908) 2024-04-25T00:38:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf81037640 1 -- 172.21.15.29:0/4980477 learned_addr learned my addr 172.21.15.29:0/4980477 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf80836640 1 --2- 172.21.15.29:0/4980477 >> v2:172.21.15.157:3300/0 conn(0x7faf7c0b4780 0x7faf7c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf81838640 1 -- 172.21.15.29:0/4980477 >> v2:172.21.15.157:3300/0 conn(0x7faf7c0b4780 msgr2=0x7faf7c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf81838640 1 --2- 172.21.15.29:0/4980477 >> v2:172.21.15.157:3300/0 conn(0x7faf7c0b4780 0x7faf7c15f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf81838640 1 -- 172.21.15.29:0/4980477 >> v1:172.21.15.29:6789/0 conn(0x7faf7c15e8d0 legacy=0x7faf7c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.891+0000 7faf81838640 1 -- 172.21.15.29:0/4980477 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c0b39c0 con 0x7faf7c0cc240 2024-04-25T00:38:27.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf81838640 1 --2- 172.21.15.29:0/4980477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf7c0cc240 0x7faf7c15c7d0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7faf7c070910 tx=0x7faf7401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b43fc2a4bb111d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:27.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf73fff640 1 -- 172.21.15.29:0/4980477 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf740b2070 con 0x7faf7c0cc240 2024-04-25T00:38:27.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf73fff640 1 -- 172.21.15.29:0/4980477 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf740a59c0 con 0x7faf7c0cc240 2024-04-25T00:38:27.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf73fff640 1 -- 172.21.15.29:0/4980477 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf740a5d90 con 0x7faf7c0cc240 2024-04-25T00:38:27.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 -- 172.21.15.29:0/4980477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf7c0cc240 msgr2=0x7faf7c15c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 --2- 172.21.15.29:0/4980477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf7c0cc240 0x7faf7c15c7d0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7faf7c070910 tx=0x7faf7401c900 comp rx=0 tx=0).stop 2024-04-25T00:38:27.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 -- 172.21.15.29:0/4980477 shutdown_connections 2024-04-25T00:38:27.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 --2- 172.21.15.29:0/4980477 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf7c0cc240 0x7faf7c15c7d0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 --2- 172.21.15.29:0/4980477 >> v2:172.21.15.157:3300/0 conn(0x7faf7c0b4780 0x7faf7c15f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 -- 172.21.15.29:0/4980477 >> 172.21.15.29:0/4980477 conn(0x7faf7c0b1870 msgr2=0x7faf7c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:27.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 -- 172.21.15.29:0/4980477 shutdown_connections 2024-04-25T00:38:27.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 -- 172.21.15.29:0/4980477 wait complete. 2024-04-25T00:38:27.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 Processor -- start 2024-04-25T00:38:27.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 -- start start 2024-04-25T00:38:27.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf7c0b4780 0x7faf7c134970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7c0cc240 0x7faf7c15a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf7c15e8d0 0x7faf7c0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c144a20 con 0x7faf7c15e8d0 2024-04-25T00:38:27.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c15d520 con 0x7faf7c0b4780 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf8329b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faf7c1442c0 con 0x7faf7c0cc240 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf81838640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf7c15e8d0 0x7faf7c0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf81838640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf7c15e8d0 0x7faf7c0cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53892/0 (socket says 172.21.15.29:53892) 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf81838640 1 -- 172.21.15.29:0/380920039 learned_addr learned my addr 172.21.15.29:0/380920039 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf81037640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf7c0b4780 0x7faf7c134970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf80836640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7c0cc240 0x7faf7c15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf81838640 1 -- 172.21.15.29:0/380920039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7c0cc240 msgr2=0x7faf7c15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf81838640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7c0cc240 0x7faf7c15a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf81838640 1 -- 172.21.15.29:0/380920039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf7c0b4780 msgr2=0x7faf7c134970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf81838640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf7c0b4780 0x7faf7c134970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.895+0000 7faf81838640 1 -- 172.21.15.29:0/380920039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c0cd180 con 0x7faf7c15e8d0 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.899+0000 7faf81838640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf7c15e8d0 0x7faf7c0cec60 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7faf740b0040 tx=0x7faf740196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.899+0000 7faf71ffb640 1 -- 172.21.15.29:0/380920039 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf740b2050 con 0x7faf7c15e8d0 2024-04-25T00:38:27.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.899+0000 7faf71ffb640 1 -- 172.21.15.29:0/380920039 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf740be070 con 0x7faf7c15e8d0 2024-04-25T00:38:27.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.899+0000 7faf71ffb640 1 -- 172.21.15.29:0/380920039 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf740adb70 con 0x7faf7c15e8d0 2024-04-25T00:38:27.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.899+0000 7faf8329b640 1 -- 172.21.15.29:0/380920039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7c0cd460 con 0x7faf7c15e8d0 2024-04-25T00:38:27.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.899+0000 7faf8329b640 1 -- 172.21.15.29:0/380920039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7c0cda20 con 0x7faf7c15e8d0 2024-04-25T00:38:27.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.899+0000 7faf8329b640 1 -- 172.21.15.29:0/380920039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7c069270 con 0x7faf7c15e8d0 2024-04-25T00:38:27.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.899+0000 7faf71ffb640 1 -- 172.21.15.29:0/380920039 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7faf740add10 con 0x7faf7c15e8d0 2024-04-25T00:38:27.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.899+0000 7faf71ffb640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faf60067ce0 0x7faf6006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:27.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.899+0000 7faf71ffb640 1 -- 172.21.15.29:0/380920039 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4857+0+0 (secure 0 0 0) 0x7faf74120f10 con 0x7faf7c15e8d0 2024-04-25T00:38:27.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.899+0000 7faf81037640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faf60067ce0 0x7faf6006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:27.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.899+0000 7faf81037640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faf60067ce0 0x7faf6006a1a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf7c13ede0 tx=0x7faf78009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:27.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:27.903+0000 7faf71ffb640 1 -- 172.21.15.29:0/380920039 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faf740c7020 con 0x7faf7c15e8d0 2024-04-25T00:38:28.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.059+0000 7faf8329b640 1 -- 172.21.15.29:0/380920039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faf7c06e7b0 con 0x7faf7c15e8d0 2024-04-25T00:38:28.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.059+0000 7faf71ffb640 1 -- 172.21.15.29:0/380920039 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) ==== 93+0+10 (secure 0 0 0) 0x7faf74016520 con 0x7faf7c15e8d0 2024-04-25T00:38:28.062 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:38:28.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.063+0000 7faf8329b640 1 -- 172.21.15.29:0/380920039 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faf60067ce0 msgr2=0x7faf6006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.063+0000 7faf8329b640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faf60067ce0 0x7faf6006a1a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf7c13ede0 tx=0x7faf78009040 comp rx=0 tx=0).stop 2024-04-25T00:38:28.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.063+0000 7faf8329b640 1 -- 172.21.15.29:0/380920039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf7c15e8d0 msgr2=0x7faf7c0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.063+0000 7faf8329b640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf7c15e8d0 0x7faf7c0cec60 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7faf740b0040 tx=0x7faf740196a0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.063+0000 7faf8329b640 1 -- 172.21.15.29:0/380920039 shutdown_connections 2024-04-25T00:38:28.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.063+0000 7faf8329b640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faf60067ce0 0x7faf6006a1a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.063+0000 7faf8329b640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf7c15e8d0 0x7faf7c0cec60 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.063+0000 7faf8329b640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf7c0cc240 0x7faf7c15a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.063+0000 7faf8329b640 1 --2- 172.21.15.29:0/380920039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf7c0b4780 0x7faf7c134970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.063+0000 7faf8329b640 1 -- 172.21.15.29:0/380920039 >> 172.21.15.29:0/380920039 conn(0x7faf7c0b1870 msgr2=0x7faf7c0af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:28.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.063+0000 7faf8329b640 1 -- 172.21.15.29:0/380920039 shutdown_connections 2024-04-25T00:38:28.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.063+0000 7faf8329b640 1 -- 172.21.15.29:0/380920039 wait complete. 2024-04-25T00:38:28.080 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:38:28.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.195+0000 7ff0a177c640 1 Processor -- start 2024-04-25T00:38:28.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff0a177c640 1 -- start start 2024-04-25T00:38:28.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff0a177c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff09c0b3140 0x7ff09c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff0a177c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff09c0b3c30 0x7ff09c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff0a177c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09c0b41e0 con 0x7ff09c0b4b10 2024-04-25T00:38:28.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff0a177c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff09c06d960 con 0x7ff09c0b3c30 2024-04-25T00:38:28.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff0a177c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff09c06dc40 con 0x7ff09c0b3140 2024-04-25T00:38:28.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff09b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff09c0b3c30 0x7ff09c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff09a7fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff09c0b4b10 0x7ff09c0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40914/0 (socket says 172.21.15.29:40914) 2024-04-25T00:38:28.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff09a7fc640 1 -- 172.21.15.29:0/2964949244 learned_addr learned my addr 172.21.15.29:0/2964949244 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:28.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff09affd640 1 --2- 172.21.15.29:0/2964949244 >> v2:172.21.15.157:3300/0 conn(0x7ff09c0b3140 0x7ff09c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff09b7fe640 1 -- 172.21.15.29:0/2964949244 >> v2:172.21.15.157:3300/0 conn(0x7ff09c0b3140 msgr2=0x7ff09c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:28.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff09b7fe640 1 --2- 172.21.15.29:0/2964949244 >> v2:172.21.15.157:3300/0 conn(0x7ff09c0b3140 0x7ff09c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff09b7fe640 1 -- 172.21.15.29:0/2964949244 >> v1:172.21.15.29:6789/0 conn(0x7ff09c0b4b10 legacy=0x7ff09c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff09b7fe640 1 -- 172.21.15.29:0/2964949244 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c06df20 con 0x7ff09c0b3c30 2024-04-25T00:38:28.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff099ffb640 1 -- 172.21.15.29:0/2964949244 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423291152 0 0) 0x7ff09c0b41e0 con 0x7ff09c0b4b10 2024-04-25T00:38:28.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff09b7fe640 1 --2- 172.21.15.29:0/2964949244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff09c0b3c30 0x7ff09c0ce240 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff0840097c0 tx=0x7ff08401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165650249ff7ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:28.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff099ffb640 1 -- 172.21.15.29:0/2964949244 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0840b0070 con 0x7ff09c0b3c30 2024-04-25T00:38:28.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff099ffb640 1 -- 172.21.15.29:0/2964949244 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0840ab600 con 0x7ff09c0b3c30 2024-04-25T00:38:28.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.199+0000 7ff099ffb640 1 -- 172.21.15.29:0/2964949244 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0840ab9d0 con 0x7ff09c0b3c30 2024-04-25T00:38:28.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 -- 172.21.15.29:0/2964949244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff09c0b3c30 msgr2=0x7ff09c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 --2- 172.21.15.29:0/2964949244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff09c0b3c30 0x7ff09c0ce240 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff0840097c0 tx=0x7ff08401c460 comp rx=0 tx=0).stop 2024-04-25T00:38:28.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 -- 172.21.15.29:0/2964949244 shutdown_connections 2024-04-25T00:38:28.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 --2- 172.21.15.29:0/2964949244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff09c0b3c30 0x7ff09c0ce240 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 --2- 172.21.15.29:0/2964949244 >> v2:172.21.15.157:3300/0 conn(0x7ff09c0b3140 0x7ff09c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 -- 172.21.15.29:0/2964949244 >> 172.21.15.29:0/2964949244 conn(0x7ff09c0b1870 msgr2=0x7ff09c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:28.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 -- 172.21.15.29:0/2964949244 shutdown_connections 2024-04-25T00:38:28.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 -- 172.21.15.29:0/2964949244 wait complete. 2024-04-25T00:38:28.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 Processor -- start 2024-04-25T00:38:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 -- start start 2024-04-25T00:38:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff09c0b3140 0x7ff09c138920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff09c0b3c30 0x7ff09c13b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff09c0b4b10 0x7ff09c13bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c168d80 con 0x7ff09c0b3140 2024-04-25T00:38:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c168c00 con 0x7ff09c0b3c30 2024-04-25T00:38:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c168f00 con 0x7ff09c0b4b10 2024-04-25T00:38:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff09affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff09c0b3140 0x7ff09c138920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff09affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff09c0b3140 0x7ff09c138920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53904/0 (socket says 172.21.15.29:53904) 2024-04-25T00:38:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff09affd640 1 -- 172.21.15.29:0/1010305933 learned_addr learned my addr 172.21.15.29:0/1010305933 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff09a7fc640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff09c0b3c30 0x7ff09c13b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff09b7fe640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff09c0b4b10 0x7ff09c13bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff09affd640 1 -- 172.21.15.29:0/1010305933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff09c0b4b10 msgr2=0x7ff09c13bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff09affd640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff09c0b4b10 0x7ff09c13bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff09affd640 1 -- 172.21.15.29:0/1010305933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff09c0b3c30 msgr2=0x7ff09c13b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff09affd640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff09c0b3c30 0x7ff09c13b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff09affd640 1 -- 172.21.15.29:0/1010305933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c158c00 con 0x7ff09c0b3140 2024-04-25T00:38:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff09affd640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff09c0b3140 0x7ff09c138920 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00d620 tx=0x7ff08c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0937fe640 1 -- 172.21.15.29:0/1010305933 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00b620 con 0x7ff09c0b3140 2024-04-25T00:38:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0937fe640 1 -- 172.21.15.29:0/1010305933 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff08c019070 con 0x7ff09c0b3140 2024-04-25T00:38:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 -- 172.21.15.29:0/1010305933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09c158ee0 con 0x7ff09c0b3140 2024-04-25T00:38:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0937fe640 1 -- 172.21.15.29:0/1010305933 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00b9e0 con 0x7ff09c0b3140 2024-04-25T00:38:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.203+0000 7ff0a177c640 1 -- 172.21.15.29:0/1010305933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09c13d950 con 0x7ff09c0b3140 2024-04-25T00:38:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.207+0000 7ff0a177c640 1 -- 172.21.15.29:0/1010305933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09c069270 con 0x7ff09c0b3140 2024-04-25T00:38:28.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.207+0000 7ff0937fe640 1 -- 172.21.15.29:0/1010305933 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7ff08c00bb80 con 0x7ff09c0b3140 2024-04-25T00:38:28.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.207+0000 7ff0937fe640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff080067cb0 0x7ff08006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.207+0000 7ff0937fe640 1 -- 172.21.15.29:0/1010305933 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4857+0+0 (secure 0 0 0) 0x7ff08c0862a0 con 0x7ff09c0b3140 2024-04-25T00:38:28.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.207+0000 7ff09a7fc640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff080067cb0 0x7ff08006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.207+0000 7ff09a7fc640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff080067cb0 0x7ff08006a170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff094007fd0 tx=0x7ff094007ee0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:28.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.211+0000 7ff0937fe640 1 -- 172.21.15.29:0/1010305933 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff08c056ce0 con 0x7ff09c0b3140 2024-04-25T00:38:28.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.371+0000 7ff0a177c640 1 -- 172.21.15.29:0/1010305933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff09c15b590 con 0x7ff09c0b3140 2024-04-25T00:38:28.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.371+0000 7ff0937fe640 1 -- 172.21.15.29:0/1010305933 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v114) ==== 84+0+65 (secure 0 0 0) 0x7ff08c0565e0 con 0x7ff09c0b3140 2024-04-25T00:38:28.375 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:28.375 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:38:28.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:28.375 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-04-25T00:38:28.375 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": ".mgr" 2024-04-25T00:38:28.376 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:28.376 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:38:28.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.375+0000 7ff0a177c640 1 -- 172.21.15.29:0/1010305933 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff080067cb0 msgr2=0x7ff08006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.375+0000 7ff0a177c640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff080067cb0 0x7ff08006a170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff094007fd0 tx=0x7ff094007ee0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.375+0000 7ff0a177c640 1 -- 172.21.15.29:0/1010305933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff09c0b3140 msgr2=0x7ff09c138920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.375+0000 7ff0a177c640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff09c0b3140 0x7ff09c138920 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00d620 tx=0x7ff08c00dae0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.375+0000 7ff0a177c640 1 -- 172.21.15.29:0/1010305933 shutdown_connections 2024-04-25T00:38:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.375+0000 7ff0a177c640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff080067cb0 0x7ff08006a170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.375+0000 7ff0a177c640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff09c0b4b10 0x7ff09c13bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.375+0000 7ff0a177c640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff09c0b3c30 0x7ff09c13b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.375+0000 7ff0a177c640 1 --2- 172.21.15.29:0/1010305933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff09c0b3140 0x7ff09c138920 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.375+0000 7ff0a177c640 1 -- 172.21.15.29:0/1010305933 >> 172.21.15.29:0/1010305933 conn(0x7ff09c0b1870 msgr2=0x7ff09c06cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.375+0000 7ff0a177c640 1 -- 172.21.15.29:0/1010305933 shutdown_connections 2024-04-25T00:38:28.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.375+0000 7ff0a177c640 1 -- 172.21.15.29:0/1010305933 wait complete. 2024-04-25T00:38:28.395 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:38:28.395 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:38:28.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2de61f6640 1 Processor -- start 2024-04-25T00:38:28.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2de61f6640 1 -- start start 2024-04-25T00:38:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2de61f6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2de015cc40 0x7f2de0071e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2de61f6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2de0072750 con 0x7f2de0147c90 2024-04-25T00:38:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2de61f6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2de0072a30 con 0x7f2de00b5bb0 2024-04-25T00:38:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2de61f6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2de0166fd0 con 0x7f2de015cc40 2024-04-25T00:38:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2ddffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2de0147c90 0x7f2de0162a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40926/0 (socket says 172.21.15.29:40926) 2024-04-25T00:38:28.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2ddffff640 1 -- 172.21.15.29:0/93717714 learned_addr learned my addr 172.21.15.29:0/93717714 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:28.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2ddf7fe640 1 --2- 172.21.15.29:0/93717714 >> v2:172.21.15.157:3300/0 conn(0x7f2de015cc40 0x7f2de0071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3031194762 0 0) 0x7f2de0072750 con 0x7f2de0147c90 2024-04-25T00:38:28.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2dc8003530 con 0x7f2de0147c90 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4021939820 0 0) 0x7f2dc8003530 con 0x7f2de0147c90 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 >> v2:172.21.15.157:3300/0 conn(0x7f2de015cc40 msgr2=0x7f2de0071e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 --2- 172.21.15.29:0/93717714 >> v2:172.21.15.157:3300/0 conn(0x7f2de015cc40 0x7f2de0071e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 >> v1:172.21.15.157:6789/0 conn(0x7f2de00b5bb0 legacy=0x7f2de01668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de01660c0 con 0x7f2de0147c90 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2dd0001680 con 0x7f2de0147c90 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 >> v1:172.21.15.29:6789/0 conn(0x7f2de0147c90 legacy=0x7f2de0162a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 --2- 172.21.15.29:0/93717714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc80039c0 0x7f2dc8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 --2- 172.21.15.29:0/93717714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dc8004810 0x7f2dc8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 --2- 172.21.15.29:0/93717714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dc80051d0 0x7f2dc800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2de0166fd0 con 0x7f2dc80039c0 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc800c170 con 0x7f2dc8004810 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc800c370 con 0x7f2dc80051d0 2024-04-25T00:38:28.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2ddf7fe640 1 --2- 172.21.15.29:0/93717714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc80039c0 0x7f2dc8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2ddeffd640 1 --2- 172.21.15.29:0/93717714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dc8004810 0x7f2dc8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2ddf7fe640 1 -- 172.21.15.29:0/93717714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dc80051d0 msgr2=0x7f2dc800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:28.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2ddf7fe640 1 --2- 172.21.15.29:0/93717714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dc80051d0 0x7f2dc800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2ddf7fe640 1 -- 172.21.15.29:0/93717714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dc8004810 msgr2=0x7f2dc8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:28.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2ddf7fe640 1 --2- 172.21.15.29:0/93717714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dc8004810 0x7f2dc8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2ddf7fe640 1 -- 172.21.15.29:0/93717714 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc8003530 con 0x7f2dc80039c0 2024-04-25T00:38:28.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.523+0000 7f2ddffff640 1 -- 172.21.15.29:0/93717714 reap_dead start 2024-04-25T00:38:28.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.527+0000 7f2ddf7fe640 1 --2- 172.21.15.29:0/93717714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc80039c0 0x7f2dc8003de0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009fc0 tx=0x7f2dd801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4fc72bf7fcdd73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:28.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.527+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80b2070 con 0x7f2dc80039c0 2024-04-25T00:38:28.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.527+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd80a5b60 con 0x7f2dc80039c0 2024-04-25T00:38:28.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.527+0000 7f2dde7fc640 1 -- 172.21.15.29:0/93717714 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8009c10 con 0x7f2dc80039c0 2024-04-25T00:38:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.527+0000 7f2de61f6640 1 -- 172.21.15.29:0/93717714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc80039c0 msgr2=0x7f2dc8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.527+0000 7f2de61f6640 1 --2- 172.21.15.29:0/93717714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc80039c0 0x7f2dc8003de0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009fc0 tx=0x7f2dd801cca0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.527+0000 7f2de61f6640 1 -- 172.21.15.29:0/93717714 shutdown_connections 2024-04-25T00:38:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.527+0000 7f2de61f6640 1 --2- 172.21.15.29:0/93717714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc80039c0 0x7f2dc8003de0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.527+0000 7f2de61f6640 1 -- 172.21.15.29:0/93717714 >> 172.21.15.29:0/93717714 conn(0x7f2de00b2880 msgr2=0x7f2de00b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.527+0000 7f2de61f6640 1 -- 172.21.15.29:0/93717714 shutdown_connections 2024-04-25T00:38:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.527+0000 7f2de61f6640 1 -- 172.21.15.29:0/93717714 wait complete. 2024-04-25T00:38:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.527+0000 7f2de61f6640 1 Processor -- start 2024-04-25T00:38:28.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.527+0000 7f2de61f6640 1 -- start start 2024-04-25T00:38:28.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2de61f6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dc80039c0 0x7f2de015bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2de61f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de0161740 0x7f2de014b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2de61f6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2de01fb9a0 0x7f2de0146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2de61f6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01484b0 con 0x7f2de01fb9a0 2024-04-25T00:38:28.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2de61f6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2de0162a40 con 0x7f2de0161740 2024-04-25T00:38:28.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2de61f6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2de015c190 con 0x7f2dc80039c0 2024-04-25T00:38:28.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2ddffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2de01fb9a0 0x7f2de0146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2ddffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2de01fb9a0 0x7f2de0146a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53924/0 (socket says 172.21.15.29:53924) 2024-04-25T00:38:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2ddffff640 1 -- 172.21.15.29:0/2085510140 learned_addr learned my addr 172.21.15.29:0/2085510140 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2ddeffd640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de0161740 0x7f2de014b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2ddf7fe640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dc80039c0 0x7f2de015bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2ddffff640 1 -- 172.21.15.29:0/2085510140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dc80039c0 msgr2=0x7f2de015bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2ddffff640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dc80039c0 0x7f2de015bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2ddffff640 1 -- 172.21.15.29:0/2085510140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de0161740 msgr2=0x7f2de014b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2ddffff640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de0161740 0x7f2de014b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2ddffff640 1 -- 172.21.15.29:0/2085510140 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de0161c40 con 0x7f2de01fb9a0 2024-04-25T00:38:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2ddffff640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2de01fb9a0 0x7f2de0146a40 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0000f70 tx=0x7f2dd000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2dc7fff640 1 -- 172.21.15.29:0/2085510140 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0007440 con 0x7f2de01fb9a0 2024-04-25T00:38:28.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2dc7fff640 1 -- 172.21.15.29:0/2085510140 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd0007620 con 0x7f2de01fb9a0 2024-04-25T00:38:28.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2de61f6640 1 -- 172.21.15.29:0/2085510140 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de01ee240 con 0x7f2de01fb9a0 2024-04-25T00:38:28.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2dc7fff640 1 -- 172.21.15.29:0/2085510140 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0004a30 con 0x7f2de01fb9a0 2024-04-25T00:38:28.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2de61f6640 1 -- 172.21.15.29:0/2085510140 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de01ee800 con 0x7f2de01fb9a0 2024-04-25T00:38:28.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2de61f6640 1 -- 172.21.15.29:0/2085510140 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de00692e0 con 0x7f2de01fb9a0 2024-04-25T00:38:28.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2dc7fff640 1 -- 172.21.15.29:0/2085510140 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f2dd0011070 con 0x7f2de01fb9a0 2024-04-25T00:38:28.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2dc7fff640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2dcc067cb0 0x7f2dcc06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2dc7fff640 1 -- 172.21.15.29:0/2085510140 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4857+0+0 (secure 0 0 0) 0x7f2dd0085690 con 0x7f2de01fb9a0 2024-04-25T00:38:28.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.531+0000 7f2ddf7fe640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2dcc067cb0 0x7f2dcc06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.535+0000 7f2ddf7fe640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2dcc067cb0 0x7f2dcc06a170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80af000 tx=0x7f2dd80a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:28.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.535+0000 7f2dc7fff640 1 -- 172.21.15.29:0/2085510140 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2dd001a060 con 0x7f2de01fb9a0 2024-04-25T00:38:28.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:28.699+0000 7f2de61f6640 1 -- 172.21.15.29:0/2085510140 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2de01ee9d0 con 0x7f2de01fb9a0 2024-04-25T00:38:29.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.667+0000 7f2dc7fff640 1 -- 172.21.15.29:0/2085510140 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v115) ==== 138+0+0 (secure 0 0 0) 0x7f2dd0056090 con 0x7f2de01fb9a0 2024-04-25T00:38:29.670 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-04-25T00:38:29.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.671+0000 7f2de61f6640 1 -- 172.21.15.29:0/2085510140 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2dcc067cb0 msgr2=0x7f2dcc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:29.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.671+0000 7f2de61f6640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2dcc067cb0 0x7f2dcc06a170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80af000 tx=0x7f2dd80a7040 comp rx=0 tx=0).stop 2024-04-25T00:38:29.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.671+0000 7f2de61f6640 1 -- 172.21.15.29:0/2085510140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2de01fb9a0 msgr2=0x7f2de0146a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:29.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.671+0000 7f2de61f6640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2de01fb9a0 0x7f2de0146a40 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0000f70 tx=0x7f2dd000c960 comp rx=0 tx=0).stop 2024-04-25T00:38:29.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.671+0000 7f2de61f6640 1 -- 172.21.15.29:0/2085510140 shutdown_connections 2024-04-25T00:38:29.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.671+0000 7f2de61f6640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2dcc067cb0 0x7f2dcc06a170 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:29.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.671+0000 7f2de61f6640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2de01fb9a0 0x7f2de0146a40 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:29.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.671+0000 7f2de61f6640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de0161740 0x7f2de014b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:29.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.671+0000 7f2de61f6640 1 --2- 172.21.15.29:0/2085510140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dc80039c0 0x7f2de015bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:29.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.671+0000 7f2de61f6640 1 -- 172.21.15.29:0/2085510140 >> 172.21.15.29:0/2085510140 conn(0x7f2de00b2880 msgr2=0x7f2de0162dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:29.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.671+0000 7f2de61f6640 1 -- 172.21.15.29:0/2085510140 shutdown_connections 2024-04-25T00:38:29.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.671+0000 7f2de61f6640 1 -- 172.21.15.29:0/2085510140 wait complete. 2024-04-25T00:38:29.700 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:38:29.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f857a2ac640 1 Processor -- start 2024-04-25T00:38:29.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f857a2ac640 1 -- start start 2024-04-25T00:38:29.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f857a2ac640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f85740b4780 0x7f857415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:29.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f857a2ac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85740cc240 0x7f857415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:29.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f857a2ac640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85740b3720 con 0x7f857415e8d0 2024-04-25T00:38:29.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f857a2ac640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f85740b42f0 con 0x7f85740b4780 2024-04-25T00:38:29.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f857a2ac640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f85740b39c0 con 0x7f85740cc240 2024-04-25T00:38:29.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f856b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f85740b4780 0x7f857415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:29.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f856b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f85740b4780 0x7f857415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53930/0 (socket says 172.21.15.29:53930) 2024-04-25T00:38:29.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f856b7fe640 1 -- 172.21.15.29:0/3240311384 learned_addr learned my addr 172.21.15.29:0/3240311384 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:29.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f8578849640 1 --2- 172.21.15.29:0/3240311384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85740cc240 0x7f857415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:29.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f856b7fe640 1 -- 172.21.15.29:0/3240311384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85740cc240 msgr2=0x7f857415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:29.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f856b7fe640 1 --2- 172.21.15.29:0/3240311384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85740cc240 0x7f857415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:29.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f856b7fe640 1 -- 172.21.15.29:0/3240311384 >> v1:172.21.15.157:6789/0 conn(0x7f857415e8d0 legacy=0x7f857415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:29.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.831+0000 7f856b7fe640 1 -- 172.21.15.29:0/3240311384 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85740b3ca0 con 0x7f85740b4780 2024-04-25T00:38:29.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.835+0000 7f856b7fe640 1 --2- 172.21.15.29:0/3240311384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85740b4780 0x7f857415f3c0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f85740b4470 tx=0x7f855c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67a287602078099 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:29.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.835+0000 7f85737fe640 1 -- 172.21.15.29:0/3240311384 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0b2070 con 0x7f85740b4780 2024-04-25T00:38:29.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.835+0000 7f85737fe640 1 -- 172.21.15.29:0/3240311384 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855c0ad640 con 0x7f85740b4780 2024-04-25T00:38:29.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.835+0000 7f85737fe640 1 -- 172.21.15.29:0/3240311384 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0ada10 con 0x7f85740b4780 2024-04-25T00:38:29.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.835+0000 7f857a2ac640 1 -- 172.21.15.29:0/3240311384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85740b4780 msgr2=0x7f857415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:29.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.835+0000 7f857a2ac640 1 --2- 172.21.15.29:0/3240311384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85740b4780 0x7f857415f3c0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f85740b4470 tx=0x7f855c01c500 comp rx=0 tx=0).stop 2024-04-25T00:38:29.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.835+0000 7f857a2ac640 1 -- 172.21.15.29:0/3240311384 shutdown_connections 2024-04-25T00:38:29.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.835+0000 7f857a2ac640 1 --2- 172.21.15.29:0/3240311384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85740cc240 0x7f857415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:29.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.835+0000 7f857a2ac640 1 --2- 172.21.15.29:0/3240311384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85740b4780 0x7f857415f3c0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:29.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.835+0000 7f857a2ac640 1 -- 172.21.15.29:0/3240311384 >> 172.21.15.29:0/3240311384 conn(0x7f85740b1870 msgr2=0x7f85740b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:29.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.835+0000 7f857a2ac640 1 -- 172.21.15.29:0/3240311384 shutdown_connections 2024-04-25T00:38:29.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.835+0000 7f857a2ac640 1 -- 172.21.15.29:0/3240311384 wait complete. 2024-04-25T00:38:29.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f857a2ac640 1 Processor -- start 2024-04-25T00:38:29.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f857a2ac640 1 -- start start 2024-04-25T00:38:29.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f857a2ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85740b4780 0x7f857414b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:29.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f857a2ac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85740cc240 0x7f85741436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:29.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f857a2ac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f857415e8d0 0x7f857406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:29.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f857a2ac640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f857415d6a0 con 0x7f85740cc240 2024-04-25T00:38:29.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f857a2ac640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f857415d520 con 0x7f85740b4780 2024-04-25T00:38:29.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f857a2ac640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8574167c70 con 0x7f857415e8d0 2024-04-25T00:38:29.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f856b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85740cc240 0x7f85741436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:29.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f856b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85740cc240 0x7f85741436e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53934/0 (socket says 172.21.15.29:53934) 2024-04-25T00:38:29.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f856b7fe640 1 -- 172.21.15.29:0/2884818193 learned_addr learned my addr 172.21.15.29:0/2884818193 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:29.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f8578849640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f857415e8d0 0x7f857406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:29.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f8573fff640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85740b4780 0x7f857414b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:29.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f856b7fe640 1 -- 172.21.15.29:0/2884818193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f857415e8d0 msgr2=0x7f857406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:29.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f856b7fe640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f857415e8d0 0x7f857406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:29.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f856b7fe640 1 -- 172.21.15.29:0/2884818193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85740b4780 msgr2=0x7f857414b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:29.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f856b7fe640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85740b4780 0x7f857414b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:29.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f856b7fe640 1 -- 172.21.15.29:0/2884818193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8574142640 con 0x7f85740cc240 2024-04-25T00:38:29.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f8578849640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f857415e8d0 0x7f857406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:29.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f856b7fe640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85740cc240 0x7f85741436e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f855c0b0040 tx=0x7f855c0192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:29.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f85717fa640 1 -- 172.21.15.29:0/2884818193 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0bd070 con 0x7f85740cc240 2024-04-25T00:38:29.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f85717fa640 1 -- 172.21.15.29:0/2884818193 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855c0b6750 con 0x7f85740cc240 2024-04-25T00:38:29.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f857a2ac640 1 -- 172.21.15.29:0/2884818193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8574142920 con 0x7f85740cc240 2024-04-25T00:38:29.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f85717fa640 1 -- 172.21.15.29:0/2884818193 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0b2070 con 0x7f85740cc240 2024-04-25T00:38:29.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f857a2ac640 1 -- 172.21.15.29:0/2884818193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8574142ee0 con 0x7f85740cc240 2024-04-25T00:38:29.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.839+0000 7f857a2ac640 1 -- 172.21.15.29:0/2884818193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8574069270 con 0x7f85740cc240 2024-04-25T00:38:29.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.843+0000 7f85717fa640 1 -- 172.21.15.29:0/2884818193 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f855c0c0070 con 0x7f85740cc240 2024-04-25T00:38:29.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.843+0000 7f85717fa640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8554067cb0 0x7f855406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:29.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.843+0000 7f8573fff640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8554067cb0 0x7f855406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:29.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.843+0000 7f85717fa640 1 -- 172.21.15.29:0/2884818193 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5227+0+0 (secure 0 0 0) 0x7f855c11fd70 con 0x7f85740cc240 2024-04-25T00:38:29.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.843+0000 7f8573fff640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8554067cb0 0x7f855406a170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f856400b810 tx=0x7f8564008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:29.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:29.847+0000 7f85717fa640 1 -- 172.21.15.29:0/2884818193 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f855c0160f0 con 0x7f85740cc240 2024-04-25T00:38:30.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.007+0000 7f857a2ac640 1 -- 172.21.15.29:0/2884818193 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f857414b980 con 0x7f85740cc240 2024-04-25T00:38:30.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.671+0000 7f85717fa640 1 -- 172.21.15.29:0/2884818193 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v116) ==== 170+0+0 (secure 0 0 0) 0x7f855c0f0750 con 0x7f85740cc240 2024-04-25T00:38:30.674 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-04-25T00:38:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.675+0000 7f857a2ac640 1 -- 172.21.15.29:0/2884818193 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8554067cb0 msgr2=0x7f855406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.675+0000 7f857a2ac640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8554067cb0 0x7f855406a170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f856400b810 tx=0x7f8564008040 comp rx=0 tx=0).stop 2024-04-25T00:38:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.675+0000 7f857a2ac640 1 -- 172.21.15.29:0/2884818193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85740cc240 msgr2=0x7f85741436e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.675+0000 7f857a2ac640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85740cc240 0x7f85741436e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f855c0b0040 tx=0x7f855c0192a0 comp rx=0 tx=0).stop 2024-04-25T00:38:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.675+0000 7f857a2ac640 1 -- 172.21.15.29:0/2884818193 shutdown_connections 2024-04-25T00:38:30.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.675+0000 7f857a2ac640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8554067cb0 0x7f855406a170 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:30.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.675+0000 7f857a2ac640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f857415e8d0 0x7f857406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:30.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.675+0000 7f857a2ac640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f85740cc240 0x7f85741436e0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:30.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.675+0000 7f857a2ac640 1 --2- 172.21.15.29:0/2884818193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85740b4780 0x7f857414b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:30.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.675+0000 7f857a2ac640 1 -- 172.21.15.29:0/2884818193 >> 172.21.15.29:0/2884818193 conn(0x7f85740b1870 msgr2=0x7f85740b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:30.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.675+0000 7f857a2ac640 1 -- 172.21.15.29:0/2884818193 shutdown_connections 2024-04-25T00:38:30.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.675+0000 7f857a2ac640 1 -- 172.21.15.29:0/2884818193 wait complete. 2024-04-25T00:38:30.716 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:38:30.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb57a252640 1 Processor -- start 2024-04-25T00:38:30.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb57a252640 1 -- start start 2024-04-25T00:38:30.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb57a252640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb5740b3140 0x7fb57406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:30.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb57a252640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb5740b4b10 0x7fb5740b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:30.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb57a252640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5740b3a60 0x7fb57415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:30.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb57a252640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb5740b41e0 con 0x7fb5740b3140 2024-04-25T00:38:30.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb57a252640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb57406d960 con 0x7fb5740b4b10 2024-04-25T00:38:30.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb57a252640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb57406dc40 con 0x7fb5740b3a60 2024-04-25T00:38:30.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb5737fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb5740b3140 0x7fb57406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:30.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb5737fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb5740b3140 0x7fb57406e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53942/0 (socket says 172.21.15.29:53942) 2024-04-25T00:38:30.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb5737fe640 1 -- 172.21.15.29:0/3422126371 learned_addr learned my addr 172.21.15.29:0/3422126371 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:30.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb573fff640 1 --2- 172.21.15.29:0/3422126371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5740b3a60 0x7fb57415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:30.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb572ffd640 1 --2- 172.21.15.29:0/3422126371 >> v2:172.21.15.157:3300/0 conn(0x7fb5740b4b10 0x7fb5740b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:30.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb5737fe640 1 -- 172.21.15.29:0/3422126371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5740b3a60 msgr2=0x7fb57415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:30.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb5737fe640 1 --2- 172.21.15.29:0/3422126371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5740b3a60 0x7fb57415c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:30.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb5737fe640 1 -- 172.21.15.29:0/3422126371 >> v2:172.21.15.157:3300/0 conn(0x7fb5740b4b10 msgr2=0x7fb5740b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:30.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb5737fe640 1 --2- 172.21.15.29:0/3422126371 >> v2:172.21.15.157:3300/0 conn(0x7fb5740b4b10 0x7fb5740b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:30.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.843+0000 7fb5737fe640 1 -- 172.21.15.29:0/3422126371 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57406df20 con 0x7fb5740b3140 2024-04-25T00:38:30.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb5737fe640 1 --2- 172.21.15.29:0/3422126371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5740b3140 0x7fb57406e440 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb568009870 tx=0x7fb56801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fb13880395f639 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:30.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb5727fc640 1 -- 172.21.15.29:0/3422126371 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5680b0070 con 0x7fb5740b3140 2024-04-25T00:38:30.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb5727fc640 1 -- 172.21.15.29:0/3422126371 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5680ab600 con 0x7fb5740b3140 2024-04-25T00:38:30.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb5727fc640 1 -- 172.21.15.29:0/3422126371 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5680ab9d0 con 0x7fb5740b3140 2024-04-25T00:38:30.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 -- 172.21.15.29:0/3422126371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5740b3140 msgr2=0x7fb57406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:30.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 --2- 172.21.15.29:0/3422126371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5740b3140 0x7fb57406e440 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb568009870 tx=0x7fb56801c520 comp rx=0 tx=0).stop 2024-04-25T00:38:30.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 -- 172.21.15.29:0/3422126371 shutdown_connections 2024-04-25T00:38:30.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 --2- 172.21.15.29:0/3422126371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5740b3a60 0x7fb57415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:30.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 --2- 172.21.15.29:0/3422126371 >> v2:172.21.15.157:3300/0 conn(0x7fb5740b4b10 0x7fb5740b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:30.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 --2- 172.21.15.29:0/3422126371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5740b3140 0x7fb57406e440 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:30.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 -- 172.21.15.29:0/3422126371 >> 172.21.15.29:0/3422126371 conn(0x7fb5740b1870 msgr2=0x7fb5740b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:30.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 -- 172.21.15.29:0/3422126371 shutdown_connections 2024-04-25T00:38:30.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 -- 172.21.15.29:0/3422126371 wait complete. 2024-04-25T00:38:30.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 Processor -- start 2024-04-25T00:38:30.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 -- start start 2024-04-25T00:38:30.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5740b3140 0x7fb574154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:30.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5740b3a60 0x7fb57415b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:30.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5740b4b10 0x7fb57415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:30.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb5740ce3c0 con 0x7fb5740b4b10 2024-04-25T00:38:30.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5740ce240 con 0x7fb5740b3a60 2024-04-25T00:38:30.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb57a252640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb5740ce540 con 0x7fb5740b3140 2024-04-25T00:38:30.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.847+0000 7fb573fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5740b4b10 0x7fb57415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:30.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb573fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5740b4b10 0x7fb57415b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53946/0 (socket says 172.21.15.29:53946) 2024-04-25T00:38:30.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb573fff640 1 -- 172.21.15.29:0/181056378 learned_addr learned my addr 172.21.15.29:0/181056378 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:30.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb5737fe640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5740b3140 0x7fb574154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:30.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb572ffd640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5740b3a60 0x7fb57415b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:30.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb573fff640 1 -- 172.21.15.29:0/181056378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5740b3140 msgr2=0x7fb574154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:30.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb573fff640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5740b3140 0x7fb574154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:30.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb573fff640 1 -- 172.21.15.29:0/181056378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5740b3a60 msgr2=0x7fb57415b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:30.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb573fff640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5740b3a60 0x7fb57415b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:30.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb573fff640 1 -- 172.21.15.29:0/181056378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57415be80 con 0x7fb5740b4b10 2024-04-25T00:38:30.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb573fff640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5740b4b10 0x7fb57415b560 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb56c007e60 tx=0x7fb56c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:30.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb55bfff640 1 -- 172.21.15.29:0/181056378 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c014620 con 0x7fb5740b4b10 2024-04-25T00:38:30.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb55bfff640 1 -- 172.21.15.29:0/181056378 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb56c019070 con 0x7fb5740b4b10 2024-04-25T00:38:30.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb57a252640 1 -- 172.21.15.29:0/181056378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57413ac20 con 0x7fb5740b4b10 2024-04-25T00:38:30.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb55bfff640 1 -- 172.21.15.29:0/181056378 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c014a20 con 0x7fb5740b4b10 2024-04-25T00:38:30.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb57a252640 1 -- 172.21.15.29:0/181056378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb574139bf0 con 0x7fb5740b4b10 2024-04-25T00:38:30.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb57a252640 1 -- 172.21.15.29:0/181056378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb574139a00 con 0x7fb5740b4b10 2024-04-25T00:38:30.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb55bfff640 1 -- 172.21.15.29:0/181056378 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fb56c014bc0 con 0x7fb5740b4b10 2024-04-25T00:38:30.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb55bfff640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb560067cb0 0x7fb56006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:30.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb55bfff640 1 -- 172.21.15.29:0/181056378 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5609+0+0 (secure 0 0 0) 0x7fb56c086c20 con 0x7fb5740b4b10 2024-04-25T00:38:30.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb5737fe640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb560067cb0 0x7fb56006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:30.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.851+0000 7fb5737fe640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb560067cb0 0x7fb56006a170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb5680ad000 tx=0x7fb568008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:30.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:30.855+0000 7fb55bfff640 1 -- 172.21.15.29:0/181056378 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb56c0571a0 con 0x7fb5740b4b10 2024-04-25T00:38:31.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.055+0000 7fb57a252640 1 -- 172.21.15.29:0/181056378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb574069270 con 0x7fb5740b4b10 2024-04-25T00:38:31.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.739+0000 7fb55bfff640 1 -- 172.21.15.29:0/181056378 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v72) ==== 174+0+0 (secure 0 0 0) 0x7fb56c056aa0 con 0x7fb5740b4b10 2024-04-25T00:38:31.753 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 18 and data pool 19 2024-04-25T00:38:31.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.755+0000 7fb57a252640 1 -- 172.21.15.29:0/181056378 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb560067cb0 msgr2=0x7fb56006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:31.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.755+0000 7fb57a252640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb560067cb0 0x7fb56006a170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb5680ad000 tx=0x7fb568008040 comp rx=0 tx=0).stop 2024-04-25T00:38:31.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.755+0000 7fb57a252640 1 -- 172.21.15.29:0/181056378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5740b4b10 msgr2=0x7fb57415b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:31.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.755+0000 7fb57a252640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5740b4b10 0x7fb57415b560 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb56c007e60 tx=0x7fb56c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:38:31.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.755+0000 7fb57a252640 1 -- 172.21.15.29:0/181056378 shutdown_connections 2024-04-25T00:38:31.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.755+0000 7fb57a252640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb560067cb0 0x7fb56006a170 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:31.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.755+0000 7fb57a252640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5740b4b10 0x7fb57415b560 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:31.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.755+0000 7fb57a252640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5740b3a60 0x7fb57415b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:31.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.755+0000 7fb57a252640 1 --2- 172.21.15.29:0/181056378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5740b3140 0x7fb574154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:31.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.755+0000 7fb57a252640 1 -- 172.21.15.29:0/181056378 >> 172.21.15.29:0/181056378 conn(0x7fb5740b1870 msgr2=0x7fb57413dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:31.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.755+0000 7fb57a252640 1 -- 172.21.15.29:0/181056378 shutdown_connections 2024-04-25T00:38:31.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.755+0000 7fb57a252640 1 -- 172.21.15.29:0/181056378 wait complete. 2024-04-25T00:38:31.782 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:31.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842309d640 1 Processor -- start 2024-04-25T00:38:31.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842309d640 1 -- start start 2024-04-25T00:38:31.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842309d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f841c0b3140 0x7f841c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:31.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842309d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f841c0b3c30 0x7f841c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:31.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842309d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f841c0b41e0 con 0x7f841c0b4b10 2024-04-25T00:38:31.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842309d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f841c06d960 con 0x7f841c0b3c30 2024-04-25T00:38:31.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842309d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f841c06dc40 con 0x7f841c0b3140 2024-04-25T00:38:31.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842163a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f841c0b3c30 0x7f841c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:31.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842163a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f841c0b3c30 0x7f841c0ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53952/0 (socket says 172.21.15.29:53952) 2024-04-25T00:38:31.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842163a640 1 -- 172.21.15.29:0/785843967 learned_addr learned my addr 172.21.15.29:0/785843967 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:31.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f8420e39640 1 --2- 172.21.15.29:0/785843967 >> v2:172.21.15.157:3300/0 conn(0x7f841c0b3140 0x7f841c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:31.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842163a640 1 -- 172.21.15.29:0/785843967 >> v2:172.21.15.157:3300/0 conn(0x7f841c0b3140 msgr2=0x7f841c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:31.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842163a640 1 --2- 172.21.15.29:0/785843967 >> v2:172.21.15.157:3300/0 conn(0x7f841c0b3140 0x7f841c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:31.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842163a640 1 -- 172.21.15.29:0/785843967 >> v1:172.21.15.157:6789/0 conn(0x7f841c0b4b10 legacy=0x7f841c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:31.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.899+0000 7f842163a640 1 -- 172.21.15.29:0/785843967 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c06df20 con 0x7f841c0b3c30 2024-04-25T00:38:31.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f842163a640 1 --2- 172.21.15.29:0/785843967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c0b3c30 0x7f841c0ce240 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f8414009e50 tx=0x7f841401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a358cb85a7d2d9d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:31.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f841b7fe640 1 -- 172.21.15.29:0/785843967 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84140a56c0 con 0x7f841c0b3c30 2024-04-25T00:38:31.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f841b7fe640 1 -- 172.21.15.29:0/785843967 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84140a5860 con 0x7f841c0b3c30 2024-04-25T00:38:31.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f841b7fe640 1 -- 172.21.15.29:0/785843967 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84140a5c30 con 0x7f841c0b3c30 2024-04-25T00:38:31.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f842309d640 1 -- 172.21.15.29:0/785843967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c0b3c30 msgr2=0x7f841c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:31.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f842309d640 1 --2- 172.21.15.29:0/785843967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c0b3c30 0x7f841c0ce240 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f8414009e50 tx=0x7f841401c9f0 comp rx=0 tx=0).stop 2024-04-25T00:38:31.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f842309d640 1 -- 172.21.15.29:0/785843967 shutdown_connections 2024-04-25T00:38:31.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f842309d640 1 --2- 172.21.15.29:0/785843967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c0b3c30 0x7f841c0ce240 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:31.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f842309d640 1 --2- 172.21.15.29:0/785843967 >> v2:172.21.15.157:3300/0 conn(0x7f841c0b3140 0x7f841c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:31.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f842309d640 1 -- 172.21.15.29:0/785843967 >> 172.21.15.29:0/785843967 conn(0x7f841c0b1870 msgr2=0x7f841c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:31.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f842309d640 1 -- 172.21.15.29:0/785843967 shutdown_connections 2024-04-25T00:38:31.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f842309d640 1 -- 172.21.15.29:0/785843967 wait complete. 2024-04-25T00:38:31.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f842309d640 1 Processor -- start 2024-04-25T00:38:31.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.903+0000 7f842309d640 1 -- start start 2024-04-25T00:38:31.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842309d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f841c0b3140 0x7f841c15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842309d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f841c0b3c30 0x7f841c1410d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842309d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c0b4b10 0x7f841c141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842309d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f841c168d80 con 0x7f841c0b4b10 2024-04-25T00:38:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842309d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f841c168c00 con 0x7f841c0b3c30 2024-04-25T00:38:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842309d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f841c168f00 con 0x7f841c0b3140 2024-04-25T00:38:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842163a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c0b4b10 0x7f841c141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842163a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c0b4b10 0x7f841c141610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53966/0 (socket says 172.21.15.29:53966) 2024-04-25T00:38:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842163a640 1 -- 172.21.15.29:0/170961524 learned_addr learned my addr 172.21.15.29:0/170961524 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:31.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f841bfff640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f841c0b3c30 0x7f841c1410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842163a640 1 -- 172.21.15.29:0/170961524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f841c0b3140 msgr2=0x7f841c15b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f8420e39640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f841c0b3140 0x7f841c15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842163a640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f841c0b3140 0x7f841c15b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842163a640 1 -- 172.21.15.29:0/170961524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f841c0b3c30 msgr2=0x7f841c1410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842163a640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f841c0b3c30 0x7f841c1410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842163a640 1 -- 172.21.15.29:0/170961524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c13d4c0 con 0x7f841c0b4b10 2024-04-25T00:38:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f841bfff640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f841c0b3c30 0x7f841c1410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842163a640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c0b4b10 0x7f841c141610 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f84140af040 tx=0x7f8414019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f84197fa640 1 -- 172.21.15.29:0/170961524 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84140b8670 con 0x7f841c0b4b10 2024-04-25T00:38:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f84197fa640 1 -- 172.21.15.29:0/170961524 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84140be070 con 0x7f841c0b4b10 2024-04-25T00:38:31.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842309d640 1 -- 172.21.15.29:0/170961524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841c13d7a0 con 0x7f841c0b4b10 2024-04-25T00:38:31.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f84197fa640 1 -- 172.21.15.29:0/170961524 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84140b8a70 con 0x7f841c0b4b10 2024-04-25T00:38:31.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842309d640 1 -- 172.21.15.29:0/170961524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841c15b810 con 0x7f841c0b4b10 2024-04-25T00:38:31.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f842309d640 1 -- 172.21.15.29:0/170961524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841c069270 con 0x7f841c0b4b10 2024-04-25T00:38:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f84197fa640 1 -- 172.21.15.29:0/170961524 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f84140b8c10 con 0x7f841c0b4b10 2024-04-25T00:38:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f84197fa640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8408067ce0 0x7f840806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f84197fa640 1 -- 172.21.15.29:0/170961524 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5709+0+0 (secure 0 0 0) 0x7f8414121220 con 0x7f841c0b4b10 2024-04-25T00:38:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.907+0000 7f8420e39640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8408067ce0 0x7f840806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:31.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.911+0000 7f8420e39640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8408067ce0 0x7f840806a1a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f841c068b60 tx=0x7f840c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:31.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:31.915+0000 7f84197fa640 1 -- 172.21.15.29:0/170961524 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f84140165e0 con 0x7f841c0b4b10 2024-04-25T00:38:32.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.079+0000 7f842309d640 1 -- 172.21.15.29:0/170961524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f841c143bd0 con 0x7f841c0b4b10 2024-04-25T00:38:32.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.079+0000 7f84197fa640 1 -- 172.21.15.29:0/170961524 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16644 (secure 0 0 0) 0x7f84140f1860 con 0x7f841c0b4b10 2024-04-25T00:38:32.082 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:32.082 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":117,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:31.060829+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:38:28.743593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:38:30.051271+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.083+0000 7f842309d640 1 -- 172.21.15.29:0/170961524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8408067ce0 msgr2=0x7f840806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.083+0000 7f842309d640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8408067ce0 0x7f840806a1a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f841c068b60 tx=0x7f840c008040 comp rx=0 tx=0).stop 2024-04-25T00:38:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.083+0000 7f842309d640 1 -- 172.21.15.29:0/170961524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c0b4b10 msgr2=0x7f841c141610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.083+0000 7f842309d640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c0b4b10 0x7f841c141610 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f84140af040 tx=0x7f8414019790 comp rx=0 tx=0).stop 2024-04-25T00:38:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.083+0000 7f842309d640 1 -- 172.21.15.29:0/170961524 shutdown_connections 2024-04-25T00:38:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.083+0000 7f842309d640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8408067ce0 0x7f840806a1a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.083+0000 7f842309d640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c0b4b10 0x7f841c141610 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.083+0000 7f842309d640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f841c0b3c30 0x7f841c1410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.083+0000 7f842309d640 1 --2- 172.21.15.29:0/170961524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f841c0b3140 0x7f841c15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.083+0000 7f842309d640 1 -- 172.21.15.29:0/170961524 >> 172.21.15.29:0/170961524 conn(0x7f841c0b1870 msgr2=0x7f841c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.083+0000 7f842309d640 1 -- 172.21.15.29:0/170961524 shutdown_connections 2024-04-25T00:38:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.083+0000 7f842309d640 1 -- 172.21.15.29:0/170961524 wait complete. 2024-04-25T00:38:32.102 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:32.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.219+0000 7fc3f74b8640 1 Processor -- start 2024-04-25T00:38:32.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.219+0000 7fc3f74b8640 1 -- start start 2024-04-25T00:38:32.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.219+0000 7fc3f74b8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc3f015c220 0x7fc3f00cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.219+0000 7fc3f74b8640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f00cdbd0 con 0x7fc3f00d0670 2024-04-25T00:38:32.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.219+0000 7fc3f74b8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f00b13e0 con 0x7fc3f00cfa10 2024-04-25T00:38:32.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.219+0000 7fc3f74b8640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc3f015c600 con 0x7fc3f015c220 2024-04-25T00:38:32.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.219+0000 7fc3f5254640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc3f015c220 0x7fc3f00cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:32.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.219+0000 7fc3f5a55640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc3f00d0670 0x7fc3f00b1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40932/0 (socket says 172.21.15.29:40932) 2024-04-25T00:38:32.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.219+0000 7fc3f5a55640 1 -- 172.21.15.29:0/4156159468 learned_addr learned my addr 172.21.15.29:0/4156159468 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:32.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.219+0000 7fc3f5254640 1 -- 172.21.15.29:0/4156159468 >> v1:172.21.15.157:6789/0 conn(0x7fc3f00cfa10 legacy=0x7fc3f00cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:32.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.219+0000 7fc3f5254640 1 -- 172.21.15.29:0/4156159468 >> v1:172.21.15.29:6789/0 conn(0x7fc3f00d0670 legacy=0x7fc3f00b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:32.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.219+0000 7fc3f5254640 1 -- 172.21.15.29:0/4156159468 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f00b16c0 con 0x7fc3f015c220 2024-04-25T00:38:32.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.219+0000 7fc3e7fff640 1 -- 172.21.15.29:0/4156159468 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3321993982 0 0) 0x7fc3f00cdbd0 con 0x7fc3f00d0670 2024-04-25T00:38:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f5254640 1 --2- 172.21.15.29:0/4156159468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f015c220 0x7fc3f00cd350 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0097c0 tx=0x7fc3ec01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fa10fbf8fb3fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3e7fff640 1 -- 172.21.15.29:0/4156159468 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0b0070 con 0x7fc3f015c220 2024-04-25T00:38:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3e7fff640 1 -- 172.21.15.29:0/4156159468 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ec0ab600 con 0x7fc3f015c220 2024-04-25T00:38:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3e7fff640 1 -- 172.21.15.29:0/4156159468 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0ab9d0 con 0x7fc3f015c220 2024-04-25T00:38:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 -- 172.21.15.29:0/4156159468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f015c220 msgr2=0x7fc3f00cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 --2- 172.21.15.29:0/4156159468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f015c220 0x7fc3f00cd350 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0097c0 tx=0x7fc3ec01c460 comp rx=0 tx=0).stop 2024-04-25T00:38:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 -- 172.21.15.29:0/4156159468 shutdown_connections 2024-04-25T00:38:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 --2- 172.21.15.29:0/4156159468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f015c220 0x7fc3f00cd350 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 -- 172.21.15.29:0/4156159468 >> 172.21.15.29:0/4156159468 conn(0x7fc3f00d1d60 msgr2=0x7fc3f00d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 -- 172.21.15.29:0/4156159468 shutdown_connections 2024-04-25T00:38:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 -- 172.21.15.29:0/4156159468 wait complete. 2024-04-25T00:38:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 Processor -- start 2024-04-25T00:38:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 -- start start 2024-04-25T00:38:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f00cfa10 0x7fc3f014be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f00d0670 0x7fc3f00705c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f015c220 0x7fc3f00c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f00b31f0 con 0x7fc3f00cfa10 2024-04-25T00:38:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f00b3070 con 0x7fc3f015c220 2024-04-25T00:38:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f74b8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f0169d10 con 0x7fc3f00d0670 2024-04-25T00:38:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f5254640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f00cfa10 0x7fc3f014be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f5254640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f00cfa10 0x7fc3f014be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53982/0 (socket says 172.21.15.29:53982) 2024-04-25T00:38:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.223+0000 7fc3f5254640 1 -- 172.21.15.29:0/1385689434 learned_addr learned my addr 172.21.15.29:0/1385689434 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3f5254640 1 -- 172.21.15.29:0/1385689434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f00d0670 msgr2=0x7fc3f00705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3f5254640 1 --2- 172.21.15.29:0/1385689434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f00d0670 0x7fc3f00705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3f5254640 1 -- 172.21.15.29:0/1385689434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f015c220 msgr2=0x7fc3f00c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3f5254640 1 --2- 172.21.15.29:0/1385689434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f015c220 0x7fc3f00c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3f5254640 1 -- 172.21.15.29:0/1385689434 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f0156c70 con 0x7fc3f00cfa10 2024-04-25T00:38:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3f5254640 1 --2- 172.21.15.29:0/1385689434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f00cfa10 0x7fc3f014be60 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0ae040 tx=0x7fc3ec01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3e5ffb640 1 -- 172.21.15.29:0/1385689434 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0b0050 con 0x7fc3f00cfa10 2024-04-25T00:38:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3e5ffb640 1 -- 172.21.15.29:0/1385689434 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ec0bc070 con 0x7fc3f00cfa10 2024-04-25T00:38:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3e5ffb640 1 -- 172.21.15.29:0/1385689434 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0aaba0 con 0x7fc3f00cfa10 2024-04-25T00:38:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3f74b8640 1 -- 172.21.15.29:0/1385689434 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f00afb60 con 0x7fc3f00cfa10 2024-04-25T00:38:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3f74b8640 1 -- 172.21.15.29:0/1385689434 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f00b0120 con 0x7fc3f00cfa10 2024-04-25T00:38:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3f74b8640 1 -- 172.21.15.29:0/1385689434 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f0156e40 con 0x7fc3f00cfa10 2024-04-25T00:38:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3e5ffb640 1 -- 172.21.15.29:0/1385689434 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fc3ec0c1460 con 0x7fc3f00cfa10 2024-04-25T00:38:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3e5ffb640 1 --2- 172.21.15.29:0/1385689434 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc3d4067cb0 0x7fc3d406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3e5ffb640 1 -- 172.21.15.29:0/1385689434 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5709+0+0 (secure 0 0 0) 0x7fc3ec120eb0 con 0x7fc3f00cfa10 2024-04-25T00:38:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3f4a53640 1 --2- 172.21.15.29:0/1385689434 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc3d4067cb0 0x7fc3d406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.227+0000 7fc3f4a53640 1 --2- 172.21.15.29:0/1385689434 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc3d4067cb0 0x7fc3d406a170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc3f015f7b0 tx=0x7fc3e0006bd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:32.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.231+0000 7fc3e5ffb640 1 -- 172.21.15.29:0/1385689434 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc3ec016050 con 0x7fc3f00cfa10 2024-04-25T00:38:32.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.395+0000 7fc3f74b8640 1 -- 172.21.15.29:0/1385689434 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3f0069270 con 0x7fc3f00cfa10 2024-04-25T00:38:32.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.399+0000 7fc3e5ffb640 1 -- 172.21.15.29:0/1385689434 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16644 (secure 0 0 0) 0x7fc3ec0f1580 con 0x7fc3f00cfa10 2024-04-25T00:38:32.401 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:32.402 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":117,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:31.060829+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:38:28.743593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:38:30.051271+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:32.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.399+0000 7fc3f74b8640 1 -- 172.21.15.29:0/1385689434 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc3d4067cb0 msgr2=0x7fc3d406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:32.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.399+0000 7fc3f74b8640 1 --2- 172.21.15.29:0/1385689434 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc3d4067cb0 0x7fc3d406a170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc3f015f7b0 tx=0x7fc3e0006bd0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.399+0000 7fc3f74b8640 1 -- 172.21.15.29:0/1385689434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f00cfa10 msgr2=0x7fc3f014be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:32.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.399+0000 7fc3f74b8640 1 --2- 172.21.15.29:0/1385689434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f00cfa10 0x7fc3f014be60 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0ae040 tx=0x7fc3ec01aca0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.399+0000 7fc3f74b8640 1 -- 172.21.15.29:0/1385689434 shutdown_connections 2024-04-25T00:38:32.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.399+0000 7fc3f74b8640 1 --2- 172.21.15.29:0/1385689434 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc3d4067cb0 0x7fc3d406a170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.399+0000 7fc3f74b8640 1 --2- 172.21.15.29:0/1385689434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f015c220 0x7fc3f00c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.399+0000 7fc3f74b8640 1 --2- 172.21.15.29:0/1385689434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f00d0670 0x7fc3f00705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.399+0000 7fc3f74b8640 1 --2- 172.21.15.29:0/1385689434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f00cfa10 0x7fc3f014be60 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.403+0000 7fc3f74b8640 1 -- 172.21.15.29:0/1385689434 >> 172.21.15.29:0/1385689434 conn(0x7fc3f00d1d60 msgr2=0x7fc3f006d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:32.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.403+0000 7fc3f74b8640 1 -- 172.21.15.29:0/1385689434 shutdown_connections 2024-04-25T00:38:32.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.403+0000 7fc3f74b8640 1 -- 172.21.15.29:0/1385689434 wait complete. 2024-04-25T00:38:32.421 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:38:32.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55cd3ba640 1 Processor -- start 2024-04-25T00:38:32.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55cd3ba640 1 -- start start 2024-04-25T00:38:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55cd3ba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c815e8d0 0x7f55c80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55cd3ba640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55c815f3c0 0x7f55c815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55cd3ba640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55c80b3720 con 0x7f55c80b4780 2024-04-25T00:38:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55cd3ba640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f55c80b42f0 con 0x7f55c815f3c0 2024-04-25T00:38:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55cd3ba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f55c80b39c0 con 0x7f55c815e8d0 2024-04-25T00:38:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c67fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f55c80b4780 0x7f55c815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40948/0 (socket says 172.21.15.29:40948) 2024-04-25T00:38:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c67fc640 1 -- 172.21.15.29:0/65655739 learned_addr learned my addr 172.21.15.29:0/65655739 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c6ffd640 1 --2- 172.21.15.29:0/65655739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c815e8d0 0x7f55c80cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2284289845 0 0) 0x7f55c80b3720 con 0x7f55c80b4780 2024-04-25T00:38:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55b4003530 con 0x7f55c80b4780 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c77fe640 1 --2- 172.21.15.29:0/65655739 >> v2:172.21.15.157:3300/0 conn(0x7f55c815f3c0 0x7f55c815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3903187477 0 0) 0x7f55b4003530 con 0x7f55c80b4780 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c815e8d0 msgr2=0x7f55c80cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 --2- 172.21.15.29:0/65655739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c815e8d0 0x7f55c80cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 >> v2:172.21.15.157:3300/0 conn(0x7f55c815f3c0 msgr2=0x7f55c815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 --2- 172.21.15.29:0/65655739 >> v2:172.21.15.157:3300/0 conn(0x7f55c815f3c0 0x7f55c815c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c80b3ca0 con 0x7f55c80b4780 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f55b0001680 con 0x7f55c80b4780 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 >> v1:172.21.15.29:6789/0 conn(0x7f55c80b4780 legacy=0x7f55c815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 --2- 172.21.15.29:0/65655739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b40039c0 0x7f55b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 --2- 172.21.15.29:0/65655739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b4004810 0x7f55b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 --2- 172.21.15.29:0/65655739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c815e8d0 0x7f55b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f55c80b42f0 con 0x7f55b40039c0 2024-04-25T00:38:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f55c80b39c0 con 0x7f55b4004810 2024-04-25T00:38:32.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f55b400b9d0 con 0x7f55c815e8d0 2024-04-25T00:38:32.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c77fe640 1 --2- 172.21.15.29:0/65655739 >> v2:172.21.15.157:3300/0 conn(0x7f55c815f3c0 0x7f55c815c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:32.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c6ffd640 1 --2- 172.21.15.29:0/65655739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b40039c0 0x7f55b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:32.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c67fc640 1 --2- 172.21.15.29:0/65655739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b4004810 0x7f55b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:32.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c6ffd640 1 -- 172.21.15.29:0/65655739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c815e8d0 msgr2=0x7f55b40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:32.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c6ffd640 1 --2- 172.21.15.29:0/65655739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c815e8d0 0x7f55b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c6ffd640 1 -- 172.21.15.29:0/65655739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b4004810 msgr2=0x7f55b4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:32.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c6ffd640 1 --2- 172.21.15.29:0/65655739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b4004810 0x7f55b4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.547+0000 7f55c6ffd640 1 -- 172.21.15.29:0/65655739 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b4003530 con 0x7f55b40039c0 2024-04-25T00:38:32.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55c6ffd640 1 --2- 172.21.15.29:0/65655739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b40039c0 0x7f55b4003de0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009ae0 tx=0x7f55b801ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299c7bf89aacad50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:32.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b80096b0 con 0x7f55b40039c0 2024-04-25T00:38:32.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b80a6490 con 0x7f55b40039c0 2024-04-25T00:38:32.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55c5ffb640 1 -- 172.21.15.29:0/65655739 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b80a6880 con 0x7f55b40039c0 2024-04-25T00:38:32.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 -- 172.21.15.29:0/65655739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b40039c0 msgr2=0x7f55b4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 --2- 172.21.15.29:0/65655739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b40039c0 0x7f55b4003de0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009ae0 tx=0x7f55b801ca80 comp rx=0 tx=0).stop 2024-04-25T00:38:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55c77fe640 1 -- 172.21.15.29:0/65655739 reap_dead start 2024-04-25T00:38:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 -- 172.21.15.29:0/65655739 shutdown_connections 2024-04-25T00:38:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 -- 172.21.15.29:0/65655739 >> 172.21.15.29:0/65655739 conn(0x7f55c80b1870 msgr2=0x7f55c80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 -- 172.21.15.29:0/65655739 shutdown_connections 2024-04-25T00:38:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 -- 172.21.15.29:0/65655739 wait complete. 2024-04-25T00:38:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 Processor -- start 2024-04-25T00:38:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 -- start start 2024-04-25T00:38:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55b40039c0 0x7f55c806dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c814d470 0x7f55c80cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55c815b1a0 0x7f55c815b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f55c815d6a0 con 0x7f55c815b1a0 2024-04-25T00:38:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f55c815d520 con 0x7f55c814d470 2024-04-25T00:38:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55cd3ba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f55c8165c60 con 0x7f55b40039c0 2024-04-25T00:38:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55c77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55c815b1a0 0x7f55c815b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55c77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55c815b1a0 0x7f55c815b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54002/0 (socket says 172.21.15.29:54002) 2024-04-25T00:38:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.551+0000 7f55c77fe640 1 -- 172.21.15.29:0/140537104 learned_addr learned my addr 172.21.15.29:0/140537104 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55c6ffd640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55b40039c0 0x7f55c806dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55c67fc640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c814d470 0x7f55c80cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55c77fe640 1 -- 172.21.15.29:0/140537104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55b40039c0 msgr2=0x7f55c806dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55c77fe640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55b40039c0 0x7f55c806dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55c77fe640 1 -- 172.21.15.29:0/140537104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c814d470 msgr2=0x7f55c80cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55c77fe640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c814d470 0x7f55c80cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55c77fe640 1 -- 172.21.15.29:0/140537104 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c8140d60 con 0x7f55c815b1a0 2024-04-25T00:38:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55c67fc640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c814d470 0x7f55c80cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55c6ffd640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55b40039c0 0x7f55c806dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55c77fe640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55c815b1a0 0x7f55c815b620 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f55c815f7a0 tx=0x7f55c000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55bf7fe640 1 -- 172.21.15.29:0/140537104 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c001a070 con 0x7f55c815b1a0 2024-04-25T00:38:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55bf7fe640 1 -- 172.21.15.29:0/140537104 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c000de50 con 0x7f55c815b1a0 2024-04-25T00:38:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55bf7fe640 1 -- 172.21.15.29:0/140537104 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0003be0 con 0x7f55c815b1a0 2024-04-25T00:38:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55cd3ba640 1 -- 172.21.15.29:0/140537104 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c80ce4b0 con 0x7f55c815b1a0 2024-04-25T00:38:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55cd3ba640 1 -- 172.21.15.29:0/140537104 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c80cf4d0 con 0x7f55c815b1a0 2024-04-25T00:38:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55cd3ba640 1 -- 172.21.15.29:0/140537104 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c8069270 con 0x7f55c815b1a0 2024-04-25T00:38:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55bf7fe640 1 -- 172.21.15.29:0/140537104 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f55c001d070 con 0x7f55c815b1a0 2024-04-25T00:38:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55bf7fe640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f55a8067cb0 0x7f55a806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55bf7fe640 1 -- 172.21.15.29:0/140537104 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5709+0+0 (secure 0 0 0) 0x7f55c0086270 con 0x7f55c815b1a0 2024-04-25T00:38:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55c6ffd640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f55a8067cb0 0x7f55a806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.555+0000 7f55c6ffd640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f55a8067cb0 0x7f55a806a170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f55b80ad000 tx=0x7f55b80046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:32.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.559+0000 7f55bf7fe640 1 -- 172.21.15.29:0/140537104 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f55c0010070 con 0x7f55c815b1a0 2024-04-25T00:38:32.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:32.759+0000 7f55cd3ba640 1 -- 172.21.15.29:0/140537104 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f55c8144250 con 0x7f55c815b1a0 2024-04-25T00:38:33.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.731+0000 7f55bf7fe640 1 -- 172.21.15.29:0/140537104 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) ==== 118+0+0 (secure 0 0 0) 0x7f55c00566c0 con 0x7f55c815b1a0 2024-04-25T00:38:33.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.735+0000 7f55cd3ba640 1 -- 172.21.15.29:0/140537104 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f55a8067cb0 msgr2=0x7f55a806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:33.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.735+0000 7f55cd3ba640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f55a8067cb0 0x7f55a806a170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f55b80ad000 tx=0x7f55b80046d0 comp rx=0 tx=0).stop 2024-04-25T00:38:33.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.735+0000 7f55cd3ba640 1 -- 172.21.15.29:0/140537104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55c815b1a0 msgr2=0x7f55c815b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:33.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.735+0000 7f55cd3ba640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55c815b1a0 0x7f55c815b620 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f55c815f7a0 tx=0x7f55c000d4a0 comp rx=0 tx=0).stop 2024-04-25T00:38:33.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.735+0000 7f55cd3ba640 1 -- 172.21.15.29:0/140537104 shutdown_connections 2024-04-25T00:38:33.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.735+0000 7f55cd3ba640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f55a8067cb0 0x7f55a806a170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:33.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.735+0000 7f55cd3ba640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55c815b1a0 0x7f55c815b620 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:33.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.735+0000 7f55cd3ba640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c814d470 0x7f55c80cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:33.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.735+0000 7f55cd3ba640 1 --2- 172.21.15.29:0/140537104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55b40039c0 0x7f55c806dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:33.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.735+0000 7f55cd3ba640 1 -- 172.21.15.29:0/140537104 >> 172.21.15.29:0/140537104 conn(0x7f55c80b1870 msgr2=0x7f55c813dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:33.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.735+0000 7f55cd3ba640 1 -- 172.21.15.29:0/140537104 shutdown_connections 2024-04-25T00:38:33.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.735+0000 7f55cd3ba640 1 -- 172.21.15.29:0/140537104 wait complete. 2024-04-25T00:38:33.757 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:38:33.757 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:38:33.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f9812619640 1 Processor -- start 2024-04-25T00:38:33.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f9812619640 1 -- start start 2024-04-25T00:38:33.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f9812619640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f980c15e8d0 0x7f980c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:33.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f9812619640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f980c15f3c0 0x7f980c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:33.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f9812619640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f980c0b3720 con 0x7f980c0b4780 2024-04-25T00:38:33.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f9812619640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f980c0b42f0 con 0x7f980c15e8d0 2024-04-25T00:38:33.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f9812619640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f980c0b39c0 con 0x7f980c15f3c0 2024-04-25T00:38:33.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f980bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f980c15e8d0 0x7f980c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:33.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f980b7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f980c0b4780 0x7f980c15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40960/0 (socket says 172.21.15.29:40960) 2024-04-25T00:38:33.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f980b7fe640 1 -- 172.21.15.29:0/3675868333 learned_addr learned my addr 172.21.15.29:0/3675868333 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:33.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f9810bb6640 1 --2- 172.21.15.29:0/3675868333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f980c15f3c0 0x7f980c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:33.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f980bfff640 1 -- 172.21.15.29:0/3675868333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f980c15f3c0 msgr2=0x7f980c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:33.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f980bfff640 1 --2- 172.21.15.29:0/3675868333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f980c15f3c0 0x7f980c15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:33.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f980bfff640 1 -- 172.21.15.29:0/3675868333 >> v1:172.21.15.29:6789/0 conn(0x7f980c0b4780 legacy=0x7f980c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:33.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f980bfff640 1 -- 172.21.15.29:0/3675868333 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c0b3ca0 con 0x7f980c15e8d0 2024-04-25T00:38:33.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.887+0000 7f9810bb6640 1 --2- 172.21.15.29:0/3675868333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f980c15f3c0 0x7f980c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:33.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f980bfff640 1 --2- 172.21.15.29:0/3675868333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f980c15e8d0 0x7f980c0cc240 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009d90 tx=0x7f97fc01c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49065aab5c3e91e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:33.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f980affd640 1 -- 172.21.15.29:0/3675868333 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0b2070 con 0x7f980c15e8d0 2024-04-25T00:38:33.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f980affd640 1 -- 172.21.15.29:0/3675868333 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc0a5820 con 0x7f980c15e8d0 2024-04-25T00:38:33.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f980affd640 1 -- 172.21.15.29:0/3675868333 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0a5bf0 con 0x7f980c15e8d0 2024-04-25T00:38:33.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f9812619640 1 -- 172.21.15.29:0/3675868333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f980c15e8d0 msgr2=0x7f980c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:33.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f9812619640 1 --2- 172.21.15.29:0/3675868333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f980c15e8d0 0x7f980c0cc240 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009d90 tx=0x7f97fc01c800 comp rx=0 tx=0).stop 2024-04-25T00:38:33.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f9812619640 1 -- 172.21.15.29:0/3675868333 shutdown_connections 2024-04-25T00:38:33.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f9812619640 1 --2- 172.21.15.29:0/3675868333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f980c15f3c0 0x7f980c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:33.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f9812619640 1 --2- 172.21.15.29:0/3675868333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f980c15e8d0 0x7f980c0cc240 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:33.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f9812619640 1 -- 172.21.15.29:0/3675868333 >> 172.21.15.29:0/3675868333 conn(0x7f980c0b1870 msgr2=0x7f980c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:33.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f9812619640 1 -- 172.21.15.29:0/3675868333 shutdown_connections 2024-04-25T00:38:33.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f9812619640 1 -- 172.21.15.29:0/3675868333 wait complete. 2024-04-25T00:38:33.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f9812619640 1 Processor -- start 2024-04-25T00:38:33.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.891+0000 7f9812619640 1 -- start start 2024-04-25T00:38:33.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9812619640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f980c0b4780 0x7f980c138540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:33.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9812619640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f980c15e8d0 0x7f980c071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:33.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9812619640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f980c15f3c0 0x7f980c072500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:33.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9812619640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f980c144a20 con 0x7f980c0b4780 2024-04-25T00:38:33.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9812619640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f980c06cf10 con 0x7f980c15f3c0 2024-04-25T00:38:33.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9812619640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f980c13fda0 con 0x7f980c15e8d0 2024-04-25T00:38:33.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f980bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f980c0b4780 0x7f980c138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:33.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f980bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f980c0b4780 0x7f980c138540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54016/0 (socket says 172.21.15.29:54016) 2024-04-25T00:38:33.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f980bfff640 1 -- 172.21.15.29:0/1572423521 learned_addr learned my addr 172.21.15.29:0/1572423521 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:33.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f980bfff640 1 -- 172.21.15.29:0/1572423521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f980c15e8d0 msgr2=0x7f980c071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:33.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f980bfff640 1 --2- 172.21.15.29:0/1572423521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f980c15e8d0 0x7f980c071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:33.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f980bfff640 1 -- 172.21.15.29:0/1572423521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f980c15f3c0 msgr2=0x7f980c072500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:33.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9810bb6640 1 --2- 172.21.15.29:0/1572423521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f980c15f3c0 0x7f980c072500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:33.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f980bfff640 1 --2- 172.21.15.29:0/1572423521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f980c15f3c0 0x7f980c072500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:33.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f980bfff640 1 -- 172.21.15.29:0/1572423521 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c13f190 con 0x7f980c0b4780 2024-04-25T00:38:33.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f980bfff640 1 --2- 172.21.15.29:0/1572423521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f980c0b4780 0x7f980c138540 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0b0040 tx=0x7f97fc0195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:33.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9808ff9640 1 -- 172.21.15.29:0/1572423521 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0b2050 con 0x7f980c0b4780 2024-04-25T00:38:33.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9808ff9640 1 -- 172.21.15.29:0/1572423521 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc0be070 con 0x7f980c0b4780 2024-04-25T00:38:33.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9812619640 1 -- 172.21.15.29:0/1572423521 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980c13f470 con 0x7f980c0b4780 2024-04-25T00:38:33.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9808ff9640 1 -- 172.21.15.29:0/1572423521 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0b9450 con 0x7f980c0b4780 2024-04-25T00:38:33.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9812619640 1 -- 172.21.15.29:0/1572423521 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f980c13fa30 con 0x7f980c0b4780 2024-04-25T00:38:33.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9812619640 1 -- 172.21.15.29:0/1572423521 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f980c069270 con 0x7f980c0b4780 2024-04-25T00:38:33.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9808ff9640 1 -- 172.21.15.29:0/1572423521 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f97fc0b95f0 con 0x7f980c0b4780 2024-04-25T00:38:33.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9808ff9640 1 --2- 172.21.15.29:0/1572423521 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f97ec067cb0 0x7f97ec06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:33.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f9808ff9640 1 -- 172.21.15.29:0/1572423521 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f97fc121b60 con 0x7f980c0b4780 2024-04-25T00:38:33.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.895+0000 7f980b7fe640 1 --2- 172.21.15.29:0/1572423521 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f97ec067cb0 0x7f97ec06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:33.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.899+0000 7f980b7fe640 1 --2- 172.21.15.29:0/1572423521 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f97ec067cb0 0x7f97ec06a170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f980c1f7760 tx=0x7f97f4001c60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:33.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:33.899+0000 7f9808ff9640 1 -- 172.21.15.29:0/1572423521 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f97fc016480 con 0x7f980c0b4780 2024-04-25T00:38:34.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.091+0000 7f9812619640 1 -- 172.21.15.29:0/1572423521 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f980c06e7b0 con 0x7f980c0b4780 2024-04-25T00:38:34.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.747+0000 7f9808ff9640 1 -- 172.21.15.29:0/1572423521 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v75) ==== 122+0+0 (secure 0 0 0) 0x7f97fc0abe20 con 0x7f980c0b4780 2024-04-25T00:38:34.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.751+0000 7f9812619640 1 -- 172.21.15.29:0/1572423521 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f97ec067cb0 msgr2=0x7f97ec06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:34.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.751+0000 7f9812619640 1 --2- 172.21.15.29:0/1572423521 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f97ec067cb0 0x7f97ec06a170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f980c1f7760 tx=0x7f97f4001c60 comp rx=0 tx=0).stop 2024-04-25T00:38:34.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.751+0000 7f9812619640 1 -- 172.21.15.29:0/1572423521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f980c0b4780 msgr2=0x7f980c138540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:34.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.751+0000 7f9812619640 1 --2- 172.21.15.29:0/1572423521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f980c0b4780 0x7f980c138540 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0b0040 tx=0x7f97fc0195a0 comp rx=0 tx=0).stop 2024-04-25T00:38:34.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.751+0000 7f9812619640 1 -- 172.21.15.29:0/1572423521 shutdown_connections 2024-04-25T00:38:34.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.751+0000 7f9812619640 1 --2- 172.21.15.29:0/1572423521 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f97ec067cb0 0x7f97ec06a170 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:34.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.751+0000 7f9812619640 1 --2- 172.21.15.29:0/1572423521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f980c15f3c0 0x7f980c072500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:34.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.751+0000 7f9812619640 1 --2- 172.21.15.29:0/1572423521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f980c15e8d0 0x7f980c071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:34.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.751+0000 7f9812619640 1 --2- 172.21.15.29:0/1572423521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f980c0b4780 0x7f980c138540 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:34.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.751+0000 7f9812619640 1 -- 172.21.15.29:0/1572423521 >> 172.21.15.29:0/1572423521 conn(0x7f980c0b1870 msgr2=0x7f980c0ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:34.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.751+0000 7f9812619640 1 -- 172.21.15.29:0/1572423521 shutdown_connections 2024-04-25T00:38:34.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.751+0000 7f9812619640 1 -- 172.21.15.29:0/1572423521 wait complete. 2024-04-25T00:38:34.774 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:38:34.775 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:38:34.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.899+0000 7f158cfc8640 1 Processor -- start 2024-04-25T00:38:34.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.899+0000 7f158cfc8640 1 -- start start 2024-04-25T00:38:34.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158cfc8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f15880cf690 0x7f15880d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:34.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158cfc8640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15880d2ea0 con 0x7f158815e200 2024-04-25T00:38:34.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158cfc8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15880cfc10 con 0x7f15880d2a90 2024-04-25T00:38:34.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158cfc8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f15880b38d0 con 0x7f15880cf690 2024-04-25T00:38:34.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158659c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f158815e200 0x7f15880d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40976/0 (socket says 172.21.15.29:40976) 2024-04-25T00:38:34.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158659c640 1 -- 172.21.15.29:0/1110289697 learned_addr learned my addr 172.21.15.29:0/1110289697 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:34.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3988074251 0 0) 0x7f1578001120 con 0x7f158815e200 2024-04-25T00:38:34.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f1585d9b640 1 --2- 172.21.15.29:0/1110289697 >> v2:172.21.15.157:3300/0 conn(0x7f15880cf690 0x7f15880d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:34.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1570003530 con 0x7f158815e200 2024-04-25T00:38:34.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2839969695 0 0) 0x7f1570003530 con 0x7f158815e200 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 >> v2:172.21.15.157:3300/0 conn(0x7f15880cf690 msgr2=0x7f15880d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 --2- 172.21.15.29:0/1110289697 >> v2:172.21.15.157:3300/0 conn(0x7f15880cf690 0x7f15880d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 >> v1:172.21.15.157:6789/0 conn(0x7f15880d2a90 legacy=0x7f15880b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15880cfef0 con 0x7f158815e200 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f1578001850 con 0x7f158815e200 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 >> v1:172.21.15.29:6789/0 conn(0x7f158815e200 legacy=0x7f15880d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 --2- 172.21.15.29:0/1110289697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15700039c0 0x7f1570003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 --2- 172.21.15.29:0/1110289697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1570004810 0x7f1570004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 --2- 172.21.15.29:0/1110289697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15700051d0 0x7f157000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f15880b38d0 con 0x7f1570004810 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f157000c120 con 0x7f15700051d0 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f157000c320 con 0x7f15700039c0 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f1585d9b640 1 --2- 172.21.15.29:0/1110289697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1570004810 0x7f1570004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:34.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f1586d9d640 1 --2- 172.21.15.29:0/1110289697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15700051d0 0x7f157000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:34.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158659c640 1 --2- 172.21.15.29:0/1110289697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15700039c0 0x7f1570003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:34.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f1585d9b640 1 -- 172.21.15.29:0/1110289697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15700039c0 msgr2=0x7f1570003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:34.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f1585d9b640 1 --2- 172.21.15.29:0/1110289697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15700039c0 0x7f1570003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:34.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f1585d9b640 1 -- 172.21.15.29:0/1110289697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15700051d0 msgr2=0x7f157000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:34.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f1585d9b640 1 --2- 172.21.15.29:0/1110289697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15700051d0 0x7f157000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:34.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f1586d9d640 1 -- 172.21.15.29:0/1110289697 reap_dead start 2024-04-25T00:38:34.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f1585d9b640 1 -- 172.21.15.29:0/1110289697 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1570003530 con 0x7f1570004810 2024-04-25T00:38:34.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.903+0000 7f158659c640 1 --2- 172.21.15.29:0/1110289697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15700039c0 0x7f1570003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:34.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f1585d9b640 1 --2- 172.21.15.29:0/1110289697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1570004810 0x7f1570004c90 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f157c00ba30 tx=0x7f157c01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef2ff682af26a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:34.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c00b600 con 0x7f1570004810 2024-04-25T00:38:34.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f157c0b1070 con 0x7f1570004810 2024-04-25T00:38:34.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158559a640 1 -- 172.21.15.29:0/1110289697 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c0ac680 con 0x7f1570004810 2024-04-25T00:38:34.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 -- 172.21.15.29:0/1110289697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1570004810 msgr2=0x7f1570004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:34.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 --2- 172.21.15.29:0/1110289697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1570004810 0x7f1570004c90 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f157c00ba30 tx=0x7f157c01c6a0 comp rx=0 tx=0).stop 2024-04-25T00:38:34.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 -- 172.21.15.29:0/1110289697 shutdown_connections 2024-04-25T00:38:34.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 --2- 172.21.15.29:0/1110289697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1570004810 0x7f1570004c90 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:34.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 -- 172.21.15.29:0/1110289697 >> 172.21.15.29:0/1110289697 conn(0x7f158809d280 msgr2=0x7f15880d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:34.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 -- 172.21.15.29:0/1110289697 shutdown_connections 2024-04-25T00:38:34.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 -- 172.21.15.29:0/1110289697 wait complete. 2024-04-25T00:38:34.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 Processor -- start 2024-04-25T00:38:34.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 -- start start 2024-04-25T00:38:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1570004810 0x7f15880ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15880cca80 0x7f15880ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15880c7aa0 0x7f15880c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f158815e5e0 con 0x7f15880c7aa0 2024-04-25T00:38:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15880b3e00 con 0x7f15880cca80 2024-04-25T00:38:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f158815e760 con 0x7f1570004810 2024-04-25T00:38:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f1586d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15880c7aa0 0x7f15880c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f1586d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15880c7aa0 0x7f15880c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54040/0 (socket says 172.21.15.29:54040) 2024-04-25T00:38:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f1586d9d640 1 -- 172.21.15.29:0/1302182272 learned_addr learned my addr 172.21.15.29:0/1302182272 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f1586d9d640 1 -- 172.21.15.29:0/1302182272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1570004810 msgr2=0x7f15880ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f1586d9d640 1 --2- 172.21.15.29:0/1302182272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1570004810 0x7f15880ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f1586d9d640 1 -- 172.21.15.29:0/1302182272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15880cca80 msgr2=0x7f15880ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f1586d9d640 1 --2- 172.21.15.29:0/1302182272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15880cca80 0x7f15880ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f1586d9d640 1 -- 172.21.15.29:0/1302182272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1588161260 con 0x7f15880c7aa0 2024-04-25T00:38:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f1586d9d640 1 --2- 172.21.15.29:0/1302182272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15880c7aa0 0x7f15880c7f20 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f158806b900 tx=0x7f158000e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f1576ffd640 1 -- 172.21.15.29:0/1302182272 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580019070 con 0x7f15880c7aa0 2024-04-25T00:38:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f1576ffd640 1 -- 172.21.15.29:0/1302182272 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1580006e30 con 0x7f15880c7aa0 2024-04-25T00:38:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f1576ffd640 1 -- 172.21.15.29:0/1302182272 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580003ba0 con 0x7f15880c7aa0 2024-04-25T00:38:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.907+0000 7f158cfc8640 1 -- 172.21.15.29:0/1302182272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15881f4950 con 0x7f15880c7aa0 2024-04-25T00:38:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.911+0000 7f158cfc8640 1 -- 172.21.15.29:0/1302182272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15880cc280 con 0x7f15880c7aa0 2024-04-25T00:38:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.911+0000 7f158cfc8640 1 -- 172.21.15.29:0/1302182272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1588069270 con 0x7f15880c7aa0 2024-04-25T00:38:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.911+0000 7f1576ffd640 1 -- 172.21.15.29:0/1302182272 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f158001c070 con 0x7f15880c7aa0 2024-04-25T00:38:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.911+0000 7f1576ffd640 1 --2- 172.21.15.29:0/1302182272 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1568067cb0 0x7f156806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.911+0000 7f1576ffd640 1 -- 172.21.15.29:0/1302182272 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f1580085980 con 0x7f15880c7aa0 2024-04-25T00:38:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.911+0000 7f158659c640 1 --2- 172.21.15.29:0/1302182272 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1568067cb0 0x7f156806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.911+0000 7f158659c640 1 --2- 172.21.15.29:0/1302182272 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1568067cb0 0x7f156806a170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15780013e0 tx=0x7f1578000f70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:34.915+0000 7f1576ffd640 1 -- 172.21.15.29:0/1302182272 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1580055ee0 con 0x7f15880c7aa0 2024-04-25T00:38:35.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.115+0000 7f158cfc8640 1 -- 172.21.15.29:0/1302182272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f158806beb0 con 0x7f15880c7aa0 2024-04-25T00:38:35.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.115+0000 7f1576ffd640 1 -- 172.21.15.29:0/1302182272 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5186 (secure 0 0 0) 0x7f15800557e0 con 0x7f15880c7aa0 2024-04-25T00:38:35.118 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:35.119 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":75,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2148970077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2148970077},{"type":"v1","addr":"172.21.15.157:6834","nonce":2148970077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3947933400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3947933400},{"type":"v1","addr":"172.21.15.157:6835","nonce":3947933400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/172449010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":172449010},{"type":"v1","addr":"172.21.15.29:6833","nonce":172449010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:38:31.061384+0000","modified":"2024-04-25T00:38:34.097007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/3206404146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3206404146},{"type":"v1","addr":"172.21.15.29:6835","nonce":3206404146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8520,"qdb_cluster":[8520]},"id":9}]} 2024-04-25T00:38:35.119 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 75 2024-04-25T00:38:35.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.119+0000 7f158cfc8640 1 -- 172.21.15.29:0/1302182272 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1568067cb0 msgr2=0x7f156806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.119+0000 7f158cfc8640 1 --2- 172.21.15.29:0/1302182272 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1568067cb0 0x7f156806a170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15780013e0 tx=0x7f1578000f70 comp rx=0 tx=0).stop 2024-04-25T00:38:35.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.119+0000 7f158cfc8640 1 -- 172.21.15.29:0/1302182272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15880c7aa0 msgr2=0x7f15880c7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.119+0000 7f158cfc8640 1 --2- 172.21.15.29:0/1302182272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15880c7aa0 0x7f15880c7f20 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f158806b900 tx=0x7f158000e9e0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.119+0000 7f158cfc8640 1 -- 172.21.15.29:0/1302182272 shutdown_connections 2024-04-25T00:38:35.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.119+0000 7f158cfc8640 1 --2- 172.21.15.29:0/1302182272 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1568067cb0 0x7f156806a170 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.119+0000 7f158cfc8640 1 --2- 172.21.15.29:0/1302182272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15880c7aa0 0x7f15880c7f20 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.119+0000 7f158cfc8640 1 --2- 172.21.15.29:0/1302182272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15880cca80 0x7f15880ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.119+0000 7f158cfc8640 1 --2- 172.21.15.29:0/1302182272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1570004810 0x7f15880ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.119+0000 7f158cfc8640 1 -- 172.21.15.29:0/1302182272 >> 172.21.15.29:0/1302182272 conn(0x7f158809d280 msgr2=0x7f15880d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:35.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.119+0000 7f158cfc8640 1 -- 172.21.15.29:0/1302182272 shutdown_connections 2024-04-25T00:38:35.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.119+0000 7f158cfc8640 1 -- 172.21.15.29:0/1302182272 wait complete. 2024-04-25T00:38:35.139 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:35.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99e4a36640 1 Processor -- start 2024-04-25T00:38:35.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99e4a36640 1 -- start start 2024-04-25T00:38:35.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99e4a36640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f99e00cf690 0x7f99e00d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99e4a36640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99e00d2a90 0x7f99e00b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99e4a36640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99e00cfc10 con 0x7f99e015e200 2024-04-25T00:38:35.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99e4a36640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f99e00d2ee0 con 0x7f99e00cf690 2024-04-25T00:38:35.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99e4a36640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f99e00b38d0 con 0x7f99e00d2a90 2024-04-25T00:38:35.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99d7fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f99e00cf690 0x7f99e00d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99ded9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99e00d2a90 0x7f99e00b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99de59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f99e015e200 0x7f99e00d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:59104/0 (socket says 172.21.15.29:59104) 2024-04-25T00:38:35.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99de59c640 1 -- 172.21.15.29:0/1816469099 learned_addr learned my addr 172.21.15.29:0/1816469099 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:35.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99d7fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f99e00cf690 0x7f99e00d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:41300/0 (socket says 172.21.15.29:41300) 2024-04-25T00:38:35.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99ded9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99e00d2a90 0x7f99e00b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:37142/0 (socket says 172.21.15.29:37142) 2024-04-25T00:38:35.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99ded9d640 1 -- 172.21.15.29:0/1816469099 >> v2:172.21.15.157:3300/0 conn(0x7f99e00cf690 msgr2=0x7f99e00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:35.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99ded9d640 1 --2- 172.21.15.29:0/1816469099 >> v2:172.21.15.157:3300/0 conn(0x7f99e00cf690 0x7f99e00d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99ded9d640 1 -- 172.21.15.29:0/1816469099 >> v1:172.21.15.157:6789/0 conn(0x7f99e015e200 legacy=0x7f99e00d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99ded9d640 1 -- 172.21.15.29:0/1816469099 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e00cfef0 con 0x7f99e00d2a90 2024-04-25T00:38:35.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.262+0000 7f99d7fff640 1 --2- 172.21.15.29:0/1816469099 >> v2:172.21.15.157:3300/0 conn(0x7f99e00cf690 0x7f99e00d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:38:35.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99ded9d640 1 --2- 172.21.15.29:0/1816469099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99e00d2a90 0x7f99e00b4320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009d90 tx=0x7f99d801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bf3eacf45ab2157 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:35.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99ddd9b640 1 -- 172.21.15.29:0/1816469099 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d80b2070 con 0x7f99e00d2a90 2024-04-25T00:38:35.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99ddd9b640 1 -- 172.21.15.29:0/1816469099 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d80a5920 con 0x7f99e00d2a90 2024-04-25T00:38:35.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99ddd9b640 1 -- 172.21.15.29:0/1816469099 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d80a5cf0 con 0x7f99e00d2a90 2024-04-25T00:38:35.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 -- 172.21.15.29:0/1816469099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99e00d2a90 msgr2=0x7f99e00b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 --2- 172.21.15.29:0/1816469099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99e00d2a90 0x7f99e00b4320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009d90 tx=0x7f99d801c970 comp rx=0 tx=0).stop 2024-04-25T00:38:35.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 -- 172.21.15.29:0/1816469099 shutdown_connections 2024-04-25T00:38:35.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 --2- 172.21.15.29:0/1816469099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99e00d2a90 0x7f99e00b4320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 --2- 172.21.15.29:0/1816469099 >> v2:172.21.15.157:3300/0 conn(0x7f99e00cf690 0x7f99e00d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 -- 172.21.15.29:0/1816469099 >> 172.21.15.29:0/1816469099 conn(0x7f99e009d280 msgr2=0x7f99e00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:35.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 -- 172.21.15.29:0/1816469099 shutdown_connections 2024-04-25T00:38:35.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 -- 172.21.15.29:0/1816469099 wait complete. 2024-04-25T00:38:35.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 Processor -- start 2024-04-25T00:38:35.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 -- start start 2024-04-25T00:38:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99e00cf690 0x7f99e01576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99e00d2a90 0x7f99e00cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e015e200 0x7f99e015c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f99e00b51f0 con 0x7f99e00d2a90 2024-04-25T00:38:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99e00b5070 con 0x7f99e015e200 2024-04-25T00:38:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99e4a36640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f99e00b5370 con 0x7f99e00cf690 2024-04-25T00:38:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99d7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99e00d2a90 0x7f99e00cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99d7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99e00d2a90 0x7f99e00cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54052/0 (socket says 172.21.15.29:54052) 2024-04-25T00:38:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.266+0000 7f99d7fff640 1 -- 172.21.15.29:0/635668191 learned_addr learned my addr 172.21.15.29:0/635668191 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99ded9d640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e015e200 0x7f99e015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99de59c640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99e00cf690 0x7f99e01576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99d7fff640 1 -- 172.21.15.29:0/635668191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99e00cf690 msgr2=0x7f99e01576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99d7fff640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99e00cf690 0x7f99e01576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99d7fff640 1 -- 172.21.15.29:0/635668191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e015e200 msgr2=0x7f99e015c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99d7fff640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e015e200 0x7f99e015c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99d7fff640 1 -- 172.21.15.29:0/635668191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e015fdc0 con 0x7f99e00d2a90 2024-04-25T00:38:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99d7fff640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99e00d2a90 0x7f99e00cd930 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f99d000ec10 tx=0x7f99d000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99d6ffd640 1 -- 172.21.15.29:0/635668191 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d0017070 con 0x7f99e00d2a90 2024-04-25T00:38:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99d6ffd640 1 -- 172.21.15.29:0/635668191 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d000cc30 con 0x7f99e00d2a90 2024-04-25T00:38:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99e4a36640 1 -- 172.21.15.29:0/635668191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e01600a0 con 0x7f99e00d2a90 2024-04-25T00:38:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99d6ffd640 1 -- 172.21.15.29:0/635668191 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d0013470 con 0x7f99e00d2a90 2024-04-25T00:38:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99e4a36640 1 -- 172.21.15.29:0/635668191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e0160940 con 0x7f99e00d2a90 2024-04-25T00:38:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99e4a36640 1 -- 172.21.15.29:0/635668191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e0069270 con 0x7f99e00d2a90 2024-04-25T00:38:35.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99d6ffd640 1 -- 172.21.15.29:0/635668191 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f99d001a070 con 0x7f99e00d2a90 2024-04-25T00:38:35.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99d6ffd640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f99c0067cb0 0x7f99c006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99d6ffd640 1 -- 172.21.15.29:0/635668191 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f99d00866a0 con 0x7f99e00d2a90 2024-04-25T00:38:35.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99de59c640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f99c0067cb0 0x7f99c006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.270+0000 7f99de59c640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f99c0067cb0 0x7f99c006a170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f99c8005890 tx=0x7f99c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:35.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.274+0000 7f99d6ffd640 1 -- 172.21.15.29:0/635668191 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f99d0056c90 con 0x7f99e00d2a90 2024-04-25T00:38:35.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.430+0000 7f99e4a36640 1 -- 172.21.15.29:0/635668191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99e015fba0 con 0x7f99e00d2a90 2024-04-25T00:38:35.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.434+0000 7f99d6ffd640 1 -- 172.21.15.29:0/635668191 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16631 (secure 0 0 0) 0x7f99d0056590 con 0x7f99e00d2a90 2024-04-25T00:38:35.438 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:35.438 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":118,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:32.730257+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:38:28.743593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:38:30.051271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:35.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.438+0000 7f99e4a36640 1 -- 172.21.15.29:0/635668191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f99c0067cb0 msgr2=0x7f99c006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.438+0000 7f99e4a36640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f99c0067cb0 0x7f99c006a170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f99c8005890 tx=0x7f99c8009040 comp rx=0 tx=0).stop 2024-04-25T00:38:35.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.438+0000 7f99e4a36640 1 -- 172.21.15.29:0/635668191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99e00d2a90 msgr2=0x7f99e00cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.438+0000 7f99e4a36640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99e00d2a90 0x7f99e00cd930 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f99d000ec10 tx=0x7f99d000c520 comp rx=0 tx=0).stop 2024-04-25T00:38:35.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.438+0000 7f99e4a36640 1 -- 172.21.15.29:0/635668191 shutdown_connections 2024-04-25T00:38:35.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.438+0000 7f99e4a36640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f99c0067cb0 0x7f99c006a170 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.438+0000 7f99e4a36640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e015e200 0x7f99e015c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.438+0000 7f99e4a36640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99e00d2a90 0x7f99e00cd930 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.438+0000 7f99e4a36640 1 --2- 172.21.15.29:0/635668191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99e00cf690 0x7f99e01576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.438+0000 7f99e4a36640 1 -- 172.21.15.29:0/635668191 >> 172.21.15.29:0/635668191 conn(0x7f99e009d280 msgr2=0x7f99e00d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:35.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.438+0000 7f99e4a36640 1 -- 172.21.15.29:0/635668191 shutdown_connections 2024-04-25T00:38:35.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.438+0000 7f99e4a36640 1 -- 172.21.15.29:0/635668191 wait complete. 2024-04-25T00:38:35.456 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:38:35.456 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:38:35.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.562+0000 7fe186372640 1 Processor -- start 2024-04-25T00:38:35.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe186372640 1 -- start start 2024-04-25T00:38:35.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe186372640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe1800cf690 0x7fe1800d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe186372640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1800d2a90 0x7fe1800b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe186372640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1800cfc10 con 0x7fe18015e200 2024-04-25T00:38:35.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe186372640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe1800d2ee0 con 0x7fe1800cf690 2024-04-25T00:38:35.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe186372640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe1800b38d0 con 0x7fe1800d2a90 2024-04-25T00:38:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe17f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe1800cf690 0x7fe1800d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe18490f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1800d2a90 0x7fe1800b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe17ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe18015e200 0x7fe1800d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:59108/0 (socket says 172.21.15.29:59108) 2024-04-25T00:38:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe17f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe1800cf690 0x7fe1800d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54056/0 (socket says 172.21.15.29:54056) 2024-04-25T00:38:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe17ffff640 1 -- 172.21.15.29:0/4273313247 learned_addr learned my addr 172.21.15.29:0/4273313247 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe17f7fe640 1 -- 172.21.15.29:0/4273313247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1800d2a90 msgr2=0x7fe1800b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe17f7fe640 1 --2- 172.21.15.29:0/4273313247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1800d2a90 0x7fe1800b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe17f7fe640 1 -- 172.21.15.29:0/4273313247 >> v1:172.21.15.157:6789/0 conn(0x7fe18015e200 legacy=0x7fe1800d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe17f7fe640 1 -- 172.21.15.29:0/4273313247 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1800cfef0 con 0x7fe1800cf690 2024-04-25T00:38:35.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.566+0000 7fe17f7fe640 1 --2- 172.21.15.29:0/4273313247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1800cf690 0x7fe1800d2550 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe1740097c0 tx=0x7fe17401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbdd1f03b4d212e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:35.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17effd640 1 -- 172.21.15.29:0/4273313247 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1740b2070 con 0x7fe1800cf690 2024-04-25T00:38:35.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17effd640 1 -- 172.21.15.29:0/4273313247 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1740ad640 con 0x7fe1800cf690 2024-04-25T00:38:35.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17effd640 1 -- 172.21.15.29:0/4273313247 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1740ada10 con 0x7fe1800cf690 2024-04-25T00:38:35.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 -- 172.21.15.29:0/4273313247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1800cf690 msgr2=0x7fe1800d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 --2- 172.21.15.29:0/4273313247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1800cf690 0x7fe1800d2550 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe1740097c0 tx=0x7fe17401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 -- 172.21.15.29:0/4273313247 shutdown_connections 2024-04-25T00:38:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 --2- 172.21.15.29:0/4273313247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1800d2a90 0x7fe1800b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 --2- 172.21.15.29:0/4273313247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1800cf690 0x7fe1800d2550 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 -- 172.21.15.29:0/4273313247 >> 172.21.15.29:0/4273313247 conn(0x7fe18009d280 msgr2=0x7fe1800d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 -- 172.21.15.29:0/4273313247 shutdown_connections 2024-04-25T00:38:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 -- 172.21.15.29:0/4273313247 wait complete. 2024-04-25T00:38:35.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 Processor -- start 2024-04-25T00:38:35.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 -- start start 2024-04-25T00:38:35.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1800cf690 0x7fe1800d42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1800d2a90 0x7fe1800c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18015e200 0x7fe1800c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe1800b51f0 con 0x7fe1800d2a90 2024-04-25T00:38:35.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe1800b5070 con 0x7fe1800cf690 2024-04-25T00:38:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe1800b5370 con 0x7fe18015e200 2024-04-25T00:38:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1800d2a90 0x7fe1800c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1800d2a90 0x7fe1800c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54070/0 (socket says 172.21.15.29:54070) 2024-04-25T00:38:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17f7fe640 1 -- 172.21.15.29:0/1134043314 learned_addr learned my addr 172.21.15.29:0/1134043314 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17f7fe640 1 -- 172.21.15.29:0/1134043314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18015e200 msgr2=0x7fe1800c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17f7fe640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18015e200 0x7fe1800c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17f7fe640 1 -- 172.21.15.29:0/1134043314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1800cf690 msgr2=0x7fe1800d42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17ffff640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1800cf690 0x7fe1800d42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17f7fe640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1800cf690 0x7fe1800d42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17f7fe640 1 -- 172.21.15.29:0/1134043314 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe180160200 con 0x7fe1800d2a90 2024-04-25T00:38:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17ffff640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1800cf690 0x7fe1800d42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17f7fe640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1800d2a90 0x7fe1800c9980 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe1740b0040 tx=0x7fe174019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17cff9640 1 -- 172.21.15.29:0/1134043314 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1740b2040 con 0x7fe1800d2a90 2024-04-25T00:38:35.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17cff9640 1 -- 172.21.15.29:0/1134043314 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1740bb070 con 0x7fe1800d2a90 2024-04-25T00:38:35.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 -- 172.21.15.29:0/1134043314 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1801604e0 con 0x7fe1800d2a90 2024-04-25T00:38:35.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe17cff9640 1 -- 172.21.15.29:0/1134043314 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174006e20 con 0x7fe1800d2a90 2024-04-25T00:38:35.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.570+0000 7fe186372640 1 -- 172.21.15.29:0/1134043314 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe180160aa0 con 0x7fe1800d2a90 2024-04-25T00:38:35.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.574+0000 7fe186372640 1 -- 172.21.15.29:0/1134043314 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe180069270 con 0x7fe1800d2a90 2024-04-25T00:38:35.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.574+0000 7fe17cff9640 1 -- 172.21.15.29:0/1134043314 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fe1740b6690 con 0x7fe1800d2a90 2024-04-25T00:38:35.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.574+0000 7fe17cff9640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe16c067ce0 0x7fe16c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.574+0000 7fe17ffff640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe16c067ce0 0x7fe16c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.574+0000 7fe17cff9640 1 -- 172.21.15.29:0/1134043314 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fe174120250 con 0x7fe1800d2a90 2024-04-25T00:38:35.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.574+0000 7fe17ffff640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe16c067ce0 0x7fe16c06a1a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe170007fd0 tx=0x7fe170009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:35.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.578+0000 7fe17cff9640 1 -- 172.21.15.29:0/1134043314 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe174016090 con 0x7fe1800d2a90 2024-04-25T00:38:35.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.730+0000 7fe186372640 1 -- 172.21.15.29:0/1134043314 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1800d22b0 con 0x7fe16c067ce0 2024-04-25T00:38:35.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.742+0000 7fe17cff9640 1 -- 172.21.15.29:0/1134043314 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7fe1800d22b0 con 0x7fe16c067ce0 2024-04-25T00:38:35.748 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:35.750 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:38:35.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.746+0000 7fe186372640 1 -- 172.21.15.29:0/1134043314 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe16c067ce0 msgr2=0x7fe16c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.746+0000 7fe186372640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe16c067ce0 0x7fe16c06a1a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe170007fd0 tx=0x7fe170009040 comp rx=0 tx=0).stop 2024-04-25T00:38:35.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.746+0000 7fe186372640 1 -- 172.21.15.29:0/1134043314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1800d2a90 msgr2=0x7fe1800c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.746+0000 7fe186372640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1800d2a90 0x7fe1800c9980 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe1740b0040 tx=0x7fe174019240 comp rx=0 tx=0).stop 2024-04-25T00:38:35.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.746+0000 7fe186372640 1 -- 172.21.15.29:0/1134043314 shutdown_connections 2024-04-25T00:38:35.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.746+0000 7fe186372640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe16c067ce0 0x7fe16c06a1a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.746+0000 7fe186372640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe18015e200 0x7fe1800c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.746+0000 7fe186372640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1800d2a90 0x7fe1800c9980 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.746+0000 7fe186372640 1 --2- 172.21.15.29:0/1134043314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1800cf690 0x7fe1800d42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.746+0000 7fe186372640 1 -- 172.21.15.29:0/1134043314 >> 172.21.15.29:0/1134043314 conn(0x7fe18009d280 msgr2=0x7fe18015d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:35.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.746+0000 7fe186372640 1 -- 172.21.15.29:0/1134043314 shutdown_connections 2024-04-25T00:38:35.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.746+0000 7fe186372640 1 -- 172.21.15.29:0/1134043314 wait complete. 2024-04-25T00:38:35.765 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-04-25T00:38:35.369854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":295444,"kb_used_data":12720,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754679276,"statfs":{"total":773094113280,"available":772791578624,"internally_reserved":0,"allocated":13025280,"data_stored":6372004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.697740"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920319+0000","last_change":"2024-04-25T00:38:31.750455+0000","last_active":"2024-04-25T00:38:33.920319+0000","last_peered":"2024-04-25T00:38:33.920319+0000","last_clean":"2024-04-25T00:38:33.920319+0000","last_became_active":"2024-04-25T00:38:31.750123+0000","last_became_peered":"2024-04-25T00:38:31.750123+0000","last_unstale":"2024-04-25T00:38:33.920319+0000","last_undegraded":"2024-04-25T00:38:33.920319+0000","last_fullsized":"2024-04-25T00:38:33.920319+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:38.678137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736223+0000","last_change":"2024-04-25T00:38:31.745202+0000","last_active":"2024-04-25T00:38:32.736223+0000","last_peered":"2024-04-25T00:38:32.736223+0000","last_clean":"2024-04-25T00:38:32.736223+0000","last_became_active":"2024-04-25T00:38:31.744799+0000","last_became_peered":"2024-04-25T00:38:31.744799+0000","last_unstale":"2024-04-25T00:38:32.736223+0000","last_undegraded":"2024-04-25T00:38:32.736223+0000","last_fullsized":"2024-04-25T00:38:32.736223+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:06.451011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.822106+0000","last_change":"2024-04-25T00:38:31.739015+0000","last_active":"2024-04-25T00:38:32.822106+0000","last_peered":"2024-04-25T00:38:32.822106+0000","last_clean":"2024-04-25T00:38:32.822106+0000","last_became_active":"2024-04-25T00:38:31.738155+0000","last_became_peered":"2024-04-25T00:38:31.738155+0000","last_unstale":"2024-04-25T00:38:32.822106+0000","last_undegraded":"2024-04-25T00:38:32.822106+0000","last_fullsized":"2024-04-25T00:38:32.822106+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:06.525021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920648+0000","last_change":"2024-04-25T00:38:31.743794+0000","last_active":"2024-04-25T00:38:33.920648+0000","last_peered":"2024-04-25T00:38:33.920648+0000","last_clean":"2024-04-25T00:38:33.920648+0000","last_became_active":"2024-04-25T00:38:31.743321+0000","last_became_peered":"2024-04-25T00:38:31.743321+0000","last_unstale":"2024-04-25T00:38:33.920648+0000","last_undegraded":"2024-04-25T00:38:33.920648+0000","last_fullsized":"2024-04-25T00:38:33.920648+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:24.320498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745181+0000","last_change":"2024-04-25T00:38:31.750696+0000","last_active":"2024-04-25T00:38:32.745181+0000","last_peered":"2024-04-25T00:38:32.745181+0000","last_clean":"2024-04-25T00:38:32.745181+0000","last_became_active":"2024-04-25T00:38:31.742792+0000","last_became_peered":"2024-04-25T00:38:31.742792+0000","last_unstale":"2024-04-25T00:38:32.745181+0000","last_undegraded":"2024-04-25T00:38:32.745181+0000","last_fullsized":"2024-04-25T00:38:32.745181+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:47.304639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.737950+0000","last_change":"2024-04-25T00:38:31.771960+0000","last_active":"2024-04-25T00:38:32.737950+0000","last_peered":"2024-04-25T00:38:32.737950+0000","last_clean":"2024-04-25T00:38:32.737950+0000","last_became_active":"2024-04-25T00:38:31.771349+0000","last_became_peered":"2024-04-25T00:38:31.771349+0000","last_unstale":"2024-04-25T00:38:32.737950+0000","last_undegraded":"2024-04-25T00:38:32.737950+0000","last_fullsized":"2024-04-25T00:38:32.737950+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:17.578134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.737330+0000","last_change":"2024-04-25T00:38:31.763195+0000","last_active":"2024-04-25T00:38:32.737330+0000","last_peered":"2024-04-25T00:38:32.737330+0000","last_clean":"2024-04-25T00:38:32.737330+0000","last_became_active":"2024-04-25T00:38:31.762439+0000","last_became_peered":"2024-04-25T00:38:31.762439+0000","last_unstale":"2024-04-25T00:38:32.737330+0000","last_undegraded":"2024-04-25T00:38:32.737330+0000","last_fullsized":"2024-04-25T00:38:32.737330+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:59.591934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821526+0000","last_change":"2024-04-25T00:38:31.748621+0000","last_active":"2024-04-25T00:38:32.821526+0000","last_peered":"2024-04-25T00:38:32.821526+0000","last_clean":"2024-04-25T00:38:32.821526+0000","last_became_active":"2024-04-25T00:38:31.747819+0000","last_became_peered":"2024-04-25T00:38:31.747819+0000","last_unstale":"2024-04-25T00:38:32.821526+0000","last_undegraded":"2024-04-25T00:38:32.821526+0000","last_fullsized":"2024-04-25T00:38:32.821526+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:40.471722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.822161+0000","last_change":"2024-04-25T00:38:31.737196+0000","last_active":"2024-04-25T00:38:32.822161+0000","last_peered":"2024-04-25T00:38:32.822161+0000","last_clean":"2024-04-25T00:38:32.822161+0000","last_became_active":"2024-04-25T00:38:31.736469+0000","last_became_peered":"2024-04-25T00:38:31.736469+0000","last_unstale":"2024-04-25T00:38:32.822161+0000","last_undegraded":"2024-04-25T00:38:32.822161+0000","last_fullsized":"2024-04-25T00:38:32.822161+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:27.701139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736635+0000","last_change":"2024-04-25T00:38:31.738296+0000","last_active":"2024-04-25T00:38:32.736635+0000","last_peered":"2024-04-25T00:38:32.736635+0000","last_clean":"2024-04-25T00:38:32.736635+0000","last_became_active":"2024-04-25T00:38:31.737927+0000","last_became_peered":"2024-04-25T00:38:31.737927+0000","last_unstale":"2024-04-25T00:38:32.736635+0000","last_undegraded":"2024-04-25T00:38:32.736635+0000","last_fullsized":"2024-04-25T00:38:32.736635+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:42.975568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920560+0000","last_change":"2024-04-25T00:38:31.742622+0000","last_active":"2024-04-25T00:38:33.920560+0000","last_peered":"2024-04-25T00:38:33.920560+0000","last_clean":"2024-04-25T00:38:33.920560+0000","last_became_active":"2024-04-25T00:38:31.742229+0000","last_became_peered":"2024-04-25T00:38:31.742229+0000","last_unstale":"2024-04-25T00:38:33.920560+0000","last_undegraded":"2024-04-25T00:38:33.920560+0000","last_fullsized":"2024-04-25T00:38:33.920560+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:13.965812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821938+0000","last_change":"2024-04-25T00:38:31.728449+0000","last_active":"2024-04-25T00:38:32.821938+0000","last_peered":"2024-04-25T00:38:32.821938+0000","last_clean":"2024-04-25T00:38:32.821938+0000","last_became_active":"2024-04-25T00:38:31.727687+0000","last_became_peered":"2024-04-25T00:38:31.727687+0000","last_unstale":"2024-04-25T00:38:32.821938+0000","last_undegraded":"2024-04-25T00:38:32.821938+0000","last_fullsized":"2024-04-25T00:38:32.821938+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:22.869254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920648+0000","last_change":"2024-04-25T00:38:31.764830+0000","last_active":"2024-04-25T00:38:33.920648+0000","last_peered":"2024-04-25T00:38:33.920648+0000","last_clean":"2024-04-25T00:38:33.920648+0000","last_became_active":"2024-04-25T00:38:31.764539+0000","last_became_peered":"2024-04-25T00:38:31.764539+0000","last_unstale":"2024-04-25T00:38:33.920648+0000","last_undegraded":"2024-04-25T00:38:33.920648+0000","last_fullsized":"2024-04-25T00:38:33.920648+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:37.877652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736209+0000","last_change":"2024-04-25T00:38:31.753441+0000","last_active":"2024-04-25T00:38:32.736209+0000","last_peered":"2024-04-25T00:38:32.736209+0000","last_clean":"2024-04-25T00:38:32.736209+0000","last_became_active":"2024-04-25T00:38:31.748127+0000","last_became_peered":"2024-04-25T00:38:31.748127+0000","last_unstale":"2024-04-25T00:38:32.736209+0000","last_undegraded":"2024-04-25T00:38:32.736209+0000","last_fullsized":"2024-04-25T00:38:32.736209+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:03.308012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920411+0000","last_change":"2024-04-25T00:38:31.763270+0000","last_active":"2024-04-25T00:38:33.920411+0000","last_peered":"2024-04-25T00:38:33.920411+0000","last_clean":"2024-04-25T00:38:33.920411+0000","last_became_active":"2024-04-25T00:38:31.762916+0000","last_became_peered":"2024-04-25T00:38:31.762916+0000","last_unstale":"2024-04-25T00:38:33.920411+0000","last_undegraded":"2024-04-25T00:38:33.920411+0000","last_fullsized":"2024-04-25T00:38:33.920411+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:53.973770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302573+0000","last_change":"2024-04-25T00:38:31.755327+0000","last_active":"2024-04-25T00:38:33.302573+0000","last_peered":"2024-04-25T00:38:33.302573+0000","last_clean":"2024-04-25T00:38:33.302573+0000","last_became_active":"2024-04-25T00:38:31.755040+0000","last_became_peered":"2024-04-25T00:38:31.755040+0000","last_unstale":"2024-04-25T00:38:33.302573+0000","last_undegraded":"2024-04-25T00:38:33.302573+0000","last_fullsized":"2024-04-25T00:38:33.302573+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:06.369170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.740700+0000","last_change":"2024-04-25T00:38:31.744770+0000","last_active":"2024-04-25T00:38:32.740700+0000","last_peered":"2024-04-25T00:38:32.740700+0000","last_clean":"2024-04-25T00:38:32.740700+0000","last_became_active":"2024-04-25T00:38:31.744033+0000","last_became_peered":"2024-04-25T00:38:31.744033+0000","last_unstale":"2024-04-25T00:38:32.740700+0000","last_undegraded":"2024-04-25T00:38:32.740700+0000","last_fullsized":"2024-04-25T00:38:32.740700+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:42.678736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302627+0000","last_change":"2024-04-25T00:38:31.761019+0000","last_active":"2024-04-25T00:38:33.302627+0000","last_peered":"2024-04-25T00:38:33.302627+0000","last_clean":"2024-04-25T00:38:33.302627+0000","last_became_active":"2024-04-25T00:38:31.760714+0000","last_became_peered":"2024-04-25T00:38:31.760714+0000","last_unstale":"2024-04-25T00:38:33.302627+0000","last_undegraded":"2024-04-25T00:38:33.302627+0000","last_fullsized":"2024-04-25T00:38:33.302627+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:11.450419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920714+0000","last_change":"2024-04-25T00:38:31.745662+0000","last_active":"2024-04-25T00:38:33.920714+0000","last_peered":"2024-04-25T00:38:33.920714+0000","last_clean":"2024-04-25T00:38:33.920714+0000","last_became_active":"2024-04-25T00:38:31.745410+0000","last_became_peered":"2024-04-25T00:38:31.745410+0000","last_unstale":"2024-04-25T00:38:33.920714+0000","last_undegraded":"2024-04-25T00:38:33.920714+0000","last_fullsized":"2024-04-25T00:38:33.920714+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:37.369926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302885+0000","last_change":"2024-04-25T00:38:31.746910+0000","last_active":"2024-04-25T00:38:33.302885+0000","last_peered":"2024-04-25T00:38:33.302885+0000","last_clean":"2024-04-25T00:38:33.302885+0000","last_became_active":"2024-04-25T00:38:31.746485+0000","last_became_peered":"2024-04-25T00:38:31.746485+0000","last_unstale":"2024-04-25T00:38:33.302885+0000","last_undegraded":"2024-04-25T00:38:33.302885+0000","last_fullsized":"2024-04-25T00:38:33.302885+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:48.095477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745766+0000","last_change":"2024-04-25T00:38:31.742114+0000","last_active":"2024-04-25T00:38:32.745766+0000","last_peered":"2024-04-25T00:38:32.745766+0000","last_clean":"2024-04-25T00:38:32.745766+0000","last_became_active":"2024-04-25T00:38:31.741368+0000","last_became_peered":"2024-04-25T00:38:31.741368+0000","last_unstale":"2024-04-25T00:38:32.745766+0000","last_undegraded":"2024-04-25T00:38:32.745766+0000","last_fullsized":"2024-04-25T00:38:32.745766+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:17.790811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.919819+0000","last_change":"2024-04-25T00:38:31.741485+0000","last_active":"2024-04-25T00:38:33.919819+0000","last_peered":"2024-04-25T00:38:33.919819+0000","last_clean":"2024-04-25T00:38:33.919819+0000","last_became_active":"2024-04-25T00:38:31.734862+0000","last_became_peered":"2024-04-25T00:38:31.734862+0000","last_unstale":"2024-04-25T00:38:33.919819+0000","last_undegraded":"2024-04-25T00:38:33.919819+0000","last_fullsized":"2024-04-25T00:38:33.919819+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:12.185180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821549+0000","last_change":"2024-04-25T00:38:31.731366+0000","last_active":"2024-04-25T00:38:32.821549+0000","last_peered":"2024-04-25T00:38:32.821549+0000","last_clean":"2024-04-25T00:38:32.821549+0000","last_became_active":"2024-04-25T00:38:31.729678+0000","last_became_peered":"2024-04-25T00:38:31.729678+0000","last_unstale":"2024-04-25T00:38:32.821549+0000","last_undegraded":"2024-04-25T00:38:32.821549+0000","last_fullsized":"2024-04-25T00:38:32.821549+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:44.984094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.919732+0000","last_change":"2024-04-25T00:38:31.745470+0000","last_active":"2024-04-25T00:38:33.919732+0000","last_peered":"2024-04-25T00:38:33.919732+0000","last_clean":"2024-04-25T00:38:33.919732+0000","last_became_active":"2024-04-25T00:38:31.745075+0000","last_became_peered":"2024-04-25T00:38:31.745075+0000","last_unstale":"2024-04-25T00:38:33.919732+0000","last_undegraded":"2024-04-25T00:38:33.919732+0000","last_fullsized":"2024-04-25T00:38:33.919732+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:08.429264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736663+0000","last_change":"2024-04-25T00:38:31.754062+0000","last_active":"2024-04-25T00:38:32.736663+0000","last_peered":"2024-04-25T00:38:32.736663+0000","last_clean":"2024-04-25T00:38:32.736663+0000","last_became_active":"2024-04-25T00:38:31.748296+0000","last_became_peered":"2024-04-25T00:38:31.748296+0000","last_unstale":"2024-04-25T00:38:32.736663+0000","last_undegraded":"2024-04-25T00:38:32.736663+0000","last_fullsized":"2024-04-25T00:38:32.736663+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:17.616702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302798+0000","last_change":"2024-04-25T00:38:31.755756+0000","last_active":"2024-04-25T00:38:33.302798+0000","last_peered":"2024-04-25T00:38:33.302798+0000","last_clean":"2024-04-25T00:38:33.302798+0000","last_became_active":"2024-04-25T00:38:31.755150+0000","last_became_peered":"2024-04-25T00:38:31.755150+0000","last_unstale":"2024-04-25T00:38:33.302798+0000","last_undegraded":"2024-04-25T00:38:33.302798+0000","last_fullsized":"2024-04-25T00:38:33.302798+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:36.801538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920730+0000","last_change":"2024-04-25T00:38:31.740031+0000","last_active":"2024-04-25T00:38:33.920730+0000","last_peered":"2024-04-25T00:38:33.920730+0000","last_clean":"2024-04-25T00:38:33.920730+0000","last_became_active":"2024-04-25T00:38:31.739657+0000","last_became_peered":"2024-04-25T00:38:31.739657+0000","last_unstale":"2024-04-25T00:38:33.920730+0000","last_undegraded":"2024-04-25T00:38:33.920730+0000","last_fullsized":"2024-04-25T00:38:33.920730+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:52.211728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739297+0000","last_change":"2024-04-25T00:38:31.732022+0000","last_active":"2024-04-25T00:38:32.739297+0000","last_peered":"2024-04-25T00:38:32.739297+0000","last_clean":"2024-04-25T00:38:32.739297+0000","last_became_active":"2024-04-25T00:38:31.731135+0000","last_became_peered":"2024-04-25T00:38:31.731135+0000","last_unstale":"2024-04-25T00:38:32.739297+0000","last_undegraded":"2024-04-25T00:38:32.739297+0000","last_fullsized":"2024-04-25T00:38:32.739297+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:17.326905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920144+0000","last_change":"2024-04-25T00:38:31.763176+0000","last_active":"2024-04-25T00:38:33.920144+0000","last_peered":"2024-04-25T00:38:33.920144+0000","last_clean":"2024-04-25T00:38:33.920144+0000","last_became_active":"2024-04-25T00:38:31.762553+0000","last_became_peered":"2024-04-25T00:38:31.762553+0000","last_unstale":"2024-04-25T00:38:33.920144+0000","last_undegraded":"2024-04-25T00:38:33.920144+0000","last_fullsized":"2024-04-25T00:38:33.920144+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:46.062663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920153+0000","last_change":"2024-04-25T00:38:31.762771+0000","last_active":"2024-04-25T00:38:33.920153+0000","last_peered":"2024-04-25T00:38:33.920153+0000","last_clean":"2024-04-25T00:38:33.920153+0000","last_became_active":"2024-04-25T00:38:31.762438+0000","last_became_peered":"2024-04-25T00:38:31.762438+0000","last_unstale":"2024-04-25T00:38:33.920153+0000","last_undegraded":"2024-04-25T00:38:33.920153+0000","last_fullsized":"2024-04-25T00:38:33.920153+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:56.381161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.744750+0000","last_change":"2024-04-25T00:38:31.755867+0000","last_active":"2024-04-25T00:38:32.744750+0000","last_peered":"2024-04-25T00:38:32.744750+0000","last_clean":"2024-04-25T00:38:32.744750+0000","last_became_active":"2024-04-25T00:38:31.754368+0000","last_became_peered":"2024-04-25T00:38:31.754368+0000","last_unstale":"2024-04-25T00:38:32.744750+0000","last_undegraded":"2024-04-25T00:38:32.744750+0000","last_fullsized":"2024-04-25T00:38:32.744750+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:11.450419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.737416+0000","last_change":"2024-04-25T00:38:31.761098+0000","last_active":"2024-04-25T00:38:32.737416+0000","last_peered":"2024-04-25T00:38:32.737416+0000","last_clean":"2024-04-25T00:38:32.737416+0000","last_became_active":"2024-04-25T00:38:31.760501+0000","last_became_peered":"2024-04-25T00:38:31.760501+0000","last_unstale":"2024-04-25T00:38:32.737416+0000","last_undegraded":"2024-04-25T00:38:32.737416+0000","last_fullsized":"2024-04-25T00:38:32.737416+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:34.431532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.303073+0000","last_change":"2024-04-25T00:38:31.760487+0000","last_active":"2024-04-25T00:38:33.303073+0000","last_peered":"2024-04-25T00:38:33.303073+0000","last_clean":"2024-04-25T00:38:33.303073+0000","last_became_active":"2024-04-25T00:38:31.760132+0000","last_became_peered":"2024-04-25T00:38:31.760132+0000","last_unstale":"2024-04-25T00:38:33.303073+0000","last_undegraded":"2024-04-25T00:38:33.303073+0000","last_fullsized":"2024-04-25T00:38:33.303073+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:47.304639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"117'4","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745583+0000","last_change":"2024-04-25T00:38:30.715226+0000","last_active":"2024-04-25T00:38:32.745583+0000","last_peered":"2024-04-25T00:38:32.745583+0000","last_clean":"2024-04-25T00:38:32.745583+0000","last_became_active":"2024-04-25T00:38:30.714540+0000","last_became_peered":"2024-04-25T00:38:30.714540+0000","last_unstale":"2024-04-25T00:38:32.745583+0000","last_undegraded":"2024-04-25T00:38:32.745583+0000","last_fullsized":"2024-04-25T00:38:32.745583+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:22.159841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.822206+0000","last_change":"2024-04-25T00:38:31.763149+0000","last_active":"2024-04-25T00:38:32.822206+0000","last_peered":"2024-04-25T00:38:32.822206+0000","last_clean":"2024-04-25T00:38:32.822206+0000","last_became_active":"2024-04-25T00:38:31.762541+0000","last_became_peered":"2024-04-25T00:38:31.762541+0000","last_unstale":"2024-04-25T00:38:32.822206+0000","last_undegraded":"2024-04-25T00:38:32.822206+0000","last_fullsized":"2024-04-25T00:38:32.822206+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:12.797380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739772+0000","last_change":"2024-04-25T00:38:30.699015+0000","last_active":"2024-04-25T00:38:32.739772+0000","last_peered":"2024-04-25T00:38:32.739772+0000","last_clean":"2024-04-25T00:38:32.739772+0000","last_became_active":"2024-04-25T00:38:30.698309+0000","last_became_peered":"2024-04-25T00:38:30.698309+0000","last_unstale":"2024-04-25T00:38:32.739772+0000","last_undegraded":"2024-04-25T00:38:32.739772+0000","last_fullsized":"2024-04-25T00:38:32.739772+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:30.060355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745357+0000","last_change":"2024-04-25T00:38:31.747829+0000","last_active":"2024-04-25T00:38:32.745357+0000","last_peered":"2024-04-25T00:38:32.745357+0000","last_clean":"2024-04-25T00:38:32.745357+0000","last_became_active":"2024-04-25T00:38:31.746279+0000","last_became_peered":"2024-04-25T00:38:31.746279+0000","last_unstale":"2024-04-25T00:38:32.745357+0000","last_undegraded":"2024-04-25T00:38:32.745357+0000","last_fullsized":"2024-04-25T00:38:32.745357+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:33.831876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920824+0000","last_change":"2024-04-25T00:38:30.701744+0000","last_active":"2024-04-25T00:38:33.920824+0000","last_peered":"2024-04-25T00:38:33.920824+0000","last_clean":"2024-04-25T00:38:33.920824+0000","last_became_active":"2024-04-25T00:38:30.700428+0000","last_became_peered":"2024-04-25T00:38:30.700428+0000","last_unstale":"2024-04-25T00:38:33.920824+0000","last_undegraded":"2024-04-25T00:38:33.920824+0000","last_fullsized":"2024-04-25T00:38:33.920824+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:12.378372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745080+0000","last_change":"2024-04-25T00:38:31.744755+0000","last_active":"2024-04-25T00:38:32.745080+0000","last_peered":"2024-04-25T00:38:32.745080+0000","last_clean":"2024-04-25T00:38:32.745080+0000","last_became_active":"2024-04-25T00:38:31.743910+0000","last_became_peered":"2024-04-25T00:38:31.743910+0000","last_unstale":"2024-04-25T00:38:32.745080+0000","last_undegraded":"2024-04-25T00:38:32.745080+0000","last_fullsized":"2024-04-25T00:38:32.745080+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:36.801538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736200+0000","last_change":"2024-04-25T00:38:30.708302+0000","last_active":"2024-04-25T00:38:32.736200+0000","last_peered":"2024-04-25T00:38:32.736200+0000","last_clean":"2024-04-25T00:38:32.736200+0000","last_became_active":"2024-04-25T00:38:30.707995+0000","last_became_peered":"2024-04-25T00:38:30.707995+0000","last_unstale":"2024-04-25T00:38:32.736200+0000","last_undegraded":"2024-04-25T00:38:32.736200+0000","last_fullsized":"2024-04-25T00:38:32.736200+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:23.309761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302265+0000","last_change":"2024-04-25T00:38:31.746913+0000","last_active":"2024-04-25T00:38:33.302265+0000","last_peered":"2024-04-25T00:38:33.302265+0000","last_clean":"2024-04-25T00:38:33.302265+0000","last_became_active":"2024-04-25T00:38:31.746549+0000","last_became_peered":"2024-04-25T00:38:31.746549+0000","last_unstale":"2024-04-25T00:38:33.302265+0000","last_undegraded":"2024-04-25T00:38:33.302265+0000","last_fullsized":"2024-04-25T00:38:33.302265+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:57.137146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736122+0000","last_change":"2024-04-25T00:38:30.702993+0000","last_active":"2024-04-25T00:38:32.736122+0000","last_peered":"2024-04-25T00:38:32.736122+0000","last_clean":"2024-04-25T00:38:32.736122+0000","last_became_active":"2024-04-25T00:38:30.702658+0000","last_became_peered":"2024-04-25T00:38:30.702658+0000","last_unstale":"2024-04-25T00:38:32.736122+0000","last_undegraded":"2024-04-25T00:38:32.736122+0000","last_fullsized":"2024-04-25T00:38:32.736122+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.866914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.738565+0000","last_change":"2024-04-25T00:38:31.760777+0000","last_active":"2024-04-25T00:38:32.738565+0000","last_peered":"2024-04-25T00:38:32.738565+0000","last_clean":"2024-04-25T00:38:32.738565+0000","last_became_active":"2024-04-25T00:38:31.759964+0000","last_became_peered":"2024-04-25T00:38:31.759964+0000","last_unstale":"2024-04-25T00:38:32.738565+0000","last_undegraded":"2024-04-25T00:38:32.738565+0000","last_fullsized":"2024-04-25T00:38:32.738565+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:04.488234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302938+0000","last_change":"2024-04-25T00:38:30.708054+0000","last_active":"2024-04-25T00:38:33.302938+0000","last_peered":"2024-04-25T00:38:33.302938+0000","last_clean":"2024-04-25T00:38:33.302938+0000","last_became_active":"2024-04-25T00:38:30.707665+0000","last_became_peered":"2024-04-25T00:38:30.707665+0000","last_unstale":"2024-04-25T00:38:33.302938+0000","last_undegraded":"2024-04-25T00:38:33.302938+0000","last_fullsized":"2024-04-25T00:38:33.302938+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:03.223501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821538+0000","last_change":"2024-04-25T00:38:31.734225+0000","last_active":"2024-04-25T00:38:32.821538+0000","last_peered":"2024-04-25T00:38:32.821538+0000","last_clean":"2024-04-25T00:38:32.821538+0000","last_became_active":"2024-04-25T00:38:31.733434+0000","last_became_peered":"2024-04-25T00:38:31.733434+0000","last_unstale":"2024-04-25T00:38:32.821538+0000","last_undegraded":"2024-04-25T00:38:32.821538+0000","last_fullsized":"2024-04-25T00:38:32.821538+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:15.662403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.737414+0000","last_change":"2024-04-25T00:38:30.717024+0000","last_active":"2024-04-25T00:38:32.737414+0000","last_peered":"2024-04-25T00:38:32.737414+0000","last_clean":"2024-04-25T00:38:32.737414+0000","last_became_active":"2024-04-25T00:38:30.716214+0000","last_became_peered":"2024-04-25T00:38:30.716214+0000","last_unstale":"2024-04-25T00:38:32.737414+0000","last_undegraded":"2024-04-25T00:38:32.737414+0000","last_fullsized":"2024-04-25T00:38:32.737414+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:21.858516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736707+0000","last_change":"2024-04-25T00:38:31.732735+0000","last_active":"2024-04-25T00:38:32.736707+0000","last_peered":"2024-04-25T00:38:32.736707+0000","last_clean":"2024-04-25T00:38:32.736707+0000","last_became_active":"2024-04-25T00:38:31.731997+0000","last_became_peered":"2024-04-25T00:38:31.731997+0000","last_unstale":"2024-04-25T00:38:32.736707+0000","last_undegraded":"2024-04-25T00:38:32.736707+0000","last_fullsized":"2024-04-25T00:38:32.736707+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:50.134707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.735734+0000","last_change":"2024-04-25T00:38:30.723006+0000","last_active":"2024-04-25T00:38:32.735734+0000","last_peered":"2024-04-25T00:38:32.735734+0000","last_clean":"2024-04-25T00:38:32.735734+0000","last_became_active":"2024-04-25T00:38:30.722115+0000","last_became_peered":"2024-04-25T00:38:30.722115+0000","last_unstale":"2024-04-25T00:38:32.735734+0000","last_undegraded":"2024-04-25T00:38:32.735734+0000","last_fullsized":"2024-04-25T00:38:32.735734+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:51.200991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920706+0000","last_change":"2024-04-25T00:38:31.749943+0000","last_active":"2024-04-25T00:38:33.920706+0000","last_peered":"2024-04-25T00:38:33.920706+0000","last_clean":"2024-04-25T00:38:33.920706+0000","last_became_active":"2024-04-25T00:38:31.749522+0000","last_became_peered":"2024-04-25T00:38:31.749522+0000","last_unstale":"2024-04-25T00:38:33.920706+0000","last_undegraded":"2024-04-25T00:38:33.920706+0000","last_fullsized":"2024-04-25T00:38:33.920706+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:02.596187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739920+0000","last_change":"2024-04-25T00:38:30.701730+0000","last_active":"2024-04-25T00:38:32.739920+0000","last_peered":"2024-04-25T00:38:32.739920+0000","last_clean":"2024-04-25T00:38:32.739920+0000","last_became_active":"2024-04-25T00:38:30.700467+0000","last_became_peered":"2024-04-25T00:38:30.700467+0000","last_unstale":"2024-04-25T00:38:32.739920+0000","last_undegraded":"2024-04-25T00:38:32.739920+0000","last_fullsized":"2024-04-25T00:38:32.739920+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.954040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739769+0000","last_change":"2024-04-25T00:38:31.747961+0000","last_active":"2024-04-25T00:38:32.739769+0000","last_peered":"2024-04-25T00:38:32.739769+0000","last_clean":"2024-04-25T00:38:32.739769+0000","last_became_active":"2024-04-25T00:38:31.745858+0000","last_became_peered":"2024-04-25T00:38:31.745858+0000","last_unstale":"2024-04-25T00:38:32.739769+0000","last_undegraded":"2024-04-25T00:38:32.739769+0000","last_fullsized":"2024-04-25T00:38:32.739769+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:08.861324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.303075+0000","last_change":"2024-04-25T00:38:30.713582+0000","last_active":"2024-04-25T00:38:33.303075+0000","last_peered":"2024-04-25T00:38:33.303075+0000","last_clean":"2024-04-25T00:38:33.303075+0000","last_became_active":"2024-04-25T00:38:30.712854+0000","last_became_peered":"2024-04-25T00:38:30.712854+0000","last_unstale":"2024-04-25T00:38:33.303075+0000","last_undegraded":"2024-04-25T00:38:33.303075+0000","last_fullsized":"2024-04-25T00:38:33.303075+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:36.922270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745486+0000","last_change":"2024-04-25T00:38:31.748426+0000","last_active":"2024-04-25T00:38:32.745486+0000","last_peered":"2024-04-25T00:38:32.745486+0000","last_clean":"2024-04-25T00:38:32.745486+0000","last_became_active":"2024-04-25T00:38:31.747148+0000","last_became_peered":"2024-04-25T00:38:31.747148+0000","last_unstale":"2024-04-25T00:38:32.745486+0000","last_undegraded":"2024-04-25T00:38:32.745486+0000","last_fullsized":"2024-04-25T00:38:32.745486+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:30.222055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"117'3","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739280+0000","last_change":"2024-04-25T00:38:30.692998+0000","last_active":"2024-04-25T00:38:32.739280+0000","last_peered":"2024-04-25T00:38:32.739280+0000","last_clean":"2024-04-25T00:38:32.739280+0000","last_became_active":"2024-04-25T00:38:30.692230+0000","last_became_peered":"2024-04-25T00:38:30.692230+0000","last_unstale":"2024-04-25T00:38:32.739280+0000","last_undegraded":"2024-04-25T00:38:32.739280+0000","last_fullsized":"2024-04-25T00:38:32.739280+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:13.723833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745175+0000","last_change":"2024-04-25T00:38:31.746776+0000","last_active":"2024-04-25T00:38:32.745175+0000","last_peered":"2024-04-25T00:38:32.745175+0000","last_clean":"2024-04-25T00:38:32.745175+0000","last_became_active":"2024-04-25T00:38:31.746081+0000","last_became_peered":"2024-04-25T00:38:31.746081+0000","last_unstale":"2024-04-25T00:38:32.745175+0000","last_undegraded":"2024-04-25T00:38:32.745175+0000","last_fullsized":"2024-04-25T00:38:32.745175+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:48.095477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"117'6","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821801+0000","last_change":"2024-04-25T00:38:30.721436+0000","last_active":"2024-04-25T00:38:32.821801+0000","last_peered":"2024-04-25T00:38:32.821801+0000","last_clean":"2024-04-25T00:38:32.821801+0000","last_became_active":"2024-04-25T00:38:30.720715+0000","last_became_peered":"2024-04-25T00:38:30.720715+0000","last_unstale":"2024-04-25T00:38:32.821801+0000","last_undegraded":"2024-04-25T00:38:32.821801+0000","last_fullsized":"2024-04-25T00:38:32.821801+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:07.850586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920419+0000","last_change":"2024-04-25T00:38:31.747824+0000","last_active":"2024-04-25T00:38:33.920419+0000","last_peered":"2024-04-25T00:38:33.920419+0000","last_clean":"2024-04-25T00:38:33.920419+0000","last_became_active":"2024-04-25T00:38:31.747055+0000","last_became_peered":"2024-04-25T00:38:31.747055+0000","last_unstale":"2024-04-25T00:38:33.920419+0000","last_undegraded":"2024-04-25T00:38:33.920419+0000","last_fullsized":"2024-04-25T00:38:33.920419+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:01.837895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920391+0000","last_change":"2024-04-25T00:38:30.711477+0000","last_active":"2024-04-25T00:38:33.920391+0000","last_peered":"2024-04-25T00:38:33.920391+0000","last_clean":"2024-04-25T00:38:33.920391+0000","last_became_active":"2024-04-25T00:38:30.711080+0000","last_became_peered":"2024-04-25T00:38:30.711080+0000","last_unstale":"2024-04-25T00:38:33.920391+0000","last_undegraded":"2024-04-25T00:38:33.920391+0000","last_fullsized":"2024-04-25T00:38:33.920391+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:37.921322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.735620+0000","last_change":"2024-04-25T00:38:31.730211+0000","last_active":"2024-04-25T00:38:32.735620+0000","last_peered":"2024-04-25T00:38:32.735620+0000","last_clean":"2024-04-25T00:38:32.735620+0000","last_became_active":"2024-04-25T00:38:31.729379+0000","last_became_peered":"2024-04-25T00:38:31.729379+0000","last_unstale":"2024-04-25T00:38:32.735620+0000","last_undegraded":"2024-04-25T00:38:32.735620+0000","last_fullsized":"2024-04-25T00:38:32.735620+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:19.409847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736436+0000","last_change":"2024-04-25T00:38:30.719068+0000","last_active":"2024-04-25T00:38:32.736436+0000","last_peered":"2024-04-25T00:38:32.736436+0000","last_clean":"2024-04-25T00:38:32.736436+0000","last_became_active":"2024-04-25T00:38:30.718528+0000","last_became_peered":"2024-04-25T00:38:30.718528+0000","last_unstale":"2024-04-25T00:38:32.736436+0000","last_undegraded":"2024-04-25T00:38:32.736436+0000","last_fullsized":"2024-04-25T00:38:32.736436+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:34.491970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821488+0000","last_change":"2024-04-25T00:38:31.731954+0000","last_active":"2024-04-25T00:38:32.821488+0000","last_peered":"2024-04-25T00:38:32.821488+0000","last_clean":"2024-04-25T00:38:32.821488+0000","last_became_active":"2024-04-25T00:38:31.730676+0000","last_became_peered":"2024-04-25T00:38:31.730676+0000","last_unstale":"2024-04-25T00:38:32.821488+0000","last_undegraded":"2024-04-25T00:38:32.821488+0000","last_fullsized":"2024-04-25T00:38:32.821488+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:51.334377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"117'8","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821799+0000","last_change":"2024-04-25T00:38:30.713594+0000","last_active":"2024-04-25T00:38:32.821799+0000","last_peered":"2024-04-25T00:38:32.821799+0000","last_clean":"2024-04-25T00:38:32.821799+0000","last_became_active":"2024-04-25T00:38:30.712756+0000","last_became_peered":"2024-04-25T00:38:30.712756+0000","last_unstale":"2024-04-25T00:38:32.821799+0000","last_undegraded":"2024-04-25T00:38:32.821799+0000","last_fullsized":"2024-04-25T00:38:32.821799+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:16.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.822032+0000","last_change":"2024-04-25T00:38:31.765700+0000","last_active":"2024-04-25T00:38:32.822032+0000","last_peered":"2024-04-25T00:38:32.822032+0000","last_clean":"2024-04-25T00:38:32.822032+0000","last_became_active":"2024-04-25T00:38:31.764958+0000","last_became_peered":"2024-04-25T00:38:31.764958+0000","last_unstale":"2024-04-25T00:38:32.822032+0000","last_undegraded":"2024-04-25T00:38:32.822032+0000","last_fullsized":"2024-04-25T00:38:32.822032+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:12.397108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.740083+0000","last_change":"2024-04-25T00:38:30.707248+0000","last_active":"2024-04-25T00:38:32.740083+0000","last_peered":"2024-04-25T00:38:32.740083+0000","last_clean":"2024-04-25T00:38:32.740083+0000","last_became_active":"2024-04-25T00:38:30.706075+0000","last_became_peered":"2024-04-25T00:38:30.706075+0000","last_unstale":"2024-04-25T00:38:32.740083+0000","last_undegraded":"2024-04-25T00:38:32.740083+0000","last_fullsized":"2024-04-25T00:38:32.740083+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:43.215015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739965+0000","last_change":"2024-04-25T00:38:31.742630+0000","last_active":"2024-04-25T00:38:32.739965+0000","last_peered":"2024-04-25T00:38:32.739965+0000","last_clean":"2024-04-25T00:38:32.739965+0000","last_became_active":"2024-04-25T00:38:31.739411+0000","last_became_peered":"2024-04-25T00:38:31.739411+0000","last_unstale":"2024-04-25T00:38:32.739965+0000","last_undegraded":"2024-04-25T00:38:32.739965+0000","last_fullsized":"2024-04-25T00:38:32.739965+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:45.433251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920251+0000","last_change":"2024-04-25T00:38:30.722849+0000","last_active":"2024-04-25T00:38:33.920251+0000","last_peered":"2024-04-25T00:38:33.920251+0000","last_clean":"2024-04-25T00:38:33.920251+0000","last_became_active":"2024-04-25T00:38:30.722273+0000","last_became_peered":"2024-04-25T00:38:30.722273+0000","last_unstale":"2024-04-25T00:38:33.920251+0000","last_undegraded":"2024-04-25T00:38:33.920251+0000","last_fullsized":"2024-04-25T00:38:33.920251+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:00.370437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.303254+0000","last_change":"2024-04-25T00:38:31.755929+0000","last_active":"2024-04-25T00:38:33.303254+0000","last_peered":"2024-04-25T00:38:33.303254+0000","last_clean":"2024-04-25T00:38:33.303254+0000","last_became_active":"2024-04-25T00:38:31.755573+0000","last_became_peered":"2024-04-25T00:38:31.755573+0000","last_unstale":"2024-04-25T00:38:33.303254+0000","last_undegraded":"2024-04-25T00:38:33.303254+0000","last_fullsized":"2024-04-25T00:38:33.303254+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:33.831876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739830+0000","last_change":"2024-04-25T00:38:30.694403+0000","last_active":"2024-04-25T00:38:32.739830+0000","last_peered":"2024-04-25T00:38:32.739830+0000","last_clean":"2024-04-25T00:38:32.739830+0000","last_became_active":"2024-04-25T00:38:30.691600+0000","last_became_peered":"2024-04-25T00:38:30.691600+0000","last_unstale":"2024-04-25T00:38:32.739830+0000","last_undegraded":"2024-04-25T00:38:32.739830+0000","last_fullsized":"2024-04-25T00:38:32.739830+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:43.973357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.737011+0000","last_change":"2024-04-25T00:38:31.741718+0000","last_active":"2024-04-25T00:38:32.737011+0000","last_peered":"2024-04-25T00:38:32.737011+0000","last_clean":"2024-04-25T00:38:32.737011+0000","last_became_active":"2024-04-25T00:38:31.741367+0000","last_became_peered":"2024-04-25T00:38:31.741367+0000","last_unstale":"2024-04-25T00:38:32.737011+0000","last_undegraded":"2024-04-25T00:38:32.737011+0000","last_fullsized":"2024-04-25T00:38:32.737011+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:58.367409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739498+0000","last_change":"2024-04-25T00:38:30.696466+0000","last_active":"2024-04-25T00:38:32.739498+0000","last_peered":"2024-04-25T00:38:32.739498+0000","last_clean":"2024-04-25T00:38:32.739498+0000","last_became_active":"2024-04-25T00:38:30.695668+0000","last_became_peered":"2024-04-25T00:38:30.695668+0000","last_unstale":"2024-04-25T00:38:32.739498+0000","last_undegraded":"2024-04-25T00:38:32.739498+0000","last_fullsized":"2024-04-25T00:38:32.739498+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:21.985091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920310+0000","last_change":"2024-04-25T00:38:31.747544+0000","last_active":"2024-04-25T00:38:33.920310+0000","last_peered":"2024-04-25T00:38:33.920310+0000","last_clean":"2024-04-25T00:38:33.920310+0000","last_became_active":"2024-04-25T00:38:31.747151+0000","last_became_peered":"2024-04-25T00:38:31.747151+0000","last_unstale":"2024-04-25T00:38:33.920310+0000","last_undegraded":"2024-04-25T00:38:33.920310+0000","last_fullsized":"2024-04-25T00:38:33.920310+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:41.608041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.738778+0000","last_change":"2024-04-25T00:38:30.706126+0000","last_active":"2024-04-25T00:38:32.738778+0000","last_peered":"2024-04-25T00:38:32.738778+0000","last_clean":"2024-04-25T00:38:32.738778+0000","last_became_active":"2024-04-25T00:38:30.705366+0000","last_became_peered":"2024-04-25T00:38:30.705366+0000","last_unstale":"2024-04-25T00:38:32.738778+0000","last_undegraded":"2024-04-25T00:38:32.738778+0000","last_fullsized":"2024-04-25T00:38:32.738778+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:24.292037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.738938+0000","last_change":"2024-04-25T00:38:31.770651+0000","last_active":"2024-04-25T00:38:32.738938+0000","last_peered":"2024-04-25T00:38:32.738938+0000","last_clean":"2024-04-25T00:38:32.738938+0000","last_became_active":"2024-04-25T00:38:31.764133+0000","last_became_peered":"2024-04-25T00:38:31.764133+0000","last_unstale":"2024-04-25T00:38:32.738938+0000","last_undegraded":"2024-04-25T00:38:32.738938+0000","last_fullsized":"2024-04-25T00:38:32.738938+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:23.170578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302549+0000","last_change":"2024-04-25T00:38:30.713264+0000","last_active":"2024-04-25T00:38:33.302549+0000","last_peered":"2024-04-25T00:38:33.302549+0000","last_clean":"2024-04-25T00:38:33.302549+0000","last_became_active":"2024-04-25T00:38:30.712613+0000","last_became_peered":"2024-04-25T00:38:30.712613+0000","last_unstale":"2024-04-25T00:38:33.302549+0000","last_undegraded":"2024-04-25T00:38:33.302549+0000","last_fullsized":"2024-04-25T00:38:33.302549+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:41.667999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.735889+0000","last_change":"2024-04-25T00:38:31.735022+0000","last_active":"2024-04-25T00:38:32.735889+0000","last_peered":"2024-04-25T00:38:32.735889+0000","last_clean":"2024-04-25T00:38:32.735889+0000","last_became_active":"2024-04-25T00:38:31.734198+0000","last_became_peered":"2024-04-25T00:38:31.734198+0000","last_unstale":"2024-04-25T00:38:32.735889+0000","last_undegraded":"2024-04-25T00:38:32.735889+0000","last_fullsized":"2024-04-25T00:38:32.735889+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:17.326905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302091+0000","last_change":"2024-04-25T00:38:30.705558+0000","last_active":"2024-04-25T00:38:33.302091+0000","last_peered":"2024-04-25T00:38:33.302091+0000","last_clean":"2024-04-25T00:38:33.302091+0000","last_became_active":"2024-04-25T00:38:30.705144+0000","last_became_peered":"2024-04-25T00:38:30.705144+0000","last_unstale":"2024-04-25T00:38:33.302091+0000","last_undegraded":"2024-04-25T00:38:33.302091+0000","last_fullsized":"2024-04-25T00:38:33.302091+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:03.477497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.744554+0000","last_change":"2024-04-25T00:38:31.756434+0000","last_active":"2024-04-25T00:38:32.744554+0000","last_peered":"2024-04-25T00:38:32.744554+0000","last_clean":"2024-04-25T00:38:32.744554+0000","last_became_active":"2024-04-25T00:38:31.755195+0000","last_became_peered":"2024-04-25T00:38:31.755195+0000","last_unstale":"2024-04-25T00:38:32.744554+0000","last_undegraded":"2024-04-25T00:38:32.744554+0000","last_fullsized":"2024-04-25T00:38:32.744554+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:57.137146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.919830+0000","last_change":"2024-04-25T00:38:30.722283+0000","last_active":"2024-04-25T00:38:33.919830+0000","last_peered":"2024-04-25T00:38:33.919830+0000","last_clean":"2024-04-25T00:38:33.919830+0000","last_became_active":"2024-04-25T00:38:30.721900+0000","last_became_peered":"2024-04-25T00:38:30.721900+0000","last_unstale":"2024-04-25T00:38:33.919830+0000","last_undegraded":"2024-04-25T00:38:33.919830+0000","last_fullsized":"2024-04-25T00:38:33.919830+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:57.081684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.740712+0000","last_change":"2024-04-25T00:38:31.744744+0000","last_active":"2024-04-25T00:38:32.740712+0000","last_peered":"2024-04-25T00:38:32.740712+0000","last_clean":"2024-04-25T00:38:32.740712+0000","last_became_active":"2024-04-25T00:38:31.741750+0000","last_became_peered":"2024-04-25T00:38:31.741750+0000","last_unstale":"2024-04-25T00:38:32.740712+0000","last_undegraded":"2024-04-25T00:38:32.740712+0000","last_fullsized":"2024-04-25T00:38:32.740712+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:37.933007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920531+0000","last_change":"2024-04-25T00:38:30.704667+0000","last_active":"2024-04-25T00:38:33.920531+0000","last_peered":"2024-04-25T00:38:33.920531+0000","last_clean":"2024-04-25T00:38:33.920531+0000","last_became_active":"2024-04-25T00:38:30.704362+0000","last_became_peered":"2024-04-25T00:38:30.704362+0000","last_unstale":"2024-04-25T00:38:33.920531+0000","last_undegraded":"2024-04-25T00:38:33.920531+0000","last_fullsized":"2024-04-25T00:38:33.920531+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:35.824741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920173+0000","last_change":"2024-04-25T00:38:31.762981+0000","last_active":"2024-04-25T00:38:33.920173+0000","last_peered":"2024-04-25T00:38:33.920173+0000","last_clean":"2024-04-25T00:38:33.920173+0000","last_became_active":"2024-04-25T00:38:31.762561+0000","last_became_peered":"2024-04-25T00:38:31.762561+0000","last_unstale":"2024-04-25T00:38:33.920173+0000","last_undegraded":"2024-04-25T00:38:33.920173+0000","last_fullsized":"2024-04-25T00:38:33.920173+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:32.895569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"118'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.747841+0000","last_change":"2024-04-25T00:38:30.712638+0000","last_active":"2024-04-25T00:38:32.747841+0000","last_peered":"2024-04-25T00:38:32.747841+0000","last_clean":"2024-04-25T00:38:32.747841+0000","last_became_active":"2024-04-25T00:38:30.703893+0000","last_became_peered":"2024-04-25T00:38:30.703893+0000","last_unstale":"2024-04-25T00:38:32.747841+0000","last_undegraded":"2024-04-25T00:38:32.747841+0000","last_fullsized":"2024-04-25T00:38:32.747841+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:03.477497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736004+0000","last_change":"2024-04-25T00:38:31.753948+0000","last_active":"2024-04-25T00:38:32.736004+0000","last_peered":"2024-04-25T00:38:32.736004+0000","last_clean":"2024-04-25T00:38:32.736004+0000","last_became_active":"2024-04-25T00:38:31.753490+0000","last_became_peered":"2024-04-25T00:38:31.753490+0000","last_unstale":"2024-04-25T00:38:32.736004+0000","last_undegraded":"2024-04-25T00:38:32.736004+0000","last_fullsized":"2024-04-25T00:38:32.736004+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:08.861324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736908+0000","last_change":"2024-04-25T00:38:30.722229+0000","last_active":"2024-04-25T00:38:32.736908+0000","last_peered":"2024-04-25T00:38:32.736908+0000","last_clean":"2024-04-25T00:38:32.736908+0000","last_became_active":"2024-04-25T00:38:30.721765+0000","last_became_peered":"2024-04-25T00:38:30.721765+0000","last_unstale":"2024-04-25T00:38:32.736908+0000","last_undegraded":"2024-04-25T00:38:32.736908+0000","last_fullsized":"2024-04-25T00:38:32.736908+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:01.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739550+0000","last_change":"2024-04-25T00:38:31.771312+0000","last_active":"2024-04-25T00:38:32.739550+0000","last_peered":"2024-04-25T00:38:32.739550+0000","last_clean":"2024-04-25T00:38:32.739550+0000","last_became_active":"2024-04-25T00:38:31.770721+0000","last_became_peered":"2024-04-25T00:38:31.770721+0000","last_unstale":"2024-04-25T00:38:32.739550+0000","last_undegraded":"2024-04-25T00:38:32.739550+0000","last_fullsized":"2024-04-25T00:38:32.739550+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:04.234238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.919859+0000","last_change":"2024-04-25T00:38:30.710482+0000","last_active":"2024-04-25T00:38:33.919859+0000","last_peered":"2024-04-25T00:38:33.919859+0000","last_clean":"2024-04-25T00:38:33.919859+0000","last_became_active":"2024-04-25T00:38:30.709789+0000","last_became_peered":"2024-04-25T00:38:30.709789+0000","last_unstale":"2024-04-25T00:38:33.919859+0000","last_undegraded":"2024-04-25T00:38:33.919859+0000","last_fullsized":"2024-04-25T00:38:33.919859+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:34.241692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.744682+0000","last_change":"2024-04-25T00:38:31.763165+0000","last_active":"2024-04-25T00:38:32.744682+0000","last_peered":"2024-04-25T00:38:32.744682+0000","last_clean":"2024-04-25T00:38:32.744682+0000","last_became_active":"2024-04-25T00:38:31.762575+0000","last_became_peered":"2024-04-25T00:38:31.762575+0000","last_unstale":"2024-04-25T00:38:32.744682+0000","last_undegraded":"2024-04-25T00:38:32.744682+0000","last_fullsized":"2024-04-25T00:38:32.744682+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:06.369170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736403+0000","last_change":"2024-04-25T00:38:30.718830+0000","last_active":"2024-04-25T00:38:32.736403+0000","last_peered":"2024-04-25T00:38:32.736403+0000","last_clean":"2024-04-25T00:38:32.736403+0000","last_became_active":"2024-04-25T00:38:30.718413+0000","last_became_peered":"2024-04-25T00:38:30.718413+0000","last_unstale":"2024-04-25T00:38:32.736403+0000","last_undegraded":"2024-04-25T00:38:32.736403+0000","last_fullsized":"2024-04-25T00:38:32.736403+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:00.827158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.738844+0000","last_change":"2024-04-25T00:38:31.750524+0000","last_active":"2024-04-25T00:38:32.738844+0000","last_peered":"2024-04-25T00:38:32.738844+0000","last_clean":"2024-04-25T00:38:32.738844+0000","last_became_active":"2024-04-25T00:38:31.749018+0000","last_became_peered":"2024-04-25T00:38:31.749018+0000","last_unstale":"2024-04-25T00:38:32.738844+0000","last_undegraded":"2024-04-25T00:38:32.738844+0000","last_fullsized":"2024-04-25T00:38:32.738844+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:19.409847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.744854+0000","last_change":"2024-04-25T00:38:30.719547+0000","last_active":"2024-04-25T00:38:32.744854+0000","last_peered":"2024-04-25T00:38:32.744854+0000","last_clean":"2024-04-25T00:38:32.744854+0000","last_became_active":"2024-04-25T00:38:30.718733+0000","last_became_peered":"2024-04-25T00:38:30.718733+0000","last_unstale":"2024-04-25T00:38:32.744854+0000","last_undegraded":"2024-04-25T00:38:32.744854+0000","last_fullsized":"2024-04-25T00:38:32.744854+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:03.223501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302317+0000","last_change":"2024-04-25T00:38:31.768366+0000","last_active":"2024-04-25T00:38:33.302317+0000","last_peered":"2024-04-25T00:38:33.302317+0000","last_clean":"2024-04-25T00:38:33.302317+0000","last_became_active":"2024-04-25T00:38:31.768051+0000","last_became_peered":"2024-04-25T00:38:31.768051+0000","last_unstale":"2024-04-25T00:38:33.302317+0000","last_undegraded":"2024-04-25T00:38:33.302317+0000","last_fullsized":"2024-04-25T00:38:33.302317+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:44.603086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302209+0000","last_change":"2024-04-25T00:38:30.724098+0000","last_active":"2024-04-25T00:38:33.302209+0000","last_peered":"2024-04-25T00:38:33.302209+0000","last_clean":"2024-04-25T00:38:33.302209+0000","last_became_active":"2024-04-25T00:38:30.723246+0000","last_became_peered":"2024-04-25T00:38:30.723246+0000","last_unstale":"2024-04-25T00:38:33.302209+0000","last_undegraded":"2024-04-25T00:38:33.302209+0000","last_fullsized":"2024-04-25T00:38:33.302209+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:22.159841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736165+0000","last_change":"2024-04-25T00:38:31.753371+0000","last_active":"2024-04-25T00:38:32.736165+0000","last_peered":"2024-04-25T00:38:32.736165+0000","last_clean":"2024-04-25T00:38:32.736165+0000","last_became_active":"2024-04-25T00:38:31.748008+0000","last_became_peered":"2024-04-25T00:38:31.748008+0000","last_unstale":"2024-04-25T00:38:32.736165+0000","last_undegraded":"2024-04-25T00:38:32.736165+0000","last_fullsized":"2024-04-25T00:38:32.736165+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:45.433251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.919928+0000","last_change":"2024-04-25T00:38:30.707209+0000","last_active":"2024-04-25T00:38:33.919928+0000","last_peered":"2024-04-25T00:38:33.919928+0000","last_clean":"2024-04-25T00:38:33.919928+0000","last_became_active":"2024-04-25T00:38:30.706440+0000","last_became_peered":"2024-04-25T00:38:30.706440+0000","last_unstale":"2024-04-25T00:38:33.919928+0000","last_undegraded":"2024-04-25T00:38:33.919928+0000","last_fullsized":"2024-04-25T00:38:33.919928+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:07.805129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.744574+0000","last_change":"2024-04-25T00:38:31.755163+0000","last_active":"2024-04-25T00:38:32.744574+0000","last_peered":"2024-04-25T00:38:32.744574+0000","last_clean":"2024-04-25T00:38:32.744574+0000","last_became_active":"2024-04-25T00:38:31.754403+0000","last_became_peered":"2024-04-25T00:38:31.754403+0000","last_unstale":"2024-04-25T00:38:32.744574+0000","last_undegraded":"2024-04-25T00:38:32.744574+0000","last_fullsized":"2024-04-25T00:38:32.744574+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:44.603086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745482+0000","last_change":"2024-04-25T00:38:30.716609+0000","last_active":"2024-04-25T00:38:32.745482+0000","last_peered":"2024-04-25T00:38:32.745482+0000","last_clean":"2024-04-25T00:38:32.745482+0000","last_became_active":"2024-04-25T00:38:30.715786+0000","last_became_peered":"2024-04-25T00:38:30.715786+0000","last_unstale":"2024-04-25T00:38:32.745482+0000","last_undegraded":"2024-04-25T00:38:32.745482+0000","last_fullsized":"2024-04-25T00:38:32.745482+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:41.667999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":396,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.744077+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:38:32.744077+0000","last_peered":"2024-04-25T00:38:32.744077+0000","last_clean":"2024-04-25T00:38:32.744077+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:38:32.744077+0000","last_undegraded":"2024-04-25T00:38:32.744077+0000","last_fullsized":"2024-04-25T00:38:32.744077+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574937,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37384,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2093056,"data_stored":1242499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574938,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":1486848,"data_stored":651493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":1482752,"data_stored":649993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":1478656,"data_stored":649951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574938,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":1482752,"data_stored":650041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574937,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":1470464,"data_stored":644197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":1474560,"data_stored":649881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574937,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37352,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334488,"statfs":{"total":96636764160,"available":96598515712,"internally_reserved":0,"allocated":2056192,"data_stored":1233949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:38:35.770 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:38:35.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.926+0000 7f422ab33640 1 Processor -- start 2024-04-25T00:38:35.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f422ab33640 1 -- start start 2024-04-25T00:38:35.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f422ab33640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f422415df70 0x7f42240b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f422ab33640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f42240b4d30 0x7f42240ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f422ab33640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f422406d980 con 0x7f422406d210 2024-04-25T00:38:35.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f422ab33640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f422406dc60 con 0x7f422415df70 2024-04-25T00:38:35.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f422ab33640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f422406df40 con 0x7f42240b4d30 2024-04-25T00:38:35.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f42288cf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f422415df70 0x7f42240b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f4223fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f422406d210 0x7f42240b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40988/0 (socket says 172.21.15.29:40988) 2024-04-25T00:38:35.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f4223fff640 1 -- 172.21.15.29:0/4156012030 learned_addr learned my addr 172.21.15.29:0/4156012030 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:35.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f42288cf640 1 -- 172.21.15.29:0/4156012030 >> v2:172.21.15.157:3300/0 conn(0x7f42240b4d30 msgr2=0x7f42240ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:35.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f42290d0640 1 --2- 172.21.15.29:0/4156012030 >> v2:172.21.15.157:3300/0 conn(0x7f42240b4d30 0x7f42240ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f42288cf640 1 --2- 172.21.15.29:0/4156012030 >> v2:172.21.15.157:3300/0 conn(0x7f42240b4d30 0x7f42240ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f42288cf640 1 -- 172.21.15.29:0/4156012030 >> v1:172.21.15.29:6789/0 conn(0x7f422406d210 legacy=0x7f42240b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f42288cf640 1 -- 172.21.15.29:0/4156012030 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422406e220 con 0x7f422415df70 2024-04-25T00:38:35.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f42237fe640 1 -- 172.21.15.29:0/4156012030 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 746552272 0 0) 0x7f422406d980 con 0x7f422406d210 2024-04-25T00:38:35.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f42290d0640 1 --2- 172.21.15.29:0/4156012030 >> v2:172.21.15.157:3300/0 conn(0x7f42240b4d30 0x7f42240ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:35.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.930+0000 7f42288cf640 1 --2- 172.21.15.29:0/4156012030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422415df70 0x7f42240b3e40 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f4224071fd0 tx=0x7f421401ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856fc4eec41a60b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:35.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42237fe640 1 -- 172.21.15.29:0/4156012030 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42140b1070 con 0x7f422415df70 2024-04-25T00:38:35.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42237fe640 1 -- 172.21.15.29:0/4156012030 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42140a8050 con 0x7f422415df70 2024-04-25T00:38:35.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42237fe640 1 -- 172.21.15.29:0/4156012030 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42140ac860 con 0x7f422415df70 2024-04-25T00:38:35.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 -- 172.21.15.29:0/4156012030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422415df70 msgr2=0x7f42240b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 --2- 172.21.15.29:0/4156012030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422415df70 0x7f42240b3e40 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f4224071fd0 tx=0x7f421401ce20 comp rx=0 tx=0).stop 2024-04-25T00:38:35.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 -- 172.21.15.29:0/4156012030 shutdown_connections 2024-04-25T00:38:35.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 --2- 172.21.15.29:0/4156012030 >> v2:172.21.15.157:3300/0 conn(0x7f42240b4d30 0x7f42240ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 --2- 172.21.15.29:0/4156012030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422415df70 0x7f42240b3e40 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 -- 172.21.15.29:0/4156012030 >> 172.21.15.29:0/4156012030 conn(0x7f42240b1870 msgr2=0x7f42240b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:35.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 -- 172.21.15.29:0/4156012030 shutdown_connections 2024-04-25T00:38:35.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 -- 172.21.15.29:0/4156012030 wait complete. 2024-04-25T00:38:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 Processor -- start 2024-04-25T00:38:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 -- start start 2024-04-25T00:38:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f422406d210 0x7f4224157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42240b4d30 0x7f42241581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422415df70 0x7f42240cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f422406eba0 con 0x7f422415df70 2024-04-25T00:38:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f422406ea20 con 0x7f42240b4d30 2024-04-25T00:38:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f42240ae5f0 con 0x7f422406d210 2024-04-25T00:38:35.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42290d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422415df70 0x7f42240cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42290d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422415df70 0x7f42240cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54096/0 (socket says 172.21.15.29:54096) 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42290d0640 1 -- 172.21.15.29:0/1375472447 learned_addr learned my addr 172.21.15.29:0/1375472447 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f4223fff640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42240b4d30 0x7f42241581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42288cf640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f422406d210 0x7f4224157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42290d0640 1 -- 172.21.15.29:0/1375472447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f422406d210 msgr2=0x7f4224157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42290d0640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f422406d210 0x7f4224157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42290d0640 1 -- 172.21.15.29:0/1375472447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42240b4d30 msgr2=0x7f42241581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42290d0640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42240b4d30 0x7f42241581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42290d0640 1 -- 172.21.15.29:0/1375472447 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42240ce460 con 0x7f422415df70 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42290d0640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422415df70 0x7f42240cd7e0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f421c00cf20 tx=0x7f421c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42217fa640 1 -- 172.21.15.29:0/1375472447 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c01a070 con 0x7f422415df70 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42217fa640 1 -- 172.21.15.29:0/1375472447 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f421c00d840 con 0x7f422415df70 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 -- 172.21.15.29:0/1375472447 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42240ce740 con 0x7f422415df70 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f42217fa640 1 -- 172.21.15.29:0/1375472447 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c00dc30 con 0x7f422415df70 2024-04-25T00:38:35.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.934+0000 7f422ab33640 1 -- 172.21.15.29:0/1375472447 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4224155790 con 0x7f422415df70 2024-04-25T00:38:35.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.938+0000 7f422ab33640 1 -- 172.21.15.29:0/1375472447 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4224069270 con 0x7f422415df70 2024-04-25T00:38:35.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.938+0000 7f42217fa640 1 -- 172.21.15.29:0/1375472447 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f421c01d070 con 0x7f422415df70 2024-04-25T00:38:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.938+0000 7f42217fa640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4210067cb0 0x7f421006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.938+0000 7f42217fa640 1 -- 172.21.15.29:0/1375472447 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f421c087050 con 0x7f422415df70 2024-04-25T00:38:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.938+0000 7f42288cf640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4210067cb0 0x7f421006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:35.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.938+0000 7f42288cf640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4210067cb0 0x7f421006a170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f42140ae000 tx=0x7f4214008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:35.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:35.942+0000 7f42217fa640 1 -- 172.21.15.29:0/1375472447 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f421c024020 con 0x7f422415df70 2024-04-25T00:38:36.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.102+0000 7f422ab33640 1 -- 172.21.15.29:0/1375472447 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42240b3330 con 0x7f4210067cb0 2024-04-25T00:38:36.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.110+0000 7f42217fa640 1 -- 172.21.15.29:0/1375472447 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7f42240b3330 con 0x7f4210067cb0 2024-04-25T00:38:36.114 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:36.116 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:38:36.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.114+0000 7f422ab33640 1 -- 172.21.15.29:0/1375472447 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4210067cb0 msgr2=0x7f421006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.114+0000 7f422ab33640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4210067cb0 0x7f421006a170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f42140ae000 tx=0x7f4214008040 comp rx=0 tx=0).stop 2024-04-25T00:38:36.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.114+0000 7f422ab33640 1 -- 172.21.15.29:0/1375472447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422415df70 msgr2=0x7f42240cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.114+0000 7f422ab33640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422415df70 0x7f42240cd7e0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f421c00cf20 tx=0x7f421c00a7d0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.114+0000 7f422ab33640 1 -- 172.21.15.29:0/1375472447 shutdown_connections 2024-04-25T00:38:36.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.114+0000 7f422ab33640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4210067cb0 0x7f421006a170 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.114+0000 7f422ab33640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422415df70 0x7f42240cd7e0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.114+0000 7f422ab33640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42240b4d30 0x7f42241581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.114+0000 7f422ab33640 1 --2- 172.21.15.29:0/1375472447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f422406d210 0x7f4224157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.114+0000 7f422ab33640 1 -- 172.21.15.29:0/1375472447 >> 172.21.15.29:0/1375472447 conn(0x7f42240b1870 msgr2=0x7f4224071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:36.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.114+0000 7f422ab33640 1 -- 172.21.15.29:0/1375472447 shutdown_connections 2024-04-25T00:38:36.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.114+0000 7f422ab33640 1 -- 172.21.15.29:0/1375472447 wait complete. 2024-04-25T00:38:36.133 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-04-25T00:38:35.369854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":295444,"kb_used_data":12720,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754679276,"statfs":{"total":773094113280,"available":772791578624,"internally_reserved":0,"allocated":13025280,"data_stored":6372004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.697740"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920319+0000","last_change":"2024-04-25T00:38:31.750455+0000","last_active":"2024-04-25T00:38:33.920319+0000","last_peered":"2024-04-25T00:38:33.920319+0000","last_clean":"2024-04-25T00:38:33.920319+0000","last_became_active":"2024-04-25T00:38:31.750123+0000","last_became_peered":"2024-04-25T00:38:31.750123+0000","last_unstale":"2024-04-25T00:38:33.920319+0000","last_undegraded":"2024-04-25T00:38:33.920319+0000","last_fullsized":"2024-04-25T00:38:33.920319+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:38.678137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736223+0000","last_change":"2024-04-25T00:38:31.745202+0000","last_active":"2024-04-25T00:38:32.736223+0000","last_peered":"2024-04-25T00:38:32.736223+0000","last_clean":"2024-04-25T00:38:32.736223+0000","last_became_active":"2024-04-25T00:38:31.744799+0000","last_became_peered":"2024-04-25T00:38:31.744799+0000","last_unstale":"2024-04-25T00:38:32.736223+0000","last_undegraded":"2024-04-25T00:38:32.736223+0000","last_fullsized":"2024-04-25T00:38:32.736223+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:06.451011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.822106+0000","last_change":"2024-04-25T00:38:31.739015+0000","last_active":"2024-04-25T00:38:32.822106+0000","last_peered":"2024-04-25T00:38:32.822106+0000","last_clean":"2024-04-25T00:38:32.822106+0000","last_became_active":"2024-04-25T00:38:31.738155+0000","last_became_peered":"2024-04-25T00:38:31.738155+0000","last_unstale":"2024-04-25T00:38:32.822106+0000","last_undegraded":"2024-04-25T00:38:32.822106+0000","last_fullsized":"2024-04-25T00:38:32.822106+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:06.525021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920648+0000","last_change":"2024-04-25T00:38:31.743794+0000","last_active":"2024-04-25T00:38:33.920648+0000","last_peered":"2024-04-25T00:38:33.920648+0000","last_clean":"2024-04-25T00:38:33.920648+0000","last_became_active":"2024-04-25T00:38:31.743321+0000","last_became_peered":"2024-04-25T00:38:31.743321+0000","last_unstale":"2024-04-25T00:38:33.920648+0000","last_undegraded":"2024-04-25T00:38:33.920648+0000","last_fullsized":"2024-04-25T00:38:33.920648+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:24.320498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745181+0000","last_change":"2024-04-25T00:38:31.750696+0000","last_active":"2024-04-25T00:38:32.745181+0000","last_peered":"2024-04-25T00:38:32.745181+0000","last_clean":"2024-04-25T00:38:32.745181+0000","last_became_active":"2024-04-25T00:38:31.742792+0000","last_became_peered":"2024-04-25T00:38:31.742792+0000","last_unstale":"2024-04-25T00:38:32.745181+0000","last_undegraded":"2024-04-25T00:38:32.745181+0000","last_fullsized":"2024-04-25T00:38:32.745181+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:47.304639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.737950+0000","last_change":"2024-04-25T00:38:31.771960+0000","last_active":"2024-04-25T00:38:32.737950+0000","last_peered":"2024-04-25T00:38:32.737950+0000","last_clean":"2024-04-25T00:38:32.737950+0000","last_became_active":"2024-04-25T00:38:31.771349+0000","last_became_peered":"2024-04-25T00:38:31.771349+0000","last_unstale":"2024-04-25T00:38:32.737950+0000","last_undegraded":"2024-04-25T00:38:32.737950+0000","last_fullsized":"2024-04-25T00:38:32.737950+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:17.578134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.737330+0000","last_change":"2024-04-25T00:38:31.763195+0000","last_active":"2024-04-25T00:38:32.737330+0000","last_peered":"2024-04-25T00:38:32.737330+0000","last_clean":"2024-04-25T00:38:32.737330+0000","last_became_active":"2024-04-25T00:38:31.762439+0000","last_became_peered":"2024-04-25T00:38:31.762439+0000","last_unstale":"2024-04-25T00:38:32.737330+0000","last_undegraded":"2024-04-25T00:38:32.737330+0000","last_fullsized":"2024-04-25T00:38:32.737330+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:59.591934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821526+0000","last_change":"2024-04-25T00:38:31.748621+0000","last_active":"2024-04-25T00:38:32.821526+0000","last_peered":"2024-04-25T00:38:32.821526+0000","last_clean":"2024-04-25T00:38:32.821526+0000","last_became_active":"2024-04-25T00:38:31.747819+0000","last_became_peered":"2024-04-25T00:38:31.747819+0000","last_unstale":"2024-04-25T00:38:32.821526+0000","last_undegraded":"2024-04-25T00:38:32.821526+0000","last_fullsized":"2024-04-25T00:38:32.821526+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:40.471722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.822161+0000","last_change":"2024-04-25T00:38:31.737196+0000","last_active":"2024-04-25T00:38:32.822161+0000","last_peered":"2024-04-25T00:38:32.822161+0000","last_clean":"2024-04-25T00:38:32.822161+0000","last_became_active":"2024-04-25T00:38:31.736469+0000","last_became_peered":"2024-04-25T00:38:31.736469+0000","last_unstale":"2024-04-25T00:38:32.822161+0000","last_undegraded":"2024-04-25T00:38:32.822161+0000","last_fullsized":"2024-04-25T00:38:32.822161+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:27.701139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736635+0000","last_change":"2024-04-25T00:38:31.738296+0000","last_active":"2024-04-25T00:38:32.736635+0000","last_peered":"2024-04-25T00:38:32.736635+0000","last_clean":"2024-04-25T00:38:32.736635+0000","last_became_active":"2024-04-25T00:38:31.737927+0000","last_became_peered":"2024-04-25T00:38:31.737927+0000","last_unstale":"2024-04-25T00:38:32.736635+0000","last_undegraded":"2024-04-25T00:38:32.736635+0000","last_fullsized":"2024-04-25T00:38:32.736635+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:42.975568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920560+0000","last_change":"2024-04-25T00:38:31.742622+0000","last_active":"2024-04-25T00:38:33.920560+0000","last_peered":"2024-04-25T00:38:33.920560+0000","last_clean":"2024-04-25T00:38:33.920560+0000","last_became_active":"2024-04-25T00:38:31.742229+0000","last_became_peered":"2024-04-25T00:38:31.742229+0000","last_unstale":"2024-04-25T00:38:33.920560+0000","last_undegraded":"2024-04-25T00:38:33.920560+0000","last_fullsized":"2024-04-25T00:38:33.920560+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:13.965812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821938+0000","last_change":"2024-04-25T00:38:31.728449+0000","last_active":"2024-04-25T00:38:32.821938+0000","last_peered":"2024-04-25T00:38:32.821938+0000","last_clean":"2024-04-25T00:38:32.821938+0000","last_became_active":"2024-04-25T00:38:31.727687+0000","last_became_peered":"2024-04-25T00:38:31.727687+0000","last_unstale":"2024-04-25T00:38:32.821938+0000","last_undegraded":"2024-04-25T00:38:32.821938+0000","last_fullsized":"2024-04-25T00:38:32.821938+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:22.869254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920648+0000","last_change":"2024-04-25T00:38:31.764830+0000","last_active":"2024-04-25T00:38:33.920648+0000","last_peered":"2024-04-25T00:38:33.920648+0000","last_clean":"2024-04-25T00:38:33.920648+0000","last_became_active":"2024-04-25T00:38:31.764539+0000","last_became_peered":"2024-04-25T00:38:31.764539+0000","last_unstale":"2024-04-25T00:38:33.920648+0000","last_undegraded":"2024-04-25T00:38:33.920648+0000","last_fullsized":"2024-04-25T00:38:33.920648+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:37.877652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736209+0000","last_change":"2024-04-25T00:38:31.753441+0000","last_active":"2024-04-25T00:38:32.736209+0000","last_peered":"2024-04-25T00:38:32.736209+0000","last_clean":"2024-04-25T00:38:32.736209+0000","last_became_active":"2024-04-25T00:38:31.748127+0000","last_became_peered":"2024-04-25T00:38:31.748127+0000","last_unstale":"2024-04-25T00:38:32.736209+0000","last_undegraded":"2024-04-25T00:38:32.736209+0000","last_fullsized":"2024-04-25T00:38:32.736209+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:03.308012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920411+0000","last_change":"2024-04-25T00:38:31.763270+0000","last_active":"2024-04-25T00:38:33.920411+0000","last_peered":"2024-04-25T00:38:33.920411+0000","last_clean":"2024-04-25T00:38:33.920411+0000","last_became_active":"2024-04-25T00:38:31.762916+0000","last_became_peered":"2024-04-25T00:38:31.762916+0000","last_unstale":"2024-04-25T00:38:33.920411+0000","last_undegraded":"2024-04-25T00:38:33.920411+0000","last_fullsized":"2024-04-25T00:38:33.920411+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:53.973770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302573+0000","last_change":"2024-04-25T00:38:31.755327+0000","last_active":"2024-04-25T00:38:33.302573+0000","last_peered":"2024-04-25T00:38:33.302573+0000","last_clean":"2024-04-25T00:38:33.302573+0000","last_became_active":"2024-04-25T00:38:31.755040+0000","last_became_peered":"2024-04-25T00:38:31.755040+0000","last_unstale":"2024-04-25T00:38:33.302573+0000","last_undegraded":"2024-04-25T00:38:33.302573+0000","last_fullsized":"2024-04-25T00:38:33.302573+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:06.369170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.740700+0000","last_change":"2024-04-25T00:38:31.744770+0000","last_active":"2024-04-25T00:38:32.740700+0000","last_peered":"2024-04-25T00:38:32.740700+0000","last_clean":"2024-04-25T00:38:32.740700+0000","last_became_active":"2024-04-25T00:38:31.744033+0000","last_became_peered":"2024-04-25T00:38:31.744033+0000","last_unstale":"2024-04-25T00:38:32.740700+0000","last_undegraded":"2024-04-25T00:38:32.740700+0000","last_fullsized":"2024-04-25T00:38:32.740700+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:42.678736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302627+0000","last_change":"2024-04-25T00:38:31.761019+0000","last_active":"2024-04-25T00:38:33.302627+0000","last_peered":"2024-04-25T00:38:33.302627+0000","last_clean":"2024-04-25T00:38:33.302627+0000","last_became_active":"2024-04-25T00:38:31.760714+0000","last_became_peered":"2024-04-25T00:38:31.760714+0000","last_unstale":"2024-04-25T00:38:33.302627+0000","last_undegraded":"2024-04-25T00:38:33.302627+0000","last_fullsized":"2024-04-25T00:38:33.302627+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:11.450419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920714+0000","last_change":"2024-04-25T00:38:31.745662+0000","last_active":"2024-04-25T00:38:33.920714+0000","last_peered":"2024-04-25T00:38:33.920714+0000","last_clean":"2024-04-25T00:38:33.920714+0000","last_became_active":"2024-04-25T00:38:31.745410+0000","last_became_peered":"2024-04-25T00:38:31.745410+0000","last_unstale":"2024-04-25T00:38:33.920714+0000","last_undegraded":"2024-04-25T00:38:33.920714+0000","last_fullsized":"2024-04-25T00:38:33.920714+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:37.369926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302885+0000","last_change":"2024-04-25T00:38:31.746910+0000","last_active":"2024-04-25T00:38:33.302885+0000","last_peered":"2024-04-25T00:38:33.302885+0000","last_clean":"2024-04-25T00:38:33.302885+0000","last_became_active":"2024-04-25T00:38:31.746485+0000","last_became_peered":"2024-04-25T00:38:31.746485+0000","last_unstale":"2024-04-25T00:38:33.302885+0000","last_undegraded":"2024-04-25T00:38:33.302885+0000","last_fullsized":"2024-04-25T00:38:33.302885+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:48.095477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745766+0000","last_change":"2024-04-25T00:38:31.742114+0000","last_active":"2024-04-25T00:38:32.745766+0000","last_peered":"2024-04-25T00:38:32.745766+0000","last_clean":"2024-04-25T00:38:32.745766+0000","last_became_active":"2024-04-25T00:38:31.741368+0000","last_became_peered":"2024-04-25T00:38:31.741368+0000","last_unstale":"2024-04-25T00:38:32.745766+0000","last_undegraded":"2024-04-25T00:38:32.745766+0000","last_fullsized":"2024-04-25T00:38:32.745766+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:17.790811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.919819+0000","last_change":"2024-04-25T00:38:31.741485+0000","last_active":"2024-04-25T00:38:33.919819+0000","last_peered":"2024-04-25T00:38:33.919819+0000","last_clean":"2024-04-25T00:38:33.919819+0000","last_became_active":"2024-04-25T00:38:31.734862+0000","last_became_peered":"2024-04-25T00:38:31.734862+0000","last_unstale":"2024-04-25T00:38:33.919819+0000","last_undegraded":"2024-04-25T00:38:33.919819+0000","last_fullsized":"2024-04-25T00:38:33.919819+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:12.185180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821549+0000","last_change":"2024-04-25T00:38:31.731366+0000","last_active":"2024-04-25T00:38:32.821549+0000","last_peered":"2024-04-25T00:38:32.821549+0000","last_clean":"2024-04-25T00:38:32.821549+0000","last_became_active":"2024-04-25T00:38:31.729678+0000","last_became_peered":"2024-04-25T00:38:31.729678+0000","last_unstale":"2024-04-25T00:38:32.821549+0000","last_undegraded":"2024-04-25T00:38:32.821549+0000","last_fullsized":"2024-04-25T00:38:32.821549+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:44.984094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.919732+0000","last_change":"2024-04-25T00:38:31.745470+0000","last_active":"2024-04-25T00:38:33.919732+0000","last_peered":"2024-04-25T00:38:33.919732+0000","last_clean":"2024-04-25T00:38:33.919732+0000","last_became_active":"2024-04-25T00:38:31.745075+0000","last_became_peered":"2024-04-25T00:38:31.745075+0000","last_unstale":"2024-04-25T00:38:33.919732+0000","last_undegraded":"2024-04-25T00:38:33.919732+0000","last_fullsized":"2024-04-25T00:38:33.919732+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:08.429264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736663+0000","last_change":"2024-04-25T00:38:31.754062+0000","last_active":"2024-04-25T00:38:32.736663+0000","last_peered":"2024-04-25T00:38:32.736663+0000","last_clean":"2024-04-25T00:38:32.736663+0000","last_became_active":"2024-04-25T00:38:31.748296+0000","last_became_peered":"2024-04-25T00:38:31.748296+0000","last_unstale":"2024-04-25T00:38:32.736663+0000","last_undegraded":"2024-04-25T00:38:32.736663+0000","last_fullsized":"2024-04-25T00:38:32.736663+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:17.616702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302798+0000","last_change":"2024-04-25T00:38:31.755756+0000","last_active":"2024-04-25T00:38:33.302798+0000","last_peered":"2024-04-25T00:38:33.302798+0000","last_clean":"2024-04-25T00:38:33.302798+0000","last_became_active":"2024-04-25T00:38:31.755150+0000","last_became_peered":"2024-04-25T00:38:31.755150+0000","last_unstale":"2024-04-25T00:38:33.302798+0000","last_undegraded":"2024-04-25T00:38:33.302798+0000","last_fullsized":"2024-04-25T00:38:33.302798+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:36.801538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920730+0000","last_change":"2024-04-25T00:38:31.740031+0000","last_active":"2024-04-25T00:38:33.920730+0000","last_peered":"2024-04-25T00:38:33.920730+0000","last_clean":"2024-04-25T00:38:33.920730+0000","last_became_active":"2024-04-25T00:38:31.739657+0000","last_became_peered":"2024-04-25T00:38:31.739657+0000","last_unstale":"2024-04-25T00:38:33.920730+0000","last_undegraded":"2024-04-25T00:38:33.920730+0000","last_fullsized":"2024-04-25T00:38:33.920730+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:52.211728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739297+0000","last_change":"2024-04-25T00:38:31.732022+0000","last_active":"2024-04-25T00:38:32.739297+0000","last_peered":"2024-04-25T00:38:32.739297+0000","last_clean":"2024-04-25T00:38:32.739297+0000","last_became_active":"2024-04-25T00:38:31.731135+0000","last_became_peered":"2024-04-25T00:38:31.731135+0000","last_unstale":"2024-04-25T00:38:32.739297+0000","last_undegraded":"2024-04-25T00:38:32.739297+0000","last_fullsized":"2024-04-25T00:38:32.739297+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:17.326905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920144+0000","last_change":"2024-04-25T00:38:31.763176+0000","last_active":"2024-04-25T00:38:33.920144+0000","last_peered":"2024-04-25T00:38:33.920144+0000","last_clean":"2024-04-25T00:38:33.920144+0000","last_became_active":"2024-04-25T00:38:31.762553+0000","last_became_peered":"2024-04-25T00:38:31.762553+0000","last_unstale":"2024-04-25T00:38:33.920144+0000","last_undegraded":"2024-04-25T00:38:33.920144+0000","last_fullsized":"2024-04-25T00:38:33.920144+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:46.062663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920153+0000","last_change":"2024-04-25T00:38:31.762771+0000","last_active":"2024-04-25T00:38:33.920153+0000","last_peered":"2024-04-25T00:38:33.920153+0000","last_clean":"2024-04-25T00:38:33.920153+0000","last_became_active":"2024-04-25T00:38:31.762438+0000","last_became_peered":"2024-04-25T00:38:31.762438+0000","last_unstale":"2024-04-25T00:38:33.920153+0000","last_undegraded":"2024-04-25T00:38:33.920153+0000","last_fullsized":"2024-04-25T00:38:33.920153+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:56.381161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.744750+0000","last_change":"2024-04-25T00:38:31.755867+0000","last_active":"2024-04-25T00:38:32.744750+0000","last_peered":"2024-04-25T00:38:32.744750+0000","last_clean":"2024-04-25T00:38:32.744750+0000","last_became_active":"2024-04-25T00:38:31.754368+0000","last_became_peered":"2024-04-25T00:38:31.754368+0000","last_unstale":"2024-04-25T00:38:32.744750+0000","last_undegraded":"2024-04-25T00:38:32.744750+0000","last_fullsized":"2024-04-25T00:38:32.744750+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:11.450419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.737416+0000","last_change":"2024-04-25T00:38:31.761098+0000","last_active":"2024-04-25T00:38:32.737416+0000","last_peered":"2024-04-25T00:38:32.737416+0000","last_clean":"2024-04-25T00:38:32.737416+0000","last_became_active":"2024-04-25T00:38:31.760501+0000","last_became_peered":"2024-04-25T00:38:31.760501+0000","last_unstale":"2024-04-25T00:38:32.737416+0000","last_undegraded":"2024-04-25T00:38:32.737416+0000","last_fullsized":"2024-04-25T00:38:32.737416+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:34.431532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.303073+0000","last_change":"2024-04-25T00:38:31.760487+0000","last_active":"2024-04-25T00:38:33.303073+0000","last_peered":"2024-04-25T00:38:33.303073+0000","last_clean":"2024-04-25T00:38:33.303073+0000","last_became_active":"2024-04-25T00:38:31.760132+0000","last_became_peered":"2024-04-25T00:38:31.760132+0000","last_unstale":"2024-04-25T00:38:33.303073+0000","last_undegraded":"2024-04-25T00:38:33.303073+0000","last_fullsized":"2024-04-25T00:38:33.303073+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:47.304639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"117'4","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745583+0000","last_change":"2024-04-25T00:38:30.715226+0000","last_active":"2024-04-25T00:38:32.745583+0000","last_peered":"2024-04-25T00:38:32.745583+0000","last_clean":"2024-04-25T00:38:32.745583+0000","last_became_active":"2024-04-25T00:38:30.714540+0000","last_became_peered":"2024-04-25T00:38:30.714540+0000","last_unstale":"2024-04-25T00:38:32.745583+0000","last_undegraded":"2024-04-25T00:38:32.745583+0000","last_fullsized":"2024-04-25T00:38:32.745583+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:22.159841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.822206+0000","last_change":"2024-04-25T00:38:31.763149+0000","last_active":"2024-04-25T00:38:32.822206+0000","last_peered":"2024-04-25T00:38:32.822206+0000","last_clean":"2024-04-25T00:38:32.822206+0000","last_became_active":"2024-04-25T00:38:31.762541+0000","last_became_peered":"2024-04-25T00:38:31.762541+0000","last_unstale":"2024-04-25T00:38:32.822206+0000","last_undegraded":"2024-04-25T00:38:32.822206+0000","last_fullsized":"2024-04-25T00:38:32.822206+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:12.797380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739772+0000","last_change":"2024-04-25T00:38:30.699015+0000","last_active":"2024-04-25T00:38:32.739772+0000","last_peered":"2024-04-25T00:38:32.739772+0000","last_clean":"2024-04-25T00:38:32.739772+0000","last_became_active":"2024-04-25T00:38:30.698309+0000","last_became_peered":"2024-04-25T00:38:30.698309+0000","last_unstale":"2024-04-25T00:38:32.739772+0000","last_undegraded":"2024-04-25T00:38:32.739772+0000","last_fullsized":"2024-04-25T00:38:32.739772+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:30.060355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745357+0000","last_change":"2024-04-25T00:38:31.747829+0000","last_active":"2024-04-25T00:38:32.745357+0000","last_peered":"2024-04-25T00:38:32.745357+0000","last_clean":"2024-04-25T00:38:32.745357+0000","last_became_active":"2024-04-25T00:38:31.746279+0000","last_became_peered":"2024-04-25T00:38:31.746279+0000","last_unstale":"2024-04-25T00:38:32.745357+0000","last_undegraded":"2024-04-25T00:38:32.745357+0000","last_fullsized":"2024-04-25T00:38:32.745357+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:33.831876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920824+0000","last_change":"2024-04-25T00:38:30.701744+0000","last_active":"2024-04-25T00:38:33.920824+0000","last_peered":"2024-04-25T00:38:33.920824+0000","last_clean":"2024-04-25T00:38:33.920824+0000","last_became_active":"2024-04-25T00:38:30.700428+0000","last_became_peered":"2024-04-25T00:38:30.700428+0000","last_unstale":"2024-04-25T00:38:33.920824+0000","last_undegraded":"2024-04-25T00:38:33.920824+0000","last_fullsized":"2024-04-25T00:38:33.920824+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:12.378372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745080+0000","last_change":"2024-04-25T00:38:31.744755+0000","last_active":"2024-04-25T00:38:32.745080+0000","last_peered":"2024-04-25T00:38:32.745080+0000","last_clean":"2024-04-25T00:38:32.745080+0000","last_became_active":"2024-04-25T00:38:31.743910+0000","last_became_peered":"2024-04-25T00:38:31.743910+0000","last_unstale":"2024-04-25T00:38:32.745080+0000","last_undegraded":"2024-04-25T00:38:32.745080+0000","last_fullsized":"2024-04-25T00:38:32.745080+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:36.801538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736200+0000","last_change":"2024-04-25T00:38:30.708302+0000","last_active":"2024-04-25T00:38:32.736200+0000","last_peered":"2024-04-25T00:38:32.736200+0000","last_clean":"2024-04-25T00:38:32.736200+0000","last_became_active":"2024-04-25T00:38:30.707995+0000","last_became_peered":"2024-04-25T00:38:30.707995+0000","last_unstale":"2024-04-25T00:38:32.736200+0000","last_undegraded":"2024-04-25T00:38:32.736200+0000","last_fullsized":"2024-04-25T00:38:32.736200+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:23.309761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302265+0000","last_change":"2024-04-25T00:38:31.746913+0000","last_active":"2024-04-25T00:38:33.302265+0000","last_peered":"2024-04-25T00:38:33.302265+0000","last_clean":"2024-04-25T00:38:33.302265+0000","last_became_active":"2024-04-25T00:38:31.746549+0000","last_became_peered":"2024-04-25T00:38:31.746549+0000","last_unstale":"2024-04-25T00:38:33.302265+0000","last_undegraded":"2024-04-25T00:38:33.302265+0000","last_fullsized":"2024-04-25T00:38:33.302265+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:57.137146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736122+0000","last_change":"2024-04-25T00:38:30.702993+0000","last_active":"2024-04-25T00:38:32.736122+0000","last_peered":"2024-04-25T00:38:32.736122+0000","last_clean":"2024-04-25T00:38:32.736122+0000","last_became_active":"2024-04-25T00:38:30.702658+0000","last_became_peered":"2024-04-25T00:38:30.702658+0000","last_unstale":"2024-04-25T00:38:32.736122+0000","last_undegraded":"2024-04-25T00:38:32.736122+0000","last_fullsized":"2024-04-25T00:38:32.736122+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.866914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.738565+0000","last_change":"2024-04-25T00:38:31.760777+0000","last_active":"2024-04-25T00:38:32.738565+0000","last_peered":"2024-04-25T00:38:32.738565+0000","last_clean":"2024-04-25T00:38:32.738565+0000","last_became_active":"2024-04-25T00:38:31.759964+0000","last_became_peered":"2024-04-25T00:38:31.759964+0000","last_unstale":"2024-04-25T00:38:32.738565+0000","last_undegraded":"2024-04-25T00:38:32.738565+0000","last_fullsized":"2024-04-25T00:38:32.738565+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:04.488234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302938+0000","last_change":"2024-04-25T00:38:30.708054+0000","last_active":"2024-04-25T00:38:33.302938+0000","last_peered":"2024-04-25T00:38:33.302938+0000","last_clean":"2024-04-25T00:38:33.302938+0000","last_became_active":"2024-04-25T00:38:30.707665+0000","last_became_peered":"2024-04-25T00:38:30.707665+0000","last_unstale":"2024-04-25T00:38:33.302938+0000","last_undegraded":"2024-04-25T00:38:33.302938+0000","last_fullsized":"2024-04-25T00:38:33.302938+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:03.223501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821538+0000","last_change":"2024-04-25T00:38:31.734225+0000","last_active":"2024-04-25T00:38:32.821538+0000","last_peered":"2024-04-25T00:38:32.821538+0000","last_clean":"2024-04-25T00:38:32.821538+0000","last_became_active":"2024-04-25T00:38:31.733434+0000","last_became_peered":"2024-04-25T00:38:31.733434+0000","last_unstale":"2024-04-25T00:38:32.821538+0000","last_undegraded":"2024-04-25T00:38:32.821538+0000","last_fullsized":"2024-04-25T00:38:32.821538+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:15.662403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.737414+0000","last_change":"2024-04-25T00:38:30.717024+0000","last_active":"2024-04-25T00:38:32.737414+0000","last_peered":"2024-04-25T00:38:32.737414+0000","last_clean":"2024-04-25T00:38:32.737414+0000","last_became_active":"2024-04-25T00:38:30.716214+0000","last_became_peered":"2024-04-25T00:38:30.716214+0000","last_unstale":"2024-04-25T00:38:32.737414+0000","last_undegraded":"2024-04-25T00:38:32.737414+0000","last_fullsized":"2024-04-25T00:38:32.737414+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:21.858516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736707+0000","last_change":"2024-04-25T00:38:31.732735+0000","last_active":"2024-04-25T00:38:32.736707+0000","last_peered":"2024-04-25T00:38:32.736707+0000","last_clean":"2024-04-25T00:38:32.736707+0000","last_became_active":"2024-04-25T00:38:31.731997+0000","last_became_peered":"2024-04-25T00:38:31.731997+0000","last_unstale":"2024-04-25T00:38:32.736707+0000","last_undegraded":"2024-04-25T00:38:32.736707+0000","last_fullsized":"2024-04-25T00:38:32.736707+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:50.134707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.735734+0000","last_change":"2024-04-25T00:38:30.723006+0000","last_active":"2024-04-25T00:38:32.735734+0000","last_peered":"2024-04-25T00:38:32.735734+0000","last_clean":"2024-04-25T00:38:32.735734+0000","last_became_active":"2024-04-25T00:38:30.722115+0000","last_became_peered":"2024-04-25T00:38:30.722115+0000","last_unstale":"2024-04-25T00:38:32.735734+0000","last_undegraded":"2024-04-25T00:38:32.735734+0000","last_fullsized":"2024-04-25T00:38:32.735734+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:51.200991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920706+0000","last_change":"2024-04-25T00:38:31.749943+0000","last_active":"2024-04-25T00:38:33.920706+0000","last_peered":"2024-04-25T00:38:33.920706+0000","last_clean":"2024-04-25T00:38:33.920706+0000","last_became_active":"2024-04-25T00:38:31.749522+0000","last_became_peered":"2024-04-25T00:38:31.749522+0000","last_unstale":"2024-04-25T00:38:33.920706+0000","last_undegraded":"2024-04-25T00:38:33.920706+0000","last_fullsized":"2024-04-25T00:38:33.920706+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:02.596187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739920+0000","last_change":"2024-04-25T00:38:30.701730+0000","last_active":"2024-04-25T00:38:32.739920+0000","last_peered":"2024-04-25T00:38:32.739920+0000","last_clean":"2024-04-25T00:38:32.739920+0000","last_became_active":"2024-04-25T00:38:30.700467+0000","last_became_peered":"2024-04-25T00:38:30.700467+0000","last_unstale":"2024-04-25T00:38:32.739920+0000","last_undegraded":"2024-04-25T00:38:32.739920+0000","last_fullsized":"2024-04-25T00:38:32.739920+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:15.954040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739769+0000","last_change":"2024-04-25T00:38:31.747961+0000","last_active":"2024-04-25T00:38:32.739769+0000","last_peered":"2024-04-25T00:38:32.739769+0000","last_clean":"2024-04-25T00:38:32.739769+0000","last_became_active":"2024-04-25T00:38:31.745858+0000","last_became_peered":"2024-04-25T00:38:31.745858+0000","last_unstale":"2024-04-25T00:38:32.739769+0000","last_undegraded":"2024-04-25T00:38:32.739769+0000","last_fullsized":"2024-04-25T00:38:32.739769+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:08.861324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.303075+0000","last_change":"2024-04-25T00:38:30.713582+0000","last_active":"2024-04-25T00:38:33.303075+0000","last_peered":"2024-04-25T00:38:33.303075+0000","last_clean":"2024-04-25T00:38:33.303075+0000","last_became_active":"2024-04-25T00:38:30.712854+0000","last_became_peered":"2024-04-25T00:38:30.712854+0000","last_unstale":"2024-04-25T00:38:33.303075+0000","last_undegraded":"2024-04-25T00:38:33.303075+0000","last_fullsized":"2024-04-25T00:38:33.303075+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:36.922270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745486+0000","last_change":"2024-04-25T00:38:31.748426+0000","last_active":"2024-04-25T00:38:32.745486+0000","last_peered":"2024-04-25T00:38:32.745486+0000","last_clean":"2024-04-25T00:38:32.745486+0000","last_became_active":"2024-04-25T00:38:31.747148+0000","last_became_peered":"2024-04-25T00:38:31.747148+0000","last_unstale":"2024-04-25T00:38:32.745486+0000","last_undegraded":"2024-04-25T00:38:32.745486+0000","last_fullsized":"2024-04-25T00:38:32.745486+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:30.222055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"117'3","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739280+0000","last_change":"2024-04-25T00:38:30.692998+0000","last_active":"2024-04-25T00:38:32.739280+0000","last_peered":"2024-04-25T00:38:32.739280+0000","last_clean":"2024-04-25T00:38:32.739280+0000","last_became_active":"2024-04-25T00:38:30.692230+0000","last_became_peered":"2024-04-25T00:38:30.692230+0000","last_unstale":"2024-04-25T00:38:32.739280+0000","last_undegraded":"2024-04-25T00:38:32.739280+0000","last_fullsized":"2024-04-25T00:38:32.739280+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:13.723833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745175+0000","last_change":"2024-04-25T00:38:31.746776+0000","last_active":"2024-04-25T00:38:32.745175+0000","last_peered":"2024-04-25T00:38:32.745175+0000","last_clean":"2024-04-25T00:38:32.745175+0000","last_became_active":"2024-04-25T00:38:31.746081+0000","last_became_peered":"2024-04-25T00:38:31.746081+0000","last_unstale":"2024-04-25T00:38:32.745175+0000","last_undegraded":"2024-04-25T00:38:32.745175+0000","last_fullsized":"2024-04-25T00:38:32.745175+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:48.095477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"117'6","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821801+0000","last_change":"2024-04-25T00:38:30.721436+0000","last_active":"2024-04-25T00:38:32.821801+0000","last_peered":"2024-04-25T00:38:32.821801+0000","last_clean":"2024-04-25T00:38:32.821801+0000","last_became_active":"2024-04-25T00:38:30.720715+0000","last_became_peered":"2024-04-25T00:38:30.720715+0000","last_unstale":"2024-04-25T00:38:32.821801+0000","last_undegraded":"2024-04-25T00:38:32.821801+0000","last_fullsized":"2024-04-25T00:38:32.821801+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:07.850586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920419+0000","last_change":"2024-04-25T00:38:31.747824+0000","last_active":"2024-04-25T00:38:33.920419+0000","last_peered":"2024-04-25T00:38:33.920419+0000","last_clean":"2024-04-25T00:38:33.920419+0000","last_became_active":"2024-04-25T00:38:31.747055+0000","last_became_peered":"2024-04-25T00:38:31.747055+0000","last_unstale":"2024-04-25T00:38:33.920419+0000","last_undegraded":"2024-04-25T00:38:33.920419+0000","last_fullsized":"2024-04-25T00:38:33.920419+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:01.837895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920391+0000","last_change":"2024-04-25T00:38:30.711477+0000","last_active":"2024-04-25T00:38:33.920391+0000","last_peered":"2024-04-25T00:38:33.920391+0000","last_clean":"2024-04-25T00:38:33.920391+0000","last_became_active":"2024-04-25T00:38:30.711080+0000","last_became_peered":"2024-04-25T00:38:30.711080+0000","last_unstale":"2024-04-25T00:38:33.920391+0000","last_undegraded":"2024-04-25T00:38:33.920391+0000","last_fullsized":"2024-04-25T00:38:33.920391+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:37.921322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.735620+0000","last_change":"2024-04-25T00:38:31.730211+0000","last_active":"2024-04-25T00:38:32.735620+0000","last_peered":"2024-04-25T00:38:32.735620+0000","last_clean":"2024-04-25T00:38:32.735620+0000","last_became_active":"2024-04-25T00:38:31.729379+0000","last_became_peered":"2024-04-25T00:38:31.729379+0000","last_unstale":"2024-04-25T00:38:32.735620+0000","last_undegraded":"2024-04-25T00:38:32.735620+0000","last_fullsized":"2024-04-25T00:38:32.735620+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:19.409847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736436+0000","last_change":"2024-04-25T00:38:30.719068+0000","last_active":"2024-04-25T00:38:32.736436+0000","last_peered":"2024-04-25T00:38:32.736436+0000","last_clean":"2024-04-25T00:38:32.736436+0000","last_became_active":"2024-04-25T00:38:30.718528+0000","last_became_peered":"2024-04-25T00:38:30.718528+0000","last_unstale":"2024-04-25T00:38:32.736436+0000","last_undegraded":"2024-04-25T00:38:32.736436+0000","last_fullsized":"2024-04-25T00:38:32.736436+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:34.491970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821488+0000","last_change":"2024-04-25T00:38:31.731954+0000","last_active":"2024-04-25T00:38:32.821488+0000","last_peered":"2024-04-25T00:38:32.821488+0000","last_clean":"2024-04-25T00:38:32.821488+0000","last_became_active":"2024-04-25T00:38:31.730676+0000","last_became_peered":"2024-04-25T00:38:31.730676+0000","last_unstale":"2024-04-25T00:38:32.821488+0000","last_undegraded":"2024-04-25T00:38:32.821488+0000","last_fullsized":"2024-04-25T00:38:32.821488+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:51.334377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"117'8","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.821799+0000","last_change":"2024-04-25T00:38:30.713594+0000","last_active":"2024-04-25T00:38:32.821799+0000","last_peered":"2024-04-25T00:38:32.821799+0000","last_clean":"2024-04-25T00:38:32.821799+0000","last_became_active":"2024-04-25T00:38:30.712756+0000","last_became_peered":"2024-04-25T00:38:30.712756+0000","last_unstale":"2024-04-25T00:38:32.821799+0000","last_undegraded":"2024-04-25T00:38:32.821799+0000","last_fullsized":"2024-04-25T00:38:32.821799+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:16.316168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.822032+0000","last_change":"2024-04-25T00:38:31.765700+0000","last_active":"2024-04-25T00:38:32.822032+0000","last_peered":"2024-04-25T00:38:32.822032+0000","last_clean":"2024-04-25T00:38:32.822032+0000","last_became_active":"2024-04-25T00:38:31.764958+0000","last_became_peered":"2024-04-25T00:38:31.764958+0000","last_unstale":"2024-04-25T00:38:32.822032+0000","last_undegraded":"2024-04-25T00:38:32.822032+0000","last_fullsized":"2024-04-25T00:38:32.822032+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:12.397108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.740083+0000","last_change":"2024-04-25T00:38:30.707248+0000","last_active":"2024-04-25T00:38:32.740083+0000","last_peered":"2024-04-25T00:38:32.740083+0000","last_clean":"2024-04-25T00:38:32.740083+0000","last_became_active":"2024-04-25T00:38:30.706075+0000","last_became_peered":"2024-04-25T00:38:30.706075+0000","last_unstale":"2024-04-25T00:38:32.740083+0000","last_undegraded":"2024-04-25T00:38:32.740083+0000","last_fullsized":"2024-04-25T00:38:32.740083+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:43.215015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739965+0000","last_change":"2024-04-25T00:38:31.742630+0000","last_active":"2024-04-25T00:38:32.739965+0000","last_peered":"2024-04-25T00:38:32.739965+0000","last_clean":"2024-04-25T00:38:32.739965+0000","last_became_active":"2024-04-25T00:38:31.739411+0000","last_became_peered":"2024-04-25T00:38:31.739411+0000","last_unstale":"2024-04-25T00:38:32.739965+0000","last_undegraded":"2024-04-25T00:38:32.739965+0000","last_fullsized":"2024-04-25T00:38:32.739965+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:45.433251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920251+0000","last_change":"2024-04-25T00:38:30.722849+0000","last_active":"2024-04-25T00:38:33.920251+0000","last_peered":"2024-04-25T00:38:33.920251+0000","last_clean":"2024-04-25T00:38:33.920251+0000","last_became_active":"2024-04-25T00:38:30.722273+0000","last_became_peered":"2024-04-25T00:38:30.722273+0000","last_unstale":"2024-04-25T00:38:33.920251+0000","last_undegraded":"2024-04-25T00:38:33.920251+0000","last_fullsized":"2024-04-25T00:38:33.920251+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:00.370437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.303254+0000","last_change":"2024-04-25T00:38:31.755929+0000","last_active":"2024-04-25T00:38:33.303254+0000","last_peered":"2024-04-25T00:38:33.303254+0000","last_clean":"2024-04-25T00:38:33.303254+0000","last_became_active":"2024-04-25T00:38:31.755573+0000","last_became_peered":"2024-04-25T00:38:31.755573+0000","last_unstale":"2024-04-25T00:38:33.303254+0000","last_undegraded":"2024-04-25T00:38:33.303254+0000","last_fullsized":"2024-04-25T00:38:33.303254+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:33.831876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739830+0000","last_change":"2024-04-25T00:38:30.694403+0000","last_active":"2024-04-25T00:38:32.739830+0000","last_peered":"2024-04-25T00:38:32.739830+0000","last_clean":"2024-04-25T00:38:32.739830+0000","last_became_active":"2024-04-25T00:38:30.691600+0000","last_became_peered":"2024-04-25T00:38:30.691600+0000","last_unstale":"2024-04-25T00:38:32.739830+0000","last_undegraded":"2024-04-25T00:38:32.739830+0000","last_fullsized":"2024-04-25T00:38:32.739830+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:43.973357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.737011+0000","last_change":"2024-04-25T00:38:31.741718+0000","last_active":"2024-04-25T00:38:32.737011+0000","last_peered":"2024-04-25T00:38:32.737011+0000","last_clean":"2024-04-25T00:38:32.737011+0000","last_became_active":"2024-04-25T00:38:31.741367+0000","last_became_peered":"2024-04-25T00:38:31.741367+0000","last_unstale":"2024-04-25T00:38:32.737011+0000","last_undegraded":"2024-04-25T00:38:32.737011+0000","last_fullsized":"2024-04-25T00:38:32.737011+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:58.367409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739498+0000","last_change":"2024-04-25T00:38:30.696466+0000","last_active":"2024-04-25T00:38:32.739498+0000","last_peered":"2024-04-25T00:38:32.739498+0000","last_clean":"2024-04-25T00:38:32.739498+0000","last_became_active":"2024-04-25T00:38:30.695668+0000","last_became_peered":"2024-04-25T00:38:30.695668+0000","last_unstale":"2024-04-25T00:38:32.739498+0000","last_undegraded":"2024-04-25T00:38:32.739498+0000","last_fullsized":"2024-04-25T00:38:32.739498+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:21.985091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920310+0000","last_change":"2024-04-25T00:38:31.747544+0000","last_active":"2024-04-25T00:38:33.920310+0000","last_peered":"2024-04-25T00:38:33.920310+0000","last_clean":"2024-04-25T00:38:33.920310+0000","last_became_active":"2024-04-25T00:38:31.747151+0000","last_became_peered":"2024-04-25T00:38:31.747151+0000","last_unstale":"2024-04-25T00:38:33.920310+0000","last_undegraded":"2024-04-25T00:38:33.920310+0000","last_fullsized":"2024-04-25T00:38:33.920310+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:41.608041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.738778+0000","last_change":"2024-04-25T00:38:30.706126+0000","last_active":"2024-04-25T00:38:32.738778+0000","last_peered":"2024-04-25T00:38:32.738778+0000","last_clean":"2024-04-25T00:38:32.738778+0000","last_became_active":"2024-04-25T00:38:30.705366+0000","last_became_peered":"2024-04-25T00:38:30.705366+0000","last_unstale":"2024-04-25T00:38:32.738778+0000","last_undegraded":"2024-04-25T00:38:32.738778+0000","last_fullsized":"2024-04-25T00:38:32.738778+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:24.292037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.738938+0000","last_change":"2024-04-25T00:38:31.770651+0000","last_active":"2024-04-25T00:38:32.738938+0000","last_peered":"2024-04-25T00:38:32.738938+0000","last_clean":"2024-04-25T00:38:32.738938+0000","last_became_active":"2024-04-25T00:38:31.764133+0000","last_became_peered":"2024-04-25T00:38:31.764133+0000","last_unstale":"2024-04-25T00:38:32.738938+0000","last_undegraded":"2024-04-25T00:38:32.738938+0000","last_fullsized":"2024-04-25T00:38:32.738938+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:23.170578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302549+0000","last_change":"2024-04-25T00:38:30.713264+0000","last_active":"2024-04-25T00:38:33.302549+0000","last_peered":"2024-04-25T00:38:33.302549+0000","last_clean":"2024-04-25T00:38:33.302549+0000","last_became_active":"2024-04-25T00:38:30.712613+0000","last_became_peered":"2024-04-25T00:38:30.712613+0000","last_unstale":"2024-04-25T00:38:33.302549+0000","last_undegraded":"2024-04-25T00:38:33.302549+0000","last_fullsized":"2024-04-25T00:38:33.302549+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:41.667999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.735889+0000","last_change":"2024-04-25T00:38:31.735022+0000","last_active":"2024-04-25T00:38:32.735889+0000","last_peered":"2024-04-25T00:38:32.735889+0000","last_clean":"2024-04-25T00:38:32.735889+0000","last_became_active":"2024-04-25T00:38:31.734198+0000","last_became_peered":"2024-04-25T00:38:31.734198+0000","last_unstale":"2024-04-25T00:38:32.735889+0000","last_undegraded":"2024-04-25T00:38:32.735889+0000","last_fullsized":"2024-04-25T00:38:32.735889+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:17.326905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302091+0000","last_change":"2024-04-25T00:38:30.705558+0000","last_active":"2024-04-25T00:38:33.302091+0000","last_peered":"2024-04-25T00:38:33.302091+0000","last_clean":"2024-04-25T00:38:33.302091+0000","last_became_active":"2024-04-25T00:38:30.705144+0000","last_became_peered":"2024-04-25T00:38:30.705144+0000","last_unstale":"2024-04-25T00:38:33.302091+0000","last_undegraded":"2024-04-25T00:38:33.302091+0000","last_fullsized":"2024-04-25T00:38:33.302091+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:03.477497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.744554+0000","last_change":"2024-04-25T00:38:31.756434+0000","last_active":"2024-04-25T00:38:32.744554+0000","last_peered":"2024-04-25T00:38:32.744554+0000","last_clean":"2024-04-25T00:38:32.744554+0000","last_became_active":"2024-04-25T00:38:31.755195+0000","last_became_peered":"2024-04-25T00:38:31.755195+0000","last_unstale":"2024-04-25T00:38:32.744554+0000","last_undegraded":"2024-04-25T00:38:32.744554+0000","last_fullsized":"2024-04-25T00:38:32.744554+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:57.137146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.919830+0000","last_change":"2024-04-25T00:38:30.722283+0000","last_active":"2024-04-25T00:38:33.919830+0000","last_peered":"2024-04-25T00:38:33.919830+0000","last_clean":"2024-04-25T00:38:33.919830+0000","last_became_active":"2024-04-25T00:38:30.721900+0000","last_became_peered":"2024-04-25T00:38:30.721900+0000","last_unstale":"2024-04-25T00:38:33.919830+0000","last_undegraded":"2024-04-25T00:38:33.919830+0000","last_fullsized":"2024-04-25T00:38:33.919830+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:57.081684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.740712+0000","last_change":"2024-04-25T00:38:31.744744+0000","last_active":"2024-04-25T00:38:32.740712+0000","last_peered":"2024-04-25T00:38:32.740712+0000","last_clean":"2024-04-25T00:38:32.740712+0000","last_became_active":"2024-04-25T00:38:31.741750+0000","last_became_peered":"2024-04-25T00:38:31.741750+0000","last_unstale":"2024-04-25T00:38:32.740712+0000","last_undegraded":"2024-04-25T00:38:32.740712+0000","last_fullsized":"2024-04-25T00:38:32.740712+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:37.933007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920531+0000","last_change":"2024-04-25T00:38:30.704667+0000","last_active":"2024-04-25T00:38:33.920531+0000","last_peered":"2024-04-25T00:38:33.920531+0000","last_clean":"2024-04-25T00:38:33.920531+0000","last_became_active":"2024-04-25T00:38:30.704362+0000","last_became_peered":"2024-04-25T00:38:30.704362+0000","last_unstale":"2024-04-25T00:38:33.920531+0000","last_undegraded":"2024-04-25T00:38:33.920531+0000","last_fullsized":"2024-04-25T00:38:33.920531+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:35.824741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.920173+0000","last_change":"2024-04-25T00:38:31.762981+0000","last_active":"2024-04-25T00:38:33.920173+0000","last_peered":"2024-04-25T00:38:33.920173+0000","last_clean":"2024-04-25T00:38:33.920173+0000","last_became_active":"2024-04-25T00:38:31.762561+0000","last_became_peered":"2024-04-25T00:38:31.762561+0000","last_unstale":"2024-04-25T00:38:33.920173+0000","last_undegraded":"2024-04-25T00:38:33.920173+0000","last_fullsized":"2024-04-25T00:38:33.920173+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:32.895569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"118'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.747841+0000","last_change":"2024-04-25T00:38:30.712638+0000","last_active":"2024-04-25T00:38:32.747841+0000","last_peered":"2024-04-25T00:38:32.747841+0000","last_clean":"2024-04-25T00:38:32.747841+0000","last_became_active":"2024-04-25T00:38:30.703893+0000","last_became_peered":"2024-04-25T00:38:30.703893+0000","last_unstale":"2024-04-25T00:38:32.747841+0000","last_undegraded":"2024-04-25T00:38:32.747841+0000","last_fullsized":"2024-04-25T00:38:32.747841+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:03.477497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736004+0000","last_change":"2024-04-25T00:38:31.753948+0000","last_active":"2024-04-25T00:38:32.736004+0000","last_peered":"2024-04-25T00:38:32.736004+0000","last_clean":"2024-04-25T00:38:32.736004+0000","last_became_active":"2024-04-25T00:38:31.753490+0000","last_became_peered":"2024-04-25T00:38:31.753490+0000","last_unstale":"2024-04-25T00:38:32.736004+0000","last_undegraded":"2024-04-25T00:38:32.736004+0000","last_fullsized":"2024-04-25T00:38:32.736004+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:08.861324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736908+0000","last_change":"2024-04-25T00:38:30.722229+0000","last_active":"2024-04-25T00:38:32.736908+0000","last_peered":"2024-04-25T00:38:32.736908+0000","last_clean":"2024-04-25T00:38:32.736908+0000","last_became_active":"2024-04-25T00:38:30.721765+0000","last_became_peered":"2024-04-25T00:38:30.721765+0000","last_unstale":"2024-04-25T00:38:32.736908+0000","last_undegraded":"2024-04-25T00:38:32.736908+0000","last_fullsized":"2024-04-25T00:38:32.736908+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:01.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.739550+0000","last_change":"2024-04-25T00:38:31.771312+0000","last_active":"2024-04-25T00:38:32.739550+0000","last_peered":"2024-04-25T00:38:32.739550+0000","last_clean":"2024-04-25T00:38:32.739550+0000","last_became_active":"2024-04-25T00:38:31.770721+0000","last_became_peered":"2024-04-25T00:38:31.770721+0000","last_unstale":"2024-04-25T00:38:32.739550+0000","last_undegraded":"2024-04-25T00:38:32.739550+0000","last_fullsized":"2024-04-25T00:38:32.739550+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:04.234238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.919859+0000","last_change":"2024-04-25T00:38:30.710482+0000","last_active":"2024-04-25T00:38:33.919859+0000","last_peered":"2024-04-25T00:38:33.919859+0000","last_clean":"2024-04-25T00:38:33.919859+0000","last_became_active":"2024-04-25T00:38:30.709789+0000","last_became_peered":"2024-04-25T00:38:30.709789+0000","last_unstale":"2024-04-25T00:38:33.919859+0000","last_undegraded":"2024-04-25T00:38:33.919859+0000","last_fullsized":"2024-04-25T00:38:33.919859+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:34.241692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.744682+0000","last_change":"2024-04-25T00:38:31.763165+0000","last_active":"2024-04-25T00:38:32.744682+0000","last_peered":"2024-04-25T00:38:32.744682+0000","last_clean":"2024-04-25T00:38:32.744682+0000","last_became_active":"2024-04-25T00:38:31.762575+0000","last_became_peered":"2024-04-25T00:38:31.762575+0000","last_unstale":"2024-04-25T00:38:32.744682+0000","last_undegraded":"2024-04-25T00:38:32.744682+0000","last_fullsized":"2024-04-25T00:38:32.744682+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:06.369170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736403+0000","last_change":"2024-04-25T00:38:30.718830+0000","last_active":"2024-04-25T00:38:32.736403+0000","last_peered":"2024-04-25T00:38:32.736403+0000","last_clean":"2024-04-25T00:38:32.736403+0000","last_became_active":"2024-04-25T00:38:30.718413+0000","last_became_peered":"2024-04-25T00:38:30.718413+0000","last_unstale":"2024-04-25T00:38:32.736403+0000","last_undegraded":"2024-04-25T00:38:32.736403+0000","last_fullsized":"2024-04-25T00:38:32.736403+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:00.827158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.738844+0000","last_change":"2024-04-25T00:38:31.750524+0000","last_active":"2024-04-25T00:38:32.738844+0000","last_peered":"2024-04-25T00:38:32.738844+0000","last_clean":"2024-04-25T00:38:32.738844+0000","last_became_active":"2024-04-25T00:38:31.749018+0000","last_became_peered":"2024-04-25T00:38:31.749018+0000","last_unstale":"2024-04-25T00:38:32.738844+0000","last_undegraded":"2024-04-25T00:38:32.738844+0000","last_fullsized":"2024-04-25T00:38:32.738844+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:19.409847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.744854+0000","last_change":"2024-04-25T00:38:30.719547+0000","last_active":"2024-04-25T00:38:32.744854+0000","last_peered":"2024-04-25T00:38:32.744854+0000","last_clean":"2024-04-25T00:38:32.744854+0000","last_became_active":"2024-04-25T00:38:30.718733+0000","last_became_peered":"2024-04-25T00:38:30.718733+0000","last_unstale":"2024-04-25T00:38:32.744854+0000","last_undegraded":"2024-04-25T00:38:32.744854+0000","last_fullsized":"2024-04-25T00:38:32.744854+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:03.223501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302317+0000","last_change":"2024-04-25T00:38:31.768366+0000","last_active":"2024-04-25T00:38:33.302317+0000","last_peered":"2024-04-25T00:38:33.302317+0000","last_clean":"2024-04-25T00:38:33.302317+0000","last_became_active":"2024-04-25T00:38:31.768051+0000","last_became_peered":"2024-04-25T00:38:31.768051+0000","last_unstale":"2024-04-25T00:38:33.302317+0000","last_undegraded":"2024-04-25T00:38:33.302317+0000","last_fullsized":"2024-04-25T00:38:33.302317+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:44.603086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.302209+0000","last_change":"2024-04-25T00:38:30.724098+0000","last_active":"2024-04-25T00:38:33.302209+0000","last_peered":"2024-04-25T00:38:33.302209+0000","last_clean":"2024-04-25T00:38:33.302209+0000","last_became_active":"2024-04-25T00:38:30.723246+0000","last_became_peered":"2024-04-25T00:38:30.723246+0000","last_unstale":"2024-04-25T00:38:33.302209+0000","last_undegraded":"2024-04-25T00:38:33.302209+0000","last_fullsized":"2024-04-25T00:38:33.302209+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:22.159841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.736165+0000","last_change":"2024-04-25T00:38:31.753371+0000","last_active":"2024-04-25T00:38:32.736165+0000","last_peered":"2024-04-25T00:38:32.736165+0000","last_clean":"2024-04-25T00:38:32.736165+0000","last_became_active":"2024-04-25T00:38:31.748008+0000","last_became_peered":"2024-04-25T00:38:31.748008+0000","last_unstale":"2024-04-25T00:38:32.736165+0000","last_undegraded":"2024-04-25T00:38:32.736165+0000","last_fullsized":"2024-04-25T00:38:32.736165+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:45.433251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:33.919928+0000","last_change":"2024-04-25T00:38:30.707209+0000","last_active":"2024-04-25T00:38:33.919928+0000","last_peered":"2024-04-25T00:38:33.919928+0000","last_clean":"2024-04-25T00:38:33.919928+0000","last_became_active":"2024-04-25T00:38:30.706440+0000","last_became_peered":"2024-04-25T00:38:30.706440+0000","last_unstale":"2024-04-25T00:38:33.919928+0000","last_undegraded":"2024-04-25T00:38:33.919928+0000","last_fullsized":"2024-04-25T00:38:33.919928+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:07.805129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.744574+0000","last_change":"2024-04-25T00:38:31.755163+0000","last_active":"2024-04-25T00:38:32.744574+0000","last_peered":"2024-04-25T00:38:32.744574+0000","last_clean":"2024-04-25T00:38:32.744574+0000","last_became_active":"2024-04-25T00:38:31.754403+0000","last_became_peered":"2024-04-25T00:38:31.754403+0000","last_unstale":"2024-04-25T00:38:32.744574+0000","last_undegraded":"2024-04-25T00:38:32.744574+0000","last_fullsized":"2024-04-25T00:38:32.744574+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:30.670801+0000","last_clean_scrub_stamp":"2024-04-25T00:38:30.670801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:44.603086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.745482+0000","last_change":"2024-04-25T00:38:30.716609+0000","last_active":"2024-04-25T00:38:32.745482+0000","last_peered":"2024-04-25T00:38:32.745482+0000","last_clean":"2024-04-25T00:38:32.745482+0000","last_became_active":"2024-04-25T00:38:30.715786+0000","last_became_peered":"2024-04-25T00:38:30.715786+0000","last_unstale":"2024-04-25T00:38:32.745482+0000","last_undegraded":"2024-04-25T00:38:32.745482+0000","last_fullsized":"2024-04-25T00:38:32.745482+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:38:29.660064+0000","last_clean_scrub_stamp":"2024-04-25T00:38:29.660064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:41.667999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":396,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T00:38:32.744077+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:38:32.744077+0000","last_peered":"2024-04-25T00:38:32.744077+0000","last_clean":"2024-04-25T00:38:32.744077+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:38:32.744077+0000","last_undegraded":"2024-04-25T00:38:32.744077+0000","last_fullsized":"2024-04-25T00:38:32.744077+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574937,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37384,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2093056,"data_stored":1242499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574938,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":1486848,"data_stored":651493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":1482752,"data_stored":649993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":1478656,"data_stored":649951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574938,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":1482752,"data_stored":650041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574937,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":1470464,"data_stored":644197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":1474560,"data_stored":649881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574937,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37352,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334488,"statfs":{"total":96636764160,"available":96598515712,"internally_reserved":0,"allocated":2056192,"data_stored":1233949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:38:36.137 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:38:36.137 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:38:36.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786c613640 1 Processor -- start 2024-04-25T00:38:36.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786c613640 1 -- start start 2024-04-25T00:38:36.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786c613640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f786415e8d0 0x7f78640cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786c613640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f786415f3c0 0x7f786415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786c613640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78640b3720 con 0x7f78640b4780 2024-04-25T00:38:36.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786c613640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f78640b42f0 con 0x7f786415f3c0 2024-04-25T00:38:36.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786c613640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f78640b39c0 con 0x7f786415e8d0 2024-04-25T00:38:36.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786abb0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f786415f3c0 0x7f786415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786abb0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f786415f3c0 0x7f786415c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54100/0 (socket says 172.21.15.29:54100) 2024-04-25T00:38:36.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786abb0640 1 -- 172.21.15.29:0/3108168438 learned_addr learned my addr 172.21.15.29:0/3108168438 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:36.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786a3af640 1 --2- 172.21.15.29:0/3108168438 >> v2:172.21.15.157:3300/0 conn(0x7f786415e8d0 0x7f78640cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786abb0640 1 -- 172.21.15.29:0/3108168438 >> v2:172.21.15.157:3300/0 conn(0x7f786415e8d0 msgr2=0x7f78640cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:36.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786abb0640 1 --2- 172.21.15.29:0/3108168438 >> v2:172.21.15.157:3300/0 conn(0x7f786415e8d0 0x7f78640cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786abb0640 1 -- 172.21.15.29:0/3108168438 >> v1:172.21.15.157:6789/0 conn(0x7f78640b4780 legacy=0x7f786415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:36.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.310+0000 7f786abb0640 1 -- 172.21.15.29:0/3108168438 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78640b3ca0 con 0x7f786415f3c0 2024-04-25T00:38:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f786abb0640 1 --2- 172.21.15.29:0/3108168438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f786415f3c0 0x7f786415c7d0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f78640b5260 tx=0x7f785c01ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd725474c4471856 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f78693ad640 1 -- 172.21.15.29:0/3108168438 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c0b4070 con 0x7f786415f3c0 2024-04-25T00:38:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f78693ad640 1 -- 172.21.15.29:0/3108168438 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f785c0a7b50 con 0x7f786415f3c0 2024-04-25T00:38:36.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f78693ad640 1 -- 172.21.15.29:0/3108168438 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c0af440 con 0x7f786415f3c0 2024-04-25T00:38:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f786c613640 1 -- 172.21.15.29:0/3108168438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f786415f3c0 msgr2=0x7f786415c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f786c613640 1 --2- 172.21.15.29:0/3108168438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f786415f3c0 0x7f786415c7d0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f78640b5260 tx=0x7f785c01ebb0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f786c613640 1 -- 172.21.15.29:0/3108168438 shutdown_connections 2024-04-25T00:38:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f786c613640 1 --2- 172.21.15.29:0/3108168438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f786415f3c0 0x7f786415c7d0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f786c613640 1 --2- 172.21.15.29:0/3108168438 >> v2:172.21.15.157:3300/0 conn(0x7f786415e8d0 0x7f78640cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f786c613640 1 -- 172.21.15.29:0/3108168438 >> 172.21.15.29:0/3108168438 conn(0x7f78640b1870 msgr2=0x7f78640b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:36.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f786c613640 1 -- 172.21.15.29:0/3108168438 shutdown_connections 2024-04-25T00:38:36.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f786c613640 1 -- 172.21.15.29:0/3108168438 wait complete. 2024-04-25T00:38:36.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f786c613640 1 Processor -- start 2024-04-25T00:38:36.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.314+0000 7f786c613640 1 -- start start 2024-04-25T00:38:36.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786c613640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78640b4780 0x7f7864157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786c613640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f786415e8d0 0x7f786413a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786c613640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f786415f3c0 0x7f78640ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786c613640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f786415d6a0 con 0x7f786415e8d0 2024-04-25T00:38:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786c613640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f786415d520 con 0x7f78640b4780 2024-04-25T00:38:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786c613640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f786406d5d0 con 0x7f786415f3c0 2024-04-25T00:38:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f7869bae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f786415e8d0 0x7f786413a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f7869bae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f786415e8d0 0x7f786413a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54114/0 (socket says 172.21.15.29:54114) 2024-04-25T00:38:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f7869bae640 1 -- 172.21.15.29:0/3645017046 learned_addr learned my addr 172.21.15.29:0/3645017046 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786a3af640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78640b4780 0x7f7864157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786abb0640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f786415f3c0 0x7f78640ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f7869bae640 1 -- 172.21.15.29:0/3645017046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f786415f3c0 msgr2=0x7f78640ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f7869bae640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f786415f3c0 0x7f78640ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f7869bae640 1 -- 172.21.15.29:0/3645017046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78640b4780 msgr2=0x7f7864157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f7869bae640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78640b4780 0x7f7864157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f7869bae640 1 -- 172.21.15.29:0/3645017046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7864158000 con 0x7f786415e8d0 2024-04-25T00:38:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786a3af640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78640b4780 0x7f7864157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f7869bae640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f786415e8d0 0x7f786413a520 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f785400c450 tx=0x7f785400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f785affd640 1 -- 172.21.15.29:0/3645017046 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7854012830 con 0x7f786415e8d0 2024-04-25T00:38:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f785affd640 1 -- 172.21.15.29:0/3645017046 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7854017070 con 0x7f786415e8d0 2024-04-25T00:38:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f785affd640 1 -- 172.21.15.29:0/3645017046 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7854012c30 con 0x7f786415e8d0 2024-04-25T00:38:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786c613640 1 -- 172.21.15.29:0/3645017046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f786413afe0 con 0x7f786415e8d0 2024-04-25T00:38:36.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786c613640 1 -- 172.21.15.29:0/3645017046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f786413b5a0 con 0x7f786415e8d0 2024-04-25T00:38:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786c613640 1 -- 172.21.15.29:0/3645017046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7864069270 con 0x7f786415e8d0 2024-04-25T00:38:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f785affd640 1 -- 172.21.15.29:0/3645017046 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f785401c460 con 0x7f786415e8d0 2024-04-25T00:38:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f785affd640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7848067cb0 0x7f784806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f785affd640 1 -- 172.21.15.29:0/3645017046 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f78540861c0 con 0x7f786415e8d0 2024-04-25T00:38:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786a3af640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7848067cb0 0x7f784806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.318+0000 7f786a3af640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7848067cb0 0x7f784806a170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7860007330 tx=0x7f786000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:36.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.322+0000 7f785affd640 1 -- 172.21.15.29:0/3645017046 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f78540566e0 con 0x7f786415e8d0 2024-04-25T00:38:36.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.542+0000 7f786c613640 1 -- 172.21.15.29:0/3645017046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f78640cc170 con 0x7f786415e8d0 2024-04-25T00:38:36.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.546+0000 7f785affd640 1 -- 172.21.15.29:0/3645017046 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7f7854055fe0 con 0x7f786415e8d0 2024-04-25T00:38:36.550 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-04-25T00:38:36.550 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A== 2024-04-25T00:38:36.550 INFO:teuthology.orchestra.run.smithi029.stdout: caps mds = "allow" 2024-04-25T00:38:36.550 INFO:teuthology.orchestra.run.smithi029.stdout: caps mon = "allow r" 2024-04-25T00:38:36.550 INFO:teuthology.orchestra.run.smithi029.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:38:36.550 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-04-25T00:38:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.550+0000 7f786c613640 1 -- 172.21.15.29:0/3645017046 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7848067cb0 msgr2=0x7f784806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.550+0000 7f786c613640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7848067cb0 0x7f784806a170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7860007330 tx=0x7f786000b040 comp rx=0 tx=0).stop 2024-04-25T00:38:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.550+0000 7f786c613640 1 -- 172.21.15.29:0/3645017046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f786415e8d0 msgr2=0x7f786413a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.550+0000 7f786c613640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f786415e8d0 0x7f786413a520 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f785400c450 tx=0x7f785400c910 comp rx=0 tx=0).stop 2024-04-25T00:38:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.550+0000 7f786c613640 1 -- 172.21.15.29:0/3645017046 shutdown_connections 2024-04-25T00:38:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.550+0000 7f786c613640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7848067cb0 0x7f784806a170 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.550+0000 7f786c613640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f786415f3c0 0x7f78640ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.550+0000 7f786c613640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f786415e8d0 0x7f786413a520 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.550+0000 7f786c613640 1 --2- 172.21.15.29:0/3645017046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78640b4780 0x7f7864157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.550+0000 7f786c613640 1 -- 172.21.15.29:0/3645017046 >> 172.21.15.29:0/3645017046 conn(0x7f78640b1870 msgr2=0x7f78640b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.550+0000 7f786c613640 1 -- 172.21.15.29:0/3645017046 shutdown_connections 2024-04-25T00:38:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.550+0000 7f786c613640 1 -- 172.21.15.29:0/3645017046 wait complete. 2024-04-25T00:38:36.569 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:38:36.665 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-04-25T00:38:36.662+0000 7f64e1602640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:38:36.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b64790640 1 Processor -- start 2024-04-25T00:38:36.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b64790640 1 -- start start 2024-04-25T00:38:36.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b64790640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9b5c0b4780 0x7f9b5c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b64790640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9b5c0cc240 0x7f9b5c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b64790640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b5c0b3720 con 0x7f9b5c15e8d0 2024-04-25T00:38:36.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b64790640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9b5c0cc650 con 0x7f9b5c0b4780 2024-04-25T00:38:36.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b64790640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9b5c0b42f0 con 0x7f9b5c0cc240 2024-04-25T00:38:36.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b6252c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9b5c15e8d0 0x7f9b5c15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40998/0 (socket says 172.21.15.29:40998) 2024-04-25T00:38:36.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b6252c640 1 -- 172.21.15.29:0/1765371350 learned_addr learned my addr 172.21.15.29:0/1765371350 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:36.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b61d2b640 1 --2- 172.21.15.29:0/1765371350 >> v2:172.21.15.29:3300/0 conn(0x7f9b5c0b4780 0x7f9b5c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b62d2d640 1 --2- 172.21.15.29:0/1765371350 >> v2:172.21.15.157:3300/0 conn(0x7f9b5c0cc240 0x7f9b5c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b6152a640 1 -- 172.21.15.29:0/1765371350 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1909200119 0 0) 0x7f9b5c0b3720 con 0x7f9b5c15e8d0 2024-04-25T00:38:36.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b6152a640 1 -- 172.21.15.29:0/1765371350 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b48003530 con 0x7f9b5c15e8d0 2024-04-25T00:38:36.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b61d2b640 1 -- 172.21.15.29:0/1765371350 >> v2:172.21.15.157:3300/0 conn(0x7f9b5c0cc240 msgr2=0x7f9b5c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:36.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b61d2b640 1 --2- 172.21.15.29:0/1765371350 >> v2:172.21.15.157:3300/0 conn(0x7f9b5c0cc240 0x7f9b5c15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b61d2b640 1 -- 172.21.15.29:0/1765371350 >> v1:172.21.15.29:6789/0 conn(0x7f9b5c15e8d0 legacy=0x7f9b5c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b61d2b640 1 -- 172.21.15.29:0/1765371350 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c0b39c0 con 0x7f9b5c0b4780 2024-04-25T00:38:36.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.686+0000 7f9b62d2d640 1 --2- 172.21.15.29:0/1765371350 >> v2:172.21.15.157:3300/0 conn(0x7f9b5c0cc240 0x7f9b5c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:36.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b61d2b640 1 --2- 172.21.15.29:0/1765371350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b5c0b4780 0x7f9b5c15f3c0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c15f9a0 tx=0x7f9b4c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3ac0e925b39c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:36.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b6152a640 1 -- 172.21.15.29:0/1765371350 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0ab650 con 0x7f9b5c0b4780 2024-04-25T00:38:36.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b6152a640 1 -- 172.21.15.29:0/1765371350 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b4c0b0070 con 0x7f9b5c0b4780 2024-04-25T00:38:36.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b6152a640 1 -- 172.21.15.29:0/1765371350 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0aba10 con 0x7f9b5c0b4780 2024-04-25T00:38:36.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 -- 172.21.15.29:0/1765371350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b5c0b4780 msgr2=0x7f9b5c15f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 --2- 172.21.15.29:0/1765371350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b5c0b4780 0x7f9b5c15f3c0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c15f9a0 tx=0x7f9b4c01c450 comp rx=0 tx=0).stop 2024-04-25T00:38:36.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 -- 172.21.15.29:0/1765371350 shutdown_connections 2024-04-25T00:38:36.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 --2- 172.21.15.29:0/1765371350 >> v2:172.21.15.157:3300/0 conn(0x7f9b5c0cc240 0x7f9b5c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 --2- 172.21.15.29:0/1765371350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b5c0b4780 0x7f9b5c15f3c0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 -- 172.21.15.29:0/1765371350 >> 172.21.15.29:0/1765371350 conn(0x7f9b5c0b1870 msgr2=0x7f9b5c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:36.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 -- 172.21.15.29:0/1765371350 shutdown_connections 2024-04-25T00:38:36.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 -- 172.21.15.29:0/1765371350 wait complete. 2024-04-25T00:38:36.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 Processor -- start 2024-04-25T00:38:36.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 -- start start 2024-04-25T00:38:36.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b5c0b4780 0x7f9b5c15b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b5c0cc240 0x7f9b5c13bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b5c15e8d0 0x7f9b5c133e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c144a20 con 0x7f9b5c0cc240 2024-04-25T00:38:36.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c15d520 con 0x7f9b5c15e8d0 2024-04-25T00:38:36.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b64790640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9b5c13c9d0 con 0x7f9b5c0b4780 2024-04-25T00:38:36.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b61d2b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b5c0cc240 0x7f9b5c13bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b61d2b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b5c0cc240 0x7f9b5c13bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54128/0 (socket says 172.21.15.29:54128) 2024-04-25T00:38:36.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b61d2b640 1 -- 172.21.15.29:0/894901589 learned_addr learned my addr 172.21.15.29:0/894901589 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:36.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.690+0000 7f9b6252c640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b5c0b4780 0x7f9b5c15b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b62d2d640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b5c15e8d0 0x7f9b5c133e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b61d2b640 1 -- 172.21.15.29:0/894901589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b5c0b4780 msgr2=0x7f9b5c15b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b61d2b640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b5c0b4780 0x7f9b5c15b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b61d2b640 1 -- 172.21.15.29:0/894901589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b5c15e8d0 msgr2=0x7f9b5c133e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b61d2b640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b5c15e8d0 0x7f9b5c133e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b61d2b640 1 -- 172.21.15.29:0/894901589 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c13ccb0 con 0x7f9b5c0cc240 2024-04-25T00:38:36.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b6252c640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b5c0b4780 0x7f9b5c15b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:36.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b62d2d640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b5c15e8d0 0x7f9b5c133e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:36.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b61d2b640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b5c0cc240 0x7f9b5c13bf10 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c01a080 tx=0x7f9b4c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:36.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b52ffd640 1 -- 172.21.15.29:0/894901589 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0bf610 con 0x7f9b5c0cc240 2024-04-25T00:38:36.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b52ffd640 1 -- 172.21.15.29:0/894901589 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b4c0bf7d0 con 0x7f9b5c0cc240 2024-04-25T00:38:36.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b64790640 1 -- 172.21.15.29:0/894901589 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5c13cf90 con 0x7f9b5c0cc240 2024-04-25T00:38:36.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b52ffd640 1 -- 172.21.15.29:0/894901589 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0bfbc0 con 0x7f9b5c0cc240 2024-04-25T00:38:36.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b64790640 1 -- 172.21.15.29:0/894901589 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5c13d550 con 0x7f9b5c0cc240 2024-04-25T00:38:36.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b64790640 1 -- 172.21.15.29:0/894901589 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b5c069270 con 0x7f9b5c0cc240 2024-04-25T00:38:36.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b52ffd640 1 -- 172.21.15.29:0/894901589 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f9b4c0bd020 con 0x7f9b5c0cc240 2024-04-25T00:38:36.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b52ffd640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9b40067cb0 0x7f9b4006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:36.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b6252c640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9b40067cb0 0x7f9b4006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:36.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b52ffd640 1 -- 172.21.15.29:0/894901589 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f9b4c120f20 con 0x7f9b5c0cc240 2024-04-25T00:38:36.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.694+0000 7f9b6252c640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9b40067cb0 0x7f9b4006a170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005890 tx=0x7f9b58009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:36.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.698+0000 7f9b52ffd640 1 -- 172.21.15.29:0/894901589 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9b4c016070 con 0x7f9b5c0cc240 2024-04-25T00:38:36.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.898+0000 7f9b64790640 1 -- 172.21.15.29:0/894901589 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b5c133be0 con 0x7f9b5c0cc240 2024-04-25T00:38:36.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.898+0000 7f9b52ffd640 1 -- 172.21.15.29:0/894901589 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5186 (secure 0 0 0) 0x7f9b4c0abe20 con 0x7f9b5c0cc240 2024-04-25T00:38:36.902 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:36.902 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":75,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2148970077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2148970077},{"type":"v1","addr":"172.21.15.157:6834","nonce":2148970077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3947933400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3947933400},{"type":"v1","addr":"172.21.15.157:6835","nonce":3947933400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/172449010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":172449010},{"type":"v1","addr":"172.21.15.29:6833","nonce":172449010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:38:31.061384+0000","modified":"2024-04-25T00:38:34.097007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/3206404146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3206404146},{"type":"v1","addr":"172.21.15.29:6835","nonce":3206404146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8520,"qdb_cluster":[8520]},"id":9}]} 2024-04-25T00:38:36.903 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 75 2024-04-25T00:38:36.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.902+0000 7f9b64790640 1 -- 172.21.15.29:0/894901589 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9b40067cb0 msgr2=0x7f9b4006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.902+0000 7f9b64790640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9b40067cb0 0x7f9b4006a170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005890 tx=0x7f9b58009040 comp rx=0 tx=0).stop 2024-04-25T00:38:36.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.902+0000 7f9b64790640 1 -- 172.21.15.29:0/894901589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b5c0cc240 msgr2=0x7f9b5c13bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:36.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.902+0000 7f9b64790640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b5c0cc240 0x7f9b5c13bf10 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c01a080 tx=0x7f9b4c01ac90 comp rx=0 tx=0).stop 2024-04-25T00:38:36.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.902+0000 7f9b64790640 1 -- 172.21.15.29:0/894901589 shutdown_connections 2024-04-25T00:38:36.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.902+0000 7f9b64790640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9b40067cb0 0x7f9b4006a170 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.902+0000 7f9b64790640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b5c15e8d0 0x7f9b5c133e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.902+0000 7f9b64790640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b5c0cc240 0x7f9b5c13bf10 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.902+0000 7f9b64790640 1 --2- 172.21.15.29:0/894901589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b5c0b4780 0x7f9b5c15b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:36.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.902+0000 7f9b64790640 1 -- 172.21.15.29:0/894901589 >> 172.21.15.29:0/894901589 conn(0x7f9b5c0b1870 msgr2=0x7f9b5c0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:36.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.902+0000 7f9b64790640 1 -- 172.21.15.29:0/894901589 shutdown_connections 2024-04-25T00:38:36.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:36.902+0000 7f9b64790640 1 -- 172.21.15.29:0/894901589 wait complete. 2024-04-25T00:38:37.324 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:38:31.061384+0000', 'modified': '2024-04-25T00:38:34.097007+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8520': {'gid': 8520, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/3206404146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 3206404146}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 3206404146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8520, 'qdb_cluster': [8520]} 2024-04-25T00:38:37.324 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:38:37.325 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:38:37.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febbb942640 1 Processor -- start 2024-04-25T00:38:37.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febbb942640 1 -- start start 2024-04-25T00:38:37.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febbb942640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb415e8d0 0x7febb40cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febbb942640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7febb415f3c0 0x7febb415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febbb942640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febb40b3720 con 0x7febb40b4780 2024-04-25T00:38:37.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febbb942640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7febb40b42f0 con 0x7febb415f3c0 2024-04-25T00:38:37.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febbb942640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7febb40b39c0 con 0x7febb415e8d0 2024-04-25T00:38:37.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febb8edd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7febb40b4780 0x7febb415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41008/0 (socket says 172.21.15.29:41008) 2024-04-25T00:38:37.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febb8edd640 1 -- 172.21.15.29:0/3603919923 learned_addr learned my addr 172.21.15.29:0/3603919923 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:37.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febb9edf640 1 --2- 172.21.15.29:0/3603919923 >> v2:172.21.15.157:3300/0 conn(0x7febb415f3c0 0x7febb415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febb96de640 1 --2- 172.21.15.29:0/3603919923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb415e8d0 0x7febb40cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4153058923 0 0) 0x7febb40b3720 con 0x7febb40b4780 2024-04-25T00:38:37.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feba0003530 con 0x7febb40b4780 2024-04-25T00:38:37.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1659116423 0 0) 0x7feba0003530 con 0x7febb40b4780 2024-04-25T00:38:37.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb415e8d0 msgr2=0x7febb40cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 --2- 172.21.15.29:0/3603919923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb415e8d0 0x7febb40cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 >> v2:172.21.15.157:3300/0 conn(0x7febb415f3c0 msgr2=0x7febb415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 --2- 172.21.15.29:0/3603919923 >> v2:172.21.15.157:3300/0 conn(0x7febb415f3c0 0x7febb415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb40b3ca0 con 0x7febb40b4780 2024-04-25T00:38:37.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7feba4001680 con 0x7febb40b4780 2024-04-25T00:38:37.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 >> v1:172.21.15.29:6789/0 conn(0x7febb40b4780 legacy=0x7febb415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 --2- 172.21.15.29:0/3603919923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feba00039c0 0x7feba0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 --2- 172.21.15.29:0/3603919923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb415e8d0 0x7feba0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 --2- 172.21.15.29:0/3603919923 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feba0004d50 0x7feba00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7febb40b42f0 con 0x7feba0004d50 2024-04-25T00:38:37.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7febb40b39c0 con 0x7feba00039c0 2024-04-25T00:38:37.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7feba000b9d0 con 0x7febb415e8d0 2024-04-25T00:38:37.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febb96de640 1 --2- 172.21.15.29:0/3603919923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feba00039c0 0x7feba0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.458+0000 7febb8edd640 1 --2- 172.21.15.29:0/3603919923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb415e8d0 0x7feba0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febb96de640 1 -- 172.21.15.29:0/3603919923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb415e8d0 msgr2=0x7feba0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febb96de640 1 --2- 172.21.15.29:0/3603919923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb415e8d0 0x7feba0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febb96de640 1 -- 172.21.15.29:0/3603919923 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feba0004d50 msgr2=0x7feba00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febb96de640 1 --2- 172.21.15.29:0/3603919923 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feba0004d50 0x7feba00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febb96de640 1 -- 172.21.15.29:0/3603919923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba0003530 con 0x7feba00039c0 2024-04-25T00:38:37.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febb96de640 1 --2- 172.21.15.29:0/3603919923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feba00039c0 0x7feba0003de0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7febb0009ac0 tx=0x7febb001ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c8e6775401c7e6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:37.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb00b2070 con 0x7feba00039c0 2024-04-25T00:38:37.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb00a5640 con 0x7feba00039c0 2024-04-25T00:38:37.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febabfff640 1 -- 172.21.15.29:0/3603919923 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0009690 con 0x7feba00039c0 2024-04-25T00:38:37.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febbb942640 1 -- 172.21.15.29:0/3603919923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feba00039c0 msgr2=0x7feba0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febbb942640 1 --2- 172.21.15.29:0/3603919923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feba00039c0 0x7feba0003de0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7febb0009ac0 tx=0x7febb001ca50 comp rx=0 tx=0).stop 2024-04-25T00:38:37.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febb9edf640 1 -- 172.21.15.29:0/3603919923 reap_dead start 2024-04-25T00:38:37.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febbb942640 1 -- 172.21.15.29:0/3603919923 shutdown_connections 2024-04-25T00:38:37.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febbb942640 1 -- 172.21.15.29:0/3603919923 >> 172.21.15.29:0/3603919923 conn(0x7febb40b1870 msgr2=0x7febb40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:37.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febbb942640 1 -- 172.21.15.29:0/3603919923 shutdown_connections 2024-04-25T00:38:37.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.462+0000 7febbb942640 1 -- 172.21.15.29:0/3603919923 wait complete. 2024-04-25T00:38:37.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febbb942640 1 Processor -- start 2024-04-25T00:38:37.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febbb942640 1 -- start start 2024-04-25T00:38:37.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febbb942640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feba00039c0 0x7febb41374a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febbb942640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febb415a0c0 0x7febb413f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febbb942640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb413f780 0x7febb4139db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febbb942640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7febb4144a20 con 0x7febb415a0c0 2024-04-25T00:38:37.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febbb942640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7febb406cf10 con 0x7feba00039c0 2024-04-25T00:38:37.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febbb942640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7febb413ab50 con 0x7febb413f780 2024-04-25T00:38:37.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febb8edd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febb415a0c0 0x7febb413f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febb96de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feba00039c0 0x7febb41374a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febb8edd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febb415a0c0 0x7febb413f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54156/0 (socket says 172.21.15.29:54156) 2024-04-25T00:38:37.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febb8edd640 1 -- 172.21.15.29:0/1078073212 learned_addr learned my addr 172.21.15.29:0/1078073212 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:37.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febb9edf640 1 --2- 172.21.15.29:0/1078073212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb413f780 0x7febb4139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febb8edd640 1 -- 172.21.15.29:0/1078073212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb413f780 msgr2=0x7febb4139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febb8edd640 1 --2- 172.21.15.29:0/1078073212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb413f780 0x7febb4139db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febb8edd640 1 -- 172.21.15.29:0/1078073212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feba00039c0 msgr2=0x7febb41374a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febb8edd640 1 --2- 172.21.15.29:0/1078073212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feba00039c0 0x7febb41374a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febb8edd640 1 -- 172.21.15.29:0/1078073212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb413ae30 con 0x7febb415a0c0 2024-04-25T00:38:37.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febb8edd640 1 --2- 172.21.15.29:0/1078073212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febb415a0c0 0x7febb413f240 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7feba4007370 tx=0x7feba4007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:37.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7feba9ffb640 1 -- 172.21.15.29:0/1078073212 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba400f960 con 0x7febb415a0c0 2024-04-25T00:38:37.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7feba9ffb640 1 -- 172.21.15.29:0/1078073212 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba4018070 con 0x7febb415a0c0 2024-04-25T00:38:37.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febbb942640 1 -- 172.21.15.29:0/1078073212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb413b110 con 0x7febb415a0c0 2024-04-25T00:38:37.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7feba9ffb640 1 -- 172.21.15.29:0/1078073212 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba400fd60 con 0x7febb415a0c0 2024-04-25T00:38:37.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.466+0000 7febbb942640 1 -- 172.21.15.29:0/1078073212 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb415a540 con 0x7febb415a0c0 2024-04-25T00:38:37.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7feba9ffb640 1 -- 172.21.15.29:0/1078073212 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7feba401c410 con 0x7febb415a0c0 2024-04-25T00:38:37.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7feba9ffb640 1 --2- 172.21.15.29:0/1078073212 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feb98067ce0 0x7feb9806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febb96de640 1 --2- 172.21.15.29:0/1078073212 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feb98067ce0 0x7feb9806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7feba9ffb640 1 -- 172.21.15.29:0/1078073212 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7feba4085e50 con 0x7febb415a0c0 2024-04-25T00:38:37.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febb96de640 1 --2- 172.21.15.29:0/1078073212 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feb98067ce0 0x7feb9806a1a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7febb00af000 tx=0x7febb00046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:37.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febbeabb000 1 Processor -- start 2024-04-25T00:38:37.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febbeabb000 1 -- start start 2024-04-25T00:38:37.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febbeabb000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561693177ef0 0x5616931782d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febbeabb000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 0x56169312e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febbeabb000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56169312ebc0 0x56169312efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febbeabb000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561692f7c5e0 con 0x561693178810 2024-04-25T00:38:37.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febbeabb000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561692f03c50 con 0x561693177ef0 2024-04-25T00:38:37.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febbeabb000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x561692f785a0 con 0x56169312ebc0 2024-04-25T00:38:37.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febb8edd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 0x56169312e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febb8edd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 0x56169312e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54164/0 (socket says 172.21.15.29:54164) 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febb8edd640 1 -- 172.21.15.29:0/468921094 learned_addr learned my addr 172.21.15.29:0/468921094 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febb96de640 1 --2- 172.21.15.29:0/468921094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561693177ef0 0x5616931782d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febb8edd640 1 -- 172.21.15.29:0/468921094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56169312ebc0 msgr2=0x56169312efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febb8edd640 1 --2- 172.21.15.29:0/468921094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56169312ebc0 0x56169312efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febb8edd640 1 -- 172.21.15.29:0/468921094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561693177ef0 msgr2=0x5616931782d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febb8edd640 1 --2- 172.21.15.29:0/468921094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561693177ef0 0x5616931782d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febb8edd640 1 -- 172.21.15.29:0/468921094 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561692f01d10 con 0x561693178810 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7febb8edd640 1 --2- 172.21.15.29:0/468921094 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 0x56169312e680 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7feba4007340 tx=0x7feba4056510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c560cf7d982c8dba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7feb9effd640 1 -- 172.21.15.29:0/468921094 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba405e9f0 con 0x561693178810 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7feb9effd640 1 -- 172.21.15.29:0/468921094 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba4052480 con 0x561693178810 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.470+0000 7feb9effd640 1 -- 172.21.15.29:0/468921094 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4055480 con 0x561693178810 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 -- 172.21.15.29:0/468921094 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 msgr2=0x56169312e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 --2- 172.21.15.29:0/468921094 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 0x56169312e680 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7feba4007340 tx=0x7feba4056510 comp rx=0 tx=0).stop 2024-04-25T00:38:37.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 -- 172.21.15.29:0/468921094 shutdown_connections 2024-04-25T00:38:37.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 --2- 172.21.15.29:0/468921094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56169312ebc0 0x56169312efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 --2- 172.21.15.29:0/468921094 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 0x56169312e680 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 --2- 172.21.15.29:0/468921094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561693177ef0 0x5616931782d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 -- 172.21.15.29:0/468921094 >> 172.21.15.29:0/468921094 conn(0x56169312ab70 msgr2=0x56169312cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:37.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 -- 172.21.15.29:0/468921094 shutdown_connections 2024-04-25T00:38:37.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 -- 172.21.15.29:0/468921094 wait complete. 2024-04-25T00:38:37.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 -- start start 2024-04-25T00:38:37.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 Processor -- start 2024-04-25T00:38:37.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169312ebc0 0x56169315cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561693177ef0 0x56169315d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 0x56169315d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561692f36520 con 0x561693178810 2024-04-25T00:38:37.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56169312a9f0 con 0x56169312ebc0 2024-04-25T00:38:37.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x561693137250 con 0x561693177ef0 2024-04-25T00:38:37.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 10 client.0 _ll_register_callbacks cb 0x5616931195c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:37.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febb9edf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 0x56169315d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febb9edf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 0x56169315d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54180/0 (socket says 172.21.15.29:54180) 2024-04-25T00:38:37.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febb9edf640 1 -- 172.21.15.29:0/1155111813 learned_addr learned my addr 172.21.15.29:0/1155111813 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febb96de640 1 --2- 172.21.15.29:0/1155111813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169312ebc0 0x56169315cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febb8edd640 1 --2- 172.21.15.29:0/1155111813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561693177ef0 0x56169315d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febb9edf640 1 -- 172.21.15.29:0/1155111813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561693177ef0 msgr2=0x56169315d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febb9edf640 1 --2- 172.21.15.29:0/1155111813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561693177ef0 0x56169315d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febb9edf640 1 -- 172.21.15.29:0/1155111813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169312ebc0 msgr2=0x56169315cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febb9edf640 1 --2- 172.21.15.29:0/1155111813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169312ebc0 0x56169315cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febb9edf640 1 -- 172.21.15.29:0/1155111813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561692f147b0 con 0x561693178810 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febb9edf640 1 --2- 172.21.15.29:0/1155111813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 0x56169315d9a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7febb415f7a0 tx=0x7febac011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba00371260412ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7feb93fff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7feb93fff640 1 -- 172.21.15.29:0/1155111813 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac01a020 con 0x561693178810 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7feb93fff640 1 -- 172.21.15.29:0/1155111813 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7febac00f970 con 0x561693178810 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 -- 172.21.15.29:0/1155111813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feba4052480 con 0x561693178810 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7feb93fff640 1 -- 172.21.15.29:0/1155111813 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7febac024050 con 0x561693178810 2024-04-25T00:38:37.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 10 client.8679 fetch_fsmap learned FSMap version 75 2024-04-25T00:38:37.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 -- 172.21.15.29:0/1155111813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x561692f03ea0 con 0x561693178810 2024-04-25T00:38:37.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7feb93fff640 1 -- 172.21.15.29:0/1155111813 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba4052480 con 0x561693178810 2024-04-25T00:38:37.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7feb93fff640 1 -- 172.21.15.29:0/1155111813 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7febac027020 con 0x561693178810 2024-04-25T00:38:37.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 10 client.8679 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T00:38:37.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 10 client.8679 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:38:37.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 20 client.8679 populate_metadata read hostname 'smithi029' 2024-04-25T00:38:37.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 --2- 172.21.15.29:0/1155111813 >> [v2:172.21.15.29:6834/3206404146,v1:172.21.15.29:6835/3206404146] conn(0x561693187f90 0x56169318a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 4 client.8679 mds_command: new command op to 8520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:38:37.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febbeabb000 1 -- 172.21.15.29:0/1155111813 --> [v2:172.21.15.29:6834/3206404146,v1:172.21.15.29:6835/3206404146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561692f17ab0 con 0x561693187f90 2024-04-25T00:38:37.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febb96de640 1 --2- 172.21.15.29:0/1155111813 >> [v2:172.21.15.29:6834/3206404146,v1:172.21.15.29:6835/3206404146] conn(0x561693187f90 0x56169318a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7feb93fff640 10 client.8679 ms_handle_connect on v2:172.21.15.29:6834/3206404146 2024-04-25T00:38:37.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.474+0000 7febb96de640 1 --2- 172.21.15.29:0/1155111813 >> [v2:172.21.15.29:6834/3206404146,v1:172.21.15.29:6835/3206404146] conn(0x561693187f90 0x56169318a380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f38548cd18cbbe2f server_cookie=abf62ddcaea3d4d1 in_seq=0 out_seq=0 2024-04-25T00:38:37.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7feb93fff640 1 -- 172.21.15.29:0/1155111813 <== mds.0 v2:172.21.15.29:6834/3206404146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561692f17ab0 con 0x561693187f90 2024-04-25T00:38:37.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7feb93fff640 10 client.8679 handle_command_reply: tid=0 2024-04-25T00:38:37.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7febbeabb000 1 client.8679 shutdown 2024-04-25T00:38:37.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7febbeabb000 1 -- 172.21.15.29:0/1155111813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 msgr2=0x56169315d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7febbeabb000 1 --2- 172.21.15.29:0/1155111813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 0x56169315d9a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7febb415f7a0 tx=0x7febac011d20 comp rx=0 tx=0).stop 2024-04-25T00:38:37.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7febbeabb000 1 -- 172.21.15.29:0/1155111813 shutdown_connections 2024-04-25T00:38:37.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7febbeabb000 1 --2- 172.21.15.29:0/1155111813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561693178810 0x56169315d9a0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7febbeabb000 1 --2- 172.21.15.29:0/1155111813 >> [v2:172.21.15.29:6834/3206404146,v1:172.21.15.29:6835/3206404146] conn(0x561693187f90 0x56169318a380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7febbeabb000 1 --2- 172.21.15.29:0/1155111813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x561693177ef0 0x56169315d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7febbeabb000 1 --2- 172.21.15.29:0/1155111813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169312ebc0 0x56169315cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7feb93fff640 0 client.8679 ms_handle_reset on v2:172.21.15.29:6834/3206404146 2024-04-25T00:38:37.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7febbeabb000 1 -- 172.21.15.29:0/1155111813 >> 172.21.15.29:0/1155111813 conn(0x56169312ab70 msgr2=0x56169312d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:37.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7febbeabb000 1 -- 172.21.15.29:0/1155111813 shutdown_connections 2024-04-25T00:38:37.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7febbeabb000 1 -- 172.21.15.29:0/1155111813 wait complete. 2024-04-25T00:38:37.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.478+0000 7febbeabb000 20 client.8679 trim_cache size 0 max 16384 2024-04-25T00:38:37.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.494+0000 7febbeabb000 1 Processor -- start 2024-04-25T00:38:37.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.494+0000 7febbeabb000 1 -- start start 2024-04-25T00:38:37.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.494+0000 7febbeabb000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169315b140 0x56169318d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.494+0000 7febbeabb000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56169312ab70 0x561693188370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.494+0000 7febbeabb000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169312ebc0 0x5616931888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.494+0000 7febbeabb000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7febb41fea60 con 0x56169315b140 2024-04-25T00:38:37.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.494+0000 7febbeabb000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7febb41fdb70 con 0x56169312ebc0 2024-04-25T00:38:37.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.494+0000 7febbeabb000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7febb413c4b0 con 0x56169312ab70 2024-04-25T00:38:37.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.494+0000 7febb96de640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169315b140 0x56169318d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.494+0000 7febb96de640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169315b140 0x56169318d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54190/0 (socket says 172.21.15.29:54190) 2024-04-25T00:38:37.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.494+0000 7febb96de640 1 -- 172.21.15.29:0/213721962 learned_addr learned my addr 172.21.15.29:0/213721962 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:37.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb96de640 1 -- 172.21.15.29:0/213721962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56169312ab70 msgr2=0x561693188370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb96de640 1 --2- 172.21.15.29:0/213721962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56169312ab70 0x561693188370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb96de640 1 -- 172.21.15.29:0/213721962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169312ebc0 msgr2=0x5616931888b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb96de640 1 --2- 172.21.15.29:0/213721962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169312ebc0 0x5616931888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb96de640 1 -- 172.21.15.29:0/213721962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561693131cc0 con 0x56169315b140 2024-04-25T00:38:37.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb96de640 1 --2- 172.21.15.29:0/213721962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169315b140 0x56169318d510 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7febb0009ac0 tx=0x7febb0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306557dbd941ae89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:37.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7feb9e7fc640 1 -- 172.21.15.29:0/213721962 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb00ce070 con 0x56169315b140 2024-04-25T00:38:37.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7feb9e7fc640 1 -- 172.21.15.29:0/213721962 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb00a5b90 con 0x56169315b140 2024-04-25T00:38:37.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7feb9e7fc640 1 -- 172.21.15.29:0/213721962 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb00b2040 con 0x56169315b140 2024-04-25T00:38:37.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 -- 172.21.15.29:0/213721962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169315b140 msgr2=0x56169318d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 --2- 172.21.15.29:0/213721962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169315b140 0x56169318d510 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7febb0009ac0 tx=0x7febb0004740 comp rx=0 tx=0).stop 2024-04-25T00:38:37.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 -- 172.21.15.29:0/213721962 shutdown_connections 2024-04-25T00:38:37.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 --2- 172.21.15.29:0/213721962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169312ebc0 0x5616931888b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 --2- 172.21.15.29:0/213721962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56169312ab70 0x561693188370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 --2- 172.21.15.29:0/213721962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169315b140 0x56169318d510 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 -- 172.21.15.29:0/213721962 >> 172.21.15.29:0/213721962 conn(0x5616931855a0 msgr2=0x561693177700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:37.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 -- 172.21.15.29:0/213721962 shutdown_connections 2024-04-25T00:38:37.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 -- 172.21.15.29:0/213721962 wait complete. 2024-04-25T00:38:37.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 -- start start 2024-04-25T00:38:37.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 Processor -- start 2024-04-25T00:38:37.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56169312ab70 0x5616931e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169312ebc0 0x56169318bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169315b140 0x56169318c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56169315bb20 con 0x56169312ebc0 2024-04-25T00:38:37.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7febb413cb00 con 0x56169315b140 2024-04-25T00:38:37.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x56169302ee80 con 0x56169312ab70 2024-04-25T00:38:37.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 10 client.0 _ll_register_callbacks cb 0x7febac011520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:37.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb8edd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169312ebc0 0x56169318bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb8edd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169312ebc0 0x56169318bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54198/0 (socket says 172.21.15.29:54198) 2024-04-25T00:38:37.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb8edd640 1 -- 172.21.15.29:0/813230883 learned_addr learned my addr 172.21.15.29:0/813230883 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:37.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb8edd640 1 -- 172.21.15.29:0/813230883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56169312ab70 msgr2=0x5616931e7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb9edf640 1 --2- 172.21.15.29:0/813230883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169315b140 0x56169318c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb8edd640 1 --2- 172.21.15.29:0/813230883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56169312ab70 0x5616931e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb8edd640 1 -- 172.21.15.29:0/813230883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169315b140 msgr2=0x56169318c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb8edd640 1 --2- 172.21.15.29:0/813230883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169315b140 0x56169318c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb8edd640 1 -- 172.21.15.29:0/813230883 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561692f04e80 con 0x56169312ebc0 2024-04-25T00:38:37.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febb8edd640 1 --2- 172.21.15.29:0/813230883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169312ebc0 0x56169318bb50 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7feba4007340 tx=0x7feba4001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1433fda074cd7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:37.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7feb93fff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:38:37.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7feb93fff640 1 -- 172.21.15.29:0/813230883 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba405e9f0 con 0x56169312ebc0 2024-04-25T00:38:37.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7feb93fff640 1 -- 172.21.15.29:0/813230883 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7feba4055470 con 0x56169312ebc0 2024-04-25T00:38:37.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 -- 172.21.15.29:0/813230883 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb00a5b90 con 0x56169312ebc0 2024-04-25T00:38:37.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7feb93fff640 1 -- 172.21.15.29:0/813230883 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7feba407a050 con 0x56169312ebc0 2024-04-25T00:38:37.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 10 client.8685 fetch_fsmap learned FSMap version 75 2024-04-25T00:38:37.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7febbeabb000 1 -- 172.21.15.29:0/813230883 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x561692f49760 con 0x56169312ebc0 2024-04-25T00:38:37.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.498+0000 7feb93fff640 1 -- 172.21.15.29:0/813230883 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb00a5b90 con 0x56169312ebc0 2024-04-25T00:38:37.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7feb93fff640 1 -- 172.21.15.29:0/813230883 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7feba407d020 con 0x56169312ebc0 2024-04-25T00:38:37.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 10 client.8685 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T00:38:37.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 10 client.8685 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:38:37.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 20 client.8685 populate_metadata read hostname 'smithi029' 2024-04-25T00:38:37.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 1 --2- 172.21.15.29:0/813230883 >> [v2:172.21.15.29:6834/3206404146,v1:172.21.15.29:6835/3206404146] conn(0x561693177ef0 0x56169318c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 4 client.8685 mds_command: new command op to 8520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:38:37.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 1 -- 172.21.15.29:0/813230883 --> [v2:172.21.15.29:6834/3206404146,v1:172.21.15.29:6835/3206404146] -- command(tid 0: {"prefix": "status"}) -- 0x561692f89b20 con 0x561693177ef0 2024-04-25T00:38:37.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febb96de640 1 --2- 172.21.15.29:0/813230883 >> [v2:172.21.15.29:6834/3206404146,v1:172.21.15.29:6835/3206404146] conn(0x561693177ef0 0x56169318c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7feb93fff640 10 client.8685 ms_handle_connect on v2:172.21.15.29:6834/3206404146 2024-04-25T00:38:37.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febb96de640 1 --2- 172.21.15.29:0/813230883 >> [v2:172.21.15.29:6834/3206404146,v1:172.21.15.29:6835/3206404146] conn(0x561693177ef0 0x56169318c5d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=707c6a75ae72a217 server_cookie=a7ea9ddc989c30ab in_seq=0 out_seq=0 2024-04-25T00:38:37.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7feb93fff640 1 -- 172.21.15.29:0/813230883 <== mds.0 v2:172.21.15.29:6834/3206404146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561692f89b20 con 0x561693177ef0 2024-04-25T00:38:37.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7feb93fff640 10 client.8685 handle_command_reply: tid=0 2024-04-25T00:38:37.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 1 client.8685 shutdown 2024-04-25T00:38:37.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 1 -- 172.21.15.29:0/813230883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169312ebc0 msgr2=0x56169318bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 1 --2- 172.21.15.29:0/813230883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169312ebc0 0x56169318bb50 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7feba4007340 tx=0x7feba4001a50 comp rx=0 tx=0).stop 2024-04-25T00:38:37.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 1 -- 172.21.15.29:0/813230883 shutdown_connections 2024-04-25T00:38:37.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 1 --2- 172.21.15.29:0/813230883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56169315b140 0x56169318c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 1 --2- 172.21.15.29:0/813230883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169312ebc0 0x56169318bb50 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 1 --2- 172.21.15.29:0/813230883 >> [v2:172.21.15.29:6834/3206404146,v1:172.21.15.29:6835/3206404146] conn(0x561693177ef0 0x56169318c5d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 1 --2- 172.21.15.29:0/813230883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56169312ab70 0x5616931e7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7feb93fff640 0 client.8685 ms_handle_reset on v2:172.21.15.29:6834/3206404146 2024-04-25T00:38:37.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 1 -- 172.21.15.29:0/813230883 >> 172.21.15.29:0/813230883 conn(0x5616931855a0 msgr2=0x56169312cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:37.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 1 -- 172.21.15.29:0/813230883 shutdown_connections 2024-04-25T00:38:37.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 1 -- 172.21.15.29:0/813230883 wait complete. 2024-04-25T00:38:37.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbeabb000 20 client.8685 trim_cache size 0 max 16384 2024-04-25T00:38:37.511 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:38:37.511 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "59afbe7d-aaa8-45e2-b468-69ee150cacea", 2024-04-25T00:38:37.512 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-04-25T00:38:37.512 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 8520, 2024-04-25T00:38:37.512 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-04-25T00:38:37.512 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-04-25T00:38:37.512 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-04-25T00:38:37.512 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 5.7716680389999997, 2024-04-25T00:38:37.512 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 75, 2024-04-25T00:38:37.512 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 118, 2024-04-25T00:38:37.512 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 118, 2024-04-25T00:38:37.512 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 11.829520016 2024-04-25T00:38:37.512 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:38:37.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbb942640 1 -- 172.21.15.29:0/1078073212 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feb98067ce0 msgr2=0x7feb9806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbb942640 1 --2- 172.21.15.29:0/1078073212 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feb98067ce0 0x7feb9806a1a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7febb00af000 tx=0x7febb00046d0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbb942640 1 -- 172.21.15.29:0/1078073212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febb415a0c0 msgr2=0x7febb413f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbb942640 1 --2- 172.21.15.29:0/1078073212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febb415a0c0 0x7febb413f240 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7feba4007370 tx=0x7feba4007770 comp rx=0 tx=0).stop 2024-04-25T00:38:37.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbb942640 1 -- 172.21.15.29:0/1078073212 shutdown_connections 2024-04-25T00:38:37.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbb942640 1 --2- 172.21.15.29:0/1078073212 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feb98067ce0 0x7feb9806a1a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbb942640 1 --2- 172.21.15.29:0/1078073212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febb413f780 0x7febb4139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbb942640 1 --2- 172.21.15.29:0/1078073212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febb415a0c0 0x7febb413f240 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbb942640 1 --2- 172.21.15.29:0/1078073212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feba00039c0 0x7febb41374a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbb942640 1 -- 172.21.15.29:0/1078073212 >> 172.21.15.29:0/1078073212 conn(0x7febb40b1870 msgr2=0x7febb415ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:37.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbb942640 1 -- 172.21.15.29:0/1078073212 shutdown_connections 2024-04-25T00:38:37.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.502+0000 7febbb942640 1 -- 172.21.15.29:0/1078073212 wait complete. 2024-04-25T00:38:37.522 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:38:37.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f321ab04640 1 Processor -- start 2024-04-25T00:38:37.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f321ab04640 1 -- start start 2024-04-25T00:38:37.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f321ab04640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32141584c0 0x7f32140715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f321ab04640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321415e930 con 0x7f32140b4770 2024-04-25T00:38:37.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f321ab04640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32140b4c40 con 0x7f32141588a0 2024-04-25T00:38:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f321ab04640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f321406d330 con 0x7f32141584c0 2024-04-25T00:38:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32190a1640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f32140b4770 0x7f32140aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41012/0 (socket says 172.21.15.29:41012) 2024-04-25T00:38:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32190a1640 1 -- 172.21.15.29:0/1023364333 learned_addr learned my addr 172.21.15.29:0/1023364333 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32188a0640 1 --2- 172.21.15.29:0/1023364333 >> v2:172.21.15.157:3300/0 conn(0x7f32141584c0 0x7f32140715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 43920276 0 0) 0x7f321415e930 con 0x7f32140b4770 2024-04-25T00:38:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31fc003530 con 0x7f32140b4770 2024-04-25T00:38:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 799323127 0 0) 0x7f31fc003530 con 0x7f32140b4770 2024-04-25T00:38:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 >> v2:172.21.15.157:3300/0 conn(0x7f32141584c0 msgr2=0x7f32140715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 --2- 172.21.15.29:0/1023364333 >> v2:172.21.15.157:3300/0 conn(0x7f32141584c0 0x7f32140715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 >> v1:172.21.15.157:6789/0 conn(0x7f32141588a0 legacy=0x7f32140b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32140b4f20 con 0x7f32140b4770 2024-04-25T00:38:37.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f3204001a00 con 0x7f32140b4770 2024-04-25T00:38:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 >> v1:172.21.15.29:6789/0 conn(0x7f32140b4770 legacy=0x7f32140aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 --2- 172.21.15.29:0/1023364333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31fc0039c0 0x7f31fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 --2- 172.21.15.29:0/1023364333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc004810 0x7f31fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 --2- 172.21.15.29:0/1023364333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0051d0 0x7f31fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc00c0f0 con 0x7f31fc004810 2024-04-25T00:38:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc00c2a0 con 0x7f31fc0051d0 2024-04-25T00:38:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f31fc00c4a0 con 0x7f31fc0039c0 2024-04-25T00:38:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f3213fff640 1 --2- 172.21.15.29:0/1023364333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc004810 0x7f31fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32190a1640 1 --2- 172.21.15.29:0/1023364333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0051d0 0x7f31fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32188a0640 1 --2- 172.21.15.29:0/1023364333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31fc0039c0 0x7f31fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f3213fff640 1 -- 172.21.15.29:0/1023364333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31fc0039c0 msgr2=0x7f31fc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f3213fff640 1 --2- 172.21.15.29:0/1023364333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31fc0039c0 0x7f31fc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f3213fff640 1 -- 172.21.15.29:0/1023364333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0051d0 msgr2=0x7f31fc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:37.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f3213fff640 1 --2- 172.21.15.29:0/1023364333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0051d0 0x7f31fc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f32190a1640 1 -- 172.21.15.29:0/1023364333 reap_dead start 2024-04-25T00:38:37.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.634+0000 7f3213fff640 1 -- 172.21.15.29:0/1023364333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc003530 con 0x7f31fc004810 2024-04-25T00:38:37.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.638+0000 7f3213fff640 1 --2- 172.21.15.29:0/1023364333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc004810 0x7f31fc004c90 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f3208001750 tx=0x7f320801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee82d6766594688b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:37.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.638+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080013a0 con 0x7f31fc004810 2024-04-25T00:38:37.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.638+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32080052f0 con 0x7f31fc004810 2024-04-25T00:38:37.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.638+0000 7f32137fe640 1 -- 172.21.15.29:0/1023364333 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208003840 con 0x7f31fc004810 2024-04-25T00:38:37.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.638+0000 7f321ab04640 1 -- 172.21.15.29:0/1023364333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc004810 msgr2=0x7f31fc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.638+0000 7f321ab04640 1 --2- 172.21.15.29:0/1023364333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc004810 0x7f31fc004c90 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f3208001750 tx=0x7f320801c3c0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.638+0000 7f321ab04640 1 -- 172.21.15.29:0/1023364333 shutdown_connections 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.638+0000 7f321ab04640 1 --2- 172.21.15.29:0/1023364333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31fc004810 0x7f31fc004c90 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.638+0000 7f321ab04640 1 -- 172.21.15.29:0/1023364333 >> 172.21.15.29:0/1023364333 conn(0x7f3214075030 msgr2=0x7f32140b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.638+0000 7f321ab04640 1 -- 172.21.15.29:0/1023364333 shutdown_connections 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.638+0000 7f321ab04640 1 -- 172.21.15.29:0/1023364333 wait complete. 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.638+0000 7f321ab04640 1 Processor -- start 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.638+0000 7f321ab04640 1 -- start start 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f321ab04640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc004810 0x7f321406e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f321ab04640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f321406d330 0x7f32141f3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f321ab04640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32140cf790 0x7f32140ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f321ab04640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f32140ae950 con 0x7f321406d330 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f321ab04640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f32140b5200 con 0x7f31fc004810 2024-04-25T00:38:37.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f321ab04640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f32140b4060 con 0x7f32140cf790 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f3213fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f321406d330 0x7f32141f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f3213fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f321406d330 0x7f32141f3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54408/0 (socket says 172.21.15.29:54408) 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f3213fff640 1 -- 172.21.15.29:0/4253941274 learned_addr learned my addr 172.21.15.29:0/4253941274 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f32188a0640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc004810 0x7f321406e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f32190a1640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32140cf790 0x7f32140ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f3213fff640 1 -- 172.21.15.29:0/4253941274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32140cf790 msgr2=0x7f32140ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f3213fff640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32140cf790 0x7f32140ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f3213fff640 1 -- 172.21.15.29:0/4253941274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc004810 msgr2=0x7f321406e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f3213fff640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc004810 0x7f321406e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f3213fff640 1 -- 172.21.15.29:0/4253941274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321414fe00 con 0x7f321406d330 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f32190a1640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32140cf790 0x7f32140ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f3213fff640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f321406d330 0x7f32141f3860 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f320801c1f0 tx=0x7f31fc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f32117fa640 1 -- 172.21.15.29:0/4253941274 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080013a0 con 0x7f321406d330 2024-04-25T00:38:37.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f32117fa640 1 -- 172.21.15.29:0/4253941274 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32080ba070 con 0x7f321406d330 2024-04-25T00:38:37.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f321ab04640 1 -- 172.21.15.29:0/4253941274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321406d830 con 0x7f321406d330 2024-04-25T00:38:37.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f32117fa640 1 -- 172.21.15.29:0/4253941274 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208007a40 con 0x7f321406d330 2024-04-25T00:38:37.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f321ab04640 1 -- 172.21.15.29:0/4253941274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321406ddf0 con 0x7f321406d330 2024-04-25T00:38:37.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f321ab04640 1 -- 172.21.15.29:0/4253941274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3214069270 con 0x7f321406d330 2024-04-25T00:38:37.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f32117fa640 1 -- 172.21.15.29:0/4253941274 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f32080052f0 con 0x7f321406d330 2024-04-25T00:38:37.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f32117fa640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3200067ce0 0x7f320006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:37.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f32117fa640 1 -- 172.21.15.29:0/4253941274 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f3208120a90 con 0x7f321406d330 2024-04-25T00:38:37.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.642+0000 7f32188a0640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3200067ce0 0x7f320006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:37.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.646+0000 7f32188a0640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3200067ce0 0x7f320006a1a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f320c00ba30 tx=0x7f320c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:37.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.650+0000 7f32117fa640 1 -- 172.21.15.29:0/4253941274 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f32080160d0 con 0x7f321406d330 2024-04-25T00:38:37.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f321ab04640 1 -- 172.21.15.29:0/4253941274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f32141569a0 con 0x7f321406d330 2024-04-25T00:38:37.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f32117fa640 1 -- 172.21.15.29:0/4253941274 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) ==== 53+0+73 (secure 0 0 0) 0x7f32080f0d60 con 0x7f321406d330 2024-04-25T00:38:37.853 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:38:37.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f321ab04640 1 -- 172.21.15.29:0/4253941274 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3200067ce0 msgr2=0x7f320006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f321ab04640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3200067ce0 0x7f320006a1a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f320c00ba30 tx=0x7f320c008040 comp rx=0 tx=0).stop 2024-04-25T00:38:37.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f321ab04640 1 -- 172.21.15.29:0/4253941274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f321406d330 msgr2=0x7f32141f3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:37.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f321ab04640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f321406d330 0x7f32141f3860 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f320801c1f0 tx=0x7f31fc005840 comp rx=0 tx=0).stop 2024-04-25T00:38:37.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f321ab04640 1 -- 172.21.15.29:0/4253941274 shutdown_connections 2024-04-25T00:38:37.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f321ab04640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3200067ce0 0x7f320006a1a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f321ab04640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32140cf790 0x7f32140ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f321ab04640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f321406d330 0x7f32141f3860 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f321ab04640 1 --2- 172.21.15.29:0/4253941274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc004810 0x7f321406e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:37.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f321ab04640 1 -- 172.21.15.29:0/4253941274 >> 172.21.15.29:0/4253941274 conn(0x7f3214075030 msgr2=0x7f3214157440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:37.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f321ab04640 1 -- 172.21.15.29:0/4253941274 shutdown_connections 2024-04-25T00:38:37.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:37.850+0000 7f321ab04640 1 -- 172.21.15.29:0/4253941274 wait complete. 2024-04-25T00:38:37.870 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:38:37.870 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:38:37.870 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-25T00:38:37.870 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:38:37.871 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:38:37.871 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:38:37.871 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:38:37.871 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:38:37.871 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:38:37.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:37.871 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-25T00:38:37.877 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:38:37.877 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:38:37.878 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:38:37.878 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:37.878 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:38:37.878 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:37.878 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:38:37.878 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:38:37.878 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-25T00:38:37.878 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:38:37.878 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:38:37.878 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-25T00:38:37.878 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42042sec preferred_lft 42042sec 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 02:42:28:a5:36:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:38:37.879 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:38:37.880 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:38:37.880 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:37.880 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:38:37.880 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:37.880 INFO:teuthology.orchestra.run.smithi029.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:38:37.880 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:38:37.880 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:38:37.880 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:37.880 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:38:37.880 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:38:37.880 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:38:37.880 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:37.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:37.881 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:38:37.929 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-25T00:38:37.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:37.931 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-25T00:38:37.981 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-25T00:38:37.981 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:38:37.981 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T00:38:37.981 INFO:teuthology.orchestra.run.smithi029.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:38:37.981 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T00:38:37.981 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:38:37.982 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:38:37.982 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:38:37.982 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:38:37.982 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:38:37.982 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:38:37.982 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:38:38.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:38:38.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:38:38.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:38.103 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:38:38.152 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:38:38.152 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:38:38.153 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:38:38.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:38.153 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-04-25T00:38:38.208 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:38:38.209 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:38:38.209 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:38:38.257 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:38:38.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:38:38.257 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:38:38.304 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:38:38.362 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:38:38.410 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:38:38.411 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:38:38.411 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:38:38.411 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:38:38.411 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:38:38.411 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:38:38.411 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:38:38.411 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:38:38.411 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:38:38.411 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:38:38.411 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:38:38.411 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:38:38.411 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:38:38.412 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:38:38.412 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:38:38.412 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:38:38.412 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:38:38.412 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:38:38.412 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:38:38.412 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:38:38.412 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:38:38.412 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:38:38.412 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:38:38.412 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:38:38.412 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:38:38.412 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:38:38.413 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:38:38.413 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:38:38.413 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:38:38.413 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:38:38.413 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:38:38.413 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:38:38.413 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:38:38.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:38.414 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:38:38.465 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:38:38.465 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:38:38.509 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:38:38.524 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:38:38.573 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:38:38.573 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:38:38.573 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:38:38.573 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:38:38.573 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:38:38.573 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:38:38.573 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:38:38.574 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:38:38.574 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:38:38.574 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:38:38.574 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:38:38.574 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:38:38.574 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:38:38.574 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:38:38.574 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:38:38.574 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:38:38.574 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:38:38.574 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:38:38.574 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:38:38.575 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:38:38.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:38.576 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:38:38.590 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:2024-04-25T00:38:38.586+0000 7f6d85644580 -1 init, newargv = 0x5563c08771b0 newargc=15 2024-04-25T00:38:38.590 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[50588]: starting ceph client 2024-04-25T00:38:38.596 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[50588]: starting fuse 2024-04-25T00:38:39.695 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:38:39.713 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:38:39.762 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:38:39.762 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:38:39.762 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:38:39.762 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:38:39.762 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:38:39.762 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:38:39.762 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:38:39.762 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:38:39.762 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:38:39.762 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:38:39.762 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:38:39.763 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:38:39.763 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:38:39.763 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:38:39.763 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:38:39.763 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:38:39.763 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:38:39.763 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:38:39.763 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:38:39.763 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:38:39.763 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:38:39.763 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:38:39.763 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:38:39.763 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:38:39.764 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:38:39.764 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:38:39.764 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:38:39.764 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:38:39.764 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:38:39.764 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:38:39.764 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:38:39.764 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:38:39.764 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:38:39.764 INFO:teuthology.orchestra.run.smithi029.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:38:39.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:39.765 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:38:39.813 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:38:39.813 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:38:39.813 DEBUG:teuthology.orchestra.run.smithi029:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:38:39.813 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-04-25T00:38:39.813 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-04-25T00:38:39.813 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-04-25T00:38:39.813 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-04-25T00:38:39.813 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:38:39.813 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:38:39.814 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-04-25T00:38:39.815 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-04-25T00:38:39.815 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-04-25T00:38:39.815 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:38:39.815 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:38:39.815 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.0")) 2024-04-25T00:38:39.815 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-04-25T00:38:39.940 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.0.50588.asok 2024-04-25T00:38:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:38:39.952 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50588.asok 2024-04-25T00:38:39.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:39.952 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50588.asok status 2024-04-25T00:38:40.109 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:38:40.109 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-04-25T00:38:40.109 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:38:40.109 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:38:40.109 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "0", 2024-04-25T00:38:40.109 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-04-25T00:38:40.109 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:38:40.110 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "50588", 2024-04-25T00:38:40.110 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-04-25T00:38:40.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:40.110 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-04-25T00:38:40.110 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-04-25T00:38:40.110 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 8697, 2024-04-25T00:38:40.110 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-04-25T00:38:40.110 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-04-25T00:38:40.110 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-04-25T00:38:40.110 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 8697 2024-04-25T00:38:40.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:40.110 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:38:40.110 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:38:40.111 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:38:40.111 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3426391697 2024-04-25T00:38:40.111 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:40.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:40.111 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:38:40.111 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:38:40.111 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:38:40.111 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3426391697 2024-04-25T00:38:40.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:38:40.111 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.8697 192.168.0.1:0/3426391697", 2024-04-25T00:38:40.111 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.1:0/3426391697", 2024-04-25T00:38:40.111 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-04-25T00:38:40.111 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 75, 2024-04-25T00:38:40.112 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 118, 2024-04-25T00:38:40.112 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:38:40.112 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-04-25T00:38:40.112 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-04-25T00:38:40.112 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:38:40.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:40.123 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-04-25T00:38:40.123 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:38:40.129 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-04-25T00:38:40.129 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:38:40.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:40.129 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:38:40.185 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:38:40.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.350+0000 7f8ed99dc640 1 Processor -- start 2024-04-25T00:38:40.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.350+0000 7f8ed99dc640 1 -- start start 2024-04-25T00:38:40.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed99dc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ed40b2b70 0x7f8ed4151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:40.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed99dc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ed40b3130 con 0x7f8ed415d980 2024-04-25T00:38:40.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed99dc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ed40b3410 con 0x7f8ed406dce0 2024-04-25T00:38:40.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed99dc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8ed40cf560 con 0x7f8ed40b2b70 2024-04-25T00:38:40.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed2ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8ed415d980 0x7f8ed406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56308/0 (socket says 172.21.15.29:56308) 2024-04-25T00:38:40.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed2ffd640 1 -- 172.21.15.29:0/1572475292 learned_addr learned my addr 172.21.15.29:0/1572475292 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:40.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed37fe640 1 --2- 172.21.15.29:0/1572475292 >> v2:172.21.15.157:3300/0 conn(0x7f8ed40b2b70 0x7f8ed4151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:40.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3096391779 0 0) 0x7f8ed40b3130 con 0x7f8ed415d980 2024-04-25T00:38:40.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ebc003450 con 0x7f8ed415d980 2024-04-25T00:38:40.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 452268696 0 0) 0x7f8ebc003450 con 0x7f8ed415d980 2024-04-25T00:38:40.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 >> v2:172.21.15.157:3300/0 conn(0x7f8ed40b2b70 msgr2=0x7f8ed4151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 --2- 172.21.15.29:0/1572475292 >> v2:172.21.15.157:3300/0 conn(0x7f8ed40b2b70 0x7f8ed4151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 >> v1:172.21.15.157:6789/0 conn(0x7f8ed406dce0 legacy=0x7f8ed4150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed40b36f0 con 0x7f8ed415d980 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8ec4001680 con 0x7f8ed415d980 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 >> v1:172.21.15.29:6789/0 conn(0x7f8ed415d980 legacy=0x7f8ed406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 --2- 172.21.15.29:0/1572475292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc003ab0 0x7f8ebc003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 --2- 172.21.15.29:0/1572475292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ebc008860 0x7f8ebc00acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 --2- 172.21.15.29:0/1572475292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc00b4d0 0x7f8ebc00d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed40cf560 con 0x7f8ebc003ab0 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc00e270 con 0x7f8ebc00b4d0 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8ebc00e470 con 0x7f8ebc008860 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed2ffd640 1 --2- 172.21.15.29:0/1572475292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc003ab0 0x7f8ebc003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed37fe640 1 --2- 172.21.15.29:0/1572475292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc00b4d0 0x7f8ebc00d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:40.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed27fc640 1 --2- 172.21.15.29:0/1572475292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ebc008860 0x7f8ebc00acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed2ffd640 1 -- 172.21.15.29:0/1572475292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ebc008860 msgr2=0x7f8ebc00acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed2ffd640 1 --2- 172.21.15.29:0/1572475292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ebc008860 0x7f8ebc00acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed2ffd640 1 -- 172.21.15.29:0/1572475292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc00b4d0 msgr2=0x7f8ebc00d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed2ffd640 1 --2- 172.21.15.29:0/1572475292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc00b4d0 0x7f8ebc00d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed2ffd640 1 -- 172.21.15.29:0/1572475292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc003450 con 0x7f8ebc003ab0 2024-04-25T00:38:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed37fe640 1 --2- 172.21.15.29:0/1572475292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc00b4d0 0x7f8ebc00d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed27fc640 1 --2- 172.21.15.29:0/1572475292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ebc008860 0x7f8ebc00acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:40.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.354+0000 7f8ed37fe640 1 -- 172.21.15.29:0/1572475292 reap_dead start 2024-04-25T00:38:40.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed2ffd640 1 --2- 172.21.15.29:0/1572475292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc003ab0 0x7f8ebc003ed0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400c1c0 tx=0x7f8ec401cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be0988f3cfe48b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:40.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4007d00 con 0x7f8ebc003ab0 2024-04-25T00:38:40.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec40a9020 con 0x7f8ebc003ab0 2024-04-25T00:38:40.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed1ffb640 1 -- 172.21.15.29:0/1572475292 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4005b20 con 0x7f8ebc003ab0 2024-04-25T00:38:40.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 -- 172.21.15.29:0/1572475292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc003ab0 msgr2=0x7f8ebc003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:40.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 --2- 172.21.15.29:0/1572475292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc003ab0 0x7f8ebc003ed0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400c1c0 tx=0x7f8ec401cbf0 comp rx=0 tx=0).stop 2024-04-25T00:38:40.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 -- 172.21.15.29:0/1572475292 shutdown_connections 2024-04-25T00:38:40.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 --2- 172.21.15.29:0/1572475292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ebc003ab0 0x7f8ebc003ed0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:40.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 -- 172.21.15.29:0/1572475292 >> 172.21.15.29:0/1572475292 conn(0x7f8ed40b1870 msgr2=0x7f8ed40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:40.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 -- 172.21.15.29:0/1572475292 shutdown_connections 2024-04-25T00:38:40.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 -- 172.21.15.29:0/1572475292 wait complete. 2024-04-25T00:38:40.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 Processor -- start 2024-04-25T00:38:40.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 -- start start 2024-04-25T00:38:40.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc003ab0 0x7f8ed4142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:40.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ed4155040 0x7f8ed4158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:40.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ed41594d0 0x7f8ed4070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:40.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4151780 con 0x7f8ed4155040 2024-04-25T00:38:40.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4151600 con 0x7f8ebc003ab0 2024-04-25T00:38:40.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed99dc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed4151900 con 0x7f8ed41594d0 2024-04-25T00:38:40.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ed4155040 0x7f8ed4158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ed4155040 0x7f8ed4158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54432/0 (socket says 172.21.15.29:54432) 2024-04-25T00:38:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed27fc640 1 -- 172.21.15.29:0/213321864 learned_addr learned my addr 172.21.15.29:0/213321864 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed2ffd640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc003ab0 0x7f8ed4142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed27fc640 1 -- 172.21.15.29:0/213321864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ed41594d0 msgr2=0x7f8ed4070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed37fe640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ed41594d0 0x7f8ed4070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed27fc640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ed41594d0 0x7f8ed4070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed27fc640 1 -- 172.21.15.29:0/213321864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc003ab0 msgr2=0x7f8ed4142190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed27fc640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc003ab0 0x7f8ed4142190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed27fc640 1 -- 172.21.15.29:0/213321864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed4072650 con 0x7f8ed4155040 2024-04-25T00:38:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.358+0000 7f8ed27fc640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ed4155040 0x7f8ed4158f90 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800d4a0 tx=0x7f8ec800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.362+0000 7f8ebbfff640 1 -- 172.21.15.29:0/213321864 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8019070 con 0x7f8ed4155040 2024-04-25T00:38:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.362+0000 7f8ebbfff640 1 -- 172.21.15.29:0/213321864 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec80054a0 con 0x7f8ed4155040 2024-04-25T00:38:40.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.362+0000 7f8ed99dc640 1 -- 172.21.15.29:0/213321864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed41e7c40 con 0x7f8ed4155040 2024-04-25T00:38:40.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.362+0000 7f8ebbfff640 1 -- 172.21.15.29:0/213321864 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8007270 con 0x7f8ed4155040 2024-04-25T00:38:40.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.362+0000 7f8ed99dc640 1 -- 172.21.15.29:0/213321864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed40722e0 con 0x7f8ed4155040 2024-04-25T00:38:40.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.362+0000 7f8ed99dc640 1 -- 172.21.15.29:0/213321864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed4069270 con 0x7f8ed4155040 2024-04-25T00:38:40.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.362+0000 7f8ebbfff640 1 -- 172.21.15.29:0/213321864 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f8ec801c070 con 0x7f8ed4155040 2024-04-25T00:38:40.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.362+0000 7f8ebbfff640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ec0067ce0 0x7f8ec006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:40.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.362+0000 7f8ebbfff640 1 -- 172.21.15.29:0/213321864 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f8ec8085b40 con 0x7f8ed4155040 2024-04-25T00:38:40.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.362+0000 7f8ed2ffd640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ec0067ce0 0x7f8ec006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:40.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.362+0000 7f8ed2ffd640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ec0067ce0 0x7f8ec006a1a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40aa000 tx=0x7f8ed40b3300 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:40.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.366+0000 7f8ebbfff640 1 -- 172.21.15.29:0/213321864 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8ec80560e0 con 0x7f8ed4155040 2024-04-25T00:38:40.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.562+0000 7f8ed99dc640 1 -- 172.21.15.29:0/213321864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8ed41598b0 con 0x7f8ed4155040 2024-04-25T00:38:40.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.566+0000 7f8ebbfff640 1 -- 172.21.15.29:0/213321864 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) ==== 53+0+73 (secure 0 0 0) 0x7f8ec80559e0 con 0x7f8ed4155040 2024-04-25T00:38:40.568 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:38:40.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.566+0000 7f8ed99dc640 1 -- 172.21.15.29:0/213321864 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ec0067ce0 msgr2=0x7f8ec006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:40.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.566+0000 7f8ed99dc640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ec0067ce0 0x7f8ec006a1a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40aa000 tx=0x7f8ed40b3300 comp rx=0 tx=0).stop 2024-04-25T00:38:40.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.566+0000 7f8ed99dc640 1 -- 172.21.15.29:0/213321864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ed4155040 msgr2=0x7f8ed4158f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:40.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.566+0000 7f8ed99dc640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ed4155040 0x7f8ed4158f90 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800d4a0 tx=0x7f8ec800d960 comp rx=0 tx=0).stop 2024-04-25T00:38:40.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.566+0000 7f8ed99dc640 1 -- 172.21.15.29:0/213321864 shutdown_connections 2024-04-25T00:38:40.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.566+0000 7f8ed99dc640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ec0067ce0 0x7f8ec006a1a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:40.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.566+0000 7f8ed99dc640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ed41594d0 0x7f8ed4070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:40.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.566+0000 7f8ed99dc640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ed4155040 0x7f8ed4158f90 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:40.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.566+0000 7f8ed99dc640 1 --2- 172.21.15.29:0/213321864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc003ab0 0x7f8ed4142190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:40.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.566+0000 7f8ed99dc640 1 -- 172.21.15.29:0/213321864 >> 172.21.15.29:0/213321864 conn(0x7f8ed40b1870 msgr2=0x7f8ed406cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:40.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.566+0000 7f8ed99dc640 1 -- 172.21.15.29:0/213321864 shutdown_connections 2024-04-25T00:38:40.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:40.566+0000 7f8ed99dc640 1 -- 172.21.15.29:0/213321864 wait complete. 2024-04-25T00:38:40.587 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:38:40.587 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:38:40.587 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-25T00:38:40.587 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:38:40.587 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:38:40.587 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:38:40.587 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:38:40.587 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:38:40.587 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:38:40.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:40.588 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-25T00:38:40.595 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:38:40.595 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:38:40.595 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:38:40.595 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:40.595 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42035sec preferred_lft 42035sec 2024-04-25T00:38:40.596 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-25T00:38:40.597 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:40.597 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:38:40.597 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:9c:8d:4b:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:38:40.597 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:38:40.597 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:40.597 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:38:40.597 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:38:40.597 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:38:40.597 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:40.597 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:38:40.597 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:40.597 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:38:40.597 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:38:40.598 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:38:40.598 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:40.598 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:38:40.598 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:38:40.598 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:38:40.598 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:38:40.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:40.599 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:38:40.651 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:38:40.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:40.653 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-25T00:38:40.704 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-25T00:38:40.704 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:38:40.704 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:38:40.704 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:38:40.704 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:38:40.704 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:38:40.704 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:38:40.704 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:38:40.704 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:38:40.705 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:38:40.705 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:38:40.705 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:38:40.705 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:38:40.705 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:38:40.705 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:38:40.705 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:38:40.705 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:38:40.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:38:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:38:40.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:38:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:38:40.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:40.830 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:38:40.836 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:38:40.836 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:38:40.836 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:38:40.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:40.837 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-25T00:38:40.890 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T00:38:40.891 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:38:40.891 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:38:40.939 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:38:40.940 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:38:40.940 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:38:40.965 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:38:40.968 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:38:40.987 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:38:41.084 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:38:41.132 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:38:41.132 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:38:41.132 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:38:41.132 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:38:41.132 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:38:41.132 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:38:41.132 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:38:41.132 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:38:41.132 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:38:41.132 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:38:41.132 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:38:41.132 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:38:41.133 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:38:41.133 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:38:41.133 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:38:41.133 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:38:41.133 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:38:41.133 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:38:41.133 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:38:41.133 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:38:41.133 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:38:41.133 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:38:41.133 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:38:41.133 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:38:41.134 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:38:41.134 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:38:41.134 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:38:41.134 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:38:41.134 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:38:41.134 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:38:41.134 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:38:41.134 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:38:41.134 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:38:41.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:41.135 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:38:41.183 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:38:41.183 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T00:38:41.228 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:38:41.242 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:38:41.292 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:38:41.292 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:38:41.292 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:38:41.292 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:38:41.292 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:38:41.292 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:38:41.292 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:38:41.292 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:38:41.292 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:38:41.292 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:38:41.292 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:38:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:38:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:38:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:38:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:38:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:38:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:38:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:38:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:38:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:38:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:38:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:38:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:38:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:38:41.294 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:38:41.294 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:38:41.294 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:38:41.294 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:38:41.294 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:38:41.294 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:38:41.294 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:38:41.294 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:38:41.294 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:38:41.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:41.295 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:38:41.308 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-04-25T00:38:41.306+0000 7fdf2c363580 -1 init, newargv = 0x55ba91191fe0 newargc=15 2024-04-25T00:38:41.308 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34753]: starting ceph client 2024-04-25T00:38:41.315 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34753]: starting fuse 2024-04-25T00:38:42.413 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:38:42.429 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:38:42.476 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:38:42.477 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:38:42.478 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:38:42.478 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:38:42.478 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:38:42.478 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:38:42.478 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:38:42.478 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:38:42.478 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:38:42.478 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:38:42.478 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:38:42.478 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:38:42.478 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:38:42.478 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:38:42.479 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:38:42.479 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:38:42.479 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:38:42.479 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:38:42.479 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:38:42.479 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:38:42.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:42.479 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:38:42.523 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:38:42.523 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-25T00:38:42.524 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-25T00:38:42.525 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-25T00:38:42.525 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:38:42.525 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-25T00:38:42.525 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:38:42.525 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:38:42.525 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:38:42.525 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-25T00:38:42.525 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-25T00:38:42.525 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-25T00:38:42.525 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:38:42.525 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:38:42.525 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-04-25T00:38:42.525 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-25T00:38:42.648 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.34753.asok 2024-04-25T00:38:42.652 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:38:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:38:42.660 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34753.asok 2024-04-25T00:38:42.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:42.660 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34753.asok status 2024-04-25T00:38:42.820 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:38:42.820 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-25T00:38:42.820 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:38:42.820 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:38:42.820 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-04-25T00:38:42.820 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-25T00:38:42.820 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T00:38:42.820 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "34753", 2024-04-25T00:38:42.821 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-25T00:38:42.821 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:38:42.821 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-25T00:38:42.821 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-25T00:38:42.821 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7222, 2024-04-25T00:38:42.821 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-25T00:38:42.821 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-25T00:38:42.821 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-25T00:38:42.821 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 7222 2024-04-25T00:38:42.821 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:38:42.821 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:38:42.821 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:38:42.821 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:38:42.822 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3837077812 2024-04-25T00:38:42.822 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-25T00:38:42.822 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:38:42.822 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:38:42.822 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:38:42.822 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:38:42.822 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3837077812 2024-04-25T00:38:42.822 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:38:42.822 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.7222 192.168.0.1:0/3837077812", 2024-04-25T00:38:42.822 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/3837077812", 2024-04-25T00:38:42.822 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-25T00:38:42.822 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 75, 2024-04-25T00:38:42.823 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 118, 2024-04-25T00:38:42.823 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:38:42.823 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-25T00:38:42.823 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-25T00:38:42.823 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:38:42.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:42.834 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-25T00:38:42.834 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T00:38:42.840 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-25T00:38:42.840 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T00:38:42.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:42.840 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:38:42.892 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-25T00:38:43.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f201b801640 1 Processor -- start 2024-04-25T00:38:43.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f201b801640 1 -- start start 2024-04-25T00:38:43.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f201b801640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f201415e180 0x7f20140d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f201b801640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f20140d2680 0x7f20140d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f201b801640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20140cfed0 con 0x7f201415e560 2024-04-25T00:38:43.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f201b801640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f20140b7100 con 0x7f20140d2680 2024-04-25T00:38:43.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f201b801640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f20140b73e0 con 0x7f201415e180 2024-04-25T00:38:43.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f2019d9e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f20140d2680 0x7f20140d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f2019d9e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f20140d2680 0x7f20140d2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54440/0 (socket says 172.21.15.29:54440) 2024-04-25T00:38:43.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f2019d9e640 1 -- 172.21.15.29:0/3879108606 learned_addr learned my addr 172.21.15.29:0/3879108606 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:43.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f201959d640 1 --2- 172.21.15.29:0/3879108606 >> v2:172.21.15.157:3300/0 conn(0x7f201415e180 0x7f20140d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f2019d9e640 1 -- 172.21.15.29:0/3879108606 >> v2:172.21.15.157:3300/0 conn(0x7f201415e180 msgr2=0x7f20140d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:43.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f2019d9e640 1 --2- 172.21.15.29:0/3879108606 >> v2:172.21.15.157:3300/0 conn(0x7f201415e180 0x7f20140d20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f2019d9e640 1 -- 172.21.15.29:0/3879108606 >> v1:172.21.15.157:6789/0 conn(0x7f201415e560 legacy=0x7f20140cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:43.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f2019d9e640 1 -- 172.21.15.29:0/3879108606 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20140b76c0 con 0x7f20140d2680 2024-04-25T00:38:43.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.030+0000 7f201959d640 1 --2- 172.21.15.29:0/3879108606 >> v2:172.21.15.157:3300/0 conn(0x7f201415e180 0x7f20140d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:43.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.034+0000 7f2019d9e640 1 --2- 172.21.15.29:0/3879108606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20140d2680 0x7f20140d2ad0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f201408d200 tx=0x7f200c01f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929faac7968dd61c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:43.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.034+0000 7f200bfff640 1 -- 172.21.15.29:0/3879108606 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0b4070 con 0x7f20140d2680 2024-04-25T00:38:43.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.034+0000 7f200bfff640 1 -- 172.21.15.29:0/3879108606 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f200c0af400 con 0x7f20140d2680 2024-04-25T00:38:43.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.034+0000 7f200bfff640 1 -- 172.21.15.29:0/3879108606 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0af7d0 con 0x7f20140d2680 2024-04-25T00:38:43.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.034+0000 7f201b801640 1 -- 172.21.15.29:0/3879108606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20140d2680 msgr2=0x7f20140d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.034+0000 7f201b801640 1 --2- 172.21.15.29:0/3879108606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20140d2680 0x7f20140d2ad0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f201408d200 tx=0x7f200c01f1a0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.034+0000 7f201b801640 1 -- 172.21.15.29:0/3879108606 shutdown_connections 2024-04-25T00:38:43.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.034+0000 7f201b801640 1 --2- 172.21.15.29:0/3879108606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20140d2680 0x7f20140d2ad0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.034+0000 7f201b801640 1 --2- 172.21.15.29:0/3879108606 >> v2:172.21.15.157:3300/0 conn(0x7f201415e180 0x7f20140d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.034+0000 7f201b801640 1 -- 172.21.15.29:0/3879108606 >> 172.21.15.29:0/3879108606 conn(0x7f20140d40a0 msgr2=0x7f20140d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:43.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.034+0000 7f201b801640 1 -- 172.21.15.29:0/3879108606 shutdown_connections 2024-04-25T00:38:43.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.034+0000 7f201b801640 1 -- 172.21.15.29:0/3879108606 wait complete. 2024-04-25T00:38:43.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f201b801640 1 Processor -- start 2024-04-25T00:38:43.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f201b801640 1 -- start start 2024-04-25T00:38:43.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f201b801640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20140d2680 0x7f20140caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f201b801640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f201415e180 0x7f201414d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f201b801640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f201415e560 0x7f201415b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f201b801640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f20140b7d60 con 0x7f201415e180 2024-04-25T00:38:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f201b801640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20140b7be0 con 0x7f201415e560 2024-04-25T00:38:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f201b801640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f20140b7ee0 con 0x7f20140d2680 2024-04-25T00:38:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2018d9c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f201415e180 0x7f201414d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2018d9c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f201415e180 0x7f201414d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54444/0 (socket says 172.21.15.29:54444) 2024-04-25T00:38:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2018d9c640 1 -- 172.21.15.29:0/2193254321 learned_addr learned my addr 172.21.15.29:0/2193254321 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2019d9e640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f201415e560 0x7f201415b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f201959d640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20140d2680 0x7f20140caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2018d9c640 1 -- 172.21.15.29:0/2193254321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20140d2680 msgr2=0x7f20140caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2018d9c640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20140d2680 0x7f20140caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2018d9c640 1 -- 172.21.15.29:0/2193254321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f201415e560 msgr2=0x7f201415b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2018d9c640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f201415e560 0x7f201415b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2018d9c640 1 -- 172.21.15.29:0/2193254321 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201414e010 con 0x7f201415e180 2024-04-25T00:38:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2018d9c640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f201415e180 0x7f201414d8a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f200400b410 tx=0x7f200400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2009ffb640 1 -- 172.21.15.29:0/2193254321 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004015030 con 0x7f201415e180 2024-04-25T00:38:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2009ffb640 1 -- 172.21.15.29:0/2193254321 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2004004020 con 0x7f201415e180 2024-04-25T00:38:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f201b801640 1 -- 172.21.15.29:0/2193254321 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201414e2f0 con 0x7f201415e180 2024-04-25T00:38:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2009ffb640 1 -- 172.21.15.29:0/2193254321 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004004450 con 0x7f201415e180 2024-04-25T00:38:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f201b801640 1 -- 172.21.15.29:0/2193254321 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201415b940 con 0x7f201415e180 2024-04-25T00:38:43.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f201b801640 1 -- 172.21.15.29:0/2193254321 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20140cb150 con 0x7f201415e180 2024-04-25T00:38:43.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2009ffb640 1 -- 172.21.15.29:0/2193254321 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f2004018070 con 0x7f201415e180 2024-04-25T00:38:43.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.038+0000 7f2009ffb640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1ff8067cb0 0x7f1ff806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.042+0000 7f2009ffb640 1 -- 172.21.15.29:0/2193254321 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f2004086090 con 0x7f201415e180 2024-04-25T00:38:43.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.042+0000 7f201959d640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1ff8067cb0 0x7f1ff806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.042+0000 7f201959d640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1ff8067cb0 0x7f1ff806a170 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f201000a9a0 tx=0x7f2010008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:43.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.046+0000 7f2009ffb640 1 -- 172.21.15.29:0/2193254321 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f20140cb150 con 0x7f201415e180 2024-04-25T00:38:43.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.242+0000 7f201b801640 1 -- 172.21.15.29:0/2193254321 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f20140692e0 con 0x7f201415e180 2024-04-25T00:38:43.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.294+0000 7f2009ffb640 1 -- 172.21.15.29:0/2193254321 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v76) ==== 136+0+0 (secure 0 0 0) 0x7f2004056560 con 0x7f201415e180 2024-04-25T00:38:43.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.298+0000 7f201b801640 1 -- 172.21.15.29:0/2193254321 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1ff8067cb0 msgr2=0x7f1ff806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.298+0000 7f201b801640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1ff8067cb0 0x7f1ff806a170 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f201000a9a0 tx=0x7f2010008040 comp rx=0 tx=0).stop 2024-04-25T00:38:43.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.298+0000 7f201b801640 1 -- 172.21.15.29:0/2193254321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f201415e180 msgr2=0x7f201414d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.298+0000 7f201b801640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f201415e180 0x7f201414d8a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f200400b410 tx=0x7f200400b8d0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.298+0000 7f201b801640 1 -- 172.21.15.29:0/2193254321 shutdown_connections 2024-04-25T00:38:43.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.298+0000 7f201b801640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1ff8067cb0 0x7f1ff806a170 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.298+0000 7f201b801640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f201415e560 0x7f201415b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.298+0000 7f201b801640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f201415e180 0x7f201414d8a0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.298+0000 7f201b801640 1 --2- 172.21.15.29:0/2193254321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20140d2680 0x7f20140caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.298+0000 7f201b801640 1 -- 172.21.15.29:0/2193254321 >> 172.21.15.29:0/2193254321 conn(0x7f20140d40a0 msgr2=0x7f20140b9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:43.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.298+0000 7f201b801640 1 -- 172.21.15.29:0/2193254321 shutdown_connections 2024-04-25T00:38:43.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.298+0000 7f201b801640 1 -- 172.21.15.29:0/2193254321 wait complete. 2024-04-25T00:38:43.320 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-25T00:38:43.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61bb1b7640 1 Processor -- start 2024-04-25T00:38:43.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61bb1b7640 1 -- start start 2024-04-25T00:38:43.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61bb1b7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61b415d980 0x7f61b406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61bb1b7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61b40b2b70 0x7f61b406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61bb1b7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f61b4150df0 0x7f61b4151280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61bb1b7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f61b40cf560 con 0x7f61b4150df0 2024-04-25T00:38:43.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61bb1b7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f61b40b3130 con 0x7f61b415d980 2024-04-25T00:38:43.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61bb1b7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f61b40b3410 con 0x7f61b40b2b70 2024-04-25T00:38:43.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61b9754640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f61b4150df0 0x7f61b4151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61b9754640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f61b4150df0 0x7f61b4151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54454/0 (socket says 172.21.15.29:54454) 2024-04-25T00:38:43.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61b9754640 1 -- 172.21.15.29:0/4227367803 learned_addr learned my addr 172.21.15.29:0/4227367803 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:43.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61b8f53640 1 --2- 172.21.15.29:0/4227367803 >> v2:172.21.15.157:3300/0 conn(0x7f61b415d980 0x7f61b406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61b3fff640 1 --2- 172.21.15.29:0/4227367803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61b40b2b70 0x7f61b406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61b9754640 1 -- 172.21.15.29:0/4227367803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61b40b2b70 msgr2=0x7f61b406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:43.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61b9754640 1 --2- 172.21.15.29:0/4227367803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61b40b2b70 0x7f61b406db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61b9754640 1 -- 172.21.15.29:0/4227367803 >> v2:172.21.15.157:3300/0 conn(0x7f61b415d980 msgr2=0x7f61b406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:43.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61b9754640 1 --2- 172.21.15.29:0/4227367803 >> v2:172.21.15.157:3300/0 conn(0x7f61b415d980 0x7f61b406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61b9754640 1 -- 172.21.15.29:0/4227367803 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b40b36f0 con 0x7f61b4150df0 2024-04-25T00:38:43.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.442+0000 7f61b8f53640 1 --2- 172.21.15.29:0/4227367803 >> v2:172.21.15.157:3300/0 conn(0x7f61b415d980 0x7f61b406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:43.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61b9754640 1 --2- 172.21.15.29:0/4227367803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61b4150df0 0x7f61b4151280 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f61b40d0ca0 tx=0x7f61ac01dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bcf51bc60f45b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:43.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61b37fe640 1 -- 172.21.15.29:0/4227367803 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac0a6800 con 0x7f61b4150df0 2024-04-25T00:38:43.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61b37fe640 1 -- 172.21.15.29:0/4227367803 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61ac0a69a0 con 0x7f61b4150df0 2024-04-25T00:38:43.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61b37fe640 1 -- 172.21.15.29:0/4227367803 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac0a6d70 con 0x7f61b4150df0 2024-04-25T00:38:43.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61bb1b7640 1 -- 172.21.15.29:0/4227367803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61b4150df0 msgr2=0x7f61b4151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61bb1b7640 1 --2- 172.21.15.29:0/4227367803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61b4150df0 0x7f61b4151280 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f61b40d0ca0 tx=0x7f61ac01dad0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61bb1b7640 1 -- 172.21.15.29:0/4227367803 shutdown_connections 2024-04-25T00:38:43.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61bb1b7640 1 --2- 172.21.15.29:0/4227367803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61b4150df0 0x7f61b4151280 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61bb1b7640 1 --2- 172.21.15.29:0/4227367803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61b40b2b70 0x7f61b406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61bb1b7640 1 --2- 172.21.15.29:0/4227367803 >> v2:172.21.15.157:3300/0 conn(0x7f61b415d980 0x7f61b406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61bb1b7640 1 -- 172.21.15.29:0/4227367803 >> 172.21.15.29:0/4227367803 conn(0x7f61b40b1870 msgr2=0x7f61b40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:43.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61bb1b7640 1 -- 172.21.15.29:0/4227367803 shutdown_connections 2024-04-25T00:38:43.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61bb1b7640 1 -- 172.21.15.29:0/4227367803 wait complete. 2024-04-25T00:38:43.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61bb1b7640 1 Processor -- start 2024-04-25T00:38:43.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.446+0000 7f61bb1b7640 1 -- start start 2024-04-25T00:38:43.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61bb1b7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61b40b2b70 0x7f61b414bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61bb1b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b4150df0 0x7f61b41f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61bb1b7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61b415d980 0x7f61b41f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61bb1b7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f61b4151940 con 0x7f61b40b2b70 2024-04-25T00:38:43.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61bb1b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61b41517c0 con 0x7f61b4150df0 2024-04-25T00:38:43.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61bb1b7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f61b415e020 con 0x7f61b415d980 2024-04-25T00:38:43.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b8f53640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61b40b2b70 0x7f61b414bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b8f53640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61b40b2b70 0x7f61b414bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54460/0 (socket says 172.21.15.29:54460) 2024-04-25T00:38:43.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b8f53640 1 -- 172.21.15.29:0/1097506323 learned_addr learned my addr 172.21.15.29:0/1097506323 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:43.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b3fff640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b4150df0 0x7f61b41f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b9754640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61b415d980 0x7f61b41f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b8f53640 1 -- 172.21.15.29:0/1097506323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61b415d980 msgr2=0x7f61b41f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b8f53640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61b415d980 0x7f61b41f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b8f53640 1 -- 172.21.15.29:0/1097506323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b4150df0 msgr2=0x7f61b41f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b8f53640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b4150df0 0x7f61b41f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b8f53640 1 -- 172.21.15.29:0/1097506323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b41f6610 con 0x7f61b40b2b70 2024-04-25T00:38:43.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b8f53640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61b40b2b70 0x7f61b414bc30 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f61a800c820 tx=0x7f61a800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:43.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b17fa640 1 -- 172.21.15.29:0/1097506323 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8015070 con 0x7f61b40b2b70 2024-04-25T00:38:43.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b17fa640 1 -- 172.21.15.29:0/1097506323 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a8007a30 con 0x7f61b40b2b70 2024-04-25T00:38:43.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b17fa640 1 -- 172.21.15.29:0/1097506323 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8007e40 con 0x7f61b40b2b70 2024-04-25T00:38:43.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61bb1b7640 1 -- 172.21.15.29:0/1097506323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b41f68f0 con 0x7f61b40b2b70 2024-04-25T00:38:43.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61bb1b7640 1 -- 172.21.15.29:0/1097506323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b41f6eb0 con 0x7f61b40b2b70 2024-04-25T00:38:43.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61bb1b7640 1 -- 172.21.15.29:0/1097506323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b4069270 con 0x7f61b40b2b70 2024-04-25T00:38:43.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b17fa640 1 -- 172.21.15.29:0/1097506323 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f61a8018070 con 0x7f61b40b2b70 2024-04-25T00:38:43.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b17fa640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6198067cb0 0x7f619806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b3fff640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6198067cb0 0x7f619806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.450+0000 7f61b17fa640 1 -- 172.21.15.29:0/1097506323 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f61a80863d0 con 0x7f61b40b2b70 2024-04-25T00:38:43.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.454+0000 7f61b3fff640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6198067cb0 0x7f619806a170 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f61b415c3f0 tx=0x7f61a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:43.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.458+0000 7f61b17fa640 1 -- 172.21.15.29:0/1097506323 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f61a8056870 con 0x7f61b40b2b70 2024-04-25T00:38:43.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.642+0000 7f61bb1b7640 1 -- 172.21.15.29:0/1097506323 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f61b40cf910 con 0x7f6198067cb0 2024-04-25T00:38:43.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.646+0000 7f61b17fa640 1 -- 172.21.15.29:0/1097506323 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f61b40cf910 con 0x7f6198067cb0 2024-04-25T00:38:43.648 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:38:43.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:38:43.648 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-04-25T00:38:43.648 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:43.648 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:38:43.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.646+0000 7f61bb1b7640 1 -- 172.21.15.29:0/1097506323 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6198067cb0 msgr2=0x7f619806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.646+0000 7f61bb1b7640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6198067cb0 0x7f619806a170 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f61b415c3f0 tx=0x7f61a0008040 comp rx=0 tx=0).stop 2024-04-25T00:38:43.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.646+0000 7f61bb1b7640 1 -- 172.21.15.29:0/1097506323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61b40b2b70 msgr2=0x7f61b414bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.646+0000 7f61bb1b7640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61b40b2b70 0x7f61b414bc30 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f61a800c820 tx=0x7f61a800cce0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.646+0000 7f61bb1b7640 1 -- 172.21.15.29:0/1097506323 shutdown_connections 2024-04-25T00:38:43.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.646+0000 7f61bb1b7640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6198067cb0 0x7f619806a170 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.646+0000 7f61bb1b7640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61b415d980 0x7f61b41f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.646+0000 7f61bb1b7640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b4150df0 0x7f61b41f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.646+0000 7f61bb1b7640 1 --2- 172.21.15.29:0/1097506323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61b40b2b70 0x7f61b414bc30 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.646+0000 7f61bb1b7640 1 -- 172.21.15.29:0/1097506323 >> 172.21.15.29:0/1097506323 conn(0x7f61b40b1870 msgr2=0x7f61b406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:43.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.646+0000 7f61bb1b7640 1 -- 172.21.15.29:0/1097506323 shutdown_connections 2024-04-25T00:38:43.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.646+0000 7f61bb1b7640 1 -- 172.21.15.29:0/1097506323 wait complete. 2024-04-25T00:38:43.668 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:38:43.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2788616640 1 Processor -- start 2024-04-25T00:38:43.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2788616640 1 -- start start 2024-04-25T00:38:43.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2788616640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27800b3140 0x7f278006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2788616640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27800b4b10 0x7f27800b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2788616640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27800b41e0 con 0x7f27800b3a60 2024-04-25T00:38:43.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2788616640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f278006d960 con 0x7f27800b4b10 2024-04-25T00:38:43.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2788616640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f278006dc40 con 0x7f27800b3140 2024-04-25T00:38:43.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2785bb1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27800b4b10 0x7f27800b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2785bb1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27800b4b10 0x7f27800b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54476/0 (socket says 172.21.15.29:54476) 2024-04-25T00:38:43.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2785bb1640 1 -- 172.21.15.29:0/257852871 learned_addr learned my addr 172.21.15.29:0/257852871 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:43.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f27863b2640 1 --2- 172.21.15.29:0/257852871 >> v2:172.21.15.157:3300/0 conn(0x7f27800b3140 0x7f278006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2785bb1640 1 -- 172.21.15.29:0/257852871 >> v2:172.21.15.157:3300/0 conn(0x7f27800b3140 msgr2=0x7f278006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:43.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2785bb1640 1 --2- 172.21.15.29:0/257852871 >> v2:172.21.15.157:3300/0 conn(0x7f27800b3140 0x7f278006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2785bb1640 1 -- 172.21.15.29:0/257852871 >> v1:172.21.15.157:6789/0 conn(0x7f27800b3a60 legacy=0x7f278015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:43.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.786+0000 7f2785bb1640 1 -- 172.21.15.29:0/257852871 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278006df20 con 0x7f27800b4b10 2024-04-25T00:38:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2785bb1640 1 --2- 172.21.15.29:0/257852871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27800b4b10 0x7f27800b3520 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f2770009870 tx=0x7f277001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bafa64ec2e08fed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f27853b0640 1 -- 172.21.15.29:0/257852871 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27700b0070 con 0x7f27800b4b10 2024-04-25T00:38:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f27853b0640 1 -- 172.21.15.29:0/257852871 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27700ab600 con 0x7f27800b4b10 2024-04-25T00:38:43.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f27853b0640 1 -- 172.21.15.29:0/257852871 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27700a4e60 con 0x7f27800b4b10 2024-04-25T00:38:43.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 -- 172.21.15.29:0/257852871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27800b4b10 msgr2=0x7f27800b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 --2- 172.21.15.29:0/257852871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27800b4b10 0x7f27800b3520 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f2770009870 tx=0x7f277001c4e0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 -- 172.21.15.29:0/257852871 shutdown_connections 2024-04-25T00:38:43.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 --2- 172.21.15.29:0/257852871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27800b4b10 0x7f27800b3520 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 --2- 172.21.15.29:0/257852871 >> v2:172.21.15.157:3300/0 conn(0x7f27800b3140 0x7f278006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 -- 172.21.15.29:0/257852871 >> 172.21.15.29:0/257852871 conn(0x7f27800b1870 msgr2=0x7f27800b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:43.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 -- 172.21.15.29:0/257852871 shutdown_connections 2024-04-25T00:38:43.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 -- 172.21.15.29:0/257852871 wait complete. 2024-04-25T00:38:43.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 Processor -- start 2024-04-25T00:38:43.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 -- start start 2024-04-25T00:38:43.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27800b3140 0x7f2780163d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27800b3a60 0x7f2780138c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27800b4b10 0x7f2780072830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2780168b70 con 0x7f27800b3140 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27801689f0 con 0x7f27800b4b10 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f2788616640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2780168cf0 con 0x7f27800b3a60 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.790+0000 7f27863b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27800b3140 0x7f2780163d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f27863b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27800b3140 0x7f2780163d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54480/0 (socket says 172.21.15.29:54480) 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f27863b2640 1 -- 172.21.15.29:0/4112998168 learned_addr learned my addr 172.21.15.29:0/4112998168 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2786bb3640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27800b4b10 0x7f2780072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2785bb1640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27800b3a60 0x7f2780138c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f27863b2640 1 -- 172.21.15.29:0/4112998168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27800b3a60 msgr2=0x7f2780138c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f27863b2640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27800b3a60 0x7f2780138c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f27863b2640 1 -- 172.21.15.29:0/4112998168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27800b4b10 msgr2=0x7f2780072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f27863b2640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27800b4b10 0x7f2780072830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f27863b2640 1 -- 172.21.15.29:0/4112998168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2780071d90 con 0x7f27800b3140 2024-04-25T00:38:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2786bb3640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27800b4b10 0x7f2780072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f27863b2640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27800b3140 0x7f2780163d00 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f277c00d5f0 tx=0x7f277c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2776ffd640 1 -- 172.21.15.29:0/4112998168 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c0125f0 con 0x7f27800b3140 2024-04-25T00:38:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2776ffd640 1 -- 172.21.15.29:0/4112998168 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f277c019070 con 0x7f27800b3140 2024-04-25T00:38:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2788616640 1 -- 172.21.15.29:0/4112998168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278015bab0 con 0x7f27800b3140 2024-04-25T00:38:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2776ffd640 1 -- 172.21.15.29:0/4112998168 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c0129b0 con 0x7f27800b3140 2024-04-25T00:38:43.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2788616640 1 -- 172.21.15.29:0/4112998168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2780072310 con 0x7f27800b3140 2024-04-25T00:38:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2788616640 1 -- 172.21.15.29:0/4112998168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2780069270 con 0x7f27800b3140 2024-04-25T00:38:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2776ffd640 1 -- 172.21.15.29:0/4112998168 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f277c012b50 con 0x7f27800b3140 2024-04-25T00:38:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2776ffd640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2764067cb0 0x7f276406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2776ffd640 1 -- 172.21.15.29:0/4112998168 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f277c086780 con 0x7f27800b3140 2024-04-25T00:38:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2785bb1640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2764067cb0 0x7f276406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:43.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.794+0000 7f2785bb1640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2764067cb0 0x7f276406a170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f27700ad000 tx=0x7f27700a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:43.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.798+0000 7f2776ffd640 1 -- 172.21.15.29:0/4112998168 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f277c056d00 con 0x7f27800b3140 2024-04-25T00:38:43.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.958+0000 7f2788616640 1 -- 172.21.15.29:0/4112998168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f27800b47a0 con 0x7f27800b3140 2024-04-25T00:38:43.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.958+0000 7f2776ffd640 1 -- 172.21.15.29:0/4112998168 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f277c056600 con 0x7f27800b3140 2024-04-25T00:38:43.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.962+0000 7f2788616640 1 -- 172.21.15.29:0/4112998168 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2764067cb0 msgr2=0x7f276406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.962+0000 7f2788616640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2764067cb0 0x7f276406a170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f27700ad000 tx=0x7f27700a6000 comp rx=0 tx=0).stop 2024-04-25T00:38:43.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.962+0000 7f2788616640 1 -- 172.21.15.29:0/4112998168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27800b3140 msgr2=0x7f2780163d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.962+0000 7f2788616640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27800b3140 0x7f2780163d00 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f277c00d5f0 tx=0x7f277c00dab0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.962+0000 7f2788616640 1 -- 172.21.15.29:0/4112998168 shutdown_connections 2024-04-25T00:38:43.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.962+0000 7f2788616640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2764067cb0 0x7f276406a170 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.962+0000 7f2788616640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27800b4b10 0x7f2780072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.962+0000 7f2788616640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27800b3a60 0x7f2780138c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.962+0000 7f2788616640 1 --2- 172.21.15.29:0/4112998168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27800b3140 0x7f2780163d00 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.962+0000 7f2788616640 1 -- 172.21.15.29:0/4112998168 >> 172.21.15.29:0/4112998168 conn(0x7f27800b1870 msgr2=0x7f27801550d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.962+0000 7f2788616640 1 -- 172.21.15.29:0/4112998168 shutdown_connections 2024-04-25T00:38:43.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:43.962+0000 7f2788616640 1 -- 172.21.15.29:0/4112998168 wait complete. 2024-04-25T00:38:43.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:43.983 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol5860/c0e50fc5-73bd-41b9-8e1b-3f3c8084454b) 2024-04-25T00:38:44.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:44.009 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol5860/c0e50fc5-73bd-41b9-8e1b-3f3c8084454b/.snap/fake) 2024-04-25T00:38:44.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:44.077 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-25T00:38:44.131 INFO:teuthology.orchestra.run.smithi029.stdout:cephfs_data 2024-04-25T00:38:44.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:44.132 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol5860/c0e50fc5-73bd-41b9-8e1b-3f3c8084454b) 2024-04-25T00:38:44.198 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:38:44.198 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol5860/.meta 2024-04-25T00:38:44.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:44.369 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp25200/subvol14261/d1941772-9e80-4658-bdf4-1031f0bf65ba) 2024-04-25T00:38:44.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:44.400 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp25200/subvol14261/d1941772-9e80-4658-bdf4-1031f0bf65ba/.snap/fake) 2024-04-25T00:38:44.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:44.476 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-25T00:38:44.528 INFO:teuthology.orchestra.run.smithi029.stdout:cephfs_data 2024-04-25T00:38:44.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:44.528 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp25200/subvol14261/d1941772-9e80-4658-bdf4-1031f0bf65ba) 2024-04-25T00:38:44.597 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:38:44.597 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp25200/subvol14261/.meta 2024-04-25T00:38:44.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:44.658 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol19552/d9d3b682-bda2-49e5-8373-9a70a669dad0) 2024-04-25T00:38:44.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:44.719 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-25T00:38:44.766 INFO:teuthology.orchestra.run.smithi029.stdout:cephfs_data 2024-04-25T00:38:44.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:44.767 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol19552/d9d3b682-bda2-49e5-8373-9a70a669dad0) 2024-04-25T00:38:44.832 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:38:44.833 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol19552/.meta 2024-04-25T00:38:44.891 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol5860 2024-04-25T00:38:45.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb3822af640 1 Processor -- start 2024-04-25T00:38:45.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb3822af640 1 -- start start 2024-04-25T00:38:45.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb3822af640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb37c15e200 0x7fb37c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb3822af640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb37c0cfc10 con 0x7fb37c0cf690 2024-04-25T00:38:45.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb3822af640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb37c0cfef0 con 0x7fb37c0d2c60 2024-04-25T00:38:45.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb3822af640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb37c0b38d0 con 0x7fb37c15e200 2024-04-25T00:38:45.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37b7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb37c0cf690 0x7fb37c0d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56316/0 (socket says 172.21.15.29:56316) 2024-04-25T00:38:45.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37b7fe640 1 -- 172.21.15.29:0/55474865 learned_addr learned my addr 172.21.15.29:0/55474865 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:45.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37bfff640 1 --2- 172.21.15.29:0/55474865 >> v2:172.21.15.157:3300/0 conn(0x7fb37c15e200 0x7fb37c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1799350858 0 0) 0x7fb37c0cfc10 con 0x7fb37c0cf690 2024-04-25T00:38:45.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb364003530 con 0x7fb37c0cf690 2024-04-25T00:38:45.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1092015491 0 0) 0x7fb364003530 con 0x7fb37c0cf690 2024-04-25T00:38:45.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 >> v2:172.21.15.157:3300/0 conn(0x7fb37c15e200 msgr2=0x7fb37c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:45.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 --2- 172.21.15.29:0/55474865 >> v2:172.21.15.157:3300/0 conn(0x7fb37c15e200 0x7fb37c0d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 >> v1:172.21.15.157:6789/0 conn(0x7fb37c0d2c60 legacy=0x7fb37c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c0b3b20 con 0x7fb37c0cf690 2024-04-25T00:38:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fb370001680 con 0x7fb37c0cf690 2024-04-25T00:38:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 >> v1:172.21.15.29:6789/0 conn(0x7fb37c0cf690 legacy=0x7fb37c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 --2- 172.21.15.29:0/55474865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3640039c0 0x7fb364003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 --2- 172.21.15.29:0/55474865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb364004810 0x7fb364004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 --2- 172.21.15.29:0/55474865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3640051d0 0x7fb36400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c0b38d0 con 0x7fb3640051d0 2024-04-25T00:38:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb36400c070 con 0x7fb364004810 2024-04-25T00:38:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb36400c270 con 0x7fb3640039c0 2024-04-25T00:38:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb38084c640 1 --2- 172.21.15.29:0/55474865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3640051d0 0x7fb36400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37bfff640 1 --2- 172.21.15.29:0/55474865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3640039c0 0x7fb364003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37b7fe640 1 --2- 172.21.15.29:0/55474865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb364004810 0x7fb364004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb38084c640 1 -- 172.21.15.29:0/55474865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3640039c0 msgr2=0x7fb364003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb38084c640 1 --2- 172.21.15.29:0/55474865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3640039c0 0x7fb364003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb38084c640 1 -- 172.21.15.29:0/55474865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb364004810 msgr2=0x7fb364004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb38084c640 1 --2- 172.21.15.29:0/55474865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb364004810 0x7fb364004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb38084c640 1 -- 172.21.15.29:0/55474865 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb364003530 con 0x7fb3640051d0 2024-04-25T00:38:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.070+0000 7fb37b7fe640 1 --2- 172.21.15.29:0/55474865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb364004810 0x7fb364004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:45.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb38084c640 1 -- 172.21.15.29:0/55474865 reap_dead start 2024-04-25T00:38:45.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb38084c640 1 --2- 172.21.15.29:0/55474865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3640051d0 0x7fb36400b9d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb374001e70 tx=0x7fb37401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6240833b69d0156 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:45.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374001a90 con 0x7fb3640051d0 2024-04-25T00:38:45.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb374003bc0 con 0x7fb3640051d0 2024-04-25T00:38:45.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb37affd640 1 -- 172.21.15.29:0/55474865 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374005020 con 0x7fb3640051d0 2024-04-25T00:38:45.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb3822af640 1 -- 172.21.15.29:0/55474865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3640051d0 msgr2=0x7fb36400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb3822af640 1 --2- 172.21.15.29:0/55474865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3640051d0 0x7fb36400b9d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb374001e70 tx=0x7fb37401c780 comp rx=0 tx=0).stop 2024-04-25T00:38:45.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb3822af640 1 -- 172.21.15.29:0/55474865 shutdown_connections 2024-04-25T00:38:45.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb3822af640 1 --2- 172.21.15.29:0/55474865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3640051d0 0x7fb36400b9d0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb3822af640 1 -- 172.21.15.29:0/55474865 >> 172.21.15.29:0/55474865 conn(0x7fb37c09d280 msgr2=0x7fb37c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb3822af640 1 -- 172.21.15.29:0/55474865 shutdown_connections 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb3822af640 1 -- 172.21.15.29:0/55474865 wait complete. 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb3822af640 1 Processor -- start 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.074+0000 7fb3822af640 1 -- start start 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb3822af640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3640051d0 0x7fb37c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb3822af640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb37c0b8ac0 0x7fb37c0b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb3822af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb37c1f4120 0x7fb37c1f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb3822af640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c15e760 con 0x7fb37c0b8ac0 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb3822af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c15e5e0 con 0x7fb37c1f4120 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb3822af640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb37c0d2550 con 0x7fb3640051d0 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb37b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb37c0b8ac0 0x7fb37c0b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb37b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb37c0b8ac0 0x7fb37c0b8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54504/0 (socket says 172.21.15.29:54504) 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb37b7fe640 1 -- 172.21.15.29:0/1473168951 learned_addr learned my addr 172.21.15.29:0/1473168951 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:45.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb38084c640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb37c1f4120 0x7fb37c1f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb37bfff640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3640051d0 0x7fb37c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb37b7fe640 1 -- 172.21.15.29:0/1473168951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3640051d0 msgr2=0x7fb37c0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb37b7fe640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3640051d0 0x7fb37c0ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb37b7fe640 1 -- 172.21.15.29:0/1473168951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb37c1f4120 msgr2=0x7fb37c1f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb37b7fe640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb37c1f4120 0x7fb37c1f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb37b7fe640 1 -- 172.21.15.29:0/1473168951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c1f75a0 con 0x7fb37c0b8ac0 2024-04-25T00:38:45.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb37bfff640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3640051d0 0x7fb37c0ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:45.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb37b7fe640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb37c0b8ac0 0x7fb37c0b8f40 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb37000d770 tx=0x7fb37000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:45.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb378ff9640 1 -- 172.21.15.29:0/1473168951 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370019070 con 0x7fb37c0b8ac0 2024-04-25T00:38:45.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb378ff9640 1 -- 172.21.15.29:0/1473168951 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3700074e0 con 0x7fb37c0b8ac0 2024-04-25T00:38:45.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb378ff9640 1 -- 172.21.15.29:0/1473168951 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370004a30 con 0x7fb37c0b8ac0 2024-04-25T00:38:45.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb3822af640 1 -- 172.21.15.29:0/1473168951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c1f7880 con 0x7fb37c0b8ac0 2024-04-25T00:38:45.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb3822af640 1 -- 172.21.15.29:0/1473168951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c1f4ae0 con 0x7fb37c0b8ac0 2024-04-25T00:38:45.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb3822af640 1 -- 172.21.15.29:0/1473168951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb37c069270 con 0x7fb37c0b8ac0 2024-04-25T00:38:45.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb378ff9640 1 -- 172.21.15.29:0/1473168951 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fb37001c070 con 0x7fb37c0b8ac0 2024-04-25T00:38:45.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb378ff9640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb368067ce0 0x7fb36806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb378ff9640 1 -- 172.21.15.29:0/1473168951 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fb370085ef0 con 0x7fb37c0b8ac0 2024-04-25T00:38:45.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb37bfff640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb368067ce0 0x7fb36806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.078+0000 7fb37bfff640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb368067ce0 0x7fb36806a1a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0ce010 tx=0x7fb36c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:45.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.082+0000 7fb378ff9640 1 -- 172.21.15.29:0/1473168951 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb370056340 con 0x7fb37c0b8ac0 2024-04-25T00:38:45.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.270+0000 7fb3822af640 1 -- 172.21.15.29:0/1473168951 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol5860", "target": ["mon-mgr", ""]}) -- 0x7fb37c1f8240 con 0x7fb368067ce0 2024-04-25T00:38:45.280 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:45.278+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:45.280 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:45.278+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:45.280 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:45.278+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:45.280 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:45.278+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:45.281 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:45.278+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:45.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.310+0000 7fb378ff9640 1 -- 172.21.15.29:0/1473168951 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+66 (secure 0 0 0) 0x7fb37c1f8240 con 0x7fb368067ce0 2024-04-25T00:38:45.314 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/_nogroup/subvol5860/c0e50fc5-73bd-41b9-8e1b-3f3c8084454b 2024-04-25T00:38:45.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.314+0000 7fb3822af640 1 -- 172.21.15.29:0/1473168951 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb368067ce0 msgr2=0x7fb36806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.314+0000 7fb3822af640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb368067ce0 0x7fb36806a1a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0ce010 tx=0x7fb36c008040 comp rx=0 tx=0).stop 2024-04-25T00:38:45.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.314+0000 7fb3822af640 1 -- 172.21.15.29:0/1473168951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb37c0b8ac0 msgr2=0x7fb37c0b8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.314+0000 7fb3822af640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb37c0b8ac0 0x7fb37c0b8f40 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb37000d770 tx=0x7fb37000dc30 comp rx=0 tx=0).stop 2024-04-25T00:38:45.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.314+0000 7fb3822af640 1 -- 172.21.15.29:0/1473168951 shutdown_connections 2024-04-25T00:38:45.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.314+0000 7fb3822af640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb368067ce0 0x7fb36806a1a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.314+0000 7fb3822af640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb37c1f4120 0x7fb37c1f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.314+0000 7fb3822af640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb37c0b8ac0 0x7fb37c0b8f40 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.314+0000 7fb3822af640 1 --2- 172.21.15.29:0/1473168951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3640051d0 0x7fb37c0ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.314+0000 7fb3822af640 1 -- 172.21.15.29:0/1473168951 >> 172.21.15.29:0/1473168951 conn(0x7fb37c09d280 msgr2=0x7fb37c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:45.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.314+0000 7fb3822af640 1 -- 172.21.15.29:0/1473168951 shutdown_connections 2024-04-25T00:38:45.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.314+0000 7fb3822af640 1 -- 172.21.15.29:0/1473168951 wait complete. 2024-04-25T00:38:45.334 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol14261 subvol_grp25200 2024-04-25T00:38:45.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9199eeb640 1 Processor -- start 2024-04-25T00:38:45.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9199eeb640 1 -- start start 2024-04-25T00:38:45.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9199eeb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91940c4d40 0x7f91940cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9199eeb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f91940b4440 0x7f91940c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9199eeb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91940cf6a0 con 0x7f9194166110 2024-04-25T00:38:45.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9199eeb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f91940b15b0 con 0x7f91940b4440 2024-04-25T00:38:45.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9199eeb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f91940cc3c0 con 0x7f91940c4d40 2024-04-25T00:38:45.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9192ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f91940b4440 0x7f91940c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9192ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f91940b4440 0x7f91940c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54520/0 (socket says 172.21.15.29:54520) 2024-04-25T00:38:45.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9192ffd640 1 -- 172.21.15.29:0/35126386 learned_addr learned my addr 172.21.15.29:0/35126386 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:45.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9192ffd640 1 -- 172.21.15.29:0/35126386 >> v2:172.21.15.157:3300/0 conn(0x7f91940c4d40 msgr2=0x7f91940cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:45.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9192ffd640 1 --2- 172.21.15.29:0/35126386 >> v2:172.21.15.157:3300/0 conn(0x7f91940c4d40 0x7f91940cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9192ffd640 1 -- 172.21.15.29:0/35126386 >> v1:172.21.15.157:6789/0 conn(0x7f9194166110 legacy=0x7f9194166540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:45.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.458+0000 7f9192ffd640 1 -- 172.21.15.29:0/35126386 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91940c5660 con 0x7f91940b4440 2024-04-25T00:38:45.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f9192ffd640 1 --2- 172.21.15.29:0/35126386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91940b4440 0x7f91940c5120 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f91940b49e0 tx=0x7f918801c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0dc8acc9b00ba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:45.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f91927fc640 1 -- 172.21.15.29:0/35126386 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91880a55e0 con 0x7f91940b4440 2024-04-25T00:38:45.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f91927fc640 1 -- 172.21.15.29:0/35126386 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91880a5780 con 0x7f91940b4440 2024-04-25T00:38:45.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f91927fc640 1 -- 172.21.15.29:0/35126386 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91880a5b50 con 0x7f91940b4440 2024-04-25T00:38:45.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f9199eeb640 1 -- 172.21.15.29:0/35126386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91940b4440 msgr2=0x7f91940c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f9199eeb640 1 --2- 172.21.15.29:0/35126386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91940b4440 0x7f91940c5120 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f91940b49e0 tx=0x7f918801c720 comp rx=0 tx=0).stop 2024-04-25T00:38:45.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f9199eeb640 1 -- 172.21.15.29:0/35126386 shutdown_connections 2024-04-25T00:38:45.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f9199eeb640 1 --2- 172.21.15.29:0/35126386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91940b4440 0x7f91940c5120 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f9199eeb640 1 --2- 172.21.15.29:0/35126386 >> v2:172.21.15.157:3300/0 conn(0x7f91940c4d40 0x7f91940cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f9199eeb640 1 -- 172.21.15.29:0/35126386 >> 172.21.15.29:0/35126386 conn(0x7f91940b3920 msgr2=0x7f91940b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:45.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f9199eeb640 1 -- 172.21.15.29:0/35126386 shutdown_connections 2024-04-25T00:38:45.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f9199eeb640 1 -- 172.21.15.29:0/35126386 wait complete. 2024-04-25T00:38:45.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f9199eeb640 1 Processor -- start 2024-04-25T00:38:45.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.462+0000 7f9199eeb640 1 -- start start 2024-04-25T00:38:45.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9199eeb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91940b4440 0x7f91941653f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9199eeb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91940c4d40 0x7f9194161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9199eeb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9194166110 0x7f9194161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9199eeb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f91940cc610 con 0x7f9194166110 2024-04-25T00:38:45.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9199eeb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f919406f780 con 0x7f91940c4d40 2024-04-25T00:38:45.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9199eeb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f919416fc60 con 0x7f91940b4440 2024-04-25T00:38:45.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9193fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9194166110 0x7f9194161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9193fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9194166110 0x7f9194161570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54534/0 (socket says 172.21.15.29:54534) 2024-04-25T00:38:45.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9193fff640 1 -- 172.21.15.29:0/3650435432 learned_addr learned my addr 172.21.15.29:0/3650435432 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:45.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9192ffd640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91940c4d40 0x7f9194161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f91937fe640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91940b4440 0x7f91941653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9193fff640 1 -- 172.21.15.29:0/3650435432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91940b4440 msgr2=0x7f91941653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9193fff640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91940b4440 0x7f91941653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9193fff640 1 -- 172.21.15.29:0/3650435432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91940c4d40 msgr2=0x7f9194161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9193fff640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91940c4d40 0x7f9194161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9193fff640 1 -- 172.21.15.29:0/3650435432 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919415c410 con 0x7f9194166110 2024-04-25T00:38:45.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9192ffd640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91940c4d40 0x7f9194161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:45.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9193fff640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9194166110 0x7f9194161570 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f918c00b440 tx=0x7f918c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:45.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f917bfff640 1 -- 172.21.15.29:0/3650435432 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c015020 con 0x7f9194166110 2024-04-25T00:38:45.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f917bfff640 1 -- 172.21.15.29:0/3650435432 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f918c011850 con 0x7f9194166110 2024-04-25T00:38:45.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9199eeb640 1 -- 172.21.15.29:0/3650435432 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91940aee50 con 0x7f9194166110 2024-04-25T00:38:45.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f917bfff640 1 -- 172.21.15.29:0/3650435432 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c011c40 con 0x7f9194166110 2024-04-25T00:38:45.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9199eeb640 1 -- 172.21.15.29:0/3650435432 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91940c9530 con 0x7f9194166110 2024-04-25T00:38:45.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f9199eeb640 1 -- 172.21.15.29:0/3650435432 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91940692e0 con 0x7f9194166110 2024-04-25T00:38:45.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f917bfff640 1 -- 172.21.15.29:0/3650435432 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f918c018020 con 0x7f9194166110 2024-04-25T00:38:45.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f917bfff640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9180067ce0 0x7f918006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f91937fe640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9180067ce0 0x7f918006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f917bfff640 1 -- 172.21.15.29:0/3650435432 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f918c085d00 con 0x7f9194166110 2024-04-25T00:38:45.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.466+0000 7f91937fe640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9180067ce0 0x7f918006a1a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9184006fd0 tx=0x7f9184008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:45.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.470+0000 7f917bfff640 1 -- 172.21.15.29:0/3650435432 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f918c01f070 con 0x7f9194166110 2024-04-25T00:38:45.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.658+0000 7f9199eeb640 1 -- 172.21.15.29:0/3650435432 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol14261", "group_name": "subvol_grp25200", "target": ["mon-mgr", ""]}) -- 0x7f9194074f90 con 0x7f9180067ce0 2024-04-25T00:38:45.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.702+0000 7f917bfff640 1 -- 172.21.15.29:0/3650435432 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7f9194074f90 con 0x7f9180067ce0 2024-04-25T00:38:45.706 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/subvol_grp25200/subvol14261/d1941772-9e80-4658-bdf4-1031f0bf65ba 2024-04-25T00:38:45.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.706+0000 7f9199eeb640 1 -- 172.21.15.29:0/3650435432 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9180067ce0 msgr2=0x7f918006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.706+0000 7f9199eeb640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9180067ce0 0x7f918006a1a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9184006fd0 tx=0x7f9184008040 comp rx=0 tx=0).stop 2024-04-25T00:38:45.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.706+0000 7f9199eeb640 1 -- 172.21.15.29:0/3650435432 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9194166110 msgr2=0x7f9194161570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.706+0000 7f9199eeb640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9194166110 0x7f9194161570 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f918c00b440 tx=0x7f918c00b900 comp rx=0 tx=0).stop 2024-04-25T00:38:45.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.706+0000 7f9199eeb640 1 -- 172.21.15.29:0/3650435432 shutdown_connections 2024-04-25T00:38:45.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.706+0000 7f9199eeb640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9180067ce0 0x7f918006a1a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.706+0000 7f9199eeb640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9194166110 0x7f9194161570 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.706+0000 7f9199eeb640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91940c4d40 0x7f9194161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.706+0000 7f9199eeb640 1 --2- 172.21.15.29:0/3650435432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91940b4440 0x7f91941653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.706+0000 7f9199eeb640 1 -- 172.21.15.29:0/3650435432 >> 172.21.15.29:0/3650435432 conn(0x7f91940b3920 msgr2=0x7f91940d1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:45.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.706+0000 7f9199eeb640 1 -- 172.21.15.29:0/3650435432 shutdown_connections 2024-04-25T00:38:45.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.706+0000 7f9199eeb640 1 -- 172.21.15.29:0/3650435432 wait complete. 2024-04-25T00:38:45.727 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol19552 2024-04-25T00:38:45.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61eb72640 1 Processor -- start 2024-04-25T00:38:45.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61eb72640 1 -- start start 2024-04-25T00:38:45.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61eb72640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e200 0x7ff6180d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61eb72640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6180cfc10 con 0x7ff6180d2c60 2024-04-25T00:38:45.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61eb72640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6180cfef0 con 0x7ff6180cf690 2024-04-25T00:38:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61eb72640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff6180b38d0 con 0x7ff61815e200 2024-04-25T00:38:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61d10f640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff6180d2c60 0x7ff6180b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56328/0 (socket says 172.21.15.29:56328) 2024-04-25T00:38:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61d10f640 1 -- 172.21.15.29:0/4074261546 learned_addr learned my addr 172.21.15.29:0/4074261546 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61c90e640 1 --2- 172.21.15.29:0/4074261546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e200 0x7ff6180d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2528954103 0 0) 0x7ff6180cfc10 con 0x7ff6180d2c60 2024-04-25T00:38:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff608003530 con 0x7ff6180d2c60 2024-04-25T00:38:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1901760499 0 0) 0x7ff608003530 con 0x7ff6180d2c60 2024-04-25T00:38:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e200 msgr2=0x7ff6180d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 --2- 172.21.15.29:0/4074261546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e200 0x7ff6180d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 >> v1:172.21.15.157:6789/0 conn(0x7ff6180cf690 legacy=0x7ff6180d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6180b3b20 con 0x7ff6180d2c60 2024-04-25T00:38:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ff610001a00 con 0x7ff6180d2c60 2024-04-25T00:38:45.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 >> v1:172.21.15.29:6789/0 conn(0x7ff6180d2c60 legacy=0x7ff6180b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 --2- 172.21.15.29:0/4074261546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6080039f0 0x7ff608003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 --2- 172.21.15.29:0/4074261546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e200 0x7ff608004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 --2- 172.21.15.29:0/4074261546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff608004d50 0x7ff6080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff6180b38d0 con 0x7ff608004d50 2024-04-25T00:38:45.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff60800ba80 con 0x7ff6080039f0 2024-04-25T00:38:45.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff60800bc80 con 0x7ff61815e200 2024-04-25T00:38:45.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61d10f640 1 --2- 172.21.15.29:0/4074261546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff608004d50 0x7ff6080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61c90e640 1 --2- 172.21.15.29:0/4074261546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6080039f0 0x7ff608003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff617fff640 1 --2- 172.21.15.29:0/4074261546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e200 0x7ff608004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61d10f640 1 -- 172.21.15.29:0/4074261546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e200 msgr2=0x7ff608004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:45.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61d10f640 1 --2- 172.21.15.29:0/4074261546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e200 0x7ff608004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61d10f640 1 -- 172.21.15.29:0/4074261546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6080039f0 msgr2=0x7ff608003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:45.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61d10f640 1 --2- 172.21.15.29:0/4074261546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6080039f0 0x7ff608003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.858+0000 7ff61d10f640 1 -- 172.21.15.29:0/4074261546 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff608003530 con 0x7ff608004d50 2024-04-25T00:38:45.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.862+0000 7ff61d10f640 1 --2- 172.21.15.29:0/4074261546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff608004d50 0x7ff6080051d0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7ff610002240 tx=0x7ff61001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2484af54f413d654 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:45.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.862+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61000a4a0 con 0x7ff608004d50 2024-04-25T00:38:45.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.862+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6100ad650 con 0x7ff608004d50 2024-04-25T00:38:45.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.862+0000 7ff6177fe640 1 -- 172.21.15.29:0/4074261546 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6100ada40 con 0x7ff608004d50 2024-04-25T00:38:45.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.862+0000 7ff61eb72640 1 -- 172.21.15.29:0/4074261546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff608004d50 msgr2=0x7ff6080051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.862+0000 7ff61eb72640 1 --2- 172.21.15.29:0/4074261546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff608004d50 0x7ff6080051d0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7ff610002240 tx=0x7ff61001d340 comp rx=0 tx=0).stop 2024-04-25T00:38:45.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.862+0000 7ff61d10f640 1 -- 172.21.15.29:0/4074261546 reap_dead start 2024-04-25T00:38:45.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.862+0000 7ff61eb72640 1 -- 172.21.15.29:0/4074261546 shutdown_connections 2024-04-25T00:38:45.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.862+0000 7ff61eb72640 1 -- 172.21.15.29:0/4074261546 >> 172.21.15.29:0/4074261546 conn(0x7ff61809d280 msgr2=0x7ff6180d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:45.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.862+0000 7ff61eb72640 1 -- 172.21.15.29:0/4074261546 shutdown_connections 2024-04-25T00:38:45.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.862+0000 7ff61eb72640 1 -- 172.21.15.29:0/4074261546 wait complete. 2024-04-25T00:38:45.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.862+0000 7ff61eb72640 1 Processor -- start 2024-04-25T00:38:45.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61eb72640 1 -- start start 2024-04-25T00:38:45.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61eb72640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff608004d50 0x7ff618161290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61eb72640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6180c7aa0 0x7ff618155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61eb72640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6180b4d10 0x7ff6180b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61eb72640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff61815e760 con 0x7ff6180c7aa0 2024-04-25T00:38:45.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61eb72640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff61815e5e0 con 0x7ff6180b4d10 2024-04-25T00:38:45.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61eb72640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff6180d47f0 con 0x7ff608004d50 2024-04-25T00:38:45.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff617fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6180c7aa0 0x7ff618155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff617fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6180c7aa0 0x7ff618155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54556/0 (socket says 172.21.15.29:54556) 2024-04-25T00:38:45.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff617fff640 1 -- 172.21.15.29:0/2420904741 learned_addr learned my addr 172.21.15.29:0/2420904741 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:45.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61d10f640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6180b4d10 0x7ff6180b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61c90e640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff608004d50 0x7ff618161290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff617fff640 1 -- 172.21.15.29:0/2420904741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff608004d50 msgr2=0x7ff618161290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff617fff640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff608004d50 0x7ff618161290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff617fff640 1 -- 172.21.15.29:0/2420904741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6180b4d10 msgr2=0x7ff6180b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:45.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff617fff640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6180b4d10 0x7ff6180b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:45.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff617fff640 1 -- 172.21.15.29:0/2420904741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff618156340 con 0x7ff6180c7aa0 2024-04-25T00:38:45.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61c90e640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff608004d50 0x7ff618161290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:45.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61d10f640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6180b4d10 0x7ff6180b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:45.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff617fff640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6180c7aa0 0x7ff618155b40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7ff60400e440 tx=0x7ff60400e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:45.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff6157fa640 1 -- 172.21.15.29:0/2420904741 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6040052e0 con 0x7ff6180c7aa0 2024-04-25T00:38:45.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff6157fa640 1 -- 172.21.15.29:0/2420904741 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff604018070 con 0x7ff6180c7aa0 2024-04-25T00:38:45.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff6157fa640 1 -- 172.21.15.29:0/2420904741 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff604005480 con 0x7ff6180c7aa0 2024-04-25T00:38:45.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61eb72640 1 -- 172.21.15.29:0/2420904741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61815f2c0 con 0x7ff6180c7aa0 2024-04-25T00:38:45.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61eb72640 1 -- 172.21.15.29:0/2420904741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6180c81f0 con 0x7ff6180c7aa0 2024-04-25T00:38:45.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff61eb72640 1 -- 172.21.15.29:0/2420904741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6180b56d0 con 0x7ff6180c7aa0 2024-04-25T00:38:45.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff6157fa640 1 -- 172.21.15.29:0/2420904741 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7ff604003840 con 0x7ff6180c7aa0 2024-04-25T00:38:45.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.866+0000 7ff6157fa640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5fc067cb0 0x7ff5fc06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:45.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.870+0000 7ff6157fa640 1 -- 172.21.15.29:0/2420904741 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7ff604085fd0 con 0x7ff6180c7aa0 2024-04-25T00:38:45.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.870+0000 7ff61c90e640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5fc067cb0 0x7ff5fc06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:45.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.870+0000 7ff61c90e640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5fc067cb0 0x7ff5fc06a170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009af0 tx=0x7ff60c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:45.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:45.874+0000 7ff6157fa640 1 -- 172.21.15.29:0/2420904741 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff618156340 con 0x7ff6180c7aa0 2024-04-25T00:38:46.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.070+0000 7ff61eb72640 1 -- 172.21.15.29:0/2420904741 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol19552", "target": ["mon-mgr", ""]}) -- 0x7ff618069270 con 0x7ff5fc067cb0 2024-04-25T00:38:46.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.102+0000 7ff6157fa640 1 -- 172.21.15.29:0/2420904741 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+144 (secure 0 0 0) 0x7ff618069270 con 0x7ff5fc067cb0 2024-04-25T00:38:46.107 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:38:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "status": { 2024-04-25T00:38:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "pending", 2024-04-25T00:38:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "source": { 2024-04-25T00:38:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "volume": "fake", 2024-04-25T00:38:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "subvolume": "fake", 2024-04-25T00:38:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "snapshot": "fake" 2024-04-25T00:38:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:38:46.108 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:38:46.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.106+0000 7ff61eb72640 1 -- 172.21.15.29:0/2420904741 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5fc067cb0 msgr2=0x7ff5fc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.106+0000 7ff61eb72640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5fc067cb0 0x7ff5fc06a170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009af0 tx=0x7ff60c008040 comp rx=0 tx=0).stop 2024-04-25T00:38:46.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.106+0000 7ff61eb72640 1 -- 172.21.15.29:0/2420904741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6180c7aa0 msgr2=0x7ff618155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.106+0000 7ff61eb72640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6180c7aa0 0x7ff618155b40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7ff60400e440 tx=0x7ff60400e900 comp rx=0 tx=0).stop 2024-04-25T00:38:46.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.106+0000 7ff61eb72640 1 -- 172.21.15.29:0/2420904741 shutdown_connections 2024-04-25T00:38:46.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.106+0000 7ff61eb72640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5fc067cb0 0x7ff5fc06a170 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.106+0000 7ff61eb72640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6180b4d10 0x7ff6180b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.106+0000 7ff61eb72640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6180c7aa0 0x7ff618155b40 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.106+0000 7ff61eb72640 1 --2- 172.21.15.29:0/2420904741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff608004d50 0x7ff618161290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.106+0000 7ff61eb72640 1 -- 172.21.15.29:0/2420904741 >> 172.21.15.29:0/2420904741 conn(0x7ff61809d280 msgr2=0x7ff61815d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:46.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.106+0000 7ff61eb72640 1 -- 172.21.15.29:0/2420904741 shutdown_connections 2024-04-25T00:38:46.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.106+0000 7ff61eb72640 1 -- 172.21.15.29:0/2420904741 wait complete. 2024-04-25T00:38:46.128 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol5860 fake 2024-04-25T00:38:46.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.258+0000 7fc00ce44640 1 Processor -- start 2024-04-25T00:38:46.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.258+0000 7fc00ce44640 1 -- start start 2024-04-25T00:38:46.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.258+0000 7fc00ce44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0081456d0 0x7fc0080b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:46.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.258+0000 7fc00ce44640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc0080b3b50 0x7fc0080cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:46.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.258+0000 7fc00ce44640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0080b4380 con 0x7fc00806cd40 2024-04-25T00:38:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.258+0000 7fc00ce44640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc0080b40a0 con 0x7fc0080b3b50 2024-04-25T00:38:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.258+0000 7fc00ce44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc0080ce480 con 0x7fc0081456d0 2024-04-25T00:38:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.258+0000 7fc005d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc0080b3b50 0x7fc0080cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.258+0000 7fc005d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc0080b3b50 0x7fc0080cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54560/0 (socket says 172.21.15.29:54560) 2024-04-25T00:38:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.258+0000 7fc005d9b640 1 -- 172.21.15.29:0/1313486900 learned_addr learned my addr 172.21.15.29:0/1313486900 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc00659c640 1 --2- 172.21.15.29:0/1313486900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0081456d0 0x7fc0080b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc005d9b640 1 -- 172.21.15.29:0/1313486900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0081456d0 msgr2=0x7fc0080b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc005d9b640 1 --2- 172.21.15.29:0/1313486900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0081456d0 0x7fc0080b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc005d9b640 1 -- 172.21.15.29:0/1313486900 >> v1:172.21.15.157:6789/0 conn(0x7fc00806cd40 legacy=0x7fc00806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc005d9b640 1 -- 172.21.15.29:0/1313486900 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0080ce760 con 0x7fc0080b3b50 2024-04-25T00:38:46.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc00659c640 1 --2- 172.21.15.29:0/1313486900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0081456d0 0x7fc0080b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:46.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc005d9b640 1 --2- 172.21.15.29:0/1313486900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0080b3b50 0x7fc0080cef60 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009870 tx=0x7fbffc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f55e4cea2eb89a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:46.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc00559a640 1 -- 172.21.15.29:0/1313486900 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0ab5f0 con 0x7fc0080b3b50 2024-04-25T00:38:46.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc00559a640 1 -- 172.21.15.29:0/1313486900 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc0b0070 con 0x7fc0080b3b50 2024-04-25T00:38:46.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc00559a640 1 -- 172.21.15.29:0/1313486900 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0ab9b0 con 0x7fc0080b3b50 2024-04-25T00:38:46.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc00ce44640 1 -- 172.21.15.29:0/1313486900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0080b3b50 msgr2=0x7fc0080cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc00ce44640 1 --2- 172.21.15.29:0/1313486900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0080b3b50 0x7fc0080cef60 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009870 tx=0x7fbffc01c560 comp rx=0 tx=0).stop 2024-04-25T00:38:46.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc00ce44640 1 -- 172.21.15.29:0/1313486900 shutdown_connections 2024-04-25T00:38:46.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc00ce44640 1 --2- 172.21.15.29:0/1313486900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0080b3b50 0x7fc0080cef60 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc00ce44640 1 --2- 172.21.15.29:0/1313486900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0081456d0 0x7fc0080b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc00ce44640 1 -- 172.21.15.29:0/1313486900 >> 172.21.15.29:0/1313486900 conn(0x7fc0080b1870 msgr2=0x7fc0080b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:46.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc00ce44640 1 -- 172.21.15.29:0/1313486900 shutdown_connections 2024-04-25T00:38:46.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.262+0000 7fc00ce44640 1 -- 172.21.15.29:0/1313486900 wait complete. 2024-04-25T00:38:46.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00ce44640 1 Processor -- start 2024-04-25T00:38:46.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00ce44640 1 -- start start 2024-04-25T00:38:46.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00ce44640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc00806cd40 0x7fc00813ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:46.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00ce44640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0080b3b50 0x7fc00815d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:46.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00ce44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0081456d0 0x7fc008164f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:46.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00ce44640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc0080cf620 con 0x7fc00806cd40 2024-04-25T00:38:46.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00ce44640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc0080cf4a0 con 0x7fc0080b3b50 2024-04-25T00:38:46.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00ce44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc008168b70 con 0x7fc0081456d0 2024-04-25T00:38:46.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc00806cd40 0x7fc00813ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:46.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc00806cd40 0x7fc00813ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54576/0 (socket says 172.21.15.29:54576) 2024-04-25T00:38:46.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00659c640 1 -- 172.21.15.29:0/1918713833 learned_addr learned my addr 172.21.15.29:0/1918713833 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:46.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc005d9b640 1 --2- 172.21.15.29:0/1918713833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0080b3b50 0x7fc00815d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:46.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00659c640 1 -- 172.21.15.29:0/1918713833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0081456d0 msgr2=0x7fc008164f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00659c640 1 --2- 172.21.15.29:0/1918713833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0081456d0 0x7fc008164f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00659c640 1 -- 172.21.15.29:0/1918713833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0080b3b50 msgr2=0x7fc00815d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00659c640 1 --2- 172.21.15.29:0/1918713833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0080b3b50 0x7fc00815d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00659c640 1 -- 172.21.15.29:0/1918713833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00813bf70 con 0x7fc00806cd40 2024-04-25T00:38:46.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00659c640 1 --2- 172.21.15.29:0/1918713833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc00806cd40 0x7fc00813ba30 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fbff800e810 tx=0x7fbff800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:46.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fbff6ffd640 1 -- 172.21.15.29:0/1918713833 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff8017070 con 0x7fc00806cd40 2024-04-25T00:38:46.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fbff6ffd640 1 -- 172.21.15.29:0/1918713833 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff800c8d0 con 0x7fc00806cd40 2024-04-25T00:38:46.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fbff6ffd640 1 -- 172.21.15.29:0/1918713833 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff800f040 con 0x7fc00806cd40 2024-04-25T00:38:46.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00ce44640 1 -- 172.21.15.29:0/1918713833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00815e000 con 0x7fc00806cd40 2024-04-25T00:38:46.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00ce44640 1 -- 172.21.15.29:0/1918713833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00815e5c0 con 0x7fc00806cd40 2024-04-25T00:38:46.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fc00ce44640 1 -- 172.21.15.29:0/1918713833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0080692e0 con 0x7fc00806cd40 2024-04-25T00:38:46.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.266+0000 7fbff6ffd640 1 -- 172.21.15.29:0/1918713833 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fbff801a070 con 0x7fc00806cd40 2024-04-25T00:38:46.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.270+0000 7fbff6ffd640 1 --2- 172.21.15.29:0/1918713833 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbfe8067cb0 0x7fbfe806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:46.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.270+0000 7fbff6ffd640 1 -- 172.21.15.29:0/1918713833 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fbff80860f0 con 0x7fc00806cd40 2024-04-25T00:38:46.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.270+0000 7fc005d9b640 1 --2- 172.21.15.29:0/1918713833 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbfe8067cb0 0x7fbfe806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:46.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.270+0000 7fc005d9b640 1 --2- 172.21.15.29:0/1918713833 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbfe8067cb0 0x7fbfe806a170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0af000 tx=0x7fbffc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:46.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.274+0000 7fbff6ffd640 1 -- 172.21.15.29:0/1918713833 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbff8056540 con 0x7fc00806cd40 2024-04-25T00:38:46.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.442+0000 7fbff6ffd640 1 -- 172.21.15.29:0/1918713833 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 20) ==== 87853+0+0 (secure 0 0 0) 0x7fbff80524b0 con 0x7fc00806cd40 2024-04-25T00:38:46.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.466+0000 7fc00ce44640 1 -- 172.21.15.29:0/1918713833 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol5860", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7fc0080b4de0 con 0x7fbfe8067cb0 2024-04-25T00:38:46.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.530+0000 7fbff6ffd640 1 -- 172.21.15.29:0/1918713833 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc0080b4de0 con 0x7fbfe8067cb0 2024-04-25T00:38:46.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.530+0000 7fc00ce44640 1 -- 172.21.15.29:0/1918713833 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbfe8067cb0 msgr2=0x7fbfe806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.530+0000 7fc00ce44640 1 --2- 172.21.15.29:0/1918713833 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbfe8067cb0 0x7fbfe806a170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0af000 tx=0x7fbffc008040 comp rx=0 tx=0).stop 2024-04-25T00:38:46.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.530+0000 7fc00ce44640 1 -- 172.21.15.29:0/1918713833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc00806cd40 msgr2=0x7fc00813ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.530+0000 7fc00ce44640 1 --2- 172.21.15.29:0/1918713833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc00806cd40 0x7fc00813ba30 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fbff800e810 tx=0x7fbff800ecd0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.534+0000 7fc00ce44640 1 -- 172.21.15.29:0/1918713833 shutdown_connections 2024-04-25T00:38:46.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.534+0000 7fc00ce44640 1 --2- 172.21.15.29:0/1918713833 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbfe8067cb0 0x7fbfe806a170 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.534+0000 7fc00ce44640 1 --2- 172.21.15.29:0/1918713833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0081456d0 0x7fc008164f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.534+0000 7fc00ce44640 1 --2- 172.21.15.29:0/1918713833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0080b3b50 0x7fc00815d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.534+0000 7fc00ce44640 1 --2- 172.21.15.29:0/1918713833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc00806cd40 0x7fc00813ba30 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.534+0000 7fc00ce44640 1 -- 172.21.15.29:0/1918713833 >> 172.21.15.29:0/1918713833 conn(0x7fc0080b1870 msgr2=0x7fc008066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.534+0000 7fc00ce44640 1 -- 172.21.15.29:0/1918713833 shutdown_connections 2024-04-25T00:38:46.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.534+0000 7fc00ce44640 1 -- 172.21.15.29:0/1918713833 wait complete. 2024-04-25T00:38:46.553 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol14261 fake subvol_grp25200 2024-04-25T00:38:46.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a6af1e640 1 Processor -- start 2024-04-25T00:38:46.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a6af1e640 1 -- start start 2024-04-25T00:38:46.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a6af1e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0a640b3140 0x7f0a6406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:46.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a6af1e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a640b4b10 0x7f0a640b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:46.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a6af1e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a640b41e0 con 0x7f0a640b3a60 2024-04-25T00:38:46.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a6af1e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f0a6406d960 con 0x7f0a640b3140 2024-04-25T00:38:46.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a6af1e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6406dc40 con 0x7f0a640b4b10 2024-04-25T00:38:46.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a68cba640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0a640b3140 0x7f0a6406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:46.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a68cba640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0a640b3140 0x7f0a6406e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54590/0 (socket says 172.21.15.29:54590) 2024-04-25T00:38:46.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a68cba640 1 -- 172.21.15.29:0/1048193359 learned_addr learned my addr 172.21.15.29:0/1048193359 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:46.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a63fff640 1 --2- 172.21.15.29:0/1048193359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a640b4b10 0x7f0a640b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:46.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a68cba640 1 -- 172.21.15.29:0/1048193359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a640b4b10 msgr2=0x7f0a640b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:46.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a68cba640 1 --2- 172.21.15.29:0/1048193359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a640b4b10 0x7f0a640b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a68cba640 1 -- 172.21.15.29:0/1048193359 >> v1:172.21.15.29:6789/0 conn(0x7f0a640b3a60 legacy=0x7f0a6415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a68cba640 1 -- 172.21.15.29:0/1048193359 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6406df20 con 0x7f0a640b3140 2024-04-25T00:38:46.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.682+0000 7f0a637fe640 1 -- 172.21.15.29:0/1048193359 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 768585956 0 0) 0x7f0a640b41e0 con 0x7f0a640b3a60 2024-04-25T00:38:46.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a68cba640 1 --2- 172.21.15.29:0/1048193359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a640b3140 0x7f0a6406e440 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009870 tx=0x7f0a5401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9514163a5f6e4c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:46.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a637fe640 1 -- 172.21.15.29:0/1048193359 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a540b0070 con 0x7f0a640b3140 2024-04-25T00:38:46.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a637fe640 1 -- 172.21.15.29:0/1048193359 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a540ab600 con 0x7f0a640b3140 2024-04-25T00:38:46.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a637fe640 1 -- 172.21.15.29:0/1048193359 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a540ab9d0 con 0x7f0a640b3140 2024-04-25T00:38:46.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 -- 172.21.15.29:0/1048193359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a640b3140 msgr2=0x7f0a6406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 --2- 172.21.15.29:0/1048193359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a640b3140 0x7f0a6406e440 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009870 tx=0x7f0a5401c560 comp rx=0 tx=0).stop 2024-04-25T00:38:46.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 -- 172.21.15.29:0/1048193359 shutdown_connections 2024-04-25T00:38:46.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 --2- 172.21.15.29:0/1048193359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a640b4b10 0x7f0a640b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 --2- 172.21.15.29:0/1048193359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a640b3140 0x7f0a6406e440 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 -- 172.21.15.29:0/1048193359 >> 172.21.15.29:0/1048193359 conn(0x7f0a640b1870 msgr2=0x7f0a640b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:46.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 -- 172.21.15.29:0/1048193359 shutdown_connections 2024-04-25T00:38:46.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 -- 172.21.15.29:0/1048193359 wait complete. 2024-04-25T00:38:46.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 Processor -- start 2024-04-25T00:38:46.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 -- start start 2024-04-25T00:38:46.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a640b3140 0x7f0a64153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:46.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a640b3a60 0x7f0a64154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:46.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a640b4b10 0x7f0a6413c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:46.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0a64168b70 con 0x7f0a640b3a60 2024-04-25T00:38:46.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641689f0 con 0x7f0a640b3140 2024-04-25T00:38:46.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a6af1e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0a64168cf0 con 0x7f0a640b4b10 2024-04-25T00:38:46.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a63fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a640b3a60 0x7f0a64154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:46.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a63fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a640b3a60 0x7f0a64154120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54596/0 (socket says 172.21.15.29:54596) 2024-04-25T00:38:46.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.686+0000 7f0a63fff640 1 -- 172.21.15.29:0/3509210448 learned_addr learned my addr 172.21.15.29:0/3509210448 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a68cba640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a640b3140 0x7f0a64153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a694bb640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a640b4b10 0x7f0a6413c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a63fff640 1 -- 172.21.15.29:0/3509210448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a640b4b10 msgr2=0x7f0a6413c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a63fff640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a640b4b10 0x7f0a6413c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a63fff640 1 -- 172.21.15.29:0/3509210448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a640b3140 msgr2=0x7f0a64153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a63fff640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a640b3140 0x7f0a64153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a63fff640 1 -- 172.21.15.29:0/3509210448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a64155480 con 0x7f0a640b3a60 2024-04-25T00:38:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a694bb640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a640b4b10 0x7f0a6413c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a63fff640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a640b3a60 0x7f0a64154120 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800d620 tx=0x7f0a5800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a617fa640 1 -- 172.21.15.29:0/3509210448 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5800b620 con 0x7f0a640b3a60 2024-04-25T00:38:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a617fa640 1 -- 172.21.15.29:0/3509210448 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a58019070 con 0x7f0a640b3a60 2024-04-25T00:38:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a617fa640 1 -- 172.21.15.29:0/3509210448 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5800ba20 con 0x7f0a640b3a60 2024-04-25T00:38:46.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a6af1e640 1 -- 172.21.15.29:0/3509210448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a64154be0 con 0x7f0a640b3a60 2024-04-25T00:38:46.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a6af1e640 1 -- 172.21.15.29:0/3509210448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a641551a0 con 0x7f0a640b3a60 2024-04-25T00:38:46.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a6af1e640 1 -- 172.21.15.29:0/3509210448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a64069270 con 0x7f0a640b3a60 2024-04-25T00:38:46.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a617fa640 1 -- 172.21.15.29:0/3509210448 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 20) ==== 87853+0+0 (secure 0 0 0) 0x7f0a5800bbc0 con 0x7f0a640b3a60 2024-04-25T00:38:46.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a617fa640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0a50067d40 0x7f0a5006a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:46.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a68cba640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0a50067d40 0x7f0a5006a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:46.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a617fa640 1 -- 172.21.15.29:0/3509210448 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f0a58086710 con 0x7f0a640b3a60 2024-04-25T00:38:46.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.690+0000 7f0a68cba640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0a50067d40 0x7f0a5006a200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a540ad000 tx=0x7f0a540a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:46.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.694+0000 7f0a617fa640 1 -- 172.21.15.29:0/3509210448 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0a64155480 con 0x7f0a640b3a60 2024-04-25T00:38:46.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.890+0000 7f0a6af1e640 1 -- 172.21.15.29:0/3509210448 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol14261", "snap_name": "fake", "group_name": "subvol_grp25200", "target": ["mon-mgr", ""]}) -- 0x7f0a6415c2e0 con 0x7f0a50067d40 2024-04-25T00:38:46.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.926+0000 7f0a617fa640 1 -- 172.21.15.29:0/3509210448 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0a6415c2e0 con 0x7f0a50067d40 2024-04-25T00:38:46.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.930+0000 7f0a6af1e640 1 -- 172.21.15.29:0/3509210448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0a50067d40 msgr2=0x7f0a5006a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.930+0000 7f0a6af1e640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0a50067d40 0x7f0a5006a200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a540ad000 tx=0x7f0a540a6000 comp rx=0 tx=0).stop 2024-04-25T00:38:46.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.930+0000 7f0a6af1e640 1 -- 172.21.15.29:0/3509210448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a640b3a60 msgr2=0x7f0a64154120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:46.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.930+0000 7f0a6af1e640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a640b3a60 0x7f0a64154120 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800d620 tx=0x7f0a5800dae0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.930+0000 7f0a6af1e640 1 -- 172.21.15.29:0/3509210448 shutdown_connections 2024-04-25T00:38:46.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.930+0000 7f0a6af1e640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0a50067d40 0x7f0a5006a200 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.930+0000 7f0a6af1e640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a640b4b10 0x7f0a6413c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.930+0000 7f0a6af1e640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a640b3a60 0x7f0a64154120 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.930+0000 7f0a6af1e640 1 --2- 172.21.15.29:0/3509210448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a640b3140 0x7f0a64153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:46.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.930+0000 7f0a6af1e640 1 -- 172.21.15.29:0/3509210448 >> 172.21.15.29:0/3509210448 conn(0x7f0a640b1870 msgr2=0x7f0a640ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:46.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.930+0000 7f0a6af1e640 1 -- 172.21.15.29:0/3509210448 shutdown_connections 2024-04-25T00:38:46.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:46.930+0000 7f0a6af1e640 1 -- 172.21.15.29:0/3509210448 wait complete. 2024-04-25T00:38:46.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:46.950 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol5860/.meta) 2024-04-25T00:38:46.968 INFO:teuthology.orchestra.run.smithi029.stdout:[GLOBAL] 2024-04-25T00:38:46.968 INFO:teuthology.orchestra.run.smithi029.stdout:version = 1 2024-04-25T00:38:46.969 INFO:teuthology.orchestra.run.smithi029.stdout:type = subvolume 2024-04-25T00:38:46.969 INFO:teuthology.orchestra.run.smithi029.stdout:path = /volumes/_nogroup/subvol5860/c0e50fc5-73bd-41b9-8e1b-3f3c8084454b 2024-04-25T00:38:46.969 INFO:teuthology.orchestra.run.smithi029.stdout:state = complete 2024-04-25T00:38:46.969 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:46.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:46.970 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp25200/subvol14261/.meta) 2024-04-25T00:38:47.027 INFO:teuthology.orchestra.run.smithi029.stdout:[GLOBAL] 2024-04-25T00:38:47.027 INFO:teuthology.orchestra.run.smithi029.stdout:version = 1 2024-04-25T00:38:47.027 INFO:teuthology.orchestra.run.smithi029.stdout:type = subvolume 2024-04-25T00:38:47.027 INFO:teuthology.orchestra.run.smithi029.stdout:path = /volumes/subvol_grp25200/subvol14261/d1941772-9e80-4658-bdf4-1031f0bf65ba 2024-04-25T00:38:47.027 INFO:teuthology.orchestra.run.smithi029.stdout:state = complete 2024-04-25T00:38:47.027 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:47.029 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol5860 2024-04-25T00:38:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.202+0000 7ffba540f640 1 Processor -- start 2024-04-25T00:38:47.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.202+0000 7ffba540f640 1 -- start start 2024-04-25T00:38:47.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffba540f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffba00b4780 0x7ffba015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffba540f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba00b3720 con 0x7ffba00cc240 2024-04-25T00:38:47.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffba540f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba00b39c0 con 0x7ffba015e8d0 2024-04-25T00:38:47.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffba540f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ffba00cc650 con 0x7ffba00b4780 2024-04-25T00:38:47.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ffba00cc240 0x7ffba015c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56350/0 (socket says 172.21.15.29:56350) 2024-04-25T00:38:47.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9f7fe640 1 -- 172.21.15.29:0/1910044141 learned_addr learned my addr 172.21.15.29:0/1910044141 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:47.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4271806539 0 0) 0x7ffba00b3720 con 0x7ffba00cc240 2024-04-25T00:38:47.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9e7fc640 1 --2- 172.21.15.29:0/1910044141 >> v2:172.21.15.157:3300/0 conn(0x7ffba00b4780 0x7ffba015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb88005740 con 0x7ffba00cc240 2024-04-25T00:38:47.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2868487622 0 0) 0x7ffb88005740 con 0x7ffba00cc240 2024-04-25T00:38:47.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 >> v2:172.21.15.157:3300/0 conn(0x7ffba00b4780 msgr2=0x7ffba015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:47.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 --2- 172.21.15.29:0/1910044141 >> v2:172.21.15.157:3300/0 conn(0x7ffba00b4780 0x7ffba015f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 >> v1:172.21.15.157:6789/0 conn(0x7ffba015e8d0 legacy=0x7ffba015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba00b3ca0 con 0x7ffba00cc240 2024-04-25T00:38:47.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ffb90001680 con 0x7ffba00cc240 2024-04-25T00:38:47.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9e7fc640 1 --2- 172.21.15.29:0/1910044141 >> v2:172.21.15.157:3300/0 conn(0x7ffba00b4780 0x7ffba015f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:47.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 >> v1:172.21.15.29:6789/0 conn(0x7ffba00cc240 legacy=0x7ffba015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 --2- 172.21.15.29:0/1910044141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb88005bd0 0x7ffb88005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 --2- 172.21.15.29:0/1910044141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb88006a20 0x7ffb88006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 --2- 172.21.15.29:0/1910044141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb880073e0 0x7ffb8800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8800e300 con 0x7ffb880073e0 2024-04-25T00:38:47.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8800e4b0 con 0x7ffb88005bd0 2024-04-25T00:38:47.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8800e6b0 con 0x7ffb88006a20 2024-04-25T00:38:47.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9f7fe640 1 --2- 172.21.15.29:0/1910044141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb880073e0 0x7ffb8800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9e7fc640 1 --2- 172.21.15.29:0/1910044141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb88006a20 0x7ffb88006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9effd640 1 --2- 172.21.15.29:0/1910044141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb88005bd0 0x7ffb88005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9f7fe640 1 -- 172.21.15.29:0/1910044141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb88006a20 msgr2=0x7ffb88006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:47.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9f7fe640 1 --2- 172.21.15.29:0/1910044141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb88006a20 0x7ffb88006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9f7fe640 1 -- 172.21.15.29:0/1910044141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb88005bd0 msgr2=0x7ffb88005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:47.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9f7fe640 1 --2- 172.21.15.29:0/1910044141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb88005bd0 0x7ffb88005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9f7fe640 1 -- 172.21.15.29:0/1910044141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb88005740 con 0x7ffb880073e0 2024-04-25T00:38:47.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9f7fe640 1 -- 172.21.15.29:0/1910044141 reap_dead start 2024-04-25T00:38:47.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9f7fe640 1 --2- 172.21.15.29:0/1910044141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb880073e0 0x7ffb8800dbe0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000c240 tx=0x7ffb9001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8483e779cdbeed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:47.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90007d20 con 0x7ffb880073e0 2024-04-25T00:38:47.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb900a9070 con 0x7ffb880073e0 2024-04-25T00:38:47.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.206+0000 7ffb9dffb640 1 -- 172.21.15.29:0/1910044141 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb900a5da0 con 0x7ffb880073e0 2024-04-25T00:38:47.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 -- 172.21.15.29:0/1910044141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb880073e0 msgr2=0x7ffb8800dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 --2- 172.21.15.29:0/1910044141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb880073e0 0x7ffb8800dbe0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000c240 tx=0x7ffb9001cc80 comp rx=0 tx=0).stop 2024-04-25T00:38:47.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 -- 172.21.15.29:0/1910044141 shutdown_connections 2024-04-25T00:38:47.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 --2- 172.21.15.29:0/1910044141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb880073e0 0x7ffb8800dbe0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 -- 172.21.15.29:0/1910044141 >> 172.21.15.29:0/1910044141 conn(0x7ffba00b1870 msgr2=0x7ffba00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:47.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 -- 172.21.15.29:0/1910044141 shutdown_connections 2024-04-25T00:38:47.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 -- 172.21.15.29:0/1910044141 wait complete. 2024-04-25T00:38:47.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 Processor -- start 2024-04-25T00:38:47.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 -- start start 2024-04-25T00:38:47.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb880073e0 0x7ffba0137f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffba00cd250 0x7ffba00ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba015a670 0x7ffba0138570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffba00b42f0 con 0x7ffba00cd250 2024-04-25T00:38:47.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffba00b3580 con 0x7ffba015a670 2024-04-25T00:38:47.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffba006cf10 con 0x7ffb880073e0 2024-04-25T00:38:47.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb9e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffba00cd250 0x7ffba00ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba015a670 0x7ffba0138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb9effd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb880073e0 0x7ffba0137f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba015a670 0x7ffba0138570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:50038/0 (socket says 172.21.15.29:50038) 2024-04-25T00:38:47.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb9f7fe640 1 -- 172.21.15.29:0/644727236 learned_addr learned my addr 172.21.15.29:0/644727236 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:47.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb9e7fc640 1 -- 172.21.15.29:0/644727236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb880073e0 msgr2=0x7ffba0137f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb9e7fc640 1 --2- 172.21.15.29:0/644727236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb880073e0 0x7ffba0137f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb9e7fc640 1 -- 172.21.15.29:0/644727236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba015a670 msgr2=0x7ffba0138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb9e7fc640 1 --2- 172.21.15.29:0/644727236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba015a670 0x7ffba0138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb9e7fc640 1 -- 172.21.15.29:0/644727236 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba00cd750 con 0x7ffba00cd250 2024-04-25T00:38:47.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb9e7fc640 1 --2- 172.21.15.29:0/644727236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffba00cd250 0x7ffba00ce7f0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009b50 tx=0x7ffb94007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:47.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb877fe640 1 -- 172.21.15.29:0/644727236 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94014610 con 0x7ffba00cd250 2024-04-25T00:38:47.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb877fe640 1 -- 172.21.15.29:0/644727236 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb94019070 con 0x7ffba00cd250 2024-04-25T00:38:47.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 -- 172.21.15.29:0/644727236 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba00cda30 con 0x7ffba00cd250 2024-04-25T00:38:47.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffb877fe640 1 -- 172.21.15.29:0/644727236 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb940149d0 con 0x7ffba00cd250 2024-04-25T00:38:47.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.210+0000 7ffba540f640 1 -- 172.21.15.29:0/644727236 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba00cdff0 con 0x7ffba00cd250 2024-04-25T00:38:47.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.214+0000 7ffba540f640 1 -- 172.21.15.29:0/644727236 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba0069270 con 0x7ffba00cd250 2024-04-25T00:38:47.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.214+0000 7ffb877fe640 1 -- 172.21.15.29:0/644727236 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 20) ==== 87853+0+0 (secure 0 0 0) 0x7ffb94014b70 con 0x7ffba00cd250 2024-04-25T00:38:47.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.214+0000 7ffb877fe640 1 --2- 172.21.15.29:0/644727236 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb8c067d70 0x7ffb8c06a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.214+0000 7ffb877fe640 1 -- 172.21.15.29:0/644727236 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7ffb940878c0 con 0x7ffba00cd250 2024-04-25T00:38:47.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.214+0000 7ffb9effd640 1 --2- 172.21.15.29:0/644727236 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb8c067d70 0x7ffb8c06a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.214+0000 7ffb9effd640 1 --2- 172.21.15.29:0/644727236 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb8c067d70 0x7ffb8c06a230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb98007fd0 tx=0x7ffb98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:47.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.218+0000 7ffb877fe640 1 -- 172.21.15.29:0/644727236 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffb94057790 con 0x7ffba00cd250 2024-04-25T00:38:47.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.406+0000 7ffba540f640 1 -- 172.21.15.29:0/644727236 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol5860", "target": ["mon-mgr", ""]}) -- 0x7ffba013e830 con 0x7ffb8c067d70 2024-04-25T00:38:47.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.478+0000 7ffb877fe640 1 -- 172.21.15.29:0/644727236 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffba013e830 con 0x7ffb8c067d70 2024-04-25T00:38:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.482+0000 7ffba540f640 1 -- 172.21.15.29:0/644727236 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb8c067d70 msgr2=0x7ffb8c06a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.482+0000 7ffba540f640 1 --2- 172.21.15.29:0/644727236 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb8c067d70 0x7ffb8c06a230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb98007fd0 tx=0x7ffb98009040 comp rx=0 tx=0).stop 2024-04-25T00:38:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.482+0000 7ffba540f640 1 -- 172.21.15.29:0/644727236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffba00cd250 msgr2=0x7ffba00ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.482+0000 7ffba540f640 1 --2- 172.21.15.29:0/644727236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffba00cd250 0x7ffba00ce7f0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009b50 tx=0x7ffb94007700 comp rx=0 tx=0).stop 2024-04-25T00:38:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.482+0000 7ffba540f640 1 -- 172.21.15.29:0/644727236 shutdown_connections 2024-04-25T00:38:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.482+0000 7ffba540f640 1 --2- 172.21.15.29:0/644727236 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb8c067d70 0x7ffb8c06a230 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.482+0000 7ffba540f640 1 --2- 172.21.15.29:0/644727236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba015a670 0x7ffba0138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.482+0000 7ffba540f640 1 --2- 172.21.15.29:0/644727236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffba00cd250 0x7ffba00ce7f0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.482+0000 7ffba540f640 1 --2- 172.21.15.29:0/644727236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb880073e0 0x7ffba0137f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.482+0000 7ffba540f640 1 -- 172.21.15.29:0/644727236 >> 172.21.15.29:0/644727236 conn(0x7ffba00b1870 msgr2=0x7ffba00728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.482+0000 7ffba540f640 1 -- 172.21.15.29:0/644727236 shutdown_connections 2024-04-25T00:38:47.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.482+0000 7ffba540f640 1 -- 172.21.15.29:0/644727236 wait complete. 2024-04-25T00:38:47.492 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:47.490+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:47.492 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:47.490+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:47.492 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:47.490+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:47.493 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:47.490+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:47.493 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:47.490+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:47.507 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol14261 subvol_grp25200 2024-04-25T00:38:47.511 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:47.506+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:47.511 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:47.506+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:47.511 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:47.506+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:47.511 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:47.506+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:47.511 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:47.506+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:38:47.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff64aa4640 1 Processor -- start 2024-04-25T00:38:47.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff64aa4640 1 -- start start 2024-04-25T00:38:47.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff64aa4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff60071a20 0x7eff600af560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff64aa4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff6006d4c0 con 0x7eff60071590 2024-04-25T00:38:47.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff64aa4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff600ae3b0 con 0x7eff601551d0 2024-04-25T00:38:47.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff64aa4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7eff60071e90 con 0x7eff60071a20 2024-04-25T00:38:47.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5dd9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7eff60071590 0x7eff600aee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56352/0 (socket says 172.21.15.29:56352) 2024-04-25T00:38:47.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5dd9b640 1 -- 172.21.15.29:0/1476555704 learned_addr learned my addr 172.21.15.29:0/1476555704 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:47.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5ed9d640 1 --2- 172.21.15.29:0/1476555704 >> v2:172.21.15.157:3300/0 conn(0x7eff60071a20 0x7eff600af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3205698790 0 0) 0x7eff48001120 con 0x7eff60071590 2024-04-25T00:38:47.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff4c003530 con 0x7eff60071590 2024-04-25T00:38:47.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1192955263 0 0) 0x7eff4c003530 con 0x7eff60071590 2024-04-25T00:38:47.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 >> v2:172.21.15.157:3300/0 conn(0x7eff60071a20 msgr2=0x7eff600af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:47.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 --2- 172.21.15.29:0/1476555704 >> v2:172.21.15.157:3300/0 conn(0x7eff60071a20 0x7eff600af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 >> v1:172.21.15.157:6789/0 conn(0x7eff601551d0 legacy=0x7eff60161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff60162730 con 0x7eff60071590 2024-04-25T00:38:47.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7eff48001850 con 0x7eff60071590 2024-04-25T00:38:47.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 >> v1:172.21.15.29:6789/0 conn(0x7eff60071590 legacy=0x7eff600aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 --2- 172.21.15.29:0/1476555704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff4c0039c0 0x7eff4c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 --2- 172.21.15.29:0/1476555704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff4c004810 0x7eff4c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 --2- 172.21.15.29:0/1476555704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff4c0051d0 0x7eff4c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7eff60071e90 con 0x7eff4c004810 2024-04-25T00:38:47.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c00c070 con 0x7eff4c0051d0 2024-04-25T00:38:47.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff4c00c270 con 0x7eff4c0039c0 2024-04-25T00:38:47.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5dd9b640 1 --2- 172.21.15.29:0/1476555704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff4c004810 0x7eff4c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5ed9d640 1 --2- 172.21.15.29:0/1476555704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff4c0051d0 0x7eff4c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5e59c640 1 --2- 172.21.15.29:0/1476555704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff4c0039c0 0x7eff4c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5dd9b640 1 -- 172.21.15.29:0/1476555704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff4c0039c0 msgr2=0x7eff4c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:47.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5dd9b640 1 --2- 172.21.15.29:0/1476555704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff4c0039c0 0x7eff4c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5dd9b640 1 -- 172.21.15.29:0/1476555704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff4c0051d0 msgr2=0x7eff4c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:47.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5dd9b640 1 --2- 172.21.15.29:0/1476555704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff4c0051d0 0x7eff4c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5ed9d640 1 -- 172.21.15.29:0/1476555704 reap_dead start 2024-04-25T00:38:47.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.630+0000 7eff5dd9b640 1 -- 172.21.15.29:0/1476555704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c003530 con 0x7eff4c004810 2024-04-25T00:38:47.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.634+0000 7eff5dd9b640 1 --2- 172.21.15.29:0/1476555704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff4c004810 0x7eff4c004c90 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7eff4800c4c0 tx=0x7eff4801f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2811bb6328048004 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:47.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.634+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4800c0e0 con 0x7eff4c004810 2024-04-25T00:38:47.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.634+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff48005b00 con 0x7eff4c004810 2024-04-25T00:38:47.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.634+0000 7eff5d59a640 1 -- 172.21.15.29:0/1476555704 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48007070 con 0x7eff4c004810 2024-04-25T00:38:47.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.634+0000 7eff64aa4640 1 -- 172.21.15.29:0/1476555704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff4c004810 msgr2=0x7eff4c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.634+0000 7eff64aa4640 1 --2- 172.21.15.29:0/1476555704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff4c004810 0x7eff4c004c90 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7eff4800c4c0 tx=0x7eff4801f3a0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.634+0000 7eff64aa4640 1 -- 172.21.15.29:0/1476555704 shutdown_connections 2024-04-25T00:38:47.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.634+0000 7eff64aa4640 1 --2- 172.21.15.29:0/1476555704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff4c004810 0x7eff4c004c90 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.634+0000 7eff64aa4640 1 -- 172.21.15.29:0/1476555704 >> 172.21.15.29:0/1476555704 conn(0x7eff600b21d0 msgr2=0x7eff600b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:47.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.634+0000 7eff64aa4640 1 -- 172.21.15.29:0/1476555704 shutdown_connections 2024-04-25T00:38:47.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.634+0000 7eff64aa4640 1 -- 172.21.15.29:0/1476555704 wait complete. 2024-04-25T00:38:47.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.634+0000 7eff64aa4640 1 Processor -- start 2024-04-25T00:38:47.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff64aa4640 1 -- start start 2024-04-25T00:38:47.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff64aa4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff4c004810 0x7eff6014bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff64aa4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff601f8190 0x7eff6006d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff64aa4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff6006dbe0 0x7eff60156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff64aa4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7eff600ae930 con 0x7eff4c004810 2024-04-25T00:38:47.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff64aa4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff600ae650 con 0x7eff601f8190 2024-04-25T00:38:47.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff64aa4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff601555b0 con 0x7eff6006dbe0 2024-04-25T00:38:47.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff5e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff4c004810 0x7eff6014bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff5e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff4c004810 0x7eff6014bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50372/0 (socket says 172.21.15.29:50372) 2024-04-25T00:38:47.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff5e59c640 1 -- 172.21.15.29:0/270055267 learned_addr learned my addr 172.21.15.29:0/270055267 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:47.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff5dd9b640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff601f8190 0x7eff6006d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff5ed9d640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff6006dbe0 0x7eff60156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff5e59c640 1 -- 172.21.15.29:0/270055267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff6006dbe0 msgr2=0x7eff60156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff5e59c640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff6006dbe0 0x7eff60156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff5e59c640 1 -- 172.21.15.29:0/270055267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff601f8190 msgr2=0x7eff6006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff5e59c640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff601f8190 0x7eff6006d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff5e59c640 1 -- 172.21.15.29:0/270055267 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff60152bd0 con 0x7eff4c004810 2024-04-25T00:38:47.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff5e59c640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff4c004810 0x7eff6014bc60 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7eff5800e8f0 tx=0x7eff5800edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:47.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff56ffd640 1 -- 172.21.15.29:0/270055267 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58003b90 con 0x7eff4c004810 2024-04-25T00:38:47.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff56ffd640 1 -- 172.21.15.29:0/270055267 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff58018070 con 0x7eff4c004810 2024-04-25T00:38:47.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff64aa4640 1 -- 172.21.15.29:0/270055267 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff601f8610 con 0x7eff4c004810 2024-04-25T00:38:47.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff56ffd640 1 -- 172.21.15.29:0/270055267 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff580050d0 con 0x7eff4c004810 2024-04-25T00:38:47.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff64aa4640 1 -- 172.21.15.29:0/270055267 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff60161290 con 0x7eff4c004810 2024-04-25T00:38:47.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff64aa4640 1 -- 172.21.15.29:0/270055267 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff600692e0 con 0x7eff4c004810 2024-04-25T00:38:47.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff56ffd640 1 -- 172.21.15.29:0/270055267 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 20) ==== 87853+0+0 (secure 0 0 0) 0x7eff58005270 con 0x7eff4c004810 2024-04-25T00:38:47.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.638+0000 7eff56ffd640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7eff40067d40 0x7eff4006a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:47.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.642+0000 7eff5dd9b640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7eff40067d40 0x7eff4006a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:47.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.642+0000 7eff56ffd640 1 -- 172.21.15.29:0/270055267 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7eff58086310 con 0x7eff4c004810 2024-04-25T00:38:47.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.642+0000 7eff5dd9b640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7eff40067d40 0x7eff4006a200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff480af000 tx=0x7eff48009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:47.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.646+0000 7eff56ffd640 1 -- 172.21.15.29:0/270055267 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7eff60152bd0 con 0x7eff4c004810 2024-04-25T00:38:47.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.834+0000 7eff64aa4640 1 -- 172.21.15.29:0/270055267 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol14261", "group_name": "subvol_grp25200", "target": ["mon-mgr", ""]}) -- 0x7eff600d0190 con 0x7eff40067d40 2024-04-25T00:38:47.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.902+0000 7eff56ffd640 1 -- 172.21.15.29:0/270055267 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7eff600d0190 con 0x7eff40067d40 2024-04-25T00:38:47.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.906+0000 7eff64aa4640 1 -- 172.21.15.29:0/270055267 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7eff40067d40 msgr2=0x7eff4006a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.906+0000 7eff64aa4640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7eff40067d40 0x7eff4006a200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff480af000 tx=0x7eff48009040 comp rx=0 tx=0).stop 2024-04-25T00:38:47.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.906+0000 7eff64aa4640 1 -- 172.21.15.29:0/270055267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff4c004810 msgr2=0x7eff6014bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:47.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.906+0000 7eff64aa4640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff4c004810 0x7eff6014bc60 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7eff5800e8f0 tx=0x7eff5800edb0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.906+0000 7eff64aa4640 1 -- 172.21.15.29:0/270055267 shutdown_connections 2024-04-25T00:38:47.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.906+0000 7eff64aa4640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7eff40067d40 0x7eff4006a200 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.906+0000 7eff64aa4640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff6006dbe0 0x7eff60156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.906+0000 7eff64aa4640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff601f8190 0x7eff6006d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.906+0000 7eff64aa4640 1 --2- 172.21.15.29:0/270055267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff4c004810 0x7eff6014bc60 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.906+0000 7eff64aa4640 1 -- 172.21.15.29:0/270055267 >> 172.21.15.29:0/270055267 conn(0x7eff600b21d0 msgr2=0x7eff6014d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.906+0000 7eff64aa4640 1 -- 172.21.15.29:0/270055267 shutdown_connections 2024-04-25T00:38:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:47.906+0000 7eff64aa4640 1 -- 172.21.15.29:0/270055267 wait complete. 2024-04-25T00:38:47.929 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol19552 2024-04-25T00:38:48.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.090+0000 7fc53803c640 1 Processor -- start 2024-04-25T00:38:48.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.090+0000 7fc53803c640 1 -- start start 2024-04-25T00:38:48.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc53803c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc5300b3140 0x7fc53006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc53803c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5300b41e0 con 0x7fc5300b3c30 2024-04-25T00:38:48.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc53803c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc53006d960 con 0x7fc5300b4b10 2024-04-25T00:38:48.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc53803c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc53006dc40 con 0x7fc5300b3140 2024-04-25T00:38:48.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc535dd8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc5300b3140 0x7fc53006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc5365d9640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc5300b3c30 0x7fc5300ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36746/0 (socket says 172.21.15.29:36746) 2024-04-25T00:38:48.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc5365d9640 1 -- 172.21.15.29:0/1635552498 learned_addr learned my addr 172.21.15.29:0/1635552498 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:48.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc535dd8640 1 -- 172.21.15.29:0/1635552498 >> v1:172.21.15.157:6789/0 conn(0x7fc5300b4b10 legacy=0x7fc5300b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:48.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc535dd8640 1 -- 172.21.15.29:0/1635552498 >> v1:172.21.15.29:6789/0 conn(0x7fc5300b3c30 legacy=0x7fc5300ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc534dd6640 1 -- 172.21.15.29:0/1635552498 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1323707970 0 0) 0x7fc5300b41e0 con 0x7fc5300b3c30 2024-04-25T00:38:48.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc535dd8640 1 -- 172.21.15.29:0/1635552498 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53006df20 con 0x7fc5300b3140 2024-04-25T00:38:48.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc535dd8640 1 --2- 172.21.15.29:0/1635552498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5300b3140 0x7fc53006e440 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009dc0 tx=0x7fc52c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dadcdc524abd5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:48.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc534dd6640 1 -- 172.21.15.29:0/1635552498 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0b2070 con 0x7fc5300b3140 2024-04-25T00:38:48.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc534dd6640 1 -- 172.21.15.29:0/1635552498 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc52c0a57d0 con 0x7fc5300b3140 2024-04-25T00:38:48.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc534dd6640 1 -- 172.21.15.29:0/1635552498 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0a5ba0 con 0x7fc5300b3140 2024-04-25T00:38:48.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc53803c640 1 -- 172.21.15.29:0/1635552498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5300b3140 msgr2=0x7fc53006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.094+0000 7fc53803c640 1 --2- 172.21.15.29:0/1635552498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5300b3140 0x7fc53006e440 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009dc0 tx=0x7fc52c01c910 comp rx=0 tx=0).stop 2024-04-25T00:38:48.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 -- 172.21.15.29:0/1635552498 shutdown_connections 2024-04-25T00:38:48.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 --2- 172.21.15.29:0/1635552498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5300b3140 0x7fc53006e440 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 -- 172.21.15.29:0/1635552498 >> 172.21.15.29:0/1635552498 conn(0x7fc5300b1870 msgr2=0x7fc5300b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:48.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 -- 172.21.15.29:0/1635552498 shutdown_connections 2024-04-25T00:38:48.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 -- 172.21.15.29:0/1635552498 wait complete. 2024-04-25T00:38:48.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 Processor -- start 2024-04-25T00:38:48.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 -- start start 2024-04-25T00:38:48.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5300b3140 0x7fc53015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5300b3c30 0x7fc530071730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5300b4b10 0x7fc530141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc530169d90 con 0x7fc5300b3c30 2024-04-25T00:38:48.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc530169c10 con 0x7fc5300b4b10 2024-04-25T00:38:48.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc530169f10 con 0x7fc5300b3140 2024-04-25T00:38:48.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc5355d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5300b3c30 0x7fc530071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc5355d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5300b3c30 0x7fc530071730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50390/0 (socket says 172.21.15.29:50390) 2024-04-25T00:38:48.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc5355d7640 1 -- 172.21.15.29:0/1470058881 learned_addr learned my addr 172.21.15.29:0/1470058881 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:48.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc535dd8640 1 --2- 172.21.15.29:0/1470058881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5300b3140 0x7fc53015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc5355d7640 1 -- 172.21.15.29:0/1470058881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5300b3140 msgr2=0x7fc53015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc5355d7640 1 --2- 172.21.15.29:0/1470058881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5300b3140 0x7fc53015a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc5355d7640 1 -- 172.21.15.29:0/1470058881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5300b4b10 msgr2=0x7fc530141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc5355d7640 1 --2- 172.21.15.29:0/1470058881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5300b4b10 0x7fc530141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc5355d7640 1 -- 172.21.15.29:0/1470058881 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53015a610 con 0x7fc5300b3c30 2024-04-25T00:38:48.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc5355d7640 1 --2- 172.21.15.29:0/1470058881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5300b3c30 0x7fc530071730 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c450 tx=0x7fc52000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:48.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc5267fc640 1 -- 172.21.15.29:0/1470058881 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520011790 con 0x7fc5300b3c30 2024-04-25T00:38:48.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc5267fc640 1 -- 172.21.15.29:0/1470058881 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc520017070 con 0x7fc5300b3c30 2024-04-25T00:38:48.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 -- 172.21.15.29:0/1470058881 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53015a8b0 con 0x7fc5300b3c30 2024-04-25T00:38:48.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc5267fc640 1 -- 172.21.15.29:0/1470058881 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520011b90 con 0x7fc5300b3c30 2024-04-25T00:38:48.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.098+0000 7fc53803c640 1 -- 172.21.15.29:0/1470058881 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc530070f30 con 0x7fc5300b3c30 2024-04-25T00:38:48.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.102+0000 7fc53803c640 1 -- 172.21.15.29:0/1470058881 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc530069270 con 0x7fc5300b3c30 2024-04-25T00:38:48.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.102+0000 7fc5267fc640 1 -- 172.21.15.29:0/1470058881 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 20) ==== 87853+0+0 (secure 0 0 0) 0x7fc520011d30 con 0x7fc5300b3c30 2024-04-25T00:38:48.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.102+0000 7fc5267fc640 1 --2- 172.21.15.29:0/1470058881 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc514067d40 0x7fc51406a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.102+0000 7fc5267fc640 1 -- 172.21.15.29:0/1470058881 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fc5200861c0 con 0x7fc5300b3c30 2024-04-25T00:38:48.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.102+0000 7fc535dd8640 1 --2- 172.21.15.29:0/1470058881 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc514067d40 0x7fc51406a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.102+0000 7fc535dd8640 1 --2- 172.21.15.29:0/1470058881 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc514067d40 0x7fc51406a200 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0af000 tx=0x7fc52c0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:48.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.106+0000 7fc5267fc640 1 -- 172.21.15.29:0/1470058881 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc520056770 con 0x7fc5300b3c30 2024-04-25T00:38:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.290+0000 7fc53803c640 1 -- 172.21.15.29:0/1470058881 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol19552", "target": ["mon-mgr", ""]}) -- 0x7fc530158200 con 0x7fc514067d40 2024-04-25T00:38:48.311 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:38:48.306+0000 7f987757d640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvol19552' is not ready for operation rm (use --force to override) 2024-04-25T00:38:48.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.306+0000 7fc5267fc640 1 -- 172.21.15.29:0/1470058881 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvol19552' is not ready for operation rm (use --force to override)) ==== 87+0+0 (secure 0 0 0) 0x7fc530158200 con 0x7fc514067d40 2024-04-25T00:38:48.312 INFO:teuthology.orchestra.run.smithi029.stderr:Error EAGAIN: subvolume 'subvol19552' is not ready for operation rm (use --force to override) 2024-04-25T00:38:48.318 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-04-25T00:38:48.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:48.319 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol19552/.meta) 2024-04-25T00:38:48.337 INFO:teuthology.orchestra.run.smithi029.stdout:[GLOBAL] 2024-04-25T00:38:48.337 INFO:teuthology.orchestra.run.smithi029.stdout:version = 1 2024-04-25T00:38:48.337 INFO:teuthology.orchestra.run.smithi029.stdout:type = clone 2024-04-25T00:38:48.337 INFO:teuthology.orchestra.run.smithi029.stdout:path = /volumes/_nogroup/subvol19552/d9d3b682-bda2-49e5-8373-9a70a669dad0 2024-04-25T00:38:48.338 INFO:teuthology.orchestra.run.smithi029.stdout:state = pending 2024-04-25T00:38:48.338 INFO:teuthology.orchestra.run.smithi029.stdout:[source] 2024-04-25T00:38:48.338 INFO:teuthology.orchestra.run.smithi029.stdout:volume = fake 2024-04-25T00:38:48.338 INFO:teuthology.orchestra.run.smithi029.stdout:subvolume = fake 2024-04-25T00:38:48.338 INFO:teuthology.orchestra.run.smithi029.stdout:snapshot = fake 2024-04-25T00:38:48.339 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol19552 --force 2024-04-25T00:38:48.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.514+0000 7fa950c75640 1 Processor -- start 2024-04-25T00:38:48.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.514+0000 7fa950c75640 1 -- start start 2024-04-25T00:38:48.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.514+0000 7fa950c75640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa94c15c100 0x7fa94c0b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.514+0000 7fa950c75640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa94c0d23a0 0x7fa94c0cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.514+0000 7fa950c75640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94c0cc9d0 con 0x7fa94c0d06f0 2024-04-25T00:38:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.514+0000 7fa950c75640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa94c0ccd00 con 0x7fa94c0d23a0 2024-04-25T00:38:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.514+0000 7fa950c75640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa94c0b3050 con 0x7fa94c15c100 2024-04-25T00:38:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.514+0000 7fa94ad9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa94c0d23a0 0x7fa94c0cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.514+0000 7fa941d9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa94c0d06f0 0x7fa94c0d1c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36758/0 (socket says 172.21.15.29:36758) 2024-04-25T00:38:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.514+0000 7fa941d9b640 1 -- 172.21.15.29:0/1218198665 learned_addr learned my addr 172.21.15.29:0/1218198665 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.514+0000 7fa94a59c640 1 --2- 172.21.15.29:0/1218198665 >> v2:172.21.15.157:3300/0 conn(0x7fa94c15c100 0x7fa94c0b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa94ad9d640 1 -- 172.21.15.29:0/1218198665 >> v2:172.21.15.157:3300/0 conn(0x7fa94c15c100 msgr2=0x7fa94c0b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa94ad9d640 1 --2- 172.21.15.29:0/1218198665 >> v2:172.21.15.157:3300/0 conn(0x7fa94c15c100 0x7fa94c0b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa94ad9d640 1 -- 172.21.15.29:0/1218198665 >> v1:172.21.15.29:6789/0 conn(0x7fa94c0d06f0 legacy=0x7fa94c0d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa949d9b640 1 -- 172.21.15.29:0/1218198665 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3064679357 0 0) 0x7fa94c0cc9d0 con 0x7fa94c0d06f0 2024-04-25T00:38:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa94ad9d640 1 -- 172.21.15.29:0/1218198665 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c0b3330 con 0x7fa94c0d23a0 2024-04-25T00:38:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa94ad9d640 1 --2- 172.21.15.29:0/1218198665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa94c0d23a0 0x7fa94c0cd190 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fa94c09fe00 tx=0x7fa94401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7ab69b3aede08f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:48.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa949d9b640 1 -- 172.21.15.29:0/1218198665 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440b2070 con 0x7fa94c0d23a0 2024-04-25T00:38:48.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa949d9b640 1 -- 172.21.15.29:0/1218198665 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9440a5870 con 0x7fa94c0d23a0 2024-04-25T00:38:48.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa949d9b640 1 -- 172.21.15.29:0/1218198665 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440a5c40 con 0x7fa94c0d23a0 2024-04-25T00:38:48.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa950c75640 1 -- 172.21.15.29:0/1218198665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa94c0d23a0 msgr2=0x7fa94c0cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa950c75640 1 --2- 172.21.15.29:0/1218198665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa94c0d23a0 0x7fa94c0cd190 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fa94c09fe00 tx=0x7fa94401c8a0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa950c75640 1 -- 172.21.15.29:0/1218198665 shutdown_connections 2024-04-25T00:38:48.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa950c75640 1 --2- 172.21.15.29:0/1218198665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa94c0d23a0 0x7fa94c0cd190 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa950c75640 1 --2- 172.21.15.29:0/1218198665 >> v2:172.21.15.157:3300/0 conn(0x7fa94c15c100 0x7fa94c0b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa950c75640 1 -- 172.21.15.29:0/1218198665 >> 172.21.15.29:0/1218198665 conn(0x7fa94c09aa30 msgr2=0x7fa94c09ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:48.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa950c75640 1 -- 172.21.15.29:0/1218198665 shutdown_connections 2024-04-25T00:38:48.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.518+0000 7fa950c75640 1 -- 172.21.15.29:0/1218198665 wait complete. 2024-04-25T00:38:48.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa950c75640 1 Processor -- start 2024-04-25T00:38:48.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa950c75640 1 -- start start 2024-04-25T00:38:48.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa950c75640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa94c0d06f0 0x7fa94c0c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa950c75640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa94c0d23a0 0x7fa94c15aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa950c75640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa94c15c100 0x7fa94c1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa950c75640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c0cdee0 con 0x7fa94c15c100 2024-04-25T00:38:48.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa950c75640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c0d0d60 con 0x7fa94c0d06f0 2024-04-25T00:38:48.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa950c75640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa94c0ce060 con 0x7fa94c0d23a0 2024-04-25T00:38:48.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa94ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa94c15c100 0x7fa94c1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa94ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa94c15c100 0x7fa94c1548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50408/0 (socket says 172.21.15.29:50408) 2024-04-25T00:38:48.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa94ad9d640 1 -- 172.21.15.29:0/2547795736 learned_addr learned my addr 172.21.15.29:0/2547795736 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:48.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa94a59c640 1 --2- 172.21.15.29:0/2547795736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa94c0d06f0 0x7fa94c0c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa94ad9d640 1 -- 172.21.15.29:0/2547795736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa94c0d23a0 msgr2=0x7fa94c15aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa94ad9d640 1 --2- 172.21.15.29:0/2547795736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa94c0d23a0 0x7fa94c15aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa94ad9d640 1 -- 172.21.15.29:0/2547795736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa94c0d06f0 msgr2=0x7fa94c0c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa94ad9d640 1 --2- 172.21.15.29:0/2547795736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa94c0d06f0 0x7fa94c0c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa94ad9d640 1 -- 172.21.15.29:0/2547795736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c0ca1e0 con 0x7fa94c15c100 2024-04-25T00:38:48.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa94ad9d640 1 --2- 172.21.15.29:0/2547795736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa94c15c100 0x7fa94c1548c0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa9440b0040 tx=0x7fa94c0b3820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:48.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa9437fe640 1 -- 172.21.15.29:0/2547795736 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440bc070 con 0x7fa94c15c100 2024-04-25T00:38:48.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa9437fe640 1 -- 172.21.15.29:0/2547795736 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa944003ba0 con 0x7fa94c15c100 2024-04-25T00:38:48.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa950c75640 1 -- 172.21.15.29:0/2547795736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94c0c6b40 con 0x7fa94c15c100 2024-04-25T00:38:48.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa9437fe640 1 -- 172.21.15.29:0/2547795736 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440b2040 con 0x7fa94c15c100 2024-04-25T00:38:48.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa950c75640 1 -- 172.21.15.29:0/2547795736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94c0ca480 con 0x7fa94c15c100 2024-04-25T00:38:48.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa9437fe640 1 -- 172.21.15.29:0/2547795736 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fa9440a56f0 con 0x7fa94c15c100 2024-04-25T00:38:48.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.522+0000 7fa950c75640 1 -- 172.21.15.29:0/2547795736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94c0692e0 con 0x7fa94c15c100 2024-04-25T00:38:48.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.526+0000 7fa9437fe640 1 --2- 172.21.15.29:0/2547795736 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa92c067e60 0x7fa92c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.526+0000 7fa94a59c640 1 --2- 172.21.15.29:0/2547795736 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa92c067e60 0x7fa92c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.526+0000 7fa9437fe640 1 -- 172.21.15.29:0/2547795736 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fa944121310 con 0x7fa94c15c100 2024-04-25T00:38:48.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.526+0000 7fa94a59c640 1 --2- 172.21.15.29:0/2547795736 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa92c067e60 0x7fa92c06a320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0096f0 tx=0x7fa93c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:48.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.530+0000 7fa9437fe640 1 -- 172.21.15.29:0/2547795736 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa944016520 con 0x7fa94c15c100 2024-04-25T00:38:48.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.714+0000 7fa950c75640 1 -- 172.21.15.29:0/2547795736 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol19552", "force": true, "target": ["mon-mgr", ""]}) -- 0x7fa94c0ca6c0 con 0x7fa92c067e60 2024-04-25T00:38:48.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.746+0000 7fa9437fe640 1 -- 172.21.15.29:0/2547795736 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa93c00e4c0 con 0x7fa92c067e60 2024-04-25T00:38:48.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.750+0000 7fa950c75640 1 -- 172.21.15.29:0/2547795736 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa92c067e60 msgr2=0x7fa92c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.750+0000 7fa950c75640 1 --2- 172.21.15.29:0/2547795736 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa92c067e60 0x7fa92c06a320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0096f0 tx=0x7fa93c008040 comp rx=0 tx=0).stop 2024-04-25T00:38:48.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.750+0000 7fa950c75640 1 -- 172.21.15.29:0/2547795736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa94c15c100 msgr2=0x7fa94c1548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.750+0000 7fa950c75640 1 --2- 172.21.15.29:0/2547795736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa94c15c100 0x7fa94c1548c0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa9440b0040 tx=0x7fa94c0b3820 comp rx=0 tx=0).stop 2024-04-25T00:38:48.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.750+0000 7fa950c75640 1 -- 172.21.15.29:0/2547795736 shutdown_connections 2024-04-25T00:38:48.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.750+0000 7fa950c75640 1 --2- 172.21.15.29:0/2547795736 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa92c067e60 0x7fa92c06a320 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.750+0000 7fa950c75640 1 --2- 172.21.15.29:0/2547795736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa94c15c100 0x7fa94c1548c0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.750+0000 7fa950c75640 1 --2- 172.21.15.29:0/2547795736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa94c0d23a0 0x7fa94c15aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.750+0000 7fa950c75640 1 --2- 172.21.15.29:0/2547795736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa94c0d06f0 0x7fa94c0c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.750+0000 7fa950c75640 1 -- 172.21.15.29:0/2547795736 >> 172.21.15.29:0/2547795736 conn(0x7fa94c09aa30 msgr2=0x7fa94c0aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:48.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.750+0000 7fa950c75640 1 -- 172.21.15.29:0/2547795736 shutdown_connections 2024-04-25T00:38:48.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.750+0000 7fa950c75640 1 -- 172.21.15.29:0/2547795736 wait complete. 2024-04-25T00:38:48.775 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-04-25T00:38:48.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.894+0000 7fb001c30640 1 Processor -- start 2024-04-25T00:38:48.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.894+0000 7fb001c30640 1 -- start start 2024-04-25T00:38:48.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7fb001c30640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faffc15e8d0 0x7faffc0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7fb001c30640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faffc15f3c0 0x7faffc15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7fb001c30640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faffc0b3720 con 0x7faffc0b4780 2024-04-25T00:38:48.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7fb001c30640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faffc0b42f0 con 0x7faffc15e8d0 2024-04-25T00:38:48.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7fb001c30640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faffc0b39c0 con 0x7faffc15f3c0 2024-04-25T00:38:48.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffaffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7faffc0b4780 0x7faffc15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36766/0 (socket says 172.21.15.29:36766) 2024-04-25T00:38:48.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffaffd640 1 -- 172.21.15.29:0/3681367613 learned_addr learned my addr 172.21.15.29:0/3681367613 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:48.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffb7fe640 1 --2- 172.21.15.29:0/3681367613 >> v2:172.21.15.157:3300/0 conn(0x7faffc15e8d0 0x7faffc0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffbfff640 1 --2- 172.21.15.29:0/3681367613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faffc15f3c0 0x7faffc15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1694029031 0 0) 0x7faffc0b3720 con 0x7faffc0b4780 2024-04-25T00:38:48.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafe8003530 con 0x7faffc0b4780 2024-04-25T00:38:48.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2104892267 0 0) 0x7fafe8003530 con 0x7faffc0b4780 2024-04-25T00:38:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faffc15f3c0 msgr2=0x7faffc15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 --2- 172.21.15.29:0/3681367613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faffc15f3c0 0x7faffc15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 >> v2:172.21.15.157:3300/0 conn(0x7faffc15e8d0 msgr2=0x7faffc0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 --2- 172.21.15.29:0/3681367613 >> v2:172.21.15.157:3300/0 conn(0x7faffc15e8d0 0x7faffc0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc0b3ca0 con 0x7faffc0b4780 2024-04-25T00:38:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fafec001a00 con 0x7faffc0b4780 2024-04-25T00:38:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 >> v1:172.21.15.29:6789/0 conn(0x7faffc0b4780 legacy=0x7faffc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 --2- 172.21.15.29:0/3681367613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafe80039c0 0x7fafe8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 --2- 172.21.15.29:0/3681367613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faffc15f3c0 0x7fafe8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 --2- 172.21.15.29:0/3681367613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafe8004d50 0x7fafe80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faffc0b42f0 con 0x7fafe8004d50 2024-04-25T00:38:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faffc0b39c0 con 0x7fafe80039c0 2024-04-25T00:38:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fafe800b9d0 con 0x7faffc15f3c0 2024-04-25T00:38:48.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffbfff640 1 --2- 172.21.15.29:0/3681367613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafe8004d50 0x7fafe80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffaffd640 1 --2- 172.21.15.29:0/3681367613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faffc15f3c0 0x7fafe8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffb7fe640 1 --2- 172.21.15.29:0/3681367613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafe80039c0 0x7fafe8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffbfff640 1 -- 172.21.15.29:0/3681367613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faffc15f3c0 msgr2=0x7fafe8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:48.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffbfff640 1 --2- 172.21.15.29:0/3681367613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faffc15f3c0 0x7fafe8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffbfff640 1 -- 172.21.15.29:0/3681367613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafe80039c0 msgr2=0x7fafe8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:48.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffbfff640 1 --2- 172.21.15.29:0/3681367613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafe80039c0 0x7fafe8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffbfff640 1 -- 172.21.15.29:0/3681367613 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe8003530 con 0x7fafe8004d50 2024-04-25T00:38:48.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.898+0000 7faffb7fe640 1 --2- 172.21.15.29:0/3681367613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafe80039c0 0x7fafe8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffbfff640 1 --2- 172.21.15.29:0/3681367613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafe8004d50 0x7fafe80051d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7faff4009ac0 tx=0x7faff401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ca4b5bbd0d4654 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff40b2070 con 0x7fafe8004d50 2024-04-25T00:38:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff40a5660 con 0x7fafe8004d50 2024-04-25T00:38:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffa7fc640 1 -- 172.21.15.29:0/3681367613 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4009690 con 0x7fafe8004d50 2024-04-25T00:38:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 -- 172.21.15.29:0/3681367613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafe8004d50 msgr2=0x7fafe80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 --2- 172.21.15.29:0/3681367613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fafe8004d50 0x7fafe80051d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7faff4009ac0 tx=0x7faff401ca70 comp rx=0 tx=0).stop 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffbfff640 1 -- 172.21.15.29:0/3681367613 reap_dead start 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 -- 172.21.15.29:0/3681367613 shutdown_connections 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 -- 172.21.15.29:0/3681367613 >> 172.21.15.29:0/3681367613 conn(0x7faffc0b1870 msgr2=0x7faffc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 -- 172.21.15.29:0/3681367613 shutdown_connections 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 -- 172.21.15.29:0/3681367613 wait complete. 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 Processor -- start 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 -- start start 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc13d690 0x7faffc14d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faffc141b50 0x7faffc141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faffc0ae7c0 0x7faffc0aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faffc144a20 con 0x7faffc13d690 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faffc06cf10 con 0x7faffc0ae7c0 2024-04-25T00:38:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7fb001c30640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faffc0ae330 con 0x7faffc141b50 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc13d690 0x7faffc14d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc13d690 0x7faffc14d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50424/0 (socket says 172.21.15.29:50424) 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffb7fe640 1 -- 172.21.15.29:0/107348715 learned_addr learned my addr 172.21.15.29:0/107348715 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffb7fe640 1 -- 172.21.15.29:0/107348715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faffc141b50 msgr2=0x7faffc141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffbfff640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faffc0ae7c0 0x7faffc0aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffb7fe640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faffc141b50 0x7faffc141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffb7fe640 1 -- 172.21.15.29:0/107348715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faffc0ae7c0 msgr2=0x7faffc0aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffb7fe640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faffc0ae7c0 0x7faffc0aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffb7fe640 1 -- 172.21.15.29:0/107348715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc0ada20 con 0x7faffc13d690 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffbfff640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faffc0ae7c0 0x7faffc0aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.902+0000 7faffb7fe640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc13d690 0x7faffc14d0e0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fafe40115f0 tx=0x7fafe4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.906+0000 7faff3fff640 1 -- 172.21.15.29:0/107348715 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe4004220 con 0x7faffc13d690 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.906+0000 7faff3fff640 1 -- 172.21.15.29:0/107348715 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe40043c0 con 0x7faffc13d690 2024-04-25T00:38:48.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.906+0000 7fb001c30640 1 -- 172.21.15.29:0/107348715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faffc0adce0 con 0x7faffc13d690 2024-04-25T00:38:48.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.906+0000 7faff3fff640 1 -- 172.21.15.29:0/107348715 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe400f910 con 0x7faffc13d690 2024-04-25T00:38:48.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.906+0000 7fb001c30640 1 -- 172.21.15.29:0/107348715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faffc13da70 con 0x7faffc13d690 2024-04-25T00:38:48.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.906+0000 7fb001c30640 1 -- 172.21.15.29:0/107348715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faffc069270 con 0x7faffc13d690 2024-04-25T00:38:48.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.906+0000 7faff3fff640 1 -- 172.21.15.29:0/107348715 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fafe400fab0 con 0x7faffc13d690 2024-04-25T00:38:48.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.906+0000 7faff3fff640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fafdc067e60 0x7fafdc06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:48.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.906+0000 7faffaffd640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fafdc067e60 0x7fafdc06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:48.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.906+0000 7faff3fff640 1 -- 172.21.15.29:0/107348715 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fafe4086da0 con 0x7faffc13d690 2024-04-25T00:38:48.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.906+0000 7faffaffd640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fafdc067e60 0x7fafdc06a320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faffc070ef0 tx=0x7fafec0012e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:48.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:48.910+0000 7faff3fff640 1 -- 172.21.15.29:0/107348715 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fafe4019030 con 0x7faffc13d690 2024-04-25T00:38:49.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.102+0000 7fb001c30640 1 -- 172.21.15.29:0/107348715 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7faffc06e7b0 con 0x7fafdc067e60 2024-04-25T00:38:49.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.110+0000 7faff3fff640 1 -- 172.21.15.29:0/107348715 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7faffc06e7b0 con 0x7fafdc067e60 2024-04-25T00:38:49.113 INFO:teuthology.orchestra.run.smithi029.stdout:[] 2024-04-25T00:38:49.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.110+0000 7fb001c30640 1 -- 172.21.15.29:0/107348715 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fafdc067e60 msgr2=0x7fafdc06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.110+0000 7fb001c30640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fafdc067e60 0x7fafdc06a320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faffc070ef0 tx=0x7fafec0012e0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.110+0000 7fb001c30640 1 -- 172.21.15.29:0/107348715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc13d690 msgr2=0x7faffc14d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.110+0000 7fb001c30640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc13d690 0x7faffc14d0e0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fafe40115f0 tx=0x7fafe4011ab0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.110+0000 7fb001c30640 1 -- 172.21.15.29:0/107348715 shutdown_connections 2024-04-25T00:38:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.110+0000 7fb001c30640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fafdc067e60 0x7fafdc06a320 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.110+0000 7fb001c30640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faffc0ae7c0 0x7faffc0aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.110+0000 7fb001c30640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faffc141b50 0x7faffc141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.110+0000 7fb001c30640 1 --2- 172.21.15.29:0/107348715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc13d690 0x7faffc14d0e0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.110+0000 7fb001c30640 1 -- 172.21.15.29:0/107348715 >> 172.21.15.29:0/107348715 conn(0x7faffc0b1870 msgr2=0x7faffc06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.114+0000 7fb001c30640 1 -- 172.21.15.29:0/107348715 shutdown_connections 2024-04-25T00:38:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.114+0000 7fb001c30640 1 -- 172.21.15.29:0/107348715 wait complete. 2024-04-25T00:38:49.133 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-25T00:38:49.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:38:49.177 INFO:teuthology.orchestra.run.smithi029.stdout:2 2024-04-25T00:38:49.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:38:49.183 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:38:49.183 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:38:49.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:38:49.308 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:38:49.308 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:38:49.308 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:38:49.309 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:38:49.316 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:49.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.490+0000 7ffb5de2c640 1 Processor -- start 2024-04-25T00:38:49.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.490+0000 7ffb5de2c640 1 -- start start 2024-04-25T00:38:49.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb5de2c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffb580b4780 0x7ffb5815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb5de2c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb580b3720 con 0x7ffb5815e8d0 2024-04-25T00:38:49.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb5de2c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb580b39c0 con 0x7ffb580cc240 2024-04-25T00:38:49.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb5de2c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ffb580cc650 con 0x7ffb580b4780 2024-04-25T00:38:49.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb577fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ffb5815e8d0 0x7ffb5815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36780/0 (socket says 172.21.15.29:36780) 2024-04-25T00:38:49.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb577fe640 1 -- 172.21.15.29:0/3798587567 learned_addr learned my addr 172.21.15.29:0/3798587567 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:49.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb56ffd640 1 --2- 172.21.15.29:0/3798587567 >> v2:172.21.15.157:3300/0 conn(0x7ffb580b4780 0x7ffb5815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4012565264 0 0) 0x7ffb580b3720 con 0x7ffb5815e8d0 2024-04-25T00:38:49.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb40003530 con 0x7ffb5815e8d0 2024-04-25T00:38:49.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 165164554 0 0) 0x7ffb40003530 con 0x7ffb5815e8d0 2024-04-25T00:38:49.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 >> v2:172.21.15.157:3300/0 conn(0x7ffb580b4780 msgr2=0x7ffb5815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 --2- 172.21.15.29:0/3798587567 >> v2:172.21.15.157:3300/0 conn(0x7ffb580b4780 0x7ffb5815f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 >> v1:172.21.15.157:6789/0 conn(0x7ffb580cc240 legacy=0x7ffb5815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb580b3ca0 con 0x7ffb5815e8d0 2024-04-25T00:38:49.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ffb48001680 con 0x7ffb5815e8d0 2024-04-25T00:38:49.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 >> v1:172.21.15.29:6789/0 conn(0x7ffb5815e8d0 legacy=0x7ffb5815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 --2- 172.21.15.29:0/3798587567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb400039c0 0x7ffb40003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 --2- 172.21.15.29:0/3798587567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb40004810 0x7ffb40004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 --2- 172.21.15.29:0/3798587567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb400051d0 0x7ffb4000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4000e250 con 0x7ffb400051d0 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4000e400 con 0x7ffb400039c0 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4000e600 con 0x7ffb40004810 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb57fff640 1 --2- 172.21.15.29:0/3798587567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb400051d0 0x7ffb4000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb56ffd640 1 --2- 172.21.15.29:0/3798587567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb40004810 0x7ffb40004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb577fe640 1 --2- 172.21.15.29:0/3798587567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb400039c0 0x7ffb40003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb57fff640 1 -- 172.21.15.29:0/3798587567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb40004810 msgr2=0x7ffb40004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb57fff640 1 --2- 172.21.15.29:0/3798587567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb40004810 0x7ffb40004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb57fff640 1 -- 172.21.15.29:0/3798587567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb400039c0 msgr2=0x7ffb40003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb57fff640 1 --2- 172.21.15.29:0/3798587567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb400039c0 0x7ffb40003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb57fff640 1 -- 172.21.15.29:0/3798587567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb40003530 con 0x7ffb400051d0 2024-04-25T00:38:49.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb577fe640 1 --2- 172.21.15.29:0/3798587567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb400039c0 0x7ffb40003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:49.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb57fff640 1 -- 172.21.15.29:0/3798587567 reap_dead start 2024-04-25T00:38:49.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb57fff640 1 --2- 172.21.15.29:0/3798587567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb400051d0 0x7ffb4000dbe0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7ffb50001e70 tx=0x7ffb5001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cd6c88dfed8fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:49.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50001a90 con 0x7ffb400051d0 2024-04-25T00:38:49.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb50003bc0 con 0x7ffb400051d0 2024-04-25T00:38:49.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.494+0000 7ffb567fc640 1 -- 172.21.15.29:0/3798587567 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50005020 con 0x7ffb400051d0 2024-04-25T00:38:49.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 -- 172.21.15.29:0/3798587567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb400051d0 msgr2=0x7ffb4000dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 --2- 172.21.15.29:0/3798587567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb400051d0 0x7ffb4000dbe0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7ffb50001e70 tx=0x7ffb5001c780 comp rx=0 tx=0).stop 2024-04-25T00:38:49.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 -- 172.21.15.29:0/3798587567 shutdown_connections 2024-04-25T00:38:49.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 --2- 172.21.15.29:0/3798587567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb400051d0 0x7ffb4000dbe0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 -- 172.21.15.29:0/3798587567 >> 172.21.15.29:0/3798587567 conn(0x7ffb580b1870 msgr2=0x7ffb580b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:49.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 -- 172.21.15.29:0/3798587567 shutdown_connections 2024-04-25T00:38:49.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 -- 172.21.15.29:0/3798587567 wait complete. 2024-04-25T00:38:49.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 Processor -- start 2024-04-25T00:38:49.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 -- start start 2024-04-25T00:38:49.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb400051d0 0x7ffb5814c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb581f3b70 0x7ffb581f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb580b07f0 0x7ffb580b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffb580b42f0 con 0x7ffb581f3b70 2024-04-25T00:38:49.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb580b3580 con 0x7ffb580b07f0 2024-04-25T00:38:49.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffb5806cf10 con 0x7ffb400051d0 2024-04-25T00:38:49.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb56ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb581f3b70 0x7ffb581f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb56ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb581f3b70 0x7ffb581f28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50452/0 (socket says 172.21.15.29:50452) 2024-04-25T00:38:49.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb56ffd640 1 -- 172.21.15.29:0/295115424 learned_addr learned my addr 172.21.15.29:0/295115424 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:49.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb57fff640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb580b07f0 0x7ffb580b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb577fe640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb400051d0 0x7ffb5814c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb56ffd640 1 -- 172.21.15.29:0/295115424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb400051d0 msgr2=0x7ffb5814c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb56ffd640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb400051d0 0x7ffb5814c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb56ffd640 1 -- 172.21.15.29:0/295115424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb580b07f0 msgr2=0x7ffb580b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb56ffd640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb580b07f0 0x7ffb580b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb56ffd640 1 -- 172.21.15.29:0/295115424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb581f3ff0 con 0x7ffb581f3b70 2024-04-25T00:38:49.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb56ffd640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb581f3b70 0x7ffb581f28d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c011590 tx=0x7ffb4c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:49.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb3ffff640 1 -- 172.21.15.29:0/295115424 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c015870 con 0x7ffb581f3b70 2024-04-25T00:38:49.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb3ffff640 1 -- 172.21.15.29:0/295115424 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb4c015a10 con 0x7ffb581f3b70 2024-04-25T00:38:49.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb3ffff640 1 -- 172.21.15.29:0/295115424 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c015dc0 con 0x7ffb581f3b70 2024-04-25T00:38:49.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 -- 172.21.15.29:0/295115424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5814c720 con 0x7ffb581f3b70 2024-04-25T00:38:49.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.498+0000 7ffb5de2c640 1 -- 172.21.15.29:0/295115424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb5814cce0 con 0x7ffb581f3b70 2024-04-25T00:38:49.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.502+0000 7ffb5de2c640 1 -- 172.21.15.29:0/295115424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb580b11b0 con 0x7ffb581f3b70 2024-04-25T00:38:49.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.502+0000 7ffb3ffff640 1 -- 172.21.15.29:0/295115424 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7ffb4c01d400 con 0x7ffb581f3b70 2024-04-25T00:38:49.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.502+0000 7ffb3ffff640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb44067e90 0x7ffb4406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.502+0000 7ffb3ffff640 1 -- 172.21.15.29:0/295115424 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7ffb4c086ae0 con 0x7ffb581f3b70 2024-04-25T00:38:49.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.502+0000 7ffb577fe640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb44067e90 0x7ffb4406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.502+0000 7ffb577fe640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb44067e90 0x7ffb4406a350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffb58139370 tx=0x7ffb48009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:49.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.506+0000 7ffb3ffff640 1 -- 172.21.15.29:0/295115424 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffb4c019030 con 0x7ffb581f3b70 2024-04-25T00:38:49.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.670+0000 7ffb5de2c640 1 -- 172.21.15.29:0/295115424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb58069270 con 0x7ffb581f3b70 2024-04-25T00:38:49.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.674+0000 7ffb3ffff640 1 -- 172.21.15.29:0/295115424 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16631 (secure 0 0 0) 0x7ffb4c056db0 con 0x7ffb581f3b70 2024-04-25T00:38:49.677 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:49.677 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":118,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:32.730257+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:38:28.743593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:38:30.051271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:49.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.674+0000 7ffb5de2c640 1 -- 172.21.15.29:0/295115424 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb44067e90 msgr2=0x7ffb4406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.674+0000 7ffb5de2c640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb44067e90 0x7ffb4406a350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffb58139370 tx=0x7ffb48009040 comp rx=0 tx=0).stop 2024-04-25T00:38:49.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.674+0000 7ffb5de2c640 1 -- 172.21.15.29:0/295115424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb581f3b70 msgr2=0x7ffb581f28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.674+0000 7ffb5de2c640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb581f3b70 0x7ffb581f28d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c011590 tx=0x7ffb4c011a50 comp rx=0 tx=0).stop 2024-04-25T00:38:49.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.674+0000 7ffb5de2c640 1 -- 172.21.15.29:0/295115424 shutdown_connections 2024-04-25T00:38:49.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.674+0000 7ffb5de2c640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb44067e90 0x7ffb4406a350 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.674+0000 7ffb5de2c640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb580b07f0 0x7ffb580b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.674+0000 7ffb5de2c640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb581f3b70 0x7ffb581f28d0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.674+0000 7ffb5de2c640 1 --2- 172.21.15.29:0/295115424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb400051d0 0x7ffb5814c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.674+0000 7ffb5de2c640 1 -- 172.21.15.29:0/295115424 >> 172.21.15.29:0/295115424 conn(0x7ffb580b1870 msgr2=0x7ffb580afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:49.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.674+0000 7ffb5de2c640 1 -- 172.21.15.29:0/295115424 shutdown_connections 2024-04-25T00:38:49.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.674+0000 7ffb5de2c640 1 -- 172.21.15.29:0/295115424 wait complete. 2024-04-25T00:38:49.697 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:38:28.743593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:38:30.051271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:38:49.697 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:38:49.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff28e52f640 1 Processor -- start 2024-04-25T00:38:49.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff28e52f640 1 -- start start 2024-04-25T00:38:49.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff28e52f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff28815e8d0 0x7ff2880cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff28e52f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2880b3720 con 0x7ff2880b4780 2024-04-25T00:38:49.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff28e52f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2880b39c0 con 0x7ff28815f3c0 2024-04-25T00:38:49.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff28e52f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff2880b42f0 con 0x7ff28815e8d0 2024-04-25T00:38:49.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff27f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff2880b4780 0x7ff28815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36792/0 (socket says 172.21.15.29:36792) 2024-04-25T00:38:49.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff27f7fe640 1 -- 172.21.15.29:0/2378480452 learned_addr learned my addr 172.21.15.29:0/2378480452 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff287fff640 1 --2- 172.21.15.29:0/2378480452 >> v2:172.21.15.157:3300/0 conn(0x7ff28815e8d0 0x7ff2880cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2263545847 0 0) 0x7ff2880b3720 con 0x7ff2880b4780 2024-04-25T00:38:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff274003530 con 0x7ff2880b4780 2024-04-25T00:38:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1402959208 0 0) 0x7ff274003530 con 0x7ff2880b4780 2024-04-25T00:38:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 >> v2:172.21.15.157:3300/0 conn(0x7ff28815e8d0 msgr2=0x7ff2880cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 --2- 172.21.15.29:0/2378480452 >> v2:172.21.15.157:3300/0 conn(0x7ff28815e8d0 0x7ff2880cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 >> v1:172.21.15.157:6789/0 conn(0x7ff28815f3c0 legacy=0x7ff28815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2880b3ca0 con 0x7ff2880b4780 2024-04-25T00:38:49.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ff270001680 con 0x7ff2880b4780 2024-04-25T00:38:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 >> v1:172.21.15.29:6789/0 conn(0x7ff2880b4780 legacy=0x7ff28815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 --2- 172.21.15.29:0/2378480452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2740039c0 0x7ff274003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 --2- 172.21.15.29:0/2378480452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff274004810 0x7ff274004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 --2- 172.21.15.29:0/2378480452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2740051d0 0x7ff27400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff2880b42f0 con 0x7ff2740039c0 2024-04-25T00:38:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff27400c070 con 0x7ff2740051d0 2024-04-25T00:38:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff27400c270 con 0x7ff274004810 2024-04-25T00:38:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff287fff640 1 --2- 172.21.15.29:0/2378480452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2740039c0 0x7ff274003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff28cacc640 1 --2- 172.21.15.29:0/2378480452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2740051d0 0x7ff27400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff27f7fe640 1 --2- 172.21.15.29:0/2378480452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff274004810 0x7ff274004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff287fff640 1 -- 172.21.15.29:0/2378480452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff274004810 msgr2=0x7ff274004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff287fff640 1 --2- 172.21.15.29:0/2378480452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff274004810 0x7ff274004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff287fff640 1 -- 172.21.15.29:0/2378480452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2740051d0 msgr2=0x7ff27400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:49.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff287fff640 1 --2- 172.21.15.29:0/2378480452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2740051d0 0x7ff27400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff287fff640 1 -- 172.21.15.29:0/2378480452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff274003530 con 0x7ff2740039c0 2024-04-25T00:38:49.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff28cacc640 1 -- 172.21.15.29:0/2378480452 reap_dead start 2024-04-25T00:38:49.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.822+0000 7ff27f7fe640 1 --2- 172.21.15.29:0/2378480452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff274004810 0x7ff274004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:49.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff287fff640 1 --2- 172.21.15.29:0/2378480452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2740039c0 0x7ff274003de0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7ff278009ae0 tx=0x7ff27801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b51ab8b1098132 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:49.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780096b0 con 0x7ff2740039c0 2024-04-25T00:38:49.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2780a6450 con 0x7ff2740039c0 2024-04-25T00:38:49.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff2877fe640 1 -- 172.21.15.29:0/2378480452 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780a6840 con 0x7ff2740039c0 2024-04-25T00:38:49.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 -- 172.21.15.29:0/2378480452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2740039c0 msgr2=0x7ff274003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 --2- 172.21.15.29:0/2378480452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2740039c0 0x7ff274003de0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7ff278009ae0 tx=0x7ff27801ca70 comp rx=0 tx=0).stop 2024-04-25T00:38:49.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 -- 172.21.15.29:0/2378480452 shutdown_connections 2024-04-25T00:38:49.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 --2- 172.21.15.29:0/2378480452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2740039c0 0x7ff274003de0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 -- 172.21.15.29:0/2378480452 >> 172.21.15.29:0/2378480452 conn(0x7ff2880b1870 msgr2=0x7ff2880b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:49.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 -- 172.21.15.29:0/2378480452 shutdown_connections 2024-04-25T00:38:49.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 -- 172.21.15.29:0/2378480452 wait complete. 2024-04-25T00:38:49.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 Processor -- start 2024-04-25T00:38:49.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 -- start start 2024-04-25T00:38:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2740039c0 0x7ff28813e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff288156950 0x7ff2881f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2881ead60 0x7ff2881eb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff288144a20 con 0x7ff2740039c0 2024-04-25T00:38:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff28815bab0 con 0x7ff2881ead60 2024-04-25T00:38:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff28e52f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff288156ec0 con 0x7ff288156950 2024-04-25T00:38:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff287fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2740039c0 0x7ff28813e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff287fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2740039c0 0x7ff28813e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50470/0 (socket says 172.21.15.29:50470) 2024-04-25T00:38:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.826+0000 7ff287fff640 1 -- 172.21.15.29:0/2982317188 learned_addr learned my addr 172.21.15.29:0/2982317188 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:49.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff28cacc640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2881ead60 0x7ff2881eb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff27f7fe640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff288156950 0x7ff2881f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff287fff640 1 -- 172.21.15.29:0/2982317188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff288156950 msgr2=0x7ff2881f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff287fff640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff288156950 0x7ff2881f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff287fff640 1 -- 172.21.15.29:0/2982317188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2881ead60 msgr2=0x7ff2881eb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff287fff640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2881ead60 0x7ff2881eb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff287fff640 1 -- 172.21.15.29:0/2982317188 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2881571a0 con 0x7ff2740039c0 2024-04-25T00:38:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff287fff640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2740039c0 0x7ff28813e730 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7ff278009ae0 tx=0x7ff278019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff2857fa640 1 -- 172.21.15.29:0/2982317188 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780a6d00 con 0x7ff2740039c0 2024-04-25T00:38:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff2857fa640 1 -- 172.21.15.29:0/2982317188 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2780b9440 con 0x7ff2740039c0 2024-04-25T00:38:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff2857fa640 1 -- 172.21.15.29:0/2982317188 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780b9810 con 0x7ff2740039c0 2024-04-25T00:38:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff28e52f640 1 -- 172.21.15.29:0/2982317188 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff288157480 con 0x7ff2740039c0 2024-04-25T00:38:49.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff28e52f640 1 -- 172.21.15.29:0/2982317188 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff288157a40 con 0x7ff2740039c0 2024-04-25T00:38:49.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff28e52f640 1 -- 172.21.15.29:0/2982317188 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2881eb720 con 0x7ff2740039c0 2024-04-25T00:38:49.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff2857fa640 1 -- 172.21.15.29:0/2982317188 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7ff2780a7430 con 0x7ff2740039c0 2024-04-25T00:38:49.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff2857fa640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff268067e60 0x7ff26806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:49.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff2857fa640 1 -- 172.21.15.29:0/2982317188 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7ff278120450 con 0x7ff2740039c0 2024-04-25T00:38:49.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff27f7fe640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff268067e60 0x7ff26806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:49.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.830+0000 7ff27f7fe640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff268067e60 0x7ff26806a320 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff2700017f0 tx=0x7ff270009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:49.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.834+0000 7ff2857fa640 1 -- 172.21.15.29:0/2982317188 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff2780bf060 con 0x7ff2740039c0 2024-04-25T00:38:50.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff28e52f640 1 -- 172.21.15.29:0/2982317188 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff288069270 con 0x7ff2740039c0 2024-04-25T00:38:50.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff2857fa640 1 -- 172.21.15.29:0/2982317188 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7ff2780ac020 con 0x7ff2740039c0 2024-04-25T00:38:50.001 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:38:50.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff28e52f640 1 -- 172.21.15.29:0/2982317188 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff268067e60 msgr2=0x7ff26806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff28e52f640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff268067e60 0x7ff26806a320 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff2700017f0 tx=0x7ff270009040 comp rx=0 tx=0).stop 2024-04-25T00:38:50.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff28e52f640 1 -- 172.21.15.29:0/2982317188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2740039c0 msgr2=0x7ff28813e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff28e52f640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2740039c0 0x7ff28813e730 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7ff278009ae0 tx=0x7ff278019810 comp rx=0 tx=0).stop 2024-04-25T00:38:50.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff28e52f640 1 -- 172.21.15.29:0/2982317188 shutdown_connections 2024-04-25T00:38:50.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff28e52f640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff268067e60 0x7ff26806a320 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff28e52f640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2881ead60 0x7ff2881eb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff28e52f640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff288156950 0x7ff2881f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff28e52f640 1 --2- 172.21.15.29:0/2982317188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2740039c0 0x7ff28813e730 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff28e52f640 1 -- 172.21.15.29:0/2982317188 >> 172.21.15.29:0/2982317188 conn(0x7ff2880b1870 msgr2=0x7ff28806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:50.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff28e52f640 1 -- 172.21.15.29:0/2982317188 shutdown_connections 2024-04-25T00:38:50.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:49.998+0000 7ff28e52f640 1 -- 172.21.15.29:0/2982317188 wait complete. 2024-04-25T00:38:50.021 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:38:50.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44be90c640 1 Processor -- start 2024-04-25T00:38:50.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44be90c640 1 -- start start 2024-04-25T00:38:50.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44be90c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44b815ecf0 0x7f44b80ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44be90c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44b815f2a0 con 0x7f44b8072990 2024-04-25T00:38:50.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44be90c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44b8071eb0 con 0x7f44b806b210 2024-04-25T00:38:50.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44be90c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f44b80b03f0 con 0x7f44b815ecf0 2024-04-25T00:38:50.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44affff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f44b8072990 0x7f44b815e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36794/0 (socket says 172.21.15.29:36794) 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44affff640 1 -- 172.21.15.29:0/3297862860 learned_addr learned my addr 172.21.15.29:0/3297862860 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44bcea9640 1 --2- 172.21.15.29:0/3297862860 >> v2:172.21.15.157:3300/0 conn(0x7f44b815ecf0 0x7f44b80ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1451879839 0 0) 0x7f44b815f2a0 con 0x7f44b8072990 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44a4003530 con 0x7f44b8072990 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 447836257 0 0) 0x7f44a4003530 con 0x7f44b8072990 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 >> v2:172.21.15.157:3300/0 conn(0x7f44b815ecf0 msgr2=0x7f44b80ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 --2- 172.21.15.29:0/3297862860 >> v2:172.21.15.157:3300/0 conn(0x7f44b815ecf0 0x7f44b80ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 >> v1:172.21.15.157:6789/0 conn(0x7f44b806b210 legacy=0x7f44b806b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b8072190 con 0x7f44b8072990 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f44a0001680 con 0x7f44b8072990 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 >> v1:172.21.15.29:6789/0 conn(0x7f44b8072990 legacy=0x7f44b815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 --2- 172.21.15.29:0/3297862860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44a40039c0 0x7f44a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 --2- 172.21.15.29:0/3297862860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a4004810 0x7f44a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 --2- 172.21.15.29:0/3297862860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a40052f0 0x7f44a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f44b80b03f0 con 0x7f44a40039c0 2024-04-25T00:38:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44a400c1c0 con 0x7f44a40052f0 2024-04-25T00:38:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44a400c3c0 con 0x7f44a4004810 2024-04-25T00:38:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b7fff640 1 --2- 172.21.15.29:0/3297862860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44a40039c0 0x7f44a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44affff640 1 --2- 172.21.15.29:0/3297862860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a4004810 0x7f44a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44bcea9640 1 --2- 172.21.15.29:0/3297862860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a40052f0 0x7f44a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b7fff640 1 -- 172.21.15.29:0/3297862860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a4004810 msgr2=0x7f44a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b7fff640 1 --2- 172.21.15.29:0/3297862860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a4004810 0x7f44a4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b7fff640 1 -- 172.21.15.29:0/3297862860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a40052f0 msgr2=0x7f44a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b7fff640 1 --2- 172.21.15.29:0/3297862860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a40052f0 0x7f44a400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44b7fff640 1 -- 172.21.15.29:0/3297862860 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a4003530 con 0x7f44a40039c0 2024-04-25T00:38:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.142+0000 7f44bcea9640 1 -- 172.21.15.29:0/3297862860 reap_dead start 2024-04-25T00:38:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.146+0000 7f44b7fff640 1 --2- 172.21.15.29:0/3297862860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44a40039c0 0x7f44a4003de0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f44a8001750 tx=0x7f44a801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c484538e85056cc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.146+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a80013a0 con 0x7f44a40039c0 2024-04-25T00:38:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.146+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a80052f0 con 0x7f44a40039c0 2024-04-25T00:38:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.146+0000 7f44b77fe640 1 -- 172.21.15.29:0/3297862860 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8003840 con 0x7f44a40039c0 2024-04-25T00:38:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.146+0000 7f44be90c640 1 -- 172.21.15.29:0/3297862860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44a40039c0 msgr2=0x7f44a4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.146+0000 7f44be90c640 1 --2- 172.21.15.29:0/3297862860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44a40039c0 0x7f44a4003de0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f44a8001750 tx=0x7f44a801c3c0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.146+0000 7f44be90c640 1 -- 172.21.15.29:0/3297862860 shutdown_connections 2024-04-25T00:38:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.146+0000 7f44be90c640 1 --2- 172.21.15.29:0/3297862860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44a40039c0 0x7f44a4003de0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.146+0000 7f44be90c640 1 -- 172.21.15.29:0/3297862860 >> 172.21.15.29:0/3297862860 conn(0x7f44b80d0c40 msgr2=0x7f44b80d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.146+0000 7f44be90c640 1 -- 172.21.15.29:0/3297862860 shutdown_connections 2024-04-25T00:38:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.146+0000 7f44be90c640 1 -- 172.21.15.29:0/3297862860 wait complete. 2024-04-25T00:38:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44be90c640 1 Processor -- start 2024-04-25T00:38:50.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44be90c640 1 -- start start 2024-04-25T00:38:50.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44be90c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a40039c0 0x7f44b8148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44be90c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44b8158ed0 0x7f44b814b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44be90c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44b814b900 0x7f44b80c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44be90c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f44b8072500 con 0x7f44b8158ed0 2024-04-25T00:38:50.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44be90c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44b815f580 con 0x7f44b814b900 2024-04-25T00:38:50.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44be90c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44b80cbf80 con 0x7f44a40039c0 2024-04-25T00:38:50.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44affff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44b8158ed0 0x7f44b814b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44affff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44b8158ed0 0x7f44b814b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50496/0 (socket says 172.21.15.29:50496) 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44affff640 1 -- 172.21.15.29:0/2902757577 learned_addr learned my addr 172.21.15.29:0/2902757577 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44b7fff640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a40039c0 0x7f44b8148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44bcea9640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44b814b900 0x7f44b80c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44affff640 1 -- 172.21.15.29:0/2902757577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a40039c0 msgr2=0x7f44b8148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44affff640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a40039c0 0x7f44b8148c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44affff640 1 -- 172.21.15.29:0/2902757577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44b814b900 msgr2=0x7f44b80c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44affff640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44b814b900 0x7f44b80c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44affff640 1 -- 172.21.15.29:0/2902757577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b806e930 con 0x7f44b8158ed0 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44affff640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44b8158ed0 0x7f44b814b3c0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f44a000e5d0 tx=0x7f44a000ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44b57fa640 1 -- 172.21.15.29:0/2902757577 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0012590 con 0x7f44b8158ed0 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44b57fa640 1 -- 172.21.15.29:0/2902757577 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a0018070 con 0x7f44b8158ed0 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44be90c640 1 -- 172.21.15.29:0/2902757577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b806ec10 con 0x7f44b8158ed0 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44b57fa640 1 -- 172.21.15.29:0/2902757577 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0012970 con 0x7f44b8158ed0 2024-04-25T00:38:50.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44be90c640 1 -- 172.21.15.29:0/2902757577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b806f1d0 con 0x7f44b8158ed0 2024-04-25T00:38:50.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44be90c640 1 -- 172.21.15.29:0/2902757577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b8069270 con 0x7f44b8158ed0 2024-04-25T00:38:50.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.150+0000 7f44b57fa640 1 -- 172.21.15.29:0/2902757577 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f44a0012b10 con 0x7f44b8158ed0 2024-04-25T00:38:50.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.154+0000 7f44b57fa640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4498067e60 0x7f449806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.154+0000 7f44b57fa640 1 -- 172.21.15.29:0/2902757577 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f44a0086370 con 0x7f44b8158ed0 2024-04-25T00:38:50.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.154+0000 7f44b7fff640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4498067e60 0x7f449806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.154+0000 7f44b7fff640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4498067e60 0x7f449806a320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f44a40057f0 tx=0x7f44a80011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:50.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.158+0000 7f44b57fa640 1 -- 172.21.15.29:0/2902757577 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f44a0056600 con 0x7f44b8158ed0 2024-04-25T00:38:50.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44be90c640 1 -- 172.21.15.29:0/2902757577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f44b814bce0 con 0x7f44b8158ed0 2024-04-25T00:38:50.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44b57fa640 1 -- 172.21.15.29:0/2902757577 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) ==== 104+0+11 (secure 0 0 0) 0x7f44a0055f00 con 0x7f44b8158ed0 2024-04-25T00:38:50.321 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:38:50.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44be90c640 1 -- 172.21.15.29:0/2902757577 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4498067e60 msgr2=0x7f449806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44be90c640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4498067e60 0x7f449806a320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f44a40057f0 tx=0x7f44a80011a0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44be90c640 1 -- 172.21.15.29:0/2902757577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44b8158ed0 msgr2=0x7f44b814b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44be90c640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44b8158ed0 0x7f44b814b3c0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f44a000e5d0 tx=0x7f44a000ea90 comp rx=0 tx=0).stop 2024-04-25T00:38:50.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44be90c640 1 -- 172.21.15.29:0/2902757577 shutdown_connections 2024-04-25T00:38:50.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44be90c640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4498067e60 0x7f449806a320 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44be90c640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44b814b900 0x7f44b80c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44be90c640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44b8158ed0 0x7f44b814b3c0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44be90c640 1 --2- 172.21.15.29:0/2902757577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44a40039c0 0x7f44b8148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44be90c640 1 -- 172.21.15.29:0/2902757577 >> 172.21.15.29:0/2902757577 conn(0x7f44b80d0c40 msgr2=0x7f44b815c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:50.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44be90c640 1 -- 172.21.15.29:0/2902757577 shutdown_connections 2024-04-25T00:38:50.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.318+0000 7f44be90c640 1 -- 172.21.15.29:0/2902757577 wait complete. 2024-04-25T00:38:50.340 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:38:50.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b7418e640 1 Processor -- start 2024-04-25T00:38:50.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b7418e640 1 -- start start 2024-04-25T00:38:50.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b7418e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b6c06b210 0x7f8b6c072990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b7418e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b6c15ecf0 0x7f8b6c0ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b7418e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b6c15f2a0 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b7418e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8b6c0b03f0 con 0x7f8b6c15ecf0 2024-04-25T00:38:50.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b7418e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6c15f580 con 0x7f8b6c06b210 2024-04-25T00:38:50.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b71f2a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b6c06b210 0x7f8b6c072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b7272b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b6c15ecf0 0x7f8b6c0ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b71729640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8b6c06b5f0 0x7f8b6c15e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:37866/0 (socket says 172.21.15.29:37866) 2024-04-25T00:38:50.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b71729640 1 -- 172.21.15.29:0/2066260427 learned_addr learned my addr 172.21.15.29:0/2066260427 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:50.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b7272b640 1 -- 172.21.15.29:0/2066260427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b6c06b210 msgr2=0x7f8b6c072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:50.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b7272b640 1 --2- 172.21.15.29:0/2066260427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b6c06b210 0x7f8b6c072990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b7272b640 1 -- 172.21.15.29:0/2066260427 >> v1:172.21.15.157:6789/0 conn(0x7f8b6c06b5f0 legacy=0x7f8b6c15e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.462+0000 7f8b7272b640 1 -- 172.21.15.29:0/2066260427 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6c071eb0 con 0x7f8b6c15ecf0 2024-04-25T00:38:50.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.466+0000 7f8b7272b640 1 --2- 172.21.15.29:0/2066260427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b6c15ecf0 0x7f8b6c0ad8b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8b640097c0 tx=0x7f8b6401c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14aa5c72268d65cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:50.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.470+0000 7f8b70f28640 1 -- 172.21.15.29:0/2066260427 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b640b0070 con 0x7f8b6c15ecf0 2024-04-25T00:38:50.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.470+0000 7f8b70f28640 1 -- 172.21.15.29:0/2066260427 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b640ab600 con 0x7f8b6c15ecf0 2024-04-25T00:38:50.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.470+0000 7f8b70f28640 1 -- 172.21.15.29:0/2066260427 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b640ab9d0 con 0x7f8b6c15ecf0 2024-04-25T00:38:50.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.470+0000 7f8b7418e640 1 -- 172.21.15.29:0/2066260427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b6c15ecf0 msgr2=0x7f8b6c0ad8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.470+0000 7f8b7418e640 1 --2- 172.21.15.29:0/2066260427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b6c15ecf0 0x7f8b6c0ad8b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8b640097c0 tx=0x7f8b6401c380 comp rx=0 tx=0).stop 2024-04-25T00:38:50.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.470+0000 7f8b7418e640 1 -- 172.21.15.29:0/2066260427 shutdown_connections 2024-04-25T00:38:50.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.470+0000 7f8b7418e640 1 --2- 172.21.15.29:0/2066260427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b6c15ecf0 0x7f8b6c0ad8b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.470+0000 7f8b7418e640 1 --2- 172.21.15.29:0/2066260427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b6c06b210 0x7f8b6c072990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.470+0000 7f8b7418e640 1 -- 172.21.15.29:0/2066260427 >> 172.21.15.29:0/2066260427 conn(0x7f8b6c0d0c40 msgr2=0x7f8b6c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:50.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.470+0000 7f8b7418e640 1 -- 172.21.15.29:0/2066260427 shutdown_connections 2024-04-25T00:38:50.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.470+0000 7f8b7418e640 1 -- 172.21.15.29:0/2066260427 wait complete. 2024-04-25T00:38:50.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.470+0000 7f8b7418e640 1 Processor -- start 2024-04-25T00:38:50.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.470+0000 7f8b7418e640 1 -- start start 2024-04-25T00:38:50.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b7418e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b6c06b210 0x7f8b6c1603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b7418e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b6c06b5f0 0x7f8b6c1608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b7418e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b6c15ecf0 0x7f8b6c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b7418e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c16af70 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b7418e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c0cbf80 con 0x7f8b6c15ecf0 2024-04-25T00:38:50.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b7418e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6c16b0f0 con 0x7f8b6c06b210 2024-04-25T00:38:50.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b71729640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b6c06b5f0 0x7f8b6c1608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b71729640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b6c06b5f0 0x7f8b6c1608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50504/0 (socket says 172.21.15.29:50504) 2024-04-25T00:38:50.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b71729640 1 -- 172.21.15.29:0/270859358 learned_addr learned my addr 172.21.15.29:0/270859358 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b7272b640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b6c15ecf0 0x7f8b6c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b71f2a640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b6c06b210 0x7f8b6c1603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b71729640 1 -- 172.21.15.29:0/270859358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b6c06b210 msgr2=0x7f8b6c1603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b71729640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b6c06b210 0x7f8b6c1603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b71729640 1 -- 172.21.15.29:0/270859358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b6c15ecf0 msgr2=0x7f8b6c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b71729640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b6c15ecf0 0x7f8b6c0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b71729640 1 -- 172.21.15.29:0/270859358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6c0b6220 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b7272b640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b6c15ecf0 0x7f8b6c0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b71729640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b6c06b5f0 0x7f8b6c1608e0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000ca70 tx=0x7f8b6000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/270859358 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60017070 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/270859358 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b6000ab70 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b7418e640 1 -- 172.21.15.29:0/270859358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6c160f10 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/270859358 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60012400 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b7418e640 1 -- 172.21.15.29:0/270859358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6c0c8b80 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b7418e640 1 -- 172.21.15.29:0/270859358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b6c0c8990 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/270859358 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f8b6001a070 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b6a7fc640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b58067e90 0x7f8b5806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/270859358 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f8b60086330 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b71f2a640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b58067e90 0x7f8b5806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.474+0000 7f8b71f2a640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b58067e90 0x7f8b5806a350 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009a50 tx=0x7f8b5c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.482+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/270859358 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8b60056610 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.642+0000 7f8b7418e640 1 -- 172.21.15.29:0/270859358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8b6c0cb8f0 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.642+0000 7f8b6a7fc640 1 -- 172.21.15.29:0/270859358 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) ==== 100+0+11 (secure 0 0 0) 0x7f8b60055f10 con 0x7f8b6c06b5f0 2024-04-25T00:38:50.646 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:38:50.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.646+0000 7f8b7418e640 1 -- 172.21.15.29:0/270859358 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b58067e90 msgr2=0x7f8b5806a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.646+0000 7f8b7418e640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b58067e90 0x7f8b5806a350 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009a50 tx=0x7f8b5c008040 comp rx=0 tx=0).stop 2024-04-25T00:38:50.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.646+0000 7f8b7418e640 1 -- 172.21.15.29:0/270859358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b6c06b5f0 msgr2=0x7f8b6c1608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.646+0000 7f8b7418e640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b6c06b5f0 0x7f8b6c1608e0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000ca70 tx=0x7f8b6000cf30 comp rx=0 tx=0).stop 2024-04-25T00:38:50.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.646+0000 7f8b7418e640 1 -- 172.21.15.29:0/270859358 shutdown_connections 2024-04-25T00:38:50.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.646+0000 7f8b7418e640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b58067e90 0x7f8b5806a350 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.646+0000 7f8b7418e640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b6c15ecf0 0x7f8b6c0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.646+0000 7f8b7418e640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b6c06b5f0 0x7f8b6c1608e0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.646+0000 7f8b7418e640 1 --2- 172.21.15.29:0/270859358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b6c06b210 0x7f8b6c1603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.646+0000 7f8b7418e640 1 -- 172.21.15.29:0/270859358 >> 172.21.15.29:0/270859358 conn(0x7f8b6c0d0c40 msgr2=0x7f8b6c0701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:50.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.646+0000 7f8b7418e640 1 -- 172.21.15.29:0/270859358 shutdown_connections 2024-04-25T00:38:50.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.646+0000 7f8b7418e640 1 -- 172.21.15.29:0/270859358 wait complete. 2024-04-25T00:38:50.665 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:38:50.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddba3d7640 1 Processor -- start 2024-04-25T00:38:50.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddba3d7640 1 -- start start 2024-04-25T00:38:50.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddba3d7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fddb40b4780 0x7fddb415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddba3d7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb40cc240 0x7fddb415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddba3d7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddb40b3720 con 0x7fddb415e8d0 2024-04-25T00:38:50.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddba3d7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fddb40b42f0 con 0x7fddb40b4780 2024-04-25T00:38:50.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddba3d7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fddb40b39c0 con 0x7fddb40cc240 2024-04-25T00:38:50.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddb3fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fddb415e8d0 0x7fddb415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36800/0 (socket says 172.21.15.29:36800) 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddb3fff640 1 -- 172.21.15.29:0/1117979032 learned_addr learned my addr 172.21.15.29:0/1117979032 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddb37fe640 1 --2- 172.21.15.29:0/1117979032 >> v2:172.21.15.157:3300/0 conn(0x7fddb40b4780 0x7fddb415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddb8974640 1 --2- 172.21.15.29:0/1117979032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb40cc240 0x7fddb415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 941334855 0 0) 0x7fddb40b3720 con 0x7fddb415e8d0 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdda0003530 con 0x7fddb415e8d0 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.786+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3542198806 0 0) 0x7fdda0003530 con 0x7fddb415e8d0 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb40cc240 msgr2=0x7fddb415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb2ffd640 1 --2- 172.21.15.29:0/1117979032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb40cc240 0x7fddb415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 >> v2:172.21.15.157:3300/0 conn(0x7fddb40b4780 msgr2=0x7fddb415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb2ffd640 1 --2- 172.21.15.29:0/1117979032 >> v2:172.21.15.157:3300/0 conn(0x7fddb40b4780 0x7fddb415f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb40b3ca0 con 0x7fddb415e8d0 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fdda4001a00 con 0x7fddb415e8d0 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb37fe640 1 --2- 172.21.15.29:0/1117979032 >> v2:172.21.15.157:3300/0 conn(0x7fddb40b4780 0x7fddb415f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:38:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 >> v1:172.21.15.29:6789/0 conn(0x7fddb415e8d0 legacy=0x7fddb415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb2ffd640 1 --2- 172.21.15.29:0/1117979032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00039c0 0x7fdda0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb2ffd640 1 --2- 172.21.15.29:0/1117979032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb40cc240 0x7fdda0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb2ffd640 1 --2- 172.21.15.29:0/1117979032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda0004d50 0x7fdda00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fddb40b42f0 con 0x7fdda00039c0 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fddb40b39c0 con 0x7fdda0004d50 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdda000b9d0 con 0x7fddb40cc240 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb3fff640 1 --2- 172.21.15.29:0/1117979032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00039c0 0x7fdda0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb37fe640 1 --2- 172.21.15.29:0/1117979032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb40cc240 0x7fdda0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb8974640 1 --2- 172.21.15.29:0/1117979032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda0004d50 0x7fdda00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb3fff640 1 -- 172.21.15.29:0/1117979032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb40cc240 msgr2=0x7fdda0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb3fff640 1 --2- 172.21.15.29:0/1117979032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb40cc240 0x7fdda0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb3fff640 1 -- 172.21.15.29:0/1117979032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda0004d50 msgr2=0x7fdda00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb3fff640 1 --2- 172.21.15.29:0/1117979032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda0004d50 0x7fdda00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.790+0000 7fddb3fff640 1 -- 172.21.15.29:0/1117979032 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda0003530 con 0x7fdda00039c0 2024-04-25T00:38:50.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddb3fff640 1 --2- 172.21.15.29:0/1117979032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00039c0 0x7fdda0003e00 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fdda400a920 tx=0x7fdda401d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec5c023567e85a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:50.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda400a4c0 con 0x7fdda00039c0 2024-04-25T00:38:50.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda40ad630 con 0x7fdda00039c0 2024-04-25T00:38:50.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddb2ffd640 1 -- 172.21.15.29:0/1117979032 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda40ad9e0 con 0x7fdda00039c0 2024-04-25T00:38:50.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 -- 172.21.15.29:0/1117979032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00039c0 msgr2=0x7fdda0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 --2- 172.21.15.29:0/1117979032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda00039c0 0x7fdda0003e00 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fdda400a920 tx=0x7fdda401d440 comp rx=0 tx=0).stop 2024-04-25T00:38:50.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddb8974640 1 -- 172.21.15.29:0/1117979032 reap_dead start 2024-04-25T00:38:50.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 -- 172.21.15.29:0/1117979032 shutdown_connections 2024-04-25T00:38:50.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 -- 172.21.15.29:0/1117979032 >> 172.21.15.29:0/1117979032 conn(0x7fddb40b1870 msgr2=0x7fddb40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:50.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 -- 172.21.15.29:0/1117979032 shutdown_connections 2024-04-25T00:38:50.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 -- 172.21.15.29:0/1117979032 wait complete. 2024-04-25T00:38:50.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 Processor -- start 2024-04-25T00:38:50.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 -- start start 2024-04-25T00:38:50.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda00039c0 0x7fddb4134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddb415a4f0 0x7fddb4159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb4072700 0x7fddb415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fddb406df20 con 0x7fddb415a4f0 2024-04-25T00:38:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fddb406dda0 con 0x7fdda00039c0 2024-04-25T00:38:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddba3d7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fddb406e0a0 con 0x7fddb4072700 2024-04-25T00:38:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddb37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddb415a4f0 0x7fddb4159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddb37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddb415a4f0 0x7fddb4159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50522/0 (socket says 172.21.15.29:50522) 2024-04-25T00:38:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.794+0000 7fddb37fe640 1 -- 172.21.15.29:0/1563679808 learned_addr learned my addr 172.21.15.29:0/1563679808 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb3fff640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda00039c0 0x7fddb4134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb8974640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb4072700 0x7fddb415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb37fe640 1 -- 172.21.15.29:0/1563679808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb4072700 msgr2=0x7fddb415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb37fe640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb4072700 0x7fddb415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb37fe640 1 -- 172.21.15.29:0/1563679808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda00039c0 msgr2=0x7fddb4134fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb37fe640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda00039c0 0x7fddb4134fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb37fe640 1 -- 172.21.15.29:0/1563679808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb41341f0 con 0x7fddb415a4f0 2024-04-25T00:38:50.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb3fff640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda00039c0 0x7fddb4134fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:50.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb37fe640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddb415a4f0 0x7fddb4159ca0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c011d00 tx=0x7fdd9c00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:50.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb0ff9640 1 -- 172.21.15.29:0/1563679808 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c00fcc0 con 0x7fddb415a4f0 2024-04-25T00:38:50.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb0ff9640 1 -- 172.21.15.29:0/1563679808 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd9c00fe60 con 0x7fddb415a4f0 2024-04-25T00:38:50.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddba3d7640 1 -- 172.21.15.29:0/1563679808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb41344d0 con 0x7fddb415a4f0 2024-04-25T00:38:50.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb0ff9640 1 -- 172.21.15.29:0/1563679808 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c004240 con 0x7fddb415a4f0 2024-04-25T00:38:50.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddba3d7640 1 -- 172.21.15.29:0/1563679808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb415a1e0 con 0x7fddb415a4f0 2024-04-25T00:38:50.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddba3d7640 1 -- 172.21.15.29:0/1563679808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb4069270 con 0x7fddb415a4f0 2024-04-25T00:38:50.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb0ff9640 1 -- 172.21.15.29:0/1563679808 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fdd9c012040 con 0x7fddb415a4f0 2024-04-25T00:38:50.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb0ff9640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdd94067e90 0x7fdd9406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:50.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb0ff9640 1 -- 172.21.15.29:0/1563679808 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fdd9c086b10 con 0x7fddb415a4f0 2024-04-25T00:38:50.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb3fff640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdd94067e90 0x7fdd9406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:50.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.798+0000 7fddb3fff640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdd94067e90 0x7fdd9406a350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdda40ad000 tx=0x7fdda4001dd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:50.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.802+0000 7fddb0ff9640 1 -- 172.21.15.29:0/1563679808 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdd9c088e20 con 0x7fddb415a4f0 2024-04-25T00:38:50.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.970+0000 7fddba3d7640 1 -- 172.21.15.29:0/1563679808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fddb415d300 con 0x7fddb415a4f0 2024-04-25T00:38:50.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.970+0000 7fddb0ff9640 1 -- 172.21.15.29:0/1563679808 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v118) ==== 80+0+0 (secure 0 0 0) 0x7fdd9c019030 con 0x7fddb415a4f0 2024-04-25T00:38:50.973 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:38:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.974+0000 7fddba3d7640 1 -- 172.21.15.29:0/1563679808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdd94067e90 msgr2=0x7fdd9406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.974+0000 7fddba3d7640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdd94067e90 0x7fdd9406a350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdda40ad000 tx=0x7fdda4001dd0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.974+0000 7fddba3d7640 1 -- 172.21.15.29:0/1563679808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddb415a4f0 msgr2=0x7fddb4159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.974+0000 7fddba3d7640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddb415a4f0 0x7fddb4159ca0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c011d00 tx=0x7fdd9c00f610 comp rx=0 tx=0).stop 2024-04-25T00:38:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.974+0000 7fddba3d7640 1 -- 172.21.15.29:0/1563679808 shutdown_connections 2024-04-25T00:38:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.974+0000 7fddba3d7640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdd94067e90 0x7fdd9406a350 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.974+0000 7fddba3d7640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddb4072700 0x7fddb415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.974+0000 7fddba3d7640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddb415a4f0 0x7fddb4159ca0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.974+0000 7fddba3d7640 1 --2- 172.21.15.29:0/1563679808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdda00039c0 0x7fddb4134fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.974+0000 7fddba3d7640 1 -- 172.21.15.29:0/1563679808 >> 172.21.15.29:0/1563679808 conn(0x7fddb40b1870 msgr2=0x7fddb40b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.974+0000 7fddba3d7640 1 -- 172.21.15.29:0/1563679808 shutdown_connections 2024-04-25T00:38:50.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:50.974+0000 7fddba3d7640 1 -- 172.21.15.29:0/1563679808 wait complete. 2024-04-25T00:38:50.991 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-04-25T00:38:50.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:50.992 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:38:51.002 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[50588]: fuse finished with error 0 and tester_r 0 2024-04-25T00:38:51.004 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:38:51.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:51.004 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:38:51.170 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:38:51.170 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:38:51.170 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:38:51.176 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:51.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd9b47640 1 Processor -- start 2024-04-25T00:38:51.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd9b47640 1 -- start start 2024-04-25T00:38:51.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd9b47640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2fd40b4780 0x7f2fd415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd9b47640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2fd40cc240 0x7f2fd415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd9b47640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd40b3720 con 0x7f2fd415e8d0 2024-04-25T00:38:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd9b47640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd40cc650 con 0x7f2fd40cc240 2024-04-25T00:38:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd9b47640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd40b42f0 con 0x7f2fd40b4780 2024-04-25T00:38:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd3fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2fd40cc240 0x7f2fd415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd3fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2fd40cc240 0x7f2fd415c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50530/0 (socket says 172.21.15.29:50530) 2024-04-25T00:38:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd3fff640 1 -- 172.21.15.29:0/3077074267 learned_addr learned my addr 172.21.15.29:0/3077074267 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd2ffd640 1 --2- 172.21.15.29:0/3077074267 >> v2:172.21.15.157:3300/0 conn(0x7f2fd40b4780 0x7f2fd415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd3fff640 1 -- 172.21.15.29:0/3077074267 >> v2:172.21.15.157:3300/0 conn(0x7f2fd40b4780 msgr2=0x7f2fd415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd3fff640 1 --2- 172.21.15.29:0/3077074267 >> v2:172.21.15.157:3300/0 conn(0x7f2fd40b4780 0x7f2fd415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd3fff640 1 -- 172.21.15.29:0/3077074267 >> v1:172.21.15.157:6789/0 conn(0x7f2fd415e8d0 legacy=0x7f2fd415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.338+0000 7f2fd3fff640 1 -- 172.21.15.29:0/3077074267 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd40b39c0 con 0x7f2fd40cc240 2024-04-25T00:38:51.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd3fff640 1 --2- 172.21.15.29:0/3077074267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd40cc240 0x7f2fd415c7d0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4070910 tx=0x7f2fcc01eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c341cd799d2dea65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:51.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd27fc640 1 -- 172.21.15.29:0/3077074267 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc00b2d0 con 0x7f2fd40cc240 2024-04-25T00:38:51.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd27fc640 1 -- 172.21.15.29:0/3077074267 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fcc00b470 con 0x7f2fd40cc240 2024-04-25T00:38:51.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd27fc640 1 -- 172.21.15.29:0/3077074267 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0a7c70 con 0x7f2fd40cc240 2024-04-25T00:38:51.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3077074267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd40cc240 msgr2=0x7f2fd415c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:51.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 --2- 172.21.15.29:0/3077074267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd40cc240 0x7f2fd415c7d0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4070910 tx=0x7f2fcc01eb80 comp rx=0 tx=0).stop 2024-04-25T00:38:51.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3077074267 shutdown_connections 2024-04-25T00:38:51.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 --2- 172.21.15.29:0/3077074267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd40cc240 0x7f2fd415c7d0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 --2- 172.21.15.29:0/3077074267 >> v2:172.21.15.157:3300/0 conn(0x7f2fd40b4780 0x7f2fd415f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3077074267 >> 172.21.15.29:0/3077074267 conn(0x7f2fd40b1870 msgr2=0x7f2fd40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:51.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3077074267 shutdown_connections 2024-04-25T00:38:51.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3077074267 wait complete. 2024-04-25T00:38:51.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 Processor -- start 2024-04-25T00:38:51.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 -- start start 2024-04-25T00:38:51.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fd40b4780 0x7f2fd4156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:51.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fd40cc240 0x7f2fd40ce430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:51.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd415e8d0 0x7f2fd41f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:51.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd4144a20 con 0x7f2fd415e8d0 2024-04-25T00:38:51.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd415d520 con 0x7f2fd40cc240 2024-04-25T00:38:51.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.342+0000 7f2fd9b47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd40cf340 con 0x7f2fd40b4780 2024-04-25T00:38:51.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd415e8d0 0x7f2fd41f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:51.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd415e8d0 0x7f2fd41f2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50536/0 (socket says 172.21.15.29:50536) 2024-04-25T00:38:51.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd3fff640 1 -- 172.21.15.29:0/3533348146 learned_addr learned my addr 172.21.15.29:0/3533348146 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:51.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd2ffd640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fd40cc240 0x7f2fd40ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:51.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd37fe640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fd40b4780 0x7f2fd4156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:51.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd3fff640 1 -- 172.21.15.29:0/3533348146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fd40b4780 msgr2=0x7f2fd4156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd3fff640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fd40b4780 0x7f2fd4156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd3fff640 1 -- 172.21.15.29:0/3533348146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fd40cc240 msgr2=0x7f2fd40ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd3fff640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fd40cc240 0x7f2fd40ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd3fff640 1 -- 172.21.15.29:0/3533348146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd40ced80 con 0x7f2fd415e8d0 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd37fe640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fd40b4780 0x7f2fd4156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd3fff640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd415e8d0 0x7f2fd41f2a50 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc01c7b0 tx=0x7f2fcc01b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fbbfff640 1 -- 172.21.15.29:0/3533348146 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0be020 con 0x7f2fd415e8d0 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fbbfff640 1 -- 172.21.15.29:0/3533348146 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fcc009800 con 0x7f2fd415e8d0 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fbbfff640 1 -- 172.21.15.29:0/3533348146 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc009bf0 con 0x7f2fd415e8d0 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3533348146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd40cf060 con 0x7f2fd415e8d0 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3533348146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd413a3b0 con 0x7f2fd415e8d0 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3533348146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd413a5b0 con 0x7f2fd415e8d0 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fbbfff640 1 -- 172.21.15.29:0/3533348146 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f2fcc0c1020 con 0x7f2fd415e8d0 2024-04-25T00:38:51.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fbbfff640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2fc0067e60 0x7f2fc006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:51.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fbbfff640 1 -- 172.21.15.29:0/3533348146 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f2fcc123d30 con 0x7f2fd415e8d0 2024-04-25T00:38:51.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd37fe640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2fc0067e60 0x7f2fc006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:51.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.346+0000 7f2fd37fe640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2fc0067e60 0x7f2fc006a320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400a850 tx=0x7f2fc4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:51.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.350+0000 7f2fbbfff640 1 -- 172.21.15.29:0/3533348146 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2fd413a5b0 con 0x7f2fd415e8d0 2024-04-25T00:38:51.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.518+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3533348146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fd4069270 con 0x7f2fd415e8d0 2024-04-25T00:38:51.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.518+0000 7f2fbbfff640 1 -- 172.21.15.29:0/3533348146 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16742 (secure 0 0 0) 0x7f2fcc0c8030 con 0x7f2fd415e8d0 2024-04-25T00:38:51.522 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:51.522 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":119,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:51.470777+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:38:28.743593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:38:30.051271+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:51.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.522+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3533348146 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2fc0067e60 msgr2=0x7f2fc006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:51.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.522+0000 7f2fd9b47640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2fc0067e60 0x7f2fc006a320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400a850 tx=0x7f2fc4008040 comp rx=0 tx=0).stop 2024-04-25T00:38:51.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.522+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3533348146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd415e8d0 msgr2=0x7f2fd41f2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:51.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.522+0000 7f2fd9b47640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd415e8d0 0x7f2fd41f2a50 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc01c7b0 tx=0x7f2fcc01b920 comp rx=0 tx=0).stop 2024-04-25T00:38:51.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.522+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3533348146 shutdown_connections 2024-04-25T00:38:51.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.522+0000 7f2fd9b47640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2fc0067e60 0x7f2fc006a320 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.522+0000 7f2fd9b47640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd415e8d0 0x7f2fd41f2a50 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.522+0000 7f2fd9b47640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fd40cc240 0x7f2fd40ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.522+0000 7f2fd9b47640 1 --2- 172.21.15.29:0/3533348146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fd40b4780 0x7f2fd4156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.522+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3533348146 >> 172.21.15.29:0/3533348146 conn(0x7f2fd40b1870 msgr2=0x7f2fd4139810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:51.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.522+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3533348146 shutdown_connections 2024-04-25T00:38:51.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.522+0000 7f2fd9b47640 1 -- 172.21.15.29:0/3533348146 wait complete. 2024-04-25T00:38:51.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:38:28.743593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:38:30.051271+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 119, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:38:51.542 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:38:51.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd32df3b640 1 Processor -- start 2024-04-25T00:38:51.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd32df3b640 1 -- start start 2024-04-25T00:38:51.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd32df3b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd32815bdb0 0x7fd3280cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:51.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd32df3b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd3280d09e0 0x7fd3280ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:51.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd32df3b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3280cd3e0 con 0x7fd32815c190 2024-04-25T00:38:51.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd32df3b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd3280cd6c0 con 0x7fd3280d09e0 2024-04-25T00:38:51.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd32df3b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd3280cd9a0 con 0x7fd32815bdb0 2024-04-25T00:38:51.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd327fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd3280d09e0 0x7fd3280ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:51.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd327fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd3280d09e0 0x7fd3280ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50542/0 (socket says 172.21.15.29:50542) 2024-04-25T00:38:51.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd327fff640 1 -- 172.21.15.29:0/3517693545 learned_addr learned my addr 172.21.15.29:0/3517693545 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:51.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd3277fe640 1 --2- 172.21.15.29:0/3517693545 >> v2:172.21.15.157:3300/0 conn(0x7fd32815bdb0 0x7fd3280cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:51.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd327fff640 1 -- 172.21.15.29:0/3517693545 >> v2:172.21.15.157:3300/0 conn(0x7fd32815bdb0 msgr2=0x7fd3280cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:51.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd327fff640 1 --2- 172.21.15.29:0/3517693545 >> v2:172.21.15.157:3300/0 conn(0x7fd32815bdb0 0x7fd3280cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd327fff640 1 -- 172.21.15.29:0/3517693545 >> v1:172.21.15.157:6789/0 conn(0x7fd32815c190 legacy=0x7fd3280d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:51.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.670+0000 7fd327fff640 1 -- 172.21.15.29:0/3517693545 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3280cdc80 con 0x7fd3280d09e0 2024-04-25T00:38:51.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd327fff640 1 --2- 172.21.15.29:0/3517693545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3280d09e0 0x7fd3280ccd60 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fd3280b4870 tx=0x7fd31801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b9ef7d454cc69d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:51.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd3267fc640 1 -- 172.21.15.29:0/3517693545 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3180b1070 con 0x7fd3280d09e0 2024-04-25T00:38:51.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd3267fc640 1 -- 172.21.15.29:0/3517693545 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3180a8050 con 0x7fd3280d09e0 2024-04-25T00:38:51.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd3267fc640 1 -- 172.21.15.29:0/3517693545 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3180ac860 con 0x7fd3280d09e0 2024-04-25T00:38:51.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd32df3b640 1 -- 172.21.15.29:0/3517693545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3280d09e0 msgr2=0x7fd3280ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:51.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd32df3b640 1 --2- 172.21.15.29:0/3517693545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3280d09e0 0x7fd3280ccd60 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fd3280b4870 tx=0x7fd31801c420 comp rx=0 tx=0).stop 2024-04-25T00:38:51.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd32df3b640 1 -- 172.21.15.29:0/3517693545 shutdown_connections 2024-04-25T00:38:51.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd32df3b640 1 --2- 172.21.15.29:0/3517693545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3280d09e0 0x7fd3280ccd60 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd32df3b640 1 --2- 172.21.15.29:0/3517693545 >> v2:172.21.15.157:3300/0 conn(0x7fd32815bdb0 0x7fd3280cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd32df3b640 1 -- 172.21.15.29:0/3517693545 >> 172.21.15.29:0/3517693545 conn(0x7fd32809aa30 msgr2=0x7fd32809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:51.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd32df3b640 1 -- 172.21.15.29:0/3517693545 shutdown_connections 2024-04-25T00:38:51.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd32df3b640 1 -- 172.21.15.29:0/3517693545 wait complete. 2024-04-25T00:38:51.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd32df3b640 1 Processor -- start 2024-04-25T00:38:51.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.674+0000 7fd32df3b640 1 -- start start 2024-04-25T00:38:51.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd32df3b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3280d09e0 0x7fd32814f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:51.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd32df3b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd32815bdb0 0x7fd3280d1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:51.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd32df3b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3281599e0 0x7fd3280d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:51.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd32df3b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd3280c6400 con 0x7fd3280d09e0 2024-04-25T00:38:51.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd32df3b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd3280c6280 con 0x7fd3281599e0 2024-04-25T00:38:51.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd32df3b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd3280c6580 con 0x7fd32815bdb0 2024-04-25T00:38:51.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd3277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3280d09e0 0x7fd32814f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:51.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd3277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3280d09e0 0x7fd32814f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50556/0 (socket says 172.21.15.29:50556) 2024-04-25T00:38:51.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd3277fe640 1 -- 172.21.15.29:0/351921815 learned_addr learned my addr 172.21.15.29:0/351921815 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd327fff640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3281599e0 0x7fd3280d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd3277fe640 1 -- 172.21.15.29:0/351921815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd32815bdb0 msgr2=0x7fd3280d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd3277fe640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd32815bdb0 0x7fd3280d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd3277fe640 1 -- 172.21.15.29:0/351921815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3281599e0 msgr2=0x7fd3280d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd3277fe640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3281599e0 0x7fd3280d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd3277fe640 1 -- 172.21.15.29:0/351921815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd328153a00 con 0x7fd3280d09e0 2024-04-25T00:38:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd327fff640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3281599e0 0x7fd3280d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd3277fe640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3280d09e0 0x7fd32814f620 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fd32000d830 tx=0x7fd32000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd30ffff640 1 -- 172.21.15.29:0/351921815 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32000b6c0 con 0x7fd3280d09e0 2024-04-25T00:38:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd30ffff640 1 -- 172.21.15.29:0/351921815 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd320019070 con 0x7fd3280d09e0 2024-04-25T00:38:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd32df3b640 1 -- 172.21.15.29:0/351921815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd328153ce0 con 0x7fd3280d09e0 2024-04-25T00:38:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd30ffff640 1 -- 172.21.15.29:0/351921815 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32000ba60 con 0x7fd3280d09e0 2024-04-25T00:38:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd32df3b640 1 -- 172.21.15.29:0/351921815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3281542a0 con 0x7fd3280d09e0 2024-04-25T00:38:51.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd32df3b640 1 -- 172.21.15.29:0/351921815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd328069270 con 0x7fd3280d09e0 2024-04-25T00:38:51.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd30ffff640 1 -- 172.21.15.29:0/351921815 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fd32001e460 con 0x7fd3280d09e0 2024-04-25T00:38:51.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd30ffff640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd314067e60 0x7fd31406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:51.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd30ffff640 1 -- 172.21.15.29:0/351921815 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fd320086d50 con 0x7fd3280d09e0 2024-04-25T00:38:51.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd326ffd640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd314067e60 0x7fd31406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:51.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.678+0000 7fd326ffd640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd314067e60 0x7fd31406a320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd328154d60 tx=0x7fd31c009230 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:51.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.682+0000 7fd30ffff640 1 -- 172.21.15.29:0/351921815 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd32008d080 con 0x7fd3280d09e0 2024-04-25T00:38:51.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.862+0000 7fd32df3b640 1 -- 172.21.15.29:0/351921815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3280b2fb0 con 0x7fd3280d09e0 2024-04-25T00:38:51.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.862+0000 7fd30ffff640 1 -- 172.21.15.29:0/351921815 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7fd320057070 con 0x7fd3280d09e0 2024-04-25T00:38:51.865 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:38:51.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.862+0000 7fd32df3b640 1 -- 172.21.15.29:0/351921815 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd314067e60 msgr2=0x7fd31406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:51.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.862+0000 7fd32df3b640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd314067e60 0x7fd31406a320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd328154d60 tx=0x7fd31c009230 comp rx=0 tx=0).stop 2024-04-25T00:38:51.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.866+0000 7fd32df3b640 1 -- 172.21.15.29:0/351921815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3280d09e0 msgr2=0x7fd32814f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:51.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.866+0000 7fd32df3b640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3280d09e0 0x7fd32814f620 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fd32000d830 tx=0x7fd32000dcf0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.866+0000 7fd32df3b640 1 -- 172.21.15.29:0/351921815 shutdown_connections 2024-04-25T00:38:51.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.866+0000 7fd32df3b640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd314067e60 0x7fd31406a320 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.866+0000 7fd32df3b640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3281599e0 0x7fd3280d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.866+0000 7fd32df3b640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd32815bdb0 0x7fd3280d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.866+0000 7fd32df3b640 1 --2- 172.21.15.29:0/351921815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3280d09e0 0x7fd32814f620 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:51.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.866+0000 7fd32df3b640 1 -- 172.21.15.29:0/351921815 >> 172.21.15.29:0/351921815 conn(0x7fd32809aa30 msgr2=0x7fd32806efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:51.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.866+0000 7fd32df3b640 1 -- 172.21.15.29:0/351921815 shutdown_connections 2024-04-25T00:38:51.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:51.866+0000 7fd32df3b640 1 -- 172.21.15.29:0/351921815 wait complete. 2024-04-25T00:38:51.885 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:38:52.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe08d74e640 1 Processor -- start 2024-04-25T00:38:52.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe08d74e640 1 -- start start 2024-04-25T00:38:52.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe08d74e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe08815d980 0x7fe08806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe08d74e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe08806db10 0x7fe088151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe08d74e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0880b3130 con 0x7fe0880b2b70 2024-04-25T00:38:52.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe08d74e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe0880cf560 con 0x7fe08815d980 2024-04-25T00:38:52.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe08d74e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe08806df60 con 0x7fe08806db10 2024-04-25T00:38:52.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe086ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe08815d980 0x7fe08806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe0867fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe0880b2b70 0x7fe088150df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36804/0 (socket says 172.21.15.29:36804) 2024-04-25T00:38:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe0867fc640 1 -- 172.21.15.29:0/2812415443 learned_addr learned my addr 172.21.15.29:0/2812415443 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe086ffd640 1 --2- 172.21.15.29:0/2812415443 >> v2:172.21.15.29:3300/0 conn(0x7fe08815d980 0x7fe08806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50570/0 (socket says 172.21.15.29:50570) 2024-04-25T00:38:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe0877fe640 1 --2- 172.21.15.29:0/2812415443 >> v2:172.21.15.157:3300/0 conn(0x7fe08806db10 0x7fe088151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe086ffd640 1 -- 172.21.15.29:0/2812415443 >> v2:172.21.15.157:3300/0 conn(0x7fe08806db10 msgr2=0x7fe088151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe086ffd640 1 --2- 172.21.15.29:0/2812415443 >> v2:172.21.15.157:3300/0 conn(0x7fe08806db10 0x7fe088151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe085ffb640 1 -- 172.21.15.29:0/2812415443 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3970026686 0 0) 0x7fe0880b3130 con 0x7fe0880b2b70 2024-04-25T00:38:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe086ffd640 1 -- 172.21.15.29:0/2812415443 >> v1:172.21.15.29:6789/0 conn(0x7fe0880b2b70 legacy=0x7fe088150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe086ffd640 1 -- 172.21.15.29:0/2812415443 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0880b3410 con 0x7fe08815d980 2024-04-25T00:38:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.014+0000 7fe0877fe640 1 --2- 172.21.15.29:0/2812415443 >> v2:172.21.15.157:3300/0 conn(0x7fe08806db10 0x7fe088151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:52.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe086ffd640 1 --2- 172.21.15.29:0/2812415443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe08815d980 0x7fe08806d5d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe088158570 tx=0x7fe07801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb541dd031bee9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:52.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe085ffb640 1 -- 172.21.15.29:0/2812415443 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780b0070 con 0x7fe08815d980 2024-04-25T00:38:52.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe085ffb640 1 -- 172.21.15.29:0/2812415443 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0780ab600 con 0x7fe08815d980 2024-04-25T00:38:52.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe085ffb640 1 -- 172.21.15.29:0/2812415443 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780ab9d0 con 0x7fe08815d980 2024-04-25T00:38:52.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 -- 172.21.15.29:0/2812415443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe08815d980 msgr2=0x7fe08806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 --2- 172.21.15.29:0/2812415443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe08815d980 0x7fe08806d5d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe088158570 tx=0x7fe07801c480 comp rx=0 tx=0).stop 2024-04-25T00:38:52.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 -- 172.21.15.29:0/2812415443 shutdown_connections 2024-04-25T00:38:52.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 --2- 172.21.15.29:0/2812415443 >> v2:172.21.15.157:3300/0 conn(0x7fe08806db10 0x7fe088151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 --2- 172.21.15.29:0/2812415443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe08815d980 0x7fe08806d5d0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 -- 172.21.15.29:0/2812415443 >> 172.21.15.29:0/2812415443 conn(0x7fe0880b1870 msgr2=0x7fe0880b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:52.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 -- 172.21.15.29:0/2812415443 shutdown_connections 2024-04-25T00:38:52.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 -- 172.21.15.29:0/2812415443 wait complete. 2024-04-25T00:38:52.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 Processor -- start 2024-04-25T00:38:52.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 -- start start 2024-04-25T00:38:52.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08806db10 0x7fe08814a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0880b2b70 0x7fe0880afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe08815d980 0x7fe088153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881663c0 con 0x7fe08815d980 2024-04-25T00:38:52.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe08815dd60 con 0x7fe0880b2b70 2024-04-25T00:38:52.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe08d74e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe088166540 con 0x7fe08806db10 2024-04-25T00:38:52.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe0877fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe08815d980 0x7fe088153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe0877fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe08815d980 0x7fe088153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50582/0 (socket says 172.21.15.29:50582) 2024-04-25T00:38:52.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe0877fe640 1 -- 172.21.15.29:0/1476365257 learned_addr learned my addr 172.21.15.29:0/1476365257 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:52.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe0877fe640 1 -- 172.21.15.29:0/1476365257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08806db10 msgr2=0x7fe08814a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe0877fe640 1 --2- 172.21.15.29:0/1476365257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08806db10 0x7fe08814a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe0877fe640 1 -- 172.21.15.29:0/1476365257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0880b2b70 msgr2=0x7fe0880afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe0877fe640 1 --2- 172.21.15.29:0/1476365257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0880b2b70 0x7fe0880afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe0877fe640 1 -- 172.21.15.29:0/1476365257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe088147a10 con 0x7fe08815d980 2024-04-25T00:38:52.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe0877fe640 1 --2- 172.21.15.29:0/1476365257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe08815d980 0x7fe088153520 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe08000cf20 tx=0x7fe08000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:52.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe06f7fe640 1 -- 172.21.15.29:0/1476365257 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08001a070 con 0x7fe08815d980 2024-04-25T00:38:52.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.018+0000 7fe06f7fe640 1 -- 172.21.15.29:0/1476365257 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe08000d840 con 0x7fe08815d980 2024-04-25T00:38:52.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.022+0000 7fe08d74e640 1 -- 172.21.15.29:0/1476365257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08814a7a0 con 0x7fe08815d980 2024-04-25T00:38:52.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.022+0000 7fe06f7fe640 1 -- 172.21.15.29:0/1476365257 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08000dc30 con 0x7fe08815d980 2024-04-25T00:38:52.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.022+0000 7fe08d74e640 1 -- 172.21.15.29:0/1476365257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe088152d20 con 0x7fe08815d980 2024-04-25T00:38:52.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.022+0000 7fe08d74e640 1 -- 172.21.15.29:0/1476365257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe088069270 con 0x7fe08815d980 2024-04-25T00:38:52.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.022+0000 7fe06f7fe640 1 -- 172.21.15.29:0/1476365257 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fe08001d070 con 0x7fe08815d980 2024-04-25T00:38:52.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.022+0000 7fe06f7fe640 1 --2- 172.21.15.29:0/1476365257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe074067e90 0x7fe07406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.022+0000 7fe06f7fe640 1 -- 172.21.15.29:0/1476365257 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fe080086f60 con 0x7fe08815d980 2024-04-25T00:38:52.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.022+0000 7fe086ffd640 1 --2- 172.21.15.29:0/1476365257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe074067e90 0x7fe07406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.022+0000 7fe086ffd640 1 --2- 172.21.15.29:0/1476365257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe074067e90 0x7fe07406a350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe0780ad000 tx=0x7fe078004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:52.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.026+0000 7fe06f7fe640 1 -- 172.21.15.29:0/1476365257 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe080057280 con 0x7fe08815d980 2024-04-25T00:38:52.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.190+0000 7fe08d74e640 1 -- 172.21.15.29:0/1476365257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe0880cf910 con 0x7fe08815d980 2024-04-25T00:38:52.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.190+0000 7fe06f7fe640 1 -- 172.21.15.29:0/1476365257 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) ==== 104+0+11 (secure 0 0 0) 0x7fe080056b80 con 0x7fe08815d980 2024-04-25T00:38:52.194 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:38:52.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.194+0000 7fe08d74e640 1 -- 172.21.15.29:0/1476365257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe074067e90 msgr2=0x7fe07406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.194+0000 7fe08d74e640 1 --2- 172.21.15.29:0/1476365257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe074067e90 0x7fe07406a350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe0780ad000 tx=0x7fe078004740 comp rx=0 tx=0).stop 2024-04-25T00:38:52.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.194+0000 7fe08d74e640 1 -- 172.21.15.29:0/1476365257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe08815d980 msgr2=0x7fe088153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.194+0000 7fe08d74e640 1 --2- 172.21.15.29:0/1476365257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe08815d980 0x7fe088153520 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe08000cf20 tx=0x7fe08000a7d0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.194+0000 7fe08d74e640 1 -- 172.21.15.29:0/1476365257 shutdown_connections 2024-04-25T00:38:52.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.194+0000 7fe08d74e640 1 --2- 172.21.15.29:0/1476365257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe074067e90 0x7fe07406a350 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.194+0000 7fe08d74e640 1 --2- 172.21.15.29:0/1476365257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe08815d980 0x7fe088153520 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.194+0000 7fe08d74e640 1 --2- 172.21.15.29:0/1476365257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0880b2b70 0x7fe0880afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.194+0000 7fe08d74e640 1 --2- 172.21.15.29:0/1476365257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe08806db10 0x7fe08814a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.194+0000 7fe08d74e640 1 -- 172.21.15.29:0/1476365257 >> 172.21.15.29:0/1476365257 conn(0x7fe0880b1870 msgr2=0x7fe08815c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:52.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.194+0000 7fe08d74e640 1 -- 172.21.15.29:0/1476365257 shutdown_connections 2024-04-25T00:38:52.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.194+0000 7fe08d74e640 1 -- 172.21.15.29:0/1476365257 wait complete. 2024-04-25T00:38:52.213 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:38:52.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d9ab36640 1 Processor -- start 2024-04-25T00:38:52.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d9ab36640 1 -- start start 2024-04-25T00:38:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d9ab36640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d940b3140 0x7f0d9406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d9ab36640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d940b4b10 0x7f0d940b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d9ab36640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0d940b3a60 0x7f0d9415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d9ab36640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f0d940b41e0 con 0x7f0d940b3a60 2024-04-25T00:38:52.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d9ab36640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0d9406d960 con 0x7f0d940b3140 2024-04-25T00:38:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d9ab36640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9406dc40 con 0x7f0d940b4b10 2024-04-25T00:38:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d990d3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0d940b3a60 0x7f0d9415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d990d3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0d940b3a60 0x7f0d9415c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50592/0 (socket says 172.21.15.29:50592) 2024-04-25T00:38:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d990d3640 1 -- 172.21.15.29:0/553351895 learned_addr learned my addr 172.21.15.29:0/553351895 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d988d2640 1 --2- 172.21.15.29:0/553351895 >> v2:172.21.15.157:3300/0 conn(0x7f0d940b3140 0x7f0d9406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d93fff640 1 --2- 172.21.15.29:0/553351895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d940b4b10 0x7f0d940b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d990d3640 1 -- 172.21.15.29:0/553351895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d940b4b10 msgr2=0x7f0d940b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d990d3640 1 --2- 172.21.15.29:0/553351895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d940b4b10 0x7f0d940b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d990d3640 1 -- 172.21.15.29:0/553351895 >> v2:172.21.15.157:3300/0 conn(0x7f0d940b3140 msgr2=0x7f0d9406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d990d3640 1 --2- 172.21.15.29:0/553351895 >> v2:172.21.15.157:3300/0 conn(0x7f0d940b3140 0x7f0d9406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.334+0000 7f0d990d3640 1 -- 172.21.15.29:0/553351895 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9406df20 con 0x7f0d940b3a60 2024-04-25T00:38:52.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d990d3640 1 --2- 172.21.15.29:0/553351895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d940b3a60 0x7f0d9415c3e0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f0d940d0bf0 tx=0x7f0d8c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff3a16fe80db8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:52.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d937fe640 1 -- 172.21.15.29:0/553351895 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c0b0070 con 0x7f0d940b3a60 2024-04-25T00:38:52.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d937fe640 1 -- 172.21.15.29:0/553351895 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8c0ab600 con 0x7f0d940b3a60 2024-04-25T00:38:52.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d937fe640 1 -- 172.21.15.29:0/553351895 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c0ab9d0 con 0x7f0d940b3a60 2024-04-25T00:38:52.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d9ab36640 1 -- 172.21.15.29:0/553351895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d940b3a60 msgr2=0x7f0d9415c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d9ab36640 1 --2- 172.21.15.29:0/553351895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d940b3a60 0x7f0d9415c3e0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f0d940d0bf0 tx=0x7f0d8c01c520 comp rx=0 tx=0).stop 2024-04-25T00:38:52.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d9ab36640 1 -- 172.21.15.29:0/553351895 shutdown_connections 2024-04-25T00:38:52.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d9ab36640 1 --2- 172.21.15.29:0/553351895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d940b3a60 0x7f0d9415c3e0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d9ab36640 1 --2- 172.21.15.29:0/553351895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d940b4b10 0x7f0d940b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d9ab36640 1 --2- 172.21.15.29:0/553351895 >> v2:172.21.15.157:3300/0 conn(0x7f0d940b3140 0x7f0d9406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d9ab36640 1 -- 172.21.15.29:0/553351895 >> 172.21.15.29:0/553351895 conn(0x7f0d940b1870 msgr2=0x7f0d940b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:52.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d9ab36640 1 -- 172.21.15.29:0/553351895 shutdown_connections 2024-04-25T00:38:52.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d9ab36640 1 -- 172.21.15.29:0/553351895 wait complete. 2024-04-25T00:38:52.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d9ab36640 1 Processor -- start 2024-04-25T00:38:52.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.338+0000 7f0d9ab36640 1 -- start start 2024-04-25T00:38:52.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d9ab36640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d940b3140 0x7f0d941580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d9ab36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d940b3a60 0x7f0d94158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d9ab36640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d940b4b10 0x7f0d94132220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d9ab36640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0d940ce3c0 con 0x7f0d940b3a60 2024-04-25T00:38:52.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d9ab36640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d940ce240 con 0x7f0d940b3140 2024-04-25T00:38:52.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d9ab36640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0d940ce540 con 0x7f0d940b4b10 2024-04-25T00:38:52.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d93fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d940b3a60 0x7f0d94158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d93fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d940b3a60 0x7f0d94158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50596/0 (socket says 172.21.15.29:50596) 2024-04-25T00:38:52.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d93fff640 1 -- 172.21.15.29:0/2476267419 learned_addr learned my addr 172.21.15.29:0/2476267419 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:52.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d988d2640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d940b3140 0x7f0d941580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d93fff640 1 -- 172.21.15.29:0/2476267419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d940b4b10 msgr2=0x7f0d94132220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d990d3640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d940b4b10 0x7f0d94132220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d93fff640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d940b4b10 0x7f0d94132220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d93fff640 1 -- 172.21.15.29:0/2476267419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d940b3140 msgr2=0x7f0d941580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d93fff640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d940b3140 0x7f0d941580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d93fff640 1 -- 172.21.15.29:0/2476267419 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9413fd80 con 0x7f0d940b3a60 2024-04-25T00:38:52.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d988d2640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d940b3140 0x7f0d941580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:52.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d93fff640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d940b3a60 0x7f0d94158630 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00d5f0 tx=0x7f0d7c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:52.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d917fa640 1 -- 172.21.15.29:0/2476267419 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0125f0 con 0x7f0d940b3a60 2024-04-25T00:38:52.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d917fa640 1 -- 172.21.15.29:0/2476267419 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d7c019070 con 0x7f0d940b3a60 2024-04-25T00:38:52.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d9ab36640 1 -- 172.21.15.29:0/2476267419 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d94158e70 con 0x7f0d940b3a60 2024-04-25T00:38:52.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d917fa640 1 -- 172.21.15.29:0/2476267419 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0129b0 con 0x7f0d940b3a60 2024-04-25T00:38:52.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d9ab36640 1 -- 172.21.15.29:0/2476267419 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d94140300 con 0x7f0d940b3a60 2024-04-25T00:38:52.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d9ab36640 1 -- 172.21.15.29:0/2476267419 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d94069270 con 0x7f0d940b3a60 2024-04-25T00:38:52.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.342+0000 7f0d917fa640 1 -- 172.21.15.29:0/2476267419 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f0d7c012b50 con 0x7f0d940b3a60 2024-04-25T00:38:52.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.346+0000 7f0d917fa640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0d74067e90 0x7f0d7406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.346+0000 7f0d917fa640 1 -- 172.21.15.29:0/2476267419 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f0d7c086b40 con 0x7f0d940b3a60 2024-04-25T00:38:52.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.346+0000 7f0d988d2640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0d74067e90 0x7f0d7406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.346+0000 7f0d988d2640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0d74067e90 0x7f0d7406a350 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009f70 tx=0x7f0d8400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:52.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.350+0000 7f0d917fa640 1 -- 172.21.15.29:0/2476267419 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0d7c056dd0 con 0x7f0d940b3a60 2024-04-25T00:38:52.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.510+0000 7f0d9ab36640 1 -- 172.21.15.29:0/2476267419 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0d94144580 con 0x7f0d940b3a60 2024-04-25T00:38:52.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.510+0000 7f0d917fa640 1 -- 172.21.15.29:0/2476267419 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) ==== 100+0+11 (secure 0 0 0) 0x7f0d7c0566d0 con 0x7f0d940b3a60 2024-04-25T00:38:52.515 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:38:52.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.514+0000 7f0d9ab36640 1 -- 172.21.15.29:0/2476267419 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0d74067e90 msgr2=0x7f0d7406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.514+0000 7f0d9ab36640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0d74067e90 0x7f0d7406a350 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009f70 tx=0x7f0d8400b040 comp rx=0 tx=0).stop 2024-04-25T00:38:52.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.514+0000 7f0d9ab36640 1 -- 172.21.15.29:0/2476267419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d940b3a60 msgr2=0x7f0d94158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.514+0000 7f0d9ab36640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d940b3a60 0x7f0d94158630 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00d5f0 tx=0x7f0d7c00dab0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.514+0000 7f0d9ab36640 1 -- 172.21.15.29:0/2476267419 shutdown_connections 2024-04-25T00:38:52.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.514+0000 7f0d9ab36640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0d74067e90 0x7f0d7406a350 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.514+0000 7f0d9ab36640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d940b4b10 0x7f0d94132220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.514+0000 7f0d9ab36640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d940b3a60 0x7f0d94158630 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.514+0000 7f0d9ab36640 1 --2- 172.21.15.29:0/2476267419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d940b3140 0x7f0d941580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.514+0000 7f0d9ab36640 1 -- 172.21.15.29:0/2476267419 >> 172.21.15.29:0/2476267419 conn(0x7f0d940b1870 msgr2=0x7f0d94144aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:52.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.514+0000 7f0d9ab36640 1 -- 172.21.15.29:0/2476267419 shutdown_connections 2024-04-25T00:38:52.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.514+0000 7f0d9ab36640 1 -- 172.21.15.29:0/2476267419 wait complete. 2024-04-25T00:38:52.535 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:38:52.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.650+0000 7f8b9a0d3640 1 Processor -- start 2024-04-25T00:38:52.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.650+0000 7f8b9a0d3640 1 -- start start 2024-04-25T00:38:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b9a0d3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b9415e8d0 0x7f8b940cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b9a0d3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b940b4780 0x7f8b9415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b9a0d3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b9415f1f0 0x7f8b9415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b9a0d3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8b940b42f0 con 0x7f8b9415f1f0 2024-04-25T00:38:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b9a0d3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8b940b3720 con 0x7f8b9415e8d0 2024-04-25T00:38:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b9a0d3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b940b39c0 con 0x7f8b940b4780 2024-04-25T00:38:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b93fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b9415f1f0 0x7f8b9415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b93fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b9415f1f0 0x7f8b9415b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50600/0 (socket says 172.21.15.29:50600) 2024-04-25T00:38:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b93fff640 1 -- 172.21.15.29:0/3699185883 learned_addr learned my addr 172.21.15.29:0/3699185883 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b937fe640 1 --2- 172.21.15.29:0/3699185883 >> v2:172.21.15.157:3300/0 conn(0x7f8b9415e8d0 0x7f8b940cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b92ffd640 1 --2- 172.21.15.29:0/3699185883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b940b4780 0x7f8b9415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b93fff640 1 -- 172.21.15.29:0/3699185883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b940b4780 msgr2=0x7f8b9415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:52.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b93fff640 1 --2- 172.21.15.29:0/3699185883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b940b4780 0x7f8b9415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b93fff640 1 -- 172.21.15.29:0/3699185883 >> v2:172.21.15.157:3300/0 conn(0x7f8b9415e8d0 msgr2=0x7f8b940cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:52.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b93fff640 1 --2- 172.21.15.29:0/3699185883 >> v2:172.21.15.157:3300/0 conn(0x7f8b9415e8d0 0x7f8b940cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b93fff640 1 -- 172.21.15.29:0/3699185883 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b940b3ca0 con 0x7f8b9415f1f0 2024-04-25T00:38:52.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b93fff640 1 --2- 172.21.15.29:0/3699185883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b9415f1f0 0x7f8b9415b2a0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f8b940b5260 tx=0x7f8b8c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f44c085b86ac21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:52.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b927fc640 1 -- 172.21.15.29:0/3699185883 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0a5820 con 0x7f8b9415f1f0 2024-04-25T00:38:52.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b927fc640 1 -- 172.21.15.29:0/3699185883 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b8c0a59c0 con 0x7f8b9415f1f0 2024-04-25T00:38:52.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.654+0000 7f8b927fc640 1 -- 172.21.15.29:0/3699185883 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0a5d90 con 0x7f8b9415f1f0 2024-04-25T00:38:52.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3699185883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b9415f1f0 msgr2=0x7f8b9415b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 --2- 172.21.15.29:0/3699185883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b9415f1f0 0x7f8b9415b2a0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f8b940b5260 tx=0x7f8b8c01c9c0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3699185883 shutdown_connections 2024-04-25T00:38:52.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 --2- 172.21.15.29:0/3699185883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b9415f1f0 0x7f8b9415b2a0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 --2- 172.21.15.29:0/3699185883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b940b4780 0x7f8b9415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 --2- 172.21.15.29:0/3699185883 >> v2:172.21.15.157:3300/0 conn(0x7f8b9415e8d0 0x7f8b940cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3699185883 >> 172.21.15.29:0/3699185883 conn(0x7f8b940b1870 msgr2=0x7f8b940b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:52.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3699185883 shutdown_connections 2024-04-25T00:38:52.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3699185883 wait complete. 2024-04-25T00:38:52.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 Processor -- start 2024-04-25T00:38:52.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 -- start start 2024-04-25T00:38:52.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b940b4780 0x7f8b9415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b9415e8d0 0x7f8b9415dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b9415f1f0 0x7f8b940b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8b94144a20 con 0x7f8b9415f1f0 2024-04-25T00:38:52.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9415c7d0 con 0x7f8b9415e8d0 2024-04-25T00:38:52.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b940713d0 con 0x7f8b940b4780 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b9415f1f0 0x7f8b940b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b9415f1f0 0x7f8b940b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50608/0 (socket says 172.21.15.29:50608) 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b93fff640 1 -- 172.21.15.29:0/3638473452 learned_addr learned my addr 172.21.15.29:0/3638473452 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b92ffd640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b9415e8d0 0x7f8b9415dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b937fe640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b940b4780 0x7f8b9415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b93fff640 1 -- 172.21.15.29:0/3638473452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b940b4780 msgr2=0x7f8b9415a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b93fff640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b940b4780 0x7f8b9415a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b93fff640 1 -- 172.21.15.29:0/3638473452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b9415e8d0 msgr2=0x7f8b9415dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b93fff640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b9415e8d0 0x7f8b9415dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b93fff640 1 -- 172.21.15.29:0/3638473452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9415e490 con 0x7f8b9415f1f0 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b937fe640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b940b4780 0x7f8b9415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b93fff640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b9415f1f0 0x7f8b940b2960 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0af040 tx=0x7f8b8c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b7bfff640 1 -- 172.21.15.29:0/3638473452 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0b8670 con 0x7f8b9415f1f0 2024-04-25T00:38:52.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b7bfff640 1 -- 172.21.15.29:0/3638473452 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b8c0be070 con 0x7f8b9415f1f0 2024-04-25T00:38:52.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b7bfff640 1 -- 172.21.15.29:0/3638473452 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0b8a70 con 0x7f8b9415f1f0 2024-04-25T00:38:52.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3638473452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b94071620 con 0x7f8b9415f1f0 2024-04-25T00:38:52.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.658+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3638473452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b94071be0 con 0x7f8b9415f1f0 2024-04-25T00:38:52.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.662+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3638473452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b94069270 con 0x7f8b9415f1f0 2024-04-25T00:38:52.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.662+0000 7f8b7bfff640 1 -- 172.21.15.29:0/3638473452 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f8b8c0b8c10 con 0x7f8b9415f1f0 2024-04-25T00:38:52.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.662+0000 7f8b7bfff640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b80067e60 0x7f8b8006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:52.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.662+0000 7f8b7bfff640 1 -- 172.21.15.29:0/3638473452 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f8b8c121930 con 0x7f8b9415f1f0 2024-04-25T00:38:52.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.662+0000 7f8b937fe640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b80067e60 0x7f8b8006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:52.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.662+0000 7f8b937fe640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b80067e60 0x7f8b8006a320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8b94072ec0 tx=0x7f8b84008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:52.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.666+0000 7f8b7bfff640 1 -- 172.21.15.29:0/3638473452 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8b8c0165e0 con 0x7f8b9415f1f0 2024-04-25T00:38:52.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.842+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3638473452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8b9406e7b0 con 0x7f8b9415f1f0 2024-04-25T00:38:52.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.842+0000 7f8b7bfff640 1 -- 172.21.15.29:0/3638473452 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) ==== 80+0+0 (secure 0 0 0) 0x7f8b8c0f1bc0 con 0x7f8b9415f1f0 2024-04-25T00:38:52.846 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:38:52.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.846+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3638473452 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b80067e60 msgr2=0x7f8b8006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.846+0000 7f8b9a0d3640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b80067e60 0x7f8b8006a320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8b94072ec0 tx=0x7f8b84008040 comp rx=0 tx=0).stop 2024-04-25T00:38:52.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.846+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3638473452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b9415f1f0 msgr2=0x7f8b940b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:52.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.846+0000 7f8b9a0d3640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b9415f1f0 0x7f8b940b2960 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0af040 tx=0x7f8b8c019760 comp rx=0 tx=0).stop 2024-04-25T00:38:52.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.846+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3638473452 shutdown_connections 2024-04-25T00:38:52.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.846+0000 7f8b9a0d3640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b80067e60 0x7f8b8006a320 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.846+0000 7f8b9a0d3640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b9415f1f0 0x7f8b940b2960 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.846+0000 7f8b9a0d3640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b9415e8d0 0x7f8b9415dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.846+0000 7f8b9a0d3640 1 --2- 172.21.15.29:0/3638473452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b940b4780 0x7f8b9415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:52.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.846+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3638473452 >> 172.21.15.29:0/3638473452 conn(0x7f8b940b1870 msgr2=0x7f8b94143b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:52.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.846+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3638473452 shutdown_connections 2024-04-25T00:38:52.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:52.846+0000 7f8b9a0d3640 1 -- 172.21.15.29:0/3638473452 wait complete. 2024-04-25T00:38:52.866 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-25T00:38:52.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:52.866 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T00:38:52.880 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34753]: fuse finished with error 0 and tester_r 0 2024-04-25T00:38:52.881 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:38:52.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:52.882 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:38:52.963 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:38:52.963 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:38:52.964 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:38:52.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:38:52.970 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:38:52.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:52.970 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:38:53.016 INFO:teuthology.orchestra.run.smithi029.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T00:38:53.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:38:53.017 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:38:53.017 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:38:53.017 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:38:53.023 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:38:53.023 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:38:53.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:38:53.023 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:38:53.071 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T00:38:53.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:38:53.072 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:38:53.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.202+0000 7f6e97326640 1 Processor -- start 2024-04-25T00:38:53.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.202+0000 7f6e97326640 1 -- start start 2024-04-25T00:38:53.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e97326640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6e9015e8d0 0x7f6e900cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e97326640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e900b4780 0x7f6e9015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e97326640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e900b3720 con 0x7f6e9015f1f0 2024-04-25T00:38:53.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e97326640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6e900b42f0 con 0x7f6e9015e8d0 2024-04-25T00:38:53.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e97326640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9015d7e0 con 0x7f6e900b4780 2024-04-25T00:38:53.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e950c2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6e9015e8d0 0x7f6e900cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e950c2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6e9015e8d0 0x7f6e900cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50618/0 (socket says 172.21.15.29:50618) 2024-04-25T00:38:53.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e950c2640 1 -- 172.21.15.29:0/2800369940 learned_addr learned my addr 172.21.15.29:0/2800369940 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:53.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e948c1640 1 --2- 172.21.15.29:0/2800369940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e900b4780 0x7f6e9015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e950c2640 1 -- 172.21.15.29:0/2800369940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e900b4780 msgr2=0x7f6e9015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:53.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e950c2640 1 --2- 172.21.15.29:0/2800369940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e900b4780 0x7f6e9015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e950c2640 1 -- 172.21.15.29:0/2800369940 >> v1:172.21.15.157:6789/0 conn(0x7f6e9015f1f0 legacy=0x7f6e9015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:53.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e950c2640 1 -- 172.21.15.29:0/2800369940 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e900b39c0 con 0x7f6e9015e8d0 2024-04-25T00:38:53.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e950c2640 1 --2- 172.21.15.29:0/2800369940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e9015e8d0 0x7f6e900cc240 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00d150 tx=0x7f6e8c01ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7dfe5eb59789ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:53.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e87fff640 1 -- 172.21.15.29:0/2800369940 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9015c1c0 con 0x7f6e9015e8d0 2024-04-25T00:38:53.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e87fff640 1 -- 172.21.15.29:0/2800369940 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e8c00b2d0 con 0x7f6e9015e8d0 2024-04-25T00:38:53.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.206+0000 7f6e87fff640 1 -- 172.21.15.29:0/2800369940 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c00b480 con 0x7f6e9015e8d0 2024-04-25T00:38:53.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 -- 172.21.15.29:0/2800369940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e9015e8d0 msgr2=0x7f6e900cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 --2- 172.21.15.29:0/2800369940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e9015e8d0 0x7f6e900cc240 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00d150 tx=0x7f6e8c01ebc0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 -- 172.21.15.29:0/2800369940 shutdown_connections 2024-04-25T00:38:53.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 --2- 172.21.15.29:0/2800369940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e900b4780 0x7f6e9015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 --2- 172.21.15.29:0/2800369940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e9015e8d0 0x7f6e900cc240 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 -- 172.21.15.29:0/2800369940 >> 172.21.15.29:0/2800369940 conn(0x7f6e900b1870 msgr2=0x7f6e900b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:53.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 -- 172.21.15.29:0/2800369940 shutdown_connections 2024-04-25T00:38:53.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 -- 172.21.15.29:0/2800369940 wait complete. 2024-04-25T00:38:53.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 Processor -- start 2024-04-25T00:38:53.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 -- start start 2024-04-25T00:38:53.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e900b4780 0x7f6e900af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9015e8d0 0x7f6e9013fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e9015f1f0 0x7f6e9013a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6e90144a20 con 0x7f6e9015f1f0 2024-04-25T00:38:53.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9015f660 con 0x7f6e9015e8d0 2024-04-25T00:38:53.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6e90140220 con 0x7f6e900b4780 2024-04-25T00:38:53.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e958c3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e9015f1f0 0x7f6e9013a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e958c3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e9015f1f0 0x7f6e9013a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50622/0 (socket says 172.21.15.29:50622) 2024-04-25T00:38:53.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e958c3640 1 -- 172.21.15.29:0/1003577793 learned_addr learned my addr 172.21.15.29:0/1003577793 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:53.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e948c1640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9015e8d0 0x7f6e9013fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e950c2640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e900b4780 0x7f6e900af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e958c3640 1 -- 172.21.15.29:0/1003577793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e900b4780 msgr2=0x7f6e900af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e958c3640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e900b4780 0x7f6e900af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e958c3640 1 -- 172.21.15.29:0/1003577793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9015e8d0 msgr2=0x7f6e9013fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e958c3640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9015e8d0 0x7f6e9013fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e958c3640 1 -- 172.21.15.29:0/1003577793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e900b0d60 con 0x7f6e9015f1f0 2024-04-25T00:38:53.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e948c1640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9015e8d0 0x7f6e9013fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:53.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e958c3640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e9015f1f0 0x7f6e9013a5d0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800b410 tx=0x7f6e8800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:53.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e85ffb640 1 -- 172.21.15.29:0/1003577793 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88015020 con 0x7f6e9015f1f0 2024-04-25T00:38:53.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e85ffb640 1 -- 172.21.15.29:0/1003577793 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e88011870 con 0x7f6e9015f1f0 2024-04-25T00:38:53.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 -- 172.21.15.29:0/1003577793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e900afcc0 con 0x7f6e9015f1f0 2024-04-25T00:38:53.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e85ffb640 1 -- 172.21.15.29:0/1003577793 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88011c60 con 0x7f6e9015f1f0 2024-04-25T00:38:53.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.210+0000 7f6e97326640 1 -- 172.21.15.29:0/1003577793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e900b0280 con 0x7f6e9015f1f0 2024-04-25T00:38:53.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.214+0000 7f6e97326640 1 -- 172.21.15.29:0/1003577793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e90069270 con 0x7f6e9015f1f0 2024-04-25T00:38:53.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.214+0000 7f6e85ffb640 1 -- 172.21.15.29:0/1003577793 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f6e88018020 con 0x7f6e9015f1f0 2024-04-25T00:38:53.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.214+0000 7f6e85ffb640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e74067e60 0x7f6e7406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.214+0000 7f6e85ffb640 1 -- 172.21.15.29:0/1003577793 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f6e880865f0 con 0x7f6e9015f1f0 2024-04-25T00:38:53.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.214+0000 7f6e950c2640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e74067e60 0x7f6e7406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.214+0000 7f6e950c2640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e74067e60 0x7f6e7406a320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0b0000 tx=0x7f6e8c0a9070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:53.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.218+0000 7f6e85ffb640 1 -- 172.21.15.29:0/1003577793 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6e8801f020 con 0x7f6e9015f1f0 2024-04-25T00:38:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.414+0000 7f6e97326640 1 -- 172.21.15.29:0/1003577793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e9006e7b0 con 0x7f6e9015f1f0 2024-04-25T00:38:53.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.414+0000 7f6e85ffb640 1 -- 172.21.15.29:0/1003577793 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5186 (secure 0 0 0) 0x7f6e880172c0 con 0x7f6e9015f1f0 2024-04-25T00:38:53.418 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:53.418 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2148970077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2148970077},{"type":"v1","addr":"172.21.15.157:6834","nonce":2148970077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3947933400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3947933400},{"type":"v1","addr":"172.21.15.157:6835","nonce":3947933400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/172449010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":172449010},{"type":"v1","addr":"172.21.15.29:6833","nonce":172449010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:38:31.061384+0000","modified":"2024-04-25T00:38:34.097007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/3206404146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3206404146},{"type":"v1","addr":"172.21.15.29:6835","nonce":3206404146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8520,"qdb_cluster":[8520]},"id":9}]} 2024-04-25T00:38:53.419 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 76 2024-04-25T00:38:53.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.418+0000 7f6e97326640 1 -- 172.21.15.29:0/1003577793 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e74067e60 msgr2=0x7f6e7406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.418+0000 7f6e97326640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e74067e60 0x7f6e7406a320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0b0000 tx=0x7f6e8c0a9070 comp rx=0 tx=0).stop 2024-04-25T00:38:53.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.418+0000 7f6e97326640 1 -- 172.21.15.29:0/1003577793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e9015f1f0 msgr2=0x7f6e9013a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.418+0000 7f6e97326640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e9015f1f0 0x7f6e9013a5d0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800b410 tx=0x7f6e8800b8d0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.418+0000 7f6e97326640 1 -- 172.21.15.29:0/1003577793 shutdown_connections 2024-04-25T00:38:53.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.418+0000 7f6e97326640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e74067e60 0x7f6e7406a320 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.418+0000 7f6e97326640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e9015f1f0 0x7f6e9013a5d0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.418+0000 7f6e97326640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9015e8d0 0x7f6e9013fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.418+0000 7f6e97326640 1 --2- 172.21.15.29:0/1003577793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e900b4780 0x7f6e900af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.418+0000 7f6e97326640 1 -- 172.21.15.29:0/1003577793 >> 172.21.15.29:0/1003577793 conn(0x7f6e900b1870 msgr2=0x7f6e900ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:53.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.418+0000 7f6e97326640 1 -- 172.21.15.29:0/1003577793 shutdown_connections 2024-04-25T00:38:53.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.418+0000 7f6e97326640 1 -- 172.21.15.29:0/1003577793 wait complete. 2024-04-25T00:38:53.436 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f66072aa640 1 Processor -- start 2024-04-25T00:38:53.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f66072aa640 1 -- start start 2024-04-25T00:38:53.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f66072aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6600162420 0x7f6600070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f66072aa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000b4e00 0x7f66000b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f66072aa640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6600071910 con 0x7f66000b30b0 2024-04-25T00:38:53.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f66072aa640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f66000b3500 con 0x7f6600162420 2024-04-25T00:38:53.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f66072aa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6600071bf0 con 0x7f66000b4e00 2024-04-25T00:38:53.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f6605847640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f66000b30b0 0x7f6600166020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36820/0 (socket says 172.21.15.29:36820) 2024-04-25T00:38:53.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f6605847640 1 -- 172.21.15.29:0/1582697735 learned_addr learned my addr 172.21.15.29:0/1582697735 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:53.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f6604845640 1 --2- 172.21.15.29:0/1582697735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000b4e00 0x7f66000b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f6605046640 1 --2- 172.21.15.29:0/1582697735 >> v2:172.21.15.157:3300/0 conn(0x7f6600162420 0x7f6600070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 841115887 0 0) 0x7f6600071910 con 0x7f66000b30b0 2024-04-25T00:38:53.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65ec003530 con 0x7f66000b30b0 2024-04-25T00:38:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2465463283 0 0) 0x7f65ec003530 con 0x7f66000b30b0 2024-04-25T00:38:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000b4e00 msgr2=0x7f66000b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 --2- 172.21.15.29:0/1582697735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000b4e00 0x7f66000b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 >> v2:172.21.15.157:3300/0 conn(0x7f6600162420 msgr2=0x7f6600070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 --2- 172.21.15.29:0/1582697735 >> v2:172.21.15.157:3300/0 conn(0x7f6600162420 0x7f6600070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6600071ed0 con 0x7f66000b30b0 2024-04-25T00:38:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f65f8001a00 con 0x7f66000b30b0 2024-04-25T00:38:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 >> v1:172.21.15.29:6789/0 conn(0x7f66000b30b0 legacy=0x7f6600166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 --2- 172.21.15.29:0/1582697735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65ec0039c0 0x7f65ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 --2- 172.21.15.29:0/1582697735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec004810 0x7f65ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 --2- 172.21.15.29:0/1582697735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000b4e00 0x7f65ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f66000b3500 con 0x7f65ec0039c0 2024-04-25T00:38:53.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6600071bf0 con 0x7f65ec004810 2024-04-25T00:38:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65ec00b9d0 con 0x7f66000b4e00 2024-04-25T00:38:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f6605046640 1 --2- 172.21.15.29:0/1582697735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65ec0039c0 0x7f65ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f6605847640 1 --2- 172.21.15.29:0/1582697735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000b4e00 0x7f65ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f6604845640 1 --2- 172.21.15.29:0/1582697735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec004810 0x7f65ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f6605046640 1 -- 172.21.15.29:0/1582697735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000b4e00 msgr2=0x7f65ec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f6605046640 1 --2- 172.21.15.29:0/1582697735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000b4e00 0x7f65ec0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f6605046640 1 -- 172.21.15.29:0/1582697735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec004810 msgr2=0x7f65ec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f6605046640 1 --2- 172.21.15.29:0/1582697735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec004810 0x7f65ec004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f6605046640 1 -- 172.21.15.29:0/1582697735 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec003530 con 0x7f65ec0039c0 2024-04-25T00:38:53.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.558+0000 7f6605847640 1 --2- 172.21.15.29:0/1582697735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000b4e00 0x7f65ec0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f6605046640 1 --2- 172.21.15.29:0/1582697735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65ec0039c0 0x7f65ec003de0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009ae0 tx=0x7f65fc01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a5c9030dcc70f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:53.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0096b0 con 0x7f65ec0039c0 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65fc0a6430 con 0x7f65ec0039c0 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f65f7fff640 1 -- 172.21.15.29:0/1582697735 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0a6820 con 0x7f65ec0039c0 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 -- 172.21.15.29:0/1582697735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65ec0039c0 msgr2=0x7f65ec003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 --2- 172.21.15.29:0/1582697735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65ec0039c0 0x7f65ec003de0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009ae0 tx=0x7f65fc01cab0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f6605847640 1 -- 172.21.15.29:0/1582697735 reap_dead start 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 -- 172.21.15.29:0/1582697735 shutdown_connections 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 -- 172.21.15.29:0/1582697735 >> 172.21.15.29:0/1582697735 conn(0x7f66000b1870 msgr2=0x7f66000b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 -- 172.21.15.29:0/1582697735 shutdown_connections 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 -- 172.21.15.29:0/1582697735 wait complete. 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 Processor -- start 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 -- start start 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec0039c0 0x7f6600164370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6600165770 0x7f6600163790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66001517e0 0x7f6600163cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f660016d3b0 con 0x7f6600165770 2024-04-25T00:38:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f66000716d0 con 0x7f65ec0039c0 2024-04-25T00:38:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.562+0000 7f66072aa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f660016d530 con 0x7f66001517e0 2024-04-25T00:38:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6604845640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6600165770 0x7f6600163790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6604845640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6600165770 0x7f6600163790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50626/0 (socket says 172.21.15.29:50626) 2024-04-25T00:38:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6604845640 1 -- 172.21.15.29:0/3035575097 learned_addr learned my addr 172.21.15.29:0/3035575097 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6605046640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec0039c0 0x7f6600164370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6605847640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66001517e0 0x7f6600163cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6604845640 1 -- 172.21.15.29:0/3035575097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66001517e0 msgr2=0x7f6600163cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6604845640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66001517e0 0x7f6600163cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6604845640 1 -- 172.21.15.29:0/3035575097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec0039c0 msgr2=0x7f6600164370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6604845640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec0039c0 0x7f6600164370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6604845640 1 -- 172.21.15.29:0/3035575097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6600165290 con 0x7f6600165770 2024-04-25T00:38:53.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6605046640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec0039c0 0x7f6600164370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:53.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6604845640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6600165770 0x7f6600163790 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f65f0010850 tx=0x7f65f0010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:53.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f65f5ffb640 1 -- 172.21.15.29:0/3035575097 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f000f040 con 0x7f6600165770 2024-04-25T00:38:53.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f65f5ffb640 1 -- 172.21.15.29:0/3035575097 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f000eb00 con 0x7f6600165770 2024-04-25T00:38:53.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f66072aa640 1 -- 172.21.15.29:0/3035575097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66001633a0 con 0x7f6600165770 2024-04-25T00:38:53.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f65f5ffb640 1 -- 172.21.15.29:0/3035575097 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f001c400 con 0x7f6600165770 2024-04-25T00:38:53.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f66072aa640 1 -- 172.21.15.29:0/3035575097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6600157cf0 con 0x7f6600165770 2024-04-25T00:38:53.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f66072aa640 1 -- 172.21.15.29:0/3035575097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6600069270 con 0x7f6600165770 2024-04-25T00:38:53.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f65f5ffb640 1 -- 172.21.15.29:0/3035575097 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f65f001c5a0 con 0x7f6600165770 2024-04-25T00:38:53.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f65f5ffb640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f65e4067e90 0x7f65e406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f65f5ffb640 1 -- 172.21.15.29:0/3035575097 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f65f0086d50 con 0x7f6600165770 2024-04-25T00:38:53.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6605046640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f65e4067e90 0x7f65e406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.566+0000 7f6605046640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f65e4067e90 0x7f65e406a350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0ad000 tx=0x7f65fc0b2040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:53.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.570+0000 7f65f5ffb640 1 -- 172.21.15.29:0/3035575097 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f65f008d080 con 0x7f6600165770 2024-04-25T00:38:53.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.734+0000 7f66072aa640 1 -- 172.21.15.29:0/3035575097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6600072fa0 con 0x7f6600165770 2024-04-25T00:38:53.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.734+0000 7f65f5ffb640 1 -- 172.21.15.29:0/3035575097 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16742 (secure 0 0 0) 0x7f65f005be20 con 0x7f6600165770 2024-04-25T00:38:53.739 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:53.739 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":119,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:51.470777+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:38:28.743593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:38:30.051271+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:53.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.738+0000 7f66072aa640 1 -- 172.21.15.29:0/3035575097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f65e4067e90 msgr2=0x7f65e406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.738+0000 7f66072aa640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f65e4067e90 0x7f65e406a350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0ad000 tx=0x7f65fc0b2040 comp rx=0 tx=0).stop 2024-04-25T00:38:53.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.738+0000 7f66072aa640 1 -- 172.21.15.29:0/3035575097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6600165770 msgr2=0x7f6600163790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.738+0000 7f66072aa640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6600165770 0x7f6600163790 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f65f0010850 tx=0x7f65f0010d10 comp rx=0 tx=0).stop 2024-04-25T00:38:53.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.738+0000 7f66072aa640 1 -- 172.21.15.29:0/3035575097 shutdown_connections 2024-04-25T00:38:53.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.738+0000 7f66072aa640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f65e4067e90 0x7f65e406a350 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.738+0000 7f66072aa640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66001517e0 0x7f6600163cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.738+0000 7f66072aa640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6600165770 0x7f6600163790 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.738+0000 7f66072aa640 1 --2- 172.21.15.29:0/3035575097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec0039c0 0x7f6600164370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.738+0000 7f66072aa640 1 -- 172.21.15.29:0/3035575097 >> 172.21.15.29:0/3035575097 conn(0x7f66000b1870 msgr2=0x7f6600066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:53.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.738+0000 7f66072aa640 1 -- 172.21.15.29:0/3035575097 shutdown_connections 2024-04-25T00:38:53.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.738+0000 7f66072aa640 1 -- 172.21.15.29:0/3035575097 wait complete. 2024-04-25T00:38:53.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:38:28.743593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:38:30.051271+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 119, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:38:53.759 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:38:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66ceb4b640 1 Processor -- start 2024-04-25T00:38:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66ceb4b640 1 -- start start 2024-04-25T00:38:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66ceb4b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66c80aec60 0x7f66c80cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66ceb4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66c80af040 0x7f66c815c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66ceb4b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66c815ccd0 con 0x7f66c815d720 2024-04-25T00:38:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66ceb4b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f66c815caa0 con 0x7f66c80aec60 2024-04-25T00:38:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66ceb4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f66c815cfb0 con 0x7f66c80af040 2024-04-25T00:38:53.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66cc8e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66c80aec60 0x7f66c80cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66cc8e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66c80aec60 0x7f66c80cfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:46898/0 (socket says 172.21.15.29:46898) 2024-04-25T00:38:53.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66c7fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66c80af040 0x7f66c815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66cd0e8640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f66c815d720 0x7f66c80b3b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:37910/0 (socket says 172.21.15.29:37910) 2024-04-25T00:38:53.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66cc8e7640 1 -- 172.21.15.29:0/2434402444 learned_addr learned my addr 172.21.15.29:0/2434402444 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:53.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66c7fff640 1 -- 172.21.15.29:0/2434402444 >> v2:172.21.15.157:3300/0 conn(0x7f66c80aec60 msgr2=0x7f66c80cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:53.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66c7fff640 1 --2- 172.21.15.29:0/2434402444 >> v2:172.21.15.157:3300/0 conn(0x7f66c80aec60 0x7f66c80cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66c7fff640 1 -- 172.21.15.29:0/2434402444 >> v1:172.21.15.157:6789/0 conn(0x7f66c815d720 legacy=0x7f66c80b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.878+0000 7f66c7fff640 1 -- 172.21.15.29:0/2434402444 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c815d290 con 0x7f66c80af040 2024-04-25T00:38:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66c7fff640 1 --2- 172.21.15.29:0/2434402444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66c80af040 0x7f66c815c560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f66b00097c0 tx=0x7f66b001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37ced24577631522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66c77fe640 1 -- 172.21.15.29:0/2434402444 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00b0070 con 0x7f66c80af040 2024-04-25T00:38:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66c77fe640 1 -- 172.21.15.29:0/2434402444 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b00ab600 con 0x7f66c80af040 2024-04-25T00:38:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66c77fe640 1 -- 172.21.15.29:0/2434402444 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00ab9d0 con 0x7f66c80af040 2024-04-25T00:38:53.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66ceb4b640 1 -- 172.21.15.29:0/2434402444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66c80af040 msgr2=0x7f66c815c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66ceb4b640 1 --2- 172.21.15.29:0/2434402444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66c80af040 0x7f66c815c560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f66b00097c0 tx=0x7f66b001c410 comp rx=0 tx=0).stop 2024-04-25T00:38:53.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66ceb4b640 1 -- 172.21.15.29:0/2434402444 shutdown_connections 2024-04-25T00:38:53.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66ceb4b640 1 --2- 172.21.15.29:0/2434402444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66c80af040 0x7f66c815c560 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66ceb4b640 1 --2- 172.21.15.29:0/2434402444 >> v2:172.21.15.157:3300/0 conn(0x7f66c80aec60 0x7f66c80cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66ceb4b640 1 -- 172.21.15.29:0/2434402444 >> 172.21.15.29:0/2434402444 conn(0x7f66c8075030 msgr2=0x7f66c80b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:53.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66ceb4b640 1 -- 172.21.15.29:0/2434402444 shutdown_connections 2024-04-25T00:38:53.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66ceb4b640 1 -- 172.21.15.29:0/2434402444 wait complete. 2024-04-25T00:38:53.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66ceb4b640 1 Processor -- start 2024-04-25T00:38:53.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.882+0000 7f66ceb4b640 1 -- start start 2024-04-25T00:38:53.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66ceb4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66c80aec60 0x7f66c8146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66ceb4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66c80af040 0x7f66c80b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66ceb4b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c815d720 0x7f66c80b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66ceb4b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8168d20 con 0x7f66c815d720 2024-04-25T00:38:53.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66ceb4b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8168ba0 con 0x7f66c80aec60 2024-04-25T00:38:53.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66ceb4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f66c8168ea0 con 0x7f66c80af040 2024-04-25T00:38:53.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66cd0e8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c815d720 0x7f66c80b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66cd0e8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c815d720 0x7f66c80b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50630/0 (socket says 172.21.15.29:50630) 2024-04-25T00:38:53.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66cd0e8640 1 -- 172.21.15.29:0/1062882240 learned_addr learned my addr 172.21.15.29:0/1062882240 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:53.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66cc8e7640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66c80aec60 0x7f66c8146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66c7fff640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66c80af040 0x7f66c80b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66cd0e8640 1 -- 172.21.15.29:0/1062882240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66c80af040 msgr2=0x7f66c80b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66cd0e8640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66c80af040 0x7f66c80b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66cd0e8640 1 -- 172.21.15.29:0/1062882240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66c80aec60 msgr2=0x7f66c8146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:53.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66cd0e8640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66c80aec60 0x7f66c8146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:53.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66cd0e8640 1 -- 172.21.15.29:0/1062882240 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c81472a0 con 0x7f66c815d720 2024-04-25T00:38:53.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66cd0e8640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c815d720 0x7f66c80b1630 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c390 tx=0x7f66b800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:53.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66c57fa640 1 -- 172.21.15.29:0/1062882240 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b80074e0 con 0x7f66c815d720 2024-04-25T00:38:53.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66c57fa640 1 -- 172.21.15.29:0/1062882240 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b8007680 con 0x7f66c815d720 2024-04-25T00:38:53.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66ceb4b640 1 -- 172.21.15.29:0/1062882240 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c8159b00 con 0x7f66c815d720 2024-04-25T00:38:53.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66c57fa640 1 -- 172.21.15.29:0/1062882240 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8004a10 con 0x7f66c815d720 2024-04-25T00:38:53.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66ceb4b640 1 -- 172.21.15.29:0/1062882240 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c815e4a0 con 0x7f66c815d720 2024-04-25T00:38:53.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66ceb4b640 1 -- 172.21.15.29:0/1062882240 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c8069270 con 0x7f66c815d720 2024-04-25T00:38:53.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66c57fa640 1 -- 172.21.15.29:0/1062882240 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f66b801a020 con 0x7f66c815d720 2024-04-25T00:38:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66c57fa640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66a8067e60 0x7f66a806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66c57fa640 1 -- 172.21.15.29:0/1062882240 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f66b8085e60 con 0x7f66c815d720 2024-04-25T00:38:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66cc8e7640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66a8067e60 0x7f66a806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:53.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.886+0000 7f66cc8e7640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66a8067e60 0x7f66a806a320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009ce0 tx=0x7f66c000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:53.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:53.890+0000 7f66c57fa640 1 -- 172.21.15.29:0/1062882240 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f66b8056180 con 0x7f66c815d720 2024-04-25T00:38:54.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.046+0000 7f66ceb4b640 1 -- 172.21.15.29:0/1062882240 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f66c80afc80 con 0x7f66c815d720 2024-04-25T00:38:54.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.046+0000 7f66c57fa640 1 -- 172.21.15.29:0/1062882240 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7f66b8055a80 con 0x7f66c815d720 2024-04-25T00:38:54.051 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:38:54.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.050+0000 7f66ceb4b640 1 -- 172.21.15.29:0/1062882240 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66a8067e60 msgr2=0x7f66a806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.050+0000 7f66ceb4b640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66a8067e60 0x7f66a806a320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009ce0 tx=0x7f66c000b040 comp rx=0 tx=0).stop 2024-04-25T00:38:54.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.050+0000 7f66ceb4b640 1 -- 172.21.15.29:0/1062882240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c815d720 msgr2=0x7f66c80b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.050+0000 7f66ceb4b640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c815d720 0x7f66c80b1630 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c390 tx=0x7f66b800c850 comp rx=0 tx=0).stop 2024-04-25T00:38:54.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.050+0000 7f66ceb4b640 1 -- 172.21.15.29:0/1062882240 shutdown_connections 2024-04-25T00:38:54.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.050+0000 7f66ceb4b640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66a8067e60 0x7f66a806a320 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.050+0000 7f66ceb4b640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c815d720 0x7f66c80b1630 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.050+0000 7f66ceb4b640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66c80af040 0x7f66c80b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.050+0000 7f66ceb4b640 1 --2- 172.21.15.29:0/1062882240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66c80aec60 0x7f66c8146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.050+0000 7f66ceb4b640 1 -- 172.21.15.29:0/1062882240 >> 172.21.15.29:0/1062882240 conn(0x7f66c8075030 msgr2=0x7f66c80ac5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:54.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.050+0000 7f66ceb4b640 1 -- 172.21.15.29:0/1062882240 shutdown_connections 2024-04-25T00:38:54.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.050+0000 7f66ceb4b640 1 -- 172.21.15.29:0/1062882240 wait complete. 2024-04-25T00:38:54.068 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:38:54.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76c5c3a640 1 Processor -- start 2024-04-25T00:38:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76c5c3a640 1 -- start start 2024-04-25T00:38:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76c5c3a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76c015e8d0 0x7f76c00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76c5c3a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76c00b3720 con 0x7f76c00b4780 2024-04-25T00:38:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76c5c3a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76c00b39c0 con 0x7f76c015f3c0 2024-04-25T00:38:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76c5c3a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f76c00b42f0 con 0x7f76c015e8d0 2024-04-25T00:38:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76beffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f76c00b4780 0x7f76c015ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36826/0 (socket says 172.21.15.29:36826) 2024-04-25T00:38:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76beffd640 1 -- 172.21.15.29:0/1909567837 learned_addr learned my addr 172.21.15.29:0/1909567837 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 689072580 0 0) 0x7f76c00b3720 con 0x7f76c00b4780 2024-04-25T00:38:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76a8003530 con 0x7f76c00b4780 2024-04-25T00:38:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76bf7fe640 1 --2- 172.21.15.29:0/1909567837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76c015e8d0 0x7f76c00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3611172688 0 0) 0x7f76a8003530 con 0x7f76c00b4780 2024-04-25T00:38:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76c015e8d0 msgr2=0x7f76c00cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 --2- 172.21.15.29:0/1909567837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76c015e8d0 0x7f76c00cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 >> v1:172.21.15.157:6789/0 conn(0x7f76c015f3c0 legacy=0x7f76c015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c00b3ca0 con 0x7f76c00b4780 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f76b0001680 con 0x7f76c00b4780 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 >> v1:172.21.15.29:6789/0 conn(0x7f76c00b4780 legacy=0x7f76c015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 --2- 172.21.15.29:0/1909567837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76a80039f0 0x7f76a8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 --2- 172.21.15.29:0/1909567837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76c015e8d0 0x7f76a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 --2- 172.21.15.29:0/1909567837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76a8004d50 0x7f76a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f76c00b42f0 con 0x7f76a80039f0 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f76a800bb60 con 0x7f76a8004d50 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f76a800bd60 con 0x7f76c015e8d0 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76bf7fe640 1 --2- 172.21.15.29:0/1909567837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76a80039f0 0x7f76a8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76beffd640 1 --2- 172.21.15.29:0/1909567837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76c015e8d0 0x7f76a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76bf7fe640 1 -- 172.21.15.29:0/1909567837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76c015e8d0 msgr2=0x7f76a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:54.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76bffff640 1 --2- 172.21.15.29:0/1909567837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76a8004d50 0x7f76a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76bf7fe640 1 --2- 172.21.15.29:0/1909567837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76c015e8d0 0x7f76a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76bf7fe640 1 -- 172.21.15.29:0/1909567837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76a8004d50 msgr2=0x7f76a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:54.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76bf7fe640 1 --2- 172.21.15.29:0/1909567837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76a8004d50 0x7f76a80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76bf7fe640 1 -- 172.21.15.29:0/1909567837 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a8003530 con 0x7f76a80039f0 2024-04-25T00:38:54.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.182+0000 7f76beffd640 1 --2- 172.21.15.29:0/1909567837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76c015e8d0 0x7f76a8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76bf7fe640 1 --2- 172.21.15.29:0/1909567837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76a80039f0 0x7f76a8003df0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f76b400ba30 tx=0x7f76b401c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e221e7aab76211e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b400b600 con 0x7f76a80039f0 2024-04-25T00:38:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76b40b1070 con 0x7f76a80039f0 2024-04-25T00:38:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76be7fc640 1 -- 172.21.15.29:0/1909567837 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b40ac850 con 0x7f76a80039f0 2024-04-25T00:38:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 -- 172.21.15.29:0/1909567837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76a80039f0 msgr2=0x7f76a8003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 --2- 172.21.15.29:0/1909567837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76a80039f0 0x7f76a8003df0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f76b400ba30 tx=0x7f76b401c670 comp rx=0 tx=0).stop 2024-04-25T00:38:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76bffff640 1 -- 172.21.15.29:0/1909567837 reap_dead start 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 -- 172.21.15.29:0/1909567837 shutdown_connections 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 -- 172.21.15.29:0/1909567837 >> 172.21.15.29:0/1909567837 conn(0x7f76c00b1870 msgr2=0x7f76c00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 -- 172.21.15.29:0/1909567837 shutdown_connections 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 -- 172.21.15.29:0/1909567837 wait complete. 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 Processor -- start 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 -- start start 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76a80039f0 0x7f76c0139770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c013ef10 0x7f76c013d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76c013d960 0x7f76c00cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0166df0 con 0x7f76c013d960 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0166c70 con 0x7f76c013ef10 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76c5c3a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f76c0166f70 con 0x7f76a80039f0 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76bffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76c013d960 0x7f76c00cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76bffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76c013d960 0x7f76c00cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50650/0 (socket says 172.21.15.29:50650) 2024-04-25T00:38:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76bffff640 1 -- 172.21.15.29:0/372812774 learned_addr learned my addr 172.21.15.29:0/372812774 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76beffd640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c013ef10 0x7f76c013d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76bf7fe640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76a80039f0 0x7f76c0139770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76bffff640 1 -- 172.21.15.29:0/372812774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76a80039f0 msgr2=0x7f76c0139770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76bffff640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76a80039f0 0x7f76c0139770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76bffff640 1 -- 172.21.15.29:0/372812774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c013ef10 msgr2=0x7f76c013d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76bffff640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c013ef10 0x7f76c013d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76bffff640 1 -- 172.21.15.29:0/372812774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c01445f0 con 0x7f76c013d960 2024-04-25T00:38:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.186+0000 7f76bffff640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76c013d960 0x7f76c00cd250 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f76c015f9a0 tx=0x7f76b800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.190+0000 7f76a7fff640 1 -- 172.21.15.29:0/372812774 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8012800 con 0x7f76c013d960 2024-04-25T00:38:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.190+0000 7f76a7fff640 1 -- 172.21.15.29:0/372812774 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76b8019070 con 0x7f76c013d960 2024-04-25T00:38:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.190+0000 7f76c5c3a640 1 -- 172.21.15.29:0/372812774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c00cc070 con 0x7f76c013d960 2024-04-25T00:38:54.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.190+0000 7f76a7fff640 1 -- 172.21.15.29:0/372812774 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8012c40 con 0x7f76c013d960 2024-04-25T00:38:54.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.190+0000 7f76c5c3a640 1 -- 172.21.15.29:0/372812774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c013de70 con 0x7f76c013d960 2024-04-25T00:38:54.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.190+0000 7f76c5c3a640 1 -- 172.21.15.29:0/372812774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c013ec00 con 0x7f76c013d960 2024-04-25T00:38:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.190+0000 7f76a7fff640 1 -- 172.21.15.29:0/372812774 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f76b801e460 con 0x7f76c013d960 2024-04-25T00:38:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.190+0000 7f76a7fff640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f76ac067e60 0x7f76ac06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.190+0000 7f76a7fff640 1 -- 172.21.15.29:0/372812774 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f76b80865d0 con 0x7f76c013d960 2024-04-25T00:38:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.190+0000 7f76bf7fe640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f76ac067e60 0x7f76ac06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.190+0000 7f76bf7fe640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f76ac067e60 0x7f76ac06a320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f76b40b0000 tx=0x7f76b40046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.194+0000 7f76a7fff640 1 -- 172.21.15.29:0/372812774 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f76b8056760 con 0x7f76c013d960 2024-04-25T00:38:54.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.354+0000 7f76c5c3a640 1 -- 172.21.15.29:0/372812774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f76c006e7b0 con 0x7f76c013d960 2024-04-25T00:38:54.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.354+0000 7f76a7fff640 1 -- 172.21.15.29:0/372812774 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) ==== 104+0+11 (secure 0 0 0) 0x7f76b8056060 con 0x7f76c013d960 2024-04-25T00:38:54.359 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:38:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.358+0000 7f76c5c3a640 1 -- 172.21.15.29:0/372812774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f76ac067e60 msgr2=0x7f76ac06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.358+0000 7f76c5c3a640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f76ac067e60 0x7f76ac06a320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f76b40b0000 tx=0x7f76b40046d0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.358+0000 7f76c5c3a640 1 -- 172.21.15.29:0/372812774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76c013d960 msgr2=0x7f76c00cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.358+0000 7f76c5c3a640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76c013d960 0x7f76c00cd250 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f76c015f9a0 tx=0x7f76b800d9f0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.358+0000 7f76c5c3a640 1 -- 172.21.15.29:0/372812774 shutdown_connections 2024-04-25T00:38:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.358+0000 7f76c5c3a640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f76ac067e60 0x7f76ac06a320 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.358+0000 7f76c5c3a640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f76c013d960 0x7f76c00cd250 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.358+0000 7f76c5c3a640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c013ef10 0x7f76c013d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.358+0000 7f76c5c3a640 1 --2- 172.21.15.29:0/372812774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76a80039f0 0x7f76c0139770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.358+0000 7f76c5c3a640 1 -- 172.21.15.29:0/372812774 >> 172.21.15.29:0/372812774 conn(0x7f76c00b1870 msgr2=0x7f76c015ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:54.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.358+0000 7f76c5c3a640 1 -- 172.21.15.29:0/372812774 shutdown_connections 2024-04-25T00:38:54.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.358+0000 7f76c5c3a640 1 -- 172.21.15.29:0/372812774 wait complete. 2024-04-25T00:38:54.378 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:38:54.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c4ddbc640 1 Processor -- start 2024-04-25T00:38:54.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c4ddbc640 1 -- start start 2024-04-25T00:38:54.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c4ddbc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4815e200 0x7f2c480d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c4ddbc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2c480d2c60 0x7f2c480b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c4ddbc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c480cfc10 con 0x7f2c480cf690 2024-04-25T00:38:54.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c4ddbc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2c480b38d0 con 0x7f2c480d2c60 2024-04-25T00:38:54.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c4ddbc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2c480cfef0 con 0x7f2c4815e200 2024-04-25T00:38:54.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c47fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2c480d2c60 0x7f2c480b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c477fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4815e200 0x7f2c480d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c46ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2c480cf690 0x7f2c480d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:37928/0 (socket says 172.21.15.29:37928) 2024-04-25T00:38:54.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c46ffd640 1 -- 172.21.15.29:0/1303989043 learned_addr learned my addr 172.21.15.29:0/1303989043 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:54.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c477fe640 1 -- 172.21.15.29:0/1303989043 >> v2:172.21.15.157:3300/0 conn(0x7f2c480d2c60 msgr2=0x7f2c480b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:54.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c477fe640 1 --2- 172.21.15.29:0/1303989043 >> v2:172.21.15.157:3300/0 conn(0x7f2c480d2c60 0x7f2c480b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c477fe640 1 -- 172.21.15.29:0/1303989043 >> v1:172.21.15.157:6789/0 conn(0x7f2c480cf690 legacy=0x7f2c480d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c477fe640 1 -- 172.21.15.29:0/1303989043 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c480b3b20 con 0x7f2c4815e200 2024-04-25T00:38:54.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.498+0000 7f2c47fff640 1 --2- 172.21.15.29:0/1303989043 >> v2:172.21.15.157:3300/0 conn(0x7f2c480d2c60 0x7f2c480b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:38:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.502+0000 7f2c477fe640 1 --2- 172.21.15.29:0/1303989043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4815e200 0x7f2c480d47f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c400097c0 tx=0x7f2c4001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d8bea2603447720 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.502+0000 7f2c467fc640 1 -- 172.21.15.29:0/1303989043 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c400aa5f0 con 0x7f2c4815e200 2024-04-25T00:38:54.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.502+0000 7f2c467fc640 1 -- 172.21.15.29:0/1303989043 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c400af070 con 0x7f2c4815e200 2024-04-25T00:38:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.502+0000 7f2c467fc640 1 -- 172.21.15.29:0/1303989043 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c400aa9b0 con 0x7f2c4815e200 2024-04-25T00:38:54.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.502+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/1303989043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4815e200 msgr2=0x7f2c480d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.502+0000 7f2c4ddbc640 1 --2- 172.21.15.29:0/1303989043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4815e200 0x7f2c480d47f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c400097c0 tx=0x7f2c4001c470 comp rx=0 tx=0).stop 2024-04-25T00:38:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.502+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/1303989043 shutdown_connections 2024-04-25T00:38:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.502+0000 7f2c4ddbc640 1 --2- 172.21.15.29:0/1303989043 >> v2:172.21.15.157:3300/0 conn(0x7f2c480d2c60 0x7f2c480b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.502+0000 7f2c4ddbc640 1 --2- 172.21.15.29:0/1303989043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4815e200 0x7f2c480d47f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.502+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/1303989043 >> 172.21.15.29:0/1303989043 conn(0x7f2c4809d280 msgr2=0x7f2c480d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.502+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/1303989043 shutdown_connections 2024-04-25T00:38:54.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.502+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/1303989043 wait complete. 2024-04-25T00:38:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c4ddbc640 1 Processor -- start 2024-04-25T00:38:54.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c4ddbc640 1 -- start start 2024-04-25T00:38:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c4ddbc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c480cf690 0x7f2c48158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c4ddbc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c480d2c60 0x7f2c481611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c4ddbc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4815e200 0x7f2c4806c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c4ddbc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2c480b51f0 con 0x7f2c480d2c60 2024-04-25T00:38:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c4ddbc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2c480b5070 con 0x7f2c480cf690 2024-04-25T00:38:54.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c4ddbc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2c480b5370 con 0x7f2c4815e200 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c46ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c480d2c60 0x7f2c481611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c46ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c480d2c60 0x7f2c481611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50656/0 (socket says 172.21.15.29:50656) 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c46ffd640 1 -- 172.21.15.29:0/3094902120 learned_addr learned my addr 172.21.15.29:0/3094902120 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c477fe640 1 --2- 172.21.15.29:0/3094902120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c480cf690 0x7f2c48158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c46ffd640 1 -- 172.21.15.29:0/3094902120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4815e200 msgr2=0x7f2c4806c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c46ffd640 1 --2- 172.21.15.29:0/3094902120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4815e200 0x7f2c4806c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c46ffd640 1 -- 172.21.15.29:0/3094902120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c480cf690 msgr2=0x7f2c48158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c46ffd640 1 --2- 172.21.15.29:0/3094902120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c480cf690 0x7f2c48158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c46ffd640 1 -- 172.21.15.29:0/3094902120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c48150400 con 0x7f2c480d2c60 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c46ffd640 1 --2- 172.21.15.29:0/3094902120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c480d2c60 0x7f2c481611b0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000c450 tx=0x7f2c3000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c3ffff640 1 -- 172.21.15.29:0/3094902120 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c30017070 con 0x7f2c480d2c60 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c3ffff640 1 -- 172.21.15.29:0/3094902120 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c30005cf0 con 0x7f2c480d2c60 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/3094902120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c481506e0 con 0x7f2c480d2c60 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c3ffff640 1 -- 172.21.15.29:0/3094902120 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c300072f0 con 0x7f2c480d2c60 2024-04-25T00:38:54.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/3094902120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4814fc90 con 0x7f2c480d2c60 2024-04-25T00:38:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.506+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/3094902120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4814ffb0 con 0x7f2c480d2c60 2024-04-25T00:38:54.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.510+0000 7f2c3ffff640 1 -- 172.21.15.29:0/3094902120 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f2c3001a070 con 0x7f2c480d2c60 2024-04-25T00:38:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.510+0000 7f2c3ffff640 1 --2- 172.21.15.29:0/3094902120 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c28067e90 0x7f2c2806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.510+0000 7f2c3ffff640 1 -- 172.21.15.29:0/3094902120 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f2c300859f0 con 0x7f2c480d2c60 2024-04-25T00:38:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.510+0000 7f2c477fe640 1 --2- 172.21.15.29:0/3094902120 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c28067e90 0x7f2c2806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.510+0000 7f2c477fe640 1 --2- 172.21.15.29:0/3094902120 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c28067e90 0x7f2c2806a350 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c400ae000 tx=0x7f2c400a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:54.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.514+0000 7f2c3ffff640 1 -- 172.21.15.29:0/3094902120 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2c4814ffb0 con 0x7f2c480d2c60 2024-04-25T00:38:54.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.666+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/3094902120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2c48069270 con 0x7f2c480d2c60 2024-04-25T00:38:54.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.666+0000 7f2c3ffff640 1 -- 172.21.15.29:0/3094902120 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) ==== 100+0+11 (secure 0 0 0) 0x7f2c30055d10 con 0x7f2c480d2c60 2024-04-25T00:38:54.670 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:38:54.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.670+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/3094902120 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c28067e90 msgr2=0x7f2c2806a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.670+0000 7f2c4ddbc640 1 --2- 172.21.15.29:0/3094902120 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c28067e90 0x7f2c2806a350 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c400ae000 tx=0x7f2c400a7000 comp rx=0 tx=0).stop 2024-04-25T00:38:54.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.670+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/3094902120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c480d2c60 msgr2=0x7f2c481611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.670+0000 7f2c4ddbc640 1 --2- 172.21.15.29:0/3094902120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c480d2c60 0x7f2c481611b0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000c450 tx=0x7f2c3000c910 comp rx=0 tx=0).stop 2024-04-25T00:38:54.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.670+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/3094902120 shutdown_connections 2024-04-25T00:38:54.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.670+0000 7f2c4ddbc640 1 --2- 172.21.15.29:0/3094902120 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c28067e90 0x7f2c2806a350 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.670+0000 7f2c4ddbc640 1 --2- 172.21.15.29:0/3094902120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c4815e200 0x7f2c4806c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.670+0000 7f2c4ddbc640 1 --2- 172.21.15.29:0/3094902120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c480d2c60 0x7f2c481611b0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.670+0000 7f2c4ddbc640 1 --2- 172.21.15.29:0/3094902120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c480cf690 0x7f2c48158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.670+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/3094902120 >> 172.21.15.29:0/3094902120 conn(0x7f2c4809d280 msgr2=0x7f2c4806d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:54.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.670+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/3094902120 shutdown_connections 2024-04-25T00:38:54.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.670+0000 7f2c4ddbc640 1 -- 172.21.15.29:0/3094902120 wait complete. 2024-04-25T00:38:54.688 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:38:54.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdad16cb640 1 Processor -- start 2024-04-25T00:38:54.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdad16cb640 1 -- start start 2024-04-25T00:38:54.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdad16cb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdacc0b3140 0x7fdacc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdad16cb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdacc0b3c30 0x7fdacc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdad16cb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdacc0b41e0 con 0x7fdacc0b4b10 2024-04-25T00:38:54.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdad16cb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdacc06d960 con 0x7fdacc0b3c30 2024-04-25T00:38:54.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdad16cb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdacc06dc40 con 0x7fdacc0b3140 2024-04-25T00:38:54.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdacb7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdacc0b3c30 0x7fdacc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdaca7fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdacc0b4b10 0x7fdacc0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:37934/0 (socket says 172.21.15.29:37934) 2024-04-25T00:38:54.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdaca7fc640 1 -- 172.21.15.29:0/2372762924 learned_addr learned my addr 172.21.15.29:0/2372762924 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:54.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdacaffd640 1 --2- 172.21.15.29:0/2372762924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdacc0b3140 0x7fdacc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdac9ffb640 1 -- 172.21.15.29:0/2372762924 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 537619530 0 0) 0x7fdacc0b41e0 con 0x7fdacc0b4b10 2024-04-25T00:38:54.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdac9ffb640 1 -- 172.21.15.29:0/2372762924 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdab4003530 con 0x7fdacc0b4b10 2024-04-25T00:38:54.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdacaffd640 1 -- 172.21.15.29:0/2372762924 >> v2:172.21.15.157:3300/0 conn(0x7fdacc0b3c30 msgr2=0x7fdacc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:54.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdacaffd640 1 --2- 172.21.15.29:0/2372762924 >> v2:172.21.15.157:3300/0 conn(0x7fdacc0b3c30 0x7fdacc0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdacaffd640 1 -- 172.21.15.29:0/2372762924 >> v1:172.21.15.157:6789/0 conn(0x7fdacc0b4b10 legacy=0x7fdacc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.806+0000 7fdacaffd640 1 -- 172.21.15.29:0/2372762924 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdacc06df20 con 0x7fdacc0b3140 2024-04-25T00:38:54.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdacaffd640 1 --2- 172.21.15.29:0/2372762924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdacc0b3140 0x7fdacc06e440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0097c0 tx=0x7fdabc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42b3a61c35efa31c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:54.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdac9ffb640 1 -- 172.21.15.29:0/2372762924 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0b0070 con 0x7fdacc0b3140 2024-04-25T00:38:54.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdac9ffb640 1 -- 172.21.15.29:0/2372762924 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdabc0ab600 con 0x7fdacc0b3140 2024-04-25T00:38:54.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdac9ffb640 1 -- 172.21.15.29:0/2372762924 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0ab9b0 con 0x7fdacc0b3140 2024-04-25T00:38:54.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdad16cb640 1 -- 172.21.15.29:0/2372762924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdacc0b3140 msgr2=0x7fdacc06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdad16cb640 1 --2- 172.21.15.29:0/2372762924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdacc0b3140 0x7fdacc06e440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0097c0 tx=0x7fdabc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdad16cb640 1 -- 172.21.15.29:0/2372762924 shutdown_connections 2024-04-25T00:38:54.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdad16cb640 1 --2- 172.21.15.29:0/2372762924 >> v2:172.21.15.157:3300/0 conn(0x7fdacc0b3c30 0x7fdacc0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdad16cb640 1 --2- 172.21.15.29:0/2372762924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdacc0b3140 0x7fdacc06e440 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdad16cb640 1 -- 172.21.15.29:0/2372762924 >> 172.21.15.29:0/2372762924 conn(0x7fdacc0b1870 msgr2=0x7fdacc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:54.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdad16cb640 1 -- 172.21.15.29:0/2372762924 shutdown_connections 2024-04-25T00:38:54.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdad16cb640 1 -- 172.21.15.29:0/2372762924 wait complete. 2024-04-25T00:38:54.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdad16cb640 1 Processor -- start 2024-04-25T00:38:54.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.810+0000 7fdad16cb640 1 -- start start 2024-04-25T00:38:54.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdad16cb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdacc0b3140 0x7fdacc070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdad16cb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdacc0b3c30 0x7fdacc137370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdad16cb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdacc0b4b10 0x7fdacc0ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdad16cb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc168d80 con 0x7fdacc0b3c30 2024-04-25T00:38:54.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdad16cb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc168c00 con 0x7fdacc0b3140 2024-04-25T00:38:54.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdad16cb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdacc168f00 con 0x7fdacc0b4b10 2024-04-25T00:38:54.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdaca7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdacc0b3c30 0x7fdacc137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdaca7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdacc0b3c30 0x7fdacc137370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50662/0 (socket says 172.21.15.29:50662) 2024-04-25T00:38:54.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdaca7fc640 1 -- 172.21.15.29:0/3053448305 learned_addr learned my addr 172.21.15.29:0/3053448305 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:54.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdacb7fe640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdacc0b4b10 0x7fdacc0ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdacaffd640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdacc0b3140 0x7fdacc070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdaca7fc640 1 -- 172.21.15.29:0/3053448305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdacc0b4b10 msgr2=0x7fdacc0ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdaca7fc640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdacc0b4b10 0x7fdacc0ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdaca7fc640 1 -- 172.21.15.29:0/3053448305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdacc0b3140 msgr2=0x7fdacc070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:54.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdaca7fc640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdacc0b3140 0x7fdacc070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:54.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdaca7fc640 1 -- 172.21.15.29:0/3053448305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdacc13a9e0 con 0x7fdacc0b3c30 2024-04-25T00:38:54.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdaca7fc640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdacc0b3c30 0x7fdacc137370 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c480 tx=0x7fdac000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:54.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdab37fe640 1 -- 172.21.15.29:0/3053448305 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0017070 con 0x7fdacc0b3c30 2024-04-25T00:38:54.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdab37fe640 1 -- 172.21.15.29:0/3053448305 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac0005cf0 con 0x7fdacc0b3c30 2024-04-25T00:38:54.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdad16cb640 1 -- 172.21.15.29:0/3053448305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdacc13acc0 con 0x7fdacc0b3c30 2024-04-25T00:38:54.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdab37fe640 1 -- 172.21.15.29:0/3053448305 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0007290 con 0x7fdacc0b3c30 2024-04-25T00:38:54.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdad16cb640 1 -- 172.21.15.29:0/3053448305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdacc154650 con 0x7fdacc0b3c30 2024-04-25T00:38:54.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdad16cb640 1 -- 172.21.15.29:0/3053448305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdacc069270 con 0x7fdacc0b3c30 2024-04-25T00:38:54.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdab37fe640 1 -- 172.21.15.29:0/3053448305 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fdac001a070 con 0x7fdacc0b3c30 2024-04-25T00:38:54.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdab37fe640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdab8067e60 0x7fdab806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:54.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdab37fe640 1 -- 172.21.15.29:0/3053448305 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fdac0085fa0 con 0x7fdacc0b3c30 2024-04-25T00:38:54.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.814+0000 7fdacaffd640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdab8067e60 0x7fdab806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:54.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.818+0000 7fdacaffd640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdab8067e60 0x7fdab806a320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0ad000 tx=0x7fdabc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:54.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:54.822+0000 7fdab37fe640 1 -- 172.21.15.29:0/3053448305 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdac0021020 con 0x7fdacc0b3c30 2024-04-25T00:38:55.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdad16cb640 1 -- 172.21.15.29:0/3053448305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdacc13b650 con 0x7fdacc0b3c30 2024-04-25T00:38:55.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdab37fe640 1 -- 172.21.15.29:0/3053448305 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5186 (secure 0 0 0) 0x7fdac00562c0 con 0x7fdacc0b3c30 2024-04-25T00:38:55.021 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:55.021 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2148970077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2148970077},{"type":"v1","addr":"172.21.15.157:6834","nonce":2148970077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3947933400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3947933400},{"type":"v1","addr":"172.21.15.157:6835","nonce":3947933400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/172449010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":172449010},{"type":"v1","addr":"172.21.15.29:6833","nonce":172449010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:38:31.061384+0000","modified":"2024-04-25T00:38:34.097007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/3206404146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3206404146},{"type":"v1","addr":"172.21.15.29:6835","nonce":3206404146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8520,"qdb_cluster":[8520]},"id":9}]} 2024-04-25T00:38:55.021 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 76 2024-04-25T00:38:55.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdad16cb640 1 -- 172.21.15.29:0/3053448305 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdab8067e60 msgr2=0x7fdab806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdad16cb640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdab8067e60 0x7fdab806a320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0ad000 tx=0x7fdabc008040 comp rx=0 tx=0).stop 2024-04-25T00:38:55.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdad16cb640 1 -- 172.21.15.29:0/3053448305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdacc0b3c30 msgr2=0x7fdacc137370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdad16cb640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdacc0b3c30 0x7fdacc137370 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c480 tx=0x7fdac000c940 comp rx=0 tx=0).stop 2024-04-25T00:38:55.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdad16cb640 1 -- 172.21.15.29:0/3053448305 shutdown_connections 2024-04-25T00:38:55.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdad16cb640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdab8067e60 0x7fdab806a320 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdad16cb640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdacc0b4b10 0x7fdacc0ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdad16cb640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdacc0b3c30 0x7fdacc137370 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdad16cb640 1 --2- 172.21.15.29:0/3053448305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdacc0b3140 0x7fdacc070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdad16cb640 1 -- 172.21.15.29:0/3053448305 >> 172.21.15.29:0/3053448305 conn(0x7fdacc0b1870 msgr2=0x7fdacc1438d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:55.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdad16cb640 1 -- 172.21.15.29:0/3053448305 shutdown_connections 2024-04-25T00:38:55.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.018+0000 7fdad16cb640 1 -- 172.21.15.29:0/3053448305 wait complete. 2024-04-25T00:38:55.042 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:55.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8ba4905640 1 Processor -- start 2024-04-25T00:38:55.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8ba4905640 1 -- start start 2024-04-25T00:38:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8ba4905640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba015e8d0 0x7f8ba00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8ba4905640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ba015f3c0 0x7f8ba015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8ba4905640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba00b3720 con 0x7f8ba00b4780 2024-04-25T00:38:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8ba4905640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8ba00b42f0 con 0x7f8ba015f3c0 2024-04-25T00:38:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8ba4905640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba00b39c0 con 0x7f8ba015e8d0 2024-04-25T00:38:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8b9e59c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba015e8d0 0x7f8ba00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8b9ed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ba015f3c0 0x7f8ba015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8b9dd9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8ba00b4780 0x7f8ba015ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:37936/0 (socket says 172.21.15.29:37936) 2024-04-25T00:38:55.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8b9dd9b640 1 -- 172.21.15.29:0/38054453 learned_addr learned my addr 172.21.15.29:0/38054453 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:55.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8b9d59a640 1 -- 172.21.15.29:0/38054453 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4035033423 0 0) 0x7f8ba00b3720 con 0x7f8ba00b4780 2024-04-25T00:38:55.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8b9d59a640 1 -- 172.21.15.29:0/38054453 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b88003530 con 0x7f8ba00b4780 2024-04-25T00:38:55.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8b9e59c640 1 -- 172.21.15.29:0/38054453 >> v2:172.21.15.157:3300/0 conn(0x7f8ba015f3c0 msgr2=0x7f8ba015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:55.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8b9e59c640 1 --2- 172.21.15.29:0/38054453 >> v2:172.21.15.157:3300/0 conn(0x7f8ba015f3c0 0x7f8ba015c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8b9e59c640 1 -- 172.21.15.29:0/38054453 >> v1:172.21.15.157:6789/0 conn(0x7f8ba00b4780 legacy=0x7f8ba015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8b9e59c640 1 -- 172.21.15.29:0/38054453 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba00b3ca0 con 0x7f8ba015e8d0 2024-04-25T00:38:55.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.150+0000 7f8b9ed9d640 1 --2- 172.21.15.29:0/38054453 >> v2:172.21.15.157:3300/0 conn(0x7f8ba015f3c0 0x7f8ba015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:38:55.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8b9e59c640 1 --2- 172.21.15.29:0/38054453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba015e8d0 0x7f8ba00cc240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8b900097c0 tx=0x7f8b9001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba021fcedaea81f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:55.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8b9d59a640 1 -- 172.21.15.29:0/38054453 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b900b2070 con 0x7f8ba015e8d0 2024-04-25T00:38:55.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8b9d59a640 1 -- 172.21.15.29:0/38054453 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b900ad680 con 0x7f8ba015e8d0 2024-04-25T00:38:55.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8b9d59a640 1 -- 172.21.15.29:0/38054453 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b900ada50 con 0x7f8ba015e8d0 2024-04-25T00:38:55.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 -- 172.21.15.29:0/38054453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba015e8d0 msgr2=0x7f8ba00cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 --2- 172.21.15.29:0/38054453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba015e8d0 0x7f8ba00cc240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8b900097c0 tx=0x7f8b9001c470 comp rx=0 tx=0).stop 2024-04-25T00:38:55.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 -- 172.21.15.29:0/38054453 shutdown_connections 2024-04-25T00:38:55.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 --2- 172.21.15.29:0/38054453 >> v2:172.21.15.157:3300/0 conn(0x7f8ba015f3c0 0x7f8ba015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 --2- 172.21.15.29:0/38054453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba015e8d0 0x7f8ba00cc240 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 -- 172.21.15.29:0/38054453 >> 172.21.15.29:0/38054453 conn(0x7f8ba00b1870 msgr2=0x7f8ba00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:55.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 -- 172.21.15.29:0/38054453 shutdown_connections 2024-04-25T00:38:55.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 -- 172.21.15.29:0/38054453 wait complete. 2024-04-25T00:38:55.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 Processor -- start 2024-04-25T00:38:55.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 -- start start 2024-04-25T00:38:55.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ba00b4780 0x7f8ba0072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ba015e8d0 0x7f8ba0136310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba015f3c0 0x7f8ba00ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba015d6a0 con 0x7f8ba00b4780 2024-04-25T00:38:55.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba015d520 con 0x7f8ba015e8d0 2024-04-25T00:38:55.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8ba4905640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba0167c70 con 0x7f8ba015f3c0 2024-04-25T00:38:55.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8b9e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ba00b4780 0x7f8ba0072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8b9e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ba00b4780 0x7f8ba0072880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50676/0 (socket says 172.21.15.29:50676) 2024-04-25T00:38:55.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.154+0000 7f8b9e59c640 1 -- 172.21.15.29:0/3578118022 learned_addr learned my addr 172.21.15.29:0/3578118022 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:55.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b9dd9b640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ba015e8d0 0x7f8ba0136310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b9ed9d640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba015f3c0 0x7f8ba00ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b9e59c640 1 -- 172.21.15.29:0/3578118022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba015f3c0 msgr2=0x7f8ba00ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b9e59c640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba015f3c0 0x7f8ba00ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b9e59c640 1 -- 172.21.15.29:0/3578118022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ba015e8d0 msgr2=0x7f8ba0136310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b9e59c640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ba015e8d0 0x7f8ba0136310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b9e59c640 1 -- 172.21.15.29:0/3578118022 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba01369d0 con 0x7f8ba00b4780 2024-04-25T00:38:55.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b9ed9d640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba015f3c0 0x7f8ba00ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:55.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b9e59c640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ba00b4780 0x7f8ba0072880 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f8b9001ab20 tx=0x7f8b9001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:55.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b8effd640 1 -- 172.21.15.29:0/3578118022 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b900bc020 con 0x7f8ba00b4780 2024-04-25T00:38:55.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b8effd640 1 -- 172.21.15.29:0/3578118022 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b900adb30 con 0x7f8ba00b4780 2024-04-25T00:38:55.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b8effd640 1 -- 172.21.15.29:0/3578118022 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b900b2040 con 0x7f8ba00b4780 2024-04-25T00:38:55.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8ba4905640 1 -- 172.21.15.29:0/3578118022 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba0136cb0 con 0x7f8ba00b4780 2024-04-25T00:38:55.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8ba4905640 1 -- 172.21.15.29:0/3578118022 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba0137270 con 0x7f8ba00b4780 2024-04-25T00:38:55.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8ba4905640 1 -- 172.21.15.29:0/3578118022 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba0069270 con 0x7f8ba00b4780 2024-04-25T00:38:55.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b8effd640 1 -- 172.21.15.29:0/3578118022 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f8b900bf020 con 0x7f8ba00b4780 2024-04-25T00:38:55.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b8effd640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b80067e90 0x7f8b8006a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b8effd640 1 -- 172.21.15.29:0/3578118022 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f8b901208c0 con 0x7f8ba00b4780 2024-04-25T00:38:55.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b9dd9b640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b80067e90 0x7f8b8006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.158+0000 7f8b9dd9b640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b80067e90 0x7f8b8006a350 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8b94007fd0 tx=0x7f8b94009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:55.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.162+0000 7f8b8effd640 1 -- 172.21.15.29:0/3578118022 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8b900c6030 con 0x7f8ba00b4780 2024-04-25T00:38:55.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.326+0000 7f8ba4905640 1 -- 172.21.15.29:0/3578118022 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ba00cc2d0 con 0x7f8ba00b4780 2024-04-25T00:38:55.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.326+0000 7f8b8effd640 1 -- 172.21.15.29:0/3578118022 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16742 (secure 0 0 0) 0x7f8b90117e20 con 0x7f8ba00b4780 2024-04-25T00:38:55.331 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:55.331 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":119,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:51.470777+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:38:28.743593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:38:30.051271+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:55.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.330+0000 7f8ba4905640 1 -- 172.21.15.29:0/3578118022 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b80067e90 msgr2=0x7f8b8006a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.330+0000 7f8ba4905640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b80067e90 0x7f8b8006a350 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8b94007fd0 tx=0x7f8b94009040 comp rx=0 tx=0).stop 2024-04-25T00:38:55.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.330+0000 7f8ba4905640 1 -- 172.21.15.29:0/3578118022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ba00b4780 msgr2=0x7f8ba0072880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.330+0000 7f8ba4905640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ba00b4780 0x7f8ba0072880 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f8b9001ab20 tx=0x7f8b9001acb0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.330+0000 7f8ba4905640 1 -- 172.21.15.29:0/3578118022 shutdown_connections 2024-04-25T00:38:55.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.330+0000 7f8ba4905640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b80067e90 0x7f8b8006a350 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.330+0000 7f8ba4905640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba015f3c0 0x7f8ba00ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.330+0000 7f8ba4905640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ba015e8d0 0x7f8ba0136310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.330+0000 7f8ba4905640 1 --2- 172.21.15.29:0/3578118022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ba00b4780 0x7f8ba0072880 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.330+0000 7f8ba4905640 1 -- 172.21.15.29:0/3578118022 >> 172.21.15.29:0/3578118022 conn(0x7f8ba00b1870 msgr2=0x7f8ba006df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:55.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.330+0000 7f8ba4905640 1 -- 172.21.15.29:0/3578118022 shutdown_connections 2024-04-25T00:38:55.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.330+0000 7f8ba4905640 1 -- 172.21.15.29:0/3578118022 wait complete. 2024-04-25T00:38:55.349 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:38:55.350 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:38:55.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86cd779640 1 Processor -- start 2024-04-25T00:38:55.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86cd779640 1 -- start start 2024-04-25T00:38:55.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86cd779640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c815df70 0x7f86c80b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86cd779640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f86c806d210 0x7f86c80b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86cd779640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86c806d980 con 0x7f86c80b4b60 2024-04-25T00:38:55.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86cd779640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f86c80b4fa0 con 0x7f86c806d210 2024-04-25T00:38:55.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86cd779640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f86c806dc60 con 0x7f86c815df70 2024-04-25T00:38:55.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86c67fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f86c806d210 0x7f86c80b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86c77fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f86c80b4b60 0x7f86c8070ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36828/0 (socket says 172.21.15.29:36828) 2024-04-25T00:38:55.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86c77fe640 1 -- 172.21.15.29:0/870427786 learned_addr learned my addr 172.21.15.29:0/870427786 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:55.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86c5ffb640 1 -- 172.21.15.29:0/870427786 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1707536087 0 0) 0x7f86c806d980 con 0x7f86c80b4b60 2024-04-25T00:38:55.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86c5ffb640 1 -- 172.21.15.29:0/870427786 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86b4003530 con 0x7f86c80b4b60 2024-04-25T00:38:55.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86c6ffd640 1 --2- 172.21.15.29:0/870427786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c815df70 0x7f86c80b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86c67fc640 1 -- 172.21.15.29:0/870427786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c815df70 msgr2=0x7f86c80b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:55.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86c67fc640 1 --2- 172.21.15.29:0/870427786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c815df70 0x7f86c80b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86c67fc640 1 -- 172.21.15.29:0/870427786 >> v1:172.21.15.29:6789/0 conn(0x7f86c80b4b60 legacy=0x7f86c8070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86c67fc640 1 -- 172.21.15.29:0/870427786 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c806df40 con 0x7f86c806d210 2024-04-25T00:38:55.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.470+0000 7f86c6ffd640 1 --2- 172.21.15.29:0/870427786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c815df70 0x7f86c80b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:55.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86c67fc640 1 --2- 172.21.15.29:0/870427786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c806d210 0x7f86c80b4620 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009d90 tx=0x7f86b001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc505f2e0ff423e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:55.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86c5ffb640 1 -- 172.21.15.29:0/870427786 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b00b2070 con 0x7f86c806d210 2024-04-25T00:38:55.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86c5ffb640 1 -- 172.21.15.29:0/870427786 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86b00a5840 con 0x7f86c806d210 2024-04-25T00:38:55.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86c5ffb640 1 -- 172.21.15.29:0/870427786 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b00a5c10 con 0x7f86c806d210 2024-04-25T00:38:55.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 -- 172.21.15.29:0/870427786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c806d210 msgr2=0x7f86c80b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 --2- 172.21.15.29:0/870427786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c806d210 0x7f86c80b4620 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009d90 tx=0x7f86b001c970 comp rx=0 tx=0).stop 2024-04-25T00:38:55.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 -- 172.21.15.29:0/870427786 shutdown_connections 2024-04-25T00:38:55.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 --2- 172.21.15.29:0/870427786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c806d210 0x7f86c80b4620 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 --2- 172.21.15.29:0/870427786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c815df70 0x7f86c80b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 -- 172.21.15.29:0/870427786 >> 172.21.15.29:0/870427786 conn(0x7f86c80b1870 msgr2=0x7f86c80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:55.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 -- 172.21.15.29:0/870427786 shutdown_connections 2024-04-25T00:38:55.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 -- 172.21.15.29:0/870427786 wait complete. 2024-04-25T00:38:55.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 Processor -- start 2024-04-25T00:38:55.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 -- start start 2024-04-25T00:38:55.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c806d210 0x7f86c8155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c80b4b60 0x7f86c80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c815df70 0x7f86c80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f86c806ea20 con 0x7f86c815df70 2024-04-25T00:38:55.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86c815e350 con 0x7f86c80b4b60 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86cd779640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f86c806eba0 con 0x7f86c806d210 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86c77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c815df70 0x7f86c80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86c77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c815df70 0x7f86c80cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50704/0 (socket says 172.21.15.29:50704) 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.474+0000 7f86c77fe640 1 -- 172.21.15.29:0/379736944 learned_addr learned my addr 172.21.15.29:0/379736944 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86c77fe640 1 -- 172.21.15.29:0/379736944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c806d210 msgr2=0x7f86c8155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86c77fe640 1 --2- 172.21.15.29:0/379736944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c806d210 0x7f86c8155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86c67fc640 1 --2- 172.21.15.29:0/379736944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c80b4b60 0x7f86c80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86c77fe640 1 -- 172.21.15.29:0/379736944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c80b4b60 msgr2=0x7f86c80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86c77fe640 1 --2- 172.21.15.29:0/379736944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c80b4b60 0x7f86c80cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86c77fe640 1 -- 172.21.15.29:0/379736944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c8155280 con 0x7f86c815df70 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86c77fe640 1 --2- 172.21.15.29:0/379736944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c815df70 0x7f86c80cfb70 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f86c000c480 tx=0x7f86c000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86bf7fe640 1 -- 172.21.15.29:0/379736944 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0017070 con 0x7f86c815df70 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86bf7fe640 1 -- 172.21.15.29:0/379736944 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c0005cf0 con 0x7f86c815df70 2024-04-25T00:38:55.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86cd779640 1 -- 172.21.15.29:0/379736944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c81554f0 con 0x7f86c815df70 2024-04-25T00:38:55.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86bf7fe640 1 -- 172.21.15.29:0/379736944 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c00072b0 con 0x7f86c815df70 2024-04-25T00:38:55.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86cd779640 1 -- 172.21.15.29:0/379736944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c813ede0 con 0x7f86c815df70 2024-04-25T00:38:55.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86cd779640 1 -- 172.21.15.29:0/379736944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c8069270 con 0x7f86c815df70 2024-04-25T00:38:55.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86bf7fe640 1 -- 172.21.15.29:0/379736944 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f86c001a070 con 0x7f86c815df70 2024-04-25T00:38:55.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86bf7fe640 1 --2- 172.21.15.29:0/379736944 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86a8067e90 0x7f86a806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86bf7fe640 1 -- 172.21.15.29:0/379736944 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f86c00859b0 con 0x7f86c815df70 2024-04-25T00:38:55.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86c6ffd640 1 --2- 172.21.15.29:0/379736944 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86a8067e90 0x7f86a806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.478+0000 7f86c6ffd640 1 --2- 172.21.15.29:0/379736944 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86a8067e90 0x7f86a806a350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f86b800a890 tx=0x7f86b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:55.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.482+0000 7f86bf7fe640 1 -- 172.21.15.29:0/379736944 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f86c0055cd0 con 0x7f86c815df70 2024-04-25T00:38:55.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.682+0000 7f86cd779640 1 -- 172.21.15.29:0/379736944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86c80d08f0 con 0x7f86c815df70 2024-04-25T00:38:55.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.682+0000 7f86bf7fe640 1 -- 172.21.15.29:0/379736944 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5186 (secure 0 0 0) 0x7f86c00555d0 con 0x7f86c815df70 2024-04-25T00:38:55.685 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:55.685 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2148970077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2148970077},{"type":"v1","addr":"172.21.15.157:6834","nonce":2148970077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3947933400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3947933400},{"type":"v1","addr":"172.21.15.157:6835","nonce":3947933400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/172449010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":172449010},{"type":"v1","addr":"172.21.15.29:6833","nonce":172449010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:38:31.061384+0000","modified":"2024-04-25T00:38:34.097007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/3206404146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3206404146},{"type":"v1","addr":"172.21.15.29:6835","nonce":3206404146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8520,"qdb_cluster":[8520]},"id":9}]} 2024-04-25T00:38:55.686 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 76 2024-04-25T00:38:55.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.682+0000 7f86cd779640 1 -- 172.21.15.29:0/379736944 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86a8067e90 msgr2=0x7f86a806a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.682+0000 7f86cd779640 1 --2- 172.21.15.29:0/379736944 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86a8067e90 0x7f86a806a350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f86b800a890 tx=0x7f86b8008040 comp rx=0 tx=0).stop 2024-04-25T00:38:55.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.682+0000 7f86cd779640 1 -- 172.21.15.29:0/379736944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c815df70 msgr2=0x7f86c80cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.682+0000 7f86cd779640 1 --2- 172.21.15.29:0/379736944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c815df70 0x7f86c80cfb70 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f86c000c480 tx=0x7f86c000c940 comp rx=0 tx=0).stop 2024-04-25T00:38:55.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.682+0000 7f86cd779640 1 -- 172.21.15.29:0/379736944 shutdown_connections 2024-04-25T00:38:55.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.682+0000 7f86cd779640 1 --2- 172.21.15.29:0/379736944 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86a8067e90 0x7f86a806a350 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.682+0000 7f86cd779640 1 --2- 172.21.15.29:0/379736944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c815df70 0x7f86c80cfb70 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.682+0000 7f86cd779640 1 --2- 172.21.15.29:0/379736944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c80b4b60 0x7f86c80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.682+0000 7f86cd779640 1 --2- 172.21.15.29:0/379736944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c806d210 0x7f86c8155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.682+0000 7f86cd779640 1 -- 172.21.15.29:0/379736944 >> 172.21.15.29:0/379736944 conn(0x7f86c80b1870 msgr2=0x7f86c8070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:55.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.686+0000 7f86cd779640 1 -- 172.21.15.29:0/379736944 shutdown_connections 2024-04-25T00:38:55.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.686+0000 7f86cd779640 1 -- 172.21.15.29:0/379736944 wait complete. 2024-04-25T00:38:55.704 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:38:55.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.818+0000 7fc487c96640 1 Processor -- start 2024-04-25T00:38:55.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc487c96640 1 -- start start 2024-04-25T00:38:55.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc487c96640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b3c30 0x7fc48006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc487c96640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4800b4b10 0x7fc4800ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc487c96640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4800b41e0 con 0x7fc4800b3140 2024-04-25T00:38:55.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc487c96640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc48006d960 con 0x7fc4800b4b10 2024-04-25T00:38:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc487c96640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc48006dc40 con 0x7fc4800b3c30 2024-04-25T00:38:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc485231640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b3c30 0x7fc48006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc486233640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4800b4b10 0x7fc4800ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc485231640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b3c30 0x7fc48006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:42666/0 (socket says 172.21.15.29:42666) 2024-04-25T00:38:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc485231640 1 -- 172.21.15.29:0/1753120296 learned_addr learned my addr 172.21.15.29:0/1753120296 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc485231640 1 -- 172.21.15.29:0/1753120296 >> v2:172.21.15.157:3300/0 conn(0x7fc4800b4b10 msgr2=0x7fc4800ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc485231640 1 --2- 172.21.15.29:0/1753120296 >> v2:172.21.15.157:3300/0 conn(0x7fc4800b4b10 0x7fc4800ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc485231640 1 -- 172.21.15.29:0/1753120296 >> v1:172.21.15.157:6789/0 conn(0x7fc4800b3140 legacy=0x7fc4800b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.822+0000 7fc485231640 1 -- 172.21.15.29:0/1753120296 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48006df20 con 0x7fc4800b3c30 2024-04-25T00:38:55.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc485231640 1 --2- 172.21.15.29:0/1753120296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b3c30 0x7fc48006e440 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc470009870 tx=0x7fc47001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26c3fd41ff40a624 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:55.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc484a30640 1 -- 172.21.15.29:0/1753120296 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4700b2070 con 0x7fc4800b3c30 2024-04-25T00:38:55.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc484a30640 1 -- 172.21.15.29:0/1753120296 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4700a9070 con 0x7fc4800b3c30 2024-04-25T00:38:55.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc484a30640 1 -- 172.21.15.29:0/1753120296 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4700ad880 con 0x7fc4800b3c30 2024-04-25T00:38:55.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 -- 172.21.15.29:0/1753120296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b3c30 msgr2=0x7fc48006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 --2- 172.21.15.29:0/1753120296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b3c30 0x7fc48006e440 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc470009870 tx=0x7fc47001c560 comp rx=0 tx=0).stop 2024-04-25T00:38:55.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 -- 172.21.15.29:0/1753120296 shutdown_connections 2024-04-25T00:38:55.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 --2- 172.21.15.29:0/1753120296 >> v2:172.21.15.157:3300/0 conn(0x7fc4800b4b10 0x7fc4800ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 --2- 172.21.15.29:0/1753120296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b3c30 0x7fc48006e440 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 -- 172.21.15.29:0/1753120296 >> 172.21.15.29:0/1753120296 conn(0x7fc4800b1870 msgr2=0x7fc4800b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:55.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 -- 172.21.15.29:0/1753120296 shutdown_connections 2024-04-25T00:38:55.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 -- 172.21.15.29:0/1753120296 wait complete. 2024-04-25T00:38:55.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 Processor -- start 2024-04-25T00:38:55.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 -- start start 2024-04-25T00:38:55.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b3140 0x7fc48015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4800b3c30 0x7fc480141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4800b4b10 0x7fc480141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc480168d80 con 0x7fc4800b3c30 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc480168c00 con 0x7fc4800b4b10 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc487c96640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc480168f00 con 0x7fc4800b3140 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc485231640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4800b3c30 0x7fc480141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc485231640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4800b3c30 0x7fc480141a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50712/0 (socket says 172.21.15.29:50712) 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.826+0000 7fc485231640 1 -- 172.21.15.29:0/1228575590 learned_addr learned my addr 172.21.15.29:0/1228575590 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc486233640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4800b4b10 0x7fc480141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc485a32640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b3140 0x7fc48015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc485231640 1 -- 172.21.15.29:0/1228575590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b3140 msgr2=0x7fc48015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc485231640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b3140 0x7fc48015ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc485231640 1 -- 172.21.15.29:0/1228575590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4800b4b10 msgr2=0x7fc480141f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc485231640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4800b4b10 0x7fc480141f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc485231640 1 -- 172.21.15.29:0/1228575590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc480067820 con 0x7fc4800b3c30 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc485a32640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b3140 0x7fc48015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:55.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc486233640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4800b4b10 0x7fc480141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:55.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc485231640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4800b3c30 0x7fc480141a20 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fc470000c10 tx=0x7fc470019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:55.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc4767fc640 1 -- 172.21.15.29:0/1228575590 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4700bc070 con 0x7fc4800b3c30 2024-04-25T00:38:55.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc4767fc640 1 -- 172.21.15.29:0/1228575590 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4700add00 con 0x7fc4800b3c30 2024-04-25T00:38:55.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc487c96640 1 -- 172.21.15.29:0/1228575590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48015dc90 con 0x7fc4800b3c30 2024-04-25T00:38:55.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc4767fc640 1 -- 172.21.15.29:0/1228575590 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4700b2040 con 0x7fc4800b3c30 2024-04-25T00:38:55.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc487c96640 1 -- 172.21.15.29:0/1228575590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48015e250 con 0x7fc4800b3c30 2024-04-25T00:38:55.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc487c96640 1 -- 172.21.15.29:0/1228575590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc480069270 con 0x7fc4800b3c30 2024-04-25T00:38:55.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc4767fc640 1 -- 172.21.15.29:0/1228575590 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fc4700bf070 con 0x7fc4800b3c30 2024-04-25T00:38:55.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc4767fc640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc464067e60 0x7fc46406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:55.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc4767fc640 1 -- 172.21.15.29:0/1228575590 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fc47011f630 con 0x7fc4800b3c30 2024-04-25T00:38:55.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc485a32640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc464067e60 0x7fc46406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:55.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.830+0000 7fc485a32640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc464067e60 0x7fc46406a320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc47c001650 tx=0x7fc47c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:55.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:55.834+0000 7fc4767fc640 1 -- 172.21.15.29:0/1228575590 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc470016150 con 0x7fc4800b3c30 2024-04-25T00:38:56.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.034+0000 7fc487c96640 1 -- 172.21.15.29:0/1228575590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc48006ee30 con 0x7fc4800b3c30 2024-04-25T00:38:56.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.034+0000 7fc4767fc640 1 -- 172.21.15.29:0/1228575590 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5186 (secure 0 0 0) 0x7fc4700ef940 con 0x7fc4800b3c30 2024-04-25T00:38:56.037 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:56.037 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2148970077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2148970077},{"type":"v1","addr":"172.21.15.157:6834","nonce":2148970077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3947933400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3947933400},{"type":"v1","addr":"172.21.15.157:6835","nonce":3947933400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/172449010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":172449010},{"type":"v1","addr":"172.21.15.29:6833","nonce":172449010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:38:31.061384+0000","modified":"2024-04-25T00:38:34.097007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/3206404146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3206404146},{"type":"v1","addr":"172.21.15.29:6835","nonce":3206404146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8520,"qdb_cluster":[8520]},"id":9}]} 2024-04-25T00:38:56.038 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 76 2024-04-25T00:38:56.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.038+0000 7fc487c96640 1 -- 172.21.15.29:0/1228575590 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc464067e60 msgr2=0x7fc46406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.038+0000 7fc487c96640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc464067e60 0x7fc46406a320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc47c001650 tx=0x7fc47c009040 comp rx=0 tx=0).stop 2024-04-25T00:38:56.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.038+0000 7fc487c96640 1 -- 172.21.15.29:0/1228575590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4800b3c30 msgr2=0x7fc480141a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.038+0000 7fc487c96640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4800b3c30 0x7fc480141a20 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fc470000c10 tx=0x7fc470019300 comp rx=0 tx=0).stop 2024-04-25T00:38:56.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.038+0000 7fc487c96640 1 -- 172.21.15.29:0/1228575590 shutdown_connections 2024-04-25T00:38:56.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.038+0000 7fc487c96640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc464067e60 0x7fc46406a320 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.038+0000 7fc487c96640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4800b4b10 0x7fc480141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.038+0000 7fc487c96640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4800b3c30 0x7fc480141a20 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.038+0000 7fc487c96640 1 --2- 172.21.15.29:0/1228575590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4800b3140 0x7fc48015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.038+0000 7fc487c96640 1 -- 172.21.15.29:0/1228575590 >> 172.21.15.29:0/1228575590 conn(0x7fc4800b1870 msgr2=0x7fc480144b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:56.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.038+0000 7fc487c96640 1 -- 172.21.15.29:0/1228575590 shutdown_connections 2024-04-25T00:38:56.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.038+0000 7fc487c96640 1 -- 172.21.15.29:0/1228575590 wait complete. 2024-04-25T00:38:56.057 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:38:56.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab1cb1d640 1 Processor -- start 2024-04-25T00:38:56.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab1cb1d640 1 -- start start 2024-04-25T00:38:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab1cb1d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab1815c190 0x7fab180d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab1cb1d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab180cd3e0 con 0x7fab1815bdb0 2024-04-25T00:38:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab1cb1d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab180cd6c0 con 0x7fab180ccf30 2024-04-25T00:38:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab1cb1d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fab180cd9a0 con 0x7fab1815c190 2024-04-25T00:38:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab1659c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fab1815bdb0 0x7fab180cc820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36830/0 (socket says 172.21.15.29:36830) 2024-04-25T00:38:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab1659c640 1 -- 172.21.15.29:0/50862330 learned_addr learned my addr 172.21.15.29:0/50862330 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:56.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab15d9b640 1 --2- 172.21.15.29:0/50862330 >> v2:172.21.15.29:3300/0 conn(0x7fab1815c190 0x7fab180d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab1559a640 1 -- 172.21.15.29:0/50862330 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1203139755 0 0) 0x7fab180cd3e0 con 0x7fab1815bdb0 2024-04-25T00:38:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab1559a640 1 -- 172.21.15.29:0/50862330 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab00003530 con 0x7fab1815bdb0 2024-04-25T00:38:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab15d9b640 1 -- 172.21.15.29:0/50862330 >> v1:172.21.15.157:6789/0 conn(0x7fab180ccf30 legacy=0x7fab180c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab15d9b640 1 -- 172.21.15.29:0/50862330 >> v1:172.21.15.29:6789/0 conn(0x7fab1815bdb0 legacy=0x7fab180cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.174+0000 7fab15d9b640 1 -- 172.21.15.29:0/50862330 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab180cdc80 con 0x7fab1815c190 2024-04-25T00:38:56.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.178+0000 7fab15d9b640 1 --2- 172.21.15.29:0/50862330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab1815c190 0x7fab180d02d0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0097c0 tx=0x7fab0c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d4428fcd6bfa34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:56.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.178+0000 7fab1559a640 1 -- 172.21.15.29:0/50862330 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0b0070 con 0x7fab1815c190 2024-04-25T00:38:56.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.178+0000 7fab1559a640 1 -- 172.21.15.29:0/50862330 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab0c0ab600 con 0x7fab1815c190 2024-04-25T00:38:56.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.178+0000 7fab1559a640 1 -- 172.21.15.29:0/50862330 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0ab9b0 con 0x7fab1815c190 2024-04-25T00:38:56.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.178+0000 7fab1cb1d640 1 -- 172.21.15.29:0/50862330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab1815c190 msgr2=0x7fab180d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.178+0000 7fab1cb1d640 1 --2- 172.21.15.29:0/50862330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab1815c190 0x7fab180d02d0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0097c0 tx=0x7fab0c01c3e0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.178+0000 7fab1cb1d640 1 -- 172.21.15.29:0/50862330 shutdown_connections 2024-04-25T00:38:56.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.178+0000 7fab1cb1d640 1 --2- 172.21.15.29:0/50862330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab1815c190 0x7fab180d02d0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.178+0000 7fab1cb1d640 1 -- 172.21.15.29:0/50862330 >> 172.21.15.29:0/50862330 conn(0x7fab1809aa30 msgr2=0x7fab1809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:56.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.178+0000 7fab1cb1d640 1 -- 172.21.15.29:0/50862330 shutdown_connections 2024-04-25T00:38:56.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.178+0000 7fab1cb1d640 1 -- 172.21.15.29:0/50862330 wait complete. 2024-04-25T00:38:56.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.178+0000 7fab1cb1d640 1 Processor -- start 2024-04-25T00:38:56.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.178+0000 7fab1cb1d640 1 -- start start 2024-04-25T00:38:56.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab1cb1d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab180ccf30 0x7fab1815cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab1cb1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab1815bdb0 0x7fab18157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab1cb1d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab18158400 0x7fab18153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab1cb1d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fab180c6b10 con 0x7fab18158400 2024-04-25T00:38:56.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab1cb1d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab180c6990 con 0x7fab180ccf30 2024-04-25T00:38:56.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab1cb1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fab18169ae0 con 0x7fab1815bdb0 2024-04-25T00:38:56.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab16d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab18158400 0x7fab18153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab16d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab18158400 0x7fab18153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50728/0 (socket says 172.21.15.29:50728) 2024-04-25T00:38:56.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab16d9d640 1 -- 172.21.15.29:0/3728094891 learned_addr learned my addr 172.21.15.29:0/3728094891 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:56.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab1659c640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab180ccf30 0x7fab1815cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab15d9b640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab1815bdb0 0x7fab18157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab16d9d640 1 -- 172.21.15.29:0/3728094891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab1815bdb0 msgr2=0x7fab18157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab16d9d640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab1815bdb0 0x7fab18157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab16d9d640 1 -- 172.21.15.29:0/3728094891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab180ccf30 msgr2=0x7fab1815cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab16d9d640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab180ccf30 0x7fab1815cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab16d9d640 1 -- 172.21.15.29:0/3728094891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1815d7e0 con 0x7fab18158400 2024-04-25T00:38:56.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab15d9b640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab1815bdb0 0x7fab18157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:56.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab1659c640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab180ccf30 0x7fab1815cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:56.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab16d9d640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab18158400 0x7fab18153be0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c770 tx=0x7fab1000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:56.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab06ffd640 1 -- 172.21.15.29:0/3728094891 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10003b80 con 0x7fab18158400 2024-04-25T00:38:56.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab06ffd640 1 -- 172.21.15.29:0/3728094891 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab10003d20 con 0x7fab18158400 2024-04-25T00:38:56.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab06ffd640 1 -- 172.21.15.29:0/3728094891 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1000aa90 con 0x7fab18158400 2024-04-25T00:38:56.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab1cb1d640 1 -- 172.21.15.29:0/3728094891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1815dac0 con 0x7fab18158400 2024-04-25T00:38:56.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab1cb1d640 1 -- 172.21.15.29:0/3728094891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1815e080 con 0x7fab18158400 2024-04-25T00:38:56.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab1cb1d640 1 -- 172.21.15.29:0/3728094891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab18069270 con 0x7fab18158400 2024-04-25T00:38:56.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab06ffd640 1 -- 172.21.15.29:0/3728094891 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fab1000ac30 con 0x7fab18158400 2024-04-25T00:38:56.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab06ffd640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faaf8067e60 0x7faaf806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab1659c640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faaf8067e60 0x7faaf806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.182+0000 7fab06ffd640 1 -- 172.21.15.29:0/3728094891 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fab10086670 con 0x7fab18158400 2024-04-25T00:38:56.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.186+0000 7fab1659c640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faaf8067e60 0x7faaf806a320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fab080011f0 tx=0x7fab08009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:56.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.186+0000 7fab06ffd640 1 -- 172.21.15.29:0/3728094891 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fab10056950 con 0x7fab18158400 2024-04-25T00:38:56.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.350+0000 7fab1cb1d640 1 -- 172.21.15.29:0/3728094891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab180d2790 con 0x7fab18158400 2024-04-25T00:38:56.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.350+0000 7fab06ffd640 1 -- 172.21.15.29:0/3728094891 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16742 (secure 0 0 0) 0x7fab10056250 con 0x7fab18158400 2024-04-25T00:38:56.355 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:38:56.355 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":119,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:51.470777+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:38:28.743593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:38:30.051271+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:38:56.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.354+0000 7fab1cb1d640 1 -- 172.21.15.29:0/3728094891 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faaf8067e60 msgr2=0x7faaf806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.354+0000 7fab1cb1d640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faaf8067e60 0x7faaf806a320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fab080011f0 tx=0x7fab08009040 comp rx=0 tx=0).stop 2024-04-25T00:38:56.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.354+0000 7fab1cb1d640 1 -- 172.21.15.29:0/3728094891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab18158400 msgr2=0x7fab18153be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.354+0000 7fab1cb1d640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab18158400 0x7fab18153be0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c770 tx=0x7fab1000cc30 comp rx=0 tx=0).stop 2024-04-25T00:38:56.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.354+0000 7fab1cb1d640 1 -- 172.21.15.29:0/3728094891 shutdown_connections 2024-04-25T00:38:56.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.354+0000 7fab1cb1d640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faaf8067e60 0x7faaf806a320 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.354+0000 7fab1cb1d640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab18158400 0x7fab18153be0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.354+0000 7fab1cb1d640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab1815bdb0 0x7fab18157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.354+0000 7fab1cb1d640 1 --2- 172.21.15.29:0/3728094891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab180ccf30 0x7fab1815cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.354+0000 7fab1cb1d640 1 -- 172.21.15.29:0/3728094891 >> 172.21.15.29:0/3728094891 conn(0x7fab1809aa30 msgr2=0x7fab180cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:56.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.354+0000 7fab1cb1d640 1 -- 172.21.15.29:0/3728094891 shutdown_connections 2024-04-25T00:38:56.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.354+0000 7fab1cb1d640 1 -- 172.21.15.29:0/3728094891 wait complete. 2024-04-25T00:38:56.374 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:38:56.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a95f4640 1 Processor -- start 2024-04-25T00:38:56.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a95f4640 1 -- start start 2024-04-25T00:38:56.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a95f4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f02a415be30 0x7f02a40ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a95f4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f02a40d0860 0x7f02a415ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a95f4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a40722e0 con 0x7f02a40ceeb0 2024-04-25T00:38:56.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a95f4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f02a415fa90 con 0x7f02a40d0860 2024-04-25T00:38:56.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a95f4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f02a40ccd80 con 0x7f02a415be30 2024-04-25T00:38:56.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a37fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f02a40d0860 0x7f02a415ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a37fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f02a40d0860 0x7f02a415ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50730/0 (socket says 172.21.15.29:50730) 2024-04-25T00:38:56.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a37fe640 1 -- 172.21.15.29:0/3266653710 learned_addr learned my addr 172.21.15.29:0/3266653710 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:56.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a2ffd640 1 --2- 172.21.15.29:0/3266653710 >> v2:172.21.15.157:3300/0 conn(0x7f02a415be30 0x7f02a40ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a37fe640 1 -- 172.21.15.29:0/3266653710 >> v2:172.21.15.157:3300/0 conn(0x7f02a415be30 msgr2=0x7f02a40ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:56.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a37fe640 1 --2- 172.21.15.29:0/3266653710 >> v2:172.21.15.157:3300/0 conn(0x7f02a415be30 0x7f02a40ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a37fe640 1 -- 172.21.15.29:0/3266653710 >> v1:172.21.15.157:6789/0 conn(0x7f02a40ceeb0 legacy=0x7f02a40d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:56.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.498+0000 7f02a37fe640 1 -- 172.21.15.29:0/3266653710 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a40cd060 con 0x7f02a40d0860 2024-04-25T00:38:56.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.502+0000 7f02a37fe640 1 --2- 172.21.15.29:0/3266653710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a40d0860 0x7f02a415ff20 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f02a4074400 tx=0x7f029401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c559031b9b0d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:56.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.502+0000 7f02a1ffb640 1 -- 172.21.15.29:0/3266653710 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02940a7070 con 0x7f02a40d0860 2024-04-25T00:38:56.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.502+0000 7f02a1ffb640 1 -- 172.21.15.29:0/3266653710 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02940b0070 con 0x7f02a40d0860 2024-04-25T00:38:56.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.502+0000 7f02a1ffb640 1 -- 172.21.15.29:0/3266653710 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02940ab890 con 0x7f02a40d0860 2024-04-25T00:38:56.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.502+0000 7f02a95f4640 1 -- 172.21.15.29:0/3266653710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a40d0860 msgr2=0x7f02a415ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.502+0000 7f02a95f4640 1 --2- 172.21.15.29:0/3266653710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a40d0860 0x7f02a415ff20 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f02a4074400 tx=0x7f029401c4b0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.502+0000 7f02a95f4640 1 -- 172.21.15.29:0/3266653710 shutdown_connections 2024-04-25T00:38:56.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.502+0000 7f02a95f4640 1 --2- 172.21.15.29:0/3266653710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a40d0860 0x7f02a415ff20 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.502+0000 7f02a95f4640 1 --2- 172.21.15.29:0/3266653710 >> v2:172.21.15.157:3300/0 conn(0x7f02a415be30 0x7f02a40ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.502+0000 7f02a95f4640 1 -- 172.21.15.29:0/3266653710 >> 172.21.15.29:0/3266653710 conn(0x7f02a4074c00 msgr2=0x7f02a406b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:56.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.502+0000 7f02a95f4640 1 -- 172.21.15.29:0/3266653710 shutdown_connections 2024-04-25T00:38:56.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.502+0000 7f02a95f4640 1 -- 172.21.15.29:0/3266653710 wait complete. 2024-04-25T00:38:56.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.502+0000 7f02a95f4640 1 Processor -- start 2024-04-25T00:38:56.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a95f4640 1 -- start start 2024-04-25T00:38:56.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a95f4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a40ceeb0 0x7f02a415d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a95f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a40d0860 0x7f02a4154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a95f4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02a415be30 0x7f02a4154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a95f4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41605e0 con 0x7f02a40ceeb0 2024-04-25T00:38:56.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a95f4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f02a4160460 con 0x7f02a40d0860 2024-04-25T00:38:56.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a95f4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f02a4160760 con 0x7f02a415be30 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a40ceeb0 0x7f02a415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a40ceeb0 0x7f02a415d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50732/0 (socket says 172.21.15.29:50732) 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a2ffd640 1 -- 172.21.15.29:0/3162010184 learned_addr learned my addr 172.21.15.29:0/3162010184 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a37fe640 1 --2- 172.21.15.29:0/3162010184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02a415be30 0x7f02a4154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a2ffd640 1 -- 172.21.15.29:0/3162010184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02a415be30 msgr2=0x7f02a4154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a2ffd640 1 --2- 172.21.15.29:0/3162010184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02a415be30 0x7f02a4154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a2ffd640 1 -- 172.21.15.29:0/3162010184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a40d0860 msgr2=0x7f02a4154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a2ffd640 1 --2- 172.21.15.29:0/3162010184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a40d0860 0x7f02a4154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a2ffd640 1 -- 172.21.15.29:0/3162010184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a40ab030 con 0x7f02a40ceeb0 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a2ffd640 1 --2- 172.21.15.29:0/3162010184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a40ceeb0 0x7f02a415d170 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f029c00ce40 tx=0x7f029c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f029b7fe640 1 -- 172.21.15.29:0/3162010184 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c003ba0 con 0x7f02a40ceeb0 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f029b7fe640 1 -- 172.21.15.29:0/3162010184 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029c003d40 con 0x7f02a40ceeb0 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a95f4640 1 -- 172.21.15.29:0/3162010184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a40ab310 con 0x7f02a40ceeb0 2024-04-25T00:38:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f029b7fe640 1 -- 172.21.15.29:0/3162010184 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c004240 con 0x7f02a40ceeb0 2024-04-25T00:38:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a95f4640 1 -- 172.21.15.29:0/3162010184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a4156790 con 0x7f02a40ceeb0 2024-04-25T00:38:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a95f4640 1 -- 172.21.15.29:0/3162010184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a4069270 con 0x7f02a40ceeb0 2024-04-25T00:38:56.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f029b7fe640 1 -- 172.21.15.29:0/3162010184 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f029c0043e0 con 0x7f02a40ceeb0 2024-04-25T00:38:56.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f029b7fe640 1 --2- 172.21.15.29:0/3162010184 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0284067e60 0x7f028406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f029b7fe640 1 -- 172.21.15.29:0/3162010184 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f029c0873e0 con 0x7f02a40ceeb0 2024-04-25T00:38:56.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.506+0000 7f02a27fc640 1 --2- 172.21.15.29:0/3162010184 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0284067e60 0x7f028406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.510+0000 7f02a27fc640 1 --2- 172.21.15.29:0/3162010184 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0284067e60 0x7f028406a320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f02a40b13c0 tx=0x7f028c006bd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:56.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.510+0000 7f029b7fe640 1 -- 172.21.15.29:0/3162010184 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f029c057560 con 0x7f02a40ceeb0 2024-04-25T00:38:56.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.706+0000 7f02a95f4640 1 -- 172.21.15.29:0/3162010184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f02a40746d0 con 0x7f02a40ceeb0 2024-04-25T00:38:56.711 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-04-25T00:38:56.706+0000 7f3ac00b5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:38:56.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.710+0000 7f029b7fe640 1 -- 172.21.15.29:0/3162010184 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) ==== 161+0+0 (secure 0 0 0) 0x7f029c056e60 con 0x7f02a40ceeb0 2024-04-25T00:38:56.714 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:38:56.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.714+0000 7f02a95f4640 1 -- 172.21.15.29:0/3162010184 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0284067e60 msgr2=0x7f028406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.714+0000 7f02a95f4640 1 --2- 172.21.15.29:0/3162010184 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0284067e60 0x7f028406a320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f02a40b13c0 tx=0x7f028c006bd0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.714+0000 7f02a95f4640 1 -- 172.21.15.29:0/3162010184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a40ceeb0 msgr2=0x7f02a415d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.714+0000 7f02a95f4640 1 --2- 172.21.15.29:0/3162010184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a40ceeb0 0x7f02a415d170 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f029c00ce40 tx=0x7f029c00a660 comp rx=0 tx=0).stop 2024-04-25T00:38:56.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.714+0000 7f02a95f4640 1 -- 172.21.15.29:0/3162010184 shutdown_connections 2024-04-25T00:38:56.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.714+0000 7f02a95f4640 1 --2- 172.21.15.29:0/3162010184 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0284067e60 0x7f028406a320 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.714+0000 7f02a95f4640 1 --2- 172.21.15.29:0/3162010184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02a415be30 0x7f02a4154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.714+0000 7f02a95f4640 1 --2- 172.21.15.29:0/3162010184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a40d0860 0x7f02a4154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.714+0000 7f02a95f4640 1 --2- 172.21.15.29:0/3162010184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a40ceeb0 0x7f02a415d170 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.714+0000 7f02a95f4640 1 -- 172.21.15.29:0/3162010184 >> 172.21.15.29:0/3162010184 conn(0x7f02a4074c00 msgr2=0x7f02a4066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:56.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.714+0000 7f02a95f4640 1 -- 172.21.15.29:0/3162010184 shutdown_connections 2024-04-25T00:38:56.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.714+0000 7f02a95f4640 1 -- 172.21.15.29:0/3162010184 wait complete. 2024-04-25T00:38:56.735 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:38:56.756 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:38:56.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.850+0000 7f6f628a9640 1 Processor -- start 2024-04-25T00:38:56.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.850+0000 7f6f628a9640 1 -- start start 2024-04-25T00:38:56.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f628a9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6f5c15a570 0x7f6f5c0715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f628a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f5c071af0 0x7f6f5c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f628a9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f5c15a950 con 0x7f6f5c0b53a0 2024-04-25T00:38:56.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f628a9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6f5c071f50 con 0x7f6f5c15a570 2024-04-25T00:38:56.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f628a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f5c0b3ee0 con 0x7f6f5c071af0 2024-04-25T00:38:56.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f5bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6f5c15a570 0x7f6f5c0715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f60e46640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6f5c0b53a0 0x7f6f5c06d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36844/0 (socket says 172.21.15.29:36844) 2024-04-25T00:38:56.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f60e46640 1 -- 172.21.15.29:0/3011601801 learned_addr learned my addr 172.21.15.29:0/3011601801 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:56.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f5b7fe640 1 --2- 172.21.15.29:0/3011601801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f5c071af0 0x7f6f5c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f5affd640 1 -- 172.21.15.29:0/3011601801 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 33309227 0 0) 0x7f6f5c15a950 con 0x7f6f5c0b53a0 2024-04-25T00:38:56.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f5affd640 1 -- 172.21.15.29:0/3011601801 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f44005740 con 0x7f6f5c0b53a0 2024-04-25T00:38:56.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f5bfff640 1 -- 172.21.15.29:0/3011601801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f5c071af0 msgr2=0x7f6f5c0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:56.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f5bfff640 1 --2- 172.21.15.29:0/3011601801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f5c071af0 0x7f6f5c0b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f5bfff640 1 -- 172.21.15.29:0/3011601801 >> v1:172.21.15.29:6789/0 conn(0x7f6f5c0b53a0 legacy=0x7f6f5c06d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f5bfff640 1 -- 172.21.15.29:0/3011601801 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5c0b4130 con 0x7f6f5c15a570 2024-04-25T00:38:56.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f5b7fe640 1 --2- 172.21.15.29:0/3011601801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f5c071af0 0x7f6f5c0b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:56.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f5bfff640 1 --2- 172.21.15.29:0/3011601801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f5c15a570 0x7f6f5c0715b0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0097c0 tx=0x7f6f4c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb91217e1c3ded4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:56.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f5affd640 1 -- 172.21.15.29:0/3011601801 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c0b1070 con 0x7f6f5c15a570 2024-04-25T00:38:56.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f5affd640 1 -- 172.21.15.29:0/3011601801 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f4c0ac640 con 0x7f6f5c15a570 2024-04-25T00:38:56.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f5affd640 1 -- 172.21.15.29:0/3011601801 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c0aca10 con 0x7f6f5c15a570 2024-04-25T00:38:56.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f628a9640 1 -- 172.21.15.29:0/3011601801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f5c15a570 msgr2=0x7f6f5c0715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f628a9640 1 --2- 172.21.15.29:0/3011601801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f5c15a570 0x7f6f5c0715b0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0097c0 tx=0x7f6f4c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f628a9640 1 -- 172.21.15.29:0/3011601801 shutdown_connections 2024-04-25T00:38:56.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f628a9640 1 --2- 172.21.15.29:0/3011601801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f5c071af0 0x7f6f5c0b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f628a9640 1 --2- 172.21.15.29:0/3011601801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f5c15a570 0x7f6f5c0715b0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.854+0000 7f6f628a9640 1 -- 172.21.15.29:0/3011601801 >> 172.21.15.29:0/3011601801 conn(0x7f6f5c075030 msgr2=0x7f6f5c0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:56.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f628a9640 1 -- 172.21.15.29:0/3011601801 shutdown_connections 2024-04-25T00:38:56.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f628a9640 1 -- 172.21.15.29:0/3011601801 wait complete. 2024-04-25T00:38:56.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f628a9640 1 Processor -- start 2024-04-25T00:38:56.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f628a9640 1 -- start start 2024-04-25T00:38:56.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f628a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f5c071af0 0x7f6f5c1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f628a9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f5c0b53a0 0x7f6f5c153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f628a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f5c15a570 0x7f6f5c14bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f628a9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c06dbc0 con 0x7f6f5c0b53a0 2024-04-25T00:38:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f628a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c06da40 con 0x7f6f5c071af0 2024-04-25T00:38:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f628a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f5c06dd40 con 0x7f6f5c15a570 2024-04-25T00:38:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f5b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f5c0b53a0 0x7f6f5c153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f5b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f5c0b53a0 0x7f6f5c153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50764/0 (socket says 172.21.15.29:50764) 2024-04-25T00:38:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f5b7fe640 1 -- 172.21.15.29:0/312085321 learned_addr learned my addr 172.21.15.29:0/312085321 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f5b7fe640 1 -- 172.21.15.29:0/312085321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f5c15a570 msgr2=0x7f6f5c14bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f5b7fe640 1 --2- 172.21.15.29:0/312085321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f5c15a570 0x7f6f5c14bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f5b7fe640 1 -- 172.21.15.29:0/312085321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f5c071af0 msgr2=0x7f6f5c1553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f5b7fe640 1 --2- 172.21.15.29:0/312085321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f5c071af0 0x7f6f5c1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f5b7fe640 1 -- 172.21.15.29:0/312085321 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5c159850 con 0x7f6f5c0b53a0 2024-04-25T00:38:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f5b7fe640 1 --2- 172.21.15.29:0/312085321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f5c0b53a0 0x7f6f5c153050 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000c7c0 tx=0x7f6f5000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f58ff9640 1 -- 172.21.15.29:0/312085321 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f50007a30 con 0x7f6f5c0b53a0 2024-04-25T00:38:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f58ff9640 1 -- 172.21.15.29:0/312085321 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f50007bd0 con 0x7f6f5c0b53a0 2024-04-25T00:38:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f628a9640 1 -- 172.21.15.29:0/312085321 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f5c14ce10 con 0x7f6f5c0b53a0 2024-04-25T00:38:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f58ff9640 1 -- 172.21.15.29:0/312085321 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f500113f0 con 0x7f6f5c0b53a0 2024-04-25T00:38:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f628a9640 1 -- 172.21.15.29:0/312085321 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f5c14d3d0 con 0x7f6f5c0b53a0 2024-04-25T00:38:56.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.858+0000 7f6f628a9640 1 -- 172.21.15.29:0/312085321 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f5c069270 con 0x7f6f5c0b53a0 2024-04-25T00:38:56.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.862+0000 7f6f58ff9640 1 -- 172.21.15.29:0/312085321 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f6f50011590 con 0x7f6f5c0b53a0 2024-04-25T00:38:56.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.862+0000 7f6f58ff9640 1 --2- 172.21.15.29:0/312085321 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6f48067e90 0x7f6f4806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.862+0000 7f6f58ff9640 1 -- 172.21.15.29:0/312085321 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5823+0+0 (secure 0 0 0) 0x7f6f50086980 con 0x7f6f5c0b53a0 2024-04-25T00:38:56.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.862+0000 7f6f5bfff640 1 --2- 172.21.15.29:0/312085321 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6f48067e90 0x7f6f4806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.862+0000 7f6f5bfff640 1 --2- 172.21.15.29:0/312085321 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6f48067e90 0x7f6f4806a350 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0ae000 tx=0x7f6f4c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:56.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:56.866+0000 7f6f58ff9640 1 -- 172.21.15.29:0/312085321 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6f50015020 con 0x7f6f5c0b53a0 2024-04-25T00:38:57.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.062+0000 7f6f628a9640 1 -- 172.21.15.29:0/312085321 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6f5c1545d0 con 0x7f6f5c0b53a0 2024-04-25T00:38:57.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.714+0000 7f6f58ff9640 1 -- 172.21.15.29:0/312085321 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) ==== 104+0+0 (secure 0 0 0) 0x7f6f50056b60 con 0x7f6f5c0b53a0 2024-04-25T00:38:57.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.714+0000 7f6f628a9640 1 -- 172.21.15.29:0/312085321 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6f48067e90 msgr2=0x7f6f4806a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:57.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.714+0000 7f6f628a9640 1 --2- 172.21.15.29:0/312085321 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6f48067e90 0x7f6f4806a350 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0ae000 tx=0x7f6f4c008040 comp rx=0 tx=0).stop 2024-04-25T00:38:57.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.714+0000 7f6f628a9640 1 -- 172.21.15.29:0/312085321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f5c0b53a0 msgr2=0x7f6f5c153050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:57.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.714+0000 7f6f628a9640 1 --2- 172.21.15.29:0/312085321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f5c0b53a0 0x7f6f5c153050 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000c7c0 tx=0x7f6f5000cc80 comp rx=0 tx=0).stop 2024-04-25T00:38:57.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.714+0000 7f6f628a9640 1 -- 172.21.15.29:0/312085321 shutdown_connections 2024-04-25T00:38:57.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.714+0000 7f6f628a9640 1 --2- 172.21.15.29:0/312085321 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6f48067e90 0x7f6f4806a350 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.714+0000 7f6f628a9640 1 --2- 172.21.15.29:0/312085321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f5c15a570 0x7f6f5c14bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.714+0000 7f6f628a9640 1 --2- 172.21.15.29:0/312085321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f5c0b53a0 0x7f6f5c153050 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.714+0000 7f6f628a9640 1 --2- 172.21.15.29:0/312085321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f5c071af0 0x7f6f5c1553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.714+0000 7f6f628a9640 1 -- 172.21.15.29:0/312085321 >> 172.21.15.29:0/312085321 conn(0x7f6f5c075030 msgr2=0x7f6f5c0b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:57.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.718+0000 7f6f628a9640 1 -- 172.21.15.29:0/312085321 shutdown_connections 2024-04-25T00:38:57.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.718+0000 7f6f628a9640 1 -- 172.21.15.29:0/312085321 wait complete. 2024-04-25T00:38:57.740 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:38:57.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf7c45640 1 Processor -- start 2024-04-25T00:38:57.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf7c45640 1 -- start start 2024-04-25T00:38:57.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf7c45640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fccf01456d0 0x7fccf00b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf7c45640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf00cf670 con 0x7fccf006cd40 2024-04-25T00:38:57.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf7c45640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf00b4380 con 0x7fccf00b3b50 2024-04-25T00:38:57.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf7c45640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fccf00b40a0 con 0x7fccf01456d0 2024-04-25T00:38:57.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf61e2640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fccf006cd40 0x7fccf006d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59394/0 (socket says 172.21.15.29:59394) 2024-04-25T00:38:57.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf61e2640 1 -- 172.21.15.29:0/2692388365 learned_addr learned my addr 172.21.15.29:0/2692388365 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:57.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf59e1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fccf01456d0 0x7fccf00b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:57.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf49df640 1 -- 172.21.15.29:0/2692388365 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3744156479 0 0) 0x7fccf00cf670 con 0x7fccf006cd40 2024-04-25T00:38:57.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf49df640 1 -- 172.21.15.29:0/2692388365 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccdc003530 con 0x7fccf006cd40 2024-04-25T00:38:57.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf59e1640 1 -- 172.21.15.29:0/2692388365 >> v1:172.21.15.157:6789/0 conn(0x7fccf00b3b50 legacy=0x7fccf00cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:57.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf59e1640 1 -- 172.21.15.29:0/2692388365 >> v1:172.21.15.29:6789/0 conn(0x7fccf006cd40 legacy=0x7fccf006d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:57.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.906+0000 7fccf59e1640 1 -- 172.21.15.29:0/2692388365 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf00ce480 con 0x7fccf01456d0 2024-04-25T00:38:57.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.910+0000 7fccf59e1640 1 --2- 172.21.15.29:0/2692388365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccf01456d0 0x7fccf00b2b70 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fccec0097c0 tx=0x7fccec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76d1a6b5a116eea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:57.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.910+0000 7fccf49df640 1 -- 172.21.15.29:0/2692388365 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec0b1070 con 0x7fccf01456d0 2024-04-25T00:38:57.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.910+0000 7fccf49df640 1 -- 172.21.15.29:0/2692388365 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccec0ac5f0 con 0x7fccf01456d0 2024-04-25T00:38:57.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.910+0000 7fccf49df640 1 -- 172.21.15.29:0/2692388365 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec0ac9a0 con 0x7fccf01456d0 2024-04-25T00:38:57.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.910+0000 7fccf7c45640 1 -- 172.21.15.29:0/2692388365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccf01456d0 msgr2=0x7fccf00b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:57.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.910+0000 7fccf7c45640 1 --2- 172.21.15.29:0/2692388365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccf01456d0 0x7fccf00b2b70 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fccec0097c0 tx=0x7fccec01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.910+0000 7fccf7c45640 1 -- 172.21.15.29:0/2692388365 shutdown_connections 2024-04-25T00:38:57.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.910+0000 7fccf7c45640 1 --2- 172.21.15.29:0/2692388365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccf01456d0 0x7fccf00b2b70 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.910+0000 7fccf7c45640 1 -- 172.21.15.29:0/2692388365 >> 172.21.15.29:0/2692388365 conn(0x7fccf00b1870 msgr2=0x7fccf00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:57.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.910+0000 7fccf7c45640 1 -- 172.21.15.29:0/2692388365 shutdown_connections 2024-04-25T00:38:57.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.910+0000 7fccf7c45640 1 -- 172.21.15.29:0/2692388365 wait complete. 2024-04-25T00:38:57.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.910+0000 7fccf7c45640 1 Processor -- start 2024-04-25T00:38:57.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.910+0000 7fccf7c45640 1 -- start start 2024-04-25T00:38:57.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf7c45640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccf006cd40 0x7fccf0144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf7c45640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccf00b3b50 0x7fccf0154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf7c45640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf01456d0 0x7fccf0153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf7c45640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fccf0169b80 con 0x7fccf00b3b50 2024-04-25T00:38:57.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf7c45640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccf00b5000 con 0x7fccf01456d0 2024-04-25T00:38:57.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf7c45640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fccf0169d00 con 0x7fccf006cd40 2024-04-25T00:38:57.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf51e0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccf00b3b50 0x7fccf0154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf51e0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccf00b3b50 0x7fccf0154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50796/0 (socket says 172.21.15.29:50796) 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf51e0640 1 -- 172.21.15.29:0/1499641913 learned_addr learned my addr 172.21.15.29:0/1499641913 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf61e2640 1 --2- 172.21.15.29:0/1499641913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf01456d0 0x7fccf0153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf51e0640 1 -- 172.21.15.29:0/1499641913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccf006cd40 msgr2=0x7fccf0144570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf51e0640 1 --2- 172.21.15.29:0/1499641913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccf006cd40 0x7fccf0144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf51e0640 1 -- 172.21.15.29:0/1499641913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf01456d0 msgr2=0x7fccf0153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf51e0640 1 --2- 172.21.15.29:0/1499641913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf01456d0 0x7fccf0153c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf51e0640 1 -- 172.21.15.29:0/1499641913 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf01556a0 con 0x7fccf00b3b50 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf51e0640 1 --2- 172.21.15.29:0/1499641913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccf00b3b50 0x7fccf0154b90 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c450 tx=0x7fcce000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fcce67fc640 1 -- 172.21.15.29:0/1499641913 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0016020 con 0x7fccf00b3b50 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fcce67fc640 1 -- 172.21.15.29:0/1499641913 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce00052d0 con 0x7fccf00b3b50 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf7c45640 1 -- 172.21.15.29:0/1499641913 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf0143a90 con 0x7fccf00b3b50 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fcce67fc640 1 -- 172.21.15.29:0/1499641913 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0007260 con 0x7fccf00b3b50 2024-04-25T00:38:57.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf7c45640 1 -- 172.21.15.29:0/1499641913 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf0144ab0 con 0x7fccf00b3b50 2024-04-25T00:38:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf7c45640 1 -- 172.21.15.29:0/1499641913 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf0143880 con 0x7fccf00b3b50 2024-04-25T00:38:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fcce67fc640 1 -- 172.21.15.29:0/1499641913 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fcce0007400 con 0x7fccf00b3b50 2024-04-25T00:38:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fcce67fc640 1 --2- 172.21.15.29:0/1499641913 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fccd4067e60 0x7fccd406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fcce67fc640 1 -- 172.21.15.29:0/1499641913 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5823+0+0 (secure 0 0 0) 0x7fcce00860a0 con 0x7fccf00b3b50 2024-04-25T00:38:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf59e1640 1 --2- 172.21.15.29:0/1499641913 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fccd4067e60 0x7fccd406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.914+0000 7fccf59e1640 1 --2- 172.21.15.29:0/1499641913 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fccd4067e60 0x7fccd406a320 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fccec0ae000 tx=0x7fccec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:57.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:57.918+0000 7fcce67fc640 1 -- 172.21.15.29:0/1499641913 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcce0020020 con 0x7fccf00b3b50 2024-04-25T00:38:58.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.078+0000 7fccf7c45640 1 -- 172.21.15.29:0/1499641913 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fccf00692e0 con 0x7fccf00b3b50 2024-04-25T00:38:58.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.722+0000 7fcce67fc640 1 -- 172.21.15.29:0/1499641913 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v122) ==== 181+0+0 (secure 0 0 0) 0x7fcce0005480 con 0x7fccf00b3b50 2024-04-25T00:38:58.725 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:38:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.726+0000 7fccf7c45640 1 -- 172.21.15.29:0/1499641913 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fccd4067e60 msgr2=0x7fccd406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.726+0000 7fccf7c45640 1 --2- 172.21.15.29:0/1499641913 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fccd4067e60 0x7fccd406a320 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fccec0ae000 tx=0x7fccec008040 comp rx=0 tx=0).stop 2024-04-25T00:38:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.726+0000 7fccf7c45640 1 -- 172.21.15.29:0/1499641913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccf00b3b50 msgr2=0x7fccf0154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.726+0000 7fccf7c45640 1 --2- 172.21.15.29:0/1499641913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccf00b3b50 0x7fccf0154b90 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c450 tx=0x7fcce000c910 comp rx=0 tx=0).stop 2024-04-25T00:38:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.726+0000 7fccf7c45640 1 -- 172.21.15.29:0/1499641913 shutdown_connections 2024-04-25T00:38:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.726+0000 7fccf7c45640 1 --2- 172.21.15.29:0/1499641913 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fccd4067e60 0x7fccd406a320 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.726+0000 7fccf7c45640 1 --2- 172.21.15.29:0/1499641913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf01456d0 0x7fccf0153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.726+0000 7fccf7c45640 1 --2- 172.21.15.29:0/1499641913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccf00b3b50 0x7fccf0154b90 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:58.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.726+0000 7fccf7c45640 1 --2- 172.21.15.29:0/1499641913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccf006cd40 0x7fccf0144570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.726+0000 7fccf7c45640 1 -- 172.21.15.29:0/1499641913 >> 172.21.15.29:0/1499641913 conn(0x7fccf00b1870 msgr2=0x7fccf00b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.726+0000 7fccf7c45640 1 -- 172.21.15.29:0/1499641913 shutdown_connections 2024-04-25T00:38:58.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.726+0000 7fccf7c45640 1 -- 172.21.15.29:0/1499641913 wait complete. 2024-04-25T00:38:58.751 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:38:58.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.890+0000 7fa4a8837640 1 Processor -- start 2024-04-25T00:38:58.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.890+0000 7fa4a8837640 1 -- start start 2024-04-25T00:38:58.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.890+0000 7fa4a8837640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4a415e1a0 0x7fa4a4071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:58.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.890+0000 7fa4a8837640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4a40b3ac0 con 0x7fa4a40b4d00 2024-04-25T00:38:58.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.890+0000 7fa4a8837640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4a40b3d60 con 0x7fa4a4070ce0 2024-04-25T00:38:58.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.890+0000 7fa4a8837640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa4a40714f0 con 0x7fa4a415e1a0 2024-04-25T00:38:58.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.890+0000 7fa4a2d9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa4a40b4d00 0x7fa4a406d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59400/0 (socket says 172.21.15.29:59400) 2024-04-25T00:38:58.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.890+0000 7fa4a2d9d640 1 -- 172.21.15.29:0/3803106193 learned_addr learned my addr 172.21.15.29:0/3803106193 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:58.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.890+0000 7fa4a259c640 1 --2- 172.21.15.29:0/3803106193 >> v2:172.21.15.157:3300/0 conn(0x7fa4a415e1a0 0x7fa4a4071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:58.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.890+0000 7fa4a159a640 1 -- 172.21.15.29:0/3803106193 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2023193324 0 0) 0x7fa4a40b3ac0 con 0x7fa4a40b4d00 2024-04-25T00:38:58.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.890+0000 7fa4a159a640 1 -- 172.21.15.29:0/3803106193 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa48c003530 con 0x7fa4a40b4d00 2024-04-25T00:38:58.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a259c640 1 -- 172.21.15.29:0/3803106193 >> v1:172.21.15.157:6789/0 conn(0x7fa4a4070ce0 legacy=0x7fa4a40b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:58.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a259c640 1 -- 172.21.15.29:0/3803106193 >> v1:172.21.15.29:6789/0 conn(0x7fa4a40b4d00 legacy=0x7fa4a406d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:58.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a259c640 1 -- 172.21.15.29:0/3803106193 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a40b4040 con 0x7fa4a415e1a0 2024-04-25T00:38:58.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a259c640 1 --2- 172.21.15.29:0/3803106193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a415e1a0 0x7fa4a4071b40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa4940097c0 tx=0x7fa49401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfe59bf5b7316d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:58.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a159a640 1 -- 172.21.15.29:0/3803106193 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4940b0070 con 0x7fa4a415e1a0 2024-04-25T00:38:58.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a159a640 1 -- 172.21.15.29:0/3803106193 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4940ab600 con 0x7fa4a415e1a0 2024-04-25T00:38:58.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a159a640 1 -- 172.21.15.29:0/3803106193 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4940ab9b0 con 0x7fa4a415e1a0 2024-04-25T00:38:58.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a8837640 1 -- 172.21.15.29:0/3803106193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a415e1a0 msgr2=0x7fa4a4071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:58.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a8837640 1 --2- 172.21.15.29:0/3803106193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a415e1a0 0x7fa4a4071b40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa4940097c0 tx=0x7fa49401c410 comp rx=0 tx=0).stop 2024-04-25T00:38:58.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a8837640 1 -- 172.21.15.29:0/3803106193 shutdown_connections 2024-04-25T00:38:58.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a8837640 1 --2- 172.21.15.29:0/3803106193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a415e1a0 0x7fa4a4071b40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:58.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a8837640 1 -- 172.21.15.29:0/3803106193 >> 172.21.15.29:0/3803106193 conn(0x7fa4a40b1870 msgr2=0x7fa4a40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:58.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a8837640 1 -- 172.21.15.29:0/3803106193 shutdown_connections 2024-04-25T00:38:58.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.894+0000 7fa4a8837640 1 -- 172.21.15.29:0/3803106193 wait complete. 2024-04-25T00:38:58.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a8837640 1 Processor -- start 2024-04-25T00:38:58.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a8837640 1 -- start start 2024-04-25T00:38:58.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a8837640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4a4070ce0 0x7fa4a4150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:58.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a8837640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4a40b4d00 0x7fa4a415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:58.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a8837640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a415e1a0 0x7fa4a415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:58.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a8837640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a406e990 con 0x7fa4a40b4d00 2024-04-25T00:38:58.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a8837640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a406e810 con 0x7fa4a415e1a0 2024-04-25T00:38:58.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a8837640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a416bd60 con 0x7fa4a4070ce0 2024-04-25T00:38:58.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a1d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4a40b4d00 0x7fa4a415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:58.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a1d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4a40b4d00 0x7fa4a415c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50812/0 (socket says 172.21.15.29:50812) 2024-04-25T00:38:58.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a1d9b640 1 -- 172.21.15.29:0/4229708249 learned_addr learned my addr 172.21.15.29:0/4229708249 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:58.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a259c640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4a4070ce0 0x7fa4a4150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:58.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a2d9d640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a415e1a0 0x7fa4a415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:58.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a1d9b640 1 -- 172.21.15.29:0/4229708249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4a4070ce0 msgr2=0x7fa4a4150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:58.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a1d9b640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4a4070ce0 0x7fa4a4150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:58.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a1d9b640 1 -- 172.21.15.29:0/4229708249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a415e1a0 msgr2=0x7fa4a415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:58.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a1d9b640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a415e1a0 0x7fa4a415ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:58.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a1d9b640 1 -- 172.21.15.29:0/4229708249 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a415c280 con 0x7fa4a40b4d00 2024-04-25T00:38:58.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a1d9b640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4a40b4d00 0x7fa4a415c4d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fa49800c3d0 tx=0x7fa49800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:58.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa492ffd640 1 -- 172.21.15.29:0/4229708249 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498017070 con 0x7fa4a40b4d00 2024-04-25T00:38:58.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa492ffd640 1 -- 172.21.15.29:0/4229708249 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4980052f0 con 0x7fa4a40b4d00 2024-04-25T00:38:58.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a8837640 1 -- 172.21.15.29:0/4229708249 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a41587c0 con 0x7fa4a40b4d00 2024-04-25T00:38:58.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa492ffd640 1 -- 172.21.15.29:0/4229708249 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498003860 con 0x7fa4a40b4d00 2024-04-25T00:38:58.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a8837640 1 -- 172.21.15.29:0/4229708249 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a4158d40 con 0x7fa4a40b4d00 2024-04-25T00:38:58.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.898+0000 7fa4a8837640 1 -- 172.21.15.29:0/4229708249 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a40692e0 con 0x7fa4a40b4d00 2024-04-25T00:38:58.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.902+0000 7fa492ffd640 1 -- 172.21.15.29:0/4229708249 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fa49801a070 con 0x7fa4a40b4d00 2024-04-25T00:38:58.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.902+0000 7fa492ffd640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa484067e60 0x7fa48406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:58.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.902+0000 7fa492ffd640 1 -- 172.21.15.29:0/4229708249 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5357+0+0 (secure 0 0 0) 0x7fa498085880 con 0x7fa4a40b4d00 2024-04-25T00:38:58.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.902+0000 7fa4a259c640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa484067e60 0x7fa48406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:58.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.902+0000 7fa4a259c640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa484067e60 0x7fa48406a320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa4940ad000 tx=0x7fa4940a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:58.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:58.906+0000 7fa492ffd640 1 -- 172.21.15.29:0/4229708249 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa498055d40 con 0x7fa4a40b4d00 2024-04-25T00:38:59.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.066+0000 7fa4a8837640 1 -- 172.21.15.29:0/4229708249 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa4a40b0560 con 0x7fa4a40b4d00 2024-04-25T00:38:59.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.730+0000 7fa492ffd640 1 -- 172.21.15.29:0/4229708249 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v123) ==== 169+0+0 (secure 0 0 0) 0x7fa498055640 con 0x7fa4a40b4d00 2024-04-25T00:38:59.733 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-04-25T00:38:59.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.734+0000 7fa4a8837640 1 -- 172.21.15.29:0/4229708249 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa484067e60 msgr2=0x7fa48406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:59.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.734+0000 7fa4a8837640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa484067e60 0x7fa48406a320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa4940ad000 tx=0x7fa4940a6000 comp rx=0 tx=0).stop 2024-04-25T00:38:59.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.734+0000 7fa4a8837640 1 -- 172.21.15.29:0/4229708249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4a40b4d00 msgr2=0x7fa4a415c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:59.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.734+0000 7fa4a8837640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4a40b4d00 0x7fa4a415c4d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fa49800c3d0 tx=0x7fa49800c890 comp rx=0 tx=0).stop 2024-04-25T00:38:59.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.734+0000 7fa4a8837640 1 -- 172.21.15.29:0/4229708249 shutdown_connections 2024-04-25T00:38:59.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.734+0000 7fa4a8837640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa484067e60 0x7fa48406a320 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:59.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.734+0000 7fa4a8837640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a415e1a0 0x7fa4a415ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:59.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.734+0000 7fa4a8837640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4a40b4d00 0x7fa4a415c4d0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:59.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.734+0000 7fa4a8837640 1 --2- 172.21.15.29:0/4229708249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4a4070ce0 0x7fa4a4150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:59.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.734+0000 7fa4a8837640 1 -- 172.21.15.29:0/4229708249 >> 172.21.15.29:0/4229708249 conn(0x7fa4a40b1870 msgr2=0x7fa4a40af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:59.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.734+0000 7fa4a8837640 1 -- 172.21.15.29:0/4229708249 shutdown_connections 2024-04-25T00:38:59.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.734+0000 7fa4a8837640 1 -- 172.21.15.29:0/4229708249 wait complete. 2024-04-25T00:38:59.760 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:38:59.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.886+0000 7f33e758f640 1 Processor -- start 2024-04-25T00:38:59.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.886+0000 7f33e758f640 1 -- start start 2024-04-25T00:38:59.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.890+0000 7f33e758f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33e015e200 0x7f33e00d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:59.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.890+0000 7f33e758f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33e00d2c60 0x7f33e00b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:59.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.890+0000 7f33e758f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33e00cfc10 con 0x7f33e00cf690 2024-04-25T00:38:59.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.890+0000 7f33e758f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f33e00b38d0 con 0x7f33e00d2c60 2024-04-25T00:38:59.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.890+0000 7f33e758f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f33e00cfef0 con 0x7f33e015e200 2024-04-25T00:38:59.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.890+0000 7f33e5b2c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33e00d2c60 0x7f33e00b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:59.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.890+0000 7f33e4b2a640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f33e00cf690 0x7f33e00d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59404/0 (socket says 172.21.15.29:59404) 2024-04-25T00:38:59.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.890+0000 7f33e4b2a640 1 -- 172.21.15.29:0/180039611 learned_addr learned my addr 172.21.15.29:0/180039611 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:38:59.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.890+0000 7f33e532b640 1 --2- 172.21.15.29:0/180039611 >> v2:172.21.15.157:3300/0 conn(0x7f33e015e200 0x7f33e00d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:59.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.890+0000 7f33e5b2c640 1 -- 172.21.15.29:0/180039611 >> v2:172.21.15.157:3300/0 conn(0x7f33e015e200 msgr2=0x7f33e00d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:59.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.890+0000 7f33e5b2c640 1 --2- 172.21.15.29:0/180039611 >> v2:172.21.15.157:3300/0 conn(0x7f33e015e200 0x7f33e00d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:59.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.890+0000 7f33e5b2c640 1 -- 172.21.15.29:0/180039611 >> v1:172.21.15.29:6789/0 conn(0x7f33e00cf690 legacy=0x7f33e00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:59.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.890+0000 7f33e5b2c640 1 -- 172.21.15.29:0/180039611 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e00b3b20 con 0x7f33e00d2c60 2024-04-25T00:38:59.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e5b2c640 1 --2- 172.21.15.29:0/180039611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33e00d2c60 0x7f33e00b4320 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f33e008d8f0 tx=0x7f33d801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f26200503697a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:59.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33d7fff640 1 -- 172.21.15.29:0/180039611 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d80b2070 con 0x7f33e00d2c60 2024-04-25T00:38:59.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33d7fff640 1 -- 172.21.15.29:0/180039611 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d80a59e0 con 0x7f33e00d2c60 2024-04-25T00:38:59.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33d7fff640 1 -- 172.21.15.29:0/180039611 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d80a5db0 con 0x7f33e00d2c60 2024-04-25T00:38:59.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 -- 172.21.15.29:0/180039611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33e00d2c60 msgr2=0x7f33e00b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:59.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 --2- 172.21.15.29:0/180039611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33e00d2c60 0x7f33e00b4320 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f33e008d8f0 tx=0x7f33d801c900 comp rx=0 tx=0).stop 2024-04-25T00:38:59.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 -- 172.21.15.29:0/180039611 shutdown_connections 2024-04-25T00:38:59.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 --2- 172.21.15.29:0/180039611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33e00d2c60 0x7f33e00b4320 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:59.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 --2- 172.21.15.29:0/180039611 >> v2:172.21.15.157:3300/0 conn(0x7f33e015e200 0x7f33e00d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:59.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 -- 172.21.15.29:0/180039611 >> 172.21.15.29:0/180039611 conn(0x7f33e009d280 msgr2=0x7f33e00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:59.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 -- 172.21.15.29:0/180039611 shutdown_connections 2024-04-25T00:38:59.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 -- 172.21.15.29:0/180039611 wait complete. 2024-04-25T00:38:59.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 Processor -- start 2024-04-25T00:38:59.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 -- start start 2024-04-25T00:38:59.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33e00cf690 0x7f33e014cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:59.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33e00d2c60 0x7f33e014dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:59.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e015e200 0x7f33e00b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:59.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f33e00b51f0 con 0x7f33e00cf690 2024-04-25T00:38:59.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33e00b5070 con 0x7f33e015e200 2024-04-25T00:38:59.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e758f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f33e00b5370 con 0x7f33e00d2c60 2024-04-25T00:38:59.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e532b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33e00cf690 0x7f33e014cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:59.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e532b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33e00cf690 0x7f33e014cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50828/0 (socket says 172.21.15.29:50828) 2024-04-25T00:38:59.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e532b640 1 -- 172.21.15.29:0/1549345519 learned_addr learned my addr 172.21.15.29:0/1549345519 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:38:59.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e5b2c640 1 --2- 172.21.15.29:0/1549345519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e015e200 0x7f33e00b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:59.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e532b640 1 -- 172.21.15.29:0/1549345519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33e00d2c60 msgr2=0x7f33e014dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:59.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e532b640 1 --2- 172.21.15.29:0/1549345519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33e00d2c60 0x7f33e014dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:59.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e532b640 1 -- 172.21.15.29:0/1549345519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e015e200 msgr2=0x7f33e00b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:59.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e532b640 1 --2- 172.21.15.29:0/1549345519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e015e200 0x7f33e00b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:59.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e532b640 1 -- 172.21.15.29:0/1549345519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e014e600 con 0x7f33e00cf690 2024-04-25T00:38:59.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33e532b640 1 --2- 172.21.15.29:0/1549345519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33e00cf690 0x7f33e014cb70 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00d5f0 tx=0x7f33dc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:59.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.894+0000 7f33d5ffb640 1 -- 172.21.15.29:0/1549345519 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0125f0 con 0x7f33e00cf690 2024-04-25T00:38:59.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.898+0000 7f33d5ffb640 1 -- 172.21.15.29:0/1549345519 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33dc019070 con 0x7f33e00cf690 2024-04-25T00:38:59.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.898+0000 7f33e758f640 1 -- 172.21.15.29:0/1549345519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e015b2a0 con 0x7f33e00cf690 2024-04-25T00:38:59.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.898+0000 7f33d5ffb640 1 -- 172.21.15.29:0/1549345519 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0129b0 con 0x7f33e00cf690 2024-04-25T00:38:59.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.898+0000 7f33e758f640 1 -- 172.21.15.29:0/1549345519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e015b860 con 0x7f33e00cf690 2024-04-25T00:38:59.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.898+0000 7f33e758f640 1 -- 172.21.15.29:0/1549345519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33e0069270 con 0x7f33e00cf690 2024-04-25T00:38:59.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.898+0000 7f33d5ffb640 1 -- 172.21.15.29:0/1549345519 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f33dc012b50 con 0x7f33e00cf690 2024-04-25T00:38:59.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.898+0000 7f33d5ffb640 1 --2- 172.21.15.29:0/1549345519 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f33c4067e60 0x7f33c406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:59.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.898+0000 7f33d5ffb640 1 -- 172.21.15.29:0/1549345519 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7f33dc086740 con 0x7f33e00cf690 2024-04-25T00:38:59.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.898+0000 7f33e4b2a640 1 --2- 172.21.15.29:0/1549345519 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f33c4067e60 0x7f33c406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:59.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.898+0000 7f33e4b2a640 1 --2- 172.21.15.29:0/1549345519 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f33c4067e60 0x7f33c406a320 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f33d0000f70 tx=0x7f33d0007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:59.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:38:59.902+0000 7f33d5ffb640 1 -- 172.21.15.29:0/1549345519 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f33e014e600 con 0x7f33e00cf690 2024-04-25T00:39:00.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.074+0000 7f33e758f640 1 -- 172.21.15.29:0/1549345519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f33e006bdf0 con 0x7f33e00cf690 2024-04-25T00:39:00.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.078+0000 7f33d5ffb640 1 -- 172.21.15.29:0/1549345519 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f33dc011340 con 0x7f33e00cf690 2024-04-25T00:39:00.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.082+0000 7f33e758f640 1 -- 172.21.15.29:0/1549345519 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f33c4067e60 msgr2=0x7f33c406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:00.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.082+0000 7f33e758f640 1 --2- 172.21.15.29:0/1549345519 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f33c4067e60 0x7f33c406a320 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f33d0000f70 tx=0x7f33d0007a90 comp rx=0 tx=0).stop 2024-04-25T00:39:00.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.082+0000 7f33e758f640 1 -- 172.21.15.29:0/1549345519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33e00cf690 msgr2=0x7f33e014cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:00.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.082+0000 7f33e758f640 1 --2- 172.21.15.29:0/1549345519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33e00cf690 0x7f33e014cb70 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00d5f0 tx=0x7f33dc00dab0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.082+0000 7f33e758f640 1 -- 172.21.15.29:0/1549345519 shutdown_connections 2024-04-25T00:39:00.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.082+0000 7f33e758f640 1 --2- 172.21.15.29:0/1549345519 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f33c4067e60 0x7f33c406a320 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.082+0000 7f33e758f640 1 --2- 172.21.15.29:0/1549345519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e015e200 0x7f33e00b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.082+0000 7f33e758f640 1 --2- 172.21.15.29:0/1549345519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33e00d2c60 0x7f33e014dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.082+0000 7f33e758f640 1 --2- 172.21.15.29:0/1549345519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33e00cf690 0x7f33e014cb70 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.082+0000 7f33e758f640 1 -- 172.21.15.29:0/1549345519 >> 172.21.15.29:0/1549345519 conn(0x7f33e009d280 msgr2=0x7f33e00d2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:00.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.082+0000 7f33e758f640 1 -- 172.21.15.29:0/1549345519 shutdown_connections 2024-04-25T00:39:00.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.082+0000 7f33e758f640 1 -- 172.21.15.29:0/1549345519 wait complete. 2024-04-25T00:39:00.101 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-04-25T00:39:00.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f95a0d61640 1 Processor -- start 2024-04-25T00:39:00.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f95a0d61640 1 -- start start 2024-04-25T00:39:00.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f95a0d61640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f959c15d980 0x7f959c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:00.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f95a0d61640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f959c0b2b70 0x7f959c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:00.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f95a0d61640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f959c151950 con 0x7f959c150df0 2024-04-25T00:39:00.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f95a0d61640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f959c0cf560 con 0x7f959c0b2b70 2024-04-25T00:39:00.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f95a0d61640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f959c0b3130 con 0x7f959c15d980 2024-04-25T00:39:00.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f959ad9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f959c150df0 0x7f959c151240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59416/0 (socket says 172.21.15.29:59416) 2024-04-25T00:39:00.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f959ad9d640 1 -- 172.21.15.29:0/943810008 learned_addr learned my addr 172.21.15.29:0/943810008 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:00.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f9599d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f959c0b2b70 0x7f959c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:00.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f959a59c640 1 --2- 172.21.15.29:0/943810008 >> v2:172.21.15.157:3300/0 conn(0x7f959c15d980 0x7f959c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:00.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f959959a640 1 -- 172.21.15.29:0/943810008 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2614824051 0 0) 0x7f959c151950 con 0x7f959c150df0 2024-04-25T00:39:00.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f959959a640 1 -- 172.21.15.29:0/943810008 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9588003530 con 0x7f959c150df0 2024-04-25T00:39:00.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f9599d9b640 1 -- 172.21.15.29:0/943810008 >> v2:172.21.15.157:3300/0 conn(0x7f959c15d980 msgr2=0x7f959c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:00.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f9599d9b640 1 --2- 172.21.15.29:0/943810008 >> v2:172.21.15.157:3300/0 conn(0x7f959c15d980 0x7f959c06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f9599d9b640 1 -- 172.21.15.29:0/943810008 >> v1:172.21.15.29:6789/0 conn(0x7f959c150df0 legacy=0x7f959c151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:00.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.218+0000 7f9599d9b640 1 -- 172.21.15.29:0/943810008 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c0b3410 con 0x7f959c0b2b70 2024-04-25T00:39:00.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f9599d9b640 1 --2- 172.21.15.29:0/943810008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f959c0b2b70 0x7f959c06db10 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f959c0d0ca0 tx=0x7f958401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7aa764b31d14bda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f959959a640 1 -- 172.21.15.29:0/943810008 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95840b1070 con 0x7f959c0b2b70 2024-04-25T00:39:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f959959a640 1 -- 172.21.15.29:0/943810008 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95840ac630 con 0x7f959c0b2b70 2024-04-25T00:39:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f959959a640 1 -- 172.21.15.29:0/943810008 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95840a8050 con 0x7f959c0b2b70 2024-04-25T00:39:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f95a0d61640 1 -- 172.21.15.29:0/943810008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f959c0b2b70 msgr2=0x7f959c06db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f95a0d61640 1 --2- 172.21.15.29:0/943810008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f959c0b2b70 0x7f959c06db10 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f959c0d0ca0 tx=0x7f958401c450 comp rx=0 tx=0).stop 2024-04-25T00:39:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f95a0d61640 1 -- 172.21.15.29:0/943810008 shutdown_connections 2024-04-25T00:39:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f95a0d61640 1 --2- 172.21.15.29:0/943810008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f959c0b2b70 0x7f959c06db10 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f95a0d61640 1 --2- 172.21.15.29:0/943810008 >> v2:172.21.15.157:3300/0 conn(0x7f959c15d980 0x7f959c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f95a0d61640 1 -- 172.21.15.29:0/943810008 >> 172.21.15.29:0/943810008 conn(0x7f959c0b1870 msgr2=0x7f959c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f95a0d61640 1 -- 172.21.15.29:0/943810008 shutdown_connections 2024-04-25T00:39:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f95a0d61640 1 -- 172.21.15.29:0/943810008 wait complete. 2024-04-25T00:39:00.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f95a0d61640 1 Processor -- start 2024-04-25T00:39:00.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.222+0000 7f95a0d61640 1 -- start start 2024-04-25T00:39:00.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f95a0d61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f959c0b2b70 0x7f959c1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:00.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f95a0d61640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f959c150df0 0x7f959c14a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f95a0d61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f959c15d980 0x7f959c14a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f95a0d61640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1663c0 con 0x7f959c150df0 2024-04-25T00:39:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f95a0d61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f959c15dd60 con 0x7f959c0b2b70 2024-04-25T00:39:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f95a0d61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f959c166540 con 0x7f959c15d980 2024-04-25T00:39:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9599d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f959c150df0 0x7f959c14a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9599d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f959c150df0 0x7f959c14a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50838/0 (socket says 172.21.15.29:50838) 2024-04-25T00:39:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9599d9b640 1 -- 172.21.15.29:0/593661031 learned_addr learned my addr 172.21.15.29:0/593661031 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f959ad9d640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f959c15d980 0x7f959c14a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f959a59c640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f959c0b2b70 0x7f959c1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9599d9b640 1 -- 172.21.15.29:0/593661031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f959c15d980 msgr2=0x7f959c14a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9599d9b640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f959c15d980 0x7f959c14a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9599d9b640 1 -- 172.21.15.29:0/593661031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f959c0b2b70 msgr2=0x7f959c1596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9599d9b640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f959c0b2b70 0x7f959c1596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9599d9b640 1 -- 172.21.15.29:0/593661031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c0cddf0 con 0x7f959c150df0 2024-04-25T00:39:00.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f959a59c640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f959c0b2b70 0x7f959c1596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:00.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9599d9b640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f959c150df0 0x7f959c14a030 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f95840af040 tx=0x7f95840191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:00.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9592ffd640 1 -- 172.21.15.29:0/593661031 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95840b1040 con 0x7f959c150df0 2024-04-25T00:39:00.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9592ffd640 1 -- 172.21.15.29:0/593661031 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95840ba070 con 0x7f959c150df0 2024-04-25T00:39:00.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9592ffd640 1 -- 172.21.15.29:0/593661031 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95840acc00 con 0x7f959c150df0 2024-04-25T00:39:00.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f95a0d61640 1 -- 172.21.15.29:0/593661031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c0ce0d0 con 0x7f959c150df0 2024-04-25T00:39:00.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f95a0d61640 1 -- 172.21.15.29:0/593661031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c159c10 con 0x7f959c150df0 2024-04-25T00:39:00.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f95a0d61640 1 -- 172.21.15.29:0/593661031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f959c158fd0 con 0x7f959c150df0 2024-04-25T00:39:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9592ffd640 1 -- 172.21.15.29:0/593661031 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f9584003620 con 0x7f959c150df0 2024-04-25T00:39:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9592ffd640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f957c067e60 0x7f957c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f9592ffd640 1 -- 172.21.15.29:0/593661031 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7f958411fe70 con 0x7f959c150df0 2024-04-25T00:39:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f959a59c640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f957c067e60 0x7f957c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:00.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.226+0000 7f959a59c640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f957c067e60 0x7f957c06a320 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f958c0097a0 tx=0x7f958c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:00.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.230+0000 7f9592ffd640 1 -- 172.21.15.29:0/593661031 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f959c158fd0 con 0x7f959c150df0 2024-04-25T00:39:00.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.454+0000 7f95a0d61640 1 -- 172.21.15.29:0/593661031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7f959c069270 con 0x7f959c150df0 2024-04-25T00:39:00.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.734+0000 7f9592ffd640 1 -- 172.21.15.29:0/593661031 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v401) ==== 148+0+0 (secure 0 0 0) 0x7f9584016070 con 0x7f959c150df0 2024-04-25T00:39:00.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.738+0000 7f95a0d61640 1 -- 172.21.15.29:0/593661031 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f957c067e60 msgr2=0x7f957c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:00.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.738+0000 7f95a0d61640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f957c067e60 0x7f957c06a320 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f958c0097a0 tx=0x7f958c008040 comp rx=0 tx=0).stop 2024-04-25T00:39:00.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.738+0000 7f95a0d61640 1 -- 172.21.15.29:0/593661031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f959c150df0 msgr2=0x7f959c14a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:00.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.738+0000 7f95a0d61640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f959c150df0 0x7f959c14a030 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f95840af040 tx=0x7f95840191f0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.738+0000 7f95a0d61640 1 -- 172.21.15.29:0/593661031 shutdown_connections 2024-04-25T00:39:00.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.738+0000 7f95a0d61640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f957c067e60 0x7f957c06a320 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.738+0000 7f95a0d61640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f959c15d980 0x7f959c14a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.738+0000 7f95a0d61640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f959c150df0 0x7f959c14a030 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.738+0000 7f95a0d61640 1 --2- 172.21.15.29:0/593661031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f959c0b2b70 0x7f959c1596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.738+0000 7f95a0d61640 1 -- 172.21.15.29:0/593661031 >> 172.21.15.29:0/593661031 conn(0x7f959c0b1870 msgr2=0x7f959c0afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:00.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.738+0000 7f95a0d61640 1 -- 172.21.15.29:0/593661031 shutdown_connections 2024-04-25T00:39:00.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.738+0000 7f95a0d61640 1 -- 172.21.15.29:0/593661031 wait complete. 2024-04-25T00:39:00.760 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-04-25T00:39:00.760 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2024-04-25T00:39:00.760 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:00.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd08c7fb640 1 Processor -- start 2024-04-25T00:39:00.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd08c7fb640 1 -- start start 2024-04-25T00:39:00.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd08c7fb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd0840b4d30 0x7fd0840b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:00.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd08c7fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08406d210 0x7fd0840ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:00.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd08c7fb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd08406d6c0 con 0x7fd08415df70 2024-04-25T00:39:00.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd08c7fb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd08406d980 con 0x7fd0840b4d30 2024-04-25T00:39:00.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd08c7fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd08406dc60 con 0x7fd08406d210 2024-04-25T00:39:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd089d96640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd0840b4d30 0x7fd0840b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd08ad98640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08406d210 0x7fd0840ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd08a597640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd08415df70 0x7fd0840b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:45596/0 (socket says 172.21.15.29:45596) 2024-04-25T00:39:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd089d96640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd0840b4d30 0x7fd0840b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:36482/0 (socket says 172.21.15.29:36482) 2024-04-25T00:39:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd08a597640 1 -- 172.21.15.29:0/752510601 learned_addr learned my addr 172.21.15.29:0/752510601 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd089d96640 1 -- 172.21.15.29:0/752510601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08406d210 msgr2=0x7fd0840ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd089d96640 1 --2- 172.21.15.29:0/752510601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08406d210 0x7fd0840ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd089d96640 1 -- 172.21.15.29:0/752510601 >> v1:172.21.15.157:6789/0 conn(0x7fd08415df70 legacy=0x7fd0840b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:00.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.882+0000 7fd089d96640 1 -- 172.21.15.29:0/752510601 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08406df40 con 0x7fd0840b4d30 2024-04-25T00:39:00.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd089d96640 1 --2- 172.21.15.29:0/752510601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0840b4d30 0x7fd0840b3e40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd0800097c0 tx=0x7fd08001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=585eceb70f30c724 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:00.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd089595640 1 -- 172.21.15.29:0/752510601 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0800a7070 con 0x7fd0840b4d30 2024-04-25T00:39:00.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd089595640 1 -- 172.21.15.29:0/752510601 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0800b0070 con 0x7fd0840b4d30 2024-04-25T00:39:00.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd089595640 1 -- 172.21.15.29:0/752510601 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0800ab890 con 0x7fd0840b4d30 2024-04-25T00:39:00.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd08c7fb640 1 -- 172.21.15.29:0/752510601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0840b4d30 msgr2=0x7fd0840b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:00.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd08c7fb640 1 --2- 172.21.15.29:0/752510601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0840b4d30 0x7fd0840b3e40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd0800097c0 tx=0x7fd08001c470 comp rx=0 tx=0).stop 2024-04-25T00:39:00.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd08c7fb640 1 -- 172.21.15.29:0/752510601 shutdown_connections 2024-04-25T00:39:00.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd08c7fb640 1 --2- 172.21.15.29:0/752510601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08406d210 0x7fd0840ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd08c7fb640 1 --2- 172.21.15.29:0/752510601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0840b4d30 0x7fd0840b3e40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd08c7fb640 1 -- 172.21.15.29:0/752510601 >> 172.21.15.29:0/752510601 conn(0x7fd0840b1870 msgr2=0x7fd0840b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:00.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd08c7fb640 1 -- 172.21.15.29:0/752510601 shutdown_connections 2024-04-25T00:39:00.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd08c7fb640 1 -- 172.21.15.29:0/752510601 wait complete. 2024-04-25T00:39:00.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd08c7fb640 1 Processor -- start 2024-04-25T00:39:00.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.886+0000 7fd08c7fb640 1 -- start start 2024-04-25T00:39:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08c7fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd08406d210 0x7fd0840cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08c7fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0840b4d30 0x7fd0841434c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08c7fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd08415df70 0x7fd0841555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08c7fb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd08406eba0 con 0x7fd08415df70 2024-04-25T00:39:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08c7fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd08406ea20 con 0x7fd08406d210 2024-04-25T00:39:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08c7fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd0840ae5f0 con 0x7fd0840b4d30 2024-04-25T00:39:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08ad98640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd08415df70 0x7fd0841555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08ad98640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd08415df70 0x7fd0841555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50854/0 (socket says 172.21.15.29:50854) 2024-04-25T00:39:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08ad98640 1 -- 172.21.15.29:0/197708053 learned_addr learned my addr 172.21.15.29:0/197708053 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:00.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd089d96640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0840b4d30 0x7fd0841434c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08a597640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd08406d210 0x7fd0840cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08ad98640 1 -- 172.21.15.29:0/197708053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0840b4d30 msgr2=0x7fd0841434c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08ad98640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0840b4d30 0x7fd0841434c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08ad98640 1 -- 172.21.15.29:0/197708053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd08406d210 msgr2=0x7fd0840cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08ad98640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd08406d210 0x7fd0840cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08ad98640 1 -- 172.21.15.29:0/197708053 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd084152e50 con 0x7fd08415df70 2024-04-25T00:39:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08ad98640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd08415df70 0x7fd0841555e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00efc0 tx=0x7fd07c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd07affd640 1 -- 172.21.15.29:0/197708053 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c018020 con 0x7fd08415df70 2024-04-25T00:39:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd07affd640 1 -- 172.21.15.29:0/197708053 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07c014480 con 0x7fd08415df70 2024-04-25T00:39:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd07affd640 1 -- 172.21.15.29:0/197708053 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c014870 con 0x7fd08415df70 2024-04-25T00:39:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08c7fb640 1 -- 172.21.15.29:0/197708053 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0840ce2d0 con 0x7fd08415df70 2024-04-25T00:39:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08c7fb640 1 -- 172.21.15.29:0/197708053 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0840cbf20 con 0x7fd08415df70 2024-04-25T00:39:00.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08c7fb640 1 -- 172.21.15.29:0/197708053 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd084069270 con 0x7fd08415df70 2024-04-25T00:39:00.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd07affd640 1 -- 172.21.15.29:0/197708053 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fd07c01b020 con 0x7fd08415df70 2024-04-25T00:39:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd07affd640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd068067e60 0x7fd06806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd07affd640 1 -- 172.21.15.29:0/197708053 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7fd07c087090 con 0x7fd08415df70 2024-04-25T00:39:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.890+0000 7fd08a597640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd068067e60 0x7fd06806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.894+0000 7fd08a597640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd068067e60 0x7fd06806a320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd074000f70 tx=0x7fd074009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:00.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:00.894+0000 7fd07affd640 1 -- 172.21.15.29:0/197708053 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd07c022070 con 0x7fd08415df70 2024-04-25T00:39:01.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.062+0000 7fd08c7fb640 1 -- 172.21.15.29:0/197708053 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0840af560 con 0x7fd08415df70 2024-04-25T00:39:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.066+0000 7fd07affd640 1 -- 172.21.15.29:0/197708053 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+12982 (secure 0 0 0) 0x7fd084152e50 con 0x7fd08415df70 2024-04-25T00:39:01.068 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:01.069 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":123,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:38:59.727452+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6835/3206404146":"2024-04-26T00:38:56.713723+0000","172.21.15.29:6834/3206404146":"2024-04-26T00:38:56.713723+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.066+0000 7fd08c7fb640 1 -- 172.21.15.29:0/197708053 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd068067e60 msgr2=0x7fd06806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.066+0000 7fd08c7fb640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd068067e60 0x7fd06806a320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd074000f70 tx=0x7fd074009040 comp rx=0 tx=0).stop 2024-04-25T00:39:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.066+0000 7fd08c7fb640 1 -- 172.21.15.29:0/197708053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd08415df70 msgr2=0x7fd0841555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.066+0000 7fd08c7fb640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd08415df70 0x7fd0841555e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00efc0 tx=0x7fd07c00c490 comp rx=0 tx=0).stop 2024-04-25T00:39:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.066+0000 7fd08c7fb640 1 -- 172.21.15.29:0/197708053 shutdown_connections 2024-04-25T00:39:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.066+0000 7fd08c7fb640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd068067e60 0x7fd06806a320 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.066+0000 7fd08c7fb640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd08415df70 0x7fd0841555e0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.066+0000 7fd08c7fb640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0840b4d30 0x7fd0841434c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.066+0000 7fd08c7fb640 1 --2- 172.21.15.29:0/197708053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd08406d210 0x7fd0840cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.066+0000 7fd08c7fb640 1 -- 172.21.15.29:0/197708053 >> 172.21.15.29:0/197708053 conn(0x7fd0840b1870 msgr2=0x7fd0840cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:01.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.066+0000 7fd08c7fb640 1 -- 172.21.15.29:0/197708053 shutdown_connections 2024-04-25T00:39:01.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.066+0000 7fd08c7fb640 1 -- 172.21.15.29:0/197708053 wait complete. 2024-04-25T00:39:01.088 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:39:01.089 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:39:01.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f6485449640 1 Processor -- start 2024-04-25T00:39:01.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f6485449640 1 -- start start 2024-04-25T00:39:01.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f6485449640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6480153030 0x7f648006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f6485449640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f64800cd200 0x7f648006b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f6485449640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f648015d580 con 0x7f648006b750 2024-04-25T00:39:01.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f6485449640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f648015d3d0 con 0x7f64800cd200 2024-04-25T00:39:01.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f6485449640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f648015c3c0 con 0x7f6480153030 2024-04-25T00:39:01.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f647e7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f64800cd200 0x7f648006b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:01.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f647f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f648006b750 0x7f64800b5ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59428/0 (socket says 172.21.15.29:59428) 2024-04-25T00:39:01.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f647f7fe640 1 -- 172.21.15.29:0/3294848881 learned_addr learned my addr 172.21.15.29:0/3294848881 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:01.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f647effd640 1 --2- 172.21.15.29:0/3294848881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6480153030 0x7f648006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:01.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f647dffb640 1 -- 172.21.15.29:0/3294848881 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3271472484 0 0) 0x7f648015d580 con 0x7f648006b750 2024-04-25T00:39:01.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f647dffb640 1 -- 172.21.15.29:0/3294848881 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6468003530 con 0x7f648006b750 2024-04-25T00:39:01.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f647e7fc640 1 -- 172.21.15.29:0/3294848881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6480153030 msgr2=0x7f648006fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:01.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f647e7fc640 1 --2- 172.21.15.29:0/3294848881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6480153030 0x7f648006fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f647e7fc640 1 -- 172.21.15.29:0/3294848881 >> v1:172.21.15.29:6789/0 conn(0x7f648006b750 legacy=0x7f64800b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:01.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.210+0000 7f647e7fc640 1 -- 172.21.15.29:0/3294848881 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648015c610 con 0x7f64800cd200 2024-04-25T00:39:01.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f647e7fc640 1 --2- 172.21.15.29:0/3294848881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64800cd200 0x7f648006b210 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f64700097c0 tx=0x7f647001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f271229bf9888d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:01.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f647dffb640 1 -- 172.21.15.29:0/3294848881 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64700b0070 con 0x7f64800cd200 2024-04-25T00:39:01.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f647dffb640 1 -- 172.21.15.29:0/3294848881 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64700ab600 con 0x7f64800cd200 2024-04-25T00:39:01.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f647dffb640 1 -- 172.21.15.29:0/3294848881 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64700ab9d0 con 0x7f64800cd200 2024-04-25T00:39:01.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f6485449640 1 -- 172.21.15.29:0/3294848881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64800cd200 msgr2=0x7f648006b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:01.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f6485449640 1 --2- 172.21.15.29:0/3294848881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64800cd200 0x7f648006b210 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f64700097c0 tx=0x7f647001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f6485449640 1 -- 172.21.15.29:0/3294848881 shutdown_connections 2024-04-25T00:39:01.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f6485449640 1 --2- 172.21.15.29:0/3294848881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64800cd200 0x7f648006b210 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f6485449640 1 --2- 172.21.15.29:0/3294848881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6480153030 0x7f648006fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f6485449640 1 -- 172.21.15.29:0/3294848881 >> 172.21.15.29:0/3294848881 conn(0x7f64800d0c10 msgr2=0x7f64800d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:01.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f6485449640 1 -- 172.21.15.29:0/3294848881 shutdown_connections 2024-04-25T00:39:01.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f6485449640 1 -- 172.21.15.29:0/3294848881 wait complete. 2024-04-25T00:39:01.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f6485449640 1 Processor -- start 2024-04-25T00:39:01.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.214+0000 7f6485449640 1 -- start start 2024-04-25T00:39:01.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f6485449640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f648006b750 0x7f64800c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f6485449640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64800cd200 0x7f64800b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f6485449640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6480153030 0x7f64800ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f6485449640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6480070550 con 0x7f6480153030 2024-04-25T00:39:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f6485449640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f64800703d0 con 0x7f64800cd200 2024-04-25T00:39:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f6485449640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6480071d80 con 0x7f648006b750 2024-04-25T00:39:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f647f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6480153030 0x7f64800ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f647f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6480153030 0x7f64800ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50868/0 (socket says 172.21.15.29:50868) 2024-04-25T00:39:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f647f7fe640 1 -- 172.21.15.29:0/3799630867 learned_addr learned my addr 172.21.15.29:0/3799630867 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f647effd640 1 --2- 172.21.15.29:0/3799630867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f648006b750 0x7f64800c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f647f7fe640 1 -- 172.21.15.29:0/3799630867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f648006b750 msgr2=0x7f64800c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f647f7fe640 1 --2- 172.21.15.29:0/3799630867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f648006b750 0x7f64800c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f647f7fe640 1 -- 172.21.15.29:0/3799630867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64800cd200 msgr2=0x7f64800b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:01.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f647f7fe640 1 --2- 172.21.15.29:0/3799630867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64800cd200 0x7f64800b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f647f7fe640 1 -- 172.21.15.29:0/3799630867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64800ce560 con 0x7f6480153030 2024-04-25T00:39:01.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f647f7fe640 1 --2- 172.21.15.29:0/3799630867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6480153030 0x7f64800ce7e0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f647800cb40 tx=0x7f647800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:01.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f64677fe640 1 -- 172.21.15.29:0/3799630867 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478017070 con 0x7f6480153030 2024-04-25T00:39:01.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f64677fe640 1 -- 172.21.15.29:0/3799630867 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f647800aca0 con 0x7f6480153030 2024-04-25T00:39:01.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f6485449640 1 -- 172.21.15.29:0/3799630867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64800c9930 con 0x7f6480153030 2024-04-25T00:39:01.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f64677fe640 1 -- 172.21.15.29:0/3799630867 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478012620 con 0x7f6480153030 2024-04-25T00:39:01.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f6485449640 1 -- 172.21.15.29:0/3799630867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64800c9ef0 con 0x7f6480153030 2024-04-25T00:39:01.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f6485449640 1 -- 172.21.15.29:0/3799630867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6480069270 con 0x7f6480153030 2024-04-25T00:39:01.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f64677fe640 1 -- 172.21.15.29:0/3799630867 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f647801a070 con 0x7f6480153030 2024-04-25T00:39:01.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f64677fe640 1 --2- 172.21.15.29:0/3799630867 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f646c067e90 0x7f646c06a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f64677fe640 1 -- 172.21.15.29:0/3799630867 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7f6478086100 con 0x7f6480153030 2024-04-25T00:39:01.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f647effd640 1 --2- 172.21.15.29:0/3799630867 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f646c067e90 0x7f646c06a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:01.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.218+0000 7f647effd640 1 --2- 172.21.15.29:0/3799630867 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f646c067e90 0x7f646c06a350 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f64740096f0 tx=0x7f6474008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:01.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.222+0000 7f64677fe640 1 -- 172.21.15.29:0/3799630867 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6478056760 con 0x7f6480153030 2024-04-25T00:39:01.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f6485449640 1 -- 172.21.15.29:0/3799630867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f64800b5fb0 con 0x7f6480153030 2024-04-25T00:39:01.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f64677fe640 1 -- 172.21.15.29:0/3799630867 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) ==== 93+0+10 (secure 0 0 0) 0x7f6478056060 con 0x7f6480153030 2024-04-25T00:39:01.388 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:39:01.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f6485449640 1 -- 172.21.15.29:0/3799630867 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f646c067e90 msgr2=0x7f646c06a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:01.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f6485449640 1 --2- 172.21.15.29:0/3799630867 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f646c067e90 0x7f646c06a350 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f64740096f0 tx=0x7f6474008040 comp rx=0 tx=0).stop 2024-04-25T00:39:01.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f6485449640 1 -- 172.21.15.29:0/3799630867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6480153030 msgr2=0x7f64800ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:01.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f6485449640 1 --2- 172.21.15.29:0/3799630867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6480153030 0x7f64800ce7e0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f647800cb40 tx=0x7f647800a060 comp rx=0 tx=0).stop 2024-04-25T00:39:01.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f6485449640 1 -- 172.21.15.29:0/3799630867 shutdown_connections 2024-04-25T00:39:01.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f6485449640 1 --2- 172.21.15.29:0/3799630867 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f646c067e90 0x7f646c06a350 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f6485449640 1 --2- 172.21.15.29:0/3799630867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6480153030 0x7f64800ce7e0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f6485449640 1 --2- 172.21.15.29:0/3799630867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64800cd200 0x7f64800b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f6485449640 1 --2- 172.21.15.29:0/3799630867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f648006b750 0x7f64800c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f6485449640 1 -- 172.21.15.29:0/3799630867 >> 172.21.15.29:0/3799630867 conn(0x7f64800d0c10 msgr2=0x7f648015a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:01.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f6485449640 1 -- 172.21.15.29:0/3799630867 shutdown_connections 2024-04-25T00:39:01.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.386+0000 7f6485449640 1 -- 172.21.15.29:0/3799630867 wait complete. 2024-04-25T00:39:01.408 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-04-25T00:39:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe0bf3a640 1 Processor -- start 2024-04-25T00:39:01.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe0bf3a640 1 -- start start 2024-04-25T00:39:01.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe0bf3a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe040d0a90 0x7fbe040b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe0bf3a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe040b17e0 con 0x7fbe0415bf00 2024-04-25T00:39:01.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe0bf3a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe040b1ac0 con 0x7fbe040cc970 2024-04-25T00:39:01.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe0bf3a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbe041600b0 con 0x7fbe040d0a90 2024-04-25T00:39:01.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe09cd6640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbe0415bf00 0x7fbe04160540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59432/0 (socket says 172.21.15.29:59432) 2024-04-25T00:39:01.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe09cd6640 1 -- 172.21.15.29:0/2966815612 learned_addr learned my addr 172.21.15.29:0/2966815612 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3460204342 0 0) 0x7fbe040b17e0 con 0x7fbe0415bf00 2024-04-25T00:39:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe0a4d7640 1 --2- 172.21.15.29:0/2966815612 >> v2:172.21.15.157:3300/0 conn(0x7fbe040d0a90 0x7fbe040b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdf0003530 con 0x7fbe0415bf00 2024-04-25T00:39:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3374110263 0 0) 0x7fbdf0003530 con 0x7fbe0415bf00 2024-04-25T00:39:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 >> v2:172.21.15.157:3300/0 conn(0x7fbe040d0a90 msgr2=0x7fbe040b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 --2- 172.21.15.29:0/2966815612 >> v2:172.21.15.157:3300/0 conn(0x7fbe040d0a90 0x7fbe040b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 >> v1:172.21.15.157:6789/0 conn(0x7fbe040cc970 legacy=0x7fbe040d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:01.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0415fd40 con 0x7fbe0415bf00 2024-04-25T00:39:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fbe00001680 con 0x7fbe0415bf00 2024-04-25T00:39:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 >> v1:172.21.15.29:6789/0 conn(0x7fbe0415bf00 legacy=0x7fbe04160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 --2- 172.21.15.29:0/2966815612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdf0003a10 0x7fbdf0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 --2- 172.21.15.29:0/2966815612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf0004810 0x7fbdf0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 --2- 172.21.15.29:0/2966815612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbdf0005220 0x7fbdf000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbe041600b0 con 0x7fbdf0005220 2024-04-25T00:39:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf000c170 con 0x7fbdf0004810 2024-04-25T00:39:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf000c370 con 0x7fbdf0003a10 2024-04-25T00:39:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe0a4d7640 1 --2- 172.21.15.29:0/2966815612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbdf0005220 0x7fbdf000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:01.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe094d5640 1 --2- 172.21.15.29:0/2966815612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf0004810 0x7fbdf0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe0a4d7640 1 -- 172.21.15.29:0/2966815612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdf0003a10 msgr2=0x7fbdf0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe0a4d7640 1 --2- 172.21.15.29:0/2966815612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdf0003a10 0x7fbdf0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe0a4d7640 1 -- 172.21.15.29:0/2966815612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf0004810 msgr2=0x7fbdf0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe0a4d7640 1 --2- 172.21.15.29:0/2966815612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf0004810 0x7fbdf0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.526+0000 7fbe0a4d7640 1 -- 172.21.15.29:0/2966815612 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf0003530 con 0x7fbdf0005220 2024-04-25T00:39:01.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0a4d7640 1 -- 172.21.15.29:0/2966815612 reap_dead start 2024-04-25T00:39:01.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0a4d7640 1 --2- 172.21.15.29:0/2966815612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbdf0005220 0x7fbdf000b9d0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fbe04072600 tx=0x7fbdfc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2515806838ecff66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:01.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc009c10 con 0x7fbdf0005220 2024-04-25T00:39:01.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdfc0a5c70 con 0x7fbdf0005220 2024-04-25T00:39:01.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe08cd4640 1 -- 172.21.15.29:0/2966815612 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc003be0 con 0x7fbdf0005220 2024-04-25T00:39:01.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/2966815612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbdf0005220 msgr2=0x7fbdf000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:01.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 --2- 172.21.15.29:0/2966815612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbdf0005220 0x7fbdf000b9d0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fbe04072600 tx=0x7fbdfc01cc60 comp rx=0 tx=0).stop 2024-04-25T00:39:01.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/2966815612 shutdown_connections 2024-04-25T00:39:01.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 --2- 172.21.15.29:0/2966815612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbdf0005220 0x7fbdf000b9d0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/2966815612 >> 172.21.15.29:0/2966815612 conn(0x7fbe04072930 msgr2=0x7fbe04072d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:01.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/2966815612 shutdown_connections 2024-04-25T00:39:01.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/2966815612 wait complete. 2024-04-25T00:39:01.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 Processor -- start 2024-04-25T00:39:01.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 -- start start 2024-04-25T00:39:01.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf0005220 0x7fbe040c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe0415b180 0x7fbe0415a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe041e5480 0x7fbe041e5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0415c460 con 0x7fbe041e5480 2024-04-25T00:39:01.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0415c2e0 con 0x7fbdf0005220 2024-04-25T00:39:01.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.530+0000 7fbe0bf3a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbe040b1170 con 0x7fbe0415b180 2024-04-25T00:39:01.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe0a4d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe041e5480 0x7fbe041e5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:01.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe0a4d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe041e5480 0x7fbe041e5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50884/0 (socket says 172.21.15.29:50884) 2024-04-25T00:39:01.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe0a4d7640 1 -- 172.21.15.29:0/3155359687 learned_addr learned my addr 172.21.15.29:0/3155359687 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:01.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe094d5640 1 --2- 172.21.15.29:0/3155359687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe0415b180 0x7fbe0415a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:01.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe0a4d7640 1 -- 172.21.15.29:0/3155359687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe0415b180 msgr2=0x7fbe0415a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:01.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe0a4d7640 1 --2- 172.21.15.29:0/3155359687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe0415b180 0x7fbe0415a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe0a4d7640 1 -- 172.21.15.29:0/3155359687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf0005220 msgr2=0x7fbe040c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:01.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe0a4d7640 1 --2- 172.21.15.29:0/3155359687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf0005220 0x7fbe040c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:01.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe0a4d7640 1 -- 172.21.15.29:0/3155359687 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe040c88f0 con 0x7fbe041e5480 2024-04-25T00:39:01.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe0a4d7640 1 --2- 172.21.15.29:0/3155359687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe041e5480 0x7fbe041e5900 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc098790 tx=0x7fbdfc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:01.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbdfa7fc640 1 -- 172.21.15.29:0/3155359687 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc003e50 con 0x7fbe041e5480 2024-04-25T00:39:01.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbdfa7fc640 1 -- 172.21.15.29:0/3155359687 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdfc0a5a70 con 0x7fbe041e5480 2024-04-25T00:39:01.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/3155359687 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe040c8bd0 con 0x7fbe041e5480 2024-04-25T00:39:01.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbdfa7fc640 1 -- 172.21.15.29:0/3155359687 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc004470 con 0x7fbe041e5480 2024-04-25T00:39:01.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/3155359687 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe040c9190 con 0x7fbe041e5480 2024-04-25T00:39:01.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/3155359687 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe040c9ed0 con 0x7fbe041e5480 2024-04-25T00:39:01.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbdfa7fc640 1 -- 172.21.15.29:0/3155359687 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fbdfc0be020 con 0x7fbe041e5480 2024-04-25T00:39:01.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbdfa7fc640 1 --2- 172.21.15.29:0/3155359687 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbde8067e60 0x7fbde806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:01.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbdfa7fc640 1 -- 172.21.15.29:0/3155359687 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7fbdfc121640 con 0x7fbe041e5480 2024-04-25T00:39:01.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe09cd6640 1 --2- 172.21.15.29:0/3155359687 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbde8067e60 0x7fbde806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:01.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.534+0000 7fbe09cd6640 1 --2- 172.21.15.29:0/3155359687 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbde8067e60 0x7fbde806a320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbe000018b0 tx=0x7fbe00001480 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:01.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.538+0000 7fbdfa7fc640 1 -- 172.21.15.29:0/3155359687 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbe040c9ed0 con 0x7fbe041e5480 2024-04-25T00:39:01.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:01.758+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/3155359687 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7fbe040b6370 con 0x7fbe041e5480 2024-04-25T00:39:02.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.742+0000 7fbdfa7fc640 1 -- 172.21.15.29:0/3155359687 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v403) ==== 154+0+0 (secure 0 0 0) 0x7fbdfc0c5020 con 0x7fbe041e5480 2024-04-25T00:39:02.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.746+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/3155359687 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbde8067e60 msgr2=0x7fbde806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.746+0000 7fbe0bf3a640 1 --2- 172.21.15.29:0/3155359687 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbde8067e60 0x7fbde806a320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbe000018b0 tx=0x7fbe00001480 comp rx=0 tx=0).stop 2024-04-25T00:39:02.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.746+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/3155359687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe041e5480 msgr2=0x7fbe041e5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.746+0000 7fbe0bf3a640 1 --2- 172.21.15.29:0/3155359687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe041e5480 0x7fbe041e5900 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc098790 tx=0x7fbdfc009750 comp rx=0 tx=0).stop 2024-04-25T00:39:02.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.746+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/3155359687 shutdown_connections 2024-04-25T00:39:02.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.746+0000 7fbe0bf3a640 1 --2- 172.21.15.29:0/3155359687 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbde8067e60 0x7fbde806a320 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.746+0000 7fbe0bf3a640 1 --2- 172.21.15.29:0/3155359687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe041e5480 0x7fbe041e5900 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.746+0000 7fbe0bf3a640 1 --2- 172.21.15.29:0/3155359687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe0415b180 0x7fbe0415a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.746+0000 7fbe0bf3a640 1 --2- 172.21.15.29:0/3155359687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf0005220 0x7fbe040c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.746+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/3155359687 >> 172.21.15.29:0/3155359687 conn(0x7fbe04072930 msgr2=0x7fbe0406e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:02.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.746+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/3155359687 shutdown_connections 2024-04-25T00:39:02.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.746+0000 7fbe0bf3a640 1 -- 172.21.15.29:0/3155359687 wait complete. 2024-04-25T00:39:02.772 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:39:02.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a33a92640 1 Processor -- start 2024-04-25T00:39:02.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a33a92640 1 -- start start 2024-04-25T00:39:02.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a33a92640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a2c0715b0 0x7f4a2c071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a33a92640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a2c0b4e60 0x7f4a2c0b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a33a92640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a2c0b4130 con 0x7f4a2c15a570 2024-04-25T00:39:02.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a33a92640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4a2c0748b0 con 0x7f4a2c0b4e60 2024-04-25T00:39:02.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a33a92640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2c071f50 con 0x7f4a2c0715b0 2024-04-25T00:39:02.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3182e640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4a2c15a570 0x7f4a2c0741a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59440/0 (socket says 172.21.15.29:59440) 2024-04-25T00:39:02.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3182e640 1 -- 172.21.15.29:0/420772272 learned_addr learned my addr 172.21.15.29:0/420772272 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:02.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3102d640 1 --2- 172.21.15.29:0/420772272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a2c0715b0 0x7f4a2c071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3202f640 1 --2- 172.21.15.29:0/420772272 >> v2:172.21.15.157:3300/0 conn(0x7f4a2c0b4e60 0x7f4a2c0b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 708370941 0 0) 0x7f4a2c0b4130 con 0x7f4a2c15a570 2024-04-25T00:39:02.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a18003530 con 0x7f4a2c15a570 2024-04-25T00:39:02.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3459019157 0 0) 0x7f4a18003530 con 0x7f4a2c15a570 2024-04-25T00:39:02.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a2c0715b0 msgr2=0x7f4a2c071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:02.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 --2- 172.21.15.29:0/420772272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a2c0715b0 0x7f4a2c071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 >> v2:172.21.15.157:3300/0 conn(0x7f4a2c0b4e60 msgr2=0x7f4a2c0b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:02.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 --2- 172.21.15.29:0/420772272 >> v2:172.21.15.157:3300/0 conn(0x7f4a2c0b4e60 0x7f4a2c0b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2c0b4410 con 0x7f4a2c15a570 2024-04-25T00:39:02.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4a24001a00 con 0x7f4a2c15a570 2024-04-25T00:39:02.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 >> v1:172.21.15.29:6789/0 conn(0x7f4a2c15a570 legacy=0x7f4a2c0741a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 --2- 172.21.15.29:0/420772272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180039f0 0x7f4a18003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 --2- 172.21.15.29:0/420772272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a18004810 0x7f4a18004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 --2- 172.21.15.29:0/420772272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a2c0715b0 0x7f4a180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c0748b0 con 0x7f4a180039f0 2024-04-25T00:39:02.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1800b9d0 con 0x7f4a18004810 2024-04-25T00:39:02.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1800bb50 con 0x7f4a2c0715b0 2024-04-25T00:39:02.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3182e640 1 --2- 172.21.15.29:0/420772272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180039f0 0x7f4a18003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3102d640 1 --2- 172.21.15.29:0/420772272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a18004810 0x7f4a18004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3182e640 1 -- 172.21.15.29:0/420772272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a2c0715b0 msgr2=0x7f4a180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:02.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3182e640 1 --2- 172.21.15.29:0/420772272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a2c0715b0 0x7f4a180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3182e640 1 -- 172.21.15.29:0/420772272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a18004810 msgr2=0x7f4a18004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:02.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3182e640 1 --2- 172.21.15.29:0/420772272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a18004810 0x7f4a18004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.902+0000 7f4a3182e640 1 -- 172.21.15.29:0/420772272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a18003530 con 0x7f4a180039f0 2024-04-25T00:39:02.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.906+0000 7f4a3182e640 1 --2- 172.21.15.29:0/420772272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180039f0 0x7f4a18003e10 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400a840 tx=0x7f4a2401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c862229e241839b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:02.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.906+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2400a460 con 0x7f4a180039f0 2024-04-25T00:39:02.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.906+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a240ad650 con 0x7f4a180039f0 2024-04-25T00:39:02.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.906+0000 7f4a3082c640 1 -- 172.21.15.29:0/420772272 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a240ada40 con 0x7f4a180039f0 2024-04-25T00:39:02.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.906+0000 7f4a33a92640 1 -- 172.21.15.29:0/420772272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180039f0 msgr2=0x7f4a18003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.906+0000 7f4a33a92640 1 --2- 172.21.15.29:0/420772272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a180039f0 0x7f4a18003e10 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400a840 tx=0x7f4a2401d360 comp rx=0 tx=0).stop 2024-04-25T00:39:02.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.906+0000 7f4a3202f640 1 -- 172.21.15.29:0/420772272 reap_dead start 2024-04-25T00:39:02.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.906+0000 7f4a33a92640 1 -- 172.21.15.29:0/420772272 shutdown_connections 2024-04-25T00:39:02.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.906+0000 7f4a33a92640 1 -- 172.21.15.29:0/420772272 >> 172.21.15.29:0/420772272 conn(0x7f4a2c075030 msgr2=0x7f4a2c0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:02.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.906+0000 7f4a33a92640 1 -- 172.21.15.29:0/420772272 shutdown_connections 2024-04-25T00:39:02.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.906+0000 7f4a33a92640 1 -- 172.21.15.29:0/420772272 wait complete. 2024-04-25T00:39:02.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.906+0000 7f4a33a92640 1 Processor -- start 2024-04-25T00:39:02.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a33a92640 1 -- start start 2024-04-25T00:39:02.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a33a92640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180039f0 0x7f4a2c0aa620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a33a92640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a2c0ab010 0x7f4a2c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a33a92640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a2c0b3800 0x7f4a2c0ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a33a92640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c0cf4f0 con 0x7f4a2c0ab010 2024-04-25T00:39:02.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a33a92640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c071da0 con 0x7f4a180039f0 2024-04-25T00:39:02.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a33a92640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2c0cf670 con 0x7f4a2c0b3800 2024-04-25T00:39:02.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a3102d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a2c0ab010 0x7f4a2c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a3102d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a2c0ab010 0x7f4a2c0b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50902/0 (socket says 172.21.15.29:50902) 2024-04-25T00:39:02.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a3102d640 1 -- 172.21.15.29:0/1238715061 learned_addr learned my addr 172.21.15.29:0/1238715061 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:02.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a3182e640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180039f0 0x7f4a2c0aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a3202f640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a2c0b3800 0x7f4a2c0ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a3102d640 1 -- 172.21.15.29:0/1238715061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a2c0b3800 msgr2=0x7f4a2c0ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a3102d640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a2c0b3800 0x7f4a2c0ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a3102d640 1 -- 172.21.15.29:0/1238715061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180039f0 msgr2=0x7f4a2c0aa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:02.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a3102d640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180039f0 0x7f4a2c0aa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:02.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a3102d640 1 -- 172.21.15.29:0/1238715061 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2c0a95c0 con 0x7f4a2c0ab010 2024-04-25T00:39:02.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a3182e640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180039f0 0x7f4a2c0aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:02.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a3102d640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a2c0ab010 0x7f4a2c0b32c0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c011560 tx=0x7f4a1c011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:02.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a227fc640 1 -- 172.21.15.29:0/1238715061 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c015930 con 0x7f4a2c0ab010 2024-04-25T00:39:02.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a227fc640 1 -- 172.21.15.29:0/1238715061 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a1c015ad0 con 0x7f4a2c0ab010 2024-04-25T00:39:02.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a33a92640 1 -- 172.21.15.29:0/1238715061 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a2c0a9860 con 0x7f4a2c0ab010 2024-04-25T00:39:02.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a227fc640 1 -- 172.21.15.29:0/1238715061 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c00f460 con 0x7f4a2c0ab010 2024-04-25T00:39:02.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a33a92640 1 -- 172.21.15.29:0/1238715061 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a2c0a9e20 con 0x7f4a2c0ab010 2024-04-25T00:39:02.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a33a92640 1 -- 172.21.15.29:0/1238715061 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a2c069270 con 0x7f4a2c0ab010 2024-04-25T00:39:02.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a227fc640 1 -- 172.21.15.29:0/1238715061 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f4a1c00f600 con 0x7f4a2c0ab010 2024-04-25T00:39:02.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a227fc640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4a10067e60 0x7f4a1006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:02.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a227fc640 1 -- 172.21.15.29:0/1238715061 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7f4a1c086cc0 con 0x7f4a2c0ab010 2024-04-25T00:39:02.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.910+0000 7f4a3182e640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4a10067e60 0x7f4a1006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:02.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.914+0000 7f4a3182e640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4a10067e60 0x7f4a1006a320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4a240ad000 tx=0x7f4a2c0b49d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:02.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:02.918+0000 7f4a227fc640 1 -- 172.21.15.29:0/1238715061 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4a1c019030 con 0x7f4a2c0ab010 2024-04-25T00:39:03.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.078+0000 7f4a33a92640 1 -- 172.21.15.29:0/1238715061 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4a2c0af790 con 0x7f4a2c0ab010 2024-04-25T00:39:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.082+0000 7f4a227fc640 1 -- 172.21.15.29:0/1238715061 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f4a1c057320 con 0x7f4a2c0ab010 2024-04-25T00:39:03.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.086+0000 7f4a33a92640 1 -- 172.21.15.29:0/1238715061 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4a10067e60 msgr2=0x7f4a1006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:03.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.086+0000 7f4a33a92640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4a10067e60 0x7f4a1006a320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4a240ad000 tx=0x7f4a2c0b49d0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.086+0000 7f4a33a92640 1 -- 172.21.15.29:0/1238715061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a2c0ab010 msgr2=0x7f4a2c0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:03.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.086+0000 7f4a33a92640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a2c0ab010 0x7f4a2c0b32c0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c011560 tx=0x7f4a1c011a20 comp rx=0 tx=0).stop 2024-04-25T00:39:03.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.086+0000 7f4a33a92640 1 -- 172.21.15.29:0/1238715061 shutdown_connections 2024-04-25T00:39:03.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.086+0000 7f4a33a92640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4a10067e60 0x7f4a1006a320 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.086+0000 7f4a33a92640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a2c0b3800 0x7f4a2c0ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.086+0000 7f4a33a92640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a2c0ab010 0x7f4a2c0b32c0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.086+0000 7f4a33a92640 1 --2- 172.21.15.29:0/1238715061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180039f0 0x7f4a2c0aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.086+0000 7f4a33a92640 1 -- 172.21.15.29:0/1238715061 >> 172.21.15.29:0/1238715061 conn(0x7f4a2c075030 msgr2=0x7f4a2c157340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:03.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.086+0000 7f4a33a92640 1 -- 172.21.15.29:0/1238715061 shutdown_connections 2024-04-25T00:39:03.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.086+0000 7f4a33a92640 1 -- 172.21.15.29:0/1238715061 wait complete. 2024-04-25T00:39:03.107 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:39:03.107 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:39:03.149 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:39:03.216 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:39:03.216 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:39:03.216 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:39:03.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:39:03.221 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:39:03.221 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:39:03.227 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:39:03.227 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:39:03.227 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:39:03.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:39:03.268 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:39:03.268 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:39:03.278 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:39:03.278 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:39:03.279 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:39:03.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705f9db640 1 Processor -- start 2024-04-25T00:39:03.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705f9db640 1 -- start start 2024-04-25T00:39:03.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705f9db640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7058162730 0x7f705806b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:03.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705f9db640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f705806b750 0x7f70580ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:03.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705f9db640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70580cba50 con 0x7f705815df40 2024-04-25T00:39:03.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705f9db640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f70580addc0 con 0x7f705806b750 2024-04-25T00:39:03.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705f9db640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f70580cbd30 con 0x7f7058162730 2024-04-25T00:39:03.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705df78640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f705806b750 0x7f70580ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:03.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705d777640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f705815df40 0x7f70580cb260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59446/0 (socket says 172.21.15.29:59446) 2024-04-25T00:39:03.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705d777640 1 -- 172.21.15.29:0/880371820 learned_addr learned my addr 172.21.15.29:0/880371820 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:03.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705cf76640 1 --2- 172.21.15.29:0/880371820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7058162730 0x7f705806b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:03.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705df78640 1 -- 172.21.15.29:0/880371820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7058162730 msgr2=0x7f705806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:03.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705df78640 1 --2- 172.21.15.29:0/880371820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7058162730 0x7f705806b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705df78640 1 -- 172.21.15.29:0/880371820 >> v1:172.21.15.29:6789/0 conn(0x7f705815df40 legacy=0x7f70580cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:03.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f705df78640 1 -- 172.21.15.29:0/880371820 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70580ada50 con 0x7f705806b750 2024-04-25T00:39:03.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.438+0000 7f704ffff640 1 -- 172.21.15.29:0/880371820 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 176377288 0 0) 0x7f70580cba50 con 0x7f705815df40 2024-04-25T00:39:03.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f705df78640 1 --2- 172.21.15.29:0/880371820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f705806b750 0x7f70580ae250 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f7050009d90 tx=0x7f705001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0e4b0b208904b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f704ffff640 1 -- 172.21.15.29:0/880371820 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70500b2070 con 0x7f705806b750 2024-04-25T00:39:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f704ffff640 1 -- 172.21.15.29:0/880371820 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70500a57a0 con 0x7f705806b750 2024-04-25T00:39:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f704ffff640 1 -- 172.21.15.29:0/880371820 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70500a5b70 con 0x7f705806b750 2024-04-25T00:39:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f705f9db640 1 -- 172.21.15.29:0/880371820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f705806b750 msgr2=0x7f70580ae250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f705f9db640 1 --2- 172.21.15.29:0/880371820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f705806b750 0x7f70580ae250 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f7050009d90 tx=0x7f705001c8e0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f705f9db640 1 -- 172.21.15.29:0/880371820 shutdown_connections 2024-04-25T00:39:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f705f9db640 1 --2- 172.21.15.29:0/880371820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f705806b750 0x7f70580ae250 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f705f9db640 1 --2- 172.21.15.29:0/880371820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7058162730 0x7f705806b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f705f9db640 1 -- 172.21.15.29:0/880371820 >> 172.21.15.29:0/880371820 conn(0x7f70580d0c10 msgr2=0x7f70580d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:03.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f705f9db640 1 -- 172.21.15.29:0/880371820 shutdown_connections 2024-04-25T00:39:03.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f705f9db640 1 -- 172.21.15.29:0/880371820 wait complete. 2024-04-25T00:39:03.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f705f9db640 1 Processor -- start 2024-04-25T00:39:03.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.442+0000 7f705f9db640 1 -- start start 2024-04-25T00:39:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705f9db640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f705806b750 0x7f705815c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705f9db640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f705815df40 0x7f705806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705f9db640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7058162730 0x7f7058157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705f9db640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f70580af330 con 0x7f7058162730 2024-04-25T00:39:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705f9db640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f70580af1b0 con 0x7f705815df40 2024-04-25T00:39:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705f9db640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f70580af4b0 con 0x7f705806b750 2024-04-25T00:39:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705df78640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7058162730 0x7f7058157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705df78640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7058162730 0x7f7058157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50922/0 (socket says 172.21.15.29:50922) 2024-04-25T00:39:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705df78640 1 -- 172.21.15.29:0/4036536127 learned_addr learned my addr 172.21.15.29:0/4036536127 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705d777640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f705806b750 0x7f705815c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705cf76640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f705815df40 0x7f705806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705df78640 1 -- 172.21.15.29:0/4036536127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f705806b750 msgr2=0x7f705815c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705df78640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f705806b750 0x7f705815c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705df78640 1 -- 172.21.15.29:0/4036536127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f705815df40 msgr2=0x7f705806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705df78640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f705815df40 0x7f705806ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705df78640 1 -- 172.21.15.29:0/4036536127 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705815cf30 con 0x7f7058162730 2024-04-25T00:39:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705cf76640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f705815df40 0x7f705806ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705df78640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7058162730 0x7f7058157db0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f7050009d90 tx=0x7f7050019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f704dffb640 1 -- 172.21.15.29:0/4036536127 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70500b2040 con 0x7f7058162730 2024-04-25T00:39:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f704dffb640 1 -- 172.21.15.29:0/4036536127 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70500bf070 con 0x7f7058162730 2024-04-25T00:39:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705f9db640 1 -- 172.21.15.29:0/4036536127 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7058150590 con 0x7f7058162730 2024-04-25T00:39:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f704dffb640 1 -- 172.21.15.29:0/4036536127 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050009450 con 0x7f7058162730 2024-04-25T00:39:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705f9db640 1 -- 172.21.15.29:0/4036536127 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705815b280 con 0x7f7058162730 2024-04-25T00:39:03.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705f9db640 1 -- 172.21.15.29:0/4036536127 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7058069270 con 0x7f7058162730 2024-04-25T00:39:03.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f704dffb640 1 -- 172.21.15.29:0/4036536127 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f70500ada30 con 0x7f7058162730 2024-04-25T00:39:03.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f704dffb640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f703c067e60 0x7f703c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:03.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f704dffb640 1 -- 172.21.15.29:0/4036536127 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7f70501217a0 con 0x7f7058162730 2024-04-25T00:39:03.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705d777640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f703c067e60 0x7f703c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:03.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.446+0000 7f705d777640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f703c067e60 0x7f703c06a320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7048000f70 tx=0x7f7048007b60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:03.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.450+0000 7f704dffb640 1 -- 172.21.15.29:0/4036536127 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7050016560 con 0x7f7058162730 2024-04-25T00:39:03.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.650+0000 7f705f9db640 1 -- 172.21.15.29:0/4036536127 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7058072a50 con 0x7f7058162730 2024-04-25T00:39:03.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.650+0000 7f704dffb640 1 -- 172.21.15.29:0/4036536127 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+3872 (secure 0 0 0) 0x7f70500f1e00 con 0x7f7058162730 2024-04-25T00:39:03.654 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:03.654 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":78,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2148970077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2148970077},{"type":"v1","addr":"172.21.15.157:6834","nonce":2148970077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3947933400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3947933400},{"type":"v1","addr":"172.21.15.157:6835","nonce":3947933400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/172449010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":172449010},{"type":"v1","addr":"172.21.15.29:6833","nonce":172449010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8931,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3929029988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3929029988},{"type":"v1","addr":"172.21.15.29:6835","nonce":3929029988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[]} 2024-04-25T00:39:03.654 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 78 2024-04-25T00:39:03.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.654+0000 7f705f9db640 1 -- 172.21.15.29:0/4036536127 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f703c067e60 msgr2=0x7f703c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:03.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.654+0000 7f705f9db640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f703c067e60 0x7f703c06a320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7048000f70 tx=0x7f7048007b60 comp rx=0 tx=0).stop 2024-04-25T00:39:03.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.654+0000 7f705f9db640 1 -- 172.21.15.29:0/4036536127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7058162730 msgr2=0x7f7058157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:03.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.654+0000 7f705f9db640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7058162730 0x7f7058157db0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f7050009d90 tx=0x7f7050019680 comp rx=0 tx=0).stop 2024-04-25T00:39:03.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.654+0000 7f705f9db640 1 -- 172.21.15.29:0/4036536127 shutdown_connections 2024-04-25T00:39:03.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.654+0000 7f705f9db640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f703c067e60 0x7f703c06a320 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.654+0000 7f705f9db640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7058162730 0x7f7058157db0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.654+0000 7f705f9db640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f705815df40 0x7f705806ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.654+0000 7f705f9db640 1 --2- 172.21.15.29:0/4036536127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f705806b750 0x7f705815c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.654+0000 7f705f9db640 1 -- 172.21.15.29:0/4036536127 >> 172.21.15.29:0/4036536127 conn(0x7f70580d0c10 msgr2=0x7f7058067160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:03.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.654+0000 7f705f9db640 1 -- 172.21.15.29:0/4036536127 shutdown_connections 2024-04-25T00:39:03.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.654+0000 7f705f9db640 1 -- 172.21.15.29:0/4036536127 wait complete. 2024-04-25T00:39:03.674 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:39:03.674 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:39:03.674 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:39:03.674 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:39:03.674 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:39:03.675 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:39:03.675 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:39:03.675 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:39:03.675 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:39:03.675 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:39:03.675 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:39:03.676 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:39:03.676 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:39:03.676 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:39:03.676 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:39:03.676 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:39:03.685 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:39:03.685 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:39:03.686 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:39:03.686 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:39:03.689 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:39:03.689 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:39:03.694 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:39:03.694 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:39:03.733 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:39:03.733 DEBUG:teuthology.parallel:result is None 2024-04-25T00:39:03.734 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:39:03.734 DEBUG:teuthology.parallel:result is None 2024-04-25T00:39:03.743 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:39:03.743 DEBUG:teuthology.parallel:result is None 2024-04-25T00:39:03.744 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:39:03.745 DEBUG:teuthology.parallel:result is None 2024-04-25T00:39:03.745 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:39:03.797 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:39:03.804 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:39:03.814 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-25T00:39:03.817 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-25T00:39:03.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f307c5df640 1 Processor -- start 2024-04-25T00:39:03.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f307c5df640 1 -- start start 2024-04-25T00:39:03.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f307c5df640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f307415f5a0 0x7f30740d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:03.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f307c5df640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30740d0300 0x7f30740d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:03.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f307c5df640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30740cdcc0 con 0x7f30740cf9f0 2024-04-25T00:39:03.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f307c5df640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f30740d0cb0 con 0x7f30740d0300 2024-04-25T00:39:03.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f307c5df640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f30740cc8a0 con 0x7f307415f5a0 2024-04-25T00:39:03.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f3079b7a640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f30740cf9f0 0x7f30740cd5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59470/0 (socket says 172.21.15.29:59470) 2024-04-25T00:39:03.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f3079b7a640 1 -- 172.21.15.29:0/1668896125 learned_addr learned my addr 172.21.15.29:0/1668896125 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:03.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f307ab7c640 1 --2- 172.21.15.29:0/1668896125 >> v2:172.21.15.29:3300/0 conn(0x7f30740d0300 0x7f30740d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:03.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f307a37b640 1 --2- 172.21.15.29:0/1668896125 >> v2:172.21.15.157:3300/0 conn(0x7f307415f5a0 0x7f30740d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:03.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f3079379640 1 -- 172.21.15.29:0/1668896125 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2779687585 0 0) 0x7f30740cdcc0 con 0x7f30740cf9f0 2024-04-25T00:39:03.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f3079379640 1 -- 172.21.15.29:0/1668896125 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3060003530 con 0x7f30740cf9f0 2024-04-25T00:39:03.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f307ab7c640 1 -- 172.21.15.29:0/1668896125 >> v2:172.21.15.157:3300/0 conn(0x7f307415f5a0 msgr2=0x7f30740d2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:03.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f307ab7c640 1 --2- 172.21.15.29:0/1668896125 >> v2:172.21.15.157:3300/0 conn(0x7f307415f5a0 0x7f30740d2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f307ab7c640 1 -- 172.21.15.29:0/1668896125 >> v1:172.21.15.29:6789/0 conn(0x7f30740cf9f0 legacy=0x7f30740cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:03.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.862+0000 7f307ab7c640 1 -- 172.21.15.29:0/1668896125 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307415f980 con 0x7f30740d0300 2024-04-25T00:39:03.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.866+0000 7f307ab7c640 1 --2- 172.21.15.29:0/1668896125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30740d0300 0x7f30740d0770 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f30740720a0 tx=0x7f306c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beab10cea70f999b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:03.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.866+0000 7f3079379640 1 -- 172.21.15.29:0/1668896125 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c0b5070 con 0x7f30740d0300 2024-04-25T00:39:03.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.866+0000 7f3079379640 1 -- 172.21.15.29:0/1668896125 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306c0a8970 con 0x7f30740d0300 2024-04-25T00:39:03.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.866+0000 7f3079379640 1 -- 172.21.15.29:0/1668896125 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c0a8d40 con 0x7f30740d0300 2024-04-25T00:39:03.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.866+0000 7f307c5df640 1 -- 172.21.15.29:0/1668896125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30740d0300 msgr2=0x7f30740d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:03.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.866+0000 7f307c5df640 1 --2- 172.21.15.29:0/1668896125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30740d0300 0x7f30740d0770 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f30740720a0 tx=0x7f306c01ec70 comp rx=0 tx=0).stop 2024-04-25T00:39:03.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.866+0000 7f307c5df640 1 -- 172.21.15.29:0/1668896125 shutdown_connections 2024-04-25T00:39:03.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.866+0000 7f307c5df640 1 --2- 172.21.15.29:0/1668896125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30740d0300 0x7f30740d0770 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.866+0000 7f307c5df640 1 --2- 172.21.15.29:0/1668896125 >> v2:172.21.15.157:3300/0 conn(0x7f307415f5a0 0x7f30740d2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.866+0000 7f307c5df640 1 -- 172.21.15.29:0/1668896125 >> 172.21.15.29:0/1668896125 conn(0x7f30740d1d20 msgr2=0x7f30740d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:03.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.866+0000 7f307c5df640 1 -- 172.21.15.29:0/1668896125 shutdown_connections 2024-04-25T00:39:03.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.866+0000 7f307c5df640 1 -- 172.21.15.29:0/1668896125 wait complete. 2024-04-25T00:39:03.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307c5df640 1 Processor -- start 2024-04-25T00:39:03.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307c5df640 1 -- start start 2024-04-25T00:39:03.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307c5df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30740cf9f0 0x7f30741ef060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:03.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307c5df640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30740d0300 0x7f30741f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:03.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307c5df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307415f5a0 0x7f307409d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:03.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307c5df640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30740ccdb0 con 0x7f30740d0300 2024-04-25T00:39:03.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307c5df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f30740ccad0 con 0x7f30740cf9f0 2024-04-25T00:39:03.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307c5df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f30740cd090 con 0x7f307415f5a0 2024-04-25T00:39:03.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f3079b7a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30740d0300 0x7f30741f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:03.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f3079b7a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30740d0300 0x7f30741f24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50966/0 (socket says 172.21.15.29:50966) 2024-04-25T00:39:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f3079b7a640 1 -- 172.21.15.29:0/4190664440 learned_addr learned my addr 172.21.15.29:0/4190664440 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307a37b640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30740cf9f0 0x7f30741ef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307ab7c640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307415f5a0 0x7f307409d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f3079b7a640 1 -- 172.21.15.29:0/4190664440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307415f5a0 msgr2=0x7f307409d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f3079b7a640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307415f5a0 0x7f307409d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f3079b7a640 1 -- 172.21.15.29:0/4190664440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30740cf9f0 msgr2=0x7f30741ef060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f3079b7a640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30740cf9f0 0x7f30741ef060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f3079b7a640 1 -- 172.21.15.29:0/4190664440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30741f2a20 con 0x7f30740d0300 2024-04-25T00:39:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307a37b640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30740cf9f0 0x7f30741ef060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307ab7c640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307415f5a0 0x7f307409d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f3079b7a640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30740d0300 0x7f30741f24e0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f306400c450 tx=0x7f306400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f306affd640 1 -- 172.21.15.29:0/4190664440 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064017070 con 0x7f30740d0300 2024-04-25T00:39:03.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f306affd640 1 -- 172.21.15.29:0/4190664440 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3064005cf0 con 0x7f30740d0300 2024-04-25T00:39:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307c5df640 1 -- 172.21.15.29:0/4190664440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307409bd20 con 0x7f30740d0300 2024-04-25T00:39:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f306affd640 1 -- 172.21.15.29:0/4190664440 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064007290 con 0x7f30740d0300 2024-04-25T00:39:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307c5df640 1 -- 172.21.15.29:0/4190664440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f307409c2e0 con 0x7f30740d0300 2024-04-25T00:39:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307c5df640 1 -- 172.21.15.29:0/4190664440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3074069270 con 0x7f30740d0300 2024-04-25T00:39:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f306affd640 1 -- 172.21.15.29:0/4190664440 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f306401a070 con 0x7f30740d0300 2024-04-25T00:39:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f306affd640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3058067e90 0x7f305806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f307a37b640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3058067e90 0x7f305806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.870+0000 7f306affd640 1 -- 172.21.15.29:0/4190664440 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4943+0+0 (secure 0 0 0) 0x7f3064085750 con 0x7f30740d0300 2024-04-25T00:39:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.874+0000 7f307a37b640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3058067e90 0x7f305806a350 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3070007440 tx=0x7f307000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:03.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:03.874+0000 7f306affd640 1 -- 172.21.15.29:0/4190664440 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f30741f2a20 con 0x7f30740d0300 2024-04-25T00:39:03.896 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T00:39:03.896 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T00:39:04.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:04.050+0000 7f307c5df640 1 -- 172.21.15.29:0/4190664440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f30740cc060 con 0x7f30740d0300 2024-04-25T00:39:05.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.106+0000 7f306affd640 1 -- 172.21.15.29:0/4190664440 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v128) ==== 97+0+0 (secure 0 0 0) 0x7f3064055db0 con 0x7f30740d0300 2024-04-25T00:39:05.110 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-04-25T00:39:05.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.110+0000 7f307c5df640 1 -- 172.21.15.29:0/4190664440 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3058067e90 msgr2=0x7f305806a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.110+0000 7f307c5df640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3058067e90 0x7f305806a350 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3070007440 tx=0x7f307000b040 comp rx=0 tx=0).stop 2024-04-25T00:39:05.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.110+0000 7f307c5df640 1 -- 172.21.15.29:0/4190664440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30740d0300 msgr2=0x7f30741f24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.110+0000 7f307c5df640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30740d0300 0x7f30741f24e0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f306400c450 tx=0x7f306400c910 comp rx=0 tx=0).stop 2024-04-25T00:39:05.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.110+0000 7f307c5df640 1 -- 172.21.15.29:0/4190664440 shutdown_connections 2024-04-25T00:39:05.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.110+0000 7f307c5df640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3058067e90 0x7f305806a350 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.110+0000 7f307c5df640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307415f5a0 0x7f307409d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.110+0000 7f307c5df640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30740d0300 0x7f30741f24e0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.110+0000 7f307c5df640 1 --2- 172.21.15.29:0/4190664440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30740cf9f0 0x7f30741ef060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.110+0000 7f307c5df640 1 -- 172.21.15.29:0/4190664440 >> 172.21.15.29:0/4190664440 conn(0x7f30740d1d20 msgr2=0x7f307415b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:05.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.110+0000 7f307c5df640 1 -- 172.21.15.29:0/4190664440 shutdown_connections 2024-04-25T00:39:05.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.110+0000 7f307c5df640 1 -- 172.21.15.29:0/4190664440 wait complete. 2024-04-25T00:39:05.135 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:39:05.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f70105cc640 1 Processor -- start 2024-04-25T00:39:05.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f70105cc640 1 -- start start 2024-04-25T00:39:05.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f70105cc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f70080b3140 0x7f700806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f70105cc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70080b4b10 0x7f70080b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f70105cc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70080b41e0 con 0x7f70080b3a60 2024-04-25T00:39:05.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f70105cc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f700806d960 con 0x7f70080b4b10 2024-04-25T00:39:05.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f70105cc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f700806dc40 con 0x7f70080b3140 2024-04-25T00:39:05.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f700db67640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70080b4b10 0x7f70080b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f700db67640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70080b4b10 0x7f70080b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50968/0 (socket says 172.21.15.29:50968) 2024-04-25T00:39:05.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f700db67640 1 -- 172.21.15.29:0/3966556014 learned_addr learned my addr 172.21.15.29:0/3966556014 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:05.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f700db67640 1 -- 172.21.15.29:0/3966556014 >> v2:172.21.15.157:3300/0 conn(0x7f70080b3140 msgr2=0x7f700806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:05.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f700db67640 1 --2- 172.21.15.29:0/3966556014 >> v2:172.21.15.157:3300/0 conn(0x7f70080b3140 0x7f700806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f700db67640 1 -- 172.21.15.29:0/3966556014 >> v1:172.21.15.29:6789/0 conn(0x7f70080b3a60 legacy=0x7f700815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.262+0000 7f700db67640 1 -- 172.21.15.29:0/3966556014 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700806df20 con 0x7f70080b4b10 2024-04-25T00:39:05.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700db67640 1 --2- 172.21.15.29:0/3966556014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70080b4b10 0x7f70080b3520 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009870 tx=0x7f6ff801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426268478bd005dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:05.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700d366640 1 -- 172.21.15.29:0/3966556014 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff80b2070 con 0x7f70080b4b10 2024-04-25T00:39:05.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700d366640 1 -- 172.21.15.29:0/3966556014 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff80ad640 con 0x7f70080b4b10 2024-04-25T00:39:05.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700d366640 1 -- 172.21.15.29:0/3966556014 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff80ada10 con 0x7f70080b4b10 2024-04-25T00:39:05.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 -- 172.21.15.29:0/3966556014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70080b4b10 msgr2=0x7f70080b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 --2- 172.21.15.29:0/3966556014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70080b4b10 0x7f70080b3520 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009870 tx=0x7f6ff801c3b0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 -- 172.21.15.29:0/3966556014 shutdown_connections 2024-04-25T00:39:05.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 --2- 172.21.15.29:0/3966556014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70080b4b10 0x7f70080b3520 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 --2- 172.21.15.29:0/3966556014 >> v2:172.21.15.157:3300/0 conn(0x7f70080b3140 0x7f700806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 -- 172.21.15.29:0/3966556014 >> 172.21.15.29:0/3966556014 conn(0x7f70080b1870 msgr2=0x7f70080b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:05.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 -- 172.21.15.29:0/3966556014 shutdown_connections 2024-04-25T00:39:05.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 -- 172.21.15.29:0/3966556014 wait complete. 2024-04-25T00:39:05.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 Processor -- start 2024-04-25T00:39:05.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 -- start start 2024-04-25T00:39:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70080b3140 0x7f7008152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70080b3a60 0x7f7008153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70080b4b10 0x7f7008153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7008145910 con 0x7f70080b3a60 2024-04-25T00:39:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f70080b3ed0 con 0x7f70080b4b10 2024-04-25T00:39:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f70105cc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7008145a90 con 0x7f70080b3140 2024-04-25T00:39:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700db67640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70080b3a60 0x7f7008153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700db67640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70080b3a60 0x7f7008153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50976/0 (socket says 172.21.15.29:50976) 2024-04-25T00:39:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700db67640 1 -- 172.21.15.29:0/3642778149 learned_addr learned my addr 172.21.15.29:0/3642778149 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700eb69640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70080b4b10 0x7f7008153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700e368640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70080b3140 0x7f7008152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700db67640 1 -- 172.21.15.29:0/3642778149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70080b3140 msgr2=0x7f7008152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700db67640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70080b3140 0x7f7008152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700db67640 1 -- 172.21.15.29:0/3642778149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70080b4b10 msgr2=0x7f7008153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700db67640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70080b4b10 0x7f7008153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700db67640 1 -- 172.21.15.29:0/3642778149 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7008136f70 con 0x7f70080b3a60 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.266+0000 7f700db67640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70080b3a60 0x7f7008153570 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80b0040 tx=0x7f6ff8019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.270+0000 7f6ffeffd640 1 -- 172.21.15.29:0/3642778149 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff80b2040 con 0x7f70080b3a60 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.270+0000 7f6ffeffd640 1 -- 172.21.15.29:0/3642778149 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff80bd070 con 0x7f70080b3a60 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.270+0000 7f70105cc640 1 -- 172.21.15.29:0/3642778149 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7008153ff0 con 0x7f70080b3a60 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.270+0000 7f6ffeffd640 1 -- 172.21.15.29:0/3642778149 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff80abbf0 con 0x7f70080b3a60 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.270+0000 7f70105cc640 1 -- 172.21.15.29:0/3642778149 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7008154580 con 0x7f70080b3a60 2024-04-25T00:39:05.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.270+0000 7f70105cc640 1 -- 172.21.15.29:0/3642778149 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7008069270 con 0x7f70080b3a60 2024-04-25T00:39:05.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.270+0000 7f6ffeffd640 1 -- 172.21.15.29:0/3642778149 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f6ff80c2460 con 0x7f70080b3a60 2024-04-25T00:39:05.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.270+0000 7f6ffeffd640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6fec067e60 0x7f6fec06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.270+0000 7f6ffeffd640 1 -- 172.21.15.29:0/3642778149 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4857+0+0 (secure 0 0 0) 0x7f6ff8120f10 con 0x7f70080b3a60 2024-04-25T00:39:05.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.270+0000 7f700e368640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6fec067e60 0x7f6fec06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.270+0000 7f700e368640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6fec067e60 0x7f6fec06a320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7004006fd0 tx=0x7f7004008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:05.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.274+0000 7f6ffeffd640 1 -- 172.21.15.29:0/3642778149 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6ff8016070 con 0x7f70080b3a60 2024-04-25T00:39:05.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.494+0000 7f70105cc640 1 -- 172.21.15.29:0/3642778149 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f700806edb0 con 0x7f70080b3a60 2024-04-25T00:39:05.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.494+0000 7f6ffeffd640 1 -- 172.21.15.29:0/3642778149 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5304 (secure 0 0 0) 0x7f6ff80c0360 con 0x7f70080b3a60 2024-04-25T00:39:05.498 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:05.499 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:39:05.499 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-04-25T00:39:05.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.499 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-04-25T00:39:05.499 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmyzgKAxAAU+nrUJgQhDX6xlARNFI/ng==", 2024-04-25T00:39:05.499 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.499 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:05.499 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:39:05.499 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:39:05.499 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.499 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.500 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-04-25T00:39:05.500 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmtyzPERAAxNUDANw1H/ZzOo+dhgzxlg==", 2024-04-25T00:39:05.500 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.500 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:05.500 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:39:05.500 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:39:05.500 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.500 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.500 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-04-25T00:39:05.500 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmYUl8CxAAj6pV1+s4a0rpWLq2EgZ+8w==", 2024-04-25T00:39:05.500 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmuAhKGhAAOLIcWUodW5TqtVgHWZl12A==", 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:39:05.501 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.502 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.502 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-04-25T00:39:05.502 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAzpClmudk6MhAA6YFyd57jwtB2pdOaad7bxQ==", 2024-04-25T00:39:05.502 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.502 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:05.502 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:05.502 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.502 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.502 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-04-25T00:39:05.503 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA2pClm76q0NRAAxWhEP2ZGoFnZcuiQyGjglw==", 2024-04-25T00:39:05.503 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.503 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:05.503 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:05.503 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.503 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.503 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-04-25T00:39:05.503 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA6pClmcgULABAA2F5WBR9Df0v4Sn875AzF7g==", 2024-04-25T00:39:05.503 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.503 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-04-25T00:39:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA9pClm9QuRBhAALFLruv4JU/uNZYMX0yNuXw==", 2024-04-25T00:39:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-04-25T00:39:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBBpClmNj4VDBAAxaxMvyaCvjynPY3f81udnw==", 2024-04-25T00:39:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-04-25T00:39:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBEpClmzBybEBAArry1lM4RIsM/txKfgBqDpw==", 2024-04-25T00:39:05.505 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-04-25T00:39:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBHpClm5HfjFhAAbFwJCpbhegLWDu0mbS8FoA==", 2024-04-25T00:39:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.506 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-04-25T00:39:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBKpClme/7oHBAAOR2iQ6CweexR7sC1PgesZw==", 2024-04-25T00:39:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-04-25T00:39:05.507 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A==", 2024-04-25T00:39:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-04-25T00:39:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:39:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-04-25T00:39:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm8W/EJxAA0HZW24Hhcwf+soSZn6jHJg==", 2024-04-25T00:39:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:39:05.508 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmGuQJJRAAhVd4D0jc6Whgyf+B6xgFMw==", 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-04-25T00:39:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmn+ciLRAAg4tQuZ13WP0RnWJaJq5fUQ==", 2024-04-25T00:39:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:39:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:39:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:39:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-04-25T00:39:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmPVihJBAA7PQge/teyRZKSkBnufZSZw==", 2024-04-25T00:39:05.510 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:39:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:39:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:39:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-04-25T00:39:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClm6WrdOhAAsY9FLCxG7PGpq2NCY4hGhQ==", 2024-04-25T00:39:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:39:05.511 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:39:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-04-25T00:39:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmeR4UOBAANs9SRolOm6E8+/4ciupk5w==", 2024-04-25T00:39:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:39:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:39:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:05.512 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:39:05.513 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:39:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.498+0000 7f70105cc640 1 -- 172.21.15.29:0/3642778149 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6fec067e60 msgr2=0x7f6fec06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.498+0000 7f70105cc640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6fec067e60 0x7f6fec06a320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7004006fd0 tx=0x7f7004008040 comp rx=0 tx=0).stop 2024-04-25T00:39:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.498+0000 7f70105cc640 1 -- 172.21.15.29:0/3642778149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70080b3a60 msgr2=0x7f7008153570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.498+0000 7f70105cc640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70080b3a60 0x7f7008153570 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80b0040 tx=0x7f6ff8019150 comp rx=0 tx=0).stop 2024-04-25T00:39:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.498+0000 7f70105cc640 1 -- 172.21.15.29:0/3642778149 shutdown_connections 2024-04-25T00:39:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.498+0000 7f70105cc640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6fec067e60 0x7f6fec06a320 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.498+0000 7f70105cc640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70080b4b10 0x7f7008153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.498+0000 7f70105cc640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70080b3a60 0x7f7008153570 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.498+0000 7f70105cc640 1 --2- 172.21.15.29:0/3642778149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70080b3140 0x7f7008152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.498+0000 7f70105cc640 1 -- 172.21.15.29:0/3642778149 >> 172.21.15.29:0/3642778149 conn(0x7f70080b1870 msgr2=0x7f7008143f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.498+0000 7f70105cc640 1 -- 172.21.15.29:0/3642778149 shutdown_connections 2024-04-25T00:39:05.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.498+0000 7f70105cc640 1 -- 172.21.15.29:0/3642778149 wait complete. 2024-04-25T00:39:05.517 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:05.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c7fa2640 1 Processor -- start 2024-04-25T00:39:05.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c7fa2640 1 -- start start 2024-04-25T00:39:05.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c7fa2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c015c220 0x7fa7c00cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c7fa2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa7c00d0670 0x7fa7c00b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c7fa2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c00cdbd0 con 0x7fa7c00cfa10 2024-04-25T00:39:05.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c7fa2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa7c015c600 con 0x7fa7c00d0670 2024-04-25T00:39:05.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c7fa2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c00b1a30 con 0x7fa7c015c220 2024-04-25T00:39:05.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c653f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa7c00d0670 0x7fa7c00b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c653f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa7c00d0670 0x7fa7c00b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50990/0 (socket says 172.21.15.29:50990) 2024-04-25T00:39:05.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c653f640 1 -- 172.21.15.29:0/1812213885 learned_addr learned my addr 172.21.15.29:0/1812213885 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:05.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c5d3e640 1 --2- 172.21.15.29:0/1812213885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c015c220 0x7fa7c00cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c653f640 1 -- 172.21.15.29:0/1812213885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c015c220 msgr2=0x7fa7c00cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:05.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c653f640 1 --2- 172.21.15.29:0/1812213885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c015c220 0x7fa7c00cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c653f640 1 -- 172.21.15.29:0/1812213885 >> v1:172.21.15.157:6789/0 conn(0x7fa7c00cfa10 legacy=0x7fa7c00cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:05.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.638+0000 7fa7c653f640 1 -- 172.21.15.29:0/1812213885 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c00b13e0 con 0x7fa7c00d0670 2024-04-25T00:39:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c653f640 1 --2- 172.21.15.29:0/1812213885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7c00d0670 0x7fa7c00b1ec0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009e50 tx=0x7fa7b801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55675bbb13d29341 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c4d3c640 1 -- 172.21.15.29:0/1812213885 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b80a5700 con 0x7fa7c00d0670 2024-04-25T00:39:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c4d3c640 1 -- 172.21.15.29:0/1812213885 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b80a58a0 con 0x7fa7c00d0670 2024-04-25T00:39:05.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c4d3c640 1 -- 172.21.15.29:0/1812213885 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b80a5c70 con 0x7fa7c00d0670 2024-04-25T00:39:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1812213885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7c00d0670 msgr2=0x7fa7c00b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c7fa2640 1 --2- 172.21.15.29:0/1812213885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7c00d0670 0x7fa7c00b1ec0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009e50 tx=0x7fa7b801ca30 comp rx=0 tx=0).stop 2024-04-25T00:39:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1812213885 shutdown_connections 2024-04-25T00:39:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c7fa2640 1 --2- 172.21.15.29:0/1812213885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7c00d0670 0x7fa7c00b1ec0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c7fa2640 1 --2- 172.21.15.29:0/1812213885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c015c220 0x7fa7c00cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1812213885 >> 172.21.15.29:0/1812213885 conn(0x7fa7c00d1d60 msgr2=0x7fa7c00d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1812213885 shutdown_connections 2024-04-25T00:39:05.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1812213885 wait complete. 2024-04-25T00:39:05.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c7fa2640 1 Processor -- start 2024-04-25T00:39:05.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c7fa2640 1 -- start start 2024-04-25T00:39:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.642+0000 7fa7c7fa2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c00cfa10 0x7fa7c015eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c7fa2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7c00d0670 0x7fa7c0145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c7fa2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c015c220 0x7fa7c01460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c7fa2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c00b3020 con 0x7fa7c00d0670 2024-04-25T00:39:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c7fa2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c00b2ea0 con 0x7fa7c015c220 2024-04-25T00:39:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c7fa2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c00b31a0 con 0x7fa7c00cfa10 2024-04-25T00:39:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c553d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7c00d0670 0x7fa7c0145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c553d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7c00d0670 0x7fa7c0145ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51004/0 (socket says 172.21.15.29:51004) 2024-04-25T00:39:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c553d640 1 -- 172.21.15.29:0/1516408150 learned_addr learned my addr 172.21.15.29:0/1516408150 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c5d3e640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c00cfa10 0x7fa7c015eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c653f640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c015c220 0x7fa7c01460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c553d640 1 -- 172.21.15.29:0/1516408150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c00cfa10 msgr2=0x7fa7c015eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c553d640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c00cfa10 0x7fa7c015eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c553d640 1 -- 172.21.15.29:0/1516408150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c015c220 msgr2=0x7fa7c01460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c553d640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c015c220 0x7fa7c01460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c553d640 1 -- 172.21.15.29:0/1516408150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c015d2c0 con 0x7fa7c00d0670 2024-04-25T00:39:05.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c653f640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c015c220 0x7fa7c01460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:05.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c5d3e640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c00cfa10 0x7fa7c015eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:05.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c553d640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7c00d0670 0x7fa7c0145ba0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400c370 tx=0x7fa7b400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:05.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7b27fc640 1 -- 172.21.15.29:0/1516408150 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40052d0 con 0x7fa7c00d0670 2024-04-25T00:39:05.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7b27fc640 1 -- 172.21.15.29:0/1516408150 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b4005470 con 0x7fa7c00d0670 2024-04-25T00:39:05.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1516408150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c015d560 con 0x7fa7c00d0670 2024-04-25T00:39:05.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7b27fc640 1 -- 172.21.15.29:0/1516408150 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4007260 con 0x7fa7c00d0670 2024-04-25T00:39:05.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1516408150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c015db20 con 0x7fa7c00d0670 2024-04-25T00:39:05.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1516408150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c0069270 con 0x7fa7c00d0670 2024-04-25T00:39:05.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7b27fc640 1 -- 172.21.15.29:0/1516408150 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fa7b4007400 con 0x7fa7c00d0670 2024-04-25T00:39:05.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7b27fc640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa7a4067e60 0x7fa7a406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7b27fc640 1 -- 172.21.15.29:0/1516408150 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4857+0+0 (secure 0 0 0) 0x7fa7b4085730 con 0x7fa7c00d0670 2024-04-25T00:39:05.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c5d3e640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa7a4067e60 0x7fa7a406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.646+0000 7fa7c5d3e640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa7a4067e60 0x7fa7a406a320 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc007a70 tx=0x7fa7bc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:05.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.650+0000 7fa7b27fc640 1 -- 172.21.15.29:0/1516408150 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa7b4055de0 con 0x7fa7c00d0670 2024-04-25T00:39:05.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.810+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1516408150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7c006fac0 con 0x7fa7c00d0670 2024-04-25T00:39:05.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.810+0000 7fa7b27fc640 1 -- 172.21.15.29:0/1516408150 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+12853 (secure 0 0 0) 0x7fa7b40556e0 con 0x7fa7c00d0670 2024-04-25T00:39:05.813 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:05.813 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":128,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:05.108995+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.814+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1516408150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa7a4067e60 msgr2=0x7fa7a406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.814+0000 7fa7c7fa2640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa7a4067e60 0x7fa7a406a320 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc007a70 tx=0x7fa7bc008040 comp rx=0 tx=0).stop 2024-04-25T00:39:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.814+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1516408150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7c00d0670 msgr2=0x7fa7c0145ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.814+0000 7fa7c7fa2640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7c00d0670 0x7fa7c0145ba0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400c370 tx=0x7fa7b400c830 comp rx=0 tx=0).stop 2024-04-25T00:39:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.814+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1516408150 shutdown_connections 2024-04-25T00:39:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.814+0000 7fa7c7fa2640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa7a4067e60 0x7fa7a406a320 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.814+0000 7fa7c7fa2640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c015c220 0x7fa7c01460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.814+0000 7fa7c7fa2640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7c00d0670 0x7fa7c0145ba0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.814+0000 7fa7c7fa2640 1 --2- 172.21.15.29:0/1516408150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c00cfa10 0x7fa7c015eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.814+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1516408150 >> 172.21.15.29:0/1516408150 conn(0x7fa7c00d1d60 msgr2=0x7fa7c00d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:05.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.814+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1516408150 shutdown_connections 2024-04-25T00:39:05.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.814+0000 7fa7c7fa2640 1 -- 172.21.15.29:0/1516408150 wait complete. 2024-04-25T00:39:05.832 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:39:05.833 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:39:05.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.946+0000 7fb97a53a640 1 Processor -- start 2024-04-25T00:39:05.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.946+0000 7fb97a53a640 1 -- start start 2024-04-25T00:39:05.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.946+0000 7fb97a53a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9740b3c30 0x7fb97406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.946+0000 7fb97a53a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9740b4b10 0x7fb9740ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.946+0000 7fb97a53a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9740b41e0 con 0x7fb9740b3140 2024-04-25T00:39:05.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb97a53a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb97406d960 con 0x7fb9740b3c30 2024-04-25T00:39:05.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb97a53a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb97406dc40 con 0x7fb9740b4b10 2024-04-25T00:39:05.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb978ad7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9740b4b10 0x7fb9740ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb9737fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9740b3c30 0x7fb97406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb9737fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9740b3c30 0x7fb97406e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:36632/0 (socket says 172.21.15.29:36632) 2024-04-25T00:39:05.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb9737fe640 1 -- 172.21.15.29:0/1492025189 learned_addr learned my addr 172.21.15.29:0/1492025189 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:05.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb973fff640 1 --1- 172.21.15.29:0/1492025189 >> v1:172.21.15.157:6789/0 conn(0x7fb9740b3140 0x7fb9740b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:45632/0 (socket says 172.21.15.29:45632) 2024-04-25T00:39:05.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb978ad7640 1 -- 172.21.15.29:0/1492025189 >> v2:172.21.15.157:3300/0 conn(0x7fb9740b3c30 msgr2=0x7fb97406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:05.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb978ad7640 1 --2- 172.21.15.29:0/1492025189 >> v2:172.21.15.157:3300/0 conn(0x7fb9740b3c30 0x7fb97406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb978ad7640 1 -- 172.21.15.29:0/1492025189 >> v1:172.21.15.157:6789/0 conn(0x7fb9740b3140 legacy=0x7fb9740b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb978ad7640 1 -- 172.21.15.29:0/1492025189 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97406df20 con 0x7fb9740b4b10 2024-04-25T00:39:05.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb9737fe640 1 --2- 172.21.15.29:0/1492025189 >> v2:172.21.15.157:3300/0 conn(0x7fb9740b3c30 0x7fb97406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:39:05.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb978ad7640 1 --2- 172.21.15.29:0/1492025189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9740b4b10 0x7fb9740ce240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009dc0 tx=0x7fb96c01c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4767b2beb4a3715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:05.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb972ffd640 1 -- 172.21.15.29:0/1492025189 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0b2070 con 0x7fb9740b4b10 2024-04-25T00:39:05.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb972ffd640 1 -- 172.21.15.29:0/1492025189 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96c0a5810 con 0x7fb9740b4b10 2024-04-25T00:39:05.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.950+0000 7fb972ffd640 1 -- 172.21.15.29:0/1492025189 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0a5be0 con 0x7fb9740b4b10 2024-04-25T00:39:05.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 -- 172.21.15.29:0/1492025189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9740b4b10 msgr2=0x7fb9740ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 --2- 172.21.15.29:0/1492025189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9740b4b10 0x7fb9740ce240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009dc0 tx=0x7fb96c01c950 comp rx=0 tx=0).stop 2024-04-25T00:39:05.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 -- 172.21.15.29:0/1492025189 shutdown_connections 2024-04-25T00:39:05.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 --2- 172.21.15.29:0/1492025189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9740b4b10 0x7fb9740ce240 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 --2- 172.21.15.29:0/1492025189 >> v2:172.21.15.157:3300/0 conn(0x7fb9740b3c30 0x7fb97406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 -- 172.21.15.29:0/1492025189 >> 172.21.15.29:0/1492025189 conn(0x7fb9740b1870 msgr2=0x7fb9740b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:05.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 -- 172.21.15.29:0/1492025189 shutdown_connections 2024-04-25T00:39:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 -- 172.21.15.29:0/1492025189 wait complete. 2024-04-25T00:39:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 Processor -- start 2024-04-25T00:39:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 -- start start 2024-04-25T00:39:05.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9740b3140 0x7fb97413fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9740b3c30 0x7fb97415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9740b4b10 0x7fb9741532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb974168d80 con 0x7fb9740b4b10 2024-04-25T00:39:05.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb974168c00 con 0x7fb9740b3140 2024-04-25T00:39:05.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb97a53a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb974168f00 con 0x7fb9740b3c30 2024-04-25T00:39:05.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb978ad7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9740b4b10 0x7fb9741532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb978ad7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9740b4b10 0x7fb9741532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51012/0 (socket says 172.21.15.29:51012) 2024-04-25T00:39:05.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb978ad7640 1 -- 172.21.15.29:0/2841916542 learned_addr learned my addr 172.21.15.29:0/2841916542 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:05.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb973fff640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9740b3140 0x7fb97413fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb9737fe640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9740b3c30 0x7fb97415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb978ad7640 1 -- 172.21.15.29:0/2841916542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9740b3c30 msgr2=0x7fb97415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb978ad7640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9740b3c30 0x7fb97415ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb978ad7640 1 -- 172.21.15.29:0/2841916542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9740b3140 msgr2=0x7fb97413fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:05.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb978ad7640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9740b3140 0x7fb97413fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:05.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.954+0000 7fb978ad7640 1 -- 172.21.15.29:0/2841916542 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97413ccf0 con 0x7fb9740b4b10 2024-04-25T00:39:05.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.958+0000 7fb978ad7640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9740b4b10 0x7fb9741532c0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fb96c01b000 tx=0x7fb96c0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:05.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.958+0000 7fb970ff9640 1 -- 172.21.15.29:0/2841916542 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0b2050 con 0x7fb9740b4b10 2024-04-25T00:39:05.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.958+0000 7fb970ff9640 1 -- 172.21.15.29:0/2841916542 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96c0be070 con 0x7fb9740b4b10 2024-04-25T00:39:05.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.958+0000 7fb970ff9640 1 -- 172.21.15.29:0/2841916542 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0ad8b0 con 0x7fb9740b4b10 2024-04-25T00:39:05.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.958+0000 7fb97a53a640 1 -- 172.21.15.29:0/2841916542 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97413efb0 con 0x7fb9740b4b10 2024-04-25T00:39:05.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.958+0000 7fb97a53a640 1 -- 172.21.15.29:0/2841916542 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97413f570 con 0x7fb9740b4b10 2024-04-25T00:39:05.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.958+0000 7fb97a53a640 1 -- 172.21.15.29:0/2841916542 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb974069270 con 0x7fb9740b4b10 2024-04-25T00:39:05.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.958+0000 7fb970ff9640 1 -- 172.21.15.29:0/2841916542 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fb96c0ada50 con 0x7fb9740b4b10 2024-04-25T00:39:05.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.958+0000 7fb970ff9640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb960067e60 0x7fb96006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:05.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.958+0000 7fb970ff9640 1 -- 172.21.15.29:0/2841916542 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4857+0+0 (secure 0 0 0) 0x7fb96c121260 con 0x7fb9740b4b10 2024-04-25T00:39:05.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.958+0000 7fb973fff640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb960067e60 0x7fb96006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:05.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.958+0000 7fb973fff640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb960067e60 0x7fb96006a320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb964000f70 tx=0x7fb964007cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:05.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:05.962+0000 7fb970ff9640 1 -- 172.21.15.29:0/2841916542 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb96c016540 con 0x7fb9740b4b10 2024-04-25T00:39:06.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.122+0000 7fb97a53a640 1 -- 172.21.15.29:0/2841916542 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb974140ad0 con 0x7fb9740b4b10 2024-04-25T00:39:06.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.126+0000 7fb970ff9640 1 -- 172.21.15.29:0/2841916542 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7fb96c0f1b30 con 0x7fb9740b4b10 2024-04-25T00:39:06.128 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:39:06.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.126+0000 7fb97a53a640 1 -- 172.21.15.29:0/2841916542 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb960067e60 msgr2=0x7fb96006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:06.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.126+0000 7fb97a53a640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb960067e60 0x7fb96006a320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb964000f70 tx=0x7fb964007cd0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.126+0000 7fb97a53a640 1 -- 172.21.15.29:0/2841916542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9740b4b10 msgr2=0x7fb9741532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:06.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.126+0000 7fb97a53a640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9740b4b10 0x7fb9741532c0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fb96c01b000 tx=0x7fb96c0196f0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.126+0000 7fb97a53a640 1 -- 172.21.15.29:0/2841916542 shutdown_connections 2024-04-25T00:39:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.126+0000 7fb97a53a640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb960067e60 0x7fb96006a320 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.126+0000 7fb97a53a640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9740b4b10 0x7fb9741532c0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.126+0000 7fb97a53a640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9740b3c30 0x7fb97415ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.126+0000 7fb97a53a640 1 --2- 172.21.15.29:0/2841916542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9740b3140 0x7fb97413fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.126+0000 7fb97a53a640 1 -- 172.21.15.29:0/2841916542 >> 172.21.15.29:0/2841916542 conn(0x7fb9740b1870 msgr2=0x7fb9741424f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.126+0000 7fb97a53a640 1 -- 172.21.15.29:0/2841916542 shutdown_connections 2024-04-25T00:39:06.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.126+0000 7fb97a53a640 1 -- 172.21.15.29:0/2841916542 wait complete. 2024-04-25T00:39:06.146 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:39:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.266+0000 7f82577a3640 1 Processor -- start 2024-04-25T00:39:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.266+0000 7f82577a3640 1 -- start start 2024-04-25T00:39:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.266+0000 7f82577a3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f82500b3c30 0x7f825006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.266+0000 7f82577a3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82500b41e0 con 0x7f82500b3140 2024-04-25T00:39:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.266+0000 7f82577a3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f825006d960 con 0x7f82500b4b10 2024-04-25T00:39:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.266+0000 7f82577a3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f825006dc40 con 0x7f82500b3c30 2024-04-25T00:39:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.266+0000 7f825553f640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f82500b3140 0x7f82500b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59486/0 (socket says 172.21.15.29:59486) 2024-04-25T00:39:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.266+0000 7f825553f640 1 -- 172.21.15.29:0/1210536454 learned_addr learned my addr 172.21.15.29:0/1210536454 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:06.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.266+0000 7f8254d3e640 1 --2- 172.21.15.29:0/1210536454 >> v2:172.21.15.29:3300/0 conn(0x7f82500b3c30 0x7f825006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:06.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.266+0000 7f8247fff640 1 -- 172.21.15.29:0/1210536454 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1082653715 0 0) 0x7f82500b41e0 con 0x7f82500b3140 2024-04-25T00:39:06.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.266+0000 7f8254d3e640 1 -- 172.21.15.29:0/1210536454 >> v1:172.21.15.157:6789/0 conn(0x7f82500b4b10 legacy=0x7f82500ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:06.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.266+0000 7f8254d3e640 1 -- 172.21.15.29:0/1210536454 >> v1:172.21.15.29:6789/0 conn(0x7f82500b3140 legacy=0x7f82500b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:06.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.266+0000 7f8254d3e640 1 -- 172.21.15.29:0/1210536454 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825006df20 con 0x7f82500b3c30 2024-04-25T00:39:06.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f8254d3e640 1 --2- 172.21.15.29:0/1210536454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82500b3c30 0x7f825006e440 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f82400097f0 tx=0x7f824001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bb21af5e5547b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:06.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f8247fff640 1 -- 172.21.15.29:0/1210536454 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82400b0070 con 0x7f82500b3c30 2024-04-25T00:39:06.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f8247fff640 1 -- 172.21.15.29:0/1210536454 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82400ab600 con 0x7f82500b3c30 2024-04-25T00:39:06.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f8247fff640 1 -- 172.21.15.29:0/1210536454 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82400ab9d0 con 0x7f82500b3c30 2024-04-25T00:39:06.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 -- 172.21.15.29:0/1210536454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82500b3c30 msgr2=0x7f825006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:06.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 --2- 172.21.15.29:0/1210536454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82500b3c30 0x7f825006e440 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f82400097f0 tx=0x7f824001c460 comp rx=0 tx=0).stop 2024-04-25T00:39:06.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 -- 172.21.15.29:0/1210536454 shutdown_connections 2024-04-25T00:39:06.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 --2- 172.21.15.29:0/1210536454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82500b3c30 0x7f825006e440 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 -- 172.21.15.29:0/1210536454 >> 172.21.15.29:0/1210536454 conn(0x7f82500b1870 msgr2=0x7f82500b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:06.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 -- 172.21.15.29:0/1210536454 shutdown_connections 2024-04-25T00:39:06.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 -- 172.21.15.29:0/1210536454 wait complete. 2024-04-25T00:39:06.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 Processor -- start 2024-04-25T00:39:06.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 -- start start 2024-04-25T00:39:06.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82500b3140 0x7f8250072670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:06.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82500b3c30 0x7f8250144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82500b4b10 0x7f825013c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8250169d90 con 0x7f82500b3140 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8250169c10 con 0x7f82500b4b10 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8250169f10 con 0x7f82500b3c30 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f825553f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82500b3140 0x7f8250072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f825553f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82500b3140 0x7f8250072670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51018/0 (socket says 172.21.15.29:51018) 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f825553f640 1 -- 172.21.15.29:0/1760659761 learned_addr learned my addr 172.21.15.29:0/1760659761 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f8254d3e640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82500b3c30 0x7f8250144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f8255d40640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82500b4b10 0x7f825013c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f825553f640 1 -- 172.21.15.29:0/1760659761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82500b3c30 msgr2=0x7f8250144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f825553f640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82500b3c30 0x7f8250144180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f825553f640 1 -- 172.21.15.29:0/1760659761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82500b4b10 msgr2=0x7f825013c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f825553f640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82500b4b10 0x7f825013c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f825553f640 1 -- 172.21.15.29:0/1760659761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825013f5e0 con 0x7f82500b3140 2024-04-25T00:39:06.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f8254d3e640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82500b3c30 0x7f8250144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:06.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f825553f640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82500b3140 0x7f8250072670 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f824c00c4b0 tx=0x7f824c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:06.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f8245ffb640 1 -- 172.21.15.29:0/1760659761 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c018070 con 0x7f82500b3140 2024-04-25T00:39:06.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f8245ffb640 1 -- 172.21.15.29:0/1760659761 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f824c007500 con 0x7f82500b3140 2024-04-25T00:39:06.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 -- 172.21.15.29:0/1760659761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825013f8c0 con 0x7f82500b3140 2024-04-25T00:39:06.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f8245ffb640 1 -- 172.21.15.29:0/1760659761 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c004a30 con 0x7f82500b3140 2024-04-25T00:39:06.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.270+0000 7f82577a3640 1 -- 172.21.15.29:0/1760659761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8250072bb0 con 0x7f82500b3140 2024-04-25T00:39:06.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.274+0000 7f82577a3640 1 -- 172.21.15.29:0/1760659761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8250069270 con 0x7f82500b3140 2024-04-25T00:39:06.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.274+0000 7f8245ffb640 1 -- 172.21.15.29:0/1760659761 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f824c01b070 con 0x7f82500b3140 2024-04-25T00:39:06.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.274+0000 7f8245ffb640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8234067e60 0x7f823406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:06.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.274+0000 7f8245ffb640 1 -- 172.21.15.29:0/1760659761 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4857+0+0 (secure 0 0 0) 0x7f824c0855f0 con 0x7f82500b3140 2024-04-25T00:39:06.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.274+0000 7f8254d3e640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8234067e60 0x7f823406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:06.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.274+0000 7f8254d3e640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8234067e60 0x7f823406a320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f82400ad000 tx=0x7f8240008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:06.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.278+0000 7f8245ffb640 1 -- 172.21.15.29:0/1760659761 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f824c055ca0 con 0x7f82500b3140 2024-04-25T00:39:06.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.438+0000 7f82577a3640 1 -- 172.21.15.29:0/1760659761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8250157fe0 con 0x7f82500b3140 2024-04-25T00:39:06.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.442+0000 7f8245ffb640 1 -- 172.21.15.29:0/1760659761 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) ==== 84+0+65 (secure 0 0 0) 0x7f824c0555a0 con 0x7f82500b3140 2024-04-25T00:39:06.444 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:06.444 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:39:06.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:06.444 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-04-25T00:39:06.444 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": ".mgr" 2024-04-25T00:39:06.444 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:06.445 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:39:06.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.442+0000 7f82577a3640 1 -- 172.21.15.29:0/1760659761 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8234067e60 msgr2=0x7f823406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:06.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.442+0000 7f82577a3640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8234067e60 0x7f823406a320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f82400ad000 tx=0x7f8240008040 comp rx=0 tx=0).stop 2024-04-25T00:39:06.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.442+0000 7f82577a3640 1 -- 172.21.15.29:0/1760659761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82500b3140 msgr2=0x7f8250072670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:06.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.442+0000 7f82577a3640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82500b3140 0x7f8250072670 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f824c00c4b0 tx=0x7f824c00c970 comp rx=0 tx=0).stop 2024-04-25T00:39:06.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.442+0000 7f82577a3640 1 -- 172.21.15.29:0/1760659761 shutdown_connections 2024-04-25T00:39:06.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.442+0000 7f82577a3640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8234067e60 0x7f823406a320 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.442+0000 7f82577a3640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82500b4b10 0x7f825013c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.442+0000 7f82577a3640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82500b3c30 0x7f8250144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.442+0000 7f82577a3640 1 --2- 172.21.15.29:0/1760659761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82500b3140 0x7f8250072670 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.442+0000 7f82577a3640 1 -- 172.21.15.29:0/1760659761 >> 172.21.15.29:0/1760659761 conn(0x7f82500b1870 msgr2=0x7f82501409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:06.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.442+0000 7f82577a3640 1 -- 172.21.15.29:0/1760659761 shutdown_connections 2024-04-25T00:39:06.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.442+0000 7f82577a3640 1 -- 172.21.15.29:0/1760659761 wait complete. 2024-04-25T00:39:06.463 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:39:06.463 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:39:06.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e15bd3640 1 Processor -- start 2024-04-25T00:39:06.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e15bd3640 1 -- start start 2024-04-25T00:39:06.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e15bd3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1015e240 0x7f0e1006b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:06.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e15bd3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e100cb330 0x7f0e100ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:06.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e15bd3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1006bcc0 con 0x7f0e100cd050 2024-04-25T00:39:06.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e15bd3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f0e10071df0 con 0x7f0e100cb330 2024-04-25T00:39:06.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e15bd3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0e100720d0 con 0x7f0e1015e240 2024-04-25T00:39:06.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e0ffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f0e100cd050 0x7f0e10161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59490/0 (socket says 172.21.15.29:59490) 2024-04-25T00:39:06.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e0ffff640 1 -- 172.21.15.29:0/990427762 learned_addr learned my addr 172.21.15.29:0/990427762 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:06.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e0effd640 1 --2- 172.21.15.29:0/990427762 >> v2:172.21.15.29:3300/0 conn(0x7f0e100cb330 0x7f0e100ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:06.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e0f7fe640 1 --2- 172.21.15.29:0/990427762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1015e240 0x7f0e1006b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:06.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e0e7fc640 1 -- 172.21.15.29:0/990427762 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3541292452 0 0) 0x7f0e1006bcc0 con 0x7f0e100cd050 2024-04-25T00:39:06.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e0e7fc640 1 -- 172.21.15.29:0/990427762 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0df0003530 con 0x7f0e100cd050 2024-04-25T00:39:06.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e0effd640 1 -- 172.21.15.29:0/990427762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1015e240 msgr2=0x7f0e1006b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:06.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e0effd640 1 --2- 172.21.15.29:0/990427762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1015e240 0x7f0e1006b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e0effd640 1 -- 172.21.15.29:0/990427762 >> v1:172.21.15.29:6789/0 conn(0x7f0e100cd050 legacy=0x7f0e10161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:06.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.578+0000 7f0e0effd640 1 -- 172.21.15.29:0/990427762 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e100723b0 con 0x7f0e100cb330 2024-04-25T00:39:06.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e0effd640 1 --2- 172.21.15.29:0/990427762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e100cb330 0x7f0e100ccb10 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f0df80097c0 tx=0x7f0df801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190fd02cd6198f5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:06.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e0e7fc640 1 -- 172.21.15.29:0/990427762 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80b5070 con 0x7f0e100cb330 2024-04-25T00:39:06.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e0e7fc640 1 -- 172.21.15.29:0/990427762 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df80a8990 con 0x7f0e100cb330 2024-04-25T00:39:06.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e0e7fc640 1 -- 172.21.15.29:0/990427762 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80a8d60 con 0x7f0e100cb330 2024-04-25T00:39:06.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 -- 172.21.15.29:0/990427762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e100cb330 msgr2=0x7f0e100ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:06.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 --2- 172.21.15.29:0/990427762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e100cb330 0x7f0e100ccb10 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f0df80097c0 tx=0x7f0df801ec90 comp rx=0 tx=0).stop 2024-04-25T00:39:06.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 -- 172.21.15.29:0/990427762 shutdown_connections 2024-04-25T00:39:06.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 --2- 172.21.15.29:0/990427762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e100cb330 0x7f0e100ccb10 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 --2- 172.21.15.29:0/990427762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1015e240 0x7f0e1006b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 -- 172.21.15.29:0/990427762 >> 172.21.15.29:0/990427762 conn(0x7f0e100d0c30 msgr2=0x7f0e100d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:06.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 -- 172.21.15.29:0/990427762 shutdown_connections 2024-04-25T00:39:06.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 -- 172.21.15.29:0/990427762 wait complete. 2024-04-25T00:39:06.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 Processor -- start 2024-04-25T00:39:06.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 -- start start 2024-04-25T00:39:06.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e100cb330 0x7f0e1015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e100cd050 0x7f0e10150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1015e240 0x7f0e1015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0e10072c50 con 0x7f0e100cd050 2024-04-25T00:39:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0e10072970 con 0x7f0e100cb330 2024-04-25T00:39:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.582+0000 7f0e15bd3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1015e620 con 0x7f0e1015e240 2024-04-25T00:39:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e100cd050 0x7f0e10150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e100cd050 0x7f0e10150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51026/0 (socket says 172.21.15.29:51026) 2024-04-25T00:39:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0effd640 1 -- 172.21.15.29:0/942507126 learned_addr learned my addr 172.21.15.29:0/942507126 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0f7fe640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e100cb330 0x7f0e1015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0effd640 1 -- 172.21.15.29:0/942507126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1015e240 msgr2=0x7f0e1015cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0ffff640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1015e240 0x7f0e1015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0effd640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1015e240 0x7f0e1015cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0effd640 1 -- 172.21.15.29:0/942507126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e100cb330 msgr2=0x7f0e1015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:06.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0effd640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e100cb330 0x7f0e1015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:06.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0effd640 1 -- 172.21.15.29:0/942507126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e10150ea0 con 0x7f0e100cd050 2024-04-25T00:39:06.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0f7fe640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e100cb330 0x7f0e1015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:06.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0effd640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e100cd050 0x7f0e10150960 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f0df801c8c0 tx=0x7f0df801ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:06.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e07fff640 1 -- 172.21.15.29:0/942507126 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80b5040 con 0x7f0e100cd050 2024-04-25T00:39:06.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e07fff640 1 -- 172.21.15.29:0/942507126 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df80c0070 con 0x7f0e100cd050 2024-04-25T00:39:06.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e15bd3640 1 -- 172.21.15.29:0/942507126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e100c7ea0 con 0x7f0e100cd050 2024-04-25T00:39:06.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e07fff640 1 -- 172.21.15.29:0/942507126 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80b0920 con 0x7f0e100cd050 2024-04-25T00:39:06.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e15bd3640 1 -- 172.21.15.29:0/942507126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e100c8420 con 0x7f0e100cd050 2024-04-25T00:39:06.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e15bd3640 1 -- 172.21.15.29:0/942507126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e100692e0 con 0x7f0e100cd050 2024-04-25T00:39:06.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e07fff640 1 -- 172.21.15.29:0/942507126 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f0df80b0ac0 con 0x7f0e100cd050 2024-04-25T00:39:06.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e07fff640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0df4067e60 0x7f0df406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:06.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e07fff640 1 -- 172.21.15.29:0/942507126 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4857+0+0 (secure 0 0 0) 0x7f0df81237e0 con 0x7f0e100cd050 2024-04-25T00:39:06.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0f7fe640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0df4067e60 0x7f0df406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:06.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.586+0000 7f0e0f7fe640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0df4067e60 0x7f0df406a320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009f50 tx=0x7f0e0000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:06.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.590+0000 7f0e07fff640 1 -- 172.21.15.29:0/942507126 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0df80187f0 con 0x7f0e100cd050 2024-04-25T00:39:06.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:06.746+0000 7f0e15bd3640 1 -- 172.21.15.29:0/942507126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0e100d1a60 con 0x7f0e100cd050 2024-04-25T00:39:07.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.110+0000 7f0e07fff640 1 -- 172.21.15.29:0/942507126 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v129) ==== 138+0+0 (secure 0 0 0) 0x7f0df80f3e90 con 0x7f0e100cd050 2024-04-25T00:39:07.115 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-04-25T00:39:07.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.114+0000 7f0e15bd3640 1 -- 172.21.15.29:0/942507126 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0df4067e60 msgr2=0x7f0df406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:07.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.114+0000 7f0e15bd3640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0df4067e60 0x7f0df406a320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009f50 tx=0x7f0e0000b040 comp rx=0 tx=0).stop 2024-04-25T00:39:07.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.114+0000 7f0e15bd3640 1 -- 172.21.15.29:0/942507126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e100cd050 msgr2=0x7f0e10150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:07.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.114+0000 7f0e15bd3640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e100cd050 0x7f0e10150960 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f0df801c8c0 tx=0x7f0df801ba30 comp rx=0 tx=0).stop 2024-04-25T00:39:07.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.114+0000 7f0e15bd3640 1 -- 172.21.15.29:0/942507126 shutdown_connections 2024-04-25T00:39:07.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.114+0000 7f0e15bd3640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0df4067e60 0x7f0df406a320 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:07.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.114+0000 7f0e15bd3640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1015e240 0x7f0e1015cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:07.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.114+0000 7f0e15bd3640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e100cd050 0x7f0e10150960 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:07.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.114+0000 7f0e15bd3640 1 --2- 172.21.15.29:0/942507126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e100cb330 0x7f0e1015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:07.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.114+0000 7f0e15bd3640 1 -- 172.21.15.29:0/942507126 >> 172.21.15.29:0/942507126 conn(0x7f0e100d0c30 msgr2=0x7f0e100672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:07.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.114+0000 7f0e15bd3640 1 -- 172.21.15.29:0/942507126 shutdown_connections 2024-04-25T00:39:07.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.114+0000 7f0e15bd3640 1 -- 172.21.15.29:0/942507126 wait complete. 2024-04-25T00:39:07.146 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:39:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f0011463640 1 Processor -- start 2024-04-25T00:39:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f0011463640 1 -- start start 2024-04-25T00:39:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f0011463640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f000c16ada0 0x7f000c0b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f0011463640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f000c0b3280 0x7f000c06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f0011463640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f000c071be0 con 0x7f000c16b180 2024-04-25T00:39:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f0011463640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f000c071ec0 con 0x7f000c0b3280 2024-04-25T00:39:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f0011463640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f000c06cd40 con 0x7f000c16ada0 2024-04-25T00:39:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f000b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f000c0b3280 0x7f000c06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f000b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f000c0b3280 0x7f000c06d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51038/0 (socket says 172.21.15.29:51038) 2024-04-25T00:39:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f000b7fe640 1 -- 172.21.15.29:0/2177835316 learned_addr learned my addr 172.21.15.29:0/2177835316 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f000affd640 1 --2- 172.21.15.29:0/2177835316 >> v2:172.21.15.157:3300/0 conn(0x7f000c16ada0 0x7f000c0b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f0009ffb640 1 -- 172.21.15.29:0/2177835316 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 250216392 0 0) 0x7f000c071be0 con 0x7f000c16b180 2024-04-25T00:39:07.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f0009ffb640 1 -- 172.21.15.29:0/2177835316 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efff4003530 con 0x7f000c16b180 2024-04-25T00:39:07.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f000b7fe640 1 -- 172.21.15.29:0/2177835316 >> v2:172.21.15.157:3300/0 conn(0x7f000c16ada0 msgr2=0x7f000c0b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:07.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f000b7fe640 1 --2- 172.21.15.29:0/2177835316 >> v2:172.21.15.157:3300/0 conn(0x7f000c16ada0 0x7f000c0b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:07.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.278+0000 7f000b7fe640 1 -- 172.21.15.29:0/2177835316 >> v1:172.21.15.157:6789/0 conn(0x7f000c16b180 legacy=0x7f000c0b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:07.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.282+0000 7f000b7fe640 1 -- 172.21.15.29:0/2177835316 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c06d020 con 0x7f000c0b3280 2024-04-25T00:39:07.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.282+0000 7f000b7fe640 1 --2- 172.21.15.29:0/2177835316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f000c0b3280 0x7f000c06d820 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f000c06c110 tx=0x7f000401cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76dcb24b4b749d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.282+0000 7f0009ffb640 1 -- 172.21.15.29:0/2177835316 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040b2070 con 0x7f000c0b3280 2024-04-25T00:39:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.282+0000 7f0009ffb640 1 -- 172.21.15.29:0/2177835316 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00040a5be0 con 0x7f000c0b3280 2024-04-25T00:39:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.282+0000 7f0009ffb640 1 -- 172.21.15.29:0/2177835316 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040ad440 con 0x7f000c0b3280 2024-04-25T00:39:07.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.282+0000 7f0011463640 1 -- 172.21.15.29:0/2177835316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f000c0b3280 msgr2=0x7f000c06d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:07.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.282+0000 7f0011463640 1 --2- 172.21.15.29:0/2177835316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f000c0b3280 0x7f000c06d820 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f000c06c110 tx=0x7f000401cd10 comp rx=0 tx=0).stop 2024-04-25T00:39:07.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.282+0000 7f0011463640 1 -- 172.21.15.29:0/2177835316 shutdown_connections 2024-04-25T00:39:07.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.282+0000 7f0011463640 1 --2- 172.21.15.29:0/2177835316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f000c0b3280 0x7f000c06d820 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:07.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.282+0000 7f0011463640 1 --2- 172.21.15.29:0/2177835316 >> v2:172.21.15.157:3300/0 conn(0x7f000c16ada0 0x7f000c0b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:07.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.282+0000 7f0011463640 1 -- 172.21.15.29:0/2177835316 >> 172.21.15.29:0/2177835316 conn(0x7f000c0b1870 msgr2=0x7f000c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:07.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.282+0000 7f0011463640 1 -- 172.21.15.29:0/2177835316 shutdown_connections 2024-04-25T00:39:07.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.282+0000 7f0011463640 1 -- 172.21.15.29:0/2177835316 wait complete. 2024-04-25T00:39:07.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f0011463640 1 Processor -- start 2024-04-25T00:39:07.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f0011463640 1 -- start start 2024-04-25T00:39:07.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f0011463640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c0b3280 0x7f000c160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:07.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f0011463640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f000c16ada0 0x7f000c16d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:07.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f0011463640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f000c16b180 0x7f000c089910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:07.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f0011463640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f000c177bd0 con 0x7f000c16b180 2024-04-25T00:39:07.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f0011463640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f000c06e780 con 0x7f000c16ada0 2024-04-25T00:39:07.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f0011463640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f000c177d50 con 0x7f000c0b3280 2024-04-25T00:39:07.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f000b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f000c16b180 0x7f000c089910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:07.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f000b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f000c16b180 0x7f000c089910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51054/0 (socket says 172.21.15.29:51054) 2024-04-25T00:39:07.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f000b7fe640 1 -- 172.21.15.29:0/2935794912 learned_addr learned my addr 172.21.15.29:0/2935794912 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:07.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f000a7fc640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f000c16ada0 0x7f000c16d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:07.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f000affd640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c0b3280 0x7f000c160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:07.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f000b7fe640 1 -- 172.21.15.29:0/2935794912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c0b3280 msgr2=0x7f000c160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:07.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f000b7fe640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c0b3280 0x7f000c160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:07.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f000b7fe640 1 -- 172.21.15.29:0/2935794912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f000c16ada0 msgr2=0x7f000c16d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:07.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f000b7fe640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f000c16ada0 0x7f000c16d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:07.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f000b7fe640 1 -- 172.21.15.29:0/2935794912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c1593e0 con 0x7f000c16b180 2024-04-25T00:39:07.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f000affd640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c0b3280 0x7f000c160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:07.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f000b7fe640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f000c16b180 0x7f000c089910 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f00040b0040 tx=0x7f0004019ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:07.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7efff37fe640 1 -- 172.21.15.29:0/2935794912 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040b2070 con 0x7f000c16b180 2024-04-25T00:39:07.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7efff37fe640 1 -- 172.21.15.29:0/2935794912 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00040bd070 con 0x7f000c16b180 2024-04-25T00:39:07.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7efff37fe640 1 -- 172.21.15.29:0/2935794912 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040074f0 con 0x7f000c16b180 2024-04-25T00:39:07.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f0011463640 1 -- 172.21.15.29:0/2935794912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000c1596c0 con 0x7f000c16b180 2024-04-25T00:39:07.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f0011463640 1 -- 172.21.15.29:0/2935794912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000c159f60 con 0x7f000c16b180 2024-04-25T00:39:07.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7f0011463640 1 -- 172.21.15.29:0/2935794912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f000c069270 con 0x7f000c16b180 2024-04-25T00:39:07.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.286+0000 7efff37fe640 1 -- 172.21.15.29:0/2935794912 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f00040b7850 con 0x7f000c16b180 2024-04-25T00:39:07.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.290+0000 7efff37fe640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efff8067e60 0x7efff806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.290+0000 7efff37fe640 1 -- 172.21.15.29:0/2935794912 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5227+0+0 (secure 0 0 0) 0x7f0004122230 con 0x7f000c16b180 2024-04-25T00:39:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.290+0000 7f000affd640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efff8067e60 0x7efff806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:07.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.290+0000 7f000affd640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efff8067e60 0x7efff806a320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f000c068770 tx=0x7efffc00b3a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:07.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.294+0000 7efff37fe640 1 -- 172.21.15.29:0/2935794912 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0004016900 con 0x7f000c16b180 2024-04-25T00:39:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:07.454+0000 7f0011463640 1 -- 172.21.15.29:0/2935794912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f000c16f330 con 0x7f000c16b180 2024-04-25T00:39:08.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.122+0000 7efff37fe640 1 -- 172.21.15.29:0/2935794912 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v130) ==== 170+0+0 (secure 0 0 0) 0x7f00040f2840 con 0x7f000c16b180 2024-04-25T00:39:08.126 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-04-25T00:39:08.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.126+0000 7f0011463640 1 -- 172.21.15.29:0/2935794912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efff8067e60 msgr2=0x7efff806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:08.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.126+0000 7f0011463640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efff8067e60 0x7efff806a320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f000c068770 tx=0x7efffc00b3a0 comp rx=0 tx=0).stop 2024-04-25T00:39:08.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.126+0000 7f0011463640 1 -- 172.21.15.29:0/2935794912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f000c16b180 msgr2=0x7f000c089910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:08.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.126+0000 7f0011463640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f000c16b180 0x7f000c089910 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f00040b0040 tx=0x7f0004019ab0 comp rx=0 tx=0).stop 2024-04-25T00:39:08.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.126+0000 7f0011463640 1 -- 172.21.15.29:0/2935794912 shutdown_connections 2024-04-25T00:39:08.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.126+0000 7f0011463640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efff8067e60 0x7efff806a320 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:08.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.126+0000 7f0011463640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f000c16b180 0x7f000c089910 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:08.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.126+0000 7f0011463640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f000c16ada0 0x7f000c16d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:08.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.126+0000 7f0011463640 1 --2- 172.21.15.29:0/2935794912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f000c0b3280 0x7f000c160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:08.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.126+0000 7f0011463640 1 -- 172.21.15.29:0/2935794912 >> 172.21.15.29:0/2935794912 conn(0x7f000c0b1870 msgr2=0x7f000c08b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:08.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.126+0000 7f0011463640 1 -- 172.21.15.29:0/2935794912 shutdown_connections 2024-04-25T00:39:08.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.126+0000 7f0011463640 1 -- 172.21.15.29:0/2935794912 wait complete. 2024-04-25T00:39:08.160 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:39:08.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa194307640 1 Processor -- start 2024-04-25T00:39:08.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa194307640 1 -- start start 2024-04-25T00:39:08.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa194307640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa18c0b3140 0x7fa18c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:08.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa194307640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa18c0b4b10 0x7fa18c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:08.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa194307640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa18c0b41e0 con 0x7fa18c0b3a60 2024-04-25T00:39:08.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa194307640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa18c06d960 con 0x7fa18c0b3140 2024-04-25T00:39:08.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa194307640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa18c06dc40 con 0x7fa18c0b4b10 2024-04-25T00:39:08.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa1920a3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa18c0b3140 0x7fa18c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:08.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa1920a3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa18c0b3140 0x7fa18c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56820/0 (socket says 172.21.15.29:56820) 2024-04-25T00:39:08.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa1920a3640 1 -- 172.21.15.29:0/3913794589 learned_addr learned my addr 172.21.15.29:0/3913794589 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:08.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa1918a2640 1 --2- 172.21.15.29:0/3913794589 >> v2:172.21.15.157:3300/0 conn(0x7fa18c0b4b10 0x7fa18c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:08.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa1920a3640 1 -- 172.21.15.29:0/3913794589 >> v2:172.21.15.157:3300/0 conn(0x7fa18c0b4b10 msgr2=0x7fa18c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:08.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa1920a3640 1 --2- 172.21.15.29:0/3913794589 >> v2:172.21.15.157:3300/0 conn(0x7fa18c0b4b10 0x7fa18c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:08.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa1920a3640 1 -- 172.21.15.29:0/3913794589 >> v1:172.21.15.157:6789/0 conn(0x7fa18c0b3a60 legacy=0x7fa18c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:08.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.286+0000 7fa1920a3640 1 -- 172.21.15.29:0/3913794589 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18c06df20 con 0x7fa18c0b3140 2024-04-25T00:39:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa1920a3640 1 --2- 172.21.15.29:0/3913794589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa18c0b3140 0x7fa18c06e440 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa188009870 tx=0x7fa18801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e176be9cf18057 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa1910a1640 1 -- 172.21.15.29:0/3913794589 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1880b0070 con 0x7fa18c0b3140 2024-04-25T00:39:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa1910a1640 1 -- 172.21.15.29:0/3913794589 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1880ab600 con 0x7fa18c0b3140 2024-04-25T00:39:08.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa1910a1640 1 -- 172.21.15.29:0/3913794589 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1880a4e60 con 0x7fa18c0b3140 2024-04-25T00:39:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 -- 172.21.15.29:0/3913794589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa18c0b3140 msgr2=0x7fa18c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 --2- 172.21.15.29:0/3913794589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa18c0b3140 0x7fa18c06e440 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa188009870 tx=0x7fa18801c4e0 comp rx=0 tx=0).stop 2024-04-25T00:39:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 -- 172.21.15.29:0/3913794589 shutdown_connections 2024-04-25T00:39:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 --2- 172.21.15.29:0/3913794589 >> v2:172.21.15.157:3300/0 conn(0x7fa18c0b4b10 0x7fa18c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 --2- 172.21.15.29:0/3913794589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa18c0b3140 0x7fa18c06e440 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 -- 172.21.15.29:0/3913794589 >> 172.21.15.29:0/3913794589 conn(0x7fa18c0b1870 msgr2=0x7fa18c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 -- 172.21.15.29:0/3913794589 shutdown_connections 2024-04-25T00:39:08.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 -- 172.21.15.29:0/3913794589 wait complete. 2024-04-25T00:39:08.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 Processor -- start 2024-04-25T00:39:08.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 -- start start 2024-04-25T00:39:08.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18c0b3140 0x7fa18c13fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa18c0b3a60 0x7fa18c1544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa18c0b4b10 0x7fa18c143950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.290+0000 7fa194307640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c168b70 con 0x7fa18c0b3a60 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa194307640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c1689f0 con 0x7fa18c0b3140 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa194307640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa18c168cf0 con 0x7fa18c0b4b10 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa1918a2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa18c0b3a60 0x7fa18c1544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa1918a2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa18c0b3a60 0x7fa18c1544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56834/0 (socket says 172.21.15.29:56834) 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa1918a2640 1 -- 172.21.15.29:0/2490451139 learned_addr learned my addr 172.21.15.29:0/2490451139 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa1920a3640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18c0b3140 0x7fa18c13fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa1918a2640 1 -- 172.21.15.29:0/2490451139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa18c0b4b10 msgr2=0x7fa18c143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa1918a2640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa18c0b4b10 0x7fa18c143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa1918a2640 1 -- 172.21.15.29:0/2490451139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18c0b3140 msgr2=0x7fa18c13fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa1918a2640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18c0b3140 0x7fa18c13fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:08.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa1918a2640 1 -- 172.21.15.29:0/2490451139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18c140330 con 0x7fa18c0b3a60 2024-04-25T00:39:08.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa1920a3640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18c0b3140 0x7fa18c13fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:08.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa1918a2640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa18c0b3a60 0x7fa18c1544e0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00d800 tx=0x7fa17c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:08.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa182ffd640 1 -- 172.21.15.29:0/2490451139 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00b6c0 con 0x7fa18c0b3a60 2024-04-25T00:39:08.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa182ffd640 1 -- 172.21.15.29:0/2490451139 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa17c019070 con 0x7fa18c0b3a60 2024-04-25T00:39:08.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa194307640 1 -- 172.21.15.29:0/2490451139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa18c140610 con 0x7fa18c0b3a60 2024-04-25T00:39:08.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa182ffd640 1 -- 172.21.15.29:0/2490451139 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00bae0 con 0x7fa18c0b3a60 2024-04-25T00:39:08.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa194307640 1 -- 172.21.15.29:0/2490451139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa18c140bd0 con 0x7fa18c0b3a60 2024-04-25T00:39:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa194307640 1 -- 172.21.15.29:0/2490451139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa18c069270 con 0x7fa18c0b3a60 2024-04-25T00:39:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa182ffd640 1 -- 172.21.15.29:0/2490451139 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fa17c00bc80 con 0x7fa18c0b3a60 2024-04-25T00:39:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa182ffd640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa170067e90 0x7fa17006a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa1920a3640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa170067e90 0x7fa17006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.294+0000 7fa182ffd640 1 -- 172.21.15.29:0/2490451139 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5609+0+0 (secure 0 0 0) 0x7fa17c087640 con 0x7fa18c0b3a60 2024-04-25T00:39:08.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.298+0000 7fa1920a3640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa170067e90 0x7fa17006a350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa1880ad000 tx=0x7fa188008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:08.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.298+0000 7fa182ffd640 1 -- 172.21.15.29:0/2490451139 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa17c0579f0 con 0x7fa18c0b3a60 2024-04-25T00:39:08.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:08.502+0000 7fa194307640 1 -- 172.21.15.29:0/2490451139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa18c0ce4e0 con 0x7fa18c0b3a60 2024-04-25T00:39:09.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.126+0000 7fa182ffd640 1 -- 172.21.15.29:0/2490451139 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v80) ==== 174+0+0 (secure 0 0 0) 0x7fa17c0572f0 con 0x7fa18c0b3a60 2024-04-25T00:39:09.129 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 20 and data pool 21 2024-04-25T00:39:09.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.130+0000 7fa194307640 1 -- 172.21.15.29:0/2490451139 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa170067e90 msgr2=0x7fa17006a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.130+0000 7fa194307640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa170067e90 0x7fa17006a350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa1880ad000 tx=0x7fa188008040 comp rx=0 tx=0).stop 2024-04-25T00:39:09.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.130+0000 7fa194307640 1 -- 172.21.15.29:0/2490451139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa18c0b3a60 msgr2=0x7fa18c1544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.130+0000 7fa194307640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa18c0b3a60 0x7fa18c1544e0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00d800 tx=0x7fa17c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.134+0000 7fa194307640 1 -- 172.21.15.29:0/2490451139 shutdown_connections 2024-04-25T00:39:09.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.134+0000 7fa194307640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa170067e90 0x7fa17006a350 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.134+0000 7fa194307640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa18c0b4b10 0x7fa18c143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.134+0000 7fa194307640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa18c0b3a60 0x7fa18c1544e0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.134+0000 7fa194307640 1 --2- 172.21.15.29:0/2490451139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18c0b3140 0x7fa18c13fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.134+0000 7fa194307640 1 -- 172.21.15.29:0/2490451139 >> 172.21.15.29:0/2490451139 conn(0x7fa18c0b1870 msgr2=0x7fa18c15b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:09.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.134+0000 7fa194307640 1 -- 172.21.15.29:0/2490451139 shutdown_connections 2024-04-25T00:39:09.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.134+0000 7fa194307640 1 -- 172.21.15.29:0/2490451139 wait complete. 2024-04-25T00:39:09.172 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:09.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441ce06640 1 Processor -- start 2024-04-25T00:39:09.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441ce06640 1 -- start start 2024-04-25T00:39:09.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441ce06640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f441815e2a0 0x7f44180d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441ce06640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44180d4d10 0x7f44180d2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441ce06640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44180cf8a0 con 0x7f44180d2680 2024-04-25T00:39:09.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441ce06640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f44180cf650 con 0x7f44180d4d10 2024-04-25T00:39:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441ce06640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44180b38d0 con 0x7f441815e2a0 2024-04-25T00:39:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f4416d9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f44180d2680 0x7f44180d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39054/0 (socket says 172.21.15.29:39054) 2024-04-25T00:39:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f4416d9d640 1 -- 172.21.15.29:0/4076952839 learned_addr learned my addr 172.21.15.29:0/4076952839 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f4415d9b640 1 --2- 172.21.15.29:0/4076952839 >> v2:172.21.15.157:3300/0 conn(0x7f44180d4d10 0x7f44180d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 651626701 0 0) 0x7f44180cf8a0 con 0x7f44180d2680 2024-04-25T00:39:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441659c640 1 --2- 172.21.15.29:0/4076952839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f441815e2a0 0x7f44180d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4400003530 con 0x7f44180d2680 2024-04-25T00:39:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2622177562 0 0) 0x7f4400003530 con 0x7f44180d2680 2024-04-25T00:39:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f441815e2a0 msgr2=0x7f44180d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 --2- 172.21.15.29:0/4076952839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f441815e2a0 0x7f44180d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 >> v2:172.21.15.157:3300/0 conn(0x7f44180d4d10 msgr2=0x7f44180d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 --2- 172.21.15.29:0/4076952839 >> v2:172.21.15.157:3300/0 conn(0x7f44180d4d10 0x7f44180d2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44180cfb80 con 0x7f44180d2680 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4410001a00 con 0x7f44180d2680 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 >> v1:172.21.15.29:6789/0 conn(0x7f44180d2680 legacy=0x7f44180d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 --2- 172.21.15.29:0/4076952839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44000039c0 0x7f4400003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 --2- 172.21.15.29:0/4076952839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f441815e2a0 0x7f4400004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 --2- 172.21.15.29:0/4076952839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4400004d50 0x7f44000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f44180cf650 con 0x7f4400004d50 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44180b38d0 con 0x7f44000039c0 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f440000ba00 con 0x7f441815e2a0 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f4416d9d640 1 --2- 172.21.15.29:0/4076952839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4400004d50 0x7f44000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f4415d9b640 1 --2- 172.21.15.29:0/4076952839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f441815e2a0 0x7f4400004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f441659c640 1 --2- 172.21.15.29:0/4076952839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44000039c0 0x7f4400003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f4416d9d640 1 -- 172.21.15.29:0/4076952839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f441815e2a0 msgr2=0x7f4400004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f4416d9d640 1 --2- 172.21.15.29:0/4076952839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f441815e2a0 0x7f4400004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f4416d9d640 1 -- 172.21.15.29:0/4076952839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44000039c0 msgr2=0x7f4400003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:09.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f4416d9d640 1 --2- 172.21.15.29:0/4076952839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44000039c0 0x7f4400003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.290+0000 7f4416d9d640 1 -- 172.21.15.29:0/4076952839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4400003530 con 0x7f4400004d50 2024-04-25T00:39:09.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.294+0000 7f4416d9d640 1 --2- 172.21.15.29:0/4076952839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4400004d50 0x7f44000051d0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f44100013e0 tx=0x7f441001fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b2800dab224c5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:09.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.294+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441000a410 con 0x7f4400004d50 2024-04-25T00:39:09.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.294+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44100aa670 con 0x7f4400004d50 2024-04-25T00:39:09.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.294+0000 7f441559a640 1 -- 172.21.15.29:0/4076952839 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44100aaa60 con 0x7f4400004d50 2024-04-25T00:39:09.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.294+0000 7f441ce06640 1 -- 172.21.15.29:0/4076952839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4400004d50 msgr2=0x7f44000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.294+0000 7f441ce06640 1 --2- 172.21.15.29:0/4076952839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4400004d50 0x7f44000051d0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f44100013e0 tx=0x7f441001fa40 comp rx=0 tx=0).stop 2024-04-25T00:39:09.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.294+0000 7f4416d9d640 1 -- 172.21.15.29:0/4076952839 reap_dead start 2024-04-25T00:39:09.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.294+0000 7f441ce06640 1 -- 172.21.15.29:0/4076952839 shutdown_connections 2024-04-25T00:39:09.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.294+0000 7f441ce06640 1 -- 172.21.15.29:0/4076952839 >> 172.21.15.29:0/4076952839 conn(0x7f44180d4030 msgr2=0x7f44180d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:09.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.294+0000 7f441ce06640 1 -- 172.21.15.29:0/4076952839 shutdown_connections 2024-04-25T00:39:09.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.294+0000 7f441ce06640 1 -- 172.21.15.29:0/4076952839 wait complete. 2024-04-25T00:39:09.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.294+0000 7f441ce06640 1 Processor -- start 2024-04-25T00:39:09.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.294+0000 7f441ce06640 1 -- start start 2024-04-25T00:39:09.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f441ce06640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4400004d50 0x7f441815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f441ce06640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44180cc6f0 0x7f441814a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f441ce06640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44180ccb70 0x7f441814a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f441ce06640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f441815e680 con 0x7f44180ccb70 2024-04-25T00:39:09.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f441ce06640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44180b3bb0 con 0x7f44180cc6f0 2024-04-25T00:39:09.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f441ce06640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f441815e800 con 0x7f4400004d50 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4416d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44180ccb70 0x7f441814a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4416d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44180ccb70 0x7f441814a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56846/0 (socket says 172.21.15.29:56846) 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4416d9d640 1 -- 172.21.15.29:0/2367003210 learned_addr learned my addr 172.21.15.29:0/2367003210 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4415d9b640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44180cc6f0 0x7f441814a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f441659c640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4400004d50 0x7f441815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4416d9d640 1 -- 172.21.15.29:0/2367003210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4400004d50 msgr2=0x7f441815ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4416d9d640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4400004d50 0x7f441815ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4416d9d640 1 -- 172.21.15.29:0/2367003210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44180cc6f0 msgr2=0x7f441814a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4416d9d640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44180cc6f0 0x7f441814a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4416d9d640 1 -- 172.21.15.29:0/2367003210 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44180cd140 con 0x7f44180ccb70 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4415d9b640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44180cc6f0 0x7f441814a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4416d9d640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44180ccb70 0x7f441814a800 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f44100013e0 tx=0x7f4400005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4406ffd640 1 -- 172.21.15.29:0/2367003210 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441000a3f0 con 0x7f44180ccb70 2024-04-25T00:39:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4406ffd640 1 -- 172.21.15.29:0/2367003210 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44100aa470 con 0x7f44180ccb70 2024-04-25T00:39:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4406ffd640 1 -- 172.21.15.29:0/2367003210 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410004d70 con 0x7f44180ccb70 2024-04-25T00:39:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f441ce06640 1 -- 172.21.15.29:0/2367003210 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44180cd420 con 0x7f44180ccb70 2024-04-25T00:39:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f441ce06640 1 -- 172.21.15.29:0/2367003210 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44180cd9e0 con 0x7f44180ccb70 2024-04-25T00:39:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f441ce06640 1 -- 172.21.15.29:0/2367003210 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4418069270 con 0x7f44180ccb70 2024-04-25T00:39:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4406ffd640 1 -- 172.21.15.29:0/2367003210 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f44100aa620 con 0x7f44180ccb70 2024-04-25T00:39:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4406ffd640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f43f8067e60 0x7f43f806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f4406ffd640 1 -- 172.21.15.29:0/2367003210 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f4410123dd0 con 0x7f44180ccb70 2024-04-25T00:39:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.298+0000 7f441659c640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f43f8067e60 0x7f43f806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.302+0000 7f441659c640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f43f8067e60 0x7f43f806a320 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4408009a30 tx=0x7f4408008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:09.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.306+0000 7f4406ffd640 1 -- 172.21.15.29:0/2367003210 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f44100af020 con 0x7f44180ccb70 2024-04-25T00:39:09.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.466+0000 7f441ce06640 1 -- 172.21.15.29:0/2367003210 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44180d3e10 con 0x7f44180ccb70 2024-04-25T00:39:09.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.470+0000 7f4406ffd640 1 -- 172.21.15.29:0/2367003210 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16644 (secure 0 0 0) 0x7f4410019750 con 0x7f44180ccb70 2024-04-25T00:39:09.473 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:09.473 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":131,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:08.508383+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:06.791988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:07.503204+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:09.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.470+0000 7f441ce06640 1 -- 172.21.15.29:0/2367003210 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f43f8067e60 msgr2=0x7f43f806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.470+0000 7f441ce06640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f43f8067e60 0x7f43f806a320 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4408009a30 tx=0x7f4408008040 comp rx=0 tx=0).stop 2024-04-25T00:39:09.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.470+0000 7f441ce06640 1 -- 172.21.15.29:0/2367003210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44180ccb70 msgr2=0x7f441814a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.470+0000 7f441ce06640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44180ccb70 0x7f441814a800 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f44100013e0 tx=0x7f4400005840 comp rx=0 tx=0).stop 2024-04-25T00:39:09.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.470+0000 7f441ce06640 1 -- 172.21.15.29:0/2367003210 shutdown_connections 2024-04-25T00:39:09.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.470+0000 7f441ce06640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f43f8067e60 0x7f43f806a320 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.470+0000 7f441ce06640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44180ccb70 0x7f441814a800 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.470+0000 7f441ce06640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44180cc6f0 0x7f441814a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.470+0000 7f441ce06640 1 --2- 172.21.15.29:0/2367003210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4400004d50 0x7f441815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.470+0000 7f441ce06640 1 -- 172.21.15.29:0/2367003210 >> 172.21.15.29:0/2367003210 conn(0x7f44180d4030 msgr2=0x7f44180b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:09.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.470+0000 7f441ce06640 1 -- 172.21.15.29:0/2367003210 shutdown_connections 2024-04-25T00:39:09.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.470+0000 7f441ce06640 1 -- 172.21.15.29:0/2367003210 wait complete. 2024-04-25T00:39:09.493 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.610+0000 7fc2fcf57640 1 Processor -- start 2024-04-25T00:39:09.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.610+0000 7fc2fcf57640 1 -- start start 2024-04-25T00:39:09.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2fcf57640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc2f806c950 0x7fc2f806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2fcf57640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc2f80cd410 0x7fc2f80cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2fcf57640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f80cc3b0 con 0x7fc2f815e440 2024-04-25T00:39:09.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2fcf57640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f806d270 con 0x7fc2f80cd410 2024-04-25T00:39:09.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2fcf57640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f80b11d0 con 0x7fc2f806c950 2024-04-25T00:39:09.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2f5d9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc2f815e440 0x7fc2f80d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39058/0 (socket says 172.21.15.29:39058) 2024-04-25T00:39:09.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2f6d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc2f80cd410 0x7fc2f80cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2f5d9b640 1 -- 172.21.15.29:0/1277242283 learned_addr learned my addr 172.21.15.29:0/1277242283 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:09.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2f659c640 1 --2- 172.21.15.29:0/1277242283 >> v2:172.21.15.157:3300/0 conn(0x7fc2f806c950 0x7fc2f806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2f6d9d640 1 -- 172.21.15.29:0/1277242283 >> v2:172.21.15.157:3300/0 conn(0x7fc2f806c950 msgr2=0x7fc2f806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:09.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2f6d9d640 1 --2- 172.21.15.29:0/1277242283 >> v2:172.21.15.157:3300/0 conn(0x7fc2f806c950 0x7fc2f806cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2f6d9d640 1 -- 172.21.15.29:0/1277242283 >> v1:172.21.15.29:6789/0 conn(0x7fc2f815e440 legacy=0x7fc2f80d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2f6d9d640 1 -- 172.21.15.29:0/1277242283 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f80cc650 con 0x7fc2f80cd410 2024-04-25T00:39:09.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2f6d9d640 1 --2- 172.21.15.29:0/1277242283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f80cd410 0x7fc2f80cd8a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fc2f806c110 tx=0x7fc2f001c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cb0bd09fc08581 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:09.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2f559a640 1 -- 172.21.15.29:0/1277242283 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00b2070 con 0x7fc2f80cd410 2024-04-25T00:39:09.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2f559a640 1 -- 172.21.15.29:0/1277242283 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f00a5820 con 0x7fc2f80cd410 2024-04-25T00:39:09.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2f559a640 1 -- 172.21.15.29:0/1277242283 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00a5bf0 con 0x7fc2f80cd410 2024-04-25T00:39:09.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2fcf57640 1 -- 172.21.15.29:0/1277242283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f80cd410 msgr2=0x7fc2f80cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2fcf57640 1 --2- 172.21.15.29:0/1277242283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f80cd410 0x7fc2f80cd8a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fc2f806c110 tx=0x7fc2f001c7c0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2fcf57640 1 -- 172.21.15.29:0/1277242283 shutdown_connections 2024-04-25T00:39:09.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2fcf57640 1 --2- 172.21.15.29:0/1277242283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f80cd410 0x7fc2f80cd8a0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2fcf57640 1 --2- 172.21.15.29:0/1277242283 >> v2:172.21.15.157:3300/0 conn(0x7fc2f806c950 0x7fc2f806cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2fcf57640 1 -- 172.21.15.29:0/1277242283 >> 172.21.15.29:0/1277242283 conn(0x7fc2f80d1d60 msgr2=0x7fc2f80d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:09.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2fcf57640 1 -- 172.21.15.29:0/1277242283 shutdown_connections 2024-04-25T00:39:09.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.614+0000 7fc2fcf57640 1 -- 172.21.15.29:0/1277242283 wait complete. 2024-04-25T00:39:09.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2fcf57640 1 Processor -- start 2024-04-25T00:39:09.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2fcf57640 1 -- start start 2024-04-25T00:39:09.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2fcf57640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2f806c950 0x7fc2f8152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2fcf57640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f80cd410 0x7fc2f814f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2fcf57640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f815e440 0x7fc2f81554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2fcf57640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f80ccef0 con 0x7fc2f80cd410 2024-04-25T00:39:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2fcf57640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f80ccc10 con 0x7fc2f806c950 2024-04-25T00:39:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2fcf57640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f806c520 con 0x7fc2f815e440 2024-04-25T00:39:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2f5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f80cd410 0x7fc2f814f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2f5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f80cd410 0x7fc2f814f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56862/0 (socket says 172.21.15.29:56862) 2024-04-25T00:39:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2f5d9b640 1 -- 172.21.15.29:0/2553058054 learned_addr learned my addr 172.21.15.29:0/2553058054 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2f659c640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2f806c950 0x7fc2f8152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2f6d9d640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f815e440 0x7fc2f81554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2f5d9b640 1 -- 172.21.15.29:0/2553058054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f815e440 msgr2=0x7fc2f81554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2f5d9b640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f815e440 0x7fc2f81554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2f5d9b640 1 -- 172.21.15.29:0/2553058054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2f806c950 msgr2=0x7fc2f8152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2f5d9b640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2f806c950 0x7fc2f8152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2f5d9b640 1 -- 172.21.15.29:0/2553058054 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f81533b0 con 0x7fc2f80cd410 2024-04-25T00:39:09.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2f659c640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2f806c950 0x7fc2f8152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:09.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2f5d9b640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f80cd410 0x7fc2f814f020 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009f30 tx=0x7fc2ec00c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:09.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2e6ffd640 1 -- 172.21.15.29:0/2553058054 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec019070 con 0x7fc2f80cd410 2024-04-25T00:39:09.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2e6ffd640 1 -- 172.21.15.29:0/2553058054 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2ec007af0 con 0x7fc2f80cd410 2024-04-25T00:39:09.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2e6ffd640 1 -- 172.21.15.29:0/2553058054 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec009070 con 0x7fc2f80cd410 2024-04-25T00:39:09.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2fcf57640 1 -- 172.21.15.29:0/2553058054 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f814fb90 con 0x7fc2f80cd410 2024-04-25T00:39:09.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.618+0000 7fc2fcf57640 1 -- 172.21.15.29:0/2553058054 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f8150150 con 0x7fc2f80cd410 2024-04-25T00:39:09.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.622+0000 7fc2fcf57640 1 -- 172.21.15.29:0/2553058054 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f8069270 con 0x7fc2f80cd410 2024-04-25T00:39:09.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.622+0000 7fc2e6ffd640 1 -- 172.21.15.29:0/2553058054 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fc2ec01c070 con 0x7fc2f80cd410 2024-04-25T00:39:09.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.622+0000 7fc2e6ffd640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2d8067e90 0x7fc2d806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.622+0000 7fc2f659c640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2d8067e90 0x7fc2d806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.622+0000 7fc2e6ffd640 1 -- 172.21.15.29:0/2553058054 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7fc2ec088540 con 0x7fc2f80cd410 2024-04-25T00:39:09.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.622+0000 7fc2f659c640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2d8067e90 0x7fc2d806a350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800b9f0 tx=0x7fc2e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:09.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.626+0000 7fc2e6ffd640 1 -- 172.21.15.29:0/2553058054 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc2ec0587d0 con 0x7fc2f80cd410 2024-04-25T00:39:09.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.790+0000 7fc2fcf57640 1 -- 172.21.15.29:0/2553058054 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2f806b6a0 con 0x7fc2f80cd410 2024-04-25T00:39:09.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.790+0000 7fc2e6ffd640 1 -- 172.21.15.29:0/2553058054 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16644 (secure 0 0 0) 0x7fc2ec0580d0 con 0x7fc2f80cd410 2024-04-25T00:39:09.795 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:09.796 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":131,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:08.508383+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:06.791988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:07.503204+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:09.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.794+0000 7fc2fcf57640 1 -- 172.21.15.29:0/2553058054 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2d8067e90 msgr2=0x7fc2d806a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.794+0000 7fc2fcf57640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2d8067e90 0x7fc2d806a350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800b9f0 tx=0x7fc2e8008040 comp rx=0 tx=0).stop 2024-04-25T00:39:09.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.794+0000 7fc2fcf57640 1 -- 172.21.15.29:0/2553058054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f80cd410 msgr2=0x7fc2f814f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.794+0000 7fc2fcf57640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f80cd410 0x7fc2f814f020 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009f30 tx=0x7fc2ec00c4f0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.794+0000 7fc2fcf57640 1 -- 172.21.15.29:0/2553058054 shutdown_connections 2024-04-25T00:39:09.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.794+0000 7fc2fcf57640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2d8067e90 0x7fc2d806a350 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.794+0000 7fc2fcf57640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f815e440 0x7fc2f81554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.794+0000 7fc2fcf57640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f80cd410 0x7fc2f814f020 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.794+0000 7fc2fcf57640 1 --2- 172.21.15.29:0/2553058054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2f806c950 0x7fc2f8152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.794+0000 7fc2fcf57640 1 -- 172.21.15.29:0/2553058054 >> 172.21.15.29:0/2553058054 conn(0x7fc2f80d1d60 msgr2=0x7fc2f806f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:09.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.794+0000 7fc2fcf57640 1 -- 172.21.15.29:0/2553058054 shutdown_connections 2024-04-25T00:39:09.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.794+0000 7fc2fcf57640 1 -- 172.21.15.29:0/2553058054 wait complete. 2024-04-25T00:39:09.815 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:39:09.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.934+0000 7f41ea996640 1 Processor -- start 2024-04-25T00:39:09.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.934+0000 7f41ea996640 1 -- start start 2024-04-25T00:39:09.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41ea996640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f41e415e250 0x7f41e40d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41ea996640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e40cfb00 0x7f41e40d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41ea996640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e40c8690 con 0x7f41e415e630 2024-04-25T00:39:09.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41ea996640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f41e40c8440 con 0x7f41e415e250 2024-04-25T00:39:09.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41ea996640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f41e40c8970 con 0x7f41e40cfb00 2024-04-25T00:39:09.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41db7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f41e415e630 0x7f41e40c9450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39068/0 (socket says 172.21.15.29:39068) 2024-04-25T00:39:09.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41db7fe640 1 -- 172.21.15.29:0/3756396139 learned_addr learned my addr 172.21.15.29:0/3756396139 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:09.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e3fff640 1 --2- 172.21.15.29:0/3756396139 >> v2:172.21.15.157:3300/0 conn(0x7f41e415e250 0x7f41e40d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e8f33640 1 --2- 172.21.15.29:0/3756396139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e40cfb00 0x7f41e40d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064599498 0 0) 0x7f41e40c8690 con 0x7f41e415e630 2024-04-25T00:39:09.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41d0003530 con 0x7f41e415e630 2024-04-25T00:39:09.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1494043492 0 0) 0x7f41d0003530 con 0x7f41e415e630 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e40cfb00 msgr2=0x7f41e40d29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 --2- 172.21.15.29:0/3756396139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e40cfb00 0x7f41e40d29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 >> v2:172.21.15.157:3300/0 conn(0x7f41e415e250 msgr2=0x7f41e40d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 --2- 172.21.15.29:0/3756396139 >> v2:172.21.15.157:3300/0 conn(0x7f41e415e250 0x7f41e40d24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e40c8c50 con 0x7f41e415e630 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f41cc0016a0 con 0x7f41e415e630 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 >> v1:172.21.15.29:6789/0 conn(0x7f41e415e630 legacy=0x7f41e40c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 --2- 172.21.15.29:0/3756396139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41d00039c0 0x7f41d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 --2- 172.21.15.29:0/3756396139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e40cfb00 0x7f41d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 --2- 172.21.15.29:0/3756396139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41d0004d50 0x7f41d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f41e40c8440 con 0x7f41d00039c0 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f41e40c8970 con 0x7f41d0004d50 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f41d000b9d0 con 0x7f41e40cfb00 2024-04-25T00:39:09.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e3fff640 1 --2- 172.21.15.29:0/3756396139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41d00039c0 0x7f41d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e8f33640 1 --2- 172.21.15.29:0/3756396139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41d0004d50 0x7f41d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41db7fe640 1 --2- 172.21.15.29:0/3756396139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e40cfb00 0x7f41d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e3fff640 1 -- 172.21.15.29:0/3756396139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e40cfb00 msgr2=0x7f41d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:09.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e3fff640 1 --2- 172.21.15.29:0/3756396139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e40cfb00 0x7f41d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e3fff640 1 -- 172.21.15.29:0/3756396139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41d0004d50 msgr2=0x7f41d00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:09.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e3fff640 1 --2- 172.21.15.29:0/3756396139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41d0004d50 0x7f41d00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e3fff640 1 -- 172.21.15.29:0/3756396139 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d0003530 con 0x7f41d00039c0 2024-04-25T00:39:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.938+0000 7f41e3fff640 1 --2- 172.21.15.29:0/3756396139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41d00039c0 0x7f41d0003de0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009ae0 tx=0x7f41d401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7504e961d6da5c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d40096b0 con 0x7f41d00039c0 2024-04-25T00:39:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d40a6490 con 0x7f41d00039c0 2024-04-25T00:39:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e37fe640 1 -- 172.21.15.29:0/3756396139 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d40a6880 con 0x7f41d00039c0 2024-04-25T00:39:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 -- 172.21.15.29:0/3756396139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41d00039c0 msgr2=0x7f41d0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 --2- 172.21.15.29:0/3756396139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41d00039c0 0x7f41d0003de0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009ae0 tx=0x7f41d401cab0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e8f33640 1 -- 172.21.15.29:0/3756396139 reap_dead start 2024-04-25T00:39:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 -- 172.21.15.29:0/3756396139 shutdown_connections 2024-04-25T00:39:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 -- 172.21.15.29:0/3756396139 >> 172.21.15.29:0/3756396139 conn(0x7f41e40d3f70 msgr2=0x7f41e40d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 -- 172.21.15.29:0/3756396139 shutdown_connections 2024-04-25T00:39:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 -- 172.21.15.29:0/3756396139 wait complete. 2024-04-25T00:39:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 Processor -- start 2024-04-25T00:39:09.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 -- start start 2024-04-25T00:39:09.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41d00039c0 0x7f41e415c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e40b6b70 0x7f41e40b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41e415b280 0x7f41e415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f41e40d2df0 con 0x7f41e415b280 2024-04-25T00:39:09.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f41e40d1ed0 con 0x7f41d00039c0 2024-04-25T00:39:09.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f41e40ca370 con 0x7f41e40b6b70 2024-04-25T00:39:09.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e8f33640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41e415b280 0x7f41e415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e8f33640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41e415b280 0x7f41e415b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56880/0 (socket says 172.21.15.29:56880) 2024-04-25T00:39:09.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e8f33640 1 -- 172.21.15.29:0/1362487878 learned_addr learned my addr 172.21.15.29:0/1362487878 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:09.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e3fff640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41d00039c0 0x7f41e415c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41db7fe640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e40b6b70 0x7f41e40b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e8f33640 1 -- 172.21.15.29:0/1362487878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e40b6b70 msgr2=0x7f41e40b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e8f33640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e40b6b70 0x7f41e40b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e8f33640 1 -- 172.21.15.29:0/1362487878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41d00039c0 msgr2=0x7f41e415c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:09.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e8f33640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41d00039c0 0x7f41e415c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:09.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e8f33640 1 -- 172.21.15.29:0/1362487878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e40b9580 con 0x7f41e415b280 2024-04-25T00:39:09.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e8f33640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41e415b280 0x7f41e415b730 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f41e4161ee0 tx=0x7f41dc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:09.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e17fa640 1 -- 172.21.15.29:0/1362487878 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc011e10 con 0x7f41e415b280 2024-04-25T00:39:09.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e17fa640 1 -- 172.21.15.29:0/1362487878 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41dc01b070 con 0x7f41e415b280 2024-04-25T00:39:09.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 -- 172.21.15.29:0/1362487878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e40b76e0 con 0x7f41e415b280 2024-04-25T00:39:09.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41e17fa640 1 -- 172.21.15.29:0/1362487878 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0039a0 con 0x7f41e415b280 2024-04-25T00:39:09.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.942+0000 7f41ea996640 1 -- 172.21.15.29:0/1362487878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e40b7ca0 con 0x7f41e415b280 2024-04-25T00:39:09.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.946+0000 7f41ea996640 1 -- 172.21.15.29:0/1362487878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e4069270 con 0x7f41e415b280 2024-04-25T00:39:09.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.946+0000 7f41e17fa640 1 -- 172.21.15.29:0/1362487878 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f41dc003b60 con 0x7f41e415b280 2024-04-25T00:39:09.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.946+0000 7f41e17fa640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f41c4067e90 0x7f41c406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:09.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.946+0000 7f41e17fa640 1 -- 172.21.15.29:0/1362487878 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5709+0+0 (secure 0 0 0) 0x7f41dc086d50 con 0x7f41e415b280 2024-04-25T00:39:09.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.946+0000 7f41e3fff640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f41c4067e90 0x7f41c406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:09.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.946+0000 7f41e3fff640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f41c4067e90 0x7f41c406a350 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f41d40ad000 tx=0x7f41d40b3040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:09.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:09.950+0000 7f41e17fa640 1 -- 172.21.15.29:0/1362487878 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f41dc08d080 con 0x7f41e415b280 2024-04-25T00:39:10.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:10.150+0000 7f41ea996640 1 -- 172.21.15.29:0/1362487878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f41e4067a10 con 0x7f41e415b280 2024-04-25T00:39:10.608 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T00:39:10.608 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T00:39:11.030 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:39:11.033 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:39:11.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.154+0000 7f41e17fa640 1 -- 172.21.15.29:0/1362487878 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v82) ==== 118+0+0 (secure 0 0 0) 0x7f41dc057070 con 0x7f41e415b280 2024-04-25T00:39:11.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.158+0000 7f41ea996640 1 -- 172.21.15.29:0/1362487878 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f41c4067e90 msgr2=0x7f41c406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:11.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.158+0000 7f41ea996640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f41c4067e90 0x7f41c406a350 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f41d40ad000 tx=0x7f41d40b3040 comp rx=0 tx=0).stop 2024-04-25T00:39:11.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.158+0000 7f41ea996640 1 -- 172.21.15.29:0/1362487878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41e415b280 msgr2=0x7f41e415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:11.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.158+0000 7f41ea996640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41e415b280 0x7f41e415b730 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f41e4161ee0 tx=0x7f41dc0115a0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.158+0000 7f41ea996640 1 -- 172.21.15.29:0/1362487878 shutdown_connections 2024-04-25T00:39:11.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.158+0000 7f41ea996640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f41c4067e90 0x7f41c406a350 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.158+0000 7f41ea996640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41e415b280 0x7f41e415b730 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.158+0000 7f41ea996640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41e40b6b70 0x7f41e40b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.158+0000 7f41ea996640 1 --2- 172.21.15.29:0/1362487878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41d00039c0 0x7f41e415c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.158+0000 7f41ea996640 1 -- 172.21.15.29:0/1362487878 >> 172.21.15.29:0/1362487878 conn(0x7f41e40d3f70 msgr2=0x7f41e415eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:11.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.158+0000 7f41ea996640 1 -- 172.21.15.29:0/1362487878 shutdown_connections 2024-04-25T00:39:11.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.158+0000 7f41ea996640 1 -- 172.21.15.29:0/1362487878 wait complete. 2024-04-25T00:39:11.185 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:39:11.185 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:39:11.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90b485e640 1 Processor -- start 2024-04-25T00:39:11.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90b485e640 1 -- start start 2024-04-25T00:39:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90b485e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90b00cf690 0x7f90b00d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90b485e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f90b00d2a90 0x7f90b00b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90b485e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b00d2ea0 con 0x7f90b015e200 2024-04-25T00:39:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90b485e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f90b00b38d0 con 0x7f90b00d2a90 2024-04-25T00:39:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90b485e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f90b00cfc10 con 0x7f90b00cf690 2024-04-25T00:39:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90ae59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f90b015e200 0x7f90b00d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39080/0 (socket says 172.21.15.29:39080) 2024-04-25T00:39:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90ae59c640 1 -- 172.21.15.29:0/2148860533 learned_addr learned my addr 172.21.15.29:0/2148860533 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90aed9d640 1 --2- 172.21.15.29:0/2148860533 >> v2:172.21.15.29:3300/0 conn(0x7f90b00d2a90 0x7f90b00b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90ad59a640 1 -- 172.21.15.29:0/2148860533 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 280699120 0 0) 0x7f90a0001120 con 0x7f90b015e200 2024-04-25T00:39:11.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90add9b640 1 --2- 172.21.15.29:0/2148860533 >> v2:172.21.15.157:3300/0 conn(0x7f90b00cf690 0x7f90b00d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90ad59a640 1 -- 172.21.15.29:0/2148860533 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f909c003530 con 0x7f90b015e200 2024-04-25T00:39:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90aed9d640 1 -- 172.21.15.29:0/2148860533 >> v2:172.21.15.157:3300/0 conn(0x7f90b00cf690 msgr2=0x7f90b00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90aed9d640 1 --2- 172.21.15.29:0/2148860533 >> v2:172.21.15.157:3300/0 conn(0x7f90b00cf690 0x7f90b00d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90aed9d640 1 -- 172.21.15.29:0/2148860533 >> v1:172.21.15.29:6789/0 conn(0x7f90b015e200 legacy=0x7f90b00d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90aed9d640 1 -- 172.21.15.29:0/2148860533 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b00cfef0 con 0x7f90b00d2a90 2024-04-25T00:39:11.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.318+0000 7f90add9b640 1 --2- 172.21.15.29:0/2148860533 >> v2:172.21.15.157:3300/0 conn(0x7f90b00cf690 0x7f90b00d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90aed9d640 1 --2- 172.21.15.29:0/2148860533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b00d2a90 0x7f90b00b4320 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f90b00d24a0 tx=0x7f90a801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a7cc9994370e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90ad59a640 1 -- 172.21.15.29:0/2148860533 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80b2070 con 0x7f90b00d2a90 2024-04-25T00:39:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90ad59a640 1 -- 172.21.15.29:0/2148860533 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a80a58e0 con 0x7f90b00d2a90 2024-04-25T00:39:11.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90ad59a640 1 -- 172.21.15.29:0/2148860533 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80a5c90 con 0x7f90b00d2a90 2024-04-25T00:39:11.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90b485e640 1 -- 172.21.15.29:0/2148860533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b00d2a90 msgr2=0x7f90b00b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:11.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90b485e640 1 --2- 172.21.15.29:0/2148860533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b00d2a90 0x7f90b00b4320 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f90b00d24a0 tx=0x7f90a801c930 comp rx=0 tx=0).stop 2024-04-25T00:39:11.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90b485e640 1 -- 172.21.15.29:0/2148860533 shutdown_connections 2024-04-25T00:39:11.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90b485e640 1 --2- 172.21.15.29:0/2148860533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b00d2a90 0x7f90b00b4320 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90b485e640 1 --2- 172.21.15.29:0/2148860533 >> v2:172.21.15.157:3300/0 conn(0x7f90b00cf690 0x7f90b00d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90b485e640 1 -- 172.21.15.29:0/2148860533 >> 172.21.15.29:0/2148860533 conn(0x7f90b009d280 msgr2=0x7f90b00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:11.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90b485e640 1 -- 172.21.15.29:0/2148860533 shutdown_connections 2024-04-25T00:39:11.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90b485e640 1 -- 172.21.15.29:0/2148860533 wait complete. 2024-04-25T00:39:11.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90b485e640 1 Processor -- start 2024-04-25T00:39:11.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.322+0000 7f90b485e640 1 -- start start 2024-04-25T00:39:11.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90b485e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b00cf690 0x7f90b0154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90b485e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00d2a90 0x7f90b0154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90b485e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b015e200 0x7f90b00c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90b485e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f90b00b51f0 con 0x7f90b015e200 2024-04-25T00:39:11.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90b485e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90b00b5070 con 0x7f90b00cf690 2024-04-25T00:39:11.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90b485e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f90b00b5370 con 0x7f90b00d2a90 2024-04-25T00:39:11.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90aed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b015e200 0x7f90b00c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90aed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b015e200 0x7f90b00c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56886/0 (socket says 172.21.15.29:56886) 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90aed9d640 1 -- 172.21.15.29:0/2474222817 learned_addr learned my addr 172.21.15.29:0/2474222817 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90ae59c640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b00cf690 0x7f90b0154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90add9b640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00d2a90 0x7f90b0154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90aed9d640 1 -- 172.21.15.29:0/2474222817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00d2a90 msgr2=0x7f90b0154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90aed9d640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00d2a90 0x7f90b0154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90aed9d640 1 -- 172.21.15.29:0/2474222817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b00cf690 msgr2=0x7f90b0154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90aed9d640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b00cf690 0x7f90b0154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90aed9d640 1 -- 172.21.15.29:0/2474222817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b00c7f60 con 0x7f90b015e200 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90ae59c640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b00cf690 0x7f90b0154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90aed9d640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b015e200 0x7f90b00c9000 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f90a801a560 tx=0x7f90a80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90a6ffd640 1 -- 172.21.15.29:0/2474222817 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80b2050 con 0x7f90b015e200 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90a6ffd640 1 -- 172.21.15.29:0/2474222817 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a80be070 con 0x7f90b015e200 2024-04-25T00:39:11.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90a6ffd640 1 -- 172.21.15.29:0/2474222817 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80ada90 con 0x7f90b015e200 2024-04-25T00:39:11.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90b485e640 1 -- 172.21.15.29:0/2474222817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b00c8240 con 0x7f90b015e200 2024-04-25T00:39:11.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90b485e640 1 -- 172.21.15.29:0/2474222817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b00c8800 con 0x7f90b015e200 2024-04-25T00:39:11.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90a6ffd640 1 -- 172.21.15.29:0/2474222817 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f90a80a5760 con 0x7f90b015e200 2024-04-25T00:39:11.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90b485e640 1 -- 172.21.15.29:0/2474222817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b0069250 con 0x7f90b015e200 2024-04-25T00:39:11.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90a6ffd640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9090067e60 0x7f909006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90a6ffd640 1 -- 172.21.15.29:0/2474222817 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f90a81219c0 con 0x7f90b015e200 2024-04-25T00:39:11.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.326+0000 7f90ae59c640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9090067e60 0x7f909006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.330+0000 7f90ae59c640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9090067e60 0x7f909006a320 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f90a00012c0 tx=0x7f90a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:11.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.334+0000 7f90a6ffd640 1 -- 172.21.15.29:0/2474222817 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f90a8016520 con 0x7f90b015e200 2024-04-25T00:39:11.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:11.530+0000 7f90b485e640 1 -- 172.21.15.29:0/2474222817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f90b00d02b0 con 0x7f90b015e200 2024-04-25T00:39:12.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.170+0000 7f90a6ffd640 1 -- 172.21.15.29:0/2474222817 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v83) ==== 122+0+0 (secure 0 0 0) 0x7f90a80f1bc0 con 0x7f90b015e200 2024-04-25T00:39:12.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.174+0000 7f90b485e640 1 -- 172.21.15.29:0/2474222817 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9090067e60 msgr2=0x7f909006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.174+0000 7f90b485e640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9090067e60 0x7f909006a320 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f90a00012c0 tx=0x7f90a0009040 comp rx=0 tx=0).stop 2024-04-25T00:39:12.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.174+0000 7f90b485e640 1 -- 172.21.15.29:0/2474222817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b015e200 msgr2=0x7f90b00c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.174+0000 7f90b485e640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b015e200 0x7f90b00c9000 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f90a801a560 tx=0x7f90a80196d0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.174+0000 7f90b485e640 1 -- 172.21.15.29:0/2474222817 shutdown_connections 2024-04-25T00:39:12.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.174+0000 7f90b485e640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9090067e60 0x7f909006a320 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.174+0000 7f90b485e640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90b015e200 0x7f90b00c9000 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.174+0000 7f90b485e640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90b00d2a90 0x7f90b0154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.174+0000 7f90b485e640 1 --2- 172.21.15.29:0/2474222817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90b00cf690 0x7f90b0154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.174+0000 7f90b485e640 1 -- 172.21.15.29:0/2474222817 >> 172.21.15.29:0/2474222817 conn(0x7f90b009d280 msgr2=0x7f90b015d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:12.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.174+0000 7f90b485e640 1 -- 172.21.15.29:0/2474222817 shutdown_connections 2024-04-25T00:39:12.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.174+0000 7f90b485e640 1 -- 172.21.15.29:0/2474222817 wait complete. 2024-04-25T00:39:12.199 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:39:12.199 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:39:12.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f3855d0f640 1 Processor -- start 2024-04-25T00:39:12.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f3855d0f640 1 -- start start 2024-04-25T00:39:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f3855d0f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38500d2bb0 0x7f38500ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f3855d0f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f385015e630 0x7f38500cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f3855d0f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38500c8690 con 0x7f385015e250 2024-04-25T00:39:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f3855d0f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f38500c8440 con 0x7f38500d2bb0 2024-04-25T00:39:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f3855d0f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f38500c8970 con 0x7f385015e630 2024-04-25T00:39:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f384effd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38500d2bb0 0x7f38500ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f384effd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38500d2bb0 0x7f38500ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56896/0 (socket says 172.21.15.29:56896) 2024-04-25T00:39:12.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f384effd640 1 -- 172.21.15.29:0/1451352301 learned_addr learned my addr 172.21.15.29:0/1451352301 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:12.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f384ffff640 1 --2- 172.21.15.29:0/1451352301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f385015e630 0x7f38500cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f384effd640 1 -- 172.21.15.29:0/1451352301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f385015e630 msgr2=0x7f38500cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:12.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f384effd640 1 --2- 172.21.15.29:0/1451352301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f385015e630 0x7f38500cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f384effd640 1 -- 172.21.15.29:0/1451352301 >> v1:172.21.15.157:6789/0 conn(0x7f385015e250 legacy=0x7f38500d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.326+0000 7f384effd640 1 -- 172.21.15.29:0/1451352301 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38500c8c50 con 0x7f38500d2bb0 2024-04-25T00:39:12.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f384effd640 1 --2- 172.21.15.29:0/1451352301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38500d2bb0 0x7f38500ceaf0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f3844009870 tx=0x7f384401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab27ef06e91236d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:12.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f384e7fc640 1 -- 172.21.15.29:0/1451352301 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38440b1070 con 0x7f38500d2bb0 2024-04-25T00:39:12.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f384e7fc640 1 -- 172.21.15.29:0/1451352301 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38440a8050 con 0x7f38500d2bb0 2024-04-25T00:39:12.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f384e7fc640 1 -- 172.21.15.29:0/1451352301 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38440ac860 con 0x7f38500d2bb0 2024-04-25T00:39:12.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f3855d0f640 1 -- 172.21.15.29:0/1451352301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38500d2bb0 msgr2=0x7f38500ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f3855d0f640 1 --2- 172.21.15.29:0/1451352301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38500d2bb0 0x7f38500ceaf0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f3844009870 tx=0x7f384401c4d0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f3855d0f640 1 -- 172.21.15.29:0/1451352301 shutdown_connections 2024-04-25T00:39:12.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f3855d0f640 1 --2- 172.21.15.29:0/1451352301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f385015e630 0x7f38500cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f3855d0f640 1 --2- 172.21.15.29:0/1451352301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38500d2bb0 0x7f38500ceaf0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f3855d0f640 1 -- 172.21.15.29:0/1451352301 >> 172.21.15.29:0/1451352301 conn(0x7f38500d3f70 msgr2=0x7f38500d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:12.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f3855d0f640 1 -- 172.21.15.29:0/1451352301 shutdown_connections 2024-04-25T00:39:12.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f3855d0f640 1 -- 172.21.15.29:0/1451352301 wait complete. 2024-04-25T00:39:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f3855d0f640 1 Processor -- start 2024-04-25T00:39:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.330+0000 7f3855d0f640 1 -- start start 2024-04-25T00:39:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3855d0f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38500d2bb0 0x7f38501618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3855d0f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f385015e250 0x7f385015b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3855d0f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f385015e630 0x7f3850151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3855d0f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f385016aff0 con 0x7f38500d2bb0 2024-04-25T00:39:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3855d0f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38500d1ed0 con 0x7f385015e630 2024-04-25T00:39:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3855d0f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f385016b170 con 0x7f385015e250 2024-04-25T00:39:12.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38500d2bb0 0x7f38501618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38500d2bb0 0x7f38501618f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56910/0 (socket says 172.21.15.29:56910) 2024-04-25T00:39:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384f7fe640 1 -- 172.21.15.29:0/3086220345 learned_addr learned my addr 172.21.15.29:0/3086220345 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384ffff640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f385015e630 0x7f3850151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384effd640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f385015e250 0x7f385015b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384f7fe640 1 -- 172.21.15.29:0/3086220345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f385015e250 msgr2=0x7f385015b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384f7fe640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f385015e250 0x7f385015b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384f7fe640 1 -- 172.21.15.29:0/3086220345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f385015e630 msgr2=0x7f3850151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384f7fe640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f385015e630 0x7f3850151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384f7fe640 1 -- 172.21.15.29:0/3086220345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3850068620 con 0x7f38500d2bb0 2024-04-25T00:39:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384effd640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f385015e250 0x7f385015b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384f7fe640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38500d2bb0 0x7f38501618f0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f384000c450 tx=0x7f384000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3837fff640 1 -- 172.21.15.29:0/3086220345 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840016020 con 0x7f38500d2bb0 2024-04-25T00:39:12.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3837fff640 1 -- 172.21.15.29:0/3086220345 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38400052d0 con 0x7f38500d2bb0 2024-04-25T00:39:12.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3855d0f640 1 -- 172.21.15.29:0/3086220345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385015c460 con 0x7f38500d2bb0 2024-04-25T00:39:12.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3837fff640 1 -- 172.21.15.29:0/3086220345 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840007260 con 0x7f38500d2bb0 2024-04-25T00:39:12.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3855d0f640 1 -- 172.21.15.29:0/3086220345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385015bea0 con 0x7f38500d2bb0 2024-04-25T00:39:12.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3855d0f640 1 -- 172.21.15.29:0/3086220345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3850069270 con 0x7f38500d2bb0 2024-04-25T00:39:12.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3837fff640 1 -- 172.21.15.29:0/3086220345 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f3840007400 con 0x7f38500d2bb0 2024-04-25T00:39:12.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3837fff640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f383c067e60 0x7f383c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384effd640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f383c067e60 0x7f383c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f3837fff640 1 -- 172.21.15.29:0/3086220345 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f3840086030 con 0x7f38500d2bb0 2024-04-25T00:39:12.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.334+0000 7f384effd640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f383c067e60 0x7f383c06a320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f38440ae000 tx=0x7f38440a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:12.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.338+0000 7f3837fff640 1 -- 172.21.15.29:0/3086220345 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3840020020 con 0x7f38500d2bb0 2024-04-25T00:39:12.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.534+0000 7f3855d0f640 1 -- 172.21.15.29:0/3086220345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3850157b70 con 0x7f38500d2bb0 2024-04-25T00:39:12.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.538+0000 7f3837fff640 1 -- 172.21.15.29:0/3086220345 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5179 (secure 0 0 0) 0x7f3840005480 con 0x7f38500d2bb0 2024-04-25T00:39:12.540 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:12.540 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":83,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7450,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/456996219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":456996219},{"type":"v1","addr":"172.21.15.157:6833","nonce":456996219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/87071339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":87071339},{"type":"v1","addr":"172.21.15.157:6835","nonce":87071339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3920208567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3920208567},{"type":"v1","addr":"172.21.15.29:6833","nonce":3920208567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:39:08.508825+0000","modified":"2024-04-25T00:39:11.535481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9021":{"gid":9021,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/619209926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":619209926},{"type":"v1","addr":"172.21.15.29:6835","nonce":619209926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9021,"qdb_cluster":[9021]},"id":10}]} 2024-04-25T00:39:12.541 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 83 2024-04-25T00:39:12.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.538+0000 7f3855d0f640 1 -- 172.21.15.29:0/3086220345 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f383c067e60 msgr2=0x7f383c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.538+0000 7f3855d0f640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f383c067e60 0x7f383c06a320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f38440ae000 tx=0x7f38440a7000 comp rx=0 tx=0).stop 2024-04-25T00:39:12.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.538+0000 7f3855d0f640 1 -- 172.21.15.29:0/3086220345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38500d2bb0 msgr2=0x7f38501618f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.538+0000 7f3855d0f640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38500d2bb0 0x7f38501618f0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f384000c450 tx=0x7f384000c910 comp rx=0 tx=0).stop 2024-04-25T00:39:12.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.538+0000 7f3855d0f640 1 -- 172.21.15.29:0/3086220345 shutdown_connections 2024-04-25T00:39:12.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.538+0000 7f3855d0f640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f383c067e60 0x7f383c06a320 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.538+0000 7f3855d0f640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f385015e630 0x7f3850151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.538+0000 7f3855d0f640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f385015e250 0x7f385015b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.538+0000 7f3855d0f640 1 --2- 172.21.15.29:0/3086220345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38500d2bb0 0x7f38501618f0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.538+0000 7f3855d0f640 1 -- 172.21.15.29:0/3086220345 >> 172.21.15.29:0/3086220345 conn(0x7f38500d3f70 msgr2=0x7f385015ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:12.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.538+0000 7f3855d0f640 1 -- 172.21.15.29:0/3086220345 shutdown_connections 2024-04-25T00:39:12.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.538+0000 7f3855d0f640 1 -- 172.21.15.29:0/3086220345 wait complete. 2024-04-25T00:39:12.560 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:12.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.674+0000 7f13eba81640 1 Processor -- start 2024-04-25T00:39:12.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.674+0000 7f13eba81640 1 -- start start 2024-04-25T00:39:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13eba81640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13e406e440 0x7f13e40b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13eba81640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e40b41e0 con 0x7f13e40b3c30 2024-04-25T00:39:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13eba81640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e406d960 con 0x7f13e40b3140 2024-04-25T00:39:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13eba81640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f13e40b5050 con 0x7f13e406e440 2024-04-25T00:39:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e901c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f13e40b3c30 0x7f13e40ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39084/0 (socket says 172.21.15.29:39084) 2024-04-25T00:39:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e901c640 1 -- 172.21.15.29:0/149137364 learned_addr learned my addr 172.21.15.29:0/149137364 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13ea01e640 1 --2- 172.21.15.29:0/149137364 >> v2:172.21.15.157:3300/0 conn(0x7f13e406e440 0x7f13e40b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3101177640 0 0) 0x7f13e40b41e0 con 0x7f13e40b3c30 2024-04-25T00:39:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13d0003530 con 0x7f13e40b3c30 2024-04-25T00:39:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1757007220 0 0) 0x7f13d0003530 con 0x7f13e40b3c30 2024-04-25T00:39:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 >> v2:172.21.15.157:3300/0 conn(0x7f13e406e440 msgr2=0x7f13e40b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 --2- 172.21.15.29:0/149137364 >> v2:172.21.15.157:3300/0 conn(0x7f13e406e440 0x7f13e40b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 >> v1:172.21.15.157:6789/0 conn(0x7f13e40b3140 legacy=0x7f13e40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e406dc40 con 0x7f13e40b3c30 2024-04-25T00:39:12.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f13e0001680 con 0x7f13e40b3c30 2024-04-25T00:39:12.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 >> v1:172.21.15.29:6789/0 conn(0x7f13e40b3c30 legacy=0x7f13e40ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 --2- 172.21.15.29:0/149137364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d00039c0 0x7f13d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 --2- 172.21.15.29:0/149137364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d0004810 0x7f13d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 --2- 172.21.15.29:0/149137364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d00051d0 0x7f13d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f13e40b5050 con 0x7f13d00039c0 2024-04-25T00:39:12.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13d000c070 con 0x7f13d00051d0 2024-04-25T00:39:12.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f13d000c270 con 0x7f13d0004810 2024-04-25T00:39:12.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e981d640 1 --2- 172.21.15.29:0/149137364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d00039c0 0x7f13d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e901c640 1 --2- 172.21.15.29:0/149137364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d0004810 0x7f13d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13ea01e640 1 --2- 172.21.15.29:0/149137364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d00051d0 0x7f13d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e981d640 1 -- 172.21.15.29:0/149137364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d0004810 msgr2=0x7f13d0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:12.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e981d640 1 --2- 172.21.15.29:0/149137364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d0004810 0x7f13d0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e981d640 1 -- 172.21.15.29:0/149137364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d00051d0 msgr2=0x7f13d000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:12.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e981d640 1 --2- 172.21.15.29:0/149137364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d00051d0 0x7f13d000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e981d640 1 -- 172.21.15.29:0/149137364 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d0003530 con 0x7f13d00039c0 2024-04-25T00:39:12.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13ea01e640 1 --2- 172.21.15.29:0/149137364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d00051d0 0x7f13d000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:12.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13ea01e640 1 -- 172.21.15.29:0/149137364 reap_dead start 2024-04-25T00:39:12.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.678+0000 7f13e901c640 1 --2- 172.21.15.29:0/149137364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d0004810 0x7f13d0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:12.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.682+0000 7f13e981d640 1 --2- 172.21.15.29:0/149137364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d00039c0 0x7f13d0003de0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f13d4001780 tx=0x7f13d401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83963521e6c75dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:12.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.682+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d40013a0 con 0x7f13d00039c0 2024-04-25T00:39:12.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.682+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d40052f0 con 0x7f13d00039c0 2024-04-25T00:39:12.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.682+0000 7f13e881b640 1 -- 172.21.15.29:0/149137364 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4003840 con 0x7f13d00039c0 2024-04-25T00:39:12.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.682+0000 7f13eba81640 1 -- 172.21.15.29:0/149137364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d00039c0 msgr2=0x7f13d0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.682+0000 7f13eba81640 1 --2- 172.21.15.29:0/149137364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d00039c0 0x7f13d0003de0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f13d4001780 tx=0x7f13d401c400 comp rx=0 tx=0).stop 2024-04-25T00:39:12.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.682+0000 7f13eba81640 1 -- 172.21.15.29:0/149137364 shutdown_connections 2024-04-25T00:39:12.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.682+0000 7f13eba81640 1 --2- 172.21.15.29:0/149137364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d00039c0 0x7f13d0003de0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.682+0000 7f13eba81640 1 -- 172.21.15.29:0/149137364 >> 172.21.15.29:0/149137364 conn(0x7f13e40b1870 msgr2=0x7f13e40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:12.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.682+0000 7f13eba81640 1 -- 172.21.15.29:0/149137364 shutdown_connections 2024-04-25T00:39:12.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.682+0000 7f13eba81640 1 -- 172.21.15.29:0/149137364 wait complete. 2024-04-25T00:39:12.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.682+0000 7f13eba81640 1 Processor -- start 2024-04-25T00:39:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.682+0000 7f13eba81640 1 -- start start 2024-04-25T00:39:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13eba81640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d00039c0 0x7f13e41504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13eba81640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e41409a0 0x7f13e4141770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13eba81640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e4141cb0 0x7f13e40af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13eba81640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f13e40b3520 con 0x7f13d00039c0 2024-04-25T00:39:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13eba81640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13e406e820 con 0x7f13e41409a0 2024-04-25T00:39:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13eba81640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f13e40b36a0 con 0x7f13e4141cb0 2024-04-25T00:39:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13e981d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d00039c0 0x7f13e41504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13e981d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d00039c0 0x7f13e41504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56920/0 (socket says 172.21.15.29:56920) 2024-04-25T00:39:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13e981d640 1 -- 172.21.15.29:0/2573783510 learned_addr learned my addr 172.21.15.29:0/2573783510 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:12.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13e981d640 1 -- 172.21.15.29:0/2573783510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e4141cb0 msgr2=0x7f13e40af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13e981d640 1 --2- 172.21.15.29:0/2573783510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e4141cb0 0x7f13e40af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13e981d640 1 -- 172.21.15.29:0/2573783510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e41409a0 msgr2=0x7f13e4141770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13e981d640 1 --2- 172.21.15.29:0/2573783510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e41409a0 0x7f13e4141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13e981d640 1 -- 172.21.15.29:0/2573783510 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e4071e90 con 0x7f13d00039c0 2024-04-25T00:39:12.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13e981d640 1 --2- 172.21.15.29:0/2573783510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d00039c0 0x7f13e41504a0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f13d4001780 tx=0x7f13d0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:12.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13da7fc640 1 -- 172.21.15.29:0/2573783510 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d40013a0 con 0x7f13d00039c0 2024-04-25T00:39:12.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13da7fc640 1 -- 172.21.15.29:0/2573783510 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d40bb070 con 0x7f13d00039c0 2024-04-25T00:39:12.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13da7fc640 1 -- 172.21.15.29:0/2573783510 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4006a20 con 0x7f13d00039c0 2024-04-25T00:39:12.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13eba81640 1 -- 172.21.15.29:0/2573783510 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e4157ae0 con 0x7f13d00039c0 2024-04-25T00:39:12.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13eba81640 1 -- 172.21.15.29:0/2573783510 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e4072110 con 0x7f13d00039c0 2024-04-25T00:39:12.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13eba81640 1 -- 172.21.15.29:0/2573783510 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e4069270 con 0x7f13d00039c0 2024-04-25T00:39:12.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13da7fc640 1 -- 172.21.15.29:0/2573783510 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f13d40b4040 con 0x7f13d00039c0 2024-04-25T00:39:12.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13da7fc640 1 --2- 172.21.15.29:0/2573783510 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13c8067e60 0x7f13c806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13da7fc640 1 -- 172.21.15.29:0/2573783510 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f13d4120b40 con 0x7f13d00039c0 2024-04-25T00:39:12.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13e901c640 1 --2- 172.21.15.29:0/2573783510 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13c8067e60 0x7f13c806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.686+0000 7f13e901c640 1 --2- 172.21.15.29:0/2573783510 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13c8067e60 0x7f13c806a320 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f13e0001210 tx=0x7f13e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:12.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.690+0000 7f13da7fc640 1 -- 172.21.15.29:0/2573783510 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f13e4071e90 con 0x7f13d00039c0 2024-04-25T00:39:12.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.846+0000 7f13eba81640 1 -- 172.21.15.29:0/2573783510 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13e4142eb0 con 0x7f13d00039c0 2024-04-25T00:39:12.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.850+0000 7f13da7fc640 1 -- 172.21.15.29:0/2573783510 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+16631 (secure 0 0 0) 0x7f13d4016110 con 0x7f13d00039c0 2024-04-25T00:39:12.854 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:12.854 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":132,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:10.130521+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:06.791988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:07.503204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:12.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.854+0000 7f13eba81640 1 -- 172.21.15.29:0/2573783510 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13c8067e60 msgr2=0x7f13c806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.854+0000 7f13eba81640 1 --2- 172.21.15.29:0/2573783510 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13c8067e60 0x7f13c806a320 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f13e0001210 tx=0x7f13e0009040 comp rx=0 tx=0).stop 2024-04-25T00:39:12.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.854+0000 7f13eba81640 1 -- 172.21.15.29:0/2573783510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d00039c0 msgr2=0x7f13e41504a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.854+0000 7f13eba81640 1 --2- 172.21.15.29:0/2573783510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d00039c0 0x7f13e41504a0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f13d4001780 tx=0x7f13d0005840 comp rx=0 tx=0).stop 2024-04-25T00:39:12.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.854+0000 7f13eba81640 1 -- 172.21.15.29:0/2573783510 shutdown_connections 2024-04-25T00:39:12.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.854+0000 7f13eba81640 1 --2- 172.21.15.29:0/2573783510 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13c8067e60 0x7f13c806a320 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.854+0000 7f13eba81640 1 --2- 172.21.15.29:0/2573783510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e4141cb0 0x7f13e40af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.854+0000 7f13eba81640 1 --2- 172.21.15.29:0/2573783510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e41409a0 0x7f13e4141770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.854+0000 7f13eba81640 1 --2- 172.21.15.29:0/2573783510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d00039c0 0x7f13e41504a0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.854+0000 7f13eba81640 1 -- 172.21.15.29:0/2573783510 >> 172.21.15.29:0/2573783510 conn(0x7f13e40b1870 msgr2=0x7f13e415cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:12.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.854+0000 7f13eba81640 1 -- 172.21.15.29:0/2573783510 shutdown_connections 2024-04-25T00:39:12.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.854+0000 7f13eba81640 1 -- 172.21.15.29:0/2573783510 wait complete. 2024-04-25T00:39:12.871 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:39:12.872 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:39:12.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.990+0000 7f610461a640 1 Processor -- start 2024-04-25T00:39:12.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.990+0000 7f610461a640 1 -- start start 2024-04-25T00:39:12.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.994+0000 7f610461a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f60fc15e2a0 0x7f60fc0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.994+0000 7f610461a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f60fc0b4040 0x7f60fc0b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:12.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.994+0000 7f610461a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60fc0cf8a0 con 0x7f60fc0d4d10 2024-04-25T00:39:12.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.994+0000 7f610461a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f60fc0d19a0 con 0x7f60fc15e2a0 2024-04-25T00:39:12.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.994+0000 7f610461a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f60fc0cf650 con 0x7f60fc0b4040 2024-04-25T00:39:12.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.994+0000 7f61023b6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f60fc15e2a0 0x7f60fc0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.994+0000 7f61023b6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f60fc15e2a0 0x7f60fc0d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56928/0 (socket says 172.21.15.29:56928) 2024-04-25T00:39:12.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.994+0000 7f61023b6640 1 -- 172.21.15.29:0/2609580574 learned_addr learned my addr 172.21.15.29:0/2609580574 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:12.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.994+0000 7f6102bb7640 1 --2- 172.21.15.29:0/2609580574 >> v2:172.21.15.157:3300/0 conn(0x7f60fc0b4040 0x7f60fc0b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:12.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.994+0000 7f61023b6640 1 -- 172.21.15.29:0/2609580574 >> v2:172.21.15.157:3300/0 conn(0x7f60fc0b4040 msgr2=0x7f60fc0b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:12.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.994+0000 7f61023b6640 1 --2- 172.21.15.29:0/2609580574 >> v2:172.21.15.157:3300/0 conn(0x7f60fc0b4040 0x7f60fc0b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.994+0000 7f61023b6640 1 -- 172.21.15.29:0/2609580574 >> v1:172.21.15.157:6789/0 conn(0x7f60fc0d4d10 legacy=0x7f60fc0d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:12.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.994+0000 7f61023b6640 1 -- 172.21.15.29:0/2609580574 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc0cfb80 con 0x7f60fc15e2a0 2024-04-25T00:39:13.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f61023b6640 1 --2- 172.21.15.29:0/2609580574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60fc15e2a0 0x7f60fc0d1c00 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f60fc08c880 tx=0x7f60f801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2bd49792759a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:13.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f61013b4640 1 -- 172.21.15.29:0/2609580574 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f80b0070 con 0x7f60fc15e2a0 2024-04-25T00:39:13.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f61013b4640 1 -- 172.21.15.29:0/2609580574 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f80ab600 con 0x7f60fc15e2a0 2024-04-25T00:39:13.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f61013b4640 1 -- 172.21.15.29:0/2609580574 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f80ab9d0 con 0x7f60fc15e2a0 2024-04-25T00:39:13.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f610461a640 1 -- 172.21.15.29:0/2609580574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60fc15e2a0 msgr2=0x7f60fc0d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f610461a640 1 --2- 172.21.15.29:0/2609580574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60fc15e2a0 0x7f60fc0d1c00 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f60fc08c880 tx=0x7f60f801c450 comp rx=0 tx=0).stop 2024-04-25T00:39:13.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f610461a640 1 -- 172.21.15.29:0/2609580574 shutdown_connections 2024-04-25T00:39:13.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f610461a640 1 --2- 172.21.15.29:0/2609580574 >> v2:172.21.15.157:3300/0 conn(0x7f60fc0b4040 0x7f60fc0b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f610461a640 1 --2- 172.21.15.29:0/2609580574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60fc15e2a0 0x7f60fc0d1c00 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f610461a640 1 -- 172.21.15.29:0/2609580574 >> 172.21.15.29:0/2609580574 conn(0x7f60fc0d4030 msgr2=0x7f60fc0d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:13.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f610461a640 1 -- 172.21.15.29:0/2609580574 shutdown_connections 2024-04-25T00:39:13.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f610461a640 1 -- 172.21.15.29:0/2609580574 wait complete. 2024-04-25T00:39:13.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f610461a640 1 Processor -- start 2024-04-25T00:39:13.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:12.998+0000 7f610461a640 1 -- start start 2024-04-25T00:39:13.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f610461a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60fc0b4040 0x7f60fc0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f610461a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60fc0d4d10 0x7f60fc1535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f610461a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc15e2a0 0x7f60fc0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f610461a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc16b070 con 0x7f60fc0b4040 2024-04-25T00:39:13.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f610461a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc16aef0 con 0x7f60fc0d4d10 2024-04-25T00:39:13.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f610461a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc16b1f0 con 0x7f60fc15e2a0 2024-04-25T00:39:13.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f61023b6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60fc0b4040 0x7f60fc0c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:13.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f61023b6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60fc0b4040 0x7f60fc0c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56932/0 (socket says 172.21.15.29:56932) 2024-04-25T00:39:13.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f61023b6640 1 -- 172.21.15.29:0/2313032987 learned_addr learned my addr 172.21.15.29:0/2313032987 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:13.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f61023b6640 1 -- 172.21.15.29:0/2313032987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc15e2a0 msgr2=0x7f60fc0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f61023b6640 1 --2- 172.21.15.29:0/2313032987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc15e2a0 0x7f60fc0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f61023b6640 1 -- 172.21.15.29:0/2313032987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60fc0d4d10 msgr2=0x7f60fc1535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f61023b6640 1 --2- 172.21.15.29:0/2313032987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60fc0d4d10 0x7f60fc1535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f61023b6640 1 -- 172.21.15.29:0/2313032987 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc153af0 con 0x7f60fc0b4040 2024-04-25T00:39:13.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f61023b6640 1 --2- 172.21.15.29:0/2313032987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60fc0b4040 0x7f60fc0c76c0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f60f80ae040 tx=0x7f60f801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:13.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f60f2ffd640 1 -- 172.21.15.29:0/2313032987 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f80ba020 con 0x7f60fc0b4040 2024-04-25T00:39:13.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f60f2ffd640 1 -- 172.21.15.29:0/2313032987 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f80b6640 con 0x7f60fc0b4040 2024-04-25T00:39:13.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f610461a640 1 -- 172.21.15.29:0/2313032987 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc158b20 con 0x7f60fc0b4040 2024-04-25T00:39:13.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f60f2ffd640 1 -- 172.21.15.29:0/2313032987 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f80b0040 con 0x7f60fc0b4040 2024-04-25T00:39:13.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f610461a640 1 -- 172.21.15.29:0/2313032987 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc1590e0 con 0x7f60fc0b4040 2024-04-25T00:39:13.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f610461a640 1 -- 172.21.15.29:0/2313032987 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc069270 con 0x7f60fc0b4040 2024-04-25T00:39:13.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f60f2ffd640 1 -- 172.21.15.29:0/2313032987 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f60f80bd020 con 0x7f60fc0b4040 2024-04-25T00:39:13.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f60f2ffd640 1 --2- 172.21.15.29:0/2313032987 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f60e0067eb0 0x7f60e006a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f60f2ffd640 1 -- 172.21.15.29:0/2313032987 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f60f8120910 con 0x7f60fc0b4040 2024-04-25T00:39:13.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.002+0000 7f6101bb5640 1 --2- 172.21.15.29:0/2313032987 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f60e0067eb0 0x7f60e006a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:13.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.006+0000 7f6101bb5640 1 --2- 172.21.15.29:0/2313032987 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f60e0067eb0 0x7f60e006a370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0a1fb0 tx=0x7f60ec006bb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:13.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.010+0000 7f60f2ffd640 1 -- 172.21.15.29:0/2313032987 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f60f80c4030 con 0x7f60fc0b4040 2024-04-25T00:39:13.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.162+0000 7f610461a640 1 -- 172.21.15.29:0/2313032987 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60fc0d3e10 con 0x7f60e0067eb0 2024-04-25T00:39:13.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.182+0000 7f60f2ffd640 1 -- 172.21.15.29:0/2313032987 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252420 (secure 0 0 0) 0x7f60fc0d3e10 con 0x7f60e0067eb0 2024-04-25T00:39:13.187 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:13.189 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:39:13.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.186+0000 7f610461a640 1 -- 172.21.15.29:0/2313032987 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f60e0067eb0 msgr2=0x7f60e006a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.186+0000 7f610461a640 1 --2- 172.21.15.29:0/2313032987 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f60e0067eb0 0x7f60e006a370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0a1fb0 tx=0x7f60ec006bb0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.186+0000 7f610461a640 1 -- 172.21.15.29:0/2313032987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60fc0b4040 msgr2=0x7f60fc0c76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.186+0000 7f610461a640 1 --2- 172.21.15.29:0/2313032987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60fc0b4040 0x7f60fc0c76c0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f60f80ae040 tx=0x7f60f801ac90 comp rx=0 tx=0).stop 2024-04-25T00:39:13.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.186+0000 7f610461a640 1 -- 172.21.15.29:0/2313032987 shutdown_connections 2024-04-25T00:39:13.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.186+0000 7f610461a640 1 --2- 172.21.15.29:0/2313032987 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f60e0067eb0 0x7f60e006a370 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.186+0000 7f610461a640 1 --2- 172.21.15.29:0/2313032987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc15e2a0 0x7f60fc0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.186+0000 7f610461a640 1 --2- 172.21.15.29:0/2313032987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60fc0d4d10 0x7f60fc1535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.186+0000 7f610461a640 1 --2- 172.21.15.29:0/2313032987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60fc0b4040 0x7f60fc0c76c0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.186+0000 7f610461a640 1 -- 172.21.15.29:0/2313032987 >> 172.21.15.29:0/2313032987 conn(0x7f60fc0d4030 msgr2=0x7f60fc0b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:13.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.186+0000 7f610461a640 1 -- 172.21.15.29:0/2313032987 shutdown_connections 2024-04-25T00:39:13.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.186+0000 7f610461a640 1 -- 172.21.15.29:0/2313032987 wait complete. 2024-04-25T00:39:13.204 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-04-25T00:39:11.380691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":300792,"kb_used_data":13896,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754673928,"statfs":{"total":773094113280,"available":772786102272,"internally_reserved":0,"allocated":14229504,"data_stored":6923008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":293654064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.262797"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.192970+0000","last_change":"2024-04-25T00:39:09.191812+0000","last_active":"2024-04-25T00:39:09.192970+0000","last_peered":"2024-04-25T00:39:09.192970+0000","last_clean":"2024-04-25T00:39:09.192970+0000","last_became_active":"2024-04-25T00:39:09.191536+0000","last_became_peered":"2024-04-25T00:39:09.191536+0000","last_unstale":"2024-04-25T00:39:09.192970+0000","last_undegraded":"2024-04-25T00:39:09.192970+0000","last_fullsized":"2024-04-25T00:39:09.192970+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:48.896950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.157212+0000","last_change":"2024-04-25T00:39:09.152951+0000","last_active":"2024-04-25T00:39:09.157212+0000","last_peered":"2024-04-25T00:39:09.157212+0000","last_clean":"2024-04-25T00:39:09.157212+0000","last_became_active":"2024-04-25T00:39:09.152679+0000","last_became_peered":"2024-04-25T00:39:09.152679+0000","last_unstale":"2024-04-25T00:39:09.157212+0000","last_undegraded":"2024-04-25T00:39:09.157212+0000","last_fullsized":"2024-04-25T00:39:09.157212+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:22.295716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201167+0000","last_change":"2024-04-25T00:39:09.198811+0000","last_active":"2024-04-25T00:39:09.201167+0000","last_peered":"2024-04-25T00:39:09.201167+0000","last_clean":"2024-04-25T00:39:09.201167+0000","last_became_active":"2024-04-25T00:39:09.192819+0000","last_became_peered":"2024-04-25T00:39:09.192819+0000","last_unstale":"2024-04-25T00:39:09.201167+0000","last_undegraded":"2024-04-25T00:39:09.201167+0000","last_fullsized":"2024-04-25T00:39:09.201167+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:25.791648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.202845+0000","last_change":"2024-04-25T00:39:09.201419+0000","last_active":"2024-04-25T00:39:09.202845+0000","last_peered":"2024-04-25T00:39:09.202845+0000","last_clean":"2024-04-25T00:39:09.202845+0000","last_became_active":"2024-04-25T00:39:09.200615+0000","last_became_peered":"2024-04-25T00:39:09.200615+0000","last_unstale":"2024-04-25T00:39:09.202845+0000","last_undegraded":"2024-04-25T00:39:09.202845+0000","last_fullsized":"2024-04-25T00:39:09.202845+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:02.368131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.191115+0000","last_change":"2024-04-25T00:39:09.186592+0000","last_active":"2024-04-25T00:39:09.191115+0000","last_peered":"2024-04-25T00:39:09.191115+0000","last_clean":"2024-04-25T00:39:09.191115+0000","last_became_active":"2024-04-25T00:39:09.185849+0000","last_became_peered":"2024-04-25T00:39:09.185849+0000","last_unstale":"2024-04-25T00:39:09.191115+0000","last_undegraded":"2024-04-25T00:39:09.191115+0000","last_fullsized":"2024-04-25T00:39:09.191115+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:41.680770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.199733+0000","last_change":"2024-04-25T00:39:09.199104+0000","last_active":"2024-04-25T00:39:09.199733+0000","last_peered":"2024-04-25T00:39:09.199733+0000","last_clean":"2024-04-25T00:39:09.199733+0000","last_became_active":"2024-04-25T00:39:09.198781+0000","last_became_peered":"2024-04-25T00:39:09.198781+0000","last_unstale":"2024-04-25T00:39:09.199733+0000","last_undegraded":"2024-04-25T00:39:09.199733+0000","last_fullsized":"2024-04-25T00:39:09.199733+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:25.542008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.168164+0000","last_change":"2024-04-25T00:39:09.167430+0000","last_active":"2024-04-25T00:39:09.168164+0000","last_peered":"2024-04-25T00:39:09.168164+0000","last_clean":"2024-04-25T00:39:09.168164+0000","last_became_active":"2024-04-25T00:39:09.167140+0000","last_became_peered":"2024-04-25T00:39:09.167140+0000","last_unstale":"2024-04-25T00:39:09.168164+0000","last_undegraded":"2024-04-25T00:39:09.168164+0000","last_fullsized":"2024-04-25T00:39:09.168164+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:07.042416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.166683+0000","last_change":"2024-04-25T00:39:09.165838+0000","last_active":"2024-04-25T00:39:09.166683+0000","last_peered":"2024-04-25T00:39:09.166683+0000","last_clean":"2024-04-25T00:39:09.166683+0000","last_became_active":"2024-04-25T00:39:09.165529+0000","last_became_peered":"2024-04-25T00:39:09.165529+0000","last_unstale":"2024-04-25T00:39:09.166683+0000","last_undegraded":"2024-04-25T00:39:09.166683+0000","last_fullsized":"2024-04-25T00:39:09.166683+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:16:34.583678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155629+0000","last_change":"2024-04-25T00:39:09.152024+0000","last_active":"2024-04-25T00:39:09.155629+0000","last_peered":"2024-04-25T00:39:09.155629+0000","last_clean":"2024-04-25T00:39:09.155629+0000","last_became_active":"2024-04-25T00:39:09.151669+0000","last_became_peered":"2024-04-25T00:39:09.151669+0000","last_unstale":"2024-04-25T00:39:09.155629+0000","last_undegraded":"2024-04-25T00:39:09.155629+0000","last_fullsized":"2024-04-25T00:39:09.155629+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:38.040554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201843+0000","last_change":"2024-04-25T00:39:09.190350+0000","last_active":"2024-04-25T00:39:09.201843+0000","last_peered":"2024-04-25T00:39:09.201843+0000","last_clean":"2024-04-25T00:39:09.201843+0000","last_became_active":"2024-04-25T00:39:09.189633+0000","last_became_peered":"2024-04-25T00:39:09.189633+0000","last_unstale":"2024-04-25T00:39:09.201843+0000","last_undegraded":"2024-04-25T00:39:09.201843+0000","last_fullsized":"2024-04-25T00:39:09.201843+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:07.042416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.166690+0000","last_change":"2024-04-25T00:39:09.165896+0000","last_active":"2024-04-25T00:39:09.166690+0000","last_peered":"2024-04-25T00:39:09.166690+0000","last_clean":"2024-04-25T00:39:09.166690+0000","last_became_active":"2024-04-25T00:39:09.165552+0000","last_became_peered":"2024-04-25T00:39:09.165552+0000","last_unstale":"2024-04-25T00:39:09.166690+0000","last_undegraded":"2024-04-25T00:39:09.166690+0000","last_fullsized":"2024-04-25T00:39:09.166690+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:22.049618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201271+0000","last_change":"2024-04-25T00:39:09.186618+0000","last_active":"2024-04-25T00:39:09.201271+0000","last_peered":"2024-04-25T00:39:09.201271+0000","last_clean":"2024-04-25T00:39:09.201271+0000","last_became_active":"2024-04-25T00:39:09.185892+0000","last_became_peered":"2024-04-25T00:39:09.185892+0000","last_unstale":"2024-04-25T00:39:09.201271+0000","last_undegraded":"2024-04-25T00:39:09.201271+0000","last_fullsized":"2024-04-25T00:39:09.201271+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:44.551679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.186940+0000","last_change":"2024-04-25T00:39:09.184541+0000","last_active":"2024-04-25T00:39:09.186940+0000","last_peered":"2024-04-25T00:39:09.186940+0000","last_clean":"2024-04-25T00:39:09.186940+0000","last_became_active":"2024-04-25T00:39:09.183778+0000","last_became_peered":"2024-04-25T00:39:09.183778+0000","last_unstale":"2024-04-25T00:39:09.186940+0000","last_undegraded":"2024-04-25T00:39:09.186940+0000","last_fullsized":"2024-04-25T00:39:09.186940+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:41.934765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.188231+0000","last_change":"2024-04-25T00:39:09.183928+0000","last_active":"2024-04-25T00:39:09.188231+0000","last_peered":"2024-04-25T00:39:09.188231+0000","last_clean":"2024-04-25T00:39:09.188231+0000","last_became_active":"2024-04-25T00:39:09.183347+0000","last_became_peered":"2024-04-25T00:39:09.183347+0000","last_unstale":"2024-04-25T00:39:09.188231+0000","last_undegraded":"2024-04-25T00:39:09.188231+0000","last_fullsized":"2024-04-25T00:39:09.188231+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:52.463025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201861+0000","last_change":"2024-04-25T00:39:09.190430+0000","last_active":"2024-04-25T00:39:09.201861+0000","last_peered":"2024-04-25T00:39:09.201861+0000","last_clean":"2024-04-25T00:39:09.201861+0000","last_became_active":"2024-04-25T00:39:09.189697+0000","last_became_peered":"2024-04-25T00:39:09.189697+0000","last_unstale":"2024-04-25T00:39:09.201861+0000","last_undegraded":"2024-04-25T00:39:09.201861+0000","last_fullsized":"2024-04-25T00:39:09.201861+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:10.600534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.158330+0000","last_change":"2024-04-25T00:39:09.157015+0000","last_active":"2024-04-25T00:39:09.158330+0000","last_peered":"2024-04-25T00:39:09.158330+0000","last_clean":"2024-04-25T00:39:09.158330+0000","last_became_active":"2024-04-25T00:39:09.156152+0000","last_became_peered":"2024-04-25T00:39:09.156152+0000","last_unstale":"2024-04-25T00:39:09.158330+0000","last_undegraded":"2024-04-25T00:39:09.158330+0000","last_fullsized":"2024-04-25T00:39:09.158330+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:41.536446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.202180+0000","last_change":"2024-04-25T00:39:09.194551+0000","last_active":"2024-04-25T00:39:09.202180+0000","last_peered":"2024-04-25T00:39:09.202180+0000","last_clean":"2024-04-25T00:39:09.202180+0000","last_became_active":"2024-04-25T00:39:09.193023+0000","last_became_peered":"2024-04-25T00:39:09.193023+0000","last_unstale":"2024-04-25T00:39:09.202180+0000","last_undegraded":"2024-04-25T00:39:09.202180+0000","last_fullsized":"2024-04-25T00:39:09.202180+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:46.496845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.199941+0000","last_change":"2024-04-25T00:39:09.195131+0000","last_active":"2024-04-25T00:39:09.199941+0000","last_peered":"2024-04-25T00:39:09.199941+0000","last_clean":"2024-04-25T00:39:09.199941+0000","last_became_active":"2024-04-25T00:39:09.192410+0000","last_became_peered":"2024-04-25T00:39:09.192410+0000","last_unstale":"2024-04-25T00:39:09.199941+0000","last_undegraded":"2024-04-25T00:39:09.199941+0000","last_fullsized":"2024-04-25T00:39:09.199941+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:52.372514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.160270+0000","last_change":"2024-04-25T00:39:09.157066+0000","last_active":"2024-04-25T00:39:09.160270+0000","last_peered":"2024-04-25T00:39:09.160270+0000","last_clean":"2024-04-25T00:39:09.160270+0000","last_became_active":"2024-04-25T00:39:09.156664+0000","last_became_peered":"2024-04-25T00:39:09.156664+0000","last_unstale":"2024-04-25T00:39:09.160270+0000","last_undegraded":"2024-04-25T00:39:09.160270+0000","last_fullsized":"2024-04-25T00:39:09.160270+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:13.920122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.158296+0000","last_change":"2024-04-25T00:39:09.156657+0000","last_active":"2024-04-25T00:39:09.158296+0000","last_peered":"2024-04-25T00:39:09.158296+0000","last_clean":"2024-04-25T00:39:09.158296+0000","last_became_active":"2024-04-25T00:39:09.156354+0000","last_became_peered":"2024-04-25T00:39:09.156354+0000","last_unstale":"2024-04-25T00:39:09.158296+0000","last_undegraded":"2024-04-25T00:39:09.158296+0000","last_fullsized":"2024-04-25T00:39:09.158296+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:52.158456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201826+0000","last_change":"2024-04-25T00:39:09.188657+0000","last_active":"2024-04-25T00:39:09.201826+0000","last_peered":"2024-04-25T00:39:09.201826+0000","last_clean":"2024-04-25T00:39:09.201826+0000","last_became_active":"2024-04-25T00:39:09.187471+0000","last_became_peered":"2024-04-25T00:39:09.187471+0000","last_unstale":"2024-04-25T00:39:09.201826+0000","last_undegraded":"2024-04-25T00:39:09.201826+0000","last_fullsized":"2024-04-25T00:39:09.201826+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:22.696723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.179340+0000","last_change":"2024-04-25T00:39:09.163036+0000","last_active":"2024-04-25T00:39:09.179340+0000","last_peered":"2024-04-25T00:39:09.179340+0000","last_clean":"2024-04-25T00:39:09.179340+0000","last_became_active":"2024-04-25T00:39:09.160768+0000","last_became_peered":"2024-04-25T00:39:09.160768+0000","last_unstale":"2024-04-25T00:39:09.179340+0000","last_undegraded":"2024-04-25T00:39:09.179340+0000","last_fullsized":"2024-04-25T00:39:09.179340+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:42.801159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.171121+0000","last_change":"2024-04-25T00:39:09.156106+0000","last_active":"2024-04-25T00:39:09.171121+0000","last_peered":"2024-04-25T00:39:09.171121+0000","last_clean":"2024-04-25T00:39:09.171121+0000","last_became_active":"2024-04-25T00:39:09.155237+0000","last_became_peered":"2024-04-25T00:39:09.155237+0000","last_unstale":"2024-04-25T00:39:09.171121+0000","last_undegraded":"2024-04-25T00:39:09.171121+0000","last_fullsized":"2024-04-25T00:39:09.171121+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:50.835641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.197032+0000","last_change":"2024-04-25T00:39:09.184470+0000","last_active":"2024-04-25T00:39:09.197032+0000","last_peered":"2024-04-25T00:39:09.197032+0000","last_clean":"2024-04-25T00:39:09.197032+0000","last_became_active":"2024-04-25T00:39:09.183743+0000","last_became_peered":"2024-04-25T00:39:09.183743+0000","last_unstale":"2024-04-25T00:39:09.197032+0000","last_undegraded":"2024-04-25T00:39:09.197032+0000","last_fullsized":"2024-04-25T00:39:09.197032+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:08.495108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155772+0000","last_change":"2024-04-25T00:39:09.153476+0000","last_active":"2024-04-25T00:39:09.155772+0000","last_peered":"2024-04-25T00:39:09.155772+0000","last_clean":"2024-04-25T00:39:09.155772+0000","last_became_active":"2024-04-25T00:39:09.153100+0000","last_became_peered":"2024-04-25T00:39:09.153100+0000","last_unstale":"2024-04-25T00:39:09.155772+0000","last_undegraded":"2024-04-25T00:39:09.155772+0000","last_fullsized":"2024-04-25T00:39:09.155772+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:49.211440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.160223+0000","last_change":"2024-04-25T00:39:09.156996+0000","last_active":"2024-04-25T00:39:09.160223+0000","last_peered":"2024-04-25T00:39:09.160223+0000","last_clean":"2024-04-25T00:39:09.160223+0000","last_became_active":"2024-04-25T00:39:09.156670+0000","last_became_peered":"2024-04-25T00:39:09.156670+0000","last_unstale":"2024-04-25T00:39:09.160223+0000","last_undegraded":"2024-04-25T00:39:09.160223+0000","last_fullsized":"2024-04-25T00:39:09.160223+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:55.380913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.193250+0000","last_change":"2024-04-25T00:39:09.192729+0000","last_active":"2024-04-25T00:39:09.193250+0000","last_peered":"2024-04-25T00:39:09.193250+0000","last_clean":"2024-04-25T00:39:09.193250+0000","last_became_active":"2024-04-25T00:39:09.192303+0000","last_became_peered":"2024-04-25T00:39:09.192303+0000","last_unstale":"2024-04-25T00:39:09.193250+0000","last_undegraded":"2024-04-25T00:39:09.193250+0000","last_fullsized":"2024-04-25T00:39:09.193250+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:55.024666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.196927+0000","last_change":"2024-04-25T00:39:09.196470+0000","last_active":"2024-04-25T00:39:09.196927+0000","last_peered":"2024-04-25T00:39:09.196927+0000","last_clean":"2024-04-25T00:39:09.196927+0000","last_became_active":"2024-04-25T00:39:09.196145+0000","last_became_peered":"2024-04-25T00:39:09.196145+0000","last_unstale":"2024-04-25T00:39:09.196927+0000","last_undegraded":"2024-04-25T00:39:09.196927+0000","last_fullsized":"2024-04-25T00:39:09.196927+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:04.851312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.192405+0000","last_change":"2024-04-25T00:39:09.191662+0000","last_active":"2024-04-25T00:39:09.192405+0000","last_peered":"2024-04-25T00:39:09.192405+0000","last_clean":"2024-04-25T00:39:09.192405+0000","last_became_active":"2024-04-25T00:39:09.191352+0000","last_became_peered":"2024-04-25T00:39:09.191352+0000","last_unstale":"2024-04-25T00:39:09.192405+0000","last_undegraded":"2024-04-25T00:39:09.192405+0000","last_fullsized":"2024-04-25T00:39:09.192405+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:43.815701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.165743+0000","last_change":"2024-04-25T00:39:09.153864+0000","last_active":"2024-04-25T00:39:09.165743+0000","last_peered":"2024-04-25T00:39:09.165743+0000","last_clean":"2024-04-25T00:39:09.165743+0000","last_became_active":"2024-04-25T00:39:09.151787+0000","last_became_peered":"2024-04-25T00:39:09.151787+0000","last_unstale":"2024-04-25T00:39:09.165743+0000","last_undegraded":"2024-04-25T00:39:09.165743+0000","last_fullsized":"2024-04-25T00:39:09.165743+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:38.827706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.182726+0000","last_change":"2024-04-25T00:39:09.182134+0000","last_active":"2024-04-25T00:39:09.182726+0000","last_peered":"2024-04-25T00:39:09.182726+0000","last_clean":"2024-04-25T00:39:09.182726+0000","last_became_active":"2024-04-25T00:39:09.181775+0000","last_became_peered":"2024-04-25T00:39:09.181775+0000","last_unstale":"2024-04-25T00:39:09.182726+0000","last_undegraded":"2024-04-25T00:39:09.182726+0000","last_fullsized":"2024-04-25T00:39:09.182726+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:36.175223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.159540+0000","last_change":"2024-04-25T00:39:09.158234+0000","last_active":"2024-04-25T00:39:09.159540+0000","last_peered":"2024-04-25T00:39:09.159540+0000","last_clean":"2024-04-25T00:39:09.159540+0000","last_became_active":"2024-04-25T00:39:09.157929+0000","last_became_peered":"2024-04-25T00:39:09.157929+0000","last_unstale":"2024-04-25T00:39:09.159540+0000","last_undegraded":"2024-04-25T00:39:09.159540+0000","last_fullsized":"2024-04-25T00:39:09.159540+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:37.038466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.202688+0000","last_change":"2024-04-25T00:39:09.197970+0000","last_active":"2024-04-25T00:39:09.202688+0000","last_peered":"2024-04-25T00:39:09.202688+0000","last_clean":"2024-04-25T00:39:09.202688+0000","last_became_active":"2024-04-25T00:39:09.197332+0000","last_became_peered":"2024-04-25T00:39:09.197332+0000","last_unstale":"2024-04-25T00:39:09.202688+0000","last_undegraded":"2024-04-25T00:39:09.202688+0000","last_fullsized":"2024-04-25T00:39:09.202688+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:04.851312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"131'4","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201896+0000","last_change":"2024-04-25T00:39:08.156446+0000","last_active":"2024-04-25T00:39:09.201896+0000","last_peered":"2024-04-25T00:39:09.201896+0000","last_clean":"2024-04-25T00:39:09.201896+0000","last_became_active":"2024-04-25T00:39:08.155076+0000","last_became_peered":"2024-04-25T00:39:08.155076+0000","last_unstale":"2024-04-25T00:39:09.201896+0000","last_undegraded":"2024-04-25T00:39:09.201896+0000","last_fullsized":"2024-04-25T00:39:09.201896+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:32.569417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155660+0000","last_change":"2024-04-25T00:39:09.152136+0000","last_active":"2024-04-25T00:39:09.155660+0000","last_peered":"2024-04-25T00:39:09.155660+0000","last_clean":"2024-04-25T00:39:09.155660+0000","last_became_active":"2024-04-25T00:39:09.151826+0000","last_became_peered":"2024-04-25T00:39:09.151826+0000","last_unstale":"2024-04-25T00:39:09.155660+0000","last_undegraded":"2024-04-25T00:39:09.155660+0000","last_fullsized":"2024-04-25T00:39:09.155660+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:08.203435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.175042+0000","last_change":"2024-04-25T00:39:08.154558+0000","last_active":"2024-04-25T00:39:09.175042+0000","last_peered":"2024-04-25T00:39:09.175042+0000","last_clean":"2024-04-25T00:39:09.175042+0000","last_became_active":"2024-04-25T00:39:08.154046+0000","last_became_peered":"2024-04-25T00:39:08.154046+0000","last_unstale":"2024-04-25T00:39:09.175042+0000","last_undegraded":"2024-04-25T00:39:09.175042+0000","last_fullsized":"2024-04-25T00:39:09.175042+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:50.223659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.160781+0000","last_change":"2024-04-25T00:39:09.159968+0000","last_active":"2024-04-25T00:39:09.160781+0000","last_peered":"2024-04-25T00:39:09.160781+0000","last_clean":"2024-04-25T00:39:09.160781+0000","last_became_active":"2024-04-25T00:39:09.159484+0000","last_became_peered":"2024-04-25T00:39:09.159484+0000","last_unstale":"2024-04-25T00:39:09.160781+0000","last_undegraded":"2024-04-25T00:39:09.160781+0000","last_fullsized":"2024-04-25T00:39:09.160781+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:34.014780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.171403+0000","last_change":"2024-04-25T00:39:08.168582+0000","last_active":"2024-04-25T00:39:09.171403+0000","last_peered":"2024-04-25T00:39:09.171403+0000","last_clean":"2024-04-25T00:39:09.171403+0000","last_became_active":"2024-04-25T00:39:08.166773+0000","last_became_peered":"2024-04-25T00:39:08.166773+0000","last_unstale":"2024-04-25T00:39:09.171403+0000","last_undegraded":"2024-04-25T00:39:09.171403+0000","last_fullsized":"2024-04-25T00:39:09.171403+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:04.006426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201653+0000","last_change":"2024-04-25T00:39:09.188040+0000","last_active":"2024-04-25T00:39:09.201653+0000","last_peered":"2024-04-25T00:39:09.201653+0000","last_clean":"2024-04-25T00:39:09.201653+0000","last_became_active":"2024-04-25T00:39:09.187252+0000","last_became_peered":"2024-04-25T00:39:09.187252+0000","last_unstale":"2024-04-25T00:39:09.201653+0000","last_undegraded":"2024-04-25T00:39:09.201653+0000","last_fullsized":"2024-04-25T00:39:09.201653+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:35.916708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.177403+0000","last_change":"2024-04-25T00:39:08.162907+0000","last_active":"2024-04-25T00:39:09.177403+0000","last_peered":"2024-04-25T00:39:09.177403+0000","last_clean":"2024-04-25T00:39:09.177403+0000","last_became_active":"2024-04-25T00:39:08.156446+0000","last_became_peered":"2024-04-25T00:39:08.156446+0000","last_unstale":"2024-04-25T00:39:09.177403+0000","last_undegraded":"2024-04-25T00:39:09.177403+0000","last_fullsized":"2024-04-25T00:39:09.177403+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:42.035329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.200405+0000","last_change":"2024-04-25T00:39:09.199971+0000","last_active":"2024-04-25T00:39:09.200405+0000","last_peered":"2024-04-25T00:39:09.200405+0000","last_clean":"2024-04-25T00:39:09.200405+0000","last_became_active":"2024-04-25T00:39:09.199627+0000","last_became_peered":"2024-04-25T00:39:09.199627+0000","last_unstale":"2024-04-25T00:39:09.200405+0000","last_undegraded":"2024-04-25T00:39:09.200405+0000","last_fullsized":"2024-04-25T00:39:09.200405+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:33.573264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.164780+0000","last_change":"2024-04-25T00:39:08.151302+0000","last_active":"2024-04-25T00:39:09.164780+0000","last_peered":"2024-04-25T00:39:09.164780+0000","last_clean":"2024-04-25T00:39:09.164780+0000","last_became_active":"2024-04-25T00:39:08.148783+0000","last_became_peered":"2024-04-25T00:39:08.148783+0000","last_unstale":"2024-04-25T00:39:09.164780+0000","last_undegraded":"2024-04-25T00:39:09.164780+0000","last_fullsized":"2024-04-25T00:39:09.164780+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:42.967705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.197570+0000","last_change":"2024-04-25T00:39:09.194869+0000","last_active":"2024-04-25T00:39:09.197570+0000","last_peered":"2024-04-25T00:39:09.197570+0000","last_clean":"2024-04-25T00:39:09.197570+0000","last_became_active":"2024-04-25T00:39:09.194200+0000","last_became_peered":"2024-04-25T00:39:09.194200+0000","last_unstale":"2024-04-25T00:39:09.197570+0000","last_undegraded":"2024-04-25T00:39:09.197570+0000","last_fullsized":"2024-04-25T00:39:09.197570+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:38.623765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.173733+0000","last_change":"2024-04-25T00:39:08.171567+0000","last_active":"2024-04-25T00:39:09.173733+0000","last_peered":"2024-04-25T00:39:09.173733+0000","last_clean":"2024-04-25T00:39:09.173733+0000","last_became_active":"2024-04-25T00:39:08.170833+0000","last_became_peered":"2024-04-25T00:39:08.170833+0000","last_unstale":"2024-04-25T00:39:09.173733+0000","last_undegraded":"2024-04-25T00:39:09.173733+0000","last_fullsized":"2024-04-25T00:39:09.173733+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:51.154609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155632+0000","last_change":"2024-04-25T00:39:09.148908+0000","last_active":"2024-04-25T00:39:09.155632+0000","last_peered":"2024-04-25T00:39:09.155632+0000","last_clean":"2024-04-25T00:39:09.155632+0000","last_became_active":"2024-04-25T00:39:09.148603+0000","last_became_peered":"2024-04-25T00:39:09.148603+0000","last_unstale":"2024-04-25T00:39:09.155632+0000","last_undegraded":"2024-04-25T00:39:09.155632+0000","last_fullsized":"2024-04-25T00:39:09.155632+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:15.292122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.140734+0000","last_change":"2024-04-25T00:39:08.171081+0000","last_active":"2024-04-25T00:39:09.140734+0000","last_peered":"2024-04-25T00:39:09.140734+0000","last_clean":"2024-04-25T00:39:09.140734+0000","last_became_active":"2024-04-25T00:39:08.170396+0000","last_became_peered":"2024-04-25T00:39:08.170396+0000","last_unstale":"2024-04-25T00:39:09.140734+0000","last_undegraded":"2024-04-25T00:39:09.140734+0000","last_fullsized":"2024-04-25T00:39:09.140734+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:48.839793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.168643+0000","last_change":"2024-04-25T00:39:09.159430+0000","last_active":"2024-04-25T00:39:09.168643+0000","last_peered":"2024-04-25T00:39:09.168643+0000","last_clean":"2024-04-25T00:39:09.168643+0000","last_became_active":"2024-04-25T00:39:09.158261+0000","last_became_peered":"2024-04-25T00:39:09.158261+0000","last_unstale":"2024-04-25T00:39:09.168643+0000","last_undegraded":"2024-04-25T00:39:09.168643+0000","last_fullsized":"2024-04-25T00:39:09.168643+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:11.878064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.139432+0000","last_change":"2024-04-25T00:39:08.172987+0000","last_active":"2024-04-25T00:39:09.139432+0000","last_peered":"2024-04-25T00:39:09.139432+0000","last_clean":"2024-04-25T00:39:09.139432+0000","last_became_active":"2024-04-25T00:39:08.171954+0000","last_became_peered":"2024-04-25T00:39:08.171954+0000","last_unstale":"2024-04-25T00:39:09.139432+0000","last_undegraded":"2024-04-25T00:39:09.139432+0000","last_fullsized":"2024-04-25T00:39:09.139432+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:11.695113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.168605+0000","last_change":"2024-04-25T00:39:09.159261+0000","last_active":"2024-04-25T00:39:09.168605+0000","last_peered":"2024-04-25T00:39:09.168605+0000","last_clean":"2024-04-25T00:39:09.168605+0000","last_became_active":"2024-04-25T00:39:09.158572+0000","last_became_peered":"2024-04-25T00:39:09.158572+0000","last_unstale":"2024-04-25T00:39:09.168605+0000","last_undegraded":"2024-04-25T00:39:09.168605+0000","last_fullsized":"2024-04-25T00:39:09.168605+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:37.038466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.136604+0000","last_change":"2024-04-25T00:39:08.154902+0000","last_active":"2024-04-25T00:39:09.136604+0000","last_peered":"2024-04-25T00:39:09.136604+0000","last_clean":"2024-04-25T00:39:09.136604+0000","last_became_active":"2024-04-25T00:39:08.154483+0000","last_became_peered":"2024-04-25T00:39:08.154483+0000","last_unstale":"2024-04-25T00:39:09.136604+0000","last_undegraded":"2024-04-25T00:39:09.136604+0000","last_fullsized":"2024-04-25T00:39:09.136604+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:33.875632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.202715+0000","last_change":"2024-04-25T00:39:09.200113+0000","last_active":"2024-04-25T00:39:09.202715+0000","last_peered":"2024-04-25T00:39:09.202715+0000","last_clean":"2024-04-25T00:39:09.202715+0000","last_became_active":"2024-04-25T00:39:09.195768+0000","last_became_peered":"2024-04-25T00:39:09.195768+0000","last_unstale":"2024-04-25T00:39:09.202715+0000","last_undegraded":"2024-04-25T00:39:09.202715+0000","last_fullsized":"2024-04-25T00:39:09.202715+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:44.534637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.175099+0000","last_change":"2024-04-25T00:39:08.168721+0000","last_active":"2024-04-25T00:39:09.175099+0000","last_peered":"2024-04-25T00:39:09.175099+0000","last_clean":"2024-04-25T00:39:09.175099+0000","last_became_active":"2024-04-25T00:39:08.167961+0000","last_became_peered":"2024-04-25T00:39:08.167961+0000","last_unstale":"2024-04-25T00:39:09.175099+0000","last_undegraded":"2024-04-25T00:39:09.175099+0000","last_fullsized":"2024-04-25T00:39:09.175099+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:32.064317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155703+0000","last_change":"2024-04-25T00:39:09.150902+0000","last_active":"2024-04-25T00:39:09.155703+0000","last_peered":"2024-04-25T00:39:09.155703+0000","last_clean":"2024-04-25T00:39:09.155703+0000","last_became_active":"2024-04-25T00:39:09.150527+0000","last_became_peered":"2024-04-25T00:39:09.150527+0000","last_unstale":"2024-04-25T00:39:09.155703+0000","last_undegraded":"2024-04-25T00:39:09.155703+0000","last_fullsized":"2024-04-25T00:39:09.155703+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:38.603505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"131'6","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.166497+0000","last_change":"2024-04-25T00:39:08.156557+0000","last_active":"2024-04-25T00:39:09.166497+0000","last_peered":"2024-04-25T00:39:09.166497+0000","last_clean":"2024-04-25T00:39:09.166497+0000","last_became_active":"2024-04-25T00:39:08.156134+0000","last_became_peered":"2024-04-25T00:39:08.156134+0000","last_unstale":"2024-04-25T00:39:09.166497+0000","last_undegraded":"2024-04-25T00:39:09.166497+0000","last_fullsized":"2024-04-25T00:39:09.166497+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:34.912860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201077+0000","last_change":"2024-04-25T00:39:09.195925+0000","last_active":"2024-04-25T00:39:09.201077+0000","last_peered":"2024-04-25T00:39:09.201077+0000","last_clean":"2024-04-25T00:39:09.201077+0000","last_became_active":"2024-04-25T00:39:09.191440+0000","last_became_peered":"2024-04-25T00:39:09.191440+0000","last_unstale":"2024-04-25T00:39:09.201077+0000","last_undegraded":"2024-04-25T00:39:09.201077+0000","last_fullsized":"2024-04-25T00:39:09.201077+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:39.278445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.190877+0000","last_change":"2024-04-25T00:39:08.158561+0000","last_active":"2024-04-25T00:39:09.190877+0000","last_peered":"2024-04-25T00:39:09.190877+0000","last_clean":"2024-04-25T00:39:09.190877+0000","last_became_active":"2024-04-25T00:39:08.157751+0000","last_became_peered":"2024-04-25T00:39:08.157751+0000","last_unstale":"2024-04-25T00:39:09.190877+0000","last_undegraded":"2024-04-25T00:39:09.190877+0000","last_fullsized":"2024-04-25T00:39:09.190877+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:10.274560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155711+0000","last_change":"2024-04-25T00:39:09.152699+0000","last_active":"2024-04-25T00:39:09.155711+0000","last_peered":"2024-04-25T00:39:09.155711+0000","last_clean":"2024-04-25T00:39:09.155711+0000","last_became_active":"2024-04-25T00:39:09.151918+0000","last_became_peered":"2024-04-25T00:39:09.151918+0000","last_unstale":"2024-04-25T00:39:09.155711+0000","last_undegraded":"2024-04-25T00:39:09.155711+0000","last_fullsized":"2024-04-25T00:39:09.155711+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:02.412132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.167024+0000","last_change":"2024-04-25T00:39:08.163557+0000","last_active":"2024-04-25T00:39:09.167024+0000","last_peered":"2024-04-25T00:39:09.167024+0000","last_clean":"2024-04-25T00:39:09.167024+0000","last_became_active":"2024-04-25T00:39:08.161998+0000","last_became_peered":"2024-04-25T00:39:08.161998+0000","last_unstale":"2024-04-25T00:39:09.167024+0000","last_undegraded":"2024-04-25T00:39:09.167024+0000","last_fullsized":"2024-04-25T00:39:09.167024+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:24.538161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.191036+0000","last_change":"2024-04-25T00:39:09.186488+0000","last_active":"2024-04-25T00:39:09.191036+0000","last_peered":"2024-04-25T00:39:09.191036+0000","last_clean":"2024-04-25T00:39:09.191036+0000","last_became_active":"2024-04-25T00:39:09.185744+0000","last_became_peered":"2024-04-25T00:39:09.185744+0000","last_unstale":"2024-04-25T00:39:09.191036+0000","last_undegraded":"2024-04-25T00:39:09.191036+0000","last_fullsized":"2024-04-25T00:39:09.191036+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:00.617110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.147940+0000","last_change":"2024-04-25T00:39:08.156570+0000","last_active":"2024-04-25T00:39:09.147940+0000","last_peered":"2024-04-25T00:39:09.147940+0000","last_clean":"2024-04-25T00:39:09.147940+0000","last_became_active":"2024-04-25T00:39:08.156089+0000","last_became_peered":"2024-04-25T00:39:08.156089+0000","last_unstale":"2024-04-25T00:39:09.147940+0000","last_undegraded":"2024-04-25T00:39:09.147940+0000","last_fullsized":"2024-04-25T00:39:09.147940+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:54.901610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.199809+0000","last_change":"2024-04-25T00:39:09.191516+0000","last_active":"2024-04-25T00:39:09.199809+0000","last_peered":"2024-04-25T00:39:09.199809+0000","last_clean":"2024-04-25T00:39:09.199809+0000","last_became_active":"2024-04-25T00:39:09.182230+0000","last_became_peered":"2024-04-25T00:39:09.182230+0000","last_unstale":"2024-04-25T00:39:09.199809+0000","last_undegraded":"2024-04-25T00:39:09.199809+0000","last_fullsized":"2024-04-25T00:39:09.199809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:07.668586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"131'8","reported_seq":20,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.200571+0000","last_change":"2024-04-25T00:39:08.161970+0000","last_active":"2024-04-25T00:39:09.200571+0000","last_peered":"2024-04-25T00:39:09.200571+0000","last_clean":"2024-04-25T00:39:09.200571+0000","last_became_active":"2024-04-25T00:39:08.161645+0000","last_became_peered":"2024-04-25T00:39:08.161645+0000","last_unstale":"2024-04-25T00:39:09.200571+0000","last_undegraded":"2024-04-25T00:39:09.200571+0000","last_fullsized":"2024-04-25T00:39:09.200571+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:49.240064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.202648+0000","last_change":"2024-04-25T00:39:09.195707+0000","last_active":"2024-04-25T00:39:09.202648+0000","last_peered":"2024-04-25T00:39:09.202648+0000","last_clean":"2024-04-25T00:39:09.202648+0000","last_became_active":"2024-04-25T00:39:09.190824+0000","last_became_peered":"2024-04-25T00:39:09.190824+0000","last_unstale":"2024-04-25T00:39:09.202648+0000","last_undegraded":"2024-04-25T00:39:09.202648+0000","last_fullsized":"2024-04-25T00:39:09.202648+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:17.003258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.183510+0000","last_change":"2024-04-25T00:39:08.163083+0000","last_active":"2024-04-25T00:39:09.183510+0000","last_peered":"2024-04-25T00:39:09.183510+0000","last_clean":"2024-04-25T00:39:09.183510+0000","last_became_active":"2024-04-25T00:39:08.162358+0000","last_became_peered":"2024-04-25T00:39:08.162358+0000","last_unstale":"2024-04-25T00:39:09.183510+0000","last_undegraded":"2024-04-25T00:39:09.183510+0000","last_fullsized":"2024-04-25T00:39:09.183510+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:40.532599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.192506+0000","last_change":"2024-04-25T00:39:09.191692+0000","last_active":"2024-04-25T00:39:09.192506+0000","last_peered":"2024-04-25T00:39:09.192506+0000","last_clean":"2024-04-25T00:39:09.192506+0000","last_became_active":"2024-04-25T00:39:09.191460+0000","last_became_peered":"2024-04-25T00:39:09.191460+0000","last_unstale":"2024-04-25T00:39:09.192506+0000","last_undegraded":"2024-04-25T00:39:09.192506+0000","last_fullsized":"2024-04-25T00:39:09.192506+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:11.878064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.192482+0000","last_change":"2024-04-25T00:39:08.159499+0000","last_active":"2024-04-25T00:39:09.192482+0000","last_peered":"2024-04-25T00:39:09.192482+0000","last_clean":"2024-04-25T00:39:09.192482+0000","last_became_active":"2024-04-25T00:39:08.159096+0000","last_became_peered":"2024-04-25T00:39:08.159096+0000","last_unstale":"2024-04-25T00:39:09.192482+0000","last_undegraded":"2024-04-25T00:39:09.192482+0000","last_fullsized":"2024-04-25T00:39:09.192482+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:57.654458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.200517+0000","last_change":"2024-04-25T00:39:09.195491+0000","last_active":"2024-04-25T00:39:09.200517+0000","last_peered":"2024-04-25T00:39:09.200517+0000","last_clean":"2024-04-25T00:39:09.200517+0000","last_became_active":"2024-04-25T00:39:09.182140+0000","last_became_peered":"2024-04-25T00:39:09.182140+0000","last_unstale":"2024-04-25T00:39:09.200517+0000","last_undegraded":"2024-04-25T00:39:09.200517+0000","last_fullsized":"2024-04-25T00:39:09.200517+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:36.089141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.176433+0000","last_change":"2024-04-25T00:39:08.150466+0000","last_active":"2024-04-25T00:39:09.176433+0000","last_peered":"2024-04-25T00:39:09.176433+0000","last_clean":"2024-04-25T00:39:09.176433+0000","last_became_active":"2024-04-25T00:39:08.149773+0000","last_became_peered":"2024-04-25T00:39:08.149773+0000","last_unstale":"2024-04-25T00:39:09.176433+0000","last_undegraded":"2024-04-25T00:39:09.176433+0000","last_fullsized":"2024-04-25T00:39:09.176433+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:04.143824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.193226+0000","last_change":"2024-04-25T00:39:09.192569+0000","last_active":"2024-04-25T00:39:09.193226+0000","last_peered":"2024-04-25T00:39:09.193226+0000","last_clean":"2024-04-25T00:39:09.193226+0000","last_became_active":"2024-04-25T00:39:09.192291+0000","last_became_peered":"2024-04-25T00:39:09.192291+0000","last_unstale":"2024-04-25T00:39:09.193226+0000","last_undegraded":"2024-04-25T00:39:09.193226+0000","last_fullsized":"2024-04-25T00:39:09.193226+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:14.248070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.135943+0000","last_change":"2024-04-25T00:39:08.152756+0000","last_active":"2024-04-25T00:39:09.135943+0000","last_peered":"2024-04-25T00:39:09.135943+0000","last_clean":"2024-04-25T00:39:09.135943+0000","last_became_active":"2024-04-25T00:39:08.152340+0000","last_became_peered":"2024-04-25T00:39:08.152340+0000","last_unstale":"2024-04-25T00:39:09.135943+0000","last_undegraded":"2024-04-25T00:39:09.135943+0000","last_fullsized":"2024-04-25T00:39:09.135943+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:27.744895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.166812+0000","last_change":"2024-04-25T00:39:09.154850+0000","last_active":"2024-04-25T00:39:09.166812+0000","last_peered":"2024-04-25T00:39:09.166812+0000","last_clean":"2024-04-25T00:39:09.166812+0000","last_became_active":"2024-04-25T00:39:09.151604+0000","last_became_peered":"2024-04-25T00:39:09.151604+0000","last_unstale":"2024-04-25T00:39:09.166812+0000","last_undegraded":"2024-04-25T00:39:09.166812+0000","last_fullsized":"2024-04-25T00:39:09.166812+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:46.262398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.183081+0000","last_change":"2024-04-25T00:39:08.159540+0000","last_active":"2024-04-25T00:39:09.183081+0000","last_peered":"2024-04-25T00:39:09.183081+0000","last_clean":"2024-04-25T00:39:09.183081+0000","last_became_active":"2024-04-25T00:39:08.159106+0000","last_became_peered":"2024-04-25T00:39:08.159106+0000","last_unstale":"2024-04-25T00:39:09.183081+0000","last_undegraded":"2024-04-25T00:39:09.183081+0000","last_fullsized":"2024-04-25T00:39:09.183081+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:21.692876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.159354+0000","last_change":"2024-04-25T00:39:09.158784+0000","last_active":"2024-04-25T00:39:09.159354+0000","last_peered":"2024-04-25T00:39:09.159354+0000","last_clean":"2024-04-25T00:39:09.159354+0000","last_became_active":"2024-04-25T00:39:09.158506+0000","last_became_peered":"2024-04-25T00:39:09.158506+0000","last_unstale":"2024-04-25T00:39:09.159354+0000","last_undegraded":"2024-04-25T00:39:09.159354+0000","last_fullsized":"2024-04-25T00:39:09.159354+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:46.496845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.164451+0000","last_change":"2024-04-25T00:39:08.153576+0000","last_active":"2024-04-25T00:39:09.164451+0000","last_peered":"2024-04-25T00:39:09.164451+0000","last_clean":"2024-04-25T00:39:09.164451+0000","last_became_active":"2024-04-25T00:39:08.152083+0000","last_became_peered":"2024-04-25T00:39:08.152083+0000","last_unstale":"2024-04-25T00:39:09.164451+0000","last_undegraded":"2024-04-25T00:39:09.164451+0000","last_fullsized":"2024-04-25T00:39:09.164451+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:01.408284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.165611+0000","last_change":"2024-04-25T00:39:09.153967+0000","last_active":"2024-04-25T00:39:09.165611+0000","last_peered":"2024-04-25T00:39:09.165611+0000","last_clean":"2024-04-25T00:39:09.165611+0000","last_became_active":"2024-04-25T00:39:09.151729+0000","last_became_peered":"2024-04-25T00:39:09.151729+0000","last_unstale":"2024-04-25T00:39:09.165611+0000","last_undegraded":"2024-04-25T00:39:09.165611+0000","last_fullsized":"2024-04-25T00:39:09.165611+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:35.538953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.172419+0000","last_change":"2024-04-25T00:39:08.157957+0000","last_active":"2024-04-25T00:39:09.172419+0000","last_peered":"2024-04-25T00:39:09.172419+0000","last_clean":"2024-04-25T00:39:09.172419+0000","last_became_active":"2024-04-25T00:39:08.155291+0000","last_became_peered":"2024-04-25T00:39:08.155291+0000","last_unstale":"2024-04-25T00:39:09.172419+0000","last_undegraded":"2024-04-25T00:39:09.172419+0000","last_fullsized":"2024-04-25T00:39:09.172419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:35.171376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.168139+0000","last_change":"2024-04-25T00:39:09.167450+0000","last_active":"2024-04-25T00:39:09.168139+0000","last_peered":"2024-04-25T00:39:09.168139+0000","last_clean":"2024-04-25T00:39:09.168139+0000","last_became_active":"2024-04-25T00:39:09.167195+0000","last_became_peered":"2024-04-25T00:39:09.167195+0000","last_unstale":"2024-04-25T00:39:09.168139+0000","last_undegraded":"2024-04-25T00:39:09.168139+0000","last_fullsized":"2024-04-25T00:39:09.168139+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:17.003258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.167387+0000","last_change":"2024-04-25T00:39:08.159847+0000","last_active":"2024-04-25T00:39:09.167387+0000","last_peered":"2024-04-25T00:39:09.167387+0000","last_clean":"2024-04-25T00:39:09.167387+0000","last_became_active":"2024-04-25T00:39:08.159154+0000","last_became_peered":"2024-04-25T00:39:08.159154+0000","last_unstale":"2024-04-25T00:39:09.167387+0000","last_undegraded":"2024-04-25T00:39:09.167387+0000","last_fullsized":"2024-04-25T00:39:09.167387+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:23.747323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.192155+0000","last_change":"2024-04-25T00:39:09.189719+0000","last_active":"2024-04-25T00:39:09.192155+0000","last_peered":"2024-04-25T00:39:09.192155+0000","last_clean":"2024-04-25T00:39:09.192155+0000","last_became_active":"2024-04-25T00:39:09.188634+0000","last_became_peered":"2024-04-25T00:39:09.188634+0000","last_unstale":"2024-04-25T00:39:09.192155+0000","last_undegraded":"2024-04-25T00:39:09.192155+0000","last_fullsized":"2024-04-25T00:39:09.192155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:20.125268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.200700+0000","last_change":"2024-04-25T00:39:08.164583+0000","last_active":"2024-04-25T00:39:09.200700+0000","last_peered":"2024-04-25T00:39:09.200700+0000","last_clean":"2024-04-25T00:39:09.200700+0000","last_became_active":"2024-04-25T00:39:08.162521+0000","last_became_peered":"2024-04-25T00:39:08.162521+0000","last_unstale":"2024-04-25T00:39:09.200700+0000","last_undegraded":"2024-04-25T00:39:09.200700+0000","last_fullsized":"2024-04-25T00:39:09.200700+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:38.966184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.199927+0000","last_change":"2024-04-25T00:39:09.191986+0000","last_active":"2024-04-25T00:39:09.199927+0000","last_peered":"2024-04-25T00:39:09.199927+0000","last_clean":"2024-04-25T00:39:09.199927+0000","last_became_active":"2024-04-25T00:39:09.190002+0000","last_became_peered":"2024-04-25T00:39:09.190002+0000","last_unstale":"2024-04-25T00:39:09.199927+0000","last_undegraded":"2024-04-25T00:39:09.199927+0000","last_fullsized":"2024-04-25T00:39:09.199927+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:55.237342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.149159+0000","last_change":"2024-04-25T00:39:08.154517+0000","last_active":"2024-04-25T00:39:09.149159+0000","last_peered":"2024-04-25T00:39:09.149159+0000","last_clean":"2024-04-25T00:39:09.149159+0000","last_became_active":"2024-04-25T00:39:08.154015+0000","last_became_peered":"2024-04-25T00:39:08.154015+0000","last_unstale":"2024-04-25T00:39:09.149159+0000","last_undegraded":"2024-04-25T00:39:09.149159+0000","last_fullsized":"2024-04-25T00:39:09.149159+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:00.260014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155702+0000","last_change":"2024-04-25T00:39:09.147903+0000","last_active":"2024-04-25T00:39:09.155702+0000","last_peered":"2024-04-25T00:39:09.155702+0000","last_clean":"2024-04-25T00:39:09.155702+0000","last_became_active":"2024-04-25T00:39:09.147583+0000","last_became_peered":"2024-04-25T00:39:09.147583+0000","last_unstale":"2024-04-25T00:39:09.155702+0000","last_undegraded":"2024-04-25T00:39:09.155702+0000","last_fullsized":"2024-04-25T00:39:09.155702+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:08.652182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.149385+0000","last_change":"2024-04-25T00:39:08.151933+0000","last_active":"2024-04-25T00:39:09.149385+0000","last_peered":"2024-04-25T00:39:09.149385+0000","last_clean":"2024-04-25T00:39:09.149385+0000","last_became_active":"2024-04-25T00:39:08.151452+0000","last_became_peered":"2024-04-25T00:39:08.151452+0000","last_unstale":"2024-04-25T00:39:09.149385+0000","last_undegraded":"2024-04-25T00:39:09.149385+0000","last_fullsized":"2024-04-25T00:39:09.149385+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:14.365515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.160704+0000","last_change":"2024-04-25T00:39:09.159585+0000","last_active":"2024-04-25T00:39:09.160704+0000","last_peered":"2024-04-25T00:39:09.160704+0000","last_clean":"2024-04-25T00:39:09.160704+0000","last_became_active":"2024-04-25T00:39:09.159303+0000","last_became_peered":"2024-04-25T00:39:09.159303+0000","last_unstale":"2024-04-25T00:39:09.160704+0000","last_undegraded":"2024-04-25T00:39:09.160704+0000","last_fullsized":"2024-04-25T00:39:09.160704+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:37.058016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.148946+0000","last_change":"2024-04-25T00:39:08.155004+0000","last_active":"2024-04-25T00:39:09.148946+0000","last_peered":"2024-04-25T00:39:09.148946+0000","last_clean":"2024-04-25T00:39:09.148946+0000","last_became_active":"2024-04-25T00:39:08.153998+0000","last_became_peered":"2024-04-25T00:39:08.153998+0000","last_unstale":"2024-04-25T00:39:09.148946+0000","last_undegraded":"2024-04-25T00:39:09.148946+0000","last_fullsized":"2024-04-25T00:39:09.148946+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:26.680458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155942+0000","last_change":"2024-04-25T00:39:09.153893+0000","last_active":"2024-04-25T00:39:09.155942+0000","last_peered":"2024-04-25T00:39:09.155942+0000","last_clean":"2024-04-25T00:39:09.155942+0000","last_became_active":"2024-04-25T00:39:09.152198+0000","last_became_peered":"2024-04-25T00:39:09.152198+0000","last_unstale":"2024-04-25T00:39:09.155942+0000","last_undegraded":"2024-04-25T00:39:09.155942+0000","last_fullsized":"2024-04-25T00:39:09.155942+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:36.537887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.143477+0000","last_change":"2024-04-25T00:39:08.172324+0000","last_active":"2024-04-25T00:39:09.143477+0000","last_peered":"2024-04-25T00:39:09.143477+0000","last_clean":"2024-04-25T00:39:09.143477+0000","last_became_active":"2024-04-25T00:39:08.171892+0000","last_became_peered":"2024-04-25T00:39:08.171892+0000","last_unstale":"2024-04-25T00:39:09.143477+0000","last_undegraded":"2024-04-25T00:39:09.143477+0000","last_fullsized":"2024-04-25T00:39:09.143477+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:02.774690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201098+0000","last_change":"2024-04-25T00:39:09.197747+0000","last_active":"2024-04-25T00:39:09.201098+0000","last_peered":"2024-04-25T00:39:09.201098+0000","last_clean":"2024-04-25T00:39:09.201098+0000","last_became_active":"2024-04-25T00:39:09.197029+0000","last_became_peered":"2024-04-25T00:39:09.197029+0000","last_unstale":"2024-04-25T00:39:09.201098+0000","last_undegraded":"2024-04-25T00:39:09.201098+0000","last_fullsized":"2024-04-25T00:39:09.201098+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:15.379539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.164877+0000","last_change":"2024-04-25T00:39:08.157128+0000","last_active":"2024-04-25T00:39:09.164877+0000","last_peered":"2024-04-25T00:39:09.164877+0000","last_clean":"2024-04-25T00:39:09.164877+0000","last_became_active":"2024-04-25T00:39:08.156406+0000","last_became_peered":"2024-04-25T00:39:08.156406+0000","last_unstale":"2024-04-25T00:39:09.164877+0000","last_undegraded":"2024-04-25T00:39:09.164877+0000","last_fullsized":"2024-04-25T00:39:09.164877+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:21.291869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201483+0000","last_change":"2024-04-25T00:39:09.186709+0000","last_active":"2024-04-25T00:39:09.201483+0000","last_peered":"2024-04-25T00:39:09.201483+0000","last_clean":"2024-04-25T00:39:09.201483+0000","last_became_active":"2024-04-25T00:39:09.185946+0000","last_became_peered":"2024-04-25T00:39:09.185946+0000","last_unstale":"2024-04-25T00:39:09.201483+0000","last_undegraded":"2024-04-25T00:39:09.201483+0000","last_fullsized":"2024-04-25T00:39:09.201483+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:03.778537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.158767+0000","last_change":"2024-04-25T00:39:08.158225+0000","last_active":"2024-04-25T00:39:09.158767+0000","last_peered":"2024-04-25T00:39:09.158767+0000","last_clean":"2024-04-25T00:39:09.158767+0000","last_became_active":"2024-04-25T00:39:08.157364+0000","last_became_peered":"2024-04-25T00:39:08.157364+0000","last_unstale":"2024-04-25T00:39:09.158767+0000","last_undegraded":"2024-04-25T00:39:09.158767+0000","last_fullsized":"2024-04-25T00:39:09.158767+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:47.893103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.168696+0000","last_change":"2024-04-25T00:39:09.161141+0000","last_active":"2024-04-25T00:39:09.168696+0000","last_peered":"2024-04-25T00:39:09.168696+0000","last_clean":"2024-04-25T00:39:09.168696+0000","last_became_active":"2024-04-25T00:39:09.160362+0000","last_became_peered":"2024-04-25T00:39:09.160362+0000","last_unstale":"2024-04-25T00:39:09.168696+0000","last_undegraded":"2024-04-25T00:39:09.168696+0000","last_fullsized":"2024-04-25T00:39:09.168696+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:55.024666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.135401+0000","last_change":"2024-04-25T00:39:08.154932+0000","last_active":"2024-04-25T00:39:09.135401+0000","last_peered":"2024-04-25T00:39:09.135401+0000","last_clean":"2024-04-25T00:39:09.135401+0000","last_became_active":"2024-04-25T00:39:08.154381+0000","last_became_peered":"2024-04-25T00:39:08.154381+0000","last_unstale":"2024-04-25T00:39:09.135401+0000","last_undegraded":"2024-04-25T00:39:09.135401+0000","last_fullsized":"2024-04-25T00:39:09.135401+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:16.874821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.160271+0000","last_change":"2024-04-25T00:39:09.159838+0000","last_active":"2024-04-25T00:39:09.160271+0000","last_peered":"2024-04-25T00:39:09.160271+0000","last_clean":"2024-04-25T00:39:09.160271+0000","last_became_active":"2024-04-25T00:39:09.159144+0000","last_became_peered":"2024-04-25T00:39:09.159144+0000","last_unstale":"2024-04-25T00:39:09.160271+0000","last_undegraded":"2024-04-25T00:39:09.160271+0000","last_fullsized":"2024-04-25T00:39:09.160271+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:10.600534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"131'3","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.193804+0000","last_change":"2024-04-25T00:39:08.164154+0000","last_active":"2024-04-25T00:39:09.193804+0000","last_peered":"2024-04-25T00:39:09.193804+0000","last_clean":"2024-04-25T00:39:09.193804+0000","last_became_active":"2024-04-25T00:39:08.160220+0000","last_became_peered":"2024-04-25T00:39:08.160220+0000","last_unstale":"2024-04-25T00:39:09.193804+0000","last_undegraded":"2024-04-25T00:39:09.193804+0000","last_fullsized":"2024-04-25T00:39:09.193804+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:13.244222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":421,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.166442+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:39:09.166442+0000","last_peered":"2024-04-25T00:39:09.166442+0000","last_clean":"2024-04-25T00:39:09.166442+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:39:09.166442+0000","last_undegraded":"2024-04-25T00:39:09.166442+0000","last_fullsized":"2024-04-25T00:39:09.166442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37516,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334324,"statfs":{"total":96636764160,"available":96598347776,"internally_reserved":0,"allocated":2224128,"data_stored":1308152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36936,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334904,"statfs":{"total":96636764160,"available":96598941696,"internally_reserved":0,"allocated":1630208,"data_stored":717784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36932,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334908,"statfs":{"total":96636764160,"available":96598945792,"internally_reserved":0,"allocated":1630208,"data_stored":717784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36936,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334904,"statfs":{"total":96636764160,"available":96598941696,"internally_reserved":0,"allocated":1630208,"data_stored":717784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36932,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334908,"statfs":{"total":96636764160,"available":96598945792,"internally_reserved":0,"allocated":1630208,"data_stored":717784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36932,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334908,"statfs":{"total":96636764160,"available":96598945792,"internally_reserved":0,"allocated":1630208,"data_stored":717784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36932,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334908,"statfs":{"total":96636764160,"available":96598945792,"internally_reserved":0,"allocated":1630208,"data_stored":717784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41676,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330164,"statfs":{"total":96636764160,"available":96594087936,"internally_reserved":0,"allocated":2224128,"data_stored":1308152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:39:13.208 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:39:13.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f3460022640 1 Processor -- start 2024-04-25T00:39:13.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f3460022640 1 -- start start 2024-04-25T00:39:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f3460022640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345815e200 0x7f34580d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f3460022640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f34580d2c60 0x7f34580b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f3460022640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34580cfc10 con 0x7f34580cf690 2024-04-25T00:39:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f3460022640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f34580b38d0 con 0x7f34580d2c60 2024-04-25T00:39:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f3460022640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f34580cfef0 con 0x7f345815e200 2024-04-25T00:39:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f345e5bf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f34580d2c60 0x7f34580b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f345e5bf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f34580d2c60 0x7f34580b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56938/0 (socket says 172.21.15.29:56938) 2024-04-25T00:39:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f345e5bf640 1 -- 172.21.15.29:0/2593026327 learned_addr learned my addr 172.21.15.29:0/2593026327 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f345ddbe640 1 --2- 172.21.15.29:0/2593026327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345815e200 0x7f34580d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f345e5bf640 1 -- 172.21.15.29:0/2593026327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345815e200 msgr2=0x7f34580d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f345e5bf640 1 --2- 172.21.15.29:0/2593026327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345815e200 0x7f34580d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f345e5bf640 1 -- 172.21.15.29:0/2593026327 >> v1:172.21.15.157:6789/0 conn(0x7f34580cf690 legacy=0x7f34580d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.362+0000 7f345e5bf640 1 -- 172.21.15.29:0/2593026327 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34580b3b20 con 0x7f34580d2c60 2024-04-25T00:39:13.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345e5bf640 1 --2- 172.21.15.29:0/2593026327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34580d2c60 0x7f34580b4320 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f3450009e50 tx=0x7f345001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b54f9d9028eda5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:13.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345cdbc640 1 -- 172.21.15.29:0/2593026327 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34500a5860 con 0x7f34580d2c60 2024-04-25T00:39:13.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345cdbc640 1 -- 172.21.15.29:0/2593026327 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34500a5a00 con 0x7f34580d2c60 2024-04-25T00:39:13.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345cdbc640 1 -- 172.21.15.29:0/2593026327 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34500a5dd0 con 0x7f34580d2c60 2024-04-25T00:39:13.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 -- 172.21.15.29:0/2593026327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34580d2c60 msgr2=0x7f34580b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 --2- 172.21.15.29:0/2593026327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34580d2c60 0x7f34580b4320 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f3450009e50 tx=0x7f345001c9c0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 -- 172.21.15.29:0/2593026327 shutdown_connections 2024-04-25T00:39:13.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 --2- 172.21.15.29:0/2593026327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34580d2c60 0x7f34580b4320 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 --2- 172.21.15.29:0/2593026327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345815e200 0x7f34580d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 -- 172.21.15.29:0/2593026327 >> 172.21.15.29:0/2593026327 conn(0x7f345809d280 msgr2=0x7f34580d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:13.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 -- 172.21.15.29:0/2593026327 shutdown_connections 2024-04-25T00:39:13.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 -- 172.21.15.29:0/2593026327 wait complete. 2024-04-25T00:39:13.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 Processor -- start 2024-04-25T00:39:13.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 -- start start 2024-04-25T00:39:13.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34580cf690 0x7f34581561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34580d2c60 0x7f34580c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345815e200 0x7f34580ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f34580b51f0 con 0x7f34580cf690 2024-04-25T00:39:13.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34580b5070 con 0x7f34580d2c60 2024-04-25T00:39:13.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f34580b5370 con 0x7f345815e200 2024-04-25T00:39:13.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345ddbe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34580cf690 0x7f34581561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:13.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345ddbe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34580cf690 0x7f34581561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56944/0 (socket says 172.21.15.29:56944) 2024-04-25T00:39:13.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345ddbe640 1 -- 172.21.15.29:0/2160846629 learned_addr learned my addr 172.21.15.29:0/2160846629 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:13.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345e5bf640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345815e200 0x7f34580ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:13.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345d5bd640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34580d2c60 0x7f34580c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:13.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345ddbe640 1 -- 172.21.15.29:0/2160846629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345815e200 msgr2=0x7f34580ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345ddbe640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345815e200 0x7f34580ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345ddbe640 1 -- 172.21.15.29:0/2160846629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34580d2c60 msgr2=0x7f34580c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345ddbe640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34580d2c60 0x7f34580c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345ddbe640 1 -- 172.21.15.29:0/2160846629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3458156f00 con 0x7f34580cf690 2024-04-25T00:39:13.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345e5bf640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345815e200 0x7f34580ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:13.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345d5bd640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34580d2c60 0x7f34580c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:13.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f345ddbe640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34580cf690 0x7f34581561d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f345400c7f0 tx=0x7f345400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:13.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f344e7fc640 1 -- 172.21.15.29:0/2160846629 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454015070 con 0x7f34580cf690 2024-04-25T00:39:13.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f344e7fc640 1 -- 172.21.15.29:0/2160846629 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3454007820 con 0x7f34580cf690 2024-04-25T00:39:13.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 -- 172.21.15.29:0/2160846629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3458155410 con 0x7f34580cf690 2024-04-25T00:39:13.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f344e7fc640 1 -- 172.21.15.29:0/2160846629 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454007c10 con 0x7f34580cf690 2024-04-25T00:39:13.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.366+0000 7f3460022640 1 -- 172.21.15.29:0/2160846629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34581559d0 con 0x7f34580cf690 2024-04-25T00:39:13.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.370+0000 7f3460022640 1 -- 172.21.15.29:0/2160846629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3458069270 con 0x7f34580cf690 2024-04-25T00:39:13.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.370+0000 7f344e7fc640 1 -- 172.21.15.29:0/2160846629 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f3454018070 con 0x7f34580cf690 2024-04-25T00:39:13.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.370+0000 7f344e7fc640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f343c067e90 0x7f343c06a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.370+0000 7f344e7fc640 1 -- 172.21.15.29:0/2160846629 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f3454086cd0 con 0x7f34580cf690 2024-04-25T00:39:13.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.370+0000 7f345d5bd640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f343c067e90 0x7f343c06a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:13.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.370+0000 7f345d5bd640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f343c067e90 0x7f343c06a350 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f34580d0c10 tx=0x7f3448008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:13.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.374+0000 7f344e7fc640 1 -- 172.21.15.29:0/2160846629 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3454056f60 con 0x7f34580cf690 2024-04-25T00:39:13.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.526+0000 7f3460022640 1 -- 172.21.15.29:0/2160846629 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f34580b19b0 con 0x7f343c067e90 2024-04-25T00:39:13.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.534+0000 7f344e7fc640 1 -- 172.21.15.29:0/2160846629 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252420 (secure 0 0 0) 0x7f34580b19b0 con 0x7f343c067e90 2024-04-25T00:39:13.540 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:13.541 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:39:13.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.538+0000 7f3460022640 1 -- 172.21.15.29:0/2160846629 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f343c067e90 msgr2=0x7f343c06a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.538+0000 7f3460022640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f343c067e90 0x7f343c06a350 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f34580d0c10 tx=0x7f3448008040 comp rx=0 tx=0).stop 2024-04-25T00:39:13.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.538+0000 7f3460022640 1 -- 172.21.15.29:0/2160846629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34580cf690 msgr2=0x7f34581561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.538+0000 7f3460022640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34580cf690 0x7f34581561d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f345400c7f0 tx=0x7f345400ccb0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.538+0000 7f3460022640 1 -- 172.21.15.29:0/2160846629 shutdown_connections 2024-04-25T00:39:13.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.538+0000 7f3460022640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f343c067e90 0x7f343c06a350 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.538+0000 7f3460022640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345815e200 0x7f34580ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.538+0000 7f3460022640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34580d2c60 0x7f34580c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.538+0000 7f3460022640 1 --2- 172.21.15.29:0/2160846629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34580cf690 0x7f34581561d0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.538+0000 7f3460022640 1 -- 172.21.15.29:0/2160846629 >> 172.21.15.29:0/2160846629 conn(0x7f345809d280 msgr2=0x7f34580b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:13.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.538+0000 7f3460022640 1 -- 172.21.15.29:0/2160846629 shutdown_connections 2024-04-25T00:39:13.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.538+0000 7f3460022640 1 -- 172.21.15.29:0/2160846629 wait complete. 2024-04-25T00:39:13.557 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-04-25T00:39:13.381076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":300792,"kb_used_data":13896,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754673928,"statfs":{"total":773094113280,"available":772786102272,"internally_reserved":0,"allocated":14229504,"data_stored":6923008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":293654064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001591"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.192970+0000","last_change":"2024-04-25T00:39:09.191812+0000","last_active":"2024-04-25T00:39:09.192970+0000","last_peered":"2024-04-25T00:39:09.192970+0000","last_clean":"2024-04-25T00:39:09.192970+0000","last_became_active":"2024-04-25T00:39:09.191536+0000","last_became_peered":"2024-04-25T00:39:09.191536+0000","last_unstale":"2024-04-25T00:39:09.192970+0000","last_undegraded":"2024-04-25T00:39:09.192970+0000","last_fullsized":"2024-04-25T00:39:09.192970+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:48.896950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.157212+0000","last_change":"2024-04-25T00:39:09.152951+0000","last_active":"2024-04-25T00:39:09.157212+0000","last_peered":"2024-04-25T00:39:09.157212+0000","last_clean":"2024-04-25T00:39:09.157212+0000","last_became_active":"2024-04-25T00:39:09.152679+0000","last_became_peered":"2024-04-25T00:39:09.152679+0000","last_unstale":"2024-04-25T00:39:09.157212+0000","last_undegraded":"2024-04-25T00:39:09.157212+0000","last_fullsized":"2024-04-25T00:39:09.157212+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:22.295716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201167+0000","last_change":"2024-04-25T00:39:09.198811+0000","last_active":"2024-04-25T00:39:09.201167+0000","last_peered":"2024-04-25T00:39:09.201167+0000","last_clean":"2024-04-25T00:39:09.201167+0000","last_became_active":"2024-04-25T00:39:09.192819+0000","last_became_peered":"2024-04-25T00:39:09.192819+0000","last_unstale":"2024-04-25T00:39:09.201167+0000","last_undegraded":"2024-04-25T00:39:09.201167+0000","last_fullsized":"2024-04-25T00:39:09.201167+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:25.791648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.202845+0000","last_change":"2024-04-25T00:39:09.201419+0000","last_active":"2024-04-25T00:39:09.202845+0000","last_peered":"2024-04-25T00:39:09.202845+0000","last_clean":"2024-04-25T00:39:09.202845+0000","last_became_active":"2024-04-25T00:39:09.200615+0000","last_became_peered":"2024-04-25T00:39:09.200615+0000","last_unstale":"2024-04-25T00:39:09.202845+0000","last_undegraded":"2024-04-25T00:39:09.202845+0000","last_fullsized":"2024-04-25T00:39:09.202845+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:02.368131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.191115+0000","last_change":"2024-04-25T00:39:09.186592+0000","last_active":"2024-04-25T00:39:09.191115+0000","last_peered":"2024-04-25T00:39:09.191115+0000","last_clean":"2024-04-25T00:39:09.191115+0000","last_became_active":"2024-04-25T00:39:09.185849+0000","last_became_peered":"2024-04-25T00:39:09.185849+0000","last_unstale":"2024-04-25T00:39:09.191115+0000","last_undegraded":"2024-04-25T00:39:09.191115+0000","last_fullsized":"2024-04-25T00:39:09.191115+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:41.680770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.199733+0000","last_change":"2024-04-25T00:39:09.199104+0000","last_active":"2024-04-25T00:39:09.199733+0000","last_peered":"2024-04-25T00:39:09.199733+0000","last_clean":"2024-04-25T00:39:09.199733+0000","last_became_active":"2024-04-25T00:39:09.198781+0000","last_became_peered":"2024-04-25T00:39:09.198781+0000","last_unstale":"2024-04-25T00:39:09.199733+0000","last_undegraded":"2024-04-25T00:39:09.199733+0000","last_fullsized":"2024-04-25T00:39:09.199733+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:25.542008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.168164+0000","last_change":"2024-04-25T00:39:09.167430+0000","last_active":"2024-04-25T00:39:09.168164+0000","last_peered":"2024-04-25T00:39:09.168164+0000","last_clean":"2024-04-25T00:39:09.168164+0000","last_became_active":"2024-04-25T00:39:09.167140+0000","last_became_peered":"2024-04-25T00:39:09.167140+0000","last_unstale":"2024-04-25T00:39:09.168164+0000","last_undegraded":"2024-04-25T00:39:09.168164+0000","last_fullsized":"2024-04-25T00:39:09.168164+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:07.042416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.166683+0000","last_change":"2024-04-25T00:39:09.165838+0000","last_active":"2024-04-25T00:39:09.166683+0000","last_peered":"2024-04-25T00:39:09.166683+0000","last_clean":"2024-04-25T00:39:09.166683+0000","last_became_active":"2024-04-25T00:39:09.165529+0000","last_became_peered":"2024-04-25T00:39:09.165529+0000","last_unstale":"2024-04-25T00:39:09.166683+0000","last_undegraded":"2024-04-25T00:39:09.166683+0000","last_fullsized":"2024-04-25T00:39:09.166683+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:16:34.583678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155629+0000","last_change":"2024-04-25T00:39:09.152024+0000","last_active":"2024-04-25T00:39:09.155629+0000","last_peered":"2024-04-25T00:39:09.155629+0000","last_clean":"2024-04-25T00:39:09.155629+0000","last_became_active":"2024-04-25T00:39:09.151669+0000","last_became_peered":"2024-04-25T00:39:09.151669+0000","last_unstale":"2024-04-25T00:39:09.155629+0000","last_undegraded":"2024-04-25T00:39:09.155629+0000","last_fullsized":"2024-04-25T00:39:09.155629+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:38.040554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201843+0000","last_change":"2024-04-25T00:39:09.190350+0000","last_active":"2024-04-25T00:39:09.201843+0000","last_peered":"2024-04-25T00:39:09.201843+0000","last_clean":"2024-04-25T00:39:09.201843+0000","last_became_active":"2024-04-25T00:39:09.189633+0000","last_became_peered":"2024-04-25T00:39:09.189633+0000","last_unstale":"2024-04-25T00:39:09.201843+0000","last_undegraded":"2024-04-25T00:39:09.201843+0000","last_fullsized":"2024-04-25T00:39:09.201843+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:07.042416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.166690+0000","last_change":"2024-04-25T00:39:09.165896+0000","last_active":"2024-04-25T00:39:09.166690+0000","last_peered":"2024-04-25T00:39:09.166690+0000","last_clean":"2024-04-25T00:39:09.166690+0000","last_became_active":"2024-04-25T00:39:09.165552+0000","last_became_peered":"2024-04-25T00:39:09.165552+0000","last_unstale":"2024-04-25T00:39:09.166690+0000","last_undegraded":"2024-04-25T00:39:09.166690+0000","last_fullsized":"2024-04-25T00:39:09.166690+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:22.049618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201271+0000","last_change":"2024-04-25T00:39:09.186618+0000","last_active":"2024-04-25T00:39:09.201271+0000","last_peered":"2024-04-25T00:39:09.201271+0000","last_clean":"2024-04-25T00:39:09.201271+0000","last_became_active":"2024-04-25T00:39:09.185892+0000","last_became_peered":"2024-04-25T00:39:09.185892+0000","last_unstale":"2024-04-25T00:39:09.201271+0000","last_undegraded":"2024-04-25T00:39:09.201271+0000","last_fullsized":"2024-04-25T00:39:09.201271+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:44.551679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.186940+0000","last_change":"2024-04-25T00:39:09.184541+0000","last_active":"2024-04-25T00:39:09.186940+0000","last_peered":"2024-04-25T00:39:09.186940+0000","last_clean":"2024-04-25T00:39:09.186940+0000","last_became_active":"2024-04-25T00:39:09.183778+0000","last_became_peered":"2024-04-25T00:39:09.183778+0000","last_unstale":"2024-04-25T00:39:09.186940+0000","last_undegraded":"2024-04-25T00:39:09.186940+0000","last_fullsized":"2024-04-25T00:39:09.186940+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:41.934765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.188231+0000","last_change":"2024-04-25T00:39:09.183928+0000","last_active":"2024-04-25T00:39:09.188231+0000","last_peered":"2024-04-25T00:39:09.188231+0000","last_clean":"2024-04-25T00:39:09.188231+0000","last_became_active":"2024-04-25T00:39:09.183347+0000","last_became_peered":"2024-04-25T00:39:09.183347+0000","last_unstale":"2024-04-25T00:39:09.188231+0000","last_undegraded":"2024-04-25T00:39:09.188231+0000","last_fullsized":"2024-04-25T00:39:09.188231+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:52.463025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201861+0000","last_change":"2024-04-25T00:39:09.190430+0000","last_active":"2024-04-25T00:39:09.201861+0000","last_peered":"2024-04-25T00:39:09.201861+0000","last_clean":"2024-04-25T00:39:09.201861+0000","last_became_active":"2024-04-25T00:39:09.189697+0000","last_became_peered":"2024-04-25T00:39:09.189697+0000","last_unstale":"2024-04-25T00:39:09.201861+0000","last_undegraded":"2024-04-25T00:39:09.201861+0000","last_fullsized":"2024-04-25T00:39:09.201861+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:10.600534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.158330+0000","last_change":"2024-04-25T00:39:09.157015+0000","last_active":"2024-04-25T00:39:09.158330+0000","last_peered":"2024-04-25T00:39:09.158330+0000","last_clean":"2024-04-25T00:39:09.158330+0000","last_became_active":"2024-04-25T00:39:09.156152+0000","last_became_peered":"2024-04-25T00:39:09.156152+0000","last_unstale":"2024-04-25T00:39:09.158330+0000","last_undegraded":"2024-04-25T00:39:09.158330+0000","last_fullsized":"2024-04-25T00:39:09.158330+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:41.536446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.202180+0000","last_change":"2024-04-25T00:39:09.194551+0000","last_active":"2024-04-25T00:39:09.202180+0000","last_peered":"2024-04-25T00:39:09.202180+0000","last_clean":"2024-04-25T00:39:09.202180+0000","last_became_active":"2024-04-25T00:39:09.193023+0000","last_became_peered":"2024-04-25T00:39:09.193023+0000","last_unstale":"2024-04-25T00:39:09.202180+0000","last_undegraded":"2024-04-25T00:39:09.202180+0000","last_fullsized":"2024-04-25T00:39:09.202180+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:46.496845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.199941+0000","last_change":"2024-04-25T00:39:09.195131+0000","last_active":"2024-04-25T00:39:09.199941+0000","last_peered":"2024-04-25T00:39:09.199941+0000","last_clean":"2024-04-25T00:39:09.199941+0000","last_became_active":"2024-04-25T00:39:09.192410+0000","last_became_peered":"2024-04-25T00:39:09.192410+0000","last_unstale":"2024-04-25T00:39:09.199941+0000","last_undegraded":"2024-04-25T00:39:09.199941+0000","last_fullsized":"2024-04-25T00:39:09.199941+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:52.372514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.160270+0000","last_change":"2024-04-25T00:39:09.157066+0000","last_active":"2024-04-25T00:39:09.160270+0000","last_peered":"2024-04-25T00:39:09.160270+0000","last_clean":"2024-04-25T00:39:09.160270+0000","last_became_active":"2024-04-25T00:39:09.156664+0000","last_became_peered":"2024-04-25T00:39:09.156664+0000","last_unstale":"2024-04-25T00:39:09.160270+0000","last_undegraded":"2024-04-25T00:39:09.160270+0000","last_fullsized":"2024-04-25T00:39:09.160270+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:13.920122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.158296+0000","last_change":"2024-04-25T00:39:09.156657+0000","last_active":"2024-04-25T00:39:09.158296+0000","last_peered":"2024-04-25T00:39:09.158296+0000","last_clean":"2024-04-25T00:39:09.158296+0000","last_became_active":"2024-04-25T00:39:09.156354+0000","last_became_peered":"2024-04-25T00:39:09.156354+0000","last_unstale":"2024-04-25T00:39:09.158296+0000","last_undegraded":"2024-04-25T00:39:09.158296+0000","last_fullsized":"2024-04-25T00:39:09.158296+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:52.158456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201826+0000","last_change":"2024-04-25T00:39:09.188657+0000","last_active":"2024-04-25T00:39:09.201826+0000","last_peered":"2024-04-25T00:39:09.201826+0000","last_clean":"2024-04-25T00:39:09.201826+0000","last_became_active":"2024-04-25T00:39:09.187471+0000","last_became_peered":"2024-04-25T00:39:09.187471+0000","last_unstale":"2024-04-25T00:39:09.201826+0000","last_undegraded":"2024-04-25T00:39:09.201826+0000","last_fullsized":"2024-04-25T00:39:09.201826+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:22.696723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.179340+0000","last_change":"2024-04-25T00:39:09.163036+0000","last_active":"2024-04-25T00:39:09.179340+0000","last_peered":"2024-04-25T00:39:09.179340+0000","last_clean":"2024-04-25T00:39:09.179340+0000","last_became_active":"2024-04-25T00:39:09.160768+0000","last_became_peered":"2024-04-25T00:39:09.160768+0000","last_unstale":"2024-04-25T00:39:09.179340+0000","last_undegraded":"2024-04-25T00:39:09.179340+0000","last_fullsized":"2024-04-25T00:39:09.179340+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:42.801159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.171121+0000","last_change":"2024-04-25T00:39:09.156106+0000","last_active":"2024-04-25T00:39:09.171121+0000","last_peered":"2024-04-25T00:39:09.171121+0000","last_clean":"2024-04-25T00:39:09.171121+0000","last_became_active":"2024-04-25T00:39:09.155237+0000","last_became_peered":"2024-04-25T00:39:09.155237+0000","last_unstale":"2024-04-25T00:39:09.171121+0000","last_undegraded":"2024-04-25T00:39:09.171121+0000","last_fullsized":"2024-04-25T00:39:09.171121+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:50.835641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.197032+0000","last_change":"2024-04-25T00:39:09.184470+0000","last_active":"2024-04-25T00:39:09.197032+0000","last_peered":"2024-04-25T00:39:09.197032+0000","last_clean":"2024-04-25T00:39:09.197032+0000","last_became_active":"2024-04-25T00:39:09.183743+0000","last_became_peered":"2024-04-25T00:39:09.183743+0000","last_unstale":"2024-04-25T00:39:09.197032+0000","last_undegraded":"2024-04-25T00:39:09.197032+0000","last_fullsized":"2024-04-25T00:39:09.197032+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:08.495108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155772+0000","last_change":"2024-04-25T00:39:09.153476+0000","last_active":"2024-04-25T00:39:09.155772+0000","last_peered":"2024-04-25T00:39:09.155772+0000","last_clean":"2024-04-25T00:39:09.155772+0000","last_became_active":"2024-04-25T00:39:09.153100+0000","last_became_peered":"2024-04-25T00:39:09.153100+0000","last_unstale":"2024-04-25T00:39:09.155772+0000","last_undegraded":"2024-04-25T00:39:09.155772+0000","last_fullsized":"2024-04-25T00:39:09.155772+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:49.211440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.160223+0000","last_change":"2024-04-25T00:39:09.156996+0000","last_active":"2024-04-25T00:39:09.160223+0000","last_peered":"2024-04-25T00:39:09.160223+0000","last_clean":"2024-04-25T00:39:09.160223+0000","last_became_active":"2024-04-25T00:39:09.156670+0000","last_became_peered":"2024-04-25T00:39:09.156670+0000","last_unstale":"2024-04-25T00:39:09.160223+0000","last_undegraded":"2024-04-25T00:39:09.160223+0000","last_fullsized":"2024-04-25T00:39:09.160223+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:55.380913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.193250+0000","last_change":"2024-04-25T00:39:09.192729+0000","last_active":"2024-04-25T00:39:09.193250+0000","last_peered":"2024-04-25T00:39:09.193250+0000","last_clean":"2024-04-25T00:39:09.193250+0000","last_became_active":"2024-04-25T00:39:09.192303+0000","last_became_peered":"2024-04-25T00:39:09.192303+0000","last_unstale":"2024-04-25T00:39:09.193250+0000","last_undegraded":"2024-04-25T00:39:09.193250+0000","last_fullsized":"2024-04-25T00:39:09.193250+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:55.024666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.196927+0000","last_change":"2024-04-25T00:39:09.196470+0000","last_active":"2024-04-25T00:39:09.196927+0000","last_peered":"2024-04-25T00:39:09.196927+0000","last_clean":"2024-04-25T00:39:09.196927+0000","last_became_active":"2024-04-25T00:39:09.196145+0000","last_became_peered":"2024-04-25T00:39:09.196145+0000","last_unstale":"2024-04-25T00:39:09.196927+0000","last_undegraded":"2024-04-25T00:39:09.196927+0000","last_fullsized":"2024-04-25T00:39:09.196927+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:04.851312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.192405+0000","last_change":"2024-04-25T00:39:09.191662+0000","last_active":"2024-04-25T00:39:09.192405+0000","last_peered":"2024-04-25T00:39:09.192405+0000","last_clean":"2024-04-25T00:39:09.192405+0000","last_became_active":"2024-04-25T00:39:09.191352+0000","last_became_peered":"2024-04-25T00:39:09.191352+0000","last_unstale":"2024-04-25T00:39:09.192405+0000","last_undegraded":"2024-04-25T00:39:09.192405+0000","last_fullsized":"2024-04-25T00:39:09.192405+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:43.815701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.165743+0000","last_change":"2024-04-25T00:39:09.153864+0000","last_active":"2024-04-25T00:39:09.165743+0000","last_peered":"2024-04-25T00:39:09.165743+0000","last_clean":"2024-04-25T00:39:09.165743+0000","last_became_active":"2024-04-25T00:39:09.151787+0000","last_became_peered":"2024-04-25T00:39:09.151787+0000","last_unstale":"2024-04-25T00:39:09.165743+0000","last_undegraded":"2024-04-25T00:39:09.165743+0000","last_fullsized":"2024-04-25T00:39:09.165743+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:38.827706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.182726+0000","last_change":"2024-04-25T00:39:09.182134+0000","last_active":"2024-04-25T00:39:09.182726+0000","last_peered":"2024-04-25T00:39:09.182726+0000","last_clean":"2024-04-25T00:39:09.182726+0000","last_became_active":"2024-04-25T00:39:09.181775+0000","last_became_peered":"2024-04-25T00:39:09.181775+0000","last_unstale":"2024-04-25T00:39:09.182726+0000","last_undegraded":"2024-04-25T00:39:09.182726+0000","last_fullsized":"2024-04-25T00:39:09.182726+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:36.175223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.159540+0000","last_change":"2024-04-25T00:39:09.158234+0000","last_active":"2024-04-25T00:39:09.159540+0000","last_peered":"2024-04-25T00:39:09.159540+0000","last_clean":"2024-04-25T00:39:09.159540+0000","last_became_active":"2024-04-25T00:39:09.157929+0000","last_became_peered":"2024-04-25T00:39:09.157929+0000","last_unstale":"2024-04-25T00:39:09.159540+0000","last_undegraded":"2024-04-25T00:39:09.159540+0000","last_fullsized":"2024-04-25T00:39:09.159540+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:37.038466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.202688+0000","last_change":"2024-04-25T00:39:09.197970+0000","last_active":"2024-04-25T00:39:09.202688+0000","last_peered":"2024-04-25T00:39:09.202688+0000","last_clean":"2024-04-25T00:39:09.202688+0000","last_became_active":"2024-04-25T00:39:09.197332+0000","last_became_peered":"2024-04-25T00:39:09.197332+0000","last_unstale":"2024-04-25T00:39:09.202688+0000","last_undegraded":"2024-04-25T00:39:09.202688+0000","last_fullsized":"2024-04-25T00:39:09.202688+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:04.851312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"131'4","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201896+0000","last_change":"2024-04-25T00:39:08.156446+0000","last_active":"2024-04-25T00:39:09.201896+0000","last_peered":"2024-04-25T00:39:09.201896+0000","last_clean":"2024-04-25T00:39:09.201896+0000","last_became_active":"2024-04-25T00:39:08.155076+0000","last_became_peered":"2024-04-25T00:39:08.155076+0000","last_unstale":"2024-04-25T00:39:09.201896+0000","last_undegraded":"2024-04-25T00:39:09.201896+0000","last_fullsized":"2024-04-25T00:39:09.201896+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:32.569417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155660+0000","last_change":"2024-04-25T00:39:09.152136+0000","last_active":"2024-04-25T00:39:09.155660+0000","last_peered":"2024-04-25T00:39:09.155660+0000","last_clean":"2024-04-25T00:39:09.155660+0000","last_became_active":"2024-04-25T00:39:09.151826+0000","last_became_peered":"2024-04-25T00:39:09.151826+0000","last_unstale":"2024-04-25T00:39:09.155660+0000","last_undegraded":"2024-04-25T00:39:09.155660+0000","last_fullsized":"2024-04-25T00:39:09.155660+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:08.203435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.175042+0000","last_change":"2024-04-25T00:39:08.154558+0000","last_active":"2024-04-25T00:39:09.175042+0000","last_peered":"2024-04-25T00:39:09.175042+0000","last_clean":"2024-04-25T00:39:09.175042+0000","last_became_active":"2024-04-25T00:39:08.154046+0000","last_became_peered":"2024-04-25T00:39:08.154046+0000","last_unstale":"2024-04-25T00:39:09.175042+0000","last_undegraded":"2024-04-25T00:39:09.175042+0000","last_fullsized":"2024-04-25T00:39:09.175042+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:50.223659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.160781+0000","last_change":"2024-04-25T00:39:09.159968+0000","last_active":"2024-04-25T00:39:09.160781+0000","last_peered":"2024-04-25T00:39:09.160781+0000","last_clean":"2024-04-25T00:39:09.160781+0000","last_became_active":"2024-04-25T00:39:09.159484+0000","last_became_peered":"2024-04-25T00:39:09.159484+0000","last_unstale":"2024-04-25T00:39:09.160781+0000","last_undegraded":"2024-04-25T00:39:09.160781+0000","last_fullsized":"2024-04-25T00:39:09.160781+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:34.014780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.171403+0000","last_change":"2024-04-25T00:39:08.168582+0000","last_active":"2024-04-25T00:39:09.171403+0000","last_peered":"2024-04-25T00:39:09.171403+0000","last_clean":"2024-04-25T00:39:09.171403+0000","last_became_active":"2024-04-25T00:39:08.166773+0000","last_became_peered":"2024-04-25T00:39:08.166773+0000","last_unstale":"2024-04-25T00:39:09.171403+0000","last_undegraded":"2024-04-25T00:39:09.171403+0000","last_fullsized":"2024-04-25T00:39:09.171403+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:04.006426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201653+0000","last_change":"2024-04-25T00:39:09.188040+0000","last_active":"2024-04-25T00:39:09.201653+0000","last_peered":"2024-04-25T00:39:09.201653+0000","last_clean":"2024-04-25T00:39:09.201653+0000","last_became_active":"2024-04-25T00:39:09.187252+0000","last_became_peered":"2024-04-25T00:39:09.187252+0000","last_unstale":"2024-04-25T00:39:09.201653+0000","last_undegraded":"2024-04-25T00:39:09.201653+0000","last_fullsized":"2024-04-25T00:39:09.201653+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:35.916708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.177403+0000","last_change":"2024-04-25T00:39:08.162907+0000","last_active":"2024-04-25T00:39:09.177403+0000","last_peered":"2024-04-25T00:39:09.177403+0000","last_clean":"2024-04-25T00:39:09.177403+0000","last_became_active":"2024-04-25T00:39:08.156446+0000","last_became_peered":"2024-04-25T00:39:08.156446+0000","last_unstale":"2024-04-25T00:39:09.177403+0000","last_undegraded":"2024-04-25T00:39:09.177403+0000","last_fullsized":"2024-04-25T00:39:09.177403+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:42.035329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.200405+0000","last_change":"2024-04-25T00:39:09.199971+0000","last_active":"2024-04-25T00:39:09.200405+0000","last_peered":"2024-04-25T00:39:09.200405+0000","last_clean":"2024-04-25T00:39:09.200405+0000","last_became_active":"2024-04-25T00:39:09.199627+0000","last_became_peered":"2024-04-25T00:39:09.199627+0000","last_unstale":"2024-04-25T00:39:09.200405+0000","last_undegraded":"2024-04-25T00:39:09.200405+0000","last_fullsized":"2024-04-25T00:39:09.200405+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:33.573264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.164780+0000","last_change":"2024-04-25T00:39:08.151302+0000","last_active":"2024-04-25T00:39:09.164780+0000","last_peered":"2024-04-25T00:39:09.164780+0000","last_clean":"2024-04-25T00:39:09.164780+0000","last_became_active":"2024-04-25T00:39:08.148783+0000","last_became_peered":"2024-04-25T00:39:08.148783+0000","last_unstale":"2024-04-25T00:39:09.164780+0000","last_undegraded":"2024-04-25T00:39:09.164780+0000","last_fullsized":"2024-04-25T00:39:09.164780+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:42.967705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.197570+0000","last_change":"2024-04-25T00:39:09.194869+0000","last_active":"2024-04-25T00:39:09.197570+0000","last_peered":"2024-04-25T00:39:09.197570+0000","last_clean":"2024-04-25T00:39:09.197570+0000","last_became_active":"2024-04-25T00:39:09.194200+0000","last_became_peered":"2024-04-25T00:39:09.194200+0000","last_unstale":"2024-04-25T00:39:09.197570+0000","last_undegraded":"2024-04-25T00:39:09.197570+0000","last_fullsized":"2024-04-25T00:39:09.197570+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:38.623765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.173733+0000","last_change":"2024-04-25T00:39:08.171567+0000","last_active":"2024-04-25T00:39:09.173733+0000","last_peered":"2024-04-25T00:39:09.173733+0000","last_clean":"2024-04-25T00:39:09.173733+0000","last_became_active":"2024-04-25T00:39:08.170833+0000","last_became_peered":"2024-04-25T00:39:08.170833+0000","last_unstale":"2024-04-25T00:39:09.173733+0000","last_undegraded":"2024-04-25T00:39:09.173733+0000","last_fullsized":"2024-04-25T00:39:09.173733+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:51.154609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155632+0000","last_change":"2024-04-25T00:39:09.148908+0000","last_active":"2024-04-25T00:39:09.155632+0000","last_peered":"2024-04-25T00:39:09.155632+0000","last_clean":"2024-04-25T00:39:09.155632+0000","last_became_active":"2024-04-25T00:39:09.148603+0000","last_became_peered":"2024-04-25T00:39:09.148603+0000","last_unstale":"2024-04-25T00:39:09.155632+0000","last_undegraded":"2024-04-25T00:39:09.155632+0000","last_fullsized":"2024-04-25T00:39:09.155632+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:15.292122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.140734+0000","last_change":"2024-04-25T00:39:08.171081+0000","last_active":"2024-04-25T00:39:09.140734+0000","last_peered":"2024-04-25T00:39:09.140734+0000","last_clean":"2024-04-25T00:39:09.140734+0000","last_became_active":"2024-04-25T00:39:08.170396+0000","last_became_peered":"2024-04-25T00:39:08.170396+0000","last_unstale":"2024-04-25T00:39:09.140734+0000","last_undegraded":"2024-04-25T00:39:09.140734+0000","last_fullsized":"2024-04-25T00:39:09.140734+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:48.839793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.168643+0000","last_change":"2024-04-25T00:39:09.159430+0000","last_active":"2024-04-25T00:39:09.168643+0000","last_peered":"2024-04-25T00:39:09.168643+0000","last_clean":"2024-04-25T00:39:09.168643+0000","last_became_active":"2024-04-25T00:39:09.158261+0000","last_became_peered":"2024-04-25T00:39:09.158261+0000","last_unstale":"2024-04-25T00:39:09.168643+0000","last_undegraded":"2024-04-25T00:39:09.168643+0000","last_fullsized":"2024-04-25T00:39:09.168643+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:11.878064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.139432+0000","last_change":"2024-04-25T00:39:08.172987+0000","last_active":"2024-04-25T00:39:09.139432+0000","last_peered":"2024-04-25T00:39:09.139432+0000","last_clean":"2024-04-25T00:39:09.139432+0000","last_became_active":"2024-04-25T00:39:08.171954+0000","last_became_peered":"2024-04-25T00:39:08.171954+0000","last_unstale":"2024-04-25T00:39:09.139432+0000","last_undegraded":"2024-04-25T00:39:09.139432+0000","last_fullsized":"2024-04-25T00:39:09.139432+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:11.695113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.168605+0000","last_change":"2024-04-25T00:39:09.159261+0000","last_active":"2024-04-25T00:39:09.168605+0000","last_peered":"2024-04-25T00:39:09.168605+0000","last_clean":"2024-04-25T00:39:09.168605+0000","last_became_active":"2024-04-25T00:39:09.158572+0000","last_became_peered":"2024-04-25T00:39:09.158572+0000","last_unstale":"2024-04-25T00:39:09.168605+0000","last_undegraded":"2024-04-25T00:39:09.168605+0000","last_fullsized":"2024-04-25T00:39:09.168605+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:37.038466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.136604+0000","last_change":"2024-04-25T00:39:08.154902+0000","last_active":"2024-04-25T00:39:09.136604+0000","last_peered":"2024-04-25T00:39:09.136604+0000","last_clean":"2024-04-25T00:39:09.136604+0000","last_became_active":"2024-04-25T00:39:08.154483+0000","last_became_peered":"2024-04-25T00:39:08.154483+0000","last_unstale":"2024-04-25T00:39:09.136604+0000","last_undegraded":"2024-04-25T00:39:09.136604+0000","last_fullsized":"2024-04-25T00:39:09.136604+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:33.875632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.202715+0000","last_change":"2024-04-25T00:39:09.200113+0000","last_active":"2024-04-25T00:39:09.202715+0000","last_peered":"2024-04-25T00:39:09.202715+0000","last_clean":"2024-04-25T00:39:09.202715+0000","last_became_active":"2024-04-25T00:39:09.195768+0000","last_became_peered":"2024-04-25T00:39:09.195768+0000","last_unstale":"2024-04-25T00:39:09.202715+0000","last_undegraded":"2024-04-25T00:39:09.202715+0000","last_fullsized":"2024-04-25T00:39:09.202715+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:44.534637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.175099+0000","last_change":"2024-04-25T00:39:08.168721+0000","last_active":"2024-04-25T00:39:09.175099+0000","last_peered":"2024-04-25T00:39:09.175099+0000","last_clean":"2024-04-25T00:39:09.175099+0000","last_became_active":"2024-04-25T00:39:08.167961+0000","last_became_peered":"2024-04-25T00:39:08.167961+0000","last_unstale":"2024-04-25T00:39:09.175099+0000","last_undegraded":"2024-04-25T00:39:09.175099+0000","last_fullsized":"2024-04-25T00:39:09.175099+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:32.064317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155703+0000","last_change":"2024-04-25T00:39:09.150902+0000","last_active":"2024-04-25T00:39:09.155703+0000","last_peered":"2024-04-25T00:39:09.155703+0000","last_clean":"2024-04-25T00:39:09.155703+0000","last_became_active":"2024-04-25T00:39:09.150527+0000","last_became_peered":"2024-04-25T00:39:09.150527+0000","last_unstale":"2024-04-25T00:39:09.155703+0000","last_undegraded":"2024-04-25T00:39:09.155703+0000","last_fullsized":"2024-04-25T00:39:09.155703+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:38.603505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"131'6","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.166497+0000","last_change":"2024-04-25T00:39:08.156557+0000","last_active":"2024-04-25T00:39:09.166497+0000","last_peered":"2024-04-25T00:39:09.166497+0000","last_clean":"2024-04-25T00:39:09.166497+0000","last_became_active":"2024-04-25T00:39:08.156134+0000","last_became_peered":"2024-04-25T00:39:08.156134+0000","last_unstale":"2024-04-25T00:39:09.166497+0000","last_undegraded":"2024-04-25T00:39:09.166497+0000","last_fullsized":"2024-04-25T00:39:09.166497+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:34.912860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201077+0000","last_change":"2024-04-25T00:39:09.195925+0000","last_active":"2024-04-25T00:39:09.201077+0000","last_peered":"2024-04-25T00:39:09.201077+0000","last_clean":"2024-04-25T00:39:09.201077+0000","last_became_active":"2024-04-25T00:39:09.191440+0000","last_became_peered":"2024-04-25T00:39:09.191440+0000","last_unstale":"2024-04-25T00:39:09.201077+0000","last_undegraded":"2024-04-25T00:39:09.201077+0000","last_fullsized":"2024-04-25T00:39:09.201077+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:39.278445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.190877+0000","last_change":"2024-04-25T00:39:08.158561+0000","last_active":"2024-04-25T00:39:09.190877+0000","last_peered":"2024-04-25T00:39:09.190877+0000","last_clean":"2024-04-25T00:39:09.190877+0000","last_became_active":"2024-04-25T00:39:08.157751+0000","last_became_peered":"2024-04-25T00:39:08.157751+0000","last_unstale":"2024-04-25T00:39:09.190877+0000","last_undegraded":"2024-04-25T00:39:09.190877+0000","last_fullsized":"2024-04-25T00:39:09.190877+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:10.274560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155711+0000","last_change":"2024-04-25T00:39:09.152699+0000","last_active":"2024-04-25T00:39:09.155711+0000","last_peered":"2024-04-25T00:39:09.155711+0000","last_clean":"2024-04-25T00:39:09.155711+0000","last_became_active":"2024-04-25T00:39:09.151918+0000","last_became_peered":"2024-04-25T00:39:09.151918+0000","last_unstale":"2024-04-25T00:39:09.155711+0000","last_undegraded":"2024-04-25T00:39:09.155711+0000","last_fullsized":"2024-04-25T00:39:09.155711+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:02.412132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.167024+0000","last_change":"2024-04-25T00:39:08.163557+0000","last_active":"2024-04-25T00:39:09.167024+0000","last_peered":"2024-04-25T00:39:09.167024+0000","last_clean":"2024-04-25T00:39:09.167024+0000","last_became_active":"2024-04-25T00:39:08.161998+0000","last_became_peered":"2024-04-25T00:39:08.161998+0000","last_unstale":"2024-04-25T00:39:09.167024+0000","last_undegraded":"2024-04-25T00:39:09.167024+0000","last_fullsized":"2024-04-25T00:39:09.167024+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:24.538161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.191036+0000","last_change":"2024-04-25T00:39:09.186488+0000","last_active":"2024-04-25T00:39:09.191036+0000","last_peered":"2024-04-25T00:39:09.191036+0000","last_clean":"2024-04-25T00:39:09.191036+0000","last_became_active":"2024-04-25T00:39:09.185744+0000","last_became_peered":"2024-04-25T00:39:09.185744+0000","last_unstale":"2024-04-25T00:39:09.191036+0000","last_undegraded":"2024-04-25T00:39:09.191036+0000","last_fullsized":"2024-04-25T00:39:09.191036+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:00.617110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.147940+0000","last_change":"2024-04-25T00:39:08.156570+0000","last_active":"2024-04-25T00:39:09.147940+0000","last_peered":"2024-04-25T00:39:09.147940+0000","last_clean":"2024-04-25T00:39:09.147940+0000","last_became_active":"2024-04-25T00:39:08.156089+0000","last_became_peered":"2024-04-25T00:39:08.156089+0000","last_unstale":"2024-04-25T00:39:09.147940+0000","last_undegraded":"2024-04-25T00:39:09.147940+0000","last_fullsized":"2024-04-25T00:39:09.147940+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:54.901610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.199809+0000","last_change":"2024-04-25T00:39:09.191516+0000","last_active":"2024-04-25T00:39:09.199809+0000","last_peered":"2024-04-25T00:39:09.199809+0000","last_clean":"2024-04-25T00:39:09.199809+0000","last_became_active":"2024-04-25T00:39:09.182230+0000","last_became_peered":"2024-04-25T00:39:09.182230+0000","last_unstale":"2024-04-25T00:39:09.199809+0000","last_undegraded":"2024-04-25T00:39:09.199809+0000","last_fullsized":"2024-04-25T00:39:09.199809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:07.668586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"131'8","reported_seq":20,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.200571+0000","last_change":"2024-04-25T00:39:08.161970+0000","last_active":"2024-04-25T00:39:09.200571+0000","last_peered":"2024-04-25T00:39:09.200571+0000","last_clean":"2024-04-25T00:39:09.200571+0000","last_became_active":"2024-04-25T00:39:08.161645+0000","last_became_peered":"2024-04-25T00:39:08.161645+0000","last_unstale":"2024-04-25T00:39:09.200571+0000","last_undegraded":"2024-04-25T00:39:09.200571+0000","last_fullsized":"2024-04-25T00:39:09.200571+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:49.240064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.202648+0000","last_change":"2024-04-25T00:39:09.195707+0000","last_active":"2024-04-25T00:39:09.202648+0000","last_peered":"2024-04-25T00:39:09.202648+0000","last_clean":"2024-04-25T00:39:09.202648+0000","last_became_active":"2024-04-25T00:39:09.190824+0000","last_became_peered":"2024-04-25T00:39:09.190824+0000","last_unstale":"2024-04-25T00:39:09.202648+0000","last_undegraded":"2024-04-25T00:39:09.202648+0000","last_fullsized":"2024-04-25T00:39:09.202648+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:17.003258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.183510+0000","last_change":"2024-04-25T00:39:08.163083+0000","last_active":"2024-04-25T00:39:09.183510+0000","last_peered":"2024-04-25T00:39:09.183510+0000","last_clean":"2024-04-25T00:39:09.183510+0000","last_became_active":"2024-04-25T00:39:08.162358+0000","last_became_peered":"2024-04-25T00:39:08.162358+0000","last_unstale":"2024-04-25T00:39:09.183510+0000","last_undegraded":"2024-04-25T00:39:09.183510+0000","last_fullsized":"2024-04-25T00:39:09.183510+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:40.532599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.192506+0000","last_change":"2024-04-25T00:39:09.191692+0000","last_active":"2024-04-25T00:39:09.192506+0000","last_peered":"2024-04-25T00:39:09.192506+0000","last_clean":"2024-04-25T00:39:09.192506+0000","last_became_active":"2024-04-25T00:39:09.191460+0000","last_became_peered":"2024-04-25T00:39:09.191460+0000","last_unstale":"2024-04-25T00:39:09.192506+0000","last_undegraded":"2024-04-25T00:39:09.192506+0000","last_fullsized":"2024-04-25T00:39:09.192506+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:11.878064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.192482+0000","last_change":"2024-04-25T00:39:08.159499+0000","last_active":"2024-04-25T00:39:09.192482+0000","last_peered":"2024-04-25T00:39:09.192482+0000","last_clean":"2024-04-25T00:39:09.192482+0000","last_became_active":"2024-04-25T00:39:08.159096+0000","last_became_peered":"2024-04-25T00:39:08.159096+0000","last_unstale":"2024-04-25T00:39:09.192482+0000","last_undegraded":"2024-04-25T00:39:09.192482+0000","last_fullsized":"2024-04-25T00:39:09.192482+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:57.654458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.200517+0000","last_change":"2024-04-25T00:39:09.195491+0000","last_active":"2024-04-25T00:39:09.200517+0000","last_peered":"2024-04-25T00:39:09.200517+0000","last_clean":"2024-04-25T00:39:09.200517+0000","last_became_active":"2024-04-25T00:39:09.182140+0000","last_became_peered":"2024-04-25T00:39:09.182140+0000","last_unstale":"2024-04-25T00:39:09.200517+0000","last_undegraded":"2024-04-25T00:39:09.200517+0000","last_fullsized":"2024-04-25T00:39:09.200517+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:36.089141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.176433+0000","last_change":"2024-04-25T00:39:08.150466+0000","last_active":"2024-04-25T00:39:09.176433+0000","last_peered":"2024-04-25T00:39:09.176433+0000","last_clean":"2024-04-25T00:39:09.176433+0000","last_became_active":"2024-04-25T00:39:08.149773+0000","last_became_peered":"2024-04-25T00:39:08.149773+0000","last_unstale":"2024-04-25T00:39:09.176433+0000","last_undegraded":"2024-04-25T00:39:09.176433+0000","last_fullsized":"2024-04-25T00:39:09.176433+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:04.143824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.193226+0000","last_change":"2024-04-25T00:39:09.192569+0000","last_active":"2024-04-25T00:39:09.193226+0000","last_peered":"2024-04-25T00:39:09.193226+0000","last_clean":"2024-04-25T00:39:09.193226+0000","last_became_active":"2024-04-25T00:39:09.192291+0000","last_became_peered":"2024-04-25T00:39:09.192291+0000","last_unstale":"2024-04-25T00:39:09.193226+0000","last_undegraded":"2024-04-25T00:39:09.193226+0000","last_fullsized":"2024-04-25T00:39:09.193226+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:14.248070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.135943+0000","last_change":"2024-04-25T00:39:08.152756+0000","last_active":"2024-04-25T00:39:09.135943+0000","last_peered":"2024-04-25T00:39:09.135943+0000","last_clean":"2024-04-25T00:39:09.135943+0000","last_became_active":"2024-04-25T00:39:08.152340+0000","last_became_peered":"2024-04-25T00:39:08.152340+0000","last_unstale":"2024-04-25T00:39:09.135943+0000","last_undegraded":"2024-04-25T00:39:09.135943+0000","last_fullsized":"2024-04-25T00:39:09.135943+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:27.744895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.166812+0000","last_change":"2024-04-25T00:39:09.154850+0000","last_active":"2024-04-25T00:39:09.166812+0000","last_peered":"2024-04-25T00:39:09.166812+0000","last_clean":"2024-04-25T00:39:09.166812+0000","last_became_active":"2024-04-25T00:39:09.151604+0000","last_became_peered":"2024-04-25T00:39:09.151604+0000","last_unstale":"2024-04-25T00:39:09.166812+0000","last_undegraded":"2024-04-25T00:39:09.166812+0000","last_fullsized":"2024-04-25T00:39:09.166812+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:46.262398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.183081+0000","last_change":"2024-04-25T00:39:08.159540+0000","last_active":"2024-04-25T00:39:09.183081+0000","last_peered":"2024-04-25T00:39:09.183081+0000","last_clean":"2024-04-25T00:39:09.183081+0000","last_became_active":"2024-04-25T00:39:08.159106+0000","last_became_peered":"2024-04-25T00:39:08.159106+0000","last_unstale":"2024-04-25T00:39:09.183081+0000","last_undegraded":"2024-04-25T00:39:09.183081+0000","last_fullsized":"2024-04-25T00:39:09.183081+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:21.692876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.159354+0000","last_change":"2024-04-25T00:39:09.158784+0000","last_active":"2024-04-25T00:39:09.159354+0000","last_peered":"2024-04-25T00:39:09.159354+0000","last_clean":"2024-04-25T00:39:09.159354+0000","last_became_active":"2024-04-25T00:39:09.158506+0000","last_became_peered":"2024-04-25T00:39:09.158506+0000","last_unstale":"2024-04-25T00:39:09.159354+0000","last_undegraded":"2024-04-25T00:39:09.159354+0000","last_fullsized":"2024-04-25T00:39:09.159354+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:46.496845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.164451+0000","last_change":"2024-04-25T00:39:08.153576+0000","last_active":"2024-04-25T00:39:09.164451+0000","last_peered":"2024-04-25T00:39:09.164451+0000","last_clean":"2024-04-25T00:39:09.164451+0000","last_became_active":"2024-04-25T00:39:08.152083+0000","last_became_peered":"2024-04-25T00:39:08.152083+0000","last_unstale":"2024-04-25T00:39:09.164451+0000","last_undegraded":"2024-04-25T00:39:09.164451+0000","last_fullsized":"2024-04-25T00:39:09.164451+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:01.408284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.165611+0000","last_change":"2024-04-25T00:39:09.153967+0000","last_active":"2024-04-25T00:39:09.165611+0000","last_peered":"2024-04-25T00:39:09.165611+0000","last_clean":"2024-04-25T00:39:09.165611+0000","last_became_active":"2024-04-25T00:39:09.151729+0000","last_became_peered":"2024-04-25T00:39:09.151729+0000","last_unstale":"2024-04-25T00:39:09.165611+0000","last_undegraded":"2024-04-25T00:39:09.165611+0000","last_fullsized":"2024-04-25T00:39:09.165611+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:35.538953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.172419+0000","last_change":"2024-04-25T00:39:08.157957+0000","last_active":"2024-04-25T00:39:09.172419+0000","last_peered":"2024-04-25T00:39:09.172419+0000","last_clean":"2024-04-25T00:39:09.172419+0000","last_became_active":"2024-04-25T00:39:08.155291+0000","last_became_peered":"2024-04-25T00:39:08.155291+0000","last_unstale":"2024-04-25T00:39:09.172419+0000","last_undegraded":"2024-04-25T00:39:09.172419+0000","last_fullsized":"2024-04-25T00:39:09.172419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:35.171376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.168139+0000","last_change":"2024-04-25T00:39:09.167450+0000","last_active":"2024-04-25T00:39:09.168139+0000","last_peered":"2024-04-25T00:39:09.168139+0000","last_clean":"2024-04-25T00:39:09.168139+0000","last_became_active":"2024-04-25T00:39:09.167195+0000","last_became_peered":"2024-04-25T00:39:09.167195+0000","last_unstale":"2024-04-25T00:39:09.168139+0000","last_undegraded":"2024-04-25T00:39:09.168139+0000","last_fullsized":"2024-04-25T00:39:09.168139+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:17.003258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.167387+0000","last_change":"2024-04-25T00:39:08.159847+0000","last_active":"2024-04-25T00:39:09.167387+0000","last_peered":"2024-04-25T00:39:09.167387+0000","last_clean":"2024-04-25T00:39:09.167387+0000","last_became_active":"2024-04-25T00:39:08.159154+0000","last_became_peered":"2024-04-25T00:39:08.159154+0000","last_unstale":"2024-04-25T00:39:09.167387+0000","last_undegraded":"2024-04-25T00:39:09.167387+0000","last_fullsized":"2024-04-25T00:39:09.167387+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:23.747323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.192155+0000","last_change":"2024-04-25T00:39:09.189719+0000","last_active":"2024-04-25T00:39:09.192155+0000","last_peered":"2024-04-25T00:39:09.192155+0000","last_clean":"2024-04-25T00:39:09.192155+0000","last_became_active":"2024-04-25T00:39:09.188634+0000","last_became_peered":"2024-04-25T00:39:09.188634+0000","last_unstale":"2024-04-25T00:39:09.192155+0000","last_undegraded":"2024-04-25T00:39:09.192155+0000","last_fullsized":"2024-04-25T00:39:09.192155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:20.125268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.200700+0000","last_change":"2024-04-25T00:39:08.164583+0000","last_active":"2024-04-25T00:39:09.200700+0000","last_peered":"2024-04-25T00:39:09.200700+0000","last_clean":"2024-04-25T00:39:09.200700+0000","last_became_active":"2024-04-25T00:39:08.162521+0000","last_became_peered":"2024-04-25T00:39:08.162521+0000","last_unstale":"2024-04-25T00:39:09.200700+0000","last_undegraded":"2024-04-25T00:39:09.200700+0000","last_fullsized":"2024-04-25T00:39:09.200700+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:38.966184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.199927+0000","last_change":"2024-04-25T00:39:09.191986+0000","last_active":"2024-04-25T00:39:09.199927+0000","last_peered":"2024-04-25T00:39:09.199927+0000","last_clean":"2024-04-25T00:39:09.199927+0000","last_became_active":"2024-04-25T00:39:09.190002+0000","last_became_peered":"2024-04-25T00:39:09.190002+0000","last_unstale":"2024-04-25T00:39:09.199927+0000","last_undegraded":"2024-04-25T00:39:09.199927+0000","last_fullsized":"2024-04-25T00:39:09.199927+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:55.237342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.149159+0000","last_change":"2024-04-25T00:39:08.154517+0000","last_active":"2024-04-25T00:39:09.149159+0000","last_peered":"2024-04-25T00:39:09.149159+0000","last_clean":"2024-04-25T00:39:09.149159+0000","last_became_active":"2024-04-25T00:39:08.154015+0000","last_became_peered":"2024-04-25T00:39:08.154015+0000","last_unstale":"2024-04-25T00:39:09.149159+0000","last_undegraded":"2024-04-25T00:39:09.149159+0000","last_fullsized":"2024-04-25T00:39:09.149159+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:00.260014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155702+0000","last_change":"2024-04-25T00:39:09.147903+0000","last_active":"2024-04-25T00:39:09.155702+0000","last_peered":"2024-04-25T00:39:09.155702+0000","last_clean":"2024-04-25T00:39:09.155702+0000","last_became_active":"2024-04-25T00:39:09.147583+0000","last_became_peered":"2024-04-25T00:39:09.147583+0000","last_unstale":"2024-04-25T00:39:09.155702+0000","last_undegraded":"2024-04-25T00:39:09.155702+0000","last_fullsized":"2024-04-25T00:39:09.155702+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:08.652182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.149385+0000","last_change":"2024-04-25T00:39:08.151933+0000","last_active":"2024-04-25T00:39:09.149385+0000","last_peered":"2024-04-25T00:39:09.149385+0000","last_clean":"2024-04-25T00:39:09.149385+0000","last_became_active":"2024-04-25T00:39:08.151452+0000","last_became_peered":"2024-04-25T00:39:08.151452+0000","last_unstale":"2024-04-25T00:39:09.149385+0000","last_undegraded":"2024-04-25T00:39:09.149385+0000","last_fullsized":"2024-04-25T00:39:09.149385+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:14.365515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.160704+0000","last_change":"2024-04-25T00:39:09.159585+0000","last_active":"2024-04-25T00:39:09.160704+0000","last_peered":"2024-04-25T00:39:09.160704+0000","last_clean":"2024-04-25T00:39:09.160704+0000","last_became_active":"2024-04-25T00:39:09.159303+0000","last_became_peered":"2024-04-25T00:39:09.159303+0000","last_unstale":"2024-04-25T00:39:09.160704+0000","last_undegraded":"2024-04-25T00:39:09.160704+0000","last_fullsized":"2024-04-25T00:39:09.160704+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:37.058016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.148946+0000","last_change":"2024-04-25T00:39:08.155004+0000","last_active":"2024-04-25T00:39:09.148946+0000","last_peered":"2024-04-25T00:39:09.148946+0000","last_clean":"2024-04-25T00:39:09.148946+0000","last_became_active":"2024-04-25T00:39:08.153998+0000","last_became_peered":"2024-04-25T00:39:08.153998+0000","last_unstale":"2024-04-25T00:39:09.148946+0000","last_undegraded":"2024-04-25T00:39:09.148946+0000","last_fullsized":"2024-04-25T00:39:09.148946+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:26.680458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.155942+0000","last_change":"2024-04-25T00:39:09.153893+0000","last_active":"2024-04-25T00:39:09.155942+0000","last_peered":"2024-04-25T00:39:09.155942+0000","last_clean":"2024-04-25T00:39:09.155942+0000","last_became_active":"2024-04-25T00:39:09.152198+0000","last_became_peered":"2024-04-25T00:39:09.152198+0000","last_unstale":"2024-04-25T00:39:09.155942+0000","last_undegraded":"2024-04-25T00:39:09.155942+0000","last_fullsized":"2024-04-25T00:39:09.155942+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:36.537887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.143477+0000","last_change":"2024-04-25T00:39:08.172324+0000","last_active":"2024-04-25T00:39:09.143477+0000","last_peered":"2024-04-25T00:39:09.143477+0000","last_clean":"2024-04-25T00:39:09.143477+0000","last_became_active":"2024-04-25T00:39:08.171892+0000","last_became_peered":"2024-04-25T00:39:08.171892+0000","last_unstale":"2024-04-25T00:39:09.143477+0000","last_undegraded":"2024-04-25T00:39:09.143477+0000","last_fullsized":"2024-04-25T00:39:09.143477+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:02.774690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201098+0000","last_change":"2024-04-25T00:39:09.197747+0000","last_active":"2024-04-25T00:39:09.201098+0000","last_peered":"2024-04-25T00:39:09.201098+0000","last_clean":"2024-04-25T00:39:09.201098+0000","last_became_active":"2024-04-25T00:39:09.197029+0000","last_became_peered":"2024-04-25T00:39:09.197029+0000","last_unstale":"2024-04-25T00:39:09.201098+0000","last_undegraded":"2024-04-25T00:39:09.201098+0000","last_fullsized":"2024-04-25T00:39:09.201098+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:15.379539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.164877+0000","last_change":"2024-04-25T00:39:08.157128+0000","last_active":"2024-04-25T00:39:09.164877+0000","last_peered":"2024-04-25T00:39:09.164877+0000","last_clean":"2024-04-25T00:39:09.164877+0000","last_became_active":"2024-04-25T00:39:08.156406+0000","last_became_peered":"2024-04-25T00:39:08.156406+0000","last_unstale":"2024-04-25T00:39:09.164877+0000","last_undegraded":"2024-04-25T00:39:09.164877+0000","last_fullsized":"2024-04-25T00:39:09.164877+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:21.291869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.201483+0000","last_change":"2024-04-25T00:39:09.186709+0000","last_active":"2024-04-25T00:39:09.201483+0000","last_peered":"2024-04-25T00:39:09.201483+0000","last_clean":"2024-04-25T00:39:09.201483+0000","last_became_active":"2024-04-25T00:39:09.185946+0000","last_became_peered":"2024-04-25T00:39:09.185946+0000","last_unstale":"2024-04-25T00:39:09.201483+0000","last_undegraded":"2024-04-25T00:39:09.201483+0000","last_fullsized":"2024-04-25T00:39:09.201483+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:03.778537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.158767+0000","last_change":"2024-04-25T00:39:08.158225+0000","last_active":"2024-04-25T00:39:09.158767+0000","last_peered":"2024-04-25T00:39:09.158767+0000","last_clean":"2024-04-25T00:39:09.158767+0000","last_became_active":"2024-04-25T00:39:08.157364+0000","last_became_peered":"2024-04-25T00:39:08.157364+0000","last_unstale":"2024-04-25T00:39:09.158767+0000","last_undegraded":"2024-04-25T00:39:09.158767+0000","last_fullsized":"2024-04-25T00:39:09.158767+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:47.893103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.168696+0000","last_change":"2024-04-25T00:39:09.161141+0000","last_active":"2024-04-25T00:39:09.168696+0000","last_peered":"2024-04-25T00:39:09.168696+0000","last_clean":"2024-04-25T00:39:09.168696+0000","last_became_active":"2024-04-25T00:39:09.160362+0000","last_became_peered":"2024-04-25T00:39:09.160362+0000","last_unstale":"2024-04-25T00:39:09.168696+0000","last_undegraded":"2024-04-25T00:39:09.168696+0000","last_fullsized":"2024-04-25T00:39:09.168696+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:55.024666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.135401+0000","last_change":"2024-04-25T00:39:08.154932+0000","last_active":"2024-04-25T00:39:09.135401+0000","last_peered":"2024-04-25T00:39:09.135401+0000","last_clean":"2024-04-25T00:39:09.135401+0000","last_became_active":"2024-04-25T00:39:08.154381+0000","last_became_peered":"2024-04-25T00:39:08.154381+0000","last_unstale":"2024-04-25T00:39:09.135401+0000","last_undegraded":"2024-04-25T00:39:09.135401+0000","last_fullsized":"2024-04-25T00:39:09.135401+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:16.874821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.160271+0000","last_change":"2024-04-25T00:39:09.159838+0000","last_active":"2024-04-25T00:39:09.160271+0000","last_peered":"2024-04-25T00:39:09.160271+0000","last_clean":"2024-04-25T00:39:09.160271+0000","last_became_active":"2024-04-25T00:39:09.159144+0000","last_became_peered":"2024-04-25T00:39:09.159144+0000","last_unstale":"2024-04-25T00:39:09.160271+0000","last_undegraded":"2024-04-25T00:39:09.160271+0000","last_fullsized":"2024-04-25T00:39:09.160271+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:08.117333+0000","last_clean_scrub_stamp":"2024-04-25T00:39:08.117333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:10.600534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"131'3","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.193804+0000","last_change":"2024-04-25T00:39:08.164154+0000","last_active":"2024-04-25T00:39:09.193804+0000","last_peered":"2024-04-25T00:39:09.193804+0000","last_clean":"2024-04-25T00:39:09.193804+0000","last_became_active":"2024-04-25T00:39:08.160220+0000","last_became_peered":"2024-04-25T00:39:08.160220+0000","last_unstale":"2024-04-25T00:39:09.193804+0000","last_undegraded":"2024-04-25T00:39:09.193804+0000","last_fullsized":"2024-04-25T00:39:09.193804+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:07.113486+0000","last_clean_scrub_stamp":"2024-04-25T00:39:07.113486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:13.244222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":421,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T00:39:09.166442+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:39:09.166442+0000","last_peered":"2024-04-25T00:39:09.166442+0000","last_clean":"2024-04-25T00:39:09.166442+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:39:09.166442+0000","last_undegraded":"2024-04-25T00:39:09.166442+0000","last_fullsized":"2024-04-25T00:39:09.166442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37516,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334324,"statfs":{"total":96636764160,"available":96598347776,"internally_reserved":0,"allocated":2224128,"data_stored":1308152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36936,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334904,"statfs":{"total":96636764160,"available":96598941696,"internally_reserved":0,"allocated":1630208,"data_stored":717784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36932,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334908,"statfs":{"total":96636764160,"available":96598945792,"internally_reserved":0,"allocated":1630208,"data_stored":717784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36936,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334904,"statfs":{"total":96636764160,"available":96598941696,"internally_reserved":0,"allocated":1630208,"data_stored":717784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36932,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334908,"statfs":{"total":96636764160,"available":96598945792,"internally_reserved":0,"allocated":1630208,"data_stored":717784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36932,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334908,"statfs":{"total":96636764160,"available":96598945792,"internally_reserved":0,"allocated":1630208,"data_stored":717784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36932,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334908,"statfs":{"total":96636764160,"available":96598945792,"internally_reserved":0,"allocated":1630208,"data_stored":717784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41676,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330164,"statfs":{"total":96636764160,"available":96594087936,"internally_reserved":0,"allocated":2224128,"data_stored":1308152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:39:13.561 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:39:13.561 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:39:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a4210640 1 Processor -- start 2024-04-25T00:39:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a4210640 1 -- start start 2024-04-25T00:39:13.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a4210640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f459c15df70 0x7f459c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a4210640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f459c06d210 0x7f459c0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a4210640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f459c0b4b60 0x7f459c070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a4210640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f459c0b4fa0 con 0x7f459c0b4b60 2024-04-25T00:39:13.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a4210640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f459c06d980 con 0x7f459c06d210 2024-04-25T00:39:13.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a4210640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f459c06dc60 con 0x7f459c15df70 2024-04-25T00:39:13.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a27ad640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f459c0b4b60 0x7f459c070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:13.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a27ad640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f459c0b4b60 0x7f459c070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56960/0 (socket says 172.21.15.29:56960) 2024-04-25T00:39:13.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a27ad640 1 -- 172.21.15.29:0/1006762482 learned_addr learned my addr 172.21.15.29:0/1006762482 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:13.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a27ad640 1 -- 172.21.15.29:0/1006762482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f459c15df70 msgr2=0x7f459c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:13.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a27ad640 1 --2- 172.21.15.29:0/1006762482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f459c15df70 0x7f459c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a27ad640 1 -- 172.21.15.29:0/1006762482 >> v2:172.21.15.157:3300/0 conn(0x7f459c06d210 msgr2=0x7f459c0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:13.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a27ad640 1 --2- 172.21.15.29:0/1006762482 >> v2:172.21.15.157:3300/0 conn(0x7f459c06d210 0x7f459c0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.726+0000 7f45a27ad640 1 -- 172.21.15.29:0/1006762482 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c06df40 con 0x7f459c0b4b60 2024-04-25T00:39:13.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a27ad640 1 --2- 172.21.15.29:0/1006762482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f459c0b4b60 0x7f459c070ce0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f459c074fb0 tx=0x7f459401c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da06bb6877fe312 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:13.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a0faa640 1 -- 172.21.15.29:0/1006762482 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45940a5580 con 0x7f459c0b4b60 2024-04-25T00:39:13.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a0faa640 1 -- 172.21.15.29:0/1006762482 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45940a5720 con 0x7f459c0b4b60 2024-04-25T00:39:13.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a0faa640 1 -- 172.21.15.29:0/1006762482 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45940a5af0 con 0x7f459c0b4b60 2024-04-25T00:39:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a4210640 1 -- 172.21.15.29:0/1006762482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f459c0b4b60 msgr2=0x7f459c070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a4210640 1 --2- 172.21.15.29:0/1006762482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f459c0b4b60 0x7f459c070ce0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f459c074fb0 tx=0x7f459401c700 comp rx=0 tx=0).stop 2024-04-25T00:39:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a4210640 1 -- 172.21.15.29:0/1006762482 shutdown_connections 2024-04-25T00:39:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a4210640 1 --2- 172.21.15.29:0/1006762482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f459c0b4b60 0x7f459c070ce0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a4210640 1 --2- 172.21.15.29:0/1006762482 >> v2:172.21.15.157:3300/0 conn(0x7f459c06d210 0x7f459c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a4210640 1 --2- 172.21.15.29:0/1006762482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f459c15df70 0x7f459c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a4210640 1 -- 172.21.15.29:0/1006762482 >> 172.21.15.29:0/1006762482 conn(0x7f459c0b1870 msgr2=0x7f459c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a4210640 1 -- 172.21.15.29:0/1006762482 shutdown_connections 2024-04-25T00:39:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a4210640 1 -- 172.21.15.29:0/1006762482 wait complete. 2024-04-25T00:39:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a4210640 1 Processor -- start 2024-04-25T00:39:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.730+0000 7f45a4210640 1 -- start start 2024-04-25T00:39:13.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a4210640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f459c06d210 0x7f459c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a4210640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f459c0b4b60 0x7f459c1486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a4210640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f459c15df70 0x7f459c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a4210640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f459c06ea20 con 0x7f459c0b4b60 2024-04-25T00:39:13.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a4210640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f459c06d650 con 0x7f459c06d210 2024-04-25T00:39:13.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a4210640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f459c06eba0 con 0x7f459c15df70 2024-04-25T00:39:13.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a17ab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f459c0b4b60 0x7f459c1486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:13.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a17ab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f459c0b4b60 0x7f459c1486d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56964/0 (socket says 172.21.15.29:56964) 2024-04-25T00:39:13.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a17ab640 1 -- 172.21.15.29:0/159495117 learned_addr learned my addr 172.21.15.29:0/159495117 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:13.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a27ad640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f459c15df70 0x7f459c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:13.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a1fac640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f459c06d210 0x7f459c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:13.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a17ab640 1 -- 172.21.15.29:0/159495117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f459c15df70 msgr2=0x7f459c15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a17ab640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f459c15df70 0x7f459c15cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a17ab640 1 -- 172.21.15.29:0/159495117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f459c06d210 msgr2=0x7f459c15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a17ab640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f459c06d210 0x7f459c15b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a17ab640 1 -- 172.21.15.29:0/159495117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c148c10 con 0x7f459c0b4b60 2024-04-25T00:39:13.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a17ab640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f459c0b4b60 0x7f459c1486d0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f458c00b550 tx=0x7f458c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:13.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45927fc640 1 -- 172.21.15.29:0/159495117 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c014070 con 0x7f459c0b4b60 2024-04-25T00:39:13.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45927fc640 1 -- 172.21.15.29:0/159495117 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458c010890 con 0x7f459c0b4b60 2024-04-25T00:39:13.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a4210640 1 -- 172.21.15.29:0/159495117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459c157490 con 0x7f459c0b4b60 2024-04-25T00:39:13.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45927fc640 1 -- 172.21.15.29:0/159495117 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c010c80 con 0x7f459c0b4b60 2024-04-25T00:39:13.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a4210640 1 -- 172.21.15.29:0/159495117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f459c157730 con 0x7f459c0b4b60 2024-04-25T00:39:13.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a4210640 1 -- 172.21.15.29:0/159495117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f459c157930 con 0x7f459c0b4b60 2024-04-25T00:39:13.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45927fc640 1 -- 172.21.15.29:0/159495117 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f458c017070 con 0x7f459c0b4b60 2024-04-25T00:39:13.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45927fc640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4580067e60 0x7f458006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:13.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45927fc640 1 -- 172.21.15.29:0/159495117 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f458c086440 con 0x7f459c0b4b60 2024-04-25T00:39:13.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a1fac640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4580067e60 0x7f458006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:13.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.734+0000 7f45a1fac640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4580067e60 0x7f458006a320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f459c156de0 tx=0x7f4598008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:13.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.738+0000 7f45927fc640 1 -- 172.21.15.29:0/159495117 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f459c157930 con 0x7f459c0b4b60 2024-04-25T00:39:13.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.958+0000 7f45a4210640 1 -- 172.21.15.29:0/159495117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f459c069270 con 0x7f459c0b4b60 2024-04-25T00:39:13.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.966+0000 7f45927fc640 1 -- 172.21.15.29:0/159495117 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f458c010e20 con 0x7f459c0b4b60 2024-04-25T00:39:13.968 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-04-25T00:39:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A== 2024-04-25T00:39:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: caps mds = "allow" 2024-04-25T00:39:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: caps mon = "allow r" 2024-04-25T00:39:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:39:13.968 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-04-25T00:39:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.966+0000 7f45a4210640 1 -- 172.21.15.29:0/159495117 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4580067e60 msgr2=0x7f458006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.966+0000 7f45a4210640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4580067e60 0x7f458006a320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f459c156de0 tx=0x7f4598008040 comp rx=0 tx=0).stop 2024-04-25T00:39:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.966+0000 7f45a4210640 1 -- 172.21.15.29:0/159495117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f459c0b4b60 msgr2=0x7f459c1486d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.966+0000 7f45a4210640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f459c0b4b60 0x7f459c1486d0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f458c00b550 tx=0x7f458c00ba10 comp rx=0 tx=0).stop 2024-04-25T00:39:13.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.966+0000 7f45a4210640 1 -- 172.21.15.29:0/159495117 shutdown_connections 2024-04-25T00:39:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.966+0000 7f45a4210640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4580067e60 0x7f458006a320 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.966+0000 7f45a4210640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f459c15df70 0x7f459c15cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.966+0000 7f45a4210640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f459c0b4b60 0x7f459c1486d0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.966+0000 7f45a4210640 1 --2- 172.21.15.29:0/159495117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f459c06d210 0x7f459c15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.966+0000 7f45a4210640 1 -- 172.21.15.29:0/159495117 >> 172.21.15.29:0/159495117 conn(0x7f459c0b1870 msgr2=0x7f459c155660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.966+0000 7f45a4210640 1 -- 172.21.15.29:0/159495117 shutdown_connections 2024-04-25T00:39:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:13.966+0000 7f45a4210640 1 -- 172.21.15.29:0/159495117 wait complete. 2024-04-25T00:39:13.988 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:39:14.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.106+0000 7f2749e51640 1 Processor -- start 2024-04-25T00:39:14.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.106+0000 7f2749e51640 1 -- start start 2024-04-25T00:39:14.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2749e51640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27440b3c30 0x7f274406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2749e51640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27440b4b10 0x7f27440ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2749e51640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27440b41e0 con 0x7f27440b3140 2024-04-25T00:39:14.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2749e51640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f274406d960 con 0x7f27440b4b10 2024-04-25T00:39:14.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2749e51640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f274406dc40 con 0x7f27440b3c30 2024-04-25T00:39:14.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27437fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f27440b3140 0x7f27440b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39090/0 (socket says 172.21.15.29:39090) 2024-04-25T00:39:14.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27437fe640 1 -- 172.21.15.29:0/584997007 learned_addr learned my addr 172.21.15.29:0/584997007 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:14.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2742ffd640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27440b3c30 0x7f274406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2743fff640 1 --2- 172.21.15.29:0/584997007 >> v2:172.21.15.157:3300/0 conn(0x7f27440b4b10 0x7f27440ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 190715256 0 0) 0x7f27440b41e0 con 0x7f27440b3140 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f272c003530 con 0x7f27440b3140 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1430912921 0 0) 0x7f272c003530 con 0x7f27440b3140 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27440b3c30 msgr2=0x7f274406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27440b3c30 0x7f274406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 >> v2:172.21.15.157:3300/0 conn(0x7f27440b4b10 msgr2=0x7f27440ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 --2- 172.21.15.29:0/584997007 >> v2:172.21.15.157:3300/0 conn(0x7f27440b4b10 0x7f27440ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274406df20 con 0x7f27440b3140 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2734001680 con 0x7f27440b3140 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 >> v1:172.21.15.29:6789/0 conn(0x7f27440b3140 legacy=0x7f27440b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f272c0039c0 0x7f272c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f272c004810 0x7f272c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27440b3c30 0x7f272c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f274406d960 con 0x7f272c004810 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f274406dc40 con 0x7f272c0039c0 2024-04-25T00:39:14.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f272c00b9d0 con 0x7f27440b3c30 2024-04-25T00:39:14.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2742ffd640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f272c004810 0x7f272c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27437fe640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f272c0039c0 0x7f272c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2743fff640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27440b3c30 0x7f272c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2742ffd640 1 -- 172.21.15.29:0/584997007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27440b3c30 msgr2=0x7f272c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2742ffd640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27440b3c30 0x7f272c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2742ffd640 1 -- 172.21.15.29:0/584997007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f272c0039c0 msgr2=0x7f272c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2742ffd640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f272c0039c0 0x7f272c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2742ffd640 1 -- 172.21.15.29:0/584997007 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c003530 con 0x7f272c004810 2024-04-25T00:39:14.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2743fff640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27440b3c30 0x7f272c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:14.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f27437fe640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f272c0039c0 0x7f272c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:14.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.110+0000 7f2742ffd640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f272c004810 0x7f272c004c90 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f2738009ac0 tx=0x7f273801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfa3efbce9384ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:14.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27380b2070 con 0x7f272c004810 2024-04-25T00:39:14.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27380a5560 con 0x7f272c004810 2024-04-25T00:39:14.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f27427fc640 1 -- 172.21.15.29:0/584997007 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738009690 con 0x7f272c004810 2024-04-25T00:39:14.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 -- 172.21.15.29:0/584997007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f272c004810 msgr2=0x7f272c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 --2- 172.21.15.29:0/584997007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f272c004810 0x7f272c004c90 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f2738009ac0 tx=0x7f273801ca50 comp rx=0 tx=0).stop 2024-04-25T00:39:14.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2743fff640 1 -- 172.21.15.29:0/584997007 reap_dead start 2024-04-25T00:39:14.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 -- 172.21.15.29:0/584997007 shutdown_connections 2024-04-25T00:39:14.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 -- 172.21.15.29:0/584997007 >> 172.21.15.29:0/584997007 conn(0x7f27440b1870 msgr2=0x7f27440b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 -- 172.21.15.29:0/584997007 shutdown_connections 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 -- 172.21.15.29:0/584997007 wait complete. 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 Processor -- start 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 -- start start 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27441f0e40 0x7f274412bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2744129410 0x7f2744129890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f274413a2c0 0x7f274413a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2744145a90 con 0x7f27441f0e40 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2744145910 con 0x7f274413a2c0 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f274406e440 con 0x7f2744129410 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f27437fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27441f0e40 0x7f274412bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f27437fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27441f0e40 0x7f274412bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56986/0 (socket says 172.21.15.29:56986) 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f27437fe640 1 -- 172.21.15.29:0/288827790 learned_addr learned my addr 172.21.15.29:0/288827790 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2742ffd640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2744129410 0x7f2744129890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2743fff640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f274413a2c0 0x7f274413a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f27437fe640 1 -- 172.21.15.29:0/288827790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2744129410 msgr2=0x7f2744129890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f27437fe640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2744129410 0x7f2744129890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f27437fe640 1 -- 172.21.15.29:0/288827790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f274413a2c0 msgr2=0x7f274413a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f27437fe640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f274413a2c0 0x7f274413a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f27437fe640 1 -- 172.21.15.29:0/288827790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274412cf00 con 0x7f27441f0e40 2024-04-25T00:39:14.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2742ffd640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2744129410 0x7f2744129890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:14.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f27437fe640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27441f0e40 0x7f274412bfd0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f2734010a90 tx=0x7f2734010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:14.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f272bfff640 1 -- 172.21.15.29:0/288827790 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273401b070 con 0x7f27441f0e40 2024-04-25T00:39:14.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f272bfff640 1 -- 172.21.15.29:0/288827790 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2734011040 con 0x7f27441f0e40 2024-04-25T00:39:14.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f272bfff640 1 -- 172.21.15.29:0/288827790 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27340143a0 con 0x7f27441f0e40 2024-04-25T00:39:14.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 -- 172.21.15.29:0/288827790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274412c940 con 0x7f27441f0e40 2024-04-25T00:39:14.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.114+0000 7f2749e51640 1 -- 172.21.15.29:0/288827790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2744128c10 con 0x7f27441f0e40 2024-04-25T00:39:14.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.118+0000 7f2749e51640 1 -- 172.21.15.29:0/288827790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2744069270 con 0x7f27441f0e40 2024-04-25T00:39:14.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.118+0000 7f272bfff640 1 -- 172.21.15.29:0/288827790 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f2734014540 con 0x7f27441f0e40 2024-04-25T00:39:14.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.118+0000 7f272bfff640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2730067e60 0x7f273006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.118+0000 7f272bfff640 1 -- 172.21.15.29:0/288827790 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f27340891e0 con 0x7f27441f0e40 2024-04-25T00:39:14.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.118+0000 7f2742ffd640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2730067e60 0x7f273006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.118+0000 7f2742ffd640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2730067e60 0x7f273006a320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f27380af000 tx=0x7f27380046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:14.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.122+0000 7f272bfff640 1 -- 172.21.15.29:0/288827790 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f27340594b0 con 0x7f27441f0e40 2024-04-25T00:39:14.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.314+0000 7f2749e51640 1 -- 172.21.15.29:0/288827790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27440753e0 con 0x7f27441f0e40 2024-04-25T00:39:14.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.318+0000 7f272bfff640 1 -- 172.21.15.29:0/288827790 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5179 (secure 0 0 0) 0x7f2734058db0 con 0x7f27441f0e40 2024-04-25T00:39:14.319 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:14.320 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":83,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7450,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/456996219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":456996219},{"type":"v1","addr":"172.21.15.157:6833","nonce":456996219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/87071339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":87071339},{"type":"v1","addr":"172.21.15.157:6835","nonce":87071339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3920208567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3920208567},{"type":"v1","addr":"172.21.15.29:6833","nonce":3920208567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:39:08.508825+0000","modified":"2024-04-25T00:39:11.535481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9021":{"gid":9021,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/619209926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":619209926},{"type":"v1","addr":"172.21.15.29:6835","nonce":619209926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9021,"qdb_cluster":[9021]},"id":10}]} 2024-04-25T00:39:14.320 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 83 2024-04-25T00:39:14.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.318+0000 7f2749e51640 1 -- 172.21.15.29:0/288827790 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2730067e60 msgr2=0x7f273006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.318+0000 7f2749e51640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2730067e60 0x7f273006a320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f27380af000 tx=0x7f27380046d0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.318+0000 7f2749e51640 1 -- 172.21.15.29:0/288827790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27441f0e40 msgr2=0x7f274412bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.318+0000 7f2749e51640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27441f0e40 0x7f274412bfd0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f2734010a90 tx=0x7f2734010f50 comp rx=0 tx=0).stop 2024-04-25T00:39:14.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.318+0000 7f2749e51640 1 -- 172.21.15.29:0/288827790 shutdown_connections 2024-04-25T00:39:14.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.318+0000 7f2749e51640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2730067e60 0x7f273006a320 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.318+0000 7f2749e51640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f274413a2c0 0x7f274413a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.318+0000 7f2749e51640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2744129410 0x7f2744129890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.318+0000 7f2749e51640 1 --2- 172.21.15.29:0/288827790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27441f0e40 0x7f274412bfd0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.318+0000 7f2749e51640 1 -- 172.21.15.29:0/288827790 >> 172.21.15.29:0/288827790 conn(0x7f27440b1870 msgr2=0x7f274415c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:14.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.318+0000 7f2749e51640 1 -- 172.21.15.29:0/288827790 shutdown_connections 2024-04-25T00:39:14.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.318+0000 7f2749e51640 1 -- 172.21.15.29:0/288827790 wait complete. 2024-04-25T00:39:14.741 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:39:08.508825+0000', 'modified': '2024-04-25T00:39:11.535481+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9021}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9021': {'gid': 9021, 'name': 'c', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/619209926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 619209926}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 619209926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9021, 'qdb_cluster': [9021]} 2024-04-25T00:39:14.742 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:39:14.742 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:39:14.762 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-04-25T00:39:14.758+0000 7fde9074a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:39:14.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d975a640 1 Processor -- start 2024-04-25T00:39:14.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d975a640 1 -- start start 2024-04-25T00:39:14.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d975a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd3d40b2e80 0x7fd3d40b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d975a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd3d406dd50 0x7fd3d406e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d975a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3d406cf90 con 0x7fd3d413d140 2024-04-25T00:39:14.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d975a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd3d406cd40 con 0x7fd3d406dd50 2024-04-25T00:39:14.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d975a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd3d406d270 con 0x7fd3d40b2e80 2024-04-25T00:39:14.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d37fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd3d406dd50 0x7fd3d406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d37fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd3d406dd50 0x7fd3d406e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56988/0 (socket says 172.21.15.29:56988) 2024-04-25T00:39:14.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d37fe640 1 -- 172.21.15.29:0/3718997176 learned_addr learned my addr 172.21.15.29:0/3718997176 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:14.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/3718997176 >> v2:172.21.15.157:3300/0 conn(0x7fd3d40b2e80 0x7fd3d40b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d37fe640 1 -- 172.21.15.29:0/3718997176 >> v2:172.21.15.157:3300/0 conn(0x7fd3d40b2e80 msgr2=0x7fd3d40b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d37fe640 1 --2- 172.21.15.29:0/3718997176 >> v2:172.21.15.157:3300/0 conn(0x7fd3d40b2e80 0x7fd3d40b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d37fe640 1 -- 172.21.15.29:0/3718997176 >> v1:172.21.15.157:6789/0 conn(0x7fd3d413d140 legacy=0x7fd3d413d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.878+0000 7fd3d37fe640 1 -- 172.21.15.29:0/3718997176 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d406d550 con 0x7fd3d406dd50 2024-04-25T00:39:14.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d37fe640 1 --2- 172.21.15.29:0/3718997176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d406dd50 0x7fd3d406e130 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40b5210 tx=0x7fd3cc01ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c5bf9a5e36298a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d1ffb640 1 -- 172.21.15.29:0/3718997176 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc00b2d0 con 0x7fd3d406dd50 2024-04-25T00:39:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d1ffb640 1 -- 172.21.15.29:0/3718997176 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3cc00b470 con 0x7fd3d406dd50 2024-04-25T00:39:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d1ffb640 1 -- 172.21.15.29:0/3718997176 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc0a7990 con 0x7fd3d406dd50 2024-04-25T00:39:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d975a640 1 -- 172.21.15.29:0/3718997176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d406dd50 msgr2=0x7fd3d406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d975a640 1 --2- 172.21.15.29:0/3718997176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d406dd50 0x7fd3d406e130 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40b5210 tx=0x7fd3cc01ebb0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d975a640 1 -- 172.21.15.29:0/3718997176 shutdown_connections 2024-04-25T00:39:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d975a640 1 --2- 172.21.15.29:0/3718997176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d406dd50 0x7fd3d406e130 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d975a640 1 --2- 172.21.15.29:0/3718997176 >> v2:172.21.15.157:3300/0 conn(0x7fd3d40b2e80 0x7fd3d40b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d975a640 1 -- 172.21.15.29:0/3718997176 >> 172.21.15.29:0/3718997176 conn(0x7fd3d40b1870 msgr2=0x7fd3d40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d975a640 1 -- 172.21.15.29:0/3718997176 shutdown_connections 2024-04-25T00:39:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d975a640 1 -- 172.21.15.29:0/3718997176 wait complete. 2024-04-25T00:39:14.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d975a640 1 Processor -- start 2024-04-25T00:39:14.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.882+0000 7fd3d975a640 1 -- start start 2024-04-25T00:39:14.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d975a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3d406dd50 0x7fd3d412f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d975a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40b2e80 0x7fd3d4160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d975a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3d413d140 0x7fd3d41ecff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d975a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d4166f90 con 0x7fd3d40b2e80 2024-04-25T00:39:14.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d975a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d415b3c0 con 0x7fd3d413d140 2024-04-25T00:39:14.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d975a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d4167110 con 0x7fd3d406dd50 2024-04-25T00:39:14.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40b2e80 0x7fd3d4160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40b2e80 0x7fd3d4160dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57002/0 (socket says 172.21.15.29:57002) 2024-04-25T00:39:14.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d27fc640 1 -- 172.21.15.29:0/2367042710 learned_addr learned my addr 172.21.15.29:0/2367042710 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:14.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d2ffd640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3d406dd50 0x7fd3d412f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d37fe640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3d413d140 0x7fd3d41ecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d27fc640 1 -- 172.21.15.29:0/2367042710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3d406dd50 msgr2=0x7fd3d412f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3d406dd50 0x7fd3d412f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d27fc640 1 -- 172.21.15.29:0/2367042710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3d413d140 msgr2=0x7fd3d41ecff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3d413d140 0x7fd3d41ecff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d27fc640 1 -- 172.21.15.29:0/2367042710 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d4161870 con 0x7fd3d40b2e80 2024-04-25T00:39:14.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40b2e80 0x7fd3d4160dd0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800d5f0 tx=0x7fd3c800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:14.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3bb7fe640 1 -- 172.21.15.29:0/2367042710 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80125f0 con 0x7fd3d40b2e80 2024-04-25T00:39:14.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3bb7fe640 1 -- 172.21.15.29:0/2367042710 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c8019070 con 0x7fd3d40b2e80 2024-04-25T00:39:14.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d975a640 1 -- 172.21.15.29:0/2367042710 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d4161b50 con 0x7fd3d40b2e80 2024-04-25T00:39:14.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3bb7fe640 1 -- 172.21.15.29:0/2367042710 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80129b0 con 0x7fd3d40b2e80 2024-04-25T00:39:14.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d975a640 1 -- 172.21.15.29:0/2367042710 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d4162110 con 0x7fd3d40b2e80 2024-04-25T00:39:14.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3bb7fe640 1 -- 172.21.15.29:0/2367042710 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fd3c8012b50 con 0x7fd3d40b2e80 2024-04-25T00:39:14.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3bb7fe640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd3c0067e60 0x7fd3c006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3d2ffd640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd3c0067e60 0x7fd3c006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.886+0000 7fd3bb7fe640 1 -- 172.21.15.29:0/2367042710 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7fd3c8086c10 con 0x7fd3d40b2e80 2024-04-25T00:39:14.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3d2ffd640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd3c0067e60 0x7fd3c006a320 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4004e30 tx=0x7fd3c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:14.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 Processor -- start 2024-04-25T00:39:14.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 -- start start 2024-04-25T00:39:14.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe75ef0 0x559e2fe762d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe76810 0x559e2fe2c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe2cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559e2fc7a5e0 con 0x559e2fe2cbc0 2024-04-25T00:39:14.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559e2fc01c50 con 0x559e2fe75ef0 2024-04-25T00:39:14.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x559e2fc765a0 con 0x559e2fe76810 2024-04-25T00:39:14.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3d37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe2cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3d37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe2cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57014/0 (socket says 172.21.15.29:57014) 2024-04-25T00:39:14.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3d37fe640 1 -- 172.21.15.29:0/2072347630 learned_addr learned my addr 172.21.15.29:0/2072347630 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:14.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3d2ffd640 1 --2- 172.21.15.29:0/2072347630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe75ef0 0x559e2fe762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2072347630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe76810 0x559e2fe2c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3d37fe640 1 -- 172.21.15.29:0/2072347630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe76810 msgr2=0x559e2fe2c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3d37fe640 1 --2- 172.21.15.29:0/2072347630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe76810 0x559e2fe2c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3d37fe640 1 -- 172.21.15.29:0/2072347630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe75ef0 msgr2=0x559e2fe762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3d37fe640 1 --2- 172.21.15.29:0/2072347630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe75ef0 0x559e2fe762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3d37fe640 1 -- 172.21.15.29:0/2072347630 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e2fbffd10 con 0x559e2fe2cbc0 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2072347630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe76810 0x559e2fe2c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3d37fe640 1 --2- 172.21.15.29:0/2072347630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe2cfa0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc01d260 tx=0x7fd3cc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d981ca3e4a05e12a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3b8ff9640 1 -- 172.21.15.29:0/2072347630 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc00b390 con 0x559e2fe2cbc0 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3b8ff9640 1 -- 172.21.15.29:0/2072347630 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3cc0be070 con 0x559e2fe2cbc0 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3b8ff9640 1 -- 172.21.15.29:0/2072347630 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc0a79b0 con 0x559e2fe2cbc0 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2072347630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 msgr2=0x559e2fe2cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2072347630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe2cfa0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc01d260 tx=0x7fd3cc004ac0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2072347630 shutdown_connections 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2072347630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe2cfa0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2072347630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe76810 0x559e2fe2c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2072347630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe75ef0 0x559e2fe762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2072347630 >> 172.21.15.29:0/2072347630 conn(0x559e2fe28b70 msgr2=0x559e2fe2af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:14.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2072347630 shutdown_connections 2024-04-25T00:39:14.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2072347630 wait complete. 2024-04-25T00:39:14.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 -- start start 2024-04-25T00:39:14.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.890+0000 7fd3dc8d3000 1 Processor -- start 2024-04-25T00:39:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe5af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe75ef0 0x559e2fe5b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe76810 0x559e2fe5b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559e2fc34520 con 0x559e2fe2cbc0 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559e2fc798f0 con 0x559e2fe75ef0 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x559e2fc7ec10 con 0x559e2fe76810 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 10 client.0 _ll_register_callbacks cb 0x559e2fe175c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe5af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe5af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57018/0 (socket says 172.21.15.29:57018) 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d2ffd640 1 -- 172.21.15.29:0/2283603102 learned_addr learned my addr 172.21.15.29:0/2283603102 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe75ef0 0x559e2fe5b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d37fe640 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe76810 0x559e2fe5b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d2ffd640 1 -- 172.21.15.29:0/2283603102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe76810 msgr2=0x559e2fe5b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d2ffd640 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe76810 0x559e2fe5b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d2ffd640 1 -- 172.21.15.29:0/2283603102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe75ef0 msgr2=0x559e2fe5b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d2ffd640 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe75ef0 0x559e2fe5b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d2ffd640 1 -- 172.21.15.29:0/2283603102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e2fc127b0 con 0x559e2fe2cbc0 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d37fe640 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe76810 0x559e2fe5b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d2ffd640 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe5af20 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4005630 tx=0x7fd3c4003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0cc349cfa32404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3b1ffb640 1 -- 172.21.15.29:0/2283603102 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4021420 con 0x559e2fe2cbc0 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3b1ffb640 1 -- 172.21.15.29:0/2283603102 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7fd3c4010040 con 0x559e2fe2cbc0 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2283603102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3cc0be070 con 0x559e2fe2cbc0 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3b1ffb640 1 -- 172.21.15.29:0/2283603102 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fd3c4003810 con 0x559e2fe2cbc0 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 10 client.9168 fetch_fsmap learned FSMap version 83 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2283603102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x559e2fc01ea0 con 0x559e2fe2cbc0 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3b1ffb640 1 -- 172.21.15.29:0/2283603102 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3cc0be070 con 0x559e2fe2cbc0 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3b1ffb640 1 -- 172.21.15.29:0/2283603102 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7fd3c4031080 con 0x559e2fe2cbc0 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 10 client.9168 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T00:39:14.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 10 client.9168 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 20 client.9168 populate_metadata read hostname 'smithi029' 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.29:6834/619209926,v1:172.21.15.29:6835/619209926] conn(0x559e2fe85f90 0x559e2fe88380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 4 client.9168 mds_command: new command op to 9021 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2283603102 --> [v2:172.21.15.29:6834/619209926,v1:172.21.15.29:6835/619209926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e2fc15ab0 con 0x559e2fe85f90 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.29:6834/619209926,v1:172.21.15.29:6835/619209926] conn(0x559e2fe85f90 0x559e2fe88380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3b1ffb640 10 client.9168 ms_handle_connect on v2:172.21.15.29:6834/619209926 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.29:6834/619209926,v1:172.21.15.29:6835/619209926] conn(0x559e2fe85f90 0x559e2fe88380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5311c0be98c48e5f server_cookie=34a938aed0eca8e7 in_seq=0 out_seq=0 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3b1ffb640 1 -- 172.21.15.29:0/2283603102 <== mds.0 v2:172.21.15.29:6834/619209926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e2fc15ab0 con 0x559e2fe85f90 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3b1ffb640 10 client.9168 handle_command_reply: tid=0 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 client.9168 shutdown 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2283603102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 msgr2=0x559e2fe5af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe5af20 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4005630 tx=0x7fd3c4003700 comp rx=0 tx=0).stop 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2283603102 shutdown_connections 2024-04-25T00:39:14.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.29:6834/619209926,v1:172.21.15.29:6835/619209926] conn(0x559e2fe85f90 0x559e2fe88380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe76810 0x559e2fe5b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe75ef0 0x559e2fe5b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3b1ffb640 0 client.9168 ms_handle_reset on v2:172.21.15.29:6834/619209926 2024-04-25T00:39:14.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2283603102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe5af20 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2283603102 >> 172.21.15.29:0/2283603102 conn(0x559e2fe28b70 msgr2=0x559e2fe2b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:14.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2283603102 shutdown_connections 2024-04-25T00:39:14.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2283603102 wait complete. 2024-04-25T00:39:14.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.894+0000 7fd3dc8d3000 20 client.9168 trim_cache size 0 max 16384 2024-04-25T00:39:14.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 Processor -- start 2024-04-25T00:39:14.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 -- start start 2024-04-25T00:39:14.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe2cbc0 0x559e2fe75330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe75ef0 0x559e2fe75870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe76810 0x559e2fe2bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d4203940 con 0x559e2fe75ef0 2024-04-25T00:39:14.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d412d530 con 0x559e2fe76810 2024-04-25T00:39:14.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d415f6d0 con 0x559e2fe2cbc0 2024-04-25T00:39:14.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3d27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe75ef0 0x559e2fe75870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3d27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe75ef0 0x559e2fe75870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57024/0 (socket says 172.21.15.29:57024) 2024-04-25T00:39:14.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3d27fc640 1 -- 172.21.15.29:0/2443751514 learned_addr learned my addr 172.21.15.29:0/2443751514 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:14.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3d27fc640 1 -- 172.21.15.29:0/2443751514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe2cbc0 msgr2=0x559e2fe75330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2443751514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe2cbc0 0x559e2fe75330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3d27fc640 1 -- 172.21.15.29:0/2443751514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe76810 msgr2=0x559e2fe2bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2443751514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe76810 0x559e2fe2bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3d27fc640 1 -- 172.21.15.29:0/2443751514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e2fc02e80 con 0x559e2fe75ef0 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2443751514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe75ef0 0x559e2fe75870 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fd3c805ee50 tx=0x7fd3c80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b5d85c5249523e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3b3fff640 1 -- 172.21.15.29:0/2443751514 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c805f380 con 0x559e2fe75ef0 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3b3fff640 1 -- 172.21.15.29:0/2443751514 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c80547b0 con 0x559e2fe75ef0 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3b3fff640 1 -- 172.21.15.29:0/2443751514 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8052d80 con 0x559e2fe75ef0 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2443751514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe75ef0 msgr2=0x559e2fe75870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2443751514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe75ef0 0x559e2fe75870 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fd3c805ee50 tx=0x7fd3c80046d0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2443751514 shutdown_connections 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2443751514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe76810 0x559e2fe2bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2443751514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe75ef0 0x559e2fe75870 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2443751514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe2cbc0 0x559e2fe75330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2443751514 >> 172.21.15.29:0/2443751514 conn(0x559e2fe28b70 msgr2=0x559e2fe74800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2443751514 shutdown_connections 2024-04-25T00:39:14.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.914+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2443751514 wait complete. 2024-04-25T00:39:14.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 -- start start 2024-04-25T00:39:14.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 Processor -- start 2024-04-25T00:39:14.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe88af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe75ef0 0x559e2fe89030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe76810 0x559e2fe89570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559e2fc02640 con 0x559e2fe2cbc0 2024-04-25T00:39:14.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d412cee0 con 0x559e2fe76810 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x559e2fc12dd0 con 0x559e2fe75ef0 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 10 client.0 _ll_register_callbacks cb 0x7fd3c40053b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe88af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe88af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57038/0 (socket says 172.21.15.29:57038) 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d2ffd640 1 -- 172.21.15.29:0/2805076027 learned_addr learned my addr 172.21.15.29:0/2805076027 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe75ef0 0x559e2fe89030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d2ffd640 1 -- 172.21.15.29:0/2805076027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe75ef0 msgr2=0x559e2fe89030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d2ffd640 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe75ef0 0x559e2fe89030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d2ffd640 1 -- 172.21.15.29:0/2805076027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe76810 msgr2=0x559e2fe89570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d37fe640 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe76810 0x559e2fe89570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d2ffd640 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe76810 0x559e2fe89570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d2ffd640 1 -- 172.21.15.29:0/2805076027 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e2fc47760 con 0x559e2fe2cbc0 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe75ef0 0x559e2fe89030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:14.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d2ffd640 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe88af0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4000c10 tx=0x7fd3c4003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e777cdbd39d24d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3b1ffb640 1 -- 172.21.15.29:0/2805076027 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4021a20 con 0x559e2fe2cbc0 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3b1ffb640 1 -- 172.21.15.29:0/2805076027 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7fd3c4021bc0 con 0x559e2fe2cbc0 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2805076027 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3c80547b0 con 0x559e2fe2cbc0 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3b1ffb640 1 -- 172.21.15.29:0/2805076027 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fd3c403f050 con 0x559e2fe2cbc0 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 10 client.9174 fetch_fsmap learned FSMap version 83 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2805076027 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x559e2fcd39e0 con 0x559e2fe2cbc0 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3b1ffb640 1 -- 172.21.15.29:0/2805076027 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c80547b0 con 0x559e2fe2cbc0 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3b1ffb640 1 -- 172.21.15.29:0/2805076027 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7fd3c4042020 con 0x559e2fe2cbc0 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 10 client.9174 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 10 client.9174 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 20 client.9174 populate_metadata read hostname 'smithi029' 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.29:6834/619209926,v1:172.21.15.29:6835/619209926] conn(0x559e2fe89ab0 0x559e2fe89e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 4 client.9174 mds_command: new command op to 9021 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:39:14.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2805076027 --> [v2:172.21.15.29:6834/619209926,v1:172.21.15.29:6835/619209926] -- command(tid 0: {"prefix": "status"}) -- 0x559e2fc87b20 con 0x559e2fe89ab0 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.29:6834/619209926,v1:172.21.15.29:6835/619209926] conn(0x559e2fe89ab0 0x559e2fe89e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d27fc640 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.29:6834/619209926,v1:172.21.15.29:6835/619209926] conn(0x559e2fe89ab0 0x559e2fe89e90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a3009be26e1b645 server_cookie=4321c8bdb17e4abc in_seq=0 out_seq=0 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3b1ffb640 10 client.9174 ms_handle_connect on v2:172.21.15.29:6834/619209926 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3b1ffb640 1 -- 172.21.15.29:0/2805076027 <== mds.0 v2:172.21.15.29:6834/619209926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x559e2fc87b20 con 0x559e2fe89ab0 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3b1ffb640 10 client.9174 handle_command_reply: tid=0 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 client.9174 shutdown 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2805076027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 msgr2=0x559e2fe88af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe88af0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4000c10 tx=0x7fd3c4003770 comp rx=0 tx=0).stop 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2805076027 shutdown_connections 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.29:6834/619209926,v1:172.21.15.29:6835/619209926] conn(0x559e2fe89ab0 0x559e2fe89e90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e2fe76810 0x559e2fe89570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559e2fe75ef0 0x559e2fe89030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 --2- 172.21.15.29:0/2805076027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559e2fe2cbc0 0x559e2fe88af0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3b1ffb640 0 client.9174 ms_handle_reset on v2:172.21.15.29:6834/619209926 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2805076027 >> 172.21.15.29:0/2805076027 conn(0x559e2fe28b70 msgr2=0x559e2fe75090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:14.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2805076027 shutdown_connections 2024-04-25T00:39:14.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 1 -- 172.21.15.29:0/2805076027 wait complete. 2024-04-25T00:39:14.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3dc8d3000 20 client.9174 trim_cache size 0 max 16384 2024-04-25T00:39:14.928 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:39:14.928 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "59afbe7d-aaa8-45e2-b468-69ee150cacea", 2024-04-25T00:39:14.928 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-04-25T00:39:14.928 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 9021, 2024-04-25T00:39:14.928 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-04-25T00:39:14.928 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-04-25T00:39:14.928 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-04-25T00:39:14.928 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 5.7932674090000003, 2024-04-25T00:39:14.928 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 83, 2024-04-25T00:39:14.928 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 132, 2024-04-25T00:39:14.929 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 132, 2024-04-25T00:39:14.929 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 11.116652975999999 2024-04-25T00:39:14.929 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:39:14.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d975a640 1 -- 172.21.15.29:0/2367042710 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd3c0067e60 msgr2=0x7fd3c006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d975a640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd3c0067e60 0x7fd3c006a320 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4004e30 tx=0x7fd3c4009040 comp rx=0 tx=0).stop 2024-04-25T00:39:14.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d975a640 1 -- 172.21.15.29:0/2367042710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40b2e80 msgr2=0x7fd3d4160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:14.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.918+0000 7fd3d975a640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40b2e80 0x7fd3d4160dd0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800d5f0 tx=0x7fd3c800dab0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.922+0000 7fd3d975a640 1 -- 172.21.15.29:0/2367042710 shutdown_connections 2024-04-25T00:39:14.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.922+0000 7fd3d975a640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd3c0067e60 0x7fd3c006a320 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.922+0000 7fd3d975a640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3d413d140 0x7fd3d41ecff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.922+0000 7fd3d975a640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40b2e80 0x7fd3d4160dd0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.922+0000 7fd3d975a640 1 --2- 172.21.15.29:0/2367042710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3d406dd50 0x7fd3d412f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:14.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.922+0000 7fd3d975a640 1 -- 172.21.15.29:0/2367042710 >> 172.21.15.29:0/2367042710 conn(0x7fd3d40b1870 msgr2=0x7fd3d40af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:14.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.922+0000 7fd3d975a640 1 -- 172.21.15.29:0/2367042710 shutdown_connections 2024-04-25T00:39:14.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:14.922+0000 7fd3d975a640 1 -- 172.21.15.29:0/2367042710 wait complete. 2024-04-25T00:39:14.940 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:39:15.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.094+0000 7f08278f8640 1 Processor -- start 2024-04-25T00:39:15.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.094+0000 7f08278f8640 1 -- start start 2024-04-25T00:39:15.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f08278f8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f08200b3140 0x7f082006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f08278f8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08200b3c30 0x7f08200ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f08278f8640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08200b41e0 con 0x7f08200b4b10 2024-04-25T00:39:15.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f08278f8640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f082006d960 con 0x7f08200b3140 2024-04-25T00:39:15.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f08278f8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f082006dc40 con 0x7f08200b3c30 2024-04-25T00:39:15.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f0825694640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f08200b3140 0x7f082006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f0824e93640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f08200b4b10 0x7f08200b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39106/0 (socket says 172.21.15.29:39106) 2024-04-25T00:39:15.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f0824e93640 1 -- 172.21.15.29:0/2943990119 learned_addr learned my addr 172.21.15.29:0/2943990119 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:15.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f0825694640 1 --2- 172.21.15.29:0/2943990119 >> v2:172.21.15.29:3300/0 conn(0x7f08200b3140 0x7f082006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57044/0 (socket says 172.21.15.29:57044) 2024-04-25T00:39:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f0825e95640 1 --2- 172.21.15.29:0/2943990119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08200b3c30 0x7f08200ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f0825694640 1 -- 172.21.15.29:0/2943990119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08200b3c30 msgr2=0x7f08200ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f0825694640 1 --2- 172.21.15.29:0/2943990119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08200b3c30 0x7f08200ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f0825694640 1 -- 172.21.15.29:0/2943990119 >> v1:172.21.15.29:6789/0 conn(0x7f08200b4b10 legacy=0x7f08200b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f0825694640 1 -- 172.21.15.29:0/2943990119 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082006df20 con 0x7f08200b3140 2024-04-25T00:39:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f0825694640 1 --2- 172.21.15.29:0/2943990119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08200b3140 0x7f082006e440 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f081c0097f0 tx=0x7f081c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6987e6b10c56f6a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f0817fff640 1 -- 172.21.15.29:0/2943990119 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c0b0070 con 0x7f08200b3140 2024-04-25T00:39:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f0817fff640 1 -- 172.21.15.29:0/2943990119 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f081c0ab600 con 0x7f08200b3140 2024-04-25T00:39:15.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.098+0000 7f0817fff640 1 -- 172.21.15.29:0/2943990119 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c0ab9d0 con 0x7f08200b3140 2024-04-25T00:39:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 -- 172.21.15.29:0/2943990119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08200b3140 msgr2=0x7f082006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 --2- 172.21.15.29:0/2943990119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08200b3140 0x7f082006e440 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f081c0097f0 tx=0x7f081c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 -- 172.21.15.29:0/2943990119 shutdown_connections 2024-04-25T00:39:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 --2- 172.21.15.29:0/2943990119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08200b3c30 0x7f08200ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 --2- 172.21.15.29:0/2943990119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08200b3140 0x7f082006e440 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 -- 172.21.15.29:0/2943990119 >> 172.21.15.29:0/2943990119 conn(0x7f08200b1870 msgr2=0x7f08200b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:15.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 -- 172.21.15.29:0/2943990119 shutdown_connections 2024-04-25T00:39:15.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 -- 172.21.15.29:0/2943990119 wait complete. 2024-04-25T00:39:15.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 Processor -- start 2024-04-25T00:39:15.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 -- start start 2024-04-25T00:39:15.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08200b3140 0x7f082013b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08200b3c30 0x7f0820156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08200b4b10 0x7f0820158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0820168d80 con 0x7f08200b3c30 2024-04-25T00:39:15.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0820168c00 con 0x7f08200b4b10 2024-04-25T00:39:15.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0820168f00 con 0x7f08200b3140 2024-04-25T00:39:15.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0824e93640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08200b3c30 0x7f0820156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0824e93640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08200b3c30 0x7f0820156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57048/0 (socket says 172.21.15.29:57048) 2024-04-25T00:39:15.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0824e93640 1 -- 172.21.15.29:0/3972707297 learned_addr learned my addr 172.21.15.29:0/3972707297 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:15.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0825694640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08200b3140 0x7f082013b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0825e95640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08200b4b10 0x7f0820158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0824e93640 1 -- 172.21.15.29:0/3972707297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08200b3140 msgr2=0x7f082013b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0824e93640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08200b3140 0x7f082013b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0824e93640 1 -- 172.21.15.29:0/3972707297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08200b4b10 msgr2=0x7f0820158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0824e93640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08200b4b10 0x7f0820158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0824e93640 1 -- 172.21.15.29:0/3972707297 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082013b620 con 0x7f08200b3c30 2024-04-25T00:39:15.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0824e93640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08200b3c30 0x7f0820156c10 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f081000c450 tx=0x7f081000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:15.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0815ffb640 1 -- 172.21.15.29:0/3972707297 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810017070 con 0x7f08200b3c30 2024-04-25T00:39:15.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0815ffb640 1 -- 172.21.15.29:0/3972707297 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0810005cf0 con 0x7f08200b3c30 2024-04-25T00:39:15.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 -- 172.21.15.29:0/3972707297 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0820157560 con 0x7f08200b3c30 2024-04-25T00:39:15.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f0815ffb640 1 -- 172.21.15.29:0/3972707297 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810007290 con 0x7f08200b3c30 2024-04-25T00:39:15.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.102+0000 7f08278f8640 1 -- 172.21.15.29:0/3972707297 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0820157b20 con 0x7f08200b3c30 2024-04-25T00:39:15.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.106+0000 7f08278f8640 1 -- 172.21.15.29:0/3972707297 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0820069270 con 0x7f08200b3c30 2024-04-25T00:39:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.106+0000 7f0815ffb640 1 -- 172.21.15.29:0/3972707297 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f081001a070 con 0x7f08200b3c30 2024-04-25T00:39:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.106+0000 7f0815ffb640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0804067e90 0x7f080406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.106+0000 7f0815ffb640 1 -- 172.21.15.29:0/3972707297 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f08100858e0 con 0x7f08200b3c30 2024-04-25T00:39:15.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.106+0000 7f0825694640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0804067e90 0x7f080406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:15.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.106+0000 7f0825694640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0804067e90 0x7f080406a350 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f081c0ad000 tx=0x7f081c0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:15.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.110+0000 7f0815ffb640 1 -- 172.21.15.29:0/3972707297 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f08100555f0 con 0x7f08200b3c30 2024-04-25T00:39:15.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f08278f8640 1 -- 172.21.15.29:0/3972707297 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0820155c80 con 0x7f08200b3c30 2024-04-25T00:39:15.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f0815ffb640 1 -- 172.21.15.29:0/3972707297 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) ==== 53+0+73 (secure 0 0 0) 0x7f08100598f0 con 0x7f08200b3c30 2024-04-25T00:39:15.312 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:39:15.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f08278f8640 1 -- 172.21.15.29:0/3972707297 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0804067e90 msgr2=0x7f080406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f08278f8640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0804067e90 0x7f080406a350 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f081c0ad000 tx=0x7f081c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:39:15.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f08278f8640 1 -- 172.21.15.29:0/3972707297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08200b3c30 msgr2=0x7f0820156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:15.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f08278f8640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08200b3c30 0x7f0820156c10 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f081000c450 tx=0x7f081000c910 comp rx=0 tx=0).stop 2024-04-25T00:39:15.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f08278f8640 1 -- 172.21.15.29:0/3972707297 shutdown_connections 2024-04-25T00:39:15.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f08278f8640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0804067e90 0x7f080406a350 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f08278f8640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08200b4b10 0x7f0820158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f08278f8640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08200b3c30 0x7f0820156c10 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f08278f8640 1 --2- 172.21.15.29:0/3972707297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08200b3140 0x7f082013b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:15.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f08278f8640 1 -- 172.21.15.29:0/3972707297 >> 172.21.15.29:0/3972707297 conn(0x7f08200b1870 msgr2=0x7f08200cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:15.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f08278f8640 1 -- 172.21.15.29:0/3972707297 shutdown_connections 2024-04-25T00:39:15.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15.310+0000 7f08278f8640 1 -- 172.21.15.29:0/3972707297 wait complete. 2024-04-25T00:39:15.331 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:39:15.331 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:39:15.331 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-25T00:39:15.331 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:39:15.331 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:15.331 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:39:15.331 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:39:15.331 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:39:15.331 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:39:15.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:15.332 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-25T00:39:15.339 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:39:15.339 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:39:15.339 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:39:15.339 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:15.339 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:39:15.339 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:15.339 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:39:15.339 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:15.339 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-25T00:39:15.339 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:39:15.339 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42004sec preferred_lft 42004sec 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 02:42:28:a5:36:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:15.340 INFO:teuthology.orchestra.run.smithi029.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:39:15.341 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:15.341 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:39:15.341 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:15.341 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:39:15.341 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:15.341 INFO:teuthology.orchestra.run.smithi029.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:39:15.341 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:39:15.341 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:39:15.341 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:15.341 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:39:15.341 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:39:15.341 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:39:15.341 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:15.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:15.342 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:39:15.395 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-25T00:39:15.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:15.397 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-25T00:39:15.446 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-25T00:39:15.446 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:39:15.446 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T00:39:15.446 INFO:teuthology.orchestra.run.smithi029.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:39:15.446 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T00:39:15.446 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:39:15.446 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:39:15.446 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:39:15.446 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:39:15.447 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:39:15.447 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:39:15.447 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:39:15.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:39:15.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:39:15.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:15.572 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:39:15.578 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:39:15.578 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:39:15.578 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:39:15.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:15.578 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-04-25T00:39:15.631 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:39:15.631 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:39:15.631 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:15.680 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:39:15.680 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:39:15.680 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:15.728 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:39:15.785 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:39:15.837 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:39:15.837 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:39:15.837 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:39:15.838 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:39:15.838 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:39:15.838 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:39:15.838 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:39:15.838 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:39:15.838 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:39:15.838 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:39:15.838 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:39:15.838 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:39:15.838 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:39:15.838 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:39:15.838 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:39:15.838 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:39:15.839 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:39:15.840 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:39:15.840 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:39:15.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:15.841 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:39:15.889 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:39:15.889 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:39:15.933 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:39:15.949 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:39:15.996 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:39:15.996 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:39:15.997 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:39:15.997 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:39:15.997 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:39:15.997 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:39:15.997 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:39:15.997 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:39:15.997 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:39:15.997 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:39:15.997 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:39:15.997 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:39:15.997 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:39:15.998 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:39:15.998 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:39:15.998 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:39:15.998 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:39:15.998 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:39:15.998 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:39:15.998 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:39:15.998 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:39:15.998 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:39:15.998 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:39:15.998 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:39:15.999 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:39:15.999 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:39:15.999 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:39:15.999 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:39:15.999 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:39:15.999 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:39:15.999 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:39:15.999 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:39:15.999 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:39:16.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:16.000 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:39:16.013 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:2024-04-25T00:39:16.010+0000 7ff380494580 -1 init, newargv = 0x55a9e37b4df0 newargc=15 2024-04-25T00:39:16.013 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[52579]: starting ceph client 2024-04-25T00:39:16.018 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[52579]: starting fuse 2024-04-25T00:39:17.119 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:39:17.136 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:39:17.186 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:39:17.186 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:39:17.186 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:39:17.186 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:39:17.186 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:39:17.186 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:39:17.186 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:39:17.186 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:39:17.186 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:39:17.186 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:39:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:39:17.188 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:39:17.188 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:39:17.188 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:39:17.188 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:39:17.188 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:39:17.188 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:39:17.188 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:39:17.188 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:39:17.188 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:39:17.188 INFO:teuthology.orchestra.run.smithi029.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:17.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:17.189 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:39:17.236 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:39:17.236 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-04-25T00:39:17.237 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-04-25T00:39:17.238 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:39:17.238 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-04-25T00:39:17.238 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:39:17.238 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:39:17.238 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:39:17.238 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-04-25T00:39:17.238 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-04-25T00:39:17.238 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-04-25T00:39:17.238 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:39:17.238 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:39:17.238 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.0")) 2024-04-25T00:39:17.238 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-04-25T00:39:17.370 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.0.52579.asok 2024-04-25T00:39:17.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:39:17.383 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52579.asok 2024-04-25T00:39:17.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:17.383 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52579.asok status 2024-04-25T00:39:17.541 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:39:17.541 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-04-25T00:39:17.541 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:39:17.541 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:39:17.541 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "0", 2024-04-25T00:39:17.541 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-04-25T00:39:17.541 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:39:17.541 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "52579", 2024-04-25T00:39:17.541 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-04-25T00:39:17.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:17.541 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 9192, 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 9192 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 748300000 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:17.542 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:39:17.543 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:39:17.543 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:39:17.543 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 748300000 2024-04-25T00:39:17.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:17.543 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.9192 192.168.0.1:0/748300000", 2024-04-25T00:39:17.543 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.1:0/748300000", 2024-04-25T00:39:17.543 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-04-25T00:39:17.543 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 83, 2024-04-25T00:39:17.543 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 132, 2024-04-25T00:39:17.543 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:39:17.543 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-04-25T00:39:17.543 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-04-25T00:39:17.543 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:39:17.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:17.555 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-04-25T00:39:17.555 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:17.562 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-04-25T00:39:17.562 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:17.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:17.562 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:17.617 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:39:17.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.786+0000 7f9af1e22640 1 Processor -- start 2024-04-25T00:39:17.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.786+0000 7f9af1e22640 1 -- start start 2024-04-25T00:39:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9af1e22640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9aec0cfa10 0x7f9aec0cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9af1e22640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aec0d04a0 0x7f9aec0b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9af1e22640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aec0cdbd0 con 0x7f9aec15c220 2024-04-25T00:39:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9af1e22640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9aec15c600 con 0x7f9aec0cfa10 2024-04-25T00:39:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9af1e22640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9aec0b1a30 con 0x7f9aec0d04a0 2024-04-25T00:39:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aeb7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9aec15c220 0x7f9aec0cd350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40464/0 (socket says 172.21.15.29:40464) 2024-04-25T00:39:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aeb7fe640 1 -- 172.21.15.29:0/3944405142 learned_addr learned my addr 172.21.15.29:0/3944405142 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1550793685 0 0) 0x7f9aec0cdbd0 con 0x7f9aec15c220 2024-04-25T00:39:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aebfff640 1 --2- 172.21.15.29:0/3944405142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aec0d04a0 0x7f9aec0b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ad4003530 con 0x7f9aec15c220 2024-04-25T00:39:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aeaffd640 1 --2- 172.21.15.29:0/3944405142 >> v2:172.21.15.157:3300/0 conn(0x7f9aec0cfa10 0x7f9aec0cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1659261588 0 0) 0x7f9ad4003530 con 0x7f9aec15c220 2024-04-25T00:39:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aec0d04a0 msgr2=0x7f9aec0b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 --2- 172.21.15.29:0/3944405142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aec0d04a0 0x7f9aec0b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 >> v2:172.21.15.157:3300/0 conn(0x7f9aec0cfa10 msgr2=0x7f9aec0cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 --2- 172.21.15.29:0/3944405142 >> v2:172.21.15.157:3300/0 conn(0x7f9aec0cfa10 0x7f9aec0cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec0b13e0 con 0x7f9aec15c220 2024-04-25T00:39:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f9adc001680 con 0x7f9aec15c220 2024-04-25T00:39:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aeaffd640 1 --2- 172.21.15.29:0/3944405142 >> v2:172.21.15.157:3300/0 conn(0x7f9aec0cfa10 0x7f9aec0cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 >> v1:172.21.15.29:6789/0 conn(0x7f9aec15c220 legacy=0x7f9aec0cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 --2- 172.21.15.29:0/3944405142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad40039c0 0x7f9ad4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 --2- 172.21.15.29:0/3944405142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ad4004810 0x7f9ad4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 --2- 172.21.15.29:0/3944405142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aec0d04a0 0x7f9ad40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec15c600 con 0x7f9ad4004810 2024-04-25T00:39:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec0b1a30 con 0x7f9ad40039c0 2024-04-25T00:39:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad400b9d0 con 0x7f9aec0d04a0 2024-04-25T00:39:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aeaffd640 1 --2- 172.21.15.29:0/3944405142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ad4004810 0x7f9ad4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aeb7fe640 1 --2- 172.21.15.29:0/3944405142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad40039c0 0x7f9ad4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aeaffd640 1 -- 172.21.15.29:0/3944405142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aec0d04a0 msgr2=0x7f9ad40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aeaffd640 1 --2- 172.21.15.29:0/3944405142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9aec0d04a0 0x7f9ad40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aeaffd640 1 -- 172.21.15.29:0/3944405142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad40039c0 msgr2=0x7f9ad4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aeaffd640 1 --2- 172.21.15.29:0/3944405142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad40039c0 0x7f9ad4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.790+0000 7f9aeaffd640 1 -- 172.21.15.29:0/3944405142 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad4003530 con 0x7f9ad4004810 2024-04-25T00:39:17.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aeaffd640 1 --2- 172.21.15.29:0/3944405142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ad4004810 0x7f9ad4004c90 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00098d0 tx=0x7f9ae001c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f48e06f563e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:17.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000f040 con 0x7f9ad4004810 2024-04-25T00:39:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae00a9680 con 0x7f9ad4004810 2024-04-25T00:39:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aea7fc640 1 -- 172.21.15.29:0/3944405142 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae00a9a30 con 0x7f9ad4004810 2024-04-25T00:39:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 -- 172.21.15.29:0/3944405142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ad4004810 msgr2=0x7f9ad4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 --2- 172.21.15.29:0/3944405142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ad4004810 0x7f9ad4004c90 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00098d0 tx=0x7f9ae001c650 comp rx=0 tx=0).stop 2024-04-25T00:39:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aebfff640 1 -- 172.21.15.29:0/3944405142 reap_dead start 2024-04-25T00:39:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 -- 172.21.15.29:0/3944405142 shutdown_connections 2024-04-25T00:39:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 -- 172.21.15.29:0/3944405142 >> 172.21.15.29:0/3944405142 conn(0x7f9aec0d1d60 msgr2=0x7f9aec0d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 -- 172.21.15.29:0/3944405142 shutdown_connections 2024-04-25T00:39:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 -- 172.21.15.29:0/3944405142 wait complete. 2024-04-25T00:39:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 Processor -- start 2024-04-25T00:39:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 -- start start 2024-04-25T00:39:17.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad4004810 0x7f9aec0ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:17.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec1f4f30 0x7f9aec1f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:17.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aec0c7fb0 0x7f9aec0c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec0b2040 con 0x7f9aec0c7fb0 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec0b1ec0 con 0x7f9aec1f4f30 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9af1e22640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9aec0b21c0 con 0x7f9ad4004810 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aec0c7fb0 0x7f9aec0c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aec0c7fb0 0x7f9aec0c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39766/0 (socket says 172.21.15.29:39766) 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aebfff640 1 -- 172.21.15.29:0/2456320179 learned_addr learned my addr 172.21.15.29:0/2456320179 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aeaffd640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec1f4f30 0x7f9aec1f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aeb7fe640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad4004810 0x7f9aec0ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aebfff640 1 -- 172.21.15.29:0/2456320179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad4004810 msgr2=0x7f9aec0ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aebfff640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad4004810 0x7f9aec0ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aebfff640 1 -- 172.21.15.29:0/2456320179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec1f4f30 msgr2=0x7f9aec1f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aebfff640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec1f4f30 0x7f9aec1f5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aebfff640 1 -- 172.21.15.29:0/2456320179 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec0cafc0 con 0x7f9aec0c7fb0 2024-04-25T00:39:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.794+0000 7f9aebfff640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aec0c7fb0 0x7f9aec0c8460 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0c5690 tx=0x7f9ae400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:17.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.798+0000 7f9ad3fff640 1 -- 172.21.15.29:0/2456320179 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae400bce0 con 0x7f9aec0c7fb0 2024-04-25T00:39:17.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.798+0000 7f9ad3fff640 1 -- 172.21.15.29:0/2456320179 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae401a070 con 0x7f9aec0c7fb0 2024-04-25T00:39:17.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.798+0000 7f9af1e22640 1 -- 172.21.15.29:0/2456320179 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aec0cb2a0 con 0x7f9aec0c7fb0 2024-04-25T00:39:17.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.798+0000 7f9ad3fff640 1 -- 172.21.15.29:0/2456320179 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4014640 con 0x7f9aec0c7fb0 2024-04-25T00:39:17.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.798+0000 7f9af1e22640 1 -- 172.21.15.29:0/2456320179 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aec0cb860 con 0x7f9aec0c7fb0 2024-04-25T00:39:17.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.798+0000 7f9af1e22640 1 -- 172.21.15.29:0/2456320179 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aec069270 con 0x7f9aec0c7fb0 2024-04-25T00:39:17.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.798+0000 7f9ad3fff640 1 -- 172.21.15.29:0/2456320179 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f9ae4013070 con 0x7f9aec0c7fb0 2024-04-25T00:39:17.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.798+0000 7f9ad3fff640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ad8067e60 0x7f9ad806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:17.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.798+0000 7f9ad3fff640 1 -- 172.21.15.29:0/2456320179 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f9ae4086800 con 0x7f9aec0c7fb0 2024-04-25T00:39:17.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.798+0000 7f9aeb7fe640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ad8067e60 0x7f9ad806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:17.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.798+0000 7f9aeb7fe640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ad8067e60 0x7f9ad806a320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9adc000fc0 tx=0x7f9adc0047f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:17.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:17.802+0000 7f9ad3fff640 1 -- 172.21.15.29:0/2456320179 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9ae4056b20 con 0x7f9aec0c7fb0 2024-04-25T00:39:18.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.002+0000 7f9af1e22640 1 -- 172.21.15.29:0/2456320179 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9aec0d2a80 con 0x7f9aec0c7fb0 2024-04-25T00:39:18.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.002+0000 7f9ad3fff640 1 -- 172.21.15.29:0/2456320179 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) ==== 53+0+73 (secure 0 0 0) 0x7f9ae4056420 con 0x7f9aec0c7fb0 2024-04-25T00:39:18.006 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:39:18.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.006+0000 7f9af1e22640 1 -- 172.21.15.29:0/2456320179 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ad8067e60 msgr2=0x7f9ad806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.006+0000 7f9af1e22640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ad8067e60 0x7f9ad806a320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9adc000fc0 tx=0x7f9adc0047f0 comp rx=0 tx=0).stop 2024-04-25T00:39:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.006+0000 7f9af1e22640 1 -- 172.21.15.29:0/2456320179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aec0c7fb0 msgr2=0x7f9aec0c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.006+0000 7f9af1e22640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aec0c7fb0 0x7f9aec0c8460 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0c5690 tx=0x7f9ae400b480 comp rx=0 tx=0).stop 2024-04-25T00:39:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.006+0000 7f9af1e22640 1 -- 172.21.15.29:0/2456320179 shutdown_connections 2024-04-25T00:39:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.006+0000 7f9af1e22640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ad8067e60 0x7f9ad806a320 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.006+0000 7f9af1e22640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aec0c7fb0 0x7f9aec0c8460 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.006+0000 7f9af1e22640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9aec1f4f30 0x7f9aec1f5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.006+0000 7f9af1e22640 1 --2- 172.21.15.29:0/2456320179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad4004810 0x7f9aec0ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.006+0000 7f9af1e22640 1 -- 172.21.15.29:0/2456320179 >> 172.21.15.29:0/2456320179 conn(0x7f9aec0d1d60 msgr2=0x7f9aec0cd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.006+0000 7f9af1e22640 1 -- 172.21.15.29:0/2456320179 shutdown_connections 2024-04-25T00:39:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:18.006+0000 7f9af1e22640 1 -- 172.21.15.29:0/2456320179 wait complete. 2024-04-25T00:39:18.025 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:39:18.025 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:39:18.025 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-25T00:39:18.026 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:39:18.026 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:18.026 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:39:18.026 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:39:18.026 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:39:18.026 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:39:18.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:18.026 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-25T00:39:18.033 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:39:18.033 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:39:18.033 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:18.034 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 41997sec preferred_lft 41997sec 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:9c:8d:4b:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:39:18.035 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:18.036 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:39:18.036 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:18.036 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:39:18.036 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:18.036 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:39:18.036 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:18.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:18.036 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:39:18.090 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:39:18.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:18.092 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-25T00:39:18.144 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-25T00:39:18.144 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:39:18.144 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:39:18.145 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:39:18.145 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:39:18.145 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:39:18.145 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:39:18.145 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:39:18.145 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:39:18.145 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:39:18.145 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:39:18.145 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:39:18.145 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:39:18.145 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:39:18.146 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:39:18.146 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:39:18.146 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:39:18.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:39:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:39:18.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:39:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:39:18.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:18.267 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:39:18.272 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:39:18.273 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:39:18.273 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:39:18.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:18.273 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-25T00:39:18.327 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T00:39:18.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:39:18.327 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:18.375 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:39:18.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:39:18.375 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:18.422 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:39:18.480 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:39:18.529 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:39:18.529 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:39:18.530 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:39:18.530 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:39:18.530 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:39:18.530 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:39:18.530 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:39:18.530 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:39:18.530 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:39:18.530 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:39:18.530 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:39:18.530 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:39:18.530 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:39:18.530 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:39:18.530 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:39:18.531 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:18.532 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:18.532 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:18.532 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:18.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:18.533 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:39:18.578 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:39:18.579 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T00:39:18.624 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:39:18.638 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:39:18.686 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:39:18.686 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:39:18.687 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:18.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:18.689 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:39:18.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-04-25T00:39:18.701+0000 7fd230520580 -1 init, newargv = 0x55c30ef6dc80 newargc=15 2024-04-25T00:39:18.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34907]: starting ceph client 2024-04-25T00:39:18.712 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34907]: starting fuse 2024-04-25T00:39:19.810 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:39:19.827 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:39:19.876 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:39:19.876 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:39:19.876 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:39:19.876 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:39:19.876 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:39:19.876 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:39:19.876 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:39:19.876 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:39:19.876 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:39:19.876 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:39:19.876 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:39:19.876 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:39:19.877 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:39:19.878 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:39:19.878 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:39:19.878 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:19.878 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:19.878 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:19.878 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:19.878 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:19.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:19.879 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:39:19.927 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:39:19.927 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:39:19.927 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:39:19.927 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-25T00:39:19.927 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:39:19.928 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:39:19.929 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:39:19.929 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-25T00:39:19.929 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-25T00:39:19.929 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-25T00:39:19.929 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:39:19.929 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:39:19.929 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-04-25T00:39:19.929 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-25T00:39:20.052 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.34907.asok 2024-04-25T00:39:20.056 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:39:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:39:20.063 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34907.asok 2024-04-25T00:39:20.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:20.063 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34907.asok status 2024-04-25T00:39:20.220 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:39:20.220 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-25T00:39:20.220 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:39:20.220 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:39:20.220 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-04-25T00:39:20.220 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-25T00:39:20.220 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T00:39:20.220 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "34907", 2024-04-25T00:39:20.220 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-25T00:39:20.220 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:39:20.220 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-25T00:39:20.221 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-25T00:39:20.221 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7543, 2024-04-25T00:39:20.221 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-25T00:39:20.221 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-25T00:39:20.221 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-25T00:39:20.221 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 7543 2024-04-25T00:39:20.221 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:39:20.221 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:39:20.221 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:39:20.221 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:39:20.221 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1506328759 2024-04-25T00:39:20.221 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-25T00:39:20.222 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:39:20.222 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:39:20.222 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:39:20.222 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:39:20.222 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1506328759 2024-04-25T00:39:20.222 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:39:20.222 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.7543 192.168.0.1:0/1506328759", 2024-04-25T00:39:20.222 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/1506328759", 2024-04-25T00:39:20.222 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-25T00:39:20.222 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 83, 2024-04-25T00:39:20.222 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 132, 2024-04-25T00:39:20.222 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:39:20.223 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-25T00:39:20.223 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-25T00:39:20.223 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:39:20.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:20.235 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-25T00:39:20.235 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:20.242 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-25T00:39:20.242 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:20.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:20.242 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:20.297 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-25T00:39:20.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f24815e1640 1 Processor -- start 2024-04-25T00:39:20.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f24815e1640 1 -- start start 2024-04-25T00:39:20.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f24815e1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f247c15e9a0 0x7f247c070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:20.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f24815e1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c0b4590 0x7f247c15ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:20.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f24815e1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f247c15f2c0 con 0x7f247c0cd1c0 2024-04-25T00:39:20.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f24815e1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f247c071ed0 con 0x7f247c15e9a0 2024-04-25T00:39:20.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f24815e1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f247c0712c0 con 0x7f247c0b4590 2024-04-25T00:39:20.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f247affd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f247c15e9a0 0x7f247c070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:20.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f247b7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f247c0cd1c0 0x7f247c072360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40472/0 (socket says 172.21.15.29:40472) 2024-04-25T00:39:20.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f247affd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f247c15e9a0 0x7f247c070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39774/0 (socket says 172.21.15.29:39774) 2024-04-25T00:39:20.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f247b7fe640 1 -- 172.21.15.29:0/830653492 learned_addr learned my addr 172.21.15.29:0/830653492 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:20.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f247a7fc640 1 --2- 172.21.15.29:0/830653492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c0b4590 0x7f247c15ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:20.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f247affd640 1 -- 172.21.15.29:0/830653492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c0b4590 msgr2=0x7f247c15ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:20.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f247affd640 1 --2- 172.21.15.29:0/830653492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c0b4590 0x7f247c15ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:20.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f247affd640 1 -- 172.21.15.29:0/830653492 >> v1:172.21.15.29:6789/0 conn(0x7f247c0cd1c0 legacy=0x7f247c072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:20.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.434+0000 7f247affd640 1 -- 172.21.15.29:0/830653492 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c0715a0 con 0x7f247c15e9a0 2024-04-25T00:39:20.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.438+0000 7f247affd640 1 --2- 172.21.15.29:0/830653492 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c15e9a0 0x7f247c070ce0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f246c0097c0 tx=0x7f246c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91f7e48e0bd4b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:20.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.438+0000 7f2479ffb640 1 -- 172.21.15.29:0/830653492 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0a7070 con 0x7f247c15e9a0 2024-04-25T00:39:20.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.438+0000 7f2479ffb640 1 -- 172.21.15.29:0/830653492 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246c0b0070 con 0x7f247c15e9a0 2024-04-25T00:39:20.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.438+0000 7f2479ffb640 1 -- 172.21.15.29:0/830653492 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0ab890 con 0x7f247c15e9a0 2024-04-25T00:39:20.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.438+0000 7f24815e1640 1 -- 172.21.15.29:0/830653492 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c15e9a0 msgr2=0x7f247c070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:20.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.438+0000 7f24815e1640 1 --2- 172.21.15.29:0/830653492 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c15e9a0 0x7f247c070ce0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f246c0097c0 tx=0x7f246c01c470 comp rx=0 tx=0).stop 2024-04-25T00:39:20.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.438+0000 7f24815e1640 1 -- 172.21.15.29:0/830653492 shutdown_connections 2024-04-25T00:39:20.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.438+0000 7f24815e1640 1 --2- 172.21.15.29:0/830653492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c0b4590 0x7f247c15ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:20.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.438+0000 7f24815e1640 1 --2- 172.21.15.29:0/830653492 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c15e9a0 0x7f247c070ce0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:20.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.438+0000 7f24815e1640 1 -- 172.21.15.29:0/830653492 >> 172.21.15.29:0/830653492 conn(0x7f247c0b1870 msgr2=0x7f247c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:20.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.438+0000 7f24815e1640 1 -- 172.21.15.29:0/830653492 shutdown_connections 2024-04-25T00:39:20.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.438+0000 7f24815e1640 1 -- 172.21.15.29:0/830653492 wait complete. 2024-04-25T00:39:20.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24815e1640 1 Processor -- start 2024-04-25T00:39:20.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24815e1640 1 -- start start 2024-04-25T00:39:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24815e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c0b4590 0x7f247c0aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24815e1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c0cd1c0 0x7f247c1436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24815e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c15e9a0 0x7f247c13e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24815e1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f247c144870 con 0x7f247c15e9a0 2024-04-25T00:39:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24815e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f247c072e80 con 0x7f247c0b4590 2024-04-25T00:39:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24815e1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f247c1449f0 con 0x7f247c0cd1c0 2024-04-25T00:39:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f247b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c15e9a0 0x7f247c13e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f247b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c15e9a0 0x7f247c13e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39790/0 (socket says 172.21.15.29:39790) 2024-04-25T00:39:20.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f247b7fe640 1 -- 172.21.15.29:0/3505166342 learned_addr learned my addr 172.21.15.29:0/3505166342 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:20.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f247affd640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c0b4590 0x7f247c0aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:20.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f247a7fc640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c0cd1c0 0x7f247c1436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:20.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f247b7fe640 1 -- 172.21.15.29:0/3505166342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c0cd1c0 msgr2=0x7f247c1436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:20.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f247b7fe640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c0cd1c0 0x7f247c1436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:20.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f247b7fe640 1 -- 172.21.15.29:0/3505166342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c0b4590 msgr2=0x7f247c0aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:20.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f247b7fe640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c0b4590 0x7f247c0aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:20.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f247b7fe640 1 -- 172.21.15.29:0/3505166342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c0aad90 con 0x7f247c15e9a0 2024-04-25T00:39:20.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f247b7fe640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c15e9a0 0x7f247c13e400 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f247400cbf0 tx=0x7f247400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:20.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24637fe640 1 -- 172.21.15.29:0/3505166342 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474017070 con 0x7f247c15e9a0 2024-04-25T00:39:20.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24637fe640 1 -- 172.21.15.29:0/3505166342 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247400ad40 con 0x7f247c15e9a0 2024-04-25T00:39:20.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24815e1640 1 -- 172.21.15.29:0/3505166342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247c0ab070 con 0x7f247c15e9a0 2024-04-25T00:39:20.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24637fe640 1 -- 172.21.15.29:0/3505166342 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474012600 con 0x7f247c15e9a0 2024-04-25T00:39:20.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24815e1640 1 -- 172.21.15.29:0/3505166342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247c0ab910 con 0x7f247c15e9a0 2024-04-25T00:39:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.442+0000 7f24815e1640 1 -- 172.21.15.29:0/3505166342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247c0ab240 con 0x7f247c15e9a0 2024-04-25T00:39:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.446+0000 7f24637fe640 1 -- 172.21.15.29:0/3505166342 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f247401a070 con 0x7f247c15e9a0 2024-04-25T00:39:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.446+0000 7f24637fe640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2468067e60 0x7f246806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.446+0000 7f247affd640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2468067e60 0x7f246806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.446+0000 7f24637fe640 1 -- 172.21.15.29:0/3505166342 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f24740863b0 con 0x7f247c15e9a0 2024-04-25T00:39:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.446+0000 7f247affd640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2468067e60 0x7f246806a320 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f246c0af000 tx=0x7f246c0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:20.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.450+0000 7f24637fe640 1 -- 172.21.15.29:0/3505166342 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2474056040 con 0x7f247c15e9a0 2024-04-25T00:39:20.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:20.650+0000 7f24815e1640 1 -- 172.21.15.29:0/3505166342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f247c0692e0 con 0x7f247c15e9a0 2024-04-25T00:39:21.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.446+0000 7f24637fe640 1 -- 172.21.15.29:0/3505166342 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v84) ==== 136+0+0 (secure 0 0 0) 0x7f247405a340 con 0x7f247c15e9a0 2024-04-25T00:39:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.446+0000 7f24815e1640 1 -- 172.21.15.29:0/3505166342 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2468067e60 msgr2=0x7f246806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.446+0000 7f24815e1640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2468067e60 0x7f246806a320 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f246c0af000 tx=0x7f246c0a7000 comp rx=0 tx=0).stop 2024-04-25T00:39:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.446+0000 7f24815e1640 1 -- 172.21.15.29:0/3505166342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c15e9a0 msgr2=0x7f247c13e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.446+0000 7f24815e1640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c15e9a0 0x7f247c13e400 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f247400cbf0 tx=0x7f247400a430 comp rx=0 tx=0).stop 2024-04-25T00:39:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.446+0000 7f24815e1640 1 -- 172.21.15.29:0/3505166342 shutdown_connections 2024-04-25T00:39:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.446+0000 7f24815e1640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2468067e60 0x7f246806a320 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.446+0000 7f24815e1640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c15e9a0 0x7f247c13e400 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.446+0000 7f24815e1640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c0cd1c0 0x7f247c1436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.446+0000 7f24815e1640 1 --2- 172.21.15.29:0/3505166342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c0b4590 0x7f247c0aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.446+0000 7f24815e1640 1 -- 172.21.15.29:0/3505166342 >> 172.21.15.29:0/3505166342 conn(0x7f247c0b1870 msgr2=0x7f247c142510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.446+0000 7f24815e1640 1 -- 172.21.15.29:0/3505166342 shutdown_connections 2024-04-25T00:39:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.446+0000 7f24815e1640 1 -- 172.21.15.29:0/3505166342 wait complete. 2024-04-25T00:39:21.471 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-25T00:39:21.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.590+0000 7f5c45a26640 1 Processor -- start 2024-04-25T00:39:21.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.590+0000 7f5c45a26640 1 -- start start 2024-04-25T00:39:21.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.590+0000 7f5c45a26640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c4015e200 0x7f5c400d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.590+0000 7f5c45a26640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c400d2c60 0x7f5c400b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.590+0000 7f5c45a26640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c400cfc10 con 0x7f5c400cf690 2024-04-25T00:39:21.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.590+0000 7f5c45a26640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5c400b38d0 con 0x7f5c4015e200 2024-04-25T00:39:21.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.590+0000 7f5c45a26640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5c400cfef0 con 0x7f5c400d2c60 2024-04-25T00:39:21.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.590+0000 7f5c3e7fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5c400cf690 0x7f5c400d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40484/0 (socket says 172.21.15.29:40484) 2024-04-25T00:39:21.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.590+0000 7f5c3e7fc640 1 -- 172.21.15.29:0/2416234384 learned_addr learned my addr 172.21.15.29:0/2416234384 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:21.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.590+0000 7f5c3effd640 1 --2- 172.21.15.29:0/2416234384 >> v2:172.21.15.157:3300/0 conn(0x7f5c4015e200 0x7f5c400d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.590+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2352115910 0 0) 0x7f5c400cfc10 con 0x7f5c400cf690 2024-04-25T00:39:21.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.590+0000 7f5c3f7fe640 1 --2- 172.21.15.29:0/2416234384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c400d2c60 0x7f5c400b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.590+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c2c003530 con 0x7f5c400cf690 2024-04-25T00:39:21.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 785853892 0 0) 0x7f5c2c003530 con 0x7f5c400cf690 2024-04-25T00:39:21.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c400d2c60 msgr2=0x7f5c400b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 --2- 172.21.15.29:0/2416234384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c400d2c60 0x7f5c400b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 >> v2:172.21.15.157:3300/0 conn(0x7f5c4015e200 msgr2=0x7f5c400d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 --2- 172.21.15.29:0/2416234384 >> v2:172.21.15.157:3300/0 conn(0x7f5c4015e200 0x7f5c400d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c400b3b20 con 0x7f5c400cf690 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f5c280016a0 con 0x7f5c400cf690 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 >> v1:172.21.15.29:6789/0 conn(0x7f5c400cf690 legacy=0x7f5c400d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 --2- 172.21.15.29:0/2416234384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c2c0039c0 0x7f5c2c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 --2- 172.21.15.29:0/2416234384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c2c004810 0x7f5c2c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 --2- 172.21.15.29:0/2416234384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c400d2c60 0x7f5c2c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5c400b38d0 con 0x7f5c2c004810 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5c400cfef0 con 0x7f5c2c0039c0 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5c2c00b9d0 con 0x7f5c400d2c60 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3e7fc640 1 --2- 172.21.15.29:0/2416234384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c2c004810 0x7f5c2c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3effd640 1 --2- 172.21.15.29:0/2416234384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c2c0039c0 0x7f5c2c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3f7fe640 1 --2- 172.21.15.29:0/2416234384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c400d2c60 0x7f5c2c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3e7fc640 1 -- 172.21.15.29:0/2416234384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c400d2c60 msgr2=0x7f5c2c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:21.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3e7fc640 1 --2- 172.21.15.29:0/2416234384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c400d2c60 0x7f5c2c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3e7fc640 1 -- 172.21.15.29:0/2416234384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c2c0039c0 msgr2=0x7f5c2c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:21.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3e7fc640 1 --2- 172.21.15.29:0/2416234384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c2c0039c0 0x7f5c2c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3e7fc640 1 -- 172.21.15.29:0/2416234384 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c003530 con 0x7f5c2c004810 2024-04-25T00:39:21.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3e7fc640 1 --2- 172.21.15.29:0/2416234384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c2c004810 0x7f5c2c004c90 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800c210 tx=0x7f5c2801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2507a3191cab9982 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:21.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28007d80 con 0x7f5c2c004810 2024-04-25T00:39:21.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c28005b00 con 0x7f5c2c004810 2024-04-25T00:39:21.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3dffb640 1 -- 172.21.15.29:0/2416234384 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28007070 con 0x7f5c2c004810 2024-04-25T00:39:21.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c45a26640 1 -- 172.21.15.29:0/2416234384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c2c004810 msgr2=0x7f5c2c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:21.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c45a26640 1 --2- 172.21.15.29:0/2416234384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c2c004810 0x7f5c2c004c90 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800c210 tx=0x7f5c2801f3f0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c3f7fe640 1 -- 172.21.15.29:0/2416234384 reap_dead start 2024-04-25T00:39:21.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c45a26640 1 -- 172.21.15.29:0/2416234384 shutdown_connections 2024-04-25T00:39:21.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c45a26640 1 -- 172.21.15.29:0/2416234384 >> 172.21.15.29:0/2416234384 conn(0x7f5c4009d280 msgr2=0x7f5c400d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:21.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c45a26640 1 -- 172.21.15.29:0/2416234384 shutdown_connections 2024-04-25T00:39:21.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.594+0000 7f5c45a26640 1 -- 172.21.15.29:0/2416234384 wait complete. 2024-04-25T00:39:21.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c45a26640 1 Processor -- start 2024-04-25T00:39:21.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c45a26640 1 -- start start 2024-04-25T00:39:21.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c45a26640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c2c004810 0x7f5c400c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c45a26640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c40151440 0x7f5c400d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c45a26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c400d4290 0x7f5c400b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c45a26640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4015e5e0 con 0x7f5c2c004810 2024-04-25T00:39:21.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c45a26640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5c400d4d30 con 0x7f5c400d4290 2024-04-25T00:39:21.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c45a26640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5c4015e760 con 0x7f5c40151440 2024-04-25T00:39:21.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c3effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c2c004810 0x7f5c400c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c3effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c2c004810 0x7f5c400c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39814/0 (socket says 172.21.15.29:39814) 2024-04-25T00:39:21.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c3effd640 1 -- 172.21.15.29:0/1379893822 learned_addr learned my addr 172.21.15.29:0/1379893822 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:21.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c3e7fc640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c40151440 0x7f5c400d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c3f7fe640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c400d4290 0x7f5c400b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c3effd640 1 -- 172.21.15.29:0/1379893822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c40151440 msgr2=0x7f5c400d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:21.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c3effd640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c40151440 0x7f5c400d3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c3effd640 1 -- 172.21.15.29:0/1379893822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c400d4290 msgr2=0x7f5c400b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c3effd640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c400d4290 0x7f5c400b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c3effd640 1 -- 172.21.15.29:0/1379893822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c400ca3d0 con 0x7f5c2c004810 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c3effd640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c2c004810 0x7f5c400c7aa0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f5c380115e0 tx=0x7f5c38011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c37fff640 1 -- 172.21.15.29:0/1379893822 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800f8f0 con 0x7f5c2c004810 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c37fff640 1 -- 172.21.15.29:0/1379893822 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c3800fa90 con 0x7f5c2c004810 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c45a26640 1 -- 172.21.15.29:0/1379893822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c400ca6b0 con 0x7f5c2c004810 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c37fff640 1 -- 172.21.15.29:0/1379893822 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800fe40 con 0x7f5c2c004810 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c45a26640 1 -- 172.21.15.29:0/1379893822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c400b6f30 con 0x7f5c2c004810 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.598+0000 7f5c45a26640 1 -- 172.21.15.29:0/1379893822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c40069270 con 0x7f5c2c004810 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.602+0000 7f5c37fff640 1 -- 172.21.15.29:0/1379893822 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f5c38015460 con 0x7f5c2c004810 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.602+0000 7f5c37fff640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5c20067e90 0x7f5c2006a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.602+0000 7f5c37fff640 1 -- 172.21.15.29:0/1379893822 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f5c380870f0 con 0x7f5c2c004810 2024-04-25T00:39:21.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.602+0000 7f5c3e7fc640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5c20067e90 0x7f5c2006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.602+0000 7f5c3e7fc640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5c20067e90 0x7f5c2006a350 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5c280ae000 tx=0x7f5c280b4040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:21.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.606+0000 7f5c37fff640 1 -- 172.21.15.29:0/1379893822 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5c38019030 con 0x7f5c2c004810 2024-04-25T00:39:21.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.782+0000 7f5c45a26640 1 -- 172.21.15.29:0/1379893822 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f5c400c7820 con 0x7f5c20067e90 2024-04-25T00:39:21.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.782+0000 7f5c37fff640 1 -- 172.21.15.29:0/1379893822 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f5c400c7820 con 0x7f5c20067e90 2024-04-25T00:39:21.787 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:39:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-04-25T00:39:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:21.787 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:39:21.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.786+0000 7f5c45a26640 1 -- 172.21.15.29:0/1379893822 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5c20067e90 msgr2=0x7f5c2006a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.786+0000 7f5c45a26640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5c20067e90 0x7f5c2006a350 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5c280ae000 tx=0x7f5c280b4040 comp rx=0 tx=0).stop 2024-04-25T00:39:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.786+0000 7f5c45a26640 1 -- 172.21.15.29:0/1379893822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c2c004810 msgr2=0x7f5c400c7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.786+0000 7f5c45a26640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c2c004810 0x7f5c400c7aa0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f5c380115e0 tx=0x7f5c38011aa0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.786+0000 7f5c45a26640 1 -- 172.21.15.29:0/1379893822 shutdown_connections 2024-04-25T00:39:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.786+0000 7f5c45a26640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5c20067e90 0x7f5c2006a350 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.786+0000 7f5c45a26640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c400d4290 0x7f5c400b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.786+0000 7f5c45a26640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c40151440 0x7f5c400d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.786+0000 7f5c45a26640 1 --2- 172.21.15.29:0/1379893822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c2c004810 0x7f5c400c7aa0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.786+0000 7f5c45a26640 1 -- 172.21.15.29:0/1379893822 >> 172.21.15.29:0/1379893822 conn(0x7f5c4009d280 msgr2=0x7f5c4006c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.786+0000 7f5c45a26640 1 -- 172.21.15.29:0/1379893822 shutdown_connections 2024-04-25T00:39:21.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.786+0000 7f5c45a26640 1 -- 172.21.15.29:0/1379893822 wait complete. 2024-04-25T00:39:21.805 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:39:21.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e9372640 1 Processor -- start 2024-04-25T00:39:21.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e9372640 1 -- start start 2024-04-25T00:39:21.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e9372640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa5e40b3e40 0x7fa5e406d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e9372640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5e406d980 con 0x7fa5e415df70 2024-04-25T00:39:21.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e9372640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5e406dc60 con 0x7fa5e40b4d30 2024-04-25T00:39:21.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e9372640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e406d750 con 0x7fa5e40b3e40 2024-04-25T00:39:21.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e2ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa5e415df70 0x7fa5e40b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40498/0 (socket says 172.21.15.29:40498) 2024-04-25T00:39:21.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e2ffd640 1 -- 172.21.15.29:0/935977369 learned_addr learned my addr 172.21.15.29:0/935977369 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:21.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e37fe640 1 --2- 172.21.15.29:0/935977369 >> v2:172.21.15.29:3300/0 conn(0x7fa5e40b3e40 0x7fa5e406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e1ffb640 1 -- 172.21.15.29:0/935977369 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423987928 0 0) 0x7fa5e406d980 con 0x7fa5e415df70 2024-04-25T00:39:21.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e1ffb640 1 -- 172.21.15.29:0/935977369 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5cc003530 con 0x7fa5e415df70 2024-04-25T00:39:21.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e37fe640 1 -- 172.21.15.29:0/935977369 >> v1:172.21.15.157:6789/0 conn(0x7fa5e40b4d30 legacy=0x7fa5e40ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:21.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e37fe640 1 -- 172.21.15.29:0/935977369 >> v1:172.21.15.29:6789/0 conn(0x7fa5e415df70 legacy=0x7fa5e40b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:21.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.930+0000 7fa5e37fe640 1 -- 172.21.15.29:0/935977369 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e406df40 con 0x7fa5e40b3e40 2024-04-25T00:39:21.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e37fe640 1 --2- 172.21.15.29:0/935977369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e40b3e40 0x7fa5e406d210 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009d90 tx=0x7fa5dc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2038f454757e40a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:21.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e1ffb640 1 -- 172.21.15.29:0/935977369 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0b2070 con 0x7fa5e40b3e40 2024-04-25T00:39:21.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e1ffb640 1 -- 172.21.15.29:0/935977369 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5dc0a5a00 con 0x7fa5e40b3e40 2024-04-25T00:39:21.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e1ffb640 1 -- 172.21.15.29:0/935977369 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0a5dd0 con 0x7fa5e40b3e40 2024-04-25T00:39:21.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 -- 172.21.15.29:0/935977369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e40b3e40 msgr2=0x7fa5e406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:21.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 --2- 172.21.15.29:0/935977369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e40b3e40 0x7fa5e406d210 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009d90 tx=0x7fa5dc01c940 comp rx=0 tx=0).stop 2024-04-25T00:39:21.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 -- 172.21.15.29:0/935977369 shutdown_connections 2024-04-25T00:39:21.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 --2- 172.21.15.29:0/935977369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e40b3e40 0x7fa5e406d210 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 -- 172.21.15.29:0/935977369 >> 172.21.15.29:0/935977369 conn(0x7fa5e40b1870 msgr2=0x7fa5e40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:21.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 -- 172.21.15.29:0/935977369 shutdown_connections 2024-04-25T00:39:21.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 -- 172.21.15.29:0/935977369 wait complete. 2024-04-25T00:39:21.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 Processor -- start 2024-04-25T00:39:21.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 -- start start 2024-04-25T00:39:21.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5e40b3e40 0x7fa5e4141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e40b4d30 0x7fa5e40cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e415df70 0x7fa5e40ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e406eba0 con 0x7fa5e40b4d30 2024-04-25T00:39:21.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e406ea20 con 0x7fa5e415df70 2024-04-25T00:39:21.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.934+0000 7fa5e9372640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e416bbc0 con 0x7fa5e40b3e40 2024-04-25T00:39:21.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e40b4d30 0x7fa5e40cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e40b4d30 0x7fa5e40cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39822/0 (socket says 172.21.15.29:39822) 2024-04-25T00:39:21.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e27fc640 1 -- 172.21.15.29:0/2011918537 learned_addr learned my addr 172.21.15.29:0/2011918537 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:21.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e2ffd640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5e40b3e40 0x7fa5e4141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e37fe640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e415df70 0x7fa5e40ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e27fc640 1 -- 172.21.15.29:0/2011918537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5e40b3e40 msgr2=0x7fa5e4141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:21.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e27fc640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5e40b3e40 0x7fa5e4141210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e27fc640 1 -- 172.21.15.29:0/2011918537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e415df70 msgr2=0x7fa5e40ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:21.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e27fc640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e415df70 0x7fa5e40ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:21.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e27fc640 1 -- 172.21.15.29:0/2011918537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e4141750 con 0x7fa5e40b4d30 2024-04-25T00:39:21.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e27fc640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e40b4d30 0x7fa5e40cdd90 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800c420 tx=0x7fa5d800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:21.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5cb7fe640 1 -- 172.21.15.29:0/2011918537 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d80052d0 con 0x7fa5e40b4d30 2024-04-25T00:39:21.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5cb7fe640 1 -- 172.21.15.29:0/2011918537 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d8005470 con 0x7fa5e40b4d30 2024-04-25T00:39:21.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e9372640 1 -- 172.21.15.29:0/2011918537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e40cae10 con 0x7fa5e40b4d30 2024-04-25T00:39:21.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5cb7fe640 1 -- 172.21.15.29:0/2011918537 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8007260 con 0x7fa5e40b4d30 2024-04-25T00:39:21.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e9372640 1 -- 172.21.15.29:0/2011918537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e40cb3d0 con 0x7fa5e40b4d30 2024-04-25T00:39:21.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e9372640 1 -- 172.21.15.29:0/2011918537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e4069270 con 0x7fa5e40b4d30 2024-04-25T00:39:21.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5cb7fe640 1 -- 172.21.15.29:0/2011918537 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fa5d8007400 con 0x7fa5e40b4d30 2024-04-25T00:39:21.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5cb7fe640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5d0067e60 0x7fa5d006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:21.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5cb7fe640 1 -- 172.21.15.29:0/2011918537 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7fa5d8085bb0 con 0x7fa5e40b4d30 2024-04-25T00:39:21.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e2ffd640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5d0067e60 0x7fa5d006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:21.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.938+0000 7fa5e2ffd640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5d0067e60 0x7fa5d006a320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40011f0 tx=0x7fa5d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:21.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:21.942+0000 7fa5cb7fe640 1 -- 172.21.15.29:0/2011918537 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa5d8055ed0 con 0x7fa5e40b4d30 2024-04-25T00:39:22.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.106+0000 7fa5e9372640 1 -- 172.21.15.29:0/2011918537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa5e40b4870 con 0x7fa5e40b4d30 2024-04-25T00:39:22.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.106+0000 7fa5cb7fe640 1 -- 172.21.15.29:0/2011918537 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7fa5d80557d0 con 0x7fa5e40b4d30 2024-04-25T00:39:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.110+0000 7fa5e9372640 1 -- 172.21.15.29:0/2011918537 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5d0067e60 msgr2=0x7fa5d006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.110+0000 7fa5e9372640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5d0067e60 0x7fa5d006a320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40011f0 tx=0x7fa5d4009040 comp rx=0 tx=0).stop 2024-04-25T00:39:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.110+0000 7fa5e9372640 1 -- 172.21.15.29:0/2011918537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e40b4d30 msgr2=0x7fa5e40cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.110+0000 7fa5e9372640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e40b4d30 0x7fa5e40cdd90 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800c420 tx=0x7fa5d800c8e0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.110+0000 7fa5e9372640 1 -- 172.21.15.29:0/2011918537 shutdown_connections 2024-04-25T00:39:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.110+0000 7fa5e9372640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5d0067e60 0x7fa5d006a320 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.110+0000 7fa5e9372640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e415df70 0x7fa5e40ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.110+0000 7fa5e9372640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e40b4d30 0x7fa5e40cdd90 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.110+0000 7fa5e9372640 1 --2- 172.21.15.29:0/2011918537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5e40b3e40 0x7fa5e4141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.110+0000 7fa5e9372640 1 -- 172.21.15.29:0/2011918537 >> 172.21.15.29:0/2011918537 conn(0x7fa5e40b1870 msgr2=0x7fa5e41593c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.110+0000 7fa5e9372640 1 -- 172.21.15.29:0/2011918537 shutdown_connections 2024-04-25T00:39:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.110+0000 7fa5e9372640 1 -- 172.21.15.29:0/2011918537 wait complete. 2024-04-25T00:39:22.129 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2024-04-25T00:39:22.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd973b85640 1 Processor -- start 2024-04-25T00:39:22.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd973b85640 1 -- start start 2024-04-25T00:39:22.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd973b85640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd96c0c4ce0 0x7fd96c0c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd973b85640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd96c0cc640 0x7fd96c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd973b85640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd96c1663e0 con 0x7fd96c0b2200 2024-04-25T00:39:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd973b85640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd96c0c5680 con 0x7fd96c0cc640 2024-04-25T00:39:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd973b85640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd96c0b25e0 con 0x7fd96c0c4ce0 2024-04-25T00:39:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd971921640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd96c0b2200 0x7fd96c06c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40514/0 (socket says 172.21.15.29:40514) 2024-04-25T00:39:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd971921640 1 -- 172.21.15.29:0/150077292 learned_addr learned my addr 172.21.15.29:0/150077292 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd972122640 1 --2- 172.21.15.29:0/150077292 >> v2:172.21.15.157:3300/0 conn(0x7fd96c0cc640 0x7fd96c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd971120640 1 --2- 172.21.15.29:0/150077292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd96c0c4ce0 0x7fd96c0c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 703178770 0 0) 0x7fd96c1663e0 con 0x7fd96c0b2200 2024-04-25T00:39:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd954003530 con 0x7fd96c0b2200 2024-04-25T00:39:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2115848486 0 0) 0x7fd954003530 con 0x7fd96c0b2200 2024-04-25T00:39:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd96c0c4ce0 msgr2=0x7fd96c0c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 --2- 172.21.15.29:0/150077292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd96c0c4ce0 0x7fd96c0c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 >> v2:172.21.15.157:3300/0 conn(0x7fd96c0cc640 msgr2=0x7fd96c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 --2- 172.21.15.29:0/150077292 >> v2:172.21.15.157:3300/0 conn(0x7fd96c0cc640 0x7fd96c0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c1666c0 con 0x7fd96c0b2200 2024-04-25T00:39:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd964001a00 con 0x7fd96c0b2200 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 >> v1:172.21.15.29:6789/0 conn(0x7fd96c0b2200 legacy=0x7fd96c06c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 --2- 172.21.15.29:0/150077292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9540039c0 0x7fd954003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 --2- 172.21.15.29:0/150077292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd96c0c4ce0 0x7fd954004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 --2- 172.21.15.29:0/150077292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd954004d50 0x7fd9540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c0b25e0 con 0x7fd9540039c0 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd95400b9d0 con 0x7fd954004d50 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd95400bb50 con 0x7fd96c0c4ce0 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd971921640 1 --2- 172.21.15.29:0/150077292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9540039c0 0x7fd954003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd972122640 1 --2- 172.21.15.29:0/150077292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd954004d50 0x7fd9540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd971120640 1 --2- 172.21.15.29:0/150077292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd96c0c4ce0 0x7fd954004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd971921640 1 -- 172.21.15.29:0/150077292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd96c0c4ce0 msgr2=0x7fd954004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd971921640 1 --2- 172.21.15.29:0/150077292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd96c0c4ce0 0x7fd954004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd971921640 1 -- 172.21.15.29:0/150077292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd954004d50 msgr2=0x7fd9540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:22.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd971921640 1 --2- 172.21.15.29:0/150077292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd954004d50 0x7fd9540051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.254+0000 7fd971921640 1 -- 172.21.15.29:0/150077292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd954003530 con 0x7fd9540039c0 2024-04-25T00:39:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.258+0000 7fd971921640 1 --2- 172.21.15.29:0/150077292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9540039c0 0x7fd954003de0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fd96400a890 tx=0x7fd96401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7848ede816d440b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.258+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96400a480 con 0x7fd9540039c0 2024-04-25T00:39:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.258+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9640a5e50 con 0x7fd9540039c0 2024-04-25T00:39:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.258+0000 7fd97091f640 1 -- 172.21.15.29:0/150077292 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640ad8b0 con 0x7fd9540039c0 2024-04-25T00:39:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.258+0000 7fd973b85640 1 -- 172.21.15.29:0/150077292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9540039c0 msgr2=0x7fd954003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.258+0000 7fd973b85640 1 --2- 172.21.15.29:0/150077292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9540039c0 0x7fd954003de0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fd96400a890 tx=0x7fd96401d380 comp rx=0 tx=0).stop 2024-04-25T00:39:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.258+0000 7fd972122640 1 -- 172.21.15.29:0/150077292 reap_dead start 2024-04-25T00:39:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.258+0000 7fd973b85640 1 -- 172.21.15.29:0/150077292 shutdown_connections 2024-04-25T00:39:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.258+0000 7fd973b85640 1 -- 172.21.15.29:0/150077292 >> 172.21.15.29:0/150077292 conn(0x7fd96c09aa30 msgr2=0x7fd96c09ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.258+0000 7fd973b85640 1 -- 172.21.15.29:0/150077292 shutdown_connections 2024-04-25T00:39:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.258+0000 7fd973b85640 1 -- 172.21.15.29:0/150077292 wait complete. 2024-04-25T00:39:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd973b85640 1 Processor -- start 2024-04-25T00:39:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd973b85640 1 -- start start 2024-04-25T00:39:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd973b85640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9540039c0 0x7fd96c152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd973b85640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd96c15d350 0x7fd96c1619e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd973b85640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd96c161f20 0x7fd96c0c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd973b85640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c06ca30 con 0x7fd9540039c0 2024-04-25T00:39:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd973b85640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c1660b0 con 0x7fd96c15d350 2024-04-25T00:39:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd973b85640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd96c06cbb0 con 0x7fd96c161f20 2024-04-25T00:39:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd971921640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9540039c0 0x7fd96c152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd971921640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9540039c0 0x7fd96c152210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39836/0 (socket says 172.21.15.29:39836) 2024-04-25T00:39:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd971921640 1 -- 172.21.15.29:0/3421592693 learned_addr learned my addr 172.21.15.29:0/3421592693 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd971120640 1 --2- 172.21.15.29:0/3421592693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd96c15d350 0x7fd96c1619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd972122640 1 --2- 172.21.15.29:0/3421592693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd96c161f20 0x7fd96c0c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd971921640 1 -- 172.21.15.29:0/3421592693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd96c161f20 msgr2=0x7fd96c0c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:22.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd971921640 1 --2- 172.21.15.29:0/3421592693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd96c161f20 0x7fd96c0c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd971921640 1 -- 172.21.15.29:0/3421592693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd96c15d350 msgr2=0x7fd96c1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:22.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd971921640 1 --2- 172.21.15.29:0/3421592693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd96c15d350 0x7fd96c1619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd971921640 1 -- 172.21.15.29:0/3421592693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c070000 con 0x7fd9540039c0 2024-04-25T00:39:22.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd971921640 1 --2- 172.21.15.29:0/3421592693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9540039c0 0x7fd96c152210 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fd96400a7b0 tx=0x7fd954005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:22.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd96a7fc640 1 -- 172.21.15.29:0/3421592693 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640ad430 con 0x7fd9540039c0 2024-04-25T00:39:22.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd96a7fc640 1 -- 172.21.15.29:0/3421592693 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9640bc070 con 0x7fd9540039c0 2024-04-25T00:39:22.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd973b85640 1 -- 172.21.15.29:0/3421592693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96c0702e0 con 0x7fd9540039c0 2024-04-25T00:39:22.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd96a7fc640 1 -- 172.21.15.29:0/3421592693 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640017c0 con 0x7fd9540039c0 2024-04-25T00:39:22.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd973b85640 1 -- 172.21.15.29:0/3421592693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96c0708a0 con 0x7fd9540039c0 2024-04-25T00:39:22.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd973b85640 1 -- 172.21.15.29:0/3421592693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd96c069270 con 0x7fd9540039c0 2024-04-25T00:39:22.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd96a7fc640 1 -- 172.21.15.29:0/3421592693 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fd9640b5070 con 0x7fd9540039c0 2024-04-25T00:39:22.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd96a7fc640 1 --2- 172.21.15.29:0/3421592693 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd958067e90 0x7fd95806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd96a7fc640 1 -- 172.21.15.29:0/3421592693 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7fd964120c50 con 0x7fd9540039c0 2024-04-25T00:39:22.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.262+0000 7fd971120640 1 --2- 172.21.15.29:0/3421592693 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd958067e90 0x7fd95806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.266+0000 7fd971120640 1 --2- 172.21.15.29:0/3421592693 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd958067e90 0x7fd95806a350 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd96c1608d0 tx=0x7fd960008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:22.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.270+0000 7fd96a7fc640 1 -- 172.21.15.29:0/3421592693 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd964017090 con 0x7fd9540039c0 2024-04-25T00:39:22.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.450+0000 7fd973b85640 1 -- 172.21.15.29:0/3421592693 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7fd96c0ced80 con 0x7fd958067e90 2024-04-25T00:39:22.454 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:22.450+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:22.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.450+0000 7fd96a7fc640 1 -- 172.21.15.29:0/3421592693 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fd96c0ced80 con 0x7fd958067e90 2024-04-25T00:39:22.455 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:22.460 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:22.460 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2024-04-25T00:39:22.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90ea042640 1 Processor -- start 2024-04-25T00:39:22.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90ea042640 1 -- start start 2024-04-25T00:39:22.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90ea042640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90e40b3c30 0x7f90e406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90ea042640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f90e40b4b10 0x7f90e40ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90ea042640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90e40b41e0 con 0x7f90e40b3140 2024-04-25T00:39:22.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90ea042640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f90e406d960 con 0x7f90e40b4b10 2024-04-25T00:39:22.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90ea042640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f90e406dc40 con 0x7f90e40b3c30 2024-04-25T00:39:22.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90e3fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f90e40b4b10 0x7f90e40ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90e3fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f90e40b4b10 0x7f90e40ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39842/0 (socket says 172.21.15.29:39842) 2024-04-25T00:39:22.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90e3fff640 1 -- 172.21.15.29:0/3274792221 learned_addr learned my addr 172.21.15.29:0/3274792221 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:22.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90daffd640 1 --2- 172.21.15.29:0/3274792221 >> v2:172.21.15.157:3300/0 conn(0x7f90e40b3c30 0x7f90e406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90e3fff640 1 -- 172.21.15.29:0/3274792221 >> v2:172.21.15.157:3300/0 conn(0x7f90e40b3c30 msgr2=0x7f90e406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:22.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90e3fff640 1 --2- 172.21.15.29:0/3274792221 >> v2:172.21.15.157:3300/0 conn(0x7f90e40b3c30 0x7f90e406e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90e3fff640 1 -- 172.21.15.29:0/3274792221 >> v1:172.21.15.29:6789/0 conn(0x7f90e40b3140 legacy=0x7f90e40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:22.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.570+0000 7f90e3fff640 1 -- 172.21.15.29:0/3274792221 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e406df20 con 0x7f90e40b4b10 2024-04-25T00:39:22.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90e3fff640 1 --2- 172.21.15.29:0/3274792221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e40b4b10 0x7f90e40ce240 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009e50 tx=0x7f90dc01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49d1600b7adf7cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:22.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90e2ffd640 1 -- 172.21.15.29:0/3274792221 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0a6510 con 0x7f90e40b4b10 2024-04-25T00:39:22.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90e2ffd640 1 -- 172.21.15.29:0/3274792221 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90dc0a66b0 con 0x7f90e40b4b10 2024-04-25T00:39:22.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90e2ffd640 1 -- 172.21.15.29:0/3274792221 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0a6a80 con 0x7f90e40b4b10 2024-04-25T00:39:22.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 -- 172.21.15.29:0/3274792221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e40b4b10 msgr2=0x7f90e40ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:22.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 --2- 172.21.15.29:0/3274792221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e40b4b10 0x7f90e40ce240 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009e50 tx=0x7f90dc01c880 comp rx=0 tx=0).stop 2024-04-25T00:39:22.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 -- 172.21.15.29:0/3274792221 shutdown_connections 2024-04-25T00:39:22.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 --2- 172.21.15.29:0/3274792221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e40b4b10 0x7f90e40ce240 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 --2- 172.21.15.29:0/3274792221 >> v2:172.21.15.157:3300/0 conn(0x7f90e40b3c30 0x7f90e406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 -- 172.21.15.29:0/3274792221 >> 172.21.15.29:0/3274792221 conn(0x7f90e40b1870 msgr2=0x7f90e40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:22.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 -- 172.21.15.29:0/3274792221 shutdown_connections 2024-04-25T00:39:22.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 -- 172.21.15.29:0/3274792221 wait complete. 2024-04-25T00:39:22.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 Processor -- start 2024-04-25T00:39:22.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 -- start start 2024-04-25T00:39:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e40b3140 0x7f90e4139210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e40b3c30 0x7f90e4154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e40b4b10 0x7f90e413d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f90e4168d80 con 0x7f90e40b3140 2024-04-25T00:39:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90e4168c00 con 0x7f90e40b3c30 2024-04-25T00:39:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90ea042640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f90e4168f00 con 0x7f90e40b4b10 2024-04-25T00:39:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.574+0000 7f90e37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e40b3140 0x7f90e4139210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e40b3140 0x7f90e4139210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39850/0 (socket says 172.21.15.29:39850) 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e37fe640 1 -- 172.21.15.29:0/3729881732 learned_addr learned my addr 172.21.15.29:0/3729881732 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90daffd640 1 --2- 172.21.15.29:0/3729881732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e40b3c30 0x7f90e4154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e3fff640 1 --2- 172.21.15.29:0/3729881732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e40b4b10 0x7f90e413d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e37fe640 1 -- 172.21.15.29:0/3729881732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e40b4b10 msgr2=0x7f90e413d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e37fe640 1 --2- 172.21.15.29:0/3729881732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e40b4b10 0x7f90e413d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e37fe640 1 -- 172.21.15.29:0/3729881732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e40b3c30 msgr2=0x7f90e4154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e37fe640 1 --2- 172.21.15.29:0/3729881732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e40b3c30 0x7f90e4154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e37fe640 1 -- 172.21.15.29:0/3729881732 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e4140290 con 0x7f90e40b3140 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90daffd640 1 --2- 172.21.15.29:0/3729881732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e40b3c30 0x7f90e4154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e37fe640 1 --2- 172.21.15.29:0/3729881732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e40b3140 0x7f90e4139210 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00c800 tx=0x7f90cc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e0ff9640 1 -- 172.21.15.29:0/3729881732 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc017070 con 0x7f90e40b3140 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e0ff9640 1 -- 172.21.15.29:0/3729881732 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90cc005290 con 0x7f90e40b3140 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e0ff9640 1 -- 172.21.15.29:0/3729881732 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc00a640 con 0x7f90e40b3140 2024-04-25T00:39:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90ea042640 1 -- 172.21.15.29:0/3729881732 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e41590d0 con 0x7f90e40b3140 2024-04-25T00:39:22.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90ea042640 1 -- 172.21.15.29:0/3729881732 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e4159370 con 0x7f90e40b3140 2024-04-25T00:39:22.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90ea042640 1 -- 172.21.15.29:0/3729881732 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e4069270 con 0x7f90e40b3140 2024-04-25T00:39:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e0ff9640 1 -- 172.21.15.29:0/3729881732 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f90cc01a070 con 0x7f90e40b3140 2024-04-25T00:39:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e0ff9640 1 --2- 172.21.15.29:0/3729881732 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90c4067e60 0x7f90c406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90e0ff9640 1 -- 172.21.15.29:0/3729881732 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f90cc085c40 con 0x7f90e40b3140 2024-04-25T00:39:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90daffd640 1 --2- 172.21.15.29:0/3729881732 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90c4067e60 0x7f90c406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.578+0000 7f90daffd640 1 --2- 172.21.15.29:0/3729881732 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90c4067e60 0x7f90c406a320 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f90e413cf10 tx=0x7f90d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:22.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.582+0000 7f90e0ff9640 1 -- 172.21.15.29:0/3729881732 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f90cc055f10 con 0x7f90e40b3140 2024-04-25T00:39:22.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.762+0000 7f90ea042640 1 -- 172.21.15.29:0/3729881732 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f90e4159d30 con 0x7f90c4067e60 2024-04-25T00:39:22.769 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:22.766+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:22.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.766+0000 7f90e0ff9640 1 -- 172.21.15.29:0/3729881732 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f90e4159d30 con 0x7f90c4067e60 2024-04-25T00:39:22.770 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:22.775 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:22.775 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2024-04-25T00:39:22.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b895f640 1 Processor -- start 2024-04-25T00:39:22.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b895f640 1 -- start start 2024-04-25T00:39:22.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b895f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93b415d980 0x7f93b406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b895f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f93b406db10 0x7f93b4151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b895f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93b40b3130 con 0x7f93b40b2b70 2024-04-25T00:39:22.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b895f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f93b40cf560 con 0x7f93b406db10 2024-04-25T00:39:22.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b895f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f93b40b3410 con 0x7f93b415d980 2024-04-25T00:39:22.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b2d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f93b406db10 0x7f93b4151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b1d9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f93b40b2b70 0x7f93b4150df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40538/0 (socket says 172.21.15.29:40538) 2024-04-25T00:39:22.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b1d9b640 1 -- 172.21.15.29:0/2961379944 learned_addr learned my addr 172.21.15.29:0/2961379944 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:22.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b259c640 1 --2- 172.21.15.29:0/2961379944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93b415d980 0x7f93b406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b2d9d640 1 -- 172.21.15.29:0/2961379944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93b415d980 msgr2=0x7f93b406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:22.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b2d9d640 1 --2- 172.21.15.29:0/2961379944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93b415d980 0x7f93b406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b2d9d640 1 -- 172.21.15.29:0/2961379944 >> v1:172.21.15.29:6789/0 conn(0x7f93b40b2b70 legacy=0x7f93b4150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:22.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.890+0000 7f93b2d9d640 1 -- 172.21.15.29:0/2961379944 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b40b36f0 con 0x7f93b406db10 2024-04-25T00:39:22.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b2d9d640 1 --2- 172.21.15.29:0/2961379944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93b406db10 0x7f93b4151500 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00bfa0 tx=0x7f93ac01eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02e7ef4f86b77da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:22.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b159a640 1 -- 172.21.15.29:0/2961379944 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac00b2d0 con 0x7f93b406db10 2024-04-25T00:39:22.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b159a640 1 -- 172.21.15.29:0/2961379944 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ac0a78d0 con 0x7f93b406db10 2024-04-25T00:39:22.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b159a640 1 -- 172.21.15.29:0/2961379944 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0a7ca0 con 0x7f93b406db10 2024-04-25T00:39:22.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 -- 172.21.15.29:0/2961379944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93b406db10 msgr2=0x7f93b4151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:22.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 --2- 172.21.15.29:0/2961379944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93b406db10 0x7f93b4151500 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00bfa0 tx=0x7f93ac01eb70 comp rx=0 tx=0).stop 2024-04-25T00:39:22.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 -- 172.21.15.29:0/2961379944 shutdown_connections 2024-04-25T00:39:22.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 --2- 172.21.15.29:0/2961379944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93b406db10 0x7f93b4151500 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 --2- 172.21.15.29:0/2961379944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93b415d980 0x7f93b406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 -- 172.21.15.29:0/2961379944 >> 172.21.15.29:0/2961379944 conn(0x7f93b40b1870 msgr2=0x7f93b40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:22.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 -- 172.21.15.29:0/2961379944 shutdown_connections 2024-04-25T00:39:22.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 -- 172.21.15.29:0/2961379944 wait complete. 2024-04-25T00:39:22.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 Processor -- start 2024-04-25T00:39:22.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 -- start start 2024-04-25T00:39:22.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b406db10 0x7f93b413df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93b40b2b70 0x7f93b40ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93b415d980 0x7f93b4159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f93b415b630 con 0x7f93b415d980 2024-04-25T00:39:22.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f93b415b4b0 con 0x7f93b406db10 2024-04-25T00:39:22.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b895f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f93b415b7b0 con 0x7f93b40b2b70 2024-04-25T00:39:22.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93b415d980 0x7f93b4159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b2d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93b415d980 0x7f93b4159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39874/0 (socket says 172.21.15.29:39874) 2024-04-25T00:39:22.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b2d9d640 1 -- 172.21.15.29:0/942299084 learned_addr learned my addr 172.21.15.29:0/942299084 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:22.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b2d9d640 1 -- 172.21.15.29:0/942299084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93b40b2b70 msgr2=0x7f93b40ce070 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:39:22.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b2d9d640 1 --2- 172.21.15.29:0/942299084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93b40b2b70 0x7f93b40ce070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b2d9d640 1 -- 172.21.15.29:0/942299084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b406db10 msgr2=0x7f93b413df20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:39:22.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.894+0000 7f93b2d9d640 1 --2- 172.21.15.29:0/942299084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b406db10 0x7f93b413df20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:22.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.898+0000 7f93b2d9d640 1 -- 172.21.15.29:0/942299084 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b40d0a50 con 0x7f93b415d980 2024-04-25T00:39:22.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.898+0000 7f93b2d9d640 1 --2- 172.21.15.29:0/942299084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93b415d980 0x7f93b4159980 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f93ac01d220 tx=0x7f93ac01b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:22.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.898+0000 7f93a2ffd640 1 -- 172.21.15.29:0/942299084 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac009750 con 0x7f93b415d980 2024-04-25T00:39:22.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.898+0000 7f93a2ffd640 1 -- 172.21.15.29:0/942299084 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ac009910 con 0x7f93b415d980 2024-04-25T00:39:22.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.898+0000 7f93a2ffd640 1 -- 172.21.15.29:0/942299084 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac009d00 con 0x7f93b415d980 2024-04-25T00:39:22.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.898+0000 7f93b895f640 1 -- 172.21.15.29:0/942299084 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b414bf60 con 0x7f93b415d980 2024-04-25T00:39:22.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.898+0000 7f93b895f640 1 -- 172.21.15.29:0/942299084 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b414c520 con 0x7f93b415d980 2024-04-25T00:39:22.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.898+0000 7f93b895f640 1 -- 172.21.15.29:0/942299084 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b4069270 con 0x7f93b415d980 2024-04-25T00:39:22.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.898+0000 7f93a2ffd640 1 -- 172.21.15.29:0/942299084 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f93ac0c2020 con 0x7f93b415d980 2024-04-25T00:39:22.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.898+0000 7f93a2ffd640 1 --2- 172.21.15.29:0/942299084 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9394067e60 0x7f939406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:22.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.898+0000 7f93b259c640 1 --2- 172.21.15.29:0/942299084 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9394067e60 0x7f939406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:22.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.898+0000 7f93a2ffd640 1 -- 172.21.15.29:0/942299084 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f93ac1232d0 con 0x7f93b415d980 2024-04-25T00:39:22.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.898+0000 7f93b259c640 1 --2- 172.21.15.29:0/942299084 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9394067e60 0x7f939406a320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f93a400a790 tx=0x7f93a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:22.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:22.902+0000 7f93a2ffd640 1 -- 172.21.15.29:0/942299084 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f93ac018760 con 0x7f93b415d980 2024-04-25T00:39:23.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.086+0000 7f93b895f640 1 -- 172.21.15.29:0/942299084 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f93b40cf910 con 0x7f9394067e60 2024-04-25T00:39:23.091 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:23.086+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.086+0000 7f93a2ffd640 1 -- 172.21.15.29:0/942299084 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f93b40cf910 con 0x7f9394067e60 2024-04-25T00:39:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:23.099 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:23.099 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2024-04-25T00:39:23.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7498ea7640 1 Processor -- start 2024-04-25T00:39:23.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7498ea7640 1 -- start start 2024-04-25T00:39:23.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7498ea7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f749415c220 0x7f74940cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7498ea7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f74940cfa10 0x7f74940cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7498ea7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74940cdbd0 con 0x7f74940d04a0 2024-04-25T00:39:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7498ea7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f74940d0900 con 0x7f74940cfa10 2024-04-25T00:39:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7498ea7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f749415c600 con 0x7f749415c220 2024-04-25T00:39:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7491d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f74940cfa10 0x7f74940cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7491d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f74940cfa10 0x7f74940cff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39884/0 (socket says 172.21.15.29:39884) 2024-04-25T00:39:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7491d9b640 1 -- 172.21.15.29:0/1813155614 learned_addr learned my addr 172.21.15.29:0/1813155614 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7492d9d640 1 --1- 172.21.15.29:0/1813155614 >> v1:172.21.15.29:6789/0 conn(0x7f74940d04a0 0x7f74940ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40542/0 (socket says 172.21.15.29:40542) 2024-04-25T00:39:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f749259c640 1 --2- 172.21.15.29:0/1813155614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f749415c220 0x7f74940cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7491d9b640 1 -- 172.21.15.29:0/1813155614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f749415c220 msgr2=0x7f74940cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7491d9b640 1 --2- 172.21.15.29:0/1813155614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f749415c220 0x7f74940cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7491d9b640 1 -- 172.21.15.29:0/1813155614 >> v1:172.21.15.29:6789/0 conn(0x7f74940d04a0 legacy=0x7f74940ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:23.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f7491d9b640 1 -- 172.21.15.29:0/1813155614 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74940b13e0 con 0x7f74940cfa10 2024-04-25T00:39:23.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.226+0000 7f749159a640 1 -- 172.21.15.29:0/1813155614 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3701967389 0 0) 0x7f74940cdbd0 con 0x7f74940d04a0 2024-04-25T00:39:23.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7491d9b640 1 --2- 172.21.15.29:0/1813155614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74940cfa10 0x7f74940cff60 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f747c009870 tx=0x7f747c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153c05b4da459cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:23.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f749159a640 1 -- 172.21.15.29:0/1813155614 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c0b2070 con 0x7f74940cfa10 2024-04-25T00:39:23.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f749159a640 1 -- 172.21.15.29:0/1813155614 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747c0a9070 con 0x7f74940cfa10 2024-04-25T00:39:23.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f749159a640 1 -- 172.21.15.29:0/1813155614 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c0ad880 con 0x7f74940cfa10 2024-04-25T00:39:23.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 -- 172.21.15.29:0/1813155614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74940cfa10 msgr2=0x7f74940cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:23.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 --2- 172.21.15.29:0/1813155614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74940cfa10 0x7f74940cff60 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f747c009870 tx=0x7f747c01c560 comp rx=0 tx=0).stop 2024-04-25T00:39:23.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 -- 172.21.15.29:0/1813155614 shutdown_connections 2024-04-25T00:39:23.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 --2- 172.21.15.29:0/1813155614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74940cfa10 0x7f74940cff60 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 --2- 172.21.15.29:0/1813155614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f749415c220 0x7f74940cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 -- 172.21.15.29:0/1813155614 >> 172.21.15.29:0/1813155614 conn(0x7f74940d1d60 msgr2=0x7f74940d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:23.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 -- 172.21.15.29:0/1813155614 shutdown_connections 2024-04-25T00:39:23.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 -- 172.21.15.29:0/1813155614 wait complete. 2024-04-25T00:39:23.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 Processor -- start 2024-04-25T00:39:23.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 -- start start 2024-04-25T00:39:23.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74940cfa10 0x7f749406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74940d04a0 0x7f749415e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f749415c220 0x7f749414ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7494169090 con 0x7f74940cfa10 2024-04-25T00:39:23.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7494168f10 con 0x7f74940d04a0 2024-04-25T00:39:23.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.230+0000 7f7498ea7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7494169210 con 0x7f749415c220 2024-04-25T00:39:23.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f749259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74940cfa10 0x7f749406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f749259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74940cfa10 0x7f749406d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39886/0 (socket says 172.21.15.29:39886) 2024-04-25T00:39:23.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f749259c640 1 -- 172.21.15.29:0/800090924 learned_addr learned my addr 172.21.15.29:0/800090924 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:23.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f749259c640 1 -- 172.21.15.29:0/800090924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f749415c220 msgr2=0x7f749414ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:23.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f749259c640 1 --2- 172.21.15.29:0/800090924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f749415c220 0x7f749414ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f749259c640 1 -- 172.21.15.29:0/800090924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74940d04a0 msgr2=0x7f749415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:23.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f749259c640 1 --2- 172.21.15.29:0/800090924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74940d04a0 0x7f749415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f749259c640 1 -- 172.21.15.29:0/800090924 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749415e7d0 con 0x7f74940cfa10 2024-04-25T00:39:23.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f749259c640 1 --2- 172.21.15.29:0/800090924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74940cfa10 0x7f749406d840 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f748400d620 tx=0x7f748400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:23.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f748affd640 1 -- 172.21.15.29:0/800090924 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74840125f0 con 0x7f74940cfa10 2024-04-25T00:39:23.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f748affd640 1 -- 172.21.15.29:0/800090924 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7484019070 con 0x7f74940cfa10 2024-04-25T00:39:23.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f7498ea7640 1 -- 172.21.15.29:0/800090924 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749414b040 con 0x7f74940cfa10 2024-04-25T00:39:23.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f748affd640 1 -- 172.21.15.29:0/800090924 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74840129b0 con 0x7f74940cfa10 2024-04-25T00:39:23.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f7498ea7640 1 -- 172.21.15.29:0/800090924 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f749415ed30 con 0x7f74940cfa10 2024-04-25T00:39:23.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f7498ea7640 1 -- 172.21.15.29:0/800090924 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7494069270 con 0x7f74940cfa10 2024-04-25T00:39:23.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f748affd640 1 -- 172.21.15.29:0/800090924 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f7484012b50 con 0x7f74940cfa10 2024-04-25T00:39:23.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f748affd640 1 --2- 172.21.15.29:0/800090924 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7474067e60 0x7f747406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f748affd640 1 -- 172.21.15.29:0/800090924 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5709+0+0 (secure 0 0 0) 0x7f7484086a40 con 0x7f74940cfa10 2024-04-25T00:39:23.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f7491d9b640 1 --2- 172.21.15.29:0/800090924 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7474067e60 0x7f747406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.234+0000 7f7491d9b640 1 --2- 172.21.15.29:0/800090924 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7474067e60 0x7f747406a320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f747c0af000 tx=0x7f747c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:23.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.238+0000 7f748affd640 1 -- 172.21.15.29:0/800090924 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f749415e7d0 con 0x7f74940cfa10 2024-04-25T00:39:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.430+0000 7f7498ea7640 1 -- 172.21.15.29:0/800090924 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f7494155180 con 0x7f7474067e60 2024-04-25T00:39:23.433 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:23.430+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:23.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.430+0000 7f748affd640 1 -- 172.21.15.29:0/800090924 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f7494155180 con 0x7f7474067e60 2024-04-25T00:39:23.433 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:23.439 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:23.440 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2024-04-25T00:39:23.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.566+0000 7fcbf02b7640 1 Processor -- start 2024-04-25T00:39:23.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.566+0000 7fcbf02b7640 1 -- start start 2024-04-25T00:39:23.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.566+0000 7fcbf02b7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbe81456d0 0x7fcbe80b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.566+0000 7fcbf02b7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbe80b3b50 0x7fcbe80cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.566+0000 7fcbf02b7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbe80b4380 con 0x7fcbe806cd40 2024-04-25T00:39:23.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.566+0000 7fcbf02b7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe80b40a0 con 0x7fcbe80b3b50 2024-04-25T00:39:23.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.566+0000 7fcbf02b7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe80ce480 con 0x7fcbe81456d0 2024-04-25T00:39:23.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbee053640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbe81456d0 0x7fcbe80b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.566+0000 7fcbed852640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbe80b3b50 0x7fcbe80cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbed852640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbe80b3b50 0x7fcbe80cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:57332/0 (socket says 172.21.15.29:57332) 2024-04-25T00:39:23.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbed852640 1 -- 172.21.15.29:0/2219335309 learned_addr learned my addr 172.21.15.29:0/2219335309 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:23.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbee854640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcbe806cd40 0x7fcbe806d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:35086/0 (socket says 172.21.15.29:35086) 2024-04-25T00:39:23.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbee053640 1 -- 172.21.15.29:0/2219335309 >> v2:172.21.15.157:3300/0 conn(0x7fcbe80b3b50 msgr2=0x7fcbe80cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:23.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbee053640 1 --2- 172.21.15.29:0/2219335309 >> v2:172.21.15.157:3300/0 conn(0x7fcbe80b3b50 0x7fcbe80cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbee053640 1 -- 172.21.15.29:0/2219335309 >> v1:172.21.15.157:6789/0 conn(0x7fcbe806cd40 legacy=0x7fcbe806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:23.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbee053640 1 -- 172.21.15.29:0/2219335309 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe80ce760 con 0x7fcbe81456d0 2024-04-25T00:39:23.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbee053640 1 --2- 172.21.15.29:0/2219335309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbe81456d0 0x7fcbe80b2b70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40097c0 tx=0x7fcbe401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da8dbe6fb56fcd0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:23.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbed051640 1 -- 172.21.15.29:0/2219335309 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe40b0070 con 0x7fcbe81456d0 2024-04-25T00:39:23.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbed051640 1 -- 172.21.15.29:0/2219335309 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe40ab600 con 0x7fcbe81456d0 2024-04-25T00:39:23.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbed051640 1 -- 172.21.15.29:0/2219335309 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe40ab9d0 con 0x7fcbe81456d0 2024-04-25T00:39:23.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbf02b7640 1 -- 172.21.15.29:0/2219335309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbe81456d0 msgr2=0x7fcbe80b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:23.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbf02b7640 1 --2- 172.21.15.29:0/2219335309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbe81456d0 0x7fcbe80b2b70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40097c0 tx=0x7fcbe401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbf02b7640 1 -- 172.21.15.29:0/2219335309 shutdown_connections 2024-04-25T00:39:23.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbf02b7640 1 --2- 172.21.15.29:0/2219335309 >> v2:172.21.15.157:3300/0 conn(0x7fcbe80b3b50 0x7fcbe80cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbf02b7640 1 --2- 172.21.15.29:0/2219335309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbe81456d0 0x7fcbe80b2b70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbf02b7640 1 -- 172.21.15.29:0/2219335309 >> 172.21.15.29:0/2219335309 conn(0x7fcbe80b1870 msgr2=0x7fcbe80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:23.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbf02b7640 1 -- 172.21.15.29:0/2219335309 shutdown_connections 2024-04-25T00:39:23.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.570+0000 7fcbf02b7640 1 -- 172.21.15.29:0/2219335309 wait complete. 2024-04-25T00:39:23.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbf02b7640 1 Processor -- start 2024-04-25T00:39:23.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbf02b7640 1 -- start start 2024-04-25T00:39:23.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbf02b7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbe806cd40 0x7fcbe813e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbf02b7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbe80b3b50 0x7fcbe815fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbf02b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe81456d0 0x7fcbe815ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbf02b7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe80cf620 con 0x7fcbe80b3b50 2024-04-25T00:39:23.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbf02b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe80cf4a0 con 0x7fcbe81456d0 2024-04-25T00:39:23.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbf02b7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe8168b70 con 0x7fcbe806cd40 2024-04-25T00:39:23.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbed852640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbe80b3b50 0x7fcbe815fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbed852640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbe80b3b50 0x7fcbe815fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39890/0 (socket says 172.21.15.29:39890) 2024-04-25T00:39:23.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbed852640 1 -- 172.21.15.29:0/3612339576 learned_addr learned my addr 172.21.15.29:0/3612339576 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:23.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbee053640 1 --2- 172.21.15.29:0/3612339576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbe806cd40 0x7fcbe813e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbee854640 1 --2- 172.21.15.29:0/3612339576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe81456d0 0x7fcbe815ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbed852640 1 -- 172.21.15.29:0/3612339576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbe806cd40 msgr2=0x7fcbe813e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbed852640 1 --2- 172.21.15.29:0/3612339576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbe806cd40 0x7fcbe813e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbed852640 1 -- 172.21.15.29:0/3612339576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe81456d0 msgr2=0x7fcbe815ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbed852640 1 --2- 172.21.15.29:0/3612339576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbe81456d0 0x7fcbe815ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbed852640 1 -- 172.21.15.29:0/3612339576 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe813d610 con 0x7fcbe80b3b50 2024-04-25T00:39:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbed852640 1 --2- 172.21.15.29:0/3612339576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbe80b3b50 0x7fcbe815fa70 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800e7f0 tx=0x7fcbd800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbdeffd640 1 -- 172.21.15.29:0/3612339576 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8017070 con 0x7fcbe80b3b50 2024-04-25T00:39:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbdeffd640 1 -- 172.21.15.29:0/3612339576 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd800c940 con 0x7fcbe80b3b50 2024-04-25T00:39:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbf02b7640 1 -- 172.21.15.29:0/3612339576 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe813ce20 con 0x7fcbe80b3b50 2024-04-25T00:39:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbdeffd640 1 -- 172.21.15.29:0/3612339576 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd800cd30 con 0x7fcbe80b3b50 2024-04-25T00:39:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbf02b7640 1 -- 172.21.15.29:0/3612339576 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe813d8d0 con 0x7fcbe80b3b50 2024-04-25T00:39:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.574+0000 7fcbf02b7640 1 -- 172.21.15.29:0/3612339576 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe80692e0 con 0x7fcbe80b3b50 2024-04-25T00:39:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.578+0000 7fcbdeffd640 1 -- 172.21.15.29:0/3612339576 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fcbd801a070 con 0x7fcbe80b3b50 2024-04-25T00:39:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.578+0000 7fcbdeffd640 1 --2- 172.21.15.29:0/3612339576 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcbcc067e90 0x7fcbcc06a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.578+0000 7fcbdeffd640 1 -- 172.21.15.29:0/3612339576 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5738+0+0 (secure 0 0 0) 0x7fcbd8086520 con 0x7fcbe80b3b50 2024-04-25T00:39:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.578+0000 7fcbee053640 1 --2- 172.21.15.29:0/3612339576 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcbcc067e90 0x7fcbcc06a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.578+0000 7fcbee053640 1 --2- 172.21.15.29:0/3612339576 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcbcc067e90 0x7fcbcc06a350 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40ad000 tx=0x7fcbe4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:23.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.582+0000 7fcbdeffd640 1 -- 172.21.15.29:0/3612339576 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcbd8056180 con 0x7fcbe80b3b50 2024-04-25T00:39:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.774+0000 7fcbf02b7640 1 -- 172.21.15.29:0/3612339576 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) -- 0x7fcbe815a850 con 0x7fcbcc067e90 2024-04-25T00:39:23.777 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:23.774+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:23.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.774+0000 7fcbdeffd640 1 -- 172.21.15.29:0/3612339576 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fcbe815a850 con 0x7fcbcc067e90 2024-04-25T00:39:23.778 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:23.783 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:23.784 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2024-04-25T00:39:23.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.906+0000 7f7c05848640 1 Processor -- start 2024-04-25T00:39:23.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.906+0000 7f7c05848640 1 -- start start 2024-04-25T00:39:23.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.910+0000 7f7c05848640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7c0015e040 0x7f7c00161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.910+0000 7f7c05848640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c00162340 0x7f7c0006b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.910+0000 7f7c05848640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c00162850 con 0x7f7c0015e420 2024-04-25T00:39:23.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.910+0000 7f7c05848640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7c000cee80 con 0x7f7c0015e040 2024-04-25T00:39:23.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.910+0000 7f7c05848640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c00162b30 con 0x7f7c00162340 2024-04-25T00:39:23.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.910+0000 7f7bff7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7c0015e420 0x7f7c000cb3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:35094/0 (socket says 172.21.15.29:35094) 2024-04-25T00:39:23.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.910+0000 7f7bfeffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7c0015e040 0x7f7c00161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.910+0000 7f7bff7fe640 1 -- 172.21.15.29:0/4252477315 learned_addr learned my addr 172.21.15.29:0/4252477315 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:23.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.910+0000 7f7bf7fff640 1 --2- 172.21.15.29:0/4252477315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c00162340 0x7f7c0006b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.910+0000 7f7bfeffd640 1 -- 172.21.15.29:0/4252477315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c00162340 msgr2=0x7f7c0006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:23.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.910+0000 7f7bfeffd640 1 --2- 172.21.15.29:0/4252477315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c00162340 0x7f7c0006b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.910+0000 7f7bfeffd640 1 -- 172.21.15.29:0/4252477315 >> v1:172.21.15.157:6789/0 conn(0x7f7c0015e420 legacy=0x7f7c000cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:23.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.910+0000 7f7bfeffd640 1 -- 172.21.15.29:0/4252477315 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c000cf0d0 con 0x7f7c0015e040 2024-04-25T00:39:23.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfeffd640 1 --2- 172.21.15.29:0/4252477315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c0015e040 0x7f7c00161e00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00097c0 tx=0x7f7bf001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a909165483106c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:23.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfe7fc640 1 -- 172.21.15.29:0/4252477315 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00b0070 con 0x7f7c0015e040 2024-04-25T00:39:23.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfe7fc640 1 -- 172.21.15.29:0/4252477315 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf00ab600 con 0x7f7c0015e040 2024-04-25T00:39:23.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfe7fc640 1 -- 172.21.15.29:0/4252477315 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00ab9d0 con 0x7f7c0015e040 2024-04-25T00:39:23.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 -- 172.21.15.29:0/4252477315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c0015e040 msgr2=0x7f7c00161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:23.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 --2- 172.21.15.29:0/4252477315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c0015e040 0x7f7c00161e00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00097c0 tx=0x7f7bf001c3e0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 -- 172.21.15.29:0/4252477315 shutdown_connections 2024-04-25T00:39:23.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 --2- 172.21.15.29:0/4252477315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c00162340 0x7f7c0006b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 --2- 172.21.15.29:0/4252477315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c0015e040 0x7f7c00161e00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 -- 172.21.15.29:0/4252477315 >> 172.21.15.29:0/4252477315 conn(0x7f7c000d0da0 msgr2=0x7f7c000d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:23.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 -- 172.21.15.29:0/4252477315 shutdown_connections 2024-04-25T00:39:23.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 -- 172.21.15.29:0/4252477315 wait complete. 2024-04-25T00:39:23.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 Processor -- start 2024-04-25T00:39:23.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 -- start start 2024-04-25T00:39:23.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c0015e040 0x7f7c0015ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c0015e420 0x7f7c00154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c00162340 0x7f7c00155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7c000cfc60 con 0x7f7c0015e040 2024-04-25T00:39:23.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c000cfae0 con 0x7f7c00162340 2024-04-25T00:39:23.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c000cfde0 con 0x7f7c0015e420 2024-04-25T00:39:23.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c0015e040 0x7f7c0015ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c0015e040 0x7f7c0015ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39892/0 (socket says 172.21.15.29:39892) 2024-04-25T00:39:23.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfeffd640 1 -- 172.21.15.29:0/3613790271 learned_addr learned my addr 172.21.15.29:0/3613790271 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:23.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfeffd640 1 -- 172.21.15.29:0/3613790271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c0015e420 msgr2=0x7f7c00154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:23.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfeffd640 1 --2- 172.21.15.29:0/3613790271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c0015e420 0x7f7c00154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfeffd640 1 -- 172.21.15.29:0/3613790271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c00162340 msgr2=0x7f7c00155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:23.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfeffd640 1 --2- 172.21.15.29:0/3613790271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c00162340 0x7f7c00155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:23.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfeffd640 1 -- 172.21.15.29:0/3613790271 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0006de80 con 0x7f7c0015e040 2024-04-25T00:39:23.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfeffd640 1 --2- 172.21.15.29:0/3613790271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c0015e040 0x7f7c0015ef70 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00ae040 tx=0x7f7bf001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:23.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfcff9640 1 -- 172.21.15.29:0/3613790271 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00b0050 con 0x7f7c0015e040 2024-04-25T00:39:23.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfcff9640 1 -- 172.21.15.29:0/3613790271 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf00b1040 con 0x7f7c0015e040 2024-04-25T00:39:23.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7bfcff9640 1 -- 172.21.15.29:0/3613790271 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0006e20 con 0x7f7c0015e040 2024-04-25T00:39:23.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.914+0000 7f7c05848640 1 -- 172.21.15.29:0/3613790271 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0006e120 con 0x7f7c0015e040 2024-04-25T00:39:23.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.918+0000 7f7c05848640 1 -- 172.21.15.29:0/3613790271 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0006e6e0 con 0x7f7c0015e040 2024-04-25T00:39:23.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.918+0000 7f7c05848640 1 -- 172.21.15.29:0/3613790271 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c000692e0 con 0x7f7c0015e040 2024-04-25T00:39:23.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.918+0000 7f7bfcff9640 1 -- 172.21.15.29:0/3613790271 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f7bf00be030 con 0x7f7c0015e040 2024-04-25T00:39:23.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.918+0000 7f7bfcff9640 1 --2- 172.21.15.29:0/3613790271 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7be0067e60 0x7f7be006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:23.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.918+0000 7f7bf7fff640 1 --2- 172.21.15.29:0/3613790271 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7be0067e60 0x7f7be006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:23.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.918+0000 7f7bfcff9640 1 -- 172.21.15.29:0/3613790271 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5738+0+0 (secure 0 0 0) 0x7f7bf0120210 con 0x7f7c0015e040 2024-04-25T00:39:23.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.918+0000 7f7bf7fff640 1 --2- 172.21.15.29:0/3613790271 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7be0067e60 0x7f7be006a320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7be8006aa0 tx=0x7f7be8006a30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:23.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:23.922+0000 7f7bfcff9640 1 -- 172.21.15.29:0/3613790271 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7bf00c5060 con 0x7f7c0015e040 2024-04-25T00:39:24.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.118+0000 7f7c05848640 1 -- 172.21.15.29:0/3613790271 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) -- 0x7f7c000d1bd0 con 0x7f7be0067e60 2024-04-25T00:39:24.122 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:24.118+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:24.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.118+0000 7f7bfcff9640 1 -- 172.21.15.29:0/3613790271 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f7c000d1bd0 con 0x7f7be0067e60 2024-04-25T00:39:24.123 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:24.128 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:24.128 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2024-04-25T00:39:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc08d7d640 1 Processor -- start 2024-04-25T00:39:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc08d7d640 1 -- start start 2024-04-25T00:39:24.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc08d7d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc040b4780 0x7fdc0415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc08d7d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc040cc240 0x7fdc0415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc08d7d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc040b3720 con 0x7fdc0415e8d0 2024-04-25T00:39:24.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc08d7d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdc040b42f0 con 0x7fdc040b4780 2024-04-25T00:39:24.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc08d7d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc040b39c0 con 0x7fdc040cc240 2024-04-25T00:39:24.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc01d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc040b4780 0x7fdc0415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc0259c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdc0415e8d0 0x7fdc0415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:35102/0 (socket says 172.21.15.29:35102) 2024-04-25T00:39:24.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc01d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc040b4780 0x7fdc0415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:57370/0 (socket says 172.21.15.29:57370) 2024-04-25T00:39:24.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc0259c640 1 -- 172.21.15.29:0/3911609302 learned_addr learned my addr 172.21.15.29:0/3911609302 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:24.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc02d9d640 1 --2- 172.21.15.29:0/3911609302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc040cc240 0x7fdc0415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc01d9b640 1 -- 172.21.15.29:0/3911609302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc040cc240 msgr2=0x7fdc0415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:24.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc01d9b640 1 --2- 172.21.15.29:0/3911609302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc040cc240 0x7fdc0415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc01d9b640 1 -- 172.21.15.29:0/3911609302 >> v1:172.21.15.157:6789/0 conn(0x7fdc0415e8d0 legacy=0x7fdc0415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc01d9b640 1 -- 172.21.15.29:0/3911609302 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc040b3ca0 con 0x7fdc040b4780 2024-04-25T00:39:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.254+0000 7fdc02d9d640 1 --2- 172.21.15.29:0/3911609302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc040cc240 0x7fdc0415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:39:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc01d9b640 1 --2- 172.21.15.29:0/3911609302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040b4780 0x7fdc0415f3c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdc040b4470 tx=0x7fdbec01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d70c5b78fc64c90d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc0159a640 1 -- 172.21.15.29:0/3911609302 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0b0070 con 0x7fdc040b4780 2024-04-25T00:39:24.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc0159a640 1 -- 172.21.15.29:0/3911609302 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbec0ab600 con 0x7fdc040b4780 2024-04-25T00:39:24.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc0159a640 1 -- 172.21.15.29:0/3911609302 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0ab9d0 con 0x7fdc040b4780 2024-04-25T00:39:24.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc08d7d640 1 -- 172.21.15.29:0/3911609302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040b4780 msgr2=0x7fdc0415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:24.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc08d7d640 1 --2- 172.21.15.29:0/3911609302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040b4780 0x7fdc0415f3c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdc040b4470 tx=0x7fdbec01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc08d7d640 1 -- 172.21.15.29:0/3911609302 shutdown_connections 2024-04-25T00:39:24.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc08d7d640 1 --2- 172.21.15.29:0/3911609302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc040cc240 0x7fdc0415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc08d7d640 1 --2- 172.21.15.29:0/3911609302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040b4780 0x7fdc0415f3c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc08d7d640 1 -- 172.21.15.29:0/3911609302 >> 172.21.15.29:0/3911609302 conn(0x7fdc040b1870 msgr2=0x7fdc040b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:24.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc08d7d640 1 -- 172.21.15.29:0/3911609302 shutdown_connections 2024-04-25T00:39:24.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc08d7d640 1 -- 172.21.15.29:0/3911609302 wait complete. 2024-04-25T00:39:24.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc08d7d640 1 Processor -- start 2024-04-25T00:39:24.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.258+0000 7fdc08d7d640 1 -- start start 2024-04-25T00:39:24.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc08d7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040b4780 0x7fdc041395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc08d7d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc040cc240 0x7fdc041381b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc08d7d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc0415e8d0 0x7fdc041386f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc08d7d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0415d6a0 con 0x7fdc0415e8d0 2024-04-25T00:39:24.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc08d7d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0415d520 con 0x7fdc040b4780 2024-04-25T00:39:24.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc08d7d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0406d5d0 con 0x7fdc040cc240 2024-04-25T00:39:24.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc02d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc0415e8d0 0x7fdc041386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc02d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc0415e8d0 0x7fdc041386f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39906/0 (socket says 172.21.15.29:39906) 2024-04-25T00:39:24.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc02d9d640 1 -- 172.21.15.29:0/3446108237 learned_addr learned my addr 172.21.15.29:0/3446108237 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:24.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc01d9b640 1 --2- 172.21.15.29:0/3446108237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc040cc240 0x7fdc041381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc0259c640 1 --2- 172.21.15.29:0/3446108237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040b4780 0x7fdc041395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc02d9d640 1 -- 172.21.15.29:0/3446108237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc040cc240 msgr2=0x7fdc041381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:24.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc02d9d640 1 --2- 172.21.15.29:0/3446108237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc040cc240 0x7fdc041381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc02d9d640 1 -- 172.21.15.29:0/3446108237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040b4780 msgr2=0x7fdc041395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:24.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc02d9d640 1 --2- 172.21.15.29:0/3446108237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040b4780 0x7fdc041395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc02d9d640 1 -- 172.21.15.29:0/3446108237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc040a9530 con 0x7fdc0415e8d0 2024-04-25T00:39:24.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc02d9d640 1 --2- 172.21.15.29:0/3446108237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc0415e8d0 0x7fdc041386f0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00efc0 tx=0x7fdbfc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:24.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdbfaffd640 1 -- 172.21.15.29:0/3446108237 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc018030 con 0x7fdc0415e8d0 2024-04-25T00:39:24.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdbfaffd640 1 -- 172.21.15.29:0/3446108237 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbfc00cde0 con 0x7fdc0415e8d0 2024-04-25T00:39:24.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc08d7d640 1 -- 172.21.15.29:0/3446108237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc040a9810 con 0x7fdc0415e8d0 2024-04-25T00:39:24.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdbfaffd640 1 -- 172.21.15.29:0/3446108237 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc003bc0 con 0x7fdc0415e8d0 2024-04-25T00:39:24.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc08d7d640 1 -- 172.21.15.29:0/3446108237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc040a9dd0 con 0x7fdc0415e8d0 2024-04-25T00:39:24.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdc08d7d640 1 -- 172.21.15.29:0/3446108237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc04069270 con 0x7fdc0415e8d0 2024-04-25T00:39:24.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.262+0000 7fdbfaffd640 1 -- 172.21.15.29:0/3446108237 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fdbfc01b070 con 0x7fdc0415e8d0 2024-04-25T00:39:24.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.266+0000 7fdbfaffd640 1 --2- 172.21.15.29:0/3446108237 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdbe4067e60 0x7fdbe406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.266+0000 7fdbfaffd640 1 -- 172.21.15.29:0/3446108237 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5738+0+0 (secure 0 0 0) 0x7fdbfc0870a0 con 0x7fdc0415e8d0 2024-04-25T00:39:24.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.266+0000 7fdc0259c640 1 --2- 172.21.15.29:0/3446108237 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdbe4067e60 0x7fdbe406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.266+0000 7fdc0259c640 1 --2- 172.21.15.29:0/3446108237 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdbe4067e60 0x7fdbe406a320 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdc040b51d0 tx=0x7fdbf4007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:24.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.270+0000 7fdbfaffd640 1 -- 172.21.15.29:0/3446108237 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdbfc022070 con 0x7fdc0415e8d0 2024-04-25T00:39:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.458+0000 7fdc08d7d640 1 -- 172.21.15.29:0/3446108237 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7fdc040cbd60 con 0x7fdbe4067e60 2024-04-25T00:39:24.463 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:24.462+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:24.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.462+0000 7fdbfaffd640 1 -- 172.21.15.29:0/3446108237 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fdc040cbd60 con 0x7fdbe4067e60 2024-04-25T00:39:24.464 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:24.470 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:24.471 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2024-04-25T00:39:24.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.586+0000 7f9a49e20640 1 Processor -- start 2024-04-25T00:39:24.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.586+0000 7f9a49e20640 1 -- start start 2024-04-25T00:39:24.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a49e20640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9a441456d0 0x7f9a440b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a49e20640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9a4406cd40 0x7f9a4406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a49e20640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a440cf670 con 0x7f9a440b3b50 2024-04-25T00:39:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a49e20640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9a440b4380 con 0x7f9a4406cd40 2024-04-25T00:39:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a49e20640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9a440b40a0 con 0x7f9a441456d0 2024-04-25T00:39:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a43fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9a4406cd40 0x7f9a4406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a43fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9a4406cd40 0x7f9a4406d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39910/0 (socket says 172.21.15.29:39910) 2024-04-25T00:39:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a43fff640 1 -- 172.21.15.29:0/3962450222 learned_addr learned my addr 172.21.15.29:0/3962450222 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a437fe640 1 --2- 172.21.15.29:0/3962450222 >> v2:172.21.15.157:3300/0 conn(0x7f9a441456d0 0x7f9a440b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a43fff640 1 -- 172.21.15.29:0/3962450222 >> v2:172.21.15.157:3300/0 conn(0x7f9a441456d0 msgr2=0x7f9a440b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a43fff640 1 --2- 172.21.15.29:0/3962450222 >> v2:172.21.15.157:3300/0 conn(0x7f9a441456d0 0x7f9a440b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a43fff640 1 -- 172.21.15.29:0/3962450222 >> v1:172.21.15.29:6789/0 conn(0x7f9a440b3b50 legacy=0x7f9a440cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:24.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a43fff640 1 -- 172.21.15.29:0/3962450222 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a440ce480 con 0x7f9a4406cd40 2024-04-25T00:39:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a43fff640 1 --2- 172.21.15.29:0/3962450222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a4406cd40 0x7f9a4406d150 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f9a44067fa0 tx=0x7f9a3c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d40de22866d3350 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a42ffd640 1 -- 172.21.15.29:0/3962450222 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0a57a0 con 0x7f9a4406cd40 2024-04-25T00:39:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a42ffd640 1 -- 172.21.15.29:0/3962450222 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a3c0a5940 con 0x7f9a4406cd40 2024-04-25T00:39:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a42ffd640 1 -- 172.21.15.29:0/3962450222 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0a5d10 con 0x7f9a4406cd40 2024-04-25T00:39:24.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a49e20640 1 -- 172.21.15.29:0/3962450222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a4406cd40 msgr2=0x7f9a4406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:24.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a49e20640 1 --2- 172.21.15.29:0/3962450222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a4406cd40 0x7f9a4406d150 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f9a44067fa0 tx=0x7f9a3c01c9f0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a49e20640 1 -- 172.21.15.29:0/3962450222 shutdown_connections 2024-04-25T00:39:24.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a49e20640 1 --2- 172.21.15.29:0/3962450222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a4406cd40 0x7f9a4406d150 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a49e20640 1 --2- 172.21.15.29:0/3962450222 >> v2:172.21.15.157:3300/0 conn(0x7f9a441456d0 0x7f9a440b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a49e20640 1 -- 172.21.15.29:0/3962450222 >> 172.21.15.29:0/3962450222 conn(0x7f9a440b1870 msgr2=0x7f9a440b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a49e20640 1 -- 172.21.15.29:0/3962450222 shutdown_connections 2024-04-25T00:39:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.590+0000 7f9a49e20640 1 -- 172.21.15.29:0/3962450222 wait complete. 2024-04-25T00:39:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a49e20640 1 Processor -- start 2024-04-25T00:39:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a49e20640 1 -- start start 2024-04-25T00:39:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a49e20640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a4406cd40 0x7f9a440b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a49e20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a440b3b50 0x7f9a441436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a49e20640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a441456d0 0x7f9a4413b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a49e20640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4406d690 con 0x7f9a441456d0 2024-04-25T00:39:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a49e20640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a440b5000 con 0x7f9a440b3b50 2024-04-25T00:39:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a49e20640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9a44168b70 con 0x7f9a4406cd40 2024-04-25T00:39:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a43fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a441456d0 0x7f9a4413b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a43fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a441456d0 0x7f9a4413b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39918/0 (socket says 172.21.15.29:39918) 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a43fff640 1 -- 172.21.15.29:0/2930096257 learned_addr learned my addr 172.21.15.29:0/2930096257 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a43fff640 1 -- 172.21.15.29:0/2930096257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a4406cd40 msgr2=0x7f9a440b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a3affd640 1 --2- 172.21.15.29:0/2930096257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a440b3b50 0x7f9a441436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a437fe640 1 --2- 172.21.15.29:0/2930096257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a4406cd40 0x7f9a440b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a43fff640 1 --2- 172.21.15.29:0/2930096257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a4406cd40 0x7f9a440b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a43fff640 1 -- 172.21.15.29:0/2930096257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a440b3b50 msgr2=0x7f9a441436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a43fff640 1 --2- 172.21.15.29:0/2930096257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a440b3b50 0x7f9a441436e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a43fff640 1 -- 172.21.15.29:0/2930096257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a44068500 con 0x7f9a441456d0 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a43fff640 1 --2- 172.21.15.29:0/2930096257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a441456d0 0x7f9a4413b280 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0af040 tx=0x7f9a3c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a40ff9640 1 -- 172.21.15.29:0/2930096257 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0b8680 con 0x7f9a441456d0 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a40ff9640 1 -- 172.21.15.29:0/2930096257 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a3c0be070 con 0x7f9a441456d0 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a49e20640 1 -- 172.21.15.29:0/2930096257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a441568f0 con 0x7f9a441456d0 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a40ff9640 1 -- 172.21.15.29:0/2930096257 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0b8a80 con 0x7f9a441456d0 2024-04-25T00:39:24.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a49e20640 1 -- 172.21.15.29:0/2930096257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a44157130 con 0x7f9a441456d0 2024-04-25T00:39:24.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a49e20640 1 -- 172.21.15.29:0/2930096257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a440692e0 con 0x7f9a441456d0 2024-04-25T00:39:24.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.594+0000 7f9a40ff9640 1 -- 172.21.15.29:0/2930096257 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f9a3c0b8c20 con 0x7f9a441456d0 2024-04-25T00:39:24.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.598+0000 7f9a40ff9640 1 --2- 172.21.15.29:0/2930096257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a24067e60 0x7f9a2406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.598+0000 7f9a40ff9640 1 -- 172.21.15.29:0/2930096257 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f9a3c121ab0 con 0x7f9a441456d0 2024-04-25T00:39:24.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.598+0000 7f9a437fe640 1 --2- 172.21.15.29:0/2930096257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a24067e60 0x7f9a2406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.598+0000 7f9a437fe640 1 --2- 172.21.15.29:0/2930096257 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a24067e60 0x7f9a2406a320 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400b800 tx=0x7f9a34008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:24.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.602+0000 7f9a40ff9640 1 -- 172.21.15.29:0/2930096257 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9a441568f0 con 0x7f9a441456d0 2024-04-25T00:39:24.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.786+0000 7f9a49e20640 1 -- 172.21.15.29:0/2930096257 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f9a440b4de0 con 0x7f9a24067e60 2024-04-25T00:39:24.789 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:24.786+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:24.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.786+0000 7f9a40ff9640 1 -- 172.21.15.29:0/2930096257 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f9a440b4de0 con 0x7f9a24067e60 2024-04-25T00:39:24.789 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:24.794 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:24.795 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2024-04-25T00:39:24.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa77d818640 1 Processor -- start 2024-04-25T00:39:24.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa77d818640 1 -- start start 2024-04-25T00:39:24.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa77d818640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7780b29c0 0x7fa7780d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa77d818640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7780d21d0 0x7fa7780cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa77d818640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7780cc9d0 con 0x7fa77815c100 2024-04-25T00:39:24.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa77d818640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa7780b2e20 con 0x7fa7780b29c0 2024-04-25T00:39:24.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa77d818640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa7780ccd00 con 0x7fa7780d21d0 2024-04-25T00:39:24.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa7777fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7780d21d0 0x7fa7780cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa7777fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7780d21d0 0x7fa7780cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:48024/0 (socket says 172.21.15.29:48024) 2024-04-25T00:39:24.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa7777fe640 1 -- 172.21.15.29:0/2281227345 learned_addr learned my addr 172.21.15.29:0/2281227345 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:24.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa7767fc640 1 --2- 172.21.15.29:0/2281227345 >> v2:172.21.15.157:3300/0 conn(0x7fa7780b29c0 0x7fa7780d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa7777fe640 1 -- 172.21.15.29:0/2281227345 >> v2:172.21.15.157:3300/0 conn(0x7fa7780b29c0 msgr2=0x7fa7780d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:24.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa7777fe640 1 --2- 172.21.15.29:0/2281227345 >> v2:172.21.15.157:3300/0 conn(0x7fa7780b29c0 0x7fa7780d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa7777fe640 1 -- 172.21.15.29:0/2281227345 >> v1:172.21.15.157:6789/0 conn(0x7fa77815c100 legacy=0x7fa7780d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:24.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.914+0000 7fa7777fe640 1 -- 172.21.15.29:0/2281227345 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7780b3050 con 0x7fa7780d21d0 2024-04-25T00:39:24.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.918+0000 7fa7777fe640 1 --2- 172.21.15.29:0/2281227345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7780d21d0 0x7fa7780cd190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa770009e50 tx=0x7fa77001c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a52d5969f3e6fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:24.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.918+0000 7fa775ffb640 1 -- 172.21.15.29:0/2281227345 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7700a5770 con 0x7fa7780d21d0 2024-04-25T00:39:24.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.918+0000 7fa775ffb640 1 -- 172.21.15.29:0/2281227345 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7700a5910 con 0x7fa7780d21d0 2024-04-25T00:39:24.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.918+0000 7fa775ffb640 1 -- 172.21.15.29:0/2281227345 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7700a5ce0 con 0x7fa7780d21d0 2024-04-25T00:39:24.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.918+0000 7fa77d818640 1 -- 172.21.15.29:0/2281227345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7780d21d0 msgr2=0x7fa7780cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:24.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.918+0000 7fa77d818640 1 --2- 172.21.15.29:0/2281227345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7780d21d0 0x7fa7780cd190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa770009e50 tx=0x7fa77001c9a0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.918+0000 7fa77d818640 1 -- 172.21.15.29:0/2281227345 shutdown_connections 2024-04-25T00:39:24.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.918+0000 7fa77d818640 1 --2- 172.21.15.29:0/2281227345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7780d21d0 0x7fa7780cd190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.918+0000 7fa77d818640 1 --2- 172.21.15.29:0/2281227345 >> v2:172.21.15.157:3300/0 conn(0x7fa7780b29c0 0x7fa7780d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.918+0000 7fa77d818640 1 -- 172.21.15.29:0/2281227345 >> 172.21.15.29:0/2281227345 conn(0x7fa77809aa30 msgr2=0x7fa77809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:24.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.918+0000 7fa77d818640 1 -- 172.21.15.29:0/2281227345 shutdown_connections 2024-04-25T00:39:24.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.918+0000 7fa77d818640 1 -- 172.21.15.29:0/2281227345 wait complete. 2024-04-25T00:39:24.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa77d818640 1 Processor -- start 2024-04-25T00:39:24.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa77d818640 1 -- start start 2024-04-25T00:39:24.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa77d818640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7780b29c0 0x7fa778159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa77d818640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7780d21d0 0x7fa77814efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa77d818640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77815c100 0x7fa77814fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa77d818640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa77815c660 con 0x7fa77815c100 2024-04-25T00:39:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa77d818640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa77815c4e0 con 0x7fa7780d21d0 2024-04-25T00:39:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa77d818640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa7780cdee0 con 0x7fa7780b29c0 2024-04-25T00:39:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa7777fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77815c100 0x7fa77814fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa7777fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77815c100 0x7fa77814fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39926/0 (socket says 172.21.15.29:39926) 2024-04-25T00:39:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa7777fe640 1 -- 172.21.15.29:0/3666155486 learned_addr learned my addr 172.21.15.29:0/3666155486 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa776ffd640 1 --2- 172.21.15.29:0/3666155486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7780b29c0 0x7fa778159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa7767fc640 1 --2- 172.21.15.29:0/3666155486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7780d21d0 0x7fa77814efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa7777fe640 1 -- 172.21.15.29:0/3666155486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7780b29c0 msgr2=0x7fa778159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa7777fe640 1 --2- 172.21.15.29:0/3666155486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7780b29c0 0x7fa778159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa7777fe640 1 -- 172.21.15.29:0/3666155486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7780d21d0 msgr2=0x7fa77814efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa7777fe640 1 --2- 172.21.15.29:0/3666155486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7780d21d0 0x7fa77814efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa7777fe640 1 -- 172.21.15.29:0/3666155486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa778164610 con 0x7fa77815c100 2024-04-25T00:39:24.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa7777fe640 1 --2- 172.21.15.29:0/3666155486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa77815c100 0x7fa77814fe70 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fa77001a5d0 tx=0x7fa770019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:24.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa75ffff640 1 -- 172.21.15.29:0/3666155486 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7700bc020 con 0x7fa77815c100 2024-04-25T00:39:24.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa75ffff640 1 -- 172.21.15.29:0/3666155486 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7700b8690 con 0x7fa77815c100 2024-04-25T00:39:24.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa77d818640 1 -- 172.21.15.29:0/3666155486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7781648f0 con 0x7fa77815c100 2024-04-25T00:39:24.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa75ffff640 1 -- 172.21.15.29:0/3666155486 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7700b8a80 con 0x7fa77815c100 2024-04-25T00:39:24.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa77d818640 1 -- 172.21.15.29:0/3666155486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa778164eb0 con 0x7fa77815c100 2024-04-25T00:39:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa77d818640 1 -- 172.21.15.29:0/3666155486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7780692e0 con 0x7fa77815c100 2024-04-25T00:39:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.922+0000 7fa75ffff640 1 -- 172.21.15.29:0/3666155486 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fa7700bf020 con 0x7fa77815c100 2024-04-25T00:39:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.926+0000 7fa75ffff640 1 --2- 172.21.15.29:0/3666155486 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa764067e60 0x7fa76406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.926+0000 7fa75ffff640 1 -- 172.21.15.29:0/3666155486 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7fa770121a50 con 0x7fa77815c100 2024-04-25T00:39:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.926+0000 7fa776ffd640 1 --2- 172.21.15.29:0/3666155486 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa764067e60 0x7fa76406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.926+0000 7fa776ffd640 1 --2- 172.21.15.29:0/3666155486 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa764067e60 0x7fa76406a320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa768000f70 tx=0x7fa768009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:24.930+0000 7fa75ffff640 1 -- 172.21.15.29:0/3666155486 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa770016620 con 0x7fa77815c100 2024-04-25T00:39:25.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.118+0000 7fa77d818640 1 -- 172.21.15.29:0/3666155486 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fa77806f730 con 0x7fa764067e60 2024-04-25T00:39:25.124 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:25.122+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.122+0000 7fa75ffff640 1 -- 172.21.15.29:0/3666155486 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fa77806f730 con 0x7fa764067e60 2024-04-25T00:39:25.125 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:25.130 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:25.131 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2024-04-25T00:39:25.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.246+0000 7f40939ed640 1 Processor -- start 2024-04-25T00:39:25.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.246+0000 7f40939ed640 1 -- start start 2024-04-25T00:39:25.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f40939ed640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f408c15e9a0 0x7f408c070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f40939ed640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f408c0712c0 con 0x7f408c0cd1c0 2024-04-25T00:39:25.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f40939ed640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f408c0715a0 con 0x7f408c0b4590 2024-04-25T00:39:25.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f40939ed640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f408c071ed0 con 0x7f408c15e9a0 2024-04-25T00:39:25.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4091f8a640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f408c0cd1c0 0x7f408c072360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40550/0 (socket says 172.21.15.29:40550) 2024-04-25T00:39:25.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4091f8a640 1 -- 172.21.15.29:0/815829705 learned_addr learned my addr 172.21.15.29:0/815829705 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:25.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 804277100 0 0) 0x7f408c0712c0 con 0x7f408c0cd1c0 2024-04-25T00:39:25.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4078003530 con 0x7f408c0cd1c0 2024-04-25T00:39:25.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4091789640 1 --2- 172.21.15.29:0/815829705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f408c15e9a0 0x7f408c070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3646312739 0 0) 0x7f4078003530 con 0x7f408c0cd1c0 2024-04-25T00:39:25.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f408c15e9a0 msgr2=0x7f408c070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 --2- 172.21.15.29:0/815829705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f408c15e9a0 0x7f408c070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 >> v1:172.21.15.157:6789/0 conn(0x7f408c0b4590 legacy=0x7f408c15ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c071880 con 0x7f408c0cd1c0 2024-04-25T00:39:25.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4084001a00 con 0x7f408c0cd1c0 2024-04-25T00:39:25.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 >> v1:172.21.15.29:6789/0 conn(0x7f408c0cd1c0 legacy=0x7f408c072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 --2- 172.21.15.29:0/815829705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f408c15e9a0 0x7f40780039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 --2- 172.21.15.29:0/815829705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4078004000 0x7f4078004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 --2- 172.21.15.29:0/815829705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4078004d50 0x7f40780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f408c071ed0 con 0x7f4078004000 2024-04-25T00:39:25.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f407800bb60 con 0x7f4078004d50 2024-04-25T00:39:25.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f407800bd60 con 0x7f408c15e9a0 2024-04-25T00:39:25.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4090f88640 1 --2- 172.21.15.29:0/815829705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4078004000 0x7f4078004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4091789640 1 --2- 172.21.15.29:0/815829705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f408c15e9a0 0x7f40780039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4091f8a640 1 --2- 172.21.15.29:0/815829705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4078004d50 0x7f40780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4090f88640 1 -- 172.21.15.29:0/815829705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f408c15e9a0 msgr2=0x7f40780039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4090f88640 1 --2- 172.21.15.29:0/815829705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f408c15e9a0 0x7f40780039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4090f88640 1 -- 172.21.15.29:0/815829705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4078004d50 msgr2=0x7f40780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4090f88640 1 --2- 172.21.15.29:0/815829705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4078004d50 0x7f40780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4090f88640 1 -- 172.21.15.29:0/815829705 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4078003530 con 0x7f4078004000 2024-04-25T00:39:25.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4091f8a640 1 --2- 172.21.15.29:0/815829705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4078004d50 0x7f40780051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:25.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4090f88640 1 --2- 172.21.15.29:0/815829705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4078004000 0x7f4078004810 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f407c007fd0 tx=0x7f407c01c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c519263cda0ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.250+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c0b0070 con 0x7f4078004000 2024-04-25T00:39:25.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f407c0ab620 con 0x7f4078004000 2024-04-25T00:39:25.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4083fff640 1 -- 172.21.15.29:0/815829705 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c0039b0 con 0x7f4078004000 2024-04-25T00:39:25.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 -- 172.21.15.29:0/815829705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4078004000 msgr2=0x7f4078004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 --2- 172.21.15.29:0/815829705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4078004000 0x7f4078004810 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f407c007fd0 tx=0x7f407c01c300 comp rx=0 tx=0).stop 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4091f8a640 1 -- 172.21.15.29:0/815829705 reap_dead start 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 -- 172.21.15.29:0/815829705 shutdown_connections 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 -- 172.21.15.29:0/815829705 >> 172.21.15.29:0/815829705 conn(0x7f408c0b1870 msgr2=0x7f408c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 -- 172.21.15.29:0/815829705 shutdown_connections 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 -- 172.21.15.29:0/815829705 wait complete. 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 Processor -- start 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 -- start start 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4078004000 0x7f408c142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f408c13e080 0x7f408c0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f408c13e500 0x7f408c158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f408c0cd650 con 0x7f408c13e500 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f408c15f700 con 0x7f4078004000 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f408c0cd7d0 con 0x7f408c13e080 2024-04-25T00:39:25.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4091f8a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f408c13e500 0x7f408c158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4091f8a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f408c13e500 0x7f408c158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39954/0 (socket says 172.21.15.29:39954) 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4091f8a640 1 -- 172.21.15.29:0/2289433409 learned_addr learned my addr 172.21.15.29:0/2289433409 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4091f8a640 1 -- 172.21.15.29:0/2289433409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f408c13e080 msgr2=0x7f408c0cec60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4091f8a640 1 --2- 172.21.15.29:0/2289433409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f408c13e080 0x7f408c0cec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4091f8a640 1 -- 172.21.15.29:0/2289433409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4078004000 msgr2=0x7f408c142bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4091f8a640 1 --2- 172.21.15.29:0/2289433409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4078004000 0x7f408c142bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4091f8a640 1 -- 172.21.15.29:0/2289433409 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c143d20 con 0x7f408c13e500 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4091f8a640 1 --2- 172.21.15.29:0/2289433409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f408c13e500 0x7f408c158600 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f4084000f90 tx=0x7f4084010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4081ffb640 1 -- 172.21.15.29:0/2289433409 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408401a070 con 0x7f408c13e500 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4081ffb640 1 -- 172.21.15.29:0/2289433409 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4084011040 con 0x7f408c13e500 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 -- 172.21.15.29:0/2289433409 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408c156140 con 0x7f408c13e500 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f4081ffb640 1 -- 172.21.15.29:0/2289433409 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408400ed80 con 0x7f408c13e500 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.254+0000 7f40939ed640 1 -- 172.21.15.29:0/2289433409 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408c157e00 con 0x7f408c13e500 2024-04-25T00:39:25.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.258+0000 7f40939ed640 1 -- 172.21.15.29:0/2289433409 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f408c0692e0 con 0x7f408c13e500 2024-04-25T00:39:25.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.258+0000 7f4081ffb640 1 -- 172.21.15.29:0/2289433409 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f408401d070 con 0x7f408c13e500 2024-04-25T00:39:25.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.258+0000 7f4081ffb640 1 --2- 172.21.15.29:0/2289433409 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4070067e60 0x7f407006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.258+0000 7f4091789640 1 --2- 172.21.15.29:0/2289433409 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4070067e60 0x7f407006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.258+0000 7f4081ffb640 1 -- 172.21.15.29:0/2289433409 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f4084088f90 con 0x7f408c13e500 2024-04-25T00:39:25.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.258+0000 7f4091789640 1 --2- 172.21.15.29:0/2289433409 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4070067e60 0x7f407006a320 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f408800d890 tx=0x7f4088008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.262+0000 7f4081ffb640 1 -- 172.21.15.29:0/2289433409 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f40840592d0 con 0x7f408c13e500 2024-04-25T00:39:25.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.446+0000 7f40939ed640 1 -- 172.21.15.29:0/2289433409 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f408c141b10 con 0x7f4070067e60 2024-04-25T00:39:25.449 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:25.446+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:25.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.446+0000 7f4081ffb640 1 -- 172.21.15.29:0/2289433409 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f408c141b10 con 0x7f4070067e60 2024-04-25T00:39:25.450 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:25.455 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:25.456 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2024-04-25T00:39:25.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.582+0000 7f9f1fdc7640 1 Processor -- start 2024-04-25T00:39:25.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.582+0000 7f9f1fdc7640 1 -- start start 2024-04-25T00:39:25.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1fdc7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f18070ce0 0x7f9f18072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1fdc7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f180712c0 con 0x7f9f1815e9a0 2024-04-25T00:39:25.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1fdc7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f180715a0 con 0x7f9f180b4590 2024-04-25T00:39:25.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1fdc7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f18071ed0 con 0x7f9f18070ce0 2024-04-25T00:39:25.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1db63640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9f1815e9a0 0x7f9f1815ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40562/0 (socket says 172.21.15.29:40562) 2024-04-25T00:39:25.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1db63640 1 -- 172.21.15.29:0/844076727 learned_addr learned my addr 172.21.15.29:0/844076727 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:25.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1778281553 0 0) 0x7f9f180712c0 con 0x7f9f1815e9a0 2024-04-25T00:39:25.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f04003530 con 0x7f9f1815e9a0 2024-04-25T00:39:25.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1e364640 1 --2- 172.21.15.29:0/844076727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f18070ce0 0x7f9f18072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2645813667 0 0) 0x7f9f04003530 con 0x7f9f1815e9a0 2024-04-25T00:39:25.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f18070ce0 msgr2=0x7f9f18072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 --2- 172.21.15.29:0/844076727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f18070ce0 0x7f9f18072360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 >> v1:172.21.15.157:6789/0 conn(0x7f9f180b4590 legacy=0x7f9f180cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f18071880 con 0x7f9f1815e9a0 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f9f14001680 con 0x7f9f1815e9a0 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 >> v1:172.21.15.29:6789/0 conn(0x7f9f1815e9a0 legacy=0x7f9f1815ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 --2- 172.21.15.29:0/844076727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f18070ce0 0x7f9f040039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 --2- 172.21.15.29:0/844076727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f04003f70 0x7f9f04004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 --2- 172.21.15.29:0/844076727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f04004d50 0x7f9f040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9f18071ed0 con 0x7f9f04003f70 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0400bab0 con 0x7f9f04004d50 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f0400bcb0 con 0x7f9f18070ce0 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1d362640 1 --2- 172.21.15.29:0/844076727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f04003f70 0x7f9f04004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1e364640 1 --2- 172.21.15.29:0/844076727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f04004d50 0x7f9f040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1db63640 1 --2- 172.21.15.29:0/844076727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f18070ce0 0x7f9f040039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1d362640 1 -- 172.21.15.29:0/844076727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f18070ce0 msgr2=0x7f9f040039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1d362640 1 --2- 172.21.15.29:0/844076727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f18070ce0 0x7f9f040039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1d362640 1 -- 172.21.15.29:0/844076727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f04004d50 msgr2=0x7f9f040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1d362640 1 --2- 172.21.15.29:0/844076727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f04004d50 0x7f9f040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1d362640 1 -- 172.21.15.29:0/844076727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f04003530 con 0x7f9f04003f70 2024-04-25T00:39:25.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.586+0000 7f9f1db63640 1 --2- 172.21.15.29:0/844076727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f18070ce0 0x7f9f040039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:25.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1d362640 1 --2- 172.21.15.29:0/844076727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f04003f70 0x7f9f04004810 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f9f08001780 tx=0x7f9f0801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4e23252892cb84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f080013a0 con 0x7f9f04003f70 2024-04-25T00:39:25.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f080a9070 con 0x7f9f04003f70 2024-04-25T00:39:25.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1cb61640 1 -- 172.21.15.29:0/844076727 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08003600 con 0x7f9f04003f70 2024-04-25T00:39:25.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 -- 172.21.15.29:0/844076727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f04003f70 msgr2=0x7f9f04004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 --2- 172.21.15.29:0/844076727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f04003f70 0x7f9f04004810 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f9f08001780 tx=0x7f9f0801c420 comp rx=0 tx=0).stop 2024-04-25T00:39:25.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1e364640 1 -- 172.21.15.29:0/844076727 reap_dead start 2024-04-25T00:39:25.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 -- 172.21.15.29:0/844076727 shutdown_connections 2024-04-25T00:39:25.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 -- 172.21.15.29:0/844076727 >> 172.21.15.29:0/844076727 conn(0x7f9f180b1870 msgr2=0x7f9f180b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:25.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 -- 172.21.15.29:0/844076727 shutdown_connections 2024-04-25T00:39:25.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 -- 172.21.15.29:0/844076727 wait complete. 2024-04-25T00:39:25.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 Processor -- start 2024-04-25T00:39:25.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 -- start start 2024-04-25T00:39:25.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f04003f70 0x7f9f181f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f1813ce10 0x7f9f180cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f180cd6c0 0x7f9f180cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9f18144870 con 0x7f9f04003f70 2024-04-25T00:39:25.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1815f700 con 0x7f9f1813ce10 2024-04-25T00:39:25.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1fdc7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f181449f0 con 0x7f9f180cd6c0 2024-04-25T00:39:25.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1db63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f04003f70 0x7f9f181f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1db63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f04003f70 0x7f9f181f1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39970/0 (socket says 172.21.15.29:39970) 2024-04-25T00:39:25.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.590+0000 7f9f1db63640 1 -- 172.21.15.29:0/3711994581 learned_addr learned my addr 172.21.15.29:0/3711994581 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:25.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f1db63640 1 -- 172.21.15.29:0/3711994581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f180cd6c0 msgr2=0x7f9f180cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f1db63640 1 --2- 172.21.15.29:0/3711994581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f180cd6c0 0x7f9f180cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f1db63640 1 -- 172.21.15.29:0/3711994581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f1813ce10 msgr2=0x7f9f180cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f1db63640 1 --2- 172.21.15.29:0/3711994581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f1813ce10 0x7f9f180cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f1db63640 1 -- 172.21.15.29:0/3711994581 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1815a9a0 con 0x7f9f04003f70 2024-04-25T00:39:25.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f1db63640 1 --2- 172.21.15.29:0/3711994581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f04003f70 0x7f9f181f1eb0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f9f14010bc0 tx=0x7f9f1400e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f0e7fc640 1 -- 172.21.15.29:0/3711994581 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1401b070 con 0x7f9f04003f70 2024-04-25T00:39:25.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f0e7fc640 1 -- 172.21.15.29:0/3711994581 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f14011040 con 0x7f9f04003f70 2024-04-25T00:39:25.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f1fdc7640 1 -- 172.21.15.29:0/3711994581 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1813acf0 con 0x7f9f04003f70 2024-04-25T00:39:25.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f0e7fc640 1 -- 172.21.15.29:0/3711994581 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f140143a0 con 0x7f9f04003f70 2024-04-25T00:39:25.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f1fdc7640 1 -- 172.21.15.29:0/3711994581 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1815a3e0 con 0x7f9f04003f70 2024-04-25T00:39:25.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f1fdc7640 1 -- 172.21.15.29:0/3711994581 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f180692e0 con 0x7f9f04003f70 2024-04-25T00:39:25.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f0e7fc640 1 -- 172.21.15.29:0/3711994581 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f9f1401e070 con 0x7f9f04003f70 2024-04-25T00:39:25.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f0e7fc640 1 --2- 172.21.15.29:0/3711994581 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9efc067e60 0x7f9efc06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f0e7fc640 1 -- 172.21.15.29:0/3711994581 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f9f140891a0 con 0x7f9f04003f70 2024-04-25T00:39:25.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f1d362640 1 --2- 172.21.15.29:0/3711994581 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9efc067e60 0x7f9efc06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.594+0000 7f9f1d362640 1 --2- 172.21.15.29:0/3711994581 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9efc067e60 0x7f9efc06a320 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9f08001750 tx=0x7f9f08001130 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.598+0000 7f9f0e7fc640 1 -- 172.21.15.29:0/3711994581 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9f140594e0 con 0x7f9f04003f70 2024-04-25T00:39:25.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.794+0000 7f9f1fdc7640 1 -- 172.21.15.29:0/3711994581 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f9f180b3c40 con 0x7f9efc067e60 2024-04-25T00:39:25.797 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:25.794+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.794+0000 7f9f0e7fc640 1 -- 172.21.15.29:0/3711994581 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f9f180b3c40 con 0x7f9efc067e60 2024-04-25T00:39:25.797 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:25.803 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:25.803 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2024-04-25T00:39:25.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb88bd5640 1 Processor -- start 2024-04-25T00:39:25.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb88bd5640 1 -- start start 2024-04-25T00:39:25.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb88bd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb8406d450 0x7fbb840b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb88bd5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb840cf670 con 0x7fbb841456d0 2024-04-25T00:39:25.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb88bd5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb840b4380 con 0x7fbb840b3b50 2024-04-25T00:39:25.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb88bd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbb840b40a0 con 0x7fbb8406d450 2024-04-25T00:39:25.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8259c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbb841456d0 0x7fbb840cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40578/0 (socket says 172.21.15.29:40578) 2024-04-25T00:39:25.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8259c640 1 -- 172.21.15.29:0/3593786185 learned_addr learned my addr 172.21.15.29:0/3593786185 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:25.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb82d9d640 1 --2- 172.21.15.29:0/3593786185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb8406d450 0x7fbb840b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2709455195 0 0) 0x7fbb840cf670 con 0x7fbb841456d0 2024-04-25T00:39:25.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb6c003530 con 0x7fbb841456d0 2024-04-25T00:39:25.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3134112310 0 0) 0x7fbb6c003530 con 0x7fbb841456d0 2024-04-25T00:39:25.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb8406d450 msgr2=0x7fbb840b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 --2- 172.21.15.29:0/3593786185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb8406d450 0x7fbb840b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 >> v1:172.21.15.157:6789/0 conn(0x7fbb840b3b50 legacy=0x7fbb8406cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb840ce480 con 0x7fbb841456d0 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fbb74001680 con 0x7fbb841456d0 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 >> v1:172.21.15.29:6789/0 conn(0x7fbb841456d0 legacy=0x7fbb840cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 --2- 172.21.15.29:0/3593786185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb6c0039f0 0x7fbb6c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 --2- 172.21.15.29:0/3593786185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb6c004810 0x7fbb6c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 --2- 172.21.15.29:0/3593786185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb8406d450 0x7fbb6c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb840b40a0 con 0x7fbb6c0039f0 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c00bb00 con 0x7fbb6c004810 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6c00bd00 con 0x7fbb8406d450 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8259c640 1 --2- 172.21.15.29:0/3593786185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb6c0039f0 0x7fbb6c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb81d9b640 1 --2- 172.21.15.29:0/3593786185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb6c004810 0x7fbb6c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb82d9d640 1 --2- 172.21.15.29:0/3593786185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb8406d450 0x7fbb6c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8259c640 1 -- 172.21.15.29:0/3593786185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb8406d450 msgr2=0x7fbb6c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8259c640 1 --2- 172.21.15.29:0/3593786185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb8406d450 0x7fbb6c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8259c640 1 -- 172.21.15.29:0/3593786185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb6c004810 msgr2=0x7fbb6c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8259c640 1 --2- 172.21.15.29:0/3593786185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb6c004810 0x7fbb6c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.930+0000 7fbb8259c640 1 -- 172.21.15.29:0/3593786185 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6c003530 con 0x7fbb6c0039f0 2024-04-25T00:39:25.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.934+0000 7fbb8259c640 1 --2- 172.21.15.29:0/3593786185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb6c0039f0 0x7fbb6c003e10 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400c240 tx=0x7fbb7401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0f805181289fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.934+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74007d20 con 0x7fbb6c0039f0 2024-04-25T00:39:25.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.934+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb740a9070 con 0x7fbb6c0039f0 2024-04-25T00:39:25.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.934+0000 7fbb8159a640 1 -- 172.21.15.29:0/3593786185 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74005d60 con 0x7fbb6c0039f0 2024-04-25T00:39:25.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.934+0000 7fbb88bd5640 1 -- 172.21.15.29:0/3593786185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb6c0039f0 msgr2=0x7fbb6c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.934+0000 7fbb88bd5640 1 --2- 172.21.15.29:0/3593786185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb6c0039f0 0x7fbb6c003e10 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400c240 tx=0x7fbb7401cc80 comp rx=0 tx=0).stop 2024-04-25T00:39:25.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.934+0000 7fbb82d9d640 1 -- 172.21.15.29:0/3593786185 reap_dead start 2024-04-25T00:39:25.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.934+0000 7fbb88bd5640 1 -- 172.21.15.29:0/3593786185 shutdown_connections 2024-04-25T00:39:25.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.934+0000 7fbb88bd5640 1 -- 172.21.15.29:0/3593786185 >> 172.21.15.29:0/3593786185 conn(0x7fbb840b1870 msgr2=0x7fbb840b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:25.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.934+0000 7fbb88bd5640 1 -- 172.21.15.29:0/3593786185 shutdown_connections 2024-04-25T00:39:25.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.934+0000 7fbb88bd5640 1 -- 172.21.15.29:0/3593786185 wait complete. 2024-04-25T00:39:25.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.934+0000 7fbb88bd5640 1 Processor -- start 2024-04-25T00:39:25.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.934+0000 7fbb88bd5640 1 -- start start 2024-04-25T00:39:25.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb88bd5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb6c0039f0 0x7fbb840aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb88bd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb841598d0 0x7fbb840ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb88bd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb840adf20 0x7fbb841405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb88bd5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84167b70 con 0x7fbb6c0039f0 2024-04-25T00:39:25.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb88bd5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb840b5000 con 0x7fbb841598d0 2024-04-25T00:39:25.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb88bd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbb84167cf0 con 0x7fbb840adf20 2024-04-25T00:39:25.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb8259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb6c0039f0 0x7fbb840aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb8259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb6c0039f0 0x7fbb840aa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39988/0 (socket says 172.21.15.29:39988) 2024-04-25T00:39:25.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb8259c640 1 -- 172.21.15.29:0/417820867 learned_addr learned my addr 172.21.15.29:0/417820867 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:25.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb81d9b640 1 --2- 172.21.15.29:0/417820867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb841598d0 0x7fbb840ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb8259c640 1 -- 172.21.15.29:0/417820867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb840adf20 msgr2=0x7fbb841405e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:39:25.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb8259c640 1 --2- 172.21.15.29:0/417820867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb840adf20 0x7fbb841405e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb8259c640 1 -- 172.21.15.29:0/417820867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb841598d0 msgr2=0x7fbb840ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb8259c640 1 --2- 172.21.15.29:0/417820867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb841598d0 0x7fbb840ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb8259c640 1 -- 172.21.15.29:0/417820867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb841f8a70 con 0x7fbb6c0039f0 2024-04-25T00:39:25.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb81d9b640 1 --2- 172.21.15.29:0/417820867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb841598d0 0x7fbb840ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:25.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb8259c640 1 --2- 172.21.15.29:0/417820867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb6c0039f0 0x7fbb840aa4e0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400c240 tx=0x7fbb6c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb72ffd640 1 -- 172.21.15.29:0/417820867 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74007d20 con 0x7fbb6c0039f0 2024-04-25T00:39:25.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb72ffd640 1 -- 172.21.15.29:0/417820867 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb74005d40 con 0x7fbb6c0039f0 2024-04-25T00:39:25.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb88bd5640 1 -- 172.21.15.29:0/417820867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb841f8d50 con 0x7fbb6c0039f0 2024-04-25T00:39:25.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb72ffd640 1 -- 172.21.15.29:0/417820867 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74001640 con 0x7fbb6c0039f0 2024-04-25T00:39:25.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb88bd5640 1 -- 172.21.15.29:0/417820867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb841f9310 con 0x7fbb6c0039f0 2024-04-25T00:39:25.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb88bd5640 1 -- 172.21.15.29:0/417820867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb840692e0 con 0x7fbb6c0039f0 2024-04-25T00:39:25.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb72ffd640 1 -- 172.21.15.29:0/417820867 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fbb74007090 con 0x7fbb6c0039f0 2024-04-25T00:39:25.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb72ffd640 1 --2- 172.21.15.29:0/417820867 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbb64067e90 0x7fbb6406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb72ffd640 1 -- 172.21.15.29:0/417820867 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7fbb7411f970 con 0x7fbb6c0039f0 2024-04-25T00:39:25.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb81d9b640 1 --2- 172.21.15.29:0/417820867 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbb64067e90 0x7fbb6406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.938+0000 7fbb81d9b640 1 --2- 172.21.15.29:0/417820867 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbb64067e90 0x7fbb6406a350 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbb78000c10 tx=0x7fbb78009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:25.942+0000 7fbb72ffd640 1 -- 172.21.15.29:0/417820867 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbb740bf060 con 0x7fbb6c0039f0 2024-04-25T00:39:26.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.130+0000 7fbb88bd5640 1 -- 172.21.15.29:0/417820867 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fbb840b4de0 con 0x7fbb64067e90 2024-04-25T00:39:26.133 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:26.130+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.130+0000 7fbb72ffd640 1 -- 172.21.15.29:0/417820867 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fbb840b4de0 con 0x7fbb64067e90 2024-04-25T00:39:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:26.139 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:26.140 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2024-04-25T00:39:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97d5e58640 1 Processor -- start 2024-04-25T00:39:26.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97d5e58640 1 -- start start 2024-04-25T00:39:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97d5e58640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f97d0161750 0x7f97d00b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97d5e58640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97d00d46e0 0x7f97d00d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97d5e58640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d00d1970 con 0x7f97d0161b30 2024-04-25T00:39:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97d5e58640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f97d00d1460 con 0x7f97d0161750 2024-04-25T00:39:26.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97d5e58640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f97d00cf720 con 0x7f97d00d46e0 2024-04-25T00:39:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97cf7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f97d0161750 0x7f97d00b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97cf7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f97d0161750 0x7f97d00b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39992/0 (socket says 172.21.15.29:39992) 2024-04-25T00:39:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97cf7fe640 1 -- 172.21.15.29:0/3461286214 learned_addr learned my addr 172.21.15.29:0/3461286214 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97cf7fe640 1 -- 172.21.15.29:0/3461286214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97d00d46e0 msgr2=0x7f97d00d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97cffff640 1 --2- 172.21.15.29:0/3461286214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97d00d46e0 0x7f97d00d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97ce7fc640 1 -- 172.21.15.29:0/3461286214 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 346034348 0 0) 0x7f97d00d1970 con 0x7f97d0161b30 2024-04-25T00:39:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97cf7fe640 1 --2- 172.21.15.29:0/3461286214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97d00d46e0 0x7f97d00d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97cf7fe640 1 -- 172.21.15.29:0/3461286214 >> v1:172.21.15.29:6789/0 conn(0x7f97d0161b30 legacy=0x7f97d00d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97cf7fe640 1 -- 172.21.15.29:0/3461286214 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d00d1690 con 0x7f97d0161750 2024-04-25T00:39:26.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.266+0000 7f97cffff640 1 --2- 172.21.15.29:0/3461286214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97d00d46e0 0x7f97d00d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.270+0000 7f97cf7fe640 1 --2- 172.21.15.29:0/3461286214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97d0161750 0x7f97d00b4e80 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009870 tx=0x7f97c001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1a247f363c28cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.270+0000 7f97ce7fc640 1 -- 172.21.15.29:0/3461286214 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c00b0070 con 0x7f97d0161750 2024-04-25T00:39:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.270+0000 7f97ce7fc640 1 -- 172.21.15.29:0/3461286214 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c00ab600 con 0x7f97d0161750 2024-04-25T00:39:26.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.270+0000 7f97ce7fc640 1 -- 172.21.15.29:0/3461286214 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c00ab9d0 con 0x7f97d0161750 2024-04-25T00:39:26.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.270+0000 7f97d5e58640 1 -- 172.21.15.29:0/3461286214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97d0161750 msgr2=0x7f97d00b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:26.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.270+0000 7f97d5e58640 1 --2- 172.21.15.29:0/3461286214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97d0161750 0x7f97d00b4e80 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009870 tx=0x7f97c001c460 comp rx=0 tx=0).stop 2024-04-25T00:39:26.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.270+0000 7f97d5e58640 1 -- 172.21.15.29:0/3461286214 shutdown_connections 2024-04-25T00:39:26.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.270+0000 7f97d5e58640 1 --2- 172.21.15.29:0/3461286214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97d00d46e0 0x7f97d00d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.270+0000 7f97d5e58640 1 --2- 172.21.15.29:0/3461286214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97d0161750 0x7f97d00b4e80 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.270+0000 7f97d5e58640 1 -- 172.21.15.29:0/3461286214 >> 172.21.15.29:0/3461286214 conn(0x7f97d009d2c0 msgr2=0x7f97d00d3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:26.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.270+0000 7f97d5e58640 1 -- 172.21.15.29:0/3461286214 shutdown_connections 2024-04-25T00:39:26.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.270+0000 7f97d5e58640 1 -- 172.21.15.29:0/3461286214 wait complete. 2024-04-25T00:39:26.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97d5e58640 1 Processor -- start 2024-04-25T00:39:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97d5e58640 1 -- start start 2024-04-25T00:39:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97d5e58640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97d00d46e0 0x7f97d00ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97d5e58640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97d0161750 0x7f97d014dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97d5e58640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d0161b30 0x7f97d0158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97d5e58640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f97d00b7ac0 con 0x7f97d0161750 2024-04-25T00:39:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97d5e58640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f97d00b7940 con 0x7f97d0161b30 2024-04-25T00:39:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97d5e58640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f97d00b7c40 con 0x7f97d00d46e0 2024-04-25T00:39:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97ceffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97d0161750 0x7f97d014dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97ceffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97d0161750 0x7f97d014dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40000/0 (socket says 172.21.15.29:40000) 2024-04-25T00:39:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97ceffd640 1 -- 172.21.15.29:0/821720687 learned_addr learned my addr 172.21.15.29:0/821720687 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:26.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97cf7fe640 1 --2- 172.21.15.29:0/821720687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97d00d46e0 0x7f97d00ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97cffff640 1 --2- 172.21.15.29:0/821720687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d0161b30 0x7f97d0158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97ceffd640 1 -- 172.21.15.29:0/821720687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97d00d46e0 msgr2=0x7f97d00ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97ceffd640 1 --2- 172.21.15.29:0/821720687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97d00d46e0 0x7f97d00ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97ceffd640 1 -- 172.21.15.29:0/821720687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d0161b30 msgr2=0x7f97d0158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97ceffd640 1 --2- 172.21.15.29:0/821720687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d0161b30 0x7f97d0158850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97ceffd640 1 -- 172.21.15.29:0/821720687 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d014fe50 con 0x7f97d0161750 2024-04-25T00:39:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97ceffd640 1 --2- 172.21.15.29:0/821720687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97d0161750 0x7f97d014dfb0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f97c400c3c0 tx=0x7f97c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97b7fff640 1 -- 172.21.15.29:0/821720687 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4017070 con 0x7f97d0161750 2024-04-25T00:39:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97b7fff640 1 -- 172.21.15.29:0/821720687 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c4003600 con 0x7f97d0161750 2024-04-25T00:39:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97d5e58640 1 -- 172.21.15.29:0/821720687 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d014f390 con 0x7f97d0161750 2024-04-25T00:39:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97b7fff640 1 -- 172.21.15.29:0/821720687 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c40039f0 con 0x7f97d0161750 2024-04-25T00:39:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97d5e58640 1 -- 172.21.15.29:0/821720687 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d014f950 con 0x7f97d0161750 2024-04-25T00:39:26.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97d5e58640 1 -- 172.21.15.29:0/821720687 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d014fb50 con 0x7f97d0161750 2024-04-25T00:39:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97b7fff640 1 -- 172.21.15.29:0/821720687 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f97c401a070 con 0x7f97d0161750 2024-04-25T00:39:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97b7fff640 1 --2- 172.21.15.29:0/821720687 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f97bc067e60 0x7f97bc06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97b7fff640 1 -- 172.21.15.29:0/821720687 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f97c4085a10 con 0x7f97d0161750 2024-04-25T00:39:26.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.274+0000 7f97cf7fe640 1 --2- 172.21.15.29:0/821720687 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f97bc067e60 0x7f97bc06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.278+0000 7f97cf7fe640 1 --2- 172.21.15.29:0/821720687 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f97bc067e60 0x7f97bc06a320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f97c00ad000 tx=0x7f97c00a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:26.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.278+0000 7f97b7fff640 1 -- 172.21.15.29:0/821720687 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f97d014fb50 con 0x7f97d0161750 2024-04-25T00:39:26.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.470+0000 7f97d5e58640 1 -- 172.21.15.29:0/821720687 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f97d0150130 con 0x7f97bc067e60 2024-04-25T00:39:26.475 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:26.470+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:26.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.474+0000 7f97b7fff640 1 -- 172.21.15.29:0/821720687 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f97d0150130 con 0x7f97bc067e60 2024-04-25T00:39:26.475 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:26.482 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:26.483 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2024-04-25T00:39:26.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe08cd6d640 1 Processor -- start 2024-04-25T00:39:26.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe08cd6d640 1 -- start start 2024-04-25T00:39:26.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe08cd6d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0880b3140 0x7fe08806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe08cd6d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe0880b4b10 0x7fe0880b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe08cd6d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0880b41e0 con 0x7fe0880b3a60 2024-04-25T00:39:26.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe08cd6d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe08806d960 con 0x7fe0880b4b10 2024-04-25T00:39:26.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe08cd6d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe08806dc40 con 0x7fe0880b3140 2024-04-25T00:39:26.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe07dd9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe0880b4b10 0x7fe0880b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe07dd9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe0880b4b10 0x7fe0880b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40010/0 (socket says 172.21.15.29:40010) 2024-04-25T00:39:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe07dd9b640 1 -- 172.21.15.29:0/735893007 learned_addr learned my addr 172.21.15.29:0/735893007 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe08659c640 1 --2- 172.21.15.29:0/735893007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0880b3140 0x7fe08806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe07dd9b640 1 -- 172.21.15.29:0/735893007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0880b3140 msgr2=0x7fe08806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe07dd9b640 1 --2- 172.21.15.29:0/735893007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0880b3140 0x7fe08806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe07dd9b640 1 -- 172.21.15.29:0/735893007 >> v1:172.21.15.157:6789/0 conn(0x7fe0880b3a60 legacy=0x7fe08815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.606+0000 7fe07dd9b640 1 -- 172.21.15.29:0/735893007 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08806df20 con 0x7fe0880b4b10 2024-04-25T00:39:26.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe07dd9b640 1 --2- 172.21.15.29:0/735893007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0880b4b10 0x7fe0880b3520 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fe070009870 tx=0x7fe07001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2306350a4d19a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:26.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe085d9b640 1 -- 172.21.15.29:0/735893007 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0700b0070 con 0x7fe0880b4b10 2024-04-25T00:39:26.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe085d9b640 1 -- 172.21.15.29:0/735893007 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0700ab600 con 0x7fe0880b4b10 2024-04-25T00:39:26.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe085d9b640 1 -- 172.21.15.29:0/735893007 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0700a4e60 con 0x7fe0880b4b10 2024-04-25T00:39:26.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe08cd6d640 1 -- 172.21.15.29:0/735893007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0880b4b10 msgr2=0x7fe0880b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:26.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe08cd6d640 1 --2- 172.21.15.29:0/735893007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0880b4b10 0x7fe0880b3520 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fe070009870 tx=0x7fe07001c520 comp rx=0 tx=0).stop 2024-04-25T00:39:26.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe08cd6d640 1 -- 172.21.15.29:0/735893007 shutdown_connections 2024-04-25T00:39:26.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe08cd6d640 1 --2- 172.21.15.29:0/735893007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0880b4b10 0x7fe0880b3520 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe08cd6d640 1 --2- 172.21.15.29:0/735893007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0880b3140 0x7fe08806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe08cd6d640 1 -- 172.21.15.29:0/735893007 >> 172.21.15.29:0/735893007 conn(0x7fe0880b1870 msgr2=0x7fe0880b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:26.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe08cd6d640 1 -- 172.21.15.29:0/735893007 shutdown_connections 2024-04-25T00:39:26.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe08cd6d640 1 -- 172.21.15.29:0/735893007 wait complete. 2024-04-25T00:39:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe08cd6d640 1 Processor -- start 2024-04-25T00:39:26.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.610+0000 7fe08cd6d640 1 -- start start 2024-04-25T00:39:26.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08cd6d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0880b3140 0x7fe088133e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08cd6d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0880b3a60 0x7fe0881343c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08cd6d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0880b4b10 0x7fe08815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08cd6d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe088168b70 con 0x7fe0880b3140 2024-04-25T00:39:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08cd6d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881689f0 con 0x7fe0880b4b10 2024-04-25T00:39:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08cd6d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe088168cf0 con 0x7fe0880b3a60 2024-04-25T00:39:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0880b3140 0x7fe088133e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0880b3140 0x7fe088133e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40012/0 (socket says 172.21.15.29:40012) 2024-04-25T00:39:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08659c640 1 -- 172.21.15.29:0/4031428609 learned_addr learned my addr 172.21.15.29:0/4031428609 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe086d9d640 1 --2- 172.21.15.29:0/4031428609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0880b4b10 0x7fe08815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe07dd9b640 1 --2- 172.21.15.29:0/4031428609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0880b3a60 0x7fe0881343c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08659c640 1 -- 172.21.15.29:0/4031428609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0880b3a60 msgr2=0x7fe0881343c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08659c640 1 --2- 172.21.15.29:0/4031428609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0880b3a60 0x7fe0881343c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08659c640 1 -- 172.21.15.29:0/4031428609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0880b4b10 msgr2=0x7fe08815b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08659c640 1 --2- 172.21.15.29:0/4031428609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0880b4b10 0x7fe08815b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08659c640 1 -- 172.21.15.29:0/4031428609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08813b9c0 con 0x7fe0880b3140 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe086d9d640 1 --2- 172.21.15.29:0/4031428609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0880b4b10 0x7fe08815b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08659c640 1 --2- 172.21.15.29:0/4031428609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0880b3140 0x7fe088133e80 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fe07800d5f0 tx=0x7fe07800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe07f7fe640 1 -- 172.21.15.29:0/4031428609 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780125f0 con 0x7fe0880b3140 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe07f7fe640 1 -- 172.21.15.29:0/4031428609 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe078019070 con 0x7fe0880b3140 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe07f7fe640 1 -- 172.21.15.29:0/4031428609 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780129b0 con 0x7fe0880b3140 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08cd6d640 1 -- 172.21.15.29:0/4031428609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08813b400 con 0x7fe0880b3140 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08cd6d640 1 -- 172.21.15.29:0/4031428609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe08813cee0 con 0x7fe0880b3140 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe08cd6d640 1 -- 172.21.15.29:0/4031428609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe088069270 con 0x7fe0880b3140 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe07f7fe640 1 -- 172.21.15.29:0/4031428609 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fe078012b50 con 0x7fe0880b3140 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe07f7fe640 1 --2- 172.21.15.29:0/4031428609 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe068067e60 0x7fe06806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe07dd9b640 1 --2- 172.21.15.29:0/4031428609 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe068067e60 0x7fe06806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe07f7fe640 1 -- 172.21.15.29:0/4031428609 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7fe078086ad0 con 0x7fe0880b3140 2024-04-25T00:39:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.614+0000 7fe07dd9b640 1 --2- 172.21.15.29:0/4031428609 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe068067e60 0x7fe06806a320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe0700ad000 tx=0x7fe0700a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:26.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.618+0000 7fe07f7fe640 1 -- 172.21.15.29:0/4031428609 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe078056e10 con 0x7fe0880b3140 2024-04-25T00:39:26.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.806+0000 7fe08cd6d640 1 -- 172.21.15.29:0/4031428609 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7fe08813fc30 con 0x7fe068067e60 2024-04-25T00:39:26.820 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:26.818+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:39:26.820 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:26.818+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:39:26.820 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:26.818+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:39:26.820 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:26.818+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:39:26.820 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:26.818+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:39:26.825 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:26.822+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:26.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.822+0000 7fe07f7fe640 1 -- 172.21.15.29:0/4031428609 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fe08813fc30 con 0x7fe068067e60 2024-04-25T00:39:26.826 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:26.830 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:26.831 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2024-04-25T00:39:26.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.946+0000 7fbae4881640 1 Processor -- start 2024-04-25T00:39:26.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.946+0000 7fbae4881640 1 -- start start 2024-04-25T00:39:26.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbae4881640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbae00b4d30 0x7fbae00b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbae4881640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbae006d210 0x7fbae00ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbae4881640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbae006d6c0 con 0x7fbae015df70 2024-04-25T00:39:26.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbae4881640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbae006d980 con 0x7fbae00b4d30 2024-04-25T00:39:26.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbae4881640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbae006dc60 con 0x7fbae006d210 2024-04-25T00:39:26.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbad5d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbae00b4d30 0x7fbae00b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbad5d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbae00b4d30 0x7fbae00b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40044/0 (socket says 172.21.15.29:40044) 2024-04-25T00:39:26.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbad5d9b640 1 -- 172.21.15.29:0/3000139818 learned_addr learned my addr 172.21.15.29:0/3000139818 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:26.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbad5d9b640 1 -- 172.21.15.29:0/3000139818 >> v2:172.21.15.157:3300/0 conn(0x7fbae006d210 msgr2=0x7fbae00ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:26.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbaded9d640 1 --2- 172.21.15.29:0/3000139818 >> v2:172.21.15.157:3300/0 conn(0x7fbae006d210 0x7fbae00ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbad5d9b640 1 --2- 172.21.15.29:0/3000139818 >> v2:172.21.15.157:3300/0 conn(0x7fbae006d210 0x7fbae00ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbad5d9b640 1 -- 172.21.15.29:0/3000139818 >> v1:172.21.15.157:6789/0 conn(0x7fbae015df70 legacy=0x7fbae00b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:26.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbad5d9b640 1 -- 172.21.15.29:0/3000139818 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae006df40 con 0x7fbae00b4d30 2024-04-25T00:39:26.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbad5d9b640 1 --2- 172.21.15.29:0/3000139818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae00b4d30 0x7fbae00b3e40 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fbae0071d10 tx=0x7fbac801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a372ffa11818b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:26.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbaddd9b640 1 -- 172.21.15.29:0/3000139818 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac80aa610 con 0x7fbae00b4d30 2024-04-25T00:39:26.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbaddd9b640 1 -- 172.21.15.29:0/3000139818 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac80af070 con 0x7fbae00b4d30 2024-04-25T00:39:26.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.950+0000 7fbaddd9b640 1 -- 172.21.15.29:0/3000139818 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac80aa9d0 con 0x7fbae00b4d30 2024-04-25T00:39:26.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 -- 172.21.15.29:0/3000139818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae00b4d30 msgr2=0x7fbae00b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:26.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 --2- 172.21.15.29:0/3000139818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae00b4d30 0x7fbae00b3e40 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fbae0071d10 tx=0x7fbac801c4b0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 -- 172.21.15.29:0/3000139818 shutdown_connections 2024-04-25T00:39:26.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 --2- 172.21.15.29:0/3000139818 >> v2:172.21.15.157:3300/0 conn(0x7fbae006d210 0x7fbae00ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 --2- 172.21.15.29:0/3000139818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae00b4d30 0x7fbae00b3e40 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 -- 172.21.15.29:0/3000139818 >> 172.21.15.29:0/3000139818 conn(0x7fbae00b1870 msgr2=0x7fbae00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:26.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 -- 172.21.15.29:0/3000139818 shutdown_connections 2024-04-25T00:39:26.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 -- 172.21.15.29:0/3000139818 wait complete. 2024-04-25T00:39:26.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 Processor -- start 2024-04-25T00:39:26.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 -- start start 2024-04-25T00:39:26.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae006d210 0x7fbae01424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbae00b4d30 0x7fbae00cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae015df70 0x7fbae00cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbae006eba0 con 0x7fbae006d210 2024-04-25T00:39:26.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbae006ea20 con 0x7fbae015df70 2024-04-25T00:39:26.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbae00ae5f0 con 0x7fbae00b4d30 2024-04-25T00:39:26.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbade59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae006d210 0x7fbae01424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbade59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae006d210 0x7fbae01424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40046/0 (socket says 172.21.15.29:40046) 2024-04-25T00:39:26.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbade59c640 1 -- 172.21.15.29:0/3314044836 learned_addr learned my addr 172.21.15.29:0/3314044836 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbad5d9b640 1 --2- 172.21.15.29:0/3314044836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbae00b4d30 0x7fbae00cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbaded9d640 1 --2- 172.21.15.29:0/3314044836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae015df70 0x7fbae00cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbade59c640 1 -- 172.21.15.29:0/3314044836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbae00b4d30 msgr2=0x7fbae00cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbade59c640 1 --2- 172.21.15.29:0/3314044836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbae00b4d30 0x7fbae00cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbade59c640 1 -- 172.21.15.29:0/3314044836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae015df70 msgr2=0x7fbae00cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbade59c640 1 --2- 172.21.15.29:0/3314044836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae015df70 0x7fbae00cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbade59c640 1 -- 172.21.15.29:0/3314044836 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae0157440 con 0x7fbae006d210 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbad5d9b640 1 --2- 172.21.15.29:0/3314044836 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbae00b4d30 0x7fbae00cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbade59c640 1 --2- 172.21.15.29:0/3314044836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae006d210 0x7fbae01424b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fbad000b410 tx=0x7fbad000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbad77fe640 1 -- 172.21.15.29:0/3314044836 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0015030 con 0x7fbae006d210 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbad77fe640 1 -- 172.21.15.29:0/3314044836 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad0004020 con 0x7fbae006d210 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 -- 172.21.15.29:0/3314044836 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae0157720 con 0x7fbae006d210 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbad77fe640 1 -- 172.21.15.29:0/3314044836 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0004450 con 0x7fbae006d210 2024-04-25T00:39:26.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.954+0000 7fbae4881640 1 -- 172.21.15.29:0/3314044836 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae0157ce0 con 0x7fbae006d210 2024-04-25T00:39:26.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.958+0000 7fbae4881640 1 -- 172.21.15.29:0/3314044836 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae0069270 con 0x7fbae006d210 2024-04-25T00:39:26.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.958+0000 7fbad77fe640 1 -- 172.21.15.29:0/3314044836 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fbad0018070 con 0x7fbae006d210 2024-04-25T00:39:26.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.958+0000 7fbad77fe640 1 --2- 172.21.15.29:0/3314044836 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbac0067e60 0x7fbac006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:26.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.958+0000 7fbad77fe640 1 -- 172.21.15.29:0/3314044836 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7fbad00862c0 con 0x7fbae006d210 2024-04-25T00:39:26.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.958+0000 7fbad5d9b640 1 --2- 172.21.15.29:0/3314044836 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbac0067e60 0x7fbac006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:26.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.958+0000 7fbad5d9b640 1 --2- 172.21.15.29:0/3314044836 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbac0067e60 0x7fbac006a320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbac80ae000 tx=0x7fbac8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:26.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:26.962+0000 7fbad77fe640 1 -- 172.21.15.29:0/3314044836 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbad0056600 con 0x7fbae006d210 2024-04-25T00:39:27.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.158+0000 7fbae4881640 1 -- 172.21.15.29:0/3314044836 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7fbae01429f0 con 0x7fbac0067e60 2024-04-25T00:39:27.162 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:27.158+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:27.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.158+0000 7fbad77fe640 1 -- 172.21.15.29:0/3314044836 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fbae01429f0 con 0x7fbac0067e60 2024-04-25T00:39:27.163 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:27.168 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:27.168 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2024-04-25T00:39:27.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.290+0000 7f1462bae640 1 Processor -- start 2024-04-25T00:39:27.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.290+0000 7f1462bae640 1 -- start start 2024-04-25T00:39:27.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.290+0000 7f1462bae640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f145c15ce60 0x7f145c06b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.290+0000 7f1462bae640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f145c06b750 0x7f145c0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.290+0000 7f1462bae640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f145c160190 con 0x7f145c156420 2024-04-25T00:39:27.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.290+0000 7f1462bae640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f145c06bb80 con 0x7f145c15ce60 2024-04-25T00:39:27.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.290+0000 7f1462bae640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f145c15c680 con 0x7f145c06b750 2024-04-25T00:39:27.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.290+0000 7f146094a640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f145c156420 0x7f145c15fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40592/0 (socket says 172.21.15.29:40592) 2024-04-25T00:39:27.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.290+0000 7f146094a640 1 -- 172.21.15.29:0/252994653 learned_addr learned my addr 172.21.15.29:0/252994653 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:27.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.290+0000 7f145bfff640 1 --2- 172.21.15.29:0/252994653 >> v2:172.21.15.29:3300/0 conn(0x7f145c15ce60 0x7f145c06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.290+0000 7f146114b640 1 --2- 172.21.15.29:0/252994653 >> v2:172.21.15.157:3300/0 conn(0x7f145c06b750 0x7f145c0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.290+0000 7f145b7fe640 1 -- 172.21.15.29:0/252994653 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2781514008 0 0) 0x7f145c160190 con 0x7f145c156420 2024-04-25T00:39:27.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.290+0000 7f145b7fe640 1 -- 172.21.15.29:0/252994653 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1444003530 con 0x7f145c156420 2024-04-25T00:39:27.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f145bfff640 1 -- 172.21.15.29:0/252994653 >> v2:172.21.15.157:3300/0 conn(0x7f145c06b750 msgr2=0x7f145c0cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:27.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f145bfff640 1 --2- 172.21.15.29:0/252994653 >> v2:172.21.15.157:3300/0 conn(0x7f145c06b750 0x7f145c0cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f145bfff640 1 -- 172.21.15.29:0/252994653 >> v1:172.21.15.29:6789/0 conn(0x7f145c156420 legacy=0x7f145c15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:27.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f145bfff640 1 -- 172.21.15.29:0/252994653 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c15d2f0 con 0x7f145c15ce60 2024-04-25T00:39:27.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f145bfff640 1 --2- 172.21.15.29:0/252994653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f145c15ce60 0x7f145c06b210 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f145c0d0bf0 tx=0x7f145001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676cb4cf05f3e32b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:27.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f145b7fe640 1 -- 172.21.15.29:0/252994653 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14500b2070 con 0x7f145c15ce60 2024-04-25T00:39:27.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f145b7fe640 1 -- 172.21.15.29:0/252994653 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14500ad640 con 0x7f145c15ce60 2024-04-25T00:39:27.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f145b7fe640 1 -- 172.21.15.29:0/252994653 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14500ada10 con 0x7f145c15ce60 2024-04-25T00:39:27.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f1462bae640 1 -- 172.21.15.29:0/252994653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f145c15ce60 msgr2=0x7f145c06b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:27.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f1462bae640 1 --2- 172.21.15.29:0/252994653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f145c15ce60 0x7f145c06b210 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f145c0d0bf0 tx=0x7f145001c480 comp rx=0 tx=0).stop 2024-04-25T00:39:27.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f1462bae640 1 -- 172.21.15.29:0/252994653 shutdown_connections 2024-04-25T00:39:27.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f1462bae640 1 --2- 172.21.15.29:0/252994653 >> v2:172.21.15.157:3300/0 conn(0x7f145c06b750 0x7f145c0cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f1462bae640 1 --2- 172.21.15.29:0/252994653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f145c15ce60 0x7f145c06b210 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f1462bae640 1 -- 172.21.15.29:0/252994653 >> 172.21.15.29:0/252994653 conn(0x7f145c0d0c40 msgr2=0x7f145c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:27.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f1462bae640 1 -- 172.21.15.29:0/252994653 shutdown_connections 2024-04-25T00:39:27.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.294+0000 7f1462bae640 1 -- 172.21.15.29:0/252994653 wait complete. 2024-04-25T00:39:27.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f1462bae640 1 Processor -- start 2024-04-25T00:39:27.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f1462bae640 1 -- start start 2024-04-25T00:39:27.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f1462bae640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f145c06b750 0x7f145c153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f1462bae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f145c156420 0x7f145c0c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f1462bae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f145c15ce60 0x7f145c071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f1462bae640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f145c156980 con 0x7f145c156420 2024-04-25T00:39:27.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f1462bae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f145c156800 con 0x7f145c15ce60 2024-04-25T00:39:27.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f1462bae640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f145c070160 con 0x7f145c06b750 2024-04-25T00:39:27.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f145bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f145c156420 0x7f145c0c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f145bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f145c156420 0x7f145c0c4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40064/0 (socket says 172.21.15.29:40064) 2024-04-25T00:39:27.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f145bfff640 1 -- 172.21.15.29:0/1584053547 learned_addr learned my addr 172.21.15.29:0/1584053547 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f146114b640 1 --2- 172.21.15.29:0/1584053547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f145c15ce60 0x7f145c071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f145bfff640 1 -- 172.21.15.29:0/1584053547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f145c06b750 msgr2=0x7f145c153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f146094a640 1 --2- 172.21.15.29:0/1584053547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f145c06b750 0x7f145c153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f145bfff640 1 --2- 172.21.15.29:0/1584053547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f145c06b750 0x7f145c153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f145bfff640 1 -- 172.21.15.29:0/1584053547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f145c15ce60 msgr2=0x7f145c071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f145bfff640 1 --2- 172.21.15.29:0/1584053547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f145c15ce60 0x7f145c071c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f145bfff640 1 -- 172.21.15.29:0/1584053547 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c1537a0 con 0x7f145c156420 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f146114b640 1 --2- 172.21.15.29:0/1584053547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f145c15ce60 0x7f145c071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f145bfff640 1 --2- 172.21.15.29:0/1584053547 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f145c156420 0x7f145c0c4e70 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f145001ab30 tx=0x7f1450019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f14597fa640 1 -- 172.21.15.29:0/1584053547 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14500b2040 con 0x7f145c156420 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f14597fa640 1 -- 172.21.15.29:0/1584053547 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14500bb070 con 0x7f145c156420 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f14597fa640 1 -- 172.21.15.29:0/1584053547 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14500b6650 con 0x7f145c156420 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f1462bae640 1 -- 172.21.15.29:0/1584053547 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145c0c53b0 con 0x7f145c156420 2024-04-25T00:39:27.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f1462bae640 1 -- 172.21.15.29:0/1584053547 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145c0c9f90 con 0x7f145c156420 2024-04-25T00:39:27.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.298+0000 7f1462bae640 1 -- 172.21.15.29:0/1584053547 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145c069270 con 0x7f145c156420 2024-04-25T00:39:27.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.302+0000 7f14597fa640 1 -- 172.21.15.29:0/1584053547 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f14500b67f0 con 0x7f145c156420 2024-04-25T00:39:27.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.302+0000 7f14597fa640 1 --2- 172.21.15.29:0/1584053547 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1448067e60 0x7f144806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.302+0000 7f14597fa640 1 -- 172.21.15.29:0/1584053547 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f1450120530 con 0x7f145c156420 2024-04-25T00:39:27.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.302+0000 7f146094a640 1 --2- 172.21.15.29:0/1584053547 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1448067e60 0x7f144806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.302+0000 7f146094a640 1 --2- 172.21.15.29:0/1584053547 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1448067e60 0x7f144806a320 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f144c0012f0 tx=0x7f144c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:27.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.306+0000 7f14597fa640 1 -- 172.21.15.29:0/1584053547 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1450016070 con 0x7f145c156420 2024-04-25T00:39:27.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.490+0000 7f1462bae640 1 -- 172.21.15.29:0/1584053547 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f145c0d1980 con 0x7f1448067e60 2024-04-25T00:39:27.495 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:27.490+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:27.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.490+0000 7f14597fa640 1 -- 172.21.15.29:0/1584053547 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f145c0d1980 con 0x7f1448067e60 2024-04-25T00:39:27.495 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:27.501 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:27.502 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2024-04-25T00:39:27.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.634+0000 7f6cdd003640 1 Processor -- start 2024-04-25T00:39:27.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.634+0000 7f6cdd003640 1 -- start start 2024-04-25T00:39:27.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.634+0000 7f6cdd003640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6cd815e440 0x7f6cd80d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.634+0000 7f6cdd003640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cd80cc3b0 con 0x7f6cd806c950 2024-04-25T00:39:27.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.634+0000 7f6cdd003640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cd80b11d0 con 0x7f6cd80d0ff0 2024-04-25T00:39:27.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.634+0000 7f6cdd003640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6cd80cc650 con 0x7f6cd815e440 2024-04-25T00:39:27.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.634+0000 7f6cd659c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6cd815e440 0x7f6cd80d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.634+0000 7f6cd6d9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6cd806c950 0x7f6cd806cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40606/0 (socket says 172.21.15.29:40606) 2024-04-25T00:39:27.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.634+0000 7f6cd6d9d640 1 -- 172.21.15.29:0/1796351937 learned_addr learned my addr 172.21.15.29:0/1796351937 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:27.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.634+0000 7f6cd659c640 1 -- 172.21.15.29:0/1796351937 >> v1:172.21.15.157:6789/0 conn(0x7f6cd80d0ff0 legacy=0x7f6cd80cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:27.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.634+0000 7f6cd659c640 1 -- 172.21.15.29:0/1796351937 >> v1:172.21.15.29:6789/0 conn(0x7f6cd806c950 legacy=0x7f6cd806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:27.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.634+0000 7f6cd659c640 1 -- 172.21.15.29:0/1796351937 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd80cc930 con 0x7f6cd815e440 2024-04-25T00:39:27.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.638+0000 7f6cd659c640 1 --2- 172.21.15.29:0/1796351937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cd815e440 0x7f6cd80d0ab0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0097c0 tx=0x7f6ccc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6250559a87087b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:27.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.638+0000 7f6cd5d9b640 1 -- 172.21.15.29:0/1796351937 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0b1070 con 0x7f6cd815e440 2024-04-25T00:39:27.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.638+0000 7f6cd5d9b640 1 -- 172.21.15.29:0/1796351937 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ccc0ac5f0 con 0x7f6cd815e440 2024-04-25T00:39:27.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.638+0000 7f6cd5d9b640 1 -- 172.21.15.29:0/1796351937 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0ac9c0 con 0x7f6cd815e440 2024-04-25T00:39:27.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.638+0000 7f6cdd003640 1 -- 172.21.15.29:0/1796351937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cd815e440 msgr2=0x7f6cd80d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:27.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.638+0000 7f6cdd003640 1 --2- 172.21.15.29:0/1796351937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cd815e440 0x7f6cd80d0ab0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0097c0 tx=0x7f6ccc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.638+0000 7f6cdd003640 1 -- 172.21.15.29:0/1796351937 shutdown_connections 2024-04-25T00:39:27.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.638+0000 7f6cdd003640 1 --2- 172.21.15.29:0/1796351937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cd815e440 0x7f6cd80d0ab0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.638+0000 7f6cdd003640 1 -- 172.21.15.29:0/1796351937 >> 172.21.15.29:0/1796351937 conn(0x7f6cd80d1d60 msgr2=0x7f6cd80d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:27.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.638+0000 7f6cdd003640 1 -- 172.21.15.29:0/1796351937 shutdown_connections 2024-04-25T00:39:27.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.638+0000 7f6cdd003640 1 -- 172.21.15.29:0/1796351937 wait complete. 2024-04-25T00:39:27.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.638+0000 7f6cdd003640 1 Processor -- start 2024-04-25T00:39:27.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cdd003640 1 -- start start 2024-04-25T00:39:27.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cdd003640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd806c950 0x7f6cd80c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cdd003640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cd80d0ff0 0x7f6cd8155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cdd003640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cd815e440 0x7f6cd81564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cdd003640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd816bfb0 con 0x7f6cd80d0ff0 2024-04-25T00:39:27.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cdd003640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd816be30 con 0x7f6cd815e440 2024-04-25T00:39:27.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cdd003640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd816c130 con 0x7f6cd806c950 2024-04-25T00:39:27.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cd659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cd80d0ff0 0x7f6cd8155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cd659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cd80d0ff0 0x7f6cd8155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43394/0 (socket says 172.21.15.29:43394) 2024-04-25T00:39:27.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cd659c640 1 -- 172.21.15.29:0/1811015430 learned_addr learned my addr 172.21.15.29:0/1811015430 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:27.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cd759e640 1 --2- 172.21.15.29:0/1811015430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cd815e440 0x7f6cd81564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cd659c640 1 -- 172.21.15.29:0/1811015430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd806c950 msgr2=0x7f6cd80c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:27.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cd6d9d640 1 --2- 172.21.15.29:0/1811015430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd806c950 0x7f6cd80c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cd659c640 1 --2- 172.21.15.29:0/1811015430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6cd806c950 0x7f6cd80c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cd659c640 1 -- 172.21.15.29:0/1811015430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cd815e440 msgr2=0x7f6cd81564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cd659c640 1 --2- 172.21.15.29:0/1811015430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cd815e440 0x7f6cd81564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cd659c640 1 -- 172.21.15.29:0/1811015430 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd8152a60 con 0x7f6cd80d0ff0 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cd659c640 1 --2- 172.21.15.29:0/1811015430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cd80d0ff0 0x7f6cd8155fa0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc01ab50 tx=0x7f6ccc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cbf7fe640 1 -- 172.21.15.29:0/1811015430 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0b1040 con 0x7f6cd80d0ff0 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cbf7fe640 1 -- 172.21.15.29:0/1811015430 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ccc0bd070 con 0x7f6cd80d0ff0 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cbf7fe640 1 -- 172.21.15.29:0/1811015430 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0b5860 con 0x7f6cd80d0ff0 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cdd003640 1 -- 172.21.15.29:0/1811015430 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd8152d40 con 0x7f6cd80d0ff0 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cdd003640 1 -- 172.21.15.29:0/1811015430 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd8153300 con 0x7f6cd80d0ff0 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cdd003640 1 -- 172.21.15.29:0/1811015430 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd8069270 con 0x7f6cd80d0ff0 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cbf7fe640 1 -- 172.21.15.29:0/1811015430 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f6ccc0b4070 con 0x7f6cd80d0ff0 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cbf7fe640 1 --2- 172.21.15.29:0/1811015430 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6cc4067e60 0x7f6cc406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cbf7fe640 1 -- 172.21.15.29:0/1811015430 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f6ccc120550 con 0x7f6cd80d0ff0 2024-04-25T00:39:27.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.642+0000 7f6cd6d9d640 1 --2- 172.21.15.29:0/1811015430 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6cc4067e60 0x7f6cc406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.646+0000 7f6cd6d9d640 1 --2- 172.21.15.29:0/1811015430 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6cc4067e60 0x7f6cc406a320 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8005890 tx=0x7f6cc8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:27.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.650+0000 7f6cbf7fe640 1 -- 172.21.15.29:0/1811015430 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6ccc0a9070 con 0x7f6cd80d0ff0 2024-04-25T00:39:27.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.826+0000 7f6cdd003640 1 -- 172.21.15.29:0/1811015430 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f6cd80b3600 con 0x7f6cc4067e60 2024-04-25T00:39:27.833 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:27.830+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:27.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.830+0000 7f6cbf7fe640 1 -- 172.21.15.29:0/1811015430 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f6cd80b3600 con 0x7f6cc4067e60 2024-04-25T00:39:27.834 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:27.841 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:27.842 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2024-04-25T00:39:27.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f05020a3640 1 Processor -- start 2024-04-25T00:39:27.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f05020a3640 1 -- start start 2024-04-25T00:39:27.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f05020a3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f04fc15df70 0x7f04fc0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f05020a3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f04fc06d210 0x7f04fc0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f05020a3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04fc06d980 con 0x7f04fc0b4b60 2024-04-25T00:39:27.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f05020a3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f04fc0b4fd0 con 0x7f04fc15df70 2024-04-25T00:39:27.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f05020a3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f04fc06dc60 con 0x7f04fc06d210 2024-04-25T00:39:27.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f04fb7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f04fc15df70 0x7f04fc0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f04fb7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f04fc15df70 0x7f04fc0b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43404/0 (socket says 172.21.15.29:43404) 2024-04-25T00:39:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f04fb7fe640 1 -- 172.21.15.29:0/3970094976 learned_addr learned my addr 172.21.15.29:0/3970094976 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f04faffd640 1 --2- 172.21.15.29:0/3970094976 >> v2:172.21.15.157:3300/0 conn(0x7f04fc06d210 0x7f04fc0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f04fb7fe640 1 -- 172.21.15.29:0/3970094976 >> v2:172.21.15.157:3300/0 conn(0x7f04fc06d210 msgr2=0x7f04fc0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f04fb7fe640 1 --2- 172.21.15.29:0/3970094976 >> v2:172.21.15.157:3300/0 conn(0x7f04fc06d210 0x7f04fc0b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f04fb7fe640 1 -- 172.21.15.29:0/3970094976 >> v1:172.21.15.157:6789/0 conn(0x7f04fc0b4b60 legacy=0x7f04fc070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f04fb7fe640 1 -- 172.21.15.29:0/3970094976 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc06df40 con 0x7f04fc15df70 2024-04-25T00:39:27.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.966+0000 7f04faffd640 1 --2- 172.21.15.29:0/3970094976 >> v2:172.21.15.157:3300/0 conn(0x7f04fc06d210 0x7f04fc0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:27.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.970+0000 7f04fb7fe640 1 --2- 172.21.15.29:0/3970094976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04fc15df70 0x7f04fc0b3e40 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0cfb70 tx=0x7f04f401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b65f3bc37335ec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:27.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.970+0000 7f04fa7fc640 1 -- 172.21.15.29:0/3970094976 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f40a56c0 con 0x7f04fc15df70 2024-04-25T00:39:27.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.970+0000 7f04fa7fc640 1 -- 172.21.15.29:0/3970094976 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f40a5860 con 0x7f04fc15df70 2024-04-25T00:39:27.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.970+0000 7f04fa7fc640 1 -- 172.21.15.29:0/3970094976 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f40a5c30 con 0x7f04fc15df70 2024-04-25T00:39:27.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.970+0000 7f05020a3640 1 -- 172.21.15.29:0/3970094976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04fc15df70 msgr2=0x7f04fc0b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:27.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.970+0000 7f05020a3640 1 --2- 172.21.15.29:0/3970094976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04fc15df70 0x7f04fc0b3e40 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0cfb70 tx=0x7f04f401c9f0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.970+0000 7f05020a3640 1 -- 172.21.15.29:0/3970094976 shutdown_connections 2024-04-25T00:39:27.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.970+0000 7f05020a3640 1 --2- 172.21.15.29:0/3970094976 >> v2:172.21.15.157:3300/0 conn(0x7f04fc06d210 0x7f04fc0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.970+0000 7f05020a3640 1 --2- 172.21.15.29:0/3970094976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04fc15df70 0x7f04fc0b3e40 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.970+0000 7f05020a3640 1 -- 172.21.15.29:0/3970094976 >> 172.21.15.29:0/3970094976 conn(0x7f04fc0b1870 msgr2=0x7f04fc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:27.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.970+0000 7f05020a3640 1 -- 172.21.15.29:0/3970094976 shutdown_connections 2024-04-25T00:39:27.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.970+0000 7f05020a3640 1 -- 172.21.15.29:0/3970094976 wait complete. 2024-04-25T00:39:27.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f05020a3640 1 Processor -- start 2024-04-25T00:39:27.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f05020a3640 1 -- start start 2024-04-25T00:39:27.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f05020a3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04fc06d210 0x7f04fc13f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f05020a3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04fc0b4b60 0x7f04fc0cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f05020a3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04fc15df70 0x7f04fc15a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f05020a3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc06ea20 con 0x7f04fc06d210 2024-04-25T00:39:27.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f05020a3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc06d620 con 0x7f04fc15df70 2024-04-25T00:39:27.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f05020a3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f04fc06eba0 con 0x7f04fc0b4b60 2024-04-25T00:39:27.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04fb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04fc06d210 0x7f04fc13f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04fb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04fc06d210 0x7f04fc13f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43414/0 (socket says 172.21.15.29:43414) 2024-04-25T00:39:27.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04fb7fe640 1 -- 172.21.15.29:0/2520510500 learned_addr learned my addr 172.21.15.29:0/2520510500 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:27.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04fbfff640 1 --2- 172.21.15.29:0/2520510500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04fc15df70 0x7f04fc15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04faffd640 1 --2- 172.21.15.29:0/2520510500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04fc0b4b60 0x7f04fc0cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04fb7fe640 1 -- 172.21.15.29:0/2520510500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04fc0b4b60 msgr2=0x7f04fc0cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:27.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04fb7fe640 1 --2- 172.21.15.29:0/2520510500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04fc0b4b60 0x7f04fc0cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04fb7fe640 1 -- 172.21.15.29:0/2520510500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04fc15df70 msgr2=0x7f04fc15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:27.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04fb7fe640 1 --2- 172.21.15.29:0/2520510500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04fc15df70 0x7f04fc15a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:27.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04fb7fe640 1 -- 172.21.15.29:0/2520510500 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc13fad0 con 0x7f04fc06d210 2024-04-25T00:39:27.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04fbfff640 1 --2- 172.21.15.29:0/2520510500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04fc15df70 0x7f04fc15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:27.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04faffd640 1 --2- 172.21.15.29:0/2520510500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04fc0b4b60 0x7f04fc0cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:27.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04fb7fe640 1 --2- 172.21.15.29:0/2520510500 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04fc06d210 0x7f04fc13f590 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f04f40af040 tx=0x7f04f4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:27.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04f3fff640 1 -- 172.21.15.29:0/2520510500 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f40b8670 con 0x7f04fc06d210 2024-04-25T00:39:27.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04f3fff640 1 -- 172.21.15.29:0/2520510500 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f40be070 con 0x7f04fc06d210 2024-04-25T00:39:27.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f05020a3640 1 -- 172.21.15.29:0/2520510500 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc159590 con 0x7f04fc06d210 2024-04-25T00:39:27.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f04f3fff640 1 -- 172.21.15.29:0/2520510500 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f40b8a50 con 0x7f04fc06d210 2024-04-25T00:39:27.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f05020a3640 1 -- 172.21.15.29:0/2520510500 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc159b50 con 0x7f04fc06d210 2024-04-25T00:39:27.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.974+0000 7f05020a3640 1 -- 172.21.15.29:0/2520510500 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04fc069270 con 0x7f04fc06d210 2024-04-25T00:39:27.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.978+0000 7f04f3fff640 1 -- 172.21.15.29:0/2520510500 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f04f40b8bf0 con 0x7f04fc06d210 2024-04-25T00:39:27.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.978+0000 7f04f3fff640 1 --2- 172.21.15.29:0/2520510500 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f04dc067e60 0x7f04dc06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:27.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.978+0000 7f04f3fff640 1 -- 172.21.15.29:0/2520510500 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f04f4122370 con 0x7f04fc06d210 2024-04-25T00:39:27.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.978+0000 7f04faffd640 1 --2- 172.21.15.29:0/2520510500 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f04dc067e60 0x7f04dc06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:27.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.978+0000 7f04faffd640 1 --2- 172.21.15.29:0/2520510500 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f04dc067e60 0x7f04dc06a320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0b32f0 tx=0x7f04e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:27.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:27.982+0000 7f04f3fff640 1 -- 172.21.15.29:0/2520510500 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f04f40165e0 con 0x7f04fc06d210 2024-04-25T00:39:28.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.170+0000 7f05020a3640 1 -- 172.21.15.29:0/2520510500 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f04fc0cf720 con 0x7f04dc067e60 2024-04-25T00:39:28.176 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:28.174+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:28.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.174+0000 7f04f3fff640 1 -- 172.21.15.29:0/2520510500 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f04fc0cf720 con 0x7f04dc067e60 2024-04-25T00:39:28.177 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:28.183 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:28.183 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2024-04-25T00:39:28.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a1df69640 1 Processor -- start 2024-04-25T00:39:28.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a1df69640 1 -- start start 2024-04-25T00:39:28.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a1df69640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1a180b3b50 0x7f1a180b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a1df69640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1a1806cd40 0x7f1a1806d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a1df69640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a180cf670 con 0x7f1a181456d0 2024-04-25T00:39:28.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a1df69640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1a180b4380 con 0x7f1a1806cd40 2024-04-25T00:39:28.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a1df69640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1a180b40a0 con 0x7f1a180b3b50 2024-04-25T00:39:28.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a177fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1a181456d0 0x7f1a180cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42100/0 (socket says 172.21.15.29:42100) 2024-04-25T00:39:28.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a17fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1a1806cd40 0x7f1a1806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a177fe640 1 -- 172.21.15.29:0/2048983597 learned_addr learned my addr 172.21.15.29:0/2048983597 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:28.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a17fff640 1 -- 172.21.15.29:0/2048983597 >> v2:172.21.15.157:3300/0 conn(0x7f1a180b3b50 msgr2=0x7f1a180b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:28.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a16ffd640 1 --2- 172.21.15.29:0/2048983597 >> v2:172.21.15.157:3300/0 conn(0x7f1a180b3b50 0x7f1a180b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a17fff640 1 --2- 172.21.15.29:0/2048983597 >> v2:172.21.15.157:3300/0 conn(0x7f1a180b3b50 0x7f1a180b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a17fff640 1 -- 172.21.15.29:0/2048983597 >> v1:172.21.15.29:6789/0 conn(0x7f1a181456d0 legacy=0x7f1a180cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a17fff640 1 -- 172.21.15.29:0/2048983597 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a180ce480 con 0x7f1a1806cd40 2024-04-25T00:39:28.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a167fc640 1 -- 172.21.15.29:0/2048983597 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3209454344 0 0) 0x7f1a180cf670 con 0x7f1a181456d0 2024-04-25T00:39:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.306+0000 7f1a17fff640 1 --2- 172.21.15.29:0/2048983597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a1806cd40 0x7f1a1806d150 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f1a180cfb50 tx=0x7f1a0801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d5fbc7d9b5ef24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a167fc640 1 -- 172.21.15.29:0/2048983597 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a080b0070 con 0x7f1a1806cd40 2024-04-25T00:39:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a167fc640 1 -- 172.21.15.29:0/2048983597 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a080ab600 con 0x7f1a1806cd40 2024-04-25T00:39:28.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a167fc640 1 -- 172.21.15.29:0/2048983597 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a080ab9d0 con 0x7f1a1806cd40 2024-04-25T00:39:28.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 -- 172.21.15.29:0/2048983597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a1806cd40 msgr2=0x7f1a1806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 --2- 172.21.15.29:0/2048983597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a1806cd40 0x7f1a1806d150 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f1a180cfb50 tx=0x7f1a0801c480 comp rx=0 tx=0).stop 2024-04-25T00:39:28.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 -- 172.21.15.29:0/2048983597 shutdown_connections 2024-04-25T00:39:28.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 --2- 172.21.15.29:0/2048983597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a1806cd40 0x7f1a1806d150 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 --2- 172.21.15.29:0/2048983597 >> v2:172.21.15.157:3300/0 conn(0x7f1a180b3b50 0x7f1a180b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 -- 172.21.15.29:0/2048983597 >> 172.21.15.29:0/2048983597 conn(0x7f1a180b1870 msgr2=0x7f1a180b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:28.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 -- 172.21.15.29:0/2048983597 shutdown_connections 2024-04-25T00:39:28.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 -- 172.21.15.29:0/2048983597 wait complete. 2024-04-25T00:39:28.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 Processor -- start 2024-04-25T00:39:28.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 -- start start 2024-04-25T00:39:28.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a1806cd40 0x7f1a1813ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a180b3b50 0x7f1a18140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a181456d0 0x7f1a18143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1806d690 con 0x7f1a180b3b50 2024-04-25T00:39:28.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1a180b5000 con 0x7f1a181456d0 2024-04-25T00:39:28.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1a18168b70 con 0x7f1a1806cd40 2024-04-25T00:39:28.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a16ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a180b3b50 0x7f1a18140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a16ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a180b3b50 0x7f1a18140aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43434/0 (socket says 172.21.15.29:43434) 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a16ffd640 1 -- 172.21.15.29:0/2185704870 learned_addr learned my addr 172.21.15.29:0/2185704870 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a177fe640 1 --2- 172.21.15.29:0/2185704870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a1806cd40 0x7f1a1813ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a17fff640 1 --2- 172.21.15.29:0/2185704870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a181456d0 0x7f1a18143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a16ffd640 1 -- 172.21.15.29:0/2185704870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a1806cd40 msgr2=0x7f1a1813ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a16ffd640 1 --2- 172.21.15.29:0/2185704870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a1806cd40 0x7f1a1813ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a16ffd640 1 -- 172.21.15.29:0/2185704870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a181456d0 msgr2=0x7f1a18143300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a16ffd640 1 --2- 172.21.15.29:0/2185704870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a181456d0 0x7f1a18143300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a16ffd640 1 -- 172.21.15.29:0/2185704870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1815d990 con 0x7f1a180b3b50 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a17fff640 1 --2- 172.21.15.29:0/2185704870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a181456d0 0x7f1a18143300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a16ffd640 1 --2- 172.21.15.29:0/2185704870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a180b3b50 0x7f1a18140aa0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000cd00 tx=0x7f1a0000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a0ffff640 1 -- 172.21.15.29:0/2185704870 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a000039a0 con 0x7f1a180b3b50 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a0ffff640 1 -- 172.21.15.29:0/2185704870 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a00003b40 con 0x7f1a180b3b50 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 -- 172.21.15.29:0/2185704870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1815dc70 con 0x7f1a180b3b50 2024-04-25T00:39:28.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a0ffff640 1 -- 172.21.15.29:0/2185704870 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a00004020 con 0x7f1a180b3b50 2024-04-25T00:39:28.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.310+0000 7f1a1df69640 1 -- 172.21.15.29:0/2185704870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1815e230 con 0x7f1a180b3b50 2024-04-25T00:39:28.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.314+0000 7f1a1df69640 1 -- 172.21.15.29:0/2185704870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a180692e0 con 0x7f1a180b3b50 2024-04-25T00:39:28.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.314+0000 7f1a0ffff640 1 -- 172.21.15.29:0/2185704870 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f1a000041c0 con 0x7f1a180b3b50 2024-04-25T00:39:28.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.314+0000 7f1a0ffff640 1 --2- 172.21.15.29:0/2185704870 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19f8067e90 0x7f19f806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.314+0000 7f1a177fe640 1 --2- 172.21.15.29:0/2185704870 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19f8067e90 0x7f19f806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.314+0000 7f1a0ffff640 1 -- 172.21.15.29:0/2185704870 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f1a00086d80 con 0x7f1a180b3b50 2024-04-25T00:39:28.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.314+0000 7f1a177fe640 1 --2- 172.21.15.29:0/2185704870 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19f8067e90 0x7f19f806a350 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1a10000f70 tx=0x7f1a10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.318+0000 7f1a0ffff640 1 -- 172.21.15.29:0/2185704870 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1a00014030 con 0x7f1a180b3b50 2024-04-25T00:39:28.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.506+0000 7f1a1df69640 1 -- 172.21.15.29:0/2185704870 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) -- 0x7f1a1813bf70 con 0x7f19f8067e90 2024-04-25T00:39:28.512 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:28.510+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:28.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.510+0000 7f1a0ffff640 1 -- 172.21.15.29:0/2185704870 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f1a1813bf70 con 0x7f19f8067e90 2024-04-25T00:39:28.513 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:28.518 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:28.518 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2024-04-25T00:39:28.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.634+0000 7f7c84068640 1 Processor -- start 2024-04-25T00:39:28.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.634+0000 7f7c84068640 1 -- start start 2024-04-25T00:39:28.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.634+0000 7f7c84068640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c7c06c950 0x7f7c7c06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.634+0000 7f7c84068640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c7c0cd410 0x7f7c7c0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c84068640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c7c0cc3b0 con 0x7f7c7c15e440 2024-04-25T00:39:28.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c84068640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7c7c06d270 con 0x7f7c7c0cd410 2024-04-25T00:39:28.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c84068640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c7c0b11d0 con 0x7f7c7c06c950 2024-04-25T00:39:28.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c81603640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7c7c15e440 0x7f7c7c0d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42108/0 (socket says 172.21.15.29:42108) 2024-04-25T00:39:28.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c81603640 1 -- 172.21.15.29:0/3324058534 learned_addr learned my addr 172.21.15.29:0/3324058534 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:28.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c82605640 1 --2- 172.21.15.29:0/3324058534 >> v2:172.21.15.29:3300/0 conn(0x7f7c7c0cd410 0x7f7c7c0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c81e04640 1 --2- 172.21.15.29:0/3324058534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c7c06c950 0x7f7c7c06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c82605640 1 -- 172.21.15.29:0/3324058534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c7c06c950 msgr2=0x7f7c7c06cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:28.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c82605640 1 --2- 172.21.15.29:0/3324058534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c7c06c950 0x7f7c7c06cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c82605640 1 -- 172.21.15.29:0/3324058534 >> v1:172.21.15.29:6789/0 conn(0x7f7c7c15e440 legacy=0x7f7c7c0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c82605640 1 -- 172.21.15.29:0/3324058534 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7c0cc650 con 0x7f7c7c0cd410 2024-04-25T00:39:28.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c82605640 1 --2- 172.21.15.29:0/3324058534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c7c0cd410 0x7f7c7c0cd8a0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009d90 tx=0x7f7c7401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cd82de1961d04c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c80e02640 1 -- 172.21.15.29:0/3324058534 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c740b2070 con 0x7f7c7c0cd410 2024-04-25T00:39:28.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c80e02640 1 -- 172.21.15.29:0/3324058534 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c740a5920 con 0x7f7c7c0cd410 2024-04-25T00:39:28.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.638+0000 7f7c80e02640 1 -- 172.21.15.29:0/3324058534 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c740a5cf0 con 0x7f7c7c0cd410 2024-04-25T00:39:28.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 -- 172.21.15.29:0/3324058534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c7c0cd410 msgr2=0x7f7c7c0cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 --2- 172.21.15.29:0/3324058534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c7c0cd410 0x7f7c7c0cd8a0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009d90 tx=0x7f7c7401c970 comp rx=0 tx=0).stop 2024-04-25T00:39:28.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 -- 172.21.15.29:0/3324058534 shutdown_connections 2024-04-25T00:39:28.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 --2- 172.21.15.29:0/3324058534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c7c0cd410 0x7f7c7c0cd8a0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 --2- 172.21.15.29:0/3324058534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c7c06c950 0x7f7c7c06cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 -- 172.21.15.29:0/3324058534 >> 172.21.15.29:0/3324058534 conn(0x7f7c7c0d1d60 msgr2=0x7f7c7c0d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:28.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 -- 172.21.15.29:0/3324058534 shutdown_connections 2024-04-25T00:39:28.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 -- 172.21.15.29:0/3324058534 wait complete. 2024-04-25T00:39:28.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 Processor -- start 2024-04-25T00:39:28.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 -- start start 2024-04-25T00:39:28.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c7c06c950 0x7f7c7c146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c7c0cd410 0x7f7c7c0b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c7c15e440 0x7f7c7c0b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c0d11c0 con 0x7f7c7c15e440 2024-04-25T00:39:28.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c06c520 con 0x7f7c7c06c950 2024-04-25T00:39:28.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c84068640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c7c16ae20 con 0x7f7c7c0cd410 2024-04-25T00:39:28.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c82605640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c7c15e440 0x7f7c7c0b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c82605640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c7c15e440 0x7f7c7c0b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43456/0 (socket says 172.21.15.29:43456) 2024-04-25T00:39:28.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c82605640 1 -- 172.21.15.29:0/620585327 learned_addr learned my addr 172.21.15.29:0/620585327 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:28.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c81603640 1 --2- 172.21.15.29:0/620585327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c7c0cd410 0x7f7c7c0b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c81e04640 1 --2- 172.21.15.29:0/620585327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c7c06c950 0x7f7c7c146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c82605640 1 -- 172.21.15.29:0/620585327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c7c0cd410 msgr2=0x7f7c7c0b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c82605640 1 --2- 172.21.15.29:0/620585327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c7c0cd410 0x7f7c7c0b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c82605640 1 -- 172.21.15.29:0/620585327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c7c06c950 msgr2=0x7f7c7c146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c82605640 1 --2- 172.21.15.29:0/620585327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c7c06c950 0x7f7c7c146f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c82605640 1 -- 172.21.15.29:0/620585327 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7c06b710 con 0x7f7c7c15e440 2024-04-25T00:39:28.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.642+0000 7f7c82605640 1 --2- 172.21.15.29:0/620585327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c7c15e440 0x7f7c7c0b2690 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f7c740b0040 tx=0x7f7c74019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.646+0000 7f7c727fc640 1 -- 172.21.15.29:0/620585327 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c740b2040 con 0x7f7c7c15e440 2024-04-25T00:39:28.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.646+0000 7f7c727fc640 1 -- 172.21.15.29:0/620585327 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c740bf070 con 0x7f7c7c15e440 2024-04-25T00:39:28.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.646+0000 7f7c727fc640 1 -- 172.21.15.29:0/620585327 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c740b8870 con 0x7f7c7c15e440 2024-04-25T00:39:28.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.646+0000 7f7c84068640 1 -- 172.21.15.29:0/620585327 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7c0c59d0 con 0x7f7c7c15e440 2024-04-25T00:39:28.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.646+0000 7f7c84068640 1 -- 172.21.15.29:0/620585327 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c7c0c5f90 con 0x7f7c7c15e440 2024-04-25T00:39:28.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.646+0000 7f7c84068640 1 -- 172.21.15.29:0/620585327 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c7c069270 con 0x7f7c7c15e440 2024-04-25T00:39:28.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.646+0000 7f7c727fc640 1 -- 172.21.15.29:0/620585327 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f7c740b8a10 con 0x7f7c7c15e440 2024-04-25T00:39:28.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.646+0000 7f7c727fc640 1 --2- 172.21.15.29:0/620585327 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c60067e60 0x7f7c6006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.646+0000 7f7c727fc640 1 -- 172.21.15.29:0/620585327 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f7c74121b60 con 0x7f7c7c15e440 2024-04-25T00:39:28.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.646+0000 7f7c81e04640 1 --2- 172.21.15.29:0/620585327 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c60067e60 0x7f7c6006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.646+0000 7f7c81e04640 1 --2- 172.21.15.29:0/620585327 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c60067e60 0x7f7c6006a320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7c780096f0 tx=0x7f7c78008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.650+0000 7f7c727fc640 1 -- 172.21.15.29:0/620585327 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7c74016560 con 0x7f7c7c15e440 2024-04-25T00:39:28.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.834+0000 7f7c84068640 1 -- 172.21.15.29:0/620585327 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7f7c7c0d2390 con 0x7f7c60067e60 2024-04-25T00:39:28.840 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:28.838+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:28.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.838+0000 7f7c727fc640 1 -- 172.21.15.29:0/620585327 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f7c7c0d2390 con 0x7f7c60067e60 2024-04-25T00:39:28.840 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:28.845 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:28.846 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2024-04-25T00:39:28.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.970+0000 7f36476b4640 1 Processor -- start 2024-04-25T00:39:28.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.970+0000 7f36476b4640 1 -- start start 2024-04-25T00:39:28.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.974+0000 7f36476b4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364015e9a0 0x7f3640070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.974+0000 7f36476b4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f36400b4590 0x7f364015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.974+0000 7f36476b4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f364015f2c0 con 0x7f36400cd1c0 2024-04-25T00:39:28.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.974+0000 7f36476b4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3640071ed0 con 0x7f36400b4590 2024-04-25T00:39:28.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.974+0000 7f36476b4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f36400712c0 con 0x7f364015e9a0 2024-04-25T00:39:28.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.974+0000 7f3645450640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364015e9a0 0x7f3640070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.974+0000 7f3644c4f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f36400b4590 0x7f364015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.974+0000 7f3645450640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364015e9a0 0x7f3640070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:60548/0 (socket says 172.21.15.29:60548) 2024-04-25T00:39:28.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.974+0000 7f3645450640 1 -- 172.21.15.29:0/4094069983 learned_addr learned my addr 172.21.15.29:0/4094069983 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:28.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.974+0000 7f3644c4f640 1 -- 172.21.15.29:0/4094069983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364015e9a0 msgr2=0x7f3640070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:28.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.974+0000 7f3644c4f640 1 --2- 172.21.15.29:0/4094069983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364015e9a0 0x7f3640070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.974+0000 7f3644c4f640 1 -- 172.21.15.29:0/4094069983 >> v1:172.21.15.157:6789/0 conn(0x7f36400cd1c0 legacy=0x7f3640072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.974+0000 7f3644c4f640 1 -- 172.21.15.29:0/4094069983 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36400715a0 con 0x7f36400b4590 2024-04-25T00:39:28.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3644c4f640 1 --2- 172.21.15.29:0/4094069983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36400b4590 0x7f364015ed80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f363000ba60 tx=0x7f363001eb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73d54c2fa1a6134a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3637fff640 1 -- 172.21.15.29:0/4094069983 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36300b4070 con 0x7f36400b4590 2024-04-25T00:39:28.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3637fff640 1 -- 172.21.15.29:0/4094069983 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36300a7b10 con 0x7f36400b4590 2024-04-25T00:39:28.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3637fff640 1 -- 172.21.15.29:0/4094069983 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36300af440 con 0x7f36400b4590 2024-04-25T00:39:28.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 -- 172.21.15.29:0/4094069983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36400b4590 msgr2=0x7f364015ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 --2- 172.21.15.29:0/4094069983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36400b4590 0x7f364015ed80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f363000ba60 tx=0x7f363001eb40 comp rx=0 tx=0).stop 2024-04-25T00:39:28.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 -- 172.21.15.29:0/4094069983 shutdown_connections 2024-04-25T00:39:28.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 --2- 172.21.15.29:0/4094069983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36400b4590 0x7f364015ed80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 --2- 172.21.15.29:0/4094069983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364015e9a0 0x7f3640070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 -- 172.21.15.29:0/4094069983 >> 172.21.15.29:0/4094069983 conn(0x7f36400b1870 msgr2=0x7f36400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:28.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 -- 172.21.15.29:0/4094069983 shutdown_connections 2024-04-25T00:39:28.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 -- 172.21.15.29:0/4094069983 wait complete. 2024-04-25T00:39:28.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 Processor -- start 2024-04-25T00:39:28.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 -- start start 2024-04-25T00:39:28.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36400b4590 0x7f3640136e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36400cd1c0 0x7f364015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f364015e9a0 0x7f3640138760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3640144870 con 0x7f36400b4590 2024-04-25T00:39:28.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3640072e80 con 0x7f364015e9a0 2024-04-25T00:39:28.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f36476b4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f36401449f0 con 0x7f36400cd1c0 2024-04-25T00:39:28.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3645450640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36400b4590 0x7f3640136e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3645450640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36400b4590 0x7f3640136e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43462/0 (socket says 172.21.15.29:43462) 2024-04-25T00:39:28.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3645450640 1 -- 172.21.15.29:0/1969895933 learned_addr learned my addr 172.21.15.29:0/1969895933 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:28.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3644c4f640 1 --2- 172.21.15.29:0/1969895933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36400cd1c0 0x7f364015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3645c51640 1 --2- 172.21.15.29:0/1969895933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f364015e9a0 0x7f3640138760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3645450640 1 -- 172.21.15.29:0/1969895933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36400cd1c0 msgr2=0x7f364015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3645450640 1 --2- 172.21.15.29:0/1969895933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36400cd1c0 0x7f364015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3645450640 1 -- 172.21.15.29:0/1969895933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f364015e9a0 msgr2=0x7f3640138760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:28.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3645450640 1 --2- 172.21.15.29:0/1969895933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f364015e9a0 0x7f3640138760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:28.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3645450640 1 -- 172.21.15.29:0/1969895933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36401376c0 con 0x7f36400b4590 2024-04-25T00:39:28.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3645c51640 1 --2- 172.21.15.29:0/1969895933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f364015e9a0 0x7f3640138760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:28.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.978+0000 7f3645450640 1 --2- 172.21.15.29:0/1969895933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36400b4590 0x7f3640136e70 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f363c00e930 tx=0x7f363c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.982+0000 7f3635ffb640 1 -- 172.21.15.29:0/1969895933 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c017070 con 0x7f36400b4590 2024-04-25T00:39:28.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.982+0000 7f3635ffb640 1 -- 172.21.15.29:0/1969895933 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f363c00ca00 con 0x7f36400b4590 2024-04-25T00:39:28.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.982+0000 7f3635ffb640 1 -- 172.21.15.29:0/1969895933 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c00cdf0 con 0x7f36400b4590 2024-04-25T00:39:28.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.982+0000 7f36476b4640 1 -- 172.21.15.29:0/1969895933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36401379a0 con 0x7f36400b4590 2024-04-25T00:39:28.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.982+0000 7f36476b4640 1 -- 172.21.15.29:0/1969895933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3640137f60 con 0x7f36400b4590 2024-04-25T00:39:28.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.982+0000 7f36476b4640 1 -- 172.21.15.29:0/1969895933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36400692e0 con 0x7f36400b4590 2024-04-25T00:39:28.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.982+0000 7f3635ffb640 1 -- 172.21.15.29:0/1969895933 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f363c01a070 con 0x7f36400b4590 2024-04-25T00:39:28.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.982+0000 7f3635ffb640 1 --2- 172.21.15.29:0/1969895933 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3624067e60 0x7f362406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:28.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.982+0000 7f3635ffb640 1 -- 172.21.15.29:0/1969895933 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f363c086a80 con 0x7f36400b4590 2024-04-25T00:39:28.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.982+0000 7f3644c4f640 1 --2- 172.21.15.29:0/1969895933 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3624067e60 0x7f362406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:28.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.982+0000 7f3644c4f640 1 --2- 172.21.15.29:0/1969895933 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3624067e60 0x7f362406a320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f36300b1000 tx=0x7f36300a9070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:28.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:28.986+0000 7f3635ffb640 1 -- 172.21.15.29:0/1969895933 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f363c0566e0 con 0x7f36400b4590 2024-04-25T00:39:29.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.166+0000 7f36476b4640 1 -- 172.21.15.29:0/1969895933 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f364006cff0 con 0x7f3624067e60 2024-04-25T00:39:29.172 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:29.170+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:29.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.170+0000 7f3635ffb640 1 -- 172.21.15.29:0/1969895933 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f364006cff0 con 0x7f3624067e60 2024-04-25T00:39:29.172 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:29.178 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:29.179 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2024-04-25T00:39:29.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.294+0000 7f5bda039640 1 Processor -- start 2024-04-25T00:39:29.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.294+0000 7f5bda039640 1 -- start start 2024-04-25T00:39:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bda039640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5bd40b4590 0x7f5bd4070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bda039640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd40cd1c0 0x7f5bd4072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bda039640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd40712c0 con 0x7f5bd415e9a0 2024-04-25T00:39:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bda039640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5bd4071ed0 con 0x7f5bd40b4590 2024-04-25T00:39:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bda039640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd40715a0 con 0x7f5bd40cd1c0 2024-04-25T00:39:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bcaffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5bd40b4590 0x7f5bd4070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bcaffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5bd40b4590 0x7f5bd4070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43470/0 (socket says 172.21.15.29:43470) 2024-04-25T00:39:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bcaffd640 1 -- 172.21.15.29:0/729700068 learned_addr learned my addr 172.21.15.29:0/729700068 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bd3fff640 1 --2- 172.21.15.29:0/729700068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd40cd1c0 0x7f5bd4072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bcaffd640 1 -- 172.21.15.29:0/729700068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd40cd1c0 msgr2=0x7f5bd4072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bcaffd640 1 --2- 172.21.15.29:0/729700068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd40cd1c0 0x7f5bd4072360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bcaffd640 1 -- 172.21.15.29:0/729700068 >> v1:172.21.15.157:6789/0 conn(0x7f5bd415e9a0 legacy=0x7f5bd415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bcaffd640 1 -- 172.21.15.29:0/729700068 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd4071880 con 0x7f5bd40b4590 2024-04-25T00:39:29.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bcaffd640 1 --2- 172.21.15.29:0/729700068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bd40b4590 0x7f5bd4070ce0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009870 tx=0x7f5bbc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405e83baf8bbcc08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:29.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bd2ffd640 1 -- 172.21.15.29:0/729700068 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc0ab640 con 0x7f5bd40b4590 2024-04-25T00:39:29.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bd2ffd640 1 -- 172.21.15.29:0/729700068 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bbc0b0070 con 0x7f5bd40b4590 2024-04-25T00:39:29.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.298+0000 7f5bd2ffd640 1 -- 172.21.15.29:0/729700068 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc0aba00 con 0x7f5bd40b4590 2024-04-25T00:39:29.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 -- 172.21.15.29:0/729700068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bd40b4590 msgr2=0x7f5bd4070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:29.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 --2- 172.21.15.29:0/729700068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bd40b4590 0x7f5bd4070ce0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009870 tx=0x7f5bbc01c460 comp rx=0 tx=0).stop 2024-04-25T00:39:29.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 -- 172.21.15.29:0/729700068 shutdown_connections 2024-04-25T00:39:29.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 --2- 172.21.15.29:0/729700068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd40cd1c0 0x7f5bd4072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:29.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 --2- 172.21.15.29:0/729700068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bd40b4590 0x7f5bd4070ce0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:29.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 -- 172.21.15.29:0/729700068 >> 172.21.15.29:0/729700068 conn(0x7f5bd40b1870 msgr2=0x7f5bd40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:29.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 -- 172.21.15.29:0/729700068 shutdown_connections 2024-04-25T00:39:29.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 -- 172.21.15.29:0/729700068 wait complete. 2024-04-25T00:39:29.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 Processor -- start 2024-04-25T00:39:29.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 -- start start 2024-04-25T00:39:29.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd40b4590 0x7f5bd4159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bd40cd1c0 0x7f5bd413ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd415e9a0 0x7f5bd415d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd40cd750 con 0x7f5bd40cd1c0 2024-04-25T00:39:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd40cd5d0 con 0x7f5bd40b4590 2024-04-25T00:39:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bda039640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd4167d40 con 0x7f5bd415e9a0 2024-04-25T00:39:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bcaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bd40cd1c0 0x7f5bd413ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bcaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bd40cd1c0 0x7f5bd413ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43478/0 (socket says 172.21.15.29:43478) 2024-04-25T00:39:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bcaffd640 1 -- 172.21.15.29:0/973337785 learned_addr learned my addr 172.21.15.29:0/973337785 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bd37fe640 1 --2- 172.21.15.29:0/973337785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd40b4590 0x7f5bd4159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bd3fff640 1 --2- 172.21.15.29:0/973337785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd415e9a0 0x7f5bd415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bcaffd640 1 -- 172.21.15.29:0/973337785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd415e9a0 msgr2=0x7f5bd415d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bcaffd640 1 --2- 172.21.15.29:0/973337785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd415e9a0 0x7f5bd415d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:29.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bcaffd640 1 -- 172.21.15.29:0/973337785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd40b4590 msgr2=0x7f5bd4159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bcaffd640 1 --2- 172.21.15.29:0/973337785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd40b4590 0x7f5bd4159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bcaffd640 1 -- 172.21.15.29:0/973337785 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd4138c60 con 0x7f5bd40cd1c0 2024-04-25T00:39:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bd37fe640 1 --2- 172.21.15.29:0/973337785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd40b4590 0x7f5bd4159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.302+0000 7f5bcaffd640 1 --2- 172.21.15.29:0/973337785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bd40cd1c0 0x7f5bd413ca50 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc01cd00 tx=0x7f5bbc01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.306+0000 7f5bd0ff9640 1 -- 172.21.15.29:0/973337785 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc0be3f0 con 0x7f5bd40cd1c0 2024-04-25T00:39:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.306+0000 7f5bd0ff9640 1 -- 172.21.15.29:0/973337785 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bbc0be590 con 0x7f5bd40cd1c0 2024-04-25T00:39:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.306+0000 7f5bda039640 1 -- 172.21.15.29:0/973337785 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd413d3c0 con 0x7f5bd40cd1c0 2024-04-25T00:39:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.306+0000 7f5bd0ff9640 1 -- 172.21.15.29:0/973337785 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc0be980 con 0x7f5bd40cd1c0 2024-04-25T00:39:29.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.306+0000 7f5bda039640 1 -- 172.21.15.29:0/973337785 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd4138f00 con 0x7f5bd40cd1c0 2024-04-25T00:39:29.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.306+0000 7f5bda039640 1 -- 172.21.15.29:0/973337785 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd40692e0 con 0x7f5bd40cd1c0 2024-04-25T00:39:29.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.306+0000 7f5bd0ff9640 1 -- 172.21.15.29:0/973337785 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f5bbc0b43f0 con 0x7f5bd40cd1c0 2024-04-25T00:39:29.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.306+0000 7f5bd0ff9640 1 --2- 172.21.15.29:0/973337785 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5bb4067e60 0x7f5bb406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:29.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.306+0000 7f5bd0ff9640 1 -- 172.21.15.29:0/973337785 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f5bbc120b90 con 0x7f5bd40cd1c0 2024-04-25T00:39:29.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.306+0000 7f5bd37fe640 1 --2- 172.21.15.29:0/973337785 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5bb4067e60 0x7f5bb406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:29.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.306+0000 7f5bd37fe640 1 --2- 172.21.15.29:0/973337785 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5bb4067e60 0x7f5bb406a320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400a850 tx=0x7f5bc4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:29.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.310+0000 7f5bd0ff9640 1 -- 172.21.15.29:0/973337785 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5bbc0c4060 con 0x7f5bd40cd1c0 2024-04-25T00:39:29.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.490+0000 7f5bda039640 1 -- 172.21.15.29:0/973337785 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f5bd413d8e0 con 0x7f5bb4067e60 2024-04-25T00:39:29.495 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:29.494+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.494+0000 7f5bd0ff9640 1 -- 172.21.15.29:0/973337785 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f5bd413d8e0 con 0x7f5bb4067e60 2024-04-25T00:39:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:29.504 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:29.504 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2024-04-25T00:39:29.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3047aeb640 1 Processor -- start 2024-04-25T00:39:29.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3047aeb640 1 -- start start 2024-04-25T00:39:29.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3047aeb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30400b3b50 0x7f30400b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:29.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3047aeb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f304006cd40 0x7f304006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:29.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3047aeb640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30400cf670 con 0x7f30401456d0 2024-04-25T00:39:29.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3047aeb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f30400b4380 con 0x7f304006cd40 2024-04-25T00:39:29.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3047aeb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f30400b40a0 con 0x7f30400b3b50 2024-04-25T00:39:29.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3046088640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f304006cd40 0x7f304006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:29.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3046088640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f304006cd40 0x7f304006d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43492/0 (socket says 172.21.15.29:43492) 2024-04-25T00:39:29.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3046088640 1 -- 172.21.15.29:0/2615918706 learned_addr learned my addr 172.21.15.29:0/2615918706 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:29.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3045086640 1 --2- 172.21.15.29:0/2615918706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30400b3b50 0x7f30400b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:29.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3046088640 1 -- 172.21.15.29:0/2615918706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30400b3b50 msgr2=0x7f30400b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:29.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3046088640 1 --2- 172.21.15.29:0/2615918706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30400b3b50 0x7f30400b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:29.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3046088640 1 -- 172.21.15.29:0/2615918706 >> v1:172.21.15.29:6789/0 conn(0x7f30401456d0 legacy=0x7f30400cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:29.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3046088640 1 -- 172.21.15.29:0/2615918706 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30400ce480 con 0x7f304006cd40 2024-04-25T00:39:29.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.626+0000 7f3045086640 1 --2- 172.21.15.29:0/2615918706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30400b3b50 0x7f30400b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:29.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.630+0000 7f3046088640 1 --2- 172.21.15.29:0/2615918706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f304006cd40 0x7f304006d170 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f3038009e50 tx=0x7f303801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbbb50fa5e9e6c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:29.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.630+0000 7f3044885640 1 -- 172.21.15.29:0/2615918706 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30380a57f0 con 0x7f304006cd40 2024-04-25T00:39:29.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.630+0000 7f3044885640 1 -- 172.21.15.29:0/2615918706 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30380a5990 con 0x7f304006cd40 2024-04-25T00:39:29.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.630+0000 7f3044885640 1 -- 172.21.15.29:0/2615918706 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30380a5d60 con 0x7f304006cd40 2024-04-25T00:39:29.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.630+0000 7f3047aeb640 1 -- 172.21.15.29:0/2615918706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f304006cd40 msgr2=0x7f304006d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:29.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.630+0000 7f3047aeb640 1 --2- 172.21.15.29:0/2615918706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f304006cd40 0x7f304006d170 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f3038009e50 tx=0x7f303801ca30 comp rx=0 tx=0).stop 2024-04-25T00:39:29.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.630+0000 7f3047aeb640 1 -- 172.21.15.29:0/2615918706 shutdown_connections 2024-04-25T00:39:29.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.630+0000 7f3047aeb640 1 --2- 172.21.15.29:0/2615918706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f304006cd40 0x7f304006d170 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:29.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.630+0000 7f3047aeb640 1 --2- 172.21.15.29:0/2615918706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30400b3b50 0x7f30400b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:29.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.630+0000 7f3047aeb640 1 -- 172.21.15.29:0/2615918706 >> 172.21.15.29:0/2615918706 conn(0x7f30400b1870 msgr2=0x7f30400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:29.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.630+0000 7f3047aeb640 1 -- 172.21.15.29:0/2615918706 shutdown_connections 2024-04-25T00:39:29.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.630+0000 7f3047aeb640 1 -- 172.21.15.29:0/2615918706 wait complete. 2024-04-25T00:39:29.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.630+0000 7f3047aeb640 1 Processor -- start 2024-04-25T00:39:29.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3047aeb640 1 -- start start 2024-04-25T00:39:29.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3047aeb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f304006cd40 0x7f304015df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:29.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3047aeb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400b3b50 0x7f304015e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:29.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3047aeb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30401456d0 0x7f30401fabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:29.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3047aeb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30400b5000 con 0x7f30400b3b50 2024-04-25T00:39:29.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3047aeb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3040145ab0 con 0x7f30401456d0 2024-04-25T00:39:29.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3047aeb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3040168b70 con 0x7f304006cd40 2024-04-25T00:39:29.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3045086640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400b3b50 0x7f304015e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:29.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3045086640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400b3b50 0x7f304015e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43500/0 (socket says 172.21.15.29:43500) 2024-04-25T00:39:29.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3045086640 1 -- 172.21.15.29:0/272926239 learned_addr learned my addr 172.21.15.29:0/272926239 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:29.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3045887640 1 --2- 172.21.15.29:0/272926239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f304006cd40 0x7f304015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:29.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3045086640 1 -- 172.21.15.29:0/272926239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f304006cd40 msgr2=0x7f304015df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3045086640 1 --2- 172.21.15.29:0/272926239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f304006cd40 0x7f304015df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3045086640 1 -- 172.21.15.29:0/272926239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30401456d0 msgr2=0x7f30401fabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3046088640 1 --2- 172.21.15.29:0/272926239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30401456d0 0x7f30401fabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3045086640 1 --2- 172.21.15.29:0/272926239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30401456d0 0x7f30401fabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3045086640 1 -- 172.21.15.29:0/272926239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30401432f0 con 0x7f30400b3b50 2024-04-25T00:39:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3045086640 1 --2- 172.21.15.29:0/272926239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30400b3b50 0x7f304015e4c0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f303000c7f0 tx=0x7f303000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f30367fc640 1 -- 172.21.15.29:0/272926239 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030015070 con 0x7f30400b3b50 2024-04-25T00:39:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f30367fc640 1 -- 172.21.15.29:0/272926239 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3030007820 con 0x7f30400b3b50 2024-04-25T00:39:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3047aeb640 1 -- 172.21.15.29:0/272926239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304013c1d0 con 0x7f30400b3b50 2024-04-25T00:39:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f30367fc640 1 -- 172.21.15.29:0/272926239 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030007c10 con 0x7f30400b3b50 2024-04-25T00:39:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3047aeb640 1 -- 172.21.15.29:0/272926239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30400723c0 con 0x7f30400b3b50 2024-04-25T00:39:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3047aeb640 1 -- 172.21.15.29:0/272926239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30400692e0 con 0x7f30400b3b50 2024-04-25T00:39:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f30367fc640 1 -- 172.21.15.29:0/272926239 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f3030018070 con 0x7f30400b3b50 2024-04-25T00:39:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f30367fc640 1 --2- 172.21.15.29:0/272926239 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3024067f20 0x7f302406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f30367fc640 1 -- 172.21.15.29:0/272926239 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f3030086a20 con 0x7f30400b3b50 2024-04-25T00:39:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.634+0000 7f3045887640 1 --2- 172.21.15.29:0/272926239 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3024067f20 0x7f302406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:29.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.638+0000 7f3045887640 1 --2- 172.21.15.29:0/272926239 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3024067f20 0x7f302406a3e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f303c001650 tx=0x7f303c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:29.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.638+0000 7f30367fc640 1 -- 172.21.15.29:0/272926239 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3030056af0 con 0x7f30400b3b50 2024-04-25T00:39:29.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.826+0000 7f3047aeb640 1 -- 172.21.15.29:0/272926239 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f3040142a90 con 0x7f3024067f20 2024-04-25T00:39:29.830 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:39:29.826+0000 7f987757d640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-04-25T00:39:29.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:29.826+0000 7f30367fc640 1 -- 172.21.15.29:0/272926239 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f3040142a90 con 0x7f3024067f20 2024-04-25T00:39:29.831 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-04-25T00:39:29.839 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:39:29.839 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:39:29.839 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:39:29.881 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:39:29.920 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:39:29.920 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:39:29.920 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:39:29.920 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:39:29.927 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:30.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9883c8e640 1 Processor -- start 2024-04-25T00:39:30.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9883c8e640 1 -- start start 2024-04-25T00:39:30.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9883c8e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f987c15e8d0 0x7f987c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9883c8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f987c15f3c0 0x7f987c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9883c8e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f987c0b3720 con 0x7f987c0b4780 2024-04-25T00:39:30.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9883c8e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f987c0b42f0 con 0x7f987c15e8d0 2024-04-25T00:39:30.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9883c8e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f987c0b39c0 con 0x7f987c15f3c0 2024-04-25T00:39:30.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9881a2a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f987c15e8d0 0x7f987c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9881229640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f987c0b4780 0x7f987c15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42130/0 (socket says 172.21.15.29:42130) 2024-04-25T00:39:30.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9881229640 1 -- 172.21.15.29:0/2343359924 learned_addr learned my addr 172.21.15.29:0/2343359924 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:30.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9881a2a640 1 -- 172.21.15.29:0/2343359924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f987c15f3c0 msgr2=0x7f987c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:30.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9881a2a640 1 --2- 172.21.15.29:0/2343359924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f987c15f3c0 0x7f987c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9881a2a640 1 -- 172.21.15.29:0/2343359924 >> v1:172.21.15.29:6789/0 conn(0x7f987c0b4780 legacy=0x7f987c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9880a28640 1 -- 172.21.15.29:0/2343359924 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1052343487 0 0) 0x7f987c0b3720 con 0x7f987c0b4780 2024-04-25T00:39:30.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9881a2a640 1 -- 172.21.15.29:0/2343359924 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c0b3ca0 con 0x7f987c15e8d0 2024-04-25T00:39:30.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.094+0000 7f9881a2a640 1 --2- 172.21.15.29:0/2343359924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f987c15e8d0 0x7f987c0cc240 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f987800ba60 tx=0x7f987801eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810b46a192695f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:30.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9880a28640 1 -- 172.21.15.29:0/2343359924 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98780b4070 con 0x7f987c15e8d0 2024-04-25T00:39:30.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9880a28640 1 -- 172.21.15.29:0/2343359924 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98780a7c30 con 0x7f987c15e8d0 2024-04-25T00:39:30.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9880a28640 1 -- 172.21.15.29:0/2343359924 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98780af440 con 0x7f987c15e8d0 2024-04-25T00:39:30.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 -- 172.21.15.29:0/2343359924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f987c15e8d0 msgr2=0x7f987c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 --2- 172.21.15.29:0/2343359924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f987c15e8d0 0x7f987c0cc240 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f987800ba60 tx=0x7f987801eb40 comp rx=0 tx=0).stop 2024-04-25T00:39:30.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 -- 172.21.15.29:0/2343359924 shutdown_connections 2024-04-25T00:39:30.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 --2- 172.21.15.29:0/2343359924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f987c15f3c0 0x7f987c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 --2- 172.21.15.29:0/2343359924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f987c15e8d0 0x7f987c0cc240 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 -- 172.21.15.29:0/2343359924 >> 172.21.15.29:0/2343359924 conn(0x7f987c0b1870 msgr2=0x7f987c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:30.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 -- 172.21.15.29:0/2343359924 shutdown_connections 2024-04-25T00:39:30.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 -- 172.21.15.29:0/2343359924 wait complete. 2024-04-25T00:39:30.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 Processor -- start 2024-04-25T00:39:30.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 -- start start 2024-04-25T00:39:30.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f987c0b4780 0x7f987c156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f987c15e8d0 0x7f987c13aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f987c15f3c0 0x7f987c13cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f987c15d6a0 con 0x7f987c15e8d0 2024-04-25T00:39:30.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f987c15d520 con 0x7f987c15f3c0 2024-04-25T00:39:30.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9883c8e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f987c06d5d0 con 0x7f987c0b4780 2024-04-25T00:39:30.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9881229640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f987c15e8d0 0x7f987c13aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9881229640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f987c15e8d0 0x7f987c13aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43524/0 (socket says 172.21.15.29:43524) 2024-04-25T00:39:30.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9881229640 1 -- 172.21.15.29:0/3992852090 learned_addr learned my addr 172.21.15.29:0/3992852090 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:30.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9881a2a640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f987c0b4780 0x7f987c156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f988222b640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f987c15f3c0 0x7f987c13cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9881229640 1 -- 172.21.15.29:0/3992852090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f987c0b4780 msgr2=0x7f987c156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9881229640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f987c0b4780 0x7f987c156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9881229640 1 -- 172.21.15.29:0/3992852090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f987c15f3c0 msgr2=0x7f987c13cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9881229640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f987c15f3c0 0x7f987c13cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9881229640 1 -- 172.21.15.29:0/3992852090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c1f8510 con 0x7f987c15e8d0 2024-04-25T00:39:30.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f9881229640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f987c15e8d0 0x7f987c13aee0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f986c0011f0 tx=0x7f986c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:30.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f98727fc640 1 -- 172.21.15.29:0/3992852090 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c018070 con 0x7f987c15e8d0 2024-04-25T00:39:30.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f98727fc640 1 -- 172.21.15.29:0/3992852090 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986c0074e0 con 0x7f987c15e8d0 2024-04-25T00:39:30.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.098+0000 7f98727fc640 1 -- 172.21.15.29:0/3992852090 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c004a10 con 0x7f987c15e8d0 2024-04-25T00:39:30.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.102+0000 7f9883c8e640 1 -- 172.21.15.29:0/3992852090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987c1f7990 con 0x7f987c15e8d0 2024-04-25T00:39:30.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.102+0000 7f9883c8e640 1 -- 172.21.15.29:0/3992852090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f987c1f7f50 con 0x7f987c15e8d0 2024-04-25T00:39:30.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.102+0000 7f9883c8e640 1 -- 172.21.15.29:0/3992852090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f987c069270 con 0x7f987c15e8d0 2024-04-25T00:39:30.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.102+0000 7f98727fc640 1 -- 172.21.15.29:0/3992852090 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f986c004bb0 con 0x7f987c15e8d0 2024-04-25T00:39:30.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.102+0000 7f98727fc640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9860067ef0 0x7f986006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.102+0000 7f98727fc640 1 -- 172.21.15.29:0/3992852090 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f986c085b50 con 0x7f987c15e8d0 2024-04-25T00:39:30.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.102+0000 7f9881a2a640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9860067ef0 0x7f986006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.102+0000 7f9881a2a640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9860067ef0 0x7f986006a3b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f98780b1000 tx=0x7f9878008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:30.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.106+0000 7f98727fc640 1 -- 172.21.15.29:0/3992852090 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f986c055d30 con 0x7f987c15e8d0 2024-04-25T00:39:30.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.262+0000 7f9883c8e640 1 -- 172.21.15.29:0/3992852090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f987c0ad8d0 con 0x7f987c15e8d0 2024-04-25T00:39:30.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.262+0000 7f98727fc640 1 -- 172.21.15.29:0/3992852090 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16680 (secure 0 0 0) 0x7f986c055630 con 0x7f987c15e8d0 2024-04-25T00:39:30.267 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:30.267 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":134,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:24.461420+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:06.791988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:07.503204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:30.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.266+0000 7f9883c8e640 1 -- 172.21.15.29:0/3992852090 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9860067ef0 msgr2=0x7f986006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.266+0000 7f9883c8e640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9860067ef0 0x7f986006a3b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f98780b1000 tx=0x7f9878008040 comp rx=0 tx=0).stop 2024-04-25T00:39:30.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.266+0000 7f9883c8e640 1 -- 172.21.15.29:0/3992852090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f987c15e8d0 msgr2=0x7f987c13aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.266+0000 7f9883c8e640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f987c15e8d0 0x7f987c13aee0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f986c0011f0 tx=0x7f986c00c9b0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.266+0000 7f9883c8e640 1 -- 172.21.15.29:0/3992852090 shutdown_connections 2024-04-25T00:39:30.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.266+0000 7f9883c8e640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9860067ef0 0x7f986006a3b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.266+0000 7f9883c8e640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f987c15f3c0 0x7f987c13cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.266+0000 7f9883c8e640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f987c15e8d0 0x7f987c13aee0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.266+0000 7f9883c8e640 1 --2- 172.21.15.29:0/3992852090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f987c0b4780 0x7f987c156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.266+0000 7f9883c8e640 1 -- 172.21.15.29:0/3992852090 >> 172.21.15.29:0/3992852090 conn(0x7f987c0b1870 msgr2=0x7f987c0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:30.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.266+0000 7f9883c8e640 1 -- 172.21.15.29:0/3992852090 shutdown_connections 2024-04-25T00:39:30.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.266+0000 7f9883c8e640 1 -- 172.21.15.29:0/3992852090 wait complete. 2024-04-25T00:39:30.284 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:39:06.791988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:39:07.503204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:39:30.285 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:39:30.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facbd9dc640 1 Processor -- start 2024-04-25T00:39:30.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facbd9dc640 1 -- start start 2024-04-25T00:39:30.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facbd9dc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80b3140 0x7facb806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facbd9dc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7facb80b4b10 0x7facb80b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facbd9dc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facb80b41e0 con 0x7facb80b3a60 2024-04-25T00:39:30.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facbd9dc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7facb806d960 con 0x7facb80b4b10 2024-04-25T00:39:30.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facbd9dc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7facb806dc40 con 0x7facb80b3140 2024-04-25T00:39:30.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facb6ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80b3140 0x7facb806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facb67fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7facb80b4b10 0x7facb80b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facb67fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7facb80b4b10 0x7facb80b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:59026/0 (socket says 172.21.15.29:59026) 2024-04-25T00:39:30.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facb77fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7facb80b3a60 0x7facb815c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:39878/0 (socket says 172.21.15.29:39878) 2024-04-25T00:39:30.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facb67fc640 1 -- 172.21.15.29:0/1683061969 learned_addr learned my addr 172.21.15.29:0/1683061969 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:30.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facb67fc640 1 -- 172.21.15.29:0/1683061969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80b3140 msgr2=0x7facb806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:30.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facb67fc640 1 --2- 172.21.15.29:0/1683061969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80b3140 0x7facb806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facb67fc640 1 -- 172.21.15.29:0/1683061969 >> v1:172.21.15.157:6789/0 conn(0x7facb80b3a60 legacy=0x7facb815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facb67fc640 1 -- 172.21.15.29:0/1683061969 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb806df20 con 0x7facb80b4b10 2024-04-25T00:39:30.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.410+0000 7facb6ffd640 1 --2- 172.21.15.29:0/1683061969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80b3140 0x7facb806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:39:30.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facb67fc640 1 --2- 172.21.15.29:0/1683061969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80b4b10 0x7facb80b3520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7facac009870 tx=0x7facac01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf0e49be29e0efcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:30.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facb5ffb640 1 -- 172.21.15.29:0/1683061969 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac0b0070 con 0x7facb80b4b10 2024-04-25T00:39:30.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facb5ffb640 1 -- 172.21.15.29:0/1683061969 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facac0ab600 con 0x7facb80b4b10 2024-04-25T00:39:30.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facb5ffb640 1 -- 172.21.15.29:0/1683061969 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac0ab9d0 con 0x7facb80b4b10 2024-04-25T00:39:30.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facbd9dc640 1 -- 172.21.15.29:0/1683061969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80b4b10 msgr2=0x7facb80b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facbd9dc640 1 --2- 172.21.15.29:0/1683061969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80b4b10 0x7facb80b3520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7facac009870 tx=0x7facac01c560 comp rx=0 tx=0).stop 2024-04-25T00:39:30.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facbd9dc640 1 -- 172.21.15.29:0/1683061969 shutdown_connections 2024-04-25T00:39:30.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facbd9dc640 1 --2- 172.21.15.29:0/1683061969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80b4b10 0x7facb80b3520 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facbd9dc640 1 --2- 172.21.15.29:0/1683061969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80b3140 0x7facb806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facbd9dc640 1 -- 172.21.15.29:0/1683061969 >> 172.21.15.29:0/1683061969 conn(0x7facb80b1870 msgr2=0x7facb80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:30.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facbd9dc640 1 -- 172.21.15.29:0/1683061969 shutdown_connections 2024-04-25T00:39:30.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facbd9dc640 1 -- 172.21.15.29:0/1683061969 wait complete. 2024-04-25T00:39:30.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facbd9dc640 1 Processor -- start 2024-04-25T00:39:30.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.414+0000 7facbd9dc640 1 -- start start 2024-04-25T00:39:30.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facbd9dc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80b3140 0x7facb815b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facbd9dc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facb80b3a60 0x7facb8071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facbd9dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80b4b10 0x7facb8071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facbd9dc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7facb8168b70 con 0x7facb80b3a60 2024-04-25T00:39:30.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facbd9dc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7facb81689f0 con 0x7facb80b4b10 2024-04-25T00:39:30.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facbd9dc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7facb8168cf0 con 0x7facb80b3140 2024-04-25T00:39:30.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facb67fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facb80b3a60 0x7facb8071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facb67fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facb80b3a60 0x7facb8071790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43536/0 (socket says 172.21.15.29:43536) 2024-04-25T00:39:30.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facb67fc640 1 -- 172.21.15.29:0/3301934001 learned_addr learned my addr 172.21.15.29:0/3301934001 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:30.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facb6ffd640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80b3140 0x7facb815b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facb77fe640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80b4b10 0x7facb8071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facb67fc640 1 -- 172.21.15.29:0/3301934001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80b3140 msgr2=0x7facb815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facb67fc640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80b3140 0x7facb815b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facb67fc640 1 -- 172.21.15.29:0/3301934001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80b4b10 msgr2=0x7facb8071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facb67fc640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80b4b10 0x7facb8071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facb67fc640 1 -- 172.21.15.29:0/3301934001 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb815b7b0 con 0x7facb80b3a60 2024-04-25T00:39:30.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facb6ffd640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80b3140 0x7facb815b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:30.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facb67fc640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facb80b3a60 0x7facb8071790 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7facac0ae040 tx=0x7facac019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:30.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7fac9ffff640 1 -- 172.21.15.29:0/3301934001 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac0b0050 con 0x7facb80b3a60 2024-04-25T00:39:30.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7fac9ffff640 1 -- 172.21.15.29:0/3301934001 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facac0b1040 con 0x7facb80b3a60 2024-04-25T00:39:30.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facbd9dc640 1 -- 172.21.15.29:0/3301934001 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb813a3c0 con 0x7facb80b3a60 2024-04-25T00:39:30.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7fac9ffff640 1 -- 172.21.15.29:0/3301934001 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac0b5850 con 0x7facb80b3a60 2024-04-25T00:39:30.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facbd9dc640 1 -- 172.21.15.29:0/3301934001 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb813a980 con 0x7facb80b3a60 2024-04-25T00:39:30.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facbd9dc640 1 -- 172.21.15.29:0/3301934001 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb8069270 con 0x7facb80b3a60 2024-04-25T00:39:30.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7fac9ffff640 1 -- 172.21.15.29:0/3301934001 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7facac0be020 con 0x7facb80b3a60 2024-04-25T00:39:30.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7fac9ffff640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faca4067ef0 0x7faca406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7fac9ffff640 1 -- 172.21.15.29:0/3301934001 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7facac120630 con 0x7facb80b3a60 2024-04-25T00:39:30.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.418+0000 7facb6ffd640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faca4067ef0 0x7faca406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.422+0000 7facb6ffd640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faca4067ef0 0x7faca406a3b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7facb8139390 tx=0x7faca8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:30.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.422+0000 7fac9ffff640 1 -- 172.21.15.29:0/3301934001 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7facac016150 con 0x7facb80b3a60 2024-04-25T00:39:30.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.586+0000 7facbd9dc640 1 -- 172.21.15.29:0/3301934001 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7facb813fba0 con 0x7facb80b3a60 2024-04-25T00:39:30.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.586+0000 7fac9ffff640 1 -- 172.21.15.29:0/3301934001 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7facac0f0990 con 0x7facb80b3a60 2024-04-25T00:39:30.590 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:39:30.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.590+0000 7facbd9dc640 1 -- 172.21.15.29:0/3301934001 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faca4067ef0 msgr2=0x7faca406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.590+0000 7facbd9dc640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faca4067ef0 0x7faca406a3b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7facb8139390 tx=0x7faca8008040 comp rx=0 tx=0).stop 2024-04-25T00:39:30.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.590+0000 7facbd9dc640 1 -- 172.21.15.29:0/3301934001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facb80b3a60 msgr2=0x7facb8071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.590+0000 7facbd9dc640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facb80b3a60 0x7facb8071790 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7facac0ae040 tx=0x7facac019300 comp rx=0 tx=0).stop 2024-04-25T00:39:30.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.590+0000 7facbd9dc640 1 -- 172.21.15.29:0/3301934001 shutdown_connections 2024-04-25T00:39:30.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.590+0000 7facbd9dc640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faca4067ef0 0x7faca406a3b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.590+0000 7facbd9dc640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80b4b10 0x7facb8071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.590+0000 7facbd9dc640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facb80b3a60 0x7facb8071790 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.590+0000 7facbd9dc640 1 --2- 172.21.15.29:0/3301934001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80b3140 0x7facb815b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.590+0000 7facbd9dc640 1 -- 172.21.15.29:0/3301934001 >> 172.21.15.29:0/3301934001 conn(0x7facb80b1870 msgr2=0x7facb80ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:30.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.590+0000 7facbd9dc640 1 -- 172.21.15.29:0/3301934001 shutdown_connections 2024-04-25T00:39:30.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.590+0000 7facbd9dc640 1 -- 172.21.15.29:0/3301934001 wait complete. 2024-04-25T00:39:30.610 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:39:30.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.730+0000 7f9f6e0c1640 1 Processor -- start 2024-04-25T00:39:30.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.730+0000 7f9f6e0c1640 1 -- start start 2024-04-25T00:39:30.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f6e0c1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f68162420 0x7f9f68070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f6e0c1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f680b4e00 0x7f9f680b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f6e0c1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f68071910 con 0x7f9f680b30b0 2024-04-25T00:39:30.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f6e0c1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9f680b3500 con 0x7f9f68162420 2024-04-25T00:39:30.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f6e0c1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f68071bf0 con 0x7f9f680b4e00 2024-04-25T00:39:30.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f677fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f68162420 0x7f9f68070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f67fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9f680b30b0 0x7f9f68166020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:39886/0 (socket says 172.21.15.29:39886) 2024-04-25T00:39:30.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f67fff640 1 -- 172.21.15.29:0/757026624 learned_addr learned my addr 172.21.15.29:0/757026624 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:30.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f5effd640 1 --2- 172.21.15.29:0/757026624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f680b4e00 0x7f9f680b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f677fe640 1 -- 172.21.15.29:0/757026624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f680b4e00 msgr2=0x7f9f680b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:30.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f677fe640 1 --2- 172.21.15.29:0/757026624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f680b4e00 0x7f9f680b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f677fe640 1 -- 172.21.15.29:0/757026624 >> v1:172.21.15.157:6789/0 conn(0x7f9f680b30b0 legacy=0x7f9f68166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f677fe640 1 -- 172.21.15.29:0/757026624 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f68071ed0 con 0x7f9f68162420 2024-04-25T00:39:30.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.734+0000 7f9f5effd640 1 --2- 172.21.15.29:0/757026624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f680b4e00 0x7f9f680b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:39:30.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f677fe640 1 --2- 172.21.15.29:0/757026624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f68162420 0x7f9f68070f80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9f580097c0 tx=0x7f9f5801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e86ea6adcf4ab734 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:30.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f66ffd640 1 -- 172.21.15.29:0/757026624 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f580b0070 con 0x7f9f68162420 2024-04-25T00:39:30.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f66ffd640 1 -- 172.21.15.29:0/757026624 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f580ab600 con 0x7f9f68162420 2024-04-25T00:39:30.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f66ffd640 1 -- 172.21.15.29:0/757026624 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f580ab9d0 con 0x7f9f68162420 2024-04-25T00:39:30.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/757026624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f68162420 msgr2=0x7f9f68070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 --2- 172.21.15.29:0/757026624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f68162420 0x7f9f68070f80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9f580097c0 tx=0x7f9f5801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/757026624 shutdown_connections 2024-04-25T00:39:30.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 --2- 172.21.15.29:0/757026624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f680b4e00 0x7f9f680b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 --2- 172.21.15.29:0/757026624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f68162420 0x7f9f68070f80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/757026624 >> 172.21.15.29:0/757026624 conn(0x7f9f680b1870 msgr2=0x7f9f680b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:30.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/757026624 shutdown_connections 2024-04-25T00:39:30.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/757026624 wait complete. 2024-04-25T00:39:30.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 Processor -- start 2024-04-25T00:39:30.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 -- start start 2024-04-25T00:39:30.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f680b30b0 0x7f9f6815a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f680b4e00 0x7f9f68164d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f68162420 0x7f9f68156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6816f3c0 con 0x7f9f680b4e00 2024-04-25T00:39:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f680716d0 con 0x7f9f680b30b0 2024-04-25T00:39:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f6e0c1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6816f540 con 0x7f9f68162420 2024-04-25T00:39:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f5effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f680b4e00 0x7f9f68164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f5effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f680b4e00 0x7f9f68164d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43552/0 (socket says 172.21.15.29:43552) 2024-04-25T00:39:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.738+0000 7f9f5effd640 1 -- 172.21.15.29:0/1692793916 learned_addr learned my addr 172.21.15.29:0/1692793916 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f677fe640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f680b30b0 0x7f9f6815a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f67fff640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f68162420 0x7f9f68156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f5effd640 1 -- 172.21.15.29:0/1692793916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f68162420 msgr2=0x7f9f68156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f5effd640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f68162420 0x7f9f68156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f5effd640 1 -- 172.21.15.29:0/1692793916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f680b30b0 msgr2=0x7f9f6815a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f5effd640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f680b30b0 0x7f9f6815a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f5effd640 1 -- 172.21.15.29:0/1692793916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f681598a0 con 0x7f9f680b4e00 2024-04-25T00:39:30.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f5effd640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f680b4e00 0x7f9f68164d00 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000ea00 tx=0x7f9f5000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f64ff9640 1 -- 172.21.15.29:0/1692793916 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50017070 con 0x7f9f680b4e00 2024-04-25T00:39:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f64ff9640 1 -- 172.21.15.29:0/1692793916 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5000ca60 con 0x7f9f680b4e00 2024-04-25T00:39:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/1692793916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f68159b80 con 0x7f9f680b4e00 2024-04-25T00:39:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f64ff9640 1 -- 172.21.15.29:0/1692793916 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000ce50 con 0x7f9f680b4e00 2024-04-25T00:39:30.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/1692793916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6815a140 con 0x7f9f680b4e00 2024-04-25T00:39:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/1692793916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f68069270 con 0x7f9f680b4e00 2024-04-25T00:39:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f64ff9640 1 -- 172.21.15.29:0/1692793916 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f9f5001a070 con 0x7f9f680b4e00 2024-04-25T00:39:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f64ff9640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f48067f20 0x7f9f4806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f64ff9640 1 -- 172.21.15.29:0/1692793916 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f9f50086630 con 0x7f9f680b4e00 2024-04-25T00:39:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f677fe640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f48067f20 0x7f9f4806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:30.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.742+0000 7f9f677fe640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f48067f20 0x7f9f4806a3e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9f580ad000 tx=0x7f9f58008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:30.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.746+0000 7f9f64ff9640 1 -- 172.21.15.29:0/1692793916 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9f500562c0 con 0x7f9f680b4e00 2024-04-25T00:39:30.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.902+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/1692793916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9f68072fa0 con 0x7f9f680b4e00 2024-04-25T00:39:30.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.902+0000 7f9f64ff9640 1 -- 172.21.15.29:0/1692793916 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) ==== 104+0+11 (secure 0 0 0) 0x7f9f5005a5c0 con 0x7f9f680b4e00 2024-04-25T00:39:30.905 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:39:30.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.902+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/1692793916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f48067f20 msgr2=0x7f9f4806a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.902+0000 7f9f6e0c1640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f48067f20 0x7f9f4806a3e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9f580ad000 tx=0x7f9f58008040 comp rx=0 tx=0).stop 2024-04-25T00:39:30.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.902+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/1692793916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f680b4e00 msgr2=0x7f9f68164d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:30.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.902+0000 7f9f6e0c1640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f680b4e00 0x7f9f68164d00 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000ea00 tx=0x7f9f5000eec0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.902+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/1692793916 shutdown_connections 2024-04-25T00:39:30.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.902+0000 7f9f6e0c1640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f48067f20 0x7f9f4806a3e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.902+0000 7f9f6e0c1640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f68162420 0x7f9f68156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.902+0000 7f9f6e0c1640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f680b4e00 0x7f9f68164d00 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.902+0000 7f9f6e0c1640 1 --2- 172.21.15.29:0/1692793916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f680b30b0 0x7f9f6815a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:30.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.902+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/1692793916 >> 172.21.15.29:0/1692793916 conn(0x7f9f680b1870 msgr2=0x7f9f680cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:30.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.906+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/1692793916 shutdown_connections 2024-04-25T00:39:30.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:30.906+0000 7f9f6e0c1640 1 -- 172.21.15.29:0/1692793916 wait complete. 2024-04-25T00:39:30.923 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:39:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.038+0000 7f9f11d03640 1 Processor -- start 2024-04-25T00:39:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.038+0000 7f9f11d03640 1 -- start start 2024-04-25T00:39:31.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.038+0000 7f9f11d03640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9f0c0b3c30 0x7f9f0c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.038+0000 7f9f11d03640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f0c0b4b10 0x7f9f0c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.038+0000 7f9f11d03640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f0c0b41e0 con 0x7f9f0c0b3140 2024-04-25T00:39:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.038+0000 7f9f11d03640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9f0c06d960 con 0x7f9f0c0b3c30 2024-04-25T00:39:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.038+0000 7f9f11d03640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9f0c06dc40 con 0x7f9f0c0b4b10 2024-04-25T00:39:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.038+0000 7f9f0affd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9f0c0b3c30 0x7f9f0c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.038+0000 7f9f0affd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9f0c0b3c30 0x7f9f0c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43568/0 (socket says 172.21.15.29:43568) 2024-04-25T00:39:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.038+0000 7f9f0b7fe640 1 --1- 172.21.15.29:0/3940646324 >> v1:172.21.15.29:6789/0 conn(0x7f9f0c0b3140 0x7f9f0c0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42142/0 (socket says 172.21.15.29:42142) 2024-04-25T00:39:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.038+0000 7f9f0affd640 1 -- 172.21.15.29:0/3940646324 learned_addr learned my addr 172.21.15.29:0/3940646324 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f0bfff640 1 --2- 172.21.15.29:0/3940646324 >> v2:172.21.15.157:3300/0 conn(0x7f9f0c0b4b10 0x7f9f0c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f0affd640 1 -- 172.21.15.29:0/3940646324 >> v2:172.21.15.157:3300/0 conn(0x7f9f0c0b4b10 msgr2=0x7f9f0c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f0affd640 1 --2- 172.21.15.29:0/3940646324 >> v2:172.21.15.157:3300/0 conn(0x7f9f0c0b4b10 0x7f9f0c0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f0affd640 1 -- 172.21.15.29:0/3940646324 >> v1:172.21.15.29:6789/0 conn(0x7f9f0c0b3140 legacy=0x7f9f0c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f0affd640 1 -- 172.21.15.29:0/3940646324 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0c06df20 con 0x7f9f0c0b3c30 2024-04-25T00:39:31.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f0a7fc640 1 -- 172.21.15.29:0/3940646324 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1887038975 0 0) 0x7f9f0c0b41e0 con 0x7f9f0c0b3140 2024-04-25T00:39:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f0affd640 1 --2- 172.21.15.29:0/3940646324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f0c0b3c30 0x7f9f0c06e440 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009870 tx=0x7f9f0001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5952861d8c7d3f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f0a7fc640 1 -- 172.21.15.29:0/3940646324 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f000b0070 con 0x7f9f0c0b3c30 2024-04-25T00:39:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f0a7fc640 1 -- 172.21.15.29:0/3940646324 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f000ab600 con 0x7f9f0c0b3c30 2024-04-25T00:39:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f0a7fc640 1 -- 172.21.15.29:0/3940646324 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f000ab9d0 con 0x7f9f0c0b3c30 2024-04-25T00:39:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f11d03640 1 -- 172.21.15.29:0/3940646324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f0c0b3c30 msgr2=0x7f9f0c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f11d03640 1 --2- 172.21.15.29:0/3940646324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f0c0b3c30 0x7f9f0c06e440 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009870 tx=0x7f9f0001c520 comp rx=0 tx=0).stop 2024-04-25T00:39:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f11d03640 1 -- 172.21.15.29:0/3940646324 shutdown_connections 2024-04-25T00:39:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f11d03640 1 --2- 172.21.15.29:0/3940646324 >> v2:172.21.15.157:3300/0 conn(0x7f9f0c0b4b10 0x7f9f0c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f11d03640 1 --2- 172.21.15.29:0/3940646324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f0c0b3c30 0x7f9f0c06e440 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f11d03640 1 -- 172.21.15.29:0/3940646324 >> 172.21.15.29:0/3940646324 conn(0x7f9f0c0b1870 msgr2=0x7f9f0c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f11d03640 1 -- 172.21.15.29:0/3940646324 shutdown_connections 2024-04-25T00:39:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.042+0000 7f9f11d03640 1 -- 172.21.15.29:0/3940646324 wait complete. 2024-04-25T00:39:31.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f11d03640 1 Processor -- start 2024-04-25T00:39:31.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f11d03640 1 -- start start 2024-04-25T00:39:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f11d03640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f0c0b3140 0x7f9f0c1432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f11d03640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f0c0b3c30 0x7f9f0c153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f11d03640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0c0b4b10 0x7f9f0c1540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f11d03640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c168d80 con 0x7f9f0c0b3140 2024-04-25T00:39:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f11d03640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c168c00 con 0x7f9f0c0b4b10 2024-04-25T00:39:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f11d03640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f0c168f00 con 0x7f9f0c0b3c30 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f0b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f0c0b3140 0x7f9f0c1432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f0b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f0c0b3140 0x7f9f0c1432d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43582/0 (socket says 172.21.15.29:43582) 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f0b7fe640 1 -- 172.21.15.29:0/2430144289 learned_addr learned my addr 172.21.15.29:0/2430144289 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f0bfff640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0c0b4b10 0x7f9f0c1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f0affd640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f0c0b3c30 0x7f9f0c153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f0b7fe640 1 -- 172.21.15.29:0/2430144289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f0c0b3c30 msgr2=0x7f9f0c153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f0b7fe640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f0c0b3c30 0x7f9f0c153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f0b7fe640 1 -- 172.21.15.29:0/2430144289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0c0b4b10 msgr2=0x7f9f0c1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f0b7fe640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0c0b4b10 0x7f9f0c1540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f0b7fe640 1 -- 172.21.15.29:0/2430144289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0c15ac60 con 0x7f9f0c0b3140 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f0bfff640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0c0b4b10 0x7f9f0c1540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f0b7fe640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f0c0b3140 0x7f9f0c1432d0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00c4b0 tx=0x7f9efc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9ef3fff640 1 -- 172.21.15.29:0/2430144289 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc018070 con 0x7f9f0c0b3140 2024-04-25T00:39:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9ef3fff640 1 -- 172.21.15.29:0/2430144289 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9efc007500 con 0x7f9f0c0b3140 2024-04-25T00:39:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f11d03640 1 -- 172.21.15.29:0/2430144289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0c15af40 con 0x7f9f0c0b3140 2024-04-25T00:39:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9ef3fff640 1 -- 172.21.15.29:0/2430144289 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc004a30 con 0x7f9f0c0b3140 2024-04-25T00:39:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f11d03640 1 -- 172.21.15.29:0/2430144289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0c15b500 con 0x7f9f0c0b3140 2024-04-25T00:39:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9f11d03640 1 -- 172.21.15.29:0/2430144289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f0c069270 con 0x7f9f0c0b3140 2024-04-25T00:39:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.046+0000 7f9ef3fff640 1 -- 172.21.15.29:0/2430144289 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f9efc01b070 con 0x7f9f0c0b3140 2024-04-25T00:39:31.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.050+0000 7f9ef3fff640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ef8067ef0 0x7f9ef806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.050+0000 7f9f0affd640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ef8067ef0 0x7f9ef806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.050+0000 7f9ef3fff640 1 -- 172.21.15.29:0/2430144289 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f9efc085a60 con 0x7f9f0c0b3140 2024-04-25T00:39:31.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.050+0000 7f9f0affd640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ef8067ef0 0x7f9ef806a3b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9f000ad000 tx=0x7f9f00008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:31.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.054+0000 7f9ef3fff640 1 -- 172.21.15.29:0/2430144289 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9efc055cd0 con 0x7f9f0c0b3140 2024-04-25T00:39:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.210+0000 7f9f11d03640 1 -- 172.21.15.29:0/2430144289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9f0c0b36c0 con 0x7f9f0c0b3140 2024-04-25T00:39:31.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.210+0000 7f9ef3fff640 1 -- 172.21.15.29:0/2430144289 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) ==== 100+0+11 (secure 0 0 0) 0x7f9efc0555d0 con 0x7f9f0c0b3140 2024-04-25T00:39:31.213 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:39:31.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.210+0000 7f9f11d03640 1 -- 172.21.15.29:0/2430144289 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ef8067ef0 msgr2=0x7f9ef806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.210+0000 7f9f11d03640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ef8067ef0 0x7f9ef806a3b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9f000ad000 tx=0x7f9f00008040 comp rx=0 tx=0).stop 2024-04-25T00:39:31.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.210+0000 7f9f11d03640 1 -- 172.21.15.29:0/2430144289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f0c0b3140 msgr2=0x7f9f0c1432d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.210+0000 7f9f11d03640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f0c0b3140 0x7f9f0c1432d0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00c4b0 tx=0x7f9efc00c970 comp rx=0 tx=0).stop 2024-04-25T00:39:31.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.210+0000 7f9f11d03640 1 -- 172.21.15.29:0/2430144289 shutdown_connections 2024-04-25T00:39:31.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.210+0000 7f9f11d03640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ef8067ef0 0x7f9ef806a3b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.210+0000 7f9f11d03640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0c0b4b10 0x7f9f0c1540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.210+0000 7f9f11d03640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f0c0b3c30 0x7f9f0c153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.210+0000 7f9f11d03640 1 --2- 172.21.15.29:0/2430144289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f0c0b3140 0x7f9f0c1432d0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.210+0000 7f9f11d03640 1 -- 172.21.15.29:0/2430144289 >> 172.21.15.29:0/2430144289 conn(0x7f9f0c0b1870 msgr2=0x7f9f0c155030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:31.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.214+0000 7f9f11d03640 1 -- 172.21.15.29:0/2430144289 shutdown_connections 2024-04-25T00:39:31.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.214+0000 7f9f11d03640 1 -- 172.21.15.29:0/2430144289 wait complete. 2024-04-25T00:39:31.231 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:39:31.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.354+0000 7f27e85bf640 1 Processor -- start 2024-04-25T00:39:31.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.354+0000 7f27e85bf640 1 -- start start 2024-04-25T00:39:31.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.354+0000 7f27e85bf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27e015e200 0x7f27e00d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.354+0000 7f27e85bf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e00cf690 0x7f27e00d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.354+0000 7f27e85bf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27e00d2a90 0x7f27e00d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.354+0000 7f27e85bf640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f27e00d2ee0 con 0x7f27e00d2a90 2024-04-25T00:39:31.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.354+0000 7f27e85bf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f27e00b38d0 con 0x7f27e015e200 2024-04-25T00:39:31.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.354+0000 7f27e85bf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f27e00cfc10 con 0x7f27e00cf690 2024-04-25T00:39:31.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.354+0000 7f27e6b5c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27e00d2a90 0x7f27e00d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.354+0000 7f27e6b5c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27e00d2a90 0x7f27e00d05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43596/0 (socket says 172.21.15.29:43596) 2024-04-25T00:39:31.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.354+0000 7f27e6b5c640 1 -- 172.21.15.29:0/794462972 learned_addr learned my addr 172.21.15.29:0/794462972 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:31.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.354+0000 7f27e5b5a640 1 --2- 172.21.15.29:0/794462972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e00cf690 0x7f27e00d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.354+0000 7f27e635b640 1 --2- 172.21.15.29:0/794462972 >> v2:172.21.15.157:3300/0 conn(0x7f27e015e200 0x7f27e00d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e6b5c640 1 -- 172.21.15.29:0/794462972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e00cf690 msgr2=0x7f27e00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:31.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e6b5c640 1 --2- 172.21.15.29:0/794462972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e00cf690 0x7f27e00d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e6b5c640 1 -- 172.21.15.29:0/794462972 >> v2:172.21.15.157:3300/0 conn(0x7f27e015e200 msgr2=0x7f27e00d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:31.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e6b5c640 1 --2- 172.21.15.29:0/794462972 >> v2:172.21.15.157:3300/0 conn(0x7f27e015e200 0x7f27e00d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e6b5c640 1 -- 172.21.15.29:0/794462972 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e00cfef0 con 0x7f27e00d2a90 2024-04-25T00:39:31.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e6b5c640 1 --2- 172.21.15.29:0/794462972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27e00d2a90 0x7f27e00d05a0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f27e008d8f0 tx=0x7f27d801c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1085d707457ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:31.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e5359640 1 -- 172.21.15.29:0/794462972 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d80a5580 con 0x7f27e00d2a90 2024-04-25T00:39:31.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e5359640 1 -- 172.21.15.29:0/794462972 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d80a5720 con 0x7f27e00d2a90 2024-04-25T00:39:31.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e5359640 1 -- 172.21.15.29:0/794462972 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d80a5af0 con 0x7f27e00d2a90 2024-04-25T00:39:31.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e85bf640 1 -- 172.21.15.29:0/794462972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27e00d2a90 msgr2=0x7f27e00d05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e85bf640 1 --2- 172.21.15.29:0/794462972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27e00d2a90 0x7f27e00d05a0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f27e008d8f0 tx=0x7f27d801c700 comp rx=0 tx=0).stop 2024-04-25T00:39:31.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e85bf640 1 -- 172.21.15.29:0/794462972 shutdown_connections 2024-04-25T00:39:31.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e85bf640 1 --2- 172.21.15.29:0/794462972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27e00d2a90 0x7f27e00d05a0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e85bf640 1 --2- 172.21.15.29:0/794462972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e00cf690 0x7f27e00d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e85bf640 1 --2- 172.21.15.29:0/794462972 >> v2:172.21.15.157:3300/0 conn(0x7f27e015e200 0x7f27e00d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e85bf640 1 -- 172.21.15.29:0/794462972 >> 172.21.15.29:0/794462972 conn(0x7f27e009d280 msgr2=0x7f27e00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:31.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e85bf640 1 -- 172.21.15.29:0/794462972 shutdown_connections 2024-04-25T00:39:31.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.358+0000 7f27e85bf640 1 -- 172.21.15.29:0/794462972 wait complete. 2024-04-25T00:39:31.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e85bf640 1 Processor -- start 2024-04-25T00:39:31.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e85bf640 1 -- start start 2024-04-25T00:39:31.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e85bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e00cf690 0x7f27e015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e85bf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e00d2a90 0x7f27e0156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e85bf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27e015e200 0x7f27e00ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e85bf640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f27e015e760 con 0x7f27e015e200 2024-04-25T00:39:31.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e85bf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27e015e5e0 con 0x7f27e00cf690 2024-04-25T00:39:31.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e85bf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f27e00b4320 con 0x7f27e00d2a90 2024-04-25T00:39:31.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e6b5c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27e015e200 0x7f27e00ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e6b5c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27e015e200 0x7f27e00ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43602/0 (socket says 172.21.15.29:43602) 2024-04-25T00:39:31.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e6b5c640 1 -- 172.21.15.29:0/614039258 learned_addr learned my addr 172.21.15.29:0/614039258 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:31.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e5b5a640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e00d2a90 0x7f27e0156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e635b640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e00cf690 0x7f27e015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e6b5c640 1 -- 172.21.15.29:0/614039258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e00d2a90 msgr2=0x7f27e0156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e6b5c640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e00d2a90 0x7f27e0156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e6b5c640 1 -- 172.21.15.29:0/614039258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e00cf690 msgr2=0x7f27e015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e6b5c640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e00cf690 0x7f27e015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e6b5c640 1 -- 172.21.15.29:0/614039258 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e0159790 con 0x7f27e015e200 2024-04-25T00:39:31.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e635b640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e00cf690 0x7f27e015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:31.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e6b5c640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27e015e200 0x7f27e00ca2b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f27d80af040 tx=0x7f27d80194a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:31.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27d6ffd640 1 -- 172.21.15.29:0/614039258 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d80bc020 con 0x7f27e015e200 2024-04-25T00:39:31.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27d6ffd640 1 -- 172.21.15.29:0/614039258 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d80b8670 con 0x7f27e015e200 2024-04-25T00:39:31.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e85bf640 1 -- 172.21.15.29:0/614039258 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e0159a70 con 0x7f27e015e200 2024-04-25T00:39:31.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27d6ffd640 1 -- 172.21.15.29:0/614039258 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8009510 con 0x7f27e015e200 2024-04-25T00:39:31.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e85bf640 1 -- 172.21.15.29:0/614039258 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e015a310 con 0x7f27e015e200 2024-04-25T00:39:31.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27e85bf640 1 -- 172.21.15.29:0/614039258 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e0069270 con 0x7f27e015e200 2024-04-25T00:39:31.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.362+0000 7f27d6ffd640 1 -- 172.21.15.29:0/614039258 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f27d80bf020 con 0x7f27e015e200 2024-04-25T00:39:31.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.366+0000 7f27d6ffd640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27c4067f20 0x7f27c406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.366+0000 7f27d6ffd640 1 -- 172.21.15.29:0/614039258 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f27d8120fd0 con 0x7f27e015e200 2024-04-25T00:39:31.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.366+0000 7f27e635b640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27c4067f20 0x7f27c406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.366+0000 7f27e635b640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27c4067f20 0x7f27c406a3e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f27e00b2e50 tx=0x7f27dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:31.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.370+0000 7f27d6ffd640 1 -- 172.21.15.29:0/614039258 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f27d80c7020 con 0x7f27e015e200 2024-04-25T00:39:31.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27e85bf640 1 -- 172.21.15.29:0/614039258 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f27e0157cc0 con 0x7f27e015e200 2024-04-25T00:39:31.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27d6ffd640 1 -- 172.21.15.29:0/614039258 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v134) ==== 80+0+0 (secure 0 0 0) 0x7f27d80165e0 con 0x7f27e015e200 2024-04-25T00:39:31.544 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:39:31.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27e85bf640 1 -- 172.21.15.29:0/614039258 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27c4067f20 msgr2=0x7f27c406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27e85bf640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27c4067f20 0x7f27c406a3e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f27e00b2e50 tx=0x7f27dc008040 comp rx=0 tx=0).stop 2024-04-25T00:39:31.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27e85bf640 1 -- 172.21.15.29:0/614039258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27e015e200 msgr2=0x7f27e00ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27e85bf640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27e015e200 0x7f27e00ca2b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f27d80af040 tx=0x7f27d80194a0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27e85bf640 1 -- 172.21.15.29:0/614039258 shutdown_connections 2024-04-25T00:39:31.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27e85bf640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27c4067f20 0x7f27c406a3e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27e85bf640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27e015e200 0x7f27e00ca2b0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27e85bf640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27e00d2a90 0x7f27e0156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27e85bf640 1 --2- 172.21.15.29:0/614039258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e00cf690 0x7f27e015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27e85bf640 1 -- 172.21.15.29:0/614039258 >> 172.21.15.29:0/614039258 conn(0x7f27e009d280 msgr2=0x7f27e00d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:31.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27e85bf640 1 -- 172.21.15.29:0/614039258 shutdown_connections 2024-04-25T00:39:31.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.542+0000 7f27e85bf640 1 -- 172.21.15.29:0/614039258 wait complete. 2024-04-25T00:39:31.562 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-04-25T00:39:31.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:31.562 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:31.573 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[52579]: fuse finished with error 0 and tester_r 0 2024-04-25T00:39:31.574 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:39:31.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:31.574 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:39:31.619 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:39:31.619 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:39:31.619 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:39:31.626 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:31.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c23800640 1 Processor -- start 2024-04-25T00:39:31.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c23800640 1 -- start start 2024-04-25T00:39:31.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c23800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c1c0b3c30 0x7f7c1c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c23800640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c1c0b4b10 0x7f7c1c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c23800640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c1c0b41e0 con 0x7f7c1c0b3140 2024-04-25T00:39:31.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c23800640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7c1c06d960 con 0x7f7c1c0b4b10 2024-04-25T00:39:31.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c23800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1c06dc40 con 0x7f7c1c0b3c30 2024-04-25T00:39:31.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c2159c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7c1c0b3140 0x7f7c1c0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42152/0 (socket says 172.21.15.29:42152) 2024-04-25T00:39:31.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c2159c640 1 -- 172.21.15.29:0/3935522008 learned_addr learned my addr 172.21.15.29:0/3935522008 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:31.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c21d9d640 1 --2- 172.21.15.29:0/3935522008 >> v2:172.21.15.29:3300/0 conn(0x7f7c1c0b4b10 0x7f7c1c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c1bfff640 1 -- 172.21.15.29:0/3935522008 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 746707441 0 0) 0x7f7c1c0b41e0 con 0x7f7c1c0b3140 2024-04-25T00:39:31.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c20d9b640 1 --2- 172.21.15.29:0/3935522008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c1c0b3c30 0x7f7c1c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c1bfff640 1 -- 172.21.15.29:0/3935522008 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c04003530 con 0x7f7c1c0b3140 2024-04-25T00:39:31.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c21d9d640 1 -- 172.21.15.29:0/3935522008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c1c0b3c30 msgr2=0x7f7c1c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:31.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c21d9d640 1 --2- 172.21.15.29:0/3935522008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c1c0b3c30 0x7f7c1c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c21d9d640 1 -- 172.21.15.29:0/3935522008 >> v1:172.21.15.29:6789/0 conn(0x7f7c1c0b3140 legacy=0x7f7c1c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.782+0000 7f7c21d9d640 1 -- 172.21.15.29:0/3935522008 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1c06df20 con 0x7f7c1c0b4b10 2024-04-25T00:39:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c21d9d640 1 --2- 172.21.15.29:0/3935522008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1c0b4b10 0x7f7c1c0ce240 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f7c140097c0 tx=0x7f7c1401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7147065c96cae8ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c1bfff640 1 -- 172.21.15.29:0/3935522008 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c140b0070 con 0x7f7c1c0b4b10 2024-04-25T00:39:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c1bfff640 1 -- 172.21.15.29:0/3935522008 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c140ab600 con 0x7f7c1c0b4b10 2024-04-25T00:39:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c1bfff640 1 -- 172.21.15.29:0/3935522008 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c140ab9d0 con 0x7f7c1c0b4b10 2024-04-25T00:39:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 -- 172.21.15.29:0/3935522008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1c0b4b10 msgr2=0x7f7c1c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 --2- 172.21.15.29:0/3935522008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1c0b4b10 0x7f7c1c0ce240 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f7c140097c0 tx=0x7f7c1401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 -- 172.21.15.29:0/3935522008 shutdown_connections 2024-04-25T00:39:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 --2- 172.21.15.29:0/3935522008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1c0b4b10 0x7f7c1c0ce240 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 --2- 172.21.15.29:0/3935522008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c1c0b3c30 0x7f7c1c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 -- 172.21.15.29:0/3935522008 >> 172.21.15.29:0/3935522008 conn(0x7f7c1c0b1870 msgr2=0x7f7c1c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 -- 172.21.15.29:0/3935522008 shutdown_connections 2024-04-25T00:39:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 -- 172.21.15.29:0/3935522008 wait complete. 2024-04-25T00:39:31.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 Processor -- start 2024-04-25T00:39:31.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 -- start start 2024-04-25T00:39:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1c0b3140 0x7f7c1c0ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c1c0b3c30 0x7f7c1c0cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c1c0b4b10 0x7f7c1c155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c168d80 con 0x7f7c1c0b3140 2024-04-25T00:39:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c168c00 con 0x7f7c1c0b4b10 2024-04-25T00:39:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c23800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1c168f00 con 0x7f7c1c0b3c30 2024-04-25T00:39:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c2159c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1c0b3140 0x7f7c1c0ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c2159c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1c0b3140 0x7f7c1c0ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43628/0 (socket says 172.21.15.29:43628) 2024-04-25T00:39:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c2159c640 1 -- 172.21.15.29:0/2960630962 learned_addr learned my addr 172.21.15.29:0/2960630962 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c20d9b640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c1c0b3c30 0x7f7c1c0cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c21d9d640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c1c0b4b10 0x7f7c1c155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c2159c640 1 -- 172.21.15.29:0/2960630962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c1c0b3c30 msgr2=0x7f7c1c0cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c2159c640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c1c0b3c30 0x7f7c1c0cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c2159c640 1 -- 172.21.15.29:0/2960630962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c1c0b4b10 msgr2=0x7f7c1c155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c2159c640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c1c0b4b10 0x7f7c1c155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.786+0000 7f7c2159c640 1 -- 172.21.15.29:0/2960630962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1c15bb10 con 0x7f7c1c0b3140 2024-04-25T00:39:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.790+0000 7f7c2159c640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1c0b3140 0x7f7c1c0ca820 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00c480 tx=0x7f7c0c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.790+0000 7f7c19ffb640 1 -- 172.21.15.29:0/2960630962 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c017070 con 0x7f7c1c0b3140 2024-04-25T00:39:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.790+0000 7f7c19ffb640 1 -- 172.21.15.29:0/2960630962 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c005cf0 con 0x7f7c1c0b3140 2024-04-25T00:39:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.790+0000 7f7c19ffb640 1 -- 172.21.15.29:0/2960630962 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c007290 con 0x7f7c1c0b3140 2024-04-25T00:39:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.790+0000 7f7c23800640 1 -- 172.21.15.29:0/2960630962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1c0c9d40 con 0x7f7c1c0b3140 2024-04-25T00:39:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.790+0000 7f7c23800640 1 -- 172.21.15.29:0/2960630962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c1c0ca300 con 0x7f7c1c0b3140 2024-04-25T00:39:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.790+0000 7f7c23800640 1 -- 172.21.15.29:0/2960630962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c1c069270 con 0x7f7c1c0b3140 2024-04-25T00:39:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.790+0000 7f7c19ffb640 1 -- 172.21.15.29:0/2960630962 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f7c0c01a070 con 0x7f7c1c0b3140 2024-04-25T00:39:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.790+0000 7f7c19ffb640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c08067ef0 0x7f7c0806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.790+0000 7f7c19ffb640 1 -- 172.21.15.29:0/2960630962 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f7c0c085ba0 con 0x7f7c1c0b3140 2024-04-25T00:39:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.790+0000 7f7c20d9b640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c08067ef0 0x7f7c0806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.790+0000 7f7c20d9b640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c08067ef0 0x7f7c0806a3b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c154ce0 tx=0x7f7c1000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:31.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.794+0000 7f7c19ffb640 1 -- 172.21.15.29:0/2960630962 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7c0c055e10 con 0x7f7c1c0b3140 2024-04-25T00:39:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.958+0000 7f7c23800640 1 -- 172.21.15.29:0/2960630962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c1c0cf480 con 0x7f7c1c0b3140 2024-04-25T00:39:31.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.958+0000 7f7c19ffb640 1 -- 172.21.15.29:0/2960630962 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16680 (secure 0 0 0) 0x7f7c0c055710 con 0x7f7c1c0b3140 2024-04-25T00:39:31.963 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:31.963 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":134,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:24.461420+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:06.791988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:07.503204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.962+0000 7f7c23800640 1 -- 172.21.15.29:0/2960630962 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c08067ef0 msgr2=0x7f7c0806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.962+0000 7f7c23800640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c08067ef0 0x7f7c0806a3b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c154ce0 tx=0x7f7c1000b040 comp rx=0 tx=0).stop 2024-04-25T00:39:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.962+0000 7f7c23800640 1 -- 172.21.15.29:0/2960630962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1c0b3140 msgr2=0x7f7c1c0ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.962+0000 7f7c23800640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1c0b3140 0x7f7c1c0ca820 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00c480 tx=0x7f7c0c00c940 comp rx=0 tx=0).stop 2024-04-25T00:39:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.962+0000 7f7c23800640 1 -- 172.21.15.29:0/2960630962 shutdown_connections 2024-04-25T00:39:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.962+0000 7f7c23800640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c08067ef0 0x7f7c0806a3b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.962+0000 7f7c23800640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c1c0b4b10 0x7f7c1c155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.962+0000 7f7c23800640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c1c0b3c30 0x7f7c1c0cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.962+0000 7f7c23800640 1 --2- 172.21.15.29:0/2960630962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1c0b3140 0x7f7c1c0ca820 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.962+0000 7f7c23800640 1 -- 172.21.15.29:0/2960630962 >> 172.21.15.29:0/2960630962 conn(0x7f7c1c0b1870 msgr2=0x7f7c1c1409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.962+0000 7f7c23800640 1 -- 172.21.15.29:0/2960630962 shutdown_connections 2024-04-25T00:39:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:31.962+0000 7f7c23800640 1 -- 172.21.15.29:0/2960630962 wait complete. 2024-04-25T00:39:31.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:39:06.791988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:39:07.503204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:39:31.982 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:39:32.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3e054d2640 1 Processor -- start 2024-04-25T00:39:32.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3e054d2640 1 -- start start 2024-04-25T00:39:32.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3e054d2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e0015e200 0x7f3e000d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3e054d2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3e000d2c60 0x7f3e000b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3e054d2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e000cfc10 con 0x7f3e000cf690 2024-04-25T00:39:32.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3e054d2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3e000b38d0 con 0x7f3e000d2c60 2024-04-25T00:39:32.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3e054d2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3e000cfef0 con 0x7f3e0015e200 2024-04-25T00:39:32.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3dff7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3e000d2c60 0x7f3e000b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3dfeffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e0015e200 0x7f3e000d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3df7fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3e000cf690 0x7f3e000d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:39890/0 (socket says 172.21.15.29:39890) 2024-04-25T00:39:32.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3df7fff640 1 -- 172.21.15.29:0/743062943 learned_addr learned my addr 172.21.15.29:0/743062943 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:32.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3dfeffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e0015e200 0x7f3e000d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:60688/0 (socket says 172.21.15.29:60688) 2024-04-25T00:39:32.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3dfeffd640 1 -- 172.21.15.29:0/743062943 >> v2:172.21.15.157:3300/0 conn(0x7f3e000d2c60 msgr2=0x7f3e000b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:32.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3dfeffd640 1 --2- 172.21.15.29:0/743062943 >> v2:172.21.15.157:3300/0 conn(0x7f3e000d2c60 0x7f3e000b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3dfeffd640 1 -- 172.21.15.29:0/743062943 >> v1:172.21.15.157:6789/0 conn(0x7f3e000cf690 legacy=0x7f3e000d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3dfeffd640 1 -- 172.21.15.29:0/743062943 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e000b3b20 con 0x7f3e0015e200 2024-04-25T00:39:32.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.110+0000 7f3dff7fe640 1 --2- 172.21.15.29:0/743062943 >> v2:172.21.15.157:3300/0 conn(0x7f3e000d2c60 0x7f3e000b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:39:32.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3dfeffd640 1 --2- 172.21.15.29:0/743062943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e0015e200 0x7f3e000d47f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3df00097c0 tx=0x7f3df001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c05c3150c7e8177 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:32.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3dfe7fc640 1 -- 172.21.15.29:0/743062943 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df00aa5f0 con 0x7f3e0015e200 2024-04-25T00:39:32.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3dfe7fc640 1 -- 172.21.15.29:0/743062943 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df00af070 con 0x7f3e0015e200 2024-04-25T00:39:32.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3dfe7fc640 1 -- 172.21.15.29:0/743062943 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df00aa9b0 con 0x7f3e0015e200 2024-04-25T00:39:32.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3e054d2640 1 -- 172.21.15.29:0/743062943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e0015e200 msgr2=0x7f3e000d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3e054d2640 1 --2- 172.21.15.29:0/743062943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e0015e200 0x7f3e000d47f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3df00097c0 tx=0x7f3df001c470 comp rx=0 tx=0).stop 2024-04-25T00:39:32.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3e054d2640 1 -- 172.21.15.29:0/743062943 shutdown_connections 2024-04-25T00:39:32.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3e054d2640 1 --2- 172.21.15.29:0/743062943 >> v2:172.21.15.157:3300/0 conn(0x7f3e000d2c60 0x7f3e000b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3e054d2640 1 --2- 172.21.15.29:0/743062943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e0015e200 0x7f3e000d47f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3e054d2640 1 -- 172.21.15.29:0/743062943 >> 172.21.15.29:0/743062943 conn(0x7f3e0009d280 msgr2=0x7f3e000d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:32.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3e054d2640 1 -- 172.21.15.29:0/743062943 shutdown_connections 2024-04-25T00:39:32.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3e054d2640 1 -- 172.21.15.29:0/743062943 wait complete. 2024-04-25T00:39:32.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3e054d2640 1 Processor -- start 2024-04-25T00:39:32.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.114+0000 7f3e054d2640 1 -- start start 2024-04-25T00:39:32.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3e054d2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e000cf690 0x7f3e00158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3e054d2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e000d2c60 0x7f3e001611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3e054d2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e0015e200 0x7f3e0006c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3e054d2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3e000b51f0 con 0x7f3e0015e200 2024-04-25T00:39:32.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3e054d2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3e000b5070 con 0x7f3e000d2c60 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3e054d2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3e000b5370 con 0x7f3e000cf690 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3dff7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e0015e200 0x7f3e0006c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3dff7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e0015e200 0x7f3e0006c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43634/0 (socket says 172.21.15.29:43634) 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3dff7fe640 1 -- 172.21.15.29:0/1457449914 learned_addr learned my addr 172.21.15.29:0/1457449914 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3dfeffd640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e000cf690 0x7f3e00158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3df7fff640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e000d2c60 0x7f3e001611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3dff7fe640 1 -- 172.21.15.29:0/1457449914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e000cf690 msgr2=0x7f3e00158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3dff7fe640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e000cf690 0x7f3e00158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3dff7fe640 1 -- 172.21.15.29:0/1457449914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e000d2c60 msgr2=0x7f3e001611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3dff7fe640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e000d2c60 0x7f3e001611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3dff7fe640 1 -- 172.21.15.29:0/1457449914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e00150400 con 0x7f3e0015e200 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3df7fff640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e000d2c60 0x7f3e001611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3dff7fe640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e0015e200 0x7f3e0006c4d0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009eb0 tx=0x7f3df800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:32.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3df77fe640 1 -- 172.21.15.29:0/1457449914 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8018020 con 0x7f3e0015e200 2024-04-25T00:39:32.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3df77fe640 1 -- 172.21.15.29:0/1457449914 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df8014480 con 0x7f3e0015e200 2024-04-25T00:39:32.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3e054d2640 1 -- 172.21.15.29:0/1457449914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e001506e0 con 0x7f3e0015e200 2024-04-25T00:39:32.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3df77fe640 1 -- 172.21.15.29:0/1457449914 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8014850 con 0x7f3e0015e200 2024-04-25T00:39:32.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3e054d2640 1 -- 172.21.15.29:0/1457449914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0014fc90 con 0x7f3e0015e200 2024-04-25T00:39:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3e054d2640 1 -- 172.21.15.29:0/1457449914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e0014ffb0 con 0x7f3e0015e200 2024-04-25T00:39:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3df77fe640 1 -- 172.21.15.29:0/1457449914 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f3df801b020 con 0x7f3e0015e200 2024-04-25T00:39:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3df77fe640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3de0067f20 0x7f3de006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.118+0000 7f3df77fe640 1 -- 172.21.15.29:0/1457449914 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f3df8087460 con 0x7f3e0015e200 2024-04-25T00:39:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.122+0000 7f3dfeffd640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3de0067f20 0x7f3de006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.122+0000 7f3dfeffd640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3de0067f20 0x7f3de006a3e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3df00ae000 tx=0x7f3df00b5040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.126+0000 7f3df77fe640 1 -- 172.21.15.29:0/1457449914 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3e0014ffb0 con 0x7f3e0015e200 2024-04-25T00:39:32.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3e054d2640 1 -- 172.21.15.29:0/1457449914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e00069270 con 0x7f3e0015e200 2024-04-25T00:39:32.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3df77fe640 1 -- 172.21.15.29:0/1457449914 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7f3df8022020 con 0x7f3e0015e200 2024-04-25T00:39:32.293 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:39:32.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3e054d2640 1 -- 172.21.15.29:0/1457449914 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3de0067f20 msgr2=0x7f3de006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3e054d2640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3de0067f20 0x7f3de006a3e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3df00ae000 tx=0x7f3df00b5040 comp rx=0 tx=0).stop 2024-04-25T00:39:32.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3e054d2640 1 -- 172.21.15.29:0/1457449914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e0015e200 msgr2=0x7f3e0006c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3e054d2640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e0015e200 0x7f3e0006c4d0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009eb0 tx=0x7f3df800c490 comp rx=0 tx=0).stop 2024-04-25T00:39:32.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3e054d2640 1 -- 172.21.15.29:0/1457449914 shutdown_connections 2024-04-25T00:39:32.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3e054d2640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3de0067f20 0x7f3de006a3e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3e054d2640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e0015e200 0x7f3e0006c4d0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3e054d2640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e000d2c60 0x7f3e001611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3e054d2640 1 --2- 172.21.15.29:0/1457449914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e000cf690 0x7f3e00158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3e054d2640 1 -- 172.21.15.29:0/1457449914 >> 172.21.15.29:0/1457449914 conn(0x7f3e0009d280 msgr2=0x7f3e0006d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:32.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3e054d2640 1 -- 172.21.15.29:0/1457449914 shutdown_connections 2024-04-25T00:39:32.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.290+0000 7f3e054d2640 1 -- 172.21.15.29:0/1457449914 wait complete. 2024-04-25T00:39:32.312 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:39:32.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.430+0000 7fd1903ed640 1 Processor -- start 2024-04-25T00:39:32.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.430+0000 7fd1903ed640 1 -- start start 2024-04-25T00:39:32.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd1903ed640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd18806dce0 0x7fd188150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd1903ed640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1880b3130 con 0x7fd18815d980 2024-04-25T00:39:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd1903ed640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1880b3410 con 0x7fd1880b2b70 2024-04-25T00:39:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd1903ed640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd1880cf560 con 0x7fd18806dce0 2024-04-25T00:39:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18e189640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd18815d980 0x7fd18806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42166/0 (socket says 172.21.15.29:42166) 2024-04-25T00:39:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18e189640 1 -- 172.21.15.29:0/2612925583 learned_addr learned my addr 172.21.15.29:0/2612925583 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d988640 1 --2- 172.21.15.29:0/2612925583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd18806dce0 0x7fd188150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737037722 0 0) 0x7fd1880b3130 con 0x7fd18815d980 2024-04-25T00:39:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd174003530 con 0x7fd18815d980 2024-04-25T00:39:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3391927909 0 0) 0x7fd174003530 con 0x7fd18815d980 2024-04-25T00:39:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd18806dce0 msgr2=0x7fd188150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 --2- 172.21.15.29:0/2612925583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd18806dce0 0x7fd188150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 >> v1:172.21.15.157:6789/0 conn(0x7fd1880b2b70 legacy=0x7fd188151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1880b36f0 con 0x7fd18815d980 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd184001680 con 0x7fd18815d980 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 >> v1:172.21.15.29:6789/0 conn(0x7fd18815d980 legacy=0x7fd18806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 --2- 172.21.15.29:0/2612925583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1740039f0 0x7fd174003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 --2- 172.21.15.29:0/2612925583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd18806dce0 0x7fd174004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 --2- 172.21.15.29:0/2612925583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd174004d50 0x7fd1740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd1880cf560 con 0x7fd1740039f0 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd17400bb60 con 0x7fd174004d50 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd17400bd60 con 0x7fd18806dce0 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18e189640 1 --2- 172.21.15.29:0/2612925583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1740039f0 0x7fd174003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18d988640 1 --2- 172.21.15.29:0/2612925583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd18806dce0 0x7fd174004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18e189640 1 -- 172.21.15.29:0/2612925583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd18806dce0 msgr2=0x7fd174004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18e189640 1 --2- 172.21.15.29:0/2612925583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd18806dce0 0x7fd174004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18e189640 1 -- 172.21.15.29:0/2612925583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd174004d50 msgr2=0x7fd1740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:32.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18e189640 1 --2- 172.21.15.29:0/2612925583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd174004d50 0x7fd1740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.434+0000 7fd18e189640 1 -- 172.21.15.29:0/2612925583 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd174003530 con 0x7fd1740039f0 2024-04-25T00:39:32.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18e189640 1 --2- 172.21.15.29:0/2612925583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1740039f0 0x7fd174003e10 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c110 tx=0x7fd18401cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf33e335724772fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:32.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184007c30 con 0x7fd1740039f0 2024-04-25T00:39:32.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1840a9020 con 0x7fd1740039f0 2024-04-25T00:39:32.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18d187640 1 -- 172.21.15.29:0/2612925583 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184005ae0 con 0x7fd1740039f0 2024-04-25T00:39:32.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 -- 172.21.15.29:0/2612925583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1740039f0 msgr2=0x7fd174003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 --2- 172.21.15.29:0/2612925583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1740039f0 0x7fd174003e10 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c110 tx=0x7fd18401cba0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18e98a640 1 -- 172.21.15.29:0/2612925583 reap_dead start 2024-04-25T00:39:32.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 -- 172.21.15.29:0/2612925583 shutdown_connections 2024-04-25T00:39:32.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 -- 172.21.15.29:0/2612925583 >> 172.21.15.29:0/2612925583 conn(0x7fd1880b1870 msgr2=0x7fd1880b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:32.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 -- 172.21.15.29:0/2612925583 shutdown_connections 2024-04-25T00:39:32.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 -- 172.21.15.29:0/2612925583 wait complete. 2024-04-25T00:39:32.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 Processor -- start 2024-04-25T00:39:32.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 -- start start 2024-04-25T00:39:32.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1740039f0 0x7fd188161bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1881472a0 0x7fd188154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1881466e0 0x7fd188146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd18815c640 con 0x7fd1881466e0 2024-04-25T00:39:32.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd18815c4c0 con 0x7fd1740039f0 2024-04-25T00:39:32.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd1903ed640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd18815c7c0 con 0x7fd1881472a0 2024-04-25T00:39:32.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18e98a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1881466e0 0x7fd188146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18e98a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1881466e0 0x7fd188146b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43660/0 (socket says 172.21.15.29:43660) 2024-04-25T00:39:32.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18e98a640 1 -- 172.21.15.29:0/3627359755 learned_addr learned my addr 172.21.15.29:0/3627359755 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:32.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18e189640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1740039f0 0x7fd188161bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18d988640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1881472a0 0x7fd188154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18e98a640 1 -- 172.21.15.29:0/3627359755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1881472a0 msgr2=0x7fd188154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18e98a640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1881472a0 0x7fd188154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18e98a640 1 -- 172.21.15.29:0/3627359755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1740039f0 msgr2=0x7fd188161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18e98a640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1740039f0 0x7fd188161bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18e98a640 1 -- 172.21.15.29:0/3627359755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18815f370 con 0x7fd1881466e0 2024-04-25T00:39:32.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.438+0000 7fd18e98a640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1881466e0 0x7fd188146b90 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fd18806e1a0 tx=0x7fd18000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:32.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.442+0000 7fd17effd640 1 -- 172.21.15.29:0/3627359755 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180017070 con 0x7fd1881466e0 2024-04-25T00:39:32.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.442+0000 7fd17effd640 1 -- 172.21.15.29:0/3627359755 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd180005d50 con 0x7fd1881466e0 2024-04-25T00:39:32.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.442+0000 7fd1903ed640 1 -- 172.21.15.29:0/3627359755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd188162130 con 0x7fd1881466e0 2024-04-25T00:39:32.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.442+0000 7fd17effd640 1 -- 172.21.15.29:0/3627359755 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18000a880 con 0x7fd1881466e0 2024-04-25T00:39:32.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.442+0000 7fd1903ed640 1 -- 172.21.15.29:0/3627359755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18815f8d0 con 0x7fd1881466e0 2024-04-25T00:39:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.442+0000 7fd1903ed640 1 -- 172.21.15.29:0/3627359755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd188069270 con 0x7fd1881466e0 2024-04-25T00:39:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.442+0000 7fd17effd640 1 -- 172.21.15.29:0/3627359755 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fd18001a070 con 0x7fd1881466e0 2024-04-25T00:39:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.442+0000 7fd17effd640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd16c067f20 0x7fd16c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.442+0000 7fd17effd640 1 -- 172.21.15.29:0/3627359755 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7fd180085b60 con 0x7fd1881466e0 2024-04-25T00:39:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.442+0000 7fd18e189640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd16c067f20 0x7fd16c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.442+0000 7fd18e189640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd16c067f20 0x7fd16c06a3e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd1840aa000 tx=0x7fd1840b0040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:32.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.446+0000 7fd17effd640 1 -- 172.21.15.29:0/3627359755 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd18815f370 con 0x7fd1881466e0 2024-04-25T00:39:32.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.602+0000 7fd1903ed640 1 -- 172.21.15.29:0/3627359755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd188158330 con 0x7fd1881466e0 2024-04-25T00:39:32.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.602+0000 7fd17effd640 1 -- 172.21.15.29:0/3627359755 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) ==== 104+0+11 (secure 0 0 0) 0x7fd180055d40 con 0x7fd1881466e0 2024-04-25T00:39:32.606 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:39:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.606+0000 7fd1903ed640 1 -- 172.21.15.29:0/3627359755 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd16c067f20 msgr2=0x7fd16c06a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.606+0000 7fd1903ed640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd16c067f20 0x7fd16c06a3e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd1840aa000 tx=0x7fd1840b0040 comp rx=0 tx=0).stop 2024-04-25T00:39:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.606+0000 7fd1903ed640 1 -- 172.21.15.29:0/3627359755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1881466e0 msgr2=0x7fd188146b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.606+0000 7fd1903ed640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1881466e0 0x7fd188146b90 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fd18806e1a0 tx=0x7fd18000cb90 comp rx=0 tx=0).stop 2024-04-25T00:39:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.606+0000 7fd1903ed640 1 -- 172.21.15.29:0/3627359755 shutdown_connections 2024-04-25T00:39:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.606+0000 7fd1903ed640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd16c067f20 0x7fd16c06a3e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.606+0000 7fd1903ed640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1881466e0 0x7fd188146b90 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.606+0000 7fd1903ed640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1881472a0 0x7fd188154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.606+0000 7fd1903ed640 1 --2- 172.21.15.29:0/3627359755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1740039f0 0x7fd188161bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.606+0000 7fd1903ed640 1 -- 172.21.15.29:0/3627359755 >> 172.21.15.29:0/3627359755 conn(0x7fd1880b1870 msgr2=0x7fd188158c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:32.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.606+0000 7fd1903ed640 1 -- 172.21.15.29:0/3627359755 shutdown_connections 2024-04-25T00:39:32.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.606+0000 7fd1903ed640 1 -- 172.21.15.29:0/3627359755 wait complete. 2024-04-25T00:39:32.623 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:39:32.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.742+0000 7f44a0897640 1 Processor -- start 2024-04-25T00:39:32.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f44a0897640 1 -- start start 2024-04-25T00:39:32.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f44a0897640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0b4780 0x7f449c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f44a0897640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f449c0cc240 0x7f449c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f44a0897640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f449c0b3720 con 0x7f449c15e8d0 2024-04-25T00:39:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f44a0897640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f449c0b42f0 con 0x7f449c0cc240 2024-04-25T00:39:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f44a0897640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f449c0b39c0 con 0x7f449c0b4780 2024-04-25T00:39:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f449ad9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f449c0cc240 0x7f449c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f449ad9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f449c0cc240 0x7f449c15c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43668/0 (socket says 172.21.15.29:43668) 2024-04-25T00:39:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f449ad9d640 1 -- 172.21.15.29:0/2294745677 learned_addr learned my addr 172.21.15.29:0/2294745677 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f4499d9b640 1 --2- 172.21.15.29:0/2294745677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0b4780 0x7f449c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f449ad9d640 1 -- 172.21.15.29:0/2294745677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0b4780 msgr2=0x7f449c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f449ad9d640 1 --2- 172.21.15.29:0/2294745677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0b4780 0x7f449c15f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f449ad9d640 1 -- 172.21.15.29:0/2294745677 >> v1:172.21.15.157:6789/0 conn(0x7f449c15e8d0 legacy=0x7f449c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.746+0000 7f449ad9d640 1 -- 172.21.15.29:0/2294745677 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449c0b3ca0 con 0x7f449c0cc240 2024-04-25T00:39:32.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449ad9d640 1 --2- 172.21.15.29:0/2294745677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449c0cc240 0x7f449c15c7d0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f4494009e50 tx=0x7f449401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce0e517ee3de036 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:32.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449959a640 1 -- 172.21.15.29:0/2294745677 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940a5740 con 0x7f449c0cc240 2024-04-25T00:39:32.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449959a640 1 -- 172.21.15.29:0/2294745677 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44940a58e0 con 0x7f449c0cc240 2024-04-25T00:39:32.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449959a640 1 -- 172.21.15.29:0/2294745677 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940a5cb0 con 0x7f449c0cc240 2024-04-25T00:39:32.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 -- 172.21.15.29:0/2294745677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449c0cc240 msgr2=0x7f449c15c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 --2- 172.21.15.29:0/2294745677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449c0cc240 0x7f449c15c7d0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f4494009e50 tx=0x7f449401c9f0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 -- 172.21.15.29:0/2294745677 shutdown_connections 2024-04-25T00:39:32.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 --2- 172.21.15.29:0/2294745677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449c0cc240 0x7f449c15c7d0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 --2- 172.21.15.29:0/2294745677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0b4780 0x7f449c15f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 -- 172.21.15.29:0/2294745677 >> 172.21.15.29:0/2294745677 conn(0x7f449c0b1870 msgr2=0x7f449c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:32.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 -- 172.21.15.29:0/2294745677 shutdown_connections 2024-04-25T00:39:32.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 -- 172.21.15.29:0/2294745677 wait complete. 2024-04-25T00:39:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 Processor -- start 2024-04-25T00:39:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 -- start start 2024-04-25T00:39:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449c0b4780 0x7f449c159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0cc240 0x7f449c13b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449c15e8d0 0x7f449c072570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f449c15d6a0 con 0x7f449c15e8d0 2024-04-25T00:39:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f449c15d520 con 0x7f449c0b4780 2024-04-25T00:39:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f449c165c60 con 0x7f449c0cc240 2024-04-25T00:39:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449c15e8d0 0x7f449c072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449c15e8d0 0x7f449c072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43680/0 (socket says 172.21.15.29:43680) 2024-04-25T00:39:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449ad9d640 1 -- 172.21.15.29:0/645332315 learned_addr learned my addr 172.21.15.29:0/645332315 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f4499d9b640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0cc240 0x7f449c13b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449a59c640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449c0b4780 0x7f449c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449ad9d640 1 -- 172.21.15.29:0/645332315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0cc240 msgr2=0x7f449c13b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449ad9d640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0cc240 0x7f449c13b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449ad9d640 1 -- 172.21.15.29:0/645332315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449c0b4780 msgr2=0x7f449c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449ad9d640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449c0b4780 0x7f449c159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449ad9d640 1 -- 172.21.15.29:0/645332315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449c157ca0 con 0x7f449c15e8d0 2024-04-25T00:39:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f449ad9d640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449c15e8d0 0x7f449c072570 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f44940af040 tx=0x7f4494019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f4492ffd640 1 -- 172.21.15.29:0/645332315 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940bc020 con 0x7f449c15e8d0 2024-04-25T00:39:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f4492ffd640 1 -- 172.21.15.29:0/645332315 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44940b8670 con 0x7f449c15e8d0 2024-04-25T00:39:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 -- 172.21.15.29:0/645332315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f449c157120 con 0x7f449c15e8d0 2024-04-25T00:39:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f4492ffd640 1 -- 172.21.15.29:0/645332315 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940b8a60 con 0x7f449c15e8d0 2024-04-25T00:39:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.750+0000 7f44a0897640 1 -- 172.21.15.29:0/645332315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f449c1576e0 con 0x7f449c15e8d0 2024-04-25T00:39:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.754+0000 7f44a0897640 1 -- 172.21.15.29:0/645332315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f449c069270 con 0x7f449c15e8d0 2024-04-25T00:39:32.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.754+0000 7f4492ffd640 1 -- 172.21.15.29:0/645332315 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f44940bf020 con 0x7f449c15e8d0 2024-04-25T00:39:32.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.754+0000 7f4492ffd640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f447c067ef0 0x7f447c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:32.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.754+0000 7f4492ffd640 1 -- 172.21.15.29:0/645332315 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f4494121830 con 0x7f449c15e8d0 2024-04-25T00:39:32.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.754+0000 7f449a59c640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f447c067ef0 0x7f447c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:32.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.754+0000 7f449a59c640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f447c067ef0 0x7f447c06a3b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f449c1f8810 tx=0x7f448c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:32.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.758+0000 7f4492ffd640 1 -- 172.21.15.29:0/645332315 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f44940c6030 con 0x7f449c15e8d0 2024-04-25T00:39:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.914+0000 7f44a0897640 1 -- 172.21.15.29:0/645332315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f449c070d70 con 0x7f449c15e8d0 2024-04-25T00:39:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.914+0000 7f4492ffd640 1 -- 172.21.15.29:0/645332315 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) ==== 100+0+11 (secure 0 0 0) 0x7f44940165e0 con 0x7f449c15e8d0 2024-04-25T00:39:32.918 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:39:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.918+0000 7f44a0897640 1 -- 172.21.15.29:0/645332315 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f447c067ef0 msgr2=0x7f447c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.918+0000 7f44a0897640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f447c067ef0 0x7f447c06a3b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f449c1f8810 tx=0x7f448c008040 comp rx=0 tx=0).stop 2024-04-25T00:39:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.918+0000 7f44a0897640 1 -- 172.21.15.29:0/645332315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449c15e8d0 msgr2=0x7f449c072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.918+0000 7f44a0897640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449c15e8d0 0x7f449c072570 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f44940af040 tx=0x7f4494019790 comp rx=0 tx=0).stop 2024-04-25T00:39:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.918+0000 7f44a0897640 1 -- 172.21.15.29:0/645332315 shutdown_connections 2024-04-25T00:39:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.918+0000 7f44a0897640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f447c067ef0 0x7f447c06a3b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.918+0000 7f44a0897640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449c15e8d0 0x7f449c072570 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.918+0000 7f44a0897640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449c0cc240 0x7f449c13b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.918+0000 7f44a0897640 1 --2- 172.21.15.29:0/645332315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449c0b4780 0x7f449c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.918+0000 7f44a0897640 1 -- 172.21.15.29:0/645332315 >> 172.21.15.29:0/645332315 conn(0x7f449c0b1870 msgr2=0x7f449c15f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.918+0000 7f44a0897640 1 -- 172.21.15.29:0/645332315 shutdown_connections 2024-04-25T00:39:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:32.918+0000 7f44a0897640 1 -- 172.21.15.29:0/645332315 wait complete. 2024-04-25T00:39:32.936 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:39:33.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1f756640 1 Processor -- start 2024-04-25T00:39:33.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1f756640 1 -- start start 2024-04-25T00:39:33.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1f756640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa180cf690 0x7ffa180d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1f756640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa180d2a90 0x7ffa180b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1f756640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa180cfc10 con 0x7ffa1815e200 2024-04-25T00:39:33.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1f756640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ffa180d2ee0 con 0x7ffa180d2a90 2024-04-25T00:39:33.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1f756640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffa180b38d0 con 0x7ffa180cf690 2024-04-25T00:39:33.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1dcf3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa180d2a90 0x7ffa180b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1dcf3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa180d2a90 0x7ffa180b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43686/0 (socket says 172.21.15.29:43686) 2024-04-25T00:39:33.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1dcf3640 1 -- 172.21.15.29:0/1249565072 learned_addr learned my addr 172.21.15.29:0/1249565072 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:33.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1ccf1640 1 --2- 172.21.15.29:0/1249565072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa180cf690 0x7ffa180d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1dcf3640 1 -- 172.21.15.29:0/1249565072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa180cf690 msgr2=0x7ffa180d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:33.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1dcf3640 1 --2- 172.21.15.29:0/1249565072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa180cf690 0x7ffa180d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1dcf3640 1 -- 172.21.15.29:0/1249565072 >> v1:172.21.15.157:6789/0 conn(0x7ffa1815e200 legacy=0x7ffa180d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:33.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.046+0000 7ffa1dcf3640 1 -- 172.21.15.29:0/1249565072 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa180cfef0 con 0x7ffa180d2a90 2024-04-25T00:39:33.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.050+0000 7ffa1dcf3640 1 --2- 172.21.15.29:0/1249565072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa180d2a90 0x7ffa180b4320 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009e50 tx=0x7ffa1001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ea333903e9a2a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:33.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.050+0000 7ffa0ffff640 1 -- 172.21.15.29:0/1249565072 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa100a5760 con 0x7ffa180d2a90 2024-04-25T00:39:33.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.050+0000 7ffa0ffff640 1 -- 172.21.15.29:0/1249565072 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa100a5900 con 0x7ffa180d2a90 2024-04-25T00:39:33.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.050+0000 7ffa0ffff640 1 -- 172.21.15.29:0/1249565072 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa100a5cd0 con 0x7ffa180d2a90 2024-04-25T00:39:33.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.050+0000 7ffa1f756640 1 -- 172.21.15.29:0/1249565072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa180d2a90 msgr2=0x7ffa180b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.050+0000 7ffa1f756640 1 --2- 172.21.15.29:0/1249565072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa180d2a90 0x7ffa180b4320 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009e50 tx=0x7ffa1001c9f0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.050+0000 7ffa1f756640 1 -- 172.21.15.29:0/1249565072 shutdown_connections 2024-04-25T00:39:33.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.050+0000 7ffa1f756640 1 --2- 172.21.15.29:0/1249565072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa180d2a90 0x7ffa180b4320 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.050+0000 7ffa1f756640 1 --2- 172.21.15.29:0/1249565072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa180cf690 0x7ffa180d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.050+0000 7ffa1f756640 1 -- 172.21.15.29:0/1249565072 >> 172.21.15.29:0/1249565072 conn(0x7ffa1809d280 msgr2=0x7ffa180d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:33.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.050+0000 7ffa1f756640 1 -- 172.21.15.29:0/1249565072 shutdown_connections 2024-04-25T00:39:33.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.050+0000 7ffa1f756640 1 -- 172.21.15.29:0/1249565072 wait complete. 2024-04-25T00:39:33.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1f756640 1 Processor -- start 2024-04-25T00:39:33.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1f756640 1 -- start start 2024-04-25T00:39:33.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1f756640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa180cf690 0x7ffa18150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1f756640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa180d2a90 0x7ffa18158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1f756640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa1815e200 0x7ffa1815e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1f756640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffa180b51f0 con 0x7ffa180cf690 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1f756640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa180b5070 con 0x7ffa1815e200 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1f756640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffa180b5370 con 0x7ffa180d2a90 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1d4f2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa180cf690 0x7ffa18150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1d4f2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa180cf690 0x7ffa18150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43696/0 (socket says 172.21.15.29:43696) 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1d4f2640 1 -- 172.21.15.29:0/3361679340 learned_addr learned my addr 172.21.15.29:0/3361679340 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1dcf3640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa1815e200 0x7ffa1815e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1ccf1640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa180d2a90 0x7ffa18158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1d4f2640 1 -- 172.21.15.29:0/3361679340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa180d2a90 msgr2=0x7ffa18158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1d4f2640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa180d2a90 0x7ffa18158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1d4f2640 1 -- 172.21.15.29:0/3361679340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa1815e200 msgr2=0x7ffa1815e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1d4f2640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa1815e200 0x7ffa1815e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1d4f2640 1 -- 172.21.15.29:0/3361679340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa180b1990 con 0x7ffa180cf690 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1d4f2640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa180cf690 0x7ffa18150e60 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400b440 tx=0x7ffa1400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa0dffb640 1 -- 172.21.15.29:0/3361679340 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14015020 con 0x7ffa180cf690 2024-04-25T00:39:33.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa0dffb640 1 -- 172.21.15.29:0/3361679340 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa14011890 con 0x7ffa180cf690 2024-04-25T00:39:33.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1f756640 1 -- 172.21.15.29:0/3361679340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa180b1c70 con 0x7ffa180cf690 2024-04-25T00:39:33.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa0dffb640 1 -- 172.21.15.29:0/3361679340 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14011c80 con 0x7ffa180cf690 2024-04-25T00:39:33.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1f756640 1 -- 172.21.15.29:0/3361679340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa18158d60 con 0x7ffa180cf690 2024-04-25T00:39:33.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.054+0000 7ffa1f756640 1 -- 172.21.15.29:0/3361679340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa18069270 con 0x7ffa180cf690 2024-04-25T00:39:33.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.058+0000 7ffa0dffb640 1 -- 172.21.15.29:0/3361679340 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7ffa14018020 con 0x7ffa180cf690 2024-04-25T00:39:33.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.058+0000 7ffa0dffb640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff9fc067ef0 0x7ff9fc06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.058+0000 7ffa0dffb640 1 -- 172.21.15.29:0/3361679340 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7ffa14086b00 con 0x7ffa180cf690 2024-04-25T00:39:33.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.058+0000 7ffa1ccf1640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff9fc067ef0 0x7ff9fc06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.058+0000 7ffa1ccf1640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff9fc067ef0 0x7ff9fc06a3b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffa18161860 tx=0x7ffa08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:33.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.062+0000 7ffa0dffb640 1 -- 172.21.15.29:0/3361679340 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffa1401f030 con 0x7ffa180cf690 2024-04-25T00:39:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa1f756640 1 -- 172.21.15.29:0/3361679340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffa18153990 con 0x7ffa180cf690 2024-04-25T00:39:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa0dffb640 1 -- 172.21.15.29:0/3361679340 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v134) ==== 80+0+0 (secure 0 0 0) 0x7ffa1400d040 con 0x7ffa180cf690 2024-04-25T00:39:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:39:33.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa1f756640 1 -- 172.21.15.29:0/3361679340 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff9fc067ef0 msgr2=0x7ff9fc06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa1f756640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff9fc067ef0 0x7ff9fc06a3b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffa18161860 tx=0x7ffa08008040 comp rx=0 tx=0).stop 2024-04-25T00:39:33.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa1f756640 1 -- 172.21.15.29:0/3361679340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa180cf690 msgr2=0x7ffa18150e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa1f756640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa180cf690 0x7ffa18150e60 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400b440 tx=0x7ffa1400b900 comp rx=0 tx=0).stop 2024-04-25T00:39:33.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa1f756640 1 -- 172.21.15.29:0/3361679340 shutdown_connections 2024-04-25T00:39:33.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa1f756640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff9fc067ef0 0x7ff9fc06a3b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa1f756640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa1815e200 0x7ffa1815e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa1f756640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa180d2a90 0x7ffa18158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa1f756640 1 --2- 172.21.15.29:0/3361679340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa180cf690 0x7ffa18150e60 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa1f756640 1 -- 172.21.15.29:0/3361679340 >> 172.21.15.29:0/3361679340 conn(0x7ffa1809d280 msgr2=0x7ffa1806cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:33.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa1f756640 1 -- 172.21.15.29:0/3361679340 shutdown_connections 2024-04-25T00:39:33.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.234+0000 7ffa1f756640 1 -- 172.21.15.29:0/3361679340 wait complete. 2024-04-25T00:39:33.255 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-25T00:39:33.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:33.255 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:33.269 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[34907]: fuse finished with error 0 and tester_r 0 2024-04-25T00:39:33.271 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:39:33.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:33.271 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:39:33.363 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:39:33.363 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:39:33.363 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:39:33.369 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:39:33.370 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:39:33.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:33.370 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:39:33.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:39:33.416 INFO:teuthology.orchestra.run.smithi029.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T00:39:33.417 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:39:33.417 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:39:33.417 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:39:33.423 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:39:33.423 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:39:33.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:33.423 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:39:33.470 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T00:39:33.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:39:33.471 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:39:33.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.594+0000 7f3722325640 1 Processor -- start 2024-04-25T00:39:33.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.594+0000 7f3722325640 1 -- start start 2024-04-25T00:39:33.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f3722325640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f371c0b3c30 0x7f371c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f3722325640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f371c0b41e0 con 0x7f371c0b4b10 2024-04-25T00:39:33.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f3722325640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f371c06d960 con 0x7f371c0b3140 2024-04-25T00:39:33.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f3722325640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f371c06dc40 con 0x7f371c0b3c30 2024-04-25T00:39:33.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f371b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f371c0b3c30 0x7f371c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f37208c2640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f371c0b4b10 0x7f371c0ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42178/0 (socket says 172.21.15.29:42178) 2024-04-25T00:39:33.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f37208c2640 1 -- 172.21.15.29:0/2893324076 learned_addr learned my addr 172.21.15.29:0/2893324076 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:33.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f371affd640 1 -- 172.21.15.29:0/2893324076 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2284159000 0 0) 0x7f371c0b41e0 con 0x7f371c0b4b10 2024-04-25T00:39:33.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f371affd640 1 -- 172.21.15.29:0/2893324076 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3704003530 con 0x7f371c0b4b10 2024-04-25T00:39:33.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f371b7fe640 1 -- 172.21.15.29:0/2893324076 >> v1:172.21.15.157:6789/0 conn(0x7f371c0b3140 legacy=0x7f371c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:33.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f371b7fe640 1 -- 172.21.15.29:0/2893324076 >> v1:172.21.15.29:6789/0 conn(0x7f371c0b4b10 legacy=0x7f371c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f371b7fe640 1 -- 172.21.15.29:0/2893324076 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371c06df20 con 0x7f371c0b3c30 2024-04-25T00:39:33.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f371b7fe640 1 --2- 172.21.15.29:0/2893324076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371c0b3c30 0x7f371c06e440 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f37100097f0 tx=0x7f371001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8680c8b0c818b86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:33.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f371affd640 1 -- 172.21.15.29:0/2893324076 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37100b0070 con 0x7f371c0b3c30 2024-04-25T00:39:33.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f371affd640 1 -- 172.21.15.29:0/2893324076 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37100ab600 con 0x7f371c0b3c30 2024-04-25T00:39:33.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f371affd640 1 -- 172.21.15.29:0/2893324076 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37100ab9d0 con 0x7f371c0b3c30 2024-04-25T00:39:33.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f3722325640 1 -- 172.21.15.29:0/2893324076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371c0b3c30 msgr2=0x7f371c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f3722325640 1 --2- 172.21.15.29:0/2893324076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371c0b3c30 0x7f371c06e440 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f37100097f0 tx=0x7f371001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f3722325640 1 -- 172.21.15.29:0/2893324076 shutdown_connections 2024-04-25T00:39:33.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.598+0000 7f3722325640 1 --2- 172.21.15.29:0/2893324076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371c0b3c30 0x7f371c06e440 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 -- 172.21.15.29:0/2893324076 >> 172.21.15.29:0/2893324076 conn(0x7f371c0b1870 msgr2=0x7f371c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:33.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 -- 172.21.15.29:0/2893324076 shutdown_connections 2024-04-25T00:39:33.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 -- 172.21.15.29:0/2893324076 wait complete. 2024-04-25T00:39:33.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 Processor -- start 2024-04-25T00:39:33.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 -- start start 2024-04-25T00:39:33.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371c0b3140 0x7f371c13a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f371c0b3c30 0x7f371c13eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f371c0b4b10 0x7f371c13be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f371c169d90 con 0x7f371c0b3140 2024-04-25T00:39:33.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f371c169c10 con 0x7f371c0b4b10 2024-04-25T00:39:33.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f371c169f10 con 0x7f371c0b3c30 2024-04-25T00:39:33.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f371bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371c0b3140 0x7f371c13a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f371bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371c0b3140 0x7f371c13a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43712/0 (socket says 172.21.15.29:43712) 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f371bfff640 1 -- 172.21.15.29:0/2896887285 learned_addr learned my addr 172.21.15.29:0/2896887285 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f371bfff640 1 -- 172.21.15.29:0/2896887285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f371c0b3c30 msgr2=0x7f371c13eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f371b7fe640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f371c0b3c30 0x7f371c13eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f37208c2640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f371c0b4b10 0x7f371c13be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f371bfff640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f371c0b3c30 0x7f371c13eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f371bfff640 1 -- 172.21.15.29:0/2896887285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f371c0b4b10 msgr2=0x7f371c13be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f371bfff640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f371c0b4b10 0x7f371c13be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f371bfff640 1 -- 172.21.15.29:0/2896887285 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371c154c00 con 0x7f371c0b3140 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f371bfff640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371c0b3140 0x7f371c13a5d0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f370c00c370 tx=0x7f370c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3718ff9640 1 -- 172.21.15.29:0/2896887285 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c0052d0 con 0x7f371c0b3140 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3718ff9640 1 -- 172.21.15.29:0/2896887285 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f370c005470 con 0x7f371c0b3140 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3718ff9640 1 -- 172.21.15.29:0/2896887285 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c007260 con 0x7f371c0b3140 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 -- 172.21.15.29:0/2896887285 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371c154ee0 con 0x7f371c0b3140 2024-04-25T00:39:33.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 -- 172.21.15.29:0/2896887285 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371c1554a0 con 0x7f371c0b3140 2024-04-25T00:39:33.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.602+0000 7f3722325640 1 -- 172.21.15.29:0/2896887285 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f371c1557d0 con 0x7f371c0b3140 2024-04-25T00:39:33.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.606+0000 7f3718ff9640 1 -- 172.21.15.29:0/2896887285 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f370c007400 con 0x7f371c0b3140 2024-04-25T00:39:33.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.606+0000 7f3718ff9640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3708067ef0 0x7f370806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.606+0000 7f3718ff9640 1 -- 172.21.15.29:0/2896887285 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f370c086150 con 0x7f371c0b3140 2024-04-25T00:39:33.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.606+0000 7f371b7fe640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3708067ef0 0x7f370806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.606+0000 7f371b7fe640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3708067ef0 0x7f370806a3b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f37100ad000 tx=0x7f3710008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:33.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.610+0000 7f3718ff9640 1 -- 172.21.15.29:0/2896887285 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f370c0563c0 con 0x7f371c0b3140 2024-04-25T00:39:33.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3722325640 1 -- 172.21.15.29:0/2896887285 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f371c069270 con 0x7f371c0b3140 2024-04-25T00:39:33.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3718ff9640 1 -- 172.21.15.29:0/2896887285 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5179 (secure 0 0 0) 0x7f370c055cc0 con 0x7f371c0b3140 2024-04-25T00:39:33.801 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:33.801 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":84,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7450,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/456996219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":456996219},{"type":"v1","addr":"172.21.15.157:6833","nonce":456996219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/87071339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":87071339},{"type":"v1","addr":"172.21.15.157:6835","nonce":87071339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3920208567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3920208567},{"type":"v1","addr":"172.21.15.29:6833","nonce":3920208567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:39:08.508825+0000","modified":"2024-04-25T00:39:11.535481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9021":{"gid":9021,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/619209926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":619209926},{"type":"v1","addr":"172.21.15.29:6835","nonce":619209926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9021,"qdb_cluster":[9021]},"id":10}]} 2024-04-25T00:39:33.801 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 84 2024-04-25T00:39:33.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3722325640 1 -- 172.21.15.29:0/2896887285 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3708067ef0 msgr2=0x7f370806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3722325640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3708067ef0 0x7f370806a3b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f37100ad000 tx=0x7f3710008040 comp rx=0 tx=0).stop 2024-04-25T00:39:33.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3722325640 1 -- 172.21.15.29:0/2896887285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371c0b3140 msgr2=0x7f371c13a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3722325640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371c0b3140 0x7f371c13a5d0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f370c00c370 tx=0x7f370c00c830 comp rx=0 tx=0).stop 2024-04-25T00:39:33.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3722325640 1 -- 172.21.15.29:0/2896887285 shutdown_connections 2024-04-25T00:39:33.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3722325640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3708067ef0 0x7f370806a3b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3722325640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f371c0b4b10 0x7f371c13be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3722325640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f371c0b3c30 0x7f371c13eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3722325640 1 --2- 172.21.15.29:0/2896887285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f371c0b3140 0x7f371c13a5d0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3722325640 1 -- 172.21.15.29:0/2896887285 >> 172.21.15.29:0/2896887285 conn(0x7f371c0b1870 msgr2=0x7f371c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3722325640 1 -- 172.21.15.29:0/2896887285 shutdown_connections 2024-04-25T00:39:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.798+0000 7f3722325640 1 -- 172.21.15.29:0/2896887285 wait complete. 2024-04-25T00:39:33.818 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:33.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad42a9640 1 Processor -- start 2024-04-25T00:39:33.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad42a9640 1 -- start start 2024-04-25T00:39:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad42a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc0b4780 0x7f5acc15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad42a9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5acc0cc240 0x7f5acc15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad42a9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5acc0b3720 con 0x7f5acc15e8d0 2024-04-25T00:39:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad42a9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5acc0b42f0 con 0x7f5acc0cc240 2024-04-25T00:39:33.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad42a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5acc0b39c0 con 0x7f5acc0b4780 2024-04-25T00:39:33.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad2846640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5acc0cc240 0x7f5acc15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad2846640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5acc0cc240 0x7f5acc15c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43720/0 (socket says 172.21.15.29:43720) 2024-04-25T00:39:33.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad2846640 1 -- 172.21.15.29:0/3184976187 learned_addr learned my addr 172.21.15.29:0/3184976187 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:33.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad1844640 1 --2- 172.21.15.29:0/3184976187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc0b4780 0x7f5acc15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad2846640 1 -- 172.21.15.29:0/3184976187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc0b4780 msgr2=0x7f5acc15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:33.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad2846640 1 --2- 172.21.15.29:0/3184976187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc0b4780 0x7f5acc15f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad2846640 1 -- 172.21.15.29:0/3184976187 >> v1:172.21.15.157:6789/0 conn(0x7f5acc15e8d0 legacy=0x7f5acc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.934+0000 7f5ad2846640 1 -- 172.21.15.29:0/3184976187 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc0b3ca0 con 0x7f5acc0cc240 2024-04-25T00:39:33.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad2846640 1 --2- 172.21.15.29:0/3184976187 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5acc0cc240 0x7f5acc15c7d0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400d150 tx=0x7f5ac401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b56403a89ccde1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:33.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad1043640 1 -- 172.21.15.29:0/3184976187 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac400b2d0 con 0x7f5acc0cc240 2024-04-25T00:39:33.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad1043640 1 -- 172.21.15.29:0/3184976187 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac400b470 con 0x7f5acc0cc240 2024-04-25T00:39:33.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad1043640 1 -- 172.21.15.29:0/3184976187 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac40a7bf0 con 0x7f5acc0cc240 2024-04-25T00:39:33.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 -- 172.21.15.29:0/3184976187 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5acc0cc240 msgr2=0x7f5acc15c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 --2- 172.21.15.29:0/3184976187 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5acc0cc240 0x7f5acc15c7d0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400d150 tx=0x7f5ac401ec30 comp rx=0 tx=0).stop 2024-04-25T00:39:33.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 -- 172.21.15.29:0/3184976187 shutdown_connections 2024-04-25T00:39:33.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 --2- 172.21.15.29:0/3184976187 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5acc0cc240 0x7f5acc15c7d0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 --2- 172.21.15.29:0/3184976187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc0b4780 0x7f5acc15f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 -- 172.21.15.29:0/3184976187 >> 172.21.15.29:0/3184976187 conn(0x7f5acc0b1870 msgr2=0x7f5acc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:33.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 -- 172.21.15.29:0/3184976187 shutdown_connections 2024-04-25T00:39:33.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 -- 172.21.15.29:0/3184976187 wait complete. 2024-04-25T00:39:33.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 Processor -- start 2024-04-25T00:39:33.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 -- start start 2024-04-25T00:39:33.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5acc0b4780 0x7f5acc15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5acc0cc240 0x7f5acc129f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc15e8d0 0x7f5acc12a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc15d6a0 con 0x7f5acc0b4780 2024-04-25T00:39:33.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc15d520 con 0x7f5acc0cc240 2024-04-25T00:39:33.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5acc06d590 con 0x7f5acc15e8d0 2024-04-25T00:39:33.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad2045640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5acc0b4780 0x7f5acc15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad2045640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5acc0b4780 0x7f5acc15a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43728/0 (socket says 172.21.15.29:43728) 2024-04-25T00:39:33.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad2045640 1 -- 172.21.15.29:0/822848926 learned_addr learned my addr 172.21.15.29:0/822848926 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:33.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad1844640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5acc0cc240 0x7f5acc129f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad2846640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc15e8d0 0x7f5acc12a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad2045640 1 -- 172.21.15.29:0/822848926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc15e8d0 msgr2=0x7f5acc12a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad2045640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc15e8d0 0x7f5acc12a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad2045640 1 -- 172.21.15.29:0/822848926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5acc0cc240 msgr2=0x7f5acc129f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:33.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad2045640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5acc0cc240 0x7f5acc129f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:33.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad2045640 1 -- 172.21.15.29:0/822848926 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc12ab20 con 0x7f5acc0b4780 2024-04-25T00:39:33.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad2045640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5acc0b4780 0x7f5acc15a1e0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800c370 tx=0x7f5ac800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:33.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ac2ffd640 1 -- 172.21.15.29:0/822848926 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac80052d0 con 0x7f5acc0b4780 2024-04-25T00:39:33.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ac2ffd640 1 -- 172.21.15.29:0/822848926 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac8005470 con 0x7f5acc0b4780 2024-04-25T00:39:33.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ad42a9640 1 -- 172.21.15.29:0/822848926 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5acc12ae00 con 0x7f5acc0b4780 2024-04-25T00:39:33.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.938+0000 7f5ac2ffd640 1 -- 172.21.15.29:0/822848926 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8007260 con 0x7f5acc0b4780 2024-04-25T00:39:33.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.942+0000 7f5ad42a9640 1 -- 172.21.15.29:0/822848926 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5acc12b3c0 con 0x7f5acc0b4780 2024-04-25T00:39:33.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.942+0000 7f5ad42a9640 1 -- 172.21.15.29:0/822848926 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5acc069270 con 0x7f5acc0b4780 2024-04-25T00:39:33.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.942+0000 7f5ac2ffd640 1 -- 172.21.15.29:0/822848926 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f5ac8007400 con 0x7f5acc0b4780 2024-04-25T00:39:33.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.942+0000 7f5ac2ffd640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5ab0067ef0 0x7f5ab006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:33.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.942+0000 7f5ac2ffd640 1 -- 172.21.15.29:0/822848926 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f5ac8085c50 con 0x7f5acc0b4780 2024-04-25T00:39:33.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.942+0000 7f5ad1844640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5ab0067ef0 0x7f5ab006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:33.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.942+0000 7f5ad1844640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5ab0067ef0 0x7f5ab006a3b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5acc1f6380 tx=0x7f5abc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:33.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:33.946+0000 7f5ac2ffd640 1 -- 172.21.15.29:0/822848926 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5ac8055e80 con 0x7f5acc0b4780 2024-04-25T00:39:34.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.110+0000 7f5ad42a9640 1 -- 172.21.15.29:0/822848926 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5acc0b4e00 con 0x7f5acc0b4780 2024-04-25T00:39:34.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.110+0000 7f5ac2ffd640 1 -- 172.21.15.29:0/822848926 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16680 (secure 0 0 0) 0x7f5ac8055780 con 0x7f5acc0b4780 2024-04-25T00:39:34.114 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:34.115 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":134,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:24.461420+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:06.791988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:07.503204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:34.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.114+0000 7f5ad42a9640 1 -- 172.21.15.29:0/822848926 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5ab0067ef0 msgr2=0x7f5ab006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.114+0000 7f5ad42a9640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5ab0067ef0 0x7f5ab006a3b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5acc1f6380 tx=0x7f5abc008040 comp rx=0 tx=0).stop 2024-04-25T00:39:34.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.114+0000 7f5ad42a9640 1 -- 172.21.15.29:0/822848926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5acc0b4780 msgr2=0x7f5acc15a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.114+0000 7f5ad42a9640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5acc0b4780 0x7f5acc15a1e0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800c370 tx=0x7f5ac800c830 comp rx=0 tx=0).stop 2024-04-25T00:39:34.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.114+0000 7f5ad42a9640 1 -- 172.21.15.29:0/822848926 shutdown_connections 2024-04-25T00:39:34.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.114+0000 7f5ad42a9640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5ab0067ef0 0x7f5ab006a3b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.114+0000 7f5ad42a9640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc15e8d0 0x7f5acc12a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.114+0000 7f5ad42a9640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5acc0cc240 0x7f5acc129f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.114+0000 7f5ad42a9640 1 --2- 172.21.15.29:0/822848926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5acc0b4780 0x7f5acc15a1e0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.114+0000 7f5ad42a9640 1 -- 172.21.15.29:0/822848926 >> 172.21.15.29:0/822848926 conn(0x7f5acc0b1870 msgr2=0x7f5acc071fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:34.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.114+0000 7f5ad42a9640 1 -- 172.21.15.29:0/822848926 shutdown_connections 2024-04-25T00:39:34.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.114+0000 7f5ad42a9640 1 -- 172.21.15.29:0/822848926 wait complete. 2024-04-25T00:39:34.134 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:39:06.791988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:39:07.503204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:39:34.135 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:39:34.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b8061640 1 Processor -- start 2024-04-25T00:39:34.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b8061640 1 -- start start 2024-04-25T00:39:34.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b8061640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32b00d18b0 0x7f32b006de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b8061640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b00cd7d0 con 0x7f32b006b210 2024-04-25T00:39:34.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b8061640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b00cdd50 con 0x7f32b0145980 2024-04-25T00:39:34.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b8061640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f32b00cf7d0 con 0x7f32b00d18b0 2024-04-25T00:39:34.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b65fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f32b006b210 0x7f32b00cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42188/0 (socket says 172.21.15.29:42188) 2024-04-25T00:39:34.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b65fe640 1 -- 172.21.15.29:0/3185354224 learned_addr learned my addr 172.21.15.29:0/3185354224 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:34.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b55fc640 1 --2- 172.21.15.29:0/3185354224 >> v2:172.21.15.157:3300/0 conn(0x7f32b00d18b0 0x7f32b006de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3998336251 0 0) 0x7f32b00cd7d0 con 0x7f32b006b210 2024-04-25T00:39:34.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f329c003530 con 0x7f32b006b210 2024-04-25T00:39:34.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1068391436 0 0) 0x7f329c003530 con 0x7f32b006b210 2024-04-25T00:39:34.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 >> v2:172.21.15.157:3300/0 conn(0x7f32b00d18b0 msgr2=0x7f32b006de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:34.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 --2- 172.21.15.29:0/3185354224 >> v2:172.21.15.157:3300/0 conn(0x7f32b00d18b0 0x7f32b006de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 >> v1:172.21.15.157:6789/0 conn(0x7f32b0145980 legacy=0x7f32b0160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b00cda70 con 0x7f32b006b210 2024-04-25T00:39:34.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f32a8001a00 con 0x7f32b006b210 2024-04-25T00:39:34.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 >> v1:172.21.15.29:6789/0 conn(0x7f32b006b210 legacy=0x7f32b00cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 --2- 172.21.15.29:0/3185354224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f329c003a10 0x7f329c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 --2- 172.21.15.29:0/3185354224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f329c004810 0x7f329c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 --2- 172.21.15.29:0/3185354224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f329c005240 0x7f329c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f32b00cf7d0 con 0x7f329c005240 2024-04-25T00:39:34.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f329c0105b0 con 0x7f329c004810 2024-04-25T00:39:34.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f329c0107b0 con 0x7f329c003a10 2024-04-25T00:39:34.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b65fe640 1 --2- 172.21.15.29:0/3185354224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f329c005240 0x7f329c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b55fc640 1 --2- 172.21.15.29:0/3185354224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f329c004810 0x7f329c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b5dfd640 1 --2- 172.21.15.29:0/3185354224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f329c003a10 0x7f329c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b65fe640 1 -- 172.21.15.29:0/3185354224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f329c003a10 msgr2=0x7f329c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:34.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b65fe640 1 --2- 172.21.15.29:0/3185354224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f329c003a10 0x7f329c003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b65fe640 1 -- 172.21.15.29:0/3185354224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f329c004810 msgr2=0x7f329c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:34.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b65fe640 1 --2- 172.21.15.29:0/3185354224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f329c004810 0x7f329c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b65fe640 1 -- 172.21.15.29:0/3185354224 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c003530 con 0x7f329c005240 2024-04-25T00:39:34.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.258+0000 7f32b55fc640 1 --2- 172.21.15.29:0/3185354224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f329c004810 0x7f329c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:34.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b65fe640 1 -- 172.21.15.29:0/3185354224 reap_dead start 2024-04-25T00:39:34.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b65fe640 1 --2- 172.21.15.29:0/3185354224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f329c005240 0x7f329c00b9d0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f32a800a790 tx=0x7f32a801f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a562c28558190047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:34.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a800a3b0 con 0x7f329c005240 2024-04-25T00:39:34.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a80ac410 con 0x7f329c005240 2024-04-25T00:39:34.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b4dfb640 1 -- 172.21.15.29:0/3185354224 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a80ac800 con 0x7f329c005240 2024-04-25T00:39:34.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 -- 172.21.15.29:0/3185354224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f329c005240 msgr2=0x7f329c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 --2- 172.21.15.29:0/3185354224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f329c005240 0x7f329c00b9d0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f32a800a790 tx=0x7f32a801f960 comp rx=0 tx=0).stop 2024-04-25T00:39:34.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 -- 172.21.15.29:0/3185354224 shutdown_connections 2024-04-25T00:39:34.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 --2- 172.21.15.29:0/3185354224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f329c005240 0x7f329c00b9d0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 -- 172.21.15.29:0/3185354224 >> 172.21.15.29:0/3185354224 conn(0x7f32b00d0c90 msgr2=0x7f32b00d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:34.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 -- 172.21.15.29:0/3185354224 shutdown_connections 2024-04-25T00:39:34.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 -- 172.21.15.29:0/3185354224 wait complete. 2024-04-25T00:39:34.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 Processor -- start 2024-04-25T00:39:34.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 -- start start 2024-04-25T00:39:34.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f329c005240 0x7f32b0148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b0141540 0x7f32b0143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b01419c0 0x7f32b0144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f32b006b7e0 con 0x7f329c005240 2024-04-25T00:39:34.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f32b006b660 con 0x7f32b01419c0 2024-04-25T00:39:34.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b8061640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f32b006b960 con 0x7f32b0141540 2024-04-25T00:39:34.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b5dfd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f329c005240 0x7f32b0148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b5dfd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f329c005240 0x7f32b0148380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43740/0 (socket says 172.21.15.29:43740) 2024-04-25T00:39:34.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b5dfd640 1 -- 172.21.15.29:0/996462649 learned_addr learned my addr 172.21.15.29:0/996462649 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:34.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b65fe640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b01419c0 0x7f32b0144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b55fc640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b0141540 0x7f32b0143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b5dfd640 1 -- 172.21.15.29:0/996462649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b0141540 msgr2=0x7f32b0143e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b5dfd640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b0141540 0x7f32b0143e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.262+0000 7f32b5dfd640 1 -- 172.21.15.29:0/996462649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b01419c0 msgr2=0x7f32b0144340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32b5dfd640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b01419c0 0x7f32b0144340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32b5dfd640 1 -- 172.21.15.29:0/996462649 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b0144a30 con 0x7f329c005240 2024-04-25T00:39:34.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32b65fe640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b01419c0 0x7f32b0144340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:34.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32b5dfd640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f329c005240 0x7f32b0148380 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00d570 tx=0x7f32ac00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:34.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32a67fc640 1 -- 172.21.15.29:0/996462649 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac005480 con 0x7f329c005240 2024-04-25T00:39:34.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32a67fc640 1 -- 172.21.15.29:0/996462649 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32ac017070 con 0x7f329c005240 2024-04-25T00:39:34.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32b8061640 1 -- 172.21.15.29:0/996462649 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b015d770 con 0x7f329c005240 2024-04-25T00:39:34.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32a67fc640 1 -- 172.21.15.29:0/996462649 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac003820 con 0x7f329c005240 2024-04-25T00:39:34.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32b8061640 1 -- 172.21.15.29:0/996462649 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b015dd30 con 0x7f329c005240 2024-04-25T00:39:34.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32b8061640 1 -- 172.21.15.29:0/996462649 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b00b2660 con 0x7f329c005240 2024-04-25T00:39:34.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32a67fc640 1 -- 172.21.15.29:0/996462649 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f32ac0039c0 con 0x7f329c005240 2024-04-25T00:39:34.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32a67fc640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3294067f20 0x7f329406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32a67fc640 1 -- 172.21.15.29:0/996462649 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f32ac0866b0 con 0x7f329c005240 2024-04-25T00:39:34.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32b55fc640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3294067f20 0x7f329406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.266+0000 7f32b55fc640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3294067f20 0x7f329406a3e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f32b00b0f20 tx=0x7f32a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:34.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.270+0000 7f32a67fc640 1 -- 172.21.15.29:0/996462649 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f32ac056810 con 0x7f329c005240 2024-04-25T00:39:34.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.434+0000 7f32b8061640 1 -- 172.21.15.29:0/996462649 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32b0073580 con 0x7f329c005240 2024-04-25T00:39:34.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.434+0000 7f32a67fc640 1 -- 172.21.15.29:0/996462649 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7f32ac056110 con 0x7f329c005240 2024-04-25T00:39:34.437 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:39:34.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.438+0000 7f32b8061640 1 -- 172.21.15.29:0/996462649 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3294067f20 msgr2=0x7f329406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.438+0000 7f32b8061640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3294067f20 0x7f329406a3e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f32b00b0f20 tx=0x7f32a0008040 comp rx=0 tx=0).stop 2024-04-25T00:39:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.438+0000 7f32b8061640 1 -- 172.21.15.29:0/996462649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f329c005240 msgr2=0x7f32b0148380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.438+0000 7f32b8061640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f329c005240 0x7f32b0148380 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00d570 tx=0x7f32ac00da30 comp rx=0 tx=0).stop 2024-04-25T00:39:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.438+0000 7f32b8061640 1 -- 172.21.15.29:0/996462649 shutdown_connections 2024-04-25T00:39:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.438+0000 7f32b8061640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3294067f20 0x7f329406a3e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.438+0000 7f32b8061640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b01419c0 0x7f32b0144340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.438+0000 7f32b8061640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b0141540 0x7f32b0143e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.438+0000 7f32b8061640 1 --2- 172.21.15.29:0/996462649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f329c005240 0x7f32b0148380 secure :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00d570 tx=0x7f32ac00da30 comp rx=0 tx=0).stop 2024-04-25T00:39:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.438+0000 7f32b8061640 1 -- 172.21.15.29:0/996462649 >> 172.21.15.29:0/996462649 conn(0x7f32b00d0c90 msgr2=0x7f32b00ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.438+0000 7f32b8061640 1 -- 172.21.15.29:0/996462649 shutdown_connections 2024-04-25T00:39:34.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.438+0000 7f32b8061640 1 -- 172.21.15.29:0/996462649 wait complete. 2024-04-25T00:39:34.457 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:39:34.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 Processor -- start 2024-04-25T00:39:34.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 -- start start 2024-04-25T00:39:34.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f237c0b3140 0x7f237c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f237c0b4b10 0x7f237c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f237c0b41e0 con 0x7f237c0b3a60 2024-04-25T00:39:34.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f237c06d960 con 0x7f237c0b3140 2024-04-25T00:39:34.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f237c06dc40 con 0x7f237c0b4b10 2024-04-25T00:39:34.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2381a02640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f237c0b3140 0x7f237c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2382203640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f237c0b3a60 0x7f237c15c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42198/0 (socket says 172.21.15.29:42198) 2024-04-25T00:39:34.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2381a02640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f237c0b3140 0x7f237c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43746/0 (socket says 172.21.15.29:43746) 2024-04-25T00:39:34.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2382203640 1 -- 172.21.15.29:0/1751947312 learned_addr learned my addr 172.21.15.29:0/1751947312 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:34.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2381201640 1 --2- 172.21.15.29:0/1751947312 >> v2:172.21.15.157:3300/0 conn(0x7f237c0b4b10 0x7f237c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2381a02640 1 -- 172.21.15.29:0/1751947312 >> v2:172.21.15.157:3300/0 conn(0x7f237c0b4b10 msgr2=0x7f237c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:34.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2381a02640 1 --2- 172.21.15.29:0/1751947312 >> v2:172.21.15.157:3300/0 conn(0x7f237c0b4b10 0x7f237c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2381a02640 1 -- 172.21.15.29:0/1751947312 >> v1:172.21.15.29:6789/0 conn(0x7f237c0b3a60 legacy=0x7f237c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2381a02640 1 -- 172.21.15.29:0/1751947312 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c06df20 con 0x7f237c0b3140 2024-04-25T00:39:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2381a02640 1 --2- 172.21.15.29:0/1751947312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f237c0b3140 0x7f237c06e440 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f23780097c0 tx=0x7f237801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d3554d9dbfa6be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2380a00640 1 -- 172.21.15.29:0/1751947312 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780b0070 con 0x7f237c0b3140 2024-04-25T00:39:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2380a00640 1 -- 172.21.15.29:0/1751947312 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23780ab600 con 0x7f237c0b3140 2024-04-25T00:39:34.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2380a00640 1 -- 172.21.15.29:0/1751947312 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780ab9d0 con 0x7f237c0b3140 2024-04-25T00:39:34.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 -- 172.21.15.29:0/1751947312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f237c0b3140 msgr2=0x7f237c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 --2- 172.21.15.29:0/1751947312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f237c0b3140 0x7f237c06e440 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f23780097c0 tx=0x7f237801c460 comp rx=0 tx=0).stop 2024-04-25T00:39:34.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 -- 172.21.15.29:0/1751947312 shutdown_connections 2024-04-25T00:39:34.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 --2- 172.21.15.29:0/1751947312 >> v2:172.21.15.157:3300/0 conn(0x7f237c0b4b10 0x7f237c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 --2- 172.21.15.29:0/1751947312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f237c0b3140 0x7f237c06e440 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 -- 172.21.15.29:0/1751947312 >> 172.21.15.29:0/1751947312 conn(0x7f237c0b1870 msgr2=0x7f237c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:34.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 -- 172.21.15.29:0/1751947312 shutdown_connections 2024-04-25T00:39:34.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 -- 172.21.15.29:0/1751947312 wait complete. 2024-04-25T00:39:34.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 Processor -- start 2024-04-25T00:39:34.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 -- start start 2024-04-25T00:39:34.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f237c0b3140 0x7f237c13de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237c0b3a60 0x7f237c1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237c0b4b10 0x7f237c137180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f237c168b70 con 0x7f237c0b3140 2024-04-25T00:39:34.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f237c1689f0 con 0x7f237c0b3a60 2024-04-25T00:39:34.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2383c66640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f237c168cf0 con 0x7f237c0b4b10 2024-04-25T00:39:34.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2381a02640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f237c0b3140 0x7f237c13de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2381a02640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f237c0b3140 0x7f237c13de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43750/0 (socket says 172.21.15.29:43750) 2024-04-25T00:39:34.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.590+0000 7f2381a02640 1 -- 172.21.15.29:0/3953139147 learned_addr learned my addr 172.21.15.29:0/3953139147 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:34.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2381201640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237c0b3a60 0x7f237c1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2381a02640 1 -- 172.21.15.29:0/3953139147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237c0b4b10 msgr2=0x7f237c137180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2382203640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237c0b4b10 0x7f237c137180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2381a02640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237c0b4b10 0x7f237c137180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2381a02640 1 -- 172.21.15.29:0/3953139147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237c0b3a60 msgr2=0x7f237c1557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2381a02640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237c0b3a60 0x7f237c1557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2381a02640 1 -- 172.21.15.29:0/3953139147 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c13e410 con 0x7f237c0b3140 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2381201640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237c0b3a60 0x7f237c1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2382203640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237c0b4b10 0x7f237c137180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2381a02640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f237c0b3140 0x7f237c13de30 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f23780ae040 tx=0x7f237801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f23727fc640 1 -- 172.21.15.29:0/3953139147 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780b0050 con 0x7f237c0b3140 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f23727fc640 1 -- 172.21.15.29:0/3953139147 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23780bc070 con 0x7f237c0b3140 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f23727fc640 1 -- 172.21.15.29:0/3953139147 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780b5850 con 0x7f237c0b3140 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2383c66640 1 -- 172.21.15.29:0/3953139147 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237c13e6f0 con 0x7f237c0b3140 2024-04-25T00:39:34.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2383c66640 1 -- 172.21.15.29:0/3953139147 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237c1422c0 con 0x7f237c0b3140 2024-04-25T00:39:34.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2383c66640 1 -- 172.21.15.29:0/3953139147 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237c069270 con 0x7f237c0b3140 2024-04-25T00:39:34.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f23727fc640 1 -- 172.21.15.29:0/3953139147 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f23780b59f0 con 0x7f237c0b3140 2024-04-25T00:39:34.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f23727fc640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2360067ef0 0x7f236006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f23727fc640 1 -- 172.21.15.29:0/3953139147 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f2378120f90 con 0x7f237c0b3140 2024-04-25T00:39:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2381201640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2360067ef0 0x7f236006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.594+0000 7f2381201640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2360067ef0 0x7f236006a3b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f237c13f0f0 tx=0x7f236c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:34.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.598+0000 7f23727fc640 1 -- 172.21.15.29:0/3953139147 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2378016050 con 0x7f237c0b3140 2024-04-25T00:39:34.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.762+0000 7f2383c66640 1 -- 172.21.15.29:0/3953139147 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f237c15db00 con 0x7f237c0b3140 2024-04-25T00:39:34.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.762+0000 7f23727fc640 1 -- 172.21.15.29:0/3953139147 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) ==== 104+0+11 (secure 0 0 0) 0x7f23780f11d0 con 0x7f237c0b3140 2024-04-25T00:39:34.765 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:39:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.766+0000 7f2383c66640 1 -- 172.21.15.29:0/3953139147 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2360067ef0 msgr2=0x7f236006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.766+0000 7f2383c66640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2360067ef0 0x7f236006a3b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f237c13f0f0 tx=0x7f236c008040 comp rx=0 tx=0).stop 2024-04-25T00:39:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.766+0000 7f2383c66640 1 -- 172.21.15.29:0/3953139147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f237c0b3140 msgr2=0x7f237c13de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.766+0000 7f2383c66640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f237c0b3140 0x7f237c13de30 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f23780ae040 tx=0x7f237801aca0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.766+0000 7f2383c66640 1 -- 172.21.15.29:0/3953139147 shutdown_connections 2024-04-25T00:39:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.766+0000 7f2383c66640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2360067ef0 0x7f236006a3b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.766+0000 7f2383c66640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237c0b4b10 0x7f237c137180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.766+0000 7f2383c66640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237c0b3a60 0x7f237c1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.766+0000 7f2383c66640 1 --2- 172.21.15.29:0/3953139147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f237c0b3140 0x7f237c13de30 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.766+0000 7f2383c66640 1 -- 172.21.15.29:0/3953139147 >> 172.21.15.29:0/3953139147 conn(0x7f237c0b1870 msgr2=0x7f237c13ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.766+0000 7f2383c66640 1 -- 172.21.15.29:0/3953139147 shutdown_connections 2024-04-25T00:39:34.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.766+0000 7f2383c66640 1 -- 172.21.15.29:0/3953139147 wait complete. 2024-04-25T00:39:34.785 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:39:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.910+0000 7f4f36f7a640 1 Processor -- start 2024-04-25T00:39:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.910+0000 7f4f36f7a640 1 -- start start 2024-04-25T00:39:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.910+0000 7f4f36f7a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4f3006c950 0x7f4f3006cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.910+0000 7f4f36f7a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f300cc3b0 con 0x7f4f300cd410 2024-04-25T00:39:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.910+0000 7f4f36f7a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f3006d270 con 0x7f4f3015e440 2024-04-25T00:39:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.910+0000 7f4f36f7a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4f300b11d0 con 0x7f4f3006c950 2024-04-25T00:39:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.910+0000 7f4f34d16640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4f3006c950 0x7f4f3006cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.910+0000 7f4f35517640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4f300cd410 0x7f4f300cd8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42202/0 (socket says 172.21.15.29:42202) 2024-04-25T00:39:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.910+0000 7f4f35517640 1 -- 172.21.15.29:0/679180084 learned_addr learned my addr 172.21.15.29:0/679180084 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:34.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.910+0000 7f4f34d16640 1 -- 172.21.15.29:0/679180084 >> v1:172.21.15.157:6789/0 conn(0x7f4f3015e440 legacy=0x7f4f300d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:34.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.910+0000 7f4f34d16640 1 -- 172.21.15.29:0/679180084 >> v1:172.21.15.29:6789/0 conn(0x7f4f300cd410 legacy=0x7f4f300cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.910+0000 7f4f34d16640 1 -- 172.21.15.29:0/679180084 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f300cc650 con 0x7f4f3006c950 2024-04-25T00:39:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f34d16640 1 --2- 172.21.15.29:0/679180084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f3006c950 0x7f4f3006cd30 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f4f200097c0 tx=0x7f4f2001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881dccc41867d3d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2f7fe640 1 -- 172.21.15.29:0/679180084 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f200b0070 con 0x7f4f3006c950 2024-04-25T00:39:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2f7fe640 1 -- 172.21.15.29:0/679180084 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f20004230 con 0x7f4f3006c950 2024-04-25T00:39:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2f7fe640 1 -- 172.21.15.29:0/679180084 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f20003620 con 0x7f4f3006c950 2024-04-25T00:39:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 -- 172.21.15.29:0/679180084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f3006c950 msgr2=0x7f4f3006cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 --2- 172.21.15.29:0/679180084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f3006c950 0x7f4f3006cd30 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f4f200097c0 tx=0x7f4f2001c460 comp rx=0 tx=0).stop 2024-04-25T00:39:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 -- 172.21.15.29:0/679180084 shutdown_connections 2024-04-25T00:39:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 --2- 172.21.15.29:0/679180084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f3006c950 0x7f4f3006cd30 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 -- 172.21.15.29:0/679180084 >> 172.21.15.29:0/679180084 conn(0x7f4f300d1d60 msgr2=0x7f4f300d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 -- 172.21.15.29:0/679180084 shutdown_connections 2024-04-25T00:39:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 -- 172.21.15.29:0/679180084 wait complete. 2024-04-25T00:39:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 Processor -- start 2024-04-25T00:39:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 -- start start 2024-04-25T00:39:34.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f300cd410 0x7f4f300c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f3015e440 0x7f4f30158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f300cbb50 0x7f4f30159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4f300d11c0 con 0x7f4f3015e440 2024-04-25T00:39:34.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3006c520 con 0x7f4f300cd410 2024-04-25T00:39:34.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4f300ce3c0 con 0x7f4f300cbb50 2024-04-25T00:39:34.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f3015e440 0x7f4f30158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f3015e440 0x7f4f30158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43762/0 (socket says 172.21.15.29:43762) 2024-04-25T00:39:34.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2ffff640 1 -- 172.21.15.29:0/3100245718 learned_addr learned my addr 172.21.15.29:0/3100245718 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:34.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2ffff640 1 -- 172.21.15.29:0/3100245718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f300cbb50 msgr2=0x7f4f30159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2ffff640 1 --2- 172.21.15.29:0/3100245718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f300cbb50 0x7f4f30159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2ffff640 1 -- 172.21.15.29:0/3100245718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f300cd410 msgr2=0x7f4f300c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:34.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2ffff640 1 --2- 172.21.15.29:0/3100245718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f300cd410 0x7f4f300c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:34.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2ffff640 1 -- 172.21.15.29:0/3100245718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f300c7f00 con 0x7f4f3015e440 2024-04-25T00:39:34.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2ffff640 1 --2- 172.21.15.29:0/3100245718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f3015e440 0x7f4f30158f50 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c450 tx=0x7f4f2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:34.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2d7fa640 1 -- 172.21.15.29:0/3100245718 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24012830 con 0x7f4f3015e440 2024-04-25T00:39:34.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2d7fa640 1 -- 172.21.15.29:0/3100245718 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f24017070 con 0x7f4f3015e440 2024-04-25T00:39:34.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 -- 172.21.15.29:0/3100245718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f300c81e0 con 0x7f4f3015e440 2024-04-25T00:39:34.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2d7fa640 1 -- 172.21.15.29:0/3100245718 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24012c30 con 0x7f4f3015e440 2024-04-25T00:39:34.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 -- 172.21.15.29:0/3100245718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f300c87a0 con 0x7f4f3015e440 2024-04-25T00:39:34.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f36f7a640 1 -- 172.21.15.29:0/3100245718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f300c83b0 con 0x7f4f3015e440 2024-04-25T00:39:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2d7fa640 1 -- 172.21.15.29:0/3100245718 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f4f2401c460 con 0x7f4f3015e440 2024-04-25T00:39:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2d7fa640 1 --2- 172.21.15.29:0/3100245718 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4f1c067ef0 0x7f4f1c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f2d7fa640 1 -- 172.21.15.29:0/3100245718 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f4f24087090 con 0x7f4f3015e440 2024-04-25T00:39:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f34d16640 1 --2- 172.21.15.29:0/3100245718 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4f1c067ef0 0x7f4f1c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:34.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.918+0000 7f4f34d16640 1 --2- 172.21.15.29:0/3100245718 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4f1c067ef0 0x7f4f1c06a3b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4f200ad000 tx=0x7f4f20004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:34.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:34.922+0000 7f4f2d7fa640 1 -- 172.21.15.29:0/3100245718 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4f24057370 con 0x7f4f3015e440 2024-04-25T00:39:35.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.082+0000 7f4f36f7a640 1 -- 172.21.15.29:0/3100245718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4f30069270 con 0x7f4f3015e440 2024-04-25T00:39:35.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.082+0000 7f4f2d7fa640 1 -- 172.21.15.29:0/3100245718 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) ==== 100+0+11 (secure 0 0 0) 0x7f4f2405d020 con 0x7f4f3015e440 2024-04-25T00:39:35.087 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:39:35.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.086+0000 7f4f36f7a640 1 -- 172.21.15.29:0/3100245718 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4f1c067ef0 msgr2=0x7f4f1c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.086+0000 7f4f36f7a640 1 --2- 172.21.15.29:0/3100245718 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4f1c067ef0 0x7f4f1c06a3b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4f200ad000 tx=0x7f4f20004740 comp rx=0 tx=0).stop 2024-04-25T00:39:35.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.086+0000 7f4f36f7a640 1 -- 172.21.15.29:0/3100245718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f3015e440 msgr2=0x7f4f30158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.086+0000 7f4f36f7a640 1 --2- 172.21.15.29:0/3100245718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f3015e440 0x7f4f30158f50 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c450 tx=0x7f4f2400c910 comp rx=0 tx=0).stop 2024-04-25T00:39:35.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.086+0000 7f4f36f7a640 1 -- 172.21.15.29:0/3100245718 shutdown_connections 2024-04-25T00:39:35.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.086+0000 7f4f36f7a640 1 --2- 172.21.15.29:0/3100245718 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4f1c067ef0 0x7f4f1c06a3b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.086+0000 7f4f36f7a640 1 --2- 172.21.15.29:0/3100245718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f300cbb50 0x7f4f30159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.086+0000 7f4f36f7a640 1 --2- 172.21.15.29:0/3100245718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f3015e440 0x7f4f30158f50 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.086+0000 7f4f36f7a640 1 --2- 172.21.15.29:0/3100245718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f300cd410 0x7f4f300c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.086+0000 7f4f36f7a640 1 -- 172.21.15.29:0/3100245718 >> 172.21.15.29:0/3100245718 conn(0x7f4f300d1d60 msgr2=0x7f4f30066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:35.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.086+0000 7f4f36f7a640 1 -- 172.21.15.29:0/3100245718 shutdown_connections 2024-04-25T00:39:35.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.086+0000 7f4f36f7a640 1 -- 172.21.15.29:0/3100245718 wait complete. 2024-04-25T00:39:35.106 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:39:35.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 Processor -- start 2024-04-25T00:39:35.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 -- start start 2024-04-25T00:39:35.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe80b3e40 0x7fefe806d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe806d980 con 0x7fefe80b4d30 2024-04-25T00:39:35.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe806dc60 con 0x7fefe815df70 2024-04-25T00:39:35.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fefe806d750 con 0x7fefe80b3e40 2024-04-25T00:39:35.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefed4a3640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fefe80b4d30 0x7fefe80ad8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42218/0 (socket says 172.21.15.29:42218) 2024-04-25T00:39:35.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefed4a3640 1 -- 172.21.15.29:0/382557670 learned_addr learned my addr 172.21.15.29:0/382557670 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:35.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3126692401 0 0) 0x7fefe806d980 con 0x7fefe80b4d30 2024-04-25T00:39:35.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefd4003530 con 0x7fefe80b4d30 2024-04-25T00:39:35.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefee4a5640 1 --2- 172.21.15.29:0/382557670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe80b3e40 0x7fefe806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 913003679 0 0) 0x7fefd4003530 con 0x7fefe80b4d30 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe80b3e40 msgr2=0x7fefe806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 --2- 172.21.15.29:0/382557670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe80b3e40 0x7fefe806d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 >> v1:172.21.15.157:6789/0 conn(0x7fefe815df70 legacy=0x7fefe80b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe806df40 con 0x7fefe80b4d30 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fefdc001a00 con 0x7fefe80b4d30 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 >> v1:172.21.15.29:6789/0 conn(0x7fefe80b4d30 legacy=0x7fefe80ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 --2- 172.21.15.29:0/382557670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefd40039f0 0x7fefd4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 --2- 172.21.15.29:0/382557670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefd4004810 0x7fefd4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 --2- 172.21.15.29:0/382557670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe80b3e40 0x7fefd4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fefd400bad0 con 0x7fefd4004810 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fefd400bc80 con 0x7fefd40039f0 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fefd400be80 con 0x7fefe80b3e40 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefed4a3640 1 --2- 172.21.15.29:0/382557670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefd4004810 0x7fefd4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefedca4640 1 --2- 172.21.15.29:0/382557670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefd40039f0 0x7fefd4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefed4a3640 1 -- 172.21.15.29:0/382557670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe80b3e40 msgr2=0x7fefd4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:35.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefee4a5640 1 --2- 172.21.15.29:0/382557670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe80b3e40 0x7fefd4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefed4a3640 1 --2- 172.21.15.29:0/382557670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe80b3e40 0x7fefd4005210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefed4a3640 1 -- 172.21.15.29:0/382557670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefd40039f0 msgr2=0x7fefd4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:35.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefed4a3640 1 --2- 172.21.15.29:0/382557670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefd40039f0 0x7fefd4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefed4a3640 1 -- 172.21.15.29:0/382557670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd4003530 con 0x7fefd4004810 2024-04-25T00:39:35.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefedca4640 1 --2- 172.21.15.29:0/382557670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefd40039f0 0x7fefd4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:35.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefed4a3640 1 --2- 172.21.15.29:0/382557670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefd4004810 0x7fefd4004c90 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00a840 tx=0x7fefdc01fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675606b6e44f4824 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:35.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc00a460 con 0x7fefd4004810 2024-04-25T00:39:35.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefdc0aa710 con 0x7fefd4004810 2024-04-25T00:39:35.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefecca2640 1 -- 172.21.15.29:0/382557670 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc0aab00 con 0x7fefd4004810 2024-04-25T00:39:35.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 -- 172.21.15.29:0/382557670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefd4004810 msgr2=0x7fefd4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 --2- 172.21.15.29:0/382557670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefd4004810 0x7fefd4004c90 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00a840 tx=0x7fefdc01fac0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefee4a5640 1 -- 172.21.15.29:0/382557670 reap_dead start 2024-04-25T00:39:35.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 -- 172.21.15.29:0/382557670 shutdown_connections 2024-04-25T00:39:35.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 -- 172.21.15.29:0/382557670 >> 172.21.15.29:0/382557670 conn(0x7fefe80b1870 msgr2=0x7fefe80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:35.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 -- 172.21.15.29:0/382557670 shutdown_connections 2024-04-25T00:39:35.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 -- 172.21.15.29:0/382557670 wait complete. 2024-04-25T00:39:35.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 Processor -- start 2024-04-25T00:39:35.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 -- start start 2024-04-25T00:39:35.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefd4004810 0x7fefe80b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefe80af220 0x7fefe8156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe8158930 0x7fefe80acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fefe815e350 con 0x7fefd4004810 2024-04-25T00:39:35.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fefe806e500 con 0x7fefe80af220 2024-04-25T00:39:35.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefeff08640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fefe806ea20 con 0x7fefe8158930 2024-04-25T00:39:35.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefedca4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefd4004810 0x7fefe80b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefedca4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefd4004810 0x7fefe80b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43790/0 (socket says 172.21.15.29:43790) 2024-04-25T00:39:35.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.222+0000 7fefedca4640 1 -- 172.21.15.29:0/4171439693 learned_addr learned my addr 172.21.15.29:0/4171439693 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:35.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefed4a3640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefe80af220 0x7fefe8156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefee4a5640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe8158930 0x7fefe80acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefedca4640 1 -- 172.21.15.29:0/4171439693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe8158930 msgr2=0x7fefe80acbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefedca4640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe8158930 0x7fefe80acbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefedca4640 1 -- 172.21.15.29:0/4171439693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefe80af220 msgr2=0x7fefe8156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefedca4640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefe80af220 0x7fefe8156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefedca4640 1 -- 172.21.15.29:0/4171439693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe80af6b0 con 0x7fefd4004810 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefedca4640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefd4004810 0x7fefe80b0750 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fefe400d770 tx=0x7fefe400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefda7fc640 1 -- 172.21.15.29:0/4171439693 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4005270 con 0x7fefd4004810 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefda7fc640 1 -- 172.21.15.29:0/4171439693 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe4017070 con 0x7fefd4004810 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefeff08640 1 -- 172.21.15.29:0/4171439693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe80af990 con 0x7fefd4004810 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefda7fc640 1 -- 172.21.15.29:0/4171439693 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe400b6b0 con 0x7fefd4004810 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefeff08640 1 -- 172.21.15.29:0/4171439693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe80aff50 con 0x7fefd4004810 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefeff08640 1 -- 172.21.15.29:0/4171439693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe8069270 con 0x7fefd4004810 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefda7fc640 1 -- 172.21.15.29:0/4171439693 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fefe400b850 con 0x7fefd4004810 2024-04-25T00:39:35.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefda7fc640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fefcc067ef0 0x7fefcc06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefda7fc640 1 -- 172.21.15.29:0/4171439693 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7fefe4086850 con 0x7fefd4004810 2024-04-25T00:39:35.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefed4a3640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fefcc067ef0 0x7fefcc06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.226+0000 7fefed4a3640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fefcc067ef0 0x7fefcc06a3b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0b1000 tx=0x7fefdc007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:35.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.230+0000 7fefda7fc640 1 -- 172.21.15.29:0/4171439693 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fefe4019310 con 0x7fefd4004810 2024-04-25T00:39:35.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.426+0000 7fefeff08640 1 -- 172.21.15.29:0/4171439693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefe8157010 con 0x7fefd4004810 2024-04-25T00:39:35.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.426+0000 7fefda7fc640 1 -- 172.21.15.29:0/4171439693 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5179 (secure 0 0 0) 0x7fefe4056920 con 0x7fefd4004810 2024-04-25T00:39:35.427 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:35.427 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":84,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7450,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/456996219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":456996219},{"type":"v1","addr":"172.21.15.157:6833","nonce":456996219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/87071339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":87071339},{"type":"v1","addr":"172.21.15.157:6835","nonce":87071339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3920208567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3920208567},{"type":"v1","addr":"172.21.15.29:6833","nonce":3920208567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:39:08.508825+0000","modified":"2024-04-25T00:39:11.535481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9021":{"gid":9021,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/619209926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":619209926},{"type":"v1","addr":"172.21.15.29:6835","nonce":619209926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9021,"qdb_cluster":[9021]},"id":10}]} 2024-04-25T00:39:35.427 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 84 2024-04-25T00:39:35.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.430+0000 7fefeff08640 1 -- 172.21.15.29:0/4171439693 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fefcc067ef0 msgr2=0x7fefcc06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.430+0000 7fefeff08640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fefcc067ef0 0x7fefcc06a3b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0b1000 tx=0x7fefdc007040 comp rx=0 tx=0).stop 2024-04-25T00:39:35.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.430+0000 7fefeff08640 1 -- 172.21.15.29:0/4171439693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefd4004810 msgr2=0x7fefe80b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.430+0000 7fefeff08640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefd4004810 0x7fefe80b0750 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fefe400d770 tx=0x7fefe400dc30 comp rx=0 tx=0).stop 2024-04-25T00:39:35.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.430+0000 7fefeff08640 1 -- 172.21.15.29:0/4171439693 shutdown_connections 2024-04-25T00:39:35.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.430+0000 7fefeff08640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fefcc067ef0 0x7fefcc06a3b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.430+0000 7fefeff08640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefe8158930 0x7fefe80acbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.430+0000 7fefeff08640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefe80af220 0x7fefe8156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.430+0000 7fefeff08640 1 --2- 172.21.15.29:0/4171439693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefd4004810 0x7fefe80b0750 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.430+0000 7fefeff08640 1 -- 172.21.15.29:0/4171439693 >> 172.21.15.29:0/4171439693 conn(0x7fefe80b1870 msgr2=0x7fefe815c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:35.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.430+0000 7fefeff08640 1 -- 172.21.15.29:0/4171439693 shutdown_connections 2024-04-25T00:39:35.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.430+0000 7fefeff08640 1 -- 172.21.15.29:0/4171439693 wait complete. 2024-04-25T00:39:35.444 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:35.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b2bb4640 1 Processor -- start 2024-04-25T00:39:35.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b2bb4640 1 -- start start 2024-04-25T00:39:35.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b2bb4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35ac15e4e0 0x7f35ac0d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b2bb4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35ac15e8c0 0x7f35ac0b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b2bb4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ac0cfe30 con 0x7f35ac0d4bf0 2024-04-25T00:39:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b2bb4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f35ac0b6f80 con 0x7f35ac15e4e0 2024-04-25T00:39:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b2bb4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f35ac0b7260 con 0x7f35ac15e8c0 2024-04-25T00:39:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35abfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f35ac0d4bf0 0x7f35ac0d23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42224/0 (socket says 172.21.15.29:42224) 2024-04-25T00:39:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35abfff640 1 -- 172.21.15.29:0/3249551884 learned_addr learned my addr 172.21.15.29:0/3249551884 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b0950640 1 --2- 172.21.15.29:0/3249551884 >> v2:172.21.15.29:3300/0 conn(0x7f35ac15e4e0 0x7f35ac0d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b1151640 1 --2- 172.21.15.29:0/3249551884 >> v2:172.21.15.157:3300/0 conn(0x7f35ac15e8c0 0x7f35ac0b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35ab7fe640 1 -- 172.21.15.29:0/3249551884 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1112935852 0 0) 0x7f35ac0cfe30 con 0x7f35ac0d4bf0 2024-04-25T00:39:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35ab7fe640 1 -- 172.21.15.29:0/3249551884 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3594003530 con 0x7f35ac0d4bf0 2024-04-25T00:39:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b0950640 1 -- 172.21.15.29:0/3249551884 >> v2:172.21.15.157:3300/0 conn(0x7f35ac15e8c0 msgr2=0x7f35ac0b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b0950640 1 --2- 172.21.15.29:0/3249551884 >> v2:172.21.15.157:3300/0 conn(0x7f35ac15e8c0 0x7f35ac0b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b0950640 1 -- 172.21.15.29:0/3249551884 >> v1:172.21.15.29:6789/0 conn(0x7f35ac0d4bf0 legacy=0x7f35ac0d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b0950640 1 -- 172.21.15.29:0/3249551884 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac0ced50 con 0x7f35ac15e4e0 2024-04-25T00:39:35.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.570+0000 7f35b1151640 1 --2- 172.21.15.29:0/3249551884 >> v2:172.21.15.157:3300/0 conn(0x7f35ac15e8c0 0x7f35ac0b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b0950640 1 --2- 172.21.15.29:0/3249551884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ac15e4e0 0x7f35ac0d1eb0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f35ac073880 tx=0x7f359c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a1480eb280f7cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35ab7fe640 1 -- 172.21.15.29:0/3249551884 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c0b5070 con 0x7f35ac15e4e0 2024-04-25T00:39:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35ab7fe640 1 -- 172.21.15.29:0/3249551884 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c0a8970 con 0x7f35ac15e4e0 2024-04-25T00:39:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35ab7fe640 1 -- 172.21.15.29:0/3249551884 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c0a8d40 con 0x7f35ac15e4e0 2024-04-25T00:39:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 -- 172.21.15.29:0/3249551884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ac15e4e0 msgr2=0x7f35ac0d1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 --2- 172.21.15.29:0/3249551884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ac15e4e0 0x7f35ac0d1eb0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f35ac073880 tx=0x7f359c01ec70 comp rx=0 tx=0).stop 2024-04-25T00:39:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 -- 172.21.15.29:0/3249551884 shutdown_connections 2024-04-25T00:39:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 --2- 172.21.15.29:0/3249551884 >> v2:172.21.15.157:3300/0 conn(0x7f35ac15e8c0 0x7f35ac0b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 --2- 172.21.15.29:0/3249551884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ac15e4e0 0x7f35ac0d1eb0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 -- 172.21.15.29:0/3249551884 >> 172.21.15.29:0/3249551884 conn(0x7f35ac0d3fb0 msgr2=0x7f35ac0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 -- 172.21.15.29:0/3249551884 shutdown_connections 2024-04-25T00:39:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 -- 172.21.15.29:0/3249551884 wait complete. 2024-04-25T00:39:35.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 Processor -- start 2024-04-25T00:39:35.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 -- start start 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35ac0d4bf0 0x7f35ac15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ac15e4e0 0x7f35ac1f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac15e8c0 0x7f35ac0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac0b7e40 con 0x7f35ac15e4e0 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac0b7cc0 con 0x7f35ac15e8c0 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b2bb4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f35ac0b7fc0 con 0x7f35ac0d4bf0 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35abfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ac15e4e0 0x7f35ac1f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35abfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ac15e4e0 0x7f35ac1f3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43812/0 (socket says 172.21.15.29:43812) 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35abfff640 1 -- 172.21.15.29:0/2169087304 learned_addr learned my addr 172.21.15.29:0/2169087304 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35abfff640 1 -- 172.21.15.29:0/2169087304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35ac0d4bf0 msgr2=0x7f35ac15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b0950640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35ac0d4bf0 0x7f35ac15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35b1151640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac15e8c0 0x7f35ac0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35abfff640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35ac0d4bf0 0x7f35ac15a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35abfff640 1 -- 172.21.15.29:0/2169087304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac15e8c0 msgr2=0x7f35ac0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35abfff640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac15e8c0 0x7f35ac0cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35abfff640 1 -- 172.21.15.29:0/2169087304 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac1f4830 con 0x7f35ac15e4e0 2024-04-25T00:39:35.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35abfff640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ac15e4e0 0x7f35ac1f3f70 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f35a000c450 tx=0x7f35a000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:35.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.574+0000 7f35a97fa640 1 -- 172.21.15.29:0/2169087304 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0017070 con 0x7f35ac15e4e0 2024-04-25T00:39:35.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.578+0000 7f35a97fa640 1 -- 172.21.15.29:0/2169087304 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35a0005cf0 con 0x7f35ac15e4e0 2024-04-25T00:39:35.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.578+0000 7f35b2bb4640 1 -- 172.21.15.29:0/2169087304 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac1f4b10 con 0x7f35ac15e4e0 2024-04-25T00:39:35.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.578+0000 7f35a97fa640 1 -- 172.21.15.29:0/2169087304 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0007290 con 0x7f35ac15e4e0 2024-04-25T00:39:35.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.578+0000 7f35b2bb4640 1 -- 172.21.15.29:0/2169087304 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac15ae00 con 0x7f35ac15e4e0 2024-04-25T00:39:35.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.578+0000 7f35b2bb4640 1 -- 172.21.15.29:0/2169087304 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac069270 con 0x7f35ac15e4e0 2024-04-25T00:39:35.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.578+0000 7f35a97fa640 1 -- 172.21.15.29:0/2169087304 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f35a001a070 con 0x7f35ac15e4e0 2024-04-25T00:39:35.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.578+0000 7f35a97fa640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3598067f20 0x7f359806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.578+0000 7f35a97fa640 1 -- 172.21.15.29:0/2169087304 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f35a0085c30 con 0x7f35ac15e4e0 2024-04-25T00:39:35.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.578+0000 7f35b0950640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3598067f20 0x7f359806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.578+0000 7f35b0950640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3598067f20 0x7f359806a3e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f359c0b2000 tx=0x7f359c0aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:35.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.582+0000 7f35a97fa640 1 -- 172.21.15.29:0/2169087304 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f35a0055d90 con 0x7f35ac15e4e0 2024-04-25T00:39:35.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.746+0000 7f35b2bb4640 1 -- 172.21.15.29:0/2169087304 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35ac0d3530 con 0x7f35ac15e4e0 2024-04-25T00:39:35.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.746+0000 7f35a97fa640 1 -- 172.21.15.29:0/2169087304 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16680 (secure 0 0 0) 0x7f35a0055690 con 0x7f35ac15e4e0 2024-04-25T00:39:35.747 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:35.747 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":134,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:24.461420+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:06.791988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:07.503204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:35.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.750+0000 7f35b2bb4640 1 -- 172.21.15.29:0/2169087304 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3598067f20 msgr2=0x7f359806a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.750+0000 7f35b2bb4640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3598067f20 0x7f359806a3e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f359c0b2000 tx=0x7f359c0aa070 comp rx=0 tx=0).stop 2024-04-25T00:39:35.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.750+0000 7f35b2bb4640 1 -- 172.21.15.29:0/2169087304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ac15e4e0 msgr2=0x7f35ac1f3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.750+0000 7f35b2bb4640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ac15e4e0 0x7f35ac1f3f70 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f35a000c450 tx=0x7f35a000c910 comp rx=0 tx=0).stop 2024-04-25T00:39:35.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.750+0000 7f35b2bb4640 1 -- 172.21.15.29:0/2169087304 shutdown_connections 2024-04-25T00:39:35.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.750+0000 7f35b2bb4640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3598067f20 0x7f359806a3e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.750+0000 7f35b2bb4640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac15e8c0 0x7f35ac0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.750+0000 7f35b2bb4640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35ac15e4e0 0x7f35ac1f3f70 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.750+0000 7f35b2bb4640 1 --2- 172.21.15.29:0/2169087304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35ac0d4bf0 0x7f35ac15a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.750+0000 7f35b2bb4640 1 -- 172.21.15.29:0/2169087304 >> 172.21.15.29:0/2169087304 conn(0x7f35ac0d3fb0 msgr2=0x7f35ac0d1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:35.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.750+0000 7f35b2bb4640 1 -- 172.21.15.29:0/2169087304 shutdown_connections 2024-04-25T00:39:35.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.750+0000 7f35b2bb4640 1 -- 172.21.15.29:0/2169087304 wait complete. 2024-04-25T00:39:35.767 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:39:35.767 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:39:35.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e4846c640 1 Processor -- start 2024-04-25T00:39:35.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e4846c640 1 -- start start 2024-04-25T00:39:35.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e4846c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400b3140 0x7f1e4006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e4846c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1e400b3c30 0x7f1e400ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e4846c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e400b41e0 con 0x7f1e400b4b10 2024-04-25T00:39:35.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e4846c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1e4006d960 con 0x7f1e400b3c30 2024-04-25T00:39:35.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e4846c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4006dc40 con 0x7f1e400b3140 2024-04-25T00:39:35.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e46a09640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1e400b3c30 0x7f1e400ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e45a07640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1e400b4b10 0x7f1e400b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42236/0 (socket says 172.21.15.29:42236) 2024-04-25T00:39:35.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e45a07640 1 -- 172.21.15.29:0/3786321771 learned_addr learned my addr 172.21.15.29:0/3786321771 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:35.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e46208640 1 --2- 172.21.15.29:0/3786321771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400b3140 0x7f1e4006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e46a09640 1 -- 172.21.15.29:0/3786321771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400b3140 msgr2=0x7f1e4006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:35.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e46a09640 1 --2- 172.21.15.29:0/3786321771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400b3140 0x7f1e4006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e46a09640 1 -- 172.21.15.29:0/3786321771 >> v1:172.21.15.29:6789/0 conn(0x7f1e400b4b10 legacy=0x7f1e400b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.890+0000 7f1e46a09640 1 -- 172.21.15.29:0/3786321771 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4006df20 con 0x7f1e400b3c30 2024-04-25T00:39:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e46a09640 1 --2- 172.21.15.29:0/3786321771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e400b3c30 0x7f1e400ce240 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009d90 tx=0x7f1e3801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa0d0e08f630a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e45206640 1 -- 172.21.15.29:0/3786321771 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e380b2070 con 0x7f1e400b3c30 2024-04-25T00:39:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e45206640 1 -- 172.21.15.29:0/3786321771 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e380a5830 con 0x7f1e400b3c30 2024-04-25T00:39:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e45206640 1 -- 172.21.15.29:0/3786321771 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e380a5c00 con 0x7f1e400b3c30 2024-04-25T00:39:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 -- 172.21.15.29:0/3786321771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e400b3c30 msgr2=0x7f1e400ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 --2- 172.21.15.29:0/3786321771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e400b3c30 0x7f1e400ce240 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009d90 tx=0x7f1e3801c970 comp rx=0 tx=0).stop 2024-04-25T00:39:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 -- 172.21.15.29:0/3786321771 shutdown_connections 2024-04-25T00:39:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 --2- 172.21.15.29:0/3786321771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e400b3c30 0x7f1e400ce240 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 --2- 172.21.15.29:0/3786321771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400b3140 0x7f1e4006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 -- 172.21.15.29:0/3786321771 >> 172.21.15.29:0/3786321771 conn(0x7f1e400b1870 msgr2=0x7f1e400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 -- 172.21.15.29:0/3786321771 shutdown_connections 2024-04-25T00:39:35.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 -- 172.21.15.29:0/3786321771 wait complete. 2024-04-25T00:39:35.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 Processor -- start 2024-04-25T00:39:35.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 -- start start 2024-04-25T00:39:35.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e400b3140 0x7f1e40071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400b3c30 0x7f1e40159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e400b4b10 0x7f1e4013d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1e40168d80 con 0x7f1e400b3140 2024-04-25T00:39:35.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e40168c00 con 0x7f1e400b4b10 2024-04-25T00:39:35.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e4846c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1e40168f00 con 0x7f1e400b3c30 2024-04-25T00:39:35.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e46208640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e400b3140 0x7f1e40071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e46208640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e400b3140 0x7f1e40071e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43824/0 (socket says 172.21.15.29:43824) 2024-04-25T00:39:35.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.894+0000 7f1e46208640 1 -- 172.21.15.29:0/856556662 learned_addr learned my addr 172.21.15.29:0/856556662 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:35.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e46a09640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e400b4b10 0x7f1e4013d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e45a07640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400b3c30 0x7f1e40159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e46208640 1 -- 172.21.15.29:0/856556662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400b3c30 msgr2=0x7f1e40159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e46208640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400b3c30 0x7f1e40159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e46208640 1 -- 172.21.15.29:0/856556662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e400b4b10 msgr2=0x7f1e4013d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:35.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e46208640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e400b4b10 0x7f1e4013d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:35.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e46208640 1 -- 172.21.15.29:0/856556662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e400723c0 con 0x7f1e400b3140 2024-04-25T00:39:35.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e46208640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e400b3140 0x7f1e40071e80 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00d5f0 tx=0x7f1e3c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:35.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e36ffd640 1 -- 172.21.15.29:0/856556662 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0125f0 con 0x7f1e400b3140 2024-04-25T00:39:35.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e36ffd640 1 -- 172.21.15.29:0/856556662 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3c019070 con 0x7f1e400b3140 2024-04-25T00:39:35.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e4846c640 1 -- 172.21.15.29:0/856556662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e40070d80 con 0x7f1e400b3140 2024-04-25T00:39:35.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e36ffd640 1 -- 172.21.15.29:0/856556662 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0129b0 con 0x7f1e400b3140 2024-04-25T00:39:35.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e4846c640 1 -- 172.21.15.29:0/856556662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e40071340 con 0x7f1e400b3140 2024-04-25T00:39:35.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e4846c640 1 -- 172.21.15.29:0/856556662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e40069270 con 0x7f1e400b3140 2024-04-25T00:39:35.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e36ffd640 1 -- 172.21.15.29:0/856556662 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f1e3c012b50 con 0x7f1e400b3140 2024-04-25T00:39:35.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e36ffd640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1e24067ef0 0x7f1e2406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:35.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e36ffd640 1 -- 172.21.15.29:0/856556662 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f1e3c086c40 con 0x7f1e400b3140 2024-04-25T00:39:35.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e45a07640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1e24067ef0 0x7f1e2406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:35.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.898+0000 7f1e45a07640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1e24067ef0 0x7f1e2406a3b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1e4006ee50 tx=0x7f1e30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:35.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:35.902+0000 7f1e36ffd640 1 -- 172.21.15.29:0/856556662 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1e3c056e20 con 0x7f1e400b3140 2024-04-25T00:39:36.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.090+0000 7f1e4846c640 1 -- 172.21.15.29:0/856556662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e400ce4b0 con 0x7f1e400b3140 2024-04-25T00:39:36.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.094+0000 7f1e36ffd640 1 -- 172.21.15.29:0/856556662 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5179 (secure 0 0 0) 0x7f1e3c056720 con 0x7f1e400b3140 2024-04-25T00:39:36.092 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:36.092 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":84,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7450,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/456996219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":456996219},{"type":"v1","addr":"172.21.15.157:6833","nonce":456996219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/87071339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":87071339},{"type":"v1","addr":"172.21.15.157:6835","nonce":87071339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3920208567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3920208567},{"type":"v1","addr":"172.21.15.29:6833","nonce":3920208567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:39:08.508825+0000","modified":"2024-04-25T00:39:11.535481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9021":{"gid":9021,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/619209926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":619209926},{"type":"v1","addr":"172.21.15.29:6835","nonce":619209926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9021,"qdb_cluster":[9021]},"id":10}]} 2024-04-25T00:39:36.093 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 84 2024-04-25T00:39:36.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.094+0000 7f1e4846c640 1 -- 172.21.15.29:0/856556662 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1e24067ef0 msgr2=0x7f1e2406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.094+0000 7f1e4846c640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1e24067ef0 0x7f1e2406a3b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1e4006ee50 tx=0x7f1e30009040 comp rx=0 tx=0).stop 2024-04-25T00:39:36.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.094+0000 7f1e4846c640 1 -- 172.21.15.29:0/856556662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e400b3140 msgr2=0x7f1e40071e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.094+0000 7f1e4846c640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e400b3140 0x7f1e40071e80 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00d5f0 tx=0x7f1e3c00dab0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.094+0000 7f1e4846c640 1 -- 172.21.15.29:0/856556662 shutdown_connections 2024-04-25T00:39:36.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.094+0000 7f1e4846c640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1e24067ef0 0x7f1e2406a3b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.094+0000 7f1e4846c640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e400b4b10 0x7f1e4013d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.094+0000 7f1e4846c640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400b3c30 0x7f1e40159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.094+0000 7f1e4846c640 1 --2- 172.21.15.29:0/856556662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e400b3140 0x7f1e40071e80 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.094+0000 7f1e4846c640 1 -- 172.21.15.29:0/856556662 >> 172.21.15.29:0/856556662 conn(0x7f1e400b1870 msgr2=0x7f1e4015ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:36.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.094+0000 7f1e4846c640 1 -- 172.21.15.29:0/856556662 shutdown_connections 2024-04-25T00:39:36.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.094+0000 7f1e4846c640 1 -- 172.21.15.29:0/856556662 wait complete. 2024-04-25T00:39:36.111 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:39:36.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc305c2a640 1 Processor -- start 2024-04-25T00:39:36.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc305c2a640 1 -- start start 2024-04-25T00:39:36.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc305c2a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc3000b3140 0x7fc30006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc305c2a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc3000b4b10 0x7fc3000b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc305c2a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3000b3a60 0x7fc30015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc305c2a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc3000b41e0 con 0x7fc3000b3140 2024-04-25T00:39:36.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc305c2a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc30006d960 con 0x7fc3000b4b10 2024-04-25T00:39:36.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc305c2a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc30006dc40 con 0x7fc3000b3a60 2024-04-25T00:39:36.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc2ff7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc3000b3140 0x7fc30006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc2ff7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc3000b3140 0x7fc30006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43836/0 (socket says 172.21.15.29:43836) 2024-04-25T00:39:36.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc2ff7fe640 1 -- 172.21.15.29:0/1698451743 learned_addr learned my addr 172.21.15.29:0/1698451743 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:36.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc2fffff640 1 --2- 172.21.15.29:0/1698451743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3000b3a60 0x7fc30015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc2f6ffd640 1 --2- 172.21.15.29:0/1698451743 >> v2:172.21.15.157:3300/0 conn(0x7fc3000b4b10 0x7fc3000b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc2ff7fe640 1 -- 172.21.15.29:0/1698451743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3000b3a60 msgr2=0x7fc30015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:36.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc2ff7fe640 1 --2- 172.21.15.29:0/1698451743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3000b3a60 0x7fc30015c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc2ff7fe640 1 -- 172.21.15.29:0/1698451743 >> v2:172.21.15.157:3300/0 conn(0x7fc3000b4b10 msgr2=0x7fc3000b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:36.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc2ff7fe640 1 --2- 172.21.15.29:0/1698451743 >> v2:172.21.15.157:3300/0 conn(0x7fc3000b4b10 0x7fc3000b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.222+0000 7fc2ff7fe640 1 -- 172.21.15.29:0/1698451743 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30006df20 con 0x7fc3000b3140 2024-04-25T00:39:36.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.226+0000 7fc2ff7fe640 1 --2- 172.21.15.29:0/1698451743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3000b3140 0x7fc30006e440 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009870 tx=0x7fc2f001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fda89bc89a9f726 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:36.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.226+0000 7fc2feffd640 1 -- 172.21.15.29:0/1698451743 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00b0070 con 0x7fc3000b3140 2024-04-25T00:39:36.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.226+0000 7fc2feffd640 1 -- 172.21.15.29:0/1698451743 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f00ab600 con 0x7fc3000b3140 2024-04-25T00:39:36.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.226+0000 7fc2feffd640 1 -- 172.21.15.29:0/1698451743 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00ab9d0 con 0x7fc3000b3140 2024-04-25T00:39:36.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.226+0000 7fc305c2a640 1 -- 172.21.15.29:0/1698451743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3000b3140 msgr2=0x7fc30006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.226+0000 7fc305c2a640 1 --2- 172.21.15.29:0/1698451743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3000b3140 0x7fc30006e440 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009870 tx=0x7fc2f001c520 comp rx=0 tx=0).stop 2024-04-25T00:39:36.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.226+0000 7fc305c2a640 1 -- 172.21.15.29:0/1698451743 shutdown_connections 2024-04-25T00:39:36.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.226+0000 7fc305c2a640 1 --2- 172.21.15.29:0/1698451743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3000b3a60 0x7fc30015c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.226+0000 7fc305c2a640 1 --2- 172.21.15.29:0/1698451743 >> v2:172.21.15.157:3300/0 conn(0x7fc3000b4b10 0x7fc3000b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.226+0000 7fc305c2a640 1 --2- 172.21.15.29:0/1698451743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3000b3140 0x7fc30006e440 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.226+0000 7fc305c2a640 1 -- 172.21.15.29:0/1698451743 >> 172.21.15.29:0/1698451743 conn(0x7fc3000b1870 msgr2=0x7fc3000b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:36.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.226+0000 7fc305c2a640 1 -- 172.21.15.29:0/1698451743 shutdown_connections 2024-04-25T00:39:36.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.226+0000 7fc305c2a640 1 -- 172.21.15.29:0/1698451743 wait complete. 2024-04-25T00:39:36.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc305c2a640 1 Processor -- start 2024-04-25T00:39:36.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc305c2a640 1 -- start start 2024-04-25T00:39:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc305c2a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3000b3140 0x7fc300144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc305c2a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3000b3a60 0x7fc300137850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc305c2a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3000b4b10 0x7fc300145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc305c2a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc3000ce3c0 con 0x7fc3000b3a60 2024-04-25T00:39:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc305c2a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3000ce240 con 0x7fc3000b4b10 2024-04-25T00:39:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc305c2a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc3000ce540 con 0x7fc3000b3140 2024-04-25T00:39:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2f6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3000b3a60 0x7fc300137850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2f6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3000b3a60 0x7fc300137850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43846/0 (socket says 172.21.15.29:43846) 2024-04-25T00:39:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2f6ffd640 1 -- 172.21.15.29:0/505549456 learned_addr learned my addr 172.21.15.29:0/505549456 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2ff7fe640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3000b3140 0x7fc300144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2fffff640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3000b4b10 0x7fc300145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2f6ffd640 1 -- 172.21.15.29:0/505549456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3000b3140 msgr2=0x7fc300144260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2f6ffd640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3000b3140 0x7fc300144260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2f6ffd640 1 -- 172.21.15.29:0/505549456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3000b4b10 msgr2=0x7fc300145b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2f6ffd640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3000b4b10 0x7fc300145b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2f6ffd640 1 -- 172.21.15.29:0/505549456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc300146060 con 0x7fc3000b3a60 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2f6ffd640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3000b3a60 0x7fc300137850 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800d5f0 tx=0x7fc2e800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2fcff9640 1 -- 172.21.15.29:0/505549456 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e80125f0 con 0x7fc3000b3a60 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2fcff9640 1 -- 172.21.15.29:0/505549456 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e8019070 con 0x7fc3000b3a60 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2fcff9640 1 -- 172.21.15.29:0/505549456 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e80129b0 con 0x7fc3000b3a60 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc305c2a640 1 -- 172.21.15.29:0/505549456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc300146300 con 0x7fc3000b3a60 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc305c2a640 1 -- 172.21.15.29:0/505549456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3001468c0 con 0x7fc3000b3a60 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc305c2a640 1 -- 172.21.15.29:0/505549456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc300069270 con 0x7fc3000b3a60 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2fcff9640 1 -- 172.21.15.29:0/505549456 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fc2e8012b50 con 0x7fc3000b3a60 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2fcff9640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2e0067f20 0x7fc2e006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2fcff9640 1 -- 172.21.15.29:0/505549456 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7fc2e8086c20 con 0x7fc3000b3a60 2024-04-25T00:39:36.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.230+0000 7fc2ff7fe640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2e0067f20 0x7fc2e006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.234+0000 7fc2ff7fe640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2e0067f20 0x7fc2e006a3e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00ad000 tx=0x7fc2f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:36.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.234+0000 7fc2fcff9640 1 -- 172.21.15.29:0/505549456 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc2e8056e00 con 0x7fc3000b3a60 2024-04-25T00:39:36.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.422+0000 7fc305c2a640 1 -- 172.21.15.29:0/505549456 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc30015ad80 con 0x7fc3000b3a60 2024-04-25T00:39:36.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.422+0000 7fc2fcff9640 1 -- 172.21.15.29:0/505549456 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5179 (secure 0 0 0) 0x7fc2e8056700 con 0x7fc3000b3a60 2024-04-25T00:39:36.422 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:36.422 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":84,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7450,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/456996219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":456996219},{"type":"v1","addr":"172.21.15.157:6833","nonce":456996219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/87071339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":87071339},{"type":"v1","addr":"172.21.15.157:6835","nonce":87071339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3920208567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3920208567},{"type":"v1","addr":"172.21.15.29:6833","nonce":3920208567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:39:08.508825+0000","modified":"2024-04-25T00:39:11.535481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9021":{"gid":9021,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/619209926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":619209926},{"type":"v1","addr":"172.21.15.29:6835","nonce":619209926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9021,"qdb_cluster":[9021]},"id":10}]} 2024-04-25T00:39:36.423 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 84 2024-04-25T00:39:36.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.426+0000 7fc305c2a640 1 -- 172.21.15.29:0/505549456 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2e0067f20 msgr2=0x7fc2e006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.426+0000 7fc305c2a640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2e0067f20 0x7fc2e006a3e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00ad000 tx=0x7fc2f0008040 comp rx=0 tx=0).stop 2024-04-25T00:39:36.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.426+0000 7fc305c2a640 1 -- 172.21.15.29:0/505549456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3000b3a60 msgr2=0x7fc300137850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.426+0000 7fc305c2a640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3000b3a60 0x7fc300137850 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800d5f0 tx=0x7fc2e800dab0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.426+0000 7fc305c2a640 1 -- 172.21.15.29:0/505549456 shutdown_connections 2024-04-25T00:39:36.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.426+0000 7fc305c2a640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2e0067f20 0x7fc2e006a3e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.426+0000 7fc305c2a640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3000b4b10 0x7fc300145b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.426+0000 7fc305c2a640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3000b3a60 0x7fc300137850 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.426+0000 7fc305c2a640 1 --2- 172.21.15.29:0/505549456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3000b3140 0x7fc300144260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.426+0000 7fc305c2a640 1 -- 172.21.15.29:0/505549456 >> 172.21.15.29:0/505549456 conn(0x7fc3000b1870 msgr2=0x7fc30015b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:36.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.426+0000 7fc305c2a640 1 -- 172.21.15.29:0/505549456 shutdown_connections 2024-04-25T00:39:36.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.426+0000 7fc305c2a640 1 -- 172.21.15.29:0/505549456 wait complete. 2024-04-25T00:39:36.440 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:36.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.550+0000 7fd0352d0640 1 Processor -- start 2024-04-25T00:39:36.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.550+0000 7fd0352d0640 1 -- start start 2024-04-25T00:39:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd0352d0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd03015e8d0 0x7fd0300cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd0352d0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0300b3720 con 0x7fd0300b4780 2024-04-25T00:39:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd0352d0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0300b39c0 con 0x7fd03015f3c0 2024-04-25T00:39:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd0352d0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd0300b42f0 con 0x7fd03015e8d0 2024-04-25T00:39:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd02effd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd03015e8d0 0x7fd0300cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd0267fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd0300b4780 0x7fd03015ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42248/0 (socket says 172.21.15.29:42248) 2024-04-25T00:39:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd0267fc640 1 -- 172.21.15.29:0/3545473745 learned_addr learned my addr 172.21.15.29:0/3545473745 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd02effd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd03015e8d0 0x7fd0300cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43860/0 (socket says 172.21.15.29:43860) 2024-04-25T00:39:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd02effd640 1 -- 172.21.15.29:0/3545473745 >> v1:172.21.15.157:6789/0 conn(0x7fd03015f3c0 legacy=0x7fd03015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd02effd640 1 -- 172.21.15.29:0/3545473745 >> v1:172.21.15.29:6789/0 conn(0x7fd0300b4780 legacy=0x7fd03015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd02e7fc640 1 -- 172.21.15.29:0/3545473745 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4008292143 0 0) 0x7fd0300b3720 con 0x7fd0300b4780 2024-04-25T00:39:36.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd02effd640 1 -- 172.21.15.29:0/3545473745 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0300b3ca0 con 0x7fd03015e8d0 2024-04-25T00:39:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd02effd640 1 --2- 172.21.15.29:0/3545473745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd03015e8d0 0x7fd0300cc240 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fd02000ba60 tx=0x7fd02001eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ce3392c801e825 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd02e7fc640 1 -- 172.21.15.29:0/3545473745 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0200b4070 con 0x7fd03015e8d0 2024-04-25T00:39:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd02e7fc640 1 -- 172.21.15.29:0/3545473745 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0200a7ad0 con 0x7fd03015e8d0 2024-04-25T00:39:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd02e7fc640 1 -- 172.21.15.29:0/3545473745 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0200af440 con 0x7fd03015e8d0 2024-04-25T00:39:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd0352d0640 1 -- 172.21.15.29:0/3545473745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd03015e8d0 msgr2=0x7fd0300cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.554+0000 7fd0352d0640 1 --2- 172.21.15.29:0/3545473745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd03015e8d0 0x7fd0300cc240 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fd02000ba60 tx=0x7fd02001eb30 comp rx=0 tx=0).stop 2024-04-25T00:39:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 -- 172.21.15.29:0/3545473745 shutdown_connections 2024-04-25T00:39:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 --2- 172.21.15.29:0/3545473745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd03015e8d0 0x7fd0300cc240 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 -- 172.21.15.29:0/3545473745 >> 172.21.15.29:0/3545473745 conn(0x7fd0300b1870 msgr2=0x7fd0300b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 -- 172.21.15.29:0/3545473745 shutdown_connections 2024-04-25T00:39:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 -- 172.21.15.29:0/3545473745 wait complete. 2024-04-25T00:39:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 Processor -- start 2024-04-25T00:39:36.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 -- start start 2024-04-25T00:39:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0300b4780 0x7fd030159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd03015e8d0 0x7fd030153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd03015f3c0 0x7fd030070f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd03006d750 con 0x7fd0300b4780 2024-04-25T00:39:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd03006d5d0 con 0x7fd03015e8d0 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd03006d8d0 con 0x7fd03015f3c0 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd02effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0300b4780 0x7fd030159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd02effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0300b4780 0x7fd030159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43874/0 (socket says 172.21.15.29:43874) 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd02effd640 1 -- 172.21.15.29:0/2867042978 learned_addr learned my addr 172.21.15.29:0/2867042978 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0267fc640 1 --2- 172.21.15.29:0/2867042978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd03015e8d0 0x7fd030153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd02effd640 1 -- 172.21.15.29:0/2867042978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd03015f3c0 msgr2=0x7fd030070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd02effd640 1 --2- 172.21.15.29:0/2867042978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd03015f3c0 0x7fd030070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd02effd640 1 -- 172.21.15.29:0/2867042978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd03015e8d0 msgr2=0x7fd030153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd02effd640 1 --2- 172.21.15.29:0/2867042978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd03015e8d0 0x7fd030153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd02effd640 1 -- 172.21.15.29:0/2867042978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03006f050 con 0x7fd0300b4780 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd02effd640 1 --2- 172.21.15.29:0/2867042978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0300b4780 0x7fd030159270 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fd0200b2040 tx=0x7fd02001b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd027fff640 1 -- 172.21.15.29:0/2867042978 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0200b4040 con 0x7fd0300b4780 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd027fff640 1 -- 172.21.15.29:0/2867042978 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0200c1070 con 0x7fd0300b4780 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 -- 172.21.15.29:0/2867042978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03015b140 con 0x7fd0300b4780 2024-04-25T00:39:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd027fff640 1 -- 172.21.15.29:0/2867042978 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0200ba470 con 0x7fd0300b4780 2024-04-25T00:39:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 -- 172.21.15.29:0/2867042978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03015b700 con 0x7fd0300b4780 2024-04-25T00:39:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.558+0000 7fd0352d0640 1 -- 172.21.15.29:0/2867042978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd030069270 con 0x7fd0300b4780 2024-04-25T00:39:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.562+0000 7fd027fff640 1 -- 172.21.15.29:0/2867042978 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fd0200ba610 con 0x7fd0300b4780 2024-04-25T00:39:36.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.562+0000 7fd027fff640 1 --2- 172.21.15.29:0/2867042978 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd010067ef0 0x7fd01006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.562+0000 7fd027fff640 1 -- 172.21.15.29:0/2867042978 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7fd0201240f0 con 0x7fd0300b4780 2024-04-25T00:39:36.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.562+0000 7fd0267fc640 1 --2- 172.21.15.29:0/2867042978 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd010067ef0 0x7fd01006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.562+0000 7fd0267fc640 1 --2- 172.21.15.29:0/2867042978 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd010067ef0 0x7fd01006a3b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd0300ce920 tx=0x7fd018009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:36.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.566+0000 7fd027fff640 1 -- 172.21.15.29:0/2867042978 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd020018720 con 0x7fd0300b4780 2024-04-25T00:39:36.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.722+0000 7fd0352d0640 1 -- 172.21.15.29:0/2867042978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0300b1210 con 0x7fd0300b4780 2024-04-25T00:39:36.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.722+0000 7fd027fff640 1 -- 172.21.15.29:0/2867042978 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16680 (secure 0 0 0) 0x7fd0200f43f0 con 0x7fd0300b4780 2024-04-25T00:39:36.723 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:36.723 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":134,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:24.461420+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:06.791988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:07.503204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:36.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.726+0000 7fd0352d0640 1 -- 172.21.15.29:0/2867042978 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd010067ef0 msgr2=0x7fd01006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.726+0000 7fd0352d0640 1 --2- 172.21.15.29:0/2867042978 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd010067ef0 0x7fd01006a3b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd0300ce920 tx=0x7fd018009040 comp rx=0 tx=0).stop 2024-04-25T00:39:36.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.726+0000 7fd0352d0640 1 -- 172.21.15.29:0/2867042978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0300b4780 msgr2=0x7fd030159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.726+0000 7fd0352d0640 1 --2- 172.21.15.29:0/2867042978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0300b4780 0x7fd030159270 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fd0200b2040 tx=0x7fd02001b8d0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.726+0000 7fd0352d0640 1 -- 172.21.15.29:0/2867042978 shutdown_connections 2024-04-25T00:39:36.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.726+0000 7fd0352d0640 1 --2- 172.21.15.29:0/2867042978 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd010067ef0 0x7fd01006a3b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.726+0000 7fd0352d0640 1 --2- 172.21.15.29:0/2867042978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd03015f3c0 0x7fd030070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.726+0000 7fd0352d0640 1 --2- 172.21.15.29:0/2867042978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd03015e8d0 0x7fd030153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.726+0000 7fd0352d0640 1 --2- 172.21.15.29:0/2867042978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0300b4780 0x7fd030159270 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.726+0000 7fd0352d0640 1 -- 172.21.15.29:0/2867042978 >> 172.21.15.29:0/2867042978 conn(0x7fd0300b1870 msgr2=0x7fd0300b0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:36.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.726+0000 7fd0352d0640 1 -- 172.21.15.29:0/2867042978 shutdown_connections 2024-04-25T00:39:36.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.726+0000 7fd0352d0640 1 -- 172.21.15.29:0/2867042978 wait complete. 2024-04-25T00:39:36.741 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:39:36.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2caa899640 1 Processor -- start 2024-04-25T00:39:36.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2caa899640 1 -- start start 2024-04-25T00:39:36.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2caa899640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca415e8d0 0x7f2ca40cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2caa899640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2ca415f3c0 0x7f2ca415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2caa899640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca40b3720 con 0x7f2ca40b4780 2024-04-25T00:39:36.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2caa899640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca40b42f0 con 0x7f2ca415f3c0 2024-04-25T00:39:36.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2caa899640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca40b39c0 con 0x7f2ca415e8d0 2024-04-25T00:39:36.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2ca8e36640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2ca415f3c0 0x7f2ca415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2ca37fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2ca40b4780 0x7f2ca415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42264/0 (socket says 172.21.15.29:42264) 2024-04-25T00:39:36.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2ca37fe640 1 -- 172.21.15.29:0/876918475 learned_addr learned my addr 172.21.15.29:0/876918475 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:36.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2ca8e36640 1 -- 172.21.15.29:0/876918475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca415e8d0 msgr2=0x7f2ca40cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:36.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2ca8e36640 1 --2- 172.21.15.29:0/876918475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca415e8d0 0x7f2ca40cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2ca8e36640 1 -- 172.21.15.29:0/876918475 >> v1:172.21.15.29:6789/0 conn(0x7f2ca40b4780 legacy=0x7f2ca415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2ca8e36640 1 -- 172.21.15.29:0/876918475 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca40b3ca0 con 0x7f2ca415f3c0 2024-04-25T00:39:36.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2ca2ffd640 1 -- 172.21.15.29:0/876918475 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 713802541 0 0) 0x7f2ca40b3720 con 0x7f2ca40b4780 2024-04-25T00:39:36.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.858+0000 7f2ca8e36640 1 --2- 172.21.15.29:0/876918475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca415f3c0 0x7f2ca415c7d0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009d90 tx=0x7f2c9c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ddedfcc2f113c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:36.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca2ffd640 1 -- 172.21.15.29:0/876918475 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0b2070 con 0x7f2ca415f3c0 2024-04-25T00:39:36.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca2ffd640 1 -- 172.21.15.29:0/876918475 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c9c0a59e0 con 0x7f2ca415f3c0 2024-04-25T00:39:36.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca2ffd640 1 -- 172.21.15.29:0/876918475 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0a5db0 con 0x7f2ca415f3c0 2024-04-25T00:39:36.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 -- 172.21.15.29:0/876918475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca415f3c0 msgr2=0x7f2ca415c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 --2- 172.21.15.29:0/876918475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca415f3c0 0x7f2ca415c7d0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009d90 tx=0x7f2c9c01c940 comp rx=0 tx=0).stop 2024-04-25T00:39:36.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 -- 172.21.15.29:0/876918475 shutdown_connections 2024-04-25T00:39:36.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 --2- 172.21.15.29:0/876918475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca415f3c0 0x7f2ca415c7d0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 --2- 172.21.15.29:0/876918475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca415e8d0 0x7f2ca40cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 -- 172.21.15.29:0/876918475 >> 172.21.15.29:0/876918475 conn(0x7f2ca40b1870 msgr2=0x7f2ca40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:36.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 -- 172.21.15.29:0/876918475 shutdown_connections 2024-04-25T00:39:36.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 -- 172.21.15.29:0/876918475 wait complete. 2024-04-25T00:39:36.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 Processor -- start 2024-04-25T00:39:36.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 -- start start 2024-04-25T00:39:36.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ca40b4780 0x7f2ca41f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca415e8d0 0x7f2ca415aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca415f3c0 0x7f2ca4139d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca415d6a0 con 0x7f2ca415f3c0 2024-04-25T00:39:36.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca415d520 con 0x7f2ca40b4780 2024-04-25T00:39:36.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca4167c70 con 0x7f2ca415e8d0 2024-04-25T00:39:36.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca8e36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca415f3c0 0x7f2ca4139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca8e36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca415f3c0 0x7f2ca4139d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43894/0 (socket says 172.21.15.29:43894) 2024-04-25T00:39:36.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca8e36640 1 -- 172.21.15.29:0/39262448 learned_addr learned my addr 172.21.15.29:0/39262448 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:36.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca37fe640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca415e8d0 0x7f2ca415aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca3fff640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ca40b4780 0x7f2ca41f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca8e36640 1 -- 172.21.15.29:0/39262448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca415e8d0 msgr2=0x7f2ca415aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca8e36640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca415e8d0 0x7f2ca415aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca8e36640 1 -- 172.21.15.29:0/39262448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ca40b4780 msgr2=0x7f2ca41f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:36.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca8e36640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ca40b4780 0x7f2ca41f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:36.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca8e36640 1 -- 172.21.15.29:0/39262448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca40b1540 con 0x7f2ca415f3c0 2024-04-25T00:39:36.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca8e36640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca415f3c0 0x7f2ca4139d30 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0b0040 tx=0x7f2c9c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:36.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca0ff9640 1 -- 172.21.15.29:0/39262448 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0bd020 con 0x7f2ca415f3c0 2024-04-25T00:39:36.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca0ff9640 1 -- 172.21.15.29:0/39262448 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c9c0ad740 con 0x7f2ca415f3c0 2024-04-25T00:39:36.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 -- 172.21.15.29:0/39262448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca415afb0 con 0x7f2ca415f3c0 2024-04-25T00:39:36.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2ca0ff9640 1 -- 172.21.15.29:0/39262448 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0b2040 con 0x7f2ca415f3c0 2024-04-25T00:39:36.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.862+0000 7f2caa899640 1 -- 172.21.15.29:0/39262448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca413a270 con 0x7f2ca415f3c0 2024-04-25T00:39:36.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.866+0000 7f2caa899640 1 -- 172.21.15.29:0/39262448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca4069270 con 0x7f2ca415f3c0 2024-04-25T00:39:36.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.866+0000 7f2ca0ff9640 1 -- 172.21.15.29:0/39262448 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f2c9c0c0020 con 0x7f2ca415f3c0 2024-04-25T00:39:36.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.866+0000 7f2ca0ff9640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c90067ef0 0x7f2c9006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:36.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.866+0000 7f2ca0ff9640 1 -- 172.21.15.29:0/39262448 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5738+0+0 (secure 0 0 0) 0x7f2c9c1219d0 con 0x7f2ca415f3c0 2024-04-25T00:39:36.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.866+0000 7f2ca3fff640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c90067ef0 0x7f2c9006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:36.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.866+0000 7f2ca3fff640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c90067ef0 0x7f2c9006a3b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2c94006fd0 tx=0x7f2c94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:36.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:36.870+0000 7f2ca0ff9640 1 -- 172.21.15.29:0/39262448 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2c9c0c7030 con 0x7f2ca415f3c0 2024-04-25T00:39:37.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.066+0000 7f2caa899640 1 -- 172.21.15.29:0/39262448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2ca406e7b0 con 0x7f2ca415f3c0 2024-04-25T00:39:37.067 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-04-25T00:39:37.066+0000 7f3ac00b5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:39:37.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.070+0000 7f2ca0ff9640 1 -- 172.21.15.29:0/39262448 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v85) ==== 161+0+0 (secure 0 0 0) 0x7f2c9c016560 con 0x7f2ca415f3c0 2024-04-25T00:39:37.070 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:39:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.074+0000 7f2caa899640 1 -- 172.21.15.29:0/39262448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c90067ef0 msgr2=0x7f2c9006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.074+0000 7f2caa899640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c90067ef0 0x7f2c9006a3b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2c94006fd0 tx=0x7f2c94008040 comp rx=0 tx=0).stop 2024-04-25T00:39:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.074+0000 7f2caa899640 1 -- 172.21.15.29:0/39262448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca415f3c0 msgr2=0x7f2ca4139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.074+0000 7f2caa899640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca415f3c0 0x7f2ca4139d30 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0b0040 tx=0x7f2c9c0196e0 comp rx=0 tx=0).stop 2024-04-25T00:39:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.074+0000 7f2caa899640 1 -- 172.21.15.29:0/39262448 shutdown_connections 2024-04-25T00:39:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.074+0000 7f2caa899640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2c90067ef0 0x7f2c9006a3b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.074+0000 7f2caa899640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca415f3c0 0x7f2ca4139d30 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.074+0000 7f2caa899640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ca415e8d0 0x7f2ca415aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.074+0000 7f2caa899640 1 --2- 172.21.15.29:0/39262448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ca40b4780 0x7f2ca41f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.074+0000 7f2caa899640 1 -- 172.21.15.29:0/39262448 >> 172.21.15.29:0/39262448 conn(0x7f2ca40b1870 msgr2=0x7f2ca40b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.074+0000 7f2caa899640 1 -- 172.21.15.29:0/39262448 shutdown_connections 2024-04-25T00:39:37.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.074+0000 7f2caa899640 1 -- 172.21.15.29:0/39262448 wait complete. 2024-04-25T00:39:37.076 INFO:tasks.ceph.mds.c.smithi029.stderr: -1747> 2024-04-25T00:39:14.758+0000 7fde9074a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:39:37.091 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:39:37.109 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:39:37.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.206+0000 7ff8a9a58640 1 Processor -- start 2024-04-25T00:39:37.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.206+0000 7ff8a9a58640 1 -- start start 2024-04-25T00:39:37.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.206+0000 7ff8a9a58640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a406b210 0x7ff8a406b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:37.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.206+0000 7ff8a9a58640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff8a4162860 0x7ff8a4071e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:37.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.206+0000 7ff8a9a58640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8a406bb30 con 0x7ff8a4072390 2024-04-25T00:39:37.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.206+0000 7ff8a9a58640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff8a4163640 con 0x7ff8a4162860 2024-04-25T00:39:37.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.206+0000 7ff8a9a58640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a4163ad0 con 0x7ff8a406b210 2024-04-25T00:39:37.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.206+0000 7ff8a37fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff8a4072390 0x7ff8a40cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42280/0 (socket says 172.21.15.29:42280) 2024-04-25T00:39:37.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.206+0000 7ff8a37fe640 1 -- 172.21.15.29:0/1228355210 learned_addr learned my addr 172.21.15.29:0/1228355210 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:37.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.206+0000 7ff8a27fc640 1 --2- 172.21.15.29:0/1228355210 >> v2:172.21.15.157:3300/0 conn(0x7ff8a4162860 0x7ff8a4071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:37.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.206+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1903461705 0 0) 0x7ff8a406bb30 con 0x7ff8a4072390 2024-04-25T00:39:37.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.206+0000 7ff8a2ffd640 1 --2- 172.21.15.29:0/1228355210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a406b210 0x7ff8a406b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:37.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.206+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff890003530 con 0x7ff8a4072390 2024-04-25T00:39:37.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4231245633 0 0) 0x7ff890003530 con 0x7ff8a4072390 2024-04-25T00:39:37.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a406b210 msgr2=0x7ff8a406b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:37.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 --2- 172.21.15.29:0/1228355210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a406b210 0x7ff8a406b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:37.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 >> v2:172.21.15.157:3300/0 conn(0x7ff8a4162860 msgr2=0x7ff8a4071e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:37.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 --2- 172.21.15.29:0/1228355210 >> v2:172.21.15.157:3300/0 conn(0x7ff8a4162860 0x7ff8a4071e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:37.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a4072bc0 con 0x7ff8a4072390 2024-04-25T00:39:37.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ff89c001a00 con 0x7ff8a4072390 2024-04-25T00:39:37.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 >> v1:172.21.15.29:6789/0 conn(0x7ff8a4072390 legacy=0x7ff8a40cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:37.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 --2- 172.21.15.29:0/1228355210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a406b210 0x7ff8900039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:37.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 --2- 172.21.15.29:0/1228355210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff890003f40 0x7ff890004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:37.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 --2- 172.21.15.29:0/1228355210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff890004d50 0x7ff8900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:37.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4163640 con 0x7ff890004d50 2024-04-25T00:39:37.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4163ad0 con 0x7ff890003f40 2024-04-25T00:39:37.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff89000ba20 con 0x7ff8a406b210 2024-04-25T00:39:37.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a37fe640 1 --2- 172.21.15.29:0/1228355210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff890004d50 0x7ff8900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:37.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a2ffd640 1 --2- 172.21.15.29:0/1228355210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a406b210 0x7ff8900039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:37.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a27fc640 1 --2- 172.21.15.29:0/1228355210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff890003f40 0x7ff890004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:37.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a37fe640 1 -- 172.21.15.29:0/1228355210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a406b210 msgr2=0x7ff8900039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:37.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a37fe640 1 --2- 172.21.15.29:0/1228355210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a406b210 0x7ff8900039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:37.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a37fe640 1 -- 172.21.15.29:0/1228355210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff890003f40 msgr2=0x7ff890004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:37.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a37fe640 1 --2- 172.21.15.29:0/1228355210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff890003f40 0x7ff890004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:37.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a37fe640 1 -- 172.21.15.29:0/1228355210 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff890003530 con 0x7ff890004d50 2024-04-25T00:39:37.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a37fe640 1 --2- 172.21.15.29:0/1228355210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff890004d50 0x7ff8900051d0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00a8d0 tx=0x7ff89c01fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a4c4e0fc60b6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:37.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c00a4c0 con 0x7ff890004d50 2024-04-25T00:39:37.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff89c0ad020 con 0x7ff890004d50 2024-04-25T00:39:37.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a1ffb640 1 -- 172.21.15.29:0/1228355210 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0a9760 con 0x7ff890004d50 2024-04-25T00:39:37.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1228355210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff890004d50 msgr2=0x7ff8900051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:37.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a9a58640 1 --2- 172.21.15.29:0/1228355210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff890004d50 0x7ff8900051d0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00a8d0 tx=0x7ff89c01fb20 comp rx=0 tx=0).stop 2024-04-25T00:39:37.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a37fe640 1 -- 172.21.15.29:0/1228355210 reap_dead start 2024-04-25T00:39:37.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1228355210 shutdown_connections 2024-04-25T00:39:37.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.210+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1228355210 >> 172.21.15.29:0/1228355210 conn(0x7ff8a40d0c50 msgr2=0x7ff8a40d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:37.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1228355210 shutdown_connections 2024-04-25T00:39:37.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1228355210 wait complete. 2024-04-25T00:39:37.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a9a58640 1 Processor -- start 2024-04-25T00:39:37.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a9a58640 1 -- start start 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a9a58640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8a415b860 0x7ff8a40b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a9a58640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a40c8710 0x7ff8a40ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a9a58640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a40ca760 0x7ff8a406f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a9a58640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41632d0 con 0x7ff8a415b860 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a9a58640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4162ff0 con 0x7ff8a40c8710 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a9a58640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a4164d90 con 0x7ff8a40ca760 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8a415b860 0x7ff8a40b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8a415b860 0x7ff8a40b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43936/0 (socket says 172.21.15.29:43936) 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a2ffd640 1 -- 172.21.15.29:0/1381955537 learned_addr learned my addr 172.21.15.29:0/1381955537 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a37fe640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a40ca760 0x7ff8a406f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a2ffd640 1 -- 172.21.15.29:0/1381955537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a40ca760 msgr2=0x7ff8a406f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a2ffd640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a40ca760 0x7ff8a406f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a2ffd640 1 -- 172.21.15.29:0/1381955537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a40c8710 msgr2=0x7ff8a40ca220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:39:37.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a2ffd640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a40c8710 0x7ff8a40ca220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:37.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a2ffd640 1 -- 172.21.15.29:0/1381955537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a415c270 con 0x7ff8a415b860 2024-04-25T00:39:37.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a37fe640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a40ca760 0x7ff8a406f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:37.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a2ffd640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8a415b860 0x7ff8a40b3970 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7ff894011610 tx=0x7ff894011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:37.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff89bfff640 1 -- 172.21.15.29:0/1381955537 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89400f8f0 con 0x7ff8a415b860 2024-04-25T00:39:37.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff89bfff640 1 -- 172.21.15.29:0/1381955537 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff89400fa90 con 0x7ff8a415b860 2024-04-25T00:39:37.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1381955537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a406e420 con 0x7ff8a415b860 2024-04-25T00:39:37.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff89bfff640 1 -- 172.21.15.29:0/1381955537 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89400fe40 con 0x7ff8a415b860 2024-04-25T00:39:37.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1381955537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a406e9e0 con 0x7ff8a415b860 2024-04-25T00:39:37.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.214+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1381955537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a4069270 con 0x7ff8a415b860 2024-04-25T00:39:37.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.218+0000 7ff89bfff640 1 -- 172.21.15.29:0/1381955537 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7ff894015460 con 0x7ff8a415b860 2024-04-25T00:39:37.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.218+0000 7ff89bfff640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff884067ef0 0x7ff88406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:37.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.218+0000 7ff8a27fc640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff884067ef0 0x7ff88406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:37.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.218+0000 7ff89bfff640 1 -- 172.21.15.29:0/1381955537 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5824+0+0 (secure 0 0 0) 0x7ff894087160 con 0x7ff8a415b860 2024-04-25T00:39:37.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.218+0000 7ff8a27fc640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff884067ef0 0x7ff88406a3b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff8a415ef30 tx=0x7ff88c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:37.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.222+0000 7ff89bfff640 1 -- 172.21.15.29:0/1381955537 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff894019030 con 0x7ff8a415b860 2024-04-25T00:39:37.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:37.422+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1381955537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff8a40cd0a0 con 0x7ff8a415b860 2024-04-25T00:39:38.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.082+0000 7ff89bfff640 1 -- 172.21.15.29:0/1381955537 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v86) ==== 104+0+0 (secure 0 0 0) 0x7ff894057380 con 0x7ff8a415b860 2024-04-25T00:39:38.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.082+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1381955537 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff884067ef0 msgr2=0x7ff88406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:38.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.082+0000 7ff8a9a58640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff884067ef0 0x7ff88406a3b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff8a415ef30 tx=0x7ff88c008040 comp rx=0 tx=0).stop 2024-04-25T00:39:38.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.082+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1381955537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8a415b860 msgr2=0x7ff8a40b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:38.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.086+0000 7ff8a9a58640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8a415b860 0x7ff8a40b3970 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7ff894011610 tx=0x7ff894011ad0 comp rx=0 tx=0).stop 2024-04-25T00:39:38.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.086+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1381955537 shutdown_connections 2024-04-25T00:39:38.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.086+0000 7ff8a9a58640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff884067ef0 0x7ff88406a3b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:38.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.086+0000 7ff8a9a58640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a40ca760 0x7ff8a406f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:38.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.086+0000 7ff8a9a58640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a40c8710 0x7ff8a40ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:38.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.086+0000 7ff8a9a58640 1 --2- 172.21.15.29:0/1381955537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8a415b860 0x7ff8a40b3970 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:38.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.086+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1381955537 >> 172.21.15.29:0/1381955537 conn(0x7ff8a40d0c50 msgr2=0x7ff8a406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:38.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.086+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1381955537 shutdown_connections 2024-04-25T00:39:38.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.086+0000 7ff8a9a58640 1 -- 172.21.15.29:0/1381955537 wait complete. 2024-04-25T00:39:38.104 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:39:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1d2b97640 1 Processor -- start 2024-04-25T00:39:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1d2b97640 1 -- start start 2024-04-25T00:39:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1d2b97640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1cc0b29c0 0x7fa1cc0d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1d2b97640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1cc0cc9d0 con 0x7fa1cc0d21d0 2024-04-25T00:39:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1d2b97640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1cc0b3050 con 0x7fa1cc15c100 2024-04-25T00:39:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1d2b97640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa1cc0b2e20 con 0x7fa1cc0b29c0 2024-04-25T00:39:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1d1134640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa1cc0d21d0 0x7fa1cc0cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52482/0 (socket says 172.21.15.29:52482) 2024-04-25T00:39:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1d1134640 1 -- 172.21.15.29:0/1989129313 learned_addr learned my addr 172.21.15.29:0/1989129313 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cbfff640 1 --2- 172.21.15.29:0/1989129313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1cc0b29c0 0x7fa1cc0d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1840140451 0 0) 0x7fa1cc0cc9d0 con 0x7fa1cc0d21d0 2024-04-25T00:39:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1b4003530 con 0x7fa1cc0d21d0 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3832437455 0 0) 0x7fa1b4003530 con 0x7fa1cc0d21d0 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1cc0b29c0 msgr2=0x7fa1cc0d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 --2- 172.21.15.29:0/1989129313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1cc0b29c0 0x7fa1cc0d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 >> v1:172.21.15.157:6789/0 conn(0x7fa1cc15c100 legacy=0x7fa1cc0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1cc0b3330 con 0x7fa1cc0d21d0 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fa1c4001a00 con 0x7fa1cc0d21d0 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 >> v1:172.21.15.29:6789/0 conn(0x7fa1cc0d21d0 legacy=0x7fa1cc0cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 --2- 172.21.15.29:0/1989129313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1cc0b29c0 0x7fa1b40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 --2- 172.21.15.29:0/1989129313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b4003f70 0x7fa1b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 --2- 172.21.15.29:0/1989129313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b4004d50 0x7fa1b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b400bb60 con 0x7fa1b4003f70 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b400bce0 con 0x7fa1b4004d50 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b400bee0 con 0x7fa1cc0b29c0 2024-04-25T00:39:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cbfff640 1 --2- 172.21.15.29:0/1989129313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b4003f70 0x7fa1b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1d0933640 1 --2- 172.21.15.29:0/1989129313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1cc0b29c0 0x7fa1b40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1d1134640 1 --2- 172.21.15.29:0/1989129313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b4004d50 0x7fa1b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cbfff640 1 -- 172.21.15.29:0/1989129313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1cc0b29c0 msgr2=0x7fa1b40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cbfff640 1 --2- 172.21.15.29:0/1989129313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1cc0b29c0 0x7fa1b40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cbfff640 1 -- 172.21.15.29:0/1989129313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b4004d50 msgr2=0x7fa1b40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cbfff640 1 --2- 172.21.15.29:0/1989129313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b4004d50 0x7fa1b40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1cbfff640 1 -- 172.21.15.29:0/1989129313 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b4003530 con 0x7fa1b4003f70 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.286+0000 7fa1d0933640 1 --2- 172.21.15.29:0/1989129313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1cc0b29c0 0x7fa1b40039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.290+0000 7fa1cbfff640 1 --2- 172.21.15.29:0/1989129313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b4003f70 0x7fa1b4004810 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009ae0 tx=0x7fa1c001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfeb59c268f94bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.290+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00096b0 con 0x7fa1b4003f70 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.290+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c00a6450 con 0x7fa1b4003f70 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.290+0000 7fa1cb7fe640 1 -- 172.21.15.29:0/1989129313 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00a6840 con 0x7fa1b4003f70 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.290+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1989129313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b4003f70 msgr2=0x7fa1b4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.290+0000 7fa1d2b97640 1 --2- 172.21.15.29:0/1989129313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b4003f70 0x7fa1b4004810 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009ae0 tx=0x7fa1c001ca70 comp rx=0 tx=0).stop 2024-04-25T00:39:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.290+0000 7fa1d1134640 1 -- 172.21.15.29:0/1989129313 reap_dead start 2024-04-25T00:39:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.290+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1989129313 shutdown_connections 2024-04-25T00:39:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.290+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1989129313 >> 172.21.15.29:0/1989129313 conn(0x7fa1cc09aa30 msgr2=0x7fa1cc09ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.290+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1989129313 shutdown_connections 2024-04-25T00:39:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.290+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1989129313 wait complete. 2024-04-25T00:39:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.290+0000 7fa1d2b97640 1 Processor -- start 2024-04-25T00:39:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.290+0000 7fa1d2b97640 1 -- start start 2024-04-25T00:39:38.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1d2b97640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1b4003f70 0x7fa1cc162780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:38.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1d2b97640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1cc14f2f0 0x7fa1cc15c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:38.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1d2b97640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1cc15ed50 0x7fa1cc15f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1d2b97640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc0ce0b0 con 0x7fa1cc14f2f0 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1d2b97640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc0ccd00 con 0x7fa1cc15ed50 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1d2b97640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa1cc0ce230 con 0x7fa1b4003f70 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1cbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1cc14f2f0 0x7fa1cc15c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1cbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1cc14f2f0 0x7fa1cc15c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37042/0 (socket says 172.21.15.29:37042) 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1cbfff640 1 -- 172.21.15.29:0/1939860986 learned_addr learned my addr 172.21.15.29:0/1939860986 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1d0933640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1b4003f70 0x7fa1cc162780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1d1134640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1cc15ed50 0x7fa1cc15f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1cbfff640 1 -- 172.21.15.29:0/1939860986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1b4003f70 msgr2=0x7fa1cc162780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1cbfff640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1b4003f70 0x7fa1cc162780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1cbfff640 1 -- 172.21.15.29:0/1939860986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1cc15ed50 msgr2=0x7fa1cc15f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1cbfff640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1cc15ed50 0x7fa1cc15f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1cbfff640 1 -- 172.21.15.29:0/1939860986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1cc15cff0 con 0x7fa1cc14f2f0 2024-04-25T00:39:38.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1cbfff640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1cc14f2f0 0x7fa1cc15c6f0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fa1c001cf50 tx=0x7fa1b4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1c97fa640 1 -- 172.21.15.29:0/1939860986 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00be070 con 0x7fa1cc14f2f0 2024-04-25T00:39:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1c97fa640 1 -- 172.21.15.29:0/1939860986 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c000f000 con 0x7fa1cc14f2f0 2024-04-25T00:39:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1c97fa640 1 -- 172.21.15.29:0/1939860986 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00a7430 con 0x7fa1cc14f2f0 2024-04-25T00:39:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1939860986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1cc15d2d0 con 0x7fa1cc14f2f0 2024-04-25T00:39:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1939860986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1cc15cc30 con 0x7fa1cc14f2f0 2024-04-25T00:39:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1939860986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1cc0692e0 con 0x7fa1cc14f2f0 2024-04-25T00:39:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1c97fa640 1 -- 172.21.15.29:0/1939860986 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fa1c00c1070 con 0x7fa1cc14f2f0 2024-04-25T00:39:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1c97fa640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1b8067f20 0x7fa1b806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1d0933640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1b8067f20 0x7fa1b806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1c97fa640 1 -- 172.21.15.29:0/1939860986 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5824+0+0 (secure 0 0 0) 0x7fa1c0120340 con 0x7fa1cc14f2f0 2024-04-25T00:39:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.294+0000 7fa1d0933640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1b8067f20 0x7fa1b806a3e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc000c10 tx=0x7fa1bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:38.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.298+0000 7fa1c97fa640 1 -- 172.21.15.29:0/1939860986 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa1c00ac020 con 0x7fa1cc14f2f0 2024-04-25T00:39:38.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:38.462+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1939860986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa1cc15ac40 con 0x7fa1cc14f2f0 2024-04-25T00:39:39.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.082+0000 7fa1c97fa640 1 -- 172.21.15.29:0/1939860986 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v136) ==== 181+0+0 (secure 0 0 0) 0x7fa1c0016780 con 0x7fa1cc14f2f0 2024-04-25T00:39:39.083 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:39:39.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.090+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1939860986 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1b8067f20 msgr2=0x7fa1b806a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.090+0000 7fa1d2b97640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1b8067f20 0x7fa1b806a3e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc000c10 tx=0x7fa1bc009040 comp rx=0 tx=0).stop 2024-04-25T00:39:39.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.090+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1939860986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1cc14f2f0 msgr2=0x7fa1cc15c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.090+0000 7fa1d2b97640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1cc14f2f0 0x7fa1cc15c6f0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fa1c001cf50 tx=0x7fa1b4005840 comp rx=0 tx=0).stop 2024-04-25T00:39:39.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.090+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1939860986 shutdown_connections 2024-04-25T00:39:39.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.090+0000 7fa1d2b97640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1b8067f20 0x7fa1b806a3e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.090+0000 7fa1d2b97640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1cc15ed50 0x7fa1cc15f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.090+0000 7fa1d2b97640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1cc14f2f0 0x7fa1cc15c6f0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.090+0000 7fa1d2b97640 1 --2- 172.21.15.29:0/1939860986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1b4003f70 0x7fa1cc162780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.090+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1939860986 >> 172.21.15.29:0/1939860986 conn(0x7fa1cc09aa30 msgr2=0x7fa1cc15b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:39.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.090+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1939860986 shutdown_connections 2024-04-25T00:39:39.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.090+0000 7fa1d2b97640 1 -- 172.21.15.29:0/1939860986 wait complete. 2024-04-25T00:39:39.111 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:39:39.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.238+0000 7f998388d640 1 Processor -- start 2024-04-25T00:39:39.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.238+0000 7f998388d640 1 -- start start 2024-04-25T00:39:39.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.238+0000 7f998388d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f997c0b4590 0x7f997c070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.238+0000 7f998388d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f997c0cd1c0 0x7f997c072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.238+0000 7f998388d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f997c0712c0 con 0x7f997c15e9a0 2024-04-25T00:39:39.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.238+0000 7f998388d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f997c071ed0 con 0x7f997c0cd1c0 2024-04-25T00:39:39.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.238+0000 7f998388d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f997c0715a0 con 0x7f997c0b4590 2024-04-25T00:39:39.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9981629640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f997c15e9a0 0x7f997c15ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52490/0 (socket says 172.21.15.29:52490) 2024-04-25T00:39:39.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9981629640 1 -- 172.21.15.29:0/677476577 learned_addr learned my addr 172.21.15.29:0/677476577 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:39.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9981e2a640 1 --2- 172.21.15.29:0/677476577 >> v2:172.21.15.29:3300/0 conn(0x7f997c0cd1c0 0x7f997c072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9980e28640 1 --2- 172.21.15.29:0/677476577 >> v2:172.21.15.157:3300/0 conn(0x7f997c0b4590 0x7f997c070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9973fff640 1 -- 172.21.15.29:0/677476577 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4275488139 0 0) 0x7f997c0712c0 con 0x7f997c15e9a0 2024-04-25T00:39:39.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9973fff640 1 -- 172.21.15.29:0/677476577 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9968003530 con 0x7f997c15e9a0 2024-04-25T00:39:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9981e2a640 1 -- 172.21.15.29:0/677476577 >> v2:172.21.15.157:3300/0 conn(0x7f997c0b4590 msgr2=0x7f997c070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9981e2a640 1 --2- 172.21.15.29:0/677476577 >> v2:172.21.15.157:3300/0 conn(0x7f997c0b4590 0x7f997c070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9981e2a640 1 -- 172.21.15.29:0/677476577 >> v1:172.21.15.29:6789/0 conn(0x7f997c15e9a0 legacy=0x7f997c15ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9981e2a640 1 -- 172.21.15.29:0/677476577 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c071880 con 0x7f997c0cd1c0 2024-04-25T00:39:39.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9981e2a640 1 --2- 172.21.15.29:0/677476577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c0cd1c0 0x7f997c072360 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f997c067fa0 tx=0x7f997401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56639c1571ab7ded server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:39.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9973fff640 1 -- 172.21.15.29:0/677476577 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99740b2070 con 0x7f997c0cd1c0 2024-04-25T00:39:39.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9973fff640 1 -- 172.21.15.29:0/677476577 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99740a5950 con 0x7f997c0cd1c0 2024-04-25T00:39:39.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f9973fff640 1 -- 172.21.15.29:0/677476577 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99740a5d20 con 0x7f997c0cd1c0 2024-04-25T00:39:39.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f998388d640 1 -- 172.21.15.29:0/677476577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c0cd1c0 msgr2=0x7f997c072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f998388d640 1 --2- 172.21.15.29:0/677476577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c0cd1c0 0x7f997c072360 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f997c067fa0 tx=0x7f997401c870 comp rx=0 tx=0).stop 2024-04-25T00:39:39.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f998388d640 1 -- 172.21.15.29:0/677476577 shutdown_connections 2024-04-25T00:39:39.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f998388d640 1 --2- 172.21.15.29:0/677476577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c0cd1c0 0x7f997c072360 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f998388d640 1 --2- 172.21.15.29:0/677476577 >> v2:172.21.15.157:3300/0 conn(0x7f997c0b4590 0x7f997c070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f998388d640 1 -- 172.21.15.29:0/677476577 >> 172.21.15.29:0/677476577 conn(0x7f997c0b1870 msgr2=0x7f997c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:39.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f998388d640 1 -- 172.21.15.29:0/677476577 shutdown_connections 2024-04-25T00:39:39.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.242+0000 7f998388d640 1 -- 172.21.15.29:0/677476577 wait complete. 2024-04-25T00:39:39.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f998388d640 1 Processor -- start 2024-04-25T00:39:39.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f998388d640 1 -- start start 2024-04-25T00:39:39.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f998388d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c0b4590 0x7f997c1528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f998388d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c0cd1c0 0x7f997c13f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f998388d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c15e9a0 0x7f997c143490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f998388d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1449f0 con 0x7f997c0cd1c0 2024-04-25T00:39:39.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f998388d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f997c144870 con 0x7f997c0b4590 2024-04-25T00:39:39.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f998388d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f997c165d30 con 0x7f997c15e9a0 2024-04-25T00:39:39.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9980e28640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c0cd1c0 0x7f997c13f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9980e28640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c0cd1c0 0x7f997c13f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37058/0 (socket says 172.21.15.29:37058) 2024-04-25T00:39:39.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9980e28640 1 -- 172.21.15.29:0/3195601524 learned_addr learned my addr 172.21.15.29:0/3195601524 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:39.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9981629640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c0b4590 0x7f997c1528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9981e2a640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c15e9a0 0x7f997c143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9980e28640 1 -- 172.21.15.29:0/3195601524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c15e9a0 msgr2=0x7f997c143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9980e28640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c15e9a0 0x7f997c143490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9980e28640 1 -- 172.21.15.29:0/3195601524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c0b4590 msgr2=0x7f997c1528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9980e28640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c0b4590 0x7f997c1528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9980e28640 1 -- 172.21.15.29:0/3195601524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c140410 con 0x7f997c0cd1c0 2024-04-25T00:39:39.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9981629640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c0b4590 0x7f997c1528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:39.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9980e28640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c0cd1c0 0x7f997c13f960 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f996c00d800 tx=0x7f996c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:39.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9971ffb640 1 -- 172.21.15.29:0/3195601524 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c00b6c0 con 0x7f997c0cd1c0 2024-04-25T00:39:39.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9971ffb640 1 -- 172.21.15.29:0/3195601524 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996c019070 con 0x7f997c0cd1c0 2024-04-25T00:39:39.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f998388d640 1 -- 172.21.15.29:0/3195601524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997c141aa0 con 0x7f997c0cd1c0 2024-04-25T00:39:39.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f9971ffb640 1 -- 172.21.15.29:0/3195601524 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c00ba60 con 0x7f997c0cd1c0 2024-04-25T00:39:39.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f998388d640 1 -- 172.21.15.29:0/3195601524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997c1542f0 con 0x7f997c0cd1c0 2024-04-25T00:39:39.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.246+0000 7f998388d640 1 -- 172.21.15.29:0/3195601524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f997c0692e0 con 0x7f997c0cd1c0 2024-04-25T00:39:39.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.250+0000 7f9971ffb640 1 -- 172.21.15.29:0/3195601524 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f996c00bc00 con 0x7f997c0cd1c0 2024-04-25T00:39:39.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.250+0000 7f9971ffb640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9960067ef0 0x7f996006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.250+0000 7f9981629640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9960067ef0 0x7f996006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.250+0000 7f9971ffb640 1 -- 172.21.15.29:0/3195601524 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5386+0+0 (secure 0 0 0) 0x7f996c086af0 con 0x7f997c0cd1c0 2024-04-25T00:39:39.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.250+0000 7f9981629640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9960067ef0 0x7f996006a3b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9978005890 tx=0x7f9978009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:39.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.254+0000 7f9971ffb640 1 -- 172.21.15.29:0/3195601524 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f996c056ec0 con 0x7f997c0cd1c0 2024-04-25T00:39:39.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:39.426+0000 7f998388d640 1 -- 172.21.15.29:0/3195601524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f997c06cff0 con 0x7f997c0cd1c0 2024-04-25T00:39:40.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.086+0000 7f9971ffb640 1 -- 172.21.15.29:0/3195601524 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v137) ==== 169+0+0 (secure 0 0 0) 0x7f996c0567c0 con 0x7f997c0cd1c0 2024-04-25T00:39:40.086 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-04-25T00:39:40.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.090+0000 7f998388d640 1 -- 172.21.15.29:0/3195601524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9960067ef0 msgr2=0x7f996006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:40.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.090+0000 7f998388d640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9960067ef0 0x7f996006a3b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9978005890 tx=0x7f9978009040 comp rx=0 tx=0).stop 2024-04-25T00:39:40.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.090+0000 7f998388d640 1 -- 172.21.15.29:0/3195601524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c0cd1c0 msgr2=0x7f997c13f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:40.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.090+0000 7f998388d640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c0cd1c0 0x7f997c13f960 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f996c00d800 tx=0x7f996c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.090+0000 7f998388d640 1 -- 172.21.15.29:0/3195601524 shutdown_connections 2024-04-25T00:39:40.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.090+0000 7f998388d640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9960067ef0 0x7f996006a3b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.090+0000 7f998388d640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c15e9a0 0x7f997c143490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.090+0000 7f998388d640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f997c0cd1c0 0x7f997c13f960 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.090+0000 7f998388d640 1 --2- 172.21.15.29:0/3195601524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c0b4590 0x7f997c1528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.090+0000 7f998388d640 1 -- 172.21.15.29:0/3195601524 >> 172.21.15.29:0/3195601524 conn(0x7f997c0b1870 msgr2=0x7f997c0cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:40.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.090+0000 7f998388d640 1 -- 172.21.15.29:0/3195601524 shutdown_connections 2024-04-25T00:39:40.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.090+0000 7f998388d640 1 -- 172.21.15.29:0/3195601524 wait complete. 2024-04-25T00:39:40.112 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:39:40.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a3a6b8640 1 Processor -- start 2024-04-25T00:39:40.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a3a6b8640 1 -- start start 2024-04-25T00:39:40.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a3a6b8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8a3415df40 0x7f8a3406c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:40.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a3a6b8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8a3406d440 0x7f8a340cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:40.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a3a6b8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a340d0100 con 0x7f8a3415e320 2024-04-25T00:39:40.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a3a6b8640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8a340d03a0 con 0x7f8a3415df40 2024-04-25T00:39:40.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a3a6b8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8a340d0680 con 0x7f8a3406d440 2024-04-25T00:39:40.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a33fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8a3415df40 0x7f8a3406c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:40.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a33fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8a3415df40 0x7f8a3406c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37064/0 (socket says 172.21.15.29:37064) 2024-04-25T00:39:40.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a33fff640 1 -- 172.21.15.29:0/827748400 learned_addr learned my addr 172.21.15.29:0/827748400 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:40.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a38c55640 1 --2- 172.21.15.29:0/827748400 >> v2:172.21.15.157:3300/0 conn(0x7f8a3406d440 0x7f8a340cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:40.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a33fff640 1 -- 172.21.15.29:0/827748400 >> v2:172.21.15.157:3300/0 conn(0x7f8a3406d440 msgr2=0x7f8a340cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:40.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a33fff640 1 --2- 172.21.15.29:0/827748400 >> v2:172.21.15.157:3300/0 conn(0x7f8a3406d440 0x7f8a340cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a33fff640 1 -- 172.21.15.29:0/827748400 >> v1:172.21.15.157:6789/0 conn(0x7f8a3415e320 legacy=0x7f8a3406cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:40.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a33fff640 1 -- 172.21.15.29:0/827748400 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a340d0960 con 0x7f8a3415df40 2024-04-25T00:39:40.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.242+0000 7f8a38c55640 1 --2- 172.21.15.29:0/827748400 >> v2:172.21.15.157:3300/0 conn(0x7f8a3406d440 0x7f8a340cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:40.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a33fff640 1 --2- 172.21.15.29:0/827748400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a3415df40 0x7f8a3406c320 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f8a340720a0 tx=0x7f8a1c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87854cb362ca5cf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:40.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a337fe640 1 -- 172.21.15.29:0/827748400 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0aa430 con 0x7f8a3415df40 2024-04-25T00:39:40.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a337fe640 1 -- 172.21.15.29:0/827748400 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c0af070 con 0x7f8a3415df40 2024-04-25T00:39:40.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a337fe640 1 -- 172.21.15.29:0/827748400 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0aa7f0 con 0x7f8a3415df40 2024-04-25T00:39:40.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/827748400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a3415df40 msgr2=0x7f8a3406c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:40.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a3a6b8640 1 --2- 172.21.15.29:0/827748400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a3415df40 0x7f8a3406c320 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f8a340720a0 tx=0x7f8a1c01c4b0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/827748400 shutdown_connections 2024-04-25T00:39:40.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a3a6b8640 1 --2- 172.21.15.29:0/827748400 >> v2:172.21.15.157:3300/0 conn(0x7f8a3406d440 0x7f8a340cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a3a6b8640 1 --2- 172.21.15.29:0/827748400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a3415df40 0x7f8a3406c320 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/827748400 >> 172.21.15.29:0/827748400 conn(0x7f8a340d1d20 msgr2=0x7f8a340d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:40.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/827748400 shutdown_connections 2024-04-25T00:39:40.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/827748400 wait complete. 2024-04-25T00:39:40.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a3a6b8640 1 Processor -- start 2024-04-25T00:39:40.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.246+0000 7f8a3a6b8640 1 -- start start 2024-04-25T00:39:40.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a3a6b8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a3406d440 0x7f8a34154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:40.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a3a6b8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a3415df40 0x7f8a34157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:40.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a3a6b8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a3415e320 0x7f8a34161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:40.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a3a6b8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a340ce970 con 0x7f8a3406d440 2024-04-25T00:39:40.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a3a6b8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8a340ce7f0 con 0x7f8a3415df40 2024-04-25T00:39:40.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a3a6b8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8a340ceaf0 con 0x7f8a3415e320 2024-04-25T00:39:40.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a3406d440 0x7f8a34154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:40.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a3406d440 0x7f8a34154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37070/0 (socket says 172.21.15.29:37070) 2024-04-25T00:39:40.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a33fff640 1 -- 172.21.15.29:0/4080108842 learned_addr learned my addr 172.21.15.29:0/4080108842 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:40.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a2bfff640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a3415df40 0x7f8a34157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:40.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a38c55640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a3415e320 0x7f8a34161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:40.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a33fff640 1 -- 172.21.15.29:0/4080108842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a3415e320 msgr2=0x7f8a34161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:40.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a33fff640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a3415e320 0x7f8a34161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a33fff640 1 -- 172.21.15.29:0/4080108842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a3415df40 msgr2=0x7f8a34157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:40.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a33fff640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a3415df40 0x7f8a34157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a33fff640 1 -- 172.21.15.29:0/4080108842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a34158410 con 0x7f8a3406d440 2024-04-25T00:39:40.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a2bfff640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a3415df40 0x7f8a34157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:40.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a33fff640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a3406d440 0x7f8a34154b00 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c01a0e0 tx=0x7f8a1c01acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:40.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a317fa640 1 -- 172.21.15.29:0/4080108842 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0bd3f0 con 0x7f8a3406d440 2024-04-25T00:39:40.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a317fa640 1 -- 172.21.15.29:0/4080108842 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c0bd590 con 0x7f8a3406d440 2024-04-25T00:39:40.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/4080108842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a34155040 con 0x7f8a3406d440 2024-04-25T00:39:40.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a317fa640 1 -- 172.21.15.29:0/4080108842 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0bd960 con 0x7f8a3406d440 2024-04-25T00:39:40.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/4080108842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a341601b0 con 0x7f8a3406d440 2024-04-25T00:39:40.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/4080108842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a34069270 con 0x7f8a3406d440 2024-04-25T00:39:40.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a317fa640 1 -- 172.21.15.29:0/4080108842 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f8a1c0b33f0 con 0x7f8a3406d440 2024-04-25T00:39:40.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a317fa640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8a14067ef0 0x7f8a1406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:40.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a317fa640 1 -- 172.21.15.29:0/4080108842 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7f8a1c1207e0 con 0x7f8a3406d440 2024-04-25T00:39:40.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a2bfff640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8a14067ef0 0x7f8a1406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:40.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.250+0000 7f8a2bfff640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8a14067ef0 0x7f8a1406a3b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8a3415d6f0 tx=0x7f8a24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:40.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.254+0000 7f8a317fa640 1 -- 172.21.15.29:0/4080108842 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8a1c0c3060 con 0x7f8a3406d440 2024-04-25T00:39:40.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.426+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/4080108842 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f8a34073260 con 0x7f8a3406d440 2024-04-25T00:39:40.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.434+0000 7f8a317fa640 1 -- 172.21.15.29:0/4080108842 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f8a1c0160d0 con 0x7f8a3406d440 2024-04-25T00:39:40.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.434+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/4080108842 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8a14067ef0 msgr2=0x7f8a1406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:40.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.434+0000 7f8a3a6b8640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8a14067ef0 0x7f8a1406a3b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8a3415d6f0 tx=0x7f8a24008040 comp rx=0 tx=0).stop 2024-04-25T00:39:40.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.434+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/4080108842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a3406d440 msgr2=0x7f8a34154b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:40.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.434+0000 7f8a3a6b8640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a3406d440 0x7f8a34154b00 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c01a0e0 tx=0x7f8a1c01acf0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.434+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/4080108842 shutdown_connections 2024-04-25T00:39:40.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.434+0000 7f8a3a6b8640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8a14067ef0 0x7f8a1406a3b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.434+0000 7f8a3a6b8640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a3415e320 0x7f8a34161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.434+0000 7f8a3a6b8640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a3415df40 0x7f8a34157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.434+0000 7f8a3a6b8640 1 --2- 172.21.15.29:0/4080108842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a3406d440 0x7f8a34154b00 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.434+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/4080108842 >> 172.21.15.29:0/4080108842 conn(0x7f8a340d1d20 msgr2=0x7f8a34066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:40.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.434+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/4080108842 shutdown_connections 2024-04-25T00:39:40.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.434+0000 7f8a3a6b8640 1 -- 172.21.15.29:0/4080108842 wait complete. 2024-04-25T00:39:40.452 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-04-25T00:39:40.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5067da5640 1 Processor -- start 2024-04-25T00:39:40.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5067da5640 1 -- start start 2024-04-25T00:39:40.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5067da5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f506015e8d0 0x7f50600cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:40.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5067da5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50600b4780 0x7f506015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:40.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5067da5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50600b3720 con 0x7f506015f1f0 2024-04-25T00:39:40.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5067da5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f50600b42f0 con 0x7f506015e8d0 2024-04-25T00:39:40.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5067da5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f506015d7e0 con 0x7f50600b4780 2024-04-25T00:39:40.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5065b41640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f506015e8d0 0x7f50600cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:40.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5066342640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f506015f1f0 0x7f506015b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52504/0 (socket says 172.21.15.29:52504) 2024-04-25T00:39:40.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5066342640 1 -- 172.21.15.29:0/3902124854 learned_addr learned my addr 172.21.15.29:0/3902124854 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:40.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5065b41640 1 -- 172.21.15.29:0/3902124854 >> v2:172.21.15.157:3300/0 conn(0x7f50600b4780 msgr2=0x7f506015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:40.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5065b41640 1 --2- 172.21.15.29:0/3902124854 >> v2:172.21.15.157:3300/0 conn(0x7f50600b4780 0x7f506015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5065b41640 1 -- 172.21.15.29:0/3902124854 >> v1:172.21.15.29:6789/0 conn(0x7f506015f1f0 legacy=0x7f506015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:40.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5064b3f640 1 -- 172.21.15.29:0/3902124854 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2834635017 0 0) 0x7f50600b3720 con 0x7f506015f1f0 2024-04-25T00:39:40.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.578+0000 7f5065b41640 1 -- 172.21.15.29:0/3902124854 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50600b39c0 con 0x7f506015e8d0 2024-04-25T00:39:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5065b41640 1 --2- 172.21.15.29:0/3902124854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f506015e8d0 0x7f50600cc240 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f5060070910 tx=0x7f505c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e452e41deab4c55d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5064b3f640 1 -- 172.21.15.29:0/3902124854 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506015c1c0 con 0x7f506015e8d0 2024-04-25T00:39:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5064b3f640 1 -- 172.21.15.29:0/3902124854 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f505c0b0070 con 0x7f506015e8d0 2024-04-25T00:39:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5064b3f640 1 -- 172.21.15.29:0/3902124854 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c0ab870 con 0x7f506015e8d0 2024-04-25T00:39:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 -- 172.21.15.29:0/3902124854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f506015e8d0 msgr2=0x7f50600cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 --2- 172.21.15.29:0/3902124854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f506015e8d0 0x7f50600cc240 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f5060070910 tx=0x7f505c01c450 comp rx=0 tx=0).stop 2024-04-25T00:39:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 -- 172.21.15.29:0/3902124854 shutdown_connections 2024-04-25T00:39:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 --2- 172.21.15.29:0/3902124854 >> v2:172.21.15.157:3300/0 conn(0x7f50600b4780 0x7f506015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 --2- 172.21.15.29:0/3902124854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f506015e8d0 0x7f50600cc240 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 -- 172.21.15.29:0/3902124854 >> 172.21.15.29:0/3902124854 conn(0x7f50600b1870 msgr2=0x7f50600b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:40.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 -- 172.21.15.29:0/3902124854 shutdown_connections 2024-04-25T00:39:40.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 -- 172.21.15.29:0/3902124854 wait complete. 2024-04-25T00:39:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 Processor -- start 2024-04-25T00:39:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 -- start start 2024-04-25T00:39:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50600b4780 0x7f50600aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f506015f1f0 0x7f506013a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5060070ce0 0x7f506015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5060144a20 con 0x7f50600b4780 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f506015f600 con 0x7f5060070ce0 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5067da5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5060071dd0 con 0x7f506015f1f0 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.582+0000 7f5065b41640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50600b4780 0x7f50600aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5066342640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5060070ce0 0x7f506015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5065b41640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50600b4780 0x7f50600aefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37084/0 (socket says 172.21.15.29:37084) 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5065b41640 1 -- 172.21.15.29:0/467801742 learned_addr learned my addr 172.21.15.29:0/467801742 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5065340640 1 --2- 172.21.15.29:0/467801742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f506015f1f0 0x7f506013a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5065b41640 1 -- 172.21.15.29:0/467801742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f506015f1f0 msgr2=0x7f506013a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5065b41640 1 --2- 172.21.15.29:0/467801742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f506015f1f0 0x7f506013a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5065b41640 1 -- 172.21.15.29:0/467801742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5060070ce0 msgr2=0x7f506015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5065b41640 1 --2- 172.21.15.29:0/467801742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5060070ce0 0x7f506015a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5065b41640 1 -- 172.21.15.29:0/467801742 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50600711c0 con 0x7f50600b4780 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5065b41640 1 --2- 172.21.15.29:0/467801742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50600b4780 0x7f50600aefa0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f505c000c10 tx=0x7f505c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f50567fc640 1 -- 172.21.15.29:0/467801742 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c006e20 con 0x7f50600b4780 2024-04-25T00:39:40.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f50567fc640 1 -- 172.21.15.29:0/467801742 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f505c004220 con 0x7f50600b4780 2024-04-25T00:39:40.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f50567fc640 1 -- 172.21.15.29:0/467801742 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c003620 con 0x7f50600b4780 2024-04-25T00:39:40.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5067da5640 1 -- 172.21.15.29:0/467801742 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50600714a0 con 0x7f50600b4780 2024-04-25T00:39:40.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5067da5640 1 -- 172.21.15.29:0/467801742 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5060071a60 con 0x7f50600b4780 2024-04-25T00:39:40.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5067da5640 1 -- 172.21.15.29:0/467801742 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50600af4e0 con 0x7f50600b4780 2024-04-25T00:39:40.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f50567fc640 1 -- 172.21.15.29:0/467801742 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f505c0037c0 con 0x7f50600b4780 2024-04-25T00:39:40.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f50567fc640 1 --2- 172.21.15.29:0/467801742 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5044067f20 0x7f504406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:40.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f50567fc640 1 -- 172.21.15.29:0/467801742 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7f505c120d40 con 0x7f50600b4780 2024-04-25T00:39:40.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5065340640 1 --2- 172.21.15.29:0/467801742 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5044067f20 0x7f504406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:40.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.586+0000 7f5065340640 1 --2- 172.21.15.29:0/467801742 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5044067f20 0x7f504406a3e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f50600cf690 tx=0x7f5050009110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:40.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.590+0000 7f50567fc640 1 -- 172.21.15.29:0/467801742 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f505c016070 con 0x7f50600b4780 2024-04-25T00:39:40.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:40.814+0000 7f5067da5640 1 -- 172.21.15.29:0/467801742 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7f5060069270 con 0x7f50600b4780 2024-04-25T00:39:41.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.110+0000 7f50567fc640 1 -- 172.21.15.29:0/467801742 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v438) ==== 151+0+0 (secure 0 0 0) 0x7f505c0f7030 con 0x7f50600b4780 2024-04-25T00:39:41.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.114+0000 7f5067da5640 1 -- 172.21.15.29:0/467801742 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5044067f20 msgr2=0x7f504406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.114+0000 7f5067da5640 1 --2- 172.21.15.29:0/467801742 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5044067f20 0x7f504406a3e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f50600cf690 tx=0x7f5050009110 comp rx=0 tx=0).stop 2024-04-25T00:39:41.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.114+0000 7f5067da5640 1 -- 172.21.15.29:0/467801742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50600b4780 msgr2=0x7f50600aefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.114+0000 7f5067da5640 1 --2- 172.21.15.29:0/467801742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50600b4780 0x7f50600aefa0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f505c000c10 tx=0x7f505c01ac90 comp rx=0 tx=0).stop 2024-04-25T00:39:41.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.114+0000 7f5067da5640 1 -- 172.21.15.29:0/467801742 shutdown_connections 2024-04-25T00:39:41.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.114+0000 7f5067da5640 1 --2- 172.21.15.29:0/467801742 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5044067f20 0x7f504406a3e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.114+0000 7f5067da5640 1 --2- 172.21.15.29:0/467801742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5060070ce0 0x7f506015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.114+0000 7f5067da5640 1 --2- 172.21.15.29:0/467801742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f506015f1f0 0x7f506013a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.114+0000 7f5067da5640 1 --2- 172.21.15.29:0/467801742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50600b4780 0x7f50600aefa0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.114+0000 7f5067da5640 1 -- 172.21.15.29:0/467801742 >> 172.21.15.29:0/467801742 conn(0x7f50600b1870 msgr2=0x7f50600af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:41.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.114+0000 7f5067da5640 1 -- 172.21.15.29:0/467801742 shutdown_connections 2024-04-25T00:39:41.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.114+0000 7f5067da5640 1 -- 172.21.15.29:0/467801742 wait complete. 2024-04-25T00:39:41.134 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-04-25T00:39:41.134 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2024-04-25T00:39:41.134 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2024-04-25T00:39:41.134 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:41.170 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:39:41.181 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:39:41.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f93863aa640 1 Processor -- start 2024-04-25T00:39:41.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f93863aa640 1 -- start start 2024-04-25T00:39:41.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f93863aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93800d4bf0 0x7f93800d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f93863aa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f938015e8c0 0x7f93800b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f93863aa640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93800cfe30 con 0x7f938015e4e0 2024-04-25T00:39:41.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f93863aa640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f93800b6f80 con 0x7f93800d4bf0 2024-04-25T00:39:41.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f93863aa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f93800b7260 con 0x7f938015e8c0 2024-04-25T00:39:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f9384947640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f938015e8c0 0x7f93800b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f937f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93800d4bf0 0x7f93800d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f9384947640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f938015e8c0 0x7f93800b7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:45572/0 (socket says 172.21.15.29:45572) 2024-04-25T00:39:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f937ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f938015e4e0 0x7f93800d1eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:36602/0 (socket says 172.21.15.29:36602) 2024-04-25T00:39:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f9384947640 1 -- 172.21.15.29:0/2352125855 learned_addr learned my addr 172.21.15.29:0/2352125855 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f937f7fe640 1 -- 172.21.15.29:0/2352125855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f938015e8c0 msgr2=0x7f93800b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f937f7fe640 1 --2- 172.21.15.29:0/2352125855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f938015e8c0 0x7f93800b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f937f7fe640 1 -- 172.21.15.29:0/2352125855 >> v1:172.21.15.157:6789/0 conn(0x7f938015e4e0 legacy=0x7f93800d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.262+0000 7f937f7fe640 1 -- 172.21.15.29:0/2352125855 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93800ced50 con 0x7f93800d4bf0 2024-04-25T00:39:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.266+0000 7f937f7fe640 1 --2- 172.21.15.29:0/2352125855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93800d4bf0 0x7f93800d25c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9368009d90 tx=0x7f936801f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46cdd0e449645d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.266+0000 7f937effd640 1 -- 172.21.15.29:0/2352125855 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93680b4070 con 0x7f93800d4bf0 2024-04-25T00:39:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.266+0000 7f937effd640 1 -- 172.21.15.29:0/2352125855 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93680af600 con 0x7f93800d4bf0 2024-04-25T00:39:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.266+0000 7f937effd640 1 -- 172.21.15.29:0/2352125855 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93680af9d0 con 0x7f93800d4bf0 2024-04-25T00:39:41.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.266+0000 7f93863aa640 1 -- 172.21.15.29:0/2352125855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93800d4bf0 msgr2=0x7f93800d25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.266+0000 7f93863aa640 1 --2- 172.21.15.29:0/2352125855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93800d4bf0 0x7f93800d25c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9368009d90 tx=0x7f936801f150 comp rx=0 tx=0).stop 2024-04-25T00:39:41.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.266+0000 7f93863aa640 1 -- 172.21.15.29:0/2352125855 shutdown_connections 2024-04-25T00:39:41.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.266+0000 7f93863aa640 1 --2- 172.21.15.29:0/2352125855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f938015e8c0 0x7f93800b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.266+0000 7f93863aa640 1 --2- 172.21.15.29:0/2352125855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93800d4bf0 0x7f93800d25c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.266+0000 7f93863aa640 1 -- 172.21.15.29:0/2352125855 >> 172.21.15.29:0/2352125855 conn(0x7f93800d3fb0 msgr2=0x7f93800d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:41.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.266+0000 7f93863aa640 1 -- 172.21.15.29:0/2352125855 shutdown_connections 2024-04-25T00:39:41.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.266+0000 7f93863aa640 1 -- 172.21.15.29:0/2352125855 wait complete. 2024-04-25T00:39:41.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.266+0000 7f93863aa640 1 Processor -- start 2024-04-25T00:39:41.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f93863aa640 1 -- start start 2024-04-25T00:39:41.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f93863aa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93800d4bf0 0x7f938014b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f93863aa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f938015e4e0 0x7f93800cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f93863aa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938015e8c0 0x7f93800cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f93863aa640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f93800b7e40 con 0x7f93800d4bf0 2024-04-25T00:39:41.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f93863aa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f93800b7cc0 con 0x7f938015e8c0 2024-04-25T00:39:41.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f93863aa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f93800b7fc0 con 0x7f938015e4e0 2024-04-25T00:39:41.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93800d4bf0 0x7f938014b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93800d4bf0 0x7f938014b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37092/0 (socket says 172.21.15.29:37092) 2024-04-25T00:39:41.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937ffff640 1 -- 172.21.15.29:0/523399838 learned_addr learned my addr 172.21.15.29:0/523399838 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:41.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937f7fe640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f938015e4e0 0x7f93800cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f9384947640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938015e8c0 0x7f93800cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937ffff640 1 -- 172.21.15.29:0/523399838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f938015e4e0 msgr2=0x7f93800cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937ffff640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f938015e4e0 0x7f93800cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937ffff640 1 -- 172.21.15.29:0/523399838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938015e8c0 msgr2=0x7f93800cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937ffff640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938015e8c0 0x7f93800cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937ffff640 1 -- 172.21.15.29:0/523399838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9380158100 con 0x7f93800d4bf0 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f9384947640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938015e8c0 0x7f93800cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937ffff640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93800d4bf0 0x7f938014b4b0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f937800c420 tx=0x7f937800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937cff9640 1 -- 172.21.15.29:0/523399838 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93780074e0 con 0x7f93800d4bf0 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937cff9640 1 -- 172.21.15.29:0/523399838 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9378007680 con 0x7f93800d4bf0 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f93863aa640 1 -- 172.21.15.29:0/523399838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93801f48a0 con 0x7f93800d4bf0 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937cff9640 1 -- 172.21.15.29:0/523399838 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378004a30 con 0x7f93800d4bf0 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f93863aa640 1 -- 172.21.15.29:0/523399838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9380158370 con 0x7f93800d4bf0 2024-04-25T00:39:41.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f93863aa640 1 -- 172.21.15.29:0/523399838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9380069270 con 0x7f93800d4bf0 2024-04-25T00:39:41.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937cff9640 1 -- 172.21.15.29:0/523399838 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f937801a020 con 0x7f93800d4bf0 2024-04-25T00:39:41.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.270+0000 7f937cff9640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9364067f20 0x7f936406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.274+0000 7f937cff9640 1 -- 172.21.15.29:0/523399838 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7f9378085c60 con 0x7f93800d4bf0 2024-04-25T00:39:41.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.274+0000 7f937f7fe640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9364067f20 0x7f936406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.274+0000 7f937f7fe640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9364067f20 0x7f936406a3e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f93680b1000 tx=0x7f93680aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:41.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.278+0000 7f937cff9640 1 -- 172.21.15.29:0/523399838 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f93801f48a0 con 0x7f93800d4bf0 2024-04-25T00:39:41.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.438+0000 7f93863aa640 1 -- 172.21.15.29:0/523399838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93800b8d40 con 0x7f93800d4bf0 2024-04-25T00:39:41.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.442+0000 7f937cff9640 1 -- 172.21.15.29:0/523399838 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+12980 (secure 0 0 0) 0x7f9378056160 con 0x7f93800d4bf0 2024-04-25T00:39:41.441 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:41.441 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":137,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:40.085367+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6835/619209926":"2024-04-26T00:39:37.069373+0000","172.21.15.29:6834/619209926":"2024-04-26T00:39:37.069373+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:41.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.442+0000 7f93863aa640 1 -- 172.21.15.29:0/523399838 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9364067f20 msgr2=0x7f936406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.442+0000 7f93863aa640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9364067f20 0x7f936406a3e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f93680b1000 tx=0x7f93680aa000 comp rx=0 tx=0).stop 2024-04-25T00:39:41.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.442+0000 7f93863aa640 1 -- 172.21.15.29:0/523399838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93800d4bf0 msgr2=0x7f938014b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.442+0000 7f93863aa640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93800d4bf0 0x7f938014b4b0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f937800c420 tx=0x7f937800c8e0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.442+0000 7f93863aa640 1 -- 172.21.15.29:0/523399838 shutdown_connections 2024-04-25T00:39:41.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.442+0000 7f93863aa640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9364067f20 0x7f936406a3e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.442+0000 7f93863aa640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938015e8c0 0x7f93800cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.442+0000 7f93863aa640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f938015e4e0 0x7f93800cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.442+0000 7f93863aa640 1 --2- 172.21.15.29:0/523399838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93800d4bf0 0x7f938014b4b0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.442+0000 7f93863aa640 1 -- 172.21.15.29:0/523399838 >> 172.21.15.29:0/523399838 conn(0x7f93800d3fb0 msgr2=0x7f93800b8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:41.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.442+0000 7f93863aa640 1 -- 172.21.15.29:0/523399838 shutdown_connections 2024-04-25T00:39:41.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.442+0000 7f93863aa640 1 -- 172.21.15.29:0/523399838 wait complete. 2024-04-25T00:39:41.460 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:39:41.460 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:39:41.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.582+0000 7f138919e640 1 Processor -- start 2024-04-25T00:39:41.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f138919e640 1 -- start start 2024-04-25T00:39:41.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f138919e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13840b3c30 0x7f138406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f138919e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f13840b4b10 0x7f13840ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f138919e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13840b41e0 con 0x7f13840b3140 2024-04-25T00:39:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f138919e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f138406d960 con 0x7f13840b4b10 2024-04-25T00:39:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f138919e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f138406dc40 con 0x7f13840b3c30 2024-04-25T00:39:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f1382d9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f13840b3140 0x7f13840b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52512/0 (socket says 172.21.15.29:52512) 2024-04-25T00:39:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f1382d9d640 1 -- 172.21.15.29:0/2556197575 learned_addr learned my addr 172.21.15.29:0/2556197575 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f138359e640 1 --2- 172.21.15.29:0/2556197575 >> v2:172.21.15.29:3300/0 conn(0x7f13840b4b10 0x7f13840ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f138259c640 1 --2- 172.21.15.29:0/2556197575 >> v2:172.21.15.157:3300/0 conn(0x7f13840b3c30 0x7f138406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f1381d9b640 1 -- 172.21.15.29:0/2556197575 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2924478513 0 0) 0x7f13840b41e0 con 0x7f13840b3140 2024-04-25T00:39:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f1381d9b640 1 -- 172.21.15.29:0/2556197575 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f136c003530 con 0x7f13840b3140 2024-04-25T00:39:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f138359e640 1 -- 172.21.15.29:0/2556197575 >> v2:172.21.15.157:3300/0 conn(0x7f13840b3c30 msgr2=0x7f138406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:41.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f138359e640 1 --2- 172.21.15.29:0/2556197575 >> v2:172.21.15.157:3300/0 conn(0x7f13840b3c30 0x7f138406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f138359e640 1 -- 172.21.15.29:0/2556197575 >> v1:172.21.15.29:6789/0 conn(0x7f13840b3140 legacy=0x7f13840b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f1381d9b640 1 -- 172.21.15.29:0/2556197575 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1698918837 0 0) 0x7f136c003530 con 0x7f13840b3140 2024-04-25T00:39:41.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.586+0000 7f138359e640 1 -- 172.21.15.29:0/2556197575 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138406df20 con 0x7f13840b4b10 2024-04-25T00:39:41.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138359e640 1 --2- 172.21.15.29:0/2556197575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13840b4b10 0x7f13840ce240 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f137c009d90 tx=0x7f137c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e845e81cec5052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:41.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f1381d9b640 1 -- 172.21.15.29:0/2556197575 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374001680 con 0x7f13840b4b10 2024-04-25T00:39:41.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f1381d9b640 1 -- 172.21.15.29:0/2556197575 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137c0a64e0 con 0x7f13840b4b10 2024-04-25T00:39:41.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f1381d9b640 1 -- 172.21.15.29:0/2556197575 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c0a68b0 con 0x7f13840b4b10 2024-04-25T00:39:41.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 -- 172.21.15.29:0/2556197575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13840b4b10 msgr2=0x7f13840ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 --2- 172.21.15.29:0/2556197575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13840b4b10 0x7f13840ce240 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f137c009d90 tx=0x7f137c01c850 comp rx=0 tx=0).stop 2024-04-25T00:39:41.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 -- 172.21.15.29:0/2556197575 shutdown_connections 2024-04-25T00:39:41.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 --2- 172.21.15.29:0/2556197575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13840b4b10 0x7f13840ce240 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 --2- 172.21.15.29:0/2556197575 >> v2:172.21.15.157:3300/0 conn(0x7f13840b3c30 0x7f138406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 -- 172.21.15.29:0/2556197575 >> 172.21.15.29:0/2556197575 conn(0x7f13840b1870 msgr2=0x7f13840b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:41.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 -- 172.21.15.29:0/2556197575 shutdown_connections 2024-04-25T00:39:41.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 -- 172.21.15.29:0/2556197575 wait complete. 2024-04-25T00:39:41.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 Processor -- start 2024-04-25T00:39:41.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 -- start start 2024-04-25T00:39:41.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13840b3140 0x7f1384158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13840b3c30 0x7f1384164bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13840b4b10 0x7f1384165130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1384168d80 con 0x7f13840b4b10 2024-04-25T00:39:41.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1384168c00 con 0x7f13840b3140 2024-04-25T00:39:41.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138919e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1384168f00 con 0x7f13840b3c30 2024-04-25T00:39:41.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138359e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13840b4b10 0x7f1384165130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138359e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13840b4b10 0x7f1384165130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37118/0 (socket says 172.21.15.29:37118) 2024-04-25T00:39:41.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138359e640 1 -- 172.21.15.29:0/4263231024 learned_addr learned my addr 172.21.15.29:0/4263231024 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:41.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f1382d9d640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13840b3140 0x7f1384158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.590+0000 7f138259c640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13840b3c30 0x7f1384164bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f138359e640 1 -- 172.21.15.29:0/4263231024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13840b3c30 msgr2=0x7f1384164bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f138359e640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13840b3c30 0x7f1384164bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f138359e640 1 -- 172.21.15.29:0/4263231024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13840b3140 msgr2=0x7f1384158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f138359e640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13840b3140 0x7f1384158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f138359e640 1 -- 172.21.15.29:0/4263231024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138415b1e0 con 0x7f13840b4b10 2024-04-25T00:39:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f138259c640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13840b3c30 0x7f1384164bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f1382d9d640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13840b3140 0x7f1384158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f138359e640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13840b4b10 0x7f1384165130 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f137c0b0040 tx=0x7f137c0195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f136b7fe640 1 -- 172.21.15.29:0/4263231024 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c009450 con 0x7f13840b4b10 2024-04-25T00:39:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f136b7fe640 1 -- 172.21.15.29:0/4263231024 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137c0c0070 con 0x7f13840b4b10 2024-04-25T00:39:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f136b7fe640 1 -- 172.21.15.29:0/4263231024 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c0ba8f0 con 0x7f13840b4b10 2024-04-25T00:39:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f138919e640 1 -- 172.21.15.29:0/4263231024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138415b4c0 con 0x7f13840b4b10 2024-04-25T00:39:41.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f138919e640 1 -- 172.21.15.29:0/4263231024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138415ba80 con 0x7f13840b4b10 2024-04-25T00:39:41.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f138919e640 1 -- 172.21.15.29:0/4263231024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1384069270 con 0x7f13840b4b10 2024-04-25T00:39:41.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f136b7fe640 1 -- 172.21.15.29:0/4263231024 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f137c0baa90 con 0x7f13840b4b10 2024-04-25T00:39:41.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f136b7fe640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1370067ef0 0x7f137006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f136b7fe640 1 -- 172.21.15.29:0/4263231024 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7f137c121d10 con 0x7f13840b4b10 2024-04-25T00:39:41.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f1382d9d640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1370067ef0 0x7f137006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.594+0000 7f1382d9d640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1370067ef0 0x7f137006a3b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1374001b80 tx=0x7f1374009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:41.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.598+0000 7f136b7fe640 1 -- 172.21.15.29:0/4263231024 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f137c016480 con 0x7f13840b4b10 2024-04-25T00:39:41.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.758+0000 7f138919e640 1 -- 172.21.15.29:0/4263231024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f13840ce2a0 con 0x7f13840b4b10 2024-04-25T00:39:41.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.758+0000 7f136b7fe640 1 -- 172.21.15.29:0/4263231024 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) ==== 93+0+10 (secure 0 0 0) 0x7f137c0f1ef0 con 0x7f13840b4b10 2024-04-25T00:39:41.758 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:39:41.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.762+0000 7f138919e640 1 -- 172.21.15.29:0/4263231024 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1370067ef0 msgr2=0x7f137006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.762+0000 7f138919e640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1370067ef0 0x7f137006a3b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1374001b80 tx=0x7f1374009040 comp rx=0 tx=0).stop 2024-04-25T00:39:41.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.762+0000 7f138919e640 1 -- 172.21.15.29:0/4263231024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13840b4b10 msgr2=0x7f1384165130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.762+0000 7f138919e640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13840b4b10 0x7f1384165130 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f137c0b0040 tx=0x7f137c0195f0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.762+0000 7f138919e640 1 -- 172.21.15.29:0/4263231024 shutdown_connections 2024-04-25T00:39:41.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.762+0000 7f138919e640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1370067ef0 0x7f137006a3b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.762+0000 7f138919e640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13840b4b10 0x7f1384165130 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.762+0000 7f138919e640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13840b3c30 0x7f1384164bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.762+0000 7f138919e640 1 --2- 172.21.15.29:0/4263231024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13840b3140 0x7f1384158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.762+0000 7f138919e640 1 -- 172.21.15.29:0/4263231024 >> 172.21.15.29:0/4263231024 conn(0x7f13840b1870 msgr2=0x7f13840b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:41.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.762+0000 7f138919e640 1 -- 172.21.15.29:0/4263231024 shutdown_connections 2024-04-25T00:39:41.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.762+0000 7f138919e640 1 -- 172.21.15.29:0/4263231024 wait complete. 2024-04-25T00:39:41.776 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-04-25T00:39:41.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa359208640 1 Processor -- start 2024-04-25T00:39:41.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa359208640 1 -- start start 2024-04-25T00:39:41.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa359208640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa35415d980 0x7fa35406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa359208640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa35406db10 0x7fa354151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa359208640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3540b3130 con 0x7fa3540b2b70 2024-04-25T00:39:41.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa359208640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa3540cf560 con 0x7fa35406db10 2024-04-25T00:39:41.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa359208640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa35406df60 con 0x7fa35415d980 2024-04-25T00:39:41.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa35359e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa35406db10 0x7fa354151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa35359e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa35406db10 0x7fa354151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37134/0 (socket says 172.21.15.29:37134) 2024-04-25T00:39:41.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa35359e640 1 -- 172.21.15.29:0/2509897349 learned_addr learned my addr 172.21.15.29:0/2509897349 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:41.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa35359e640 1 -- 172.21.15.29:0/2509897349 >> v2:172.21.15.157:3300/0 conn(0x7fa35415d980 msgr2=0x7fa35406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:41.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa35359e640 1 --2- 172.21.15.29:0/2509897349 >> v2:172.21.15.157:3300/0 conn(0x7fa35415d980 0x7fa35406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa35359e640 1 -- 172.21.15.29:0/2509897349 >> v1:172.21.15.157:6789/0 conn(0x7fa3540b2b70 legacy=0x7fa354150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:41.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.898+0000 7fa35359e640 1 -- 172.21.15.29:0/2509897349 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3540b3410 con 0x7fa35406db10 2024-04-25T00:39:41.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa35359e640 1 --2- 172.21.15.29:0/2509897349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa35406db10 0x7fa354151500 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fa3540d0ca0 tx=0x7fa34c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f943ac29aec3e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:41.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa351d9b640 1 -- 172.21.15.29:0/2509897349 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0a56e0 con 0x7fa35406db10 2024-04-25T00:39:41.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa351d9b640 1 -- 172.21.15.29:0/2509897349 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34c0a5880 con 0x7fa35406db10 2024-04-25T00:39:41.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa351d9b640 1 -- 172.21.15.29:0/2509897349 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0a5c50 con 0x7fa35406db10 2024-04-25T00:39:41.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 -- 172.21.15.29:0/2509897349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa35406db10 msgr2=0x7fa354151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 --2- 172.21.15.29:0/2509897349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa35406db10 0x7fa354151500 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fa3540d0ca0 tx=0x7fa34c01c9b0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 -- 172.21.15.29:0/2509897349 shutdown_connections 2024-04-25T00:39:41.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 --2- 172.21.15.29:0/2509897349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa35406db10 0x7fa354151500 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 --2- 172.21.15.29:0/2509897349 >> v2:172.21.15.157:3300/0 conn(0x7fa35415d980 0x7fa35406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 -- 172.21.15.29:0/2509897349 >> 172.21.15.29:0/2509897349 conn(0x7fa3540b1870 msgr2=0x7fa3540b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:41.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 -- 172.21.15.29:0/2509897349 shutdown_connections 2024-04-25T00:39:41.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 -- 172.21.15.29:0/2509897349 wait complete. 2024-04-25T00:39:41.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 Processor -- start 2024-04-25T00:39:41.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 -- start start 2024-04-25T00:39:41.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa35406db10 0x7fa35413b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3540b2b70 0x7fa354155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35415d980 0x7fa3540720a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa35415b4b0 con 0x7fa3540b2b70 2024-04-25T00:39:41.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa35415dd60 con 0x7fa35415d980 2024-04-25T00:39:41.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa359208640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa35415b630 con 0x7fa35406db10 2024-04-25T00:39:41.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa35259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3540b2b70 0x7fa354155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa35259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3540b2b70 0x7fa354155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37138/0 (socket says 172.21.15.29:37138) 2024-04-25T00:39:41.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.902+0000 7fa35259c640 1 -- 172.21.15.29:0/3819015134 learned_addr learned my addr 172.21.15.29:0/3819015134 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:41.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa35359e640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35415d980 0x7fa3540720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa352d9d640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa35406db10 0x7fa35413b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa35259c640 1 -- 172.21.15.29:0/3819015134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa35406db10 msgr2=0x7fa35413b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa35259c640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa35406db10 0x7fa35413b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa35259c640 1 -- 172.21.15.29:0/3819015134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35415d980 msgr2=0x7fa3540720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:41.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa35259c640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35415d980 0x7fa3540720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:41.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa35259c640 1 -- 172.21.15.29:0/3819015134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35413bc40 con 0x7fa3540b2b70 2024-04-25T00:39:41.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa35359e640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35415d980 0x7fa3540720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:41.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa352d9d640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa35406db10 0x7fa35413b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:41.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa35259c640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3540b2b70 0x7fa354155ce0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fa34800c450 tx=0x7fa34800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:41.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa33b7fe640 1 -- 172.21.15.29:0/3819015134 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348012830 con 0x7fa3540b2b70 2024-04-25T00:39:41.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa33b7fe640 1 -- 172.21.15.29:0/3819015134 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa348017070 con 0x7fa3540b2b70 2024-04-25T00:39:41.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa359208640 1 -- 172.21.15.29:0/3819015134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3541577f0 con 0x7fa3540b2b70 2024-04-25T00:39:41.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa33b7fe640 1 -- 172.21.15.29:0/3819015134 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348012c30 con 0x7fa3540b2b70 2024-04-25T00:39:41.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa359208640 1 -- 172.21.15.29:0/3819015134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3541554e0 con 0x7fa3540b2b70 2024-04-25T00:39:41.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa359208640 1 -- 172.21.15.29:0/3819015134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa354069270 con 0x7fa3540b2b70 2024-04-25T00:39:41.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa33b7fe640 1 -- 172.21.15.29:0/3819015134 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fa34801c460 con 0x7fa3540b2b70 2024-04-25T00:39:41.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa33b7fe640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa340067ef0 0x7fa34006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:41.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa33b7fe640 1 -- 172.21.15.29:0/3819015134 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7fa348086d20 con 0x7fa3540b2b70 2024-04-25T00:39:41.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa352d9d640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa340067ef0 0x7fa34006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:41.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.906+0000 7fa352d9d640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa340067ef0 0x7fa34006a3b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa34400a870 tx=0x7fa344008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:41.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:41.910+0000 7fa33b7fe640 1 -- 172.21.15.29:0/3819015134 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa348057320 con 0x7fa3540b2b70 2024-04-25T00:39:42.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:42.130+0000 7fa359208640 1 -- 172.21.15.29:0/3819015134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7fa3540cf910 con 0x7fa3540b2b70 2024-04-25T00:39:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.118+0000 7fa33b7fe640 1 -- 172.21.15.29:0/3819015134 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v440) ==== 152+0+0 (secure 0 0 0) 0x7fa348056c20 con 0x7fa3540b2b70 2024-04-25T00:39:43.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.122+0000 7fa359208640 1 -- 172.21.15.29:0/3819015134 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa340067ef0 msgr2=0x7fa34006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:43.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.122+0000 7fa359208640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa340067ef0 0x7fa34006a3b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa34400a870 tx=0x7fa344008040 comp rx=0 tx=0).stop 2024-04-25T00:39:43.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.122+0000 7fa359208640 1 -- 172.21.15.29:0/3819015134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3540b2b70 msgr2=0x7fa354155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:43.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.122+0000 7fa359208640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3540b2b70 0x7fa354155ce0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fa34800c450 tx=0x7fa34800c910 comp rx=0 tx=0).stop 2024-04-25T00:39:43.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.122+0000 7fa359208640 1 -- 172.21.15.29:0/3819015134 shutdown_connections 2024-04-25T00:39:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.122+0000 7fa359208640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa340067ef0 0x7fa34006a3b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.122+0000 7fa359208640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35415d980 0x7fa3540720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.122+0000 7fa359208640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa3540b2b70 0x7fa354155ce0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.122+0000 7fa359208640 1 --2- 172.21.15.29:0/3819015134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa35406db10 0x7fa35413b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.122+0000 7fa359208640 1 -- 172.21.15.29:0/3819015134 >> 172.21.15.29:0/3819015134 conn(0x7fa3540b1870 msgr2=0x7fa3540b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.122+0000 7fa359208640 1 -- 172.21.15.29:0/3819015134 shutdown_connections 2024-04-25T00:39:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.122+0000 7fa359208640 1 -- 172.21.15.29:0/3819015134 wait complete. 2024-04-25T00:39:43.140 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:39:43.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4f4cc640 1 Processor -- start 2024-04-25T00:39:43.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4f4cc640 1 -- start start 2024-04-25T00:39:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4f4cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4806e440 0x7f9c480b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4f4cc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c480b41e0 con 0x7f9c480b3c30 2024-04-25T00:39:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4f4cc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c4806d960 con 0x7f9c480b3140 2024-04-25T00:39:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4f4cc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9c480b5050 con 0x7f9c4806e440 2024-04-25T00:39:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4ca67640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9c480b3c30 0x7f9c480ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52528/0 (socket says 172.21.15.29:52528) 2024-04-25T00:39:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4ca67640 1 -- 172.21.15.29:0/3671286047 learned_addr learned my addr 172.21.15.29:0/3671286047 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4da69640 1 --2- 172.21.15.29:0/3671286047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4806e440 0x7f9c480b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:43.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2786539615 0 0) 0x7f9c480b41e0 con 0x7f9c480b3c30 2024-04-25T00:39:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c34003530 con 0x7f9c480b3c30 2024-04-25T00:39:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 145889206 0 0) 0x7f9c34003530 con 0x7f9c480b3c30 2024-04-25T00:39:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4806e440 msgr2=0x7f9c480b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 --2- 172.21.15.29:0/3671286047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4806e440 0x7f9c480b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 >> v1:172.21.15.157:6789/0 conn(0x7f9c480b3140 legacy=0x7f9c480b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4806dc40 con 0x7f9c480b3c30 2024-04-25T00:39:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f9c38001680 con 0x7f9c480b3c30 2024-04-25T00:39:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 >> v1:172.21.15.29:6789/0 conn(0x7f9c480b3c30 legacy=0x7f9c480ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 --2- 172.21.15.29:0/3671286047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c340039f0 0x7f9c34003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 --2- 172.21.15.29:0/3671286047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4806e440 0x7f9c34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 --2- 172.21.15.29:0/3671286047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c34004d50 0x7f9c340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9c480b5050 con 0x7f9c34004d50 2024-04-25T00:39:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3400ba80 con 0x7f9c340039f0 2024-04-25T00:39:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9c3400bc80 con 0x7f9c4806e440 2024-04-25T00:39:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4da69640 1 --2- 172.21.15.29:0/3671286047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c34004d50 0x7f9c340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4ca67640 1 --2- 172.21.15.29:0/3671286047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4806e440 0x7f9c34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4d268640 1 --2- 172.21.15.29:0/3671286047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c340039f0 0x7f9c34003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4da69640 1 -- 172.21.15.29:0/3671286047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4806e440 msgr2=0x7f9c34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4da69640 1 --2- 172.21.15.29:0/3671286047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4806e440 0x7f9c34004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4da69640 1 -- 172.21.15.29:0/3671286047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c340039f0 msgr2=0x7f9c34003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4da69640 1 --2- 172.21.15.29:0/3671286047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c340039f0 0x7f9c34003e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4da69640 1 -- 172.21.15.29:0/3671286047 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c34003530 con 0x7f9c34004d50 2024-04-25T00:39:43.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.278+0000 7f9c4ca67640 1 --2- 172.21.15.29:0/3671286047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4806e440 0x7f9c34004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4da69640 1 --2- 172.21.15.29:0/3671286047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c34004d50 0x7f9c340051d0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009f90 tx=0x7f9c4001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1f7c5dc9c08738 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c400b2070 con 0x7f9c34004d50 2024-04-25T00:39:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c400a5c50 con 0x7f9c34004d50 2024-04-25T00:39:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c3ffff640 1 -- 172.21.15.29:0/3671286047 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40009c10 con 0x7f9c34004d50 2024-04-25T00:39:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/3671286047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c34004d50 msgr2=0x7f9c340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 --2- 172.21.15.29:0/3671286047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c34004d50 0x7f9c340051d0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009f90 tx=0x7f9c4001cc40 comp rx=0 tx=0).stop 2024-04-25T00:39:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4da69640 1 -- 172.21.15.29:0/3671286047 reap_dead start 2024-04-25T00:39:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/3671286047 shutdown_connections 2024-04-25T00:39:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/3671286047 >> 172.21.15.29:0/3671286047 conn(0x7f9c480b1870 msgr2=0x7f9c480b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/3671286047 shutdown_connections 2024-04-25T00:39:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/3671286047 wait complete. 2024-04-25T00:39:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 Processor -- start 2024-04-25T00:39:43.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 -- start start 2024-04-25T00:39:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c480cbf10 0x7f9c480cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c480c8580 0x7f9c480c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c481f7290 0x7f9c481f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9c48145a90 con 0x7f9c480c8580 2024-04-25T00:39:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9c48145910 con 0x7f9c480cbf10 2024-04-25T00:39:43.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.282+0000 7f9c4f4cc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9c480b4b10 con 0x7f9c481f7290 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4ca67640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c480c8580 0x7f9c480c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4ca67640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c480c8580 0x7f9c480c5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37144/0 (socket says 172.21.15.29:37144) 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4ca67640 1 -- 172.21.15.29:0/743675809 learned_addr learned my addr 172.21.15.29:0/743675809 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4da69640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c481f7290 0x7f9c481f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4d268640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c480cbf10 0x7f9c480cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4ca67640 1 -- 172.21.15.29:0/743675809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c481f7290 msgr2=0x7f9c481f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4ca67640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c481f7290 0x7f9c481f7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4ca67640 1 -- 172.21.15.29:0/743675809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c480cbf10 msgr2=0x7f9c480cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4ca67640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c480cbf10 0x7f9c480cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4ca67640 1 -- 172.21.15.29:0/743675809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c480c7540 con 0x7f9c480c8580 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4ca67640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c480c8580 0x7f9c480c5240 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800d8d0 tx=0x7f9c3800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c3dffb640 1 -- 172.21.15.29:0/743675809 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38019070 con 0x7f9c480c8580 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c3dffb640 1 -- 172.21.15.29:0/743675809 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c38004cc0 con 0x7f9c480c8580 2024-04-25T00:39:43.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/743675809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c480c77e0 con 0x7f9c480c8580 2024-04-25T00:39:43.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c3dffb640 1 -- 172.21.15.29:0/743675809 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c380072d0 con 0x7f9c480c8580 2024-04-25T00:39:43.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/743675809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c480c7da0 con 0x7f9c480c8580 2024-04-25T00:39:43.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/743675809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c48069270 con 0x7f9c480c8580 2024-04-25T00:39:43.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c3dffb640 1 -- 172.21.15.29:0/743675809 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f9c3801c070 con 0x7f9c480c8580 2024-04-25T00:39:43.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c3dffb640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9c2c067ef0 0x7f9c2c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:43.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c3dffb640 1 -- 172.21.15.29:0/743675809 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7f9c38086170 con 0x7f9c480c8580 2024-04-25T00:39:43.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4d268640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9c2c067ef0 0x7f9c2c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:43.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.286+0000 7f9c4d268640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9c2c067ef0 0x7f9c2c06a3b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9c440015c0 tx=0x7f9c44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:43.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.290+0000 7f9c3dffb640 1 -- 172.21.15.29:0/743675809 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9c380565f0 con 0x7f9c480c8580 2024-04-25T00:39:43.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.454+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/743675809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9c480d05e0 con 0x7f9c480c8580 2024-04-25T00:39:43.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.458+0000 7f9c3dffb640 1 -- 172.21.15.29:0/743675809 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f9c38055ef0 con 0x7f9c480c8580 2024-04-25T00:39:43.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.458+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/743675809 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9c2c067ef0 msgr2=0x7f9c2c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:43.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.458+0000 7f9c4f4cc640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9c2c067ef0 0x7f9c2c06a3b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9c440015c0 tx=0x7f9c44009040 comp rx=0 tx=0).stop 2024-04-25T00:39:43.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.458+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/743675809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c480c8580 msgr2=0x7f9c480c5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:43.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.458+0000 7f9c4f4cc640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c480c8580 0x7f9c480c5240 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800d8d0 tx=0x7f9c3800dd90 comp rx=0 tx=0).stop 2024-04-25T00:39:43.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.458+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/743675809 shutdown_connections 2024-04-25T00:39:43.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.458+0000 7f9c4f4cc640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9c2c067ef0 0x7f9c2c06a3b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.458+0000 7f9c4f4cc640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c481f7290 0x7f9c481f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.458+0000 7f9c4f4cc640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c480c8580 0x7f9c480c5240 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.458+0000 7f9c4f4cc640 1 --2- 172.21.15.29:0/743675809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c480cbf10 0x7f9c480cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.458+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/743675809 >> 172.21.15.29:0/743675809 conn(0x7f9c480b1870 msgr2=0x7f9c4815d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:43.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.462+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/743675809 shutdown_connections 2024-04-25T00:39:43.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.462+0000 7f9c4f4cc640 1 -- 172.21.15.29:0/743675809 wait complete. 2024-04-25T00:39:43.476 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:39:43.476 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:39:43.521 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:39:43.584 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:39:43.584 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:39:43.584 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:39:43.589 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:39:43.589 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:39:43.589 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:39:43.595 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:39:43.595 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:39:43.595 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:39:43.640 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:39:43.640 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:39:43.640 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:39:43.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:39:43.645 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:39:43.646 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:39:43.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f5933363640 1 Processor -- start 2024-04-25T00:39:43.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f5933363640 1 -- start start 2024-04-25T00:39:43.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f5933363640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f592c06c950 0x7f592c06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:43.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f5933363640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f592c0cc3b0 con 0x7f592c15e440 2024-04-25T00:39:43.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f5933363640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f592c06d270 con 0x7f592c0cd410 2024-04-25T00:39:43.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f5933363640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f592c0b11d0 con 0x7f592c06c950 2024-04-25T00:39:43.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f59310ff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f592c06c950 0x7f592c06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:43.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f59308fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f592c15e440 0x7f592c0d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52538/0 (socket says 172.21.15.29:52538) 2024-04-25T00:39:43.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f59308fe640 1 -- 172.21.15.29:0/4230342754 learned_addr learned my addr 172.21.15.29:0/4230342754 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:43.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f5923fff640 1 -- 172.21.15.29:0/4230342754 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4151079104 0 0) 0x7f592c0cc3b0 con 0x7f592c15e440 2024-04-25T00:39:43.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f5923fff640 1 -- 172.21.15.29:0/4230342754 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5918003530 con 0x7f592c15e440 2024-04-25T00:39:43.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f59310ff640 1 -- 172.21.15.29:0/4230342754 >> v1:172.21.15.157:6789/0 conn(0x7f592c0cd410 legacy=0x7f592c0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:43.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f59310ff640 1 -- 172.21.15.29:0/4230342754 >> v1:172.21.15.29:6789/0 conn(0x7f592c15e440 legacy=0x7f592c0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:43.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.814+0000 7f59310ff640 1 -- 172.21.15.29:0/4230342754 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592c0cc650 con 0x7f592c06c950 2024-04-25T00:39:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f59310ff640 1 --2- 172.21.15.29:0/4230342754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592c06c950 0x7f592c06cd30 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f591c0097c0 tx=0x7f591c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebf57a92d2e3669 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5923fff640 1 -- 172.21.15.29:0/4230342754 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c0b0070 con 0x7f592c06c950 2024-04-25T00:39:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5923fff640 1 -- 172.21.15.29:0/4230342754 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f591c004230 con 0x7f592c06c950 2024-04-25T00:39:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5923fff640 1 -- 172.21.15.29:0/4230342754 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c003620 con 0x7f592c06c950 2024-04-25T00:39:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 -- 172.21.15.29:0/4230342754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592c06c950 msgr2=0x7f592c06cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 --2- 172.21.15.29:0/4230342754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592c06c950 0x7f592c06cd30 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f591c0097c0 tx=0x7f591c01c460 comp rx=0 tx=0).stop 2024-04-25T00:39:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 -- 172.21.15.29:0/4230342754 shutdown_connections 2024-04-25T00:39:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 --2- 172.21.15.29:0/4230342754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592c06c950 0x7f592c06cd30 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 -- 172.21.15.29:0/4230342754 >> 172.21.15.29:0/4230342754 conn(0x7f592c0d1d60 msgr2=0x7f592c0d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 -- 172.21.15.29:0/4230342754 shutdown_connections 2024-04-25T00:39:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 -- 172.21.15.29:0/4230342754 wait complete. 2024-04-25T00:39:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 Processor -- start 2024-04-25T00:39:43.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 -- start start 2024-04-25T00:39:43.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592c06c950 0x7f592c14f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592c0cd410 0x7f592c14d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592c15e440 0x7f592c0c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f592c0d11c0 con 0x7f592c0cd410 2024-04-25T00:39:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f592c06c520 con 0x7f592c06c950 2024-04-25T00:39:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5933363640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f592c0ce3c0 con 0x7f592c15e440 2024-04-25T00:39:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f59308fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592c0cd410 0x7f592c14d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f59308fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592c0cd410 0x7f592c14d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37170/0 (socket says 172.21.15.29:37170) 2024-04-25T00:39:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f59308fe640 1 -- 172.21.15.29:0/1163740799 learned_addr learned my addr 172.21.15.29:0/1163740799 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f59310ff640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592c06c950 0x7f592c14f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5931900640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592c15e440 0x7f592c0c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f59308fe640 1 -- 172.21.15.29:0/1163740799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592c15e440 msgr2=0x7f592c0c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f59308fe640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592c15e440 0x7f592c0c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f59308fe640 1 -- 172.21.15.29:0/1163740799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592c06c950 msgr2=0x7f592c14f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f59308fe640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592c06c950 0x7f592c14f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f59308fe640 1 -- 172.21.15.29:0/1163740799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592c0c69a0 con 0x7f592c0cd410 2024-04-25T00:39:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f5931900640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592c15e440 0x7f592c0c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.818+0000 7f59310ff640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592c06c950 0x7f592c14f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.822+0000 7f59308fe640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592c0cd410 0x7f592c14d6d0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f592800c4d0 tx=0x7f592800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.822+0000 7f5921ffb640 1 -- 172.21.15.29:0/1163740799 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5928017070 con 0x7f592c0cd410 2024-04-25T00:39:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.822+0000 7f5921ffb640 1 -- 172.21.15.29:0/1163740799 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5928005cf0 con 0x7f592c0cd410 2024-04-25T00:39:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.822+0000 7f5933363640 1 -- 172.21.15.29:0/1163740799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592c0c6c80 con 0x7f592c0cd410 2024-04-25T00:39:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.822+0000 7f5921ffb640 1 -- 172.21.15.29:0/1163740799 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5928007270 con 0x7f592c0cd410 2024-04-25T00:39:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.822+0000 7f5933363640 1 -- 172.21.15.29:0/1163740799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f592c0c7240 con 0x7f592c0cd410 2024-04-25T00:39:43.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.822+0000 7f5933363640 1 -- 172.21.15.29:0/1163740799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f592c069270 con 0x7f592c0cd410 2024-04-25T00:39:43.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.822+0000 7f5921ffb640 1 -- 172.21.15.29:0/1163740799 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f592801a070 con 0x7f592c0cd410 2024-04-25T00:39:43.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.822+0000 7f5921ffb640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5910067ef0 0x7f591006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:43.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.822+0000 7f5921ffb640 1 -- 172.21.15.29:0/1163740799 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7f5928085b80 con 0x7f592c0cd410 2024-04-25T00:39:43.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.822+0000 7f59310ff640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5910067ef0 0x7f591006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:43.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.822+0000 7f59310ff640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5910067ef0 0x7f591006a3b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f591c0ad000 tx=0x7f591c0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:43.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:43.826+0000 7f5921ffb640 1 -- 172.21.15.29:0/1163740799 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5928055bc0 con 0x7f592c0cd410 2024-04-25T00:39:44.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5933363640 1 -- 172.21.15.29:0/1163740799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f592c0cbe60 con 0x7f592c0cd410 2024-04-25T00:39:44.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5921ffb640 1 -- 172.21.15.29:0/1163740799 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+3863 (secure 0 0 0) 0x7f5928059ec0 con 0x7f592c0cd410 2024-04-25T00:39:44.024 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:44.025 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":86,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7450,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/456996219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":456996219},{"type":"v1","addr":"172.21.15.157:6833","nonce":456996219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/87071339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":87071339},{"type":"v1","addr":"172.21.15.157:6835","nonce":87071339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3920208567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3920208567},{"type":"v1","addr":"172.21.15.29:6833","nonce":3920208567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9537,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/206426973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":206426973},{"type":"v1","addr":"172.21.15.29:6835","nonce":206426973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[]} 2024-04-25T00:39:44.025 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 86 2024-04-25T00:39:44.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5933363640 1 -- 172.21.15.29:0/1163740799 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5910067ef0 msgr2=0x7f591006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:44.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5933363640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5910067ef0 0x7f591006a3b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f591c0ad000 tx=0x7f591c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:39:44.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5933363640 1 -- 172.21.15.29:0/1163740799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592c0cd410 msgr2=0x7f592c14d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:44.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5933363640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592c0cd410 0x7f592c14d6d0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f592800c4d0 tx=0x7f592800c990 comp rx=0 tx=0).stop 2024-04-25T00:39:44.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5933363640 1 -- 172.21.15.29:0/1163740799 shutdown_connections 2024-04-25T00:39:44.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5933363640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5910067ef0 0x7f591006a3b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:44.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5933363640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592c15e440 0x7f592c0c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:44.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5933363640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f592c0cd410 0x7f592c14d6d0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:44.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5933363640 1 --2- 172.21.15.29:0/1163740799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f592c06c950 0x7f592c14f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:44.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5933363640 1 -- 172.21.15.29:0/1163740799 >> 172.21.15.29:0/1163740799 conn(0x7f592c0d1d60 msgr2=0x7f592c0d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:44.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5933363640 1 -- 172.21.15.29:0/1163740799 shutdown_connections 2024-04-25T00:39:44.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.026+0000 7f5933363640 1 -- 172.21.15.29:0/1163740799 wait complete. 2024-04-25T00:39:44.044 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:39:44.044 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:39:44.044 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:39:44.044 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:39:44.044 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:39:44.045 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:39:44.045 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:39:44.045 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:39:44.045 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:39:44.045 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:39:44.045 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:39:44.045 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:39:44.045 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:39:44.046 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:39:44.046 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:39:44.046 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:39:44.059 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:39:44.059 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:39:44.060 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:39:44.060 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:39:44.061 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:39:44.061 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:39:44.064 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:39:44.064 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:39:44.107 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:39:44.108 DEBUG:teuthology.parallel:result is None 2024-04-25T00:39:44.109 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:39:44.109 DEBUG:teuthology.parallel:result is None 2024-04-25T00:39:44.109 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:39:44.110 DEBUG:teuthology.parallel:result is None 2024-04-25T00:39:44.110 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:39:44.110 DEBUG:teuthology.parallel:result is None 2024-04-25T00:39:44.111 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:39:44.175 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:39:44.178 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:39:44.180 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-25T00:39:44.185 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-25T00:39:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.230+0000 7f6fa4c05640 1 Processor -- start 2024-04-25T00:39:44.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.230+0000 7f6fa4c05640 1 -- start start 2024-04-25T00:39:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6fa4c05640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6fa006d580 0x7f6fa0162450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6fa4c05640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fa0143a90 con 0x7f6fa00d08a0 2024-04-25T00:39:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6fa4c05640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fa00d0c80 con 0x7f6fa00b3a90 2024-04-25T00:39:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6fa4c05640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa0144660 con 0x7f6fa006d580 2024-04-25T00:39:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6f9e59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6fa00d08a0 0x7f6fa00b3380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52570/0 (socket says 172.21.15.29:52570) 2024-04-25T00:39:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6f9e59c640 1 -- 172.21.15.29:0/2211488556 learned_addr learned my addr 172.21.15.29:0/2211488556 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6f9ed9d640 1 --2- 172.21.15.29:0/2211488556 >> v2:172.21.15.157:3300/0 conn(0x7f6fa006d580 0x7f6fa0162450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6f9d59a640 1 -- 172.21.15.29:0/2211488556 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 189421988 0 0) 0x7f6fa0143a90 con 0x7f6fa00d08a0 2024-04-25T00:39:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6f9d59a640 1 -- 172.21.15.29:0/2211488556 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f88003530 con 0x7f6fa00d08a0 2024-04-25T00:39:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6f9ed9d640 1 -- 172.21.15.29:0/2211488556 >> v1:172.21.15.157:6789/0 conn(0x7f6fa00b3a90 legacy=0x7f6fa0161d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6f9ed9d640 1 -- 172.21.15.29:0/2211488556 >> v1:172.21.15.29:6789/0 conn(0x7f6fa00d08a0 legacy=0x7f6fa00b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:44.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6f9ed9d640 1 -- 172.21.15.29:0/2211488556 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa0143d30 con 0x7f6fa006d580 2024-04-25T00:39:44.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6f9ed9d640 1 --2- 172.21.15.29:0/2211488556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fa006d580 0x7f6fa0162450 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009d90 tx=0x7f6f9801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94663e06374da525 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:44.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6f9d59a640 1 -- 172.21.15.29:0/2211488556 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f980b2070 con 0x7f6fa006d580 2024-04-25T00:39:44.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6f9d59a640 1 -- 172.21.15.29:0/2211488556 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f980a5970 con 0x7f6fa006d580 2024-04-25T00:39:44.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.234+0000 7f6f9d59a640 1 -- 172.21.15.29:0/2211488556 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f980a5d40 con 0x7f6fa006d580 2024-04-25T00:39:44.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 -- 172.21.15.29:0/2211488556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fa006d580 msgr2=0x7f6fa0162450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:44.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 --2- 172.21.15.29:0/2211488556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fa006d580 0x7f6fa0162450 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009d90 tx=0x7f6f9801c8b0 comp rx=0 tx=0).stop 2024-04-25T00:39:44.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 -- 172.21.15.29:0/2211488556 shutdown_connections 2024-04-25T00:39:44.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 --2- 172.21.15.29:0/2211488556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fa006d580 0x7f6fa0162450 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:44.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 -- 172.21.15.29:0/2211488556 >> 172.21.15.29:0/2211488556 conn(0x7f6fa0074c00 msgr2=0x7f6fa006b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:44.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 -- 172.21.15.29:0/2211488556 shutdown_connections 2024-04-25T00:39:44.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 -- 172.21.15.29:0/2211488556 wait complete. 2024-04-25T00:39:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 Processor -- start 2024-04-25T00:39:44.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 -- start start 2024-04-25T00:39:44.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fa006d580 0x7f6fa015d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fa00b3a90 0x7f6fa0138130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fa00d08a0 0x7f6fa015d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa0162b10 con 0x7f6fa00b3a90 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa0162990 con 0x7f6fa00d08a0 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa016c800 con 0x7f6fa006d580 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f9dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fa00b3a90 0x7f6fa0138130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f9dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fa00b3a90 0x7f6fa0138130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37208/0 (socket says 172.21.15.29:37208) 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f9dd9b640 1 -- 172.21.15.29:0/710721659 learned_addr learned my addr 172.21.15.29:0/710721659 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f9e59c640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fa006d580 0x7f6fa015d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f9ed9d640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fa00d08a0 0x7f6fa015d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f9dd9b640 1 -- 172.21.15.29:0/710721659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fa006d580 msgr2=0x7f6fa015d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f9dd9b640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fa006d580 0x7f6fa015d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f9dd9b640 1 -- 172.21.15.29:0/710721659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fa00d08a0 msgr2=0x7f6fa015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:44.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f9dd9b640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fa00d08a0 0x7f6fa015d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f9dd9b640 1 -- 172.21.15.29:0/710721659 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa0138670 con 0x7f6fa00b3a90 2024-04-25T00:39:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f9e59c640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fa006d580 0x7f6fa015d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f9dd9b640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fa00b3a90 0x7f6fa0138130 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400c500 tx=0x7f6f9400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f8effd640 1 -- 172.21.15.29:0/710721659 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94016020 con 0x7f6fa00b3a90 2024-04-25T00:39:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f8effd640 1 -- 172.21.15.29:0/710721659 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f94005220 con 0x7f6fa00b3a90 2024-04-25T00:39:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6f8effd640 1 -- 172.21.15.29:0/710721659 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94007030 con 0x7f6fa00b3a90 2024-04-25T00:39:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 -- 172.21.15.29:0/710721659 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa01456a0 con 0x7f6fa00b3a90 2024-04-25T00:39:44.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.238+0000 7f6fa4c05640 1 -- 172.21.15.29:0/710721659 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa0137930 con 0x7f6fa00b3a90 2024-04-25T00:39:44.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.242+0000 7f6fa4c05640 1 -- 172.21.15.29:0/710721659 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa0069270 con 0x7f6fa00b3a90 2024-04-25T00:39:44.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.242+0000 7f6f8effd640 1 -- 172.21.15.29:0/710721659 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f6f940071f0 con 0x7f6fa00b3a90 2024-04-25T00:39:44.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.242+0000 7f6f8effd640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6f80067f20 0x7f6f8006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:44.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.242+0000 7f6f8effd640 1 -- 172.21.15.29:0/710721659 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7f6f94085eb0 con 0x7f6fa00b3a90 2024-04-25T00:39:44.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.242+0000 7f6f9e59c640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6f80067f20 0x7f6f8006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:44.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.242+0000 7f6f9e59c640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6f80067f20 0x7f6f8006a3e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6f90005890 tx=0x7f6f90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:44.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.246+0000 7f6f8effd640 1 -- 172.21.15.29:0/710721659 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6f94020020 con 0x7f6fa00b3a90 2024-04-25T00:39:44.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:44.414+0000 7f6fa4c05640 1 -- 172.21.15.29:0/710721659 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6fa00cf520 con 0x7f6fa00b3a90 2024-04-25T00:39:45.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.142+0000 7f6f8effd640 1 -- 172.21.15.29:0/710721659 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v139) ==== 97+0+0 (secure 0 0 0) 0x7f6f940563c0 con 0x7f6fa00b3a90 2024-04-25T00:39:45.143 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-04-25T00:39:45.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.146+0000 7f6fa4c05640 1 -- 172.21.15.29:0/710721659 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6f80067f20 msgr2=0x7f6f8006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.146+0000 7f6fa4c05640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6f80067f20 0x7f6f8006a3e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6f90005890 tx=0x7f6f90009040 comp rx=0 tx=0).stop 2024-04-25T00:39:45.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.146+0000 7f6fa4c05640 1 -- 172.21.15.29:0/710721659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fa00b3a90 msgr2=0x7f6fa0138130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.146+0000 7f6fa4c05640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fa00b3a90 0x7f6fa0138130 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400c500 tx=0x7f6f9400c9c0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.150+0000 7f6fa4c05640 1 -- 172.21.15.29:0/710721659 shutdown_connections 2024-04-25T00:39:45.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.150+0000 7f6fa4c05640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6f80067f20 0x7f6f8006a3e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.150+0000 7f6fa4c05640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fa00d08a0 0x7f6fa015d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.150+0000 7f6fa4c05640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fa00b3a90 0x7f6fa0138130 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.150+0000 7f6fa4c05640 1 --2- 172.21.15.29:0/710721659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fa006d580 0x7f6fa015d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.150+0000 7f6fa4c05640 1 -- 172.21.15.29:0/710721659 >> 172.21.15.29:0/710721659 conn(0x7f6fa0074c00 msgr2=0x7f6fa0161d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:45.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.150+0000 7f6fa4c05640 1 -- 172.21.15.29:0/710721659 shutdown_connections 2024-04-25T00:39:45.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.150+0000 7f6fa4c05640 1 -- 172.21.15.29:0/710721659 wait complete. 2024-04-25T00:39:45.171 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:39:45.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f678085f640 1 Processor -- start 2024-04-25T00:39:45.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f678085f640 1 -- start start 2024-04-25T00:39:45.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f678085f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f677c0b12d0 0x7f677c0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f678085f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f677c0d12a0 con 0x7f677c06c530 2024-04-25T00:39:45.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f678085f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f677c06ced0 con 0x7f677c0d1a10 2024-04-25T00:39:45.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f678085f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f677c0b1720 con 0x7f677c0b12d0 2024-04-25T00:39:45.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f6779d9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f677c06c530 0x7f677c0b1ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52572/0 (socket says 172.21.15.29:52572) 2024-04-25T00:39:45.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f6779d9b640 1 -- 172.21.15.29:0/3054512082 learned_addr learned my addr 172.21.15.29:0/3054512082 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:45.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f677ad9d640 1 --2- 172.21.15.29:0/3054512082 >> v2:172.21.15.29:3300/0 conn(0x7f677c0b12d0 0x7f677c0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f677959a640 1 -- 172.21.15.29:0/3054512082 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1419586985 0 0) 0x7f677c0d12a0 con 0x7f677c06c530 2024-04-25T00:39:45.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f677959a640 1 -- 172.21.15.29:0/3054512082 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6768003530 con 0x7f677c06c530 2024-04-25T00:39:45.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f677ad9d640 1 -- 172.21.15.29:0/3054512082 >> v1:172.21.15.157:6789/0 conn(0x7f677c0d1a10 legacy=0x7f677c0d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:45.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f677ad9d640 1 -- 172.21.15.29:0/3054512082 >> v1:172.21.15.29:6789/0 conn(0x7f677c06c530 legacy=0x7f677c0b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.302+0000 7f677ad9d640 1 -- 172.21.15.29:0/3054512082 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c06d1b0 con 0x7f677c0b12d0 2024-04-25T00:39:45.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.306+0000 7f677ad9d640 1 --2- 172.21.15.29:0/3054512082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f677c0b12d0 0x7f677c0b21e0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f67740097c0 tx=0x7f677401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d50ea15615446b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:45.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.306+0000 7f677959a640 1 -- 172.21.15.29:0/3054512082 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67740b0070 con 0x7f677c0b12d0 2024-04-25T00:39:45.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.306+0000 7f677959a640 1 -- 172.21.15.29:0/3054512082 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6774004230 con 0x7f677c0b12d0 2024-04-25T00:39:45.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.306+0000 7f677959a640 1 -- 172.21.15.29:0/3054512082 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774003620 con 0x7f677c0b12d0 2024-04-25T00:39:45.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.306+0000 7f678085f640 1 -- 172.21.15.29:0/3054512082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f677c0b12d0 msgr2=0x7f677c0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.306+0000 7f678085f640 1 --2- 172.21.15.29:0/3054512082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f677c0b12d0 0x7f677c0b21e0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f67740097c0 tx=0x7f677401c460 comp rx=0 tx=0).stop 2024-04-25T00:39:45.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.306+0000 7f678085f640 1 -- 172.21.15.29:0/3054512082 shutdown_connections 2024-04-25T00:39:45.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.306+0000 7f678085f640 1 --2- 172.21.15.29:0/3054512082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f677c0b12d0 0x7f677c0b21e0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.306+0000 7f678085f640 1 -- 172.21.15.29:0/3054512082 >> 172.21.15.29:0/3054512082 conn(0x7f677c09aa30 msgr2=0x7f677c09ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:45.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.306+0000 7f678085f640 1 -- 172.21.15.29:0/3054512082 shutdown_connections 2024-04-25T00:39:45.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.306+0000 7f678085f640 1 -- 172.21.15.29:0/3054512082 wait complete. 2024-04-25T00:39:45.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.306+0000 7f678085f640 1 Processor -- start 2024-04-25T00:39:45.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.306+0000 7f678085f640 1 -- start start 2024-04-25T00:39:45.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f678085f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f677c06c530 0x7f677c14cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f678085f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f677c0b12d0 0x7f677c0c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f678085f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f677c0d1a10 0x7f677c0c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f678085f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f677c15e730 con 0x7f677c0d1a10 2024-04-25T00:39:45.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f678085f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f677c15e5b0 con 0x7f677c0b12d0 2024-04-25T00:39:45.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f678085f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f677c06d490 con 0x7f677c06c530 2024-04-25T00:39:45.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f677ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f677c0d1a10 0x7f677c0c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f677ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f677c0d1a10 0x7f677c0c7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37216/0 (socket says 172.21.15.29:37216) 2024-04-25T00:39:45.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f677ad9d640 1 -- 172.21.15.29:0/3294566175 learned_addr learned my addr 172.21.15.29:0/3294566175 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:45.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f6779d9b640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f677c0b12d0 0x7f677c0c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f677a59c640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f677c06c530 0x7f677c14cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f677ad9d640 1 -- 172.21.15.29:0/3294566175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f677c06c530 msgr2=0x7f677c14cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f677ad9d640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f677c06c530 0x7f677c14cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f677ad9d640 1 -- 172.21.15.29:0/3294566175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f677c0b12d0 msgr2=0x7f677c0c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f677ad9d640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f677c0b12d0 0x7f677c0c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f677ad9d640 1 -- 172.21.15.29:0/3294566175 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c0c86c0 con 0x7f677c0d1a10 2024-04-25T00:39:45.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f677ad9d640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f677c0d1a10 0x7f677c0c7690 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f67740ae040 tx=0x7f6774019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:45.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f6772ffd640 1 -- 172.21.15.29:0/3294566175 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67740b0040 con 0x7f677c0d1a10 2024-04-25T00:39:45.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f6772ffd640 1 -- 172.21.15.29:0/3294566175 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67740bb070 con 0x7f677c0d1a10 2024-04-25T00:39:45.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f6772ffd640 1 -- 172.21.15.29:0/3294566175 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774006e20 con 0x7f677c0d1a10 2024-04-25T00:39:45.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f678085f640 1 -- 172.21.15.29:0/3294566175 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c0c7e20 con 0x7f677c0d1a10 2024-04-25T00:39:45.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f678085f640 1 -- 172.21.15.29:0/3294566175 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c0c83e0 con 0x7f677c0d1a10 2024-04-25T00:39:45.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f678085f640 1 -- 172.21.15.29:0/3294566175 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677c069270 con 0x7f677c0d1a10 2024-04-25T00:39:45.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f6772ffd640 1 -- 172.21.15.29:0/3294566175 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f67740b3070 con 0x7f677c0d1a10 2024-04-25T00:39:45.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f6772ffd640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f675c067ef0 0x7f675c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f6772ffd640 1 -- 172.21.15.29:0/3294566175 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4857+0+0 (secure 0 0 0) 0x7f67741201c0 con 0x7f677c0d1a10 2024-04-25T00:39:45.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f677a59c640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f675c067ef0 0x7f675c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.310+0000 7f677a59c640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f675c067ef0 0x7f675c06a3b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f676400b810 tx=0x7f6764008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:45.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.314+0000 7f6772ffd640 1 -- 172.21.15.29:0/3294566175 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6774016050 con 0x7f677c0d1a10 2024-04-25T00:39:45.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.530+0000 7f678085f640 1 -- 172.21.15.29:0/3294566175 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f677c0c88f0 con 0x7f677c0d1a10 2024-04-25T00:39:45.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.534+0000 7f6772ffd640 1 -- 172.21.15.29:0/3294566175 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) ==== 80+0+5304 (secure 0 0 0) 0x7f67740f0770 con 0x7f677c0d1a10 2024-04-25T00:39:45.531 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:45.532 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:39:45.532 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-04-25T00:39:45.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.532 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-04-25T00:39:45.532 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmyzgKAxAAU+nrUJgQhDX6xlARNFI/ng==", 2024-04-25T00:39:45.532 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.532 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:45.532 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:39:45.532 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:39:45.532 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.532 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmtyzPERAAxNUDANw1H/ZzOo+dhgzxlg==", 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmYUl8CxAAj6pV1+s4a0rpWLq2EgZ+8w==", 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:45.534 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:39:45.534 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:39:45.534 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.534 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.534 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-04-25T00:39:45.534 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmuAhKGhAAOLIcWUodW5TqtVgHWZl12A==", 2024-04-25T00:39:45.534 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.534 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:45.534 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:39:45.534 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:39:45.534 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.535 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.535 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-04-25T00:39:45.535 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAzpClmudk6MhAA6YFyd57jwtB2pdOaad7bxQ==", 2024-04-25T00:39:45.535 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.535 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:45.535 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:45.535 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.535 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.535 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA2pClm76q0NRAAxWhEP2ZGoFnZcuiQyGjglw==", 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA6pClmcgULABAA2F5WBR9Df0v4Sn875AzF7g==", 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:45.536 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA9pClm9QuRBhAALFLruv4JU/uNZYMX0yNuXw==", 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBBpClmNj4VDBAAxaxMvyaCvjynPY3f81udnw==", 2024-04-25T00:39:45.537 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBEpClmzBybEBAArry1lM4RIsM/txKfgBqDpw==", 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBHpClm5HfjFhAAbFwJCpbhegLWDu0mbS8FoA==", 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBKpClme/7oHBAAOR2iQ6CweexR7sC1PgesZw==", 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.539 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A==", 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.540 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm8W/EJxAA0HZW24Hhcwf+soSZn6jHJg==", 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmGuQJJRAAhVd4D0jc6Whgyf+B6xgFMw==", 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:39:45.541 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:39:45.542 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.542 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-04-25T00:39:45.542 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmn+ciLRAAg4tQuZ13WP0RnWJaJq5fUQ==", 2024-04-25T00:39:45.542 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.542 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:39:45.542 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:39:45.542 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:39:45.542 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:39:45.542 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmPVihJBAA7PQge/teyRZKSkBnufZSZw==", 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClm6WrdOhAAsY9FLCxG7PGpq2NCY4hGhQ==", 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:39:45.543 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmeR4UOBAANs9SRolOm6E8+/4ciupk5w==", 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:39:45.544 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:39:45.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.534+0000 7f678085f640 1 -- 172.21.15.29:0/3294566175 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f675c067ef0 msgr2=0x7f675c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.534+0000 7f678085f640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f675c067ef0 0x7f675c06a3b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f676400b810 tx=0x7f6764008040 comp rx=0 tx=0).stop 2024-04-25T00:39:45.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.534+0000 7f678085f640 1 -- 172.21.15.29:0/3294566175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f677c0d1a10 msgr2=0x7f677c0c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.534+0000 7f678085f640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f677c0d1a10 0x7f677c0c7690 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f67740ae040 tx=0x7f6774019200 comp rx=0 tx=0).stop 2024-04-25T00:39:45.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.534+0000 7f678085f640 1 -- 172.21.15.29:0/3294566175 shutdown_connections 2024-04-25T00:39:45.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.534+0000 7f678085f640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f675c067ef0 0x7f675c06a3b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.534+0000 7f678085f640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f677c0d1a10 0x7f677c0c7690 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.534+0000 7f678085f640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f677c0b12d0 0x7f677c0c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.534+0000 7f678085f640 1 --2- 172.21.15.29:0/3294566175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f677c06c530 0x7f677c14cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.534+0000 7f678085f640 1 -- 172.21.15.29:0/3294566175 >> 172.21.15.29:0/3294566175 conn(0x7f677c09aa30 msgr2=0x7f677c15d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:45.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.534+0000 7f678085f640 1 -- 172.21.15.29:0/3294566175 shutdown_connections 2024-04-25T00:39:45.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.534+0000 7f678085f640 1 -- 172.21.15.29:0/3294566175 wait complete. 2024-04-25T00:39:45.549 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:45.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.661+0000 7fa6d82be640 1 Processor -- start 2024-04-25T00:39:45.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.661+0000 7fa6d82be640 1 -- start start 2024-04-25T00:39:45.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d82be640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6d015e4e0 0x7fa6d00d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d82be640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa6d00d4bf0 0x7fa6d00d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d82be640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6d00cfe30 con 0x7fa6d015e8c0 2024-04-25T00:39:45.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d82be640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa6d00b6f80 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d82be640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa6d00b7260 con 0x7fa6d015e4e0 2024-04-25T00:39:45.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d5859640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa6d00d4bf0 0x7fa6d00d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d5859640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa6d00d4bf0 0x7fa6d00d23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37222/0 (socket says 172.21.15.29:37222) 2024-04-25T00:39:45.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d5859640 1 -- 172.21.15.29:0/3007431146 learned_addr learned my addr 172.21.15.29:0/3007431146 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:45.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d605a640 1 --2- 172.21.15.29:0/3007431146 >> v2:172.21.15.157:3300/0 conn(0x7fa6d015e4e0 0x7fa6d00d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d5859640 1 -- 172.21.15.29:0/3007431146 >> v2:172.21.15.157:3300/0 conn(0x7fa6d015e4e0 msgr2=0x7fa6d00d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:45.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d5859640 1 --2- 172.21.15.29:0/3007431146 >> v2:172.21.15.157:3300/0 conn(0x7fa6d015e4e0 0x7fa6d00d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d5859640 1 -- 172.21.15.29:0/3007431146 >> v1:172.21.15.29:6789/0 conn(0x7fa6d015e8c0 legacy=0x7fa6d00d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d5859640 1 -- 172.21.15.29:0/3007431146 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d00ced50 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d5859640 1 --2- 172.21.15.29:0/3007431146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6d00d4bf0 0x7fa6d00d23f0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0073380 tx=0x7fa6c001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef1dd827646a046 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:45.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d5058640 1 -- 172.21.15.29:0/3007431146 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c00b4070 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d5058640 1 -- 172.21.15.29:0/3007431146 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c00af400 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.665+0000 7fa6d5058640 1 -- 172.21.15.29:0/3007431146 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c00af7d0 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 -- 172.21.15.29:0/3007431146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6d00d4bf0 msgr2=0x7fa6d00d23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 --2- 172.21.15.29:0/3007431146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6d00d4bf0 0x7fa6d00d23f0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0073380 tx=0x7fa6c001f1e0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 -- 172.21.15.29:0/3007431146 shutdown_connections 2024-04-25T00:39:45.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 --2- 172.21.15.29:0/3007431146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6d00d4bf0 0x7fa6d00d23f0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 --2- 172.21.15.29:0/3007431146 >> v2:172.21.15.157:3300/0 conn(0x7fa6d015e4e0 0x7fa6d00d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 -- 172.21.15.29:0/3007431146 >> 172.21.15.29:0/3007431146 conn(0x7fa6d00d3fb0 msgr2=0x7fa6d00d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:45.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 -- 172.21.15.29:0/3007431146 shutdown_connections 2024-04-25T00:39:45.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 -- 172.21.15.29:0/3007431146 wait complete. 2024-04-25T00:39:45.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 Processor -- start 2024-04-25T00:39:45.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 -- start start 2024-04-25T00:39:45.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6d00d4bf0 0x7fa6d015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d015e4e0 0x7fa6d01583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d015e8c0 0x7fa6d0160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d00d2930 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d00b8a20 con 0x7fa6d015e8c0 2024-04-25T00:39:45.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d00d2ab0 con 0x7fa6d015e4e0 2024-04-25T00:39:45.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d605a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6d00d4bf0 0x7fa6d015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d605a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6d00d4bf0 0x7fa6d015ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37234/0 (socket says 172.21.15.29:37234) 2024-04-25T00:39:45.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d605a640 1 -- 172.21.15.29:0/1591391633 learned_addr learned my addr 172.21.15.29:0/1591391633 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:45.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d605a640 1 -- 172.21.15.29:0/1591391633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d015e4e0 msgr2=0x7fa6d01583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d605a640 1 --2- 172.21.15.29:0/1591391633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d015e4e0 0x7fa6d01583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d605a640 1 -- 172.21.15.29:0/1591391633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d015e8c0 msgr2=0x7fa6d0160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d605a640 1 --2- 172.21.15.29:0/1591391633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d015e8c0 0x7fa6d0160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d605a640 1 -- 172.21.15.29:0/1591391633 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d0161a90 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d605a640 1 --2- 172.21.15.29:0/1591391633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6d00d4bf0 0x7fa6d015ed30 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400d800 tx=0x7fa6c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:45.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6ceffd640 1 -- 172.21.15.29:0/1591391633 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c400b6c0 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6ceffd640 1 -- 172.21.15.29:0/1591391633 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c4019070 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 -- 172.21.15.29:0/1591391633 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d01617b0 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6ceffd640 1 -- 172.21.15.29:0/1591391633 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c400ba60 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 -- 172.21.15.29:0/1591391633 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d0158fc0 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.669+0000 7fa6d82be640 1 -- 172.21.15.29:0/1591391633 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d0069270 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.673+0000 7fa6ceffd640 1 -- 172.21.15.29:0/1591391633 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fa6c400bc00 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.673+0000 7fa6ceffd640 1 --2- 172.21.15.29:0/1591391633 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa6bc067ef0 0x7fa6bc06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.673+0000 7fa6ceffd640 1 -- 172.21.15.29:0/1591391633 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4857+0+0 (secure 0 0 0) 0x7fa6c4087330 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.673+0000 7fa6d5859640 1 --2- 172.21.15.29:0/1591391633 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa6bc067ef0 0x7fa6bc06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.673+0000 7fa6d5859640 1 --2- 172.21.15.29:0/1591391633 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa6bc067ef0 0x7fa6bc06a3b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00b1000 tx=0x7fa6c0004ac0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:45.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.677+0000 7fa6ceffd640 1 -- 172.21.15.29:0/1591391633 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa6c4057a10 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.837+0000 7fa6d82be640 1 -- 172.21.15.29:0/1591391633 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6d00d0ab0 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.837+0000 7fa6ceffd640 1 -- 172.21.15.29:0/1591391633 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+12853 (secure 0 0 0) 0x7fa6c4057310 con 0x7fa6d00d4bf0 2024-04-25T00:39:45.839 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:45.839 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":141,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:45.158213+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:45.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.841+0000 7fa6d82be640 1 -- 172.21.15.29:0/1591391633 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa6bc067ef0 msgr2=0x7fa6bc06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.841+0000 7fa6d82be640 1 --2- 172.21.15.29:0/1591391633 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa6bc067ef0 0x7fa6bc06a3b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00b1000 tx=0x7fa6c0004ac0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.841+0000 7fa6d82be640 1 -- 172.21.15.29:0/1591391633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6d00d4bf0 msgr2=0x7fa6d015ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.841+0000 7fa6d82be640 1 --2- 172.21.15.29:0/1591391633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6d00d4bf0 0x7fa6d015ed30 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400d800 tx=0x7fa6c400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.841+0000 7fa6d82be640 1 -- 172.21.15.29:0/1591391633 shutdown_connections 2024-04-25T00:39:45.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.841+0000 7fa6d82be640 1 --2- 172.21.15.29:0/1591391633 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa6bc067ef0 0x7fa6bc06a3b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.841+0000 7fa6d82be640 1 --2- 172.21.15.29:0/1591391633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d015e8c0 0x7fa6d0160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.841+0000 7fa6d82be640 1 --2- 172.21.15.29:0/1591391633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d015e4e0 0x7fa6d01583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.841+0000 7fa6d82be640 1 --2- 172.21.15.29:0/1591391633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6d00d4bf0 0x7fa6d015ed30 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.841+0000 7fa6d82be640 1 -- 172.21.15.29:0/1591391633 >> 172.21.15.29:0/1591391633 conn(0x7fa6d00d3fb0 msgr2=0x7fa6d00cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:45.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.841+0000 7fa6d82be640 1 -- 172.21.15.29:0/1591391633 shutdown_connections 2024-04-25T00:39:45.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.841+0000 7fa6d82be640 1 -- 172.21.15.29:0/1591391633 wait complete. 2024-04-25T00:39:45.857 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:39:45.857 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:39:45.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf54f4d640 1 Processor -- start 2024-04-25T00:39:45.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf54f4d640 1 -- start start 2024-04-25T00:39:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf54f4d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf5015e410 0x7faf500b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf54f4d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faf5006bc20 0x7faf50070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf54f4d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf500adaf0 con 0x7faf500afa50 2024-04-25T00:39:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf54f4d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7faf500ad8a0 con 0x7faf5006bc20 2024-04-25T00:39:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf54f4d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faf500ae420 con 0x7faf5015e410 2024-04-25T00:39:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf4ed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faf5006bc20 0x7faf50070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf4ed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faf5006bc20 0x7faf50070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37248/0 (socket says 172.21.15.29:37248) 2024-04-25T00:39:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf4ed9d640 1 -- 172.21.15.29:0/3543253357 learned_addr learned my addr 172.21.15.29:0/3543253357 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf4dd9b640 1 --2- 172.21.15.29:0/3543253357 >> v2:172.21.15.157:3300/0 conn(0x7faf5015e410 0x7faf500b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf4ed9d640 1 -- 172.21.15.29:0/3543253357 >> v2:172.21.15.157:3300/0 conn(0x7faf5015e410 msgr2=0x7faf500b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:45.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf4ed9d640 1 --2- 172.21.15.29:0/3543253357 >> v2:172.21.15.157:3300/0 conn(0x7faf5015e410 0x7faf500b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf4ed9d640 1 -- 172.21.15.29:0/3543253357 >> v1:172.21.15.157:6789/0 conn(0x7faf500afa50 legacy=0x7faf500afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:45.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.985+0000 7faf4ed9d640 1 -- 172.21.15.29:0/3543253357 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf500addd0 con 0x7faf5006bc20 2024-04-25T00:39:45.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4ed9d640 1 --2- 172.21.15.29:0/3543253357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf5006bc20 0x7faf50070330 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7faf50050510 tx=0x7faf4801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d011a9f1316ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:45.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4d59a640 1 -- 172.21.15.29:0/3543253357 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf480a5760 con 0x7faf5006bc20 2024-04-25T00:39:45.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4d59a640 1 -- 172.21.15.29:0/3543253357 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf480a5900 con 0x7faf5006bc20 2024-04-25T00:39:45.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4d59a640 1 -- 172.21.15.29:0/3543253357 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf480a5cd0 con 0x7faf5006bc20 2024-04-25T00:39:45.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 -- 172.21.15.29:0/3543253357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf5006bc20 msgr2=0x7faf50070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 --2- 172.21.15.29:0/3543253357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf5006bc20 0x7faf50070330 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7faf50050510 tx=0x7faf4801c980 comp rx=0 tx=0).stop 2024-04-25T00:39:45.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 -- 172.21.15.29:0/3543253357 shutdown_connections 2024-04-25T00:39:45.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 --2- 172.21.15.29:0/3543253357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf5006bc20 0x7faf50070330 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 --2- 172.21.15.29:0/3543253357 >> v2:172.21.15.157:3300/0 conn(0x7faf5015e410 0x7faf500b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 -- 172.21.15.29:0/3543253357 >> 172.21.15.29:0/3543253357 conn(0x7faf500d0c90 msgr2=0x7faf500d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:45.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 -- 172.21.15.29:0/3543253357 shutdown_connections 2024-04-25T00:39:45.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 -- 172.21.15.29:0/3543253357 wait complete. 2024-04-25T00:39:45.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 Processor -- start 2024-04-25T00:39:45.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 -- start start 2024-04-25T00:39:45.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf5006bc20 0x7faf50157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf500afa50 0x7faf50167000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf5015e410 0x7faf50167540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faf500af450 con 0x7faf500afa50 2024-04-25T00:39:45.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faf500af2d0 con 0x7faf5015e410 2024-04-25T00:39:45.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf54f4d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faf500af5d0 con 0x7faf5006bc20 2024-04-25T00:39:45.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf500afa50 0x7faf50167000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf500afa50 0x7faf50167000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37258/0 (socket says 172.21.15.29:37258) 2024-04-25T00:39:45.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4dd9b640 1 -- 172.21.15.29:0/2363387919 learned_addr learned my addr 172.21.15.29:0/2363387919 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:45.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4ed9d640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf5015e410 0x7faf50167540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4e59c640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf5006bc20 0x7faf50157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4dd9b640 1 -- 172.21.15.29:0/2363387919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf5006bc20 msgr2=0x7faf50157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4dd9b640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf5006bc20 0x7faf50157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4dd9b640 1 -- 172.21.15.29:0/2363387919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf5015e410 msgr2=0x7faf50167540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:45.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4dd9b640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf5015e410 0x7faf50167540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:45.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4dd9b640 1 -- 172.21.15.29:0/2363387919 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf500ccea0 con 0x7faf500afa50 2024-04-25T00:39:45.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.989+0000 7faf4ed9d640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf5015e410 0x7faf50167540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:45.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.993+0000 7faf4dd9b640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf500afa50 0x7faf50167000 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7faf4400d800 tx=0x7faf4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:45.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.993+0000 7faf3effd640 1 -- 172.21.15.29:0/2363387919 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4400b6c0 con 0x7faf500afa50 2024-04-25T00:39:45.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.993+0000 7faf3effd640 1 -- 172.21.15.29:0/2363387919 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf44019070 con 0x7faf500afa50 2024-04-25T00:39:45.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.993+0000 7faf54f4d640 1 -- 172.21.15.29:0/2363387919 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf500cd110 con 0x7faf500afa50 2024-04-25T00:39:45.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.993+0000 7faf3effd640 1 -- 172.21.15.29:0/2363387919 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4400ba60 con 0x7faf500afa50 2024-04-25T00:39:45.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.993+0000 7faf54f4d640 1 -- 172.21.15.29:0/2363387919 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf500cd6d0 con 0x7faf500afa50 2024-04-25T00:39:45.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.993+0000 7faf54f4d640 1 -- 172.21.15.29:0/2363387919 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf50069270 con 0x7faf500afa50 2024-04-25T00:39:45.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.993+0000 7faf3effd640 1 -- 172.21.15.29:0/2363387919 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7faf4400bc00 con 0x7faf500afa50 2024-04-25T00:39:45.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.993+0000 7faf3effd640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faf30067ef0 0x7faf3006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:45.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.993+0000 7faf3effd640 1 -- 172.21.15.29:0/2363387919 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4857+0+0 (secure 0 0 0) 0x7faf44087340 con 0x7faf500afa50 2024-04-25T00:39:45.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.993+0000 7faf4e59c640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faf30067ef0 0x7faf3006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:45.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.993+0000 7faf4e59c640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faf30067ef0 0x7faf3006a3b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faf4000a750 tx=0x7faf40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:45.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:45.997+0000 7faf3effd640 1 -- 172.21.15.29:0/2363387919 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faf440579a0 con 0x7faf500afa50 2024-04-25T00:39:46.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.153+0000 7faf54f4d640 1 -- 172.21.15.29:0/2363387919 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faf50155490 con 0x7faf500afa50 2024-04-25T00:39:46.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.153+0000 7faf3effd640 1 -- 172.21.15.29:0/2363387919 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) ==== 93+0+10 (secure 0 0 0) 0x7faf440572a0 con 0x7faf500afa50 2024-04-25T00:39:46.154 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:39:46.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.157+0000 7faf54f4d640 1 -- 172.21.15.29:0/2363387919 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faf30067ef0 msgr2=0x7faf3006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:46.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.157+0000 7faf54f4d640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faf30067ef0 0x7faf3006a3b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faf4000a750 tx=0x7faf40008040 comp rx=0 tx=0).stop 2024-04-25T00:39:46.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.157+0000 7faf54f4d640 1 -- 172.21.15.29:0/2363387919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf500afa50 msgr2=0x7faf50167000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:46.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.157+0000 7faf54f4d640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf500afa50 0x7faf50167000 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7faf4400d800 tx=0x7faf4400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.157+0000 7faf54f4d640 1 -- 172.21.15.29:0/2363387919 shutdown_connections 2024-04-25T00:39:46.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.157+0000 7faf54f4d640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faf30067ef0 0x7faf3006a3b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.157+0000 7faf54f4d640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf5015e410 0x7faf50167540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.157+0000 7faf54f4d640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf500afa50 0x7faf50167000 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.157+0000 7faf54f4d640 1 --2- 172.21.15.29:0/2363387919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf5006bc20 0x7faf50157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.157+0000 7faf54f4d640 1 -- 172.21.15.29:0/2363387919 >> 172.21.15.29:0/2363387919 conn(0x7faf500d0c90 msgr2=0x7faf5014af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:46.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.157+0000 7faf54f4d640 1 -- 172.21.15.29:0/2363387919 shutdown_connections 2024-04-25T00:39:46.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.157+0000 7faf54f4d640 1 -- 172.21.15.29:0/2363387919 wait complete. 2024-04-25T00:39:46.173 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:39:46.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29fcbc640 1 Processor -- start 2024-04-25T00:39:46.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29fcbc640 1 -- start start 2024-04-25T00:39:46.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29fcbc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb29815e200 0x7fb2980d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:46.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29fcbc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb2980d2c60 0x7fb2980b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:46.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29fcbc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2980cfc10 con 0x7fb2980cf690 2024-04-25T00:39:46.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29fcbc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb2980b38d0 con 0x7fb2980d2c60 2024-04-25T00:39:46.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29fcbc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb2980cfef0 con 0x7fb29815e200 2024-04-25T00:39:46.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29e259640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb2980d2c60 0x7fb2980b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:46.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29d257640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb2980cf690 0x7fb2980d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52598/0 (socket says 172.21.15.29:52598) 2024-04-25T00:39:46.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29d257640 1 -- 172.21.15.29:0/2629693825 learned_addr learned my addr 172.21.15.29:0/2629693825 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:46.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29da58640 1 --2- 172.21.15.29:0/2629693825 >> v2:172.21.15.157:3300/0 conn(0x7fb29815e200 0x7fb2980d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:46.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29e259640 1 -- 172.21.15.29:0/2629693825 >> v2:172.21.15.157:3300/0 conn(0x7fb29815e200 msgr2=0x7fb2980d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:46.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29e259640 1 --2- 172.21.15.29:0/2629693825 >> v2:172.21.15.157:3300/0 conn(0x7fb29815e200 0x7fb2980d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29e259640 1 -- 172.21.15.29:0/2629693825 >> v1:172.21.15.29:6789/0 conn(0x7fb2980cf690 legacy=0x7fb2980d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:46.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29e259640 1 -- 172.21.15.29:0/2629693825 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2980b3b20 con 0x7fb2980d2c60 2024-04-25T00:39:46.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.293+0000 7fb29ca56640 1 -- 172.21.15.29:0/2629693825 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 961795785 0 0) 0x7fb2980cfc10 con 0x7fb2980cf690 2024-04-25T00:39:46.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29e259640 1 --2- 172.21.15.29:0/2629693825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2980d2c60 0x7fb2980b4320 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fb29808d8f0 tx=0x7fb29001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e8c7735f5b1e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:46.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29ca56640 1 -- 172.21.15.29:0/2629693825 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900b2070 con 0x7fb2980d2c60 2024-04-25T00:39:46.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29ca56640 1 -- 172.21.15.29:0/2629693825 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2900a59e0 con 0x7fb2980d2c60 2024-04-25T00:39:46.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29ca56640 1 -- 172.21.15.29:0/2629693825 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900a5db0 con 0x7fb2980d2c60 2024-04-25T00:39:46.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 -- 172.21.15.29:0/2629693825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2980d2c60 msgr2=0x7fb2980b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:46.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 --2- 172.21.15.29:0/2629693825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2980d2c60 0x7fb2980b4320 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fb29808d8f0 tx=0x7fb29001c900 comp rx=0 tx=0).stop 2024-04-25T00:39:46.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 -- 172.21.15.29:0/2629693825 shutdown_connections 2024-04-25T00:39:46.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 --2- 172.21.15.29:0/2629693825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2980d2c60 0x7fb2980b4320 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 --2- 172.21.15.29:0/2629693825 >> v2:172.21.15.157:3300/0 conn(0x7fb29815e200 0x7fb2980d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 -- 172.21.15.29:0/2629693825 >> 172.21.15.29:0/2629693825 conn(0x7fb29809d280 msgr2=0x7fb2980d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:46.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 -- 172.21.15.29:0/2629693825 shutdown_connections 2024-04-25T00:39:46.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 -- 172.21.15.29:0/2629693825 wait complete. 2024-04-25T00:39:46.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 Processor -- start 2024-04-25T00:39:46.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 -- start start 2024-04-25T00:39:46.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2980cf690 0x7fb2980cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:46.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980d2c60 0x7fb2981583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:46.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb29815e200 0x7fb298158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:46.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2980b51f0 con 0x7fb29815e200 2024-04-25T00:39:46.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2980b5070 con 0x7fb2980cf690 2024-04-25T00:39:46.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2980b5370 con 0x7fb2980d2c60 2024-04-25T00:39:46.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29e259640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb29815e200 0x7fb298158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:46.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29e259640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb29815e200 0x7fb298158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37268/0 (socket says 172.21.15.29:37268) 2024-04-25T00:39:46.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29e259640 1 -- 172.21.15.29:0/3487893704 learned_addr learned my addr 172.21.15.29:0/3487893704 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:46.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29da58640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2980cf690 0x7fb2980cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:46.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29d257640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980d2c60 0x7fb2981583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:46.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29e259640 1 -- 172.21.15.29:0/3487893704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980d2c60 msgr2=0x7fb2981583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:46.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29e259640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980d2c60 0x7fb2981583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29e259640 1 -- 172.21.15.29:0/3487893704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2980cf690 msgr2=0x7fb2980cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:46.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29e259640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2980cf690 0x7fb2980cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29e259640 1 -- 172.21.15.29:0/3487893704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2980c8600 con 0x7fb29815e200 2024-04-25T00:39:46.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29da58640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2980cf690 0x7fb2980cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:46.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29e259640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb29815e200 0x7fb298158920 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fb2900b0040 tx=0x7fb2900196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:46.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb28e7fc640 1 -- 172.21.15.29:0/3487893704 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900b2040 con 0x7fb29815e200 2024-04-25T00:39:46.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb28e7fc640 1 -- 172.21.15.29:0/3487893704 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2900bf070 con 0x7fb29815e200 2024-04-25T00:39:46.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb28e7fc640 1 -- 172.21.15.29:0/3487893704 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900ad900 con 0x7fb29815e200 2024-04-25T00:39:46.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.297+0000 7fb29fcbc640 1 -- 172.21.15.29:0/3487893704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2980c88e0 con 0x7fb29815e200 2024-04-25T00:39:46.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.301+0000 7fb29fcbc640 1 -- 172.21.15.29:0/3487893704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2980c7d80 con 0x7fb29815e200 2024-04-25T00:39:46.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.301+0000 7fb29fcbc640 1 -- 172.21.15.29:0/3487893704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb298069270 con 0x7fb29815e200 2024-04-25T00:39:46.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.301+0000 7fb28e7fc640 1 -- 172.21.15.29:0/3487893704 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fb2900adaa0 con 0x7fb29815e200 2024-04-25T00:39:46.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.301+0000 7fb28e7fc640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb27c067ef0 0x7fb27c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:46.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.301+0000 7fb28e7fc640 1 -- 172.21.15.29:0/3487893704 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4857+0+0 (secure 0 0 0) 0x7fb290121770 con 0x7fb29815e200 2024-04-25T00:39:46.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.301+0000 7fb29da58640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb27c067ef0 0x7fb27c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:46.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.301+0000 7fb29da58640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb27c067ef0 0x7fb27c06a3b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb2940097a0 tx=0x7fb294008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:46.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.305+0000 7fb28e7fc640 1 -- 172.21.15.29:0/3487893704 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb290016520 con 0x7fb29815e200 2024-04-25T00:39:46.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.461+0000 7fb29fcbc640 1 -- 172.21.15.29:0/3487893704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb2980d1f50 con 0x7fb29815e200 2024-04-25T00:39:46.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.461+0000 7fb28e7fc640 1 -- 172.21.15.29:0/3487893704 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v141) ==== 84+0+65 (secure 0 0 0) 0x7fb2900f1db0 con 0x7fb29815e200 2024-04-25T00:39:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:46.463 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:39:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:39:46.463 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-04-25T00:39:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": ".mgr" 2024-04-25T00:39:46.464 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:46.464 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:39:46.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.465+0000 7fb29fcbc640 1 -- 172.21.15.29:0/3487893704 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb27c067ef0 msgr2=0x7fb27c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:46.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.465+0000 7fb29fcbc640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb27c067ef0 0x7fb27c06a3b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb2940097a0 tx=0x7fb294008040 comp rx=0 tx=0).stop 2024-04-25T00:39:46.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.465+0000 7fb29fcbc640 1 -- 172.21.15.29:0/3487893704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb29815e200 msgr2=0x7fb298158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:46.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.465+0000 7fb29fcbc640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb29815e200 0x7fb298158920 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fb2900b0040 tx=0x7fb2900196a0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.465+0000 7fb29fcbc640 1 -- 172.21.15.29:0/3487893704 shutdown_connections 2024-04-25T00:39:46.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.465+0000 7fb29fcbc640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb27c067ef0 0x7fb27c06a3b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.465+0000 7fb29fcbc640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb29815e200 0x7fb298158920 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.465+0000 7fb29fcbc640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980d2c60 0x7fb2981583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.465+0000 7fb29fcbc640 1 --2- 172.21.15.29:0/3487893704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2980cf690 0x7fb2980cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.465+0000 7fb29fcbc640 1 -- 172.21.15.29:0/3487893704 >> 172.21.15.29:0/3487893704 conn(0x7fb29809d280 msgr2=0x7fb2980b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:46.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.465+0000 7fb29fcbc640 1 -- 172.21.15.29:0/3487893704 shutdown_connections 2024-04-25T00:39:46.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.465+0000 7fb29fcbc640 1 -- 172.21.15.29:0/3487893704 wait complete. 2024-04-25T00:39:46.480 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:39:46.481 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:39:46.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.609+0000 7f250b7ae640 1 Processor -- start 2024-04-25T00:39:46.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.609+0000 7f250b7ae640 1 -- start start 2024-04-25T00:39:46.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.609+0000 7f250b7ae640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f25040b3b50 0x7f25040b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:46.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.609+0000 7f250b7ae640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f250406cd40 0x7f250406d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.609+0000 7f250b7ae640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25040cf670 con 0x7f25041456d0 2024-04-25T00:39:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.609+0000 7f250b7ae640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f25040b4380 con 0x7f25040b3b50 2024-04-25T00:39:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.609+0000 7f250b7ae640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f25040b40a0 con 0x7f250406cd40 2024-04-25T00:39:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.613+0000 7f2508d49640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f25040b3b50 0x7f25040b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.613+0000 7f2508d49640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f25040b3b50 0x7f25040b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37282/0 (socket says 172.21.15.29:37282) 2024-04-25T00:39:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.613+0000 7f2508d49640 1 -- 172.21.15.29:0/3196060880 learned_addr learned my addr 172.21.15.29:0/3196060880 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.613+0000 7f2509d4b640 1 --2- 172.21.15.29:0/3196060880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f250406cd40 0x7f250406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.613+0000 7f2508d49640 1 -- 172.21.15.29:0/3196060880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f250406cd40 msgr2=0x7f250406d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.613+0000 7f2508d49640 1 --2- 172.21.15.29:0/3196060880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f250406cd40 0x7f250406d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.613+0000 7f2508d49640 1 -- 172.21.15.29:0/3196060880 >> v1:172.21.15.157:6789/0 conn(0x7f25041456d0 legacy=0x7f25040cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:46.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.613+0000 7f2508d49640 1 -- 172.21.15.29:0/3196060880 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25040ce480 con 0x7f25040b3b50 2024-04-25T00:39:46.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.613+0000 7f2508d49640 1 --2- 172.21.15.29:0/3196060880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25040b3b50 0x7f25040b2b70 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009870 tx=0x7f24f801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1099eb2f8fd79ce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:46.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.613+0000 7f24f7fff640 1 -- 172.21.15.29:0/3196060880 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f80ab650 con 0x7f25040b3b50 2024-04-25T00:39:46.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.613+0000 7f24f7fff640 1 -- 172.21.15.29:0/3196060880 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f80b0070 con 0x7f25040b3b50 2024-04-25T00:39:46.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.613+0000 7f24f7fff640 1 -- 172.21.15.29:0/3196060880 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f80aba10 con 0x7f25040b3b50 2024-04-25T00:39:46.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 -- 172.21.15.29:0/3196060880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25040b3b50 msgr2=0x7f25040b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:46.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 --2- 172.21.15.29:0/3196060880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25040b3b50 0x7f25040b2b70 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009870 tx=0x7f24f801c560 comp rx=0 tx=0).stop 2024-04-25T00:39:46.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 -- 172.21.15.29:0/3196060880 shutdown_connections 2024-04-25T00:39:46.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 --2- 172.21.15.29:0/3196060880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f250406cd40 0x7f250406d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 --2- 172.21.15.29:0/3196060880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25040b3b50 0x7f25040b2b70 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 -- 172.21.15.29:0/3196060880 >> 172.21.15.29:0/3196060880 conn(0x7f25040b1870 msgr2=0x7f25040b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:46.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 -- 172.21.15.29:0/3196060880 shutdown_connections 2024-04-25T00:39:46.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 -- 172.21.15.29:0/3196060880 wait complete. 2024-04-25T00:39:46.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 Processor -- start 2024-04-25T00:39:46.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 -- start start 2024-04-25T00:39:46.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f250406cd40 0x7f250415ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:46.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25040b3b50 0x7f250415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:46.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25041456d0 0x7f250415b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:46.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f25040b5000 con 0x7f250406cd40 2024-04-25T00:39:46.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2504145ab0 con 0x7f25041456d0 2024-04-25T00:39:46.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2504168b70 con 0x7f25040b3b50 2024-04-25T00:39:46.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250954a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f250406cd40 0x7f250415ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:46.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250954a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f250406cd40 0x7f250415ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37288/0 (socket says 172.21.15.29:37288) 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250954a640 1 -- 172.21.15.29:0/58597611 learned_addr learned my addr 172.21.15.29:0/58597611 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f2509d4b640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25041456d0 0x7f250415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f2508d49640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25040b3b50 0x7f250415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250954a640 1 -- 172.21.15.29:0/58597611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25040b3b50 msgr2=0x7f250415a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250954a640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25040b3b50 0x7f250415a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250954a640 1 -- 172.21.15.29:0/58597611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25041456d0 msgr2=0x7f250415b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250954a640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25041456d0 0x7f250415b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250954a640 1 -- 172.21.15.29:0/58597611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2504164ef0 con 0x7f250406cd40 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250954a640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f250406cd40 0x7f250415ff80 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f250000c420 tx=0x7f250000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f24f5ffb640 1 -- 172.21.15.29:0/58597611 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25000052d0 con 0x7f250406cd40 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f24f5ffb640 1 -- 172.21.15.29:0/58597611 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2500005470 con 0x7f250406cd40 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 -- 172.21.15.29:0/58597611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25041651d0 con 0x7f250406cd40 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f24f5ffb640 1 -- 172.21.15.29:0/58597611 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500007260 con 0x7f250406cd40 2024-04-25T00:39:46.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.617+0000 7f250b7ae640 1 -- 172.21.15.29:0/58597611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2504165790 con 0x7f250406cd40 2024-04-25T00:39:46.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.621+0000 7f250b7ae640 1 -- 172.21.15.29:0/58597611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25040692e0 con 0x7f250406cd40 2024-04-25T00:39:46.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.621+0000 7f24f5ffb640 1 -- 172.21.15.29:0/58597611 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f2500007400 con 0x7f250406cd40 2024-04-25T00:39:46.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.621+0000 7f24f5ffb640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f24e8067ef0 0x7f24e806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:46.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.621+0000 7f24f5ffb640 1 -- 172.21.15.29:0/58597611 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4857+0+0 (secure 0 0 0) 0x7f2500085a60 con 0x7f250406cd40 2024-04-25T00:39:46.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.621+0000 7f2508d49640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f24e8067ef0 0x7f24e806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:46.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.621+0000 7f2508d49640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f24e8067ef0 0x7f24e806a3b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f24f80af000 tx=0x7f24f80a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:46.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.625+0000 7f24f5ffb640 1 -- 172.21.15.29:0/58597611 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2500055f70 con 0x7f250406cd40 2024-04-25T00:39:46.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:46.781+0000 7f250b7ae640 1 -- 172.21.15.29:0/58597611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f25040b4de0 con 0x7f250406cd40 2024-04-25T00:39:47.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.161+0000 7f24f5ffb640 1 -- 172.21.15.29:0/58597611 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v142) ==== 138+0+0 (secure 0 0 0) 0x7f2500055870 con 0x7f250406cd40 2024-04-25T00:39:47.163 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-04-25T00:39:47.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.165+0000 7f250b7ae640 1 -- 172.21.15.29:0/58597611 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f24e8067ef0 msgr2=0x7f24e806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:47.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.165+0000 7f250b7ae640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f24e8067ef0 0x7f24e806a3b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f24f80af000 tx=0x7f24f80a7000 comp rx=0 tx=0).stop 2024-04-25T00:39:47.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.165+0000 7f250b7ae640 1 -- 172.21.15.29:0/58597611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f250406cd40 msgr2=0x7f250415ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:47.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.165+0000 7f250b7ae640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f250406cd40 0x7f250415ff80 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f250000c420 tx=0x7f250000c8e0 comp rx=0 tx=0).stop 2024-04-25T00:39:47.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.165+0000 7f250b7ae640 1 -- 172.21.15.29:0/58597611 shutdown_connections 2024-04-25T00:39:47.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.165+0000 7f250b7ae640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f24e8067ef0 0x7f24e806a3b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:47.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.165+0000 7f250b7ae640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25041456d0 0x7f250415b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:47.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.165+0000 7f250b7ae640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25040b3b50 0x7f250415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:47.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.165+0000 7f250b7ae640 1 --2- 172.21.15.29:0/58597611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f250406cd40 0x7f250415ff80 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:47.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.165+0000 7f250b7ae640 1 -- 172.21.15.29:0/58597611 >> 172.21.15.29:0/58597611 conn(0x7f25040b1870 msgr2=0x7f2504144860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:47.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.165+0000 7f250b7ae640 1 -- 172.21.15.29:0/58597611 shutdown_connections 2024-04-25T00:39:47.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.165+0000 7f250b7ae640 1 -- 172.21.15.29:0/58597611 wait complete. 2024-04-25T00:39:47.194 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:39:47.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.325+0000 7fc4aa200640 1 Processor -- start 2024-04-25T00:39:47.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.325+0000 7fc4aa200640 1 -- start start 2024-04-25T00:39:47.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.325+0000 7fc4aa200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a40b3140 0x7fc4a406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:47.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.325+0000 7fc4aa200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4a40b4b10 0x7fc4a40b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:47.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.325+0000 7fc4aa200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4a40b41e0 con 0x7fc4a40b3a60 2024-04-25T00:39:47.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.325+0000 7fc4aa200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc4a406d960 con 0x7fc4a40b4b10 2024-04-25T00:39:47.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.325+0000 7fc4aa200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a406dc40 con 0x7fc4a40b3140 2024-04-25T00:39:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.325+0000 7fc4a37fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a40b3140 0x7fc4a406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.325+0000 7fc4a2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4a40b4b10 0x7fc4a40b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.325+0000 7fc4a2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4a40b4b10 0x7fc4a40b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:36012/0 (socket says 172.21.15.29:36012) 2024-04-25T00:39:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.325+0000 7fc4a2ffd640 1 -- 172.21.15.29:0/4293299707 learned_addr learned my addr 172.21.15.29:0/4293299707 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.329+0000 7fc4a37fe640 1 -- 172.21.15.29:0/4293299707 >> v2:172.21.15.157:3300/0 conn(0x7fc4a40b4b10 msgr2=0x7fc4a40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.329+0000 7fc4a37fe640 1 --2- 172.21.15.29:0/4293299707 >> v2:172.21.15.157:3300/0 conn(0x7fc4a40b4b10 0x7fc4a40b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.329+0000 7fc4a37fe640 1 -- 172.21.15.29:0/4293299707 >> v1:172.21.15.157:6789/0 conn(0x7fc4a40b3a60 legacy=0x7fc4a415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.329+0000 7fc4a37fe640 1 -- 172.21.15.29:0/4293299707 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a406df20 con 0x7fc4a40b3140 2024-04-25T00:39:47.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.329+0000 7fc4a2ffd640 1 --2- 172.21.15.29:0/4293299707 >> v2:172.21.15.157:3300/0 conn(0x7fc4a40b4b10 0x7fc4a40b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:39:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.333+0000 7fc4a37fe640 1 --2- 172.21.15.29:0/4293299707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a40b3140 0x7fc4a406e440 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc4940097c0 tx=0x7fc49401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebdfb315a75188cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:47.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.333+0000 7fc4a27fc640 1 -- 172.21.15.29:0/4293299707 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940b0070 con 0x7fc4a40b3140 2024-04-25T00:39:47.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.333+0000 7fc4a27fc640 1 -- 172.21.15.29:0/4293299707 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4940ab600 con 0x7fc4a40b3140 2024-04-25T00:39:47.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.333+0000 7fc4a27fc640 1 -- 172.21.15.29:0/4293299707 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940ab9d0 con 0x7fc4a40b3140 2024-04-25T00:39:47.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.333+0000 7fc4aa200640 1 -- 172.21.15.29:0/4293299707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a40b3140 msgr2=0x7fc4a406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:47.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.333+0000 7fc4aa200640 1 --2- 172.21.15.29:0/4293299707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a40b3140 0x7fc4a406e440 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc4940097c0 tx=0x7fc49401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:39:47.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.333+0000 7fc4aa200640 1 -- 172.21.15.29:0/4293299707 shutdown_connections 2024-04-25T00:39:47.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.333+0000 7fc4aa200640 1 --2- 172.21.15.29:0/4293299707 >> v2:172.21.15.157:3300/0 conn(0x7fc4a40b4b10 0x7fc4a40b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:47.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.333+0000 7fc4aa200640 1 --2- 172.21.15.29:0/4293299707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a40b3140 0x7fc4a406e440 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:47.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.333+0000 7fc4aa200640 1 -- 172.21.15.29:0/4293299707 >> 172.21.15.29:0/4293299707 conn(0x7fc4a40b1870 msgr2=0x7fc4a40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:47.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.333+0000 7fc4aa200640 1 -- 172.21.15.29:0/4293299707 shutdown_connections 2024-04-25T00:39:47.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.333+0000 7fc4aa200640 1 -- 172.21.15.29:0/4293299707 wait complete. 2024-04-25T00:39:47.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4aa200640 1 Processor -- start 2024-04-25T00:39:47.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4aa200640 1 -- start start 2024-04-25T00:39:47.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4aa200640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a40b3140 0x7fc4a415b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:47.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4aa200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a40b3a60 0x7fc4a4071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:47.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4aa200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a40b4b10 0x7fc4a413a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:47.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4aa200640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a4168b70 con 0x7fc4a40b3140 2024-04-25T00:39:47.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4aa200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a41689f0 con 0x7fc4a40b3a60 2024-04-25T00:39:47.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4aa200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a4168cf0 con 0x7fc4a40b4b10 2024-04-25T00:39:47.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4a37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a40b3140 0x7fc4a415b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:47.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4a37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a40b3140 0x7fc4a415b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37290/0 (socket says 172.21.15.29:37290) 2024-04-25T00:39:47.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4a37fe640 1 -- 172.21.15.29:0/2262367411 learned_addr learned my addr 172.21.15.29:0/2262367411 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:47.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4a3fff640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a40b4b10 0x7fc4a413a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:47.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4a2ffd640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a40b3a60 0x7fc4a4071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:47.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4a37fe640 1 -- 172.21.15.29:0/2262367411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a40b4b10 msgr2=0x7fc4a413a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:47.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4a37fe640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a40b4b10 0x7fc4a413a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:47.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4a37fe640 1 -- 172.21.15.29:0/2262367411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a40b3a60 msgr2=0x7fc4a4071960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:47.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4a37fe640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a40b3a60 0x7fc4a4071960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:47.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4a37fe640 1 -- 172.21.15.29:0/2262367411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a415b930 con 0x7fc4a40b3140 2024-04-25T00:39:47.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4a37fe640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a40b3140 0x7fc4a415b350 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fc4940ae040 tx=0x7fc49401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:47.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc48bfff640 1 -- 172.21.15.29:0/2262367411 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940b0050 con 0x7fc4a40b3140 2024-04-25T00:39:47.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc48bfff640 1 -- 172.21.15.29:0/2262367411 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4940b1040 con 0x7fc4a40b3140 2024-04-25T00:39:47.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4aa200640 1 -- 172.21.15.29:0/2262367411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a4070e80 con 0x7fc4a40b3140 2024-04-25T00:39:47.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc48bfff640 1 -- 172.21.15.29:0/2262367411 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494006e20 con 0x7fc4a40b3140 2024-04-25T00:39:47.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4aa200640 1 -- 172.21.15.29:0/2262367411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a415abe0 con 0x7fc4a40b3140 2024-04-25T00:39:47.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.337+0000 7fc4aa200640 1 -- 172.21.15.29:0/2262367411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a4069270 con 0x7fc4a40b3140 2024-04-25T00:39:47.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.341+0000 7fc48bfff640 1 -- 172.21.15.29:0/2262367411 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fc4940be030 con 0x7fc4a40b3140 2024-04-25T00:39:47.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.341+0000 7fc48bfff640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc490067ef0 0x7fc49006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:47.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.341+0000 7fc48bfff640 1 -- 172.21.15.29:0/2262367411 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5227+0+0 (secure 0 0 0) 0x7fc494120a00 con 0x7fc4a40b3140 2024-04-25T00:39:47.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.341+0000 7fc4a2ffd640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc490067ef0 0x7fc49006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:47.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.341+0000 7fc4a2ffd640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc490067ef0 0x7fc49006a3b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4142440 tx=0x7fc498008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:47.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.345+0000 7fc48bfff640 1 -- 172.21.15.29:0/2262367411 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc4940c5060 con 0x7fc4a40b3140 2024-04-25T00:39:47.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:47.505+0000 7fc4aa200640 1 -- 172.21.15.29:0/2262367411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc4a41667c0 con 0x7fc4a40b3140 2024-04-25T00:39:48.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.173+0000 7fc48bfff640 1 -- 172.21.15.29:0/2262367411 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v143) ==== 170+0+0 (secure 0 0 0) 0x7fc4940a4e20 con 0x7fc4a40b3140 2024-04-25T00:39:48.174 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-04-25T00:39:48.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.177+0000 7fc4aa200640 1 -- 172.21.15.29:0/2262367411 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc490067ef0 msgr2=0x7fc49006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:48.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.177+0000 7fc4aa200640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc490067ef0 0x7fc49006a3b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4142440 tx=0x7fc498008040 comp rx=0 tx=0).stop 2024-04-25T00:39:48.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.177+0000 7fc4aa200640 1 -- 172.21.15.29:0/2262367411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a40b3140 msgr2=0x7fc4a415b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:48.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.177+0000 7fc4aa200640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a40b3140 0x7fc4a415b350 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fc4940ae040 tx=0x7fc49401ace0 comp rx=0 tx=0).stop 2024-04-25T00:39:48.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.177+0000 7fc4aa200640 1 -- 172.21.15.29:0/2262367411 shutdown_connections 2024-04-25T00:39:48.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.177+0000 7fc4aa200640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc490067ef0 0x7fc49006a3b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:48.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.177+0000 7fc4aa200640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4a40b4b10 0x7fc4a413a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:48.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.177+0000 7fc4aa200640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a40b3a60 0x7fc4a4071960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:48.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.177+0000 7fc4aa200640 1 --2- 172.21.15.29:0/2262367411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a40b3140 0x7fc4a415b350 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.177+0000 7fc4aa200640 1 -- 172.21.15.29:0/2262367411 >> 172.21.15.29:0/2262367411 conn(0x7fc4a40b1870 msgr2=0x7fc4a415d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.177+0000 7fc4aa200640 1 -- 172.21.15.29:0/2262367411 shutdown_connections 2024-04-25T00:39:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.177+0000 7fc4aa200640 1 -- 172.21.15.29:0/2262367411 wait complete. 2024-04-25T00:39:48.210 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:39:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f77755e4640 1 Processor -- start 2024-04-25T00:39:48.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f77755e4640 1 -- start start 2024-04-25T00:39:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f77755e4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777015e8d0 0x7f77700cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f77755e4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77700b4780 0x7f777015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f77755e4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f777015f1f0 0x7f777015b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f77755e4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f77700b42f0 con 0x7f777015f1f0 2024-04-25T00:39:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f77755e4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f77700b3720 con 0x7f77700b4780 2024-04-25T00:39:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f77755e4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f77700b39c0 con 0x7f777015e8d0 2024-04-25T00:39:48.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f776f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f777015f1f0 0x7f777015b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f776f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f777015f1f0 0x7f777015b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49156/0 (socket says 172.21.15.29:49156) 2024-04-25T00:39:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f776f7fe640 1 -- 172.21.15.29:0/1923582778 learned_addr learned my addr 172.21.15.29:0/1923582778 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f776effd640 1 --2- 172.21.15.29:0/1923582778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777015e8d0 0x7f77700cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f7767fff640 1 --2- 172.21.15.29:0/1923582778 >> v2:172.21.15.157:3300/0 conn(0x7f77700b4780 0x7f777015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f776f7fe640 1 -- 172.21.15.29:0/1923582778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777015e8d0 msgr2=0x7f77700cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f776f7fe640 1 --2- 172.21.15.29:0/1923582778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777015e8d0 0x7f77700cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f776f7fe640 1 -- 172.21.15.29:0/1923582778 >> v2:172.21.15.157:3300/0 conn(0x7f77700b4780 msgr2=0x7f777015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f776f7fe640 1 --2- 172.21.15.29:0/1923582778 >> v2:172.21.15.157:3300/0 conn(0x7f77700b4780 0x7f777015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f776f7fe640 1 -- 172.21.15.29:0/1923582778 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77700b3ca0 con 0x7f777015f1f0 2024-04-25T00:39:48.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.337+0000 7f7767fff640 1 --2- 172.21.15.29:0/1923582778 >> v2:172.21.15.157:3300/0 conn(0x7f77700b4780 0x7f777015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:48.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.341+0000 7f776f7fe640 1 --2- 172.21.15.29:0/1923582778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f777015f1f0 0x7f777015b2a0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f7770074fb0 tx=0x7f776801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9eddedc57f6f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:48.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.341+0000 7f776e7fc640 1 -- 172.21.15.29:0/1923582778 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680a5820 con 0x7f777015f1f0 2024-04-25T00:39:48.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.341+0000 7f776e7fc640 1 -- 172.21.15.29:0/1923582778 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77680a59c0 con 0x7f777015f1f0 2024-04-25T00:39:48.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.341+0000 7f776e7fc640 1 -- 172.21.15.29:0/1923582778 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680a5d90 con 0x7f777015f1f0 2024-04-25T00:39:48.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.341+0000 7f77755e4640 1 -- 172.21.15.29:0/1923582778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f777015f1f0 msgr2=0x7f777015b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:48.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.341+0000 7f77755e4640 1 --2- 172.21.15.29:0/1923582778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f777015f1f0 0x7f777015b2a0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f7770074fb0 tx=0x7f776801c9c0 comp rx=0 tx=0).stop 2024-04-25T00:39:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.341+0000 7f77755e4640 1 -- 172.21.15.29:0/1923582778 shutdown_connections 2024-04-25T00:39:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.341+0000 7f77755e4640 1 --2- 172.21.15.29:0/1923582778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f777015f1f0 0x7f777015b2a0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.341+0000 7f77755e4640 1 --2- 172.21.15.29:0/1923582778 >> v2:172.21.15.157:3300/0 conn(0x7f77700b4780 0x7f777015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.341+0000 7f77755e4640 1 --2- 172.21.15.29:0/1923582778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777015e8d0 0x7f77700cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.341+0000 7f77755e4640 1 -- 172.21.15.29:0/1923582778 >> 172.21.15.29:0/1923582778 conn(0x7f77700b1870 msgr2=0x7f77700b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.341+0000 7f77755e4640 1 -- 172.21.15.29:0/1923582778 shutdown_connections 2024-04-25T00:39:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.341+0000 7f77755e4640 1 -- 172.21.15.29:0/1923582778 wait complete. 2024-04-25T00:39:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77755e4640 1 Processor -- start 2024-04-25T00:39:48.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77755e4640 1 -- start start 2024-04-25T00:39:48.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77755e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77700b4780 0x7f7770135c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:48.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77755e4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777015e8d0 0x7f777015a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:48.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77755e4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f777015f1f0 0x7f7770138470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:48.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77755e4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7770144a20 con 0x7f777015f1f0 2024-04-25T00:39:48.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77755e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f777015c7d0 con 0x7f77700b4780 2024-04-25T00:39:48.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77755e4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7770138f10 con 0x7f777015e8d0 2024-04-25T00:39:48.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f776f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f777015f1f0 0x7f7770138470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:48.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f776f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f777015f1f0 0x7f7770138470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49162/0 (socket says 172.21.15.29:49162) 2024-04-25T00:39:48.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f776f7fe640 1 -- 172.21.15.29:0/490262860 learned_addr learned my addr 172.21.15.29:0/490262860 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f7767fff640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777015e8d0 0x7f777015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f776effd640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77700b4780 0x7f7770135c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f776f7fe640 1 -- 172.21.15.29:0/490262860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777015e8d0 msgr2=0x7f777015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f776f7fe640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777015e8d0 0x7f777015a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f776f7fe640 1 -- 172.21.15.29:0/490262860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77700b4780 msgr2=0x7f7770135c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f776f7fe640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77700b4780 0x7f7770135c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f776f7fe640 1 -- 172.21.15.29:0/490262860 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77701361c0 con 0x7f777015f1f0 2024-04-25T00:39:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f776f7fe640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f777015f1f0 0x7f7770138470 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f77680af040 tx=0x7f7768019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77677fe640 1 -- 172.21.15.29:0/490262860 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680b8670 con 0x7f777015f1f0 2024-04-25T00:39:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77677fe640 1 -- 172.21.15.29:0/490262860 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77680be070 con 0x7f777015f1f0 2024-04-25T00:39:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77755e4640 1 -- 172.21.15.29:0/490262860 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77701391b0 con 0x7f777015f1f0 2024-04-25T00:39:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77677fe640 1 -- 172.21.15.29:0/490262860 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680b8a90 con 0x7f777015f1f0 2024-04-25T00:39:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77755e4640 1 -- 172.21.15.29:0/490262860 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7770139770 con 0x7f777015f1f0 2024-04-25T00:39:48.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.345+0000 7f77755e4640 1 -- 172.21.15.29:0/490262860 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7770069270 con 0x7f777015f1f0 2024-04-25T00:39:48.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.349+0000 7f77677fe640 1 -- 172.21.15.29:0/490262860 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f77680b8c30 con 0x7f777015f1f0 2024-04-25T00:39:48.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.349+0000 7f77677fe640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7750067ef0 0x7f775006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:48.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.349+0000 7f77677fe640 1 -- 172.21.15.29:0/490262860 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5609+0+0 (secure 0 0 0) 0x7f7768122390 con 0x7f777015f1f0 2024-04-25T00:39:48.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.349+0000 7f776effd640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7750067ef0 0x7f775006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:48.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.349+0000 7f776effd640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7750067ef0 0x7f775006a3b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f77701f85b0 tx=0x7f7760008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:48.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.353+0000 7f77677fe640 1 -- 172.21.15.29:0/490262860 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f77680165e0 con 0x7f777015f1f0 2024-04-25T00:39:48.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:48.553+0000 7f77755e4640 1 -- 172.21.15.29:0/490262860 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f777006e7b0 con 0x7f777015f1f0 2024-04-25T00:39:49.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.225+0000 7f77677fe640 1 -- 172.21.15.29:0/490262860 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v89) ==== 174+0+0 (secure 0 0 0) 0x7f77680f2750 con 0x7f777015f1f0 2024-04-25T00:39:49.229 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 22 and data pool 23 2024-04-25T00:39:49.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.237+0000 7f77755e4640 1 -- 172.21.15.29:0/490262860 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7750067ef0 msgr2=0x7f775006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.237+0000 7f77755e4640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7750067ef0 0x7f775006a3b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f77701f85b0 tx=0x7f7760008040 comp rx=0 tx=0).stop 2024-04-25T00:39:49.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.237+0000 7f77755e4640 1 -- 172.21.15.29:0/490262860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f777015f1f0 msgr2=0x7f7770138470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.237+0000 7f77755e4640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f777015f1f0 0x7f7770138470 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f77680af040 tx=0x7f7768019760 comp rx=0 tx=0).stop 2024-04-25T00:39:49.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.237+0000 7f77755e4640 1 -- 172.21.15.29:0/490262860 shutdown_connections 2024-04-25T00:39:49.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.237+0000 7f77755e4640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7750067ef0 0x7f775006a3b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.237+0000 7f77755e4640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f777015f1f0 0x7f7770138470 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.237+0000 7f77755e4640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777015e8d0 0x7f777015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.237+0000 7f77755e4640 1 --2- 172.21.15.29:0/490262860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77700b4780 0x7f7770135c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.237+0000 7f77755e4640 1 -- 172.21.15.29:0/490262860 >> 172.21.15.29:0/490262860 conn(0x7f77700b1870 msgr2=0x7f77700b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:49.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.237+0000 7f77755e4640 1 -- 172.21.15.29:0/490262860 shutdown_connections 2024-04-25T00:39:49.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.237+0000 7f77755e4640 1 -- 172.21.15.29:0/490262860 wait complete. 2024-04-25T00:39:49.260 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:49.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.377+0000 7fc22eaa5640 1 Processor -- start 2024-04-25T00:39:49.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.377+0000 7fc22eaa5640 1 -- start start 2024-04-25T00:39:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc22eaa5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc22815e8d0 0x7fc2280cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc22eaa5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2280b3720 con 0x7fc2280b4780 2024-04-25T00:39:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc22eaa5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2280b39c0 con 0x7fc22815f3c0 2024-04-25T00:39:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc22eaa5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc2280b42f0 con 0x7fc22815e8d0 2024-04-25T00:39:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc227fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc2280b4780 0x7fc22815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37710/0 (socket says 172.21.15.29:37710) 2024-04-25T00:39:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc22c841640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc22815e8d0 0x7fc2280cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc227fff640 1 -- 172.21.15.29:0/1882657903 learned_addr learned my addr 172.21.15.29:0/1882657903 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc2277fe640 1 -- 172.21.15.29:0/1882657903 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2886846740 0 0) 0x7fc2280b3720 con 0x7fc2280b4780 2024-04-25T00:39:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc2277fe640 1 -- 172.21.15.29:0/1882657903 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc214003530 con 0x7fc2280b4780 2024-04-25T00:39:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc22c841640 1 -- 172.21.15.29:0/1882657903 >> v1:172.21.15.157:6789/0 conn(0x7fc22815f3c0 legacy=0x7fc22815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc22c841640 1 -- 172.21.15.29:0/1882657903 >> v1:172.21.15.29:6789/0 conn(0x7fc2280b4780 legacy=0x7fc22815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc22c841640 1 -- 172.21.15.29:0/1882657903 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2280b3ca0 con 0x7fc22815e8d0 2024-04-25T00:39:49.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc22c841640 1 --2- 172.21.15.29:0/1882657903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22815e8d0 0x7fc2280cc240 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fc220009d90 tx=0x7fc22001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6961c4797e8021 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:49.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc2277fe640 1 -- 172.21.15.29:0/1882657903 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2200b2070 con 0x7fc22815e8d0 2024-04-25T00:39:49.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc2277fe640 1 -- 172.21.15.29:0/1882657903 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2200a58c0 con 0x7fc22815e8d0 2024-04-25T00:39:49.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.381+0000 7fc2277fe640 1 -- 172.21.15.29:0/1882657903 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2200a5c70 con 0x7fc22815e8d0 2024-04-25T00:39:49.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1882657903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22815e8d0 msgr2=0x7fc2280cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 --2- 172.21.15.29:0/1882657903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22815e8d0 0x7fc2280cc240 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fc220009d90 tx=0x7fc22001c930 comp rx=0 tx=0).stop 2024-04-25T00:39:49.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1882657903 shutdown_connections 2024-04-25T00:39:49.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 --2- 172.21.15.29:0/1882657903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22815e8d0 0x7fc2280cc240 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1882657903 >> 172.21.15.29:0/1882657903 conn(0x7fc2280b1870 msgr2=0x7fc2280b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:49.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1882657903 shutdown_connections 2024-04-25T00:39:49.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1882657903 wait complete. 2024-04-25T00:39:49.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 Processor -- start 2024-04-25T00:39:49.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 -- start start 2024-04-25T00:39:49.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280b4780 0x7fc2281395e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:49.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22815e8d0 0x7fc228142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:49.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc22815f3c0 0x7fc2280b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:49.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc228168e00 con 0x7fc22815e8d0 2024-04-25T00:39:49.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc228168c80 con 0x7fc2280b4780 2024-04-25T00:39:49.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc228168f80 con 0x7fc22815f3c0 2024-04-25T00:39:49.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc227fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22815e8d0 0x7fc228142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc227fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22815e8d0 0x7fc228142e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49174/0 (socket says 172.21.15.29:49174) 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc227fff640 1 -- 172.21.15.29:0/1864294917 learned_addr learned my addr 172.21.15.29:0/1864294917 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22c841640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280b4780 0x7fc2281395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22d042640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc22815f3c0 0x7fc2280b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc227fff640 1 -- 172.21.15.29:0/1864294917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc22815f3c0 msgr2=0x7fc2280b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc227fff640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc22815f3c0 0x7fc2280b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc227fff640 1 -- 172.21.15.29:0/1864294917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280b4780 msgr2=0x7fc2281395e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc227fff640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280b4780 0x7fc2281395e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc227fff640 1 -- 172.21.15.29:0/1864294917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc228139b20 con 0x7fc22815e8d0 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc227fff640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22815e8d0 0x7fc228142e50 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fc21000c450 tx=0x7fc21000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc2257fa640 1 -- 172.21.15.29:0/1864294917 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210017070 con 0x7fc22815e8d0 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc2257fa640 1 -- 172.21.15.29:0/1864294917 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc210005cf0 con 0x7fc22815e8d0 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1864294917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2280cd7e0 con 0x7fc22815e8d0 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc2257fa640 1 -- 172.21.15.29:0/1864294917 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210007290 con 0x7fc22815e8d0 2024-04-25T00:39:49.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.385+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1864294917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2280cdda0 con 0x7fc22815e8d0 2024-04-25T00:39:49.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.389+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1864294917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc228069270 con 0x7fc22815e8d0 2024-04-25T00:39:49.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.389+0000 7fc2257fa640 1 -- 172.21.15.29:0/1864294917 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fc21001a070 con 0x7fc22815e8d0 2024-04-25T00:39:49.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.389+0000 7fc2257fa640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc208067ef0 0x7fc20806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:49.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.389+0000 7fc2257fa640 1 -- 172.21.15.29:0/1864294917 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7fc210085c40 con 0x7fc22815e8d0 2024-04-25T00:39:49.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.389+0000 7fc22c841640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc208067ef0 0x7fc20806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:49.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.389+0000 7fc22c841640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc208067ef0 0x7fc20806a3b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc2200af000 tx=0x7fc2200a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:49.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.393+0000 7fc2257fa640 1 -- 172.21.15.29:0/1864294917 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc210055e00 con 0x7fc22815e8d0 2024-04-25T00:39:49.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.553+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1864294917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc22813d510 con 0x7fc22815e8d0 2024-04-25T00:39:49.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.557+0000 7fc2257fa640 1 -- 172.21.15.29:0/1864294917 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16636 (secure 0 0 0) 0x7fc210055700 con 0x7fc22815e8d0 2024-04-25T00:39:49.558 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:49.558 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":144,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:48.555897+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:46.824888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:47.550815+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:49.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.561+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1864294917 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc208067ef0 msgr2=0x7fc20806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.561+0000 7fc22eaa5640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc208067ef0 0x7fc20806a3b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc2200af000 tx=0x7fc2200a7070 comp rx=0 tx=0).stop 2024-04-25T00:39:49.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.561+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1864294917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22815e8d0 msgr2=0x7fc228142e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.561+0000 7fc22eaa5640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22815e8d0 0x7fc228142e50 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fc21000c450 tx=0x7fc21000c910 comp rx=0 tx=0).stop 2024-04-25T00:39:49.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.561+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1864294917 shutdown_connections 2024-04-25T00:39:49.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.561+0000 7fc22eaa5640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc208067ef0 0x7fc20806a3b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.561+0000 7fc22eaa5640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc22815f3c0 0x7fc2280b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.561+0000 7fc22eaa5640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22815e8d0 0x7fc228142e50 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.561+0000 7fc22eaa5640 1 --2- 172.21.15.29:0/1864294917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280b4780 0x7fc2281395e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.561+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1864294917 >> 172.21.15.29:0/1864294917 conn(0x7fc2280b1870 msgr2=0x7fc2280afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:49.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.561+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1864294917 shutdown_connections 2024-04-25T00:39:49.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.561+0000 7fc22eaa5640 1 -- 172.21.15.29:0/1864294917 wait complete. 2024-04-25T00:39:49.577 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:49.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde818ac640 1 Processor -- start 2024-04-25T00:39:49.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde818ac640 1 -- start start 2024-04-25T00:39:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde818ac640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fde7c0b3140 0x7fde7c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde818ac640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fde7c0b3c30 0x7fde7c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde818ac640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde7c0b41e0 con 0x7fde7c0b4b10 2024-04-25T00:39:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde818ac640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fde7c06d960 con 0x7fde7c0b3c30 2024-04-25T00:39:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde818ac640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fde7c06dc40 con 0x7fde7c0b3140 2024-04-25T00:39:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde7a7fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fde7c0b4b10 0x7fde7c0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37714/0 (socket says 172.21.15.29:37714) 2024-04-25T00:39:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde7b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fde7c0b3c30 0x7fde7c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde7a7fc640 1 -- 172.21.15.29:0/4183873999 learned_addr learned my addr 172.21.15.29:0/4183873999 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:49.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde79ffb640 1 -- 172.21.15.29:0/4183873999 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2606080095 0 0) 0x7fde7c0b41e0 con 0x7fde7c0b4b10 2024-04-25T00:39:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde7affd640 1 --2- 172.21.15.29:0/4183873999 >> v2:172.21.15.157:3300/0 conn(0x7fde7c0b3140 0x7fde7c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde79ffb640 1 -- 172.21.15.29:0/4183873999 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde68003530 con 0x7fde7c0b4b10 2024-04-25T00:39:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde7b7fe640 1 -- 172.21.15.29:0/4183873999 >> v2:172.21.15.157:3300/0 conn(0x7fde7c0b3140 msgr2=0x7fde7c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde7b7fe640 1 --2- 172.21.15.29:0/4183873999 >> v2:172.21.15.157:3300/0 conn(0x7fde7c0b3140 0x7fde7c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde7b7fe640 1 -- 172.21.15.29:0/4183873999 >> v1:172.21.15.29:6789/0 conn(0x7fde7c0b4b10 legacy=0x7fde7c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.697+0000 7fde7b7fe640 1 -- 172.21.15.29:0/4183873999 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c06df20 con 0x7fde7c0b3c30 2024-04-25T00:39:49.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde7b7fe640 1 --2- 172.21.15.29:0/4183873999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde7c0b3c30 0x7fde7c0ce240 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fde740097c0 tx=0x7fde7401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac1bfd4e12ee373 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:49.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde79ffb640 1 -- 172.21.15.29:0/4183873999 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde740b0070 con 0x7fde7c0b3c30 2024-04-25T00:39:49.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde79ffb640 1 -- 172.21.15.29:0/4183873999 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde740ab600 con 0x7fde7c0b3c30 2024-04-25T00:39:49.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde79ffb640 1 -- 172.21.15.29:0/4183873999 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde740ab9b0 con 0x7fde7c0b3c30 2024-04-25T00:39:49.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 -- 172.21.15.29:0/4183873999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde7c0b3c30 msgr2=0x7fde7c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 --2- 172.21.15.29:0/4183873999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde7c0b3c30 0x7fde7c0ce240 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fde740097c0 tx=0x7fde7401c460 comp rx=0 tx=0).stop 2024-04-25T00:39:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 -- 172.21.15.29:0/4183873999 shutdown_connections 2024-04-25T00:39:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 --2- 172.21.15.29:0/4183873999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde7c0b3c30 0x7fde7c0ce240 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 --2- 172.21.15.29:0/4183873999 >> v2:172.21.15.157:3300/0 conn(0x7fde7c0b3140 0x7fde7c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 -- 172.21.15.29:0/4183873999 >> 172.21.15.29:0/4183873999 conn(0x7fde7c0b1870 msgr2=0x7fde7c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 -- 172.21.15.29:0/4183873999 shutdown_connections 2024-04-25T00:39:49.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 -- 172.21.15.29:0/4183873999 wait complete. 2024-04-25T00:39:49.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 Processor -- start 2024-04-25T00:39:49.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 -- start start 2024-04-25T00:39:49.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde7c0b3140 0x7fde7c12b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:49.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde7c0b3c30 0x7fde7c158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:49.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde7c0b4b10 0x7fde7c15ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:49.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c168d80 con 0x7fde7c0b3140 2024-04-25T00:39:49.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c168c00 con 0x7fde7c0b4b10 2024-04-25T00:39:49.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde818ac640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c168f00 con 0x7fde7c0b3c30 2024-04-25T00:39:49.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde7affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde7c0b3140 0x7fde7c12b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde7affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde7c0b3140 0x7fde7c12b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49194/0 (socket says 172.21.15.29:49194) 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde7affd640 1 -- 172.21.15.29:0/3803196045 learned_addr learned my addr 172.21.15.29:0/3803196045 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde7a7fc640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde7c0b3c30 0x7fde7c158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde7affd640 1 -- 172.21.15.29:0/3803196045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde7c0b3c30 msgr2=0x7fde7c158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde7affd640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde7c0b3c30 0x7fde7c158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde7b7fe640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde7c0b4b10 0x7fde7c15ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde7affd640 1 -- 172.21.15.29:0/3803196045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde7c0b4b10 msgr2=0x7fde7c15ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde7affd640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde7c0b4b10 0x7fde7c15ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde7affd640 1 -- 172.21.15.29:0/3803196045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c12bd80 con 0x7fde7c0b3140 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde7affd640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde7c0b3140 0x7fde7c12b840 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00ccd0 tx=0x7fde6c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde73fff640 1 -- 172.21.15.29:0/3803196045 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0039a0 con 0x7fde7c0b3140 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde73fff640 1 -- 172.21.15.29:0/3803196045 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde6c003b40 con 0x7fde7c0b3140 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.701+0000 7fde73fff640 1 -- 172.21.15.29:0/3803196045 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c004020 con 0x7fde7c0b3140 2024-04-25T00:39:49.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.705+0000 7fde818ac640 1 -- 172.21.15.29:0/3803196045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c15dec0 con 0x7fde7c0b3140 2024-04-25T00:39:49.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.705+0000 7fde818ac640 1 -- 172.21.15.29:0/3803196045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c15e480 con 0x7fde7c0b3140 2024-04-25T00:39:49.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.705+0000 7fde818ac640 1 -- 172.21.15.29:0/3803196045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7c069270 con 0x7fde7c0b3140 2024-04-25T00:39:49.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.705+0000 7fde73fff640 1 -- 172.21.15.29:0/3803196045 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fde6c0041c0 con 0x7fde7c0b3140 2024-04-25T00:39:49.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.705+0000 7fde73fff640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fde5c067ef0 0x7fde5c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:49.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.705+0000 7fde73fff640 1 -- 172.21.15.29:0/3803196045 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7fde6c086db0 con 0x7fde7c0b3140 2024-04-25T00:39:49.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.705+0000 7fde7a7fc640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fde5c067ef0 0x7fde5c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:49.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.705+0000 7fde7a7fc640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fde5c067ef0 0x7fde5c06a3b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fde64007fd0 tx=0x7fde64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:49.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.709+0000 7fde73fff640 1 -- 172.21.15.29:0/3803196045 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fde7c12bd80 con 0x7fde7c0b3140 2024-04-25T00:39:49.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.869+0000 7fde818ac640 1 -- 172.21.15.29:0/3803196045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde7c158d80 con 0x7fde7c0b3140 2024-04-25T00:39:49.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.869+0000 7fde73fff640 1 -- 172.21.15.29:0/3803196045 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16636 (secure 0 0 0) 0x7fde6c014030 con 0x7fde7c0b3140 2024-04-25T00:39:49.871 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:49.871 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":144,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:48.555897+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:46.824888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:47.550815+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.873+0000 7fde818ac640 1 -- 172.21.15.29:0/3803196045 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fde5c067ef0 msgr2=0x7fde5c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.873+0000 7fde818ac640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fde5c067ef0 0x7fde5c06a3b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fde64007fd0 tx=0x7fde64009040 comp rx=0 tx=0).stop 2024-04-25T00:39:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.873+0000 7fde818ac640 1 -- 172.21.15.29:0/3803196045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde7c0b3140 msgr2=0x7fde7c12b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.873+0000 7fde818ac640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde7c0b3140 0x7fde7c12b840 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00ccd0 tx=0x7fde6c00a5c0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.873+0000 7fde818ac640 1 -- 172.21.15.29:0/3803196045 shutdown_connections 2024-04-25T00:39:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.873+0000 7fde818ac640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fde5c067ef0 0x7fde5c06a3b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.873+0000 7fde818ac640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde7c0b4b10 0x7fde7c15ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.873+0000 7fde818ac640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde7c0b3c30 0x7fde7c158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.873+0000 7fde818ac640 1 --2- 172.21.15.29:0/3803196045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde7c0b3140 0x7fde7c12b840 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.873+0000 7fde818ac640 1 -- 172.21.15.29:0/3803196045 >> 172.21.15.29:0/3803196045 conn(0x7fde7c0b1870 msgr2=0x7fde7c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.873+0000 7fde818ac640 1 -- 172.21.15.29:0/3803196045 shutdown_connections 2024-04-25T00:39:49.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:49.873+0000 7fde818ac640 1 -- 172.21.15.29:0/3803196045 wait complete. 2024-04-25T00:39:49.889 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:39:50.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.013+0000 7fdcd763b640 1 Processor -- start 2024-04-25T00:39:50.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.013+0000 7fdcd763b640 1 -- start start 2024-04-25T00:39:50.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd763b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdcd00b3c30 0x7fdcd006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:50.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd763b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcd00b41e0 con 0x7fdcd00b3140 2024-04-25T00:39:50.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd763b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcd006d960 con 0x7fdcd00b4b10 2024-04-25T00:39:50.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd763b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdcd006dc40 con 0x7fdcd00b3c30 2024-04-25T00:39:50.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd53d7640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdcd00b3140 0x7fdcd00b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37728/0 (socket says 172.21.15.29:37728) 2024-04-25T00:39:50.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd53d7640 1 -- 172.21.15.29:0/2823865433 learned_addr learned my addr 172.21.15.29:0/2823865433 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:50.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd4bd6640 1 --2- 172.21.15.29:0/2823865433 >> v2:172.21.15.157:3300/0 conn(0x7fdcd00b3c30 0x7fdcd006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:50.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2362349544 0 0) 0x7fdcd00b41e0 con 0x7fdcd00b3140 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcbc003530 con 0x7fdcd00b3140 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 102888577 0 0) 0x7fdcbc003530 con 0x7fdcd00b3140 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 >> v2:172.21.15.157:3300/0 conn(0x7fdcd00b3c30 msgr2=0x7fdcd006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 --2- 172.21.15.29:0/2823865433 >> v2:172.21.15.157:3300/0 conn(0x7fdcd00b3c30 0x7fdcd006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 >> v1:172.21.15.157:6789/0 conn(0x7fdcd00b4b10 legacy=0x7fdcd00ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd006df20 con 0x7fdcd00b3140 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fdcc8001680 con 0x7fdcd00b3140 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 >> v1:172.21.15.29:6789/0 conn(0x7fdcd00b3140 legacy=0x7fdcd00b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 --2- 172.21.15.29:0/2823865433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc0039f0 0x7fdcbc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 --2- 172.21.15.29:0/2823865433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc004810 0x7fdcbc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 --2- 172.21.15.29:0/2823865433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcbc0051d0 0x7fdcbc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd006dc40 con 0x7fdcbc004810 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc00c0a0 con 0x7fdcbc0039f0 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdcbc00c2a0 con 0x7fdcbc0051d0 2024-04-25T00:39:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd4bd6640 1 --2- 172.21.15.29:0/2823865433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc004810 0x7fdcbc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd5bd8640 1 --2- 172.21.15.29:0/2823865433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcbc0051d0 0x7fdcbc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd53d7640 1 --2- 172.21.15.29:0/2823865433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc0039f0 0x7fdcbc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd4bd6640 1 -- 172.21.15.29:0/2823865433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcbc0051d0 msgr2=0x7fdcbc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd4bd6640 1 --2- 172.21.15.29:0/2823865433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcbc0051d0 0x7fdcbc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd4bd6640 1 -- 172.21.15.29:0/2823865433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc0039f0 msgr2=0x7fdcbc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd4bd6640 1 --2- 172.21.15.29:0/2823865433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc0039f0 0x7fdcbc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd4bd6640 1 -- 172.21.15.29:0/2823865433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc003530 con 0x7fdcbc004810 2024-04-25T00:39:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd53d7640 1 --2- 172.21.15.29:0/2823865433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcbc0039f0 0x7fdcbc003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd5bd8640 1 --2- 172.21.15.29:0/2823865433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcbc0051d0 0x7fdcbc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:50.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.017+0000 7fdcd5bd8640 1 -- 172.21.15.29:0/2823865433 reap_dead start 2024-04-25T00:39:50.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd4bd6640 1 --2- 172.21.15.29:0/2823865433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc004810 0x7fdcbc004c90 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009ae0 tx=0x7fdcc001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339c4bd7e601c3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:50.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc00b2070 con 0x7fdcbc004810 2024-04-25T00:39:50.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc00a5660 con 0x7fdcbc004810 2024-04-25T00:39:50.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcc7fff640 1 -- 172.21.15.29:0/2823865433 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc00096b0 con 0x7fdcbc004810 2024-04-25T00:39:50.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 -- 172.21.15.29:0/2823865433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc004810 msgr2=0x7fdcbc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:50.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 --2- 172.21.15.29:0/2823865433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc004810 0x7fdcbc004c90 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009ae0 tx=0x7fdcc001ca70 comp rx=0 tx=0).stop 2024-04-25T00:39:50.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 -- 172.21.15.29:0/2823865433 shutdown_connections 2024-04-25T00:39:50.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 --2- 172.21.15.29:0/2823865433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc004810 0x7fdcbc004c90 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:50.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 -- 172.21.15.29:0/2823865433 >> 172.21.15.29:0/2823865433 conn(0x7fdcd00b1870 msgr2=0x7fdcd00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:50.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 -- 172.21.15.29:0/2823865433 shutdown_connections 2024-04-25T00:39:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 -- 172.21.15.29:0/2823865433 wait complete. 2024-04-25T00:39:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 Processor -- start 2024-04-25T00:39:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 -- start start 2024-04-25T00:39:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc004810 0x7fdcd013f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcd0141a20 0x7fdcd013c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcd01e31d0 0x7fdcd01e3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd0145a90 con 0x7fdcbc004810 2024-04-25T00:39:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd0145910 con 0x7fdcd01e31d0 2024-04-25T00:39:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd006e440 con 0x7fdcd0141a20 2024-04-25T00:39:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd53d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc004810 0x7fdcd013f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd53d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc004810 0x7fdcd013f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49226/0 (socket says 172.21.15.29:49226) 2024-04-25T00:39:50.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd53d7640 1 -- 172.21.15.29:0/1774189513 learned_addr learned my addr 172.21.15.29:0/1774189513 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:50.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd4bd6640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcd0141a20 0x7fdcd013c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:50.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd5bd8640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcd01e31d0 0x7fdcd01e3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:50.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd53d7640 1 -- 172.21.15.29:0/1774189513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcd0141a20 msgr2=0x7fdcd013c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:50.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd53d7640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcd0141a20 0x7fdcd013c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:50.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd53d7640 1 -- 172.21.15.29:0/1774189513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcd01e31d0 msgr2=0x7fdcd01e3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:50.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd53d7640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcd01e31d0 0x7fdcd01e3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:50.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd53d7640 1 -- 172.21.15.29:0/1774189513 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd013fd80 con 0x7fdcbc004810 2024-04-25T00:39:50.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd53d7640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc004810 0x7fdcd013f6f0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8010c40 tx=0x7fdcc800e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:50.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcc5ffb640 1 -- 172.21.15.29:0/1774189513 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc801b070 con 0x7fdcbc004810 2024-04-25T00:39:50.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcc5ffb640 1 -- 172.21.15.29:0/1774189513 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc8011040 con 0x7fdcbc004810 2024-04-25T00:39:50.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcd763b640 1 -- 172.21.15.29:0/1774189513 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd0140060 con 0x7fdcbc004810 2024-04-25T00:39:50.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.021+0000 7fdcc5ffb640 1 -- 172.21.15.29:0/1774189513 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80145c0 con 0x7fdcbc004810 2024-04-25T00:39:50.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.025+0000 7fdcd763b640 1 -- 172.21.15.29:0/1774189513 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd0140620 con 0x7fdcbc004810 2024-04-25T00:39:50.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.025+0000 7fdcd763b640 1 -- 172.21.15.29:0/1774189513 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd0069270 con 0x7fdcbc004810 2024-04-25T00:39:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.025+0000 7fdcc5ffb640 1 -- 172.21.15.29:0/1774189513 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fdcc801e070 con 0x7fdcbc004810 2024-04-25T00:39:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.025+0000 7fdcc5ffb640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdcb4067ef0 0x7fdcb406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.025+0000 7fdcc5ffb640 1 -- 172.21.15.29:0/1774189513 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7fdcc8089570 con 0x7fdcbc004810 2024-04-25T00:39:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.025+0000 7fdcd4bd6640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdcb4067ef0 0x7fdcb406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:50.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.025+0000 7fdcd4bd6640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdcb4067ef0 0x7fdcb406a3b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00af000 tx=0x7fdcc0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:50.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.029+0000 7fdcc5ffb640 1 -- 172.21.15.29:0/1774189513 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdcc8059730 con 0x7fdcbc004810 2024-04-25T00:39:50.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:50.221+0000 7fdcd763b640 1 -- 172.21.15.29:0/1774189513 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdcd013d1e0 con 0x7fdcbc004810 2024-04-25T00:39:51.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.229+0000 7fdcc5ffb640 1 -- 172.21.15.29:0/1774189513 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v91) ==== 118+0+0 (secure 0 0 0) 0x7fdcc8059030 con 0x7fdcbc004810 2024-04-25T00:39:51.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.233+0000 7fdcd763b640 1 -- 172.21.15.29:0/1774189513 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdcb4067ef0 msgr2=0x7fdcb406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:51.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.233+0000 7fdcd763b640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdcb4067ef0 0x7fdcb406a3b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00af000 tx=0x7fdcc0008040 comp rx=0 tx=0).stop 2024-04-25T00:39:51.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.233+0000 7fdcd763b640 1 -- 172.21.15.29:0/1774189513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc004810 msgr2=0x7fdcd013f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:51.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.233+0000 7fdcd763b640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc004810 0x7fdcd013f6f0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8010c40 tx=0x7fdcc800e4d0 comp rx=0 tx=0).stop 2024-04-25T00:39:51.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.233+0000 7fdcd763b640 1 -- 172.21.15.29:0/1774189513 shutdown_connections 2024-04-25T00:39:51.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.233+0000 7fdcd763b640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdcb4067ef0 0x7fdcb406a3b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:51.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.233+0000 7fdcd763b640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcd01e31d0 0x7fdcd01e3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:51.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.233+0000 7fdcd763b640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcd0141a20 0x7fdcd013c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:51.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.233+0000 7fdcd763b640 1 --2- 172.21.15.29:0/1774189513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc004810 0x7fdcd013f6f0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:51.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.233+0000 7fdcd763b640 1 -- 172.21.15.29:0/1774189513 >> 172.21.15.29:0/1774189513 conn(0x7fdcd00b1870 msgr2=0x7fdcd013d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:51.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.233+0000 7fdcd763b640 1 -- 172.21.15.29:0/1774189513 shutdown_connections 2024-04-25T00:39:51.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.233+0000 7fdcd763b640 1 -- 172.21.15.29:0/1774189513 wait complete. 2024-04-25T00:39:51.253 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:39:51.253 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:39:51.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152ea61640 1 Processor -- start 2024-04-25T00:39:51.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152ea61640 1 -- start start 2024-04-25T00:39:51.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152ea61640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f152806b210 0x7f152806db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:51.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152ea61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15280d18f0 0x7f152815f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:51.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152ea61640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f152815f730 con 0x7f152806b5f0 2024-04-25T00:39:51.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152ea61640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f15280d0410 con 0x7f152806b210 2024-04-25T00:39:51.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152ea61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f15280ce880 con 0x7f15280d18f0 2024-04-25T00:39:51.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152cffe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f152806b5f0 0x7f1528071fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37730/0 (socket says 172.21.15.29:37730) 2024-04-25T00:39:51.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152cffe640 1 -- 172.21.15.29:0/1088581554 learned_addr learned my addr 172.21.15.29:0/1088581554 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:51.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1527fff640 1 --2- 172.21.15.29:0/1088581554 >> v2:172.21.15.157:3300/0 conn(0x7f152806b210 0x7f152806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:51.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 345689457 0 0) 0x7f1520001120 con 0x7f152806b5f0 2024-04-25T00:39:51.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1510003530 con 0x7f152806b5f0 2024-04-25T00:39:51.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f15277fe640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15280d18f0 0x7f152815f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:51.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2906714609 0 0) 0x7f1510003530 con 0x7f152806b5f0 2024-04-25T00:39:51.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15280d18f0 msgr2=0x7f152815f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15280d18f0 0x7f152815f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 >> v2:172.21.15.157:3300/0 conn(0x7f152806b210 msgr2=0x7f152806db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 --2- 172.21.15.29:0/1088581554 >> v2:172.21.15.157:3300/0 conn(0x7f152806b210 0x7f152806db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15280d06b0 con 0x7f152806b5f0 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f1520001850 con 0x7f152806b5f0 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f15277fe640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15280d18f0 0x7f152815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 >> v1:172.21.15.29:6789/0 conn(0x7f152806b5f0 legacy=0x7f1528071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15280d18f0 0x7f15100039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1510003f00 0x7f1510004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1510004d50 0x7f15100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f15280d0410 con 0x7f1510004d50 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15280ce880 con 0x7f1510003f00 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f151000ba00 con 0x7f15280d18f0 2024-04-25T00:39:51.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152cffe640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1510004d50 0x7f15100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:51.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f1527fff640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15280d18f0 0x7f15100039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:51.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f15277fe640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1510003f00 0x7f1510004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:51.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152cffe640 1 -- 172.21.15.29:0/1088581554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15280d18f0 msgr2=0x7f15100039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:51.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152cffe640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15280d18f0 0x7f15100039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:51.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152cffe640 1 -- 172.21.15.29:0/1088581554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1510003f00 msgr2=0x7f1510004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:51.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152cffe640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1510003f00 0x7f1510004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:51.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f152cffe640 1 -- 172.21.15.29:0/1088581554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1510003530 con 0x7f1510004d50 2024-04-25T00:39:51.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.385+0000 7f15277fe640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1510003f00 0x7f1510004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:51.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.389+0000 7f152cffe640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1510004d50 0x7f15100051d0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f152000c490 tx=0x7f152001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298f9ae3ffc61fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.389+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152000c0e0 con 0x7f1510004d50 2024-04-25T00:39:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.389+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15200a9020 con 0x7f1510004d50 2024-04-25T00:39:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.389+0000 7f1526ffd640 1 -- 172.21.15.29:0/1088581554 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15200a5ba0 con 0x7f1510004d50 2024-04-25T00:39:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.389+0000 7f152ea61640 1 -- 172.21.15.29:0/1088581554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1510004d50 msgr2=0x7f15100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:51.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.389+0000 7f152ea61640 1 --2- 172.21.15.29:0/1088581554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1510004d50 0x7f15100051d0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f152000c490 tx=0x7f152001cca0 comp rx=0 tx=0).stop 2024-04-25T00:39:51.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.389+0000 7f152cffe640 1 -- 172.21.15.29:0/1088581554 reap_dead start 2024-04-25T00:39:51.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.389+0000 7f152ea61640 1 -- 172.21.15.29:0/1088581554 shutdown_connections 2024-04-25T00:39:51.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.389+0000 7f152ea61640 1 -- 172.21.15.29:0/1088581554 >> 172.21.15.29:0/1088581554 conn(0x7f15280d0cb0 msgr2=0x7f15280d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:51.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.389+0000 7f152ea61640 1 -- 172.21.15.29:0/1088581554 shutdown_connections 2024-04-25T00:39:51.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.389+0000 7f152ea61640 1 -- 172.21.15.29:0/1088581554 wait complete. 2024-04-25T00:39:51.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.389+0000 7f152ea61640 1 Processor -- start 2024-04-25T00:39:51.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.389+0000 7f152ea61640 1 -- start start 2024-04-25T00:39:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f152ea61640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15281572c0 0x7f15280ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f152ea61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15281576a0 0x7f15280aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f152ea61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152815b350 0x7f152815b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f152ea61640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f15280cee00 con 0x7f15281572c0 2024-04-25T00:39:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f152ea61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15280ceb20 con 0x7f15281576a0 2024-04-25T00:39:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f152ea61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f152806ba70 con 0x7f152815b350 2024-04-25T00:39:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1527fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15281572c0 0x7f15280ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1527fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15281572c0 0x7f15280ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49248/0 (socket says 172.21.15.29:49248) 2024-04-25T00:39:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1527fff640 1 -- 172.21.15.29:0/3390713463 learned_addr learned my addr 172.21.15.29:0/3390713463 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f15277fe640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15281576a0 0x7f15280aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f152cffe640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152815b350 0x7f152815b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1527fff640 1 -- 172.21.15.29:0/3390713463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152815b350 msgr2=0x7f152815b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1527fff640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152815b350 0x7f152815b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1527fff640 1 -- 172.21.15.29:0/3390713463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15281576a0 msgr2=0x7f15280aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1527fff640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15281576a0 0x7f15280aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1527fff640 1 -- 172.21.15.29:0/3390713463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152815bd10 con 0x7f15281572c0 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f152cffe640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152815b350 0x7f152815b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1527fff640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15281572c0 0x7f15280ae900 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f151c011980 tx=0x7f151c011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1524ff9640 1 -- 172.21.15.29:0/3390713463 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c00faf0 con 0x7f15281572c0 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1524ff9640 1 -- 172.21.15.29:0/3390713463 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151c00fc90 con 0x7f15281572c0 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f152ea61640 1 -- 172.21.15.29:0/3390713463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15281f3cc0 con 0x7f15281572c0 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1524ff9640 1 -- 172.21.15.29:0/3390713463 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c014440 con 0x7f15281572c0 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f152ea61640 1 -- 172.21.15.29:0/3390713463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15281f3f80 con 0x7f15281572c0 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f152ea61640 1 -- 172.21.15.29:0/3390713463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15281f46a0 con 0x7f15281572c0 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1524ff9640 1 -- 172.21.15.29:0/3390713463 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f151c012040 con 0x7f15281572c0 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1524ff9640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1514067ef0 0x7f151406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f15277fe640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1514067ef0 0x7f151406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:51.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.393+0000 7f1524ff9640 1 -- 172.21.15.29:0/3390713463 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f151c086e40 con 0x7f15281572c0 2024-04-25T00:39:51.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.397+0000 7f15277fe640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1514067ef0 0x7f151406a3b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f15180095c0 tx=0x7f151800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:51.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.401+0000 7f1524ff9640 1 -- 172.21.15.29:0/3390713463 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f151c08b070 con 0x7f15281572c0 2024-04-25T00:39:51.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:51.597+0000 7f152ea61640 1 -- 172.21.15.29:0/3390713463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1528069250 con 0x7f15281572c0 2024-04-25T00:39:52.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.237+0000 7f1524ff9640 1 -- 172.21.15.29:0/3390713463 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v92) ==== 122+0+0 (secure 0 0 0) 0x7f151c019030 con 0x7f15281572c0 2024-04-25T00:39:52.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.241+0000 7f152ea61640 1 -- 172.21.15.29:0/3390713463 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1514067ef0 msgr2=0x7f151406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:52.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.241+0000 7f152ea61640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1514067ef0 0x7f151406a3b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f15180095c0 tx=0x7f151800b040 comp rx=0 tx=0).stop 2024-04-25T00:39:52.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.241+0000 7f152ea61640 1 -- 172.21.15.29:0/3390713463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15281572c0 msgr2=0x7f15280ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:52.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.241+0000 7f152ea61640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15281572c0 0x7f15280ae900 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f151c011980 tx=0x7f151c011e40 comp rx=0 tx=0).stop 2024-04-25T00:39:52.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.241+0000 7f152ea61640 1 -- 172.21.15.29:0/3390713463 shutdown_connections 2024-04-25T00:39:52.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.241+0000 7f152ea61640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1514067ef0 0x7f151406a3b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.241+0000 7f152ea61640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152815b350 0x7f152815b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.241+0000 7f152ea61640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15281576a0 0x7f15280aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.241+0000 7f152ea61640 1 --2- 172.21.15.29:0/3390713463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15281572c0 0x7f15280ae900 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.241+0000 7f152ea61640 1 -- 172.21.15.29:0/3390713463 >> 172.21.15.29:0/3390713463 conn(0x7f15280d0cb0 msgr2=0x7f152806db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:52.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.241+0000 7f152ea61640 1 -- 172.21.15.29:0/3390713463 shutdown_connections 2024-04-25T00:39:52.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.241+0000 7f152ea61640 1 -- 172.21.15.29:0/3390713463 wait complete. 2024-04-25T00:39:52.264 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:39:52.264 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:39:52.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2857749640 1 Processor -- start 2024-04-25T00:39:52.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2857749640 1 -- start start 2024-04-25T00:39:52.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2857749640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f285015c4c0 0x7f28500b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:52.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2857749640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28500b3a10 0x7f28500715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:52.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2857749640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28500afe20 con 0x7f28500ce100 2024-04-25T00:39:52.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2857749640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f285015c9f0 con 0x7f28500b3a10 2024-04-25T00:39:52.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2857749640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f285015ccd0 con 0x7f285015c4c0 2024-04-25T00:39:52.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2855ce6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28500b3a10 0x7f28500715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:52.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2855ce6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28500b3a10 0x7f28500715b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49256/0 (socket says 172.21.15.29:49256) 2024-04-25T00:39:52.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2855ce6640 1 -- 172.21.15.29:0/2472928948 learned_addr learned my addr 172.21.15.29:0/2472928948 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:52.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2854ce4640 1 --2- 172.21.15.29:0/2472928948 >> v2:172.21.15.157:3300/0 conn(0x7f285015c4c0 0x7f28500b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:52.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2855ce6640 1 -- 172.21.15.29:0/2472928948 >> v2:172.21.15.157:3300/0 conn(0x7f285015c4c0 msgr2=0x7f28500b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:52.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2855ce6640 1 --2- 172.21.15.29:0/2472928948 >> v2:172.21.15.157:3300/0 conn(0x7f285015c4c0 0x7f28500b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2855ce6640 1 -- 172.21.15.29:0/2472928948 >> v1:172.21.15.157:6789/0 conn(0x7f28500ce100 legacy=0x7f28500cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:52.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.389+0000 7f2855ce6640 1 -- 172.21.15.29:0/2472928948 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285015cfb0 con 0x7f28500b3a10 2024-04-25T00:39:52.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2855ce6640 1 --2- 172.21.15.29:0/2472928948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28500b3a10 0x7f28500715b0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f28500cfa10 tx=0x7f284801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc837755c438364 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:52.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2847fff640 1 -- 172.21.15.29:0/2472928948 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28480ab620 con 0x7f28500b3a10 2024-04-25T00:39:52.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2847fff640 1 -- 172.21.15.29:0/2472928948 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28480b0070 con 0x7f28500b3a10 2024-04-25T00:39:52.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2847fff640 1 -- 172.21.15.29:0/2472928948 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28480ab9e0 con 0x7f28500b3a10 2024-04-25T00:39:52.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 -- 172.21.15.29:0/2472928948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28500b3a10 msgr2=0x7f28500715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:52.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 --2- 172.21.15.29:0/2472928948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28500b3a10 0x7f28500715b0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f28500cfa10 tx=0x7f284801c420 comp rx=0 tx=0).stop 2024-04-25T00:39:52.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 -- 172.21.15.29:0/2472928948 shutdown_connections 2024-04-25T00:39:52.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 --2- 172.21.15.29:0/2472928948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28500b3a10 0x7f28500715b0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 --2- 172.21.15.29:0/2472928948 >> v2:172.21.15.157:3300/0 conn(0x7f285015c4c0 0x7f28500b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 -- 172.21.15.29:0/2472928948 >> 172.21.15.29:0/2472928948 conn(0x7f2850075030 msgr2=0x7f28500b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:52.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 -- 172.21.15.29:0/2472928948 shutdown_connections 2024-04-25T00:39:52.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 -- 172.21.15.29:0/2472928948 wait complete. 2024-04-25T00:39:52.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 Processor -- start 2024-04-25T00:39:52.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 -- start start 2024-04-25T00:39:52.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28500ce100 0x7f28501f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:52.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f285015c4c0 0x7f28501fa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:52.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28500ad8a0 0x7f28500add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:52.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2850071f00 con 0x7f28500ad8a0 2024-04-25T00:39:52.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f28500b59f0 con 0x7f285015c4c0 2024-04-25T00:39:52.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2857749640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2850168c20 con 0x7f28500ce100 2024-04-25T00:39:52.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2855ce6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28500ad8a0 0x7f28500add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:52.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2855ce6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28500ad8a0 0x7f28500add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49262/0 (socket says 172.21.15.29:49262) 2024-04-25T00:39:52.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2855ce6640 1 -- 172.21.15.29:0/4263469386 learned_addr learned my addr 172.21.15.29:0/4263469386 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:52.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f28554e5640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28500ce100 0x7f28501f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:52.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.393+0000 7f2854ce4640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f285015c4c0 0x7f28501fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:52.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2855ce6640 1 -- 172.21.15.29:0/4263469386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28500ce100 msgr2=0x7f28501f8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:52.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2855ce6640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28500ce100 0x7f28501f8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2855ce6640 1 -- 172.21.15.29:0/4263469386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f285015c4c0 msgr2=0x7f28501fa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:52.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2855ce6640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f285015c4c0 0x7f28501fa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2855ce6640 1 -- 172.21.15.29:0/4263469386 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28501f8370 con 0x7f28500ad8a0 2024-04-25T00:39:52.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2855ce6640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28500ad8a0 0x7f28500add20 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f284801aad0 tx=0x7f284801ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:52.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2845ffb640 1 -- 172.21.15.29:0/4263469386 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28480abd00 con 0x7f28500ad8a0 2024-04-25T00:39:52.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2845ffb640 1 -- 172.21.15.29:0/4263469386 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28480be3f0 con 0x7f28500ad8a0 2024-04-25T00:39:52.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2857749640 1 -- 172.21.15.29:0/4263469386 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285006dc10 con 0x7f28500ad8a0 2024-04-25T00:39:52.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2845ffb640 1 -- 172.21.15.29:0/4263469386 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28480be7a0 con 0x7f28500ad8a0 2024-04-25T00:39:52.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2857749640 1 -- 172.21.15.29:0/4263469386 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28501fa990 con 0x7f28500ad8a0 2024-04-25T00:39:52.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2857749640 1 -- 172.21.15.29:0/4263469386 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2850069270 con 0x7f28500ad8a0 2024-04-25T00:39:52.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2845ffb640 1 -- 172.21.15.29:0/4263469386 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f28480c7440 con 0x7f28500ad8a0 2024-04-25T00:39:52.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2845ffb640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2834067ef0 0x7f283406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:52.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f2845ffb640 1 -- 172.21.15.29:0/4263469386 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f2848120da0 con 0x7f28500ad8a0 2024-04-25T00:39:52.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f28554e5640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2834067ef0 0x7f283406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:52.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.397+0000 7f28554e5640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2834067ef0 0x7f283406a3b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f284c001dc0 tx=0x7f284c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:52.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.401+0000 7f2845ffb640 1 -- 172.21.15.29:0/4263469386 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2848126020 con 0x7f28500ad8a0 2024-04-25T00:39:52.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.593+0000 7f2857749640 1 -- 172.21.15.29:0/4263469386 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2850156940 con 0x7f28500ad8a0 2024-04-25T00:39:52.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.593+0000 7f2845ffb640 1 -- 172.21.15.29:0/4263469386 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5185 (secure 0 0 0) 0x7f28480160d0 con 0x7f28500ad8a0 2024-04-25T00:39:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:52.595 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":92,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3075204329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3075204329},{"type":"v1","addr":"172.21.15.157:6833","nonce":3075204329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3377772964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3377772964},{"type":"v1","addr":"172.21.15.157:6835","nonce":3377772964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":9621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/897131811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":897131811},{"type":"v1","addr":"172.21.15.29:6833","nonce":897131811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:39:48.556647+0000","modified":"2024-04-25T00:39:51.600640+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9624":{"gid":9624,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/821427804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":821427804},{"type":"v1","addr":"172.21.15.29:6835","nonce":821427804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9624,"qdb_cluster":[9624]},"id":11}]} 2024-04-25T00:39:52.596 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 92 2024-04-25T00:39:52.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.597+0000 7f2857749640 1 -- 172.21.15.29:0/4263469386 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2834067ef0 msgr2=0x7f283406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:52.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.597+0000 7f2857749640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2834067ef0 0x7f283406a3b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f284c001dc0 tx=0x7f284c009040 comp rx=0 tx=0).stop 2024-04-25T00:39:52.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.597+0000 7f2857749640 1 -- 172.21.15.29:0/4263469386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28500ad8a0 msgr2=0x7f28500add20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:52.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.597+0000 7f2857749640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28500ad8a0 0x7f28500add20 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f284801aad0 tx=0x7f284801ac60 comp rx=0 tx=0).stop 2024-04-25T00:39:52.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.597+0000 7f2857749640 1 -- 172.21.15.29:0/4263469386 shutdown_connections 2024-04-25T00:39:52.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.597+0000 7f2857749640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2834067ef0 0x7f283406a3b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.597+0000 7f2857749640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28500ad8a0 0x7f28500add20 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.597+0000 7f2857749640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f285015c4c0 0x7f28501fa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.597+0000 7f2857749640 1 --2- 172.21.15.29:0/4263469386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28500ce100 0x7f28501f8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.597+0000 7f2857749640 1 -- 172.21.15.29:0/4263469386 >> 172.21.15.29:0/4263469386 conn(0x7f2850075030 msgr2=0x7f2850066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:52.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.597+0000 7f2857749640 1 -- 172.21.15.29:0/4263469386 shutdown_connections 2024-04-25T00:39:52.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.597+0000 7f2857749640 1 -- 172.21.15.29:0/4263469386 wait complete. 2024-04-25T00:39:52.614 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:39:52.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd58b522640 1 Processor -- start 2024-04-25T00:39:52.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd58b522640 1 -- start start 2024-04-25T00:39:52.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd58b522640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd5840b3c30 0x7fd58406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd58b522640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5840b4b10 0x7fd5840ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd58b522640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5840b41e0 con 0x7fd5840b3140 2024-04-25T00:39:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd58b522640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd58406d960 con 0x7fd5840b3c30 2024-04-25T00:39:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd58b522640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd58406dc40 con 0x7fd5840b4b10 2024-04-25T00:39:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd588abd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd5840b3c30 0x7fd58406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd588abd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd5840b3c30 0x7fd58406e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49270/0 (socket says 172.21.15.29:49270) 2024-04-25T00:39:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd588abd640 1 -- 172.21.15.29:0/4245661017 learned_addr learned my addr 172.21.15.29:0/4245661017 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:52.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd589abf640 1 --2- 172.21.15.29:0/4245661017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5840b4b10 0x7fd5840ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:52.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd588abd640 1 -- 172.21.15.29:0/4245661017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5840b4b10 msgr2=0x7fd5840ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:52.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd588abd640 1 --2- 172.21.15.29:0/4245661017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5840b4b10 0x7fd5840ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd588abd640 1 -- 172.21.15.29:0/4245661017 >> v1:172.21.15.157:6789/0 conn(0x7fd5840b3140 legacy=0x7fd5840b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:52.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd588abd640 1 -- 172.21.15.29:0/4245661017 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58406df20 con 0x7fd5840b3c30 2024-04-25T00:39:52.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.737+0000 7fd589abf640 1 --2- 172.21.15.29:0/4245661017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5840b4b10 0x7fd5840ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:52.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd588abd640 1 --2- 172.21.15.29:0/4245661017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5840b3c30 0x7fd58406e440 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fd5740098a0 tx=0x7fd57401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e0974614564f71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:52.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd57bfff640 1 -- 172.21.15.29:0/4245661017 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5740b0070 con 0x7fd5840b3c30 2024-04-25T00:39:52.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd57bfff640 1 -- 172.21.15.29:0/4245661017 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5740ab600 con 0x7fd5840b3c30 2024-04-25T00:39:52.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd57bfff640 1 -- 172.21.15.29:0/4245661017 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5740ab9d0 con 0x7fd5840b3c30 2024-04-25T00:39:52.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 -- 172.21.15.29:0/4245661017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5840b3c30 msgr2=0x7fd58406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:52.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 --2- 172.21.15.29:0/4245661017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5840b3c30 0x7fd58406e440 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fd5740098a0 tx=0x7fd57401c500 comp rx=0 tx=0).stop 2024-04-25T00:39:52.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 -- 172.21.15.29:0/4245661017 shutdown_connections 2024-04-25T00:39:52.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 --2- 172.21.15.29:0/4245661017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5840b4b10 0x7fd5840ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 --2- 172.21.15.29:0/4245661017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5840b3c30 0x7fd58406e440 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 -- 172.21.15.29:0/4245661017 >> 172.21.15.29:0/4245661017 conn(0x7fd5840b1870 msgr2=0x7fd5840b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:52.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 -- 172.21.15.29:0/4245661017 shutdown_connections 2024-04-25T00:39:52.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 -- 172.21.15.29:0/4245661017 wait complete. 2024-04-25T00:39:52.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 Processor -- start 2024-04-25T00:39:52.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 -- start start 2024-04-25T00:39:52.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5840b3140 0x7fd5841397c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:52.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5840b3c30 0x7fd584143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:52.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5840b4b10 0x7fd584144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:52.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd584168d80 con 0x7fd5840b4b10 2024-04-25T00:39:52.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd584168c00 con 0x7fd5840b3c30 2024-04-25T00:39:52.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd58b522640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd584168f00 con 0x7fd5840b3140 2024-04-25T00:39:52.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd589abf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5840b4b10 0x7fd584144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:52.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd589abf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5840b4b10 0x7fd584144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49282/0 (socket says 172.21.15.29:49282) 2024-04-25T00:39:52.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd589abf640 1 -- 172.21.15.29:0/792112222 learned_addr learned my addr 172.21.15.29:0/792112222 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:52.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd5892be640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5840b3140 0x7fd5841397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:52.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd588abd640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5840b3c30 0x7fd584143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:52.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd589abf640 1 -- 172.21.15.29:0/792112222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5840b3140 msgr2=0x7fd5841397c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:52.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd589abf640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5840b3140 0x7fd5841397c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd589abf640 1 -- 172.21.15.29:0/792112222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5840b3c30 msgr2=0x7fd584143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:52.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd589abf640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5840b3c30 0x7fd584143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd589abf640 1 -- 172.21.15.29:0/792112222 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd584159bb0 con 0x7fd5840b4b10 2024-04-25T00:39:52.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd589abf640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5840b4b10 0x7fd584144460 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00cef0 tx=0x7fd57c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:52.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.741+0000 7fd579ffb640 1 -- 172.21.15.29:0/792112222 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c01a070 con 0x7fd5840b4b10 2024-04-25T00:39:52.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.745+0000 7fd579ffb640 1 -- 172.21.15.29:0/792112222 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd57c00d860 con 0x7fd5840b4b10 2024-04-25T00:39:52.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.745+0000 7fd58b522640 1 -- 172.21.15.29:0/792112222 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd584159e50 con 0x7fd5840b4b10 2024-04-25T00:39:52.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.745+0000 7fd579ffb640 1 -- 172.21.15.29:0/792112222 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00dc50 con 0x7fd5840b4b10 2024-04-25T00:39:52.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.745+0000 7fd58b522640 1 -- 172.21.15.29:0/792112222 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58415a410 con 0x7fd5840b4b10 2024-04-25T00:39:52.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.745+0000 7fd58b522640 1 -- 172.21.15.29:0/792112222 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd584069270 con 0x7fd5840b4b10 2024-04-25T00:39:52.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.745+0000 7fd579ffb640 1 -- 172.21.15.29:0/792112222 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fd57c01d070 con 0x7fd5840b4b10 2024-04-25T00:39:52.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.745+0000 7fd579ffb640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd568067ef0 0x7fd56806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:52.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.745+0000 7fd579ffb640 1 -- 172.21.15.29:0/792112222 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fd57c087030 con 0x7fd5840b4b10 2024-04-25T00:39:52.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.745+0000 7fd5892be640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd568067ef0 0x7fd56806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:52.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.745+0000 7fd5892be640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd568067ef0 0x7fd56806a3b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd58000b810 tx=0x7fd580008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:52.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.749+0000 7fd579ffb640 1 -- 172.21.15.29:0/792112222 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd57c057380 con 0x7fd5840b4b10 2024-04-25T00:39:52.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.913+0000 7fd58b522640 1 -- 172.21.15.29:0/792112222 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5840ce310 con 0x7fd5840b4b10 2024-04-25T00:39:52.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.913+0000 7fd579ffb640 1 -- 172.21.15.29:0/792112222 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16623 (secure 0 0 0) 0x7fd57c056c80 con 0x7fd5840b4b10 2024-04-25T00:39:52.915 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:52.915 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":145,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:50.179122+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:46.824888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:47.550815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:39:52.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.917+0000 7fd58b522640 1 -- 172.21.15.29:0/792112222 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd568067ef0 msgr2=0x7fd56806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:52.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.917+0000 7fd58b522640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd568067ef0 0x7fd56806a3b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd58000b810 tx=0x7fd580008040 comp rx=0 tx=0).stop 2024-04-25T00:39:52.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.917+0000 7fd58b522640 1 -- 172.21.15.29:0/792112222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5840b4b10 msgr2=0x7fd584144460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:52.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.917+0000 7fd58b522640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5840b4b10 0x7fd584144460 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00cef0 tx=0x7fd57c00a710 comp rx=0 tx=0).stop 2024-04-25T00:39:52.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.917+0000 7fd58b522640 1 -- 172.21.15.29:0/792112222 shutdown_connections 2024-04-25T00:39:52.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.917+0000 7fd58b522640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd568067ef0 0x7fd56806a3b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.917+0000 7fd58b522640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5840b4b10 0x7fd584144460 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.917+0000 7fd58b522640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5840b3c30 0x7fd584143f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.917+0000 7fd58b522640 1 --2- 172.21.15.29:0/792112222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5840b3140 0x7fd5841397c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:52.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.917+0000 7fd58b522640 1 -- 172.21.15.29:0/792112222 >> 172.21.15.29:0/792112222 conn(0x7fd5840b1870 msgr2=0x7fd5840b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:52.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.917+0000 7fd58b522640 1 -- 172.21.15.29:0/792112222 shutdown_connections 2024-04-25T00:39:52.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:52.917+0000 7fd58b522640 1 -- 172.21.15.29:0/792112222 wait complete. 2024-04-25T00:39:52.935 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:39:52.935 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:39:53.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.049+0000 7f1cdcb72640 1 Processor -- start 2024-04-25T00:39:53.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.049+0000 7f1cdcb72640 1 -- start start 2024-04-25T00:39:53.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cdcb72640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1cd806e440 0x7f1cd80b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cdcb72640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd80b41e0 con 0x7f1cd80b3140 2024-04-25T00:39:53.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cdcb72640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd806d960 con 0x7f1cd80b3c30 2024-04-25T00:39:53.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cdcb72640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd80b5050 con 0x7f1cd806e440 2024-04-25T00:39:53.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd659c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1cd80b3140 0x7f1cd80b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37732/0 (socket says 172.21.15.29:37732) 2024-04-25T00:39:53.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd659c640 1 -- 172.21.15.29:0/908454465 learned_addr learned my addr 172.21.15.29:0/908454465 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd6d9d640 1 --2- 172.21.15.29:0/908454465 >> v2:172.21.15.157:3300/0 conn(0x7f1cd806e440 0x7f1cd80b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3714705902 0 0) 0x7f1cd80b41e0 con 0x7f1cd80b3140 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cc0003530 con 0x7f1cd80b3140 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2206236439 0 0) 0x7f1cc0003530 con 0x7f1cd80b3140 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 >> v2:172.21.15.157:3300/0 conn(0x7f1cd806e440 msgr2=0x7f1cd80b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 --2- 172.21.15.29:0/908454465 >> v2:172.21.15.157:3300/0 conn(0x7f1cd806e440 0x7f1cd80b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 >> v1:172.21.15.157:6789/0 conn(0x7f1cd80b3c30 legacy=0x7f1cd80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd806dc40 con 0x7f1cd80b3140 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f1ccc001680 con 0x7f1cd80b3140 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 >> v1:172.21.15.29:6789/0 conn(0x7f1cd80b3140 legacy=0x7f1cd80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 --2- 172.21.15.29:0/908454465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cc00039c0 0x7f1cc0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 --2- 172.21.15.29:0/908454465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cc0004810 0x7f1cc0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 --2- 172.21.15.29:0/908454465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cc00051d0 0x7f1cc000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd80b5050 con 0x7f1cc00051d0 2024-04-25T00:39:53.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc000c070 con 0x7f1cc0004810 2024-04-25T00:39:53.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc000c270 con 0x7f1cc00039c0 2024-04-25T00:39:53.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd6d9d640 1 --2- 172.21.15.29:0/908454465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cc00051d0 0x7f1cc000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd5d9b640 1 --2- 172.21.15.29:0/908454465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cc0004810 0x7f1cc0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd659c640 1 --2- 172.21.15.29:0/908454465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cc00039c0 0x7f1cc0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd6d9d640 1 -- 172.21.15.29:0/908454465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cc00039c0 msgr2=0x7f1cc0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:53.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd6d9d640 1 --2- 172.21.15.29:0/908454465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cc00039c0 0x7f1cc0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd6d9d640 1 -- 172.21.15.29:0/908454465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cc0004810 msgr2=0x7f1cc0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:53.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd6d9d640 1 --2- 172.21.15.29:0/908454465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cc0004810 0x7f1cc0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd6d9d640 1 -- 172.21.15.29:0/908454465 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc0003530 con 0x7f1cc00051d0 2024-04-25T00:39:53.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.053+0000 7f1cd659c640 1 --2- 172.21.15.29:0/908454465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cc00039c0 0x7f1cc0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:53.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd6d9d640 1 -- 172.21.15.29:0/908454465 reap_dead start 2024-04-25T00:39:53.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd6d9d640 1 --2- 172.21.15.29:0/908454465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cc00051d0 0x7f1cc000b9d0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009fa0 tx=0x7f1cd001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875004398f4cfb76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00b2070 con 0x7f1cc00051d0 2024-04-25T00:39:53.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd00a5d50 con 0x7f1cc00051d0 2024-04-25T00:39:53.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd559a640 1 -- 172.21.15.29:0/908454465 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0009bf0 con 0x7f1cc00051d0 2024-04-25T00:39:53.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 -- 172.21.15.29:0/908454465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cc00051d0 msgr2=0x7f1cc000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 --2- 172.21.15.29:0/908454465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cc00051d0 0x7f1cc000b9d0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009fa0 tx=0x7f1cd001cc80 comp rx=0 tx=0).stop 2024-04-25T00:39:53.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 -- 172.21.15.29:0/908454465 shutdown_connections 2024-04-25T00:39:53.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 --2- 172.21.15.29:0/908454465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cc00051d0 0x7f1cc000b9d0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 -- 172.21.15.29:0/908454465 >> 172.21.15.29:0/908454465 conn(0x7f1cd80b1870 msgr2=0x7f1cd80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:53.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 -- 172.21.15.29:0/908454465 shutdown_connections 2024-04-25T00:39:53.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 -- 172.21.15.29:0/908454465 wait complete. 2024-04-25T00:39:53.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 Processor -- start 2024-04-25T00:39:53.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 -- start start 2024-04-25T00:39:53.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cc00051d0 0x7f1cd813c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd8157740 0x7f1cd8157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cd8152350 0x7f1cd81527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd8145a90 con 0x7f1cc00051d0 2024-04-25T00:39:53.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd8145910 con 0x7f1cd8157740 2024-04-25T00:39:53.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cdcb72640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd80b4b10 con 0x7f1cd8152350 2024-04-25T00:39:53.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cc00051d0 0x7f1cd813c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cc00051d0 0x7f1cd813c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49300/0 (socket says 172.21.15.29:49300) 2024-04-25T00:39:53.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd659c640 1 -- 172.21.15.29:0/3365677469 learned_addr learned my addr 172.21.15.29:0/3365677469 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:53.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd6d9d640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cd8152350 0x7f1cd81527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd5d9b640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd8157740 0x7f1cd8157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd659c640 1 -- 172.21.15.29:0/3365677469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cd8152350 msgr2=0x7f1cd81527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd659c640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cd8152350 0x7f1cd81527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd659c640 1 -- 172.21.15.29:0/3365677469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd8157740 msgr2=0x7f1cd8157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd659c640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd8157740 0x7f1cd8157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.057+0000 7f1cd659c640 1 -- 172.21.15.29:0/3365677469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd81f1b60 con 0x7f1cc00051d0 2024-04-25T00:39:53.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.061+0000 7f1cd659c640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cc00051d0 0x7f1cd813c7f0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc001210 tx=0x7f1ccc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.061+0000 7f1cc6ffd640 1 -- 172.21.15.29:0/3365677469 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc019070 con 0x7f1cc00051d0 2024-04-25T00:39:53.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.061+0000 7f1cc6ffd640 1 -- 172.21.15.29:0/3365677469 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ccc0074e0 con 0x7f1cc00051d0 2024-04-25T00:39:53.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.061+0000 7f1cdcb72640 1 -- 172.21.15.29:0/3365677469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd8153080 con 0x7f1cc00051d0 2024-04-25T00:39:53.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.061+0000 7f1cc6ffd640 1 -- 172.21.15.29:0/3365677469 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0049f0 con 0x7f1cc00051d0 2024-04-25T00:39:53.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.061+0000 7f1cdcb72640 1 -- 172.21.15.29:0/3365677469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd8153640 con 0x7f1cc00051d0 2024-04-25T00:39:53.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.061+0000 7f1cdcb72640 1 -- 172.21.15.29:0/3365677469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd8069250 con 0x7f1cc00051d0 2024-04-25T00:39:53.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.061+0000 7f1cc6ffd640 1 -- 172.21.15.29:0/3365677469 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f1ccc01c070 con 0x7f1cc00051d0 2024-04-25T00:39:53.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.061+0000 7f1cc6ffd640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1cb8067f20 0x7f1cb806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.061+0000 7f1cd5d9b640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1cb8067f20 0x7f1cb806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.061+0000 7f1cc6ffd640 1 -- 172.21.15.29:0/3365677469 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f1ccc086370 con 0x7f1cc00051d0 2024-04-25T00:39:53.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.061+0000 7f1cd5d9b640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1cb8067f20 0x7f1cb806a3e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1cd815b690 tx=0x7f1cc8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.065+0000 7f1cc6ffd640 1 -- 172.21.15.29:0/3365677469 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1ccc056460 con 0x7f1cc00051d0 2024-04-25T00:39:53.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.217+0000 7f1cdcb72640 1 -- 172.21.15.29:0/3365677469 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1cd80b38a0 con 0x7f1cb8067f20 2024-04-25T00:39:53.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.233+0000 7f1cc6ffd640 1 -- 172.21.15.29:0/3365677469 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252413 (secure 0 0 0) 0x7f1cd80b38a0 con 0x7f1cb8067f20 2024-04-25T00:39:53.236 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:53.238 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:39:53.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.237+0000 7f1cdcb72640 1 -- 172.21.15.29:0/3365677469 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1cb8067f20 msgr2=0x7f1cb806a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.237+0000 7f1cdcb72640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1cb8067f20 0x7f1cb806a3e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1cd815b690 tx=0x7f1cc8009040 comp rx=0 tx=0).stop 2024-04-25T00:39:53.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.237+0000 7f1cdcb72640 1 -- 172.21.15.29:0/3365677469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cc00051d0 msgr2=0x7f1cd813c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.237+0000 7f1cdcb72640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cc00051d0 0x7f1cd813c7f0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc001210 tx=0x7f1ccc00dd40 comp rx=0 tx=0).stop 2024-04-25T00:39:53.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.237+0000 7f1cdcb72640 1 -- 172.21.15.29:0/3365677469 shutdown_connections 2024-04-25T00:39:53.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.237+0000 7f1cdcb72640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1cb8067f20 0x7f1cb806a3e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.237+0000 7f1cdcb72640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cd8152350 0x7f1cd81527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.237+0000 7f1cdcb72640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd8157740 0x7f1cd8157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.237+0000 7f1cdcb72640 1 --2- 172.21.15.29:0/3365677469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cc00051d0 0x7f1cd813c7f0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.237+0000 7f1cdcb72640 1 -- 172.21.15.29:0/3365677469 >> 172.21.15.29:0/3365677469 conn(0x7f1cd80b1870 msgr2=0x7f1cd80ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:53.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.237+0000 7f1cdcb72640 1 -- 172.21.15.29:0/3365677469 shutdown_connections 2024-04-25T00:39:53.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.237+0000 7f1cdcb72640 1 -- 172.21.15.29:0/3365677469 wait complete. 2024-04-25T00:39:53.254 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-04-25T00:39:51.392095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306268,"kb_used_data":15208,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754668452,"statfs":{"total":773094113280,"available":772780494848,"internally_reserved":0,"allocated":15572992,"data_stored":7524512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.225899"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.233217+0000","last_change":"2024-04-25T00:39:49.230883+0000","last_active":"2024-04-25T00:39:49.233217+0000","last_peered":"2024-04-25T00:39:49.233217+0000","last_clean":"2024-04-25T00:39:49.233217+0000","last_became_active":"2024-04-25T00:39:49.230326+0000","last_became_peered":"2024-04-25T00:39:49.230326+0000","last_unstale":"2024-04-25T00:39:49.233217+0000","last_undegraded":"2024-04-25T00:39:49.233217+0000","last_fullsized":"2024-04-25T00:39:49.233217+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:26.408586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.229501+0000","last_change":"2024-04-25T00:39:49.227757+0000","last_active":"2024-04-25T00:39:49.229501+0000","last_peered":"2024-04-25T00:39:49.229501+0000","last_clean":"2024-04-25T00:39:49.229501+0000","last_became_active":"2024-04-25T00:39:49.227102+0000","last_became_peered":"2024-04-25T00:39:49.227102+0000","last_unstale":"2024-04-25T00:39:49.229501+0000","last_undegraded":"2024-04-25T00:39:49.229501+0000","last_fullsized":"2024-04-25T00:39:49.229501+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:24.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.221724+0000","last_change":"2024-04-25T00:39:49.216106+0000","last_active":"2024-04-25T00:39:49.221724+0000","last_peered":"2024-04-25T00:39:49.221724+0000","last_clean":"2024-04-25T00:39:49.221724+0000","last_became_active":"2024-04-25T00:39:49.211605+0000","last_became_peered":"2024-04-25T00:39:49.211605+0000","last_unstale":"2024-04-25T00:39:49.221724+0000","last_undegraded":"2024-04-25T00:39:49.221724+0000","last_fullsized":"2024-04-25T00:39:49.221724+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:29.679950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.223801+0000","last_change":"2024-04-25T00:39:49.213789+0000","last_active":"2024-04-25T00:39:49.223801+0000","last_peered":"2024-04-25T00:39:49.223801+0000","last_clean":"2024-04-25T00:39:49.223801+0000","last_became_active":"2024-04-25T00:39:49.211043+0000","last_became_peered":"2024-04-25T00:39:49.211043+0000","last_unstale":"2024-04-25T00:39:49.223801+0000","last_undegraded":"2024-04-25T00:39:49.223801+0000","last_fullsized":"2024-04-25T00:39:49.223801+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:13.116403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.214214+0000","last_change":"2024-04-25T00:39:49.211378+0000","last_active":"2024-04-25T00:39:49.214214+0000","last_peered":"2024-04-25T00:39:49.214214+0000","last_clean":"2024-04-25T00:39:49.214214+0000","last_became_active":"2024-04-25T00:39:49.210693+0000","last_became_peered":"2024-04-25T00:39:49.210693+0000","last_unstale":"2024-04-25T00:39:49.214214+0000","last_undegraded":"2024-04-25T00:39:49.214214+0000","last_fullsized":"2024-04-25T00:39:49.214214+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:13.621503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.214066+0000","last_change":"2024-04-25T00:39:49.210790+0000","last_active":"2024-04-25T00:39:49.214066+0000","last_peered":"2024-04-25T00:39:49.214066+0000","last_clean":"2024-04-25T00:39:49.214066+0000","last_became_active":"2024-04-25T00:39:49.210011+0000","last_became_peered":"2024-04-25T00:39:49.210011+0000","last_unstale":"2024-04-25T00:39:49.214066+0000","last_undegraded":"2024-04-25T00:39:49.214066+0000","last_fullsized":"2024-04-25T00:39:49.214066+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:45.058512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.231963+0000","last_change":"2024-04-25T00:39:49.222555+0000","last_active":"2024-04-25T00:39:49.231963+0000","last_peered":"2024-04-25T00:39:49.231963+0000","last_clean":"2024-04-25T00:39:49.231963+0000","last_became_active":"2024-04-25T00:39:49.221627+0000","last_became_peered":"2024-04-25T00:39:49.221627+0000","last_unstale":"2024-04-25T00:39:49.231963+0000","last_undegraded":"2024-04-25T00:39:49.231963+0000","last_fullsized":"2024-04-25T00:39:49.231963+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:21.291228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.215033+0000","last_change":"2024-04-25T00:39:49.212394+0000","last_active":"2024-04-25T00:39:49.215033+0000","last_peered":"2024-04-25T00:39:49.215033+0000","last_clean":"2024-04-25T00:39:49.215033+0000","last_became_active":"2024-04-25T00:39:49.210667+0000","last_became_peered":"2024-04-25T00:39:49.210667+0000","last_unstale":"2024-04-25T00:39:49.215033+0000","last_undegraded":"2024-04-25T00:39:49.215033+0000","last_fullsized":"2024-04-25T00:39:49.215033+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:20.018270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.233047+0000","last_change":"2024-04-25T00:39:49.231597+0000","last_active":"2024-04-25T00:39:49.233047+0000","last_peered":"2024-04-25T00:39:49.233047+0000","last_clean":"2024-04-25T00:39:49.233047+0000","last_became_active":"2024-04-25T00:39:49.230883+0000","last_became_peered":"2024-04-25T00:39:49.230883+0000","last_unstale":"2024-04-25T00:39:49.233047+0000","last_undegraded":"2024-04-25T00:39:49.233047+0000","last_fullsized":"2024-04-25T00:39:49.233047+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:43.826776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.210727+0000","last_change":"2024-04-25T00:39:49.206851+0000","last_active":"2024-04-25T00:39:49.210727+0000","last_peered":"2024-04-25T00:39:49.210727+0000","last_clean":"2024-04-25T00:39:49.210727+0000","last_became_active":"2024-04-25T00:39:49.206093+0000","last_became_peered":"2024-04-25T00:39:49.206093+0000","last_unstale":"2024-04-25T00:39:49.210727+0000","last_undegraded":"2024-04-25T00:39:49.210727+0000","last_fullsized":"2024-04-25T00:39:49.210727+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:16.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.221431+0000","last_change":"2024-04-25T00:39:49.218840+0000","last_active":"2024-04-25T00:39:49.221431+0000","last_peered":"2024-04-25T00:39:49.221431+0000","last_clean":"2024-04-25T00:39:49.221431+0000","last_became_active":"2024-04-25T00:39:49.218051+0000","last_became_peered":"2024-04-25T00:39:49.218051+0000","last_unstale":"2024-04-25T00:39:49.221431+0000","last_undegraded":"2024-04-25T00:39:49.221431+0000","last_fullsized":"2024-04-25T00:39:49.221431+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:13.116403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.229829+0000","last_change":"2024-04-25T00:39:49.227651+0000","last_active":"2024-04-25T00:39:49.229829+0000","last_peered":"2024-04-25T00:39:49.229829+0000","last_clean":"2024-04-25T00:39:49.229829+0000","last_became_active":"2024-04-25T00:39:49.226904+0000","last_became_peered":"2024-04-25T00:39:49.226904+0000","last_unstale":"2024-04-25T00:39:49.229829+0000","last_undegraded":"2024-04-25T00:39:49.229829+0000","last_fullsized":"2024-04-25T00:39:49.229829+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:48.543347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.219526+0000","last_change":"2024-04-25T00:39:49.210054+0000","last_active":"2024-04-25T00:39:49.219526+0000","last_peered":"2024-04-25T00:39:49.219526+0000","last_clean":"2024-04-25T00:39:49.219526+0000","last_became_active":"2024-04-25T00:39:49.208527+0000","last_became_peered":"2024-04-25T00:39:49.208527+0000","last_unstale":"2024-04-25T00:39:49.219526+0000","last_undegraded":"2024-04-25T00:39:49.219526+0000","last_fullsized":"2024-04-25T00:39:49.219526+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:42.416370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.221654+0000","last_change":"2024-04-25T00:39:49.215573+0000","last_active":"2024-04-25T00:39:49.221654+0000","last_peered":"2024-04-25T00:39:49.221654+0000","last_clean":"2024-04-25T00:39:49.221654+0000","last_became_active":"2024-04-25T00:39:49.214978+0000","last_became_peered":"2024-04-25T00:39:49.214978+0000","last_unstale":"2024-04-25T00:39:49.221654+0000","last_undegraded":"2024-04-25T00:39:49.221654+0000","last_fullsized":"2024-04-25T00:39:49.221654+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:25.924034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.232560+0000","last_change":"2024-04-25T00:39:49.228981+0000","last_active":"2024-04-25T00:39:49.232560+0000","last_peered":"2024-04-25T00:39:49.232560+0000","last_clean":"2024-04-25T00:39:49.232560+0000","last_became_active":"2024-04-25T00:39:49.228117+0000","last_became_peered":"2024-04-25T00:39:49.228117+0000","last_unstale":"2024-04-25T00:39:49.232560+0000","last_undegraded":"2024-04-25T00:39:49.232560+0000","last_fullsized":"2024-04-25T00:39:49.232560+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:33.480144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.239078+0000","last_change":"2024-04-25T00:39:49.234413+0000","last_active":"2024-04-25T00:39:49.239078+0000","last_peered":"2024-04-25T00:39:49.239078+0000","last_clean":"2024-04-25T00:39:49.239078+0000","last_became_active":"2024-04-25T00:39:49.233340+0000","last_became_peered":"2024-04-25T00:39:49.233340+0000","last_unstale":"2024-04-25T00:39:49.239078+0000","last_undegraded":"2024-04-25T00:39:49.239078+0000","last_fullsized":"2024-04-25T00:39:49.239078+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:44.606319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.213016+0000","last_change":"2024-04-25T00:39:49.206514+0000","last_active":"2024-04-25T00:39:49.213016+0000","last_peered":"2024-04-25T00:39:49.213016+0000","last_clean":"2024-04-25T00:39:49.213016+0000","last_became_active":"2024-04-25T00:39:49.205736+0000","last_became_peered":"2024-04-25T00:39:49.205736+0000","last_unstale":"2024-04-25T00:39:49.213016+0000","last_undegraded":"2024-04-25T00:39:49.213016+0000","last_fullsized":"2024-04-25T00:39:49.213016+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:05.839887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.223030+0000","last_change":"2024-04-25T00:39:49.217231+0000","last_active":"2024-04-25T00:39:49.223030+0000","last_peered":"2024-04-25T00:39:49.223030+0000","last_clean":"2024-04-25T00:39:49.223030+0000","last_became_active":"2024-04-25T00:39:49.216408+0000","last_became_peered":"2024-04-25T00:39:49.216408+0000","last_unstale":"2024-04-25T00:39:49.223030+0000","last_undegraded":"2024-04-25T00:39:49.223030+0000","last_fullsized":"2024-04-25T00:39:49.223030+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:44.856903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.214462+0000","last_change":"2024-04-25T00:39:49.212204+0000","last_active":"2024-04-25T00:39:49.214462+0000","last_peered":"2024-04-25T00:39:49.214462+0000","last_clean":"2024-04-25T00:39:49.214462+0000","last_became_active":"2024-04-25T00:39:49.211143+0000","last_became_peered":"2024-04-25T00:39:49.211143+0000","last_unstale":"2024-04-25T00:39:49.214462+0000","last_undegraded":"2024-04-25T00:39:49.214462+0000","last_fullsized":"2024-04-25T00:39:49.214462+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:23.087415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.235383+0000","last_change":"2024-04-25T00:39:49.234693+0000","last_active":"2024-04-25T00:39:49.235383+0000","last_peered":"2024-04-25T00:39:49.235383+0000","last_clean":"2024-04-25T00:39:49.235383+0000","last_became_active":"2024-04-25T00:39:49.234083+0000","last_became_peered":"2024-04-25T00:39:49.234083+0000","last_unstale":"2024-04-25T00:39:49.235383+0000","last_undegraded":"2024-04-25T00:39:49.235383+0000","last_fullsized":"2024-04-25T00:39:49.235383+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:31.275746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.212827+0000","last_change":"2024-04-25T00:39:49.203242+0000","last_active":"2024-04-25T00:39:49.212827+0000","last_peered":"2024-04-25T00:39:49.212827+0000","last_clean":"2024-04-25T00:39:49.212827+0000","last_became_active":"2024-04-25T00:39:49.202455+0000","last_became_peered":"2024-04-25T00:39:49.202455+0000","last_unstale":"2024-04-25T00:39:49.212827+0000","last_undegraded":"2024-04-25T00:39:49.212827+0000","last_fullsized":"2024-04-25T00:39:49.212827+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:19.326684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.210675+0000","last_change":"2024-04-25T00:39:49.206790+0000","last_active":"2024-04-25T00:39:49.210675+0000","last_peered":"2024-04-25T00:39:49.210675+0000","last_clean":"2024-04-25T00:39:49.210675+0000","last_became_active":"2024-04-25T00:39:49.206064+0000","last_became_peered":"2024-04-25T00:39:49.206064+0000","last_unstale":"2024-04-25T00:39:49.210675+0000","last_undegraded":"2024-04-25T00:39:49.210675+0000","last_fullsized":"2024-04-25T00:39:49.210675+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:02.343955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.222320+0000","last_change":"2024-04-25T00:39:49.215773+0000","last_active":"2024-04-25T00:39:49.222320+0000","last_peered":"2024-04-25T00:39:49.222320+0000","last_clean":"2024-04-25T00:39:49.222320+0000","last_became_active":"2024-04-25T00:39:49.213902+0000","last_became_peered":"2024-04-25T00:39:49.213902+0000","last_unstale":"2024-04-25T00:39:49.222320+0000","last_undegraded":"2024-04-25T00:39:49.222320+0000","last_fullsized":"2024-04-25T00:39:49.222320+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:52.747200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.197544+0000","last_change":"2024-04-25T00:39:49.196382+0000","last_active":"2024-04-25T00:39:49.197544+0000","last_peered":"2024-04-25T00:39:49.197544+0000","last_clean":"2024-04-25T00:39:49.197544+0000","last_became_active":"2024-04-25T00:39:49.195594+0000","last_became_peered":"2024-04-25T00:39:49.195594+0000","last_unstale":"2024-04-25T00:39:49.197544+0000","last_undegraded":"2024-04-25T00:39:49.197544+0000","last_fullsized":"2024-04-25T00:39:49.197544+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:42.460371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.219790+0000","last_change":"2024-04-25T00:39:49.211938+0000","last_active":"2024-04-25T00:39:49.219790+0000","last_peered":"2024-04-25T00:39:49.219790+0000","last_clean":"2024-04-25T00:39:49.219790+0000","last_became_active":"2024-04-25T00:39:49.211217+0000","last_became_peered":"2024-04-25T00:39:49.211217+0000","last_unstale":"2024-04-25T00:39:49.219790+0000","last_undegraded":"2024-04-25T00:39:49.219790+0000","last_fullsized":"2024-04-25T00:39:49.219790+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:25.657793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.239132+0000","last_change":"2024-04-25T00:39:49.234342+0000","last_active":"2024-04-25T00:39:49.239132+0000","last_peered":"2024-04-25T00:39:49.239132+0000","last_clean":"2024-04-25T00:39:49.239132+0000","last_became_active":"2024-04-25T00:39:49.233080+0000","last_became_peered":"2024-04-25T00:39:49.233080+0000","last_unstale":"2024-04-25T00:39:49.239132+0000","last_undegraded":"2024-04-25T00:39:49.239132+0000","last_fullsized":"2024-04-25T00:39:49.239132+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:35.870599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.221872+0000","last_change":"2024-04-25T00:39:49.218588+0000","last_active":"2024-04-25T00:39:49.221872+0000","last_peered":"2024-04-25T00:39:49.221872+0000","last_clean":"2024-04-25T00:39:49.221872+0000","last_became_active":"2024-04-25T00:39:49.217764+0000","last_became_peered":"2024-04-25T00:39:49.217764+0000","last_unstale":"2024-04-25T00:39:49.221872+0000","last_undegraded":"2024-04-25T00:39:49.221872+0000","last_fullsized":"2024-04-25T00:39:49.221872+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:54.864697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.223719+0000","last_change":"2024-04-25T00:39:49.211800+0000","last_active":"2024-04-25T00:39:49.223719+0000","last_peered":"2024-04-25T00:39:49.223719+0000","last_clean":"2024-04-25T00:39:49.223719+0000","last_became_active":"2024-04-25T00:39:49.208589+0000","last_became_peered":"2024-04-25T00:39:49.208589+0000","last_unstale":"2024-04-25T00:39:49.223719+0000","last_undegraded":"2024-04-25T00:39:49.223719+0000","last_fullsized":"2024-04-25T00:39:49.223719+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:20.018270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.222021+0000","last_change":"2024-04-25T00:39:49.218199+0000","last_active":"2024-04-25T00:39:49.222021+0000","last_peered":"2024-04-25T00:39:49.222021+0000","last_clean":"2024-04-25T00:39:49.222021+0000","last_became_active":"2024-04-25T00:39:49.217632+0000","last_became_peered":"2024-04-25T00:39:49.217632+0000","last_unstale":"2024-04-25T00:39:49.222021+0000","last_undegraded":"2024-04-25T00:39:49.222021+0000","last_fullsized":"2024-04-25T00:39:49.222021+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:31.460582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.232973+0000","last_change":"2024-04-25T00:39:49.230120+0000","last_active":"2024-04-25T00:39:49.232973+0000","last_peered":"2024-04-25T00:39:49.232973+0000","last_clean":"2024-04-25T00:39:49.232973+0000","last_became_active":"2024-04-25T00:39:49.229596+0000","last_became_peered":"2024-04-25T00:39:49.229596+0000","last_unstale":"2024-04-25T00:39:49.232973+0000","last_undegraded":"2024-04-25T00:39:49.232973+0000","last_fullsized":"2024-04-25T00:39:49.232973+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:33.825755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.231303+0000","last_change":"2024-04-25T00:39:49.227599+0000","last_active":"2024-04-25T00:39:49.231303+0000","last_peered":"2024-04-25T00:39:49.231303+0000","last_clean":"2024-04-25T00:39:49.231303+0000","last_became_active":"2024-04-25T00:39:49.226985+0000","last_became_peered":"2024-04-25T00:39:49.226985+0000","last_unstale":"2024-04-25T00:39:49.231303+0000","last_undegraded":"2024-04-25T00:39:49.231303+0000","last_fullsized":"2024-04-25T00:39:49.231303+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:41.509064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.222945+0000","last_change":"2024-04-25T00:39:49.217554+0000","last_active":"2024-04-25T00:39:49.222945+0000","last_peered":"2024-04-25T00:39:49.222945+0000","last_clean":"2024-04-25T00:39:49.222945+0000","last_became_active":"2024-04-25T00:39:49.216293+0000","last_became_peered":"2024-04-25T00:39:49.216293+0000","last_unstale":"2024-04-25T00:39:49.222945+0000","last_undegraded":"2024-04-25T00:39:49.222945+0000","last_fullsized":"2024-04-25T00:39:49.222945+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:10.565622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.239171+0000","last_change":"2024-04-25T00:39:49.234826+0000","last_active":"2024-04-25T00:39:49.239171+0000","last_peered":"2024-04-25T00:39:49.239171+0000","last_clean":"2024-04-25T00:39:49.239171+0000","last_became_active":"2024-04-25T00:39:49.233781+0000","last_became_peered":"2024-04-25T00:39:49.233781+0000","last_unstale":"2024-04-25T00:39:49.239171+0000","last_undegraded":"2024-04-25T00:39:49.239171+0000","last_fullsized":"2024-04-25T00:39:49.239171+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:05.766025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"144'4","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.247477+0000","last_change":"2024-04-25T00:39:48.218565+0000","last_active":"2024-04-25T00:39:49.247477+0000","last_peered":"2024-04-25T00:39:49.247477+0000","last_clean":"2024-04-25T00:39:49.247477+0000","last_became_active":"2024-04-25T00:39:48.217775+0000","last_became_peered":"2024-04-25T00:39:48.217775+0000","last_unstale":"2024-04-25T00:39:49.247477+0000","last_undegraded":"2024-04-25T00:39:49.247477+0000","last_fullsized":"2024-04-25T00:39:49.247477+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:29.921543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.224352+0000","last_change":"2024-04-25T00:39:49.214684+0000","last_active":"2024-04-25T00:39:49.224352+0000","last_peered":"2024-04-25T00:39:49.224352+0000","last_clean":"2024-04-25T00:39:49.224352+0000","last_became_active":"2024-04-25T00:39:49.213681+0000","last_became_peered":"2024-04-25T00:39:49.213681+0000","last_unstale":"2024-04-25T00:39:49.224352+0000","last_undegraded":"2024-04-25T00:39:49.224352+0000","last_fullsized":"2024-04-25T00:39:49.224352+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:25:34.809939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.245981+0000","last_change":"2024-04-25T00:39:48.221297+0000","last_active":"2024-04-25T00:39:49.245981+0000","last_peered":"2024-04-25T00:39:49.245981+0000","last_clean":"2024-04-25T00:39:49.245981+0000","last_became_active":"2024-04-25T00:39:48.220433+0000","last_became_peered":"2024-04-25T00:39:48.220433+0000","last_unstale":"2024-04-25T00:39:49.245981+0000","last_undegraded":"2024-04-25T00:39:49.245981+0000","last_fullsized":"2024-04-25T00:39:49.245981+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:23.579138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.234217+0000","last_change":"2024-04-25T00:39:49.233391+0000","last_active":"2024-04-25T00:39:49.234217+0000","last_peered":"2024-04-25T00:39:49.234217+0000","last_clean":"2024-04-25T00:39:49.234217+0000","last_became_active":"2024-04-25T00:39:49.233071+0000","last_became_peered":"2024-04-25T00:39:49.233071+0000","last_unstale":"2024-04-25T00:39:49.234217+0000","last_undegraded":"2024-04-25T00:39:49.234217+0000","last_fullsized":"2024-04-25T00:39:49.234217+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:30.883880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"144'2","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.245166+0000","last_change":"2024-04-25T00:39:48.212240+0000","last_active":"2024-04-25T00:39:49.245166+0000","last_peered":"2024-04-25T00:39:49.245166+0000","last_clean":"2024-04-25T00:39:49.245166+0000","last_became_active":"2024-04-25T00:39:48.212019+0000","last_became_peered":"2024-04-25T00:39:48.212019+0000","last_unstale":"2024-04-25T00:39:49.245166+0000","last_undegraded":"2024-04-25T00:39:49.245166+0000","last_fullsized":"2024-04-25T00:39:49.245166+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:01.340217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.226184+0000","last_change":"2024-04-25T00:39:49.223430+0000","last_active":"2024-04-25T00:39:49.226184+0000","last_peered":"2024-04-25T00:39:49.226184+0000","last_clean":"2024-04-25T00:39:49.226184+0000","last_became_active":"2024-04-25T00:39:49.222857+0000","last_became_peered":"2024-04-25T00:39:49.222857+0000","last_unstale":"2024-04-25T00:39:49.226184+0000","last_undegraded":"2024-04-25T00:39:49.226184+0000","last_fullsized":"2024-04-25T00:39:49.226184+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:57.926907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"144'2","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.245547+0000","last_change":"2024-04-25T00:39:48.210806+0000","last_active":"2024-04-25T00:39:49.245547+0000","last_peered":"2024-04-25T00:39:49.245547+0000","last_clean":"2024-04-25T00:39:49.245547+0000","last_became_active":"2024-04-25T00:39:48.210483+0000","last_became_peered":"2024-04-25T00:39:48.210483+0000","last_unstale":"2024-04-25T00:39:49.245547+0000","last_undegraded":"2024-04-25T00:39:49.245547+0000","last_fullsized":"2024-04-25T00:39:49.245547+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:15.219724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.231797+0000","last_change":"2024-04-25T00:39:49.220040+0000","last_active":"2024-04-25T00:39:49.231797+0000","last_peered":"2024-04-25T00:39:49.231797+0000","last_clean":"2024-04-25T00:39:49.231797+0000","last_became_active":"2024-04-25T00:39:49.212352+0000","last_became_peered":"2024-04-25T00:39:49.212352+0000","last_unstale":"2024-04-25T00:39:49.231797+0000","last_undegraded":"2024-04-25T00:39:49.231797+0000","last_fullsized":"2024-04-25T00:39:49.231797+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:10.565622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.248614+0000","last_change":"2024-04-25T00:39:48.199249+0000","last_active":"2024-04-25T00:39:49.248614+0000","last_peered":"2024-04-25T00:39:49.248614+0000","last_clean":"2024-04-25T00:39:49.248614+0000","last_became_active":"2024-04-25T00:39:48.198551+0000","last_became_peered":"2024-04-25T00:39:48.198551+0000","last_unstale":"2024-04-25T00:39:49.248614+0000","last_undegraded":"2024-04-25T00:39:49.248614+0000","last_fullsized":"2024-04-25T00:39:49.248614+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:25.306899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.224657+0000","last_change":"2024-04-25T00:39:49.222069+0000","last_active":"2024-04-25T00:39:49.224657+0000","last_peered":"2024-04-25T00:39:49.224657+0000","last_clean":"2024-04-25T00:39:49.224657+0000","last_became_active":"2024-04-25T00:39:49.221287+0000","last_became_peered":"2024-04-25T00:39:49.221287+0000","last_unstale":"2024-04-25T00:39:49.224657+0000","last_undegraded":"2024-04-25T00:39:49.224657+0000","last_fullsized":"2024-04-25T00:39:49.224657+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:24.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.241665+0000","last_change":"2024-04-25T00:39:48.203960+0000","last_active":"2024-04-25T00:39:49.241665+0000","last_peered":"2024-04-25T00:39:49.241665+0000","last_clean":"2024-04-25T00:39:49.241665+0000","last_became_active":"2024-04-25T00:39:48.203624+0000","last_became_peered":"2024-04-25T00:39:48.203624+0000","last_unstale":"2024-04-25T00:39:49.241665+0000","last_undegraded":"2024-04-25T00:39:49.241665+0000","last_fullsized":"2024-04-25T00:39:49.241665+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:13.923980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.231917+0000","last_change":"2024-04-25T00:39:49.230318+0000","last_active":"2024-04-25T00:39:49.231917+0000","last_peered":"2024-04-25T00:39:49.231917+0000","last_clean":"2024-04-25T00:39:49.231917+0000","last_became_active":"2024-04-25T00:39:49.229670+0000","last_became_peered":"2024-04-25T00:39:49.229670+0000","last_unstale":"2024-04-25T00:39:49.231917+0000","last_undegraded":"2024-04-25T00:39:49.231917+0000","last_fullsized":"2024-04-25T00:39:49.231917+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:15.964947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.187225+0000","last_change":"2024-04-25T00:39:48.208654+0000","last_active":"2024-04-25T00:39:49.187225+0000","last_peered":"2024-04-25T00:39:49.187225+0000","last_clean":"2024-04-25T00:39:49.187225+0000","last_became_active":"2024-04-25T00:39:48.207944+0000","last_became_peered":"2024-04-25T00:39:48.207944+0000","last_unstale":"2024-04-25T00:39:49.187225+0000","last_undegraded":"2024-04-25T00:39:49.187225+0000","last_fullsized":"2024-04-25T00:39:49.187225+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:54.336623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.229905+0000","last_change":"2024-04-25T00:39:49.229085+0000","last_active":"2024-04-25T00:39:49.229905+0000","last_peered":"2024-04-25T00:39:49.229905+0000","last_clean":"2024-04-25T00:39:49.229905+0000","last_became_active":"2024-04-25T00:39:49.227899+0000","last_became_peered":"2024-04-25T00:39:49.227899+0000","last_unstale":"2024-04-25T00:39:49.229905+0000","last_undegraded":"2024-04-25T00:39:49.229905+0000","last_fullsized":"2024-04-25T00:39:49.229905+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:26.310637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.200293+0000","last_change":"2024-04-25T00:39:48.202592+0000","last_active":"2024-04-25T00:39:49.200293+0000","last_peered":"2024-04-25T00:39:49.200293+0000","last_clean":"2024-04-25T00:39:49.200293+0000","last_became_active":"2024-04-25T00:39:48.201278+0000","last_became_peered":"2024-04-25T00:39:48.201278+0000","last_unstale":"2024-04-25T00:39:49.200293+0000","last_undegraded":"2024-04-25T00:39:49.200293+0000","last_fullsized":"2024-04-25T00:39:49.200293+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:44.611053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.232469+0000","last_change":"2024-04-25T00:39:49.224978+0000","last_active":"2024-04-25T00:39:49.232469+0000","last_peered":"2024-04-25T00:39:49.232469+0000","last_clean":"2024-04-25T00:39:49.232469+0000","last_became_active":"2024-04-25T00:39:49.223975+0000","last_became_peered":"2024-04-25T00:39:49.223975+0000","last_unstale":"2024-04-25T00:39:49.232469+0000","last_undegraded":"2024-04-25T00:39:49.232469+0000","last_fullsized":"2024-04-25T00:39:49.232469+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:21.291228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.201958+0000","last_change":"2024-04-25T00:39:48.213283+0000","last_active":"2024-04-25T00:39:49.201958+0000","last_peered":"2024-04-25T00:39:49.201958+0000","last_clean":"2024-04-25T00:39:49.201958+0000","last_became_active":"2024-04-25T00:39:48.212855+0000","last_became_peered":"2024-04-25T00:39:48.212855+0000","last_unstale":"2024-04-25T00:39:49.201958+0000","last_undegraded":"2024-04-25T00:39:49.201958+0000","last_fullsized":"2024-04-25T00:39:49.201958+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:15.133642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.232546+0000","last_change":"2024-04-25T00:39:49.225609+0000","last_active":"2024-04-25T00:39:49.232546+0000","last_peered":"2024-04-25T00:39:49.232546+0000","last_clean":"2024-04-25T00:39:49.232546+0000","last_became_active":"2024-04-25T00:39:49.224116+0000","last_became_peered":"2024-04-25T00:39:49.224116+0000","last_unstale":"2024-04-25T00:39:49.232546+0000","last_undegraded":"2024-04-25T00:39:49.232546+0000","last_fullsized":"2024-04-25T00:39:49.232546+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:06.226827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"144'3","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.246185+0000","last_change":"2024-04-25T00:39:48.216287+0000","last_active":"2024-04-25T00:39:49.246185+0000","last_peered":"2024-04-25T00:39:49.246185+0000","last_clean":"2024-04-25T00:39:49.246185+0000","last_became_active":"2024-04-25T00:39:48.215891+0000","last_became_peered":"2024-04-25T00:39:48.215891+0000","last_unstale":"2024-04-25T00:39:49.246185+0000","last_undegraded":"2024-04-25T00:39:49.246185+0000","last_fullsized":"2024-04-25T00:39:49.246185+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:31.417015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.225293+0000","last_change":"2024-04-25T00:39:49.218880+0000","last_active":"2024-04-25T00:39:49.225293+0000","last_peered":"2024-04-25T00:39:49.225293+0000","last_clean":"2024-04-25T00:39:49.225293+0000","last_became_active":"2024-04-25T00:39:49.218163+0000","last_became_peered":"2024-04-25T00:39:49.218163+0000","last_unstale":"2024-04-25T00:39:49.225293+0000","last_undegraded":"2024-04-25T00:39:49.225293+0000","last_fullsized":"2024-04-25T00:39:49.225293+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:26.413930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"144'6","reported_seq":18,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.254614+0000","last_change":"2024-04-25T00:39:48.217818+0000","last_active":"2024-04-25T00:39:49.254614+0000","last_peered":"2024-04-25T00:39:49.254614+0000","last_clean":"2024-04-25T00:39:49.254614+0000","last_became_active":"2024-04-25T00:39:48.217139+0000","last_became_peered":"2024-04-25T00:39:48.217139+0000","last_unstale":"2024-04-25T00:39:49.254614+0000","last_undegraded":"2024-04-25T00:39:49.254614+0000","last_fullsized":"2024-04-25T00:39:49.254614+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:56.047759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.205757+0000","last_change":"2024-04-25T00:39:49.204673+0000","last_active":"2024-04-25T00:39:49.205757+0000","last_peered":"2024-04-25T00:39:49.205757+0000","last_clean":"2024-04-25T00:39:49.205757+0000","last_became_active":"2024-04-25T00:39:49.203765+0000","last_became_peered":"2024-04-25T00:39:49.203765+0000","last_unstale":"2024-04-25T00:39:49.205757+0000","last_undegraded":"2024-04-25T00:39:49.205757+0000","last_fullsized":"2024-04-25T00:39:49.205757+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:45.614791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"144'2","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.242600+0000","last_change":"2024-04-25T00:39:48.204304+0000","last_active":"2024-04-25T00:39:49.242600+0000","last_peered":"2024-04-25T00:39:49.242600+0000","last_clean":"2024-04-25T00:39:49.242600+0000","last_became_active":"2024-04-25T00:39:48.203530+0000","last_became_peered":"2024-04-25T00:39:48.203530+0000","last_unstale":"2024-04-25T00:39:49.242600+0000","last_undegraded":"2024-04-25T00:39:49.242600+0000","last_fullsized":"2024-04-25T00:39:49.242600+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:17.085055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.225824+0000","last_change":"2024-04-25T00:39:49.218401+0000","last_active":"2024-04-25T00:39:49.225824+0000","last_peered":"2024-04-25T00:39:49.225824+0000","last_clean":"2024-04-25T00:39:49.225824+0000","last_became_active":"2024-04-25T00:39:49.217670+0000","last_became_peered":"2024-04-25T00:39:49.217670+0000","last_unstale":"2024-04-25T00:39:49.225824+0000","last_undegraded":"2024-04-25T00:39:49.225824+0000","last_fullsized":"2024-04-25T00:39:49.225824+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:44.856903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.195434+0000","last_change":"2024-04-25T00:39:48.203171+0000","last_active":"2024-04-25T00:39:49.195434+0000","last_peered":"2024-04-25T00:39:49.195434+0000","last_clean":"2024-04-25T00:39:49.195434+0000","last_became_active":"2024-04-25T00:39:48.201377+0000","last_became_peered":"2024-04-25T00:39:48.201377+0000","last_unstale":"2024-04-25T00:39:49.195434+0000","last_undegraded":"2024-04-25T00:39:49.195434+0000","last_fullsized":"2024-04-25T00:39:49.195434+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:16.553143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.219952+0000","last_change":"2024-04-25T00:39:49.210008+0000","last_active":"2024-04-25T00:39:49.219952+0000","last_peered":"2024-04-25T00:39:49.219952+0000","last_clean":"2024-04-25T00:39:49.219952+0000","last_became_active":"2024-04-25T00:39:49.209192+0000","last_became_peered":"2024-04-25T00:39:49.209192+0000","last_unstale":"2024-04-25T00:39:49.219952+0000","last_undegraded":"2024-04-25T00:39:49.219952+0000","last_fullsized":"2024-04-25T00:39:49.219952+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:13.621503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.183857+0000","last_change":"2024-04-25T00:39:48.220233+0000","last_active":"2024-04-25T00:39:49.183857+0000","last_peered":"2024-04-25T00:39:49.183857+0000","last_clean":"2024-04-25T00:39:49.183857+0000","last_became_active":"2024-04-25T00:39:48.220019+0000","last_became_peered":"2024-04-25T00:39:48.220019+0000","last_unstale":"2024-04-25T00:39:49.183857+0000","last_undegraded":"2024-04-25T00:39:49.183857+0000","last_fullsized":"2024-04-25T00:39:49.183857+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:47.247936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.226030+0000","last_change":"2024-04-25T00:39:49.221941+0000","last_active":"2024-04-25T00:39:49.226030+0000","last_peered":"2024-04-25T00:39:49.226030+0000","last_clean":"2024-04-25T00:39:49.226030+0000","last_became_active":"2024-04-25T00:39:49.221024+0000","last_became_peered":"2024-04-25T00:39:49.221024+0000","last_unstale":"2024-04-25T00:39:49.226030+0000","last_undegraded":"2024-04-25T00:39:49.226030+0000","last_fullsized":"2024-04-25T00:39:49.226030+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:55.577398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"144'8","reported_seq":20,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.252860+0000","last_change":"2024-04-25T00:39:48.217842+0000","last_active":"2024-04-25T00:39:49.252860+0000","last_peered":"2024-04-25T00:39:49.252860+0000","last_clean":"2024-04-25T00:39:49.252860+0000","last_became_active":"2024-04-25T00:39:48.217012+0000","last_became_peered":"2024-04-25T00:39:48.217012+0000","last_unstale":"2024-04-25T00:39:49.252860+0000","last_undegraded":"2024-04-25T00:39:49.252860+0000","last_fullsized":"2024-04-25T00:39:49.252860+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:25.541346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.223724+0000","last_change":"2024-04-25T00:39:49.216642+0000","last_active":"2024-04-25T00:39:49.223724+0000","last_peered":"2024-04-25T00:39:49.223724+0000","last_clean":"2024-04-25T00:39:49.223724+0000","last_became_active":"2024-04-25T00:39:49.215816+0000","last_became_peered":"2024-04-25T00:39:49.215816+0000","last_unstale":"2024-04-25T00:39:49.223724+0000","last_undegraded":"2024-04-25T00:39:49.223724+0000","last_fullsized":"2024-04-25T00:39:49.223724+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:48.543347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.247662+0000","last_change":"2024-04-25T00:39:48.223144+0000","last_active":"2024-04-25T00:39:49.247662+0000","last_peered":"2024-04-25T00:39:49.247662+0000","last_clean":"2024-04-25T00:39:49.247662+0000","last_became_active":"2024-04-25T00:39:48.222357+0000","last_became_peered":"2024-04-25T00:39:48.222357+0000","last_unstale":"2024-04-25T00:39:49.247662+0000","last_undegraded":"2024-04-25T00:39:49.247662+0000","last_fullsized":"2024-04-25T00:39:49.247662+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:21.845660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.224307+0000","last_change":"2024-04-25T00:39:49.219449+0000","last_active":"2024-04-25T00:39:49.224307+0000","last_peered":"2024-04-25T00:39:49.224307+0000","last_clean":"2024-04-25T00:39:49.224307+0000","last_became_active":"2024-04-25T00:39:49.218718+0000","last_became_peered":"2024-04-25T00:39:49.218718+0000","last_unstale":"2024-04-25T00:39:49.224307+0000","last_undegraded":"2024-04-25T00:39:49.224307+0000","last_fullsized":"2024-04-25T00:39:49.224307+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:35.483449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"144'2","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.245024+0000","last_change":"2024-04-25T00:39:48.208442+0000","last_active":"2024-04-25T00:39:49.245024+0000","last_peered":"2024-04-25T00:39:49.245024+0000","last_clean":"2024-04-25T00:39:49.245024+0000","last_became_active":"2024-04-25T00:39:48.207686+0000","last_became_peered":"2024-04-25T00:39:48.207686+0000","last_unstale":"2024-04-25T00:39:49.245024+0000","last_undegraded":"2024-04-25T00:39:49.245024+0000","last_fullsized":"2024-04-25T00:39:49.245024+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:29.880142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.225504+0000","last_change":"2024-04-25T00:39:49.223344+0000","last_active":"2024-04-25T00:39:49.225504+0000","last_peered":"2024-04-25T00:39:49.225504+0000","last_clean":"2024-04-25T00:39:49.225504+0000","last_became_active":"2024-04-25T00:39:49.222300+0000","last_became_peered":"2024-04-25T00:39:49.222300+0000","last_unstale":"2024-04-25T00:39:49.225504+0000","last_undegraded":"2024-04-25T00:39:49.225504+0000","last_fullsized":"2024-04-25T00:39:49.225504+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:43.826776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.249031+0000","last_change":"2024-04-25T00:39:48.218587+0000","last_active":"2024-04-25T00:39:49.249031+0000","last_peered":"2024-04-25T00:39:49.249031+0000","last_clean":"2024-04-25T00:39:49.249031+0000","last_became_active":"2024-04-25T00:39:48.217789+0000","last_became_peered":"2024-04-25T00:39:48.217789+0000","last_unstale":"2024-04-25T00:39:49.249031+0000","last_undegraded":"2024-04-25T00:39:49.249031+0000","last_fullsized":"2024-04-25T00:39:49.249031+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:39.132893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.231981+0000","last_change":"2024-04-25T00:39:49.222258+0000","last_active":"2024-04-25T00:39:49.231981+0000","last_peered":"2024-04-25T00:39:49.231981+0000","last_clean":"2024-04-25T00:39:49.231981+0000","last_became_active":"2024-04-25T00:39:49.221375+0000","last_became_peered":"2024-04-25T00:39:49.221375+0000","last_unstale":"2024-04-25T00:39:49.231981+0000","last_undegraded":"2024-04-25T00:39:49.231981+0000","last_fullsized":"2024-04-25T00:39:49.231981+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:35.483449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.215441+0000","last_change":"2024-04-25T00:39:48.216499+0000","last_active":"2024-04-25T00:39:49.215441+0000","last_peered":"2024-04-25T00:39:49.215441+0000","last_clean":"2024-04-25T00:39:49.215441+0000","last_became_active":"2024-04-25T00:39:48.215144+0000","last_became_peered":"2024-04-25T00:39:48.215144+0000","last_unstale":"2024-04-25T00:39:49.215441+0000","last_undegraded":"2024-04-25T00:39:49.215441+0000","last_fullsized":"2024-04-25T00:39:49.215441+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:39.744644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.219448+0000","last_change":"2024-04-25T00:39:49.209517+0000","last_active":"2024-04-25T00:39:49.219448+0000","last_peered":"2024-04-25T00:39:49.219448+0000","last_clean":"2024-04-25T00:39:49.219448+0000","last_became_active":"2024-04-25T00:39:49.208739+0000","last_became_peered":"2024-04-25T00:39:49.208739+0000","last_unstale":"2024-04-25T00:39:49.219448+0000","last_undegraded":"2024-04-25T00:39:49.219448+0000","last_fullsized":"2024-04-25T00:39:49.219448+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:18.672004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"144'2","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.245522+0000","last_change":"2024-04-25T00:39:48.217863+0000","last_active":"2024-04-25T00:39:49.245522+0000","last_peered":"2024-04-25T00:39:49.245522+0000","last_clean":"2024-04-25T00:39:49.245522+0000","last_became_active":"2024-04-25T00:39:48.217416+0000","last_became_peered":"2024-04-25T00:39:48.217416+0000","last_unstale":"2024-04-25T00:39:49.245522+0000","last_undegraded":"2024-04-25T00:39:49.245522+0000","last_fullsized":"2024-04-25T00:39:49.245522+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:41.456633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.235826+0000","last_change":"2024-04-25T00:39:49.231001+0000","last_active":"2024-04-25T00:39:49.235826+0000","last_peered":"2024-04-25T00:39:49.235826+0000","last_clean":"2024-04-25T00:39:49.235826+0000","last_became_active":"2024-04-25T00:39:49.230523+0000","last_became_peered":"2024-04-25T00:39:49.230523+0000","last_unstale":"2024-04-25T00:39:49.235826+0000","last_undegraded":"2024-04-25T00:39:49.235826+0000","last_fullsized":"2024-04-25T00:39:49.235826+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:06.226827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.199718+0000","last_change":"2024-04-25T00:39:48.213061+0000","last_active":"2024-04-25T00:39:49.199718+0000","last_peered":"2024-04-25T00:39:49.199718+0000","last_clean":"2024-04-25T00:39:49.199718+0000","last_became_active":"2024-04-25T00:39:48.212258+0000","last_became_peered":"2024-04-25T00:39:48.212258+0000","last_unstale":"2024-04-25T00:39:49.199718+0000","last_undegraded":"2024-04-25T00:39:49.199718+0000","last_fullsized":"2024-04-25T00:39:49.199718+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:01.496286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.233840+0000","last_change":"2024-04-25T00:39:49.232020+0000","last_active":"2024-04-25T00:39:49.233840+0000","last_peered":"2024-04-25T00:39:49.233840+0000","last_clean":"2024-04-25T00:39:49.233840+0000","last_became_active":"2024-04-25T00:39:49.229792+0000","last_became_peered":"2024-04-25T00:39:49.229792+0000","last_unstale":"2024-04-25T00:39:49.233840+0000","last_undegraded":"2024-04-25T00:39:49.233840+0000","last_fullsized":"2024-04-25T00:39:49.233840+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:00.398931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.183176+0000","last_change":"2024-04-25T00:39:48.210946+0000","last_active":"2024-04-25T00:39:49.183176+0000","last_peered":"2024-04-25T00:39:49.183176+0000","last_clean":"2024-04-25T00:39:49.183176+0000","last_became_active":"2024-04-25T00:39:48.210628+0000","last_became_peered":"2024-04-25T00:39:48.210628+0000","last_unstale":"2024-04-25T00:39:49.183176+0000","last_undegraded":"2024-04-25T00:39:49.183176+0000","last_fullsized":"2024-04-25T00:39:49.183176+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:47.696683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.222378+0000","last_change":"2024-04-25T00:39:49.216387+0000","last_active":"2024-04-25T00:39:49.222378+0000","last_peered":"2024-04-25T00:39:49.222378+0000","last_clean":"2024-04-25T00:39:49.222378+0000","last_became_active":"2024-04-25T00:39:49.214971+0000","last_became_peered":"2024-04-25T00:39:49.214971+0000","last_unstale":"2024-04-25T00:39:49.222378+0000","last_undegraded":"2024-04-25T00:39:49.222378+0000","last_fullsized":"2024-04-25T00:39:49.222378+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:18.875945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.205459+0000","last_change":"2024-04-25T00:39:48.219079+0000","last_active":"2024-04-25T00:39:49.205459+0000","last_peered":"2024-04-25T00:39:49.205459+0000","last_clean":"2024-04-25T00:39:49.205459+0000","last_became_active":"2024-04-25T00:39:48.218019+0000","last_became_peered":"2024-04-25T00:39:48.218019+0000","last_unstale":"2024-04-25T00:39:49.205459+0000","last_undegraded":"2024-04-25T00:39:49.205459+0000","last_fullsized":"2024-04-25T00:39:49.205459+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:10.209249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.234890+0000","last_change":"2024-04-25T00:39:49.233626+0000","last_active":"2024-04-25T00:39:49.234890+0000","last_peered":"2024-04-25T00:39:49.234890+0000","last_clean":"2024-04-25T00:39:49.234890+0000","last_became_active":"2024-04-25T00:39:49.232913+0000","last_became_peered":"2024-04-25T00:39:49.232913+0000","last_unstale":"2024-04-25T00:39:49.234890+0000","last_undegraded":"2024-04-25T00:39:49.234890+0000","last_fullsized":"2024-04-25T00:39:49.234890+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:35.870599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"144'2","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.247239+0000","last_change":"2024-04-25T00:39:48.218728+0000","last_active":"2024-04-25T00:39:49.247239+0000","last_peered":"2024-04-25T00:39:49.247239+0000","last_clean":"2024-04-25T00:39:49.247239+0000","last_became_active":"2024-04-25T00:39:48.218421+0000","last_became_peered":"2024-04-25T00:39:48.218421+0000","last_unstale":"2024-04-25T00:39:49.247239+0000","last_undegraded":"2024-04-25T00:39:49.247239+0000","last_fullsized":"2024-04-25T00:39:49.247239+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:08.646024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.225720+0000","last_change":"2024-04-25T00:39:49.213115+0000","last_active":"2024-04-25T00:39:49.225720+0000","last_peered":"2024-04-25T00:39:49.225720+0000","last_clean":"2024-04-25T00:39:49.225720+0000","last_became_active":"2024-04-25T00:39:49.212378+0000","last_became_peered":"2024-04-25T00:39:49.212378+0000","last_unstale":"2024-04-25T00:39:49.225720+0000","last_undegraded":"2024-04-25T00:39:49.225720+0000","last_fullsized":"2024-04-25T00:39:49.225720+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:35.105674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.190185+0000","last_change":"2024-04-25T00:39:48.200756+0000","last_active":"2024-04-25T00:39:49.190185+0000","last_peered":"2024-04-25T00:39:49.190185+0000","last_clean":"2024-04-25T00:39:49.190185+0000","last_became_active":"2024-04-25T00:39:48.199947+0000","last_became_peered":"2024-04-25T00:39:48.199947+0000","last_unstale":"2024-04-25T00:39:49.190185+0000","last_undegraded":"2024-04-25T00:39:49.190185+0000","last_fullsized":"2024-04-25T00:39:49.190185+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:31.507526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.224400+0000","last_change":"2024-04-25T00:39:49.219425+0000","last_active":"2024-04-25T00:39:49.224400+0000","last_peered":"2024-04-25T00:39:49.224400+0000","last_clean":"2024-04-25T00:39:49.224400+0000","last_became_active":"2024-04-25T00:39:49.218644+0000","last_became_peered":"2024-04-25T00:39:49.218644+0000","last_unstale":"2024-04-25T00:39:49.224400+0000","last_undegraded":"2024-04-25T00:39:49.224400+0000","last_fullsized":"2024-04-25T00:39:49.224400+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:15.587192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.191614+0000","last_change":"2024-04-25T00:39:48.218559+0000","last_active":"2024-04-25T00:39:49.191614+0000","last_peered":"2024-04-25T00:39:49.191614+0000","last_clean":"2024-04-25T00:39:49.191614+0000","last_became_active":"2024-04-25T00:39:48.217837+0000","last_became_peered":"2024-04-25T00:39:48.217837+0000","last_unstale":"2024-04-25T00:39:49.191614+0000","last_undegraded":"2024-04-25T00:39:49.191614+0000","last_fullsized":"2024-04-25T00:39:49.191614+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:47.247936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.234761+0000","last_change":"2024-04-25T00:39:49.227432+0000","last_active":"2024-04-25T00:39:49.234761+0000","last_peered":"2024-04-25T00:39:49.234761+0000","last_clean":"2024-04-25T00:39:49.234761+0000","last_became_active":"2024-04-25T00:39:49.226779+0000","last_became_peered":"2024-04-25T00:39:49.226779+0000","last_unstale":"2024-04-25T00:39:49.234761+0000","last_undegraded":"2024-04-25T00:39:49.234761+0000","last_fullsized":"2024-04-25T00:39:49.234761+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:03.282823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.201153+0000","last_change":"2024-04-25T00:39:48.201947+0000","last_active":"2024-04-25T00:39:49.201153+0000","last_peered":"2024-04-25T00:39:49.201153+0000","last_clean":"2024-04-25T00:39:49.201153+0000","last_became_active":"2024-04-25T00:39:48.201320+0000","last_became_peered":"2024-04-25T00:39:48.201320+0000","last_unstale":"2024-04-25T00:39:49.201153+0000","last_undegraded":"2024-04-25T00:39:49.201153+0000","last_fullsized":"2024-04-25T00:39:49.201153+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:12.000446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.222838+0000","last_change":"2024-04-25T00:39:49.216888+0000","last_active":"2024-04-25T00:39:49.222838+0000","last_peered":"2024-04-25T00:39:49.222838+0000","last_clean":"2024-04-25T00:39:49.222838+0000","last_became_active":"2024-04-25T00:39:49.216055+0000","last_became_peered":"2024-04-25T00:39:49.216055+0000","last_unstale":"2024-04-25T00:39:49.222838+0000","last_undegraded":"2024-04-25T00:39:49.222838+0000","last_fullsized":"2024-04-25T00:39:49.222838+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:35.105674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.206625+0000","last_change":"2024-04-25T00:39:48.210702+0000","last_active":"2024-04-25T00:39:49.206625+0000","last_peered":"2024-04-25T00:39:49.206625+0000","last_clean":"2024-04-25T00:39:49.206625+0000","last_became_active":"2024-04-25T00:39:48.209288+0000","last_became_peered":"2024-04-25T00:39:48.209288+0000","last_unstale":"2024-04-25T00:39:49.206625+0000","last_undegraded":"2024-04-25T00:39:49.206625+0000","last_fullsized":"2024-04-25T00:39:49.206625+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:46.086917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.231529+0000","last_change":"2024-04-25T00:39:49.227563+0000","last_active":"2024-04-25T00:39:49.231529+0000","last_peered":"2024-04-25T00:39:49.231529+0000","last_clean":"2024-04-25T00:39:49.231529+0000","last_became_active":"2024-04-25T00:39:49.226791+0000","last_became_peered":"2024-04-25T00:39:49.226791+0000","last_unstale":"2024-04-25T00:39:49.231529+0000","last_undegraded":"2024-04-25T00:39:49.231529+0000","last_fullsized":"2024-04-25T00:39:49.231529+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:07.877866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.183672+0000","last_change":"2024-04-25T00:39:48.194638+0000","last_active":"2024-04-25T00:39:49.183672+0000","last_peered":"2024-04-25T00:39:49.183672+0000","last_clean":"2024-04-25T00:39:49.183672+0000","last_became_active":"2024-04-25T00:39:48.194112+0000","last_became_peered":"2024-04-25T00:39:48.194112+0000","last_unstale":"2024-04-25T00:39:49.183672+0000","last_undegraded":"2024-04-25T00:39:49.183672+0000","last_fullsized":"2024-04-25T00:39:49.183672+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:54.336623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.238966+0000","last_change":"2024-04-25T00:39:49.233947+0000","last_active":"2024-04-25T00:39:49.238966+0000","last_peered":"2024-04-25T00:39:49.238966+0000","last_clean":"2024-04-25T00:39:49.238966+0000","last_became_active":"2024-04-25T00:39:49.233626+0000","last_became_peered":"2024-04-25T00:39:49.233626+0000","last_unstale":"2024-04-25T00:39:49.238966+0000","last_undegraded":"2024-04-25T00:39:49.238966+0000","last_fullsized":"2024-04-25T00:39:49.238966+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:03.282823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.188716+0000","last_change":"2024-04-25T00:39:48.218683+0000","last_active":"2024-04-25T00:39:49.188716+0000","last_peered":"2024-04-25T00:39:49.188716+0000","last_clean":"2024-04-25T00:39:49.188716+0000","last_became_active":"2024-04-25T00:39:48.212117+0000","last_became_peered":"2024-04-25T00:39:48.212117+0000","last_unstale":"2024-04-25T00:39:49.188716+0000","last_undegraded":"2024-04-25T00:39:49.188716+0000","last_fullsized":"2024-04-25T00:39:49.188716+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:17.085055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.226352+0000","last_change":"2024-04-25T00:39:49.223908+0000","last_active":"2024-04-25T00:39:49.226352+0000","last_peered":"2024-04-25T00:39:49.226352+0000","last_clean":"2024-04-25T00:39:49.226352+0000","last_became_active":"2024-04-25T00:39:49.223142+0000","last_became_peered":"2024-04-25T00:39:49.223142+0000","last_unstale":"2024-04-25T00:39:49.226352+0000","last_undegraded":"2024-04-25T00:39:49.226352+0000","last_fullsized":"2024-04-25T00:39:49.226352+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:08.796982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.203043+0000","last_change":"2024-04-25T00:39:48.208379+0000","last_active":"2024-04-25T00:39:49.203043+0000","last_peered":"2024-04-25T00:39:49.203043+0000","last_clean":"2024-04-25T00:39:49.203043+0000","last_became_active":"2024-04-25T00:39:48.207572+0000","last_became_peered":"2024-04-25T00:39:48.207572+0000","last_unstale":"2024-04-25T00:39:49.203043+0000","last_undegraded":"2024-04-25T00:39:49.203043+0000","last_fullsized":"2024-04-25T00:39:49.203043+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:49.645035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.235176+0000","last_change":"2024-04-25T00:39:49.233894+0000","last_active":"2024-04-25T00:39:49.235176+0000","last_peered":"2024-04-25T00:39:49.235176+0000","last_clean":"2024-04-25T00:39:49.235176+0000","last_became_active":"2024-04-25T00:39:49.233340+0000","last_became_peered":"2024-04-25T00:39:49.233340+0000","last_unstale":"2024-04-25T00:39:49.235176+0000","last_undegraded":"2024-04-25T00:39:49.235176+0000","last_fullsized":"2024-04-25T00:39:49.235176+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:44.606319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.206960+0000","last_change":"2024-04-25T00:39:48.208815+0000","last_active":"2024-04-25T00:39:49.206960+0000","last_peered":"2024-04-25T00:39:49.206960+0000","last_clean":"2024-04-25T00:39:49.206960+0000","last_became_active":"2024-04-25T00:39:48.208041+0000","last_became_peered":"2024-04-25T00:39:48.208041+0000","last_unstale":"2024-04-25T00:39:49.206960+0000","last_undegraded":"2024-04-25T00:39:49.206960+0000","last_fullsized":"2024-04-25T00:39:49.206960+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:43.895813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":445,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.205709+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:39:49.205709+0000","last_peered":"2024-04-25T00:39:49.205709+0000","last_clean":"2024-04-25T00:39:49.205709+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:39:49.205709+0000","last_undegraded":"2024-04-25T00:39:49.205709+0000","last_fullsized":"2024-04-25T00:39:49.205709+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37676,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334164,"statfs":{"total":96636764160,"available":96598183936,"internally_reserved":0,"allocated":2392064,"data_stored":1383340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37096,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334744,"statfs":{"total":96636764160,"available":96598777856,"internally_reserved":0,"allocated":1798144,"data_stored":792972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37100,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1798144,"data_stored":792972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37100,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1798144,"data_stored":792972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41260,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330580,"statfs":{"total":96636764160,"available":96594513920,"internally_reserved":0,"allocated":1798144,"data_stored":792972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37100,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1798144,"data_stored":792972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37100,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1798144,"data_stored":792972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41836,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330004,"statfs":{"total":96636764160,"available":96593924096,"internally_reserved":0,"allocated":2392064,"data_stored":1383340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:39:53.259 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:39:53.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.405+0000 7fd66efcb640 1 Processor -- start 2024-04-25T00:39:53.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.405+0000 7fd66efcb640 1 -- start start 2024-04-25T00:39:53.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.405+0000 7fd66efcb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd66815e8d0 0x7fd6680cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.405+0000 7fd66efcb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd66815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.405+0000 7fd66efcb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6680b3720 con 0x7fd6680b4780 2024-04-25T00:39:53.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.405+0000 7fd66efcb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd6680b42f0 con 0x7fd66815e8d0 2024-04-25T00:39:53.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.405+0000 7fd66efcb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd6680b39c0 con 0x7fd66815f3c0 2024-04-25T00:39:53.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.409+0000 7fd66d568640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd66815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.409+0000 7fd66d568640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd66815c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:42326/0 (socket says 172.21.15.29:42326) 2024-04-25T00:39:53.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.409+0000 7fd66d568640 1 -- 172.21.15.29:0/1666747077 learned_addr learned my addr 172.21.15.29:0/1666747077 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:53.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.409+0000 7fd66cd67640 1 --2- 172.21.15.29:0/1666747077 >> v2:172.21.15.157:3300/0 conn(0x7fd66815e8d0 0x7fd6680cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.409+0000 7fd66d568640 1 -- 172.21.15.29:0/1666747077 >> v2:172.21.15.157:3300/0 conn(0x7fd66815e8d0 msgr2=0x7fd6680cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:53.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.409+0000 7fd66d568640 1 --2- 172.21.15.29:0/1666747077 >> v2:172.21.15.157:3300/0 conn(0x7fd66815e8d0 0x7fd6680cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.409+0000 7fd66d568640 1 -- 172.21.15.29:0/1666747077 >> v1:172.21.15.157:6789/0 conn(0x7fd6680b4780 legacy=0x7fd66815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.409+0000 7fd66d568640 1 -- 172.21.15.29:0/1666747077 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6680b3ca0 con 0x7fd66815f3c0 2024-04-25T00:39:53.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.409+0000 7fd66d568640 1 --2- 172.21.15.29:0/1666747077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd66815c7d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd660009e50 tx=0x7fd66001ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=834f855fa63ebe57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd6677fe640 1 -- 172.21.15.29:0/1666747077 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6600a58b0 con 0x7fd66815f3c0 2024-04-25T00:39:53.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd6677fe640 1 -- 172.21.15.29:0/1666747077 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6600a5a50 con 0x7fd66815f3c0 2024-04-25T00:39:53.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd6677fe640 1 -- 172.21.15.29:0/1666747077 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6600a5e20 con 0x7fd66815f3c0 2024-04-25T00:39:53.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 -- 172.21.15.29:0/1666747077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 msgr2=0x7fd66815c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 --2- 172.21.15.29:0/1666747077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd66815c7d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd660009e50 tx=0x7fd66001ca00 comp rx=0 tx=0).stop 2024-04-25T00:39:53.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 -- 172.21.15.29:0/1666747077 shutdown_connections 2024-04-25T00:39:53.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 --2- 172.21.15.29:0/1666747077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd66815c7d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 --2- 172.21.15.29:0/1666747077 >> v2:172.21.15.157:3300/0 conn(0x7fd66815e8d0 0x7fd6680cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 -- 172.21.15.29:0/1666747077 >> 172.21.15.29:0/1666747077 conn(0x7fd6680b1870 msgr2=0x7fd6680b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:53.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 -- 172.21.15.29:0/1666747077 shutdown_connections 2024-04-25T00:39:53.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 -- 172.21.15.29:0/1666747077 wait complete. 2024-04-25T00:39:53.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 Processor -- start 2024-04-25T00:39:53.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 -- start start 2024-04-25T00:39:53.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6680b4780 0x7fd66815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd66815e8d0 0x7fd668072740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd668143930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd66815d6a0 con 0x7fd6680b4780 2024-04-25T00:39:53.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd66815d520 con 0x7fd66815e8d0 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66efcb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd668167c70 con 0x7fd66815f3c0 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66cd67640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6680b4780 0x7fd66815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66cd67640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6680b4780 0x7fd66815a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49310/0 (socket says 172.21.15.29:49310) 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66cd67640 1 -- 172.21.15.29:0/4010208592 learned_addr learned my addr 172.21.15.29:0/4010208592 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66d568640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd668143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd667fff640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd66815e8d0 0x7fd668072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66cd67640 1 -- 172.21.15.29:0/4010208592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 msgr2=0x7fd668143930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66cd67640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd668143930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66cd67640 1 -- 172.21.15.29:0/4010208592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd66815e8d0 msgr2=0x7fd668072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66cd67640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd66815e8d0 0x7fd668072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66cd67640 1 -- 172.21.15.29:0/4010208592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6680b0340 con 0x7fd6680b4780 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd667fff640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd66815e8d0 0x7fd668072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd66cd67640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6680b4780 0x7fd66815a5d0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00e930 tx=0x7fd65c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd6657fa640 1 -- 172.21.15.29:0/4010208592 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c017070 con 0x7fd6680b4780 2024-04-25T00:39:53.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd6657fa640 1 -- 172.21.15.29:0/4010208592 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd65c00ca00 con 0x7fd6680b4780 2024-04-25T00:39:53.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.413+0000 7fd6657fa640 1 -- 172.21.15.29:0/4010208592 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c00cdb0 con 0x7fd6680b4780 2024-04-25T00:39:53.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.417+0000 7fd66efcb640 1 -- 172.21.15.29:0/4010208592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6680712a0 con 0x7fd6680b4780 2024-04-25T00:39:53.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.417+0000 7fd66efcb640 1 -- 172.21.15.29:0/4010208592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd668071860 con 0x7fd6680b4780 2024-04-25T00:39:53.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.417+0000 7fd66efcb640 1 -- 172.21.15.29:0/4010208592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd668069270 con 0x7fd6680b4780 2024-04-25T00:39:53.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.417+0000 7fd6657fa640 1 -- 172.21.15.29:0/4010208592 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fd65c01a070 con 0x7fd6680b4780 2024-04-25T00:39:53.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.417+0000 7fd6657fa640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd654067ef0 0x7fd65406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.417+0000 7fd667fff640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd654067ef0 0x7fd65406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.417+0000 7fd6657fa640 1 -- 172.21.15.29:0/4010208592 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fd65c0867f0 con 0x7fd6680b4780 2024-04-25T00:39:53.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.417+0000 7fd667fff640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd654067ef0 0x7fd65406a3b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd66813a290 tx=0x7fd658009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.421+0000 7fd6657fa640 1 -- 172.21.15.29:0/4010208592 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd65c056b00 con 0x7fd6680b4780 2024-04-25T00:39:53.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.573+0000 7fd66efcb640 1 -- 172.21.15.29:0/4010208592 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd66806e7b0 con 0x7fd654067ef0 2024-04-25T00:39:53.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.589+0000 7fd6657fa640 1 -- 172.21.15.29:0/4010208592 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252413 (secure 0 0 0) 0x7fd66806e7b0 con 0x7fd654067ef0 2024-04-25T00:39:53.592 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:53.593 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:39:53.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.593+0000 7fd66efcb640 1 -- 172.21.15.29:0/4010208592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd654067ef0 msgr2=0x7fd65406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.593+0000 7fd66efcb640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd654067ef0 0x7fd65406a3b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd66813a290 tx=0x7fd658009040 comp rx=0 tx=0).stop 2024-04-25T00:39:53.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.593+0000 7fd66efcb640 1 -- 172.21.15.29:0/4010208592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6680b4780 msgr2=0x7fd66815a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.593+0000 7fd66efcb640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6680b4780 0x7fd66815a5d0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00e930 tx=0x7fd65c00edf0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.593+0000 7fd66efcb640 1 -- 172.21.15.29:0/4010208592 shutdown_connections 2024-04-25T00:39:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.593+0000 7fd66efcb640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd654067ef0 0x7fd65406a3b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.593+0000 7fd66efcb640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd66815f3c0 0x7fd668143930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.593+0000 7fd66efcb640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd66815e8d0 0x7fd668072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.593+0000 7fd66efcb640 1 --2- 172.21.15.29:0/4010208592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6680b4780 0x7fd66815a5d0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.593+0000 7fd66efcb640 1 -- 172.21.15.29:0/4010208592 >> 172.21.15.29:0/4010208592 conn(0x7fd6680b1870 msgr2=0x7fd6680cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.593+0000 7fd66efcb640 1 -- 172.21.15.29:0/4010208592 shutdown_connections 2024-04-25T00:39:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.593+0000 7fd66efcb640 1 -- 172.21.15.29:0/4010208592 wait complete. 2024-04-25T00:39:53.610 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-04-25T00:39:53.392464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306268,"kb_used_data":15208,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754668452,"statfs":{"total":773094113280,"available":772780494848,"internally_reserved":0,"allocated":15572992,"data_stored":7524512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001227"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.233217+0000","last_change":"2024-04-25T00:39:49.230883+0000","last_active":"2024-04-25T00:39:49.233217+0000","last_peered":"2024-04-25T00:39:49.233217+0000","last_clean":"2024-04-25T00:39:49.233217+0000","last_became_active":"2024-04-25T00:39:49.230326+0000","last_became_peered":"2024-04-25T00:39:49.230326+0000","last_unstale":"2024-04-25T00:39:49.233217+0000","last_undegraded":"2024-04-25T00:39:49.233217+0000","last_fullsized":"2024-04-25T00:39:49.233217+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:26.408586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.229501+0000","last_change":"2024-04-25T00:39:49.227757+0000","last_active":"2024-04-25T00:39:49.229501+0000","last_peered":"2024-04-25T00:39:49.229501+0000","last_clean":"2024-04-25T00:39:49.229501+0000","last_became_active":"2024-04-25T00:39:49.227102+0000","last_became_peered":"2024-04-25T00:39:49.227102+0000","last_unstale":"2024-04-25T00:39:49.229501+0000","last_undegraded":"2024-04-25T00:39:49.229501+0000","last_fullsized":"2024-04-25T00:39:49.229501+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:24.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.221724+0000","last_change":"2024-04-25T00:39:49.216106+0000","last_active":"2024-04-25T00:39:49.221724+0000","last_peered":"2024-04-25T00:39:49.221724+0000","last_clean":"2024-04-25T00:39:49.221724+0000","last_became_active":"2024-04-25T00:39:49.211605+0000","last_became_peered":"2024-04-25T00:39:49.211605+0000","last_unstale":"2024-04-25T00:39:49.221724+0000","last_undegraded":"2024-04-25T00:39:49.221724+0000","last_fullsized":"2024-04-25T00:39:49.221724+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:29.679950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.223801+0000","last_change":"2024-04-25T00:39:49.213789+0000","last_active":"2024-04-25T00:39:49.223801+0000","last_peered":"2024-04-25T00:39:49.223801+0000","last_clean":"2024-04-25T00:39:49.223801+0000","last_became_active":"2024-04-25T00:39:49.211043+0000","last_became_peered":"2024-04-25T00:39:49.211043+0000","last_unstale":"2024-04-25T00:39:49.223801+0000","last_undegraded":"2024-04-25T00:39:49.223801+0000","last_fullsized":"2024-04-25T00:39:49.223801+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:13.116403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.214214+0000","last_change":"2024-04-25T00:39:49.211378+0000","last_active":"2024-04-25T00:39:49.214214+0000","last_peered":"2024-04-25T00:39:49.214214+0000","last_clean":"2024-04-25T00:39:49.214214+0000","last_became_active":"2024-04-25T00:39:49.210693+0000","last_became_peered":"2024-04-25T00:39:49.210693+0000","last_unstale":"2024-04-25T00:39:49.214214+0000","last_undegraded":"2024-04-25T00:39:49.214214+0000","last_fullsized":"2024-04-25T00:39:49.214214+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:13.621503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.214066+0000","last_change":"2024-04-25T00:39:49.210790+0000","last_active":"2024-04-25T00:39:49.214066+0000","last_peered":"2024-04-25T00:39:49.214066+0000","last_clean":"2024-04-25T00:39:49.214066+0000","last_became_active":"2024-04-25T00:39:49.210011+0000","last_became_peered":"2024-04-25T00:39:49.210011+0000","last_unstale":"2024-04-25T00:39:49.214066+0000","last_undegraded":"2024-04-25T00:39:49.214066+0000","last_fullsized":"2024-04-25T00:39:49.214066+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:45.058512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.231963+0000","last_change":"2024-04-25T00:39:49.222555+0000","last_active":"2024-04-25T00:39:49.231963+0000","last_peered":"2024-04-25T00:39:49.231963+0000","last_clean":"2024-04-25T00:39:49.231963+0000","last_became_active":"2024-04-25T00:39:49.221627+0000","last_became_peered":"2024-04-25T00:39:49.221627+0000","last_unstale":"2024-04-25T00:39:49.231963+0000","last_undegraded":"2024-04-25T00:39:49.231963+0000","last_fullsized":"2024-04-25T00:39:49.231963+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:21.291228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.215033+0000","last_change":"2024-04-25T00:39:49.212394+0000","last_active":"2024-04-25T00:39:49.215033+0000","last_peered":"2024-04-25T00:39:49.215033+0000","last_clean":"2024-04-25T00:39:49.215033+0000","last_became_active":"2024-04-25T00:39:49.210667+0000","last_became_peered":"2024-04-25T00:39:49.210667+0000","last_unstale":"2024-04-25T00:39:49.215033+0000","last_undegraded":"2024-04-25T00:39:49.215033+0000","last_fullsized":"2024-04-25T00:39:49.215033+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:20.018270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.233047+0000","last_change":"2024-04-25T00:39:49.231597+0000","last_active":"2024-04-25T00:39:49.233047+0000","last_peered":"2024-04-25T00:39:49.233047+0000","last_clean":"2024-04-25T00:39:49.233047+0000","last_became_active":"2024-04-25T00:39:49.230883+0000","last_became_peered":"2024-04-25T00:39:49.230883+0000","last_unstale":"2024-04-25T00:39:49.233047+0000","last_undegraded":"2024-04-25T00:39:49.233047+0000","last_fullsized":"2024-04-25T00:39:49.233047+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:43.826776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.210727+0000","last_change":"2024-04-25T00:39:49.206851+0000","last_active":"2024-04-25T00:39:49.210727+0000","last_peered":"2024-04-25T00:39:49.210727+0000","last_clean":"2024-04-25T00:39:49.210727+0000","last_became_active":"2024-04-25T00:39:49.206093+0000","last_became_peered":"2024-04-25T00:39:49.206093+0000","last_unstale":"2024-04-25T00:39:49.210727+0000","last_undegraded":"2024-04-25T00:39:49.210727+0000","last_fullsized":"2024-04-25T00:39:49.210727+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:16.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.221431+0000","last_change":"2024-04-25T00:39:49.218840+0000","last_active":"2024-04-25T00:39:49.221431+0000","last_peered":"2024-04-25T00:39:49.221431+0000","last_clean":"2024-04-25T00:39:49.221431+0000","last_became_active":"2024-04-25T00:39:49.218051+0000","last_became_peered":"2024-04-25T00:39:49.218051+0000","last_unstale":"2024-04-25T00:39:49.221431+0000","last_undegraded":"2024-04-25T00:39:49.221431+0000","last_fullsized":"2024-04-25T00:39:49.221431+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:13.116403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.229829+0000","last_change":"2024-04-25T00:39:49.227651+0000","last_active":"2024-04-25T00:39:49.229829+0000","last_peered":"2024-04-25T00:39:49.229829+0000","last_clean":"2024-04-25T00:39:49.229829+0000","last_became_active":"2024-04-25T00:39:49.226904+0000","last_became_peered":"2024-04-25T00:39:49.226904+0000","last_unstale":"2024-04-25T00:39:49.229829+0000","last_undegraded":"2024-04-25T00:39:49.229829+0000","last_fullsized":"2024-04-25T00:39:49.229829+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:48.543347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.219526+0000","last_change":"2024-04-25T00:39:49.210054+0000","last_active":"2024-04-25T00:39:49.219526+0000","last_peered":"2024-04-25T00:39:49.219526+0000","last_clean":"2024-04-25T00:39:49.219526+0000","last_became_active":"2024-04-25T00:39:49.208527+0000","last_became_peered":"2024-04-25T00:39:49.208527+0000","last_unstale":"2024-04-25T00:39:49.219526+0000","last_undegraded":"2024-04-25T00:39:49.219526+0000","last_fullsized":"2024-04-25T00:39:49.219526+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:42.416370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.221654+0000","last_change":"2024-04-25T00:39:49.215573+0000","last_active":"2024-04-25T00:39:49.221654+0000","last_peered":"2024-04-25T00:39:49.221654+0000","last_clean":"2024-04-25T00:39:49.221654+0000","last_became_active":"2024-04-25T00:39:49.214978+0000","last_became_peered":"2024-04-25T00:39:49.214978+0000","last_unstale":"2024-04-25T00:39:49.221654+0000","last_undegraded":"2024-04-25T00:39:49.221654+0000","last_fullsized":"2024-04-25T00:39:49.221654+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:25.924034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.232560+0000","last_change":"2024-04-25T00:39:49.228981+0000","last_active":"2024-04-25T00:39:49.232560+0000","last_peered":"2024-04-25T00:39:49.232560+0000","last_clean":"2024-04-25T00:39:49.232560+0000","last_became_active":"2024-04-25T00:39:49.228117+0000","last_became_peered":"2024-04-25T00:39:49.228117+0000","last_unstale":"2024-04-25T00:39:49.232560+0000","last_undegraded":"2024-04-25T00:39:49.232560+0000","last_fullsized":"2024-04-25T00:39:49.232560+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:33.480144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.239078+0000","last_change":"2024-04-25T00:39:49.234413+0000","last_active":"2024-04-25T00:39:49.239078+0000","last_peered":"2024-04-25T00:39:49.239078+0000","last_clean":"2024-04-25T00:39:49.239078+0000","last_became_active":"2024-04-25T00:39:49.233340+0000","last_became_peered":"2024-04-25T00:39:49.233340+0000","last_unstale":"2024-04-25T00:39:49.239078+0000","last_undegraded":"2024-04-25T00:39:49.239078+0000","last_fullsized":"2024-04-25T00:39:49.239078+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:44.606319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.213016+0000","last_change":"2024-04-25T00:39:49.206514+0000","last_active":"2024-04-25T00:39:49.213016+0000","last_peered":"2024-04-25T00:39:49.213016+0000","last_clean":"2024-04-25T00:39:49.213016+0000","last_became_active":"2024-04-25T00:39:49.205736+0000","last_became_peered":"2024-04-25T00:39:49.205736+0000","last_unstale":"2024-04-25T00:39:49.213016+0000","last_undegraded":"2024-04-25T00:39:49.213016+0000","last_fullsized":"2024-04-25T00:39:49.213016+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:05.839887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.223030+0000","last_change":"2024-04-25T00:39:49.217231+0000","last_active":"2024-04-25T00:39:49.223030+0000","last_peered":"2024-04-25T00:39:49.223030+0000","last_clean":"2024-04-25T00:39:49.223030+0000","last_became_active":"2024-04-25T00:39:49.216408+0000","last_became_peered":"2024-04-25T00:39:49.216408+0000","last_unstale":"2024-04-25T00:39:49.223030+0000","last_undegraded":"2024-04-25T00:39:49.223030+0000","last_fullsized":"2024-04-25T00:39:49.223030+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:44.856903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.214462+0000","last_change":"2024-04-25T00:39:49.212204+0000","last_active":"2024-04-25T00:39:49.214462+0000","last_peered":"2024-04-25T00:39:49.214462+0000","last_clean":"2024-04-25T00:39:49.214462+0000","last_became_active":"2024-04-25T00:39:49.211143+0000","last_became_peered":"2024-04-25T00:39:49.211143+0000","last_unstale":"2024-04-25T00:39:49.214462+0000","last_undegraded":"2024-04-25T00:39:49.214462+0000","last_fullsized":"2024-04-25T00:39:49.214462+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:23.087415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.235383+0000","last_change":"2024-04-25T00:39:49.234693+0000","last_active":"2024-04-25T00:39:49.235383+0000","last_peered":"2024-04-25T00:39:49.235383+0000","last_clean":"2024-04-25T00:39:49.235383+0000","last_became_active":"2024-04-25T00:39:49.234083+0000","last_became_peered":"2024-04-25T00:39:49.234083+0000","last_unstale":"2024-04-25T00:39:49.235383+0000","last_undegraded":"2024-04-25T00:39:49.235383+0000","last_fullsized":"2024-04-25T00:39:49.235383+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:31.275746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.212827+0000","last_change":"2024-04-25T00:39:49.203242+0000","last_active":"2024-04-25T00:39:49.212827+0000","last_peered":"2024-04-25T00:39:49.212827+0000","last_clean":"2024-04-25T00:39:49.212827+0000","last_became_active":"2024-04-25T00:39:49.202455+0000","last_became_peered":"2024-04-25T00:39:49.202455+0000","last_unstale":"2024-04-25T00:39:49.212827+0000","last_undegraded":"2024-04-25T00:39:49.212827+0000","last_fullsized":"2024-04-25T00:39:49.212827+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:19.326684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.210675+0000","last_change":"2024-04-25T00:39:49.206790+0000","last_active":"2024-04-25T00:39:49.210675+0000","last_peered":"2024-04-25T00:39:49.210675+0000","last_clean":"2024-04-25T00:39:49.210675+0000","last_became_active":"2024-04-25T00:39:49.206064+0000","last_became_peered":"2024-04-25T00:39:49.206064+0000","last_unstale":"2024-04-25T00:39:49.210675+0000","last_undegraded":"2024-04-25T00:39:49.210675+0000","last_fullsized":"2024-04-25T00:39:49.210675+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:02.343955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.222320+0000","last_change":"2024-04-25T00:39:49.215773+0000","last_active":"2024-04-25T00:39:49.222320+0000","last_peered":"2024-04-25T00:39:49.222320+0000","last_clean":"2024-04-25T00:39:49.222320+0000","last_became_active":"2024-04-25T00:39:49.213902+0000","last_became_peered":"2024-04-25T00:39:49.213902+0000","last_unstale":"2024-04-25T00:39:49.222320+0000","last_undegraded":"2024-04-25T00:39:49.222320+0000","last_fullsized":"2024-04-25T00:39:49.222320+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:52.747200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.197544+0000","last_change":"2024-04-25T00:39:49.196382+0000","last_active":"2024-04-25T00:39:49.197544+0000","last_peered":"2024-04-25T00:39:49.197544+0000","last_clean":"2024-04-25T00:39:49.197544+0000","last_became_active":"2024-04-25T00:39:49.195594+0000","last_became_peered":"2024-04-25T00:39:49.195594+0000","last_unstale":"2024-04-25T00:39:49.197544+0000","last_undegraded":"2024-04-25T00:39:49.197544+0000","last_fullsized":"2024-04-25T00:39:49.197544+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:42.460371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.219790+0000","last_change":"2024-04-25T00:39:49.211938+0000","last_active":"2024-04-25T00:39:49.219790+0000","last_peered":"2024-04-25T00:39:49.219790+0000","last_clean":"2024-04-25T00:39:49.219790+0000","last_became_active":"2024-04-25T00:39:49.211217+0000","last_became_peered":"2024-04-25T00:39:49.211217+0000","last_unstale":"2024-04-25T00:39:49.219790+0000","last_undegraded":"2024-04-25T00:39:49.219790+0000","last_fullsized":"2024-04-25T00:39:49.219790+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:25.657793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.239132+0000","last_change":"2024-04-25T00:39:49.234342+0000","last_active":"2024-04-25T00:39:49.239132+0000","last_peered":"2024-04-25T00:39:49.239132+0000","last_clean":"2024-04-25T00:39:49.239132+0000","last_became_active":"2024-04-25T00:39:49.233080+0000","last_became_peered":"2024-04-25T00:39:49.233080+0000","last_unstale":"2024-04-25T00:39:49.239132+0000","last_undegraded":"2024-04-25T00:39:49.239132+0000","last_fullsized":"2024-04-25T00:39:49.239132+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:35.870599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.221872+0000","last_change":"2024-04-25T00:39:49.218588+0000","last_active":"2024-04-25T00:39:49.221872+0000","last_peered":"2024-04-25T00:39:49.221872+0000","last_clean":"2024-04-25T00:39:49.221872+0000","last_became_active":"2024-04-25T00:39:49.217764+0000","last_became_peered":"2024-04-25T00:39:49.217764+0000","last_unstale":"2024-04-25T00:39:49.221872+0000","last_undegraded":"2024-04-25T00:39:49.221872+0000","last_fullsized":"2024-04-25T00:39:49.221872+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:54.864697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.223719+0000","last_change":"2024-04-25T00:39:49.211800+0000","last_active":"2024-04-25T00:39:49.223719+0000","last_peered":"2024-04-25T00:39:49.223719+0000","last_clean":"2024-04-25T00:39:49.223719+0000","last_became_active":"2024-04-25T00:39:49.208589+0000","last_became_peered":"2024-04-25T00:39:49.208589+0000","last_unstale":"2024-04-25T00:39:49.223719+0000","last_undegraded":"2024-04-25T00:39:49.223719+0000","last_fullsized":"2024-04-25T00:39:49.223719+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:20.018270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.222021+0000","last_change":"2024-04-25T00:39:49.218199+0000","last_active":"2024-04-25T00:39:49.222021+0000","last_peered":"2024-04-25T00:39:49.222021+0000","last_clean":"2024-04-25T00:39:49.222021+0000","last_became_active":"2024-04-25T00:39:49.217632+0000","last_became_peered":"2024-04-25T00:39:49.217632+0000","last_unstale":"2024-04-25T00:39:49.222021+0000","last_undegraded":"2024-04-25T00:39:49.222021+0000","last_fullsized":"2024-04-25T00:39:49.222021+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:31.460582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.232973+0000","last_change":"2024-04-25T00:39:49.230120+0000","last_active":"2024-04-25T00:39:49.232973+0000","last_peered":"2024-04-25T00:39:49.232973+0000","last_clean":"2024-04-25T00:39:49.232973+0000","last_became_active":"2024-04-25T00:39:49.229596+0000","last_became_peered":"2024-04-25T00:39:49.229596+0000","last_unstale":"2024-04-25T00:39:49.232973+0000","last_undegraded":"2024-04-25T00:39:49.232973+0000","last_fullsized":"2024-04-25T00:39:49.232973+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:33.825755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.231303+0000","last_change":"2024-04-25T00:39:49.227599+0000","last_active":"2024-04-25T00:39:49.231303+0000","last_peered":"2024-04-25T00:39:49.231303+0000","last_clean":"2024-04-25T00:39:49.231303+0000","last_became_active":"2024-04-25T00:39:49.226985+0000","last_became_peered":"2024-04-25T00:39:49.226985+0000","last_unstale":"2024-04-25T00:39:49.231303+0000","last_undegraded":"2024-04-25T00:39:49.231303+0000","last_fullsized":"2024-04-25T00:39:49.231303+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:41.509064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.222945+0000","last_change":"2024-04-25T00:39:49.217554+0000","last_active":"2024-04-25T00:39:49.222945+0000","last_peered":"2024-04-25T00:39:49.222945+0000","last_clean":"2024-04-25T00:39:49.222945+0000","last_became_active":"2024-04-25T00:39:49.216293+0000","last_became_peered":"2024-04-25T00:39:49.216293+0000","last_unstale":"2024-04-25T00:39:49.222945+0000","last_undegraded":"2024-04-25T00:39:49.222945+0000","last_fullsized":"2024-04-25T00:39:49.222945+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:10.565622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.239171+0000","last_change":"2024-04-25T00:39:49.234826+0000","last_active":"2024-04-25T00:39:49.239171+0000","last_peered":"2024-04-25T00:39:49.239171+0000","last_clean":"2024-04-25T00:39:49.239171+0000","last_became_active":"2024-04-25T00:39:49.233781+0000","last_became_peered":"2024-04-25T00:39:49.233781+0000","last_unstale":"2024-04-25T00:39:49.239171+0000","last_undegraded":"2024-04-25T00:39:49.239171+0000","last_fullsized":"2024-04-25T00:39:49.239171+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:05.766025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"144'4","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.247477+0000","last_change":"2024-04-25T00:39:48.218565+0000","last_active":"2024-04-25T00:39:49.247477+0000","last_peered":"2024-04-25T00:39:49.247477+0000","last_clean":"2024-04-25T00:39:49.247477+0000","last_became_active":"2024-04-25T00:39:48.217775+0000","last_became_peered":"2024-04-25T00:39:48.217775+0000","last_unstale":"2024-04-25T00:39:49.247477+0000","last_undegraded":"2024-04-25T00:39:49.247477+0000","last_fullsized":"2024-04-25T00:39:49.247477+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:29.921543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.224352+0000","last_change":"2024-04-25T00:39:49.214684+0000","last_active":"2024-04-25T00:39:49.224352+0000","last_peered":"2024-04-25T00:39:49.224352+0000","last_clean":"2024-04-25T00:39:49.224352+0000","last_became_active":"2024-04-25T00:39:49.213681+0000","last_became_peered":"2024-04-25T00:39:49.213681+0000","last_unstale":"2024-04-25T00:39:49.224352+0000","last_undegraded":"2024-04-25T00:39:49.224352+0000","last_fullsized":"2024-04-25T00:39:49.224352+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:25:34.809939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.245981+0000","last_change":"2024-04-25T00:39:48.221297+0000","last_active":"2024-04-25T00:39:49.245981+0000","last_peered":"2024-04-25T00:39:49.245981+0000","last_clean":"2024-04-25T00:39:49.245981+0000","last_became_active":"2024-04-25T00:39:48.220433+0000","last_became_peered":"2024-04-25T00:39:48.220433+0000","last_unstale":"2024-04-25T00:39:49.245981+0000","last_undegraded":"2024-04-25T00:39:49.245981+0000","last_fullsized":"2024-04-25T00:39:49.245981+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:23.579138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.234217+0000","last_change":"2024-04-25T00:39:49.233391+0000","last_active":"2024-04-25T00:39:49.234217+0000","last_peered":"2024-04-25T00:39:49.234217+0000","last_clean":"2024-04-25T00:39:49.234217+0000","last_became_active":"2024-04-25T00:39:49.233071+0000","last_became_peered":"2024-04-25T00:39:49.233071+0000","last_unstale":"2024-04-25T00:39:49.234217+0000","last_undegraded":"2024-04-25T00:39:49.234217+0000","last_fullsized":"2024-04-25T00:39:49.234217+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:30.883880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"144'2","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.245166+0000","last_change":"2024-04-25T00:39:48.212240+0000","last_active":"2024-04-25T00:39:49.245166+0000","last_peered":"2024-04-25T00:39:49.245166+0000","last_clean":"2024-04-25T00:39:49.245166+0000","last_became_active":"2024-04-25T00:39:48.212019+0000","last_became_peered":"2024-04-25T00:39:48.212019+0000","last_unstale":"2024-04-25T00:39:49.245166+0000","last_undegraded":"2024-04-25T00:39:49.245166+0000","last_fullsized":"2024-04-25T00:39:49.245166+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:01.340217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.226184+0000","last_change":"2024-04-25T00:39:49.223430+0000","last_active":"2024-04-25T00:39:49.226184+0000","last_peered":"2024-04-25T00:39:49.226184+0000","last_clean":"2024-04-25T00:39:49.226184+0000","last_became_active":"2024-04-25T00:39:49.222857+0000","last_became_peered":"2024-04-25T00:39:49.222857+0000","last_unstale":"2024-04-25T00:39:49.226184+0000","last_undegraded":"2024-04-25T00:39:49.226184+0000","last_fullsized":"2024-04-25T00:39:49.226184+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:57.926907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"144'2","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.245547+0000","last_change":"2024-04-25T00:39:48.210806+0000","last_active":"2024-04-25T00:39:49.245547+0000","last_peered":"2024-04-25T00:39:49.245547+0000","last_clean":"2024-04-25T00:39:49.245547+0000","last_became_active":"2024-04-25T00:39:48.210483+0000","last_became_peered":"2024-04-25T00:39:48.210483+0000","last_unstale":"2024-04-25T00:39:49.245547+0000","last_undegraded":"2024-04-25T00:39:49.245547+0000","last_fullsized":"2024-04-25T00:39:49.245547+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:15.219724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.231797+0000","last_change":"2024-04-25T00:39:49.220040+0000","last_active":"2024-04-25T00:39:49.231797+0000","last_peered":"2024-04-25T00:39:49.231797+0000","last_clean":"2024-04-25T00:39:49.231797+0000","last_became_active":"2024-04-25T00:39:49.212352+0000","last_became_peered":"2024-04-25T00:39:49.212352+0000","last_unstale":"2024-04-25T00:39:49.231797+0000","last_undegraded":"2024-04-25T00:39:49.231797+0000","last_fullsized":"2024-04-25T00:39:49.231797+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:10.565622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.248614+0000","last_change":"2024-04-25T00:39:48.199249+0000","last_active":"2024-04-25T00:39:49.248614+0000","last_peered":"2024-04-25T00:39:49.248614+0000","last_clean":"2024-04-25T00:39:49.248614+0000","last_became_active":"2024-04-25T00:39:48.198551+0000","last_became_peered":"2024-04-25T00:39:48.198551+0000","last_unstale":"2024-04-25T00:39:49.248614+0000","last_undegraded":"2024-04-25T00:39:49.248614+0000","last_fullsized":"2024-04-25T00:39:49.248614+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:25.306899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.224657+0000","last_change":"2024-04-25T00:39:49.222069+0000","last_active":"2024-04-25T00:39:49.224657+0000","last_peered":"2024-04-25T00:39:49.224657+0000","last_clean":"2024-04-25T00:39:49.224657+0000","last_became_active":"2024-04-25T00:39:49.221287+0000","last_became_peered":"2024-04-25T00:39:49.221287+0000","last_unstale":"2024-04-25T00:39:49.224657+0000","last_undegraded":"2024-04-25T00:39:49.224657+0000","last_fullsized":"2024-04-25T00:39:49.224657+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:24.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.241665+0000","last_change":"2024-04-25T00:39:48.203960+0000","last_active":"2024-04-25T00:39:49.241665+0000","last_peered":"2024-04-25T00:39:49.241665+0000","last_clean":"2024-04-25T00:39:49.241665+0000","last_became_active":"2024-04-25T00:39:48.203624+0000","last_became_peered":"2024-04-25T00:39:48.203624+0000","last_unstale":"2024-04-25T00:39:49.241665+0000","last_undegraded":"2024-04-25T00:39:49.241665+0000","last_fullsized":"2024-04-25T00:39:49.241665+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:13.923980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.231917+0000","last_change":"2024-04-25T00:39:49.230318+0000","last_active":"2024-04-25T00:39:49.231917+0000","last_peered":"2024-04-25T00:39:49.231917+0000","last_clean":"2024-04-25T00:39:49.231917+0000","last_became_active":"2024-04-25T00:39:49.229670+0000","last_became_peered":"2024-04-25T00:39:49.229670+0000","last_unstale":"2024-04-25T00:39:49.231917+0000","last_undegraded":"2024-04-25T00:39:49.231917+0000","last_fullsized":"2024-04-25T00:39:49.231917+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:15.964947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.187225+0000","last_change":"2024-04-25T00:39:48.208654+0000","last_active":"2024-04-25T00:39:49.187225+0000","last_peered":"2024-04-25T00:39:49.187225+0000","last_clean":"2024-04-25T00:39:49.187225+0000","last_became_active":"2024-04-25T00:39:48.207944+0000","last_became_peered":"2024-04-25T00:39:48.207944+0000","last_unstale":"2024-04-25T00:39:49.187225+0000","last_undegraded":"2024-04-25T00:39:49.187225+0000","last_fullsized":"2024-04-25T00:39:49.187225+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:54.336623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.229905+0000","last_change":"2024-04-25T00:39:49.229085+0000","last_active":"2024-04-25T00:39:49.229905+0000","last_peered":"2024-04-25T00:39:49.229905+0000","last_clean":"2024-04-25T00:39:49.229905+0000","last_became_active":"2024-04-25T00:39:49.227899+0000","last_became_peered":"2024-04-25T00:39:49.227899+0000","last_unstale":"2024-04-25T00:39:49.229905+0000","last_undegraded":"2024-04-25T00:39:49.229905+0000","last_fullsized":"2024-04-25T00:39:49.229905+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:26.310637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.200293+0000","last_change":"2024-04-25T00:39:48.202592+0000","last_active":"2024-04-25T00:39:49.200293+0000","last_peered":"2024-04-25T00:39:49.200293+0000","last_clean":"2024-04-25T00:39:49.200293+0000","last_became_active":"2024-04-25T00:39:48.201278+0000","last_became_peered":"2024-04-25T00:39:48.201278+0000","last_unstale":"2024-04-25T00:39:49.200293+0000","last_undegraded":"2024-04-25T00:39:49.200293+0000","last_fullsized":"2024-04-25T00:39:49.200293+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:44.611053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.232469+0000","last_change":"2024-04-25T00:39:49.224978+0000","last_active":"2024-04-25T00:39:49.232469+0000","last_peered":"2024-04-25T00:39:49.232469+0000","last_clean":"2024-04-25T00:39:49.232469+0000","last_became_active":"2024-04-25T00:39:49.223975+0000","last_became_peered":"2024-04-25T00:39:49.223975+0000","last_unstale":"2024-04-25T00:39:49.232469+0000","last_undegraded":"2024-04-25T00:39:49.232469+0000","last_fullsized":"2024-04-25T00:39:49.232469+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:21.291228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.201958+0000","last_change":"2024-04-25T00:39:48.213283+0000","last_active":"2024-04-25T00:39:49.201958+0000","last_peered":"2024-04-25T00:39:49.201958+0000","last_clean":"2024-04-25T00:39:49.201958+0000","last_became_active":"2024-04-25T00:39:48.212855+0000","last_became_peered":"2024-04-25T00:39:48.212855+0000","last_unstale":"2024-04-25T00:39:49.201958+0000","last_undegraded":"2024-04-25T00:39:49.201958+0000","last_fullsized":"2024-04-25T00:39:49.201958+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:15.133642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.232546+0000","last_change":"2024-04-25T00:39:49.225609+0000","last_active":"2024-04-25T00:39:49.232546+0000","last_peered":"2024-04-25T00:39:49.232546+0000","last_clean":"2024-04-25T00:39:49.232546+0000","last_became_active":"2024-04-25T00:39:49.224116+0000","last_became_peered":"2024-04-25T00:39:49.224116+0000","last_unstale":"2024-04-25T00:39:49.232546+0000","last_undegraded":"2024-04-25T00:39:49.232546+0000","last_fullsized":"2024-04-25T00:39:49.232546+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:06.226827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"144'3","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.246185+0000","last_change":"2024-04-25T00:39:48.216287+0000","last_active":"2024-04-25T00:39:49.246185+0000","last_peered":"2024-04-25T00:39:49.246185+0000","last_clean":"2024-04-25T00:39:49.246185+0000","last_became_active":"2024-04-25T00:39:48.215891+0000","last_became_peered":"2024-04-25T00:39:48.215891+0000","last_unstale":"2024-04-25T00:39:49.246185+0000","last_undegraded":"2024-04-25T00:39:49.246185+0000","last_fullsized":"2024-04-25T00:39:49.246185+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:31.417015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.225293+0000","last_change":"2024-04-25T00:39:49.218880+0000","last_active":"2024-04-25T00:39:49.225293+0000","last_peered":"2024-04-25T00:39:49.225293+0000","last_clean":"2024-04-25T00:39:49.225293+0000","last_became_active":"2024-04-25T00:39:49.218163+0000","last_became_peered":"2024-04-25T00:39:49.218163+0000","last_unstale":"2024-04-25T00:39:49.225293+0000","last_undegraded":"2024-04-25T00:39:49.225293+0000","last_fullsized":"2024-04-25T00:39:49.225293+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:26.413930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"144'6","reported_seq":18,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.254614+0000","last_change":"2024-04-25T00:39:48.217818+0000","last_active":"2024-04-25T00:39:49.254614+0000","last_peered":"2024-04-25T00:39:49.254614+0000","last_clean":"2024-04-25T00:39:49.254614+0000","last_became_active":"2024-04-25T00:39:48.217139+0000","last_became_peered":"2024-04-25T00:39:48.217139+0000","last_unstale":"2024-04-25T00:39:49.254614+0000","last_undegraded":"2024-04-25T00:39:49.254614+0000","last_fullsized":"2024-04-25T00:39:49.254614+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:56.047759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.205757+0000","last_change":"2024-04-25T00:39:49.204673+0000","last_active":"2024-04-25T00:39:49.205757+0000","last_peered":"2024-04-25T00:39:49.205757+0000","last_clean":"2024-04-25T00:39:49.205757+0000","last_became_active":"2024-04-25T00:39:49.203765+0000","last_became_peered":"2024-04-25T00:39:49.203765+0000","last_unstale":"2024-04-25T00:39:49.205757+0000","last_undegraded":"2024-04-25T00:39:49.205757+0000","last_fullsized":"2024-04-25T00:39:49.205757+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:45.614791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"144'2","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.242600+0000","last_change":"2024-04-25T00:39:48.204304+0000","last_active":"2024-04-25T00:39:49.242600+0000","last_peered":"2024-04-25T00:39:49.242600+0000","last_clean":"2024-04-25T00:39:49.242600+0000","last_became_active":"2024-04-25T00:39:48.203530+0000","last_became_peered":"2024-04-25T00:39:48.203530+0000","last_unstale":"2024-04-25T00:39:49.242600+0000","last_undegraded":"2024-04-25T00:39:49.242600+0000","last_fullsized":"2024-04-25T00:39:49.242600+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:17.085055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.225824+0000","last_change":"2024-04-25T00:39:49.218401+0000","last_active":"2024-04-25T00:39:49.225824+0000","last_peered":"2024-04-25T00:39:49.225824+0000","last_clean":"2024-04-25T00:39:49.225824+0000","last_became_active":"2024-04-25T00:39:49.217670+0000","last_became_peered":"2024-04-25T00:39:49.217670+0000","last_unstale":"2024-04-25T00:39:49.225824+0000","last_undegraded":"2024-04-25T00:39:49.225824+0000","last_fullsized":"2024-04-25T00:39:49.225824+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:44.856903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.195434+0000","last_change":"2024-04-25T00:39:48.203171+0000","last_active":"2024-04-25T00:39:49.195434+0000","last_peered":"2024-04-25T00:39:49.195434+0000","last_clean":"2024-04-25T00:39:49.195434+0000","last_became_active":"2024-04-25T00:39:48.201377+0000","last_became_peered":"2024-04-25T00:39:48.201377+0000","last_unstale":"2024-04-25T00:39:49.195434+0000","last_undegraded":"2024-04-25T00:39:49.195434+0000","last_fullsized":"2024-04-25T00:39:49.195434+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:16.553143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.219952+0000","last_change":"2024-04-25T00:39:49.210008+0000","last_active":"2024-04-25T00:39:49.219952+0000","last_peered":"2024-04-25T00:39:49.219952+0000","last_clean":"2024-04-25T00:39:49.219952+0000","last_became_active":"2024-04-25T00:39:49.209192+0000","last_became_peered":"2024-04-25T00:39:49.209192+0000","last_unstale":"2024-04-25T00:39:49.219952+0000","last_undegraded":"2024-04-25T00:39:49.219952+0000","last_fullsized":"2024-04-25T00:39:49.219952+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:13.621503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.183857+0000","last_change":"2024-04-25T00:39:48.220233+0000","last_active":"2024-04-25T00:39:49.183857+0000","last_peered":"2024-04-25T00:39:49.183857+0000","last_clean":"2024-04-25T00:39:49.183857+0000","last_became_active":"2024-04-25T00:39:48.220019+0000","last_became_peered":"2024-04-25T00:39:48.220019+0000","last_unstale":"2024-04-25T00:39:49.183857+0000","last_undegraded":"2024-04-25T00:39:49.183857+0000","last_fullsized":"2024-04-25T00:39:49.183857+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:47.247936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.226030+0000","last_change":"2024-04-25T00:39:49.221941+0000","last_active":"2024-04-25T00:39:49.226030+0000","last_peered":"2024-04-25T00:39:49.226030+0000","last_clean":"2024-04-25T00:39:49.226030+0000","last_became_active":"2024-04-25T00:39:49.221024+0000","last_became_peered":"2024-04-25T00:39:49.221024+0000","last_unstale":"2024-04-25T00:39:49.226030+0000","last_undegraded":"2024-04-25T00:39:49.226030+0000","last_fullsized":"2024-04-25T00:39:49.226030+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:55.577398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"144'8","reported_seq":20,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.252860+0000","last_change":"2024-04-25T00:39:48.217842+0000","last_active":"2024-04-25T00:39:49.252860+0000","last_peered":"2024-04-25T00:39:49.252860+0000","last_clean":"2024-04-25T00:39:49.252860+0000","last_became_active":"2024-04-25T00:39:48.217012+0000","last_became_peered":"2024-04-25T00:39:48.217012+0000","last_unstale":"2024-04-25T00:39:49.252860+0000","last_undegraded":"2024-04-25T00:39:49.252860+0000","last_fullsized":"2024-04-25T00:39:49.252860+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:25.541346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.223724+0000","last_change":"2024-04-25T00:39:49.216642+0000","last_active":"2024-04-25T00:39:49.223724+0000","last_peered":"2024-04-25T00:39:49.223724+0000","last_clean":"2024-04-25T00:39:49.223724+0000","last_became_active":"2024-04-25T00:39:49.215816+0000","last_became_peered":"2024-04-25T00:39:49.215816+0000","last_unstale":"2024-04-25T00:39:49.223724+0000","last_undegraded":"2024-04-25T00:39:49.223724+0000","last_fullsized":"2024-04-25T00:39:49.223724+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:48.543347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.247662+0000","last_change":"2024-04-25T00:39:48.223144+0000","last_active":"2024-04-25T00:39:49.247662+0000","last_peered":"2024-04-25T00:39:49.247662+0000","last_clean":"2024-04-25T00:39:49.247662+0000","last_became_active":"2024-04-25T00:39:48.222357+0000","last_became_peered":"2024-04-25T00:39:48.222357+0000","last_unstale":"2024-04-25T00:39:49.247662+0000","last_undegraded":"2024-04-25T00:39:49.247662+0000","last_fullsized":"2024-04-25T00:39:49.247662+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:21.845660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.224307+0000","last_change":"2024-04-25T00:39:49.219449+0000","last_active":"2024-04-25T00:39:49.224307+0000","last_peered":"2024-04-25T00:39:49.224307+0000","last_clean":"2024-04-25T00:39:49.224307+0000","last_became_active":"2024-04-25T00:39:49.218718+0000","last_became_peered":"2024-04-25T00:39:49.218718+0000","last_unstale":"2024-04-25T00:39:49.224307+0000","last_undegraded":"2024-04-25T00:39:49.224307+0000","last_fullsized":"2024-04-25T00:39:49.224307+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:35.483449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"144'2","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.245024+0000","last_change":"2024-04-25T00:39:48.208442+0000","last_active":"2024-04-25T00:39:49.245024+0000","last_peered":"2024-04-25T00:39:49.245024+0000","last_clean":"2024-04-25T00:39:49.245024+0000","last_became_active":"2024-04-25T00:39:48.207686+0000","last_became_peered":"2024-04-25T00:39:48.207686+0000","last_unstale":"2024-04-25T00:39:49.245024+0000","last_undegraded":"2024-04-25T00:39:49.245024+0000","last_fullsized":"2024-04-25T00:39:49.245024+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:29.880142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.225504+0000","last_change":"2024-04-25T00:39:49.223344+0000","last_active":"2024-04-25T00:39:49.225504+0000","last_peered":"2024-04-25T00:39:49.225504+0000","last_clean":"2024-04-25T00:39:49.225504+0000","last_became_active":"2024-04-25T00:39:49.222300+0000","last_became_peered":"2024-04-25T00:39:49.222300+0000","last_unstale":"2024-04-25T00:39:49.225504+0000","last_undegraded":"2024-04-25T00:39:49.225504+0000","last_fullsized":"2024-04-25T00:39:49.225504+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:43.826776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.249031+0000","last_change":"2024-04-25T00:39:48.218587+0000","last_active":"2024-04-25T00:39:49.249031+0000","last_peered":"2024-04-25T00:39:49.249031+0000","last_clean":"2024-04-25T00:39:49.249031+0000","last_became_active":"2024-04-25T00:39:48.217789+0000","last_became_peered":"2024-04-25T00:39:48.217789+0000","last_unstale":"2024-04-25T00:39:49.249031+0000","last_undegraded":"2024-04-25T00:39:49.249031+0000","last_fullsized":"2024-04-25T00:39:49.249031+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:39.132893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.231981+0000","last_change":"2024-04-25T00:39:49.222258+0000","last_active":"2024-04-25T00:39:49.231981+0000","last_peered":"2024-04-25T00:39:49.231981+0000","last_clean":"2024-04-25T00:39:49.231981+0000","last_became_active":"2024-04-25T00:39:49.221375+0000","last_became_peered":"2024-04-25T00:39:49.221375+0000","last_unstale":"2024-04-25T00:39:49.231981+0000","last_undegraded":"2024-04-25T00:39:49.231981+0000","last_fullsized":"2024-04-25T00:39:49.231981+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:35.483449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.215441+0000","last_change":"2024-04-25T00:39:48.216499+0000","last_active":"2024-04-25T00:39:49.215441+0000","last_peered":"2024-04-25T00:39:49.215441+0000","last_clean":"2024-04-25T00:39:49.215441+0000","last_became_active":"2024-04-25T00:39:48.215144+0000","last_became_peered":"2024-04-25T00:39:48.215144+0000","last_unstale":"2024-04-25T00:39:49.215441+0000","last_undegraded":"2024-04-25T00:39:49.215441+0000","last_fullsized":"2024-04-25T00:39:49.215441+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:39.744644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.219448+0000","last_change":"2024-04-25T00:39:49.209517+0000","last_active":"2024-04-25T00:39:49.219448+0000","last_peered":"2024-04-25T00:39:49.219448+0000","last_clean":"2024-04-25T00:39:49.219448+0000","last_became_active":"2024-04-25T00:39:49.208739+0000","last_became_peered":"2024-04-25T00:39:49.208739+0000","last_unstale":"2024-04-25T00:39:49.219448+0000","last_undegraded":"2024-04-25T00:39:49.219448+0000","last_fullsized":"2024-04-25T00:39:49.219448+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:18.672004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"144'2","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.245522+0000","last_change":"2024-04-25T00:39:48.217863+0000","last_active":"2024-04-25T00:39:49.245522+0000","last_peered":"2024-04-25T00:39:49.245522+0000","last_clean":"2024-04-25T00:39:49.245522+0000","last_became_active":"2024-04-25T00:39:48.217416+0000","last_became_peered":"2024-04-25T00:39:48.217416+0000","last_unstale":"2024-04-25T00:39:49.245522+0000","last_undegraded":"2024-04-25T00:39:49.245522+0000","last_fullsized":"2024-04-25T00:39:49.245522+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:41.456633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.235826+0000","last_change":"2024-04-25T00:39:49.231001+0000","last_active":"2024-04-25T00:39:49.235826+0000","last_peered":"2024-04-25T00:39:49.235826+0000","last_clean":"2024-04-25T00:39:49.235826+0000","last_became_active":"2024-04-25T00:39:49.230523+0000","last_became_peered":"2024-04-25T00:39:49.230523+0000","last_unstale":"2024-04-25T00:39:49.235826+0000","last_undegraded":"2024-04-25T00:39:49.235826+0000","last_fullsized":"2024-04-25T00:39:49.235826+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:06.226827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.199718+0000","last_change":"2024-04-25T00:39:48.213061+0000","last_active":"2024-04-25T00:39:49.199718+0000","last_peered":"2024-04-25T00:39:49.199718+0000","last_clean":"2024-04-25T00:39:49.199718+0000","last_became_active":"2024-04-25T00:39:48.212258+0000","last_became_peered":"2024-04-25T00:39:48.212258+0000","last_unstale":"2024-04-25T00:39:49.199718+0000","last_undegraded":"2024-04-25T00:39:49.199718+0000","last_fullsized":"2024-04-25T00:39:49.199718+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:01.496286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.233840+0000","last_change":"2024-04-25T00:39:49.232020+0000","last_active":"2024-04-25T00:39:49.233840+0000","last_peered":"2024-04-25T00:39:49.233840+0000","last_clean":"2024-04-25T00:39:49.233840+0000","last_became_active":"2024-04-25T00:39:49.229792+0000","last_became_peered":"2024-04-25T00:39:49.229792+0000","last_unstale":"2024-04-25T00:39:49.233840+0000","last_undegraded":"2024-04-25T00:39:49.233840+0000","last_fullsized":"2024-04-25T00:39:49.233840+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:00.398931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.183176+0000","last_change":"2024-04-25T00:39:48.210946+0000","last_active":"2024-04-25T00:39:49.183176+0000","last_peered":"2024-04-25T00:39:49.183176+0000","last_clean":"2024-04-25T00:39:49.183176+0000","last_became_active":"2024-04-25T00:39:48.210628+0000","last_became_peered":"2024-04-25T00:39:48.210628+0000","last_unstale":"2024-04-25T00:39:49.183176+0000","last_undegraded":"2024-04-25T00:39:49.183176+0000","last_fullsized":"2024-04-25T00:39:49.183176+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:47.696683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.222378+0000","last_change":"2024-04-25T00:39:49.216387+0000","last_active":"2024-04-25T00:39:49.222378+0000","last_peered":"2024-04-25T00:39:49.222378+0000","last_clean":"2024-04-25T00:39:49.222378+0000","last_became_active":"2024-04-25T00:39:49.214971+0000","last_became_peered":"2024-04-25T00:39:49.214971+0000","last_unstale":"2024-04-25T00:39:49.222378+0000","last_undegraded":"2024-04-25T00:39:49.222378+0000","last_fullsized":"2024-04-25T00:39:49.222378+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:18.875945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.205459+0000","last_change":"2024-04-25T00:39:48.219079+0000","last_active":"2024-04-25T00:39:49.205459+0000","last_peered":"2024-04-25T00:39:49.205459+0000","last_clean":"2024-04-25T00:39:49.205459+0000","last_became_active":"2024-04-25T00:39:48.218019+0000","last_became_peered":"2024-04-25T00:39:48.218019+0000","last_unstale":"2024-04-25T00:39:49.205459+0000","last_undegraded":"2024-04-25T00:39:49.205459+0000","last_fullsized":"2024-04-25T00:39:49.205459+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:10.209249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.234890+0000","last_change":"2024-04-25T00:39:49.233626+0000","last_active":"2024-04-25T00:39:49.234890+0000","last_peered":"2024-04-25T00:39:49.234890+0000","last_clean":"2024-04-25T00:39:49.234890+0000","last_became_active":"2024-04-25T00:39:49.232913+0000","last_became_peered":"2024-04-25T00:39:49.232913+0000","last_unstale":"2024-04-25T00:39:49.234890+0000","last_undegraded":"2024-04-25T00:39:49.234890+0000","last_fullsized":"2024-04-25T00:39:49.234890+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:35.870599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"144'2","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.247239+0000","last_change":"2024-04-25T00:39:48.218728+0000","last_active":"2024-04-25T00:39:49.247239+0000","last_peered":"2024-04-25T00:39:49.247239+0000","last_clean":"2024-04-25T00:39:49.247239+0000","last_became_active":"2024-04-25T00:39:48.218421+0000","last_became_peered":"2024-04-25T00:39:48.218421+0000","last_unstale":"2024-04-25T00:39:49.247239+0000","last_undegraded":"2024-04-25T00:39:49.247239+0000","last_fullsized":"2024-04-25T00:39:49.247239+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:08.646024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.225720+0000","last_change":"2024-04-25T00:39:49.213115+0000","last_active":"2024-04-25T00:39:49.225720+0000","last_peered":"2024-04-25T00:39:49.225720+0000","last_clean":"2024-04-25T00:39:49.225720+0000","last_became_active":"2024-04-25T00:39:49.212378+0000","last_became_peered":"2024-04-25T00:39:49.212378+0000","last_unstale":"2024-04-25T00:39:49.225720+0000","last_undegraded":"2024-04-25T00:39:49.225720+0000","last_fullsized":"2024-04-25T00:39:49.225720+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:35.105674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.190185+0000","last_change":"2024-04-25T00:39:48.200756+0000","last_active":"2024-04-25T00:39:49.190185+0000","last_peered":"2024-04-25T00:39:49.190185+0000","last_clean":"2024-04-25T00:39:49.190185+0000","last_became_active":"2024-04-25T00:39:48.199947+0000","last_became_peered":"2024-04-25T00:39:48.199947+0000","last_unstale":"2024-04-25T00:39:49.190185+0000","last_undegraded":"2024-04-25T00:39:49.190185+0000","last_fullsized":"2024-04-25T00:39:49.190185+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:31.507526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.224400+0000","last_change":"2024-04-25T00:39:49.219425+0000","last_active":"2024-04-25T00:39:49.224400+0000","last_peered":"2024-04-25T00:39:49.224400+0000","last_clean":"2024-04-25T00:39:49.224400+0000","last_became_active":"2024-04-25T00:39:49.218644+0000","last_became_peered":"2024-04-25T00:39:49.218644+0000","last_unstale":"2024-04-25T00:39:49.224400+0000","last_undegraded":"2024-04-25T00:39:49.224400+0000","last_fullsized":"2024-04-25T00:39:49.224400+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:15.587192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.191614+0000","last_change":"2024-04-25T00:39:48.218559+0000","last_active":"2024-04-25T00:39:49.191614+0000","last_peered":"2024-04-25T00:39:49.191614+0000","last_clean":"2024-04-25T00:39:49.191614+0000","last_became_active":"2024-04-25T00:39:48.217837+0000","last_became_peered":"2024-04-25T00:39:48.217837+0000","last_unstale":"2024-04-25T00:39:49.191614+0000","last_undegraded":"2024-04-25T00:39:49.191614+0000","last_fullsized":"2024-04-25T00:39:49.191614+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:47.247936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.234761+0000","last_change":"2024-04-25T00:39:49.227432+0000","last_active":"2024-04-25T00:39:49.234761+0000","last_peered":"2024-04-25T00:39:49.234761+0000","last_clean":"2024-04-25T00:39:49.234761+0000","last_became_active":"2024-04-25T00:39:49.226779+0000","last_became_peered":"2024-04-25T00:39:49.226779+0000","last_unstale":"2024-04-25T00:39:49.234761+0000","last_undegraded":"2024-04-25T00:39:49.234761+0000","last_fullsized":"2024-04-25T00:39:49.234761+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:03.282823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.201153+0000","last_change":"2024-04-25T00:39:48.201947+0000","last_active":"2024-04-25T00:39:49.201153+0000","last_peered":"2024-04-25T00:39:49.201153+0000","last_clean":"2024-04-25T00:39:49.201153+0000","last_became_active":"2024-04-25T00:39:48.201320+0000","last_became_peered":"2024-04-25T00:39:48.201320+0000","last_unstale":"2024-04-25T00:39:49.201153+0000","last_undegraded":"2024-04-25T00:39:49.201153+0000","last_fullsized":"2024-04-25T00:39:49.201153+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:12.000446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.222838+0000","last_change":"2024-04-25T00:39:49.216888+0000","last_active":"2024-04-25T00:39:49.222838+0000","last_peered":"2024-04-25T00:39:49.222838+0000","last_clean":"2024-04-25T00:39:49.222838+0000","last_became_active":"2024-04-25T00:39:49.216055+0000","last_became_peered":"2024-04-25T00:39:49.216055+0000","last_unstale":"2024-04-25T00:39:49.222838+0000","last_undegraded":"2024-04-25T00:39:49.222838+0000","last_fullsized":"2024-04-25T00:39:49.222838+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:35.105674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.206625+0000","last_change":"2024-04-25T00:39:48.210702+0000","last_active":"2024-04-25T00:39:49.206625+0000","last_peered":"2024-04-25T00:39:49.206625+0000","last_clean":"2024-04-25T00:39:49.206625+0000","last_became_active":"2024-04-25T00:39:48.209288+0000","last_became_peered":"2024-04-25T00:39:48.209288+0000","last_unstale":"2024-04-25T00:39:49.206625+0000","last_undegraded":"2024-04-25T00:39:49.206625+0000","last_fullsized":"2024-04-25T00:39:49.206625+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:46.086917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.231529+0000","last_change":"2024-04-25T00:39:49.227563+0000","last_active":"2024-04-25T00:39:49.231529+0000","last_peered":"2024-04-25T00:39:49.231529+0000","last_clean":"2024-04-25T00:39:49.231529+0000","last_became_active":"2024-04-25T00:39:49.226791+0000","last_became_peered":"2024-04-25T00:39:49.226791+0000","last_unstale":"2024-04-25T00:39:49.231529+0000","last_undegraded":"2024-04-25T00:39:49.231529+0000","last_fullsized":"2024-04-25T00:39:49.231529+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:07.877866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.183672+0000","last_change":"2024-04-25T00:39:48.194638+0000","last_active":"2024-04-25T00:39:49.183672+0000","last_peered":"2024-04-25T00:39:49.183672+0000","last_clean":"2024-04-25T00:39:49.183672+0000","last_became_active":"2024-04-25T00:39:48.194112+0000","last_became_peered":"2024-04-25T00:39:48.194112+0000","last_unstale":"2024-04-25T00:39:49.183672+0000","last_undegraded":"2024-04-25T00:39:49.183672+0000","last_fullsized":"2024-04-25T00:39:49.183672+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:54.336623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.238966+0000","last_change":"2024-04-25T00:39:49.233947+0000","last_active":"2024-04-25T00:39:49.238966+0000","last_peered":"2024-04-25T00:39:49.238966+0000","last_clean":"2024-04-25T00:39:49.238966+0000","last_became_active":"2024-04-25T00:39:49.233626+0000","last_became_peered":"2024-04-25T00:39:49.233626+0000","last_unstale":"2024-04-25T00:39:49.238966+0000","last_undegraded":"2024-04-25T00:39:49.238966+0000","last_fullsized":"2024-04-25T00:39:49.238966+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:03.282823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.188716+0000","last_change":"2024-04-25T00:39:48.218683+0000","last_active":"2024-04-25T00:39:49.188716+0000","last_peered":"2024-04-25T00:39:49.188716+0000","last_clean":"2024-04-25T00:39:49.188716+0000","last_became_active":"2024-04-25T00:39:48.212117+0000","last_became_peered":"2024-04-25T00:39:48.212117+0000","last_unstale":"2024-04-25T00:39:49.188716+0000","last_undegraded":"2024-04-25T00:39:49.188716+0000","last_fullsized":"2024-04-25T00:39:49.188716+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:17.085055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.226352+0000","last_change":"2024-04-25T00:39:49.223908+0000","last_active":"2024-04-25T00:39:49.226352+0000","last_peered":"2024-04-25T00:39:49.226352+0000","last_clean":"2024-04-25T00:39:49.226352+0000","last_became_active":"2024-04-25T00:39:49.223142+0000","last_became_peered":"2024-04-25T00:39:49.223142+0000","last_unstale":"2024-04-25T00:39:49.226352+0000","last_undegraded":"2024-04-25T00:39:49.226352+0000","last_fullsized":"2024-04-25T00:39:49.226352+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:08.796982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.203043+0000","last_change":"2024-04-25T00:39:48.208379+0000","last_active":"2024-04-25T00:39:49.203043+0000","last_peered":"2024-04-25T00:39:49.203043+0000","last_clean":"2024-04-25T00:39:49.203043+0000","last_became_active":"2024-04-25T00:39:48.207572+0000","last_became_peered":"2024-04-25T00:39:48.207572+0000","last_unstale":"2024-04-25T00:39:49.203043+0000","last_undegraded":"2024-04-25T00:39:49.203043+0000","last_fullsized":"2024-04-25T00:39:49.203043+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:49.645035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.235176+0000","last_change":"2024-04-25T00:39:49.233894+0000","last_active":"2024-04-25T00:39:49.235176+0000","last_peered":"2024-04-25T00:39:49.235176+0000","last_clean":"2024-04-25T00:39:49.235176+0000","last_became_active":"2024-04-25T00:39:49.233340+0000","last_became_peered":"2024-04-25T00:39:49.233340+0000","last_unstale":"2024-04-25T00:39:49.235176+0000","last_undegraded":"2024-04-25T00:39:49.235176+0000","last_fullsized":"2024-04-25T00:39:49.235176+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:48.165572+0000","last_clean_scrub_stamp":"2024-04-25T00:39:48.165572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:44.606319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.206960+0000","last_change":"2024-04-25T00:39:48.208815+0000","last_active":"2024-04-25T00:39:49.206960+0000","last_peered":"2024-04-25T00:39:49.206960+0000","last_clean":"2024-04-25T00:39:49.206960+0000","last_became_active":"2024-04-25T00:39:48.208041+0000","last_became_peered":"2024-04-25T00:39:48.208041+0000","last_unstale":"2024-04-25T00:39:49.206960+0000","last_undegraded":"2024-04-25T00:39:49.206960+0000","last_fullsized":"2024-04-25T00:39:49.206960+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:39:47.161834+0000","last_clean_scrub_stamp":"2024-04-25T00:39:47.161834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:43.895813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":445,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-25T00:39:49.205709+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:39:49.205709+0000","last_peered":"2024-04-25T00:39:49.205709+0000","last_clean":"2024-04-25T00:39:49.205709+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:39:49.205709+0000","last_undegraded":"2024-04-25T00:39:49.205709+0000","last_fullsized":"2024-04-25T00:39:49.205709+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37676,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334164,"statfs":{"total":96636764160,"available":96598183936,"internally_reserved":0,"allocated":2392064,"data_stored":1383340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37096,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334744,"statfs":{"total":96636764160,"available":96598777856,"internally_reserved":0,"allocated":1798144,"data_stored":792972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37100,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1798144,"data_stored":792972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37100,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1798144,"data_stored":792972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41260,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330580,"statfs":{"total":96636764160,"available":96594513920,"internally_reserved":0,"allocated":1798144,"data_stored":792972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37100,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1798144,"data_stored":792972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37100,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1798144,"data_stored":792972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41836,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330004,"statfs":{"total":96636764160,"available":96593924096,"internally_reserved":0,"allocated":2392064,"data_stored":1383340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:39:53.614 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:39:53.615 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:39:53.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.781+0000 7fe4d73ce640 1 Processor -- start 2024-04-25T00:39:53.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.781+0000 7fe4d73ce640 1 -- start start 2024-04-25T00:39:53.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.781+0000 7fe4d73ce640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe4d015e250 0x7fe4d00d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.781+0000 7fe4d73ce640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe4d00cfb00 0x7fe4d00d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.781+0000 7fe4d73ce640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d00c8690 con 0x7fe4d015e630 2024-04-25T00:39:53.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.781+0000 7fe4d73ce640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d00c8440 con 0x7fe4d00cfb00 2024-04-25T00:39:53.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.781+0000 7fe4d73ce640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d00c8970 con 0x7fe4d015e250 2024-04-25T00:39:53.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.781+0000 7fe4d596b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe4d00cfb00 0x7fe4d00d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.781+0000 7fe4d596b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe4d00cfb00 0x7fe4d00d29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49326/0 (socket says 172.21.15.29:49326) 2024-04-25T00:39:53.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.781+0000 7fe4d596b640 1 -- 172.21.15.29:0/3417179230 learned_addr learned my addr 172.21.15.29:0/3417179230 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:53.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d516a640 1 --2- 172.21.15.29:0/3417179230 >> v2:172.21.15.157:3300/0 conn(0x7fe4d015e250 0x7fe4d00d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d596b640 1 -- 172.21.15.29:0/3417179230 >> v2:172.21.15.157:3300/0 conn(0x7fe4d015e250 msgr2=0x7fe4d00d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:53.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d596b640 1 --2- 172.21.15.29:0/3417179230 >> v2:172.21.15.157:3300/0 conn(0x7fe4d015e250 0x7fe4d00d24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d596b640 1 -- 172.21.15.29:0/3417179230 >> v1:172.21.15.157:6789/0 conn(0x7fe4d015e630 legacy=0x7fe4d00c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d596b640 1 -- 172.21.15.29:0/3417179230 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d00c8c50 con 0x7fe4d00cfb00 2024-04-25T00:39:53.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d596b640 1 --2- 172.21.15.29:0/3417179230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4d00cfb00 0x7fe4d00d29e0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00d3160 tx=0x7fe4c801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79265a462549f065 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4c7fff640 1 -- 172.21.15.29:0/3417179230 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80a5620 con 0x7fe4d00cfb00 2024-04-25T00:39:53.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4c7fff640 1 -- 172.21.15.29:0/3417179230 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c80a57c0 con 0x7fe4d00cfb00 2024-04-25T00:39:53.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4c7fff640 1 -- 172.21.15.29:0/3417179230 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80a5b90 con 0x7fe4d00cfb00 2024-04-25T00:39:53.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d73ce640 1 -- 172.21.15.29:0/3417179230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4d00cfb00 msgr2=0x7fe4d00d29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d73ce640 1 --2- 172.21.15.29:0/3417179230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4d00cfb00 0x7fe4d00d29e0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00d3160 tx=0x7fe4c801c960 comp rx=0 tx=0).stop 2024-04-25T00:39:53.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d73ce640 1 -- 172.21.15.29:0/3417179230 shutdown_connections 2024-04-25T00:39:53.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d73ce640 1 --2- 172.21.15.29:0/3417179230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4d00cfb00 0x7fe4d00d29e0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d73ce640 1 --2- 172.21.15.29:0/3417179230 >> v2:172.21.15.157:3300/0 conn(0x7fe4d015e250 0x7fe4d00d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d73ce640 1 -- 172.21.15.29:0/3417179230 >> 172.21.15.29:0/3417179230 conn(0x7fe4d00d3f70 msgr2=0x7fe4d00d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:53.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d73ce640 1 -- 172.21.15.29:0/3417179230 shutdown_connections 2024-04-25T00:39:53.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.785+0000 7fe4d73ce640 1 -- 172.21.15.29:0/3417179230 wait complete. 2024-04-25T00:39:53.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d73ce640 1 Processor -- start 2024-04-25T00:39:53.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d73ce640 1 -- start start 2024-04-25T00:39:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d73ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4d00cfb00 0x7fe4d00cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d73ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4d015e250 0x7fe4d015d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d73ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4d015e630 0x7fe4d00b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d73ce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d00ca370 con 0x7fe4d015e630 2024-04-25T00:39:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d73ce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d00d1ed0 con 0x7fe4d00cfb00 2024-04-25T00:39:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d73ce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d00ca4f0 con 0x7fe4d015e250 2024-04-25T00:39:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d596b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4d015e630 0x7fe4d00b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d596b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4d015e630 0x7fe4d00b7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49340/0 (socket says 172.21.15.29:49340) 2024-04-25T00:39:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d596b640 1 -- 172.21.15.29:0/1654994813 learned_addr learned my addr 172.21.15.29:0/1654994813 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d4969640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4d015e250 0x7fe4d015d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d516a640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4d00cfb00 0x7fe4d00cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d596b640 1 -- 172.21.15.29:0/1654994813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4d015e250 msgr2=0x7fe4d015d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d596b640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4d015e250 0x7fe4d015d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d596b640 1 -- 172.21.15.29:0/1654994813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4d00cfb00 msgr2=0x7fe4d00cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d596b640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4d00cfb00 0x7fe4d00cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d596b640 1 -- 172.21.15.29:0/1654994813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d015c0d0 con 0x7fe4d015e630 2024-04-25T00:39:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d516a640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4d00cfb00 0x7fe4d00cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d596b640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4d015e630 0x7fe4d00b7620 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8019570 tx=0x7fe4c8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4c5ffb640 1 -- 172.21.15.29:0/1654994813 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80bc020 con 0x7fe4d015e630 2024-04-25T00:39:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4c5ffb640 1 -- 172.21.15.29:0/1654994813 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c80b8670 con 0x7fe4d015e630 2024-04-25T00:39:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4c5ffb640 1 -- 172.21.15.29:0/1654994813 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80b8a60 con 0x7fe4d015e630 2024-04-25T00:39:53.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d73ce640 1 -- 172.21.15.29:0/1654994813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d015c3b0 con 0x7fe4d015e630 2024-04-25T00:39:53.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d73ce640 1 -- 172.21.15.29:0/1654994813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d015c970 con 0x7fe4d015e630 2024-04-25T00:39:53.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.789+0000 7fe4d73ce640 1 -- 172.21.15.29:0/1654994813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d0069270 con 0x7fe4d015e630 2024-04-25T00:39:53.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.793+0000 7fe4c5ffb640 1 -- 172.21.15.29:0/1654994813 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fe4c80bf020 con 0x7fe4d015e630 2024-04-25T00:39:53.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.793+0000 7fe4c5ffb640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe4b4067ef0 0x7fe4b406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.793+0000 7fe4c5ffb640 1 -- 172.21.15.29:0/1654994813 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fe4c8121840 con 0x7fe4d015e630 2024-04-25T00:39:53.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.793+0000 7fe4d516a640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe4b4067ef0 0x7fe4b406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.793+0000 7fe4d516a640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe4b4067ef0 0x7fe4b406a3b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00c79b0 tx=0x7fe4cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:53.797+0000 7fe4c5ffb640 1 -- 172.21.15.29:0/1654994813 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe4c80c6030 con 0x7fe4d015e630 2024-04-25T00:39:54.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.009+0000 7fe4d73ce640 1 -- 172.21.15.29:0/1654994813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe4d00b5e90 con 0x7fe4d015e630 2024-04-25T00:39:54.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.013+0000 7fe4c5ffb640 1 -- 172.21.15.29:0/1654994813 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) ==== 190+0+147 (secure 0 0 0) 0x7fe4c80165e0 con 0x7fe4d015e630 2024-04-25T00:39:54.014 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-04-25T00:39:54.015 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A== 2024-04-25T00:39:54.015 INFO:teuthology.orchestra.run.smithi029.stdout: caps mds = "allow" 2024-04-25T00:39:54.015 INFO:teuthology.orchestra.run.smithi029.stdout: caps mon = "allow r" 2024-04-25T00:39:54.015 INFO:teuthology.orchestra.run.smithi029.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:39:54.015 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-04-25T00:39:54.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.017+0000 7fe4d73ce640 1 -- 172.21.15.29:0/1654994813 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe4b4067ef0 msgr2=0x7fe4b406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.017+0000 7fe4d73ce640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe4b4067ef0 0x7fe4b406a3b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00c79b0 tx=0x7fe4cc008040 comp rx=0 tx=0).stop 2024-04-25T00:39:54.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.017+0000 7fe4d73ce640 1 -- 172.21.15.29:0/1654994813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4d015e630 msgr2=0x7fe4d00b7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.017+0000 7fe4d73ce640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4d015e630 0x7fe4d00b7620 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8019570 tx=0x7fe4c8019700 comp rx=0 tx=0).stop 2024-04-25T00:39:54.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.017+0000 7fe4d73ce640 1 -- 172.21.15.29:0/1654994813 shutdown_connections 2024-04-25T00:39:54.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.017+0000 7fe4d73ce640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe4b4067ef0 0x7fe4b406a3b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.017+0000 7fe4d73ce640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4d015e630 0x7fe4d00b7620 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.017+0000 7fe4d73ce640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4d015e250 0x7fe4d015d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.017+0000 7fe4d73ce640 1 --2- 172.21.15.29:0/1654994813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4d00cfb00 0x7fe4d00cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.017+0000 7fe4d73ce640 1 -- 172.21.15.29:0/1654994813 >> 172.21.15.29:0/1654994813 conn(0x7fe4d00d3f70 msgr2=0x7fe4d0161100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:54.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.017+0000 7fe4d73ce640 1 -- 172.21.15.29:0/1654994813 shutdown_connections 2024-04-25T00:39:54.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.017+0000 7fe4d73ce640 1 -- 172.21.15.29:0/1654994813 wait complete. 2024-04-25T00:39:54.032 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:39:54.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.153+0000 7f8af67c8640 1 Processor -- start 2024-04-25T00:39:54.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.153+0000 7f8af67c8640 1 -- start start 2024-04-25T00:39:54.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8af67c8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8af015df70 0x7f8af00b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8af67c8640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8af006d980 con 0x7f8af00b4d30 2024-04-25T00:39:54.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8af67c8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8af006dc60 con 0x7f8af006d210 2024-04-25T00:39:54.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8af67c8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8af006df40 con 0x7f8af015df70 2024-04-25T00:39:54.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8af4d65640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8af00b4d30 0x7f8af00ad8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37748/0 (socket says 172.21.15.29:37748) 2024-04-25T00:39:54.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8af4d65640 1 -- 172.21.15.29:0/883946031 learned_addr learned my addr 172.21.15.29:0/883946031 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:54.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aeffff640 1 --2- 172.21.15.29:0/883946031 >> v2:172.21.15.157:3300/0 conn(0x7f8af015df70 0x7f8af00b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 313652255 0 0) 0x7f8af006d980 con 0x7f8af00b4d30 2024-04-25T00:39:54.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8adc003530 con 0x7f8af00b4d30 2024-04-25T00:39:54.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3901607491 0 0) 0x7f8adc003530 con 0x7f8af00b4d30 2024-04-25T00:39:54.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 >> v2:172.21.15.157:3300/0 conn(0x7f8af015df70 msgr2=0x7f8af00b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:54.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 --2- 172.21.15.29:0/883946031 >> v2:172.21.15.157:3300/0 conn(0x7f8af015df70 0x7f8af00b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 >> v1:172.21.15.157:6789/0 conn(0x7f8af006d210 legacy=0x7f8af00b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af006e220 con 0x7f8af00b4d30 2024-04-25T00:39:54.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8ae8001a00 con 0x7f8af00b4d30 2024-04-25T00:39:54.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 >> v1:172.21.15.29:6789/0 conn(0x7f8af00b4d30 legacy=0x7f8af00ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 --2- 172.21.15.29:0/883946031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8adc0039c0 0x7f8adc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 --2- 172.21.15.29:0/883946031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8adc004810 0x7f8adc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 --2- 172.21.15.29:0/883946031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8adc0051d0 0x7f8adc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8af006df40 con 0x7f8adc0051d0 2024-04-25T00:39:54.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc00c170 con 0x7f8adc0039c0 2024-04-25T00:39:54.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8adc00c370 con 0x7f8adc004810 2024-04-25T00:39:54.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8af4d65640 1 --2- 172.21.15.29:0/883946031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8adc0051d0 0x7f8adc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8ae77fe640 1 --2- 172.21.15.29:0/883946031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8adc004810 0x7f8adc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8aeffff640 1 --2- 172.21.15.29:0/883946031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8adc0039c0 0x7f8adc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8af4d65640 1 -- 172.21.15.29:0/883946031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8adc004810 msgr2=0x7f8adc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:54.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8af4d65640 1 --2- 172.21.15.29:0/883946031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8adc004810 0x7f8adc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8af4d65640 1 -- 172.21.15.29:0/883946031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8adc0039c0 msgr2=0x7f8adc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:54.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8af4d65640 1 --2- 172.21.15.29:0/883946031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8adc0039c0 0x7f8adc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8af4d65640 1 -- 172.21.15.29:0/883946031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8adc003530 con 0x7f8adc0051d0 2024-04-25T00:39:54.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.157+0000 7f8ae77fe640 1 --2- 172.21.15.29:0/883946031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8adc004810 0x7f8adc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:54.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af4d65640 1 -- 172.21.15.29:0/883946031 reap_dead start 2024-04-25T00:39:54.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af4d65640 1 --2- 172.21.15.29:0/883946031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8adc0051d0 0x7f8adc00b9d0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8010c70 tx=0x7f8ae8027f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806cee82dd8c087f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:54.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8010890 con 0x7f8adc0051d0 2024-04-25T00:39:54.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae80b1bd0 con 0x7f8adc0051d0 2024-04-25T00:39:54.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8aef7fe640 1 -- 172.21.15.29:0/883946031 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80b9440 con 0x7f8adc0051d0 2024-04-25T00:39:54.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 -- 172.21.15.29:0/883946031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8adc0051d0 msgr2=0x7f8adc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 --2- 172.21.15.29:0/883946031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8adc0051d0 0x7f8adc00b9d0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8010c70 tx=0x7f8ae8027f70 comp rx=0 tx=0).stop 2024-04-25T00:39:54.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 -- 172.21.15.29:0/883946031 shutdown_connections 2024-04-25T00:39:54.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 --2- 172.21.15.29:0/883946031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8adc0051d0 0x7f8adc00b9d0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 -- 172.21.15.29:0/883946031 >> 172.21.15.29:0/883946031 conn(0x7f8af00b1870 msgr2=0x7f8af00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:54.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 -- 172.21.15.29:0/883946031 shutdown_connections 2024-04-25T00:39:54.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 -- 172.21.15.29:0/883946031 wait complete. 2024-04-25T00:39:54.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 Processor -- start 2024-04-25T00:39:54.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 -- start start 2024-04-25T00:39:54.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8adc0051d0 0x7f8af01564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8af0152120 0x7f8af0142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8af01525a0 0x7f8af0142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8af006d6a0 con 0x7f8adc0051d0 2024-04-25T00:39:54.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8af015e350 con 0x7f8af01525a0 2024-04-25T00:39:54.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af67c8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8af006ea20 con 0x7f8af0152120 2024-04-25T00:39:54.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8aeffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8adc0051d0 0x7f8af01564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8aeffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8adc0051d0 0x7f8af01564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49360/0 (socket says 172.21.15.29:49360) 2024-04-25T00:39:54.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8aeffff640 1 -- 172.21.15.29:0/1551801 learned_addr learned my addr 172.21.15.29:0/1551801 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:54.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8ae77fe640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8af0152120 0x7f8af0142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8af4d65640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8af01525a0 0x7f8af0142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8aeffff640 1 -- 172.21.15.29:0/1551801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8af0152120 msgr2=0x7f8af0142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8aeffff640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8af0152120 0x7f8af0142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8aeffff640 1 -- 172.21.15.29:0/1551801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8af01525a0 msgr2=0x7f8af0142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8aeffff640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8af01525a0 0x7f8af0142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.161+0000 7f8aeffff640 1 -- 172.21.15.29:0/1551801 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af0152b00 con 0x7f8adc0051d0 2024-04-25T00:39:54.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.165+0000 7f8aeffff640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8adc0051d0 0x7f8af01564a0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0011560 tx=0x7f8ae0011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:54.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.165+0000 7f8aed7fa640 1 -- 172.21.15.29:0/1551801 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0015930 con 0x7f8adc0051d0 2024-04-25T00:39:54.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.165+0000 7f8aed7fa640 1 -- 172.21.15.29:0/1551801 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae0015ad0 con 0x7f8adc0051d0 2024-04-25T00:39:54.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.165+0000 7f8af67c8640 1 -- 172.21.15.29:0/1551801 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af0152de0 con 0x7f8adc0051d0 2024-04-25T00:39:54.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.165+0000 7f8aed7fa640 1 -- 172.21.15.29:0/1551801 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae000f460 con 0x7f8adc0051d0 2024-04-25T00:39:54.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.165+0000 7f8af67c8640 1 -- 172.21.15.29:0/1551801 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af0154100 con 0x7f8adc0051d0 2024-04-25T00:39:54.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.165+0000 7f8af67c8640 1 -- 172.21.15.29:0/1551801 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af0069270 con 0x7f8adc0051d0 2024-04-25T00:39:54.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.165+0000 7f8aed7fa640 1 -- 172.21.15.29:0/1551801 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f8ae000f600 con 0x7f8adc0051d0 2024-04-25T00:39:54.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.165+0000 7f8aed7fa640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ad0067ef0 0x7f8ad006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.165+0000 7f8aed7fa640 1 -- 172.21.15.29:0/1551801 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f8ae0087490 con 0x7f8adc0051d0 2024-04-25T00:39:54.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.165+0000 7f8ae77fe640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ad0067ef0 0x7f8ad006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.165+0000 7f8ae77fe640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ad0067ef0 0x7f8ad006a3b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8af0159e20 tx=0x7f8ad8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:54.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.169+0000 7f8aed7fa640 1 -- 172.21.15.29:0/1551801 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8ae0019030 con 0x7f8adc0051d0 2024-04-25T00:39:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.373+0000 7f8af67c8640 1 -- 172.21.15.29:0/1551801 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8af0158b90 con 0x7f8adc0051d0 2024-04-25T00:39:54.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.373+0000 7f8aed7fa640 1 -- 172.21.15.29:0/1551801 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5185 (secure 0 0 0) 0x7f8ae0057790 con 0x7f8adc0051d0 2024-04-25T00:39:54.373 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:39:54.374 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":92,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3075204329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3075204329},{"type":"v1","addr":"172.21.15.157:6833","nonce":3075204329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3377772964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3377772964},{"type":"v1","addr":"172.21.15.157:6835","nonce":3377772964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":9621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/897131811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":897131811},{"type":"v1","addr":"172.21.15.29:6833","nonce":897131811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:39:48.556647+0000","modified":"2024-04-25T00:39:51.600640+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9624":{"gid":9624,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/821427804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":821427804},{"type":"v1","addr":"172.21.15.29:6835","nonce":821427804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9624,"qdb_cluster":[9624]},"id":11}]} 2024-04-25T00:39:54.374 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 92 2024-04-25T00:39:54.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.377+0000 7f8af67c8640 1 -- 172.21.15.29:0/1551801 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ad0067ef0 msgr2=0x7f8ad006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.377+0000 7f8af67c8640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ad0067ef0 0x7f8ad006a3b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8af0159e20 tx=0x7f8ad8009040 comp rx=0 tx=0).stop 2024-04-25T00:39:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.377+0000 7f8af67c8640 1 -- 172.21.15.29:0/1551801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8adc0051d0 msgr2=0x7f8af01564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.377+0000 7f8af67c8640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8adc0051d0 0x7f8af01564a0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0011560 tx=0x7f8ae0011a20 comp rx=0 tx=0).stop 2024-04-25T00:39:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.377+0000 7f8af67c8640 1 -- 172.21.15.29:0/1551801 shutdown_connections 2024-04-25T00:39:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.377+0000 7f8af67c8640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8ad0067ef0 0x7f8ad006a3b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.377+0000 7f8af67c8640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8af01525a0 0x7f8af0142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.377+0000 7f8af67c8640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8af0152120 0x7f8af0142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.377+0000 7f8af67c8640 1 --2- 172.21.15.29:0/1551801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8adc0051d0 0x7f8af01564a0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.377+0000 7f8af67c8640 1 -- 172.21.15.29:0/1551801 >> 172.21.15.29:0/1551801 conn(0x7f8af00b1870 msgr2=0x7f8af0157b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.377+0000 7f8af67c8640 1 -- 172.21.15.29:0/1551801 shutdown_connections 2024-04-25T00:39:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.377+0000 7f8af67c8640 1 -- 172.21.15.29:0/1551801 wait complete. 2024-04-25T00:39:54.795 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:39:48.556647+0000', 'modified': '2024-04-25T00:39:51.600640+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9624}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9624': {'gid': 9624, 'name': 'c', 'rank': 0, 'incarnation': 89, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/821427804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 821427804}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 821427804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9624, 'qdb_cluster': [9624]} 2024-04-25T00:39:54.795 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:39:54.795 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:39:54.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.933+0000 7efd58ff0640 1 Processor -- start 2024-04-25T00:39:54.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.933+0000 7efd58ff0640 1 -- start start 2024-04-25T00:39:54.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd58ff0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efd5415bdb0 0x7efd540cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd58ff0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5415c190 0x7efd540ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd58ff0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd540cd3e0 con 0x7efd540d02d0 2024-04-25T00:39:54.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd58ff0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7efd540cd6c0 con 0x7efd5415bdb0 2024-04-25T00:39:54.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd58ff0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efd540cd9a0 con 0x7efd5415c190 2024-04-25T00:39:54.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd5259c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efd5415bdb0 0x7efd540cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd5259c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efd5415bdb0 0x7efd540cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49376/0 (socket says 172.21.15.29:49376) 2024-04-25T00:39:54.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd5259c640 1 -- 172.21.15.29:0/1470639884 learned_addr learned my addr 172.21.15.29:0/1470639884 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:54.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd51d9b640 1 --2- 172.21.15.29:0/1470639884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5415c190 0x7efd540ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd5259c640 1 -- 172.21.15.29:0/1470639884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5415c190 msgr2=0x7efd540ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:54.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd5259c640 1 --2- 172.21.15.29:0/1470639884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5415c190 0x7efd540ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd5259c640 1 -- 172.21.15.29:0/1470639884 >> v1:172.21.15.29:6789/0 conn(0x7efd540d02d0 legacy=0x7efd540d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd5259c640 1 -- 172.21.15.29:0/1470639884 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd540cdc80 con 0x7efd5415bdb0 2024-04-25T00:39:54.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd51d9b640 1 --2- 172.21.15.29:0/1470639884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5415c190 0x7efd540ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:54.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd5259c640 1 --2- 172.21.15.29:0/1470639884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd5415bdb0 0x7efd540cc820 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7efd44009870 tx=0x7efd4401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adea4363263b878 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:54.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd5159a640 1 -- 172.21.15.29:0/1470639884 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd440ab660 con 0x7efd5415bdb0 2024-04-25T00:39:54.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd5159a640 1 -- 172.21.15.29:0/1470639884 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd440b0070 con 0x7efd5415bdb0 2024-04-25T00:39:54.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.937+0000 7efd5159a640 1 -- 172.21.15.29:0/1470639884 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd440aba20 con 0x7efd5415bdb0 2024-04-25T00:39:54.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 -- 172.21.15.29:0/1470639884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd5415bdb0 msgr2=0x7efd540cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 --2- 172.21.15.29:0/1470639884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd5415bdb0 0x7efd540cc820 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7efd44009870 tx=0x7efd4401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 -- 172.21.15.29:0/1470639884 shutdown_connections 2024-04-25T00:39:54.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 --2- 172.21.15.29:0/1470639884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5415c190 0x7efd540ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 --2- 172.21.15.29:0/1470639884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd5415bdb0 0x7efd540cc820 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 -- 172.21.15.29:0/1470639884 >> 172.21.15.29:0/1470639884 conn(0x7efd5409aa30 msgr2=0x7efd5409ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:54.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 -- 172.21.15.29:0/1470639884 shutdown_connections 2024-04-25T00:39:54.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 -- 172.21.15.29:0/1470639884 wait complete. 2024-04-25T00:39:54.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 Processor -- start 2024-04-25T00:39:54.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 -- start start 2024-04-25T00:39:54.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd540d02d0 0x7efd540c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd5415bdb0 0x7efd5406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd5406ddb0 0x7efd540ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efd54168ad0 con 0x7efd5415bdb0 2024-04-25T00:39:54.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efd54168c50 con 0x7efd5406ddb0 2024-04-25T00:39:54.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efd54168dd0 con 0x7efd540d02d0 2024-04-25T00:39:54.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd51d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd5415bdb0 0x7efd5406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd51d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd5415bdb0 0x7efd5406d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49378/0 (socket says 172.21.15.29:49378) 2024-04-25T00:39:54.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd51d9b640 1 -- 172.21.15.29:0/152530925 learned_addr learned my addr 172.21.15.29:0/152530925 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:54.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd5259c640 1 --2- 172.21.15.29:0/152530925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd540d02d0 0x7efd540c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd51d9b640 1 -- 172.21.15.29:0/152530925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd540d02d0 msgr2=0x7efd540c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd51d9b640 1 --2- 172.21.15.29:0/152530925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd540d02d0 0x7efd540c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd51d9b640 1 -- 172.21.15.29:0/152530925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd5406ddb0 msgr2=0x7efd540ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd51d9b640 1 --2- 172.21.15.29:0/152530925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd5406ddb0 0x7efd540ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd51d9b640 1 -- 172.21.15.29:0/152530925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd540c8f80 con 0x7efd5415bdb0 2024-04-25T00:39:54.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd51d9b640 1 --2- 172.21.15.29:0/152530925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd5415bdb0 0x7efd5406d870 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7efd4800d800 tx=0x7efd4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:54.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd42ffd640 1 -- 172.21.15.29:0/152530925 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4800b6c0 con 0x7efd5415bdb0 2024-04-25T00:39:54.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd42ffd640 1 -- 172.21.15.29:0/152530925 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd48019070 con 0x7efd5415bdb0 2024-04-25T00:39:54.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd42ffd640 1 -- 172.21.15.29:0/152530925 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4800baa0 con 0x7efd5415bdb0 2024-04-25T00:39:54.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 -- 172.21.15.29:0/152530925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd540c9540 con 0x7efd5415bdb0 2024-04-25T00:39:54.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.941+0000 7efd58ff0640 1 -- 172.21.15.29:0/152530925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd540c9b00 con 0x7efd5415bdb0 2024-04-25T00:39:54.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd42ffd640 1 -- 172.21.15.29:0/152530925 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7efd4800bc40 con 0x7efd5415bdb0 2024-04-25T00:39:54.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd42ffd640 1 --2- 172.21.15.29:0/152530925 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd34067ef0 0x7efd3406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd42ffd640 1 -- 172.21.15.29:0/152530925 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7efd48086b50 con 0x7efd5415bdb0 2024-04-25T00:39:54.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd5259c640 1 --2- 172.21.15.29:0/152530925 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd34067ef0 0x7efd3406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd5259c640 1 --2- 172.21.15.29:0/152530925 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd34067ef0 0x7efd3406a3b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efd440af000 tx=0x7efd440a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:54.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd5c169000 1 Processor -- start 2024-04-25T00:39:54.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd5c169000 1 -- start start 2024-04-25T00:39:54.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd5c169000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323eef0 0x5555f323f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd5c169000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 0x5555f31f5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd5c169000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 0x5555f31f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd5c169000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5555f30435e0 con 0x5555f31f5bc0 2024-04-25T00:39:54.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd5c169000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5555f2fcac50 con 0x5555f323f810 2024-04-25T00:39:54.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd5c169000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5555f303f5a0 con 0x5555f323eef0 2024-04-25T00:39:54.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd52d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 0x5555f31f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd52d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 0x5555f31f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49380/0 (socket says 172.21.15.29:49380) 2024-04-25T00:39:54.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd52d9d640 1 -- 172.21.15.29:0/3248650835 learned_addr learned my addr 172.21.15.29:0/3248650835 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:54.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd5259c640 1 --2- 172.21.15.29:0/3248650835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323eef0 0x5555f323f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd51d9b640 1 --2- 172.21.15.29:0/3248650835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 0x5555f31f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd52d9d640 1 -- 172.21.15.29:0/3248650835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323eef0 msgr2=0x5555f323f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:54.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd52d9d640 1 --2- 172.21.15.29:0/3248650835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323eef0 0x5555f323f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd52d9d640 1 -- 172.21.15.29:0/3248650835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 msgr2=0x5555f31f5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:54.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd52d9d640 1 --2- 172.21.15.29:0/3248650835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 0x5555f31f5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd52d9d640 1 -- 172.21.15.29:0/3248650835 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f2fc8d10 con 0x5555f31f5bc0 2024-04-25T00:39:54.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd51d9b640 1 --2- 172.21.15.29:0/3248650835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 0x5555f31f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:54.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd52d9d640 1 --2- 172.21.15.29:0/3248650835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 0x5555f31f5fa0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7efd4c001d50 tx=0x7efd4c00c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e101a4116171d8d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:54.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd33fff640 1 -- 172.21.15.29:0/3248650835 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c006d60 con 0x5555f31f5bc0 2024-04-25T00:39:54.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd33fff640 1 -- 172.21.15.29:0/3248650835 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd4c00cac0 con 0x5555f31f5bc0 2024-04-25T00:39:54.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.945+0000 7efd33fff640 1 -- 172.21.15.29:0/3248650835 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c005020 con 0x5555f31f5bc0 2024-04-25T00:39:54.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 -- 172.21.15.29:0/3248650835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 msgr2=0x5555f31f5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 --2- 172.21.15.29:0/3248650835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 0x5555f31f5fa0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7efd4c001d50 tx=0x7efd4c00c0e0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 -- 172.21.15.29:0/3248650835 shutdown_connections 2024-04-25T00:39:54.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 --2- 172.21.15.29:0/3248650835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 0x5555f31f5fa0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 --2- 172.21.15.29:0/3248650835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 0x5555f31f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 --2- 172.21.15.29:0/3248650835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323eef0 0x5555f323f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 -- 172.21.15.29:0/3248650835 >> 172.21.15.29:0/3248650835 conn(0x5555f31f1b70 msgr2=0x5555f31f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:54.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 -- 172.21.15.29:0/3248650835 shutdown_connections 2024-04-25T00:39:54.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 -- 172.21.15.29:0/3248650835 wait complete. 2024-04-25T00:39:54.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 -- start start 2024-04-25T00:39:54.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 Processor -- start 2024-04-25T00:39:54.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f31f5bc0 0x5555f3223f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 0x5555f3224460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323f810 0x5555f32249a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5555f2ffd520 con 0x5555f323eef0 2024-04-25T00:39:54.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5555f31f19f0 con 0x5555f31f5bc0 2024-04-25T00:39:54.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5555f2fdbdd0 con 0x5555f323f810 2024-04-25T00:39:54.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 10 client.0 _ll_register_callbacks cb 0x5555f31e05c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:54.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd51d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 0x5555f3224460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd51d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 0x5555f3224460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49396/0 (socket says 172.21.15.29:49396) 2024-04-25T00:39:54.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd51d9b640 1 -- 172.21.15.29:0/790549042 learned_addr learned my addr 172.21.15.29:0/790549042 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:54.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5259c640 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f31f5bc0 0x5555f3223f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd52d9d640 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323f810 0x5555f32249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd51d9b640 1 -- 172.21.15.29:0/790549042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323f810 msgr2=0x5555f32249a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:54.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd51d9b640 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323f810 0x5555f32249a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd51d9b640 1 -- 172.21.15.29:0/790549042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f31f5bc0 msgr2=0x5555f3223f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:54.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd51d9b640 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f31f5bc0 0x5555f3223f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd51d9b640 1 -- 172.21.15.29:0/790549042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f2fdb7b0 con 0x5555f323eef0 2024-04-25T00:39:54.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd52d9d640 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323f810 0x5555f32249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:54.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd51d9b640 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 0x5555f3224460 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7efd4805ed00 tx=0x7efd48056e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517b8b57c605a99e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:54.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd317fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:39:54.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd317fa640 1 -- 172.21.15.29:0/790549042 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4806e070 con 0x5555f323eef0 2024-04-25T00:39:54.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd317fa640 1 -- 172.21.15.29:0/790549042 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7efd4805bd90 con 0x5555f323eef0 2024-04-25T00:39:54.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 -- 172.21.15.29:0/790549042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd4c00cac0 con 0x5555f323eef0 2024-04-25T00:39:54.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd317fa640 1 -- 172.21.15.29:0/790549042 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7efd4805f230 con 0x5555f323eef0 2024-04-25T00:39:54.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 10 client.9762 fetch_fsmap learned FSMap version 92 2024-04-25T00:39:54.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 -- 172.21.15.29:0/790549042 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x5555f2fcaea0 con 0x5555f323eef0 2024-04-25T00:39:54.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd317fa640 1 -- 172.21.15.29:0/790549042 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd48081030 con 0x5555f323eef0 2024-04-25T00:39:54.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd317fa640 1 -- 172.21.15.29:0/790549042 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 92) ==== 3494+0+0 (secure 0 0 0) 0x7efd48013070 con 0x5555f323eef0 2024-04-25T00:39:54.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 10 client.9762 fetch_fsmap finished waiting for FSMap version 92 2024-04-25T00:39:54.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 10 client.9762 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:39:54.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 20 client.9762 populate_metadata read hostname 'smithi029' 2024-04-25T00:39:54.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.29:6834/821427804,v1:172.21.15.29:6835/821427804] conn(0x5555f324ef90 0x5555f3251380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 4 client.9762 mds_command: new command op to 9624 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:39:54.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5c169000 1 -- 172.21.15.29:0/790549042 --> [v2:172.21.15.29:6834/821427804,v1:172.21.15.29:6835/821427804] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555f2fdeab0 con 0x5555f324ef90 2024-04-25T00:39:54.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5259c640 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.29:6834/821427804,v1:172.21.15.29:6835/821427804] conn(0x5555f324ef90 0x5555f3251380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd5259c640 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.29:6834/821427804,v1:172.21.15.29:6835/821427804] conn(0x5555f324ef90 0x5555f3251380 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61024b45b72ffe57 server_cookie=cc68603a7f6d2292 in_seq=0 out_seq=0 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.949+0000 7efd317fa640 10 client.9762 ms_handle_connect on v2:172.21.15.29:6834/821427804 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd317fa640 1 -- 172.21.15.29:0/790549042 <== mds.0 v2:172.21.15.29:6834/821427804 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555f2fdeab0 con 0x5555f324ef90 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd317fa640 10 client.9762 handle_command_reply: tid=0 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd5c169000 1 client.9762 shutdown 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd5c169000 1 -- 172.21.15.29:0/790549042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 msgr2=0x5555f3224460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd5c169000 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 0x5555f3224460 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7efd4805ed00 tx=0x7efd48056e10 comp rx=0 tx=0).stop 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd5c169000 1 -- 172.21.15.29:0/790549042 shutdown_connections 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd5c169000 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.29:6834/821427804,v1:172.21.15.29:6835/821427804] conn(0x5555f324ef90 0x5555f3251380 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd5c169000 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323f810 0x5555f32249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd5c169000 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 0x5555f3224460 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd317fa640 0 client.9762 ms_handle_reset on v2:172.21.15.29:6834/821427804 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd5c169000 1 --2- 172.21.15.29:0/790549042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f31f5bc0 0x5555f3223f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd5c169000 1 -- 172.21.15.29:0/790549042 >> 172.21.15.29:0/790549042 conn(0x5555f31f1b70 msgr2=0x5555f31f47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:54.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd5c169000 1 -- 172.21.15.29:0/790549042 shutdown_connections 2024-04-25T00:39:54.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd5c169000 1 -- 172.21.15.29:0/790549042 wait complete. 2024-04-25T00:39:54.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.953+0000 7efd5c169000 20 client.9762 trim_cache size 0 max 16384 2024-04-25T00:39:54.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.969+0000 7efd5c169000 1 Processor -- start 2024-04-25T00:39:54.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.969+0000 7efd5c169000 1 -- start start 2024-04-25T00:39:54.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.969+0000 7efd5c169000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f31f5bc0 0x5555f32aa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.969+0000 7efd5c169000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 0x5555f32aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.969+0000 7efd5c169000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 0x5555f32aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.969+0000 7efd5c169000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efd54202920 con 0x5555f323eef0 2024-04-25T00:39:54.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.969+0000 7efd5c169000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efd540b5b50 con 0x5555f323f810 2024-04-25T00:39:54.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.969+0000 7efd5c169000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efd540b61a0 con 0x5555f31f5bc0 2024-04-25T00:39:54.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.969+0000 7efd51d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 0x5555f32aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.969+0000 7efd51d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 0x5555f32aaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49410/0 (socket says 172.21.15.29:49410) 2024-04-25T00:39:54.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.969+0000 7efd51d9b640 1 -- 172.21.15.29:0/4184529764 learned_addr learned my addr 172.21.15.29:0/4184529764 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:54.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5259c640 1 --2- 172.21.15.29:0/4184529764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f31f5bc0 0x5555f32aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd51d9b640 1 -- 172.21.15.29:0/4184529764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f31f5bc0 msgr2=0x5555f32aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:54.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd51d9b640 1 --2- 172.21.15.29:0/4184529764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f31f5bc0 0x5555f32aa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd51d9b640 1 -- 172.21.15.29:0/4184529764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 msgr2=0x5555f32aaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:54.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd51d9b640 1 --2- 172.21.15.29:0/4184529764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 0x5555f32aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd51d9b640 1 -- 172.21.15.29:0/4184529764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f2fcbe80 con 0x5555f323eef0 2024-04-25T00:39:54.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5259c640 1 --2- 172.21.15.29:0/4184529764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f31f5bc0 0x5555f32aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:54.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd51d9b640 1 --2- 172.21.15.29:0/4184529764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 0x5555f32aaa40 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7efd48006f10 tx=0x7efd4806b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb22e5d593b61112 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:54.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd337fe640 1 -- 172.21.15.29:0/4184529764 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4806e040 con 0x5555f323eef0 2024-04-25T00:39:54.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd337fe640 1 -- 172.21.15.29:0/4184529764 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd48081020 con 0x5555f323eef0 2024-04-25T00:39:54.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd337fe640 1 -- 172.21.15.29:0/4184529764 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4805bb90 con 0x5555f323eef0 2024-04-25T00:39:54.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 -- 172.21.15.29:0/4184529764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 msgr2=0x5555f32aaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 --2- 172.21.15.29:0/4184529764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 0x5555f32aaa40 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7efd48006f10 tx=0x7efd4806b000 comp rx=0 tx=0).stop 2024-04-25T00:39:54.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 -- 172.21.15.29:0/4184529764 shutdown_connections 2024-04-25T00:39:54.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 --2- 172.21.15.29:0/4184529764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 0x5555f32aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 --2- 172.21.15.29:0/4184529764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f323eef0 0x5555f32aaa40 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 --2- 172.21.15.29:0/4184529764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f31f5bc0 0x5555f32aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 -- 172.21.15.29:0/4184529764 >> 172.21.15.29:0/4184529764 conn(0x5555f31f1b70 msgr2=0x5555f324c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:54.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 -- 172.21.15.29:0/4184529764 shutdown_connections 2024-04-25T00:39:54.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 -- 172.21.15.29:0/4184529764 wait complete. 2024-04-25T00:39:54.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 -- start start 2024-04-25T00:39:54.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 Processor -- start 2024-04-25T00:39:54.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 0x5555f32af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323eef0 0x5555f32af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 0x5555f32ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5555f2ff7c40 con 0x5555f31f5bc0 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5555f30f5e80 con 0x5555f323f810 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5555f300c810 con 0x5555f323eef0 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 0x5555f32af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 10 client.0 _ll_register_callbacks cb 0x7efd480046d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 0x5555f32af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49424/0 (socket says 172.21.15.29:49424) 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5259c640 1 -- 172.21.15.29:0/2621946126 learned_addr learned my addr 172.21.15.29:0/2621946126 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd51d9b640 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323eef0 0x5555f32af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd52d9d640 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 0x5555f32ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5259c640 1 -- 172.21.15.29:0/2621946126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323eef0 msgr2=0x5555f32af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5259c640 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323eef0 0x5555f32af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5259c640 1 -- 172.21.15.29:0/2621946126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 msgr2=0x5555f32ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5259c640 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 0x5555f32ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5259c640 1 -- 172.21.15.29:0/2621946126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f3010760 con 0x5555f31f5bc0 2024-04-25T00:39:54.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd52d9d640 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 0x5555f32ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:54.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5259c640 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 0x5555f32af050 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7efd44008010 tx=0x7efd440046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d3d29d3e0442c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:54.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd317fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:39:54.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd317fa640 1 -- 172.21.15.29:0/2621946126 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd440cd270 con 0x5555f31f5bc0 2024-04-25T00:39:54.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 -- 172.21.15.29:0/2621946126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd48081020 con 0x5555f31f5bc0 2024-04-25T00:39:54.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd317fa640 1 -- 172.21.15.29:0/2621946126 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7efd440c1ac0 con 0x5555f31f5bc0 2024-04-25T00:39:54.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd317fa640 1 -- 172.21.15.29:0/2621946126 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7efd440c1cc0 con 0x5555f31f5bc0 2024-04-25T00:39:54.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 10 client.9768 fetch_fsmap learned FSMap version 92 2024-04-25T00:39:54.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 -- 172.21.15.29:0/2621946126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x5555f309c9e0 con 0x5555f31f5bc0 2024-04-25T00:39:54.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd317fa640 1 -- 172.21.15.29:0/2621946126 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd48081020 con 0x5555f31f5bc0 2024-04-25T00:39:54.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd317fa640 1 -- 172.21.15.29:0/2621946126 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 92) ==== 3494+0+0 (secure 0 0 0) 0x7efd440cd410 con 0x5555f31f5bc0 2024-04-25T00:39:54.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 10 client.9768 fetch_fsmap finished waiting for FSMap version 92 2024-04-25T00:39:54.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 10 client.9768 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:39:54.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 20 client.9768 populate_metadata read hostname 'smithi029' 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.29:6834/821427804,v1:172.21.15.29:6835/821427804] conn(0x5555f32abc30 0x5555f32ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 4 client.9768 mds_command: new command op to 9624 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.973+0000 7efd5c169000 1 -- 172.21.15.29:0/2621946126 --> [v2:172.21.15.29:6834/821427804,v1:172.21.15.29:6835/821427804] -- command(tid 0: {"prefix": "status"}) -- 0x5555f3050b20 con 0x5555f32abc30 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd51d9b640 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.29:6834/821427804,v1:172.21.15.29:6835/821427804] conn(0x5555f32abc30 0x5555f32ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd51d9b640 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.29:6834/821427804,v1:172.21.15.29:6835/821427804] conn(0x5555f32abc30 0x5555f32ac010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb2cecdc183f215 server_cookie=9cfc5940029ff912 in_seq=0 out_seq=0 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd317fa640 10 client.9768 ms_handle_connect on v2:172.21.15.29:6834/821427804 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd317fa640 1 -- 172.21.15.29:0/2621946126 <== mds.0 v2:172.21.15.29:6834/821427804 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5555f3050b20 con 0x5555f32abc30 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd317fa640 10 client.9768 handle_command_reply: tid=0 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd5c169000 1 client.9768 shutdown 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd5c169000 1 -- 172.21.15.29:0/2621946126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 msgr2=0x5555f32af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd5c169000 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 0x5555f32af050 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7efd44008010 tx=0x7efd440046d0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd5c169000 1 -- 172.21.15.29:0/2621946126 shutdown_connections 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd5c169000 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.29:6834/821427804,v1:172.21.15.29:6835/821427804] conn(0x5555f32abc30 0x5555f32ac010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd5c169000 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555f323f810 0x5555f32ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd5c169000 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5555f323eef0 0x5555f32af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd317fa640 0 client.9768 ms_handle_reset on v2:172.21.15.29:6834/821427804 2024-04-25T00:39:54.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd5c169000 1 --2- 172.21.15.29:0/2621946126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5555f31f5bc0 0x5555f32af050 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd5c169000 1 -- 172.21.15.29:0/2621946126 >> 172.21.15.29:0/2621946126 conn(0x5555f31f1b70 msgr2=0x5555f323d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:54.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd5c169000 1 -- 172.21.15.29:0/2621946126 shutdown_connections 2024-04-25T00:39:54.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd5c169000 1 -- 172.21.15.29:0/2621946126 wait complete. 2024-04-25T00:39:54.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd5c169000 20 client.9768 trim_cache size 0 max 16384 2024-04-25T00:39:54.983 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:39:54.983 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "59afbe7d-aaa8-45e2-b468-69ee150cacea", 2024-04-25T00:39:54.983 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-04-25T00:39:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 9624, 2024-04-25T00:39:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-04-25T00:39:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-04-25T00:39:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-04-25T00:39:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 5.7506582719999999, 2024-04-25T00:39:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 92, 2024-04-25T00:39:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 145, 2024-04-25T00:39:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 145, 2024-04-25T00:39:54.984 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 10.795903955 2024-04-25T00:39:54.984 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:39:54.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd58ff0640 1 -- 172.21.15.29:0/152530925 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd34067ef0 msgr2=0x7efd3406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd58ff0640 1 --2- 172.21.15.29:0/152530925 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd34067ef0 0x7efd3406a3b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efd440af000 tx=0x7efd440a7000 comp rx=0 tx=0).stop 2024-04-25T00:39:54.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd58ff0640 1 -- 172.21.15.29:0/152530925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd5415bdb0 msgr2=0x7efd5406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:54.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd58ff0640 1 --2- 172.21.15.29:0/152530925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd5415bdb0 0x7efd5406d870 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7efd4800d800 tx=0x7efd4800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd58ff0640 1 -- 172.21.15.29:0/152530925 shutdown_connections 2024-04-25T00:39:54.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd58ff0640 1 --2- 172.21.15.29:0/152530925 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd34067ef0 0x7efd3406a3b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd58ff0640 1 --2- 172.21.15.29:0/152530925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd5406ddb0 0x7efd540ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd58ff0640 1 --2- 172.21.15.29:0/152530925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd5415bdb0 0x7efd5406d870 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd58ff0640 1 --2- 172.21.15.29:0/152530925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd540d02d0 0x7efd540c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:54.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd58ff0640 1 -- 172.21.15.29:0/152530925 >> 172.21.15.29:0/152530925 conn(0x7efd5409aa30 msgr2=0x7efd54159520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:54.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd58ff0640 1 -- 172.21.15.29:0/152530925 shutdown_connections 2024-04-25T00:39:54.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:54.977+0000 7efd58ff0640 1 -- 172.21.15.29:0/152530925 wait complete. 2024-04-25T00:39:54.993 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:39:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.105+0000 7efd40681640 1 Processor -- start 2024-04-25T00:39:55.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.105+0000 7efd40681640 1 -- start start 2024-04-25T00:39:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.109+0000 7efd40681640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd3815e330 0x7efd3806db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.109+0000 7efd40681640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd380cfe90 con 0x7efd3806f270 2024-04-25T00:39:55.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.109+0000 7efd40681640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd380d0170 con 0x7efd3806b210 2024-04-25T00:39:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.109+0000 7efd40681640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efd380d0450 con 0x7efd3815e330 2024-04-25T00:39:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.109+0000 7efd3e41d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7efd3806f270 0x7efd3806f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37758/0 (socket says 172.21.15.29:37758) 2024-04-25T00:39:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.109+0000 7efd3e41d640 1 -- 172.21.15.29:0/540403653 learned_addr learned my addr 172.21.15.29:0/540403653 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.109+0000 7efd3dc1c640 1 --2- 172.21.15.29:0/540403653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd3815e330 0x7efd3806db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.109+0000 7efd3d41b640 1 -- 172.21.15.29:0/540403653 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1317929328 0 0) 0x7efd380cfe90 con 0x7efd3806f270 2024-04-25T00:39:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.109+0000 7efd3d41b640 1 -- 172.21.15.29:0/540403653 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd24003530 con 0x7efd3806f270 2024-04-25T00:39:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.109+0000 7efd3dc1c640 1 -- 172.21.15.29:0/540403653 >> v1:172.21.15.157:6789/0 conn(0x7efd3806b210 legacy=0x7efd3806b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.109+0000 7efd3d41b640 1 -- 172.21.15.29:0/540403653 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 609816131 0 0) 0x7efd24003530 con 0x7efd3806f270 2024-04-25T00:39:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.109+0000 7efd3dc1c640 1 -- 172.21.15.29:0/540403653 >> v1:172.21.15.29:6789/0 conn(0x7efd3806f270 legacy=0x7efd3806f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.109+0000 7efd3dc1c640 1 -- 172.21.15.29:0/540403653 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd380d0730 con 0x7efd3815e330 2024-04-25T00:39:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd3dc1c640 1 --2- 172.21.15.29:0/540403653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd3815e330 0x7efd3806db90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd280097f0 tx=0x7efd2801c220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8db70a83a19a24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd3d41b640 1 -- 172.21.15.29:0/540403653 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd280b0070 con 0x7efd3815e330 2024-04-25T00:39:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd3d41b640 1 -- 172.21.15.29:0/540403653 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd280ab400 con 0x7efd3815e330 2024-04-25T00:39:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd3d41b640 1 -- 172.21.15.29:0/540403653 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34001680 con 0x7efd3815e330 2024-04-25T00:39:55.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 -- 172.21.15.29:0/540403653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd3815e330 msgr2=0x7efd3806db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 --2- 172.21.15.29:0/540403653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd3815e330 0x7efd3806db90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd280097f0 tx=0x7efd2801c220 comp rx=0 tx=0).stop 2024-04-25T00:39:55.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 -- 172.21.15.29:0/540403653 shutdown_connections 2024-04-25T00:39:55.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 --2- 172.21.15.29:0/540403653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd3815e330 0x7efd3806db90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 -- 172.21.15.29:0/540403653 >> 172.21.15.29:0/540403653 conn(0x7efd380b2920 msgr2=0x7efd380b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:55.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 -- 172.21.15.29:0/540403653 shutdown_connections 2024-04-25T00:39:55.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 -- 172.21.15.29:0/540403653 wait complete. 2024-04-25T00:39:55.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 Processor -- start 2024-04-25T00:39:55.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 -- start start 2024-04-25T00:39:55.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd3806b210 0x7efd380afa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd3806f270 0x7efd380ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd38148e00 0x7efd380cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efd3816bf50 con 0x7efd3806b210 2024-04-25T00:39:55.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efd3815f460 con 0x7efd3806f270 2024-04-25T00:39:55.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd40681640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efd3816c0d0 con 0x7efd38148e00 2024-04-25T00:39:55.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd3e41d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd3806b210 0x7efd380afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd3e41d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd3806b210 0x7efd380afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49428/0 (socket says 172.21.15.29:49428) 2024-04-25T00:39:55.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd3e41d640 1 -- 172.21.15.29:0/4163542503 learned_addr learned my addr 172.21.15.29:0/4163542503 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:55.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd3dc1c640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd3806f270 0x7efd380ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd3ec1e640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd38148e00 0x7efd380cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.113+0000 7efd3e41d640 1 -- 172.21.15.29:0/4163542503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd38148e00 msgr2=0x7efd380cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd3e41d640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd38148e00 0x7efd380cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd3e41d640 1 -- 172.21.15.29:0/4163542503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd3806f270 msgr2=0x7efd380ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd3e41d640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd3806f270 0x7efd380ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd3e41d640 1 -- 172.21.15.29:0/4163542503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd38150bb0 con 0x7efd3806b210 2024-04-25T00:39:55.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd3dc1c640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd3806f270 0x7efd380ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:55.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd3e41d640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd3806b210 0x7efd380afa90 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7efd34007fd0 tx=0x7efd3400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:55.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd2effd640 1 -- 172.21.15.29:0/4163542503 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34018070 con 0x7efd3806b210 2024-04-25T00:39:55.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd2effd640 1 -- 172.21.15.29:0/4163542503 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd340072d0 con 0x7efd3806b210 2024-04-25T00:39:55.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd40681640 1 -- 172.21.15.29:0/4163542503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd38150e20 con 0x7efd3806b210 2024-04-25T00:39:55.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd2effd640 1 -- 172.21.15.29:0/4163542503 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd340047f0 con 0x7efd3806b210 2024-04-25T00:39:55.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd40681640 1 -- 172.21.15.29:0/4163542503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd381513e0 con 0x7efd3806b210 2024-04-25T00:39:55.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd40681640 1 -- 172.21.15.29:0/4163542503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd38069270 con 0x7efd3806b210 2024-04-25T00:39:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd2effd640 1 -- 172.21.15.29:0/4163542503 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7efd3401b070 con 0x7efd3806b210 2024-04-25T00:39:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd2effd640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd1c067ef0 0x7efd1c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd2effd640 1 -- 172.21.15.29:0/4163542503 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7efd34085d20 con 0x7efd3806b210 2024-04-25T00:39:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd3dc1c640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd1c067ef0 0x7efd1c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.117+0000 7efd3dc1c640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd1c067ef0 0x7efd1c06a3b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efd280ad000 tx=0x7efd28008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:55.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.121+0000 7efd2effd640 1 -- 172.21.15.29:0/4163542503 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efd34055f20 con 0x7efd3806b210 2024-04-25T00:39:55.149 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-04-25T00:39:55.149+0000 7f6c0cd00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:39:55.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.321+0000 7efd40681640 1 -- 172.21.15.29:0/4163542503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efd38153440 con 0x7efd3806b210 2024-04-25T00:39:55.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.321+0000 7efd2effd640 1 -- 172.21.15.29:0/4163542503 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v92) ==== 53+0+73 (secure 0 0 0) 0x7efd34055820 con 0x7efd3806b210 2024-04-25T00:39:55.322 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:39:55.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.325+0000 7efd40681640 1 -- 172.21.15.29:0/4163542503 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd1c067ef0 msgr2=0x7efd1c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.325+0000 7efd40681640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd1c067ef0 0x7efd1c06a3b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efd280ad000 tx=0x7efd28008040 comp rx=0 tx=0).stop 2024-04-25T00:39:55.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.325+0000 7efd40681640 1 -- 172.21.15.29:0/4163542503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd3806b210 msgr2=0x7efd380afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:55.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.325+0000 7efd40681640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd3806b210 0x7efd380afa90 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7efd34007fd0 tx=0x7efd3400cbe0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.325+0000 7efd40681640 1 -- 172.21.15.29:0/4163542503 shutdown_connections 2024-04-25T00:39:55.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.325+0000 7efd40681640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efd1c067ef0 0x7efd1c06a3b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.325+0000 7efd40681640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd38148e00 0x7efd380cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.325+0000 7efd40681640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd3806f270 0x7efd380ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.325+0000 7efd40681640 1 --2- 172.21.15.29:0/4163542503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd3806b210 0x7efd380afa90 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:55.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.325+0000 7efd40681640 1 -- 172.21.15.29:0/4163542503 >> 172.21.15.29:0/4163542503 conn(0x7efd380b2920 msgr2=0x7efd38152550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:55.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.325+0000 7efd40681640 1 -- 172.21.15.29:0/4163542503 shutdown_connections 2024-04-25T00:39:55.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55.325+0000 7efd40681640 1 -- 172.21.15.29:0/4163542503 wait complete. 2024-04-25T00:39:55.341 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:39:55.342 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:39:55.342 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-25T00:39:55.342 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:39:55.342 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:55.342 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:39:55.342 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:39:55.342 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:39:55.342 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:39:55.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:55.343 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-25T00:39:55.348 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:39:55.349 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:39:55.349 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:39:55.349 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:55.349 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:39:55.349 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:55.349 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:39:55.349 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:55.349 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-25T00:39:55.349 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:39:55.349 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:55.349 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-25T00:39:55.349 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:39:55.349 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:55.350 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-25T00:39:55.350 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:39:55.350 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 41964sec preferred_lft 41964sec 2024-04-25T00:39:55.350 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-04-25T00:39:55.350 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:55.350 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:39:55.350 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:55.350 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-25T00:39:55.350 INFO:teuthology.orchestra.run.smithi029.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:39:55.350 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 02:42:28:a5:36:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:55.350 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:39:55.350 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:55.350 INFO:teuthology.orchestra.run.smithi029.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:39:55.351 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:55.351 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:39:55.351 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:55.351 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:39:55.351 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:55.351 INFO:teuthology.orchestra.run.smithi029.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:39:55.351 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:39:55.351 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:39:55.351 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:55.351 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:39:55.351 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:39:55.351 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:39:55.351 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:55.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:55.352 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:39:55.399 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-25T00:39:55.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:55.401 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-25T00:39:55.449 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-25T00:39:55.449 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:39:55.449 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T00:39:55.449 INFO:teuthology.orchestra.run.smithi029.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:39:55.449 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T00:39:55.449 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:39:55.450 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:39:55.450 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:39:55.450 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:39:55.450 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:39:55.450 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:39:55.450 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:39:55.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:39:55.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:39:55.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:55.574 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:39:55.620 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:39:55.621 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:39:55.621 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:39:55.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:55.621 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-04-25T00:39:55.677 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:39:55.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:39:55.677 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:55.728 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:39:55.728 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:39:55.729 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:55.780 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:39:55.837 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:39:55.885 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:39:55.885 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:39:55.885 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:39:55.886 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:39:55.886 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:39:55.886 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:39:55.886 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:39:55.886 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:39:55.886 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:39:55.886 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:39:55.886 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:39:55.886 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:39:55.886 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:39:55.886 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:39:55.887 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:39:55.887 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:39:55.887 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:39:55.887 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:39:55.887 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:39:55.887 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:39:55.887 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:39:55.887 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:39:55.887 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:39:55.887 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:39:55.887 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:39:55.887 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:39:55.887 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:39:55.888 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:39:55.888 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:39:55.888 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:39:55.888 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:39:55.888 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:39:55.888 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:39:55.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:55.889 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:39:55.936 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:39:55.936 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:39:55.981 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:39:56.019 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:39:56.069 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:39:56.069 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:39:56.069 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:39:56.069 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:39:56.069 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:39:56.069 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:39:56.069 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:39:56.069 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:39:56.069 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:39:56.069 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:39:56.069 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:39:56.070 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:39:56.071 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:39:56.071 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:39:56.071 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:39:56.071 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:39:56.071 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:39:56.071 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:39:56.071 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:39:56.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:56.071 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:39:56.072 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:2024-04-25T00:39:56.069+0000 7f66eedf7580 -1 init, newargv = 0x56210657edc0 newargc=15 2024-04-25T00:39:56.072 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[54812]: starting ceph client 2024-04-25T00:39:56.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[54812]: starting fuse 2024-04-25T00:39:56.080 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:39:56.080 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:39:56.080 DEBUG:teuthology.orchestra.run.smithi029:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:39:56.080 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-04-25T00:39:56.081 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:39:56.082 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-04-25T00:39:56.082 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:39:56.082 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:39:56.082 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:39:56.082 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-04-25T00:39:56.082 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-04-25T00:39:56.082 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-04-25T00:39:56.082 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:39:56.082 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:39:56.082 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.0")) 2024-04-25T00:39:56.082 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-04-25T00:39:56.190 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.0.54812.asok 2024-04-25T00:39:56.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:39:56.202 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54812.asok 2024-04-25T00:39:56.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:56.202 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54812.asok status 2024-04-25T00:39:56.363 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:39:56.363 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-04-25T00:39:56.363 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:39:56.363 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:39:56.363 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "0", 2024-04-25T00:39:56.363 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-04-25T00:39:56.363 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:39:56.363 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "54812", 2024-04-25T00:39:56.363 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-04-25T00:39:56.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:56.363 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-04-25T00:39:56.363 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 9783, 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 9783 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2114779879 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:39:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:39:56.365 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2114779879 2024-04-25T00:39:56.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:39:56.365 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.9783 192.168.0.1:0/2114779879", 2024-04-25T00:39:56.365 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.1:0/2114779879", 2024-04-25T00:39:56.365 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-04-25T00:39:56.365 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 92, 2024-04-25T00:39:56.365 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 145, 2024-04-25T00:39:56.365 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:39:56.365 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-04-25T00:39:56.365 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-04-25T00:39:56.365 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:39:56.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:56.376 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-04-25T00:39:56.377 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:56.382 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-04-25T00:39:56.382 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:56.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:56.383 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:39:56.437 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:39:56.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f7473640 1 Processor -- start 2024-04-25T00:39:56.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f7473640 1 -- start start 2024-04-25T00:39:56.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f7473640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f015e200 0x7ff4f00d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:56.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f7473640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff4f00cf690 0x7ff4f00d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:56.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f7473640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff4f00d2a90 0x7ff4f00d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:56.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f7473640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff4f00d2ee0 con 0x7ff4f00cf690 2024-04-25T00:39:56.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f7473640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff4f00b38d0 con 0x7ff4f00d2a90 2024-04-25T00:39:56.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f7473640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f00cfc10 con 0x7ff4f015e200 2024-04-25T00:39:56.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f4a0e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff4f00cf690 0x7ff4f00d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:56.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f4a0e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff4f00cf690 0x7ff4f00d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49440/0 (socket says 172.21.15.29:49440) 2024-04-25T00:39:56.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f4a0e640 1 -- 172.21.15.29:0/1402177020 learned_addr learned my addr 172.21.15.29:0/1402177020 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:56.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f520f640 1 --2- 172.21.15.29:0/1402177020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f015e200 0x7ff4f00d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:56.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f5a10640 1 --2- 172.21.15.29:0/1402177020 >> v2:172.21.15.157:3300/0 conn(0x7ff4f00d2a90 0x7ff4f00d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:56.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f4a0e640 1 -- 172.21.15.29:0/1402177020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f015e200 msgr2=0x7ff4f00d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:56.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f4a0e640 1 --2- 172.21.15.29:0/1402177020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f015e200 0x7ff4f00d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:56.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f4a0e640 1 -- 172.21.15.29:0/1402177020 >> v2:172.21.15.157:3300/0 conn(0x7ff4f00d2a90 msgr2=0x7ff4f00d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:56.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f4a0e640 1 --2- 172.21.15.29:0/1402177020 >> v2:172.21.15.157:3300/0 conn(0x7ff4f00d2a90 0x7ff4f00d05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:56.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.609+0000 7ff4f4a0e640 1 -- 172.21.15.29:0/1402177020 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f00cfef0 con 0x7ff4f00cf690 2024-04-25T00:39:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4f4a0e640 1 --2- 172.21.15.29:0/1402177020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f00cf690 0x7ff4f00d2550 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7ff4f008d3b0 tx=0x7ff4e401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc04205bb5c3e0c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4effff640 1 -- 172.21.15.29:0/1402177020 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e40b0070 con 0x7ff4f00cf690 2024-04-25T00:39:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4effff640 1 -- 172.21.15.29:0/1402177020 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e40ab600 con 0x7ff4f00cf690 2024-04-25T00:39:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4effff640 1 -- 172.21.15.29:0/1402177020 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e40ab9d0 con 0x7ff4f00cf690 2024-04-25T00:39:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4f7473640 1 -- 172.21.15.29:0/1402177020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f00cf690 msgr2=0x7ff4f00d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4f7473640 1 --2- 172.21.15.29:0/1402177020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f00cf690 0x7ff4f00d2550 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7ff4f008d3b0 tx=0x7ff4e401c230 comp rx=0 tx=0).stop 2024-04-25T00:39:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4f7473640 1 -- 172.21.15.29:0/1402177020 shutdown_connections 2024-04-25T00:39:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4f7473640 1 --2- 172.21.15.29:0/1402177020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f00cf690 0x7ff4f00d2550 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4f7473640 1 --2- 172.21.15.29:0/1402177020 >> v2:172.21.15.157:3300/0 conn(0x7ff4f00d2a90 0x7ff4f00d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4f7473640 1 --2- 172.21.15.29:0/1402177020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f015e200 0x7ff4f00d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4f7473640 1 -- 172.21.15.29:0/1402177020 >> 172.21.15.29:0/1402177020 conn(0x7ff4f009d280 msgr2=0x7ff4f00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4f7473640 1 -- 172.21.15.29:0/1402177020 shutdown_connections 2024-04-25T00:39:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4f7473640 1 -- 172.21.15.29:0/1402177020 wait complete. 2024-04-25T00:39:56.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4f7473640 1 Processor -- start 2024-04-25T00:39:56.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.613+0000 7ff4f7473640 1 -- start start 2024-04-25T00:39:56.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f7473640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f00cf690 0x7ff4f0154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f7473640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f00d2a90 0x7ff4f006bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f7473640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f015e200 0x7ff4f006c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f7473640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f015e760 con 0x7ff4f015e200 2024-04-25T00:39:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f7473640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f015e5e0 con 0x7ff4f00d2a90 2024-04-25T00:39:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f7473640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f00b4320 con 0x7ff4f00cf690 2024-04-25T00:39:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f5a10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f015e200 0x7ff4f006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f5a10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f015e200 0x7ff4f006c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49444/0 (socket says 172.21.15.29:49444) 2024-04-25T00:39:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f5a10640 1 -- 172.21.15.29:0/367485252 learned_addr learned my addr 172.21.15.29:0/367485252 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f4a0e640 1 --2- 172.21.15.29:0/367485252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f00d2a90 0x7ff4f006bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f5a10640 1 -- 172.21.15.29:0/367485252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f00cf690 msgr2=0x7ff4f0154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f5a10640 1 --2- 172.21.15.29:0/367485252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f00cf690 0x7ff4f0154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f5a10640 1 -- 172.21.15.29:0/367485252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f00d2a90 msgr2=0x7ff4f006bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f5a10640 1 --2- 172.21.15.29:0/367485252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f00d2a90 0x7ff4f006bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f5a10640 1 -- 172.21.15.29:0/367485252 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f01546a0 con 0x7ff4f015e200 2024-04-25T00:39:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f5a10640 1 --2- 172.21.15.29:0/367485252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f015e200 0x7ff4f006c070 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8007e60 tx=0x7ff4e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4edffb640 1 -- 172.21.15.29:0/367485252 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8014620 con 0x7ff4f015e200 2024-04-25T00:39:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4edffb640 1 -- 172.21.15.29:0/367485252 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e8019070 con 0x7ff4f015e200 2024-04-25T00:39:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f7473640 1 -- 172.21.15.29:0/367485252 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f01611e0 con 0x7ff4f015e200 2024-04-25T00:39:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4edffb640 1 -- 172.21.15.29:0/367485252 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8014a20 con 0x7ff4f015e200 2024-04-25T00:39:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f7473640 1 -- 172.21.15.29:0/367485252 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f00c7c40 con 0x7ff4f015e200 2024-04-25T00:39:56.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f7473640 1 -- 172.21.15.29:0/367485252 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f0161700 con 0x7ff4f015e200 2024-04-25T00:39:56.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4edffb640 1 -- 172.21.15.29:0/367485252 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7ff4e8014bc0 con 0x7ff4f015e200 2024-04-25T00:39:56.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4edffb640 1 --2- 172.21.15.29:0/367485252 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff4dc067ef0 0x7ff4dc06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:56.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4edffb640 1 -- 172.21.15.29:0/367485252 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7ff4e8086ff0 con 0x7ff4f015e200 2024-04-25T00:39:56.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.617+0000 7ff4f520f640 1 --2- 172.21.15.29:0/367485252 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff4dc067ef0 0x7ff4dc06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:56.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.621+0000 7ff4f520f640 1 --2- 172.21.15.29:0/367485252 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff4dc067ef0 0x7ff4dc06a3b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0158180 tx=0x7ff4e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:56.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.621+0000 7ff4edffb640 1 -- 172.21.15.29:0/367485252 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff4f0161700 con 0x7ff4f015e200 2024-04-25T00:39:56.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4f7473640 1 -- 172.21.15.29:0/367485252 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff4f0069270 con 0x7ff4f015e200 2024-04-25T00:39:56.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4edffb640 1 -- 172.21.15.29:0/367485252 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v92) ==== 53+0+73 (secure 0 0 0) 0x7ff4e8057280 con 0x7ff4f015e200 2024-04-25T00:39:56.821 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:39:56.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4f7473640 1 -- 172.21.15.29:0/367485252 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff4dc067ef0 msgr2=0x7ff4dc06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:56.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4f7473640 1 --2- 172.21.15.29:0/367485252 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff4dc067ef0 0x7ff4dc06a3b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0158180 tx=0x7ff4e0008040 comp rx=0 tx=0).stop 2024-04-25T00:39:56.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4f7473640 1 -- 172.21.15.29:0/367485252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f015e200 msgr2=0x7ff4f006c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:56.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4f7473640 1 --2- 172.21.15.29:0/367485252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f015e200 0x7ff4f006c070 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8007e60 tx=0x7ff4e800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:39:56.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4f7473640 1 -- 172.21.15.29:0/367485252 shutdown_connections 2024-04-25T00:39:56.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4f7473640 1 --2- 172.21.15.29:0/367485252 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff4dc067ef0 0x7ff4dc06a3b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:56.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4f7473640 1 --2- 172.21.15.29:0/367485252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f015e200 0x7ff4f006c070 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:56.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4f7473640 1 --2- 172.21.15.29:0/367485252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f00d2a90 0x7ff4f006bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:56.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4f7473640 1 --2- 172.21.15.29:0/367485252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f00cf690 0x7ff4f0154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:56.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4f7473640 1 -- 172.21.15.29:0/367485252 >> 172.21.15.29:0/367485252 conn(0x7ff4f009d280 msgr2=0x7ff4f00b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:56.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4f7473640 1 -- 172.21.15.29:0/367485252 shutdown_connections 2024-04-25T00:39:56.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:56.821+0000 7ff4f7473640 1 -- 172.21.15.29:0/367485252 wait complete. 2024-04-25T00:39:56.839 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:39:56.839 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:39:56.839 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-25T00:39:56.839 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:39:56.839 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:56.839 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:39:56.839 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:39:56.839 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:39:56.839 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:39:56.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:56.840 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-25T00:39:56.847 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:39:56.847 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:39:56.847 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:39:56.847 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:56.847 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:39:56.848 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:56.848 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:39:56.848 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:56.848 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-25T00:39:56.848 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:39:56.848 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:56.848 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-25T00:39:56.848 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:39:56.848 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:56.848 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 41958sec preferred_lft 41958sec 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:9c:8d:4b:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:56.849 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:39:56.850 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:56.850 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:39:56.850 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:56.850 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:39:56.850 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:56.850 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:39:56.850 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:39:56.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:56.850 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:39:56.902 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:39:56.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:56.903 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-25T00:39:56.955 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-25T00:39:56.956 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:39:56.956 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:39:56.956 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:39:56.956 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:39:56.956 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:39:56.956 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:39:56.956 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:39:56.956 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:39:56.956 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:39:56.956 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:39:56.957 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:39:56.957 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:39:56.957 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:39:56.957 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:39:56.957 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:39:56.957 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:39:57.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:39:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:39:57.072 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:39:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:39:57.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:57.080 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:39:57.085 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:39:57.086 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:39:57.086 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:39:57.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:57.086 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-25T00:39:57.143 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T00:39:57.143 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:39:57.143 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:57.190 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:39:57.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:39:57.191 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:57.238 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:39:57.296 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:39:57.347 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:39:57.347 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:39:57.347 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:39:57.347 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:39:57.347 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:39:57.348 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:57.349 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:57.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:57.350 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:39:57.394 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:39:57.394 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T00:39:57.439 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:39:57.452 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:39:57.498 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:39:57.498 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:39:57.499 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:39:57.500 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:57.501 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:57.501 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:57.501 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:57.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:57.502 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:39:57.519 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-04-25T00:39:57.517+0000 7fb80cc5a580 -1 init, newargv = 0x558acaec9fc0 newargc=15 2024-04-25T00:39:57.519 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[35063]: starting ceph client 2024-04-25T00:39:57.526 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[35063]: starting fuse 2024-04-25T00:39:58.624 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:39:58.640 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:39:58.689 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:39:58.689 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:39:58.689 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:39:58.689 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:39:58.689 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:39:58.689 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:39:58.689 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:39:58.689 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:39:58.689 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:39:58.690 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:39:58.691 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:39:58.691 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:39:58.691 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:39:58.691 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:39:58.691 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:39:58.691 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:39:58.691 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:58.691 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:39:58.691 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:58.691 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:39:58.691 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:58.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:58.692 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:39:58.739 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:39:58.739 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:39:58.739 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:39:58.740 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-25T00:39:58.741 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:39:58.741 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:39:58.741 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:39:58.741 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-25T00:39:58.741 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-25T00:39:58.741 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-25T00:39:58.741 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:39:58.741 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:39:58.741 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-04-25T00:39:58.741 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-25T00:39:58.864 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.35063.asok 2024-04-25T00:39:58.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:39:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:39:58.876 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35063.asok 2024-04-25T00:39:58.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:58.877 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35063.asok status 2024-04-25T00:39:59.030 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:39:59.031 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-25T00:39:59.031 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:39:59.031 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:39:59.031 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-04-25T00:39:59.031 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-25T00:39:59.031 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T00:39:59.031 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "35063", 2024-04-25T00:39:59.031 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-25T00:39:59.031 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:39:59.031 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-25T00:39:59.031 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-25T00:39:59.031 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7858, 2024-04-25T00:39:59.031 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 7858 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3697213800 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3697213800 2024-04-25T00:39:59.032 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:39:59.033 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.7858 192.168.0.1:0/3697213800", 2024-04-25T00:39:59.033 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/3697213800", 2024-04-25T00:39:59.033 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-25T00:39:59.033 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 92, 2024-04-25T00:39:59.033 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 145, 2024-04-25T00:39:59.033 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:39:59.033 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-25T00:39:59.033 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-25T00:39:59.033 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:39:59.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:59.045 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-25T00:39:59.045 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:59.051 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-25T00:39:59.051 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:59.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:39:59.051 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:39:59.106 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-25T00:39:59.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.245+0000 7fef07bfc640 1 Processor -- start 2024-04-25T00:39:59.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.245+0000 7fef07bfc640 1 -- start start 2024-04-25T00:39:59.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.245+0000 7fef07bfc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef000b3b50 0x7fef000b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:59.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.245+0000 7fef07bfc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef000cf670 con 0x7fef0006cd40 2024-04-25T00:39:59.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.245+0000 7fef07bfc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef000b4380 con 0x7fef001456d0 2024-04-25T00:39:59.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.245+0000 7fef07bfc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fef000b40a0 con 0x7fef000b3b50 2024-04-25T00:39:59.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.245+0000 7fef06199640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fef0006cd40 0x7fef0006d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34598/0 (socket says 172.21.15.29:34598) 2024-04-25T00:39:59.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.245+0000 7fef06199640 1 -- 172.21.15.29:0/3341573957 learned_addr learned my addr 172.21.15.29:0/3341573957 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:39:59.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.245+0000 7fef05197640 1 --2- 172.21.15.29:0/3341573957 >> v2:172.21.15.157:3300/0 conn(0x7fef000b3b50 0x7fef000b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:59.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.245+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1883802812 0 0) 0x7fef000cf670 con 0x7fef0006cd40 2024-04-25T00:39:59.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.245+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feeec003530 con 0x7fef0006cd40 2024-04-25T00:39:59.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2045735058 0 0) 0x7feeec003530 con 0x7fef0006cd40 2024-04-25T00:39:59.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 >> v2:172.21.15.157:3300/0 conn(0x7fef000b3b50 msgr2=0x7fef000b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:59.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 --2- 172.21.15.29:0/3341573957 >> v2:172.21.15.157:3300/0 conn(0x7fef000b3b50 0x7fef000b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:59.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 >> v1:172.21.15.157:6789/0 conn(0x7fef001456d0 legacy=0x7fef000cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:59.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef000ce480 con 0x7fef0006cd40 2024-04-25T00:39:59.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7feef8001a00 con 0x7fef0006cd40 2024-04-25T00:39:59.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 >> v1:172.21.15.29:6789/0 conn(0x7fef0006cd40 legacy=0x7fef0006d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:59.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 --2- 172.21.15.29:0/3341573957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec0039c0 0x7feeec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:59.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 --2- 172.21.15.29:0/3341573957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeec004810 0x7feeec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:59.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 --2- 172.21.15.29:0/3341573957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feeec0051d0 0x7feeec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:59.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fef000b40a0 con 0x7feeec0051d0 2024-04-25T00:39:59.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feeec00c070 con 0x7feeec0039c0 2024-04-25T00:39:59.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7feeec00c270 con 0x7feeec004810 2024-04-25T00:39:59.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef06199640 1 --2- 172.21.15.29:0/3341573957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feeec0051d0 0x7feeec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:59.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef05998640 1 --2- 172.21.15.29:0/3341573957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec0039c0 0x7feeec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:59.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef05197640 1 --2- 172.21.15.29:0/3341573957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeec004810 0x7feeec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:59.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef06199640 1 -- 172.21.15.29:0/3341573957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeec004810 msgr2=0x7feeec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:59.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef06199640 1 --2- 172.21.15.29:0/3341573957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeec004810 0x7feeec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:59.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef06199640 1 -- 172.21.15.29:0/3341573957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec0039c0 msgr2=0x7feeec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:59.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef06199640 1 --2- 172.21.15.29:0/3341573957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec0039c0 0x7feeec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:59.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef06199640 1 -- 172.21.15.29:0/3341573957 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec003530 con 0x7feeec0051d0 2024-04-25T00:39:59.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef06199640 1 -- 172.21.15.29:0/3341573957 reap_dead start 2024-04-25T00:39:59.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef06199640 1 --2- 172.21.15.29:0/3341573957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feeec0051d0 0x7feeec00b9d0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7feef800a830 tx=0x7feef801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5c25c899f98851 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:59.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef800a480 con 0x7feeec0051d0 2024-04-25T00:39:59.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef80ac640 con 0x7feeec0051d0 2024-04-25T00:39:59.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.249+0000 7fef04996640 1 -- 172.21.15.29:0/3341573957 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef80aca30 con 0x7feeec0051d0 2024-04-25T00:39:59.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 -- 172.21.15.29:0/3341573957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feeec0051d0 msgr2=0x7feeec00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:59.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 --2- 172.21.15.29:0/3341573957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feeec0051d0 0x7feeec00b9d0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7feef800a830 tx=0x7feef801d360 comp rx=0 tx=0).stop 2024-04-25T00:39:59.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 -- 172.21.15.29:0/3341573957 shutdown_connections 2024-04-25T00:39:59.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 --2- 172.21.15.29:0/3341573957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feeec0051d0 0x7feeec00b9d0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:59.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 -- 172.21.15.29:0/3341573957 >> 172.21.15.29:0/3341573957 conn(0x7fef000b1870 msgr2=0x7fef000b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:59.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 -- 172.21.15.29:0/3341573957 shutdown_connections 2024-04-25T00:39:59.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 -- 172.21.15.29:0/3341573957 wait complete. 2024-04-25T00:39:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 Processor -- start 2024-04-25T00:39:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 -- start start 2024-04-25T00:39:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec0051d0 0x7fef001f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef001f2280 0x7fef001536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef001f2700 0x7fef00153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fef00145ab0 con 0x7fef001f2700 2024-04-25T00:39:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef000cea40 con 0x7feeec0051d0 2024-04-25T00:39:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fef000b5000 con 0x7fef001f2280 2024-04-25T00:39:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef06199640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef001f2700 0x7fef00153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef06199640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef001f2700 0x7fef00153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50206/0 (socket says 172.21.15.29:50206) 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef06199640 1 -- 172.21.15.29:0/2471111895 learned_addr learned my addr 172.21.15.29:0/2471111895 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef05197640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef001f2280 0x7fef001536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef05998640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec0051d0 0x7fef001f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef06199640 1 -- 172.21.15.29:0/2471111895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef001f2280 msgr2=0x7fef001536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef06199640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef001f2280 0x7fef001536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef06199640 1 -- 172.21.15.29:0/2471111895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec0051d0 msgr2=0x7fef001f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef06199640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec0051d0 0x7fef001f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef06199640 1 -- 172.21.15.29:0/2471111895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0015b100 con 0x7fef001f2700 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef05197640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef001f2280 0x7fef001536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef06199640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef001f2700 0x7fef00153c10 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7feef801dc00 tx=0x7feeec005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7feef67fc640 1 -- 172.21.15.29:0/2471111895 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef80b0030 con 0x7fef001f2700 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7feef67fc640 1 -- 172.21.15.29:0/2471111895 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef80019c0 con 0x7fef001f2700 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 -- 172.21.15.29:0/2471111895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef001514f0 con 0x7fef001f2700 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7feef67fc640 1 -- 172.21.15.29:0/2471111895 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef80aca10 con 0x7fef001f2700 2024-04-25T00:39:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.253+0000 7fef07bfc640 1 -- 172.21.15.29:0/2471111895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef00151790 con 0x7fef001f2700 2024-04-25T00:39:59.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.257+0000 7fef07bfc640 1 -- 172.21.15.29:0/2471111895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef000692e0 con 0x7fef001f2700 2024-04-25T00:39:59.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.257+0000 7feef67fc640 1 -- 172.21.15.29:0/2471111895 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7feef80b7420 con 0x7fef001f2700 2024-04-25T00:39:59.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.257+0000 7feef67fc640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feee4067ef0 0x7feee406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:59.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.257+0000 7feef67fc640 1 -- 172.21.15.29:0/2471111895 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7feef8120850 con 0x7fef001f2700 2024-04-25T00:39:59.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.257+0000 7fef05998640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feee4067ef0 0x7feee406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:59.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.257+0000 7fef05998640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feee4067ef0 0x7feee406a3b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7feefc000c10 tx=0x7feefc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:59.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.261+0000 7feef67fc640 1 -- 172.21.15.29:0/2471111895 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feef8017070 con 0x7fef001f2700 2024-04-25T00:39:59.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:39:59.465+0000 7fef07bfc640 1 -- 172.21.15.29:0/2471111895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fef000b2db0 con 0x7fef001f2700 2024-04-25T00:40:00.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.457+0000 7feef67fc640 1 -- 172.21.15.29:0/2471111895 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v93) ==== 136+0+0 (secure 0 0 0) 0x7feef80f0b00 con 0x7fef001f2700 2024-04-25T00:40:00.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.461+0000 7fef07bfc640 1 -- 172.21.15.29:0/2471111895 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feee4067ef0 msgr2=0x7feee406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:00.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.461+0000 7fef07bfc640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feee4067ef0 0x7feee406a3b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7feefc000c10 tx=0x7feefc009040 comp rx=0 tx=0).stop 2024-04-25T00:40:00.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.461+0000 7fef07bfc640 1 -- 172.21.15.29:0/2471111895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef001f2700 msgr2=0x7fef00153c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:00.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.461+0000 7fef07bfc640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef001f2700 0x7fef00153c10 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7feef801dc00 tx=0x7feeec005840 comp rx=0 tx=0).stop 2024-04-25T00:40:00.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.461+0000 7fef07bfc640 1 -- 172.21.15.29:0/2471111895 shutdown_connections 2024-04-25T00:40:00.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.461+0000 7fef07bfc640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7feee4067ef0 0x7feee406a3b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.461+0000 7fef07bfc640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef001f2700 0x7fef00153c10 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.461+0000 7fef07bfc640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef001f2280 0x7fef001536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.461+0000 7fef07bfc640 1 --2- 172.21.15.29:0/2471111895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec0051d0 0x7fef001f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.461+0000 7fef07bfc640 1 -- 172.21.15.29:0/2471111895 >> 172.21.15.29:0/2471111895 conn(0x7fef000b1870 msgr2=0x7fef0015a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:00.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.461+0000 7fef07bfc640 1 -- 172.21.15.29:0/2471111895 shutdown_connections 2024-04-25T00:40:00.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.461+0000 7fef07bfc640 1 -- 172.21.15.29:0/2471111895 wait complete. 2024-04-25T00:40:00.484 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-25T00:40:00.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f449e813640 1 Processor -- start 2024-04-25T00:40:00.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f449e813640 1 -- start start 2024-04-25T00:40:00.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f449e813640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f449815f3c0 0x7f44980cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:00.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f449e813640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44980b3720 con 0x7f449815e8d0 2024-04-25T00:40:00.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f449e813640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44980b39c0 con 0x7f44980b4780 2024-04-25T00:40:00.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f449e813640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f44980b42f0 con 0x7f449815f3c0 2024-04-25T00:40:00.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f4497fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f449815e8d0 0x7f449815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34610/0 (socket says 172.21.15.29:34610) 2024-04-25T00:40:00.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f4497fff640 1 -- 172.21.15.29:0/942137305 learned_addr learned my addr 172.21.15.29:0/942137305 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:00.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f449cdb0640 1 --2- 172.21.15.29:0/942137305 >> v2:172.21.15.29:3300/0 conn(0x7f449815f3c0 0x7f44980cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:00.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f4496ffd640 1 -- 172.21.15.29:0/942137305 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 290796376 0 0) 0x7f44980b3720 con 0x7f449815e8d0 2024-04-25T00:40:00.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f4496ffd640 1 -- 172.21.15.29:0/942137305 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4484003530 con 0x7f449815e8d0 2024-04-25T00:40:00.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f449cdb0640 1 -- 172.21.15.29:0/942137305 >> v1:172.21.15.157:6789/0 conn(0x7f44980b4780 legacy=0x7f449815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:00.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f449cdb0640 1 -- 172.21.15.29:0/942137305 >> v1:172.21.15.29:6789/0 conn(0x7f449815e8d0 legacy=0x7f449815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:00.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.613+0000 7f449cdb0640 1 -- 172.21.15.29:0/942137305 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44980b3ca0 con 0x7f449815f3c0 2024-04-25T00:40:00.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449cdb0640 1 --2- 172.21.15.29:0/942137305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449815f3c0 0x7f44980cc240 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f4490009d90 tx=0x7f449001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd96bb498f6c231e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:00.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f4496ffd640 1 -- 172.21.15.29:0/942137305 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44900b2070 con 0x7f449815f3c0 2024-04-25T00:40:00.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f4496ffd640 1 -- 172.21.15.29:0/942137305 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44900a58c0 con 0x7f449815f3c0 2024-04-25T00:40:00.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f4496ffd640 1 -- 172.21.15.29:0/942137305 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44900a5c90 con 0x7f449815f3c0 2024-04-25T00:40:00.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 -- 172.21.15.29:0/942137305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449815f3c0 msgr2=0x7f44980cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:00.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 --2- 172.21.15.29:0/942137305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449815f3c0 0x7f44980cc240 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f4490009d90 tx=0x7f449001c930 comp rx=0 tx=0).stop 2024-04-25T00:40:00.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 -- 172.21.15.29:0/942137305 shutdown_connections 2024-04-25T00:40:00.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 --2- 172.21.15.29:0/942137305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449815f3c0 0x7f44980cc240 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 -- 172.21.15.29:0/942137305 >> 172.21.15.29:0/942137305 conn(0x7f44980b1870 msgr2=0x7f44980b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:00.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 -- 172.21.15.29:0/942137305 shutdown_connections 2024-04-25T00:40:00.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 -- 172.21.15.29:0/942137305 wait complete. 2024-04-25T00:40:00.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 Processor -- start 2024-04-25T00:40:00.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 -- start start 2024-04-25T00:40:00.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44980b4780 0x7f449813f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:00.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449815e8d0 0x7f449813d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:00.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449815f3c0 0x7f4498159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:00.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4498168e00 con 0x7f449815f3c0 2024-04-25T00:40:00.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4498168c80 con 0x7f449815e8d0 2024-04-25T00:40:00.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449e813640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4498168f80 con 0x7f44980b4780 2024-04-25T00:40:00.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449cdb0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449815f3c0 0x7f4498159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:00.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449cdb0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449815f3c0 0x7f4498159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50230/0 (socket says 172.21.15.29:50230) 2024-04-25T00:40:00.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.617+0000 7f449cdb0640 1 -- 172.21.15.29:0/731072298 learned_addr learned my addr 172.21.15.29:0/731072298 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:00.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f44977fe640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449815e8d0 0x7f449813d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:00.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f4497fff640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44980b4780 0x7f449813f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:00.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f449cdb0640 1 -- 172.21.15.29:0/731072298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44980b4780 msgr2=0x7f449813f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:00.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f449cdb0640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44980b4780 0x7f449813f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f449cdb0640 1 -- 172.21.15.29:0/731072298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449815e8d0 msgr2=0x7f449813d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:00.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f449cdb0640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449815e8d0 0x7f449813d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f449cdb0640 1 -- 172.21.15.29:0/731072298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449813fe70 con 0x7f449815f3c0 2024-04-25T00:40:00.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f449cdb0640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449815f3c0 0x7f4498159c50 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f449001a560 tx=0x7f44900196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:00.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f4494ff9640 1 -- 172.21.15.29:0/731072298 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44900b2050 con 0x7f449815f3c0 2024-04-25T00:40:00.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f4494ff9640 1 -- 172.21.15.29:0/731072298 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44900be070 con 0x7f449815f3c0 2024-04-25T00:40:00.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f449e813640 1 -- 172.21.15.29:0/731072298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4498140150 con 0x7f449815f3c0 2024-04-25T00:40:00.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f4494ff9640 1 -- 172.21.15.29:0/731072298 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44900ad850 con 0x7f449815f3c0 2024-04-25T00:40:00.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f449e813640 1 -- 172.21.15.29:0/731072298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4498140710 con 0x7f449815f3c0 2024-04-25T00:40:00.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f449e813640 1 -- 172.21.15.29:0/731072298 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4498069270 con 0x7f449815f3c0 2024-04-25T00:40:00.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f4494ff9640 1 -- 172.21.15.29:0/731072298 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f44900ad9f0 con 0x7f449815f3c0 2024-04-25T00:40:00.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f4494ff9640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4478067ef0 0x7f447806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:00.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f4494ff9640 1 -- 172.21.15.29:0/731072298 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f4490121910 con 0x7f449815f3c0 2024-04-25T00:40:00.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f4497fff640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4478067ef0 0x7f447806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:00.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.621+0000 7f4497fff640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4478067ef0 0x7f447806a3b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4488005890 tx=0x7f4488009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:00.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.625+0000 7f4494ff9640 1 -- 172.21.15.29:0/731072298 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4490016520 con 0x7f449815f3c0 2024-04-25T00:40:00.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.809+0000 7f449e813640 1 -- 172.21.15.29:0/731072298 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f44980cc120 con 0x7f4478067ef0 2024-04-25T00:40:00.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.813+0000 7f4494ff9640 1 -- 172.21.15.29:0/731072298 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f44980cc120 con 0x7f4478067ef0 2024-04-25T00:40:00.812 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:40:00.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:00.812 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-04-25T00:40:00.812 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:00.812 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:40:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.813+0000 7f449e813640 1 -- 172.21.15.29:0/731072298 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4478067ef0 msgr2=0x7f447806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.813+0000 7f449e813640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4478067ef0 0x7f447806a3b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4488005890 tx=0x7f4488009040 comp rx=0 tx=0).stop 2024-04-25T00:40:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.813+0000 7f449e813640 1 -- 172.21.15.29:0/731072298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449815f3c0 msgr2=0x7f4498159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.813+0000 7f449e813640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449815f3c0 0x7f4498159c50 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f449001a560 tx=0x7f44900196d0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.813+0000 7f449e813640 1 -- 172.21.15.29:0/731072298 shutdown_connections 2024-04-25T00:40:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.813+0000 7f449e813640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4478067ef0 0x7f447806a3b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.813+0000 7f449e813640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f449815f3c0 0x7f4498159c50 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.813+0000 7f449e813640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f449815e8d0 0x7f449813d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.813+0000 7f449e813640 1 --2- 172.21.15.29:0/731072298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44980b4780 0x7f449813f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.813+0000 7f449e813640 1 -- 172.21.15.29:0/731072298 >> 172.21.15.29:0/731072298 conn(0x7f44980b1870 msgr2=0x7f449806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.813+0000 7f449e813640 1 -- 172.21.15.29:0/731072298 shutdown_connections 2024-04-25T00:40:00.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.813+0000 7f449e813640 1 -- 172.21.15.29:0/731072298 wait complete. 2024-04-25T00:40:00.831 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:40:00.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca6964640 1 Processor -- start 2024-04-25T00:40:00.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca6964640 1 -- start start 2024-04-25T00:40:00.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca6964640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faca00b3c30 0x7faca006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:00.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca6964640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faca00b4b10 0x7faca00ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:00.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca6964640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca00b41e0 con 0x7faca00b3140 2024-04-25T00:40:00.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca6964640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faca006d960 con 0x7faca00b4b10 2024-04-25T00:40:00.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca6964640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faca006dc40 con 0x7faca00b3c30 2024-04-25T00:40:00.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca4f01640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faca00b4b10 0x7faca00ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:00.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7fac9f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faca00b3c30 0x7faca006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:00.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca4f01640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faca00b4b10 0x7faca00ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:50614/0 (socket says 172.21.15.29:50614) 2024-04-25T00:40:00.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca4f01640 1 -- 172.21.15.29:0/2551046472 learned_addr learned my addr 172.21.15.29:0/2551046472 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:00.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7fac9ffff640 1 --1- 172.21.15.29:0/2551046472 >> v1:172.21.15.157:6789/0 conn(0x7faca00b3140 0x7faca00b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:35570/0 (socket says 172.21.15.29:35570) 2024-04-25T00:40:00.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca4f01640 1 -- 172.21.15.29:0/2551046472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faca00b3c30 msgr2=0x7faca006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:00.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca4f01640 1 --2- 172.21.15.29:0/2551046472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faca00b3c30 0x7faca006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca4f01640 1 -- 172.21.15.29:0/2551046472 >> v1:172.21.15.157:6789/0 conn(0x7faca00b3140 legacy=0x7faca00b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:00.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.957+0000 7faca4f01640 1 -- 172.21.15.29:0/2551046472 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca006df20 con 0x7faca00b4b10 2024-04-25T00:40:00.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7faca4f01640 1 --2- 172.21.15.29:0/2551046472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca00b4b10 0x7faca00ce240 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fac98009e50 tx=0x7fac9801ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b069e24f2d6c99fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:00.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7fac9effd640 1 -- 172.21.15.29:0/2551046472 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac980a5700 con 0x7faca00b4b10 2024-04-25T00:40:00.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7fac9effd640 1 -- 172.21.15.29:0/2551046472 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac980a58a0 con 0x7faca00b4b10 2024-04-25T00:40:00.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7fac9effd640 1 -- 172.21.15.29:0/2551046472 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac980a5c70 con 0x7faca00b4b10 2024-04-25T00:40:00.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7faca6964640 1 -- 172.21.15.29:0/2551046472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca00b4b10 msgr2=0x7faca00ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:00.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7faca6964640 1 --2- 172.21.15.29:0/2551046472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca00b4b10 0x7faca00ce240 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fac98009e50 tx=0x7fac9801ca30 comp rx=0 tx=0).stop 2024-04-25T00:40:00.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7faca6964640 1 -- 172.21.15.29:0/2551046472 shutdown_connections 2024-04-25T00:40:00.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7faca6964640 1 --2- 172.21.15.29:0/2551046472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca00b4b10 0x7faca00ce240 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7faca6964640 1 --2- 172.21.15.29:0/2551046472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faca00b3c30 0x7faca006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7faca6964640 1 -- 172.21.15.29:0/2551046472 >> 172.21.15.29:0/2551046472 conn(0x7faca00b1870 msgr2=0x7faca00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:00.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7faca6964640 1 -- 172.21.15.29:0/2551046472 shutdown_connections 2024-04-25T00:40:00.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7faca6964640 1 -- 172.21.15.29:0/2551046472 wait complete. 2024-04-25T00:40:00.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7faca6964640 1 Processor -- start 2024-04-25T00:40:00.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.961+0000 7faca6964640 1 -- start start 2024-04-25T00:40:00.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7faca6964640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faca00b3140 0x7faca015ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:00.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7faca6964640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca00b3c30 0x7faca01376c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:00.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7faca6964640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faca00b4b10 0x7faca015e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:00.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7faca6964640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faca0168d80 con 0x7faca00b3140 2024-04-25T00:40:00.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7faca6964640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faca0168c00 con 0x7faca00b3c30 2024-04-25T00:40:00.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7faca6964640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faca0168f00 con 0x7faca00b4b10 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faca00b3140 0x7faca015ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faca00b3140 0x7faca015ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50236/0 (socket says 172.21.15.29:50236) 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9ffff640 1 -- 172.21.15.29:0/2310655858 learned_addr learned my addr 172.21.15.29:0/2310655858 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7faca4f01640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faca00b4b10 0x7faca015e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9f7fe640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca00b3c30 0x7faca01376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9ffff640 1 -- 172.21.15.29:0/2310655858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faca00b4b10 msgr2=0x7faca015e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9ffff640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faca00b4b10 0x7faca015e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9ffff640 1 -- 172.21.15.29:0/2310655858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca00b3c30 msgr2=0x7faca01376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9ffff640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca00b3c30 0x7faca01376c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9ffff640 1 -- 172.21.15.29:0/2310655858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca0137c00 con 0x7faca00b3140 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9ffff640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faca00b3140 0x7faca015ba60 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c420 tx=0x7fac9000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9cff9640 1 -- 172.21.15.29:0/2310655858 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900074e0 con 0x7faca00b3140 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9cff9640 1 -- 172.21.15.29:0/2310655858 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac90007680 con 0x7faca00b3140 2024-04-25T00:40:00.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9cff9640 1 -- 172.21.15.29:0/2310655858 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90004a30 con 0x7faca00b3140 2024-04-25T00:40:00.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7faca6964640 1 -- 172.21.15.29:0/2310655858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca015b200 con 0x7faca00b3140 2024-04-25T00:40:00.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7faca6964640 1 -- 172.21.15.29:0/2310655858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca0138d50 con 0x7faca00b3140 2024-04-25T00:40:00.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7faca6964640 1 -- 172.21.15.29:0/2310655858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca0069270 con 0x7faca00b3140 2024-04-25T00:40:00.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9cff9640 1 -- 172.21.15.29:0/2310655858 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fac9001a020 con 0x7faca00b3140 2024-04-25T00:40:00.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9cff9640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fac8c067ef0 0x7fac8c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:00.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9cff9640 1 -- 172.21.15.29:0/2310655858 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fac90085ff0 con 0x7faca00b3140 2024-04-25T00:40:00.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9f7fe640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fac8c067ef0 0x7fac8c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:00.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.965+0000 7fac9f7fe640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fac8c067ef0 0x7fac8c06a3b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7faca0155040 tx=0x7fac94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:00.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:00.969+0000 7fac9cff9640 1 -- 172.21.15.29:0/2310655858 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fac90056240 con 0x7faca00b3140 2024-04-25T00:40:01.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.129+0000 7faca6964640 1 -- 172.21.15.29:0/2310655858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7faca00b3860 con 0x7faca00b3140 2024-04-25T00:40:01.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.129+0000 7fac9cff9640 1 -- 172.21.15.29:0/2310655858 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fac90055b40 con 0x7faca00b3140 2024-04-25T00:40:01.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.133+0000 7faca6964640 1 -- 172.21.15.29:0/2310655858 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fac8c067ef0 msgr2=0x7fac8c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:01.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.133+0000 7faca6964640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fac8c067ef0 0x7fac8c06a3b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7faca0155040 tx=0x7fac94008040 comp rx=0 tx=0).stop 2024-04-25T00:40:01.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.133+0000 7faca6964640 1 -- 172.21.15.29:0/2310655858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faca00b3140 msgr2=0x7faca015ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:01.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.133+0000 7faca6964640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faca00b3140 0x7faca015ba60 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c420 tx=0x7fac9000c8e0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.133+0000 7faca6964640 1 -- 172.21.15.29:0/2310655858 shutdown_connections 2024-04-25T00:40:01.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.133+0000 7faca6964640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fac8c067ef0 0x7fac8c06a3b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.133+0000 7faca6964640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faca00b4b10 0x7faca015e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.133+0000 7faca6964640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca00b3c30 0x7faca01376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.133+0000 7faca6964640 1 --2- 172.21.15.29:0/2310655858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faca00b3140 0x7faca015ba60 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.133+0000 7faca6964640 1 -- 172.21.15.29:0/2310655858 >> 172.21.15.29:0/2310655858 conn(0x7faca00b1870 msgr2=0x7faca0143f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:01.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.133+0000 7faca6964640 1 -- 172.21.15.29:0/2310655858 shutdown_connections 2024-04-25T00:40:01.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.133+0000 7faca6964640 1 -- 172.21.15.29:0/2310655858 wait complete. 2024-04-25T00:40:01.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:01.148 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol54280) 2024-04-25T00:40:01.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:01.170 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp8010/subvol291) 2024-04-25T00:40:01.224 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol54280 2024-04-25T00:40:01.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fee110640 1 Processor -- start 2024-04-25T00:40:01.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fee110640 1 -- start start 2024-04-25T00:40:01.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fee110640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1fe80d1c10 0x7f1fe80d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:01.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fee110640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1fe815c290 0x7f1fe80cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:01.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fee110640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe80af770 con 0x7f1fe80aea40 2024-04-25T00:40:01.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fee110640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1fe80b0380 con 0x7f1fe80d1c10 2024-04-25T00:40:01.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fee110640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1fe80afa50 con 0x7f1fe815c290 2024-04-25T00:40:01.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fe77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1fe80d1c10 0x7f1fe80d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:01.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fe77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1fe80d1c10 0x7f1fe80d1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50248/0 (socket says 172.21.15.29:50248) 2024-04-25T00:40:01.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fe77fe640 1 -- 172.21.15.29:0/3695495821 learned_addr learned my addr 172.21.15.29:0/3695495821 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:01.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fe6ffd640 1 --2- 172.21.15.29:0/3695495821 >> v2:172.21.15.157:3300/0 conn(0x7f1fe815c290 0x7f1fe80cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:01.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fe77fe640 1 -- 172.21.15.29:0/3695495821 >> v2:172.21.15.157:3300/0 conn(0x7f1fe815c290 msgr2=0x7f1fe80cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:01.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fe77fe640 1 --2- 172.21.15.29:0/3695495821 >> v2:172.21.15.157:3300/0 conn(0x7f1fe815c290 0x7f1fe80cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fe77fe640 1 -- 172.21.15.29:0/3695495821 >> v1:172.21.15.157:6789/0 conn(0x7f1fe80aea40 legacy=0x7f1fe80aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:01.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fe77fe640 1 -- 172.21.15.29:0/3695495821 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe80afd30 con 0x7f1fe80d1c10 2024-04-25T00:40:01.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.393+0000 7f1fe6ffd640 1 --2- 172.21.15.29:0/3695495821 >> v2:172.21.15.157:3300/0 conn(0x7f1fe815c290 0x7f1fe80cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:01.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.397+0000 7f1fe77fe640 1 --2- 172.21.15.29:0/3695495821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe80d1c10 0x7f1fe80d1ff0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f1fe815fb10 tx=0x7f1fd801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0f62b1a5f5b80b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:01.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.397+0000 7f1fe67fc640 1 -- 172.21.15.29:0/3695495821 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80ab5f0 con 0x7f1fe80d1c10 2024-04-25T00:40:01.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.397+0000 7f1fe67fc640 1 -- 172.21.15.29:0/3695495821 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd80b0070 con 0x7f1fe80d1c10 2024-04-25T00:40:01.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.397+0000 7f1fe67fc640 1 -- 172.21.15.29:0/3695495821 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80ab9b0 con 0x7f1fe80d1c10 2024-04-25T00:40:01.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.397+0000 7f1fee110640 1 -- 172.21.15.29:0/3695495821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe80d1c10 msgr2=0x7f1fe80d1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:01.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.397+0000 7f1fee110640 1 --2- 172.21.15.29:0/3695495821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe80d1c10 0x7f1fe80d1ff0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f1fe815fb10 tx=0x7f1fd801c4e0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.397+0000 7f1fee110640 1 -- 172.21.15.29:0/3695495821 shutdown_connections 2024-04-25T00:40:01.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.397+0000 7f1fee110640 1 --2- 172.21.15.29:0/3695495821 >> v2:172.21.15.157:3300/0 conn(0x7f1fe815c290 0x7f1fe80cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.397+0000 7f1fee110640 1 --2- 172.21.15.29:0/3695495821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe80d1c10 0x7f1fe80d1ff0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.397+0000 7f1fee110640 1 -- 172.21.15.29:0/3695495821 >> 172.21.15.29:0/3695495821 conn(0x7f1fe809aa30 msgr2=0x7f1fe809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:01.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.397+0000 7f1fee110640 1 -- 172.21.15.29:0/3695495821 shutdown_connections 2024-04-25T00:40:01.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.397+0000 7f1fee110640 1 -- 172.21.15.29:0/3695495821 wait complete. 2024-04-25T00:40:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fee110640 1 Processor -- start 2024-04-25T00:40:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fee110640 1 -- start start 2024-04-25T00:40:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fee110640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe80aea40 0x7f1fe806f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fee110640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe80d1c10 0x7f1fe806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fee110640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe815c290 0x7f1fe8152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fee110640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe80b11a0 con 0x7f1fe80d1c10 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fee110640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe80b1020 con 0x7f1fe815c290 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fee110640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe80b1320 con 0x7f1fe80aea40 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fe6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe80d1c10 0x7f1fe806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fe6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe80d1c10 0x7f1fe806fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50262/0 (socket says 172.21.15.29:50262) 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fe6ffd640 1 -- 172.21.15.29:0/2593873740 learned_addr learned my addr 172.21.15.29:0/2593873740 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fe77fe640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe80aea40 0x7f1fe806f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fe7fff640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe815c290 0x7f1fe8152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fe6ffd640 1 -- 172.21.15.29:0/2593873740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe80aea40 msgr2=0x7f1fe806f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fe6ffd640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe80aea40 0x7f1fe806f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fe6ffd640 1 -- 172.21.15.29:0/2593873740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe815c290 msgr2=0x7f1fe8152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fe6ffd640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe815c290 0x7f1fe8152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fe6ffd640 1 -- 172.21.15.29:0/2593873740 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe81541b0 con 0x7f1fe80d1c10 2024-04-25T00:40:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fe6ffd640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe80d1c10 0x7f1fe806fca0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000c820 tx=0x7f1fd000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fdffff640 1 -- 172.21.15.29:0/2593873740 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0015070 con 0x7f1fe80d1c10 2024-04-25T00:40:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fdffff640 1 -- 172.21.15.29:0/2593873740 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd0007a30 con 0x7f1fe80d1c10 2024-04-25T00:40:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fdffff640 1 -- 172.21.15.29:0/2593873740 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0007e40 con 0x7f1fe80d1c10 2024-04-25T00:40:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fee110640 1 -- 172.21.15.29:0/2593873740 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe8154490 con 0x7f1fe80d1c10 2024-04-25T00:40:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fee110640 1 -- 172.21.15.29:0/2593873740 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe8154a50 con 0x7f1fe80d1c10 2024-04-25T00:40:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.401+0000 7f1fee110640 1 -- 172.21.15.29:0/2593873740 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe8069270 con 0x7f1fe80d1c10 2024-04-25T00:40:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.405+0000 7f1fdffff640 1 -- 172.21.15.29:0/2593873740 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f1fd0018070 con 0x7f1fe80d1c10 2024-04-25T00:40:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.405+0000 7f1fdffff640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fc8067ef0 0x7f1fc806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.405+0000 7f1fdffff640 1 -- 172.21.15.29:0/2593873740 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f1fd00867c0 con 0x7f1fe80d1c10 2024-04-25T00:40:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.405+0000 7f1fe77fe640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fc8067ef0 0x7f1fc806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.405+0000 7f1fe77fe640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fc8067ef0 0x7f1fc806a3b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80af000 tx=0x7f1fd80a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:01.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.409+0000 7f1fdffff640 1 -- 172.21.15.29:0/2593873740 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1fd0056940 con 0x7f1fe80d1c10 2024-04-25T00:40:01.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.589+0000 7f1fee110640 1 -- 172.21.15.29:0/2593873740 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol54280", "target": ["mon-mgr", ""]}) -- 0x7f1fe80b7b10 con 0x7f1fc8067ef0 2024-04-25T00:40:01.597 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:01.597+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:01.597 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:01.597+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:01.597 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:01.597+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:01.597 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:01.597+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:01.597 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:01.597+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:01.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.629+0000 7f1fdffff640 1 -- 172.21.15.29:0/2593873740 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f1fe80b7b10 con 0x7f1fc8067ef0 2024-04-25T00:40:01.630 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/_nogroup/subvol54280 2024-04-25T00:40:01.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.633+0000 7f1fee110640 1 -- 172.21.15.29:0/2593873740 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fc8067ef0 msgr2=0x7f1fc806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:01.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.633+0000 7f1fee110640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fc8067ef0 0x7f1fc806a3b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80af000 tx=0x7f1fd80a7000 comp rx=0 tx=0).stop 2024-04-25T00:40:01.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.633+0000 7f1fee110640 1 -- 172.21.15.29:0/2593873740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe80d1c10 msgr2=0x7f1fe806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:01.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.633+0000 7f1fee110640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe80d1c10 0x7f1fe806fca0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000c820 tx=0x7f1fd000cce0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.633+0000 7f1fee110640 1 -- 172.21.15.29:0/2593873740 shutdown_connections 2024-04-25T00:40:01.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.633+0000 7f1fee110640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1fc8067ef0 0x7f1fc806a3b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.633+0000 7f1fee110640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe815c290 0x7f1fe8152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.633+0000 7f1fee110640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe80d1c10 0x7f1fe806fca0 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.633+0000 7f1fee110640 1 --2- 172.21.15.29:0/2593873740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fe80aea40 0x7f1fe806f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.633+0000 7f1fee110640 1 -- 172.21.15.29:0/2593873740 >> 172.21.15.29:0/2593873740 conn(0x7f1fe809aa30 msgr2=0x7f1fe80b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:01.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.633+0000 7f1fee110640 1 -- 172.21.15.29:0/2593873740 shutdown_connections 2024-04-25T00:40:01.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.633+0000 7f1fee110640 1 -- 172.21.15.29:0/2593873740 wait complete. 2024-04-25T00:40:01.647 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol291 subvol_grp8010 2024-04-25T00:40:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba654e0640 1 Processor -- start 2024-04-25T00:40:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba654e0640 1 -- start start 2024-04-25T00:40:01.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba654e0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fba6015c120 0x7fba600d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:01.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba654e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600ced70 0x7fba600b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:01.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba654e0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba600d0b50 con 0x7fba600d28d0 2024-04-25T00:40:01.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba654e0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fba600b4d10 con 0x7fba6015c120 2024-04-25T00:40:01.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba654e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba600b4ff0 con 0x7fba600ced70 2024-04-25T00:40:01.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba5e7fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fba600d28d0 0x7fba600ce660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34624/0 (socket says 172.21.15.29:34624) 2024-04-25T00:40:01.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba5e7fc640 1 -- 172.21.15.29:0/3597498990 learned_addr learned my addr 172.21.15.29:0/3597498990 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:01.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba5effd640 1 --2- 172.21.15.29:0/3597498990 >> v2:172.21.15.29:3300/0 conn(0x7fba6015c120 0x7fba600d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:01.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba5dffb640 1 -- 172.21.15.29:0/3597498990 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2903187118 0 0) 0x7fba600d0b50 con 0x7fba600d28d0 2024-04-25T00:40:01.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba5f7fe640 1 --2- 172.21.15.29:0/3597498990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600ced70 0x7fba600b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:01.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba5dffb640 1 -- 172.21.15.29:0/3597498990 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba4c003530 con 0x7fba600d28d0 2024-04-25T00:40:01.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba5effd640 1 -- 172.21.15.29:0/3597498990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600ced70 msgr2=0x7fba600b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:01.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba5effd640 1 --2- 172.21.15.29:0/3597498990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600ced70 0x7fba600b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba5effd640 1 -- 172.21.15.29:0/3597498990 >> v1:172.21.15.29:6789/0 conn(0x7fba600d28d0 legacy=0x7fba600ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:01.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba5dffb640 1 -- 172.21.15.29:0/3597498990 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3506576717 0 0) 0x7fba4c003530 con 0x7fba600d28d0 2024-04-25T00:40:01.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.765+0000 7fba5effd640 1 -- 172.21.15.29:0/3597498990 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba600b52d0 con 0x7fba6015c120 2024-04-25T00:40:01.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba5effd640 1 --2- 172.21.15.29:0/3597498990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba6015c120 0x7fba600d0300 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fba50009dc0 tx=0x7fba5001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81328f2d37e3ba13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:01.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba5dffb640 1 -- 172.21.15.29:0/3597498990 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba500b2070 con 0x7fba6015c120 2024-04-25T00:40:01.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba5dffb640 1 -- 172.21.15.29:0/3597498990 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba500a5610 con 0x7fba6015c120 2024-04-25T00:40:01.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba5dffb640 1 -- 172.21.15.29:0/3597498990 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48001680 con 0x7fba6015c120 2024-04-25T00:40:01.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 -- 172.21.15.29:0/3597498990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba6015c120 msgr2=0x7fba600d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:01.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 --2- 172.21.15.29:0/3597498990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba6015c120 0x7fba600d0300 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fba50009dc0 tx=0x7fba5001c810 comp rx=0 tx=0).stop 2024-04-25T00:40:01.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 -- 172.21.15.29:0/3597498990 shutdown_connections 2024-04-25T00:40:01.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 --2- 172.21.15.29:0/3597498990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba600ced70 0x7fba600b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 --2- 172.21.15.29:0/3597498990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba6015c120 0x7fba600d0300 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 -- 172.21.15.29:0/3597498990 >> 172.21.15.29:0/3597498990 conn(0x7fba600d1d20 msgr2=0x7fba600d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:01.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 -- 172.21.15.29:0/3597498990 shutdown_connections 2024-04-25T00:40:01.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 -- 172.21.15.29:0/3597498990 wait complete. 2024-04-25T00:40:01.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 Processor -- start 2024-04-25T00:40:01.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 -- start start 2024-04-25T00:40:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba600ced70 0x7fba600c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba600d28d0 0x7fba6015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba6015c120 0x7fba6014d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fba600b6910 con 0x7fba600ced70 2024-04-25T00:40:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba600b6790 con 0x7fba600d28d0 2024-04-25T00:40:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba654e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba600b6a90 con 0x7fba6015c120 2024-04-25T00:40:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba5effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba600ced70 0x7fba600c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba5effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba600ced70 0x7fba600c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50304/0 (socket says 172.21.15.29:50304) 2024-04-25T00:40:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba5effd640 1 -- 172.21.15.29:0/3034040771 learned_addr learned my addr 172.21.15.29:0/3034040771 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.769+0000 7fba5effd640 1 -- 172.21.15.29:0/3034040771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba6015c120 msgr2=0x7fba6014d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba5effd640 1 --2- 172.21.15.29:0/3034040771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba6015c120 0x7fba6014d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba5effd640 1 -- 172.21.15.29:0/3034040771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba600d28d0 msgr2=0x7fba6015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba5effd640 1 --2- 172.21.15.29:0/3034040771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba600d28d0 0x7fba6015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba5effd640 1 -- 172.21.15.29:0/3034040771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba600c6540 con 0x7fba600ced70 2024-04-25T00:40:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba5effd640 1 --2- 172.21.15.29:0/3034040771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba600ced70 0x7fba600c7170 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fba500b0040 tx=0x7fba50019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba577fe640 1 -- 172.21.15.29:0/3034040771 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba500b2050 con 0x7fba600ced70 2024-04-25T00:40:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba577fe640 1 -- 172.21.15.29:0/3034040771 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba500be070 con 0x7fba600ced70 2024-04-25T00:40:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba654e0640 1 -- 172.21.15.29:0/3034040771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba600c6820 con 0x7fba600ced70 2024-04-25T00:40:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba577fe640 1 -- 172.21.15.29:0/3034040771 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50009450 con 0x7fba600ced70 2024-04-25T00:40:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba654e0640 1 -- 172.21.15.29:0/3034040771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba600c76b0 con 0x7fba600ced70 2024-04-25T00:40:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba654e0640 1 -- 172.21.15.29:0/3034040771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba600692e0 con 0x7fba600ced70 2024-04-25T00:40:01.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba577fe640 1 -- 172.21.15.29:0/3034040771 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fba500ada50 con 0x7fba600ced70 2024-04-25T00:40:01.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba577fe640 1 --2- 172.21.15.29:0/3034040771 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fba40067ef0 0x7fba4006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:01.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba5e7fc640 1 --2- 172.21.15.29:0/3034040771 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fba40067ef0 0x7fba4006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:01.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba577fe640 1 -- 172.21.15.29:0/3034040771 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fba501218e0 con 0x7fba600ced70 2024-04-25T00:40:01.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.773+0000 7fba5e7fc640 1 --2- 172.21.15.29:0/3034040771 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fba40067ef0 0x7fba4006a3b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fba480016d0 tx=0x7fba48001190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.777+0000 7fba577fe640 1 -- 172.21.15.29:0/3034040771 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fba500164a0 con 0x7fba600ced70 2024-04-25T00:40:01.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.965+0000 7fba654e0640 1 -- 172.21.15.29:0/3034040771 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol291", "group_name": "subvol_grp8010", "target": ["mon-mgr", ""]}) -- 0x7fba6015e3a0 con 0x7fba40067ef0 2024-04-25T00:40:01.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.993+0000 7fba577fe640 1 -- 172.21.15.29:0/3034040771 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+34 (secure 0 0 0) 0x7fba48007d90 con 0x7fba40067ef0 2024-04-25T00:40:01.995 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/subvol_grp8010/subvol291 2024-04-25T00:40:01.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.997+0000 7fba654e0640 1 -- 172.21.15.29:0/3034040771 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fba40067ef0 msgr2=0x7fba4006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:01.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.997+0000 7fba654e0640 1 --2- 172.21.15.29:0/3034040771 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fba40067ef0 0x7fba4006a3b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fba480016d0 tx=0x7fba48001190 comp rx=0 tx=0).stop 2024-04-25T00:40:01.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.997+0000 7fba654e0640 1 -- 172.21.15.29:0/3034040771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba600ced70 msgr2=0x7fba600c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:01.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.997+0000 7fba654e0640 1 --2- 172.21.15.29:0/3034040771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba600ced70 0x7fba600c7170 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fba500b0040 tx=0x7fba50019610 comp rx=0 tx=0).stop 2024-04-25T00:40:01.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.997+0000 7fba654e0640 1 -- 172.21.15.29:0/3034040771 shutdown_connections 2024-04-25T00:40:01.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.997+0000 7fba654e0640 1 --2- 172.21.15.29:0/3034040771 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fba40067ef0 0x7fba4006a3b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.997+0000 7fba654e0640 1 --2- 172.21.15.29:0/3034040771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba6015c120 0x7fba6014d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.997+0000 7fba654e0640 1 --2- 172.21.15.29:0/3034040771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba600d28d0 0x7fba6015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.997+0000 7fba654e0640 1 --2- 172.21.15.29:0/3034040771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba600ced70 0x7fba600c7170 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:01.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.997+0000 7fba654e0640 1 -- 172.21.15.29:0/3034040771 >> 172.21.15.29:0/3034040771 conn(0x7fba600d1d20 msgr2=0x7fba6015b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:01.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.997+0000 7fba654e0640 1 -- 172.21.15.29:0/3034040771 shutdown_connections 2024-04-25T00:40:01.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:01.997+0000 7fba654e0640 1 -- 172.21.15.29:0/3034040771 wait complete. 2024-04-25T00:40:02.012 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol54280 2024-04-25T00:40:02.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.133+0000 7f348b60a640 1 Processor -- start 2024-04-25T00:40:02.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f348b60a640 1 -- start start 2024-04-25T00:40:02.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f348b60a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415f3c0 0x7f34840cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:02.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f348b60a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34840b3720 con 0x7f34840b4780 2024-04-25T00:40:02.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f348b60a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34840b39c0 con 0x7f348415e8d0 2024-04-25T00:40:02.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f348b60a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f34840b42f0 con 0x7f348415f3c0 2024-04-25T00:40:02.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f3488ba5640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f34840b4780 0x7f348415c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34634/0 (socket says 172.21.15.29:34634) 2024-04-25T00:40:02.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f3488ba5640 1 -- 172.21.15.29:0/430285827 learned_addr learned my addr 172.21.15.29:0/430285827 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f3489ba7640 1 --2- 172.21.15.29:0/430285827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415f3c0 0x7f34840cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3897767468 0 0) 0x7f34840b3720 con 0x7f34840b4780 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3470005740 con 0x7f34840b4780 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1191896598 0 0) 0x7f3470005740 con 0x7f34840b4780 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415f3c0 msgr2=0x7f34840cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 --2- 172.21.15.29:0/430285827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415f3c0 0x7f34840cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 >> v1:172.21.15.157:6789/0 conn(0x7f348415e8d0 legacy=0x7f348415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34840b3ca0 con 0x7f34840b4780 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f3474001680 con 0x7f34840b4780 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 >> v1:172.21.15.29:6789/0 conn(0x7f34840b4780 legacy=0x7f348415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 --2- 172.21.15.29:0/430285827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415f3c0 0x7f3470005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 --2- 172.21.15.29:0/430285827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3470006180 0x7f3470006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 --2- 172.21.15.29:0/430285827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3470006f60 0x7f34700073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:02.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f34840b42f0 con 0x7f3470006f60 2024-04-25T00:40:02.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f347000dcc0 con 0x7f3470006180 2024-04-25T00:40:02.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f347000dec0 con 0x7f348415f3c0 2024-04-25T00:40:02.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f3489ba7640 1 --2- 172.21.15.29:0/430285827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3470006f60 0x7f34700073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:02.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f34893a6640 1 --2- 172.21.15.29:0/430285827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415f3c0 0x7f3470005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:02.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f3488ba5640 1 --2- 172.21.15.29:0/430285827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3470006180 0x7f3470006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:02.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f3489ba7640 1 -- 172.21.15.29:0/430285827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415f3c0 msgr2=0x7f3470005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:02.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f3489ba7640 1 --2- 172.21.15.29:0/430285827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415f3c0 0x7f3470005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:02.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f3489ba7640 1 -- 172.21.15.29:0/430285827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3470006180 msgr2=0x7f3470006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:02.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f3489ba7640 1 --2- 172.21.15.29:0/430285827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3470006180 0x7f3470006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:02.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.137+0000 7f3489ba7640 1 -- 172.21.15.29:0/430285827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3470005740 con 0x7f3470006f60 2024-04-25T00:40:02.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f3489ba7640 1 --2- 172.21.15.29:0/430285827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3470006f60 0x7f34700073e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f347c009ae0 tx=0x7f347c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cda208d15dd7d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:02.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c0b2070 con 0x7f3470006f60 2024-04-25T00:40:02.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347c0a5660 con 0x7f3470006f60 2024-04-25T00:40:02.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f347bfff640 1 -- 172.21.15.29:0/430285827 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c0096b0 con 0x7f3470006f60 2024-04-25T00:40:02.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 -- 172.21.15.29:0/430285827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3470006f60 msgr2=0x7f34700073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:02.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 --2- 172.21.15.29:0/430285827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3470006f60 0x7f34700073e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f347c009ae0 tx=0x7f347c01ca70 comp rx=0 tx=0).stop 2024-04-25T00:40:02.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f3489ba7640 1 -- 172.21.15.29:0/430285827 reap_dead start 2024-04-25T00:40:02.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 -- 172.21.15.29:0/430285827 shutdown_connections 2024-04-25T00:40:02.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 -- 172.21.15.29:0/430285827 >> 172.21.15.29:0/430285827 conn(0x7f34840b1870 msgr2=0x7f34840b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:02.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 -- 172.21.15.29:0/430285827 shutdown_connections 2024-04-25T00:40:02.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 -- 172.21.15.29:0/430285827 wait complete. 2024-04-25T00:40:02.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 Processor -- start 2024-04-25T00:40:02.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 -- start start 2024-04-25T00:40:02.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3470006f60 0x7f34840b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:02.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415a3d0 0x7f348415a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:02.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3484142e40 0x7f3484143280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:02.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3484144a20 con 0x7f3470006f60 2024-04-25T00:40:02.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f348406cf10 con 0x7f3484142e40 2024-04-25T00:40:02.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f348b60a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3484141530 con 0x7f348415a3d0 2024-04-25T00:40:02.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f34893a6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3470006f60 0x7f34840b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:02.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f34893a6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3470006f60 0x7f34840b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50326/0 (socket says 172.21.15.29:50326) 2024-04-25T00:40:02.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.141+0000 7f34893a6640 1 -- 172.21.15.29:0/886179303 learned_addr learned my addr 172.21.15.29:0/886179303 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:02.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f3488ba5640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415a3d0 0x7f348415a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:02.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f34893a6640 1 -- 172.21.15.29:0/886179303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415a3d0 msgr2=0x7f348415a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:02.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f34893a6640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415a3d0 0x7f348415a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:02.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f3489ba7640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3484142e40 0x7f3484143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:02.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f34893a6640 1 -- 172.21.15.29:0/886179303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3484142e40 msgr2=0x7f3484143280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:02.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f34893a6640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3484142e40 0x7f3484143280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:02.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f34893a6640 1 -- 172.21.15.29:0/886179303 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34841409b0 con 0x7f3470006f60 2024-04-25T00:40:02.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f3488ba5640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415a3d0 0x7f348415a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:02.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f34893a6640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3470006f60 0x7f34840b2bc0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f348000fbc0 tx=0x7f348000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:02.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f3479ffb640 1 -- 172.21.15.29:0/886179303 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348000dc30 con 0x7f3470006f60 2024-04-25T00:40:02.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f3479ffb640 1 -- 172.21.15.29:0/886179303 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3480018070 con 0x7f3470006f60 2024-04-25T00:40:02.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f348b60a640 1 -- 172.21.15.29:0/886179303 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3484140c90 con 0x7f3470006f60 2024-04-25T00:40:02.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f3479ffb640 1 -- 172.21.15.29:0/886179303 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480003980 con 0x7f3470006f60 2024-04-25T00:40:02.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f348b60a640 1 -- 172.21.15.29:0/886179303 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3484141250 con 0x7f3470006f60 2024-04-25T00:40:02.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f348b60a640 1 -- 172.21.15.29:0/886179303 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3484069270 con 0x7f3470006f60 2024-04-25T00:40:02.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f3479ffb640 1 -- 172.21.15.29:0/886179303 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f3480003b20 con 0x7f3470006f60 2024-04-25T00:40:02.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f3479ffb640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3468067f20 0x7f346806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:02.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f3479ffb640 1 -- 172.21.15.29:0/886179303 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f3480086930 con 0x7f3470006f60 2024-04-25T00:40:02.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f3488ba5640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3468067f20 0x7f346806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:02.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.145+0000 7f3488ba5640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3468067f20 0x7f346806a3e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3474001210 tx=0x7f3474001160 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:02.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.149+0000 7f3479ffb640 1 -- 172.21.15.29:0/886179303 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3480056bc0 con 0x7f3470006f60 2024-04-25T00:40:02.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.337+0000 7f348b60a640 1 -- 172.21.15.29:0/886179303 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol54280", "target": ["mon-mgr", ""]}) -- 0x7f348406e7b0 con 0x7f3468067f20 2024-04-25T00:40:02.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.345+0000 7f3479ffb640 1 -- 172.21.15.29:0/886179303 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f348406e7b0 con 0x7f3468067f20 2024-04-25T00:40:02.347 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/_nogroup/subvol54280 2024-04-25T00:40:02.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.349+0000 7f348b60a640 1 -- 172.21.15.29:0/886179303 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3468067f20 msgr2=0x7f346806a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:02.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.349+0000 7f348b60a640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3468067f20 0x7f346806a3e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3474001210 tx=0x7f3474001160 comp rx=0 tx=0).stop 2024-04-25T00:40:02.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.349+0000 7f348b60a640 1 -- 172.21.15.29:0/886179303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3470006f60 msgr2=0x7f34840b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:02.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.349+0000 7f348b60a640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3470006f60 0x7f34840b2bc0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f348000fbc0 tx=0x7f348000d520 comp rx=0 tx=0).stop 2024-04-25T00:40:02.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.349+0000 7f348b60a640 1 -- 172.21.15.29:0/886179303 shutdown_connections 2024-04-25T00:40:02.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.349+0000 7f348b60a640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3468067f20 0x7f346806a3e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:02.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.349+0000 7f348b60a640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3484142e40 0x7f3484143280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:02.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.349+0000 7f348b60a640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f348415a3d0 0x7f348415a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:02.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.349+0000 7f348b60a640 1 --2- 172.21.15.29:0/886179303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3470006f60 0x7f34840b2bc0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:02.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.349+0000 7f348b60a640 1 -- 172.21.15.29:0/886179303 >> 172.21.15.29:0/886179303 conn(0x7f34840b1870 msgr2=0x7f348413e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:02.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.349+0000 7f348b60a640 1 -- 172.21.15.29:0/886179303 shutdown_connections 2024-04-25T00:40:02.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:02.349+0000 7f348b60a640 1 -- 172.21.15.29:0/886179303 wait complete. 2024-04-25T00:40:02.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:02.367 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/76236127effd7750c42ba6050a6e0f7a.meta) 2024-04-25T00:40:04.195 INFO:teuthology.orchestra.run.smithi029.stdout:[GLOBAL] 2024-04-25T00:40:04.196 INFO:teuthology.orchestra.run.smithi029.stdout:version = 1 2024-04-25T00:40:04.196 INFO:teuthology.orchestra.run.smithi029.stdout:type = subvolume 2024-04-25T00:40:04.196 INFO:teuthology.orchestra.run.smithi029.stdout:path = /volumes/_nogroup/subvol54280 2024-04-25T00:40:04.196 INFO:teuthology.orchestra.run.smithi029.stdout:state = complete 2024-04-25T00:40:04.196 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:04.198 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol291 subvol_grp8010 2024-04-25T00:40:04.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a5d510640 1 Processor -- start 2024-04-25T00:40:04.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a5d510640 1 -- start start 2024-04-25T00:40:04.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a5d510640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9a5815e9a0 0x7f9a58070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:04.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a5d510640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9a580cd1c0 0x7f9a58072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:04.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a5d510640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a580712c0 con 0x7f9a580b4590 2024-04-25T00:40:04.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a5d510640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9a58071ed0 con 0x7f9a580cd1c0 2024-04-25T00:40:04.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a5d510640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9a580715a0 con 0x7f9a5815e9a0 2024-04-25T00:40:04.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a567fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9a580b4590 0x7f9a5815ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34650/0 (socket says 172.21.15.29:34650) 2024-04-25T00:40:04.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a567fc640 1 -- 172.21.15.29:0/2704768258 learned_addr learned my addr 172.21.15.29:0/2704768258 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:04.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a577fe640 1 --2- 172.21.15.29:0/2704768258 >> v2:172.21.15.29:3300/0 conn(0x7f9a580cd1c0 0x7f9a58072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:04.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a56ffd640 1 --2- 172.21.15.29:0/2704768258 >> v2:172.21.15.157:3300/0 conn(0x7f9a5815e9a0 0x7f9a58070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:04.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a55ffb640 1 -- 172.21.15.29:0/2704768258 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1372192463 0 0) 0x7f9a580712c0 con 0x7f9a580b4590 2024-04-25T00:40:04.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a55ffb640 1 -- 172.21.15.29:0/2704768258 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a40005740 con 0x7f9a580b4590 2024-04-25T00:40:04.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a577fe640 1 -- 172.21.15.29:0/2704768258 >> v2:172.21.15.157:3300/0 conn(0x7f9a5815e9a0 msgr2=0x7f9a58070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:04.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a577fe640 1 --2- 172.21.15.29:0/2704768258 >> v2:172.21.15.157:3300/0 conn(0x7f9a5815e9a0 0x7f9a58070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:04.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a577fe640 1 -- 172.21.15.29:0/2704768258 >> v1:172.21.15.29:6789/0 conn(0x7f9a580b4590 legacy=0x7f9a5815ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:04.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.341+0000 7f9a577fe640 1 -- 172.21.15.29:0/2704768258 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a58071880 con 0x7f9a580cd1c0 2024-04-25T00:40:04.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a577fe640 1 --2- 172.21.15.29:0/2704768258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a580cd1c0 0x7f9a58072360 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f9a580b5260 tx=0x7f9a5001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764a1f061f4ecb50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:04.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a55ffb640 1 -- 172.21.15.29:0/2704768258 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a500b2070 con 0x7f9a580cd1c0 2024-04-25T00:40:04.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a55ffb640 1 -- 172.21.15.29:0/2704768258 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a500a5950 con 0x7f9a580cd1c0 2024-04-25T00:40:04.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a55ffb640 1 -- 172.21.15.29:0/2704768258 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a500a5d20 con 0x7f9a580cd1c0 2024-04-25T00:40:04.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 -- 172.21.15.29:0/2704768258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a580cd1c0 msgr2=0x7f9a58072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:04.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 --2- 172.21.15.29:0/2704768258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a580cd1c0 0x7f9a58072360 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f9a580b5260 tx=0x7f9a5001c870 comp rx=0 tx=0).stop 2024-04-25T00:40:04.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 -- 172.21.15.29:0/2704768258 shutdown_connections 2024-04-25T00:40:04.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 --2- 172.21.15.29:0/2704768258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a580cd1c0 0x7f9a58072360 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:04.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 --2- 172.21.15.29:0/2704768258 >> v2:172.21.15.157:3300/0 conn(0x7f9a5815e9a0 0x7f9a58070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:04.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 -- 172.21.15.29:0/2704768258 >> 172.21.15.29:0/2704768258 conn(0x7f9a580b1870 msgr2=0x7f9a580b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:04.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 -- 172.21.15.29:0/2704768258 shutdown_connections 2024-04-25T00:40:04.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 -- 172.21.15.29:0/2704768258 wait complete. 2024-04-25T00:40:04.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 Processor -- start 2024-04-25T00:40:04.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 -- start start 2024-04-25T00:40:04.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a580b4590 0x7f9a5815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:04.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a580cd1c0 0x7f9a5813e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:04.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a5815e9a0 0x7f9a58142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:04.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581449f0 con 0x7f9a580cd1c0 2024-04-25T00:40:04.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58144870 con 0x7f9a5815e9a0 2024-04-25T00:40:04.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.345+0000 7f9a5d510640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5806d5d0 con 0x7f9a580b4590 2024-04-25T00:40:04.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a567fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a580cd1c0 0x7f9a5813e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:04.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a567fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a580cd1c0 0x7f9a5813e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50332/0 (socket says 172.21.15.29:50332) 2024-04-25T00:40:04.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a567fc640 1 -- 172.21.15.29:0/3206054654 learned_addr learned my addr 172.21.15.29:0/3206054654 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:04.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a56ffd640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a580b4590 0x7f9a5815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:04.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a577fe640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a5815e9a0 0x7f9a58142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:04.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a567fc640 1 -- 172.21.15.29:0/3206054654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a580b4590 msgr2=0x7f9a5815a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:04.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a567fc640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a580b4590 0x7f9a5815a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:04.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a567fc640 1 -- 172.21.15.29:0/3206054654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a5815e9a0 msgr2=0x7f9a58142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:04.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a567fc640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a5815e9a0 0x7f9a58142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:04.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a567fc640 1 -- 172.21.15.29:0/3206054654 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5813fae0 con 0x7f9a580cd1c0 2024-04-25T00:40:04.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a577fe640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a5815e9a0 0x7f9a58142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:04.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a567fc640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a580cd1c0 0x7f9a5813e880 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c500 tx=0x7f9a4c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:04.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a3f7fe640 1 -- 172.21.15.29:0/3206054654 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c017070 con 0x7f9a580cd1c0 2024-04-25T00:40:04.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a5d510640 1 -- 172.21.15.29:0/3206054654 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5813ef80 con 0x7f9a580cd1c0 2024-04-25T00:40:04.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a3f7fe640 1 -- 172.21.15.29:0/3206054654 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a4c005cf0 con 0x7f9a580cd1c0 2024-04-25T00:40:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a3f7fe640 1 -- 172.21.15.29:0/3206054654 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0072f0 con 0x7f9a580cd1c0 2024-04-25T00:40:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a5d510640 1 -- 172.21.15.29:0/3206054654 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a5815b260 con 0x7f9a580cd1c0 2024-04-25T00:40:04.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a5d510640 1 -- 172.21.15.29:0/3206054654 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a580692e0 con 0x7f9a580cd1c0 2024-04-25T00:40:04.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a3f7fe640 1 -- 172.21.15.29:0/3206054654 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 23) ==== 88057+0+0 (secure 0 0 0) 0x7f9a4c01a070 con 0x7f9a580cd1c0 2024-04-25T00:40:04.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a3f7fe640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a44067f80 0x7f9a4406a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:04.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a3f7fe640 1 -- 172.21.15.29:0/3206054654 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f9a4c085c30 con 0x7f9a580cd1c0 2024-04-25T00:40:04.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a56ffd640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a44067f80 0x7f9a4406a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:04.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.349+0000 7f9a56ffd640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a44067f80 0x7f9a4406a440 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9a5813fcd0 tx=0x7f9a48008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:04.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.353+0000 7f9a3f7fe640 1 -- 172.21.15.29:0/3206054654 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9a4c055710 con 0x7f9a580cd1c0 2024-04-25T00:40:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.545+0000 7f9a5d510640 1 -- 172.21.15.29:0/3206054654 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol291", "group_name": "subvol_grp8010", "target": ["mon-mgr", ""]}) -- 0x7f9a5806cff0 con 0x7f9a44067f80 2024-04-25T00:40:04.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.553+0000 7f9a3f7fe640 1 -- 172.21.15.29:0/3206054654 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+34 (secure 0 0 0) 0x7f9a5806cff0 con 0x7f9a44067f80 2024-04-25T00:40:04.554 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/subvol_grp8010/subvol291 2024-04-25T00:40:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.557+0000 7f9a5d510640 1 -- 172.21.15.29:0/3206054654 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a44067f80 msgr2=0x7f9a4406a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.557+0000 7f9a5d510640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a44067f80 0x7f9a4406a440 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9a5813fcd0 tx=0x7f9a48008040 comp rx=0 tx=0).stop 2024-04-25T00:40:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.557+0000 7f9a5d510640 1 -- 172.21.15.29:0/3206054654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a580cd1c0 msgr2=0x7f9a5813e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.557+0000 7f9a5d510640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a580cd1c0 0x7f9a5813e880 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c500 tx=0x7f9a4c00c9c0 comp rx=0 tx=0).stop 2024-04-25T00:40:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.557+0000 7f9a5d510640 1 -- 172.21.15.29:0/3206054654 shutdown_connections 2024-04-25T00:40:04.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.557+0000 7f9a5d510640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9a44067f80 0x7f9a4406a440 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:04.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.557+0000 7f9a5d510640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a5815e9a0 0x7f9a58142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:04.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.557+0000 7f9a5d510640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a580cd1c0 0x7f9a5813e880 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:04.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.557+0000 7f9a5d510640 1 --2- 172.21.15.29:0/3206054654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a580b4590 0x7f9a5815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:04.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.557+0000 7f9a5d510640 1 -- 172.21.15.29:0/3206054654 >> 172.21.15.29:0/3206054654 conn(0x7f9a580b1870 msgr2=0x7f9a5815e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:04.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.557+0000 7f9a5d510640 1 -- 172.21.15.29:0/3206054654 shutdown_connections 2024-04-25T00:40:04.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:04.557+0000 7f9a5d510640 1 -- 172.21.15.29:0/3206054654 wait complete. 2024-04-25T00:40:04.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:04.574 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/5beac64c18a1c28b40b5ce0e7ddec4a1.meta) 2024-04-25T00:40:09.192 INFO:teuthology.orchestra.run.smithi029.stdout:[GLOBAL] 2024-04-25T00:40:09.192 INFO:teuthology.orchestra.run.smithi029.stdout:version = 1 2024-04-25T00:40:09.192 INFO:teuthology.orchestra.run.smithi029.stdout:type = subvolume 2024-04-25T00:40:09.192 INFO:teuthology.orchestra.run.smithi029.stdout:path = /volumes/subvol_grp8010/subvol291 2024-04-25T00:40:09.193 INFO:teuthology.orchestra.run.smithi029.stdout:state = complete 2024-04-25T00:40:09.193 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:09.195 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol54280 2024-04-25T00:40:09.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.333+0000 7f1414d7d640 1 Processor -- start 2024-04-25T00:40:09.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.333+0000 7f1414d7d640 1 -- start start 2024-04-25T00:40:09.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.333+0000 7f1414d7d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b3140 0x7f141006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:09.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.333+0000 7f1414d7d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f14100b4b10 0x7f14100b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:09.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.333+0000 7f1414d7d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14100b41e0 con 0x7f14100b3a60 2024-04-25T00:40:09.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.333+0000 7f1414d7d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f141006d960 con 0x7f14100b4b10 2024-04-25T00:40:09.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.333+0000 7f1414d7d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f141006dc40 con 0x7f14100b3140 2024-04-25T00:40:09.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.337+0000 7f140dd9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f14100b4b10 0x7f14100b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:09.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.337+0000 7f140dd9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f14100b4b10 0x7f14100b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52022/0 (socket says 172.21.15.29:52022) 2024-04-25T00:40:09.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.337+0000 7f140ed9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f14100b3a60 0x7f141015c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59046/0 (socket says 172.21.15.29:59046) 2024-04-25T00:40:09.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.337+0000 7f140dd9b640 1 -- 172.21.15.29:0/791207131 learned_addr learned my addr 172.21.15.29:0/791207131 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:09.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.337+0000 7f140e59c640 1 --2- 172.21.15.29:0/791207131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b3140 0x7f141006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:09.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.337+0000 7f140dd9b640 1 -- 172.21.15.29:0/791207131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b3140 msgr2=0x7f141006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:09.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.337+0000 7f140dd9b640 1 --2- 172.21.15.29:0/791207131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b3140 0x7f141006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.337+0000 7f140dd9b640 1 -- 172.21.15.29:0/791207131 >> v1:172.21.15.29:6789/0 conn(0x7f14100b3a60 legacy=0x7f141015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:09.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.337+0000 7f140dd9b640 1 -- 172.21.15.29:0/791207131 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141006df20 con 0x7f14100b4b10 2024-04-25T00:40:09.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.337+0000 7f140dd9b640 1 --2- 172.21.15.29:0/791207131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14100b4b10 0x7f14100b3520 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f1404009870 tx=0x7f140401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b002e04a59a9c44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:09.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.337+0000 7f140d59a640 1 -- 172.21.15.29:0/791207131 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14040b0070 con 0x7f14100b4b10 2024-04-25T00:40:09.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.337+0000 7f140d59a640 1 -- 172.21.15.29:0/791207131 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14040ab600 con 0x7f14100b4b10 2024-04-25T00:40:09.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.337+0000 7f140d59a640 1 -- 172.21.15.29:0/791207131 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14040ab9d0 con 0x7f14100b4b10 2024-04-25T00:40:09.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 -- 172.21.15.29:0/791207131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14100b4b10 msgr2=0x7f14100b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:09.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 --2- 172.21.15.29:0/791207131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14100b4b10 0x7f14100b3520 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f1404009870 tx=0x7f140401c560 comp rx=0 tx=0).stop 2024-04-25T00:40:09.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 -- 172.21.15.29:0/791207131 shutdown_connections 2024-04-25T00:40:09.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 --2- 172.21.15.29:0/791207131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14100b4b10 0x7f14100b3520 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 --2- 172.21.15.29:0/791207131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b3140 0x7f141006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 -- 172.21.15.29:0/791207131 >> 172.21.15.29:0/791207131 conn(0x7f14100b1870 msgr2=0x7f14100b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:09.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 -- 172.21.15.29:0/791207131 shutdown_connections 2024-04-25T00:40:09.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 -- 172.21.15.29:0/791207131 wait complete. 2024-04-25T00:40:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 Processor -- start 2024-04-25T00:40:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 -- start start 2024-04-25T00:40:09.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14100b3140 0x7f1410136d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14100b3a60 0x7f1410137240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b4b10 0x7f1410140870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1410168b70 con 0x7f14100b3140 2024-04-25T00:40:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f14101689f0 con 0x7f14100b3a60 2024-04-25T00:40:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1410168cf0 con 0x7f14100b4b10 2024-04-25T00:40:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f140e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14100b3140 0x7f1410136d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f140e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14100b3140 0x7f1410136d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52024/0 (socket says 172.21.15.29:52024) 2024-04-25T00:40:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f140e59c640 1 -- 172.21.15.29:0/734042043 learned_addr learned my addr 172.21.15.29:0/734042043 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f140ed9d640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b4b10 0x7f1410140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f140dd9b640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14100b3a60 0x7f1410137240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f140e59c640 1 -- 172.21.15.29:0/734042043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b4b10 msgr2=0x7f1410140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:09.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f140e59c640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b4b10 0x7f1410140870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f140e59c640 1 -- 172.21.15.29:0/734042043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14100b3a60 msgr2=0x7f1410137240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f140e59c640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14100b3a60 0x7f1410137240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f140e59c640 1 -- 172.21.15.29:0/734042043 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141013b0a0 con 0x7f14100b3140 2024-04-25T00:40:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f140dd9b640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14100b3a60 0x7f1410137240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f140e59c640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14100b3140 0x7f1410136d00 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f140000d5f0 tx=0x7f140000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f13feffd640 1 -- 172.21.15.29:0/734042043 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14000125f0 con 0x7f14100b3140 2024-04-25T00:40:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f13feffd640 1 -- 172.21.15.29:0/734042043 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1400019070 con 0x7f14100b3140 2024-04-25T00:40:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f13feffd640 1 -- 172.21.15.29:0/734042043 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14000129b0 con 0x7f14100b3140 2024-04-25T00:40:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 -- 172.21.15.29:0/734042043 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141013a2e0 con 0x7f14100b3140 2024-04-25T00:40:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.341+0000 7f1414d7d640 1 -- 172.21.15.29:0/734042043 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141013a8a0 con 0x7f14100b3140 2024-04-25T00:40:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.345+0000 7f1414d7d640 1 -- 172.21.15.29:0/734042043 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1410069270 con 0x7f14100b3140 2024-04-25T00:40:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.345+0000 7f13feffd640 1 -- 172.21.15.29:0/734042043 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 23) ==== 88057+0+0 (secure 0 0 0) 0x7f1400012b50 con 0x7f14100b3140 2024-04-25T00:40:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.345+0000 7f13feffd640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13f0067fb0 0x7f13f006a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.345+0000 7f140dd9b640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13f0067fb0 0x7f13f006a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:09.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.345+0000 7f13feffd640 1 -- 172.21.15.29:0/734042043 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f1400086b10 con 0x7f14100b3140 2024-04-25T00:40:09.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.345+0000 7f140dd9b640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13f0067fb0 0x7f13f006a470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f14040ad000 tx=0x7f14040a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:09.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.349+0000 7f13feffd640 1 -- 172.21.15.29:0/734042043 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1400011340 con 0x7f14100b3140 2024-04-25T00:40:09.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.537+0000 7f1414d7d640 1 -- 172.21.15.29:0/734042043 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol54280", "target": ["mon-mgr", ""]}) -- 0x7f141015bac0 con 0x7f13f0067fb0 2024-04-25T00:40:09.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.561+0000 7f13feffd640 1 -- 172.21.15.29:0/734042043 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f14040ab580 con 0x7f13f0067fb0 2024-04-25T00:40:09.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.561+0000 7f1414d7d640 1 -- 172.21.15.29:0/734042043 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13f0067fb0 msgr2=0x7f13f006a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:09.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.561+0000 7f1414d7d640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13f0067fb0 0x7f13f006a470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f14040ad000 tx=0x7f14040a6000 comp rx=0 tx=0).stop 2024-04-25T00:40:09.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.561+0000 7f1414d7d640 1 -- 172.21.15.29:0/734042043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14100b3140 msgr2=0x7f1410136d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:09.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.561+0000 7f1414d7d640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14100b3140 0x7f1410136d00 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f140000d5f0 tx=0x7f140000dab0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.561+0000 7f1414d7d640 1 -- 172.21.15.29:0/734042043 shutdown_connections 2024-04-25T00:40:09.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.561+0000 7f1414d7d640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13f0067fb0 0x7f13f006a470 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.561+0000 7f1414d7d640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b4b10 0x7f1410140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.561+0000 7f1414d7d640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14100b3a60 0x7f1410137240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.561+0000 7f1414d7d640 1 --2- 172.21.15.29:0/734042043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14100b3140 0x7f1410136d00 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.561+0000 7f1414d7d640 1 -- 172.21.15.29:0/734042043 >> 172.21.15.29:0/734042043 conn(0x7f14100b1870 msgr2=0x7f14100ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:09.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.561+0000 7f1414d7d640 1 -- 172.21.15.29:0/734042043 shutdown_connections 2024-04-25T00:40:09.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.561+0000 7f1414d7d640 1 -- 172.21.15.29:0/734042043 wait complete. 2024-04-25T00:40:09.565 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:09.565+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:09.565 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:09.565+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:09.565 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:09.565+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:09.565 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:09.565+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:09.565 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:09.565+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:09.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol291 subvol_grp8010 2024-04-25T00:40:09.585 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:09.585+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:09.585 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:09.585+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:09.585 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:09.585+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:09.585 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:09.585+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:09.585 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:09.585+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:09.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f3155deb640 1 Processor -- start 2024-04-25T00:40:09.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f3155deb640 1 -- start start 2024-04-25T00:40:09.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f3155deb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3150156300 0x7f3150141400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:09.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f3155deb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31500cf250 0x7f315015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:09.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f3155deb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f315006b210 0x7f315006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:09.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f3155deb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f315006bba0 con 0x7f315006b210 2024-04-25T00:40:09.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f3155deb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f315015d2b0 con 0x7f31500cf250 2024-04-25T00:40:09.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f3155deb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f31500cf8c0 con 0x7f3150156300 2024-04-25T00:40:09.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f314ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f315006b210 0x7f315006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:09.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f314ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f315006b210 0x7f315006b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52060/0 (socket says 172.21.15.29:52060) 2024-04-25T00:40:09.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f314ffff640 1 -- 172.21.15.29:0/2218153672 learned_addr learned my addr 172.21.15.29:0/2218153672 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:09.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f314f7fe640 1 --2- 172.21.15.29:0/2218153672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3150156300 0x7f3150141400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:09.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f314effd640 1 --2- 172.21.15.29:0/2218153672 >> v2:172.21.15.157:3300/0 conn(0x7f31500cf250 0x7f315015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:09.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f314ffff640 1 -- 172.21.15.29:0/2218153672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3150156300 msgr2=0x7f3150141400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:09.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f314ffff640 1 --2- 172.21.15.29:0/2218153672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3150156300 0x7f3150141400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f314ffff640 1 -- 172.21.15.29:0/2218153672 >> v2:172.21.15.157:3300/0 conn(0x7f31500cf250 msgr2=0x7f315015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:09.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f314ffff640 1 --2- 172.21.15.29:0/2218153672 >> v2:172.21.15.157:3300/0 conn(0x7f31500cf250 0x7f315015cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.705+0000 7f314ffff640 1 -- 172.21.15.29:0/2218153672 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31500cfba0 con 0x7f315006b210 2024-04-25T00:40:09.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f314ffff640 1 --2- 172.21.15.29:0/2218153672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f315006b210 0x7f315006b660 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f3150073290 tx=0x7f314801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde5072fd6e517f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:09.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f314e7fc640 1 -- 172.21.15.29:0/2218153672 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31480a5760 con 0x7f315006b210 2024-04-25T00:40:09.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f314e7fc640 1 -- 172.21.15.29:0/2218153672 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31480a5900 con 0x7f315006b210 2024-04-25T00:40:09.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f314e7fc640 1 -- 172.21.15.29:0/2218153672 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31480a5cd0 con 0x7f315006b210 2024-04-25T00:40:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f3155deb640 1 -- 172.21.15.29:0/2218153672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f315006b210 msgr2=0x7f315006b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f3155deb640 1 --2- 172.21.15.29:0/2218153672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f315006b210 0x7f315006b660 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f3150073290 tx=0x7f314801c9f0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f3155deb640 1 -- 172.21.15.29:0/2218153672 shutdown_connections 2024-04-25T00:40:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f3155deb640 1 --2- 172.21.15.29:0/2218153672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f315006b210 0x7f315006b660 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f3155deb640 1 --2- 172.21.15.29:0/2218153672 >> v2:172.21.15.157:3300/0 conn(0x7f31500cf250 0x7f315015cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f3155deb640 1 --2- 172.21.15.29:0/2218153672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3150156300 0x7f3150141400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f3155deb640 1 -- 172.21.15.29:0/2218153672 >> 172.21.15.29:0/2218153672 conn(0x7f31500b2880 msgr2=0x7f31500b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f3155deb640 1 -- 172.21.15.29:0/2218153672 shutdown_connections 2024-04-25T00:40:09.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f3155deb640 1 -- 172.21.15.29:0/2218153672 wait complete. 2024-04-25T00:40:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f3155deb640 1 Processor -- start 2024-04-25T00:40:09.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.709+0000 7f3155deb640 1 -- start start 2024-04-25T00:40:09.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3155deb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f315006b210 0x7f31501551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:09.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3155deb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31500cf250 0x7f315013e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:09.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3155deb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3150156300 0x7f315013ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:09.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3155deb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3150156860 con 0x7f3150156300 2024-04-25T00:40:09.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3155deb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f31501566e0 con 0x7f315006b210 2024-04-25T00:40:09.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3155deb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3150072600 con 0x7f31500cf250 2024-04-25T00:40:09.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f314ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3150156300 0x7f315013ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:09.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f314ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3150156300 0x7f315013ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52076/0 (socket says 172.21.15.29:52076) 2024-04-25T00:40:09.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f314ffff640 1 -- 172.21.15.29:0/3646421899 learned_addr learned my addr 172.21.15.29:0/3646421899 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:09.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f314f7fe640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f315006b210 0x7f31501551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:09.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f314effd640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31500cf250 0x7f315013e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:09.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f314ffff640 1 -- 172.21.15.29:0/3646421899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31500cf250 msgr2=0x7f315013e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:09.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f314ffff640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31500cf250 0x7f315013e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f314ffff640 1 -- 172.21.15.29:0/3646421899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f315006b210 msgr2=0x7f31501551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:09.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f314ffff640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f315006b210 0x7f31501551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f314ffff640 1 -- 172.21.15.29:0/3646421899 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3150159c60 con 0x7f3150156300 2024-04-25T00:40:09.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f314ffff640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3150156300 0x7f315013ee30 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f31480af040 tx=0x7f3148019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:09.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3137fff640 1 -- 172.21.15.29:0/3646421899 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31480bc020 con 0x7f3150156300 2024-04-25T00:40:09.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3137fff640 1 -- 172.21.15.29:0/3646421899 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31480b8690 con 0x7f3150156300 2024-04-25T00:40:09.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3155deb640 1 -- 172.21.15.29:0/3646421899 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315006e1d0 con 0x7f3150156300 2024-04-25T00:40:09.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3137fff640 1 -- 172.21.15.29:0/3646421899 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31480b8a80 con 0x7f3150156300 2024-04-25T00:40:09.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3155deb640 1 -- 172.21.15.29:0/3646421899 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31501596a0 con 0x7f3150156300 2024-04-25T00:40:09.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3155deb640 1 -- 172.21.15.29:0/3646421899 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31500692e0 con 0x7f3150156300 2024-04-25T00:40:09.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3137fff640 1 -- 172.21.15.29:0/3646421899 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 23) ==== 88057+0+0 (secure 0 0 0) 0x7f31480bf020 con 0x7f3150156300 2024-04-25T00:40:09.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3137fff640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f313c067f80 0x7f313c06a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:09.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f3137fff640 1 -- 172.21.15.29:0/3646421899 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f3148121670 con 0x7f3150156300 2024-04-25T00:40:09.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.713+0000 7f314f7fe640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f313c067f80 0x7f313c06a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:09.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.717+0000 7f314f7fe640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f313c067f80 0x7f313c06a440 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f315013ba40 tx=0x7f3144008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:09.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.721+0000 7f3137fff640 1 -- 172.21.15.29:0/3646421899 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f31480c6030 con 0x7f3150156300 2024-04-25T00:40:09.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.905+0000 7f3155deb640 1 -- 172.21.15.29:0/3646421899 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol291", "group_name": "subvol_grp8010", "target": ["mon-mgr", ""]}) -- 0x7f315013b790 con 0x7f313c067f80 2024-04-25T00:40:09.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.921+0000 7f3137fff640 1 -- 172.21.15.29:0/3646421899 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f315013b790 con 0x7f313c067f80 2024-04-25T00:40:09.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.925+0000 7f3155deb640 1 -- 172.21.15.29:0/3646421899 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f313c067f80 msgr2=0x7f313c06a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:09.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.925+0000 7f3155deb640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f313c067f80 0x7f313c06a440 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f315013ba40 tx=0x7f3144008040 comp rx=0 tx=0).stop 2024-04-25T00:40:09.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.925+0000 7f3155deb640 1 -- 172.21.15.29:0/3646421899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3150156300 msgr2=0x7f315013ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:09.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.925+0000 7f3155deb640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3150156300 0x7f315013ee30 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f31480af040 tx=0x7f3148019790 comp rx=0 tx=0).stop 2024-04-25T00:40:09.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.925+0000 7f3155deb640 1 -- 172.21.15.29:0/3646421899 shutdown_connections 2024-04-25T00:40:09.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.925+0000 7f3155deb640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f313c067f80 0x7f313c06a440 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.925+0000 7f3155deb640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3150156300 0x7f315013ee30 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.925+0000 7f3155deb640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31500cf250 0x7f315013e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.925+0000 7f3155deb640 1 --2- 172.21.15.29:0/3646421899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f315006b210 0x7f31501551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:09.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.925+0000 7f3155deb640 1 -- 172.21.15.29:0/3646421899 >> 172.21.15.29:0/3646421899 conn(0x7f31500b2880 msgr2=0x7f31501404f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:09.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.925+0000 7f3155deb640 1 -- 172.21.15.29:0/3646421899 shutdown_connections 2024-04-25T00:40:09.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:09.925+0000 7f3155deb640 1 -- 172.21.15.29:0/3646421899 wait complete. 2024-04-25T00:40:09.942 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-25T00:40:10.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:40:10.027 INFO:teuthology.orchestra.run.smithi029.stdout:2 2024-04-25T00:40:10.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:40:10.033 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp8010 2024-04-25T00:40:10.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b44327640 1 Processor -- start 2024-04-25T00:40:10.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b44327640 1 -- start start 2024-04-25T00:40:10.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b44327640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b3c15df70 0x7f8b3c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b44327640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c06d210 0x7f8b3c0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b44327640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b3c06d980 con 0x7f8b3c0b4b60 2024-04-25T00:40:10.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b44327640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8b3c0b4fa0 con 0x7f8b3c15df70 2024-04-25T00:40:10.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b44327640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3c06dc60 con 0x7f8b3c06d210 2024-04-25T00:40:10.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b420c3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b3c15df70 0x7f8b3c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b420c3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b3c15df70 0x7f8b3c0b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52086/0 (socket says 172.21.15.29:52086) 2024-04-25T00:40:10.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b420c3640 1 -- 172.21.15.29:0/2829989466 learned_addr learned my addr 172.21.15.29:0/2829989466 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:10.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b418c2640 1 --2- 172.21.15.29:0/2829989466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c06d210 0x7f8b3c0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b420c3640 1 -- 172.21.15.29:0/2829989466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c06d210 msgr2=0x7f8b3c0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:10.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b420c3640 1 --2- 172.21.15.29:0/2829989466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c06d210 0x7f8b3c0b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b420c3640 1 -- 172.21.15.29:0/2829989466 >> v1:172.21.15.157:6789/0 conn(0x7f8b3c0b4b60 legacy=0x7f8b3c070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.201+0000 7f8b420c3640 1 -- 172.21.15.29:0/2829989466 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c06df40 con 0x7f8b3c15df70 2024-04-25T00:40:10.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b420c3640 1 --2- 172.21.15.29:0/2829989466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15df70 0x7f8b3c0b3e40 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009870 tx=0x7f8b3801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0b6a539eb8844f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:10.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b410c1640 1 -- 172.21.15.29:0/2829989466 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380b0070 con 0x7f8b3c15df70 2024-04-25T00:40:10.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b410c1640 1 -- 172.21.15.29:0/2829989466 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b380ab600 con 0x7f8b3c15df70 2024-04-25T00:40:10.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b410c1640 1 -- 172.21.15.29:0/2829989466 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380ab9d0 con 0x7f8b3c15df70 2024-04-25T00:40:10.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 -- 172.21.15.29:0/2829989466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15df70 msgr2=0x7f8b3c0b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 --2- 172.21.15.29:0/2829989466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15df70 0x7f8b3c0b3e40 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009870 tx=0x7f8b3801c560 comp rx=0 tx=0).stop 2024-04-25T00:40:10.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 -- 172.21.15.29:0/2829989466 shutdown_connections 2024-04-25T00:40:10.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 --2- 172.21.15.29:0/2829989466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c06d210 0x7f8b3c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 --2- 172.21.15.29:0/2829989466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15df70 0x7f8b3c0b3e40 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 -- 172.21.15.29:0/2829989466 >> 172.21.15.29:0/2829989466 conn(0x7f8b3c0b1870 msgr2=0x7f8b3c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:10.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 -- 172.21.15.29:0/2829989466 shutdown_connections 2024-04-25T00:40:10.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 -- 172.21.15.29:0/2829989466 wait complete. 2024-04-25T00:40:10.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 Processor -- start 2024-04-25T00:40:10.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 -- start start 2024-04-25T00:40:10.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c06d210 0x7f8b3c1599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4b60 0x7f8b3c0b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c15df70 0x7f8b3c15ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c06ea20 con 0x7f8b3c06d210 2024-04-25T00:40:10.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c06d5f0 con 0x7f8b3c15df70 2024-04-25T00:40:10.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b44327640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3c06eba0 con 0x7f8b3c0b4b60 2024-04-25T00:40:10.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b420c3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c06d210 0x7f8b3c1599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b420c3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c06d210 0x7f8b3c1599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52102/0 (socket says 172.21.15.29:52102) 2024-04-25T00:40:10.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b420c3640 1 -- 172.21.15.29:0/2144614749 learned_addr learned my addr 172.21.15.29:0/2144614749 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:10.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b428c4640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c15df70 0x7f8b3c15ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b418c2640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4b60 0x7f8b3c0b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b420c3640 1 -- 172.21.15.29:0/2144614749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4b60 msgr2=0x7f8b3c0b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b420c3640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4b60 0x7f8b3c0b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b420c3640 1 -- 172.21.15.29:0/2144614749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c15df70 msgr2=0x7f8b3c15ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b420c3640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c15df70 0x7f8b3c15ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b420c3640 1 -- 172.21.15.29:0/2144614749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c1569d0 con 0x7f8b3c06d210 2024-04-25T00:40:10.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.205+0000 7f8b428c4640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c15df70 0x7f8b3c15ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:10.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.209+0000 7f8b420c3640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c06d210 0x7f8b3c1599a0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f8b380ae040 tx=0x7f8b3801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:10.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.209+0000 7f8b32ffd640 1 -- 172.21.15.29:0/2144614749 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380ba020 con 0x7f8b3c06d210 2024-04-25T00:40:10.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.209+0000 7f8b32ffd640 1 -- 172.21.15.29:0/2144614749 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b380b6420 con 0x7f8b3c06d210 2024-04-25T00:40:10.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.209+0000 7f8b44327640 1 -- 172.21.15.29:0/2144614749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3c156cb0 con 0x7f8b3c06d210 2024-04-25T00:40:10.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.209+0000 7f8b32ffd640 1 -- 172.21.15.29:0/2144614749 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380b0040 con 0x7f8b3c06d210 2024-04-25T00:40:10.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.209+0000 7f8b44327640 1 -- 172.21.15.29:0/2144614749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b3c157270 con 0x7f8b3c06d210 2024-04-25T00:40:10.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.209+0000 7f8b44327640 1 -- 172.21.15.29:0/2144614749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b3c069270 con 0x7f8b3c06d210 2024-04-25T00:40:10.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.209+0000 7f8b32ffd640 1 -- 172.21.15.29:0/2144614749 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 23) ==== 88057+0+0 (secure 0 0 0) 0x7f8b380bd020 con 0x7f8b3c06d210 2024-04-25T00:40:10.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.209+0000 7f8b32ffd640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b20067f80 0x7f8b2006a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.209+0000 7f8b32ffd640 1 -- 172.21.15.29:0/2144614749 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f8b38120ad0 con 0x7f8b3c06d210 2024-04-25T00:40:10.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.209+0000 7f8b418c2640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b20067f80 0x7f8b2006a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.209+0000 7f8b418c2640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b20067f80 0x7f8b2006a440 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c15bcc0 tx=0x7f8b2c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:10.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.213+0000 7f8b32ffd640 1 -- 172.21.15.29:0/2144614749 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8b380c4030 con 0x7f8b3c06d210 2024-04-25T00:40:10.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.389+0000 7f8b44327640 1 -- 172.21.15.29:0/2144614749 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp8010", "target": ["mon-mgr", ""]}) -- 0x7f8b3c1574b0 con 0x7f8b20067f80 2024-04-25T00:40:10.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.401+0000 7f8b32ffd640 1 -- 172.21.15.29:0/2144614749 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8b2c009820 con 0x7f8b20067f80 2024-04-25T00:40:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.401+0000 7f8b44327640 1 -- 172.21.15.29:0/2144614749 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b20067f80 msgr2=0x7f8b2006a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.401+0000 7f8b44327640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b20067f80 0x7f8b2006a440 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c15bcc0 tx=0x7f8b2c008040 comp rx=0 tx=0).stop 2024-04-25T00:40:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.401+0000 7f8b44327640 1 -- 172.21.15.29:0/2144614749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c06d210 msgr2=0x7f8b3c1599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.401+0000 7f8b44327640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c06d210 0x7f8b3c1599a0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f8b380ae040 tx=0x7f8b3801ada0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.401+0000 7f8b44327640 1 -- 172.21.15.29:0/2144614749 shutdown_connections 2024-04-25T00:40:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.401+0000 7f8b44327640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b20067f80 0x7f8b2006a440 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.401+0000 7f8b44327640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c15df70 0x7f8b3c15ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.401+0000 7f8b44327640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4b60 0x7f8b3c0b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.401+0000 7f8b44327640 1 --2- 172.21.15.29:0/2144614749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c06d210 0x7f8b3c1599a0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.401+0000 7f8b44327640 1 -- 172.21.15.29:0/2144614749 >> 172.21.15.29:0/2144614749 conn(0x7f8b3c0b1870 msgr2=0x7f8b3c15be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:10.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.401+0000 7f8b44327640 1 -- 172.21.15.29:0/2144614749 shutdown_connections 2024-04-25T00:40:10.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.401+0000 7f8b44327640 1 -- 172.21.15.29:0/2144614749 wait complete. 2024-04-25T00:40:10.417 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:40:10.417 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:40:10.452 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:40:10.492 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:40:10.492 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:40:10.492 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:40:10.492 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:40:10.500 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:40:10.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84f03e8640 1 Processor -- start 2024-04-25T00:40:10.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84f03e8640 1 -- start start 2024-04-25T00:40:10.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84f03e8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f84e806b210 0x7f84e8072990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84f03e8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84e815ecf0 0x7f84e80ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84f03e8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e815f2a0 con 0x7f84e806b5f0 2024-04-25T00:40:10.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84f03e8640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f84e80b03f0 con 0x7f84e806b210 2024-04-25T00:40:10.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84f03e8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f84e815f580 con 0x7f84e815ecf0 2024-04-25T00:40:10.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84ee184640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f84e806b210 0x7f84e8072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84ee184640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f84e806b210 0x7f84e8072990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52110/0 (socket says 172.21.15.29:52110) 2024-04-25T00:40:10.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84ee184640 1 -- 172.21.15.29:0/2694742374 learned_addr learned my addr 172.21.15.29:0/2694742374 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:10.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84ee985640 1 --2- 172.21.15.29:0/2694742374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84e815ecf0 0x7f84e80ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84ee184640 1 -- 172.21.15.29:0/2694742374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84e815ecf0 msgr2=0x7f84e80ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:10.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84ee184640 1 --2- 172.21.15.29:0/2694742374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84e815ecf0 0x7f84e80ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84ee184640 1 -- 172.21.15.29:0/2694742374 >> v1:172.21.15.157:6789/0 conn(0x7f84e806b5f0 legacy=0x7f84e815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.673+0000 7f84ee184640 1 -- 172.21.15.29:0/2694742374 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e8071eb0 con 0x7f84e806b210 2024-04-25T00:40:10.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.677+0000 7f84ee184640 1 --2- 172.21.15.29:0/2694742374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84e806b210 0x7f84e8072990 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009870 tx=0x7f84e401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b963853683f2309b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:10.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.677+0000 7f84ed182640 1 -- 172.21.15.29:0/2694742374 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e40b0070 con 0x7f84e806b210 2024-04-25T00:40:10.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.677+0000 7f84ed182640 1 -- 172.21.15.29:0/2694742374 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e4004230 con 0x7f84e806b210 2024-04-25T00:40:10.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.677+0000 7f84ed182640 1 -- 172.21.15.29:0/2694742374 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4003620 con 0x7f84e806b210 2024-04-25T00:40:10.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.677+0000 7f84f03e8640 1 -- 172.21.15.29:0/2694742374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84e806b210 msgr2=0x7f84e8072990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.677+0000 7f84f03e8640 1 --2- 172.21.15.29:0/2694742374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84e806b210 0x7f84e8072990 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009870 tx=0x7f84e401c440 comp rx=0 tx=0).stop 2024-04-25T00:40:10.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.677+0000 7f84f03e8640 1 -- 172.21.15.29:0/2694742374 shutdown_connections 2024-04-25T00:40:10.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.677+0000 7f84f03e8640 1 --2- 172.21.15.29:0/2694742374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84e815ecf0 0x7f84e80ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.677+0000 7f84f03e8640 1 --2- 172.21.15.29:0/2694742374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84e806b210 0x7f84e8072990 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.677+0000 7f84f03e8640 1 -- 172.21.15.29:0/2694742374 >> 172.21.15.29:0/2694742374 conn(0x7f84e80d0c40 msgr2=0x7f84e80d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:10.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.677+0000 7f84f03e8640 1 -- 172.21.15.29:0/2694742374 shutdown_connections 2024-04-25T00:40:10.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.677+0000 7f84f03e8640 1 -- 172.21.15.29:0/2694742374 wait complete. 2024-04-25T00:40:10.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84f03e8640 1 Processor -- start 2024-04-25T00:40:10.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84f03e8640 1 -- start start 2024-04-25T00:40:10.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84f03e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84e806b210 0x7f84e8157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84f03e8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84e806b5f0 0x7f84e80c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84f03e8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84e815ecf0 0x7f84e80c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84f03e8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f84e816b0f0 con 0x7f84e815ecf0 2024-04-25T00:40:10.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84f03e8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f84e816af70 con 0x7f84e806b210 2024-04-25T00:40:10.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84f03e8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f84e816b270 con 0x7f84e806b5f0 2024-04-25T00:40:10.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84ee985640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84e815ecf0 0x7f84e80c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84ee985640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84e815ecf0 0x7f84e80c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52120/0 (socket says 172.21.15.29:52120) 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84ee985640 1 -- 172.21.15.29:0/917448373 learned_addr learned my addr 172.21.15.29:0/917448373 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84ed983640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84e806b5f0 0x7f84e80c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84ee184640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84e806b210 0x7f84e8157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84ee985640 1 -- 172.21.15.29:0/917448373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84e806b5f0 msgr2=0x7f84e80c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84ee985640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84e806b5f0 0x7f84e80c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84ee985640 1 -- 172.21.15.29:0/917448373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84e806b210 msgr2=0x7f84e8157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84ee985640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84e806b210 0x7f84e8157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84ee985640 1 -- 172.21.15.29:0/917448373 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e806ece0 con 0x7f84e815ecf0 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84ee985640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84e815ecf0 0x7f84e80c8250 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f84e0007e70 tx=0x7f84e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84deffd640 1 -- 172.21.15.29:0/917448373 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0014620 con 0x7f84e815ecf0 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84deffd640 1 -- 172.21.15.29:0/917448373 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e0019070 con 0x7f84e815ecf0 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84deffd640 1 -- 172.21.15.29:0/917448373 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0014a20 con 0x7f84e815ecf0 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84f03e8640 1 -- 172.21.15.29:0/917448373 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e806efc0 con 0x7f84e815ecf0 2024-04-25T00:40:10.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84f03e8640 1 -- 172.21.15.29:0/917448373 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e806f580 con 0x7f84e815ecf0 2024-04-25T00:40:10.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84f03e8640 1 -- 172.21.15.29:0/917448373 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e8069270 con 0x7f84e815ecf0 2024-04-25T00:40:10.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.681+0000 7f84deffd640 1 -- 172.21.15.29:0/917448373 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 23) ==== 88057+0+0 (secure 0 0 0) 0x7f84e0014bc0 con 0x7f84e815ecf0 2024-04-25T00:40:10.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.685+0000 7f84deffd640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f84cc067f80 0x7f84cc06a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.685+0000 7f84deffd640 1 -- 172.21.15.29:0/917448373 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f84e0087050 con 0x7f84e815ecf0 2024-04-25T00:40:10.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.685+0000 7f84ee184640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f84cc067f80 0x7f84cc06a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.685+0000 7f84ee184640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f84cc067f80 0x7f84cc06a440 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84e40ad000 tx=0x7f84e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:10.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.689+0000 7f84deffd640 1 -- 172.21.15.29:0/917448373 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f84e0057300 con 0x7f84e815ecf0 2024-04-25T00:40:10.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.845+0000 7f84f03e8640 1 -- 172.21.15.29:0/917448373 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84e80d1a70 con 0x7f84e815ecf0 2024-04-25T00:40:10.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.845+0000 7f84deffd640 1 -- 172.21.15.29:0/917448373 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16623 (secure 0 0 0) 0x7f84e0056c00 con 0x7f84e815ecf0 2024-04-25T00:40:10.845 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:10.845 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":145,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:50.179122+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:46.824888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:47.550815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:40:10.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.849+0000 7f84f03e8640 1 -- 172.21.15.29:0/917448373 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f84cc067f80 msgr2=0x7f84cc06a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.849+0000 7f84f03e8640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f84cc067f80 0x7f84cc06a440 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84e40ad000 tx=0x7f84e4008040 comp rx=0 tx=0).stop 2024-04-25T00:40:10.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.849+0000 7f84f03e8640 1 -- 172.21.15.29:0/917448373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84e815ecf0 msgr2=0x7f84e80c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.849+0000 7f84f03e8640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84e815ecf0 0x7f84e80c8250 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f84e0007e70 tx=0x7f84e000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.849+0000 7f84f03e8640 1 -- 172.21.15.29:0/917448373 shutdown_connections 2024-04-25T00:40:10.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.849+0000 7f84f03e8640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f84cc067f80 0x7f84cc06a440 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.849+0000 7f84f03e8640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84e815ecf0 0x7f84e80c8250 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.849+0000 7f84f03e8640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84e806b5f0 0x7f84e80c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.849+0000 7f84f03e8640 1 --2- 172.21.15.29:0/917448373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84e806b210 0x7f84e8157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.849+0000 7f84f03e8640 1 -- 172.21.15.29:0/917448373 >> 172.21.15.29:0/917448373 conn(0x7f84e80d0c40 msgr2=0x7f84e815c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:10.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.849+0000 7f84f03e8640 1 -- 172.21.15.29:0/917448373 shutdown_connections 2024-04-25T00:40:10.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.849+0000 7f84f03e8640 1 -- 172.21.15.29:0/917448373 wait complete. 2024-04-25T00:40:10.864 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:39:46.824888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:39:47.550815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:40:10.865 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:40:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.977+0000 7f93b1195640 1 Processor -- start 2024-04-25T00:40:10.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.977+0000 7f93b1195640 1 -- start start 2024-04-25T00:40:10.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93b1195640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93ac15e200 0x7f93ac0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93b1195640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93ac0d2c60 0x7f93ac0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93b1195640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93ac0cfc10 con 0x7f93ac0cf690 2024-04-25T00:40:10.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93b1195640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f93ac0b38d0 con 0x7f93ac15e200 2024-04-25T00:40:10.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93b1195640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f93ac0cfef0 con 0x7f93ac0d2c60 2024-04-25T00:40:10.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93aa59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f93ac0cf690 0x7f93ac0d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59060/0 (socket says 172.21.15.29:59060) 2024-04-25T00:40:10.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93aa59c640 1 -- 172.21.15.29:0/3449964261 learned_addr learned my addr 172.21.15.29:0/3449964261 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93ab59e640 1 --2- 172.21.15.29:0/3449964261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93ac0d2c60 0x7f93ac0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93aad9d640 1 --2- 172.21.15.29:0/3449964261 >> v2:172.21.15.157:3300/0 conn(0x7f93ac15e200 0x7f93ac0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 903419904 0 0) 0x7f93ac0cfc10 con 0x7f93ac0cf690 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9394003530 con 0x7f93ac0cf690 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1185681044 0 0) 0x7f9394003530 con 0x7f93ac0cf690 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93ac0d2c60 msgr2=0x7f93ac0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 --2- 172.21.15.29:0/3449964261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93ac0d2c60 0x7f93ac0b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 >> v2:172.21.15.157:3300/0 conn(0x7f93ac15e200 msgr2=0x7f93ac0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 --2- 172.21.15.29:0/3449964261 >> v2:172.21.15.157:3300/0 conn(0x7f93ac15e200 0x7f93ac0d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac0b3b20 con 0x7f93ac0cf690 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f93a00016a0 con 0x7f93ac0cf690 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 >> v1:172.21.15.29:6789/0 conn(0x7f93ac0cf690 legacy=0x7f93ac0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 --2- 172.21.15.29:0/3449964261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93940039c0 0x7f9394003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 --2- 172.21.15.29:0/3449964261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93ac0d2c60 0x7f9394004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 --2- 172.21.15.29:0/3449964261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9394004d50 0x7f93940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac0b38d0 con 0x7f93940039c0 2024-04-25T00:40:10.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac0cfef0 con 0x7f9394004d50 2024-04-25T00:40:10.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f939400b9d0 con 0x7f93ac0d2c60 2024-04-25T00:40:10.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93aad9d640 1 --2- 172.21.15.29:0/3449964261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93940039c0 0x7f9394003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93aa59c640 1 --2- 172.21.15.29:0/3449964261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93ac0d2c60 0x7f9394004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93aad9d640 1 -- 172.21.15.29:0/3449964261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93ac0d2c60 msgr2=0x7f9394004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:10.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93aad9d640 1 --2- 172.21.15.29:0/3449964261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93ac0d2c60 0x7f9394004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93aad9d640 1 -- 172.21.15.29:0/3449964261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9394004d50 msgr2=0x7f93940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:10.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93aad9d640 1 --2- 172.21.15.29:0/3449964261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9394004d50 0x7f93940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93aad9d640 1 -- 172.21.15.29:0/3449964261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9394003530 con 0x7f93940039c0 2024-04-25T00:40:10.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.981+0000 7f93aa59c640 1 --2- 172.21.15.29:0/3449964261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93ac0d2c60 0x7f9394004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93aad9d640 1 --2- 172.21.15.29:0/3449964261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93940039c0 0x7f9394003de0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f939c009b10 tx=0x7f939c01ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282a73d7d58d6f4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c0b2070 con 0x7f93940039c0 2024-04-25T00:40:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939c0a5670 con 0x7f93940039c0 2024-04-25T00:40:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93a9d9b640 1 -- 172.21.15.29:0/3449964261 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c0096b0 con 0x7f93940039c0 2024-04-25T00:40:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 -- 172.21.15.29:0/3449964261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93940039c0 msgr2=0x7f9394003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 --2- 172.21.15.29:0/3449964261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93940039c0 0x7f9394003de0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f939c009b10 tx=0x7f939c01ca80 comp rx=0 tx=0).stop 2024-04-25T00:40:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93ab59e640 1 -- 172.21.15.29:0/3449964261 reap_dead start 2024-04-25T00:40:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 -- 172.21.15.29:0/3449964261 shutdown_connections 2024-04-25T00:40:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 -- 172.21.15.29:0/3449964261 >> 172.21.15.29:0/3449964261 conn(0x7f93ac09d280 msgr2=0x7f93ac0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 -- 172.21.15.29:0/3449964261 shutdown_connections 2024-04-25T00:40:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 -- 172.21.15.29:0/3449964261 wait complete. 2024-04-25T00:40:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 Processor -- start 2024-04-25T00:40:10.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 -- start start 2024-04-25T00:40:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93940039c0 0x7f93ac0cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93ac155170 0x7f93ac06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93ac1556c0 0x7f93ac06c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac0b4cb0 con 0x7f93ac1556c0 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac0b4b30 con 0x7f93ac155170 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.985+0000 7f93b1195640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f93ac0b4e30 con 0x7f93940039c0 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93ab59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93ac1556c0 0x7f93ac06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93ab59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93ac1556c0 0x7f93ac06c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52128/0 (socket says 172.21.15.29:52128) 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93ab59e640 1 -- 172.21.15.29:0/1514013459 learned_addr learned my addr 172.21.15.29:0/1514013459 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93aad9d640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93940039c0 0x7f93ac0cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93aa59c640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93ac155170 0x7f93ac06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93ab59e640 1 -- 172.21.15.29:0/1514013459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93940039c0 msgr2=0x7f93ac0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93ab59e640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93940039c0 0x7f93ac0cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93ab59e640 1 -- 172.21.15.29:0/1514013459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93ac155170 msgr2=0x7f93ac06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93ab59e640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93ac155170 0x7f93ac06bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93ab59e640 1 -- 172.21.15.29:0/1514013459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac0c7df0 con 0x7f93ac1556c0 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93ab59e640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93ac1556c0 0x7f93ac06c070 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f93ac08d3b0 tx=0x7f93a400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:10.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93937fe640 1 -- 172.21.15.29:0/1514013459 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4018020 con 0x7f93ac1556c0 2024-04-25T00:40:10.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93937fe640 1 -- 172.21.15.29:0/1514013459 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93a400ba80 con 0x7f93ac1556c0 2024-04-25T00:40:10.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93b1195640 1 -- 172.21.15.29:0/1514013459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ac0c80d0 con 0x7f93ac1556c0 2024-04-25T00:40:10.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93937fe640 1 -- 172.21.15.29:0/1514013459 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4014400 con 0x7f93ac1556c0 2024-04-25T00:40:10.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93b1195640 1 -- 172.21.15.29:0/1514013459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ac0c8690 con 0x7f93ac1556c0 2024-04-25T00:40:10.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93b1195640 1 -- 172.21.15.29:0/1514013459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ac069270 con 0x7f93ac1556c0 2024-04-25T00:40:10.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93937fe640 1 -- 172.21.15.29:0/1514013459 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 23) ==== 88057+0+0 (secure 0 0 0) 0x7f93a401b020 con 0x7f93ac1556c0 2024-04-25T00:40:10.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93937fe640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9398067fb0 0x7f939806a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:10.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93937fe640 1 -- 172.21.15.29:0/1514013459 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f93a4086ee0 con 0x7f93ac1556c0 2024-04-25T00:40:10.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93aad9d640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9398067fb0 0x7f939806a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:10.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.989+0000 7f93aad9d640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9398067fb0 0x7f939806a470 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f939c0af000 tx=0x7f939c0046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:10.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:10.993+0000 7f93937fe640 1 -- 172.21.15.29:0/1514013459 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f93a408a030 con 0x7f93ac1556c0 2024-04-25T00:40:11.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.153+0000 7f93b1195640 1 -- 172.21.15.29:0/1514013459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f93ac0b3480 con 0x7f93ac1556c0 2024-04-25T00:40:11.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.153+0000 7f93937fe640 1 -- 172.21.15.29:0/1514013459 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7f93a4022030 con 0x7f93ac1556c0 2024-04-25T00:40:11.155 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:40:11.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.157+0000 7f93b1195640 1 -- 172.21.15.29:0/1514013459 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9398067fb0 msgr2=0x7f939806a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.157+0000 7f93b1195640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9398067fb0 0x7f939806a470 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f939c0af000 tx=0x7f939c0046d0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.157+0000 7f93b1195640 1 -- 172.21.15.29:0/1514013459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93ac1556c0 msgr2=0x7f93ac06c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.157+0000 7f93b1195640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93ac1556c0 0x7f93ac06c070 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f93ac08d3b0 tx=0x7f93a400b480 comp rx=0 tx=0).stop 2024-04-25T00:40:11.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.157+0000 7f93b1195640 1 -- 172.21.15.29:0/1514013459 shutdown_connections 2024-04-25T00:40:11.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.157+0000 7f93b1195640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9398067fb0 0x7f939806a470 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.157+0000 7f93b1195640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93ac1556c0 0x7f93ac06c070 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.157+0000 7f93b1195640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93ac155170 0x7f93ac06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.157+0000 7f93b1195640 1 --2- 172.21.15.29:0/1514013459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93940039c0 0x7f93ac0cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.157+0000 7f93b1195640 1 -- 172.21.15.29:0/1514013459 >> 172.21.15.29:0/1514013459 conn(0x7f93ac09d280 msgr2=0x7f93ac0d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:11.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.157+0000 7f93b1195640 1 -- 172.21.15.29:0/1514013459 shutdown_connections 2024-04-25T00:40:11.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.157+0000 7f93b1195640 1 -- 172.21.15.29:0/1514013459 wait complete. 2024-04-25T00:40:11.173 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:40:11.250 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:40:11.253 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:40:11.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.301+0000 7f7c1e5f1640 1 Processor -- start 2024-04-25T00:40:11.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.301+0000 7f7c1e5f1640 1 -- start start 2024-04-25T00:40:11.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c1e5f1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c180cf690 0x7f7c180d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c1e5f1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c180d2ea0 con 0x7f7c1815e200 2024-04-25T00:40:11.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c1e5f1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c180cfc10 con 0x7f7c180d2a90 2024-04-25T00:40:11.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c1e5f1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7c180b38d0 con 0x7f7c180cf690 2024-04-25T00:40:11.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c17fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7c1815e200 0x7f7c180d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59068/0 (socket says 172.21.15.29:59068) 2024-04-25T00:40:11.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c17fff640 1 -- 172.21.15.29:0/1924133909 learned_addr learned my addr 172.21.15.29:0/1924133909 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:11.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c177fe640 1 --2- 172.21.15.29:0/1924133909 >> v2:172.21.15.29:3300/0 conn(0x7f7c180cf690 0x7f7c180d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1521104952 0 0) 0x7f7c08001120 con 0x7f7c1815e200 2024-04-25T00:40:11.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c00003530 con 0x7f7c1815e200 2024-04-25T00:40:11.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3560538517 0 0) 0x7f7c00003530 con 0x7f7c1815e200 2024-04-25T00:40:11.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 >> v2:172.21.15.29:3300/0 conn(0x7f7c180cf690 msgr2=0x7f7c180d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:11.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 --2- 172.21.15.29:0/1924133909 >> v2:172.21.15.29:3300/0 conn(0x7f7c180cf690 0x7f7c180d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 >> v1:172.21.15.157:6789/0 conn(0x7f7c180d2a90 legacy=0x7f7c180b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c180cfef0 con 0x7f7c1815e200 2024-04-25T00:40:11.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f7c08001850 con 0x7f7c1815e200 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 >> v1:172.21.15.29:6789/0 conn(0x7f7c1815e200 legacy=0x7f7c180d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 --2- 172.21.15.29:0/1924133909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c000039c0 0x7f7c00003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 --2- 172.21.15.29:0/1924133909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c00004810 0x7f7c00004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 --2- 172.21.15.29:0/1924133909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c000051d0 0x7f7c0000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7c180b38d0 con 0x7f7c000039c0 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0000c070 con 0x7f7c00004810 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c0000c270 con 0x7f7c000051d0 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c17fff640 1 --2- 172.21.15.29:0/1924133909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c000039c0 0x7f7c00003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c177fe640 1 --2- 172.21.15.29:0/1924133909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c00004810 0x7f7c00004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c17fff640 1 -- 172.21.15.29:0/1924133909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c000051d0 msgr2=0x7f7c0000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c17fff640 1 --2- 172.21.15.29:0/1924133909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c000051d0 0x7f7c0000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c17fff640 1 -- 172.21.15.29:0/1924133909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c00004810 msgr2=0x7f7c00004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c17fff640 1 --2- 172.21.15.29:0/1924133909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c00004810 0x7f7c00004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c17fff640 1 -- 172.21.15.29:0/1924133909 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c00003530 con 0x7f7c000039c0 2024-04-25T00:40:11.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c1cb8e640 1 -- 172.21.15.29:0/1924133909 reap_dead start 2024-04-25T00:40:11.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c17fff640 1 --2- 172.21.15.29:0/1924133909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c000039c0 0x7f7c00003de0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800c4c0 tx=0x7f7c0801cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c56c108f5dcbc81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:11.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0800c0e0 con 0x7f7c000039c0 2024-04-25T00:40:11.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c080a9070 con 0x7f7c000039c0 2024-04-25T00:40:11.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.305+0000 7f7c16ffd640 1 -- 172.21.15.29:0/1924133909 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08005d80 con 0x7f7c000039c0 2024-04-25T00:40:11.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/1924133909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c000039c0 msgr2=0x7f7c00003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 --2- 172.21.15.29:0/1924133909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c000039c0 0x7f7c00003de0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800c4c0 tx=0x7f7c0801cc70 comp rx=0 tx=0).stop 2024-04-25T00:40:11.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/1924133909 shutdown_connections 2024-04-25T00:40:11.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 --2- 172.21.15.29:0/1924133909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c000039c0 0x7f7c00003de0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/1924133909 >> 172.21.15.29:0/1924133909 conn(0x7f7c1809d280 msgr2=0x7f7c180d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:11.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/1924133909 shutdown_connections 2024-04-25T00:40:11.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/1924133909 wait complete. 2024-04-25T00:40:11.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 Processor -- start 2024-04-25T00:40:11.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 -- start start 2024-04-25T00:40:11.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c000039c0 0x7f7c180cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c1815e7f0 0x7f7c1815c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c180b1920 0x7f7c180b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7c180d0720 con 0x7f7c000039c0 2024-04-25T00:40:11.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c180d05a0 con 0x7f7c1815e7f0 2024-04-25T00:40:11.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c180d08a0 con 0x7f7c180b1920 2024-04-25T00:40:11.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c17fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c000039c0 0x7f7c180cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c17fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c000039c0 0x7f7c180cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52162/0 (socket says 172.21.15.29:52162) 2024-04-25T00:40:11.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c17fff640 1 -- 172.21.15.29:0/3699473981 learned_addr learned my addr 172.21.15.29:0/3699473981 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:11.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c177fe640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c1815e7f0 0x7f7c1815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1cb8e640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c180b1920 0x7f7c180b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c17fff640 1 -- 172.21.15.29:0/3699473981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c180b1920 msgr2=0x7f7c180b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c17fff640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c180b1920 0x7f7c180b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c17fff640 1 -- 172.21.15.29:0/3699473981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c1815e7f0 msgr2=0x7f7c1815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c17fff640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c1815e7f0 0x7f7c1815c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c17fff640 1 -- 172.21.15.29:0/3699473981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1815ecf0 con 0x7f7c000039c0 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c17fff640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c000039c0 0x7f7c180cdb40 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f7c0801caa0 tx=0x7f7c00005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c14ff9640 1 -- 172.21.15.29:0/3699473981 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08001120 con 0x7f7c000039c0 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c14ff9640 1 -- 172.21.15.29:0/3699473981 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c080a5860 con 0x7f7c000039c0 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/3699473981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1815efd0 con 0x7f7c000039c0 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c14ff9640 1 -- 172.21.15.29:0/3699473981 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c080a5c50 con 0x7f7c000039c0 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.309+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/3699473981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c1815f590 con 0x7f7c000039c0 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.313+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/3699473981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c18069270 con 0x7f7c000039c0 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.313+0000 7f7c14ff9640 1 -- 172.21.15.29:0/3699473981 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 23) ==== 88057+0+0 (secure 0 0 0) 0x7f7c08007180 con 0x7f7c000039c0 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.313+0000 7f7c14ff9640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c04067fb0 0x7f7c0406a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.313+0000 7f7c14ff9640 1 -- 172.21.15.29:0/3699473981 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f7c081202e0 con 0x7f7c000039c0 2024-04-25T00:40:11.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.313+0000 7f7c177fe640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c04067fb0 0x7f7c0406a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.313+0000 7f7c177fe640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c04067fb0 0x7f7c0406a470 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7c180b3300 tx=0x7f7c0c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:11.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.317+0000 7f7c14ff9640 1 -- 172.21.15.29:0/3699473981 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7c180d2ea0 con 0x7f7c000039c0 2024-04-25T00:40:11.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.473+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/3699473981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7c180ce4a0 con 0x7f7c000039c0 2024-04-25T00:40:11.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.477+0000 7f7c14ff9640 1 -- 172.21.15.29:0/3699473981 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) ==== 104+0+11 (secure 0 0 0) 0x7f7c080f0400 con 0x7f7c000039c0 2024-04-25T00:40:11.476 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:40:11.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.477+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/3699473981 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c04067fb0 msgr2=0x7f7c0406a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.477+0000 7f7c1e5f1640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c04067fb0 0x7f7c0406a470 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7c180b3300 tx=0x7f7c0c00b040 comp rx=0 tx=0).stop 2024-04-25T00:40:11.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.477+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/3699473981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c000039c0 msgr2=0x7f7c180cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.477+0000 7f7c1e5f1640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c000039c0 0x7f7c180cdb40 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f7c0801caa0 tx=0x7f7c00005840 comp rx=0 tx=0).stop 2024-04-25T00:40:11.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.477+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/3699473981 shutdown_connections 2024-04-25T00:40:11.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.477+0000 7f7c1e5f1640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7c04067fb0 0x7f7c0406a470 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.477+0000 7f7c1e5f1640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c180b1920 0x7f7c180b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.477+0000 7f7c1e5f1640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c1815e7f0 0x7f7c1815c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.477+0000 7f7c1e5f1640 1 --2- 172.21.15.29:0/3699473981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c000039c0 0x7f7c180cdb40 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.477+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/3699473981 >> 172.21.15.29:0/3699473981 conn(0x7f7c1809d280 msgr2=0x7f7c180d3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:11.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.477+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/3699473981 shutdown_connections 2024-04-25T00:40:11.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.477+0000 7f7c1e5f1640 1 -- 172.21.15.29:0/3699473981 wait complete. 2024-04-25T00:40:11.494 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:40:11.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb713eff640 1 Processor -- start 2024-04-25T00:40:11.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb713eff640 1 -- start start 2024-04-25T00:40:11.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb713eff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70c0b3c30 0x7fb70c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb713eff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb70c0b4b10 0x7fb70c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb713eff640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb70c0b41e0 con 0x7fb70c0b3140 2024-04-25T00:40:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb713eff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb70c06d960 con 0x7fb70c0b4b10 2024-04-25T00:40:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb713eff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb70c06dc40 con 0x7fb70c0b3c30 2024-04-25T00:40:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb711c9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb70c0b3140 0x7fb70c0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59070/0 (socket says 172.21.15.29:59070) 2024-04-25T00:40:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb711c9b640 1 -- 172.21.15.29:0/4112748066 learned_addr learned my addr 172.21.15.29:0/4112748066 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1075755421 0 0) 0x7fb70c0b41e0 con 0x7fb70c0b3140 2024-04-25T00:40:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6f8003530 con 0x7fb70c0b3140 2024-04-25T00:40:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb71149a640 1 --2- 172.21.15.29:0/4112748066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70c0b3c30 0x7fb70c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb71249c640 1 --2- 172.21.15.29:0/4112748066 >> v2:172.21.15.157:3300/0 conn(0x7fb70c0b4b10 0x7fb70c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 849535254 0 0) 0x7fb6f8003530 con 0x7fb70c0b3140 2024-04-25T00:40:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70c0b3c30 msgr2=0x7fb70c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 --2- 172.21.15.29:0/4112748066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70c0b3c30 0x7fb70c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 >> v2:172.21.15.157:3300/0 conn(0x7fb70c0b4b10 msgr2=0x7fb70c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:11.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 --2- 172.21.15.29:0/4112748066 >> v2:172.21.15.157:3300/0 conn(0x7fb70c0b4b10 0x7fb70c0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c06df20 con 0x7fb70c0b3140 2024-04-25T00:40:11.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fb708001680 con 0x7fb70c0b3140 2024-04-25T00:40:11.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 >> v1:172.21.15.29:6789/0 conn(0x7fb70c0b3140 legacy=0x7fb70c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 --2- 172.21.15.29:0/4112748066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6f80039c0 0x7fb6f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 --2- 172.21.15.29:0/4112748066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70c0b3c30 0x7fb6f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 --2- 172.21.15.29:0/4112748066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6f8004d50 0x7fb6f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c06d960 con 0x7fb6f80039c0 2024-04-25T00:40:11.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c06dc40 con 0x7fb6f8004d50 2024-04-25T00:40:11.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f800bb30 con 0x7fb70c0b3c30 2024-04-25T00:40:11.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb711c9b640 1 --2- 172.21.15.29:0/4112748066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6f80039c0 0x7fb6f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb71149a640 1 --2- 172.21.15.29:0/4112748066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70c0b3c30 0x7fb6f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb71249c640 1 --2- 172.21.15.29:0/4112748066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6f8004d50 0x7fb6f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb711c9b640 1 -- 172.21.15.29:0/4112748066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70c0b3c30 msgr2=0x7fb6f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:11.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb711c9b640 1 --2- 172.21.15.29:0/4112748066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70c0b3c30 0x7fb6f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb711c9b640 1 -- 172.21.15.29:0/4112748066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6f8004d50 msgr2=0x7fb6f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:11.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb711c9b640 1 --2- 172.21.15.29:0/4112748066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6f8004d50 0x7fb6f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.653+0000 7fb711c9b640 1 -- 172.21.15.29:0/4112748066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f8003530 con 0x7fb6f80039c0 2024-04-25T00:40:11.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb711c9b640 1 --2- 172.21.15.29:0/4112748066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6f80039c0 0x7fb6f8003de0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c190 tx=0x7fb70801cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d763623c1e8c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:11.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708007c90 con 0x7fb6f80039c0 2024-04-25T00:40:11.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7080a9020 con 0x7fb6f80039c0 2024-04-25T00:40:11.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb710c99640 1 -- 172.21.15.29:0/4112748066 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708005ae0 con 0x7fb6f80039c0 2024-04-25T00:40:11.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 -- 172.21.15.29:0/4112748066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6f80039c0 msgr2=0x7fb6f8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 --2- 172.21.15.29:0/4112748066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6f80039c0 0x7fb6f8003de0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c190 tx=0x7fb70801cc00 comp rx=0 tx=0).stop 2024-04-25T00:40:11.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb71249c640 1 -- 172.21.15.29:0/4112748066 reap_dead start 2024-04-25T00:40:11.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 -- 172.21.15.29:0/4112748066 shutdown_connections 2024-04-25T00:40:11.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 -- 172.21.15.29:0/4112748066 >> 172.21.15.29:0/4112748066 conn(0x7fb70c0b1870 msgr2=0x7fb70c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:11.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 -- 172.21.15.29:0/4112748066 shutdown_connections 2024-04-25T00:40:11.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 -- 172.21.15.29:0/4112748066 wait complete. 2024-04-25T00:40:11.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 Processor -- start 2024-04-25T00:40:11.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 -- start start 2024-04-25T00:40:11.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6f80039c0 0x7fb70c15e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70c156b40 0x7fb70c141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70c141d70 0x7fb70c0713a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c166d70 con 0x7fb6f80039c0 2024-04-25T00:40:11.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c166bf0 con 0x7fb70c156b40 2024-04-25T00:40:11.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb713eff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb70c166ef0 con 0x7fb70c141d70 2024-04-25T00:40:11.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb711c9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6f80039c0 0x7fb70c15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb711c9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6f80039c0 0x7fb70c15e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52184/0 (socket says 172.21.15.29:52184) 2024-04-25T00:40:11.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb711c9b640 1 -- 172.21.15.29:0/1214812489 learned_addr learned my addr 172.21.15.29:0/1214812489 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:11.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb71249c640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70c141d70 0x7fb70c0713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb71149a640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70c156b40 0x7fb70c141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb711c9b640 1 -- 172.21.15.29:0/1214812489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70c141d70 msgr2=0x7fb70c0713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb711c9b640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70c141d70 0x7fb70c0713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb711c9b640 1 -- 172.21.15.29:0/1214812489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70c156b40 msgr2=0x7fb70c141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb711c9b640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70c156b40 0x7fb70c141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb711c9b640 1 -- 172.21.15.29:0/1214812489 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c15e8c0 con 0x7fb6f80039c0 2024-04-25T00:40:11.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb71149a640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70c156b40 0x7fb70c141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:11.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.657+0000 7fb711c9b640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6f80039c0 0x7fb70c15e270 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c190 tx=0x7fb6f8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:11.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.661+0000 7fb7027fc640 1 -- 172.21.15.29:0/1214812489 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708004ab0 con 0x7fb6f80039c0 2024-04-25T00:40:11.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.661+0000 7fb7027fc640 1 -- 172.21.15.29:0/1214812489 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb708004c50 con 0x7fb6f80039c0 2024-04-25T00:40:11.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.661+0000 7fb713eff640 1 -- 172.21.15.29:0/1214812489 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70c15eba0 con 0x7fb6f80039c0 2024-04-25T00:40:11.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.661+0000 7fb7027fc640 1 -- 172.21.15.29:0/1214812489 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7080070d0 con 0x7fb6f80039c0 2024-04-25T00:40:11.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.661+0000 7fb713eff640 1 -- 172.21.15.29:0/1214812489 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70c070e80 con 0x7fb6f80039c0 2024-04-25T00:40:11.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.661+0000 7fb713eff640 1 -- 172.21.15.29:0/1214812489 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70c15f0c0 con 0x7fb6f80039c0 2024-04-25T00:40:11.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.661+0000 7fb7027fc640 1 -- 172.21.15.29:0/1214812489 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fb708007290 con 0x7fb6f80039c0 2024-04-25T00:40:11.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.661+0000 7fb7027fc640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb6f00680a0 0x7fb6f006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.661+0000 7fb7027fc640 1 -- 172.21.15.29:0/1214812489 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fb70811dc80 con 0x7fb6f80039c0 2024-04-25T00:40:11.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.661+0000 7fb71149a640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb6f00680a0 0x7fb6f006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.661+0000 7fb71149a640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb6f00680a0 0x7fb6f006a560 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb70c072ec0 tx=0x7fb6fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:11.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.665+0000 7fb7027fc640 1 -- 172.21.15.29:0/1214812489 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb70c15f0c0 con 0x7fb6f80039c0 2024-04-25T00:40:11.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.825+0000 7fb713eff640 1 -- 172.21.15.29:0/1214812489 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb70c069270 con 0x7fb6f80039c0 2024-04-25T00:40:11.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.829+0000 7fb7027fc640 1 -- 172.21.15.29:0/1214812489 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) ==== 100+0+11 (secure 0 0 0) 0x7fb708016910 con 0x7fb6f80039c0 2024-04-25T00:40:11.828 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:40:11.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.829+0000 7fb713eff640 1 -- 172.21.15.29:0/1214812489 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb6f00680a0 msgr2=0x7fb6f006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.829+0000 7fb713eff640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb6f00680a0 0x7fb6f006a560 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb70c072ec0 tx=0x7fb6fc008040 comp rx=0 tx=0).stop 2024-04-25T00:40:11.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.829+0000 7fb713eff640 1 -- 172.21.15.29:0/1214812489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6f80039c0 msgr2=0x7fb70c15e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.829+0000 7fb713eff640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6f80039c0 0x7fb70c15e270 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c190 tx=0x7fb6f8005840 comp rx=0 tx=0).stop 2024-04-25T00:40:11.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.829+0000 7fb713eff640 1 -- 172.21.15.29:0/1214812489 shutdown_connections 2024-04-25T00:40:11.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.829+0000 7fb713eff640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb6f00680a0 0x7fb6f006a560 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.829+0000 7fb713eff640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb70c141d70 0x7fb70c0713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.829+0000 7fb713eff640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70c156b40 0x7fb70c141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.829+0000 7fb713eff640 1 --2- 172.21.15.29:0/1214812489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6f80039c0 0x7fb70c15e270 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.829+0000 7fb713eff640 1 -- 172.21.15.29:0/1214812489 >> 172.21.15.29:0/1214812489 conn(0x7fb70c0b1870 msgr2=0x7fb70c161940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:11.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.829+0000 7fb713eff640 1 -- 172.21.15.29:0/1214812489 shutdown_connections 2024-04-25T00:40:11.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.829+0000 7fb713eff640 1 -- 172.21.15.29:0/1214812489 wait complete. 2024-04-25T00:40:11.848 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:40:11.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359fdbe640 1 Processor -- start 2024-04-25T00:40:11.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359fdbe640 1 -- start start 2024-04-25T00:40:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359fdbe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f359815e4e0 0x7f35980d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359fdbe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f359815e8c0 0x7f35980b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359fdbe640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35980cfe30 con 0x7f35980d4bf0 2024-04-25T00:40:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359fdbe640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f35980b6f80 con 0x7f359815e8c0 2024-04-25T00:40:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359fdbe640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f35980b7260 con 0x7f359815e4e0 2024-04-25T00:40:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359e35b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f359815e8c0 0x7f35980b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359d359640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f35980d4bf0 0x7f35980d23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59078/0 (socket says 172.21.15.29:59078) 2024-04-25T00:40:11.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359d359640 1 -- 172.21.15.29:0/2810030994 learned_addr learned my addr 172.21.15.29:0/2810030994 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:11.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359db5a640 1 --2- 172.21.15.29:0/2810030994 >> v2:172.21.15.157:3300/0 conn(0x7f359815e4e0 0x7f35980d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359e35b640 1 -- 172.21.15.29:0/2810030994 >> v2:172.21.15.157:3300/0 conn(0x7f359815e4e0 msgr2=0x7f35980d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:11.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359e35b640 1 --2- 172.21.15.29:0/2810030994 >> v2:172.21.15.157:3300/0 conn(0x7f359815e4e0 0x7f35980d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359e35b640 1 -- 172.21.15.29:0/2810030994 >> v1:172.21.15.29:6789/0 conn(0x7f35980d4bf0 legacy=0x7f35980d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359cb58640 1 -- 172.21.15.29:0/2810030994 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 104312682 0 0) 0x7f35980cfe30 con 0x7f35980d4bf0 2024-04-25T00:40:11.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.969+0000 7f359e35b640 1 -- 172.21.15.29:0/2810030994 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35980ced50 con 0x7f359815e8c0 2024-04-25T00:40:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359e35b640 1 --2- 172.21.15.29:0/2810030994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f359815e8c0 0x7f35980b7780 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f3598073380 tx=0x7f359001f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e00ba544b8fdcd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359cb58640 1 -- 172.21.15.29:0/2810030994 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35900b4070 con 0x7f359815e8c0 2024-04-25T00:40:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359cb58640 1 -- 172.21.15.29:0/2810030994 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35900af600 con 0x7f359815e8c0 2024-04-25T00:40:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359cb58640 1 -- 172.21.15.29:0/2810030994 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35900af9d0 con 0x7f359815e8c0 2024-04-25T00:40:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 -- 172.21.15.29:0/2810030994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f359815e8c0 msgr2=0x7f35980b7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 --2- 172.21.15.29:0/2810030994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f359815e8c0 0x7f35980b7780 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f3598073380 tx=0x7f359001f110 comp rx=0 tx=0).stop 2024-04-25T00:40:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 -- 172.21.15.29:0/2810030994 shutdown_connections 2024-04-25T00:40:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 --2- 172.21.15.29:0/2810030994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f359815e8c0 0x7f35980b7780 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 --2- 172.21.15.29:0/2810030994 >> v2:172.21.15.157:3300/0 conn(0x7f359815e4e0 0x7f35980d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 -- 172.21.15.29:0/2810030994 >> 172.21.15.29:0/2810030994 conn(0x7f35980d3fb0 msgr2=0x7f35980d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 -- 172.21.15.29:0/2810030994 shutdown_connections 2024-04-25T00:40:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 -- 172.21.15.29:0/2810030994 wait complete. 2024-04-25T00:40:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 Processor -- start 2024-04-25T00:40:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 -- start start 2024-04-25T00:40:11.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35980d4bf0 0x7f35980ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f359815e4e0 0x7f3598152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f359815e8c0 0x7f359806d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f35980b7e40 con 0x7f35980d4bf0 2024-04-25T00:40:11.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35980b7cc0 con 0x7f359815e4e0 2024-04-25T00:40:11.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359fdbe640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f35980b7fc0 con 0x7f359815e8c0 2024-04-25T00:40:11.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359db5a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35980d4bf0 0x7f35980ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359db5a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35980d4bf0 0x7f35980ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52208/0 (socket says 172.21.15.29:52208) 2024-04-25T00:40:11.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359db5a640 1 -- 172.21.15.29:0/1239051247 learned_addr learned my addr 172.21.15.29:0/1239051247 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:11.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359e35b640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f359815e8c0 0x7f359806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359d359640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f359815e4e0 0x7f3598152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359db5a640 1 -- 172.21.15.29:0/1239051247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f359815e8c0 msgr2=0x7f359806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359db5a640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f359815e8c0 0x7f359806d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359db5a640 1 -- 172.21.15.29:0/1239051247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f359815e4e0 msgr2=0x7f3598152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:11.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359db5a640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f359815e4e0 0x7f3598152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:11.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.973+0000 7f359db5a640 1 -- 172.21.15.29:0/1239051247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35980c9610 con 0x7f35980d4bf0 2024-04-25T00:40:11.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.977+0000 7f359db5a640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35980d4bf0 0x7f35980ca6b0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f359400d830 tx=0x7f359400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:11.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.977+0000 7f358e7fc640 1 -- 172.21.15.29:0/1239051247 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359400b6c0 con 0x7f35980d4bf0 2024-04-25T00:40:11.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.977+0000 7f358e7fc640 1 -- 172.21.15.29:0/1239051247 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3594019070 con 0x7f35980d4bf0 2024-04-25T00:40:11.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.977+0000 7f359fdbe640 1 -- 172.21.15.29:0/1239051247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35980c98f0 con 0x7f35980d4bf0 2024-04-25T00:40:11.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.977+0000 7f358e7fc640 1 -- 172.21.15.29:0/1239051247 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359400ba60 con 0x7f35980d4bf0 2024-04-25T00:40:11.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.977+0000 7f359fdbe640 1 -- 172.21.15.29:0/1239051247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35980c9eb0 con 0x7f35980d4bf0 2024-04-25T00:40:11.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.977+0000 7f359fdbe640 1 -- 172.21.15.29:0/1239051247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3598069270 con 0x7f35980d4bf0 2024-04-25T00:40:11.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.977+0000 7f358e7fc640 1 -- 172.21.15.29:0/1239051247 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f359401e460 con 0x7f35980d4bf0 2024-04-25T00:40:11.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.977+0000 7f358e7fc640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f357c0680a0 0x7f357c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:11.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.977+0000 7f358e7fc640 1 -- 172.21.15.29:0/1239051247 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f3594087580 con 0x7f35980d4bf0 2024-04-25T00:40:11.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.977+0000 7f359d359640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f357c0680a0 0x7f357c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:11.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.977+0000 7f359d359640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f357c0680a0 0x7f357c06a560 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f359815c420 tx=0x7f3588007b60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:11.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:11.981+0000 7f358e7fc640 1 -- 172.21.15.29:0/1239051247 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3594057130 con 0x7f35980d4bf0 2024-04-25T00:40:12.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.153+0000 7f359fdbe640 1 -- 172.21.15.29:0/1239051247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f35980751b0 con 0x7f35980d4bf0 2024-04-25T00:40:12.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.157+0000 7f358e7fc640 1 -- 172.21.15.29:0/1239051247 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) ==== 80+0+0 (secure 0 0 0) 0x7f3594056a30 con 0x7f35980d4bf0 2024-04-25T00:40:12.156 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:40:12.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.157+0000 7f359fdbe640 1 -- 172.21.15.29:0/1239051247 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f357c0680a0 msgr2=0x7f357c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.157+0000 7f359fdbe640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f357c0680a0 0x7f357c06a560 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f359815c420 tx=0x7f3588007b60 comp rx=0 tx=0).stop 2024-04-25T00:40:12.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.157+0000 7f359fdbe640 1 -- 172.21.15.29:0/1239051247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35980d4bf0 msgr2=0x7f35980ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.157+0000 7f359fdbe640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35980d4bf0 0x7f35980ca6b0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f359400d830 tx=0x7f359400dcf0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.157+0000 7f359fdbe640 1 -- 172.21.15.29:0/1239051247 shutdown_connections 2024-04-25T00:40:12.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.157+0000 7f359fdbe640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f357c0680a0 0x7f357c06a560 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.157+0000 7f359fdbe640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f359815e8c0 0x7f359806d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.157+0000 7f359fdbe640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f359815e4e0 0x7f3598152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.157+0000 7f359fdbe640 1 --2- 172.21.15.29:0/1239051247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35980d4bf0 0x7f35980ca6b0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.157+0000 7f359fdbe640 1 -- 172.21.15.29:0/1239051247 >> 172.21.15.29:0/1239051247 conn(0x7f35980d3fb0 msgr2=0x7f359815d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:12.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.157+0000 7f359fdbe640 1 -- 172.21.15.29:0/1239051247 shutdown_connections 2024-04-25T00:40:12.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.157+0000 7f359fdbe640 1 -- 172.21.15.29:0/1239051247 wait complete. 2024-04-25T00:40:12.174 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-04-25T00:40:12.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:12.174 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:40:12.185 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[54812]: fuse finished with error 0 and tester_r 0 2024-04-25T00:40:12.187 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:40:12.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:12.187 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:40:12.275 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:40:12.276 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:40:12.276 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:40:12.283 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:40:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8f4fa640 1 Processor -- start 2024-04-25T00:40:12.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8f4fa640 1 -- start start 2024-04-25T00:40:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8f4fa640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3a88161620 0x7f3a8806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8f4fa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a880cd8f0 0x7f3a880d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8f4fa640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a88161a00 con 0x7f3a8806d270 2024-04-25T00:40:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8f4fa640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f3a8806d6d0 con 0x7f3a88161620 2024-04-25T00:40:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8f4fa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a880cdd80 con 0x7f3a880cd8f0 2024-04-25T00:40:12.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8d296640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3a88161620 0x7f3a8806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8d296640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3a88161620 0x7f3a8806cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52218/0 (socket says 172.21.15.29:52218) 2024-04-25T00:40:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8d296640 1 -- 172.21.15.29:0/4257281334 learned_addr learned my addr 172.21.15.29:0/4257281334 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8da97640 1 --2- 172.21.15.29:0/4257281334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a880cd8f0 0x7f3a880d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8d296640 1 -- 172.21.15.29:0/4257281334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a880cd8f0 msgr2=0x7f3a880d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8d296640 1 --2- 172.21.15.29:0/4257281334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a880cd8f0 0x7f3a880d0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8d296640 1 -- 172.21.15.29:0/4257281334 >> v1:172.21.15.29:6789/0 conn(0x7f3a8806d270 legacy=0x7f3a880cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.445+0000 7f3a8d296640 1 -- 172.21.15.29:0/4257281334 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a880cdfb0 con 0x7f3a88161620 2024-04-25T00:40:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8d296640 1 --2- 172.21.15.29:0/4257281334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a88161620 0x7f3a8806cd30 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009870 tx=0x7f3a7c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf64b30e7ac2e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a7bfff640 1 -- 172.21.15.29:0/4257281334 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0a95f0 con 0x7f3a88161620 2024-04-25T00:40:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a7bfff640 1 -- 172.21.15.29:0/4257281334 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a7c0a9790 con 0x7f3a88161620 2024-04-25T00:40:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a7bfff640 1 -- 172.21.15.29:0/4257281334 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0a9b60 con 0x7f3a88161620 2024-04-25T00:40:12.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/4257281334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a88161620 msgr2=0x7f3a8806cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 --2- 172.21.15.29:0/4257281334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a88161620 0x7f3a8806cd30 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009870 tx=0x7f3a7c01c530 comp rx=0 tx=0).stop 2024-04-25T00:40:12.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/4257281334 shutdown_connections 2024-04-25T00:40:12.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 --2- 172.21.15.29:0/4257281334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a880cd8f0 0x7f3a880d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 --2- 172.21.15.29:0/4257281334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a88161620 0x7f3a8806cd30 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/4257281334 >> 172.21.15.29:0/4257281334 conn(0x7f3a8809aa30 msgr2=0x7f3a8809ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:12.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/4257281334 shutdown_connections 2024-04-25T00:40:12.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/4257281334 wait complete. 2024-04-25T00:40:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 Processor -- start 2024-04-25T00:40:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 -- start start 2024-04-25T00:40:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8806d270 0x7f3a881501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cd8f0 0x7f3a8815ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a88161620 0x7f3a8815b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a880d1ad0 con 0x7f3a88161620 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3a880d0860 con 0x7f3a880cd8f0 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8f4fa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a880d1c50 con 0x7f3a8806d270 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8da97640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a88161620 0x7f3a8815b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8da97640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a88161620 0x7f3a8815b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52222/0 (socket says 172.21.15.29:52222) 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8da97640 1 -- 172.21.15.29:0/15307896 learned_addr learned my addr 172.21.15.29:0/15307896 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8d296640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8806d270 0x7f3a881501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8da97640 1 -- 172.21.15.29:0/15307896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8806d270 msgr2=0x7f3a881501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8da97640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8806d270 0x7f3a881501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8ca95640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cd8f0 0x7f3a8815ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8da97640 1 -- 172.21.15.29:0/15307896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cd8f0 msgr2=0x7f3a8815ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8da97640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cd8f0 0x7f3a8815ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8da97640 1 -- 172.21.15.29:0/15307896 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a88150730 con 0x7f3a88161620 2024-04-25T00:40:12.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.449+0000 7f3a8da97640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a88161620 0x7f3a8815b360 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000cd70 tx=0x7f3a8000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:12.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.453+0000 7f3a79ffb640 1 -- 172.21.15.29:0/15307896 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80003ba0 con 0x7f3a88161620 2024-04-25T00:40:12.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.453+0000 7f3a79ffb640 1 -- 172.21.15.29:0/15307896 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a80003d40 con 0x7f3a88161620 2024-04-25T00:40:12.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.453+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/15307896 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a880c7db0 con 0x7f3a88161620 2024-04-25T00:40:12.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.453+0000 7f3a79ffb640 1 -- 172.21.15.29:0/15307896 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80004240 con 0x7f3a88161620 2024-04-25T00:40:12.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.453+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/15307896 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a880c8370 con 0x7f3a88161620 2024-04-25T00:40:12.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.453+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/15307896 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a88069270 con 0x7f3a88161620 2024-04-25T00:40:12.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.453+0000 7f3a79ffb640 1 -- 172.21.15.29:0/15307896 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f3a800043e0 con 0x7f3a88161620 2024-04-25T00:40:12.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.453+0000 7f3a79ffb640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3a6c0680a0 0x7f3a6c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.453+0000 7f3a79ffb640 1 -- 172.21.15.29:0/15307896 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f3a80085010 con 0x7f3a88161620 2024-04-25T00:40:12.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.453+0000 7f3a8d296640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3a6c0680a0 0x7f3a6c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.453+0000 7f3a8d296640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3a6c0680a0 0x7f3a6c06a560 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0ae000 tx=0x7f3a7c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:12.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.457+0000 7f3a79ffb640 1 -- 172.21.15.29:0/15307896 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3a80057600 con 0x7f3a88161620 2024-04-25T00:40:12.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.617+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/15307896 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a8815f870 con 0x7f3a88161620 2024-04-25T00:40:12.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.621+0000 7f3a79ffb640 1 -- 172.21.15.29:0/15307896 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16623 (secure 0 0 0) 0x7f3a80056f00 con 0x7f3a88161620 2024-04-25T00:40:12.620 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:12.620 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":145,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:50.179122+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:46.824888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:47.550815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:40:12.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.621+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/15307896 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3a6c0680a0 msgr2=0x7f3a6c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.621+0000 7f3a8f4fa640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3a6c0680a0 0x7f3a6c06a560 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0ae000 tx=0x7f3a7c009040 comp rx=0 tx=0).stop 2024-04-25T00:40:12.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.621+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/15307896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a88161620 msgr2=0x7f3a8815b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.621+0000 7f3a8f4fa640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a88161620 0x7f3a8815b360 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000cd70 tx=0x7f3a8000a660 comp rx=0 tx=0).stop 2024-04-25T00:40:12.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.621+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/15307896 shutdown_connections 2024-04-25T00:40:12.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.621+0000 7f3a8f4fa640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3a6c0680a0 0x7f3a6c06a560 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.621+0000 7f3a8f4fa640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a88161620 0x7f3a8815b360 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.621+0000 7f3a8f4fa640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cd8f0 0x7f3a8815ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.621+0000 7f3a8f4fa640 1 --2- 172.21.15.29:0/15307896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8806d270 0x7f3a881501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.621+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/15307896 >> 172.21.15.29:0/15307896 conn(0x7f3a8809aa30 msgr2=0x7f3a880705b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:12.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.621+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/15307896 shutdown_connections 2024-04-25T00:40:12.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.621+0000 7f3a8f4fa640 1 -- 172.21.15.29:0/15307896 wait complete. 2024-04-25T00:40:12.638 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:39:46.824888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:39:47.550815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:40:12.638 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:40:12.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d762c640 1 Processor -- start 2024-04-25T00:40:12.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d762c640 1 -- start start 2024-04-25T00:40:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d762c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6d015e200 0x7fd6d00d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d762c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6d00cfc10 con 0x7fd6d00cf690 2024-04-25T00:40:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d762c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6d00cfef0 con 0x7fd6d00d2c60 2024-04-25T00:40:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d762c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d00b38d0 con 0x7fd6d015e200 2024-04-25T00:40:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d4bc7640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd6d00cf690 0x7fd6d00d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59090/0 (socket says 172.21.15.29:59090) 2024-04-25T00:40:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d4bc7640 1 -- 172.21.15.29:0/3527101322 learned_addr learned my addr 172.21.15.29:0/3527101322 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d53c8640 1 --2- 172.21.15.29:0/3527101322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6d015e200 0x7fd6d00d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3223936305 0 0) 0x7fd6d00cfc10 con 0x7fd6d00cf690 2024-04-25T00:40:12.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6bc003530 con 0x7fd6d00cf690 2024-04-25T00:40:12.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2603959359 0 0) 0x7fd6bc003530 con 0x7fd6d00cf690 2024-04-25T00:40:12.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6d015e200 msgr2=0x7fd6d00d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:12.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 --2- 172.21.15.29:0/3527101322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6d015e200 0x7fd6d00d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 >> v1:172.21.15.157:6789/0 conn(0x7fd6d00d2c60 legacy=0x7fd6d00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d00b3b20 con 0x7fd6d00cf690 2024-04-25T00:40:12.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd6c0001680 con 0x7fd6d00cf690 2024-04-25T00:40:12.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 >> v1:172.21.15.29:6789/0 conn(0x7fd6d00cf690 legacy=0x7fd6d00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 --2- 172.21.15.29:0/3527101322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6bc0039f0 0x7fd6bc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 --2- 172.21.15.29:0/3527101322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6bc004810 0x7fd6bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 --2- 172.21.15.29:0/3527101322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6d015e200 0x7fd6bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d00b38d0 con 0x7fd6bc004810 2024-04-25T00:40:12.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc00ba80 con 0x7fd6bc0039f0 2024-04-25T00:40:12.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd6bc00bc80 con 0x7fd6d015e200 2024-04-25T00:40:12.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d4bc7640 1 --2- 172.21.15.29:0/3527101322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6bc004810 0x7fd6bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d53c8640 1 --2- 172.21.15.29:0/3527101322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6bc0039f0 0x7fd6bc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d5bc9640 1 --2- 172.21.15.29:0/3527101322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6d015e200 0x7fd6bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d4bc7640 1 -- 172.21.15.29:0/3527101322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6d015e200 msgr2=0x7fd6bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:12.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d4bc7640 1 --2- 172.21.15.29:0/3527101322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6d015e200 0x7fd6bc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d4bc7640 1 -- 172.21.15.29:0/3527101322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6bc0039f0 msgr2=0x7fd6bc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:12.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d4bc7640 1 --2- 172.21.15.29:0/3527101322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6bc0039f0 0x7fd6bc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.765+0000 7fd6d4bc7640 1 -- 172.21.15.29:0/3527101322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc003530 con 0x7fd6bc004810 2024-04-25T00:40:12.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d4bc7640 1 --2- 172.21.15.29:0/3527101322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6bc004810 0x7fd6bc004c90 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0001ec0 tx=0x7fd6c001f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af298854e2c7253 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:12.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0007d40 con 0x7fd6bc004810 2024-04-25T00:40:12.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c0005b00 con 0x7fd6bc004810 2024-04-25T00:40:12.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6c7fff640 1 -- 172.21.15.29:0/3527101322 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0007070 con 0x7fd6bc004810 2024-04-25T00:40:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 -- 172.21.15.29:0/3527101322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6bc004810 msgr2=0x7fd6bc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 --2- 172.21.15.29:0/3527101322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6bc004810 0x7fd6bc004c90 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0001ec0 tx=0x7fd6c001f3b0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d5bc9640 1 -- 172.21.15.29:0/3527101322 reap_dead start 2024-04-25T00:40:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 -- 172.21.15.29:0/3527101322 shutdown_connections 2024-04-25T00:40:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 -- 172.21.15.29:0/3527101322 >> 172.21.15.29:0/3527101322 conn(0x7fd6d009d280 msgr2=0x7fd6d00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 -- 172.21.15.29:0/3527101322 shutdown_connections 2024-04-25T00:40:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 -- 172.21.15.29:0/3527101322 wait complete. 2024-04-25T00:40:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 Processor -- start 2024-04-25T00:40:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 -- start start 2024-04-25T00:40:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6bc004810 0x7fd6d00b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6d014e0a0 0x7fd6d014acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6d00b6b70 0x7fd6d00b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d015e760 con 0x7fd6d00b6b70 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d015e5e0 con 0x7fd6d014e0a0 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d762c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d00d47f0 con 0x7fd6bc004810 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d5bc9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6d00b6b70 0x7fd6d00b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d5bc9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6d00b6b70 0x7fd6d00b7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52244/0 (socket says 172.21.15.29:52244) 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.769+0000 7fd6d5bc9640 1 -- 172.21.15.29:0/1052820625 learned_addr learned my addr 172.21.15.29:0/1052820625 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6d53c8640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6bc004810 0x7fd6d00b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6d5bc9640 1 -- 172.21.15.29:0/1052820625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6bc004810 msgr2=0x7fd6d00b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6d4bc7640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6d014e0a0 0x7fd6d014acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6d5bc9640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6bc004810 0x7fd6d00b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6d5bc9640 1 -- 172.21.15.29:0/1052820625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6d014e0a0 msgr2=0x7fd6d014acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6d5bc9640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6d014e0a0 0x7fd6d014acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6d5bc9640 1 -- 172.21.15.29:0/1052820625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d00b9220 con 0x7fd6d00b6b70 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6d5bc9640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6d00b6b70 0x7fd6d00b7020 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0161d40 tx=0x7fd6c800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:12.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6c5ffb640 1 -- 172.21.15.29:0/1052820625 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8003980 con 0x7fd6d00b6b70 2024-04-25T00:40:12.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6c5ffb640 1 -- 172.21.15.29:0/1052820625 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c8018070 con 0x7fd6d00b6b70 2024-04-25T00:40:12.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6d762c640 1 -- 172.21.15.29:0/1052820625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d00b9500 con 0x7fd6d00b6b70 2024-04-25T00:40:12.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6c5ffb640 1 -- 172.21.15.29:0/1052820625 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8003d60 con 0x7fd6d00b6b70 2024-04-25T00:40:12.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6d762c640 1 -- 172.21.15.29:0/1052820625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d00b9ac0 con 0x7fd6d00b6b70 2024-04-25T00:40:12.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6d762c640 1 -- 172.21.15.29:0/1052820625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d0069270 con 0x7fd6d00b6b70 2024-04-25T00:40:12.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6c5ffb640 1 -- 172.21.15.29:0/1052820625 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fd6c80050d0 con 0x7fd6d00b6b70 2024-04-25T00:40:12.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6c5ffb640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd6b40680a0 0x7fd6b406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:12.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6c5ffb640 1 -- 172.21.15.29:0/1052820625 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fd6c8086c30 con 0x7fd6d00b6b70 2024-04-25T00:40:12.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6d53c8640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd6b40680a0 0x7fd6b406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:12.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.773+0000 7fd6d53c8640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd6b40680a0 0x7fd6b406a560 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009a30 tx=0x7fd6cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:12.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.777+0000 7fd6c5ffb640 1 -- 172.21.15.29:0/1052820625 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd6c80567e0 con 0x7fd6d00b6b70 2024-04-25T00:40:12.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.937+0000 7fd6d762c640 1 -- 172.21.15.29:0/1052820625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6d00ce4a0 con 0x7fd6d00b6b70 2024-04-25T00:40:12.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.937+0000 7fd6c5ffb640 1 -- 172.21.15.29:0/1052820625 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7fd6c80560e0 con 0x7fd6d00b6b70 2024-04-25T00:40:12.939 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:40:12.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.941+0000 7fd6d762c640 1 -- 172.21.15.29:0/1052820625 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd6b40680a0 msgr2=0x7fd6b406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.941+0000 7fd6d762c640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd6b40680a0 0x7fd6b406a560 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009a30 tx=0x7fd6cc008040 comp rx=0 tx=0).stop 2024-04-25T00:40:12.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.941+0000 7fd6d762c640 1 -- 172.21.15.29:0/1052820625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6d00b6b70 msgr2=0x7fd6d00b7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:12.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.941+0000 7fd6d762c640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6d00b6b70 0x7fd6d00b7020 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0161d40 tx=0x7fd6c800ec10 comp rx=0 tx=0).stop 2024-04-25T00:40:12.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.941+0000 7fd6d762c640 1 -- 172.21.15.29:0/1052820625 shutdown_connections 2024-04-25T00:40:12.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.941+0000 7fd6d762c640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd6b40680a0 0x7fd6b406a560 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.941+0000 7fd6d762c640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6d00b6b70 0x7fd6d00b7020 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.941+0000 7fd6d762c640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6d014e0a0 0x7fd6d014acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.941+0000 7fd6d762c640 1 --2- 172.21.15.29:0/1052820625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6bc004810 0x7fd6d00b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:12.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.941+0000 7fd6d762c640 1 -- 172.21.15.29:0/1052820625 >> 172.21.15.29:0/1052820625 conn(0x7fd6d009d280 msgr2=0x7fd6d00d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:12.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.941+0000 7fd6d762c640 1 -- 172.21.15.29:0/1052820625 shutdown_connections 2024-04-25T00:40:12.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:12.941+0000 7fd6d762c640 1 -- 172.21.15.29:0/1052820625 wait complete. 2024-04-25T00:40:12.959 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:40:13.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa5a118a640 1 Processor -- start 2024-04-25T00:40:13.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa5a118a640 1 -- start start 2024-04-25T00:40:13.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa5a118a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa59c15bdb0 0x7fa59c0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa5a118a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa59c15c190 0x7fa59c0ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa5a118a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa59c0d02d0 0x7fa59c0d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa5a118a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa59c0cd3e0 con 0x7fa59c15bdb0 2024-04-25T00:40:13.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa5a118a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa59c0cd6c0 con 0x7fa59c15c190 2024-04-25T00:40:13.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa5a118a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa59c0cd9a0 con 0x7fa59c0d02d0 2024-04-25T00:40:13.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa59ad9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa59c15bdb0 0x7fa59c0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa59ad9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa59c15bdb0 0x7fa59c0cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52248/0 (socket says 172.21.15.29:52248) 2024-04-25T00:40:13.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa59ad9d640 1 -- 172.21.15.29:0/4164036809 learned_addr learned my addr 172.21.15.29:0/4164036809 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:13.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa59b59e640 1 --2- 172.21.15.29:0/4164036809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa59c0d02d0 0x7fa59c0d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa59ad9d640 1 -- 172.21.15.29:0/4164036809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa59c0d02d0 msgr2=0x7fa59c0d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:13.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa593fff640 1 --2- 172.21.15.29:0/4164036809 >> v2:172.21.15.157:3300/0 conn(0x7fa59c15c190 0x7fa59c0ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa59ad9d640 1 --2- 172.21.15.29:0/4164036809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa59c0d02d0 0x7fa59c0d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa59ad9d640 1 -- 172.21.15.29:0/4164036809 >> v2:172.21.15.157:3300/0 conn(0x7fa59c15c190 msgr2=0x7fa59c0ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:13.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa59ad9d640 1 --2- 172.21.15.29:0/4164036809 >> v2:172.21.15.157:3300/0 conn(0x7fa59c15c190 0x7fa59c0ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa59ad9d640 1 -- 172.21.15.29:0/4164036809 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c0cdc80 con 0x7fa59c15bdb0 2024-04-25T00:40:13.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.089+0000 7fa593fff640 1 --2- 172.21.15.29:0/4164036809 >> v2:172.21.15.157:3300/0 conn(0x7fa59c15c190 0x7fa59c0ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:13.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa59ad9d640 1 --2- 172.21.15.29:0/4164036809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa59c15bdb0 0x7fa59c0cc820 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0cc360 tx=0x7fa58c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec64c1af64408fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:13.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa59a59c640 1 -- 172.21.15.29:0/4164036809 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0b0070 con 0x7fa59c15bdb0 2024-04-25T00:40:13.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa59a59c640 1 -- 172.21.15.29:0/4164036809 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa58c0ab600 con 0x7fa59c15bdb0 2024-04-25T00:40:13.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa59a59c640 1 -- 172.21.15.29:0/4164036809 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0ab9d0 con 0x7fa59c15bdb0 2024-04-25T00:40:13.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 -- 172.21.15.29:0/4164036809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa59c15bdb0 msgr2=0x7fa59c0cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 --2- 172.21.15.29:0/4164036809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa59c15bdb0 0x7fa59c0cc820 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0cc360 tx=0x7fa58c01c490 comp rx=0 tx=0).stop 2024-04-25T00:40:13.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 -- 172.21.15.29:0/4164036809 shutdown_connections 2024-04-25T00:40:13.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 --2- 172.21.15.29:0/4164036809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa59c0d02d0 0x7fa59c0d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 --2- 172.21.15.29:0/4164036809 >> v2:172.21.15.157:3300/0 conn(0x7fa59c15c190 0x7fa59c0ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 --2- 172.21.15.29:0/4164036809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa59c15bdb0 0x7fa59c0cc820 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 -- 172.21.15.29:0/4164036809 >> 172.21.15.29:0/4164036809 conn(0x7fa59c09aa30 msgr2=0x7fa59c09ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:13.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 -- 172.21.15.29:0/4164036809 shutdown_connections 2024-04-25T00:40:13.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 -- 172.21.15.29:0/4164036809 wait complete. 2024-04-25T00:40:13.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 Processor -- start 2024-04-25T00:40:13.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 -- start start 2024-04-25T00:40:13.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa59c0d02d0 0x7fa59c0caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa59c15bdb0 0x7fa59c0c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa59c1570c0 0x7fa59c0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c0d2560 con 0x7fa59c1570c0 2024-04-25T00:40:13.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c0d23e0 con 0x7fa59c0d02d0 2024-04-25T00:40:13.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa5a118a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa59c0d26e0 con 0x7fa59c15bdb0 2024-04-25T00:40:13.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa59b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa59c1570c0 0x7fa59c0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa59b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa59c1570c0 0x7fa59c0c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52264/0 (socket says 172.21.15.29:52264) 2024-04-25T00:40:13.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa59b59e640 1 -- 172.21.15.29:0/3450443333 learned_addr learned my addr 172.21.15.29:0/3450443333 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:13.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa59ad9d640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa59c0d02d0 0x7fa59c0caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa593fff640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa59c15bdb0 0x7fa59c0c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa59b59e640 1 -- 172.21.15.29:0/3450443333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa59c15bdb0 msgr2=0x7fa59c0c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa59b59e640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa59c15bdb0 0x7fa59c0c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa59b59e640 1 -- 172.21.15.29:0/3450443333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa59c0d02d0 msgr2=0x7fa59c0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa59b59e640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa59c0d02d0 0x7fa59c0caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.093+0000 7fa59b59e640 1 -- 172.21.15.29:0/3450443333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c0c9d30 con 0x7fa59c1570c0 2024-04-25T00:40:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.097+0000 7fa59b59e640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa59c1570c0 0x7fa59c0c7180 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fa594007e90 tx=0x7fa59400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.097+0000 7fa5937fe640 1 -- 172.21.15.29:0/3450443333 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594014420 con 0x7fa59c1570c0 2024-04-25T00:40:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.097+0000 7fa5937fe640 1 -- 172.21.15.29:0/3450443333 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa594019070 con 0x7fa59c1570c0 2024-04-25T00:40:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.097+0000 7fa5a118a640 1 -- 172.21.15.29:0/3450443333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59c06c530 con 0x7fa59c1570c0 2024-04-25T00:40:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.097+0000 7fa5937fe640 1 -- 172.21.15.29:0/3450443333 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594014800 con 0x7fa59c1570c0 2024-04-25T00:40:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.097+0000 7fa5a118a640 1 -- 172.21.15.29:0/3450443333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59c06caf0 con 0x7fa59c1570c0 2024-04-25T00:40:13.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.097+0000 7fa5a118a640 1 -- 172.21.15.29:0/3450443333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59c069270 con 0x7fa59c1570c0 2024-04-25T00:40:13.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.097+0000 7fa5937fe640 1 -- 172.21.15.29:0/3450443333 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fa5940149a0 con 0x7fa59c1570c0 2024-04-25T00:40:13.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.097+0000 7fa5937fe640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa57c0680d0 0x7fa57c06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.097+0000 7fa5937fe640 1 -- 172.21.15.29:0/3450443333 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fa594087630 con 0x7fa59c1570c0 2024-04-25T00:40:13.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.097+0000 7fa59ad9d640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa57c0680d0 0x7fa57c06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.097+0000 7fa59ad9d640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa57c0680d0 0x7fa57c06a590 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0ad000 tx=0x7fa58c0a4f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:13.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.101+0000 7fa5937fe640 1 -- 172.21.15.29:0/3450443333 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa594057290 con 0x7fa59c1570c0 2024-04-25T00:40:13.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.257+0000 7fa5a118a640 1 -- 172.21.15.29:0/3450443333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa59c09f9d0 con 0x7fa59c1570c0 2024-04-25T00:40:13.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.257+0000 7fa5937fe640 1 -- 172.21.15.29:0/3450443333 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) ==== 104+0+11 (secure 0 0 0) 0x7fa594056b90 con 0x7fa59c1570c0 2024-04-25T00:40:13.259 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:40:13.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.261+0000 7fa5a118a640 1 -- 172.21.15.29:0/3450443333 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa57c0680d0 msgr2=0x7fa57c06a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.261+0000 7fa5a118a640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa57c0680d0 0x7fa57c06a590 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0ad000 tx=0x7fa58c0a4f90 comp rx=0 tx=0).stop 2024-04-25T00:40:13.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.261+0000 7fa5a118a640 1 -- 172.21.15.29:0/3450443333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa59c1570c0 msgr2=0x7fa59c0c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.261+0000 7fa5a118a640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa59c1570c0 0x7fa59c0c7180 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fa594007e90 tx=0x7fa59400e4b0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.261+0000 7fa5a118a640 1 -- 172.21.15.29:0/3450443333 shutdown_connections 2024-04-25T00:40:13.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.261+0000 7fa5a118a640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa57c0680d0 0x7fa57c06a590 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.261+0000 7fa5a118a640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa59c1570c0 0x7fa59c0c7180 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.261+0000 7fa5a118a640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa59c15bdb0 0x7fa59c0c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.261+0000 7fa5a118a640 1 --2- 172.21.15.29:0/3450443333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa59c0d02d0 0x7fa59c0caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.261+0000 7fa5a118a640 1 -- 172.21.15.29:0/3450443333 >> 172.21.15.29:0/3450443333 conn(0x7fa59c09aa30 msgr2=0x7fa59c070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:13.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.261+0000 7fa5a118a640 1 -- 172.21.15.29:0/3450443333 shutdown_connections 2024-04-25T00:40:13.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.261+0000 7fa5a118a640 1 -- 172.21.15.29:0/3450443333 wait complete. 2024-04-25T00:40:13.277 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:40:13.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.401+0000 7f389f0d1640 1 Processor -- start 2024-04-25T00:40:13.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f389f0d1640 1 -- start start 2024-04-25T00:40:13.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f389f0d1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38980d2bb0 0x7f38980ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f389f0d1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f389815e630 0x7f38980cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f389f0d1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38980c8690 con 0x7f389815e250 2024-04-25T00:40:13.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f389f0d1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f38980c8440 con 0x7f389815e630 2024-04-25T00:40:13.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f389f0d1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f38980c8970 con 0x7f38980d2bb0 2024-04-25T00:40:13.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f389d66e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f389815e630 0x7f38980cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f389d66e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f389815e630 0x7f38980cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52278/0 (socket says 172.21.15.29:52278) 2024-04-25T00:40:13.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f389d66e640 1 -- 172.21.15.29:0/1253878188 learned_addr learned my addr 172.21.15.29:0/1253878188 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:13.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f3897fff640 1 --2- 172.21.15.29:0/1253878188 >> v2:172.21.15.157:3300/0 conn(0x7f38980d2bb0 0x7f38980ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f389d66e640 1 -- 172.21.15.29:0/1253878188 >> v2:172.21.15.157:3300/0 conn(0x7f38980d2bb0 msgr2=0x7f38980ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:13.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f389d66e640 1 --2- 172.21.15.29:0/1253878188 >> v2:172.21.15.157:3300/0 conn(0x7f38980d2bb0 0x7f38980ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f389d66e640 1 -- 172.21.15.29:0/1253878188 >> v1:172.21.15.157:6789/0 conn(0x7f389815e250 legacy=0x7f38980d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.405+0000 7f389d66e640 1 -- 172.21.15.29:0/1253878188 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38980c8c50 con 0x7f389815e630 2024-04-25T00:40:13.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f389d66e640 1 --2- 172.21.15.29:0/1253878188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389815e630 0x7f38980cf030 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f3898067f10 tx=0x7f388801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19eb2bd8d5966a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:13.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f38977fe640 1 -- 172.21.15.29:0/1253878188 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38880b1070 con 0x7f389815e630 2024-04-25T00:40:13.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f38977fe640 1 -- 172.21.15.29:0/1253878188 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38880ac600 con 0x7f389815e630 2024-04-25T00:40:13.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f38977fe640 1 -- 172.21.15.29:0/1253878188 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38880ac9d0 con 0x7f389815e630 2024-04-25T00:40:13.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f389f0d1640 1 -- 172.21.15.29:0/1253878188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389815e630 msgr2=0x7f38980cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f389f0d1640 1 --2- 172.21.15.29:0/1253878188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389815e630 0x7f38980cf030 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f3898067f10 tx=0x7f388801c490 comp rx=0 tx=0).stop 2024-04-25T00:40:13.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f389f0d1640 1 -- 172.21.15.29:0/1253878188 shutdown_connections 2024-04-25T00:40:13.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f389f0d1640 1 --2- 172.21.15.29:0/1253878188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389815e630 0x7f38980cf030 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f389f0d1640 1 --2- 172.21.15.29:0/1253878188 >> v2:172.21.15.157:3300/0 conn(0x7f38980d2bb0 0x7f38980ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f389f0d1640 1 -- 172.21.15.29:0/1253878188 >> 172.21.15.29:0/1253878188 conn(0x7f38980d3f70 msgr2=0x7f38980d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:13.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f389f0d1640 1 -- 172.21.15.29:0/1253878188 shutdown_connections 2024-04-25T00:40:13.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f389f0d1640 1 -- 172.21.15.29:0/1253878188 wait complete. 2024-04-25T00:40:13.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f389f0d1640 1 Processor -- start 2024-04-25T00:40:13.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.409+0000 7f389f0d1640 1 -- start start 2024-04-25T00:40:13.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f389f0d1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38980d2bb0 0x7f389815ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f389f0d1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389815e250 0x7f38980cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f389f0d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389815e630 0x7f3898153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f389f0d1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f389816aff0 con 0x7f389815e250 2024-04-25T00:40:13.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f389f0d1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38980d1ed0 con 0x7f389815e630 2024-04-25T00:40:13.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f389f0d1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f389816b170 con 0x7f38980d2bb0 2024-04-25T00:40:13.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f3897fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389815e250 0x7f38980cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f3897fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389815e250 0x7f38980cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52290/0 (socket says 172.21.15.29:52290) 2024-04-25T00:40:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f3897fff640 1 -- 172.21.15.29:0/1939144694 learned_addr learned my addr 172.21.15.29:0/1939144694 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f389ce6d640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38980d2bb0 0x7f389815ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f389d66e640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389815e630 0x7f3898153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f3897fff640 1 -- 172.21.15.29:0/1939144694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38980d2bb0 msgr2=0x7f389815ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f3897fff640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38980d2bb0 0x7f389815ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f3897fff640 1 -- 172.21.15.29:0/1939144694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389815e630 msgr2=0x7f3898153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f3897fff640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389815e630 0x7f3898153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f3897fff640 1 -- 172.21.15.29:0/1939144694 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38980cbeb0 con 0x7f389815e250 2024-04-25T00:40:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f3897fff640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389815e250 0x7f38980cb970 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f388c00e9e0 tx=0x7f388c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f38957fa640 1 -- 172.21.15.29:0/1939144694 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c017070 con 0x7f389815e250 2024-04-25T00:40:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f38957fa640 1 -- 172.21.15.29:0/1939144694 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388c00c9d0 con 0x7f389815e250 2024-04-25T00:40:13.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f389f0d1640 1 -- 172.21.15.29:0/1939144694 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38980cc130 con 0x7f389815e250 2024-04-25T00:40:13.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f38957fa640 1 -- 172.21.15.29:0/1939144694 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c00cda0 con 0x7f389815e250 2024-04-25T00:40:13.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f389f0d1640 1 -- 172.21.15.29:0/1939144694 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38980cc6f0 con 0x7f389815e250 2024-04-25T00:40:13.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f389f0d1640 1 -- 172.21.15.29:0/1939144694 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3898069270 con 0x7f389815e250 2024-04-25T00:40:13.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f38957fa640 1 -- 172.21.15.29:0/1939144694 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f388c01a070 con 0x7f389815e250 2024-04-25T00:40:13.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f38957fa640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f38840680a0 0x7f388406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f38957fa640 1 -- 172.21.15.29:0/1939144694 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f388c084ad0 con 0x7f389815e250 2024-04-25T00:40:13.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f389ce6d640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f38840680a0 0x7f388406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.413+0000 7f389ce6d640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f38840680a0 0x7f388406a560 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f38900012f0 tx=0x7f3890009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:13.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.417+0000 7f38957fa640 1 -- 172.21.15.29:0/1939144694 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f38980cbeb0 con 0x7f389815e250 2024-04-25T00:40:13.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.581+0000 7f389f0d1640 1 -- 172.21.15.29:0/1939144694 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f38980b6010 con 0x7f389815e250 2024-04-25T00:40:13.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.581+0000 7f38957fa640 1 -- 172.21.15.29:0/1939144694 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) ==== 100+0+11 (secure 0 0 0) 0x7f388c056f00 con 0x7f389815e250 2024-04-25T00:40:13.583 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:40:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.585+0000 7f389f0d1640 1 -- 172.21.15.29:0/1939144694 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f38840680a0 msgr2=0x7f388406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.585+0000 7f389f0d1640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f38840680a0 0x7f388406a560 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f38900012f0 tx=0x7f3890009040 comp rx=0 tx=0).stop 2024-04-25T00:40:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.585+0000 7f389f0d1640 1 -- 172.21.15.29:0/1939144694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389815e250 msgr2=0x7f38980cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.585+0000 7f389f0d1640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389815e250 0x7f38980cb970 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f388c00e9e0 tx=0x7f388c00eea0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.585+0000 7f389f0d1640 1 -- 172.21.15.29:0/1939144694 shutdown_connections 2024-04-25T00:40:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.585+0000 7f389f0d1640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f38840680a0 0x7f388406a560 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.585+0000 7f389f0d1640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389815e630 0x7f3898153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.585+0000 7f389f0d1640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389815e250 0x7f38980cb970 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.585+0000 7f389f0d1640 1 --2- 172.21.15.29:0/1939144694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38980d2bb0 0x7f389815ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.585+0000 7f389f0d1640 1 -- 172.21.15.29:0/1939144694 >> 172.21.15.29:0/1939144694 conn(0x7f38980d3f70 msgr2=0x7f389806d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.585+0000 7f389f0d1640 1 -- 172.21.15.29:0/1939144694 shutdown_connections 2024-04-25T00:40:13.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.585+0000 7f389f0d1640 1 -- 172.21.15.29:0/1939144694 wait complete. 2024-04-25T00:40:13.603 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:40:13.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.721+0000 7f0e2a3ba640 1 Processor -- start 2024-04-25T00:40:13.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e2a3ba640 1 -- start start 2024-04-25T00:40:13.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e2a3ba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e240b4780 0x7f0e2415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e2a3ba640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e240cc240 0x7f0e2415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e2a3ba640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e240b3720 con 0x7f0e2415e8d0 2024-04-25T00:40:13.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e2a3ba640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f0e240b42f0 con 0x7f0e240cc240 2024-04-25T00:40:13.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e2a3ba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0e240b39c0 con 0x7f0e240b4780 2024-04-25T00:40:13.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e28957640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e240cc240 0x7f0e2415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e28957640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e240cc240 0x7f0e2415c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52298/0 (socket says 172.21.15.29:52298) 2024-04-25T00:40:13.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e28957640 1 -- 172.21.15.29:0/2605314381 learned_addr learned my addr 172.21.15.29:0/2605314381 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:13.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e237fe640 1 --2- 172.21.15.29:0/2605314381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e240b4780 0x7f0e2415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e28957640 1 -- 172.21.15.29:0/2605314381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e240b4780 msgr2=0x7f0e2415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:13.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e28957640 1 --2- 172.21.15.29:0/2605314381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e240b4780 0x7f0e2415f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e28957640 1 -- 172.21.15.29:0/2605314381 >> v1:172.21.15.29:6789/0 conn(0x7f0e2415e8d0 legacy=0x7f0e2415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e28957640 1 -- 172.21.15.29:0/2605314381 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e240b3ca0 con 0x7f0e240cc240 2024-04-25T00:40:13.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.725+0000 7f0e237fe640 1 --2- 172.21.15.29:0/2605314381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e240b4780 0x7f0e2415f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:13.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e28957640 1 --2- 172.21.15.29:0/2605314381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e240cc240 0x7f0e2415c7d0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009e50 tx=0x7f0e1c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54d6f7e8872bd3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:13.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e22ffd640 1 -- 172.21.15.29:0/2605314381 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0a57d0 con 0x7f0e240cc240 2024-04-25T00:40:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e22ffd640 1 -- 172.21.15.29:0/2605314381 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e1c0a5970 con 0x7f0e240cc240 2024-04-25T00:40:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e22ffd640 1 -- 172.21.15.29:0/2605314381 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0a5d40 con 0x7f0e240cc240 2024-04-25T00:40:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/2605314381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e240cc240 msgr2=0x7f0e2415c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 --2- 172.21.15.29:0/2605314381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e240cc240 0x7f0e2415c7d0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009e50 tx=0x7f0e1c01ca30 comp rx=0 tx=0).stop 2024-04-25T00:40:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/2605314381 shutdown_connections 2024-04-25T00:40:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 --2- 172.21.15.29:0/2605314381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e240cc240 0x7f0e2415c7d0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 --2- 172.21.15.29:0/2605314381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e240b4780 0x7f0e2415f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/2605314381 >> 172.21.15.29:0/2605314381 conn(0x7f0e240b1870 msgr2=0x7f0e240b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:13.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/2605314381 shutdown_connections 2024-04-25T00:40:13.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/2605314381 wait complete. 2024-04-25T00:40:13.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 Processor -- start 2024-04-25T00:40:13.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 -- start start 2024-04-25T00:40:13.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e240b4780 0x7f0e24072580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e240cc240 0x7f0e2413e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2415e8d0 0x7f0e240ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2415d6a0 con 0x7f0e240cc240 2024-04-25T00:40:13.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2415d520 con 0x7f0e2415e8d0 2024-04-25T00:40:13.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e2a3ba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0e24167c70 con 0x7f0e240b4780 2024-04-25T00:40:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e237fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e240cc240 0x7f0e2413e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e237fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e240cc240 0x7f0e2413e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52304/0 (socket says 172.21.15.29:52304) 2024-04-25T00:40:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e237fe640 1 -- 172.21.15.29:0/1639921403 learned_addr learned my addr 172.21.15.29:0/1639921403 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e23fff640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e240b4780 0x7f0e24072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e237fe640 1 -- 172.21.15.29:0/1639921403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e240b4780 msgr2=0x7f0e24072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e237fe640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e240b4780 0x7f0e24072580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e28957640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2415e8d0 0x7f0e240ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e237fe640 1 -- 172.21.15.29:0/1639921403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2415e8d0 msgr2=0x7f0e240ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e237fe640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2415e8d0 0x7f0e240ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e237fe640 1 -- 172.21.15.29:0/1639921403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e240af6b0 con 0x7f0e240cc240 2024-04-25T00:40:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e237fe640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e240cc240 0x7f0e2413e280 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400c7c0 tx=0x7f0e1400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e20ff9640 1 -- 172.21.15.29:0/1639921403 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14007a30 con 0x7f0e240cc240 2024-04-25T00:40:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.729+0000 7f0e20ff9640 1 -- 172.21.15.29:0/1639921403 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e14007bd0 con 0x7f0e240cc240 2024-04-25T00:40:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.733+0000 7f0e20ff9640 1 -- 172.21.15.29:0/1639921403 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e140113f0 con 0x7f0e240cc240 2024-04-25T00:40:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.733+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/1639921403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e240af990 con 0x7f0e240cc240 2024-04-25T00:40:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.733+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/1639921403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e240aff50 con 0x7f0e240cc240 2024-04-25T00:40:13.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.733+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/1639921403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e24069270 con 0x7f0e240cc240 2024-04-25T00:40:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.733+0000 7f0e20ff9640 1 -- 172.21.15.29:0/1639921403 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f0e14011590 con 0x7f0e240cc240 2024-04-25T00:40:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.733+0000 7f0e20ff9640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0e040680d0 0x7f0e0406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.733+0000 7f0e23fff640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0e040680d0 0x7f0e0406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.733+0000 7f0e20ff9640 1 -- 172.21.15.29:0/1639921403 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f0e140847f0 con 0x7f0e240cc240 2024-04-25T00:40:13.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.733+0000 7f0e23fff640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0e040680d0 0x7f0e0406a590 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0e241439f0 tx=0x7f0e0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:13.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.737+0000 7f0e20ff9640 1 -- 172.21.15.29:0/1639921403 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0e14015020 con 0x7f0e240cc240 2024-04-25T00:40:13.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.913+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/1639921403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0e2406e7b0 con 0x7f0e240cc240 2024-04-25T00:40:13.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.913+0000 7f0e20ff9640 1 -- 172.21.15.29:0/1639921403 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) ==== 80+0+0 (secure 0 0 0) 0x7f0e14056c20 con 0x7f0e240cc240 2024-04-25T00:40:13.913 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:40:13.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.913+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/1639921403 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0e040680d0 msgr2=0x7f0e0406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.913+0000 7f0e2a3ba640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0e040680d0 0x7f0e0406a590 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0e241439f0 tx=0x7f0e0c009040 comp rx=0 tx=0).stop 2024-04-25T00:40:13.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.913+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/1639921403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e240cc240 msgr2=0x7f0e2413e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:13.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.913+0000 7f0e2a3ba640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e240cc240 0x7f0e2413e280 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400c7c0 tx=0x7f0e1400cc80 comp rx=0 tx=0).stop 2024-04-25T00:40:13.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.913+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/1639921403 shutdown_connections 2024-04-25T00:40:13.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.913+0000 7f0e2a3ba640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0e040680d0 0x7f0e0406a590 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.913+0000 7f0e2a3ba640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2415e8d0 0x7f0e240ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.913+0000 7f0e2a3ba640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e240cc240 0x7f0e2413e280 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.913+0000 7f0e2a3ba640 1 --2- 172.21.15.29:0/1639921403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e240b4780 0x7f0e24072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:13.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.913+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/1639921403 >> 172.21.15.29:0/1639921403 conn(0x7f0e240b1870 msgr2=0x7f0e240ad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:13.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.913+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/1639921403 shutdown_connections 2024-04-25T00:40:13.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:13.917+0000 7f0e2a3ba640 1 -- 172.21.15.29:0/1639921403 wait complete. 2024-04-25T00:40:13.932 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-25T00:40:13.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:13.932 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T00:40:13.945 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[35063]: fuse finished with error 0 and tester_r 0 2024-04-25T00:40:13.947 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:40:13.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:13.947 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:40:14.114 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:40:14.115 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:40:14.115 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:40:14.121 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:40:14.121 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:40:14.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:14.121 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:40:14.168 INFO:teuthology.orchestra.run.smithi029.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T00:40:14.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:40:14.170 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:40:14.170 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:40:14.170 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:40:14.176 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:40:14.176 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:40:14.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:14.177 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:40:14.222 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T00:40:14.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:40:14.223 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:40:14.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8093e9640 1 Processor -- start 2024-04-25T00:40:14.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8093e9640 1 -- start start 2024-04-25T00:40:14.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8093e9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd80406dce0 0x7fd804150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:14.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8093e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd8040b2b70 0x7fd804151330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:14.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8093e9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8040b3130 con 0x7fd80415d980 2024-04-25T00:40:14.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8093e9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd8040cf560 con 0x7fd80406dce0 2024-04-25T00:40:14.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8093e9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd8040b3410 con 0x7fd8040b2b70 2024-04-25T00:40:14.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8027fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd80406dce0 0x7fd804150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:14.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8027fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd80406dce0 0x7fd804150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52314/0 (socket says 172.21.15.29:52314) 2024-04-25T00:40:14.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8027fc640 1 -- 172.21.15.29:0/1636882498 learned_addr learned my addr 172.21.15.29:0/1636882498 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:14.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8037fe640 1 --2- 172.21.15.29:0/1636882498 >> v2:172.21.15.157:3300/0 conn(0x7fd8040b2b70 0x7fd804151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:14.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8027fc640 1 -- 172.21.15.29:0/1636882498 >> v2:172.21.15.157:3300/0 conn(0x7fd8040b2b70 msgr2=0x7fd804151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:14.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8027fc640 1 --2- 172.21.15.29:0/1636882498 >> v2:172.21.15.157:3300/0 conn(0x7fd8040b2b70 0x7fd804151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8027fc640 1 -- 172.21.15.29:0/1636882498 >> v1:172.21.15.157:6789/0 conn(0x7fd80415d980 legacy=0x7fd80406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:14.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8027fc640 1 -- 172.21.15.29:0/1636882498 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8040b36f0 con 0x7fd80406dce0 2024-04-25T00:40:14.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8037fe640 1 --2- 172.21.15.29:0/1636882498 >> v2:172.21.15.157:3300/0 conn(0x7fd8040b2b70 0x7fd804151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:14.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.357+0000 7fd8027fc640 1 --2- 172.21.15.29:0/1636882498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd80406dce0 0x7fd804150df0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fd80406e1a0 tx=0x7fd7f801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df43f93fd2354201 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:14.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd801ffb640 1 -- 172.21.15.29:0/1636882498 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80a5750 con 0x7fd80406dce0 2024-04-25T00:40:14.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd801ffb640 1 -- 172.21.15.29:0/1636882498 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f80a58f0 con 0x7fd80406dce0 2024-04-25T00:40:14.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd801ffb640 1 -- 172.21.15.29:0/1636882498 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80a5cc0 con 0x7fd80406dce0 2024-04-25T00:40:14.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 -- 172.21.15.29:0/1636882498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd80406dce0 msgr2=0x7fd804150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:14.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 --2- 172.21.15.29:0/1636882498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd80406dce0 0x7fd804150df0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fd80406e1a0 tx=0x7fd7f801c980 comp rx=0 tx=0).stop 2024-04-25T00:40:14.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 -- 172.21.15.29:0/1636882498 shutdown_connections 2024-04-25T00:40:14.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 --2- 172.21.15.29:0/1636882498 >> v2:172.21.15.157:3300/0 conn(0x7fd8040b2b70 0x7fd804151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 --2- 172.21.15.29:0/1636882498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd80406dce0 0x7fd804150df0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 -- 172.21.15.29:0/1636882498 >> 172.21.15.29:0/1636882498 conn(0x7fd8040b1870 msgr2=0x7fd8040b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:14.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 -- 172.21.15.29:0/1636882498 shutdown_connections 2024-04-25T00:40:14.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 -- 172.21.15.29:0/1636882498 wait complete. 2024-04-25T00:40:14.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 Processor -- start 2024-04-25T00:40:14.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 -- start start 2024-04-25T00:40:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd80406dce0 0x7fd8041f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8040b2b70 0x7fd80414a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd80415d980 0x7fd80414a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd80406cd40 con 0x7fd8040b2b70 2024-04-25T00:40:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd804151870 con 0x7fd80406dce0 2024-04-25T00:40:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8093e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd80406cec0 con 0x7fd80415d980 2024-04-25T00:40:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8027fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8040b2b70 0x7fd80414a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8027fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8040b2b70 0x7fd80414a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52326/0 (socket says 172.21.15.29:52326) 2024-04-25T00:40:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8027fc640 1 -- 172.21.15.29:0/3309477421 learned_addr learned my addr 172.21.15.29:0/3309477421 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8037fe640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd80415d980 0x7fd80414a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd802ffd640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd80406dce0 0x7fd8041f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8027fc640 1 -- 172.21.15.29:0/3309477421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd80415d980 msgr2=0x7fd80414a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:14.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8027fc640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd80415d980 0x7fd80414a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8027fc640 1 -- 172.21.15.29:0/3309477421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd80406dce0 msgr2=0x7fd8041f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:14.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8027fc640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd80406dce0 0x7fd8041f6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8027fc640 1 -- 172.21.15.29:0/3309477421 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd804151150 con 0x7fd8040b2b70 2024-04-25T00:40:14.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.361+0000 7fd8027fc640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8040b2b70 0x7fd80414a390 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80af040 tx=0x7fd7f8019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:14.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.365+0000 7fd7eb7fe640 1 -- 172.21.15.29:0/3309477421 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80b8880 con 0x7fd8040b2b70 2024-04-25T00:40:14.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.365+0000 7fd7eb7fe640 1 -- 172.21.15.29:0/3309477421 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f80be070 con 0x7fd8040b2b70 2024-04-25T00:40:14.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.365+0000 7fd8093e9640 1 -- 172.21.15.29:0/3309477421 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd804158640 con 0x7fd8040b2b70 2024-04-25T00:40:14.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.365+0000 7fd7eb7fe640 1 -- 172.21.15.29:0/3309477421 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80b8ca0 con 0x7fd8040b2b70 2024-04-25T00:40:14.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.365+0000 7fd8093e9640 1 -- 172.21.15.29:0/3309477421 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd804158c00 con 0x7fd8040b2b70 2024-04-25T00:40:14.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.365+0000 7fd8093e9640 1 -- 172.21.15.29:0/3309477421 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd804069270 con 0x7fd8040b2b70 2024-04-25T00:40:14.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.365+0000 7fd7eb7fe640 1 -- 172.21.15.29:0/3309477421 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fd7f80c3460 con 0x7fd8040b2b70 2024-04-25T00:40:14.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.365+0000 7fd7eb7fe640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd7f00680a0 0x7fd7f006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:14.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.365+0000 7fd7eb7fe640 1 -- 172.21.15.29:0/3309477421 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fd7f811f9b0 con 0x7fd8040b2b70 2024-04-25T00:40:14.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.365+0000 7fd802ffd640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd7f00680a0 0x7fd7f006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:14.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.365+0000 7fd802ffd640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd7f00680a0 0x7fd7f006a560 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd8041503b0 tx=0x7fd7f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.369+0000 7fd7eb7fe640 1 -- 172.21.15.29:0/3309477421 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd7f80165a0 con 0x7fd8040b2b70 2024-04-25T00:40:14.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.561+0000 7fd8093e9640 1 -- 172.21.15.29:0/3309477421 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8040cf910 con 0x7fd8040b2b70 2024-04-25T00:40:14.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.561+0000 7fd7eb7fe640 1 -- 172.21.15.29:0/3309477421 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5185 (secure 0 0 0) 0x7fd7f80f1ea0 con 0x7fd8040b2b70 2024-04-25T00:40:14.563 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:14.563 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":93,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3075204329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3075204329},{"type":"v1","addr":"172.21.15.157:6833","nonce":3075204329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3377772964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3377772964},{"type":"v1","addr":"172.21.15.157:6835","nonce":3377772964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":9621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/897131811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":897131811},{"type":"v1","addr":"172.21.15.29:6833","nonce":897131811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:39:48.556647+0000","modified":"2024-04-25T00:39:51.600640+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9624":{"gid":9624,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/821427804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":821427804},{"type":"v1","addr":"172.21.15.29:6835","nonce":821427804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9624,"qdb_cluster":[9624]},"id":11}]} 2024-04-25T00:40:14.563 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 93 2024-04-25T00:40:14.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.565+0000 7fd8093e9640 1 -- 172.21.15.29:0/3309477421 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd7f00680a0 msgr2=0x7fd7f006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:14.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.565+0000 7fd8093e9640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd7f00680a0 0x7fd7f006a560 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd8041503b0 tx=0x7fd7f4008040 comp rx=0 tx=0).stop 2024-04-25T00:40:14.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.565+0000 7fd8093e9640 1 -- 172.21.15.29:0/3309477421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8040b2b70 msgr2=0x7fd80414a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:14.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.565+0000 7fd8093e9640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8040b2b70 0x7fd80414a390 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80af040 tx=0x7fd7f8019720 comp rx=0 tx=0).stop 2024-04-25T00:40:14.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.565+0000 7fd8093e9640 1 -- 172.21.15.29:0/3309477421 shutdown_connections 2024-04-25T00:40:14.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.565+0000 7fd8093e9640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd7f00680a0 0x7fd7f006a560 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.565+0000 7fd8093e9640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd80415d980 0x7fd80414a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.565+0000 7fd8093e9640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8040b2b70 0x7fd80414a390 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.565+0000 7fd8093e9640 1 --2- 172.21.15.29:0/3309477421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd80406dce0 0x7fd8041f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.565+0000 7fd8093e9640 1 -- 172.21.15.29:0/3309477421 >> 172.21.15.29:0/3309477421 conn(0x7fd8040b1870 msgr2=0x7fd80414e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:14.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.565+0000 7fd8093e9640 1 -- 172.21.15.29:0/3309477421 shutdown_connections 2024-04-25T00:40:14.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.565+0000 7fd8093e9640 1 -- 172.21.15.29:0/3309477421 wait complete. 2024-04-25T00:40:14.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:40:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.701+0000 7fe54a25c640 1 Processor -- start 2024-04-25T00:40:14.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.701+0000 7fe54a25c640 1 -- start start 2024-04-25T00:40:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.701+0000 7fe54a25c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe54415f3c0 0x7fe5440cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.701+0000 7fe54a25c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5440b3720 con 0x7fe5440b4780 2024-04-25T00:40:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.701+0000 7fe54a25c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5440b39c0 con 0x7fe54415e8d0 2024-04-25T00:40:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.701+0000 7fe54a25c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe5440b42f0 con 0x7fe54415f3c0 2024-04-25T00:40:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.701+0000 7fe542ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe5440b4780 0x7fe54415c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59112/0 (socket says 172.21.15.29:59112) 2024-04-25T00:40:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.701+0000 7fe542ffd640 1 -- 172.21.15.29:0/3364258 learned_addr learned my addr 172.21.15.29:0/3364258 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.701+0000 7fe543fff640 1 --2- 172.21.15.29:0/3364258 >> v2:172.21.15.29:3300/0 conn(0x7fe54415f3c0 0x7fe5440cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe5427fc640 1 -- 172.21.15.29:0/3364258 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2662092142 0 0) 0x7fe5440b3720 con 0x7fe5440b4780 2024-04-25T00:40:14.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe5427fc640 1 -- 172.21.15.29:0/3364258 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe52c003530 con 0x7fe5440b4780 2024-04-25T00:40:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe543fff640 1 -- 172.21.15.29:0/3364258 >> v1:172.21.15.157:6789/0 conn(0x7fe54415e8d0 legacy=0x7fe54415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe543fff640 1 -- 172.21.15.29:0/3364258 >> v1:172.21.15.29:6789/0 conn(0x7fe5440b4780 legacy=0x7fe54415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe543fff640 1 -- 172.21.15.29:0/3364258 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5440b3ca0 con 0x7fe54415f3c0 2024-04-25T00:40:14.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe543fff640 1 --2- 172.21.15.29:0/3364258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe54415f3c0 0x7fe5440cc240 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009d90 tx=0x7fe53c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f079c0cdd1b385 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe5427fc640 1 -- 172.21.15.29:0/3364258 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0b2070 con 0x7fe54415f3c0 2024-04-25T00:40:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe5427fc640 1 -- 172.21.15.29:0/3364258 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53c0a58c0 con 0x7fe54415f3c0 2024-04-25T00:40:14.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe5427fc640 1 -- 172.21.15.29:0/3364258 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0a5c90 con 0x7fe54415f3c0 2024-04-25T00:40:14.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe54a25c640 1 -- 172.21.15.29:0/3364258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe54415f3c0 msgr2=0x7fe5440cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:14.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe54a25c640 1 --2- 172.21.15.29:0/3364258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe54415f3c0 0x7fe5440cc240 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009d90 tx=0x7fe53c01c930 comp rx=0 tx=0).stop 2024-04-25T00:40:14.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe54a25c640 1 -- 172.21.15.29:0/3364258 shutdown_connections 2024-04-25T00:40:14.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe54a25c640 1 --2- 172.21.15.29:0/3364258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe54415f3c0 0x7fe5440cc240 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe54a25c640 1 -- 172.21.15.29:0/3364258 >> 172.21.15.29:0/3364258 conn(0x7fe5440b1870 msgr2=0x7fe5440b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:14.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe54a25c640 1 -- 172.21.15.29:0/3364258 shutdown_connections 2024-04-25T00:40:14.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.705+0000 7fe54a25c640 1 -- 172.21.15.29:0/3364258 wait complete. 2024-04-25T00:40:14.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe54a25c640 1 Processor -- start 2024-04-25T00:40:14.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe54a25c640 1 -- start start 2024-04-25T00:40:14.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe54a25c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5440b4780 0x7fe5441f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:14.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe54a25c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe54415e8d0 0x7fe54416a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:14.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe54a25c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54415f3c0 0x7fe54416a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:14.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe54a25c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe54406d710 con 0x7fe54415e8d0 2024-04-25T00:40:14.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe54a25c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe54406d590 con 0x7fe5440b4780 2024-04-25T00:40:14.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe54a25c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe54406d890 con 0x7fe54415f3c0 2024-04-25T00:40:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe542ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe54415e8d0 0x7fe54416a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe542ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe54415e8d0 0x7fe54416a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52356/0 (socket says 172.21.15.29:52356) 2024-04-25T00:40:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe542ffd640 1 -- 172.21.15.29:0/1061784420 learned_addr learned my addr 172.21.15.29:0/1061784420 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe5437fe640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5440b4780 0x7fe5441f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe543fff640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54415f3c0 0x7fe54416a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe542ffd640 1 -- 172.21.15.29:0/1061784420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54415f3c0 msgr2=0x7fe54416a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe542ffd640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54415f3c0 0x7fe54416a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe542ffd640 1 -- 172.21.15.29:0/1061784420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5440b4780 msgr2=0x7fe5441f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe542ffd640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5440b4780 0x7fe5441f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe542ffd640 1 -- 172.21.15.29:0/1061784420 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5440ada20 con 0x7fe54415e8d0 2024-04-25T00:40:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe543fff640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54415f3c0 0x7fe54416a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe5437fe640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5440b4780 0x7fe5441f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:14.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe542ffd640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe54415e8d0 0x7fe54416a1d0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c450 tx=0x7fe53800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:14.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe52bfff640 1 -- 172.21.15.29:0/1061784420 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538017070 con 0x7fe54415e8d0 2024-04-25T00:40:14.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe52bfff640 1 -- 172.21.15.29:0/1061784420 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe538005cf0 con 0x7fe54415e8d0 2024-04-25T00:40:14.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe52bfff640 1 -- 172.21.15.29:0/1061784420 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538007290 con 0x7fe54415e8d0 2024-04-25T00:40:14.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe54a25c640 1 -- 172.21.15.29:0/1061784420 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5440add00 con 0x7fe54415e8d0 2024-04-25T00:40:14.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe54a25c640 1 -- 172.21.15.29:0/1061784420 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5440ae2c0 con 0x7fe54415e8d0 2024-04-25T00:40:14.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.709+0000 7fe54a25c640 1 -- 172.21.15.29:0/1061784420 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe544069270 con 0x7fe54415e8d0 2024-04-25T00:40:14.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.713+0000 7fe52bfff640 1 -- 172.21.15.29:0/1061784420 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fe53801a070 con 0x7fe54415e8d0 2024-04-25T00:40:14.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.713+0000 7fe52bfff640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe5300680a0 0x7fe53006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:14.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.713+0000 7fe5437fe640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe5300680a0 0x7fe53006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:14.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.713+0000 7fe52bfff640 1 -- 172.21.15.29:0/1061784420 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fe538083a60 con 0x7fe54415e8d0 2024-04-25T00:40:14.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.713+0000 7fe5437fe640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe5300680a0 0x7fe53006a560 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe54415f390 tx=0x7fe534009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:14.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.717+0000 7fe52bfff640 1 -- 172.21.15.29:0/1061784420 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe538055e90 con 0x7fe54415e8d0 2024-04-25T00:40:14.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.877+0000 7fe54a25c640 1 -- 172.21.15.29:0/1061784420 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe54406e7b0 con 0x7fe54415e8d0 2024-04-25T00:40:14.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.881+0000 7fe52bfff640 1 -- 172.21.15.29:0/1061784420 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16623 (secure 0 0 0) 0x7fe538055790 con 0x7fe54415e8d0 2024-04-25T00:40:14.881 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:14.881 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":145,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:50.179122+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:46.824888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:47.550815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:40:14.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.885+0000 7fe54a25c640 1 -- 172.21.15.29:0/1061784420 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe5300680a0 msgr2=0x7fe53006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:14.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.885+0000 7fe54a25c640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe5300680a0 0x7fe53006a560 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe54415f390 tx=0x7fe534009040 comp rx=0 tx=0).stop 2024-04-25T00:40:14.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.885+0000 7fe54a25c640 1 -- 172.21.15.29:0/1061784420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe54415e8d0 msgr2=0x7fe54416a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:14.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.885+0000 7fe54a25c640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe54415e8d0 0x7fe54416a1d0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c450 tx=0x7fe53800c910 comp rx=0 tx=0).stop 2024-04-25T00:40:14.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.885+0000 7fe54a25c640 1 -- 172.21.15.29:0/1061784420 shutdown_connections 2024-04-25T00:40:14.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.885+0000 7fe54a25c640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe5300680a0 0x7fe53006a560 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.885+0000 7fe54a25c640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54415f3c0 0x7fe54416a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.885+0000 7fe54a25c640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe54415e8d0 0x7fe54416a1d0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.885+0000 7fe54a25c640 1 --2- 172.21.15.29:0/1061784420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5440b4780 0x7fe5441f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:14.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.885+0000 7fe54a25c640 1 -- 172.21.15.29:0/1061784420 >> 172.21.15.29:0/1061784420 conn(0x7fe5440b1870 msgr2=0x7fe5440cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:14.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.885+0000 7fe54a25c640 1 -- 172.21.15.29:0/1061784420 shutdown_connections 2024-04-25T00:40:14.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:14.885+0000 7fe54a25c640 1 -- 172.21.15.29:0/1061784420 wait complete. 2024-04-25T00:40:14.901 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:39:46.824888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:39:47.550815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:40:14.902 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:40:15.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b40f48640 1 Processor -- start 2024-04-25T00:40:15.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b40f48640 1 -- start start 2024-04-25T00:40:15.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b40f48640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b3c15e8d0 0x7f8b3c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b40f48640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4780 0x7f8b3c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b40f48640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b3c15f1f0 0x7f8b3c15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b40f48640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8b3c0b42f0 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b40f48640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8b3c0b3720 con 0x7f8b3c15f1f0 2024-04-25T00:40:15.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b40f48640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3c0b39c0 con 0x7f8b3c0b4780 2024-04-25T00:40:15.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b3a59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b3c15e8d0 0x7f8b3c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b3a59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b3c15e8d0 0x7f8b3c0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52364/0 (socket says 172.21.15.29:52364) 2024-04-25T00:40:15.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b3a59c640 1 -- 172.21.15.29:0/2211034128 learned_addr learned my addr 172.21.15.29:0/2211034128 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:15.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b39d9b640 1 --2- 172.21.15.29:0/2211034128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4780 0x7f8b3c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b3ad9d640 1 --2- 172.21.15.29:0/2211034128 >> v2:172.21.15.157:3300/0 conn(0x7f8b3c15f1f0 0x7f8b3c15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b3a59c640 1 -- 172.21.15.29:0/2211034128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4780 msgr2=0x7f8b3c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:15.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b3a59c640 1 --2- 172.21.15.29:0/2211034128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4780 0x7f8b3c15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b3a59c640 1 -- 172.21.15.29:0/2211034128 >> v2:172.21.15.157:3300/0 conn(0x7f8b3c15f1f0 msgr2=0x7f8b3c15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:15.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b3a59c640 1 --2- 172.21.15.29:0/2211034128 >> v2:172.21.15.157:3300/0 conn(0x7f8b3c15f1f0 0x7f8b3c15b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b3a59c640 1 -- 172.21.15.29:0/2211034128 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c0b3ca0 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.021+0000 7f8b3ad9d640 1 --2- 172.21.15.29:0/2211034128 >> v2:172.21.15.157:3300/0 conn(0x7f8b3c15f1f0 0x7f8b3c15b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:15.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.025+0000 7f8b3a59c640 1 --2- 172.21.15.29:0/2211034128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15e8d0 0x7f8b3c0cc240 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0708a0 tx=0x7f8b2c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdac1d0a86d1a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:15.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.025+0000 7f8b3959a640 1 -- 172.21.15.29:0/2211034128 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0a5820 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.025+0000 7f8b3959a640 1 -- 172.21.15.29:0/2211034128 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b2c0a59c0 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.025+0000 7f8b3959a640 1 -- 172.21.15.29:0/2211034128 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0a5d90 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.025+0000 7f8b40f48640 1 -- 172.21.15.29:0/2211034128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15e8d0 msgr2=0x7f8b3c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.025+0000 7f8b40f48640 1 --2- 172.21.15.29:0/2211034128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15e8d0 0x7f8b3c0cc240 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0708a0 tx=0x7f8b2c01c9c0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.025+0000 7f8b40f48640 1 -- 172.21.15.29:0/2211034128 shutdown_connections 2024-04-25T00:40:15.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.025+0000 7f8b40f48640 1 --2- 172.21.15.29:0/2211034128 >> v2:172.21.15.157:3300/0 conn(0x7f8b3c15f1f0 0x7f8b3c15b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.025+0000 7f8b40f48640 1 --2- 172.21.15.29:0/2211034128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4780 0x7f8b3c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.025+0000 7f8b40f48640 1 --2- 172.21.15.29:0/2211034128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15e8d0 0x7f8b3c0cc240 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.025+0000 7f8b40f48640 1 -- 172.21.15.29:0/2211034128 >> 172.21.15.29:0/2211034128 conn(0x7f8b3c0b1870 msgr2=0x7f8b3c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.025+0000 7f8b40f48640 1 -- 172.21.15.29:0/2211034128 shutdown_connections 2024-04-25T00:40:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.025+0000 7f8b40f48640 1 -- 172.21.15.29:0/2211034128 wait complete. 2024-04-25T00:40:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b40f48640 1 Processor -- start 2024-04-25T00:40:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b40f48640 1 -- start start 2024-04-25T00:40:15.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b40f48640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4780 0x7f8b3c137e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b40f48640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15e8d0 0x7f8b3c1375c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b40f48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c15f1f0 0x7f8b3c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b40f48640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c15f600 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b40f48640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c15c7d0 con 0x7f8b3c15f1f0 2024-04-25T00:40:15.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b40f48640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3c138580 con 0x7f8b3c0b4780 2024-04-25T00:40:15.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b39d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15e8d0 0x7f8b3c1375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b39d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15e8d0 0x7f8b3c1375c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52378/0 (socket says 172.21.15.29:52378) 2024-04-25T00:40:15.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b39d9b640 1 -- 172.21.15.29:0/1588801230 learned_addr learned my addr 172.21.15.29:0/1588801230 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:15.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b3a59c640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4780 0x7f8b3c137e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b3ad9d640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c15f1f0 0x7f8b3c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b39d9b640 1 -- 172.21.15.29:0/1588801230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4780 msgr2=0x7f8b3c137e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b39d9b640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4780 0x7f8b3c137e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b39d9b640 1 -- 172.21.15.29:0/1588801230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c15f1f0 msgr2=0x7f8b3c0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b39d9b640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c15f1f0 0x7f8b3c0cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b39d9b640 1 -- 172.21.15.29:0/1588801230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c138860 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b3a59c640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4780 0x7f8b3c137e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b39d9b640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15e8d0 0x7f8b3c1375c0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400d800 tx=0x7f8b2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b32ffd640 1 -- 172.21.15.29:0/1588801230 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24018020 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b32ffd640 1 -- 172.21.15.29:0/1588801230 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b240096a0 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b40f48640 1 -- 172.21.15.29:0/1588801230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3c138b40 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b32ffd640 1 -- 172.21.15.29:0/1588801230 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24009a50 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b40f48640 1 -- 172.21.15.29:0/1588801230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b3c139100 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b40f48640 1 -- 172.21.15.29:0/1588801230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b3c069270 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b32ffd640 1 -- 172.21.15.29:0/1588801230 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f8b2401b020 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b32ffd640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b1c0680d0 0x7f8b1c06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b32ffd640 1 -- 172.21.15.29:0/1588801230 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f8b24084be0 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.029+0000 7f8b3a59c640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b1c0680d0 0x7f8b1c06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.033+0000 7f8b3a59c640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b1c0680d0 0x7f8b1c06a590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0ae000 tx=0x7f8b2c0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:15.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.033+0000 7f8b32ffd640 1 -- 172.21.15.29:0/1588801230 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8b24022020 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.197+0000 7f8b40f48640 1 -- 172.21.15.29:0/1588801230 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8b3c072d20 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.197+0000 7f8b32ffd640 1 -- 172.21.15.29:0/1588801230 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7f8b24057010 con 0x7f8b3c15e8d0 2024-04-25T00:40:15.197 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:40:15.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.197+0000 7f8b40f48640 1 -- 172.21.15.29:0/1588801230 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b1c0680d0 msgr2=0x7f8b1c06a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.197+0000 7f8b40f48640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b1c0680d0 0x7f8b1c06a590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0ae000 tx=0x7f8b2c0a7070 comp rx=0 tx=0).stop 2024-04-25T00:40:15.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.197+0000 7f8b40f48640 1 -- 172.21.15.29:0/1588801230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15e8d0 msgr2=0x7f8b3c1375c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.197+0000 7f8b40f48640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15e8d0 0x7f8b3c1375c0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400d800 tx=0x7f8b2400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.197+0000 7f8b40f48640 1 -- 172.21.15.29:0/1588801230 shutdown_connections 2024-04-25T00:40:15.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.197+0000 7f8b40f48640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b1c0680d0 0x7f8b1c06a590 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.197+0000 7f8b40f48640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c15f1f0 0x7f8b3c0cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.197+0000 7f8b40f48640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3c15e8d0 0x7f8b3c1375c0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.197+0000 7f8b40f48640 1 --2- 172.21.15.29:0/1588801230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0b4780 0x7f8b3c137e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.197+0000 7f8b40f48640 1 -- 172.21.15.29:0/1588801230 >> 172.21.15.29:0/1588801230 conn(0x7f8b3c0b1870 msgr2=0x7f8b3c070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:15.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.201+0000 7f8b40f48640 1 -- 172.21.15.29:0/1588801230 shutdown_connections 2024-04-25T00:40:15.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.201+0000 7f8b40f48640 1 -- 172.21.15.29:0/1588801230 wait complete. 2024-04-25T00:40:15.217 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:40:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa417270640 1 Processor -- start 2024-04-25T00:40:15.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa417270640 1 -- start start 2024-04-25T00:40:15.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa417270640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4101629a0 0x7fa410163510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa417270640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa410163a50 0x7fa4100b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa417270640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa410162d80 con 0x7fa4100b09a0 2024-04-25T00:40:15.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa417270640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa410163080 con 0x7fa4101629a0 2024-04-25T00:40:15.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa417270640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa410071850 con 0x7fa410163a50 2024-04-25T00:40:15.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa41480b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa410163a50 0x7fa4100b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa41500c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4101629a0 0x7fa410163510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa41480b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa410163a50 0x7fa4100b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:32842/0 (socket says 172.21.15.29:32842) 2024-04-25T00:40:15.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa41480b640 1 -- 172.21.15.29:0/309441672 learned_addr learned my addr 172.21.15.29:0/309441672 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:15.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa41480b640 1 -- 172.21.15.29:0/309441672 >> v2:172.21.15.157:3300/0 conn(0x7fa4101629a0 msgr2=0x7fa410163510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:15.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa41480b640 1 --2- 172.21.15.29:0/309441672 >> v2:172.21.15.157:3300/0 conn(0x7fa4101629a0 0x7fa410163510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa41480b640 1 -- 172.21.15.29:0/309441672 >> v1:172.21.15.157:6789/0 conn(0x7fa4100b09a0 legacy=0x7fa4100c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.337+0000 7fa41480b640 1 -- 172.21.15.29:0/309441672 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa410074340 con 0x7fa410163a50 2024-04-25T00:40:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.341+0000 7fa41480b640 1 --2- 172.21.15.29:0/309441672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa410163a50 0x7fa4100b0460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa408009e50 tx=0x7fa40801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de9350a038dae4b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.341+0000 7fa407fff640 1 -- 172.21.15.29:0/309441672 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080a5700 con 0x7fa410163a50 2024-04-25T00:40:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.341+0000 7fa407fff640 1 -- 172.21.15.29:0/309441672 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4080a58a0 con 0x7fa410163a50 2024-04-25T00:40:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.341+0000 7fa407fff640 1 -- 172.21.15.29:0/309441672 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080a5c70 con 0x7fa410163a50 2024-04-25T00:40:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.341+0000 7fa417270640 1 -- 172.21.15.29:0/309441672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa410163a50 msgr2=0x7fa4100b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.341+0000 7fa417270640 1 --2- 172.21.15.29:0/309441672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa410163a50 0x7fa4100b0460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa408009e50 tx=0x7fa40801ca30 comp rx=0 tx=0).stop 2024-04-25T00:40:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.341+0000 7fa417270640 1 -- 172.21.15.29:0/309441672 shutdown_connections 2024-04-25T00:40:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.341+0000 7fa417270640 1 --2- 172.21.15.29:0/309441672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa410163a50 0x7fa4100b0460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.341+0000 7fa417270640 1 --2- 172.21.15.29:0/309441672 >> v2:172.21.15.157:3300/0 conn(0x7fa4101629a0 0x7fa410163510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.341+0000 7fa417270640 1 -- 172.21.15.29:0/309441672 >> 172.21.15.29:0/309441672 conn(0x7fa4100b1ec0 msgr2=0x7fa4100b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.341+0000 7fa417270640 1 -- 172.21.15.29:0/309441672 shutdown_connections 2024-04-25T00:40:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.341+0000 7fa417270640 1 -- 172.21.15.29:0/309441672 wait complete. 2024-04-25T00:40:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa417270640 1 Processor -- start 2024-04-25T00:40:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa417270640 1 -- start start 2024-04-25T00:40:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa417270640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4100b09a0 0x7fa410098b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa417270640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4101629a0 0x7fa4100c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa417270640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa410163a50 0x7fa4100addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa417270640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100af460 con 0x7fa410163a50 2024-04-25T00:40:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa417270640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100af2e0 con 0x7fa4100b09a0 2024-04-25T00:40:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa417270640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa41006bf80 con 0x7fa4101629a0 2024-04-25T00:40:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa41580d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa410163a50 0x7fa4100addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa41580d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa410163a50 0x7fa4100addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52386/0 (socket says 172.21.15.29:52386) 2024-04-25T00:40:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa41580d640 1 -- 172.21.15.29:0/3256711514 learned_addr learned my addr 172.21.15.29:0/3256711514 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa41500c640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4100b09a0 0x7fa410098b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa41480b640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4101629a0 0x7fa4100c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa41580d640 1 -- 172.21.15.29:0/3256711514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4101629a0 msgr2=0x7fa4100c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa41580d640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4101629a0 0x7fa4100c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa41580d640 1 -- 172.21.15.29:0/3256711514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4100b09a0 msgr2=0x7fa410098b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa41580d640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4100b09a0 0x7fa410098b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa41580d640 1 -- 172.21.15.29:0/3256711514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4101654d0 con 0x7fa410163a50 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa41500c640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4100b09a0 0x7fa410098b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa41580d640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa410163a50 0x7fa4100addd0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c390 tx=0x7fa40c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa405ffb640 1 -- 172.21.15.29:0/3256711514 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0052d0 con 0x7fa410163a50 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa405ffb640 1 -- 172.21.15.29:0/3256711514 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c005470 con 0x7fa410163a50 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa405ffb640 1 -- 172.21.15.29:0/3256711514 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c007240 con 0x7fa410163a50 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa417270640 1 -- 172.21.15.29:0/3256711514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4101657b0 con 0x7fa410163a50 2024-04-25T00:40:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa417270640 1 -- 172.21.15.29:0/3256711514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa410165d70 con 0x7fa410163a50 2024-04-25T00:40:15.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa417270640 1 -- 172.21.15.29:0/3256711514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa410069270 con 0x7fa410163a50 2024-04-25T00:40:15.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.345+0000 7fa405ffb640 1 -- 172.21.15.29:0/3256711514 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fa40c0073e0 con 0x7fa410163a50 2024-04-25T00:40:15.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.349+0000 7fa405ffb640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa3fc0680a0 0x7fa3fc06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.349+0000 7fa405ffb640 1 -- 172.21.15.29:0/3256711514 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fa40c083a40 con 0x7fa410163a50 2024-04-25T00:40:15.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.349+0000 7fa41500c640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa3fc0680a0 0x7fa3fc06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.349+0000 7fa41500c640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa3fc0680a0 0x7fa3fc06a560 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa400009bb0 tx=0x7fa400008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:15.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.353+0000 7fa405ffb640 1 -- 172.21.15.29:0/3256711514 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa40c055e70 con 0x7fa410163a50 2024-04-25T00:40:15.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa417270640 1 -- 172.21.15.29:0/3256711514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa4100726d0 con 0x7fa410163a50 2024-04-25T00:40:15.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa405ffb640 1 -- 172.21.15.29:0/3256711514 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) ==== 104+0+11 (secure 0 0 0) 0x7fa40c001010 con 0x7fa410163a50 2024-04-25T00:40:15.517 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:40:15.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa417270640 1 -- 172.21.15.29:0/3256711514 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa3fc0680a0 msgr2=0x7fa3fc06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa417270640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa3fc0680a0 0x7fa3fc06a560 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa400009bb0 tx=0x7fa400008040 comp rx=0 tx=0).stop 2024-04-25T00:40:15.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa417270640 1 -- 172.21.15.29:0/3256711514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa410163a50 msgr2=0x7fa4100addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa417270640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa410163a50 0x7fa4100addd0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c390 tx=0x7fa40c00c850 comp rx=0 tx=0).stop 2024-04-25T00:40:15.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa417270640 1 -- 172.21.15.29:0/3256711514 shutdown_connections 2024-04-25T00:40:15.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa417270640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa3fc0680a0 0x7fa3fc06a560 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa417270640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa410163a50 0x7fa4100addd0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa417270640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4101629a0 0x7fa4100c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa417270640 1 --2- 172.21.15.29:0/3256711514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4100b09a0 0x7fa410098b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa417270640 1 -- 172.21.15.29:0/3256711514 >> 172.21.15.29:0/3256711514 conn(0x7fa4100b1ec0 msgr2=0x7fa410157c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:15.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa417270640 1 -- 172.21.15.29:0/3256711514 shutdown_connections 2024-04-25T00:40:15.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.517+0000 7fa417270640 1 -- 172.21.15.29:0/3256711514 wait complete. 2024-04-25T00:40:15.537 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:40:15.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.665+0000 7fd20de1b640 1 Processor -- start 2024-04-25T00:40:15.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.665+0000 7fd20de1b640 1 -- start start 2024-04-25T00:40:15.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.665+0000 7fd20de1b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd2080afa50 0x7fd2080b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.665+0000 7fd20de1b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20806bc20 0x7fd208070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.665+0000 7fd20de1b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2080adaf0 con 0x7fd20815e410 2024-04-25T00:40:15.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.665+0000 7fd20de1b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd2080ad8a0 con 0x7fd2080afa50 2024-04-25T00:40:15.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.665+0000 7fd20de1b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd2080ae420 con 0x7fd20806bc20 2024-04-25T00:40:15.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd2077fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd2080afa50 0x7fd2080b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd2077fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd2080afa50 0x7fd2080b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52402/0 (socket says 172.21.15.29:52402) 2024-04-25T00:40:15.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd2077fe640 1 -- 172.21.15.29:0/3951465510 learned_addr learned my addr 172.21.15.29:0/3951465510 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:15.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd207fff640 1 --2- 172.21.15.29:0/3951465510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20806bc20 0x7fd208070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd2077fe640 1 -- 172.21.15.29:0/3951465510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20806bc20 msgr2=0x7fd208070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:15.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd2077fe640 1 --2- 172.21.15.29:0/3951465510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20806bc20 0x7fd208070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd2077fe640 1 -- 172.21.15.29:0/3951465510 >> v1:172.21.15.157:6789/0 conn(0x7fd20815e410 legacy=0x7fd2080afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:15.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd2077fe640 1 -- 172.21.15.29:0/3951465510 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2080addd0 con 0x7fd2080afa50 2024-04-25T00:40:15.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd2077fe640 1 --2- 172.21.15.29:0/3951465510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2080afa50 0x7fd2080b3b80 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009870 tx=0x7fd1f801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67573df503edfa11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:15.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd206ffd640 1 -- 172.21.15.29:0/3951465510 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80b1070 con 0x7fd2080afa50 2024-04-25T00:40:15.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd206ffd640 1 -- 172.21.15.29:0/3951465510 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f80ac630 con 0x7fd2080afa50 2024-04-25T00:40:15.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd206ffd640 1 -- 172.21.15.29:0/3951465510 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80a8050 con 0x7fd2080afa50 2024-04-25T00:40:15.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd20de1b640 1 -- 172.21.15.29:0/3951465510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2080afa50 msgr2=0x7fd2080b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd20de1b640 1 --2- 172.21.15.29:0/3951465510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2080afa50 0x7fd2080b3b80 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009870 tx=0x7fd1f801c4f0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd20de1b640 1 -- 172.21.15.29:0/3951465510 shutdown_connections 2024-04-25T00:40:15.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd20de1b640 1 --2- 172.21.15.29:0/3951465510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20806bc20 0x7fd208070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd20de1b640 1 --2- 172.21.15.29:0/3951465510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2080afa50 0x7fd2080b3b80 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd20de1b640 1 -- 172.21.15.29:0/3951465510 >> 172.21.15.29:0/3951465510 conn(0x7fd2080d0c90 msgr2=0x7fd2080d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:15.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd20de1b640 1 -- 172.21.15.29:0/3951465510 shutdown_connections 2024-04-25T00:40:15.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.669+0000 7fd20de1b640 1 -- 172.21.15.29:0/3951465510 wait complete. 2024-04-25T00:40:15.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd20de1b640 1 Processor -- start 2024-04-25T00:40:15.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd20de1b640 1 -- start start 2024-04-25T00:40:15.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd20de1b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd20806bc20 0x7fd2080cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd20de1b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2080afa50 0x7fd2080ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd20de1b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20815e410 0x7fd208152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd20de1b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd2080af490 con 0x7fd2080afa50 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd20de1b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd2080af310 con 0x7fd20806bc20 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd20de1b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd2080af610 con 0x7fd20815e410 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd1fffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2080afa50 0x7fd2080ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd207fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20815e410 0x7fd208152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd1fffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2080afa50 0x7fd2080ce1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52408/0 (socket says 172.21.15.29:52408) 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd1fffff640 1 -- 172.21.15.29:0/1056629515 learned_addr learned my addr 172.21.15.29:0/1056629515 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd2077fe640 1 --2- 172.21.15.29:0/1056629515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd20806bc20 0x7fd2080cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd1fffff640 1 -- 172.21.15.29:0/1056629515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20815e410 msgr2=0x7fd208152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd1fffff640 1 --2- 172.21.15.29:0/1056629515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20815e410 0x7fd208152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd1fffff640 1 -- 172.21.15.29:0/1056629515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd20806bc20 msgr2=0x7fd2080cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd1fffff640 1 --2- 172.21.15.29:0/1056629515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd20806bc20 0x7fd2080cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd1fffff640 1 -- 172.21.15.29:0/1056629515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2080ce720 con 0x7fd2080afa50 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd1fffff640 1 --2- 172.21.15.29:0/1056629515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2080afa50 0x7fd2080ce1e0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000b410 tx=0x7fd1f000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:15.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd204ff9640 1 -- 172.21.15.29:0/1056629515 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0015030 con 0x7fd2080afa50 2024-04-25T00:40:15.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd204ff9640 1 -- 172.21.15.29:0/1056629515 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f0004020 con 0x7fd2080afa50 2024-04-25T00:40:15.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd20de1b640 1 -- 172.21.15.29:0/1056629515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2080cecf0 con 0x7fd2080afa50 2024-04-25T00:40:15.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd204ff9640 1 -- 172.21.15.29:0/1056629515 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0004450 con 0x7fd2080afa50 2024-04-25T00:40:15.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd20de1b640 1 -- 172.21.15.29:0/1056629515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2080cf2b0 con 0x7fd2080afa50 2024-04-25T00:40:15.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.673+0000 7fd20de1b640 1 -- 172.21.15.29:0/1056629515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd208069270 con 0x7fd2080afa50 2024-04-25T00:40:15.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.677+0000 7fd204ff9640 1 -- 172.21.15.29:0/1056629515 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fd1f0018070 con 0x7fd2080afa50 2024-04-25T00:40:15.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.677+0000 7fd204ff9640 1 --2- 172.21.15.29:0/1056629515 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd1e80680a0 0x7fd1e806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.677+0000 7fd2077fe640 1 --2- 172.21.15.29:0/1056629515 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd1e80680a0 0x7fd1e806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.677+0000 7fd204ff9640 1 -- 172.21.15.29:0/1056629515 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fd1f00869e0 con 0x7fd2080afa50 2024-04-25T00:40:15.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.677+0000 7fd2077fe640 1 --2- 172.21.15.29:0/1056629515 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd1e80680a0 0x7fd1e806a560 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80ae000 tx=0x7fd1f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:15.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.681+0000 7fd204ff9640 1 -- 172.21.15.29:0/1056629515 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd1f0056590 con 0x7fd2080afa50 2024-04-25T00:40:15.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd20de1b640 1 -- 172.21.15.29:0/1056629515 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd208155640 con 0x7fd2080afa50 2024-04-25T00:40:15.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd204ff9640 1 -- 172.21.15.29:0/1056629515 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) ==== 100+0+11 (secure 0 0 0) 0x7fd1f0055e90 con 0x7fd2080afa50 2024-04-25T00:40:15.840 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:40:15.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd20de1b640 1 -- 172.21.15.29:0/1056629515 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd1e80680a0 msgr2=0x7fd1e806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd20de1b640 1 --2- 172.21.15.29:0/1056629515 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd1e80680a0 0x7fd1e806a560 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80ae000 tx=0x7fd1f8008040 comp rx=0 tx=0).stop 2024-04-25T00:40:15.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd20de1b640 1 -- 172.21.15.29:0/1056629515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2080afa50 msgr2=0x7fd2080ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd20de1b640 1 --2- 172.21.15.29:0/1056629515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2080afa50 0x7fd2080ce1e0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000b410 tx=0x7fd1f000b8d0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd20de1b640 1 -- 172.21.15.29:0/1056629515 shutdown_connections 2024-04-25T00:40:15.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd20de1b640 1 --2- 172.21.15.29:0/1056629515 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd1e80680a0 0x7fd1e806a560 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd20de1b640 1 --2- 172.21.15.29:0/1056629515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20815e410 0x7fd208152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd20de1b640 1 --2- 172.21.15.29:0/1056629515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2080afa50 0x7fd2080ce1e0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd20de1b640 1 --2- 172.21.15.29:0/1056629515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd20806bc20 0x7fd2080cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd20de1b640 1 -- 172.21.15.29:0/1056629515 >> 172.21.15.29:0/1056629515 conn(0x7fd2080d0c90 msgr2=0x7fd2080afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:15.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd20de1b640 1 -- 172.21.15.29:0/1056629515 shutdown_connections 2024-04-25T00:40:15.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.841+0000 7fd20de1b640 1 -- 172.21.15.29:0/1056629515 wait complete. 2024-04-25T00:40:15.860 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:40:15.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.973+0000 7f8c09527640 1 Processor -- start 2024-04-25T00:40:15.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.973+0000 7f8c09527640 1 -- start start 2024-04-25T00:40:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.973+0000 7f8c09527640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8c0415bdb0 0x7f8c040cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.973+0000 7f8c09527640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c040cd3e0 con 0x7f8c040d09e0 2024-04-25T00:40:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.973+0000 7f8c09527640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c040cd6c0 con 0x7f8c0415c190 2024-04-25T00:40:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.973+0000 7f8c09527640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8c040cd9a0 con 0x7f8c0415bdb0 2024-04-25T00:40:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.973+0000 7f8c02ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8c0415bdb0 0x7f8c040cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.973+0000 7f8c02ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8c0415bdb0 0x7f8c040cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52412/0 (socket says 172.21.15.29:52412) 2024-04-25T00:40:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.973+0000 7f8c02ffd640 1 -- 172.21.15.29:0/2040882916 learned_addr learned my addr 172.21.15.29:0/2040882916 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.973+0000 7f8c02ffd640 1 -- 172.21.15.29:0/2040882916 >> v1:172.21.15.157:6789/0 conn(0x7f8c0415c190 legacy=0x7f8c040d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.973+0000 7f8c02ffd640 1 -- 172.21.15.29:0/2040882916 >> v1:172.21.15.29:6789/0 conn(0x7f8c040d09e0 legacy=0x7f8c040c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.973+0000 7f8c02ffd640 1 -- 172.21.15.29:0/2040882916 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c040cdc80 con 0x7f8c0415bdb0 2024-04-25T00:40:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c02ffd640 1 --2- 172.21.15.29:0/2040882916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c0415bdb0 0x7f8c040cc820 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009870 tx=0x7f8bec01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dab9659219446e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c01ffb640 1 -- 172.21.15.29:0/2040882916 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0b0070 con 0x7f8c0415bdb0 2024-04-25T00:40:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c01ffb640 1 -- 172.21.15.29:0/2040882916 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bec0ab600 con 0x7f8c0415bdb0 2024-04-25T00:40:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c01ffb640 1 -- 172.21.15.29:0/2040882916 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0ab9d0 con 0x7f8c0415bdb0 2024-04-25T00:40:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 -- 172.21.15.29:0/2040882916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c0415bdb0 msgr2=0x7f8c040cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 --2- 172.21.15.29:0/2040882916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c0415bdb0 0x7f8c040cc820 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009870 tx=0x7f8bec01c460 comp rx=0 tx=0).stop 2024-04-25T00:40:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 -- 172.21.15.29:0/2040882916 shutdown_connections 2024-04-25T00:40:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 --2- 172.21.15.29:0/2040882916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c0415bdb0 0x7f8c040cc820 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 -- 172.21.15.29:0/2040882916 >> 172.21.15.29:0/2040882916 conn(0x7f8c0409aa30 msgr2=0x7f8c0409ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 -- 172.21.15.29:0/2040882916 shutdown_connections 2024-04-25T00:40:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 -- 172.21.15.29:0/2040882916 wait complete. 2024-04-25T00:40:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 Processor -- start 2024-04-25T00:40:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 -- start start 2024-04-25T00:40:15.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c040d09e0 0x7f8c040b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c0415bdb0 0x7f8c040b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c0414ded0 0x7f8c04157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8c040c6b10 con 0x7f8c040d09e0 2024-04-25T00:40:15.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c040c6990 con 0x7f8c0415bdb0 2024-04-25T00:40:15.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c09527640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8c04169ae0 con 0x7f8c0414ded0 2024-04-25T00:40:15.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c02ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c040d09e0 0x7f8c040b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c02ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c040d09e0 0x7f8c040b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52428/0 (socket says 172.21.15.29:52428) 2024-04-25T00:40:15.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c02ffd640 1 -- 172.21.15.29:0/2232745129 learned_addr learned my addr 172.21.15.29:0/2232745129 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:15.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c027fc640 1 --2- 172.21.15.29:0/2232745129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c0415bdb0 0x7f8c040b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c02ffd640 1 -- 172.21.15.29:0/2232745129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c0414ded0 msgr2=0x7f8c04157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c02ffd640 1 --2- 172.21.15.29:0/2232745129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c0414ded0 0x7f8c04157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c02ffd640 1 -- 172.21.15.29:0/2232745129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c0415bdb0 msgr2=0x7f8c040b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:15.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c02ffd640 1 --2- 172.21.15.29:0/2232745129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c0415bdb0 0x7f8c040b6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:15.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.977+0000 7f8c02ffd640 1 -- 172.21.15.29:0/2232745129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c040b7350 con 0x7f8c040d09e0 2024-04-25T00:40:15.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.981+0000 7f8c02ffd640 1 --2- 172.21.15.29:0/2232745129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c040d09e0 0x7f8c040b4870 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0ae040 tx=0x7f8bec01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:15.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.981+0000 7f8bfb7fe640 1 -- 172.21.15.29:0/2232745129 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0ba020 con 0x7f8c040d09e0 2024-04-25T00:40:15.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.981+0000 7f8bfb7fe640 1 -- 172.21.15.29:0/2232745129 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bec0b6640 con 0x7f8c040d09e0 2024-04-25T00:40:15.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.981+0000 7f8c09527640 1 -- 172.21.15.29:0/2232745129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c040b7630 con 0x7f8c040d09e0 2024-04-25T00:40:15.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.981+0000 7f8bfb7fe640 1 -- 172.21.15.29:0/2232745129 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0b0040 con 0x7f8c040d09e0 2024-04-25T00:40:15.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.981+0000 7f8c09527640 1 -- 172.21.15.29:0/2232745129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c04151230 con 0x7f8c040d09e0 2024-04-25T00:40:15.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.981+0000 7f8c09527640 1 -- 172.21.15.29:0/2232745129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c04069270 con 0x7f8c040d09e0 2024-04-25T00:40:15.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.981+0000 7f8bfb7fe640 1 -- 172.21.15.29:0/2232745129 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f8bec0bd020 con 0x7f8c040d09e0 2024-04-25T00:40:15.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.981+0000 7f8bfb7fe640 1 --2- 172.21.15.29:0/2232745129 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8be80680d0 0x7f8be806a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.981+0000 7f8c027fc640 1 --2- 172.21.15.29:0/2232745129 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8be80680d0 0x7f8be806a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.981+0000 7f8bfb7fe640 1 -- 172.21.15.29:0/2232745129 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f8bec11e980 con 0x7f8c040d09e0 2024-04-25T00:40:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.981+0000 7f8c027fc640 1 --2- 172.21.15.29:0/2232745129 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8be80680d0 0x7f8be806a590 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8c041f8490 tx=0x7f8bfc0072d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:15.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:15.985+0000 7f8bfb7fe640 1 -- 172.21.15.29:0/2232745129 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8bec0c4030 con 0x7f8c040d09e0 2024-04-25T00:40:16.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.173+0000 7f8c09527640 1 -- 172.21.15.29:0/2232745129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c04151400 con 0x7f8c040d09e0 2024-04-25T00:40:16.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.173+0000 7f8bfb7fe640 1 -- 172.21.15.29:0/2232745129 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5185 (secure 0 0 0) 0x7f8bec016110 con 0x7f8c040d09e0 2024-04-25T00:40:16.175 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:16.175 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":93,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3075204329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3075204329},{"type":"v1","addr":"172.21.15.157:6833","nonce":3075204329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3377772964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3377772964},{"type":"v1","addr":"172.21.15.157:6835","nonce":3377772964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":9621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/897131811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":897131811},{"type":"v1","addr":"172.21.15.29:6833","nonce":897131811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:39:48.556647+0000","modified":"2024-04-25T00:39:51.600640+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9624":{"gid":9624,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/821427804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":821427804},{"type":"v1","addr":"172.21.15.29:6835","nonce":821427804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9624,"qdb_cluster":[9624]},"id":11}]} 2024-04-25T00:40:16.175 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 93 2024-04-25T00:40:16.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.177+0000 7f8c09527640 1 -- 172.21.15.29:0/2232745129 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8be80680d0 msgr2=0x7f8be806a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:16.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.177+0000 7f8c09527640 1 --2- 172.21.15.29:0/2232745129 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8be80680d0 0x7f8be806a590 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8c041f8490 tx=0x7f8bfc0072d0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.177+0000 7f8c09527640 1 -- 172.21.15.29:0/2232745129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c040d09e0 msgr2=0x7f8c040b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:16.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.177+0000 7f8c09527640 1 --2- 172.21.15.29:0/2232745129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c040d09e0 0x7f8c040b4870 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0ae040 tx=0x7f8bec01aca0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.177+0000 7f8c09527640 1 -- 172.21.15.29:0/2232745129 shutdown_connections 2024-04-25T00:40:16.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.177+0000 7f8c09527640 1 --2- 172.21.15.29:0/2232745129 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8be80680d0 0x7f8be806a590 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.177+0000 7f8c09527640 1 --2- 172.21.15.29:0/2232745129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c0414ded0 0x7f8c04157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.177+0000 7f8c09527640 1 --2- 172.21.15.29:0/2232745129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c0415bdb0 0x7f8c040b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.177+0000 7f8c09527640 1 --2- 172.21.15.29:0/2232745129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c040d09e0 0x7f8c040b4870 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.177+0000 7f8c09527640 1 -- 172.21.15.29:0/2232745129 >> 172.21.15.29:0/2232745129 conn(0x7f8c0409aa30 msgr2=0x7f8c041595b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:16.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.177+0000 7f8c09527640 1 -- 172.21.15.29:0/2232745129 shutdown_connections 2024-04-25T00:40:16.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.177+0000 7f8c09527640 1 -- 172.21.15.29:0/2232745129 wait complete. 2024-04-25T00:40:16.193 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:40:16.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc8ecee640 1 Processor -- start 2024-04-25T00:40:16.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc8ecee640 1 -- start start 2024-04-25T00:40:16.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc8ecee640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbc88162420 0x7fbc88070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc8ecee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbc880b4e00 0x7fbc880b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc8ecee640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc880b30b0 0x7fbc88166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc8ecee640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbc880b3500 con 0x7fbc880b4e00 2024-04-25T00:40:16.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc8ecee640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbc88071910 con 0x7fbc88162420 2024-04-25T00:40:16.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc8ecee640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbc88071bf0 con 0x7fbc880b30b0 2024-04-25T00:40:16.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc87fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbc880b4e00 0x7fbc880b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc87fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbc880b4e00 0x7fbc880b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52438/0 (socket says 172.21.15.29:52438) 2024-04-25T00:40:16.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc87fff640 1 -- 172.21.15.29:0/2404212261 learned_addr learned my addr 172.21.15.29:0/2404212261 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:16.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc8ca8a640 1 --2- 172.21.15.29:0/2404212261 >> v2:172.21.15.157:3300/0 conn(0x7fbc88162420 0x7fbc88070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc8d28b640 1 --2- 172.21.15.29:0/2404212261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc880b30b0 0x7fbc88166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc87fff640 1 -- 172.21.15.29:0/2404212261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc880b30b0 msgr2=0x7fbc88166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:16.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc87fff640 1 --2- 172.21.15.29:0/2404212261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc880b30b0 0x7fbc88166020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc87fff640 1 -- 172.21.15.29:0/2404212261 >> v2:172.21.15.157:3300/0 conn(0x7fbc88162420 msgr2=0x7fbc88070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:16.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc87fff640 1 --2- 172.21.15.29:0/2404212261 >> v2:172.21.15.157:3300/0 conn(0x7fbc88162420 0x7fbc88070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc87fff640 1 -- 172.21.15.29:0/2404212261 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc88071ed0 con 0x7fbc880b4e00 2024-04-25T00:40:16.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.313+0000 7fbc8ca8a640 1 --2- 172.21.15.29:0/2404212261 >> v2:172.21.15.157:3300/0 conn(0x7fbc88162420 0x7fbc88070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:16.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc87fff640 1 --2- 172.21.15.29:0/2404212261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc880b4e00 0x7fbc880b2b70 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fbc88067e70 tx=0x7fbc7c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4afc224ab299294 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:16.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc877fe640 1 -- 172.21.15.29:0/2404212261 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0b0070 con 0x7fbc880b4e00 2024-04-25T00:40:16.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc877fe640 1 -- 172.21.15.29:0/2404212261 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc7c0ab600 con 0x7fbc880b4e00 2024-04-25T00:40:16.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc877fe640 1 -- 172.21.15.29:0/2404212261 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0ab9d0 con 0x7fbc880b4e00 2024-04-25T00:40:16.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc8ecee640 1 -- 172.21.15.29:0/2404212261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc880b4e00 msgr2=0x7fbc880b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:16.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc8ecee640 1 --2- 172.21.15.29:0/2404212261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc880b4e00 0x7fbc880b2b70 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fbc88067e70 tx=0x7fbc7c01c260 comp rx=0 tx=0).stop 2024-04-25T00:40:16.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc8ecee640 1 -- 172.21.15.29:0/2404212261 shutdown_connections 2024-04-25T00:40:16.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc8ecee640 1 --2- 172.21.15.29:0/2404212261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc880b4e00 0x7fbc880b2b70 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc8ecee640 1 --2- 172.21.15.29:0/2404212261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc880b30b0 0x7fbc88166020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc8ecee640 1 --2- 172.21.15.29:0/2404212261 >> v2:172.21.15.157:3300/0 conn(0x7fbc88162420 0x7fbc88070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc8ecee640 1 -- 172.21.15.29:0/2404212261 >> 172.21.15.29:0/2404212261 conn(0x7fbc880b1870 msgr2=0x7fbc880b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:16.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc8ecee640 1 -- 172.21.15.29:0/2404212261 shutdown_connections 2024-04-25T00:40:16.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc8ecee640 1 -- 172.21.15.29:0/2404212261 wait complete. 2024-04-25T00:40:16.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc8ecee640 1 Processor -- start 2024-04-25T00:40:16.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.317+0000 7fbc8ecee640 1 -- start start 2024-04-25T00:40:16.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ecee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc880b30b0 0x7fbc88145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ecee640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc880b4e00 0x7fbc88158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ecee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc88162420 0x7fbc88156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ecee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8806cf60 con 0x7fbc880b30b0 2024-04-25T00:40:16.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ecee640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbc880716d0 con 0x7fbc88162420 2024-04-25T00:40:16.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ecee640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbc8806d0e0 con 0x7fbc880b4e00 2024-04-25T00:40:16.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ca8a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc880b30b0 0x7fbc88145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ca8a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc880b30b0 0x7fbc88145c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52440/0 (socket says 172.21.15.29:52440) 2024-04-25T00:40:16.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ca8a640 1 -- 172.21.15.29:0/1688986823 learned_addr learned my addr 172.21.15.29:0/1688986823 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:16.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ca8a640 1 -- 172.21.15.29:0/1688986823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc880b4e00 msgr2=0x7fbc88158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:16.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc87fff640 1 --2- 172.21.15.29:0/1688986823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc880b4e00 0x7fbc88158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ca8a640 1 --2- 172.21.15.29:0/1688986823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc880b4e00 0x7fbc88158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ca8a640 1 -- 172.21.15.29:0/1688986823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc88162420 msgr2=0x7fbc88156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:16.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ca8a640 1 --2- 172.21.15.29:0/1688986823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc88162420 0x7fbc88156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ca8a640 1 -- 172.21.15.29:0/1688986823 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc88159710 con 0x7fbc880b30b0 2024-04-25T00:40:16.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ca8a640 1 --2- 172.21.15.29:0/1688986823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc880b30b0 0x7fbc88145c70 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800c7f0 tx=0x7fbc7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:16.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc857fa640 1 -- 172.21.15.29:0/1688986823 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78015070 con 0x7fbc880b30b0 2024-04-25T00:40:16.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc857fa640 1 -- 172.21.15.29:0/1688986823 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc78007820 con 0x7fbc880b30b0 2024-04-25T00:40:16.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc857fa640 1 -- 172.21.15.29:0/1688986823 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78007c10 con 0x7fbc880b30b0 2024-04-25T00:40:16.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ecee640 1 -- 172.21.15.29:0/1688986823 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc881599f0 con 0x7fbc880b30b0 2024-04-25T00:40:16.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ecee640 1 -- 172.21.15.29:0/1688986823 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc88159fb0 con 0x7fbc880b30b0 2024-04-25T00:40:16.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc8ecee640 1 -- 172.21.15.29:0/1688986823 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc88069270 con 0x7fbc880b30b0 2024-04-25T00:40:16.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc857fa640 1 -- 172.21.15.29:0/1688986823 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fbc78018070 con 0x7fbc880b30b0 2024-04-25T00:40:16.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc857fa640 1 --2- 172.21.15.29:0/1688986823 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc740680a0 0x7fbc7406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc857fa640 1 -- 172.21.15.29:0/1688986823 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fbc78087190 con 0x7fbc880b30b0 2024-04-25T00:40:16.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.321+0000 7fbc87fff640 1 --2- 172.21.15.29:0/1688986823 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc740680a0 0x7fbc7406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.325+0000 7fbc87fff640 1 --2- 172.21.15.29:0/1688986823 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc740680a0 0x7fbc7406a560 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0ad000 tx=0x7fbc7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:16.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.325+0000 7fbc857fa640 1 -- 172.21.15.29:0/1688986823 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbc78056c30 con 0x7fbc880b30b0 2024-04-25T00:40:16.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.485+0000 7fbc8ecee640 1 -- 172.21.15.29:0/1688986823 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc881461b0 con 0x7fbc880b30b0 2024-04-25T00:40:16.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.489+0000 7fbc857fa640 1 -- 172.21.15.29:0/1688986823 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16623 (secure 0 0 0) 0x7fbc78056530 con 0x7fbc880b30b0 2024-04-25T00:40:16.689 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:16.689 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":145,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:50.179122+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:46.824888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:47.550815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:40:16.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.489+0000 7fbc8ecee640 1 -- 172.21.15.29:0/1688986823 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc740680a0 msgr2=0x7fbc7406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:16.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.489+0000 7fbc8ecee640 1 --2- 172.21.15.29:0/1688986823 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc740680a0 0x7fbc7406a560 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0ad000 tx=0x7fbc7c008040 comp rx=0 tx=0).stop 2024-04-25T00:40:16.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.489+0000 7fbc8ecee640 1 -- 172.21.15.29:0/1688986823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc880b30b0 msgr2=0x7fbc88145c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:16.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.489+0000 7fbc8ecee640 1 --2- 172.21.15.29:0/1688986823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc880b30b0 0x7fbc88145c70 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800c7f0 tx=0x7fbc7800ccb0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.489+0000 7fbc8ecee640 1 -- 172.21.15.29:0/1688986823 shutdown_connections 2024-04-25T00:40:16.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.489+0000 7fbc8ecee640 1 --2- 172.21.15.29:0/1688986823 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbc740680a0 0x7fbc7406a560 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.489+0000 7fbc8ecee640 1 --2- 172.21.15.29:0/1688986823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc88162420 0x7fbc88156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.489+0000 7fbc8ecee640 1 --2- 172.21.15.29:0/1688986823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc880b4e00 0x7fbc88158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.489+0000 7fbc8ecee640 1 --2- 172.21.15.29:0/1688986823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc880b30b0 0x7fbc88145c70 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.489+0000 7fbc8ecee640 1 -- 172.21.15.29:0/1688986823 >> 172.21.15.29:0/1688986823 conn(0x7fbc880b1870 msgr2=0x7fbc8815b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:16.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.489+0000 7fbc8ecee640 1 -- 172.21.15.29:0/1688986823 shutdown_connections 2024-04-25T00:40:16.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.489+0000 7fbc8ecee640 1 -- 172.21.15.29:0/1688986823 wait complete. 2024-04-25T00:40:16.692 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:40:16.692 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:40:16.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f7372a19640 1 Processor -- start 2024-04-25T00:40:16.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f7372a19640 1 -- start start 2024-04-25T00:40:16.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f7372a19640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f736c158af0 0x7f736c0c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f7372a19640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f736c0725d0 0x7f736c0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f7372a19640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f736c158ed0 con 0x7f736c0c70b0 2024-04-25T00:40:16.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f7372a19640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f736c15f650 con 0x7f736c158af0 2024-04-25T00:40:16.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f7372a19640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f736c15ed20 con 0x7f736c0725d0 2024-04-25T00:40:16.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f73637fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f736c0c70b0 0x7f736c071ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59138/0 (socket says 172.21.15.29:59138) 2024-04-25T00:40:16.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f73637fe640 1 -- 172.21.15.29:0/2343064144 learned_addr learned my addr 172.21.15.29:0/2343064144 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:16.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f7370fb6640 1 --2- 172.21.15.29:0/2343064144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f736c0725d0 0x7f736c0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736bfff640 1 --2- 172.21.15.29:0/2343064144 >> v2:172.21.15.157:3300/0 conn(0x7f736c158af0 0x7f736c0c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1675067630 0 0) 0x7f7354001120 con 0x7f736c0c70b0 2024-04-25T00:40:16.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7358003530 con 0x7f736c0c70b0 2024-04-25T00:40:16.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3970717382 0 0) 0x7f7358003530 con 0x7f736c0c70b0 2024-04-25T00:40:16.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f736c0725d0 msgr2=0x7f736c0cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:16.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 --2- 172.21.15.29:0/2343064144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f736c0725d0 0x7f736c0cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 >> v2:172.21.15.157:3300/0 conn(0x7f736c158af0 msgr2=0x7f736c0c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:16.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 --2- 172.21.15.29:0/2343064144 >> v2:172.21.15.157:3300/0 conn(0x7f736c158af0 0x7f736c0c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736c15f000 con 0x7f736c0c70b0 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f7354001850 con 0x7f736c0c70b0 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 >> v1:172.21.15.29:6789/0 conn(0x7f736c0c70b0 legacy=0x7f736c071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 --2- 172.21.15.29:0/2343064144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73580039c0 0x7f7358003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 --2- 172.21.15.29:0/2343064144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7358004810 0x7f7358004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 --2- 172.21.15.29:0/2343064144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f736c0725d0 0x7f73580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f736c15f650 con 0x7f7358004810 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f736c15ed20 con 0x7f73580039c0 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f735800b9d0 con 0x7f736c0725d0 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f73637fe640 1 --2- 172.21.15.29:0/2343064144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7358004810 0x7f7358004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f7370fb6640 1 --2- 172.21.15.29:0/2343064144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f736c0725d0 0x7f73580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f736bfff640 1 --2- 172.21.15.29:0/2343064144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73580039c0 0x7f7358003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f73637fe640 1 -- 172.21.15.29:0/2343064144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f736c0725d0 msgr2=0x7f73580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f73637fe640 1 --2- 172.21.15.29:0/2343064144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f736c0725d0 0x7f73580051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f73637fe640 1 -- 172.21.15.29:0/2343064144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73580039c0 msgr2=0x7f7358003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:16.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f73637fe640 1 --2- 172.21.15.29:0/2343064144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73580039c0 0x7f7358003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.825+0000 7f73637fe640 1 -- 172.21.15.29:0/2343064144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7358003530 con 0x7f7358004810 2024-04-25T00:40:16.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.829+0000 7f73637fe640 1 --2- 172.21.15.29:0/2343064144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7358004810 0x7f7358004c90 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f735400c4c0 tx=0x7f735401f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb72130e2ed0074 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:16.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.829+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735400c0e0 con 0x7f7358004810 2024-04-25T00:40:16.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.829+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7354005b00 con 0x7f7358004810 2024-04-25T00:40:16.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.829+0000 7f736b7fe640 1 -- 172.21.15.29:0/2343064144 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7354007070 con 0x7f7358004810 2024-04-25T00:40:16.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.829+0000 7f7372a19640 1 -- 172.21.15.29:0/2343064144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7358004810 msgr2=0x7f7358004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:16.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.829+0000 7f7372a19640 1 --2- 172.21.15.29:0/2343064144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7358004810 0x7f7358004c90 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f735400c4c0 tx=0x7f735401f440 comp rx=0 tx=0).stop 2024-04-25T00:40:16.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.829+0000 7f7370fb6640 1 -- 172.21.15.29:0/2343064144 reap_dead start 2024-04-25T00:40:16.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.829+0000 7f7372a19640 1 -- 172.21.15.29:0/2343064144 shutdown_connections 2024-04-25T00:40:16.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.829+0000 7f7372a19640 1 -- 172.21.15.29:0/2343064144 >> 172.21.15.29:0/2343064144 conn(0x7f736c0d0c50 msgr2=0x7f736c0d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:16.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.829+0000 7f7372a19640 1 -- 172.21.15.29:0/2343064144 shutdown_connections 2024-04-25T00:40:16.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.829+0000 7f7372a19640 1 -- 172.21.15.29:0/2343064144 wait complete. 2024-04-25T00:40:16.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f7372a19640 1 Processor -- start 2024-04-25T00:40:16.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f7372a19640 1 -- start start 2024-04-25T00:40:16.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f7372a19640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7358004810 0x7f736c0b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f7372a19640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f736c150970 0x7f736c14e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f7372a19640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f736c14d380 0x7f736c151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f7372a19640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f736c06b390 con 0x7f736c150970 2024-04-25T00:40:16.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f7372a19640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f736c06b210 con 0x7f736c14d380 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f7372a19640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f736c06b510 con 0x7f7358004810 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f73637fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f736c150970 0x7f736c14e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f73637fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f736c150970 0x7f736c14e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52456/0 (socket says 172.21.15.29:52456) 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f73637fe640 1 -- 172.21.15.29:0/4068829768 learned_addr learned my addr 172.21.15.29:0/4068829768 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f736bfff640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7358004810 0x7f736c0b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f7370fb6640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f736c14d380 0x7f736c151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f73637fe640 1 -- 172.21.15.29:0/4068829768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7358004810 msgr2=0x7f736c0b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f73637fe640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7358004810 0x7f736c0b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f73637fe640 1 -- 172.21.15.29:0/4068829768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f736c14d380 msgr2=0x7f736c151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f73637fe640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f736c14d380 0x7f736c151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f73637fe640 1 -- 172.21.15.29:0/4068829768 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736c0b4730 con 0x7f736c150970 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f73637fe640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f736c150970 0x7f736c14e240 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f735400c4c0 tx=0x7f7358005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f73697fa640 1 -- 172.21.15.29:0/4068829768 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7354005d70 con 0x7f736c150970 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f73697fa640 1 -- 172.21.15.29:0/4068829768 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7354001610 con 0x7f736c150970 2024-04-25T00:40:16.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f7372a19640 1 -- 172.21.15.29:0/4068829768 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f736c151130 con 0x7f736c150970 2024-04-25T00:40:16.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f73697fa640 1 -- 172.21.15.29:0/4068829768 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73540019e0 con 0x7f736c150970 2024-04-25T00:40:16.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f7372a19640 1 -- 172.21.15.29:0/4068829768 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f736c1516f0 con 0x7f736c150970 2024-04-25T00:40:16.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f7372a19640 1 -- 172.21.15.29:0/4068829768 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f736c13af90 con 0x7f736c150970 2024-04-25T00:40:16.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.833+0000 7f73697fa640 1 -- 172.21.15.29:0/4068829768 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f73540ba030 con 0x7f736c150970 2024-04-25T00:40:16.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.837+0000 7f73697fa640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f734c0680d0 0x7f734c06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:16.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.837+0000 7f736bfff640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f734c0680d0 0x7f734c06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:16.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.837+0000 7f73697fa640 1 -- 172.21.15.29:0/4068829768 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f7354120c10 con 0x7f736c150970 2024-04-25T00:40:16.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.837+0000 7f736bfff640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f734c0680d0 0x7f734c06a590 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f736c14d330 tx=0x7f735c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:16.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:16.841+0000 7f73697fa640 1 -- 172.21.15.29:0/4068829768 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f736c158ed0 con 0x7f736c150970 2024-04-25T00:40:17.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.029+0000 7f7372a19640 1 -- 172.21.15.29:0/4068829768 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f736c069270 con 0x7f736c150970 2024-04-25T00:40:17.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.029+0000 7f73697fa640 1 -- 172.21.15.29:0/4068829768 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5185 (secure 0 0 0) 0x7f73540f3040 con 0x7f736c150970 2024-04-25T00:40:17.030 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:17.030 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":93,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3075204329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3075204329},{"type":"v1","addr":"172.21.15.157:6833","nonce":3075204329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3377772964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3377772964},{"type":"v1","addr":"172.21.15.157:6835","nonce":3377772964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":9621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/897131811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":897131811},{"type":"v1","addr":"172.21.15.29:6833","nonce":897131811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:39:48.556647+0000","modified":"2024-04-25T00:39:51.600640+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9624":{"gid":9624,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/821427804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":821427804},{"type":"v1","addr":"172.21.15.29:6835","nonce":821427804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9624,"qdb_cluster":[9624]},"id":11}]} 2024-04-25T00:40:17.030 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 93 2024-04-25T00:40:17.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.033+0000 7f7372a19640 1 -- 172.21.15.29:0/4068829768 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f734c0680d0 msgr2=0x7f734c06a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.033+0000 7f7372a19640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f734c0680d0 0x7f734c06a590 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f736c14d330 tx=0x7f735c008040 comp rx=0 tx=0).stop 2024-04-25T00:40:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.033+0000 7f7372a19640 1 -- 172.21.15.29:0/4068829768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f736c150970 msgr2=0x7f736c14e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.033+0000 7f7372a19640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f736c150970 0x7f736c14e240 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f735400c4c0 tx=0x7f7358005840 comp rx=0 tx=0).stop 2024-04-25T00:40:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.033+0000 7f7372a19640 1 -- 172.21.15.29:0/4068829768 shutdown_connections 2024-04-25T00:40:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.033+0000 7f7372a19640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f734c0680d0 0x7f734c06a590 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.033+0000 7f7372a19640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f736c14d380 0x7f736c151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.033+0000 7f7372a19640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f736c150970 0x7f736c14e240 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.033+0000 7f7372a19640 1 --2- 172.21.15.29:0/4068829768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7358004810 0x7f736c0b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.033+0000 7f7372a19640 1 -- 172.21.15.29:0/4068829768 >> 172.21.15.29:0/4068829768 conn(0x7f736c0d0c50 msgr2=0x7f736c0b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.033+0000 7f7372a19640 1 -- 172.21.15.29:0/4068829768 shutdown_connections 2024-04-25T00:40:17.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.033+0000 7f7372a19640 1 -- 172.21.15.29:0/4068829768 wait complete. 2024-04-25T00:40:17.048 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:40:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.157+0000 7f98bb7b2640 1 Processor -- start 2024-04-25T00:40:17.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.157+0000 7f98bb7b2640 1 -- start start 2024-04-25T00:40:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98bb7b2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98b415e200 0x7f98b40d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98bb7b2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f98b40d2c60 0x7f98b40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98bb7b2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98b40cfc10 con 0x7f98b40cf690 2024-04-25T00:40:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98bb7b2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f98b40b38d0 con 0x7f98b40d2c60 2024-04-25T00:40:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98bb7b2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f98b40cfef0 con 0x7f98b415e200 2024-04-25T00:40:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98b8d4d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f98b40cf690 0x7f98b40d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59140/0 (socket says 172.21.15.29:59140) 2024-04-25T00:40:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98b8d4d640 1 -- 172.21.15.29:0/1229217881 learned_addr learned my addr 172.21.15.29:0/1229217881 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98b9d4f640 1 --2- 172.21.15.29:0/1229217881 >> v2:172.21.15.29:3300/0 conn(0x7f98b40d2c60 0x7f98b40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98b954e640 1 --2- 172.21.15.29:0/1229217881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98b415e200 0x7f98b40d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98b9d4f640 1 -- 172.21.15.29:0/1229217881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98b415e200 msgr2=0x7f98b40d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:17.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98b9d4f640 1 --2- 172.21.15.29:0/1229217881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98b415e200 0x7f98b40d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98b9d4f640 1 -- 172.21.15.29:0/1229217881 >> v1:172.21.15.29:6789/0 conn(0x7f98b40cf690 legacy=0x7f98b40d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98b9d4f640 1 -- 172.21.15.29:0/1229217881 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b40b3b20 con 0x7f98b40d2c60 2024-04-25T00:40:17.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98b9d4f640 1 --2- 172.21.15.29:0/1229217881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98b40d2c60 0x7f98b40b4320 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009d90 tx=0x7f98ac01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c713d91533d5dbc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:17.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98abfff640 1 -- 172.21.15.29:0/1229217881 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0b2070 con 0x7f98b40d2c60 2024-04-25T00:40:17.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98abfff640 1 -- 172.21.15.29:0/1229217881 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac0a5a20 con 0x7f98b40d2c60 2024-04-25T00:40:17.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98abfff640 1 -- 172.21.15.29:0/1229217881 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0a5df0 con 0x7f98b40d2c60 2024-04-25T00:40:17.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98bb7b2640 1 -- 172.21.15.29:0/1229217881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98b40d2c60 msgr2=0x7f98b40b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98bb7b2640 1 --2- 172.21.15.29:0/1229217881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98b40d2c60 0x7f98b40b4320 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009d90 tx=0x7f98ac01c940 comp rx=0 tx=0).stop 2024-04-25T00:40:17.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98bb7b2640 1 -- 172.21.15.29:0/1229217881 shutdown_connections 2024-04-25T00:40:17.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98bb7b2640 1 --2- 172.21.15.29:0/1229217881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98b40d2c60 0x7f98b40b4320 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98bb7b2640 1 --2- 172.21.15.29:0/1229217881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98b415e200 0x7f98b40d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98bb7b2640 1 -- 172.21.15.29:0/1229217881 >> 172.21.15.29:0/1229217881 conn(0x7f98b409d280 msgr2=0x7f98b40d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:17.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98bb7b2640 1 -- 172.21.15.29:0/1229217881 shutdown_connections 2024-04-25T00:40:17.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.161+0000 7f98bb7b2640 1 -- 172.21.15.29:0/1229217881 wait complete. 2024-04-25T00:40:17.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98bb7b2640 1 Processor -- start 2024-04-25T00:40:17.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98bb7b2640 1 -- start start 2024-04-25T00:40:17.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98bb7b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98b40cf690 0x7f98b4155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98bb7b2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98b40d2c60 0x7f98b40c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98bb7b2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98b415e200 0x7f98b40b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98bb7b2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f98b40b51f0 con 0x7f98b40cf690 2024-04-25T00:40:17.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98bb7b2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f98b40b5070 con 0x7f98b415e200 2024-04-25T00:40:17.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98bb7b2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f98b40b5370 con 0x7f98b40d2c60 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98b954e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98b40cf690 0x7f98b4155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98b954e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98b40cf690 0x7f98b4155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52480/0 (socket says 172.21.15.29:52480) 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98b954e640 1 -- 172.21.15.29:0/3670570261 learned_addr learned my addr 172.21.15.29:0/3670570261 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98b8d4d640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98b40d2c60 0x7f98b40c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98b9d4f640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98b415e200 0x7f98b40b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98b954e640 1 -- 172.21.15.29:0/3670570261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98b40d2c60 msgr2=0x7f98b40c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98b954e640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98b40d2c60 0x7f98b40c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98b954e640 1 -- 172.21.15.29:0/3670570261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98b415e200 msgr2=0x7f98b40b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98b954e640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98b415e200 0x7f98b40b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98b954e640 1 -- 172.21.15.29:0/3670570261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b40c91e0 con 0x7f98b40cf690 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98b954e640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98b40cf690 0x7f98b4155380 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f98b000d5f0 tx=0x7f98b000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98a9ffb640 1 -- 172.21.15.29:0/3670570261 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b00125f0 con 0x7f98b40cf690 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98a9ffb640 1 -- 172.21.15.29:0/3670570261 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b0019070 con 0x7f98b40cf690 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98bb7b2640 1 -- 172.21.15.29:0/3670570261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b40c94c0 con 0x7f98b40cf690 2024-04-25T00:40:17.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98a9ffb640 1 -- 172.21.15.29:0/3670570261 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b00129d0 con 0x7f98b40cf690 2024-04-25T00:40:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98bb7b2640 1 -- 172.21.15.29:0/3670570261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b40c9a80 con 0x7f98b40cf690 2024-04-25T00:40:17.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.165+0000 7f98bb7b2640 1 -- 172.21.15.29:0/3670570261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b4069270 con 0x7f98b40cf690 2024-04-25T00:40:17.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.169+0000 7f98a9ffb640 1 -- 172.21.15.29:0/3670570261 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f98b0012b70 con 0x7f98b40cf690 2024-04-25T00:40:17.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.169+0000 7f98a9ffb640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f98980680a0 0x7f989806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.169+0000 7f98a9ffb640 1 -- 172.21.15.29:0/3670570261 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f98b0087970 con 0x7f98b40cf690 2024-04-25T00:40:17.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.169+0000 7f98b8d4d640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f98980680a0 0x7f989806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.169+0000 7f98b8d4d640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f98980680a0 0x7f989806a560 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f98b40b6890 tx=0x7f98a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:17.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.173+0000 7f98a9ffb640 1 -- 172.21.15.29:0/3670570261 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f98b0057a70 con 0x7f98b40cf690 2024-04-25T00:40:17.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.361+0000 7f98bb7b2640 1 -- 172.21.15.29:0/3670570261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98b41558c0 con 0x7f98b40cf690 2024-04-25T00:40:17.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.361+0000 7f98a9ffb640 1 -- 172.21.15.29:0/3670570261 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5185 (secure 0 0 0) 0x7f98b0057370 con 0x7f98b40cf690 2024-04-25T00:40:17.361 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:17.362 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":93,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3075204329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3075204329},{"type":"v1","addr":"172.21.15.157:6833","nonce":3075204329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3377772964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3377772964},{"type":"v1","addr":"172.21.15.157:6835","nonce":3377772964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":9621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/897131811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":897131811},{"type":"v1","addr":"172.21.15.29:6833","nonce":897131811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:39:48.556647+0000","modified":"2024-04-25T00:39:51.600640+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9624":{"gid":9624,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/821427804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":821427804},{"type":"v1","addr":"172.21.15.29:6835","nonce":821427804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9624,"qdb_cluster":[9624]},"id":11}]} 2024-04-25T00:40:17.362 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 93 2024-04-25T00:40:17.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.365+0000 7f98bb7b2640 1 -- 172.21.15.29:0/3670570261 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f98980680a0 msgr2=0x7f989806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.365+0000 7f98bb7b2640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f98980680a0 0x7f989806a560 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f98b40b6890 tx=0x7f98a4009040 comp rx=0 tx=0).stop 2024-04-25T00:40:17.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.365+0000 7f98bb7b2640 1 -- 172.21.15.29:0/3670570261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98b40cf690 msgr2=0x7f98b4155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.365+0000 7f98bb7b2640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98b40cf690 0x7f98b4155380 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f98b000d5f0 tx=0x7f98b000dab0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.365+0000 7f98bb7b2640 1 -- 172.21.15.29:0/3670570261 shutdown_connections 2024-04-25T00:40:17.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.365+0000 7f98bb7b2640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f98980680a0 0x7f989806a560 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.365+0000 7f98bb7b2640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98b415e200 0x7f98b40b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.365+0000 7f98bb7b2640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98b40d2c60 0x7f98b40c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.365+0000 7f98bb7b2640 1 --2- 172.21.15.29:0/3670570261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98b40cf690 0x7f98b4155380 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.365+0000 7f98bb7b2640 1 -- 172.21.15.29:0/3670570261 >> 172.21.15.29:0/3670570261 conn(0x7f98b409d280 msgr2=0x7f98b406d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:17.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.365+0000 7f98bb7b2640 1 -- 172.21.15.29:0/3670570261 shutdown_connections 2024-04-25T00:40:17.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.365+0000 7f98bb7b2640 1 -- 172.21.15.29:0/3670570261 wait complete. 2024-04-25T00:40:17.380 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:40:17.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.485+0000 7f27a2977640 1 Processor -- start 2024-04-25T00:40:17.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.485+0000 7f27a2977640 1 -- start start 2024-04-25T00:40:17.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f27a2977640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f279c15e200 0x7f279c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f27a2977640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f279c0cfc10 con 0x7f279c0cf690 2024-04-25T00:40:17.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f27a2977640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f279c0cfef0 con 0x7f279c0d2c60 2024-04-25T00:40:17.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f27a2977640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f279c0b38d0 con 0x7f279c15e200 2024-04-25T00:40:17.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f279bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f279c15e200 0x7f279c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f279bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f279c15e200 0x7f279c0d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52484/0 (socket says 172.21.15.29:52484) 2024-04-25T00:40:17.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f279bfff640 1 -- 172.21.15.29:0/3904766203 learned_addr learned my addr 172.21.15.29:0/3904766203 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:17.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f279bfff640 1 -- 172.21.15.29:0/3904766203 >> v1:172.21.15.157:6789/0 conn(0x7f279c0d2c60 legacy=0x7f279c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:17.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f279bfff640 1 -- 172.21.15.29:0/3904766203 >> v1:172.21.15.29:6789/0 conn(0x7f279c0cf690 legacy=0x7f279c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f279bfff640 1 -- 172.21.15.29:0/3904766203 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c0b3b20 con 0x7f279c15e200 2024-04-25T00:40:17.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f279bfff640 1 --2- 172.21.15.29:0/3904766203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f279c15e200 0x7f279c0d47f0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f278c009e50 tx=0x7f278c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b40b6a5be3dac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:17.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f279affd640 1 -- 172.21.15.29:0/3904766203 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c0a58a0 con 0x7f279c15e200 2024-04-25T00:40:17.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f279affd640 1 -- 172.21.15.29:0/3904766203 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f278c0a5a40 con 0x7f279c15e200 2024-04-25T00:40:17.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f279affd640 1 -- 172.21.15.29:0/3904766203 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c0a5e10 con 0x7f279c15e200 2024-04-25T00:40:17.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f27a2977640 1 -- 172.21.15.29:0/3904766203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f279c15e200 msgr2=0x7f279c0d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.489+0000 7f27a2977640 1 --2- 172.21.15.29:0/3904766203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f279c15e200 0x7f279c0d47f0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f278c009e50 tx=0x7f278c01c9c0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 -- 172.21.15.29:0/3904766203 shutdown_connections 2024-04-25T00:40:17.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 --2- 172.21.15.29:0/3904766203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f279c15e200 0x7f279c0d47f0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 -- 172.21.15.29:0/3904766203 >> 172.21.15.29:0/3904766203 conn(0x7f279c09d280 msgr2=0x7f279c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:17.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 -- 172.21.15.29:0/3904766203 shutdown_connections 2024-04-25T00:40:17.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 -- 172.21.15.29:0/3904766203 wait complete. 2024-04-25T00:40:17.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 Processor -- start 2024-04-25T00:40:17.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 -- start start 2024-04-25T00:40:17.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0cf690 0x7f279c0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c0d2c60 0x7f279c0b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f279c15e200 0x7f279c0b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f279c0b53c0 con 0x7f279c15e200 2024-04-25T00:40:17.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f279c0b5240 con 0x7f279c0cf690 2024-04-25T00:40:17.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f279c0b5540 con 0x7f279c0d2c60 2024-04-25T00:40:17.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a0f14640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f279c15e200 0x7f279c0b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a0f14640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f279c15e200 0x7f279c0b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52496/0 (socket says 172.21.15.29:52496) 2024-04-25T00:40:17.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a0f14640 1 -- 172.21.15.29:0/1818548549 learned_addr learned my addr 172.21.15.29:0/1818548549 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:17.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f279b7fe640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c0d2c60 0x7f279c0b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f279bfff640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0cf690 0x7f279c0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a0f14640 1 -- 172.21.15.29:0/1818548549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c0d2c60 msgr2=0x7f279c0b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a0f14640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c0d2c60 0x7f279c0b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a0f14640 1 -- 172.21.15.29:0/1818548549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0cf690 msgr2=0x7f279c0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a0f14640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0cf690 0x7f279c0c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a0f14640 1 -- 172.21.15.29:0/1818548549 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c15ecc0 con 0x7f279c15e200 2024-04-25T00:40:17.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f279b7fe640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c0d2c60 0x7f279c0b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:17.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a0f14640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f279c15e200 0x7f279c0b19c0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f279400b410 tx=0x7f279400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:17.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f2798ff9640 1 -- 172.21.15.29:0/1818548549 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794015020 con 0x7f279c15e200 2024-04-25T00:40:17.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f2798ff9640 1 -- 172.21.15.29:0/1818548549 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2794011870 con 0x7f279c15e200 2024-04-25T00:40:17.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 -- 172.21.15.29:0/1818548549 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279c15efa0 con 0x7f279c15e200 2024-04-25T00:40:17.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f2798ff9640 1 -- 172.21.15.29:0/1818548549 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794011c60 con 0x7f279c15e200 2024-04-25T00:40:17.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.493+0000 7f27a2977640 1 -- 172.21.15.29:0/1818548549 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279c15f560 con 0x7f279c15e200 2024-04-25T00:40:17.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.497+0000 7f27a2977640 1 -- 172.21.15.29:0/1818548549 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f279c069270 con 0x7f279c15e200 2024-04-25T00:40:17.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.497+0000 7f2798ff9640 1 -- 172.21.15.29:0/1818548549 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f2794018020 con 0x7f279c15e200 2024-04-25T00:40:17.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.497+0000 7f2798ff9640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27880680d0 0x7f278806a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.497+0000 7f279bfff640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27880680d0 0x7f278806a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.497+0000 7f2798ff9640 1 -- 172.21.15.29:0/1818548549 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f27940841f0 con 0x7f279c15e200 2024-04-25T00:40:17.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.497+0000 7f279bfff640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27880680d0 0x7f278806a590 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f278c0ae000 tx=0x7f278c0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:17.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.501+0000 7f2798ff9640 1 -- 172.21.15.29:0/1818548549 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f279401f030 con 0x7f279c15e200 2024-04-25T00:40:17.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.657+0000 7f27a2977640 1 -- 172.21.15.29:0/1818548549 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f279c0d37c0 con 0x7f279c15e200 2024-04-25T00:40:17.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.661+0000 7f2798ff9640 1 -- 172.21.15.29:0/1818548549 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16623 (secure 0 0 0) 0x7f2794056620 con 0x7f279c15e200 2024-04-25T00:40:17.660 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:17.660 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":145,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:39:50.179122+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:39:46.824888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:39:47.550815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:40:17.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.661+0000 7f27a2977640 1 -- 172.21.15.29:0/1818548549 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27880680d0 msgr2=0x7f278806a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.661+0000 7f27a2977640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27880680d0 0x7f278806a590 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f278c0ae000 tx=0x7f278c0a7070 comp rx=0 tx=0).stop 2024-04-25T00:40:17.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.661+0000 7f27a2977640 1 -- 172.21.15.29:0/1818548549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f279c15e200 msgr2=0x7f279c0b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.661+0000 7f27a2977640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f279c15e200 0x7f279c0b19c0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f279400b410 tx=0x7f279400b8d0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.661+0000 7f27a2977640 1 -- 172.21.15.29:0/1818548549 shutdown_connections 2024-04-25T00:40:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.661+0000 7f27a2977640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27880680d0 0x7f278806a590 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.661+0000 7f27a2977640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f279c15e200 0x7f279c0b19c0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.661+0000 7f27a2977640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c0d2c60 0x7f279c0b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.661+0000 7f27a2977640 1 --2- 172.21.15.29:0/1818548549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0cf690 0x7f279c0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.661+0000 7f27a2977640 1 -- 172.21.15.29:0/1818548549 >> 172.21.15.29:0/1818548549 conn(0x7f279c09d280 msgr2=0x7f279c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.661+0000 7f27a2977640 1 -- 172.21.15.29:0/1818548549 shutdown_connections 2024-04-25T00:40:17.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.661+0000 7f27a2977640 1 -- 172.21.15.29:0/1818548549 wait complete. 2024-04-25T00:40:17.678 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:40:17.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.789+0000 7f056d805640 1 Processor -- start 2024-04-25T00:40:17.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.789+0000 7f056d805640 1 -- start start 2024-04-25T00:40:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f056d805640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f056806dce0 0x7f0568150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f056d805640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05680b2b70 0x7f0568151330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f056d805640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05680b3130 con 0x7f056815d980 2024-04-25T00:40:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f056d805640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f05680cf560 con 0x7f056806dce0 2024-04-25T00:40:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f056d805640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f05680b3410 con 0x7f05680b2b70 2024-04-25T00:40:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f0566ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f056815d980 0x7f056806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38290/0 (socket says 172.21.15.29:38290) 2024-04-25T00:40:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f0566ffd640 1 -- 172.21.15.29:0/3710570014 learned_addr learned my addr 172.21.15.29:0/3710570014 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05677fe640 1 --2- 172.21.15.29:0/3710570014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05680b2b70 0x7f0568151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f055e7fc640 1 --2- 172.21.15.29:0/3710570014 >> v2:172.21.15.157:3300/0 conn(0x7f056806dce0 0x7f0568150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3957974684 0 0) 0x7f05680b3130 con 0x7f056815d980 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0554004640 con 0x7f056815d980 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2777438513 0 0) 0x7f0554004640 con 0x7f056815d980 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05680b2b70 msgr2=0x7f0568151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 --2- 172.21.15.29:0/3710570014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05680b2b70 0x7f0568151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 >> v2:172.21.15.157:3300/0 conn(0x7f056806dce0 msgr2=0x7f0568150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 --2- 172.21.15.29:0/3710570014 >> v2:172.21.15.157:3300/0 conn(0x7f056806dce0 0x7f0568150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05680b36f0 con 0x7f056815d980 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f0558001680 con 0x7f056815d980 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 >> v1:172.21.15.29:6789/0 conn(0x7f056815d980 legacy=0x7f056806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 --2- 172.21.15.29:0/3710570014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05680b2b70 0x7f0554004ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 --2- 172.21.15.29:0/3710570014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0554005050 0x7f0554005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 --2- 172.21.15.29:0/3710570014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0554005e60 0x7f05540062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f05680cf560 con 0x7f0554005050 2024-04-25T00:40:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05680b3410 con 0x7f0554005e60 2024-04-25T00:40:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f055400cae0 con 0x7f05680b2b70 2024-04-25T00:40:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f055e7fc640 1 --2- 172.21.15.29:0/3710570014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0554005050 0x7f0554005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f055e7fc640 1 -- 172.21.15.29:0/3710570014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05680b2b70 msgr2=0x7f0554004ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f055e7fc640 1 --2- 172.21.15.29:0/3710570014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05680b2b70 0x7f0554004ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f055e7fc640 1 -- 172.21.15.29:0/3710570014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0554005e60 msgr2=0x7f05540062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f055e7fc640 1 --2- 172.21.15.29:0/3710570014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0554005e60 0x7f05540062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.793+0000 7f055e7fc640 1 -- 172.21.15.29:0/3710570014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0554004640 con 0x7f0554005050 2024-04-25T00:40:17.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f055e7fc640 1 --2- 172.21.15.29:0/3710570014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0554005050 0x7f0554005920 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f0550009ae0 tx=0x7f055001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848981432ce6f9dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:17.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05500b2070 con 0x7f0554005050 2024-04-25T00:40:17.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05500a56a0 con 0x7f0554005050 2024-04-25T00:40:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f05667fc640 1 -- 172.21.15.29:0/3710570014 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05500096b0 con 0x7f0554005050 2024-04-25T00:40:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 -- 172.21.15.29:0/3710570014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0554005050 msgr2=0x7f0554005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 --2- 172.21.15.29:0/3710570014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0554005050 0x7f0554005920 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f0550009ae0 tx=0x7f055001ca50 comp rx=0 tx=0).stop 2024-04-25T00:40:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f05677fe640 1 -- 172.21.15.29:0/3710570014 reap_dead start 2024-04-25T00:40:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 -- 172.21.15.29:0/3710570014 shutdown_connections 2024-04-25T00:40:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 -- 172.21.15.29:0/3710570014 >> 172.21.15.29:0/3710570014 conn(0x7f05680b1870 msgr2=0x7f05680b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 -- 172.21.15.29:0/3710570014 shutdown_connections 2024-04-25T00:40:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 -- 172.21.15.29:0/3710570014 wait complete. 2024-04-25T00:40:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 Processor -- start 2024-04-25T00:40:17.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 -- start start 2024-04-25T00:40:17.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0568146690 0x7f05681576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0568157bf0 0x7f056814f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f056814fc60 0x7f05681534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f056815dd60 con 0x7f0568146690 2024-04-25T00:40:17.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05680b3cb0 con 0x7f056814fc60 2024-04-25T00:40:17.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f056d805640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0568151330 con 0x7f0568157bf0 2024-04-25T00:40:17.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f0566ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0568146690 0x7f05681576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f05677fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f056814fc60 0x7f05681534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f05677fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f056814fc60 0x7f05681534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:56438/0 (socket says 172.21.15.29:56438) 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f0566ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0568146690 0x7f05681576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52966/0 (socket says 172.21.15.29:52966) 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f055e7fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0568157bf0 0x7f056814f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f05677fe640 1 -- 172.21.15.29:0/4151682180 learned_addr learned my addr 172.21.15.29:0/4151682180 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f0566ffd640 1 -- 172.21.15.29:0/4151682180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0568157bf0 msgr2=0x7f056814f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f0566ffd640 1 --2- 172.21.15.29:0/4151682180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0568157bf0 0x7f056814f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f0566ffd640 1 -- 172.21.15.29:0/4151682180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f056814fc60 msgr2=0x7f05681534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f0566ffd640 1 --2- 172.21.15.29:0/4151682180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f056814fc60 0x7f05681534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f0566ffd640 1 -- 172.21.15.29:0/4151682180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0568158230 con 0x7f0568146690 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.797+0000 7f0566ffd640 1 --2- 172.21.15.29:0/4151682180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0568146690 0x7f05681576b0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f0558011a40 tx=0x7f0558011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.801+0000 7f0564ff9640 1 -- 172.21.15.29:0/4151682180 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05580078f0 con 0x7f0568146690 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.801+0000 7f0564ff9640 1 -- 172.21.15.29:0/4151682180 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0558007ad0 con 0x7f0568146690 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.801+0000 7f0564ff9640 1 -- 172.21.15.29:0/4151682180 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055800a0d0 con 0x7f0568146690 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.801+0000 7f056d805640 1 -- 172.21.15.29:0/4151682180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0568152a00 con 0x7f0568146690 2024-04-25T00:40:17.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.801+0000 7f056d805640 1 -- 172.21.15.29:0/4151682180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0568152fc0 con 0x7f0568146690 2024-04-25T00:40:17.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.801+0000 7f056d805640 1 -- 172.21.15.29:0/4151682180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0568069270 con 0x7f0568146690 2024-04-25T00:40:17.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.801+0000 7f0564ff9640 1 -- 172.21.15.29:0/4151682180 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f055800a270 con 0x7f0568146690 2024-04-25T00:40:17.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.801+0000 7f0564ff9640 1 --2- 172.21.15.29:0/4151682180 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f05480680d0 0x7f054806a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:17.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.801+0000 7f0564ff9640 1 -- 172.21.15.29:0/4151682180 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f055808b100 con 0x7f0568146690 2024-04-25T00:40:17.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.801+0000 7f055e7fc640 1 --2- 172.21.15.29:0/4151682180 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f05480680d0 0x7f054806a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:17.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.801+0000 7f055e7fc640 1 --2- 172.21.15.29:0/4151682180 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f05480680d0 0x7f054806a590 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f05500af000 tx=0x7f05500046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:17.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.805+0000 7f0564ff9640 1 -- 172.21.15.29:0/4151682180 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f055805cee0 con 0x7f0568146690 2024-04-25T00:40:17.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:17.997+0000 7f056d805640 1 -- 172.21.15.29:0/4151682180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f05680cf910 con 0x7f0568146690 2024-04-25T00:40:17.999 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-04-25T00:40:17.997+0000 7f3ac00b5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:40:18.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.001+0000 7f0564ff9640 1 -- 172.21.15.29:0/4151682180 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v94) ==== 161+0+0 (secure 0 0 0) 0x7f05580611e0 con 0x7f0568146690 2024-04-25T00:40:18.001 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:40:18.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.005+0000 7f056d805640 1 -- 172.21.15.29:0/4151682180 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f05480680d0 msgr2=0x7f054806a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:18.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.005+0000 7f056d805640 1 --2- 172.21.15.29:0/4151682180 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f05480680d0 0x7f054806a590 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f05500af000 tx=0x7f05500046d0 comp rx=0 tx=0).stop 2024-04-25T00:40:18.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.005+0000 7f056d805640 1 -- 172.21.15.29:0/4151682180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0568146690 msgr2=0x7f05681576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:18.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.005+0000 7f056d805640 1 --2- 172.21.15.29:0/4151682180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0568146690 0x7f05681576b0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f0558011a40 tx=0x7f0558011f00 comp rx=0 tx=0).stop 2024-04-25T00:40:18.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.005+0000 7f056d805640 1 -- 172.21.15.29:0/4151682180 shutdown_connections 2024-04-25T00:40:18.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.005+0000 7f056d805640 1 --2- 172.21.15.29:0/4151682180 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f05480680d0 0x7f054806a590 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:18.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.005+0000 7f056d805640 1 --2- 172.21.15.29:0/4151682180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f056814fc60 0x7f05681534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:18.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.005+0000 7f056d805640 1 --2- 172.21.15.29:0/4151682180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0568157bf0 0x7f056814f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:18.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.005+0000 7f056d805640 1 --2- 172.21.15.29:0/4151682180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0568146690 0x7f05681576b0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:18.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.005+0000 7f056d805640 1 -- 172.21.15.29:0/4151682180 >> 172.21.15.29:0/4151682180 conn(0x7f05680b1870 msgr2=0x7f05680b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:18.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.009+0000 7f056d805640 1 -- 172.21.15.29:0/4151682180 shutdown_connections 2024-04-25T00:40:18.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.009+0000 7f056d805640 1 -- 172.21.15.29:0/4151682180 wait complete. 2024-04-25T00:40:18.024 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:40:18.044 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:40:18.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22fce1640 1 Processor -- start 2024-04-25T00:40:18.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22fce1640 1 -- start start 2024-04-25T00:40:18.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22fce1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe22806dd40 0x7fe2280715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:18.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22fce1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe228071af0 0x7fe2280aef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:18.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22fce1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe22806e3d0 con 0x7fe22815e160 2024-04-25T00:40:18.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22fce1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe22806e1a0 con 0x7fe22806dd40 2024-04-25T00:40:18.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22fce1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe22806e6b0 con 0x7fe228071af0 2024-04-25T00:40:18.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22e27e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe228071af0 0x7fe2280aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:18.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22d27c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe22806dd40 0x7fe2280715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:18.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22e27e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe228071af0 0x7fe2280aef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:34556/0 (socket says 172.21.15.29:34556) 2024-04-25T00:40:18.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22e27e640 1 -- 172.21.15.29:0/3422218123 learned_addr learned my addr 172.21.15.29:0/3422218123 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:18.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22e27e640 1 -- 172.21.15.29:0/3422218123 >> v2:172.21.15.157:3300/0 conn(0x7fe22806dd40 msgr2=0x7fe2280715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:18.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22e27e640 1 --2- 172.21.15.29:0/3422218123 >> v2:172.21.15.157:3300/0 conn(0x7fe22806dd40 0x7fe2280715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:18.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22e27e640 1 -- 172.21.15.29:0/3422218123 >> v1:172.21.15.157:6789/0 conn(0x7fe22815e160 legacy=0x7fe2280b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:18.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.153+0000 7fe22e27e640 1 -- 172.21.15.29:0/3422218123 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2280ae700 con 0x7fe228071af0 2024-04-25T00:40:18.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22e27e640 1 --2- 172.21.15.29:0/3422218123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe228071af0 0x7fe2280aef00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe220009870 tx=0x7fe22001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6f9481fd3fab6a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:18.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22ca7b640 1 -- 172.21.15.29:0/3422218123 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2200b0070 con 0x7fe228071af0 2024-04-25T00:40:18.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22ca7b640 1 -- 172.21.15.29:0/3422218123 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe220004230 con 0x7fe228071af0 2024-04-25T00:40:18.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22ca7b640 1 -- 172.21.15.29:0/3422218123 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220003620 con 0x7fe228071af0 2024-04-25T00:40:18.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22fce1640 1 -- 172.21.15.29:0/3422218123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe228071af0 msgr2=0x7fe2280aef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:18.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22fce1640 1 --2- 172.21.15.29:0/3422218123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe228071af0 0x7fe2280aef00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe220009870 tx=0x7fe22001c440 comp rx=0 tx=0).stop 2024-04-25T00:40:18.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22fce1640 1 -- 172.21.15.29:0/3422218123 shutdown_connections 2024-04-25T00:40:18.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22fce1640 1 --2- 172.21.15.29:0/3422218123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe228071af0 0x7fe2280aef00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:18.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22fce1640 1 --2- 172.21.15.29:0/3422218123 >> v2:172.21.15.157:3300/0 conn(0x7fe22806dd40 0x7fe2280715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:18.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22fce1640 1 -- 172.21.15.29:0/3422218123 >> 172.21.15.29:0/3422218123 conn(0x7fe228075030 msgr2=0x7fe2280b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:18.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22fce1640 1 -- 172.21.15.29:0/3422218123 shutdown_connections 2024-04-25T00:40:18.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22fce1640 1 -- 172.21.15.29:0/3422218123 wait complete. 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22fce1640 1 Processor -- start 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.157+0000 7fe22fce1640 1 -- start start 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22fce1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe22806dd40 0x7fe22815a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22fce1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe228071af0 0x7fe22815fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22fce1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22815e160 0x7fe2281602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22fce1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe2280ccfe0 con 0x7fe22806dd40 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22fce1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe2280cce60 con 0x7fe22815e160 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22fce1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe22816ada0 con 0x7fe228071af0 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22da7d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe22806dd40 0x7fe22815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22da7d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe22806dd40 0x7fe22815a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53000/0 (socket says 172.21.15.29:53000) 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22da7d640 1 -- 172.21.15.29:0/395740191 learned_addr learned my addr 172.21.15.29:0/395740191 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22d27c640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe228071af0 0x7fe22815fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22e27e640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22815e160 0x7fe2281602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22da7d640 1 -- 172.21.15.29:0/395740191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe228071af0 msgr2=0x7fe22815fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22da7d640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe228071af0 0x7fe22815fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22da7d640 1 -- 172.21.15.29:0/395740191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22815e160 msgr2=0x7fe2281602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22da7d640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22815e160 0x7fe2281602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22da7d640 1 -- 172.21.15.29:0/395740191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22815bdd0 con 0x7fe22806dd40 2024-04-25T00:40:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22e27e640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22815e160 0x7fe2281602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22da7d640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe22806dd40 0x7fe22815a000 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00c3a0 tx=0x7fe21c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe2267fc640 1 -- 172.21.15.29:0/395740191 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c0052d0 con 0x7fe22806dd40 2024-04-25T00:40:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe2267fc640 1 -- 172.21.15.29:0/395740191 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe21c005470 con 0x7fe22806dd40 2024-04-25T00:40:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22fce1640 1 -- 172.21.15.29:0/395740191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22815c0b0 con 0x7fe22806dd40 2024-04-25T00:40:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe2267fc640 1 -- 172.21.15.29:0/395740191 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c007260 con 0x7fe22806dd40 2024-04-25T00:40:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22fce1640 1 -- 172.21.15.29:0/395740191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22815f2b0 con 0x7fe22806dd40 2024-04-25T00:40:18.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22fce1640 1 -- 172.21.15.29:0/395740191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe228069270 con 0x7fe22806dd40 2024-04-25T00:40:18.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe2267fc640 1 -- 172.21.15.29:0/395740191 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fe21c007400 con 0x7fe22806dd40 2024-04-25T00:40:18.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe2267fc640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe2140680a0 0x7fe21406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:18.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22d27c640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe2140680a0 0x7fe21406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:18.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe2267fc640 1 -- 172.21.15.29:0/395740191 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5795+0+0 (secure 0 0 0) 0x7fe21c083ad0 con 0x7fe22806dd40 2024-04-25T00:40:18.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.161+0000 7fe22d27c640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe2140680a0 0x7fe21406a560 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe2281571d0 tx=0x7fe21800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:18.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.165+0000 7fe2267fc640 1 -- 172.21.15.29:0/395740191 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe21c055f00 con 0x7fe22806dd40 2024-04-25T00:40:18.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:18.357+0000 7fe22fce1640 1 -- 172.21.15.29:0/395740191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe228158bb0 con 0x7fe22806dd40 2024-04-25T00:40:19.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.005+0000 7fe2267fc640 1 -- 172.21.15.29:0/395740191 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v95) ==== 104+0+0 (secure 0 0 0) 0x7fe21c05c0d0 con 0x7fe22806dd40 2024-04-25T00:40:19.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.005+0000 7fe22fce1640 1 -- 172.21.15.29:0/395740191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe2140680a0 msgr2=0x7fe21406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:19.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.005+0000 7fe22fce1640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe2140680a0 0x7fe21406a560 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe2281571d0 tx=0x7fe21800b040 comp rx=0 tx=0).stop 2024-04-25T00:40:19.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.005+0000 7fe22fce1640 1 -- 172.21.15.29:0/395740191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe22806dd40 msgr2=0x7fe22815a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:19.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.005+0000 7fe22fce1640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe22806dd40 0x7fe22815a000 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00c3a0 tx=0x7fe21c00c860 comp rx=0 tx=0).stop 2024-04-25T00:40:19.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.005+0000 7fe22fce1640 1 -- 172.21.15.29:0/395740191 shutdown_connections 2024-04-25T00:40:19.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.005+0000 7fe22fce1640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe2140680a0 0x7fe21406a560 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:19.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.005+0000 7fe22fce1640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22815e160 0x7fe2281602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:19.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.005+0000 7fe22fce1640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe228071af0 0x7fe22815fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:19.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.005+0000 7fe22fce1640 1 --2- 172.21.15.29:0/395740191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe22806dd40 0x7fe22815a000 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:19.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.005+0000 7fe22fce1640 1 -- 172.21.15.29:0/395740191 >> 172.21.15.29:0/395740191 conn(0x7fe228075030 msgr2=0x7fe22815c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:19.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.005+0000 7fe22fce1640 1 -- 172.21.15.29:0/395740191 shutdown_connections 2024-04-25T00:40:19.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.005+0000 7fe22fce1640 1 -- 172.21.15.29:0/395740191 wait complete. 2024-04-25T00:40:19.028 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:40:19.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.201+0000 7f9f48086640 1 Processor -- start 2024-04-25T00:40:19.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.201+0000 7f9f48086640 1 -- start start 2024-04-25T00:40:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f48086640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9f40161730 0x7f9f400b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f48086640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f400b2ad0 0x7f9f4006c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f48086640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f4006ca70 0x7f9f400d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f48086640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9f400b2100 con 0x7f9f40161730 2024-04-25T00:40:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f48086640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9f4006d5f0 con 0x7f9f4006ca70 2024-04-25T00:40:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f48086640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f4006d030 con 0x7f9f400b2ad0 2024-04-25T00:40:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f45e22640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9f40161730 0x7f9f400b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f45e22640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9f40161730 0x7f9f400b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53010/0 (socket says 172.21.15.29:53010) 2024-04-25T00:40:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f45e22640 1 -- 172.21.15.29:0/57823394 learned_addr learned my addr 172.21.15.29:0/57823394 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f46623640 1 --2- 172.21.15.29:0/57823394 >> v2:172.21.15.157:3300/0 conn(0x7f9f4006ca70 0x7f9f400d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f45621640 1 --2- 172.21.15.29:0/57823394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f400b2ad0 0x7f9f4006c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f45e22640 1 -- 172.21.15.29:0/57823394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f400b2ad0 msgr2=0x7f9f4006c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:19.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f45e22640 1 --2- 172.21.15.29:0/57823394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f400b2ad0 0x7f9f4006c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:19.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f45e22640 1 -- 172.21.15.29:0/57823394 >> v2:172.21.15.157:3300/0 conn(0x7f9f4006ca70 msgr2=0x7f9f400d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:19.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f45e22640 1 --2- 172.21.15.29:0/57823394 >> v2:172.21.15.157:3300/0 conn(0x7f9f4006ca70 0x7f9f400d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:19.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f45e22640 1 -- 172.21.15.29:0/57823394 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4006d310 con 0x7f9f40161730 2024-04-25T00:40:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f45e22640 1 --2- 172.21.15.29:0/57823394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f40161730 0x7f9f400b2590 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f9f40072e10 tx=0x7f9f3c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa72900104d2bb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f44e20640 1 -- 172.21.15.29:0/57823394 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c0b0070 con 0x7f9f40161730 2024-04-25T00:40:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f44e20640 1 -- 172.21.15.29:0/57823394 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f3c0ab600 con 0x7f9f40161730 2024-04-25T00:40:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.205+0000 7f9f44e20640 1 -- 172.21.15.29:0/57823394 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c0ab9d0 con 0x7f9f40161730 2024-04-25T00:40:19.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 -- 172.21.15.29:0/57823394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f40161730 msgr2=0x7f9f400b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:19.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 --2- 172.21.15.29:0/57823394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f40161730 0x7f9f400b2590 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f9f40072e10 tx=0x7f9f3c01c3d0 comp rx=0 tx=0).stop 2024-04-25T00:40:19.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 -- 172.21.15.29:0/57823394 shutdown_connections 2024-04-25T00:40:19.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 --2- 172.21.15.29:0/57823394 >> v2:172.21.15.157:3300/0 conn(0x7f9f4006ca70 0x7f9f400d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:19.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 --2- 172.21.15.29:0/57823394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f400b2ad0 0x7f9f4006c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:19.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 --2- 172.21.15.29:0/57823394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f40161730 0x7f9f400b2590 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:19.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 -- 172.21.15.29:0/57823394 >> 172.21.15.29:0/57823394 conn(0x7f9f4009aa30 msgr2=0x7f9f4009ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:19.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 -- 172.21.15.29:0/57823394 shutdown_connections 2024-04-25T00:40:19.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 -- 172.21.15.29:0/57823394 wait complete. 2024-04-25T00:40:19.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 Processor -- start 2024-04-25T00:40:19.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 -- start start 2024-04-25T00:40:19.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f4006ca70 0x7f9f40154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:19.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f400b2ad0 0x7f9f4015a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:19.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f40161730 0x7f9f4015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:19.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40161e80 con 0x7f9f40161730 2024-04-25T00:40:19.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f400d09e0 con 0x7f9f400b2ad0 2024-04-25T00:40:19.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f40162000 con 0x7f9f4006ca70 2024-04-25T00:40:19.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f46623640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f40161730 0x7f9f4015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f46623640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f40161730 0x7f9f4015aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53016/0 (socket says 172.21.15.29:53016) 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f46623640 1 -- 172.21.15.29:0/973524040 learned_addr learned my addr 172.21.15.29:0/973524040 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f45621640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f400b2ad0 0x7f9f4015a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f45e22640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f4006ca70 0x7f9f40154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f46623640 1 -- 172.21.15.29:0/973524040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f4006ca70 msgr2=0x7f9f40154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f46623640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f4006ca70 0x7f9f40154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f46623640 1 -- 172.21.15.29:0/973524040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f400b2ad0 msgr2=0x7f9f4015a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f46623640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f400b2ad0 0x7f9f4015a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f46623640 1 -- 172.21.15.29:0/973524040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f400aeb50 con 0x7f9f40161730 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f46623640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f40161730 0x7f9f4015aaa0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f9f38007e60 tx=0x7f9f3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f367fc640 1 -- 172.21.15.29:0/973524040 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38014620 con 0x7f9f40161730 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f367fc640 1 -- 172.21.15.29:0/973524040 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f38019070 con 0x7f9f40161730 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 -- 172.21.15.29:0/973524040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f400aedc0 con 0x7f9f40161730 2024-04-25T00:40:19.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f367fc640 1 -- 172.21.15.29:0/973524040 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38014a20 con 0x7f9f40161730 2024-04-25T00:40:19.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.209+0000 7f9f48086640 1 -- 172.21.15.29:0/973524040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f400af380 con 0x7f9f40161730 2024-04-25T00:40:19.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.213+0000 7f9f48086640 1 -- 172.21.15.29:0/973524040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f400692e0 con 0x7f9f40161730 2024-04-25T00:40:19.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.213+0000 7f9f367fc640 1 -- 172.21.15.29:0/973524040 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f9f38014bc0 con 0x7f9f40161730 2024-04-25T00:40:19.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.213+0000 7f9f367fc640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f240680a0 0x7f9f2406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:19.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.213+0000 7f9f367fc640 1 -- 172.21.15.29:0/973524040 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5795+0+0 (secure 0 0 0) 0x7f9f38084ee0 con 0x7f9f40161730 2024-04-25T00:40:19.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.213+0000 7f9f45e22640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f240680a0 0x7f9f2406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:19.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.213+0000 7f9f45e22640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f240680a0 0x7f9f2406a560 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0ad000 tx=0x7f9f3c0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:19.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.217+0000 7f9f367fc640 1 -- 172.21.15.29:0/973524040 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9f38057310 con 0x7f9f40161730 2024-04-25T00:40:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:19.377+0000 7f9f48086640 1 -- 172.21.15.29:0/973524040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9f4006dce0 con 0x7f9f40161730 2024-04-25T00:40:20.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.013+0000 7f9f367fc640 1 -- 172.21.15.29:0/973524040 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) ==== 181+0+0 (secure 0 0 0) 0x7f9f38056c10 con 0x7f9f40161730 2024-04-25T00:40:20.012 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:40:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.013+0000 7f9f48086640 1 -- 172.21.15.29:0/973524040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f240680a0 msgr2=0x7f9f2406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.013+0000 7f9f48086640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f240680a0 0x7f9f2406a560 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0ad000 tx=0x7f9f3c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:40:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.013+0000 7f9f48086640 1 -- 172.21.15.29:0/973524040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f40161730 msgr2=0x7f9f4015aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.013+0000 7f9f48086640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f40161730 0x7f9f4015aaa0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f9f38007e60 tx=0x7f9f3800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.013+0000 7f9f48086640 1 -- 172.21.15.29:0/973524040 shutdown_connections 2024-04-25T00:40:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.013+0000 7f9f48086640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9f240680a0 0x7f9f2406a560 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.013+0000 7f9f48086640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f40161730 0x7f9f4015aaa0 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.013+0000 7f9f48086640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f400b2ad0 0x7f9f4015a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.013+0000 7f9f48086640 1 --2- 172.21.15.29:0/973524040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f4006ca70 0x7f9f40154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.013+0000 7f9f48086640 1 -- 172.21.15.29:0/973524040 >> 172.21.15.29:0/973524040 conn(0x7f9f4009aa30 msgr2=0x7f9f400ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.017+0000 7f9f48086640 1 -- 172.21.15.29:0/973524040 shutdown_connections 2024-04-25T00:40:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.017+0000 7f9f48086640 1 -- 172.21.15.29:0/973524040 wait complete. 2024-04-25T00:40:20.038 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:40:20.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.177+0000 7f076a0f0640 1 Processor -- start 2024-04-25T00:40:20.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.177+0000 7f076a0f0640 1 -- start start 2024-04-25T00:40:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.177+0000 7f076a0f0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0764151640 0x7f076406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.177+0000 7f076a0f0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076406db10 0x7f076415e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.177+0000 7f076a0f0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07640b40a0 con 0x7f07640b2b70 2024-04-25T00:40:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.177+0000 7f076a0f0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f07640b3000 con 0x7f0764151640 2024-04-25T00:40:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.177+0000 7f076a0f0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f07640b32e0 con 0x7f076406db10 2024-04-25T00:40:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f0763fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076406db10 0x7f076415e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f0763fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076406db10 0x7f076415e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:34596/0 (socket says 172.21.15.29:34596) 2024-04-25T00:40:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f0763fff640 1 -- 172.21.15.29:0/2598580783 learned_addr learned my addr 172.21.15.29:0/2598580783 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f07637fe640 1 --2- 172.21.15.29:0/2598580783 >> v2:172.21.15.157:3300/0 conn(0x7f0764151640 0x7f076406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f0763fff640 1 -- 172.21.15.29:0/2598580783 >> v2:172.21.15.157:3300/0 conn(0x7f0764151640 msgr2=0x7f076406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f0763fff640 1 --2- 172.21.15.29:0/2598580783 >> v2:172.21.15.157:3300/0 conn(0x7f0764151640 0x7f076406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f0763fff640 1 -- 172.21.15.29:0/2598580783 >> v1:172.21.15.157:6789/0 conn(0x7f07640b2b70 legacy=0x7f076415dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f0763fff640 1 -- 172.21.15.29:0/2598580783 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076406df30 con 0x7f076406db10 2024-04-25T00:40:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f0763fff640 1 --2- 172.21.15.29:0/2598580783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076406db10 0x7f076415e6e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f075c009e50 tx=0x7f075c01c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f74dc5b7c8ac1eff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f07627fc640 1 -- 172.21.15.29:0/2598580783 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c0a55c0 con 0x7f076406db10 2024-04-25T00:40:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f07627fc640 1 -- 172.21.15.29:0/2598580783 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f075c0a5760 con 0x7f076406db10 2024-04-25T00:40:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f07627fc640 1 -- 172.21.15.29:0/2598580783 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c0a5b30 con 0x7f076406db10 2024-04-25T00:40:20.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f076a0f0640 1 -- 172.21.15.29:0/2598580783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076406db10 msgr2=0x7f076415e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:20.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f076a0f0640 1 --2- 172.21.15.29:0/2598580783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076406db10 0x7f076415e6e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f075c009e50 tx=0x7f075c01c9a0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f076a0f0640 1 -- 172.21.15.29:0/2598580783 shutdown_connections 2024-04-25T00:40:20.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f076a0f0640 1 --2- 172.21.15.29:0/2598580783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076406db10 0x7f076415e6e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f076a0f0640 1 --2- 172.21.15.29:0/2598580783 >> v2:172.21.15.157:3300/0 conn(0x7f0764151640 0x7f076406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.181+0000 7f076a0f0640 1 -- 172.21.15.29:0/2598580783 >> 172.21.15.29:0/2598580783 conn(0x7f07640b1870 msgr2=0x7f07640b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:20.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f076a0f0640 1 -- 172.21.15.29:0/2598580783 shutdown_connections 2024-04-25T00:40:20.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f076a0f0640 1 -- 172.21.15.29:0/2598580783 wait complete. 2024-04-25T00:40:20.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f076a0f0640 1 Processor -- start 2024-04-25T00:40:20.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f076a0f0640 1 -- start start 2024-04-25T00:40:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f076a0f0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076406db10 0x7f076413b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f076a0f0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0764151640 0x7f0764151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f076a0f0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0764151f90 0x7f076415d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f076a0f0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f076415eda0 con 0x7f0764151f90 2024-04-25T00:40:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f076a0f0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f076415ec20 con 0x7f076406db10 2024-04-25T00:40:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f076a0f0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f076415c410 con 0x7f0764151640 2024-04-25T00:40:20.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f0763fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0764151f90 0x7f076415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f0763fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0764151f90 0x7f076415d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53026/0 (socket says 172.21.15.29:53026) 2024-04-25T00:40:20.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f0763fff640 1 -- 172.21.15.29:0/2743036874 learned_addr learned my addr 172.21.15.29:0/2743036874 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:20.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f0762ffd640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0764151640 0x7f0764151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f07637fe640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076406db10 0x7f076413b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f0763fff640 1 -- 172.21.15.29:0/2743036874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0764151640 msgr2=0x7f0764151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:20.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f0763fff640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0764151640 0x7f0764151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f0763fff640 1 -- 172.21.15.29:0/2743036874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076406db10 msgr2=0x7f076413b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:20.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f0763fff640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076406db10 0x7f076413b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f0763fff640 1 -- 172.21.15.29:0/2743036874 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07641524a0 con 0x7f0764151f90 2024-04-25T00:40:20.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f0763fff640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0764151f90 0x7f076415d360 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f075c0af040 tx=0x7f075c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:20.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f074bfff640 1 -- 172.21.15.29:0/2743036874 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c009510 con 0x7f0764151f90 2024-04-25T00:40:20.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f074bfff640 1 -- 172.21.15.29:0/2743036874 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f075c0b8470 con 0x7f0764151f90 2024-04-25T00:40:20.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f074bfff640 1 -- 172.21.15.29:0/2743036874 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c0b88a0 con 0x7f0764151f90 2024-04-25T00:40:20.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f076a0f0640 1 -- 172.21.15.29:0/2743036874 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0764152780 con 0x7f0764151f90 2024-04-25T00:40:20.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.185+0000 7f076a0f0640 1 -- 172.21.15.29:0/2743036874 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0764152d40 con 0x7f0764151f90 2024-04-25T00:40:20.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.189+0000 7f076a0f0640 1 -- 172.21.15.29:0/2743036874 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07640692e0 con 0x7f0764151f90 2024-04-25T00:40:20.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.189+0000 7f074bfff640 1 -- 172.21.15.29:0/2743036874 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f075c0c0020 con 0x7f0764151f90 2024-04-25T00:40:20.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.189+0000 7f074bfff640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f07500680a0 0x7f075006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.189+0000 7f074bfff640 1 -- 172.21.15.29:0/2743036874 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5357+0+0 (secure 0 0 0) 0x7f075c121530 con 0x7f0764151f90 2024-04-25T00:40:20.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.189+0000 7f07637fe640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f07500680a0 0x7f075006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.189+0000 7f07637fe640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f07500680a0 0x7f075006a560 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0754009a50 tx=0x7f0754008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:20.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.193+0000 7f074bfff640 1 -- 172.21.15.29:0/2743036874 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f075c016620 con 0x7f0764151f90 2024-04-25T00:40:20.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:20.353+0000 7f076a0f0640 1 -- 172.21.15.29:0/2743036874 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f07640ced70 con 0x7f0764151f90 2024-04-25T00:40:21.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.013+0000 7f074bfff640 1 -- 172.21.15.29:0/2743036874 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) ==== 169+0+0 (secure 0 0 0) 0x7f075c0f1530 con 0x7f0764151f90 2024-04-25T00:40:21.015 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-04-25T00:40:21.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.017+0000 7f076a0f0640 1 -- 172.21.15.29:0/2743036874 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f07500680a0 msgr2=0x7f075006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.017+0000 7f076a0f0640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f07500680a0 0x7f075006a560 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0754009a50 tx=0x7f0754008040 comp rx=0 tx=0).stop 2024-04-25T00:40:21.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.017+0000 7f076a0f0640 1 -- 172.21.15.29:0/2743036874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0764151f90 msgr2=0x7f076415d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.017+0000 7f076a0f0640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0764151f90 0x7f076415d360 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f075c0af040 tx=0x7f075c019740 comp rx=0 tx=0).stop 2024-04-25T00:40:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.021+0000 7f076a0f0640 1 -- 172.21.15.29:0/2743036874 shutdown_connections 2024-04-25T00:40:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.021+0000 7f076a0f0640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f07500680a0 0x7f075006a560 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.021+0000 7f076a0f0640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0764151f90 0x7f076415d360 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.021+0000 7f076a0f0640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0764151640 0x7f0764151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.021+0000 7f076a0f0640 1 --2- 172.21.15.29:0/2743036874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076406db10 0x7f076413b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.021+0000 7f076a0f0640 1 -- 172.21.15.29:0/2743036874 >> 172.21.15.29:0/2743036874 conn(0x7f07640b1870 msgr2=0x7f0764070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.021+0000 7f076a0f0640 1 -- 172.21.15.29:0/2743036874 shutdown_connections 2024-04-25T00:40:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.021+0000 7f076a0f0640 1 -- 172.21.15.29:0/2743036874 wait complete. 2024-04-25T00:40:21.048 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:40:21.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.177+0000 7f17fa44d640 1 Processor -- start 2024-04-25T00:40:21.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17fa44d640 1 -- start start 2024-04-25T00:40:21.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17fa44d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17f415e8d0 0x7f17f40cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17fa44d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f17f415f3c0 0x7f17f415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17fa44d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17f40b3720 con 0x7f17f40b4780 2024-04-25T00:40:21.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17fa44d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f17f40b42f0 con 0x7f17f415e8d0 2024-04-25T00:40:21.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17fa44d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f17f40b39c0 con 0x7f17f415f3c0 2024-04-25T00:40:21.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17f3fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17f415e8d0 0x7f17f40cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17f3fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17f415e8d0 0x7f17f40cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53036/0 (socket says 172.21.15.29:53036) 2024-04-25T00:40:21.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17f3fff640 1 -- 172.21.15.29:0/152280396 learned_addr learned my addr 172.21.15.29:0/152280396 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:21.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17f89ea640 1 --2- 172.21.15.29:0/152280396 >> v2:172.21.15.157:3300/0 conn(0x7f17f415f3c0 0x7f17f415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17f3fff640 1 -- 172.21.15.29:0/152280396 >> v2:172.21.15.157:3300/0 conn(0x7f17f415f3c0 msgr2=0x7f17f415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:21.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17f3fff640 1 --2- 172.21.15.29:0/152280396 >> v2:172.21.15.157:3300/0 conn(0x7f17f415f3c0 0x7f17f415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17f3fff640 1 -- 172.21.15.29:0/152280396 >> v1:172.21.15.157:6789/0 conn(0x7f17f40b4780 legacy=0x7f17f415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:21.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17f3fff640 1 -- 172.21.15.29:0/152280396 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f40b3ca0 con 0x7f17f415e8d0 2024-04-25T00:40:21.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17f89ea640 1 --2- 172.21.15.29:0/152280396 >> v2:172.21.15.157:3300/0 conn(0x7f17f415f3c0 0x7f17f415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:21.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.181+0000 7f17f3fff640 1 --2- 172.21.15.29:0/152280396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f415e8d0 0x7f17f40cc240 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f17f415f9a0 tx=0x7f17e401e6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3654982a7e5276d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:21.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17f2ffd640 1 -- 172.21.15.29:0/152280396 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e40b2600 con 0x7f17f415e8d0 2024-04-25T00:40:21.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17f2ffd640 1 -- 172.21.15.29:0/152280396 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17e40b27a0 con 0x7f17f415e8d0 2024-04-25T00:40:21.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17f2ffd640 1 -- 172.21.15.29:0/152280396 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e40b2b70 con 0x7f17f415e8d0 2024-04-25T00:40:21.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 -- 172.21.15.29:0/152280396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f415e8d0 msgr2=0x7f17f40cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 --2- 172.21.15.29:0/152280396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f415e8d0 0x7f17f40cc240 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f17f415f9a0 tx=0x7f17e401e6f0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 -- 172.21.15.29:0/152280396 shutdown_connections 2024-04-25T00:40:21.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 --2- 172.21.15.29:0/152280396 >> v2:172.21.15.157:3300/0 conn(0x7f17f415f3c0 0x7f17f415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 --2- 172.21.15.29:0/152280396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f415e8d0 0x7f17f40cc240 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 -- 172.21.15.29:0/152280396 >> 172.21.15.29:0/152280396 conn(0x7f17f40b1870 msgr2=0x7f17f40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:21.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 -- 172.21.15.29:0/152280396 shutdown_connections 2024-04-25T00:40:21.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 -- 172.21.15.29:0/152280396 wait complete. 2024-04-25T00:40:21.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 Processor -- start 2024-04-25T00:40:21.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 -- start start 2024-04-25T00:40:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17f40b4780 0x7f17f4147710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f415e8d0 0x7f17f40b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f415f3c0 0x7f17f40b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f17f415d6a0 con 0x7f17f415f3c0 2024-04-25T00:40:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17f415d520 con 0x7f17f415e8d0 2024-04-25T00:40:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17fa44d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f17f406d5d0 con 0x7f17f40b4780 2024-04-25T00:40:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17f89ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f415f3c0 0x7f17f40b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17f89ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f415f3c0 0x7f17f40b2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53040/0 (socket says 172.21.15.29:53040) 2024-04-25T00:40:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17f89ea640 1 -- 172.21.15.29:0/4092117366 learned_addr learned my addr 172.21.15.29:0/4092117366 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17f3fff640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17f40b4780 0x7f17f4147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.185+0000 7f17f37fe640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f415e8d0 0x7f17f40b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f89ea640 1 -- 172.21.15.29:0/4092117366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17f40b4780 msgr2=0x7f17f4147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f89ea640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17f40b4780 0x7f17f4147710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f89ea640 1 -- 172.21.15.29:0/4092117366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f415e8d0 msgr2=0x7f17f40b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f89ea640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f415e8d0 0x7f17f40b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f89ea640 1 -- 172.21.15.29:0/4092117366 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f4139f40 con 0x7f17f415f3c0 2024-04-25T00:40:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f89ea640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f415f3c0 0x7f17f40b2ea0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00cd80 tx=0x7f17ec00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f0ff9640 1 -- 172.21.15.29:0/4092117366 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00f680 con 0x7f17f415f3c0 2024-04-25T00:40:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f0ff9640 1 -- 172.21.15.29:0/4092117366 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17ec00f840 con 0x7f17f415f3c0 2024-04-25T00:40:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f0ff9640 1 -- 172.21.15.29:0/4092117366 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00fc30 con 0x7f17f415f3c0 2024-04-25T00:40:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17fa44d640 1 -- 172.21.15.29:0/4092117366 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f413a220 con 0x7f17f415f3c0 2024-04-25T00:40:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17fa44d640 1 -- 172.21.15.29:0/4092117366 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f413a7e0 con 0x7f17f415f3c0 2024-04-25T00:40:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17fa44d640 1 -- 172.21.15.29:0/4092117366 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f4069270 con 0x7f17f415f3c0 2024-04-25T00:40:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f0ff9640 1 -- 172.21.15.29:0/4092117366 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f17ec00e3e0 con 0x7f17f415f3c0 2024-04-25T00:40:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f0ff9640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17e00680a0 0x7f17e006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f0ff9640 1 -- 172.21.15.29:0/4092117366 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f17ec085460 con 0x7f17f415f3c0 2024-04-25T00:40:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f3fff640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17e00680a0 0x7f17e006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.189+0000 7f17f3fff640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17e00680a0 0x7f17e006a560 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f17e40b1000 tx=0x7f17e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.193+0000 7f17f0ff9640 1 -- 172.21.15.29:0/4092117366 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f17ec014030 con 0x7f17f415f3c0 2024-04-25T00:40:21.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.361+0000 7f17fa44d640 1 -- 172.21.15.29:0/4092117366 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f17f406d960 con 0x7f17f415f3c0 2024-04-25T00:40:21.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.365+0000 7f17f0ff9640 1 -- 172.21.15.29:0/4092117366 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f17ec057890 con 0x7f17f415f3c0 2024-04-25T00:40:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.369+0000 7f17fa44d640 1 -- 172.21.15.29:0/4092117366 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17e00680a0 msgr2=0x7f17e006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.369+0000 7f17fa44d640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17e00680a0 0x7f17e006a560 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f17e40b1000 tx=0x7f17e4008040 comp rx=0 tx=0).stop 2024-04-25T00:40:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.369+0000 7f17fa44d640 1 -- 172.21.15.29:0/4092117366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f415f3c0 msgr2=0x7f17f40b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.369+0000 7f17fa44d640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f415f3c0 0x7f17f40b2ea0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00cd80 tx=0x7f17ec00a650 comp rx=0 tx=0).stop 2024-04-25T00:40:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.369+0000 7f17fa44d640 1 -- 172.21.15.29:0/4092117366 shutdown_connections 2024-04-25T00:40:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.369+0000 7f17fa44d640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f17e00680a0 0x7f17e006a560 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.369+0000 7f17fa44d640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f415f3c0 0x7f17f40b2ea0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.369+0000 7f17fa44d640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f415e8d0 0x7f17f40b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.369+0000 7f17fa44d640 1 --2- 172.21.15.29:0/4092117366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17f40b4780 0x7f17f4147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.369+0000 7f17fa44d640 1 -- 172.21.15.29:0/4092117366 >> 172.21.15.29:0/4092117366 conn(0x7f17f40b1870 msgr2=0x7f17f413c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:21.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.369+0000 7f17fa44d640 1 -- 172.21.15.29:0/4092117366 shutdown_connections 2024-04-25T00:40:21.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.369+0000 7f17fa44d640 1 -- 172.21.15.29:0/4092117366 wait complete. 2024-04-25T00:40:21.386 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-04-25T00:40:21.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78bb844640 1 Processor -- start 2024-04-25T00:40:21.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78bb844640 1 -- start start 2024-04-25T00:40:21.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78bb844640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b406e440 0x7f78b40b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78bb844640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78b40b41e0 con 0x7f78b40b3140 2024-04-25T00:40:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78bb844640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78b406d960 con 0x7f78b40b3c30 2024-04-25T00:40:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78bb844640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f78b40b5050 con 0x7f78b406e440 2024-04-25T00:40:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78b95e0640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f78b40b3140 0x7f78b40b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38316/0 (socket says 172.21.15.29:38316) 2024-04-25T00:40:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78b95e0640 1 -- 172.21.15.29:0/2535923140 learned_addr learned my addr 172.21.15.29:0/2535923140 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78b9de1640 1 --2- 172.21.15.29:0/2535923140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b406e440 0x7f78b40b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1744406382 0 0) 0x7f78b40b41e0 con 0x7f78b40b3140 2024-04-25T00:40:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78a0003530 con 0x7f78b40b3140 2024-04-25T00:40:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2839624363 0 0) 0x7f78a0003530 con 0x7f78b40b3140 2024-04-25T00:40:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b406e440 msgr2=0x7f78b40b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 --2- 172.21.15.29:0/2535923140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b406e440 0x7f78b40b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 >> v1:172.21.15.157:6789/0 conn(0x7f78b40b3c30 legacy=0x7f78b40ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b406dc40 con 0x7f78b40b3140 2024-04-25T00:40:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f78b0001680 con 0x7f78b40b3140 2024-04-25T00:40:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 >> v1:172.21.15.29:6789/0 conn(0x7f78b40b3140 legacy=0x7f78b40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 --2- 172.21.15.29:0/2535923140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78a00039f0 0x7f78a0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 --2- 172.21.15.29:0/2535923140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78a0004810 0x7f78a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 --2- 172.21.15.29:0/2535923140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b406e440 0x7f78a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f78b40b5050 con 0x7f78a00039f0 2024-04-25T00:40:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78a000ba80 con 0x7f78a0004810 2024-04-25T00:40:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f78a000bc80 con 0x7f78b406e440 2024-04-25T00:40:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78b95e0640 1 --2- 172.21.15.29:0/2535923140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78a00039f0 0x7f78a0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78b9de1640 1 --2- 172.21.15.29:0/2535923140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b406e440 0x7f78a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78b8ddf640 1 --2- 172.21.15.29:0/2535923140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78a0004810 0x7f78a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78b95e0640 1 -- 172.21.15.29:0/2535923140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b406e440 msgr2=0x7f78a00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78b95e0640 1 --2- 172.21.15.29:0/2535923140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b406e440 0x7f78a00051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78b95e0640 1 -- 172.21.15.29:0/2535923140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78a0004810 msgr2=0x7f78a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:21.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78b95e0640 1 --2- 172.21.15.29:0/2535923140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78a0004810 0x7f78a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78b95e0640 1 -- 172.21.15.29:0/2535923140 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a0003530 con 0x7f78a00039f0 2024-04-25T00:40:21.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.513+0000 7f78b8ddf640 1 --2- 172.21.15.29:0/2535923140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78a0004810 0x7f78a0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:21.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.517+0000 7f78b95e0640 1 --2- 172.21.15.29:0/2535923140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78a00039f0 0x7f78a0003e10 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f78b0001ec0 tx=0x7f78b001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede7bf20dbcee8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:21.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.517+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0007d40 con 0x7f78a00039f0 2024-04-25T00:40:21.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.517+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b00a9070 con 0x7f78a00039f0 2024-04-25T00:40:21.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.517+0000 7f78abfff640 1 -- 172.21.15.29:0/2535923140 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0005d40 con 0x7f78a00039f0 2024-04-25T00:40:21.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.517+0000 7f78bb844640 1 -- 172.21.15.29:0/2535923140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78a00039f0 msgr2=0x7f78a0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.517+0000 7f78bb844640 1 --2- 172.21.15.29:0/2535923140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78a00039f0 0x7f78a0003e10 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f78b0001ec0 tx=0x7f78b001cc40 comp rx=0 tx=0).stop 2024-04-25T00:40:21.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.517+0000 7f78b9de1640 1 -- 172.21.15.29:0/2535923140 reap_dead start 2024-04-25T00:40:21.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.517+0000 7f78bb844640 1 -- 172.21.15.29:0/2535923140 shutdown_connections 2024-04-25T00:40:21.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.517+0000 7f78bb844640 1 -- 172.21.15.29:0/2535923140 >> 172.21.15.29:0/2535923140 conn(0x7f78b40b1870 msgr2=0x7f78b40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:21.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.517+0000 7f78bb844640 1 -- 172.21.15.29:0/2535923140 shutdown_connections 2024-04-25T00:40:21.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.517+0000 7f78bb844640 1 -- 172.21.15.29:0/2535923140 wait complete. 2024-04-25T00:40:21.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.517+0000 7f78bb844640 1 Processor -- start 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.517+0000 7f78bb844640 1 -- start start 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78bb844640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4138ed0 0x7f78b4141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78bb844640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78b414d370 0x7f78b4141c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78bb844640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b41573f0 0x7f78b4157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78bb844640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f78b4167d80 con 0x7f78b414d370 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78bb844640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78b4167c00 con 0x7f78b4138ed0 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78bb844640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f78b4167f00 con 0x7f78b41573f0 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b8ddf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78b414d370 0x7f78b4141c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b8ddf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78b414d370 0x7f78b4141c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53054/0 (socket says 172.21.15.29:53054) 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b8ddf640 1 -- 172.21.15.29:0/2124058752 learned_addr learned my addr 172.21.15.29:0/2124058752 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b95e0640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4138ed0 0x7f78b4141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b9de1640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b41573f0 0x7f78b4157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b8ddf640 1 -- 172.21.15.29:0/2124058752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b41573f0 msgr2=0x7f78b4157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b8ddf640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b41573f0 0x7f78b4157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b8ddf640 1 -- 172.21.15.29:0/2124058752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4138ed0 msgr2=0x7f78b4141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b8ddf640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4138ed0 0x7f78b4141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b8ddf640 1 -- 172.21.15.29:0/2124058752 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b414f7a0 con 0x7f78b414d370 2024-04-25T00:40:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b95e0640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4138ed0 0x7f78b4141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b8ddf640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78b414d370 0x7f78b4141c40 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f78a400d540 tx=0x7f78a400da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78a9ffb640 1 -- 172.21.15.29:0/2124058752 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4005480 con 0x7f78b414d370 2024-04-25T00:40:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78a9ffb640 1 -- 172.21.15.29:0/2124058752 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78a4017070 con 0x7f78b414d370 2024-04-25T00:40:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78bb844640 1 -- 172.21.15.29:0/2124058752 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b414fa80 con 0x7f78b414d370 2024-04-25T00:40:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78a9ffb640 1 -- 172.21.15.29:0/2124058752 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4003840 con 0x7f78b414d370 2024-04-25T00:40:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78bb844640 1 -- 172.21.15.29:0/2124058752 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b4150040 con 0x7f78b414d370 2024-04-25T00:40:21.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78bb844640 1 -- 172.21.15.29:0/2124058752 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b4069270 con 0x7f78b414d370 2024-04-25T00:40:21.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78a9ffb640 1 -- 172.21.15.29:0/2124058752 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f78a40039e0 con 0x7f78b414d370 2024-04-25T00:40:21.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78a9ffb640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f78980680d0 0x7f789806a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:21.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78a9ffb640 1 -- 172.21.15.29:0/2124058752 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f78a4084480 con 0x7f78b414d370 2024-04-25T00:40:21.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b95e0640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f78980680d0 0x7f789806a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:21.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.521+0000 7f78b95e0640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f78980680d0 0x7f789806a590 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f78b00ab000 tx=0x7f78b00a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:21.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.525+0000 7f78a9ffb640 1 -- 172.21.15.29:0/2124058752 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f78a400f2f0 con 0x7f78b414d370 2024-04-25T00:40:21.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:21.749+0000 7f78bb844640 1 -- 172.21.15.29:0/2124058752 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f78b415b0b0 con 0x7f78b414d370 2024-04-25T00:40:22.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.017+0000 7f78a9ffb640 1 -- 172.21.15.29:0/2124058752 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v475) ==== 149+0+0 (secure 0 0 0) 0x7f78a40568b0 con 0x7f78b414d370 2024-04-25T00:40:22.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.021+0000 7f78bb844640 1 -- 172.21.15.29:0/2124058752 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f78980680d0 msgr2=0x7f789806a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.021+0000 7f78bb844640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f78980680d0 0x7f789806a590 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f78b00ab000 tx=0x7f78b00a7070 comp rx=0 tx=0).stop 2024-04-25T00:40:22.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.021+0000 7f78bb844640 1 -- 172.21.15.29:0/2124058752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78b414d370 msgr2=0x7f78b4141c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.021+0000 7f78bb844640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78b414d370 0x7f78b4141c40 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f78a400d540 tx=0x7f78a400da00 comp rx=0 tx=0).stop 2024-04-25T00:40:22.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.021+0000 7f78bb844640 1 -- 172.21.15.29:0/2124058752 shutdown_connections 2024-04-25T00:40:22.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.021+0000 7f78bb844640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f78980680d0 0x7f789806a590 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.021+0000 7f78bb844640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b41573f0 0x7f78b4157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.021+0000 7f78bb844640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78b414d370 0x7f78b4141c40 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.021+0000 7f78bb844640 1 --2- 172.21.15.29:0/2124058752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4138ed0 0x7f78b4141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.021+0000 7f78bb844640 1 -- 172.21.15.29:0/2124058752 >> 172.21.15.29:0/2124058752 conn(0x7f78b40b1870 msgr2=0x7f78b415b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:22.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.021+0000 7f78bb844640 1 -- 172.21.15.29:0/2124058752 shutdown_connections 2024-04-25T00:40:22.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.021+0000 7f78bb844640 1 -- 172.21.15.29:0/2124058752 wait complete. 2024-04-25T00:40:22.044 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2024-04-25T00:40:22.044 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-04-25T00:40:22.044 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-04-25T00:40:22.045 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:40:22.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc751b6b640 1 Processor -- start 2024-04-25T00:40:22.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc751b6b640 1 -- start start 2024-04-25T00:40:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc751b6b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc74c0cf690 0x7fc74c0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc751b6b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc74c0d2a90 0x7fc74c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc751b6b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc74c0d2ea0 con 0x7fc74c15e200 2024-04-25T00:40:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc751b6b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc74c0b38d0 con 0x7fc74c0d2a90 2024-04-25T00:40:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc751b6b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc74c0cfc10 con 0x7fc74c0cf690 2024-04-25T00:40:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc74bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc74c0d2a90 0x7fc74c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc74bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc74c0d2a90 0x7fc74c0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53058/0 (socket says 172.21.15.29:53058) 2024-04-25T00:40:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc74bfff640 1 -- 172.21.15.29:0/2197991847 learned_addr learned my addr 172.21.15.29:0/2197991847 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:22.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc74affd640 1 --2- 172.21.15.29:0/2197991847 >> v2:172.21.15.157:3300/0 conn(0x7fc74c0cf690 0x7fc74c0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc74bfff640 1 -- 172.21.15.29:0/2197991847 >> v2:172.21.15.157:3300/0 conn(0x7fc74c0cf690 msgr2=0x7fc74c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:22.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc74bfff640 1 --2- 172.21.15.29:0/2197991847 >> v2:172.21.15.157:3300/0 conn(0x7fc74c0cf690 0x7fc74c0d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc74bfff640 1 -- 172.21.15.29:0/2197991847 >> v1:172.21.15.157:6789/0 conn(0x7fc74c15e200 legacy=0x7fc74c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.177+0000 7fc74bfff640 1 -- 172.21.15.29:0/2197991847 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74c0cfef0 con 0x7fc74c0d2a90 2024-04-25T00:40:22.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc74bfff640 1 --2- 172.21.15.29:0/2197991847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc74c0d2a90 0x7fc74c0b4320 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0d24a0 tx=0x7fc74401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983ff819be756d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:22.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc74a7fc640 1 -- 172.21.15.29:0/2197991847 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7440a57a0 con 0x7fc74c0d2a90 2024-04-25T00:40:22.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc74a7fc640 1 -- 172.21.15.29:0/2197991847 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7440a5940 con 0x7fc74c0d2a90 2024-04-25T00:40:22.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc74a7fc640 1 -- 172.21.15.29:0/2197991847 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7440a5d10 con 0x7fc74c0d2a90 2024-04-25T00:40:22.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc751b6b640 1 -- 172.21.15.29:0/2197991847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc74c0d2a90 msgr2=0x7fc74c0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc751b6b640 1 --2- 172.21.15.29:0/2197991847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc74c0d2a90 0x7fc74c0b4320 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0d24a0 tx=0x7fc74401c9f0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc751b6b640 1 -- 172.21.15.29:0/2197991847 shutdown_connections 2024-04-25T00:40:22.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc751b6b640 1 --2- 172.21.15.29:0/2197991847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc74c0d2a90 0x7fc74c0b4320 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc751b6b640 1 --2- 172.21.15.29:0/2197991847 >> v2:172.21.15.157:3300/0 conn(0x7fc74c0cf690 0x7fc74c0d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc751b6b640 1 -- 172.21.15.29:0/2197991847 >> 172.21.15.29:0/2197991847 conn(0x7fc74c09d280 msgr2=0x7fc74c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:22.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc751b6b640 1 -- 172.21.15.29:0/2197991847 shutdown_connections 2024-04-25T00:40:22.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc751b6b640 1 -- 172.21.15.29:0/2197991847 wait complete. 2024-04-25T00:40:22.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc751b6b640 1 Processor -- start 2024-04-25T00:40:22.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.181+0000 7fc751b6b640 1 -- start start 2024-04-25T00:40:22.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc751b6b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc74c0cf690 0x7fc74c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc751b6b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc74c0d2a90 0x7fc74c14c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc751b6b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc74c15e200 0x7fc74c0d0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc751b6b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c0b51f0 con 0x7fc74c0d2a90 2024-04-25T00:40:22.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc751b6b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c0b5070 con 0x7fc74c15e200 2024-04-25T00:40:22.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc751b6b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc74c0b5370 con 0x7fc74c0cf690 2024-04-25T00:40:22.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc74c0d2a90 0x7fc74c14c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc74c0d2a90 0x7fc74c14c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53062/0 (socket says 172.21.15.29:53062) 2024-04-25T00:40:22.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74affd640 1 -- 172.21.15.29:0/431341948 learned_addr learned my addr 172.21.15.29:0/431341948 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:22.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74b7fe640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc74c0cf690 0x7fc74c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74bfff640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc74c15e200 0x7fc74c0d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74affd640 1 -- 172.21.15.29:0/431341948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc74c0cf690 msgr2=0x7fc74c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74affd640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc74c0cf690 0x7fc74c15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74affd640 1 -- 172.21.15.29:0/431341948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc74c15e200 msgr2=0x7fc74c0d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74affd640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc74c15e200 0x7fc74c0d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74affd640 1 -- 172.21.15.29:0/431341948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74c15c070 con 0x7fc74c0d2a90 2024-04-25T00:40:22.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74b7fe640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc74c0cf690 0x7fc74c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:22.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74affd640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc74c0d2a90 0x7fc74c14c940 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fc74000e930 tx=0x7fc74000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:22.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc733fff640 1 -- 172.21.15.29:0/431341948 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740017070 con 0x7fc74c0d2a90 2024-04-25T00:40:22.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc733fff640 1 -- 172.21.15.29:0/431341948 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc74000ca00 con 0x7fc74c0d2a90 2024-04-25T00:40:22.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc751b6b640 1 -- 172.21.15.29:0/431341948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc74c14d5b0 con 0x7fc74c0d2a90 2024-04-25T00:40:22.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc733fff640 1 -- 172.21.15.29:0/431341948 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74000cdb0 con 0x7fc74c0d2a90 2024-04-25T00:40:22.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc751b6b640 1 -- 172.21.15.29:0/431341948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc74c14db70 con 0x7fc74c0d2a90 2024-04-25T00:40:22.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc751b6b640 1 -- 172.21.15.29:0/431341948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc74c069270 con 0x7fc74c0d2a90 2024-04-25T00:40:22.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc733fff640 1 -- 172.21.15.29:0/431341948 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fc74001a070 con 0x7fc74c0d2a90 2024-04-25T00:40:22.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc733fff640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc7380680a0 0x7fc73806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc733fff640 1 -- 172.21.15.29:0/431341948 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7fc7400846a0 con 0x7fc74c0d2a90 2024-04-25T00:40:22.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74b7fe640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc7380680a0 0x7fc73806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.185+0000 7fc74b7fe640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc7380680a0 0x7fc73806a560 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc73c001650 tx=0x7fc73c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:22.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.189+0000 7fc733fff640 1 -- 172.21.15.29:0/431341948 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc740056b90 con 0x7fc74c0d2a90 2024-04-25T00:40:22.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.349+0000 7fc751b6b640 1 -- 172.21.15.29:0/431341948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc74c0d0460 con 0x7fc74c0d2a90 2024-04-25T00:40:22.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.353+0000 7fc733fff640 1 -- 172.21.15.29:0/431341948 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+12980 (secure 0 0 0) 0x7fc74007de20 con 0x7fc74c0d2a90 2024-04-25T00:40:22.353 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:22.353 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":148,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:40:21.014151+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6835/821427804":"2024-04-26T00:40:18.001837+0000","172.21.15.29:6834/821427804":"2024-04-26T00:40:18.001837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:40:22.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.353+0000 7fc751b6b640 1 -- 172.21.15.29:0/431341948 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc7380680a0 msgr2=0x7fc73806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.353+0000 7fc751b6b640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc7380680a0 0x7fc73806a560 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc73c001650 tx=0x7fc73c009040 comp rx=0 tx=0).stop 2024-04-25T00:40:22.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.353+0000 7fc751b6b640 1 -- 172.21.15.29:0/431341948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc74c0d2a90 msgr2=0x7fc74c14c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.353+0000 7fc751b6b640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc74c0d2a90 0x7fc74c14c940 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fc74000e930 tx=0x7fc74000edf0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.353+0000 7fc751b6b640 1 -- 172.21.15.29:0/431341948 shutdown_connections 2024-04-25T00:40:22.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.353+0000 7fc751b6b640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc7380680a0 0x7fc73806a560 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.353+0000 7fc751b6b640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc74c15e200 0x7fc74c0d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.353+0000 7fc751b6b640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc74c0d2a90 0x7fc74c14c940 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.353+0000 7fc751b6b640 1 --2- 172.21.15.29:0/431341948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc74c0cf690 0x7fc74c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.353+0000 7fc751b6b640 1 -- 172.21.15.29:0/431341948 >> 172.21.15.29:0/431341948 conn(0x7fc74c09d280 msgr2=0x7fc74c0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:22.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.357+0000 7fc751b6b640 1 -- 172.21.15.29:0/431341948 shutdown_connections 2024-04-25T00:40:22.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.357+0000 7fc751b6b640 1 -- 172.21.15.29:0/431341948 wait complete. 2024-04-25T00:40:22.372 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:40:22.372 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:40:22.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dda576640 1 Processor -- start 2024-04-25T00:40:22.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dda576640 1 -- start start 2024-04-25T00:40:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dda576640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9dd415e200 0x7f9dd40d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dda576640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40cf690 0x7f9dd40d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dda576640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dd40cfc10 con 0x7f9dd40d2a90 2024-04-25T00:40:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dda576640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd40d2ee0 con 0x7f9dd415e200 2024-04-25T00:40:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dda576640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd40b38d0 con 0x7f9dd40cf690 2024-04-25T00:40:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd8b13640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9dd40d2a90 0x7f9dd40d05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38320/0 (socket says 172.21.15.29:38320) 2024-04-25T00:40:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd8b13640 1 -- 172.21.15.29:0/3073602154 learned_addr learned my addr 172.21.15.29:0/3073602154 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd37fe640 1 --2- 172.21.15.29:0/3073602154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40cf690 0x7f9dd40d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd3fff640 1 --2- 172.21.15.29:0/3073602154 >> v2:172.21.15.157:3300/0 conn(0x7f9dd415e200 0x7f9dd40d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4044744281 0 0) 0x7f9dd40cfc10 con 0x7f9dd40d2a90 2024-04-25T00:40:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9dbc003530 con 0x7f9dd40d2a90 2024-04-25T00:40:22.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2011492839 0 0) 0x7f9dbc003530 con 0x7f9dd40d2a90 2024-04-25T00:40:22.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40cf690 msgr2=0x7f9dd40d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:22.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 --2- 172.21.15.29:0/3073602154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40cf690 0x7f9dd40d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 >> v2:172.21.15.157:3300/0 conn(0x7f9dd415e200 msgr2=0x7f9dd40d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:22.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 --2- 172.21.15.29:0/3073602154 >> v2:172.21.15.157:3300/0 conn(0x7f9dd415e200 0x7f9dd40d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd40cfef0 con 0x7f9dd40d2a90 2024-04-25T00:40:22.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f9dcc001a00 con 0x7f9dd40d2a90 2024-04-25T00:40:22.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 >> v1:172.21.15.29:6789/0 conn(0x7f9dd40d2a90 legacy=0x7f9dd40d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 --2- 172.21.15.29:0/3073602154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0039c0 0x7f9dbc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 --2- 172.21.15.29:0/3073602154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40cf690 0x7f9dbc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 --2- 172.21.15.29:0/3073602154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dbc004d50 0x7f9dbc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd40d2ee0 con 0x7f9dbc004d50 2024-04-25T00:40:22.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd40b38d0 con 0x7f9dbc0039c0 2024-04-25T00:40:22.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9dbc00b9d0 con 0x7f9dd40cf690 2024-04-25T00:40:22.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd8b13640 1 --2- 172.21.15.29:0/3073602154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dbc004d50 0x7f9dbc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.501+0000 7f9dd3fff640 1 --2- 172.21.15.29:0/3073602154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0039c0 0x7f9dbc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dd37fe640 1 --2- 172.21.15.29:0/3073602154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40cf690 0x7f9dbc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dd8b13640 1 -- 172.21.15.29:0/3073602154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40cf690 msgr2=0x7f9dbc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:22.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dd8b13640 1 --2- 172.21.15.29:0/3073602154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40cf690 0x7f9dbc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dd8b13640 1 -- 172.21.15.29:0/3073602154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0039c0 msgr2=0x7f9dbc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:22.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dd8b13640 1 --2- 172.21.15.29:0/3073602154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0039c0 0x7f9dbc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dd8b13640 1 -- 172.21.15.29:0/3073602154 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc003530 con 0x7f9dbc004d50 2024-04-25T00:40:22.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dd37fe640 1 --2- 172.21.15.29:0/3073602154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40cf690 0x7f9dbc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:22.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dd8b13640 1 --2- 172.21.15.29:0/3073602154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dbc004d50 0x7f9dbc0051d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00a8a0 tx=0x7f9dcc01d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63637ee224b20185 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:22.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc00a4c0 con 0x7f9dbc004d50 2024-04-25T00:40:22.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc0a9020 con 0x7f9dbc004d50 2024-04-25T00:40:22.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dd2ffd640 1 -- 172.21.15.29:0/3073602154 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0072d0 con 0x7f9dbc004d50 2024-04-25T00:40:22.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dda576640 1 -- 172.21.15.29:0/3073602154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dbc004d50 msgr2=0x7f9dbc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dda576640 1 --2- 172.21.15.29:0/3073602154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dbc004d50 0x7f9dbc0051d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00a8a0 tx=0x7f9dcc01d3c0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dd8b13640 1 -- 172.21.15.29:0/3073602154 reap_dead start 2024-04-25T00:40:22.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dda576640 1 -- 172.21.15.29:0/3073602154 shutdown_connections 2024-04-25T00:40:22.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dda576640 1 -- 172.21.15.29:0/3073602154 >> 172.21.15.29:0/3073602154 conn(0x7f9dd409d280 msgr2=0x7f9dd40d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dda576640 1 -- 172.21.15.29:0/3073602154 shutdown_connections 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.505+0000 7f9dda576640 1 -- 172.21.15.29:0/3073602154 wait complete. 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dda576640 1 Processor -- start 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dda576640 1 -- start start 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dda576640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dbc004d50 0x7f9dd40b8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dda576640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40b6eb0 0x7f9dd40cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dda576640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd4149ff0 0x7f9dd414a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dda576640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd40d4d30 con 0x7f9dbc004d50 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dda576640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd40b3e00 con 0x7f9dd4149ff0 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dda576640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd415e5e0 con 0x7f9dd40b6eb0 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dbc004d50 0x7f9dd40b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dbc004d50 0x7f9dd40b8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53076/0 (socket says 172.21.15.29:53076) 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd3fff640 1 -- 172.21.15.29:0/1300735512 learned_addr learned my addr 172.21.15.29:0/1300735512 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:22.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd8b13640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd4149ff0 0x7f9dd414a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd37fe640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40b6eb0 0x7f9dd40cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd3fff640 1 -- 172.21.15.29:0/1300735512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40b6eb0 msgr2=0x7f9dd40cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd3fff640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40b6eb0 0x7f9dd40cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd3fff640 1 -- 172.21.15.29:0/1300735512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd4149ff0 msgr2=0x7f9dd414a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd3fff640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd4149ff0 0x7f9dd414a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd3fff640 1 -- 172.21.15.29:0/1300735512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd40b9010 con 0x7f9dbc004d50 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd8b13640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd4149ff0 0x7f9dd414a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd3fff640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dbc004d50 0x7f9dd40b8880 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4011640 tx=0x7f9dc4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd0ff9640 1 -- 172.21.15.29:0/1300735512 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc400f920 con 0x7f9dbc004d50 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd0ff9640 1 -- 172.21.15.29:0/1300735512 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc400fac0 con 0x7f9dbc004d50 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dda576640 1 -- 172.21.15.29:0/1300735512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd40b92f0 con 0x7f9dbc004d50 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd0ff9640 1 -- 172.21.15.29:0/1300735512 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc400fe70 con 0x7f9dbc004d50 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dda576640 1 -- 172.21.15.29:0/1300735512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd40b98b0 con 0x7f9dbc004d50 2024-04-25T00:40:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dda576640 1 -- 172.21.15.29:0/1300735512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd415f930 con 0x7f9dbc004d50 2024-04-25T00:40:22.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd0ff9640 1 -- 172.21.15.29:0/1300735512 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f9dc4015460 con 0x7f9dbc004d50 2024-04-25T00:40:22.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.509+0000 7f9dd0ff9640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9dc00680d0 0x7f9dc006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.513+0000 7f9dd0ff9640 1 -- 172.21.15.29:0/1300735512 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f9dc4087170 con 0x7f9dbc004d50 2024-04-25T00:40:22.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.513+0000 7f9dd37fe640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9dc00680d0 0x7f9dc006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.513+0000 7f9dd37fe640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9dc00680d0 0x7f9dc006a590 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4161500 tx=0x7f9dc8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:22.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.517+0000 7f9dd0ff9640 1 -- 172.21.15.29:0/1300735512 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9dc4019030 con 0x7f9dbc004d50 2024-04-25T00:40:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.673+0000 7f9dda576640 1 -- 172.21.15.29:0/1300735512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9dd4069270 con 0x7f9dbc004d50 2024-04-25T00:40:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.673+0000 7f9dd0ff9640 1 -- 172.21.15.29:0/1300735512 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) ==== 93+0+10 (secure 0 0 0) 0x7f9dc4057430 con 0x7f9dbc004d50 2024-04-25T00:40:22.674 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:40:22.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.677+0000 7f9dda576640 1 -- 172.21.15.29:0/1300735512 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9dc00680d0 msgr2=0x7f9dc006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.677+0000 7f9dda576640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9dc00680d0 0x7f9dc006a590 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4161500 tx=0x7f9dc8008040 comp rx=0 tx=0).stop 2024-04-25T00:40:22.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.677+0000 7f9dda576640 1 -- 172.21.15.29:0/1300735512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dbc004d50 msgr2=0x7f9dd40b8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.677+0000 7f9dda576640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dbc004d50 0x7f9dd40b8880 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4011640 tx=0x7f9dc4011b00 comp rx=0 tx=0).stop 2024-04-25T00:40:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.677+0000 7f9dda576640 1 -- 172.21.15.29:0/1300735512 shutdown_connections 2024-04-25T00:40:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.677+0000 7f9dda576640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9dc00680d0 0x7f9dc006a590 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.677+0000 7f9dda576640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dd4149ff0 0x7f9dd414a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.677+0000 7f9dda576640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dd40b6eb0 0x7f9dd40cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.677+0000 7f9dda576640 1 --2- 172.21.15.29:0/1300735512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9dbc004d50 0x7f9dd40b8880 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.677+0000 7f9dda576640 1 -- 172.21.15.29:0/1300735512 >> 172.21.15.29:0/1300735512 conn(0x7f9dd409d280 msgr2=0x7f9dd40cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.677+0000 7f9dda576640 1 -- 172.21.15.29:0/1300735512 shutdown_connections 2024-04-25T00:40:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.677+0000 7f9dda576640 1 -- 172.21.15.29:0/1300735512 wait complete. 2024-04-25T00:40:22.694 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-04-25T00:40:22.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.813+0000 7f80ad720640 1 Processor -- start 2024-04-25T00:40:22.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.813+0000 7f80ad720640 1 -- start start 2024-04-25T00:40:22.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80ad720640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f80a815e8d0 0x7f80a80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80ad720640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f80a815f3c0 0x7f80a815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80ad720640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a80b3720 con 0x7f80a80b4780 2024-04-25T00:40:22.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80ad720640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f80a80b42f0 con 0x7f80a815f3c0 2024-04-25T00:40:22.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80ad720640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f80a80b39c0 con 0x7f80a815e8d0 2024-04-25T00:40:22.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80a77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f80a815f3c0 0x7f80a815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80a77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f80a815f3c0 0x7f80a815c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53092/0 (socket says 172.21.15.29:53092) 2024-04-25T00:40:22.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80a77fe640 1 -- 172.21.15.29:0/580097469 learned_addr learned my addr 172.21.15.29:0/580097469 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:22.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80a6ffd640 1 --2- 172.21.15.29:0/580097469 >> v2:172.21.15.157:3300/0 conn(0x7f80a815e8d0 0x7f80a80cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80a77fe640 1 -- 172.21.15.29:0/580097469 >> v2:172.21.15.157:3300/0 conn(0x7f80a815e8d0 msgr2=0x7f80a80cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:22.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80a77fe640 1 --2- 172.21.15.29:0/580097469 >> v2:172.21.15.157:3300/0 conn(0x7f80a815e8d0 0x7f80a80cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80a77fe640 1 -- 172.21.15.29:0/580097469 >> v1:172.21.15.157:6789/0 conn(0x7f80a80b4780 legacy=0x7f80a815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80a77fe640 1 -- 172.21.15.29:0/580097469 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a80b3ca0 con 0x7f80a815f3c0 2024-04-25T00:40:22.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80a77fe640 1 --2- 172.21.15.29:0/580097469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80a815f3c0 0x7f80a815c7d0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f80a80b5260 tx=0x7f80a001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22866932c2afe6cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:22.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80a5ffb640 1 -- 172.21.15.29:0/580097469 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a00a5790 con 0x7f80a815f3c0 2024-04-25T00:40:22.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80a5ffb640 1 -- 172.21.15.29:0/580097469 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a00a5930 con 0x7f80a815f3c0 2024-04-25T00:40:22.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.817+0000 7f80a5ffb640 1 -- 172.21.15.29:0/580097469 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a00a5d00 con 0x7f80a815f3c0 2024-04-25T00:40:22.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 -- 172.21.15.29:0/580097469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80a815f3c0 msgr2=0x7f80a815c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 --2- 172.21.15.29:0/580097469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80a815f3c0 0x7f80a815c7d0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f80a80b5260 tx=0x7f80a001c9f0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 -- 172.21.15.29:0/580097469 shutdown_connections 2024-04-25T00:40:22.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 --2- 172.21.15.29:0/580097469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80a815f3c0 0x7f80a815c7d0 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 --2- 172.21.15.29:0/580097469 >> v2:172.21.15.157:3300/0 conn(0x7f80a815e8d0 0x7f80a80cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 -- 172.21.15.29:0/580097469 >> 172.21.15.29:0/580097469 conn(0x7f80a80b1870 msgr2=0x7f80a80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:22.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 -- 172.21.15.29:0/580097469 shutdown_connections 2024-04-25T00:40:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 -- 172.21.15.29:0/580097469 wait complete. 2024-04-25T00:40:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 Processor -- start 2024-04-25T00:40:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 -- start start 2024-04-25T00:40:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a80b4780 0x7f80a813c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a815e8d0 0x7f80a8160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80a815f3c0 0x7f80a8161460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f80a815d6a0 con 0x7f80a815f3c0 2024-04-25T00:40:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f80a815d520 con 0x7f80a815e8d0 2024-04-25T00:40:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f80a8167c70 con 0x7f80a80b4780 2024-04-25T00:40:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80a77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80a815f3c0 0x7f80a8161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80a77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80a815f3c0 0x7f80a8161460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53096/0 (socket says 172.21.15.29:53096) 2024-04-25T00:40:22.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80a77fe640 1 -- 172.21.15.29:0/408540916 learned_addr learned my addr 172.21.15.29:0/408540916 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:22.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80a6ffd640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a80b4780 0x7f80a813c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80a67fc640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a815e8d0 0x7f80a8160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80a77fe640 1 -- 172.21.15.29:0/408540916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a80b4780 msgr2=0x7f80a813c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80a77fe640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a80b4780 0x7f80a813c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80a77fe640 1 -- 172.21.15.29:0/408540916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a815e8d0 msgr2=0x7f80a8160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:22.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80a77fe640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a815e8d0 0x7f80a8160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:22.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80a77fe640 1 -- 172.21.15.29:0/408540916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a813ce80 con 0x7f80a815f3c0 2024-04-25T00:40:22.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80a6ffd640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a80b4780 0x7f80a813c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:22.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80a67fc640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a815e8d0 0x7f80a8160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:22.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80a77fe640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80a815f3c0 0x7f80a8161460 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f80a0019600 tx=0x7f80a0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:22.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f808f7fe640 1 -- 172.21.15.29:0/408540916 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a00bc020 con 0x7f80a815f3c0 2024-04-25T00:40:22.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f808f7fe640 1 -- 172.21.15.29:0/408540916 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a00b8690 con 0x7f80a815f3c0 2024-04-25T00:40:22.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 -- 172.21.15.29:0/408540916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a813b0f0 con 0x7f80a815f3c0 2024-04-25T00:40:22.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f808f7fe640 1 -- 172.21.15.29:0/408540916 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a00b8a80 con 0x7f80a815f3c0 2024-04-25T00:40:22.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.821+0000 7f80ad720640 1 -- 172.21.15.29:0/408540916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a813b6b0 con 0x7f80a815f3c0 2024-04-25T00:40:22.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.825+0000 7f80ad720640 1 -- 172.21.15.29:0/408540916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a8069270 con 0x7f80a815f3c0 2024-04-25T00:40:22.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.825+0000 7f808f7fe640 1 -- 172.21.15.29:0/408540916 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f80a00bf020 con 0x7f80a815f3c0 2024-04-25T00:40:22.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.825+0000 7f808f7fe640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f80940680a0 0x7f809406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:22.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.825+0000 7f808f7fe640 1 -- 172.21.15.29:0/408540916 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f80a011f760 con 0x7f80a815f3c0 2024-04-25T00:40:22.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.825+0000 7f80a6ffd640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f80940680a0 0x7f809406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:22.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.825+0000 7f80a6ffd640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f80940680a0 0x7f809406a560 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8098007a70 tx=0x7f8098008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:22.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:22.829+0000 7f808f7fe640 1 -- 172.21.15.29:0/408540916 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f80a00c6030 con 0x7f80a815f3c0 2024-04-25T00:40:23.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:23.045+0000 7f80ad720640 1 -- 172.21.15.29:0/408540916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7f80a806e7b0 con 0x7f80a815f3c0 2024-04-25T00:40:24.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.025+0000 7f808f7fe640 1 -- 172.21.15.29:0/408540916 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v477) ==== 148+0+0 (secure 0 0 0) 0x7f80a00165e0 con 0x7f80a815f3c0 2024-04-25T00:40:24.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.029+0000 7f80ad720640 1 -- 172.21.15.29:0/408540916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f80940680a0 msgr2=0x7f809406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.029+0000 7f80ad720640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f80940680a0 0x7f809406a560 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8098007a70 tx=0x7f8098008040 comp rx=0 tx=0).stop 2024-04-25T00:40:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.029+0000 7f80ad720640 1 -- 172.21.15.29:0/408540916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80a815f3c0 msgr2=0x7f80a8161460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.029+0000 7f80ad720640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80a815f3c0 0x7f80a8161460 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f80a0019600 tx=0x7f80a0019790 comp rx=0 tx=0).stop 2024-04-25T00:40:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.029+0000 7f80ad720640 1 -- 172.21.15.29:0/408540916 shutdown_connections 2024-04-25T00:40:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.029+0000 7f80ad720640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f80940680a0 0x7f809406a560 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.029+0000 7f80ad720640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80a815f3c0 0x7f80a8161460 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.029+0000 7f80ad720640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a815e8d0 0x7f80a8160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.029+0000 7f80ad720640 1 --2- 172.21.15.29:0/408540916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a80b4780 0x7f80a813c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.029+0000 7f80ad720640 1 -- 172.21.15.29:0/408540916 >> 172.21.15.29:0/408540916 conn(0x7f80a80b1870 msgr2=0x7f80a80d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.029+0000 7f80ad720640 1 -- 172.21.15.29:0/408540916 shutdown_connections 2024-04-25T00:40:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.029+0000 7f80ad720640 1 -- 172.21.15.29:0/408540916 wait complete. 2024-04-25T00:40:24.050 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:40:24.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.181+0000 7fc2cbe52640 1 Processor -- start 2024-04-25T00:40:24.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.181+0000 7fc2cbe52640 1 -- start start 2024-04-25T00:40:24.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.181+0000 7fc2cbe52640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc2c40b4d30 0x7fc2c40b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:24.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.181+0000 7fc2cbe52640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2c406d6c0 con 0x7fc2c415df70 2024-04-25T00:40:24.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.181+0000 7fc2cbe52640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2c406d980 con 0x7fc2c406d210 2024-04-25T00:40:24.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.181+0000 7fc2cbe52640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc2c406dc60 con 0x7fc2c40b4d30 2024-04-25T00:40:24.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.181+0000 7fc2c93ed640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc2c40b4d30 0x7fc2c40b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:24.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.181+0000 7fc2c93ed640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc2c40b4d30 0x7fc2c40b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53100/0 (socket says 172.21.15.29:53100) 2024-04-25T00:40:24.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.181+0000 7fc2c93ed640 1 -- 172.21.15.29:0/986872443 learned_addr learned my addr 172.21.15.29:0/986872443 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:24.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.181+0000 7fc2c93ed640 1 -- 172.21.15.29:0/986872443 >> v1:172.21.15.157:6789/0 conn(0x7fc2c406d210 legacy=0x7fc2c40ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:24.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.181+0000 7fc2c93ed640 1 -- 172.21.15.29:0/986872443 >> v1:172.21.15.29:6789/0 conn(0x7fc2c415df70 legacy=0x7fc2c40b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.181+0000 7fc2c93ed640 1 -- 172.21.15.29:0/986872443 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c406df40 con 0x7fc2c40b4d30 2024-04-25T00:40:24.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.181+0000 7fc2c8bec640 1 -- 172.21.15.29:0/986872443 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2142510051 0 0) 0x7fc2c406d6c0 con 0x7fc2c415df70 2024-04-25T00:40:24.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2c93ed640 1 --2- 172.21.15.29:0/986872443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c40b4d30 0x7fc2c40b3e40 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009870 tx=0x7fc2b401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe85f41e1ed211e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:24.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2c8bec640 1 -- 172.21.15.29:0/986872443 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40aa650 con 0x7fc2c40b4d30 2024-04-25T00:40:24.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2c8bec640 1 -- 172.21.15.29:0/986872443 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b40af070 con 0x7fc2c40b4d30 2024-04-25T00:40:24.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2c8bec640 1 -- 172.21.15.29:0/986872443 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40aaa10 con 0x7fc2c40b4d30 2024-04-25T00:40:24.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 -- 172.21.15.29:0/986872443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c40b4d30 msgr2=0x7fc2c40b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 --2- 172.21.15.29:0/986872443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c40b4d30 0x7fc2c40b3e40 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009870 tx=0x7fc2b401c4f0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 -- 172.21.15.29:0/986872443 shutdown_connections 2024-04-25T00:40:24.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 --2- 172.21.15.29:0/986872443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c40b4d30 0x7fc2c40b3e40 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 -- 172.21.15.29:0/986872443 >> 172.21.15.29:0/986872443 conn(0x7fc2c40b1870 msgr2=0x7fc2c40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:24.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 -- 172.21.15.29:0/986872443 shutdown_connections 2024-04-25T00:40:24.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 -- 172.21.15.29:0/986872443 wait complete. 2024-04-25T00:40:24.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 Processor -- start 2024-04-25T00:40:24.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 -- start start 2024-04-25T00:40:24.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c406d210 0x7fc2c40cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:24.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c40b4d30 0x7fc2c41fa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:24.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2c415df70 0x7fc2c41fa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:24.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c406eba0 con 0x7fc2c406d210 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c406ea20 con 0x7fc2c40b4d30 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc2c416bbc0 con 0x7fc2c415df70 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2c9bee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c406d210 0x7fc2c40cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2c9bee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c406d210 0x7fc2c40cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53106/0 (socket says 172.21.15.29:53106) 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2c9bee640 1 -- 172.21.15.29:0/1031625275 learned_addr learned my addr 172.21.15.29:0/1031625275 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2c9bee640 1 -- 172.21.15.29:0/1031625275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2c415df70 msgr2=0x7fc2c41fa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2c9bee640 1 --2- 172.21.15.29:0/1031625275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2c415df70 0x7fc2c41fa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2c9bee640 1 -- 172.21.15.29:0/1031625275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c40b4d30 msgr2=0x7fc2c41fa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2c9bee640 1 --2- 172.21.15.29:0/1031625275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c40b4d30 0x7fc2c41fa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2c9bee640 1 -- 172.21.15.29:0/1031625275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c4157bc0 con 0x7fc2c406d210 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2c9bee640 1 --2- 172.21.15.29:0/1031625275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c406d210 0x7fc2c40cbd80 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0001ec0 tx=0x7fc2c000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2ba7fc640 1 -- 172.21.15.29:0/1031625275 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0017070 con 0x7fc2c406d210 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2ba7fc640 1 -- 172.21.15.29:0/1031625275 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c0003600 con 0x7fc2c406d210 2024-04-25T00:40:24.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 -- 172.21.15.29:0/1031625275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c40afcd0 con 0x7fc2c406d210 2024-04-25T00:40:24.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2ba7fc640 1 -- 172.21.15.29:0/1031625275 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00039f0 con 0x7fc2c406d210 2024-04-25T00:40:24.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.185+0000 7fc2cbe52640 1 -- 172.21.15.29:0/1031625275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c40b0530 con 0x7fc2c406d210 2024-04-25T00:40:24.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.189+0000 7fc2cbe52640 1 -- 172.21.15.29:0/1031625275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c4069270 con 0x7fc2c406d210 2024-04-25T00:40:24.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.189+0000 7fc2ba7fc640 1 -- 172.21.15.29:0/1031625275 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fc2c001a070 con 0x7fc2c406d210 2024-04-25T00:40:24.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.189+0000 7fc2ba7fc640 1 --2- 172.21.15.29:0/1031625275 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2a80680a0 0x7fc2a806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:24.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.189+0000 7fc2ba7fc640 1 -- 172.21.15.29:0/1031625275 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7fc2c00839f0 con 0x7fc2c406d210 2024-04-25T00:40:24.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.189+0000 7fc2c93ed640 1 --2- 172.21.15.29:0/1031625275 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2a80680a0 0x7fc2a806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:24.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.189+0000 7fc2c93ed640 1 --2- 172.21.15.29:0/1031625275 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2a80680a0 0x7fc2a806a560 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40ae000 tx=0x7fc2b4004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:24.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.193+0000 7fc2ba7fc640 1 -- 172.21.15.29:0/1031625275 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc2c0055e20 con 0x7fc2c406d210 2024-04-25T00:40:24.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.353+0000 7fc2cbe52640 1 -- 172.21.15.29:0/1031625275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc2c40cf330 con 0x7fc2c406d210 2024-04-25T00:40:24.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.357+0000 7fc2ba7fc640 1 -- 172.21.15.29:0/1031625275 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7fc2c0055720 con 0x7fc2c406d210 2024-04-25T00:40:24.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.361+0000 7fc2cbe52640 1 -- 172.21.15.29:0/1031625275 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2a80680a0 msgr2=0x7fc2a806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.361+0000 7fc2cbe52640 1 --2- 172.21.15.29:0/1031625275 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2a80680a0 0x7fc2a806a560 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40ae000 tx=0x7fc2b4004740 comp rx=0 tx=0).stop 2024-04-25T00:40:24.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.361+0000 7fc2cbe52640 1 -- 172.21.15.29:0/1031625275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c406d210 msgr2=0x7fc2c40cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.361+0000 7fc2cbe52640 1 --2- 172.21.15.29:0/1031625275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c406d210 0x7fc2c40cbd80 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0001ec0 tx=0x7fc2c000c8e0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.361+0000 7fc2cbe52640 1 -- 172.21.15.29:0/1031625275 shutdown_connections 2024-04-25T00:40:24.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.361+0000 7fc2cbe52640 1 --2- 172.21.15.29:0/1031625275 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc2a80680a0 0x7fc2a806a560 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.361+0000 7fc2cbe52640 1 --2- 172.21.15.29:0/1031625275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2c415df70 0x7fc2c41fa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.361+0000 7fc2cbe52640 1 --2- 172.21.15.29:0/1031625275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c40b4d30 0x7fc2c41fa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.361+0000 7fc2cbe52640 1 --2- 172.21.15.29:0/1031625275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c406d210 0x7fc2c40cbd80 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.361+0000 7fc2cbe52640 1 -- 172.21.15.29:0/1031625275 >> 172.21.15.29:0/1031625275 conn(0x7fc2c40b1870 msgr2=0x7fc2c4154d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:24.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.361+0000 7fc2cbe52640 1 -- 172.21.15.29:0/1031625275 shutdown_connections 2024-04-25T00:40:24.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.361+0000 7fc2cbe52640 1 -- 172.21.15.29:0/1031625275 wait complete. 2024-04-25T00:40:24.379 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:40:24.379 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:40:24.433 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:40:24.484 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:40:24.484 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:40:24.484 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:40:24.490 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:40:24.491 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:40:24.493 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:40:24.499 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:40:24.499 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:40:24.499 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:40:24.536 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:40:24.537 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:40:24.537 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:40:24.546 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:40:24.546 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:40:24.547 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:40:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f2024ee1640 1 Processor -- start 2024-04-25T00:40:24.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f2024ee1640 1 -- start start 2024-04-25T00:40:24.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f2024ee1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202006e440 0x7f20200b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:24.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f2024ee1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20200b41e0 con 0x7f20200b3140 2024-04-25T00:40:24.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f2024ee1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f202006d960 con 0x7f20200b3c30 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f2024ee1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f20200b5050 con 0x7f202006e440 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201e59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f20200b3140 0x7f20200b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38350/0 (socket says 172.21.15.29:38350) 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201e59c640 1 -- 172.21.15.29:0/1095380333 learned_addr learned my addr 172.21.15.29:0/1095380333 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201ed9d640 1 --2- 172.21.15.29:0/1095380333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202006e440 0x7f20200b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1933444110 0 0) 0x7f20200b41e0 con 0x7f20200b3140 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f200c003530 con 0x7f20200b3140 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 752057820 0 0) 0x7f200c003530 con 0x7f20200b3140 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202006e440 msgr2=0x7f20200b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 --2- 172.21.15.29:0/1095380333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202006e440 0x7f20200b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 >> v1:172.21.15.157:6789/0 conn(0x7f20200b3c30 legacy=0x7f20200ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202006dc40 con 0x7f20200b3140 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f2008001680 con 0x7f20200b3140 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 >> v1:172.21.15.29:6789/0 conn(0x7f20200b3140 legacy=0x7f20200b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 --2- 172.21.15.29:0/1095380333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202006e440 0x7f200c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 --2- 172.21.15.29:0/1095380333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c003f70 0x7f200c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 --2- 172.21.15.29:0/1095380333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f200c004d50 0x7f200c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f20200b5050 con 0x7f200c004d50 2024-04-25T00:40:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f200c00bab0 con 0x7f200c003f70 2024-04-25T00:40:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f200c00bcb0 con 0x7f202006e440 2024-04-25T00:40:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201ed9d640 1 --2- 172.21.15.29:0/1095380333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f200c004d50 0x7f200c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f2017fff640 1 --2- 172.21.15.29:0/1095380333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c003f70 0x7f200c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201ed9d640 1 -- 172.21.15.29:0/1095380333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202006e440 msgr2=0x7f200c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201ed9d640 1 --2- 172.21.15.29:0/1095380333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f202006e440 0x7f200c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201ed9d640 1 -- 172.21.15.29:0/1095380333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c003f70 msgr2=0x7f200c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201ed9d640 1 --2- 172.21.15.29:0/1095380333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c003f70 0x7f200c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.717+0000 7f201ed9d640 1 -- 172.21.15.29:0/1095380333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c003530 con 0x7f200c004d50 2024-04-25T00:40:24.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f201ed9d640 1 --2- 172.21.15.29:0/1095380333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f200c004d50 0x7f200c0051d0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f2018009fa0 tx=0x7f201801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af56d4c09dfac230 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:24.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180b2070 con 0x7f200c004d50 2024-04-25T00:40:24.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20180a5d50 con 0x7f200c004d50 2024-04-25T00:40:24.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f201dd9b640 1 -- 172.21.15.29:0/1095380333 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018009bf0 con 0x7f200c004d50 2024-04-25T00:40:24.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 -- 172.21.15.29:0/1095380333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f200c004d50 msgr2=0x7f200c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 --2- 172.21.15.29:0/1095380333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f200c004d50 0x7f200c0051d0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f2018009fa0 tx=0x7f201801cc80 comp rx=0 tx=0).stop 2024-04-25T00:40:24.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f201ed9d640 1 -- 172.21.15.29:0/1095380333 reap_dead start 2024-04-25T00:40:24.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 -- 172.21.15.29:0/1095380333 shutdown_connections 2024-04-25T00:40:24.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 -- 172.21.15.29:0/1095380333 >> 172.21.15.29:0/1095380333 conn(0x7f20200b1870 msgr2=0x7f20200b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:24.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 -- 172.21.15.29:0/1095380333 shutdown_connections 2024-04-25T00:40:24.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 -- 172.21.15.29:0/1095380333 wait complete. 2024-04-25T00:40:24.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 Processor -- start 2024-04-25T00:40:24.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 -- start start 2024-04-25T00:40:24.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c004d50 0x7f20200b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f202015e3e0 0x7f202015e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20200a9490 0x7f20200a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2020145a90 con 0x7f202015e3e0 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2020145910 con 0x7f200c004d50 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.721+0000 7f2024ee1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f20200b4b10 con 0x7f20200a9490 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2017fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f202015e3e0 0x7f202015e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2017fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f202015e3e0 0x7f202015e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53118/0 (socket says 172.21.15.29:53118) 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2017fff640 1 -- 172.21.15.29:0/2000074248 learned_addr learned my addr 172.21.15.29:0/2000074248 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f201e59c640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c004d50 0x7f20200b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f201ed9d640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20200a9490 0x7f20200a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2017fff640 1 -- 172.21.15.29:0/2000074248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20200a9490 msgr2=0x7f20200a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2017fff640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20200a9490 0x7f20200a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2017fff640 1 -- 172.21.15.29:0/2000074248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c004d50 msgr2=0x7f20200b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2017fff640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c004d50 0x7f20200b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2017fff640 1 -- 172.21.15.29:0/2000074248 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202014c250 con 0x7f202015e3e0 2024-04-25T00:40:24.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2017fff640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f202015e3e0 0x7f202015e840 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f201000e550 tx=0x7f201000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:24.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2016ffd640 1 -- 172.21.15.29:0/2000074248 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20100052e0 con 0x7f202015e3e0 2024-04-25T00:40:24.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2016ffd640 1 -- 172.21.15.29:0/2000074248 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2010018070 con 0x7f202015e3e0 2024-04-25T00:40:24.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2024ee1640 1 -- 172.21.15.29:0/2000074248 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202015ef80 con 0x7f202015e3e0 2024-04-25T00:40:24.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2016ffd640 1 -- 172.21.15.29:0/2000074248 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010003820 con 0x7f202015e3e0 2024-04-25T00:40:24.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2024ee1640 1 -- 172.21.15.29:0/2000074248 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f202014c510 con 0x7f202015e3e0 2024-04-25T00:40:24.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2024ee1640 1 -- 172.21.15.29:0/2000074248 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2020069270 con 0x7f202015e3e0 2024-04-25T00:40:24.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2016ffd640 1 -- 172.21.15.29:0/2000074248 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f20100039c0 con 0x7f202015e3e0 2024-04-25T00:40:24.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2016ffd640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f20000680d0 0x7f200006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:24.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f2016ffd640 1 -- 172.21.15.29:0/2000074248 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f2010083f30 con 0x7f202015e3e0 2024-04-25T00:40:24.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f201e59c640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f20000680d0 0x7f200006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:24.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.725+0000 7f201e59c640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f20000680d0 0x7f200006a590 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2008001810 tx=0x7f20080016e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:24.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.729+0000 7f2016ffd640 1 -- 172.21.15.29:0/2000074248 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2010056360 con 0x7f202015e3e0 2024-04-25T00:40:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.925+0000 7f2024ee1640 1 -- 172.21.15.29:0/2000074248 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2020143930 con 0x7f202015e3e0 2024-04-25T00:40:24.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.925+0000 7f2016ffd640 1 -- 172.21.15.29:0/2000074248 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+3873 (secure 0 0 0) 0x7f2010005480 con 0x7f202015e3e0 2024-04-25T00:40:24.927 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:24.927 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":95,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6923,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3075204329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3075204329},{"type":"v1","addr":"172.21.15.157:6833","nonce":3075204329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":6926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3377772964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3377772964},{"type":"v1","addr":"172.21.15.157:6835","nonce":3377772964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":9621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/897131811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":897131811},{"type":"v1","addr":"172.21.15.29:6833","nonce":897131811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/1992310340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1992310340},{"type":"v1","addr":"172.21.15.29:6835","nonce":1992310340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[]} 2024-04-25T00:40:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 95 2024-04-25T00:40:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.929+0000 7f2024ee1640 1 -- 172.21.15.29:0/2000074248 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f20000680d0 msgr2=0x7f200006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.929+0000 7f2024ee1640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f20000680d0 0x7f200006a590 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2008001810 tx=0x7f20080016e0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.929+0000 7f2024ee1640 1 -- 172.21.15.29:0/2000074248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f202015e3e0 msgr2=0x7f202015e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.929+0000 7f2024ee1640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f202015e3e0 0x7f202015e840 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f201000e550 tx=0x7f201000ea10 comp rx=0 tx=0).stop 2024-04-25T00:40:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.929+0000 7f2024ee1640 1 -- 172.21.15.29:0/2000074248 shutdown_connections 2024-04-25T00:40:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.929+0000 7f2024ee1640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f20000680d0 0x7f200006a590 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.929+0000 7f2024ee1640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20200a9490 0x7f20200a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.929+0000 7f2024ee1640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f202015e3e0 0x7f202015e840 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.929+0000 7f2024ee1640 1 --2- 172.21.15.29:0/2000074248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200c004d50 0x7f20200b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.929+0000 7f2024ee1640 1 -- 172.21.15.29:0/2000074248 >> 172.21.15.29:0/2000074248 conn(0x7f20200b1870 msgr2=0x7f20201446f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.929+0000 7f2024ee1640 1 -- 172.21.15.29:0/2000074248 shutdown_connections 2024-04-25T00:40:24.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:24.929+0000 7f2024ee1640 1 -- 172.21.15.29:0/2000074248 wait complete. 2024-04-25T00:40:24.946 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:40:24.946 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:40:24.946 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:40:24.946 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:40:24.946 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:40:24.947 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:40:24.947 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:40:24.947 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:40:24.947 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:40:24.947 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:40:24.947 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:40:24.947 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:40:24.948 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:40:24.948 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:40:24.948 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:40:24.948 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:40:24.956 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:40:24.956 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:40:24.957 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:40:24.958 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:40:24.964 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:40:24.965 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:40:24.966 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:40:24.966 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:40:25.000 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:40:25.000 DEBUG:teuthology.parallel:result is None 2024-04-25T00:40:25.001 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:40:25.001 DEBUG:teuthology.parallel:result is None 2024-04-25T00:40:25.010 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:40:25.011 DEBUG:teuthology.parallel:result is None 2024-04-25T00:40:25.012 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:40:25.012 DEBUG:teuthology.parallel:result is None 2024-04-25T00:40:25.012 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:40:25.062 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:40:25.066 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:40:25.082 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-25T00:40:25.087 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-25T00:40:25.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.125+0000 7f586efce640 1 Processor -- start 2024-04-25T00:40:25.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.125+0000 7f586efce640 1 -- start start 2024-04-25T00:40:25.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.125+0000 7f586efce640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f58680b3c30 0x7f586806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:25.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.125+0000 7f586efce640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58680b41e0 con 0x7f58680b4b10 2024-04-25T00:40:25.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f586efce640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f586806d960 con 0x7f58680b3140 2024-04-25T00:40:25.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f586efce640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f586806dc40 con 0x7f58680b3c30 2024-04-25T00:40:25.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f5867fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f58680b3c30 0x7f586806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:25.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f586d56b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f58680b4b10 0x7f58680ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38370/0 (socket says 172.21.15.29:38370) 2024-04-25T00:40:25.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f586d56b640 1 -- 172.21.15.29:0/3859799849 learned_addr learned my addr 172.21.15.29:0/3859799849 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:25.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f58677fe640 1 -- 172.21.15.29:0/3859799849 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1806270371 0 0) 0x7f58680b41e0 con 0x7f58680b4b10 2024-04-25T00:40:25.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f58677fe640 1 -- 172.21.15.29:0/3859799849 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5850003530 con 0x7f58680b4b10 2024-04-25T00:40:25.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f5867fff640 1 -- 172.21.15.29:0/3859799849 >> v1:172.21.15.157:6789/0 conn(0x7f58680b3140 legacy=0x7f58680b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:25.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f5867fff640 1 -- 172.21.15.29:0/3859799849 >> v1:172.21.15.29:6789/0 conn(0x7f58680b4b10 legacy=0x7f58680ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:25.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f5867fff640 1 -- 172.21.15.29:0/3859799849 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586806df20 con 0x7f58680b3c30 2024-04-25T00:40:25.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f5867fff640 1 --2- 172.21.15.29:0/3859799849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58680b3c30 0x7f586806e440 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f585c0097f0 tx=0x7f585c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fecea45765eb195 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:25.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f58677fe640 1 -- 172.21.15.29:0/3859799849 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c0b0070 con 0x7f58680b3c30 2024-04-25T00:40:25.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f58677fe640 1 -- 172.21.15.29:0/3859799849 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585c0ab600 con 0x7f58680b3c30 2024-04-25T00:40:25.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f58677fe640 1 -- 172.21.15.29:0/3859799849 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c0ab9d0 con 0x7f58680b3c30 2024-04-25T00:40:25.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f586efce640 1 -- 172.21.15.29:0/3859799849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58680b3c30 msgr2=0x7f586806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:25.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f586efce640 1 --2- 172.21.15.29:0/3859799849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58680b3c30 0x7f586806e440 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f585c0097f0 tx=0x7f585c01c460 comp rx=0 tx=0).stop 2024-04-25T00:40:25.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f586efce640 1 -- 172.21.15.29:0/3859799849 shutdown_connections 2024-04-25T00:40:25.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f586efce640 1 --2- 172.21.15.29:0/3859799849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58680b3c30 0x7f586806e440 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:25.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f586efce640 1 -- 172.21.15.29:0/3859799849 >> 172.21.15.29:0/3859799849 conn(0x7f58680b1870 msgr2=0x7f58680b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:25.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f586efce640 1 -- 172.21.15.29:0/3859799849 shutdown_connections 2024-04-25T00:40:25.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.129+0000 7f586efce640 1 -- 172.21.15.29:0/3859799849 wait complete. 2024-04-25T00:40:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586efce640 1 Processor -- start 2024-04-25T00:40:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586efce640 1 -- start start 2024-04-25T00:40:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586efce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58680b3140 0x7f58681395e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586efce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58680b3c30 0x7f58681565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:25.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586efce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58680b4b10 0x7f5868156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:25.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586efce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5868169d90 con 0x7f58680b3140 2024-04-25T00:40:25.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586efce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5868169c10 con 0x7f58680b4b10 2024-04-25T00:40:25.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586efce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5868169f10 con 0x7f58680b3c30 2024-04-25T00:40:25.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586cd6a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58680b3140 0x7f58681395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:25.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586cd6a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58680b3140 0x7f58681395e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53160/0 (socket says 172.21.15.29:53160) 2024-04-25T00:40:25.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586cd6a640 1 -- 172.21.15.29:0/2529533384 learned_addr learned my addr 172.21.15.29:0/2529533384 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:25.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f5867fff640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58680b3c30 0x7f58681565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:25.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586d56b640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58680b4b10 0x7f5868156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:25.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586cd6a640 1 -- 172.21.15.29:0/2529533384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58680b3c30 msgr2=0x7f58681565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:25.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586cd6a640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58680b3c30 0x7f58681565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:25.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586cd6a640 1 -- 172.21.15.29:0/2529533384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58680b4b10 msgr2=0x7f5868156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:25.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586cd6a640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58680b4b10 0x7f5868156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:25.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586cd6a640 1 -- 172.21.15.29:0/2529533384 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58681677f0 con 0x7f58680b3140 2024-04-25T00:40:25.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f5867fff640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58680b3c30 0x7f58681565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:25.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586cd6a640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58680b3140 0x7f58681395e0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f585800ba50 tx=0x7f585800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:25.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f58657fa640 1 -- 172.21.15.29:0/2529533384 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858019070 con 0x7f58680b3140 2024-04-25T00:40:25.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f58657fa640 1 -- 172.21.15.29:0/2529533384 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585800cc20 con 0x7f58680b3140 2024-04-25T00:40:25.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586efce640 1 -- 172.21.15.29:0/2529533384 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586813a400 con 0x7f58680b3140 2024-04-25T00:40:25.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f58657fa640 1 -- 172.21.15.29:0/2529533384 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858014440 con 0x7f58680b3140 2024-04-25T00:40:25.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586efce640 1 -- 172.21.15.29:0/2529533384 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586813a6a0 con 0x7f58680b3140 2024-04-25T00:40:25.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f586efce640 1 -- 172.21.15.29:0/2529533384 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586813ad60 con 0x7f58680b3140 2024-04-25T00:40:25.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.133+0000 7f58657fa640 1 -- 172.21.15.29:0/2529533384 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f585801c070 con 0x7f58680b3140 2024-04-25T00:40:25.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.137+0000 7f58657fa640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f58540680a0 0x7f585406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:25.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.137+0000 7f5867fff640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f58540680a0 0x7f585406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:25.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.137+0000 7f58657fa640 1 -- 172.21.15.29:0/2529533384 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f5858086940 con 0x7f58680b3140 2024-04-25T00:40:25.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.137+0000 7f5867fff640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f58540680a0 0x7f585406a560 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f585c0ad000 tx=0x7f585c0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:25.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.141+0000 7f58657fa640 1 -- 172.21.15.29:0/2529533384 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f586813ad60 con 0x7f58680b3140 2024-04-25T00:40:25.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:25.313+0000 7f586efce640 1 -- 172.21.15.29:0/2529533384 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5868069270 con 0x7f58680b3140 2024-04-25T00:40:26.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.045+0000 7f58657fa640 1 -- 172.21.15.29:0/2529533384 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v150) ==== 97+0+0 (secure 0 0 0) 0x7f5858056110 con 0x7f58680b3140 2024-04-25T00:40:26.044 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-04-25T00:40:26.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.045+0000 7f586efce640 1 -- 172.21.15.29:0/2529533384 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f58540680a0 msgr2=0x7f585406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.045+0000 7f586efce640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f58540680a0 0x7f585406a560 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f585c0ad000 tx=0x7f585c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:40:26.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.045+0000 7f586efce640 1 -- 172.21.15.29:0/2529533384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58680b3140 msgr2=0x7f58681395e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.045+0000 7f586efce640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58680b3140 0x7f58681395e0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f585800ba50 tx=0x7f585800bf10 comp rx=0 tx=0).stop 2024-04-25T00:40:26.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.045+0000 7f586efce640 1 -- 172.21.15.29:0/2529533384 shutdown_connections 2024-04-25T00:40:26.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.045+0000 7f586efce640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f58540680a0 0x7f585406a560 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.045+0000 7f586efce640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58680b4b10 0x7f5868156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.045+0000 7f586efce640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58680b3c30 0x7f58681565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.045+0000 7f586efce640 1 --2- 172.21.15.29:0/2529533384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58680b3140 0x7f58681395e0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.045+0000 7f586efce640 1 -- 172.21.15.29:0/2529533384 >> 172.21.15.29:0/2529533384 conn(0x7f58680b1870 msgr2=0x7f5868143410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:26.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.049+0000 7f586efce640 1 -- 172.21.15.29:0/2529533384 shutdown_connections 2024-04-25T00:40:26.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.049+0000 7f586efce640 1 -- 172.21.15.29:0/2529533384 wait complete. 2024-04-25T00:40:26.070 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:40:26.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.193+0000 7f621c8fb640 1 Processor -- start 2024-04-25T00:40:26.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.193+0000 7f621c8fb640 1 -- start start 2024-04-25T00:40:26.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621c8fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62180b3140 0x7f621806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621c8fb640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62180b41e0 con 0x7f62180b4b10 2024-04-25T00:40:26.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621c8fb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f621806d960 con 0x7f62180b3c30 2024-04-25T00:40:26.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621c8fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f621806dc40 con 0x7f62180b3140 2024-04-25T00:40:26.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f6215d9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f62180b4b10 0x7f62180b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38384/0 (socket says 172.21.15.29:38384) 2024-04-25T00:40:26.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f6215d9b640 1 -- 172.21.15.29:0/3202334130 learned_addr learned my addr 172.21.15.29:0/3202334130 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:26.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621659c640 1 --2- 172.21.15.29:0/3202334130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62180b3140 0x7f621806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 72508206 0 0) 0x7f62180b41e0 con 0x7f62180b4b10 2024-04-25T00:40:26.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6200003530 con 0x7f62180b4b10 2024-04-25T00:40:26.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2152432985 0 0) 0x7f6200003530 con 0x7f62180b4b10 2024-04-25T00:40:26.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62180b3140 msgr2=0x7f621806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:26.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 --2- 172.21.15.29:0/3202334130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62180b3140 0x7f621806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 >> v1:172.21.15.157:6789/0 conn(0x7f62180b3c30 legacy=0x7f62180ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621806df20 con 0x7f62180b4b10 2024-04-25T00:40:26.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f620c001680 con 0x7f62180b4b10 2024-04-25T00:40:26.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 >> v1:172.21.15.29:6789/0 conn(0x7f62180b4b10 legacy=0x7f62180b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 --2- 172.21.15.29:0/3202334130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62000039f0 0x7f6200003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 --2- 172.21.15.29:0/3202334130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6200004810 0x7f6200004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 --2- 172.21.15.29:0/3202334130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62180b3140 0x7f62000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f621806dc40 con 0x7f62000039f0 2024-04-25T00:40:26.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6200009a90 con 0x7f6200004810 2024-04-25T00:40:26.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f62000145d0 con 0x7f62180b3140 2024-04-25T00:40:26.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621659c640 1 --2- 172.21.15.29:0/3202334130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62000039f0 0x7f6200003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f6215d9b640 1 --2- 172.21.15.29:0/3202334130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6200004810 0x7f6200004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f6216d9d640 1 --2- 172.21.15.29:0/3202334130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62180b3140 0x7f62000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621659c640 1 -- 172.21.15.29:0/3202334130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62180b3140 msgr2=0x7f62000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:26.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621659c640 1 --2- 172.21.15.29:0/3202334130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62180b3140 0x7f62000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621659c640 1 -- 172.21.15.29:0/3202334130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6200004810 msgr2=0x7f6200004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:26.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621659c640 1 --2- 172.21.15.29:0/3202334130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6200004810 0x7f6200004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.197+0000 7f621659c640 1 -- 172.21.15.29:0/3202334130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6200003530 con 0x7f62000039f0 2024-04-25T00:40:26.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f621659c640 1 --2- 172.21.15.29:0/3202334130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62000039f0 0x7f6200003e10 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f6210009fc0 tx=0x7f621001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91655a36647857f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:26.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62100b2070 con 0x7f62000039f0 2024-04-25T00:40:26.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62100a5a70 con 0x7f62000039f0 2024-04-25T00:40:26.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f621559a640 1 -- 172.21.15.29:0/3202334130 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210009c10 con 0x7f62000039f0 2024-04-25T00:40:26.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f621c8fb640 1 -- 172.21.15.29:0/3202334130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62000039f0 msgr2=0x7f6200003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f621c8fb640 1 --2- 172.21.15.29:0/3202334130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62000039f0 0x7f6200003e10 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f6210009fc0 tx=0x7f621001cc80 comp rx=0 tx=0).stop 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f6216d9d640 1 -- 172.21.15.29:0/3202334130 reap_dead start 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f621c8fb640 1 -- 172.21.15.29:0/3202334130 shutdown_connections 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f621c8fb640 1 -- 172.21.15.29:0/3202334130 >> 172.21.15.29:0/3202334130 conn(0x7f62180b1870 msgr2=0x7f62180b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f621c8fb640 1 -- 172.21.15.29:0/3202334130 shutdown_connections 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f621c8fb640 1 -- 172.21.15.29:0/3202334130 wait complete. 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f621c8fb640 1 Processor -- start 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f621c8fb640 1 -- start start 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.201+0000 7f621c8fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6218155660 0x7f6218144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f621c8fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6218155a40 0x7f6218141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f621c8fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62180cff60 0x7f62180d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f621c8fb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f62180710b0 con 0x7f62180cff60 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f621c8fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6218070f30 con 0x7f6218155a40 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f621c8fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6218071230 con 0x7f6218155660 2024-04-25T00:40:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6216d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62180cff60 0x7f62180d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6216d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62180cff60 0x7f62180d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53166/0 (socket says 172.21.15.29:53166) 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6216d9d640 1 -- 172.21.15.29:0/2259969125 learned_addr learned my addr 172.21.15.29:0/2259969125 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6215d9b640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6218155a40 0x7f6218141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f621659c640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6218155660 0x7f6218144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6216d9d640 1 -- 172.21.15.29:0/2259969125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6218155660 msgr2=0x7f6218144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6216d9d640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6218155660 0x7f6218144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6216d9d640 1 -- 172.21.15.29:0/2259969125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6218155a40 msgr2=0x7f6218141e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6216d9d640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6218155a40 0x7f6218141e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6216d9d640 1 -- 172.21.15.29:0/2259969125 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62180ce200 con 0x7f62180cff60 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6216d9d640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62180cff60 0x7f62180d03e0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f6208000c10 tx=0x7f620800e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6206ffd640 1 -- 172.21.15.29:0/2259969125 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208019070 con 0x7f62180cff60 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6206ffd640 1 -- 172.21.15.29:0/2259969125 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6208007e30 con 0x7f62180cff60 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f621c8fb640 1 -- 172.21.15.29:0/2259969125 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6218142360 con 0x7f62180cff60 2024-04-25T00:40:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6206ffd640 1 -- 172.21.15.29:0/2259969125 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208003600 con 0x7f62180cff60 2024-04-25T00:40:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f621c8fb640 1 -- 172.21.15.29:0/2259969125 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6218143f10 con 0x7f62180cff60 2024-04-25T00:40:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f621c8fb640 1 -- 172.21.15.29:0/2259969125 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62180d0920 con 0x7f62180cff60 2024-04-25T00:40:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6206ffd640 1 -- 172.21.15.29:0/2259969125 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f620801c070 con 0x7f62180cff60 2024-04-25T00:40:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6206ffd640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f61f80680a0 0x7f61f806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f621659c640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f61f80680a0 0x7f61f806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f6206ffd640 1 -- 172.21.15.29:0/2259969125 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4857+0+0 (secure 0 0 0) 0x7f6208083ad0 con 0x7f62180cff60 2024-04-25T00:40:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.205+0000 7f621659c640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f61f80680a0 0x7f61f806a560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f62100af000 tx=0x7f62100a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.209+0000 7f6206ffd640 1 -- 172.21.15.29:0/2259969125 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6208055e70 con 0x7f62180cff60 2024-04-25T00:40:26.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.429+0000 7f621c8fb640 1 -- 172.21.15.29:0/2259969125 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6218069270 con 0x7f62180cff60 2024-04-25T00:40:26.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.429+0000 7f6206ffd640 1 -- 172.21.15.29:0/2259969125 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) ==== 80+0+5304 (secure 0 0 0) 0x7f6208085e20 con 0x7f62180cff60 2024-04-25T00:40:26.430 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:26.431 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:40:26.431 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-04-25T00:40:26.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.431 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-04-25T00:40:26.431 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmyzgKAxAAU+nrUJgQhDX6xlARNFI/ng==", 2024-04-25T00:40:26.432 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.432 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:40:26.432 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:40:26.432 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:40:26.432 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.432 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.433 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-04-25T00:40:26.433 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmtyzPERAAxNUDANw1H/ZzOo+dhgzxlg==", 2024-04-25T00:40:26.433 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.433 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:40:26.433 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:40:26.434 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:40:26.434 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.434 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.435 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-04-25T00:40:26.435 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmYUl8CxAAj6pV1+s4a0rpWLq2EgZ+8w==", 2024-04-25T00:40:26.435 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.435 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:40:26.435 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:40:26.435 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:40:26.436 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.436 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-04-25T00:40:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmuAhKGhAAOLIcWUodW5TqtVgHWZl12A==", 2024-04-25T00:40:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:40:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:40:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:40:26.438 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.438 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.438 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-04-25T00:40:26.438 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAzpClmudk6MhAA6YFyd57jwtB2pdOaad7bxQ==", 2024-04-25T00:40:26.438 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.438 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:40:26.439 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:40:26.439 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.439 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.439 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-04-25T00:40:26.439 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA2pClm76q0NRAAxWhEP2ZGoFnZcuiQyGjglw==", 2024-04-25T00:40:26.439 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.439 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:40:26.439 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:40:26.439 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.440 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-04-25T00:40:26.440 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA6pClmcgULABAA2F5WBR9Df0v4Sn875AzF7g==", 2024-04-25T00:40:26.440 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.440 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:40:26.440 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:40:26.440 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.441 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-04-25T00:40:26.441 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA9pClm9QuRBhAALFLruv4JU/uNZYMX0yNuXw==", 2024-04-25T00:40:26.441 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.441 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:40:26.441 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:40:26.442 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.442 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.442 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-04-25T00:40:26.442 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBBpClmNj4VDBAAxaxMvyaCvjynPY3f81udnw==", 2024-04-25T00:40:26.442 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.442 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:40:26.442 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:40:26.443 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.443 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.443 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-04-25T00:40:26.443 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBEpClmzBybEBAArry1lM4RIsM/txKfgBqDpw==", 2024-04-25T00:40:26.443 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.443 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:40:26.443 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:40:26.444 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.444 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.444 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-04-25T00:40:26.444 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBHpClm5HfjFhAAbFwJCpbhegLWDu0mbS8FoA==", 2024-04-25T00:40:26.444 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.444 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:40:26.444 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:40:26.444 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.445 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-04-25T00:40:26.445 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBKpClme/7oHBAAOR2iQ6CweexR7sC1PgesZw==", 2024-04-25T00:40:26.445 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.445 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:40:26.445 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:40:26.445 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.446 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-04-25T00:40:26.446 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A==", 2024-04-25T00:40:26.446 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.446 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:40:26.446 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-04-25T00:40:26.446 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:40:26.446 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.447 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-04-25T00:40:26.447 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm8W/EJxAA0HZW24Hhcwf+soSZn6jHJg==", 2024-04-25T00:40:26.447 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.447 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:40:26.447 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:40:26.447 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:40:26.447 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:40:26.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.448 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-04-25T00:40:26.448 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmGuQJJRAAhVd4D0jc6Whgyf+B6xgFMw==", 2024-04-25T00:40:26.448 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.448 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:40:26.448 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:40:26.448 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:40:26.448 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:40:26.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.449 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-04-25T00:40:26.449 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmn+ciLRAAg4tQuZ13WP0RnWJaJq5fUQ==", 2024-04-25T00:40:26.449 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.449 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:40:26.449 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:40:26.450 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:40:26.450 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:40:26.450 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.451 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-04-25T00:40:26.451 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmPVihJBAA7PQge/teyRZKSkBnufZSZw==", 2024-04-25T00:40:26.451 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.451 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:40:26.451 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:40:26.452 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:40:26.452 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.452 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.452 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-04-25T00:40:26.453 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClm6WrdOhAAsY9FLCxG7PGpq2NCY4hGhQ==", 2024-04-25T00:40:26.453 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.453 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:40:26.453 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:40:26.453 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.453 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:26.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:26.454 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-04-25T00:40:26.454 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmeR4UOBAANs9SRolOm6E8+/4ciupk5w==", 2024-04-25T00:40:26.454 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:40:26.454 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:40:26.455 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:40:26.455 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:40:26.455 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.455 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:26.455 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:40:26.455 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:40:26.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.433+0000 7f621c8fb640 1 -- 172.21.15.29:0/2259969125 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f61f80680a0 msgr2=0x7f61f806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.433+0000 7f621c8fb640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f61f80680a0 0x7f61f806a560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f62100af000 tx=0x7f62100a7040 comp rx=0 tx=0).stop 2024-04-25T00:40:26.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.433+0000 7f621c8fb640 1 -- 172.21.15.29:0/2259969125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62180cff60 msgr2=0x7f62180d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.433+0000 7f621c8fb640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62180cff60 0x7f62180d03e0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f6208000c10 tx=0x7f620800e7e0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.433+0000 7f621c8fb640 1 -- 172.21.15.29:0/2259969125 shutdown_connections 2024-04-25T00:40:26.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.433+0000 7f621c8fb640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f61f80680a0 0x7f61f806a560 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.433+0000 7f621c8fb640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62180cff60 0x7f62180d03e0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.433+0000 7f621c8fb640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6218155a40 0x7f6218141e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.433+0000 7f621c8fb640 1 --2- 172.21.15.29:0/2259969125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6218155660 0x7f6218144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.433+0000 7f621c8fb640 1 -- 172.21.15.29:0/2259969125 >> 172.21.15.29:0/2259969125 conn(0x7f62180b1870 msgr2=0x7f62180727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:26.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.433+0000 7f621c8fb640 1 -- 172.21.15.29:0/2259969125 shutdown_connections 2024-04-25T00:40:26.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.433+0000 7f621c8fb640 1 -- 172.21.15.29:0/2259969125 wait complete. 2024-04-25T00:40:26.460 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:40:26.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.585+0000 7f9d6d470640 1 Processor -- start 2024-04-25T00:40:26.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d6d470640 1 -- start start 2024-04-25T00:40:26.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d6d470640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d6815e200 0x7f9d680d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d6d470640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9d680cf690 0x7f9d680d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d6d470640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d680d2ea0 con 0x7f9d680d2a90 2024-04-25T00:40:26.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d6d470640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9d680b38d0 con 0x7f9d680cf690 2024-04-25T00:40:26.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d6d470640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9d680cfc10 con 0x7f9d6815e200 2024-04-25T00:40:26.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d667fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9d680cf690 0x7f9d680d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d667fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9d680cf690 0x7f9d680d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53182/0 (socket says 172.21.15.29:53182) 2024-04-25T00:40:26.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d667fc640 1 -- 172.21.15.29:0/1112599213 learned_addr learned my addr 172.21.15.29:0/1112599213 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:26.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d66ffd640 1 --2- 172.21.15.29:0/1112599213 >> v2:172.21.15.157:3300/0 conn(0x7f9d6815e200 0x7f9d680d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d667fc640 1 -- 172.21.15.29:0/1112599213 >> v2:172.21.15.157:3300/0 conn(0x7f9d6815e200 msgr2=0x7f9d680d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:26.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d667fc640 1 --2- 172.21.15.29:0/1112599213 >> v2:172.21.15.157:3300/0 conn(0x7f9d6815e200 0x7f9d680d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d667fc640 1 -- 172.21.15.29:0/1112599213 >> v1:172.21.15.157:6789/0 conn(0x7f9d680d2a90 legacy=0x7f9d680d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:26.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d667fc640 1 -- 172.21.15.29:0/1112599213 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d680cfef0 con 0x7f9d680cf690 2024-04-25T00:40:26.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d667fc640 1 --2- 172.21.15.29:0/1112599213 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d680cf690 0x7f9d680d2550 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f9d6808d8f0 tx=0x7f9d5c01c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51113dacc18cb4c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:26.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.589+0000 7f9d65ffb640 1 -- 172.21.15.29:0/1112599213 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0b0070 con 0x7f9d680cf690 2024-04-25T00:40:26.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d65ffb640 1 -- 172.21.15.29:0/1112599213 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d5c0ab600 con 0x7f9d680cf690 2024-04-25T00:40:26.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d65ffb640 1 -- 172.21.15.29:0/1112599213 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0ab9d0 con 0x7f9d680cf690 2024-04-25T00:40:26.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 -- 172.21.15.29:0/1112599213 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d680cf690 msgr2=0x7f9d680d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 --2- 172.21.15.29:0/1112599213 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d680cf690 0x7f9d680d2550 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f9d6808d8f0 tx=0x7f9d5c01c2d0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 -- 172.21.15.29:0/1112599213 shutdown_connections 2024-04-25T00:40:26.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 --2- 172.21.15.29:0/1112599213 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d680cf690 0x7f9d680d2550 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 --2- 172.21.15.29:0/1112599213 >> v2:172.21.15.157:3300/0 conn(0x7f9d6815e200 0x7f9d680d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 -- 172.21.15.29:0/1112599213 >> 172.21.15.29:0/1112599213 conn(0x7f9d6809d280 msgr2=0x7f9d680d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:26.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 -- 172.21.15.29:0/1112599213 shutdown_connections 2024-04-25T00:40:26.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 -- 172.21.15.29:0/1112599213 wait complete. 2024-04-25T00:40:26.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 Processor -- start 2024-04-25T00:40:26.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 -- start start 2024-04-25T00:40:26.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d680cf690 0x7f9d681555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d680d2a90 0x7f9d6815f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d6815e200 0x7f9d6815f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6815e760 con 0x7f9d680d2a90 2024-04-25T00:40:26.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6815e5e0 con 0x7f9d6815e200 2024-04-25T00:40:26.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9d6816ae30 con 0x7f9d680cf690 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d667fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d680d2a90 0x7f9d6815f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d667fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d680d2a90 0x7f9d6815f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53192/0 (socket says 172.21.15.29:53192) 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d667fc640 1 -- 172.21.15.29:0/2795153466 learned_addr learned my addr 172.21.15.29:0/2795153466 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d66ffd640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d680cf690 0x7f9d681555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d677fe640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d6815e200 0x7f9d6815f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d667fc640 1 -- 172.21.15.29:0/2795153466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d680cf690 msgr2=0x7f9d681555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d667fc640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d680cf690 0x7f9d681555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d667fc640 1 -- 172.21.15.29:0/2795153466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d6815e200 msgr2=0x7f9d6815f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d667fc640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d6815e200 0x7f9d6815f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d667fc640 1 -- 172.21.15.29:0/2795153466 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6815bb90 con 0x7f9d680d2a90 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d667fc640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d680d2a90 0x7f9d6815f2d0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0ae040 tx=0x7f9d5c019070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d4f7fe640 1 -- 172.21.15.29:0/2795153466 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0bb070 con 0x7f9d680d2a90 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d4f7fe640 1 -- 172.21.15.29:0/2795153466 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d5c0b6430 con 0x7f9d680d2a90 2024-04-25T00:40:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 -- 172.21.15.29:0/2795153466 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d68155e20 con 0x7f9d680d2a90 2024-04-25T00:40:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d4f7fe640 1 -- 172.21.15.29:0/2795153466 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0b0040 con 0x7f9d680d2a90 2024-04-25T00:40:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.593+0000 7f9d6d470640 1 -- 172.21.15.29:0/2795153466 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d681563e0 con 0x7f9d680d2a90 2024-04-25T00:40:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.597+0000 7f9d6d470640 1 -- 172.21.15.29:0/2795153466 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d68069270 con 0x7f9d680d2a90 2024-04-25T00:40:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.597+0000 7f9d4f7fe640 1 -- 172.21.15.29:0/2795153466 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f9d5c0be070 con 0x7f9d680d2a90 2024-04-25T00:40:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.597+0000 7f9d4f7fe640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d540680d0 0x7f9d5406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.597+0000 7f9d4f7fe640 1 -- 172.21.15.29:0/2795153466 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4857+0+0 (secure 0 0 0) 0x7f9d5c11dc50 con 0x7f9d680d2a90 2024-04-25T00:40:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.597+0000 7f9d66ffd640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d540680d0 0x7f9d5406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.597+0000 7f9d66ffd640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d540680d0 0x7f9d5406a590 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009790 tx=0x7f9d58008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:26.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.601+0000 7f9d4f7fe640 1 -- 172.21.15.29:0/2795153466 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9d680d2ea0 con 0x7f9d680d2a90 2024-04-25T00:40:26.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.769+0000 7f9d6d470640 1 -- 172.21.15.29:0/2795153466 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d6806cdd0 con 0x7f9d680d2a90 2024-04-25T00:40:26.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.769+0000 7f9d4f7fe640 1 -- 172.21.15.29:0/2795153466 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+12853 (secure 0 0 0) 0x7f9d5c0f0080 con 0x7f9d680d2a90 2024-04-25T00:40:26.770 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:26.771 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":152,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:40:26.055233+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:40:26.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.773+0000 7f9d6d470640 1 -- 172.21.15.29:0/2795153466 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d540680d0 msgr2=0x7f9d5406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.773+0000 7f9d6d470640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d540680d0 0x7f9d5406a590 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009790 tx=0x7f9d58008040 comp rx=0 tx=0).stop 2024-04-25T00:40:26.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.773+0000 7f9d6d470640 1 -- 172.21.15.29:0/2795153466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d680d2a90 msgr2=0x7f9d6815f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.773+0000 7f9d6d470640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d680d2a90 0x7f9d6815f2d0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0ae040 tx=0x7f9d5c019070 comp rx=0 tx=0).stop 2024-04-25T00:40:26.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.773+0000 7f9d6d470640 1 -- 172.21.15.29:0/2795153466 shutdown_connections 2024-04-25T00:40:26.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.773+0000 7f9d6d470640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d540680d0 0x7f9d5406a590 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.773+0000 7f9d6d470640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d6815e200 0x7f9d6815f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.773+0000 7f9d6d470640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d680d2a90 0x7f9d6815f2d0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.773+0000 7f9d6d470640 1 --2- 172.21.15.29:0/2795153466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d680cf690 0x7f9d681555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.773+0000 7f9d6d470640 1 -- 172.21.15.29:0/2795153466 >> 172.21.15.29:0/2795153466 conn(0x7f9d6809d280 msgr2=0x7f9d68157660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:26.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.773+0000 7f9d6d470640 1 -- 172.21.15.29:0/2795153466 shutdown_connections 2024-04-25T00:40:26.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.773+0000 7f9d6d470640 1 -- 172.21.15.29:0/2795153466 wait complete. 2024-04-25T00:40:26.790 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:40:26.790 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:40:26.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8ff71b640 1 Processor -- start 2024-04-25T00:40:26.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8ff71b640 1 -- start start 2024-04-25T00:40:26.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8ff71b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff8f806e440 0x7ff8f80b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8ff71b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f80b41e0 con 0x7ff8f80b3c30 2024-04-25T00:40:26.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8ff71b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f806d960 con 0x7ff8f80b3140 2024-04-25T00:40:26.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8ff71b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f80b5050 con 0x7ff8f806e440 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8fccb6640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff8f80b3c30 0x7ff8f80ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38392/0 (socket says 172.21.15.29:38392) 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8fccb6640 1 -- 172.21.15.29:0/2964672634 learned_addr learned my addr 172.21.15.29:0/2964672634 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8fdcb8640 1 --2- 172.21.15.29:0/2964672634 >> v2:172.21.15.157:3300/0 conn(0x7ff8f806e440 0x7ff8f80b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 186669293 0 0) 0x7ff8f80b41e0 con 0x7ff8f80b3c30 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8e4003530 con 0x7ff8f80b3c30 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1849406479 0 0) 0x7ff8e4003530 con 0x7ff8f80b3c30 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 >> v2:172.21.15.157:3300/0 conn(0x7ff8f806e440 msgr2=0x7ff8f80b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 --2- 172.21.15.29:0/2964672634 >> v2:172.21.15.157:3300/0 conn(0x7ff8f806e440 0x7ff8f80b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 >> v1:172.21.15.157:6789/0 conn(0x7ff8f80b3140 legacy=0x7ff8f80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f806dc40 con 0x7ff8f80b3c30 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ff8e8001680 con 0x7ff8f80b3c30 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 >> v1:172.21.15.29:6789/0 conn(0x7ff8f80b3c30 legacy=0x7ff8f80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 --2- 172.21.15.29:0/2964672634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8e40039c0 0x7ff8e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 --2- 172.21.15.29:0/2964672634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e4004810 0x7ff8e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 --2- 172.21.15.29:0/2964672634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8e40051d0 0x7ff8e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f80b5050 con 0x7ff8e4004810 2024-04-25T00:40:26.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e400c070 con 0x7ff8e40051d0 2024-04-25T00:40:26.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff8e400c270 con 0x7ff8e40039c0 2024-04-25T00:40:26.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8fccb6640 1 --2- 172.21.15.29:0/2964672634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e4004810 0x7ff8e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8fdcb8640 1 --2- 172.21.15.29:0/2964672634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8e40051d0 0x7ff8e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8fd4b7640 1 --2- 172.21.15.29:0/2964672634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8e40039c0 0x7ff8e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8fccb6640 1 -- 172.21.15.29:0/2964672634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8e40039c0 msgr2=0x7ff8e4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:26.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8fccb6640 1 --2- 172.21.15.29:0/2964672634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8e40039c0 0x7ff8e4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8fccb6640 1 -- 172.21.15.29:0/2964672634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8e40051d0 msgr2=0x7ff8e400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:26.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8fccb6640 1 --2- 172.21.15.29:0/2964672634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8e40051d0 0x7ff8e400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8fdcb8640 1 -- 172.21.15.29:0/2964672634 reap_dead start 2024-04-25T00:40:26.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.921+0000 7ff8fccb6640 1 -- 172.21.15.29:0/2964672634 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e4003530 con 0x7ff8e4004810 2024-04-25T00:40:26.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.925+0000 7ff8fccb6640 1 --2- 172.21.15.29:0/2964672634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e4004810 0x7ff8e4004c90 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800c240 tx=0x7ff8e801f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca1a29f5cf1f5cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:26.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.925+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8007d20 con 0x7ff8e4004810 2024-04-25T00:40:26.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.925+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e8005ae0 con 0x7ff8e4004810 2024-04-25T00:40:26.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.925+0000 7ff8effff640 1 -- 172.21.15.29:0/2964672634 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8007070 con 0x7ff8e4004810 2024-04-25T00:40:26.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.925+0000 7ff8ff71b640 1 -- 172.21.15.29:0/2964672634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e4004810 msgr2=0x7ff8e4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.925+0000 7ff8ff71b640 1 --2- 172.21.15.29:0/2964672634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e4004810 0x7ff8e4004c90 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800c240 tx=0x7ff8e801f3b0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.925+0000 7ff8ff71b640 1 -- 172.21.15.29:0/2964672634 shutdown_connections 2024-04-25T00:40:26.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.925+0000 7ff8ff71b640 1 --2- 172.21.15.29:0/2964672634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e4004810 0x7ff8e4004c90 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.925+0000 7ff8ff71b640 1 -- 172.21.15.29:0/2964672634 >> 172.21.15.29:0/2964672634 conn(0x7ff8f80b1870 msgr2=0x7ff8f80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:26.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.925+0000 7ff8ff71b640 1 -- 172.21.15.29:0/2964672634 shutdown_connections 2024-04-25T00:40:26.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.925+0000 7ff8ff71b640 1 -- 172.21.15.29:0/2964672634 wait complete. 2024-04-25T00:40:26.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.925+0000 7ff8ff71b640 1 Processor -- start 2024-04-25T00:40:26.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.925+0000 7ff8ff71b640 1 -- start start 2024-04-25T00:40:26.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8ff71b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e4004810 0x7ff8f81542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8ff71b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8f8154800 0x7ff8f80aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8ff71b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8f80af160 0x7ff8f81581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8ff71b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8145a90 con 0x7ff8e4004810 2024-04-25T00:40:26.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8ff71b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8145910 con 0x7ff8f80af160 2024-04-25T00:40:26.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8ff71b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f80b4b10 con 0x7ff8f8154800 2024-04-25T00:40:26.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8fd4b7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e4004810 0x7ff8f81542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8fd4b7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e4004810 0x7ff8f81542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53206/0 (socket says 172.21.15.29:53206) 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8fd4b7640 1 -- 172.21.15.29:0/3183189223 learned_addr learned my addr 172.21.15.29:0/3183189223 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8fdcb8640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8f80af160 0x7ff8f81581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8fccb6640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8f8154800 0x7ff8f80aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8fd4b7640 1 -- 172.21.15.29:0/3183189223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8f8154800 msgr2=0x7ff8f80aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8fd4b7640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8f8154800 0x7ff8f80aec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8fd4b7640 1 -- 172.21.15.29:0/3183189223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8f80af160 msgr2=0x7ff8f81581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8fd4b7640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8f80af160 0x7ff8f81581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8fd4b7640 1 -- 172.21.15.29:0/3183189223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f813aad0 con 0x7ff8e4004810 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8fd4b7640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e4004810 0x7ff8f81542c0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400e8d0 tx=0x7ff8f400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8edffb640 1 -- 172.21.15.29:0/3183189223 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4003b90 con 0x7ff8e4004810 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8edffb640 1 -- 172.21.15.29:0/3183189223 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f4018070 con 0x7ff8e4004810 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8edffb640 1 -- 172.21.15.29:0/3183189223 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40050d0 con 0x7ff8e4004810 2024-04-25T00:40:26.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8ff71b640 1 -- 172.21.15.29:0/3183189223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f813adb0 con 0x7ff8e4004810 2024-04-25T00:40:26.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8ff71b640 1 -- 172.21.15.29:0/3183189223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f813b370 con 0x7ff8e4004810 2024-04-25T00:40:26.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8ff71b640 1 -- 172.21.15.29:0/3183189223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f8154d40 con 0x7ff8e4004810 2024-04-25T00:40:26.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8edffb640 1 -- 172.21.15.29:0/3183189223 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7ff8f4005270 con 0x7ff8e4004810 2024-04-25T00:40:26.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8edffb640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff8dc0680d0 0x7ff8dc06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:26.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8edffb640 1 -- 172.21.15.29:0/3183189223 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4857+0+0 (secure 0 0 0) 0x7ff8f40847b0 con 0x7ff8e4004810 2024-04-25T00:40:26.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.929+0000 7ff8fccb6640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff8dc0680d0 0x7ff8dc06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:26.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.933+0000 7ff8fccb6640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff8dc0680d0 0x7ff8dc06a590 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80af000 tx=0x7ff8e80aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:26.937+0000 7ff8edffb640 1 -- 172.21.15.29:0/3183189223 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff8f401a310 con 0x7ff8e4004810 2024-04-25T00:40:27.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.101+0000 7ff8ff71b640 1 -- 172.21.15.29:0/3183189223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff8f8069270 con 0x7ff8e4004810 2024-04-25T00:40:27.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.101+0000 7ff8edffb640 1 -- 172.21.15.29:0/3183189223 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v152) ==== 93+0+10 (secure 0 0 0) 0x7ff8f4056b60 con 0x7ff8e4004810 2024-04-25T00:40:27.100 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:40:27.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.101+0000 7ff8ff71b640 1 -- 172.21.15.29:0/3183189223 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff8dc0680d0 msgr2=0x7ff8dc06a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:27.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.101+0000 7ff8ff71b640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff8dc0680d0 0x7ff8dc06a590 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80af000 tx=0x7ff8e80aa000 comp rx=0 tx=0).stop 2024-04-25T00:40:27.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.101+0000 7ff8ff71b640 1 -- 172.21.15.29:0/3183189223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e4004810 msgr2=0x7ff8f81542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:27.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.101+0000 7ff8ff71b640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e4004810 0x7ff8f81542c0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400e8d0 tx=0x7ff8f400ed90 comp rx=0 tx=0).stop 2024-04-25T00:40:27.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.101+0000 7ff8ff71b640 1 -- 172.21.15.29:0/3183189223 shutdown_connections 2024-04-25T00:40:27.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.101+0000 7ff8ff71b640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff8dc0680d0 0x7ff8dc06a590 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.101+0000 7ff8ff71b640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8f80af160 0x7ff8f81581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.101+0000 7ff8ff71b640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8f8154800 0x7ff8f80aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.101+0000 7ff8ff71b640 1 --2- 172.21.15.29:0/3183189223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8e4004810 0x7ff8f81542c0 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.105+0000 7ff8ff71b640 1 -- 172.21.15.29:0/3183189223 >> 172.21.15.29:0/3183189223 conn(0x7ff8f80b1870 msgr2=0x7ff8f80b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:27.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.105+0000 7ff8ff71b640 1 -- 172.21.15.29:0/3183189223 shutdown_connections 2024-04-25T00:40:27.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.105+0000 7ff8ff71b640 1 -- 172.21.15.29:0/3183189223 wait complete. 2024-04-25T00:40:27.121 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:40:27.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.241+0000 7f1703bcc640 1 Processor -- start 2024-04-25T00:40:27.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.241+0000 7f1703bcc640 1 -- start start 2024-04-25T00:40:27.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1703bcc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16fc0b3140 0x7f16fc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1703bcc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f16fc0b4b10 0x7f16fc0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1703bcc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16fc0b41e0 con 0x7f16fc0b3a60 2024-04-25T00:40:27.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1703bcc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f16fc06d960 con 0x7f16fc0b4b10 2024-04-25T00:40:27.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1703bcc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f16fc06dc40 con 0x7f16fc0b3140 2024-04-25T00:40:27.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1702169640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f16fc0b3a60 0x7f16fc15c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38400/0 (socket says 172.21.15.29:38400) 2024-04-25T00:40:27.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1702169640 1 -- 172.21.15.29:0/1743043996 learned_addr learned my addr 172.21.15.29:0/1743043996 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:27.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1701167640 1 --2- 172.21.15.29:0/1743043996 >> v2:172.21.15.157:3300/0 conn(0x7f16fc0b4b10 0x7f16fc0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1701968640 1 --2- 172.21.15.29:0/1743043996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16fc0b3140 0x7f16fc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 689081893 0 0) 0x7f16fc0b41e0 con 0x7f16fc0b3a60 2024-04-25T00:40:27.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16e4003530 con 0x7f16fc0b3a60 2024-04-25T00:40:27.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 244617327 0 0) 0x7f16e4003530 con 0x7f16fc0b3a60 2024-04-25T00:40:27.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16fc0b3140 msgr2=0x7f16fc06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:27.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 --2- 172.21.15.29:0/1743043996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16fc0b3140 0x7f16fc06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 >> v2:172.21.15.157:3300/0 conn(0x7f16fc0b4b10 msgr2=0x7f16fc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 --2- 172.21.15.29:0/1743043996 >> v2:172.21.15.157:3300/0 conn(0x7f16fc0b4b10 0x7f16fc0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc06df20 con 0x7f16fc0b3a60 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f16ec001680 con 0x7f16fc0b3a60 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 >> v1:172.21.15.29:6789/0 conn(0x7f16fc0b3a60 legacy=0x7f16fc15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 --2- 172.21.15.29:0/1743043996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16fc0b3140 0x7f16e40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 --2- 172.21.15.29:0/1743043996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16e4003f40 0x7f16e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 --2- 172.21.15.29:0/1743043996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16e4004d50 0x7f16e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc06d960 con 0x7f16e4004d50 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc06dc40 con 0x7f16e4003f40 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f16e400b9d0 con 0x7f16fc0b3140 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1702169640 1 --2- 172.21.15.29:0/1743043996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16e4004d50 0x7f16e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1701167640 1 --2- 172.21.15.29:0/1743043996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16e4003f40 0x7f16e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1701968640 1 --2- 172.21.15.29:0/1743043996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16fc0b3140 0x7f16e40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1702169640 1 -- 172.21.15.29:0/1743043996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16fc0b3140 msgr2=0x7f16e40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:27.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1702169640 1 --2- 172.21.15.29:0/1743043996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16fc0b3140 0x7f16e40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1702169640 1 -- 172.21.15.29:0/1743043996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16e4003f40 msgr2=0x7f16e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:27.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1702169640 1 --2- 172.21.15.29:0/1743043996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16e4003f40 0x7f16e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1702169640 1 -- 172.21.15.29:0/1743043996 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e4003530 con 0x7f16e4004d50 2024-04-25T00:40:27.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.245+0000 7f1701167640 1 --2- 172.21.15.29:0/1743043996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16e4003f40 0x7f16e4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:27.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1702169640 1 --2- 172.21.15.29:0/1743043996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16e4004d50 0x7f16e40051d0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00c240 tx=0x7f16ec01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda594028c9d2197 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:27.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec007d60 con 0x7f16e4004d50 2024-04-25T00:40:27.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16ec0a9020 con 0x7f16e4004d50 2024-04-25T00:40:27.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1700966640 1 -- 172.21.15.29:0/1743043996 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0a5bc0 con 0x7f16e4004d50 2024-04-25T00:40:27.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 -- 172.21.15.29:0/1743043996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16e4004d50 msgr2=0x7f16e40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:27.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 --2- 172.21.15.29:0/1743043996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16e4004d50 0x7f16e40051d0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00c240 tx=0x7f16ec01cc60 comp rx=0 tx=0).stop 2024-04-25T00:40:27.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1702169640 1 -- 172.21.15.29:0/1743043996 reap_dead start 2024-04-25T00:40:27.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 -- 172.21.15.29:0/1743043996 shutdown_connections 2024-04-25T00:40:27.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 -- 172.21.15.29:0/1743043996 >> 172.21.15.29:0/1743043996 conn(0x7f16fc0b1870 msgr2=0x7f16fc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:27.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 -- 172.21.15.29:0/1743043996 shutdown_connections 2024-04-25T00:40:27.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 -- 172.21.15.29:0/1743043996 wait complete. 2024-04-25T00:40:27.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 Processor -- start 2024-04-25T00:40:27.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 -- start start 2024-04-25T00:40:27.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16fc0726d0 0x7f16fc1f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16fc1f3fc0 0x7f16fc13b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16fc13b610 0x7f16fc0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc145910 con 0x7f16fc13b610 2024-04-25T00:40:27.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc0b4f70 con 0x7f16fc0726d0 2024-04-25T00:40:27.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1703bcc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f16fc145a90 con 0x7f16fc1f3fc0 2024-04-25T00:40:27.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1702169640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16fc13b610 0x7f16fc0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1702169640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16fc13b610 0x7f16fc0ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53226/0 (socket says 172.21.15.29:53226) 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1702169640 1 -- 172.21.15.29:0/1329772769 learned_addr learned my addr 172.21.15.29:0/1329772769 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1701968640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16fc0726d0 0x7f16fc1f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.249+0000 7f1701167640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16fc1f3fc0 0x7f16fc13b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f1702169640 1 -- 172.21.15.29:0/1329772769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16fc1f3fc0 msgr2=0x7f16fc13b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f1702169640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16fc1f3fc0 0x7f16fc13b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f1702169640 1 -- 172.21.15.29:0/1329772769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16fc0726d0 msgr2=0x7f16fc1f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f1702169640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16fc0726d0 0x7f16fc1f3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f1702169640 1 -- 172.21.15.29:0/1329772769 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc1f45b0 con 0x7f16fc13b610 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f1702169640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16fc13b610 0x7f16fc0ad8a0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00c240 tx=0x7f16e4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f16fa7fc640 1 -- 172.21.15.29:0/1329772769 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec001ca0 con 0x7f16fc13b610 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f16fa7fc640 1 -- 172.21.15.29:0/1329772769 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16ec004ab0 con 0x7f16fc13b610 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f1703bcc640 1 -- 172.21.15.29:0/1329772769 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16fc1f4890 con 0x7f16fc13b610 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f16fa7fc640 1 -- 172.21.15.29:0/1329772769 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec000e50 con 0x7f16fc13b610 2024-04-25T00:40:27.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f1703bcc640 1 -- 172.21.15.29:0/1329772769 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16fc072ab0 con 0x7f16fc13b610 2024-04-25T00:40:27.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f1703bcc640 1 -- 172.21.15.29:0/1329772769 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16fc13a6c0 con 0x7f16fc13b610 2024-04-25T00:40:27.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f16fa7fc640 1 -- 172.21.15.29:0/1329772769 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f16ec0bd060 con 0x7f16fc13b610 2024-04-25T00:40:27.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f16fa7fc640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f16e80680a0 0x7f16e806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f16fa7fc640 1 -- 172.21.15.29:0/1329772769 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4857+0+0 (secure 0 0 0) 0x7f16ec11da40 con 0x7f16fc13b610 2024-04-25T00:40:27.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f1701968640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f16e80680a0 0x7f16e806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.253+0000 7f1701968640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f16e80680a0 0x7f16e806a560 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f16fc13ef30 tx=0x7f16f400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:27.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.257+0000 7f16fa7fc640 1 -- 172.21.15.29:0/1329772769 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f16ec0169d0 con 0x7f16fc13b610 2024-04-25T00:40:27.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.421+0000 7f1703bcc640 1 -- 172.21.15.29:0/1329772769 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f16fc069270 con 0x7f16fc13b610 2024-04-25T00:40:27.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.421+0000 7f16fa7fc640 1 -- 172.21.15.29:0/1329772769 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v152) ==== 84+0+65 (secure 0 0 0) 0x7f16ec0bd350 con 0x7f16fc13b610 2024-04-25T00:40:27.421 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:27.421 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:40:27.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:27.421 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-04-25T00:40:27.421 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": ".mgr" 2024-04-25T00:40:27.421 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:27.421 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:40:27.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.421+0000 7f1703bcc640 1 -- 172.21.15.29:0/1329772769 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f16e80680a0 msgr2=0x7f16e806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:27.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.421+0000 7f1703bcc640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f16e80680a0 0x7f16e806a560 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f16fc13ef30 tx=0x7f16f400b040 comp rx=0 tx=0).stop 2024-04-25T00:40:27.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.421+0000 7f1703bcc640 1 -- 172.21.15.29:0/1329772769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16fc13b610 msgr2=0x7f16fc0ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:27.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.421+0000 7f1703bcc640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16fc13b610 0x7f16fc0ad8a0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00c240 tx=0x7f16e4005840 comp rx=0 tx=0).stop 2024-04-25T00:40:27.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.421+0000 7f1703bcc640 1 -- 172.21.15.29:0/1329772769 shutdown_connections 2024-04-25T00:40:27.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.421+0000 7f1703bcc640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f16e80680a0 0x7f16e806a560 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.421+0000 7f1703bcc640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16fc13b610 0x7f16fc0ad8a0 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.421+0000 7f1703bcc640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16fc1f3fc0 0x7f16fc13b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.421+0000 7f1703bcc640 1 --2- 172.21.15.29:0/1329772769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16fc0726d0 0x7f16fc1f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.421+0000 7f1703bcc640 1 -- 172.21.15.29:0/1329772769 >> 172.21.15.29:0/1329772769 conn(0x7f16fc0b1870 msgr2=0x7f16fc06cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:27.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.425+0000 7f1703bcc640 1 -- 172.21.15.29:0/1329772769 shutdown_connections 2024-04-25T00:40:27.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.425+0000 7f1703bcc640 1 -- 172.21.15.29:0/1329772769 wait complete. 2024-04-25T00:40:27.441 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:40:27.442 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:40:27.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.577+0000 7f01980b0640 1 Processor -- start 2024-04-25T00:40:27.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.577+0000 7f01980b0640 1 -- start start 2024-04-25T00:40:27.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.577+0000 7f01980b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019006cf60 0x7f01900c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.577+0000 7f01980b0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01900c87e0 con 0x7f019015e290 2024-04-25T00:40:27.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.577+0000 7f01980b0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01900cfa40 con 0x7f01900c82d0 2024-04-25T00:40:27.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.577+0000 7f01980b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f019015e670 con 0x7f019006cf60 2024-04-25T00:40:27.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.577+0000 7f0195e4c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f019015e290 0x7f019006c850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38412/0 (socket says 172.21.15.29:38412) 2024-04-25T00:40:27.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.577+0000 7f0195e4c640 1 -- 172.21.15.29:0/3911392076 learned_addr learned my addr 172.21.15.29:0/3911392076 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:27.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f019564b640 1 --2- 172.21.15.29:0/3911392076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019006cf60 0x7f01900c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2862847561 0 0) 0x7f01900c87e0 con 0x7f019015e290 2024-04-25T00:40:27.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f017c003530 con 0x7f019015e290 2024-04-25T00:40:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 376781095 0 0) 0x7f01900cfa40 con 0x7f01900c82d0 2024-04-25T00:40:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01900c87e0 con 0x7f01900c82d0 2024-04-25T00:40:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3936164516 0 0) 0x7f017c003530 con 0x7f019015e290 2024-04-25T00:40:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019006cf60 msgr2=0x7f01900c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 --2- 172.21.15.29:0/3911392076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019006cf60 0x7f01900c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 >> v1:172.21.15.157:6789/0 conn(0x7f01900c82d0 legacy=0x7f01900d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01900cfd20 con 0x7f019015e290 2024-04-25T00:40:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f018c001680 con 0x7f019015e290 2024-04-25T00:40:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 >> v1:172.21.15.29:6789/0 conn(0x7f019015e290 legacy=0x7f019006c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 --2- 172.21.15.29:0/3911392076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f017c003b80 0x7f017c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 --2- 172.21.15.29:0/3911392076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019006cf60 0x7f017c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 --2- 172.21.15.29:0/3911392076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f017c004d50 0x7f017c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f019015e670 con 0x7f017c004d50 2024-04-25T00:40:27.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f017c00ba80 con 0x7f017c003b80 2024-04-25T00:40:27.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f017c00bc30 con 0x7f019006cf60 2024-04-25T00:40:27.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f019664d640 1 --2- 172.21.15.29:0/3911392076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f017c004d50 0x7f017c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0195e4c640 1 --2- 172.21.15.29:0/3911392076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f017c003b80 0x7f017c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f019564b640 1 --2- 172.21.15.29:0/3911392076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019006cf60 0x7f017c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f019664d640 1 -- 172.21.15.29:0/3911392076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019006cf60 msgr2=0x7f017c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:27.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f019664d640 1 --2- 172.21.15.29:0/3911392076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019006cf60 0x7f017c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f019664d640 1 -- 172.21.15.29:0/3911392076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f017c003b80 msgr2=0x7f017c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:27.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f019664d640 1 --2- 172.21.15.29:0/3911392076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f017c003b80 0x7f017c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f019664d640 1 -- 172.21.15.29:0/3911392076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c003530 con 0x7f017c004d50 2024-04-25T00:40:27.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0195e4c640 1 --2- 172.21.15.29:0/3911392076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f017c003b80 0x7f017c003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:27.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f019664d640 1 --2- 172.21.15.29:0/3911392076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f017c004d50 0x7f017c0051d0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f0188001fd0 tx=0x7f018801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a5a37a10895721 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:27.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188001c20 con 0x7f017c004d50 2024-04-25T00:40:27.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0188003ba0 con 0x7f017c004d50 2024-04-25T00:40:27.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.581+0000 7f0194e4a640 1 -- 172.21.15.29:0/3911392076 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01880050d0 con 0x7f017c004d50 2024-04-25T00:40:27.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 -- 172.21.15.29:0/3911392076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f017c004d50 msgr2=0x7f017c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:27.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 --2- 172.21.15.29:0/3911392076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f017c004d50 0x7f017c0051d0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f0188001fd0 tx=0x7f018801c900 comp rx=0 tx=0).stop 2024-04-25T00:40:27.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f019664d640 1 -- 172.21.15.29:0/3911392076 reap_dead start 2024-04-25T00:40:27.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 -- 172.21.15.29:0/3911392076 shutdown_connections 2024-04-25T00:40:27.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 -- 172.21.15.29:0/3911392076 >> 172.21.15.29:0/3911392076 conn(0x7f01900b3a00 msgr2=0x7f01900b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:27.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 -- 172.21.15.29:0/3911392076 shutdown_connections 2024-04-25T00:40:27.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 -- 172.21.15.29:0/3911392076 wait complete. 2024-04-25T00:40:27.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 Processor -- start 2024-04-25T00:40:27.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 -- start start 2024-04-25T00:40:27.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f017c004d50 0x7f019015c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f019009c100 0x7f019009c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01900b22d0 0x7f01900b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f019006d4a0 con 0x7f019009c100 2024-04-25T00:40:27.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01900c8ac0 con 0x7f01900b22d0 2024-04-25T00:40:27.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f01980b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f019006d620 con 0x7f017c004d50 2024-04-25T00:40:27.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f019564b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f019009c100 0x7f019009c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f019564b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f019009c100 0x7f019009c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53238/0 (socket says 172.21.15.29:53238) 2024-04-25T00:40:27.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f019564b640 1 -- 172.21.15.29:0/1562068307 learned_addr learned my addr 172.21.15.29:0/1562068307 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:27.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f019664d640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01900b22d0 0x7f01900b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f019564b640 1 -- 172.21.15.29:0/1562068307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f017c004d50 msgr2=0x7f019015c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:27.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f0195e4c640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f017c004d50 0x7f019015c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f019564b640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f017c004d50 0x7f019015c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f019564b640 1 -- 172.21.15.29:0/1562068307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01900b22d0 msgr2=0x7f01900b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:27.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f019564b640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01900b22d0 0x7f01900b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:27.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f019564b640 1 -- 172.21.15.29:0/1562068307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019015c780 con 0x7f019009c100 2024-04-25T00:40:27.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.585+0000 7f019664d640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01900b22d0 0x7f01900b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:27.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.589+0000 7f019564b640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f019009c100 0x7f019009c580 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f01800115e0 tx=0x7f0180011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:27.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.589+0000 7f01867fc640 1 -- 172.21.15.29:0/1562068307 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180015870 con 0x7f019009c100 2024-04-25T00:40:27.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.589+0000 7f01867fc640 1 -- 172.21.15.29:0/1562068307 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0180015a10 con 0x7f019009c100 2024-04-25T00:40:27.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.589+0000 7f01867fc640 1 -- 172.21.15.29:0/1562068307 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180015dc0 con 0x7f019009c100 2024-04-25T00:40:27.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.589+0000 7f01980b0640 1 -- 172.21.15.29:0/1562068307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01900aee10 con 0x7f019009c100 2024-04-25T00:40:27.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.589+0000 7f01980b0640 1 -- 172.21.15.29:0/1562068307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01900af3d0 con 0x7f019009c100 2024-04-25T00:40:27.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.589+0000 7f01980b0640 1 -- 172.21.15.29:0/1562068307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01900692e0 con 0x7f019009c100 2024-04-25T00:40:27.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.589+0000 7f01867fc640 1 -- 172.21.15.29:0/1562068307 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f018000f480 con 0x7f019009c100 2024-04-25T00:40:27.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.589+0000 7f01867fc640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f01740680d0 0x7f017406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:27.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.589+0000 7f01867fc640 1 -- 172.21.15.29:0/1562068307 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4857+0+0 (secure 0 0 0) 0x7f0180084fc0 con 0x7f019009c100 2024-04-25T00:40:27.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.589+0000 7f0195e4c640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f01740680d0 0x7f017406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:27.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.589+0000 7f0195e4c640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f01740680d0 0x7f017406a590 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f018c000c10 tx=0x7f018c001c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:27.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.593+0000 7f01867fc640 1 -- 172.21.15.29:0/1562068307 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f019015c780 con 0x7f019009c100 2024-04-25T00:40:27.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:27.753+0000 7f01980b0640 1 -- 172.21.15.29:0/1562068307 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f01900b7fe0 con 0x7f019009c100 2024-04-25T00:40:28.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.049+0000 7f01867fc640 1 -- 172.21.15.29:0/1562068307 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v153) ==== 138+0+0 (secure 0 0 0) 0x7f0180019030 con 0x7f019009c100 2024-04-25T00:40:28.051 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-04-25T00:40:28.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.053+0000 7f01980b0640 1 -- 172.21.15.29:0/1562068307 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f01740680d0 msgr2=0x7f017406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:28.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.053+0000 7f01980b0640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f01740680d0 0x7f017406a590 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f018c000c10 tx=0x7f018c001c00 comp rx=0 tx=0).stop 2024-04-25T00:40:28.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.053+0000 7f01980b0640 1 -- 172.21.15.29:0/1562068307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f019009c100 msgr2=0x7f019009c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:28.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.053+0000 7f01980b0640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f019009c100 0x7f019009c580 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f01800115e0 tx=0x7f0180011aa0 comp rx=0 tx=0).stop 2024-04-25T00:40:28.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.053+0000 7f01980b0640 1 -- 172.21.15.29:0/1562068307 shutdown_connections 2024-04-25T00:40:28.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.053+0000 7f01980b0640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f01740680d0 0x7f017406a590 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:28.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.053+0000 7f01980b0640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01900b22d0 0x7f01900b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:28.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.053+0000 7f01980b0640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f019009c100 0x7f019009c580 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:28.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.053+0000 7f01980b0640 1 --2- 172.21.15.29:0/1562068307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f017c004d50 0x7f019015c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:28.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.053+0000 7f01980b0640 1 -- 172.21.15.29:0/1562068307 >> 172.21.15.29:0/1562068307 conn(0x7f01900b3a00 msgr2=0x7f019006f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:28.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.053+0000 7f01980b0640 1 -- 172.21.15.29:0/1562068307 shutdown_connections 2024-04-25T00:40:28.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.053+0000 7f01980b0640 1 -- 172.21.15.29:0/1562068307 wait complete. 2024-04-25T00:40:28.082 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:40:28.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f6718a14640 1 Processor -- start 2024-04-25T00:40:28.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f6718a14640 1 -- start start 2024-04-25T00:40:28.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f6718a14640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f671406d440 0x7f67140cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:28.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f6718a14640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67140cd720 0x7f67140d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:28.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f6718a14640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6714161a00 con 0x7f6714161620 2024-04-25T00:40:28.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f6718a14640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f67140cdfb0 con 0x7f671406d440 2024-04-25T00:40:28.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f6718a14640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f67140cffc0 con 0x7f67140cd720 2024-04-25T00:40:28.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f6711d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f671406d440 0x7f67140cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:28.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f671259c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6714161620 0x7f671406cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55726/0 (socket says 172.21.15.29:55726) 2024-04-25T00:40:28.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f671259c640 1 -- 172.21.15.29:0/1993587371 learned_addr learned my addr 172.21.15.29:0/1993587371 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:28.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f6712d9d640 1 --2- 172.21.15.29:0/1993587371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67140cd720 0x7f67140d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:28.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f6711d9b640 1 -- 172.21.15.29:0/1993587371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67140cd720 msgr2=0x7f67140d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:28.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f6711d9b640 1 --2- 172.21.15.29:0/1993587371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67140cd720 0x7f67140d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:28.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f6711d9b640 1 -- 172.21.15.29:0/1993587371 >> v1:172.21.15.29:6789/0 conn(0x7f6714161620 legacy=0x7f671406cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:28.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f6711d9b640 1 -- 172.21.15.29:0/1993587371 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67140d02a0 con 0x7f671406d440 2024-04-25T00:40:28.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.217+0000 7f671159a640 1 -- 172.21.15.29:0/1993587371 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3987455027 0 0) 0x7f6714161a00 con 0x7f6714161620 2024-04-25T00:40:28.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6711d9b640 1 --2- 172.21.15.29:0/1993587371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f671406d440 0x7f67140cd1e0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f67080097c0 tx=0x7f670801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236554789584bbf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:28.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f671159a640 1 -- 172.21.15.29:0/1993587371 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67080a7070 con 0x7f671406d440 2024-04-25T00:40:28.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f671159a640 1 -- 172.21.15.29:0/1993587371 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67080b0070 con 0x7f671406d440 2024-04-25T00:40:28.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f671159a640 1 -- 172.21.15.29:0/1993587371 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67080ab890 con 0x7f671406d440 2024-04-25T00:40:28.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 -- 172.21.15.29:0/1993587371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f671406d440 msgr2=0x7f67140cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:28.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 --2- 172.21.15.29:0/1993587371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f671406d440 0x7f67140cd1e0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f67080097c0 tx=0x7f670801c470 comp rx=0 tx=0).stop 2024-04-25T00:40:28.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 -- 172.21.15.29:0/1993587371 shutdown_connections 2024-04-25T00:40:28.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 --2- 172.21.15.29:0/1993587371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67140cd720 0x7f67140d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:28.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 --2- 172.21.15.29:0/1993587371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f671406d440 0x7f67140cd1e0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:28.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 -- 172.21.15.29:0/1993587371 >> 172.21.15.29:0/1993587371 conn(0x7f671409aa30 msgr2=0x7f671409ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:28.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 -- 172.21.15.29:0/1993587371 shutdown_connections 2024-04-25T00:40:28.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 -- 172.21.15.29:0/1993587371 wait complete. 2024-04-25T00:40:28.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 Processor -- start 2024-04-25T00:40:28.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 -- start start 2024-04-25T00:40:28.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f671406d440 0x7f67140b7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:28.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67140cd720 0x7f6714151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:28.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6714161620 0x7f671406f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:28.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f67140d1ad0 con 0x7f671406d440 2024-04-25T00:40:28.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f67140d1c50 con 0x7f67140cd720 2024-04-25T00:40:28.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f6718a14640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f67140d1dd0 con 0x7f6714161620 2024-04-25T00:40:28.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f671259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f671406d440 0x7f67140b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:28.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f671259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f671406d440 0x7f67140b7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59852/0 (socket says 172.21.15.29:59852) 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.221+0000 7f671259c640 1 -- 172.21.15.29:0/1120102290 learned_addr learned my addr 172.21.15.29:0/1120102290 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f6711d9b640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67140cd720 0x7f6714151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f6712d9d640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6714161620 0x7f671406f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f671259c640 1 -- 172.21.15.29:0/1120102290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6714161620 msgr2=0x7f671406f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f671259c640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6714161620 0x7f671406f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f671259c640 1 -- 172.21.15.29:0/1120102290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67140cd720 msgr2=0x7f6714151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f671259c640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67140cd720 0x7f6714151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f671259c640 1 -- 172.21.15.29:0/1120102290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67140ca2f0 con 0x7f671406d440 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f671259c640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f671406d440 0x7f67140b7470 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f670c00cb40 tx=0x7f670c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f6702ffd640 1 -- 172.21.15.29:0/1120102290 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c017070 con 0x7f671406d440 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f6702ffd640 1 -- 172.21.15.29:0/1120102290 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670c00aca0 con 0x7f671406d440 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f6718a14640 1 -- 172.21.15.29:0/1120102290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67140ca5d0 con 0x7f671406d440 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f6702ffd640 1 -- 172.21.15.29:0/1120102290 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c012620 con 0x7f671406d440 2024-04-25T00:40:28.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f6718a14640 1 -- 172.21.15.29:0/1120102290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67140cab90 con 0x7f671406d440 2024-04-25T00:40:28.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f6718a14640 1 -- 172.21.15.29:0/1120102290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6714069270 con 0x7f671406d440 2024-04-25T00:40:28.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f6702ffd640 1 -- 172.21.15.29:0/1120102290 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f670c01a070 con 0x7f671406d440 2024-04-25T00:40:28.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f6702ffd640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66f40680d0 0x7f66f406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:28.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f6702ffd640 1 -- 172.21.15.29:0/1120102290 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5227+0+0 (secure 0 0 0) 0x7f670c086e50 con 0x7f671406d440 2024-04-25T00:40:28.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f6711d9b640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66f40680d0 0x7f66f406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:28.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.225+0000 7f6711d9b640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66f40680d0 0x7f66f406a590 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f67080af000 tx=0x7f67080a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:28.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.229+0000 7f6702ffd640 1 -- 172.21.15.29:0/1120102290 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f670c021020 con 0x7f671406d440 2024-04-25T00:40:28.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:28.385+0000 7f6718a14640 1 -- 172.21.15.29:0/1120102290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f671409b470 con 0x7f671406d440 2024-04-25T00:40:29.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.057+0000 7f6702ffd640 1 -- 172.21.15.29:0/1120102290 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v154) ==== 170+0+0 (secure 0 0 0) 0x7f670c056c90 con 0x7f671406d440 2024-04-25T00:40:29.056 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-04-25T00:40:29.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.057+0000 7f6718a14640 1 -- 172.21.15.29:0/1120102290 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66f40680d0 msgr2=0x7f66f406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:29.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.057+0000 7f6718a14640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66f40680d0 0x7f66f406a590 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f67080af000 tx=0x7f67080a7000 comp rx=0 tx=0).stop 2024-04-25T00:40:29.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.057+0000 7f6718a14640 1 -- 172.21.15.29:0/1120102290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f671406d440 msgr2=0x7f67140b7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:29.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.057+0000 7f6718a14640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f671406d440 0x7f67140b7470 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f670c00cb40 tx=0x7f670c00a060 comp rx=0 tx=0).stop 2024-04-25T00:40:29.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.057+0000 7f6718a14640 1 -- 172.21.15.29:0/1120102290 shutdown_connections 2024-04-25T00:40:29.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.057+0000 7f6718a14640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f66f40680d0 0x7f66f406a590 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:29.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.057+0000 7f6718a14640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6714161620 0x7f671406f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:29.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.057+0000 7f6718a14640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67140cd720 0x7f6714151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:29.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.057+0000 7f6718a14640 1 --2- 172.21.15.29:0/1120102290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f671406d440 0x7f67140b7470 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:29.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.057+0000 7f6718a14640 1 -- 172.21.15.29:0/1120102290 >> 172.21.15.29:0/1120102290 conn(0x7f671409aa30 msgr2=0x7f67140ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:29.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.057+0000 7f6718a14640 1 -- 172.21.15.29:0/1120102290 shutdown_connections 2024-04-25T00:40:29.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.057+0000 7f6718a14640 1 -- 172.21.15.29:0/1120102290 wait complete. 2024-04-25T00:40:29.090 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:40:29.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.209+0000 7fc105bf7640 1 Processor -- start 2024-04-25T00:40:29.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.209+0000 7fc105bf7640 1 -- start start 2024-04-25T00:40:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.209+0000 7fc105bf7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc1000b3c30 0x7fc10006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.209+0000 7fc105bf7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1000b41e0 con 0x7fc1000b4b10 2024-04-25T00:40:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.209+0000 7fc105bf7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc10006d960 con 0x7fc1000b3140 2024-04-25T00:40:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.209+0000 7fc105bf7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc10006dc40 con 0x7fc1000b3c30 2024-04-25T00:40:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc0feffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc1000b3c30 0x7fc10006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc0fffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc1000b4b10 0x7fc1000ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55730/0 (socket says 172.21.15.29:55730) 2024-04-25T00:40:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc0fffff640 1 -- 172.21.15.29:0/2754671153 learned_addr learned my addr 172.21.15.29:0/2754671153 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc0feffd640 1 -- 172.21.15.29:0/2754671153 >> v1:172.21.15.157:6789/0 conn(0x7fc1000b3140 legacy=0x7fc1000b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc0feffd640 1 -- 172.21.15.29:0/2754671153 >> v1:172.21.15.29:6789/0 conn(0x7fc1000b4b10 legacy=0x7fc1000ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc0feffd640 1 -- 172.21.15.29:0/2754671153 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10006df20 con 0x7fc1000b3c30 2024-04-25T00:40:29.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc0feffd640 1 --2- 172.21.15.29:0/2754671153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1000b3c30 0x7fc10006e440 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40097f0 tx=0x7fc0f401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef82b9c420dcc678 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:29.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc0fe7fc640 1 -- 172.21.15.29:0/2754671153 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40b0070 con 0x7fc1000b3c30 2024-04-25T00:40:29.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc0fe7fc640 1 -- 172.21.15.29:0/2754671153 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f40ab600 con 0x7fc1000b3c30 2024-04-25T00:40:29.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc0fe7fc640 1 -- 172.21.15.29:0/2754671153 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40ab9d0 con 0x7fc1000b3c30 2024-04-25T00:40:29.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc105bf7640 1 -- 172.21.15.29:0/2754671153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1000b3c30 msgr2=0x7fc10006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:29.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc105bf7640 1 --2- 172.21.15.29:0/2754671153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1000b3c30 0x7fc10006e440 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40097f0 tx=0x7fc0f401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:40:29.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc105bf7640 1 -- 172.21.15.29:0/2754671153 shutdown_connections 2024-04-25T00:40:29.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc105bf7640 1 --2- 172.21.15.29:0/2754671153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1000b3c30 0x7fc10006e440 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:29.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc105bf7640 1 -- 172.21.15.29:0/2754671153 >> 172.21.15.29:0/2754671153 conn(0x7fc1000b1870 msgr2=0x7fc1000b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:29.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc105bf7640 1 -- 172.21.15.29:0/2754671153 shutdown_connections 2024-04-25T00:40:29.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.213+0000 7fc105bf7640 1 -- 172.21.15.29:0/2754671153 wait complete. 2024-04-25T00:40:29.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc105bf7640 1 Processor -- start 2024-04-25T00:40:29.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc105bf7640 1 -- start start 2024-04-25T00:40:29.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc105bf7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1000b3140 0x7fc100153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:29.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc105bf7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1000b3c30 0x7fc100159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:29.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc105bf7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1000b4b10 0x7fc100154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:29.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc105bf7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc100169d90 con 0x7fc1000b4b10 2024-04-25T00:40:29.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc105bf7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc100169c10 con 0x7fc1000b3140 2024-04-25T00:40:29.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc105bf7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc100169f10 con 0x7fc1000b3c30 2024-04-25T00:40:29.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1000b4b10 0x7fc100154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1000b4b10 0x7fc100154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59878/0 (socket says 172.21.15.29:59878) 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0fffff640 1 -- 172.21.15.29:0/2274370074 learned_addr learned my addr 172.21.15.29:0/2274370074 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0feffd640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1000b3c30 0x7fc100159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0ff7fe640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1000b3140 0x7fc100153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0fffff640 1 -- 172.21.15.29:0/2274370074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1000b3c30 msgr2=0x7fc100159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0fffff640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1000b3c30 0x7fc100159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0fffff640 1 -- 172.21.15.29:0/2274370074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1000b3140 msgr2=0x7fc100153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0fffff640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1000b3140 0x7fc100153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0fffff640 1 -- 172.21.15.29:0/2274370074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10015a8a0 con 0x7fc1000b4b10 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0ff7fe640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1000b3140 0x7fc100153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0fffff640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1000b4b10 0x7fc100154430 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800cbc0 tx=0x7fc0f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0e7fff640 1 -- 172.21.15.29:0/2274370074 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8017070 con 0x7fc1000b4b10 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0e7fff640 1 -- 172.21.15.29:0/2274370074 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f800ad40 con 0x7fc1000b4b10 2024-04-25T00:40:29.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc0e7fff640 1 -- 172.21.15.29:0/2274370074 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8012600 con 0x7fc1000b4b10 2024-04-25T00:40:29.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc105bf7640 1 -- 172.21.15.29:0/2274370074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10015ab80 con 0x7fc1000b4b10 2024-04-25T00:40:29.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc105bf7640 1 -- 172.21.15.29:0/2274370074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10015b140 con 0x7fc1000b4b10 2024-04-25T00:40:29.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.217+0000 7fc105bf7640 1 -- 172.21.15.29:0/2274370074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc100069270 con 0x7fc1000b4b10 2024-04-25T00:40:29.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.221+0000 7fc0e7fff640 1 -- 172.21.15.29:0/2274370074 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fc0f801a070 con 0x7fc1000b4b10 2024-04-25T00:40:29.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.221+0000 7fc0e7fff640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc0ec0680d0 0x7fc0ec06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:29.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.221+0000 7fc0e7fff640 1 -- 172.21.15.29:0/2274370074 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5609+0+0 (secure 0 0 0) 0x7fc0f8084320 con 0x7fc1000b4b10 2024-04-25T00:40:29.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.221+0000 7fc0ff7fe640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc0ec0680d0 0x7fc0ec06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:29.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.221+0000 7fc0ff7fe640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc0ec0680d0 0x7fc0ec06a590 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0000f70 tx=0x7fc0f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:29.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.225+0000 7fc0e7fff640 1 -- 172.21.15.29:0/2274370074 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc0f8056080 con 0x7fc1000b4b10 2024-04-25T00:40:29.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:29.413+0000 7fc105bf7640 1 -- 172.21.15.29:0/2274370074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc1000cf6d0 con 0x7fc1000b4b10 2024-04-25T00:40:30.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.093+0000 7fc0e7fff640 1 -- 172.21.15.29:0/2274370074 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v98) ==== 174+0+0 (secure 0 0 0) 0x7fc0f805a380 con 0x7fc1000b4b10 2024-04-25T00:40:30.093 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 24 and data pool 25 2024-04-25T00:40:30.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.109+0000 7fc105bf7640 1 -- 172.21.15.29:0/2274370074 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc0ec0680d0 msgr2=0x7fc0ec06a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.109+0000 7fc105bf7640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc0ec0680d0 0x7fc0ec06a590 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0000f70 tx=0x7fc0f0009040 comp rx=0 tx=0).stop 2024-04-25T00:40:30.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.109+0000 7fc105bf7640 1 -- 172.21.15.29:0/2274370074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1000b4b10 msgr2=0x7fc100154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.109+0000 7fc105bf7640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1000b4b10 0x7fc100154430 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800cbc0 tx=0x7fc0f800a430 comp rx=0 tx=0).stop 2024-04-25T00:40:30.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.113+0000 7fc105bf7640 1 -- 172.21.15.29:0/2274370074 shutdown_connections 2024-04-25T00:40:30.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.113+0000 7fc105bf7640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc0ec0680d0 0x7fc0ec06a590 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.113+0000 7fc105bf7640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1000b4b10 0x7fc100154430 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.113+0000 7fc105bf7640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1000b3c30 0x7fc100159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.113+0000 7fc105bf7640 1 --2- 172.21.15.29:0/2274370074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1000b3140 0x7fc100153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.113+0000 7fc105bf7640 1 -- 172.21.15.29:0/2274370074 >> 172.21.15.29:0/2274370074 conn(0x7fc1000b1870 msgr2=0x7fc10015b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:30.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.117+0000 7fc105bf7640 1 -- 172.21.15.29:0/2274370074 shutdown_connections 2024-04-25T00:40:30.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.117+0000 7fc105bf7640 1 -- 172.21.15.29:0/2274370074 wait complete. 2024-04-25T00:40:30.142 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:40:30.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.253+0000 7f9d36b74640 1 Processor -- start 2024-04-25T00:40:30.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.253+0000 7f9d36b74640 1 -- start start 2024-04-25T00:40:30.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d36b74640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d30161800 0x7f9d300d4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d36b74640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d300d1e10 0x7f9d300d24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d36b74640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d300d1330 con 0x7f9d300d2a30 2024-04-25T00:40:30.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d36b74640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9d300d1610 con 0x7f9d300d1e10 2024-04-25T00:40:30.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d36b74640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9d300b34a0 con 0x7f9d30161800 2024-04-25T00:40:30.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d2ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d300d1e10 0x7f9d300d24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d34910640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d30161800 0x7f9d300d4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d2ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d300d1e10 0x7f9d300d24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:59028/0 (socket says 172.21.15.29:59028) 2024-04-25T00:40:30.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d35111640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9d300d2a30 0x7f9d300ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:59376/0 (socket says 172.21.15.29:59376) 2024-04-25T00:40:30.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d2ffff640 1 -- 172.21.15.29:0/2553498539 learned_addr learned my addr 172.21.15.29:0/2553498539 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:30.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d2ffff640 1 -- 172.21.15.29:0/2553498539 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d30161800 msgr2=0x7f9d300d4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:30.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d2ffff640 1 --2- 172.21.15.29:0/2553498539 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d30161800 0x7f9d300d4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d2ffff640 1 -- 172.21.15.29:0/2553498539 >> v1:172.21.15.157:6789/0 conn(0x7f9d300d2a30 legacy=0x7f9d300ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d2ffff640 1 -- 172.21.15.29:0/2553498539 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d300b3780 con 0x7f9d300d1e10 2024-04-25T00:40:30.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.257+0000 7f9d2ffff640 1 --2- 172.21.15.29:0/2553498539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d300d1e10 0x7f9d300d24f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009870 tx=0x7f9d1c01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=846ae5de7a89a130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:30.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2f7fe640 1 -- 172.21.15.29:0/2553498539 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c0b0070 con 0x7f9d300d1e10 2024-04-25T00:40:30.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2f7fe640 1 -- 172.21.15.29:0/2553498539 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d1c0ab600 con 0x7f9d300d1e10 2024-04-25T00:40:30.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2f7fe640 1 -- 172.21.15.29:0/2553498539 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c0ab9d0 con 0x7f9d300d1e10 2024-04-25T00:40:30.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 -- 172.21.15.29:0/2553498539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d300d1e10 msgr2=0x7f9d300d24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 --2- 172.21.15.29:0/2553498539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d300d1e10 0x7f9d300d24f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009870 tx=0x7f9d1c01c4d0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 -- 172.21.15.29:0/2553498539 shutdown_connections 2024-04-25T00:40:30.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 --2- 172.21.15.29:0/2553498539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d300d1e10 0x7f9d300d24f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 --2- 172.21.15.29:0/2553498539 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d30161800 0x7f9d300d4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 -- 172.21.15.29:0/2553498539 >> 172.21.15.29:0/2553498539 conn(0x7f9d300d3f10 msgr2=0x7f9d300d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:30.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 -- 172.21.15.29:0/2553498539 shutdown_connections 2024-04-25T00:40:30.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 -- 172.21.15.29:0/2553498539 wait complete. 2024-04-25T00:40:30.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 Processor -- start 2024-04-25T00:40:30.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 -- start start 2024-04-25T00:40:30.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d300d1e10 0x7f9d300cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d300d2a30 0x7f9d300b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d30161800 0x7f9d300b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3016b1f0 con 0x7f9d300d2a30 2024-04-25T00:40:30.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9d300d1110 con 0x7f9d300d1e10 2024-04-25T00:40:30.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3016b370 con 0x7f9d30161800 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d300d2a30 0x7f9d300b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d300d2a30 0x7f9d300b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59880/0 (socket says 172.21.15.29:59880) 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2ffff640 1 -- 172.21.15.29:0/3781157215 learned_addr learned my addr 172.21.15.29:0/3781157215 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d34910640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d300d1e10 0x7f9d300cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d35111640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d30161800 0x7f9d300b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2ffff640 1 -- 172.21.15.29:0/3781157215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d30161800 msgr2=0x7f9d300b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2ffff640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d30161800 0x7f9d300b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2ffff640 1 -- 172.21.15.29:0/3781157215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d300d1e10 msgr2=0x7f9d300cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2ffff640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d300d1e10 0x7f9d300cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2ffff640 1 -- 172.21.15.29:0/3781157215 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d300ccd30 con 0x7f9d300d2a30 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d34910640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d300d1e10 0x7f9d300cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2ffff640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d300d2a30 0x7f9d300b1b30 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0ae040 tx=0x7f9d1c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2d7fa640 1 -- 172.21.15.29:0/3781157215 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c0b0040 con 0x7f9d300d2a30 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2d7fa640 1 -- 172.21.15.29:0/3781157215 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d1c0b4410 con 0x7f9d300d2a30 2024-04-25T00:40:30.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 -- 172.21.15.29:0/3781157215 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d300cc120 con 0x7f9d300d2a30 2024-04-25T00:40:30.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d2d7fa640 1 -- 172.21.15.29:0/3781157215 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c0b4820 con 0x7f9d300d2a30 2024-04-25T00:40:30.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.261+0000 7f9d36b74640 1 -- 172.21.15.29:0/3781157215 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d300cc6e0 con 0x7f9d300d2a30 2024-04-25T00:40:30.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.265+0000 7f9d36b74640 1 -- 172.21.15.29:0/3781157215 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d30069270 con 0x7f9d300d2a30 2024-04-25T00:40:30.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.265+0000 7f9d2d7fa640 1 -- 172.21.15.29:0/3781157215 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f9d1c0b49c0 con 0x7f9d300d2a30 2024-04-25T00:40:30.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.265+0000 7f9d2d7fa640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d140680a0 0x7f9d1406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.265+0000 7f9d2d7fa640 1 -- 172.21.15.29:0/3781157215 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7f9d1c11ed30 con 0x7f9d300d2a30 2024-04-25T00:40:30.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.265+0000 7f9d34910640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d140680a0 0x7f9d1406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.265+0000 7f9d34910640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d140680a0 0x7f9d1406a560 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009bb0 tx=0x7f9d24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:30.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.269+0000 7f9d2d7fa640 1 -- 172.21.15.29:0/3781157215 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9d1c016150 con 0x7f9d300d2a30 2024-04-25T00:40:30.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.433+0000 7f9d36b74640 1 -- 172.21.15.29:0/3781157215 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d300b5aa0 con 0x7f9d300d2a30 2024-04-25T00:40:30.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.433+0000 7f9d2d7fa640 1 -- 172.21.15.29:0/3781157215 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+16644 (secure 0 0 0) 0x7f9d1c0f1160 con 0x7f9d300d2a30 2024-04-25T00:40:30.435 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:30.436 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":155,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:40:29.418735+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:40:27.796735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:40:28.430727+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:40:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.437+0000 7f9d36b74640 1 -- 172.21.15.29:0/3781157215 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d140680a0 msgr2=0x7f9d1406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.437+0000 7f9d36b74640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d140680a0 0x7f9d1406a560 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009bb0 tx=0x7f9d24008040 comp rx=0 tx=0).stop 2024-04-25T00:40:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.437+0000 7f9d36b74640 1 -- 172.21.15.29:0/3781157215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d300d2a30 msgr2=0x7f9d300b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.437+0000 7f9d36b74640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d300d2a30 0x7f9d300b1b30 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0ae040 tx=0x7f9d1c019270 comp rx=0 tx=0).stop 2024-04-25T00:40:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.437+0000 7f9d36b74640 1 -- 172.21.15.29:0/3781157215 shutdown_connections 2024-04-25T00:40:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.437+0000 7f9d36b74640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9d140680a0 0x7f9d1406a560 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.437+0000 7f9d36b74640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d30161800 0x7f9d300b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.437+0000 7f9d36b74640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d300d2a30 0x7f9d300b1b30 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.437+0000 7f9d36b74640 1 --2- 172.21.15.29:0/3781157215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d300d1e10 0x7f9d300cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.437+0000 7f9d36b74640 1 -- 172.21.15.29:0/3781157215 >> 172.21.15.29:0/3781157215 conn(0x7f9d300d3f10 msgr2=0x7f9d300b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.437+0000 7f9d36b74640 1 -- 172.21.15.29:0/3781157215 shutdown_connections 2024-04-25T00:40:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.437+0000 7f9d36b74640 1 -- 172.21.15.29:0/3781157215 wait complete. 2024-04-25T00:40:30.456 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:40:30.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d71fb640 1 Processor -- start 2024-04-25T00:40:30.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d71fb640 1 -- start start 2024-04-25T00:40:30.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d71fb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff2d00b3140 0x7ff2d006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d71fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2d00b3c30 0x7ff2d00ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d71fb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2d00b41e0 con 0x7ff2d00b4b10 2024-04-25T00:40:30.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d71fb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff2d006d960 con 0x7ff2d00b3140 2024-04-25T00:40:30.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d71fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff2d006dc40 con 0x7ff2d00b3c30 2024-04-25T00:40:30.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d4f97640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff2d00b3140 0x7ff2d006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d4f97640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff2d00b3140 0x7ff2d006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59888/0 (socket says 172.21.15.29:59888) 2024-04-25T00:40:30.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d4f97640 1 -- 172.21.15.29:0/3582819563 learned_addr learned my addr 172.21.15.29:0/3582819563 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:30.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d5798640 1 --2- 172.21.15.29:0/3582819563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2d00b3c30 0x7ff2d00ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d4f97640 1 -- 172.21.15.29:0/3582819563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2d00b3c30 msgr2=0x7ff2d00ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:30.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d4f97640 1 --2- 172.21.15.29:0/3582819563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2d00b3c30 0x7ff2d00ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d4f97640 1 -- 172.21.15.29:0/3582819563 >> v1:172.21.15.157:6789/0 conn(0x7ff2d00b4b10 legacy=0x7ff2d00b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.577+0000 7ff2d4f97640 1 -- 172.21.15.29:0/3582819563 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d006df20 con 0x7ff2d00b3140 2024-04-25T00:40:30.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d4f97640 1 --2- 172.21.15.29:0/3582819563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2d00b3140 0x7ff2d006e440 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80098a0 tx=0x7ff2b801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a33be800d72fab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:30.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2cf7fe640 1 -- 172.21.15.29:0/3582819563 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80b0070 con 0x7ff2d00b3140 2024-04-25T00:40:30.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2cf7fe640 1 -- 172.21.15.29:0/3582819563 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b80ab600 con 0x7ff2d00b3140 2024-04-25T00:40:30.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2cf7fe640 1 -- 172.21.15.29:0/3582819563 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80ab9d0 con 0x7ff2d00b3140 2024-04-25T00:40:30.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 -- 172.21.15.29:0/3582819563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2d00b3140 msgr2=0x7ff2d006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 --2- 172.21.15.29:0/3582819563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2d00b3140 0x7ff2d006e440 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80098a0 tx=0x7ff2b801c540 comp rx=0 tx=0).stop 2024-04-25T00:40:30.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 -- 172.21.15.29:0/3582819563 shutdown_connections 2024-04-25T00:40:30.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 --2- 172.21.15.29:0/3582819563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2d00b3c30 0x7ff2d00ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 --2- 172.21.15.29:0/3582819563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2d00b3140 0x7ff2d006e440 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 -- 172.21.15.29:0/3582819563 >> 172.21.15.29:0/3582819563 conn(0x7ff2d00b1870 msgr2=0x7ff2d00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:30.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 -- 172.21.15.29:0/3582819563 shutdown_connections 2024-04-25T00:40:30.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 -- 172.21.15.29:0/3582819563 wait complete. 2024-04-25T00:40:30.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 Processor -- start 2024-04-25T00:40:30.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 -- start start 2024-04-25T00:40:30.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2d00b3140 0x7ff2d012f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2d00b3c30 0x7ff2d0141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2d00b4b10 0x7ff2d0141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d0168d80 con 0x7ff2d00b3140 2024-04-25T00:40:30.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d0168c00 con 0x7ff2d00b3c30 2024-04-25T00:40:30.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d71fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff2d0168f00 con 0x7ff2d00b4b10 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d4f97640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2d00b3140 0x7ff2d012f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d4f97640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2d00b3140 0x7ff2d012f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59900/0 (socket says 172.21.15.29:59900) 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d4f97640 1 -- 172.21.15.29:0/1003270 learned_addr learned my addr 172.21.15.29:0/1003270 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.581+0000 7ff2d5798640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2d00b4b10 0x7ff2d0141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2cffff640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2d00b3c30 0x7ff2d0141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2d4f97640 1 -- 172.21.15.29:0/1003270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2d00b4b10 msgr2=0x7ff2d0141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2d4f97640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2d00b4b10 0x7ff2d0141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2d4f97640 1 -- 172.21.15.29:0/1003270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2d00b3c30 msgr2=0x7ff2d0141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2d4f97640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2d00b3c30 0x7ff2d0141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2d4f97640 1 -- 172.21.15.29:0/1003270 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d00710c0 con 0x7ff2d00b3140 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2d4f97640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2d00b3140 0x7ff2d012f760 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7ff2b801abf0 tx=0x7ff2b80192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2cd7fa640 1 -- 172.21.15.29:0/1003270 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80b0040 con 0x7ff2d00b3140 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2cd7fa640 1 -- 172.21.15.29:0/1003270 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b80b6660 con 0x7ff2d00b3140 2024-04-25T00:40:30.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2d71fb640 1 -- 172.21.15.29:0/1003270 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d00713a0 con 0x7ff2d00b3140 2024-04-25T00:40:30.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2cd7fa640 1 -- 172.21.15.29:0/1003270 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80b6a50 con 0x7ff2d00b3140 2024-04-25T00:40:30.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2d71fb640 1 -- 172.21.15.29:0/1003270 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d0071960 con 0x7ff2d00b3140 2024-04-25T00:40:30.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2d71fb640 1 -- 172.21.15.29:0/1003270 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d0069270 con 0x7ff2d00b3140 2024-04-25T00:40:30.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2cd7fa640 1 -- 172.21.15.29:0/1003270 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7ff2b80bd060 con 0x7ff2d00b3140 2024-04-25T00:40:30.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2cd7fa640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff2b00680a0 0x7ff2b006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2cffff640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff2b00680a0 0x7ff2b006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2cd7fa640 1 -- 172.21.15.29:0/1003270 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7ff2b811e4c0 con 0x7ff2d00b3140 2024-04-25T00:40:30.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.585+0000 7ff2cffff640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff2b00680a0 0x7ff2b006a560 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00b2b90 tx=0x7ff2c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:30.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.589+0000 7ff2cd7fa640 1 -- 172.21.15.29:0/1003270 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff2b8016130 con 0x7ff2d00b3140 2024-04-25T00:40:30.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.765+0000 7ff2d71fb640 1 -- 172.21.15.29:0/1003270 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2d0154c90 con 0x7ff2d00b3140 2024-04-25T00:40:30.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.765+0000 7ff2cd7fa640 1 -- 172.21.15.29:0/1003270 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+16644 (secure 0 0 0) 0x7ff2b80f08f0 con 0x7ff2d00b3140 2024-04-25T00:40:30.766 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:30.766 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":155,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:40:29.418735+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:40:27.796735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:40:28.430727+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:40:30.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.769+0000 7ff2d71fb640 1 -- 172.21.15.29:0/1003270 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff2b00680a0 msgr2=0x7ff2b006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.769+0000 7ff2d71fb640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff2b00680a0 0x7ff2b006a560 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00b2b90 tx=0x7ff2c0009040 comp rx=0 tx=0).stop 2024-04-25T00:40:30.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.769+0000 7ff2d71fb640 1 -- 172.21.15.29:0/1003270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2d00b3140 msgr2=0x7ff2d012f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.769+0000 7ff2d71fb640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2d00b3140 0x7ff2d012f760 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7ff2b801abf0 tx=0x7ff2b80192e0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.769+0000 7ff2d71fb640 1 -- 172.21.15.29:0/1003270 shutdown_connections 2024-04-25T00:40:30.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.769+0000 7ff2d71fb640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff2b00680a0 0x7ff2b006a560 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.769+0000 7ff2d71fb640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2d00b4b10 0x7ff2d0141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.769+0000 7ff2d71fb640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2d00b3c30 0x7ff2d0141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.769+0000 7ff2d71fb640 1 --2- 172.21.15.29:0/1003270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2d00b3140 0x7ff2d012f760 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.769+0000 7ff2d71fb640 1 -- 172.21.15.29:0/1003270 >> 172.21.15.29:0/1003270 conn(0x7ff2d00b1870 msgr2=0x7ff2d00cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:30.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.769+0000 7ff2d71fb640 1 -- 172.21.15.29:0/1003270 shutdown_connections 2024-04-25T00:40:30.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.769+0000 7ff2d71fb640 1 -- 172.21.15.29:0/1003270 wait complete. 2024-04-25T00:40:30.786 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:40:30.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f822a2ca640 1 Processor -- start 2024-04-25T00:40:30.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f822a2ca640 1 -- start start 2024-04-25T00:40:30.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f822a2ca640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f822415d980 0x7f822406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f822a2ca640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f82240b2b70 0x7f822406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f822a2ca640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82240b3130 con 0x7f8224150df0 2024-04-25T00:40:30.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f822a2ca640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f82240cf560 con 0x7f82240b2b70 2024-04-25T00:40:30.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f822a2ca640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8224151990 con 0x7f822415d980 2024-04-25T00:40:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f82237fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f82240b2b70 0x7f822406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f8228867640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8224150df0 0x7f8224151280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55742/0 (socket says 172.21.15.29:55742) 2024-04-25T00:40:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f8228867640 1 -- 172.21.15.29:0/2867241882 learned_addr learned my addr 172.21.15.29:0/2867241882 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f8222ffd640 1 -- 172.21.15.29:0/2867241882 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 554557657 0 0) 0x7f82240b3130 con 0x7f8224150df0 2024-04-25T00:40:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f82237fe640 1 -- 172.21.15.29:0/2867241882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f822415d980 msgr2=0x7f822406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:30.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f8223fff640 1 --2- 172.21.15.29:0/2867241882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f822415d980 0x7f822406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f82237fe640 1 --2- 172.21.15.29:0/2867241882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f822415d980 0x7f822406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f82237fe640 1 -- 172.21.15.29:0/2867241882 >> v1:172.21.15.29:6789/0 conn(0x7f8224150df0 legacy=0x7f8224151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.917+0000 7f82237fe640 1 -- 172.21.15.29:0/2867241882 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82240b3410 con 0x7f82240b2b70 2024-04-25T00:40:30.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f82237fe640 1 --2- 172.21.15.29:0/2867241882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82240b2b70 0x7f822406db10 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f821800a960 tx=0x7f821801d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea56a98e15d46f8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:30.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f8222ffd640 1 -- 172.21.15.29:0/2867241882 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180b3070 con 0x7f82240b2b70 2024-04-25T00:40:30.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f8222ffd640 1 -- 172.21.15.29:0/2867241882 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82180a6870 con 0x7f82240b2b70 2024-04-25T00:40:30.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f8222ffd640 1 -- 172.21.15.29:0/2867241882 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821800b040 con 0x7f82240b2b70 2024-04-25T00:40:30.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 -- 172.21.15.29:0/2867241882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82240b2b70 msgr2=0x7f822406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 --2- 172.21.15.29:0/2867241882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82240b2b70 0x7f822406db10 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f821800a960 tx=0x7f821801d840 comp rx=0 tx=0).stop 2024-04-25T00:40:30.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 -- 172.21.15.29:0/2867241882 shutdown_connections 2024-04-25T00:40:30.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 --2- 172.21.15.29:0/2867241882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82240b2b70 0x7f822406db10 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 --2- 172.21.15.29:0/2867241882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f822415d980 0x7f822406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 -- 172.21.15.29:0/2867241882 >> 172.21.15.29:0/2867241882 conn(0x7f82240b1870 msgr2=0x7f82240b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 -- 172.21.15.29:0/2867241882 shutdown_connections 2024-04-25T00:40:30.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 -- 172.21.15.29:0/2867241882 wait complete. 2024-04-25T00:40:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 Processor -- start 2024-04-25T00:40:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 -- start start 2024-04-25T00:40:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82240b2b70 0x7f82241494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8224150df0 0x7f8224155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f822415d980 0x7f8224156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f822415b4b0 con 0x7f8224150df0 2024-04-25T00:40:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f822415dd60 con 0x7f822415d980 2024-04-25T00:40:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f822a2ca640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f822415b630 con 0x7f82240b2b70 2024-04-25T00:40:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f82237fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8224150df0 0x7f8224155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f82237fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8224150df0 0x7f8224155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59916/0 (socket says 172.21.15.29:59916) 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.921+0000 7f82237fe640 1 -- 172.21.15.29:0/3447074778 learned_addr learned my addr 172.21.15.29:0/3447074778 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f8228867640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f822415d980 0x7f8224156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f8223fff640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82240b2b70 0x7f82241494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f82237fe640 1 -- 172.21.15.29:0/3447074778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82240b2b70 msgr2=0x7f82241494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f82237fe640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82240b2b70 0x7f82241494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f82237fe640 1 -- 172.21.15.29:0/3447074778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f822415d980 msgr2=0x7f8224156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f82237fe640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f822415d980 0x7f8224156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f82237fe640 1 -- 172.21.15.29:0/3447074778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8224157bf0 con 0x7f8224150df0 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f8223fff640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82240b2b70 0x7f82241494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f8228867640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f822415d980 0x7f8224156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f82237fe640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8224150df0 0x7f8224155060 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f821801bef0 tx=0x7f821801a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f8220ff9640 1 -- 172.21.15.29:0/3447074778 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180b3040 con 0x7f8224150df0 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f8220ff9640 1 -- 172.21.15.29:0/3447074778 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82180bb070 con 0x7f8224150df0 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f822a2ca640 1 -- 172.21.15.29:0/3447074778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82240cc9a0 con 0x7f8224150df0 2024-04-25T00:40:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f8220ff9640 1 -- 172.21.15.29:0/3447074778 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180aeab0 con 0x7f8224150df0 2024-04-25T00:40:30.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f822a2ca640 1 -- 172.21.15.29:0/3447074778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82241499e0 con 0x7f8224150df0 2024-04-25T00:40:30.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f822a2ca640 1 -- 172.21.15.29:0/3447074778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8224069270 con 0x7f8224150df0 2024-04-25T00:40:30.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f8220ff9640 1 -- 172.21.15.29:0/3447074778 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f82180aec50 con 0x7f8224150df0 2024-04-25T00:40:30.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f8220ff9640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f82100680a0 0x7f821006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:30.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f8220ff9640 1 -- 172.21.15.29:0/3447074778 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7f8218120ac0 con 0x7f8224150df0 2024-04-25T00:40:30.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f8223fff640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f82100680a0 0x7f821006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:30.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.925+0000 7f8223fff640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f82100680a0 0x7f821006a560 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8224150330 tx=0x7f821c007420 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:30.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:30.929+0000 7f8220ff9640 1 -- 172.21.15.29:0/3447074778 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8218127070 con 0x7f8224150df0 2024-04-25T00:40:31.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:31.125+0000 7f822a2ca640 1 -- 172.21.15.29:0/3447074778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f82240cf910 con 0x7f8224150df0 2024-04-25T00:40:32.054 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-04-25T00:40:32.053+0000 7f6af494d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:40:32.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.113+0000 7f8220ff9640 1 -- 172.21.15.29:0/3447074778 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v101) ==== 118+0+0 (secure 0 0 0) 0x7f8218017460 con 0x7f8224150df0 2024-04-25T00:40:32.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.117+0000 7f822a2ca640 1 -- 172.21.15.29:0/3447074778 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f82100680a0 msgr2=0x7f821006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:32.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.117+0000 7f822a2ca640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f82100680a0 0x7f821006a560 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8224150330 tx=0x7f821c007420 comp rx=0 tx=0).stop 2024-04-25T00:40:32.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.117+0000 7f822a2ca640 1 -- 172.21.15.29:0/3447074778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8224150df0 msgr2=0x7f8224155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:32.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.117+0000 7f822a2ca640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8224150df0 0x7f8224155060 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f821801bef0 tx=0x7f821801a5e0 comp rx=0 tx=0).stop 2024-04-25T00:40:32.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.117+0000 7f822a2ca640 1 -- 172.21.15.29:0/3447074778 shutdown_connections 2024-04-25T00:40:32.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.117+0000 7f822a2ca640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f82100680a0 0x7f821006a560 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:32.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.117+0000 7f822a2ca640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f822415d980 0x7f8224156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:32.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.117+0000 7f822a2ca640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8224150df0 0x7f8224155060 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:32.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.117+0000 7f822a2ca640 1 --2- 172.21.15.29:0/3447074778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82240b2b70 0x7f82241494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:32.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.117+0000 7f822a2ca640 1 -- 172.21.15.29:0/3447074778 >> 172.21.15.29:0/3447074778 conn(0x7f82240b1870 msgr2=0x7f82240afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:32.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.117+0000 7f822a2ca640 1 -- 172.21.15.29:0/3447074778 shutdown_connections 2024-04-25T00:40:32.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.117+0000 7f822a2ca640 1 -- 172.21.15.29:0/3447074778 wait complete. 2024-04-25T00:40:32.140 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:40:32.140 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:40:32.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.265+0000 7fd7a5523640 1 Processor -- start 2024-04-25T00:40:32.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.265+0000 7fd7a5523640 1 -- start start 2024-04-25T00:40:32.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd7a5523640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7a015e8d0 0x7fd7a00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:32.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd7a5523640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd7a00b4780 0x7fd7a015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:32.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd7a5523640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a015f1f0 0x7fd7a015b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:32.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd7a5523640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a00b42f0 con 0x7fd7a00b4780 2024-04-25T00:40:32.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd7a5523640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a00b3720 con 0x7fd7a015e8d0 2024-04-25T00:40:32.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd7a5523640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a00b39c0 con 0x7fd7a015f1f0 2024-04-25T00:40:32.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd79e7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd7a00b4780 0x7fd7a015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:32.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd79e7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd7a00b4780 0x7fd7a015ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59920/0 (socket says 172.21.15.29:59920) 2024-04-25T00:40:32.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd79e7fc640 1 -- 172.21.15.29:0/107433129 learned_addr learned my addr 172.21.15.29:0/107433129 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:32.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd79effd640 1 --2- 172.21.15.29:0/107433129 >> v2:172.21.15.157:3300/0 conn(0x7fd7a015e8d0 0x7fd7a00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:32.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd79f7fe640 1 --2- 172.21.15.29:0/107433129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a015f1f0 0x7fd7a015b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:32.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd79e7fc640 1 -- 172.21.15.29:0/107433129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a015f1f0 msgr2=0x7fd7a015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:32.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd79e7fc640 1 --2- 172.21.15.29:0/107433129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a015f1f0 0x7fd7a015b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:32.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd79e7fc640 1 -- 172.21.15.29:0/107433129 >> v2:172.21.15.157:3300/0 conn(0x7fd7a015e8d0 msgr2=0x7fd7a00cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:32.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd79e7fc640 1 --2- 172.21.15.29:0/107433129 >> v2:172.21.15.157:3300/0 conn(0x7fd7a015e8d0 0x7fd7a00cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:32.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd79e7fc640 1 -- 172.21.15.29:0/107433129 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a00b3ca0 con 0x7fd7a00b4780 2024-04-25T00:40:32.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.269+0000 7fd79effd640 1 --2- 172.21.15.29:0/107433129 >> v2:172.21.15.157:3300/0 conn(0x7fd7a015e8d0 0x7fd7a00cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:32.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79e7fc640 1 --2- 172.21.15.29:0/107433129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a00b4780 0x7fd7a015ecb0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00b5260 tx=0x7fd79401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadb57ad814b63d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:32.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79dffb640 1 -- 172.21.15.29:0/107433129 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7940aa5f0 con 0x7fd7a00b4780 2024-04-25T00:40:32.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79dffb640 1 -- 172.21.15.29:0/107433129 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7940af070 con 0x7fd7a00b4780 2024-04-25T00:40:32.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79dffb640 1 -- 172.21.15.29:0/107433129 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7940aa9b0 con 0x7fd7a00b4780 2024-04-25T00:40:32.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 -- 172.21.15.29:0/107433129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a00b4780 msgr2=0x7fd7a015ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:32.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 --2- 172.21.15.29:0/107433129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a00b4780 0x7fd7a015ecb0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00b5260 tx=0x7fd79401c4f0 comp rx=0 tx=0).stop 2024-04-25T00:40:32.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 -- 172.21.15.29:0/107433129 shutdown_connections 2024-04-25T00:40:32.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 --2- 172.21.15.29:0/107433129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a00b4780 0x7fd7a015ecb0 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:32.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 --2- 172.21.15.29:0/107433129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a015f1f0 0x7fd7a015b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:32.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 --2- 172.21.15.29:0/107433129 >> v2:172.21.15.157:3300/0 conn(0x7fd7a015e8d0 0x7fd7a00cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:32.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 -- 172.21.15.29:0/107433129 >> 172.21.15.29:0/107433129 conn(0x7fd7a00b1870 msgr2=0x7fd7a00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:32.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 -- 172.21.15.29:0/107433129 shutdown_connections 2024-04-25T00:40:32.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 -- 172.21.15.29:0/107433129 wait complete. 2024-04-25T00:40:32.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 Processor -- start 2024-04-25T00:40:32.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 -- start start 2024-04-25T00:40:32.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7a00b4780 0x7fd7a0159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:32.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a015e8d0 0x7fd7a01f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a015f1f0 0x7fd7a01f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a015f600 con 0x7fd7a015f1f0 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a015c7d0 con 0x7fd7a00b4780 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a01388e0 con 0x7fd7a015e8d0 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a015f1f0 0x7fd7a01f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a015f1f0 0x7fd7a01f84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59926/0 (socket says 172.21.15.29:59926) 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79f7fe640 1 -- 172.21.15.29:0/637502529 learned_addr learned my addr 172.21.15.29:0/637502529 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79e7fc640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a015e8d0 0x7fd7a01f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79effd640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7a00b4780 0x7fd7a0159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79f7fe640 1 -- 172.21.15.29:0/637502529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a015e8d0 msgr2=0x7fd7a01f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79f7fe640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a015e8d0 0x7fd7a01f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79f7fe640 1 -- 172.21.15.29:0/637502529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7a00b4780 msgr2=0x7fd7a0159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79f7fe640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7a00b4780 0x7fd7a0159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79f7fe640 1 -- 172.21.15.29:0/637502529 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a01f93d0 con 0x7fd7a015f1f0 2024-04-25T00:40:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79effd640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7a00b4780 0x7fd7a0159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79e7fc640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a015e8d0 0x7fd7a01f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd79f7fe640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a015f1f0 0x7fd7a01f84c0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fd798007e60 tx=0x7fd79800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7877fe640 1 -- 172.21.15.29:0/637502529 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798014420 con 0x7fd7a015f1f0 2024-04-25T00:40:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7877fe640 1 -- 172.21.15.29:0/637502529 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd798019070 con 0x7fd7a015f1f0 2024-04-25T00:40:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7a5523640 1 -- 172.21.15.29:0/637502529 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a01f8e10 con 0x7fd7a015f1f0 2024-04-25T00:40:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.273+0000 7fd7877fe640 1 -- 172.21.15.29:0/637502529 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798014820 con 0x7fd7a015f1f0 2024-04-25T00:40:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.277+0000 7fd7a5523640 1 -- 172.21.15.29:0/637502529 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a01f52c0 con 0x7fd7a015f1f0 2024-04-25T00:40:32.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.277+0000 7fd7a5523640 1 -- 172.21.15.29:0/637502529 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a01f9200 con 0x7fd7a015f1f0 2024-04-25T00:40:32.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.277+0000 7fd7877fe640 1 -- 172.21.15.29:0/637502529 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fd7980149c0 con 0x7fd7a015f1f0 2024-04-25T00:40:32.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.277+0000 7fd7877fe640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd78c0680a0 0x7fd78c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:32.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.277+0000 7fd7877fe640 1 -- 172.21.15.29:0/637502529 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fd798087630 con 0x7fd7a015f1f0 2024-04-25T00:40:32.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.277+0000 7fd79effd640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd78c0680a0 0x7fd78c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:32.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.277+0000 7fd79effd640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd78c0680a0 0x7fd78c06a560 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd790006fd0 tx=0x7fd790008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:32.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.281+0000 7fd7877fe640 1 -- 172.21.15.29:0/637502529 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd798057290 con 0x7fd7a015f1f0 2024-04-25T00:40:32.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:32.481+0000 7fd7a5523640 1 -- 172.21.15.29:0/637502529 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd7a0069270 con 0x7fd7a015f1f0 2024-04-25T00:40:33.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.117+0000 7fd7877fe640 1 -- 172.21.15.29:0/637502529 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v102) ==== 122+0+0 (secure 0 0 0) 0x7fd798056b90 con 0x7fd7a015f1f0 2024-04-25T00:40:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.117+0000 7fd7a5523640 1 -- 172.21.15.29:0/637502529 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd78c0680a0 msgr2=0x7fd78c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.117+0000 7fd7a5523640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd78c0680a0 0x7fd78c06a560 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd790006fd0 tx=0x7fd790008040 comp rx=0 tx=0).stop 2024-04-25T00:40:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.117+0000 7fd7a5523640 1 -- 172.21.15.29:0/637502529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a015f1f0 msgr2=0x7fd7a01f84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.117+0000 7fd7a5523640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a015f1f0 0x7fd7a01f84c0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fd798007e60 tx=0x7fd79800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.117+0000 7fd7a5523640 1 -- 172.21.15.29:0/637502529 shutdown_connections 2024-04-25T00:40:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.117+0000 7fd7a5523640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd78c0680a0 0x7fd78c06a560 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.117+0000 7fd7a5523640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a015f1f0 0x7fd7a01f84c0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.117+0000 7fd7a5523640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a015e8d0 0x7fd7a01f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.117+0000 7fd7a5523640 1 --2- 172.21.15.29:0/637502529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7a00b4780 0x7fd7a0159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.117+0000 7fd7a5523640 1 -- 172.21.15.29:0/637502529 >> 172.21.15.29:0/637502529 conn(0x7fd7a00b1870 msgr2=0x7fd7a00af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:33.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.117+0000 7fd7a5523640 1 -- 172.21.15.29:0/637502529 shutdown_connections 2024-04-25T00:40:33.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.117+0000 7fd7a5523640 1 -- 172.21.15.29:0/637502529 wait complete. 2024-04-25T00:40:33.141 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:40:33.141 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:40:33.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f56517a6640 1 Processor -- start 2024-04-25T00:40:33.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f56517a6640 1 -- start start 2024-04-25T00:40:33.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f56517a6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c0d09c0 0x7f564c15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f56517a6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f564c143880 0x7f564c143d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f56517a6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f564c0b3380 0x7f564c0b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f56517a6640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f564c0b3d30 con 0x7f564c143880 2024-04-25T00:40:33.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f56517a6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f564c15ee20 con 0x7f564c0b3380 2024-04-25T00:40:33.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f56517a6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f564c1443f0 con 0x7f564c0d09c0 2024-04-25T00:40:33.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f564a7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f564c143880 0x7f564c143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f564a7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f564c143880 0x7f564c143d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59942/0 (socket says 172.21.15.29:59942) 2024-04-25T00:40:33.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f564a7fc640 1 -- 172.21.15.29:0/2962555061 learned_addr learned my addr 172.21.15.29:0/2962555061 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:33.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f564affd640 1 --2- 172.21.15.29:0/2962555061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c0d09c0 0x7f564c15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f564b7fe640 1 --2- 172.21.15.29:0/2962555061 >> v2:172.21.15.157:3300/0 conn(0x7f564c0b3380 0x7f564c0b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f564a7fc640 1 -- 172.21.15.29:0/2962555061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c0d09c0 msgr2=0x7f564c15f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:33.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f564a7fc640 1 --2- 172.21.15.29:0/2962555061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c0d09c0 0x7f564c15f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f564a7fc640 1 -- 172.21.15.29:0/2962555061 >> v2:172.21.15.157:3300/0 conn(0x7f564c0b3380 msgr2=0x7f564c0b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:33.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f564a7fc640 1 --2- 172.21.15.29:0/2962555061 >> v2:172.21.15.157:3300/0 conn(0x7f564c0b3380 0x7f564c0b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f564a7fc640 1 -- 172.21.15.29:0/2962555061 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564c15e7d0 con 0x7f564c143880 2024-04-25T00:40:33.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.269+0000 7f564b7fe640 1 --2- 172.21.15.29:0/2962555061 >> v2:172.21.15.157:3300/0 conn(0x7f564c0b3380 0x7f564c0b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:33.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f564a7fc640 1 --2- 172.21.15.29:0/2962555061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f564c143880 0x7f564c143d00 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f564c0713b0 tx=0x7f563401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0578237a0f4535 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:33.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f5649ffb640 1 -- 172.21.15.29:0/2962555061 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340ab5f0 con 0x7f564c143880 2024-04-25T00:40:33.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f5649ffb640 1 -- 172.21.15.29:0/2962555061 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56340b0070 con 0x7f564c143880 2024-04-25T00:40:33.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f5649ffb640 1 -- 172.21.15.29:0/2962555061 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340ab9b0 con 0x7f564c143880 2024-04-25T00:40:33.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 -- 172.21.15.29:0/2962555061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f564c143880 msgr2=0x7f564c143d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 --2- 172.21.15.29:0/2962555061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f564c143880 0x7f564c143d00 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f564c0713b0 tx=0x7f563401c520 comp rx=0 tx=0).stop 2024-04-25T00:40:33.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 -- 172.21.15.29:0/2962555061 shutdown_connections 2024-04-25T00:40:33.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 --2- 172.21.15.29:0/2962555061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f564c143880 0x7f564c143d00 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 --2- 172.21.15.29:0/2962555061 >> v2:172.21.15.157:3300/0 conn(0x7f564c0b3380 0x7f564c0b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 --2- 172.21.15.29:0/2962555061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c0d09c0 0x7f564c15f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 -- 172.21.15.29:0/2962555061 >> 172.21.15.29:0/2962555061 conn(0x7f564c074c00 msgr2=0x7f564c06b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:33.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 -- 172.21.15.29:0/2962555061 shutdown_connections 2024-04-25T00:40:33.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 -- 172.21.15.29:0/2962555061 wait complete. 2024-04-25T00:40:33.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 Processor -- start 2024-04-25T00:40:33.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 -- start start 2024-04-25T00:40:33.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c0b3380 0x7f564c157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564c0d09c0 0x7f564c15fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f564c143880 0x7f564c15ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f564c162640 con 0x7f564c143880 2024-04-25T00:40:33.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1624c0 con 0x7f564c0d09c0 2024-04-25T00:40:33.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f56517a6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f564c1627c0 con 0x7f564c0b3380 2024-04-25T00:40:33.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f564b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f564c143880 0x7f564c15ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f564b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f564c143880 0x7f564c15ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59944/0 (socket says 172.21.15.29:59944) 2024-04-25T00:40:33.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f564b7fe640 1 -- 172.21.15.29:0/1785370868 learned_addr learned my addr 172.21.15.29:0/1785370868 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:33.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f564affd640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c0b3380 0x7f564c157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f564b7fe640 1 -- 172.21.15.29:0/1785370868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c0b3380 msgr2=0x7f564c157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.273+0000 7f564b7fe640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c0b3380 0x7f564c157450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f564b7fe640 1 -- 172.21.15.29:0/1785370868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564c0d09c0 msgr2=0x7f564c15fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f564a7fc640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564c0d09c0 0x7f564c15fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f564b7fe640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564c0d09c0 0x7f564c15fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f564b7fe640 1 -- 172.21.15.29:0/1785370868 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564c157990 con 0x7f564c143880 2024-04-25T00:40:33.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f564b7fe640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f564c143880 0x7f564c15ff50 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f564400c820 tx=0x7f564400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:33.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f56437fe640 1 -- 172.21.15.29:0/1785370868 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644015070 con 0x7f564c143880 2024-04-25T00:40:33.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f56437fe640 1 -- 172.21.15.29:0/1785370868 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5644007a30 con 0x7f564c143880 2024-04-25T00:40:33.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f56437fe640 1 -- 172.21.15.29:0/1785370868 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644007e20 con 0x7f564c143880 2024-04-25T00:40:33.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f56517a6640 1 -- 172.21.15.29:0/1785370868 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564c15ccf0 con 0x7f564c143880 2024-04-25T00:40:33.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f56517a6640 1 -- 172.21.15.29:0/1785370868 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f564c15d2b0 con 0x7f564c143880 2024-04-25T00:40:33.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f56517a6640 1 -- 172.21.15.29:0/1785370868 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f564c15dba0 con 0x7f564c143880 2024-04-25T00:40:33.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f56437fe640 1 -- 172.21.15.29:0/1785370868 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f5644018070 con 0x7f564c143880 2024-04-25T00:40:33.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f56437fe640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f562c068120 0x7f562c06a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f56437fe640 1 -- 172.21.15.29:0/1785370868 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f5644086df0 con 0x7f564c143880 2024-04-25T00:40:33.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f564affd640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f562c068120 0x7f562c06a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.277+0000 7f564affd640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f562c068120 0x7f562c06a5e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f563c009f30 tx=0x7f563c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:33.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.281+0000 7f56437fe640 1 -- 172.21.15.29:0/1785370868 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5644056a10 con 0x7f564c143880 2024-04-25T00:40:33.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56517a6640 1 -- 172.21.15.29:0/1785370868 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f564c069270 con 0x7f564c143880 2024-04-25T00:40:33.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56437fe640 1 -- 172.21.15.29:0/1785370868 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5196 (secure 0 0 0) 0x7f5644056310 con 0x7f564c143880 2024-04-25T00:40:33.477 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:33.477 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2677876688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2677876688},{"type":"v1","addr":"172.21.15.157:6833","nonce":2677876688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3340901266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3340901266},{"type":"v1","addr":"172.21.15.157:6835","nonce":3340901266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":10092,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/567284262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":567284262},{"type":"v1","addr":"172.21.15.29:6833","nonce":567284262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:40:29.419450+0000","modified":"2024-04-25T00:40:32.484311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10095":{"gid":10095,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/4183931647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":4183931647},{"type":"v1","addr":"172.21.15.29:6835","nonce":4183931647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10095,"qdb_cluster":[10095]},"id":12}]} 2024-04-25T00:40:33.478 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 102 2024-04-25T00:40:33.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56517a6640 1 -- 172.21.15.29:0/1785370868 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f562c068120 msgr2=0x7f562c06a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56517a6640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f562c068120 0x7f562c06a5e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f563c009f30 tx=0x7f563c00b040 comp rx=0 tx=0).stop 2024-04-25T00:40:33.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56517a6640 1 -- 172.21.15.29:0/1785370868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f564c143880 msgr2=0x7f564c15ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56517a6640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f564c143880 0x7f564c15ff50 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f564400c820 tx=0x7f564400cce0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56517a6640 1 -- 172.21.15.29:0/1785370868 shutdown_connections 2024-04-25T00:40:33.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56517a6640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f562c068120 0x7f562c06a5e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56517a6640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f564c143880 0x7f564c15ff50 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56517a6640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564c0d09c0 0x7f564c15fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56517a6640 1 --2- 172.21.15.29:0/1785370868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c0b3380 0x7f564c157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56517a6640 1 -- 172.21.15.29:0/1785370868 >> 172.21.15.29:0/1785370868 conn(0x7f564c074c00 msgr2=0x7f564c142d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:33.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56517a6640 1 -- 172.21.15.29:0/1785370868 shutdown_connections 2024-04-25T00:40:33.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.477+0000 7f56517a6640 1 -- 172.21.15.29:0/1785370868 wait complete. 2024-04-25T00:40:33.495 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:40:33.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfe0216640 1 Processor -- start 2024-04-25T00:40:33.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfe0216640 1 -- start start 2024-04-25T00:40:33.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfe0216640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd80b3140 0x7fbfd806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfe0216640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbfd80b3c30 0x7fbfd80ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfe0216640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd80b41e0 con 0x7fbfd80b4b10 2024-04-25T00:40:33.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfe0216640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbfd806d960 con 0x7fbfd80b3c30 2024-04-25T00:40:33.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfe0216640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd806dc40 con 0x7fbfd80b3140 2024-04-25T00:40:33.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfddfb2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd80b3140 0x7fbfd806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfde7b3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbfd80b3c30 0x7fbfd80ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfdd7b1640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbfd80b4b10 0x7fbfd80b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:59402/0 (socket says 172.21.15.29:59402) 2024-04-25T00:40:33.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfdd7b1640 1 -- 172.21.15.29:0/938089947 learned_addr learned my addr 172.21.15.29:0/938089947 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:33.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfddfb2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd80b3140 0x7fbfd806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:58262/0 (socket says 172.21.15.29:58262) 2024-04-25T00:40:33.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfde7b3640 1 -- 172.21.15.29:0/938089947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd80b3140 msgr2=0x7fbfd806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:33.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfde7b3640 1 --2- 172.21.15.29:0/938089947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd80b3140 0x7fbfd806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfde7b3640 1 -- 172.21.15.29:0/938089947 >> v1:172.21.15.157:6789/0 conn(0x7fbfd80b4b10 legacy=0x7fbfd80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfde7b3640 1 -- 172.21.15.29:0/938089947 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd806df20 con 0x7fbfd80b3c30 2024-04-25T00:40:33.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfdcfb0640 1 -- 172.21.15.29:0/938089947 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2423396417 0 0) 0x7fbfd80b41e0 con 0x7fbfd80b4b10 2024-04-25T00:40:33.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.617+0000 7fbfddfb2640 1 --2- 172.21.15.29:0/938089947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd80b3140 0x7fbfd806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:40:33.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfde7b3640 1 --2- 172.21.15.29:0/938089947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd80b3c30 0x7fbfd80ce240 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00097c0 tx=0x7fbfd001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc01615fcd3db7e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:33.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfdcfb0640 1 -- 172.21.15.29:0/938089947 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00b0070 con 0x7fbfd80b3c30 2024-04-25T00:40:33.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfdcfb0640 1 -- 172.21.15.29:0/938089947 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd00ab600 con 0x7fbfd80b3c30 2024-04-25T00:40:33.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfdcfb0640 1 -- 172.21.15.29:0/938089947 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00ab9d0 con 0x7fbfd80b3c30 2024-04-25T00:40:33.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 -- 172.21.15.29:0/938089947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd80b3c30 msgr2=0x7fbfd80ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 --2- 172.21.15.29:0/938089947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd80b3c30 0x7fbfd80ce240 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00097c0 tx=0x7fbfd001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 -- 172.21.15.29:0/938089947 shutdown_connections 2024-04-25T00:40:33.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 --2- 172.21.15.29:0/938089947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd80b3c30 0x7fbfd80ce240 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 --2- 172.21.15.29:0/938089947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd80b3140 0x7fbfd806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 -- 172.21.15.29:0/938089947 >> 172.21.15.29:0/938089947 conn(0x7fbfd80b1870 msgr2=0x7fbfd80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:33.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 -- 172.21.15.29:0/938089947 shutdown_connections 2024-04-25T00:40:33.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 -- 172.21.15.29:0/938089947 wait complete. 2024-04-25T00:40:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 Processor -- start 2024-04-25T00:40:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 -- start start 2024-04-25T00:40:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd80b3140 0x7fbfd815b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd80b3c30 0x7fbfd80c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfd80b4b10 0x7fbfd80b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd8168d80 con 0x7fbfd80b4b10 2024-04-25T00:40:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd8168c00 con 0x7fbfd80b3c30 2024-04-25T00:40:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.621+0000 7fbfe0216640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd8168f00 con 0x7fbfd80b3140 2024-04-25T00:40:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfde7b3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfd80b4b10 0x7fbfd80b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfde7b3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfd80b4b10 0x7fbfd80b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59952/0 (socket says 172.21.15.29:59952) 2024-04-25T00:40:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfde7b3640 1 -- 172.21.15.29:0/776571736 learned_addr learned my addr 172.21.15.29:0/776571736 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfdd7b1640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd80b3c30 0x7fbfd80c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfddfb2640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd80b3140 0x7fbfd815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfde7b3640 1 -- 172.21.15.29:0/776571736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd80b3140 msgr2=0x7fbfd815b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfde7b3640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd80b3140 0x7fbfd815b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfde7b3640 1 -- 172.21.15.29:0/776571736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd80b3c30 msgr2=0x7fbfd80c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfde7b3640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd80b3c30 0x7fbfd80c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfde7b3640 1 -- 172.21.15.29:0/776571736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd80c5d70 con 0x7fbfd80b4b10 2024-04-25T00:40:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfde7b3640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfd80b4b10 0x7fbfd80b0360 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00ae040 tx=0x7fbfd001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfce7fc640 1 -- 172.21.15.29:0/776571736 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00bd070 con 0x7fbfd80b4b10 2024-04-25T00:40:33.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfce7fc640 1 -- 172.21.15.29:0/776571736 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd0004030 con 0x7fbfd80b4b10 2024-04-25T00:40:33.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfe0216640 1 -- 172.21.15.29:0/776571736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd80c6050 con 0x7fbfd80b4b10 2024-04-25T00:40:33.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfce7fc640 1 -- 172.21.15.29:0/776571736 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00b0040 con 0x7fbfd80b4b10 2024-04-25T00:40:33.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfe0216640 1 -- 172.21.15.29:0/776571736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd80c6610 con 0x7fbfd80b4b10 2024-04-25T00:40:33.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfe0216640 1 -- 172.21.15.29:0/776571736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd8069270 con 0x7fbfd80b4b10 2024-04-25T00:40:33.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfce7fc640 1 -- 172.21.15.29:0/776571736 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fbfd00c0070 con 0x7fbfd80b4b10 2024-04-25T00:40:33.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfce7fc640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbfbc0680a0 0x7fbfbc06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfddfb2640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbfbc0680a0 0x7fbfbc06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfce7fc640 1 -- 172.21.15.29:0/776571736 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fbfd011dff0 con 0x7fbfd80b4b10 2024-04-25T00:40:33.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.625+0000 7fbfddfb2640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbfbc0680a0 0x7fbfbc06a560 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80c7100 tx=0x7fbfd400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:33.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.629+0000 7fbfce7fc640 1 -- 172.21.15.29:0/776571736 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbfd0016090 con 0x7fbfd80b4b10 2024-04-25T00:40:33.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.793+0000 7fbfe0216640 1 -- 172.21.15.29:0/776571736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbfd80cf6d0 con 0x7fbfd80b4b10 2024-04-25T00:40:33.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.793+0000 7fbfce7fc640 1 -- 172.21.15.29:0/776571736 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16631 (secure 0 0 0) 0x7fbfd00f0420 con 0x7fbfd80b4b10 2024-04-25T00:40:33.795 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:33.795 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":156,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:40:31.094463+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:40:27.796735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:40:28.430727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:40:33.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.797+0000 7fbfe0216640 1 -- 172.21.15.29:0/776571736 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbfbc0680a0 msgr2=0x7fbfbc06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.797+0000 7fbfe0216640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbfbc0680a0 0x7fbfbc06a560 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80c7100 tx=0x7fbfd400b040 comp rx=0 tx=0).stop 2024-04-25T00:40:33.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.797+0000 7fbfe0216640 1 -- 172.21.15.29:0/776571736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfd80b4b10 msgr2=0x7fbfd80b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.797+0000 7fbfe0216640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfd80b4b10 0x7fbfd80b0360 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00ae040 tx=0x7fbfd001ace0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.797+0000 7fbfe0216640 1 -- 172.21.15.29:0/776571736 shutdown_connections 2024-04-25T00:40:33.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.797+0000 7fbfe0216640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbfbc0680a0 0x7fbfbc06a560 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.797+0000 7fbfe0216640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfd80b4b10 0x7fbfd80b0360 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.797+0000 7fbfe0216640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd80b3c30 0x7fbfd80c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.797+0000 7fbfe0216640 1 --2- 172.21.15.29:0/776571736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfd80b3140 0x7fbfd815b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.797+0000 7fbfe0216640 1 -- 172.21.15.29:0/776571736 >> 172.21.15.29:0/776571736 conn(0x7fbfd80b1870 msgr2=0x7fbfd813f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:33.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.797+0000 7fbfe0216640 1 -- 172.21.15.29:0/776571736 shutdown_connections 2024-04-25T00:40:33.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.797+0000 7fbfe0216640 1 -- 172.21.15.29:0/776571736 wait complete. 2024-04-25T00:40:33.814 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:40:33.815 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:40:33.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f19642ea640 1 Processor -- start 2024-04-25T00:40:33.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f19642ea640 1 -- start start 2024-04-25T00:40:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f19642ea640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0b4780 0x7f195c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f19642ea640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f195c0cc240 0x7f195c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f19642ea640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f195c0b3720 con 0x7f195c15e8d0 2024-04-25T00:40:33.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f19642ea640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f195c0b42f0 con 0x7f195c0cc240 2024-04-25T00:40:33.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f19642ea640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f195c0b39c0 con 0x7f195c0b4780 2024-04-25T00:40:33.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f1961885640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0b4780 0x7f195c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f1962887640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f195c0cc240 0x7f195c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f1961885640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0b4780 0x7f195c15f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:58282/0 (socket says 172.21.15.29:58282) 2024-04-25T00:40:33.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f1961885640 1 -- 172.21.15.29:0/662010983 learned_addr learned my addr 172.21.15.29:0/662010983 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:33.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f1962086640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f195c15e8d0 0x7f195c15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:59408/0 (socket says 172.21.15.29:59408) 2024-04-25T00:40:33.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f1961885640 1 -- 172.21.15.29:0/662010983 >> v2:172.21.15.157:3300/0 conn(0x7f195c0cc240 msgr2=0x7f195c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:33.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f1961885640 1 --2- 172.21.15.29:0/662010983 >> v2:172.21.15.157:3300/0 conn(0x7f195c0cc240 0x7f195c15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f1961885640 1 -- 172.21.15.29:0/662010983 >> v1:172.21.15.157:6789/0 conn(0x7f195c15e8d0 legacy=0x7f195c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.933+0000 7f1961885640 1 -- 172.21.15.29:0/662010983 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195c0b3ca0 con 0x7f195c0b4780 2024-04-25T00:40:33.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f1961885640 1 --2- 172.21.15.29:0/662010983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0b4780 0x7f195c15f3c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f194c00bb10 tx=0x7f194c01ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83f93cc4b0267137 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:33.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f1961084640 1 -- 172.21.15.29:0/662010983 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c0b4070 con 0x7f195c0b4780 2024-04-25T00:40:33.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f1961084640 1 -- 172.21.15.29:0/662010983 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194c0a7bd0 con 0x7f195c0b4780 2024-04-25T00:40:33.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f1961084640 1 -- 172.21.15.29:0/662010983 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c0af440 con 0x7f195c0b4780 2024-04-25T00:40:33.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 -- 172.21.15.29:0/662010983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0b4780 msgr2=0x7f195c15f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 --2- 172.21.15.29:0/662010983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0b4780 0x7f195c15f3c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f194c00bb10 tx=0x7f194c01ec30 comp rx=0 tx=0).stop 2024-04-25T00:40:33.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 -- 172.21.15.29:0/662010983 shutdown_connections 2024-04-25T00:40:33.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 --2- 172.21.15.29:0/662010983 >> v2:172.21.15.157:3300/0 conn(0x7f195c0cc240 0x7f195c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 --2- 172.21.15.29:0/662010983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0b4780 0x7f195c15f3c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 -- 172.21.15.29:0/662010983 >> 172.21.15.29:0/662010983 conn(0x7f195c0b1870 msgr2=0x7f195c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:33.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 -- 172.21.15.29:0/662010983 shutdown_connections 2024-04-25T00:40:33.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 -- 172.21.15.29:0/662010983 wait complete. 2024-04-25T00:40:33.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 Processor -- start 2024-04-25T00:40:33.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 -- start start 2024-04-25T00:40:33.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f195c0b4780 0x7f195c158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0cc240 0x7f195c0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f195c15e8d0 0x7f195c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f195c15d6a0 con 0x7f195c15e8d0 2024-04-25T00:40:33.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f195c15d520 con 0x7f195c0b4780 2024-04-25T00:40:33.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f19642ea640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f195c06d5d0 con 0x7f195c0cc240 2024-04-25T00:40:33.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.937+0000 7f1962887640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f195c15e8d0 0x7f195c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1962887640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f195c15e8d0 0x7f195c15a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59956/0 (socket says 172.21.15.29:59956) 2024-04-25T00:40:33.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1962887640 1 -- 172.21.15.29:0/3329270630 learned_addr learned my addr 172.21.15.29:0/3329270630 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:33.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1961885640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0cc240 0x7f195c0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1962086640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f195c0b4780 0x7f195c158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1962887640 1 -- 172.21.15.29:0/3329270630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0cc240 msgr2=0x7f195c0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1962887640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0cc240 0x7f195c0aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1962887640 1 -- 172.21.15.29:0/3329270630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f195c0b4780 msgr2=0x7f195c158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:33.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1962887640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f195c0b4780 0x7f195c158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:33.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1962887640 1 -- 172.21.15.29:0/3329270630 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195c0a9900 con 0x7f195c15e8d0 2024-04-25T00:40:33.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1962887640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f195c15e8d0 0x7f195c15a0f0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f195400efc0 tx=0x7f195400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:33.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1952ffd640 1 -- 172.21.15.29:0/3329270630 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954018020 con 0x7f195c15e8d0 2024-04-25T00:40:33.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1952ffd640 1 -- 172.21.15.29:0/3329270630 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1954014480 con 0x7f195c15e8d0 2024-04-25T00:40:33.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f19642ea640 1 -- 172.21.15.29:0/3329270630 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195c159610 con 0x7f195c15e8d0 2024-04-25T00:40:33.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1952ffd640 1 -- 172.21.15.29:0/3329270630 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954014870 con 0x7f195c15e8d0 2024-04-25T00:40:33.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f19642ea640 1 -- 172.21.15.29:0/3329270630 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f195c159bd0 con 0x7f195c15e8d0 2024-04-25T00:40:33.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f19642ea640 1 -- 172.21.15.29:0/3329270630 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f195c069270 con 0x7f195c15e8d0 2024-04-25T00:40:33.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1952ffd640 1 -- 172.21.15.29:0/3329270630 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f195401b020 con 0x7f195c15e8d0 2024-04-25T00:40:33.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1952ffd640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19400680a0 0x7f194006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:33.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1952ffd640 1 -- 172.21.15.29:0/3329270630 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f1954087da0 con 0x7f195c15e8d0 2024-04-25T00:40:33.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1962086640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19400680a0 0x7f194006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:33.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.941+0000 7f1962086640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19400680a0 0x7f194006a560 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1958000f70 tx=0x7f1958007cf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:33.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:33.945+0000 7f1952ffd640 1 -- 172.21.15.29:0/3329270630 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1954022070 con 0x7f195c15e8d0 2024-04-25T00:40:34.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.097+0000 7f19642ea640 1 -- 172.21.15.29:0/3329270630 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f195c06e7b0 con 0x7f19400680a0 2024-04-25T00:40:34.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.101+0000 7f1952ffd640 1 -- 172.21.15.29:0/3329270630 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247088 (secure 0 0 0) 0x7f195c06e7b0 con 0x7f19400680a0 2024-04-25T00:40:34.108 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:34.109 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:40:34.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.105+0000 7f19642ea640 1 -- 172.21.15.29:0/3329270630 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19400680a0 msgr2=0x7f194006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.105+0000 7f19642ea640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19400680a0 0x7f194006a560 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1958000f70 tx=0x7f1958007cf0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.105+0000 7f19642ea640 1 -- 172.21.15.29:0/3329270630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f195c15e8d0 msgr2=0x7f195c15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.105+0000 7f19642ea640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f195c15e8d0 0x7f195c15a0f0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f195400efc0 tx=0x7f195400c490 comp rx=0 tx=0).stop 2024-04-25T00:40:34.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.105+0000 7f19642ea640 1 -- 172.21.15.29:0/3329270630 shutdown_connections 2024-04-25T00:40:34.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.105+0000 7f19642ea640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f19400680a0 0x7f194006a560 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.105+0000 7f19642ea640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f195c15e8d0 0x7f195c15a0f0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.105+0000 7f19642ea640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0cc240 0x7f195c0aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.105+0000 7f19642ea640 1 --2- 172.21.15.29:0/3329270630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f195c0b4780 0x7f195c158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.105+0000 7f19642ea640 1 -- 172.21.15.29:0/3329270630 >> 172.21.15.29:0/3329270630 conn(0x7f195c0b1870 msgr2=0x7f195c0af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.105+0000 7f19642ea640 1 -- 172.21.15.29:0/3329270630 shutdown_connections 2024-04-25T00:40:34.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.105+0000 7f19642ea640 1 -- 172.21.15.29:0/3329270630 wait complete. 2024-04-25T00:40:34.122 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-04-25T00:40:33.405476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307328,"kb_used_data":16244,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667392,"statfs":{"total":773094113280,"available":772779409408,"internally_reserved":0,"allocated":16633856,"data_stored":7986187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.350831"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068067+0000","last_change":"2024-04-25T00:40:29.068067+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068067+0000","last_undegraded":"2024-04-25T00:40:29.068067+0000","last_fullsized":"2024-04-25T00:40:29.068067+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.077656+0000","last_change":"2024-04-25T00:40:29.077656+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.077656+0000","last_undegraded":"2024-04-25T00:40:29.077656+0000","last_fullsized":"2024-04-25T00:40:29.077656+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.085806+0000","last_change":"2024-04-25T00:40:29.085806+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.085806+0000","last_undegraded":"2024-04-25T00:40:29.085806+0000","last_fullsized":"2024-04-25T00:40:29.085806+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076453+0000","last_change":"2024-04-25T00:40:29.076453+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076453+0000","last_undegraded":"2024-04-25T00:40:29.076453+0000","last_fullsized":"2024-04-25T00:40:29.076453+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.083171+0000","last_change":"2024-04-25T00:40:29.083171+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.083171+0000","last_undegraded":"2024-04-25T00:40:29.083171+0000","last_fullsized":"2024-04-25T00:40:29.083171+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076571+0000","last_change":"2024-04-25T00:40:29.076571+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076571+0000","last_undegraded":"2024-04-25T00:40:29.076571+0000","last_fullsized":"2024-04-25T00:40:29.076571+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067060+0000","last_change":"2024-04-25T00:40:29.067060+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067060+0000","last_undegraded":"2024-04-25T00:40:29.067060+0000","last_fullsized":"2024-04-25T00:40:29.067060+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.078988+0000","last_change":"2024-04-25T00:40:29.078988+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.078988+0000","last_undegraded":"2024-04-25T00:40:29.078988+0000","last_fullsized":"2024-04-25T00:40:29.078988+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.066446+0000","last_change":"2024-04-25T00:40:29.066446+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.066446+0000","last_undegraded":"2024-04-25T00:40:29.066446+0000","last_fullsized":"2024-04-25T00:40:29.066446+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.071624+0000","last_change":"2024-04-25T00:40:29.071624+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.071624+0000","last_undegraded":"2024-04-25T00:40:29.071624+0000","last_fullsized":"2024-04-25T00:40:29.071624+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065659+0000","last_change":"2024-04-25T00:40:29.065659+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065659+0000","last_undegraded":"2024-04-25T00:40:29.065659+0000","last_fullsized":"2024-04-25T00:40:29.065659+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.075405+0000","last_change":"2024-04-25T00:40:29.075405+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.075405+0000","last_undegraded":"2024-04-25T00:40:29.075405+0000","last_fullsized":"2024-04-25T00:40:29.075405+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.079997+0000","last_change":"2024-04-25T00:40:29.079997+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.079997+0000","last_undegraded":"2024-04-25T00:40:29.079997+0000","last_fullsized":"2024-04-25T00:40:29.079997+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067927+0000","last_change":"2024-04-25T00:40:29.067927+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067927+0000","last_undegraded":"2024-04-25T00:40:29.067927+0000","last_fullsized":"2024-04-25T00:40:29.067927+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.080999+0000","last_change":"2024-04-25T00:40:29.080999+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.080999+0000","last_undegraded":"2024-04-25T00:40:29.080999+0000","last_fullsized":"2024-04-25T00:40:29.080999+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.078469+0000","last_change":"2024-04-25T00:40:29.078469+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.078469+0000","last_undegraded":"2024-04-25T00:40:29.078469+0000","last_fullsized":"2024-04-25T00:40:29.078469+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082514+0000","last_change":"2024-04-25T00:40:29.082514+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082514+0000","last_undegraded":"2024-04-25T00:40:29.082514+0000","last_fullsized":"2024-04-25T00:40:29.082514+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068120+0000","last_change":"2024-04-25T00:40:29.068120+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068120+0000","last_undegraded":"2024-04-25T00:40:29.068120+0000","last_fullsized":"2024-04-25T00:40:29.068120+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.069605+0000","last_change":"2024-04-25T00:40:29.069605+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.069605+0000","last_undegraded":"2024-04-25T00:40:29.069605+0000","last_fullsized":"2024-04-25T00:40:29.069605+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072609+0000","last_change":"2024-04-25T00:40:29.072609+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072609+0000","last_undegraded":"2024-04-25T00:40:29.072609+0000","last_fullsized":"2024-04-25T00:40:29.072609+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082713+0000","last_change":"2024-04-25T00:40:29.082713+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082713+0000","last_undegraded":"2024-04-25T00:40:29.082713+0000","last_fullsized":"2024-04-25T00:40:29.082713+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.069538+0000","last_change":"2024-04-25T00:40:29.069538+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.069538+0000","last_undegraded":"2024-04-25T00:40:29.069538+0000","last_fullsized":"2024-04-25T00:40:29.069538+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.085695+0000","last_change":"2024-04-25T00:40:29.085695+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.085695+0000","last_undegraded":"2024-04-25T00:40:29.085695+0000","last_fullsized":"2024-04-25T00:40:29.085695+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.077654+0000","last_change":"2024-04-25T00:40:29.077654+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.077654+0000","last_undegraded":"2024-04-25T00:40:29.077654+0000","last_fullsized":"2024-04-25T00:40:29.077654+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.073015+0000","last_change":"2024-04-25T00:40:29.073015+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.073015+0000","last_undegraded":"2024-04-25T00:40:29.073015+0000","last_fullsized":"2024-04-25T00:40:29.073015+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065060+0000","last_change":"2024-04-25T00:40:29.065060+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065060+0000","last_undegraded":"2024-04-25T00:40:29.065060+0000","last_fullsized":"2024-04-25T00:40:29.065060+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082493+0000","last_change":"2024-04-25T00:40:29.082493+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082493+0000","last_undegraded":"2024-04-25T00:40:29.082493+0000","last_fullsized":"2024-04-25T00:40:29.082493+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072143+0000","last_change":"2024-04-25T00:40:29.072143+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072143+0000","last_undegraded":"2024-04-25T00:40:29.072143+0000","last_fullsized":"2024-04-25T00:40:29.072143+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072837+0000","last_change":"2024-04-25T00:40:29.072837+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072837+0000","last_undegraded":"2024-04-25T00:40:29.072837+0000","last_fullsized":"2024-04-25T00:40:29.072837+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065859+0000","last_change":"2024-04-25T00:40:29.065859+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065859+0000","last_undegraded":"2024-04-25T00:40:29.065859+0000","last_fullsized":"2024-04-25T00:40:29.065859+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082748+0000","last_change":"2024-04-25T00:40:29.082748+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082748+0000","last_undegraded":"2024-04-25T00:40:29.082748+0000","last_fullsized":"2024-04-25T00:40:29.082748+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.071805+0000","last_change":"2024-04-25T00:40:29.071805+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.071805+0000","last_undegraded":"2024-04-25T00:40:29.071805+0000","last_fullsized":"2024-04-25T00:40:29.071805+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.074871+0000","last_change":"2024-04-25T00:40:29.074871+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.074871+0000","last_undegraded":"2024-04-25T00:40:29.074871+0000","last_fullsized":"2024-04-25T00:40:29.074871+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.100723+0000","last_change":"2024-04-25T00:40:29.094752+0000","last_active":"2024-04-25T00:40:29.100723+0000","last_peered":"2024-04-25T00:40:29.100723+0000","last_clean":"2024-04-25T00:40:29.100723+0000","last_became_active":"2024-04-25T00:40:29.094472+0000","last_became_peered":"2024-04-25T00:40:29.094472+0000","last_unstale":"2024-04-25T00:40:29.100723+0000","last_undegraded":"2024-04-25T00:40:29.100723+0000","last_fullsized":"2024-04-25T00:40:29.100723+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:59.895661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.070618+0000","last_change":"2024-04-25T00:40:29.070618+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.070618+0000","last_undegraded":"2024-04-25T00:40:29.070618+0000","last_fullsized":"2024-04-25T00:40:29.070618+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.088790+0000","last_change":"2024-04-25T00:40:29.085706+0000","last_active":"2024-04-25T00:40:29.088790+0000","last_peered":"2024-04-25T00:40:29.088790+0000","last_clean":"2024-04-25T00:40:29.088790+0000","last_became_active":"2024-04-25T00:40:29.084970+0000","last_became_peered":"2024-04-25T00:40:29.084970+0000","last_unstale":"2024-04-25T00:40:29.088790+0000","last_undegraded":"2024-04-25T00:40:29.088790+0000","last_fullsized":"2024-04-25T00:40:29.088790+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:57.516592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076415+0000","last_change":"2024-04-25T00:40:29.076415+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076415+0000","last_undegraded":"2024-04-25T00:40:29.076415+0000","last_fullsized":"2024-04-25T00:40:29.076415+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.099678+0000","last_change":"2024-04-25T00:40:29.098108+0000","last_active":"2024-04-25T00:40:29.099678+0000","last_peered":"2024-04-25T00:40:29.099678+0000","last_clean":"2024-04-25T00:40:29.099678+0000","last_became_active":"2024-04-25T00:40:29.097478+0000","last_became_peered":"2024-04-25T00:40:29.097478+0000","last_unstale":"2024-04-25T00:40:29.099678+0000","last_undegraded":"2024-04-25T00:40:29.099678+0000","last_fullsized":"2024-04-25T00:40:29.099678+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:14.983065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067036+0000","last_change":"2024-04-25T00:40:29.067036+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067036+0000","last_undegraded":"2024-04-25T00:40:29.067036+0000","last_fullsized":"2024-04-25T00:40:29.067036+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.101617+0000","last_change":"2024-04-25T00:40:29.096877+0000","last_active":"2024-04-25T00:40:29.101617+0000","last_peered":"2024-04-25T00:40:29.101617+0000","last_clean":"2024-04-25T00:40:29.101617+0000","last_became_active":"2024-04-25T00:40:29.096218+0000","last_became_peered":"2024-04-25T00:40:29.096218+0000","last_unstale":"2024-04-25T00:40:29.101617+0000","last_undegraded":"2024-04-25T00:40:29.101617+0000","last_fullsized":"2024-04-25T00:40:29.101617+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:25.492182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.069078+0000","last_change":"2024-04-25T00:40:29.069078+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.069078+0000","last_undegraded":"2024-04-25T00:40:29.069078+0000","last_fullsized":"2024-04-25T00:40:29.069078+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.095362+0000","last_change":"2024-04-25T00:40:29.087901+0000","last_active":"2024-04-25T00:40:29.095362+0000","last_peered":"2024-04-25T00:40:29.095362+0000","last_clean":"2024-04-25T00:40:29.095362+0000","last_became_active":"2024-04-25T00:40:29.087264+0000","last_became_peered":"2024-04-25T00:40:29.087264+0000","last_unstale":"2024-04-25T00:40:29.095362+0000","last_undegraded":"2024-04-25T00:40:29.095362+0000","last_fullsized":"2024-04-25T00:40:29.095362+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:56.100853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.074690+0000","last_change":"2024-04-25T00:40:29.074690+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.074690+0000","last_undegraded":"2024-04-25T00:40:29.074690+0000","last_fullsized":"2024-04-25T00:40:29.074690+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.097916+0000","last_change":"2024-04-25T00:40:29.096769+0000","last_active":"2024-04-25T00:40:29.097916+0000","last_peered":"2024-04-25T00:40:29.097916+0000","last_clean":"2024-04-25T00:40:29.097916+0000","last_became_active":"2024-04-25T00:40:29.096339+0000","last_became_peered":"2024-04-25T00:40:29.096339+0000","last_unstale":"2024-04-25T00:40:29.097916+0000","last_undegraded":"2024-04-25T00:40:29.097916+0000","last_fullsized":"2024-04-25T00:40:29.097916+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.904588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.064376+0000","last_change":"2024-04-25T00:40:29.064376+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.064376+0000","last_undegraded":"2024-04-25T00:40:29.064376+0000","last_fullsized":"2024-04-25T00:40:29.064376+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.090108+0000","last_change":"2024-04-25T00:40:29.082537+0000","last_active":"2024-04-25T00:40:29.090108+0000","last_peered":"2024-04-25T00:40:29.090108+0000","last_clean":"2024-04-25T00:40:29.090108+0000","last_became_active":"2024-04-25T00:40:29.081768+0000","last_became_peered":"2024-04-25T00:40:29.081768+0000","last_unstale":"2024-04-25T00:40:29.090108+0000","last_undegraded":"2024-04-25T00:40:29.090108+0000","last_fullsized":"2024-04-25T00:40:29.090108+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:23.222465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076804+0000","last_change":"2024-04-25T00:40:29.076804+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076804+0000","last_undegraded":"2024-04-25T00:40:29.076804+0000","last_fullsized":"2024-04-25T00:40:29.076804+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.097546+0000","last_change":"2024-04-25T00:40:29.091407+0000","last_active":"2024-04-25T00:40:29.097546+0000","last_peered":"2024-04-25T00:40:29.097546+0000","last_clean":"2024-04-25T00:40:29.097546+0000","last_became_active":"2024-04-25T00:40:29.090686+0000","last_became_peered":"2024-04-25T00:40:29.090686+0000","last_unstale":"2024-04-25T00:40:29.097546+0000","last_undegraded":"2024-04-25T00:40:29.097546+0000","last_fullsized":"2024-04-25T00:40:29.097546+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:43.160214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068208+0000","last_change":"2024-04-25T00:40:29.068208+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068208+0000","last_undegraded":"2024-04-25T00:40:29.068208+0000","last_fullsized":"2024-04-25T00:40:29.068208+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.101528+0000","last_change":"2024-04-25T00:40:29.100639+0000","last_active":"2024-04-25T00:40:29.101528+0000","last_peered":"2024-04-25T00:40:29.101528+0000","last_clean":"2024-04-25T00:40:29.101528+0000","last_became_active":"2024-04-25T00:40:29.100348+0000","last_became_peered":"2024-04-25T00:40:29.100348+0000","last_unstale":"2024-04-25T00:40:29.101528+0000","last_undegraded":"2024-04-25T00:40:29.101528+0000","last_fullsized":"2024-04-25T00:40:29.101528+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:28.420738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067576+0000","last_change":"2024-04-25T00:40:29.067576+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067576+0000","last_undegraded":"2024-04-25T00:40:29.067576+0000","last_fullsized":"2024-04-25T00:40:29.067576+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.095407+0000","last_change":"2024-04-25T00:40:29.088110+0000","last_active":"2024-04-25T00:40:29.095407+0000","last_peered":"2024-04-25T00:40:29.095407+0000","last_clean":"2024-04-25T00:40:29.095407+0000","last_became_active":"2024-04-25T00:40:29.087595+0000","last_became_peered":"2024-04-25T00:40:29.087595+0000","last_unstale":"2024-04-25T00:40:29.095407+0000","last_undegraded":"2024-04-25T00:40:29.095407+0000","last_fullsized":"2024-04-25T00:40:29.095407+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:42.221346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.069149+0000","last_change":"2024-04-25T00:40:29.069149+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.069149+0000","last_undegraded":"2024-04-25T00:40:29.069149+0000","last_fullsized":"2024-04-25T00:40:29.069149+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.100608+0000","last_change":"2024-04-25T00:40:29.099633+0000","last_active":"2024-04-25T00:40:29.100608+0000","last_peered":"2024-04-25T00:40:29.100608+0000","last_clean":"2024-04-25T00:40:29.100608+0000","last_became_active":"2024-04-25T00:40:29.099060+0000","last_became_peered":"2024-04-25T00:40:29.099060+0000","last_unstale":"2024-04-25T00:40:29.100608+0000","last_undegraded":"2024-04-25T00:40:29.100608+0000","last_fullsized":"2024-04-25T00:40:29.100608+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:24.734293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076030+0000","last_change":"2024-04-25T00:40:29.076030+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076030+0000","last_undegraded":"2024-04-25T00:40:29.076030+0000","last_fullsized":"2024-04-25T00:40:29.076030+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.092384+0000","last_change":"2024-04-25T00:40:29.088653+0000","last_active":"2024-04-25T00:40:29.092384+0000","last_peered":"2024-04-25T00:40:29.092384+0000","last_clean":"2024-04-25T00:40:29.092384+0000","last_became_active":"2024-04-25T00:40:29.087923+0000","last_became_peered":"2024-04-25T00:40:29.087923+0000","last_unstale":"2024-04-25T00:40:29.092384+0000","last_undegraded":"2024-04-25T00:40:29.092384+0000","last_fullsized":"2024-04-25T00:40:29.092384+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:13.385254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065511+0000","last_change":"2024-04-25T00:40:29.065511+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065511+0000","last_undegraded":"2024-04-25T00:40:29.065511+0000","last_fullsized":"2024-04-25T00:40:29.065511+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.093663+0000","last_change":"2024-04-25T00:40:29.088591+0000","last_active":"2024-04-25T00:40:29.093663+0000","last_peered":"2024-04-25T00:40:29.093663+0000","last_clean":"2024-04-25T00:40:29.093663+0000","last_became_active":"2024-04-25T00:40:29.088270+0000","last_became_peered":"2024-04-25T00:40:29.088270+0000","last_unstale":"2024-04-25T00:40:29.093663+0000","last_undegraded":"2024-04-25T00:40:29.093663+0000","last_fullsized":"2024-04-25T00:40:29.093663+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.904588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.064117+0000","last_change":"2024-04-25T00:40:29.064117+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.064117+0000","last_undegraded":"2024-04-25T00:40:29.064117+0000","last_fullsized":"2024-04-25T00:40:29.064117+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.101315+0000","last_change":"2024-04-25T00:40:29.100762+0000","last_active":"2024-04-25T00:40:29.101315+0000","last_peered":"2024-04-25T00:40:29.101315+0000","last_clean":"2024-04-25T00:40:29.101315+0000","last_became_active":"2024-04-25T00:40:29.100159+0000","last_became_peered":"2024-04-25T00:40:29.100159+0000","last_unstale":"2024-04-25T00:40:29.101315+0000","last_undegraded":"2024-04-25T00:40:29.101315+0000","last_fullsized":"2024-04-25T00:40:29.101315+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:53.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.071314+0000","last_change":"2024-04-25T00:40:29.071314+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.071314+0000","last_undegraded":"2024-04-25T00:40:29.071314+0000","last_fullsized":"2024-04-25T00:40:29.071314+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.092551+0000","last_change":"2024-04-25T00:40:29.089205+0000","last_active":"2024-04-25T00:40:29.092551+0000","last_peered":"2024-04-25T00:40:29.092551+0000","last_clean":"2024-04-25T00:40:29.092551+0000","last_became_active":"2024-04-25T00:40:29.088497+0000","last_became_peered":"2024-04-25T00:40:29.088497+0000","last_unstale":"2024-04-25T00:40:29.092551+0000","last_undegraded":"2024-04-25T00:40:29.092551+0000","last_fullsized":"2024-04-25T00:40:29.092551+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:38.339737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.071161+0000","last_change":"2024-04-25T00:40:29.071161+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.071161+0000","last_undegraded":"2024-04-25T00:40:29.071161+0000","last_fullsized":"2024-04-25T00:40:29.071161+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.095713+0000","last_change":"2024-04-25T00:40:29.093336+0000","last_active":"2024-04-25T00:40:29.095713+0000","last_peered":"2024-04-25T00:40:29.095713+0000","last_clean":"2024-04-25T00:40:29.095713+0000","last_became_active":"2024-04-25T00:40:29.092530+0000","last_became_peered":"2024-04-25T00:40:29.092530+0000","last_unstale":"2024-04-25T00:40:29.095713+0000","last_undegraded":"2024-04-25T00:40:29.095713+0000","last_fullsized":"2024-04-25T00:40:29.095713+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:53.498894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065433+0000","last_change":"2024-04-25T00:40:29.065433+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065433+0000","last_undegraded":"2024-04-25T00:40:29.065433+0000","last_fullsized":"2024-04-25T00:40:29.065433+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.091444+0000","last_change":"2024-04-25T00:40:29.090555+0000","last_active":"2024-04-25T00:40:29.091444+0000","last_peered":"2024-04-25T00:40:29.091444+0000","last_clean":"2024-04-25T00:40:29.091444+0000","last_became_active":"2024-04-25T00:40:29.089983+0000","last_became_peered":"2024-04-25T00:40:29.089983+0000","last_unstale":"2024-04-25T00:40:29.091444+0000","last_undegraded":"2024-04-25T00:40:29.091444+0000","last_fullsized":"2024-04-25T00:40:29.091444+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:31.202069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.064890+0000","last_change":"2024-04-25T00:40:29.064890+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.064890+0000","last_undegraded":"2024-04-25T00:40:29.064890+0000","last_fullsized":"2024-04-25T00:40:29.064890+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.092463+0000","last_change":"2024-04-25T00:40:29.088835+0000","last_active":"2024-04-25T00:40:29.092463+0000","last_peered":"2024-04-25T00:40:29.092463+0000","last_clean":"2024-04-25T00:40:29.092463+0000","last_became_active":"2024-04-25T00:40:29.088099+0000","last_became_peered":"2024-04-25T00:40:29.088099+0000","last_unstale":"2024-04-25T00:40:29.092463+0000","last_undegraded":"2024-04-25T00:40:29.092463+0000","last_fullsized":"2024-04-25T00:40:29.092463+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:11.663375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.073123+0000","last_change":"2024-04-25T00:40:29.073123+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.073123+0000","last_undegraded":"2024-04-25T00:40:29.073123+0000","last_fullsized":"2024-04-25T00:40:29.073123+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.098666+0000","last_change":"2024-04-25T00:40:29.095787+0000","last_active":"2024-04-25T00:40:29.098666+0000","last_peered":"2024-04-25T00:40:29.098666+0000","last_clean":"2024-04-25T00:40:29.098666+0000","last_became_active":"2024-04-25T00:40:29.094547+0000","last_became_peered":"2024-04-25T00:40:29.094547+0000","last_unstale":"2024-04-25T00:40:29.098666+0000","last_undegraded":"2024-04-25T00:40:29.098666+0000","last_fullsized":"2024-04-25T00:40:29.098666+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:15.747990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068646+0000","last_change":"2024-04-25T00:40:29.068646+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068646+0000","last_undegraded":"2024-04-25T00:40:29.068646+0000","last_fullsized":"2024-04-25T00:40:29.068646+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.096168+0000","last_change":"2024-04-25T00:40:29.095275+0000","last_active":"2024-04-25T00:40:29.096168+0000","last_peered":"2024-04-25T00:40:29.096168+0000","last_clean":"2024-04-25T00:40:29.096168+0000","last_became_active":"2024-04-25T00:40:29.094973+0000","last_became_peered":"2024-04-25T00:40:29.094973+0000","last_unstale":"2024-04-25T00:40:29.096168+0000","last_undegraded":"2024-04-25T00:40:29.096168+0000","last_fullsized":"2024-04-25T00:40:29.096168+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:52.881575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.064353+0000","last_change":"2024-04-25T00:40:29.064353+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.064353+0000","last_undegraded":"2024-04-25T00:40:29.064353+0000","last_fullsized":"2024-04-25T00:40:29.064353+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.101103+0000","last_change":"2024-04-25T00:40:29.099837+0000","last_active":"2024-04-25T00:40:29.101103+0000","last_peered":"2024-04-25T00:40:29.101103+0000","last_clean":"2024-04-25T00:40:29.101103+0000","last_became_active":"2024-04-25T00:40:29.096352+0000","last_became_peered":"2024-04-25T00:40:29.096352+0000","last_unstale":"2024-04-25T00:40:29.101103+0000","last_undegraded":"2024-04-25T00:40:29.101103+0000","last_fullsized":"2024-04-25T00:40:29.101103+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:52.993794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076771+0000","last_change":"2024-04-25T00:40:29.076771+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076771+0000","last_undegraded":"2024-04-25T00:40:29.076771+0000","last_fullsized":"2024-04-25T00:40:29.076771+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.098071+0000","last_change":"2024-04-25T00:40:29.097310+0000","last_active":"2024-04-25T00:40:29.098071+0000","last_peered":"2024-04-25T00:40:29.098071+0000","last_clean":"2024-04-25T00:40:29.098071+0000","last_became_active":"2024-04-25T00:40:29.096881+0000","last_became_peered":"2024-04-25T00:40:29.096881+0000","last_unstale":"2024-04-25T00:40:29.098071+0000","last_undegraded":"2024-04-25T00:40:29.098071+0000","last_fullsized":"2024-04-25T00:40:29.098071+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:25.492182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082449+0000","last_change":"2024-04-25T00:40:29.082449+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082449+0000","last_undegraded":"2024-04-25T00:40:29.082449+0000","last_fullsized":"2024-04-25T00:40:29.082449+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.088718+0000","last_change":"2024-04-25T00:40:29.083696+0000","last_active":"2024-04-25T00:40:29.088718+0000","last_peered":"2024-04-25T00:40:29.088718+0000","last_clean":"2024-04-25T00:40:29.088718+0000","last_became_active":"2024-04-25T00:40:29.082983+0000","last_became_peered":"2024-04-25T00:40:29.082983+0000","last_unstale":"2024-04-25T00:40:29.088718+0000","last_undegraded":"2024-04-25T00:40:29.088718+0000","last_fullsized":"2024-04-25T00:40:29.088718+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:13.703146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072503+0000","last_change":"2024-04-25T00:40:29.072503+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072503+0000","last_undegraded":"2024-04-25T00:40:29.072503+0000","last_fullsized":"2024-04-25T00:40:29.072503+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.099838+0000","last_change":"2024-04-25T00:40:29.098261+0000","last_active":"2024-04-25T00:40:29.099838+0000","last_peered":"2024-04-25T00:40:29.099838+0000","last_clean":"2024-04-25T00:40:29.099838+0000","last_became_active":"2024-04-25T00:40:29.097558+0000","last_became_peered":"2024-04-25T00:40:29.097558+0000","last_unstale":"2024-04-25T00:40:29.099838+0000","last_undegraded":"2024-04-25T00:40:29.099838+0000","last_fullsized":"2024-04-25T00:40:29.099838+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:01.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.070128+0000","last_change":"2024-04-25T00:40:29.070128+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.070128+0000","last_undegraded":"2024-04-25T00:40:29.070128+0000","last_fullsized":"2024-04-25T00:40:29.070128+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.096865+0000","last_change":"2024-04-25T00:40:29.096322+0000","last_active":"2024-04-25T00:40:29.096865+0000","last_peered":"2024-04-25T00:40:29.096865+0000","last_clean":"2024-04-25T00:40:29.096865+0000","last_became_active":"2024-04-25T00:40:29.096034+0000","last_became_peered":"2024-04-25T00:40:29.096034+0000","last_unstale":"2024-04-25T00:40:29.096865+0000","last_undegraded":"2024-04-25T00:40:29.096865+0000","last_fullsized":"2024-04-25T00:40:29.096865+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:53.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067461+0000","last_change":"2024-04-25T00:40:29.067461+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067461+0000","last_undegraded":"2024-04-25T00:40:29.067461+0000","last_fullsized":"2024-04-25T00:40:29.067461+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.093696+0000","last_change":"2024-04-25T00:40:29.090943+0000","last_active":"2024-04-25T00:40:29.093696+0000","last_peered":"2024-04-25T00:40:29.093696+0000","last_clean":"2024-04-25T00:40:29.093696+0000","last_became_active":"2024-04-25T00:40:29.090524+0000","last_became_peered":"2024-04-25T00:40:29.090524+0000","last_unstale":"2024-04-25T00:40:29.093696+0000","last_undegraded":"2024-04-25T00:40:29.093696+0000","last_fullsized":"2024-04-25T00:40:29.093696+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:40.259597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068430+0000","last_change":"2024-04-25T00:40:29.068430+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068430+0000","last_undegraded":"2024-04-25T00:40:29.068430+0000","last_fullsized":"2024-04-25T00:40:29.068430+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.098189+0000","last_change":"2024-04-25T00:40:29.097144+0000","last_active":"2024-04-25T00:40:29.098189+0000","last_peered":"2024-04-25T00:40:29.098189+0000","last_clean":"2024-04-25T00:40:29.098189+0000","last_became_active":"2024-04-25T00:40:29.096558+0000","last_became_peered":"2024-04-25T00:40:29.096558+0000","last_unstale":"2024-04-25T00:40:29.098189+0000","last_undegraded":"2024-04-25T00:40:29.098189+0000","last_fullsized":"2024-04-25T00:40:29.098189+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:40.259597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068589+0000","last_change":"2024-04-25T00:40:29.068589+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068589+0000","last_undegraded":"2024-04-25T00:40:29.068589+0000","last_fullsized":"2024-04-25T00:40:29.068589+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.079341+0000","last_change":"2024-04-25T00:40:29.077975+0000","last_active":"2024-04-25T00:40:29.079341+0000","last_peered":"2024-04-25T00:40:29.079341+0000","last_clean":"2024-04-25T00:40:29.079341+0000","last_became_active":"2024-04-25T00:40:29.077218+0000","last_became_peered":"2024-04-25T00:40:29.077218+0000","last_unstale":"2024-04-25T00:40:29.079341+0000","last_undegraded":"2024-04-25T00:40:29.079341+0000","last_fullsized":"2024-04-25T00:40:29.079341+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:21.386455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068479+0000","last_change":"2024-04-25T00:40:29.068479+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068479+0000","last_undegraded":"2024-04-25T00:40:29.068479+0000","last_fullsized":"2024-04-25T00:40:29.068479+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.098857+0000","last_change":"2024-04-25T00:40:29.096474+0000","last_active":"2024-04-25T00:40:29.098857+0000","last_peered":"2024-04-25T00:40:29.098857+0000","last_clean":"2024-04-25T00:40:29.098857+0000","last_became_active":"2024-04-25T00:40:29.095752+0000","last_became_peered":"2024-04-25T00:40:29.095752+0000","last_unstale":"2024-04-25T00:40:29.098857+0000","last_undegraded":"2024-04-25T00:40:29.098857+0000","last_fullsized":"2024-04-25T00:40:29.098857+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:42.017864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.074826+0000","last_change":"2024-04-25T00:40:29.074826+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.074826+0000","last_undegraded":"2024-04-25T00:40:29.074826+0000","last_fullsized":"2024-04-25T00:40:29.074826+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.099838+0000","last_change":"2024-04-25T00:40:29.084462+0000","last_active":"2024-04-25T00:40:29.099838+0000","last_peered":"2024-04-25T00:40:29.099838+0000","last_clean":"2024-04-25T00:40:29.099838+0000","last_became_active":"2024-04-25T00:40:29.082507+0000","last_became_peered":"2024-04-25T00:40:29.082507+0000","last_unstale":"2024-04-25T00:40:29.099838+0000","last_undegraded":"2024-04-25T00:40:29.099838+0000","last_fullsized":"2024-04-25T00:40:29.099838+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:52.881575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072774+0000","last_change":"2024-04-25T00:40:29.072774+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072774+0000","last_undegraded":"2024-04-25T00:40:29.072774+0000","last_fullsized":"2024-04-25T00:40:29.072774+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.095525+0000","last_change":"2024-04-25T00:40:29.094661+0000","last_active":"2024-04-25T00:40:29.095525+0000","last_peered":"2024-04-25T00:40:29.095525+0000","last_clean":"2024-04-25T00:40:29.095525+0000","last_became_active":"2024-04-25T00:40:29.094039+0000","last_became_peered":"2024-04-25T00:40:29.094039+0000","last_unstale":"2024-04-25T00:40:29.095525+0000","last_undegraded":"2024-04-25T00:40:29.095525+0000","last_fullsized":"2024-04-25T00:40:29.095525+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:49.527153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067669+0000","last_change":"2024-04-25T00:40:29.067669+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067669+0000","last_undegraded":"2024-04-25T00:40:29.067669+0000","last_fullsized":"2024-04-25T00:40:29.067669+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.102744+0000","last_change":"2024-04-25T00:40:29.101828+0000","last_active":"2024-04-25T00:40:29.102744+0000","last_peered":"2024-04-25T00:40:29.102744+0000","last_clean":"2024-04-25T00:40:29.102744+0000","last_became_active":"2024-04-25T00:40:29.101030+0000","last_became_peered":"2024-04-25T00:40:29.101030+0000","last_unstale":"2024-04-25T00:40:29.102744+0000","last_undegraded":"2024-04-25T00:40:29.102744+0000","last_fullsized":"2024-04-25T00:40:29.102744+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:08.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":462,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.070740+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:40:29.070740+0000","last_peered":"2024-04-25T00:40:29.070740+0000","last_clean":"2024-04-25T00:40:29.070740+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:40:29.070740+0000","last_undegraded":"2024-04-25T00:40:29.070740+0000","last_fullsized":"2024-04-25T00:40:29.070740+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574960,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37808,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334032,"statfs":{"total":96636764160,"available":96598048768,"internally_reserved":0,"allocated":2523136,"data_stored":1440265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574961,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1941504,"data_stored":856172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41388,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330452,"statfs":{"total":96636764160,"available":96594382848,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37232,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574960,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41968,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329872,"statfs":{"total":96636764160,"available":96593788928,"internally_reserved":0,"allocated":2523136,"data_stored":1440265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:40:34.126 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:40:34.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.285+0000 7fb92f354640 1 Processor -- start 2024-04-25T00:40:34.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.285+0000 7fb92f354640 1 -- start start 2024-04-25T00:40:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb92f354640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb9280b3140 0x7fb92806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb92f354640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9280b41e0 con 0x7fb9280b4b10 2024-04-25T00:40:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb92f354640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb92806d960 con 0x7fb9280b3c30 2024-04-25T00:40:34.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb92f354640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb92806dc40 con 0x7fb9280b3140 2024-04-25T00:40:34.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb92d0f0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb9280b3140 0x7fb92806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb92c8ef640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb9280b4b10 0x7fb9280b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55750/0 (socket says 172.21.15.29:55750) 2024-04-25T00:40:34.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb92c8ef640 1 -- 172.21.15.29:0/624268447 learned_addr learned my addr 172.21.15.29:0/624268447 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:34.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb92d0f0640 1 -- 172.21.15.29:0/624268447 >> v1:172.21.15.157:6789/0 conn(0x7fb9280b3c30 legacy=0x7fb9280ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb92d0f0640 1 -- 172.21.15.29:0/624268447 >> v1:172.21.15.29:6789/0 conn(0x7fb9280b4b10 legacy=0x7fb9280b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb92d0f0640 1 -- 172.21.15.29:0/624268447 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92806df20 con 0x7fb9280b3140 2024-04-25T00:40:34.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb91ffff640 1 -- 172.21.15.29:0/624268447 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 434880129 0 0) 0x7fb9280b41e0 con 0x7fb9280b4b10 2024-04-25T00:40:34.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb92d0f0640 1 --2- 172.21.15.29:0/624268447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9280b3140 0x7fb92806e440 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7fb9240097f0 tx=0x7fb92401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fa2da8a7b22e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb91ffff640 1 -- 172.21.15.29:0/624268447 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240b0070 con 0x7fb9280b3140 2024-04-25T00:40:34.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb91ffff640 1 -- 172.21.15.29:0/624268447 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9240ab600 con 0x7fb9280b3140 2024-04-25T00:40:34.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.289+0000 7fb91ffff640 1 -- 172.21.15.29:0/624268447 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240ab9d0 con 0x7fb9280b3140 2024-04-25T00:40:34.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 -- 172.21.15.29:0/624268447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9280b3140 msgr2=0x7fb92806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 --2- 172.21.15.29:0/624268447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9280b3140 0x7fb92806e440 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7fb9240097f0 tx=0x7fb92401c460 comp rx=0 tx=0).stop 2024-04-25T00:40:34.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 -- 172.21.15.29:0/624268447 shutdown_connections 2024-04-25T00:40:34.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 --2- 172.21.15.29:0/624268447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9280b3140 0x7fb92806e440 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 -- 172.21.15.29:0/624268447 >> 172.21.15.29:0/624268447 conn(0x7fb9280b1870 msgr2=0x7fb9280b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 -- 172.21.15.29:0/624268447 shutdown_connections 2024-04-25T00:40:34.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 -- 172.21.15.29:0/624268447 wait complete. 2024-04-25T00:40:34.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 Processor -- start 2024-04-25T00:40:34.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 -- start start 2024-04-25T00:40:34.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9280b3140 0x7fb928153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9280b3c30 0x7fb92813dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9280b4b10 0x7fb92813e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb928169d90 con 0x7fb9280b3c30 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb928169c10 con 0x7fb9280b4b10 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb928169f10 con 0x7fb9280b3140 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92c8ef640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9280b3c30 0x7fb92813dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92c8ef640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9280b3c30 0x7fb92813dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59976/0 (socket says 172.21.15.29:59976) 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92c8ef640 1 -- 172.21.15.29:0/973139765 learned_addr learned my addr 172.21.15.29:0/973139765 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92d8f1640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9280b4b10 0x7fb92813e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92d0f0640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9280b3140 0x7fb928153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92c8ef640 1 -- 172.21.15.29:0/973139765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9280b3140 msgr2=0x7fb928153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92c8ef640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9280b3140 0x7fb928153440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92c8ef640 1 -- 172.21.15.29:0/973139765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9280b4b10 msgr2=0x7fb92813e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92c8ef640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9280b4b10 0x7fb92813e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92c8ef640 1 -- 172.21.15.29:0/973139765 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb928154de0 con 0x7fb9280b3c30 2024-04-25T00:40:34.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92d8f1640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9280b4b10 0x7fb92813e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:34.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92c8ef640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9280b3c30 0x7fb92813dde0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fb9180011f0 tx=0x7fb91800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb91dffb640 1 -- 172.21.15.29:0/973139765 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918018070 con 0x7fb9280b3c30 2024-04-25T00:40:34.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb91dffb640 1 -- 172.21.15.29:0/973139765 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9180074e0 con 0x7fb9280b3c30 2024-04-25T00:40:34.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb92f354640 1 -- 172.21.15.29:0/973139765 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb928128ca0 con 0x7fb9280b3c30 2024-04-25T00:40:34.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.293+0000 7fb91dffb640 1 -- 172.21.15.29:0/973139765 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918004a10 con 0x7fb9280b3c30 2024-04-25T00:40:34.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.297+0000 7fb92f354640 1 -- 172.21.15.29:0/973139765 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb928129220 con 0x7fb9280b3c30 2024-04-25T00:40:34.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.297+0000 7fb92f354640 1 -- 172.21.15.29:0/973139765 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb928069270 con 0x7fb9280b3c30 2024-04-25T00:40:34.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.297+0000 7fb91dffb640 1 -- 172.21.15.29:0/973139765 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fb91801b070 con 0x7fb9280b3c30 2024-04-25T00:40:34.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.297+0000 7fb91dffb640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb90c0680d0 0x7fb90c06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.297+0000 7fb92d0f0640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb90c0680d0 0x7fb90c06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.297+0000 7fb91dffb640 1 -- 172.21.15.29:0/973139765 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fb9180861e0 con 0x7fb9280b3c30 2024-04-25T00:40:34.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.297+0000 7fb92d0f0640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb90c0680d0 0x7fb90c06a590 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb9240ad000 tx=0x7fb924008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.301+0000 7fb91dffb640 1 -- 172.21.15.29:0/973139765 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb918055dd0 con 0x7fb9280b3c30 2024-04-25T00:40:34.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.453+0000 7fb92f354640 1 -- 172.21.15.29:0/973139765 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb928157000 con 0x7fb90c0680d0 2024-04-25T00:40:34.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.473+0000 7fb91dffb640 1 -- 172.21.15.29:0/973139765 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247088 (secure 0 0 0) 0x7fb928157000 con 0x7fb90c0680d0 2024-04-25T00:40:34.475 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:34.476 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:40:34.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.477+0000 7fb92f354640 1 -- 172.21.15.29:0/973139765 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb90c0680d0 msgr2=0x7fb90c06a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.477+0000 7fb92f354640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb90c0680d0 0x7fb90c06a590 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb9240ad000 tx=0x7fb924008040 comp rx=0 tx=0).stop 2024-04-25T00:40:34.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.477+0000 7fb92f354640 1 -- 172.21.15.29:0/973139765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9280b3c30 msgr2=0x7fb92813dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.477+0000 7fb92f354640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9280b3c30 0x7fb92813dde0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fb9180011f0 tx=0x7fb91800c9b0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.477+0000 7fb92f354640 1 -- 172.21.15.29:0/973139765 shutdown_connections 2024-04-25T00:40:34.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.477+0000 7fb92f354640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb90c0680d0 0x7fb90c06a590 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.477+0000 7fb92f354640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9280b4b10 0x7fb92813e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.477+0000 7fb92f354640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9280b3c30 0x7fb92813dde0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.477+0000 7fb92f354640 1 --2- 172.21.15.29:0/973139765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9280b3140 0x7fb928153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.477+0000 7fb92f354640 1 -- 172.21.15.29:0/973139765 >> 172.21.15.29:0/973139765 conn(0x7fb9280b1870 msgr2=0x7fb9280cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.477+0000 7fb92f354640 1 -- 172.21.15.29:0/973139765 shutdown_connections 2024-04-25T00:40:34.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.477+0000 7fb92f354640 1 -- 172.21.15.29:0/973139765 wait complete. 2024-04-25T00:40:34.491 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-04-25T00:40:33.405476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307328,"kb_used_data":16244,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667392,"statfs":{"total":773094113280,"available":772779409408,"internally_reserved":0,"allocated":16633856,"data_stored":7986187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.350831"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068067+0000","last_change":"2024-04-25T00:40:29.068067+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068067+0000","last_undegraded":"2024-04-25T00:40:29.068067+0000","last_fullsized":"2024-04-25T00:40:29.068067+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.077656+0000","last_change":"2024-04-25T00:40:29.077656+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.077656+0000","last_undegraded":"2024-04-25T00:40:29.077656+0000","last_fullsized":"2024-04-25T00:40:29.077656+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.085806+0000","last_change":"2024-04-25T00:40:29.085806+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.085806+0000","last_undegraded":"2024-04-25T00:40:29.085806+0000","last_fullsized":"2024-04-25T00:40:29.085806+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076453+0000","last_change":"2024-04-25T00:40:29.076453+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076453+0000","last_undegraded":"2024-04-25T00:40:29.076453+0000","last_fullsized":"2024-04-25T00:40:29.076453+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.083171+0000","last_change":"2024-04-25T00:40:29.083171+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.083171+0000","last_undegraded":"2024-04-25T00:40:29.083171+0000","last_fullsized":"2024-04-25T00:40:29.083171+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076571+0000","last_change":"2024-04-25T00:40:29.076571+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076571+0000","last_undegraded":"2024-04-25T00:40:29.076571+0000","last_fullsized":"2024-04-25T00:40:29.076571+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067060+0000","last_change":"2024-04-25T00:40:29.067060+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067060+0000","last_undegraded":"2024-04-25T00:40:29.067060+0000","last_fullsized":"2024-04-25T00:40:29.067060+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.078988+0000","last_change":"2024-04-25T00:40:29.078988+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.078988+0000","last_undegraded":"2024-04-25T00:40:29.078988+0000","last_fullsized":"2024-04-25T00:40:29.078988+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.066446+0000","last_change":"2024-04-25T00:40:29.066446+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.066446+0000","last_undegraded":"2024-04-25T00:40:29.066446+0000","last_fullsized":"2024-04-25T00:40:29.066446+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.071624+0000","last_change":"2024-04-25T00:40:29.071624+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.071624+0000","last_undegraded":"2024-04-25T00:40:29.071624+0000","last_fullsized":"2024-04-25T00:40:29.071624+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065659+0000","last_change":"2024-04-25T00:40:29.065659+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065659+0000","last_undegraded":"2024-04-25T00:40:29.065659+0000","last_fullsized":"2024-04-25T00:40:29.065659+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.075405+0000","last_change":"2024-04-25T00:40:29.075405+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.075405+0000","last_undegraded":"2024-04-25T00:40:29.075405+0000","last_fullsized":"2024-04-25T00:40:29.075405+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.079997+0000","last_change":"2024-04-25T00:40:29.079997+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.079997+0000","last_undegraded":"2024-04-25T00:40:29.079997+0000","last_fullsized":"2024-04-25T00:40:29.079997+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067927+0000","last_change":"2024-04-25T00:40:29.067927+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067927+0000","last_undegraded":"2024-04-25T00:40:29.067927+0000","last_fullsized":"2024-04-25T00:40:29.067927+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.080999+0000","last_change":"2024-04-25T00:40:29.080999+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.080999+0000","last_undegraded":"2024-04-25T00:40:29.080999+0000","last_fullsized":"2024-04-25T00:40:29.080999+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.078469+0000","last_change":"2024-04-25T00:40:29.078469+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.078469+0000","last_undegraded":"2024-04-25T00:40:29.078469+0000","last_fullsized":"2024-04-25T00:40:29.078469+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082514+0000","last_change":"2024-04-25T00:40:29.082514+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082514+0000","last_undegraded":"2024-04-25T00:40:29.082514+0000","last_fullsized":"2024-04-25T00:40:29.082514+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068120+0000","last_change":"2024-04-25T00:40:29.068120+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068120+0000","last_undegraded":"2024-04-25T00:40:29.068120+0000","last_fullsized":"2024-04-25T00:40:29.068120+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.069605+0000","last_change":"2024-04-25T00:40:29.069605+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.069605+0000","last_undegraded":"2024-04-25T00:40:29.069605+0000","last_fullsized":"2024-04-25T00:40:29.069605+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072609+0000","last_change":"2024-04-25T00:40:29.072609+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072609+0000","last_undegraded":"2024-04-25T00:40:29.072609+0000","last_fullsized":"2024-04-25T00:40:29.072609+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082713+0000","last_change":"2024-04-25T00:40:29.082713+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082713+0000","last_undegraded":"2024-04-25T00:40:29.082713+0000","last_fullsized":"2024-04-25T00:40:29.082713+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.069538+0000","last_change":"2024-04-25T00:40:29.069538+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.069538+0000","last_undegraded":"2024-04-25T00:40:29.069538+0000","last_fullsized":"2024-04-25T00:40:29.069538+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.085695+0000","last_change":"2024-04-25T00:40:29.085695+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.085695+0000","last_undegraded":"2024-04-25T00:40:29.085695+0000","last_fullsized":"2024-04-25T00:40:29.085695+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.077654+0000","last_change":"2024-04-25T00:40:29.077654+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.077654+0000","last_undegraded":"2024-04-25T00:40:29.077654+0000","last_fullsized":"2024-04-25T00:40:29.077654+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.073015+0000","last_change":"2024-04-25T00:40:29.073015+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.073015+0000","last_undegraded":"2024-04-25T00:40:29.073015+0000","last_fullsized":"2024-04-25T00:40:29.073015+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065060+0000","last_change":"2024-04-25T00:40:29.065060+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065060+0000","last_undegraded":"2024-04-25T00:40:29.065060+0000","last_fullsized":"2024-04-25T00:40:29.065060+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082493+0000","last_change":"2024-04-25T00:40:29.082493+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082493+0000","last_undegraded":"2024-04-25T00:40:29.082493+0000","last_fullsized":"2024-04-25T00:40:29.082493+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072143+0000","last_change":"2024-04-25T00:40:29.072143+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072143+0000","last_undegraded":"2024-04-25T00:40:29.072143+0000","last_fullsized":"2024-04-25T00:40:29.072143+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072837+0000","last_change":"2024-04-25T00:40:29.072837+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072837+0000","last_undegraded":"2024-04-25T00:40:29.072837+0000","last_fullsized":"2024-04-25T00:40:29.072837+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065859+0000","last_change":"2024-04-25T00:40:29.065859+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065859+0000","last_undegraded":"2024-04-25T00:40:29.065859+0000","last_fullsized":"2024-04-25T00:40:29.065859+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082748+0000","last_change":"2024-04-25T00:40:29.082748+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082748+0000","last_undegraded":"2024-04-25T00:40:29.082748+0000","last_fullsized":"2024-04-25T00:40:29.082748+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.071805+0000","last_change":"2024-04-25T00:40:29.071805+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.071805+0000","last_undegraded":"2024-04-25T00:40:29.071805+0000","last_fullsized":"2024-04-25T00:40:29.071805+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.074871+0000","last_change":"2024-04-25T00:40:29.074871+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.074871+0000","last_undegraded":"2024-04-25T00:40:29.074871+0000","last_fullsized":"2024-04-25T00:40:29.074871+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.100723+0000","last_change":"2024-04-25T00:40:29.094752+0000","last_active":"2024-04-25T00:40:29.100723+0000","last_peered":"2024-04-25T00:40:29.100723+0000","last_clean":"2024-04-25T00:40:29.100723+0000","last_became_active":"2024-04-25T00:40:29.094472+0000","last_became_peered":"2024-04-25T00:40:29.094472+0000","last_unstale":"2024-04-25T00:40:29.100723+0000","last_undegraded":"2024-04-25T00:40:29.100723+0000","last_fullsized":"2024-04-25T00:40:29.100723+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:59.895661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.070618+0000","last_change":"2024-04-25T00:40:29.070618+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.070618+0000","last_undegraded":"2024-04-25T00:40:29.070618+0000","last_fullsized":"2024-04-25T00:40:29.070618+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.088790+0000","last_change":"2024-04-25T00:40:29.085706+0000","last_active":"2024-04-25T00:40:29.088790+0000","last_peered":"2024-04-25T00:40:29.088790+0000","last_clean":"2024-04-25T00:40:29.088790+0000","last_became_active":"2024-04-25T00:40:29.084970+0000","last_became_peered":"2024-04-25T00:40:29.084970+0000","last_unstale":"2024-04-25T00:40:29.088790+0000","last_undegraded":"2024-04-25T00:40:29.088790+0000","last_fullsized":"2024-04-25T00:40:29.088790+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:57.516592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076415+0000","last_change":"2024-04-25T00:40:29.076415+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076415+0000","last_undegraded":"2024-04-25T00:40:29.076415+0000","last_fullsized":"2024-04-25T00:40:29.076415+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.099678+0000","last_change":"2024-04-25T00:40:29.098108+0000","last_active":"2024-04-25T00:40:29.099678+0000","last_peered":"2024-04-25T00:40:29.099678+0000","last_clean":"2024-04-25T00:40:29.099678+0000","last_became_active":"2024-04-25T00:40:29.097478+0000","last_became_peered":"2024-04-25T00:40:29.097478+0000","last_unstale":"2024-04-25T00:40:29.099678+0000","last_undegraded":"2024-04-25T00:40:29.099678+0000","last_fullsized":"2024-04-25T00:40:29.099678+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:14.983065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067036+0000","last_change":"2024-04-25T00:40:29.067036+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067036+0000","last_undegraded":"2024-04-25T00:40:29.067036+0000","last_fullsized":"2024-04-25T00:40:29.067036+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.101617+0000","last_change":"2024-04-25T00:40:29.096877+0000","last_active":"2024-04-25T00:40:29.101617+0000","last_peered":"2024-04-25T00:40:29.101617+0000","last_clean":"2024-04-25T00:40:29.101617+0000","last_became_active":"2024-04-25T00:40:29.096218+0000","last_became_peered":"2024-04-25T00:40:29.096218+0000","last_unstale":"2024-04-25T00:40:29.101617+0000","last_undegraded":"2024-04-25T00:40:29.101617+0000","last_fullsized":"2024-04-25T00:40:29.101617+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:25.492182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.069078+0000","last_change":"2024-04-25T00:40:29.069078+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.069078+0000","last_undegraded":"2024-04-25T00:40:29.069078+0000","last_fullsized":"2024-04-25T00:40:29.069078+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.095362+0000","last_change":"2024-04-25T00:40:29.087901+0000","last_active":"2024-04-25T00:40:29.095362+0000","last_peered":"2024-04-25T00:40:29.095362+0000","last_clean":"2024-04-25T00:40:29.095362+0000","last_became_active":"2024-04-25T00:40:29.087264+0000","last_became_peered":"2024-04-25T00:40:29.087264+0000","last_unstale":"2024-04-25T00:40:29.095362+0000","last_undegraded":"2024-04-25T00:40:29.095362+0000","last_fullsized":"2024-04-25T00:40:29.095362+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:56.100853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.074690+0000","last_change":"2024-04-25T00:40:29.074690+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.074690+0000","last_undegraded":"2024-04-25T00:40:29.074690+0000","last_fullsized":"2024-04-25T00:40:29.074690+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.097916+0000","last_change":"2024-04-25T00:40:29.096769+0000","last_active":"2024-04-25T00:40:29.097916+0000","last_peered":"2024-04-25T00:40:29.097916+0000","last_clean":"2024-04-25T00:40:29.097916+0000","last_became_active":"2024-04-25T00:40:29.096339+0000","last_became_peered":"2024-04-25T00:40:29.096339+0000","last_unstale":"2024-04-25T00:40:29.097916+0000","last_undegraded":"2024-04-25T00:40:29.097916+0000","last_fullsized":"2024-04-25T00:40:29.097916+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.904588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.064376+0000","last_change":"2024-04-25T00:40:29.064376+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.064376+0000","last_undegraded":"2024-04-25T00:40:29.064376+0000","last_fullsized":"2024-04-25T00:40:29.064376+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.090108+0000","last_change":"2024-04-25T00:40:29.082537+0000","last_active":"2024-04-25T00:40:29.090108+0000","last_peered":"2024-04-25T00:40:29.090108+0000","last_clean":"2024-04-25T00:40:29.090108+0000","last_became_active":"2024-04-25T00:40:29.081768+0000","last_became_peered":"2024-04-25T00:40:29.081768+0000","last_unstale":"2024-04-25T00:40:29.090108+0000","last_undegraded":"2024-04-25T00:40:29.090108+0000","last_fullsized":"2024-04-25T00:40:29.090108+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:23.222465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076804+0000","last_change":"2024-04-25T00:40:29.076804+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076804+0000","last_undegraded":"2024-04-25T00:40:29.076804+0000","last_fullsized":"2024-04-25T00:40:29.076804+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.097546+0000","last_change":"2024-04-25T00:40:29.091407+0000","last_active":"2024-04-25T00:40:29.097546+0000","last_peered":"2024-04-25T00:40:29.097546+0000","last_clean":"2024-04-25T00:40:29.097546+0000","last_became_active":"2024-04-25T00:40:29.090686+0000","last_became_peered":"2024-04-25T00:40:29.090686+0000","last_unstale":"2024-04-25T00:40:29.097546+0000","last_undegraded":"2024-04-25T00:40:29.097546+0000","last_fullsized":"2024-04-25T00:40:29.097546+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:43.160214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068208+0000","last_change":"2024-04-25T00:40:29.068208+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068208+0000","last_undegraded":"2024-04-25T00:40:29.068208+0000","last_fullsized":"2024-04-25T00:40:29.068208+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.101528+0000","last_change":"2024-04-25T00:40:29.100639+0000","last_active":"2024-04-25T00:40:29.101528+0000","last_peered":"2024-04-25T00:40:29.101528+0000","last_clean":"2024-04-25T00:40:29.101528+0000","last_became_active":"2024-04-25T00:40:29.100348+0000","last_became_peered":"2024-04-25T00:40:29.100348+0000","last_unstale":"2024-04-25T00:40:29.101528+0000","last_undegraded":"2024-04-25T00:40:29.101528+0000","last_fullsized":"2024-04-25T00:40:29.101528+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:28.420738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067576+0000","last_change":"2024-04-25T00:40:29.067576+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067576+0000","last_undegraded":"2024-04-25T00:40:29.067576+0000","last_fullsized":"2024-04-25T00:40:29.067576+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.095407+0000","last_change":"2024-04-25T00:40:29.088110+0000","last_active":"2024-04-25T00:40:29.095407+0000","last_peered":"2024-04-25T00:40:29.095407+0000","last_clean":"2024-04-25T00:40:29.095407+0000","last_became_active":"2024-04-25T00:40:29.087595+0000","last_became_peered":"2024-04-25T00:40:29.087595+0000","last_unstale":"2024-04-25T00:40:29.095407+0000","last_undegraded":"2024-04-25T00:40:29.095407+0000","last_fullsized":"2024-04-25T00:40:29.095407+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:42.221346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.069149+0000","last_change":"2024-04-25T00:40:29.069149+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.069149+0000","last_undegraded":"2024-04-25T00:40:29.069149+0000","last_fullsized":"2024-04-25T00:40:29.069149+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.100608+0000","last_change":"2024-04-25T00:40:29.099633+0000","last_active":"2024-04-25T00:40:29.100608+0000","last_peered":"2024-04-25T00:40:29.100608+0000","last_clean":"2024-04-25T00:40:29.100608+0000","last_became_active":"2024-04-25T00:40:29.099060+0000","last_became_peered":"2024-04-25T00:40:29.099060+0000","last_unstale":"2024-04-25T00:40:29.100608+0000","last_undegraded":"2024-04-25T00:40:29.100608+0000","last_fullsized":"2024-04-25T00:40:29.100608+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:24.734293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076030+0000","last_change":"2024-04-25T00:40:29.076030+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076030+0000","last_undegraded":"2024-04-25T00:40:29.076030+0000","last_fullsized":"2024-04-25T00:40:29.076030+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.092384+0000","last_change":"2024-04-25T00:40:29.088653+0000","last_active":"2024-04-25T00:40:29.092384+0000","last_peered":"2024-04-25T00:40:29.092384+0000","last_clean":"2024-04-25T00:40:29.092384+0000","last_became_active":"2024-04-25T00:40:29.087923+0000","last_became_peered":"2024-04-25T00:40:29.087923+0000","last_unstale":"2024-04-25T00:40:29.092384+0000","last_undegraded":"2024-04-25T00:40:29.092384+0000","last_fullsized":"2024-04-25T00:40:29.092384+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:13.385254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065511+0000","last_change":"2024-04-25T00:40:29.065511+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065511+0000","last_undegraded":"2024-04-25T00:40:29.065511+0000","last_fullsized":"2024-04-25T00:40:29.065511+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.093663+0000","last_change":"2024-04-25T00:40:29.088591+0000","last_active":"2024-04-25T00:40:29.093663+0000","last_peered":"2024-04-25T00:40:29.093663+0000","last_clean":"2024-04-25T00:40:29.093663+0000","last_became_active":"2024-04-25T00:40:29.088270+0000","last_became_peered":"2024-04-25T00:40:29.088270+0000","last_unstale":"2024-04-25T00:40:29.093663+0000","last_undegraded":"2024-04-25T00:40:29.093663+0000","last_fullsized":"2024-04-25T00:40:29.093663+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.904588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.064117+0000","last_change":"2024-04-25T00:40:29.064117+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.064117+0000","last_undegraded":"2024-04-25T00:40:29.064117+0000","last_fullsized":"2024-04-25T00:40:29.064117+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.101315+0000","last_change":"2024-04-25T00:40:29.100762+0000","last_active":"2024-04-25T00:40:29.101315+0000","last_peered":"2024-04-25T00:40:29.101315+0000","last_clean":"2024-04-25T00:40:29.101315+0000","last_became_active":"2024-04-25T00:40:29.100159+0000","last_became_peered":"2024-04-25T00:40:29.100159+0000","last_unstale":"2024-04-25T00:40:29.101315+0000","last_undegraded":"2024-04-25T00:40:29.101315+0000","last_fullsized":"2024-04-25T00:40:29.101315+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:53.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.071314+0000","last_change":"2024-04-25T00:40:29.071314+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.071314+0000","last_undegraded":"2024-04-25T00:40:29.071314+0000","last_fullsized":"2024-04-25T00:40:29.071314+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.092551+0000","last_change":"2024-04-25T00:40:29.089205+0000","last_active":"2024-04-25T00:40:29.092551+0000","last_peered":"2024-04-25T00:40:29.092551+0000","last_clean":"2024-04-25T00:40:29.092551+0000","last_became_active":"2024-04-25T00:40:29.088497+0000","last_became_peered":"2024-04-25T00:40:29.088497+0000","last_unstale":"2024-04-25T00:40:29.092551+0000","last_undegraded":"2024-04-25T00:40:29.092551+0000","last_fullsized":"2024-04-25T00:40:29.092551+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:38.339737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.071161+0000","last_change":"2024-04-25T00:40:29.071161+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.071161+0000","last_undegraded":"2024-04-25T00:40:29.071161+0000","last_fullsized":"2024-04-25T00:40:29.071161+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.095713+0000","last_change":"2024-04-25T00:40:29.093336+0000","last_active":"2024-04-25T00:40:29.095713+0000","last_peered":"2024-04-25T00:40:29.095713+0000","last_clean":"2024-04-25T00:40:29.095713+0000","last_became_active":"2024-04-25T00:40:29.092530+0000","last_became_peered":"2024-04-25T00:40:29.092530+0000","last_unstale":"2024-04-25T00:40:29.095713+0000","last_undegraded":"2024-04-25T00:40:29.095713+0000","last_fullsized":"2024-04-25T00:40:29.095713+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:53.498894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065433+0000","last_change":"2024-04-25T00:40:29.065433+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065433+0000","last_undegraded":"2024-04-25T00:40:29.065433+0000","last_fullsized":"2024-04-25T00:40:29.065433+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.091444+0000","last_change":"2024-04-25T00:40:29.090555+0000","last_active":"2024-04-25T00:40:29.091444+0000","last_peered":"2024-04-25T00:40:29.091444+0000","last_clean":"2024-04-25T00:40:29.091444+0000","last_became_active":"2024-04-25T00:40:29.089983+0000","last_became_peered":"2024-04-25T00:40:29.089983+0000","last_unstale":"2024-04-25T00:40:29.091444+0000","last_undegraded":"2024-04-25T00:40:29.091444+0000","last_fullsized":"2024-04-25T00:40:29.091444+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:31.202069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.064890+0000","last_change":"2024-04-25T00:40:29.064890+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.064890+0000","last_undegraded":"2024-04-25T00:40:29.064890+0000","last_fullsized":"2024-04-25T00:40:29.064890+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.092463+0000","last_change":"2024-04-25T00:40:29.088835+0000","last_active":"2024-04-25T00:40:29.092463+0000","last_peered":"2024-04-25T00:40:29.092463+0000","last_clean":"2024-04-25T00:40:29.092463+0000","last_became_active":"2024-04-25T00:40:29.088099+0000","last_became_peered":"2024-04-25T00:40:29.088099+0000","last_unstale":"2024-04-25T00:40:29.092463+0000","last_undegraded":"2024-04-25T00:40:29.092463+0000","last_fullsized":"2024-04-25T00:40:29.092463+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:11.663375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.073123+0000","last_change":"2024-04-25T00:40:29.073123+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.073123+0000","last_undegraded":"2024-04-25T00:40:29.073123+0000","last_fullsized":"2024-04-25T00:40:29.073123+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.098666+0000","last_change":"2024-04-25T00:40:29.095787+0000","last_active":"2024-04-25T00:40:29.098666+0000","last_peered":"2024-04-25T00:40:29.098666+0000","last_clean":"2024-04-25T00:40:29.098666+0000","last_became_active":"2024-04-25T00:40:29.094547+0000","last_became_peered":"2024-04-25T00:40:29.094547+0000","last_unstale":"2024-04-25T00:40:29.098666+0000","last_undegraded":"2024-04-25T00:40:29.098666+0000","last_fullsized":"2024-04-25T00:40:29.098666+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:15.747990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068646+0000","last_change":"2024-04-25T00:40:29.068646+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068646+0000","last_undegraded":"2024-04-25T00:40:29.068646+0000","last_fullsized":"2024-04-25T00:40:29.068646+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.096168+0000","last_change":"2024-04-25T00:40:29.095275+0000","last_active":"2024-04-25T00:40:29.096168+0000","last_peered":"2024-04-25T00:40:29.096168+0000","last_clean":"2024-04-25T00:40:29.096168+0000","last_became_active":"2024-04-25T00:40:29.094973+0000","last_became_peered":"2024-04-25T00:40:29.094973+0000","last_unstale":"2024-04-25T00:40:29.096168+0000","last_undegraded":"2024-04-25T00:40:29.096168+0000","last_fullsized":"2024-04-25T00:40:29.096168+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:52.881575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.064353+0000","last_change":"2024-04-25T00:40:29.064353+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.064353+0000","last_undegraded":"2024-04-25T00:40:29.064353+0000","last_fullsized":"2024-04-25T00:40:29.064353+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.101103+0000","last_change":"2024-04-25T00:40:29.099837+0000","last_active":"2024-04-25T00:40:29.101103+0000","last_peered":"2024-04-25T00:40:29.101103+0000","last_clean":"2024-04-25T00:40:29.101103+0000","last_became_active":"2024-04-25T00:40:29.096352+0000","last_became_peered":"2024-04-25T00:40:29.096352+0000","last_unstale":"2024-04-25T00:40:29.101103+0000","last_undegraded":"2024-04-25T00:40:29.101103+0000","last_fullsized":"2024-04-25T00:40:29.101103+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:52.993794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076771+0000","last_change":"2024-04-25T00:40:29.076771+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076771+0000","last_undegraded":"2024-04-25T00:40:29.076771+0000","last_fullsized":"2024-04-25T00:40:29.076771+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.098071+0000","last_change":"2024-04-25T00:40:29.097310+0000","last_active":"2024-04-25T00:40:29.098071+0000","last_peered":"2024-04-25T00:40:29.098071+0000","last_clean":"2024-04-25T00:40:29.098071+0000","last_became_active":"2024-04-25T00:40:29.096881+0000","last_became_peered":"2024-04-25T00:40:29.096881+0000","last_unstale":"2024-04-25T00:40:29.098071+0000","last_undegraded":"2024-04-25T00:40:29.098071+0000","last_fullsized":"2024-04-25T00:40:29.098071+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:25.492182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082449+0000","last_change":"2024-04-25T00:40:29.082449+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082449+0000","last_undegraded":"2024-04-25T00:40:29.082449+0000","last_fullsized":"2024-04-25T00:40:29.082449+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.088718+0000","last_change":"2024-04-25T00:40:29.083696+0000","last_active":"2024-04-25T00:40:29.088718+0000","last_peered":"2024-04-25T00:40:29.088718+0000","last_clean":"2024-04-25T00:40:29.088718+0000","last_became_active":"2024-04-25T00:40:29.082983+0000","last_became_peered":"2024-04-25T00:40:29.082983+0000","last_unstale":"2024-04-25T00:40:29.088718+0000","last_undegraded":"2024-04-25T00:40:29.088718+0000","last_fullsized":"2024-04-25T00:40:29.088718+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:13.703146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072503+0000","last_change":"2024-04-25T00:40:29.072503+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072503+0000","last_undegraded":"2024-04-25T00:40:29.072503+0000","last_fullsized":"2024-04-25T00:40:29.072503+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.099838+0000","last_change":"2024-04-25T00:40:29.098261+0000","last_active":"2024-04-25T00:40:29.099838+0000","last_peered":"2024-04-25T00:40:29.099838+0000","last_clean":"2024-04-25T00:40:29.099838+0000","last_became_active":"2024-04-25T00:40:29.097558+0000","last_became_peered":"2024-04-25T00:40:29.097558+0000","last_unstale":"2024-04-25T00:40:29.099838+0000","last_undegraded":"2024-04-25T00:40:29.099838+0000","last_fullsized":"2024-04-25T00:40:29.099838+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:01.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.070128+0000","last_change":"2024-04-25T00:40:29.070128+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.070128+0000","last_undegraded":"2024-04-25T00:40:29.070128+0000","last_fullsized":"2024-04-25T00:40:29.070128+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.096865+0000","last_change":"2024-04-25T00:40:29.096322+0000","last_active":"2024-04-25T00:40:29.096865+0000","last_peered":"2024-04-25T00:40:29.096865+0000","last_clean":"2024-04-25T00:40:29.096865+0000","last_became_active":"2024-04-25T00:40:29.096034+0000","last_became_peered":"2024-04-25T00:40:29.096034+0000","last_unstale":"2024-04-25T00:40:29.096865+0000","last_undegraded":"2024-04-25T00:40:29.096865+0000","last_fullsized":"2024-04-25T00:40:29.096865+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:53.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067461+0000","last_change":"2024-04-25T00:40:29.067461+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067461+0000","last_undegraded":"2024-04-25T00:40:29.067461+0000","last_fullsized":"2024-04-25T00:40:29.067461+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.093696+0000","last_change":"2024-04-25T00:40:29.090943+0000","last_active":"2024-04-25T00:40:29.093696+0000","last_peered":"2024-04-25T00:40:29.093696+0000","last_clean":"2024-04-25T00:40:29.093696+0000","last_became_active":"2024-04-25T00:40:29.090524+0000","last_became_peered":"2024-04-25T00:40:29.090524+0000","last_unstale":"2024-04-25T00:40:29.093696+0000","last_undegraded":"2024-04-25T00:40:29.093696+0000","last_fullsized":"2024-04-25T00:40:29.093696+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:40.259597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068430+0000","last_change":"2024-04-25T00:40:29.068430+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068430+0000","last_undegraded":"2024-04-25T00:40:29.068430+0000","last_fullsized":"2024-04-25T00:40:29.068430+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.098189+0000","last_change":"2024-04-25T00:40:29.097144+0000","last_active":"2024-04-25T00:40:29.098189+0000","last_peered":"2024-04-25T00:40:29.098189+0000","last_clean":"2024-04-25T00:40:29.098189+0000","last_became_active":"2024-04-25T00:40:29.096558+0000","last_became_peered":"2024-04-25T00:40:29.096558+0000","last_unstale":"2024-04-25T00:40:29.098189+0000","last_undegraded":"2024-04-25T00:40:29.098189+0000","last_fullsized":"2024-04-25T00:40:29.098189+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:40.259597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068589+0000","last_change":"2024-04-25T00:40:29.068589+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068589+0000","last_undegraded":"2024-04-25T00:40:29.068589+0000","last_fullsized":"2024-04-25T00:40:29.068589+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.079341+0000","last_change":"2024-04-25T00:40:29.077975+0000","last_active":"2024-04-25T00:40:29.079341+0000","last_peered":"2024-04-25T00:40:29.079341+0000","last_clean":"2024-04-25T00:40:29.079341+0000","last_became_active":"2024-04-25T00:40:29.077218+0000","last_became_peered":"2024-04-25T00:40:29.077218+0000","last_unstale":"2024-04-25T00:40:29.079341+0000","last_undegraded":"2024-04-25T00:40:29.079341+0000","last_fullsized":"2024-04-25T00:40:29.079341+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:21.386455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068479+0000","last_change":"2024-04-25T00:40:29.068479+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068479+0000","last_undegraded":"2024-04-25T00:40:29.068479+0000","last_fullsized":"2024-04-25T00:40:29.068479+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.098857+0000","last_change":"2024-04-25T00:40:29.096474+0000","last_active":"2024-04-25T00:40:29.098857+0000","last_peered":"2024-04-25T00:40:29.098857+0000","last_clean":"2024-04-25T00:40:29.098857+0000","last_became_active":"2024-04-25T00:40:29.095752+0000","last_became_peered":"2024-04-25T00:40:29.095752+0000","last_unstale":"2024-04-25T00:40:29.098857+0000","last_undegraded":"2024-04-25T00:40:29.098857+0000","last_fullsized":"2024-04-25T00:40:29.098857+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:42.017864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.074826+0000","last_change":"2024-04-25T00:40:29.074826+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.074826+0000","last_undegraded":"2024-04-25T00:40:29.074826+0000","last_fullsized":"2024-04-25T00:40:29.074826+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.099838+0000","last_change":"2024-04-25T00:40:29.084462+0000","last_active":"2024-04-25T00:40:29.099838+0000","last_peered":"2024-04-25T00:40:29.099838+0000","last_clean":"2024-04-25T00:40:29.099838+0000","last_became_active":"2024-04-25T00:40:29.082507+0000","last_became_peered":"2024-04-25T00:40:29.082507+0000","last_unstale":"2024-04-25T00:40:29.099838+0000","last_undegraded":"2024-04-25T00:40:29.099838+0000","last_fullsized":"2024-04-25T00:40:29.099838+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:52.881575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072774+0000","last_change":"2024-04-25T00:40:29.072774+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072774+0000","last_undegraded":"2024-04-25T00:40:29.072774+0000","last_fullsized":"2024-04-25T00:40:29.072774+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.095525+0000","last_change":"2024-04-25T00:40:29.094661+0000","last_active":"2024-04-25T00:40:29.095525+0000","last_peered":"2024-04-25T00:40:29.095525+0000","last_clean":"2024-04-25T00:40:29.095525+0000","last_became_active":"2024-04-25T00:40:29.094039+0000","last_became_peered":"2024-04-25T00:40:29.094039+0000","last_unstale":"2024-04-25T00:40:29.095525+0000","last_undegraded":"2024-04-25T00:40:29.095525+0000","last_fullsized":"2024-04-25T00:40:29.095525+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:49.527153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067669+0000","last_change":"2024-04-25T00:40:29.067669+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067669+0000","last_undegraded":"2024-04-25T00:40:29.067669+0000","last_fullsized":"2024-04-25T00:40:29.067669+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.102744+0000","last_change":"2024-04-25T00:40:29.101828+0000","last_active":"2024-04-25T00:40:29.102744+0000","last_peered":"2024-04-25T00:40:29.102744+0000","last_clean":"2024-04-25T00:40:29.102744+0000","last_became_active":"2024-04-25T00:40:29.101030+0000","last_became_peered":"2024-04-25T00:40:29.101030+0000","last_unstale":"2024-04-25T00:40:29.102744+0000","last_undegraded":"2024-04-25T00:40:29.102744+0000","last_fullsized":"2024-04-25T00:40:29.102744+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:08.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":462,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.070740+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:40:29.070740+0000","last_peered":"2024-04-25T00:40:29.070740+0000","last_clean":"2024-04-25T00:40:29.070740+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:40:29.070740+0000","last_undegraded":"2024-04-25T00:40:29.070740+0000","last_fullsized":"2024-04-25T00:40:29.070740+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574960,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37808,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334032,"statfs":{"total":96636764160,"available":96598048768,"internally_reserved":0,"allocated":2523136,"data_stored":1440265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574961,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1941504,"data_stored":856172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41388,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330452,"statfs":{"total":96636764160,"available":96594382848,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37232,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574960,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41968,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329872,"statfs":{"total":96636764160,"available":96593788928,"internally_reserved":0,"allocated":2523136,"data_stored":1440265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:40:34.495 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:40:34.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff787257640 1 Processor -- start 2024-04-25T00:40:34.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff787257640 1 -- start start 2024-04-25T00:40:34.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff787257640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800b3140 0x7ff78006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff787257640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff7800b4b10 0x7ff7800b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff787257640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff7800b3a60 0x7ff78015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff787257640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff7800b41e0 con 0x7ff7800b4b10 2024-04-25T00:40:34.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff787257640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff78006d960 con 0x7ff7800b3a60 2024-04-25T00:40:34.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff787257640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff78006dc40 con 0x7ff7800b3140 2024-04-25T00:40:34.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff77ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff7800b4b10 0x7ff7800b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff77ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff7800b4b10 0x7ff7800b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59988/0 (socket says 172.21.15.29:59988) 2024-04-25T00:40:34.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff77ffff640 1 -- 172.21.15.29:0/3148936743 learned_addr learned my addr 172.21.15.29:0/3148936743 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:34.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff7857f4640 1 --2- 172.21.15.29:0/3148936743 >> v2:172.21.15.157:3300/0 conn(0x7ff7800b3a60 0x7ff78015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff784ff3640 1 --2- 172.21.15.29:0/3148936743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800b3140 0x7ff78006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff77ffff640 1 -- 172.21.15.29:0/3148936743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800b3140 msgr2=0x7ff78006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff77ffff640 1 --2- 172.21.15.29:0/3148936743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800b3140 0x7ff78006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff77ffff640 1 -- 172.21.15.29:0/3148936743 >> v2:172.21.15.157:3300/0 conn(0x7ff7800b3a60 msgr2=0x7ff78015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff77ffff640 1 --2- 172.21.15.29:0/3148936743 >> v2:172.21.15.157:3300/0 conn(0x7ff7800b3a60 0x7ff78015c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.657+0000 7ff77ffff640 1 -- 172.21.15.29:0/3148936743 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78006df20 con 0x7ff7800b4b10 2024-04-25T00:40:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff77ffff640 1 --2- 172.21.15.29:0/3148936743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7800b4b10 0x7ff7800b3520 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7ff7800b4790 tx=0x7ff77001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c7e10113edf910 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff77f7fe640 1 -- 172.21.15.29:0/3148936743 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7700b0070 con 0x7ff7800b4b10 2024-04-25T00:40:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff77f7fe640 1 -- 172.21.15.29:0/3148936743 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7700ab600 con 0x7ff7800b4b10 2024-04-25T00:40:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff77f7fe640 1 -- 172.21.15.29:0/3148936743 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7700ab9d0 con 0x7ff7800b4b10 2024-04-25T00:40:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 -- 172.21.15.29:0/3148936743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7800b4b10 msgr2=0x7ff7800b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 --2- 172.21.15.29:0/3148936743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7800b4b10 0x7ff7800b3520 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7ff7800b4790 tx=0x7ff77001c520 comp rx=0 tx=0).stop 2024-04-25T00:40:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 -- 172.21.15.29:0/3148936743 shutdown_connections 2024-04-25T00:40:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 --2- 172.21.15.29:0/3148936743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7800b4b10 0x7ff7800b3520 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 --2- 172.21.15.29:0/3148936743 >> v2:172.21.15.157:3300/0 conn(0x7ff7800b3a60 0x7ff78015c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 --2- 172.21.15.29:0/3148936743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800b3140 0x7ff78006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 -- 172.21.15.29:0/3148936743 >> 172.21.15.29:0/3148936743 conn(0x7ff7800b1870 msgr2=0x7ff7800b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 -- 172.21.15.29:0/3148936743 shutdown_connections 2024-04-25T00:40:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 -- 172.21.15.29:0/3148936743 wait complete. 2024-04-25T00:40:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 Processor -- start 2024-04-25T00:40:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 -- start start 2024-04-25T00:40:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7800b3140 0x7ff780152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7800b3a60 0x7ff780139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800b4b10 0x7ff780071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff7800ce3c0 con 0x7ff7800b3140 2024-04-25T00:40:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff7800ce240 con 0x7ff7800b3a60 2024-04-25T00:40:34.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff787257640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff7800ce540 con 0x7ff7800b4b10 2024-04-25T00:40:34.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff784ff3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7800b3140 0x7ff780152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff784ff3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7800b3140 0x7ff780152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59990/0 (socket says 172.21.15.29:59990) 2024-04-25T00:40:34.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff784ff3640 1 -- 172.21.15.29:0/475562325 learned_addr learned my addr 172.21.15.29:0/475562325 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:34.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff77ffff640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7800b3a60 0x7ff780139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.661+0000 7ff7857f4640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800b4b10 0x7ff780071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff784ff3640 1 -- 172.21.15.29:0/475562325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800b4b10 msgr2=0x7ff780071cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff784ff3640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800b4b10 0x7ff780071cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff784ff3640 1 -- 172.21.15.29:0/475562325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7800b3a60 msgr2=0x7ff780139ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff784ff3640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7800b3a60 0x7ff780139ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff784ff3640 1 -- 172.21.15.29:0/475562325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff780072c40 con 0x7ff7800b3140 2024-04-25T00:40:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff784ff3640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7800b3140 0x7ff780152900 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7ff77400d800 tx=0x7ff77400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff77d7fa640 1 -- 172.21.15.29:0/475562325 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774018020 con 0x7ff7800b3140 2024-04-25T00:40:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff77d7fa640 1 -- 172.21.15.29:0/475562325 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7740096c0 con 0x7ff7800b3140 2024-04-25T00:40:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff787257640 1 -- 172.21.15.29:0/475562325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff780157d20 con 0x7ff7800b3140 2024-04-25T00:40:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff77d7fa640 1 -- 172.21.15.29:0/475562325 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774009ab0 con 0x7ff7800b3140 2024-04-25T00:40:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff787257640 1 -- 172.21.15.29:0/475562325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff780071790 con 0x7ff7800b3140 2024-04-25T00:40:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff787257640 1 -- 172.21.15.29:0/475562325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff780069270 con 0x7ff7800b3140 2024-04-25T00:40:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff77d7fa640 1 -- 172.21.15.29:0/475562325 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7ff77401b020 con 0x7ff7800b3140 2024-04-25T00:40:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff77d7fa640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff76c0680a0 0x7ff76c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff77d7fa640 1 -- 172.21.15.29:0/475562325 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7ff774087530 con 0x7ff7800b3140 2024-04-25T00:40:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff77ffff640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff76c0680a0 0x7ff76c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.665+0000 7ff77ffff640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff76c0680a0 0x7ff76c06a560 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff7700ad000 tx=0x7ff7700a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.669+0000 7ff77d7fa640 1 -- 172.21.15.29:0/475562325 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff774022020 con 0x7ff7800b3140 2024-04-25T00:40:34.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.829+0000 7ff787257640 1 -- 172.21.15.29:0/475562325 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff7800719d0 con 0x7ff76c0680a0 2024-04-25T00:40:34.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.845+0000 7ff77d7fa640 1 -- 172.21.15.29:0/475562325 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247088 (secure 0 0 0) 0x7ff7700ab480 con 0x7ff76c0680a0 2024-04-25T00:40:34.847 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:34.848 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:40:34.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.849+0000 7ff787257640 1 -- 172.21.15.29:0/475562325 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff76c0680a0 msgr2=0x7ff76c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.849+0000 7ff787257640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff76c0680a0 0x7ff76c06a560 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff7700ad000 tx=0x7ff7700a6000 comp rx=0 tx=0).stop 2024-04-25T00:40:34.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.849+0000 7ff787257640 1 -- 172.21.15.29:0/475562325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7800b3140 msgr2=0x7ff780152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.849+0000 7ff787257640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7800b3140 0x7ff780152900 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7ff77400d800 tx=0x7ff77400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.849+0000 7ff787257640 1 -- 172.21.15.29:0/475562325 shutdown_connections 2024-04-25T00:40:34.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.849+0000 7ff787257640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff76c0680a0 0x7ff76c06a560 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.849+0000 7ff787257640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7800b4b10 0x7ff780071cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.849+0000 7ff787257640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7800b3a60 0x7ff780139ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.849+0000 7ff787257640 1 --2- 172.21.15.29:0/475562325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7800b3140 0x7ff780152900 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.849+0000 7ff787257640 1 -- 172.21.15.29:0/475562325 >> 172.21.15.29:0/475562325 conn(0x7ff7800b1870 msgr2=0x7ff78006cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.849+0000 7ff787257640 1 -- 172.21.15.29:0/475562325 shutdown_connections 2024-04-25T00:40:34.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:34.849+0000 7ff787257640 1 -- 172.21.15.29:0/475562325 wait complete. 2024-04-25T00:40:34.866 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-04-25T00:40:33.405476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307328,"kb_used_data":16244,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667392,"statfs":{"total":773094113280,"available":772779409408,"internally_reserved":0,"allocated":16633856,"data_stored":7986187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.350831"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068067+0000","last_change":"2024-04-25T00:40:29.068067+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068067+0000","last_undegraded":"2024-04-25T00:40:29.068067+0000","last_fullsized":"2024-04-25T00:40:29.068067+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.077656+0000","last_change":"2024-04-25T00:40:29.077656+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.077656+0000","last_undegraded":"2024-04-25T00:40:29.077656+0000","last_fullsized":"2024-04-25T00:40:29.077656+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.085806+0000","last_change":"2024-04-25T00:40:29.085806+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.085806+0000","last_undegraded":"2024-04-25T00:40:29.085806+0000","last_fullsized":"2024-04-25T00:40:29.085806+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076453+0000","last_change":"2024-04-25T00:40:29.076453+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076453+0000","last_undegraded":"2024-04-25T00:40:29.076453+0000","last_fullsized":"2024-04-25T00:40:29.076453+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.083171+0000","last_change":"2024-04-25T00:40:29.083171+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.083171+0000","last_undegraded":"2024-04-25T00:40:29.083171+0000","last_fullsized":"2024-04-25T00:40:29.083171+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076571+0000","last_change":"2024-04-25T00:40:29.076571+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076571+0000","last_undegraded":"2024-04-25T00:40:29.076571+0000","last_fullsized":"2024-04-25T00:40:29.076571+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067060+0000","last_change":"2024-04-25T00:40:29.067060+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067060+0000","last_undegraded":"2024-04-25T00:40:29.067060+0000","last_fullsized":"2024-04-25T00:40:29.067060+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.078988+0000","last_change":"2024-04-25T00:40:29.078988+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.078988+0000","last_undegraded":"2024-04-25T00:40:29.078988+0000","last_fullsized":"2024-04-25T00:40:29.078988+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.066446+0000","last_change":"2024-04-25T00:40:29.066446+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.066446+0000","last_undegraded":"2024-04-25T00:40:29.066446+0000","last_fullsized":"2024-04-25T00:40:29.066446+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.071624+0000","last_change":"2024-04-25T00:40:29.071624+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.071624+0000","last_undegraded":"2024-04-25T00:40:29.071624+0000","last_fullsized":"2024-04-25T00:40:29.071624+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065659+0000","last_change":"2024-04-25T00:40:29.065659+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065659+0000","last_undegraded":"2024-04-25T00:40:29.065659+0000","last_fullsized":"2024-04-25T00:40:29.065659+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.075405+0000","last_change":"2024-04-25T00:40:29.075405+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.075405+0000","last_undegraded":"2024-04-25T00:40:29.075405+0000","last_fullsized":"2024-04-25T00:40:29.075405+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.079997+0000","last_change":"2024-04-25T00:40:29.079997+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.079997+0000","last_undegraded":"2024-04-25T00:40:29.079997+0000","last_fullsized":"2024-04-25T00:40:29.079997+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067927+0000","last_change":"2024-04-25T00:40:29.067927+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067927+0000","last_undegraded":"2024-04-25T00:40:29.067927+0000","last_fullsized":"2024-04-25T00:40:29.067927+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.080999+0000","last_change":"2024-04-25T00:40:29.080999+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.080999+0000","last_undegraded":"2024-04-25T00:40:29.080999+0000","last_fullsized":"2024-04-25T00:40:29.080999+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.078469+0000","last_change":"2024-04-25T00:40:29.078469+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.078469+0000","last_undegraded":"2024-04-25T00:40:29.078469+0000","last_fullsized":"2024-04-25T00:40:29.078469+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082514+0000","last_change":"2024-04-25T00:40:29.082514+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082514+0000","last_undegraded":"2024-04-25T00:40:29.082514+0000","last_fullsized":"2024-04-25T00:40:29.082514+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068120+0000","last_change":"2024-04-25T00:40:29.068120+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068120+0000","last_undegraded":"2024-04-25T00:40:29.068120+0000","last_fullsized":"2024-04-25T00:40:29.068120+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.069605+0000","last_change":"2024-04-25T00:40:29.069605+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.069605+0000","last_undegraded":"2024-04-25T00:40:29.069605+0000","last_fullsized":"2024-04-25T00:40:29.069605+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072609+0000","last_change":"2024-04-25T00:40:29.072609+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072609+0000","last_undegraded":"2024-04-25T00:40:29.072609+0000","last_fullsized":"2024-04-25T00:40:29.072609+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082713+0000","last_change":"2024-04-25T00:40:29.082713+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082713+0000","last_undegraded":"2024-04-25T00:40:29.082713+0000","last_fullsized":"2024-04-25T00:40:29.082713+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.069538+0000","last_change":"2024-04-25T00:40:29.069538+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.069538+0000","last_undegraded":"2024-04-25T00:40:29.069538+0000","last_fullsized":"2024-04-25T00:40:29.069538+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.085695+0000","last_change":"2024-04-25T00:40:29.085695+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.085695+0000","last_undegraded":"2024-04-25T00:40:29.085695+0000","last_fullsized":"2024-04-25T00:40:29.085695+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.077654+0000","last_change":"2024-04-25T00:40:29.077654+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.077654+0000","last_undegraded":"2024-04-25T00:40:29.077654+0000","last_fullsized":"2024-04-25T00:40:29.077654+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.073015+0000","last_change":"2024-04-25T00:40:29.073015+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.073015+0000","last_undegraded":"2024-04-25T00:40:29.073015+0000","last_fullsized":"2024-04-25T00:40:29.073015+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065060+0000","last_change":"2024-04-25T00:40:29.065060+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065060+0000","last_undegraded":"2024-04-25T00:40:29.065060+0000","last_fullsized":"2024-04-25T00:40:29.065060+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082493+0000","last_change":"2024-04-25T00:40:29.082493+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082493+0000","last_undegraded":"2024-04-25T00:40:29.082493+0000","last_fullsized":"2024-04-25T00:40:29.082493+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072143+0000","last_change":"2024-04-25T00:40:29.072143+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072143+0000","last_undegraded":"2024-04-25T00:40:29.072143+0000","last_fullsized":"2024-04-25T00:40:29.072143+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072837+0000","last_change":"2024-04-25T00:40:29.072837+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072837+0000","last_undegraded":"2024-04-25T00:40:29.072837+0000","last_fullsized":"2024-04-25T00:40:29.072837+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065859+0000","last_change":"2024-04-25T00:40:29.065859+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065859+0000","last_undegraded":"2024-04-25T00:40:29.065859+0000","last_fullsized":"2024-04-25T00:40:29.065859+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082748+0000","last_change":"2024-04-25T00:40:29.082748+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082748+0000","last_undegraded":"2024-04-25T00:40:29.082748+0000","last_fullsized":"2024-04-25T00:40:29.082748+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.071805+0000","last_change":"2024-04-25T00:40:29.071805+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.071805+0000","last_undegraded":"2024-04-25T00:40:29.071805+0000","last_fullsized":"2024-04-25T00:40:29.071805+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.074871+0000","last_change":"2024-04-25T00:40:29.074871+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.074871+0000","last_undegraded":"2024-04-25T00:40:29.074871+0000","last_fullsized":"2024-04-25T00:40:29.074871+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.100723+0000","last_change":"2024-04-25T00:40:29.094752+0000","last_active":"2024-04-25T00:40:29.100723+0000","last_peered":"2024-04-25T00:40:29.100723+0000","last_clean":"2024-04-25T00:40:29.100723+0000","last_became_active":"2024-04-25T00:40:29.094472+0000","last_became_peered":"2024-04-25T00:40:29.094472+0000","last_unstale":"2024-04-25T00:40:29.100723+0000","last_undegraded":"2024-04-25T00:40:29.100723+0000","last_fullsized":"2024-04-25T00:40:29.100723+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:59.895661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.070618+0000","last_change":"2024-04-25T00:40:29.070618+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.070618+0000","last_undegraded":"2024-04-25T00:40:29.070618+0000","last_fullsized":"2024-04-25T00:40:29.070618+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.088790+0000","last_change":"2024-04-25T00:40:29.085706+0000","last_active":"2024-04-25T00:40:29.088790+0000","last_peered":"2024-04-25T00:40:29.088790+0000","last_clean":"2024-04-25T00:40:29.088790+0000","last_became_active":"2024-04-25T00:40:29.084970+0000","last_became_peered":"2024-04-25T00:40:29.084970+0000","last_unstale":"2024-04-25T00:40:29.088790+0000","last_undegraded":"2024-04-25T00:40:29.088790+0000","last_fullsized":"2024-04-25T00:40:29.088790+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:57.516592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076415+0000","last_change":"2024-04-25T00:40:29.076415+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076415+0000","last_undegraded":"2024-04-25T00:40:29.076415+0000","last_fullsized":"2024-04-25T00:40:29.076415+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.099678+0000","last_change":"2024-04-25T00:40:29.098108+0000","last_active":"2024-04-25T00:40:29.099678+0000","last_peered":"2024-04-25T00:40:29.099678+0000","last_clean":"2024-04-25T00:40:29.099678+0000","last_became_active":"2024-04-25T00:40:29.097478+0000","last_became_peered":"2024-04-25T00:40:29.097478+0000","last_unstale":"2024-04-25T00:40:29.099678+0000","last_undegraded":"2024-04-25T00:40:29.099678+0000","last_fullsized":"2024-04-25T00:40:29.099678+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:14.983065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067036+0000","last_change":"2024-04-25T00:40:29.067036+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067036+0000","last_undegraded":"2024-04-25T00:40:29.067036+0000","last_fullsized":"2024-04-25T00:40:29.067036+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.101617+0000","last_change":"2024-04-25T00:40:29.096877+0000","last_active":"2024-04-25T00:40:29.101617+0000","last_peered":"2024-04-25T00:40:29.101617+0000","last_clean":"2024-04-25T00:40:29.101617+0000","last_became_active":"2024-04-25T00:40:29.096218+0000","last_became_peered":"2024-04-25T00:40:29.096218+0000","last_unstale":"2024-04-25T00:40:29.101617+0000","last_undegraded":"2024-04-25T00:40:29.101617+0000","last_fullsized":"2024-04-25T00:40:29.101617+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:25.492182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.069078+0000","last_change":"2024-04-25T00:40:29.069078+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.069078+0000","last_undegraded":"2024-04-25T00:40:29.069078+0000","last_fullsized":"2024-04-25T00:40:29.069078+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.095362+0000","last_change":"2024-04-25T00:40:29.087901+0000","last_active":"2024-04-25T00:40:29.095362+0000","last_peered":"2024-04-25T00:40:29.095362+0000","last_clean":"2024-04-25T00:40:29.095362+0000","last_became_active":"2024-04-25T00:40:29.087264+0000","last_became_peered":"2024-04-25T00:40:29.087264+0000","last_unstale":"2024-04-25T00:40:29.095362+0000","last_undegraded":"2024-04-25T00:40:29.095362+0000","last_fullsized":"2024-04-25T00:40:29.095362+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:56.100853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.074690+0000","last_change":"2024-04-25T00:40:29.074690+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.074690+0000","last_undegraded":"2024-04-25T00:40:29.074690+0000","last_fullsized":"2024-04-25T00:40:29.074690+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.097916+0000","last_change":"2024-04-25T00:40:29.096769+0000","last_active":"2024-04-25T00:40:29.097916+0000","last_peered":"2024-04-25T00:40:29.097916+0000","last_clean":"2024-04-25T00:40:29.097916+0000","last_became_active":"2024-04-25T00:40:29.096339+0000","last_became_peered":"2024-04-25T00:40:29.096339+0000","last_unstale":"2024-04-25T00:40:29.097916+0000","last_undegraded":"2024-04-25T00:40:29.097916+0000","last_fullsized":"2024-04-25T00:40:29.097916+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.904588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.064376+0000","last_change":"2024-04-25T00:40:29.064376+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.064376+0000","last_undegraded":"2024-04-25T00:40:29.064376+0000","last_fullsized":"2024-04-25T00:40:29.064376+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.090108+0000","last_change":"2024-04-25T00:40:29.082537+0000","last_active":"2024-04-25T00:40:29.090108+0000","last_peered":"2024-04-25T00:40:29.090108+0000","last_clean":"2024-04-25T00:40:29.090108+0000","last_became_active":"2024-04-25T00:40:29.081768+0000","last_became_peered":"2024-04-25T00:40:29.081768+0000","last_unstale":"2024-04-25T00:40:29.090108+0000","last_undegraded":"2024-04-25T00:40:29.090108+0000","last_fullsized":"2024-04-25T00:40:29.090108+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:23.222465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076804+0000","last_change":"2024-04-25T00:40:29.076804+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076804+0000","last_undegraded":"2024-04-25T00:40:29.076804+0000","last_fullsized":"2024-04-25T00:40:29.076804+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.097546+0000","last_change":"2024-04-25T00:40:29.091407+0000","last_active":"2024-04-25T00:40:29.097546+0000","last_peered":"2024-04-25T00:40:29.097546+0000","last_clean":"2024-04-25T00:40:29.097546+0000","last_became_active":"2024-04-25T00:40:29.090686+0000","last_became_peered":"2024-04-25T00:40:29.090686+0000","last_unstale":"2024-04-25T00:40:29.097546+0000","last_undegraded":"2024-04-25T00:40:29.097546+0000","last_fullsized":"2024-04-25T00:40:29.097546+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:43.160214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068208+0000","last_change":"2024-04-25T00:40:29.068208+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068208+0000","last_undegraded":"2024-04-25T00:40:29.068208+0000","last_fullsized":"2024-04-25T00:40:29.068208+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.101528+0000","last_change":"2024-04-25T00:40:29.100639+0000","last_active":"2024-04-25T00:40:29.101528+0000","last_peered":"2024-04-25T00:40:29.101528+0000","last_clean":"2024-04-25T00:40:29.101528+0000","last_became_active":"2024-04-25T00:40:29.100348+0000","last_became_peered":"2024-04-25T00:40:29.100348+0000","last_unstale":"2024-04-25T00:40:29.101528+0000","last_undegraded":"2024-04-25T00:40:29.101528+0000","last_fullsized":"2024-04-25T00:40:29.101528+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:28.420738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067576+0000","last_change":"2024-04-25T00:40:29.067576+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067576+0000","last_undegraded":"2024-04-25T00:40:29.067576+0000","last_fullsized":"2024-04-25T00:40:29.067576+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.095407+0000","last_change":"2024-04-25T00:40:29.088110+0000","last_active":"2024-04-25T00:40:29.095407+0000","last_peered":"2024-04-25T00:40:29.095407+0000","last_clean":"2024-04-25T00:40:29.095407+0000","last_became_active":"2024-04-25T00:40:29.087595+0000","last_became_peered":"2024-04-25T00:40:29.087595+0000","last_unstale":"2024-04-25T00:40:29.095407+0000","last_undegraded":"2024-04-25T00:40:29.095407+0000","last_fullsized":"2024-04-25T00:40:29.095407+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:42.221346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.069149+0000","last_change":"2024-04-25T00:40:29.069149+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.069149+0000","last_undegraded":"2024-04-25T00:40:29.069149+0000","last_fullsized":"2024-04-25T00:40:29.069149+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.100608+0000","last_change":"2024-04-25T00:40:29.099633+0000","last_active":"2024-04-25T00:40:29.100608+0000","last_peered":"2024-04-25T00:40:29.100608+0000","last_clean":"2024-04-25T00:40:29.100608+0000","last_became_active":"2024-04-25T00:40:29.099060+0000","last_became_peered":"2024-04-25T00:40:29.099060+0000","last_unstale":"2024-04-25T00:40:29.100608+0000","last_undegraded":"2024-04-25T00:40:29.100608+0000","last_fullsized":"2024-04-25T00:40:29.100608+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:24.734293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076030+0000","last_change":"2024-04-25T00:40:29.076030+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076030+0000","last_undegraded":"2024-04-25T00:40:29.076030+0000","last_fullsized":"2024-04-25T00:40:29.076030+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.092384+0000","last_change":"2024-04-25T00:40:29.088653+0000","last_active":"2024-04-25T00:40:29.092384+0000","last_peered":"2024-04-25T00:40:29.092384+0000","last_clean":"2024-04-25T00:40:29.092384+0000","last_became_active":"2024-04-25T00:40:29.087923+0000","last_became_peered":"2024-04-25T00:40:29.087923+0000","last_unstale":"2024-04-25T00:40:29.092384+0000","last_undegraded":"2024-04-25T00:40:29.092384+0000","last_fullsized":"2024-04-25T00:40:29.092384+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:13.385254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065511+0000","last_change":"2024-04-25T00:40:29.065511+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065511+0000","last_undegraded":"2024-04-25T00:40:29.065511+0000","last_fullsized":"2024-04-25T00:40:29.065511+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.093663+0000","last_change":"2024-04-25T00:40:29.088591+0000","last_active":"2024-04-25T00:40:29.093663+0000","last_peered":"2024-04-25T00:40:29.093663+0000","last_clean":"2024-04-25T00:40:29.093663+0000","last_became_active":"2024-04-25T00:40:29.088270+0000","last_became_peered":"2024-04-25T00:40:29.088270+0000","last_unstale":"2024-04-25T00:40:29.093663+0000","last_undegraded":"2024-04-25T00:40:29.093663+0000","last_fullsized":"2024-04-25T00:40:29.093663+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.904588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.064117+0000","last_change":"2024-04-25T00:40:29.064117+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.064117+0000","last_undegraded":"2024-04-25T00:40:29.064117+0000","last_fullsized":"2024-04-25T00:40:29.064117+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.101315+0000","last_change":"2024-04-25T00:40:29.100762+0000","last_active":"2024-04-25T00:40:29.101315+0000","last_peered":"2024-04-25T00:40:29.101315+0000","last_clean":"2024-04-25T00:40:29.101315+0000","last_became_active":"2024-04-25T00:40:29.100159+0000","last_became_peered":"2024-04-25T00:40:29.100159+0000","last_unstale":"2024-04-25T00:40:29.101315+0000","last_undegraded":"2024-04-25T00:40:29.101315+0000","last_fullsized":"2024-04-25T00:40:29.101315+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:53.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.071314+0000","last_change":"2024-04-25T00:40:29.071314+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.071314+0000","last_undegraded":"2024-04-25T00:40:29.071314+0000","last_fullsized":"2024-04-25T00:40:29.071314+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.092551+0000","last_change":"2024-04-25T00:40:29.089205+0000","last_active":"2024-04-25T00:40:29.092551+0000","last_peered":"2024-04-25T00:40:29.092551+0000","last_clean":"2024-04-25T00:40:29.092551+0000","last_became_active":"2024-04-25T00:40:29.088497+0000","last_became_peered":"2024-04-25T00:40:29.088497+0000","last_unstale":"2024-04-25T00:40:29.092551+0000","last_undegraded":"2024-04-25T00:40:29.092551+0000","last_fullsized":"2024-04-25T00:40:29.092551+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:38.339737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.071161+0000","last_change":"2024-04-25T00:40:29.071161+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.071161+0000","last_undegraded":"2024-04-25T00:40:29.071161+0000","last_fullsized":"2024-04-25T00:40:29.071161+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.095713+0000","last_change":"2024-04-25T00:40:29.093336+0000","last_active":"2024-04-25T00:40:29.095713+0000","last_peered":"2024-04-25T00:40:29.095713+0000","last_clean":"2024-04-25T00:40:29.095713+0000","last_became_active":"2024-04-25T00:40:29.092530+0000","last_became_peered":"2024-04-25T00:40:29.092530+0000","last_unstale":"2024-04-25T00:40:29.095713+0000","last_undegraded":"2024-04-25T00:40:29.095713+0000","last_fullsized":"2024-04-25T00:40:29.095713+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:53.498894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.065433+0000","last_change":"2024-04-25T00:40:29.065433+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.065433+0000","last_undegraded":"2024-04-25T00:40:29.065433+0000","last_fullsized":"2024-04-25T00:40:29.065433+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.091444+0000","last_change":"2024-04-25T00:40:29.090555+0000","last_active":"2024-04-25T00:40:29.091444+0000","last_peered":"2024-04-25T00:40:29.091444+0000","last_clean":"2024-04-25T00:40:29.091444+0000","last_became_active":"2024-04-25T00:40:29.089983+0000","last_became_peered":"2024-04-25T00:40:29.089983+0000","last_unstale":"2024-04-25T00:40:29.091444+0000","last_undegraded":"2024-04-25T00:40:29.091444+0000","last_fullsized":"2024-04-25T00:40:29.091444+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:31.202069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.064890+0000","last_change":"2024-04-25T00:40:29.064890+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.064890+0000","last_undegraded":"2024-04-25T00:40:29.064890+0000","last_fullsized":"2024-04-25T00:40:29.064890+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.092463+0000","last_change":"2024-04-25T00:40:29.088835+0000","last_active":"2024-04-25T00:40:29.092463+0000","last_peered":"2024-04-25T00:40:29.092463+0000","last_clean":"2024-04-25T00:40:29.092463+0000","last_became_active":"2024-04-25T00:40:29.088099+0000","last_became_peered":"2024-04-25T00:40:29.088099+0000","last_unstale":"2024-04-25T00:40:29.092463+0000","last_undegraded":"2024-04-25T00:40:29.092463+0000","last_fullsized":"2024-04-25T00:40:29.092463+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:11.663375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.073123+0000","last_change":"2024-04-25T00:40:29.073123+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.073123+0000","last_undegraded":"2024-04-25T00:40:29.073123+0000","last_fullsized":"2024-04-25T00:40:29.073123+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.098666+0000","last_change":"2024-04-25T00:40:29.095787+0000","last_active":"2024-04-25T00:40:29.098666+0000","last_peered":"2024-04-25T00:40:29.098666+0000","last_clean":"2024-04-25T00:40:29.098666+0000","last_became_active":"2024-04-25T00:40:29.094547+0000","last_became_peered":"2024-04-25T00:40:29.094547+0000","last_unstale":"2024-04-25T00:40:29.098666+0000","last_undegraded":"2024-04-25T00:40:29.098666+0000","last_fullsized":"2024-04-25T00:40:29.098666+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:15.747990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068646+0000","last_change":"2024-04-25T00:40:29.068646+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068646+0000","last_undegraded":"2024-04-25T00:40:29.068646+0000","last_fullsized":"2024-04-25T00:40:29.068646+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.096168+0000","last_change":"2024-04-25T00:40:29.095275+0000","last_active":"2024-04-25T00:40:29.096168+0000","last_peered":"2024-04-25T00:40:29.096168+0000","last_clean":"2024-04-25T00:40:29.096168+0000","last_became_active":"2024-04-25T00:40:29.094973+0000","last_became_peered":"2024-04-25T00:40:29.094973+0000","last_unstale":"2024-04-25T00:40:29.096168+0000","last_undegraded":"2024-04-25T00:40:29.096168+0000","last_fullsized":"2024-04-25T00:40:29.096168+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:52.881575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.064353+0000","last_change":"2024-04-25T00:40:29.064353+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.064353+0000","last_undegraded":"2024-04-25T00:40:29.064353+0000","last_fullsized":"2024-04-25T00:40:29.064353+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.101103+0000","last_change":"2024-04-25T00:40:29.099837+0000","last_active":"2024-04-25T00:40:29.101103+0000","last_peered":"2024-04-25T00:40:29.101103+0000","last_clean":"2024-04-25T00:40:29.101103+0000","last_became_active":"2024-04-25T00:40:29.096352+0000","last_became_peered":"2024-04-25T00:40:29.096352+0000","last_unstale":"2024-04-25T00:40:29.101103+0000","last_undegraded":"2024-04-25T00:40:29.101103+0000","last_fullsized":"2024-04-25T00:40:29.101103+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:52.993794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.076771+0000","last_change":"2024-04-25T00:40:29.076771+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.076771+0000","last_undegraded":"2024-04-25T00:40:29.076771+0000","last_fullsized":"2024-04-25T00:40:29.076771+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.098071+0000","last_change":"2024-04-25T00:40:29.097310+0000","last_active":"2024-04-25T00:40:29.098071+0000","last_peered":"2024-04-25T00:40:29.098071+0000","last_clean":"2024-04-25T00:40:29.098071+0000","last_became_active":"2024-04-25T00:40:29.096881+0000","last_became_peered":"2024-04-25T00:40:29.096881+0000","last_unstale":"2024-04-25T00:40:29.098071+0000","last_undegraded":"2024-04-25T00:40:29.098071+0000","last_fullsized":"2024-04-25T00:40:29.098071+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:25.492182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.082449+0000","last_change":"2024-04-25T00:40:29.082449+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.082449+0000","last_undegraded":"2024-04-25T00:40:29.082449+0000","last_fullsized":"2024-04-25T00:40:29.082449+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.088718+0000","last_change":"2024-04-25T00:40:29.083696+0000","last_active":"2024-04-25T00:40:29.088718+0000","last_peered":"2024-04-25T00:40:29.088718+0000","last_clean":"2024-04-25T00:40:29.088718+0000","last_became_active":"2024-04-25T00:40:29.082983+0000","last_became_peered":"2024-04-25T00:40:29.082983+0000","last_unstale":"2024-04-25T00:40:29.088718+0000","last_undegraded":"2024-04-25T00:40:29.088718+0000","last_fullsized":"2024-04-25T00:40:29.088718+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:13.703146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072503+0000","last_change":"2024-04-25T00:40:29.072503+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072503+0000","last_undegraded":"2024-04-25T00:40:29.072503+0000","last_fullsized":"2024-04-25T00:40:29.072503+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.099838+0000","last_change":"2024-04-25T00:40:29.098261+0000","last_active":"2024-04-25T00:40:29.099838+0000","last_peered":"2024-04-25T00:40:29.099838+0000","last_clean":"2024-04-25T00:40:29.099838+0000","last_became_active":"2024-04-25T00:40:29.097558+0000","last_became_peered":"2024-04-25T00:40:29.097558+0000","last_unstale":"2024-04-25T00:40:29.099838+0000","last_undegraded":"2024-04-25T00:40:29.099838+0000","last_fullsized":"2024-04-25T00:40:29.099838+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:01.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.070128+0000","last_change":"2024-04-25T00:40:29.070128+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.070128+0000","last_undegraded":"2024-04-25T00:40:29.070128+0000","last_fullsized":"2024-04-25T00:40:29.070128+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.096865+0000","last_change":"2024-04-25T00:40:29.096322+0000","last_active":"2024-04-25T00:40:29.096865+0000","last_peered":"2024-04-25T00:40:29.096865+0000","last_clean":"2024-04-25T00:40:29.096865+0000","last_became_active":"2024-04-25T00:40:29.096034+0000","last_became_peered":"2024-04-25T00:40:29.096034+0000","last_unstale":"2024-04-25T00:40:29.096865+0000","last_undegraded":"2024-04-25T00:40:29.096865+0000","last_fullsized":"2024-04-25T00:40:29.096865+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:53.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067461+0000","last_change":"2024-04-25T00:40:29.067461+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067461+0000","last_undegraded":"2024-04-25T00:40:29.067461+0000","last_fullsized":"2024-04-25T00:40:29.067461+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.093696+0000","last_change":"2024-04-25T00:40:29.090943+0000","last_active":"2024-04-25T00:40:29.093696+0000","last_peered":"2024-04-25T00:40:29.093696+0000","last_clean":"2024-04-25T00:40:29.093696+0000","last_became_active":"2024-04-25T00:40:29.090524+0000","last_became_peered":"2024-04-25T00:40:29.090524+0000","last_unstale":"2024-04-25T00:40:29.093696+0000","last_undegraded":"2024-04-25T00:40:29.093696+0000","last_fullsized":"2024-04-25T00:40:29.093696+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:40.259597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068430+0000","last_change":"2024-04-25T00:40:29.068430+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068430+0000","last_undegraded":"2024-04-25T00:40:29.068430+0000","last_fullsized":"2024-04-25T00:40:29.068430+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.098189+0000","last_change":"2024-04-25T00:40:29.097144+0000","last_active":"2024-04-25T00:40:29.098189+0000","last_peered":"2024-04-25T00:40:29.098189+0000","last_clean":"2024-04-25T00:40:29.098189+0000","last_became_active":"2024-04-25T00:40:29.096558+0000","last_became_peered":"2024-04-25T00:40:29.096558+0000","last_unstale":"2024-04-25T00:40:29.098189+0000","last_undegraded":"2024-04-25T00:40:29.098189+0000","last_fullsized":"2024-04-25T00:40:29.098189+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:40.259597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068589+0000","last_change":"2024-04-25T00:40:29.068589+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068589+0000","last_undegraded":"2024-04-25T00:40:29.068589+0000","last_fullsized":"2024-04-25T00:40:29.068589+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.079341+0000","last_change":"2024-04-25T00:40:29.077975+0000","last_active":"2024-04-25T00:40:29.079341+0000","last_peered":"2024-04-25T00:40:29.079341+0000","last_clean":"2024-04-25T00:40:29.079341+0000","last_became_active":"2024-04-25T00:40:29.077218+0000","last_became_peered":"2024-04-25T00:40:29.077218+0000","last_unstale":"2024-04-25T00:40:29.079341+0000","last_undegraded":"2024-04-25T00:40:29.079341+0000","last_fullsized":"2024-04-25T00:40:29.079341+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:21.386455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.068479+0000","last_change":"2024-04-25T00:40:29.068479+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.068479+0000","last_undegraded":"2024-04-25T00:40:29.068479+0000","last_fullsized":"2024-04-25T00:40:29.068479+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.098857+0000","last_change":"2024-04-25T00:40:29.096474+0000","last_active":"2024-04-25T00:40:29.098857+0000","last_peered":"2024-04-25T00:40:29.098857+0000","last_clean":"2024-04-25T00:40:29.098857+0000","last_became_active":"2024-04-25T00:40:29.095752+0000","last_became_peered":"2024-04-25T00:40:29.095752+0000","last_unstale":"2024-04-25T00:40:29.098857+0000","last_undegraded":"2024-04-25T00:40:29.098857+0000","last_fullsized":"2024-04-25T00:40:29.098857+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:42.017864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.074826+0000","last_change":"2024-04-25T00:40:29.074826+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.074826+0000","last_undegraded":"2024-04-25T00:40:29.074826+0000","last_fullsized":"2024-04-25T00:40:29.074826+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.099838+0000","last_change":"2024-04-25T00:40:29.084462+0000","last_active":"2024-04-25T00:40:29.099838+0000","last_peered":"2024-04-25T00:40:29.099838+0000","last_clean":"2024-04-25T00:40:29.099838+0000","last_became_active":"2024-04-25T00:40:29.082507+0000","last_became_peered":"2024-04-25T00:40:29.082507+0000","last_unstale":"2024-04-25T00:40:29.099838+0000","last_undegraded":"2024-04-25T00:40:29.099838+0000","last_fullsized":"2024-04-25T00:40:29.099838+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:52.881575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.072774+0000","last_change":"2024-04-25T00:40:29.072774+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.072774+0000","last_undegraded":"2024-04-25T00:40:29.072774+0000","last_fullsized":"2024-04-25T00:40:29.072774+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.095525+0000","last_change":"2024-04-25T00:40:29.094661+0000","last_active":"2024-04-25T00:40:29.095525+0000","last_peered":"2024-04-25T00:40:29.095525+0000","last_clean":"2024-04-25T00:40:29.095525+0000","last_became_active":"2024-04-25T00:40:29.094039+0000","last_became_peered":"2024-04-25T00:40:29.094039+0000","last_unstale":"2024-04-25T00:40:29.095525+0000","last_undegraded":"2024-04-25T00:40:29.095525+0000","last_fullsized":"2024-04-25T00:40:29.095525+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:49.527153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":154,"state":"creating+peering","last_fresh":"2024-04-25T00:40:29.067669+0000","last_change":"2024-04-25T00:40:29.067669+0000","last_active":"2024-04-25T00:40:29.053452+0000","last_peered":"2024-04-25T00:40:29.053452+0000","last_clean":"2024-04-25T00:40:29.053452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:40:29.067669+0000","last_undegraded":"2024-04-25T00:40:29.067669+0000","last_fullsized":"2024-04-25T00:40:29.067669+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.102744+0000","last_change":"2024-04-25T00:40:29.101828+0000","last_active":"2024-04-25T00:40:29.102744+0000","last_peered":"2024-04-25T00:40:29.102744+0000","last_clean":"2024-04-25T00:40:29.102744+0000","last_became_active":"2024-04-25T00:40:29.101030+0000","last_became_peered":"2024-04-25T00:40:29.101030+0000","last_unstale":"2024-04-25T00:40:29.102744+0000","last_undegraded":"2024-04-25T00:40:29.102744+0000","last_fullsized":"2024-04-25T00:40:29.102744+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:08.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":462,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T00:40:29.070740+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:40:29.070740+0000","last_peered":"2024-04-25T00:40:29.070740+0000","last_clean":"2024-04-25T00:40:29.070740+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:40:29.070740+0000","last_undegraded":"2024-04-25T00:40:29.070740+0000","last_fullsized":"2024-04-25T00:40:29.070740+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574960,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37808,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334032,"statfs":{"total":96636764160,"available":96598048768,"internally_reserved":0,"allocated":2523136,"data_stored":1440265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574961,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1941504,"data_stored":856172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41388,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330452,"statfs":{"total":96636764160,"available":96594382848,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37232,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1929216,"data_stored":849897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574960,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41968,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329872,"statfs":{"total":96636764160,"available":96593788928,"internally_reserved":0,"allocated":2523136,"data_stored":1440265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:40:34.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3f is not active+clean 2024-04-25T00:40:34.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.068067+0000', 'last_change': '2024-04-25T00:40:29.068067+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.068067+0000', 'last_undegraded': '2024-04-25T00:40:29.068067+0000', 'last_fullsized': '2024-04-25T00:40:29.068067+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:40:34.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3e is not active+clean 2024-04-25T00:40:34.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.077656+0000', 'last_change': '2024-04-25T00:40:29.077656+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.077656+0000', 'last_undegraded': '2024-04-25T00:40:29.077656+0000', 'last_fullsized': '2024-04-25T00:40:29.077656+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:40:34.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3d is not active+clean 2024-04-25T00:40:34.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.085806+0000', 'last_change': '2024-04-25T00:40:29.085806+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.085806+0000', 'last_undegraded': '2024-04-25T00:40:29.085806+0000', 'last_fullsized': '2024-04-25T00:40:29.085806+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:40:34.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3c is not active+clean 2024-04-25T00:40:34.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.076453+0000', 'last_change': '2024-04-25T00:40:29.076453+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.076453+0000', 'last_undegraded': '2024-04-25T00:40:29.076453+0000', 'last_fullsized': '2024-04-25T00:40:29.076453+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:40:34.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3b is not active+clean 2024-04-25T00:40:34.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.083171+0000', 'last_change': '2024-04-25T00:40:29.083171+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.083171+0000', 'last_undegraded': '2024-04-25T00:40:29.083171+0000', 'last_fullsized': '2024-04-25T00:40:29.083171+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:40:34.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3a is not active+clean 2024-04-25T00:40:34.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.076571+0000', 'last_change': '2024-04-25T00:40:29.076571+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.076571+0000', 'last_undegraded': '2024-04-25T00:40:29.076571+0000', 'last_fullsized': '2024-04-25T00:40:29.076571+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:40:34.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.39 is not active+clean 2024-04-25T00:40:34.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.067060+0000', 'last_change': '2024-04-25T00:40:29.067060+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.067060+0000', 'last_undegraded': '2024-04-25T00:40:29.067060+0000', 'last_fullsized': '2024-04-25T00:40:29.067060+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:40:34.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.38 is not active+clean 2024-04-25T00:40:34.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.078988+0000', 'last_change': '2024-04-25T00:40:29.078988+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.078988+0000', 'last_undegraded': '2024-04-25T00:40:29.078988+0000', 'last_fullsized': '2024-04-25T00:40:29.078988+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:40:34.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.37 is not active+clean 2024-04-25T00:40:34.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.066446+0000', 'last_change': '2024-04-25T00:40:29.066446+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.066446+0000', 'last_undegraded': '2024-04-25T00:40:29.066446+0000', 'last_fullsized': '2024-04-25T00:40:29.066446+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:40:34.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.36 is not active+clean 2024-04-25T00:40:34.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.071624+0000', 'last_change': '2024-04-25T00:40:29.071624+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.071624+0000', 'last_undegraded': '2024-04-25T00:40:29.071624+0000', 'last_fullsized': '2024-04-25T00:40:29.071624+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:40:34.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.35 is not active+clean 2024-04-25T00:40:34.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.065659+0000', 'last_change': '2024-04-25T00:40:29.065659+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.065659+0000', 'last_undegraded': '2024-04-25T00:40:29.065659+0000', 'last_fullsized': '2024-04-25T00:40:29.065659+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:40:34.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.34 is not active+clean 2024-04-25T00:40:34.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.075405+0000', 'last_change': '2024-04-25T00:40:29.075405+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.075405+0000', 'last_undegraded': '2024-04-25T00:40:29.075405+0000', 'last_fullsized': '2024-04-25T00:40:29.075405+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:40:34.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.33 is not active+clean 2024-04-25T00:40:34.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.079997+0000', 'last_change': '2024-04-25T00:40:29.079997+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.079997+0000', 'last_undegraded': '2024-04-25T00:40:29.079997+0000', 'last_fullsized': '2024-04-25T00:40:29.079997+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:40:34.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.32 is not active+clean 2024-04-25T00:40:34.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.067927+0000', 'last_change': '2024-04-25T00:40:29.067927+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.067927+0000', 'last_undegraded': '2024-04-25T00:40:29.067927+0000', 'last_fullsized': '2024-04-25T00:40:29.067927+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:40:34.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.31 is not active+clean 2024-04-25T00:40:34.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.080999+0000', 'last_change': '2024-04-25T00:40:29.080999+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.080999+0000', 'last_undegraded': '2024-04-25T00:40:29.080999+0000', 'last_fullsized': '2024-04-25T00:40:29.080999+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:40:34.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.30 is not active+clean 2024-04-25T00:40:34.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.078469+0000', 'last_change': '2024-04-25T00:40:29.078469+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.078469+0000', 'last_undegraded': '2024-04-25T00:40:29.078469+0000', 'last_fullsized': '2024-04-25T00:40:29.078469+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:40:34.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2f is not active+clean 2024-04-25T00:40:34.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.082514+0000', 'last_change': '2024-04-25T00:40:29.082514+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.082514+0000', 'last_undegraded': '2024-04-25T00:40:29.082514+0000', 'last_fullsized': '2024-04-25T00:40:29.082514+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:40:34.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2e is not active+clean 2024-04-25T00:40:34.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.068120+0000', 'last_change': '2024-04-25T00:40:29.068120+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.068120+0000', 'last_undegraded': '2024-04-25T00:40:29.068120+0000', 'last_fullsized': '2024-04-25T00:40:29.068120+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:40:34.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2d is not active+clean 2024-04-25T00:40:34.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.069605+0000', 'last_change': '2024-04-25T00:40:29.069605+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.069605+0000', 'last_undegraded': '2024-04-25T00:40:29.069605+0000', 'last_fullsized': '2024-04-25T00:40:29.069605+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:40:34.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2c is not active+clean 2024-04-25T00:40:34.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.072609+0000', 'last_change': '2024-04-25T00:40:29.072609+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.072609+0000', 'last_undegraded': '2024-04-25T00:40:29.072609+0000', 'last_fullsized': '2024-04-25T00:40:29.072609+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:40:34.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2b is not active+clean 2024-04-25T00:40:34.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.082713+0000', 'last_change': '2024-04-25T00:40:29.082713+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.082713+0000', 'last_undegraded': '2024-04-25T00:40:29.082713+0000', 'last_fullsized': '2024-04-25T00:40:29.082713+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:40:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2a is not active+clean 2024-04-25T00:40:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.069538+0000', 'last_change': '2024-04-25T00:40:29.069538+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.069538+0000', 'last_undegraded': '2024-04-25T00:40:29.069538+0000', 'last_fullsized': '2024-04-25T00:40:29.069538+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:40:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.29 is not active+clean 2024-04-25T00:40:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.085695+0000', 'last_change': '2024-04-25T00:40:29.085695+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.085695+0000', 'last_undegraded': '2024-04-25T00:40:29.085695+0000', 'last_fullsized': '2024-04-25T00:40:29.085695+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:40:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.28 is not active+clean 2024-04-25T00:40:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.077654+0000', 'last_change': '2024-04-25T00:40:29.077654+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.077654+0000', 'last_undegraded': '2024-04-25T00:40:29.077654+0000', 'last_fullsized': '2024-04-25T00:40:29.077654+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:40:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.27 is not active+clean 2024-04-25T00:40:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.073015+0000', 'last_change': '2024-04-25T00:40:29.073015+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.073015+0000', 'last_undegraded': '2024-04-25T00:40:29.073015+0000', 'last_fullsized': '2024-04-25T00:40:29.073015+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:40:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.26 is not active+clean 2024-04-25T00:40:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.065060+0000', 'last_change': '2024-04-25T00:40:29.065060+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.065060+0000', 'last_undegraded': '2024-04-25T00:40:29.065060+0000', 'last_fullsized': '2024-04-25T00:40:29.065060+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:40:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.25 is not active+clean 2024-04-25T00:40:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.082493+0000', 'last_change': '2024-04-25T00:40:29.082493+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.082493+0000', 'last_undegraded': '2024-04-25T00:40:29.082493+0000', 'last_fullsized': '2024-04-25T00:40:29.082493+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:40:34.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.24 is not active+clean 2024-04-25T00:40:34.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.072143+0000', 'last_change': '2024-04-25T00:40:29.072143+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.072143+0000', 'last_undegraded': '2024-04-25T00:40:29.072143+0000', 'last_fullsized': '2024-04-25T00:40:29.072143+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:40:34.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.23 is not active+clean 2024-04-25T00:40:34.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.072837+0000', 'last_change': '2024-04-25T00:40:29.072837+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.072837+0000', 'last_undegraded': '2024-04-25T00:40:29.072837+0000', 'last_fullsized': '2024-04-25T00:40:29.072837+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:40:34.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.22 is not active+clean 2024-04-25T00:40:34.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.065859+0000', 'last_change': '2024-04-25T00:40:29.065859+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.065859+0000', 'last_undegraded': '2024-04-25T00:40:29.065859+0000', 'last_fullsized': '2024-04-25T00:40:29.065859+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:40:34.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.21 is not active+clean 2024-04-25T00:40:34.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.082748+0000', 'last_change': '2024-04-25T00:40:29.082748+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.082748+0000', 'last_undegraded': '2024-04-25T00:40:29.082748+0000', 'last_fullsized': '2024-04-25T00:40:29.082748+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:40:34.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.20 is not active+clean 2024-04-25T00:40:34.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.071805+0000', 'last_change': '2024-04-25T00:40:29.071805+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.071805+0000', 'last_undegraded': '2024-04-25T00:40:29.071805+0000', 'last_fullsized': '2024-04-25T00:40:29.071805+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:40:34.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1e is not active+clean 2024-04-25T00:40:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.074871+0000', 'last_change': '2024-04-25T00:40:29.074871+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.074871+0000', 'last_undegraded': '2024-04-25T00:40:29.074871+0000', 'last_fullsized': '2024-04-25T00:40:29.074871+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:40:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1f is not active+clean 2024-04-25T00:40:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.070618+0000', 'last_change': '2024-04-25T00:40:29.070618+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.070618+0000', 'last_undegraded': '2024-04-25T00:40:29.070618+0000', 'last_fullsized': '2024-04-25T00:40:29.070618+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:40:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1c is not active+clean 2024-04-25T00:40:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.076415+0000', 'last_change': '2024-04-25T00:40:29.076415+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.076415+0000', 'last_undegraded': '2024-04-25T00:40:29.076415+0000', 'last_fullsized': '2024-04-25T00:40:29.076415+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:40:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1d is not active+clean 2024-04-25T00:40:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.067036+0000', 'last_change': '2024-04-25T00:40:29.067036+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.067036+0000', 'last_undegraded': '2024-04-25T00:40:29.067036+0000', 'last_fullsized': '2024-04-25T00:40:29.067036+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:40:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1a is not active+clean 2024-04-25T00:40:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.069078+0000', 'last_change': '2024-04-25T00:40:29.069078+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.069078+0000', 'last_undegraded': '2024-04-25T00:40:29.069078+0000', 'last_fullsized': '2024-04-25T00:40:29.069078+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:40:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1b is not active+clean 2024-04-25T00:40:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.074690+0000', 'last_change': '2024-04-25T00:40:29.074690+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.074690+0000', 'last_undegraded': '2024-04-25T00:40:29.074690+0000', 'last_fullsized': '2024-04-25T00:40:29.074690+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:40:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.19 is not active+clean 2024-04-25T00:40:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.064376+0000', 'last_change': '2024-04-25T00:40:29.064376+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.064376+0000', 'last_undegraded': '2024-04-25T00:40:29.064376+0000', 'last_fullsized': '2024-04-25T00:40:29.064376+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:40:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.16 is not active+clean 2024-04-25T00:40:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.076804+0000', 'last_change': '2024-04-25T00:40:29.076804+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.076804+0000', 'last_undegraded': '2024-04-25T00:40:29.076804+0000', 'last_fullsized': '2024-04-25T00:40:29.076804+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:40:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.17 is not active+clean 2024-04-25T00:40:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.068208+0000', 'last_change': '2024-04-25T00:40:29.068208+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.068208+0000', 'last_undegraded': '2024-04-25T00:40:29.068208+0000', 'last_fullsized': '2024-04-25T00:40:29.068208+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:40:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.14 is not active+clean 2024-04-25T00:40:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.067576+0000', 'last_change': '2024-04-25T00:40:29.067576+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.067576+0000', 'last_undegraded': '2024-04-25T00:40:29.067576+0000', 'last_fullsized': '2024-04-25T00:40:29.067576+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:40:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.15 is not active+clean 2024-04-25T00:40:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.069149+0000', 'last_change': '2024-04-25T00:40:29.069149+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.069149+0000', 'last_undegraded': '2024-04-25T00:40:29.069149+0000', 'last_fullsized': '2024-04-25T00:40:29.069149+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:40:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.12 is not active+clean 2024-04-25T00:40:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.076030+0000', 'last_change': '2024-04-25T00:40:29.076030+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.076030+0000', 'last_undegraded': '2024-04-25T00:40:29.076030+0000', 'last_fullsized': '2024-04-25T00:40:29.076030+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:40:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.13 is not active+clean 2024-04-25T00:40:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.065511+0000', 'last_change': '2024-04-25T00:40:29.065511+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.065511+0000', 'last_undegraded': '2024-04-25T00:40:29.065511+0000', 'last_fullsized': '2024-04-25T00:40:29.065511+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:40:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.10 is not active+clean 2024-04-25T00:40:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.064117+0000', 'last_change': '2024-04-25T00:40:29.064117+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.064117+0000', 'last_undegraded': '2024-04-25T00:40:29.064117+0000', 'last_fullsized': '2024-04-25T00:40:29.064117+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:40:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.11 is not active+clean 2024-04-25T00:40:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.071314+0000', 'last_change': '2024-04-25T00:40:29.071314+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.071314+0000', 'last_undegraded': '2024-04-25T00:40:29.071314+0000', 'last_fullsized': '2024-04-25T00:40:29.071314+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:40:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.e is not active+clean 2024-04-25T00:40:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.071161+0000', 'last_change': '2024-04-25T00:40:29.071161+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.071161+0000', 'last_undegraded': '2024-04-25T00:40:29.071161+0000', 'last_fullsized': '2024-04-25T00:40:29.071161+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:40:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.f is not active+clean 2024-04-25T00:40:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.065433+0000', 'last_change': '2024-04-25T00:40:29.065433+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.065433+0000', 'last_undegraded': '2024-04-25T00:40:29.065433+0000', 'last_fullsized': '2024-04-25T00:40:29.065433+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:40:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.c is not active+clean 2024-04-25T00:40:34.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.064890+0000', 'last_change': '2024-04-25T00:40:29.064890+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.064890+0000', 'last_undegraded': '2024-04-25T00:40:29.064890+0000', 'last_fullsized': '2024-04-25T00:40:29.064890+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:40:34.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.d is not active+clean 2024-04-25T00:40:34.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.073123+0000', 'last_change': '2024-04-25T00:40:29.073123+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.073123+0000', 'last_undegraded': '2024-04-25T00:40:29.073123+0000', 'last_fullsized': '2024-04-25T00:40:29.073123+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:40:34.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.a is not active+clean 2024-04-25T00:40:34.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.068646+0000', 'last_change': '2024-04-25T00:40:29.068646+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.068646+0000', 'last_undegraded': '2024-04-25T00:40:29.068646+0000', 'last_fullsized': '2024-04-25T00:40:29.068646+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:40:34.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.b is not active+clean 2024-04-25T00:40:34.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.064353+0000', 'last_change': '2024-04-25T00:40:29.064353+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.064353+0000', 'last_undegraded': '2024-04-25T00:40:29.064353+0000', 'last_fullsized': '2024-04-25T00:40:29.064353+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:40:34.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.8 is not active+clean 2024-04-25T00:40:34.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.076771+0000', 'last_change': '2024-04-25T00:40:29.076771+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.076771+0000', 'last_undegraded': '2024-04-25T00:40:29.076771+0000', 'last_fullsized': '2024-04-25T00:40:29.076771+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:40:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.9 is not active+clean 2024-04-25T00:40:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.082449+0000', 'last_change': '2024-04-25T00:40:29.082449+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.082449+0000', 'last_undegraded': '2024-04-25T00:40:29.082449+0000', 'last_fullsized': '2024-04-25T00:40:29.082449+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:40:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.6 is not active+clean 2024-04-25T00:40:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.072503+0000', 'last_change': '2024-04-25T00:40:29.072503+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.072503+0000', 'last_undegraded': '2024-04-25T00:40:29.072503+0000', 'last_fullsized': '2024-04-25T00:40:29.072503+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:40:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.7 is not active+clean 2024-04-25T00:40:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.070128+0000', 'last_change': '2024-04-25T00:40:29.070128+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.070128+0000', 'last_undegraded': '2024-04-25T00:40:29.070128+0000', 'last_fullsized': '2024-04-25T00:40:29.070128+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:40:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.4 is not active+clean 2024-04-25T00:40:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.067461+0000', 'last_change': '2024-04-25T00:40:29.067461+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.067461+0000', 'last_undegraded': '2024-04-25T00:40:29.067461+0000', 'last_fullsized': '2024-04-25T00:40:29.067461+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:40:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.5 is not active+clean 2024-04-25T00:40:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.068430+0000', 'last_change': '2024-04-25T00:40:29.068430+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.068430+0000', 'last_undegraded': '2024-04-25T00:40:29.068430+0000', 'last_fullsized': '2024-04-25T00:40:29.068430+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:40:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2 is not active+clean 2024-04-25T00:40:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.068589+0000', 'last_change': '2024-04-25T00:40:29.068589+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.068589+0000', 'last_undegraded': '2024-04-25T00:40:29.068589+0000', 'last_fullsized': '2024-04-25T00:40:29.068589+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:40:34.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3 is not active+clean 2024-04-25T00:40:34.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.068479+0000', 'last_change': '2024-04-25T00:40:29.068479+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.068479+0000', 'last_undegraded': '2024-04-25T00:40:29.068479+0000', 'last_fullsized': '2024-04-25T00:40:29.068479+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:40:34.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.0 is not active+clean 2024-04-25T00:40:34.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.074826+0000', 'last_change': '2024-04-25T00:40:29.074826+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.074826+0000', 'last_undegraded': '2024-04-25T00:40:29.074826+0000', 'last_fullsized': '2024-04-25T00:40:29.074826+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:40:34.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1 is not active+clean 2024-04-25T00:40:34.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.072774+0000', 'last_change': '2024-04-25T00:40:29.072774+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.072774+0000', 'last_undegraded': '2024-04-25T00:40:29.072774+0000', 'last_fullsized': '2024-04-25T00:40:29.072774+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:40:34.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.18 is not active+clean 2024-04-25T00:40:34.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 154, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:40:29.067669+0000', 'last_change': '2024-04-25T00:40:29.067669+0000', 'last_active': '2024-04-25T00:40:29.053452+0000', 'last_peered': '2024-04-25T00:40:29.053452+0000', 'last_clean': '2024-04-25T00:40:29.053452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:40:29.067669+0000', 'last_undegraded': '2024-04-25T00:40:29.067669+0000', 'last_fullsized': '2024-04-25T00:40:29.067669+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'last_clean_scrub_stamp': '2024-04-25T00:40:29.053452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:40:34.883 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:40:35.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.025+0000 7f5760407640 1 Processor -- start 2024-04-25T00:40:35.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.025+0000 7f5760407640 1 -- start start 2024-04-25T00:40:35.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f5760407640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57580b3140 0x7f575806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:35.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f5760407640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57580b4b10 0x7f57580b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:35.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f5760407640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57580b41e0 con 0x7f57580b3a60 2024-04-25T00:40:35.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f5760407640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f575806d960 con 0x7f57580b3140 2024-04-25T00:40:35.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f5760407640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f575806dc40 con 0x7f57580b4b10 2024-04-25T00:40:35.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f575e1a3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57580b3140 0x7f575806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:35.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f575e9a4640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f57580b3a60 0x7f575815c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55756/0 (socket says 172.21.15.29:55756) 2024-04-25T00:40:35.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f575e9a4640 1 -- 172.21.15.29:0/204691289 learned_addr learned my addr 172.21.15.29:0/204691289 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:35.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f575d9a2640 1 --2- 172.21.15.29:0/204691289 >> v2:172.21.15.157:3300/0 conn(0x7f57580b4b10 0x7f57580b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:35.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f575e1a3640 1 -- 172.21.15.29:0/204691289 >> v2:172.21.15.157:3300/0 conn(0x7f57580b4b10 msgr2=0x7f57580b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:35.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f575e1a3640 1 --2- 172.21.15.29:0/204691289 >> v2:172.21.15.157:3300/0 conn(0x7f57580b4b10 0x7f57580b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f575e1a3640 1 -- 172.21.15.29:0/204691289 >> v1:172.21.15.29:6789/0 conn(0x7f57580b3a60 legacy=0x7f575815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:35.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f575d1a1640 1 -- 172.21.15.29:0/204691289 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3286178694 0 0) 0x7f57580b41e0 con 0x7f57580b3a60 2024-04-25T00:40:35.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f575e1a3640 1 -- 172.21.15.29:0/204691289 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575806df20 con 0x7f57580b3140 2024-04-25T00:40:35.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f575e1a3640 1 --2- 172.21.15.29:0/204691289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57580b3140 0x7f575806e440 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f57540097c0 tx=0x7f575401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802f58bc481a295f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:35.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f575d1a1640 1 -- 172.21.15.29:0/204691289 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57540b0070 con 0x7f57580b3140 2024-04-25T00:40:35.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f575d1a1640 1 -- 172.21.15.29:0/204691289 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57540ab600 con 0x7f57580b3140 2024-04-25T00:40:35.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.029+0000 7f575d1a1640 1 -- 172.21.15.29:0/204691289 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57540ab9d0 con 0x7f57580b3140 2024-04-25T00:40:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 -- 172.21.15.29:0/204691289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57580b3140 msgr2=0x7f575806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 --2- 172.21.15.29:0/204691289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57580b3140 0x7f575806e440 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f57540097c0 tx=0x7f575401c460 comp rx=0 tx=0).stop 2024-04-25T00:40:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 -- 172.21.15.29:0/204691289 shutdown_connections 2024-04-25T00:40:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 --2- 172.21.15.29:0/204691289 >> v2:172.21.15.157:3300/0 conn(0x7f57580b4b10 0x7f57580b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 --2- 172.21.15.29:0/204691289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57580b3140 0x7f575806e440 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 -- 172.21.15.29:0/204691289 >> 172.21.15.29:0/204691289 conn(0x7f57580b1870 msgr2=0x7f57580b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 -- 172.21.15.29:0/204691289 shutdown_connections 2024-04-25T00:40:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 -- 172.21.15.29:0/204691289 wait complete. 2024-04-25T00:40:35.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 Processor -- start 2024-04-25T00:40:35.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 -- start start 2024-04-25T00:40:35.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57580b3140 0x7f575813a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:35.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57580b3a60 0x7f575815f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:35.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57580b4b10 0x7f5758139a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:35.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5758168b70 con 0x7f57580b3140 2024-04-25T00:40:35.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57581689f0 con 0x7f57580b3a60 2024-04-25T00:40:35.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5758168cf0 con 0x7f57580b4b10 2024-04-25T00:40:35.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f575e1a3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57580b3140 0x7f575813a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:35.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f575e1a3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57580b3140 0x7f575813a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60022/0 (socket says 172.21.15.29:60022) 2024-04-25T00:40:35.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f575e1a3640 1 -- 172.21.15.29:0/1807600838 learned_addr learned my addr 172.21.15.29:0/1807600838 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:35.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f575e9a4640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57580b4b10 0x7f5758139a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:35.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f575d9a2640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57580b3a60 0x7f575815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:35.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f575e1a3640 1 -- 172.21.15.29:0/1807600838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57580b4b10 msgr2=0x7f5758139a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:35.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f575e1a3640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57580b4b10 0x7f5758139a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f575e1a3640 1 -- 172.21.15.29:0/1807600838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57580b3a60 msgr2=0x7f575815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:35.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f575e1a3640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57580b3a60 0x7f575815f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f575e1a3640 1 -- 172.21.15.29:0/1807600838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575813ab80 con 0x7f57580b3140 2024-04-25T00:40:35.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f575d9a2640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57580b3a60 0x7f575815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:35.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f575e1a3640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57580b3140 0x7f575813a2e0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f57540ae040 tx=0x7f575401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:35.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f574effd640 1 -- 172.21.15.29:0/1807600838 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57540b0050 con 0x7f57580b3140 2024-04-25T00:40:35.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f574effd640 1 -- 172.21.15.29:0/1807600838 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57540bc070 con 0x7f57580b3140 2024-04-25T00:40:35.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f574effd640 1 -- 172.21.15.29:0/1807600838 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57540b5850 con 0x7f57580b3140 2024-04-25T00:40:35.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 -- 172.21.15.29:0/1807600838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5758157f60 con 0x7f57580b3140 2024-04-25T00:40:35.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.033+0000 7f5760407640 1 -- 172.21.15.29:0/1807600838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5758158520 con 0x7f57580b3140 2024-04-25T00:40:35.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.037+0000 7f5760407640 1 -- 172.21.15.29:0/1807600838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5758069270 con 0x7f57580b3140 2024-04-25T00:40:35.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.037+0000 7f574effd640 1 -- 172.21.15.29:0/1807600838 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f57540b59f0 con 0x7f57580b3140 2024-04-25T00:40:35.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.037+0000 7f574effd640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f573c0680a0 0x7f573c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:35.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.037+0000 7f574effd640 1 -- 172.21.15.29:0/1807600838 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f5754121510 con 0x7f57580b3140 2024-04-25T00:40:35.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.037+0000 7f575d9a2640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f573c0680a0 0x7f573c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:35.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.037+0000 7f575d9a2640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f573c0680a0 0x7f573c06a560 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f57580b1650 tx=0x7f5748008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:35.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.041+0000 7f574effd640 1 -- 172.21.15.29:0/1807600838 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f575813ab80 con 0x7f57580b3140 2024-04-25T00:40:35.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.261+0000 7f5760407640 1 -- 172.21.15.29:0/1807600838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f575815bdb0 con 0x7f57580b3140 2024-04-25T00:40:35.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.261+0000 7f574effd640 1 -- 172.21.15.29:0/1807600838 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1459 (secure 0 0 0) 0x7f575411fe20 con 0x7f57580b3140 2024-04-25T00:40:35.262 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:35.262 INFO:teuthology.orchestra.run.smithi029.stdout:{"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":562,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":156,"num_osds":8,"num_up_osds":8,"osd_up_since":1714005083,"num_in_osds":8,"osd_in_since":1714005074,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":314703872,"bytes_avail":772779409408,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":102,"id":12,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":12,"rank":0,"name":"c","status":"up:active","gid":10095}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-25T00:39:37.388610+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:40:35.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.265+0000 7f5760407640 1 -- 172.21.15.29:0/1807600838 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f573c0680a0 msgr2=0x7f573c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:35.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.265+0000 7f5760407640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f573c0680a0 0x7f573c06a560 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f57580b1650 tx=0x7f5748008040 comp rx=0 tx=0).stop 2024-04-25T00:40:35.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.265+0000 7f5760407640 1 -- 172.21.15.29:0/1807600838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57580b3140 msgr2=0x7f575813a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:35.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.265+0000 7f5760407640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57580b3140 0x7f575813a2e0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f57540ae040 tx=0x7f575401aca0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.265+0000 7f5760407640 1 -- 172.21.15.29:0/1807600838 shutdown_connections 2024-04-25T00:40:35.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.265+0000 7f5760407640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f573c0680a0 0x7f573c06a560 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.265+0000 7f5760407640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57580b4b10 0x7f5758139a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.265+0000 7f5760407640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57580b3a60 0x7f575815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.265+0000 7f5760407640 1 --2- 172.21.15.29:0/1807600838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57580b3140 0x7f575813a2e0 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.265+0000 7f5760407640 1 -- 172.21.15.29:0/1807600838 >> 172.21.15.29:0/1807600838 conn(0x7f57580b1870 msgr2=0x7f575815b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:35.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.265+0000 7f5760407640 1 -- 172.21.15.29:0/1807600838 shutdown_connections 2024-04-25T00:40:35.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.265+0000 7f5760407640 1 -- 172.21.15.29:0/1807600838 wait complete. 2024-04-25T00:40:35.280 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T00:40:35.281 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:40:35.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde09552640 1 Processor -- start 2024-04-25T00:40:35.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde09552640 1 -- start start 2024-04-25T00:40:35.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde09552640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fde040b3140 0x7fde0406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:35.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde09552640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde040b3c30 0x7fde040ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:35.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde09552640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde040b41e0 con 0x7fde040b4b10 2024-04-25T00:40:35.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde09552640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fde0406d960 con 0x7fde040b3140 2024-04-25T00:40:35.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde09552640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fde0406dc40 con 0x7fde040b3c30 2024-04-25T00:40:35.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde02ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fde040b3140 0x7fde0406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:35.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde02ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fde040b3140 0x7fde0406e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60034/0 (socket says 172.21.15.29:60034) 2024-04-25T00:40:35.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde02ffd640 1 -- 172.21.15.29:0/4135834764 learned_addr learned my addr 172.21.15.29:0/4135834764 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:35.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde037fe640 1 --2- 172.21.15.29:0/4135834764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde040b3c30 0x7fde040ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:35.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde02ffd640 1 -- 172.21.15.29:0/4135834764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde040b3c30 msgr2=0x7fde040ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:35.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde02ffd640 1 --2- 172.21.15.29:0/4135834764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde040b3c30 0x7fde040ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde02ffd640 1 -- 172.21.15.29:0/4135834764 >> v1:172.21.15.157:6789/0 conn(0x7fde040b4b10 legacy=0x7fde040b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:35.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde02ffd640 1 -- 172.21.15.29:0/4135834764 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0406df20 con 0x7fde040b3140 2024-04-25T00:40:35.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde037fe640 1 --2- 172.21.15.29:0/4135834764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde040b3c30 0x7fde040ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:35.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.393+0000 7fde02ffd640 1 --2- 172.21.15.29:0/4135834764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde040b3140 0x7fde0406e440 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fddf40098a0 tx=0x7fddf401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae058bc12454b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:35.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde027fc640 1 -- 172.21.15.29:0/4135834764 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf40b0070 con 0x7fde040b3140 2024-04-25T00:40:35.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde027fc640 1 -- 172.21.15.29:0/4135834764 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf40ab600 con 0x7fde040b3140 2024-04-25T00:40:35.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde027fc640 1 -- 172.21.15.29:0/4135834764 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf40ab9d0 con 0x7fde040b3140 2024-04-25T00:40:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 -- 172.21.15.29:0/4135834764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde040b3140 msgr2=0x7fde0406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 --2- 172.21.15.29:0/4135834764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde040b3140 0x7fde0406e440 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fddf40098a0 tx=0x7fddf401c500 comp rx=0 tx=0).stop 2024-04-25T00:40:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 -- 172.21.15.29:0/4135834764 shutdown_connections 2024-04-25T00:40:35.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 --2- 172.21.15.29:0/4135834764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde040b3c30 0x7fde040ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 --2- 172.21.15.29:0/4135834764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde040b3140 0x7fde0406e440 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 -- 172.21.15.29:0/4135834764 >> 172.21.15.29:0/4135834764 conn(0x7fde040b1870 msgr2=0x7fde040b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:35.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 -- 172.21.15.29:0/4135834764 shutdown_connections 2024-04-25T00:40:35.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 -- 172.21.15.29:0/4135834764 wait complete. 2024-04-25T00:40:35.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 Processor -- start 2024-04-25T00:40:35.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 -- start start 2024-04-25T00:40:35.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde040b3140 0x7fde0415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:35.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde040b3c30 0x7fde04139750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:35.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde040b4b10 0x7fde0415b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:35.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde04168d80 con 0x7fde040b4b10 2024-04-25T00:40:35.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fde04168c00 con 0x7fde040b3140 2024-04-25T00:40:35.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fde04168f00 con 0x7fde040b3c30 2024-04-25T00:40:35.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde037fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde040b4b10 0x7fde0415b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:35.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde037fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde040b4b10 0x7fde0415b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60040/0 (socket says 172.21.15.29:60040) 2024-04-25T00:40:35.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde037fe640 1 -- 172.21.15.29:0/136492727 learned_addr learned my addr 172.21.15.29:0/136492727 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:35.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde02ffd640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde040b3140 0x7fde0415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:35.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fddfbfff640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde040b3c30 0x7fde04139750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:35.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde037fe640 1 -- 172.21.15.29:0/136492727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde040b3c30 msgr2=0x7fde04139750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:35.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde037fe640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde040b3c30 0x7fde04139750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde037fe640 1 -- 172.21.15.29:0/136492727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde040b3140 msgr2=0x7fde0415a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:35.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde037fe640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde040b3140 0x7fde0415a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde037fe640 1 -- 172.21.15.29:0/136492727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0415bee0 con 0x7fde040b4b10 2024-04-25T00:40:35.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde037fe640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde040b4b10 0x7fde0415b290 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00cef0 tx=0x7fddfc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:35.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fddfb7fe640 1 -- 172.21.15.29:0/136492727 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc01a070 con 0x7fde040b4b10 2024-04-25T00:40:35.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fddfb7fe640 1 -- 172.21.15.29:0/136492727 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddfc00d860 con 0x7fde040b4b10 2024-04-25T00:40:35.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 -- 172.21.15.29:0/136492727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde04139c90 con 0x7fde040b4b10 2024-04-25T00:40:35.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fddfb7fe640 1 -- 172.21.15.29:0/136492727 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00dc50 con 0x7fde040b4b10 2024-04-25T00:40:35.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.397+0000 7fde09552640 1 -- 172.21.15.29:0/136492727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde04159920 con 0x7fde040b4b10 2024-04-25T00:40:35.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.401+0000 7fde09552640 1 -- 172.21.15.29:0/136492727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde04069270 con 0x7fde040b4b10 2024-04-25T00:40:35.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.401+0000 7fddfb7fe640 1 -- 172.21.15.29:0/136492727 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fddfc01d070 con 0x7fde040b4b10 2024-04-25T00:40:35.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.401+0000 7fddfb7fe640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdde4068120 0x7fdde406a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:35.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.401+0000 7fddfb7fe640 1 -- 172.21.15.29:0/136492727 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fddfc087670 con 0x7fde040b4b10 2024-04-25T00:40:35.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.401+0000 7fde02ffd640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdde4068120 0x7fdde406a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:35.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.401+0000 7fde02ffd640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdde4068120 0x7fdde406a5e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fddf40ad000 tx=0x7fddf4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:35.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.405+0000 7fddfb7fe640 1 -- 172.21.15.29:0/136492727 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fddfc057420 con 0x7fde040b4b10 2024-04-25T00:40:35.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.557+0000 7fde09552640 1 -- 172.21.15.29:0/136492727 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fde040753e0 con 0x7fdde4068120 2024-04-25T00:40:35.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.565+0000 7fddfb7fe640 1 -- 172.21.15.29:0/136492727 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7fde040753e0 con 0x7fdde4068120 2024-04-25T00:40:35.570 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:35.571 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:40:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.573+0000 7fde09552640 1 -- 172.21.15.29:0/136492727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdde4068120 msgr2=0x7fdde406a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.573+0000 7fde09552640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdde4068120 0x7fdde406a5e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fddf40ad000 tx=0x7fddf4008040 comp rx=0 tx=0).stop 2024-04-25T00:40:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.573+0000 7fde09552640 1 -- 172.21.15.29:0/136492727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde040b4b10 msgr2=0x7fde0415b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.573+0000 7fde09552640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde040b4b10 0x7fde0415b290 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00cef0 tx=0x7fddfc00a710 comp rx=0 tx=0).stop 2024-04-25T00:40:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.573+0000 7fde09552640 1 -- 172.21.15.29:0/136492727 shutdown_connections 2024-04-25T00:40:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.573+0000 7fde09552640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdde4068120 0x7fdde406a5e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.573+0000 7fde09552640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde040b4b10 0x7fde0415b290 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.573+0000 7fde09552640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde040b3c30 0x7fde04139750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.573+0000 7fde09552640 1 --2- 172.21.15.29:0/136492727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde040b3140 0x7fde0415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.573+0000 7fde09552640 1 -- 172.21.15.29:0/136492727 >> 172.21.15.29:0/136492727 conn(0x7fde040b1870 msgr2=0x7fde04142fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.573+0000 7fde09552640 1 -- 172.21.15.29:0/136492727 shutdown_connections 2024-04-25T00:40:35.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:35.573+0000 7fde09552640 1 -- 172.21.15.29:0/136492727 wait complete. 2024-04-25T00:40:35.588 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-04-25T00:40:35.406074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307656,"kb_used_data":16584,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667064,"statfs":{"total":773094113280,"available":772779073536,"internally_reserved":0,"allocated":16982016,"data_stored":8139532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.346518"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109537+0000","last_change":"2024-04-25T00:40:30.132436+0000","last_active":"2024-04-25T00:40:31.109537+0000","last_peered":"2024-04-25T00:40:31.109537+0000","last_clean":"2024-04-25T00:40:31.109537+0000","last_became_active":"2024-04-25T00:40:30.123577+0000","last_became_peered":"2024-04-25T00:40:30.123577+0000","last_unstale":"2024-04-25T00:40:31.109537+0000","last_undegraded":"2024-04-25T00:40:31.109537+0000","last_fullsized":"2024-04-25T00:40:31.109537+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:39.019304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.127473+0000","last_change":"2024-04-25T00:40:30.114772+0000","last_active":"2024-04-25T00:40:32.127473+0000","last_peered":"2024-04-25T00:40:32.127473+0000","last_clean":"2024-04-25T00:40:32.127473+0000","last_became_active":"2024-04-25T00:40:30.109499+0000","last_became_peered":"2024-04-25T00:40:30.109499+0000","last_unstale":"2024-04-25T00:40:32.127473+0000","last_undegraded":"2024-04-25T00:40:32.127473+0000","last_fullsized":"2024-04-25T00:40:32.127473+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:28.342905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109259+0000","last_change":"2024-04-25T00:40:30.117242+0000","last_active":"2024-04-25T00:40:31.109259+0000","last_peered":"2024-04-25T00:40:31.109259+0000","last_clean":"2024-04-25T00:40:31.109259+0000","last_became_active":"2024-04-25T00:40:30.116846+0000","last_became_peered":"2024-04-25T00:40:30.116846+0000","last_unstale":"2024-04-25T00:40:31.109259+0000","last_undegraded":"2024-04-25T00:40:31.109259+0000","last_fullsized":"2024-04-25T00:40:31.109259+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:11.813161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.127096+0000","last_change":"2024-04-25T00:40:30.127364+0000","last_active":"2024-04-25T00:40:32.127096+0000","last_peered":"2024-04-25T00:40:32.127096+0000","last_clean":"2024-04-25T00:40:32.127096+0000","last_became_active":"2024-04-25T00:40:30.124178+0000","last_became_peered":"2024-04-25T00:40:30.124178+0000","last_unstale":"2024-04-25T00:40:32.127096+0000","last_undegraded":"2024-04-25T00:40:32.127096+0000","last_fullsized":"2024-04-25T00:40:32.127096+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:14.368023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532596+0000","last_change":"2024-04-25T00:40:30.104621+0000","last_active":"2024-04-25T00:40:31.532596+0000","last_peered":"2024-04-25T00:40:31.532596+0000","last_clean":"2024-04-25T00:40:31.532596+0000","last_became_active":"2024-04-25T00:40:30.104243+0000","last_became_peered":"2024-04-25T00:40:30.104243+0000","last_unstale":"2024-04-25T00:40:31.532596+0000","last_undegraded":"2024-04-25T00:40:31.532596+0000","last_fullsized":"2024-04-25T00:40:31.532596+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:09.790480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109376+0000","last_change":"2024-04-25T00:40:30.106794+0000","last_active":"2024-04-25T00:40:31.109376+0000","last_peered":"2024-04-25T00:40:31.109376+0000","last_clean":"2024-04-25T00:40:31.109376+0000","last_became_active":"2024-04-25T00:40:30.106065+0000","last_became_peered":"2024-04-25T00:40:30.106065+0000","last_unstale":"2024-04-25T00:40:31.109376+0000","last_undegraded":"2024-04-25T00:40:31.109376+0000","last_fullsized":"2024-04-25T00:40:31.109376+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:41.286811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.110009+0000","last_change":"2024-04-25T00:40:30.128835+0000","last_active":"2024-04-25T00:40:31.110009+0000","last_peered":"2024-04-25T00:40:31.110009+0000","last_clean":"2024-04-25T00:40:31.110009+0000","last_became_active":"2024-04-25T00:40:30.128221+0000","last_became_peered":"2024-04-25T00:40:30.128221+0000","last_unstale":"2024-04-25T00:40:31.110009+0000","last_undegraded":"2024-04-25T00:40:31.110009+0000","last_fullsized":"2024-04-25T00:40:31.110009+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:41.286811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.122767+0000","last_change":"2024-04-25T00:40:30.135572+0000","last_active":"2024-04-25T00:40:32.122767+0000","last_peered":"2024-04-25T00:40:32.122767+0000","last_clean":"2024-04-25T00:40:32.122767+0000","last_became_active":"2024-04-25T00:40:30.135273+0000","last_became_peered":"2024-04-25T00:40:30.135273+0000","last_unstale":"2024-04-25T00:40:32.122767+0000","last_undegraded":"2024-04-25T00:40:32.122767+0000","last_fullsized":"2024-04-25T00:40:32.122767+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:54.538024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.113034+0000","last_change":"2024-04-25T00:40:30.110598+0000","last_active":"2024-04-25T00:40:31.113034+0000","last_peered":"2024-04-25T00:40:31.113034+0000","last_clean":"2024-04-25T00:40:31.113034+0000","last_became_active":"2024-04-25T00:40:30.109826+0000","last_became_peered":"2024-04-25T00:40:30.109826+0000","last_unstale":"2024-04-25T00:40:31.113034+0000","last_undegraded":"2024-04-25T00:40:31.113034+0000","last_fullsized":"2024-04-25T00:40:31.113034+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:16.758479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327463+0000","last_change":"2024-04-25T00:40:30.109734+0000","last_active":"2024-04-25T00:40:31.327463+0000","last_peered":"2024-04-25T00:40:31.327463+0000","last_clean":"2024-04-25T00:40:31.327463+0000","last_became_active":"2024-04-25T00:40:30.109492+0000","last_became_peered":"2024-04-25T00:40:30.109492+0000","last_unstale":"2024-04-25T00:40:31.327463+0000","last_undegraded":"2024-04-25T00:40:31.327463+0000","last_fullsized":"2024-04-25T00:40:31.327463+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:38.456637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.112145+0000","last_change":"2024-04-25T00:40:30.095770+0000","last_active":"2024-04-25T00:40:31.112145+0000","last_peered":"2024-04-25T00:40:31.112145+0000","last_clean":"2024-04-25T00:40:31.112145+0000","last_became_active":"2024-04-25T00:40:30.095052+0000","last_became_peered":"2024-04-25T00:40:30.095052+0000","last_unstale":"2024-04-25T00:40:31.112145+0000","last_undegraded":"2024-04-25T00:40:31.112145+0000","last_fullsized":"2024-04-25T00:40:31.112145+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:28.490785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.127300+0000","last_change":"2024-04-25T00:40:30.114507+0000","last_active":"2024-04-25T00:40:32.127300+0000","last_peered":"2024-04-25T00:40:32.127300+0000","last_clean":"2024-04-25T00:40:32.127300+0000","last_became_active":"2024-04-25T00:40:30.113692+0000","last_became_peered":"2024-04-25T00:40:30.113692+0000","last_unstale":"2024-04-25T00:40:32.127300+0000","last_undegraded":"2024-04-25T00:40:32.127300+0000","last_fullsized":"2024-04-25T00:40:32.127300+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:41.286811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.126843+0000","last_change":"2024-04-25T00:40:30.133750+0000","last_active":"2024-04-25T00:40:32.126843+0000","last_peered":"2024-04-25T00:40:32.126843+0000","last_clean":"2024-04-25T00:40:32.126843+0000","last_became_active":"2024-04-25T00:40:30.133095+0000","last_became_peered":"2024-04-25T00:40:30.133095+0000","last_unstale":"2024-04-25T00:40:32.126843+0000","last_undegraded":"2024-04-25T00:40:32.126843+0000","last_fullsized":"2024-04-25T00:40:32.126843+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:11.813161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111579+0000","last_change":"2024-04-25T00:40:30.111376+0000","last_active":"2024-04-25T00:40:31.111579+0000","last_peered":"2024-04-25T00:40:31.111579+0000","last_clean":"2024-04-25T00:40:31.111579+0000","last_became_active":"2024-04-25T00:40:30.110992+0000","last_became_peered":"2024-04-25T00:40:30.110992+0000","last_unstale":"2024-04-25T00:40:31.111579+0000","last_undegraded":"2024-04-25T00:40:31.111579+0000","last_fullsized":"2024-04-25T00:40:31.111579+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:16.371328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.126698+0000","last_change":"2024-04-25T00:40:30.125877+0000","last_active":"2024-04-25T00:40:32.126698+0000","last_peered":"2024-04-25T00:40:32.126698+0000","last_clean":"2024-04-25T00:40:32.126698+0000","last_became_active":"2024-04-25T00:40:30.109087+0000","last_became_peered":"2024-04-25T00:40:30.109087+0000","last_unstale":"2024-04-25T00:40:32.126698+0000","last_undegraded":"2024-04-25T00:40:32.126698+0000","last_fullsized":"2024-04-25T00:40:32.126698+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:21.024511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123593+0000","last_change":"2024-04-25T00:40:30.115666+0000","last_active":"2024-04-25T00:40:32.123593+0000","last_peered":"2024-04-25T00:40:32.123593+0000","last_clean":"2024-04-25T00:40:32.123593+0000","last_became_active":"2024-04-25T00:40:30.115335+0000","last_became_peered":"2024-04-25T00:40:30.115335+0000","last_unstale":"2024-04-25T00:40:32.123593+0000","last_undegraded":"2024-04-25T00:40:32.123593+0000","last_fullsized":"2024-04-25T00:40:32.123593+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:49.124306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533179+0000","last_change":"2024-04-25T00:40:30.110030+0000","last_active":"2024-04-25T00:40:31.533179+0000","last_peered":"2024-04-25T00:40:31.533179+0000","last_clean":"2024-04-25T00:40:31.533179+0000","last_became_active":"2024-04-25T00:40:30.109685+0000","last_became_peered":"2024-04-25T00:40:30.109685+0000","last_unstale":"2024-04-25T00:40:31.533179+0000","last_undegraded":"2024-04-25T00:40:31.533179+0000","last_fullsized":"2024-04-25T00:40:31.533179+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:58.444761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109382+0000","last_change":"2024-04-25T00:40:30.105168+0000","last_active":"2024-04-25T00:40:31.109382+0000","last_peered":"2024-04-25T00:40:31.109382+0000","last_clean":"2024-04-25T00:40:31.109382+0000","last_became_active":"2024-04-25T00:40:30.104401+0000","last_became_peered":"2024-04-25T00:40:30.104401+0000","last_unstale":"2024-04-25T00:40:31.109382+0000","last_undegraded":"2024-04-25T00:40:31.109382+0000","last_fullsized":"2024-04-25T00:40:31.109382+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:49.124306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327133+0000","last_change":"2024-04-25T00:40:30.116393+0000","last_active":"2024-04-25T00:40:31.327133+0000","last_peered":"2024-04-25T00:40:31.327133+0000","last_clean":"2024-04-25T00:40:31.327133+0000","last_became_active":"2024-04-25T00:40:30.115406+0000","last_became_peered":"2024-04-25T00:40:30.115406+0000","last_unstale":"2024-04-25T00:40:31.327133+0000","last_undegraded":"2024-04-25T00:40:31.327133+0000","last_fullsized":"2024-04-25T00:40:31.327133+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:49.124306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533095+0000","last_change":"2024-04-25T00:40:30.119807+0000","last_active":"2024-04-25T00:40:31.533095+0000","last_peered":"2024-04-25T00:40:31.533095+0000","last_clean":"2024-04-25T00:40:31.533095+0000","last_became_active":"2024-04-25T00:40:30.119510+0000","last_became_peered":"2024-04-25T00:40:30.119510+0000","last_unstale":"2024-04-25T00:40:31.533095+0000","last_undegraded":"2024-04-25T00:40:31.533095+0000","last_fullsized":"2024-04-25T00:40:31.533095+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:39.713681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533044+0000","last_change":"2024-04-25T00:40:30.106305+0000","last_active":"2024-04-25T00:40:31.533044+0000","last_peered":"2024-04-25T00:40:31.533044+0000","last_clean":"2024-04-25T00:40:31.533044+0000","last_became_active":"2024-04-25T00:40:30.104321+0000","last_became_peered":"2024-04-25T00:40:30.104321+0000","last_unstale":"2024-04-25T00:40:31.533044+0000","last_undegraded":"2024-04-25T00:40:31.533044+0000","last_fullsized":"2024-04-25T00:40:31.533044+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:56.242863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327402+0000","last_change":"2024-04-25T00:40:30.112605+0000","last_active":"2024-04-25T00:40:31.327402+0000","last_peered":"2024-04-25T00:40:31.327402+0000","last_clean":"2024-04-25T00:40:31.327402+0000","last_became_active":"2024-04-25T00:40:30.112325+0000","last_became_peered":"2024-04-25T00:40:30.112325+0000","last_unstale":"2024-04-25T00:40:31.327402+0000","last_undegraded":"2024-04-25T00:40:31.327402+0000","last_fullsized":"2024-04-25T00:40:31.327402+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:25.442863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109793+0000","last_change":"2024-04-25T00:40:30.106950+0000","last_active":"2024-04-25T00:40:31.109793+0000","last_peered":"2024-04-25T00:40:31.109793+0000","last_clean":"2024-04-25T00:40:31.109793+0000","last_became_active":"2024-04-25T00:40:30.106672+0000","last_became_peered":"2024-04-25T00:40:30.106672+0000","last_unstale":"2024-04-25T00:40:31.109793+0000","last_undegraded":"2024-04-25T00:40:31.109793+0000","last_fullsized":"2024-04-25T00:40:31.109793+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:36.066823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123130+0000","last_change":"2024-04-25T00:40:30.103963+0000","last_active":"2024-04-25T00:40:32.123130+0000","last_peered":"2024-04-25T00:40:32.123130+0000","last_clean":"2024-04-25T00:40:32.123130+0000","last_became_active":"2024-04-25T00:40:30.103476+0000","last_became_peered":"2024-04-25T00:40:30.103476+0000","last_unstale":"2024-04-25T00:40:32.123130+0000","last_undegraded":"2024-04-25T00:40:32.123130+0000","last_fullsized":"2024-04-25T00:40:32.123130+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:42.110955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.127872+0000","last_change":"2024-04-25T00:40:30.126333+0000","last_active":"2024-04-25T00:40:32.127872+0000","last_peered":"2024-04-25T00:40:32.127872+0000","last_clean":"2024-04-25T00:40:32.127872+0000","last_became_active":"2024-04-25T00:40:30.125851+0000","last_became_peered":"2024-04-25T00:40:30.125851+0000","last_unstale":"2024-04-25T00:40:32.127872+0000","last_undegraded":"2024-04-25T00:40:32.127872+0000","last_fullsized":"2024-04-25T00:40:32.127872+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:36.066823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111800+0000","last_change":"2024-04-25T00:40:30.129130+0000","last_active":"2024-04-25T00:40:31.111800+0000","last_peered":"2024-04-25T00:40:31.111800+0000","last_clean":"2024-04-25T00:40:31.111800+0000","last_became_active":"2024-04-25T00:40:30.113758+0000","last_became_peered":"2024-04-25T00:40:30.113758+0000","last_unstale":"2024-04-25T00:40:31.111800+0000","last_undegraded":"2024-04-25T00:40:31.111800+0000","last_fullsized":"2024-04-25T00:40:31.111800+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:05.372029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109368+0000","last_change":"2024-04-25T00:40:30.105721+0000","last_active":"2024-04-25T00:40:31.109368+0000","last_peered":"2024-04-25T00:40:31.109368+0000","last_clean":"2024-04-25T00:40:31.109368+0000","last_became_active":"2024-04-25T00:40:30.105374+0000","last_became_peered":"2024-04-25T00:40:30.105374+0000","last_unstale":"2024-04-25T00:40:31.109368+0000","last_undegraded":"2024-04-25T00:40:31.109368+0000","last_fullsized":"2024-04-25T00:40:31.109368+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:14.368023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533317+0000","last_change":"2024-04-25T00:40:30.109593+0000","last_active":"2024-04-25T00:40:31.533317+0000","last_peered":"2024-04-25T00:40:31.533317+0000","last_clean":"2024-04-25T00:40:31.533317+0000","last_became_active":"2024-04-25T00:40:30.109276+0000","last_became_peered":"2024-04-25T00:40:30.109276+0000","last_unstale":"2024-04-25T00:40:31.533317+0000","last_undegraded":"2024-04-25T00:40:31.533317+0000","last_fullsized":"2024-04-25T00:40:31.533317+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:50.537642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.126845+0000","last_change":"2024-04-25T00:40:30.107733+0000","last_active":"2024-04-25T00:40:32.126845+0000","last_peered":"2024-04-25T00:40:32.126845+0000","last_clean":"2024-04-25T00:40:32.126845+0000","last_became_active":"2024-04-25T00:40:30.107062+0000","last_became_peered":"2024-04-25T00:40:30.107062+0000","last_unstale":"2024-04-25T00:40:32.126845+0000","last_undegraded":"2024-04-25T00:40:32.126845+0000","last_fullsized":"2024-04-25T00:40:32.126845+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:41:21.636262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111993+0000","last_change":"2024-04-25T00:40:30.099120+0000","last_active":"2024-04-25T00:40:31.111993+0000","last_peered":"2024-04-25T00:40:31.111993+0000","last_clean":"2024-04-25T00:40:31.111993+0000","last_became_active":"2024-04-25T00:40:30.098319+0000","last_became_peered":"2024-04-25T00:40:30.098319+0000","last_unstale":"2024-04-25T00:40:31.111993+0000","last_undegraded":"2024-04-25T00:40:31.111993+0000","last_fullsized":"2024-04-25T00:40:31.111993+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:02.179108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109175+0000","last_change":"2024-04-25T00:40:30.115433+0000","last_active":"2024-04-25T00:40:31.109175+0000","last_peered":"2024-04-25T00:40:31.109175+0000","last_clean":"2024-04-25T00:40:31.109175+0000","last_became_active":"2024-04-25T00:40:30.114982+0000","last_became_peered":"2024-04-25T00:40:30.114982+0000","last_unstale":"2024-04-25T00:40:31.109175+0000","last_undegraded":"2024-04-25T00:40:31.109175+0000","last_fullsized":"2024-04-25T00:40:31.109175+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:41:21.636262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532379+0000","last_change":"2024-04-25T00:40:30.130523+0000","last_active":"2024-04-25T00:40:31.532379+0000","last_peered":"2024-04-25T00:40:31.532379+0000","last_clean":"2024-04-25T00:40:31.532379+0000","last_became_active":"2024-04-25T00:40:30.130164+0000","last_became_peered":"2024-04-25T00:40:30.130164+0000","last_unstale":"2024-04-25T00:40:31.532379+0000","last_undegraded":"2024-04-25T00:40:31.532379+0000","last_fullsized":"2024-04-25T00:40:31.532379+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:57.823829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.124073+0000","last_change":"2024-04-25T00:40:30.115851+0000","last_active":"2024-04-25T00:40:32.124073+0000","last_peered":"2024-04-25T00:40:32.124073+0000","last_clean":"2024-04-25T00:40:32.124073+0000","last_became_active":"2024-04-25T00:40:30.115406+0000","last_became_peered":"2024-04-25T00:40:30.115406+0000","last_unstale":"2024-04-25T00:40:32.124073+0000","last_undegraded":"2024-04-25T00:40:32.124073+0000","last_fullsized":"2024-04-25T00:40:32.124073+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:31.042850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"155'4","reported_seq":18,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533065+0000","last_change":"2024-04-25T00:40:29.094752+0000","last_active":"2024-04-25T00:40:31.533065+0000","last_peered":"2024-04-25T00:40:31.533065+0000","last_clean":"2024-04-25T00:40:31.533065+0000","last_became_active":"2024-04-25T00:40:29.094472+0000","last_became_peered":"2024-04-25T00:40:29.094472+0000","last_unstale":"2024-04-25T00:40:31.533065+0000","last_undegraded":"2024-04-25T00:40:31.533065+0000","last_fullsized":"2024-04-25T00:40:31.533065+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:42.622353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.328400+0000","last_change":"2024-04-25T00:40:30.114712+0000","last_active":"2024-04-25T00:40:31.328400+0000","last_peered":"2024-04-25T00:40:31.328400+0000","last_clean":"2024-04-25T00:40:31.328400+0000","last_became_active":"2024-04-25T00:40:30.114451+0000","last_became_peered":"2024-04-25T00:40:30.114451+0000","last_unstale":"2024-04-25T00:40:31.328400+0000","last_undegraded":"2024-04-25T00:40:31.328400+0000","last_fullsized":"2024-04-25T00:40:31.328400+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:41:59.378161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.112120+0000","last_change":"2024-04-25T00:40:29.085706+0000","last_active":"2024-04-25T00:40:31.112120+0000","last_peered":"2024-04-25T00:40:31.112120+0000","last_clean":"2024-04-25T00:40:31.112120+0000","last_became_active":"2024-04-25T00:40:29.084970+0000","last_became_peered":"2024-04-25T00:40:29.084970+0000","last_unstale":"2024-04-25T00:40:31.112120+0000","last_undegraded":"2024-04-25T00:40:31.112120+0000","last_fullsized":"2024-04-25T00:40:31.112120+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:48.057565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111179+0000","last_change":"2024-04-25T00:40:30.113159+0000","last_active":"2024-04-25T00:40:31.111179+0000","last_peered":"2024-04-25T00:40:31.111179+0000","last_clean":"2024-04-25T00:40:31.111179+0000","last_became_active":"2024-04-25T00:40:30.112489+0000","last_became_peered":"2024-04-25T00:40:30.112489+0000","last_unstale":"2024-04-25T00:40:31.111179+0000","last_undegraded":"2024-04-25T00:40:31.111179+0000","last_fullsized":"2024-04-25T00:40:31.111179+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:01.301393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109471+0000","last_change":"2024-04-25T00:40:29.098108+0000","last_active":"2024-04-25T00:40:31.109471+0000","last_peered":"2024-04-25T00:40:31.109471+0000","last_clean":"2024-04-25T00:40:31.109471+0000","last_became_active":"2024-04-25T00:40:29.097478+0000","last_became_peered":"2024-04-25T00:40:29.097478+0000","last_unstale":"2024-04-25T00:40:31.109471+0000","last_undegraded":"2024-04-25T00:40:31.109471+0000","last_fullsized":"2024-04-25T00:40:31.109471+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:43.160214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.112458+0000","last_change":"2024-04-25T00:40:30.132249+0000","last_active":"2024-04-25T00:40:31.112458+0000","last_peered":"2024-04-25T00:40:31.112458+0000","last_clean":"2024-04-25T00:40:31.112458+0000","last_became_active":"2024-04-25T00:40:30.131607+0000","last_became_peered":"2024-04-25T00:40:30.131607+0000","last_unstale":"2024-04-25T00:40:31.112458+0000","last_undegraded":"2024-04-25T00:40:31.112458+0000","last_fullsized":"2024-04-25T00:40:31.112458+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:44.170703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.127403+0000","last_change":"2024-04-25T00:40:29.096877+0000","last_active":"2024-04-25T00:40:32.127403+0000","last_peered":"2024-04-25T00:40:32.127403+0000","last_clean":"2024-04-25T00:40:32.127403+0000","last_became_active":"2024-04-25T00:40:29.096218+0000","last_became_peered":"2024-04-25T00:40:29.096218+0000","last_unstale":"2024-04-25T00:40:32.127403+0000","last_undegraded":"2024-04-25T00:40:32.127403+0000","last_fullsized":"2024-04-25T00:40:32.127403+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:15.360840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.326938+0000","last_change":"2024-04-25T00:40:30.113296+0000","last_active":"2024-04-25T00:40:31.326938+0000","last_peered":"2024-04-25T00:40:31.326938+0000","last_clean":"2024-04-25T00:40:31.326938+0000","last_became_active":"2024-04-25T00:40:30.113042+0000","last_became_peered":"2024-04-25T00:40:30.113042+0000","last_unstale":"2024-04-25T00:40:31.326938+0000","last_undegraded":"2024-04-25T00:40:31.326938+0000","last_fullsized":"2024-04-25T00:40:31.326938+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:51.016368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532876+0000","last_change":"2024-04-25T00:40:29.087901+0000","last_active":"2024-04-25T00:40:31.532876+0000","last_peered":"2024-04-25T00:40:31.532876+0000","last_clean":"2024-04-25T00:40:31.532876+0000","last_became_active":"2024-04-25T00:40:29.087264+0000","last_became_peered":"2024-04-25T00:40:29.087264+0000","last_unstale":"2024-04-25T00:40:31.532876+0000","last_undegraded":"2024-04-25T00:40:31.532876+0000","last_fullsized":"2024-04-25T00:40:31.532876+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:30.526164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123443+0000","last_change":"2024-04-25T00:40:30.115689+0000","last_active":"2024-04-25T00:40:32.123443+0000","last_peered":"2024-04-25T00:40:32.123443+0000","last_clean":"2024-04-25T00:40:32.123443+0000","last_became_active":"2024-04-25T00:40:30.115014+0000","last_became_peered":"2024-04-25T00:40:30.115014+0000","last_unstale":"2024-04-25T00:40:32.123443+0000","last_undegraded":"2024-04-25T00:40:32.123443+0000","last_fullsized":"2024-04-25T00:40:32.123443+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:51.016368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123444+0000","last_change":"2024-04-25T00:40:29.096769+0000","last_active":"2024-04-25T00:40:32.123444+0000","last_peered":"2024-04-25T00:40:32.123444+0000","last_clean":"2024-04-25T00:40:32.123444+0000","last_became_active":"2024-04-25T00:40:29.096339+0000","last_became_peered":"2024-04-25T00:40:29.096339+0000","last_unstale":"2024-04-25T00:40:32.123444+0000","last_undegraded":"2024-04-25T00:40:32.123444+0000","last_fullsized":"2024-04-25T00:40:32.123444+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:23.222465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.110010+0000","last_change":"2024-04-25T00:40:30.112484+0000","last_active":"2024-04-25T00:40:31.110010+0000","last_peered":"2024-04-25T00:40:31.110010+0000","last_clean":"2024-04-25T00:40:31.110010+0000","last_became_active":"2024-04-25T00:40:30.096322+0000","last_became_peered":"2024-04-25T00:40:30.096322+0000","last_unstale":"2024-04-25T00:40:31.110010+0000","last_undegraded":"2024-04-25T00:40:31.110010+0000","last_fullsized":"2024-04-25T00:40:31.110010+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:07.296465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.106816+0000","last_change":"2024-04-25T00:40:29.082537+0000","last_active":"2024-04-25T00:40:31.106816+0000","last_peered":"2024-04-25T00:40:31.106816+0000","last_clean":"2024-04-25T00:40:31.106816+0000","last_became_active":"2024-04-25T00:40:29.081768+0000","last_became_peered":"2024-04-25T00:40:29.081768+0000","last_unstale":"2024-04-25T00:40:31.106816+0000","last_undegraded":"2024-04-25T00:40:31.106816+0000","last_fullsized":"2024-04-25T00:40:31.106816+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:55.940190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109505+0000","last_change":"2024-04-25T00:40:30.112614+0000","last_active":"2024-04-25T00:40:31.109505+0000","last_peered":"2024-04-25T00:40:31.109505+0000","last_clean":"2024-04-25T00:40:31.109505+0000","last_became_active":"2024-04-25T00:40:30.112312+0000","last_became_peered":"2024-04-25T00:40:30.112312+0000","last_unstale":"2024-04-25T00:40:31.109505+0000","last_undegraded":"2024-04-25T00:40:31.109505+0000","last_fullsized":"2024-04-25T00:40:31.109505+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:28.342905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111523+0000","last_change":"2024-04-25T00:40:29.091407+0000","last_active":"2024-04-25T00:40:31.111523+0000","last_peered":"2024-04-25T00:40:31.111523+0000","last_clean":"2024-04-25T00:40:31.111523+0000","last_became_active":"2024-04-25T00:40:29.090686+0000","last_became_peered":"2024-04-25T00:40:29.090686+0000","last_unstale":"2024-04-25T00:40:31.111523+0000","last_undegraded":"2024-04-25T00:40:31.111523+0000","last_fullsized":"2024-04-25T00:40:31.111523+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:45.643415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123150+0000","last_change":"2024-04-25T00:40:30.118062+0000","last_active":"2024-04-25T00:40:32.123150+0000","last_peered":"2024-04-25T00:40:32.123150+0000","last_clean":"2024-04-25T00:40:32.123150+0000","last_became_active":"2024-04-25T00:40:30.117710+0000","last_became_peered":"2024-04-25T00:40:30.117710+0000","last_unstale":"2024-04-25T00:40:32.123150+0000","last_undegraded":"2024-04-25T00:40:32.123150+0000","last_fullsized":"2024-04-25T00:40:32.123150+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:51.726915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533110+0000","last_change":"2024-04-25T00:40:29.100639+0000","last_active":"2024-04-25T00:40:31.533110+0000","last_peered":"2024-04-25T00:40:31.533110+0000","last_clean":"2024-04-25T00:40:31.533110+0000","last_became_active":"2024-04-25T00:40:29.100348+0000","last_became_peered":"2024-04-25T00:40:29.100348+0000","last_unstale":"2024-04-25T00:40:31.533110+0000","last_undegraded":"2024-04-25T00:40:31.533110+0000","last_fullsized":"2024-04-25T00:40:31.533110+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:55.842338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111373+0000","last_change":"2024-04-25T00:40:30.136453+0000","last_active":"2024-04-25T00:40:31.111373+0000","last_peered":"2024-04-25T00:40:31.111373+0000","last_clean":"2024-04-25T00:40:31.111373+0000","last_became_active":"2024-04-25T00:40:30.135882+0000","last_became_peered":"2024-04-25T00:40:30.135882+0000","last_unstale":"2024-04-25T00:40:31.111373+0000","last_undegraded":"2024-04-25T00:40:31.111373+0000","last_fullsized":"2024-04-25T00:40:31.111373+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:09.079933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"155'3","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532724+0000","last_change":"2024-04-25T00:40:29.088110+0000","last_active":"2024-04-25T00:40:31.532724+0000","last_peered":"2024-04-25T00:40:31.532724+0000","last_clean":"2024-04-25T00:40:31.532724+0000","last_became_active":"2024-04-25T00:40:29.087595+0000","last_became_peered":"2024-04-25T00:40:29.087595+0000","last_unstale":"2024-04-25T00:40:31.532724+0000","last_undegraded":"2024-04-25T00:40:31.532724+0000","last_fullsized":"2024-04-25T00:40:31.532724+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:23.704167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327252+0000","last_change":"2024-04-25T00:40:30.108118+0000","last_active":"2024-04-25T00:40:31.327252+0000","last_peered":"2024-04-25T00:40:31.327252+0000","last_clean":"2024-04-25T00:40:31.327252+0000","last_became_active":"2024-04-25T00:40:30.107850+0000","last_became_peered":"2024-04-25T00:40:30.107850+0000","last_unstale":"2024-04-25T00:40:31.327252+0000","last_undegraded":"2024-04-25T00:40:31.327252+0000","last_fullsized":"2024-04-25T00:40:31.327252+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:39.019304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"155'6","reported_seq":20,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109260+0000","last_change":"2024-04-25T00:40:29.099633+0000","last_active":"2024-04-25T00:40:31.109260+0000","last_peered":"2024-04-25T00:40:31.109260+0000","last_clean":"2024-04-25T00:40:31.109260+0000","last_became_active":"2024-04-25T00:40:29.099060+0000","last_became_peered":"2024-04-25T00:40:29.099060+0000","last_unstale":"2024-04-25T00:40:31.109260+0000","last_undegraded":"2024-04-25T00:40:31.109260+0000","last_fullsized":"2024-04-25T00:40:31.109260+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:01.168619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109428+0000","last_change":"2024-04-25T00:40:30.110005+0000","last_active":"2024-04-25T00:40:31.109428+0000","last_peered":"2024-04-25T00:40:31.109428+0000","last_clean":"2024-04-25T00:40:31.109428+0000","last_became_active":"2024-04-25T00:40:30.109578+0000","last_became_peered":"2024-04-25T00:40:30.109578+0000","last_unstale":"2024-04-25T00:40:31.109428+0000","last_undegraded":"2024-04-25T00:40:31.109428+0000","last_fullsized":"2024-04-25T00:40:31.109428+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:07.296465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.108569+0000","last_change":"2024-04-25T00:40:29.088653+0000","last_active":"2024-04-25T00:40:31.108569+0000","last_peered":"2024-04-25T00:40:31.108569+0000","last_clean":"2024-04-25T00:40:31.108569+0000","last_became_active":"2024-04-25T00:40:29.087923+0000","last_became_peered":"2024-04-25T00:40:29.087923+0000","last_unstale":"2024-04-25T00:40:31.108569+0000","last_undegraded":"2024-04-25T00:40:31.108569+0000","last_fullsized":"2024-04-25T00:40:31.108569+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:10.274642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.110246+0000","last_change":"2024-04-25T00:40:30.128973+0000","last_active":"2024-04-25T00:40:31.110246+0000","last_peered":"2024-04-25T00:40:31.110246+0000","last_clean":"2024-04-25T00:40:31.110246+0000","last_became_active":"2024-04-25T00:40:30.113153+0000","last_became_peered":"2024-04-25T00:40:30.113153+0000","last_unstale":"2024-04-25T00:40:31.110246+0000","last_undegraded":"2024-04-25T00:40:31.110246+0000","last_fullsized":"2024-04-25T00:40:31.110246+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:28.342905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327629+0000","last_change":"2024-04-25T00:40:29.088591+0000","last_active":"2024-04-25T00:40:31.327629+0000","last_peered":"2024-04-25T00:40:31.327629+0000","last_clean":"2024-04-25T00:40:31.327629+0000","last_became_active":"2024-04-25T00:40:29.088270+0000","last_became_peered":"2024-04-25T00:40:29.088270+0000","last_unstale":"2024-04-25T00:40:31.327629+0000","last_undegraded":"2024-04-25T00:40:31.327629+0000","last_fullsized":"2024-04-25T00:40:31.327629+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:23.222465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111219+0000","last_change":"2024-04-25T00:40:30.118420+0000","last_active":"2024-04-25T00:40:31.111219+0000","last_peered":"2024-04-25T00:40:31.111219+0000","last_clean":"2024-04-25T00:40:31.111219+0000","last_became_active":"2024-04-25T00:40:30.117750+0000","last_became_peered":"2024-04-25T00:40:30.117750+0000","last_unstale":"2024-04-25T00:40:31.111219+0000","last_undegraded":"2024-04-25T00:40:31.111219+0000","last_fullsized":"2024-04-25T00:40:31.111219+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:51.453502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.122687+0000","last_change":"2024-04-25T00:40:29.100762+0000","last_active":"2024-04-25T00:40:32.122687+0000","last_peered":"2024-04-25T00:40:32.122687+0000","last_clean":"2024-04-25T00:40:32.122687+0000","last_became_active":"2024-04-25T00:40:29.100159+0000","last_became_peered":"2024-04-25T00:40:29.100159+0000","last_unstale":"2024-04-25T00:40:32.122687+0000","last_undegraded":"2024-04-25T00:40:32.122687+0000","last_fullsized":"2024-04-25T00:40:32.122687+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:13.385254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.326896+0000","last_change":"2024-04-25T00:40:30.137144+0000","last_active":"2024-04-25T00:40:31.326896+0000","last_peered":"2024-04-25T00:40:31.326896+0000","last_clean":"2024-04-25T00:40:31.326896+0000","last_became_active":"2024-04-25T00:40:30.136889+0000","last_became_peered":"2024-04-25T00:40:30.136889+0000","last_unstale":"2024-04-25T00:40:31.326896+0000","last_undegraded":"2024-04-25T00:40:31.326896+0000","last_fullsized":"2024-04-25T00:40:31.326896+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:06.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"155'8","reported_seq":22,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.106984+0000","last_change":"2024-04-25T00:40:29.089205+0000","last_active":"2024-04-25T00:40:31.106984+0000","last_peered":"2024-04-25T00:40:31.106984+0000","last_clean":"2024-04-25T00:40:31.106984+0000","last_became_active":"2024-04-25T00:40:29.088497+0000","last_became_peered":"2024-04-25T00:40:29.088497+0000","last_unstale":"2024-04-25T00:40:31.106984+0000","last_undegraded":"2024-04-25T00:40:31.106984+0000","last_fullsized":"2024-04-25T00:40:31.106984+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:10.767826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327704+0000","last_change":"2024-04-25T00:40:30.137077+0000","last_active":"2024-04-25T00:40:31.327704+0000","last_peered":"2024-04-25T00:40:31.327704+0000","last_clean":"2024-04-25T00:40:31.327704+0000","last_became_active":"2024-04-25T00:40:30.136823+0000","last_became_peered":"2024-04-25T00:40:30.136823+0000","last_unstale":"2024-04-25T00:40:31.327704+0000","last_undegraded":"2024-04-25T00:40:31.327704+0000","last_fullsized":"2024-04-25T00:40:31.327704+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:21.729541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533413+0000","last_change":"2024-04-25T00:40:29.093336+0000","last_active":"2024-04-25T00:40:31.533413+0000","last_peered":"2024-04-25T00:40:31.533413+0000","last_clean":"2024-04-25T00:40:31.533413+0000","last_became_active":"2024-04-25T00:40:29.092530+0000","last_became_peered":"2024-04-25T00:40:29.092530+0000","last_unstale":"2024-04-25T00:40:31.533413+0000","last_undegraded":"2024-04-25T00:40:31.533413+0000","last_fullsized":"2024-04-25T00:40:31.533413+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:06.422475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.113226+0000","last_change":"2024-04-25T00:40:30.119303+0000","last_active":"2024-04-25T00:40:31.113226+0000","last_peered":"2024-04-25T00:40:31.113226+0000","last_clean":"2024-04-25T00:40:31.113226+0000","last_became_active":"2024-04-25T00:40:30.118527+0000","last_became_peered":"2024-04-25T00:40:30.118527+0000","last_unstale":"2024-04-25T00:40:31.113226+0000","last_undegraded":"2024-04-25T00:40:31.113226+0000","last_fullsized":"2024-04-25T00:40:31.113226+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:25.494199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.124155+0000","last_change":"2024-04-25T00:40:29.090555+0000","last_active":"2024-04-25T00:40:32.124155+0000","last_peered":"2024-04-25T00:40:32.124155+0000","last_clean":"2024-04-25T00:40:32.124155+0000","last_became_active":"2024-04-25T00:40:29.089983+0000","last_became_peered":"2024-04-25T00:40:29.089983+0000","last_unstale":"2024-04-25T00:40:32.124155+0000","last_undegraded":"2024-04-25T00:40:32.124155+0000","last_fullsized":"2024-04-25T00:40:32.124155+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:11.663375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.112152+0000","last_change":"2024-04-25T00:40:30.096459+0000","last_active":"2024-04-25T00:40:31.112152+0000","last_peered":"2024-04-25T00:40:31.112152+0000","last_clean":"2024-04-25T00:40:31.112152+0000","last_became_active":"2024-04-25T00:40:30.095676+0000","last_became_peered":"2024-04-25T00:40:30.095676+0000","last_unstale":"2024-04-25T00:40:31.112152+0000","last_undegraded":"2024-04-25T00:40:31.112152+0000","last_fullsized":"2024-04-25T00:40:31.112152+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:47.114707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109123+0000","last_change":"2024-04-25T00:40:29.088835+0000","last_active":"2024-04-25T00:40:31.109123+0000","last_peered":"2024-04-25T00:40:31.109123+0000","last_clean":"2024-04-25T00:40:31.109123+0000","last_became_active":"2024-04-25T00:40:29.088099+0000","last_became_peered":"2024-04-25T00:40:29.088099+0000","last_unstale":"2024-04-25T00:40:31.109123+0000","last_undegraded":"2024-04-25T00:40:31.109123+0000","last_fullsized":"2024-04-25T00:40:31.109123+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:45:13.430795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.127103+0000","last_change":"2024-04-25T00:40:30.119352+0000","last_active":"2024-04-25T00:40:32.127103+0000","last_peered":"2024-04-25T00:40:32.127103+0000","last_clean":"2024-04-25T00:40:32.127103+0000","last_became_active":"2024-04-25T00:40:30.118647+0000","last_became_peered":"2024-04-25T00:40:30.118647+0000","last_unstale":"2024-04-25T00:40:32.127103+0000","last_undegraded":"2024-04-25T00:40:32.127103+0000","last_fullsized":"2024-04-25T00:40:32.127103+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:07.296465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111457+0000","last_change":"2024-04-25T00:40:29.095787+0000","last_active":"2024-04-25T00:40:31.111457+0000","last_peered":"2024-04-25T00:40:31.111457+0000","last_clean":"2024-04-25T00:40:31.111457+0000","last_became_active":"2024-04-25T00:40:29.094547+0000","last_became_peered":"2024-04-25T00:40:29.094547+0000","last_unstale":"2024-04-25T00:40:31.111457+0000","last_undegraded":"2024-04-25T00:40:31.111457+0000","last_fullsized":"2024-04-25T00:40:31.111457+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:24.483710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327374+0000","last_change":"2024-04-25T00:40:30.109213+0000","last_active":"2024-04-25T00:40:31.327374+0000","last_peered":"2024-04-25T00:40:31.327374+0000","last_clean":"2024-04-25T00:40:31.327374+0000","last_became_active":"2024-04-25T00:40:30.108785+0000","last_became_peered":"2024-04-25T00:40:30.108785+0000","last_unstale":"2024-04-25T00:40:31.327374+0000","last_undegraded":"2024-04-25T00:40:31.327374+0000","last_fullsized":"2024-04-25T00:40:31.327374+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:31.042850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109143+0000","last_change":"2024-04-25T00:40:29.095275+0000","last_active":"2024-04-25T00:40:31.109143+0000","last_peered":"2024-04-25T00:40:31.109143+0000","last_clean":"2024-04-25T00:40:31.109143+0000","last_became_active":"2024-04-25T00:40:29.094973+0000","last_became_peered":"2024-04-25T00:40:29.094973+0000","last_unstale":"2024-04-25T00:40:31.109143+0000","last_undegraded":"2024-04-25T00:40:31.109143+0000","last_fullsized":"2024-04-25T00:40:31.109143+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:50.443013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.110421+0000","last_change":"2024-04-25T00:40:30.129513+0000","last_active":"2024-04-25T00:40:31.110421+0000","last_peered":"2024-04-25T00:40:31.110421+0000","last_clean":"2024-04-25T00:40:31.110421+0000","last_became_active":"2024-04-25T00:40:30.113812+0000","last_became_peered":"2024-04-25T00:40:30.113812+0000","last_unstale":"2024-04-25T00:40:31.110421+0000","last_undegraded":"2024-04-25T00:40:31.110421+0000","last_fullsized":"2024-04-25T00:40:31.110421+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:36.066823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532754+0000","last_change":"2024-04-25T00:40:29.099837+0000","last_active":"2024-04-25T00:40:31.532754+0000","last_peered":"2024-04-25T00:40:31.532754+0000","last_clean":"2024-04-25T00:40:31.532754+0000","last_became_active":"2024-04-25T00:40:29.096352+0000","last_became_peered":"2024-04-25T00:40:29.096352+0000","last_unstale":"2024-04-25T00:40:31.532754+0000","last_undegraded":"2024-04-25T00:40:31.532754+0000","last_fullsized":"2024-04-25T00:40:31.532754+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:04.477309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.126260+0000","last_change":"2024-04-25T00:40:30.109579+0000","last_active":"2024-04-25T00:40:32.126260+0000","last_peered":"2024-04-25T00:40:32.126260+0000","last_clean":"2024-04-25T00:40:32.126260+0000","last_became_active":"2024-04-25T00:40:30.108793+0000","last_became_peered":"2024-04-25T00:40:30.108793+0000","last_unstale":"2024-04-25T00:40:32.126260+0000","last_undegraded":"2024-04-25T00:40:32.126260+0000","last_fullsized":"2024-04-25T00:40:32.126260+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:43.387904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109271+0000","last_change":"2024-04-25T00:40:29.097310+0000","last_active":"2024-04-25T00:40:31.109271+0000","last_peered":"2024-04-25T00:40:31.109271+0000","last_clean":"2024-04-25T00:40:31.109271+0000","last_became_active":"2024-04-25T00:40:29.096881+0000","last_became_peered":"2024-04-25T00:40:29.096881+0000","last_unstale":"2024-04-25T00:40:31.109271+0000","last_undegraded":"2024-04-25T00:40:31.109271+0000","last_fullsized":"2024-04-25T00:40:31.109271+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:46.104218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532371+0000","last_change":"2024-04-25T00:40:30.104225+0000","last_active":"2024-04-25T00:40:31.532371+0000","last_peered":"2024-04-25T00:40:31.532371+0000","last_clean":"2024-04-25T00:40:31.532371+0000","last_became_active":"2024-04-25T00:40:30.103790+0000","last_became_peered":"2024-04-25T00:40:30.103790+0000","last_unstale":"2024-04-25T00:40:31.532371+0000","last_undegraded":"2024-04-25T00:40:31.532371+0000","last_fullsized":"2024-04-25T00:40:31.532371+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:43.028353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111497+0000","last_change":"2024-04-25T00:40:29.083696+0000","last_active":"2024-04-25T00:40:31.111497+0000","last_peered":"2024-04-25T00:40:31.111497+0000","last_clean":"2024-04-25T00:40:31.111497+0000","last_became_active":"2024-04-25T00:40:29.082983+0000","last_became_peered":"2024-04-25T00:40:29.082983+0000","last_unstale":"2024-04-25T00:40:31.111497+0000","last_undegraded":"2024-04-25T00:40:31.111497+0000","last_fullsized":"2024-04-25T00:40:31.111497+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:08.779991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532823+0000","last_change":"2024-04-25T00:40:30.108256+0000","last_active":"2024-04-25T00:40:31.532823+0000","last_peered":"2024-04-25T00:40:31.532823+0000","last_clean":"2024-04-25T00:40:31.532823+0000","last_became_active":"2024-04-25T00:40:30.107857+0000","last_became_peered":"2024-04-25T00:40:30.107857+0000","last_unstale":"2024-04-25T00:40:31.532823+0000","last_undegraded":"2024-04-25T00:40:31.532823+0000","last_fullsized":"2024-04-25T00:40:31.532823+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:58.358658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109387+0000","last_change":"2024-04-25T00:40:29.098261+0000","last_active":"2024-04-25T00:40:31.109387+0000","last_peered":"2024-04-25T00:40:31.109387+0000","last_clean":"2024-04-25T00:40:31.109387+0000","last_became_active":"2024-04-25T00:40:29.097558+0000","last_became_peered":"2024-04-25T00:40:29.097558+0000","last_unstale":"2024-04-25T00:40:31.109387+0000","last_undegraded":"2024-04-25T00:40:31.109387+0000","last_fullsized":"2024-04-25T00:40:31.109387+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:05.401875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.328353+0000","last_change":"2024-04-25T00:40:30.112378+0000","last_active":"2024-04-25T00:40:31.328353+0000","last_peered":"2024-04-25T00:40:31.328353+0000","last_clean":"2024-04-25T00:40:31.328353+0000","last_became_active":"2024-04-25T00:40:30.112051+0000","last_became_peered":"2024-04-25T00:40:30.112051+0000","last_unstale":"2024-04-25T00:40:31.328353+0000","last_undegraded":"2024-04-25T00:40:31.328353+0000","last_fullsized":"2024-04-25T00:40:31.328353+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:31.152730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327734+0000","last_change":"2024-04-25T00:40:29.096322+0000","last_active":"2024-04-25T00:40:31.327734+0000","last_peered":"2024-04-25T00:40:31.327734+0000","last_clean":"2024-04-25T00:40:31.327734+0000","last_became_active":"2024-04-25T00:40:29.096034+0000","last_became_peered":"2024-04-25T00:40:29.096034+0000","last_unstale":"2024-04-25T00:40:31.327734+0000","last_undegraded":"2024-04-25T00:40:31.327734+0000","last_fullsized":"2024-04-25T00:40:31.327734+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:11.663375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123266+0000","last_change":"2024-04-25T00:40:30.106946+0000","last_active":"2024-04-25T00:40:32.123266+0000","last_peered":"2024-04-25T00:40:32.123266+0000","last_clean":"2024-04-25T00:40:32.123266+0000","last_became_active":"2024-04-25T00:40:30.106677+0000","last_became_peered":"2024-04-25T00:40:30.106677+0000","last_unstale":"2024-04-25T00:40:32.123266+0000","last_undegraded":"2024-04-25T00:40:32.123266+0000","last_fullsized":"2024-04-25T00:40:32.123266+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:06.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327426+0000","last_change":"2024-04-25T00:40:29.090943+0000","last_active":"2024-04-25T00:40:31.327426+0000","last_peered":"2024-04-25T00:40:31.327426+0000","last_clean":"2024-04-25T00:40:31.327426+0000","last_became_active":"2024-04-25T00:40:29.090524+0000","last_became_peered":"2024-04-25T00:40:29.090524+0000","last_unstale":"2024-04-25T00:40:31.327426+0000","last_undegraded":"2024-04-25T00:40:31.327426+0000","last_fullsized":"2024-04-25T00:40:31.327426+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:13.385254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123783+0000","last_change":"2024-04-25T00:40:30.115306+0000","last_active":"2024-04-25T00:40:32.123783+0000","last_peered":"2024-04-25T00:40:32.123783+0000","last_clean":"2024-04-25T00:40:32.123783+0000","last_became_active":"2024-04-25T00:40:30.114966+0000","last_became_peered":"2024-04-25T00:40:30.114966+0000","last_unstale":"2024-04-25T00:40:32.123783+0000","last_undegraded":"2024-04-25T00:40:32.123783+0000","last_fullsized":"2024-04-25T00:40:32.123783+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:39.019304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.122922+0000","last_change":"2024-04-25T00:40:29.097144+0000","last_active":"2024-04-25T00:40:32.122922+0000","last_peered":"2024-04-25T00:40:32.122922+0000","last_clean":"2024-04-25T00:40:32.122922+0000","last_became_active":"2024-04-25T00:40:29.096558+0000","last_became_peered":"2024-04-25T00:40:29.096558+0000","last_unstale":"2024-04-25T00:40:32.122922+0000","last_undegraded":"2024-04-25T00:40:32.122922+0000","last_fullsized":"2024-04-25T00:40:32.122922+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:38.339737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.107695+0000","last_change":"2024-04-25T00:40:30.108781+0000","last_active":"2024-04-25T00:40:31.107695+0000","last_peered":"2024-04-25T00:40:31.107695+0000","last_clean":"2024-04-25T00:40:31.107695+0000","last_became_active":"2024-04-25T00:40:30.107905+0000","last_became_peered":"2024-04-25T00:40:30.107905+0000","last_unstale":"2024-04-25T00:40:31.107695+0000","last_undegraded":"2024-04-25T00:40:31.107695+0000","last_fullsized":"2024-04-25T00:40:31.107695+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:06.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.112681+0000","last_change":"2024-04-25T00:40:29.077975+0000","last_active":"2024-04-25T00:40:31.112681+0000","last_peered":"2024-04-25T00:40:31.112681+0000","last_clean":"2024-04-25T00:40:31.112681+0000","last_became_active":"2024-04-25T00:40:29.077218+0000","last_became_peered":"2024-04-25T00:40:29.077218+0000","last_unstale":"2024-04-25T00:40:31.112681+0000","last_undegraded":"2024-04-25T00:40:31.112681+0000","last_fullsized":"2024-04-25T00:40:31.112681+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:57.348169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.108265+0000","last_change":"2024-04-25T00:40:30.131943+0000","last_active":"2024-04-25T00:40:31.108265+0000","last_peered":"2024-04-25T00:40:31.108265+0000","last_clean":"2024-04-25T00:40:31.108265+0000","last_became_active":"2024-04-25T00:40:30.131502+0000","last_became_peered":"2024-04-25T00:40:30.131502+0000","last_unstale":"2024-04-25T00:40:31.108265+0000","last_undegraded":"2024-04-25T00:40:31.108265+0000","last_fullsized":"2024-04-25T00:40:31.108265+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:51.016368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.112097+0000","last_change":"2024-04-25T00:40:29.096474+0000","last_active":"2024-04-25T00:40:31.112097+0000","last_peered":"2024-04-25T00:40:31.112097+0000","last_clean":"2024-04-25T00:40:31.112097+0000","last_became_active":"2024-04-25T00:40:29.095752+0000","last_became_peered":"2024-04-25T00:40:29.095752+0000","last_unstale":"2024-04-25T00:40:31.112097+0000","last_undegraded":"2024-04-25T00:40:31.112097+0000","last_fullsized":"2024-04-25T00:40:31.112097+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:55.232375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.126336+0000","last_change":"2024-04-25T00:40:30.115688+0000","last_active":"2024-04-25T00:40:32.126336+0000","last_peered":"2024-04-25T00:40:32.126336+0000","last_clean":"2024-04-25T00:40:32.126336+0000","last_became_active":"2024-04-25T00:40:30.109711+0000","last_became_peered":"2024-04-25T00:40:30.109711+0000","last_unstale":"2024-04-25T00:40:32.126336+0000","last_undegraded":"2024-04-25T00:40:32.126336+0000","last_fullsized":"2024-04-25T00:40:32.126336+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:52.100866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.126668+0000","last_change":"2024-04-25T00:40:29.084462+0000","last_active":"2024-04-25T00:40:32.126668+0000","last_peered":"2024-04-25T00:40:32.126668+0000","last_clean":"2024-04-25T00:40:32.126668+0000","last_became_active":"2024-04-25T00:40:29.082507+0000","last_became_peered":"2024-04-25T00:40:29.082507+0000","last_unstale":"2024-04-25T00:40:32.126668+0000","last_undegraded":"2024-04-25T00:40:32.126668+0000","last_fullsized":"2024-04-25T00:40:32.126668+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:24.734293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532534+0000","last_change":"2024-04-25T00:40:30.104073+0000","last_active":"2024-04-25T00:40:31.532534+0000","last_peered":"2024-04-25T00:40:31.532534+0000","last_clean":"2024-04-25T00:40:31.532534+0000","last_became_active":"2024-04-25T00:40:30.103701+0000","last_became_peered":"2024-04-25T00:40:30.103701+0000","last_unstale":"2024-04-25T00:40:31.532534+0000","last_undegraded":"2024-04-25T00:40:31.532534+0000","last_fullsized":"2024-04-25T00:40:31.532534+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:49.068054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109098+0000","last_change":"2024-04-25T00:40:29.094661+0000","last_active":"2024-04-25T00:40:31.109098+0000","last_peered":"2024-04-25T00:40:31.109098+0000","last_clean":"2024-04-25T00:40:31.109098+0000","last_became_active":"2024-04-25T00:40:29.094039+0000","last_became_peered":"2024-04-25T00:40:29.094039+0000","last_unstale":"2024-04-25T00:40:31.109098+0000","last_undegraded":"2024-04-25T00:40:31.109098+0000","last_fullsized":"2024-04-25T00:40:31.109098+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:15.360840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.326595+0000","last_change":"2024-04-25T00:40:30.108874+0000","last_active":"2024-04-25T00:40:31.326595+0000","last_peered":"2024-04-25T00:40:31.326595+0000","last_clean":"2024-04-25T00:40:31.326595+0000","last_became_active":"2024-04-25T00:40:30.108573+0000","last_became_peered":"2024-04-25T00:40:30.108573+0000","last_unstale":"2024-04-25T00:40:31.326595+0000","last_undegraded":"2024-04-25T00:40:31.326595+0000","last_fullsized":"2024-04-25T00:40:31.326595+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:42.110955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.107442+0000","last_change":"2024-04-25T00:40:29.101828+0000","last_active":"2024-04-25T00:40:31.107442+0000","last_peered":"2024-04-25T00:40:31.107442+0000","last_clean":"2024-04-25T00:40:31.107442+0000","last_became_active":"2024-04-25T00:40:29.101030+0000","last_became_peered":"2024-04-25T00:40:29.101030+0000","last_unstale":"2024-04-25T00:40:31.107442+0000","last_undegraded":"2024-04-25T00:40:31.107442+0000","last_fullsized":"2024-04-25T00:40:31.107442+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:47.720411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":466,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.107285+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:40:31.107285+0000","last_peered":"2024-04-25T00:40:31.107285+0000","last_clean":"2024-04-25T00:40:31.107285+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:40:31.107285+0000","last_undegraded":"2024-04-25T00:40:31.107285+0000","last_fullsized":"2024-04-25T00:40:31.107285+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574961,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37848,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333992,"statfs":{"total":96636764160,"available":96598007808,"internally_reserved":0,"allocated":2568192,"data_stored":1459719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37260,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334580,"statfs":{"total":96636764160,"available":96598609920,"internally_reserved":0,"allocated":1966080,"data_stored":869247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574962,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":1966080,"data_stored":869247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574961,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37284,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334556,"statfs":{"total":96636764160,"available":96598585344,"internally_reserved":0,"allocated":1982464,"data_stored":871441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574962,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41436,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330404,"statfs":{"total":96636764160,"available":96594333696,"internally_reserved":0,"allocated":1978368,"data_stored":869441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574961,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37276,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334564,"statfs":{"total":96636764160,"available":96598593536,"internally_reserved":0,"allocated":1978368,"data_stored":869941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37280,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334560,"statfs":{"total":96636764160,"available":96598589440,"internally_reserved":0,"allocated":1978368,"data_stored":870859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574961,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42008,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329832,"statfs":{"total":96636764160,"available":96593747968,"internally_reserved":0,"allocated":2564096,"data_stored":1459637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:40:38.593 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:40:38.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.725+0000 7f87a09da640 1 Processor -- start 2024-04-25T00:40:38.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.725+0000 7f87a09da640 1 -- start start 2024-04-25T00:40:38.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f87a09da640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f879c0b3140 0x7f879c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:38.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f87a09da640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0b4b10 0x7f879c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:38.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f87a09da640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f879c0b41e0 con 0x7f879c0b3a60 2024-04-25T00:40:38.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f87a09da640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f879c06d960 con 0x7f879c0b3140 2024-04-25T00:40:38.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f87a09da640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f879c06dc40 con 0x7f879c0b4b10 2024-04-25T00:40:38.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f879ad9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f879c0b3a60 0x7f879c15c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54400/0 (socket says 172.21.15.29:54400) 2024-04-25T00:40:38.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f879ad9d640 1 -- 172.21.15.29:0/3490136920 learned_addr learned my addr 172.21.15.29:0/3490136920 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:38.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f879a59c640 1 --2- 172.21.15.29:0/3490136920 >> v2:172.21.15.29:3300/0 conn(0x7f879c0b3140 0x7f879c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:38.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f8793fff640 1 --2- 172.21.15.29:0/3490136920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0b4b10 0x7f879c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:38.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f8799d9b640 1 -- 172.21.15.29:0/3490136920 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3411443076 0 0) 0x7f879c0b41e0 con 0x7f879c0b3a60 2024-04-25T00:40:38.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f879a59c640 1 -- 172.21.15.29:0/3490136920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0b4b10 msgr2=0x7f879c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:38.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f879a59c640 1 --2- 172.21.15.29:0/3490136920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0b4b10 0x7f879c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:38.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f879a59c640 1 -- 172.21.15.29:0/3490136920 >> v1:172.21.15.29:6789/0 conn(0x7f879c0b3a60 legacy=0x7f879c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:38.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f879a59c640 1 -- 172.21.15.29:0/3490136920 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c06df20 con 0x7f879c0b3140 2024-04-25T00:40:38.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f879a59c640 1 --2- 172.21.15.29:0/3490136920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f879c0b3140 0x7f879c06e440 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f878c0097c0 tx=0x7f878c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b4d0e019295364 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:38.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f8799d9b640 1 -- 172.21.15.29:0/3490136920 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0b0070 con 0x7f879c0b3140 2024-04-25T00:40:38.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f8799d9b640 1 -- 172.21.15.29:0/3490136920 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878c0ab600 con 0x7f879c0b3140 2024-04-25T00:40:38.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.729+0000 7f8799d9b640 1 -- 172.21.15.29:0/3490136920 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0ab9d0 con 0x7f879c0b3140 2024-04-25T00:40:38.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 -- 172.21.15.29:0/3490136920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f879c0b3140 msgr2=0x7f879c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:38.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 --2- 172.21.15.29:0/3490136920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f879c0b3140 0x7f879c06e440 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f878c0097c0 tx=0x7f878c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:40:38.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 -- 172.21.15.29:0/3490136920 shutdown_connections 2024-04-25T00:40:38.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 --2- 172.21.15.29:0/3490136920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0b4b10 0x7f879c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:38.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 --2- 172.21.15.29:0/3490136920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f879c0b3140 0x7f879c06e440 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:38.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 -- 172.21.15.29:0/3490136920 >> 172.21.15.29:0/3490136920 conn(0x7f879c0b1870 msgr2=0x7f879c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:38.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 -- 172.21.15.29:0/3490136920 shutdown_connections 2024-04-25T00:40:38.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 -- 172.21.15.29:0/3490136920 wait complete. 2024-04-25T00:40:38.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 Processor -- start 2024-04-25T00:40:38.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 -- start start 2024-04-25T00:40:38.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f879c0b3140 0x7f879c1551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:38.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f879c0b3a60 0x7f879c153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:38.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0b4b10 0x7f879c157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:38.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f879c168b70 con 0x7f879c0b3a60 2024-04-25T00:40:38.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f879c1689f0 con 0x7f879c0b3140 2024-04-25T00:40:38.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f879c168cf0 con 0x7f879c0b4b10 2024-04-25T00:40:38.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f8793fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f879c0b3a60 0x7f879c153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:38.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f8793fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f879c0b3a60 0x7f879c153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60868/0 (socket says 172.21.15.29:60868) 2024-04-25T00:40:38.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f8793fff640 1 -- 172.21.15.29:0/4120934410 learned_addr learned my addr 172.21.15.29:0/4120934410 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:38.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f879ad9d640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0b4b10 0x7f879c157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:38.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f879a59c640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f879c0b3140 0x7f879c1551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:38.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f8793fff640 1 -- 172.21.15.29:0/4120934410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0b4b10 msgr2=0x7f879c157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:38.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f8793fff640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0b4b10 0x7f879c157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:38.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f8793fff640 1 -- 172.21.15.29:0/4120934410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f879c0b3140 msgr2=0x7f879c1551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:38.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f8793fff640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f879c0b3140 0x7f879c1551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:38.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f8793fff640 1 -- 172.21.15.29:0/4120934410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c158540 con 0x7f879c0b3a60 2024-04-25T00:40:38.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f879ad9d640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0b4b10 0x7f879c157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:38.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f879a59c640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f879c0b3140 0x7f879c1551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:38.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f8793fff640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f879c0b3a60 0x7f879c153c40 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f878400d5f0 tx=0x7f878400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:38.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f8792ffd640 1 -- 172.21.15.29:0/4120934410 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87840125f0 con 0x7f879c0b3a60 2024-04-25T00:40:38.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f8792ffd640 1 -- 172.21.15.29:0/4120934410 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8784019070 con 0x7f879c0b3a60 2024-04-25T00:40:38.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f8792ffd640 1 -- 172.21.15.29:0/4120934410 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87840129b0 con 0x7f879c0b3a60 2024-04-25T00:40:38.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 -- 172.21.15.29:0/4120934410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879c158820 con 0x7f879c0b3a60 2024-04-25T00:40:38.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.733+0000 7f87a09da640 1 -- 172.21.15.29:0/4120934410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f879c158de0 con 0x7f879c0b3a60 2024-04-25T00:40:38.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.737+0000 7f87a09da640 1 -- 172.21.15.29:0/4120934410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879c1591c0 con 0x7f879c0b3a60 2024-04-25T00:40:38.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.737+0000 7f8792ffd640 1 -- 172.21.15.29:0/4120934410 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f8784012b50 con 0x7f879c0b3a60 2024-04-25T00:40:38.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.737+0000 7f8792ffd640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f87800680d0 0x7f878006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:38.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.737+0000 7f8792ffd640 1 -- 172.21.15.29:0/4120934410 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f8784084a30 con 0x7f879c0b3a60 2024-04-25T00:40:38.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.737+0000 7f879a59c640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f87800680d0 0x7f878006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:38.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.737+0000 7f879a59c640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f87800680d0 0x7f878006a590 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f878c0ad000 tx=0x7f878c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:38.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.741+0000 7f8792ffd640 1 -- 172.21.15.29:0/4120934410 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f879c1591c0 con 0x7f879c0b3a60 2024-04-25T00:40:38.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.901+0000 7f87a09da640 1 -- 172.21.15.29:0/4120934410 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f879c069270 con 0x7f87800680d0 2024-04-25T00:40:38.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.917+0000 7f8792ffd640 1 -- 172.21.15.29:0/4120934410 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7f879c069270 con 0x7f87800680d0 2024-04-25T00:40:38.919 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:38.920 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:40:38.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.921+0000 7f87a09da640 1 -- 172.21.15.29:0/4120934410 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f87800680d0 msgr2=0x7f878006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:38.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.921+0000 7f87a09da640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f87800680d0 0x7f878006a590 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f878c0ad000 tx=0x7f878c008040 comp rx=0 tx=0).stop 2024-04-25T00:40:38.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.921+0000 7f87a09da640 1 -- 172.21.15.29:0/4120934410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f879c0b3a60 msgr2=0x7f879c153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:38.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.921+0000 7f87a09da640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f879c0b3a60 0x7f879c153c40 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f878400d5f0 tx=0x7f878400dab0 comp rx=0 tx=0).stop 2024-04-25T00:40:38.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.921+0000 7f87a09da640 1 -- 172.21.15.29:0/4120934410 shutdown_connections 2024-04-25T00:40:38.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.921+0000 7f87a09da640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f87800680d0 0x7f878006a590 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:38.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.921+0000 7f87a09da640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0b4b10 0x7f879c157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:38.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.921+0000 7f87a09da640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f879c0b3a60 0x7f879c153c40 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:38.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.921+0000 7f87a09da640 1 --2- 172.21.15.29:0/4120934410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f879c0b3140 0x7f879c1551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:38.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.921+0000 7f87a09da640 1 -- 172.21.15.29:0/4120934410 >> 172.21.15.29:0/4120934410 conn(0x7f879c0b1870 msgr2=0x7f879c06cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:38.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.921+0000 7f87a09da640 1 -- 172.21.15.29:0/4120934410 shutdown_connections 2024-04-25T00:40:38.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:38.921+0000 7f87a09da640 1 -- 172.21.15.29:0/4120934410 wait complete. 2024-04-25T00:40:38.938 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-04-25T00:40:37.406679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307656,"kb_used_data":16584,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754667064,"statfs":{"total":773094113280,"available":772779073536,"internally_reserved":0,"allocated":16982016,"data_stored":8139532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002615"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109537+0000","last_change":"2024-04-25T00:40:30.132436+0000","last_active":"2024-04-25T00:40:31.109537+0000","last_peered":"2024-04-25T00:40:31.109537+0000","last_clean":"2024-04-25T00:40:31.109537+0000","last_became_active":"2024-04-25T00:40:30.123577+0000","last_became_peered":"2024-04-25T00:40:30.123577+0000","last_unstale":"2024-04-25T00:40:31.109537+0000","last_undegraded":"2024-04-25T00:40:31.109537+0000","last_fullsized":"2024-04-25T00:40:31.109537+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:39.019304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.127473+0000","last_change":"2024-04-25T00:40:30.114772+0000","last_active":"2024-04-25T00:40:32.127473+0000","last_peered":"2024-04-25T00:40:32.127473+0000","last_clean":"2024-04-25T00:40:32.127473+0000","last_became_active":"2024-04-25T00:40:30.109499+0000","last_became_peered":"2024-04-25T00:40:30.109499+0000","last_unstale":"2024-04-25T00:40:32.127473+0000","last_undegraded":"2024-04-25T00:40:32.127473+0000","last_fullsized":"2024-04-25T00:40:32.127473+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:28.342905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109259+0000","last_change":"2024-04-25T00:40:30.117242+0000","last_active":"2024-04-25T00:40:31.109259+0000","last_peered":"2024-04-25T00:40:31.109259+0000","last_clean":"2024-04-25T00:40:31.109259+0000","last_became_active":"2024-04-25T00:40:30.116846+0000","last_became_peered":"2024-04-25T00:40:30.116846+0000","last_unstale":"2024-04-25T00:40:31.109259+0000","last_undegraded":"2024-04-25T00:40:31.109259+0000","last_fullsized":"2024-04-25T00:40:31.109259+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:11.813161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.127096+0000","last_change":"2024-04-25T00:40:30.127364+0000","last_active":"2024-04-25T00:40:32.127096+0000","last_peered":"2024-04-25T00:40:32.127096+0000","last_clean":"2024-04-25T00:40:32.127096+0000","last_became_active":"2024-04-25T00:40:30.124178+0000","last_became_peered":"2024-04-25T00:40:30.124178+0000","last_unstale":"2024-04-25T00:40:32.127096+0000","last_undegraded":"2024-04-25T00:40:32.127096+0000","last_fullsized":"2024-04-25T00:40:32.127096+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:14.368023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532596+0000","last_change":"2024-04-25T00:40:30.104621+0000","last_active":"2024-04-25T00:40:31.532596+0000","last_peered":"2024-04-25T00:40:31.532596+0000","last_clean":"2024-04-25T00:40:31.532596+0000","last_became_active":"2024-04-25T00:40:30.104243+0000","last_became_peered":"2024-04-25T00:40:30.104243+0000","last_unstale":"2024-04-25T00:40:31.532596+0000","last_undegraded":"2024-04-25T00:40:31.532596+0000","last_fullsized":"2024-04-25T00:40:31.532596+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:09.790480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109376+0000","last_change":"2024-04-25T00:40:30.106794+0000","last_active":"2024-04-25T00:40:31.109376+0000","last_peered":"2024-04-25T00:40:31.109376+0000","last_clean":"2024-04-25T00:40:31.109376+0000","last_became_active":"2024-04-25T00:40:30.106065+0000","last_became_peered":"2024-04-25T00:40:30.106065+0000","last_unstale":"2024-04-25T00:40:31.109376+0000","last_undegraded":"2024-04-25T00:40:31.109376+0000","last_fullsized":"2024-04-25T00:40:31.109376+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:41.286811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.110009+0000","last_change":"2024-04-25T00:40:30.128835+0000","last_active":"2024-04-25T00:40:31.110009+0000","last_peered":"2024-04-25T00:40:31.110009+0000","last_clean":"2024-04-25T00:40:31.110009+0000","last_became_active":"2024-04-25T00:40:30.128221+0000","last_became_peered":"2024-04-25T00:40:30.128221+0000","last_unstale":"2024-04-25T00:40:31.110009+0000","last_undegraded":"2024-04-25T00:40:31.110009+0000","last_fullsized":"2024-04-25T00:40:31.110009+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:41.286811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.122767+0000","last_change":"2024-04-25T00:40:30.135572+0000","last_active":"2024-04-25T00:40:32.122767+0000","last_peered":"2024-04-25T00:40:32.122767+0000","last_clean":"2024-04-25T00:40:32.122767+0000","last_became_active":"2024-04-25T00:40:30.135273+0000","last_became_peered":"2024-04-25T00:40:30.135273+0000","last_unstale":"2024-04-25T00:40:32.122767+0000","last_undegraded":"2024-04-25T00:40:32.122767+0000","last_fullsized":"2024-04-25T00:40:32.122767+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:54.538024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.113034+0000","last_change":"2024-04-25T00:40:30.110598+0000","last_active":"2024-04-25T00:40:31.113034+0000","last_peered":"2024-04-25T00:40:31.113034+0000","last_clean":"2024-04-25T00:40:31.113034+0000","last_became_active":"2024-04-25T00:40:30.109826+0000","last_became_peered":"2024-04-25T00:40:30.109826+0000","last_unstale":"2024-04-25T00:40:31.113034+0000","last_undegraded":"2024-04-25T00:40:31.113034+0000","last_fullsized":"2024-04-25T00:40:31.113034+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:16.758479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327463+0000","last_change":"2024-04-25T00:40:30.109734+0000","last_active":"2024-04-25T00:40:31.327463+0000","last_peered":"2024-04-25T00:40:31.327463+0000","last_clean":"2024-04-25T00:40:31.327463+0000","last_became_active":"2024-04-25T00:40:30.109492+0000","last_became_peered":"2024-04-25T00:40:30.109492+0000","last_unstale":"2024-04-25T00:40:31.327463+0000","last_undegraded":"2024-04-25T00:40:31.327463+0000","last_fullsized":"2024-04-25T00:40:31.327463+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:38.456637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.112145+0000","last_change":"2024-04-25T00:40:30.095770+0000","last_active":"2024-04-25T00:40:31.112145+0000","last_peered":"2024-04-25T00:40:31.112145+0000","last_clean":"2024-04-25T00:40:31.112145+0000","last_became_active":"2024-04-25T00:40:30.095052+0000","last_became_peered":"2024-04-25T00:40:30.095052+0000","last_unstale":"2024-04-25T00:40:31.112145+0000","last_undegraded":"2024-04-25T00:40:31.112145+0000","last_fullsized":"2024-04-25T00:40:31.112145+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:28.490785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.127300+0000","last_change":"2024-04-25T00:40:30.114507+0000","last_active":"2024-04-25T00:40:32.127300+0000","last_peered":"2024-04-25T00:40:32.127300+0000","last_clean":"2024-04-25T00:40:32.127300+0000","last_became_active":"2024-04-25T00:40:30.113692+0000","last_became_peered":"2024-04-25T00:40:30.113692+0000","last_unstale":"2024-04-25T00:40:32.127300+0000","last_undegraded":"2024-04-25T00:40:32.127300+0000","last_fullsized":"2024-04-25T00:40:32.127300+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:41.286811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.126843+0000","last_change":"2024-04-25T00:40:30.133750+0000","last_active":"2024-04-25T00:40:32.126843+0000","last_peered":"2024-04-25T00:40:32.126843+0000","last_clean":"2024-04-25T00:40:32.126843+0000","last_became_active":"2024-04-25T00:40:30.133095+0000","last_became_peered":"2024-04-25T00:40:30.133095+0000","last_unstale":"2024-04-25T00:40:32.126843+0000","last_undegraded":"2024-04-25T00:40:32.126843+0000","last_fullsized":"2024-04-25T00:40:32.126843+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:11.813161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111579+0000","last_change":"2024-04-25T00:40:30.111376+0000","last_active":"2024-04-25T00:40:31.111579+0000","last_peered":"2024-04-25T00:40:31.111579+0000","last_clean":"2024-04-25T00:40:31.111579+0000","last_became_active":"2024-04-25T00:40:30.110992+0000","last_became_peered":"2024-04-25T00:40:30.110992+0000","last_unstale":"2024-04-25T00:40:31.111579+0000","last_undegraded":"2024-04-25T00:40:31.111579+0000","last_fullsized":"2024-04-25T00:40:31.111579+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:16.371328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.126698+0000","last_change":"2024-04-25T00:40:30.125877+0000","last_active":"2024-04-25T00:40:32.126698+0000","last_peered":"2024-04-25T00:40:32.126698+0000","last_clean":"2024-04-25T00:40:32.126698+0000","last_became_active":"2024-04-25T00:40:30.109087+0000","last_became_peered":"2024-04-25T00:40:30.109087+0000","last_unstale":"2024-04-25T00:40:32.126698+0000","last_undegraded":"2024-04-25T00:40:32.126698+0000","last_fullsized":"2024-04-25T00:40:32.126698+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:21.024511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123593+0000","last_change":"2024-04-25T00:40:30.115666+0000","last_active":"2024-04-25T00:40:32.123593+0000","last_peered":"2024-04-25T00:40:32.123593+0000","last_clean":"2024-04-25T00:40:32.123593+0000","last_became_active":"2024-04-25T00:40:30.115335+0000","last_became_peered":"2024-04-25T00:40:30.115335+0000","last_unstale":"2024-04-25T00:40:32.123593+0000","last_undegraded":"2024-04-25T00:40:32.123593+0000","last_fullsized":"2024-04-25T00:40:32.123593+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:49.124306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533179+0000","last_change":"2024-04-25T00:40:30.110030+0000","last_active":"2024-04-25T00:40:31.533179+0000","last_peered":"2024-04-25T00:40:31.533179+0000","last_clean":"2024-04-25T00:40:31.533179+0000","last_became_active":"2024-04-25T00:40:30.109685+0000","last_became_peered":"2024-04-25T00:40:30.109685+0000","last_unstale":"2024-04-25T00:40:31.533179+0000","last_undegraded":"2024-04-25T00:40:31.533179+0000","last_fullsized":"2024-04-25T00:40:31.533179+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:58.444761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109382+0000","last_change":"2024-04-25T00:40:30.105168+0000","last_active":"2024-04-25T00:40:31.109382+0000","last_peered":"2024-04-25T00:40:31.109382+0000","last_clean":"2024-04-25T00:40:31.109382+0000","last_became_active":"2024-04-25T00:40:30.104401+0000","last_became_peered":"2024-04-25T00:40:30.104401+0000","last_unstale":"2024-04-25T00:40:31.109382+0000","last_undegraded":"2024-04-25T00:40:31.109382+0000","last_fullsized":"2024-04-25T00:40:31.109382+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:49.124306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327133+0000","last_change":"2024-04-25T00:40:30.116393+0000","last_active":"2024-04-25T00:40:31.327133+0000","last_peered":"2024-04-25T00:40:31.327133+0000","last_clean":"2024-04-25T00:40:31.327133+0000","last_became_active":"2024-04-25T00:40:30.115406+0000","last_became_peered":"2024-04-25T00:40:30.115406+0000","last_unstale":"2024-04-25T00:40:31.327133+0000","last_undegraded":"2024-04-25T00:40:31.327133+0000","last_fullsized":"2024-04-25T00:40:31.327133+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:49.124306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533095+0000","last_change":"2024-04-25T00:40:30.119807+0000","last_active":"2024-04-25T00:40:31.533095+0000","last_peered":"2024-04-25T00:40:31.533095+0000","last_clean":"2024-04-25T00:40:31.533095+0000","last_became_active":"2024-04-25T00:40:30.119510+0000","last_became_peered":"2024-04-25T00:40:30.119510+0000","last_unstale":"2024-04-25T00:40:31.533095+0000","last_undegraded":"2024-04-25T00:40:31.533095+0000","last_fullsized":"2024-04-25T00:40:31.533095+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:39.713681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533044+0000","last_change":"2024-04-25T00:40:30.106305+0000","last_active":"2024-04-25T00:40:31.533044+0000","last_peered":"2024-04-25T00:40:31.533044+0000","last_clean":"2024-04-25T00:40:31.533044+0000","last_became_active":"2024-04-25T00:40:30.104321+0000","last_became_peered":"2024-04-25T00:40:30.104321+0000","last_unstale":"2024-04-25T00:40:31.533044+0000","last_undegraded":"2024-04-25T00:40:31.533044+0000","last_fullsized":"2024-04-25T00:40:31.533044+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:56.242863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327402+0000","last_change":"2024-04-25T00:40:30.112605+0000","last_active":"2024-04-25T00:40:31.327402+0000","last_peered":"2024-04-25T00:40:31.327402+0000","last_clean":"2024-04-25T00:40:31.327402+0000","last_became_active":"2024-04-25T00:40:30.112325+0000","last_became_peered":"2024-04-25T00:40:30.112325+0000","last_unstale":"2024-04-25T00:40:31.327402+0000","last_undegraded":"2024-04-25T00:40:31.327402+0000","last_fullsized":"2024-04-25T00:40:31.327402+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:25.442863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109793+0000","last_change":"2024-04-25T00:40:30.106950+0000","last_active":"2024-04-25T00:40:31.109793+0000","last_peered":"2024-04-25T00:40:31.109793+0000","last_clean":"2024-04-25T00:40:31.109793+0000","last_became_active":"2024-04-25T00:40:30.106672+0000","last_became_peered":"2024-04-25T00:40:30.106672+0000","last_unstale":"2024-04-25T00:40:31.109793+0000","last_undegraded":"2024-04-25T00:40:31.109793+0000","last_fullsized":"2024-04-25T00:40:31.109793+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:36.066823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123130+0000","last_change":"2024-04-25T00:40:30.103963+0000","last_active":"2024-04-25T00:40:32.123130+0000","last_peered":"2024-04-25T00:40:32.123130+0000","last_clean":"2024-04-25T00:40:32.123130+0000","last_became_active":"2024-04-25T00:40:30.103476+0000","last_became_peered":"2024-04-25T00:40:30.103476+0000","last_unstale":"2024-04-25T00:40:32.123130+0000","last_undegraded":"2024-04-25T00:40:32.123130+0000","last_fullsized":"2024-04-25T00:40:32.123130+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:42.110955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.127872+0000","last_change":"2024-04-25T00:40:30.126333+0000","last_active":"2024-04-25T00:40:32.127872+0000","last_peered":"2024-04-25T00:40:32.127872+0000","last_clean":"2024-04-25T00:40:32.127872+0000","last_became_active":"2024-04-25T00:40:30.125851+0000","last_became_peered":"2024-04-25T00:40:30.125851+0000","last_unstale":"2024-04-25T00:40:32.127872+0000","last_undegraded":"2024-04-25T00:40:32.127872+0000","last_fullsized":"2024-04-25T00:40:32.127872+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:36.066823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111800+0000","last_change":"2024-04-25T00:40:30.129130+0000","last_active":"2024-04-25T00:40:31.111800+0000","last_peered":"2024-04-25T00:40:31.111800+0000","last_clean":"2024-04-25T00:40:31.111800+0000","last_became_active":"2024-04-25T00:40:30.113758+0000","last_became_peered":"2024-04-25T00:40:30.113758+0000","last_unstale":"2024-04-25T00:40:31.111800+0000","last_undegraded":"2024-04-25T00:40:31.111800+0000","last_fullsized":"2024-04-25T00:40:31.111800+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:05.372029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109368+0000","last_change":"2024-04-25T00:40:30.105721+0000","last_active":"2024-04-25T00:40:31.109368+0000","last_peered":"2024-04-25T00:40:31.109368+0000","last_clean":"2024-04-25T00:40:31.109368+0000","last_became_active":"2024-04-25T00:40:30.105374+0000","last_became_peered":"2024-04-25T00:40:30.105374+0000","last_unstale":"2024-04-25T00:40:31.109368+0000","last_undegraded":"2024-04-25T00:40:31.109368+0000","last_fullsized":"2024-04-25T00:40:31.109368+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:14.368023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533317+0000","last_change":"2024-04-25T00:40:30.109593+0000","last_active":"2024-04-25T00:40:31.533317+0000","last_peered":"2024-04-25T00:40:31.533317+0000","last_clean":"2024-04-25T00:40:31.533317+0000","last_became_active":"2024-04-25T00:40:30.109276+0000","last_became_peered":"2024-04-25T00:40:30.109276+0000","last_unstale":"2024-04-25T00:40:31.533317+0000","last_undegraded":"2024-04-25T00:40:31.533317+0000","last_fullsized":"2024-04-25T00:40:31.533317+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:50.537642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.126845+0000","last_change":"2024-04-25T00:40:30.107733+0000","last_active":"2024-04-25T00:40:32.126845+0000","last_peered":"2024-04-25T00:40:32.126845+0000","last_clean":"2024-04-25T00:40:32.126845+0000","last_became_active":"2024-04-25T00:40:30.107062+0000","last_became_peered":"2024-04-25T00:40:30.107062+0000","last_unstale":"2024-04-25T00:40:32.126845+0000","last_undegraded":"2024-04-25T00:40:32.126845+0000","last_fullsized":"2024-04-25T00:40:32.126845+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:41:21.636262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111993+0000","last_change":"2024-04-25T00:40:30.099120+0000","last_active":"2024-04-25T00:40:31.111993+0000","last_peered":"2024-04-25T00:40:31.111993+0000","last_clean":"2024-04-25T00:40:31.111993+0000","last_became_active":"2024-04-25T00:40:30.098319+0000","last_became_peered":"2024-04-25T00:40:30.098319+0000","last_unstale":"2024-04-25T00:40:31.111993+0000","last_undegraded":"2024-04-25T00:40:31.111993+0000","last_fullsized":"2024-04-25T00:40:31.111993+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:02.179108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109175+0000","last_change":"2024-04-25T00:40:30.115433+0000","last_active":"2024-04-25T00:40:31.109175+0000","last_peered":"2024-04-25T00:40:31.109175+0000","last_clean":"2024-04-25T00:40:31.109175+0000","last_became_active":"2024-04-25T00:40:30.114982+0000","last_became_peered":"2024-04-25T00:40:30.114982+0000","last_unstale":"2024-04-25T00:40:31.109175+0000","last_undegraded":"2024-04-25T00:40:31.109175+0000","last_fullsized":"2024-04-25T00:40:31.109175+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:41:21.636262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532379+0000","last_change":"2024-04-25T00:40:30.130523+0000","last_active":"2024-04-25T00:40:31.532379+0000","last_peered":"2024-04-25T00:40:31.532379+0000","last_clean":"2024-04-25T00:40:31.532379+0000","last_became_active":"2024-04-25T00:40:30.130164+0000","last_became_peered":"2024-04-25T00:40:30.130164+0000","last_unstale":"2024-04-25T00:40:31.532379+0000","last_undegraded":"2024-04-25T00:40:31.532379+0000","last_fullsized":"2024-04-25T00:40:31.532379+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:57.823829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.124073+0000","last_change":"2024-04-25T00:40:30.115851+0000","last_active":"2024-04-25T00:40:32.124073+0000","last_peered":"2024-04-25T00:40:32.124073+0000","last_clean":"2024-04-25T00:40:32.124073+0000","last_became_active":"2024-04-25T00:40:30.115406+0000","last_became_peered":"2024-04-25T00:40:30.115406+0000","last_unstale":"2024-04-25T00:40:32.124073+0000","last_undegraded":"2024-04-25T00:40:32.124073+0000","last_fullsized":"2024-04-25T00:40:32.124073+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:31.042850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"155'4","reported_seq":18,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533065+0000","last_change":"2024-04-25T00:40:29.094752+0000","last_active":"2024-04-25T00:40:31.533065+0000","last_peered":"2024-04-25T00:40:31.533065+0000","last_clean":"2024-04-25T00:40:31.533065+0000","last_became_active":"2024-04-25T00:40:29.094472+0000","last_became_peered":"2024-04-25T00:40:29.094472+0000","last_unstale":"2024-04-25T00:40:31.533065+0000","last_undegraded":"2024-04-25T00:40:31.533065+0000","last_fullsized":"2024-04-25T00:40:31.533065+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:42.622353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.328400+0000","last_change":"2024-04-25T00:40:30.114712+0000","last_active":"2024-04-25T00:40:31.328400+0000","last_peered":"2024-04-25T00:40:31.328400+0000","last_clean":"2024-04-25T00:40:31.328400+0000","last_became_active":"2024-04-25T00:40:30.114451+0000","last_became_peered":"2024-04-25T00:40:30.114451+0000","last_unstale":"2024-04-25T00:40:31.328400+0000","last_undegraded":"2024-04-25T00:40:31.328400+0000","last_fullsized":"2024-04-25T00:40:31.328400+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:41:59.378161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.112120+0000","last_change":"2024-04-25T00:40:29.085706+0000","last_active":"2024-04-25T00:40:31.112120+0000","last_peered":"2024-04-25T00:40:31.112120+0000","last_clean":"2024-04-25T00:40:31.112120+0000","last_became_active":"2024-04-25T00:40:29.084970+0000","last_became_peered":"2024-04-25T00:40:29.084970+0000","last_unstale":"2024-04-25T00:40:31.112120+0000","last_undegraded":"2024-04-25T00:40:31.112120+0000","last_fullsized":"2024-04-25T00:40:31.112120+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:48.057565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111179+0000","last_change":"2024-04-25T00:40:30.113159+0000","last_active":"2024-04-25T00:40:31.111179+0000","last_peered":"2024-04-25T00:40:31.111179+0000","last_clean":"2024-04-25T00:40:31.111179+0000","last_became_active":"2024-04-25T00:40:30.112489+0000","last_became_peered":"2024-04-25T00:40:30.112489+0000","last_unstale":"2024-04-25T00:40:31.111179+0000","last_undegraded":"2024-04-25T00:40:31.111179+0000","last_fullsized":"2024-04-25T00:40:31.111179+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:01.301393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109471+0000","last_change":"2024-04-25T00:40:29.098108+0000","last_active":"2024-04-25T00:40:31.109471+0000","last_peered":"2024-04-25T00:40:31.109471+0000","last_clean":"2024-04-25T00:40:31.109471+0000","last_became_active":"2024-04-25T00:40:29.097478+0000","last_became_peered":"2024-04-25T00:40:29.097478+0000","last_unstale":"2024-04-25T00:40:31.109471+0000","last_undegraded":"2024-04-25T00:40:31.109471+0000","last_fullsized":"2024-04-25T00:40:31.109471+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:43.160214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.112458+0000","last_change":"2024-04-25T00:40:30.132249+0000","last_active":"2024-04-25T00:40:31.112458+0000","last_peered":"2024-04-25T00:40:31.112458+0000","last_clean":"2024-04-25T00:40:31.112458+0000","last_became_active":"2024-04-25T00:40:30.131607+0000","last_became_peered":"2024-04-25T00:40:30.131607+0000","last_unstale":"2024-04-25T00:40:31.112458+0000","last_undegraded":"2024-04-25T00:40:31.112458+0000","last_fullsized":"2024-04-25T00:40:31.112458+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:44.170703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.127403+0000","last_change":"2024-04-25T00:40:29.096877+0000","last_active":"2024-04-25T00:40:32.127403+0000","last_peered":"2024-04-25T00:40:32.127403+0000","last_clean":"2024-04-25T00:40:32.127403+0000","last_became_active":"2024-04-25T00:40:29.096218+0000","last_became_peered":"2024-04-25T00:40:29.096218+0000","last_unstale":"2024-04-25T00:40:32.127403+0000","last_undegraded":"2024-04-25T00:40:32.127403+0000","last_fullsized":"2024-04-25T00:40:32.127403+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:15.360840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.326938+0000","last_change":"2024-04-25T00:40:30.113296+0000","last_active":"2024-04-25T00:40:31.326938+0000","last_peered":"2024-04-25T00:40:31.326938+0000","last_clean":"2024-04-25T00:40:31.326938+0000","last_became_active":"2024-04-25T00:40:30.113042+0000","last_became_peered":"2024-04-25T00:40:30.113042+0000","last_unstale":"2024-04-25T00:40:31.326938+0000","last_undegraded":"2024-04-25T00:40:31.326938+0000","last_fullsized":"2024-04-25T00:40:31.326938+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:51.016368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532876+0000","last_change":"2024-04-25T00:40:29.087901+0000","last_active":"2024-04-25T00:40:31.532876+0000","last_peered":"2024-04-25T00:40:31.532876+0000","last_clean":"2024-04-25T00:40:31.532876+0000","last_became_active":"2024-04-25T00:40:29.087264+0000","last_became_peered":"2024-04-25T00:40:29.087264+0000","last_unstale":"2024-04-25T00:40:31.532876+0000","last_undegraded":"2024-04-25T00:40:31.532876+0000","last_fullsized":"2024-04-25T00:40:31.532876+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:30.526164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123443+0000","last_change":"2024-04-25T00:40:30.115689+0000","last_active":"2024-04-25T00:40:32.123443+0000","last_peered":"2024-04-25T00:40:32.123443+0000","last_clean":"2024-04-25T00:40:32.123443+0000","last_became_active":"2024-04-25T00:40:30.115014+0000","last_became_peered":"2024-04-25T00:40:30.115014+0000","last_unstale":"2024-04-25T00:40:32.123443+0000","last_undegraded":"2024-04-25T00:40:32.123443+0000","last_fullsized":"2024-04-25T00:40:32.123443+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:51.016368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123444+0000","last_change":"2024-04-25T00:40:29.096769+0000","last_active":"2024-04-25T00:40:32.123444+0000","last_peered":"2024-04-25T00:40:32.123444+0000","last_clean":"2024-04-25T00:40:32.123444+0000","last_became_active":"2024-04-25T00:40:29.096339+0000","last_became_peered":"2024-04-25T00:40:29.096339+0000","last_unstale":"2024-04-25T00:40:32.123444+0000","last_undegraded":"2024-04-25T00:40:32.123444+0000","last_fullsized":"2024-04-25T00:40:32.123444+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:23.222465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.110010+0000","last_change":"2024-04-25T00:40:30.112484+0000","last_active":"2024-04-25T00:40:31.110010+0000","last_peered":"2024-04-25T00:40:31.110010+0000","last_clean":"2024-04-25T00:40:31.110010+0000","last_became_active":"2024-04-25T00:40:30.096322+0000","last_became_peered":"2024-04-25T00:40:30.096322+0000","last_unstale":"2024-04-25T00:40:31.110010+0000","last_undegraded":"2024-04-25T00:40:31.110010+0000","last_fullsized":"2024-04-25T00:40:31.110010+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:07.296465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.106816+0000","last_change":"2024-04-25T00:40:29.082537+0000","last_active":"2024-04-25T00:40:31.106816+0000","last_peered":"2024-04-25T00:40:31.106816+0000","last_clean":"2024-04-25T00:40:31.106816+0000","last_became_active":"2024-04-25T00:40:29.081768+0000","last_became_peered":"2024-04-25T00:40:29.081768+0000","last_unstale":"2024-04-25T00:40:31.106816+0000","last_undegraded":"2024-04-25T00:40:31.106816+0000","last_fullsized":"2024-04-25T00:40:31.106816+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:55.940190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109505+0000","last_change":"2024-04-25T00:40:30.112614+0000","last_active":"2024-04-25T00:40:31.109505+0000","last_peered":"2024-04-25T00:40:31.109505+0000","last_clean":"2024-04-25T00:40:31.109505+0000","last_became_active":"2024-04-25T00:40:30.112312+0000","last_became_peered":"2024-04-25T00:40:30.112312+0000","last_unstale":"2024-04-25T00:40:31.109505+0000","last_undegraded":"2024-04-25T00:40:31.109505+0000","last_fullsized":"2024-04-25T00:40:31.109505+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:28.342905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111523+0000","last_change":"2024-04-25T00:40:29.091407+0000","last_active":"2024-04-25T00:40:31.111523+0000","last_peered":"2024-04-25T00:40:31.111523+0000","last_clean":"2024-04-25T00:40:31.111523+0000","last_became_active":"2024-04-25T00:40:29.090686+0000","last_became_peered":"2024-04-25T00:40:29.090686+0000","last_unstale":"2024-04-25T00:40:31.111523+0000","last_undegraded":"2024-04-25T00:40:31.111523+0000","last_fullsized":"2024-04-25T00:40:31.111523+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:45.643415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123150+0000","last_change":"2024-04-25T00:40:30.118062+0000","last_active":"2024-04-25T00:40:32.123150+0000","last_peered":"2024-04-25T00:40:32.123150+0000","last_clean":"2024-04-25T00:40:32.123150+0000","last_became_active":"2024-04-25T00:40:30.117710+0000","last_became_peered":"2024-04-25T00:40:30.117710+0000","last_unstale":"2024-04-25T00:40:32.123150+0000","last_undegraded":"2024-04-25T00:40:32.123150+0000","last_fullsized":"2024-04-25T00:40:32.123150+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:51.726915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533110+0000","last_change":"2024-04-25T00:40:29.100639+0000","last_active":"2024-04-25T00:40:31.533110+0000","last_peered":"2024-04-25T00:40:31.533110+0000","last_clean":"2024-04-25T00:40:31.533110+0000","last_became_active":"2024-04-25T00:40:29.100348+0000","last_became_peered":"2024-04-25T00:40:29.100348+0000","last_unstale":"2024-04-25T00:40:31.533110+0000","last_undegraded":"2024-04-25T00:40:31.533110+0000","last_fullsized":"2024-04-25T00:40:31.533110+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:55.842338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111373+0000","last_change":"2024-04-25T00:40:30.136453+0000","last_active":"2024-04-25T00:40:31.111373+0000","last_peered":"2024-04-25T00:40:31.111373+0000","last_clean":"2024-04-25T00:40:31.111373+0000","last_became_active":"2024-04-25T00:40:30.135882+0000","last_became_peered":"2024-04-25T00:40:30.135882+0000","last_unstale":"2024-04-25T00:40:31.111373+0000","last_undegraded":"2024-04-25T00:40:31.111373+0000","last_fullsized":"2024-04-25T00:40:31.111373+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:09.079933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"155'3","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532724+0000","last_change":"2024-04-25T00:40:29.088110+0000","last_active":"2024-04-25T00:40:31.532724+0000","last_peered":"2024-04-25T00:40:31.532724+0000","last_clean":"2024-04-25T00:40:31.532724+0000","last_became_active":"2024-04-25T00:40:29.087595+0000","last_became_peered":"2024-04-25T00:40:29.087595+0000","last_unstale":"2024-04-25T00:40:31.532724+0000","last_undegraded":"2024-04-25T00:40:31.532724+0000","last_fullsized":"2024-04-25T00:40:31.532724+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:23.704167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327252+0000","last_change":"2024-04-25T00:40:30.108118+0000","last_active":"2024-04-25T00:40:31.327252+0000","last_peered":"2024-04-25T00:40:31.327252+0000","last_clean":"2024-04-25T00:40:31.327252+0000","last_became_active":"2024-04-25T00:40:30.107850+0000","last_became_peered":"2024-04-25T00:40:30.107850+0000","last_unstale":"2024-04-25T00:40:31.327252+0000","last_undegraded":"2024-04-25T00:40:31.327252+0000","last_fullsized":"2024-04-25T00:40:31.327252+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:39.019304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"155'6","reported_seq":20,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109260+0000","last_change":"2024-04-25T00:40:29.099633+0000","last_active":"2024-04-25T00:40:31.109260+0000","last_peered":"2024-04-25T00:40:31.109260+0000","last_clean":"2024-04-25T00:40:31.109260+0000","last_became_active":"2024-04-25T00:40:29.099060+0000","last_became_peered":"2024-04-25T00:40:29.099060+0000","last_unstale":"2024-04-25T00:40:31.109260+0000","last_undegraded":"2024-04-25T00:40:31.109260+0000","last_fullsized":"2024-04-25T00:40:31.109260+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:01.168619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109428+0000","last_change":"2024-04-25T00:40:30.110005+0000","last_active":"2024-04-25T00:40:31.109428+0000","last_peered":"2024-04-25T00:40:31.109428+0000","last_clean":"2024-04-25T00:40:31.109428+0000","last_became_active":"2024-04-25T00:40:30.109578+0000","last_became_peered":"2024-04-25T00:40:30.109578+0000","last_unstale":"2024-04-25T00:40:31.109428+0000","last_undegraded":"2024-04-25T00:40:31.109428+0000","last_fullsized":"2024-04-25T00:40:31.109428+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:07.296465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.108569+0000","last_change":"2024-04-25T00:40:29.088653+0000","last_active":"2024-04-25T00:40:31.108569+0000","last_peered":"2024-04-25T00:40:31.108569+0000","last_clean":"2024-04-25T00:40:31.108569+0000","last_became_active":"2024-04-25T00:40:29.087923+0000","last_became_peered":"2024-04-25T00:40:29.087923+0000","last_unstale":"2024-04-25T00:40:31.108569+0000","last_undegraded":"2024-04-25T00:40:31.108569+0000","last_fullsized":"2024-04-25T00:40:31.108569+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:10.274642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.110246+0000","last_change":"2024-04-25T00:40:30.128973+0000","last_active":"2024-04-25T00:40:31.110246+0000","last_peered":"2024-04-25T00:40:31.110246+0000","last_clean":"2024-04-25T00:40:31.110246+0000","last_became_active":"2024-04-25T00:40:30.113153+0000","last_became_peered":"2024-04-25T00:40:30.113153+0000","last_unstale":"2024-04-25T00:40:31.110246+0000","last_undegraded":"2024-04-25T00:40:31.110246+0000","last_fullsized":"2024-04-25T00:40:31.110246+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:28.342905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327629+0000","last_change":"2024-04-25T00:40:29.088591+0000","last_active":"2024-04-25T00:40:31.327629+0000","last_peered":"2024-04-25T00:40:31.327629+0000","last_clean":"2024-04-25T00:40:31.327629+0000","last_became_active":"2024-04-25T00:40:29.088270+0000","last_became_peered":"2024-04-25T00:40:29.088270+0000","last_unstale":"2024-04-25T00:40:31.327629+0000","last_undegraded":"2024-04-25T00:40:31.327629+0000","last_fullsized":"2024-04-25T00:40:31.327629+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:23.222465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111219+0000","last_change":"2024-04-25T00:40:30.118420+0000","last_active":"2024-04-25T00:40:31.111219+0000","last_peered":"2024-04-25T00:40:31.111219+0000","last_clean":"2024-04-25T00:40:31.111219+0000","last_became_active":"2024-04-25T00:40:30.117750+0000","last_became_peered":"2024-04-25T00:40:30.117750+0000","last_unstale":"2024-04-25T00:40:31.111219+0000","last_undegraded":"2024-04-25T00:40:31.111219+0000","last_fullsized":"2024-04-25T00:40:31.111219+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:51.453502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.122687+0000","last_change":"2024-04-25T00:40:29.100762+0000","last_active":"2024-04-25T00:40:32.122687+0000","last_peered":"2024-04-25T00:40:32.122687+0000","last_clean":"2024-04-25T00:40:32.122687+0000","last_became_active":"2024-04-25T00:40:29.100159+0000","last_became_peered":"2024-04-25T00:40:29.100159+0000","last_unstale":"2024-04-25T00:40:32.122687+0000","last_undegraded":"2024-04-25T00:40:32.122687+0000","last_fullsized":"2024-04-25T00:40:32.122687+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:13.385254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.326896+0000","last_change":"2024-04-25T00:40:30.137144+0000","last_active":"2024-04-25T00:40:31.326896+0000","last_peered":"2024-04-25T00:40:31.326896+0000","last_clean":"2024-04-25T00:40:31.326896+0000","last_became_active":"2024-04-25T00:40:30.136889+0000","last_became_peered":"2024-04-25T00:40:30.136889+0000","last_unstale":"2024-04-25T00:40:31.326896+0000","last_undegraded":"2024-04-25T00:40:31.326896+0000","last_fullsized":"2024-04-25T00:40:31.326896+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:06.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"155'8","reported_seq":22,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.106984+0000","last_change":"2024-04-25T00:40:29.089205+0000","last_active":"2024-04-25T00:40:31.106984+0000","last_peered":"2024-04-25T00:40:31.106984+0000","last_clean":"2024-04-25T00:40:31.106984+0000","last_became_active":"2024-04-25T00:40:29.088497+0000","last_became_peered":"2024-04-25T00:40:29.088497+0000","last_unstale":"2024-04-25T00:40:31.106984+0000","last_undegraded":"2024-04-25T00:40:31.106984+0000","last_fullsized":"2024-04-25T00:40:31.106984+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:10.767826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327704+0000","last_change":"2024-04-25T00:40:30.137077+0000","last_active":"2024-04-25T00:40:31.327704+0000","last_peered":"2024-04-25T00:40:31.327704+0000","last_clean":"2024-04-25T00:40:31.327704+0000","last_became_active":"2024-04-25T00:40:30.136823+0000","last_became_peered":"2024-04-25T00:40:30.136823+0000","last_unstale":"2024-04-25T00:40:31.327704+0000","last_undegraded":"2024-04-25T00:40:31.327704+0000","last_fullsized":"2024-04-25T00:40:31.327704+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:21.729541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.533413+0000","last_change":"2024-04-25T00:40:29.093336+0000","last_active":"2024-04-25T00:40:31.533413+0000","last_peered":"2024-04-25T00:40:31.533413+0000","last_clean":"2024-04-25T00:40:31.533413+0000","last_became_active":"2024-04-25T00:40:29.092530+0000","last_became_peered":"2024-04-25T00:40:29.092530+0000","last_unstale":"2024-04-25T00:40:31.533413+0000","last_undegraded":"2024-04-25T00:40:31.533413+0000","last_fullsized":"2024-04-25T00:40:31.533413+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:06.422475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.113226+0000","last_change":"2024-04-25T00:40:30.119303+0000","last_active":"2024-04-25T00:40:31.113226+0000","last_peered":"2024-04-25T00:40:31.113226+0000","last_clean":"2024-04-25T00:40:31.113226+0000","last_became_active":"2024-04-25T00:40:30.118527+0000","last_became_peered":"2024-04-25T00:40:30.118527+0000","last_unstale":"2024-04-25T00:40:31.113226+0000","last_undegraded":"2024-04-25T00:40:31.113226+0000","last_fullsized":"2024-04-25T00:40:31.113226+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:25.494199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.124155+0000","last_change":"2024-04-25T00:40:29.090555+0000","last_active":"2024-04-25T00:40:32.124155+0000","last_peered":"2024-04-25T00:40:32.124155+0000","last_clean":"2024-04-25T00:40:32.124155+0000","last_became_active":"2024-04-25T00:40:29.089983+0000","last_became_peered":"2024-04-25T00:40:29.089983+0000","last_unstale":"2024-04-25T00:40:32.124155+0000","last_undegraded":"2024-04-25T00:40:32.124155+0000","last_fullsized":"2024-04-25T00:40:32.124155+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:11.663375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.112152+0000","last_change":"2024-04-25T00:40:30.096459+0000","last_active":"2024-04-25T00:40:31.112152+0000","last_peered":"2024-04-25T00:40:31.112152+0000","last_clean":"2024-04-25T00:40:31.112152+0000","last_became_active":"2024-04-25T00:40:30.095676+0000","last_became_peered":"2024-04-25T00:40:30.095676+0000","last_unstale":"2024-04-25T00:40:31.112152+0000","last_undegraded":"2024-04-25T00:40:31.112152+0000","last_fullsized":"2024-04-25T00:40:31.112152+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:47.114707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109123+0000","last_change":"2024-04-25T00:40:29.088835+0000","last_active":"2024-04-25T00:40:31.109123+0000","last_peered":"2024-04-25T00:40:31.109123+0000","last_clean":"2024-04-25T00:40:31.109123+0000","last_became_active":"2024-04-25T00:40:29.088099+0000","last_became_peered":"2024-04-25T00:40:29.088099+0000","last_unstale":"2024-04-25T00:40:31.109123+0000","last_undegraded":"2024-04-25T00:40:31.109123+0000","last_fullsized":"2024-04-25T00:40:31.109123+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:45:13.430795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.127103+0000","last_change":"2024-04-25T00:40:30.119352+0000","last_active":"2024-04-25T00:40:32.127103+0000","last_peered":"2024-04-25T00:40:32.127103+0000","last_clean":"2024-04-25T00:40:32.127103+0000","last_became_active":"2024-04-25T00:40:30.118647+0000","last_became_peered":"2024-04-25T00:40:30.118647+0000","last_unstale":"2024-04-25T00:40:32.127103+0000","last_undegraded":"2024-04-25T00:40:32.127103+0000","last_fullsized":"2024-04-25T00:40:32.127103+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:07.296465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111457+0000","last_change":"2024-04-25T00:40:29.095787+0000","last_active":"2024-04-25T00:40:31.111457+0000","last_peered":"2024-04-25T00:40:31.111457+0000","last_clean":"2024-04-25T00:40:31.111457+0000","last_became_active":"2024-04-25T00:40:29.094547+0000","last_became_peered":"2024-04-25T00:40:29.094547+0000","last_unstale":"2024-04-25T00:40:31.111457+0000","last_undegraded":"2024-04-25T00:40:31.111457+0000","last_fullsized":"2024-04-25T00:40:31.111457+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:24.483710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327374+0000","last_change":"2024-04-25T00:40:30.109213+0000","last_active":"2024-04-25T00:40:31.327374+0000","last_peered":"2024-04-25T00:40:31.327374+0000","last_clean":"2024-04-25T00:40:31.327374+0000","last_became_active":"2024-04-25T00:40:30.108785+0000","last_became_peered":"2024-04-25T00:40:30.108785+0000","last_unstale":"2024-04-25T00:40:31.327374+0000","last_undegraded":"2024-04-25T00:40:31.327374+0000","last_fullsized":"2024-04-25T00:40:31.327374+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:31.042850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109143+0000","last_change":"2024-04-25T00:40:29.095275+0000","last_active":"2024-04-25T00:40:31.109143+0000","last_peered":"2024-04-25T00:40:31.109143+0000","last_clean":"2024-04-25T00:40:31.109143+0000","last_became_active":"2024-04-25T00:40:29.094973+0000","last_became_peered":"2024-04-25T00:40:29.094973+0000","last_unstale":"2024-04-25T00:40:31.109143+0000","last_undegraded":"2024-04-25T00:40:31.109143+0000","last_fullsized":"2024-04-25T00:40:31.109143+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:50.443013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.110421+0000","last_change":"2024-04-25T00:40:30.129513+0000","last_active":"2024-04-25T00:40:31.110421+0000","last_peered":"2024-04-25T00:40:31.110421+0000","last_clean":"2024-04-25T00:40:31.110421+0000","last_became_active":"2024-04-25T00:40:30.113812+0000","last_became_peered":"2024-04-25T00:40:30.113812+0000","last_unstale":"2024-04-25T00:40:31.110421+0000","last_undegraded":"2024-04-25T00:40:31.110421+0000","last_fullsized":"2024-04-25T00:40:31.110421+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:36.066823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532754+0000","last_change":"2024-04-25T00:40:29.099837+0000","last_active":"2024-04-25T00:40:31.532754+0000","last_peered":"2024-04-25T00:40:31.532754+0000","last_clean":"2024-04-25T00:40:31.532754+0000","last_became_active":"2024-04-25T00:40:29.096352+0000","last_became_peered":"2024-04-25T00:40:29.096352+0000","last_unstale":"2024-04-25T00:40:31.532754+0000","last_undegraded":"2024-04-25T00:40:31.532754+0000","last_fullsized":"2024-04-25T00:40:31.532754+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:04.477309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.126260+0000","last_change":"2024-04-25T00:40:30.109579+0000","last_active":"2024-04-25T00:40:32.126260+0000","last_peered":"2024-04-25T00:40:32.126260+0000","last_clean":"2024-04-25T00:40:32.126260+0000","last_became_active":"2024-04-25T00:40:30.108793+0000","last_became_peered":"2024-04-25T00:40:30.108793+0000","last_unstale":"2024-04-25T00:40:32.126260+0000","last_undegraded":"2024-04-25T00:40:32.126260+0000","last_fullsized":"2024-04-25T00:40:32.126260+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:43.387904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109271+0000","last_change":"2024-04-25T00:40:29.097310+0000","last_active":"2024-04-25T00:40:31.109271+0000","last_peered":"2024-04-25T00:40:31.109271+0000","last_clean":"2024-04-25T00:40:31.109271+0000","last_became_active":"2024-04-25T00:40:29.096881+0000","last_became_peered":"2024-04-25T00:40:29.096881+0000","last_unstale":"2024-04-25T00:40:31.109271+0000","last_undegraded":"2024-04-25T00:40:31.109271+0000","last_fullsized":"2024-04-25T00:40:31.109271+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:46.104218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532371+0000","last_change":"2024-04-25T00:40:30.104225+0000","last_active":"2024-04-25T00:40:31.532371+0000","last_peered":"2024-04-25T00:40:31.532371+0000","last_clean":"2024-04-25T00:40:31.532371+0000","last_became_active":"2024-04-25T00:40:30.103790+0000","last_became_peered":"2024-04-25T00:40:30.103790+0000","last_unstale":"2024-04-25T00:40:31.532371+0000","last_undegraded":"2024-04-25T00:40:31.532371+0000","last_fullsized":"2024-04-25T00:40:31.532371+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:43.028353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.111497+0000","last_change":"2024-04-25T00:40:29.083696+0000","last_active":"2024-04-25T00:40:31.111497+0000","last_peered":"2024-04-25T00:40:31.111497+0000","last_clean":"2024-04-25T00:40:31.111497+0000","last_became_active":"2024-04-25T00:40:29.082983+0000","last_became_peered":"2024-04-25T00:40:29.082983+0000","last_unstale":"2024-04-25T00:40:31.111497+0000","last_undegraded":"2024-04-25T00:40:31.111497+0000","last_fullsized":"2024-04-25T00:40:31.111497+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:08.779991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532823+0000","last_change":"2024-04-25T00:40:30.108256+0000","last_active":"2024-04-25T00:40:31.532823+0000","last_peered":"2024-04-25T00:40:31.532823+0000","last_clean":"2024-04-25T00:40:31.532823+0000","last_became_active":"2024-04-25T00:40:30.107857+0000","last_became_peered":"2024-04-25T00:40:30.107857+0000","last_unstale":"2024-04-25T00:40:31.532823+0000","last_undegraded":"2024-04-25T00:40:31.532823+0000","last_fullsized":"2024-04-25T00:40:31.532823+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:58.358658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109387+0000","last_change":"2024-04-25T00:40:29.098261+0000","last_active":"2024-04-25T00:40:31.109387+0000","last_peered":"2024-04-25T00:40:31.109387+0000","last_clean":"2024-04-25T00:40:31.109387+0000","last_became_active":"2024-04-25T00:40:29.097558+0000","last_became_peered":"2024-04-25T00:40:29.097558+0000","last_unstale":"2024-04-25T00:40:31.109387+0000","last_undegraded":"2024-04-25T00:40:31.109387+0000","last_fullsized":"2024-04-25T00:40:31.109387+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:05.401875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.328353+0000","last_change":"2024-04-25T00:40:30.112378+0000","last_active":"2024-04-25T00:40:31.328353+0000","last_peered":"2024-04-25T00:40:31.328353+0000","last_clean":"2024-04-25T00:40:31.328353+0000","last_became_active":"2024-04-25T00:40:30.112051+0000","last_became_peered":"2024-04-25T00:40:30.112051+0000","last_unstale":"2024-04-25T00:40:31.328353+0000","last_undegraded":"2024-04-25T00:40:31.328353+0000","last_fullsized":"2024-04-25T00:40:31.328353+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:31.152730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327734+0000","last_change":"2024-04-25T00:40:29.096322+0000","last_active":"2024-04-25T00:40:31.327734+0000","last_peered":"2024-04-25T00:40:31.327734+0000","last_clean":"2024-04-25T00:40:31.327734+0000","last_became_active":"2024-04-25T00:40:29.096034+0000","last_became_peered":"2024-04-25T00:40:29.096034+0000","last_unstale":"2024-04-25T00:40:31.327734+0000","last_undegraded":"2024-04-25T00:40:31.327734+0000","last_fullsized":"2024-04-25T00:40:31.327734+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:11.663375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123266+0000","last_change":"2024-04-25T00:40:30.106946+0000","last_active":"2024-04-25T00:40:32.123266+0000","last_peered":"2024-04-25T00:40:32.123266+0000","last_clean":"2024-04-25T00:40:32.123266+0000","last_became_active":"2024-04-25T00:40:30.106677+0000","last_became_peered":"2024-04-25T00:40:30.106677+0000","last_unstale":"2024-04-25T00:40:32.123266+0000","last_undegraded":"2024-04-25T00:40:32.123266+0000","last_fullsized":"2024-04-25T00:40:32.123266+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:06.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.327426+0000","last_change":"2024-04-25T00:40:29.090943+0000","last_active":"2024-04-25T00:40:31.327426+0000","last_peered":"2024-04-25T00:40:31.327426+0000","last_clean":"2024-04-25T00:40:31.327426+0000","last_became_active":"2024-04-25T00:40:29.090524+0000","last_became_peered":"2024-04-25T00:40:29.090524+0000","last_unstale":"2024-04-25T00:40:31.327426+0000","last_undegraded":"2024-04-25T00:40:31.327426+0000","last_fullsized":"2024-04-25T00:40:31.327426+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:13.385254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.123783+0000","last_change":"2024-04-25T00:40:30.115306+0000","last_active":"2024-04-25T00:40:32.123783+0000","last_peered":"2024-04-25T00:40:32.123783+0000","last_clean":"2024-04-25T00:40:32.123783+0000","last_became_active":"2024-04-25T00:40:30.114966+0000","last_became_peered":"2024-04-25T00:40:30.114966+0000","last_unstale":"2024-04-25T00:40:32.123783+0000","last_undegraded":"2024-04-25T00:40:32.123783+0000","last_fullsized":"2024-04-25T00:40:32.123783+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:39.019304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.122922+0000","last_change":"2024-04-25T00:40:29.097144+0000","last_active":"2024-04-25T00:40:32.122922+0000","last_peered":"2024-04-25T00:40:32.122922+0000","last_clean":"2024-04-25T00:40:32.122922+0000","last_became_active":"2024-04-25T00:40:29.096558+0000","last_became_peered":"2024-04-25T00:40:29.096558+0000","last_unstale":"2024-04-25T00:40:32.122922+0000","last_undegraded":"2024-04-25T00:40:32.122922+0000","last_fullsized":"2024-04-25T00:40:32.122922+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:38.339737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.107695+0000","last_change":"2024-04-25T00:40:30.108781+0000","last_active":"2024-04-25T00:40:31.107695+0000","last_peered":"2024-04-25T00:40:31.107695+0000","last_clean":"2024-04-25T00:40:31.107695+0000","last_became_active":"2024-04-25T00:40:30.107905+0000","last_became_peered":"2024-04-25T00:40:30.107905+0000","last_unstale":"2024-04-25T00:40:31.107695+0000","last_undegraded":"2024-04-25T00:40:31.107695+0000","last_fullsized":"2024-04-25T00:40:31.107695+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:06.771363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.112681+0000","last_change":"2024-04-25T00:40:29.077975+0000","last_active":"2024-04-25T00:40:31.112681+0000","last_peered":"2024-04-25T00:40:31.112681+0000","last_clean":"2024-04-25T00:40:31.112681+0000","last_became_active":"2024-04-25T00:40:29.077218+0000","last_became_peered":"2024-04-25T00:40:29.077218+0000","last_unstale":"2024-04-25T00:40:31.112681+0000","last_undegraded":"2024-04-25T00:40:31.112681+0000","last_fullsized":"2024-04-25T00:40:31.112681+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:57.348169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.108265+0000","last_change":"2024-04-25T00:40:30.131943+0000","last_active":"2024-04-25T00:40:31.108265+0000","last_peered":"2024-04-25T00:40:31.108265+0000","last_clean":"2024-04-25T00:40:31.108265+0000","last_became_active":"2024-04-25T00:40:30.131502+0000","last_became_peered":"2024-04-25T00:40:30.131502+0000","last_unstale":"2024-04-25T00:40:31.108265+0000","last_undegraded":"2024-04-25T00:40:31.108265+0000","last_fullsized":"2024-04-25T00:40:31.108265+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:51.016368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.112097+0000","last_change":"2024-04-25T00:40:29.096474+0000","last_active":"2024-04-25T00:40:31.112097+0000","last_peered":"2024-04-25T00:40:31.112097+0000","last_clean":"2024-04-25T00:40:31.112097+0000","last_became_active":"2024-04-25T00:40:29.095752+0000","last_became_peered":"2024-04-25T00:40:29.095752+0000","last_unstale":"2024-04-25T00:40:31.112097+0000","last_undegraded":"2024-04-25T00:40:31.112097+0000","last_fullsized":"2024-04-25T00:40:31.112097+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:55.232375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.126336+0000","last_change":"2024-04-25T00:40:30.115688+0000","last_active":"2024-04-25T00:40:32.126336+0000","last_peered":"2024-04-25T00:40:32.126336+0000","last_clean":"2024-04-25T00:40:32.126336+0000","last_became_active":"2024-04-25T00:40:30.109711+0000","last_became_peered":"2024-04-25T00:40:30.109711+0000","last_unstale":"2024-04-25T00:40:32.126336+0000","last_undegraded":"2024-04-25T00:40:32.126336+0000","last_fullsized":"2024-04-25T00:40:32.126336+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:52.100866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:32.126668+0000","last_change":"2024-04-25T00:40:29.084462+0000","last_active":"2024-04-25T00:40:32.126668+0000","last_peered":"2024-04-25T00:40:32.126668+0000","last_clean":"2024-04-25T00:40:32.126668+0000","last_became_active":"2024-04-25T00:40:29.082507+0000","last_became_peered":"2024-04-25T00:40:29.082507+0000","last_unstale":"2024-04-25T00:40:32.126668+0000","last_undegraded":"2024-04-25T00:40:32.126668+0000","last_fullsized":"2024-04-25T00:40:32.126668+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:24.734293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.532534+0000","last_change":"2024-04-25T00:40:30.104073+0000","last_active":"2024-04-25T00:40:31.532534+0000","last_peered":"2024-04-25T00:40:31.532534+0000","last_clean":"2024-04-25T00:40:31.532534+0000","last_became_active":"2024-04-25T00:40:30.103701+0000","last_became_peered":"2024-04-25T00:40:30.103701+0000","last_unstale":"2024-04-25T00:40:31.532534+0000","last_undegraded":"2024-04-25T00:40:31.532534+0000","last_fullsized":"2024-04-25T00:40:31.532534+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:49.068054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.109098+0000","last_change":"2024-04-25T00:40:29.094661+0000","last_active":"2024-04-25T00:40:31.109098+0000","last_peered":"2024-04-25T00:40:31.109098+0000","last_clean":"2024-04-25T00:40:31.109098+0000","last_became_active":"2024-04-25T00:40:29.094039+0000","last_became_peered":"2024-04-25T00:40:29.094039+0000","last_unstale":"2024-04-25T00:40:31.109098+0000","last_undegraded":"2024-04-25T00:40:31.109098+0000","last_fullsized":"2024-04-25T00:40:31.109098+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:15.360840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.326595+0000","last_change":"2024-04-25T00:40:30.108874+0000","last_active":"2024-04-25T00:40:31.326595+0000","last_peered":"2024-04-25T00:40:31.326595+0000","last_clean":"2024-04-25T00:40:31.326595+0000","last_became_active":"2024-04-25T00:40:30.108573+0000","last_became_peered":"2024-04-25T00:40:30.108573+0000","last_unstale":"2024-04-25T00:40:31.326595+0000","last_undegraded":"2024-04-25T00:40:31.326595+0000","last_fullsized":"2024-04-25T00:40:31.326595+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:29.053452+0000","last_clean_scrub_stamp":"2024-04-25T00:40:29.053452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:42.110955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.107442+0000","last_change":"2024-04-25T00:40:29.101828+0000","last_active":"2024-04-25T00:40:31.107442+0000","last_peered":"2024-04-25T00:40:31.107442+0000","last_clean":"2024-04-25T00:40:31.107442+0000","last_became_active":"2024-04-25T00:40:29.101030+0000","last_became_peered":"2024-04-25T00:40:29.101030+0000","last_unstale":"2024-04-25T00:40:31.107442+0000","last_undegraded":"2024-04-25T00:40:31.107442+0000","last_fullsized":"2024-04-25T00:40:31.107442+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:40:28.042963+0000","last_clean_scrub_stamp":"2024-04-25T00:40:28.042963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:47.720411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":466,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T00:40:31.107285+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:40:31.107285+0000","last_peered":"2024-04-25T00:40:31.107285+0000","last_clean":"2024-04-25T00:40:31.107285+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:40:31.107285+0000","last_undegraded":"2024-04-25T00:40:31.107285+0000","last_fullsized":"2024-04-25T00:40:31.107285+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574961,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37848,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333992,"statfs":{"total":96636764160,"available":96598007808,"internally_reserved":0,"allocated":2568192,"data_stored":1459719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37260,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334580,"statfs":{"total":96636764160,"available":96598609920,"internally_reserved":0,"allocated":1966080,"data_stored":869247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574962,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":1966080,"data_stored":869247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574961,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37284,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334556,"statfs":{"total":96636764160,"available":96598585344,"internally_reserved":0,"allocated":1982464,"data_stored":871441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574962,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41436,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330404,"statfs":{"total":96636764160,"available":96594333696,"internally_reserved":0,"allocated":1978368,"data_stored":869441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574961,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37276,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334564,"statfs":{"total":96636764160,"available":96598593536,"internally_reserved":0,"allocated":1978368,"data_stored":869941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37280,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334560,"statfs":{"total":96636764160,"available":96598589440,"internally_reserved":0,"allocated":1978368,"data_stored":870859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574961,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42008,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329832,"statfs":{"total":96636764160,"available":96593747968,"internally_reserved":0,"allocated":2564096,"data_stored":1459637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:40:38.942 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:40:38.942 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:40:39.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.113+0000 7f13eca99640 1 Processor -- start 2024-04-25T00:40:39.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.113+0000 7f13eca99640 1 -- start start 2024-04-25T00:40:39.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13eca99640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13e80b3c30 0x7f13e806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13eca99640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e80b41e0 con 0x7f13e80b3140 2024-04-25T00:40:39.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13eca99640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e806d960 con 0x7f13e80b4b10 2024-04-25T00:40:39.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13eca99640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f13e806dc40 con 0x7f13e80b3c30 2024-04-25T00:40:39.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e659c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f13e80b3140 0x7f13e80b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54416/0 (socket says 172.21.15.29:54416) 2024-04-25T00:40:39.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e659c640 1 -- 172.21.15.29:0/3886983899 learned_addr learned my addr 172.21.15.29:0/3886983899 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:39.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e5d9b640 1 --2- 172.21.15.29:0/3886983899 >> v2:172.21.15.157:3300/0 conn(0x7f13e80b3c30 0x7f13e806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4057996361 0 0) 0x7f13e80b41e0 con 0x7f13e80b3140 2024-04-25T00:40:39.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13d4003530 con 0x7f13e80b3140 2024-04-25T00:40:39.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1906773123 0 0) 0x7f13d4003530 con 0x7f13e80b3140 2024-04-25T00:40:39.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 >> v2:172.21.15.157:3300/0 conn(0x7f13e80b3c30 msgr2=0x7f13e806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:39.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 --2- 172.21.15.29:0/3886983899 >> v2:172.21.15.157:3300/0 conn(0x7f13e80b3c30 0x7f13e806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 >> v1:172.21.15.157:6789/0 conn(0x7f13e80b4b10 legacy=0x7f13e80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e806df20 con 0x7f13e80b3140 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f13e0001680 con 0x7f13e80b3140 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 >> v1:172.21.15.29:6789/0 conn(0x7f13e80b3140 legacy=0x7f13e80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 --2- 172.21.15.29:0/3886983899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d40039f0 0x7f13d4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 --2- 172.21.15.29:0/3886983899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d4004810 0x7f13d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 --2- 172.21.15.29:0/3886983899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d40051d0 0x7f13d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f13e806dc40 con 0x7f13d40051d0 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13d400c0a0 con 0x7f13d40039f0 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f13d400c2a0 con 0x7f13d4004810 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e6d9d640 1 --2- 172.21.15.29:0/3886983899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d40051d0 0x7f13d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e659c640 1 --2- 172.21.15.29:0/3886983899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d40039f0 0x7f13d4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e5d9b640 1 --2- 172.21.15.29:0/3886983899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d4004810 0x7f13d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e6d9d640 1 -- 172.21.15.29:0/3886983899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d4004810 msgr2=0x7f13d4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:39.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e6d9d640 1 --2- 172.21.15.29:0/3886983899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d4004810 0x7f13d4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e6d9d640 1 -- 172.21.15.29:0/3886983899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d40039f0 msgr2=0x7f13d4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:39.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e6d9d640 1 --2- 172.21.15.29:0/3886983899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d40039f0 0x7f13d4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e6d9d640 1 -- 172.21.15.29:0/3886983899 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d4003530 con 0x7f13d40051d0 2024-04-25T00:40:39.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.117+0000 7f13e6d9d640 1 -- 172.21.15.29:0/3886983899 reap_dead start 2024-04-25T00:40:39.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e6d9d640 1 --2- 172.21.15.29:0/3886983899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d40051d0 0x7f13d400b9d0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f13d8001e70 tx=0x7f13d801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47e96caf3c5449b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:39.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8001a90 con 0x7f13d40051d0 2024-04-25T00:40:39.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d8003ba0 con 0x7f13d40051d0 2024-04-25T00:40:39.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e559a640 1 -- 172.21.15.29:0/3886983899 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8005020 con 0x7f13d40051d0 2024-04-25T00:40:39.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 -- 172.21.15.29:0/3886983899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d40051d0 msgr2=0x7f13d400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 --2- 172.21.15.29:0/3886983899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d40051d0 0x7f13d400b9d0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f13d8001e70 tx=0x7f13d801c780 comp rx=0 tx=0).stop 2024-04-25T00:40:39.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 -- 172.21.15.29:0/3886983899 shutdown_connections 2024-04-25T00:40:39.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 --2- 172.21.15.29:0/3886983899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13d40051d0 0x7f13d400b9d0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 -- 172.21.15.29:0/3886983899 >> 172.21.15.29:0/3886983899 conn(0x7f13e80b1870 msgr2=0x7f13e80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:39.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 -- 172.21.15.29:0/3886983899 shutdown_connections 2024-04-25T00:40:39.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 -- 172.21.15.29:0/3886983899 wait complete. 2024-04-25T00:40:39.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 Processor -- start 2024-04-25T00:40:39.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 -- start start 2024-04-25T00:40:39.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d40051d0 0x7f13e80acad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13e81f3af0 0x7f13e80ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e80ae720 0x7f13e80aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f13e8145a90 con 0x7f13e81f3af0 2024-04-25T00:40:39.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13e8145910 con 0x7f13e80ae720 2024-04-25T00:40:39.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f13e806e440 con 0x7f13d40051d0 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13e81f3af0 0x7f13e80ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13e81f3af0 0x7f13e80ac140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60894/0 (socket says 172.21.15.29:60894) 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e5d9b640 1 -- 172.21.15.29:0/2322641514 learned_addr learned my addr 172.21.15.29:0/2322641514 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e6d9d640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e80ae720 0x7f13e80aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e659c640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d40051d0 0x7f13e80acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e5d9b640 1 -- 172.21.15.29:0/2322641514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d40051d0 msgr2=0x7f13e80acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e5d9b640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d40051d0 0x7f13e80acad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e5d9b640 1 -- 172.21.15.29:0/2322641514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e80ae720 msgr2=0x7f13e80aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e5d9b640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e80ae720 0x7f13e80aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e5d9b640 1 -- 172.21.15.29:0/2322641514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e81f3f30 con 0x7f13e81f3af0 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e659c640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d40051d0 0x7f13e80acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13e5d9b640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13e81f3af0 0x7f13e80ac140 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f13d0011590 tx=0x7f13d0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13deffd640 1 -- 172.21.15.29:0/2322641514 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0015930 con 0x7f13e81f3af0 2024-04-25T00:40:39.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13deffd640 1 -- 172.21.15.29:0/2322641514 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d0015ad0 con 0x7f13e81f3af0 2024-04-25T00:40:39.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.121+0000 7f13eca99640 1 -- 172.21.15.29:0/2322641514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e80a9490 con 0x7f13e81f3af0 2024-04-25T00:40:39.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.125+0000 7f13deffd640 1 -- 172.21.15.29:0/2322641514 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d000f460 con 0x7f13e81f3af0 2024-04-25T00:40:39.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.125+0000 7f13eca99640 1 -- 172.21.15.29:0/2322641514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e80ad470 con 0x7f13e81f3af0 2024-04-25T00:40:39.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.125+0000 7f13eca99640 1 -- 172.21.15.29:0/2322641514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e8069270 con 0x7f13e81f3af0 2024-04-25T00:40:39.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.125+0000 7f13deffd640 1 -- 172.21.15.29:0/2322641514 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f13d000f600 con 0x7f13e81f3af0 2024-04-25T00:40:39.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.125+0000 7f13deffd640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13c80680a0 0x7f13c806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.125+0000 7f13deffd640 1 -- 172.21.15.29:0/2322641514 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f13d00874b0 con 0x7f13e81f3af0 2024-04-25T00:40:39.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.125+0000 7f13e659c640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13c80680a0 0x7f13c806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.125+0000 7f13e659c640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13c80680a0 0x7f13c806a560 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f13e00017c0 tx=0x7f13e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:39.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.129+0000 7f13deffd640 1 -- 172.21.15.29:0/2322641514 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f13e81f3f30 con 0x7f13e81f3af0 2024-04-25T00:40:39.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.345+0000 7f13eca99640 1 -- 172.21.15.29:0/2322641514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f13e80ac8b0 con 0x7f13e81f3af0 2024-04-25T00:40:39.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.353+0000 7f13deffd640 1 -- 172.21.15.29:0/2322641514 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v27) ==== 190+0+147 (secure 0 0 0) 0x7f13d0019030 con 0x7f13e81f3af0 2024-04-25T00:40:39.351 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-04-25T00:40:39.351 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A== 2024-04-25T00:40:39.352 INFO:teuthology.orchestra.run.smithi029.stdout: caps mds = "allow" 2024-04-25T00:40:39.352 INFO:teuthology.orchestra.run.smithi029.stdout: caps mon = "allow r" 2024-04-25T00:40:39.352 INFO:teuthology.orchestra.run.smithi029.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:40:39.352 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-04-25T00:40:39.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.353+0000 7f13eca99640 1 -- 172.21.15.29:0/2322641514 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13c80680a0 msgr2=0x7f13c806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.353+0000 7f13eca99640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13c80680a0 0x7f13c806a560 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f13e00017c0 tx=0x7f13e0009040 comp rx=0 tx=0).stop 2024-04-25T00:40:39.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.353+0000 7f13eca99640 1 -- 172.21.15.29:0/2322641514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13e81f3af0 msgr2=0x7f13e80ac140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.353+0000 7f13eca99640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13e81f3af0 0x7f13e80ac140 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f13d0011590 tx=0x7f13d0011a50 comp rx=0 tx=0).stop 2024-04-25T00:40:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.353+0000 7f13eca99640 1 -- 172.21.15.29:0/2322641514 shutdown_connections 2024-04-25T00:40:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.353+0000 7f13eca99640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f13c80680a0 0x7f13c806a560 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.353+0000 7f13eca99640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e80ae720 0x7f13e80aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.353+0000 7f13eca99640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13e81f3af0 0x7f13e80ac140 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.353+0000 7f13eca99640 1 --2- 172.21.15.29:0/2322641514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d40051d0 0x7f13e80acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.353+0000 7f13eca99640 1 -- 172.21.15.29:0/2322641514 >> 172.21.15.29:0/2322641514 conn(0x7f13e80b1870 msgr2=0x7f13e80ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.353+0000 7f13eca99640 1 -- 172.21.15.29:0/2322641514 shutdown_connections 2024-04-25T00:40:39.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.353+0000 7f13eca99640 1 -- 172.21.15.29:0/2322641514 wait complete. 2024-04-25T00:40:39.370 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:40:39.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc716c60640 1 Processor -- start 2024-04-25T00:40:39.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc716c60640 1 -- start start 2024-04-25T00:40:39.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc716c60640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7100cf690 0x7fc7100d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc716c60640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7100cfc10 con 0x7fc7100d2a90 2024-04-25T00:40:39.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc716c60640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7100cfef0 con 0x7fc71015e200 2024-04-25T00:40:39.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc716c60640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc7100d2ee0 con 0x7fc7100cf690 2024-04-25T00:40:39.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc7151fd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc7100d2a90 0x7fc7100b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54418/0 (socket says 172.21.15.29:54418) 2024-04-25T00:40:39.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc7151fd640 1 -- 172.21.15.29:0/3651179565 learned_addr learned my addr 172.21.15.29:0/3651179565 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:39.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70ffff640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7100cf690 0x7fc7100d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1463213491 0 0) 0x7fc7100cfc10 con 0x7fc7100d2a90 2024-04-25T00:40:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6f8003530 con 0x7fc7100d2a90 2024-04-25T00:40:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 458022630 0 0) 0x7fc6f8003530 con 0x7fc7100d2a90 2024-04-25T00:40:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7100cf690 msgr2=0x7fc7100d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7100cf690 0x7fc7100d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 >> v1:172.21.15.157:6789/0 conn(0x7fc71015e200 legacy=0x7fc7100d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7100b3b20 con 0x7fc7100d2a90 2024-04-25T00:40:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fc708001a00 con 0x7fc7100d2a90 2024-04-25T00:40:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 >> v1:172.21.15.29:6789/0 conn(0x7fc7100d2a90 legacy=0x7fc7100b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f80039f0 0x7fc6f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7100cf690 0x7fc6f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6f8004d50 0x7fc6f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc7100d2ee0 con 0x7fc6f80039f0 2024-04-25T00:40:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f800ba80 con 0x7fc6f8004d50 2024-04-25T00:40:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f800bc80 con 0x7fc7100cf690 2024-04-25T00:40:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc7149fc640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f80039f0 0x7fc6f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70ffff640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7100cf690 0x7fc6f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc7151fd640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6f8004d50 0x7fc6f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc7149fc640 1 -- 172.21.15.29:0/3651179565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7100cf690 msgr2=0x7fc6f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc7149fc640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7100cf690 0x7fc6f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc7149fc640 1 -- 172.21.15.29:0/3651179565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6f8004d50 msgr2=0x7fc6f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc7149fc640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6f8004d50 0x7fc6f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc7149fc640 1 -- 172.21.15.29:0/3651179565 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f8003530 con 0x7fc6f80039f0 2024-04-25T00:40:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc7151fd640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6f8004d50 0x7fc6f80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:39.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.493+0000 7fc70ffff640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7100cf690 0x7fc6f8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:39.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.497+0000 7fc7149fc640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f80039f0 0x7fc6f8003e10 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fc700001750 tx=0x7fc70001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62e37932fcf0124 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:39.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.497+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7000013a0 con 0x7fc6f80039f0 2024-04-25T00:40:39.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.497+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7000052f0 con 0x7fc6f80039f0 2024-04-25T00:40:39.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.497+0000 7fc70f7fe640 1 -- 172.21.15.29:0/3651179565 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700003840 con 0x7fc6f80039f0 2024-04-25T00:40:39.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.497+0000 7fc716c60640 1 -- 172.21.15.29:0/3651179565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f80039f0 msgr2=0x7fc6f8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.497+0000 7fc716c60640 1 --2- 172.21.15.29:0/3651179565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f80039f0 0x7fc6f8003e10 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fc700001750 tx=0x7fc70001c3e0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.497+0000 7fc7151fd640 1 -- 172.21.15.29:0/3651179565 reap_dead start 2024-04-25T00:40:39.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.497+0000 7fc716c60640 1 -- 172.21.15.29:0/3651179565 shutdown_connections 2024-04-25T00:40:39.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.497+0000 7fc716c60640 1 -- 172.21.15.29:0/3651179565 >> 172.21.15.29:0/3651179565 conn(0x7fc71009d280 msgr2=0x7fc7100d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:39.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.497+0000 7fc716c60640 1 -- 172.21.15.29:0/3651179565 shutdown_connections 2024-04-25T00:40:39.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.497+0000 7fc716c60640 1 -- 172.21.15.29:0/3651179565 wait complete. 2024-04-25T00:40:39.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc716c60640 1 Processor -- start 2024-04-25T00:40:39.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc716c60640 1 -- start start 2024-04-25T00:40:39.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc716c60640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6f80039f0 0x7fc7101674f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc716c60640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc710168c20 0x7fc710156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc716c60640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc71015e8e0 0x7fc71015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc716c60640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc7100b55d0 con 0x7fc710168c20 2024-04-25T00:40:39.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc716c60640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7100b5450 con 0x7fc71015e8e0 2024-04-25T00:40:39.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc716c60640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc7100b38d0 con 0x7fc6f80039f0 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc70ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc710168c20 0x7fc710156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc70ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc710168c20 0x7fc710156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60908/0 (socket says 172.21.15.29:60908) 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc70ffff640 1 -- 172.21.15.29:0/675402966 learned_addr learned my addr 172.21.15.29:0/675402966 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc7151fd640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc71015e8e0 0x7fc71015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc7149fc640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6f80039f0 0x7fc7101674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc70ffff640 1 -- 172.21.15.29:0/675402966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6f80039f0 msgr2=0x7fc7101674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc70ffff640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6f80039f0 0x7fc7101674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc70ffff640 1 -- 172.21.15.29:0/675402966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc71015e8e0 msgr2=0x7fc71015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc70ffff640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc71015e8e0 0x7fc71015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc70ffff640 1 -- 172.21.15.29:0/675402966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc710157400 con 0x7fc710168c20 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc7149fc640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6f80039f0 0x7fc7101674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc70ffff640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc710168c20 0x7fc710156ec0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fc7040108d0 tx=0x7fc704010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc70d7fa640 1 -- 172.21.15.29:0/675402966 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70400f040 con 0x7fc710168c20 2024-04-25T00:40:39.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc70d7fa640 1 -- 172.21.15.29:0/675402966 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc704004220 con 0x7fc710168c20 2024-04-25T00:40:39.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc716c60640 1 -- 172.21.15.29:0/675402966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc710167b80 con 0x7fc710168c20 2024-04-25T00:40:39.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc70d7fa640 1 -- 172.21.15.29:0/675402966 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70400e960 con 0x7fc710168c20 2024-04-25T00:40:39.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc716c60640 1 -- 172.21.15.29:0/675402966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc710168140 con 0x7fc710168c20 2024-04-25T00:40:39.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.501+0000 7fc70d7fa640 1 -- 172.21.15.29:0/675402966 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fc70400eb00 con 0x7fc710168c20 2024-04-25T00:40:39.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.505+0000 7fc716c60640 1 -- 172.21.15.29:0/675402966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc710069270 con 0x7fc710168c20 2024-04-25T00:40:39.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.505+0000 7fc70d7fa640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc6fc0680a0 0x7fc6fc06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:39.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.505+0000 7fc7149fc640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc6fc0680a0 0x7fc6fc06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:39.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.505+0000 7fc70d7fa640 1 -- 172.21.15.29:0/675402966 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fc704084650 con 0x7fc710168c20 2024-04-25T00:40:39.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.505+0000 7fc7149fc640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc6fc0680a0 0x7fc6fc06a560 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc6f80057f0 tx=0x7fc7000011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:39.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.509+0000 7fc70d7fa640 1 -- 172.21.15.29:0/675402966 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc704018030 con 0x7fc710168c20 2024-04-25T00:40:39.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.705+0000 7fc716c60640 1 -- 172.21.15.29:0/675402966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7100008e0 con 0x7fc710168c20 2024-04-25T00:40:39.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.705+0000 7fc70d7fa640 1 -- 172.21.15.29:0/675402966 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5196 (secure 0 0 0) 0x7fc704056a80 con 0x7fc710168c20 2024-04-25T00:40:39.706 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:39.706 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2677876688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2677876688},{"type":"v1","addr":"172.21.15.157:6833","nonce":2677876688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3340901266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3340901266},{"type":"v1","addr":"172.21.15.157:6835","nonce":3340901266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":10092,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/567284262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":567284262},{"type":"v1","addr":"172.21.15.29:6833","nonce":567284262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:40:29.419450+0000","modified":"2024-04-25T00:40:32.484311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10095":{"gid":10095,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/4183931647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":4183931647},{"type":"v1","addr":"172.21.15.29:6835","nonce":4183931647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10095,"qdb_cluster":[10095]},"id":12}]} 2024-04-25T00:40:39.706 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 102 2024-04-25T00:40:39.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.709+0000 7fc716c60640 1 -- 172.21.15.29:0/675402966 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc6fc0680a0 msgr2=0x7fc6fc06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.709+0000 7fc716c60640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc6fc0680a0 0x7fc6fc06a560 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc6f80057f0 tx=0x7fc7000011a0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.709+0000 7fc716c60640 1 -- 172.21.15.29:0/675402966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc710168c20 msgr2=0x7fc710156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:39.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.709+0000 7fc716c60640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc710168c20 0x7fc710156ec0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fc7040108d0 tx=0x7fc704010d90 comp rx=0 tx=0).stop 2024-04-25T00:40:39.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.709+0000 7fc716c60640 1 -- 172.21.15.29:0/675402966 shutdown_connections 2024-04-25T00:40:39.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.709+0000 7fc716c60640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc6fc0680a0 0x7fc6fc06a560 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.709+0000 7fc716c60640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc71015e8e0 0x7fc71015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.709+0000 7fc716c60640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc710168c20 0x7fc710156ec0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.709+0000 7fc716c60640 1 --2- 172.21.15.29:0/675402966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6f80039f0 0x7fc7101674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:39.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.709+0000 7fc716c60640 1 -- 172.21.15.29:0/675402966 >> 172.21.15.29:0/675402966 conn(0x7fc71009d280 msgr2=0x7fc710157f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:39.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.709+0000 7fc716c60640 1 -- 172.21.15.29:0/675402966 shutdown_connections 2024-04-25T00:40:39.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:39.709+0000 7fc716c60640 1 -- 172.21.15.29:0/675402966 wait complete. 2024-04-25T00:40:40.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:40:29.419450+0000', 'modified': '2024-04-25T00:40:32.484311+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10095}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10095': {'gid': 10095, 'name': 'c', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/4183931647', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 4183931647}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 4183931647}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10095, 'qdb_cluster': [10095]} 2024-04-25T00:40:40.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:40:40.129 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:40:40.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.265+0000 7f7dbc900640 1 Processor -- start 2024-04-25T00:40:40.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.265+0000 7f7dbc900640 1 -- start start 2024-04-25T00:40:40.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.265+0000 7f7dbc900640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7db80b2b70 0x7f7db8151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.265+0000 7f7dbc900640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db80b3130 con 0x7f7db806dce0 2024-04-25T00:40:40.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.265+0000 7f7dbc900640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db80b3410 con 0x7f7db815d980 2024-04-25T00:40:40.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.265+0000 7f7dbc900640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7db80cf560 con 0x7f7db80b2b70 2024-04-25T00:40:40.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.265+0000 7f7db5d9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7db806dce0 0x7f7db8150df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54424/0 (socket says 172.21.15.29:54424) 2024-04-25T00:40:40.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.265+0000 7f7db5d9b640 1 -- 172.21.15.29:0/1236299490 learned_addr learned my addr 172.21.15.29:0/1236299490 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.265+0000 7f7db6d9d640 1 --2- 172.21.15.29:0/1236299490 >> v2:172.21.15.157:3300/0 conn(0x7f7db80b2b70 0x7f7db8151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.265+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3435006956 0 0) 0x7f7db80b3130 con 0x7f7db806dce0 2024-04-25T00:40:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7da4003530 con 0x7f7db806dce0 2024-04-25T00:40:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4191630671 0 0) 0x7f7da4003530 con 0x7f7db806dce0 2024-04-25T00:40:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 >> v2:172.21.15.157:3300/0 conn(0x7f7db80b2b70 msgr2=0x7f7db8151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 --2- 172.21.15.29:0/1236299490 >> v2:172.21.15.157:3300/0 conn(0x7f7db80b2b70 0x7f7db8151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 >> v1:172.21.15.157:6789/0 conn(0x7f7db815d980 legacy=0x7f7db806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db80b36f0 con 0x7f7db806dce0 2024-04-25T00:40:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f7da0001680 con 0x7f7db806dce0 2024-04-25T00:40:40.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 >> v1:172.21.15.29:6789/0 conn(0x7f7db806dce0 legacy=0x7f7db8150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 --2- 172.21.15.29:0/1236299490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40039c0 0x7f7da4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 --2- 172.21.15.29:0/1236299490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da4004370 0x7f7da4004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 --2- 172.21.15.29:0/1236299490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da4005030 0x7f7da400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7db80cf560 con 0x7f7da40039c0 2024-04-25T00:40:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db659c640 1 --2- 172.21.15.29:0/1236299490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40039c0 0x7f7da4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7da400bf70 con 0x7f7da4004370 2024-04-25T00:40:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7da400c170 con 0x7f7da4005030 2024-04-25T00:40:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/1236299490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da4004370 0x7f7da4004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db659c640 1 -- 172.21.15.29:0/1236299490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da4005030 msgr2=0x7f7da400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db6d9d640 1 --2- 172.21.15.29:0/1236299490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da4005030 0x7f7da400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db659c640 1 --2- 172.21.15.29:0/1236299490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da4005030 0x7f7da400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db659c640 1 -- 172.21.15.29:0/1236299490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da4004370 msgr2=0x7f7da4004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db659c640 1 --2- 172.21.15.29:0/1236299490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da4004370 0x7f7da4004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db659c640 1 -- 172.21.15.29:0/1236299490 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da4003530 con 0x7f7da40039c0 2024-04-25T00:40:40.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db6d9d640 1 -- 172.21.15.29:0/1236299490 reap_dead start 2024-04-25T00:40:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db659c640 1 --2- 172.21.15.29:0/1236299490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40039c0 0x7f7da4003da0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f7da8001860 tx=0x7f7da801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3f96c2be8a2cae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80b1070 con 0x7f7da40039c0 2024-04-25T00:40:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da80ac410 con 0x7f7da40039c0 2024-04-25T00:40:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7db559a640 1 -- 172.21.15.29:0/1236299490 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8001460 con 0x7f7da40039c0 2024-04-25T00:40:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7dbc900640 1 -- 172.21.15.29:0/1236299490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40039c0 msgr2=0x7f7da4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7dbc900640 1 --2- 172.21.15.29:0/1236299490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40039c0 0x7f7da4003da0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f7da8001860 tx=0x7f7da801c4f0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7dbc900640 1 -- 172.21.15.29:0/1236299490 shutdown_connections 2024-04-25T00:40:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7dbc900640 1 --2- 172.21.15.29:0/1236299490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40039c0 0x7f7da4003da0 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.269+0000 7f7dbc900640 1 -- 172.21.15.29:0/1236299490 >> 172.21.15.29:0/1236299490 conn(0x7f7db80b1870 msgr2=0x7f7db80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:40.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7dbc900640 1 -- 172.21.15.29:0/1236299490 shutdown_connections 2024-04-25T00:40:40.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7dbc900640 1 -- 172.21.15.29:0/1236299490 wait complete. 2024-04-25T00:40:40.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7dbc900640 1 Processor -- start 2024-04-25T00:40:40.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7dbc900640 1 -- start start 2024-04-25T00:40:40.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7dbc900640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40039c0 0x7f7db8157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7dbc900640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db8155060 0x7f7db81f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7dbc900640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7db80cfb30 0x7f7db80cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7dbc900640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7db806db60 con 0x7f7da40039c0 2024-04-25T00:40:40.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7dbc900640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7db806d9e0 con 0x7f7db8155060 2024-04-25T00:40:40.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7dbc900640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7db8151600 con 0x7f7db80cfb30 2024-04-25T00:40:40.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7db659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40039c0 0x7f7db8157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7db659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40039c0 0x7f7db8157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60922/0 (socket says 172.21.15.29:60922) 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7db659c640 1 -- 172.21.15.29:0/3660966350 learned_addr learned my addr 172.21.15.29:0/3660966350 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db8155060 0x7f7db81f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7db6d9d640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7db80cfb30 0x7f7db80cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7db659c640 1 -- 172.21.15.29:0/3660966350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7db80cfb30 msgr2=0x7f7db80cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7db659c640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7db80cfb30 0x7f7db80cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7db659c640 1 -- 172.21.15.29:0/3660966350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db8155060 msgr2=0x7f7db81f6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7db659c640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db8155060 0x7f7db81f6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7db659c640 1 -- 172.21.15.29:0/3660966350 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db8158d30 con 0x7f7da40039c0 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7db6d9d640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7db80cfb30 0x7f7db80cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7db659c640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40039c0 0x7f7db8157aa0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f7da8000c10 tx=0x7f7da4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7daeffd640 1 -- 172.21.15.29:0/3660966350 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80b1040 con 0x7f7da40039c0 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7daeffd640 1 -- 172.21.15.29:0/3660966350 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da80067e0 con 0x7f7da40039c0 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7dbc900640 1 -- 172.21.15.29:0/3660966350 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db8159010 con 0x7f7da40039c0 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7daeffd640 1 -- 172.21.15.29:0/3660966350 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8006b90 con 0x7f7da40039c0 2024-04-25T00:40:40.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.273+0000 7f7dbc900640 1 -- 172.21.15.29:0/3660966350 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db81595d0 con 0x7f7da40039c0 2024-04-25T00:40:40.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7daeffd640 1 -- 172.21.15.29:0/3660966350 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f7da80bc060 con 0x7f7da40039c0 2024-04-25T00:40:40.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7daeffd640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d980680d0 0x7f7d9806a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7daeffd640 1 -- 172.21.15.29:0/3660966350 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f7da811d3f0 con 0x7f7da40039c0 2024-04-25T00:40:40.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d980680d0 0x7f7d9806a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d980680d0 0x7f7d9806a590 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7da0001b30 tx=0x7f7da000a040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:40.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7dbfa79000 1 Processor -- start 2024-04-25T00:40:40.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7dbfa79000 1 -- start start 2024-04-25T00:40:40.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7dbfa79000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c95d0ef0 0x55b2c95d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7dbfa79000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c95d1810 0x55b2c9587680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7dbfa79000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c9587bc0 0x55b2c9587fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7dbfa79000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b2c93d55e0 con 0x55b2c95d0ef0 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7dbfa79000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b2c935cc50 con 0x55b2c95d1810 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7dbfa79000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55b2c93d15a0 con 0x55b2c9587bc0 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7db659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c95d0ef0 0x55b2c95d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7db659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c95d0ef0 0x55b2c95d12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60936/0 (socket says 172.21.15.29:60936) 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7db659c640 1 -- 172.21.15.29:0/1345390113 learned_addr learned my addr 172.21.15.29:0/1345390113 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7db6d9d640 1 --2- 172.21.15.29:0/1345390113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c9587bc0 0x55b2c9587fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/1345390113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c95d1810 0x55b2c9587680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7db659c640 1 -- 172.21.15.29:0/1345390113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c9587bc0 msgr2=0x55b2c9587fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7db659c640 1 --2- 172.21.15.29:0/1345390113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c9587bc0 0x55b2c9587fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7db659c640 1 -- 172.21.15.29:0/1345390113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c95d1810 msgr2=0x55b2c9587680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7db659c640 1 --2- 172.21.15.29:0/1345390113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c95d1810 0x55b2c9587680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7db659c640 1 -- 172.21.15.29:0/1345390113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c935ad10 con 0x55b2c95d0ef0 2024-04-25T00:40:40.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7db659c640 1 --2- 172.21.15.29:0/1345390113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c95d0ef0 0x55b2c95d12d0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f7da80af040 tx=0x7f7da80ef8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa46ca6c9234c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7d97fff640 1 -- 172.21.15.29:0/1345390113 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80f7d00 con 0x55b2c95d0ef0 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7d97fff640 1 -- 172.21.15.29:0/1345390113 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da80ed1c0 con 0x55b2c95d0ef0 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.277+0000 7f7d97fff640 1 -- 172.21.15.29:0/1345390113 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8006d30 con 0x55b2c95d0ef0 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 -- 172.21.15.29:0/1345390113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c95d0ef0 msgr2=0x55b2c95d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/1345390113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c95d0ef0 0x55b2c95d12d0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f7da80af040 tx=0x7f7da80ef8e0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 -- 172.21.15.29:0/1345390113 shutdown_connections 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/1345390113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c9587bc0 0x55b2c9587fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/1345390113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c95d1810 0x55b2c9587680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/1345390113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c95d0ef0 0x55b2c95d12d0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 -- 172.21.15.29:0/1345390113 >> 172.21.15.29:0/1345390113 conn(0x55b2c9583b70 msgr2=0x55b2c9585f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 -- 172.21.15.29:0/1345390113 shutdown_connections 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 -- 172.21.15.29:0/1345390113 wait complete. 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 -- start start 2024-04-25T00:40:40.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 Processor -- start 2024-04-25T00:40:40.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c95b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95d0ef0 0x55b2c95b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c95d1810 0x55b2c95b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b2c93d55e0 con 0x55b2c9587bc0 2024-04-25T00:40:40.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b2c938f520 con 0x55b2c95d1810 2024-04-25T00:40:40.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55b2c93d15a0 con 0x55b2c95d0ef0 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 10 client.0 _ll_register_callbacks cb 0x55b2c95725c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c95b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c95b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60952/0 (socket says 172.21.15.29:60952) 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db659c640 1 -- 172.21.15.29:0/596453342 learned_addr learned my addr 172.21.15.29:0/596453342 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95d0ef0 0x55b2c95b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db6d9d640 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c95d1810 0x55b2c95b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db659c640 1 -- 172.21.15.29:0/596453342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95d0ef0 msgr2=0x55b2c95b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db659c640 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95d0ef0 0x55b2c95b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db659c640 1 -- 172.21.15.29:0/596453342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c95d1810 msgr2=0x55b2c95b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db659c640 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c95d1810 0x55b2c95b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db659c640 1 -- 172.21.15.29:0/596453342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c936d7b0 con 0x55b2c9587bc0 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95d0ef0 0x55b2c95b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db6d9d640 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c95d1810 0x55b2c95b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:40.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db659c640 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c95b52c0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f7da80ef060 tx=0x7f7da8007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f19a8b51ec62e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7d957fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7d957fa640 1 -- 172.21.15.29:0/596453342 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80f3170 con 0x55b2c9587bc0 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7d957fa640 1 -- 172.21.15.29:0/596453342 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f7da80f7870 con 0x55b2c9587bc0 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 -- 172.21.15.29:0/596453342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da80ed1c0 con 0x55b2c9587bc0 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7d957fa640 1 -- 172.21.15.29:0/596453342 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f7da80f7a70 con 0x55b2c9587bc0 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 10 client.10266 fetch_fsmap learned FSMap version 102 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 -- 172.21.15.29:0/596453342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x55b2c935cea0 con 0x55b2c9587bc0 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7d957fa640 1 -- 172.21.15.29:0/596453342 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da80ed1c0 con 0x55b2c9587bc0 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7d957fa640 1 -- 172.21.15.29:0/596453342 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 102) ==== 3494+0+0 (secure 0 0 0) 0x7f7da80b4070 con 0x55b2c9587bc0 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 10 client.10266 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 10 client.10266 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 20 client.10266 populate_metadata read hostname 'smithi029' 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.29:6834/4183931647,v1:172.21.15.29:6835/4183931647] conn(0x55b2c95b65e0 0x55b2c95e2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 4 client.10266 mds_command: new command op to 10095 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:40:40.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7dbfa79000 1 -- 172.21.15.29:0/596453342 --> [v2:172.21.15.29:6834/4183931647,v1:172.21.15.29:6835/4183931647] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2c9370ab0 con 0x55b2c95b65e0 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.29:6834/4183931647,v1:172.21.15.29:6835/4183931647] conn(0x55b2c95b65e0 0x55b2c95e2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.29:6834/4183931647,v1:172.21.15.29:6835/4183931647] conn(0x55b2c95b65e0 0x55b2c95e2460 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a313657810a4160 server_cookie=671ca4f0f6b0b2ad in_seq=0 out_seq=0 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.281+0000 7f7d957fa640 10 client.10266 ms_handle_connect on v2:172.21.15.29:6834/4183931647 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7d957fa640 1 -- 172.21.15.29:0/596453342 <== mds.0 v2:172.21.15.29:6834/4183931647 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2c9370ab0 con 0x55b2c95b65e0 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7d957fa640 10 client.10266 handle_command_reply: tid=0 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7dbfa79000 1 client.10266 shutdown 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7dbfa79000 1 -- 172.21.15.29:0/596453342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 msgr2=0x55b2c95b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c95b52c0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f7da80ef060 tx=0x7f7da8007cb0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7dbfa79000 1 -- 172.21.15.29:0/596453342 shutdown_connections 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.29:6834/4183931647,v1:172.21.15.29:6835/4183931647] conn(0x55b2c95b65e0 0x55b2c95e2460 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c95d1810 0x55b2c95b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7d957fa640 0 client.10266 ms_handle_reset on v2:172.21.15.29:6834/4183931647 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95d0ef0 0x55b2c95b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/596453342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c95b52c0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7dbfa79000 1 -- 172.21.15.29:0/596453342 >> 172.21.15.29:0/596453342 conn(0x55b2c9583b70 msgr2=0x55b2c95867f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:40.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7dbfa79000 1 -- 172.21.15.29:0/596453342 shutdown_connections 2024-04-25T00:40:40.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7dbfa79000 1 -- 172.21.15.29:0/596453342 wait complete. 2024-04-25T00:40:40.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.285+0000 7f7dbfa79000 20 client.10266 trim_cache size 0 max 16384 2024-04-25T00:40:40.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.301+0000 7f7dbfa79000 1 Processor -- start 2024-04-25T00:40:40.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.301+0000 7f7dbfa79000 1 -- start start 2024-04-25T00:40:40.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.301+0000 7f7dbfa79000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95b5d00 0x55b2c9586f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.301+0000 7f7dbfa79000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c9583b70 0x55b2c95874b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.301+0000 7f7dbfa79000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c95e2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.301+0000 7f7dbfa79000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7db8202b00 con 0x55b2c9587bc0 2024-04-25T00:40:40.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.301+0000 7f7dbfa79000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81f7cd0 con 0x55b2c9583b70 2024-04-25T00:40:40.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.301+0000 7f7dbfa79000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7db81f8320 con 0x55b2c95b5d00 2024-04-25T00:40:40.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.301+0000 7f7db6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c95e2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.301+0000 7f7db6d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c95e2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60956/0 (socket says 172.21.15.29:60956) 2024-04-25T00:40:40.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.301+0000 7f7db6d9d640 1 -- 172.21.15.29:0/2410727381 learned_addr learned my addr 172.21.15.29:0/2410727381 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:40.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/2410727381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c9583b70 0x55b2c95874b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db659c640 1 --2- 172.21.15.29:0/2410727381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95b5d00 0x55b2c9586f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db6d9d640 1 -- 172.21.15.29:0/2410727381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95b5d00 msgr2=0x55b2c9586f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db6d9d640 1 --2- 172.21.15.29:0/2410727381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95b5d00 0x55b2c9586f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db6d9d640 1 -- 172.21.15.29:0/2410727381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c9583b70 msgr2=0x55b2c95874b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db6d9d640 1 --2- 172.21.15.29:0/2410727381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c9583b70 0x55b2c95874b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db6d9d640 1 -- 172.21.15.29:0/2410727381 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c935de80 con 0x55b2c9587bc0 2024-04-25T00:40:40.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/2410727381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c9583b70 0x55b2c95874b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:40.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db6d9d640 1 --2- 172.21.15.29:0/2410727381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c95e2460 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f7db80b4d50 tx=0x7f7db0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e18a2d3ce73fafe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:40.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7d977fe640 1 -- 172.21.15.29:0/2410727381 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db00045b0 con 0x55b2c9587bc0 2024-04-25T00:40:40.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7d977fe640 1 -- 172.21.15.29:0/2410727381 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db001c070 con 0x55b2c9587bc0 2024-04-25T00:40:40.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7d977fe640 1 -- 172.21.15.29:0/2410727381 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0016400 con 0x55b2c9587bc0 2024-04-25T00:40:40.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 -- 172.21.15.29:0/2410727381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 msgr2=0x55b2c95e2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/2410727381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c95e2460 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f7db80b4d50 tx=0x7f7db0009750 comp rx=0 tx=0).stop 2024-04-25T00:40:40.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 -- 172.21.15.29:0/2410727381 shutdown_connections 2024-04-25T00:40:40.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/2410727381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c95e2460 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/2410727381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c9583b70 0x55b2c95874b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/2410727381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95b5d00 0x55b2c9586f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 -- 172.21.15.29:0/2410727381 >> 172.21.15.29:0/2410727381 conn(0x55b2c95b44f0 msgr2=0x55b2c958cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:40.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 -- 172.21.15.29:0/2410727381 shutdown_connections 2024-04-25T00:40:40.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 -- 172.21.15.29:0/2410727381 wait complete. 2024-04-25T00:40:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 -- start start 2024-04-25T00:40:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 Processor -- start 2024-04-25T00:40:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c9583b70 0x55b2c96413c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c9641900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95b5d00 0x55b2c963d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b2c93d15a0 con 0x55b2c9587bc0 2024-04-25T00:40:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b2c93d48f0 con 0x55b2c9583b70 2024-04-25T00:40:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55b2c93a2340 con 0x55b2c95b5d00 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 10 client.0 _ll_register_callbacks cb 0x7f7da8001520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c9641900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db5d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c9641900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60962/0 (socket says 172.21.15.29:60962) 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db5d9b640 1 -- 172.21.15.29:0/286120162 learned_addr learned my addr 172.21.15.29:0/286120162 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db659c640 1 --2- 172.21.15.29:0/286120162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c9583b70 0x55b2c96413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db6d9d640 1 --2- 172.21.15.29:0/286120162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95b5d00 0x55b2c963d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db5d9b640 1 -- 172.21.15.29:0/286120162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95b5d00 msgr2=0x55b2c963d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/286120162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95b5d00 0x55b2c963d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db5d9b640 1 -- 172.21.15.29:0/286120162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c9583b70 msgr2=0x55b2c96413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/286120162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c9583b70 0x55b2c96413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db5d9b640 1 -- 172.21.15.29:0/286120162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c93a2760 con 0x55b2c9587bc0 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7db5d9b640 1 --2- 172.21.15.29:0/286120162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c9641900 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f7db80aeb50 tx=0x7f7da0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1798695c4ff92418 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7d957fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:40:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7d957fa640 1 -- 172.21.15.29:0/286120162 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da00273b0 con 0x55b2c9587bc0 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7d957fa640 1 -- 172.21.15.29:0/286120162 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f7da0027550 con 0x55b2c9587bc0 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 -- 172.21.15.29:0/286120162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7db001c070 con 0x55b2c9587bc0 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7d957fa640 1 -- 172.21.15.29:0/286120162 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f7da0027750 con 0x55b2c9587bc0 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 10 client.10272 fetch_fsmap learned FSMap version 102 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7dbfa79000 1 -- 172.21.15.29:0/286120162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x55b2c942e9e0 con 0x55b2c9587bc0 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.305+0000 7f7d957fa640 1 -- 172.21.15.29:0/286120162 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db001c070 con 0x55b2c9587bc0 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7d957fa640 1 -- 172.21.15.29:0/286120162 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 102) ==== 3494+0+0 (secure 0 0 0) 0x7f7da0025410 con 0x55b2c9587bc0 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 10 client.10272 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 10 client.10272 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 20 client.10272 populate_metadata read hostname 'smithi029' 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/286120162 >> [v2:172.21.15.29:6834/4183931647,v1:172.21.15.29:6835/4183931647] conn(0x55b2c95d0ef0 0x55b2c963e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 4 client.10272 mds_command: new command op to 10095 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 1 -- 172.21.15.29:0/286120162 --> [v2:172.21.15.29:6834/4183931647,v1:172.21.15.29:6835/4183931647] -- command(tid 0: {"prefix": "status"}) -- 0x55b2c93e2b20 con 0x55b2c95d0ef0 2024-04-25T00:40:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7db659c640 1 --2- 172.21.15.29:0/286120162 >> [v2:172.21.15.29:6834/4183931647,v1:172.21.15.29:6835/4183931647] conn(0x55b2c95d0ef0 0x55b2c963e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7db659c640 1 --2- 172.21.15.29:0/286120162 >> [v2:172.21.15.29:6834/4183931647,v1:172.21.15.29:6835/4183931647] conn(0x55b2c95d0ef0 0x55b2c963e660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=514504384f1d7bcb server_cookie=1645905f08fcb0f5 in_seq=0 out_seq=0 2024-04-25T00:40:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7d957fa640 10 client.10272 ms_handle_connect on v2:172.21.15.29:6834/4183931647 2024-04-25T00:40:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7d957fa640 1 -- 172.21.15.29:0/286120162 <== mds.0 v2:172.21.15.29:6834/4183931647 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x55b2c93e2b20 con 0x55b2c95d0ef0 2024-04-25T00:40:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7d957fa640 10 client.10272 handle_command_reply: tid=0 2024-04-25T00:40:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 1 client.10272 shutdown 2024-04-25T00:40:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 1 -- 172.21.15.29:0/286120162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 msgr2=0x55b2c9641900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/286120162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c9641900 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f7db80aeb50 tx=0x7f7da0013000 comp rx=0 tx=0).stop 2024-04-25T00:40:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 1 -- 172.21.15.29:0/286120162 shutdown_connections 2024-04-25T00:40:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/286120162 >> [v2:172.21.15.29:6834/4183931647,v1:172.21.15.29:6835/4183931647] conn(0x55b2c95d0ef0 0x55b2c963e660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/286120162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b2c95b5d00 0x55b2c963d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/286120162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2c9587bc0 0x55b2c9641900 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7d957fa640 0 client.10272 ms_handle_reset on v2:172.21.15.29:6834/4183931647 2024-04-25T00:40:40.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 1 --2- 172.21.15.29:0/286120162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c9583b70 0x55b2c96413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 1 -- 172.21.15.29:0/286120162 >> 172.21.15.29:0/286120162 conn(0x55b2c95b44f0 msgr2=0x55b2c9585f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 1 -- 172.21.15.29:0/286120162 shutdown_connections 2024-04-25T00:40:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 1 -- 172.21.15.29:0/286120162 wait complete. 2024-04-25T00:40:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbfa79000 20 client.10272 trim_cache size 0 max 16384 2024-04-25T00:40:40.315 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:40:40.315 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "59afbe7d-aaa8-45e2-b468-69ee150cacea", 2024-04-25T00:40:40.315 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-04-25T00:40:40.315 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 10095, 2024-04-25T00:40:40.315 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-04-25T00:40:40.315 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-04-25T00:40:40.315 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-04-25T00:40:40.316 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 10.215159027, 2024-04-25T00:40:40.316 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 102, 2024-04-25T00:40:40.316 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 156, 2024-04-25T00:40:40.316 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 156, 2024-04-25T00:40:40.316 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 15.240637228000001 2024-04-25T00:40:40.316 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:40:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbc900640 1 -- 172.21.15.29:0/3660966350 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d980680d0 msgr2=0x7f7d9806a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbc900640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d980680d0 0x7f7d9806a590 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7da0001b30 tx=0x7f7da000a040 comp rx=0 tx=0).stop 2024-04-25T00:40:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbc900640 1 -- 172.21.15.29:0/3660966350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40039c0 msgr2=0x7f7db8157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbc900640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40039c0 0x7f7db8157aa0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f7da8000c10 tx=0x7f7da4005840 comp rx=0 tx=0).stop 2024-04-25T00:40:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbc900640 1 -- 172.21.15.29:0/3660966350 shutdown_connections 2024-04-25T00:40:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbc900640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d980680d0 0x7f7d9806a590 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbc900640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7db80cfb30 0x7f7db80cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbc900640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db8155060 0x7f7db81f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbc900640 1 --2- 172.21.15.29:0/3660966350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40039c0 0x7f7db8157aa0 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbc900640 1 -- 172.21.15.29:0/3660966350 >> 172.21.15.29:0/3660966350 conn(0x7f7db80b1870 msgr2=0x7f7db815cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:40.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbc900640 1 -- 172.21.15.29:0/3660966350 shutdown_connections 2024-04-25T00:40:40.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.309+0000 7f7dbc900640 1 -- 172.21.15.29:0/3660966350 wait complete. 2024-04-25T00:40:40.325 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:40:40.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.441+0000 7ffa98f4c640 1 Processor -- start 2024-04-25T00:40:40.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.441+0000 7ffa98f4c640 1 -- start start 2024-04-25T00:40:40.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa98f4c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa940d0c70 0x7ffa940d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa98f4c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa940b10e0 con 0x7ffa940c4d30 2024-04-25T00:40:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa98f4c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa940b3640 con 0x7ffa940aeab0 2024-04-25T00:40:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa98f4c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ffa94162c40 con 0x7ffa940d0c70 2024-04-25T00:40:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9259c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ffa940c4d30 0x7ffa940c5110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54432/0 (socket says 172.21.15.29:54432) 2024-04-25T00:40:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9259c640 1 -- 172.21.15.29:0/1524380121 learned_addr learned my addr 172.21.15.29:0/1524380121 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa92d9d640 1 --2- 172.21.15.29:0/1524380121 >> v2:172.21.15.157:3300/0 conn(0x7ffa940d0c70 0x7ffa940d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1150140047 0 0) 0x7ffa940b10e0 con 0x7ffa940c4d30 2024-04-25T00:40:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa80003530 con 0x7ffa940c4d30 2024-04-25T00:40:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 333009161 0 0) 0x7ffa80003530 con 0x7ffa940c4d30 2024-04-25T00:40:40.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 >> v2:172.21.15.157:3300/0 conn(0x7ffa940d0c70 msgr2=0x7ffa940d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 --2- 172.21.15.29:0/1524380121 >> v2:172.21.15.157:3300/0 conn(0x7ffa940d0c70 0x7ffa940d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 >> v1:172.21.15.157:6789/0 conn(0x7ffa940aeab0 legacy=0x7ffa9406c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa940b3080 con 0x7ffa940c4d30 2024-04-25T00:40:40.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ffa8c001a00 con 0x7ffa940c4d30 2024-04-25T00:40:40.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 >> v1:172.21.15.29:6789/0 conn(0x7ffa940c4d30 legacy=0x7ffa940c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 --2- 172.21.15.29:0/1524380121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa800039c0 0x7ffa80003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 --2- 172.21.15.29:0/1524380121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa80004810 0x7ffa80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 --2- 172.21.15.29:0/1524380121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa800052f0 0x7ffa8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffa94162c40 con 0x7ffa80004810 2024-04-25T00:40:40.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8000c1c0 con 0x7ffa800039c0 2024-04-25T00:40:40.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8000c3c0 con 0x7ffa800052f0 2024-04-25T00:40:40.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa91d9b640 1 --2- 172.21.15.29:0/1524380121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa80004810 0x7ffa80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9259c640 1 --2- 172.21.15.29:0/1524380121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa800039c0 0x7ffa80003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa91d9b640 1 -- 172.21.15.29:0/1524380121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa800052f0 msgr2=0x7ffa8000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa91d9b640 1 --2- 172.21.15.29:0/1524380121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa800052f0 0x7ffa8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa91d9b640 1 -- 172.21.15.29:0/1524380121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa800039c0 msgr2=0x7ffa80003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:40.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa91d9b640 1 --2- 172.21.15.29:0/1524380121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa800039c0 0x7ffa80003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa91d9b640 1 -- 172.21.15.29:0/1524380121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa80003530 con 0x7ffa80004810 2024-04-25T00:40:40.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa92d9d640 1 -- 172.21.15.29:0/1524380121 reap_dead start 2024-04-25T00:40:40.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa91d9b640 1 --2- 172.21.15.29:0/1524380121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa80004810 0x7ffa80004c90 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c001750 tx=0x7ffa7c01c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c925988e620c4a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:40.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0013a0 con 0x7ffa80004810 2024-04-25T00:40:40.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c0050d0 con 0x7ffa80004810 2024-04-25T00:40:40.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.445+0000 7ffa9159a640 1 -- 172.21.15.29:0/1524380121 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0054c0 con 0x7ffa80004810 2024-04-25T00:40:40.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 -- 172.21.15.29:0/1524380121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa80004810 msgr2=0x7ffa80004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 --2- 172.21.15.29:0/1524380121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa80004810 0x7ffa80004c90 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c001750 tx=0x7ffa7c01c1c0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 -- 172.21.15.29:0/1524380121 shutdown_connections 2024-04-25T00:40:40.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 --2- 172.21.15.29:0/1524380121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa80004810 0x7ffa80004c90 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 -- 172.21.15.29:0/1524380121 >> 172.21.15.29:0/1524380121 conn(0x7ffa940d1d60 msgr2=0x7ffa940d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:40.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 -- 172.21.15.29:0/1524380121 shutdown_connections 2024-04-25T00:40:40.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 -- 172.21.15.29:0/1524380121 wait complete. 2024-04-25T00:40:40.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 Processor -- start 2024-04-25T00:40:40.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 -- start start 2024-04-25T00:40:40.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa80004810 0x7ffa9415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa941637d0 0x7ffa9406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa94163c50 0x7ffa9406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffa940c5520 con 0x7ffa941637d0 2024-04-25T00:40:40.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa940b2570 con 0x7ffa94163c50 2024-04-25T00:40:40.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffa940c56a0 con 0x7ffa80004810 2024-04-25T00:40:40.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa91d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa941637d0 0x7ffa9406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa91d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa941637d0 0x7ffa9406eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60974/0 (socket says 172.21.15.29:60974) 2024-04-25T00:40:40.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa91d9b640 1 -- 172.21.15.29:0/304575265 learned_addr learned my addr 172.21.15.29:0/304575265 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:40.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa92d9d640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa94163c50 0x7ffa9406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa9259c640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa80004810 0x7ffa9415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa91d9b640 1 -- 172.21.15.29:0/304575265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa80004810 msgr2=0x7ffa9415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa91d9b640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa80004810 0x7ffa9415dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa91d9b640 1 -- 172.21.15.29:0/304575265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa94163c50 msgr2=0x7ffa9406f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa91d9b640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa94163c50 0x7ffa9406f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa91d9b640 1 -- 172.21.15.29:0/304575265 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa940bc0f0 con 0x7ffa941637d0 2024-04-25T00:40:40.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa92d9d640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa94163c50 0x7ffa9406f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:40.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa91d9b640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa941637d0 0x7ffa9406eea0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c005040 tx=0x7ffa80005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:40.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa8affd640 1 -- 172.21.15.29:0/304575265 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0053c0 con 0x7ffa941637d0 2024-04-25T00:40:40.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa8affd640 1 -- 172.21.15.29:0/304575265 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c0b9070 con 0x7ffa941637d0 2024-04-25T00:40:40.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 -- 172.21.15.29:0/304575265 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa94164220 con 0x7ffa941637d0 2024-04-25T00:40:40.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa8affd640 1 -- 172.21.15.29:0/304575265 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c007280 con 0x7ffa941637d0 2024-04-25T00:40:40.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.449+0000 7ffa98f4c640 1 -- 172.21.15.29:0/304575265 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa941647e0 con 0x7ffa941637d0 2024-04-25T00:40:40.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.453+0000 7ffa98f4c640 1 -- 172.21.15.29:0/304575265 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa94069270 con 0x7ffa941637d0 2024-04-25T00:40:40.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.453+0000 7ffa8affd640 1 -- 172.21.15.29:0/304575265 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7ffa7c007420 con 0x7ffa941637d0 2024-04-25T00:40:40.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.453+0000 7ffa8affd640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffa740680a0 0x7ffa7406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:40.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.453+0000 7ffa8affd640 1 -- 172.21.15.29:0/304575265 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7ffa7c120240 con 0x7ffa941637d0 2024-04-25T00:40:40.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.453+0000 7ffa9259c640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffa740680a0 0x7ffa7406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:40.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.453+0000 7ffa9259c640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffa740680a0 0x7ffa7406a560 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c005d20 tx=0x7ffa8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:40.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.457+0000 7ffa8affd640 1 -- 172.21.15.29:0/304575265 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffa940b3640 con 0x7ffa941637d0 2024-04-25T00:40:40.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa98f4c640 1 -- 172.21.15.29:0/304575265 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffa940bc300 con 0x7ffa941637d0 2024-04-25T00:40:40.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa8affd640 1 -- 172.21.15.29:0/304575265 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v102) ==== 53+0+73 (secure 0 0 0) 0x7ffa7c0f0170 con 0x7ffa941637d0 2024-04-25T00:40:40.653 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:40:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa98f4c640 1 -- 172.21.15.29:0/304575265 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffa740680a0 msgr2=0x7ffa7406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa98f4c640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffa740680a0 0x7ffa7406a560 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c005d20 tx=0x7ffa8c009040 comp rx=0 tx=0).stop 2024-04-25T00:40:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa98f4c640 1 -- 172.21.15.29:0/304575265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa941637d0 msgr2=0x7ffa9406eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa98f4c640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa941637d0 0x7ffa9406eea0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c005040 tx=0x7ffa80005840 comp rx=0 tx=0).stop 2024-04-25T00:40:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa98f4c640 1 -- 172.21.15.29:0/304575265 shutdown_connections 2024-04-25T00:40:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa98f4c640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffa740680a0 0x7ffa7406a560 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa98f4c640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa94163c50 0x7ffa9406f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa98f4c640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa941637d0 0x7ffa9406eea0 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa98f4c640 1 --2- 172.21.15.29:0/304575265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa80004810 0x7ffa9415dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:40.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa98f4c640 1 -- 172.21.15.29:0/304575265 >> 172.21.15.29:0/304575265 conn(0x7ffa940d1d60 msgr2=0x7ffa940b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa98f4c640 1 -- 172.21.15.29:0/304575265 shutdown_connections 2024-04-25T00:40:40.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40.653+0000 7ffa98f4c640 1 -- 172.21.15.29:0/304575265 wait complete. 2024-04-25T00:40:40.671 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:40:40.671 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:40:40.671 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-25T00:40:40.671 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:40:40.671 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:40:40.671 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:40:40.671 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:40:40.671 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:40:40.671 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:40:40.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:40.672 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-25T00:40:40.678 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:40:40.678 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:40:40.678 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:40:40.678 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:40.678 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:40:40.678 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:40.678 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:40:40.678 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:40:40.679 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-25T00:40:40.679 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:40:40.679 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:40:40.679 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-25T00:40:40.679 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:40:40.679 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:40:40.679 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-25T00:40:40.679 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:40:40.679 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 41919sec preferred_lft 41919sec 2024-04-25T00:40:40.679 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-04-25T00:40:40.679 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:40.679 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:40:40.679 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 02:42:28:a5:36:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:40:40.680 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:40.681 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:40:40.681 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:40:40.681 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:40:40.681 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:40.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:40.682 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:40:40.728 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-25T00:40:40.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:40.729 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-25T00:40:40.777 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-25T00:40:40.777 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:40:40.778 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T00:40:40.778 INFO:teuthology.orchestra.run.smithi029.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:40:40.778 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T00:40:40.778 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:40:40.778 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:40:40.778 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:40:40.779 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:40:40.779 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:40:40.779 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:40:40.779 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:40:40.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:40:40.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:40:40.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:40.902 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:40:40.908 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:40:40.908 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:40:40.908 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:40:40.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:40.909 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-04-25T00:40:40.964 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:40:40.965 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:40:40.965 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:40:41.012 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:40:41.013 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:40:41.013 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:40:41.060 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:40:41.118 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:40:41.166 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:40:41.167 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:40:41.167 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:40:41.167 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:40:41.167 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:40:41.167 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:40:41.167 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:40:41.167 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:40:41.167 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:40:41.167 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:40:41.167 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:40:41.167 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:40:41.168 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:40:41.169 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:40:41.169 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:40:41.169 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:40:41.169 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:40:41.169 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:40:41.169 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:40:41.169 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:40:41.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:41.170 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:40:41.216 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:40:41.217 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:40:41.264 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:40:41.278 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:40:41.324 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:40:41.324 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:40:41.324 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:40:41.324 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:40:41.324 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:40:41.324 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:40:41.325 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:40:41.325 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:40:41.325 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:40:41.325 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:40:41.325 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:40:41.325 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:40:41.325 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:40:41.325 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:40:41.325 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:40:41.325 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:40:41.325 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:40:41.325 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:40:41.325 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:40:41.326 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:40:41.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:41.327 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:40:41.343 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:2024-04-25T00:40:41.341+0000 7fe0a0217580 -1 init, newargv = 0x562d0f332570 newargc=15 2024-04-25T00:40:41.343 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[56788]: starting ceph client 2024-04-25T00:40:41.349 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:40:41.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[56788]: starting fuse 2024-04-25T00:40:41.351 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:40:42.449 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:40:42.465 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:40:42.514 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:40:42.515 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:40:42.515 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:40:42.515 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:40:42.515 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:40:42.515 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:40:42.515 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:40:42.515 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:40:42.515 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:40:42.515 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:40:42.515 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:40:42.515 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:40:42.515 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:40:42.515 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:40:42.516 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:40:42.517 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:40:42.517 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:40:42.517 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:40:42.517 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:40:42.517 INFO:teuthology.orchestra.run.smithi029.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:40:42.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:42.518 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:40:42.565 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:40:42.566 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:40:42.566 DEBUG:teuthology.orchestra.run.smithi029:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:40:42.566 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-04-25T00:40:42.566 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-04-25T00:40:42.566 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-04-25T00:40:42.566 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-04-25T00:40:42.566 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:40:42.566 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-04-25T00:40:42.566 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:40:42.566 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-04-25T00:40:42.566 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:40:42.566 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.0")) 2024-04-25T00:40:42.567 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-04-25T00:40:42.693 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.0.56788.asok 2024-04-25T00:40:42.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:40:42.705 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56788.asok 2024-04-25T00:40:42.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:42.706 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56788.asok status 2024-04-25T00:40:42.860 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:40:42.860 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-04-25T00:40:42.860 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:40:42.860 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:40:42.860 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "0", 2024-04-25T00:40:42.860 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-04-25T00:40:42.860 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:40:42.860 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "56788", 2024-04-25T00:40:42.860 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-04-25T00:40:42.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:42.861 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-04-25T00:40:42.861 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-04-25T00:40:42.861 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 10287, 2024-04-25T00:40:42.861 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-04-25T00:40:42.861 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-04-25T00:40:42.861 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-04-25T00:40:42.861 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 10287 2024-04-25T00:40:42.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:42.861 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:40:42.861 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:40:42.861 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:40:42.861 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2428372806 2024-04-25T00:40:42.861 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 2428372806 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.10287 192.168.0.1:0/2428372806", 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.1:0/2428372806", 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 102, 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 156, 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-04-25T00:40:42.862 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-04-25T00:40:42.863 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:40:42.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:42.874 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-04-25T00:40:42.874 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:40:42.881 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-04-25T00:40:42.881 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:40:42.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:42.882 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:40:42.937 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:40:43.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.105+0000 7f8272de8640 1 Processor -- start 2024-04-25T00:40:43.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.105+0000 7f8272de8640 1 -- start start 2024-04-25T00:40:43.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f8272de8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c15e8d0 0x7f826c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:43.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f8272de8640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f826c0b3720 con 0x7f826c15f3c0 2024-04-25T00:40:43.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f8272de8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f826c0b39c0 con 0x7f826c0b4780 2024-04-25T00:40:43.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f8272de8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f826c0b42f0 con 0x7f826c15e8d0 2024-04-25T00:40:43.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f8271385640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f826c15f3c0 0x7f826c15c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54448/0 (socket says 172.21.15.29:54448) 2024-04-25T00:40:43.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f8271385640 1 -- 172.21.15.29:0/520767441 learned_addr learned my addr 172.21.15.29:0/520767441 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:43.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f8270b84640 1 --2- 172.21.15.29:0/520767441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c15e8d0 0x7f826c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1598079156 0 0) 0x7f826c0b3720 con 0x7f826c15f3c0 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8258003430 con 0x7f826c15f3c0 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 140289605 0 0) 0x7f8258003430 con 0x7f826c15f3c0 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c15e8d0 msgr2=0x7f826c0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 --2- 172.21.15.29:0/520767441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c15e8d0 0x7f826c0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 >> v1:172.21.15.157:6789/0 conn(0x7f826c0b4780 legacy=0x7f826c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c0b3ca0 con 0x7f826c15f3c0 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8264001a00 con 0x7f826c15f3c0 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 >> v1:172.21.15.29:6789/0 conn(0x7f826c15f3c0 legacy=0x7f826c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 --2- 172.21.15.29:0/520767441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8258004ad0 0x7f8258004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 --2- 172.21.15.29:0/520767441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82580036b0 0x7f8258003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 --2- 172.21.15.29:0/520767441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c15e8d0 0x7f825800fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f826c0b42f0 con 0x7f8258004ad0 2024-04-25T00:40:43.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f82580044b0 con 0x7f82580036b0 2024-04-25T00:40:43.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f82580100c0 con 0x7f826c15e8d0 2024-04-25T00:40:43.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f8270b84640 1 --2- 172.21.15.29:0/520767441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8258004ad0 0x7f8258004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:43.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826bfff640 1 --2- 172.21.15.29:0/520767441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82580036b0 0x7f8258003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:43.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f8270b84640 1 -- 172.21.15.29:0/520767441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c15e8d0 msgr2=0x7f825800fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:43.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f8270b84640 1 --2- 172.21.15.29:0/520767441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c15e8d0 0x7f825800fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:43.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f8270b84640 1 -- 172.21.15.29:0/520767441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82580036b0 msgr2=0x7f8258003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:43.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f8270b84640 1 --2- 172.21.15.29:0/520767441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82580036b0 0x7f8258003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:43.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f8270b84640 1 -- 172.21.15.29:0/520767441 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8258003430 con 0x7f8258004ad0 2024-04-25T00:40:43.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.109+0000 7f826bfff640 1 --2- 172.21.15.29:0/520767441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82580036b0 0x7f8258003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:43.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8270b84640 1 --2- 172.21.15.29:0/520767441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8258004ad0 0x7f8258004ef0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f825c009ae0 tx=0x7f825c01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b146196c1d5deac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:43.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0b2070 con 0x7f8258004ad0 2024-04-25T00:40:43.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825c0a5560 con 0x7f8258004ad0 2024-04-25T00:40:43.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f826b7fe640 1 -- 172.21.15.29:0/520767441 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0096b0 con 0x7f8258004ad0 2024-04-25T00:40:43.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 -- 172.21.15.29:0/520767441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8258004ad0 msgr2=0x7f8258004ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:43.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 --2- 172.21.15.29:0/520767441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8258004ad0 0x7f8258004ef0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f825c009ae0 tx=0x7f825c01ca20 comp rx=0 tx=0).stop 2024-04-25T00:40:43.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8271385640 1 -- 172.21.15.29:0/520767441 reap_dead start 2024-04-25T00:40:43.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 -- 172.21.15.29:0/520767441 shutdown_connections 2024-04-25T00:40:43.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 -- 172.21.15.29:0/520767441 >> 172.21.15.29:0/520767441 conn(0x7f826c0b1870 msgr2=0x7f826c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:43.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 -- 172.21.15.29:0/520767441 shutdown_connections 2024-04-25T00:40:43.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 -- 172.21.15.29:0/520767441 wait complete. 2024-04-25T00:40:43.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 Processor -- start 2024-04-25T00:40:43.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 -- start start 2024-04-25T00:40:43.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258004ad0 0x7f826c070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:43.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c071220 0x7f826c1f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:43.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c1f6ee0 0x7f826c1f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:43.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f826c06d710 con 0x7f826c071220 2024-04-25T00:40:43.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f826c06d590 con 0x7f8258004ad0 2024-04-25T00:40:43.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f8272de8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f826c06d890 con 0x7f826c1f6ee0 2024-04-25T00:40:43.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f826bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c071220 0x7f826c1f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:43.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f826bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c071220 0x7f826c1f69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:32770/0 (socket says 172.21.15.29:32770) 2024-04-25T00:40:43.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.113+0000 7f826bfff640 1 -- 172.21.15.29:0/2721852524 learned_addr learned my addr 172.21.15.29:0/2721852524 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:43.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f8270b84640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258004ad0 0x7f826c070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:43.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f8271385640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c1f6ee0 0x7f826c1f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:43.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f826bfff640 1 -- 172.21.15.29:0/2721852524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c1f6ee0 msgr2=0x7f826c1f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f826bfff640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c1f6ee0 0x7f826c1f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f826bfff640 1 -- 172.21.15.29:0/2721852524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258004ad0 msgr2=0x7f826c070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f826bfff640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258004ad0 0x7f826c070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f826bfff640 1 -- 172.21.15.29:0/2721852524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c0717a0 con 0x7f826c071220 2024-04-25T00:40:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f8271385640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c1f6ee0 0x7f826c1f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f826bfff640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c071220 0x7f826c1f69a0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f825400d770 tx=0x7f825400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f82697fa640 1 -- 172.21.15.29:0/2721852524 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254005270 con 0x7f826c071220 2024-04-25T00:40:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f82697fa640 1 -- 172.21.15.29:0/2721852524 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8254017070 con 0x7f826c071220 2024-04-25T00:40:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f8272de8640 1 -- 172.21.15.29:0/2721852524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826c071a80 con 0x7f826c071220 2024-04-25T00:40:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f82697fa640 1 -- 172.21.15.29:0/2721852524 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825400b6b0 con 0x7f826c071220 2024-04-25T00:40:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f8272de8640 1 -- 172.21.15.29:0/2721852524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826c072040 con 0x7f826c071220 2024-04-25T00:40:43.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f8272de8640 1 -- 172.21.15.29:0/2721852524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826c069270 con 0x7f826c071220 2024-04-25T00:40:43.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f82697fa640 1 -- 172.21.15.29:0/2721852524 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f825400b850 con 0x7f826c071220 2024-04-25T00:40:43.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f82697fa640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f824c0680d0 0x7f824c06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:43.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f82697fa640 1 -- 172.21.15.29:0/2721852524 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f8254086b90 con 0x7f826c071220 2024-04-25T00:40:43.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f8270b84640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f824c0680d0 0x7f824c06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:43.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.117+0000 7f8270b84640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f824c0680d0 0x7f824c06a590 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f825c0af000 tx=0x7f825c0046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:43.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.121+0000 7f82697fa640 1 -- 172.21.15.29:0/2721852524 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f82540102d0 con 0x7f826c071220 2024-04-25T00:40:43.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.321+0000 7f8272de8640 1 -- 172.21.15.29:0/2721852524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f826c06e7b0 con 0x7f826c071220 2024-04-25T00:40:43.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.321+0000 7f82697fa640 1 -- 172.21.15.29:0/2721852524 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v102) ==== 53+0+73 (secure 0 0 0) 0x7f82540565a0 con 0x7f826c071220 2024-04-25T00:40:43.323 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:40:43.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.325+0000 7f8272de8640 1 -- 172.21.15.29:0/2721852524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f824c0680d0 msgr2=0x7f824c06a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:43.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.325+0000 7f8272de8640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f824c0680d0 0x7f824c06a590 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f825c0af000 tx=0x7f825c0046d0 comp rx=0 tx=0).stop 2024-04-25T00:40:43.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.325+0000 7f8272de8640 1 -- 172.21.15.29:0/2721852524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c071220 msgr2=0x7f826c1f69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:43.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.325+0000 7f8272de8640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c071220 0x7f826c1f69a0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f825400d770 tx=0x7f825400dc30 comp rx=0 tx=0).stop 2024-04-25T00:40:43.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.325+0000 7f8272de8640 1 -- 172.21.15.29:0/2721852524 shutdown_connections 2024-04-25T00:40:43.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.325+0000 7f8272de8640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f824c0680d0 0x7f824c06a590 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:43.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.325+0000 7f8272de8640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f826c1f6ee0 0x7f826c1f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:43.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.325+0000 7f8272de8640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c071220 0x7f826c1f69a0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:43.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.325+0000 7f8272de8640 1 --2- 172.21.15.29:0/2721852524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258004ad0 0x7f826c070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:43.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.325+0000 7f8272de8640 1 -- 172.21.15.29:0/2721852524 >> 172.21.15.29:0/2721852524 conn(0x7f826c0b1870 msgr2=0x7f826c0b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.325+0000 7f8272de8640 1 -- 172.21.15.29:0/2721852524 shutdown_connections 2024-04-25T00:40:43.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:43.325+0000 7f8272de8640 1 -- 172.21.15.29:0/2721852524 wait complete. 2024-04-25T00:40:43.342 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:40:43.342 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:40:43.343 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-25T00:40:43.343 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:40:43.343 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:40:43.343 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:40:43.343 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:40:43.343 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:40:43.343 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:40:43.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:43.343 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-25T00:40:43.350 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:40:43.350 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:40:43.351 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 41912sec preferred_lft 41912sec 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:9c:8d:4b:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:40:43.352 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:43.353 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:40:43.353 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:40:43.353 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:40:43.353 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:43.353 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:40:43.353 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:40:43.353 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:40:43.353 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:40:43.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:43.353 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:40:43.406 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:40:43.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:43.408 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-25T00:40:43.459 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-25T00:40:43.459 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:40:43.459 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:40:43.459 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:40:43.459 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:40:43.459 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:40:43.459 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:40:43.459 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:40:43.459 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:40:43.459 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:40:43.459 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:40:43.460 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:40:43.460 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:40:43.460 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:40:43.460 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:40:43.460 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:40:43.460 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:40:43.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:40:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:40:43.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:40:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:40:43.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:43.586 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:40:43.591 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:40:43.591 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:40:43.591 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:40:43.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:43.592 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-25T00:40:43.646 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T00:40:43.646 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:40:43.646 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:40:43.695 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:40:43.695 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:40:43.695 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:40:43.746 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:40:43.804 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:40:43.851 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:40:43.851 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:40:43.851 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:40:43.851 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:40:43.851 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:40:43.852 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:40:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:40:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:40:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:40:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:40:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:40:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:40:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:40:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:40:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:40:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:40:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:40:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:40:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:40:43.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:43.854 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:40:43.902 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:40:43.903 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T00:40:43.948 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:40:43.963 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:40:44.011 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:40:44.011 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:40:44.011 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:40:44.012 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:40:44.012 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:40:44.012 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:40:44.012 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:40:44.012 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:40:44.012 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:40:44.012 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:40:44.012 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:40:44.012 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:40:44.012 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:40:44.013 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:40:44.013 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:40:44.013 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:40:44.013 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:40:44.013 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:40:44.013 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:40:44.013 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:40:44.013 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:40:44.013 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:40:44.014 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:40:44.014 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:40:44.014 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:40:44.014 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:40:44.014 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:40:44.014 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:40:44.014 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:40:44.014 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:40:44.015 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:40:44.015 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:40:44.015 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:40:44.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:44.015 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:40:44.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-04-25T00:40:44.028+0000 7f545b021580 -1 init, newargv = 0x5625ca9efbf0 newargc=15 2024-04-25T00:40:44.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[35219]: starting ceph client 2024-04-25T00:40:44.034 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[35219]: starting fuse 2024-04-25T00:40:45.133 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:40:45.149 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:40:45.195 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:40:45.196 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:40:45.196 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:40:45.196 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:40:45.196 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:40:45.196 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:40:45.196 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:40:45.196 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:40:45.196 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:40:45.196 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:40:45.196 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:40:45.196 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:40:45.196 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:40:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:40:45.198 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:40:45.198 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:40:45.198 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:40:45.198 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:40:45.198 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:40:45.198 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:40:45.198 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:40:45.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:45.199 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:40:45.246 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:40:45.247 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:40:45.247 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:40:45.247 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-25T00:40:45.247 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-25T00:40:45.247 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-25T00:40:45.247 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-25T00:40:45.247 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:40:45.247 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-25T00:40:45.247 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-25T00:40:45.248 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-25T00:40:45.249 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:40:45.249 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:40:45.249 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-04-25T00:40:45.249 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-25T00:40:45.371 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.35219.asok 2024-04-25T00:40:45.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:40:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:40:45.383 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35219.asok 2024-04-25T00:40:45.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:45.383 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35219.asok status 2024-04-25T00:40:45.538 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:40:45.538 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-25T00:40:45.538 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:40:45.538 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:40:45.538 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-04-25T00:40:45.538 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-25T00:40:45.538 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T00:40:45.538 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "35219", 2024-04-25T00:40:45.538 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7295, 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 7295 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2121957492 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-25T00:40:45.539 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:40:45.540 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:40:45.540 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:40:45.540 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:40:45.540 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2121957492 2024-04-25T00:40:45.540 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:40:45.540 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.7295 192.168.0.1:0/2121957492", 2024-04-25T00:40:45.540 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/2121957492", 2024-04-25T00:40:45.540 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-25T00:40:45.540 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 102, 2024-04-25T00:40:45.540 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 156, 2024-04-25T00:40:45.540 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:40:45.540 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-25T00:40:45.540 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-25T00:40:45.541 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:40:45.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:45.552 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-25T00:40:45.552 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T00:40:45.558 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-25T00:40:45.558 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T00:40:45.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:45.558 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:40:45.615 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-25T00:40:45.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60d184640 1 Processor -- start 2024-04-25T00:40:45.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60d184640 1 -- start start 2024-04-25T00:40:45.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60d184640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60815ae20 0x7fa60806f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:45.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60d184640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa60806fa40 0x7fa608071e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:45.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60d184640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6080b4f90 con 0x7fa6080d0820 2024-04-25T00:40:45.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60d184640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa60815b200 con 0x7fa60806fa40 2024-04-25T00:40:45.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60d184640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa60806f070 con 0x7fa60815ae20 2024-04-25T00:40:45.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa60806fa40 0x7fa608071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:45.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa606d9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60815ae20 0x7fa60806f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:45.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa60806fa40 0x7fa608071e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:44804/0 (socket says 172.21.15.29:44804) 2024-04-25T00:40:45.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60759e640 1 -- 172.21.15.29:0/3052658647 learned_addr learned my addr 172.21.15.29:0/3052658647 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:45.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa6080d0820 0x7fa6080b4880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:53828/0 (socket says 172.21.15.29:53828) 2024-04-25T00:40:45.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60759e640 1 -- 172.21.15.29:0/3052658647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60815ae20 msgr2=0x7fa60806f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:45.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60759e640 1 --2- 172.21.15.29:0/3052658647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60815ae20 0x7fa60806f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:45.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60759e640 1 -- 172.21.15.29:0/3052658647 >> v1:172.21.15.157:6789/0 conn(0x7fa6080d0820 legacy=0x7fa6080b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:45.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.757+0000 7fa60759e640 1 -- 172.21.15.29:0/3052658647 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6080b5260 con 0x7fa60806fa40 2024-04-25T00:40:45.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.761+0000 7fa60759e640 1 --2- 172.21.15.29:0/3052658647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60806fa40 0x7fa608071e50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa600009870 tx=0x7fa60001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7353e5d3450a8a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:45.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.761+0000 7fa605d9b640 1 -- 172.21.15.29:0/3052658647 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000b1070 con 0x7fa60806fa40 2024-04-25T00:40:45.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.761+0000 7fa605d9b640 1 -- 172.21.15.29:0/3052658647 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6000ac410 con 0x7fa60806fa40 2024-04-25T00:40:45.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.761+0000 7fa605d9b640 1 -- 172.21.15.29:0/3052658647 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000ac7e0 con 0x7fa60806fa40 2024-04-25T00:40:45.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.761+0000 7fa60d184640 1 -- 172.21.15.29:0/3052658647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60806fa40 msgr2=0x7fa608071e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:45.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.761+0000 7fa60d184640 1 --2- 172.21.15.29:0/3052658647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60806fa40 0x7fa608071e50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa600009870 tx=0x7fa60001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:40:45.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.761+0000 7fa60d184640 1 -- 172.21.15.29:0/3052658647 shutdown_connections 2024-04-25T00:40:45.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.761+0000 7fa60d184640 1 --2- 172.21.15.29:0/3052658647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60806fa40 0x7fa608071e50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:45.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.761+0000 7fa60d184640 1 --2- 172.21.15.29:0/3052658647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60815ae20 0x7fa60806f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:45.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.761+0000 7fa60d184640 1 -- 172.21.15.29:0/3052658647 >> 172.21.15.29:0/3052658647 conn(0x7fa6080b2880 msgr2=0x7fa6080b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:45.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.761+0000 7fa60d184640 1 -- 172.21.15.29:0/3052658647 shutdown_connections 2024-04-25T00:40:45.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.761+0000 7fa60d184640 1 -- 172.21.15.29:0/3052658647 wait complete. 2024-04-25T00:40:45.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60d184640 1 Processor -- start 2024-04-25T00:40:45.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60d184640 1 -- start start 2024-04-25T00:40:45.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60d184640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa60815ae20 0x7fa608156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:45.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60d184640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60806b470 0x7fa6080cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:45.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60d184640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60806b8f0 0x7fa60813c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:45.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60d184640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa608167a90 con 0x7fa60815ae20 2024-04-25T00:40:45.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60d184640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6080ce9c0 con 0x7fa60806b470 2024-04-25T00:40:45.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60d184640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa608167c10 con 0x7fa60806b8f0 2024-04-25T00:40:45.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa606d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa60815ae20 0x7fa608156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:45.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa606d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa60815ae20 0x7fa608156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:32782/0 (socket says 172.21.15.29:32782) 2024-04-25T00:40:45.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa606d9d640 1 -- 172.21.15.29:0/967710551 learned_addr learned my addr 172.21.15.29:0/967710551 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:45.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60659c640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60806b470 0x7fa6080cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:45.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60759e640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60806b8f0 0x7fa60813c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:45.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa606d9d640 1 -- 172.21.15.29:0/967710551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60806b8f0 msgr2=0x7fa60813c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:45.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa606d9d640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60806b8f0 0x7fa60813c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:45.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa606d9d640 1 -- 172.21.15.29:0/967710551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60806b470 msgr2=0x7fa6080cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:45.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa606d9d640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60806b470 0x7fa6080cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:45.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa606d9d640 1 -- 172.21.15.29:0/967710551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6080cbe00 con 0x7fa60815ae20 2024-04-25T00:40:45.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60659c640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60806b470 0x7fa6080cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:45.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa606d9d640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa60815ae20 0x7fa608156950 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800efc0 tx=0x7fa5f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:45.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa5ff7fe640 1 -- 172.21.15.29:0/967710551 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8018020 con 0x7fa60815ae20 2024-04-25T00:40:45.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa5ff7fe640 1 -- 172.21.15.29:0/967710551 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f8014480 con 0x7fa60815ae20 2024-04-25T00:40:45.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60d184640 1 -- 172.21.15.29:0/967710551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa608155e70 con 0x7fa60815ae20 2024-04-25T00:40:45.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa5ff7fe640 1 -- 172.21.15.29:0/967710551 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8014850 con 0x7fa60815ae20 2024-04-25T00:40:45.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60d184640 1 -- 172.21.15.29:0/967710551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa608156430 con 0x7fa60815ae20 2024-04-25T00:40:45.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.765+0000 7fa60d184640 1 -- 172.21.15.29:0/967710551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6080692e0 con 0x7fa60815ae20 2024-04-25T00:40:45.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.769+0000 7fa5ff7fe640 1 -- 172.21.15.29:0/967710551 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fa5f801b020 con 0x7fa60815ae20 2024-04-25T00:40:45.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.769+0000 7fa5ff7fe640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5e80680a0 0x7fa5e806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:45.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.769+0000 7fa5ff7fe640 1 -- 172.21.15.29:0/967710551 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fa5f8087e80 con 0x7fa60815ae20 2024-04-25T00:40:45.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.769+0000 7fa60659c640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5e80680a0 0x7fa5e806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:45.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.769+0000 7fa60659c640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5e80680a0 0x7fa5e806a560 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa60815e380 tx=0x7fa5f0007d90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:45.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.773+0000 7fa5ff7fe640 1 -- 172.21.15.29:0/967710551 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa5f8022070 con 0x7fa60815ae20 2024-04-25T00:40:45.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:45.969+0000 7fa60d184640 1 -- 172.21.15.29:0/967710551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fa608157990 con 0x7fa60815ae20 2024-04-25T00:40:46.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.493+0000 7fa5ff7fe640 1 -- 172.21.15.29:0/967710551 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v103) ==== 136+0+0 (secure 0 0 0) 0x7fa5f8010040 con 0x7fa60815ae20 2024-04-25T00:40:46.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.497+0000 7fa60d184640 1 -- 172.21.15.29:0/967710551 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5e80680a0 msgr2=0x7fa5e806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:46.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.497+0000 7fa60d184640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5e80680a0 0x7fa5e806a560 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa60815e380 tx=0x7fa5f0007d90 comp rx=0 tx=0).stop 2024-04-25T00:40:46.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.497+0000 7fa60d184640 1 -- 172.21.15.29:0/967710551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa60815ae20 msgr2=0x7fa608156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:46.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.497+0000 7fa60d184640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa60815ae20 0x7fa608156950 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800efc0 tx=0x7fa5f800c490 comp rx=0 tx=0).stop 2024-04-25T00:40:46.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.497+0000 7fa60d184640 1 -- 172.21.15.29:0/967710551 shutdown_connections 2024-04-25T00:40:46.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.497+0000 7fa60d184640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa5e80680a0 0x7fa5e806a560 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.497+0000 7fa60d184640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60806b8f0 0x7fa60813c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.497+0000 7fa60d184640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60806b470 0x7fa6080cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.497+0000 7fa60d184640 1 --2- 172.21.15.29:0/967710551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa60815ae20 0x7fa608156950 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.497+0000 7fa60d184640 1 -- 172.21.15.29:0/967710551 >> 172.21.15.29:0/967710551 conn(0x7fa6080b2880 msgr2=0x7fa6080b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:46.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.497+0000 7fa60d184640 1 -- 172.21.15.29:0/967710551 shutdown_connections 2024-04-25T00:40:46.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.497+0000 7fa60d184640 1 -- 172.21.15.29:0/967710551 wait complete. 2024-04-25T00:40:46.519 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-25T00:40:46.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f353a863640 1 Processor -- start 2024-04-25T00:40:46.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f353a863640 1 -- start start 2024-04-25T00:40:46.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f353a863640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f353415c4a0 0x7f353415c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:46.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f353a863640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f353415cdc0 0x7f35340cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:46.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f353a863640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35340cecd0 con 0x7f353406cc60 2024-04-25T00:40:46.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f353a863640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f35340b25c0 con 0x7f353415cdc0 2024-04-25T00:40:46.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f353a863640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f35340cefb0 con 0x7f353415c4a0 2024-04-25T00:40:46.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f353a062640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f353406cc60 0x7f35340b3130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54458/0 (socket says 172.21.15.29:54458) 2024-04-25T00:40:46.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f353a062640 1 -- 172.21.15.29:0/3448684650 learned_addr learned my addr 172.21.15.29:0/3448684650 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:46.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f3539060640 1 --2- 172.21.15.29:0/3448684650 >> v2:172.21.15.29:3300/0 conn(0x7f353415cdc0 0x7f35340cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:46.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f3539861640 1 --2- 172.21.15.29:0/3448684650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f353415c4a0 0x7f353415c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:46.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f353885f640 1 -- 172.21.15.29:0/3448684650 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2651329037 0 0) 0x7f35340cecd0 con 0x7f353406cc60 2024-04-25T00:40:46.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f353885f640 1 -- 172.21.15.29:0/3448684650 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3520003530 con 0x7f353406cc60 2024-04-25T00:40:46.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f3539060640 1 -- 172.21.15.29:0/3448684650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f353415c4a0 msgr2=0x7f353415c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:46.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f3539060640 1 --2- 172.21.15.29:0/3448684650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f353415c4a0 0x7f353415c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f3539060640 1 -- 172.21.15.29:0/3448684650 >> v1:172.21.15.29:6789/0 conn(0x7f353406cc60 legacy=0x7f35340b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:46.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.649+0000 7f3539060640 1 -- 172.21.15.29:0/3448684650 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35340cf290 con 0x7f353415cdc0 2024-04-25T00:40:46.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f3539060640 1 --2- 172.21.15.29:0/3448684650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f353415cdc0 0x7f35340cdff0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f35240097c0 tx=0x7f352401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42de8c173630208 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:46.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f353885f640 1 -- 172.21.15.29:0/3448684650 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35240b0070 con 0x7f353415cdc0 2024-04-25T00:40:46.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f353885f640 1 -- 172.21.15.29:0/3448684650 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35240ab600 con 0x7f353415cdc0 2024-04-25T00:40:46.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f353885f640 1 -- 172.21.15.29:0/3448684650 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35240ab9d0 con 0x7f353415cdc0 2024-04-25T00:40:46.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f353a863640 1 -- 172.21.15.29:0/3448684650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f353415cdc0 msgr2=0x7f35340cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:46.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f353a863640 1 --2- 172.21.15.29:0/3448684650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f353415cdc0 0x7f35340cdff0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f35240097c0 tx=0x7f352401c410 comp rx=0 tx=0).stop 2024-04-25T00:40:46.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f353a863640 1 -- 172.21.15.29:0/3448684650 shutdown_connections 2024-04-25T00:40:46.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f353a863640 1 --2- 172.21.15.29:0/3448684650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f353415cdc0 0x7f35340cdff0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f353a863640 1 --2- 172.21.15.29:0/3448684650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f353415c4a0 0x7f353415c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f353a863640 1 -- 172.21.15.29:0/3448684650 >> 172.21.15.29:0/3448684650 conn(0x7f35340b1770 msgr2=0x7f35340b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:46.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f353a863640 1 -- 172.21.15.29:0/3448684650 shutdown_connections 2024-04-25T00:40:46.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f353a863640 1 -- 172.21.15.29:0/3448684650 wait complete. 2024-04-25T00:40:46.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f353a863640 1 Processor -- start 2024-04-25T00:40:46.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.653+0000 7f353a863640 1 -- start start 2024-04-25T00:40:46.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f353a863640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f353406cc60 0x7f3534159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:46.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f353a863640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f353415c4a0 0x7f3534159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:46.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f353a863640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f353415cdc0 0x7f353414e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:46.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f353a863640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3534160b60 con 0x7f353406cc60 2024-04-25T00:40:46.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f353a863640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35341609e0 con 0x7f353415cdc0 2024-04-25T00:40:46.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f353a863640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3534160ce0 con 0x7f353415c4a0 2024-04-25T00:40:46.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f3539861640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f353406cc60 0x7f3534159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:46.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f3539861640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f353406cc60 0x7f3534159130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:32802/0 (socket says 172.21.15.29:32802) 2024-04-25T00:40:46.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f3539861640 1 -- 172.21.15.29:0/211248338 learned_addr learned my addr 172.21.15.29:0/211248338 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:46.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f3539861640 1 -- 172.21.15.29:0/211248338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f353415c4a0 msgr2=0x7f3534159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:46.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f3539861640 1 --2- 172.21.15.29:0/211248338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f353415c4a0 0x7f3534159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f3539861640 1 -- 172.21.15.29:0/211248338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f353415cdc0 msgr2=0x7f353414e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:46.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f3539861640 1 --2- 172.21.15.29:0/211248338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f353415cdc0 0x7f353414e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f3539861640 1 -- 172.21.15.29:0/211248338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353414f4a0 con 0x7f353406cc60 2024-04-25T00:40:46.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f3539861640 1 --2- 172.21.15.29:0/211248338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f353406cc60 0x7f3534159130 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f353000d830 tx=0x7f353000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:46.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f352a7fc640 1 -- 172.21.15.29:0/211248338 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353000b6c0 con 0x7f353406cc60 2024-04-25T00:40:46.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f352a7fc640 1 -- 172.21.15.29:0/211248338 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3530019070 con 0x7f353406cc60 2024-04-25T00:40:46.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f353a863640 1 -- 172.21.15.29:0/211248338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3534154470 con 0x7f353406cc60 2024-04-25T00:40:46.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f352a7fc640 1 -- 172.21.15.29:0/211248338 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353000ba60 con 0x7f353406cc60 2024-04-25T00:40:46.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f353a863640 1 -- 172.21.15.29:0/211248338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f353413a2e0 con 0x7f353406cc60 2024-04-25T00:40:46.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f353a863640 1 -- 172.21.15.29:0/211248338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3534069200 con 0x7f353406cc60 2024-04-25T00:40:46.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f352a7fc640 1 -- 172.21.15.29:0/211248338 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f353001e460 con 0x7f353406cc60 2024-04-25T00:40:46.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f352a7fc640 1 --2- 172.21.15.29:0/211248338 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f35180680a0 0x7f351806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:46.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f3539060640 1 --2- 172.21.15.29:0/211248338 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f35180680a0 0x7f351806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:46.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.657+0000 7f352a7fc640 1 -- 172.21.15.29:0/211248338 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f3530084d60 con 0x7f353406cc60 2024-04-25T00:40:46.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.661+0000 7f3539060640 1 --2- 172.21.15.29:0/211248338 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f35180680a0 0x7f351806a560 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f35240ad000 tx=0x7f3524004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:46.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.665+0000 7f352a7fc640 1 -- 172.21.15.29:0/211248338 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3530057190 con 0x7f353406cc60 2024-04-25T00:40:46.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.845+0000 7f353a863640 1 -- 172.21.15.29:0/211248338 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f35340cc270 con 0x7f35180680a0 2024-04-25T00:40:46.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.849+0000 7f352a7fc640 1 -- 172.21.15.29:0/211248338 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f35340cc270 con 0x7f35180680a0 2024-04-25T00:40:46.848 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:40:46.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:40:46.849 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-04-25T00:40:46.849 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:40:46.849 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:40:46.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.849+0000 7f353a863640 1 -- 172.21.15.29:0/211248338 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f35180680a0 msgr2=0x7f351806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:46.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.849+0000 7f353a863640 1 --2- 172.21.15.29:0/211248338 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f35180680a0 0x7f351806a560 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f35240ad000 tx=0x7f3524004740 comp rx=0 tx=0).stop 2024-04-25T00:40:46.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.849+0000 7f353a863640 1 -- 172.21.15.29:0/211248338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f353406cc60 msgr2=0x7f3534159130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:46.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.849+0000 7f353a863640 1 --2- 172.21.15.29:0/211248338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f353406cc60 0x7f3534159130 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f353000d830 tx=0x7f353000dcf0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.849+0000 7f353a863640 1 -- 172.21.15.29:0/211248338 shutdown_connections 2024-04-25T00:40:46.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.849+0000 7f353a863640 1 --2- 172.21.15.29:0/211248338 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f35180680a0 0x7f351806a560 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.849+0000 7f353a863640 1 --2- 172.21.15.29:0/211248338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f353415cdc0 0x7f353414e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.849+0000 7f353a863640 1 --2- 172.21.15.29:0/211248338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f353415c4a0 0x7f3534159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.849+0000 7f353a863640 1 --2- 172.21.15.29:0/211248338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f353406cc60 0x7f3534159130 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.849+0000 7f353a863640 1 -- 172.21.15.29:0/211248338 >> 172.21.15.29:0/211248338 conn(0x7f35340b1770 msgr2=0x7f35340cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:46.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.849+0000 7f353a863640 1 -- 172.21.15.29:0/211248338 shutdown_connections 2024-04-25T00:40:46.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.849+0000 7f353a863640 1 -- 172.21.15.29:0/211248338 wait complete. 2024-04-25T00:40:46.867 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:40:46.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.989+0000 7f86e4ede640 1 Processor -- start 2024-04-25T00:40:46.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.989+0000 7f86e4ede640 1 -- start start 2024-04-25T00:40:46.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86e4ede640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f86e015f3c0 0x7f86e00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86e4ede640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86e00b3720 con 0x7f86e00b4780 2024-04-25T00:40:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86e4ede640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86e00b39c0 con 0x7f86e015e8d0 2024-04-25T00:40:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86e4ede640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f86e00b42f0 con 0x7f86e015f3c0 2024-04-25T00:40:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86ddd9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f86e00b4780 0x7f86e015c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54464/0 (socket says 172.21.15.29:54464) 2024-04-25T00:40:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86ddd9b640 1 -- 172.21.15.29:0/1380106078 learned_addr learned my addr 172.21.15.29:0/1380106078 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86ded9d640 1 --2- 172.21.15.29:0/1380106078 >> v2:172.21.15.157:3300/0 conn(0x7f86e015f3c0 0x7f86e00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3773902527 0 0) 0x7f86e00b3720 con 0x7f86e00b4780 2024-04-25T00:40:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86c8003530 con 0x7f86e00b4780 2024-04-25T00:40:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2244177199 0 0) 0x7f86c8003530 con 0x7f86e00b4780 2024-04-25T00:40:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 >> v2:172.21.15.157:3300/0 conn(0x7f86e015f3c0 msgr2=0x7f86e00cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 --2- 172.21.15.29:0/1380106078 >> v2:172.21.15.157:3300/0 conn(0x7f86e015f3c0 0x7f86e00cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 >> v1:172.21.15.157:6789/0 conn(0x7f86e015e8d0 legacy=0x7f86e015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e00b3ca0 con 0x7f86e00b4780 2024-04-25T00:40:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f86d4001680 con 0x7f86e00b4780 2024-04-25T00:40:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 >> v1:172.21.15.29:6789/0 conn(0x7f86e00b4780 legacy=0x7f86e015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 --2- 172.21.15.29:0/1380106078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c80039c0 0x7f86c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 --2- 172.21.15.29:0/1380106078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c8004810 0x7f86c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 --2- 172.21.15.29:0/1380106078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c80051d0 0x7f86c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f86e00b42f0 con 0x7f86c80039c0 2024-04-25T00:40:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86c800c070 con 0x7f86c80051d0 2024-04-25T00:40:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f86c800c270 con 0x7f86c8004810 2024-04-25T00:40:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86de59c640 1 --2- 172.21.15.29:0/1380106078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c80039c0 0x7f86c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86de59c640 1 -- 172.21.15.29:0/1380106078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c8004810 msgr2=0x7f86c8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86ddd9b640 1 --2- 172.21.15.29:0/1380106078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c8004810 0x7f86c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:46.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86de59c640 1 --2- 172.21.15.29:0/1380106078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c8004810 0x7f86c8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86de59c640 1 -- 172.21.15.29:0/1380106078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c80051d0 msgr2=0x7f86c800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:46.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86de59c640 1 --2- 172.21.15.29:0/1380106078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c80051d0 0x7f86c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86de59c640 1 -- 172.21.15.29:0/1380106078 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c8003530 con 0x7f86c80039c0 2024-04-25T00:40:46.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.993+0000 7f86ded9d640 1 -- 172.21.15.29:0/1380106078 reap_dead start 2024-04-25T00:40:46.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86de59c640 1 --2- 172.21.15.29:0/1380106078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c80039c0 0x7f86c8003de0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f86d0001780 tx=0x7f86d001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a68b178d515abb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:46.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d00013a0 con 0x7f86c80039c0 2024-04-25T00:40:46.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d00052f0 con 0x7f86c80039c0 2024-04-25T00:40:46.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86dd59a640 1 -- 172.21.15.29:0/1380106078 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0003840 con 0x7f86c80039c0 2024-04-25T00:40:46.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 -- 172.21.15.29:0/1380106078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c80039c0 msgr2=0x7f86c8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:46.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 --2- 172.21.15.29:0/1380106078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c80039c0 0x7f86c8003de0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f86d0001780 tx=0x7f86d001c400 comp rx=0 tx=0).stop 2024-04-25T00:40:46.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 -- 172.21.15.29:0/1380106078 shutdown_connections 2024-04-25T00:40:46.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 --2- 172.21.15.29:0/1380106078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86c80039c0 0x7f86c8003de0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:46.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 -- 172.21.15.29:0/1380106078 >> 172.21.15.29:0/1380106078 conn(0x7f86e00b1870 msgr2=0x7f86e00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:46.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 -- 172.21.15.29:0/1380106078 shutdown_connections 2024-04-25T00:40:46.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 -- 172.21.15.29:0/1380106078 wait complete. 2024-04-25T00:40:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 Processor -- start 2024-04-25T00:40:46.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 -- start start 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86e0138900 0x7f86e013bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e013c140 0x7f86e015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e015d230 0x7f86e0071d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f86e0144a20 con 0x7f86e0138900 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86e015bab0 con 0x7f86e013c140 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86e4ede640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f86e0136a10 con 0x7f86e015d230 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86de59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86e0138900 0x7f86e013bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86de59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86e0138900 0x7f86e013bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:32826/0 (socket says 172.21.15.29:32826) 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86de59c640 1 -- 172.21.15.29:0/851335160 learned_addr learned my addr 172.21.15.29:0/851335160 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86ddd9b640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e013c140 0x7f86e015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86ded9d640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e015d230 0x7f86e0071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86de59c640 1 -- 172.21.15.29:0/851335160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e015d230 msgr2=0x7f86e0071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86de59c640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e015d230 0x7f86e0071d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86de59c640 1 -- 172.21.15.29:0/851335160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e013c140 msgr2=0x7f86e015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86de59c640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e013c140 0x7f86e015ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86de59c640 1 -- 172.21.15.29:0/851335160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e01366c0 con 0x7f86e0138900 2024-04-25T00:40:47.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:46.997+0000 7f86ddd9b640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e013c140 0x7f86e015ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:47.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.001+0000 7f86de59c640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86e0138900 0x7f86e013bc00 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f86d0005040 tx=0x7f86c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:47.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.001+0000 7f86ceffd640 1 -- 172.21.15.29:0/851335160 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d00013a0 con 0x7f86e0138900 2024-04-25T00:40:47.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.001+0000 7f86ceffd640 1 -- 172.21.15.29:0/851335160 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d00ba070 con 0x7f86e0138900 2024-04-25T00:40:47.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.001+0000 7f86e4ede640 1 -- 172.21.15.29:0/851335160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e013c6a0 con 0x7f86e0138900 2024-04-25T00:40:47.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.001+0000 7f86ceffd640 1 -- 172.21.15.29:0/851335160 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0007a40 con 0x7f86e0138900 2024-04-25T00:40:47.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.001+0000 7f86e4ede640 1 -- 172.21.15.29:0/851335160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e013cc60 con 0x7f86e0138900 2024-04-25T00:40:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.001+0000 7f86e4ede640 1 -- 172.21.15.29:0/851335160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e0069270 con 0x7f86e0138900 2024-04-25T00:40:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.001+0000 7f86ceffd640 1 -- 172.21.15.29:0/851335160 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f86d0005290 con 0x7f86e0138900 2024-04-25T00:40:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.001+0000 7f86ceffd640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86c00680d0 0x7f86c006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.001+0000 7f86ddd9b640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86c00680d0 0x7f86c006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.001+0000 7f86ceffd640 1 -- 172.21.15.29:0/851335160 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f86d011e4f0 con 0x7f86e0138900 2024-04-25T00:40:47.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.001+0000 7f86ddd9b640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86c00680d0 0x7f86c006a590 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f86d4000c10 tx=0x7f86d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:47.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.005+0000 7f86ceffd640 1 -- 172.21.15.29:0/851335160 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f86d0016110 con 0x7f86e0138900 2024-04-25T00:40:47.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.173+0000 7f86e4ede640 1 -- 172.21.15.29:0/851335160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f86e015d610 con 0x7f86e0138900 2024-04-25T00:40:47.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.173+0000 7f86ceffd640 1 -- 172.21.15.29:0/851335160 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f86d00f0920 con 0x7f86e0138900 2024-04-25T00:40:47.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.177+0000 7f86e4ede640 1 -- 172.21.15.29:0/851335160 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86c00680d0 msgr2=0x7f86c006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.177+0000 7f86e4ede640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86c00680d0 0x7f86c006a590 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f86d4000c10 tx=0x7f86d4009040 comp rx=0 tx=0).stop 2024-04-25T00:40:47.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.177+0000 7f86e4ede640 1 -- 172.21.15.29:0/851335160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86e0138900 msgr2=0x7f86e013bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.177+0000 7f86e4ede640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86e0138900 0x7f86e013bc00 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f86d0005040 tx=0x7f86c8005840 comp rx=0 tx=0).stop 2024-04-25T00:40:47.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.177+0000 7f86e4ede640 1 -- 172.21.15.29:0/851335160 shutdown_connections 2024-04-25T00:40:47.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.177+0000 7f86e4ede640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86c00680d0 0x7f86c006a590 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.177+0000 7f86e4ede640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e015d230 0x7f86e0071d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.177+0000 7f86e4ede640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e013c140 0x7f86e015ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.177+0000 7f86e4ede640 1 --2- 172.21.15.29:0/851335160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86e0138900 0x7f86e013bc00 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.177+0000 7f86e4ede640 1 -- 172.21.15.29:0/851335160 >> 172.21.15.29:0/851335160 conn(0x7f86e00b1870 msgr2=0x7f86e013fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:47.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.177+0000 7f86e4ede640 1 -- 172.21.15.29:0/851335160 shutdown_connections 2024-04-25T00:40:47.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.177+0000 7f86e4ede640 1 -- 172.21.15.29:0/851335160 wait complete. 2024-04-25T00:40:47.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:47.193 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol58400/64dbb795-e33f-48b8-bc46-3cb0be58bf74) 2024-04-25T00:40:47.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:47.221 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-25T00:40:47.270 INFO:teuthology.orchestra.run.smithi029.stdout:cephfs_data 2024-04-25T00:40:47.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:47.271 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol58400/64dbb795-e33f-48b8-bc46-3cb0be58bf74) 2024-04-25T00:40:47.332 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:40:47.333 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol58400/.meta 2024-04-25T00:40:47.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:47.490 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp79360/subvol76361/3784ea05-266a-4ec3-8766-cea78eb3e822) 2024-04-25T00:40:47.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:47.517 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-25T00:40:47.571 INFO:teuthology.orchestra.run.smithi029.stdout:cephfs_data 2024-04-25T00:40:47.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:47.571 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp79360/subvol76361/3784ea05-266a-4ec3-8766-cea78eb3e822) 2024-04-25T00:40:47.639 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:40:47.639 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp79360/subvol76361/.meta 2024-04-25T00:40:47.699 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol58400 2024-04-25T00:40:47.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73e0b36640 1 Processor -- start 2024-04-25T00:40:47.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73e0b36640 1 -- start start 2024-04-25T00:40:47.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73e0b36640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73dc0b3140 0x7f73dc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73e0b36640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73dc0b41e0 con 0x7f73dc0b3c30 2024-04-25T00:40:47.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73e0b36640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73dc06d960 con 0x7f73dc0b4b10 2024-04-25T00:40:47.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73e0b36640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f73dc06dc40 con 0x7f73dc0b3140 2024-04-25T00:40:47.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73dad9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f73dc0b3c30 0x7f73dc0ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37456/0 (socket says 172.21.15.29:37456) 2024-04-25T00:40:47.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73dad9d640 1 -- 172.21.15.29:0/1803993650 learned_addr learned my addr 172.21.15.29:0/1803993650 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:47.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73da59c640 1 --2- 172.21.15.29:0/1803993650 >> v2:172.21.15.157:3300/0 conn(0x7f73dc0b3140 0x7f73dc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1099056592 0 0) 0x7f73dc0b41e0 con 0x7f73dc0b3c30 2024-04-25T00:40:47.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73c8003530 con 0x7f73dc0b3c30 2024-04-25T00:40:47.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1384045841 0 0) 0x7f73c8003530 con 0x7f73dc0b3c30 2024-04-25T00:40:47.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 >> v2:172.21.15.157:3300/0 conn(0x7f73dc0b3140 msgr2=0x7f73dc06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:47.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 --2- 172.21.15.29:0/1803993650 >> v2:172.21.15.157:3300/0 conn(0x7f73dc0b3140 0x7f73dc06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 >> v1:172.21.15.157:6789/0 conn(0x7f73dc0b4b10 legacy=0x7f73dc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc06df20 con 0x7f73dc0b3c30 2024-04-25T00:40:47.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f73d4001a00 con 0x7f73dc0b3c30 2024-04-25T00:40:47.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 >> v1:172.21.15.29:6789/0 conn(0x7f73dc0b3c30 legacy=0x7f73dc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 --2- 172.21.15.29:0/1803993650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73c80039c0 0x7f73c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 --2- 172.21.15.29:0/1803993650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73c8004810 0x7f73c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 --2- 172.21.15.29:0/1803993650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73c80051d0 0x7f73c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc06dc40 con 0x7f73c80051d0 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f73c800c070 con 0x7f73c80039c0 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f73c800c270 con 0x7f73c8004810 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73dad9d640 1 --2- 172.21.15.29:0/1803993650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73c80051d0 0x7f73c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73d9d9b640 1 --2- 172.21.15.29:0/1803993650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73c8004810 0x7f73c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73da59c640 1 --2- 172.21.15.29:0/1803993650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73c80039c0 0x7f73c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73dad9d640 1 -- 172.21.15.29:0/1803993650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73c8004810 msgr2=0x7f73c8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73dad9d640 1 --2- 172.21.15.29:0/1803993650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73c8004810 0x7f73c8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73dad9d640 1 -- 172.21.15.29:0/1803993650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73c80039c0 msgr2=0x7f73c8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73dad9d640 1 --2- 172.21.15.29:0/1803993650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73c80039c0 0x7f73c8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73dad9d640 1 -- 172.21.15.29:0/1803993650 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c8003530 con 0x7f73c80051d0 2024-04-25T00:40:47.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.881+0000 7f73da59c640 1 --2- 172.21.15.29:0/1803993650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73c80039c0 0x7f73c8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:47.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.885+0000 7f73dad9d640 1 -- 172.21.15.29:0/1803993650 reap_dead start 2024-04-25T00:40:47.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.885+0000 7f73dad9d640 1 --2- 172.21.15.29:0/1803993650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73c80051d0 0x7f73c800b9d0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f73d400a860 tx=0x7f73d401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6154ab42790a7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:47.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.885+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d400a480 con 0x7f73c80051d0 2024-04-25T00:40:47.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.885+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d40ad630 con 0x7f73c80051d0 2024-04-25T00:40:47.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.885+0000 7f73d959a640 1 -- 172.21.15.29:0/1803993650 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d40ada20 con 0x7f73c80051d0 2024-04-25T00:40:47.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.885+0000 7f73e0b36640 1 -- 172.21.15.29:0/1803993650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73c80051d0 msgr2=0x7f73c800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.885+0000 7f73e0b36640 1 --2- 172.21.15.29:0/1803993650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73c80051d0 0x7f73c800b9d0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f73d400a860 tx=0x7f73d401d380 comp rx=0 tx=0).stop 2024-04-25T00:40:47.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.885+0000 7f73e0b36640 1 -- 172.21.15.29:0/1803993650 shutdown_connections 2024-04-25T00:40:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.885+0000 7f73e0b36640 1 --2- 172.21.15.29:0/1803993650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73c80051d0 0x7f73c800b9d0 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.885+0000 7f73e0b36640 1 -- 172.21.15.29:0/1803993650 >> 172.21.15.29:0/1803993650 conn(0x7f73dc0b1870 msgr2=0x7f73dc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.885+0000 7f73e0b36640 1 -- 172.21.15.29:0/1803993650 shutdown_connections 2024-04-25T00:40:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.885+0000 7f73e0b36640 1 -- 172.21.15.29:0/1803993650 wait complete. 2024-04-25T00:40:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73e0b36640 1 Processor -- start 2024-04-25T00:40:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73e0b36640 1 -- start start 2024-04-25T00:40:47.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73e0b36640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73c80051d0 0x7f73dc13d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73e0b36640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73dc13ea70 0x7f73dc15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73e0b36640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73dc143480 0x7f73dc143900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73e0b36640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc145a90 con 0x7f73dc143480 2024-04-25T00:40:47.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73e0b36640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc145910 con 0x7f73c80051d0 2024-04-25T00:40:47.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73e0b36640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f73dc06e440 con 0x7f73dc13ea70 2024-04-25T00:40:47.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73dad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73dc143480 0x7f73dc143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73dad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73dc143480 0x7f73dc143900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36304/0 (socket says 172.21.15.29:36304) 2024-04-25T00:40:47.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73dad9d640 1 -- 172.21.15.29:0/1839978495 learned_addr learned my addr 172.21.15.29:0/1839978495 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:47.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73da59c640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73c80051d0 0x7f73dc13d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73d9d9b640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73dc13ea70 0x7f73dc15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73dad9d640 1 -- 172.21.15.29:0/1839978495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73dc13ea70 msgr2=0x7f73dc15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73dad9d640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73dc13ea70 0x7f73dc15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73dad9d640 1 -- 172.21.15.29:0/1839978495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73c80051d0 msgr2=0x7f73dc13d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:47.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73dad9d640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73c80051d0 0x7f73dc13d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:47.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73dad9d640 1 -- 172.21.15.29:0/1839978495 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc13d5f0 con 0x7f73dc143480 2024-04-25T00:40:47.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73dad9d640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73dc143480 0x7f73dc143900 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f73d401dc20 tx=0x7f73c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:47.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73d2ffd640 1 -- 172.21.15.29:0/1839978495 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d40bc070 con 0x7f73dc143480 2024-04-25T00:40:47.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73d2ffd640 1 -- 172.21.15.29:0/1839978495 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d4001c20 con 0x7f73dc143480 2024-04-25T00:40:47.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73d2ffd640 1 -- 172.21.15.29:0/1839978495 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d400a480 con 0x7f73dc143480 2024-04-25T00:40:47.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73e0b36640 1 -- 172.21.15.29:0/1839978495 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73dc13f2a0 con 0x7f73dc143480 2024-04-25T00:40:47.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73e0b36640 1 -- 172.21.15.29:0/1839978495 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73dc1390c0 con 0x7f73dc143480 2024-04-25T00:40:47.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73e0b36640 1 -- 172.21.15.29:0/1839978495 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73dc069270 con 0x7f73dc143480 2024-04-25T00:40:47.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.889+0000 7f73d2ffd640 1 -- 172.21.15.29:0/1839978495 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f73d40bf070 con 0x7f73dc143480 2024-04-25T00:40:47.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.893+0000 7f73d2ffd640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f73bc0680a0 0x7f73bc06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:47.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.893+0000 7f73d2ffd640 1 -- 172.21.15.29:0/1839978495 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f73d411e500 con 0x7f73dc143480 2024-04-25T00:40:47.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.893+0000 7f73da59c640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f73bc0680a0 0x7f73bc06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:47.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.893+0000 7f73da59c640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f73bc0680a0 0x7f73bc06a560 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009a20 tx=0x7f73c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:47.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:47.897+0000 7f73d2ffd640 1 -- 172.21.15.29:0/1839978495 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f73d40b4070 con 0x7f73dc143480 2024-04-25T00:40:48.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.085+0000 7f73e0b36640 1 -- 172.21.15.29:0/1839978495 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol58400", "target": ["mon-mgr", ""]}) -- 0x7f73dc1408b0 con 0x7f73bc0680a0 2024-04-25T00:40:48.092 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:48.093+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:48.092 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:48.093+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:48.093 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:48.093+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:48.093 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:48.093+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:48.093 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:48.093+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:48.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.141+0000 7f73d2ffd640 1 -- 172.21.15.29:0/1839978495 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f73dc1408b0 con 0x7f73bc0680a0 2024-04-25T00:40:48.143 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/_nogroup/subvol58400/64dbb795-e33f-48b8-bc46-3cb0be58bf74 2024-04-25T00:40:48.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.145+0000 7f73e0b36640 1 -- 172.21.15.29:0/1839978495 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f73bc0680a0 msgr2=0x7f73bc06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.145+0000 7f73e0b36640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f73bc0680a0 0x7f73bc06a560 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009a20 tx=0x7f73c4008040 comp rx=0 tx=0).stop 2024-04-25T00:40:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.145+0000 7f73e0b36640 1 -- 172.21.15.29:0/1839978495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73dc143480 msgr2=0x7f73dc143900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.145+0000 7f73e0b36640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73dc143480 0x7f73dc143900 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f73d401dc20 tx=0x7f73c8005840 comp rx=0 tx=0).stop 2024-04-25T00:40:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.145+0000 7f73e0b36640 1 -- 172.21.15.29:0/1839978495 shutdown_connections 2024-04-25T00:40:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.145+0000 7f73e0b36640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f73bc0680a0 0x7f73bc06a560 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.145+0000 7f73e0b36640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73dc143480 0x7f73dc143900 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.145+0000 7f73e0b36640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73dc13ea70 0x7f73dc15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.145+0000 7f73e0b36640 1 --2- 172.21.15.29:0/1839978495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73c80051d0 0x7f73dc13d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.145+0000 7f73e0b36640 1 -- 172.21.15.29:0/1839978495 >> 172.21.15.29:0/1839978495 conn(0x7f73dc0b1870 msgr2=0x7f73dc0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.145+0000 7f73e0b36640 1 -- 172.21.15.29:0/1839978495 shutdown_connections 2024-04-25T00:40:48.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.145+0000 7f73e0b36640 1 -- 172.21.15.29:0/1839978495 wait complete. 2024-04-25T00:40:48.161 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol76361 subvol_grp79360 2024-04-25T00:40:48.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faee0f82640 1 Processor -- start 2024-04-25T00:40:48.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faee0f82640 1 -- start start 2024-04-25T00:40:48.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faee0f82640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faedc151640 0x7faedc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faee0f82640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faedc0b2b70 0x7faedc06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faee0f82640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faedc0b40a0 con 0x7faedc15dfd0 2024-04-25T00:40:48.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faee0f82640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7faedc0b3000 con 0x7faedc151640 2024-04-25T00:40:48.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faee0f82640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faedc0b32e0 con 0x7faedc0b2b70 2024-04-25T00:40:48.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faeda59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faedc151640 0x7faedc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faeda59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faedc151640 0x7faedc06d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36320/0 (socket says 172.21.15.29:36320) 2024-04-25T00:40:48.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faeda59c640 1 -- 172.21.15.29:0/3394198122 learned_addr learned my addr 172.21.15.29:0/3394198122 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:48.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faed9d9b640 1 --2- 172.21.15.29:0/3394198122 >> v2:172.21.15.157:3300/0 conn(0x7faedc0b2b70 0x7faedc06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faeda59c640 1 -- 172.21.15.29:0/3394198122 >> v2:172.21.15.157:3300/0 conn(0x7faedc0b2b70 msgr2=0x7faedc06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:48.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faeda59c640 1 --2- 172.21.15.29:0/3394198122 >> v2:172.21.15.157:3300/0 conn(0x7faedc0b2b70 0x7faedc06db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faeda59c640 1 -- 172.21.15.29:0/3394198122 >> v1:172.21.15.157:6789/0 conn(0x7faedc15dfd0 legacy=0x7faedc15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:48.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.285+0000 7faeda59c640 1 -- 172.21.15.29:0/3394198122 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc0b35c0 con 0x7faedc151640 2024-04-25T00:40:48.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.289+0000 7faeda59c640 1 --2- 172.21.15.29:0/3394198122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faedc151640 0x7faedc06d5d0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7faedc151a20 tx=0x7faecc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84eae4191ea10e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:48.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.289+0000 7faed959a640 1 -- 172.21.15.29:0/3394198122 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0b0070 con 0x7faedc151640 2024-04-25T00:40:48.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.289+0000 7faed959a640 1 -- 172.21.15.29:0/3394198122 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faecc0ab600 con 0x7faedc151640 2024-04-25T00:40:48.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.289+0000 7faed959a640 1 -- 172.21.15.29:0/3394198122 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0ab9d0 con 0x7faedc151640 2024-04-25T00:40:48.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.289+0000 7faee0f82640 1 -- 172.21.15.29:0/3394198122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faedc151640 msgr2=0x7faedc06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.289+0000 7faee0f82640 1 --2- 172.21.15.29:0/3394198122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faedc151640 0x7faedc06d5d0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7faedc151a20 tx=0x7faecc01c450 comp rx=0 tx=0).stop 2024-04-25T00:40:48.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.289+0000 7faee0f82640 1 -- 172.21.15.29:0/3394198122 shutdown_connections 2024-04-25T00:40:48.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.289+0000 7faee0f82640 1 --2- 172.21.15.29:0/3394198122 >> v2:172.21.15.157:3300/0 conn(0x7faedc0b2b70 0x7faedc06db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.289+0000 7faee0f82640 1 --2- 172.21.15.29:0/3394198122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faedc151640 0x7faedc06d5d0 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.289+0000 7faee0f82640 1 -- 172.21.15.29:0/3394198122 >> 172.21.15.29:0/3394198122 conn(0x7faedc0b1870 msgr2=0x7faedc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:48.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.289+0000 7faee0f82640 1 -- 172.21.15.29:0/3394198122 shutdown_connections 2024-04-25T00:40:48.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.289+0000 7faee0f82640 1 -- 172.21.15.29:0/3394198122 wait complete. 2024-04-25T00:40:48.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faee0f82640 1 Processor -- start 2024-04-25T00:40:48.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faee0f82640 1 -- start start 2024-04-25T00:40:48.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faee0f82640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faedc151640 0x7faedc14a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faee0f82640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faedc15dfd0 0x7faedc0cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faee0f82640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faedc0ce040 0x7faedc159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faee0f82640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faedc167700 con 0x7faedc15dfd0 2024-04-25T00:40:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faee0f82640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faedc167580 con 0x7faedc151640 2024-04-25T00:40:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faee0f82640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faedc167880 con 0x7faedc0ce040 2024-04-25T00:40:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faed9d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faedc15dfd0 0x7faedc0cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faed9d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faedc15dfd0 0x7faedc0cdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36324/0 (socket says 172.21.15.29:36324) 2024-04-25T00:40:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faed9d9b640 1 -- 172.21.15.29:0/2793383888 learned_addr learned my addr 172.21.15.29:0/2793383888 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faeda59c640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faedc151640 0x7faedc14a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faedad9d640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faedc0ce040 0x7faedc159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faed9d9b640 1 -- 172.21.15.29:0/2793383888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faedc0ce040 msgr2=0x7faedc159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faed9d9b640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faedc0ce040 0x7faedc159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faed9d9b640 1 -- 172.21.15.29:0/2793383888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faedc151640 msgr2=0x7faedc14a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faed9d9b640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faedc151640 0x7faedc14a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faed9d9b640 1 -- 172.21.15.29:0/2793383888 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc14f480 con 0x7faedc15dfd0 2024-04-25T00:40:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faedad9d640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faedc0ce040 0x7faedc159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faed9d9b640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faedc15dfd0 0x7faedc0cdb00 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7faed000d5f0 tx=0x7faed000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faecaffd640 1 -- 172.21.15.29:0/2793383888 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed00125f0 con 0x7faedc15dfd0 2024-04-25T00:40:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faecaffd640 1 -- 172.21.15.29:0/2793383888 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed0019070 con 0x7faedc15dfd0 2024-04-25T00:40:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faee0f82640 1 -- 172.21.15.29:0/2793383888 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faedc14f760 con 0x7faedc15dfd0 2024-04-25T00:40:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faecaffd640 1 -- 172.21.15.29:0/2793383888 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed00129d0 con 0x7faedc15dfd0 2024-04-25T00:40:48.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faee0f82640 1 -- 172.21.15.29:0/2793383888 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faedc14fd20 con 0x7faedc15dfd0 2024-04-25T00:40:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faee0f82640 1 -- 172.21.15.29:0/2793383888 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faedc0692e0 con 0x7faedc15dfd0 2024-04-25T00:40:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.293+0000 7faecaffd640 1 -- 172.21.15.29:0/2793383888 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7faed0012b70 con 0x7faedc15dfd0 2024-04-25T00:40:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.297+0000 7faecaffd640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faebc0680a0 0x7faebc06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.297+0000 7faecaffd640 1 -- 172.21.15.29:0/2793383888 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7faed0084af0 con 0x7faedc15dfd0 2024-04-25T00:40:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.297+0000 7faeda59c640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faebc0680a0 0x7faebc06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.297+0000 7faeda59c640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faebc0680a0 0x7faebc06a560 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7faecc0ad000 tx=0x7faecc0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:48.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.301+0000 7faecaffd640 1 -- 172.21.15.29:0/2793383888 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faed0056f20 con 0x7faedc15dfd0 2024-04-25T00:40:48.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.489+0000 7faee0f82640 1 -- 172.21.15.29:0/2793383888 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol76361", "group_name": "subvol_grp79360", "target": ["mon-mgr", ""]}) -- 0x7faedc06e1d0 con 0x7faebc0680a0 2024-04-25T00:40:48.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.573+0000 7faecaffd640 1 -- 172.21.15.29:0/2793383888 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7faedc06e1d0 con 0x7faebc0680a0 2024-04-25T00:40:48.572 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/subvol_grp79360/subvol76361/3784ea05-266a-4ec3-8766-cea78eb3e822 2024-04-25T00:40:48.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.573+0000 7faee0f82640 1 -- 172.21.15.29:0/2793383888 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faebc0680a0 msgr2=0x7faebc06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.573+0000 7faee0f82640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faebc0680a0 0x7faebc06a560 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7faecc0ad000 tx=0x7faecc0a6000 comp rx=0 tx=0).stop 2024-04-25T00:40:48.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.573+0000 7faee0f82640 1 -- 172.21.15.29:0/2793383888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faedc15dfd0 msgr2=0x7faedc0cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.573+0000 7faee0f82640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faedc15dfd0 0x7faedc0cdb00 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7faed000d5f0 tx=0x7faed000dab0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.573+0000 7faee0f82640 1 -- 172.21.15.29:0/2793383888 shutdown_connections 2024-04-25T00:40:48.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.573+0000 7faee0f82640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7faebc0680a0 0x7faebc06a560 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.573+0000 7faee0f82640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faedc0ce040 0x7faedc159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.573+0000 7faee0f82640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faedc15dfd0 0x7faedc0cdb00 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.573+0000 7faee0f82640 1 --2- 172.21.15.29:0/2793383888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faedc151640 0x7faedc14a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.573+0000 7faee0f82640 1 -- 172.21.15.29:0/2793383888 >> 172.21.15.29:0/2793383888 conn(0x7faedc0b1870 msgr2=0x7faedc070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:48.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.577+0000 7faee0f82640 1 -- 172.21.15.29:0/2793383888 shutdown_connections 2024-04-25T00:40:48.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.577+0000 7faee0f82640 1 -- 172.21.15.29:0/2793383888 wait complete. 2024-04-25T00:40:48.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:48.595 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol58400/.meta) 2024-04-25T00:40:48.612 INFO:teuthology.orchestra.run.smithi029.stdout:[GLOBAL] 2024-04-25T00:40:48.613 INFO:teuthology.orchestra.run.smithi029.stdout:version = 2 2024-04-25T00:40:48.613 INFO:teuthology.orchestra.run.smithi029.stdout:type = subvolume 2024-04-25T00:40:48.613 INFO:teuthology.orchestra.run.smithi029.stdout:path = /volumes/_nogroup/subvol58400/64dbb795-e33f-48b8-bc46-3cb0be58bf74 2024-04-25T00:40:48.613 INFO:teuthology.orchestra.run.smithi029.stdout:state = complete 2024-04-25T00:40:48.613 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:48.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:48.614 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp79360/subvol76361/.meta) 2024-04-25T00:40:48.670 INFO:teuthology.orchestra.run.smithi029.stdout:[GLOBAL] 2024-04-25T00:40:48.670 INFO:teuthology.orchestra.run.smithi029.stdout:version = 2 2024-04-25T00:40:48.670 INFO:teuthology.orchestra.run.smithi029.stdout:type = subvolume 2024-04-25T00:40:48.670 INFO:teuthology.orchestra.run.smithi029.stdout:path = /volumes/subvol_grp79360/subvol76361/3784ea05-266a-4ec3-8766-cea78eb3e822 2024-04-25T00:40:48.670 INFO:teuthology.orchestra.run.smithi029.stdout:state = complete 2024-04-25T00:40:48.670 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:48.671 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol58400 2024-04-25T00:40:48.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.845+0000 7fc5d367e640 1 Processor -- start 2024-04-25T00:40:48.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.845+0000 7fc5d367e640 1 -- start start 2024-04-25T00:40:48.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d367e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b4d30 0x7fc5cc0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d367e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5cc06d980 con 0x7fc5cc06d210 2024-04-25T00:40:48.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d367e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5cc06dc60 con 0x7fc5cc15df70 2024-04-25T00:40:48.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d367e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc5cc06df40 con 0x7fc5cc0b4d30 2024-04-25T00:40:48.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d1c1b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc5cc06d210 0x7fc5cc0ad8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37472/0 (socket says 172.21.15.29:37472) 2024-04-25T00:40:48.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d1c1b640 1 -- 172.21.15.29:0/1574041350 learned_addr learned my addr 172.21.15.29:0/1574041350 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:48.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d0c19640 1 --2- 172.21.15.29:0/1574041350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b4d30 0x7fc5cc0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3286272625 0 0) 0x7fc5cc06d980 con 0x7fc5cc06d210 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5b8003530 con 0x7fc5cc06d210 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3459027170 0 0) 0x7fc5b8003530 con 0x7fc5cc06d210 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b4d30 msgr2=0x7fc5cc0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 --2- 172.21.15.29:0/1574041350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b4d30 0x7fc5cc0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 >> v1:172.21.15.157:6789/0 conn(0x7fc5cc15df70 legacy=0x7fc5cc0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc06e220 con 0x7fc5cc06d210 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fc5c4001a00 con 0x7fc5cc06d210 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 >> v1:172.21.15.29:6789/0 conn(0x7fc5cc06d210 legacy=0x7fc5cc0ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 --2- 172.21.15.29:0/1574041350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b4d30 0x7fc5b80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 --2- 172.21.15.29:0/1574041350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5b8003f70 0x7fc5b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 --2- 172.21.15.29:0/1574041350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5b8004d50 0x7fc5b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc06df40 con 0x7fc5b8004d50 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b800bab0 con 0x7fc5b8003f70 2024-04-25T00:40:48.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b800bcb0 con 0x7fc5cc0b4d30 2024-04-25T00:40:48.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d1c1b640 1 --2- 172.21.15.29:0/1574041350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5b8004d50 0x7fc5b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d141a640 1 --2- 172.21.15.29:0/1574041350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b4d30 0x7fc5b80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d0c19640 1 --2- 172.21.15.29:0/1574041350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5b8003f70 0x7fc5b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d1c1b640 1 -- 172.21.15.29:0/1574041350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b4d30 msgr2=0x7fc5b80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:48.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d1c1b640 1 --2- 172.21.15.29:0/1574041350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b4d30 0x7fc5b80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d1c1b640 1 -- 172.21.15.29:0/1574041350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5b8003f70 msgr2=0x7fc5b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:48.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d1c1b640 1 --2- 172.21.15.29:0/1574041350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5b8003f70 0x7fc5b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d1c1b640 1 -- 172.21.15.29:0/1574041350 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b8003530 con 0x7fc5b8004d50 2024-04-25T00:40:48.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.849+0000 7fc5d141a640 1 --2- 172.21.15.29:0/1574041350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b4d30 0x7fc5b80039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:48.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d1c1b640 1 --2- 172.21.15.29:0/1574041350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5b8004d50 0x7fc5b80051d0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400a840 tx=0x7fc5c401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7982a393f979c9aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400a460 con 0x7fc5b8004d50 2024-04-25T00:40:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c40ad650 con 0x7fc5b8004d50 2024-04-25T00:40:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5c3fff640 1 -- 172.21.15.29:0/1574041350 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40ada80 con 0x7fc5b8004d50 2024-04-25T00:40:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 -- 172.21.15.29:0/1574041350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5b8004d50 msgr2=0x7fc5b80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 --2- 172.21.15.29:0/1574041350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5b8004d50 0x7fc5b80051d0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400a840 tx=0x7fc5c401d360 comp rx=0 tx=0).stop 2024-04-25T00:40:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d1c1b640 1 -- 172.21.15.29:0/1574041350 reap_dead start 2024-04-25T00:40:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 -- 172.21.15.29:0/1574041350 shutdown_connections 2024-04-25T00:40:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 -- 172.21.15.29:0/1574041350 >> 172.21.15.29:0/1574041350 conn(0x7fc5cc0b1870 msgr2=0x7fc5cc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 -- 172.21.15.29:0/1574041350 shutdown_connections 2024-04-25T00:40:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 -- 172.21.15.29:0/1574041350 wait complete. 2024-04-25T00:40:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 Processor -- start 2024-04-25T00:40:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 -- start start 2024-04-25T00:40:48.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5cc142650 0x7fc5cc158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5cc152d50 0x7fc5cc0b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b07b0 0x7fc5cc1f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc15e350 con 0x7fc5cc142650 2024-04-25T00:40:48.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc06d680 con 0x7fc5cc152d50 2024-04-25T00:40:48.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d367e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc5cc06ea20 con 0x7fc5cc0b07b0 2024-04-25T00:40:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d141a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5cc142650 0x7fc5cc158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d141a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5cc142650 0x7fc5cc158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36334/0 (socket says 172.21.15.29:36334) 2024-04-25T00:40:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d141a640 1 -- 172.21.15.29:0/222042350 learned_addr learned my addr 172.21.15.29:0/222042350 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d1c1b640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b07b0 0x7fc5cc1f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d0c19640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5cc152d50 0x7fc5cc0b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d141a640 1 -- 172.21.15.29:0/222042350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b07b0 msgr2=0x7fc5cc1f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d141a640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b07b0 0x7fc5cc1f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d141a640 1 -- 172.21.15.29:0/222042350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5cc152d50 msgr2=0x7fc5cc0b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d141a640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5cc152d50 0x7fc5cc0b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.853+0000 7fc5d141a640 1 -- 172.21.15.29:0/222042350 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc158d90 con 0x7fc5cc142650 2024-04-25T00:40:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.857+0000 7fc5d141a640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5cc142650 0x7fc5cc158420 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e7b0 tx=0x7fc5c800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.857+0000 7fc5c1ffb640 1 -- 172.21.15.29:0/222042350 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80050d0 con 0x7fc5cc142650 2024-04-25T00:40:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.857+0000 7fc5c1ffb640 1 -- 172.21.15.29:0/222042350 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c8018070 con 0x7fc5cc142650 2024-04-25T00:40:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.857+0000 7fc5d367e640 1 -- 172.21.15.29:0/222042350 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc159070 con 0x7fc5cc142650 2024-04-25T00:40:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.857+0000 7fc5c1ffb640 1 -- 172.21.15.29:0/222042350 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800c6c0 con 0x7fc5cc142650 2024-04-25T00:40:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.857+0000 7fc5d367e640 1 -- 172.21.15.29:0/222042350 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc155040 con 0x7fc5cc142650 2024-04-25T00:40:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.857+0000 7fc5d367e640 1 -- 172.21.15.29:0/222042350 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5cc069270 con 0x7fc5cc142650 2024-04-25T00:40:48.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.857+0000 7fc5c1ffb640 1 -- 172.21.15.29:0/222042350 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fc5c800c860 con 0x7fc5cc142650 2024-04-25T00:40:48.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.857+0000 7fc5c1ffb640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc5b00680d0 0x7fc5b006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.857+0000 7fc5c1ffb640 1 -- 172.21.15.29:0/222042350 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fc5c80843c0 con 0x7fc5cc142650 2024-04-25T00:40:48.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.857+0000 7fc5d0c19640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc5b00680d0 0x7fc5b006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.857+0000 7fc5d0c19640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc5b00680d0 0x7fc5b006a590 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc157ea0 tx=0x7fc5bc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:48.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:48.861+0000 7fc5c1ffb640 1 -- 172.21.15.29:0/222042350 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc5c80568b0 con 0x7fc5cc142650 2024-04-25T00:40:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.053+0000 7fc5d367e640 1 -- 172.21.15.29:0/222042350 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol58400", "target": ["mon-mgr", ""]}) -- 0x7fc5cc159590 con 0x7fc5b00680d0 2024-04-25T00:40:49.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.077+0000 7fc5c1ffb640 1 -- 172.21.15.29:0/222042350 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc5cc159590 con 0x7fc5b00680d0 2024-04-25T00:40:49.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.077+0000 7fc5d367e640 1 -- 172.21.15.29:0/222042350 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc5b00680d0 msgr2=0x7fc5b006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:49.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.077+0000 7fc5d367e640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc5b00680d0 0x7fc5b006a590 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc157ea0 tx=0x7fc5bc008040 comp rx=0 tx=0).stop 2024-04-25T00:40:49.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.077+0000 7fc5d367e640 1 -- 172.21.15.29:0/222042350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5cc142650 msgr2=0x7fc5cc158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:49.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.077+0000 7fc5d367e640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5cc142650 0x7fc5cc158420 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e7b0 tx=0x7fc5c800ec70 comp rx=0 tx=0).stop 2024-04-25T00:40:49.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.077+0000 7fc5d367e640 1 -- 172.21.15.29:0/222042350 shutdown_connections 2024-04-25T00:40:49.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.077+0000 7fc5d367e640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc5b00680d0 0x7fc5b006a590 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:49.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.077+0000 7fc5d367e640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5cc0b07b0 0x7fc5cc1f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:49.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.077+0000 7fc5d367e640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5cc152d50 0x7fc5cc0b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:49.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.077+0000 7fc5d367e640 1 --2- 172.21.15.29:0/222042350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5cc142650 0x7fc5cc158420 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:49.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.077+0000 7fc5d367e640 1 -- 172.21.15.29:0/222042350 >> 172.21.15.29:0/222042350 conn(0x7fc5cc0b1870 msgr2=0x7fc5cc15d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:49.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.077+0000 7fc5d367e640 1 -- 172.21.15.29:0/222042350 shutdown_connections 2024-04-25T00:40:49.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.077+0000 7fc5d367e640 1 -- 172.21.15.29:0/222042350 wait complete. 2024-04-25T00:40:49.083 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:49.081+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:49.083 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:49.081+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:49.083 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:49.081+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:49.083 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:49.081+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:49.083 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:49.081+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:49.098 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol76361 subvol_grp79360 2024-04-25T00:40:49.099 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:49.101+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:49.099 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:49.101+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:49.100 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:49.101+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:49.100 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:49.101+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:49.100 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:40:49.101+0000 7f987a583640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:40:49.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.221+0000 7fcee33c0640 1 Processor -- start 2024-04-25T00:40:49.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.221+0000 7fcee33c0640 1 -- start start 2024-04-25T00:40:49.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.221+0000 7fcee33c0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcedc15e560 0x7fcedc0d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:49.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.221+0000 7fcee33c0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcedc0d2680 0x7fcedc0d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:49.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.221+0000 7fcee33c0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcedc0cfed0 con 0x7fcedc15e180 2024-04-25T00:40:49.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.221+0000 7fcee33c0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fcedc0b7100 con 0x7fcedc15e560 2024-04-25T00:40:49.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.221+0000 7fcee33c0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcedc0b73e0 con 0x7fcedc0d2680 2024-04-25T00:40:49.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.221+0000 7fcee095b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcedc15e560 0x7fcedc0d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:49.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.221+0000 7fcee115c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcedc15e180 0x7fcedc0cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37476/0 (socket says 172.21.15.29:37476) 2024-04-25T00:40:49.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.221+0000 7fcee115c640 1 -- 172.21.15.29:0/3685489400 learned_addr learned my addr 172.21.15.29:0/3685489400 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:49.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee195d640 1 --2- 172.21.15.29:0/3685489400 >> v2:172.21.15.157:3300/0 conn(0x7fcedc0d2680 0x7fcedc0d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:49.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee095b640 1 -- 172.21.15.29:0/3685489400 >> v2:172.21.15.157:3300/0 conn(0x7fcedc0d2680 msgr2=0x7fcedc0d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:49.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee095b640 1 --2- 172.21.15.29:0/3685489400 >> v2:172.21.15.157:3300/0 conn(0x7fcedc0d2680 0x7fcedc0d2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:49.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee095b640 1 -- 172.21.15.29:0/3685489400 >> v1:172.21.15.29:6789/0 conn(0x7fcedc15e180 legacy=0x7fcedc0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:49.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fced3fff640 1 -- 172.21.15.29:0/3685489400 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3379079991 0 0) 0x7fcedc0cfed0 con 0x7fcedc15e180 2024-04-25T00:40:49.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee095b640 1 -- 172.21.15.29:0/3685489400 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc0b76c0 con 0x7fcedc15e560 2024-04-25T00:40:49.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee095b640 1 --2- 172.21.15.29:0/3685489400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcedc15e560 0x7fcedc0d20e0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0ceac0 tx=0x7fcecc01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcfdf6d716dfd36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:49.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fced3fff640 1 -- 172.21.15.29:0/3685489400 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0b5070 con 0x7fcedc15e560 2024-04-25T00:40:49.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fced3fff640 1 -- 172.21.15.29:0/3685489400 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcecc0a8970 con 0x7fcedc15e560 2024-04-25T00:40:49.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fced3fff640 1 -- 172.21.15.29:0/3685489400 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0a8d40 con 0x7fcedc15e560 2024-04-25T00:40:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee33c0640 1 -- 172.21.15.29:0/3685489400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcedc15e560 msgr2=0x7fcedc0d20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee33c0640 1 --2- 172.21.15.29:0/3685489400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcedc15e560 0x7fcedc0d20e0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0ceac0 tx=0x7fcecc01ec70 comp rx=0 tx=0).stop 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee33c0640 1 -- 172.21.15.29:0/3685489400 shutdown_connections 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee33c0640 1 --2- 172.21.15.29:0/3685489400 >> v2:172.21.15.157:3300/0 conn(0x7fcedc0d2680 0x7fcedc0d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee33c0640 1 --2- 172.21.15.29:0/3685489400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcedc15e560 0x7fcedc0d20e0 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee33c0640 1 -- 172.21.15.29:0/3685489400 >> 172.21.15.29:0/3685489400 conn(0x7fcedc0d40a0 msgr2=0x7fcedc0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee33c0640 1 -- 172.21.15.29:0/3685489400 shutdown_connections 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.225+0000 7fcee33c0640 1 -- 172.21.15.29:0/3685489400 wait complete. 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee33c0640 1 Processor -- start 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee33c0640 1 -- start start 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee33c0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcedc0d2680 0x7fcedc0c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee33c0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcedc15e180 0x7fcedc15b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee33c0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcedc15e560 0x7fcedc0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee33c0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc0b7d60 con 0x7fcedc15e180 2024-04-25T00:40:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee33c0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc0b7be0 con 0x7fcedc0d2680 2024-04-25T00:40:49.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee33c0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcedc0b7ee0 con 0x7fcedc15e560 2024-04-25T00:40:49.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee095b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcedc15e180 0x7fcedc15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:49.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee095b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcedc15e180 0x7fcedc15b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36372/0 (socket says 172.21.15.29:36372) 2024-04-25T00:40:49.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee095b640 1 -- 172.21.15.29:0/2623626191 learned_addr learned my addr 172.21.15.29:0/2623626191 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:49.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee115c640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcedc0d2680 0x7fcedc0c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:49.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee195d640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcedc15e560 0x7fcedc0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:49.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee095b640 1 -- 172.21.15.29:0/2623626191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcedc15e560 msgr2=0x7fcedc0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:49.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee095b640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcedc15e560 0x7fcedc0cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:49.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee095b640 1 -- 172.21.15.29:0/2623626191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcedc0d2680 msgr2=0x7fcedc0c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:49.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee095b640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcedc0d2680 0x7fcedc0c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:49.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee095b640 1 -- 172.21.15.29:0/2623626191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc0ca220 con 0x7fcedc15e180 2024-04-25T00:40:49.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee095b640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcedc15e180 0x7fcedc15b730 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7fcecc01c8a0 tx=0x7fcecc01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:49.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fced1ffb640 1 -- 172.21.15.29:0/2623626191 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0c0070 con 0x7fcedc15e180 2024-04-25T00:40:49.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fced1ffb640 1 -- 172.21.15.29:0/2623626191 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcecc004230 con 0x7fcedc15e180 2024-04-25T00:40:49.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee33c0640 1 -- 172.21.15.29:0/2623626191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcedc0ca500 con 0x7fcedc15e180 2024-04-25T00:40:49.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fced1ffb640 1 -- 172.21.15.29:0/2623626191 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0b5040 con 0x7fcedc15e180 2024-04-25T00:40:49.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee33c0640 1 -- 172.21.15.29:0/2623626191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcedc0caac0 con 0x7fcedc15e180 2024-04-25T00:40:49.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fcee33c0640 1 -- 172.21.15.29:0/2623626191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcedc0692e0 con 0x7fcedc15e180 2024-04-25T00:40:49.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.229+0000 7fced1ffb640 1 -- 172.21.15.29:0/2623626191 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fcecc0c3070 con 0x7fcedc15e180 2024-04-25T00:40:49.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.233+0000 7fced1ffb640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcec00680a0 0x7fcec006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:49.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.233+0000 7fced1ffb640 1 -- 172.21.15.29:0/2623626191 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fcecc123630 con 0x7fcedc15e180 2024-04-25T00:40:49.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.233+0000 7fcee115c640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcec00680a0 0x7fcec006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:49.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.233+0000 7fcee115c640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcec00680a0 0x7fcec006a560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fced8000f70 tx=0x7fced8007b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:49.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.237+0000 7fced1ffb640 1 -- 172.21.15.29:0/2623626191 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcecc0187d0 con 0x7fcedc15e180 2024-04-25T00:40:49.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.425+0000 7fcee33c0640 1 -- 172.21.15.29:0/2623626191 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol76361", "group_name": "subvol_grp79360", "target": ["mon-mgr", ""]}) -- 0x7fcedc15dbd0 con 0x7fcec00680a0 2024-04-25T00:40:49.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:49.525+0000 7fced1ffb640 1 -- 172.21.15.29:0/2623626191 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fcecc0ef4d0 con 0x7fcedc15e180 2024-04-25T00:40:55.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.101+0000 7fced1ffb640 1 -- 172.21.15.29:0/2623626191 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcedc15dbd0 con 0x7fcec00680a0 2024-04-25T00:40:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.105+0000 7fcee33c0640 1 -- 172.21.15.29:0/2623626191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcec00680a0 msgr2=0x7fcec006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.105+0000 7fcee33c0640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcec00680a0 0x7fcec006a560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fced8000f70 tx=0x7fced8007b80 comp rx=0 tx=0).stop 2024-04-25T00:40:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.105+0000 7fcee33c0640 1 -- 172.21.15.29:0/2623626191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcedc15e180 msgr2=0x7fcedc15b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.105+0000 7fcee33c0640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcedc15e180 0x7fcedc15b730 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7fcecc01c8a0 tx=0x7fcecc01ba10 comp rx=0 tx=0).stop 2024-04-25T00:40:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.105+0000 7fcee33c0640 1 -- 172.21.15.29:0/2623626191 shutdown_connections 2024-04-25T00:40:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.105+0000 7fcee33c0640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcec00680a0 0x7fcec006a560 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.105+0000 7fcee33c0640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcedc15e560 0x7fcedc0cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.105+0000 7fcee33c0640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcedc15e180 0x7fcedc15b730 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.105+0000 7fcee33c0640 1 --2- 172.21.15.29:0/2623626191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcedc0d2680 0x7fcedc0c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.105+0000 7fcee33c0640 1 -- 172.21.15.29:0/2623626191 >> 172.21.15.29:0/2623626191 conn(0x7fcedc0d40a0 msgr2=0x7fcedc0cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.105+0000 7fcee33c0640 1 -- 172.21.15.29:0/2623626191 shutdown_connections 2024-04-25T00:40:55.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.105+0000 7fcee33c0640 1 -- 172.21.15.29:0/2623626191 wait complete. 2024-04-25T00:40:55.126 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-25T00:40:55.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:40:55.185 INFO:teuthology.orchestra.run.smithi029.stdout:2 2024-04-25T00:40:55.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:40:55.193 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:40:55.193 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:40:55.237 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:40:55.280 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:40:55.280 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:40:55.280 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:40:55.280 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:40:55.285 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:40:55.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.457+0000 7f02710d7640 1 Processor -- start 2024-04-25T00:40:55.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.457+0000 7f02710d7640 1 -- start start 2024-04-25T00:40:55.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f02710d7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f026c15e4e0 0x7f026c0d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f02710d7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f026c0cfe30 con 0x7f026c15e8c0 2024-04-25T00:40:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f02710d7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f026c0b6f80 con 0x7f026c0d4bf0 2024-04-25T00:40:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f02710d7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f026c0b7260 con 0x7f026c15e4e0 2024-04-25T00:40:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f026b59e640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f026c15e8c0 0x7f026c0b7780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37478/0 (socket says 172.21.15.29:37478) 2024-04-25T00:40:55.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f026b59e640 1 -- 172.21.15.29:0/761333503 learned_addr learned my addr 172.21.15.29:0/761333503 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f026ad9d640 1 --2- 172.21.15.29:0/761333503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f026c15e4e0 0x7f026c0d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 203997843 0 0) 0x7f026c0cfe30 con 0x7f026c15e8c0 2024-04-25T00:40:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0254003530 con 0x7f026c15e8c0 2024-04-25T00:40:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1398165839 0 0) 0x7f0254003530 con 0x7f026c15e8c0 2024-04-25T00:40:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f026c15e4e0 msgr2=0x7f026c0d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 --2- 172.21.15.29:0/761333503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f026c15e4e0 0x7f026c0d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 >> v1:172.21.15.157:6789/0 conn(0x7f026c0d4bf0 legacy=0x7f026c0d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026c0ced50 con 0x7f026c15e8c0 2024-04-25T00:40:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f0264001a00 con 0x7f026c15e8c0 2024-04-25T00:40:55.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 >> v1:172.21.15.29:6789/0 conn(0x7f026c15e8c0 legacy=0x7f026c0b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 --2- 172.21.15.29:0/761333503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02540039f0 0x7f0254003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 --2- 172.21.15.29:0/761333503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f026c15e4e0 0x7f0254004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 --2- 172.21.15.29:0/761333503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0254004d50 0x7f02540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f026c0b7260 con 0x7f0254004d50 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f025400bb10 con 0x7f02540039f0 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f025400bd10 con 0x7f026c15e4e0 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f026b59e640 1 --2- 172.21.15.29:0/761333503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0254004d50 0x7f02540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f026ad9d640 1 --2- 172.21.15.29:0/761333503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02540039f0 0x7f0254003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f026a59c640 1 --2- 172.21.15.29:0/761333503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f026c15e4e0 0x7f0254004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f026b59e640 1 -- 172.21.15.29:0/761333503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f026c15e4e0 msgr2=0x7f0254004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f026b59e640 1 --2- 172.21.15.29:0/761333503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f026c15e4e0 0x7f0254004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f026b59e640 1 -- 172.21.15.29:0/761333503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02540039f0 msgr2=0x7f0254003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f026b59e640 1 --2- 172.21.15.29:0/761333503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02540039f0 0x7f0254003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.461+0000 7f026b59e640 1 -- 172.21.15.29:0/761333503 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0254003530 con 0x7f0254004d50 2024-04-25T00:40:55.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f026b59e640 1 --2- 172.21.15.29:0/761333503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0254004d50 0x7f02540051d0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f0264002240 tx=0x7f026401d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ded0510f01b8a43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:55.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026400a4a0 con 0x7f0254004d50 2024-04-25T00:40:55.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02640ad650 con 0x7f0254004d50 2024-04-25T00:40:55.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f0269d9b640 1 -- 172.21.15.29:0/761333503 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02640ada40 con 0x7f0254004d50 2024-04-25T00:40:55.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 -- 172.21.15.29:0/761333503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0254004d50 msgr2=0x7f02540051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 --2- 172.21.15.29:0/761333503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0254004d50 0x7f02540051d0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f0264002240 tx=0x7f026401d340 comp rx=0 tx=0).stop 2024-04-25T00:40:55.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f026b59e640 1 -- 172.21.15.29:0/761333503 reap_dead start 2024-04-25T00:40:55.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 -- 172.21.15.29:0/761333503 shutdown_connections 2024-04-25T00:40:55.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 -- 172.21.15.29:0/761333503 >> 172.21.15.29:0/761333503 conn(0x7f026c0d3fb0 msgr2=0x7f026c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:55.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 -- 172.21.15.29:0/761333503 shutdown_connections 2024-04-25T00:40:55.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 -- 172.21.15.29:0/761333503 wait complete. 2024-04-25T00:40:55.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 Processor -- start 2024-04-25T00:40:55.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 -- start start 2024-04-25T00:40:55.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0254004d50 0x7f026c0a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f026c0ccc10 0x7f026c0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f026c0c7cf0 0x7f026c0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f026c0d2030 con 0x7f026c0ccc10 2024-04-25T00:40:55.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f026c0d1eb0 con 0x7f0254004d50 2024-04-25T00:40:55.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02710d7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f026c0d21b0 con 0x7f026c0c7cf0 2024-04-25T00:40:55.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f026a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f026c0ccc10 0x7f026c0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f026a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f026c0ccc10 0x7f026c0cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36394/0 (socket says 172.21.15.29:36394) 2024-04-25T00:40:55.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f026a59c640 1 -- 172.21.15.29:0/1069679793 learned_addr learned my addr 172.21.15.29:0/1069679793 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:55.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f026ad9d640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0254004d50 0x7f026c0a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f026b59e640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f026c0c7cf0 0x7f026c0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f026a59c640 1 -- 172.21.15.29:0/1069679793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f026c0c7cf0 msgr2=0x7f026c0c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f026a59c640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f026c0c7cf0 0x7f026c0c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f026a59c640 1 -- 172.21.15.29:0/1069679793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0254004d50 msgr2=0x7f026c0a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f026a59c640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0254004d50 0x7f026c0a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f026a59c640 1 -- 172.21.15.29:0/1069679793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026c0cd780 con 0x7f026c0ccc10 2024-04-25T00:40:55.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f026a59c640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f026c0ccc10 0x7f026c0cd090 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f026000e4e0 tx=0x7f026000e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:55.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02537fe640 1 -- 172.21.15.29:0/1069679793 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02600052e0 con 0x7f026c0ccc10 2024-04-25T00:40:55.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.465+0000 7f02537fe640 1 -- 172.21.15.29:0/1069679793 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0260018070 con 0x7f026c0ccc10 2024-04-25T00:40:55.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.469+0000 7f02710d7640 1 -- 172.21.15.29:0/1069679793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026c0cda60 con 0x7f026c0ccc10 2024-04-25T00:40:55.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.469+0000 7f02537fe640 1 -- 172.21.15.29:0/1069679793 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260003820 con 0x7f026c0ccc10 2024-04-25T00:40:55.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.469+0000 7f02710d7640 1 -- 172.21.15.29:0/1069679793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f026c158620 con 0x7f026c0ccc10 2024-04-25T00:40:55.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.469+0000 7f02710d7640 1 -- 172.21.15.29:0/1069679793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f026c069270 con 0x7f026c0ccc10 2024-04-25T00:40:55.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.469+0000 7f02537fe640 1 -- 172.21.15.29:0/1069679793 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f02600039c0 con 0x7f026c0ccc10 2024-04-25T00:40:55.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.469+0000 7f02537fe640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0258068280 0x7f025806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.469+0000 7f02537fe640 1 -- 172.21.15.29:0/1069679793 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f0260083d10 con 0x7f026c0ccc10 2024-04-25T00:40:55.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.469+0000 7f026ad9d640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0258068280 0x7f025806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.469+0000 7f026ad9d640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0258068280 0x7f025806a740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f025c009a30 tx=0x7f025c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:55.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.473+0000 7f02537fe640 1 -- 172.21.15.29:0/1069679793 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0260056450 con 0x7f026c0ccc10 2024-04-25T00:40:55.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.633+0000 7f02710d7640 1 -- 172.21.15.29:0/1069679793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f026c0ce2e0 con 0x7f026c0ccc10 2024-04-25T00:40:55.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.637+0000 7f02537fe640 1 -- 172.21.15.29:0/1069679793 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16631 (secure 0 0 0) 0x7f0260005480 con 0x7f026c0ccc10 2024-04-25T00:40:55.638 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:55.638 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":156,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:40:31.094463+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:40:27.796735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:40:28.430727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:40:55.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.641+0000 7f02710d7640 1 -- 172.21.15.29:0/1069679793 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0258068280 msgr2=0x7f025806a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.641+0000 7f02710d7640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0258068280 0x7f025806a740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f025c009a30 tx=0x7f025c008040 comp rx=0 tx=0).stop 2024-04-25T00:40:55.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.641+0000 7f02710d7640 1 -- 172.21.15.29:0/1069679793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f026c0ccc10 msgr2=0x7f026c0cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.641+0000 7f02710d7640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f026c0ccc10 0x7f026c0cd090 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f026000e4e0 tx=0x7f026000e9a0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.641+0000 7f02710d7640 1 -- 172.21.15.29:0/1069679793 shutdown_connections 2024-04-25T00:40:55.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.641+0000 7f02710d7640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0258068280 0x7f025806a740 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.641+0000 7f02710d7640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f026c0c7cf0 0x7f026c0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.641+0000 7f02710d7640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f026c0ccc10 0x7f026c0cd090 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.641+0000 7f02710d7640 1 --2- 172.21.15.29:0/1069679793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0254004d50 0x7f026c0a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.641+0000 7f02710d7640 1 -- 172.21.15.29:0/1069679793 >> 172.21.15.29:0/1069679793 conn(0x7f026c0d3fb0 msgr2=0x7f026c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:55.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.641+0000 7f02710d7640 1 -- 172.21.15.29:0/1069679793 shutdown_connections 2024-04-25T00:40:55.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.641+0000 7f02710d7640 1 -- 172.21.15.29:0/1069679793 wait complete. 2024-04-25T00:40:55.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:40:27.796735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:40:28.430727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:40:55.659 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:40:55.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a25eed640 1 Processor -- start 2024-04-25T00:40:55.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a25eed640 1 -- start start 2024-04-25T00:40:55.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a25eed640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a2015e8d0 0x7f7a200cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a25eed640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a200b3720 con 0x7f7a2015f3c0 2024-04-25T00:40:55.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a25eed640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a200b39c0 con 0x7f7a200b4780 2024-04-25T00:40:55.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a25eed640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7a200b42f0 con 0x7f7a2015e8d0 2024-04-25T00:40:55.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1ffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7a2015f3c0 0x7f7a2015c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37492/0 (socket says 172.21.15.29:37492) 2024-04-25T00:40:55.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1ffff640 1 -- 172.21.15.29:0/26973931 learned_addr learned my addr 172.21.15.29:0/26973931 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1f7fe640 1 --2- 172.21.15.29:0/26973931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a2015e8d0 0x7f7a200cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3261098826 0 0) 0x7f7a200b3720 con 0x7f7a2015f3c0 2024-04-25T00:40:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a08005740 con 0x7f7a2015f3c0 2024-04-25T00:40:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2039657074 0 0) 0x7f7a08005740 con 0x7f7a2015f3c0 2024-04-25T00:40:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a2015e8d0 msgr2=0x7f7a200cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 --2- 172.21.15.29:0/26973931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a2015e8d0 0x7f7a200cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 >> v1:172.21.15.157:6789/0 conn(0x7f7a200b4780 legacy=0x7f7a2015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a200b3ca0 con 0x7f7a2015f3c0 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f7a18001680 con 0x7f7a2015f3c0 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 >> v1:172.21.15.29:6789/0 conn(0x7f7a2015f3c0 legacy=0x7f7a2015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 --2- 172.21.15.29:0/26973931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a08005c00 0x7f7a08006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 --2- 172.21.15.29:0/26973931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a08006a20 0x7f7a08006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 --2- 172.21.15.29:0/26973931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a2015e8d0 0x7f7a080073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7a200b42f0 con 0x7f7a08006a20 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0800dc90 con 0x7f7a08005c00 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7a0800de90 con 0x7f7a2015e8d0 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1effd640 1 --2- 172.21.15.29:0/26973931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a08006a20 0x7f7a08006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1ffff640 1 --2- 172.21.15.29:0/26973931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a2015e8d0 0x7f7a080073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1f7fe640 1 --2- 172.21.15.29:0/26973931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a08005c00 0x7f7a08006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1effd640 1 -- 172.21.15.29:0/26973931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a2015e8d0 msgr2=0x7f7a080073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:55.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1effd640 1 --2- 172.21.15.29:0/26973931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a2015e8d0 0x7f7a080073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1effd640 1 -- 172.21.15.29:0/26973931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a08005c00 msgr2=0x7f7a08006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:55.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1effd640 1 --2- 172.21.15.29:0/26973931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a08005c00 0x7f7a08006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.785+0000 7f7a1effd640 1 -- 172.21.15.29:0/26973931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a08005740 con 0x7f7a08006a20 2024-04-25T00:40:55.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.789+0000 7f7a1effd640 1 --2- 172.21.15.29:0/26973931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a08006a20 0x7f7a08006ea0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f7a14001750 tx=0x7f7a1401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff50a0f20fe4d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:55.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.789+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a140013a0 con 0x7f7a08006a20 2024-04-25T00:40:55.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.789+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a140052f0 con 0x7f7a08006a20 2024-04-25T00:40:55.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.789+0000 7f7a1e7fc640 1 -- 172.21.15.29:0/26973931 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14003840 con 0x7f7a08006a20 2024-04-25T00:40:55.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.789+0000 7f7a25eed640 1 -- 172.21.15.29:0/26973931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a08006a20 msgr2=0x7f7a08006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.789+0000 7f7a25eed640 1 --2- 172.21.15.29:0/26973931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a08006a20 0x7f7a08006ea0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f7a14001750 tx=0x7f7a1401c3c0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.789+0000 7f7a1ffff640 1 -- 172.21.15.29:0/26973931 reap_dead start 2024-04-25T00:40:55.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.789+0000 7f7a25eed640 1 -- 172.21.15.29:0/26973931 shutdown_connections 2024-04-25T00:40:55.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.789+0000 7f7a25eed640 1 -- 172.21.15.29:0/26973931 >> 172.21.15.29:0/26973931 conn(0x7f7a200b1870 msgr2=0x7f7a200b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:55.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.789+0000 7f7a25eed640 1 -- 172.21.15.29:0/26973931 shutdown_connections 2024-04-25T00:40:55.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.789+0000 7f7a25eed640 1 -- 172.21.15.29:0/26973931 wait complete. 2024-04-25T00:40:55.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.789+0000 7f7a25eed640 1 Processor -- start 2024-04-25T00:40:55.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.789+0000 7f7a25eed640 1 -- start start 2024-04-25T00:40:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a25eed640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a08006a20 0x7f7a20072210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a25eed640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a20070ce0 0x7f7a201f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a25eed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a20159e80 0x7f7a2015a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a25eed640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20144a20 con 0x7f7a20159e80 2024-04-25T00:40:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a25eed640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2006cf10 con 0x7f7a20070ce0 2024-04-25T00:40:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a25eed640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7a20071170 con 0x7f7a08006a20 2024-04-25T00:40:55.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a1ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a20159e80 0x7f7a2015a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a1ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a20159e80 0x7f7a2015a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36414/0 (socket says 172.21.15.29:36414) 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a1ffff640 1 -- 172.21.15.29:0/2246471886 learned_addr learned my addr 172.21.15.29:0/2246471886 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a1f7fe640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a08006a20 0x7f7a20072210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a1effd640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a20070ce0 0x7f7a201f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a1ffff640 1 -- 172.21.15.29:0/2246471886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a08006a20 msgr2=0x7f7a20072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a1ffff640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a08006a20 0x7f7a20072210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a1ffff640 1 -- 172.21.15.29:0/2246471886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a20070ce0 msgr2=0x7f7a201f39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a1ffff640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a20070ce0 0x7f7a201f39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a1ffff640 1 -- 172.21.15.29:0/2246471886 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a20071450 con 0x7f7a20159e80 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a1ffff640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a20159e80 0x7f7a2015a330 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f7a2015f7a0 tx=0x7f7a18011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a07fff640 1 -- 172.21.15.29:0/2246471886 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18010040 con 0x7f7a20159e80 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a07fff640 1 -- 172.21.15.29:0/2246471886 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a1801a070 con 0x7f7a20159e80 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a25eed640 1 -- 172.21.15.29:0/2246471886 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a20071730 con 0x7f7a20159e80 2024-04-25T00:40:55.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a07fff640 1 -- 172.21.15.29:0/2246471886 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1800fb80 con 0x7f7a20159e80 2024-04-25T00:40:55.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a25eed640 1 -- 172.21.15.29:0/2246471886 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2015a870 con 0x7f7a20159e80 2024-04-25T00:40:55.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a25eed640 1 -- 172.21.15.29:0/2246471886 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a20069270 con 0x7f7a20159e80 2024-04-25T00:40:55.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a07fff640 1 -- 172.21.15.29:0/2246471886 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f7a1800fd20 con 0x7f7a20159e80 2024-04-25T00:40:55.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a07fff640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7a0c068280 0x7f7a0c06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:55.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a07fff640 1 -- 172.21.15.29:0/2246471886 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f7a180872d0 con 0x7f7a20159e80 2024-04-25T00:40:55.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.793+0000 7f7a1f7fe640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7a0c068280 0x7f7a0c06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:55.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.797+0000 7f7a1f7fe640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7a0c068280 0x7f7a0c06a740 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7a10007520 tx=0x7f7a1000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:55.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.801+0000 7f7a07fff640 1 -- 172.21.15.29:0/2246471886 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7a18059a10 con 0x7f7a20159e80 2024-04-25T00:40:55.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.964+0000 7f7a25eed640 1 -- 172.21.15.29:0/2246471886 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7a2013ba90 con 0x7f7a20159e80 2024-04-25T00:40:55.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.968+0000 7f7a07fff640 1 -- 172.21.15.29:0/2246471886 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) ==== 93+0+10 (secure 0 0 0) 0x7f7a18059310 con 0x7f7a20159e80 2024-04-25T00:40:55.968 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:40:55.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.968+0000 7f7a25eed640 1 -- 172.21.15.29:0/2246471886 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7a0c068280 msgr2=0x7f7a0c06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.968+0000 7f7a25eed640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7a0c068280 0x7f7a0c06a740 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7a10007520 tx=0x7f7a1000b040 comp rx=0 tx=0).stop 2024-04-25T00:40:55.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.968+0000 7f7a25eed640 1 -- 172.21.15.29:0/2246471886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a20159e80 msgr2=0x7f7a2015a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:55.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.968+0000 7f7a25eed640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a20159e80 0x7f7a2015a330 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f7a2015f7a0 tx=0x7f7a18011b60 comp rx=0 tx=0).stop 2024-04-25T00:40:55.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.968+0000 7f7a25eed640 1 -- 172.21.15.29:0/2246471886 shutdown_connections 2024-04-25T00:40:55.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.968+0000 7f7a25eed640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7a0c068280 0x7f7a0c06a740 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.968+0000 7f7a25eed640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a20159e80 0x7f7a2015a330 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.968+0000 7f7a25eed640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a20070ce0 0x7f7a201f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.968+0000 7f7a25eed640 1 --2- 172.21.15.29:0/2246471886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a08006a20 0x7f7a20072210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:55.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.968+0000 7f7a25eed640 1 -- 172.21.15.29:0/2246471886 >> 172.21.15.29:0/2246471886 conn(0x7f7a200b1870 msgr2=0x7f7a2015df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:55.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.968+0000 7f7a25eed640 1 -- 172.21.15.29:0/2246471886 shutdown_connections 2024-04-25T00:40:55.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:55.968+0000 7f7a25eed640 1 -- 172.21.15.29:0/2246471886 wait complete. 2024-04-25T00:40:55.986 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:40:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.100+0000 7f0f5137a640 1 Processor -- start 2024-04-25T00:40:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.100+0000 7f0f5137a640 1 -- start start 2024-04-25T00:40:56.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f5137a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f4c071e10 0x7f0f4c15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f5137a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0f4c0ce820 0x7f0f4c0cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f5137a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f4c06e580 con 0x7f0f4c0721f0 2024-04-25T00:40:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f5137a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f0f4c06f5e0 con 0x7f0f4c0ce820 2024-04-25T00:40:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f5137a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0f4c0b0370 con 0x7f0f4c071e10 2024-04-25T00:40:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f4b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0f4c0ce820 0x7f0f4c0cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f4b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0f4c0ce820 0x7f0f4c0cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36428/0 (socket says 172.21.15.29:36428) 2024-04-25T00:40:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f4b7fe640 1 -- 172.21.15.29:0/2860222134 learned_addr learned my addr 172.21.15.29:0/2860222134 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f4affd640 1 --2- 172.21.15.29:0/2860222134 >> v2:172.21.15.157:3300/0 conn(0x7f0f4c071e10 0x7f0f4c15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f4b7fe640 1 -- 172.21.15.29:0/2860222134 >> v2:172.21.15.157:3300/0 conn(0x7f0f4c071e10 msgr2=0x7f0f4c15ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f4b7fe640 1 --2- 172.21.15.29:0/2860222134 >> v2:172.21.15.157:3300/0 conn(0x7f0f4c071e10 0x7f0f4c15ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f4b7fe640 1 -- 172.21.15.29:0/2860222134 >> v1:172.21.15.157:6789/0 conn(0x7f0f4c0721f0 legacy=0x7f0f4c072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f4b7fe640 1 -- 172.21.15.29:0/2860222134 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c06e820 con 0x7f0f4c0ce820 2024-04-25T00:40:56.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f4affd640 1 --2- 172.21.15.29:0/2860222134 >> v2:172.21.15.157:3300/0 conn(0x7f0f4c071e10 0x7f0f4c15ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f4b7fe640 1 --2- 172.21.15.29:0/2860222134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f4c0ce820 0x7f0f4c0cec30 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c067fc0 tx=0x7f0f4401eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7938a270058008f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f49ffb640 1 -- 172.21.15.29:0/2860222134 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f440b4070 con 0x7f0f4c0ce820 2024-04-25T00:40:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f49ffb640 1 -- 172.21.15.29:0/2860222134 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f440a7c00 con 0x7f0f4c0ce820 2024-04-25T00:40:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.104+0000 7f0f49ffb640 1 -- 172.21.15.29:0/2860222134 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f440af440 con 0x7f0f4c0ce820 2024-04-25T00:40:56.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 -- 172.21.15.29:0/2860222134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f4c0ce820 msgr2=0x7f0f4c0cec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 --2- 172.21.15.29:0/2860222134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f4c0ce820 0x7f0f4c0cec30 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c067fc0 tx=0x7f0f4401eb30 comp rx=0 tx=0).stop 2024-04-25T00:40:56.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 -- 172.21.15.29:0/2860222134 shutdown_connections 2024-04-25T00:40:56.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 --2- 172.21.15.29:0/2860222134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f4c0ce820 0x7f0f4c0cec30 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 --2- 172.21.15.29:0/2860222134 >> v2:172.21.15.157:3300/0 conn(0x7f0f4c071e10 0x7f0f4c15ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 -- 172.21.15.29:0/2860222134 >> 172.21.15.29:0/2860222134 conn(0x7f0f4c0b2920 msgr2=0x7f0f4c0b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 -- 172.21.15.29:0/2860222134 shutdown_connections 2024-04-25T00:40:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 -- 172.21.15.29:0/2860222134 wait complete. 2024-04-25T00:40:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 Processor -- start 2024-04-25T00:40:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 -- start start 2024-04-25T00:40:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f4c071e10 0x7f0f4c1f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f4c0721f0 0x7f0f4c159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f4c0ce820 0x7f0f4c1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c15c150 con 0x7f0f4c0ce820 2024-04-25T00:40:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c15bfd0 con 0x7f0f4c071e10 2024-04-25T00:40:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4c06b210 con 0x7f0f4c0721f0 2024-04-25T00:40:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f4b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f4c0ce820 0x7f0f4c1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f4b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f4c0ce820 0x7f0f4c1599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36434/0 (socket says 172.21.15.29:36434) 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f4b7fe640 1 -- 172.21.15.29:0/3338010538 learned_addr learned my addr 172.21.15.29:0/3338010538 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f4affd640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f4c071e10 0x7f0f4c1f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f4a7fc640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f4c0721f0 0x7f0f4c159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f4b7fe640 1 -- 172.21.15.29:0/3338010538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f4c0721f0 msgr2=0x7f0f4c159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f4b7fe640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f4c0721f0 0x7f0f4c159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f4b7fe640 1 -- 172.21.15.29:0/3338010538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f4c071e10 msgr2=0x7f0f4c1f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f4b7fe640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f4c071e10 0x7f0f4c1f5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f4b7fe640 1 -- 172.21.15.29:0/3338010538 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c1f2eb0 con 0x7f0f4c0ce820 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f4b7fe640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f4c0ce820 0x7f0f4c1599d0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400b2c0 tx=0x7f0f4401b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f437fe640 1 -- 172.21.15.29:0/3338010538 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f440b4040 con 0x7f0f4c0ce820 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f437fe640 1 -- 172.21.15.29:0/3338010538 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f440bf070 con 0x7f0f4c0ce820 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 -- 172.21.15.29:0/3338010538 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4c1f3190 con 0x7f0f4c0ce820 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f437fe640 1 -- 172.21.15.29:0/3338010538 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f440ba680 con 0x7f0f4c0ce820 2024-04-25T00:40:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.108+0000 7f0f5137a640 1 -- 172.21.15.29:0/3338010538 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f4c1f2bb0 con 0x7f0f4c0ce820 2024-04-25T00:40:56.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.112+0000 7f0f5137a640 1 -- 172.21.15.29:0/3338010538 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f4c069270 con 0x7f0f4c0ce820 2024-04-25T00:40:56.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.112+0000 7f0f437fe640 1 -- 172.21.15.29:0/3338010538 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f0f440ba820 con 0x7f0f4c0ce820 2024-04-25T00:40:56.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.112+0000 7f0f437fe640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0f2c068250 0x7f0f2c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.112+0000 7f0f4affd640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0f2c068250 0x7f0f2c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.112+0000 7f0f437fe640 1 -- 172.21.15.29:0/3338010538 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f0f44121910 con 0x7f0f4c0ce820 2024-04-25T00:40:56.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.112+0000 7f0f4affd640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0f2c068250 0x7f0f2c06a710 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c15bb40 tx=0x7f0f3c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:56.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.116+0000 7f0f437fe640 1 -- 172.21.15.29:0/3338010538 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0f440187e0 con 0x7f0f4c0ce820 2024-04-25T00:40:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.272+0000 7f0f5137a640 1 -- 172.21.15.29:0/3338010538 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0f4c06e050 con 0x7f0f4c0ce820 2024-04-25T00:40:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.272+0000 7f0f437fe640 1 -- 172.21.15.29:0/3338010538 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) ==== 104+0+11 (secure 0 0 0) 0x7f0f440f4050 con 0x7f0f4c0ce820 2024-04-25T00:40:56.275 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:40:56.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.276+0000 7f0f5137a640 1 -- 172.21.15.29:0/3338010538 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0f2c068250 msgr2=0x7f0f2c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.276+0000 7f0f5137a640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0f2c068250 0x7f0f2c06a710 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c15bb40 tx=0x7f0f3c00b040 comp rx=0 tx=0).stop 2024-04-25T00:40:56.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.276+0000 7f0f5137a640 1 -- 172.21.15.29:0/3338010538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f4c0ce820 msgr2=0x7f0f4c1599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.276+0000 7f0f5137a640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f4c0ce820 0x7f0f4c1599d0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400b2c0 tx=0x7f0f4401b8d0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.276+0000 7f0f5137a640 1 -- 172.21.15.29:0/3338010538 shutdown_connections 2024-04-25T00:40:56.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.276+0000 7f0f5137a640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0f2c068250 0x7f0f2c06a710 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.276+0000 7f0f5137a640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f4c0ce820 0x7f0f4c1599d0 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.276+0000 7f0f5137a640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f4c0721f0 0x7f0f4c159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.276+0000 7f0f5137a640 1 --2- 172.21.15.29:0/3338010538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f4c071e10 0x7f0f4c1f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.276+0000 7f0f5137a640 1 -- 172.21.15.29:0/3338010538 >> 172.21.15.29:0/3338010538 conn(0x7f0f4c0b2920 msgr2=0x7f0f4c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:56.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.276+0000 7f0f5137a640 1 -- 172.21.15.29:0/3338010538 shutdown_connections 2024-04-25T00:40:56.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.276+0000 7f0f5137a640 1 -- 172.21.15.29:0/3338010538 wait complete. 2024-04-25T00:40:56.293 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:40:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.408+0000 7f8c7aae6640 1 Processor -- start 2024-04-25T00:40:56.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.408+0000 7f8c7aae6640 1 -- start start 2024-04-25T00:40:56.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c7aae6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8c740d2c60 0x7f8c74168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c7aae6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c740cfc10 con 0x7f8c740cf690 2024-04-25T00:40:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c7aae6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c740cfef0 con 0x7f8c7415e200 2024-04-25T00:40:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c7aae6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8c740b38d0 con 0x7f8c740d2c60 2024-04-25T00:40:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c73fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8c740cf690 0x7f8c740d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37498/0 (socket says 172.21.15.29:37498) 2024-04-25T00:40:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c73fff640 1 -- 172.21.15.29:0/1813132215 learned_addr learned my addr 172.21.15.29:0/1813132215 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:56.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c79083640 1 --2- 172.21.15.29:0/1813132215 >> v2:172.21.15.157:3300/0 conn(0x7f8c740d2c60 0x7f8c74168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2350386203 0 0) 0x7f8c740cfc10 con 0x7f8c740cf690 2024-04-25T00:40:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c64003530 con 0x7f8c740cf690 2024-04-25T00:40:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 369173657 0 0) 0x7f8c64003530 con 0x7f8c740cf690 2024-04-25T00:40:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 >> v2:172.21.15.157:3300/0 conn(0x7f8c740d2c60 msgr2=0x7f8c74168e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 --2- 172.21.15.29:0/1813132215 >> v2:172.21.15.157:3300/0 conn(0x7f8c740d2c60 0x7f8c74168e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 >> v1:172.21.15.157:6789/0 conn(0x7f8c7415e200 legacy=0x7f8c740d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c740b3b20 con 0x7f8c740cf690 2024-04-25T00:40:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8c60001680 con 0x7f8c740cf690 2024-04-25T00:40:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 >> v1:172.21.15.29:6789/0 conn(0x7f8c740cf690 legacy=0x7f8c740d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 --2- 172.21.15.29:0/1813132215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c640039c0 0x7f8c64003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 --2- 172.21.15.29:0/1813132215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c64004810 0x7f8c64004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 --2- 172.21.15.29:0/1813132215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c640051d0 0x7f8c6400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8c740b38d0 con 0x7f8c64004810 2024-04-25T00:40:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6400c070 con 0x7f8c640039c0 2024-04-25T00:40:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8c6400c270 con 0x7f8c640051d0 2024-04-25T00:40:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c73fff640 1 --2- 172.21.15.29:0/1813132215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c64004810 0x7f8c64004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c78882640 1 --2- 172.21.15.29:0/1813132215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c640039c0 0x7f8c64003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c79083640 1 --2- 172.21.15.29:0/1813132215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c640051d0 0x7f8c6400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c73fff640 1 -- 172.21.15.29:0/1813132215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c640051d0 msgr2=0x7f8c6400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c73fff640 1 --2- 172.21.15.29:0/1813132215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c640051d0 0x7f8c6400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c73fff640 1 -- 172.21.15.29:0/1813132215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c640039c0 msgr2=0x7f8c64003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c73fff640 1 --2- 172.21.15.29:0/1813132215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c640039c0 0x7f8c64003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c79083640 1 -- 172.21.15.29:0/1813132215 reap_dead start 2024-04-25T00:40:56.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.412+0000 7f8c73fff640 1 -- 172.21.15.29:0/1813132215 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c64003530 con 0x7f8c64004810 2024-04-25T00:40:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c73fff640 1 --2- 172.21.15.29:0/1813132215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c64004810 0x7f8c64004c90 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000c210 tx=0x7f8c6001f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b82f1327546edf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60007d40 con 0x7f8c64004810 2024-04-25T00:40:56.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c60005b00 con 0x7f8c64004810 2024-04-25T00:40:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c737fe640 1 -- 172.21.15.29:0/1813132215 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60007070 con 0x7f8c64004810 2024-04-25T00:40:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 -- 172.21.15.29:0/1813132215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c64004810 msgr2=0x7f8c64004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 --2- 172.21.15.29:0/1813132215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c64004810 0x7f8c64004c90 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000c210 tx=0x7f8c6001f370 comp rx=0 tx=0).stop 2024-04-25T00:40:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 -- 172.21.15.29:0/1813132215 shutdown_connections 2024-04-25T00:40:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 --2- 172.21.15.29:0/1813132215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c64004810 0x7f8c64004c90 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 -- 172.21.15.29:0/1813132215 >> 172.21.15.29:0/1813132215 conn(0x7f8c7409d280 msgr2=0x7f8c740d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 -- 172.21.15.29:0/1813132215 shutdown_connections 2024-04-25T00:40:56.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 -- 172.21.15.29:0/1813132215 wait complete. 2024-04-25T00:40:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 Processor -- start 2024-04-25T00:40:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 -- start start 2024-04-25T00:40:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c64004810 0x7f8c741534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c740b6b70 0x7f8c74160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c74160c40 0x7f8c740c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7415e760 con 0x7f8c74160c40 2024-04-25T00:40:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7415e5e0 con 0x7f8c64004810 2024-04-25T00:40:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8c740d2550 con 0x7f8c740b6b70 2024-04-25T00:40:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c79083640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c74160c40 0x7f8c740c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c79083640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c74160c40 0x7f8c740c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36462/0 (socket says 172.21.15.29:36462) 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c79083640 1 -- 172.21.15.29:0/4168729698 learned_addr learned my addr 172.21.15.29:0/4168729698 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c73fff640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c740b6b70 0x7f8c74160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c78882640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c64004810 0x7f8c741534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c79083640 1 -- 172.21.15.29:0/4168729698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c740b6b70 msgr2=0x7f8c74160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c79083640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c740b6b70 0x7f8c74160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c79083640 1 -- 172.21.15.29:0/4168729698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c64004810 msgr2=0x7f8c741534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c79083640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c64004810 0x7f8c741534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c79083640 1 -- 172.21.15.29:0/4168729698 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7415e8e0 con 0x7f8c74160c40 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c79083640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c74160c40 0x7f8c740c8b70 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f8c740b6a70 tx=0x7f8c6c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c717fa640 1 -- 172.21.15.29:0/4168729698 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c011db0 con 0x7f8c74160c40 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c717fa640 1 -- 172.21.15.29:0/4168729698 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c01b070 con 0x7f8c74160c40 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 -- 172.21.15.29:0/4168729698 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c740c9880 con 0x7f8c74160c40 2024-04-25T00:40:56.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c717fa640 1 -- 172.21.15.29:0/4168729698 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0039a0 con 0x7f8c74160c40 2024-04-25T00:40:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.416+0000 7f8c7aae6640 1 -- 172.21.15.29:0/4168729698 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c740c9e40 con 0x7f8c74160c40 2024-04-25T00:40:56.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.420+0000 7f8c7aae6640 1 -- 172.21.15.29:0/4168729698 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c74069270 con 0x7f8c74160c40 2024-04-25T00:40:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.420+0000 7f8c717fa640 1 -- 172.21.15.29:0/4168729698 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f8c6c003b40 con 0x7f8c74160c40 2024-04-25T00:40:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.420+0000 7f8c717fa640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8c58068250 0x7f8c5806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.420+0000 7f8c717fa640 1 -- 172.21.15.29:0/4168729698 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f8c6c084b10 con 0x7f8c74160c40 2024-04-25T00:40:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.420+0000 7f8c78882640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8c58068250 0x7f8c5806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.420+0000 7f8c78882640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8c58068250 0x7f8c5806a710 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8c68000c10 tx=0x7f8c68009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:56.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.424+0000 7f8c717fa640 1 -- 172.21.15.29:0/4168729698 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8c6c057250 con 0x7f8c74160c40 2024-04-25T00:40:56.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c7aae6640 1 -- 172.21.15.29:0/4168729698 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8c740b46a0 con 0x7f8c74160c40 2024-04-25T00:40:56.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c717fa640 1 -- 172.21.15.29:0/4168729698 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) ==== 100+0+11 (secure 0 0 0) 0x7f8c6c056b50 con 0x7f8c74160c40 2024-04-25T00:40:56.580 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:40:56.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c7aae6640 1 -- 172.21.15.29:0/4168729698 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8c58068250 msgr2=0x7f8c5806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c7aae6640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8c58068250 0x7f8c5806a710 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8c68000c10 tx=0x7f8c68009040 comp rx=0 tx=0).stop 2024-04-25T00:40:56.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c7aae6640 1 -- 172.21.15.29:0/4168729698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c74160c40 msgr2=0x7f8c740c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c7aae6640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c74160c40 0x7f8c740c8b70 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f8c740b6a70 tx=0x7f8c6c0115a0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c7aae6640 1 -- 172.21.15.29:0/4168729698 shutdown_connections 2024-04-25T00:40:56.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c7aae6640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8c58068250 0x7f8c5806a710 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c7aae6640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c74160c40 0x7f8c740c8b70 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c7aae6640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c740b6b70 0x7f8c74160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c7aae6640 1 --2- 172.21.15.29:0/4168729698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c64004810 0x7f8c741534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c7aae6640 1 -- 172.21.15.29:0/4168729698 >> 172.21.15.29:0/4168729698 conn(0x7f8c7409d280 msgr2=0x7f8c7406c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:56.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c7aae6640 1 -- 172.21.15.29:0/4168729698 shutdown_connections 2024-04-25T00:40:56.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.580+0000 7f8c7aae6640 1 -- 172.21.15.29:0/4168729698 wait complete. 2024-04-25T00:40:56.598 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:40:56.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f280251e640 1 Processor -- start 2024-04-25T00:40:56.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f280251e640 1 -- start start 2024-04-25T00:40:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f280251e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27fc0b3140 0x7f27fc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f280251e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27fc0b3c30 0x7f27fc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f280251e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27fc0b41e0 con 0x7f27fc0b4b10 2024-04-25T00:40:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f280251e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f27fc06d960 con 0x7f27fc0b3140 2024-04-25T00:40:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f280251e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f27fc06dc40 con 0x7f27fc0b3c30 2024-04-25T00:40:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f27fbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27fc0b3140 0x7f27fc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f27fbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27fc0b3140 0x7f27fc06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36464/0 (socket says 172.21.15.29:36464) 2024-04-25T00:40:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f27fbfff640 1 -- 172.21.15.29:0/3003915683 learned_addr learned my addr 172.21.15.29:0/3003915683 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f27fb7fe640 1 --1- 172.21.15.29:0/3003915683 >> v1:172.21.15.29:6789/0 conn(0x7f27fc0b4b10 0x7f27fc0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37504/0 (socket says 172.21.15.29:37504) 2024-04-25T00:40:56.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f2800abb640 1 --2- 172.21.15.29:0/3003915683 >> v2:172.21.15.157:3300/0 conn(0x7f27fc0b3c30 0x7f27fc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f27fbfff640 1 -- 172.21.15.29:0/3003915683 >> v2:172.21.15.157:3300/0 conn(0x7f27fc0b3c30 msgr2=0x7f27fc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:56.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f27fbfff640 1 --2- 172.21.15.29:0/3003915683 >> v2:172.21.15.157:3300/0 conn(0x7f27fc0b3c30 0x7f27fc0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f27fbfff640 1 -- 172.21.15.29:0/3003915683 >> v1:172.21.15.29:6789/0 conn(0x7f27fc0b4b10 legacy=0x7f27fc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f27fbfff640 1 -- 172.21.15.29:0/3003915683 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc06df20 con 0x7f27fc0b3140 2024-04-25T00:40:56.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.720+0000 7f27faffd640 1 -- 172.21.15.29:0/3003915683 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394896888 0 0) 0x7f27fc0b41e0 con 0x7f27fc0b4b10 2024-04-25T00:40:56.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f27fbfff640 1 --2- 172.21.15.29:0/3003915683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27fc0b3140 0x7f27fc06e440 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009870 tx=0x7f27ec01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd1da1ba145d0bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:56.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f27faffd640 1 -- 172.21.15.29:0/3003915683 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0b0070 con 0x7f27fc0b3140 2024-04-25T00:40:56.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f27faffd640 1 -- 172.21.15.29:0/3003915683 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27ec0ab600 con 0x7f27fc0b3140 2024-04-25T00:40:56.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f27faffd640 1 -- 172.21.15.29:0/3003915683 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0ab9d0 con 0x7f27fc0b3140 2024-04-25T00:40:56.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 -- 172.21.15.29:0/3003915683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27fc0b3140 msgr2=0x7f27fc06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 --2- 172.21.15.29:0/3003915683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27fc0b3140 0x7f27fc06e440 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009870 tx=0x7f27ec01c520 comp rx=0 tx=0).stop 2024-04-25T00:40:56.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 -- 172.21.15.29:0/3003915683 shutdown_connections 2024-04-25T00:40:56.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 --2- 172.21.15.29:0/3003915683 >> v2:172.21.15.157:3300/0 conn(0x7f27fc0b3c30 0x7f27fc0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 --2- 172.21.15.29:0/3003915683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27fc0b3140 0x7f27fc06e440 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 -- 172.21.15.29:0/3003915683 >> 172.21.15.29:0/3003915683 conn(0x7f27fc0b1870 msgr2=0x7f27fc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:56.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 -- 172.21.15.29:0/3003915683 shutdown_connections 2024-04-25T00:40:56.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 -- 172.21.15.29:0/3003915683 wait complete. 2024-04-25T00:40:56.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 Processor -- start 2024-04-25T00:40:56.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 -- start start 2024-04-25T00:40:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27fc0b3140 0x7f27fc136b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27fc0b3c30 0x7f27fc13e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27fc0b4b10 0x7f27fc13e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc166d70 con 0x7f27fc0b3c30 2024-04-25T00:40:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc166bf0 con 0x7f27fc0b3140 2024-04-25T00:40:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f280251e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f27fc166ef0 con 0x7f27fc0b4b10 2024-04-25T00:40:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27fc0b3c30 0x7f27fc13e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27fc0b3c30 0x7f27fc13e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36470/0 (socket says 172.21.15.29:36470) 2024-04-25T00:40:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f27fb7fe640 1 -- 172.21.15.29:0/404306220 learned_addr learned my addr 172.21.15.29:0/404306220 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.724+0000 7f27fbfff640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27fc0b3140 0x7f27fc136b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f2800abb640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27fc0b4b10 0x7f27fc13e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27fb7fe640 1 -- 172.21.15.29:0/404306220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27fc0b4b10 msgr2=0x7f27fc13e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27fb7fe640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27fc0b4b10 0x7f27fc13e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27fb7fe640 1 -- 172.21.15.29:0/404306220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27fc0b3140 msgr2=0x7f27fc136b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27fb7fe640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27fc0b3140 0x7f27fc136b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27fb7fe640 1 -- 172.21.15.29:0/404306220 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc140420 con 0x7f27fc0b3c30 2024-04-25T00:40:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27fbfff640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27fc0b3140 0x7f27fc136b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27fb7fe640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27fc0b3c30 0x7f27fc13e2d0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f27f000c480 tx=0x7f27f000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27f8ff9640 1 -- 172.21.15.29:0/404306220 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0018070 con 0x7f27fc0b3c30 2024-04-25T00:40:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27f8ff9640 1 -- 172.21.15.29:0/404306220 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27f0007500 con 0x7f27fc0b3c30 2024-04-25T00:40:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27f8ff9640 1 -- 172.21.15.29:0/404306220 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0004a30 con 0x7f27fc0b3c30 2024-04-25T00:40:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f280251e640 1 -- 172.21.15.29:0/404306220 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc155a40 con 0x7f27fc0b3c30 2024-04-25T00:40:56.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f280251e640 1 -- 172.21.15.29:0/404306220 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc1409a0 con 0x7f27fc0b3c30 2024-04-25T00:40:56.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f280251e640 1 -- 172.21.15.29:0/404306220 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27fc140ba0 con 0x7f27fc0b3c30 2024-04-25T00:40:56.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27f8ff9640 1 -- 172.21.15.29:0/404306220 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f27f001b070 con 0x7f27fc0b3c30 2024-04-25T00:40:56.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27f8ff9640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27e8068280 0x7f27e806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:56.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27f8ff9640 1 -- 172.21.15.29:0/404306220 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f27f00836d0 con 0x7f27fc0b3c30 2024-04-25T00:40:56.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27fbfff640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27e8068280 0x7f27e806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:56.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.728+0000 7f27fbfff640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27e8068280 0x7f27e806a740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0ad000 tx=0x7f27ec0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:56.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.732+0000 7f27f8ff9640 1 -- 172.21.15.29:0/404306220 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f27fc140ba0 con 0x7f27fc0b3c30 2024-04-25T00:40:56.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.908+0000 7f280251e640 1 -- 172.21.15.29:0/404306220 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f27fc069270 con 0x7f27fc0b3c30 2024-04-25T00:40:56.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.908+0000 7f27f8ff9640 1 -- 172.21.15.29:0/404306220 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v156) ==== 80+0+0 (secure 0 0 0) 0x7f27f0055e10 con 0x7f27fc0b3c30 2024-04-25T00:40:56.909 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:40:56.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.912+0000 7f280251e640 1 -- 172.21.15.29:0/404306220 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27e8068280 msgr2=0x7f27e806a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.912+0000 7f280251e640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27e8068280 0x7f27e806a740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0ad000 tx=0x7f27ec0a6000 comp rx=0 tx=0).stop 2024-04-25T00:40:56.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.912+0000 7f280251e640 1 -- 172.21.15.29:0/404306220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27fc0b3c30 msgr2=0x7f27fc13e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:56.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.912+0000 7f280251e640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27fc0b3c30 0x7f27fc13e2d0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f27f000c480 tx=0x7f27f000c940 comp rx=0 tx=0).stop 2024-04-25T00:40:56.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.912+0000 7f280251e640 1 -- 172.21.15.29:0/404306220 shutdown_connections 2024-04-25T00:40:56.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.912+0000 7f280251e640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f27e8068280 0x7f27e806a740 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.912+0000 7f280251e640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27fc0b4b10 0x7f27fc13e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.912+0000 7f280251e640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27fc0b3c30 0x7f27fc13e2d0 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.912+0000 7f280251e640 1 --2- 172.21.15.29:0/404306220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27fc0b3140 0x7f27fc136b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:56.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.912+0000 7f280251e640 1 -- 172.21.15.29:0/404306220 >> 172.21.15.29:0/404306220 conn(0x7f27fc0b1870 msgr2=0x7f27fc13d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:56.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.912+0000 7f280251e640 1 -- 172.21.15.29:0/404306220 shutdown_connections 2024-04-25T00:40:56.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:56.912+0000 7f280251e640 1 -- 172.21.15.29:0/404306220 wait complete. 2024-04-25T00:40:56.929 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-04-25T00:40:56.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:56.929 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:40:56.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[56788]: fuse finished with error 0 and tester_r 0 2024-04-25T00:40:56.942 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:40:56.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:56.942 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:40:57.112 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:40:57.112 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:40:57.112 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:40:57.119 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:40:57.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.284+0000 7fd796ca3640 1 Processor -- start 2024-04-25T00:40:57.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.284+0000 7fd796ca3640 1 -- start start 2024-04-25T00:40:57.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd796ca3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd79015e200 0x7fd7900d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd796ca3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7900d2c60 0x7fd7900b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd796ca3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7900cfc10 con 0x7fd7900cf690 2024-04-25T00:40:57.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd796ca3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd7900b38d0 con 0x7fd7900d2c60 2024-04-25T00:40:57.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd796ca3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd7900cfef0 con 0x7fd79015e200 2024-04-25T00:40:57.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd795240640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7900d2c60 0x7fd7900b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd794a3f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd79015e200 0x7fd7900d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd795240640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7900d2c60 0x7fd7900b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:42454/0 (socket says 172.21.15.29:42454) 2024-04-25T00:40:57.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd795240640 1 -- 172.21.15.29:0/3402726473 learned_addr learned my addr 172.21.15.29:0/3402726473 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:57.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd78ffff640 1 --1- 172.21.15.29:0/3402726473 >> v1:172.21.15.157:6789/0 conn(0x7fd7900cf690 0x7fd7900d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:33736/0 (socket says 172.21.15.29:33736) 2024-04-25T00:40:57.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd795240640 1 -- 172.21.15.29:0/3402726473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd79015e200 msgr2=0x7fd7900d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:57.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd795240640 1 --2- 172.21.15.29:0/3402726473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd79015e200 0x7fd7900d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd795240640 1 -- 172.21.15.29:0/3402726473 >> v1:172.21.15.157:6789/0 conn(0x7fd7900cf690 legacy=0x7fd7900d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.288+0000 7fd795240640 1 -- 172.21.15.29:0/3402726473 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7900b3b20 con 0x7fd7900d2c60 2024-04-25T00:40:57.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd795240640 1 --2- 172.21.15.29:0/3402726473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7900d2c60 0x7fd7900b4320 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd788009e50 tx=0x7fd78801ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc24f8a50defc0ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:57.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd78f7fe640 1 -- 172.21.15.29:0/3402726473 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7880a58f0 con 0x7fd7900d2c60 2024-04-25T00:40:57.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd78f7fe640 1 -- 172.21.15.29:0/3402726473 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7880a5a90 con 0x7fd7900d2c60 2024-04-25T00:40:57.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd78f7fe640 1 -- 172.21.15.29:0/3402726473 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7880a5e60 con 0x7fd7900d2c60 2024-04-25T00:40:57.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 -- 172.21.15.29:0/3402726473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7900d2c60 msgr2=0x7fd7900b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 --2- 172.21.15.29:0/3402726473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7900d2c60 0x7fd7900b4320 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd788009e50 tx=0x7fd78801ca00 comp rx=0 tx=0).stop 2024-04-25T00:40:57.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 -- 172.21.15.29:0/3402726473 shutdown_connections 2024-04-25T00:40:57.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 --2- 172.21.15.29:0/3402726473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7900d2c60 0x7fd7900b4320 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 --2- 172.21.15.29:0/3402726473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd79015e200 0x7fd7900d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 -- 172.21.15.29:0/3402726473 >> 172.21.15.29:0/3402726473 conn(0x7fd79009d280 msgr2=0x7fd7900d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:57.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 -- 172.21.15.29:0/3402726473 shutdown_connections 2024-04-25T00:40:57.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 -- 172.21.15.29:0/3402726473 wait complete. 2024-04-25T00:40:57.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 Processor -- start 2024-04-25T00:40:57.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 -- start start 2024-04-25T00:40:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7900cf690 0x7fd7900b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7900d2c60 0x7fd7901599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd79015e200 0x7fd7901609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7900b51f0 con 0x7fd79015e200 2024-04-25T00:40:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd7900b5070 con 0x7fd7900d2c60 2024-04-25T00:40:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd796ca3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd7900b5370 con 0x7fd7900cf690 2024-04-25T00:40:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd795240640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd79015e200 0x7fd7901609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd795240640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd79015e200 0x7fd7901609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36476/0 (socket says 172.21.15.29:36476) 2024-04-25T00:40:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd795240640 1 -- 172.21.15.29:0/3663431150 learned_addr learned my addr 172.21.15.29:0/3663431150 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd794a3f640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7900cf690 0x7fd7900b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd78ffff640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7900d2c60 0x7fd7901599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd795240640 1 -- 172.21.15.29:0/3663431150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7900cf690 msgr2=0x7fd7900b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd795240640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7900cf690 0x7fd7900b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd795240640 1 -- 172.21.15.29:0/3663431150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7900d2c60 msgr2=0x7fd7901599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd795240640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7900d2c60 0x7fd7901599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.292+0000 7fd795240640 1 -- 172.21.15.29:0/3663431150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7900ce380 con 0x7fd79015e200 2024-04-25T00:40:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.296+0000 7fd795240640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd79015e200 0x7fd7901609c0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fd7880af040 tx=0x7fd7880197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.296+0000 7fd78d7fa640 1 -- 172.21.15.29:0/3663431150 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7880a58f0 con 0x7fd79015e200 2024-04-25T00:40:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.296+0000 7fd78d7fa640 1 -- 172.21.15.29:0/3663431150 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd788007c50 con 0x7fd79015e200 2024-04-25T00:40:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.296+0000 7fd796ca3640 1 -- 172.21.15.29:0/3663431150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7900b96d0 con 0x7fd79015e200 2024-04-25T00:40:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.296+0000 7fd78d7fa640 1 -- 172.21.15.29:0/3663431150 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd788009510 con 0x7fd79015e200 2024-04-25T00:40:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.296+0000 7fd796ca3640 1 -- 172.21.15.29:0/3663431150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7900b9c90 con 0x7fd79015e200 2024-04-25T00:40:57.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.296+0000 7fd796ca3640 1 -- 172.21.15.29:0/3663431150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd790069270 con 0x7fd79015e200 2024-04-25T00:40:57.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.296+0000 7fd78d7fa640 1 -- 172.21.15.29:0/3663431150 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fd7880c0020 con 0x7fd79015e200 2024-04-25T00:40:57.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.296+0000 7fd78d7fa640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd77c068250 0x7fd77c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.296+0000 7fd78d7fa640 1 -- 172.21.15.29:0/3663431150 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fd78811fb20 con 0x7fd79015e200 2024-04-25T00:40:57.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.296+0000 7fd794a3f640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd77c068250 0x7fd77c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.296+0000 7fd794a3f640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd77c068250 0x7fd77c06a710 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd7900d08c0 tx=0x7fd780008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:57.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.300+0000 7fd78d7fa640 1 -- 172.21.15.29:0/3663431150 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd7880c7020 con 0x7fd79015e200 2024-04-25T00:40:57.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.464+0000 7fd796ca3640 1 -- 172.21.15.29:0/3663431150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd79015d080 con 0x7fd79015e200 2024-04-25T00:40:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.464+0000 7fd78d7fa640 1 -- 172.21.15.29:0/3663431150 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16631 (secure 0 0 0) 0x7fd788016620 con 0x7fd79015e200 2024-04-25T00:40:57.466 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:57.466 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":156,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:40:31.094463+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:40:27.796735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:40:28.430727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:40:57.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.468+0000 7fd796ca3640 1 -- 172.21.15.29:0/3663431150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd77c068250 msgr2=0x7fd77c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.468+0000 7fd796ca3640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd77c068250 0x7fd77c06a710 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd7900d08c0 tx=0x7fd780008040 comp rx=0 tx=0).stop 2024-04-25T00:40:57.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.468+0000 7fd796ca3640 1 -- 172.21.15.29:0/3663431150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd79015e200 msgr2=0x7fd7901609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.468+0000 7fd796ca3640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd79015e200 0x7fd7901609c0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fd7880af040 tx=0x7fd7880197a0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.468+0000 7fd796ca3640 1 -- 172.21.15.29:0/3663431150 shutdown_connections 2024-04-25T00:40:57.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.468+0000 7fd796ca3640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd77c068250 0x7fd77c06a710 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.468+0000 7fd796ca3640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd79015e200 0x7fd7901609c0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.468+0000 7fd796ca3640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7900d2c60 0x7fd7901599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.468+0000 7fd796ca3640 1 --2- 172.21.15.29:0/3663431150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7900cf690 0x7fd7900b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.468+0000 7fd796ca3640 1 -- 172.21.15.29:0/3663431150 >> 172.21.15.29:0/3663431150 conn(0x7fd79009d280 msgr2=0x7fd79006cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:57.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.468+0000 7fd796ca3640 1 -- 172.21.15.29:0/3663431150 shutdown_connections 2024-04-25T00:40:57.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.468+0000 7fd796ca3640 1 -- 172.21.15.29:0/3663431150 wait complete. 2024-04-25T00:40:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:40:27.796735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:40:28.430727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:40:57.486 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:40:57.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b369ec640 1 Processor -- start 2024-04-25T00:40:57.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b369ec640 1 -- start start 2024-04-25T00:40:57.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b369ec640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b300701a0 0x7f5b3015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b369ec640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b3015d2b0 con 0x7f5b3015f930 2024-04-25T00:40:57.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b369ec640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b3015fef0 con 0x7f5b30070580 2024-04-25T00:40:57.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b369ec640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5b301601d0 con 0x7f5b300701a0 2024-04-25T00:40:57.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b34f89640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5b3015f930 0x7f5b3006b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37516/0 (socket says 172.21.15.29:37516) 2024-04-25T00:40:57.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b34f89640 1 -- 172.21.15.29:0/697055898 learned_addr learned my addr 172.21.15.29:0/697055898 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:57.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2ffff640 1 --2- 172.21.15.29:0/697055898 >> v2:172.21.15.157:3300/0 conn(0x7f5b300701a0 0x7f5b3015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1130093308 0 0) 0x7f5b3015d2b0 con 0x7f5b3015f930 2024-04-25T00:40:57.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b18003530 con 0x7f5b3015f930 2024-04-25T00:40:57.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1178990805 0 0) 0x7f5b18003530 con 0x7f5b3015f930 2024-04-25T00:40:57.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 >> v2:172.21.15.157:3300/0 conn(0x7f5b300701a0 msgr2=0x7f5b3015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:57.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 --2- 172.21.15.29:0/697055898 >> v2:172.21.15.157:3300/0 conn(0x7f5b300701a0 0x7f5b3015cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 >> v1:172.21.15.157:6789/0 conn(0x7f5b30070580 legacy=0x7f5b3013f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b301604b0 con 0x7f5b3015f930 2024-04-25T00:40:57.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f5b28001680 con 0x7f5b3015f930 2024-04-25T00:40:57.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 >> v1:172.21.15.29:6789/0 conn(0x7f5b3015f930 legacy=0x7f5b3006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 --2- 172.21.15.29:0/697055898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b180039c0 0x7f5b18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 --2- 172.21.15.29:0/697055898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b18004810 0x7f5b18004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 --2- 172.21.15.29:0/697055898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b180051d0 0x7f5b1800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301601d0 con 0x7f5b180051d0 2024-04-25T00:40:57.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1800c070 con 0x7f5b180039c0 2024-04-25T00:40:57.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5b1800c270 con 0x7f5b18004810 2024-04-25T00:40:57.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b34f89640 1 --2- 172.21.15.29:0/697055898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b180051d0 0x7f5b1800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2ffff640 1 --2- 172.21.15.29:0/697055898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b180039c0 0x7f5b18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b2f7fe640 1 --2- 172.21.15.29:0/697055898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b18004810 0x7f5b18004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b34f89640 1 -- 172.21.15.29:0/697055898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b18004810 msgr2=0x7f5b18004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:57.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b34f89640 1 --2- 172.21.15.29:0/697055898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b18004810 0x7f5b18004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b34f89640 1 -- 172.21.15.29:0/697055898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b180039c0 msgr2=0x7f5b18003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:57.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b34f89640 1 --2- 172.21.15.29:0/697055898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b180039c0 0x7f5b18003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.616+0000 7f5b34f89640 1 -- 172.21.15.29:0/697055898 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b18003530 con 0x7f5b180051d0 2024-04-25T00:40:57.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b34f89640 1 -- 172.21.15.29:0/697055898 reap_dead start 2024-04-25T00:40:57.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b34f89640 1 --2- 172.21.15.29:0/697055898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b180051d0 0x7f5b1800b9d0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800c290 tx=0x7f5b2801cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7160aae8bd685ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:57.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28007d40 con 0x7f5b180051d0 2024-04-25T00:40:57.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b280a9070 con 0x7f5b180051d0 2024-04-25T00:40:57.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b2effd640 1 -- 172.21.15.29:0/697055898 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b280a5dd0 con 0x7f5b180051d0 2024-04-25T00:40:57.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b369ec640 1 -- 172.21.15.29:0/697055898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b180051d0 msgr2=0x7f5b1800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b369ec640 1 --2- 172.21.15.29:0/697055898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b180051d0 0x7f5b1800b9d0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800c290 tx=0x7f5b2801cd00 comp rx=0 tx=0).stop 2024-04-25T00:40:57.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b369ec640 1 -- 172.21.15.29:0/697055898 shutdown_connections 2024-04-25T00:40:57.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b369ec640 1 --2- 172.21.15.29:0/697055898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b180051d0 0x7f5b1800b9d0 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b369ec640 1 -- 172.21.15.29:0/697055898 >> 172.21.15.29:0/697055898 conn(0x7f5b300d0c30 msgr2=0x7f5b300d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:57.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b369ec640 1 -- 172.21.15.29:0/697055898 shutdown_connections 2024-04-25T00:40:57.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b369ec640 1 -- 172.21.15.29:0/697055898 wait complete. 2024-04-25T00:40:57.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b369ec640 1 Processor -- start 2024-04-25T00:40:57.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.620+0000 7f5b369ec640 1 -- start start 2024-04-25T00:40:57.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b369ec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b180051d0 0x7f5b300cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b369ec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b300c5af0 0x7f5b301f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b369ec640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b3006ee80 0x7f5b3006f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b369ec640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3015cd70 con 0x7f5b3006ee80 2024-04-25T00:40:57.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b369ec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30071df0 con 0x7f5b300c5af0 2024-04-25T00:40:57.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b369ec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5b3015cef0 con 0x7f5b180051d0 2024-04-25T00:40:57.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b34f89640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b3006ee80 0x7f5b3006f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b34f89640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b3006ee80 0x7f5b3006f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36504/0 (socket says 172.21.15.29:36504) 2024-04-25T00:40:57.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b34f89640 1 -- 172.21.15.29:0/4048065894 learned_addr learned my addr 172.21.15.29:0/4048065894 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:57.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b2ffff640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b180051d0 0x7f5b300cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b2f7fe640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b300c5af0 0x7f5b301f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b34f89640 1 -- 172.21.15.29:0/4048065894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b180051d0 msgr2=0x7f5b300cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b34f89640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b180051d0 0x7f5b300cfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b34f89640 1 -- 172.21.15.29:0/4048065894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b300c5af0 msgr2=0x7f5b301f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b34f89640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b300c5af0 0x7f5b301f8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b34f89640 1 -- 172.21.15.29:0/4048065894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3015d580 con 0x7f5b3006ee80 2024-04-25T00:40:57.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b2ffff640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b180051d0 0x7f5b300cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:57.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b2f7fe640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b300c5af0 0x7f5b301f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:57.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b34f89640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b3006ee80 0x7f5b3006f2e0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f5b28000c10 tx=0x7f5b18005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:57.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b2cff9640 1 -- 172.21.15.29:0/4048065894 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28001c90 con 0x7f5b3006ee80 2024-04-25T00:40:57.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b2cff9640 1 -- 172.21.15.29:0/4048065894 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b280049f0 con 0x7f5b3006ee80 2024-04-25T00:40:57.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b369ec640 1 -- 172.21.15.29:0/4048065894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b300c5f50 con 0x7f5b3006ee80 2024-04-25T00:40:57.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b2cff9640 1 -- 172.21.15.29:0/4048065894 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28004de0 con 0x7f5b3006ee80 2024-04-25T00:40:57.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b369ec640 1 -- 172.21.15.29:0/4048065894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3015d820 con 0x7f5b3006ee80 2024-04-25T00:40:57.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.624+0000 7f5b369ec640 1 -- 172.21.15.29:0/4048065894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b30069270 con 0x7f5b3006ee80 2024-04-25T00:40:57.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.628+0000 7f5b2cff9640 1 -- 172.21.15.29:0/4048065894 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f5b280b6060 con 0x7f5b3006ee80 2024-04-25T00:40:57.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.628+0000 7f5b2cff9640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5b1c068250 0x7f5b1c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.628+0000 7f5b2cff9640 1 -- 172.21.15.29:0/4048065894 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f5b2811d240 con 0x7f5b3006ee80 2024-04-25T00:40:57.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.628+0000 7f5b2ffff640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5b1c068250 0x7f5b1c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.628+0000 7f5b2ffff640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5b1c068250 0x7f5b1c06a710 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5b20007440 tx=0x7f5b2000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:57.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.632+0000 7f5b2cff9640 1 -- 172.21.15.29:0/4048065894 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5b28016a10 con 0x7f5b3006ee80 2024-04-25T00:40:57.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.792+0000 7f5b369ec640 1 -- 172.21.15.29:0/4048065894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5b300d1890 con 0x7f5b3006ee80 2024-04-25T00:40:57.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.792+0000 7f5b2cff9640 1 -- 172.21.15.29:0/4048065894 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) ==== 93+0+10 (secure 0 0 0) 0x7f5b280ef980 con 0x7f5b3006ee80 2024-04-25T00:40:57.795 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:40:57.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.796+0000 7f5b369ec640 1 -- 172.21.15.29:0/4048065894 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5b1c068250 msgr2=0x7f5b1c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.796+0000 7f5b369ec640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5b1c068250 0x7f5b1c06a710 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5b20007440 tx=0x7f5b2000b040 comp rx=0 tx=0).stop 2024-04-25T00:40:57.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.796+0000 7f5b369ec640 1 -- 172.21.15.29:0/4048065894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b3006ee80 msgr2=0x7f5b3006f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.796+0000 7f5b369ec640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b3006ee80 0x7f5b3006f2e0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f5b28000c10 tx=0x7f5b18005840 comp rx=0 tx=0).stop 2024-04-25T00:40:57.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.796+0000 7f5b369ec640 1 -- 172.21.15.29:0/4048065894 shutdown_connections 2024-04-25T00:40:57.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.796+0000 7f5b369ec640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5b1c068250 0x7f5b1c06a710 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.796+0000 7f5b369ec640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b3006ee80 0x7f5b3006f2e0 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.796+0000 7f5b369ec640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b300c5af0 0x7f5b301f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.796+0000 7f5b369ec640 1 --2- 172.21.15.29:0/4048065894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b180051d0 0x7f5b300cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.796+0000 7f5b369ec640 1 -- 172.21.15.29:0/4048065894 >> 172.21.15.29:0/4048065894 conn(0x7f5b300d0c30 msgr2=0x7f5b3013fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:57.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.796+0000 7f5b369ec640 1 -- 172.21.15.29:0/4048065894 shutdown_connections 2024-04-25T00:40:57.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.796+0000 7f5b369ec640 1 -- 172.21.15.29:0/4048065894 wait complete. 2024-04-25T00:40:57.815 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:40:57.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e483d9640 1 Processor -- start 2024-04-25T00:40:57.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e483d9640 1 -- start start 2024-04-25T00:40:57.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e483d9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8e4015d980 0x7f8e4006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e483d9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8e4006db10 0x7f8e40151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e483d9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e400b3130 con 0x7f8e400b2b70 2024-04-25T00:40:57.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e483d9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8e400cf560 con 0x7f8e4006db10 2024-04-25T00:40:57.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e483d9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8e4006df60 con 0x7f8e4015d980 2024-04-25T00:40:57.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e46976640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8e4006db10 0x7f8e40151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e46976640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8e4006db10 0x7f8e40151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48576/0 (socket says 172.21.15.29:48576) 2024-04-25T00:40:57.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e46976640 1 -- 172.21.15.29:0/2660516909 learned_addr learned my addr 172.21.15.29:0/2660516909 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:57.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e46175640 1 --2- 172.21.15.29:0/2660516909 >> v2:172.21.15.157:3300/0 conn(0x7f8e4015d980 0x7f8e4006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e46976640 1 -- 172.21.15.29:0/2660516909 >> v2:172.21.15.157:3300/0 conn(0x7f8e4015d980 msgr2=0x7f8e4006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:57.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e46976640 1 --2- 172.21.15.29:0/2660516909 >> v2:172.21.15.157:3300/0 conn(0x7f8e4015d980 0x7f8e4006d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e46976640 1 -- 172.21.15.29:0/2660516909 >> v1:172.21.15.29:6789/0 conn(0x7f8e400b2b70 legacy=0x7f8e40150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.940+0000 7f8e46976640 1 -- 172.21.15.29:0/2660516909 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e400b3410 con 0x7f8e4006db10 2024-04-25T00:40:57.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e46976640 1 --2- 172.21.15.29:0/2660516909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e4006db10 0x7f8e40151500 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f8e400d0ca0 tx=0x7f8e3801db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9810b04dff0d4812 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:57.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e45173640 1 -- 172.21.15.29:0/2660516909 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e380a6830 con 0x7f8e4006db10 2024-04-25T00:40:57.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e45173640 1 -- 172.21.15.29:0/2660516909 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e380a69d0 con 0x7f8e4006db10 2024-04-25T00:40:57.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e45173640 1 -- 172.21.15.29:0/2660516909 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e380a6da0 con 0x7f8e4006db10 2024-04-25T00:40:57.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 -- 172.21.15.29:0/2660516909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e4006db10 msgr2=0x7f8e40151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 --2- 172.21.15.29:0/2660516909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e4006db10 0x7f8e40151500 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f8e400d0ca0 tx=0x7f8e3801db00 comp rx=0 tx=0).stop 2024-04-25T00:40:57.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 -- 172.21.15.29:0/2660516909 shutdown_connections 2024-04-25T00:40:57.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 --2- 172.21.15.29:0/2660516909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e4006db10 0x7f8e40151500 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 --2- 172.21.15.29:0/2660516909 >> v2:172.21.15.157:3300/0 conn(0x7f8e4015d980 0x7f8e4006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 -- 172.21.15.29:0/2660516909 >> 172.21.15.29:0/2660516909 conn(0x7f8e400b1870 msgr2=0x7f8e400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:57.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 -- 172.21.15.29:0/2660516909 shutdown_connections 2024-04-25T00:40:57.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 -- 172.21.15.29:0/2660516909 wait complete. 2024-04-25T00:40:57.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 Processor -- start 2024-04-25T00:40:57.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 -- start start 2024-04-25T00:40:57.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e4006db10 0x7f8e40142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e400b2b70 0x7f8e400cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e4015d980 0x7f8e4015bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4015b4b0 con 0x7f8e4006db10 2024-04-25T00:40:57.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4015dd60 con 0x7f8e4015d980 2024-04-25T00:40:57.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e483d9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4015b630 con 0x7f8e400b2b70 2024-04-25T00:40:57.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e46175640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e4006db10 0x7f8e40142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e46175640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e4006db10 0x7f8e40142190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48594/0 (socket says 172.21.15.29:48594) 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.944+0000 7f8e46175640 1 -- 172.21.15.29:0/552829473 learned_addr learned my addr 172.21.15.29:0/552829473 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e46976640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e4015d980 0x7f8e4015bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e45974640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e400b2b70 0x7f8e400cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e46175640 1 -- 172.21.15.29:0/552829473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e400b2b70 msgr2=0x7f8e400cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e46175640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e400b2b70 0x7f8e400cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e46175640 1 -- 172.21.15.29:0/552829473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e4015d980 msgr2=0x7f8e4015bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e46175640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e4015d980 0x7f8e4015bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e46175640 1 -- 172.21.15.29:0/552829473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e40155310 con 0x7f8e4006db10 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e46976640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e4015d980 0x7f8e4015bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e45974640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e400b2b70 0x7f8e400cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e46175640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e4006db10 0x7f8e40142190 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000d620 tx=0x7f8e3000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e36ffd640 1 -- 172.21.15.29:0/552829473 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3000b620 con 0x7f8e4006db10 2024-04-25T00:40:57.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e36ffd640 1 -- 172.21.15.29:0/552829473 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e30019070 con 0x7f8e4006db10 2024-04-25T00:40:57.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e483d9640 1 -- 172.21.15.29:0/552829473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e400ccc50 con 0x7f8e4006db10 2024-04-25T00:40:57.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e36ffd640 1 -- 172.21.15.29:0/552829473 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3000b9e0 con 0x7f8e4006db10 2024-04-25T00:40:57.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e483d9640 1 -- 172.21.15.29:0/552829473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e400cd210 con 0x7f8e4006db10 2024-04-25T00:40:57.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e483d9640 1 -- 172.21.15.29:0/552829473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e40069270 con 0x7f8e4006db10 2024-04-25T00:40:57.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e36ffd640 1 -- 172.21.15.29:0/552829473 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f8e3000bb80 con 0x7f8e4006db10 2024-04-25T00:40:57.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e36ffd640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8e24068280 0x7f8e2406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:57.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e45974640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8e24068280 0x7f8e2406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:57.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e36ffd640 1 -- 172.21.15.29:0/552829473 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f8e30084790 con 0x7f8e4006db10 2024-04-25T00:40:57.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.948+0000 7f8e45974640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8e24068280 0x7f8e2406a740 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8e4015c980 tx=0x7f8e3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:57.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:57.952+0000 7f8e36ffd640 1 -- 172.21.15.29:0/552829473 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8e30056ed0 con 0x7f8e4006db10 2024-04-25T00:40:58.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.116+0000 7f8e483d9640 1 -- 172.21.15.29:0/552829473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8e400cf910 con 0x7f8e4006db10 2024-04-25T00:40:58.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.116+0000 7f8e36ffd640 1 -- 172.21.15.29:0/552829473 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) ==== 104+0+11 (secure 0 0 0) 0x7f8e300567d0 con 0x7f8e4006db10 2024-04-25T00:40:58.117 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:40:58.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.116+0000 7f8e483d9640 1 -- 172.21.15.29:0/552829473 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8e24068280 msgr2=0x7f8e2406a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.116+0000 7f8e483d9640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8e24068280 0x7f8e2406a740 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8e4015c980 tx=0x7f8e3c009040 comp rx=0 tx=0).stop 2024-04-25T00:40:58.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.116+0000 7f8e483d9640 1 -- 172.21.15.29:0/552829473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e4006db10 msgr2=0x7f8e40142190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.116+0000 7f8e483d9640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e4006db10 0x7f8e40142190 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000d620 tx=0x7f8e3000dae0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.116+0000 7f8e483d9640 1 -- 172.21.15.29:0/552829473 shutdown_connections 2024-04-25T00:40:58.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.116+0000 7f8e483d9640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8e24068280 0x7f8e2406a740 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.116+0000 7f8e483d9640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e4015d980 0x7f8e4015bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.116+0000 7f8e483d9640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e400b2b70 0x7f8e400cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.120+0000 7f8e483d9640 1 --2- 172.21.15.29:0/552829473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e4006db10 0x7f8e40142190 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.120+0000 7f8e483d9640 1 -- 172.21.15.29:0/552829473 >> 172.21.15.29:0/552829473 conn(0x7f8e400b1870 msgr2=0x7f8e40151200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:58.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.120+0000 7f8e483d9640 1 -- 172.21.15.29:0/552829473 shutdown_connections 2024-04-25T00:40:58.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.120+0000 7f8e483d9640 1 -- 172.21.15.29:0/552829473 wait complete. 2024-04-25T00:40:58.136 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:40:58.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8e058da640 1 Processor -- start 2024-04-25T00:40:58.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8e058da640 1 -- start start 2024-04-25T00:40:58.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8e058da640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8e0006b210 0x7f8e00072990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:58.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8e058da640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8e0015ecf0 0x7f8e000ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:58.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8e058da640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e0015f2a0 con 0x7f8e0006b5f0 2024-04-25T00:40:58.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8e058da640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8e000b03f0 con 0x7f8e0006b210 2024-04-25T00:40:58.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8e058da640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8e0015f580 con 0x7f8e0015ecf0 2024-04-25T00:40:58.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8dfeffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8e0006b210 0x7f8e00072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:58.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8dfeffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8e0006b210 0x7f8e00072990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48608/0 (socket says 172.21.15.29:48608) 2024-04-25T00:40:58.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8dfeffd640 1 -- 172.21.15.29:0/1418067444 learned_addr learned my addr 172.21.15.29:0/1418067444 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:58.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8dff7fe640 1 --2- 172.21.15.29:0/1418067444 >> v2:172.21.15.157:3300/0 conn(0x7f8e0015ecf0 0x7f8e000ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:58.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8dfeffd640 1 -- 172.21.15.29:0/1418067444 >> v2:172.21.15.157:3300/0 conn(0x7f8e0015ecf0 msgr2=0x7f8e000ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:58.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8dfeffd640 1 --2- 172.21.15.29:0/1418067444 >> v2:172.21.15.157:3300/0 conn(0x7f8e0015ecf0 0x7f8e000ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8dfeffd640 1 -- 172.21.15.29:0/1418067444 >> v1:172.21.15.29:6789/0 conn(0x7f8e0006b5f0 legacy=0x7f8e0015e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.260+0000 7f8dfeffd640 1 -- 172.21.15.29:0/1418067444 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e00071eb0 con 0x7f8e0006b210 2024-04-25T00:40:58.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8dfeffd640 1 --2- 172.21.15.29:0/1418067444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e0006b210 0x7f8e00072990 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f8e00071d10 tx=0x7f8df401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb655c7c09a403d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:58.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8dfdffb640 1 -- 172.21.15.29:0/1418067444 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df40b1070 con 0x7f8e0006b210 2024-04-25T00:40:58.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8dfdffb640 1 -- 172.21.15.29:0/1418067444 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df40ac640 con 0x7f8e0006b210 2024-04-25T00:40:58.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8dfdffb640 1 -- 172.21.15.29:0/1418067444 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df40aca10 con 0x7f8e0006b210 2024-04-25T00:40:58.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8e058da640 1 -- 172.21.15.29:0/1418067444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e0006b210 msgr2=0x7f8e00072990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8e058da640 1 --2- 172.21.15.29:0/1418067444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e0006b210 0x7f8e00072990 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f8e00071d10 tx=0x7f8df401c400 comp rx=0 tx=0).stop 2024-04-25T00:40:58.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8e058da640 1 -- 172.21.15.29:0/1418067444 shutdown_connections 2024-04-25T00:40:58.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8e058da640 1 --2- 172.21.15.29:0/1418067444 >> v2:172.21.15.157:3300/0 conn(0x7f8e0015ecf0 0x7f8e000ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8e058da640 1 --2- 172.21.15.29:0/1418067444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e0006b210 0x7f8e00072990 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8e058da640 1 -- 172.21.15.29:0/1418067444 >> 172.21.15.29:0/1418067444 conn(0x7f8e000d0c40 msgr2=0x7f8e000d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:58.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8e058da640 1 -- 172.21.15.29:0/1418067444 shutdown_connections 2024-04-25T00:40:58.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8e058da640 1 -- 172.21.15.29:0/1418067444 wait complete. 2024-04-25T00:40:58.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8e058da640 1 Processor -- start 2024-04-25T00:40:58.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.264+0000 7f8e058da640 1 -- start start 2024-04-25T00:40:58.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8e058da640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e0006b210 0x7f8e000b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:58.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8e058da640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006b5f0 0x7f8e0015ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:58.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8e058da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e0015ecf0 0x7f8e0015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:58.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8e058da640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0016af70 con 0x7f8e0006b210 2024-04-25T00:40:58.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8e058da640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8e000cbf80 con 0x7f8e0015ecf0 2024-04-25T00:40:58.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8e058da640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0016b0f0 con 0x7f8e0006b5f0 2024-04-25T00:40:58.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e0006b210 0x7f8e000b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:58.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e0006b210 0x7f8e000b5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48624/0 (socket says 172.21.15.29:48624) 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8dfeffd640 1 -- 172.21.15.29:0/777075585 learned_addr learned my addr 172.21.15.29:0/777075585 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8dff7fe640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e0015ecf0 0x7f8e0015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8dfe7fc640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006b5f0 0x7f8e0015ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8dfeffd640 1 -- 172.21.15.29:0/777075585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006b5f0 msgr2=0x7f8e0015ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8dfeffd640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006b5f0 0x7f8e0015ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8dfeffd640 1 -- 172.21.15.29:0/777075585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e0015ecf0 msgr2=0x7f8e0015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8dfeffd640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e0015ecf0 0x7f8e0015b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8dfeffd640 1 -- 172.21.15.29:0/777075585 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e00158130 con 0x7f8e0006b210 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8dfe7fc640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006b5f0 0x7f8e0015ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8dfeffd640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e0006b210 0x7f8e000b5c80 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f8df40af040 tx=0x7f8df401ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8de7fff640 1 -- 172.21.15.29:0/777075585 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df40bb020 con 0x7f8e0006b210 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8de7fff640 1 -- 172.21.15.29:0/777075585 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df40b7640 con 0x7f8e0006b210 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8e058da640 1 -- 172.21.15.29:0/777075585 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e00158410 con 0x7f8e0006b210 2024-04-25T00:40:58.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8de7fff640 1 -- 172.21.15.29:0/777075585 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df40b1040 con 0x7f8e0006b210 2024-04-25T00:40:58.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8e058da640 1 -- 172.21.15.29:0/777075585 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e000b61c0 con 0x7f8e0006b210 2024-04-25T00:40:58.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8e058da640 1 -- 172.21.15.29:0/777075585 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e00069270 con 0x7f8e0006b210 2024-04-25T00:40:58.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8de7fff640 1 -- 172.21.15.29:0/777075585 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f8df40be020 con 0x7f8e0006b210 2024-04-25T00:40:58.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8de7fff640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8dec068280 0x7f8dec06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:58.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8de7fff640 1 -- 172.21.15.29:0/777075585 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f8df411e690 con 0x7f8e0006b210 2024-04-25T00:40:58.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.268+0000 7f8dfe7fc640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8dec068280 0x7f8dec06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:58.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.272+0000 7f8dfe7fc640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8dec068280 0x7f8dec06a740 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8e001572b0 tx=0x7f8df0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:58.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.276+0000 7f8de7fff640 1 -- 172.21.15.29:0/777075585 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8df40c5030 con 0x7f8e0006b210 2024-04-25T00:40:58.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.432+0000 7f8e058da640 1 -- 172.21.15.29:0/777075585 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8e001585e0 con 0x7f8e0006b210 2024-04-25T00:40:58.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.436+0000 7f8de7fff640 1 -- 172.21.15.29:0/777075585 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) ==== 100+0+11 (secure 0 0 0) 0x7f8df4016110 con 0x7f8e0006b210 2024-04-25T00:40:58.436 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:40:58.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.436+0000 7f8e058da640 1 -- 172.21.15.29:0/777075585 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8dec068280 msgr2=0x7f8dec06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.436+0000 7f8e058da640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8dec068280 0x7f8dec06a740 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8e001572b0 tx=0x7f8df0009040 comp rx=0 tx=0).stop 2024-04-25T00:40:58.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.436+0000 7f8e058da640 1 -- 172.21.15.29:0/777075585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e0006b210 msgr2=0x7f8e000b5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.436+0000 7f8e058da640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e0006b210 0x7f8e000b5c80 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f8df40af040 tx=0x7f8df401ac40 comp rx=0 tx=0).stop 2024-04-25T00:40:58.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.436+0000 7f8e058da640 1 -- 172.21.15.29:0/777075585 shutdown_connections 2024-04-25T00:40:58.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.436+0000 7f8e058da640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8dec068280 0x7f8dec06a740 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.436+0000 7f8e058da640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e0015ecf0 0x7f8e0015b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.436+0000 7f8e058da640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e0006b5f0 0x7f8e0015ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.436+0000 7f8e058da640 1 --2- 172.21.15.29:0/777075585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e0006b210 0x7f8e000b5c80 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.436+0000 7f8e058da640 1 -- 172.21.15.29:0/777075585 >> 172.21.15.29:0/777075585 conn(0x7f8e000d0c40 msgr2=0x7f8e000b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:58.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.436+0000 7f8e058da640 1 -- 172.21.15.29:0/777075585 shutdown_connections 2024-04-25T00:40:58.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.436+0000 7f8e058da640 1 -- 172.21.15.29:0/777075585 wait complete. 2024-04-25T00:40:58.455 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:40:58.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11fcdd640 1 Processor -- start 2024-04-25T00:40:58.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11fcdd640 1 -- start start 2024-04-25T00:40:58.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11fcdd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb11815e060 0x7fb11815c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:58.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11fcdd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb118072490 0x7fb1180cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:58.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11fcdd640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb11815e850 con 0x7fb11815bce0 2024-04-25T00:40:58.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11fcdd640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb11815f700 con 0x7fb11815e060 2024-04-25T00:40:58.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11fcdd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb11815eaf0 con 0x7fb118072490 2024-04-25T00:40:58.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11da79640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb11815e060 0x7fb11815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:58.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11d278640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb11815bce0 0x7fb118071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33900/0 (socket says 172.21.15.29:33900) 2024-04-25T00:40:58.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11da79640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb11815e060 0x7fb11815c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48638/0 (socket says 172.21.15.29:48638) 2024-04-25T00:40:58.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11d278640 1 -- 172.21.15.29:0/675718083 learned_addr learned my addr 172.21.15.29:0/675718083 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:58.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11e27a640 1 --2- 172.21.15.29:0/675718083 >> v2:172.21.15.157:3300/0 conn(0x7fb118072490 0x7fb1180cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:58.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11da79640 1 -- 172.21.15.29:0/675718083 >> v2:172.21.15.157:3300/0 conn(0x7fb118072490 msgr2=0x7fb1180cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:58.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11da79640 1 --2- 172.21.15.29:0/675718083 >> v2:172.21.15.157:3300/0 conn(0x7fb118072490 0x7fb1180cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11da79640 1 -- 172.21.15.29:0/675718083 >> v1:172.21.15.29:6789/0 conn(0x7fb11815bce0 legacy=0x7fb118071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.584+0000 7fb11da79640 1 -- 172.21.15.29:0/675718083 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11815edd0 con 0x7fb11815e060 2024-04-25T00:40:58.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11da79640 1 --2- 172.21.15.29:0/675718083 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb11815e060 0x7fb11815c420 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fb11815f880 tx=0x7fb11001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379d2f3b829bd60b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:58.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11ca77640 1 -- 172.21.15.29:0/675718083 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1100b2070 con 0x7fb11815e060 2024-04-25T00:40:58.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11ca77640 1 -- 172.21.15.29:0/675718083 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1100a57c0 con 0x7fb11815e060 2024-04-25T00:40:58.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11ca77640 1 -- 172.21.15.29:0/675718083 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1100a5b90 con 0x7fb11815e060 2024-04-25T00:40:58.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11fcdd640 1 -- 172.21.15.29:0/675718083 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb11815e060 msgr2=0x7fb11815c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11fcdd640 1 --2- 172.21.15.29:0/675718083 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb11815e060 0x7fb11815c420 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fb11815f880 tx=0x7fb11001c900 comp rx=0 tx=0).stop 2024-04-25T00:40:58.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11fcdd640 1 -- 172.21.15.29:0/675718083 shutdown_connections 2024-04-25T00:40:58.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11fcdd640 1 --2- 172.21.15.29:0/675718083 >> v2:172.21.15.157:3300/0 conn(0x7fb118072490 0x7fb1180cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11fcdd640 1 --2- 172.21.15.29:0/675718083 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb11815e060 0x7fb11815c420 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11fcdd640 1 -- 172.21.15.29:0/675718083 >> 172.21.15.29:0/675718083 conn(0x7fb11806c320 msgr2=0x7fb11806c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:58.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11fcdd640 1 -- 172.21.15.29:0/675718083 shutdown_connections 2024-04-25T00:40:58.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11fcdd640 1 -- 172.21.15.29:0/675718083 wait complete. 2024-04-25T00:40:58.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11fcdd640 1 Processor -- start 2024-04-25T00:40:58.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.588+0000 7fb11fcdd640 1 -- start start 2024-04-25T00:40:58.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11fcdd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb118072490 0x7fb1180c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:58.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11fcdd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb11815bce0 0x7fb11814ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:58.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11fcdd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb11815e060 0x7fb1180d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:58.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11fcdd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb11815e5c0 con 0x7fb11815e060 2024-04-25T00:40:58.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11fcdd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb11815e440 con 0x7fb118072490 2024-04-25T00:40:58.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11fcdd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb11806f610 con 0x7fb11815bce0 2024-04-25T00:40:58.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11e27a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb11815e060 0x7fb1180d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:58.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11e27a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb11815e060 0x7fb1180d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48646/0 (socket says 172.21.15.29:48646) 2024-04-25T00:40:58.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11e27a640 1 -- 172.21.15.29:0/659308748 learned_addr learned my addr 172.21.15.29:0/659308748 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:58.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11da79640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb118072490 0x7fb1180c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11d278640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb11815bce0 0x7fb11814ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11e27a640 1 -- 172.21.15.29:0/659308748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb11815bce0 msgr2=0x7fb11814ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11e27a640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb11815bce0 0x7fb11814ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11e27a640 1 -- 172.21.15.29:0/659308748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb118072490 msgr2=0x7fb1180c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11e27a640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb118072490 0x7fb1180c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11e27a640 1 -- 172.21.15.29:0/659308748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1180c7ee0 con 0x7fb11815e060 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11da79640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb118072490 0x7fb1180c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11d278640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb11815bce0 0x7fb11814ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11e27a640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb11815e060 0x7fb1180d09e0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fb11400d500 tx=0x7fb11400d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb10a7fc640 1 -- 172.21.15.29:0/659308748 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1140125f0 con 0x7fb11815e060 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb10a7fc640 1 -- 172.21.15.29:0/659308748 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb114019070 con 0x7fb11815e060 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb10a7fc640 1 -- 172.21.15.29:0/659308748 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1140129d0 con 0x7fb11815e060 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11fcdd640 1 -- 172.21.15.29:0/659308748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1180c81c0 con 0x7fb11815e060 2024-04-25T00:40:58.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11fcdd640 1 -- 172.21.15.29:0/659308748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1180c8780 con 0x7fb11815e060 2024-04-25T00:40:58.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb11fcdd640 1 -- 172.21.15.29:0/659308748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb118069270 con 0x7fb11815e060 2024-04-25T00:40:58.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb10a7fc640 1 -- 172.21.15.29:0/659308748 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fb114012b70 con 0x7fb11815e060 2024-04-25T00:40:58.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.592+0000 7fb10a7fc640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb0fc068250 0x7fb0fc06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:58.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.596+0000 7fb10a7fc640 1 -- 172.21.15.29:0/659308748 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fb114084570 con 0x7fb11815e060 2024-04-25T00:40:58.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.596+0000 7fb11da79640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb0fc068250 0x7fb0fc06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:58.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.596+0000 7fb11da79640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb0fc068250 0x7fb0fc06a710 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb1100af000 tx=0x7fb1100a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:58.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.600+0000 7fb10a7fc640 1 -- 172.21.15.29:0/659308748 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb114023020 con 0x7fb11815e060 2024-04-25T00:40:58.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.768+0000 7fb11fcdd640 1 -- 172.21.15.29:0/659308748 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb1180c6f40 con 0x7fb11815e060 2024-04-25T00:40:58.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.772+0000 7fb10a7fc640 1 -- 172.21.15.29:0/659308748 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v156) ==== 80+0+0 (secure 0 0 0) 0x7fb114056c30 con 0x7fb11815e060 2024-04-25T00:40:58.772 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:40:58.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.772+0000 7fb11fcdd640 1 -- 172.21.15.29:0/659308748 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb0fc068250 msgr2=0x7fb0fc06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.772+0000 7fb11fcdd640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb0fc068250 0x7fb0fc06a710 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb1100af000 tx=0x7fb1100a7070 comp rx=0 tx=0).stop 2024-04-25T00:40:58.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.772+0000 7fb11fcdd640 1 -- 172.21.15.29:0/659308748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb11815e060 msgr2=0x7fb1180d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:58.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.772+0000 7fb11fcdd640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb11815e060 0x7fb1180d09e0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fb11400d500 tx=0x7fb11400d9c0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.772+0000 7fb11fcdd640 1 -- 172.21.15.29:0/659308748 shutdown_connections 2024-04-25T00:40:58.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.772+0000 7fb11fcdd640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb0fc068250 0x7fb0fc06a710 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.772+0000 7fb11fcdd640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb11815e060 0x7fb1180d09e0 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.772+0000 7fb11fcdd640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb11815bce0 0x7fb11814ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.772+0000 7fb11fcdd640 1 --2- 172.21.15.29:0/659308748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb118072490 0x7fb1180c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:58.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.772+0000 7fb11fcdd640 1 -- 172.21.15.29:0/659308748 >> 172.21.15.29:0/659308748 conn(0x7fb11806c320 msgr2=0x7fb118152e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:58.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.772+0000 7fb11fcdd640 1 -- 172.21.15.29:0/659308748 shutdown_connections 2024-04-25T00:40:58.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:58.772+0000 7fb11fcdd640 1 -- 172.21.15.29:0/659308748 wait complete. 2024-04-25T00:40:58.792 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-25T00:40:58.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:58.792 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T00:40:58.805 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[35219]: fuse finished with error 0 and tester_r 0 2024-04-25T00:40:58.807 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:40:58.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:58.807 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:40:58.882 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:40:58.882 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:40:58.882 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:40:58.888 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:40:58.889 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:40:58.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:58.889 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:40:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T00:40:58.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:40:58.937 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:40:58.937 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:40:58.937 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:40:58.942 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:40:58.942 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:40:58.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:40:58.942 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:40:58.990 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T00:40:58.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:40:58.991 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:40:59.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4c5537640 1 Processor -- start 2024-04-25T00:40:59.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4c5537640 1 -- start start 2024-04-25T00:40:59.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4c5537640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c00b3140 0x7fc4c006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4c5537640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4c00b3c30 0x7fc4c00ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4c5537640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4c00b41e0 con 0x7fc4c00b4b10 2024-04-25T00:40:59.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4c5537640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c006d960 con 0x7fc4c00b3c30 2024-04-25T00:40:59.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4c5537640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c006dc40 con 0x7fc4c00b3140 2024-04-25T00:40:59.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4be7fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc4c00b4b10 0x7fc4c00b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33910/0 (socket says 172.21.15.29:33910) 2024-04-25T00:40:59.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4be7fc640 1 -- 172.21.15.29:0/384617121 learned_addr learned my addr 172.21.15.29:0/384617121 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:59.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4beffd640 1 --2- 172.21.15.29:0/384617121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c00b3140 0x7fc4c006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bf7fe640 1 --2- 172.21.15.29:0/384617121 >> v2:172.21.15.157:3300/0 conn(0x7fc4c00b3c30 0x7fc4c00ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 94910341 0 0) 0x7fc4c00b41e0 con 0x7fc4c00b4b10 2024-04-25T00:40:59.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4a8003530 con 0x7fc4c00b4b10 2024-04-25T00:40:59.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 52234216 0 0) 0x7fc4a8003530 con 0x7fc4c00b4b10 2024-04-25T00:40:59.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c00b3140 msgr2=0x7fc4c006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:59.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 --2- 172.21.15.29:0/384617121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c00b3140 0x7fc4c006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 >> v2:172.21.15.157:3300/0 conn(0x7fc4c00b3c30 msgr2=0x7fc4c00ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:59.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 --2- 172.21.15.29:0/384617121 >> v2:172.21.15.157:3300/0 conn(0x7fc4c00b3c30 0x7fc4c00ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c006df20 con 0x7fc4c00b4b10 2024-04-25T00:40:59.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fc4b4001680 con 0x7fc4c00b4b10 2024-04-25T00:40:59.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 >> v1:172.21.15.29:6789/0 conn(0x7fc4c00b4b10 legacy=0x7fc4c00b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 --2- 172.21.15.29:0/384617121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80039c0 0x7fc4a8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 --2- 172.21.15.29:0/384617121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c00b3140 0x7fc4a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 --2- 172.21.15.29:0/384617121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a8004d50 0x7fc4a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c006d960 con 0x7fc4a80039c0 2024-04-25T00:40:59.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c006dc40 con 0x7fc4a8004d50 2024-04-25T00:40:59.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a800b9d0 con 0x7fc4c00b3140 2024-04-25T00:40:59.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4beffd640 1 --2- 172.21.15.29:0/384617121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80039c0 0x7fc4a8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4be7fc640 1 --2- 172.21.15.29:0/384617121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c00b3140 0x7fc4a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bf7fe640 1 --2- 172.21.15.29:0/384617121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a8004d50 0x7fc4a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4beffd640 1 -- 172.21.15.29:0/384617121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c00b3140 msgr2=0x7fc4a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:59.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4beffd640 1 --2- 172.21.15.29:0/384617121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c00b3140 0x7fc4a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4beffd640 1 -- 172.21.15.29:0/384617121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a8004d50 msgr2=0x7fc4a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:59.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4beffd640 1 --2- 172.21.15.29:0/384617121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a8004d50 0x7fc4a80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4beffd640 1 -- 172.21.15.29:0/384617121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a8003530 con 0x7fc4a80039c0 2024-04-25T00:40:59.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.124+0000 7fc4bf7fe640 1 --2- 172.21.15.29:0/384617121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a8004d50 0x7fc4a80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:59.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.128+0000 7fc4beffd640 1 --2- 172.21.15.29:0/384617121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80039c0 0x7fc4a8003de0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009ae0 tx=0x7fc4b001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91709f312c0677e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:59.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.128+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00096b0 con 0x7fc4a80039c0 2024-04-25T00:40:59.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.128+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b00a6450 con 0x7fc4a80039c0 2024-04-25T00:40:59.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.128+0000 7fc4bdffb640 1 -- 172.21.15.29:0/384617121 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00a6840 con 0x7fc4a80039c0 2024-04-25T00:40:59.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.128+0000 7fc4c5537640 1 -- 172.21.15.29:0/384617121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80039c0 msgr2=0x7fc4a8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.128+0000 7fc4c5537640 1 --2- 172.21.15.29:0/384617121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80039c0 0x7fc4a8003de0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009ae0 tx=0x7fc4b001ca70 comp rx=0 tx=0).stop 2024-04-25T00:40:59.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.128+0000 7fc4bf7fe640 1 -- 172.21.15.29:0/384617121 reap_dead start 2024-04-25T00:40:59.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.128+0000 7fc4c5537640 1 -- 172.21.15.29:0/384617121 shutdown_connections 2024-04-25T00:40:59.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.128+0000 7fc4c5537640 1 -- 172.21.15.29:0/384617121 >> 172.21.15.29:0/384617121 conn(0x7fc4c00b1870 msgr2=0x7fc4c00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:59.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.128+0000 7fc4c5537640 1 -- 172.21.15.29:0/384617121 shutdown_connections 2024-04-25T00:40:59.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.128+0000 7fc4c5537640 1 -- 172.21.15.29:0/384617121 wait complete. 2024-04-25T00:40:59.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.128+0000 7fc4c5537640 1 Processor -- start 2024-04-25T00:40:59.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.128+0000 7fc4c5537640 1 -- start start 2024-04-25T00:40:59.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4c5537640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80039c0 0x7fc4c0136bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4c5537640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c0072770 0x7fc4c015b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4c5537640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c013c8f0 0x7fc4c013cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4c5537640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0166d70 con 0x7fc4a80039c0 2024-04-25T00:40:59.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4c5537640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0166bf0 con 0x7fc4c013c8f0 2024-04-25T00:40:59.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4c5537640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c0166ef0 con 0x7fc4c0072770 2024-04-25T00:40:59.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4beffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80039c0 0x7fc4c0136bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4beffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80039c0 0x7fc4c0136bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48668/0 (socket says 172.21.15.29:48668) 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4beffd640 1 -- 172.21.15.29:0/1760912105 learned_addr learned my addr 172.21.15.29:0/1760912105 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4be7fc640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c0072770 0x7fc4c015b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4bf7fe640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c013c8f0 0x7fc4c013cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4beffd640 1 -- 172.21.15.29:0/1760912105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c0072770 msgr2=0x7fc4c015b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4beffd640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c0072770 0x7fc4c015b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4beffd640 1 -- 172.21.15.29:0/1760912105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c013c8f0 msgr2=0x7fc4c013cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4beffd640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c013c8f0 0x7fc4c013cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4beffd640 1 -- 172.21.15.29:0/1760912105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c0137240 con 0x7fc4a80039c0 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4beffd640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80039c0 0x7fc4c0136bf0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0098260 tx=0x7fc4a8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4a77fe640 1 -- 172.21.15.29:0/1760912105 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00a6600 con 0x7fc4a80039c0 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4a77fe640 1 -- 172.21.15.29:0/1760912105 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b000f000 con 0x7fc4a80039c0 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4c5537640 1 -- 172.21.15.29:0/1760912105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c0137520 con 0x7fc4a80039c0 2024-04-25T00:40:59.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4a77fe640 1 -- 172.21.15.29:0/1760912105 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0004020 con 0x7fc4a80039c0 2024-04-25T00:40:59.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4c5537640 1 -- 172.21.15.29:0/1760912105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c0137ae0 con 0x7fc4a80039c0 2024-04-25T00:40:59.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4c5537640 1 -- 172.21.15.29:0/1760912105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c0069270 con 0x7fc4a80039c0 2024-04-25T00:40:59.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4a77fe640 1 -- 172.21.15.29:0/1760912105 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fc4b00c2060 con 0x7fc4a80039c0 2024-04-25T00:40:59.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4a77fe640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc4ac068280 0x7fc4ac06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4a77fe640 1 -- 172.21.15.29:0/1760912105 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fc4b011e150 con 0x7fc4a80039c0 2024-04-25T00:40:59.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.132+0000 7fc4be7fc640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc4ac068280 0x7fc4ac06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.136+0000 7fc4be7fc640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc4ac068280 0x7fc4ac06a740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc4c01fa7d0 tx=0x7fc4b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:59.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.140+0000 7fc4a77fe640 1 -- 172.21.15.29:0/1760912105 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc4b00ac020 con 0x7fc4a80039c0 2024-04-25T00:40:59.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.332+0000 7fc4c5537640 1 -- 172.21.15.29:0/1760912105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4c013f2a0 con 0x7fc4a80039c0 2024-04-25T00:40:59.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.332+0000 7fc4a77fe640 1 -- 172.21.15.29:0/1760912105 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5196 (secure 0 0 0) 0x7fc4b0016780 con 0x7fc4a80039c0 2024-04-25T00:40:59.335 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:59.335 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2677876688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2677876688},{"type":"v1","addr":"172.21.15.157:6833","nonce":2677876688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3340901266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3340901266},{"type":"v1","addr":"172.21.15.157:6835","nonce":3340901266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":10092,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/567284262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":567284262},{"type":"v1","addr":"172.21.15.29:6833","nonce":567284262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:40:29.419450+0000","modified":"2024-04-25T00:40:32.484311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10095":{"gid":10095,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/4183931647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":4183931647},{"type":"v1","addr":"172.21.15.29:6835","nonce":4183931647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10095,"qdb_cluster":[10095]},"id":12}]} 2024-04-25T00:40:59.335 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 103 2024-04-25T00:40:59.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.336+0000 7fc4c5537640 1 -- 172.21.15.29:0/1760912105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc4ac068280 msgr2=0x7fc4ac06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.336+0000 7fc4c5537640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc4ac068280 0x7fc4ac06a740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc4c01fa7d0 tx=0x7fc4b4009040 comp rx=0 tx=0).stop 2024-04-25T00:40:59.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.336+0000 7fc4c5537640 1 -- 172.21.15.29:0/1760912105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80039c0 msgr2=0x7fc4c0136bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.336+0000 7fc4c5537640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80039c0 0x7fc4c0136bf0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0098260 tx=0x7fc4a8005840 comp rx=0 tx=0).stop 2024-04-25T00:40:59.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.336+0000 7fc4c5537640 1 -- 172.21.15.29:0/1760912105 shutdown_connections 2024-04-25T00:40:59.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.336+0000 7fc4c5537640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc4ac068280 0x7fc4ac06a740 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.336+0000 7fc4c5537640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c013c8f0 0x7fc4c013cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.336+0000 7fc4c5537640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c0072770 0x7fc4c015b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.336+0000 7fc4c5537640 1 --2- 172.21.15.29:0/1760912105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a80039c0 0x7fc4c0136bf0 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.336+0000 7fc4c5537640 1 -- 172.21.15.29:0/1760912105 >> 172.21.15.29:0/1760912105 conn(0x7fc4c00b1870 msgr2=0x7fc4c015c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:59.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.336+0000 7fc4c5537640 1 -- 172.21.15.29:0/1760912105 shutdown_connections 2024-04-25T00:40:59.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.336+0000 7fc4c5537640 1 -- 172.21.15.29:0/1760912105 wait complete. 2024-04-25T00:40:59.354 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:40:59.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.468+0000 7f4b2b64e640 1 Processor -- start 2024-04-25T00:40:59.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.468+0000 7f4b2b64e640 1 -- start start 2024-04-25T00:40:59.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b2b64e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b240cf690 0x7f4b240d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b2b64e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b240cfc10 con 0x7f4b2415e200 2024-04-25T00:40:59.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b2b64e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b240cfef0 con 0x7f4b240d2a90 2024-04-25T00:40:59.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b2b64e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b240d2ee0 con 0x7f4b240cf690 2024-04-25T00:40:59.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b293ea640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4b2415e200 0x7f4b240d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33926/0 (socket says 172.21.15.29:33926) 2024-04-25T00:40:59.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b293ea640 1 -- 172.21.15.29:0/2011252680 learned_addr learned my addr 172.21.15.29:0/2011252680 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:40:59.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b28be9640 1 --2- 172.21.15.29:0/2011252680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b240cf690 0x7f4b240d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1155633790 0 0) 0x7f4b240cfc10 con 0x7f4b2415e200 2024-04-25T00:40:59.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b10003530 con 0x7f4b2415e200 2024-04-25T00:40:59.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3219073942 0 0) 0x7f4b10003530 con 0x7f4b2415e200 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b240cf690 msgr2=0x7f4b240d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 --2- 172.21.15.29:0/2011252680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b240cf690 0x7f4b240d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 >> v1:172.21.15.157:6789/0 conn(0x7f4b240d2a90 legacy=0x7f4b240b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b240b3b20 con 0x7f4b2415e200 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4b20001a00 con 0x7f4b2415e200 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 >> v1:172.21.15.29:6789/0 conn(0x7f4b2415e200 legacy=0x7f4b240d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 --2- 172.21.15.29:0/2011252680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b100039f0 0x7f4b10003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 --2- 172.21.15.29:0/2011252680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b240cf690 0x7f4b10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 --2- 172.21.15.29:0/2011252680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b10004d50 0x7f4b100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4b240d2ee0 con 0x7f4b10004d50 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1000bb60 con 0x7f4b100039f0 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1000bd30 con 0x7f4b240cf690 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b29beb640 1 --2- 172.21.15.29:0/2011252680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b10004d50 0x7f4b100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b293ea640 1 --2- 172.21.15.29:0/2011252680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b100039f0 0x7f4b10003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b28be9640 1 --2- 172.21.15.29:0/2011252680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b240cf690 0x7f4b10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b29beb640 1 -- 172.21.15.29:0/2011252680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b240cf690 msgr2=0x7f4b10004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:59.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b29beb640 1 --2- 172.21.15.29:0/2011252680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b240cf690 0x7f4b10004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b29beb640 1 -- 172.21.15.29:0/2011252680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b100039f0 msgr2=0x7f4b10003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:59.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b29beb640 1 --2- 172.21.15.29:0/2011252680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b100039f0 0x7f4b10003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.472+0000 7f4b29beb640 1 -- 172.21.15.29:0/2011252680 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b10003530 con 0x7f4b10004d50 2024-04-25T00:40:59.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b29beb640 1 --2- 172.21.15.29:0/2011252680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b10004d50 0x7f4b100051d0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0017b0 tx=0x7f4b1c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a296751f7d6fad92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:59.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c0013a0 con 0x7f4b10004d50 2024-04-25T00:40:59.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1c0052f0 con 0x7f4b10004d50 2024-04-25T00:40:59.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b1bfff640 1 -- 172.21.15.29:0/2011252680 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c003840 con 0x7f4b10004d50 2024-04-25T00:40:59.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2011252680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b10004d50 msgr2=0x7f4b100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 --2- 172.21.15.29:0/2011252680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b10004d50 0x7f4b100051d0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0017b0 tx=0x7f4b1c01c400 comp rx=0 tx=0).stop 2024-04-25T00:40:59.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b29beb640 1 -- 172.21.15.29:0/2011252680 reap_dead start 2024-04-25T00:40:59.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2011252680 shutdown_connections 2024-04-25T00:40:59.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2011252680 >> 172.21.15.29:0/2011252680 conn(0x7f4b2409d280 msgr2=0x7f4b240d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:59.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2011252680 shutdown_connections 2024-04-25T00:40:59.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2011252680 wait complete. 2024-04-25T00:40:59.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 Processor -- start 2024-04-25T00:40:59.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 -- start start 2024-04-25T00:40:59.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b240b6110 0x7f4b240c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240b64f0 0x7f4b240c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b2415f930 0x7f4b2415fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2415e5e0 con 0x7f4b240b64f0 2024-04-25T00:40:59.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b240b38d0 con 0x7f4b240b6110 2024-04-25T00:40:59.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2415e760 con 0x7f4b2415f930 2024-04-25T00:40:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b28be9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240b64f0 0x7f4b240c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b28be9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240b64f0 0x7f4b240c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48682/0 (socket says 172.21.15.29:48682) 2024-04-25T00:40:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b28be9640 1 -- 172.21.15.29:0/2297620790 learned_addr learned my addr 172.21.15.29:0/2297620790 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b293ea640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b240b6110 0x7f4b240c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b29beb640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b2415f930 0x7f4b2415fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b28be9640 1 -- 172.21.15.29:0/2297620790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b2415f930 msgr2=0x7f4b2415fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b28be9640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b2415f930 0x7f4b2415fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b28be9640 1 -- 172.21.15.29:0/2297620790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b240b6110 msgr2=0x7f4b240c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b28be9640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b240b6110 0x7f4b240c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b28be9640 1 -- 172.21.15.29:0/2297620790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b240cb1e0 con 0x7f4b240b64f0 2024-04-25T00:40:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b293ea640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b240b6110 0x7f4b240c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b28be9640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240b64f0 0x7f4b240c9cf0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400f580 tx=0x7f4b1400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:59.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b19ffb640 1 -- 172.21.15.29:0/2297620790 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14019020 con 0x7f4b240b64f0 2024-04-25T00:40:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b19ffb640 1 -- 172.21.15.29:0/2297620790 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b14014870 con 0x7f4b240b64f0 2024-04-25T00:40:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2297620790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b240cb4c0 con 0x7f4b240b64f0 2024-04-25T00:40:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b19ffb640 1 -- 172.21.15.29:0/2297620790 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14014c40 con 0x7f4b240b64f0 2024-04-25T00:40:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.476+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2297620790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b240cba80 con 0x7f4b240b64f0 2024-04-25T00:40:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.480+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2297620790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b24069270 con 0x7f4b240b64f0 2024-04-25T00:40:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.480+0000 7f4b19ffb640 1 -- 172.21.15.29:0/2297620790 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f4b1401c020 con 0x7f4b240b64f0 2024-04-25T00:40:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.480+0000 7f4b19ffb640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4b08068280 0x7f4b0806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.480+0000 7f4b19ffb640 1 -- 172.21.15.29:0/2297620790 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f4b14084c80 con 0x7f4b240b64f0 2024-04-25T00:40:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.480+0000 7f4b293ea640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4b08068280 0x7f4b0806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.480+0000 7f4b293ea640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4b08068280 0x7f4b0806a740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4b20001340 tx=0x7f4b20001520 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:59.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.484+0000 7f4b19ffb640 1 -- 172.21.15.29:0/2297620790 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4b14023020 con 0x7f4b240b64f0 2024-04-25T00:40:59.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.648+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2297620790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b24067900 con 0x7f4b240b64f0 2024-04-25T00:40:59.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.648+0000 7f4b19ffb640 1 -- 172.21.15.29:0/2297620790 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16631 (secure 0 0 0) 0x7f4b14026350 con 0x7f4b240b64f0 2024-04-25T00:40:59.651 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:40:59.651 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":156,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:40:31.094463+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:40:27.796735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:40:28.430727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:40:59.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.652+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2297620790 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4b08068280 msgr2=0x7f4b0806a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.652+0000 7f4b2b64e640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4b08068280 0x7f4b0806a740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4b20001340 tx=0x7f4b20001520 comp rx=0 tx=0).stop 2024-04-25T00:40:59.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.652+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2297620790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240b64f0 msgr2=0x7f4b240c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.652+0000 7f4b2b64e640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240b64f0 0x7f4b240c9cf0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400f580 tx=0x7f4b1400fa40 comp rx=0 tx=0).stop 2024-04-25T00:40:59.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.652+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2297620790 shutdown_connections 2024-04-25T00:40:59.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.652+0000 7f4b2b64e640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4b08068280 0x7f4b0806a740 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.652+0000 7f4b2b64e640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b2415f930 0x7f4b2415fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.652+0000 7f4b2b64e640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240b64f0 0x7f4b240c9cf0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.652+0000 7f4b2b64e640 1 --2- 172.21.15.29:0/2297620790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b240b6110 0x7f4b240c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.652+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2297620790 >> 172.21.15.29:0/2297620790 conn(0x7f4b2409d280 msgr2=0x7f4b2406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:59.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.652+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2297620790 shutdown_connections 2024-04-25T00:40:59.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.652+0000 7f4b2b64e640 1 -- 172.21.15.29:0/2297620790 wait complete. 2024-04-25T00:40:59.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:40:27.796735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:40:28.430727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:40:59.671 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:40:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d2e018640 1 Processor -- start 2024-04-25T00:40:59.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d2e018640 1 -- start start 2024-04-25T00:40:59.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d2e018640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3d280b3140 0x7f3d2806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d2e018640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3d280b3c30 0x7f3d280ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d2e018640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d280b41e0 con 0x7f3d280b4b10 2024-04-25T00:40:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d2e018640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f3d2806d960 con 0x7f3d280b3c30 2024-04-25T00:40:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d2e018640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3d2806dc40 con 0x7f3d280b3140 2024-04-25T00:40:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d27fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3d280b3c30 0x7f3d280ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d27fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3d280b3c30 0x7f3d280ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48694/0 (socket says 172.21.15.29:48694) 2024-04-25T00:40:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d27fff640 1 -- 172.21.15.29:0/276535693 learned_addr learned my addr 172.21.15.29:0/276535693 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d277fe640 1 --2- 172.21.15.29:0/276535693 >> v2:172.21.15.157:3300/0 conn(0x7f3d280b3140 0x7f3d2806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d27fff640 1 -- 172.21.15.29:0/276535693 >> v2:172.21.15.157:3300/0 conn(0x7f3d280b3140 msgr2=0x7f3d2806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d27fff640 1 --2- 172.21.15.29:0/276535693 >> v2:172.21.15.157:3300/0 conn(0x7f3d280b3140 0x7f3d2806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d27fff640 1 -- 172.21.15.29:0/276535693 >> v1:172.21.15.157:6789/0 conn(0x7f3d280b4b10 legacy=0x7f3d280b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.784+0000 7f3d27fff640 1 -- 172.21.15.29:0/276535693 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2806df20 con 0x7f3d280b3c30 2024-04-25T00:40:59.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d27fff640 1 --2- 172.21.15.29:0/276535693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d280b3c30 0x7f3d280ce240 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009870 tx=0x7f3d2001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5194028ad2173775 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:59.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d26ffd640 1 -- 172.21.15.29:0/276535693 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d200b0070 con 0x7f3d280b3c30 2024-04-25T00:40:59.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d26ffd640 1 -- 172.21.15.29:0/276535693 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d200ab600 con 0x7f3d280b3c30 2024-04-25T00:40:59.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d26ffd640 1 -- 172.21.15.29:0/276535693 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d200ab9d0 con 0x7f3d280b3c30 2024-04-25T00:40:59.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 -- 172.21.15.29:0/276535693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d280b3c30 msgr2=0x7f3d280ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 --2- 172.21.15.29:0/276535693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d280b3c30 0x7f3d280ce240 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009870 tx=0x7f3d2001c520 comp rx=0 tx=0).stop 2024-04-25T00:40:59.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 -- 172.21.15.29:0/276535693 shutdown_connections 2024-04-25T00:40:59.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 --2- 172.21.15.29:0/276535693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d280b3c30 0x7f3d280ce240 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 --2- 172.21.15.29:0/276535693 >> v2:172.21.15.157:3300/0 conn(0x7f3d280b3140 0x7f3d2806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 -- 172.21.15.29:0/276535693 >> 172.21.15.29:0/276535693 conn(0x7f3d280b1870 msgr2=0x7f3d280b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:59.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 -- 172.21.15.29:0/276535693 shutdown_connections 2024-04-25T00:40:59.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 -- 172.21.15.29:0/276535693 wait complete. 2024-04-25T00:40:59.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 Processor -- start 2024-04-25T00:40:59.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 -- start start 2024-04-25T00:40:59.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d280b3140 0x7f3d28152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d280b3c30 0x7f3d2813c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d280b4b10 0x7f3d28138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28168d80 con 0x7f3d280b3c30 2024-04-25T00:40:59.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28168c00 con 0x7f3d280b4b10 2024-04-25T00:40:59.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.788+0000 7f3d2e018640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3d28168f00 con 0x7f3d280b3140 2024-04-25T00:40:59.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d1effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d280b3c30 0x7f3d2813c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d1effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d280b3c30 0x7f3d2813c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48702/0 (socket says 172.21.15.29:48702) 2024-04-25T00:40:59.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d1effd640 1 -- 172.21.15.29:0/669089081 learned_addr learned my addr 172.21.15.29:0/669089081 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:40:59.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d27fff640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d280b4b10 0x7f3d28138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d1effd640 1 -- 172.21.15.29:0/669089081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d280b3140 msgr2=0x7f3d28152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d277fe640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d280b3140 0x7f3d28152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d1effd640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d280b3140 0x7f3d28152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d1effd640 1 -- 172.21.15.29:0/669089081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d280b4b10 msgr2=0x7f3d28138ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d1effd640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d280b4b10 0x7f3d28138ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d1effd640 1 -- 172.21.15.29:0/669089081 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d28153ad0 con 0x7f3d280b3c30 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d27fff640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d280b4b10 0x7f3d28138ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d277fe640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d280b3140 0x7f3d28152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d1effd640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d280b3c30 0x7f3d2813c1b0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000c450 tx=0x7f3d1000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d24ff9640 1 -- 172.21.15.29:0/669089081 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10016020 con 0x7f3d280b3c30 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d24ff9640 1 -- 172.21.15.29:0/669089081 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d100052d0 con 0x7f3d280b3c30 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d2e018640 1 -- 172.21.15.29:0/669089081 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d28153510 con 0x7f3d280b3c30 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d24ff9640 1 -- 172.21.15.29:0/669089081 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10007260 con 0x7f3d280b3c30 2024-04-25T00:40:59.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d2e018640 1 -- 172.21.15.29:0/669089081 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d2815acd0 con 0x7f3d280b3c30 2024-04-25T00:40:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d2e018640 1 -- 172.21.15.29:0/669089081 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d28069270 con 0x7f3d280b3c30 2024-04-25T00:40:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d24ff9640 1 -- 172.21.15.29:0/669089081 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f3d10007400 con 0x7f3d280b3c30 2024-04-25T00:40:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d24ff9640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3d08068250 0x7f3d0806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d24ff9640 1 -- 172.21.15.29:0/669089081 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f3d10084bd0 con 0x7f3d280b3c30 2024-04-25T00:40:59.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d277fe640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3d08068250 0x7f3d0806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:59.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.792+0000 7f3d277fe640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3d08068250 0x7f3d0806a710 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009f50 tx=0x7f3d1800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:59.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.796+0000 7f3d24ff9640 1 -- 172.21.15.29:0/669089081 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3d10020020 con 0x7f3d280b3c30 2024-04-25T00:40:59.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.956+0000 7f3d2e018640 1 -- 172.21.15.29:0/669089081 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3d280753e0 con 0x7f3d280b3c30 2024-04-25T00:40:59.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.960+0000 7f3d24ff9640 1 -- 172.21.15.29:0/669089081 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) ==== 93+0+10 (secure 0 0 0) 0x7f3d10005480 con 0x7f3d280b3c30 2024-04-25T00:40:59.960 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:40:59.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.960+0000 7f3d2e018640 1 -- 172.21.15.29:0/669089081 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3d08068250 msgr2=0x7f3d0806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.960+0000 7f3d2e018640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3d08068250 0x7f3d0806a710 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009f50 tx=0x7f3d1800b040 comp rx=0 tx=0).stop 2024-04-25T00:40:59.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.960+0000 7f3d2e018640 1 -- 172.21.15.29:0/669089081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d280b3c30 msgr2=0x7f3d2813c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:59.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.960+0000 7f3d2e018640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d280b3c30 0x7f3d2813c1b0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000c450 tx=0x7f3d1000c910 comp rx=0 tx=0).stop 2024-04-25T00:40:59.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.960+0000 7f3d2e018640 1 -- 172.21.15.29:0/669089081 shutdown_connections 2024-04-25T00:40:59.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.960+0000 7f3d2e018640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3d08068250 0x7f3d0806a710 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.960+0000 7f3d2e018640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d280b4b10 0x7f3d28138ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.960+0000 7f3d2e018640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d280b3c30 0x7f3d2813c1b0 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.960+0000 7f3d2e018640 1 --2- 172.21.15.29:0/669089081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d280b3140 0x7f3d28152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:59.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.960+0000 7f3d2e018640 1 -- 172.21.15.29:0/669089081 >> 172.21.15.29:0/669089081 conn(0x7f3d280b1870 msgr2=0x7f3d28140da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:59.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.960+0000 7f3d2e018640 1 -- 172.21.15.29:0/669089081 shutdown_connections 2024-04-25T00:40:59.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:40:59.960+0000 7f3d2e018640 1 -- 172.21.15.29:0/669089081 wait complete. 2024-04-25T00:40:59.978 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:41:00.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98595ef640 1 Processor -- start 2024-04-25T00:41:00.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98595ef640 1 -- start start 2024-04-25T00:41:00.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98595ef640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f985415e200 0x7f98540d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98595ef640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98540cfc10 con 0x7f98540d2c60 2024-04-25T00:41:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98595ef640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98540cfef0 con 0x7f98540cf690 2024-04-25T00:41:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98595ef640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f98540b38d0 con 0x7f985415e200 2024-04-25T00:41:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98537fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f98540d2c60 0x7f98540b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33932/0 (socket says 172.21.15.29:33932) 2024-04-25T00:41:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98537fe640 1 -- 172.21.15.29:0/3040687315 learned_addr learned my addr 172.21.15.29:0/3040687315 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9852ffd640 1 --2- 172.21.15.29:0/3040687315 >> v2:172.21.15.157:3300/0 conn(0x7f985415e200 0x7f98540d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 955150427 0 0) 0x7f98540cfc10 con 0x7f98540d2c60 2024-04-25T00:41:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f983c003530 con 0x7f98540d2c60 2024-04-25T00:41:00.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3859300133 0 0) 0x7f983c003530 con 0x7f98540d2c60 2024-04-25T00:41:00.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 >> v2:172.21.15.157:3300/0 conn(0x7f985415e200 msgr2=0x7f98540d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:00.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 --2- 172.21.15.29:0/3040687315 >> v2:172.21.15.157:3300/0 conn(0x7f985415e200 0x7f98540d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 >> v1:172.21.15.157:6789/0 conn(0x7f98540cf690 legacy=0x7f98540d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98540b3b20 con 0x7f98540d2c60 2024-04-25T00:41:00.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f984c001680 con 0x7f98540d2c60 2024-04-25T00:41:00.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 >> v1:172.21.15.29:6789/0 conn(0x7f98540d2c60 legacy=0x7f98540b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 --2- 172.21.15.29:0/3040687315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f983c0039c0 0x7f983c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 --2- 172.21.15.29:0/3040687315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f983c004810 0x7f983c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 --2- 172.21.15.29:0/3040687315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f983c0051d0 0x7f983c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f98540b38d0 con 0x7f983c0051d0 2024-04-25T00:41:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f983c00c070 con 0x7f983c0039c0 2024-04-25T00:41:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f983c00c270 con 0x7f983c004810 2024-04-25T00:41:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98537fe640 1 --2- 172.21.15.29:0/3040687315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f983c0051d0 0x7f983c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f9852ffd640 1 --2- 172.21.15.29:0/3040687315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f983c0039c0 0x7f983c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98527fc640 1 --2- 172.21.15.29:0/3040687315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f983c004810 0x7f983c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98537fe640 1 -- 172.21.15.29:0/3040687315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f983c004810 msgr2=0x7f983c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98537fe640 1 --2- 172.21.15.29:0/3040687315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f983c004810 0x7f983c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98537fe640 1 -- 172.21.15.29:0/3040687315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f983c0039c0 msgr2=0x7f983c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98537fe640 1 --2- 172.21.15.29:0/3040687315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f983c0039c0 0x7f983c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.092+0000 7f98537fe640 1 -- 172.21.15.29:0/3040687315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983c003530 con 0x7f983c0051d0 2024-04-25T00:41:00.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f98537fe640 1 -- 172.21.15.29:0/3040687315 reap_dead start 2024-04-25T00:41:00.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f98537fe640 1 --2- 172.21.15.29:0/3040687315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f983c0051d0 0x7f983c00b9d0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c210 tx=0x7f984c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be92cc43b4f8a854 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:00.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c007d40 con 0x7f983c0051d0 2024-04-25T00:41:00.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984c0a9070 con 0x7f983c0051d0 2024-04-25T00:41:00.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f9851ffb640 1 -- 172.21.15.29:0/3040687315 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c0a5e00 con 0x7f983c0051d0 2024-04-25T00:41:00.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f98595ef640 1 -- 172.21.15.29:0/3040687315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f983c0051d0 msgr2=0x7f983c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f98595ef640 1 --2- 172.21.15.29:0/3040687315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f983c0051d0 0x7f983c00b9d0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c210 tx=0x7f984c01cca0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f98595ef640 1 -- 172.21.15.29:0/3040687315 shutdown_connections 2024-04-25T00:41:00.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f98595ef640 1 --2- 172.21.15.29:0/3040687315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f983c0051d0 0x7f983c00b9d0 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f98595ef640 1 -- 172.21.15.29:0/3040687315 >> 172.21.15.29:0/3040687315 conn(0x7f985409d280 msgr2=0x7f98540d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f98595ef640 1 -- 172.21.15.29:0/3040687315 shutdown_connections 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f98595ef640 1 -- 172.21.15.29:0/3040687315 wait complete. 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f98595ef640 1 Processor -- start 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.096+0000 7f98595ef640 1 -- start start 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f98595ef640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f983c0051d0 0x7f9854160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f98595ef640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985415e8e0 0x7f9854153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f98595ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985415d390 0x7f985415d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f98595ef640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f985415e760 con 0x7f983c0051d0 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f98595ef640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f985415e5e0 con 0x7f985415d390 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f98595ef640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f98540d2550 con 0x7f985415e8e0 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f9852ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f983c0051d0 0x7f9854160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f9852ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f983c0051d0 0x7f9854160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48724/0 (socket says 172.21.15.29:48724) 2024-04-25T00:41:00.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f9852ffd640 1 -- 172.21.15.29:0/2315222574 learned_addr learned my addr 172.21.15.29:0/2315222574 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f98537fe640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985415d390 0x7f985415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f9852ffd640 1 -- 172.21.15.29:0/2315222574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985415e8e0 msgr2=0x7f9854153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f98527fc640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985415e8e0 0x7f9854153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f9852ffd640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985415e8e0 0x7f9854153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f9852ffd640 1 -- 172.21.15.29:0/2315222574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985415d390 msgr2=0x7f985415d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f9852ffd640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985415d390 0x7f985415d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f9852ffd640 1 -- 172.21.15.29:0/2315222574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985415f740 con 0x7f983c0051d0 2024-04-25T00:41:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f9852ffd640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f983c0051d0 0x7f9854160eb0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f9844007f00 tx=0x7f98440115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f983b7fe640 1 -- 172.21.15.29:0/2315222574 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984401a020 con 0x7f983c0051d0 2024-04-25T00:41:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f983b7fe640 1 -- 172.21.15.29:0/2315222574 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9844016470 con 0x7f983c0051d0 2024-04-25T00:41:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f98595ef640 1 -- 172.21.15.29:0/2315222574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98541613f0 con 0x7f983c0051d0 2024-04-25T00:41:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f983b7fe640 1 -- 172.21.15.29:0/2315222574 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844016860 con 0x7f983c0051d0 2024-04-25T00:41:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f98595ef640 1 -- 172.21.15.29:0/2315222574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f985415f180 con 0x7f983c0051d0 2024-04-25T00:41:00.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f98595ef640 1 -- 172.21.15.29:0/2315222574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f985415dd50 con 0x7f983c0051d0 2024-04-25T00:41:00.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f983b7fe640 1 -- 172.21.15.29:0/2315222574 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f984401d020 con 0x7f983c0051d0 2024-04-25T00:41:00.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f983b7fe640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9840068280 0x7f984006a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f98527fc640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9840068280 0x7f984006a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f983b7fe640 1 -- 172.21.15.29:0/2315222574 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f98440851d0 con 0x7f983c0051d0 2024-04-25T00:41:00.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.100+0000 7f98527fc640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9840068280 0x7f984006a740 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f98540ce670 tx=0x7f9848009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:00.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.104+0000 7f983b7fe640 1 -- 172.21.15.29:0/2315222574 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9844024030 con 0x7f983c0051d0 2024-04-25T00:41:00.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.268+0000 7f98595ef640 1 -- 172.21.15.29:0/2315222574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f98540c8380 con 0x7f983c0051d0 2024-04-25T00:41:00.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.268+0000 7f983b7fe640 1 -- 172.21.15.29:0/2315222574 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) ==== 104+0+11 (secure 0 0 0) 0x7f9844057910 con 0x7f983c0051d0 2024-04-25T00:41:00.271 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:41:00.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.272+0000 7f98595ef640 1 -- 172.21.15.29:0/2315222574 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9840068280 msgr2=0x7f984006a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.272+0000 7f98595ef640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9840068280 0x7f984006a740 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f98540ce670 tx=0x7f9848009040 comp rx=0 tx=0).stop 2024-04-25T00:41:00.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.272+0000 7f98595ef640 1 -- 172.21.15.29:0/2315222574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f983c0051d0 msgr2=0x7f9854160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.272+0000 7f98595ef640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f983c0051d0 0x7f9854160eb0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f9844007f00 tx=0x7f98440115a0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.272+0000 7f98595ef640 1 -- 172.21.15.29:0/2315222574 shutdown_connections 2024-04-25T00:41:00.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.272+0000 7f98595ef640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9840068280 0x7f984006a740 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.272+0000 7f98595ef640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985415d390 0x7f985415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.272+0000 7f98595ef640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985415e8e0 0x7f9854153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.272+0000 7f98595ef640 1 --2- 172.21.15.29:0/2315222574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f983c0051d0 0x7f9854160eb0 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.272+0000 7f98595ef640 1 -- 172.21.15.29:0/2315222574 >> 172.21.15.29:0/2315222574 conn(0x7f985409d280 msgr2=0x7f98540b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:00.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.272+0000 7f98595ef640 1 -- 172.21.15.29:0/2315222574 shutdown_connections 2024-04-25T00:41:00.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.272+0000 7f98595ef640 1 -- 172.21.15.29:0/2315222574 wait complete. 2024-04-25T00:41:00.290 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:41:00.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf998e640 1 Processor -- start 2024-04-25T00:41:00.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf998e640 1 -- start start 2024-04-25T00:41:00.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf998e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1cf40b3c30 0x7f1cf406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf998e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1cf40b4b10 0x7f1cf40ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf998e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cf40b41e0 con 0x7f1cf40b3140 2024-04-25T00:41:00.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf998e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1cf406d960 con 0x7f1cf40b4b10 2024-04-25T00:41:00.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf998e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1cf406dc40 con 0x7f1cf40b3c30 2024-04-25T00:41:00.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf37fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1cf40b4b10 0x7f1cf40ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf37fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1cf40b4b10 0x7f1cf40ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48732/0 (socket says 172.21.15.29:48732) 2024-04-25T00:41:00.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf37fe640 1 -- 172.21.15.29:0/3359687669 learned_addr learned my addr 172.21.15.29:0/3359687669 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:00.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf27fc640 1 --2- 172.21.15.29:0/3359687669 >> v2:172.21.15.157:3300/0 conn(0x7f1cf40b3c30 0x7f1cf406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf37fe640 1 -- 172.21.15.29:0/3359687669 >> v2:172.21.15.157:3300/0 conn(0x7f1cf40b3c30 msgr2=0x7f1cf406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:00.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf37fe640 1 --2- 172.21.15.29:0/3359687669 >> v2:172.21.15.157:3300/0 conn(0x7f1cf40b3c30 0x7f1cf406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf37fe640 1 -- 172.21.15.29:0/3359687669 >> v1:172.21.15.157:6789/0 conn(0x7f1cf40b3140 legacy=0x7f1cf40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:00.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.412+0000 7f1cf37fe640 1 -- 172.21.15.29:0/3359687669 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf406df20 con 0x7f1cf40b4b10 2024-04-25T00:41:00.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf37fe640 1 --2- 172.21.15.29:0/3359687669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cf40b4b10 0x7f1cf40ce240 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009e50 tx=0x7f1cec01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6292d5eba1eb05ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:00.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf1ffb640 1 -- 172.21.15.29:0/3359687669 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0a5670 con 0x7f1cf40b4b10 2024-04-25T00:41:00.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf1ffb640 1 -- 172.21.15.29:0/3359687669 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cec0a5810 con 0x7f1cf40b4b10 2024-04-25T00:41:00.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf1ffb640 1 -- 172.21.15.29:0/3359687669 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0a5be0 con 0x7f1cf40b4b10 2024-04-25T00:41:00.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf998e640 1 -- 172.21.15.29:0/3359687669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cf40b4b10 msgr2=0x7f1cf40ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf998e640 1 --2- 172.21.15.29:0/3359687669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cf40b4b10 0x7f1cf40ce240 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009e50 tx=0x7f1cec01c9b0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf998e640 1 -- 172.21.15.29:0/3359687669 shutdown_connections 2024-04-25T00:41:00.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf998e640 1 --2- 172.21.15.29:0/3359687669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cf40b4b10 0x7f1cf40ce240 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf998e640 1 --2- 172.21.15.29:0/3359687669 >> v2:172.21.15.157:3300/0 conn(0x7f1cf40b3c30 0x7f1cf406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf998e640 1 -- 172.21.15.29:0/3359687669 >> 172.21.15.29:0/3359687669 conn(0x7f1cf40b1870 msgr2=0x7f1cf40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:00.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf998e640 1 -- 172.21.15.29:0/3359687669 shutdown_connections 2024-04-25T00:41:00.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf998e640 1 -- 172.21.15.29:0/3359687669 wait complete. 2024-04-25T00:41:00.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf998e640 1 Processor -- start 2024-04-25T00:41:00.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.416+0000 7f1cf998e640 1 -- start start 2024-04-25T00:41:00.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf998e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cf40b3140 0x7f1cf4140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf998e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf40b3c30 0x7f1cf4139630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf998e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf40b4b10 0x7f1cf413c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf998e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4168d80 con 0x7f1cf40b3140 2024-04-25T00:41:00.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf998e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4168c00 con 0x7f1cf40b3c30 2024-04-25T00:41:00.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf998e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf4168f00 con 0x7f1cf40b4b10 2024-04-25T00:41:00.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cf40b3140 0x7f1cf4140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cf40b3140 0x7f1cf4140010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48742/0 (socket says 172.21.15.29:48742) 2024-04-25T00:41:00.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf2ffd640 1 -- 172.21.15.29:0/3311766141 learned_addr learned my addr 172.21.15.29:0/3311766141 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:00.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf27fc640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf40b3c30 0x7f1cf4139630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf37fe640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf40b4b10 0x7f1cf413c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf2ffd640 1 -- 172.21.15.29:0/3311766141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf40b4b10 msgr2=0x7f1cf413c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf2ffd640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf40b4b10 0x7f1cf413c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf2ffd640 1 -- 172.21.15.29:0/3311766141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf40b3c30 msgr2=0x7f1cf4139630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf2ffd640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf40b3c30 0x7f1cf4139630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf2ffd640 1 -- 172.21.15.29:0/3311766141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf4139b70 con 0x7f1cf40b3140 2024-04-25T00:41:00.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf37fe640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf40b4b10 0x7f1cf413c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:00.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf2ffd640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cf40b3140 0x7f1cf4140010 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400c450 tx=0x7f1ce400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:00.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cdbfff640 1 -- 172.21.15.29:0/3311766141 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4012830 con 0x7f1cf40b3140 2024-04-25T00:41:00.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cdbfff640 1 -- 172.21.15.29:0/3311766141 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce4017070 con 0x7f1cf40b3140 2024-04-25T00:41:00.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf998e640 1 -- 172.21.15.29:0/3311766141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf413cd60 con 0x7f1cf40b3140 2024-04-25T00:41:00.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cdbfff640 1 -- 172.21.15.29:0/3311766141 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4012c30 con 0x7f1cf40b3140 2024-04-25T00:41:00.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf998e640 1 -- 172.21.15.29:0/3311766141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf413bd40 con 0x7f1cf40b3140 2024-04-25T00:41:00.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf998e640 1 -- 172.21.15.29:0/3311766141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf4069270 con 0x7f1cf40b3140 2024-04-25T00:41:00.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cdbfff640 1 -- 172.21.15.29:0/3311766141 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f1ce401c460 con 0x7f1cf40b3140 2024-04-25T00:41:00.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cdbfff640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1ce0068250 0x7f1ce006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cdbfff640 1 -- 172.21.15.29:0/3311766141 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f1ce40841f0 con 0x7f1cf40b3140 2024-04-25T00:41:00.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf27fc640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1ce0068250 0x7f1ce006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.420+0000 7f1cf27fc640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1ce0068250 0x7f1ce006a710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1cf41f8180 tx=0x7f1ce8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:00.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.424+0000 7f1cdbfff640 1 -- 172.21.15.29:0/3311766141 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1cf413cd60 con 0x7f1cf40b3140 2024-04-25T00:41:00.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.580+0000 7f1cf998e640 1 -- 172.21.15.29:0/3311766141 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1cf4155cd0 con 0x7f1cf40b3140 2024-04-25T00:41:00.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.584+0000 7f1cdbfff640 1 -- 172.21.15.29:0/3311766141 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) ==== 100+0+11 (secure 0 0 0) 0x7f1ce40568b0 con 0x7f1cf40b3140 2024-04-25T00:41:00.583 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:41:00.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.584+0000 7f1cf998e640 1 -- 172.21.15.29:0/3311766141 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1ce0068250 msgr2=0x7f1ce006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.584+0000 7f1cf998e640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1ce0068250 0x7f1ce006a710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1cf41f8180 tx=0x7f1ce8008040 comp rx=0 tx=0).stop 2024-04-25T00:41:00.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.584+0000 7f1cf998e640 1 -- 172.21.15.29:0/3311766141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cf40b3140 msgr2=0x7f1cf4140010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.584+0000 7f1cf998e640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cf40b3140 0x7f1cf4140010 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400c450 tx=0x7f1ce400c910 comp rx=0 tx=0).stop 2024-04-25T00:41:00.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.584+0000 7f1cf998e640 1 -- 172.21.15.29:0/3311766141 shutdown_connections 2024-04-25T00:41:00.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.584+0000 7f1cf998e640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1ce0068250 0x7f1ce006a710 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.584+0000 7f1cf998e640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cf40b4b10 0x7f1cf413c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.584+0000 7f1cf998e640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf40b3c30 0x7f1cf4139630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.584+0000 7f1cf998e640 1 --2- 172.21.15.29:0/3311766141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cf40b3140 0x7f1cf4140010 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.584+0000 7f1cf998e640 1 -- 172.21.15.29:0/3311766141 >> 172.21.15.29:0/3311766141 conn(0x7f1cf40b1870 msgr2=0x7f1cf40cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:00.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.584+0000 7f1cf998e640 1 -- 172.21.15.29:0/3311766141 shutdown_connections 2024-04-25T00:41:00.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.584+0000 7f1cf998e640 1 -- 172.21.15.29:0/3311766141 wait complete. 2024-04-25T00:41:00.601 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:41:00.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.716+0000 7fea99e5f640 1 Processor -- start 2024-04-25T00:41:00.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.716+0000 7fea99e5f640 1 -- start start 2024-04-25T00:41:00.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea99e5f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fea940b3140 0x7fea9406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea99e5f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea940b3c30 0x7fea940ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea99e5f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea940b41e0 con 0x7fea940b4b10 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea99e5f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fea9406d960 con 0x7fea940b3140 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea99e5f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fea9406dc40 con 0x7fea940b3c30 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea937fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fea940b3140 0x7fea9406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea92ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fea940b4b10 0x7fea940b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33948/0 (socket says 172.21.15.29:33948) 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea92ffd640 1 -- 172.21.15.29:0/3497012852 learned_addr learned my addr 172.21.15.29:0/3497012852 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea93fff640 1 --2- 172.21.15.29:0/3497012852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea940b3c30 0x7fea940ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea927fc640 1 -- 172.21.15.29:0/3497012852 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1418112686 0 0) 0x7fea940b41e0 con 0x7fea940b4b10 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea927fc640 1 -- 172.21.15.29:0/3497012852 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea80003530 con 0x7fea940b4b10 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea937fe640 1 -- 172.21.15.29:0/3497012852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea940b3c30 msgr2=0x7fea940ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea937fe640 1 --2- 172.21.15.29:0/3497012852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea940b3c30 0x7fea940ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea937fe640 1 -- 172.21.15.29:0/3497012852 >> v1:172.21.15.29:6789/0 conn(0x7fea940b4b10 legacy=0x7fea940b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea937fe640 1 -- 172.21.15.29:0/3497012852 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9406df20 con 0x7fea940b3140 2024-04-25T00:41:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea93fff640 1 --2- 172.21.15.29:0/3497012852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea940b3c30 0x7fea940ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:00.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea937fe640 1 --2- 172.21.15.29:0/3497012852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea940b3140 0x7fea9406e440 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009dc0 tx=0x7fea8c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2991e1e4aac3e77d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:00.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea927fc640 1 -- 172.21.15.29:0/3497012852 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0b2070 con 0x7fea940b3140 2024-04-25T00:41:00.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea927fc640 1 -- 172.21.15.29:0/3497012852 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea8c0a5810 con 0x7fea940b3140 2024-04-25T00:41:00.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.720+0000 7fea927fc640 1 -- 172.21.15.29:0/3497012852 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0a5be0 con 0x7fea940b3140 2024-04-25T00:41:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 -- 172.21.15.29:0/3497012852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea940b3140 msgr2=0x7fea9406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 --2- 172.21.15.29:0/3497012852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea940b3140 0x7fea9406e440 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009dc0 tx=0x7fea8c01c950 comp rx=0 tx=0).stop 2024-04-25T00:41:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 -- 172.21.15.29:0/3497012852 shutdown_connections 2024-04-25T00:41:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 --2- 172.21.15.29:0/3497012852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea940b3c30 0x7fea940ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 --2- 172.21.15.29:0/3497012852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea940b3140 0x7fea9406e440 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 -- 172.21.15.29:0/3497012852 >> 172.21.15.29:0/3497012852 conn(0x7fea940b1870 msgr2=0x7fea940b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 -- 172.21.15.29:0/3497012852 shutdown_connections 2024-04-25T00:41:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 -- 172.21.15.29:0/3497012852 wait complete. 2024-04-25T00:41:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 Processor -- start 2024-04-25T00:41:00.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 -- start start 2024-04-25T00:41:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea940b3140 0x7fea94071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea940b3c30 0x7fea94070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea940b4b10 0x7fea9413a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fea94168d80 con 0x7fea940b4b10 2024-04-25T00:41:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea94168c00 con 0x7fea940b3c30 2024-04-25T00:41:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fea94168f00 con 0x7fea940b3140 2024-04-25T00:41:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea93fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea940b4b10 0x7fea9413a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea93fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea940b4b10 0x7fea9413a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48754/0 (socket says 172.21.15.29:48754) 2024-04-25T00:41:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea93fff640 1 -- 172.21.15.29:0/3490778320 learned_addr learned my addr 172.21.15.29:0/3490778320 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea92ffd640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea940b3c30 0x7fea94070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea937fe640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea940b3140 0x7fea94071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea93fff640 1 -- 172.21.15.29:0/3490778320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea940b3140 msgr2=0x7fea94071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea93fff640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea940b3140 0x7fea94071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea93fff640 1 -- 172.21.15.29:0/3490778320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea940b3c30 msgr2=0x7fea94070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea93fff640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea940b3c30 0x7fea94070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea93fff640 1 -- 172.21.15.29:0/3490778320 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9415d4b0 con 0x7fea940b4b10 2024-04-25T00:41:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea92ffd640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea940b3c30 0x7fea94070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea937fe640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea940b3140 0x7fea94071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea93fff640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea940b4b10 0x7fea9413a5f0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fea8400e810 tx=0x7fea8400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea8bfff640 1 -- 172.21.15.29:0/3490778320 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84017070 con 0x7fea940b4b10 2024-04-25T00:41:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea8bfff640 1 -- 172.21.15.29:0/3490778320 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea8400c6d0 con 0x7fea940b4b10 2024-04-25T00:41:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 -- 172.21.15.29:0/3490778320 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea94072390 con 0x7fea940b4b10 2024-04-25T00:41:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea8bfff640 1 -- 172.21.15.29:0/3490778320 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8400f040 con 0x7fea940b4b10 2024-04-25T00:41:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.724+0000 7fea99e5f640 1 -- 172.21.15.29:0/3490778320 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9415da30 con 0x7fea940b4b10 2024-04-25T00:41:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.728+0000 7fea99e5f640 1 -- 172.21.15.29:0/3490778320 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea9415dc30 con 0x7fea940b4b10 2024-04-25T00:41:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.728+0000 7fea8bfff640 1 -- 172.21.15.29:0/3490778320 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fea8401a070 con 0x7fea940b4b10 2024-04-25T00:41:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.728+0000 7fea8bfff640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fea74068250 0x7fea7406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.728+0000 7fea8bfff640 1 -- 172.21.15.29:0/3490778320 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fea84083fb0 con 0x7fea940b4b10 2024-04-25T00:41:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.728+0000 7fea937fe640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fea74068250 0x7fea7406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.728+0000 7fea937fe640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fea74068250 0x7fea7406a710 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0af000 tx=0x7fea8c0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:00.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.732+0000 7fea8bfff640 1 -- 172.21.15.29:0/3490778320 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fea9415dc30 con 0x7fea940b4b10 2024-04-25T00:41:00.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.928+0000 7fea99e5f640 1 -- 172.21.15.29:0/3490778320 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea94069270 con 0x7fea940b4b10 2024-04-25T00:41:00.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.928+0000 7fea8bfff640 1 -- 172.21.15.29:0/3490778320 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5196 (secure 0 0 0) 0x7fea840566f0 con 0x7fea940b4b10 2024-04-25T00:41:00.929 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:00.929 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2677876688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2677876688},{"type":"v1","addr":"172.21.15.157:6833","nonce":2677876688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3340901266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3340901266},{"type":"v1","addr":"172.21.15.157:6835","nonce":3340901266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":10092,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/567284262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":567284262},{"type":"v1","addr":"172.21.15.29:6833","nonce":567284262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:40:29.419450+0000","modified":"2024-04-25T00:40:32.484311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10095":{"gid":10095,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/4183931647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":4183931647},{"type":"v1","addr":"172.21.15.29:6835","nonce":4183931647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10095,"qdb_cluster":[10095]},"id":12}]} 2024-04-25T00:41:00.929 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 103 2024-04-25T00:41:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.932+0000 7fea99e5f640 1 -- 172.21.15.29:0/3490778320 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fea74068250 msgr2=0x7fea7406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.932+0000 7fea99e5f640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fea74068250 0x7fea7406a710 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0af000 tx=0x7fea8c0a7070 comp rx=0 tx=0).stop 2024-04-25T00:41:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.932+0000 7fea99e5f640 1 -- 172.21.15.29:0/3490778320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea940b4b10 msgr2=0x7fea9413a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.932+0000 7fea99e5f640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea940b4b10 0x7fea9413a5f0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fea8400e810 tx=0x7fea8400ecd0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.932+0000 7fea99e5f640 1 -- 172.21.15.29:0/3490778320 shutdown_connections 2024-04-25T00:41:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.932+0000 7fea99e5f640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fea74068250 0x7fea7406a710 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.932+0000 7fea99e5f640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea940b4b10 0x7fea9413a5f0 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.932+0000 7fea99e5f640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea940b3c30 0x7fea94070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.932+0000 7fea99e5f640 1 --2- 172.21.15.29:0/3490778320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea940b3140 0x7fea94071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.932+0000 7fea99e5f640 1 -- 172.21.15.29:0/3490778320 >> 172.21.15.29:0/3490778320 conn(0x7fea940b1870 msgr2=0x7fea94158320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.932+0000 7fea99e5f640 1 -- 172.21.15.29:0/3490778320 shutdown_connections 2024-04-25T00:41:00.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:00.932+0000 7fea99e5f640 1 -- 172.21.15.29:0/3490778320 wait complete. 2024-04-25T00:41:00.947 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:01.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.056+0000 7f7ed5581640 1 Processor -- start 2024-04-25T00:41:01.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.056+0000 7f7ed5581640 1 -- start start 2024-04-25T00:41:01.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ed5581640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed006dce0 0x7f7ed0150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ed5581640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7ed00b2b70 0x7f7ed0151330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ed5581640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed00b3130 con 0x7f7ed015d980 2024-04-25T00:41:01.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ed5581640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed00cf560 con 0x7f7ed00b2b70 2024-04-25T00:41:01.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ed5581640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed00b3410 con 0x7f7ed006dce0 2024-04-25T00:41:01.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7eceffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7ed015d980 0x7f7ed006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33958/0 (socket says 172.21.15.29:33958) 2024-04-25T00:41:01.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7eceffd640 1 -- 172.21.15.29:0/4191375594 learned_addr learned my addr 172.21.15.29:0/4191375594 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:01.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecf7fe640 1 --2- 172.21.15.29:0/4191375594 >> v2:172.21.15.157:3300/0 conn(0x7f7ed00b2b70 0x7f7ed0151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ece7fc640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed006dce0 0x7f7ed0150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3302525871 0 0) 0x7f7ed00b3130 con 0x7f7ed015d980 2024-04-25T00:41:01.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7eb8003530 con 0x7f7ed015d980 2024-04-25T00:41:01.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3076667614 0 0) 0x7f7eb8003530 con 0x7f7ed015d980 2024-04-25T00:41:01.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed006dce0 msgr2=0x7f7ed0150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed006dce0 0x7f7ed0150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 >> v2:172.21.15.157:3300/0 conn(0x7f7ed00b2b70 msgr2=0x7f7ed0151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 --2- 172.21.15.29:0/4191375594 >> v2:172.21.15.157:3300/0 conn(0x7f7ed00b2b70 0x7f7ed0151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed00b36f0 con 0x7f7ed015d980 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f7ec8001680 con 0x7f7ed015d980 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 >> v1:172.21.15.29:6789/0 conn(0x7f7ed015d980 legacy=0x7f7ed006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed006dce0 0x7f7eb80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eb8003f40 0x7f7eb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb8004d50 0x7f7eb80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed00cf560 con 0x7f7eb8004d50 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed00b3410 con 0x7f7eb8003f40 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb800ba20 con 0x7f7ed006dce0 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecf7fe640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb8004d50 0x7f7eb80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ece7fc640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eb8003f40 0x7f7eb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7eceffd640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed006dce0 0x7f7eb80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecf7fe640 1 -- 172.21.15.29:0/4191375594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed006dce0 msgr2=0x7f7eb80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecf7fe640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed006dce0 0x7f7eb80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecf7fe640 1 -- 172.21.15.29:0/4191375594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eb8003f40 msgr2=0x7f7eb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecf7fe640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eb8003f40 0x7f7eb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecf7fe640 1 -- 172.21.15.29:0/4191375594 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb8003530 con 0x7f7eb8004d50 2024-04-25T00:41:01.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7eceffd640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed006dce0 0x7f7eb80039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:01.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ece7fc640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eb8003f40 0x7f7eb8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:01.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecf7fe640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb8004d50 0x7f7eb80051d0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000e170 tx=0x7f7ec001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45d325473b45ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec00b2070 con 0x7f7eb8004d50 2024-04-25T00:41:01.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec00a5cb0 con 0x7f7eb8004d50 2024-04-25T00:41:01.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.060+0000 7f7ecdffb640 1 -- 172.21.15.29:0/4191375594 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0009c10 con 0x7f7eb8004d50 2024-04-25T00:41:01.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 -- 172.21.15.29:0/4191375594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb8004d50 msgr2=0x7f7eb80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 --2- 172.21.15.29:0/4191375594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb8004d50 0x7f7eb80051d0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000e170 tx=0x7f7ec001cca0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ecf7fe640 1 -- 172.21.15.29:0/4191375594 reap_dead start 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 -- 172.21.15.29:0/4191375594 shutdown_connections 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 -- 172.21.15.29:0/4191375594 >> 172.21.15.29:0/4191375594 conn(0x7f7ed00b1870 msgr2=0x7f7ed00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 -- 172.21.15.29:0/4191375594 shutdown_connections 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 -- 172.21.15.29:0/4191375594 wait complete. 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 Processor -- start 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 -- start start 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed01479c0 0x7f7ed0151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed0147da0 0x7f7ed0153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ed0148d50 0x7f7ed0154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed015dd60 con 0x7f7ed0148d50 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed00b3cb0 con 0x7f7ed01479c0 2024-04-25T00:41:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed0151870 con 0x7f7ed0147da0 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ecf7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ed0148d50 0x7f7ed0154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ecf7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ed0148d50 0x7f7ed0154230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48782/0 (socket says 172.21.15.29:48782) 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ecf7fe640 1 -- 172.21.15.29:0/2313924177 learned_addr learned my addr 172.21.15.29:0/2313924177 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7eceffd640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed01479c0 0x7f7ed0151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ece7fc640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed0147da0 0x7f7ed0153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ecf7fe640 1 -- 172.21.15.29:0/2313924177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed0147da0 msgr2=0x7f7ed0153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ecf7fe640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed0147da0 0x7f7ed0153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ecf7fe640 1 -- 172.21.15.29:0/2313924177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed01479c0 msgr2=0x7f7ed0151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ecf7fe640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed01479c0 0x7f7ed0151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ecf7fe640 1 -- 172.21.15.29:0/2313924177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed00cca50 con 0x7f7ed0148d50 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7eceffd640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed01479c0 0x7f7ed0151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ecf7fe640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ed0148d50 0x7f7ed0154230 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009f90 tx=0x7f7ec0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7eb77fe640 1 -- 172.21.15.29:0/2313924177 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec00b2040 con 0x7f7ed0148d50 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7eb77fe640 1 -- 172.21.15.29:0/2313924177 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec00039a0 con 0x7f7ed0148d50 2024-04-25T00:41:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7eb77fe640 1 -- 172.21.15.29:0/2313924177 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec00a5e60 con 0x7f7ed0148d50 2024-04-25T00:41:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 -- 172.21.15.29:0/2313924177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed0072950 con 0x7f7ed0148d50 2024-04-25T00:41:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.064+0000 7f7ed5581640 1 -- 172.21.15.29:0/2313924177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed00ce720 con 0x7f7ed0148d50 2024-04-25T00:41:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.068+0000 7f7ed5581640 1 -- 172.21.15.29:0/2313924177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed0069270 con 0x7f7ed0148d50 2024-04-25T00:41:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.068+0000 7f7eb77fe640 1 -- 172.21.15.29:0/2313924177 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f7ec00ad8d0 con 0x7f7ed0148d50 2024-04-25T00:41:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.068+0000 7f7eb77fe640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7ebc068250 0x7f7ebc06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.068+0000 7f7eb77fe640 1 -- 172.21.15.29:0/2313924177 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f7ec011dd20 con 0x7f7ed0148d50 2024-04-25T00:41:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.068+0000 7f7eceffd640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7ebc068250 0x7f7ebc06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.068+0000 7f7eceffd640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7ebc068250 0x7f7ebc06a710 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8000c10 tx=0x7f7ec80014f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.072+0000 7f7eb77fe640 1 -- 172.21.15.29:0/2313924177 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7ec0016a30 con 0x7f7ed0148d50 2024-04-25T00:41:01.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.236+0000 7f7ed5581640 1 -- 172.21.15.29:0/2313924177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ed00ccec0 con 0x7f7ed0148d50 2024-04-25T00:41:01.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.236+0000 7f7eb77fe640 1 -- 172.21.15.29:0/2313924177 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16631 (secure 0 0 0) 0x7f7ec00efe10 con 0x7f7ed0148d50 2024-04-25T00:41:01.238 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:01.238 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":156,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:40:31.094463+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:40:27.796735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:40:28.430727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:01.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.240+0000 7f7ed5581640 1 -- 172.21.15.29:0/2313924177 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7ebc068250 msgr2=0x7f7ebc06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.240+0000 7f7ed5581640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7ebc068250 0x7f7ebc06a710 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8000c10 tx=0x7f7ec80014f0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.240+0000 7f7ed5581640 1 -- 172.21.15.29:0/2313924177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ed0148d50 msgr2=0x7f7ed0154230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.240+0000 7f7ed5581640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ed0148d50 0x7f7ed0154230 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009f90 tx=0x7f7ec0009750 comp rx=0 tx=0).stop 2024-04-25T00:41:01.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.240+0000 7f7ed5581640 1 -- 172.21.15.29:0/2313924177 shutdown_connections 2024-04-25T00:41:01.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.240+0000 7f7ed5581640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7ebc068250 0x7f7ebc06a710 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.240+0000 7f7ed5581640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ed0148d50 0x7f7ed0154230 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.240+0000 7f7ed5581640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed0147da0 0x7f7ed0153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.240+0000 7f7ed5581640 1 --2- 172.21.15.29:0/2313924177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed01479c0 0x7f7ed0151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.240+0000 7f7ed5581640 1 -- 172.21.15.29:0/2313924177 >> 172.21.15.29:0/2313924177 conn(0x7f7ed00b1870 msgr2=0x7f7ed014be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:01.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.240+0000 7f7ed5581640 1 -- 172.21.15.29:0/2313924177 shutdown_connections 2024-04-25T00:41:01.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.240+0000 7f7ed5581640 1 -- 172.21.15.29:0/2313924177 wait complete. 2024-04-25T00:41:01.258 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:41:01.259 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:41:01.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32be5b9640 1 Processor -- start 2024-04-25T00:41:01.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32be5b9640 1 -- start start 2024-04-25T00:41:01.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32be5b9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f32b80b3140 0x7f32b806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32be5b9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b80b4b10 0x7f32b80b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32be5b9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b80b41e0 con 0x7f32b80b3a60 2024-04-25T00:41:01.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32be5b9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f32b806d960 con 0x7f32b80b3140 2024-04-25T00:41:01.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32be5b9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f32b806dc40 con 0x7f32b80b4b10 2024-04-25T00:41:01.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32b7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f32b80b3140 0x7f32b806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32b7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f32b80b3140 0x7f32b806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48790/0 (socket says 172.21.15.29:48790) 2024-04-25T00:41:01.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32b7fff640 1 -- 172.21.15.29:0/1937765886 learned_addr learned my addr 172.21.15.29:0/1937765886 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:01.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32b77fe640 1 --2- 172.21.15.29:0/1937765886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b80b4b10 0x7f32b80b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32b7fff640 1 -- 172.21.15.29:0/1937765886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b80b4b10 msgr2=0x7f32b80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32b7fff640 1 --2- 172.21.15.29:0/1937765886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b80b4b10 0x7f32b80b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32b7fff640 1 -- 172.21.15.29:0/1937765886 >> v1:172.21.15.157:6789/0 conn(0x7f32b80b3a60 legacy=0x7f32b815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32b7fff640 1 -- 172.21.15.29:0/1937765886 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b806df20 con 0x7f32b80b3140 2024-04-25T00:41:01.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.376+0000 7f32b77fe640 1 --2- 172.21.15.29:0/1937765886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b80b4b10 0x7f32b80b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:01.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32b7fff640 1 --2- 172.21.15.29:0/1937765886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32b80b3140 0x7f32b806e440 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009870 tx=0x7f32a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c9dd2e1689266f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32b6ffd640 1 -- 172.21.15.29:0/1937765886 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a80b0070 con 0x7f32b80b3140 2024-04-25T00:41:01.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32b6ffd640 1 -- 172.21.15.29:0/1937765886 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a80ab600 con 0x7f32b80b3140 2024-04-25T00:41:01.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32b6ffd640 1 -- 172.21.15.29:0/1937765886 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a80a4e60 con 0x7f32b80b3140 2024-04-25T00:41:01.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32be5b9640 1 -- 172.21.15.29:0/1937765886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32b80b3140 msgr2=0x7f32b806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32be5b9640 1 --2- 172.21.15.29:0/1937765886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32b80b3140 0x7f32b806e440 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009870 tx=0x7f32a801c520 comp rx=0 tx=0).stop 2024-04-25T00:41:01.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32be5b9640 1 -- 172.21.15.29:0/1937765886 shutdown_connections 2024-04-25T00:41:01.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32be5b9640 1 --2- 172.21.15.29:0/1937765886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b80b4b10 0x7f32b80b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32be5b9640 1 --2- 172.21.15.29:0/1937765886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32b80b3140 0x7f32b806e440 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32be5b9640 1 -- 172.21.15.29:0/1937765886 >> 172.21.15.29:0/1937765886 conn(0x7f32b80b1870 msgr2=0x7f32b80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:01.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32be5b9640 1 -- 172.21.15.29:0/1937765886 shutdown_connections 2024-04-25T00:41:01.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32be5b9640 1 -- 172.21.15.29:0/1937765886 wait complete. 2024-04-25T00:41:01.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32be5b9640 1 Processor -- start 2024-04-25T00:41:01.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.380+0000 7f32be5b9640 1 -- start start 2024-04-25T00:41:01.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32be5b9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b80b3140 0x7f32b813e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32be5b9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32b80b3a60 0x7f32b8139250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32be5b9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b80b4b10 0x7f32b8143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32be5b9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8168b70 con 0x7f32b80b3a60 2024-04-25T00:41:01.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32be5b9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81689f0 con 0x7f32b80b3140 2024-04-25T00:41:01.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32be5b9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f32b8168cf0 con 0x7f32b80b4b10 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32b80b3a60 0x7f32b8139250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32b80b3a60 0x7f32b8139250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48800/0 (socket says 172.21.15.29:48800) 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b77fe640 1 -- 172.21.15.29:0/2257219035 learned_addr learned my addr 172.21.15.29:0/2257219035 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b77fe640 1 -- 172.21.15.29:0/2257219035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b80b4b10 msgr2=0x7f32b8143ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b77fe640 1 --2- 172.21.15.29:0/2257219035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b80b4b10 0x7f32b8143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b77fe640 1 -- 172.21.15.29:0/2257219035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b80b3140 msgr2=0x7f32b813e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b77fe640 1 --2- 172.21.15.29:0/2257219035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b80b3140 0x7f32b813e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b77fe640 1 -- 172.21.15.29:0/2257219035 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b8139790 con 0x7f32b80b3a60 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b77fe640 1 --2- 172.21.15.29:0/2257219035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32b80b3a60 0x7f32b8139250 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00c820 tx=0x7f32ac00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b4ff9640 1 -- 172.21.15.29:0/2257219035 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac015070 con 0x7f32b80b3a60 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b4ff9640 1 -- 172.21.15.29:0/2257219035 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32ac007a30 con 0x7f32b80b3a60 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32be5b9640 1 -- 172.21.15.29:0/2257219035 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b8139a20 con 0x7f32b80b3a60 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b4ff9640 1 -- 172.21.15.29:0/2257219035 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac007e40 con 0x7f32b80b3a60 2024-04-25T00:41:01.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32be5b9640 1 -- 172.21.15.29:0/2257219035 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b8139fe0 con 0x7f32b80b3a60 2024-04-25T00:41:01.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32be5b9640 1 -- 172.21.15.29:0/2257219035 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b8069270 con 0x7f32b80b3a60 2024-04-25T00:41:01.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b4ff9640 1 -- 172.21.15.29:0/2257219035 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f32ac018070 con 0x7f32b80b3a60 2024-04-25T00:41:01.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b4ff9640 1 --2- 172.21.15.29:0/2257219035 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f32a4068250 0x7f32a406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b4ff9640 1 -- 172.21.15.29:0/2257219035 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f32ac0842c0 con 0x7f32b80b3a60 2024-04-25T00:41:01.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b7fff640 1 --2- 172.21.15.29:0/2257219035 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f32a4068250 0x7f32a406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.384+0000 7f32b7fff640 1 --2- 172.21.15.29:0/2257219035 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f32a4068250 0x7f32a406a710 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f32a80ad000 tx=0x7f32a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.388+0000 7f32b4ff9640 1 -- 172.21.15.29:0/2257219035 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f32ac056a00 con 0x7f32b80b3a60 2024-04-25T00:41:01.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.580+0000 7f32be5b9640 1 -- 172.21.15.29:0/2257219035 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32b81559e0 con 0x7f32b80b3a60 2024-04-25T00:41:01.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.580+0000 7f32b4ff9640 1 -- 172.21.15.29:0/2257219035 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5196 (secure 0 0 0) 0x7f32ac056300 con 0x7f32b80b3a60 2024-04-25T00:41:01.582 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:01.582 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2677876688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2677876688},{"type":"v1","addr":"172.21.15.157:6833","nonce":2677876688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3340901266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3340901266},{"type":"v1","addr":"172.21.15.157:6835","nonce":3340901266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":10092,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/567284262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":567284262},{"type":"v1","addr":"172.21.15.29:6833","nonce":567284262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:40:29.419450+0000","modified":"2024-04-25T00:40:32.484311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10095":{"gid":10095,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/4183931647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":4183931647},{"type":"v1","addr":"172.21.15.29:6835","nonce":4183931647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10095,"qdb_cluster":[10095]},"id":12}]} 2024-04-25T00:41:01.583 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 103 2024-04-25T00:41:01.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.584+0000 7f32be5b9640 1 -- 172.21.15.29:0/2257219035 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f32a4068250 msgr2=0x7f32a406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.584+0000 7f32be5b9640 1 --2- 172.21.15.29:0/2257219035 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f32a4068250 0x7f32a406a710 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f32a80ad000 tx=0x7f32a8008040 comp rx=0 tx=0).stop 2024-04-25T00:41:01.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.584+0000 7f32be5b9640 1 -- 172.21.15.29:0/2257219035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32b80b3a60 msgr2=0x7f32b8139250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.584+0000 7f32be5b9640 1 --2- 172.21.15.29:0/2257219035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32b80b3a60 0x7f32b8139250 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00c820 tx=0x7f32ac00cce0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.584+0000 7f32be5b9640 1 -- 172.21.15.29:0/2257219035 shutdown_connections 2024-04-25T00:41:01.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.584+0000 7f32be5b9640 1 --2- 172.21.15.29:0/2257219035 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f32a4068250 0x7f32a406a710 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.584+0000 7f32be5b9640 1 --2- 172.21.15.29:0/2257219035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b80b4b10 0x7f32b8143ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.584+0000 7f32be5b9640 1 --2- 172.21.15.29:0/2257219035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32b80b3a60 0x7f32b8139250 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.584+0000 7f32be5b9640 1 --2- 172.21.15.29:0/2257219035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b80b3140 0x7f32b813e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.584+0000 7f32be5b9640 1 -- 172.21.15.29:0/2257219035 >> 172.21.15.29:0/2257219035 conn(0x7f32b80b1870 msgr2=0x7f32b8154f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:01.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.584+0000 7f32be5b9640 1 -- 172.21.15.29:0/2257219035 shutdown_connections 2024-04-25T00:41:01.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.584+0000 7f32be5b9640 1 -- 172.21.15.29:0/2257219035 wait complete. 2024-04-25T00:41:01.600 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:41:01.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab84dd4640 1 Processor -- start 2024-04-25T00:41:01.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab84dd4640 1 -- start start 2024-04-25T00:41:01.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab84dd4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fab8006ea50 0x7fab800d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab84dd4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab8006ef10 0x7fab800d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab84dd4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab800722e0 con 0x7fab8015be30 2024-04-25T00:41:01.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab84dd4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fab8015fa90 con 0x7fab8006ef10 2024-04-25T00:41:01.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab84dd4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fab800ccd80 con 0x7fab8006ea50 2024-04-25T00:41:01.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab7ed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab8006ef10 0x7fab800d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab7ed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab8006ef10 0x7fab800d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48812/0 (socket says 172.21.15.29:48812) 2024-04-25T00:41:01.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab7ed9d640 1 -- 172.21.15.29:0/3446413977 learned_addr learned my addr 172.21.15.29:0/3446413977 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:01.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab7ed9d640 1 -- 172.21.15.29:0/3446413977 >> v2:172.21.15.157:3300/0 conn(0x7fab8006ea50 msgr2=0x7fab800d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab7ed9d640 1 --2- 172.21.15.29:0/3446413977 >> v2:172.21.15.157:3300/0 conn(0x7fab8006ea50 0x7fab800d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab7ed9d640 1 -- 172.21.15.29:0/3446413977 >> v1:172.21.15.157:6789/0 conn(0x7fab8015be30 legacy=0x7fab800ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab7ed9d640 1 -- 172.21.15.29:0/3446413977 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab800cd060 con 0x7fab8006ef10 2024-04-25T00:41:01.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.712+0000 7fab7ed9d640 1 --2- 172.21.15.29:0/3446413977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab8006ef10 0x7fab800d0690 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fab800746a0 tx=0x7fab7801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c5fb20cebdeb95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab7dd9b640 1 -- 172.21.15.29:0/3446413977 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab780a5800 con 0x7fab8006ef10 2024-04-25T00:41:01.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab7dd9b640 1 -- 172.21.15.29:0/3446413977 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab780a59a0 con 0x7fab8006ef10 2024-04-25T00:41:01.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab7dd9b640 1 -- 172.21.15.29:0/3446413977 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab780a5d70 con 0x7fab8006ef10 2024-04-25T00:41:01.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 -- 172.21.15.29:0/3446413977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab8006ef10 msgr2=0x7fab800d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 --2- 172.21.15.29:0/3446413977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab8006ef10 0x7fab800d0690 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fab800746a0 tx=0x7fab7801c980 comp rx=0 tx=0).stop 2024-04-25T00:41:01.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 -- 172.21.15.29:0/3446413977 shutdown_connections 2024-04-25T00:41:01.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 --2- 172.21.15.29:0/3446413977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab8006ef10 0x7fab800d0690 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 --2- 172.21.15.29:0/3446413977 >> v2:172.21.15.157:3300/0 conn(0x7fab8006ea50 0x7fab800d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 -- 172.21.15.29:0/3446413977 >> 172.21.15.29:0/3446413977 conn(0x7fab80074c00 msgr2=0x7fab8006b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:01.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 -- 172.21.15.29:0/3446413977 shutdown_connections 2024-04-25T00:41:01.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 -- 172.21.15.29:0/3446413977 wait complete. 2024-04-25T00:41:01.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 Processor -- start 2024-04-25T00:41:01.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 -- start start 2024-04-25T00:41:01.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab8006ea50 0x7fab8015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab8006ef10 0x7fab800b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab8015be30 0x7fab80151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fab80168da0 con 0x7fab8006ef10 2024-04-25T00:41:01.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab80071f70 con 0x7fab8006ea50 2024-04-25T00:41:01.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fab80168f20 con 0x7fab8015be30 2024-04-25T00:41:01.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab77fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab8006ef10 0x7fab800b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab77fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab8006ef10 0x7fab800b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48816/0 (socket says 172.21.15.29:48816) 2024-04-25T00:41:01.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab77fff640 1 -- 172.21.15.29:0/3250149407 learned_addr learned my addr 172.21.15.29:0/3250149407 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:01.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab7e59c640 1 --2- 172.21.15.29:0/3250149407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab8006ea50 0x7fab8015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab77fff640 1 -- 172.21.15.29:0/3250149407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab8015be30 msgr2=0x7fab80151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab77fff640 1 --2- 172.21.15.29:0/3250149407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab8015be30 0x7fab80151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab77fff640 1 -- 172.21.15.29:0/3250149407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab8006ea50 msgr2=0x7fab8015b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab77fff640 1 --2- 172.21.15.29:0/3250149407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab8006ea50 0x7fab8015b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab77fff640 1 -- 172.21.15.29:0/3250149407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab80152040 con 0x7fab8006ef10 2024-04-25T00:41:01.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab77fff640 1 --2- 172.21.15.29:0/3250149407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab8006ef10 0x7fab800b0e50 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fab6800b550 tx=0x7fab6800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab76ffd640 1 -- 172.21.15.29:0/3250149407 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68014070 con 0x7fab8006ef10 2024-04-25T00:41:01.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab76ffd640 1 -- 172.21.15.29:0/3250149407 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab68010890 con 0x7fab8006ef10 2024-04-25T00:41:01.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 -- 172.21.15.29:0/3250149407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab800b2b90 con 0x7fab8006ef10 2024-04-25T00:41:01.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab76ffd640 1 -- 172.21.15.29:0/3250149407 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68010c80 con 0x7fab8006ef10 2024-04-25T00:41:01.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.716+0000 7fab84dd4640 1 -- 172.21.15.29:0/3250149407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab800b0370 con 0x7fab8006ef10 2024-04-25T00:41:01.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.720+0000 7fab84dd4640 1 -- 172.21.15.29:0/3250149407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab80069270 con 0x7fab8006ef10 2024-04-25T00:41:01.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.720+0000 7fab76ffd640 1 -- 172.21.15.29:0/3250149407 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fab68017070 con 0x7fab8006ef10 2024-04-25T00:41:01.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.720+0000 7fab76ffd640 1 --2- 172.21.15.29:0/3250149407 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab60068250 0x7fab6006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.720+0000 7fab76ffd640 1 -- 172.21.15.29:0/3250149407 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fab68084710 con 0x7fab8006ef10 2024-04-25T00:41:01.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.720+0000 7fab7e59c640 1 --2- 172.21.15.29:0/3250149407 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab60068250 0x7fab6006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.720+0000 7fab7e59c640 1 --2- 172.21.15.29:0/3250149407 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab60068250 0x7fab6006a710 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fab7000a790 tx=0x7fab70008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.724+0000 7fab76ffd640 1 -- 172.21.15.29:0/3250149407 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fab68056e50 con 0x7fab8006ef10 2024-04-25T00:41:01.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.912+0000 7fab84dd4640 1 -- 172.21.15.29:0/3250149407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab800746d0 con 0x7fab8006ef10 2024-04-25T00:41:01.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.912+0000 7fab76ffd640 1 -- 172.21.15.29:0/3250149407 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5196 (secure 0 0 0) 0x7fab68010e20 con 0x7fab8006ef10 2024-04-25T00:41:01.914 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:01.914 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2677876688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2677876688},{"type":"v1","addr":"172.21.15.157:6833","nonce":2677876688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3340901266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3340901266},{"type":"v1","addr":"172.21.15.157:6835","nonce":3340901266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":10092,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/567284262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":567284262},{"type":"v1","addr":"172.21.15.29:6833","nonce":567284262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:40:29.419450+0000","modified":"2024-04-25T00:40:32.484311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10095":{"gid":10095,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/4183931647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":4183931647},{"type":"v1","addr":"172.21.15.29:6835","nonce":4183931647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10095,"qdb_cluster":[10095]},"id":12}]} 2024-04-25T00:41:01.914 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 103 2024-04-25T00:41:01.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.916+0000 7fab84dd4640 1 -- 172.21.15.29:0/3250149407 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab60068250 msgr2=0x7fab6006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.916+0000 7fab84dd4640 1 --2- 172.21.15.29:0/3250149407 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab60068250 0x7fab6006a710 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fab7000a790 tx=0x7fab70008040 comp rx=0 tx=0).stop 2024-04-25T00:41:01.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.916+0000 7fab84dd4640 1 -- 172.21.15.29:0/3250149407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab8006ef10 msgr2=0x7fab800b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.916+0000 7fab84dd4640 1 --2- 172.21.15.29:0/3250149407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab8006ef10 0x7fab800b0e50 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fab6800b550 tx=0x7fab6800ba10 comp rx=0 tx=0).stop 2024-04-25T00:41:01.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.916+0000 7fab84dd4640 1 -- 172.21.15.29:0/3250149407 shutdown_connections 2024-04-25T00:41:01.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.916+0000 7fab84dd4640 1 --2- 172.21.15.29:0/3250149407 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fab60068250 0x7fab6006a710 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.916+0000 7fab84dd4640 1 --2- 172.21.15.29:0/3250149407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab8015be30 0x7fab80151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.916+0000 7fab84dd4640 1 --2- 172.21.15.29:0/3250149407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab8006ef10 0x7fab800b0e50 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.916+0000 7fab84dd4640 1 --2- 172.21.15.29:0/3250149407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab8006ea50 0x7fab8015b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.916+0000 7fab84dd4640 1 -- 172.21.15.29:0/3250149407 >> 172.21.15.29:0/3250149407 conn(0x7fab80074c00 msgr2=0x7fab80157e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:01.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.916+0000 7fab84dd4640 1 -- 172.21.15.29:0/3250149407 shutdown_connections 2024-04-25T00:41:01.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:01.916+0000 7fab84dd4640 1 -- 172.21.15.29:0/3250149407 wait complete. 2024-04-25T00:41:01.932 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:02.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e2b955640 1 Processor -- start 2024-04-25T00:41:02.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e2b955640 1 -- start start 2024-04-25T00:41:02.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e2b955640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9e240b3140 0x7f9e2406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e2b955640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e240b4b10 0x7f9e240b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e2b955640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e240b3a60 0x7f9e2415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e2b955640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9e240b41e0 con 0x7f9e240b3140 2024-04-25T00:41:02.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e2b955640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9e2406d960 con 0x7f9e240b4b10 2024-04-25T00:41:02.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e2b955640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9e2406dc40 con 0x7f9e240b3a60 2024-04-25T00:41:02.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e296f1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9e240b3140 0x7f9e2406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e296f1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9e240b3140 0x7f9e2406e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48832/0 (socket says 172.21.15.29:48832) 2024-04-25T00:41:02.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e296f1640 1 -- 172.21.15.29:0/3388328166 learned_addr learned my addr 172.21.15.29:0/3388328166 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:02.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e29ef2640 1 --2- 172.21.15.29:0/3388328166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e240b3a60 0x7f9e2415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e28ef0640 1 --2- 172.21.15.29:0/3388328166 >> v2:172.21.15.157:3300/0 conn(0x7f9e240b4b10 0x7f9e240b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e296f1640 1 -- 172.21.15.29:0/3388328166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e240b3a60 msgr2=0x7f9e2415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:02.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e296f1640 1 --2- 172.21.15.29:0/3388328166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e240b3a60 0x7f9e2415c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e296f1640 1 -- 172.21.15.29:0/3388328166 >> v2:172.21.15.157:3300/0 conn(0x7f9e240b4b10 msgr2=0x7f9e240b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:02.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e296f1640 1 --2- 172.21.15.29:0/3388328166 >> v2:172.21.15.157:3300/0 conn(0x7f9e240b4b10 0x7f9e240b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.052+0000 7f9e296f1640 1 -- 172.21.15.29:0/3388328166 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2406df20 con 0x7f9e240b3140 2024-04-25T00:41:02.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e296f1640 1 --2- 172.21.15.29:0/3388328166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e240b3140 0x7f9e2406e440 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009870 tx=0x7f9e2001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7981bfc184a33750 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:02.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e1bfff640 1 -- 172.21.15.29:0/3388328166 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e200b0070 con 0x7f9e240b3140 2024-04-25T00:41:02.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e1bfff640 1 -- 172.21.15.29:0/3388328166 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e200ab600 con 0x7f9e240b3140 2024-04-25T00:41:02.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e1bfff640 1 -- 172.21.15.29:0/3388328166 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e200ab9d0 con 0x7f9e240b3140 2024-04-25T00:41:02.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 -- 172.21.15.29:0/3388328166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e240b3140 msgr2=0x7f9e2406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 --2- 172.21.15.29:0/3388328166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e240b3140 0x7f9e2406e440 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009870 tx=0x7f9e2001c520 comp rx=0 tx=0).stop 2024-04-25T00:41:02.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 -- 172.21.15.29:0/3388328166 shutdown_connections 2024-04-25T00:41:02.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 --2- 172.21.15.29:0/3388328166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e240b3a60 0x7f9e2415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 --2- 172.21.15.29:0/3388328166 >> v2:172.21.15.157:3300/0 conn(0x7f9e240b4b10 0x7f9e240b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 --2- 172.21.15.29:0/3388328166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e240b3140 0x7f9e2406e440 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 -- 172.21.15.29:0/3388328166 >> 172.21.15.29:0/3388328166 conn(0x7f9e240b1870 msgr2=0x7f9e240b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:02.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 -- 172.21.15.29:0/3388328166 shutdown_connections 2024-04-25T00:41:02.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 -- 172.21.15.29:0/3388328166 wait complete. 2024-04-25T00:41:02.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 Processor -- start 2024-04-25T00:41:02.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 -- start start 2024-04-25T00:41:02.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e240b3140 0x7f9e24144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e240b3a60 0x7f9e24137850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e240b4b10 0x7f9e24145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9e240ce3c0 con 0x7f9e240b3a60 2024-04-25T00:41:02.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9e240ce240 con 0x7f9e240b3140 2024-04-25T00:41:02.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e2b955640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9e240ce540 con 0x7f9e240b4b10 2024-04-25T00:41:02.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e28ef0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e240b3a60 0x7f9e24137850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e28ef0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e240b3a60 0x7f9e24137850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48836/0 (socket says 172.21.15.29:48836) 2024-04-25T00:41:02.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.056+0000 7f9e28ef0640 1 -- 172.21.15.29:0/3970463891 learned_addr learned my addr 172.21.15.29:0/3970463891 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:02.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e296f1640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e240b3140 0x7f9e24144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e29ef2640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e240b4b10 0x7f9e24145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e28ef0640 1 -- 172.21.15.29:0/3970463891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e240b4b10 msgr2=0x7f9e24145b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e28ef0640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e240b4b10 0x7f9e24145b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e28ef0640 1 -- 172.21.15.29:0/3970463891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e240b3140 msgr2=0x7f9e24144260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e28ef0640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e240b3140 0x7f9e24144260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e28ef0640 1 -- 172.21.15.29:0/3970463891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e24146060 con 0x7f9e240b3a60 2024-04-25T00:41:02.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e29ef2640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e240b4b10 0x7f9e24145b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:02.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e28ef0640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e240b3a60 0x7f9e24137850 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400d5f0 tx=0x7f9e1400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:02.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e19ffb640 1 -- 172.21.15.29:0/3970463891 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e140125f0 con 0x7f9e240b3a60 2024-04-25T00:41:02.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e19ffb640 1 -- 172.21.15.29:0/3970463891 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e14019070 con 0x7f9e240b3a60 2024-04-25T00:41:02.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e2b955640 1 -- 172.21.15.29:0/3970463891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e24146300 con 0x7f9e240b3a60 2024-04-25T00:41:02.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e19ffb640 1 -- 172.21.15.29:0/3970463891 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e140129b0 con 0x7f9e240b3a60 2024-04-25T00:41:02.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e2b955640 1 -- 172.21.15.29:0/3970463891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e241468c0 con 0x7f9e240b3a60 2024-04-25T00:41:02.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e2b955640 1 -- 172.21.15.29:0/3970463891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e24069270 con 0x7f9e240b3a60 2024-04-25T00:41:02.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e19ffb640 1 -- 172.21.15.29:0/3970463891 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f9e14012b50 con 0x7f9e240b3a60 2024-04-25T00:41:02.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e19ffb640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9e08068280 0x7f9e0806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e296f1640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9e08068280 0x7f9e0806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e19ffb640 1 -- 172.21.15.29:0/3970463891 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f9e140847f0 con 0x7f9e240b3a60 2024-04-25T00:41:02.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.060+0000 7f9e296f1640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9e08068280 0x7f9e0806a740 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9e200ad000 tx=0x7f9e20008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:02.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.064+0000 7f9e19ffb640 1 -- 172.21.15.29:0/3970463891 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9e14056f30 con 0x7f9e240b3a60 2024-04-25T00:41:02.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.232+0000 7f9e2b955640 1 -- 172.21.15.29:0/3970463891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e2415ad80 con 0x7f9e240b3a60 2024-04-25T00:41:02.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.232+0000 7f9e19ffb640 1 -- 172.21.15.29:0/3970463891 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16631 (secure 0 0 0) 0x7f9e14056830 con 0x7f9e240b3a60 2024-04-25T00:41:02.234 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:02.235 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":156,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:40:31.094463+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:40:27.796735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:40:28.430727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:02.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.236+0000 7f9e2b955640 1 -- 172.21.15.29:0/3970463891 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9e08068280 msgr2=0x7f9e0806a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.236+0000 7f9e2b955640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9e08068280 0x7f9e0806a740 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9e200ad000 tx=0x7f9e20008040 comp rx=0 tx=0).stop 2024-04-25T00:41:02.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.236+0000 7f9e2b955640 1 -- 172.21.15.29:0/3970463891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e240b3a60 msgr2=0x7f9e24137850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.236+0000 7f9e2b955640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e240b3a60 0x7f9e24137850 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400d5f0 tx=0x7f9e1400dab0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.236+0000 7f9e2b955640 1 -- 172.21.15.29:0/3970463891 shutdown_connections 2024-04-25T00:41:02.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.236+0000 7f9e2b955640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9e08068280 0x7f9e0806a740 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.236+0000 7f9e2b955640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e240b4b10 0x7f9e24145b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.236+0000 7f9e2b955640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e240b3a60 0x7f9e24137850 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.236+0000 7f9e2b955640 1 --2- 172.21.15.29:0/3970463891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e240b3140 0x7f9e24144260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.236+0000 7f9e2b955640 1 -- 172.21.15.29:0/3970463891 >> 172.21.15.29:0/3970463891 conn(0x7f9e240b1870 msgr2=0x7f9e2415b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:02.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.236+0000 7f9e2b955640 1 -- 172.21.15.29:0/3970463891 shutdown_connections 2024-04-25T00:41:02.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.236+0000 7f9e2b955640 1 -- 172.21.15.29:0/3970463891 wait complete. 2024-04-25T00:41:02.254 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:41:02.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bc0251640 1 Processor -- start 2024-04-25T00:41:02.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bc0251640 1 -- start start 2024-04-25T00:41:02.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bc0251640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3bb80b2b70 0x7f3bb8151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bc0251640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb80b3130 con 0x7f3bb806dce0 2024-04-25T00:41:02.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bc0251640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb80b3410 con 0x7f3bb815d980 2024-04-25T00:41:02.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bc0251640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb80cf560 con 0x7f3bb80b2b70 2024-04-25T00:41:02.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbd7ec640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3bb806dce0 0x7f3bb8150df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33974/0 (socket says 172.21.15.29:33974) 2024-04-25T00:41:02.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbd7ec640 1 -- 172.21.15.29:0/130473834 learned_addr learned my addr 172.21.15.29:0/130473834 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:02.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbe7ee640 1 --2- 172.21.15.29:0/130473834 >> v2:172.21.15.157:3300/0 conn(0x7f3bb80b2b70 0x7f3bb8151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4186669160 0 0) 0x7f3bb80b3130 con 0x7f3bb806dce0 2024-04-25T00:41:02.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ba4003530 con 0x7f3bb806dce0 2024-04-25T00:41:02.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2381548502 0 0) 0x7f3ba4003530 con 0x7f3bb806dce0 2024-04-25T00:41:02.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 >> v2:172.21.15.157:3300/0 conn(0x7f3bb80b2b70 msgr2=0x7f3bb8151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:02.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 --2- 172.21.15.29:0/130473834 >> v2:172.21.15.157:3300/0 conn(0x7f3bb80b2b70 0x7f3bb8151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 >> v1:172.21.15.157:6789/0 conn(0x7f3bb815d980 legacy=0x7f3bb806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb80b36f0 con 0x7f3bb806dce0 2024-04-25T00:41:02.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f3ba8001680 con 0x7f3bb806dce0 2024-04-25T00:41:02.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 >> v1:172.21.15.29:6789/0 conn(0x7f3bb806dce0 legacy=0x7f3bb8150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 --2- 172.21.15.29:0/130473834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ba40039c0 0x7f3ba4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 --2- 172.21.15.29:0/130473834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4004810 0x7f3ba4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 --2- 172.21.15.29:0/130473834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba40051d0 0x7f3ba400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb80cf560 con 0x7f3ba4004810 2024-04-25T00:41:02.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba400d180 con 0x7f3ba40051d0 2024-04-25T00:41:02.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba400d380 con 0x7f3ba40039c0 2024-04-25T00:41:02.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbd7ec640 1 --2- 172.21.15.29:0/130473834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4004810 0x7f3ba4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbe7ee640 1 --2- 172.21.15.29:0/130473834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba40051d0 0x7f3ba400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbdfed640 1 --2- 172.21.15.29:0/130473834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ba40039c0 0x7f3ba4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbd7ec640 1 -- 172.21.15.29:0/130473834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ba40039c0 msgr2=0x7f3ba4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:02.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbd7ec640 1 --2- 172.21.15.29:0/130473834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ba40039c0 0x7f3ba4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbd7ec640 1 -- 172.21.15.29:0/130473834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba40051d0 msgr2=0x7f3ba400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:02.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbd7ec640 1 --2- 172.21.15.29:0/130473834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba40051d0 0x7f3ba400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbd7ec640 1 -- 172.21.15.29:0/130473834 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba4003530 con 0x7f3ba4004810 2024-04-25T00:41:02.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.372+0000 7f3bbe7ee640 1 --2- 172.21.15.29:0/130473834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba40051d0 0x7f3ba400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:02.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.376+0000 7f3bbe7ee640 1 -- 172.21.15.29:0/130473834 reap_dead start 2024-04-25T00:41:02.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.376+0000 7f3bbd7ec640 1 --2- 172.21.15.29:0/130473834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4004810 0x7f3ba4004c90 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c1c0 tx=0x7f3ba801f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f66b34ca1876fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:02.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.376+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8007d00 con 0x7f3ba4004810 2024-04-25T00:41:02.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.376+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba8005b00 con 0x7f3ba4004810 2024-04-25T00:41:02.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.376+0000 7f3bbcfeb640 1 -- 172.21.15.29:0/130473834 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8007070 con 0x7f3ba4004810 2024-04-25T00:41:02.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.376+0000 7f3bc0251640 1 -- 172.21.15.29:0/130473834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4004810 msgr2=0x7f3ba4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.376+0000 7f3bc0251640 1 --2- 172.21.15.29:0/130473834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4004810 0x7f3ba4004c90 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c1c0 tx=0x7f3ba801f390 comp rx=0 tx=0).stop 2024-04-25T00:41:02.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.376+0000 7f3bc0251640 1 -- 172.21.15.29:0/130473834 shutdown_connections 2024-04-25T00:41:02.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.376+0000 7f3bc0251640 1 --2- 172.21.15.29:0/130473834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4004810 0x7f3ba4004c90 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.376+0000 7f3bc0251640 1 -- 172.21.15.29:0/130473834 >> 172.21.15.29:0/130473834 conn(0x7f3bb80b1870 msgr2=0x7f3bb80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:02.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.376+0000 7f3bc0251640 1 -- 172.21.15.29:0/130473834 shutdown_connections 2024-04-25T00:41:02.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.376+0000 7f3bc0251640 1 -- 172.21.15.29:0/130473834 wait complete. 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bc0251640 1 Processor -- start 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bc0251640 1 -- start start 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bc0251640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4004810 0x7f3bb81527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bc0251640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb8151180 0x7f3bb8153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bc0251640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bb8147730 0x7f3bb81568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bc0251640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb815dd60 con 0x7f3ba4004810 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bc0251640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb80b3cb0 con 0x7f3bb8147730 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bc0251640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb8150df0 con 0x7f3bb8151180 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bbdfed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4004810 0x7f3bb81527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bbdfed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4004810 0x7f3bb81527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48860/0 (socket says 172.21.15.29:48860) 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bbdfed640 1 -- 172.21.15.29:0/1860538259 learned_addr learned my addr 172.21.15.29:0/1860538259 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bbd7ec640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb8151180 0x7f3bb8153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bbe7ee640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bb8147730 0x7f3bb81568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bbdfed640 1 -- 172.21.15.29:0/1860538259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb8151180 msgr2=0x7f3bb8153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bbdfed640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb8151180 0x7f3bb8153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bbdfed640 1 -- 172.21.15.29:0/1860538259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bb8147730 msgr2=0x7f3bb81568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bbdfed640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bb8147730 0x7f3bb81568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bbdfed640 1 -- 172.21.15.29:0/1860538259 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb8151720 con 0x7f3ba4004810 2024-04-25T00:41:02.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bbdfed640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4004810 0x7f3bb81527c0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400e4d0 tx=0x7f3bb400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:02.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bae7fc640 1 -- 172.21.15.29:0/1860538259 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40053a0 con 0x7f3ba4004810 2024-04-25T00:41:02.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bae7fc640 1 -- 172.21.15.29:0/1860538259 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb4018070 con 0x7f3ba4004810 2024-04-25T00:41:02.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bc0251640 1 -- 172.21.15.29:0/1860538259 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb8151a00 con 0x7f3ba4004810 2024-04-25T00:41:02.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bae7fc640 1 -- 172.21.15.29:0/1860538259 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4003820 con 0x7f3ba4004810 2024-04-25T00:41:02.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bc0251640 1 -- 172.21.15.29:0/1860538259 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb8151fc0 con 0x7f3ba4004810 2024-04-25T00:41:02.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bc0251640 1 -- 172.21.15.29:0/1860538259 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb8069270 con 0x7f3ba4004810 2024-04-25T00:41:02.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bae7fc640 1 -- 172.21.15.29:0/1860538259 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f3bb40039c0 con 0x7f3ba4004810 2024-04-25T00:41:02.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.380+0000 7f3bae7fc640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3b9c068280 0x7f3b9c06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.384+0000 7f3bbd7ec640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3b9c068280 0x7f3b9c06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.384+0000 7f3bae7fc640 1 -- 172.21.15.29:0/1860538259 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f3bb40840f0 con 0x7f3ba4004810 2024-04-25T00:41:02.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.384+0000 7f3bbd7ec640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3b9c068280 0x7f3b9c06a740 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80ae000 tx=0x7f3ba8007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:02.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.388+0000 7f3bae7fc640 1 -- 172.21.15.29:0/1860538259 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3bb4056830 con 0x7f3ba4004810 2024-04-25T00:41:02.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.576+0000 7f3bc0251640 1 -- 172.21.15.29:0/1860538259 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3bb80cf910 con 0x7f3ba4004810 2024-04-25T00:41:02.579 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-04-25T00:41:02.580+0000 7f3ac00b5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:41:02.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.580+0000 7f3bae7fc640 1 -- 172.21.15.29:0/1860538259 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v104) ==== 161+0+0 (secure 0 0 0) 0x7f3bb4056130 con 0x7f3ba4004810 2024-04-25T00:41:02.582 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:41:02.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.584+0000 7f3bc0251640 1 -- 172.21.15.29:0/1860538259 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3b9c068280 msgr2=0x7f3b9c06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.584+0000 7f3bc0251640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3b9c068280 0x7f3b9c06a740 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80ae000 tx=0x7f3ba8007040 comp rx=0 tx=0).stop 2024-04-25T00:41:02.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.584+0000 7f3bc0251640 1 -- 172.21.15.29:0/1860538259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4004810 msgr2=0x7f3bb81527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.584+0000 7f3bc0251640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4004810 0x7f3bb81527c0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400e4d0 tx=0x7f3bb400e990 comp rx=0 tx=0).stop 2024-04-25T00:41:02.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.584+0000 7f3bc0251640 1 -- 172.21.15.29:0/1860538259 shutdown_connections 2024-04-25T00:41:02.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.584+0000 7f3bc0251640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3b9c068280 0x7f3b9c06a740 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.584+0000 7f3bc0251640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bb8147730 0x7f3bb81568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.584+0000 7f3bc0251640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb8151180 0x7f3bb8153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.584+0000 7f3bc0251640 1 --2- 172.21.15.29:0/1860538259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4004810 0x7f3bb81527c0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.584+0000 7f3bc0251640 1 -- 172.21.15.29:0/1860538259 >> 172.21.15.29:0/1860538259 conn(0x7f3bb80b1870 msgr2=0x7f3bb815b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:02.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.584+0000 7f3bc0251640 1 -- 172.21.15.29:0/1860538259 shutdown_connections 2024-04-25T00:41:02.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.584+0000 7f3bc0251640 1 -- 172.21.15.29:0/1860538259 wait complete. 2024-04-25T00:41:02.602 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:41:02.631 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:41:02.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e8601640 1 Processor -- start 2024-04-25T00:41:02.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e8601640 1 -- start start 2024-04-25T00:41:02.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e8601640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc3e00b3e40 0x7fc3e006d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e8601640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e006d980 con 0x7fc3e015df70 2024-04-25T00:41:02.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e8601640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e006dc60 con 0x7fc3e00b4d30 2024-04-25T00:41:02.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e8601640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e006d750 con 0x7fc3e00b3e40 2024-04-25T00:41:02.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e639d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc3e015df70 0x7fc3e00b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33982/0 (socket says 172.21.15.29:33982) 2024-04-25T00:41:02.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e639d640 1 -- 172.21.15.29:0/1467703817 learned_addr learned my addr 172.21.15.29:0/1467703817 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:02.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e6b9e640 1 --2- 172.21.15.29:0/1467703817 >> v2:172.21.15.157:3300/0 conn(0x7fc3e00b3e40 0x7fc3e006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1861973261 0 0) 0x7fc3e006d980 con 0x7fc3e015df70 2024-04-25T00:41:02.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3cc003530 con 0x7fc3e015df70 2024-04-25T00:41:02.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2959214189 0 0) 0x7fc3cc003530 con 0x7fc3e015df70 2024-04-25T00:41:02.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 >> v2:172.21.15.157:3300/0 conn(0x7fc3e00b3e40 msgr2=0x7fc3e006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:02.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 --2- 172.21.15.29:0/1467703817 >> v2:172.21.15.157:3300/0 conn(0x7fc3e00b3e40 0x7fc3e006d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 >> v1:172.21.15.157:6789/0 conn(0x7fc3e00b4d30 legacy=0x7fc3e00ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e006df40 con 0x7fc3e015df70 2024-04-25T00:41:02.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fc3dc001680 con 0x7fc3e015df70 2024-04-25T00:41:02.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 >> v1:172.21.15.29:6789/0 conn(0x7fc3e015df70 legacy=0x7fc3e00b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 --2- 172.21.15.29:0/1467703817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc0039c0 0x7fc3cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 --2- 172.21.15.29:0/1467703817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc004810 0x7fc3cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 --2- 172.21.15.29:0/1467703817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3cc0051d0 0x7fc3cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc00c040 con 0x7fc3cc004810 2024-04-25T00:41:02.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc00c1f0 con 0x7fc3cc0039c0 2024-04-25T00:41:02.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc3cc00c3f0 con 0x7fc3cc0051d0 2024-04-25T00:41:02.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e5b9c640 1 --2- 172.21.15.29:0/1467703817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc004810 0x7fc3cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e639d640 1 --2- 172.21.15.29:0/1467703817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc0039c0 0x7fc3cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e5b9c640 1 -- 172.21.15.29:0/1467703817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3cc0051d0 msgr2=0x7fc3cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:02.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e5b9c640 1 --2- 172.21.15.29:0/1467703817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3cc0051d0 0x7fc3cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e5b9c640 1 -- 172.21.15.29:0/1467703817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc0039c0 msgr2=0x7fc3cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:02.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e5b9c640 1 --2- 172.21.15.29:0/1467703817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc0039c0 0x7fc3cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e5b9c640 1 -- 172.21.15.29:0/1467703817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc003530 con 0x7fc3cc004810 2024-04-25T00:41:02.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e6b9e640 1 -- 172.21.15.29:0/1467703817 reap_dead start 2024-04-25T00:41:02.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.716+0000 7fc3e639d640 1 --2- 172.21.15.29:0/1467703817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc0039c0 0x7fc3cc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:02.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e5b9c640 1 --2- 172.21.15.29:0/1467703817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc004810 0x7fc3cc004c90 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0001780 tx=0x7fc3d001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa875cd7afcf746 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:02.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d00013a0 con 0x7fc3cc004810 2024-04-25T00:41:02.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d00052f0 con 0x7fc3cc004810 2024-04-25T00:41:02.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e539b640 1 -- 172.21.15.29:0/1467703817 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0003840 con 0x7fc3cc004810 2024-04-25T00:41:02.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 -- 172.21.15.29:0/1467703817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc004810 msgr2=0x7fc3cc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 --2- 172.21.15.29:0/1467703817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc004810 0x7fc3cc004c90 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0001780 tx=0x7fc3d001c400 comp rx=0 tx=0).stop 2024-04-25T00:41:02.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 -- 172.21.15.29:0/1467703817 shutdown_connections 2024-04-25T00:41:02.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 --2- 172.21.15.29:0/1467703817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3cc004810 0x7fc3cc004c90 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 -- 172.21.15.29:0/1467703817 >> 172.21.15.29:0/1467703817 conn(0x7fc3e00b1870 msgr2=0x7fc3e00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:02.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 -- 172.21.15.29:0/1467703817 shutdown_connections 2024-04-25T00:41:02.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 -- 172.21.15.29:0/1467703817 wait complete. 2024-04-25T00:41:02.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 Processor -- start 2024-04-25T00:41:02.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 -- start start 2024-04-25T00:41:02.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc004810 0x7fc3e01e3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3e01e55d0 0x7fc3e014d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3e00cdd90 0x7fc3e00ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e015e350 con 0x7fc3e00cdd90 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e006e500 con 0x7fc3cc004810 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e8601640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e006ea20 con 0x7fc3e01e55d0 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e6b9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3e00cdd90 0x7fc3e00ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e6b9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3e00cdd90 0x7fc3e00ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48880/0 (socket says 172.21.15.29:48880) 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e6b9e640 1 -- 172.21.15.29:0/4241520445 learned_addr learned my addr 172.21.15.29:0/4241520445 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e639d640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc004810 0x7fc3e01e3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e5b9c640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3e01e55d0 0x7fc3e014d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e6b9e640 1 -- 172.21.15.29:0/4241520445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3e01e55d0 msgr2=0x7fc3e014d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e6b9e640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3e01e55d0 0x7fc3e014d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e6b9e640 1 -- 172.21.15.29:0/4241520445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc004810 msgr2=0x7fc3e01e3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e6b9e640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc004810 0x7fc3e01e3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.720+0000 7fc3e6b9e640 1 -- 172.21.15.29:0/4241520445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e014dbb0 con 0x7fc3e00cdd90 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.724+0000 7fc3e6b9e640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3e00cdd90 0x7fc3e00ce210 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00725c0 tx=0x7fc3d800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:02.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.724+0000 7fc3d6ffd640 1 -- 172.21.15.29:0/4241520445 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d800bba0 con 0x7fc3e00cdd90 2024-04-25T00:41:02.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.724+0000 7fc3d6ffd640 1 -- 172.21.15.29:0/4241520445 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d801a070 con 0x7fc3e00cdd90 2024-04-25T00:41:02.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.724+0000 7fc3e8601640 1 -- 172.21.15.29:0/4241520445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e01f3cc0 con 0x7fc3e00cdd90 2024-04-25T00:41:02.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.724+0000 7fc3d6ffd640 1 -- 172.21.15.29:0/4241520445 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8014400 con 0x7fc3e00cdd90 2024-04-25T00:41:02.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.724+0000 7fc3e8601640 1 -- 172.21.15.29:0/4241520445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e01f3f80 con 0x7fc3e00cdd90 2024-04-25T00:41:02.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.724+0000 7fc3e8601640 1 -- 172.21.15.29:0/4241520445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e00ce750 con 0x7fc3e00cdd90 2024-04-25T00:41:02.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.724+0000 7fc3d6ffd640 1 -- 172.21.15.29:0/4241520445 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fc3d8013070 con 0x7fc3e00cdd90 2024-04-25T00:41:02.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.724+0000 7fc3d6ffd640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc3c4068280 0x7fc3c406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:02.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.724+0000 7fc3e639d640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc3c4068280 0x7fc3c406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:02.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.724+0000 7fc3d6ffd640 1 -- 172.21.15.29:0/4241520445 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5795+0+0 (secure 0 0 0) 0x7fc3d80846b0 con 0x7fc3e00cdd90 2024-04-25T00:41:02.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.724+0000 7fc3e639d640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc3c4068280 0x7fc3c406a740 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc000c10 tx=0x7fc3dc001440 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:02.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.728+0000 7fc3d6ffd640 1 -- 172.21.15.29:0/4241520445 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc3d8024020 con 0x7fc3e00cdd90 2024-04-25T00:41:02.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:02.920+0000 7fc3e8601640 1 -- 172.21.15.29:0/4241520445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc3e0069270 con 0x7fc3e00cdd90 2024-04-25T00:41:03.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.584+0000 7fc3d6ffd640 1 -- 172.21.15.29:0/4241520445 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v105) ==== 104+0+0 (secure 0 0 0) 0x7fc3d8056df0 con 0x7fc3e00cdd90 2024-04-25T00:41:03.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.588+0000 7fc3e8601640 1 -- 172.21.15.29:0/4241520445 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc3c4068280 msgr2=0x7fc3c406a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:03.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.588+0000 7fc3e8601640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc3c4068280 0x7fc3c406a740 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc000c10 tx=0x7fc3dc001440 comp rx=0 tx=0).stop 2024-04-25T00:41:03.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.588+0000 7fc3e8601640 1 -- 172.21.15.29:0/4241520445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3e00cdd90 msgr2=0x7fc3e00ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:03.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.588+0000 7fc3e8601640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3e00cdd90 0x7fc3e00ce210 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00725c0 tx=0x7fc3d800ef30 comp rx=0 tx=0).stop 2024-04-25T00:41:03.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.588+0000 7fc3e8601640 1 -- 172.21.15.29:0/4241520445 shutdown_connections 2024-04-25T00:41:03.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.588+0000 7fc3e8601640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc3c4068280 0x7fc3c406a740 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:03.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.588+0000 7fc3e8601640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3e00cdd90 0x7fc3e00ce210 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:03.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.588+0000 7fc3e8601640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3e01e55d0 0x7fc3e014d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:03.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.588+0000 7fc3e8601640 1 --2- 172.21.15.29:0/4241520445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3cc004810 0x7fc3e01e3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:03.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.588+0000 7fc3e8601640 1 -- 172.21.15.29:0/4241520445 >> 172.21.15.29:0/4241520445 conn(0x7fc3e00b1870 msgr2=0x7fc3e015b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:03.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.588+0000 7fc3e8601640 1 -- 172.21.15.29:0/4241520445 shutdown_connections 2024-04-25T00:41:03.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.588+0000 7fc3e8601640 1 -- 172.21.15.29:0/4241520445 wait complete. 2024-04-25T00:41:03.614 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:41:03.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.792+0000 7f189b1c1640 1 Processor -- start 2024-04-25T00:41:03.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.792+0000 7f189b1c1640 1 -- start start 2024-04-25T00:41:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f189b1c1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189415e9a0 0x7f1894070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f189b1c1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f18940b4590 0x7f189415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f189b1c1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18940cd1c0 0x7f1894072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f189b1c1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1894071ed0 con 0x7f18940b4590 2024-04-25T00:41:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f189b1c1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f189415f2c0 con 0x7f18940cd1c0 2024-04-25T00:41:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f189b1c1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f18940712c0 con 0x7f189415e9a0 2024-04-25T00:41:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f1893fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f18940b4590 0x7f189415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f1893fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f18940b4590 0x7f189415ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48884/0 (socket says 172.21.15.29:48884) 2024-04-25T00:41:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f1893fff640 1 -- 172.21.15.29:0/1326392788 learned_addr learned my addr 172.21.15.29:0/1326392788 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f1898f5d640 1 --2- 172.21.15.29:0/1326392788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189415e9a0 0x7f1894070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f189975e640 1 --2- 172.21.15.29:0/1326392788 >> v2:172.21.15.157:3300/0 conn(0x7f18940cd1c0 0x7f1894072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:03.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f1893fff640 1 -- 172.21.15.29:0/1326392788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189415e9a0 msgr2=0x7f1894070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:03.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f1893fff640 1 --2- 172.21.15.29:0/1326392788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189415e9a0 0x7f1894070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:03.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f1893fff640 1 -- 172.21.15.29:0/1326392788 >> v2:172.21.15.157:3300/0 conn(0x7f18940cd1c0 msgr2=0x7f1894072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:03.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f1893fff640 1 --2- 172.21.15.29:0/1326392788 >> v2:172.21.15.157:3300/0 conn(0x7f18940cd1c0 0x7f1894072360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:03.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f1893fff640 1 -- 172.21.15.29:0/1326392788 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18940715a0 con 0x7f18940b4590 2024-04-25T00:41:03.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f1893fff640 1 --2- 172.21.15.29:0/1326392788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18940b4590 0x7f189415ed80 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f1894144cc0 tx=0x7f188801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864da12a0beb6972 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:03.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f18937fe640 1 -- 172.21.15.29:0/1326392788 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18880ab620 con 0x7f18940b4590 2024-04-25T00:41:03.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f18937fe640 1 -- 172.21.15.29:0/1326392788 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18880b0070 con 0x7f18940b4590 2024-04-25T00:41:03.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.796+0000 7f18937fe640 1 -- 172.21.15.29:0/1326392788 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18880ab9e0 con 0x7f18940b4590 2024-04-25T00:41:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 -- 172.21.15.29:0/1326392788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18940b4590 msgr2=0x7f189415ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 --2- 172.21.15.29:0/1326392788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18940b4590 0x7f189415ed80 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f1894144cc0 tx=0x7f188801c460 comp rx=0 tx=0).stop 2024-04-25T00:41:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 -- 172.21.15.29:0/1326392788 shutdown_connections 2024-04-25T00:41:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 --2- 172.21.15.29:0/1326392788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18940b4590 0x7f189415ed80 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 --2- 172.21.15.29:0/1326392788 >> v2:172.21.15.157:3300/0 conn(0x7f18940cd1c0 0x7f1894072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 --2- 172.21.15.29:0/1326392788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f189415e9a0 0x7f1894070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 -- 172.21.15.29:0/1326392788 >> 172.21.15.29:0/1326392788 conn(0x7f18940b1870 msgr2=0x7f18940b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 -- 172.21.15.29:0/1326392788 shutdown_connections 2024-04-25T00:41:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 -- 172.21.15.29:0/1326392788 wait complete. 2024-04-25T00:41:03.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 Processor -- start 2024-04-25T00:41:03.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 -- start start 2024-04-25T00:41:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18940b4590 0x7f1894140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18940cd1c0 0x7f18940cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189415e9a0 0x7f18941571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f18941449f0 con 0x7f189415e9a0 2024-04-25T00:41:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1894144870 con 0x7f18940b4590 2024-04-25T00:41:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189b1c1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f18940cd5d0 con 0x7f18940cd1c0 2024-04-25T00:41:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189975e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189415e9a0 0x7f18941571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189975e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189415e9a0 0x7f18941571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48898/0 (socket says 172.21.15.29:48898) 2024-04-25T00:41:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189975e640 1 -- 172.21.15.29:0/238454900 learned_addr learned my addr 172.21.15.29:0/238454900 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f1898f5d640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18940b4590 0x7f1894140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f1893fff640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18940cd1c0 0x7f18940cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189975e640 1 -- 172.21.15.29:0/238454900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18940cd1c0 msgr2=0x7f18940cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189975e640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18940cd1c0 0x7f18940cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189975e640 1 -- 172.21.15.29:0/238454900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18940b4590 msgr2=0x7f1894140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189975e640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18940b4590 0x7f1894140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189975e640 1 -- 172.21.15.29:0/238454900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18940ce450 con 0x7f189415e9a0 2024-04-25T00:41:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f1898f5d640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18940b4590 0x7f1894140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f189975e640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189415e9a0 0x7f18941571b0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f188c00d5d0 tx=0x7f188c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.800+0000 7f18917fa640 1 -- 172.21.15.29:0/238454900 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c018020 con 0x7f189415e9a0 2024-04-25T00:41:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.804+0000 7f18917fa640 1 -- 172.21.15.29:0/238454900 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f188c014620 con 0x7f189415e9a0 2024-04-25T00:41:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.804+0000 7f18917fa640 1 -- 172.21.15.29:0/238454900 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c0149f0 con 0x7f189415e9a0 2024-04-25T00:41:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.804+0000 7f189b1c1640 1 -- 172.21.15.29:0/238454900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18940ce6f0 con 0x7f189415e9a0 2024-04-25T00:41:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.804+0000 7f189b1c1640 1 -- 172.21.15.29:0/238454900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1894157c00 con 0x7f189415e9a0 2024-04-25T00:41:03.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.804+0000 7f189b1c1640 1 -- 172.21.15.29:0/238454900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18940692e0 con 0x7f189415e9a0 2024-04-25T00:41:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.804+0000 7f18917fa640 1 -- 172.21.15.29:0/238454900 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f188c01b020 con 0x7f189415e9a0 2024-04-25T00:41:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.804+0000 7f18917fa640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1880068250 0x7f188006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.804+0000 7f1898f5d640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1880068250 0x7f188006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.804+0000 7f18917fa640 1 -- 172.21.15.29:0/238454900 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5795+0+0 (secure 0 0 0) 0x7f188c0848c0 con 0x7f189415e9a0 2024-04-25T00:41:03.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.804+0000 7f1898f5d640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1880068250 0x7f188006a710 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1894159a70 tx=0x7f188400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:03.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.808+0000 7f18917fa640 1 -- 172.21.15.29:0/238454900 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f188c022020 con 0x7f189415e9a0 2024-04-25T00:41:03.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:03.972+0000 7f189b1c1640 1 -- 172.21.15.29:0/238454900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f189406cff0 con 0x7f189415e9a0 2024-04-25T00:41:04.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.592+0000 7f18917fa640 1 -- 172.21.15.29:0/238454900 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v158) ==== 181+0+0 (secure 0 0 0) 0x7f188c057000 con 0x7f189415e9a0 2024-04-25T00:41:04.593 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:41:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.596+0000 7f189b1c1640 1 -- 172.21.15.29:0/238454900 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1880068250 msgr2=0x7f188006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.596+0000 7f189b1c1640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1880068250 0x7f188006a710 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1894159a70 tx=0x7f188400b040 comp rx=0 tx=0).stop 2024-04-25T00:41:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.596+0000 7f189b1c1640 1 -- 172.21.15.29:0/238454900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189415e9a0 msgr2=0x7f18941571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.596+0000 7f189b1c1640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189415e9a0 0x7f18941571b0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f188c00d5d0 tx=0x7f188c00da90 comp rx=0 tx=0).stop 2024-04-25T00:41:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.596+0000 7f189b1c1640 1 -- 172.21.15.29:0/238454900 shutdown_connections 2024-04-25T00:41:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.596+0000 7f189b1c1640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1880068250 0x7f188006a710 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.596+0000 7f189b1c1640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189415e9a0 0x7f18941571b0 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.596+0000 7f189b1c1640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18940cd1c0 0x7f18940cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.596+0000 7f189b1c1640 1 --2- 172.21.15.29:0/238454900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18940b4590 0x7f1894140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.596+0000 7f189b1c1640 1 -- 172.21.15.29:0/238454900 >> 172.21.15.29:0/238454900 conn(0x7f18940b1870 msgr2=0x7f1894140eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.596+0000 7f189b1c1640 1 -- 172.21.15.29:0/238454900 shutdown_connections 2024-04-25T00:41:04.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.596+0000 7f189b1c1640 1 -- 172.21.15.29:0/238454900 wait complete. 2024-04-25T00:41:04.620 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:41:04.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19af2df640 1 Processor -- start 2024-04-25T00:41:04.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19af2df640 1 -- start start 2024-04-25T00:41:04.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19af2df640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19a8071b40 0x7f19a80b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:04.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19af2df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a80b4b30 0x7f19a806d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:04.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19af2df640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19a80b3ac0 con 0x7f19a815e1a0 2024-04-25T00:41:04.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19af2df640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f19a80714f0 con 0x7f19a8071b40 2024-04-25T00:41:04.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19af2df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f19a80b2d80 con 0x7f19a80b4b30 2024-04-25T00:41:04.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19ac87a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19a8071b40 0x7f19a80b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:04.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19ac87a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19a8071b40 0x7f19a80b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48910/0 (socket says 172.21.15.29:48910) 2024-04-25T00:41:04.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19ac87a640 1 -- 172.21.15.29:0/2981178288 learned_addr learned my addr 172.21.15.29:0/2981178288 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:04.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19ad87c640 1 --2- 172.21.15.29:0/2981178288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a80b4b30 0x7f19a806d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:04.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19ac87a640 1 -- 172.21.15.29:0/2981178288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a80b4b30 msgr2=0x7f19a806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:04.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19ac87a640 1 --2- 172.21.15.29:0/2981178288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a80b4b30 0x7f19a806d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:04.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19ac87a640 1 -- 172.21.15.29:0/2981178288 >> v1:172.21.15.157:6789/0 conn(0x7f19a815e1a0 legacy=0x7f19a8070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:04.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.752+0000 7f19ac87a640 1 -- 172.21.15.29:0/2981178288 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a80b3d60 con 0x7f19a8071b40 2024-04-25T00:41:04.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.756+0000 7f19ac87a640 1 --2- 172.21.15.29:0/2981178288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19a8071b40 0x7f19a80b45f0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f1998009870 tx=0x7f199801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a0c9b091b16072 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:04.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.756+0000 7f199ffff640 1 -- 172.21.15.29:0/2981178288 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19980b0070 con 0x7f19a8071b40 2024-04-25T00:41:04.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.756+0000 7f199ffff640 1 -- 172.21.15.29:0/2981178288 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19980ab600 con 0x7f19a8071b40 2024-04-25T00:41:04.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.756+0000 7f199ffff640 1 -- 172.21.15.29:0/2981178288 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19980ab9d0 con 0x7f19a8071b40 2024-04-25T00:41:04.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.756+0000 7f19af2df640 1 -- 172.21.15.29:0/2981178288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19a8071b40 msgr2=0x7f19a80b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:04.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.756+0000 7f19af2df640 1 --2- 172.21.15.29:0/2981178288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19a8071b40 0x7f19a80b45f0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f1998009870 tx=0x7f199801c490 comp rx=0 tx=0).stop 2024-04-25T00:41:04.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.756+0000 7f19af2df640 1 -- 172.21.15.29:0/2981178288 shutdown_connections 2024-04-25T00:41:04.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.756+0000 7f19af2df640 1 --2- 172.21.15.29:0/2981178288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a80b4b30 0x7f19a806d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:04.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.756+0000 7f19af2df640 1 --2- 172.21.15.29:0/2981178288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19a8071b40 0x7f19a80b45f0 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:04.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.756+0000 7f19af2df640 1 -- 172.21.15.29:0/2981178288 >> 172.21.15.29:0/2981178288 conn(0x7f19a80b1870 msgr2=0x7f19a80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:04.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.756+0000 7f19af2df640 1 -- 172.21.15.29:0/2981178288 shutdown_connections 2024-04-25T00:41:04.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.756+0000 7f19af2df640 1 -- 172.21.15.29:0/2981178288 wait complete. 2024-04-25T00:41:04.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19af2df640 1 Processor -- start 2024-04-25T00:41:04.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19af2df640 1 -- start start 2024-04-25T00:41:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19af2df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071b40 0x7f19a815c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19af2df640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19a80b4b30 0x7f19a814dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19af2df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a815e1a0 0x7f19a80cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19af2df640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f19a806e780 con 0x7f19a80b4b30 2024-04-25T00:41:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19af2df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f19a806e600 con 0x7f19a815e1a0 2024-04-25T00:41:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19af2df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f19a806e900 con 0x7f19a8071b40 2024-04-25T00:41:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19ac87a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19a80b4b30 0x7f19a814dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19ac87a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19a80b4b30 0x7f19a814dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48922/0 (socket says 172.21.15.29:48922) 2024-04-25T00:41:04.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19ac87a640 1 -- 172.21.15.29:0/1150708294 learned_addr learned my addr 172.21.15.29:0/1150708294 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19ad87c640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a815e1a0 0x7f19a80cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19ad07b640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071b40 0x7f19a815c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19ac87a640 1 -- 172.21.15.29:0/1150708294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071b40 msgr2=0x7f19a815c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19ac87a640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071b40 0x7f19a815c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19ac87a640 1 -- 172.21.15.29:0/1150708294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a815e1a0 msgr2=0x7f19a80cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19ac87a640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a815e1a0 0x7f19a80cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19ac87a640 1 -- 172.21.15.29:0/1150708294 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a815c8f0 con 0x7f19a80b4b30 2024-04-25T00:41:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19ac87a640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19a80b4b30 0x7f19a814dce0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f199801ab40 tx=0x7f1998019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f199dffb640 1 -- 172.21.15.29:0/1150708294 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19980b8020 con 0x7f19a80b4b30 2024-04-25T00:41:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f199dffb640 1 -- 172.21.15.29:0/1150708294 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19980b46d0 con 0x7f19a80b4b30 2024-04-25T00:41:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19af2df640 1 -- 172.21.15.29:0/1150708294 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a815d760 con 0x7f19a80b4b30 2024-04-25T00:41:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f199dffb640 1 -- 172.21.15.29:0/1150708294 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19980b0040 con 0x7f19a80b4b30 2024-04-25T00:41:04.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.760+0000 7f19af2df640 1 -- 172.21.15.29:0/1150708294 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a815bbb0 con 0x7f19a80b4b30 2024-04-25T00:41:04.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.764+0000 7f19af2df640 1 -- 172.21.15.29:0/1150708294 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a815b500 con 0x7f19a80b4b30 2024-04-25T00:41:04.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.764+0000 7f199dffb640 1 -- 172.21.15.29:0/1150708294 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f19980bb020 con 0x7f19a80b4b30 2024-04-25T00:41:04.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.764+0000 7f199dffb640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f198c068250 0x7f198c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:04.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.764+0000 7f199dffb640 1 -- 172.21.15.29:0/1150708294 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5357+0+0 (secure 0 0 0) 0x7f199811e0d0 con 0x7f19a80b4b30 2024-04-25T00:41:04.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.764+0000 7f19ad07b640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f198c068250 0x7f198c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:04.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.764+0000 7f19ad07b640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f198c068250 0x7f198c06a710 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f19a400b810 tx=0x7f19a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:04.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.768+0000 7f199dffb640 1 -- 172.21.15.29:0/1150708294 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f19980c2030 con 0x7f19a80b4b30 2024-04-25T00:41:04.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:04.936+0000 7f19af2df640 1 -- 172.21.15.29:0/1150708294 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f19a80692e0 con 0x7f19a80b4b30 2024-04-25T00:41:05.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.596+0000 7f199dffb640 1 -- 172.21.15.29:0/1150708294 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v159) ==== 169+0+0 (secure 0 0 0) 0x7f1998016110 con 0x7f19a80b4b30 2024-04-25T00:41:05.596 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-04-25T00:41:05.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.600+0000 7f19af2df640 1 -- 172.21.15.29:0/1150708294 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f198c068250 msgr2=0x7f198c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:05.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.600+0000 7f19af2df640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f198c068250 0x7f198c06a710 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f19a400b810 tx=0x7f19a4008040 comp rx=0 tx=0).stop 2024-04-25T00:41:05.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.600+0000 7f19af2df640 1 -- 172.21.15.29:0/1150708294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19a80b4b30 msgr2=0x7f19a814dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:05.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.600+0000 7f19af2df640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19a80b4b30 0x7f19a814dce0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f199801ab40 tx=0x7f1998019230 comp rx=0 tx=0).stop 2024-04-25T00:41:05.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.604+0000 7f19af2df640 1 -- 172.21.15.29:0/1150708294 shutdown_connections 2024-04-25T00:41:05.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.604+0000 7f19af2df640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f198c068250 0x7f198c06a710 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.604+0000 7f19af2df640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a815e1a0 0x7f19a80cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.604+0000 7f19af2df640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19a80b4b30 0x7f19a814dce0 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.604+0000 7f19af2df640 1 --2- 172.21.15.29:0/1150708294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19a8071b40 0x7f19a815c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.604+0000 7f19af2df640 1 -- 172.21.15.29:0/1150708294 >> 172.21.15.29:0/1150708294 conn(0x7f19a80b1870 msgr2=0x7f19a8070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:05.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.604+0000 7f19af2df640 1 -- 172.21.15.29:0/1150708294 shutdown_connections 2024-04-25T00:41:05.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.604+0000 7f19af2df640 1 -- 172.21.15.29:0/1150708294 wait complete. 2024-04-25T00:41:05.635 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:41:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa634b95640 1 Processor -- start 2024-04-25T00:41:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa634b95640 1 -- start start 2024-04-25T00:41:05.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa634b95640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6300b4780 0x7fa63015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:05.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa634b95640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa6300cc240 0x7fa63015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:05.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa634b95640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6300b3720 con 0x7fa63015e8d0 2024-04-25T00:41:05.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa634b95640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa6300b42f0 con 0x7fa6300cc240 2024-04-25T00:41:05.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa634b95640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa6300b39c0 con 0x7fa6300b4780 2024-04-25T00:41:05.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa62ed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa6300cc240 0x7fa63015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:05.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa62ed9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa6300cc240 0x7fa63015c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48928/0 (socket says 172.21.15.29:48928) 2024-04-25T00:41:05.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa62ed9d640 1 -- 172.21.15.29:0/3210589930 learned_addr learned my addr 172.21.15.29:0/3210589930 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:05.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa62dd9b640 1 --2- 172.21.15.29:0/3210589930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6300b4780 0x7fa63015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:05.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa62ed9d640 1 -- 172.21.15.29:0/3210589930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6300b4780 msgr2=0x7fa63015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:05.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa62ed9d640 1 --2- 172.21.15.29:0/3210589930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6300b4780 0x7fa63015f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa62ed9d640 1 -- 172.21.15.29:0/3210589930 >> v1:172.21.15.157:6789/0 conn(0x7fa63015e8d0 legacy=0x7fa63015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:05.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa62ed9d640 1 -- 172.21.15.29:0/3210589930 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6300b3ca0 con 0x7fa6300cc240 2024-04-25T00:41:05.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.760+0000 7fa62dd9b640 1 --2- 172.21.15.29:0/3210589930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6300b4780 0x7fa63015f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:05.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa62ed9d640 1 --2- 172.21.15.29:0/3210589930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6300cc240 0x7fa63015c7d0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fa62800d150 tx=0x7fa62801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a187d34934d25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:05.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa62d59a640 1 -- 172.21.15.29:0/3210589930 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62800b2d0 con 0x7fa6300cc240 2024-04-25T00:41:05.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa62d59a640 1 -- 172.21.15.29:0/3210589930 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa62800b470 con 0x7fa6300cc240 2024-04-25T00:41:05.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa62d59a640 1 -- 172.21.15.29:0/3210589930 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6280a7b70 con 0x7fa6300cc240 2024-04-25T00:41:05.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa634b95640 1 -- 172.21.15.29:0/3210589930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6300cc240 msgr2=0x7fa63015c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:05.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa634b95640 1 --2- 172.21.15.29:0/3210589930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6300cc240 0x7fa63015c7d0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fa62800d150 tx=0x7fa62801ebf0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa634b95640 1 -- 172.21.15.29:0/3210589930 shutdown_connections 2024-04-25T00:41:05.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa634b95640 1 --2- 172.21.15.29:0/3210589930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6300cc240 0x7fa63015c7d0 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa634b95640 1 --2- 172.21.15.29:0/3210589930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6300b4780 0x7fa63015f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa634b95640 1 -- 172.21.15.29:0/3210589930 >> 172.21.15.29:0/3210589930 conn(0x7fa6300b1870 msgr2=0x7fa6300b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:05.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa634b95640 1 -- 172.21.15.29:0/3210589930 shutdown_connections 2024-04-25T00:41:05.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa634b95640 1 -- 172.21.15.29:0/3210589930 wait complete. 2024-04-25T00:41:05.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa634b95640 1 Processor -- start 2024-04-25T00:41:05.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.764+0000 7fa634b95640 1 -- start start 2024-04-25T00:41:05.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa634b95640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6300b4780 0x7fa63013bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:05.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa634b95640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6300cc240 0x7fa630158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:05.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa634b95640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa63015e8d0 0x7fa6301585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:05.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa634b95640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa63015d6a0 con 0x7fa63015e8d0 2024-04-25T00:41:05.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa634b95640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa63015d520 con 0x7fa6300b4780 2024-04-25T00:41:05.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa634b95640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa63006d590 con 0x7fa6300cc240 2024-04-25T00:41:05.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa63015e8d0 0x7fa6301585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:05.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa63015e8d0 0x7fa6301585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48936/0 (socket says 172.21.15.29:48936) 2024-04-25T00:41:05.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62ed9d640 1 -- 172.21.15.29:0/915934653 learned_addr learned my addr 172.21.15.29:0/915934653 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:05.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62e59c640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6300b4780 0x7fa63013bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:05.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62dd9b640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6300cc240 0x7fa630158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:05.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62ed9d640 1 -- 172.21.15.29:0/915934653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6300cc240 msgr2=0x7fa630158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:05.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62ed9d640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6300cc240 0x7fa630158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62ed9d640 1 -- 172.21.15.29:0/915934653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6300b4780 msgr2=0x7fa63013bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:05.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62ed9d640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6300b4780 0x7fa63013bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62ed9d640 1 -- 172.21.15.29:0/915934653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63015a900 con 0x7fa63015e8d0 2024-04-25T00:41:05.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62e59c640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6300b4780 0x7fa63013bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:05.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62ed9d640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa63015e8d0 0x7fa6301585a0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fa62800d150 tx=0x7fa62801b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:05.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa61effd640 1 -- 172.21.15.29:0/915934653 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6280be020 con 0x7fa63015e8d0 2024-04-25T00:41:05.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa61effd640 1 -- 172.21.15.29:0/915934653 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa628009800 con 0x7fa63015e8d0 2024-04-25T00:41:05.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa61effd640 1 -- 172.21.15.29:0/915934653 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628009bf0 con 0x7fa63015e8d0 2024-04-25T00:41:05.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa634b95640 1 -- 172.21.15.29:0/915934653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63013ae30 con 0x7fa63015e8d0 2024-04-25T00:41:05.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa634b95640 1 -- 172.21.15.29:0/915934653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa63013b3f0 con 0x7fa63015e8d0 2024-04-25T00:41:05.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa634b95640 1 -- 172.21.15.29:0/915934653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa630069270 con 0x7fa63015e8d0 2024-04-25T00:41:05.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa61effd640 1 -- 172.21.15.29:0/915934653 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fa6280c1020 con 0x7fa63015e8d0 2024-04-25T00:41:05.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa61effd640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa610068280 0x7fa61006a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:05.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa61effd640 1 -- 172.21.15.29:0/915934653 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7fa6281210e0 con 0x7fa63015e8d0 2024-04-25T00:41:05.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62e59c640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa610068280 0x7fa61006a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:05.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.768+0000 7fa62e59c640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa610068280 0x7fa61006a740 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa63013d4f0 tx=0x7fa620008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:05.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.772+0000 7fa61effd640 1 -- 172.21.15.29:0/915934653 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa6280c9020 con 0x7fa63015e8d0 2024-04-25T00:41:05.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.944+0000 7fa634b95640 1 -- 172.21.15.29:0/915934653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa63006e7b0 con 0x7fa63015e8d0 2024-04-25T00:41:05.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.952+0000 7fa61effd640 1 -- 172.21.15.29:0/915934653 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fa63015a900 con 0x7fa63015e8d0 2024-04-25T00:41:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.956+0000 7fa634b95640 1 -- 172.21.15.29:0/915934653 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa610068280 msgr2=0x7fa61006a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.956+0000 7fa634b95640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa610068280 0x7fa61006a740 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa63013d4f0 tx=0x7fa620008040 comp rx=0 tx=0).stop 2024-04-25T00:41:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.956+0000 7fa634b95640 1 -- 172.21.15.29:0/915934653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa63015e8d0 msgr2=0x7fa6301585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.956+0000 7fa634b95640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa63015e8d0 0x7fa6301585a0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fa62800d150 tx=0x7fa62801b990 comp rx=0 tx=0).stop 2024-04-25T00:41:05.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.956+0000 7fa634b95640 1 -- 172.21.15.29:0/915934653 shutdown_connections 2024-04-25T00:41:05.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.956+0000 7fa634b95640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa610068280 0x7fa61006a740 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.956+0000 7fa634b95640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa63015e8d0 0x7fa6301585a0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.956+0000 7fa634b95640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6300cc240 0x7fa630158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.956+0000 7fa634b95640 1 --2- 172.21.15.29:0/915934653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6300b4780 0x7fa63013bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:05.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.956+0000 7fa634b95640 1 -- 172.21.15.29:0/915934653 >> 172.21.15.29:0/915934653 conn(0x7fa6300b1870 msgr2=0x7fa63015ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:05.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.956+0000 7fa634b95640 1 -- 172.21.15.29:0/915934653 shutdown_connections 2024-04-25T00:41:05.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:05.956+0000 7fa634b95640 1 -- 172.21.15.29:0/915934653 wait complete. 2024-04-25T00:41:05.975 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-04-25T00:41:06.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa210fec640 1 Processor -- start 2024-04-25T00:41:06.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa210fec640 1 -- start start 2024-04-25T00:41:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa210fec640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa20c15e250 0x7fa20c0d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa210fec640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa20c15e630 0x7fa20c0d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa210fec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa20c0c9450 0x7fa20c0c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa210fec640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa20c0c8440 con 0x7fa20c15e630 2024-04-25T00:41:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa210fec640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa20c0c8690 con 0x7fa20c15e250 2024-04-25T00:41:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa210fec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa20c0c8970 con 0x7fa20c0c9450 2024-04-25T00:41:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa209d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa20c15e630 0x7fa20c0d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa209d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa20c15e630 0x7fa20c0d29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48948/0 (socket says 172.21.15.29:48948) 2024-04-25T00:41:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa209d9b640 1 -- 172.21.15.29:0/1912473292 learned_addr learned my addr 172.21.15.29:0/1912473292 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa20ad9d640 1 --2- 172.21.15.29:0/1912473292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa20c0c9450 0x7fa20c0c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa20a59c640 1 --2- 172.21.15.29:0/1912473292 >> v2:172.21.15.157:3300/0 conn(0x7fa20c15e250 0x7fa20c0d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa209d9b640 1 -- 172.21.15.29:0/1912473292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa20c0c9450 msgr2=0x7fa20c0c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa209d9b640 1 --2- 172.21.15.29:0/1912473292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa20c0c9450 0x7fa20c0c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa209d9b640 1 -- 172.21.15.29:0/1912473292 >> v2:172.21.15.157:3300/0 conn(0x7fa20c15e250 msgr2=0x7fa20c0d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa209d9b640 1 --2- 172.21.15.29:0/1912473292 >> v2:172.21.15.157:3300/0 conn(0x7fa20c15e250 0x7fa20c0d24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa209d9b640 1 -- 172.21.15.29:0/1912473292 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c0c8c50 con 0x7fa20c15e630 2024-04-25T00:41:06.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa20a59c640 1 --2- 172.21.15.29:0/1912473292 >> v2:172.21.15.157:3300/0 conn(0x7fa20c15e250 0x7fa20c0d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.100+0000 7fa209d9b640 1 --2- 172.21.15.29:0/1912473292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa20c15e630 0x7fa20c0d29e0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0d3160 tx=0x7fa20001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea9c38d5c24a291 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa20959a640 1 -- 172.21.15.29:0/1912473292 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2000b2070 con 0x7fa20c15e630 2024-04-25T00:41:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa20959a640 1 -- 172.21.15.29:0/1912473292 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2000a9070 con 0x7fa20c15e630 2024-04-25T00:41:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa20959a640 1 -- 172.21.15.29:0/1912473292 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2000ad880 con 0x7fa20c15e630 2024-04-25T00:41:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 -- 172.21.15.29:0/1912473292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa20c15e630 msgr2=0x7fa20c0d29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 --2- 172.21.15.29:0/1912473292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa20c15e630 0x7fa20c0d29e0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0d3160 tx=0x7fa20001c490 comp rx=0 tx=0).stop 2024-04-25T00:41:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 -- 172.21.15.29:0/1912473292 shutdown_connections 2024-04-25T00:41:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 --2- 172.21.15.29:0/1912473292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa20c15e630 0x7fa20c0d29e0 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 --2- 172.21.15.29:0/1912473292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa20c0c9450 0x7fa20c0c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 --2- 172.21.15.29:0/1912473292 >> v2:172.21.15.157:3300/0 conn(0x7fa20c15e250 0x7fa20c0d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 -- 172.21.15.29:0/1912473292 >> 172.21.15.29:0/1912473292 conn(0x7fa20c0d3f70 msgr2=0x7fa20c0d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 -- 172.21.15.29:0/1912473292 shutdown_connections 2024-04-25T00:41:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 -- 172.21.15.29:0/1912473292 wait complete. 2024-04-25T00:41:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 Processor -- start 2024-04-25T00:41:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 -- start start 2024-04-25T00:41:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa20c0c9450 0x7fa20c0a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa20c15e250 0x7fa20c09e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa20c15e630 0x7fa20c0cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c0ceaf0 con 0x7fa20c15e630 2024-04-25T00:41:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c0d1ed0 con 0x7fa20c0c9450 2024-04-25T00:41:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa20c0cec70 con 0x7fa20c15e250 2024-04-25T00:41:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa20ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa20c15e630 0x7fa20c0cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa20ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa20c15e630 0x7fa20c0cb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48952/0 (socket says 172.21.15.29:48952) 2024-04-25T00:41:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa20ad9d640 1 -- 172.21.15.29:0/1030459760 learned_addr learned my addr 172.21.15.29:0/1030459760 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa20a59c640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa20c0c9450 0x7fa20c0a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa209d9b640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa20c15e250 0x7fa20c09e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa20ad9d640 1 -- 172.21.15.29:0/1030459760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa20c15e250 msgr2=0x7fa20c09e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa20ad9d640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa20c15e250 0x7fa20c09e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa20ad9d640 1 -- 172.21.15.29:0/1030459760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa20c0c9450 msgr2=0x7fa20c0a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa20ad9d640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa20c0c9450 0x7fa20c0a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa20ad9d640 1 -- 172.21.15.29:0/1030459760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c0b6d10 con 0x7fa20c15e630 2024-04-25T00:41:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa209d9b640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa20c15e250 0x7fa20c09e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa20ad9d640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa20c15e630 0x7fa20c0cb9c0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fa20400c590 tx=0x7fa20400ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa1faffd640 1 -- 172.21.15.29:0/1030459760 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204015070 con 0x7fa20c15e630 2024-04-25T00:41:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa1faffd640 1 -- 172.21.15.29:0/1030459760 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa204004220 con 0x7fa20c15e630 2024-04-25T00:41:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 -- 172.21.15.29:0/1030459760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20c0b6ff0 con 0x7fa20c15e630 2024-04-25T00:41:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa1faffd640 1 -- 172.21.15.29:0/1030459760 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204003620 con 0x7fa20c15e630 2024-04-25T00:41:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.104+0000 7fa210fec640 1 -- 172.21.15.29:0/1030459760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20c0b75b0 con 0x7fa20c15e630 2024-04-25T00:41:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.108+0000 7fa210fec640 1 -- 172.21.15.29:0/1030459760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa20c0b9dc0 con 0x7fa20c15e630 2024-04-25T00:41:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.108+0000 7fa1faffd640 1 -- 172.21.15.29:0/1030459760 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fa2040037c0 con 0x7fa20c15e630 2024-04-25T00:41:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.108+0000 7fa1faffd640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1ec068250 0x7fa1ec06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.108+0000 7fa1faffd640 1 -- 172.21.15.29:0/1030459760 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7fa2040840e0 con 0x7fa20c15e630 2024-04-25T00:41:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.108+0000 7fa20a59c640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1ec068250 0x7fa1ec06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:06.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.108+0000 7fa20a59c640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1ec068250 0x7fa1ec06a710 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0a0e50 tx=0x7fa1fc007810 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:06.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.112+0000 7fa1faffd640 1 -- 172.21.15.29:0/1030459760 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa2040561d0 con 0x7fa20c15e630 2024-04-25T00:41:06.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.340+0000 7fa210fec640 1 -- 172.21.15.29:0/1030459760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7fa20c069250 con 0x7fa20c15e630 2024-04-25T00:41:06.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.608+0000 7fa1faffd640 1 -- 172.21.15.29:0/1030459760 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v514) ==== 145+0+0 (secure 0 0 0) 0x7fa20405a4d0 con 0x7fa20c15e630 2024-04-25T00:41:06.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.612+0000 7fa210fec640 1 -- 172.21.15.29:0/1030459760 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1ec068250 msgr2=0x7fa1ec06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:06.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.612+0000 7fa210fec640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1ec068250 0x7fa1ec06a710 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0a0e50 tx=0x7fa1fc007810 comp rx=0 tx=0).stop 2024-04-25T00:41:06.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.612+0000 7fa210fec640 1 -- 172.21.15.29:0/1030459760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa20c15e630 msgr2=0x7fa20c0cb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:06.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.612+0000 7fa210fec640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa20c15e630 0x7fa20c0cb9c0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fa20400c590 tx=0x7fa20400ca50 comp rx=0 tx=0).stop 2024-04-25T00:41:06.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.612+0000 7fa210fec640 1 -- 172.21.15.29:0/1030459760 shutdown_connections 2024-04-25T00:41:06.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.612+0000 7fa210fec640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1ec068250 0x7fa1ec06a710 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.612+0000 7fa210fec640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa20c15e630 0x7fa20c0cb9c0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.612+0000 7fa210fec640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa20c15e250 0x7fa20c09e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.612+0000 7fa210fec640 1 --2- 172.21.15.29:0/1030459760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa20c0c9450 0x7fa20c0a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.612+0000 7fa210fec640 1 -- 172.21.15.29:0/1030459760 >> 172.21.15.29:0/1030459760 conn(0x7fa20c0d3f70 msgr2=0x7fa20c15d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:06.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.612+0000 7fa210fec640 1 -- 172.21.15.29:0/1030459760 shutdown_connections 2024-04-25T00:41:06.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.612+0000 7fa210fec640 1 -- 172.21.15.29:0/1030459760 wait complete. 2024-04-25T00:41:06.634 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-04-25T00:41:06.634 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2024-04-25T00:41:06.634 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-04-25T00:41:06.635 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:06.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.764+0000 7fba43080640 1 Processor -- start 2024-04-25T00:41:06.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba43080640 1 -- start start 2024-04-25T00:41:06.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba43080640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fba3c15e200 0x7fba3c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba43080640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fba3c0cf690 0x7fba3c0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba43080640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba3c0d2a90 0x7fba3c0d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba43080640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fba3c0d2ee0 con 0x7fba3c15e200 2024-04-25T00:41:06.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba43080640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fba3c0b38d0 con 0x7fba3c0cf690 2024-04-25T00:41:06.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba43080640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c0cfc10 con 0x7fba3c0d2a90 2024-04-25T00:41:06.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba40e1c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fba3c15e200 0x7fba3c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:06.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba40e1c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fba3c15e200 0x7fba3c0d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48956/0 (socket says 172.21.15.29:48956) 2024-04-25T00:41:06.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba40e1c640 1 -- 172.21.15.29:0/2670724671 learned_addr learned my addr 172.21.15.29:0/2670724671 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:06.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba40e1c640 1 -- 172.21.15.29:0/2670724671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba3c0d2a90 msgr2=0x7fba3c0d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:06.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba40e1c640 1 --2- 172.21.15.29:0/2670724671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba3c0d2a90 0x7fba3c0d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba40e1c640 1 -- 172.21.15.29:0/2670724671 >> v2:172.21.15.157:3300/0 conn(0x7fba3c0cf690 msgr2=0x7fba3c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:06.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba40e1c640 1 --2- 172.21.15.29:0/2670724671 >> v2:172.21.15.157:3300/0 conn(0x7fba3c0cf690 0x7fba3c0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.768+0000 7fba40e1c640 1 -- 172.21.15.29:0/2670724671 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c0cfef0 con 0x7fba3c15e200 2024-04-25T00:41:06.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba40e1c640 1 --2- 172.21.15.29:0/2670724671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba3c15e200 0x7fba3c0d47f0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0d24a0 tx=0x7fba3001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e1470c5c7c61bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:06.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba3b7fe640 1 -- 172.21.15.29:0/2670724671 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba300b0070 con 0x7fba3c15e200 2024-04-25T00:41:06.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba3b7fe640 1 -- 172.21.15.29:0/2670724671 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba300ab600 con 0x7fba3c15e200 2024-04-25T00:41:06.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba3b7fe640 1 -- 172.21.15.29:0/2670724671 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba300ab9d0 con 0x7fba3c15e200 2024-04-25T00:41:06.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 -- 172.21.15.29:0/2670724671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba3c15e200 msgr2=0x7fba3c0d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:06.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 --2- 172.21.15.29:0/2670724671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba3c15e200 0x7fba3c0d47f0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0d24a0 tx=0x7fba3001c230 comp rx=0 tx=0).stop 2024-04-25T00:41:06.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 -- 172.21.15.29:0/2670724671 shutdown_connections 2024-04-25T00:41:06.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 --2- 172.21.15.29:0/2670724671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba3c0d2a90 0x7fba3c0d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 --2- 172.21.15.29:0/2670724671 >> v2:172.21.15.157:3300/0 conn(0x7fba3c0cf690 0x7fba3c0d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 --2- 172.21.15.29:0/2670724671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba3c15e200 0x7fba3c0d47f0 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 -- 172.21.15.29:0/2670724671 >> 172.21.15.29:0/2670724671 conn(0x7fba3c09d280 msgr2=0x7fba3c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:06.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 -- 172.21.15.29:0/2670724671 shutdown_connections 2024-04-25T00:41:06.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 -- 172.21.15.29:0/2670724671 wait complete. 2024-04-25T00:41:06.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 Processor -- start 2024-04-25T00:41:06.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 -- start start 2024-04-25T00:41:06.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba3c0cf690 0x7fba3c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba3c0d2a90 0x7fba3c161070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3c15e200 0x7fba3c0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c15e760 con 0x7fba3c0cf690 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c15e5e0 con 0x7fba3c15e200 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba43080640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c0b4320 con 0x7fba3c0d2a90 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba40e1c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba3c0cf690 0x7fba3c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba40e1c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba3c0cf690 0x7fba3c15cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48966/0 (socket says 172.21.15.29:48966) 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba40e1c640 1 -- 172.21.15.29:0/3227147862 learned_addr learned my addr 172.21.15.29:0/3227147862 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba4161d640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3c15e200 0x7fba3c0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba3bfff640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba3c0d2a90 0x7fba3c161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba40e1c640 1 -- 172.21.15.29:0/3227147862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba3c0d2a90 msgr2=0x7fba3c161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba40e1c640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba3c0d2a90 0x7fba3c161070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba40e1c640 1 -- 172.21.15.29:0/3227147862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3c15e200 msgr2=0x7fba3c0b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba40e1c640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3c15e200 0x7fba3c0b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.772+0000 7fba40e1c640 1 -- 172.21.15.29:0/3227147862 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c0c9b80 con 0x7fba3c0cf690 2024-04-25T00:41:06.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.776+0000 7fba40e1c640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba3c0cf690 0x7fba3c15cdf0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fba300ae040 tx=0x7fba3001aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:06.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.776+0000 7fba397fa640 1 -- 172.21.15.29:0/3227147862 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba300b0040 con 0x7fba3c0cf690 2024-04-25T00:41:06.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.776+0000 7fba397fa640 1 -- 172.21.15.29:0/3227147862 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba300bc070 con 0x7fba3c0cf690 2024-04-25T00:41:06.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.776+0000 7fba43080640 1 -- 172.21.15.29:0/3227147862 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3c0c9e60 con 0x7fba3c0cf690 2024-04-25T00:41:06.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.776+0000 7fba397fa640 1 -- 172.21.15.29:0/3227147862 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba300b6840 con 0x7fba3c0cf690 2024-04-25T00:41:06.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.776+0000 7fba43080640 1 -- 172.21.15.29:0/3227147862 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3c0ca420 con 0x7fba3c0cf690 2024-04-25T00:41:06.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.776+0000 7fba43080640 1 -- 172.21.15.29:0/3227147862 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba3c069270 con 0x7fba3c0cf690 2024-04-25T00:41:06.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.776+0000 7fba397fa640 1 -- 172.21.15.29:0/3227147862 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fba300b69e0 con 0x7fba3c0cf690 2024-04-25T00:41:06.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.776+0000 7fba397fa640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fba20068250 0x7fba2006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:06.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.776+0000 7fba397fa640 1 -- 172.21.15.29:0/3227147862 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7fba3011e880 con 0x7fba3c0cf690 2024-04-25T00:41:06.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.776+0000 7fba3bfff640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fba20068250 0x7fba2006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:06.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.776+0000 7fba3bfff640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fba20068250 0x7fba2006a710 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fba28006fd0 tx=0x7fba28008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:06.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.780+0000 7fba397fa640 1 -- 172.21.15.29:0/3227147862 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fba30016110 con 0x7fba3c0cf690 2024-04-25T00:41:06.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.944+0000 7fba43080640 1 -- 172.21.15.29:0/3227147862 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba3c06bdd0 con 0x7fba3c0cf690 2024-04-25T00:41:06.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.944+0000 7fba397fa640 1 -- 172.21.15.29:0/3227147862 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) ==== 74+0+12982 (secure 0 0 0) 0x7fba300f0fc0 con 0x7fba3c0cf690 2024-04-25T00:41:06.947 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:06.947 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":159,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:41:05.595669+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6835/4183931647":"2024-04-26T00:41:02.581792+0000","172.21.15.29:6834/4183931647":"2024-04-26T00:41:02.581792+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:06.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.948+0000 7fba43080640 1 -- 172.21.15.29:0/3227147862 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fba20068250 msgr2=0x7fba2006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:06.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.948+0000 7fba43080640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fba20068250 0x7fba2006a710 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fba28006fd0 tx=0x7fba28008040 comp rx=0 tx=0).stop 2024-04-25T00:41:06.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.948+0000 7fba43080640 1 -- 172.21.15.29:0/3227147862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba3c0cf690 msgr2=0x7fba3c15cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:06.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.948+0000 7fba43080640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba3c0cf690 0x7fba3c15cdf0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fba300ae040 tx=0x7fba3001aa70 comp rx=0 tx=0).stop 2024-04-25T00:41:06.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.948+0000 7fba43080640 1 -- 172.21.15.29:0/3227147862 shutdown_connections 2024-04-25T00:41:06.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.948+0000 7fba43080640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fba20068250 0x7fba2006a710 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.948+0000 7fba43080640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3c15e200 0x7fba3c0b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.948+0000 7fba43080640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba3c0d2a90 0x7fba3c161070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.948+0000 7fba43080640 1 --2- 172.21.15.29:0/3227147862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba3c0cf690 0x7fba3c15cdf0 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:06.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.948+0000 7fba43080640 1 -- 172.21.15.29:0/3227147862 >> 172.21.15.29:0/3227147862 conn(0x7fba3c09d280 msgr2=0x7fba3c0b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:06.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.948+0000 7fba43080640 1 -- 172.21.15.29:0/3227147862 shutdown_connections 2024-04-25T00:41:06.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:06.948+0000 7fba43080640 1 -- 172.21.15.29:0/3227147862 wait complete. 2024-04-25T00:41:06.968 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:41:06.968 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:41:07.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc32d3b5640 1 Processor -- start 2024-04-25T00:41:07.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc32d3b5640 1 -- start start 2024-04-25T00:41:07.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc32d3b5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc32815d980 0x7fc32806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc32d3b5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3280b2b70 0x7fc32806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc32d3b5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3280b3130 con 0x7fc328150df0 2024-04-25T00:41:07.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc32d3b5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc3280cf560 con 0x7fc32815d980 2024-04-25T00:41:07.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc32d3b5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc328151990 con 0x7fc3280b2b70 2024-04-25T00:41:07.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc3277fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc328150df0 0x7fc328151280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33992/0 (socket says 172.21.15.29:33992) 2024-04-25T00:41:07.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc3277fe640 1 -- 172.21.15.29:0/1542359745 learned_addr learned my addr 172.21.15.29:0/1542359745 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:07.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc326ffd640 1 --2- 172.21.15.29:0/1542359745 >> v2:172.21.15.157:3300/0 conn(0x7fc32815d980 0x7fc32806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc3267fc640 1 --2- 172.21.15.29:0/1542359745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3280b2b70 0x7fc32806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 242489051 0 0) 0x7fc3280b3130 con 0x7fc328150df0 2024-04-25T00:41:07.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc314003530 con 0x7fc328150df0 2024-04-25T00:41:07.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 130304549 0 0) 0x7fc314003530 con 0x7fc328150df0 2024-04-25T00:41:07.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3280b2b70 msgr2=0x7fc32806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:07.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 --2- 172.21.15.29:0/1542359745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3280b2b70 0x7fc32806db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 >> v2:172.21.15.157:3300/0 conn(0x7fc32815d980 msgr2=0x7fc32806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:07.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 --2- 172.21.15.29:0/1542359745 >> v2:172.21.15.157:3300/0 conn(0x7fc32815d980 0x7fc32806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3280b3410 con 0x7fc328150df0 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fc318001680 con 0x7fc328150df0 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 >> v1:172.21.15.29:6789/0 conn(0x7fc328150df0 legacy=0x7fc328151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 --2- 172.21.15.29:0/1542359745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3280b2b70 0x7fc3140039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 --2- 172.21.15.29:0/1542359745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc314003f40 0x7fc314004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 --2- 172.21.15.29:0/1542359745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc314004d50 0x7fc3140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc3280cf560 con 0x7fc314003f40 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc328151990 con 0x7fc314004d50 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc31400dbe0 con 0x7fc3280b2b70 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc3267fc640 1 --2- 172.21.15.29:0/1542359745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc314003f40 0x7fc314004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc326ffd640 1 --2- 172.21.15.29:0/1542359745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3280b2b70 0x7fc3140039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc3277fe640 1 --2- 172.21.15.29:0/1542359745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc314004d50 0x7fc3140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc3267fc640 1 -- 172.21.15.29:0/1542359745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3280b2b70 msgr2=0x7fc3140039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc3267fc640 1 --2- 172.21.15.29:0/1542359745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3280b2b70 0x7fc3140039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc3267fc640 1 -- 172.21.15.29:0/1542359745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc314004d50 msgr2=0x7fc3140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:07.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc3267fc640 1 --2- 172.21.15.29:0/1542359745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc314004d50 0x7fc3140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc3267fc640 1 -- 172.21.15.29:0/1542359745 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc314003530 con 0x7fc314003f40 2024-04-25T00:41:07.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.100+0000 7fc3277fe640 1 --2- 172.21.15.29:0/1542359745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc314004d50 0x7fc3140051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:07.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.104+0000 7fc3267fc640 1 --2- 172.21.15.29:0/1542359745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc314003f40 0x7fc314004810 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fc310009fc0 tx=0x7fc31001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78eb8361f850e49b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:07.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.104+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3100b2070 con 0x7fc314003f40 2024-04-25T00:41:07.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.104+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3100a5c90 con 0x7fc314003f40 2024-04-25T00:41:07.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.104+0000 7fc325ffb640 1 -- 172.21.15.29:0/1542359745 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310009c10 con 0x7fc314003f40 2024-04-25T00:41:07.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.104+0000 7fc32d3b5640 1 -- 172.21.15.29:0/1542359745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc314003f40 msgr2=0x7fc314004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:07.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.104+0000 7fc32d3b5640 1 --2- 172.21.15.29:0/1542359745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc314003f40 0x7fc314004810 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fc310009fc0 tx=0x7fc31001cc20 comp rx=0 tx=0).stop 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.104+0000 7fc3277fe640 1 -- 172.21.15.29:0/1542359745 reap_dead start 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.104+0000 7fc32d3b5640 1 -- 172.21.15.29:0/1542359745 shutdown_connections 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.104+0000 7fc32d3b5640 1 -- 172.21.15.29:0/1542359745 >> 172.21.15.29:0/1542359745 conn(0x7fc3280b1870 msgr2=0x7fc3280b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.104+0000 7fc32d3b5640 1 -- 172.21.15.29:0/1542359745 shutdown_connections 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.104+0000 7fc32d3b5640 1 -- 172.21.15.29:0/1542359745 wait complete. 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.104+0000 7fc32d3b5640 1 Processor -- start 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.104+0000 7fc32d3b5640 1 -- start start 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc32d3b5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc314003f40 0x7fc328155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc32d3b5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3280cccf0 0x7fc32814f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc32d3b5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3280aec20 0x7fc3281573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc32d3b5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc3280b3cb0 con 0x7fc3280cccf0 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc32d3b5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3280b39d0 con 0x7fc314003f40 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc32d3b5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc32815dd60 con 0x7fc3280aec20 2024-04-25T00:41:07.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc3267fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3280cccf0 0x7fc32814f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc3267fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3280cccf0 0x7fc32814f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:48994/0 (socket says 172.21.15.29:48994) 2024-04-25T00:41:07.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc3267fc640 1 -- 172.21.15.29:0/3757485960 learned_addr learned my addr 172.21.15.29:0/3757485960 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:07.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc3277fe640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3280aec20 0x7fc3281573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc326ffd640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc314003f40 0x7fc328155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc3267fc640 1 -- 172.21.15.29:0/3757485960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3280aec20 msgr2=0x7fc3281573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:07.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc3267fc640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3280aec20 0x7fc3281573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc3267fc640 1 -- 172.21.15.29:0/3757485960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc314003f40 msgr2=0x7fc328155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:07.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc3267fc640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc314003f40 0x7fc328155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc3267fc640 1 -- 172.21.15.29:0/3757485960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3280ad8a0 con 0x7fc3280cccf0 2024-04-25T00:41:07.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc326ffd640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc314003f40 0x7fc328155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:07.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc3267fc640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3280cccf0 0x7fc32814f720 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fc31001c690 tx=0x7fc310009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:07.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc31f7fe640 1 -- 172.21.15.29:0/3757485960 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3100bd070 con 0x7fc3280cccf0 2024-04-25T00:41:07.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc31f7fe640 1 -- 172.21.15.29:0/3757485960 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3100adbc0 con 0x7fc3280cccf0 2024-04-25T00:41:07.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc32d3b5640 1 -- 172.21.15.29:0/3757485960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3280adb80 con 0x7fc3280cccf0 2024-04-25T00:41:07.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc31f7fe640 1 -- 172.21.15.29:0/3757485960 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3100b2040 con 0x7fc3280cccf0 2024-04-25T00:41:07.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc32d3b5640 1 -- 172.21.15.29:0/3757485960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3280ae140 con 0x7fc3280cccf0 2024-04-25T00:41:07.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc32d3b5640 1 -- 172.21.15.29:0/3757485960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc328069270 con 0x7fc3280cccf0 2024-04-25T00:41:07.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc31f7fe640 1 -- 172.21.15.29:0/3757485960 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fc3100c0070 con 0x7fc3280cccf0 2024-04-25T00:41:07.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc31f7fe640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc308068250 0x7fc30806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc326ffd640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc308068250 0x7fc30806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.108+0000 7fc31f7fe640 1 -- 172.21.15.29:0/3757485960 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7fc31011e5d0 con 0x7fc3280cccf0 2024-04-25T00:41:07.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.112+0000 7fc326ffd640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc308068250 0x7fc30806a710 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc320009b50 tx=0x7fc320008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:07.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.116+0000 7fc31f7fe640 1 -- 172.21.15.29:0/3757485960 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc310016990 con 0x7fc3280cccf0 2024-04-25T00:41:07.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc32d3b5640 1 -- 172.21.15.29:0/3757485960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc3280cf910 con 0x7fc3280cccf0 2024-04-25T00:41:07.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc31f7fe640 1 -- 172.21.15.29:0/3757485960 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) ==== 93+0+10 (secure 0 0 0) 0x7fc3100f0630 con 0x7fc3280cccf0 2024-04-25T00:41:07.277 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:41:07.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc32d3b5640 1 -- 172.21.15.29:0/3757485960 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc308068250 msgr2=0x7fc30806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:07.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc32d3b5640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc308068250 0x7fc30806a710 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc320009b50 tx=0x7fc320008040 comp rx=0 tx=0).stop 2024-04-25T00:41:07.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc32d3b5640 1 -- 172.21.15.29:0/3757485960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3280cccf0 msgr2=0x7fc32814f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:07.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc32d3b5640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3280cccf0 0x7fc32814f720 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fc31001c690 tx=0x7fc310009750 comp rx=0 tx=0).stop 2024-04-25T00:41:07.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc32d3b5640 1 -- 172.21.15.29:0/3757485960 shutdown_connections 2024-04-25T00:41:07.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc32d3b5640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc308068250 0x7fc30806a710 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc32d3b5640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3280aec20 0x7fc3281573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc32d3b5640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3280cccf0 0x7fc32814f720 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc32d3b5640 1 --2- 172.21.15.29:0/3757485960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc314003f40 0x7fc328155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc32d3b5640 1 -- 172.21.15.29:0/3757485960 >> 172.21.15.29:0/3757485960 conn(0x7fc3280b1870 msgr2=0x7fc3280af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:07.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc32d3b5640 1 -- 172.21.15.29:0/3757485960 shutdown_connections 2024-04-25T00:41:07.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.276+0000 7fc32d3b5640 1 -- 172.21.15.29:0/3757485960 wait complete. 2024-04-25T00:41:07.296 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-04-25T00:41:07.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.420+0000 7f378c22a640 1 Processor -- start 2024-04-25T00:41:07.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.420+0000 7f378c22a640 1 -- start start 2024-04-25T00:41:07.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f378c22a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f37840d4bf0 0x7f37840d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f378c22a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378415e8c0 0x7f37840b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f378c22a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37840cfe30 con 0x7f378415e4e0 2024-04-25T00:41:07.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f378c22a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f37840b6f80 con 0x7f37840d4bf0 2024-04-25T00:41:07.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f378c22a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f37840b7260 con 0x7f378415e8c0 2024-04-25T00:41:07.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f378a7c7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378415e8c0 0x7f37840b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f378a7c7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378415e8c0 0x7f37840b7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:40976/0 (socket says 172.21.15.29:40976) 2024-04-25T00:41:07.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f3789fc6640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f378415e4e0 0x7f37840d1eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:55248/0 (socket says 172.21.15.29:55248) 2024-04-25T00:41:07.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f37897c5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f37840d4bf0 0x7f37840d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f378a7c7640 1 -- 172.21.15.29:0/3697108192 learned_addr learned my addr 172.21.15.29:0/3697108192 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:07.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f378a7c7640 1 -- 172.21.15.29:0/3697108192 >> v2:172.21.15.157:3300/0 conn(0x7f37840d4bf0 msgr2=0x7f37840d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f378a7c7640 1 --2- 172.21.15.29:0/3697108192 >> v2:172.21.15.157:3300/0 conn(0x7f37840d4bf0 0x7f37840d25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f378a7c7640 1 -- 172.21.15.29:0/3697108192 >> v1:172.21.15.157:6789/0 conn(0x7f378415e4e0 legacy=0x7f37840d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f378a7c7640 1 -- 172.21.15.29:0/3697108192 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37840ced50 con 0x7f378415e8c0 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f37897c5640 1 --2- 172.21.15.29:0/3697108192 >> v2:172.21.15.157:3300/0 conn(0x7f37840d4bf0 0x7f37840d25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f378a7c7640 1 --2- 172.21.15.29:0/3697108192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378415e8c0 0x7f37840b7780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3774009870 tx=0x7f377401ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0ca26f88c9d661 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f3788fc4640 1 -- 172.21.15.29:0/3697108192 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37740b5070 con 0x7f378415e8c0 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f3788fc4640 1 -- 172.21.15.29:0/3697108192 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37740a8a50 con 0x7f378415e8c0 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.424+0000 7f3788fc4640 1 -- 172.21.15.29:0/3697108192 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37740a8e20 con 0x7f378415e8c0 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 -- 172.21.15.29:0/3697108192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378415e8c0 msgr2=0x7f37840b7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 --2- 172.21.15.29:0/3697108192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378415e8c0 0x7f37840b7780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3774009870 tx=0x7f377401ed50 comp rx=0 tx=0).stop 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 -- 172.21.15.29:0/3697108192 shutdown_connections 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 --2- 172.21.15.29:0/3697108192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f378415e8c0 0x7f37840b7780 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 --2- 172.21.15.29:0/3697108192 >> v2:172.21.15.157:3300/0 conn(0x7f37840d4bf0 0x7f37840d25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 -- 172.21.15.29:0/3697108192 >> 172.21.15.29:0/3697108192 conn(0x7f37840d3fb0 msgr2=0x7f37840d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:07.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 -- 172.21.15.29:0/3697108192 shutdown_connections 2024-04-25T00:41:07.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 -- 172.21.15.29:0/3697108192 wait complete. 2024-04-25T00:41:07.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 Processor -- start 2024-04-25T00:41:07.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 -- start start 2024-04-25T00:41:07.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37840d4bf0 0x7f37840c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f378415e4e0 0x7f37840c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f378415e8c0 0x7f3784158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f37840b7e40 con 0x7f378415e4e0 2024-04-25T00:41:07.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f37840b7cc0 con 0x7f378415e8c0 2024-04-25T00:41:07.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f37840b7fc0 con 0x7f37840d4bf0 2024-04-25T00:41:07.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f37897c5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f378415e4e0 0x7f37840c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f37897c5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f378415e4e0 0x7f37840c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49010/0 (socket says 172.21.15.29:49010) 2024-04-25T00:41:07.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f37897c5640 1 -- 172.21.15.29:0/3304532592 learned_addr learned my addr 172.21.15.29:0/3304532592 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f3789fc6640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37840d4bf0 0x7f37840c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378a7c7640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f378415e8c0 0x7f3784158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f37897c5640 1 -- 172.21.15.29:0/3304532592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37840d4bf0 msgr2=0x7f37840c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f37897c5640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37840d4bf0 0x7f37840c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f37897c5640 1 -- 172.21.15.29:0/3304532592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f378415e8c0 msgr2=0x7f3784158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f37897c5640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f378415e8c0 0x7f3784158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f37897c5640 1 -- 172.21.15.29:0/3304532592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378415eee0 con 0x7f378415e4e0 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378a7c7640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f378415e8c0 0x7f3784158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f3789fc6640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37840d4bf0 0x7f37840c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f37897c5640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f378415e4e0 0x7f37840c8cb0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f377800ebe0 tx=0x7f377800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f37827fc640 1 -- 172.21.15.29:0/3304532592 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778017070 con 0x7f378415e4e0 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f37827fc640 1 -- 172.21.15.29:0/3304532592 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f377800cc30 con 0x7f378415e4e0 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f37827fc640 1 -- 172.21.15.29:0/3304532592 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778013470 con 0x7f378415e4e0 2024-04-25T00:41:07.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 -- 172.21.15.29:0/3304532592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378415f1c0 con 0x7f378415e4e0 2024-04-25T00:41:07.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.428+0000 7f378c22a640 1 -- 172.21.15.29:0/3304532592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378415f780 con 0x7f378415e4e0 2024-04-25T00:41:07.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.432+0000 7f378c22a640 1 -- 172.21.15.29:0/3304532592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3784069270 con 0x7f378415e4e0 2024-04-25T00:41:07.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.432+0000 7f37827fc640 1 -- 172.21.15.29:0/3304532592 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f377801a070 con 0x7f378415e4e0 2024-04-25T00:41:07.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.432+0000 7f37827fc640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3770068250 0x7f377006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:07.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.432+0000 7f3789fc6640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3770068250 0x7f377006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:07.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.432+0000 7f37827fc640 1 -- 172.21.15.29:0/3304532592 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7f3778084720 con 0x7f378415e4e0 2024-04-25T00:41:07.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.432+0000 7f3789fc6640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3770068250 0x7f377006a710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f37840d3c90 tx=0x7f377c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:07.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.436+0000 7f37827fc640 1 -- 172.21.15.29:0/3304532592 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3778056e60 con 0x7f378415e4e0 2024-04-25T00:41:07.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:07.656+0000 7f378c22a640 1 -- 172.21.15.29:0/3304532592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f37840d38d0 con 0x7f378415e4e0 2024-04-25T00:41:08.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.620+0000 7f37827fc640 1 -- 172.21.15.29:0/3304532592 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v516) ==== 166+0+0 (secure 0 0 0) 0x7f3778056760 con 0x7f378415e4e0 2024-04-25T00:41:08.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.620+0000 7f378c22a640 1 -- 172.21.15.29:0/3304532592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3770068250 msgr2=0x7f377006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:08.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.620+0000 7f378c22a640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3770068250 0x7f377006a710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f37840d3c90 tx=0x7f377c009040 comp rx=0 tx=0).stop 2024-04-25T00:41:08.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.620+0000 7f378c22a640 1 -- 172.21.15.29:0/3304532592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f378415e4e0 msgr2=0x7f37840c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:08.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.620+0000 7f378c22a640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f378415e4e0 0x7f37840c8cb0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f377800ebe0 tx=0x7f377800c520 comp rx=0 tx=0).stop 2024-04-25T00:41:08.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.620+0000 7f378c22a640 1 -- 172.21.15.29:0/3304532592 shutdown_connections 2024-04-25T00:41:08.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.620+0000 7f378c22a640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3770068250 0x7f377006a710 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.620+0000 7f378c22a640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f378415e8c0 0x7f3784158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.620+0000 7f378c22a640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f378415e4e0 0x7f37840c8cb0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.620+0000 7f378c22a640 1 --2- 172.21.15.29:0/3304532592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37840d4bf0 0x7f37840c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.620+0000 7f378c22a640 1 -- 172.21.15.29:0/3304532592 >> 172.21.15.29:0/3304532592 conn(0x7f37840d3fb0 msgr2=0x7f3784066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:08.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.620+0000 7f378c22a640 1 -- 172.21.15.29:0/3304532592 shutdown_connections 2024-04-25T00:41:08.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.620+0000 7f378c22a640 1 -- 172.21.15.29:0/3304532592 wait complete. 2024-04-25T00:41:08.645 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:41:08.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.776+0000 7fb76c4eb640 1 Processor -- start 2024-04-25T00:41:08.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.776+0000 7fb76c4eb640 1 -- start start 2024-04-25T00:41:08.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.776+0000 7fb76c4eb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb764153030 0x7fb76406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:08.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.776+0000 7fb76c4eb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb76406b920 0x7fb7640b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:08.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.776+0000 7fb76c4eb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb76415d580 con 0x7fb7640cd200 2024-04-25T00:41:08.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.776+0000 7fb76c4eb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb76415d3d0 con 0x7fb764153030 2024-04-25T00:41:08.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.776+0000 7fb76c4eb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb76415c3c0 con 0x7fb76406b920 2024-04-25T00:41:08.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.780+0000 7fb76aa88640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb76406b920 0x7fb7640b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:08.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.780+0000 7fb76a287640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb764153030 0x7fb76406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:08.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.780+0000 7fb76a287640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb764153030 0x7fb76406fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:55160/0 (socket says 172.21.15.29:55160) 2024-04-25T00:41:08.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.780+0000 7fb769a86640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb7640cd200 0x7fb76406b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:53730/0 (socket says 172.21.15.29:53730) 2024-04-25T00:41:08.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.780+0000 7fb76a287640 1 -- 172.21.15.29:0/3807690618 learned_addr learned my addr 172.21.15.29:0/3807690618 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:08.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.780+0000 7fb76aa88640 1 -- 172.21.15.29:0/3807690618 >> v2:172.21.15.157:3300/0 conn(0x7fb764153030 msgr2=0x7fb76406fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:08.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.780+0000 7fb76aa88640 1 --2- 172.21.15.29:0/3807690618 >> v2:172.21.15.157:3300/0 conn(0x7fb764153030 0x7fb76406fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.780+0000 7fb76aa88640 1 -- 172.21.15.29:0/3807690618 >> v1:172.21.15.157:6789/0 conn(0x7fb7640cd200 legacy=0x7fb76406b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:08.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.780+0000 7fb76aa88640 1 -- 172.21.15.29:0/3807690618 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76415c610 con 0x7fb76406b920 2024-04-25T00:41:08.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.780+0000 7fb76aa88640 1 --2- 172.21.15.29:0/3807690618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb76406b920 0x7fb7640b5ac0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009d90 tx=0x7fb75c01c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=470edb57248a3ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:08.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.780+0000 7fb769285640 1 -- 172.21.15.29:0/3807690618 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0b2070 con 0x7fb76406b920 2024-04-25T00:41:08.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.780+0000 7fb769285640 1 -- 172.21.15.29:0/3807690618 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75c0a5740 con 0x7fb76406b920 2024-04-25T00:41:08.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.780+0000 7fb769285640 1 -- 172.21.15.29:0/3807690618 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0a5b10 con 0x7fb76406b920 2024-04-25T00:41:08.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 -- 172.21.15.29:0/3807690618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb76406b920 msgr2=0x7fb7640b5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:08.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 --2- 172.21.15.29:0/3807690618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb76406b920 0x7fb7640b5ac0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009d90 tx=0x7fb75c01c790 comp rx=0 tx=0).stop 2024-04-25T00:41:08.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 -- 172.21.15.29:0/3807690618 shutdown_connections 2024-04-25T00:41:08.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 --2- 172.21.15.29:0/3807690618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb76406b920 0x7fb7640b5ac0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 --2- 172.21.15.29:0/3807690618 >> v2:172.21.15.157:3300/0 conn(0x7fb764153030 0x7fb76406fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 -- 172.21.15.29:0/3807690618 >> 172.21.15.29:0/3807690618 conn(0x7fb7640d0c10 msgr2=0x7fb7640d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:08.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 -- 172.21.15.29:0/3807690618 shutdown_connections 2024-04-25T00:41:08.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 -- 172.21.15.29:0/3807690618 wait complete. 2024-04-25T00:41:08.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 Processor -- start 2024-04-25T00:41:08.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 -- start start 2024-04-25T00:41:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb76406b920 0x7fb76414e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7640cd200 0x7fb76414e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb764153030 0x7fb7640c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb764070550 con 0x7fb7640cd200 2024-04-25T00:41:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7640703d0 con 0x7fb76406b920 2024-04-25T00:41:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb764071d80 con 0x7fb764153030 2024-04-25T00:41:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb769a86640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7640cd200 0x7fb76414e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb769a86640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7640cd200 0x7fb76414e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52720/0 (socket says 172.21.15.29:52720) 2024-04-25T00:41:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb769a86640 1 -- 172.21.15.29:0/403839905 learned_addr learned my addr 172.21.15.29:0/403839905 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76aa88640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb764153030 0x7fb7640c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76a287640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb76406b920 0x7fb76414e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:08.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb769a86640 1 -- 172.21.15.29:0/403839905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb764153030 msgr2=0x7fb7640c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb769a86640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb764153030 0x7fb7640c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb769a86640 1 -- 172.21.15.29:0/403839905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb76406b920 msgr2=0x7fb76414e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb769a86640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb76406b920 0x7fb76414e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb769a86640 1 -- 172.21.15.29:0/403839905 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76414ee80 con 0x7fb7640cd200 2024-04-25T00:41:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76a287640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb76406b920 0x7fb76414e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76aa88640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb764153030 0x7fb7640c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb769a86640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7640cd200 0x7fb76414e620 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c500 tx=0x7fb75400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb75affd640 1 -- 172.21.15.29:0/403839905 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754018070 con 0x7fb7640cd200 2024-04-25T00:41:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb75affd640 1 -- 172.21.15.29:0/403839905 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb754007500 con 0x7fb7640cd200 2024-04-25T00:41:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb75affd640 1 -- 172.21.15.29:0/403839905 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754004a10 con 0x7fb7640cd200 2024-04-25T00:41:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 -- 172.21.15.29:0/403839905 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76414f160 con 0x7fb7640cd200 2024-04-25T00:41:08.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.784+0000 7fb76c4eb640 1 -- 172.21.15.29:0/403839905 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76414f720 con 0x7fb7640cd200 2024-04-25T00:41:08.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.788+0000 7fb76c4eb640 1 -- 172.21.15.29:0/403839905 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb764069270 con 0x7fb7640cd200 2024-04-25T00:41:08.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.788+0000 7fb75affd640 1 -- 172.21.15.29:0/403839905 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fb75401b070 con 0x7fb7640cd200 2024-04-25T00:41:08.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.788+0000 7fb75affd640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb748068250 0x7fb74806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:08.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.788+0000 7fb75affd640 1 -- 172.21.15.29:0/403839905 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7fb7540836f0 con 0x7fb7640cd200 2024-04-25T00:41:08.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.788+0000 7fb76a287640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb748068250 0x7fb74806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:08.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.788+0000 7fb76a287640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb748068250 0x7fb74806a710 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb760009b00 tx=0x7fb760008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:08.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.792+0000 7fb75affd640 1 -- 172.21.15.29:0/403839905 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb754055e30 con 0x7fb7640cd200 2024-04-25T00:41:08.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.956+0000 7fb76c4eb640 1 -- 172.21.15.29:0/403839905 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb7640cb0d0 con 0x7fb7640cd200 2024-04-25T00:41:08.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.960+0000 7fb75affd640 1 -- 172.21.15.29:0/403839905 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7fb754055730 con 0x7fb7640cd200 2024-04-25T00:41:08.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.960+0000 7fb76c4eb640 1 -- 172.21.15.29:0/403839905 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb748068250 msgr2=0x7fb74806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:08.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.960+0000 7fb76c4eb640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb748068250 0x7fb74806a710 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb760009b00 tx=0x7fb760008040 comp rx=0 tx=0).stop 2024-04-25T00:41:08.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.960+0000 7fb76c4eb640 1 -- 172.21.15.29:0/403839905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7640cd200 msgr2=0x7fb76414e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:08.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.960+0000 7fb76c4eb640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7640cd200 0x7fb76414e620 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c500 tx=0x7fb75400c9c0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.960+0000 7fb76c4eb640 1 -- 172.21.15.29:0/403839905 shutdown_connections 2024-04-25T00:41:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.960+0000 7fb76c4eb640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb748068250 0x7fb74806a710 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.960+0000 7fb76c4eb640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb764153030 0x7fb7640c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.960+0000 7fb76c4eb640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7640cd200 0x7fb76414e620 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.960+0000 7fb76c4eb640 1 --2- 172.21.15.29:0/403839905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb76406b920 0x7fb76414e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.960+0000 7fb76c4eb640 1 -- 172.21.15.29:0/403839905 >> 172.21.15.29:0/403839905 conn(0x7fb7640d0c10 msgr2=0x7fb76415dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.960+0000 7fb76c4eb640 1 -- 172.21.15.29:0/403839905 shutdown_connections 2024-04-25T00:41:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:08.960+0000 7fb76c4eb640 1 -- 172.21.15.29:0/403839905 wait complete. 2024-04-25T00:41:08.979 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:41:08.979 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:41:09.016 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:41:09.076 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:41:09.076 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:41:09.076 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:41:09.081 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:41:09.081 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:41:09.081 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:41:09.087 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:41:09.087 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:41:09.087 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:41:09.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:41:09.128 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:41:09.128 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:41:09.134 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:41:09.134 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:41:09.134 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:41:09.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.296+0000 7f5206998640 1 Processor -- start 2024-04-25T00:41:09.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.296+0000 7f5206998640 1 -- start start 2024-04-25T00:41:09.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f5206998640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52000b3140 0x7f520006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:09.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f5206998640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f52000b4b10 0x7f52000b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:09.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f5206998640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52000b41e0 con 0x7f52000b3a60 2024-04-25T00:41:09.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f5206998640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f520006d960 con 0x7f52000b4b10 2024-04-25T00:41:09.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f5206998640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f520006dc40 con 0x7f52000b3140 2024-04-25T00:41:09.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f5204f35640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f52000b3a60 0x7f520015c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59730/0 (socket says 172.21.15.29:59730) 2024-04-25T00:41:09.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f5204f35640 1 -- 172.21.15.29:0/1880851324 learned_addr learned my addr 172.21.15.29:0/1880851324 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51ff7fe640 1 --2- 172.21.15.29:0/1880851324 >> v2:172.21.15.157:3300/0 conn(0x7f52000b4b10 0x7f52000b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51fffff640 1 --2- 172.21.15.29:0/1880851324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52000b3140 0x7f520006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3294476355 0 0) 0x7f52000b41e0 con 0x7f52000b3a60 2024-04-25T00:41:09.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51e8003530 con 0x7f52000b3a60 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 649555910 0 0) 0x7f51e8003530 con 0x7f52000b3a60 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52000b3140 msgr2=0x7f520006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 --2- 172.21.15.29:0/1880851324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52000b3140 0x7f520006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 >> v2:172.21.15.157:3300/0 conn(0x7f52000b4b10 msgr2=0x7f52000b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 --2- 172.21.15.29:0/1880851324 >> v2:172.21.15.157:3300/0 conn(0x7f52000b4b10 0x7f52000b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520006df20 con 0x7f52000b3a60 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f51f8001a00 con 0x7f52000b3a60 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 >> v1:172.21.15.29:6789/0 conn(0x7f52000b3a60 legacy=0x7f520015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 --2- 172.21.15.29:0/1880851324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e80039c0 0x7f51e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 --2- 172.21.15.29:0/1880851324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51e8004810 0x7f51e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 --2- 172.21.15.29:0/1880851324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52000b3140 0x7f51e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f520006d960 con 0x7f51e80039c0 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f520006dc40 con 0x7f51e8004810 2024-04-25T00:41:09.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f51e800b9d0 con 0x7f52000b3140 2024-04-25T00:41:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51fffff640 1 --2- 172.21.15.29:0/1880851324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e80039c0 0x7f51e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51ff7fe640 1 --2- 172.21.15.29:0/1880851324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51e8004810 0x7f51e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f5204f35640 1 --2- 172.21.15.29:0/1880851324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52000b3140 0x7f51e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51fffff640 1 -- 172.21.15.29:0/1880851324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52000b3140 msgr2=0x7f51e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51fffff640 1 --2- 172.21.15.29:0/1880851324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52000b3140 0x7f51e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51fffff640 1 -- 172.21.15.29:0/1880851324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51e8004810 msgr2=0x7f51e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51fffff640 1 --2- 172.21.15.29:0/1880851324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51e8004810 0x7f51e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f51fffff640 1 -- 172.21.15.29:0/1880851324 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e8003530 con 0x7f51e80039c0 2024-04-25T00:41:09.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.300+0000 7f5204f35640 1 --2- 172.21.15.29:0/1880851324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52000b3140 0x7f51e80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:09.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f51fffff640 1 --2- 172.21.15.29:0/1880851324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e80039c0 0x7f51e8003de0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009ae0 tx=0x7f51f001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9048a32ae0808af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:09.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f00096b0 con 0x7f51e80039c0 2024-04-25T00:41:09.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f00af070 con 0x7f51e80039c0 2024-04-25T00:41:09.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f51feffd640 1 -- 172.21.15.29:0/1880851324 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f00aa640 con 0x7f51e80039c0 2024-04-25T00:41:09.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 -- 172.21.15.29:0/1880851324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e80039c0 msgr2=0x7f51e8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:09.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 --2- 172.21.15.29:0/1880851324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e80039c0 0x7f51e8003de0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009ae0 tx=0x7f51f001ca70 comp rx=0 tx=0).stop 2024-04-25T00:41:09.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5204f35640 1 -- 172.21.15.29:0/1880851324 reap_dead start 2024-04-25T00:41:09.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 -- 172.21.15.29:0/1880851324 shutdown_connections 2024-04-25T00:41:09.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 -- 172.21.15.29:0/1880851324 >> 172.21.15.29:0/1880851324 conn(0x7f52000b1870 msgr2=0x7f52000b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:09.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 -- 172.21.15.29:0/1880851324 shutdown_connections 2024-04-25T00:41:09.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 -- 172.21.15.29:0/1880851324 wait complete. 2024-04-25T00:41:09.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 Processor -- start 2024-04-25T00:41:09.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 -- start start 2024-04-25T00:41:09.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e80039c0 0x7f5200072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:09.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5200140850 0x7f5200136f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5200135320 0x7f52001357a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5200166b60 con 0x7f51e80039c0 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52001669e0 con 0x7f5200140850 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f5206998640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5200166ce0 con 0x7f5200135320 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f51fffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e80039c0 0x7f5200072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f51fffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e80039c0 0x7f5200072920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52744/0 (socket says 172.21.15.29:52744) 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f51fffff640 1 -- 172.21.15.29:0/821922700 learned_addr learned my addr 172.21.15.29:0/821922700 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.304+0000 7f51ff7fe640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5200140850 0x7f5200136f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f5204f35640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5200135320 0x7f52001357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51fffff640 1 -- 172.21.15.29:0/821922700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5200135320 msgr2=0x7f52001357a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51fffff640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5200135320 0x7f52001357a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51fffff640 1 -- 172.21.15.29:0/821922700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5200140850 msgr2=0x7f5200136f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51fffff640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5200140850 0x7f5200136f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:09.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51fffff640 1 -- 172.21.15.29:0/821922700 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5200137900 con 0x7f51e80039c0 2024-04-25T00:41:09.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f5204f35640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5200135320 0x7f52001357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:09.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51fffff640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e80039c0 0x7f5200072920 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f51f0098070 tx=0x7f51e8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:09.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51fcff9640 1 -- 172.21.15.29:0/821922700 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f000f000 con 0x7f51e80039c0 2024-04-25T00:41:09.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51fcff9640 1 -- 172.21.15.29:0/821922700 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f0004020 con 0x7f51e80039c0 2024-04-25T00:41:09.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f5206998640 1 -- 172.21.15.29:0/821922700 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520013f170 con 0x7f51e80039c0 2024-04-25T00:41:09.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51fcff9640 1 -- 172.21.15.29:0/821922700 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f00043f0 con 0x7f51e80039c0 2024-04-25T00:41:09.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f5206998640 1 -- 172.21.15.29:0/821922700 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52001412a0 con 0x7f51e80039c0 2024-04-25T00:41:09.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f5206998640 1 -- 172.21.15.29:0/821922700 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5200069270 con 0x7f51e80039c0 2024-04-25T00:41:09.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51fcff9640 1 -- 172.21.15.29:0/821922700 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f51f00a9070 con 0x7f51e80039c0 2024-04-25T00:41:09.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51fcff9640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f51ec068280 0x7f51ec06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:09.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51fcff9640 1 -- 172.21.15.29:0/821922700 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4943+0+0 (secure 0 0 0) 0x7f51f011e5d0 con 0x7f51e80039c0 2024-04-25T00:41:09.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51ff7fe640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f51ec068280 0x7f51ec06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:09.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.308+0000 7f51ff7fe640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f51ec068280 0x7f51ec06a740 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5200138620 tx=0x7f51f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:09.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.312+0000 7f51fcff9640 1 -- 172.21.15.29:0/821922700 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f51f0016780 con 0x7f51e80039c0 2024-04-25T00:41:09.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.512+0000 7f5206998640 1 -- 172.21.15.29:0/821922700 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5200141f00 con 0x7f51e80039c0 2024-04-25T00:41:09.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.512+0000 7f51fcff9640 1 -- 172.21.15.29:0/821922700 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+3873 (secure 0 0 0) 0x7f51f00f0d10 con 0x7f51e80039c0 2024-04-25T00:41:09.514 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:09.514 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":105,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2677876688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2677876688},{"type":"v1","addr":"172.21.15.157:6833","nonce":2677876688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3340901266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3340901266},{"type":"v1","addr":"172.21.15.157:6835","nonce":3340901266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":10092,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/567284262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":567284262},{"type":"v1","addr":"172.21.15.29:6833","nonce":567284262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":10500,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/263667798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":263667798},{"type":"v1","addr":"172.21.15.29:6835","nonce":263667798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105}],"filesystems":[]} 2024-04-25T00:41:09.514 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 105 2024-04-25T00:41:09.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.516+0000 7f5206998640 1 -- 172.21.15.29:0/821922700 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f51ec068280 msgr2=0x7f51ec06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:09.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.516+0000 7f5206998640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f51ec068280 0x7f51ec06a740 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5200138620 tx=0x7f51f4008040 comp rx=0 tx=0).stop 2024-04-25T00:41:09.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.516+0000 7f5206998640 1 -- 172.21.15.29:0/821922700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e80039c0 msgr2=0x7f5200072920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:09.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.516+0000 7f5206998640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e80039c0 0x7f5200072920 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f51f0098070 tx=0x7f51e8005840 comp rx=0 tx=0).stop 2024-04-25T00:41:09.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.516+0000 7f5206998640 1 -- 172.21.15.29:0/821922700 shutdown_connections 2024-04-25T00:41:09.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.516+0000 7f5206998640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f51ec068280 0x7f51ec06a740 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:09.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.516+0000 7f5206998640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5200135320 0x7f52001357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:09.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.516+0000 7f5206998640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5200140850 0x7f5200136f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:09.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.516+0000 7f5206998640 1 --2- 172.21.15.29:0/821922700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e80039c0 0x7f5200072920 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:09.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.516+0000 7f5206998640 1 -- 172.21.15.29:0/821922700 >> 172.21.15.29:0/821922700 conn(0x7f52000b1870 msgr2=0x7f520015eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:09.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.516+0000 7f5206998640 1 -- 172.21.15.29:0/821922700 shutdown_connections 2024-04-25T00:41:09.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:09.516+0000 7f5206998640 1 -- 172.21.15.29:0/821922700 wait complete. 2024-04-25T00:41:09.535 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:41:09.535 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:41:09.535 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:41:09.535 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:41:09.535 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:41:09.535 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:41:09.536 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:41:09.536 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:41:09.536 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:41:09.536 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:41:09.536 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:41:09.536 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:41:09.536 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:41:09.537 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:41:09.537 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:41:09.537 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:41:09.546 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:41:09.546 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:41:09.550 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:41:09.550 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:41:09.556 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:41:09.556 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:41:09.589 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:41:09.589 DEBUG:teuthology.parallel:result is None 2024-04-25T00:41:09.598 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:41:09.599 DEBUG:teuthology.parallel:result is None 2024-04-25T00:41:09.600 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:41:09.600 DEBUG:teuthology.parallel:result is None 2024-04-25T00:41:09.660 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-04-25T00:41:09.669 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-25T00:41:09.670 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-25T00:41:11.396 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:41:11.399 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:41:11.505 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T00:41:15.546 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:41:15.547 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:41:15.549 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:41:15.549 DEBUG:teuthology.parallel:result is None 2024-04-25T00:41:15.550 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:41:15.628 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:41:15.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.680+0000 7fd9ca97a640 1 Processor -- start 2024-04-25T00:41:15.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.680+0000 7fd9ca97a640 1 -- start start 2024-04-25T00:41:15.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.680+0000 7fd9ca97a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9c4140ca0 0x7fd9c4165a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.680+0000 7fd9ca97a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9c40cf680 con 0x7fd9c413deb0 2024-04-25T00:41:15.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.680+0000 7fd9ca97a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9c40cf960 con 0x7fd9c40cf1f0 2024-04-25T00:41:15.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.680+0000 7fd9ca97a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd9c40cfc40 con 0x7fd9c4140ca0 2024-04-25T00:41:15.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.680+0000 7fd9c3fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd9c413deb0 0x7fd9c40ceae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59744/0 (socket says 172.21.15.29:59744) 2024-04-25T00:41:15.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.680+0000 7fd9c3fff640 1 -- 172.21.15.29:0/2614062394 learned_addr learned my addr 172.21.15.29:0/2614062394 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:15.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c8f17640 1 --2- 172.21.15.29:0/2614062394 >> v2:172.21.15.157:3300/0 conn(0x7fd9c4140ca0 0x7fd9c4165a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1002143126 0 0) 0x7fd9c40cf680 con 0x7fd9c413deb0 2024-04-25T00:41:15.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9ac003530 con 0x7fd9c413deb0 2024-04-25T00:41:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4272227515 0 0) 0x7fd9ac003530 con 0x7fd9c413deb0 2024-04-25T00:41:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 >> v2:172.21.15.157:3300/0 conn(0x7fd9c4140ca0 msgr2=0x7fd9c4165a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 --2- 172.21.15.29:0/2614062394 >> v2:172.21.15.157:3300/0 conn(0x7fd9c4140ca0 0x7fd9c4165a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 >> v1:172.21.15.157:6789/0 conn(0x7fd9c40cf1f0 legacy=0x7fd9c4140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c406d4d0 con 0x7fd9c413deb0 2024-04-25T00:41:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fd9b4001680 con 0x7fd9c413deb0 2024-04-25T00:41:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 >> v1:172.21.15.29:6789/0 conn(0x7fd9c413deb0 legacy=0x7fd9c40ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 --2- 172.21.15.29:0/2614062394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9ac0039c0 0x7fd9ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 --2- 172.21.15.29:0/2614062394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9ac004810 0x7fd9ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 --2- 172.21.15.29:0/2614062394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9ac0051d0 0x7fd9ac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c40cfc40 con 0x7fd9ac0039c0 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac00c170 con 0x7fd9ac0051d0 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ac00c370 con 0x7fd9ac004810 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c3fff640 1 --2- 172.21.15.29:0/2614062394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9ac0039c0 0x7fd9ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c8f17640 1 --2- 172.21.15.29:0/2614062394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9ac0051d0 0x7fd9ac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c37fe640 1 --2- 172.21.15.29:0/2614062394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9ac004810 0x7fd9ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c3fff640 1 -- 172.21.15.29:0/2614062394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9ac004810 msgr2=0x7fd9ac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c3fff640 1 --2- 172.21.15.29:0/2614062394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9ac004810 0x7fd9ac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c3fff640 1 -- 172.21.15.29:0/2614062394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9ac0051d0 msgr2=0x7fd9ac00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c3fff640 1 --2- 172.21.15.29:0/2614062394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9ac0051d0 0x7fd9ac00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c8f17640 1 -- 172.21.15.29:0/2614062394 reap_dead start 2024-04-25T00:41:15.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c3fff640 1 -- 172.21.15.29:0/2614062394 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac003530 con 0x7fd9ac0039c0 2024-04-25T00:41:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c3fff640 1 --2- 172.21.15.29:0/2614062394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9ac0039c0 0x7fd9ac003de0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400c240 tx=0x7fd9b401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5568e5765dbf55f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40aa070 con 0x7fd9ac0039c0 2024-04-25T00:41:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b4005ae0 con 0x7fd9ac0039c0 2024-04-25T00:41:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.684+0000 7fd9c2ffd640 1 -- 172.21.15.29:0/2614062394 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4007d20 con 0x7fd9ac0039c0 2024-04-25T00:41:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 -- 172.21.15.29:0/2614062394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9ac0039c0 msgr2=0x7fd9ac003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 --2- 172.21.15.29:0/2614062394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9ac0039c0 0x7fd9ac003de0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400c240 tx=0x7fd9b401cc40 comp rx=0 tx=0).stop 2024-04-25T00:41:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 -- 172.21.15.29:0/2614062394 shutdown_connections 2024-04-25T00:41:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 --2- 172.21.15.29:0/2614062394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9ac0039c0 0x7fd9ac003de0 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 -- 172.21.15.29:0/2614062394 >> 172.21.15.29:0/2614062394 conn(0x7fd9c4075030 msgr2=0x7fd9c40b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 -- 172.21.15.29:0/2614062394 shutdown_connections 2024-04-25T00:41:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 -- 172.21.15.29:0/2614062394 wait complete. 2024-04-25T00:41:15.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 Processor -- start 2024-04-25T00:41:15.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 -- start start 2024-04-25T00:41:15.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9ac0039c0 0x7fd9c4159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9c41551b0 0x7fd9c4155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c40c5a80 0x7fd9c40c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c415f290 con 0x7fd9c41551b0 2024-04-25T00:41:15.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c415f110 con 0x7fd9c40c5a80 2024-04-25T00:41:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9c415f410 con 0x7fd9ac0039c0 2024-04-25T00:41:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9c41551b0 0x7fd9c4155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9c41551b0 0x7fd9c4155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52788/0 (socket says 172.21.15.29:52788) 2024-04-25T00:41:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c37fe640 1 -- 172.21.15.29:0/3610380844 learned_addr learned my addr 172.21.15.29:0/3610380844 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c3fff640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9ac0039c0 0x7fd9c4159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c8f17640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c40c5a80 0x7fd9c40c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c37fe640 1 -- 172.21.15.29:0/3610380844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9ac0039c0 msgr2=0x7fd9c4159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c37fe640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9ac0039c0 0x7fd9c4159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c37fe640 1 -- 172.21.15.29:0/3610380844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c40c5a80 msgr2=0x7fd9c40c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c37fe640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c40c5a80 0x7fd9c40c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c37fe640 1 -- 172.21.15.29:0/3610380844 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c4155b70 con 0x7fd9c41551b0 2024-04-25T00:41:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c3fff640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9ac0039c0 0x7fd9c4159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c8f17640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c40c5a80 0x7fd9c40c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c37fe640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9c41551b0 0x7fd9c4155630 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800e4b0 tx=0x7fd9b800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c0ff9640 1 -- 172.21.15.29:0/3610380844 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b80052e0 con 0x7fd9c41551b0 2024-04-25T00:41:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c0ff9640 1 -- 172.21.15.29:0/3610380844 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b8018070 con 0x7fd9c41551b0 2024-04-25T00:41:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 -- 172.21.15.29:0/3610380844 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c40c67f0 con 0x7fd9c41551b0 2024-04-25T00:41:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9c0ff9640 1 -- 172.21.15.29:0/3610380844 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8005480 con 0x7fd9c41551b0 2024-04-25T00:41:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 -- 172.21.15.29:0/3610380844 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c4143330 con 0x7fd9c41551b0 2024-04-25T00:41:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.688+0000 7fd9ca97a640 1 -- 172.21.15.29:0/3610380844 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c4069270 con 0x7fd9c41551b0 2024-04-25T00:41:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.692+0000 7fd9c0ff9640 1 -- 172.21.15.29:0/3610380844 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fd9b8003840 con 0x7fd9c41551b0 2024-04-25T00:41:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.692+0000 7fd9c0ff9640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9b0068250 0x7fd9b006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.692+0000 7fd9c0ff9640 1 -- 172.21.15.29:0/3610380844 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4943+0+0 (secure 0 0 0) 0x7fd9b8084080 con 0x7fd9c41551b0 2024-04-25T00:41:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.692+0000 7fd9c3fff640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9b0068250 0x7fd9b006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.692+0000 7fd9c3fff640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9b0068250 0x7fd9b006a710 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40ac000 tx=0x7fd9b40b1040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:15.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.696+0000 7fd9c0ff9640 1 -- 172.21.15.29:0/3610380844 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd9b80567c0 con 0x7fd9c41551b0 2024-04-25T00:41:15.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:15.868+0000 7fd9ca97a640 1 -- 172.21.15.29:0/3610380844 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd9c415e470 con 0x7fd9c41551b0 2024-04-25T00:41:16.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.656+0000 7fd9c0ff9640 1 -- 172.21.15.29:0/3610380844 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v164) ==== 97+0+0 (secure 0 0 0) 0x7fd9b80560c0 con 0x7fd9c41551b0 2024-04-25T00:41:16.656 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-04-25T00:41:16.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.660+0000 7fd9ca97a640 1 -- 172.21.15.29:0/3610380844 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9b0068250 msgr2=0x7fd9b006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:16.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.660+0000 7fd9ca97a640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9b0068250 0x7fd9b006a710 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40ac000 tx=0x7fd9b40b1040 comp rx=0 tx=0).stop 2024-04-25T00:41:16.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.660+0000 7fd9ca97a640 1 -- 172.21.15.29:0/3610380844 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9c41551b0 msgr2=0x7fd9c4155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:16.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.660+0000 7fd9ca97a640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9c41551b0 0x7fd9c4155630 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800e4b0 tx=0x7fd9b800e970 comp rx=0 tx=0).stop 2024-04-25T00:41:16.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.660+0000 7fd9ca97a640 1 -- 172.21.15.29:0/3610380844 shutdown_connections 2024-04-25T00:41:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.660+0000 7fd9ca97a640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd9b0068250 0x7fd9b006a710 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.660+0000 7fd9ca97a640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9c40c5a80 0x7fd9c40c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.660+0000 7fd9ca97a640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9c41551b0 0x7fd9c4155630 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.660+0000 7fd9ca97a640 1 --2- 172.21.15.29:0/3610380844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9ac0039c0 0x7fd9c4159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.660+0000 7fd9ca97a640 1 -- 172.21.15.29:0/3610380844 >> 172.21.15.29:0/3610380844 conn(0x7fd9c4075030 msgr2=0x7fd9c415a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.660+0000 7fd9ca97a640 1 -- 172.21.15.29:0/3610380844 shutdown_connections 2024-04-25T00:41:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.660+0000 7fd9ca97a640 1 -- 172.21.15.29:0/3610380844 wait complete. 2024-04-25T00:41:16.684 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:41:16.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.812+0000 7f63b417a640 1 Processor -- start 2024-04-25T00:41:16.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.812+0000 7f63b417a640 1 -- start start 2024-04-25T00:41:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.812+0000 7f63b417a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f63ac15e8d0 0x7f63ac0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.812+0000 7f63b417a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63ac0b3720 con 0x7f63ac0b4780 2024-04-25T00:41:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.812+0000 7f63b417a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63ac0b39c0 con 0x7f63ac15f3c0 2024-04-25T00:41:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.812+0000 7f63b417a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f63ac0b42f0 con 0x7f63ac15e8d0 2024-04-25T00:41:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.812+0000 7f63b1f16640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f63ac15e8d0 0x7f63ac0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:16.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.812+0000 7f63b1715640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f63ac0b4780 0x7f63ac15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59748/0 (socket says 172.21.15.29:59748) 2024-04-25T00:41:16.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.812+0000 7f63b1f16640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f63ac15e8d0 0x7f63ac0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52798/0 (socket says 172.21.15.29:52798) 2024-04-25T00:41:16.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.812+0000 7f63b1715640 1 -- 172.21.15.29:0/1272423877 learned_addr learned my addr 172.21.15.29:0/1272423877 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:16.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.812+0000 7f63b1f16640 1 -- 172.21.15.29:0/1272423877 >> v1:172.21.15.157:6789/0 conn(0x7f63ac15f3c0 legacy=0x7f63ac15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:16.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.812+0000 7f63b1f16640 1 -- 172.21.15.29:0/1272423877 >> v1:172.21.15.29:6789/0 conn(0x7f63ac0b4780 legacy=0x7f63ac15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:16.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.812+0000 7f63b1f16640 1 -- 172.21.15.29:0/1272423877 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac0b3ca0 con 0x7f63ac15e8d0 2024-04-25T00:41:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b1f16640 1 --2- 172.21.15.29:0/1272423877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63ac15e8d0 0x7f63ac0cc240 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f63a80097c0 tx=0x7f63a801e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357192f711099a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b0f14640 1 -- 172.21.15.29:0/1272423877 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a80ab610 con 0x7f63ac15e8d0 2024-04-25T00:41:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b0f14640 1 -- 172.21.15.29:0/1272423877 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a80ab7b0 con 0x7f63ac15e8d0 2024-04-25T00:41:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b0f14640 1 -- 172.21.15.29:0/1272423877 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a80abb80 con 0x7f63ac15e8d0 2024-04-25T00:41:16.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 -- 172.21.15.29:0/1272423877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63ac15e8d0 msgr2=0x7f63ac0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:16.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 --2- 172.21.15.29:0/1272423877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63ac15e8d0 0x7f63ac0cc240 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f63a80097c0 tx=0x7f63a801e670 comp rx=0 tx=0).stop 2024-04-25T00:41:16.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 -- 172.21.15.29:0/1272423877 shutdown_connections 2024-04-25T00:41:16.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 --2- 172.21.15.29:0/1272423877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63ac15e8d0 0x7f63ac0cc240 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:16.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 -- 172.21.15.29:0/1272423877 >> 172.21.15.29:0/1272423877 conn(0x7f63ac0b1870 msgr2=0x7f63ac0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:16.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 -- 172.21.15.29:0/1272423877 shutdown_connections 2024-04-25T00:41:16.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 -- 172.21.15.29:0/1272423877 wait complete. 2024-04-25T00:41:16.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 Processor -- start 2024-04-25T00:41:16.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 -- start start 2024-04-25T00:41:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac0b4780 0x7f63ac157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63ac15e8d0 0x7f63ac157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63ac15f3c0 0x7f63ac1583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac06d710 con 0x7f63ac15e8d0 2024-04-25T00:41:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac06d590 con 0x7f63ac0b4780 2024-04-25T00:41:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b417a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f63ac06d890 con 0x7f63ac15f3c0 2024-04-25T00:41:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b1715640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63ac15e8d0 0x7f63ac157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b1715640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63ac15e8d0 0x7f63ac157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52804/0 (socket says 172.21.15.29:52804) 2024-04-25T00:41:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.816+0000 7f63b1715640 1 -- 172.21.15.29:0/2084699428 learned_addr learned my addr 172.21.15.29:0/2084699428 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b1f16640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac0b4780 0x7f63ac157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b2717640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63ac15f3c0 0x7f63ac1583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b1715640 1 -- 172.21.15.29:0/2084699428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63ac15f3c0 msgr2=0x7f63ac1583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b1715640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63ac15f3c0 0x7f63ac1583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b1715640 1 -- 172.21.15.29:0/2084699428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac0b4780 msgr2=0x7f63ac157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b1715640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac0b4780 0x7f63ac157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b1715640 1 -- 172.21.15.29:0/2084699428 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac157760 con 0x7f63ac15e8d0 2024-04-25T00:41:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b2717640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63ac15f3c0 0x7f63ac1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b1715640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63ac15e8d0 0x7f63ac157e80 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f639c00c420 tx=0x7f639c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63a27fc640 1 -- 172.21.15.29:0/2084699428 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c017070 con 0x7f63ac15e8d0 2024-04-25T00:41:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63a27fc640 1 -- 172.21.15.29:0/2084699428 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f639c005cf0 con 0x7f63ac15e8d0 2024-04-25T00:41:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b417a640 1 -- 172.21.15.29:0/2084699428 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ac1394f0 con 0x7f63ac15e8d0 2024-04-25T00:41:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63a27fc640 1 -- 172.21.15.29:0/2084699428 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c0072f0 con 0x7f63ac15e8d0 2024-04-25T00:41:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b417a640 1 -- 172.21.15.29:0/2084699428 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ac139ab0 con 0x7f63ac15e8d0 2024-04-25T00:41:16.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b417a640 1 -- 172.21.15.29:0/2084699428 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ac06e7b0 con 0x7f63ac15e8d0 2024-04-25T00:41:16.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63a27fc640 1 -- 172.21.15.29:0/2084699428 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f639c01a070 con 0x7f63ac15e8d0 2024-04-25T00:41:16.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63a27fc640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6390068250 0x7f639006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:16.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63a27fc640 1 -- 172.21.15.29:0/2084699428 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4857+0+0 (secure 0 0 0) 0x7f639c083c70 con 0x7f63ac15e8d0 2024-04-25T00:41:16.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b1f16640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6390068250 0x7f639006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:16.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.820+0000 7f63b1f16640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6390068250 0x7f639006a710 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f63a80b0000 tx=0x7f63a80a9000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:16.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:16.824+0000 7f63a27fc640 1 -- 172.21.15.29:0/2084699428 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f639c021020 con 0x7f63ac15e8d0 2024-04-25T00:41:17.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.040+0000 7f63b417a640 1 -- 172.21.15.29:0/2084699428 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f63ac069250 con 0x7f63ac15e8d0 2024-04-25T00:41:17.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.044+0000 7f63a27fc640 1 -- 172.21.15.29:0/2084699428 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v27) ==== 80+0+5304 (secure 0 0 0) 0x7f639c0563b0 con 0x7f63ac15e8d0 2024-04-25T00:41:17.044 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:17.044 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:41:17.044 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-04-25T00:41:17.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.044 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-04-25T00:41:17.044 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmyzgKAxAAU+nrUJgQhDX6xlARNFI/ng==", 2024-04-25T00:41:17.044 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:41:17.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:41:17.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:41:17.045 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.045 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.045 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-04-25T00:41:17.045 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmtyzPERAAxNUDANw1H/ZzOo+dhgzxlg==", 2024-04-25T00:41:17.045 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:41:17.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:41:17.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:41:17.046 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.046 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.046 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-04-25T00:41:17.046 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmYUl8CxAAj6pV1+s4a0rpWLq2EgZ+8w==", 2024-04-25T00:41:17.046 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:41:17.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:41:17.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:41:17.047 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.047 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.047 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-04-25T00:41:17.047 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmuAhKGhAAOLIcWUodW5TqtVgHWZl12A==", 2024-04-25T00:41:17.047 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:41:17.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:41:17.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:41:17.047 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.047 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.048 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-04-25T00:41:17.048 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAzpClmudk6MhAA6YFyd57jwtB2pdOaad7bxQ==", 2024-04-25T00:41:17.048 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:41:17.048 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:41:17.048 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.048 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.048 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-04-25T00:41:17.048 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA2pClm76q0NRAAxWhEP2ZGoFnZcuiQyGjglw==", 2024-04-25T00:41:17.049 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.049 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:41:17.049 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:41:17.049 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.049 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.049 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-04-25T00:41:17.049 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA6pClmcgULABAA2F5WBR9Df0v4Sn875AzF7g==", 2024-04-25T00:41:17.049 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.049 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:41:17.049 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:41:17.050 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.050 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.050 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-04-25T00:41:17.050 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQA9pClm9QuRBhAALFLruv4JU/uNZYMX0yNuXw==", 2024-04-25T00:41:17.050 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.050 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:41:17.050 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:41:17.050 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.050 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.051 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-04-25T00:41:17.051 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBBpClmNj4VDBAAxaxMvyaCvjynPY3f81udnw==", 2024-04-25T00:41:17.051 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.051 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:41:17.051 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:41:17.051 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.051 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.051 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-04-25T00:41:17.051 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBEpClmzBybEBAArry1lM4RIsM/txKfgBqDpw==", 2024-04-25T00:41:17.052 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.052 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:41:17.052 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:41:17.052 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.052 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.052 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-04-25T00:41:17.052 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBHpClm5HfjFhAAbFwJCpbhegLWDu0mbS8FoA==", 2024-04-25T00:41:17.052 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.052 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:41:17.052 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:41:17.053 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.053 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.053 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-04-25T00:41:17.053 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBKpClme/7oHBAAOR2iQ6CweexR7sC1PgesZw==", 2024-04-25T00:41:17.053 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.053 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-04-25T00:41:17.053 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-04-25T00:41:17.053 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.053 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.054 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-04-25T00:41:17.054 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A==", 2024-04-25T00:41:17.054 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.054 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:41:17.054 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-04-25T00:41:17.054 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:41:17.054 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.054 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-04-25T00:41:17.055 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClm8W/EJxAA0HZW24Hhcwf+soSZn6jHJg==", 2024-04-25T00:41:17.055 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.055 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:41:17.055 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:41:17.055 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:41:17.055 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:41:17.055 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.055 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-04-25T00:41:17.055 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmGuQJJRAAhVd4D0jc6Whgyf+B6xgFMw==", 2024-04-25T00:41:17.055 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.055 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:41:17.056 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:41:17.056 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:41:17.056 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:41:17.056 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.056 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-04-25T00:41:17.056 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAypClmn+ciLRAAg4tQuZ13WP0RnWJaJq5fUQ==", 2024-04-25T00:41:17.056 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.056 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-04-25T00:41:17.057 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-04-25T00:41:17.057 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-04-25T00:41:17.057 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-04-25T00:41:17.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.057 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-04-25T00:41:17.057 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmPVihJBAA7PQge/teyRZKSkBnufZSZw==", 2024-04-25T00:41:17.057 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.057 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:41:17.057 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-04-25T00:41:17.057 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-04-25T00:41:17.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.058 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.058 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-04-25T00:41:17.058 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClm6WrdOhAAsY9FLCxG7PGpq2NCY4hGhQ==", 2024-04-25T00:41:17.058 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.058 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:41:17.058 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:41:17.058 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.058 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:17.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:17.059 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-04-25T00:41:17.059 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQAxpClmeR4UOBAANs9SRolOm6E8+/4ciupk5w==", 2024-04-25T00:41:17.059 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-04-25T00:41:17.059 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-04-25T00:41:17.059 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-04-25T00:41:17.059 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-04-25T00:41:17.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:17.059 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-04-25T00:41:17.059 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:41:17.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.044+0000 7f63b417a640 1 -- 172.21.15.29:0/2084699428 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6390068250 msgr2=0x7f639006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.044+0000 7f63b417a640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6390068250 0x7f639006a710 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f63a80b0000 tx=0x7f63a80a9000 comp rx=0 tx=0).stop 2024-04-25T00:41:17.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.044+0000 7f63b417a640 1 -- 172.21.15.29:0/2084699428 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63ac15e8d0 msgr2=0x7f63ac157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.044+0000 7f63b417a640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63ac15e8d0 0x7f63ac157e80 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f639c00c420 tx=0x7f639c00c8e0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.044+0000 7f63b417a640 1 -- 172.21.15.29:0/2084699428 shutdown_connections 2024-04-25T00:41:17.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.044+0000 7f63b417a640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6390068250 0x7f639006a710 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.044+0000 7f63b417a640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63ac15f3c0 0x7f63ac1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.044+0000 7f63b417a640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63ac15e8d0 0x7f63ac157e80 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.044+0000 7f63b417a640 1 --2- 172.21.15.29:0/2084699428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac0b4780 0x7f63ac157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.044+0000 7f63b417a640 1 -- 172.21.15.29:0/2084699428 >> 172.21.15.29:0/2084699428 conn(0x7f63ac0b1870 msgr2=0x7f63ac0d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:17.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.044+0000 7f63b417a640 1 -- 172.21.15.29:0/2084699428 shutdown_connections 2024-04-25T00:41:17.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.044+0000 7f63b417a640 1 -- 172.21.15.29:0/2084699428 wait complete. 2024-04-25T00:41:17.064 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:17.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9ade03640 1 Processor -- start 2024-04-25T00:41:17.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9ade03640 1 -- start start 2024-04-25T00:41:17.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9ade03640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9a80cf690 0x7fe9a80d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9ade03640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe9a80d2a90 0x7fe9a80b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9ade03640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9a80d2ea0 con 0x7fe9a815e200 2024-04-25T00:41:17.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9ade03640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a80b38d0 con 0x7fe9a80d2a90 2024-04-25T00:41:17.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9ade03640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a80cfc10 con 0x7fe9a80cf690 2024-04-25T00:41:17.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9a77fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe9a815e200 0x7fe9a80d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59760/0 (socket says 172.21.15.29:59760) 2024-04-25T00:41:17.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9a7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe9a80d2a90 0x7fe9a80b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9a77fe640 1 -- 172.21.15.29:0/1143256030 learned_addr learned my addr 172.21.15.29:0/1143256030 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:17.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9a6ffd640 1 --2- 172.21.15.29:0/1143256030 >> v2:172.21.15.157:3300/0 conn(0x7fe9a80cf690 0x7fe9a80d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9a7fff640 1 -- 172.21.15.29:0/1143256030 >> v2:172.21.15.157:3300/0 conn(0x7fe9a80cf690 msgr2=0x7fe9a80d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:17.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9a7fff640 1 --2- 172.21.15.29:0/1143256030 >> v2:172.21.15.157:3300/0 conn(0x7fe9a80cf690 0x7fe9a80d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9a7fff640 1 -- 172.21.15.29:0/1143256030 >> v1:172.21.15.29:6789/0 conn(0x7fe9a815e200 legacy=0x7fe9a80d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9a7fff640 1 -- 172.21.15.29:0/1143256030 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a80cfef0 con 0x7fe9a80d2a90 2024-04-25T00:41:17.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9a67fc640 1 -- 172.21.15.29:0/1143256030 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 201472445 0 0) 0x7fe9980014a0 con 0x7fe9a815e200 2024-04-25T00:41:17.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.192+0000 7fe9a6ffd640 1 --2- 172.21.15.29:0/1143256030 >> v2:172.21.15.157:3300/0 conn(0x7fe9a80cf690 0x7fe9a80d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a7fff640 1 --2- 172.21.15.29:0/1143256030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9a80d2a90 0x7fe9a80b4320 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80d24a0 tx=0x7fe9a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46dd8ad95a187d03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a67fc640 1 -- 172.21.15.29:0/1143256030 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a00b0070 con 0x7fe9a80d2a90 2024-04-25T00:41:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a67fc640 1 -- 172.21.15.29:0/1143256030 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a00ab600 con 0x7fe9a80d2a90 2024-04-25T00:41:17.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a67fc640 1 -- 172.21.15.29:0/1143256030 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a00ab9d0 con 0x7fe9a80d2a90 2024-04-25T00:41:17.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 -- 172.21.15.29:0/1143256030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9a80d2a90 msgr2=0x7fe9a80b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 --2- 172.21.15.29:0/1143256030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9a80d2a90 0x7fe9a80b4320 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80d24a0 tx=0x7fe9a001c480 comp rx=0 tx=0).stop 2024-04-25T00:41:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 -- 172.21.15.29:0/1143256030 shutdown_connections 2024-04-25T00:41:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 --2- 172.21.15.29:0/1143256030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9a80d2a90 0x7fe9a80b4320 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 --2- 172.21.15.29:0/1143256030 >> v2:172.21.15.157:3300/0 conn(0x7fe9a80cf690 0x7fe9a80d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 -- 172.21.15.29:0/1143256030 >> 172.21.15.29:0/1143256030 conn(0x7fe9a809d280 msgr2=0x7fe9a80d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 -- 172.21.15.29:0/1143256030 shutdown_connections 2024-04-25T00:41:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 -- 172.21.15.29:0/1143256030 wait complete. 2024-04-25T00:41:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 Processor -- start 2024-04-25T00:41:17.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 -- start start 2024-04-25T00:41:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9a80cf690 0x7fe9a815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a80d2a90 0x7fe9a814eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9a815e200 0x7fe9a815c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a80b51f0 con 0x7fe9a80cf690 2024-04-25T00:41:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a80b5070 con 0x7fe9a815e200 2024-04-25T00:41:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9ade03640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a80b5370 con 0x7fe9a80d2a90 2024-04-25T00:41:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9a80cf690 0x7fe9a815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9a80cf690 0x7fe9a815bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52826/0 (socket says 172.21.15.29:52826) 2024-04-25T00:41:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a77fe640 1 -- 172.21.15.29:0/2360335092 learned_addr learned my addr 172.21.15.29:0/2360335092 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a7fff640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9a815e200 0x7fe9a815c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a6ffd640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a80d2a90 0x7fe9a814eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a77fe640 1 -- 172.21.15.29:0/2360335092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a80d2a90 msgr2=0x7fe9a814eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a77fe640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a80d2a90 0x7fe9a814eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a77fe640 1 -- 172.21.15.29:0/2360335092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9a815e200 msgr2=0x7fe9a815c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a77fe640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9a815e200 0x7fe9a815c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a77fe640 1 -- 172.21.15.29:0/2360335092 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a815d270 con 0x7fe9a80cf690 2024-04-25T00:41:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a6ffd640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a80d2a90 0x7fe9a814eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.196+0000 7fe9a77fe640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9a80cf690 0x7fe9a815bfc0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fe99800c7c0 tx=0x7fe99800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.200+0000 7fe98ffff640 1 -- 172.21.15.29:0/2360335092 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998018070 con 0x7fe9a80cf690 2024-04-25T00:41:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.200+0000 7fe98ffff640 1 -- 172.21.15.29:0/2360335092 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99800a640 con 0x7fe9a80cf690 2024-04-25T00:41:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.200+0000 7fe9ade03640 1 -- 172.21.15.29:0/2360335092 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a815b200 con 0x7fe9a80cf690 2024-04-25T00:41:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.200+0000 7fe98ffff640 1 -- 172.21.15.29:0/2360335092 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99800aa30 con 0x7fe9a80cf690 2024-04-25T00:41:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.200+0000 7fe9ade03640 1 -- 172.21.15.29:0/2360335092 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a815b7c0 con 0x7fe9a80cf690 2024-04-25T00:41:17.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.200+0000 7fe9ade03640 1 -- 172.21.15.29:0/2360335092 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a8069270 con 0x7fe9a80cf690 2024-04-25T00:41:17.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.200+0000 7fe98ffff640 1 -- 172.21.15.29:0/2360335092 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fe99801b070 con 0x7fe9a80cf690 2024-04-25T00:41:17.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.200+0000 7fe98ffff640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe994068250 0x7fe99406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.200+0000 7fe9a6ffd640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe994068250 0x7fe99406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.200+0000 7fe98ffff640 1 -- 172.21.15.29:0/2360335092 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4857+0+0 (secure 0 0 0) 0x7fe998083a30 con 0x7fe9a80cf690 2024-04-25T00:41:17.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.200+0000 7fe9a6ffd640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe994068250 0x7fe99406a710 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8150e30 tx=0x7fe99c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:17.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.204+0000 7fe98ffff640 1 -- 172.21.15.29:0/2360335092 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe9a80d2ea0 con 0x7fe9a80cf690 2024-04-25T00:41:17.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.368+0000 7fe9ade03640 1 -- 172.21.15.29:0/2360335092 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9a80d4c00 con 0x7fe9a80cf690 2024-04-25T00:41:17.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.368+0000 7fe98ffff640 1 -- 172.21.15.29:0/2360335092 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) ==== 74+0+12853 (secure 0 0 0) 0x7fe998056170 con 0x7fe9a80cf690 2024-04-25T00:41:17.369 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:17.369 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":164,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:41:16.656203+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:17.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.368+0000 7fe9ade03640 1 -- 172.21.15.29:0/2360335092 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe994068250 msgr2=0x7fe99406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.368+0000 7fe9ade03640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe994068250 0x7fe99406a710 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8150e30 tx=0x7fe99c008040 comp rx=0 tx=0).stop 2024-04-25T00:41:17.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.368+0000 7fe9ade03640 1 -- 172.21.15.29:0/2360335092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9a80cf690 msgr2=0x7fe9a815bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.368+0000 7fe9ade03640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9a80cf690 0x7fe9a815bfc0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fe99800c7c0 tx=0x7fe99800cc80 comp rx=0 tx=0).stop 2024-04-25T00:41:17.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.368+0000 7fe9ade03640 1 -- 172.21.15.29:0/2360335092 shutdown_connections 2024-04-25T00:41:17.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.368+0000 7fe9ade03640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe994068250 0x7fe99406a710 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.368+0000 7fe9ade03640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9a815e200 0x7fe9a815c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.368+0000 7fe9ade03640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a80d2a90 0x7fe9a814eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.368+0000 7fe9ade03640 1 --2- 172.21.15.29:0/2360335092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9a80cf690 0x7fe9a815bfc0 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.368+0000 7fe9ade03640 1 -- 172.21.15.29:0/2360335092 >> 172.21.15.29:0/2360335092 conn(0x7fe9a809d280 msgr2=0x7fe9a80d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:17.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.372+0000 7fe9ade03640 1 -- 172.21.15.29:0/2360335092 shutdown_connections 2024-04-25T00:41:17.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.372+0000 7fe9ade03640 1 -- 172.21.15.29:0/2360335092 wait complete. 2024-04-25T00:41:17.388 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:41:17.389 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:41:17.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.508+0000 7fc4368f6640 1 Processor -- start 2024-04-25T00:41:17.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc4368f6640 1 -- start start 2024-04-25T00:41:17.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc4368f6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4300b3c30 0x7fc43006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc4368f6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4300b41e0 con 0x7fc4300b3140 2024-04-25T00:41:17.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc4368f6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc43006d960 con 0x7fc4300b4b10 2024-04-25T00:41:17.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc4368f6640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc43006dc40 con 0x7fc4300b3c30 2024-04-25T00:41:17.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc42f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4300b3c30 0x7fc43006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc42f7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4300b3c30 0x7fc43006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52838/0 (socket says 172.21.15.29:52838) 2024-04-25T00:41:17.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc42ffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc4300b3140 0x7fc4300b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59762/0 (socket says 172.21.15.29:59762) 2024-04-25T00:41:17.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc42f7fe640 1 -- 172.21.15.29:0/1046576632 learned_addr learned my addr 172.21.15.29:0/1046576632 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:17.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc42f7fe640 1 -- 172.21.15.29:0/1046576632 >> v1:172.21.15.157:6789/0 conn(0x7fc4300b4b10 legacy=0x7fc4300ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:17.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc42f7fe640 1 -- 172.21.15.29:0/1046576632 >> v1:172.21.15.29:6789/0 conn(0x7fc4300b3140 legacy=0x7fc4300b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc42f7fe640 1 -- 172.21.15.29:0/1046576632 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43006df20 con 0x7fc4300b3c30 2024-04-25T00:41:17.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc42effd640 1 -- 172.21.15.29:0/1046576632 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 792720653 0 0) 0x7fc4300b41e0 con 0x7fc4300b3140 2024-04-25T00:41:17.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.512+0000 7fc42f7fe640 1 --2- 172.21.15.29:0/1046576632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4300b3c30 0x7fc43006e440 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fc4240098a0 tx=0x7fc42401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710f1a1fe8bda977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:17.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc42effd640 1 -- 172.21.15.29:0/1046576632 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4240b0070 con 0x7fc4300b3c30 2024-04-25T00:41:17.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc42effd640 1 -- 172.21.15.29:0/1046576632 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4240ab600 con 0x7fc4300b3c30 2024-04-25T00:41:17.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc42effd640 1 -- 172.21.15.29:0/1046576632 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4240ab9d0 con 0x7fc4300b3c30 2024-04-25T00:41:17.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 -- 172.21.15.29:0/1046576632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4300b3c30 msgr2=0x7fc43006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 --2- 172.21.15.29:0/1046576632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4300b3c30 0x7fc43006e440 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fc4240098a0 tx=0x7fc42401c500 comp rx=0 tx=0).stop 2024-04-25T00:41:17.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 -- 172.21.15.29:0/1046576632 shutdown_connections 2024-04-25T00:41:17.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 --2- 172.21.15.29:0/1046576632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4300b3c30 0x7fc43006e440 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 -- 172.21.15.29:0/1046576632 >> 172.21.15.29:0/1046576632 conn(0x7fc4300b1870 msgr2=0x7fc4300b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:17.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 -- 172.21.15.29:0/1046576632 shutdown_connections 2024-04-25T00:41:17.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 -- 172.21.15.29:0/1046576632 wait complete. 2024-04-25T00:41:17.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 Processor -- start 2024-04-25T00:41:17.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 -- start start 2024-04-25T00:41:17.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4300b3140 0x7fc43013c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4300b3c30 0x7fc430158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4300b4b10 0x7fc430159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc430169d90 con 0x7fc4300b4b10 2024-04-25T00:41:17.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc430169c10 con 0x7fc4300b3140 2024-04-25T00:41:17.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc4368f6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc430169f10 con 0x7fc4300b3c30 2024-04-25T00:41:17.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc434e93640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4300b4b10 0x7fc430159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc434e93640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4300b4b10 0x7fc430159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52848/0 (socket says 172.21.15.29:52848) 2024-04-25T00:41:17.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc434e93640 1 -- 172.21.15.29:0/3534067063 learned_addr learned my addr 172.21.15.29:0/3534067063 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:17.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc42f7fe640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4300b3c30 0x7fc430158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc42ffff640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4300b3140 0x7fc43013c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc434e93640 1 -- 172.21.15.29:0/3534067063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4300b3c30 msgr2=0x7fc430158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc434e93640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4300b3c30 0x7fc430158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc434e93640 1 -- 172.21.15.29:0/3534067063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4300b3140 msgr2=0x7fc43013c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc434e93640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4300b3140 0x7fc43013c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc434e93640 1 -- 172.21.15.29:0/3534067063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4301529c0 con 0x7fc4300b4b10 2024-04-25T00:41:17.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.516+0000 7fc434e93640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4300b4b10 0x7fc430159f40 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fc42800c780 tx=0x7fc42800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:17.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.520+0000 7fc42cff9640 1 -- 172.21.15.29:0/3534067063 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc428004b70 con 0x7fc4300b4b10 2024-04-25T00:41:17.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.520+0000 7fc42cff9640 1 -- 172.21.15.29:0/3534067063 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc428004d10 con 0x7fc4300b4b10 2024-04-25T00:41:17.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.520+0000 7fc4368f6640 1 -- 172.21.15.29:0/3534067063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc430152f40 con 0x7fc4300b4b10 2024-04-25T00:41:17.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.520+0000 7fc42cff9640 1 -- 172.21.15.29:0/3534067063 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42800aa90 con 0x7fc4300b4b10 2024-04-25T00:41:17.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.520+0000 7fc4368f6640 1 -- 172.21.15.29:0/3534067063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc430152c60 con 0x7fc4300b4b10 2024-04-25T00:41:17.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.520+0000 7fc4368f6640 1 -- 172.21.15.29:0/3534067063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc430069270 con 0x7fc4300b4b10 2024-04-25T00:41:17.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.520+0000 7fc42cff9640 1 -- 172.21.15.29:0/3534067063 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fc42800ac30 con 0x7fc4300b4b10 2024-04-25T00:41:17.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.520+0000 7fc42cff9640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc41c068250 0x7fc41c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.520+0000 7fc42cff9640 1 -- 172.21.15.29:0/3534067063 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4857+0+0 (secure 0 0 0) 0x7fc428084980 con 0x7fc4300b4b10 2024-04-25T00:41:17.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.520+0000 7fc42ffff640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc41c068250 0x7fc41c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.520+0000 7fc42ffff640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc41c068250 0x7fc41c06a710 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc420000f70 tx=0x7fc420007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:17.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.524+0000 7fc42cff9640 1 -- 172.21.15.29:0/3534067063 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc4280570c0 con 0x7fc4300b4b10 2024-04-25T00:41:17.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc4368f6640 1 -- 172.21.15.29:0/3534067063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc430067560 con 0x7fc4300b4b10 2024-04-25T00:41:17.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc42cff9640 1 -- 172.21.15.29:0/3534067063 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v164) ==== 93+0+10 (secure 0 0 0) 0x7fc4280569c0 con 0x7fc4300b4b10 2024-04-25T00:41:17.688 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:41:17.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc4368f6640 1 -- 172.21.15.29:0/3534067063 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc41c068250 msgr2=0x7fc41c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc4368f6640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc41c068250 0x7fc41c06a710 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc420000f70 tx=0x7fc420007c10 comp rx=0 tx=0).stop 2024-04-25T00:41:17.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc4368f6640 1 -- 172.21.15.29:0/3534067063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4300b4b10 msgr2=0x7fc430159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc4368f6640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4300b4b10 0x7fc430159f40 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fc42800c780 tx=0x7fc42800cc40 comp rx=0 tx=0).stop 2024-04-25T00:41:17.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc4368f6640 1 -- 172.21.15.29:0/3534067063 shutdown_connections 2024-04-25T00:41:17.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc4368f6640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc41c068250 0x7fc41c06a710 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc4368f6640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4300b4b10 0x7fc430159f40 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc4368f6640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4300b3c30 0x7fc430158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc4368f6640 1 --2- 172.21.15.29:0/3534067063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4300b3140 0x7fc43013c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc4368f6640 1 -- 172.21.15.29:0/3534067063 >> 172.21.15.29:0/3534067063 conn(0x7fc4300b1870 msgr2=0x7fc4301435f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:17.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc4368f6640 1 -- 172.21.15.29:0/3534067063 shutdown_connections 2024-04-25T00:41:17.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.688+0000 7fc4368f6640 1 -- 172.21.15.29:0/3534067063 wait complete. 2024-04-25T00:41:17.707 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:41:17.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23d5698640 1 Processor -- start 2024-04-25T00:41:17.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23d5698640 1 -- start start 2024-04-25T00:41:17.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23d5698640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f23d015e200 0x7f23d00d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23d5698640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f23d00cf690 0x7f23d00d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23d5698640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d00d2ea0 con 0x7f23d00d2a90 2024-04-25T00:41:17.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23d5698640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f23d00b38d0 con 0x7f23d015e200 2024-04-25T00:41:17.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23d5698640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f23d00cfc10 con 0x7f23d00cf690 2024-04-25T00:41:17.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23ceffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f23d015e200 0x7f23d00d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23ceffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f23d015e200 0x7f23d00d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46352/0 (socket says 172.21.15.29:46352) 2024-04-25T00:41:17.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23ceffd640 1 -- 172.21.15.29:0/3980369233 learned_addr learned my addr 172.21.15.29:0/3980369233 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:17.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23ce7fc640 1 --2- 172.21.15.29:0/3980369233 >> v2:172.21.15.157:3300/0 conn(0x7f23d00cf690 0x7f23d00d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23ceffd640 1 -- 172.21.15.29:0/3980369233 >> v2:172.21.15.157:3300/0 conn(0x7f23d00cf690 msgr2=0x7f23d00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:17.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23ceffd640 1 --2- 172.21.15.29:0/3980369233 >> v2:172.21.15.157:3300/0 conn(0x7f23d00cf690 0x7f23d00d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23ceffd640 1 -- 172.21.15.29:0/3980369233 >> v1:172.21.15.157:6789/0 conn(0x7f23d00d2a90 legacy=0x7f23d00d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:17.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.832+0000 7f23ceffd640 1 -- 172.21.15.29:0/3980369233 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d00cfef0 con 0x7f23d015e200 2024-04-25T00:41:17.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.836+0000 7f23ceffd640 1 --2- 172.21.15.29:0/3980369233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23d015e200 0x7f23d00d47f0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f23d00d24a0 tx=0x7f23c001c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f1d963b827c09e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:17.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.836+0000 7f23cdffb640 1 -- 172.21.15.29:0/3980369233 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c00b0070 con 0x7f23d015e200 2024-04-25T00:41:17.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.836+0000 7f23cdffb640 1 -- 172.21.15.29:0/3980369233 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c00ab600 con 0x7f23d015e200 2024-04-25T00:41:17.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.836+0000 7f23cdffb640 1 -- 172.21.15.29:0/3980369233 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c00ab9d0 con 0x7f23d015e200 2024-04-25T00:41:17.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.836+0000 7f23d5698640 1 -- 172.21.15.29:0/3980369233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23d015e200 msgr2=0x7f23d00d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.836+0000 7f23d5698640 1 --2- 172.21.15.29:0/3980369233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23d015e200 0x7f23d00d47f0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f23d00d24a0 tx=0x7f23c001c2d0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.836+0000 7f23d5698640 1 -- 172.21.15.29:0/3980369233 shutdown_connections 2024-04-25T00:41:17.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.836+0000 7f23d5698640 1 --2- 172.21.15.29:0/3980369233 >> v2:172.21.15.157:3300/0 conn(0x7f23d00cf690 0x7f23d00d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.836+0000 7f23d5698640 1 --2- 172.21.15.29:0/3980369233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23d015e200 0x7f23d00d47f0 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.836+0000 7f23d5698640 1 -- 172.21.15.29:0/3980369233 >> 172.21.15.29:0/3980369233 conn(0x7f23d009d280 msgr2=0x7f23d00d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:17.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.836+0000 7f23d5698640 1 -- 172.21.15.29:0/3980369233 shutdown_connections 2024-04-25T00:41:17.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.836+0000 7f23d5698640 1 -- 172.21.15.29:0/3980369233 wait complete. 2024-04-25T00:41:17.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.836+0000 7f23d5698640 1 Processor -- start 2024-04-25T00:41:17.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23d5698640 1 -- start start 2024-04-25T00:41:17.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23d5698640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23d00cf690 0x7f23d00c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23d5698640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23d00d2a90 0x7f23d014fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23d5698640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23d015e200 0x7f23d0154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23d5698640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f23d015e760 con 0x7f23d015e200 2024-04-25T00:41:17.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23d5698640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f23d015e5e0 con 0x7f23d00cf690 2024-04-25T00:41:17.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23d5698640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f23d016ae30 con 0x7f23d00d2a90 2024-04-25T00:41:17.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23cf7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23d015e200 0x7f23d0154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23cf7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23d015e200 0x7f23d0154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46364/0 (socket says 172.21.15.29:46364) 2024-04-25T00:41:17.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23cf7fe640 1 -- 172.21.15.29:0/3053898820 learned_addr learned my addr 172.21.15.29:0/3053898820 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23ceffd640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23d00cf690 0x7f23d00c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23ce7fc640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23d00d2a90 0x7f23d014fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23cf7fe640 1 -- 172.21.15.29:0/3053898820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23d00d2a90 msgr2=0x7f23d014fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23cf7fe640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23d00d2a90 0x7f23d014fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23cf7fe640 1 -- 172.21.15.29:0/3053898820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23d00cf690 msgr2=0x7f23d00c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23cf7fe640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23d00cf690 0x7f23d00c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23cf7fe640 1 -- 172.21.15.29:0/3053898820 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d00c9a80 con 0x7f23d015e200 2024-04-25T00:41:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23cf7fe640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23d015e200 0x7f23d0154500 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f23c800b9a0 tx=0x7f23c800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23b77fe640 1 -- 172.21.15.29:0/3053898820 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8019070 con 0x7f23d015e200 2024-04-25T00:41:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23b77fe640 1 -- 172.21.15.29:0/3053898820 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c800cba0 con 0x7f23d015e200 2024-04-25T00:41:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23b77fe640 1 -- 172.21.15.29:0/3053898820 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8014440 con 0x7f23d015e200 2024-04-25T00:41:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23d5698640 1 -- 172.21.15.29:0/3053898820 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d00c9d60 con 0x7f23d015e200 2024-04-25T00:41:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23d5698640 1 -- 172.21.15.29:0/3053898820 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d00ca320 con 0x7f23d015e200 2024-04-25T00:41:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23d5698640 1 -- 172.21.15.29:0/3053898820 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d0069270 con 0x7f23d015e200 2024-04-25T00:41:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.840+0000 7f23b77fe640 1 -- 172.21.15.29:0/3053898820 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f23c801c070 con 0x7f23d015e200 2024-04-25T00:41:17.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.844+0000 7f23b77fe640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f23bc068280 0x7f23bc06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:17.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.844+0000 7f23b77fe640 1 -- 172.21.15.29:0/3053898820 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4857+0+0 (secure 0 0 0) 0x7f23c8084000 con 0x7f23d015e200 2024-04-25T00:41:17.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.844+0000 7f23ceffd640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f23bc068280 0x7f23bc06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:17.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.844+0000 7f23ceffd640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f23bc068280 0x7f23bc06a740 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f23c00ad000 tx=0x7f23c00a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:17.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:17.848+0000 7f23b77fe640 1 -- 172.21.15.29:0/3053898820 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f23c8056740 con 0x7f23d015e200 2024-04-25T00:41:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.008+0000 7f23d5698640 1 -- 172.21.15.29:0/3053898820 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f23d00c7230 con 0x7f23d015e200 2024-04-25T00:41:18.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.008+0000 7f23b77fe640 1 -- 172.21.15.29:0/3053898820 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v164) ==== 84+0+65 (secure 0 0 0) 0x7f23c8056040 con 0x7f23d015e200 2024-04-25T00:41:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:18.010 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:41:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-04-25T00:41:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": ".mgr" 2024-04-25T00:41:18.010 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:18.010 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:41:18.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.012+0000 7f23d5698640 1 -- 172.21.15.29:0/3053898820 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f23bc068280 msgr2=0x7f23bc06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:18.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.012+0000 7f23d5698640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f23bc068280 0x7f23bc06a740 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f23c00ad000 tx=0x7f23c00a6000 comp rx=0 tx=0).stop 2024-04-25T00:41:18.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.012+0000 7f23d5698640 1 -- 172.21.15.29:0/3053898820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23d015e200 msgr2=0x7f23d0154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:18.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.012+0000 7f23d5698640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23d015e200 0x7f23d0154500 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f23c800b9a0 tx=0x7f23c800be60 comp rx=0 tx=0).stop 2024-04-25T00:41:18.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.012+0000 7f23d5698640 1 -- 172.21.15.29:0/3053898820 shutdown_connections 2024-04-25T00:41:18.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.012+0000 7f23d5698640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f23bc068280 0x7f23bc06a740 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.012+0000 7f23d5698640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23d015e200 0x7f23d0154500 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.012+0000 7f23d5698640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23d00d2a90 0x7f23d014fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.012+0000 7f23d5698640 1 --2- 172.21.15.29:0/3053898820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23d00cf690 0x7f23d00c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.012+0000 7f23d5698640 1 -- 172.21.15.29:0/3053898820 >> 172.21.15.29:0/3053898820 conn(0x7f23d009d280 msgr2=0x7f23d00b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.012+0000 7f23d5698640 1 -- 172.21.15.29:0/3053898820 shutdown_connections 2024-04-25T00:41:18.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.012+0000 7f23d5698640 1 -- 172.21.15.29:0/3053898820 wait complete. 2024-04-25T00:41:18.029 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:41:18.029 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:41:18.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.152+0000 7fbf5faf2640 1 Processor -- start 2024-04-25T00:41:18.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.152+0000 7fbf5faf2640 1 -- start start 2024-04-25T00:41:18.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.152+0000 7fbf5faf2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf580b3b50 0x7fbf580b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:18.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.152+0000 7fbf5faf2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf580cf670 con 0x7fbf581456d0 2024-04-25T00:41:18.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.152+0000 7fbf5faf2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf580b4380 con 0x7fbf5806cd40 2024-04-25T00:41:18.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.152+0000 7fbf5faf2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbf580b40a0 con 0x7fbf580b3b50 2024-04-25T00:41:18.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.152+0000 7fbf5d08d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf580b3b50 0x7fbf580b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:18.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.152+0000 7fbf5d88e640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbf581456d0 0x7fbf580cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:51548/0 (socket says 172.21.15.29:51548) 2024-04-25T00:41:18.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.152+0000 7fbf5d88e640 1 -- 172.21.15.29:0/740810635 learned_addr learned my addr 172.21.15.29:0/740810635 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:18.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.152+0000 7fbf5d08d640 1 -- 172.21.15.29:0/740810635 >> v1:172.21.15.157:6789/0 conn(0x7fbf5806cd40 legacy=0x7fbf5806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:18.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.152+0000 7fbf5d08d640 1 -- 172.21.15.29:0/740810635 >> v1:172.21.15.29:6789/0 conn(0x7fbf581456d0 legacy=0x7fbf580cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:18.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.152+0000 7fbf5d08d640 1 -- 172.21.15.29:0/740810635 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf580ce480 con 0x7fbf580b3b50 2024-04-25T00:41:18.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5d08d640 1 --2- 172.21.15.29:0/740810635 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf580b3b50 0x7fbf580b2b70 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fbf480097c0 tx=0x7fbf4801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c4a714005af598 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:18.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5c88c640 1 -- 172.21.15.29:0/740810635 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf480b0070 con 0x7fbf580b3b50 2024-04-25T00:41:18.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5c88c640 1 -- 172.21.15.29:0/740810635 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf48004230 con 0x7fbf580b3b50 2024-04-25T00:41:18.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5c88c640 1 -- 172.21.15.29:0/740810635 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48003620 con 0x7fbf580b3b50 2024-04-25T00:41:18.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 -- 172.21.15.29:0/740810635 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf580b3b50 msgr2=0x7fbf580b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:18.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 --2- 172.21.15.29:0/740810635 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf580b3b50 0x7fbf580b2b70 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fbf480097c0 tx=0x7fbf4801c460 comp rx=0 tx=0).stop 2024-04-25T00:41:18.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 -- 172.21.15.29:0/740810635 shutdown_connections 2024-04-25T00:41:18.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 --2- 172.21.15.29:0/740810635 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf580b3b50 0x7fbf580b2b70 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 -- 172.21.15.29:0/740810635 >> 172.21.15.29:0/740810635 conn(0x7fbf580b1870 msgr2=0x7fbf580b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:18.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 -- 172.21.15.29:0/740810635 shutdown_connections 2024-04-25T00:41:18.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 -- 172.21.15.29:0/740810635 wait complete. 2024-04-25T00:41:18.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 Processor -- start 2024-04-25T00:41:18.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 -- start start 2024-04-25T00:41:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5806cd40 0x7fbf58140b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf580b3b50 0x7fbf58154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf581456d0 0x7fbf58155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58169b80 con 0x7fbf580b3b50 2024-04-25T00:41:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf580b5000 con 0x7fbf581456d0 2024-04-25T00:41:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5faf2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbf58169d00 con 0x7fbf5806cd40 2024-04-25T00:41:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5d08d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf580b3b50 0x7fbf58154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5d08d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf580b3b50 0x7fbf58154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46384/0 (socket says 172.21.15.29:46384) 2024-04-25T00:41:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5d08d640 1 -- 172.21.15.29:0/2647898059 learned_addr learned my addr 172.21.15.29:0/2647898059 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.156+0000 7fbf5e08f640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf581456d0 0x7fbf58155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf5d88e640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5806cd40 0x7fbf58140b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:18.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf5d08d640 1 -- 172.21.15.29:0/2647898059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5806cd40 msgr2=0x7fbf58140b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf5d08d640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5806cd40 0x7fbf58140b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf5d08d640 1 -- 172.21.15.29:0/2647898059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf581456d0 msgr2=0x7fbf58155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf5d08d640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf581456d0 0x7fbf58155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf5d08d640 1 -- 172.21.15.29:0/2647898059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5813fd80 con 0x7fbf580b3b50 2024-04-25T00:41:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf5d08d640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf580b3b50 0x7fbf58154490 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fbf480ae040 tx=0x7fbf48019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf4e7fc640 1 -- 172.21.15.29:0/2647898059 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf480b0040 con 0x7fbf580b3b50 2024-04-25T00:41:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf4e7fc640 1 -- 172.21.15.29:0/2647898059 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf480bb070 con 0x7fbf580b3b50 2024-04-25T00:41:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf5faf2640 1 -- 172.21.15.29:0/2647898059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf58140060 con 0x7fbf580b3b50 2024-04-25T00:41:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf4e7fc640 1 -- 172.21.15.29:0/2647898059 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf480b5480 con 0x7fbf580b3b50 2024-04-25T00:41:18.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf5faf2640 1 -- 172.21.15.29:0/2647898059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf58140620 con 0x7fbf580b3b50 2024-04-25T00:41:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf5faf2640 1 -- 172.21.15.29:0/2647898059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf580692e0 con 0x7fbf580b3b50 2024-04-25T00:41:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf4e7fc640 1 -- 172.21.15.29:0/2647898059 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fbf480b3070 con 0x7fbf580b3b50 2024-04-25T00:41:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf4e7fc640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbf3c068250 0x7fbf3c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf4e7fc640 1 -- 172.21.15.29:0/2647898059 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4857+0+0 (secure 0 0 0) 0x7fbf4811e370 con 0x7fbf580b3b50 2024-04-25T00:41:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf5d88e640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbf3c068250 0x7fbf3c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:18.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.160+0000 7fbf5d88e640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbf3c068250 0x7fbf3c06a710 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbf54000f70 tx=0x7fbf54009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:18.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.164+0000 7fbf4e7fc640 1 -- 172.21.15.29:0/2647898059 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbf48016050 con 0x7fbf580b3b50 2024-04-25T00:41:18.218 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T00:41:18.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.324+0000 7fbf5faf2640 1 -- 172.21.15.29:0/2647898059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fbf581407f0 con 0x7fbf580b3b50 2024-04-25T00:41:18.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.676+0000 7fbf4e7fc640 1 -- 172.21.15.29:0/2647898059 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v165) ==== 138+0+0 (secure 0 0 0) 0x7fbf480f0a20 con 0x7fbf580b3b50 2024-04-25T00:41:18.676 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-04-25T00:41:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.676+0000 7fbf5faf2640 1 -- 172.21.15.29:0/2647898059 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbf3c068250 msgr2=0x7fbf3c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.676+0000 7fbf5faf2640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbf3c068250 0x7fbf3c06a710 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbf54000f70 tx=0x7fbf54009040 comp rx=0 tx=0).stop 2024-04-25T00:41:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.676+0000 7fbf5faf2640 1 -- 172.21.15.29:0/2647898059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf580b3b50 msgr2=0x7fbf58154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.676+0000 7fbf5faf2640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf580b3b50 0x7fbf58154490 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fbf480ae040 tx=0x7fbf48019200 comp rx=0 tx=0).stop 2024-04-25T00:41:18.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.676+0000 7fbf5faf2640 1 -- 172.21.15.29:0/2647898059 shutdown_connections 2024-04-25T00:41:18.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.680+0000 7fbf5faf2640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fbf3c068250 0x7fbf3c06a710 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.680+0000 7fbf5faf2640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf581456d0 0x7fbf58155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.680+0000 7fbf5faf2640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf580b3b50 0x7fbf58154490 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.680+0000 7fbf5faf2640 1 --2- 172.21.15.29:0/2647898059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf5806cd40 0x7fbf58140b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.680+0000 7fbf5faf2640 1 -- 172.21.15.29:0/2647898059 >> 172.21.15.29:0/2647898059 conn(0x7fbf580b1870 msgr2=0x7fbf5806d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:18.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.680+0000 7fbf5faf2640 1 -- 172.21.15.29:0/2647898059 shutdown_connections 2024-04-25T00:41:18.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.680+0000 7fbf5faf2640 1 -- 172.21.15.29:0/2647898059 wait complete. 2024-04-25T00:41:18.708 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:41:18.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.844+0000 7f9b8be3f640 1 Processor -- start 2024-04-25T00:41:18.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.844+0000 7f9b8be3f640 1 -- start start 2024-04-25T00:41:18.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b8be3f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9b8415e2a0 0x7f9b840d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:18.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b8be3f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b840cf8a0 con 0x7f9b840b4040 2024-04-25T00:41:18.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b8be3f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b840cfb80 con 0x7f9b840d4d10 2024-04-25T00:41:18.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b8be3f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9b840d19a0 con 0x7f9b8415e2a0 2024-04-25T00:41:18.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b89bdb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9b8415e2a0 0x7f9b840d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b8a3dc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9b840b4040 0x7f9b840b44b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:51556/0 (socket says 172.21.15.29:51556) 2024-04-25T00:41:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b89bdb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9b8415e2a0 0x7f9b840d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46394/0 (socket says 172.21.15.29:46394) 2024-04-25T00:41:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b8a3dc640 1 -- 172.21.15.29:0/1642057294 learned_addr learned my addr 172.21.15.29:0/1642057294 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b89bdb640 1 -- 172.21.15.29:0/1642057294 >> v1:172.21.15.157:6789/0 conn(0x7f9b840d4d10 legacy=0x7f9b840d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b89bdb640 1 -- 172.21.15.29:0/1642057294 >> v1:172.21.15.29:6789/0 conn(0x7f9b840b4040 legacy=0x7f9b840b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b89bdb640 1 -- 172.21.15.29:0/1642057294 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b840cfe60 con 0x7f9b8415e2a0 2024-04-25T00:41:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b88bd9640 1 -- 172.21.15.29:0/1642057294 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937523110 0 0) 0x7f9b840cf8a0 con 0x7f9b840b4040 2024-04-25T00:41:18.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b89bdb640 1 --2- 172.21.15.29:0/1642057294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b8415e2a0 0x7f9b840d1c00 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f9b800097c0 tx=0x7f9b8001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90559b1a071f831c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:18.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b88bd9640 1 -- 172.21.15.29:0/1642057294 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800b0070 con 0x7f9b8415e2a0 2024-04-25T00:41:18.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b88bd9640 1 -- 172.21.15.29:0/1642057294 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b800ab600 con 0x7f9b8415e2a0 2024-04-25T00:41:18.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.848+0000 7f9b88bd9640 1 -- 172.21.15.29:0/1642057294 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800a4e60 con 0x7f9b8415e2a0 2024-04-25T00:41:18.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1642057294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b8415e2a0 msgr2=0x7f9b840d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 --2- 172.21.15.29:0/1642057294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b8415e2a0 0x7f9b840d1c00 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f9b800097c0 tx=0x7f9b8001c3d0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1642057294 shutdown_connections 2024-04-25T00:41:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 --2- 172.21.15.29:0/1642057294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b8415e2a0 0x7f9b840d1c00 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1642057294 >> 172.21.15.29:0/1642057294 conn(0x7f9b840d4030 msgr2=0x7f9b840d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1642057294 shutdown_connections 2024-04-25T00:41:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1642057294 wait complete. 2024-04-25T00:41:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 Processor -- start 2024-04-25T00:41:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 -- start start 2024-04-25T00:41:18.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b840b4040 0x7f9b8414e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:18.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b840d4d10 0x7f9b8414fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:18.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b8415e2a0 0x7f9b840b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:18.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9b840b3bb0 con 0x7f9b840d4d10 2024-04-25T00:41:18.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b840b38d0 con 0x7f9b8415e2a0 2024-04-25T00:41:18.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9b840d2a60 con 0x7f9b840b4040 2024-04-25T00:41:18.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b893da640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b840d4d10 0x7f9b8414fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:18.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b893da640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b840d4d10 0x7f9b8414fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46400/0 (socket says 172.21.15.29:46400) 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b893da640 1 -- 172.21.15.29:0/1462300729 learned_addr learned my addr 172.21.15.29:0/1462300729 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8a3dc640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b8415e2a0 0x7f9b840b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b89bdb640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b840b4040 0x7f9b8414e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b893da640 1 -- 172.21.15.29:0/1462300729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b840b4040 msgr2=0x7f9b8414e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b893da640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b840b4040 0x7f9b8414e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b893da640 1 -- 172.21.15.29:0/1462300729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b8415e2a0 msgr2=0x7f9b840b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b893da640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b8415e2a0 0x7f9b840b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b893da640 1 -- 172.21.15.29:0/1462300729 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b841503e0 con 0x7f9b840d4d10 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b893da640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b840d4d10 0x7f9b8414fea0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400b410 tx=0x7f9b7400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b7a7fc640 1 -- 172.21.15.29:0/1462300729 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74015030 con 0x7f9b840d4d10 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b7a7fc640 1 -- 172.21.15.29:0/1462300729 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b74004020 con 0x7f9b840d4d10 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b7a7fc640 1 -- 172.21.15.29:0/1462300729 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74004410 con 0x7f9b840d4d10 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1462300729 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b840b77f0 con 0x7f9b840d4d10 2024-04-25T00:41:18.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.852+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1462300729 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b840cbfc0 con 0x7f9b840d4d10 2024-04-25T00:41:18.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.856+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1462300729 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b84069270 con 0x7f9b840d4d10 2024-04-25T00:41:18.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.856+0000 7f9b7a7fc640 1 -- 172.21.15.29:0/1462300729 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f9b74018070 con 0x7f9b840d4d10 2024-04-25T00:41:18.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.856+0000 7f9b7a7fc640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9b68068250 0x7f9b6806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:18.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.856+0000 7f9b7a7fc640 1 -- 172.21.15.29:0/1462300729 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5227+0+0 (secure 0 0 0) 0x7f9b74083f40 con 0x7f9b840d4d10 2024-04-25T00:41:18.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.856+0000 7f9b89bdb640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9b68068250 0x7f9b6806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:18.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.856+0000 7f9b89bdb640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9b68068250 0x7f9b6806a710 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9b800ad000 tx=0x7f9b800a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:18.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:18.860+0000 7f9b7a7fc640 1 -- 172.21.15.29:0/1462300729 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9b74056680 con 0x7f9b840d4d10 2024-04-25T00:41:19.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.024+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1462300729 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9b840b57c0 con 0x7f9b840d4d10 2024-04-25T00:41:19.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.684+0000 7f9b7a7fc640 1 -- 172.21.15.29:0/1462300729 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v166) ==== 170+0+0 (secure 0 0 0) 0x7f9b74055f80 con 0x7f9b840d4d10 2024-04-25T00:41:19.688 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-04-25T00:41:19.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.692+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1462300729 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9b68068250 msgr2=0x7f9b6806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:19.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.692+0000 7f9b8be3f640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9b68068250 0x7f9b6806a710 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9b800ad000 tx=0x7f9b800a6000 comp rx=0 tx=0).stop 2024-04-25T00:41:19.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.692+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1462300729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b840d4d10 msgr2=0x7f9b8414fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:19.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.692+0000 7f9b8be3f640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b840d4d10 0x7f9b8414fea0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400b410 tx=0x7f9b7400b8d0 comp rx=0 tx=0).stop 2024-04-25T00:41:19.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.692+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1462300729 shutdown_connections 2024-04-25T00:41:19.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.692+0000 7f9b8be3f640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9b68068250 0x7f9b6806a710 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:19.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.692+0000 7f9b8be3f640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b8415e2a0 0x7f9b840b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:19.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.692+0000 7f9b8be3f640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b840d4d10 0x7f9b8414fea0 secure :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400b410 tx=0x7f9b7400b8d0 comp rx=0 tx=0).stop 2024-04-25T00:41:19.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.692+0000 7f9b8be3f640 1 --2- 172.21.15.29:0/1462300729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b840b4040 0x7f9b8414e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:19.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.692+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1462300729 >> 172.21.15.29:0/1462300729 conn(0x7f9b840d4030 msgr2=0x7f9b840d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:19.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.692+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1462300729 shutdown_connections 2024-04-25T00:41:19.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.692+0000 7f9b8be3f640 1 -- 172.21.15.29:0/1462300729 wait complete. 2024-04-25T00:41:19.734 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:41:19.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fce2da640 1 Processor -- start 2024-04-25T00:41:19.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fce2da640 1 -- start start 2024-04-25T00:41:19.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fce2da640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9fc815bfb0 0x7f9fc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:19.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fce2da640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc80b5000 0x7f9fc80b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:19.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fce2da640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fc815f170 con 0x7f9fc81555b0 2024-04-25T00:41:19.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fce2da640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9fc80b5990 con 0x7f9fc815bfb0 2024-04-25T00:41:19.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fce2da640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc815f450 con 0x7f9fc80b5000 2024-04-25T00:41:19.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fc77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9fc815bfb0 0x7f9fc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:19.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fc7fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9fc81555b0 0x7f9fc815e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:51572/0 (socket says 172.21.15.29:51572) 2024-04-25T00:41:19.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fc7fff640 1 -- 172.21.15.29:0/304876902 learned_addr learned my addr 172.21.15.29:0/304876902 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:19.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fc77fe640 1 -- 172.21.15.29:0/304876902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc80b5000 msgr2=0x7f9fc80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:19.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fc77fe640 1 --2- 172.21.15.29:0/304876902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc80b5000 0x7f9fc80b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:19.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fc77fe640 1 -- 172.21.15.29:0/304876902 >> v1:172.21.15.29:6789/0 conn(0x7f9fc81555b0 legacy=0x7f9fc815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:19.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.860+0000 7f9fc77fe640 1 -- 172.21.15.29:0/304876902 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc815f730 con 0x7f9fc815bfb0 2024-04-25T00:41:19.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fc77fe640 1 --2- 172.21.15.29:0/304876902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc815bfb0 0x7f9fc806d330 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00097c0 tx=0x7f9fb001c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78872d1a03291b64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:19.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fc6ffd640 1 -- 172.21.15.29:0/304876902 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00b0070 con 0x7f9fc815bfb0 2024-04-25T00:41:19.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fc6ffd640 1 -- 172.21.15.29:0/304876902 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb00ab600 con 0x7f9fc815bfb0 2024-04-25T00:41:19.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fc6ffd640 1 -- 172.21.15.29:0/304876902 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00ab9d0 con 0x7f9fc815bfb0 2024-04-25T00:41:19.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 -- 172.21.15.29:0/304876902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc815bfb0 msgr2=0x7f9fc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:19.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 --2- 172.21.15.29:0/304876902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc815bfb0 0x7f9fc806d330 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00097c0 tx=0x7f9fb001c300 comp rx=0 tx=0).stop 2024-04-25T00:41:19.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 -- 172.21.15.29:0/304876902 shutdown_connections 2024-04-25T00:41:19.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 --2- 172.21.15.29:0/304876902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc80b5000 0x7f9fc80b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:19.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 --2- 172.21.15.29:0/304876902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc815bfb0 0x7f9fc806d330 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:19.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 -- 172.21.15.29:0/304876902 >> 172.21.15.29:0/304876902 conn(0x7f9fc8075030 msgr2=0x7f9fc80b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:19.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 -- 172.21.15.29:0/304876902 shutdown_connections 2024-04-25T00:41:19.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 -- 172.21.15.29:0/304876902 wait complete. 2024-04-25T00:41:19.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 Processor -- start 2024-04-25T00:41:19.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 -- start start 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc80b5000 0x7f9fc8143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc81555b0 0x7f9fc815c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc815bfb0 0x7f9fc815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc80cfdb0 con 0x7f9fc80b5000 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc80cfc30 con 0x7f9fc815bfb0 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fce2da640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc8167cc0 con 0x7f9fc81555b0 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fc7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc80b5000 0x7f9fc8143800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fc7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc80b5000 0x7f9fc8143800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46418/0 (socket says 172.21.15.29:46418) 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fc7fff640 1 -- 172.21.15.29:0/449855628 learned_addr learned my addr 172.21.15.29:0/449855628 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fc7fff640 1 -- 172.21.15.29:0/449855628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc81555b0 msgr2=0x7f9fc815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fc7fff640 1 --2- 172.21.15.29:0/449855628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc81555b0 0x7f9fc815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fcc877640 1 --2- 172.21.15.29:0/449855628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc815bfb0 0x7f9fc815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.864+0000 7f9fc7fff640 1 -- 172.21.15.29:0/449855628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc815bfb0 msgr2=0x7f9fc815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:19.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fc7fff640 1 --2- 172.21.15.29:0/449855628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc815bfb0 0x7f9fc815cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:19.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fc7fff640 1 -- 172.21.15.29:0/449855628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc80cdb30 con 0x7f9fc80b5000 2024-04-25T00:41:19.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fc7fff640 1 --2- 172.21.15.29:0/449855628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc80b5000 0x7f9fc8143800 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000c780 tx=0x7f9fc000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:19.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fc4ff9640 1 -- 172.21.15.29:0/449855628 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0017070 con 0x7f9fc80b5000 2024-04-25T00:41:19.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fc4ff9640 1 -- 172.21.15.29:0/449855628 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc0005a80 con 0x7f9fc80b5000 2024-04-25T00:41:19.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fce2da640 1 -- 172.21.15.29:0/449855628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc81593e0 con 0x7f9fc80b5000 2024-04-25T00:41:19.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fc4ff9640 1 -- 172.21.15.29:0/449855628 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0007070 con 0x7f9fc80b5000 2024-04-25T00:41:19.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fce2da640 1 -- 172.21.15.29:0/449855628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc81599a0 con 0x7f9fc80b5000 2024-04-25T00:41:19.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fce2da640 1 -- 172.21.15.29:0/449855628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc8069270 con 0x7f9fc80b5000 2024-04-25T00:41:19.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fc4ff9640 1 -- 172.21.15.29:0/449855628 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f9fc001a070 con 0x7f9fc80b5000 2024-04-25T00:41:19.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fc4ff9640 1 --2- 172.21.15.29:0/449855628 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9fa8068280 0x7f9fa806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:19.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fc4ff9640 1 -- 172.21.15.29:0/449855628 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5609+0+0 (secure 0 0 0) 0x7f9fc00838f0 con 0x7f9fc80b5000 2024-04-25T00:41:19.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fc77fe640 1 --2- 172.21.15.29:0/449855628 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9fa8068280 0x7f9fa806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:19.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.868+0000 7f9fc77fe640 1 --2- 172.21.15.29:0/449855628 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9fa8068280 0x7f9fa806a740 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00ad000 tx=0x7f9fb0004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:19.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:19.872+0000 7f9fc4ff9640 1 -- 172.21.15.29:0/449855628 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9fc0056060 con 0x7f9fc80b5000 2024-04-25T00:41:20.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.076+0000 7f9fce2da640 1 -- 172.21.15.29:0/449855628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9fc8068d70 con 0x7f9fc80b5000 2024-04-25T00:41:20.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.712+0000 7f9fc4ff9640 1 -- 172.21.15.29:0/449855628 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v108) ==== 174+0+0 (secure 0 0 0) 0x7f9fc0055960 con 0x7f9fc80b5000 2024-04-25T00:41:20.717 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 26 and data pool 27 2024-04-25T00:41:20.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.736+0000 7f9fce2da640 1 -- 172.21.15.29:0/449855628 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9fa8068280 msgr2=0x7f9fa806a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:20.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.736+0000 7f9fce2da640 1 --2- 172.21.15.29:0/449855628 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9fa8068280 0x7f9fa806a740 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00ad000 tx=0x7f9fb0004740 comp rx=0 tx=0).stop 2024-04-25T00:41:20.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.736+0000 7f9fce2da640 1 -- 172.21.15.29:0/449855628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc80b5000 msgr2=0x7f9fc8143800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:20.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.736+0000 7f9fce2da640 1 --2- 172.21.15.29:0/449855628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc80b5000 0x7f9fc8143800 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000c780 tx=0x7f9fc000cc40 comp rx=0 tx=0).stop 2024-04-25T00:41:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.736+0000 7f9fce2da640 1 -- 172.21.15.29:0/449855628 shutdown_connections 2024-04-25T00:41:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.736+0000 7f9fce2da640 1 --2- 172.21.15.29:0/449855628 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9fa8068280 0x7f9fa806a740 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.736+0000 7f9fce2da640 1 --2- 172.21.15.29:0/449855628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc815bfb0 0x7f9fc815cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.736+0000 7f9fce2da640 1 --2- 172.21.15.29:0/449855628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc81555b0 0x7f9fc815c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.736+0000 7f9fce2da640 1 --2- 172.21.15.29:0/449855628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fc80b5000 0x7f9fc8143800 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.736+0000 7f9fce2da640 1 -- 172.21.15.29:0/449855628 >> 172.21.15.29:0/449855628 conn(0x7f9fc8075030 msgr2=0x7f9fc806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.736+0000 7f9fce2da640 1 -- 172.21.15.29:0/449855628 shutdown_connections 2024-04-25T00:41:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.736+0000 7f9fce2da640 1 -- 172.21.15.29:0/449855628 wait complete. 2024-04-25T00:41:20.760 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:20.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb8a935640 1 Processor -- start 2024-04-25T00:41:20.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb8a935640 1 -- start start 2024-04-25T00:41:20.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb8a935640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffb8415ecf0 0x7ffb840ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:20.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb8a935640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb8415f2a0 con 0x7ffb84072990 2024-04-25T00:41:20.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb8a935640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb84071eb0 con 0x7ffb8406b210 2024-04-25T00:41:20.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb8a935640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ffb840b03f0 con 0x7ffb8415ecf0 2024-04-25T00:41:20.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb7bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ffb84072990 0x7ffb8415e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:51574/0 (socket says 172.21.15.29:51574) 2024-04-25T00:41:20.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb7bfff640 1 -- 172.21.15.29:0/3153808246 learned_addr learned my addr 172.21.15.29:0/3153808246 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:20.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb88ed2640 1 --2- 172.21.15.29:0/3153808246 >> v2:172.21.15.29:3300/0 conn(0x7ffb8415ecf0 0x7ffb840ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:20.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb837fe640 1 -- 172.21.15.29:0/3153808246 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3962567543 0 0) 0x7ffb8415f2a0 con 0x7ffb84072990 2024-04-25T00:41:20.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb837fe640 1 -- 172.21.15.29:0/3153808246 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb70003530 con 0x7ffb84072990 2024-04-25T00:41:20.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb88ed2640 1 -- 172.21.15.29:0/3153808246 >> v1:172.21.15.157:6789/0 conn(0x7ffb8406b210 legacy=0x7ffb8406b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:20.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb88ed2640 1 -- 172.21.15.29:0/3153808246 >> v1:172.21.15.29:6789/0 conn(0x7ffb84072990 legacy=0x7ffb8415e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:20.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.880+0000 7ffb88ed2640 1 -- 172.21.15.29:0/3153808246 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb84072190 con 0x7ffb8415ecf0 2024-04-25T00:41:20.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb88ed2640 1 --2- 172.21.15.29:0/3153808246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb8415ecf0 0x7ffb840ad8b0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009d90 tx=0x7ffb7c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33778bde8d2201e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:20.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb837fe640 1 -- 172.21.15.29:0/3153808246 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0b2070 con 0x7ffb8415ecf0 2024-04-25T00:41:20.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb837fe640 1 -- 172.21.15.29:0/3153808246 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c0a5710 con 0x7ffb8415ecf0 2024-04-25T00:41:20.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb837fe640 1 -- 172.21.15.29:0/3153808246 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0a5ae0 con 0x7ffb8415ecf0 2024-04-25T00:41:20.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb8a935640 1 -- 172.21.15.29:0/3153808246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb8415ecf0 msgr2=0x7ffb840ad8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:20.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb8a935640 1 --2- 172.21.15.29:0/3153808246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb8415ecf0 0x7ffb840ad8b0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009d90 tx=0x7ffb7c01c850 comp rx=0 tx=0).stop 2024-04-25T00:41:20.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb8a935640 1 -- 172.21.15.29:0/3153808246 shutdown_connections 2024-04-25T00:41:20.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb8a935640 1 --2- 172.21.15.29:0/3153808246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb8415ecf0 0x7ffb840ad8b0 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:20.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb8a935640 1 -- 172.21.15.29:0/3153808246 >> 172.21.15.29:0/3153808246 conn(0x7ffb840d0c40 msgr2=0x7ffb840d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:20.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb8a935640 1 -- 172.21.15.29:0/3153808246 shutdown_connections 2024-04-25T00:41:20.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb8a935640 1 -- 172.21.15.29:0/3153808246 wait complete. 2024-04-25T00:41:20.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb8a935640 1 Processor -- start 2024-04-25T00:41:20.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb8a935640 1 -- start start 2024-04-25T00:41:20.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb8a935640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb8406b210 0x7ffb8413b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:20.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.884+0000 7ffb8a935640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb84072990 0x7ffb8406f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:20.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb8a935640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb8415ecf0 0x7ffb84151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:20.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb8a935640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffb840cbf80 con 0x7ffb8415ecf0 2024-04-25T00:41:20.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb8a935640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb84072500 con 0x7ffb84072990 2024-04-25T00:41:20.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb8a935640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8416c9a0 con 0x7ffb8406b210 2024-04-25T00:41:20.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb88ed2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb8415ecf0 0x7ffb84151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:20.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb88ed2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb8415ecf0 0x7ffb84151c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46436/0 (socket says 172.21.15.29:46436) 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb88ed2640 1 -- 172.21.15.29:0/3421728831 learned_addr learned my addr 172.21.15.29:0/3421728831 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb83fff640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb8406b210 0x7ffb8413b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb88ed2640 1 -- 172.21.15.29:0/3421728831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb8406b210 msgr2=0x7ffb8413b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb88ed2640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb8406b210 0x7ffb8413b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb88ed2640 1 -- 172.21.15.29:0/3421728831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb84072990 msgr2=0x7ffb8406f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb88ed2640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb84072990 0x7ffb8406f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb88ed2640 1 -- 172.21.15.29:0/3421728831 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8413b700 con 0x7ffb8415ecf0 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb83fff640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb8406b210 0x7ffb8413b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb88ed2640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb8415ecf0 0x7ffb84151c70 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c01a480 tx=0x7ffb7c0195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb817fa640 1 -- 172.21.15.29:0/3421728831 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0b2050 con 0x7ffb8415ecf0 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb817fa640 1 -- 172.21.15.29:0/3421728831 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c0be070 con 0x7ffb8415ecf0 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb817fa640 1 -- 172.21.15.29:0/3421728831 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c009450 con 0x7ffb8415ecf0 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb8a935640 1 -- 172.21.15.29:0/3421728831 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb84153290 con 0x7ffb8415ecf0 2024-04-25T00:41:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb8a935640 1 -- 172.21.15.29:0/3421728831 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb841389c0 con 0x7ffb8415ecf0 2024-04-25T00:41:20.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb8a935640 1 -- 172.21.15.29:0/3421728831 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb841387d0 con 0x7ffb8415ecf0 2024-04-25T00:41:20.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb817fa640 1 -- 172.21.15.29:0/3421728831 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7ffb7c0adb80 con 0x7ffb8415ecf0 2024-04-25T00:41:20.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb817fa640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb64068250 0x7ffb6406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:20.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb817fa640 1 -- 172.21.15.29:0/3421728831 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7ffb7c11f570 con 0x7ffb8415ecf0 2024-04-25T00:41:20.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb83fff640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb64068250 0x7ffb6406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:20.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.888+0000 7ffb83fff640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb64068250 0x7ffb6406a710 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ffb841619c0 tx=0x7ffb74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:20.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:20.892+0000 7ffb817fa640 1 -- 172.21.15.29:0/3421728831 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffb7c016560 con 0x7ffb8415ecf0 2024-04-25T00:41:21.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.060+0000 7ffb8a935640 1 -- 172.21.15.29:0/3421728831 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb840d1a70 con 0x7ffb8415ecf0 2024-04-25T00:41:21.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.060+0000 7ffb817fa640 1 -- 172.21.15.29:0/3421728831 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16648 (secure 0 0 0) 0x7ffb7c0f1cb0 con 0x7ffb8415ecf0 2024-04-25T00:41:21.061 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:21.062 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":167,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:41:20.082374+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:41:18.370636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:41:19.067830+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:21.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.064+0000 7ffb8a935640 1 -- 172.21.15.29:0/3421728831 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb64068250 msgr2=0x7ffb6406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:21.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.064+0000 7ffb8a935640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb64068250 0x7ffb6406a710 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ffb841619c0 tx=0x7ffb74009040 comp rx=0 tx=0).stop 2024-04-25T00:41:21.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.064+0000 7ffb8a935640 1 -- 172.21.15.29:0/3421728831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb8415ecf0 msgr2=0x7ffb84151c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:21.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.064+0000 7ffb8a935640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb8415ecf0 0x7ffb84151c70 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c01a480 tx=0x7ffb7c0195f0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.064+0000 7ffb8a935640 1 -- 172.21.15.29:0/3421728831 shutdown_connections 2024-04-25T00:41:21.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.064+0000 7ffb8a935640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ffb64068250 0x7ffb6406a710 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.064+0000 7ffb8a935640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb8415ecf0 0x7ffb84151c70 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.064+0000 7ffb8a935640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb84072990 0x7ffb8406f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.064+0000 7ffb8a935640 1 --2- 172.21.15.29:0/3421728831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb8406b210 0x7ffb8413b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.064+0000 7ffb8a935640 1 -- 172.21.15.29:0/3421728831 >> 172.21.15.29:0/3421728831 conn(0x7ffb840d0c40 msgr2=0x7ffb840cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.064+0000 7ffb8a935640 1 -- 172.21.15.29:0/3421728831 shutdown_connections 2024-04-25T00:41:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.064+0000 7ffb8a935640 1 -- 172.21.15.29:0/3421728831 wait complete. 2024-04-25T00:41:21.082 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:21.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.200+0000 7f6bc0728640 1 Processor -- start 2024-04-25T00:41:21.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.200+0000 7f6bc0728640 1 -- start start 2024-04-25T00:41:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.200+0000 7f6bc0728640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6bb80b2b70 0x7f6bb8151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.200+0000 7f6bc0728640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bb80b3130 con 0x7f6bb815d980 2024-04-25T00:41:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.200+0000 7f6bc0728640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bb80b3410 con 0x7f6bb806dce0 2024-04-25T00:41:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.200+0000 7f6bc0728640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f6bb80cf560 con 0x7f6bb80b2b70 2024-04-25T00:41:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.200+0000 7f6bbe4c4640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6bb815d980 0x7f6bb806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:51576/0 (socket says 172.21.15.29:51576) 2024-04-25T00:41:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.200+0000 7f6bbe4c4640 1 -- 172.21.15.29:0/905515226 learned_addr learned my addr 172.21.15.29:0/905515226 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbecc5640 1 --2- 172.21.15.29:0/905515226 >> v2:172.21.15.157:3300/0 conn(0x7f6bb80b2b70 0x7f6bb8151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1819667864 0 0) 0x7f6bb80b3130 con 0x7f6bb815d980 2024-04-25T00:41:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ba4003530 con 0x7f6bb815d980 2024-04-25T00:41:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 138575522 0 0) 0x7f6ba4003530 con 0x7f6bb815d980 2024-04-25T00:41:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 >> v2:172.21.15.157:3300/0 conn(0x7f6bb80b2b70 msgr2=0x7f6bb8151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 --2- 172.21.15.29:0/905515226 >> v2:172.21.15.157:3300/0 conn(0x7f6bb80b2b70 0x7f6bb8151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 >> v1:172.21.15.157:6789/0 conn(0x7f6bb806dce0 legacy=0x7f6bb8150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb80b36f0 con 0x7f6bb815d980 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f6bb4001680 con 0x7f6bb815d980 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 >> v1:172.21.15.29:6789/0 conn(0x7f6bb815d980 legacy=0x7f6bb806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 --2- 172.21.15.29:0/905515226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ba40039c0 0x7f6ba4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 --2- 172.21.15.29:0/905515226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba4004810 0x7f6ba4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 --2- 172.21.15.29:0/905515226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba40051d0 0x7f6ba400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb80cf560 con 0x7f6ba4004810 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba400c070 con 0x7f6ba40039c0 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba400c270 con 0x7f6ba40051d0 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbdcc3640 1 --2- 172.21.15.29:0/905515226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba4004810 0x7f6ba4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbe4c4640 1 --2- 172.21.15.29:0/905515226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ba40039c0 0x7f6ba4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbecc5640 1 --2- 172.21.15.29:0/905515226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba40051d0 0x7f6ba400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbdcc3640 1 -- 172.21.15.29:0/905515226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba40051d0 msgr2=0x7f6ba400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:21.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbdcc3640 1 --2- 172.21.15.29:0/905515226 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba40051d0 0x7f6ba400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbdcc3640 1 -- 172.21.15.29:0/905515226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ba40039c0 msgr2=0x7f6ba4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:21.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbdcc3640 1 --2- 172.21.15.29:0/905515226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ba40039c0 0x7f6ba4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbdcc3640 1 -- 172.21.15.29:0/905515226 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba4003530 con 0x7f6ba4004810 2024-04-25T00:41:21.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbecc5640 1 -- 172.21.15.29:0/905515226 reap_dead start 2024-04-25T00:41:21.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbdcc3640 1 --2- 172.21.15.29:0/905515226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba4004810 0x7f6ba4004c90 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8001780 tx=0x7f6ba801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ab9a8491ae2b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:21.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80013a0 con 0x7f6ba4004810 2024-04-25T00:41:21.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba80052f0 con 0x7f6ba4004810 2024-04-25T00:41:21.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.204+0000 7f6bbd4c2640 1 -- 172.21.15.29:0/905515226 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8003840 con 0x7f6ba4004810 2024-04-25T00:41:21.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 -- 172.21.15.29:0/905515226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba4004810 msgr2=0x7f6ba4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:21.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 --2- 172.21.15.29:0/905515226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba4004810 0x7f6ba4004c90 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8001780 tx=0x7f6ba801c400 comp rx=0 tx=0).stop 2024-04-25T00:41:21.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 -- 172.21.15.29:0/905515226 shutdown_connections 2024-04-25T00:41:21.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 --2- 172.21.15.29:0/905515226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba4004810 0x7f6ba4004c90 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 -- 172.21.15.29:0/905515226 >> 172.21.15.29:0/905515226 conn(0x7f6bb80b1870 msgr2=0x7f6bb80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:21.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 -- 172.21.15.29:0/905515226 shutdown_connections 2024-04-25T00:41:21.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 -- 172.21.15.29:0/905515226 wait complete. 2024-04-25T00:41:21.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 Processor -- start 2024-04-25T00:41:21.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 -- start start 2024-04-25T00:41:21.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba4004810 0x7f6bb80cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb8158fb0 0x7f6bb80ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6bb81491c0 0x7f6bb8149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb815dd60 con 0x7f6bb81491c0 2024-04-25T00:41:21.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb80b3cb0 con 0x7f6bb8158fb0 2024-04-25T00:41:21.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb8150df0 con 0x7f6ba4004810 2024-04-25T00:41:21.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bbecc5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6bb81491c0 0x7f6bb8149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bbecc5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6bb81491c0 0x7f6bb8149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46452/0 (socket says 172.21.15.29:46452) 2024-04-25T00:41:21.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bbecc5640 1 -- 172.21.15.29:0/3828060953 learned_addr learned my addr 172.21.15.29:0/3828060953 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bbe4c4640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba4004810 0x7f6bb80cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bbdcc3640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb8158fb0 0x7f6bb80ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bbecc5640 1 -- 172.21.15.29:0/3828060953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba4004810 msgr2=0x7f6bb80cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bbecc5640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba4004810 0x7f6bb80cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bbecc5640 1 -- 172.21.15.29:0/3828060953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb8158fb0 msgr2=0x7f6bb80ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bbecc5640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb8158fb0 0x7f6bb80ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bbecc5640 1 -- 172.21.15.29:0/3828060953 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb8149b80 con 0x7f6bb81491c0 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bbdcc3640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb8158fb0 0x7f6bb80ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bbecc5640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6bb81491c0 0x7f6bb8149640 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8158570 tx=0x7f6bb000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6baeffd640 1 -- 172.21.15.29:0/3828060953 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb000fe80 con 0x7f6bb81491c0 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6baeffd640 1 -- 172.21.15.29:0/3828060953 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb00039a0 con 0x7f6bb81491c0 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 -- 172.21.15.29:0/3828060953 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb8159550 con 0x7f6bb81491c0 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6baeffd640 1 -- 172.21.15.29:0/3828060953 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0003d70 con 0x7f6bb81491c0 2024-04-25T00:41:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.208+0000 7f6bc0728640 1 -- 172.21.15.29:0/3828060953 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb8159b10 con 0x7f6bb81491c0 2024-04-25T00:41:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.212+0000 7f6bc0728640 1 -- 172.21.15.29:0/3828060953 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb8069270 con 0x7f6bb81491c0 2024-04-25T00:41:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.212+0000 7f6baeffd640 1 -- 172.21.15.29:0/3828060953 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f6bb001d020 con 0x7f6bb81491c0 2024-04-25T00:41:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.212+0000 7f6baeffd640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6b9c068280 0x7f6b9c06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.212+0000 7f6bbe4c4640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6b9c068280 0x7f6b9c06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.212+0000 7f6baeffd640 1 -- 172.21.15.29:0/3828060953 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f6bb0084c00 con 0x7f6bb81491c0 2024-04-25T00:41:21.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.212+0000 7f6bbe4c4640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6b9c068280 0x7f6b9c06a740 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4001000 tx=0x7f6bb4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:21.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.216+0000 7f6baeffd640 1 -- 172.21.15.29:0/3828060953 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6bb0024020 con 0x7f6bb81491c0 2024-04-25T00:41:21.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.380+0000 7f6bc0728640 1 -- 172.21.15.29:0/3828060953 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6bb80cf910 con 0x7f6bb81491c0 2024-04-25T00:41:21.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.380+0000 7f6baeffd640 1 -- 172.21.15.29:0/3828060953 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16648 (secure 0 0 0) 0x7f6bb00572b0 con 0x7f6bb81491c0 2024-04-25T00:41:21.382 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:21.382 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":167,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:41:20.082374+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:41:18.370636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:41:19.067830+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.384+0000 7f6bc0728640 1 -- 172.21.15.29:0/3828060953 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6b9c068280 msgr2=0x7f6b9c06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.384+0000 7f6bc0728640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6b9c068280 0x7f6b9c06a740 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4001000 tx=0x7f6bb4009040 comp rx=0 tx=0).stop 2024-04-25T00:41:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.384+0000 7f6bc0728640 1 -- 172.21.15.29:0/3828060953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6bb81491c0 msgr2=0x7f6bb8149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.384+0000 7f6bc0728640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6bb81491c0 0x7f6bb8149640 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8158570 tx=0x7f6bb000f520 comp rx=0 tx=0).stop 2024-04-25T00:41:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.384+0000 7f6bc0728640 1 -- 172.21.15.29:0/3828060953 shutdown_connections 2024-04-25T00:41:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.384+0000 7f6bc0728640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6b9c068280 0x7f6b9c06a740 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.384+0000 7f6bc0728640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6bb81491c0 0x7f6bb8149640 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.384+0000 7f6bc0728640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb8158fb0 0x7f6bb80ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.384+0000 7f6bc0728640 1 --2- 172.21.15.29:0/3828060953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ba4004810 0x7f6bb80cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.384+0000 7f6bc0728640 1 -- 172.21.15.29:0/3828060953 >> 172.21.15.29:0/3828060953 conn(0x7f6bb80b1870 msgr2=0x7f6bb815c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.384+0000 7f6bc0728640 1 -- 172.21.15.29:0/3828060953 shutdown_connections 2024-04-25T00:41:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.384+0000 7f6bc0728640 1 -- 172.21.15.29:0/3828060953 wait complete. 2024-04-25T00:41:21.402 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:41:21.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2f52f1640 1 Processor -- start 2024-04-25T00:41:21.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2f52f1640 1 -- start start 2024-04-25T00:41:21.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2f52f1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe2f00d1a10 0x7fe2f006c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2f52f1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe2f00b1ad0 0x7fe2f00b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2f52f1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f00d12a0 con 0x7fe2f00b12d0 2024-04-25T00:41:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2f52f1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f00b1730 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2f52f1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f006ced0 con 0x7fe2f00d1a10 2024-04-25T00:41:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2ef7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe2f00b1ad0 0x7fe2f00b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2ee7fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe2f00b12d0 0x7fe2f00d1df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:51580/0 (socket says 172.21.15.29:51580) 2024-04-25T00:41:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2ef7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe2f00b1ad0 0x7fe2f00b1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46460/0 (socket says 172.21.15.29:46460) 2024-04-25T00:41:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2ee7fc640 1 -- 172.21.15.29:0/3796950489 learned_addr learned my addr 172.21.15.29:0/3796950489 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2eeffd640 1 --2- 172.21.15.29:0/3796950489 >> v2:172.21.15.157:3300/0 conn(0x7fe2f00d1a10 0x7fe2f006c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2edffb640 1 -- 172.21.15.29:0/3796950489 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3903902325 0 0) 0x7fe2f00d12a0 con 0x7fe2f00b12d0 2024-04-25T00:41:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2edffb640 1 -- 172.21.15.29:0/3796950489 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2d8003530 con 0x7fe2f00b12d0 2024-04-25T00:41:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2ef7fe640 1 -- 172.21.15.29:0/3796950489 >> v2:172.21.15.157:3300/0 conn(0x7fe2f00d1a10 msgr2=0x7fe2f006c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:21.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2ef7fe640 1 --2- 172.21.15.29:0/3796950489 >> v2:172.21.15.157:3300/0 conn(0x7fe2f00d1a10 0x7fe2f006c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2ef7fe640 1 -- 172.21.15.29:0/3796950489 >> v1:172.21.15.29:6789/0 conn(0x7fe2f00b12d0 legacy=0x7fe2f00d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:21.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.532+0000 7fe2ef7fe640 1 -- 172.21.15.29:0/3796950489 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f006d1b0 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2ef7fe640 1 --2- 172.21.15.29:0/3796950489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2f00b1ad0 0x7fe2f00b1f20 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00cf860 tx=0x7fe2e801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870288b6d2b33e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2edffb640 1 -- 172.21.15.29:0/3796950489 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80b2070 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2edffb640 1 -- 172.21.15.29:0/3796950489 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e80a58c0 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2edffb640 1 -- 172.21.15.29:0/3796950489 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80a5c70 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 -- 172.21.15.29:0/3796950489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2f00b1ad0 msgr2=0x7fe2f00b1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:21.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 --2- 172.21.15.29:0/3796950489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2f00b1ad0 0x7fe2f00b1f20 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00cf860 tx=0x7fe2e801c900 comp rx=0 tx=0).stop 2024-04-25T00:41:21.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 -- 172.21.15.29:0/3796950489 shutdown_connections 2024-04-25T00:41:21.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 --2- 172.21.15.29:0/3796950489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2f00b1ad0 0x7fe2f00b1f20 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 --2- 172.21.15.29:0/3796950489 >> v2:172.21.15.157:3300/0 conn(0x7fe2f00d1a10 0x7fe2f006c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 -- 172.21.15.29:0/3796950489 >> 172.21.15.29:0/3796950489 conn(0x7fe2f009aa30 msgr2=0x7fe2f009ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:21.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 -- 172.21.15.29:0/3796950489 shutdown_connections 2024-04-25T00:41:21.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 -- 172.21.15.29:0/3796950489 wait complete. 2024-04-25T00:41:21.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 Processor -- start 2024-04-25T00:41:21.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 -- start start 2024-04-25T00:41:21.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2f00b12d0 0x7fe2f015e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2f00b1ad0 0x7fe2f014fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2f00d1a10 0x7fe2f0153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f006d610 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f006d490 con 0x7fe2f00d1a10 2024-04-25T00:41:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2f52f1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f016ae00 con 0x7fe2f00b12d0 2024-04-25T00:41:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2ee7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2f00b1ad0 0x7fe2f014fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2ee7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2f00b1ad0 0x7fe2f014fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46472/0 (socket says 172.21.15.29:46472) 2024-04-25T00:41:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2ee7fc640 1 -- 172.21.15.29:0/1142826590 learned_addr learned my addr 172.21.15.29:0/1142826590 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2ef7fe640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2f00d1a10 0x7fe2f0153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2eeffd640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2f00b12d0 0x7fe2f015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2ee7fc640 1 -- 172.21.15.29:0/1142826590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2f00b12d0 msgr2=0x7fe2f015e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.536+0000 7fe2ee7fc640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2f00b12d0 0x7fe2f015e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2ee7fc640 1 -- 172.21.15.29:0/1142826590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2f00d1a10 msgr2=0x7fe2f0153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2ee7fc640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2f00d1a10 0x7fe2f0153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2ee7fc640 1 -- 172.21.15.29:0/1142826590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f00b60f0 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2eeffd640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2f00b12d0 0x7fe2f015e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2ee7fc640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2f00b1ad0 0x7fe2f014fb90 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400c3f0 tx=0x7fe2e400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2d77fe640 1 -- 172.21.15.29:0/1142826590 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4017070 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2d77fe640 1 -- 172.21.15.29:0/1142826590 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e4005cf0 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2f52f1640 1 -- 172.21.15.29:0/1142826590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f01571f0 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2d77fe640 1 -- 172.21.15.29:0/1142826590 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e40072f0 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2f52f1640 1 -- 172.21.15.29:0/1142826590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f01577b0 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2f52f1640 1 -- 172.21.15.29:0/1142826590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f009b6e0 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2d77fe640 1 -- 172.21.15.29:0/1142826590 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fe2e401a070 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2d77fe640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe2dc068280 0x7fe2dc06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:21.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2d77fe640 1 -- 172.21.15.29:0/1142826590 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fe2e4083ca0 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2eeffd640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe2dc068280 0x7fe2dc06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:21.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.540+0000 7fe2eeffd640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe2dc068280 0x7fe2dc06a740 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe2f01fa4e0 tx=0x7fe2e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:21.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.544+0000 7fe2d77fe640 1 -- 172.21.15.29:0/1142826590 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe2e4021020 con 0x7fe2f00b1ad0 2024-04-25T00:41:21.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:21.748+0000 7fe2f52f1640 1 -- 172.21.15.29:0/1142826590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe2f01579b0 con 0x7fe2f00b1ad0 2024-04-25T00:41:22.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.716+0000 7fe2d77fe640 1 -- 172.21.15.29:0/1142826590 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v111) ==== 118+0+0 (secure 0 0 0) 0x7fe2f01579b0 con 0x7fe2f00b1ad0 2024-04-25T00:41:22.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.720+0000 7fe2f52f1640 1 -- 172.21.15.29:0/1142826590 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe2dc068280 msgr2=0x7fe2dc06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:22.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.720+0000 7fe2f52f1640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe2dc068280 0x7fe2dc06a740 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe2f01fa4e0 tx=0x7fe2e0008040 comp rx=0 tx=0).stop 2024-04-25T00:41:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.720+0000 7fe2f52f1640 1 -- 172.21.15.29:0/1142826590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2f00b1ad0 msgr2=0x7fe2f014fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.720+0000 7fe2f52f1640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2f00b1ad0 0x7fe2f014fb90 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400c3f0 tx=0x7fe2e400c8b0 comp rx=0 tx=0).stop 2024-04-25T00:41:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.724+0000 7fe2f52f1640 1 -- 172.21.15.29:0/1142826590 shutdown_connections 2024-04-25T00:41:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.724+0000 7fe2f52f1640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe2dc068280 0x7fe2dc06a740 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.724+0000 7fe2f52f1640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2f00d1a10 0x7fe2f0153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.724+0000 7fe2f52f1640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2f00b1ad0 0x7fe2f014fb90 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.724+0000 7fe2f52f1640 1 --2- 172.21.15.29:0/1142826590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2f00b12d0 0x7fe2f015e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.724+0000 7fe2f52f1640 1 -- 172.21.15.29:0/1142826590 >> 172.21.15.29:0/1142826590 conn(0x7fe2f009aa30 msgr2=0x7fe2f015d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.724+0000 7fe2f52f1640 1 -- 172.21.15.29:0/1142826590 shutdown_connections 2024-04-25T00:41:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.724+0000 7fe2f52f1640 1 -- 172.21.15.29:0/1142826590 wait complete. 2024-04-25T00:41:22.744 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:41:22.744 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:41:22.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fccf145b640 1 Processor -- start 2024-04-25T00:41:22.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fccf145b640 1 -- start start 2024-04-25T00:41:22.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fccf145b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccec06d440 0x7fccec0cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:22.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fccf145b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fccec0cd720 0x7fccec0d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:22.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fccf145b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccec161a00 con 0x7fccec161620 2024-04-25T00:41:22.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fccf145b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fccec0cdfb0 con 0x7fccec0cd720 2024-04-25T00:41:22.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fccf145b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fccec0cffc0 con 0x7fccec06d440 2024-04-25T00:41:22.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcceaffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fccec161620 0x7fccec06cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:51590/0 (socket says 172.21.15.29:51590) 2024-04-25T00:41:22.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcceaffd640 1 -- 172.21.15.29:0/3717276107 learned_addr learned my addr 172.21.15.29:0/3717276107 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:22.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fccea7fc640 1 --2- 172.21.15.29:0/3717276107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccec06d440 0x7fccec0cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:22.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcceb7fe640 1 --2- 172.21.15.29:0/3717276107 >> v2:172.21.15.157:3300/0 conn(0x7fccec0cd720 0x7fccec0d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:22.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1642026960 0 0) 0x7fccec161a00 con 0x7fccec161620 2024-04-25T00:41:22.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccd8003530 con 0x7fccec161620 2024-04-25T00:41:22.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 679881145 0 0) 0x7fccd8003530 con 0x7fccec161620 2024-04-25T00:41:22.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccec06d440 msgr2=0x7fccec0cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:22.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 --2- 172.21.15.29:0/3717276107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccec06d440 0x7fccec0cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:22.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 >> v2:172.21.15.157:3300/0 conn(0x7fccec0cd720 msgr2=0x7fccec0d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:22.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 --2- 172.21.15.29:0/3717276107 >> v2:172.21.15.157:3300/0 conn(0x7fccec0cd720 0x7fccec0d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:22.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec0d02a0 con 0x7fccec161620 2024-04-25T00:41:22.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fcce4001a00 con 0x7fccec161620 2024-04-25T00:41:22.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 >> v1:172.21.15.29:6789/0 conn(0x7fccec161620 legacy=0x7fccec06cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:22.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 --2- 172.21.15.29:0/3717276107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccec06d440 0x7fccd80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:22.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 --2- 172.21.15.29:0/3717276107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd8003f40 0x7fccd8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:22.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 --2- 172.21.15.29:0/3717276107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccd8004d50 0x7fccd80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:22.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fccec0cdfb0 con 0x7fccd8004d50 2024-04-25T00:41:22.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccec0cffc0 con 0x7fccd8003f40 2024-04-25T00:41:22.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fccd800b9d0 con 0x7fccec06d440 2024-04-25T00:41:22.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcceb7fe640 1 --2- 172.21.15.29:0/3717276107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccd8004d50 0x7fccd80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:22.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.876+0000 7fcceaffd640 1 --2- 172.21.15.29:0/3717276107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccec06d440 0x7fccd80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:22.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fccea7fc640 1 --2- 172.21.15.29:0/3717276107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd8003f40 0x7fccd8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:22.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fcceb7fe640 1 -- 172.21.15.29:0/3717276107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccec06d440 msgr2=0x7fccd80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:22.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fcceb7fe640 1 --2- 172.21.15.29:0/3717276107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccec06d440 0x7fccd80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:22.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fcceb7fe640 1 -- 172.21.15.29:0/3717276107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd8003f40 msgr2=0x7fccd8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:22.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fcceb7fe640 1 --2- 172.21.15.29:0/3717276107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd8003f40 0x7fccd8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:22.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fcceb7fe640 1 -- 172.21.15.29:0/3717276107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd8003530 con 0x7fccd8004d50 2024-04-25T00:41:22.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fccea7fc640 1 --2- 172.21.15.29:0/3717276107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd8003f40 0x7fccd8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:22.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fcceb7fe640 1 --2- 172.21.15.29:0/3717276107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccd8004d50 0x7fccd80051d0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009ae0 tx=0x7fccdc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51e435657388812 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:22.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0b2070 con 0x7fccd8004d50 2024-04-25T00:41:22.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccdc0a56a0 con 0x7fccd8004d50 2024-04-25T00:41:22.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fcce9ffb640 1 -- 172.21.15.29:0/3717276107 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0096b0 con 0x7fccd8004d50 2024-04-25T00:41:22.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fccf145b640 1 -- 172.21.15.29:0/3717276107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccd8004d50 msgr2=0x7fccd80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:22.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fccf145b640 1 --2- 172.21.15.29:0/3717276107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccd8004d50 0x7fccd80051d0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009ae0 tx=0x7fccdc01ca50 comp rx=0 tx=0).stop 2024-04-25T00:41:22.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fcceb7fe640 1 -- 172.21.15.29:0/3717276107 reap_dead start 2024-04-25T00:41:22.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fccf145b640 1 -- 172.21.15.29:0/3717276107 shutdown_connections 2024-04-25T00:41:22.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fccf145b640 1 -- 172.21.15.29:0/3717276107 >> 172.21.15.29:0/3717276107 conn(0x7fccec09aa30 msgr2=0x7fccec09ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:22.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fccf145b640 1 -- 172.21.15.29:0/3717276107 shutdown_connections 2024-04-25T00:41:22.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.880+0000 7fccf145b640 1 -- 172.21.15.29:0/3717276107 wait complete. 2024-04-25T00:41:22.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fccf145b640 1 Processor -- start 2024-04-25T00:41:22.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fccf145b640 1 -- start start 2024-04-25T00:41:22.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fccf145b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec1522a0 0x7fccec0cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:22.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fccf145b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccec09e920 0x7fccec0d1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:22.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fccf145b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec0d1650 0x7fccec1f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:22.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fccf145b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fccec0cdbb0 con 0x7fccec0d1650 2024-04-25T00:41:22.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fccf145b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccec0d0860 con 0x7fccec1522a0 2024-04-25T00:41:22.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fccf145b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fccec0cdd30 con 0x7fccec09e920 2024-04-25T00:41:22.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcceb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec0d1650 0x7fccec1f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:22.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcceb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec0d1650 0x7fccec1f6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46494/0 (socket says 172.21.15.29:46494) 2024-04-25T00:41:22.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcceb7fe640 1 -- 172.21.15.29:0/2589812167 learned_addr learned my addr 172.21.15.29:0/2589812167 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:22.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcceaffd640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec1522a0 0x7fccec0cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:22.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fccea7fc640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccec09e920 0x7fccec0d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:22.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcceb7fe640 1 -- 172.21.15.29:0/2589812167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccec09e920 msgr2=0x7fccec0d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:22.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcceb7fe640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccec09e920 0x7fccec0d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:22.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcceb7fe640 1 -- 172.21.15.29:0/2589812167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec1522a0 msgr2=0x7fccec0cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:22.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcceb7fe640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec1522a0 0x7fccec0cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:22.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcceb7fe640 1 -- 172.21.15.29:0/2589812167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec152680 con 0x7fccec0d1650 2024-04-25T00:41:22.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcceb7fe640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec0d1650 0x7fccec1f6f50 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0b0040 tx=0x7fccdc019850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:22.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcce37fe640 1 -- 172.21.15.29:0/2589812167 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0b2040 con 0x7fccec0d1650 2024-04-25T00:41:22.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcce37fe640 1 -- 172.21.15.29:0/2589812167 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccdc004020 con 0x7fccec0d1650 2024-04-25T00:41:22.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcce37fe640 1 -- 172.21.15.29:0/2589812167 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0096b0 con 0x7fccec0d1650 2024-04-25T00:41:22.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fccf145b640 1 -- 172.21.15.29:0/2589812167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccec09f100 con 0x7fccec0d1650 2024-04-25T00:41:22.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fccf145b640 1 -- 172.21.15.29:0/2589812167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccec09f6c0 con 0x7fccec0d1650 2024-04-25T00:41:22.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fccf145b640 1 -- 172.21.15.29:0/2589812167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccec06ccb0 con 0x7fccec0d1650 2024-04-25T00:41:22.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.884+0000 7fcce37fe640 1 -- 172.21.15.29:0/2589812167 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fccdc0bd060 con 0x7fccec0d1650 2024-04-25T00:41:22.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.888+0000 7fcce37fe640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcccc068250 0x7fcccc06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:22.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.888+0000 7fcce37fe640 1 -- 172.21.15.29:0/2589812167 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7fccdc11d650 con 0x7fccec0d1650 2024-04-25T00:41:22.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.888+0000 7fcceaffd640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcccc068250 0x7fcccc06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:22.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.888+0000 7fcceaffd640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcccc068250 0x7fcccc06a710 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcce4000f90 tx=0x7fcce4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:22.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:22.892+0000 7fcce37fe640 1 -- 172.21.15.29:0/2589812167 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fccdc0167c0 con 0x7fccec0d1650 2024-04-25T00:41:23.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.088+0000 7fccf145b640 1 -- 172.21.15.29:0/2589812167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fccec09f890 con 0x7fccec0d1650 2024-04-25T00:41:23.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.728+0000 7fcce37fe640 1 -- 172.21.15.29:0/2589812167 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v112) ==== 122+0+0 (secure 0 0 0) 0x7fccdc0efd90 con 0x7fccec0d1650 2024-04-25T00:41:23.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.728+0000 7fccf145b640 1 -- 172.21.15.29:0/2589812167 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcccc068250 msgr2=0x7fcccc06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:23.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.728+0000 7fccf145b640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcccc068250 0x7fcccc06a710 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcce4000f90 tx=0x7fcce4009040 comp rx=0 tx=0).stop 2024-04-25T00:41:23.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.728+0000 7fccf145b640 1 -- 172.21.15.29:0/2589812167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec0d1650 msgr2=0x7fccec1f6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:23.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.728+0000 7fccf145b640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec0d1650 0x7fccec1f6f50 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0b0040 tx=0x7fccdc019850 comp rx=0 tx=0).stop 2024-04-25T00:41:23.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.728+0000 7fccf145b640 1 -- 172.21.15.29:0/2589812167 shutdown_connections 2024-04-25T00:41:23.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.728+0000 7fccf145b640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcccc068250 0x7fcccc06a710 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:23.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.728+0000 7fccf145b640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec0d1650 0x7fccec1f6f50 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:23.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.728+0000 7fccf145b640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccec09e920 0x7fccec0d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:23.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.728+0000 7fccf145b640 1 --2- 172.21.15.29:0/2589812167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec1522a0 0x7fccec0cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:23.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.728+0000 7fccf145b640 1 -- 172.21.15.29:0/2589812167 >> 172.21.15.29:0/2589812167 conn(0x7fccec09aa30 msgr2=0x7fccec0cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:23.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.732+0000 7fccf145b640 1 -- 172.21.15.29:0/2589812167 shutdown_connections 2024-04-25T00:41:23.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.732+0000 7fccf145b640 1 -- 172.21.15.29:0/2589812167 wait complete. 2024-04-25T00:41:23.752 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:41:23.752 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:41:23.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe06be4b640 1 Processor -- start 2024-04-25T00:41:23.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe06be4b640 1 -- start start 2024-04-25T00:41:23.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe06be4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0640cf9f0 0x7fe0640d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:23.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe06be4b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0640cdcc0 con 0x7fe06415f5a0 2024-04-25T00:41:23.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe06be4b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe06415f980 con 0x7fe0640d0300 2024-04-25T00:41:23.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe06be4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe0640cc8a0 con 0x7fe0640cf9f0 2024-04-25T00:41:23.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe069be7640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe06415f5a0 0x7fe0640cd5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:51592/0 (socket says 172.21.15.29:51592) 2024-04-25T00:41:23.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe069be7640 1 -- 172.21.15.29:0/194621969 learned_addr learned my addr 172.21.15.29:0/194621969 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:23.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe0693e6640 1 --2- 172.21.15.29:0/194621969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0640cf9f0 0x7fe0640d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:23.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 903458142 0 0) 0x7fe0640cdcc0 con 0x7fe06415f5a0 2024-04-25T00:41:23.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe04c003530 con 0x7fe06415f5a0 2024-04-25T00:41:23.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4179309901 0 0) 0x7fe04c003530 con 0x7fe06415f5a0 2024-04-25T00:41:23.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0640cf9f0 msgr2=0x7fe0640d2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:23.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 --2- 172.21.15.29:0/194621969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0640cf9f0 0x7fe0640d2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:23.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 >> v1:172.21.15.157:6789/0 conn(0x7fe0640d0300 legacy=0x7fe0640c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:23.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0640cf3e0 con 0x7fe06415f5a0 2024-04-25T00:41:23.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fe05c001680 con 0x7fe06415f5a0 2024-04-25T00:41:23.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 >> v1:172.21.15.29:6789/0 conn(0x7fe06415f5a0 legacy=0x7fe0640cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:23.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 --2- 172.21.15.29:0/194621969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe04c0039f0 0x7fe04c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:23.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 --2- 172.21.15.29:0/194621969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0640cf9f0 0x7fe04c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:23.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 --2- 172.21.15.29:0/194621969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe04c004d50 0x7fe04c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:23.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c00bb10 con 0x7fe04c004d50 2024-04-25T00:41:23.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c00bc90 con 0x7fe04c0039f0 2024-04-25T00:41:23.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe04c00be90 con 0x7fe0640cf9f0 2024-04-25T00:41:23.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe06a3e8640 1 --2- 172.21.15.29:0/194621969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe04c004d50 0x7fe04c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:23.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe0693e6640 1 --2- 172.21.15.29:0/194621969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0640cf9f0 0x7fe04c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:23.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe069be7640 1 --2- 172.21.15.29:0/194621969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe04c0039f0 0x7fe04c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:23.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe06a3e8640 1 -- 172.21.15.29:0/194621969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0640cf9f0 msgr2=0x7fe04c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:23.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe06a3e8640 1 --2- 172.21.15.29:0/194621969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0640cf9f0 0x7fe04c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:23.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe06a3e8640 1 -- 172.21.15.29:0/194621969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe04c0039f0 msgr2=0x7fe04c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:23.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe06a3e8640 1 --2- 172.21.15.29:0/194621969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe04c0039f0 0x7fe04c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:23.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.880+0000 7fe06a3e8640 1 -- 172.21.15.29:0/194621969 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04c003530 con 0x7fe04c004d50 2024-04-25T00:41:23.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.884+0000 7fe06a3e8640 1 --2- 172.21.15.29:0/194621969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe04c004d50 0x7fe04c0051d0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fe054001750 tx=0x7fe05401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956da4b24cfb57eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:23.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.884+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0540013a0 con 0x7fe04c004d50 2024-04-25T00:41:23.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.884+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0540052f0 con 0x7fe04c004d50 2024-04-25T00:41:23.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.884+0000 7fe068be5640 1 -- 172.21.15.29:0/194621969 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054003840 con 0x7fe04c004d50 2024-04-25T00:41:23.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.884+0000 7fe06be4b640 1 -- 172.21.15.29:0/194621969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe04c004d50 msgr2=0x7fe04c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:23.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.884+0000 7fe06be4b640 1 --2- 172.21.15.29:0/194621969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe04c004d50 0x7fe04c0051d0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fe054001750 tx=0x7fe05401c3e0 comp rx=0 tx=0).stop 2024-04-25T00:41:23.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.884+0000 7fe06a3e8640 1 -- 172.21.15.29:0/194621969 reap_dead start 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.884+0000 7fe06be4b640 1 -- 172.21.15.29:0/194621969 shutdown_connections 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.884+0000 7fe06be4b640 1 -- 172.21.15.29:0/194621969 >> 172.21.15.29:0/194621969 conn(0x7fe0640d1d20 msgr2=0x7fe0640d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.884+0000 7fe06be4b640 1 -- 172.21.15.29:0/194621969 shutdown_connections 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.884+0000 7fe06be4b640 1 -- 172.21.15.29:0/194621969 wait complete. 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.884+0000 7fe06be4b640 1 Processor -- start 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.884+0000 7fe06be4b640 1 -- start start 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe06be4b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe04c004d50 0x7fe06414daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe06be4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06409e3d0 0x7fe06409e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe06be4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0640c9e50 0x7fe0640ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe06be4b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe0640cd090 con 0x7fe04c004d50 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe06be4b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe0640ccdb0 con 0x7fe06409e3d0 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe06be4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe0640cefb0 con 0x7fe0640c9e50 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe069be7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe04c004d50 0x7fe06414daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe069be7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe04c004d50 0x7fe06414daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46514/0 (socket says 172.21.15.29:46514) 2024-04-25T00:41:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe069be7640 1 -- 172.21.15.29:0/3944651396 learned_addr learned my addr 172.21.15.29:0/3944651396 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe069be7640 1 -- 172.21.15.29:0/3944651396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0640c9e50 msgr2=0x7fe0640ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe06a3e8640 1 --2- 172.21.15.29:0/3944651396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0640c9e50 0x7fe0640ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe069be7640 1 --2- 172.21.15.29:0/3944651396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0640c9e50 0x7fe0640ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe069be7640 1 -- 172.21.15.29:0/3944651396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06409e3d0 msgr2=0x7fe06409e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe069be7640 1 --2- 172.21.15.29:0/3944651396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06409e3d0 0x7fe06409e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe069be7640 1 -- 172.21.15.29:0/3944651396 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0640d2130 con 0x7fe04c004d50 2024-04-25T00:41:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe069be7640 1 --2- 172.21.15.29:0/3944651396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe04c004d50 0x7fe06414daf0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00e810 tx=0x7fe05c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe0627fc640 1 -- 172.21.15.29:0/3944651396 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00c870 con 0x7fe04c004d50 2024-04-25T00:41:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe0627fc640 1 -- 172.21.15.29:0/3944651396 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe05c018070 con 0x7fe04c004d50 2024-04-25T00:41:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe0627fc640 1 -- 172.21.15.29:0/3944651396 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00cc70 con 0x7fe04c004d50 2024-04-25T00:41:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe06be4b640 1 -- 172.21.15.29:0/3944651396 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06414e3f0 con 0x7fe04c004d50 2024-04-25T00:41:23.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe06be4b640 1 -- 172.21.15.29:0/3944651396 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe06414e690 con 0x7fe04c004d50 2024-04-25T00:41:23.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe06be4b640 1 -- 172.21.15.29:0/3944651396 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe064069270 con 0x7fe04c004d50 2024-04-25T00:41:23.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe0627fc640 1 -- 172.21.15.29:0/3944651396 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fe05c01d460 con 0x7fe04c004d50 2024-04-25T00:41:23.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe0627fc640 1 --2- 172.21.15.29:0/3944651396 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe050068250 0x7fe05006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:23.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe0627fc640 1 -- 172.21.15.29:0/3944651396 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7fe05c084340 con 0x7fe04c004d50 2024-04-25T00:41:23.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe0693e6640 1 --2- 172.21.15.29:0/3944651396 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe050068250 0x7fe05006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:23.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.888+0000 7fe0693e6640 1 --2- 172.21.15.29:0/3944651396 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe050068250 0x7fe05006a710 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe06415ba40 tx=0x7fe058009750 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:23.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:23.892+0000 7fe0627fc640 1 -- 172.21.15.29:0/3944651396 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe05c0569f0 con 0x7fe04c004d50 2024-04-25T00:41:24.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.084+0000 7fe06be4b640 1 -- 172.21.15.29:0/3944651396 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0640ce1c0 con 0x7fe04c004d50 2024-04-25T00:41:24.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.084+0000 7fe0627fc640 1 -- 172.21.15.29:0/3944651396 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5195 (secure 0 0 0) 0x7fe05c0562f0 con 0x7fe04c004d50 2024-04-25T00:41:24.086 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:24.086 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":112,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7427,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/929849226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":929849226},{"type":"v1","addr":"172.21.15.157:6834","nonce":929849226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1459365951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1459365951},{"type":"v1","addr":"172.21.15.29:6833","nonce":1459365951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1011867355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1011867355},{"type":"v1","addr":"172.21.15.157:6835","nonce":1011867355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:41:20.082895+0000","modified":"2024-04-25T00:41:23.093215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10578":{"gid":10578,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/85311029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":85311029},{"type":"v1","addr":"172.21.15.29:6835","nonce":85311029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10578,"qdb_cluster":[10578]},"id":13}]} 2024-04-25T00:41:24.087 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 112 2024-04-25T00:41:24.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.088+0000 7fe06be4b640 1 -- 172.21.15.29:0/3944651396 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe050068250 msgr2=0x7fe05006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.088+0000 7fe06be4b640 1 --2- 172.21.15.29:0/3944651396 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe050068250 0x7fe05006a710 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe06415ba40 tx=0x7fe058009750 comp rx=0 tx=0).stop 2024-04-25T00:41:24.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.088+0000 7fe06be4b640 1 -- 172.21.15.29:0/3944651396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe04c004d50 msgr2=0x7fe06414daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.088+0000 7fe06be4b640 1 --2- 172.21.15.29:0/3944651396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe04c004d50 0x7fe06414daf0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00e810 tx=0x7fe05c00ecd0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.088+0000 7fe06be4b640 1 -- 172.21.15.29:0/3944651396 shutdown_connections 2024-04-25T00:41:24.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.088+0000 7fe06be4b640 1 --2- 172.21.15.29:0/3944651396 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fe050068250 0x7fe05006a710 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.088+0000 7fe06be4b640 1 --2- 172.21.15.29:0/3944651396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0640c9e50 0x7fe0640ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.088+0000 7fe06be4b640 1 --2- 172.21.15.29:0/3944651396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06409e3d0 0x7fe06409e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.088+0000 7fe06be4b640 1 --2- 172.21.15.29:0/3944651396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe04c004d50 0x7fe06414daf0 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.088+0000 7fe06be4b640 1 -- 172.21.15.29:0/3944651396 >> 172.21.15.29:0/3944651396 conn(0x7fe0640d1d20 msgr2=0x7fe0640c5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:24.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.088+0000 7fe06be4b640 1 -- 172.21.15.29:0/3944651396 shutdown_connections 2024-04-25T00:41:24.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.088+0000 7fe06be4b640 1 -- 172.21.15.29:0/3944651396 wait complete. 2024-04-25T00:41:24.105 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:24.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdcaa0c2640 1 Processor -- start 2024-04-25T00:41:24.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdcaa0c2640 1 -- start start 2024-04-25T00:41:24.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdcaa0c2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e200 0x7fdca40d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdcaa0c2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdca40cf690 0x7fdca40d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdcaa0c2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdca40cfc10 con 0x7fdca40d2a90 2024-04-25T00:41:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdcaa0c2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdca40d2ee0 con 0x7fdca40cf690 2024-04-25T00:41:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdcaa0c2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdca40b38d0 con 0x7fdca415e200 2024-04-25T00:41:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdca2ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdca40cf690 0x7fdca40d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdca2ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdca40cf690 0x7fdca40d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46528/0 (socket says 172.21.15.29:46528) 2024-04-25T00:41:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdca2ffd640 1 -- 172.21.15.29:0/3329722156 learned_addr learned my addr 172.21.15.29:0/3329722156 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdca37fe640 1 --2- 172.21.15.29:0/3329722156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e200 0x7fdca40d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdca2ffd640 1 -- 172.21.15.29:0/3329722156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e200 msgr2=0x7fdca40d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:24.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdca2ffd640 1 --2- 172.21.15.29:0/3329722156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e200 0x7fdca40d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdca2ffd640 1 -- 172.21.15.29:0/3329722156 >> v1:172.21.15.157:6789/0 conn(0x7fdca40d2a90 legacy=0x7fdca40d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.228+0000 7fdca2ffd640 1 -- 172.21.15.29:0/3329722156 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca40cfef0 con 0x7fdca40cf690 2024-04-25T00:41:24.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca2ffd640 1 --2- 172.21.15.29:0/3329722156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdca40cf690 0x7fdca40d2550 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009870 tx=0x7fdc9401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9289d9bae67e2bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:24.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca27fc640 1 -- 172.21.15.29:0/3329722156 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc940a95f0 con 0x7fdca40cf690 2024-04-25T00:41:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca27fc640 1 -- 172.21.15.29:0/3329722156 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc940a9790 con 0x7fdca40cf690 2024-04-25T00:41:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca27fc640 1 -- 172.21.15.29:0/3329722156 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc940a9b60 con 0x7fdca40cf690 2024-04-25T00:41:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/3329722156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdca40cf690 msgr2=0x7fdca40d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 --2- 172.21.15.29:0/3329722156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdca40cf690 0x7fdca40d2550 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009870 tx=0x7fdc9401c530 comp rx=0 tx=0).stop 2024-04-25T00:41:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/3329722156 shutdown_connections 2024-04-25T00:41:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 --2- 172.21.15.29:0/3329722156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdca40cf690 0x7fdca40d2550 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 --2- 172.21.15.29:0/3329722156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca415e200 0x7fdca40d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/3329722156 >> 172.21.15.29:0/3329722156 conn(0x7fdca409d280 msgr2=0x7fdca40d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/3329722156 shutdown_connections 2024-04-25T00:41:24.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/3329722156 wait complete. 2024-04-25T00:41:24.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 Processor -- start 2024-04-25T00:41:24.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 -- start start 2024-04-25T00:41:24.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca40cf690 0x7fdca415fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca40d2a90 0x7fdca40cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdca415e200 0x7fdca4155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdca415e760 con 0x7fdca415e200 2024-04-25T00:41:24.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdca415e5e0 con 0x7fdca40d2a90 2024-04-25T00:41:24.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdcaa0c2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdca416ae30 con 0x7fdca40cf690 2024-04-25T00:41:24.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdca415e200 0x7fdca4155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdca415e200 0x7fdca4155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46534/0 (socket says 172.21.15.29:46534) 2024-04-25T00:41:24.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca3fff640 1 -- 172.21.15.29:0/2561222461 learned_addr learned my addr 172.21.15.29:0/2561222461 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:24.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca37fe640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca40cf690 0x7fdca415fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca2ffd640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca40d2a90 0x7fdca40cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca3fff640 1 -- 172.21.15.29:0/2561222461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca40cf690 msgr2=0x7fdca415fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca3fff640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca40cf690 0x7fdca415fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca3fff640 1 -- 172.21.15.29:0/2561222461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca40d2a90 msgr2=0x7fdca40cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca3fff640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca40d2a90 0x7fdca40cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.232+0000 7fdca3fff640 1 -- 172.21.15.29:0/2561222461 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca41534d0 con 0x7fdca415e200 2024-04-25T00:41:24.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.236+0000 7fdca3fff640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdca415e200 0x7fdca4155d60 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00c730 tx=0x7fdc9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:24.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.236+0000 7fdc9bfff640 1 -- 172.21.15.29:0/2561222461 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c003b80 con 0x7fdca415e200 2024-04-25T00:41:24.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.236+0000 7fdc9bfff640 1 -- 172.21.15.29:0/2561222461 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc9c003d20 con 0x7fdca415e200 2024-04-25T00:41:24.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.236+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/2561222461 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca41537b0 con 0x7fdca415e200 2024-04-25T00:41:24.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.236+0000 7fdc9bfff640 1 -- 172.21.15.29:0/2561222461 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00aa30 con 0x7fdca415e200 2024-04-25T00:41:24.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.236+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/2561222461 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca4153d70 con 0x7fdca415e200 2024-04-25T00:41:24.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.236+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/2561222461 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca4069270 con 0x7fdca415e200 2024-04-25T00:41:24.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.236+0000 7fdc9bfff640 1 -- 172.21.15.29:0/2561222461 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fdc9c00abd0 con 0x7fdca415e200 2024-04-25T00:41:24.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.236+0000 7fdc9bfff640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc84068250 0x7fdc8406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.236+0000 7fdc9bfff640 1 -- 172.21.15.29:0/2561222461 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7fdc9c084330 con 0x7fdca415e200 2024-04-25T00:41:24.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.236+0000 7fdca37fe640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc84068250 0x7fdc8406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.236+0000 7fdca37fe640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc84068250 0x7fdc8406a710 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00b9d0 tx=0x7fdc8c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:24.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.240+0000 7fdc9bfff640 1 -- 172.21.15.29:0/2561222461 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdc9c08a030 con 0x7fdca415e200 2024-04-25T00:41:24.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.408+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/2561222461 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdca40b5890 con 0x7fdca415e200 2024-04-25T00:41:24.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.412+0000 7fdc9bfff640 1 -- 172.21.15.29:0/2561222461 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) ==== 74+0+16635 (secure 0 0 0) 0x7fdc9c056a70 con 0x7fdca415e200 2024-04-25T00:41:24.412 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:24.413 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":168,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:41:21.712101+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:41:18.370636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:41:19.067830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:24.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.412+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/2561222461 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc84068250 msgr2=0x7fdc8406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.412+0000 7fdcaa0c2640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc84068250 0x7fdc8406a710 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00b9d0 tx=0x7fdc8c008040 comp rx=0 tx=0).stop 2024-04-25T00:41:24.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.412+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/2561222461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdca415e200 msgr2=0x7fdca4155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.412+0000 7fdcaa0c2640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdca415e200 0x7fdca4155d60 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00c730 tx=0x7fdc9c00cbf0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.412+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/2561222461 shutdown_connections 2024-04-25T00:41:24.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.412+0000 7fdcaa0c2640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc84068250 0x7fdc8406a710 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.412+0000 7fdcaa0c2640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdca415e200 0x7fdca4155d60 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.412+0000 7fdcaa0c2640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca40d2a90 0x7fdca40cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.412+0000 7fdcaa0c2640 1 --2- 172.21.15.29:0/2561222461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca40cf690 0x7fdca415fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.412+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/2561222461 >> 172.21.15.29:0/2561222461 conn(0x7fdca409d280 msgr2=0x7fdca415d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:24.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.412+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/2561222461 shutdown_connections 2024-04-25T00:41:24.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.412+0000 7fdcaa0c2640 1 -- 172.21.15.29:0/2561222461 wait complete. 2024-04-25T00:41:24.432 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:41:24.433 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:41:24.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.548+0000 7f298a5ca640 1 Processor -- start 2024-04-25T00:41:24.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.552+0000 7f298a5ca640 1 -- start start 2024-04-25T00:41:24.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.552+0000 7f298a5ca640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f29840b4780 0x7f298415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.552+0000 7f298a5ca640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29840b3720 con 0x7f298415e8d0 2024-04-25T00:41:24.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.552+0000 7f298a5ca640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29840b39c0 con 0x7f29840cc240 2024-04-25T00:41:24.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.552+0000 7f298a5ca640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f29840cc650 con 0x7f29840b4780 2024-04-25T00:41:24.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.552+0000 7f297bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f29840b4780 0x7f298415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.552+0000 7f2983fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f298415e8d0 0x7f298415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:51600/0 (socket says 172.21.15.29:51600) 2024-04-25T00:41:24.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.552+0000 7f2983fff640 1 -- 172.21.15.29:0/2560788683 learned_addr learned my addr 172.21.15.29:0/2560788683 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:24.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.552+0000 7f297bfff640 1 -- 172.21.15.29:0/2560788683 >> v1:172.21.15.157:6789/0 conn(0x7f29840cc240 legacy=0x7f298415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:24.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.552+0000 7f297bfff640 1 -- 172.21.15.29:0/2560788683 >> v1:172.21.15.29:6789/0 conn(0x7f298415e8d0 legacy=0x7f298415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.552+0000 7f297bfff640 1 -- 172.21.15.29:0/2560788683 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29840b3ca0 con 0x7f29840b4780 2024-04-25T00:41:24.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.552+0000 7f29837fe640 1 -- 172.21.15.29:0/2560788683 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 313590990 0 0) 0x7f29840b3720 con 0x7f298415e8d0 2024-04-25T00:41:24.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.552+0000 7f297bfff640 1 --2- 172.21.15.29:0/2560788683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29840b4780 0x7f298415f3c0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f296c0097c0 tx=0x7f296c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d0afacb83bcb3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:24.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f29837fe640 1 -- 172.21.15.29:0/2560788683 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c0b0070 con 0x7f29840b4780 2024-04-25T00:41:24.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f29837fe640 1 -- 172.21.15.29:0/2560788683 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296c0ab600 con 0x7f29840b4780 2024-04-25T00:41:24.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f29837fe640 1 -- 172.21.15.29:0/2560788683 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c0ab9d0 con 0x7f29840b4780 2024-04-25T00:41:24.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 -- 172.21.15.29:0/2560788683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29840b4780 msgr2=0x7f298415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 --2- 172.21.15.29:0/2560788683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29840b4780 0x7f298415f3c0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f296c0097c0 tx=0x7f296c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 -- 172.21.15.29:0/2560788683 shutdown_connections 2024-04-25T00:41:24.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 --2- 172.21.15.29:0/2560788683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29840b4780 0x7f298415f3c0 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 -- 172.21.15.29:0/2560788683 >> 172.21.15.29:0/2560788683 conn(0x7f29840b1870 msgr2=0x7f29840b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:24.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 -- 172.21.15.29:0/2560788683 shutdown_connections 2024-04-25T00:41:24.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 -- 172.21.15.29:0/2560788683 wait complete. 2024-04-25T00:41:24.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 Processor -- start 2024-04-25T00:41:24.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 -- start start 2024-04-25T00:41:24.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29840b4780 0x7f2984159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29840cc240 0x7f298415a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298415e8d0 0x7f2984138c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2984168e00 con 0x7f29840b4780 2024-04-25T00:41:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2984168c80 con 0x7f298415e8d0 2024-04-25T00:41:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2984168f80 con 0x7f29840cc240 2024-04-25T00:41:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f2983fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29840b4780 0x7f2984159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f2983fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29840b4780 0x7f2984159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46548/0 (socket says 172.21.15.29:46548) 2024-04-25T00:41:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f2983fff640 1 -- 172.21.15.29:0/3917753078 learned_addr learned my addr 172.21.15.29:0/3917753078 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f2988b67640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298415e8d0 0x7f2984138c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f2983fff640 1 -- 172.21.15.29:0/3917753078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29840cc240 msgr2=0x7f298415a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f2983fff640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29840cc240 0x7f298415a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f2983fff640 1 -- 172.21.15.29:0/3917753078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298415e8d0 msgr2=0x7f2984138c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f2983fff640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298415e8d0 0x7f2984138c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f2983fff640 1 -- 172.21.15.29:0/3917753078 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29840ce220 con 0x7f29840b4780 2024-04-25T00:41:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f2988b67640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298415e8d0 0x7f2984138c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:24.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f2983fff640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29840b4780 0x7f2984159fb0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f297400c410 tx=0x7f297400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:24.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f29817fa640 1 -- 172.21.15.29:0/3917753078 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29740074e0 con 0x7f29840b4780 2024-04-25T00:41:24.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f29817fa640 1 -- 172.21.15.29:0/3917753078 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29740047f0 con 0x7f29840b4780 2024-04-25T00:41:24.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 -- 172.21.15.29:0/3917753078 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29841f67c0 con 0x7f29840b4780 2024-04-25T00:41:24.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f29817fa640 1 -- 172.21.15.29:0/3917753078 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974007680 con 0x7f29840b4780 2024-04-25T00:41:24.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.556+0000 7f298a5ca640 1 -- 172.21.15.29:0/3917753078 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29841319b0 con 0x7f29840b4780 2024-04-25T00:41:24.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.560+0000 7f298a5ca640 1 -- 172.21.15.29:0/3917753078 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2984069270 con 0x7f29840b4780 2024-04-25T00:41:24.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.560+0000 7f29817fa640 1 -- 172.21.15.29:0/3917753078 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f2974011070 con 0x7f29840b4780 2024-04-25T00:41:24.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.560+0000 7f29817fa640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2964068250 0x7f296406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.560+0000 7f29817fa640 1 -- 172.21.15.29:0/3917753078 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f2974084040 con 0x7f29840b4780 2024-04-25T00:41:24.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.560+0000 7f297bfff640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2964068250 0x7f296406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.560+0000 7f297bfff640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2964068250 0x7f296406a710 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f296c0ad000 tx=0x7f296c004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.564+0000 7f29817fa640 1 -- 172.21.15.29:0/3917753078 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2974056780 con 0x7f29840b4780 2024-04-25T00:41:24.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.724+0000 7f298a5ca640 1 -- 172.21.15.29:0/3917753078 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f298413ae10 con 0x7f2964068250 2024-04-25T00:41:24.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.736+0000 7f29817fa640 1 -- 172.21.15.29:0/3917753078 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240298 (secure 0 0 0) 0x7f298413ae10 con 0x7f2964068250 2024-04-25T00:41:24.738 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:24.740 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:41:24.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.740+0000 7f298a5ca640 1 -- 172.21.15.29:0/3917753078 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2964068250 msgr2=0x7f296406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.740+0000 7f298a5ca640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2964068250 0x7f296406a710 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f296c0ad000 tx=0x7f296c004740 comp rx=0 tx=0).stop 2024-04-25T00:41:24.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.740+0000 7f298a5ca640 1 -- 172.21.15.29:0/3917753078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29840b4780 msgr2=0x7f2984159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.740+0000 7f298a5ca640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29840b4780 0x7f2984159fb0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f297400c410 tx=0x7f297400c8d0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.740+0000 7f298a5ca640 1 -- 172.21.15.29:0/3917753078 shutdown_connections 2024-04-25T00:41:24.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.740+0000 7f298a5ca640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2964068250 0x7f296406a710 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.740+0000 7f298a5ca640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298415e8d0 0x7f2984138c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.740+0000 7f298a5ca640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29840cc240 0x7f298415a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.740+0000 7f298a5ca640 1 --2- 172.21.15.29:0/3917753078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29840b4780 0x7f2984159fb0 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.740+0000 7f298a5ca640 1 -- 172.21.15.29:0/3917753078 >> 172.21.15.29:0/3917753078 conn(0x7f29840b1870 msgr2=0x7f2984139f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:24.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.740+0000 7f298a5ca640 1 -- 172.21.15.29:0/3917753078 shutdown_connections 2024-04-25T00:41:24.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.740+0000 7f298a5ca640 1 -- 172.21.15.29:0/3917753078 wait complete. 2024-04-25T00:41:24.757 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-04-25T00:41:23.420502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":308468,"kb_used_data":17412,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754666252,"statfs":{"total":773094113280,"available":772778242048,"internally_reserved":0,"allocated":17829888,"data_stored":8510581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.741384"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.688947+0000","last_change":"2024-04-25T00:41:18.688947+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.688947+0000","last_undegraded":"2024-04-25T00:41:18.688947+0000","last_fullsized":"2024-04-25T00:41:18.688947+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.684337+0000","last_change":"2024-04-25T00:41:18.684337+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.684337+0000","last_undegraded":"2024-04-25T00:41:18.684337+0000","last_fullsized":"2024-04-25T00:41:18.684337+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.690505+0000","last_change":"2024-04-25T00:41:18.690505+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.690505+0000","last_undegraded":"2024-04-25T00:41:18.690505+0000","last_fullsized":"2024-04-25T00:41:18.690505+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.692018+0000","last_change":"2024-04-25T00:41:18.692018+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.692018+0000","last_undegraded":"2024-04-25T00:41:18.692018+0000","last_fullsized":"2024-04-25T00:41:18.692018+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.687408+0000","last_change":"2024-04-25T00:41:18.687408+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.687408+0000","last_undegraded":"2024-04-25T00:41:18.687408+0000","last_fullsized":"2024-04-25T00:41:18.687408+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.688430+0000","last_change":"2024-04-25T00:41:18.688430+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.688430+0000","last_undegraded":"2024-04-25T00:41:18.688430+0000","last_fullsized":"2024-04-25T00:41:18.688430+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.683330+0000","last_change":"2024-04-25T00:41:18.683330+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.683330+0000","last_undegraded":"2024-04-25T00:41:18.683330+0000","last_fullsized":"2024-04-25T00:41:18.683330+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.686924+0000","last_change":"2024-04-25T00:41:18.686924+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.686924+0000","last_undegraded":"2024-04-25T00:41:18.686924+0000","last_fullsized":"2024-04-25T00:41:18.686924+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.685578+0000","last_change":"2024-04-25T00:41:18.685578+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.685578+0000","last_undegraded":"2024-04-25T00:41:18.685578+0000","last_fullsized":"2024-04-25T00:41:18.685578+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.686308+0000","last_change":"2024-04-25T00:41:18.686308+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.686308+0000","last_undegraded":"2024-04-25T00:41:18.686308+0000","last_fullsized":"2024-04-25T00:41:18.686308+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.690730+0000","last_change":"2024-04-25T00:41:18.690730+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.690730+0000","last_undegraded":"2024-04-25T00:41:18.690730+0000","last_fullsized":"2024-04-25T00:41:18.690730+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.683301+0000","last_change":"2024-04-25T00:41:18.683301+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.683301+0000","last_undegraded":"2024-04-25T00:41:18.683301+0000","last_fullsized":"2024-04-25T00:41:18.683301+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.684457+0000","last_change":"2024-04-25T00:41:18.684457+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.684457+0000","last_undegraded":"2024-04-25T00:41:18.684457+0000","last_fullsized":"2024-04-25T00:41:18.684457+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.684809+0000","last_change":"2024-04-25T00:41:18.684809+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.684809+0000","last_undegraded":"2024-04-25T00:41:18.684809+0000","last_fullsized":"2024-04-25T00:41:18.684809+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.689094+0000","last_change":"2024-04-25T00:41:18.689094+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.689094+0000","last_undegraded":"2024-04-25T00:41:18.689094+0000","last_fullsized":"2024-04-25T00:41:18.689094+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.686305+0000","last_change":"2024-04-25T00:41:18.686305+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.686305+0000","last_undegraded":"2024-04-25T00:41:18.686305+0000","last_fullsized":"2024-04-25T00:41:18.686305+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.683176+0000","last_change":"2024-04-25T00:41:18.683176+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.683176+0000","last_undegraded":"2024-04-25T00:41:18.683176+0000","last_fullsized":"2024-04-25T00:41:18.683176+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.685022+0000","last_change":"2024-04-25T00:41:18.685022+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.685022+0000","last_undegraded":"2024-04-25T00:41:18.685022+0000","last_fullsized":"2024-04-25T00:41:18.685022+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.689455+0000","last_change":"2024-04-25T00:41:18.689455+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.689455+0000","last_undegraded":"2024-04-25T00:41:18.689455+0000","last_fullsized":"2024-04-25T00:41:18.689455+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.684780+0000","last_change":"2024-04-25T00:41:18.684780+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.684780+0000","last_undegraded":"2024-04-25T00:41:18.684780+0000","last_fullsized":"2024-04-25T00:41:18.684780+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.689739+0000","last_change":"2024-04-25T00:41:18.689739+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.689739+0000","last_undegraded":"2024-04-25T00:41:18.689739+0000","last_fullsized":"2024-04-25T00:41:18.689739+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.689897+0000","last_change":"2024-04-25T00:41:18.689897+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.689897+0000","last_undegraded":"2024-04-25T00:41:18.689897+0000","last_fullsized":"2024-04-25T00:41:18.689897+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.686451+0000","last_change":"2024-04-25T00:41:18.686451+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.686451+0000","last_undegraded":"2024-04-25T00:41:18.686451+0000","last_fullsized":"2024-04-25T00:41:18.686451+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.690245+0000","last_change":"2024-04-25T00:41:18.690245+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.690245+0000","last_undegraded":"2024-04-25T00:41:18.690245+0000","last_fullsized":"2024-04-25T00:41:18.690245+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.690810+0000","last_change":"2024-04-25T00:41:18.690810+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.690810+0000","last_undegraded":"2024-04-25T00:41:18.690810+0000","last_fullsized":"2024-04-25T00:41:18.690810+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.686457+0000","last_change":"2024-04-25T00:41:18.686457+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.686457+0000","last_undegraded":"2024-04-25T00:41:18.686457+0000","last_fullsized":"2024-04-25T00:41:18.686457+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.688800+0000","last_change":"2024-04-25T00:41:18.688800+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.688800+0000","last_undegraded":"2024-04-25T00:41:18.688800+0000","last_fullsized":"2024-04-25T00:41:18.688800+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.690706+0000","last_change":"2024-04-25T00:41:18.690706+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.690706+0000","last_undegraded":"2024-04-25T00:41:18.690706+0000","last_fullsized":"2024-04-25T00:41:18.690706+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.686435+0000","last_change":"2024-04-25T00:41:18.686435+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.686435+0000","last_undegraded":"2024-04-25T00:41:18.686435+0000","last_fullsized":"2024-04-25T00:41:18.686435+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.683274+0000","last_change":"2024-04-25T00:41:18.683274+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.683274+0000","last_undegraded":"2024-04-25T00:41:18.683274+0000","last_fullsized":"2024-04-25T00:41:18.683274+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.687688+0000","last_change":"2024-04-25T00:41:18.687688+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.687688+0000","last_undegraded":"2024-04-25T00:41:18.687688+0000","last_fullsized":"2024-04-25T00:41:18.687688+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.687787+0000","last_change":"2024-04-25T00:41:18.687787+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.687787+0000","last_undegraded":"2024-04-25T00:41:18.687787+0000","last_fullsized":"2024-04-25T00:41:18.687787+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":483,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T00:41:18.693860+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:41:18.693860+0000","last_peered":"2024-04-25T00:41:18.693860+0000","last_clean":"2024-04-25T00:41:18.693860+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:41:18.693860+0000","last_undegraded":"2024-04-25T00:41:18.693860+0000","last_fullsized":"2024-04-25T00:41:18.693860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574970,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37948,"kb_used_data":2604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333892,"statfs":{"total":96636764160,"available":96597905408,"internally_reserved":0,"allocated":2666496,"data_stored":1502963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37376,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334464,"statfs":{"total":96636764160,"available":96598491136,"internally_reserved":0,"allocated":2084864,"data_stored":918412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37376,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334464,"statfs":{"total":96636764160,"available":96598491136,"internally_reserved":0,"allocated":2084864,"data_stored":918412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574970,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37364,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334476,"statfs":{"total":96636764160,"available":96598503424,"internally_reserved":0,"allocated":2072576,"data_stored":912595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574971,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41536,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330304,"statfs":{"total":96636764160,"available":96594231296,"internally_reserved":0,"allocated":2084864,"data_stored":918412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37368,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334472,"statfs":{"total":96636764160,"available":96598499328,"internally_reserved":0,"allocated":2072576,"data_stored":912595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37384,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2084864,"data_stored":918412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574970,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42116,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329724,"statfs":{"total":96636764160,"available":96593637376,"internally_reserved":0,"allocated":2678784,"data_stored":1508780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:41:24.761 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:41:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.924+0000 7f86318d0640 1 Processor -- start 2024-04-25T00:41:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.924+0000 7f86318d0640 1 -- start start 2024-04-25T00:41:24.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.928+0000 7f86318d0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f862c15e8d0 0x7f862c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.928+0000 7f86318d0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c0b4780 0x7f862c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.928+0000 7f86318d0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f862c0b3720 con 0x7f862c15f1f0 2024-04-25T00:41:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.928+0000 7f86318d0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f862c0b42f0 con 0x7f862c15e8d0 2024-04-25T00:41:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.928+0000 7f86318d0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f862c15d7e0 con 0x7f862c0b4780 2024-04-25T00:41:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.928+0000 7f862affd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f862c15e8d0 0x7f862c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.928+0000 7f862affd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f862c15e8d0 0x7f862c0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:45574/0 (socket says 172.21.15.29:45574) 2024-04-25T00:41:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.928+0000 7f862affd640 1 -- 172.21.15.29:0/3347987931 learned_addr learned my addr 172.21.15.29:0/3347987931 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.928+0000 7f862a7fc640 1 --2- 172.21.15.29:0/3347987931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c0b4780 0x7f862c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.928+0000 7f862affd640 1 -- 172.21.15.29:0/3347987931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c0b4780 msgr2=0x7f862c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.928+0000 7f862affd640 1 --2- 172.21.15.29:0/3347987931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c0b4780 0x7f862c15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.928+0000 7f862affd640 1 -- 172.21.15.29:0/3347987931 >> v1:172.21.15.157:6789/0 conn(0x7f862c15f1f0 legacy=0x7f862c15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.928+0000 7f862affd640 1 -- 172.21.15.29:0/3347987931 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c0b39c0 con 0x7f862c15e8d0 2024-04-25T00:41:24.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f862affd640 1 --2- 172.21.15.29:0/3347987931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c15e8d0 0x7f862c0cc240 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f861c009870 tx=0x7f861c01c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38b463bfb330d2f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:24.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f8629ffb640 1 -- 172.21.15.29:0/3347987931 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c15c1c0 con 0x7f862c15e8d0 2024-04-25T00:41:24.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f8629ffb640 1 -- 172.21.15.29:0/3347987931 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861c0a95f0 con 0x7f862c15e8d0 2024-04-25T00:41:24.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f8629ffb640 1 -- 172.21.15.29:0/3347987931 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c0a99c0 con 0x7f862c15e8d0 2024-04-25T00:41:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f86318d0640 1 -- 172.21.15.29:0/3347987931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c15e8d0 msgr2=0x7f862c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f86318d0640 1 --2- 172.21.15.29:0/3347987931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c15e8d0 0x7f862c0cc240 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f861c009870 tx=0x7f861c01c530 comp rx=0 tx=0).stop 2024-04-25T00:41:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f86318d0640 1 -- 172.21.15.29:0/3347987931 shutdown_connections 2024-04-25T00:41:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f86318d0640 1 --2- 172.21.15.29:0/3347987931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c0b4780 0x7f862c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f86318d0640 1 --2- 172.21.15.29:0/3347987931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c15e8d0 0x7f862c0cc240 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f86318d0640 1 -- 172.21.15.29:0/3347987931 >> 172.21.15.29:0/3347987931 conn(0x7f862c0b1870 msgr2=0x7f862c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f86318d0640 1 -- 172.21.15.29:0/3347987931 shutdown_connections 2024-04-25T00:41:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f86318d0640 1 -- 172.21.15.29:0/3347987931 wait complete. 2024-04-25T00:41:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f86318d0640 1 Processor -- start 2024-04-25T00:41:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f86318d0640 1 -- start start 2024-04-25T00:41:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f86318d0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c0b4780 0x7f862c13d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.932+0000 7f86318d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f862c15e8d0 0x7f862c1f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f86318d0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c15f1f0 0x7f862c0ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f86318d0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f862c144a20 con 0x7f862c15e8d0 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f86318d0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f862c15f600 con 0x7f862c0b4780 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f86318d0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f862c071150 con 0x7f862c15f1f0 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f862c15e8d0 0x7f862c1f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f862c15e8d0 0x7f862c1f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46550/0 (socket says 172.21.15.29:46550) 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862a7fc640 1 -- 172.21.15.29:0/4239686429 learned_addr learned my addr 172.21.15.29:0/4239686429 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862b7fe640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c15f1f0 0x7f862c0ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862affd640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c0b4780 0x7f862c13d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862a7fc640 1 -- 172.21.15.29:0/4239686429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c15f1f0 msgr2=0x7f862c0ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862a7fc640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c15f1f0 0x7f862c0ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862a7fc640 1 -- 172.21.15.29:0/4239686429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c0b4780 msgr2=0x7f862c13d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862a7fc640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c0b4780 0x7f862c13d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:24.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862a7fc640 1 -- 172.21.15.29:0/4239686429 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c071430 con 0x7f862c15e8d0 2024-04-25T00:41:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862affd640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c0b4780 0x7f862c13d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862a7fc640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f862c15e8d0 0x7f862c1f6f60 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f862000e9b0 tx=0x7f862000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f8613fff640 1 -- 172.21.15.29:0/4239686429 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620017070 con 0x7f862c15e8d0 2024-04-25T00:41:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f8613fff640 1 -- 172.21.15.29:0/4239686429 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862000c9a0 con 0x7f862c15e8d0 2024-04-25T00:41:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f86318d0640 1 -- 172.21.15.29:0/4239686429 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862c071710 con 0x7f862c15e8d0 2024-04-25T00:41:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f8613fff640 1 -- 172.21.15.29:0/4239686429 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862000cd70 con 0x7f862c15e8d0 2024-04-25T00:41:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f86318d0640 1 -- 172.21.15.29:0/4239686429 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862c13dd10 con 0x7f862c15e8d0 2024-04-25T00:41:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f86318d0640 1 -- 172.21.15.29:0/4239686429 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862c0718e0 con 0x7f862c15e8d0 2024-04-25T00:41:24.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f8613fff640 1 -- 172.21.15.29:0/4239686429 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f862001a070 con 0x7f862c15e8d0 2024-04-25T00:41:24.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f8613fff640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8618068280 0x7f861806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:24.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f8613fff640 1 -- 172.21.15.29:0/4239686429 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f86200844b0 con 0x7f862c15e8d0 2024-04-25T00:41:24.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862affd640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8618068280 0x7f861806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:24.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.936+0000 7f862affd640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8618068280 0x7f861806a740 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f861c0ae000 tx=0x7f861c0b3040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:24.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:24.940+0000 7f8613fff640 1 -- 172.21.15.29:0/4239686429 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8620056b70 con 0x7f862c15e8d0 2024-04-25T00:41:25.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.100+0000 7f86318d0640 1 -- 172.21.15.29:0/4239686429 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f862c071b10 con 0x7f8618068280 2024-04-25T00:41:25.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.112+0000 7f8613fff640 1 -- 172.21.15.29:0/4239686429 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240298 (secure 0 0 0) 0x7f861c0b15f0 con 0x7f8618068280 2024-04-25T00:41:25.114 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:25.115 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:41:25.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.112+0000 7f86318d0640 1 -- 172.21.15.29:0/4239686429 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8618068280 msgr2=0x7f861806a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.112+0000 7f86318d0640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8618068280 0x7f861806a740 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f861c0ae000 tx=0x7f861c0b3040 comp rx=0 tx=0).stop 2024-04-25T00:41:25.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.112+0000 7f86318d0640 1 -- 172.21.15.29:0/4239686429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f862c15e8d0 msgr2=0x7f862c1f6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.112+0000 7f86318d0640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f862c15e8d0 0x7f862c1f6f60 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f862000e9b0 tx=0x7f862000ee70 comp rx=0 tx=0).stop 2024-04-25T00:41:25.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.116+0000 7f86318d0640 1 -- 172.21.15.29:0/4239686429 shutdown_connections 2024-04-25T00:41:25.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.116+0000 7f86318d0640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8618068280 0x7f861806a740 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.116+0000 7f86318d0640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c15f1f0 0x7f862c0ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.116+0000 7f86318d0640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f862c15e8d0 0x7f862c1f6f60 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.116+0000 7f86318d0640 1 --2- 172.21.15.29:0/4239686429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c0b4780 0x7f862c13d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.116+0000 7f86318d0640 1 -- 172.21.15.29:0/4239686429 >> 172.21.15.29:0/4239686429 conn(0x7f862c0b1870 msgr2=0x7f862c0afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.116+0000 7f86318d0640 1 -- 172.21.15.29:0/4239686429 shutdown_connections 2024-04-25T00:41:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.116+0000 7f86318d0640 1 -- 172.21.15.29:0/4239686429 wait complete. 2024-04-25T00:41:25.133 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-04-25T00:41:23.420502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":308468,"kb_used_data":17412,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754666252,"statfs":{"total":773094113280,"available":772778242048,"internally_reserved":0,"allocated":17829888,"data_stored":8510581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.741384"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.688947+0000","last_change":"2024-04-25T00:41:18.688947+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.688947+0000","last_undegraded":"2024-04-25T00:41:18.688947+0000","last_fullsized":"2024-04-25T00:41:18.688947+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.684337+0000","last_change":"2024-04-25T00:41:18.684337+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.684337+0000","last_undegraded":"2024-04-25T00:41:18.684337+0000","last_fullsized":"2024-04-25T00:41:18.684337+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.690505+0000","last_change":"2024-04-25T00:41:18.690505+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.690505+0000","last_undegraded":"2024-04-25T00:41:18.690505+0000","last_fullsized":"2024-04-25T00:41:18.690505+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.692018+0000","last_change":"2024-04-25T00:41:18.692018+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.692018+0000","last_undegraded":"2024-04-25T00:41:18.692018+0000","last_fullsized":"2024-04-25T00:41:18.692018+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.687408+0000","last_change":"2024-04-25T00:41:18.687408+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.687408+0000","last_undegraded":"2024-04-25T00:41:18.687408+0000","last_fullsized":"2024-04-25T00:41:18.687408+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.688430+0000","last_change":"2024-04-25T00:41:18.688430+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.688430+0000","last_undegraded":"2024-04-25T00:41:18.688430+0000","last_fullsized":"2024-04-25T00:41:18.688430+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.683330+0000","last_change":"2024-04-25T00:41:18.683330+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.683330+0000","last_undegraded":"2024-04-25T00:41:18.683330+0000","last_fullsized":"2024-04-25T00:41:18.683330+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.686924+0000","last_change":"2024-04-25T00:41:18.686924+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.686924+0000","last_undegraded":"2024-04-25T00:41:18.686924+0000","last_fullsized":"2024-04-25T00:41:18.686924+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.685578+0000","last_change":"2024-04-25T00:41:18.685578+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.685578+0000","last_undegraded":"2024-04-25T00:41:18.685578+0000","last_fullsized":"2024-04-25T00:41:18.685578+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.686308+0000","last_change":"2024-04-25T00:41:18.686308+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.686308+0000","last_undegraded":"2024-04-25T00:41:18.686308+0000","last_fullsized":"2024-04-25T00:41:18.686308+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.690730+0000","last_change":"2024-04-25T00:41:18.690730+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.690730+0000","last_undegraded":"2024-04-25T00:41:18.690730+0000","last_fullsized":"2024-04-25T00:41:18.690730+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.683301+0000","last_change":"2024-04-25T00:41:18.683301+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.683301+0000","last_undegraded":"2024-04-25T00:41:18.683301+0000","last_fullsized":"2024-04-25T00:41:18.683301+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.684457+0000","last_change":"2024-04-25T00:41:18.684457+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.684457+0000","last_undegraded":"2024-04-25T00:41:18.684457+0000","last_fullsized":"2024-04-25T00:41:18.684457+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.684809+0000","last_change":"2024-04-25T00:41:18.684809+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.684809+0000","last_undegraded":"2024-04-25T00:41:18.684809+0000","last_fullsized":"2024-04-25T00:41:18.684809+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.689094+0000","last_change":"2024-04-25T00:41:18.689094+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.689094+0000","last_undegraded":"2024-04-25T00:41:18.689094+0000","last_fullsized":"2024-04-25T00:41:18.689094+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.686305+0000","last_change":"2024-04-25T00:41:18.686305+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.686305+0000","last_undegraded":"2024-04-25T00:41:18.686305+0000","last_fullsized":"2024-04-25T00:41:18.686305+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.683176+0000","last_change":"2024-04-25T00:41:18.683176+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.683176+0000","last_undegraded":"2024-04-25T00:41:18.683176+0000","last_fullsized":"2024-04-25T00:41:18.683176+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.685022+0000","last_change":"2024-04-25T00:41:18.685022+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.685022+0000","last_undegraded":"2024-04-25T00:41:18.685022+0000","last_fullsized":"2024-04-25T00:41:18.685022+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.689455+0000","last_change":"2024-04-25T00:41:18.689455+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.689455+0000","last_undegraded":"2024-04-25T00:41:18.689455+0000","last_fullsized":"2024-04-25T00:41:18.689455+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.684780+0000","last_change":"2024-04-25T00:41:18.684780+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.684780+0000","last_undegraded":"2024-04-25T00:41:18.684780+0000","last_fullsized":"2024-04-25T00:41:18.684780+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.689739+0000","last_change":"2024-04-25T00:41:18.689739+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.689739+0000","last_undegraded":"2024-04-25T00:41:18.689739+0000","last_fullsized":"2024-04-25T00:41:18.689739+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.689897+0000","last_change":"2024-04-25T00:41:18.689897+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.689897+0000","last_undegraded":"2024-04-25T00:41:18.689897+0000","last_fullsized":"2024-04-25T00:41:18.689897+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.686451+0000","last_change":"2024-04-25T00:41:18.686451+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.686451+0000","last_undegraded":"2024-04-25T00:41:18.686451+0000","last_fullsized":"2024-04-25T00:41:18.686451+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.690245+0000","last_change":"2024-04-25T00:41:18.690245+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.690245+0000","last_undegraded":"2024-04-25T00:41:18.690245+0000","last_fullsized":"2024-04-25T00:41:18.690245+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.690810+0000","last_change":"2024-04-25T00:41:18.690810+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.690810+0000","last_undegraded":"2024-04-25T00:41:18.690810+0000","last_fullsized":"2024-04-25T00:41:18.690810+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.686457+0000","last_change":"2024-04-25T00:41:18.686457+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.686457+0000","last_undegraded":"2024-04-25T00:41:18.686457+0000","last_fullsized":"2024-04-25T00:41:18.686457+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.688800+0000","last_change":"2024-04-25T00:41:18.688800+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.688800+0000","last_undegraded":"2024-04-25T00:41:18.688800+0000","last_fullsized":"2024-04-25T00:41:18.688800+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.690706+0000","last_change":"2024-04-25T00:41:18.690706+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.690706+0000","last_undegraded":"2024-04-25T00:41:18.690706+0000","last_fullsized":"2024-04-25T00:41:18.690706+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.686435+0000","last_change":"2024-04-25T00:41:18.686435+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.686435+0000","last_undegraded":"2024-04-25T00:41:18.686435+0000","last_fullsized":"2024-04-25T00:41:18.686435+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.683274+0000","last_change":"2024-04-25T00:41:18.683274+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.683274+0000","last_undegraded":"2024-04-25T00:41:18.683274+0000","last_fullsized":"2024-04-25T00:41:18.683274+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.687688+0000","last_change":"2024-04-25T00:41:18.687688+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.687688+0000","last_undegraded":"2024-04-25T00:41:18.687688+0000","last_fullsized":"2024-04-25T00:41:18.687688+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:41:19.678552+0000","last_change":"2024-04-25T00:41:19.678552+0000","last_active":"2024-04-25T00:41:19.678552+0000","last_peered":"2024-04-25T00:41:19.678552+0000","last_clean":"2024-04-25T00:41:19.678552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:19.678552+0000","last_undegraded":"2024-04-25T00:41:19.678552+0000","last_fullsized":"2024-04-25T00:41:19.678552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-04-25T00:41:18.687787+0000","last_change":"2024-04-25T00:41:18.687787+0000","last_active":"2024-04-25T00:41:18.666477+0000","last_peered":"2024-04-25T00:41:18.666477+0000","last_clean":"2024-04-25T00:41:18.666477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:41:18.687787+0000","last_undegraded":"2024-04-25T00:41:18.687787+0000","last_fullsized":"2024-04-25T00:41:18.687787+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":483,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T00:41:18.693860+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:41:18.693860+0000","last_peered":"2024-04-25T00:41:18.693860+0000","last_clean":"2024-04-25T00:41:18.693860+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:41:18.693860+0000","last_undegraded":"2024-04-25T00:41:18.693860+0000","last_fullsized":"2024-04-25T00:41:18.693860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574970,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37948,"kb_used_data":2604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333892,"statfs":{"total":96636764160,"available":96597905408,"internally_reserved":0,"allocated":2666496,"data_stored":1502963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37376,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334464,"statfs":{"total":96636764160,"available":96598491136,"internally_reserved":0,"allocated":2084864,"data_stored":918412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37376,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334464,"statfs":{"total":96636764160,"available":96598491136,"internally_reserved":0,"allocated":2084864,"data_stored":918412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574970,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37364,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334476,"statfs":{"total":96636764160,"available":96598503424,"internally_reserved":0,"allocated":2072576,"data_stored":912595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574971,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41536,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330304,"statfs":{"total":96636764160,"available":96594231296,"internally_reserved":0,"allocated":2084864,"data_stored":918412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37368,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334472,"statfs":{"total":96636764160,"available":96598499328,"internally_reserved":0,"allocated":2072576,"data_stored":912595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37384,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2084864,"data_stored":918412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574970,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42116,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329724,"statfs":{"total":96636764160,"available":96593637376,"internally_reserved":0,"allocated":2678784,"data_stored":1508780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:41:25.136 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:41:25.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.296+0000 7f7881fe5640 1 Processor -- start 2024-04-25T00:41:25.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.296+0000 7f7881fe5640 1 -- start start 2024-04-25T00:41:25.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.300+0000 7f7881fe5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f787c06dce0 0x7f787c150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.300+0000 7f7881fe5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f787c0b2b70 0x7f787c151330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.300+0000 7f7881fe5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f787c0b3130 con 0x7f787c15d980 2024-04-25T00:41:25.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.300+0000 7f7881fe5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f787c0cf560 con 0x7f787c06dce0 2024-04-25T00:41:25.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.300+0000 7f7881fe5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f787c0b3410 con 0x7f787c0b2b70 2024-04-25T00:41:25.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.300+0000 7f787affd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f787c06dce0 0x7f787c150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.300+0000 7f787bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f787c0b2b70 0x7f787c151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.300+0000 7f787affd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f787c06dce0 0x7f787c150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:45600/0 (socket says 172.21.15.29:45600) 2024-04-25T00:41:25.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.300+0000 7f787affd640 1 -- 172.21.15.29:0/812208378 learned_addr learned my addr 172.21.15.29:0/812208378 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:25.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.300+0000 7f787affd640 1 -- 172.21.15.29:0/812208378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f787c0b2b70 msgr2=0x7f787c151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:25.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.300+0000 7f787affd640 1 --2- 172.21.15.29:0/812208378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f787c0b2b70 0x7f787c151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.300+0000 7f787affd640 1 -- 172.21.15.29:0/812208378 >> v1:172.21.15.157:6789/0 conn(0x7f787c15d980 legacy=0x7f787c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.300+0000 7f787affd640 1 -- 172.21.15.29:0/812208378 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c0b36f0 con 0x7f787c06dce0 2024-04-25T00:41:25.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787affd640 1 --2- 172.21.15.29:0/812208378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f787c06dce0 0x7f787c150df0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f786400bb10 tx=0x7f786401ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c59f71cfb770d2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:25.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787a7fc640 1 -- 172.21.15.29:0/812208378 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78640b4070 con 0x7f787c06dce0 2024-04-25T00:41:25.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787a7fc640 1 -- 172.21.15.29:0/812208378 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78640a7b60 con 0x7f787c06dce0 2024-04-25T00:41:25.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787a7fc640 1 -- 172.21.15.29:0/812208378 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78640af440 con 0x7f787c06dce0 2024-04-25T00:41:25.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 -- 172.21.15.29:0/812208378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f787c06dce0 msgr2=0x7f787c150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 --2- 172.21.15.29:0/812208378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f787c06dce0 0x7f787c150df0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f786400bb10 tx=0x7f786401ec00 comp rx=0 tx=0).stop 2024-04-25T00:41:25.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 -- 172.21.15.29:0/812208378 shutdown_connections 2024-04-25T00:41:25.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 --2- 172.21.15.29:0/812208378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f787c0b2b70 0x7f787c151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 --2- 172.21.15.29:0/812208378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f787c06dce0 0x7f787c150df0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 -- 172.21.15.29:0/812208378 >> 172.21.15.29:0/812208378 conn(0x7f787c0b1870 msgr2=0x7f787c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:25.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 -- 172.21.15.29:0/812208378 shutdown_connections 2024-04-25T00:41:25.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 -- 172.21.15.29:0/812208378 wait complete. 2024-04-25T00:41:25.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 Processor -- start 2024-04-25T00:41:25.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 -- start start 2024-04-25T00:41:25.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f787c06dce0 0x7f787c0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f787c0b2b70 0x7f787c14d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f787c15d980 0x7f787c14a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f787c15b4b0 con 0x7f787c15d980 2024-04-25T00:41:25.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f787c151870 con 0x7f787c06dce0 2024-04-25T00:41:25.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f7881fe5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f787c15b630 con 0x7f787c0b2b70 2024-04-25T00:41:25.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f787c15d980 0x7f787c14a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f787c15d980 0x7f787c14a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46564/0 (socket says 172.21.15.29:46564) 2024-04-25T00:41:25.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787bfff640 1 -- 172.21.15.29:0/1892747787 learned_addr learned my addr 172.21.15.29:0/1892747787 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:25.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787affd640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f787c0b2b70 0x7f787c14d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787b7fe640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f787c06dce0 0x7f787c0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787bfff640 1 -- 172.21.15.29:0/1892747787 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f787c0b2b70 msgr2=0x7f787c14d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787bfff640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f787c0b2b70 0x7f787c14d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787bfff640 1 -- 172.21.15.29:0/1892747787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f787c06dce0 msgr2=0x7f787c0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787bfff640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f787c06dce0 0x7f787c0ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787bfff640 1 -- 172.21.15.29:0/1892747787 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c0cc870 con 0x7f787c15d980 2024-04-25T00:41:25.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.304+0000 7f787bfff640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f787c15d980 0x7f787c14a5b0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f787400efc0 tx=0x7f787400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:25.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.308+0000 7f7873fff640 1 -- 172.21.15.29:0/1892747787 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874018020 con 0x7f787c15d980 2024-04-25T00:41:25.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.308+0000 7f7881fe5640 1 -- 172.21.15.29:0/1892747787 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787c0ccb50 con 0x7f787c15d980 2024-04-25T00:41:25.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.308+0000 7f7873fff640 1 -- 172.21.15.29:0/1892747787 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7874014480 con 0x7f787c15d980 2024-04-25T00:41:25.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.308+0000 7f7873fff640 1 -- 172.21.15.29:0/1892747787 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874014870 con 0x7f787c15d980 2024-04-25T00:41:25.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.308+0000 7f7881fe5640 1 -- 172.21.15.29:0/1892747787 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787c0cd110 con 0x7f787c15d980 2024-04-25T00:41:25.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.308+0000 7f7881fe5640 1 -- 172.21.15.29:0/1892747787 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787c069270 con 0x7f787c15d980 2024-04-25T00:41:25.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.308+0000 7f7873fff640 1 -- 172.21.15.29:0/1892747787 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f787401b020 con 0x7f787c15d980 2024-04-25T00:41:25.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.308+0000 7f7873fff640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f785c068280 0x7f785c06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.308+0000 7f7873fff640 1 -- 172.21.15.29:0/1892747787 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f7874084ef0 con 0x7f787c15d980 2024-04-25T00:41:25.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.308+0000 7f787b7fe640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f785c068280 0x7f785c06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.308+0000 7f787b7fe640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f785c068280 0x7f785c06a740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f787c0cc2e0 tx=0x7f786c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:25.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.312+0000 7f7873fff640 1 -- 172.21.15.29:0/1892747787 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7874022070 con 0x7f787c15d980 2024-04-25T00:41:25.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.472+0000 7f7881fe5640 1 -- 172.21.15.29:0/1892747787 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f787c06e1d0 con 0x7f785c068280 2024-04-25T00:41:25.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.488+0000 7f7873fff640 1 -- 172.21.15.29:0/1892747787 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252492 (secure 0 0 0) 0x7f787c06e1d0 con 0x7f785c068280 2024-04-25T00:41:25.493 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:25.495 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:41:25.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.496+0000 7f7881fe5640 1 -- 172.21.15.29:0/1892747787 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f785c068280 msgr2=0x7f785c06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.496+0000 7f7881fe5640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f785c068280 0x7f785c06a740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f787c0cc2e0 tx=0x7f786c009040 comp rx=0 tx=0).stop 2024-04-25T00:41:25.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.496+0000 7f7881fe5640 1 -- 172.21.15.29:0/1892747787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f787c15d980 msgr2=0x7f787c14a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.496+0000 7f7881fe5640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f787c15d980 0x7f787c14a5b0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f787400efc0 tx=0x7f787400c490 comp rx=0 tx=0).stop 2024-04-25T00:41:25.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.496+0000 7f7881fe5640 1 -- 172.21.15.29:0/1892747787 shutdown_connections 2024-04-25T00:41:25.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.496+0000 7f7881fe5640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f785c068280 0x7f785c06a740 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.496+0000 7f7881fe5640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f787c15d980 0x7f787c14a5b0 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.496+0000 7f7881fe5640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f787c0b2b70 0x7f787c14d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.496+0000 7f7881fe5640 1 --2- 172.21.15.29:0/1892747787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f787c06dce0 0x7f787c0ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.496+0000 7f7881fe5640 1 -- 172.21.15.29:0/1892747787 >> 172.21.15.29:0/1892747787 conn(0x7f787c0b1870 msgr2=0x7f787c15bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:25.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.496+0000 7f7881fe5640 1 -- 172.21.15.29:0/1892747787 shutdown_connections 2024-04-25T00:41:25.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.496+0000 7f7881fe5640 1 -- 172.21.15.29:0/1892747787 wait complete. 2024-04-25T00:41:25.513 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-04-25T00:41:25.421054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":313028,"kb_used_data":17800,"kb_used_omap":12,"kb_used_meta":295091,"kb_avail":754661692,"statfs":{"total":773094113280,"available":772773572608,"internally_reserved":0,"allocated":18227200,"data_stored":8687652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":302173744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001699"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736869+0000","last_change":"2024-04-25T00:41:20.738051+0000","last_active":"2024-04-25T00:41:21.736869+0000","last_peered":"2024-04-25T00:41:21.736869+0000","last_clean":"2024-04-25T00:41:21.736869+0000","last_became_active":"2024-04-25T00:41:20.737775+0000","last_became_peered":"2024-04-25T00:41:20.737775+0000","last_unstale":"2024-04-25T00:41:21.736869+0000","last_undegraded":"2024-04-25T00:41:21.736869+0000","last_fullsized":"2024-04-25T00:41:21.736869+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:00.559458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728319+0000","last_change":"2024-04-25T00:41:20.754563+0000","last_active":"2024-04-25T00:41:21.728319+0000","last_peered":"2024-04-25T00:41:21.728319+0000","last_clean":"2024-04-25T00:41:21.728319+0000","last_became_active":"2024-04-25T00:41:20.753984+0000","last_became_peered":"2024-04-25T00:41:20.753984+0000","last_unstale":"2024-04-25T00:41:21.728319+0000","last_undegraded":"2024-04-25T00:41:21.728319+0000","last_fullsized":"2024-04-25T00:41:21.728319+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:01.910232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856753+0000","last_change":"2024-04-25T00:41:20.722575+0000","last_active":"2024-04-25T00:41:21.856753+0000","last_peered":"2024-04-25T00:41:21.856753+0000","last_clean":"2024-04-25T00:41:21.856753+0000","last_became_active":"2024-04-25T00:41:20.722097+0000","last_became_peered":"2024-04-25T00:41:20.722097+0000","last_unstale":"2024-04-25T00:41:21.856753+0000","last_undegraded":"2024-04-25T00:41:21.856753+0000","last_fullsized":"2024-04-25T00:41:21.856753+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:59.739712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724539+0000","last_change":"2024-04-25T00:41:20.754229+0000","last_active":"2024-04-25T00:41:21.724539+0000","last_peered":"2024-04-25T00:41:21.724539+0000","last_clean":"2024-04-25T00:41:21.724539+0000","last_became_active":"2024-04-25T00:41:20.753591+0000","last_became_peered":"2024-04-25T00:41:20.753591+0000","last_unstale":"2024-04-25T00:41:21.724539+0000","last_undegraded":"2024-04-25T00:41:21.724539+0000","last_fullsized":"2024-04-25T00:41:21.724539+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:14.056858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855011+0000","last_change":"2024-04-25T00:41:20.729817+0000","last_active":"2024-04-25T00:41:21.855011+0000","last_peered":"2024-04-25T00:41:21.855011+0000","last_clean":"2024-04-25T00:41:21.855011+0000","last_became_active":"2024-04-25T00:41:20.729299+0000","last_became_peered":"2024-04-25T00:41:20.729299+0000","last_unstale":"2024-04-25T00:41:21.855011+0000","last_undegraded":"2024-04-25T00:41:21.855011+0000","last_fullsized":"2024-04-25T00:41:21.855011+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:50.003262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853047+0000","last_change":"2024-04-25T00:41:20.738031+0000","last_active":"2024-04-25T00:41:21.853047+0000","last_peered":"2024-04-25T00:41:21.853047+0000","last_clean":"2024-04-25T00:41:21.853047+0000","last_became_active":"2024-04-25T00:41:20.737694+0000","last_became_peered":"2024-04-25T00:41:20.737694+0000","last_unstale":"2024-04-25T00:41:21.853047+0000","last_undegraded":"2024-04-25T00:41:21.853047+0000","last_fullsized":"2024-04-25T00:41:21.853047+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:12.354642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736461+0000","last_change":"2024-04-25T00:41:20.756463+0000","last_active":"2024-04-25T00:41:21.736461+0000","last_peered":"2024-04-25T00:41:21.736461+0000","last_clean":"2024-04-25T00:41:21.736461+0000","last_became_active":"2024-04-25T00:41:20.755935+0000","last_became_peered":"2024-04-25T00:41:20.755935+0000","last_unstale":"2024-04-25T00:41:21.736461+0000","last_undegraded":"2024-04-25T00:41:21.736461+0000","last_fullsized":"2024-04-25T00:41:21.736461+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:50.605977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852627+0000","last_change":"2024-04-25T00:41:20.730367+0000","last_active":"2024-04-25T00:41:21.852627+0000","last_peered":"2024-04-25T00:41:21.852627+0000","last_clean":"2024-04-25T00:41:21.852627+0000","last_became_active":"2024-04-25T00:41:20.729765+0000","last_became_peered":"2024-04-25T00:41:20.729765+0000","last_unstale":"2024-04-25T00:41:21.852627+0000","last_undegraded":"2024-04-25T00:41:21.852627+0000","last_fullsized":"2024-04-25T00:41:21.852627+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:29.081738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728881+0000","last_change":"2024-04-25T00:41:20.731164+0000","last_active":"2024-04-25T00:41:21.728881+0000","last_peered":"2024-04-25T00:41:21.728881+0000","last_clean":"2024-04-25T00:41:21.728881+0000","last_became_active":"2024-04-25T00:41:20.729720+0000","last_became_peered":"2024-04-25T00:41:20.729720+0000","last_unstale":"2024-04-25T00:41:21.728881+0000","last_undegraded":"2024-04-25T00:41:21.728881+0000","last_fullsized":"2024-04-25T00:41:21.728881+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:54.493042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853480+0000","last_change":"2024-04-25T00:41:20.728265+0000","last_active":"2024-04-25T00:41:21.853480+0000","last_peered":"2024-04-25T00:41:21.853480+0000","last_clean":"2024-04-25T00:41:21.853480+0000","last_became_active":"2024-04-25T00:41:20.727840+0000","last_became_peered":"2024-04-25T00:41:20.727840+0000","last_unstale":"2024-04-25T00:41:21.853480+0000","last_undegraded":"2024-04-25T00:41:21.853480+0000","last_fullsized":"2024-04-25T00:41:21.853480+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:25.159973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856794+0000","last_change":"2024-04-25T00:41:20.733820+0000","last_active":"2024-04-25T00:41:21.856794+0000","last_peered":"2024-04-25T00:41:21.856794+0000","last_clean":"2024-04-25T00:41:21.856794+0000","last_became_active":"2024-04-25T00:41:20.733516+0000","last_became_peered":"2024-04-25T00:41:20.733516+0000","last_unstale":"2024-04-25T00:41:21.856794+0000","last_undegraded":"2024-04-25T00:41:21.856794+0000","last_fullsized":"2024-04-25T00:41:21.856794+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:47.575779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.727368+0000","last_change":"2024-04-25T00:41:20.733108+0000","last_active":"2024-04-25T00:41:21.727368+0000","last_peered":"2024-04-25T00:41:21.727368+0000","last_clean":"2024-04-25T00:41:21.727368+0000","last_became_active":"2024-04-25T00:41:20.728236+0000","last_became_peered":"2024-04-25T00:41:20.728236+0000","last_unstale":"2024-04-25T00:41:21.727368+0000","last_undegraded":"2024-04-25T00:41:21.727368+0000","last_fullsized":"2024-04-25T00:41:21.727368+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:57.396464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.727972+0000","last_change":"2024-04-25T00:41:20.726309+0000","last_active":"2024-04-25T00:41:21.727972+0000","last_peered":"2024-04-25T00:41:21.727972+0000","last_clean":"2024-04-25T00:41:21.727972+0000","last_became_active":"2024-04-25T00:41:20.725444+0000","last_became_peered":"2024-04-25T00:41:20.725444+0000","last_unstale":"2024-04-25T00:41:21.727972+0000","last_undegraded":"2024-04-25T00:41:21.727972+0000","last_fullsized":"2024-04-25T00:41:21.727972+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:41.641469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736632+0000","last_change":"2024-04-25T00:41:20.727682+0000","last_active":"2024-04-25T00:41:21.736632+0000","last_peered":"2024-04-25T00:41:21.736632+0000","last_clean":"2024-04-25T00:41:21.736632+0000","last_became_active":"2024-04-25T00:41:20.727409+0000","last_became_peered":"2024-04-25T00:41:20.727409+0000","last_unstale":"2024-04-25T00:41:21.736632+0000","last_undegraded":"2024-04-25T00:41:21.736632+0000","last_fullsized":"2024-04-25T00:41:21.736632+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:48.999599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736447+0000","last_change":"2024-04-25T00:41:20.725702+0000","last_active":"2024-04-25T00:41:21.736447+0000","last_peered":"2024-04-25T00:41:21.736447+0000","last_clean":"2024-04-25T00:41:21.736447+0000","last_became_active":"2024-04-25T00:41:20.725313+0000","last_became_peered":"2024-04-25T00:41:20.725313+0000","last_unstale":"2024-04-25T00:41:21.736447+0000","last_undegraded":"2024-04-25T00:41:21.736447+0000","last_fullsized":"2024-04-25T00:41:21.736447+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:05.336967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852398+0000","last_change":"2024-04-25T00:41:20.734076+0000","last_active":"2024-04-25T00:41:21.852398+0000","last_peered":"2024-04-25T00:41:21.852398+0000","last_clean":"2024-04-25T00:41:21.852398+0000","last_became_active":"2024-04-25T00:41:20.733757+0000","last_became_peered":"2024-04-25T00:41:20.733757+0000","last_unstale":"2024-04-25T00:41:21.852398+0000","last_undegraded":"2024-04-25T00:41:21.852398+0000","last_fullsized":"2024-04-25T00:41:21.852398+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:16.226179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737048+0000","last_change":"2024-04-25T00:41:20.732582+0000","last_active":"2024-04-25T00:41:21.737048+0000","last_peered":"2024-04-25T00:41:21.737048+0000","last_clean":"2024-04-25T00:41:21.737048+0000","last_became_active":"2024-04-25T00:41:20.732276+0000","last_became_peered":"2024-04-25T00:41:20.732276+0000","last_unstale":"2024-04-25T00:41:21.737048+0000","last_undegraded":"2024-04-25T00:41:21.737048+0000","last_fullsized":"2024-04-25T00:41:21.737048+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:44.801392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853934+0000","last_change":"2024-04-25T00:41:20.729067+0000","last_active":"2024-04-25T00:41:21.853934+0000","last_peered":"2024-04-25T00:41:21.853934+0000","last_clean":"2024-04-25T00:41:21.853934+0000","last_became_active":"2024-04-25T00:41:20.728765+0000","last_became_peered":"2024-04-25T00:41:20.728765+0000","last_unstale":"2024-04-25T00:41:21.853934+0000","last_undegraded":"2024-04-25T00:41:21.853934+0000","last_fullsized":"2024-04-25T00:41:21.853934+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:31.971109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854502+0000","last_change":"2024-04-25T00:41:20.728640+0000","last_active":"2024-04-25T00:41:21.854502+0000","last_peered":"2024-04-25T00:41:21.854502+0000","last_clean":"2024-04-25T00:41:21.854502+0000","last_became_active":"2024-04-25T00:41:20.728290+0000","last_became_peered":"2024-04-25T00:41:20.728290+0000","last_unstale":"2024-04-25T00:41:21.854502+0000","last_undegraded":"2024-04-25T00:41:21.854502+0000","last_fullsized":"2024-04-25T00:41:21.854502+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:05.336967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855556+0000","last_change":"2024-04-25T00:41:20.729925+0000","last_active":"2024-04-25T00:41:21.855556+0000","last_peered":"2024-04-25T00:41:21.855556+0000","last_clean":"2024-04-25T00:41:21.855556+0000","last_became_active":"2024-04-25T00:41:20.729597+0000","last_became_peered":"2024-04-25T00:41:20.729597+0000","last_unstale":"2024-04-25T00:41:21.855556+0000","last_undegraded":"2024-04-25T00:41:21.855556+0000","last_fullsized":"2024-04-25T00:41:21.855556+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:27.879965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725550+0000","last_change":"2024-04-25T00:41:20.724283+0000","last_active":"2024-04-25T00:41:21.725550+0000","last_peered":"2024-04-25T00:41:21.725550+0000","last_clean":"2024-04-25T00:41:21.725550+0000","last_became_active":"2024-04-25T00:41:20.723569+0000","last_became_peered":"2024-04-25T00:41:20.723569+0000","last_unstale":"2024-04-25T00:41:21.725550+0000","last_undegraded":"2024-04-25T00:41:21.725550+0000","last_fullsized":"2024-04-25T00:41:21.725550+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:19.901587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852904+0000","last_change":"2024-04-25T00:41:20.746173+0000","last_active":"2024-04-25T00:41:21.852904+0000","last_peered":"2024-04-25T00:41:21.852904+0000","last_clean":"2024-04-25T00:41:21.852904+0000","last_became_active":"2024-04-25T00:41:20.745450+0000","last_became_peered":"2024-04-25T00:41:20.745450+0000","last_unstale":"2024-04-25T00:41:21.852904+0000","last_undegraded":"2024-04-25T00:41:21.852904+0000","last_fullsized":"2024-04-25T00:41:21.852904+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:12.354642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852684+0000","last_change":"2024-04-25T00:41:20.752770+0000","last_active":"2024-04-25T00:41:21.852684+0000","last_peered":"2024-04-25T00:41:21.852684+0000","last_clean":"2024-04-25T00:41:21.852684+0000","last_became_active":"2024-04-25T00:41:20.751527+0000","last_became_peered":"2024-04-25T00:41:20.751527+0000","last_unstale":"2024-04-25T00:41:21.852684+0000","last_undegraded":"2024-04-25T00:41:21.852684+0000","last_fullsized":"2024-04-25T00:41:21.852684+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:29.081738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724561+0000","last_change":"2024-04-25T00:41:20.726726+0000","last_active":"2024-04-25T00:41:21.724561+0000","last_peered":"2024-04-25T00:41:21.724561+0000","last_clean":"2024-04-25T00:41:21.724561+0000","last_became_active":"2024-04-25T00:41:20.726043+0000","last_became_peered":"2024-04-25T00:41:20.726043+0000","last_unstale":"2024-04-25T00:41:21.724561+0000","last_undegraded":"2024-04-25T00:41:21.724561+0000","last_fullsized":"2024-04-25T00:41:21.724561+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:23.550089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728079+0000","last_change":"2024-04-25T00:41:20.742119+0000","last_active":"2024-04-25T00:41:21.728079+0000","last_peered":"2024-04-25T00:41:21.728079+0000","last_clean":"2024-04-25T00:41:21.728079+0000","last_became_active":"2024-04-25T00:41:20.739704+0000","last_became_peered":"2024-04-25T00:41:20.739704+0000","last_unstale":"2024-04-25T00:41:21.728079+0000","last_undegraded":"2024-04-25T00:41:21.728079+0000","last_fullsized":"2024-04-25T00:41:21.728079+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:05.066385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736778+0000","last_change":"2024-04-25T00:41:20.756207+0000","last_active":"2024-04-25T00:41:21.736778+0000","last_peered":"2024-04-25T00:41:21.736778+0000","last_clean":"2024-04-25T00:41:21.736778+0000","last_became_active":"2024-04-25T00:41:20.755848+0000","last_became_peered":"2024-04-25T00:41:20.755848+0000","last_unstale":"2024-04-25T00:41:21.736778+0000","last_undegraded":"2024-04-25T00:41:21.736778+0000","last_fullsized":"2024-04-25T00:41:21.736778+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:04.914485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.727995+0000","last_change":"2024-04-25T00:41:20.728570+0000","last_active":"2024-04-25T00:41:21.727995+0000","last_peered":"2024-04-25T00:41:21.727995+0000","last_clean":"2024-04-25T00:41:21.727995+0000","last_became_active":"2024-04-25T00:41:20.727806+0000","last_became_peered":"2024-04-25T00:41:20.727806+0000","last_unstale":"2024-04-25T00:41:21.727995+0000","last_undegraded":"2024-04-25T00:41:21.727995+0000","last_fullsized":"2024-04-25T00:41:21.727995+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:39.749407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856704+0000","last_change":"2024-04-25T00:41:20.730663+0000","last_active":"2024-04-25T00:41:21.856704+0000","last_peered":"2024-04-25T00:41:21.856704+0000","last_clean":"2024-04-25T00:41:21.856704+0000","last_became_active":"2024-04-25T00:41:20.730330+0000","last_became_peered":"2024-04-25T00:41:20.730330+0000","last_unstale":"2024-04-25T00:41:21.856704+0000","last_undegraded":"2024-04-25T00:41:21.856704+0000","last_fullsized":"2024-04-25T00:41:21.856704+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:52.725891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.726305+0000","last_change":"2024-04-25T00:41:20.728144+0000","last_active":"2024-04-25T00:41:21.726305+0000","last_peered":"2024-04-25T00:41:21.726305+0000","last_clean":"2024-04-25T00:41:21.726305+0000","last_became_active":"2024-04-25T00:41:20.727266+0000","last_became_peered":"2024-04-25T00:41:20.727266+0000","last_unstale":"2024-04-25T00:41:21.726305+0000","last_undegraded":"2024-04-25T00:41:21.726305+0000","last_fullsized":"2024-04-25T00:41:21.726305+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:32.736055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.726950+0000","last_change":"2024-04-25T00:41:20.730975+0000","last_active":"2024-04-25T00:41:21.726950+0000","last_peered":"2024-04-25T00:41:21.726950+0000","last_clean":"2024-04-25T00:41:21.726950+0000","last_became_active":"2024-04-25T00:41:20.730142+0000","last_became_peered":"2024-04-25T00:41:20.730142+0000","last_unstale":"2024-04-25T00:41:21.726950+0000","last_undegraded":"2024-04-25T00:41:21.726950+0000","last_fullsized":"2024-04-25T00:41:21.726950+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:47.575779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852645+0000","last_change":"2024-04-25T00:41:20.723764+0000","last_active":"2024-04-25T00:41:21.852645+0000","last_peered":"2024-04-25T00:41:21.852645+0000","last_clean":"2024-04-25T00:41:21.852645+0000","last_became_active":"2024-04-25T00:41:20.723442+0000","last_became_peered":"2024-04-25T00:41:20.723442+0000","last_unstale":"2024-04-25T00:41:21.852645+0000","last_undegraded":"2024-04-25T00:41:21.852645+0000","last_fullsized":"2024-04-25T00:41:21.852645+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:19.901587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736358+0000","last_change":"2024-04-25T00:41:20.725607+0000","last_active":"2024-04-25T00:41:21.736358+0000","last_peered":"2024-04-25T00:41:21.736358+0000","last_clean":"2024-04-25T00:41:21.736358+0000","last_became_active":"2024-04-25T00:41:20.725195+0000","last_became_peered":"2024-04-25T00:41:20.725195+0000","last_unstale":"2024-04-25T00:41:21.736358+0000","last_undegraded":"2024-04-25T00:41:21.736358+0000","last_fullsized":"2024-04-25T00:41:21.736358+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:55.977495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728742+0000","last_change":"2024-04-25T00:41:20.733094+0000","last_active":"2024-04-25T00:41:21.728742+0000","last_peered":"2024-04-25T00:41:21.728742+0000","last_clean":"2024-04-25T00:41:21.728742+0000","last_became_active":"2024-04-25T00:41:20.732348+0000","last_became_peered":"2024-04-25T00:41:20.732348+0000","last_unstale":"2024-04-25T00:41:21.728742+0000","last_undegraded":"2024-04-25T00:41:21.728742+0000","last_fullsized":"2024-04-25T00:41:21.728742+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:57.037464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"167'4","reported_seq":18,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729495+0000","last_change":"2024-04-25T00:41:19.730561+0000","last_active":"2024-04-25T00:41:21.729495+0000","last_peered":"2024-04-25T00:41:21.729495+0000","last_clean":"2024-04-25T00:41:21.729495+0000","last_became_active":"2024-04-25T00:41:19.729475+0000","last_became_peered":"2024-04-25T00:41:19.729475+0000","last_unstale":"2024-04-25T00:41:21.729495+0000","last_undegraded":"2024-04-25T00:41:21.729495+0000","last_fullsized":"2024-04-25T00:41:21.729495+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:58.727637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853286+0000","last_change":"2024-04-25T00:41:20.754267+0000","last_active":"2024-04-25T00:41:21.853286+0000","last_peered":"2024-04-25T00:41:21.853286+0000","last_clean":"2024-04-25T00:41:21.853286+0000","last_became_active":"2024-04-25T00:41:20.753526+0000","last_became_peered":"2024-04-25T00:41:20.753526+0000","last_unstale":"2024-04-25T00:41:21.853286+0000","last_undegraded":"2024-04-25T00:41:21.853286+0000","last_fullsized":"2024-04-25T00:41:21.853286+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:27.879965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729086+0000","last_change":"2024-04-25T00:41:19.728196+0000","last_active":"2024-04-25T00:41:21.729086+0000","last_peered":"2024-04-25T00:41:21.729086+0000","last_clean":"2024-04-25T00:41:21.729086+0000","last_became_active":"2024-04-25T00:41:19.727469+0000","last_became_peered":"2024-04-25T00:41:19.727469+0000","last_unstale":"2024-04-25T00:41:21.729086+0000","last_undegraded":"2024-04-25T00:41:21.729086+0000","last_fullsized":"2024-04-25T00:41:21.729086+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:18.103810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736679+0000","last_change":"2024-04-25T00:41:20.757686+0000","last_active":"2024-04-25T00:41:21.736679+0000","last_peered":"2024-04-25T00:41:21.736679+0000","last_clean":"2024-04-25T00:41:21.736679+0000","last_became_active":"2024-04-25T00:41:20.757425+0000","last_became_peered":"2024-04-25T00:41:20.757425+0000","last_unstale":"2024-04-25T00:41:21.736679+0000","last_undegraded":"2024-04-25T00:41:21.736679+0000","last_fullsized":"2024-04-25T00:41:21.736679+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:29.413633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856910+0000","last_change":"2024-04-25T00:41:19.714844+0000","last_active":"2024-04-25T00:41:21.856910+0000","last_peered":"2024-04-25T00:41:21.856910+0000","last_clean":"2024-04-25T00:41:21.856910+0000","last_became_active":"2024-04-25T00:41:19.714584+0000","last_became_peered":"2024-04-25T00:41:19.714584+0000","last_unstale":"2024-04-25T00:41:21.856910+0000","last_undegraded":"2024-04-25T00:41:21.856910+0000","last_fullsized":"2024-04-25T00:41:21.856910+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:17.955930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854015+0000","last_change":"2024-04-25T00:41:20.729450+0000","last_active":"2024-04-25T00:41:21.854015+0000","last_peered":"2024-04-25T00:41:21.854015+0000","last_clean":"2024-04-25T00:41:21.854015+0000","last_became_active":"2024-04-25T00:41:20.728986+0000","last_became_peered":"2024-04-25T00:41:20.728986+0000","last_unstale":"2024-04-25T00:41:21.854015+0000","last_undegraded":"2024-04-25T00:41:21.854015+0000","last_fullsized":"2024-04-25T00:41:21.854015+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:44.845619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725827+0000","last_change":"2024-04-25T00:41:19.727073+0000","last_active":"2024-04-25T00:41:21.725827+0000","last_peered":"2024-04-25T00:41:21.725827+0000","last_clean":"2024-04-25T00:41:21.725827+0000","last_became_active":"2024-04-25T00:41:19.726232+0000","last_became_peered":"2024-04-25T00:41:19.726232+0000","last_unstale":"2024-04-25T00:41:21.725827+0000","last_undegraded":"2024-04-25T00:41:21.725827+0000","last_fullsized":"2024-04-25T00:41:21.725827+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:12.458015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724621+0000","last_change":"2024-04-25T00:41:20.754166+0000","last_active":"2024-04-25T00:41:21.724621+0000","last_peered":"2024-04-25T00:41:21.724621+0000","last_clean":"2024-04-25T00:41:21.724621+0000","last_became_active":"2024-04-25T00:41:20.753650+0000","last_became_peered":"2024-04-25T00:41:20.753650+0000","last_unstale":"2024-04-25T00:41:21.724621+0000","last_undegraded":"2024-04-25T00:41:21.724621+0000","last_fullsized":"2024-04-25T00:41:21.724621+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:17.802309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853585+0000","last_change":"2024-04-25T00:41:19.723133+0000","last_active":"2024-04-25T00:41:21.853585+0000","last_peered":"2024-04-25T00:41:21.853585+0000","last_clean":"2024-04-25T00:41:21.853585+0000","last_became_active":"2024-04-25T00:41:19.722748+0000","last_became_peered":"2024-04-25T00:41:19.722748+0000","last_unstale":"2024-04-25T00:41:21.853585+0000","last_undegraded":"2024-04-25T00:41:21.853585+0000","last_fullsized":"2024-04-25T00:41:21.853585+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:53.480966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852317+0000","last_change":"2024-04-25T00:41:20.720406+0000","last_active":"2024-04-25T00:41:21.852317+0000","last_peered":"2024-04-25T00:41:21.852317+0000","last_clean":"2024-04-25T00:41:21.852317+0000","last_became_active":"2024-04-25T00:41:20.720072+0000","last_became_peered":"2024-04-25T00:41:20.720072+0000","last_unstale":"2024-04-25T00:41:21.852317+0000","last_undegraded":"2024-04-25T00:41:21.852317+0000","last_fullsized":"2024-04-25T00:41:21.852317+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:16.067964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852079+0000","last_change":"2024-04-25T00:41:19.712612+0000","last_active":"2024-04-25T00:41:21.852079+0000","last_peered":"2024-04-25T00:41:21.852079+0000","last_clean":"2024-04-25T00:41:21.852079+0000","last_became_active":"2024-04-25T00:41:19.710382+0000","last_became_peered":"2024-04-25T00:41:19.710382+0000","last_unstale":"2024-04-25T00:41:21.852079+0000","last_undegraded":"2024-04-25T00:41:21.852079+0000","last_fullsized":"2024-04-25T00:41:21.852079+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:41.339940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725041+0000","last_change":"2024-04-25T00:41:20.755342+0000","last_active":"2024-04-25T00:41:21.725041+0000","last_peered":"2024-04-25T00:41:21.725041+0000","last_clean":"2024-04-25T00:41:21.725041+0000","last_became_active":"2024-04-25T00:41:20.754876+0000","last_became_peered":"2024-04-25T00:41:20.754876+0000","last_unstale":"2024-04-25T00:41:21.725041+0000","last_undegraded":"2024-04-25T00:41:21.725041+0000","last_fullsized":"2024-04-25T00:41:21.725041+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:16.226179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852140+0000","last_change":"2024-04-25T00:41:19.723756+0000","last_active":"2024-04-25T00:41:21.852140+0000","last_peered":"2024-04-25T00:41:21.852140+0000","last_clean":"2024-04-25T00:41:21.852140+0000","last_became_active":"2024-04-25T00:41:19.722504+0000","last_became_peered":"2024-04-25T00:41:19.722504+0000","last_unstale":"2024-04-25T00:41:21.852140+0000","last_undegraded":"2024-04-25T00:41:21.852140+0000","last_fullsized":"2024-04-25T00:41:21.852140+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:53.480966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728998+0000","last_change":"2024-04-25T00:41:20.715594+0000","last_active":"2024-04-25T00:41:21.728998+0000","last_peered":"2024-04-25T00:41:21.728998+0000","last_clean":"2024-04-25T00:41:21.728998+0000","last_became_active":"2024-04-25T00:41:20.712098+0000","last_became_peered":"2024-04-25T00:41:20.712098+0000","last_unstale":"2024-04-25T00:41:21.728998+0000","last_undegraded":"2024-04-25T00:41:21.728998+0000","last_fullsized":"2024-04-25T00:41:21.728998+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:53.784754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855238+0000","last_change":"2024-04-25T00:41:19.727211+0000","last_active":"2024-04-25T00:41:21.855238+0000","last_peered":"2024-04-25T00:41:21.855238+0000","last_clean":"2024-04-25T00:41:21.855238+0000","last_became_active":"2024-04-25T00:41:19.726926+0000","last_became_peered":"2024-04-25T00:41:19.726926+0000","last_unstale":"2024-04-25T00:41:21.855238+0000","last_undegraded":"2024-04-25T00:41:21.855238+0000","last_fullsized":"2024-04-25T00:41:21.855238+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:04.054309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856661+0000","last_change":"2024-04-25T00:41:20.723273+0000","last_active":"2024-04-25T00:41:21.856661+0000","last_peered":"2024-04-25T00:41:21.856661+0000","last_clean":"2024-04-25T00:41:21.856661+0000","last_became_active":"2024-04-25T00:41:20.722938+0000","last_became_peered":"2024-04-25T00:41:20.722938+0000","last_unstale":"2024-04-25T00:41:21.856661+0000","last_undegraded":"2024-04-25T00:41:21.856661+0000","last_fullsized":"2024-04-25T00:41:21.856661+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:29.975327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853121+0000","last_change":"2024-04-25T00:41:19.733298+0000","last_active":"2024-04-25T00:41:21.853121+0000","last_peered":"2024-04-25T00:41:21.853121+0000","last_clean":"2024-04-25T00:41:21.853121+0000","last_became_active":"2024-04-25T00:41:19.732514+0000","last_became_peered":"2024-04-25T00:41:19.732514+0000","last_unstale":"2024-04-25T00:41:21.853121+0000","last_undegraded":"2024-04-25T00:41:21.853121+0000","last_fullsized":"2024-04-25T00:41:21.853121+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:56.384389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724912+0000","last_change":"2024-04-25T00:41:20.716394+0000","last_active":"2024-04-25T00:41:21.724912+0000","last_peered":"2024-04-25T00:41:21.724912+0000","last_clean":"2024-04-25T00:41:21.724912+0000","last_became_active":"2024-04-25T00:41:20.715624+0000","last_became_peered":"2024-04-25T00:41:20.715624+0000","last_unstale":"2024-04-25T00:41:21.724912+0000","last_undegraded":"2024-04-25T00:41:21.724912+0000","last_fullsized":"2024-04-25T00:41:21.724912+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:23.460044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"167'3","reported_seq":17,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854498+0000","last_change":"2024-04-25T00:41:19.714290+0000","last_active":"2024-04-25T00:41:21.854498+0000","last_peered":"2024-04-25T00:41:21.854498+0000","last_clean":"2024-04-25T00:41:21.854498+0000","last_became_active":"2024-04-25T00:41:19.713997+0000","last_became_peered":"2024-04-25T00:41:19.713997+0000","last_unstale":"2024-04-25T00:41:21.854498+0000","last_undegraded":"2024-04-25T00:41:21.854498+0000","last_fullsized":"2024-04-25T00:41:21.854498+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:47.272504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737071+0000","last_change":"2024-04-25T00:41:20.729697+0000","last_active":"2024-04-25T00:41:21.737071+0000","last_peered":"2024-04-25T00:41:21.737071+0000","last_clean":"2024-04-25T00:41:21.737071+0000","last_became_active":"2024-04-25T00:41:20.728724+0000","last_became_peered":"2024-04-25T00:41:20.728724+0000","last_unstale":"2024-04-25T00:41:21.737071+0000","last_undegraded":"2024-04-25T00:41:21.737071+0000","last_fullsized":"2024-04-25T00:41:21.737071+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:54.350976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"167'6","reported_seq":20,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.726112+0000","last_change":"2024-04-25T00:41:19.726821+0000","last_active":"2024-04-25T00:41:21.726112+0000","last_peered":"2024-04-25T00:41:21.726112+0000","last_clean":"2024-04-25T00:41:21.726112+0000","last_became_active":"2024-04-25T00:41:19.726146+0000","last_became_peered":"2024-04-25T00:41:19.726146+0000","last_unstale":"2024-04-25T00:41:21.726112+0000","last_undegraded":"2024-04-25T00:41:21.726112+0000","last_fullsized":"2024-04-25T00:41:21.726112+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:59.876465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853444+0000","last_change":"2024-04-25T00:41:20.731223+0000","last_active":"2024-04-25T00:41:21.853444+0000","last_peered":"2024-04-25T00:41:21.853444+0000","last_clean":"2024-04-25T00:41:21.853444+0000","last_became_active":"2024-04-25T00:41:20.730734+0000","last_became_peered":"2024-04-25T00:41:20.730734+0000","last_unstale":"2024-04-25T00:41:21.853444+0000","last_undegraded":"2024-04-25T00:41:21.853444+0000","last_fullsized":"2024-04-25T00:41:21.853444+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:21.777831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852468+0000","last_change":"2024-04-25T00:41:19.715184+0000","last_active":"2024-04-25T00:41:21.852468+0000","last_peered":"2024-04-25T00:41:21.852468+0000","last_clean":"2024-04-25T00:41:21.852468+0000","last_became_active":"2024-04-25T00:41:19.713383+0000","last_became_peered":"2024-04-25T00:41:19.713383+0000","last_unstale":"2024-04-25T00:41:21.852468+0000","last_undegraded":"2024-04-25T00:41:21.852468+0000","last_fullsized":"2024-04-25T00:41:21.852468+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:41.339940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853053+0000","last_change":"2024-04-25T00:41:20.726316+0000","last_active":"2024-04-25T00:41:21.853053+0000","last_peered":"2024-04-25T00:41:21.853053+0000","last_clean":"2024-04-25T00:41:21.853053+0000","last_became_active":"2024-04-25T00:41:20.724262+0000","last_became_peered":"2024-04-25T00:41:20.724262+0000","last_unstale":"2024-04-25T00:41:21.853053+0000","last_undegraded":"2024-04-25T00:41:21.853053+0000","last_fullsized":"2024-04-25T00:41:21.853053+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:53.784754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852580+0000","last_change":"2024-04-25T00:41:19.724753+0000","last_active":"2024-04-25T00:41:21.852580+0000","last_peered":"2024-04-25T00:41:21.852580+0000","last_clean":"2024-04-25T00:41:21.852580+0000","last_became_active":"2024-04-25T00:41:19.721745+0000","last_became_peered":"2024-04-25T00:41:19.721745+0000","last_unstale":"2024-04-25T00:41:21.852580+0000","last_undegraded":"2024-04-25T00:41:21.852580+0000","last_fullsized":"2024-04-25T00:41:21.852580+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:31.723980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856313+0000","last_change":"2024-04-25T00:41:20.727156+0000","last_active":"2024-04-25T00:41:21.856313+0000","last_peered":"2024-04-25T00:41:21.856313+0000","last_clean":"2024-04-25T00:41:21.856313+0000","last_became_active":"2024-04-25T00:41:20.725938+0000","last_became_peered":"2024-04-25T00:41:20.725938+0000","last_unstale":"2024-04-25T00:41:21.856313+0000","last_undegraded":"2024-04-25T00:41:21.856313+0000","last_fullsized":"2024-04-25T00:41:21.856313+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:03.298965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852561+0000","last_change":"2024-04-25T00:41:19.725909+0000","last_active":"2024-04-25T00:41:21.852561+0000","last_peered":"2024-04-25T00:41:21.852561+0000","last_clean":"2024-04-25T00:41:21.852561+0000","last_became_active":"2024-04-25T00:41:19.725301+0000","last_became_peered":"2024-04-25T00:41:19.725301+0000","last_unstale":"2024-04-25T00:41:21.852561+0000","last_undegraded":"2024-04-25T00:41:21.852561+0000","last_fullsized":"2024-04-25T00:41:21.852561+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:59.876465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736566+0000","last_change":"2024-04-25T00:41:20.746071+0000","last_active":"2024-04-25T00:41:21.736566+0000","last_peered":"2024-04-25T00:41:21.736566+0000","last_clean":"2024-04-25T00:41:21.736566+0000","last_became_active":"2024-04-25T00:41:20.745714+0000","last_became_peered":"2024-04-25T00:41:20.745714+0000","last_unstale":"2024-04-25T00:41:21.736566+0000","last_undegraded":"2024-04-25T00:41:21.736566+0000","last_fullsized":"2024-04-25T00:41:21.736566+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:10.415430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"167'8","reported_seq":22,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855744+0000","last_change":"2024-04-25T00:41:19.720510+0000","last_active":"2024-04-25T00:41:21.855744+0000","last_peered":"2024-04-25T00:41:21.855744+0000","last_clean":"2024-04-25T00:41:21.855744+0000","last_became_active":"2024-04-25T00:41:19.720185+0000","last_became_peered":"2024-04-25T00:41:19.720185+0000","last_unstale":"2024-04-25T00:41:21.855744+0000","last_undegraded":"2024-04-25T00:41:21.855744+0000","last_fullsized":"2024-04-25T00:41:21.855744+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:50.914418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729150+0000","last_change":"2024-04-25T00:41:20.710672+0000","last_active":"2024-04-25T00:41:21.729150+0000","last_peered":"2024-04-25T00:41:21.729150+0000","last_clean":"2024-04-25T00:41:21.729150+0000","last_became_active":"2024-04-25T00:41:20.709976+0000","last_became_peered":"2024-04-25T00:41:20.709976+0000","last_unstale":"2024-04-25T00:41:21.729150+0000","last_undegraded":"2024-04-25T00:41:21.729150+0000","last_fullsized":"2024-04-25T00:41:21.729150+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:45.163125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855527+0000","last_change":"2024-04-25T00:41:19.721839+0000","last_active":"2024-04-25T00:41:21.855527+0000","last_peered":"2024-04-25T00:41:21.855527+0000","last_clean":"2024-04-25T00:41:21.855527+0000","last_became_active":"2024-04-25T00:41:19.721547+0000","last_became_peered":"2024-04-25T00:41:19.721547+0000","last_unstale":"2024-04-25T00:41:21.855527+0000","last_undegraded":"2024-04-25T00:41:21.855527+0000","last_fullsized":"2024-04-25T00:41:21.855527+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:00.898156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728440+0000","last_change":"2024-04-25T00:41:20.729031+0000","last_active":"2024-04-25T00:41:21.728440+0000","last_peered":"2024-04-25T00:41:21.728440+0000","last_clean":"2024-04-25T00:41:21.728440+0000","last_became_active":"2024-04-25T00:41:20.724259+0000","last_became_peered":"2024-04-25T00:41:20.724259+0000","last_unstale":"2024-04-25T00:41:21.728440+0000","last_undegraded":"2024-04-25T00:41:21.728440+0000","last_fullsized":"2024-04-25T00:41:21.728440+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:02.403416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729314+0000","last_change":"2024-04-25T00:41:19.707832+0000","last_active":"2024-04-25T00:41:21.729314+0000","last_peered":"2024-04-25T00:41:21.729314+0000","last_clean":"2024-04-25T00:41:21.729314+0000","last_became_active":"2024-04-25T00:41:19.706600+0000","last_became_peered":"2024-04-25T00:41:19.706600+0000","last_unstale":"2024-04-25T00:41:21.729314+0000","last_undegraded":"2024-04-25T00:41:21.729314+0000","last_fullsized":"2024-04-25T00:41:21.729314+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:09.353188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855887+0000","last_change":"2024-04-25T00:41:20.736978+0000","last_active":"2024-04-25T00:41:21.855887+0000","last_peered":"2024-04-25T00:41:21.855887+0000","last_clean":"2024-04-25T00:41:21.855887+0000","last_became_active":"2024-04-25T00:41:20.736654+0000","last_became_peered":"2024-04-25T00:41:20.736654+0000","last_unstale":"2024-04-25T00:41:21.855887+0000","last_undegraded":"2024-04-25T00:41:21.855887+0000","last_fullsized":"2024-04-25T00:41:21.855887+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:14.858055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854660+0000","last_change":"2024-04-25T00:41:19.724832+0000","last_active":"2024-04-25T00:41:21.854660+0000","last_peered":"2024-04-25T00:41:21.854660+0000","last_clean":"2024-04-25T00:41:21.854660+0000","last_became_active":"2024-04-25T00:41:19.724496+0000","last_became_peered":"2024-04-25T00:41:19.724496+0000","last_unstale":"2024-04-25T00:41:21.854660+0000","last_undegraded":"2024-04-25T00:41:21.854660+0000","last_fullsized":"2024-04-25T00:41:21.854660+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:25.361018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854330+0000","last_change":"2024-04-25T00:41:20.727581+0000","last_active":"2024-04-25T00:41:21.854330+0000","last_peered":"2024-04-25T00:41:21.854330+0000","last_clean":"2024-04-25T00:41:21.854330+0000","last_became_active":"2024-04-25T00:41:20.723303+0000","last_became_peered":"2024-04-25T00:41:20.723303+0000","last_unstale":"2024-04-25T00:41:21.854330+0000","last_undegraded":"2024-04-25T00:41:21.854330+0000","last_fullsized":"2024-04-25T00:41:21.854330+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:55.977495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856037+0000","last_change":"2024-04-25T00:41:19.720256+0000","last_active":"2024-04-25T00:41:21.856037+0000","last_peered":"2024-04-25T00:41:21.856037+0000","last_clean":"2024-04-25T00:41:21.856037+0000","last_became_active":"2024-04-25T00:41:19.719906+0000","last_became_peered":"2024-04-25T00:41:19.719906+0000","last_unstale":"2024-04-25T00:41:21.856037+0000","last_undegraded":"2024-04-25T00:41:21.856037+0000","last_fullsized":"2024-04-25T00:41:21.856037+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:58.692958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853309+0000","last_change":"2024-04-25T00:41:20.727981+0000","last_active":"2024-04-25T00:41:21.853309+0000","last_peered":"2024-04-25T00:41:21.853309+0000","last_clean":"2024-04-25T00:41:21.853309+0000","last_became_active":"2024-04-25T00:41:20.727650+0000","last_became_peered":"2024-04-25T00:41:20.727650+0000","last_unstale":"2024-04-25T00:41:21.853309+0000","last_undegraded":"2024-04-25T00:41:21.853309+0000","last_fullsized":"2024-04-25T00:41:21.853309+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:24.427571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852863+0000","last_change":"2024-04-25T00:41:19.725591+0000","last_active":"2024-04-25T00:41:21.852863+0000","last_peered":"2024-04-25T00:41:21.852863+0000","last_clean":"2024-04-25T00:41:21.852863+0000","last_became_active":"2024-04-25T00:41:19.725213+0000","last_became_peered":"2024-04-25T00:41:19.725213+0000","last_unstale":"2024-04-25T00:41:21.852863+0000","last_undegraded":"2024-04-25T00:41:21.852863+0000","last_fullsized":"2024-04-25T00:41:21.852863+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:35.912276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856172+0000","last_change":"2024-04-25T00:41:20.728184+0000","last_active":"2024-04-25T00:41:21.856172+0000","last_peered":"2024-04-25T00:41:21.856172+0000","last_clean":"2024-04-25T00:41:21.856172+0000","last_became_active":"2024-04-25T00:41:20.726395+0000","last_became_peered":"2024-04-25T00:41:20.726395+0000","last_unstale":"2024-04-25T00:41:21.856172+0000","last_undegraded":"2024-04-25T00:41:21.856172+0000","last_fullsized":"2024-04-25T00:41:21.856172+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:05.020843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737304+0000","last_change":"2024-04-25T00:41:19.726228+0000","last_active":"2024-04-25T00:41:21.737304+0000","last_peered":"2024-04-25T00:41:21.737304+0000","last_clean":"2024-04-25T00:41:21.737304+0000","last_became_active":"2024-04-25T00:41:19.725747+0000","last_became_peered":"2024-04-25T00:41:19.725747+0000","last_unstale":"2024-04-25T00:41:21.737304+0000","last_undegraded":"2024-04-25T00:41:21.737304+0000","last_fullsized":"2024-04-25T00:41:21.737304+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:00.329987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724987+0000","last_change":"2024-04-25T00:41:20.747974+0000","last_active":"2024-04-25T00:41:21.724987+0000","last_peered":"2024-04-25T00:41:21.724987+0000","last_clean":"2024-04-25T00:41:21.724987+0000","last_became_active":"2024-04-25T00:41:20.747202+0000","last_became_peered":"2024-04-25T00:41:20.747202+0000","last_unstale":"2024-04-25T00:41:21.724987+0000","last_undegraded":"2024-04-25T00:41:21.724987+0000","last_fullsized":"2024-04-25T00:41:21.724987+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:52.725891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736508+0000","last_change":"2024-04-25T00:41:19.724386+0000","last_active":"2024-04-25T00:41:21.736508+0000","last_peered":"2024-04-25T00:41:21.736508+0000","last_clean":"2024-04-25T00:41:21.736508+0000","last_became_active":"2024-04-25T00:41:19.724122+0000","last_became_peered":"2024-04-25T00:41:19.724122+0000","last_unstale":"2024-04-25T00:41:21.736508+0000","last_undegraded":"2024-04-25T00:41:21.736508+0000","last_fullsized":"2024-04-25T00:41:21.736508+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:58.585739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852918+0000","last_change":"2024-04-25T00:41:20.751369+0000","last_active":"2024-04-25T00:41:21.852918+0000","last_peered":"2024-04-25T00:41:21.852918+0000","last_clean":"2024-04-25T00:41:21.852918+0000","last_became_active":"2024-04-25T00:41:20.746068+0000","last_became_peered":"2024-04-25T00:41:20.746068+0000","last_unstale":"2024-04-25T00:41:21.852918+0000","last_undegraded":"2024-04-25T00:41:21.852918+0000","last_fullsized":"2024-04-25T00:41:21.852918+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:21.777831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852447+0000","last_change":"2024-04-25T00:41:19.729543+0000","last_active":"2024-04-25T00:41:21.852447+0000","last_peered":"2024-04-25T00:41:21.852447+0000","last_clean":"2024-04-25T00:41:21.852447+0000","last_became_active":"2024-04-25T00:41:19.729146+0000","last_became_peered":"2024-04-25T00:41:19.729146+0000","last_unstale":"2024-04-25T00:41:21.852447+0000","last_undegraded":"2024-04-25T00:41:21.852447+0000","last_fullsized":"2024-04-25T00:41:21.852447+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:56.025389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728090+0000","last_change":"2024-04-25T00:41:20.732303+0000","last_active":"2024-04-25T00:41:21.728090+0000","last_peered":"2024-04-25T00:41:21.728090+0000","last_clean":"2024-04-25T00:41:21.728090+0000","last_became_active":"2024-04-25T00:41:20.727064+0000","last_became_peered":"2024-04-25T00:41:20.727064+0000","last_unstale":"2024-04-25T00:41:21.728090+0000","last_undegraded":"2024-04-25T00:41:21.728090+0000","last_fullsized":"2024-04-25T00:41:21.728090+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:29.644405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856443+0000","last_change":"2024-04-25T00:41:19.711806+0000","last_active":"2024-04-25T00:41:21.856443+0000","last_peered":"2024-04-25T00:41:21.856443+0000","last_clean":"2024-04-25T00:41:21.856443+0000","last_became_active":"2024-04-25T00:41:19.711478+0000","last_became_peered":"2024-04-25T00:41:19.711478+0000","last_unstale":"2024-04-25T00:41:21.856443+0000","last_undegraded":"2024-04-25T00:41:21.856443+0000","last_fullsized":"2024-04-25T00:41:21.856443+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:25.679848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737205+0000","last_change":"2024-04-25T00:41:20.729141+0000","last_active":"2024-04-25T00:41:21.737205+0000","last_peered":"2024-04-25T00:41:21.737205+0000","last_clean":"2024-04-25T00:41:21.737205+0000","last_became_active":"2024-04-25T00:41:20.728844+0000","last_became_peered":"2024-04-25T00:41:20.728844+0000","last_unstale":"2024-04-25T00:41:21.737205+0000","last_undegraded":"2024-04-25T00:41:21.737205+0000","last_fullsized":"2024-04-25T00:41:21.737205+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:09.108727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"168'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729185+0000","last_change":"2024-04-25T00:41:19.726585+0000","last_active":"2024-04-25T00:41:21.729185+0000","last_peered":"2024-04-25T00:41:21.729185+0000","last_clean":"2024-04-25T00:41:21.729185+0000","last_became_active":"2024-04-25T00:41:19.725579+0000","last_became_peered":"2024-04-25T00:41:19.725579+0000","last_unstale":"2024-04-25T00:41:21.729185+0000","last_undegraded":"2024-04-25T00:41:21.729185+0000","last_fullsized":"2024-04-25T00:41:21.729185+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:58.553384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.727539+0000","last_change":"2024-04-25T00:41:20.724729+0000","last_active":"2024-04-25T00:41:21.727539+0000","last_peered":"2024-04-25T00:41:21.727539+0000","last_clean":"2024-04-25T00:41:21.727539+0000","last_became_active":"2024-04-25T00:41:20.723781+0000","last_became_peered":"2024-04-25T00:41:20.723781+0000","last_unstale":"2024-04-25T00:41:21.727539+0000","last_undegraded":"2024-04-25T00:41:21.727539+0000","last_fullsized":"2024-04-25T00:41:21.727539+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:39.356001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854985+0000","last_change":"2024-04-25T00:41:19.713087+0000","last_active":"2024-04-25T00:41:21.854985+0000","last_peered":"2024-04-25T00:41:21.854985+0000","last_clean":"2024-04-25T00:41:21.854985+0000","last_became_active":"2024-04-25T00:41:19.710399+0000","last_became_peered":"2024-04-25T00:41:19.710399+0000","last_unstale":"2024-04-25T00:41:21.854985+0000","last_undegraded":"2024-04-25T00:41:21.854985+0000","last_fullsized":"2024-04-25T00:41:21.854985+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:01.391340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853040+0000","last_change":"2024-04-25T00:41:20.752301+0000","last_active":"2024-04-25T00:41:21.853040+0000","last_peered":"2024-04-25T00:41:21.853040+0000","last_clean":"2024-04-25T00:41:21.853040+0000","last_became_active":"2024-04-25T00:41:20.751497+0000","last_became_peered":"2024-04-25T00:41:20.751497+0000","last_unstale":"2024-04-25T00:41:21.853040+0000","last_undegraded":"2024-04-25T00:41:21.853040+0000","last_fullsized":"2024-04-25T00:41:21.853040+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:50.003262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856627+0000","last_change":"2024-04-25T00:41:19.727532+0000","last_active":"2024-04-25T00:41:21.856627+0000","last_peered":"2024-04-25T00:41:21.856627+0000","last_clean":"2024-04-25T00:41:21.856627+0000","last_became_active":"2024-04-25T00:41:19.727155+0000","last_became_peered":"2024-04-25T00:41:19.727155+0000","last_unstale":"2024-04-25T00:41:21.856627+0000","last_undegraded":"2024-04-25T00:41:21.856627+0000","last_fullsized":"2024-04-25T00:41:21.856627+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:54.985055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737267+0000","last_change":"2024-04-25T00:41:20.756146+0000","last_active":"2024-04-25T00:41:21.737267+0000","last_peered":"2024-04-25T00:41:21.737267+0000","last_clean":"2024-04-25T00:41:21.737267+0000","last_became_active":"2024-04-25T00:41:20.755869+0000","last_became_peered":"2024-04-25T00:41:20.755869+0000","last_unstale":"2024-04-25T00:41:21.737267+0000","last_undegraded":"2024-04-25T00:41:21.737267+0000","last_fullsized":"2024-04-25T00:41:21.737267+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:02.047170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724768+0000","last_change":"2024-04-25T00:41:19.725155+0000","last_active":"2024-04-25T00:41:21.724768+0000","last_peered":"2024-04-25T00:41:21.724768+0000","last_clean":"2024-04-25T00:41:21.724768+0000","last_became_active":"2024-04-25T00:41:19.723795+0000","last_became_peered":"2024-04-25T00:41:19.723795+0000","last_unstale":"2024-04-25T00:41:21.724768+0000","last_undegraded":"2024-04-25T00:41:21.724768+0000","last_fullsized":"2024-04-25T00:41:21.724768+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:17.151602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728540+0000","last_change":"2024-04-25T00:41:20.733781+0000","last_active":"2024-04-25T00:41:21.728540+0000","last_peered":"2024-04-25T00:41:21.728540+0000","last_clean":"2024-04-25T00:41:21.728540+0000","last_became_active":"2024-04-25T00:41:20.726560+0000","last_became_peered":"2024-04-25T00:41:20.726560+0000","last_unstale":"2024-04-25T00:41:21.728540+0000","last_undegraded":"2024-04-25T00:41:21.728540+0000","last_fullsized":"2024-04-25T00:41:21.728540+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:21.667951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853894+0000","last_change":"2024-04-25T00:41:19.722319+0000","last_active":"2024-04-25T00:41:21.853894+0000","last_peered":"2024-04-25T00:41:21.853894+0000","last_clean":"2024-04-25T00:41:21.853894+0000","last_became_active":"2024-04-25T00:41:19.721932+0000","last_became_peered":"2024-04-25T00:41:19.721932+0000","last_unstale":"2024-04-25T00:41:21.853894+0000","last_undegraded":"2024-04-25T00:41:21.853894+0000","last_fullsized":"2024-04-25T00:41:21.853894+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:44.151050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855598+0000","last_change":"2024-04-25T00:41:20.735352+0000","last_active":"2024-04-25T00:41:21.855598+0000","last_peered":"2024-04-25T00:41:21.855598+0000","last_clean":"2024-04-25T00:41:21.855598+0000","last_became_active":"2024-04-25T00:41:20.733136+0000","last_became_peered":"2024-04-25T00:41:20.733136+0000","last_unstale":"2024-04-25T00:41:21.855598+0000","last_undegraded":"2024-04-25T00:41:21.855598+0000","last_fullsized":"2024-04-25T00:41:21.855598+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:45.237690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852820+0000","last_change":"2024-04-25T00:41:19.732832+0000","last_active":"2024-04-25T00:41:21.852820+0000","last_peered":"2024-04-25T00:41:21.852820+0000","last_clean":"2024-04-25T00:41:21.852820+0000","last_became_active":"2024-04-25T00:41:19.731714+0000","last_became_peered":"2024-04-25T00:41:19.731714+0000","last_unstale":"2024-04-25T00:41:21.852820+0000","last_undegraded":"2024-04-25T00:41:21.852820+0000","last_fullsized":"2024-04-25T00:41:21.852820+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:44.151050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725714+0000","last_change":"2024-04-25T00:41:20.717857+0000","last_active":"2024-04-25T00:41:21.725714+0000","last_peered":"2024-04-25T00:41:21.725714+0000","last_clean":"2024-04-25T00:41:21.725714+0000","last_became_active":"2024-04-25T00:41:20.717105+0000","last_became_peered":"2024-04-25T00:41:20.717105+0000","last_unstale":"2024-04-25T00:41:21.725714+0000","last_undegraded":"2024-04-25T00:41:21.725714+0000","last_fullsized":"2024-04-25T00:41:21.725714+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:59.739712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.726474+0000","last_change":"2024-04-25T00:41:19.714001+0000","last_active":"2024-04-25T00:41:21.726474+0000","last_peered":"2024-04-25T00:41:21.726474+0000","last_clean":"2024-04-25T00:41:21.726474+0000","last_became_active":"2024-04-25T00:41:19.713267+0000","last_became_peered":"2024-04-25T00:41:19.713267+0000","last_unstale":"2024-04-25T00:41:21.726474+0000","last_undegraded":"2024-04-25T00:41:21.726474+0000","last_fullsized":"2024-04-25T00:41:21.726474+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:46.563704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737058+0000","last_change":"2024-04-25T00:41:20.726216+0000","last_active":"2024-04-25T00:41:21.737058+0000","last_peered":"2024-04-25T00:41:21.737058+0000","last_clean":"2024-04-25T00:41:21.737058+0000","last_became_active":"2024-04-25T00:41:20.725564+0000","last_became_peered":"2024-04-25T00:41:20.725564+0000","last_unstale":"2024-04-25T00:41:21.737058+0000","last_undegraded":"2024-04-25T00:41:21.737058+0000","last_fullsized":"2024-04-25T00:41:21.737058+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:27.898758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728110+0000","last_change":"2024-04-25T00:41:19.729890+0000","last_active":"2024-04-25T00:41:21.728110+0000","last_peered":"2024-04-25T00:41:21.728110+0000","last_clean":"2024-04-25T00:41:21.728110+0000","last_became_active":"2024-04-25T00:41:19.728730+0000","last_became_peered":"2024-04-25T00:41:19.728730+0000","last_unstale":"2024-04-25T00:41:21.728110+0000","last_undegraded":"2024-04-25T00:41:21.728110+0000","last_fullsized":"2024-04-25T00:41:21.728110+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:38.343926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":489,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724956+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:41:21.724956+0000","last_peered":"2024-04-25T00:41:21.724956+0000","last_clean":"2024-04-25T00:41:21.724956+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:41:21.724956+0000","last_undegraded":"2024-04-25T00:41:21.724956+0000","last_fullsized":"2024-04-25T00:41:21.724956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38004,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333836,"statfs":{"total":96636764160,"available":96597848064,"internally_reserved":0,"allocated":2727936,"data_stored":1529742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574972,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37416,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334424,"statfs":{"total":96636764160,"available":96598450176,"internally_reserved":0,"allocated":2125824,"data_stored":938344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574972,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37420,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334420,"statfs":{"total":96636764160,"available":96598446080,"internally_reserved":0,"allocated":2125824,"data_stored":938344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574971,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41600,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330240,"statfs":{"total":96636764160,"available":96594165760,"internally_reserved":0,"allocated":2142208,"data_stored":939478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574972,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41588,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330252,"statfs":{"total":96636764160,"available":96594178048,"internally_reserved":0,"allocated":2134016,"data_stored":937944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37416,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334424,"statfs":{"total":96636764160,"available":96598450176,"internally_reserved":0,"allocated":2121728,"data_stored":937762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2129920,"data_stored":937818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574971,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42156,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329684,"statfs":{"total":96636764160,"available":96593596416,"internally_reserved":0,"allocated":2719744,"data_stored":1528220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:41:25.518 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:41:25.659 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-04-25T00:41:25.656+0000 7f40d112b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:41:25.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.680+0000 7f7d10fa3640 1 Processor -- start 2024-04-25T00:41:25.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.680+0000 7f7d10fa3640 1 -- start start 2024-04-25T00:41:25.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.680+0000 7f7d10fa3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d0c15f3c0 0x7f7d0c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.680+0000 7f7d10fa3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d0c0b3720 con 0x7f7d0c0b4780 2024-04-25T00:41:25.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.680+0000 7f7d10fa3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d0c0b39c0 con 0x7f7d0c15e8d0 2024-04-25T00:41:25.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.680+0000 7f7d10fa3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7d0c0b42f0 con 0x7f7d0c15f3c0 2024-04-25T00:41:25.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.680+0000 7f7d09d9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7d0c0b4780 0x7f7d0c15c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:51612/0 (socket says 172.21.15.29:51612) 2024-04-25T00:41:25.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.680+0000 7f7d09d9b640 1 -- 172.21.15.29:0/2993909241 learned_addr learned my addr 172.21.15.29:0/2993909241 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:25.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0ad9d640 1 --2- 172.21.15.29:0/2993909241 >> v2:172.21.15.157:3300/0 conn(0x7f7d0c15f3c0 0x7f7d0c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3564114775 0 0) 0x7f7d0c0b3720 con 0x7f7d0c0b4780 2024-04-25T00:41:25.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cf8003530 con 0x7f7d0c0b4780 2024-04-25T00:41:25.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 30840970 0 0) 0x7f7cf8003530 con 0x7f7d0c0b4780 2024-04-25T00:41:25.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 >> v2:172.21.15.157:3300/0 conn(0x7f7d0c15f3c0 msgr2=0x7f7d0c0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:25.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 --2- 172.21.15.29:0/2993909241 >> v2:172.21.15.157:3300/0 conn(0x7f7d0c15f3c0 0x7f7d0c0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 >> v1:172.21.15.157:6789/0 conn(0x7f7d0c15e8d0 legacy=0x7f7d0c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c0b3ca0 con 0x7f7d0c0b4780 2024-04-25T00:41:25.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f7cfc001680 con 0x7f7d0c0b4780 2024-04-25T00:41:25.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 >> v1:172.21.15.29:6789/0 conn(0x7f7d0c0b4780 legacy=0x7f7d0c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 --2- 172.21.15.29:0/2993909241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf80039c0 0x7f7cf8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 --2- 172.21.15.29:0/2993909241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf8004810 0x7f7cf8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 --2- 172.21.15.29:0/2993909241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cf80051d0 0x7f7cf800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c0b42f0 con 0x7f7cf80051d0 2024-04-25T00:41:25.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf800c070 con 0x7f7cf80039c0 2024-04-25T00:41:25.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf800c270 con 0x7f7cf8004810 2024-04-25T00:41:25.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0ad9d640 1 --2- 172.21.15.29:0/2993909241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cf80051d0 0x7f7cf800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0a59c640 1 --2- 172.21.15.29:0/2993909241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf80039c0 0x7f7cf8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d09d9b640 1 --2- 172.21.15.29:0/2993909241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf8004810 0x7f7cf8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0ad9d640 1 -- 172.21.15.29:0/2993909241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf8004810 msgr2=0x7f7cf8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:25.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0ad9d640 1 --2- 172.21.15.29:0/2993909241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf8004810 0x7f7cf8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0ad9d640 1 -- 172.21.15.29:0/2993909241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf80039c0 msgr2=0x7f7cf8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:25.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0ad9d640 1 --2- 172.21.15.29:0/2993909241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf80039c0 0x7f7cf8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0ad9d640 1 -- 172.21.15.29:0/2993909241 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf8003530 con 0x7f7cf80051d0 2024-04-25T00:41:25.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0ad9d640 1 -- 172.21.15.29:0/2993909241 reap_dead start 2024-04-25T00:41:25.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0ad9d640 1 --2- 172.21.15.29:0/2993909241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cf80051d0 0x7f7cf800b9d0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009ac0 tx=0x7f7cf401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2049f36d2d1eeb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:25.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4009690 con 0x7f7cf80051d0 2024-04-25T00:41:25.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf40a5680 con 0x7f7cf80051d0 2024-04-25T00:41:25.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.684+0000 7f7d0959a640 1 -- 172.21.15.29:0/2993909241 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40a5a50 con 0x7f7cf80051d0 2024-04-25T00:41:25.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 -- 172.21.15.29:0/2993909241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cf80051d0 msgr2=0x7f7cf800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 --2- 172.21.15.29:0/2993909241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cf80051d0 0x7f7cf800b9d0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009ac0 tx=0x7f7cf401ca90 comp rx=0 tx=0).stop 2024-04-25T00:41:25.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 -- 172.21.15.29:0/2993909241 shutdown_connections 2024-04-25T00:41:25.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 --2- 172.21.15.29:0/2993909241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cf80051d0 0x7f7cf800b9d0 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 -- 172.21.15.29:0/2993909241 >> 172.21.15.29:0/2993909241 conn(0x7f7d0c0b1870 msgr2=0x7f7d0c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:25.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 -- 172.21.15.29:0/2993909241 shutdown_connections 2024-04-25T00:41:25.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 -- 172.21.15.29:0/2993909241 wait complete. 2024-04-25T00:41:25.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 Processor -- start 2024-04-25T00:41:25.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 -- start start 2024-04-25T00:41:25.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf80051d0 0x7f7d0c1380b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d0c138d40 0x7f7d0c1385f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d0c1f7990 0x7f7d0c1f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c06cf10 con 0x7f7d0c138d40 2024-04-25T00:41:25.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c0b3580 con 0x7f7d0c1f7990 2024-04-25T00:41:25.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c144a20 con 0x7f7cf80051d0 2024-04-25T00:41:25.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d09d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d0c138d40 0x7f7d0c1385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d09d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d0c138d40 0x7f7d0c1385f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46594/0 (socket says 172.21.15.29:46594) 2024-04-25T00:41:25.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d09d9b640 1 -- 172.21.15.29:0/718145486 learned_addr learned my addr 172.21.15.29:0/718145486 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:25.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d0ad9d640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d0c1f7990 0x7f7d0c1f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d0a59c640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf80051d0 0x7f7d0c1380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d09d9b640 1 -- 172.21.15.29:0/718145486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf80051d0 msgr2=0x7f7d0c1380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d09d9b640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf80051d0 0x7f7d0c1380b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d09d9b640 1 -- 172.21.15.29:0/718145486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d0c1f7990 msgr2=0x7f7d0c1f7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d09d9b640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d0c1f7990 0x7f7d0c1f7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d09d9b640 1 -- 172.21.15.29:0/718145486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c15c4b0 con 0x7f7d0c138d40 2024-04-25T00:41:25.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d09d9b640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d0c138d40 0x7f7d0c1385f0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00e7b0 tx=0x7f7cfc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:25.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d02ffd640 1 -- 172.21.15.29:0/718145486 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc00c870 con 0x7f7d0c138d40 2024-04-25T00:41:25.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d02ffd640 1 -- 172.21.15.29:0/718145486 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc018070 con 0x7f7d0c138d40 2024-04-25T00:41:25.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 -- 172.21.15.29:0/718145486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0c1442a0 con 0x7f7d0c138d40 2024-04-25T00:41:25.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d02ffd640 1 -- 172.21.15.29:0/718145486 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc00cc50 con 0x7f7d0c138d40 2024-04-25T00:41:25.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 -- 172.21.15.29:0/718145486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0c1413b0 con 0x7f7d0c138d40 2024-04-25T00:41:25.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.688+0000 7f7d10fa3640 1 -- 172.21.15.29:0/718145486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0c069270 con 0x7f7d0c138d40 2024-04-25T00:41:25.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.692+0000 7f7d02ffd640 1 -- 172.21.15.29:0/718145486 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f7cfc01d460 con 0x7f7d0c138d40 2024-04-25T00:41:25.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.692+0000 7f7d02ffd640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7cec068250 0x7f7cec06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:25.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.692+0000 7f7d02ffd640 1 -- 172.21.15.29:0/718145486 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f7cfc084370 con 0x7f7d0c138d40 2024-04-25T00:41:25.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.692+0000 7f7d0a59c640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7cec068250 0x7f7cec06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:25.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.692+0000 7f7d0a59c640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7cec068250 0x7f7cec06a710 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7d04003ea0 tx=0x7f7d04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:25.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.696+0000 7f7d02ffd640 1 -- 172.21.15.29:0/718145486 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7cfc056ab0 con 0x7f7d0c138d40 2024-04-25T00:41:25.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.920+0000 7f7d10fa3640 1 -- 172.21.15.29:0/718145486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7d0c15e390 con 0x7f7d0c138d40 2024-04-25T00:41:25.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.920+0000 7f7d02ffd640 1 -- 172.21.15.29:0/718145486 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1438 (secure 0 0 0) 0x7f7cfc0563b0 con 0x7f7d0c138d40 2024-04-25T00:41:25.922 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:25.922 INFO:teuthology.orchestra.run.smithi029.stdout:{"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":613,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":168,"num_osds":8,"num_up_osds":8,"osd_up_since":1714005083,"num_in_osds":8,"osd_in_since":1714005074,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":320540672,"bytes_avail":772773572608,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":112,"id":13,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":13,"rank":0,"name":"c","status":"up:active","gid":10578}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-25T00:41:03.414909+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:41:25.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.924+0000 7f7d10fa3640 1 -- 172.21.15.29:0/718145486 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7cec068250 msgr2=0x7f7cec06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.924+0000 7f7d10fa3640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7cec068250 0x7f7cec06a710 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7d04003ea0 tx=0x7f7d04009040 comp rx=0 tx=0).stop 2024-04-25T00:41:25.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.924+0000 7f7d10fa3640 1 -- 172.21.15.29:0/718145486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d0c138d40 msgr2=0x7f7d0c1385f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:25.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.924+0000 7f7d10fa3640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d0c138d40 0x7f7d0c1385f0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00e7b0 tx=0x7f7cfc00ec70 comp rx=0 tx=0).stop 2024-04-25T00:41:25.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.924+0000 7f7d10fa3640 1 -- 172.21.15.29:0/718145486 shutdown_connections 2024-04-25T00:41:25.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.924+0000 7f7d10fa3640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7cec068250 0x7f7cec06a710 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.924+0000 7f7d10fa3640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d0c1f7990 0x7f7d0c1f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.924+0000 7f7d10fa3640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d0c138d40 0x7f7d0c1385f0 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.924+0000 7f7d10fa3640 1 --2- 172.21.15.29:0/718145486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf80051d0 0x7f7d0c1380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:25.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.924+0000 7f7d10fa3640 1 -- 172.21.15.29:0/718145486 >> 172.21.15.29:0/718145486 conn(0x7f7d0c0b1870 msgr2=0x7f7d0c15d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:25.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.924+0000 7f7d10fa3640 1 -- 172.21.15.29:0/718145486 shutdown_connections 2024-04-25T00:41:25.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:25.924+0000 7f7d10fa3640 1 -- 172.21.15.29:0/718145486 wait complete. 2024-04-25T00:41:25.941 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T00:41:25.941 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:41:26.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95e1e41640 1 Processor -- start 2024-04-25T00:41:26.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95e1e41640 1 -- start start 2024-04-25T00:41:26.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95e1e41640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95dc15e170 0x7f95dc161d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:26.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95e1e41640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc15e550 0x7f95dc15f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:26.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95e1e41640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f95dc15f570 0x7f95dc06b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:26.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95e1e41640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f95dc1622b0 con 0x7f95dc15f570 2024-04-25T00:41:26.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95e1e41640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f95dc162580 con 0x7f95dc15e170 2024-04-25T00:41:26.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95e1e41640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f95dc162860 con 0x7f95dc15e550 2024-04-25T00:41:26.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95dbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f95dc15f570 0x7f95dc06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:26.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95dbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f95dc15f570 0x7f95dc06b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46610/0 (socket says 172.21.15.29:46610) 2024-04-25T00:41:26.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95dbfff640 1 -- 172.21.15.29:0/3792102024 learned_addr learned my addr 172.21.15.29:0/3792102024 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:26.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95daffd640 1 --2- 172.21.15.29:0/3792102024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc15e550 0x7f95dc15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:26.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95db7fe640 1 --2- 172.21.15.29:0/3792102024 >> v2:172.21.15.157:3300/0 conn(0x7f95dc15e170 0x7f95dc161d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:26.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95dbfff640 1 -- 172.21.15.29:0/3792102024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc15e550 msgr2=0x7f95dc15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:26.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95dbfff640 1 --2- 172.21.15.29:0/3792102024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc15e550 0x7f95dc15f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:26.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95dbfff640 1 -- 172.21.15.29:0/3792102024 >> v2:172.21.15.157:3300/0 conn(0x7f95dc15e170 msgr2=0x7f95dc161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:26.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95dbfff640 1 --2- 172.21.15.29:0/3792102024 >> v2:172.21.15.157:3300/0 conn(0x7f95dc15e170 0x7f95dc161d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:26.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95dbfff640 1 -- 172.21.15.29:0/3792102024 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95dc162b40 con 0x7f95dc15f570 2024-04-25T00:41:26.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.060+0000 7f95db7fe640 1 --2- 172.21.15.29:0/3792102024 >> v2:172.21.15.157:3300/0 conn(0x7f95dc15e170 0x7f95dc161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:26.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95dbfff640 1 --2- 172.21.15.29:0/3792102024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95dc15f570 0x7f95dc06b210 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0d1d30 tx=0x7f95d401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b26abd7231d504c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:26.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95da7fc640 1 -- 172.21.15.29:0/3792102024 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d40a5750 con 0x7f95dc15f570 2024-04-25T00:41:26.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95da7fc640 1 -- 172.21.15.29:0/3792102024 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d40a58f0 con 0x7f95dc15f570 2024-04-25T00:41:26.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95da7fc640 1 -- 172.21.15.29:0/3792102024 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d40a5cc0 con 0x7f95dc15f570 2024-04-25T00:41:26.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 -- 172.21.15.29:0/3792102024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95dc15f570 msgr2=0x7f95dc06b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:26.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 --2- 172.21.15.29:0/3792102024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95dc15f570 0x7f95dc06b210 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0d1d30 tx=0x7f95d401c9c0 comp rx=0 tx=0).stop 2024-04-25T00:41:26.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 -- 172.21.15.29:0/3792102024 shutdown_connections 2024-04-25T00:41:26.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 --2- 172.21.15.29:0/3792102024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95dc15f570 0x7f95dc06b210 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:26.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 --2- 172.21.15.29:0/3792102024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc15e550 0x7f95dc15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:26.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 --2- 172.21.15.29:0/3792102024 >> v2:172.21.15.157:3300/0 conn(0x7f95dc15e170 0x7f95dc161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:26.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 -- 172.21.15.29:0/3792102024 >> 172.21.15.29:0/3792102024 conn(0x7f95dc0d0c30 msgr2=0x7f95dc0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:26.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 -- 172.21.15.29:0/3792102024 shutdown_connections 2024-04-25T00:41:26.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 -- 172.21.15.29:0/3792102024 wait complete. 2024-04-25T00:41:26.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 Processor -- start 2024-04-25T00:41:26.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 -- start start 2024-04-25T00:41:26.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95dc15e170 0x7f95dc14e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:26.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95dc15e550 0x7f95dc15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:26.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc15da30 0x7f95dc06e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:26.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc071df0 con 0x7f95dc15e170 2024-04-25T00:41:26.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc0c6720 con 0x7f95dc15e550 2024-04-25T00:41:26.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.064+0000 7f95e1e41640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f95dc071f70 con 0x7f95dc15da30 2024-04-25T00:41:26.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95db7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95dc15e170 0x7f95dc14e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:26.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95db7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95dc15e170 0x7f95dc14e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46626/0 (socket says 172.21.15.29:46626) 2024-04-25T00:41:26.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95db7fe640 1 -- 172.21.15.29:0/803463097 learned_addr learned my addr 172.21.15.29:0/803463097 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:26.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95daffd640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95dc15e550 0x7f95dc15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:26.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95dbfff640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc15da30 0x7f95dc06e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:26.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95db7fe640 1 -- 172.21.15.29:0/803463097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc15da30 msgr2=0x7f95dc06e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:26.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95db7fe640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc15da30 0x7f95dc06e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:26.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95db7fe640 1 -- 172.21.15.29:0/803463097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95dc15e550 msgr2=0x7f95dc15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:26.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95db7fe640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95dc15e550 0x7f95dc15d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:26.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95db7fe640 1 -- 172.21.15.29:0/803463097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95dc14e5e0 con 0x7f95dc15e170 2024-04-25T00:41:26.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95db7fe640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95dc15e170 0x7f95dc14e0a0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00c7c0 tx=0x7f95cc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:26.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95c3fff640 1 -- 172.21.15.29:0/803463097 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc015070 con 0x7f95dc15e170 2024-04-25T00:41:26.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95c3fff640 1 -- 172.21.15.29:0/803463097 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95cc007a30 con 0x7f95dc15e170 2024-04-25T00:41:26.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95e1e41640 1 -- 172.21.15.29:0/803463097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95dc14fad0 con 0x7f95dc15e170 2024-04-25T00:41:26.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95c3fff640 1 -- 172.21.15.29:0/803463097 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc007e00 con 0x7f95dc15e170 2024-04-25T00:41:26.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95e1e41640 1 -- 172.21.15.29:0/803463097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95dc14ed10 con 0x7f95dc15e170 2024-04-25T00:41:26.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95e1e41640 1 -- 172.21.15.29:0/803463097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95dc069270 con 0x7f95dc15e170 2024-04-25T00:41:26.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95c3fff640 1 -- 172.21.15.29:0/803463097 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f95cc018070 con 0x7f95dc15e170 2024-04-25T00:41:26.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95c3fff640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f95c8068250 0x7f95c806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:26.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95daffd640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f95c8068250 0x7f95c806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:26.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95c3fff640 1 -- 172.21.15.29:0/803463097 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f95cc084310 con 0x7f95dc15e170 2024-04-25T00:41:26.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.068+0000 7f95daffd640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f95c8068250 0x7f95c806a710 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f95dc158e10 tx=0x7f95d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:26.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.072+0000 7f95c3fff640 1 -- 172.21.15.29:0/803463097 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f95dc14e5e0 con 0x7f95dc15e170 2024-04-25T00:41:26.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.232+0000 7f95e1e41640 1 -- 172.21.15.29:0/803463097 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95dc14e7e0 con 0x7f95c8068250 2024-04-25T00:41:26.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.252+0000 7f95c3fff640 1 -- 172.21.15.29:0/803463097 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252492 (secure 0 0 0) 0x7f95dc14e7e0 con 0x7f95c8068250 2024-04-25T00:41:26.255 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:26.256 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:41:26.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.256+0000 7f95e1e41640 1 -- 172.21.15.29:0/803463097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f95c8068250 msgr2=0x7f95c806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:26.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.256+0000 7f95e1e41640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f95c8068250 0x7f95c806a710 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f95dc158e10 tx=0x7f95d0008040 comp rx=0 tx=0).stop 2024-04-25T00:41:26.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.256+0000 7f95e1e41640 1 -- 172.21.15.29:0/803463097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95dc15e170 msgr2=0x7f95dc14e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:26.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.256+0000 7f95e1e41640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95dc15e170 0x7f95dc14e0a0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00c7c0 tx=0x7f95cc00cc80 comp rx=0 tx=0).stop 2024-04-25T00:41:26.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.256+0000 7f95e1e41640 1 -- 172.21.15.29:0/803463097 shutdown_connections 2024-04-25T00:41:26.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.256+0000 7f95e1e41640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f95c8068250 0x7f95c806a710 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:26.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.256+0000 7f95e1e41640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95dc15da30 0x7f95dc06e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:26.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.256+0000 7f95e1e41640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95dc15e550 0x7f95dc15d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:26.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.256+0000 7f95e1e41640 1 --2- 172.21.15.29:0/803463097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95dc15e170 0x7f95dc14e0a0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:26.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.256+0000 7f95e1e41640 1 -- 172.21.15.29:0/803463097 >> 172.21.15.29:0/803463097 conn(0x7f95dc0d0c30 msgr2=0x7f95dc066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:26.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.256+0000 7f95e1e41640 1 -- 172.21.15.29:0/803463097 shutdown_connections 2024-04-25T00:41:26.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:26.256+0000 7f95e1e41640 1 -- 172.21.15.29:0/803463097 wait complete. 2024-04-25T00:41:26.273 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-04-25T00:41:25.421054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":313028,"kb_used_data":17800,"kb_used_omap":12,"kb_used_meta":295091,"kb_avail":754661692,"statfs":{"total":773094113280,"available":772773572608,"internally_reserved":0,"allocated":18227200,"data_stored":8687652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":302173744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001699"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736869+0000","last_change":"2024-04-25T00:41:20.738051+0000","last_active":"2024-04-25T00:41:21.736869+0000","last_peered":"2024-04-25T00:41:21.736869+0000","last_clean":"2024-04-25T00:41:21.736869+0000","last_became_active":"2024-04-25T00:41:20.737775+0000","last_became_peered":"2024-04-25T00:41:20.737775+0000","last_unstale":"2024-04-25T00:41:21.736869+0000","last_undegraded":"2024-04-25T00:41:21.736869+0000","last_fullsized":"2024-04-25T00:41:21.736869+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:00.559458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728319+0000","last_change":"2024-04-25T00:41:20.754563+0000","last_active":"2024-04-25T00:41:21.728319+0000","last_peered":"2024-04-25T00:41:21.728319+0000","last_clean":"2024-04-25T00:41:21.728319+0000","last_became_active":"2024-04-25T00:41:20.753984+0000","last_became_peered":"2024-04-25T00:41:20.753984+0000","last_unstale":"2024-04-25T00:41:21.728319+0000","last_undegraded":"2024-04-25T00:41:21.728319+0000","last_fullsized":"2024-04-25T00:41:21.728319+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:01.910232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856753+0000","last_change":"2024-04-25T00:41:20.722575+0000","last_active":"2024-04-25T00:41:21.856753+0000","last_peered":"2024-04-25T00:41:21.856753+0000","last_clean":"2024-04-25T00:41:21.856753+0000","last_became_active":"2024-04-25T00:41:20.722097+0000","last_became_peered":"2024-04-25T00:41:20.722097+0000","last_unstale":"2024-04-25T00:41:21.856753+0000","last_undegraded":"2024-04-25T00:41:21.856753+0000","last_fullsized":"2024-04-25T00:41:21.856753+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:59.739712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724539+0000","last_change":"2024-04-25T00:41:20.754229+0000","last_active":"2024-04-25T00:41:21.724539+0000","last_peered":"2024-04-25T00:41:21.724539+0000","last_clean":"2024-04-25T00:41:21.724539+0000","last_became_active":"2024-04-25T00:41:20.753591+0000","last_became_peered":"2024-04-25T00:41:20.753591+0000","last_unstale":"2024-04-25T00:41:21.724539+0000","last_undegraded":"2024-04-25T00:41:21.724539+0000","last_fullsized":"2024-04-25T00:41:21.724539+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:14.056858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855011+0000","last_change":"2024-04-25T00:41:20.729817+0000","last_active":"2024-04-25T00:41:21.855011+0000","last_peered":"2024-04-25T00:41:21.855011+0000","last_clean":"2024-04-25T00:41:21.855011+0000","last_became_active":"2024-04-25T00:41:20.729299+0000","last_became_peered":"2024-04-25T00:41:20.729299+0000","last_unstale":"2024-04-25T00:41:21.855011+0000","last_undegraded":"2024-04-25T00:41:21.855011+0000","last_fullsized":"2024-04-25T00:41:21.855011+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:50.003262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853047+0000","last_change":"2024-04-25T00:41:20.738031+0000","last_active":"2024-04-25T00:41:21.853047+0000","last_peered":"2024-04-25T00:41:21.853047+0000","last_clean":"2024-04-25T00:41:21.853047+0000","last_became_active":"2024-04-25T00:41:20.737694+0000","last_became_peered":"2024-04-25T00:41:20.737694+0000","last_unstale":"2024-04-25T00:41:21.853047+0000","last_undegraded":"2024-04-25T00:41:21.853047+0000","last_fullsized":"2024-04-25T00:41:21.853047+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:12.354642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736461+0000","last_change":"2024-04-25T00:41:20.756463+0000","last_active":"2024-04-25T00:41:21.736461+0000","last_peered":"2024-04-25T00:41:21.736461+0000","last_clean":"2024-04-25T00:41:21.736461+0000","last_became_active":"2024-04-25T00:41:20.755935+0000","last_became_peered":"2024-04-25T00:41:20.755935+0000","last_unstale":"2024-04-25T00:41:21.736461+0000","last_undegraded":"2024-04-25T00:41:21.736461+0000","last_fullsized":"2024-04-25T00:41:21.736461+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:50.605977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852627+0000","last_change":"2024-04-25T00:41:20.730367+0000","last_active":"2024-04-25T00:41:21.852627+0000","last_peered":"2024-04-25T00:41:21.852627+0000","last_clean":"2024-04-25T00:41:21.852627+0000","last_became_active":"2024-04-25T00:41:20.729765+0000","last_became_peered":"2024-04-25T00:41:20.729765+0000","last_unstale":"2024-04-25T00:41:21.852627+0000","last_undegraded":"2024-04-25T00:41:21.852627+0000","last_fullsized":"2024-04-25T00:41:21.852627+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:29.081738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728881+0000","last_change":"2024-04-25T00:41:20.731164+0000","last_active":"2024-04-25T00:41:21.728881+0000","last_peered":"2024-04-25T00:41:21.728881+0000","last_clean":"2024-04-25T00:41:21.728881+0000","last_became_active":"2024-04-25T00:41:20.729720+0000","last_became_peered":"2024-04-25T00:41:20.729720+0000","last_unstale":"2024-04-25T00:41:21.728881+0000","last_undegraded":"2024-04-25T00:41:21.728881+0000","last_fullsized":"2024-04-25T00:41:21.728881+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:54.493042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853480+0000","last_change":"2024-04-25T00:41:20.728265+0000","last_active":"2024-04-25T00:41:21.853480+0000","last_peered":"2024-04-25T00:41:21.853480+0000","last_clean":"2024-04-25T00:41:21.853480+0000","last_became_active":"2024-04-25T00:41:20.727840+0000","last_became_peered":"2024-04-25T00:41:20.727840+0000","last_unstale":"2024-04-25T00:41:21.853480+0000","last_undegraded":"2024-04-25T00:41:21.853480+0000","last_fullsized":"2024-04-25T00:41:21.853480+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:25.159973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856794+0000","last_change":"2024-04-25T00:41:20.733820+0000","last_active":"2024-04-25T00:41:21.856794+0000","last_peered":"2024-04-25T00:41:21.856794+0000","last_clean":"2024-04-25T00:41:21.856794+0000","last_became_active":"2024-04-25T00:41:20.733516+0000","last_became_peered":"2024-04-25T00:41:20.733516+0000","last_unstale":"2024-04-25T00:41:21.856794+0000","last_undegraded":"2024-04-25T00:41:21.856794+0000","last_fullsized":"2024-04-25T00:41:21.856794+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:47.575779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.727368+0000","last_change":"2024-04-25T00:41:20.733108+0000","last_active":"2024-04-25T00:41:21.727368+0000","last_peered":"2024-04-25T00:41:21.727368+0000","last_clean":"2024-04-25T00:41:21.727368+0000","last_became_active":"2024-04-25T00:41:20.728236+0000","last_became_peered":"2024-04-25T00:41:20.728236+0000","last_unstale":"2024-04-25T00:41:21.727368+0000","last_undegraded":"2024-04-25T00:41:21.727368+0000","last_fullsized":"2024-04-25T00:41:21.727368+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:57.396464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.727972+0000","last_change":"2024-04-25T00:41:20.726309+0000","last_active":"2024-04-25T00:41:21.727972+0000","last_peered":"2024-04-25T00:41:21.727972+0000","last_clean":"2024-04-25T00:41:21.727972+0000","last_became_active":"2024-04-25T00:41:20.725444+0000","last_became_peered":"2024-04-25T00:41:20.725444+0000","last_unstale":"2024-04-25T00:41:21.727972+0000","last_undegraded":"2024-04-25T00:41:21.727972+0000","last_fullsized":"2024-04-25T00:41:21.727972+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:41.641469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736632+0000","last_change":"2024-04-25T00:41:20.727682+0000","last_active":"2024-04-25T00:41:21.736632+0000","last_peered":"2024-04-25T00:41:21.736632+0000","last_clean":"2024-04-25T00:41:21.736632+0000","last_became_active":"2024-04-25T00:41:20.727409+0000","last_became_peered":"2024-04-25T00:41:20.727409+0000","last_unstale":"2024-04-25T00:41:21.736632+0000","last_undegraded":"2024-04-25T00:41:21.736632+0000","last_fullsized":"2024-04-25T00:41:21.736632+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:48.999599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736447+0000","last_change":"2024-04-25T00:41:20.725702+0000","last_active":"2024-04-25T00:41:21.736447+0000","last_peered":"2024-04-25T00:41:21.736447+0000","last_clean":"2024-04-25T00:41:21.736447+0000","last_became_active":"2024-04-25T00:41:20.725313+0000","last_became_peered":"2024-04-25T00:41:20.725313+0000","last_unstale":"2024-04-25T00:41:21.736447+0000","last_undegraded":"2024-04-25T00:41:21.736447+0000","last_fullsized":"2024-04-25T00:41:21.736447+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:05.336967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852398+0000","last_change":"2024-04-25T00:41:20.734076+0000","last_active":"2024-04-25T00:41:21.852398+0000","last_peered":"2024-04-25T00:41:21.852398+0000","last_clean":"2024-04-25T00:41:21.852398+0000","last_became_active":"2024-04-25T00:41:20.733757+0000","last_became_peered":"2024-04-25T00:41:20.733757+0000","last_unstale":"2024-04-25T00:41:21.852398+0000","last_undegraded":"2024-04-25T00:41:21.852398+0000","last_fullsized":"2024-04-25T00:41:21.852398+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:16.226179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737048+0000","last_change":"2024-04-25T00:41:20.732582+0000","last_active":"2024-04-25T00:41:21.737048+0000","last_peered":"2024-04-25T00:41:21.737048+0000","last_clean":"2024-04-25T00:41:21.737048+0000","last_became_active":"2024-04-25T00:41:20.732276+0000","last_became_peered":"2024-04-25T00:41:20.732276+0000","last_unstale":"2024-04-25T00:41:21.737048+0000","last_undegraded":"2024-04-25T00:41:21.737048+0000","last_fullsized":"2024-04-25T00:41:21.737048+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:44.801392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853934+0000","last_change":"2024-04-25T00:41:20.729067+0000","last_active":"2024-04-25T00:41:21.853934+0000","last_peered":"2024-04-25T00:41:21.853934+0000","last_clean":"2024-04-25T00:41:21.853934+0000","last_became_active":"2024-04-25T00:41:20.728765+0000","last_became_peered":"2024-04-25T00:41:20.728765+0000","last_unstale":"2024-04-25T00:41:21.853934+0000","last_undegraded":"2024-04-25T00:41:21.853934+0000","last_fullsized":"2024-04-25T00:41:21.853934+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:31.971109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854502+0000","last_change":"2024-04-25T00:41:20.728640+0000","last_active":"2024-04-25T00:41:21.854502+0000","last_peered":"2024-04-25T00:41:21.854502+0000","last_clean":"2024-04-25T00:41:21.854502+0000","last_became_active":"2024-04-25T00:41:20.728290+0000","last_became_peered":"2024-04-25T00:41:20.728290+0000","last_unstale":"2024-04-25T00:41:21.854502+0000","last_undegraded":"2024-04-25T00:41:21.854502+0000","last_fullsized":"2024-04-25T00:41:21.854502+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:05.336967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855556+0000","last_change":"2024-04-25T00:41:20.729925+0000","last_active":"2024-04-25T00:41:21.855556+0000","last_peered":"2024-04-25T00:41:21.855556+0000","last_clean":"2024-04-25T00:41:21.855556+0000","last_became_active":"2024-04-25T00:41:20.729597+0000","last_became_peered":"2024-04-25T00:41:20.729597+0000","last_unstale":"2024-04-25T00:41:21.855556+0000","last_undegraded":"2024-04-25T00:41:21.855556+0000","last_fullsized":"2024-04-25T00:41:21.855556+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:27.879965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725550+0000","last_change":"2024-04-25T00:41:20.724283+0000","last_active":"2024-04-25T00:41:21.725550+0000","last_peered":"2024-04-25T00:41:21.725550+0000","last_clean":"2024-04-25T00:41:21.725550+0000","last_became_active":"2024-04-25T00:41:20.723569+0000","last_became_peered":"2024-04-25T00:41:20.723569+0000","last_unstale":"2024-04-25T00:41:21.725550+0000","last_undegraded":"2024-04-25T00:41:21.725550+0000","last_fullsized":"2024-04-25T00:41:21.725550+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:19.901587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852904+0000","last_change":"2024-04-25T00:41:20.746173+0000","last_active":"2024-04-25T00:41:21.852904+0000","last_peered":"2024-04-25T00:41:21.852904+0000","last_clean":"2024-04-25T00:41:21.852904+0000","last_became_active":"2024-04-25T00:41:20.745450+0000","last_became_peered":"2024-04-25T00:41:20.745450+0000","last_unstale":"2024-04-25T00:41:21.852904+0000","last_undegraded":"2024-04-25T00:41:21.852904+0000","last_fullsized":"2024-04-25T00:41:21.852904+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:12.354642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852684+0000","last_change":"2024-04-25T00:41:20.752770+0000","last_active":"2024-04-25T00:41:21.852684+0000","last_peered":"2024-04-25T00:41:21.852684+0000","last_clean":"2024-04-25T00:41:21.852684+0000","last_became_active":"2024-04-25T00:41:20.751527+0000","last_became_peered":"2024-04-25T00:41:20.751527+0000","last_unstale":"2024-04-25T00:41:21.852684+0000","last_undegraded":"2024-04-25T00:41:21.852684+0000","last_fullsized":"2024-04-25T00:41:21.852684+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:29.081738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724561+0000","last_change":"2024-04-25T00:41:20.726726+0000","last_active":"2024-04-25T00:41:21.724561+0000","last_peered":"2024-04-25T00:41:21.724561+0000","last_clean":"2024-04-25T00:41:21.724561+0000","last_became_active":"2024-04-25T00:41:20.726043+0000","last_became_peered":"2024-04-25T00:41:20.726043+0000","last_unstale":"2024-04-25T00:41:21.724561+0000","last_undegraded":"2024-04-25T00:41:21.724561+0000","last_fullsized":"2024-04-25T00:41:21.724561+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:23.550089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728079+0000","last_change":"2024-04-25T00:41:20.742119+0000","last_active":"2024-04-25T00:41:21.728079+0000","last_peered":"2024-04-25T00:41:21.728079+0000","last_clean":"2024-04-25T00:41:21.728079+0000","last_became_active":"2024-04-25T00:41:20.739704+0000","last_became_peered":"2024-04-25T00:41:20.739704+0000","last_unstale":"2024-04-25T00:41:21.728079+0000","last_undegraded":"2024-04-25T00:41:21.728079+0000","last_fullsized":"2024-04-25T00:41:21.728079+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:05.066385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736778+0000","last_change":"2024-04-25T00:41:20.756207+0000","last_active":"2024-04-25T00:41:21.736778+0000","last_peered":"2024-04-25T00:41:21.736778+0000","last_clean":"2024-04-25T00:41:21.736778+0000","last_became_active":"2024-04-25T00:41:20.755848+0000","last_became_peered":"2024-04-25T00:41:20.755848+0000","last_unstale":"2024-04-25T00:41:21.736778+0000","last_undegraded":"2024-04-25T00:41:21.736778+0000","last_fullsized":"2024-04-25T00:41:21.736778+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:04.914485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.727995+0000","last_change":"2024-04-25T00:41:20.728570+0000","last_active":"2024-04-25T00:41:21.727995+0000","last_peered":"2024-04-25T00:41:21.727995+0000","last_clean":"2024-04-25T00:41:21.727995+0000","last_became_active":"2024-04-25T00:41:20.727806+0000","last_became_peered":"2024-04-25T00:41:20.727806+0000","last_unstale":"2024-04-25T00:41:21.727995+0000","last_undegraded":"2024-04-25T00:41:21.727995+0000","last_fullsized":"2024-04-25T00:41:21.727995+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:39.749407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856704+0000","last_change":"2024-04-25T00:41:20.730663+0000","last_active":"2024-04-25T00:41:21.856704+0000","last_peered":"2024-04-25T00:41:21.856704+0000","last_clean":"2024-04-25T00:41:21.856704+0000","last_became_active":"2024-04-25T00:41:20.730330+0000","last_became_peered":"2024-04-25T00:41:20.730330+0000","last_unstale":"2024-04-25T00:41:21.856704+0000","last_undegraded":"2024-04-25T00:41:21.856704+0000","last_fullsized":"2024-04-25T00:41:21.856704+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:52.725891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.726305+0000","last_change":"2024-04-25T00:41:20.728144+0000","last_active":"2024-04-25T00:41:21.726305+0000","last_peered":"2024-04-25T00:41:21.726305+0000","last_clean":"2024-04-25T00:41:21.726305+0000","last_became_active":"2024-04-25T00:41:20.727266+0000","last_became_peered":"2024-04-25T00:41:20.727266+0000","last_unstale":"2024-04-25T00:41:21.726305+0000","last_undegraded":"2024-04-25T00:41:21.726305+0000","last_fullsized":"2024-04-25T00:41:21.726305+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:32.736055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.726950+0000","last_change":"2024-04-25T00:41:20.730975+0000","last_active":"2024-04-25T00:41:21.726950+0000","last_peered":"2024-04-25T00:41:21.726950+0000","last_clean":"2024-04-25T00:41:21.726950+0000","last_became_active":"2024-04-25T00:41:20.730142+0000","last_became_peered":"2024-04-25T00:41:20.730142+0000","last_unstale":"2024-04-25T00:41:21.726950+0000","last_undegraded":"2024-04-25T00:41:21.726950+0000","last_fullsized":"2024-04-25T00:41:21.726950+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:47.575779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852645+0000","last_change":"2024-04-25T00:41:20.723764+0000","last_active":"2024-04-25T00:41:21.852645+0000","last_peered":"2024-04-25T00:41:21.852645+0000","last_clean":"2024-04-25T00:41:21.852645+0000","last_became_active":"2024-04-25T00:41:20.723442+0000","last_became_peered":"2024-04-25T00:41:20.723442+0000","last_unstale":"2024-04-25T00:41:21.852645+0000","last_undegraded":"2024-04-25T00:41:21.852645+0000","last_fullsized":"2024-04-25T00:41:21.852645+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:19.901587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736358+0000","last_change":"2024-04-25T00:41:20.725607+0000","last_active":"2024-04-25T00:41:21.736358+0000","last_peered":"2024-04-25T00:41:21.736358+0000","last_clean":"2024-04-25T00:41:21.736358+0000","last_became_active":"2024-04-25T00:41:20.725195+0000","last_became_peered":"2024-04-25T00:41:20.725195+0000","last_unstale":"2024-04-25T00:41:21.736358+0000","last_undegraded":"2024-04-25T00:41:21.736358+0000","last_fullsized":"2024-04-25T00:41:21.736358+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:55.977495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728742+0000","last_change":"2024-04-25T00:41:20.733094+0000","last_active":"2024-04-25T00:41:21.728742+0000","last_peered":"2024-04-25T00:41:21.728742+0000","last_clean":"2024-04-25T00:41:21.728742+0000","last_became_active":"2024-04-25T00:41:20.732348+0000","last_became_peered":"2024-04-25T00:41:20.732348+0000","last_unstale":"2024-04-25T00:41:21.728742+0000","last_undegraded":"2024-04-25T00:41:21.728742+0000","last_fullsized":"2024-04-25T00:41:21.728742+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:57.037464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"167'4","reported_seq":18,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729495+0000","last_change":"2024-04-25T00:41:19.730561+0000","last_active":"2024-04-25T00:41:21.729495+0000","last_peered":"2024-04-25T00:41:21.729495+0000","last_clean":"2024-04-25T00:41:21.729495+0000","last_became_active":"2024-04-25T00:41:19.729475+0000","last_became_peered":"2024-04-25T00:41:19.729475+0000","last_unstale":"2024-04-25T00:41:21.729495+0000","last_undegraded":"2024-04-25T00:41:21.729495+0000","last_fullsized":"2024-04-25T00:41:21.729495+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:58.727637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853286+0000","last_change":"2024-04-25T00:41:20.754267+0000","last_active":"2024-04-25T00:41:21.853286+0000","last_peered":"2024-04-25T00:41:21.853286+0000","last_clean":"2024-04-25T00:41:21.853286+0000","last_became_active":"2024-04-25T00:41:20.753526+0000","last_became_peered":"2024-04-25T00:41:20.753526+0000","last_unstale":"2024-04-25T00:41:21.853286+0000","last_undegraded":"2024-04-25T00:41:21.853286+0000","last_fullsized":"2024-04-25T00:41:21.853286+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:27.879965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729086+0000","last_change":"2024-04-25T00:41:19.728196+0000","last_active":"2024-04-25T00:41:21.729086+0000","last_peered":"2024-04-25T00:41:21.729086+0000","last_clean":"2024-04-25T00:41:21.729086+0000","last_became_active":"2024-04-25T00:41:19.727469+0000","last_became_peered":"2024-04-25T00:41:19.727469+0000","last_unstale":"2024-04-25T00:41:21.729086+0000","last_undegraded":"2024-04-25T00:41:21.729086+0000","last_fullsized":"2024-04-25T00:41:21.729086+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:18.103810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736679+0000","last_change":"2024-04-25T00:41:20.757686+0000","last_active":"2024-04-25T00:41:21.736679+0000","last_peered":"2024-04-25T00:41:21.736679+0000","last_clean":"2024-04-25T00:41:21.736679+0000","last_became_active":"2024-04-25T00:41:20.757425+0000","last_became_peered":"2024-04-25T00:41:20.757425+0000","last_unstale":"2024-04-25T00:41:21.736679+0000","last_undegraded":"2024-04-25T00:41:21.736679+0000","last_fullsized":"2024-04-25T00:41:21.736679+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:29.413633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856910+0000","last_change":"2024-04-25T00:41:19.714844+0000","last_active":"2024-04-25T00:41:21.856910+0000","last_peered":"2024-04-25T00:41:21.856910+0000","last_clean":"2024-04-25T00:41:21.856910+0000","last_became_active":"2024-04-25T00:41:19.714584+0000","last_became_peered":"2024-04-25T00:41:19.714584+0000","last_unstale":"2024-04-25T00:41:21.856910+0000","last_undegraded":"2024-04-25T00:41:21.856910+0000","last_fullsized":"2024-04-25T00:41:21.856910+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:17.955930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854015+0000","last_change":"2024-04-25T00:41:20.729450+0000","last_active":"2024-04-25T00:41:21.854015+0000","last_peered":"2024-04-25T00:41:21.854015+0000","last_clean":"2024-04-25T00:41:21.854015+0000","last_became_active":"2024-04-25T00:41:20.728986+0000","last_became_peered":"2024-04-25T00:41:20.728986+0000","last_unstale":"2024-04-25T00:41:21.854015+0000","last_undegraded":"2024-04-25T00:41:21.854015+0000","last_fullsized":"2024-04-25T00:41:21.854015+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:44.845619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725827+0000","last_change":"2024-04-25T00:41:19.727073+0000","last_active":"2024-04-25T00:41:21.725827+0000","last_peered":"2024-04-25T00:41:21.725827+0000","last_clean":"2024-04-25T00:41:21.725827+0000","last_became_active":"2024-04-25T00:41:19.726232+0000","last_became_peered":"2024-04-25T00:41:19.726232+0000","last_unstale":"2024-04-25T00:41:21.725827+0000","last_undegraded":"2024-04-25T00:41:21.725827+0000","last_fullsized":"2024-04-25T00:41:21.725827+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:12.458015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724621+0000","last_change":"2024-04-25T00:41:20.754166+0000","last_active":"2024-04-25T00:41:21.724621+0000","last_peered":"2024-04-25T00:41:21.724621+0000","last_clean":"2024-04-25T00:41:21.724621+0000","last_became_active":"2024-04-25T00:41:20.753650+0000","last_became_peered":"2024-04-25T00:41:20.753650+0000","last_unstale":"2024-04-25T00:41:21.724621+0000","last_undegraded":"2024-04-25T00:41:21.724621+0000","last_fullsized":"2024-04-25T00:41:21.724621+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:17.802309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853585+0000","last_change":"2024-04-25T00:41:19.723133+0000","last_active":"2024-04-25T00:41:21.853585+0000","last_peered":"2024-04-25T00:41:21.853585+0000","last_clean":"2024-04-25T00:41:21.853585+0000","last_became_active":"2024-04-25T00:41:19.722748+0000","last_became_peered":"2024-04-25T00:41:19.722748+0000","last_unstale":"2024-04-25T00:41:21.853585+0000","last_undegraded":"2024-04-25T00:41:21.853585+0000","last_fullsized":"2024-04-25T00:41:21.853585+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:53.480966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852317+0000","last_change":"2024-04-25T00:41:20.720406+0000","last_active":"2024-04-25T00:41:21.852317+0000","last_peered":"2024-04-25T00:41:21.852317+0000","last_clean":"2024-04-25T00:41:21.852317+0000","last_became_active":"2024-04-25T00:41:20.720072+0000","last_became_peered":"2024-04-25T00:41:20.720072+0000","last_unstale":"2024-04-25T00:41:21.852317+0000","last_undegraded":"2024-04-25T00:41:21.852317+0000","last_fullsized":"2024-04-25T00:41:21.852317+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:16.067964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852079+0000","last_change":"2024-04-25T00:41:19.712612+0000","last_active":"2024-04-25T00:41:21.852079+0000","last_peered":"2024-04-25T00:41:21.852079+0000","last_clean":"2024-04-25T00:41:21.852079+0000","last_became_active":"2024-04-25T00:41:19.710382+0000","last_became_peered":"2024-04-25T00:41:19.710382+0000","last_unstale":"2024-04-25T00:41:21.852079+0000","last_undegraded":"2024-04-25T00:41:21.852079+0000","last_fullsized":"2024-04-25T00:41:21.852079+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:41.339940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725041+0000","last_change":"2024-04-25T00:41:20.755342+0000","last_active":"2024-04-25T00:41:21.725041+0000","last_peered":"2024-04-25T00:41:21.725041+0000","last_clean":"2024-04-25T00:41:21.725041+0000","last_became_active":"2024-04-25T00:41:20.754876+0000","last_became_peered":"2024-04-25T00:41:20.754876+0000","last_unstale":"2024-04-25T00:41:21.725041+0000","last_undegraded":"2024-04-25T00:41:21.725041+0000","last_fullsized":"2024-04-25T00:41:21.725041+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:16.226179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852140+0000","last_change":"2024-04-25T00:41:19.723756+0000","last_active":"2024-04-25T00:41:21.852140+0000","last_peered":"2024-04-25T00:41:21.852140+0000","last_clean":"2024-04-25T00:41:21.852140+0000","last_became_active":"2024-04-25T00:41:19.722504+0000","last_became_peered":"2024-04-25T00:41:19.722504+0000","last_unstale":"2024-04-25T00:41:21.852140+0000","last_undegraded":"2024-04-25T00:41:21.852140+0000","last_fullsized":"2024-04-25T00:41:21.852140+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:53.480966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728998+0000","last_change":"2024-04-25T00:41:20.715594+0000","last_active":"2024-04-25T00:41:21.728998+0000","last_peered":"2024-04-25T00:41:21.728998+0000","last_clean":"2024-04-25T00:41:21.728998+0000","last_became_active":"2024-04-25T00:41:20.712098+0000","last_became_peered":"2024-04-25T00:41:20.712098+0000","last_unstale":"2024-04-25T00:41:21.728998+0000","last_undegraded":"2024-04-25T00:41:21.728998+0000","last_fullsized":"2024-04-25T00:41:21.728998+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:53.784754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855238+0000","last_change":"2024-04-25T00:41:19.727211+0000","last_active":"2024-04-25T00:41:21.855238+0000","last_peered":"2024-04-25T00:41:21.855238+0000","last_clean":"2024-04-25T00:41:21.855238+0000","last_became_active":"2024-04-25T00:41:19.726926+0000","last_became_peered":"2024-04-25T00:41:19.726926+0000","last_unstale":"2024-04-25T00:41:21.855238+0000","last_undegraded":"2024-04-25T00:41:21.855238+0000","last_fullsized":"2024-04-25T00:41:21.855238+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:04.054309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856661+0000","last_change":"2024-04-25T00:41:20.723273+0000","last_active":"2024-04-25T00:41:21.856661+0000","last_peered":"2024-04-25T00:41:21.856661+0000","last_clean":"2024-04-25T00:41:21.856661+0000","last_became_active":"2024-04-25T00:41:20.722938+0000","last_became_peered":"2024-04-25T00:41:20.722938+0000","last_unstale":"2024-04-25T00:41:21.856661+0000","last_undegraded":"2024-04-25T00:41:21.856661+0000","last_fullsized":"2024-04-25T00:41:21.856661+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:29.975327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853121+0000","last_change":"2024-04-25T00:41:19.733298+0000","last_active":"2024-04-25T00:41:21.853121+0000","last_peered":"2024-04-25T00:41:21.853121+0000","last_clean":"2024-04-25T00:41:21.853121+0000","last_became_active":"2024-04-25T00:41:19.732514+0000","last_became_peered":"2024-04-25T00:41:19.732514+0000","last_unstale":"2024-04-25T00:41:21.853121+0000","last_undegraded":"2024-04-25T00:41:21.853121+0000","last_fullsized":"2024-04-25T00:41:21.853121+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:56.384389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724912+0000","last_change":"2024-04-25T00:41:20.716394+0000","last_active":"2024-04-25T00:41:21.724912+0000","last_peered":"2024-04-25T00:41:21.724912+0000","last_clean":"2024-04-25T00:41:21.724912+0000","last_became_active":"2024-04-25T00:41:20.715624+0000","last_became_peered":"2024-04-25T00:41:20.715624+0000","last_unstale":"2024-04-25T00:41:21.724912+0000","last_undegraded":"2024-04-25T00:41:21.724912+0000","last_fullsized":"2024-04-25T00:41:21.724912+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:23.460044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"167'3","reported_seq":17,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854498+0000","last_change":"2024-04-25T00:41:19.714290+0000","last_active":"2024-04-25T00:41:21.854498+0000","last_peered":"2024-04-25T00:41:21.854498+0000","last_clean":"2024-04-25T00:41:21.854498+0000","last_became_active":"2024-04-25T00:41:19.713997+0000","last_became_peered":"2024-04-25T00:41:19.713997+0000","last_unstale":"2024-04-25T00:41:21.854498+0000","last_undegraded":"2024-04-25T00:41:21.854498+0000","last_fullsized":"2024-04-25T00:41:21.854498+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:47.272504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737071+0000","last_change":"2024-04-25T00:41:20.729697+0000","last_active":"2024-04-25T00:41:21.737071+0000","last_peered":"2024-04-25T00:41:21.737071+0000","last_clean":"2024-04-25T00:41:21.737071+0000","last_became_active":"2024-04-25T00:41:20.728724+0000","last_became_peered":"2024-04-25T00:41:20.728724+0000","last_unstale":"2024-04-25T00:41:21.737071+0000","last_undegraded":"2024-04-25T00:41:21.737071+0000","last_fullsized":"2024-04-25T00:41:21.737071+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:54.350976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"167'6","reported_seq":20,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.726112+0000","last_change":"2024-04-25T00:41:19.726821+0000","last_active":"2024-04-25T00:41:21.726112+0000","last_peered":"2024-04-25T00:41:21.726112+0000","last_clean":"2024-04-25T00:41:21.726112+0000","last_became_active":"2024-04-25T00:41:19.726146+0000","last_became_peered":"2024-04-25T00:41:19.726146+0000","last_unstale":"2024-04-25T00:41:21.726112+0000","last_undegraded":"2024-04-25T00:41:21.726112+0000","last_fullsized":"2024-04-25T00:41:21.726112+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:59.876465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853444+0000","last_change":"2024-04-25T00:41:20.731223+0000","last_active":"2024-04-25T00:41:21.853444+0000","last_peered":"2024-04-25T00:41:21.853444+0000","last_clean":"2024-04-25T00:41:21.853444+0000","last_became_active":"2024-04-25T00:41:20.730734+0000","last_became_peered":"2024-04-25T00:41:20.730734+0000","last_unstale":"2024-04-25T00:41:21.853444+0000","last_undegraded":"2024-04-25T00:41:21.853444+0000","last_fullsized":"2024-04-25T00:41:21.853444+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:21.777831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852468+0000","last_change":"2024-04-25T00:41:19.715184+0000","last_active":"2024-04-25T00:41:21.852468+0000","last_peered":"2024-04-25T00:41:21.852468+0000","last_clean":"2024-04-25T00:41:21.852468+0000","last_became_active":"2024-04-25T00:41:19.713383+0000","last_became_peered":"2024-04-25T00:41:19.713383+0000","last_unstale":"2024-04-25T00:41:21.852468+0000","last_undegraded":"2024-04-25T00:41:21.852468+0000","last_fullsized":"2024-04-25T00:41:21.852468+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:41.339940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853053+0000","last_change":"2024-04-25T00:41:20.726316+0000","last_active":"2024-04-25T00:41:21.853053+0000","last_peered":"2024-04-25T00:41:21.853053+0000","last_clean":"2024-04-25T00:41:21.853053+0000","last_became_active":"2024-04-25T00:41:20.724262+0000","last_became_peered":"2024-04-25T00:41:20.724262+0000","last_unstale":"2024-04-25T00:41:21.853053+0000","last_undegraded":"2024-04-25T00:41:21.853053+0000","last_fullsized":"2024-04-25T00:41:21.853053+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:53.784754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852580+0000","last_change":"2024-04-25T00:41:19.724753+0000","last_active":"2024-04-25T00:41:21.852580+0000","last_peered":"2024-04-25T00:41:21.852580+0000","last_clean":"2024-04-25T00:41:21.852580+0000","last_became_active":"2024-04-25T00:41:19.721745+0000","last_became_peered":"2024-04-25T00:41:19.721745+0000","last_unstale":"2024-04-25T00:41:21.852580+0000","last_undegraded":"2024-04-25T00:41:21.852580+0000","last_fullsized":"2024-04-25T00:41:21.852580+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:31.723980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856313+0000","last_change":"2024-04-25T00:41:20.727156+0000","last_active":"2024-04-25T00:41:21.856313+0000","last_peered":"2024-04-25T00:41:21.856313+0000","last_clean":"2024-04-25T00:41:21.856313+0000","last_became_active":"2024-04-25T00:41:20.725938+0000","last_became_peered":"2024-04-25T00:41:20.725938+0000","last_unstale":"2024-04-25T00:41:21.856313+0000","last_undegraded":"2024-04-25T00:41:21.856313+0000","last_fullsized":"2024-04-25T00:41:21.856313+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:03.298965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852561+0000","last_change":"2024-04-25T00:41:19.725909+0000","last_active":"2024-04-25T00:41:21.852561+0000","last_peered":"2024-04-25T00:41:21.852561+0000","last_clean":"2024-04-25T00:41:21.852561+0000","last_became_active":"2024-04-25T00:41:19.725301+0000","last_became_peered":"2024-04-25T00:41:19.725301+0000","last_unstale":"2024-04-25T00:41:21.852561+0000","last_undegraded":"2024-04-25T00:41:21.852561+0000","last_fullsized":"2024-04-25T00:41:21.852561+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:59.876465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736566+0000","last_change":"2024-04-25T00:41:20.746071+0000","last_active":"2024-04-25T00:41:21.736566+0000","last_peered":"2024-04-25T00:41:21.736566+0000","last_clean":"2024-04-25T00:41:21.736566+0000","last_became_active":"2024-04-25T00:41:20.745714+0000","last_became_peered":"2024-04-25T00:41:20.745714+0000","last_unstale":"2024-04-25T00:41:21.736566+0000","last_undegraded":"2024-04-25T00:41:21.736566+0000","last_fullsized":"2024-04-25T00:41:21.736566+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:10.415430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"167'8","reported_seq":22,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855744+0000","last_change":"2024-04-25T00:41:19.720510+0000","last_active":"2024-04-25T00:41:21.855744+0000","last_peered":"2024-04-25T00:41:21.855744+0000","last_clean":"2024-04-25T00:41:21.855744+0000","last_became_active":"2024-04-25T00:41:19.720185+0000","last_became_peered":"2024-04-25T00:41:19.720185+0000","last_unstale":"2024-04-25T00:41:21.855744+0000","last_undegraded":"2024-04-25T00:41:21.855744+0000","last_fullsized":"2024-04-25T00:41:21.855744+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:50.914418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729150+0000","last_change":"2024-04-25T00:41:20.710672+0000","last_active":"2024-04-25T00:41:21.729150+0000","last_peered":"2024-04-25T00:41:21.729150+0000","last_clean":"2024-04-25T00:41:21.729150+0000","last_became_active":"2024-04-25T00:41:20.709976+0000","last_became_peered":"2024-04-25T00:41:20.709976+0000","last_unstale":"2024-04-25T00:41:21.729150+0000","last_undegraded":"2024-04-25T00:41:21.729150+0000","last_fullsized":"2024-04-25T00:41:21.729150+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:45.163125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855527+0000","last_change":"2024-04-25T00:41:19.721839+0000","last_active":"2024-04-25T00:41:21.855527+0000","last_peered":"2024-04-25T00:41:21.855527+0000","last_clean":"2024-04-25T00:41:21.855527+0000","last_became_active":"2024-04-25T00:41:19.721547+0000","last_became_peered":"2024-04-25T00:41:19.721547+0000","last_unstale":"2024-04-25T00:41:21.855527+0000","last_undegraded":"2024-04-25T00:41:21.855527+0000","last_fullsized":"2024-04-25T00:41:21.855527+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:00.898156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728440+0000","last_change":"2024-04-25T00:41:20.729031+0000","last_active":"2024-04-25T00:41:21.728440+0000","last_peered":"2024-04-25T00:41:21.728440+0000","last_clean":"2024-04-25T00:41:21.728440+0000","last_became_active":"2024-04-25T00:41:20.724259+0000","last_became_peered":"2024-04-25T00:41:20.724259+0000","last_unstale":"2024-04-25T00:41:21.728440+0000","last_undegraded":"2024-04-25T00:41:21.728440+0000","last_fullsized":"2024-04-25T00:41:21.728440+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:02.403416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729314+0000","last_change":"2024-04-25T00:41:19.707832+0000","last_active":"2024-04-25T00:41:21.729314+0000","last_peered":"2024-04-25T00:41:21.729314+0000","last_clean":"2024-04-25T00:41:21.729314+0000","last_became_active":"2024-04-25T00:41:19.706600+0000","last_became_peered":"2024-04-25T00:41:19.706600+0000","last_unstale":"2024-04-25T00:41:21.729314+0000","last_undegraded":"2024-04-25T00:41:21.729314+0000","last_fullsized":"2024-04-25T00:41:21.729314+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:09.353188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855887+0000","last_change":"2024-04-25T00:41:20.736978+0000","last_active":"2024-04-25T00:41:21.855887+0000","last_peered":"2024-04-25T00:41:21.855887+0000","last_clean":"2024-04-25T00:41:21.855887+0000","last_became_active":"2024-04-25T00:41:20.736654+0000","last_became_peered":"2024-04-25T00:41:20.736654+0000","last_unstale":"2024-04-25T00:41:21.855887+0000","last_undegraded":"2024-04-25T00:41:21.855887+0000","last_fullsized":"2024-04-25T00:41:21.855887+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:14.858055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854660+0000","last_change":"2024-04-25T00:41:19.724832+0000","last_active":"2024-04-25T00:41:21.854660+0000","last_peered":"2024-04-25T00:41:21.854660+0000","last_clean":"2024-04-25T00:41:21.854660+0000","last_became_active":"2024-04-25T00:41:19.724496+0000","last_became_peered":"2024-04-25T00:41:19.724496+0000","last_unstale":"2024-04-25T00:41:21.854660+0000","last_undegraded":"2024-04-25T00:41:21.854660+0000","last_fullsized":"2024-04-25T00:41:21.854660+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:25.361018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854330+0000","last_change":"2024-04-25T00:41:20.727581+0000","last_active":"2024-04-25T00:41:21.854330+0000","last_peered":"2024-04-25T00:41:21.854330+0000","last_clean":"2024-04-25T00:41:21.854330+0000","last_became_active":"2024-04-25T00:41:20.723303+0000","last_became_peered":"2024-04-25T00:41:20.723303+0000","last_unstale":"2024-04-25T00:41:21.854330+0000","last_undegraded":"2024-04-25T00:41:21.854330+0000","last_fullsized":"2024-04-25T00:41:21.854330+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:55.977495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856037+0000","last_change":"2024-04-25T00:41:19.720256+0000","last_active":"2024-04-25T00:41:21.856037+0000","last_peered":"2024-04-25T00:41:21.856037+0000","last_clean":"2024-04-25T00:41:21.856037+0000","last_became_active":"2024-04-25T00:41:19.719906+0000","last_became_peered":"2024-04-25T00:41:19.719906+0000","last_unstale":"2024-04-25T00:41:21.856037+0000","last_undegraded":"2024-04-25T00:41:21.856037+0000","last_fullsized":"2024-04-25T00:41:21.856037+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:58.692958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853309+0000","last_change":"2024-04-25T00:41:20.727981+0000","last_active":"2024-04-25T00:41:21.853309+0000","last_peered":"2024-04-25T00:41:21.853309+0000","last_clean":"2024-04-25T00:41:21.853309+0000","last_became_active":"2024-04-25T00:41:20.727650+0000","last_became_peered":"2024-04-25T00:41:20.727650+0000","last_unstale":"2024-04-25T00:41:21.853309+0000","last_undegraded":"2024-04-25T00:41:21.853309+0000","last_fullsized":"2024-04-25T00:41:21.853309+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:24.427571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852863+0000","last_change":"2024-04-25T00:41:19.725591+0000","last_active":"2024-04-25T00:41:21.852863+0000","last_peered":"2024-04-25T00:41:21.852863+0000","last_clean":"2024-04-25T00:41:21.852863+0000","last_became_active":"2024-04-25T00:41:19.725213+0000","last_became_peered":"2024-04-25T00:41:19.725213+0000","last_unstale":"2024-04-25T00:41:21.852863+0000","last_undegraded":"2024-04-25T00:41:21.852863+0000","last_fullsized":"2024-04-25T00:41:21.852863+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:35.912276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856172+0000","last_change":"2024-04-25T00:41:20.728184+0000","last_active":"2024-04-25T00:41:21.856172+0000","last_peered":"2024-04-25T00:41:21.856172+0000","last_clean":"2024-04-25T00:41:21.856172+0000","last_became_active":"2024-04-25T00:41:20.726395+0000","last_became_peered":"2024-04-25T00:41:20.726395+0000","last_unstale":"2024-04-25T00:41:21.856172+0000","last_undegraded":"2024-04-25T00:41:21.856172+0000","last_fullsized":"2024-04-25T00:41:21.856172+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:05.020843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737304+0000","last_change":"2024-04-25T00:41:19.726228+0000","last_active":"2024-04-25T00:41:21.737304+0000","last_peered":"2024-04-25T00:41:21.737304+0000","last_clean":"2024-04-25T00:41:21.737304+0000","last_became_active":"2024-04-25T00:41:19.725747+0000","last_became_peered":"2024-04-25T00:41:19.725747+0000","last_unstale":"2024-04-25T00:41:21.737304+0000","last_undegraded":"2024-04-25T00:41:21.737304+0000","last_fullsized":"2024-04-25T00:41:21.737304+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:00.329987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724987+0000","last_change":"2024-04-25T00:41:20.747974+0000","last_active":"2024-04-25T00:41:21.724987+0000","last_peered":"2024-04-25T00:41:21.724987+0000","last_clean":"2024-04-25T00:41:21.724987+0000","last_became_active":"2024-04-25T00:41:20.747202+0000","last_became_peered":"2024-04-25T00:41:20.747202+0000","last_unstale":"2024-04-25T00:41:21.724987+0000","last_undegraded":"2024-04-25T00:41:21.724987+0000","last_fullsized":"2024-04-25T00:41:21.724987+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:52.725891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736508+0000","last_change":"2024-04-25T00:41:19.724386+0000","last_active":"2024-04-25T00:41:21.736508+0000","last_peered":"2024-04-25T00:41:21.736508+0000","last_clean":"2024-04-25T00:41:21.736508+0000","last_became_active":"2024-04-25T00:41:19.724122+0000","last_became_peered":"2024-04-25T00:41:19.724122+0000","last_unstale":"2024-04-25T00:41:21.736508+0000","last_undegraded":"2024-04-25T00:41:21.736508+0000","last_fullsized":"2024-04-25T00:41:21.736508+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:58.585739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852918+0000","last_change":"2024-04-25T00:41:20.751369+0000","last_active":"2024-04-25T00:41:21.852918+0000","last_peered":"2024-04-25T00:41:21.852918+0000","last_clean":"2024-04-25T00:41:21.852918+0000","last_became_active":"2024-04-25T00:41:20.746068+0000","last_became_peered":"2024-04-25T00:41:20.746068+0000","last_unstale":"2024-04-25T00:41:21.852918+0000","last_undegraded":"2024-04-25T00:41:21.852918+0000","last_fullsized":"2024-04-25T00:41:21.852918+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:21.777831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852447+0000","last_change":"2024-04-25T00:41:19.729543+0000","last_active":"2024-04-25T00:41:21.852447+0000","last_peered":"2024-04-25T00:41:21.852447+0000","last_clean":"2024-04-25T00:41:21.852447+0000","last_became_active":"2024-04-25T00:41:19.729146+0000","last_became_peered":"2024-04-25T00:41:19.729146+0000","last_unstale":"2024-04-25T00:41:21.852447+0000","last_undegraded":"2024-04-25T00:41:21.852447+0000","last_fullsized":"2024-04-25T00:41:21.852447+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:56.025389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728090+0000","last_change":"2024-04-25T00:41:20.732303+0000","last_active":"2024-04-25T00:41:21.728090+0000","last_peered":"2024-04-25T00:41:21.728090+0000","last_clean":"2024-04-25T00:41:21.728090+0000","last_became_active":"2024-04-25T00:41:20.727064+0000","last_became_peered":"2024-04-25T00:41:20.727064+0000","last_unstale":"2024-04-25T00:41:21.728090+0000","last_undegraded":"2024-04-25T00:41:21.728090+0000","last_fullsized":"2024-04-25T00:41:21.728090+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:29.644405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856443+0000","last_change":"2024-04-25T00:41:19.711806+0000","last_active":"2024-04-25T00:41:21.856443+0000","last_peered":"2024-04-25T00:41:21.856443+0000","last_clean":"2024-04-25T00:41:21.856443+0000","last_became_active":"2024-04-25T00:41:19.711478+0000","last_became_peered":"2024-04-25T00:41:19.711478+0000","last_unstale":"2024-04-25T00:41:21.856443+0000","last_undegraded":"2024-04-25T00:41:21.856443+0000","last_fullsized":"2024-04-25T00:41:21.856443+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:25.679848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737205+0000","last_change":"2024-04-25T00:41:20.729141+0000","last_active":"2024-04-25T00:41:21.737205+0000","last_peered":"2024-04-25T00:41:21.737205+0000","last_clean":"2024-04-25T00:41:21.737205+0000","last_became_active":"2024-04-25T00:41:20.728844+0000","last_became_peered":"2024-04-25T00:41:20.728844+0000","last_unstale":"2024-04-25T00:41:21.737205+0000","last_undegraded":"2024-04-25T00:41:21.737205+0000","last_fullsized":"2024-04-25T00:41:21.737205+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:09.108727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"168'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729185+0000","last_change":"2024-04-25T00:41:19.726585+0000","last_active":"2024-04-25T00:41:21.729185+0000","last_peered":"2024-04-25T00:41:21.729185+0000","last_clean":"2024-04-25T00:41:21.729185+0000","last_became_active":"2024-04-25T00:41:19.725579+0000","last_became_peered":"2024-04-25T00:41:19.725579+0000","last_unstale":"2024-04-25T00:41:21.729185+0000","last_undegraded":"2024-04-25T00:41:21.729185+0000","last_fullsized":"2024-04-25T00:41:21.729185+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:58.553384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.727539+0000","last_change":"2024-04-25T00:41:20.724729+0000","last_active":"2024-04-25T00:41:21.727539+0000","last_peered":"2024-04-25T00:41:21.727539+0000","last_clean":"2024-04-25T00:41:21.727539+0000","last_became_active":"2024-04-25T00:41:20.723781+0000","last_became_peered":"2024-04-25T00:41:20.723781+0000","last_unstale":"2024-04-25T00:41:21.727539+0000","last_undegraded":"2024-04-25T00:41:21.727539+0000","last_fullsized":"2024-04-25T00:41:21.727539+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:39.356001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854985+0000","last_change":"2024-04-25T00:41:19.713087+0000","last_active":"2024-04-25T00:41:21.854985+0000","last_peered":"2024-04-25T00:41:21.854985+0000","last_clean":"2024-04-25T00:41:21.854985+0000","last_became_active":"2024-04-25T00:41:19.710399+0000","last_became_peered":"2024-04-25T00:41:19.710399+0000","last_unstale":"2024-04-25T00:41:21.854985+0000","last_undegraded":"2024-04-25T00:41:21.854985+0000","last_fullsized":"2024-04-25T00:41:21.854985+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:01.391340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853040+0000","last_change":"2024-04-25T00:41:20.752301+0000","last_active":"2024-04-25T00:41:21.853040+0000","last_peered":"2024-04-25T00:41:21.853040+0000","last_clean":"2024-04-25T00:41:21.853040+0000","last_became_active":"2024-04-25T00:41:20.751497+0000","last_became_peered":"2024-04-25T00:41:20.751497+0000","last_unstale":"2024-04-25T00:41:21.853040+0000","last_undegraded":"2024-04-25T00:41:21.853040+0000","last_fullsized":"2024-04-25T00:41:21.853040+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:50.003262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856627+0000","last_change":"2024-04-25T00:41:19.727532+0000","last_active":"2024-04-25T00:41:21.856627+0000","last_peered":"2024-04-25T00:41:21.856627+0000","last_clean":"2024-04-25T00:41:21.856627+0000","last_became_active":"2024-04-25T00:41:19.727155+0000","last_became_peered":"2024-04-25T00:41:19.727155+0000","last_unstale":"2024-04-25T00:41:21.856627+0000","last_undegraded":"2024-04-25T00:41:21.856627+0000","last_fullsized":"2024-04-25T00:41:21.856627+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:54.985055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737267+0000","last_change":"2024-04-25T00:41:20.756146+0000","last_active":"2024-04-25T00:41:21.737267+0000","last_peered":"2024-04-25T00:41:21.737267+0000","last_clean":"2024-04-25T00:41:21.737267+0000","last_became_active":"2024-04-25T00:41:20.755869+0000","last_became_peered":"2024-04-25T00:41:20.755869+0000","last_unstale":"2024-04-25T00:41:21.737267+0000","last_undegraded":"2024-04-25T00:41:21.737267+0000","last_fullsized":"2024-04-25T00:41:21.737267+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:02.047170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724768+0000","last_change":"2024-04-25T00:41:19.725155+0000","last_active":"2024-04-25T00:41:21.724768+0000","last_peered":"2024-04-25T00:41:21.724768+0000","last_clean":"2024-04-25T00:41:21.724768+0000","last_became_active":"2024-04-25T00:41:19.723795+0000","last_became_peered":"2024-04-25T00:41:19.723795+0000","last_unstale":"2024-04-25T00:41:21.724768+0000","last_undegraded":"2024-04-25T00:41:21.724768+0000","last_fullsized":"2024-04-25T00:41:21.724768+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:17.151602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728540+0000","last_change":"2024-04-25T00:41:20.733781+0000","last_active":"2024-04-25T00:41:21.728540+0000","last_peered":"2024-04-25T00:41:21.728540+0000","last_clean":"2024-04-25T00:41:21.728540+0000","last_became_active":"2024-04-25T00:41:20.726560+0000","last_became_peered":"2024-04-25T00:41:20.726560+0000","last_unstale":"2024-04-25T00:41:21.728540+0000","last_undegraded":"2024-04-25T00:41:21.728540+0000","last_fullsized":"2024-04-25T00:41:21.728540+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:21.667951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853894+0000","last_change":"2024-04-25T00:41:19.722319+0000","last_active":"2024-04-25T00:41:21.853894+0000","last_peered":"2024-04-25T00:41:21.853894+0000","last_clean":"2024-04-25T00:41:21.853894+0000","last_became_active":"2024-04-25T00:41:19.721932+0000","last_became_peered":"2024-04-25T00:41:19.721932+0000","last_unstale":"2024-04-25T00:41:21.853894+0000","last_undegraded":"2024-04-25T00:41:21.853894+0000","last_fullsized":"2024-04-25T00:41:21.853894+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:44.151050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855598+0000","last_change":"2024-04-25T00:41:20.735352+0000","last_active":"2024-04-25T00:41:21.855598+0000","last_peered":"2024-04-25T00:41:21.855598+0000","last_clean":"2024-04-25T00:41:21.855598+0000","last_became_active":"2024-04-25T00:41:20.733136+0000","last_became_peered":"2024-04-25T00:41:20.733136+0000","last_unstale":"2024-04-25T00:41:21.855598+0000","last_undegraded":"2024-04-25T00:41:21.855598+0000","last_fullsized":"2024-04-25T00:41:21.855598+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:45.237690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852820+0000","last_change":"2024-04-25T00:41:19.732832+0000","last_active":"2024-04-25T00:41:21.852820+0000","last_peered":"2024-04-25T00:41:21.852820+0000","last_clean":"2024-04-25T00:41:21.852820+0000","last_became_active":"2024-04-25T00:41:19.731714+0000","last_became_peered":"2024-04-25T00:41:19.731714+0000","last_unstale":"2024-04-25T00:41:21.852820+0000","last_undegraded":"2024-04-25T00:41:21.852820+0000","last_fullsized":"2024-04-25T00:41:21.852820+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:44.151050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725714+0000","last_change":"2024-04-25T00:41:20.717857+0000","last_active":"2024-04-25T00:41:21.725714+0000","last_peered":"2024-04-25T00:41:21.725714+0000","last_clean":"2024-04-25T00:41:21.725714+0000","last_became_active":"2024-04-25T00:41:20.717105+0000","last_became_peered":"2024-04-25T00:41:20.717105+0000","last_unstale":"2024-04-25T00:41:21.725714+0000","last_undegraded":"2024-04-25T00:41:21.725714+0000","last_fullsized":"2024-04-25T00:41:21.725714+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:59.739712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.726474+0000","last_change":"2024-04-25T00:41:19.714001+0000","last_active":"2024-04-25T00:41:21.726474+0000","last_peered":"2024-04-25T00:41:21.726474+0000","last_clean":"2024-04-25T00:41:21.726474+0000","last_became_active":"2024-04-25T00:41:19.713267+0000","last_became_peered":"2024-04-25T00:41:19.713267+0000","last_unstale":"2024-04-25T00:41:21.726474+0000","last_undegraded":"2024-04-25T00:41:21.726474+0000","last_fullsized":"2024-04-25T00:41:21.726474+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:46.563704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737058+0000","last_change":"2024-04-25T00:41:20.726216+0000","last_active":"2024-04-25T00:41:21.737058+0000","last_peered":"2024-04-25T00:41:21.737058+0000","last_clean":"2024-04-25T00:41:21.737058+0000","last_became_active":"2024-04-25T00:41:20.725564+0000","last_became_peered":"2024-04-25T00:41:20.725564+0000","last_unstale":"2024-04-25T00:41:21.737058+0000","last_undegraded":"2024-04-25T00:41:21.737058+0000","last_fullsized":"2024-04-25T00:41:21.737058+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:27.898758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728110+0000","last_change":"2024-04-25T00:41:19.729890+0000","last_active":"2024-04-25T00:41:21.728110+0000","last_peered":"2024-04-25T00:41:21.728110+0000","last_clean":"2024-04-25T00:41:21.728110+0000","last_became_active":"2024-04-25T00:41:19.728730+0000","last_became_peered":"2024-04-25T00:41:19.728730+0000","last_unstale":"2024-04-25T00:41:21.728110+0000","last_undegraded":"2024-04-25T00:41:21.728110+0000","last_fullsized":"2024-04-25T00:41:21.728110+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:38.343926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":489,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724956+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:41:21.724956+0000","last_peered":"2024-04-25T00:41:21.724956+0000","last_clean":"2024-04-25T00:41:21.724956+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:41:21.724956+0000","last_undegraded":"2024-04-25T00:41:21.724956+0000","last_fullsized":"2024-04-25T00:41:21.724956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38004,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333836,"statfs":{"total":96636764160,"available":96597848064,"internally_reserved":0,"allocated":2727936,"data_stored":1529742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574972,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37416,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334424,"statfs":{"total":96636764160,"available":96598450176,"internally_reserved":0,"allocated":2125824,"data_stored":938344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574972,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37420,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334420,"statfs":{"total":96636764160,"available":96598446080,"internally_reserved":0,"allocated":2125824,"data_stored":938344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574971,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41600,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330240,"statfs":{"total":96636764160,"available":96594165760,"internally_reserved":0,"allocated":2142208,"data_stored":939478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574972,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41588,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330252,"statfs":{"total":96636764160,"available":96594178048,"internally_reserved":0,"allocated":2134016,"data_stored":937944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37416,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334424,"statfs":{"total":96636764160,"available":96598450176,"internally_reserved":0,"allocated":2121728,"data_stored":937762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2129920,"data_stored":937818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574971,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42156,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329684,"statfs":{"total":96636764160,"available":96593596416,"internally_reserved":0,"allocated":2719744,"data_stored":1528220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:41:29.278 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:41:29.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f406649b640 1 Processor -- start 2024-04-25T00:41:29.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f406649b640 1 -- start start 2024-04-25T00:41:29.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f406649b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f406006dce0 0x7f4060150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:29.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f406649b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f40600b2b70 0x7f4060151330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:29.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f406649b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40600b3130 con 0x7f406015d980 2024-04-25T00:41:29.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f406649b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f40600cf560 con 0x7f40600b2b70 2024-04-25T00:41:29.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f406649b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f40600b3410 con 0x7f406006dce0 2024-04-25T00:41:29.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f4064a38640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f40600b2b70 0x7f4060151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:29.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f4064a38640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f40600b2b70 0x7f4060151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37084/0 (socket says 172.21.15.29:37084) 2024-04-25T00:41:29.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f4064a38640 1 -- 172.21.15.29:0/3657387865 learned_addr learned my addr 172.21.15.29:0/3657387865 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:29.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f405f7fe640 1 --2- 172.21.15.29:0/3657387865 >> v2:172.21.15.157:3300/0 conn(0x7f406006dce0 0x7f4060150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:29.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f4064a38640 1 -- 172.21.15.29:0/3657387865 >> v2:172.21.15.157:3300/0 conn(0x7f406006dce0 msgr2=0x7f4060150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:29.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f4064a38640 1 --2- 172.21.15.29:0/3657387865 >> v2:172.21.15.157:3300/0 conn(0x7f406006dce0 0x7f4060150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f4064a38640 1 -- 172.21.15.29:0/3657387865 >> v1:172.21.15.157:6789/0 conn(0x7f406015d980 legacy=0x7f406006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:29.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.412+0000 7f4064a38640 1 -- 172.21.15.29:0/3657387865 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40600b36f0 con 0x7f40600b2b70 2024-04-25T00:41:29.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f4064a38640 1 --2- 172.21.15.29:0/3657387865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40600b2b70 0x7f4060151330 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f40600b2720 tx=0x7f405801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc488a518bc8e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:29.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f405effd640 1 -- 172.21.15.29:0/3657387865 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580a5800 con 0x7f40600b2b70 2024-04-25T00:41:29.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f405effd640 1 -- 172.21.15.29:0/3657387865 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40580a59a0 con 0x7f40600b2b70 2024-04-25T00:41:29.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f405effd640 1 -- 172.21.15.29:0/3657387865 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580a5d70 con 0x7f40600b2b70 2024-04-25T00:41:29.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f406649b640 1 -- 172.21.15.29:0/3657387865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40600b2b70 msgr2=0x7f4060151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:29.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f406649b640 1 --2- 172.21.15.29:0/3657387865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40600b2b70 0x7f4060151330 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f40600b2720 tx=0x7f405801c9c0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f406649b640 1 -- 172.21.15.29:0/3657387865 shutdown_connections 2024-04-25T00:41:29.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f406649b640 1 --2- 172.21.15.29:0/3657387865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40600b2b70 0x7f4060151330 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f406649b640 1 --2- 172.21.15.29:0/3657387865 >> v2:172.21.15.157:3300/0 conn(0x7f406006dce0 0x7f4060150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f406649b640 1 -- 172.21.15.29:0/3657387865 >> 172.21.15.29:0/3657387865 conn(0x7f40600b1870 msgr2=0x7f40600b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:29.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f406649b640 1 -- 172.21.15.29:0/3657387865 shutdown_connections 2024-04-25T00:41:29.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f406649b640 1 -- 172.21.15.29:0/3657387865 wait complete. 2024-04-25T00:41:29.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f406649b640 1 Processor -- start 2024-04-25T00:41:29.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.416+0000 7f406649b640 1 -- start start 2024-04-25T00:41:29.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f406649b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f406006dce0 0x7f406014ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:29.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f406649b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40600b2b70 0x7f40600ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:29.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f406649b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f406015d980 0x7f40601595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:29.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f406649b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f406015c5c0 con 0x7f406006dce0 2024-04-25T00:41:29.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f406649b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4060151870 con 0x7f406015d980 2024-04-25T00:41:29.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f406649b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f406015c740 con 0x7f40600b2b70 2024-04-25T00:41:29.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f406006dce0 0x7f406014ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:29.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f406006dce0 0x7f406014ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37098/0 (socket says 172.21.15.29:37098) 2024-04-25T00:41:29.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405ffff640 1 -- 172.21.15.29:0/2224459203 learned_addr learned my addr 172.21.15.29:0/2224459203 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:29.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f4064a38640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f406015d980 0x7f40601595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:29.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405f7fe640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40600b2b70 0x7f40600ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:29.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405ffff640 1 -- 172.21.15.29:0/2224459203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40600b2b70 msgr2=0x7f40600ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:29.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405ffff640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40600b2b70 0x7f40600ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405ffff640 1 -- 172.21.15.29:0/2224459203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f406015d980 msgr2=0x7f40601595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:29.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405ffff640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f406015d980 0x7f40601595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405ffff640 1 -- 172.21.15.29:0/2224459203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40601fb330 con 0x7f406006dce0 2024-04-25T00:41:29.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405ffff640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f406006dce0 0x7f406014ad90 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f405000c370 tx=0x7f405000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:29.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405cff9640 1 -- 172.21.15.29:0/2224459203 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40500052d0 con 0x7f406006dce0 2024-04-25T00:41:29.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405cff9640 1 -- 172.21.15.29:0/2224459203 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4050005470 con 0x7f406006dce0 2024-04-25T00:41:29.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f406649b640 1 -- 172.21.15.29:0/2224459203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406014b2d0 con 0x7f406006dce0 2024-04-25T00:41:29.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405cff9640 1 -- 172.21.15.29:0/2224459203 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050007260 con 0x7f406006dce0 2024-04-25T00:41:29.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f406649b640 1 -- 172.21.15.29:0/2224459203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40601faa90 con 0x7f406006dce0 2024-04-25T00:41:29.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f406649b640 1 -- 172.21.15.29:0/2224459203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4060069270 con 0x7f406006dce0 2024-04-25T00:41:29.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405cff9640 1 -- 172.21.15.29:0/2224459203 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f4050007400 con 0x7f406006dce0 2024-04-25T00:41:29.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405cff9640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f404c068250 0x7f404c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:29.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405cff9640 1 -- 172.21.15.29:0/2224459203 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f4050083750 con 0x7f406006dce0 2024-04-25T00:41:29.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405f7fe640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f404c068250 0x7f404c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:29.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.420+0000 7f405f7fe640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f404c068250 0x7f404c06a710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f406015a470 tx=0x7f4054009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:29.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.424+0000 7f405cff9640 1 -- 172.21.15.29:0/2224459203 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4050055840 con 0x7f406006dce0 2024-04-25T00:41:29.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.584+0000 7f406649b640 1 -- 172.21.15.29:0/2224459203 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4060151ca0 con 0x7f404c068250 2024-04-25T00:41:29.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.592+0000 7f405cff9640 1 -- 172.21.15.29:0/2224459203 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252376 (secure 0 0 0) 0x7f4060151ca0 con 0x7f404c068250 2024-04-25T00:41:29.594 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:29.595 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:41:29.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.592+0000 7f406649b640 1 -- 172.21.15.29:0/2224459203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f404c068250 msgr2=0x7f404c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:29.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.592+0000 7f406649b640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f404c068250 0x7f404c06a710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f406015a470 tx=0x7f4054009190 comp rx=0 tx=0).stop 2024-04-25T00:41:29.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.592+0000 7f406649b640 1 -- 172.21.15.29:0/2224459203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f406006dce0 msgr2=0x7f406014ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:29.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.592+0000 7f406649b640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f406006dce0 0x7f406014ad90 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f405000c370 tx=0x7f405000c830 comp rx=0 tx=0).stop 2024-04-25T00:41:29.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.592+0000 7f406649b640 1 -- 172.21.15.29:0/2224459203 shutdown_connections 2024-04-25T00:41:29.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.592+0000 7f406649b640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f404c068250 0x7f404c06a710 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.592+0000 7f406649b640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f406015d980 0x7f40601595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.592+0000 7f406649b640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40600b2b70 0x7f40600ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.592+0000 7f406649b640 1 --2- 172.21.15.29:0/2224459203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f406006dce0 0x7f406014ad90 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.592+0000 7f406649b640 1 -- 172.21.15.29:0/2224459203 >> 172.21.15.29:0/2224459203 conn(0x7f40600b1870 msgr2=0x7f40600af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:29.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.596+0000 7f406649b640 1 -- 172.21.15.29:0/2224459203 shutdown_connections 2024-04-25T00:41:29.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.596+0000 7f406649b640 1 -- 172.21.15.29:0/2224459203 wait complete. 2024-04-25T00:41:29.612 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-04-25T00:41:29.422460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":313028,"kb_used_data":17800,"kb_used_omap":12,"kb_used_meta":295091,"kb_avail":754661692,"statfs":{"total":773094113280,"available":772773572608,"internally_reserved":0,"allocated":18227200,"data_stored":8687652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":302173744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.728889"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736869+0000","last_change":"2024-04-25T00:41:20.738051+0000","last_active":"2024-04-25T00:41:21.736869+0000","last_peered":"2024-04-25T00:41:21.736869+0000","last_clean":"2024-04-25T00:41:21.736869+0000","last_became_active":"2024-04-25T00:41:20.737775+0000","last_became_peered":"2024-04-25T00:41:20.737775+0000","last_unstale":"2024-04-25T00:41:21.736869+0000","last_undegraded":"2024-04-25T00:41:21.736869+0000","last_fullsized":"2024-04-25T00:41:21.736869+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:00.559458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728319+0000","last_change":"2024-04-25T00:41:20.754563+0000","last_active":"2024-04-25T00:41:21.728319+0000","last_peered":"2024-04-25T00:41:21.728319+0000","last_clean":"2024-04-25T00:41:21.728319+0000","last_became_active":"2024-04-25T00:41:20.753984+0000","last_became_peered":"2024-04-25T00:41:20.753984+0000","last_unstale":"2024-04-25T00:41:21.728319+0000","last_undegraded":"2024-04-25T00:41:21.728319+0000","last_fullsized":"2024-04-25T00:41:21.728319+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:01.910232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856753+0000","last_change":"2024-04-25T00:41:20.722575+0000","last_active":"2024-04-25T00:41:21.856753+0000","last_peered":"2024-04-25T00:41:21.856753+0000","last_clean":"2024-04-25T00:41:21.856753+0000","last_became_active":"2024-04-25T00:41:20.722097+0000","last_became_peered":"2024-04-25T00:41:20.722097+0000","last_unstale":"2024-04-25T00:41:21.856753+0000","last_undegraded":"2024-04-25T00:41:21.856753+0000","last_fullsized":"2024-04-25T00:41:21.856753+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:59.739712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724539+0000","last_change":"2024-04-25T00:41:20.754229+0000","last_active":"2024-04-25T00:41:21.724539+0000","last_peered":"2024-04-25T00:41:21.724539+0000","last_clean":"2024-04-25T00:41:21.724539+0000","last_became_active":"2024-04-25T00:41:20.753591+0000","last_became_peered":"2024-04-25T00:41:20.753591+0000","last_unstale":"2024-04-25T00:41:21.724539+0000","last_undegraded":"2024-04-25T00:41:21.724539+0000","last_fullsized":"2024-04-25T00:41:21.724539+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:14.056858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855011+0000","last_change":"2024-04-25T00:41:20.729817+0000","last_active":"2024-04-25T00:41:21.855011+0000","last_peered":"2024-04-25T00:41:21.855011+0000","last_clean":"2024-04-25T00:41:21.855011+0000","last_became_active":"2024-04-25T00:41:20.729299+0000","last_became_peered":"2024-04-25T00:41:20.729299+0000","last_unstale":"2024-04-25T00:41:21.855011+0000","last_undegraded":"2024-04-25T00:41:21.855011+0000","last_fullsized":"2024-04-25T00:41:21.855011+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:50.003262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853047+0000","last_change":"2024-04-25T00:41:20.738031+0000","last_active":"2024-04-25T00:41:21.853047+0000","last_peered":"2024-04-25T00:41:21.853047+0000","last_clean":"2024-04-25T00:41:21.853047+0000","last_became_active":"2024-04-25T00:41:20.737694+0000","last_became_peered":"2024-04-25T00:41:20.737694+0000","last_unstale":"2024-04-25T00:41:21.853047+0000","last_undegraded":"2024-04-25T00:41:21.853047+0000","last_fullsized":"2024-04-25T00:41:21.853047+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:12.354642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736461+0000","last_change":"2024-04-25T00:41:20.756463+0000","last_active":"2024-04-25T00:41:21.736461+0000","last_peered":"2024-04-25T00:41:21.736461+0000","last_clean":"2024-04-25T00:41:21.736461+0000","last_became_active":"2024-04-25T00:41:20.755935+0000","last_became_peered":"2024-04-25T00:41:20.755935+0000","last_unstale":"2024-04-25T00:41:21.736461+0000","last_undegraded":"2024-04-25T00:41:21.736461+0000","last_fullsized":"2024-04-25T00:41:21.736461+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:50.605977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852627+0000","last_change":"2024-04-25T00:41:20.730367+0000","last_active":"2024-04-25T00:41:21.852627+0000","last_peered":"2024-04-25T00:41:21.852627+0000","last_clean":"2024-04-25T00:41:21.852627+0000","last_became_active":"2024-04-25T00:41:20.729765+0000","last_became_peered":"2024-04-25T00:41:20.729765+0000","last_unstale":"2024-04-25T00:41:21.852627+0000","last_undegraded":"2024-04-25T00:41:21.852627+0000","last_fullsized":"2024-04-25T00:41:21.852627+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:29.081738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728881+0000","last_change":"2024-04-25T00:41:20.731164+0000","last_active":"2024-04-25T00:41:21.728881+0000","last_peered":"2024-04-25T00:41:21.728881+0000","last_clean":"2024-04-25T00:41:21.728881+0000","last_became_active":"2024-04-25T00:41:20.729720+0000","last_became_peered":"2024-04-25T00:41:20.729720+0000","last_unstale":"2024-04-25T00:41:21.728881+0000","last_undegraded":"2024-04-25T00:41:21.728881+0000","last_fullsized":"2024-04-25T00:41:21.728881+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:54.493042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853480+0000","last_change":"2024-04-25T00:41:20.728265+0000","last_active":"2024-04-25T00:41:21.853480+0000","last_peered":"2024-04-25T00:41:21.853480+0000","last_clean":"2024-04-25T00:41:21.853480+0000","last_became_active":"2024-04-25T00:41:20.727840+0000","last_became_peered":"2024-04-25T00:41:20.727840+0000","last_unstale":"2024-04-25T00:41:21.853480+0000","last_undegraded":"2024-04-25T00:41:21.853480+0000","last_fullsized":"2024-04-25T00:41:21.853480+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:25.159973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856794+0000","last_change":"2024-04-25T00:41:20.733820+0000","last_active":"2024-04-25T00:41:21.856794+0000","last_peered":"2024-04-25T00:41:21.856794+0000","last_clean":"2024-04-25T00:41:21.856794+0000","last_became_active":"2024-04-25T00:41:20.733516+0000","last_became_peered":"2024-04-25T00:41:20.733516+0000","last_unstale":"2024-04-25T00:41:21.856794+0000","last_undegraded":"2024-04-25T00:41:21.856794+0000","last_fullsized":"2024-04-25T00:41:21.856794+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:47.575779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.727368+0000","last_change":"2024-04-25T00:41:20.733108+0000","last_active":"2024-04-25T00:41:21.727368+0000","last_peered":"2024-04-25T00:41:21.727368+0000","last_clean":"2024-04-25T00:41:21.727368+0000","last_became_active":"2024-04-25T00:41:20.728236+0000","last_became_peered":"2024-04-25T00:41:20.728236+0000","last_unstale":"2024-04-25T00:41:21.727368+0000","last_undegraded":"2024-04-25T00:41:21.727368+0000","last_fullsized":"2024-04-25T00:41:21.727368+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:57.396464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.727972+0000","last_change":"2024-04-25T00:41:20.726309+0000","last_active":"2024-04-25T00:41:21.727972+0000","last_peered":"2024-04-25T00:41:21.727972+0000","last_clean":"2024-04-25T00:41:21.727972+0000","last_became_active":"2024-04-25T00:41:20.725444+0000","last_became_peered":"2024-04-25T00:41:20.725444+0000","last_unstale":"2024-04-25T00:41:21.727972+0000","last_undegraded":"2024-04-25T00:41:21.727972+0000","last_fullsized":"2024-04-25T00:41:21.727972+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:41.641469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736632+0000","last_change":"2024-04-25T00:41:20.727682+0000","last_active":"2024-04-25T00:41:21.736632+0000","last_peered":"2024-04-25T00:41:21.736632+0000","last_clean":"2024-04-25T00:41:21.736632+0000","last_became_active":"2024-04-25T00:41:20.727409+0000","last_became_peered":"2024-04-25T00:41:20.727409+0000","last_unstale":"2024-04-25T00:41:21.736632+0000","last_undegraded":"2024-04-25T00:41:21.736632+0000","last_fullsized":"2024-04-25T00:41:21.736632+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:48.999599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736447+0000","last_change":"2024-04-25T00:41:20.725702+0000","last_active":"2024-04-25T00:41:21.736447+0000","last_peered":"2024-04-25T00:41:21.736447+0000","last_clean":"2024-04-25T00:41:21.736447+0000","last_became_active":"2024-04-25T00:41:20.725313+0000","last_became_peered":"2024-04-25T00:41:20.725313+0000","last_unstale":"2024-04-25T00:41:21.736447+0000","last_undegraded":"2024-04-25T00:41:21.736447+0000","last_fullsized":"2024-04-25T00:41:21.736447+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:05.336967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852398+0000","last_change":"2024-04-25T00:41:20.734076+0000","last_active":"2024-04-25T00:41:21.852398+0000","last_peered":"2024-04-25T00:41:21.852398+0000","last_clean":"2024-04-25T00:41:21.852398+0000","last_became_active":"2024-04-25T00:41:20.733757+0000","last_became_peered":"2024-04-25T00:41:20.733757+0000","last_unstale":"2024-04-25T00:41:21.852398+0000","last_undegraded":"2024-04-25T00:41:21.852398+0000","last_fullsized":"2024-04-25T00:41:21.852398+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:16.226179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737048+0000","last_change":"2024-04-25T00:41:20.732582+0000","last_active":"2024-04-25T00:41:21.737048+0000","last_peered":"2024-04-25T00:41:21.737048+0000","last_clean":"2024-04-25T00:41:21.737048+0000","last_became_active":"2024-04-25T00:41:20.732276+0000","last_became_peered":"2024-04-25T00:41:20.732276+0000","last_unstale":"2024-04-25T00:41:21.737048+0000","last_undegraded":"2024-04-25T00:41:21.737048+0000","last_fullsized":"2024-04-25T00:41:21.737048+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:44.801392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853934+0000","last_change":"2024-04-25T00:41:20.729067+0000","last_active":"2024-04-25T00:41:21.853934+0000","last_peered":"2024-04-25T00:41:21.853934+0000","last_clean":"2024-04-25T00:41:21.853934+0000","last_became_active":"2024-04-25T00:41:20.728765+0000","last_became_peered":"2024-04-25T00:41:20.728765+0000","last_unstale":"2024-04-25T00:41:21.853934+0000","last_undegraded":"2024-04-25T00:41:21.853934+0000","last_fullsized":"2024-04-25T00:41:21.853934+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:31.971109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854502+0000","last_change":"2024-04-25T00:41:20.728640+0000","last_active":"2024-04-25T00:41:21.854502+0000","last_peered":"2024-04-25T00:41:21.854502+0000","last_clean":"2024-04-25T00:41:21.854502+0000","last_became_active":"2024-04-25T00:41:20.728290+0000","last_became_peered":"2024-04-25T00:41:20.728290+0000","last_unstale":"2024-04-25T00:41:21.854502+0000","last_undegraded":"2024-04-25T00:41:21.854502+0000","last_fullsized":"2024-04-25T00:41:21.854502+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:05.336967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855556+0000","last_change":"2024-04-25T00:41:20.729925+0000","last_active":"2024-04-25T00:41:21.855556+0000","last_peered":"2024-04-25T00:41:21.855556+0000","last_clean":"2024-04-25T00:41:21.855556+0000","last_became_active":"2024-04-25T00:41:20.729597+0000","last_became_peered":"2024-04-25T00:41:20.729597+0000","last_unstale":"2024-04-25T00:41:21.855556+0000","last_undegraded":"2024-04-25T00:41:21.855556+0000","last_fullsized":"2024-04-25T00:41:21.855556+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:27.879965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725550+0000","last_change":"2024-04-25T00:41:20.724283+0000","last_active":"2024-04-25T00:41:21.725550+0000","last_peered":"2024-04-25T00:41:21.725550+0000","last_clean":"2024-04-25T00:41:21.725550+0000","last_became_active":"2024-04-25T00:41:20.723569+0000","last_became_peered":"2024-04-25T00:41:20.723569+0000","last_unstale":"2024-04-25T00:41:21.725550+0000","last_undegraded":"2024-04-25T00:41:21.725550+0000","last_fullsized":"2024-04-25T00:41:21.725550+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:19.901587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852904+0000","last_change":"2024-04-25T00:41:20.746173+0000","last_active":"2024-04-25T00:41:21.852904+0000","last_peered":"2024-04-25T00:41:21.852904+0000","last_clean":"2024-04-25T00:41:21.852904+0000","last_became_active":"2024-04-25T00:41:20.745450+0000","last_became_peered":"2024-04-25T00:41:20.745450+0000","last_unstale":"2024-04-25T00:41:21.852904+0000","last_undegraded":"2024-04-25T00:41:21.852904+0000","last_fullsized":"2024-04-25T00:41:21.852904+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:12.354642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852684+0000","last_change":"2024-04-25T00:41:20.752770+0000","last_active":"2024-04-25T00:41:21.852684+0000","last_peered":"2024-04-25T00:41:21.852684+0000","last_clean":"2024-04-25T00:41:21.852684+0000","last_became_active":"2024-04-25T00:41:20.751527+0000","last_became_peered":"2024-04-25T00:41:20.751527+0000","last_unstale":"2024-04-25T00:41:21.852684+0000","last_undegraded":"2024-04-25T00:41:21.852684+0000","last_fullsized":"2024-04-25T00:41:21.852684+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:29.081738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724561+0000","last_change":"2024-04-25T00:41:20.726726+0000","last_active":"2024-04-25T00:41:21.724561+0000","last_peered":"2024-04-25T00:41:21.724561+0000","last_clean":"2024-04-25T00:41:21.724561+0000","last_became_active":"2024-04-25T00:41:20.726043+0000","last_became_peered":"2024-04-25T00:41:20.726043+0000","last_unstale":"2024-04-25T00:41:21.724561+0000","last_undegraded":"2024-04-25T00:41:21.724561+0000","last_fullsized":"2024-04-25T00:41:21.724561+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:23.550089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728079+0000","last_change":"2024-04-25T00:41:20.742119+0000","last_active":"2024-04-25T00:41:21.728079+0000","last_peered":"2024-04-25T00:41:21.728079+0000","last_clean":"2024-04-25T00:41:21.728079+0000","last_became_active":"2024-04-25T00:41:20.739704+0000","last_became_peered":"2024-04-25T00:41:20.739704+0000","last_unstale":"2024-04-25T00:41:21.728079+0000","last_undegraded":"2024-04-25T00:41:21.728079+0000","last_fullsized":"2024-04-25T00:41:21.728079+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:05.066385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736778+0000","last_change":"2024-04-25T00:41:20.756207+0000","last_active":"2024-04-25T00:41:21.736778+0000","last_peered":"2024-04-25T00:41:21.736778+0000","last_clean":"2024-04-25T00:41:21.736778+0000","last_became_active":"2024-04-25T00:41:20.755848+0000","last_became_peered":"2024-04-25T00:41:20.755848+0000","last_unstale":"2024-04-25T00:41:21.736778+0000","last_undegraded":"2024-04-25T00:41:21.736778+0000","last_fullsized":"2024-04-25T00:41:21.736778+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:04.914485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.727995+0000","last_change":"2024-04-25T00:41:20.728570+0000","last_active":"2024-04-25T00:41:21.727995+0000","last_peered":"2024-04-25T00:41:21.727995+0000","last_clean":"2024-04-25T00:41:21.727995+0000","last_became_active":"2024-04-25T00:41:20.727806+0000","last_became_peered":"2024-04-25T00:41:20.727806+0000","last_unstale":"2024-04-25T00:41:21.727995+0000","last_undegraded":"2024-04-25T00:41:21.727995+0000","last_fullsized":"2024-04-25T00:41:21.727995+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:39.749407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856704+0000","last_change":"2024-04-25T00:41:20.730663+0000","last_active":"2024-04-25T00:41:21.856704+0000","last_peered":"2024-04-25T00:41:21.856704+0000","last_clean":"2024-04-25T00:41:21.856704+0000","last_became_active":"2024-04-25T00:41:20.730330+0000","last_became_peered":"2024-04-25T00:41:20.730330+0000","last_unstale":"2024-04-25T00:41:21.856704+0000","last_undegraded":"2024-04-25T00:41:21.856704+0000","last_fullsized":"2024-04-25T00:41:21.856704+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:52.725891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.726305+0000","last_change":"2024-04-25T00:41:20.728144+0000","last_active":"2024-04-25T00:41:21.726305+0000","last_peered":"2024-04-25T00:41:21.726305+0000","last_clean":"2024-04-25T00:41:21.726305+0000","last_became_active":"2024-04-25T00:41:20.727266+0000","last_became_peered":"2024-04-25T00:41:20.727266+0000","last_unstale":"2024-04-25T00:41:21.726305+0000","last_undegraded":"2024-04-25T00:41:21.726305+0000","last_fullsized":"2024-04-25T00:41:21.726305+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:32.736055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.726950+0000","last_change":"2024-04-25T00:41:20.730975+0000","last_active":"2024-04-25T00:41:21.726950+0000","last_peered":"2024-04-25T00:41:21.726950+0000","last_clean":"2024-04-25T00:41:21.726950+0000","last_became_active":"2024-04-25T00:41:20.730142+0000","last_became_peered":"2024-04-25T00:41:20.730142+0000","last_unstale":"2024-04-25T00:41:21.726950+0000","last_undegraded":"2024-04-25T00:41:21.726950+0000","last_fullsized":"2024-04-25T00:41:21.726950+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:47.575779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852645+0000","last_change":"2024-04-25T00:41:20.723764+0000","last_active":"2024-04-25T00:41:21.852645+0000","last_peered":"2024-04-25T00:41:21.852645+0000","last_clean":"2024-04-25T00:41:21.852645+0000","last_became_active":"2024-04-25T00:41:20.723442+0000","last_became_peered":"2024-04-25T00:41:20.723442+0000","last_unstale":"2024-04-25T00:41:21.852645+0000","last_undegraded":"2024-04-25T00:41:21.852645+0000","last_fullsized":"2024-04-25T00:41:21.852645+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:19.901587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736358+0000","last_change":"2024-04-25T00:41:20.725607+0000","last_active":"2024-04-25T00:41:21.736358+0000","last_peered":"2024-04-25T00:41:21.736358+0000","last_clean":"2024-04-25T00:41:21.736358+0000","last_became_active":"2024-04-25T00:41:20.725195+0000","last_became_peered":"2024-04-25T00:41:20.725195+0000","last_unstale":"2024-04-25T00:41:21.736358+0000","last_undegraded":"2024-04-25T00:41:21.736358+0000","last_fullsized":"2024-04-25T00:41:21.736358+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:55.977495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728742+0000","last_change":"2024-04-25T00:41:20.733094+0000","last_active":"2024-04-25T00:41:21.728742+0000","last_peered":"2024-04-25T00:41:21.728742+0000","last_clean":"2024-04-25T00:41:21.728742+0000","last_became_active":"2024-04-25T00:41:20.732348+0000","last_became_peered":"2024-04-25T00:41:20.732348+0000","last_unstale":"2024-04-25T00:41:21.728742+0000","last_undegraded":"2024-04-25T00:41:21.728742+0000","last_fullsized":"2024-04-25T00:41:21.728742+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:57.037464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"167'4","reported_seq":18,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729495+0000","last_change":"2024-04-25T00:41:19.730561+0000","last_active":"2024-04-25T00:41:21.729495+0000","last_peered":"2024-04-25T00:41:21.729495+0000","last_clean":"2024-04-25T00:41:21.729495+0000","last_became_active":"2024-04-25T00:41:19.729475+0000","last_became_peered":"2024-04-25T00:41:19.729475+0000","last_unstale":"2024-04-25T00:41:21.729495+0000","last_undegraded":"2024-04-25T00:41:21.729495+0000","last_fullsized":"2024-04-25T00:41:21.729495+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:58.727637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853286+0000","last_change":"2024-04-25T00:41:20.754267+0000","last_active":"2024-04-25T00:41:21.853286+0000","last_peered":"2024-04-25T00:41:21.853286+0000","last_clean":"2024-04-25T00:41:21.853286+0000","last_became_active":"2024-04-25T00:41:20.753526+0000","last_became_peered":"2024-04-25T00:41:20.753526+0000","last_unstale":"2024-04-25T00:41:21.853286+0000","last_undegraded":"2024-04-25T00:41:21.853286+0000","last_fullsized":"2024-04-25T00:41:21.853286+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:27.879965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729086+0000","last_change":"2024-04-25T00:41:19.728196+0000","last_active":"2024-04-25T00:41:21.729086+0000","last_peered":"2024-04-25T00:41:21.729086+0000","last_clean":"2024-04-25T00:41:21.729086+0000","last_became_active":"2024-04-25T00:41:19.727469+0000","last_became_peered":"2024-04-25T00:41:19.727469+0000","last_unstale":"2024-04-25T00:41:21.729086+0000","last_undegraded":"2024-04-25T00:41:21.729086+0000","last_fullsized":"2024-04-25T00:41:21.729086+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:18.103810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736679+0000","last_change":"2024-04-25T00:41:20.757686+0000","last_active":"2024-04-25T00:41:21.736679+0000","last_peered":"2024-04-25T00:41:21.736679+0000","last_clean":"2024-04-25T00:41:21.736679+0000","last_became_active":"2024-04-25T00:41:20.757425+0000","last_became_peered":"2024-04-25T00:41:20.757425+0000","last_unstale":"2024-04-25T00:41:21.736679+0000","last_undegraded":"2024-04-25T00:41:21.736679+0000","last_fullsized":"2024-04-25T00:41:21.736679+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:29.413633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856910+0000","last_change":"2024-04-25T00:41:19.714844+0000","last_active":"2024-04-25T00:41:21.856910+0000","last_peered":"2024-04-25T00:41:21.856910+0000","last_clean":"2024-04-25T00:41:21.856910+0000","last_became_active":"2024-04-25T00:41:19.714584+0000","last_became_peered":"2024-04-25T00:41:19.714584+0000","last_unstale":"2024-04-25T00:41:21.856910+0000","last_undegraded":"2024-04-25T00:41:21.856910+0000","last_fullsized":"2024-04-25T00:41:21.856910+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:17.955930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854015+0000","last_change":"2024-04-25T00:41:20.729450+0000","last_active":"2024-04-25T00:41:21.854015+0000","last_peered":"2024-04-25T00:41:21.854015+0000","last_clean":"2024-04-25T00:41:21.854015+0000","last_became_active":"2024-04-25T00:41:20.728986+0000","last_became_peered":"2024-04-25T00:41:20.728986+0000","last_unstale":"2024-04-25T00:41:21.854015+0000","last_undegraded":"2024-04-25T00:41:21.854015+0000","last_fullsized":"2024-04-25T00:41:21.854015+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:44.845619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725827+0000","last_change":"2024-04-25T00:41:19.727073+0000","last_active":"2024-04-25T00:41:21.725827+0000","last_peered":"2024-04-25T00:41:21.725827+0000","last_clean":"2024-04-25T00:41:21.725827+0000","last_became_active":"2024-04-25T00:41:19.726232+0000","last_became_peered":"2024-04-25T00:41:19.726232+0000","last_unstale":"2024-04-25T00:41:21.725827+0000","last_undegraded":"2024-04-25T00:41:21.725827+0000","last_fullsized":"2024-04-25T00:41:21.725827+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:12.458015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724621+0000","last_change":"2024-04-25T00:41:20.754166+0000","last_active":"2024-04-25T00:41:21.724621+0000","last_peered":"2024-04-25T00:41:21.724621+0000","last_clean":"2024-04-25T00:41:21.724621+0000","last_became_active":"2024-04-25T00:41:20.753650+0000","last_became_peered":"2024-04-25T00:41:20.753650+0000","last_unstale":"2024-04-25T00:41:21.724621+0000","last_undegraded":"2024-04-25T00:41:21.724621+0000","last_fullsized":"2024-04-25T00:41:21.724621+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:17.802309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853585+0000","last_change":"2024-04-25T00:41:19.723133+0000","last_active":"2024-04-25T00:41:21.853585+0000","last_peered":"2024-04-25T00:41:21.853585+0000","last_clean":"2024-04-25T00:41:21.853585+0000","last_became_active":"2024-04-25T00:41:19.722748+0000","last_became_peered":"2024-04-25T00:41:19.722748+0000","last_unstale":"2024-04-25T00:41:21.853585+0000","last_undegraded":"2024-04-25T00:41:21.853585+0000","last_fullsized":"2024-04-25T00:41:21.853585+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:53.480966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852317+0000","last_change":"2024-04-25T00:41:20.720406+0000","last_active":"2024-04-25T00:41:21.852317+0000","last_peered":"2024-04-25T00:41:21.852317+0000","last_clean":"2024-04-25T00:41:21.852317+0000","last_became_active":"2024-04-25T00:41:20.720072+0000","last_became_peered":"2024-04-25T00:41:20.720072+0000","last_unstale":"2024-04-25T00:41:21.852317+0000","last_undegraded":"2024-04-25T00:41:21.852317+0000","last_fullsized":"2024-04-25T00:41:21.852317+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:16.067964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852079+0000","last_change":"2024-04-25T00:41:19.712612+0000","last_active":"2024-04-25T00:41:21.852079+0000","last_peered":"2024-04-25T00:41:21.852079+0000","last_clean":"2024-04-25T00:41:21.852079+0000","last_became_active":"2024-04-25T00:41:19.710382+0000","last_became_peered":"2024-04-25T00:41:19.710382+0000","last_unstale":"2024-04-25T00:41:21.852079+0000","last_undegraded":"2024-04-25T00:41:21.852079+0000","last_fullsized":"2024-04-25T00:41:21.852079+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:41.339940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725041+0000","last_change":"2024-04-25T00:41:20.755342+0000","last_active":"2024-04-25T00:41:21.725041+0000","last_peered":"2024-04-25T00:41:21.725041+0000","last_clean":"2024-04-25T00:41:21.725041+0000","last_became_active":"2024-04-25T00:41:20.754876+0000","last_became_peered":"2024-04-25T00:41:20.754876+0000","last_unstale":"2024-04-25T00:41:21.725041+0000","last_undegraded":"2024-04-25T00:41:21.725041+0000","last_fullsized":"2024-04-25T00:41:21.725041+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:16.226179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852140+0000","last_change":"2024-04-25T00:41:19.723756+0000","last_active":"2024-04-25T00:41:21.852140+0000","last_peered":"2024-04-25T00:41:21.852140+0000","last_clean":"2024-04-25T00:41:21.852140+0000","last_became_active":"2024-04-25T00:41:19.722504+0000","last_became_peered":"2024-04-25T00:41:19.722504+0000","last_unstale":"2024-04-25T00:41:21.852140+0000","last_undegraded":"2024-04-25T00:41:21.852140+0000","last_fullsized":"2024-04-25T00:41:21.852140+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:53.480966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728998+0000","last_change":"2024-04-25T00:41:20.715594+0000","last_active":"2024-04-25T00:41:21.728998+0000","last_peered":"2024-04-25T00:41:21.728998+0000","last_clean":"2024-04-25T00:41:21.728998+0000","last_became_active":"2024-04-25T00:41:20.712098+0000","last_became_peered":"2024-04-25T00:41:20.712098+0000","last_unstale":"2024-04-25T00:41:21.728998+0000","last_undegraded":"2024-04-25T00:41:21.728998+0000","last_fullsized":"2024-04-25T00:41:21.728998+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:53.784754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855238+0000","last_change":"2024-04-25T00:41:19.727211+0000","last_active":"2024-04-25T00:41:21.855238+0000","last_peered":"2024-04-25T00:41:21.855238+0000","last_clean":"2024-04-25T00:41:21.855238+0000","last_became_active":"2024-04-25T00:41:19.726926+0000","last_became_peered":"2024-04-25T00:41:19.726926+0000","last_unstale":"2024-04-25T00:41:21.855238+0000","last_undegraded":"2024-04-25T00:41:21.855238+0000","last_fullsized":"2024-04-25T00:41:21.855238+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:04.054309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856661+0000","last_change":"2024-04-25T00:41:20.723273+0000","last_active":"2024-04-25T00:41:21.856661+0000","last_peered":"2024-04-25T00:41:21.856661+0000","last_clean":"2024-04-25T00:41:21.856661+0000","last_became_active":"2024-04-25T00:41:20.722938+0000","last_became_peered":"2024-04-25T00:41:20.722938+0000","last_unstale":"2024-04-25T00:41:21.856661+0000","last_undegraded":"2024-04-25T00:41:21.856661+0000","last_fullsized":"2024-04-25T00:41:21.856661+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:29.975327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853121+0000","last_change":"2024-04-25T00:41:19.733298+0000","last_active":"2024-04-25T00:41:21.853121+0000","last_peered":"2024-04-25T00:41:21.853121+0000","last_clean":"2024-04-25T00:41:21.853121+0000","last_became_active":"2024-04-25T00:41:19.732514+0000","last_became_peered":"2024-04-25T00:41:19.732514+0000","last_unstale":"2024-04-25T00:41:21.853121+0000","last_undegraded":"2024-04-25T00:41:21.853121+0000","last_fullsized":"2024-04-25T00:41:21.853121+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:56.384389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724912+0000","last_change":"2024-04-25T00:41:20.716394+0000","last_active":"2024-04-25T00:41:21.724912+0000","last_peered":"2024-04-25T00:41:21.724912+0000","last_clean":"2024-04-25T00:41:21.724912+0000","last_became_active":"2024-04-25T00:41:20.715624+0000","last_became_peered":"2024-04-25T00:41:20.715624+0000","last_unstale":"2024-04-25T00:41:21.724912+0000","last_undegraded":"2024-04-25T00:41:21.724912+0000","last_fullsized":"2024-04-25T00:41:21.724912+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:23.460044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"167'3","reported_seq":17,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854498+0000","last_change":"2024-04-25T00:41:19.714290+0000","last_active":"2024-04-25T00:41:21.854498+0000","last_peered":"2024-04-25T00:41:21.854498+0000","last_clean":"2024-04-25T00:41:21.854498+0000","last_became_active":"2024-04-25T00:41:19.713997+0000","last_became_peered":"2024-04-25T00:41:19.713997+0000","last_unstale":"2024-04-25T00:41:21.854498+0000","last_undegraded":"2024-04-25T00:41:21.854498+0000","last_fullsized":"2024-04-25T00:41:21.854498+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:47.272504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737071+0000","last_change":"2024-04-25T00:41:20.729697+0000","last_active":"2024-04-25T00:41:21.737071+0000","last_peered":"2024-04-25T00:41:21.737071+0000","last_clean":"2024-04-25T00:41:21.737071+0000","last_became_active":"2024-04-25T00:41:20.728724+0000","last_became_peered":"2024-04-25T00:41:20.728724+0000","last_unstale":"2024-04-25T00:41:21.737071+0000","last_undegraded":"2024-04-25T00:41:21.737071+0000","last_fullsized":"2024-04-25T00:41:21.737071+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:54.350976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"167'6","reported_seq":20,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.726112+0000","last_change":"2024-04-25T00:41:19.726821+0000","last_active":"2024-04-25T00:41:21.726112+0000","last_peered":"2024-04-25T00:41:21.726112+0000","last_clean":"2024-04-25T00:41:21.726112+0000","last_became_active":"2024-04-25T00:41:19.726146+0000","last_became_peered":"2024-04-25T00:41:19.726146+0000","last_unstale":"2024-04-25T00:41:21.726112+0000","last_undegraded":"2024-04-25T00:41:21.726112+0000","last_fullsized":"2024-04-25T00:41:21.726112+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:59.876465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853444+0000","last_change":"2024-04-25T00:41:20.731223+0000","last_active":"2024-04-25T00:41:21.853444+0000","last_peered":"2024-04-25T00:41:21.853444+0000","last_clean":"2024-04-25T00:41:21.853444+0000","last_became_active":"2024-04-25T00:41:20.730734+0000","last_became_peered":"2024-04-25T00:41:20.730734+0000","last_unstale":"2024-04-25T00:41:21.853444+0000","last_undegraded":"2024-04-25T00:41:21.853444+0000","last_fullsized":"2024-04-25T00:41:21.853444+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:21.777831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852468+0000","last_change":"2024-04-25T00:41:19.715184+0000","last_active":"2024-04-25T00:41:21.852468+0000","last_peered":"2024-04-25T00:41:21.852468+0000","last_clean":"2024-04-25T00:41:21.852468+0000","last_became_active":"2024-04-25T00:41:19.713383+0000","last_became_peered":"2024-04-25T00:41:19.713383+0000","last_unstale":"2024-04-25T00:41:21.852468+0000","last_undegraded":"2024-04-25T00:41:21.852468+0000","last_fullsized":"2024-04-25T00:41:21.852468+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:41.339940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853053+0000","last_change":"2024-04-25T00:41:20.726316+0000","last_active":"2024-04-25T00:41:21.853053+0000","last_peered":"2024-04-25T00:41:21.853053+0000","last_clean":"2024-04-25T00:41:21.853053+0000","last_became_active":"2024-04-25T00:41:20.724262+0000","last_became_peered":"2024-04-25T00:41:20.724262+0000","last_unstale":"2024-04-25T00:41:21.853053+0000","last_undegraded":"2024-04-25T00:41:21.853053+0000","last_fullsized":"2024-04-25T00:41:21.853053+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:53.784754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852580+0000","last_change":"2024-04-25T00:41:19.724753+0000","last_active":"2024-04-25T00:41:21.852580+0000","last_peered":"2024-04-25T00:41:21.852580+0000","last_clean":"2024-04-25T00:41:21.852580+0000","last_became_active":"2024-04-25T00:41:19.721745+0000","last_became_peered":"2024-04-25T00:41:19.721745+0000","last_unstale":"2024-04-25T00:41:21.852580+0000","last_undegraded":"2024-04-25T00:41:21.852580+0000","last_fullsized":"2024-04-25T00:41:21.852580+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:31.723980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856313+0000","last_change":"2024-04-25T00:41:20.727156+0000","last_active":"2024-04-25T00:41:21.856313+0000","last_peered":"2024-04-25T00:41:21.856313+0000","last_clean":"2024-04-25T00:41:21.856313+0000","last_became_active":"2024-04-25T00:41:20.725938+0000","last_became_peered":"2024-04-25T00:41:20.725938+0000","last_unstale":"2024-04-25T00:41:21.856313+0000","last_undegraded":"2024-04-25T00:41:21.856313+0000","last_fullsized":"2024-04-25T00:41:21.856313+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:03.298965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852561+0000","last_change":"2024-04-25T00:41:19.725909+0000","last_active":"2024-04-25T00:41:21.852561+0000","last_peered":"2024-04-25T00:41:21.852561+0000","last_clean":"2024-04-25T00:41:21.852561+0000","last_became_active":"2024-04-25T00:41:19.725301+0000","last_became_peered":"2024-04-25T00:41:19.725301+0000","last_unstale":"2024-04-25T00:41:21.852561+0000","last_undegraded":"2024-04-25T00:41:21.852561+0000","last_fullsized":"2024-04-25T00:41:21.852561+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:59.876465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736566+0000","last_change":"2024-04-25T00:41:20.746071+0000","last_active":"2024-04-25T00:41:21.736566+0000","last_peered":"2024-04-25T00:41:21.736566+0000","last_clean":"2024-04-25T00:41:21.736566+0000","last_became_active":"2024-04-25T00:41:20.745714+0000","last_became_peered":"2024-04-25T00:41:20.745714+0000","last_unstale":"2024-04-25T00:41:21.736566+0000","last_undegraded":"2024-04-25T00:41:21.736566+0000","last_fullsized":"2024-04-25T00:41:21.736566+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:10.415430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"167'8","reported_seq":22,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855744+0000","last_change":"2024-04-25T00:41:19.720510+0000","last_active":"2024-04-25T00:41:21.855744+0000","last_peered":"2024-04-25T00:41:21.855744+0000","last_clean":"2024-04-25T00:41:21.855744+0000","last_became_active":"2024-04-25T00:41:19.720185+0000","last_became_peered":"2024-04-25T00:41:19.720185+0000","last_unstale":"2024-04-25T00:41:21.855744+0000","last_undegraded":"2024-04-25T00:41:21.855744+0000","last_fullsized":"2024-04-25T00:41:21.855744+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:50.914418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729150+0000","last_change":"2024-04-25T00:41:20.710672+0000","last_active":"2024-04-25T00:41:21.729150+0000","last_peered":"2024-04-25T00:41:21.729150+0000","last_clean":"2024-04-25T00:41:21.729150+0000","last_became_active":"2024-04-25T00:41:20.709976+0000","last_became_peered":"2024-04-25T00:41:20.709976+0000","last_unstale":"2024-04-25T00:41:21.729150+0000","last_undegraded":"2024-04-25T00:41:21.729150+0000","last_fullsized":"2024-04-25T00:41:21.729150+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:45.163125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855527+0000","last_change":"2024-04-25T00:41:19.721839+0000","last_active":"2024-04-25T00:41:21.855527+0000","last_peered":"2024-04-25T00:41:21.855527+0000","last_clean":"2024-04-25T00:41:21.855527+0000","last_became_active":"2024-04-25T00:41:19.721547+0000","last_became_peered":"2024-04-25T00:41:19.721547+0000","last_unstale":"2024-04-25T00:41:21.855527+0000","last_undegraded":"2024-04-25T00:41:21.855527+0000","last_fullsized":"2024-04-25T00:41:21.855527+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:00.898156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728440+0000","last_change":"2024-04-25T00:41:20.729031+0000","last_active":"2024-04-25T00:41:21.728440+0000","last_peered":"2024-04-25T00:41:21.728440+0000","last_clean":"2024-04-25T00:41:21.728440+0000","last_became_active":"2024-04-25T00:41:20.724259+0000","last_became_peered":"2024-04-25T00:41:20.724259+0000","last_unstale":"2024-04-25T00:41:21.728440+0000","last_undegraded":"2024-04-25T00:41:21.728440+0000","last_fullsized":"2024-04-25T00:41:21.728440+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:02.403416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729314+0000","last_change":"2024-04-25T00:41:19.707832+0000","last_active":"2024-04-25T00:41:21.729314+0000","last_peered":"2024-04-25T00:41:21.729314+0000","last_clean":"2024-04-25T00:41:21.729314+0000","last_became_active":"2024-04-25T00:41:19.706600+0000","last_became_peered":"2024-04-25T00:41:19.706600+0000","last_unstale":"2024-04-25T00:41:21.729314+0000","last_undegraded":"2024-04-25T00:41:21.729314+0000","last_fullsized":"2024-04-25T00:41:21.729314+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:09.353188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855887+0000","last_change":"2024-04-25T00:41:20.736978+0000","last_active":"2024-04-25T00:41:21.855887+0000","last_peered":"2024-04-25T00:41:21.855887+0000","last_clean":"2024-04-25T00:41:21.855887+0000","last_became_active":"2024-04-25T00:41:20.736654+0000","last_became_peered":"2024-04-25T00:41:20.736654+0000","last_unstale":"2024-04-25T00:41:21.855887+0000","last_undegraded":"2024-04-25T00:41:21.855887+0000","last_fullsized":"2024-04-25T00:41:21.855887+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:14.858055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854660+0000","last_change":"2024-04-25T00:41:19.724832+0000","last_active":"2024-04-25T00:41:21.854660+0000","last_peered":"2024-04-25T00:41:21.854660+0000","last_clean":"2024-04-25T00:41:21.854660+0000","last_became_active":"2024-04-25T00:41:19.724496+0000","last_became_peered":"2024-04-25T00:41:19.724496+0000","last_unstale":"2024-04-25T00:41:21.854660+0000","last_undegraded":"2024-04-25T00:41:21.854660+0000","last_fullsized":"2024-04-25T00:41:21.854660+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:25.361018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854330+0000","last_change":"2024-04-25T00:41:20.727581+0000","last_active":"2024-04-25T00:41:21.854330+0000","last_peered":"2024-04-25T00:41:21.854330+0000","last_clean":"2024-04-25T00:41:21.854330+0000","last_became_active":"2024-04-25T00:41:20.723303+0000","last_became_peered":"2024-04-25T00:41:20.723303+0000","last_unstale":"2024-04-25T00:41:21.854330+0000","last_undegraded":"2024-04-25T00:41:21.854330+0000","last_fullsized":"2024-04-25T00:41:21.854330+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:55.977495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856037+0000","last_change":"2024-04-25T00:41:19.720256+0000","last_active":"2024-04-25T00:41:21.856037+0000","last_peered":"2024-04-25T00:41:21.856037+0000","last_clean":"2024-04-25T00:41:21.856037+0000","last_became_active":"2024-04-25T00:41:19.719906+0000","last_became_peered":"2024-04-25T00:41:19.719906+0000","last_unstale":"2024-04-25T00:41:21.856037+0000","last_undegraded":"2024-04-25T00:41:21.856037+0000","last_fullsized":"2024-04-25T00:41:21.856037+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:58.692958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853309+0000","last_change":"2024-04-25T00:41:20.727981+0000","last_active":"2024-04-25T00:41:21.853309+0000","last_peered":"2024-04-25T00:41:21.853309+0000","last_clean":"2024-04-25T00:41:21.853309+0000","last_became_active":"2024-04-25T00:41:20.727650+0000","last_became_peered":"2024-04-25T00:41:20.727650+0000","last_unstale":"2024-04-25T00:41:21.853309+0000","last_undegraded":"2024-04-25T00:41:21.853309+0000","last_fullsized":"2024-04-25T00:41:21.853309+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:24.427571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852863+0000","last_change":"2024-04-25T00:41:19.725591+0000","last_active":"2024-04-25T00:41:21.852863+0000","last_peered":"2024-04-25T00:41:21.852863+0000","last_clean":"2024-04-25T00:41:21.852863+0000","last_became_active":"2024-04-25T00:41:19.725213+0000","last_became_peered":"2024-04-25T00:41:19.725213+0000","last_unstale":"2024-04-25T00:41:21.852863+0000","last_undegraded":"2024-04-25T00:41:21.852863+0000","last_fullsized":"2024-04-25T00:41:21.852863+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:35.912276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856172+0000","last_change":"2024-04-25T00:41:20.728184+0000","last_active":"2024-04-25T00:41:21.856172+0000","last_peered":"2024-04-25T00:41:21.856172+0000","last_clean":"2024-04-25T00:41:21.856172+0000","last_became_active":"2024-04-25T00:41:20.726395+0000","last_became_peered":"2024-04-25T00:41:20.726395+0000","last_unstale":"2024-04-25T00:41:21.856172+0000","last_undegraded":"2024-04-25T00:41:21.856172+0000","last_fullsized":"2024-04-25T00:41:21.856172+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:05.020843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737304+0000","last_change":"2024-04-25T00:41:19.726228+0000","last_active":"2024-04-25T00:41:21.737304+0000","last_peered":"2024-04-25T00:41:21.737304+0000","last_clean":"2024-04-25T00:41:21.737304+0000","last_became_active":"2024-04-25T00:41:19.725747+0000","last_became_peered":"2024-04-25T00:41:19.725747+0000","last_unstale":"2024-04-25T00:41:21.737304+0000","last_undegraded":"2024-04-25T00:41:21.737304+0000","last_fullsized":"2024-04-25T00:41:21.737304+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:00.329987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724987+0000","last_change":"2024-04-25T00:41:20.747974+0000","last_active":"2024-04-25T00:41:21.724987+0000","last_peered":"2024-04-25T00:41:21.724987+0000","last_clean":"2024-04-25T00:41:21.724987+0000","last_became_active":"2024-04-25T00:41:20.747202+0000","last_became_peered":"2024-04-25T00:41:20.747202+0000","last_unstale":"2024-04-25T00:41:21.724987+0000","last_undegraded":"2024-04-25T00:41:21.724987+0000","last_fullsized":"2024-04-25T00:41:21.724987+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:52.725891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.736508+0000","last_change":"2024-04-25T00:41:19.724386+0000","last_active":"2024-04-25T00:41:21.736508+0000","last_peered":"2024-04-25T00:41:21.736508+0000","last_clean":"2024-04-25T00:41:21.736508+0000","last_became_active":"2024-04-25T00:41:19.724122+0000","last_became_peered":"2024-04-25T00:41:19.724122+0000","last_unstale":"2024-04-25T00:41:21.736508+0000","last_undegraded":"2024-04-25T00:41:21.736508+0000","last_fullsized":"2024-04-25T00:41:21.736508+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:58.585739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852918+0000","last_change":"2024-04-25T00:41:20.751369+0000","last_active":"2024-04-25T00:41:21.852918+0000","last_peered":"2024-04-25T00:41:21.852918+0000","last_clean":"2024-04-25T00:41:21.852918+0000","last_became_active":"2024-04-25T00:41:20.746068+0000","last_became_peered":"2024-04-25T00:41:20.746068+0000","last_unstale":"2024-04-25T00:41:21.852918+0000","last_undegraded":"2024-04-25T00:41:21.852918+0000","last_fullsized":"2024-04-25T00:41:21.852918+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:21.777831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852447+0000","last_change":"2024-04-25T00:41:19.729543+0000","last_active":"2024-04-25T00:41:21.852447+0000","last_peered":"2024-04-25T00:41:21.852447+0000","last_clean":"2024-04-25T00:41:21.852447+0000","last_became_active":"2024-04-25T00:41:19.729146+0000","last_became_peered":"2024-04-25T00:41:19.729146+0000","last_unstale":"2024-04-25T00:41:21.852447+0000","last_undegraded":"2024-04-25T00:41:21.852447+0000","last_fullsized":"2024-04-25T00:41:21.852447+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:56.025389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728090+0000","last_change":"2024-04-25T00:41:20.732303+0000","last_active":"2024-04-25T00:41:21.728090+0000","last_peered":"2024-04-25T00:41:21.728090+0000","last_clean":"2024-04-25T00:41:21.728090+0000","last_became_active":"2024-04-25T00:41:20.727064+0000","last_became_peered":"2024-04-25T00:41:20.727064+0000","last_unstale":"2024-04-25T00:41:21.728090+0000","last_undegraded":"2024-04-25T00:41:21.728090+0000","last_fullsized":"2024-04-25T00:41:21.728090+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:29.644405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"167'2","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856443+0000","last_change":"2024-04-25T00:41:19.711806+0000","last_active":"2024-04-25T00:41:21.856443+0000","last_peered":"2024-04-25T00:41:21.856443+0000","last_clean":"2024-04-25T00:41:21.856443+0000","last_became_active":"2024-04-25T00:41:19.711478+0000","last_became_peered":"2024-04-25T00:41:19.711478+0000","last_unstale":"2024-04-25T00:41:21.856443+0000","last_undegraded":"2024-04-25T00:41:21.856443+0000","last_fullsized":"2024-04-25T00:41:21.856443+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:25.679848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737205+0000","last_change":"2024-04-25T00:41:20.729141+0000","last_active":"2024-04-25T00:41:21.737205+0000","last_peered":"2024-04-25T00:41:21.737205+0000","last_clean":"2024-04-25T00:41:21.737205+0000","last_became_active":"2024-04-25T00:41:20.728844+0000","last_became_peered":"2024-04-25T00:41:20.728844+0000","last_unstale":"2024-04-25T00:41:21.737205+0000","last_undegraded":"2024-04-25T00:41:21.737205+0000","last_fullsized":"2024-04-25T00:41:21.737205+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:09.108727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"168'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.729185+0000","last_change":"2024-04-25T00:41:19.726585+0000","last_active":"2024-04-25T00:41:21.729185+0000","last_peered":"2024-04-25T00:41:21.729185+0000","last_clean":"2024-04-25T00:41:21.729185+0000","last_became_active":"2024-04-25T00:41:19.725579+0000","last_became_peered":"2024-04-25T00:41:19.725579+0000","last_unstale":"2024-04-25T00:41:21.729185+0000","last_undegraded":"2024-04-25T00:41:21.729185+0000","last_fullsized":"2024-04-25T00:41:21.729185+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:58.553384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.727539+0000","last_change":"2024-04-25T00:41:20.724729+0000","last_active":"2024-04-25T00:41:21.727539+0000","last_peered":"2024-04-25T00:41:21.727539+0000","last_clean":"2024-04-25T00:41:21.727539+0000","last_became_active":"2024-04-25T00:41:20.723781+0000","last_became_peered":"2024-04-25T00:41:20.723781+0000","last_unstale":"2024-04-25T00:41:21.727539+0000","last_undegraded":"2024-04-25T00:41:21.727539+0000","last_fullsized":"2024-04-25T00:41:21.727539+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:39.356001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.854985+0000","last_change":"2024-04-25T00:41:19.713087+0000","last_active":"2024-04-25T00:41:21.854985+0000","last_peered":"2024-04-25T00:41:21.854985+0000","last_clean":"2024-04-25T00:41:21.854985+0000","last_became_active":"2024-04-25T00:41:19.710399+0000","last_became_peered":"2024-04-25T00:41:19.710399+0000","last_unstale":"2024-04-25T00:41:21.854985+0000","last_undegraded":"2024-04-25T00:41:21.854985+0000","last_fullsized":"2024-04-25T00:41:21.854985+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:01.391340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853040+0000","last_change":"2024-04-25T00:41:20.752301+0000","last_active":"2024-04-25T00:41:21.853040+0000","last_peered":"2024-04-25T00:41:21.853040+0000","last_clean":"2024-04-25T00:41:21.853040+0000","last_became_active":"2024-04-25T00:41:20.751497+0000","last_became_peered":"2024-04-25T00:41:20.751497+0000","last_unstale":"2024-04-25T00:41:21.853040+0000","last_undegraded":"2024-04-25T00:41:21.853040+0000","last_fullsized":"2024-04-25T00:41:21.853040+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:50.003262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.856627+0000","last_change":"2024-04-25T00:41:19.727532+0000","last_active":"2024-04-25T00:41:21.856627+0000","last_peered":"2024-04-25T00:41:21.856627+0000","last_clean":"2024-04-25T00:41:21.856627+0000","last_became_active":"2024-04-25T00:41:19.727155+0000","last_became_peered":"2024-04-25T00:41:19.727155+0000","last_unstale":"2024-04-25T00:41:21.856627+0000","last_undegraded":"2024-04-25T00:41:21.856627+0000","last_fullsized":"2024-04-25T00:41:21.856627+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:54.985055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737267+0000","last_change":"2024-04-25T00:41:20.756146+0000","last_active":"2024-04-25T00:41:21.737267+0000","last_peered":"2024-04-25T00:41:21.737267+0000","last_clean":"2024-04-25T00:41:21.737267+0000","last_became_active":"2024-04-25T00:41:20.755869+0000","last_became_peered":"2024-04-25T00:41:20.755869+0000","last_unstale":"2024-04-25T00:41:21.737267+0000","last_undegraded":"2024-04-25T00:41:21.737267+0000","last_fullsized":"2024-04-25T00:41:21.737267+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:02.047170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.724768+0000","last_change":"2024-04-25T00:41:19.725155+0000","last_active":"2024-04-25T00:41:21.724768+0000","last_peered":"2024-04-25T00:41:21.724768+0000","last_clean":"2024-04-25T00:41:21.724768+0000","last_became_active":"2024-04-25T00:41:19.723795+0000","last_became_peered":"2024-04-25T00:41:19.723795+0000","last_unstale":"2024-04-25T00:41:21.724768+0000","last_undegraded":"2024-04-25T00:41:21.724768+0000","last_fullsized":"2024-04-25T00:41:21.724768+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:17.151602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728540+0000","last_change":"2024-04-25T00:41:20.733781+0000","last_active":"2024-04-25T00:41:21.728540+0000","last_peered":"2024-04-25T00:41:21.728540+0000","last_clean":"2024-04-25T00:41:21.728540+0000","last_became_active":"2024-04-25T00:41:20.726560+0000","last_became_peered":"2024-04-25T00:41:20.726560+0000","last_unstale":"2024-04-25T00:41:21.728540+0000","last_undegraded":"2024-04-25T00:41:21.728540+0000","last_fullsized":"2024-04-25T00:41:21.728540+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:21.667951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.853894+0000","last_change":"2024-04-25T00:41:19.722319+0000","last_active":"2024-04-25T00:41:21.853894+0000","last_peered":"2024-04-25T00:41:21.853894+0000","last_clean":"2024-04-25T00:41:21.853894+0000","last_became_active":"2024-04-25T00:41:19.721932+0000","last_became_peered":"2024-04-25T00:41:19.721932+0000","last_unstale":"2024-04-25T00:41:21.853894+0000","last_undegraded":"2024-04-25T00:41:21.853894+0000","last_fullsized":"2024-04-25T00:41:21.853894+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:44.151050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.855598+0000","last_change":"2024-04-25T00:41:20.735352+0000","last_active":"2024-04-25T00:41:21.855598+0000","last_peered":"2024-04-25T00:41:21.855598+0000","last_clean":"2024-04-25T00:41:21.855598+0000","last_became_active":"2024-04-25T00:41:20.733136+0000","last_became_peered":"2024-04-25T00:41:20.733136+0000","last_unstale":"2024-04-25T00:41:21.855598+0000","last_undegraded":"2024-04-25T00:41:21.855598+0000","last_fullsized":"2024-04-25T00:41:21.855598+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:45.237690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.852820+0000","last_change":"2024-04-25T00:41:19.732832+0000","last_active":"2024-04-25T00:41:21.852820+0000","last_peered":"2024-04-25T00:41:21.852820+0000","last_clean":"2024-04-25T00:41:21.852820+0000","last_became_active":"2024-04-25T00:41:19.731714+0000","last_became_peered":"2024-04-25T00:41:19.731714+0000","last_unstale":"2024-04-25T00:41:21.852820+0000","last_undegraded":"2024-04-25T00:41:21.852820+0000","last_fullsized":"2024-04-25T00:41:21.852820+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:44.151050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725714+0000","last_change":"2024-04-25T00:41:20.717857+0000","last_active":"2024-04-25T00:41:21.725714+0000","last_peered":"2024-04-25T00:41:21.725714+0000","last_clean":"2024-04-25T00:41:21.725714+0000","last_became_active":"2024-04-25T00:41:20.717105+0000","last_became_peered":"2024-04-25T00:41:20.717105+0000","last_unstale":"2024-04-25T00:41:21.725714+0000","last_undegraded":"2024-04-25T00:41:21.725714+0000","last_fullsized":"2024-04-25T00:41:21.725714+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:59.739712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.726474+0000","last_change":"2024-04-25T00:41:19.714001+0000","last_active":"2024-04-25T00:41:21.726474+0000","last_peered":"2024-04-25T00:41:21.726474+0000","last_clean":"2024-04-25T00:41:21.726474+0000","last_became_active":"2024-04-25T00:41:19.713267+0000","last_became_peered":"2024-04-25T00:41:19.713267+0000","last_unstale":"2024-04-25T00:41:21.726474+0000","last_undegraded":"2024-04-25T00:41:21.726474+0000","last_fullsized":"2024-04-25T00:41:21.726474+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:46.563704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.737058+0000","last_change":"2024-04-25T00:41:20.726216+0000","last_active":"2024-04-25T00:41:21.737058+0000","last_peered":"2024-04-25T00:41:21.737058+0000","last_clean":"2024-04-25T00:41:21.737058+0000","last_became_active":"2024-04-25T00:41:20.725564+0000","last_became_peered":"2024-04-25T00:41:20.725564+0000","last_unstale":"2024-04-25T00:41:21.737058+0000","last_undegraded":"2024-04-25T00:41:21.737058+0000","last_fullsized":"2024-04-25T00:41:21.737058+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:19.678552+0000","last_clean_scrub_stamp":"2024-04-25T00:41:19.678552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:27.898758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"167'1","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.728110+0000","last_change":"2024-04-25T00:41:19.729890+0000","last_active":"2024-04-25T00:41:21.728110+0000","last_peered":"2024-04-25T00:41:21.728110+0000","last_clean":"2024-04-25T00:41:21.728110+0000","last_became_active":"2024-04-25T00:41:19.728730+0000","last_became_peered":"2024-04-25T00:41:19.728730+0000","last_unstale":"2024-04-25T00:41:21.728110+0000","last_undegraded":"2024-04-25T00:41:21.728110+0000","last_fullsized":"2024-04-25T00:41:21.728110+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:41:18.666477+0000","last_clean_scrub_stamp":"2024-04-25T00:41:18.666477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:38.343926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":490,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725087+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:41:21.725087+0000","last_peered":"2024-04-25T00:41:21.725087+0000","last_clean":"2024-04-25T00:41:21.725087+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:41:21.725087+0000","last_undegraded":"2024-04-25T00:41:21.725087+0000","last_fullsized":"2024-04-25T00:41:21.725087+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574972,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38004,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333836,"statfs":{"total":96636764160,"available":96597848064,"internally_reserved":0,"allocated":2727936,"data_stored":1529742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37416,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334424,"statfs":{"total":96636764160,"available":96598450176,"internally_reserved":0,"allocated":2125824,"data_stored":938344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574973,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37420,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334420,"statfs":{"total":96636764160,"available":96598446080,"internally_reserved":0,"allocated":2125824,"data_stored":938344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574972,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41600,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330240,"statfs":{"total":96636764160,"available":96594165760,"internally_reserved":0,"allocated":2142208,"data_stored":939478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41588,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330252,"statfs":{"total":96636764160,"available":96594178048,"internally_reserved":0,"allocated":2134016,"data_stored":937944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574972,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37416,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334424,"statfs":{"total":96636764160,"available":96598450176,"internally_reserved":0,"allocated":2121728,"data_stored":937762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2129920,"data_stored":937818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574972,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42156,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329684,"statfs":{"total":96636764160,"available":96593596416,"internally_reserved":0,"allocated":2719744,"data_stored":1528220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:41:29.616 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:41:29.616 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:41:29.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.788+0000 7f6001e13640 1 Processor -- start 2024-04-25T00:41:29.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.788+0000 7f6001e13640 1 -- start start 2024-04-25T00:41:29.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.788+0000 7f6001e13640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ffc15e8d0 0x7f5ffc0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:29.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.788+0000 7f6001e13640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc0b4780 0x7f5ffc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:29.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.788+0000 7f6001e13640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ffc0b3720 con 0x7f5ffc15f1f0 2024-04-25T00:41:29.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.788+0000 7f6001e13640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5ffc0b42f0 con 0x7f5ffc15e8d0 2024-04-25T00:41:29.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.788+0000 7f6001e13640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5ffc15d7e0 con 0x7f5ffc0b4780 2024-04-25T00:41:29.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.792+0000 7f5ffb7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ffc15e8d0 0x7f5ffc0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:29.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.792+0000 7f5ffaffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc0b4780 0x7f5ffc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:29.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.792+0000 7f5ffb7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ffc15e8d0 0x7f5ffc0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:44066/0 (socket says 172.21.15.29:44066) 2024-04-25T00:41:29.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.792+0000 7f5ffb7fe640 1 -- 172.21.15.29:0/1671842453 learned_addr learned my addr 172.21.15.29:0/1671842453 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:29.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.792+0000 7f5ffb7fe640 1 -- 172.21.15.29:0/1671842453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc0b4780 msgr2=0x7f5ffc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:29.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.792+0000 7f5ffb7fe640 1 --2- 172.21.15.29:0/1671842453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc0b4780 0x7f5ffc15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.792+0000 7f5ffb7fe640 1 -- 172.21.15.29:0/1671842453 >> v1:172.21.15.157:6789/0 conn(0x7f5ffc15f1f0 legacy=0x7f5ffc15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:29.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.792+0000 7f5ffb7fe640 1 -- 172.21.15.29:0/1671842453 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc0b39c0 con 0x7f5ffc15e8d0 2024-04-25T00:41:29.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.792+0000 7f5ffb7fe640 1 --2- 172.21.15.29:0/1671842453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc15e8d0 0x7f5ffc0cc240 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009870 tx=0x7f5fec01c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88ca7864a6249268 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:29.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.792+0000 7f5ffa7fc640 1 -- 172.21.15.29:0/1671842453 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc15c1c0 con 0x7f5ffc15e8d0 2024-04-25T00:41:29.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.792+0000 7f5ffa7fc640 1 -- 172.21.15.29:0/1671842453 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fec0a95f0 con 0x7f5ffc15e8d0 2024-04-25T00:41:29.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.792+0000 7f5ffa7fc640 1 -- 172.21.15.29:0/1671842453 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0a99c0 con 0x7f5ffc15e8d0 2024-04-25T00:41:29.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 -- 172.21.15.29:0/1671842453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc15e8d0 msgr2=0x7f5ffc0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:29.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 --2- 172.21.15.29:0/1671842453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc15e8d0 0x7f5ffc0cc240 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009870 tx=0x7f5fec01c530 comp rx=0 tx=0).stop 2024-04-25T00:41:29.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 -- 172.21.15.29:0/1671842453 shutdown_connections 2024-04-25T00:41:29.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 --2- 172.21.15.29:0/1671842453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc0b4780 0x7f5ffc15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 --2- 172.21.15.29:0/1671842453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc15e8d0 0x7f5ffc0cc240 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 -- 172.21.15.29:0/1671842453 >> 172.21.15.29:0/1671842453 conn(0x7f5ffc0b1870 msgr2=0x7f5ffc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:29.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 -- 172.21.15.29:0/1671842453 shutdown_connections 2024-04-25T00:41:29.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 -- 172.21.15.29:0/1671842453 wait complete. 2024-04-25T00:41:29.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 Processor -- start 2024-04-25T00:41:29.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 -- start start 2024-04-25T00:41:29.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc0b4780 0x7f5ffc158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:29.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc15e8d0 0x7f5ffc0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:29.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ffc15f1f0 0x7f5ffc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:29.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc144a20 con 0x7f5ffc15f1f0 2024-04-25T00:41:29.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc15f600 con 0x7f5ffc15e8d0 2024-04-25T00:41:29.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5ffc13e800 con 0x7f5ffc0b4780 2024-04-25T00:41:29.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ffc15f1f0 0x7f5ffc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:29.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ffc15f1f0 0x7f5ffc06d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37112/0 (socket says 172.21.15.29:37112) 2024-04-25T00:41:29.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5ffbfff640 1 -- 172.21.15.29:0/148525218 learned_addr learned my addr 172.21.15.29:0/148525218 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:29.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5ffaffd640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc15e8d0 0x7f5ffc0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:29.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5ffb7fe640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc0b4780 0x7f5ffc158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:29.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5ffbfff640 1 -- 172.21.15.29:0/148525218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc0b4780 msgr2=0x7f5ffc158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:29.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5ffbfff640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc0b4780 0x7f5ffc158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5ffbfff640 1 -- 172.21.15.29:0/148525218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc15e8d0 msgr2=0x7f5ffc0b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:29.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5ffbfff640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc15e8d0 0x7f5ffc0b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5ffbfff640 1 -- 172.21.15.29:0/148525218 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc13eae0 con 0x7f5ffc15f1f0 2024-04-25T00:41:29.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5ffaffd640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc15e8d0 0x7f5ffc0b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:29.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5ffbfff640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ffc15f1f0 0x7f5ffc06d5d0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400c760 tx=0x7f5ff400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:29.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5fe3fff640 1 -- 172.21.15.29:0/148525218 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4016020 con 0x7f5ffc15f1f0 2024-04-25T00:41:29.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5fe3fff640 1 -- 172.21.15.29:0/148525218 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff4006d60 con 0x7f5ffc15f1f0 2024-04-25T00:41:29.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 -- 172.21.15.29:0/148525218 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ffc13edc0 con 0x7f5ffc15f1f0 2024-04-25T00:41:29.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f5fe3fff640 1 -- 172.21.15.29:0/148525218 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4003bc0 con 0x7f5ffc15f1f0 2024-04-25T00:41:29.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.796+0000 7f6001e13640 1 -- 172.21.15.29:0/148525218 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ffc13f380 con 0x7f5ffc15f1f0 2024-04-25T00:41:29.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.800+0000 7f6001e13640 1 -- 172.21.15.29:0/148525218 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ffc069270 con 0x7f5ffc15f1f0 2024-04-25T00:41:29.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.800+0000 7f5fe3fff640 1 -- 172.21.15.29:0/148525218 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f5ff40050d0 con 0x7f5ffc15f1f0 2024-04-25T00:41:29.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.800+0000 7f5fe3fff640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fe8068250 0x7f5fe806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:29.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.800+0000 7f5fe3fff640 1 -- 172.21.15.29:0/148525218 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f5ff4084000 con 0x7f5ffc15f1f0 2024-04-25T00:41:29.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.800+0000 7f5ffb7fe640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fe8068250 0x7f5fe806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:29.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.800+0000 7f5ffb7fe640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fe8068250 0x7f5fe806a710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0ae000 tx=0x7f5fec0b3040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:29.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:29.804+0000 7f5fe3fff640 1 -- 172.21.15.29:0/148525218 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5ff4020020 con 0x7f5ffc15f1f0 2024-04-25T00:41:30.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.032+0000 7f6001e13640 1 -- 172.21.15.29:0/148525218 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5ffc06e7b0 con 0x7f5ffc15f1f0 2024-04-25T00:41:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.036+0000 7f5fe3fff640 1 -- 172.21.15.29:0/148525218 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v28) ==== 190+0+147 (secure 0 0 0) 0x7f5ff4056740 con 0x7f5ffc15f1f0 2024-04-25T00:41:30.036 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-04-25T00:41:30.036 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQAypClm5cgeIBAAjLjg6xX11JetOS2oCqgz6A== 2024-04-25T00:41:30.036 INFO:teuthology.orchestra.run.smithi029.stdout: caps mds = "allow" 2024-04-25T00:41:30.036 INFO:teuthology.orchestra.run.smithi029.stdout: caps mon = "allow r" 2024-04-25T00:41:30.036 INFO:teuthology.orchestra.run.smithi029.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:41:30.037 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-04-25T00:41:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.036+0000 7f6001e13640 1 -- 172.21.15.29:0/148525218 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fe8068250 msgr2=0x7f5fe806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.036+0000 7f6001e13640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fe8068250 0x7f5fe806a710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0ae000 tx=0x7f5fec0b3040 comp rx=0 tx=0).stop 2024-04-25T00:41:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.036+0000 7f6001e13640 1 -- 172.21.15.29:0/148525218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ffc15f1f0 msgr2=0x7f5ffc06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.036+0000 7f6001e13640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ffc15f1f0 0x7f5ffc06d5d0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400c760 tx=0x7f5ff400cb60 comp rx=0 tx=0).stop 2024-04-25T00:41:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.036+0000 7f6001e13640 1 -- 172.21.15.29:0/148525218 shutdown_connections 2024-04-25T00:41:30.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.036+0000 7f6001e13640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fe8068250 0x7f5fe806a710 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.036+0000 7f6001e13640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ffc15f1f0 0x7f5ffc06d5d0 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.036+0000 7f6001e13640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ffc15e8d0 0x7f5ffc0b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.036+0000 7f6001e13640 1 --2- 172.21.15.29:0/148525218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ffc0b4780 0x7f5ffc158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.036+0000 7f6001e13640 1 -- 172.21.15.29:0/148525218 >> 172.21.15.29:0/148525218 conn(0x7f5ffc0b1870 msgr2=0x7f5ffc13a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:30.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.040+0000 7f6001e13640 1 -- 172.21.15.29:0/148525218 shutdown_connections 2024-04-25T00:41:30.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.040+0000 7f6001e13640 1 -- 172.21.15.29:0/148525218 wait complete. 2024-04-25T00:41:30.055 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:41:30.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1c1ab640 1 Processor -- start 2024-04-25T00:41:30.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1c1ab640 1 -- start start 2024-04-25T00:41:30.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1c1ab640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c140b3140 0x7f4c1406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1c1ab640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4c140b3c30 0x7f4c140ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1c1ab640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c140b41e0 con 0x7f4c140b4b10 2024-04-25T00:41:30.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1c1ab640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4c1406d960 con 0x7f4c140b3c30 2024-04-25T00:41:30.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1c1ab640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4c1406dc40 con 0x7f4c140b3140 2024-04-25T00:41:30.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1a748640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4c140b3c30 0x7f4c140ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c19f47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c140b3140 0x7f4c1406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1a748640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4c140b3c30 0x7f4c140ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37126/0 (socket says 172.21.15.29:37126) 2024-04-25T00:41:30.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1a748640 1 -- 172.21.15.29:0/4195241058 learned_addr learned my addr 172.21.15.29:0/4195241058 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1a748640 1 -- 172.21.15.29:0/4195241058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c140b3140 msgr2=0x7f4c1406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1a748640 1 --2- 172.21.15.29:0/4195241058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c140b3140 0x7f4c1406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1a748640 1 -- 172.21.15.29:0/4195241058 >> v1:172.21.15.157:6789/0 conn(0x7f4c140b4b10 legacy=0x7f4c140b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.164+0000 7f4c1a748640 1 -- 172.21.15.29:0/4195241058 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1406df20 con 0x7f4c140b3c30 2024-04-25T00:41:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c1a748640 1 --2- 172.21.15.29:0/4195241058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c140b3c30 0x7f4c140ce240 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009e50 tx=0x7f4c1001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8adb604c67860e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:30.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c18f45640 1 -- 172.21.15.29:0/4195241058 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c100a5700 con 0x7f4c140b3c30 2024-04-25T00:41:30.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c18f45640 1 -- 172.21.15.29:0/4195241058 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c100a58a0 con 0x7f4c140b3c30 2024-04-25T00:41:30.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c18f45640 1 -- 172.21.15.29:0/4195241058 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c100a5c70 con 0x7f4c140b3c30 2024-04-25T00:41:30.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/4195241058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c140b3c30 msgr2=0x7f4c140ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c1c1ab640 1 --2- 172.21.15.29:0/4195241058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c140b3c30 0x7f4c140ce240 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009e50 tx=0x7f4c1001ca30 comp rx=0 tx=0).stop 2024-04-25T00:41:30.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/4195241058 shutdown_connections 2024-04-25T00:41:30.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c1c1ab640 1 --2- 172.21.15.29:0/4195241058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c140b3c30 0x7f4c140ce240 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c1c1ab640 1 --2- 172.21.15.29:0/4195241058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c140b3140 0x7f4c1406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/4195241058 >> 172.21.15.29:0/4195241058 conn(0x7f4c140b1870 msgr2=0x7f4c140b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:30.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/4195241058 shutdown_connections 2024-04-25T00:41:30.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/4195241058 wait complete. 2024-04-25T00:41:30.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c1c1ab640 1 Processor -- start 2024-04-25T00:41:30.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.168+0000 7f4c1c1ab640 1 -- start start 2024-04-25T00:41:30.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c1c1ab640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c140b3140 0x7f4c14134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c1c1ab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c140b3c30 0x7f4c14132ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c1c1ab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c140b4b10 0x7f4c141402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c1c1ab640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14168d80 con 0x7f4c140b3c30 2024-04-25T00:41:30.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c1c1ab640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14168c00 con 0x7f4c140b4b10 2024-04-25T00:41:30.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c1c1ab640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4c14168f00 con 0x7f4c140b3140 2024-04-25T00:41:30.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c19746640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c140b3c30 0x7f4c14132ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c19746640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c140b3c30 0x7f4c14132ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37128/0 (socket says 172.21.15.29:37128) 2024-04-25T00:41:30.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c19746640 1 -- 172.21.15.29:0/1606641774 learned_addr learned my addr 172.21.15.29:0/1606641774 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:30.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c19746640 1 -- 172.21.15.29:0/1606641774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c140b3140 msgr2=0x7f4c14134fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c19746640 1 --2- 172.21.15.29:0/1606641774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c140b3140 0x7f4c14134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c1a748640 1 --2- 172.21.15.29:0/1606641774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c140b4b10 0x7f4c141402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c19746640 1 -- 172.21.15.29:0/1606641774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c140b4b10 msgr2=0x7f4c141402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c19746640 1 --2- 172.21.15.29:0/1606641774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c140b4b10 0x7f4c141402a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c19746640 1 -- 172.21.15.29:0/1606641774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c14133410 con 0x7f4c140b3c30 2024-04-25T00:41:30.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c19746640 1 --2- 172.21.15.29:0/1606641774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c140b3c30 0x7f4c14132ca0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400c420 tx=0x7f4c0400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:30.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c0a7fc640 1 -- 172.21.15.29:0/1606641774 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c040052d0 con 0x7f4c140b3c30 2024-04-25T00:41:30.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c0a7fc640 1 -- 172.21.15.29:0/1606641774 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c04005470 con 0x7f4c140b3c30 2024-04-25T00:41:30.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/1606641774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c141336f0 con 0x7f4c140b3c30 2024-04-25T00:41:30.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c0a7fc640 1 -- 172.21.15.29:0/1606641774 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04007260 con 0x7f4c140b3c30 2024-04-25T00:41:30.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/1606641774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c14133cb0 con 0x7f4c140b3c30 2024-04-25T00:41:30.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/1606641774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c14069270 con 0x7f4c140b3c30 2024-04-25T00:41:30.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c0a7fc640 1 -- 172.21.15.29:0/1606641774 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f4c04007400 con 0x7f4c140b3c30 2024-04-25T00:41:30.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c0a7fc640 1 --2- 172.21.15.29:0/1606641774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4bf8068250 0x7f4bf806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c0a7fc640 1 -- 172.21.15.29:0/1606641774 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f4c040838c0 con 0x7f4c140b3c30 2024-04-25T00:41:30.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c19f47640 1 --2- 172.21.15.29:0/1606641774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4bf8068250 0x7f4bf806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.172+0000 7f4c19f47640 1 --2- 172.21.15.29:0/1606641774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4bf8068250 0x7f4bf806a710 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c006aa0 tx=0x7f4c0c006a30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:30.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.176+0000 7f4c0a7fc640 1 -- 172.21.15.29:0/1606641774 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4c04056000 con 0x7f4c140b3c30 2024-04-25T00:41:30.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.364+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/1606641774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c14143500 con 0x7f4c140b3c30 2024-04-25T00:41:30.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.368+0000 7f4c0a7fc640 1 -- 172.21.15.29:0/1606641774 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5195 (secure 0 0 0) 0x7f4c04055900 con 0x7f4c140b3c30 2024-04-25T00:41:30.367 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:30.368 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":112,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7427,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/929849226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":929849226},{"type":"v1","addr":"172.21.15.157:6834","nonce":929849226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1459365951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1459365951},{"type":"v1","addr":"172.21.15.29:6833","nonce":1459365951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1011867355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1011867355},{"type":"v1","addr":"172.21.15.157:6835","nonce":1011867355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:41:20.082895+0000","modified":"2024-04-25T00:41:23.093215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10578":{"gid":10578,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/85311029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":85311029},{"type":"v1","addr":"172.21.15.29:6835","nonce":85311029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10578,"qdb_cluster":[10578]},"id":13}]} 2024-04-25T00:41:30.368 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 112 2024-04-25T00:41:30.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.368+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/1606641774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4bf8068250 msgr2=0x7f4bf806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.368+0000 7f4c1c1ab640 1 --2- 172.21.15.29:0/1606641774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4bf8068250 0x7f4bf806a710 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c006aa0 tx=0x7f4c0c006a30 comp rx=0 tx=0).stop 2024-04-25T00:41:30.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.368+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/1606641774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c140b3c30 msgr2=0x7f4c14132ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.368+0000 7f4c1c1ab640 1 --2- 172.21.15.29:0/1606641774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c140b3c30 0x7f4c14132ca0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400c420 tx=0x7f4c0400c8e0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.368+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/1606641774 shutdown_connections 2024-04-25T00:41:30.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.368+0000 7f4c1c1ab640 1 --2- 172.21.15.29:0/1606641774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4bf8068250 0x7f4bf806a710 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.368+0000 7f4c1c1ab640 1 --2- 172.21.15.29:0/1606641774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c140b4b10 0x7f4c141402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.368+0000 7f4c1c1ab640 1 --2- 172.21.15.29:0/1606641774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c140b3c30 0x7f4c14132ca0 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.368+0000 7f4c1c1ab640 1 --2- 172.21.15.29:0/1606641774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c140b3140 0x7f4c14134fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.368+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/1606641774 >> 172.21.15.29:0/1606641774 conn(0x7f4c140b1870 msgr2=0x7f4c14143f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:30.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.368+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/1606641774 shutdown_connections 2024-04-25T00:41:30.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.368+0000 7f4c1c1ab640 1 -- 172.21.15.29:0/1606641774 wait complete. 2024-04-25T00:41:30.789 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:41:20.082895+0000', 'modified': '2024-04-25T00:41:23.093215+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10578}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10578': {'gid': 10578, 'name': 'c', 'rank': 0, 'incarnation': 108, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/85311029', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 85311029}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 85311029}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10578, 'qdb_cluster': [10578]} 2024-04-25T00:41:30.789 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:41:30.790 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:41:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.924+0000 7fa4f4461640 1 Processor -- start 2024-04-25T00:41:30.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.924+0000 7fa4f4461640 1 -- start start 2024-04-25T00:41:30.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.924+0000 7fa4f4461640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa4ec15d980 0x7fa4ec06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.924+0000 7fa4f4461640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4ec0b2b70 0x7fa4ec06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.924+0000 7fa4f4461640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ec151950 con 0x7fa4ec150df0 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.924+0000 7fa4f4461640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa4ec0cf560 con 0x7fa4ec15d980 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.924+0000 7fa4f4461640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa4ec0b3130 con 0x7fa4ec0b2b70 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.924+0000 7fa4f29fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa4ec150df0 0x7fa4ec151240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35816/0 (socket says 172.21.15.29:35816) 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.924+0000 7fa4f29fe640 1 -- 172.21.15.29:0/2824303950 learned_addr learned my addr 172.21.15.29:0/2824303950 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.924+0000 7fa4f21fd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa4ec15d980 0x7fa4ec06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.928+0000 7fa4f19fc640 1 --2- 172.21.15.29:0/2824303950 >> v2:172.21.15.157:3300/0 conn(0x7fa4ec0b2b70 0x7fa4ec06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.928+0000 7fa4f11fb640 1 -- 172.21.15.29:0/2824303950 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 819110970 0 0) 0x7fa4ec151950 con 0x7fa4ec150df0 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.928+0000 7fa4f11fb640 1 -- 172.21.15.29:0/2824303950 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4d8003450 con 0x7fa4ec150df0 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.928+0000 7fa4f21fd640 1 -- 172.21.15.29:0/2824303950 >> v2:172.21.15.157:3300/0 conn(0x7fa4ec0b2b70 msgr2=0x7fa4ec06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.928+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/2824303950 >> v2:172.21.15.157:3300/0 conn(0x7fa4ec0b2b70 0x7fa4ec06db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.928+0000 7fa4f21fd640 1 -- 172.21.15.29:0/2824303950 >> v1:172.21.15.29:6789/0 conn(0x7fa4ec150df0 legacy=0x7fa4ec151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.928+0000 7fa4f21fd640 1 -- 172.21.15.29:0/2824303950 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec0b3410 con 0x7fa4ec15d980 2024-04-25T00:41:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/2824303950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4ec15d980 0x7fa4ec06d5d0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0b2720 tx=0x7fa4e001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43d544d5be18182 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f11fb640 1 -- 172.21.15.29:0/2824303950 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00b1070 con 0x7fa4ec15d980 2024-04-25T00:41:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f11fb640 1 -- 172.21.15.29:0/2824303950 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e00ac630 con 0x7fa4ec15d980 2024-04-25T00:41:30.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f11fb640 1 -- 172.21.15.29:0/2824303950 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00a8050 con 0x7fa4ec15d980 2024-04-25T00:41:30.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 -- 172.21.15.29:0/2824303950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4ec15d980 msgr2=0x7fa4ec06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 --2- 172.21.15.29:0/2824303950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4ec15d980 0x7fa4ec06d5d0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0b2720 tx=0x7fa4e001c450 comp rx=0 tx=0).stop 2024-04-25T00:41:30.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 -- 172.21.15.29:0/2824303950 shutdown_connections 2024-04-25T00:41:30.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 --2- 172.21.15.29:0/2824303950 >> v2:172.21.15.157:3300/0 conn(0x7fa4ec0b2b70 0x7fa4ec06db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 --2- 172.21.15.29:0/2824303950 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4ec15d980 0x7fa4ec06d5d0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 -- 172.21.15.29:0/2824303950 >> 172.21.15.29:0/2824303950 conn(0x7fa4ec0b1870 msgr2=0x7fa4ec0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:30.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 -- 172.21.15.29:0/2824303950 shutdown_connections 2024-04-25T00:41:30.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 -- 172.21.15.29:0/2824303950 wait complete. 2024-04-25T00:41:30.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 Processor -- start 2024-04-25T00:41:30.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 -- start start 2024-04-25T00:41:30.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4ec0b2b70 0x7fa4ec14ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4ec150df0 0x7fa4ec070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4ec15d980 0x7fa4ec071220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec15b4b0 con 0x7fa4ec15d980 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec15dd60 con 0x7fa4ec150df0 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.932+0000 7fa4f4461640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa4ec15b630 con 0x7fa4ec0b2b70 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f29fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4ec15d980 0x7fa4ec071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f29fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4ec15d980 0x7fa4ec071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37148/0 (socket says 172.21.15.29:37148) 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f29fe640 1 -- 172.21.15.29:0/2253120839 learned_addr learned my addr 172.21.15.29:0/2253120839 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4ec0b2b70 0x7fa4ec14ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f19fc640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4ec150df0 0x7fa4ec070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f29fe640 1 -- 172.21.15.29:0/2253120839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4ec0b2b70 msgr2=0x7fa4ec14ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f29fe640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4ec0b2b70 0x7fa4ec14ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f29fe640 1 -- 172.21.15.29:0/2253120839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4ec150df0 msgr2=0x7fa4ec070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f29fe640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4ec150df0 0x7fa4ec070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f29fe640 1 -- 172.21.15.29:0/2253120839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec0b00f0 con 0x7fa4ec15d980 2024-04-25T00:41:30.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4ec0b2b70 0x7fa4ec14ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:30.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f19fc640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4ec150df0 0x7fa4ec070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:30.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f29fe640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4ec15d980 0x7fa4ec071220 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400cb30 tx=0x7fa4e400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:30.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4deffd640 1 -- 172.21.15.29:0/2253120839 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4017070 con 0x7fa4ec15d980 2024-04-25T00:41:30.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4deffd640 1 -- 172.21.15.29:0/2253120839 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e400aca0 con 0x7fa4ec15d980 2024-04-25T00:41:30.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f4461640 1 -- 172.21.15.29:0/2253120839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec0b03d0 con 0x7fa4ec15d980 2024-04-25T00:41:30.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4deffd640 1 -- 172.21.15.29:0/2253120839 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4012600 con 0x7fa4ec15d980 2024-04-25T00:41:30.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f4461640 1 -- 172.21.15.29:0/2253120839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec0b0990 con 0x7fa4ec15d980 2024-04-25T00:41:30.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4deffd640 1 -- 172.21.15.29:0/2253120839 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fa4e401a070 con 0x7fa4ec15d980 2024-04-25T00:41:30.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4deffd640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa4d0068250 0x7fa4d006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4deffd640 1 -- 172.21.15.29:0/2253120839 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7fa4e40840e0 con 0x7fa4ec15d980 2024-04-25T00:41:30.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa4d0068250 0x7fa4d006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa4d0068250 0x7fa4d006a710 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00ae000 tx=0x7fa4e00a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:30.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f75da000 1 Processor -- start 2024-04-25T00:41:30.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.936+0000 7fa4f75da000 1 -- start start 2024-04-25T00:41:30.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a679736ef0 0x55a6797372d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679737810 0x55a6796ed680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 0x55a6796edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a67953b5e0 con 0x55a6796edbc0 2024-04-25T00:41:30.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a6794c2c50 con 0x55a679736ef0 2024-04-25T00:41:30.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55a6795375a0 con 0x55a679737810 2024-04-25T00:41:30.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f29fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 0x55a6796edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f29fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 0x55a6796edfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37158/0 (socket says 172.21.15.29:37158) 2024-04-25T00:41:30.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f29fe640 1 -- 172.21.15.29:0/2351735360 learned_addr learned my addr 172.21.15.29:0/2351735360 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:30.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f29fe640 1 -- 172.21.15.29:0/2351735360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679737810 msgr2=0x55a6796ed680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:30.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/2351735360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a679736ef0 0x55a6797372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f19fc640 1 --2- 172.21.15.29:0/2351735360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679737810 0x55a6796ed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f29fe640 1 --2- 172.21.15.29:0/2351735360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679737810 0x55a6796ed680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f29fe640 1 -- 172.21.15.29:0/2351735360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a679736ef0 msgr2=0x55a6797372d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:30.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f29fe640 1 --2- 172.21.15.29:0/2351735360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a679736ef0 0x55a6797372d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f29fe640 1 -- 172.21.15.29:0/2351735360 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6794c0d10 con 0x55a6796edbc0 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f29fe640 1 --2- 172.21.15.29:0/2351735360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 0x55a6796edfa0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4001590 tx=0x7fa4e4056320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1058d9bc311a76c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4cffff640 1 -- 172.21.15.29:0/2351735360 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e405a1b0 con 0x55a6796edbc0 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4cffff640 1 -- 172.21.15.29:0/2351735360 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e4052840 con 0x55a6796edbc0 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4cffff640 1 -- 172.21.15.29:0/2351735360 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e405ea70 con 0x55a6796edbc0 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 -- 172.21.15.29:0/2351735360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 msgr2=0x55a6796edfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 --2- 172.21.15.29:0/2351735360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 0x55a6796edfa0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4001590 tx=0x7fa4e4056320 comp rx=0 tx=0).stop 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 -- 172.21.15.29:0/2351735360 shutdown_connections 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 --2- 172.21.15.29:0/2351735360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 0x55a6796edfa0 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 --2- 172.21.15.29:0/2351735360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679737810 0x55a6796ed680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 --2- 172.21.15.29:0/2351735360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a679736ef0 0x55a6797372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 -- 172.21.15.29:0/2351735360 >> 172.21.15.29:0/2351735360 conn(0x55a6796e9b70 msgr2=0x55a6796ebf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 -- 172.21.15.29:0/2351735360 shutdown_connections 2024-04-25T00:41:30.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 -- 172.21.15.29:0/2351735360 wait complete. 2024-04-25T00:41:30.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 -- start start 2024-04-25T00:41:30.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 Processor -- start 2024-04-25T00:41:30.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 0x55a67971bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a679736ef0 0x55a67971c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679737810 0x55a67971c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a6794f5520 con 0x55a6796edbc0 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a6794e1810 con 0x55a679736ef0 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55a6796e99f0 con 0x55a679737810 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f75da000 10 client.0 _ll_register_callbacks cb 0x55a6796d85c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f21fd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 0x55a67971bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f21fd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 0x55a67971bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37170/0 (socket says 172.21.15.29:37170) 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f21fd640 1 -- 172.21.15.29:0/259878179 learned_addr learned my addr 172.21.15.29:0/259878179 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.940+0000 7fa4f19fc640 1 --2- 172.21.15.29:0/259878179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a679736ef0 0x55a67971c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f21fd640 1 -- 172.21.15.29:0/259878179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679737810 msgr2=0x55a67971c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/259878179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679737810 0x55a67971c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f21fd640 1 -- 172.21.15.29:0/259878179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a679736ef0 msgr2=0x55a67971c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/259878179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a679736ef0 0x55a67971c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f21fd640 1 -- 172.21.15.29:0/259878179 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6794d37b0 con 0x55a6796edbc0 2024-04-25T00:41:30.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/259878179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 0x55a67971bf20 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0097f20 tx=0x7fa4e00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba0c4e201394fca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4cd7fa640 1 -- 172.21.15.29:0/259878179 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00b1040 con 0x55a6796edbc0 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4cd7fa640 1 -- 172.21.15.29:0/259878179 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7fa4e00b6720 con 0x55a6796edbc0 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 -- 172.21.15.29:0/259878179 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e4052840 con 0x55a6796edbc0 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4cd7fa640 1 -- 172.21.15.29:0/259878179 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7fa4e00cf050 con 0x55a6796edbc0 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 10 client.10743 fetch_fsmap learned FSMap version 112 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 -- 172.21.15.29:0/259878179 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55a6794c2ea0 con 0x55a6796edbc0 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4cd7fa640 1 -- 172.21.15.29:0/259878179 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e4052840 con 0x55a6796edbc0 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4cd7fa640 1 -- 172.21.15.29:0/259878179 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 112) ==== 3494+0+0 (secure 0 0 0) 0x7fa4e00d2020 con 0x55a6796edbc0 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 10 client.10743 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 10 client.10743 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 20 client.10743 populate_metadata read hostname 'smithi029' 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 --2- 172.21.15.29:0/259878179 >> [v2:172.21.15.29:6834/85311029,v1:172.21.15.29:6835/85311029] conn(0x55a679746f90 0x55a679749380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 4 client.10743 mds_command: new command op to 10578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 -- 172.21.15.29:0/259878179 --> [v2:172.21.15.29:6834/85311029,v1:172.21.15.29:6835/85311029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6794d6ab0 con 0x55a679746f90 2024-04-25T00:41:30.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f19fc640 1 --2- 172.21.15.29:0/259878179 >> [v2:172.21.15.29:6834/85311029,v1:172.21.15.29:6835/85311029] conn(0x55a679746f90 0x55a679749380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f19fc640 1 --2- 172.21.15.29:0/259878179 >> [v2:172.21.15.29:6834/85311029,v1:172.21.15.29:6835/85311029] conn(0x55a679746f90 0x55a679749380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af592e8352977d91 server_cookie=72bc6af615348b0b in_seq=0 out_seq=0 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4cd7fa640 10 client.10743 ms_handle_connect on v2:172.21.15.29:6834/85311029 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4cd7fa640 1 -- 172.21.15.29:0/259878179 <== mds.0 v2:172.21.15.29:6834/85311029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6794d6ab0 con 0x55a679746f90 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4cd7fa640 10 client.10743 handle_command_reply: tid=0 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 client.10743 shutdown 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 -- 172.21.15.29:0/259878179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 msgr2=0x55a67971bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 --2- 172.21.15.29:0/259878179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 0x55a67971bf20 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0097f20 tx=0x7fa4e00046d0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 -- 172.21.15.29:0/259878179 shutdown_connections 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 --2- 172.21.15.29:0/259878179 >> [v2:172.21.15.29:6834/85311029,v1:172.21.15.29:6835/85311029] conn(0x55a679746f90 0x55a679749380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 --2- 172.21.15.29:0/259878179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679737810 0x55a67971c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 --2- 172.21.15.29:0/259878179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a679736ef0 0x55a67971c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4cd7fa640 0 client.10743 ms_handle_reset on v2:172.21.15.29:6834/85311029 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 --2- 172.21.15.29:0/259878179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6796edbc0 0x55a67971bf20 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 -- 172.21.15.29:0/259878179 >> 172.21.15.29:0/259878179 conn(0x55a6796e9b70 msgr2=0x55a6796ec7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:30.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 -- 172.21.15.29:0/259878179 shutdown_connections 2024-04-25T00:41:30.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 1 -- 172.21.15.29:0/259878179 wait complete. 2024-04-25T00:41:30.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.944+0000 7fa4f75da000 20 client.10743 trim_cache size 0 max 16384 2024-04-25T00:41:30.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 Processor -- start 2024-04-25T00:41:30.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 -- start start 2024-04-25T00:41:30.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6796edbc0 0x55a6796ed0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679736ef0 0x55a6796ebf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679737810 0x55a6796ec4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec203190 con 0x55a679736ef0 2024-04-25T00:41:30.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec0cd060 con 0x55a6796edbc0 2024-04-25T00:41:30.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa4ec156c60 con 0x55a679737810 2024-04-25T00:41:30.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f19fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679736ef0 0x55a6796ebf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f19fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679736ef0 0x55a6796ebf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37172/0 (socket says 172.21.15.29:37172) 2024-04-25T00:41:30.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f19fc640 1 -- 172.21.15.29:0/3357721953 learned_addr learned my addr 172.21.15.29:0/3357721953 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:30.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f19fc640 1 -- 172.21.15.29:0/3357721953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679737810 msgr2=0x55a6796ec4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:30.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f19fc640 1 --2- 172.21.15.29:0/3357721953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679737810 0x55a6796ec4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f19fc640 1 -- 172.21.15.29:0/3357721953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6796edbc0 msgr2=0x55a6796ed0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:30.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f19fc640 1 --2- 172.21.15.29:0/3357721953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6796edbc0 0x55a6796ed0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f19fc640 1 -- 172.21.15.29:0/3357721953 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6794c3e80 con 0x55a679736ef0 2024-04-25T00:41:30.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f19fc640 1 --2- 172.21.15.29:0/3357721953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679736ef0 0x55a6796ebf60 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec1575a0 tx=0x7fa4e800d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385e1c70f3c2c8c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:30.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4cf7fe640 1 -- 172.21.15.29:0/3357721953 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8016ac0 con 0x55a679736ef0 2024-04-25T00:41:30.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4cf7fe640 1 -- 172.21.15.29:0/3357721953 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e8016c60 con 0x55a679736ef0 2024-04-25T00:41:30.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4cf7fe640 1 -- 172.21.15.29:0/3357721953 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8004230 con 0x55a679736ef0 2024-04-25T00:41:30.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 -- 172.21.15.29:0/3357721953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679736ef0 msgr2=0x55a6796ebf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 --2- 172.21.15.29:0/3357721953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679736ef0 0x55a6796ebf60 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec1575a0 tx=0x7fa4e800d870 comp rx=0 tx=0).stop 2024-04-25T00:41:30.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 -- 172.21.15.29:0/3357721953 shutdown_connections 2024-04-25T00:41:30.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 --2- 172.21.15.29:0/3357721953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679737810 0x55a6796ec4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 --2- 172.21.15.29:0/3357721953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679736ef0 0x55a6796ebf60 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 --2- 172.21.15.29:0/3357721953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6796edbc0 0x55a6796ed0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 -- 172.21.15.29:0/3357721953 >> 172.21.15.29:0/3357721953 conn(0x55a6796e9b70 msgr2=0x55a6797444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:30.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 -- 172.21.15.29:0/3357721953 shutdown_connections 2024-04-25T00:41:30.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 -- 172.21.15.29:0/3357721953 wait complete. 2024-04-25T00:41:30.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 -- start start 2024-04-25T00:41:30.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.964+0000 7fa4f75da000 1 Processor -- start 2024-04-25T00:41:30.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6796edbc0 0x55a6797a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679736ef0 0x55a6797a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679737810 0x55a679749690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a6795ede80 con 0x55a679737810 2024-04-25T00:41:30.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec0cca10 con 0x55a6796edbc0 2024-04-25T00:41:30.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55a6794efc40 con 0x55a679736ef0 2024-04-25T00:41:30.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 10 client.0 _ll_register_callbacks cb 0x7fa4e00af070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:30.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f29fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679737810 0x55a679749690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f29fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679737810 0x55a679749690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37178/0 (socket says 172.21.15.29:37178) 2024-04-25T00:41:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f29fe640 1 -- 172.21.15.29:0/1249509051 learned_addr learned my addr 172.21.15.29:0/1249509051 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f19fc640 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679736ef0 0x55a6797a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6796edbc0 0x55a6797a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f29fe640 1 -- 172.21.15.29:0/1249509051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679736ef0 msgr2=0x55a6797a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f29fe640 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679736ef0 0x55a6797a7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f29fe640 1 -- 172.21.15.29:0/1249509051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6796edbc0 msgr2=0x55a6797a77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f29fe640 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6796edbc0 0x55a6797a77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f29fe640 1 -- 172.21.15.29:0/1249509051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a679508760 con 0x55a679737810 2024-04-25T00:41:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f19fc640 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679736ef0 0x55a6797a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6796edbc0 0x55a6797a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f29fe640 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679737810 0x55a679749690 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4055ed0 tx=0x7fa4e406e2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad3effcd5138bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-25T00:41:30.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4cd7fa640 1 -- 172.21.15.29:0/1249509051 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e40561d0 con 0x55a679737810 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4cd7fa640 1 -- 172.21.15.29:0/1249509051 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7fa4e405e870 con 0x55a679737810 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 -- 172.21.15.29:0/1249509051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e8016c60 con 0x55a679737810 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4cd7fa640 1 -- 172.21.15.29:0/1249509051 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7fa4e405ea90 con 0x55a679737810 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 10 client.10749 fetch_fsmap learned FSMap version 112 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 -- 172.21.15.29:0/1249509051 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55a6795949e0 con 0x55a679737810 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4cd7fa640 1 -- 172.21.15.29:0/1249509051 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e8016c60 con 0x55a679737810 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4cd7fa640 1 -- 172.21.15.29:0/1249509051 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 112) ==== 3494+0+0 (secure 0 0 0) 0x7fa4e407db70 con 0x55a679737810 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 10 client.10749 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 10 client.10749 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 20 client.10749 populate_metadata read hostname 'smithi029' 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.29:6834/85311029,v1:172.21.15.29:6835/85311029] conn(0x55a679749bd0 0x55a679749fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 4 client.10749 mds_command: new command op to 10578 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 -- 172.21.15.29:0/1249509051 --> [v2:172.21.15.29:6834/85311029,v1:172.21.15.29:6835/85311029] -- command(tid 0: {"prefix": "status"}) -- 0x55a679548b20 con 0x55a679749bd0 2024-04-25T00:41:30.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.29:6834/85311029,v1:172.21.15.29:6835/85311029] conn(0x55a679749bd0 0x55a679749fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f21fd640 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.29:6834/85311029,v1:172.21.15.29:6835/85311029] conn(0x55a679749bd0 0x55a679749fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae28c6f335c1126 server_cookie=2cd71f230b1e9a69 in_seq=0 out_seq=0 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4cd7fa640 10 client.10749 ms_handle_connect on v2:172.21.15.29:6834/85311029 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4cd7fa640 1 -- 172.21.15.29:0/1249509051 <== mds.0 v2:172.21.15.29:6834/85311029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55a679548b20 con 0x55a679749bd0 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4cd7fa640 10 client.10749 handle_command_reply: tid=0 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 client.10749 shutdown 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 -- 172.21.15.29:0/1249509051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679737810 msgr2=0x55a679749690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679737810 0x55a679749690 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4055ed0 tx=0x7fa4e406e2a0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 -- 172.21.15.29:0/1249509051 shutdown_connections 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.29:6834/85311029,v1:172.21.15.29:6835/85311029] conn(0x55a679749bd0 0x55a679749fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a679737810 0x55a679749690 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a679736ef0 0x55a6797a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4cd7fa640 0 client.10749 ms_handle_reset on v2:172.21.15.29:6834/85311029 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 --2- 172.21.15.29:0/1249509051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6796edbc0 0x55a6797a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 -- 172.21.15.29:0/1249509051 >> 172.21.15.29:0/1249509051 conn(0x55a6796e9b70 msgr2=0x55a679735800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:30.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 -- 172.21.15.29:0/1249509051 shutdown_connections 2024-04-25T00:41:30.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 1 -- 172.21.15.29:0/1249509051 wait complete. 2024-04-25T00:41:30.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f75da000 20 client.10749 trim_cache size 0 max 16384 2024-04-25T00:41:30.976 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:41:30.976 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "59afbe7d-aaa8-45e2-b468-69ee150cacea", 2024-04-25T00:41:30.976 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-04-25T00:41:30.976 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 10578, 2024-04-25T00:41:30.976 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-04-25T00:41:30.976 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-04-25T00:41:30.977 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-04-25T00:41:30.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 10.256321798, 2024-04-25T00:41:30.977 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 112, 2024-04-25T00:41:30.977 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 168, 2024-04-25T00:41:30.977 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 168, 2024-04-25T00:41:30.977 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 15.340327893 2024-04-25T00:41:30.977 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:41:30.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f4461640 1 -- 172.21.15.29:0/2253120839 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa4d0068250 msgr2=0x7fa4d006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f4461640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa4d0068250 0x7fa4d006a710 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00ae000 tx=0x7fa4e00a7000 comp rx=0 tx=0).stop 2024-04-25T00:41:30.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f4461640 1 -- 172.21.15.29:0/2253120839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4ec15d980 msgr2=0x7fa4ec071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:30.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f4461640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4ec15d980 0x7fa4ec071220 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400cb30 tx=0x7fa4e400a060 comp rx=0 tx=0).stop 2024-04-25T00:41:30.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f4461640 1 -- 172.21.15.29:0/2253120839 shutdown_connections 2024-04-25T00:41:30.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f4461640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa4d0068250 0x7fa4d006a710 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f4461640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4ec15d980 0x7fa4ec071220 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f4461640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4ec150df0 0x7fa4ec070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f4461640 1 --2- 172.21.15.29:0/2253120839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4ec0b2b70 0x7fa4ec14ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:30.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f4461640 1 -- 172.21.15.29:0/2253120839 >> 172.21.15.29:0/2253120839 conn(0x7fa4ec0b1870 msgr2=0x7fa4ec150550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:30.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f4461640 1 -- 172.21.15.29:0/2253120839 shutdown_connections 2024-04-25T00:41:30.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:30.968+0000 7fa4f4461640 1 -- 172.21.15.29:0/2253120839 wait complete. 2024-04-25T00:41:30.986 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:41:31.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.092+0000 7fcf5e2b4640 1 Processor -- start 2024-04-25T00:41:31.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.092+0000 7fcf5e2b4640 1 -- start start 2024-04-25T00:41:31.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf5e2b4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcf5815e8d0 0x7fcf580cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:31.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf5e2b4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5815f3c0 0x7fcf5815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:31.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf5e2b4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf580b3720 con 0x7fcf580b4780 2024-04-25T00:41:31.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf5e2b4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fcf580b42f0 con 0x7fcf5815e8d0 2024-04-25T00:41:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf5e2b4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcf580b39c0 con 0x7fcf5815f3c0 2024-04-25T00:41:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf57fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcf5815e8d0 0x7fcf580cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf4ffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcf580b4780 0x7fcf5815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35820/0 (socket says 172.21.15.29:35820) 2024-04-25T00:41:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf4ffff640 1 -- 172.21.15.29:0/3776365022 learned_addr learned my addr 172.21.15.29:0/3776365022 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf5c851640 1 --2- 172.21.15.29:0/3776365022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5815f3c0 0x7fcf5815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf57fff640 1 -- 172.21.15.29:0/3776365022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5815f3c0 msgr2=0x7fcf5815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf57fff640 1 --2- 172.21.15.29:0/3776365022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5815f3c0 0x7fcf5815c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf57fff640 1 -- 172.21.15.29:0/3776365022 >> v1:172.21.15.29:6789/0 conn(0x7fcf580b4780 legacy=0x7fcf5815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf57fff640 1 -- 172.21.15.29:0/3776365022 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf580b3ca0 con 0x7fcf5815e8d0 2024-04-25T00:41:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf577fe640 1 -- 172.21.15.29:0/3776365022 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3237343319 0 0) 0x7fcf580b3720 con 0x7fcf580b4780 2024-04-25T00:41:31.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf57fff640 1 --2- 172.21.15.29:0/3776365022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf5815e8d0 0x7fcf580cc240 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7fcf480097c0 tx=0x7fcf4801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc348201e5b7fc88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:31.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf577fe640 1 -- 172.21.15.29:0/3776365022 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf480b2070 con 0x7fcf5815e8d0 2024-04-25T00:41:31.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf577fe640 1 -- 172.21.15.29:0/3776365022 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf480ad680 con 0x7fcf5815e8d0 2024-04-25T00:41:31.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf577fe640 1 -- 172.21.15.29:0/3776365022 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf480ada50 con 0x7fcf5815e8d0 2024-04-25T00:41:31.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/3776365022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf5815e8d0 msgr2=0x7fcf580cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:31.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf5e2b4640 1 --2- 172.21.15.29:0/3776365022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf5815e8d0 0x7fcf580cc240 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7fcf480097c0 tx=0x7fcf4801c470 comp rx=0 tx=0).stop 2024-04-25T00:41:31.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/3776365022 shutdown_connections 2024-04-25T00:41:31.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf5e2b4640 1 --2- 172.21.15.29:0/3776365022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5815f3c0 0x7fcf5815c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:31.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf5e2b4640 1 --2- 172.21.15.29:0/3776365022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf5815e8d0 0x7fcf580cc240 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:31.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/3776365022 >> 172.21.15.29:0/3776365022 conn(0x7fcf580b1870 msgr2=0x7fcf580b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:31.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.096+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/3776365022 shutdown_connections 2024-04-25T00:41:31.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/3776365022 wait complete. 2024-04-25T00:41:31.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf5e2b4640 1 Processor -- start 2024-04-25T00:41:31.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf5e2b4640 1 -- start start 2024-04-25T00:41:31.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf5e2b4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf580b4780 0x7fcf58072890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:31.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf5e2b4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf5815e8d0 0x7fcf580afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:31.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf5e2b4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5815f3c0 0x7fcf581f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:31.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf5e2b4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5815d6a0 con 0x7fcf5815e8d0 2024-04-25T00:41:31.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf5e2b4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5815d520 con 0x7fcf580b4780 2024-04-25T00:41:31.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf5e2b4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcf58167c70 con 0x7fcf5815f3c0 2024-04-25T00:41:31.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf4ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf5815e8d0 0x7fcf580afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:31.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf4ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf5815e8d0 0x7fcf580afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37202/0 (socket says 172.21.15.29:37202) 2024-04-25T00:41:31.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf4ffff640 1 -- 172.21.15.29:0/181845892 learned_addr learned my addr 172.21.15.29:0/181845892 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:31.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf5c851640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5815f3c0 0x7fcf581f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:31.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf57fff640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf580b4780 0x7fcf58072890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:31.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf4ffff640 1 -- 172.21.15.29:0/181845892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5815f3c0 msgr2=0x7fcf581f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:31.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf4ffff640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5815f3c0 0x7fcf581f6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:31.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf4ffff640 1 -- 172.21.15.29:0/181845892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf580b4780 msgr2=0x7fcf58072890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:31.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf4ffff640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf580b4780 0x7fcf58072890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:31.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf4ffff640 1 -- 172.21.15.29:0/181845892 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5806d270 con 0x7fcf5815e8d0 2024-04-25T00:41:31.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf4ffff640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf5815e8d0 0x7fcf580afa10 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fcf400011f0 tx=0x7fcf4000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:31.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf557fa640 1 -- 172.21.15.29:0/181845892 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40018070 con 0x7fcf5815e8d0 2024-04-25T00:41:31.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf557fa640 1 -- 172.21.15.29:0/181845892 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf400074e0 con 0x7fcf5815e8d0 2024-04-25T00:41:31.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/181845892 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5806e390 con 0x7fcf5815e8d0 2024-04-25T00:41:31.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf557fa640 1 -- 172.21.15.29:0/181845892 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40004a10 con 0x7fcf5815e8d0 2024-04-25T00:41:31.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/181845892 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf581437a0 con 0x7fcf5815e8d0 2024-04-25T00:41:31.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.100+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/181845892 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf58069270 con 0x7fcf5815e8d0 2024-04-25T00:41:31.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.104+0000 7fcf557fa640 1 -- 172.21.15.29:0/181845892 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fcf4001b070 con 0x7fcf5815e8d0 2024-04-25T00:41:31.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.104+0000 7fcf557fa640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcf38068280 0x7fcf3806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:31.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.104+0000 7fcf557fa640 1 -- 172.21.15.29:0/181845892 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7fcf400836c0 con 0x7fcf5815e8d0 2024-04-25T00:41:31.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.104+0000 7fcf57fff640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcf38068280 0x7fcf3806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:31.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.104+0000 7fcf57fff640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcf38068280 0x7fcf3806a740 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcf480af000 tx=0x7fcf48008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:31.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.108+0000 7fcf557fa640 1 -- 172.21.15.29:0/181845892 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcf40055e00 con 0x7fcf5815e8d0 2024-04-25T00:41:31.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.296+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/181845892 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcf580753e0 con 0x7fcf5815e8d0 2024-04-25T00:41:31.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.296+0000 7fcf557fa640 1 -- 172.21.15.29:0/181845892 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) ==== 53+0+73 (secure 0 0 0) 0x7fcf40055700 con 0x7fcf5815e8d0 2024-04-25T00:41:31.299 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:41:31.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.300+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/181845892 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcf38068280 msgr2=0x7fcf3806a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:31.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.300+0000 7fcf5e2b4640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcf38068280 0x7fcf3806a740 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcf480af000 tx=0x7fcf48008040 comp rx=0 tx=0).stop 2024-04-25T00:41:31.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.300+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/181845892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf5815e8d0 msgr2=0x7fcf580afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:31.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.300+0000 7fcf5e2b4640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf5815e8d0 0x7fcf580afa10 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fcf400011f0 tx=0x7fcf4000c9b0 comp rx=0 tx=0).stop 2024-04-25T00:41:31.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.300+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/181845892 shutdown_connections 2024-04-25T00:41:31.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.300+0000 7fcf5e2b4640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fcf38068280 0x7fcf3806a740 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:31.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.300+0000 7fcf5e2b4640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5815f3c0 0x7fcf581f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:31.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.300+0000 7fcf5e2b4640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf5815e8d0 0x7fcf580afa10 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:31.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.300+0000 7fcf5e2b4640 1 --2- 172.21.15.29:0/181845892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf580b4780 0x7fcf58072890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:31.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.300+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/181845892 >> 172.21.15.29:0/181845892 conn(0x7fcf580b1870 msgr2=0x7fcf580d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:31.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.300+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/181845892 shutdown_connections 2024-04-25T00:41:31.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31.300+0000 7fcf5e2b4640 1 -- 172.21.15.29:0/181845892 wait complete. 2024-04-25T00:41:31.316 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:41:31.316 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:41:31.316 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-25T00:41:31.316 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:41:31.316 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:41:31.316 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:41:31.317 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:41:31.317 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:41:31.317 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:41:31.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:31.317 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-25T00:41:31.323 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:41:31.323 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:41:31.323 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:41:31.323 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:31.323 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-25T00:41:31.323 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:31.323 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:41:31.323 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:41:31.323 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-25T00:41:31.323 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:41:31.323 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:41:31.324 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-25T00:41:31.324 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:41:31.324 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:41:31.324 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-25T00:41:31.324 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:41:31.324 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 41868sec preferred_lft 41868sec 2024-04-25T00:41:31.324 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-04-25T00:41:31.324 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:31.324 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:41:31.324 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:41:31.324 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-25T00:41:31.324 INFO:teuthology.orchestra.run.smithi029.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 02:42:28:a5:36:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:41:31.325 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:41:31.326 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:41:31.326 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:31.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:31.326 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:41:31.378 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-25T00:41:31.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:31.380 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-25T00:41:31.428 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-25T00:41:31.428 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:41:31.429 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T00:41:31.429 INFO:teuthology.orchestra.run.smithi029.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:41:31.429 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T00:41:31.429 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:41:31.429 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:41:31.429 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-25T00:41:31.429 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:41:31.429 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:41:31.429 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:41:31.429 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-25T00:41:31.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:41:31.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:41:31.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:31.554 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:41:31.600 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:41:31.600 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:41:31.600 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:41:31.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:31.600 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-04-25T00:41:31.653 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:41:31.654 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:41:31.654 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:41:31.700 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:41:31.700 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:41:31.700 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:41:31.747 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:41:31.806 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:41:31.853 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:41:31.853 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:41:31.853 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:41:31.853 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:41:31.853 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:41:31.853 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:41:31.853 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:41:31.854 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:41:31.854 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:41:31.854 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:41:31.854 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:41:31.854 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:41:31.854 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:41:31.854 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:41:31.854 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:41:31.854 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:41:31.854 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:41:31.854 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:41:31.854 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:41:31.854 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:41:31.855 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:41:31.855 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:41:31.855 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:41:31.855 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:41:31.855 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:41:31.855 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:41:31.855 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:41:31.855 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:41:31.855 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:41:31.855 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:41:31.855 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:41:31.855 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:41:31.855 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:41:31.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:31.856 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:41:31.904 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:41:31.904 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:41:31.949 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:41:31.964 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:41:32.012 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:41:32.012 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:41:32.012 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:41:32.012 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:41:32.012 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:41:32.013 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:41:32.014 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:41:32.014 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:41:32.014 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:41:32.014 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:41:32.014 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:41:32.014 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:41:32.014 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:41:32.014 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:41:32.014 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:41:32.014 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:41:32.015 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:41:32.015 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:41:32.015 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:41:32.015 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:41:32.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:32.016 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:41:32.028 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:2024-04-25T00:41:32.028+0000 7fa89668e580 -1 init, newargv = 0x55799bb8ee80 newargc=15 2024-04-25T00:41:32.028 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[58708]: starting ceph client 2024-04-25T00:41:32.035 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[58708]: starting fuse 2024-04-25T00:41:33.134 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-04-25T00:41:33.150 DEBUG:teuthology.orchestra.run.smithi029:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:41:33.198 INFO:teuthology.orchestra.run.smithi029.stdout:/sys 2024-04-25T00:41:33.198 INFO:teuthology.orchestra.run.smithi029.stdout:/proc 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/dev 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/pts 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/run 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/ 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/security 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/shm 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/run/lock 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/cgroup 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/pstore 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/bpf 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/hugepages 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/dev/mqueue 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/debug 2024-04-25T00:41:33.199 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/tracing 2024-04-25T00:41:33.200 INFO:teuthology.orchestra.run.smithi029.stdout:/proc/fs/nfsd 2024-04-25T00:41:33.201 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/fs/fuse/connections 2024-04-25T00:41:33.201 INFO:teuthology.orchestra.run.smithi029.stdout:/sys/kernel/config 2024-04-25T00:41:33.201 INFO:teuthology.orchestra.run.smithi029.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:41:33.201 INFO:teuthology.orchestra.run.smithi029.stdout:/run/rpc_pipefs 2024-04-25T00:41:33.201 INFO:teuthology.orchestra.run.smithi029.stdout:/run/user/1000 2024-04-25T00:41:33.202 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph 2024-04-25T00:41:33.202 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:41:33.202 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:41:33.202 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:41:33.202 INFO:teuthology.orchestra.run.smithi029.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:41:33.202 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns 2024-04-25T00:41:33.202 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:41:33.202 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:41:33.202 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:41:33.202 INFO:teuthology.orchestra.run.smithi029.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:41:33.202 INFO:teuthology.orchestra.run.smithi029.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:41:33.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:33.204 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-04-25T00:41:33.253 INFO:teuthology.orchestra.run.smithi029.stdout:45 2024-04-25T00:41:33.253 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:41:33.254 DEBUG:teuthology.orchestra.run.smithi029:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:41:33.254 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-04-25T00:41:33.254 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-04-25T00:41:33.254 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-04-25T00:41:33.254 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-04-25T00:41:33.254 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:41:33.254 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-04-25T00:41:33.254 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:41:33.254 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-04-25T00:41:33.254 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:41:33.254 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> 2024-04-25T00:41:33.255 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.0")) 2024-04-25T00:41:33.256 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-04-25T00:41:33.380 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.0.58708.asok 2024-04-25T00:41:33.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:41:33.393 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.58708.asok 2024-04-25T00:41:33.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:33.393 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.58708.asok status 2024-04-25T00:41:33.552 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:41:33.552 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-04-25T00:41:33.552 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:41:33.552 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:41:33.552 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "0", 2024-04-25T00:41:33.552 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-04-25T00:41:33.552 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:41:33.552 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "58708", 2024-04-25T00:41:33.553 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-04-25T00:41:33.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:33.553 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-04-25T00:41:33.553 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-04-25T00:41:33.553 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 10764, 2024-04-25T00:41:33.553 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-04-25T00:41:33.553 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-04-25T00:41:33.553 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-04-25T00:41:33.553 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 10764 2024-04-25T00:41:33.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:33.553 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:41:33.553 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:41:33.553 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:41:33.554 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3281378362 2024-04-25T00:41:33.554 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:33.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:33.554 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-04-25T00:41:33.554 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-04-25T00:41:33.554 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:41:33.554 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 3281378362 2024-04-25T00:41:33.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-04-25T00:41:33.554 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.10764 192.168.0.1:0/3281378362", 2024-04-25T00:41:33.554 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.1:0/3281378362", 2024-04-25T00:41:33.554 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-04-25T00:41:33.554 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 112, 2024-04-25T00:41:33.554 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 168, 2024-04-25T00:41:33.555 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:41:33.555 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-04-25T00:41:33.555 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-04-25T00:41:33.555 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:41:33.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:33.567 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-04-25T00:41:33.567 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:41:33.573 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-04-25T00:41:33.574 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:41:33.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:33.574 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:41:33.629 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:41:33.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.800+0000 7f7da970e640 1 Processor -- start 2024-04-25T00:41:33.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.800+0000 7f7da970e640 1 -- start start 2024-04-25T00:41:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da970e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7da40b3140 0x7f7da406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da970e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7da40b3c30 0x7f7da40ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da970e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7da40b41e0 con 0x7f7da40b4b10 2024-04-25T00:41:33.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da970e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7da406d960 con 0x7f7da40b3140 2024-04-25T00:41:33.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da970e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7da406dc40 con 0x7f7da40b3c30 2024-04-25T00:41:33.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da27fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7da40b4b10 0x7f7da40b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35830/0 (socket says 172.21.15.29:35830) 2024-04-25T00:41:33.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da27fc640 1 -- 172.21.15.29:0/1352165853 learned_addr learned my addr 172.21.15.29:0/1352165853 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:33.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da2ffd640 1 --2- 172.21.15.29:0/1352165853 >> v2:172.21.15.29:3300/0 conn(0x7f7da40b3140 0x7f7da406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:33.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da37fe640 1 --2- 172.21.15.29:0/1352165853 >> v2:172.21.15.157:3300/0 conn(0x7f7da40b3c30 0x7f7da40ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:33.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da1ffb640 1 -- 172.21.15.29:0/1352165853 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1697912979 0 0) 0x7f7da40b41e0 con 0x7f7da40b4b10 2024-04-25T00:41:33.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da1ffb640 1 -- 172.21.15.29:0/1352165853 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d8c003530 con 0x7f7da40b4b10 2024-04-25T00:41:33.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da2ffd640 1 -- 172.21.15.29:0/1352165853 >> v2:172.21.15.157:3300/0 conn(0x7f7da40b3c30 msgr2=0x7f7da40ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:33.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da2ffd640 1 --2- 172.21.15.29:0/1352165853 >> v2:172.21.15.157:3300/0 conn(0x7f7da40b3c30 0x7f7da40ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:33.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da2ffd640 1 -- 172.21.15.29:0/1352165853 >> v1:172.21.15.29:6789/0 conn(0x7f7da40b4b10 legacy=0x7f7da40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:33.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da2ffd640 1 -- 172.21.15.29:0/1352165853 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da406df20 con 0x7f7da40b3140 2024-04-25T00:41:33.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.804+0000 7f7da37fe640 1 --2- 172.21.15.29:0/1352165853 >> v2:172.21.15.157:3300/0 conn(0x7f7da40b3c30 0x7f7da40ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:33.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da2ffd640 1 --2- 172.21.15.29:0/1352165853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40b3140 0x7f7da406e440 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f7d940097c0 tx=0x7f7d9401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897ad9cd1af934d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:33.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da1ffb640 1 -- 172.21.15.29:0/1352165853 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d940b0070 con 0x7f7da40b3140 2024-04-25T00:41:33.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da1ffb640 1 -- 172.21.15.29:0/1352165853 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d940ab600 con 0x7f7da40b3140 2024-04-25T00:41:33.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da1ffb640 1 -- 172.21.15.29:0/1352165853 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d940ab9d0 con 0x7f7da40b3140 2024-04-25T00:41:33.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da970e640 1 -- 172.21.15.29:0/1352165853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40b3140 msgr2=0x7f7da406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:33.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da970e640 1 --2- 172.21.15.29:0/1352165853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40b3140 0x7f7da406e440 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f7d940097c0 tx=0x7f7d9401c460 comp rx=0 tx=0).stop 2024-04-25T00:41:33.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da970e640 1 -- 172.21.15.29:0/1352165853 shutdown_connections 2024-04-25T00:41:33.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da970e640 1 --2- 172.21.15.29:0/1352165853 >> v2:172.21.15.157:3300/0 conn(0x7f7da40b3c30 0x7f7da40ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:33.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da970e640 1 --2- 172.21.15.29:0/1352165853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40b3140 0x7f7da406e440 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:33.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da970e640 1 -- 172.21.15.29:0/1352165853 >> 172.21.15.29:0/1352165853 conn(0x7f7da40b1870 msgr2=0x7f7da40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:33.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da970e640 1 -- 172.21.15.29:0/1352165853 shutdown_connections 2024-04-25T00:41:33.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da970e640 1 -- 172.21.15.29:0/1352165853 wait complete. 2024-04-25T00:41:33.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da970e640 1 Processor -- start 2024-04-25T00:41:33.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.808+0000 7f7da970e640 1 -- start start 2024-04-25T00:41:33.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da970e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da40b3140 0x7f7da41385d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da970e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40b3c30 0x7f7da413de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da970e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da40b4b10 0x7f7da4139d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da970e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4168d80 con 0x7f7da40b3c30 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da970e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4168c00 con 0x7f7da40b4b10 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da970e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7da4168f00 con 0x7f7da40b3140 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40b3c30 0x7f7da413de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40b3c30 0x7f7da413de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37220/0 (socket says 172.21.15.29:37220) 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da27fc640 1 -- 172.21.15.29:0/3137760760 learned_addr learned my addr 172.21.15.29:0/3137760760 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da37fe640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da40b4b10 0x7f7da4139d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da2ffd640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da40b3140 0x7f7da41385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da27fc640 1 -- 172.21.15.29:0/3137760760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da40b3140 msgr2=0x7f7da41385d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da27fc640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da40b3140 0x7f7da41385d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da27fc640 1 -- 172.21.15.29:0/3137760760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da40b4b10 msgr2=0x7f7da4139d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:33.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da27fc640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da40b4b10 0x7f7da4139d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:33.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da27fc640 1 -- 172.21.15.29:0/3137760760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da413be50 con 0x7f7da40b3c30 2024-04-25T00:41:33.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da2ffd640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da40b3140 0x7f7da41385d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:33.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da27fc640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40b3c30 0x7f7da413de30 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800c480 tx=0x7f7d9800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:33.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7d8b7fe640 1 -- 172.21.15.29:0/3137760760 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98017070 con 0x7f7da40b3c30 2024-04-25T00:41:33.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7d8b7fe640 1 -- 172.21.15.29:0/3137760760 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d98005cf0 con 0x7f7da40b3c30 2024-04-25T00:41:33.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7d8b7fe640 1 -- 172.21.15.29:0/3137760760 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98007290 con 0x7f7da40b3c30 2024-04-25T00:41:33.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da970e640 1 -- 172.21.15.29:0/3137760760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da413e370 con 0x7f7da40b3c30 2024-04-25T00:41:33.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da970e640 1 -- 172.21.15.29:0/3137760760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da413d4c0 con 0x7f7da40b3c30 2024-04-25T00:41:33.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7da970e640 1 -- 172.21.15.29:0/3137760760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da4069270 con 0x7f7da40b3c30 2024-04-25T00:41:33.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7d8b7fe640 1 -- 172.21.15.29:0/3137760760 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f7d9801a070 con 0x7f7da40b3c30 2024-04-25T00:41:33.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.812+0000 7f7d8b7fe640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d90068250 0x7f7d9006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:33.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.816+0000 7f7d8b7fe640 1 -- 172.21.15.29:0/3137760760 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f7d98083870 con 0x7f7da40b3c30 2024-04-25T00:41:33.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.816+0000 7f7da2ffd640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d90068250 0x7f7d9006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:33.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.816+0000 7f7da2ffd640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d90068250 0x7f7d9006a710 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7d940ad000 tx=0x7f7d94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:33.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:33.820+0000 7f7d8b7fe640 1 -- 172.21.15.29:0/3137760760 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7d98055fb0 con 0x7f7da40b3c30 2024-04-25T00:41:34.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.016+0000 7f7da970e640 1 -- 172.21.15.29:0/3137760760 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7da415c410 con 0x7f7da40b3c30 2024-04-25T00:41:34.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.016+0000 7f7d8b7fe640 1 -- 172.21.15.29:0/3137760760 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) ==== 53+0+73 (secure 0 0 0) 0x7f7d980558b0 con 0x7f7da40b3c30 2024-04-25T00:41:34.019 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:41:34.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.020+0000 7f7da970e640 1 -- 172.21.15.29:0/3137760760 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d90068250 msgr2=0x7f7d9006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:34.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.020+0000 7f7da970e640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d90068250 0x7f7d9006a710 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7d940ad000 tx=0x7f7d94008040 comp rx=0 tx=0).stop 2024-04-25T00:41:34.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.020+0000 7f7da970e640 1 -- 172.21.15.29:0/3137760760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40b3c30 msgr2=0x7f7da413de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:34.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.020+0000 7f7da970e640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40b3c30 0x7f7da413de30 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800c480 tx=0x7f7d9800c940 comp rx=0 tx=0).stop 2024-04-25T00:41:34.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.020+0000 7f7da970e640 1 -- 172.21.15.29:0/3137760760 shutdown_connections 2024-04-25T00:41:34.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.020+0000 7f7da970e640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d90068250 0x7f7d9006a710 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:34.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.020+0000 7f7da970e640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da40b4b10 0x7f7da4139d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:34.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.020+0000 7f7da970e640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7da40b3c30 0x7f7da413de30 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:34.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.020+0000 7f7da970e640 1 --2- 172.21.15.29:0/3137760760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da40b3140 0x7f7da41385d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:34.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.020+0000 7f7da970e640 1 -- 172.21.15.29:0/3137760760 >> 172.21.15.29:0/3137760760 conn(0x7f7da40b1870 msgr2=0x7f7da415cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:34.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.020+0000 7f7da970e640 1 -- 172.21.15.29:0/3137760760 shutdown_connections 2024-04-25T00:41:34.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:34.020+0000 7f7da970e640 1 -- 172.21.15.29:0/3137760760 wait complete. 2024-04-25T00:41:34.039 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:41:34.039 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:41:34.039 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-25T00:41:34.039 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:41:34.039 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:41:34.039 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:41:34.039 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:41:34.039 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:41:34.039 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:41:34.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:34.040 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-25T00:41:34.048 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:41:34.048 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:41:34.048 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:41:34.048 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:34.048 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-25T00:41:34.048 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:34.048 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:41:34.048 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:41:34.048 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-25T00:41:34.048 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 41861sec preferred_lft 41861sec 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:9c:8d:4b:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:41:34.049 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:41:34.050 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:41:34.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:34.051 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:41:34.107 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-25T00:41:34.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:34.110 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-25T00:41:34.163 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-25T00:41:34.163 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:41:34.164 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:41:34.164 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:41:34.164 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:41:34.164 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:41:34.164 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:41:34.164 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:41:34.165 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:41:34.165 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:41:34.165 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:41:34.165 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:41:34.165 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-25T00:41:34.165 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:41:34.165 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:41:34.166 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:41:34.166 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-25T00:41:34.251 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:41:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:41:34.280 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:41:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:41:34.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:34.287 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:41:34.293 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:41:34.293 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:41:34.293 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:41:34.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:34.293 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-25T00:41:34.346 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T00:41:34.346 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:41:34.346 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:41:34.394 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:41:34.394 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:41:34.394 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:41:34.442 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:41:34.499 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:41:34.547 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:41:34.547 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:41:34.547 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:41:34.547 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:41:34.547 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:41:34.547 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:41:34.548 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:41:34.549 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:41:34.549 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:41:34.549 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:41:34.549 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:41:34.549 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:41:34.549 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:41:34.549 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:41:34.549 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:41:34.549 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:41:34.549 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:41:34.549 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:41:34.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:34.550 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:41:34.598 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:41:34.598 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T00:41:34.643 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:41:34.657 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:41:34.703 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:41:34.703 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:41:34.703 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:41:34.703 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:41:34.703 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:41:34.703 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:41:34.703 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:41:34.703 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:41:34.703 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:41:34.703 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:41:34.703 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:41:34.704 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:41:34.704 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:41:34.704 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:41:34.704 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:41:34.704 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:41:34.704 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:41:34.704 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:41:34.704 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:41:34.704 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:41:34.704 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:41:34.704 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:41:34.704 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:41:34.704 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:41:34.705 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:41:34.705 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:41:34.705 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:41:34.705 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:41:34.705 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:41:34.705 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:41:34.705 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:41:34.705 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:41:34.705 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:41:34.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:34.706 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:41:34.724 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-04-25T00:41:34.723+0000 7ffb0cde4580 -1 init, newargv = 0x558b93de3f50 newargc=15 2024-04-25T00:41:34.724 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[35373]: starting ceph client 2024-04-25T00:41:34.731 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[35373]: starting fuse 2024-04-25T00:41:35.830 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-25T00:41:35.847 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:41:35.895 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/run/lock 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-25T00:41:35.896 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/fs/nfsd 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/run/rpc_pipefs 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-25T00:41:35.897 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:41:35.898 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:41:35.898 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:41:35.898 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:41:35.898 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:41:35.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:35.899 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-25T00:41:35.946 INFO:teuthology.orchestra.run.smithi157.stdout:45 2024-04-25T00:41:35.947 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:41:35.947 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:41:35.947 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-25T00:41:35.947 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-25T00:41:35.947 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-25T00:41:35.947 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-25T00:41:35.947 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:41:35.947 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-25T00:41:35.947 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T00:41:35.947 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-25T00:41:35.947 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:41:35.948 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-25T00:41:35.949 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-04-25T00:41:35.949 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-25T00:41:36.069 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.35373.asok 2024-04-25T00:41:36.073 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-25T00:41:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:41:36.081 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35373.asok 2024-04-25T00:41:36.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:36.081 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35373.asok status 2024-04-25T00:41:36.240 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-25T00:41:36.241 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-25T00:41:36.241 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:41:36.241 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:41:36.241 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-04-25T00:41:36.241 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-25T00:41:36.241 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T00:41:36.241 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "35373", 2024-04-25T00:41:36.241 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-25T00:41:36.241 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:41:36.241 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-25T00:41:36.241 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-25T00:41:36.241 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 8530, 2024-04-25T00:41:36.241 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-25T00:41:36.242 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-25T00:41:36.242 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-25T00:41:36.242 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 8530 2024-04-25T00:41:36.242 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:41:36.242 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:41:36.242 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:41:36.242 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:41:36.242 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2399121581 2024-04-25T00:41:36.242 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-25T00:41:36.242 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:41:36.242 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-25T00:41:36.242 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-25T00:41:36.242 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:41:36.243 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2399121581 2024-04-25T00:41:36.243 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-25T00:41:36.243 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.8530 192.168.0.1:0/2399121581", 2024-04-25T00:41:36.243 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/2399121581", 2024-04-25T00:41:36.243 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-25T00:41:36.243 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 112, 2024-04-25T00:41:36.243 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 168, 2024-04-25T00:41:36.243 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:41:36.243 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-25T00:41:36.243 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-25T00:41:36.243 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-25T00:41:36.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:36.256 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-25T00:41:36.256 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T00:41:36.263 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-25T00:41:36.263 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T00:41:36.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:36.263 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:41:36.317 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-25T00:41:36.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.456+0000 7f77c45e1640 1 Processor -- start 2024-04-25T00:41:36.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.456+0000 7f77c45e1640 1 -- start start 2024-04-25T00:41:36.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c45e1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc15bfb0 0x7f77bc06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c45e1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77bc06b920 0x7f77bc15fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c45e1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77bc15c6b0 con 0x7f77bc0703d0 2024-04-25T00:41:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c45e1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f77bc15c950 con 0x7f77bc06b920 2024-04-25T00:41:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c45e1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f77bc15cc30 con 0x7f77bc15bfb0 2024-04-25T00:41:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c2b7e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77bc06b920 0x7f77bc15fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c237d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc15bfb0 0x7f77bc06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c2b7e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77bc06b920 0x7f77bc15fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:44188/0 (socket says 172.21.15.29:44188) 2024-04-25T00:41:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c2b7e640 1 -- 172.21.15.29:0/2576780083 learned_addr learned my addr 172.21.15.29:0/2576780083 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c2b7e640 1 -- 172.21.15.29:0/2576780083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc15bfb0 msgr2=0x7f77bc06fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:36.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c2b7e640 1 --2- 172.21.15.29:0/2576780083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc15bfb0 0x7f77bc06fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c2b7e640 1 -- 172.21.15.29:0/2576780083 >> v1:172.21.15.157:6789/0 conn(0x7f77bc0703d0 legacy=0x7f77bc06b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:36.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c2b7e640 1 -- 172.21.15.29:0/2576780083 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc15cf10 con 0x7f77bc06b920 2024-04-25T00:41:36.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.460+0000 7f77c237d640 1 --2- 172.21.15.29:0/2576780083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc15bfb0 0x7f77bc06fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:36.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.464+0000 7f77c2b7e640 1 --2- 172.21.15.29:0/2576780083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc06b920 0x7f77bc15fb90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009e50 tx=0x7f77b401c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb51f92bf8374e3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:36.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.464+0000 7f77c137b640 1 -- 172.21.15.29:0/2576780083 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b40a5660 con 0x7f77bc06b920 2024-04-25T00:41:36.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.464+0000 7f77c137b640 1 -- 172.21.15.29:0/2576780083 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b40a5800 con 0x7f77bc06b920 2024-04-25T00:41:36.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.464+0000 7f77c137b640 1 -- 172.21.15.29:0/2576780083 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b40a5bd0 con 0x7f77bc06b920 2024-04-25T00:41:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.464+0000 7f77c45e1640 1 -- 172.21.15.29:0/2576780083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc06b920 msgr2=0x7f77bc15fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.464+0000 7f77c45e1640 1 --2- 172.21.15.29:0/2576780083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc06b920 0x7f77bc15fb90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009e50 tx=0x7f77b401c9a0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.464+0000 7f77c45e1640 1 -- 172.21.15.29:0/2576780083 shutdown_connections 2024-04-25T00:41:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.464+0000 7f77c45e1640 1 --2- 172.21.15.29:0/2576780083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc06b920 0x7f77bc15fb90 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.464+0000 7f77c45e1640 1 --2- 172.21.15.29:0/2576780083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc15bfb0 0x7f77bc06fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.464+0000 7f77c45e1640 1 -- 172.21.15.29:0/2576780083 >> 172.21.15.29:0/2576780083 conn(0x7f77bc0d0da0 msgr2=0x7f77bc0d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.464+0000 7f77c45e1640 1 -- 172.21.15.29:0/2576780083 shutdown_connections 2024-04-25T00:41:36.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.464+0000 7f77c45e1640 1 -- 172.21.15.29:0/2576780083 wait complete. 2024-04-25T00:41:36.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c45e1640 1 Processor -- start 2024-04-25T00:41:36.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c45e1640 1 -- start start 2024-04-25T00:41:36.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c45e1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc06b920 0x7f77bc0b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c45e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc0703d0 0x7f77bc0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c45e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc15bfb0 0x7f77bc0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c45e1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc1690b0 con 0x7f77bc15bfb0 2024-04-25T00:41:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c45e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc168f30 con 0x7f77bc0703d0 2024-04-25T00:41:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c45e1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f77bc169230 con 0x7f77bc06b920 2024-04-25T00:41:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c2b7e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc15bfb0 0x7f77bc0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c2b7e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc15bfb0 0x7f77bc0c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37232/0 (socket says 172.21.15.29:37232) 2024-04-25T00:41:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c2b7e640 1 -- 172.21.15.29:0/2855613845 learned_addr learned my addr 172.21.15.29:0/2855613845 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c237d640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc06b920 0x7f77bc0b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c1b7c640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc0703d0 0x7f77bc0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c2b7e640 1 -- 172.21.15.29:0/2855613845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc06b920 msgr2=0x7f77bc0b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:36.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c2b7e640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc06b920 0x7f77bc0b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c2b7e640 1 -- 172.21.15.29:0/2855613845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc0703d0 msgr2=0x7f77bc0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:36.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c2b7e640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc0703d0 0x7f77bc0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c2b7e640 1 -- 172.21.15.29:0/2855613845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc1577e0 con 0x7f77bc15bfb0 2024-04-25T00:41:36.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c237d640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc06b920 0x7f77bc0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:36.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c2b7e640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc15bfb0 0x7f77bc0c5400 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f77b401a5d0 tx=0x7f77b4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:36.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77b2ffd640 1 -- 172.21.15.29:0/2855613845 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b40b8470 con 0x7f77bc15bfb0 2024-04-25T00:41:36.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77b2ffd640 1 -- 172.21.15.29:0/2855613845 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b40b8610 con 0x7f77bc15bfb0 2024-04-25T00:41:36.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77b2ffd640 1 -- 172.21.15.29:0/2855613845 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b40b8a00 con 0x7f77bc15bfb0 2024-04-25T00:41:36.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c45e1640 1 -- 172.21.15.29:0/2855613845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77bc0ceac0 con 0x7f77bc15bfb0 2024-04-25T00:41:36.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c45e1640 1 -- 172.21.15.29:0/2855613845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77bc0cf080 con 0x7f77bc15bfb0 2024-04-25T00:41:36.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77c45e1640 1 -- 172.21.15.29:0/2855613845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77bc0692e0 con 0x7f77bc15bfb0 2024-04-25T00:41:36.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.468+0000 7f77b2ffd640 1 -- 172.21.15.29:0/2855613845 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f77b40c0020 con 0x7f77bc15bfb0 2024-04-25T00:41:36.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.472+0000 7f77b2ffd640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f77a0068250 0x7f77a006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:36.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.472+0000 7f77c237d640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f77a0068250 0x7f77a006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:36.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.472+0000 7f77b2ffd640 1 -- 172.21.15.29:0/2855613845 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f77b411eb30 con 0x7f77bc15bfb0 2024-04-25T00:41:36.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.472+0000 7f77c237d640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f77a0068250 0x7f77a006a710 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009bc0 tx=0x7f77b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:36.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.476+0000 7f77b2ffd640 1 -- 172.21.15.29:0/2855613845 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f77b40c7020 con 0x7f77bc15bfb0 2024-04-25T00:41:36.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.672+0000 7f77c45e1640 1 -- 172.21.15.29:0/2855613845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f77bc068d70 con 0x7f77bc15bfb0 2024-04-25T00:41:36.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.772+0000 7f77b2ffd640 1 -- 172.21.15.29:0/2855613845 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v113) ==== 136+0+0 (secure 0 0 0) 0x7f77b4016620 con 0x7f77bc15bfb0 2024-04-25T00:41:36.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.776+0000 7f77c45e1640 1 -- 172.21.15.29:0/2855613845 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f77a0068250 msgr2=0x7f77a006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:36.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.776+0000 7f77c45e1640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f77a0068250 0x7f77a006a710 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009bc0 tx=0x7f77b8008040 comp rx=0 tx=0).stop 2024-04-25T00:41:36.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.776+0000 7f77c45e1640 1 -- 172.21.15.29:0/2855613845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc15bfb0 msgr2=0x7f77bc0c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:36.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.776+0000 7f77c45e1640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc15bfb0 0x7f77bc0c5400 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f77b401a5d0 tx=0x7f77b4019740 comp rx=0 tx=0).stop 2024-04-25T00:41:36.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.776+0000 7f77c45e1640 1 -- 172.21.15.29:0/2855613845 shutdown_connections 2024-04-25T00:41:36.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.776+0000 7f77c45e1640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f77a0068250 0x7f77a006a710 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.776+0000 7f77c45e1640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc15bfb0 0x7f77bc0c5400 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.776+0000 7f77c45e1640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc0703d0 0x7f77bc0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.776+0000 7f77c45e1640 1 --2- 172.21.15.29:0/2855613845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc06b920 0x7f77bc0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.776+0000 7f77c45e1640 1 -- 172.21.15.29:0/2855613845 >> 172.21.15.29:0/2855613845 conn(0x7f77bc0d0da0 msgr2=0x7f77bc1585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:36.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.776+0000 7f77c45e1640 1 -- 172.21.15.29:0/2855613845 shutdown_connections 2024-04-25T00:41:36.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.776+0000 7f77c45e1640 1 -- 172.21.15.29:0/2855613845 wait complete. 2024-04-25T00:41:36.793 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-25T00:41:36.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eb64f1640 1 Processor -- start 2024-04-25T00:41:36.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eb64f1640 1 -- start start 2024-04-25T00:41:36.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eb64f1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9eb00d08a0 0x7f9eb00b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:36.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eb64f1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9eb006d580 0x7f9eb00b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:36.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eb64f1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb0143a90 con 0x7f9eb0147860 2024-04-25T00:41:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eb64f1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb0144660 con 0x7f9eb00d08a0 2024-04-25T00:41:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eb64f1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb00d0c80 con 0x7f9eb006d580 2024-04-25T00:41:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eaffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9eb00d08a0 0x7f9eb00b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eb4a8e640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9eb0147860 0x7f9eb0071560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35836/0 (socket says 172.21.15.29:35836) 2024-04-25T00:41:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eb4a8e640 1 -- 172.21.15.29:0/2947070070 learned_addr learned my addr 172.21.15.29:0/2947070070 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eaf7fe640 1 --2- 172.21.15.29:0/2947070070 >> v2:172.21.15.157:3300/0 conn(0x7f9eb006d580 0x7f9eb00b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eaeffd640 1 -- 172.21.15.29:0/2947070070 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3834170633 0 0) 0x7f9eb0143a90 con 0x7f9eb0147860 2024-04-25T00:41:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eaeffd640 1 -- 172.21.15.29:0/2947070070 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e9c003530 con 0x7f9eb0147860 2024-04-25T00:41:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eaffff640 1 -- 172.21.15.29:0/2947070070 >> v2:172.21.15.157:3300/0 conn(0x7f9eb006d580 msgr2=0x7f9eb00b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eaffff640 1 --2- 172.21.15.29:0/2947070070 >> v2:172.21.15.157:3300/0 conn(0x7f9eb006d580 0x7f9eb00b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eaffff640 1 -- 172.21.15.29:0/2947070070 >> v1:172.21.15.29:6789/0 conn(0x7f9eb0147860 legacy=0x7f9eb0071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eaffff640 1 -- 172.21.15.29:0/2947070070 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb0143d30 con 0x7f9eb00d08a0 2024-04-25T00:41:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.908+0000 7f9eaf7fe640 1 --2- 172.21.15.29:0/2947070070 >> v2:172.21.15.157:3300/0 conn(0x7f9eb006d580 0x7f9eb00b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:36.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eaffff640 1 --2- 172.21.15.29:0/2947070070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00d08a0 0x7f9eb00b3380 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f9eb01447e0 tx=0x7f9ea801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cc45bd7cbe8a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:36.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eaeffd640 1 -- 172.21.15.29:0/2947070070 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea80b1070 con 0x7f9eb00d08a0 2024-04-25T00:41:36.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eaeffd640 1 -- 172.21.15.29:0/2947070070 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea80ac620 con 0x7f9eb00d08a0 2024-04-25T00:41:36.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eaeffd640 1 -- 172.21.15.29:0/2947070070 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea80ac9f0 con 0x7f9eb00d08a0 2024-04-25T00:41:36.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 -- 172.21.15.29:0/2947070070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00d08a0 msgr2=0x7f9eb00b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:36.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 --2- 172.21.15.29:0/2947070070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00d08a0 0x7f9eb00b3380 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f9eb01447e0 tx=0x7f9ea801c3a0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 -- 172.21.15.29:0/2947070070 shutdown_connections 2024-04-25T00:41:36.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 --2- 172.21.15.29:0/2947070070 >> v2:172.21.15.157:3300/0 conn(0x7f9eb006d580 0x7f9eb00b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 --2- 172.21.15.29:0/2947070070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00d08a0 0x7f9eb00b3380 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 -- 172.21.15.29:0/2947070070 >> 172.21.15.29:0/2947070070 conn(0x7f9eb0074c00 msgr2=0x7f9eb006b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:36.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 -- 172.21.15.29:0/2947070070 shutdown_connections 2024-04-25T00:41:36.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 -- 172.21.15.29:0/2947070070 wait complete. 2024-04-25T00:41:36.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 Processor -- start 2024-04-25T00:41:36.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 -- start start 2024-04-25T00:41:36.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb006d580 0x7f9eb00b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:36.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb00d08a0 0x7f9eb015c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:36.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb0147860 0x7f9eb013d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:36.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb016b160 con 0x7f9eb0147860 2024-04-25T00:41:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb016afe0 con 0x7f9eb00d08a0 2024-04-25T00:41:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.912+0000 7f9eb64f1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb016b2e0 con 0x7f9eb006d580 2024-04-25T00:41:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eb4a8e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb0147860 0x7f9eb013d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eb4a8e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb0147860 0x7f9eb013d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37254/0 (socket says 172.21.15.29:37254) 2024-04-25T00:41:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eb4a8e640 1 -- 172.21.15.29:0/563668250 learned_addr learned my addr 172.21.15.29:0/563668250 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eaffff640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb006d580 0x7f9eb00b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eb4a8e640 1 -- 172.21.15.29:0/563668250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb006d580 msgr2=0x7f9eb00b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eb4a8e640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb006d580 0x7f9eb00b1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eb4a8e640 1 -- 172.21.15.29:0/563668250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb00d08a0 msgr2=0x7f9eb015c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eaf7fe640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb00d08a0 0x7f9eb015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eb4a8e640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb00d08a0 0x7f9eb015c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eb4a8e640 1 -- 172.21.15.29:0/563668250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb01402d0 con 0x7f9eb0147860 2024-04-25T00:41:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eb4a8e640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb0147860 0x7f9eb013d250 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000cb10 tx=0x7f9ea000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eacff9640 1 -- 172.21.15.29:0/563668250 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0017070 con 0x7f9eb0147860 2024-04-25T00:41:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eacff9640 1 -- 172.21.15.29:0/563668250 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea000ac70 con 0x7f9eb0147860 2024-04-25T00:41:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eacff9640 1 -- 172.21.15.29:0/563668250 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000ae60 con 0x7f9eb0147860 2024-04-25T00:41:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eb64f1640 1 -- 172.21.15.29:0/563668250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb0140570 con 0x7f9eb0147860 2024-04-25T00:41:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eb64f1640 1 -- 172.21.15.29:0/563668250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb0146300 con 0x7f9eb0147860 2024-04-25T00:41:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eb64f1640 1 -- 172.21.15.29:0/563668250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb0069270 con 0x7f9eb0147860 2024-04-25T00:41:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eacff9640 1 -- 172.21.15.29:0/563668250 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f9ea001a070 con 0x7f9eb0147860 2024-04-25T00:41:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eacff9640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9e90068250 0x7f9e9006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eacff9640 1 -- 172.21.15.29:0/563668250 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f9ea00841b0 con 0x7f9eb0147860 2024-04-25T00:41:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eaffff640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9e90068250 0x7f9e9006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.916+0000 7f9eaffff640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9e90068250 0x7f9e9006a710 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80ae000 tx=0x7f9ea8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:36.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:36.920+0000 7f9eacff9640 1 -- 172.21.15.29:0/563668250 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9ea00568f0 con 0x7f9eb0147860 2024-04-25T00:41:37.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.104+0000 7f9eb64f1640 1 -- 172.21.15.29:0/563668250 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f9eb006b9f0 con 0x7f9e90068250 2024-04-25T00:41:37.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.104+0000 7f9eacff9640 1 -- 172.21.15.29:0/563668250 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f9eb006b9f0 con 0x7f9e90068250 2024-04-25T00:41:37.107 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-04-25T00:41:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-04-25T00:41:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-04-25T00:41:37.107 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-04-25T00:41:37.107 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-04-25T00:41:37.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.108+0000 7f9eb64f1640 1 -- 172.21.15.29:0/563668250 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9e90068250 msgr2=0x7f9e9006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:37.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.108+0000 7f9eb64f1640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9e90068250 0x7f9e9006a710 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80ae000 tx=0x7f9ea8008040 comp rx=0 tx=0).stop 2024-04-25T00:41:37.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.108+0000 7f9eb64f1640 1 -- 172.21.15.29:0/563668250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb0147860 msgr2=0x7f9eb013d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:37.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.108+0000 7f9eb64f1640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb0147860 0x7f9eb013d250 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000cb10 tx=0x7f9ea000a060 comp rx=0 tx=0).stop 2024-04-25T00:41:37.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.108+0000 7f9eb64f1640 1 -- 172.21.15.29:0/563668250 shutdown_connections 2024-04-25T00:41:37.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.108+0000 7f9eb64f1640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9e90068250 0x7f9e9006a710 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.108+0000 7f9eb64f1640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb0147860 0x7f9eb013d250 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.108+0000 7f9eb64f1640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb00d08a0 0x7f9eb015c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.108+0000 7f9eb64f1640 1 --2- 172.21.15.29:0/563668250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb006d580 0x7f9eb00b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.108+0000 7f9eb64f1640 1 -- 172.21.15.29:0/563668250 >> 172.21.15.29:0/563668250 conn(0x7f9eb0074c00 msgr2=0x7f9eb00b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:37.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.108+0000 7f9eb64f1640 1 -- 172.21.15.29:0/563668250 shutdown_connections 2024-04-25T00:41:37.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.108+0000 7f9eb64f1640 1 -- 172.21.15.29:0/563668250 wait complete. 2024-04-25T00:41:37.125 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:41:37.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.252+0000 7f90aff35640 1 Processor -- start 2024-04-25T00:41:37.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.252+0000 7f90aff35640 1 -- start start 2024-04-25T00:41:37.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.252+0000 7f90aff35640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90a80b3140 0x7f90a806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.252+0000 7f90aff35640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a80b41e0 con 0x7f90a80b4b10 2024-04-25T00:41:37.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.252+0000 7f90aff35640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a806d960 con 0x7f90a80b3c30 2024-04-25T00:41:37.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.252+0000 7f90aff35640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f90a806dc40 con 0x7f90a80b3140 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90ad4d0640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f90a80b4b10 0x7f90a80b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35846/0 (socket says 172.21.15.29:35846) 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90ad4d0640 1 -- 172.21.15.29:0/917409001 learned_addr learned my addr 172.21.15.29:0/917409001 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90adcd1640 1 --2- 172.21.15.29:0/917409001 >> v2:172.21.15.157:3300/0 conn(0x7f90a80b3140 0x7f90a806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2334582271 0 0) 0x7f90a80b41e0 con 0x7f90a80b4b10 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9094003530 con 0x7f90a80b4b10 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4194267413 0 0) 0x7f9094003530 con 0x7f90a80b4b10 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 >> v2:172.21.15.157:3300/0 conn(0x7f90a80b3140 msgr2=0x7f90a806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 --2- 172.21.15.29:0/917409001 >> v2:172.21.15.157:3300/0 conn(0x7f90a80b3140 0x7f90a806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 >> v1:172.21.15.157:6789/0 conn(0x7f90a80b3c30 legacy=0x7f90a80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a806df20 con 0x7f90a80b4b10 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f9098001680 con 0x7f90a80b4b10 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 >> v1:172.21.15.29:6789/0 conn(0x7f90a80b4b10 legacy=0x7f90a80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 --2- 172.21.15.29:0/917409001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90940039c0 0x7f9094003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 --2- 172.21.15.29:0/917409001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094004810 0x7f9094004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 --2- 172.21.15.29:0/917409001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90940051d0 0x7f909400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f90a806dc40 con 0x7f9094004810 2024-04-25T00:41:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f909400c070 con 0x7f90940039c0 2024-04-25T00:41:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f909400c270 con 0x7f90940051d0 2024-04-25T00:41:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90ad4d0640 1 --2- 172.21.15.29:0/917409001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094004810 0x7f9094004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90ae4d2640 1 --2- 172.21.15.29:0/917409001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90940051d0 0x7f909400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90adcd1640 1 --2- 172.21.15.29:0/917409001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90940039c0 0x7f9094003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90ad4d0640 1 -- 172.21.15.29:0/917409001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90940051d0 msgr2=0x7f909400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90ad4d0640 1 --2- 172.21.15.29:0/917409001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90940051d0 0x7f909400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90ad4d0640 1 -- 172.21.15.29:0/917409001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90940039c0 msgr2=0x7f9094003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90ad4d0640 1 --2- 172.21.15.29:0/917409001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90940039c0 0x7f9094003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90ad4d0640 1 -- 172.21.15.29:0/917409001 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9094003530 con 0x7f9094004810 2024-04-25T00:41:37.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90ae4d2640 1 -- 172.21.15.29:0/917409001 reap_dead start 2024-04-25T00:41:37.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90ad4d0640 1 --2- 172.21.15.29:0/917409001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094004810 0x7f9094004c90 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f909800c240 tx=0x7f909801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11aeb42682b01a6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:37.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098007d20 con 0x7f9094004810 2024-04-25T00:41:37.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9098005ae0 con 0x7f9094004810 2024-04-25T00:41:37.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.256+0000 7f90acccf640 1 -- 172.21.15.29:0/917409001 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098007070 con 0x7f9094004810 2024-04-25T00:41:37.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 -- 172.21.15.29:0/917409001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094004810 msgr2=0x7f9094004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:37.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 --2- 172.21.15.29:0/917409001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094004810 0x7f9094004c90 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f909800c240 tx=0x7f909801f3f0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 -- 172.21.15.29:0/917409001 shutdown_connections 2024-04-25T00:41:37.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 --2- 172.21.15.29:0/917409001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094004810 0x7f9094004c90 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 -- 172.21.15.29:0/917409001 >> 172.21.15.29:0/917409001 conn(0x7f90a80b1870 msgr2=0x7f90a80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:37.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 -- 172.21.15.29:0/917409001 shutdown_connections 2024-04-25T00:41:37.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 -- 172.21.15.29:0/917409001 wait complete. 2024-04-25T00:41:37.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 Processor -- start 2024-04-25T00:41:37.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 -- start start 2024-04-25T00:41:37.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094004810 0x7f90a814c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a815ef50 0x7f90a814c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a81381b0 0x7f90a8138610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f90a8145a90 con 0x7f9094004810 2024-04-25T00:41:37.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90a8145910 con 0x7f90a81381b0 2024-04-25T00:41:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f90a806e440 con 0x7f90a815ef50 2024-04-25T00:41:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90adcd1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094004810 0x7f90a814c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90adcd1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094004810 0x7f90a814c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37276/0 (socket says 172.21.15.29:37276) 2024-04-25T00:41:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90adcd1640 1 -- 172.21.15.29:0/3212616865 learned_addr learned my addr 172.21.15.29:0/3212616865 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90ae4d2640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a81381b0 0x7f90a8138610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90ad4d0640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a815ef50 0x7f90a814c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90adcd1640 1 -- 172.21.15.29:0/3212616865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a815ef50 msgr2=0x7f90a814c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90adcd1640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a815ef50 0x7f90a814c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90adcd1640 1 -- 172.21.15.29:0/3212616865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a81381b0 msgr2=0x7f90a8138610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90adcd1640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a81381b0 0x7f90a8138610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90adcd1640 1 -- 172.21.15.29:0/3212616865 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a8142da0 con 0x7f9094004810 2024-04-25T00:41:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90ad4d0640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a815ef50 0x7f90a814c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:37.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90ae4d2640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a81381b0 0x7f90a8138610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:37.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90adcd1640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094004810 0x7f90a814c9b0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f90a000f3b0 tx=0x7f90a000f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:37.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f909e7fc640 1 -- 172.21.15.29:0/3212616865 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0016620 con 0x7f9094004810 2024-04-25T00:41:37.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f909e7fc640 1 -- 172.21.15.29:0/3212616865 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a001b070 con 0x7f9094004810 2024-04-25T00:41:37.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 -- 172.21.15.29:0/3212616865 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a8138b50 con 0x7f9094004810 2024-04-25T00:41:37.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f909e7fc640 1 -- 172.21.15.29:0/3212616865 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0016a20 con 0x7f9094004810 2024-04-25T00:41:37.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.260+0000 7f90aff35640 1 -- 172.21.15.29:0/3212616865 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a815ca70 con 0x7f9094004810 2024-04-25T00:41:37.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.264+0000 7f90aff35640 1 -- 172.21.15.29:0/3212616865 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a8069270 con 0x7f9094004810 2024-04-25T00:41:37.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.264+0000 7f909e7fc640 1 -- 172.21.15.29:0/3212616865 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f90a0016bc0 con 0x7f9094004810 2024-04-25T00:41:37.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.264+0000 7f909e7fc640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f908c068250 0x7f908c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.264+0000 7f909e7fc640 1 -- 172.21.15.29:0/3212616865 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f90a0085330 con 0x7f9094004810 2024-04-25T00:41:37.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.264+0000 7f90ad4d0640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f908c068250 0x7f908c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:37.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.264+0000 7f90ad4d0640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f908c068250 0x7f908c06a710 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f90980af000 tx=0x7f90980aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:37.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.268+0000 7f909e7fc640 1 -- 172.21.15.29:0/3212616865 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f90a0057aa0 con 0x7f9094004810 2024-04-25T00:41:37.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f90aff35640 1 -- 172.21.15.29:0/3212616865 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f90a80b3800 con 0x7f9094004810 2024-04-25T00:41:37.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f909e7fc640 1 -- 172.21.15.29:0/3212616865 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f90a00573a0 con 0x7f9094004810 2024-04-25T00:41:37.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f90aff35640 1 -- 172.21.15.29:0/3212616865 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f908c068250 msgr2=0x7f908c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:37.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f90aff35640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f908c068250 0x7f908c06a710 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f90980af000 tx=0x7f90980aa000 comp rx=0 tx=0).stop 2024-04-25T00:41:37.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f90aff35640 1 -- 172.21.15.29:0/3212616865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094004810 msgr2=0x7f90a814c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:37.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f90aff35640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094004810 0x7f90a814c9b0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f90a000f3b0 tx=0x7f90a000f870 comp rx=0 tx=0).stop 2024-04-25T00:41:37.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f90aff35640 1 -- 172.21.15.29:0/3212616865 shutdown_connections 2024-04-25T00:41:37.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f90aff35640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f908c068250 0x7f908c06a710 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f90aff35640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a81381b0 0x7f90a8138610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f90aff35640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a815ef50 0x7f90a814c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f90aff35640 1 --2- 172.21.15.29:0/3212616865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094004810 0x7f90a814c9b0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f90aff35640 1 -- 172.21.15.29:0/3212616865 >> 172.21.15.29:0/3212616865 conn(0x7f90a80b1870 msgr2=0x7f90a81434e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:37.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f90aff35640 1 -- 172.21.15.29:0/3212616865 shutdown_connections 2024-04-25T00:41:37.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.432+0000 7f90aff35640 1 -- 172.21.15.29:0/3212616865 wait complete. 2024-04-25T00:41:37.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:37.451 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp14870/subvol61070) 2024-04-25T00:41:37.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:37.474 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-04-25T00:41:37.522 INFO:teuthology.orchestra.run.smithi029.stdout:cephfs_data 2024-04-25T00:41:37.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:37.523 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp14870/subvol61070) 2024-04-25T00:41:37.583 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:41:37.583 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp14870/subvol61070/.meta 2024-04-25T00:41:37.643 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol61070 subvol_grp14870 2024-04-25T00:41:37.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f7854640 1 Processor -- start 2024-04-25T00:41:37.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f7854640 1 -- start start 2024-04-25T00:41:37.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f7854640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f53f015c100 0x7f53f00b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f7854640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53f00d23a0 0x7f53f00cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f7854640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f00cc9d0 con 0x7f53f00d06f0 2024-04-25T00:41:37.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f7854640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f53f00ccd00 con 0x7f53f015c100 2024-04-25T00:41:37.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f7854640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f53f00b3050 con 0x7f53f00d23a0 2024-04-25T00:41:37.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f4def640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f53f00d06f0 0x7f53f00d1c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59958/0 (socket says 172.21.15.29:59958) 2024-04-25T00:41:37.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f4def640 1 -- 172.21.15.29:0/1700339408 learned_addr learned my addr 172.21.15.29:0/1700339408 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:37.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f55f0640 1 --2- 172.21.15.29:0/1700339408 >> v2:172.21.15.29:3300/0 conn(0x7f53f015c100 0x7f53f00b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:37.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53e7fff640 1 -- 172.21.15.29:0/1700339408 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3121323108 0 0) 0x7f53f00cc9d0 con 0x7f53f00d06f0 2024-04-25T00:41:37.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53e7fff640 1 -- 172.21.15.29:0/1700339408 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53dc003530 con 0x7f53f00d06f0 2024-04-25T00:41:37.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f5df1640 1 --2- 172.21.15.29:0/1700339408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53f00d23a0 0x7f53f00cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:37.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f55f0640 1 -- 172.21.15.29:0/1700339408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53f00d23a0 msgr2=0x7f53f00cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:37.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f55f0640 1 --2- 172.21.15.29:0/1700339408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53f00d23a0 0x7f53f00cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f55f0640 1 -- 172.21.15.29:0/1700339408 >> v1:172.21.15.29:6789/0 conn(0x7f53f00d06f0 legacy=0x7f53f00d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:37.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f55f0640 1 -- 172.21.15.29:0/1700339408 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f00b3330 con 0x7f53f015c100 2024-04-25T00:41:37.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.808+0000 7f53f5df1640 1 --2- 172.21.15.29:0/1700339408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53f00d23a0 0x7f53f00cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:37.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f55f0640 1 --2- 172.21.15.29:0/1700339408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53f015c100 0x7f53f00b29c0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0097c0 tx=0x7f53ec01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856da6bfae088643 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:37.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53e7fff640 1 -- 172.21.15.29:0/1700339408 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec0b0070 con 0x7f53f015c100 2024-04-25T00:41:37.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53e7fff640 1 -- 172.21.15.29:0/1700339408 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ec0ab600 con 0x7f53f015c100 2024-04-25T00:41:37.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53e7fff640 1 -- 172.21.15.29:0/1700339408 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec0ab9d0 con 0x7f53f015c100 2024-04-25T00:41:37.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 -- 172.21.15.29:0/1700339408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53f015c100 msgr2=0x7f53f00b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:37.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 --2- 172.21.15.29:0/1700339408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53f015c100 0x7f53f00b29c0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0097c0 tx=0x7f53ec01c380 comp rx=0 tx=0).stop 2024-04-25T00:41:37.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 -- 172.21.15.29:0/1700339408 shutdown_connections 2024-04-25T00:41:37.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 --2- 172.21.15.29:0/1700339408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53f00d23a0 0x7f53f00cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 --2- 172.21.15.29:0/1700339408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53f015c100 0x7f53f00b29c0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 -- 172.21.15.29:0/1700339408 >> 172.21.15.29:0/1700339408 conn(0x7f53f009aa30 msgr2=0x7f53f009ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:37.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 -- 172.21.15.29:0/1700339408 shutdown_connections 2024-04-25T00:41:37.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 -- 172.21.15.29:0/1700339408 wait complete. 2024-04-25T00:41:37.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 Processor -- start 2024-04-25T00:41:37.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 -- start start 2024-04-25T00:41:37.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f00d06f0 0x7f53f014e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53f00d23a0 0x7f53f00b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53f015c100 0x7f53f00b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f53f00ce2f0 con 0x7f53f015c100 2024-04-25T00:41:37.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53f00ce170 con 0x7f53f00d06f0 2024-04-25T00:41:37.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.812+0000 7f53f7854640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f53f00ce470 con 0x7f53f00d23a0 2024-04-25T00:41:37.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f5df1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53f015c100 0x7f53f00b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:37.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f5df1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53f015c100 0x7f53f00b6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53540/0 (socket says 172.21.15.29:53540) 2024-04-25T00:41:37.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f5df1640 1 -- 172.21.15.29:0/1003189986 learned_addr learned my addr 172.21.15.29:0/1003189986 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:37.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f5df1640 1 -- 172.21.15.29:0/1003189986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53f00d23a0 msgr2=0x7f53f00b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:37.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f5df1640 1 --2- 172.21.15.29:0/1003189986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53f00d23a0 0x7f53f00b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f5df1640 1 -- 172.21.15.29:0/1003189986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f00d06f0 msgr2=0x7f53f014e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:37.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f5df1640 1 --2- 172.21.15.29:0/1003189986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f00d06f0 0x7f53f014e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:37.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f5df1640 1 -- 172.21.15.29:0/1003189986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f0152560 con 0x7f53f015c100 2024-04-25T00:41:37.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f5df1640 1 --2- 172.21.15.29:0/1003189986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53f015c100 0x7f53f00b6db0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f53e800cf20 tx=0x7f53e800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:37.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53e5ffb640 1 -- 172.21.15.29:0/1003189986 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e801a070 con 0x7f53f015c100 2024-04-25T00:41:37.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53e5ffb640 1 -- 172.21.15.29:0/1003189986 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e800d840 con 0x7f53f015c100 2024-04-25T00:41:37.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f7854640 1 -- 172.21.15.29:0/1003189986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f015a8c0 con 0x7f53f015c100 2024-04-25T00:41:37.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53e5ffb640 1 -- 172.21.15.29:0/1003189986 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e800dc30 con 0x7f53f015c100 2024-04-25T00:41:37.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f7854640 1 -- 172.21.15.29:0/1003189986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f015aba0 con 0x7f53f015c100 2024-04-25T00:41:37.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f7854640 1 -- 172.21.15.29:0/1003189986 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f00692e0 con 0x7f53f015c100 2024-04-25T00:41:37.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53e5ffb640 1 -- 172.21.15.29:0/1003189986 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f53e801d070 con 0x7f53f015c100 2024-04-25T00:41:37.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53e5ffb640 1 --2- 172.21.15.29:0/1003189986 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f53d4068250 0x7f53d406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:37.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53e5ffb640 1 -- 172.21.15.29:0/1003189986 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f53e8084c80 con 0x7f53f015c100 2024-04-25T00:41:37.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f55f0640 1 --2- 172.21.15.29:0/1003189986 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f53d4068250 0x7f53d406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:37.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.816+0000 7f53f55f0640 1 --2- 172.21.15.29:0/1003189986 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f53d4068250 0x7f53d406a710 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0ad000 tx=0x7f53ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:37.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:37.820+0000 7f53e5ffb640 1 -- 172.21.15.29:0/1003189986 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f53e80573c0 con 0x7f53f015c100 2024-04-25T00:41:38.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.000+0000 7f53f7854640 1 -- 172.21.15.29:0/1003189986 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol61070", "group_name": "subvol_grp14870", "target": ["mon-mgr", ""]}) -- 0x7f53f00af810 con 0x7f53d4068250 2024-04-25T00:41:38.009 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:38.008+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:38.009 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:38.008+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:38.009 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:38.008+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:38.009 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:38.008+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:38.009 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:38.008+0000 7f987757d640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:38.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.052+0000 7f53e5ffb640 1 -- 172.21.15.29:0/1003189986 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f53f00af810 con 0x7f53d4068250 2024-04-25T00:41:38.053 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/subvol_grp14870/subvol61070 2024-04-25T00:41:38.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.056+0000 7f53f7854640 1 -- 172.21.15.29:0/1003189986 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f53d4068250 msgr2=0x7f53d406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:38.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.056+0000 7f53f7854640 1 --2- 172.21.15.29:0/1003189986 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f53d4068250 0x7f53d406a710 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0ad000 tx=0x7f53ec008040 comp rx=0 tx=0).stop 2024-04-25T00:41:38.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.056+0000 7f53f7854640 1 -- 172.21.15.29:0/1003189986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53f015c100 msgr2=0x7f53f00b6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:38.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.056+0000 7f53f7854640 1 --2- 172.21.15.29:0/1003189986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53f015c100 0x7f53f00b6db0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f53e800cf20 tx=0x7f53e800a7d0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.056+0000 7f53f7854640 1 -- 172.21.15.29:0/1003189986 shutdown_connections 2024-04-25T00:41:38.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.056+0000 7f53f7854640 1 --2- 172.21.15.29:0/1003189986 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f53d4068250 0x7f53d406a710 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.056+0000 7f53f7854640 1 --2- 172.21.15.29:0/1003189986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53f015c100 0x7f53f00b6db0 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.056+0000 7f53f7854640 1 --2- 172.21.15.29:0/1003189986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53f00d23a0 0x7f53f00b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.056+0000 7f53f7854640 1 --2- 172.21.15.29:0/1003189986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f00d06f0 0x7f53f014e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.056+0000 7f53f7854640 1 -- 172.21.15.29:0/1003189986 >> 172.21.15.29:0/1003189986 conn(0x7f53f009aa30 msgr2=0x7f53f00b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:38.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.056+0000 7f53f7854640 1 -- 172.21.15.29:0/1003189986 shutdown_connections 2024-04-25T00:41:38.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.056+0000 7f53f7854640 1 -- 172.21.15.29:0/1003189986 wait complete. 2024-04-25T00:41:38.071 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol61070 subvol_grp14870 2024-04-25T00:41:38.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.188+0000 7f88a69d9640 1 Processor -- start 2024-04-25T00:41:38.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.188+0000 7f88a69d9640 1 -- start start 2024-04-25T00:41:38.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.188+0000 7f88a69d9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f88a00b8bf0 0x7f88a00d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:38.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.188+0000 7f88a69d9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88a00cfed0 con 0x7f88a015e560 2024-04-25T00:41:38.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.188+0000 7f88a69d9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88a00b7100 con 0x7f88a015e180 2024-04-25T00:41:38.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.188+0000 7f88a69d9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f88a00b73e0 con 0x7f88a00b8bf0 2024-04-25T00:41:38.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.188+0000 7f8897fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f88a015e560 0x7f88a00d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59964/0 (socket says 172.21.15.29:59964) 2024-04-25T00:41:38.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.188+0000 7f8897fff640 1 -- 172.21.15.29:0/903889802 learned_addr learned my addr 172.21.15.29:0/903889802 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:38.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f88a4f76640 1 --2- 172.21.15.29:0/903889802 >> v2:172.21.15.157:3300/0 conn(0x7f88a00b8bf0 0x7f88a00d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:38.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1122677257 0 0) 0x7f88a00cfed0 con 0x7f88a015e560 2024-04-25T00:41:38.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f888c003530 con 0x7f88a015e560 2024-04-25T00:41:38.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4249267002 0 0) 0x7f888c003530 con 0x7f88a015e560 2024-04-25T00:41:38.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 >> v2:172.21.15.157:3300/0 conn(0x7f88a00b8bf0 msgr2=0x7f88a00d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:38.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 --2- 172.21.15.29:0/903889802 >> v2:172.21.15.157:3300/0 conn(0x7f88a00b8bf0 0x7f88a00d20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 >> v1:172.21.15.157:6789/0 conn(0x7f88a015e180 legacy=0x7f88a00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:38.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a00b76c0 con 0x7f88a015e560 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8888001680 con 0x7f88a015e560 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 >> v1:172.21.15.29:6789/0 conn(0x7f88a015e560 legacy=0x7f88a00d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 --2- 172.21.15.29:0/903889802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f888c0039c0 0x7f888c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 --2- 172.21.15.29:0/903889802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f888c004810 0x7f888c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 --2- 172.21.15.29:0/903889802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f888c0051d0 0x7f888c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f88a00b73e0 con 0x7f888c0051d0 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f888c00c080 con 0x7f888c0039c0 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f888c00c280 con 0x7f888c004810 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f88a4f76640 1 --2- 172.21.15.29:0/903889802 >> v2:172.21.15.157:3300/0 conn(0x7f88a00b8bf0 0x7f88a00d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f88a4f76640 1 --2- 172.21.15.29:0/903889802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f888c0051d0 0x7f888c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889ffff640 1 --2- 172.21.15.29:0/903889802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f888c0039c0 0x7f888c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f8897fff640 1 --2- 172.21.15.29:0/903889802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f888c004810 0x7f888c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f88a4f76640 1 -- 172.21.15.29:0/903889802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f888c004810 msgr2=0x7f888c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:38.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f88a4f76640 1 --2- 172.21.15.29:0/903889802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f888c004810 0x7f888c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f88a4f76640 1 -- 172.21.15.29:0/903889802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f888c0039c0 msgr2=0x7f888c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:38.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f88a4f76640 1 --2- 172.21.15.29:0/903889802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f888c0039c0 0x7f888c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f88a4f76640 1 -- 172.21.15.29:0/903889802 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888c003530 con 0x7f888c0051d0 2024-04-25T00:41:38.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f88a4f76640 1 -- 172.21.15.29:0/903889802 reap_dead start 2024-04-25T00:41:38.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f88a4f76640 1 --2- 172.21.15.29:0/903889802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f888c0051d0 0x7f888c00b9d0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f889800e350 tx=0x7f889801ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30519963b914f61f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:38.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898009dc0 con 0x7f888c0051d0 2024-04-25T00:41:38.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88980039a0 con 0x7f888c0051d0 2024-04-25T00:41:38.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.192+0000 7f889f7fe640 1 -- 172.21.15.29:0/903889802 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898003d90 con 0x7f888c0051d0 2024-04-25T00:41:38.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 -- 172.21.15.29:0/903889802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f888c0051d0 msgr2=0x7f888c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:38.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 --2- 172.21.15.29:0/903889802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f888c0051d0 0x7f888c00b9d0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f889800e350 tx=0x7f889801ce60 comp rx=0 tx=0).stop 2024-04-25T00:41:38.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 -- 172.21.15.29:0/903889802 shutdown_connections 2024-04-25T00:41:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 --2- 172.21.15.29:0/903889802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f888c0051d0 0x7f888c00b9d0 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 -- 172.21.15.29:0/903889802 >> 172.21.15.29:0/903889802 conn(0x7f88a00d40a0 msgr2=0x7f88a00d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 -- 172.21.15.29:0/903889802 shutdown_connections 2024-04-25T00:41:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 -- 172.21.15.29:0/903889802 wait complete. 2024-04-25T00:41:38.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 Processor -- start 2024-04-25T00:41:38.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 -- start start 2024-04-25T00:41:38.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f888c0051d0 0x7f88a006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:38.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88a00cdc80 0x7f88a00cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:38.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88a00ccfb0 0x7f88a01510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:38.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f88a00d2470 con 0x7f88a00ccfb0 2024-04-25T00:41:38.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f88a00d22f0 con 0x7f888c0051d0 2024-04-25T00:41:38.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f88a00cf680 con 0x7f88a00cdc80 2024-04-25T00:41:38.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a4f76640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88a00ccfb0 0x7f88a01510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:38.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a4f76640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88a00ccfb0 0x7f88a01510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53590/0 (socket says 172.21.15.29:53590) 2024-04-25T00:41:38.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a4f76640 1 -- 172.21.15.29:0/1197875293 learned_addr learned my addr 172.21.15.29:0/1197875293 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a4f76640 1 -- 172.21.15.29:0/1197875293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88a00cdc80 msgr2=0x7f88a00cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a4f76640 1 --2- 172.21.15.29:0/1197875293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88a00cdc80 0x7f88a00cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a4f76640 1 -- 172.21.15.29:0/1197875293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f888c0051d0 msgr2=0x7f88a006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a4f76640 1 --2- 172.21.15.29:0/1197875293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f888c0051d0 0x7f88a006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a4f76640 1 -- 172.21.15.29:0/1197875293 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a00cb210 con 0x7f88a00ccfb0 2024-04-25T00:41:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a4f76640 1 --2- 172.21.15.29:0/1197875293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88a00ccfb0 0x7f88a01510e0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f889801cea0 tx=0x7f8898009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f889d7fa640 1 -- 172.21.15.29:0/1197875293 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898009dc0 con 0x7f88a00ccfb0 2024-04-25T00:41:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f889d7fa640 1 -- 172.21.15.29:0/1197875293 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88980047e0 con 0x7f88a00ccfb0 2024-04-25T00:41:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 -- 172.21.15.29:0/1197875293 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a00cb480 con 0x7f88a00ccfb0 2024-04-25T00:41:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f889d7fa640 1 -- 172.21.15.29:0/1197875293 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88980b6660 con 0x7f88a00ccfb0 2024-04-25T00:41:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.196+0000 7f88a69d9640 1 -- 172.21.15.29:0/1197875293 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a00cba40 con 0x7f88a00ccfb0 2024-04-25T00:41:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.200+0000 7f88a69d9640 1 -- 172.21.15.29:0/1197875293 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a00692e0 con 0x7f88a00ccfb0 2024-04-25T00:41:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.200+0000 7f889d7fa640 1 -- 172.21.15.29:0/1197875293 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f88980bf060 con 0x7f88a00ccfb0 2024-04-25T00:41:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.200+0000 7f889d7fa640 1 --2- 172.21.15.29:0/1197875293 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8880068280 0x7f888006a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.200+0000 7f889d7fa640 1 -- 172.21.15.29:0/1197875293 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f889811edd0 con 0x7f88a00ccfb0 2024-04-25T00:41:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.200+0000 7f889ffff640 1 --2- 172.21.15.29:0/1197875293 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8880068280 0x7f888006a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:38.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.200+0000 7f889ffff640 1 --2- 172.21.15.29:0/1197875293 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8880068280 0x7f888006a740 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8890001500 tx=0x7f8890009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:38.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.204+0000 7f889d7fa640 1 -- 172.21.15.29:0/1197875293 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8898016b70 con 0x7f88a00ccfb0 2024-04-25T00:41:38.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.392+0000 7f88a69d9640 1 -- 172.21.15.29:0/1197875293 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol61070", "group_name": "subvol_grp14870", "target": ["mon-mgr", ""]}) -- 0x7f88a00d4d10 con 0x7f8880068280 2024-04-25T00:41:38.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.404+0000 7f889d7fa640 1 -- 172.21.15.29:0/1197875293 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f88a00d4d10 con 0x7f8880068280 2024-04-25T00:41:38.404 INFO:teuthology.orchestra.run.smithi029.stdout:/volumes/subvol_grp14870/subvol61070 2024-04-25T00:41:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.404+0000 7f88a69d9640 1 -- 172.21.15.29:0/1197875293 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8880068280 msgr2=0x7f888006a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.404+0000 7f88a69d9640 1 --2- 172.21.15.29:0/1197875293 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8880068280 0x7f888006a740 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8890001500 tx=0x7f8890009040 comp rx=0 tx=0).stop 2024-04-25T00:41:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.404+0000 7f88a69d9640 1 -- 172.21.15.29:0/1197875293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88a00ccfb0 msgr2=0x7f88a01510e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.404+0000 7f88a69d9640 1 --2- 172.21.15.29:0/1197875293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88a00ccfb0 0x7f88a01510e0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f889801cea0 tx=0x7f8898009750 comp rx=0 tx=0).stop 2024-04-25T00:41:38.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.404+0000 7f88a69d9640 1 -- 172.21.15.29:0/1197875293 shutdown_connections 2024-04-25T00:41:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.404+0000 7f88a69d9640 1 --2- 172.21.15.29:0/1197875293 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8880068280 0x7f888006a740 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.404+0000 7f88a69d9640 1 --2- 172.21.15.29:0/1197875293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88a00ccfb0 0x7f88a01510e0 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.404+0000 7f88a69d9640 1 --2- 172.21.15.29:0/1197875293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88a00cdc80 0x7f88a00cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.404+0000 7f88a69d9640 1 --2- 172.21.15.29:0/1197875293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f888c0051d0 0x7f88a006cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.404+0000 7f88a69d9640 1 -- 172.21.15.29:0/1197875293 >> 172.21.15.29:0/1197875293 conn(0x7f88a00d40a0 msgr2=0x7f88a0153320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.404+0000 7f88a69d9640 1 -- 172.21.15.29:0/1197875293 shutdown_connections 2024-04-25T00:41:38.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:38.404+0000 7f88a69d9640 1 -- 172.21.15.29:0/1197875293 wait complete. 2024-04-25T00:41:38.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:38.424 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/04aff59c5a7fb794c0ee5bb905860aee.meta) 2024-04-25T00:41:40.641 INFO:teuthology.orchestra.run.smithi029.stdout:[GLOBAL] 2024-04-25T00:41:40.641 INFO:teuthology.orchestra.run.smithi029.stdout:version = 1 2024-04-25T00:41:40.641 INFO:teuthology.orchestra.run.smithi029.stdout:type = subvolume 2024-04-25T00:41:40.641 INFO:teuthology.orchestra.run.smithi029.stdout:path = /volumes/subvol_grp14870/subvol61070 2024-04-25T00:41:40.641 INFO:teuthology.orchestra.run.smithi029.stdout:state = complete 2024-04-25T00:41:40.641 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:40.643 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol61070 subvol_grp14870 2024-04-25T00:41:40.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee7b3c640 1 Processor -- start 2024-04-25T00:41:40.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee7b3c640 1 -- start start 2024-04-25T00:41:40.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee7b3c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9ee015e180 0x7f9ee00d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee7b3c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9ee00d2680 0x7f9ee00d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee7b3c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ee00cfed0 con 0x7f9ee015e560 2024-04-25T00:41:40.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee7b3c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9ee00b7100 con 0x7f9ee015e180 2024-04-25T00:41:40.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee7b3c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9ee00b73e0 con 0x7f9ee00d2680 2024-04-25T00:41:40.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee58d8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9ee015e180 0x7f9ee00d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee50d7640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9ee015e560 0x7f9ee00cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59980/0 (socket says 172.21.15.29:59980) 2024-04-25T00:41:40.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee50d7640 1 -- 172.21.15.29:0/2553874688 learned_addr learned my addr 172.21.15.29:0/2553874688 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:40.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee60d9640 1 --2- 172.21.15.29:0/2553874688 >> v2:172.21.15.157:3300/0 conn(0x7f9ee00d2680 0x7f9ee00d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee58d8640 1 -- 172.21.15.29:0/2553874688 >> v2:172.21.15.157:3300/0 conn(0x7f9ee00d2680 msgr2=0x7f9ee00d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:40.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee58d8640 1 --2- 172.21.15.29:0/2553874688 >> v2:172.21.15.157:3300/0 conn(0x7f9ee00d2680 0x7f9ee00d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee58d8640 1 -- 172.21.15.29:0/2553874688 >> v1:172.21.15.29:6789/0 conn(0x7f9ee015e560 legacy=0x7f9ee00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:40.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee58d8640 1 -- 172.21.15.29:0/2553874688 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee00b76c0 con 0x7f9ee015e180 2024-04-25T00:41:40.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee48d6640 1 -- 172.21.15.29:0/2553874688 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 588008165 0 0) 0x7f9ee00cfed0 con 0x7f9ee015e560 2024-04-25T00:41:40.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.784+0000 7f9ee60d9640 1 --2- 172.21.15.29:0/2553874688 >> v2:172.21.15.157:3300/0 conn(0x7f9ee00d2680 0x7f9ee00d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee58d8640 1 --2- 172.21.15.29:0/2553874688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ee015e180 0x7f9ee00d20e0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00ceac0 tx=0x7f9ed401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af71deabfc0516e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee48d6640 1 -- 172.21.15.29:0/2553874688 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40b5070 con 0x7f9ee015e180 2024-04-25T00:41:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee48d6640 1 -- 172.21.15.29:0/2553874688 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed40a8970 con 0x7f9ee015e180 2024-04-25T00:41:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee48d6640 1 -- 172.21.15.29:0/2553874688 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40a8d40 con 0x7f9ee015e180 2024-04-25T00:41:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/2553874688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ee015e180 msgr2=0x7f9ee00d20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 --2- 172.21.15.29:0/2553874688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ee015e180 0x7f9ee00d20e0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00ceac0 tx=0x7f9ed401ec70 comp rx=0 tx=0).stop 2024-04-25T00:41:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/2553874688 shutdown_connections 2024-04-25T00:41:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 --2- 172.21.15.29:0/2553874688 >> v2:172.21.15.157:3300/0 conn(0x7f9ee00d2680 0x7f9ee00d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 --2- 172.21.15.29:0/2553874688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ee015e180 0x7f9ee00d20e0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/2553874688 >> 172.21.15.29:0/2553874688 conn(0x7f9ee00d40a0 msgr2=0x7f9ee00d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/2553874688 shutdown_connections 2024-04-25T00:41:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/2553874688 wait complete. 2024-04-25T00:41:40.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 Processor -- start 2024-04-25T00:41:40.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 -- start start 2024-04-25T00:41:40.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ee00d2680 0x7f9ee01603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee015e180 0x7f9ee015a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee015e560 0x7f9ee01f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee00b7d60 con 0x7f9ee00d2680 2024-04-25T00:41:40.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee00b7be0 con 0x7f9ee015e180 2024-04-25T00:41:40.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee7b3c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee00b7ee0 con 0x7f9ee015e560 2024-04-25T00:41:40.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.788+0000 7f9ee58d8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ee00d2680 0x7f9ee01603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee58d8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ee00d2680 0x7f9ee01603c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53598/0 (socket says 172.21.15.29:53598) 2024-04-25T00:41:40.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee58d8640 1 -- 172.21.15.29:0/3064754201 learned_addr learned my addr 172.21.15.29:0/3064754201 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:40.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee50d7640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee015e180 0x7f9ee015a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee60d9640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee015e560 0x7f9ee01f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee58d8640 1 -- 172.21.15.29:0/3064754201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee015e560 msgr2=0x7f9ee01f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:40.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee58d8640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee015e560 0x7f9ee01f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee58d8640 1 -- 172.21.15.29:0/3064754201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee015e180 msgr2=0x7f9ee015a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:40.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee58d8640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee015e180 0x7f9ee015a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:40.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee58d8640 1 -- 172.21.15.29:0/3064754201 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee00c7e00 con 0x7f9ee00d2680 2024-04-25T00:41:40.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee50d7640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee015e180 0x7f9ee015a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:40.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee60d9640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee015e560 0x7f9ee01f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:40.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee58d8640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ee00d2680 0x7f9ee01603c0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40b3040 tx=0x7f9ed401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:40.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ed27fc640 1 -- 172.21.15.29:0/3064754201 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40bf070 con 0x7f9ee00d2680 2024-04-25T00:41:40.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ed27fc640 1 -- 172.21.15.29:0/3064754201 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed40b07c0 con 0x7f9ee00d2680 2024-04-25T00:41:40.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ed27fc640 1 -- 172.21.15.29:0/3064754201 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40b5040 con 0x7f9ee00d2680 2024-04-25T00:41:40.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/3064754201 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee00c80e0 con 0x7f9ee00d2680 2024-04-25T00:41:40.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/3064754201 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee00c86a0 con 0x7f9ee00d2680 2024-04-25T00:41:40.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/3064754201 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee00692e0 con 0x7f9ee00d2680 2024-04-25T00:41:40.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ed27fc640 1 -- 172.21.15.29:0/3064754201 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f9ed40b0bb0 con 0x7f9ee00d2680 2024-04-25T00:41:40.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ed27fc640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ec4068250 0x7f9ec406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:40.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee50d7640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ec4068250 0x7f9ec406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:40.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ed27fc640 1 -- 172.21.15.29:0/3064754201 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f9ed4120e60 con 0x7f9ee00d2680 2024-04-25T00:41:40.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.792+0000 7f9ee50d7640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ec4068250 0x7f9ec406a710 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00ca080 tx=0x7f9edc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:40.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.796+0000 7f9ed27fc640 1 -- 172.21.15.29:0/3064754201 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9ed40187d0 con 0x7f9ee00d2680 2024-04-25T00:41:40.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.808+0000 7f9ed27fc640 1 -- 172.21.15.29:0/3064754201 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 26) ==== 88363+0+0 (secure 0 0 0) 0x7f9ed40ef510 con 0x7f9ee00d2680 2024-04-25T00:41:40.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:40.984+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/3064754201 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol61070", "group_name": "subvol_grp14870", "target": ["mon-mgr", ""]}) -- 0x7f9ee00b5650 con 0x7f9ec4068250 2024-04-25T00:41:41.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.012+0000 7f9ed27fc640 1 -- 172.21.15.29:0/3064754201 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ee00b5650 con 0x7f9ec4068250 2024-04-25T00:41:41.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.012+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/3064754201 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ec4068250 msgr2=0x7f9ec406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.012+0000 7f9ee7b3c640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ec4068250 0x7f9ec406a710 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00ca080 tx=0x7f9edc009040 comp rx=0 tx=0).stop 2024-04-25T00:41:41.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.012+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/3064754201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ee00d2680 msgr2=0x7f9ee01603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.012+0000 7f9ee7b3c640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ee00d2680 0x7f9ee01603c0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40b3040 tx=0x7f9ed401ba10 comp rx=0 tx=0).stop 2024-04-25T00:41:41.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.012+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/3064754201 shutdown_connections 2024-04-25T00:41:41.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.012+0000 7f9ee7b3c640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f9ec4068250 0x7f9ec406a710 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.012+0000 7f9ee7b3c640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee015e560 0x7f9ee01f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.012+0000 7f9ee7b3c640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee015e180 0x7f9ee015a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.012+0000 7f9ee7b3c640 1 --2- 172.21.15.29:0/3064754201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ee00d2680 0x7f9ee01603c0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.012+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/3064754201 >> 172.21.15.29:0/3064754201 conn(0x7f9ee00d40a0 msgr2=0x7f9ee006c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:41.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.012+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/3064754201 shutdown_connections 2024-04-25T00:41:41.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.012+0000 7f9ee7b3c640 1 -- 172.21.15.29:0/3064754201 wait complete. 2024-04-25T00:41:41.019 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:41.020+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:41.019 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:41.020+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:41.019 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:41.020+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:41.019 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:41.020+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:41.019 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:41.020+0000 7f9879d82640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:41.035 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-04-25T00:41:41.038 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:41.036+0000 7f9878d80640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:41.038 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:41.036+0000 7f9878d80640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:41.039 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:41.036+0000 7f9878d80640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:41.039 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:41.036+0000 7f9878d80640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:41.039 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-04-25T00:41:41.036+0000 7f9878d80640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:41:41.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:41:41.080 INFO:teuthology.orchestra.run.smithi029.stdout:2 2024-04-25T00:41:41.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:41:41.086 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp14870 2024-04-25T00:41:41.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.248+0000 7f8bb02d9640 1 Processor -- start 2024-04-25T00:41:41.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bb02d9640 1 -- start start 2024-04-25T00:41:41.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bb02d9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba806e440 0x7f8ba80b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bb02d9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba80b41e0 con 0x7f8ba80b3140 2024-04-25T00:41:41.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bb02d9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba806d960 con 0x7f8ba80b3c30 2024-04-25T00:41:41.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bb02d9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba80b5050 con 0x7f8ba806e440 2024-04-25T00:41:41.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bae075640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8ba80b3140 0x7f8ba80b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59988/0 (socket says 172.21.15.29:59988) 2024-04-25T00:41:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bae075640 1 -- 172.21.15.29:0/1602986137 learned_addr learned my addr 172.21.15.29:0/1602986137 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bae876640 1 --2- 172.21.15.29:0/1602986137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba806e440 0x7f8ba80b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 651433338 0 0) 0x7f8ba80b41e0 con 0x7f8ba80b3140 2024-04-25T00:41:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b94003530 con 0x7f8ba80b3140 2024-04-25T00:41:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4014324305 0 0) 0x7f8b94003530 con 0x7f8ba80b3140 2024-04-25T00:41:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba806e440 msgr2=0x7f8ba80b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 --2- 172.21.15.29:0/1602986137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba806e440 0x7f8ba80b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 >> v1:172.21.15.157:6789/0 conn(0x7f8ba80b3c30 legacy=0x7f8ba80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba806dc40 con 0x7f8ba80b3140 2024-04-25T00:41:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f8ba4001680 con 0x7f8ba80b3140 2024-04-25T00:41:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 >> v1:172.21.15.29:6789/0 conn(0x7f8ba80b3140 legacy=0x7f8ba80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 --2- 172.21.15.29:0/1602986137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b940039f0 0x7f8b94003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 --2- 172.21.15.29:0/1602986137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b94004810 0x7f8b94004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 --2- 172.21.15.29:0/1602986137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba806e440 0x7f8b940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba80b5050 con 0x7f8b94004810 2024-04-25T00:41:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9400ba80 con 0x7f8b940039f0 2024-04-25T00:41:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b9400bc80 con 0x7f8ba806e440 2024-04-25T00:41:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad874640 1 --2- 172.21.15.29:0/1602986137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b94004810 0x7f8b94004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bae075640 1 --2- 172.21.15.29:0/1602986137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b940039f0 0x7f8b94003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bae876640 1 --2- 172.21.15.29:0/1602986137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba806e440 0x7f8b940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad874640 1 -- 172.21.15.29:0/1602986137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba806e440 msgr2=0x7f8b940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad874640 1 --2- 172.21.15.29:0/1602986137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba806e440 0x7f8b940051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad874640 1 -- 172.21.15.29:0/1602986137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b940039f0 msgr2=0x7f8b94003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad874640 1 --2- 172.21.15.29:0/1602986137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b940039f0 0x7f8b94003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bad874640 1 -- 172.21.15.29:0/1602986137 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b94003530 con 0x7f8b94004810 2024-04-25T00:41:41.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.252+0000 7f8bae075640 1 --2- 172.21.15.29:0/1602986137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b940039f0 0x7f8b94003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:41.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.256+0000 7f8bad874640 1 --2- 172.21.15.29:0/1602986137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b94004810 0x7f8b94004c90 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f8b98001750 tx=0x7f8b9801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a89800d08470e39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:41.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.256+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b980013a0 con 0x7f8b94004810 2024-04-25T00:41:41.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.256+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b980052f0 con 0x7f8b94004810 2024-04-25T00:41:41.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.256+0000 7f8bad073640 1 -- 172.21.15.29:0/1602986137 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98003840 con 0x7f8b94004810 2024-04-25T00:41:41.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.256+0000 7f8bb02d9640 1 -- 172.21.15.29:0/1602986137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b94004810 msgr2=0x7f8b94004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.256+0000 7f8bb02d9640 1 --2- 172.21.15.29:0/1602986137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b94004810 0x7f8b94004c90 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f8b98001750 tx=0x7f8b9801c3c0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.256+0000 7f8bae876640 1 -- 172.21.15.29:0/1602986137 reap_dead start 2024-04-25T00:41:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.256+0000 7f8bb02d9640 1 -- 172.21.15.29:0/1602986137 shutdown_connections 2024-04-25T00:41:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.256+0000 7f8bb02d9640 1 -- 172.21.15.29:0/1602986137 >> 172.21.15.29:0/1602986137 conn(0x7f8ba80b1870 msgr2=0x7f8ba80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.256+0000 7f8bb02d9640 1 -- 172.21.15.29:0/1602986137 shutdown_connections 2024-04-25T00:41:41.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.256+0000 7f8bb02d9640 1 -- 172.21.15.29:0/1602986137 wait complete. 2024-04-25T00:41:41.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.256+0000 7f8bb02d9640 1 Processor -- start 2024-04-25T00:41:41.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.256+0000 7f8bb02d9640 1 -- start start 2024-04-25T00:41:41.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bb02d9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b94004810 0x7f8ba81394c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bb02d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ba8141be0 0x7f8ba8142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bb02d9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba8145c10 0x7f8ba8146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bb02d9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8145a90 con 0x7f8b94004810 2024-04-25T00:41:41.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bb02d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8145910 con 0x7f8ba8141be0 2024-04-25T00:41:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bb02d9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba80b4b10 con 0x7f8ba8145c10 2024-04-25T00:41:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bae075640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b94004810 0x7f8ba81394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bae075640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b94004810 0x7f8ba81394c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53656/0 (socket says 172.21.15.29:53656) 2024-04-25T00:41:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bae075640 1 -- 172.21.15.29:0/548125246 learned_addr learned my addr 172.21.15.29:0/548125246 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bae876640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba8145c10 0x7f8ba8146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bae075640 1 -- 172.21.15.29:0/548125246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba8145c10 msgr2=0x7f8ba8146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bad874640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ba8141be0 0x7f8ba8142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bae075640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba8145c10 0x7f8ba8146070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bae075640 1 -- 172.21.15.29:0/548125246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ba8141be0 msgr2=0x7f8ba8142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bae075640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ba8141be0 0x7f8ba8142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bae075640 1 -- 172.21.15.29:0/548125246 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba8140b40 con 0x7f8b94004810 2024-04-25T00:41:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bae075640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b94004810 0x7f8ba81394c0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4010b90 tx=0x7f8ba400e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8b9effd640 1 -- 172.21.15.29:0/548125246 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba401b070 con 0x7f8b94004810 2024-04-25T00:41:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8b9effd640 1 -- 172.21.15.29:0/548125246 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba4011040 con 0x7f8b94004810 2024-04-25T00:41:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8b9effd640 1 -- 172.21.15.29:0/548125246 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba40143a0 con 0x7f8b94004810 2024-04-25T00:41:41.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bb02d9640 1 -- 172.21.15.29:0/548125246 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba8140e20 con 0x7f8b94004810 2024-04-25T00:41:41.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bb02d9640 1 -- 172.21.15.29:0/548125246 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba81413e0 con 0x7f8b94004810 2024-04-25T00:41:41.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8bb02d9640 1 -- 172.21.15.29:0/548125246 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba8069270 con 0x7f8b94004810 2024-04-25T00:41:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8b9effd640 1 -- 172.21.15.29:0/548125246 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 26) ==== 88363+0+0 (secure 0 0 0) 0x7f8ba4014540 con 0x7f8b94004810 2024-04-25T00:41:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.260+0000 7f8b9effd640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b8c0682e0 0x7f8b8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.264+0000 7f8bad874640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b8c0682e0 0x7f8b8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.264+0000 7f8b9effd640 1 -- 172.21.15.29:0/548125246 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f8ba4087020 con 0x7f8b94004810 2024-04-25T00:41:41.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.264+0000 7f8bad874640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b8c0682e0 0x7f8b8c06a7a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8b940057f0 tx=0x7f8b980011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:41.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.268+0000 7f8b9effd640 1 -- 172.21.15.29:0/548125246 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8ba4059610 con 0x7f8b94004810 2024-04-25T00:41:41.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.444+0000 7f8bb02d9640 1 -- 172.21.15.29:0/548125246 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp14870", "target": ["mon-mgr", ""]}) -- 0x7f8ba80753e0 con 0x7f8b8c0682e0 2024-04-25T00:41:41.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.452+0000 7f8b9effd640 1 -- 172.21.15.29:0/548125246 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ba80753e0 con 0x7f8b8c0682e0 2024-04-25T00:41:41.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.452+0000 7f8bb02d9640 1 -- 172.21.15.29:0/548125246 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b8c0682e0 msgr2=0x7f8b8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.452+0000 7f8bb02d9640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b8c0682e0 0x7f8b8c06a7a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8b940057f0 tx=0x7f8b980011a0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.452+0000 7f8bb02d9640 1 -- 172.21.15.29:0/548125246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b94004810 msgr2=0x7f8ba81394c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.452+0000 7f8bb02d9640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b94004810 0x7f8ba81394c0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4010b90 tx=0x7f8ba400e4d0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.452+0000 7f8bb02d9640 1 -- 172.21.15.29:0/548125246 shutdown_connections 2024-04-25T00:41:41.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.452+0000 7f8bb02d9640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b8c0682e0 0x7f8b8c06a7a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.452+0000 7f8bb02d9640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ba8145c10 0x7f8ba8146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.452+0000 7f8bb02d9640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ba8141be0 0x7f8ba8142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.452+0000 7f8bb02d9640 1 --2- 172.21.15.29:0/548125246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b94004810 0x7f8ba81394c0 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.452+0000 7f8bb02d9640 1 -- 172.21.15.29:0/548125246 >> 172.21.15.29:0/548125246 conn(0x7f8ba80b1870 msgr2=0x7f8ba813d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:41.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.456+0000 7f8bb02d9640 1 -- 172.21.15.29:0/548125246 shutdown_connections 2024-04-25T00:41:41.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.456+0000 7f8bb02d9640 1 -- 172.21.15.29:0/548125246 wait complete. 2024-04-25T00:41:41.470 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:41:41.470 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:41:41.517 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:41:41.555 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:41:41.555 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:41:41.556 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:41:41.556 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:41:41.561 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:41.603 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:41:41.605 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:41:41.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f870086d640 1 Processor -- start 2024-04-25T00:41:41.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f870086d640 1 -- start start 2024-04-25T00:41:41.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f870086d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f86fc0b8790 0x7f86fc1692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f870086d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86fc0d2cd0 con 0x7f86fc15e4e0 2024-04-25T00:41:41.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f870086d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86fc0cfe30 con 0x7f86fc0d4bf0 2024-04-25T00:41:41.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f870086d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f86fc0b6f80 con 0x7f86fc0b8790 2024-04-25T00:41:41.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86fa59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f86fc15e4e0 0x7f86fc0d1eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59990/0 (socket says 172.21.15.29:59990) 2024-04-25T00:41:41.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86fa59c640 1 -- 172.21.15.29:0/1728207322 learned_addr learned my addr 172.21.15.29:0/1728207322 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:41.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86fad9d640 1 --2- 172.21.15.29:0/1728207322 >> v2:172.21.15.157:3300/0 conn(0x7f86fc0b8790 0x7f86fc1692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3983127297 0 0) 0x7f86fc0d2cd0 con 0x7f86fc15e4e0 2024-04-25T00:41:41.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86e8003530 con 0x7f86fc15e4e0 2024-04-25T00:41:41.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4290637283 0 0) 0x7f86e8003530 con 0x7f86fc15e4e0 2024-04-25T00:41:41.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 >> v2:172.21.15.157:3300/0 conn(0x7f86fc0b8790 msgr2=0x7f86fc1692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:41.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 --2- 172.21.15.29:0/1728207322 >> v2:172.21.15.157:3300/0 conn(0x7f86fc0b8790 0x7f86fc1692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 >> v1:172.21.15.157:6789/0 conn(0x7f86fc0d4bf0 legacy=0x7f86fc0d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc0b7260 con 0x7f86fc15e4e0 2024-04-25T00:41:41.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f86ec001680 con 0x7f86fc15e4e0 2024-04-25T00:41:41.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 >> v1:172.21.15.29:6789/0 conn(0x7f86fc15e4e0 legacy=0x7f86fc0d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 --2- 172.21.15.29:0/1728207322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e80039c0 0x7f86e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 --2- 172.21.15.29:0/1728207322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86e8004810 0x7f86e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 --2- 172.21.15.29:0/1728207322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e80051d0 0x7f86e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc0b6f80 con 0x7f86e8004810 2024-04-25T00:41:41.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86e800c080 con 0x7f86e80051d0 2024-04-25T00:41:41.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f86e800c280 con 0x7f86e80039c0 2024-04-25T00:41:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f3fff640 1 --2- 172.21.15.29:0/1728207322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86e8004810 0x7f86e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86fad9d640 1 --2- 172.21.15.29:0/1728207322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e80051d0 0x7f86e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86fa59c640 1 --2- 172.21.15.29:0/1728207322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e80039c0 0x7f86e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f3fff640 1 -- 172.21.15.29:0/1728207322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e80039c0 msgr2=0x7f86e8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f3fff640 1 --2- 172.21.15.29:0/1728207322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e80039c0 0x7f86e8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f3fff640 1 -- 172.21.15.29:0/1728207322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e80051d0 msgr2=0x7f86e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f3fff640 1 --2- 172.21.15.29:0/1728207322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e80051d0 0x7f86e800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86f3fff640 1 -- 172.21.15.29:0/1728207322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e8003530 con 0x7f86e8004810 2024-04-25T00:41:41.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.728+0000 7f86fad9d640 1 -- 172.21.15.29:0/1728207322 reap_dead start 2024-04-25T00:41:41.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.732+0000 7f86f3fff640 1 --2- 172.21.15.29:0/1728207322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86e8004810 0x7f86e8004c90 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f86e4001780 tx=0x7f86e401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bebc03ed5878628 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:41.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.732+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e40013a0 con 0x7f86e8004810 2024-04-25T00:41:41.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.732+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e40052f0 con 0x7f86e8004810 2024-04-25T00:41:41.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.732+0000 7f86f9d9b640 1 -- 172.21.15.29:0/1728207322 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4003840 con 0x7f86e8004810 2024-04-25T00:41:41.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.732+0000 7f870086d640 1 -- 172.21.15.29:0/1728207322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86e8004810 msgr2=0x7f86e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.732+0000 7f870086d640 1 --2- 172.21.15.29:0/1728207322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86e8004810 0x7f86e8004c90 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f86e4001780 tx=0x7f86e401c400 comp rx=0 tx=0).stop 2024-04-25T00:41:41.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.732+0000 7f870086d640 1 -- 172.21.15.29:0/1728207322 shutdown_connections 2024-04-25T00:41:41.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.732+0000 7f870086d640 1 --2- 172.21.15.29:0/1728207322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86e8004810 0x7f86e8004c90 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.732+0000 7f870086d640 1 -- 172.21.15.29:0/1728207322 >> 172.21.15.29:0/1728207322 conn(0x7f86fc0d3fb0 msgr2=0x7f86fc0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:41.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.732+0000 7f870086d640 1 -- 172.21.15.29:0/1728207322 shutdown_connections 2024-04-25T00:41:41.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.732+0000 7f870086d640 1 -- 172.21.15.29:0/1728207322 wait complete. 2024-04-25T00:41:41.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f870086d640 1 Processor -- start 2024-04-25T00:41:41.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f870086d640 1 -- start start 2024-04-25T00:41:41.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f870086d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e8004810 0x7f86fc14b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f870086d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86fc0cd7f0 0x7f86fc0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f870086d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86fc0cbfa0 0x7f86fc14bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f870086d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc15ea40 con 0x7f86fc0cd7f0 2024-04-25T00:41:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f870086d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc15e8c0 con 0x7f86fc0cbfa0 2024-04-25T00:41:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f870086d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f86fc15ebc0 con 0x7f86e8004810 2024-04-25T00:41:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86fc0cd7f0 0x7f86fc0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86fc0cd7f0 0x7f86fc0cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53666/0 (socket says 172.21.15.29:53666) 2024-04-25T00:41:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f3fff640 1 -- 172.21.15.29:0/2202342628 learned_addr learned my addr 172.21.15.29:0/2202342628 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86fa59c640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e8004810 0x7f86fc14b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86fad9d640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86fc0cbfa0 0x7f86fc14bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f3fff640 1 -- 172.21.15.29:0/2202342628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e8004810 msgr2=0x7f86fc14b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f3fff640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e8004810 0x7f86fc14b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f3fff640 1 -- 172.21.15.29:0/2202342628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86fc0cbfa0 msgr2=0x7f86fc14bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f3fff640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86fc0cbfa0 0x7f86fc14bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f3fff640 1 -- 172.21.15.29:0/2202342628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc0cc750 con 0x7f86fc0cd7f0 2024-04-25T00:41:41.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86fa59c640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e8004810 0x7f86fc14b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f3fff640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86fc0cd7f0 0x7f86fc0cba60 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f86e4000c10 tx=0x7f86e8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f2ffd640 1 -- 172.21.15.29:0/2202342628 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4007a40 con 0x7f86fc0cd7f0 2024-04-25T00:41:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f2ffd640 1 -- 172.21.15.29:0/2202342628 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e40ba070 con 0x7f86fc0cd7f0 2024-04-25T00:41:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f2ffd640 1 -- 172.21.15.29:0/2202342628 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4007250 con 0x7f86fc0cd7f0 2024-04-25T00:41:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f870086d640 1 -- 172.21.15.29:0/2202342628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc0cca30 con 0x7f86fc0cd7f0 2024-04-25T00:41:41.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f870086d640 1 -- 172.21.15.29:0/2202342628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc0ccff0 con 0x7f86fc0cd7f0 2024-04-25T00:41:41.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f870086d640 1 -- 172.21.15.29:0/2202342628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86fc069270 con 0x7f86fc0cd7f0 2024-04-25T00:41:41.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f2ffd640 1 -- 172.21.15.29:0/2202342628 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 26) ==== 88363+0+0 (secure 0 0 0) 0x7f86e40052f0 con 0x7f86fc0cd7f0 2024-04-25T00:41:41.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f2ffd640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86dc068310 0x7f86dc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:41.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86f2ffd640 1 -- 172.21.15.29:0/2202342628 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f86e411e810 con 0x7f86fc0cd7f0 2024-04-25T00:41:41.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.736+0000 7f86fa59c640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86dc068310 0x7f86dc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:41.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.740+0000 7f86fa59c640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86dc068310 0x7f86dc06a7d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f86ec000c10 tx=0x7f86ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:41.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.744+0000 7f86f2ffd640 1 -- 172.21.15.29:0/2202342628 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f86e4016110 con 0x7f86fc0cd7f0 2024-04-25T00:41:41.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.908+0000 7f870086d640 1 -- 172.21.15.29:0/2202342628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86fc0d2d80 con 0x7f86fc0cd7f0 2024-04-25T00:41:41.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.908+0000 7f86f2ffd640 1 -- 172.21.15.29:0/2202342628 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) ==== 74+0+16635 (secure 0 0 0) 0x7f86e40f0e00 con 0x7f86fc0cd7f0 2024-04-25T00:41:41.911 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:41.911 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":168,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:41:21.712101+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:41:18.370636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:41:19.067830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:41.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.912+0000 7f870086d640 1 -- 172.21.15.29:0/2202342628 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86dc068310 msgr2=0x7f86dc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.912+0000 7f870086d640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86dc068310 0x7f86dc06a7d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f86ec000c10 tx=0x7f86ec009040 comp rx=0 tx=0).stop 2024-04-25T00:41:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.912+0000 7f870086d640 1 -- 172.21.15.29:0/2202342628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86fc0cd7f0 msgr2=0x7f86fc0cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.912+0000 7f870086d640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86fc0cd7f0 0x7f86fc0cba60 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f86e4000c10 tx=0x7f86e8005840 comp rx=0 tx=0).stop 2024-04-25T00:41:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.912+0000 7f870086d640 1 -- 172.21.15.29:0/2202342628 shutdown_connections 2024-04-25T00:41:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.912+0000 7f870086d640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f86dc068310 0x7f86dc06a7d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.912+0000 7f870086d640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86fc0cbfa0 0x7f86fc14bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.912+0000 7f870086d640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86fc0cd7f0 0x7f86fc0cba60 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.912+0000 7f870086d640 1 --2- 172.21.15.29:0/2202342628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e8004810 0x7f86fc14b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.912+0000 7f870086d640 1 -- 172.21.15.29:0/2202342628 >> 172.21.15.29:0/2202342628 conn(0x7f86fc0d3fb0 msgr2=0x7f86fc066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.912+0000 7f870086d640 1 -- 172.21.15.29:0/2202342628 shutdown_connections 2024-04-25T00:41:41.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:41.912+0000 7f870086d640 1 -- 172.21.15.29:0/2202342628 wait complete. 2024-04-25T00:41:41.930 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:41:18.370636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:41:19.067830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:41:41.931 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:41:42.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d38a74640 1 Processor -- start 2024-04-25T00:41:42.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d38a74640 1 -- start start 2024-04-25T00:41:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d38a74640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1d340b3c30 0x7f1d3406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d38a74640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d340b4b10 0x7f1d340ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d38a74640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d340b41e0 con 0x7f1d340b3140 2024-04-25T00:41:42.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d38a74640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1d3406d960 con 0x7f1d340b3c30 2024-04-25T00:41:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d38a74640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3406dc40 con 0x7f1d340b4b10 2024-04-25T00:41:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3259c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1d340b3140 0x7f1d340b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59992/0 (socket says 172.21.15.29:59992) 2024-04-25T00:41:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3259c640 1 -- 172.21.15.29:0/461281691 learned_addr learned my addr 172.21.15.29:0/461281691 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d32d9d640 1 --2- 172.21.15.29:0/461281691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d340b4b10 0x7f1d340ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d31d9b640 1 --2- 172.21.15.29:0/461281691 >> v2:172.21.15.157:3300/0 conn(0x7f1d340b3c30 0x7f1d3406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4155883307 0 0) 0x7f1d340b41e0 con 0x7f1d340b3140 2024-04-25T00:41:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d1c003530 con 0x7f1d340b3140 2024-04-25T00:41:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 564782469 0 0) 0x7f1d1c003530 con 0x7f1d340b3140 2024-04-25T00:41:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d340b4b10 msgr2=0x7f1d340ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 --2- 172.21.15.29:0/461281691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d340b4b10 0x7f1d340ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 >> v2:172.21.15.157:3300/0 conn(0x7f1d340b3c30 msgr2=0x7f1d3406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 --2- 172.21.15.29:0/461281691 >> v2:172.21.15.157:3300/0 conn(0x7f1d340b3c30 0x7f1d3406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3406df20 con 0x7f1d340b3140 2024-04-25T00:41:42.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f1d24001680 con 0x7f1d340b3140 2024-04-25T00:41:42.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 >> v1:172.21.15.29:6789/0 conn(0x7f1d340b3140 legacy=0x7f1d340b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 --2- 172.21.15.29:0/461281691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d1c0039c0 0x7f1d1c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 --2- 172.21.15.29:0/461281691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d1c004810 0x7f1d1c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 --2- 172.21.15.29:0/461281691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d340b4b10 0x7f1d1c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3406d960 con 0x7f1d1c004810 2024-04-25T00:41:42.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3406dc40 con 0x7f1d1c0039c0 2024-04-25T00:41:42.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1d1c00b9d0 con 0x7f1d340b4b10 2024-04-25T00:41:42.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d31d9b640 1 --2- 172.21.15.29:0/461281691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d1c004810 0x7f1d1c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d3259c640 1 --2- 172.21.15.29:0/461281691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d1c0039c0 0x7f1d1c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d32d9d640 1 --2- 172.21.15.29:0/461281691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d340b4b10 0x7f1d1c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d31d9b640 1 -- 172.21.15.29:0/461281691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d340b4b10 msgr2=0x7f1d1c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d31d9b640 1 --2- 172.21.15.29:0/461281691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d340b4b10 0x7f1d1c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d31d9b640 1 -- 172.21.15.29:0/461281691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d1c0039c0 msgr2=0x7f1d1c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d31d9b640 1 --2- 172.21.15.29:0/461281691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d1c0039c0 0x7f1d1c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d31d9b640 1 -- 172.21.15.29:0/461281691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c003530 con 0x7f1d1c004810 2024-04-25T00:41:42.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.100+0000 7f1d32d9d640 1 --2- 172.21.15.29:0/461281691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d340b4b10 0x7f1d1c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:42.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d31d9b640 1 --2- 172.21.15.29:0/461281691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d1c004810 0x7f1d1c004c90 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009ae0 tx=0x7f1d2801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b7630a0ab114c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d280096b0 con 0x7f1d1c004810 2024-04-25T00:41:42.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d280a64c0 con 0x7f1d1c004810 2024-04-25T00:41:42.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d3159a640 1 -- 172.21.15.29:0/461281691 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d280a68b0 con 0x7f1d1c004810 2024-04-25T00:41:42.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 -- 172.21.15.29:0/461281691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d1c004810 msgr2=0x7f1d1c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 --2- 172.21.15.29:0/461281691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d1c004810 0x7f1d1c004c90 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009ae0 tx=0x7f1d2801cab0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d32d9d640 1 -- 172.21.15.29:0/461281691 reap_dead start 2024-04-25T00:41:42.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 -- 172.21.15.29:0/461281691 shutdown_connections 2024-04-25T00:41:42.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 -- 172.21.15.29:0/461281691 >> 172.21.15.29:0/461281691 conn(0x7f1d340b1870 msgr2=0x7f1d340b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 -- 172.21.15.29:0/461281691 shutdown_connections 2024-04-25T00:41:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 -- 172.21.15.29:0/461281691 wait complete. 2024-04-25T00:41:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 Processor -- start 2024-04-25T00:41:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 -- start start 2024-04-25T00:41:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d1c004810 0x7f1d3413c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d34141830 0x7f1d34141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d34157d20 0x7f1d341581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d34145a90 con 0x7f1d34157d20 2024-04-25T00:41:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d34145910 con 0x7f1d1c004810 2024-04-25T00:41:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d38a74640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1d340ce240 con 0x7f1d34141830 2024-04-25T00:41:42.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d32d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d34157d20 0x7f1d341581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d32d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d34157d20 0x7f1d341581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53676/0 (socket says 172.21.15.29:53676) 2024-04-25T00:41:42.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.104+0000 7f1d32d9d640 1 -- 172.21.15.29:0/2118021160 learned_addr learned my addr 172.21.15.29:0/2118021160 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:42.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d3259c640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d1c004810 0x7f1d3413c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d31d9b640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d34141830 0x7f1d34141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d32d9d640 1 -- 172.21.15.29:0/2118021160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d34141830 msgr2=0x7f1d34141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d32d9d640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d34141830 0x7f1d34141cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d32d9d640 1 -- 172.21.15.29:0/2118021160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d1c004810 msgr2=0x7f1d3413c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d32d9d640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d1c004810 0x7f1d3413c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d32d9d640 1 -- 172.21.15.29:0/2118021160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3413cb30 con 0x7f1d34157d20 2024-04-25T00:41:42.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d31d9b640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d34141830 0x7f1d34141cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:42.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d32d9d640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d34157d20 0x7f1d341581d0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f1d340b4ef0 tx=0x7f1d2c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d22ffd640 1 -- 172.21.15.29:0/2118021160 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c01c070 con 0x7f1d34157d20 2024-04-25T00:41:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d22ffd640 1 -- 172.21.15.29:0/2118021160 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d2c017400 con 0x7f1d34157d20 2024-04-25T00:41:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d38a74640 1 -- 172.21.15.29:0/2118021160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d34070ce0 con 0x7f1d34157d20 2024-04-25T00:41:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d22ffd640 1 -- 172.21.15.29:0/2118021160 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0177d0 con 0x7f1d34157d20 2024-04-25T00:41:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d38a74640 1 -- 172.21.15.29:0/2118021160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d34071260 con 0x7f1d34157d20 2024-04-25T00:41:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d38a74640 1 -- 172.21.15.29:0/2118021160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d34069270 con 0x7f1d34157d20 2024-04-25T00:41:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d22ffd640 1 -- 172.21.15.29:0/2118021160 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 26) ==== 88363+0+0 (secure 0 0 0) 0x7f1d2c01f070 con 0x7f1d34157d20 2024-04-25T00:41:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d22ffd640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1d14068310 0x7f1d1406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d22ffd640 1 -- 172.21.15.29:0/2118021160 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f1d2c0848b0 con 0x7f1d34157d20 2024-04-25T00:41:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d3259c640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1d14068310 0x7f1d1406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.108+0000 7f1d3259c640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1d14068310 0x7f1d1406a7d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1d24001ec0 tx=0x7f1d24009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.112+0000 7f1d22ffd640 1 -- 172.21.15.29:0/2118021160 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1d2c012070 con 0x7f1d34157d20 2024-04-25T00:41:42.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.272+0000 7f1d38a74640 1 -- 172.21.15.29:0/2118021160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1d34144f90 con 0x7f1d34157d20 2024-04-25T00:41:42.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.276+0000 7f1d22ffd640 1 -- 172.21.15.29:0/2118021160 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v168) ==== 93+0+10 (secure 0 0 0) 0x7f1d2c056c80 con 0x7f1d34157d20 2024-04-25T00:41:42.275 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:41:42.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.276+0000 7f1d38a74640 1 -- 172.21.15.29:0/2118021160 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1d14068310 msgr2=0x7f1d1406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.276+0000 7f1d38a74640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1d14068310 0x7f1d1406a7d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1d24001ec0 tx=0x7f1d24009040 comp rx=0 tx=0).stop 2024-04-25T00:41:42.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.276+0000 7f1d38a74640 1 -- 172.21.15.29:0/2118021160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d34157d20 msgr2=0x7f1d341581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.276+0000 7f1d38a74640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d34157d20 0x7f1d341581d0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f1d340b4ef0 tx=0x7f1d2c00f4a0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.276+0000 7f1d38a74640 1 -- 172.21.15.29:0/2118021160 shutdown_connections 2024-04-25T00:41:42.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.276+0000 7f1d38a74640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1d14068310 0x7f1d1406a7d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.276+0000 7f1d38a74640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d34157d20 0x7f1d341581d0 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.276+0000 7f1d38a74640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d34141830 0x7f1d34141cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.276+0000 7f1d38a74640 1 --2- 172.21.15.29:0/2118021160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d1c004810 0x7f1d3413c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.276+0000 7f1d38a74640 1 -- 172.21.15.29:0/2118021160 >> 172.21.15.29:0/2118021160 conn(0x7f1d340b1870 msgr2=0x7f1d340cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:42.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.276+0000 7f1d38a74640 1 -- 172.21.15.29:0/2118021160 shutdown_connections 2024-04-25T00:41:42.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.276+0000 7f1d38a74640 1 -- 172.21.15.29:0/2118021160 wait complete. 2024-04-25T00:41:42.294 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:41:42.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d102aa640 1 Processor -- start 2024-04-25T00:41:42.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d102aa640 1 -- start start 2024-04-25T00:41:42.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d102aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d080b3140 0x7f0d0806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d102aa640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0d080b4b10 0x7f0d080b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d102aa640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d080b41e0 con 0x7f0d080b3a60 2024-04-25T00:41:42.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d102aa640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f0d0806d960 con 0x7f0d080b4b10 2024-04-25T00:41:42.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d102aa640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0d0806dc40 con 0x7f0d080b3140 2024-04-25T00:41:42.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d0d845640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0d080b4b10 0x7f0d080b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d0d845640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0d080b4b10 0x7f0d080b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53680/0 (socket says 172.21.15.29:53680) 2024-04-25T00:41:42.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d0d845640 1 -- 172.21.15.29:0/3931219106 learned_addr learned my addr 172.21.15.29:0/3931219106 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:42.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d0e046640 1 --2- 172.21.15.29:0/3931219106 >> v2:172.21.15.157:3300/0 conn(0x7f0d080b3140 0x7f0d0806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d0d845640 1 -- 172.21.15.29:0/3931219106 >> v2:172.21.15.157:3300/0 conn(0x7f0d080b3140 msgr2=0x7f0d0806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d0d845640 1 --2- 172.21.15.29:0/3931219106 >> v2:172.21.15.157:3300/0 conn(0x7f0d080b3140 0x7f0d0806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d0d845640 1 -- 172.21.15.29:0/3931219106 >> v1:172.21.15.157:6789/0 conn(0x7f0d080b3a60 legacy=0x7f0d0815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d0d845640 1 -- 172.21.15.29:0/3931219106 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0806df20 con 0x7f0d080b4b10 2024-04-25T00:41:42.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.412+0000 7f0d0e046640 1 --2- 172.21.15.29:0/3931219106 >> v2:172.21.15.157:3300/0 conn(0x7f0d080b3140 0x7f0d0806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:42.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.416+0000 7f0d0d845640 1 --2- 172.21.15.29:0/3931219106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d080b4b10 0x7f0d080b3520 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009870 tx=0x7f0cf801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c2f54f03e51c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.416+0000 7f0d0d044640 1 -- 172.21.15.29:0/3931219106 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80b0070 con 0x7f0d080b4b10 2024-04-25T00:41:42.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.416+0000 7f0d0d044640 1 -- 172.21.15.29:0/3931219106 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf80ab600 con 0x7f0d080b4b10 2024-04-25T00:41:42.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.416+0000 7f0d0d044640 1 -- 172.21.15.29:0/3931219106 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80a4e60 con 0x7f0d080b4b10 2024-04-25T00:41:42.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.416+0000 7f0d102aa640 1 -- 172.21.15.29:0/3931219106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d080b4b10 msgr2=0x7f0d080b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.416+0000 7f0d102aa640 1 --2- 172.21.15.29:0/3931219106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d080b4b10 0x7f0d080b3520 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009870 tx=0x7f0cf801c4e0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.416+0000 7f0d102aa640 1 -- 172.21.15.29:0/3931219106 shutdown_connections 2024-04-25T00:41:42.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.416+0000 7f0d102aa640 1 --2- 172.21.15.29:0/3931219106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d080b4b10 0x7f0d080b3520 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.416+0000 7f0d102aa640 1 --2- 172.21.15.29:0/3931219106 >> v2:172.21.15.157:3300/0 conn(0x7f0d080b3140 0x7f0d0806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.416+0000 7f0d102aa640 1 -- 172.21.15.29:0/3931219106 >> 172.21.15.29:0/3931219106 conn(0x7f0d080b1870 msgr2=0x7f0d080b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:42.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.416+0000 7f0d102aa640 1 -- 172.21.15.29:0/3931219106 shutdown_connections 2024-04-25T00:41:42.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.416+0000 7f0d102aa640 1 -- 172.21.15.29:0/3931219106 wait complete. 2024-04-25T00:41:42.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d102aa640 1 Processor -- start 2024-04-25T00:41:42.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d102aa640 1 -- start start 2024-04-25T00:41:42.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d102aa640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080b3140 0x7f0d08139650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d102aa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d080b3a60 0x7f0d08139ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d102aa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d080b4b10 0x7f0d0813a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d102aa640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08168b70 con 0x7f0d080b4b10 2024-04-25T00:41:42.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d102aa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081689f0 con 0x7f0d080b3a60 2024-04-25T00:41:42.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d102aa640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0d08168cf0 con 0x7f0d080b3140 2024-04-25T00:41:42.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d0e847640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d080b4b10 0x7f0d0813a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d0e847640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d080b4b10 0x7f0d0813a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53682/0 (socket says 172.21.15.29:53682) 2024-04-25T00:41:42.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d0e847640 1 -- 172.21.15.29:0/254939231 learned_addr learned my addr 172.21.15.29:0/254939231 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:42.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d0e046640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080b3140 0x7f0d08139650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d0d845640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d080b3a60 0x7f0d08139ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d0e847640 1 -- 172.21.15.29:0/254939231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080b3140 msgr2=0x7f0d08139650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d0e847640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080b3140 0x7f0d08139650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d0e847640 1 -- 172.21.15.29:0/254939231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d080b3a60 msgr2=0x7f0d08139ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d0e847640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d080b3a60 0x7f0d08139ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d0e847640 1 -- 172.21.15.29:0/254939231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0813cbb0 con 0x7f0d080b4b10 2024-04-25T00:41:42.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d0e046640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080b3140 0x7f0d08139650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:42.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d0d845640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d080b3a60 0x7f0d08139ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:42.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d0e847640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d080b4b10 0x7f0d0813a3e0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c7b0 tx=0x7f0d0000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0cfeffd640 1 -- 172.21.15.29:0/254939231 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00004b70 con 0x7f0d080b4b10 2024-04-25T00:41:42.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0cfeffd640 1 -- 172.21.15.29:0/254939231 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d00004d10 con 0x7f0d080b4b10 2024-04-25T00:41:42.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d102aa640 1 -- 172.21.15.29:0/254939231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d08153840 con 0x7f0d080b4b10 2024-04-25T00:41:42.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0cfeffd640 1 -- 172.21.15.29:0/254939231 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0000aa90 con 0x7f0d080b4b10 2024-04-25T00:41:42.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d102aa640 1 -- 172.21.15.29:0/254939231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d08153e00 con 0x7f0d080b4b10 2024-04-25T00:41:42.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.420+0000 7f0d102aa640 1 -- 172.21.15.29:0/254939231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d08069270 con 0x7f0d080b4b10 2024-04-25T00:41:42.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.424+0000 7f0cfeffd640 1 -- 172.21.15.29:0/254939231 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 26) ==== 88363+0+0 (secure 0 0 0) 0x7f0d0000ac30 con 0x7f0d080b4b10 2024-04-25T00:41:42.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.424+0000 7f0cfeffd640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0cec0682e0 0x7f0cec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.424+0000 7f0cfeffd640 1 -- 172.21.15.29:0/254939231 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f0d00084d70 con 0x7f0d080b4b10 2024-04-25T00:41:42.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.424+0000 7f0d0e046640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0cec0682e0 0x7f0cec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.424+0000 7f0d0e046640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0cec0682e0 0x7f0cec06a7a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0d04008890 tx=0x7f0d04008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.428+0000 7f0cfeffd640 1 -- 172.21.15.29:0/254939231 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0d000570e0 con 0x7f0d080b4b10 2024-04-25T00:41:42.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0d102aa640 1 -- 172.21.15.29:0/254939231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0d080b46d0 con 0x7f0d080b4b10 2024-04-25T00:41:42.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0cfeffd640 1 -- 172.21.15.29:0/254939231 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v168) ==== 104+0+11 (secure 0 0 0) 0x7f0d000569e0 con 0x7f0d080b4b10 2024-04-25T00:41:42.588 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:41:42.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0d102aa640 1 -- 172.21.15.29:0/254939231 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0cec0682e0 msgr2=0x7f0cec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0d102aa640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0cec0682e0 0x7f0cec06a7a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0d04008890 tx=0x7f0d04008040 comp rx=0 tx=0).stop 2024-04-25T00:41:42.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0d102aa640 1 -- 172.21.15.29:0/254939231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d080b4b10 msgr2=0x7f0d0813a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0d102aa640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d080b4b10 0x7f0d0813a3e0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c7b0 tx=0x7f0d0000cc70 comp rx=0 tx=0).stop 2024-04-25T00:41:42.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0d102aa640 1 -- 172.21.15.29:0/254939231 shutdown_connections 2024-04-25T00:41:42.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0d102aa640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0cec0682e0 0x7f0cec06a7a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0d102aa640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d080b4b10 0x7f0d0813a3e0 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0d102aa640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d080b3a60 0x7f0d08139ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0d102aa640 1 --2- 172.21.15.29:0/254939231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d080b3140 0x7f0d08139650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0d102aa640 1 -- 172.21.15.29:0/254939231 >> 172.21.15.29:0/254939231 conn(0x7f0d080b1870 msgr2=0x7f0d081550d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:42.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0d102aa640 1 -- 172.21.15.29:0/254939231 shutdown_connections 2024-04-25T00:41:42.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.588+0000 7f0d102aa640 1 -- 172.21.15.29:0/254939231 wait complete. 2024-04-25T00:41:42.607 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:41:42.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.720+0000 7f1af5d26640 1 Processor -- start 2024-04-25T00:41:42.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.720+0000 7f1af5d26640 1 -- start start 2024-04-25T00:41:42.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1af5d26640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1af015e8d0 0x7f1af00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1af5d26640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af00b3720 con 0x7f1af015f3c0 2024-04-25T00:41:42.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1af5d26640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af00b39c0 con 0x7f1af00b4780 2024-04-25T00:41:42.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1af5d26640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1af00b42f0 con 0x7f1af015e8d0 2024-04-25T00:41:42.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aeffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1af015f3c0 0x7f1af015c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59994/0 (socket says 172.21.15.29:59994) 2024-04-25T00:41:42.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aeffff640 1 -- 172.21.15.29:0/3766187915 learned_addr learned my addr 172.21.15.29:0/3766187915 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:42.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aef7fe640 1 --2- 172.21.15.29:0/3766187915 >> v2:172.21.15.157:3300/0 conn(0x7f1af015e8d0 0x7f1af00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 764969511 0 0) 0x7f1af00b3720 con 0x7f1af015f3c0 2024-04-25T00:41:42.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ad8003530 con 0x7f1af015f3c0 2024-04-25T00:41:42.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1526738859 0 0) 0x7f1ad8003530 con 0x7f1af015f3c0 2024-04-25T00:41:42.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 >> v2:172.21.15.157:3300/0 conn(0x7f1af015e8d0 msgr2=0x7f1af00cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 --2- 172.21.15.29:0/3766187915 >> v2:172.21.15.157:3300/0 conn(0x7f1af015e8d0 0x7f1af00cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 >> v1:172.21.15.157:6789/0 conn(0x7f1af00b4780 legacy=0x7f1af015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af00b3ca0 con 0x7f1af015f3c0 2024-04-25T00:41:42.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f1ae8001a00 con 0x7f1af015f3c0 2024-04-25T00:41:42.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 >> v1:172.21.15.29:6789/0 conn(0x7f1af015f3c0 legacy=0x7f1af015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 --2- 172.21.15.29:0/3766187915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad80039c0 0x7f1ad8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 --2- 172.21.15.29:0/3766187915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad8004810 0x7f1ad8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 --2- 172.21.15.29:0/3766187915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad80051d0 0x7f1ad800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1af00b42f0 con 0x7f1ad80051d0 2024-04-25T00:41:42.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad800c070 con 0x7f1ad80039c0 2024-04-25T00:41:42.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad800c270 con 0x7f1ad8004810 2024-04-25T00:41:42.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aeffff640 1 --2- 172.21.15.29:0/3766187915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad80051d0 0x7f1ad800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aef7fe640 1 --2- 172.21.15.29:0/3766187915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad80039c0 0x7f1ad8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aeeffd640 1 --2- 172.21.15.29:0/3766187915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad8004810 0x7f1ad8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aeffff640 1 -- 172.21.15.29:0/3766187915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad8004810 msgr2=0x7f1ad8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aeffff640 1 --2- 172.21.15.29:0/3766187915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad8004810 0x7f1ad8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aeffff640 1 -- 172.21.15.29:0/3766187915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad80039c0 msgr2=0x7f1ad8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aeffff640 1 --2- 172.21.15.29:0/3766187915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad80039c0 0x7f1ad8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aeffff640 1 -- 172.21.15.29:0/3766187915 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad8003530 con 0x7f1ad80051d0 2024-04-25T00:41:42.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.724+0000 7f1aef7fe640 1 --2- 172.21.15.29:0/3766187915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad80039c0 0x7f1ad8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:42.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1aeffff640 1 -- 172.21.15.29:0/3766187915 reap_dead start 2024-04-25T00:41:42.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1aeffff640 1 --2- 172.21.15.29:0/3766187915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad80051d0 0x7f1ad800b9d0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800a850 tx=0x7f1ae801d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d29590156814790 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae800a4a0 con 0x7f1ad80051d0 2024-04-25T00:41:42.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae80ad650 con 0x7f1ad80051d0 2024-04-25T00:41:42.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1aee7fc640 1 -- 172.21.15.29:0/3766187915 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80ada40 con 0x7f1ad80051d0 2024-04-25T00:41:42.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 -- 172.21.15.29:0/3766187915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad80051d0 msgr2=0x7f1ad800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 --2- 172.21.15.29:0/3766187915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad80051d0 0x7f1ad800b9d0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800a850 tx=0x7f1ae801d3a0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 -- 172.21.15.29:0/3766187915 shutdown_connections 2024-04-25T00:41:42.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 --2- 172.21.15.29:0/3766187915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ad80051d0 0x7f1ad800b9d0 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 -- 172.21.15.29:0/3766187915 >> 172.21.15.29:0/3766187915 conn(0x7f1af00b1870 msgr2=0x7f1af00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:42.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 -- 172.21.15.29:0/3766187915 shutdown_connections 2024-04-25T00:41:42.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 -- 172.21.15.29:0/3766187915 wait complete. 2024-04-25T00:41:42.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 Processor -- start 2024-04-25T00:41:42.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 -- start start 2024-04-25T00:41:42.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad80051d0 0x7f1af0138ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1af0137580 0x7f1af015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af01f28c0 0x7f1af0139640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1af006cf10 con 0x7f1af01f28c0 2024-04-25T00:41:42.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1af00b3580 con 0x7f1af0137580 2024-04-25T00:41:42.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.728+0000 7f1af5d26640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1af0144a20 con 0x7f1ad80051d0 2024-04-25T00:41:42.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aeffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af01f28c0 0x7f1af0139640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aeffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af01f28c0 0x7f1af0139640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53690/0 (socket says 172.21.15.29:53690) 2024-04-25T00:41:42.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aeffff640 1 -- 172.21.15.29:0/4162240194 learned_addr learned my addr 172.21.15.29:0/4162240194 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:42.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aef7fe640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad80051d0 0x7f1af0138ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aeeffd640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1af0137580 0x7f1af015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aeffff640 1 -- 172.21.15.29:0/4162240194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad80051d0 msgr2=0x7f1af0138ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aeffff640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad80051d0 0x7f1af0138ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aeffff640 1 -- 172.21.15.29:0/4162240194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1af0137580 msgr2=0x7f1af015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aeffff640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1af0137580 0x7f1af015b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aeffff640 1 -- 172.21.15.29:0/4162240194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af013d820 con 0x7f1af01f28c0 2024-04-25T00:41:42.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aeeffd640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1af0137580 0x7f1af015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:42.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aeffff640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af01f28c0 0x7f1af0139640 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f1ae801dc40 tx=0x7f1ad8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1ad7fff640 1 -- 172.21.15.29:0/4162240194 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8001940 con 0x7f1af01f28c0 2024-04-25T00:41:42.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1ad7fff640 1 -- 172.21.15.29:0/4162240194 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae80b4070 con 0x7f1af01f28c0 2024-04-25T00:41:42.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1af5d26640 1 -- 172.21.15.29:0/4162240194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af0137a40 con 0x7f1af01f28c0 2024-04-25T00:41:42.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1ad7fff640 1 -- 172.21.15.29:0/4162240194 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80ada00 con 0x7f1af01f28c0 2024-04-25T00:41:42.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1af5d26640 1 -- 172.21.15.29:0/4162240194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af0138000 con 0x7f1af01f28c0 2024-04-25T00:41:42.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1af5d26640 1 -- 172.21.15.29:0/4162240194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af0069270 con 0x7f1af01f28c0 2024-04-25T00:41:42.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1ad7fff640 1 -- 172.21.15.29:0/4162240194 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 26) ==== 88363+0+0 (secure 0 0 0) 0x7f1ae80b6070 con 0x7f1af01f28c0 2024-04-25T00:41:42.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1ad7fff640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1adc068310 0x7f1adc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1ad7fff640 1 -- 172.21.15.29:0/4162240194 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f1ae811e190 con 0x7f1af01f28c0 2024-04-25T00:41:42.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aef7fe640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1adc068310 0x7f1adc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.732+0000 7f1aef7fe640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1adc068310 0x7f1adc06a7d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1af013d420 tx=0x7f1ae0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.736+0000 7f1ad7fff640 1 -- 172.21.15.29:0/4162240194 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1af013d820 con 0x7f1af01f28c0 2024-04-25T00:41:42.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.836+0000 7f1ad7fff640 1 -- 172.21.15.29:0/4162240194 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f1ae80ec690 con 0x7f1af01f28c0 2024-04-25T00:41:42.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.896+0000 7f1af5d26640 1 -- 172.21.15.29:0/4162240194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1af006e7b0 con 0x7f1af01f28c0 2024-04-25T00:41:42.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.896+0000 7f1ad7fff640 1 -- 172.21.15.29:0/4162240194 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v168) ==== 100+0+11 (secure 0 0 0) 0x7f1ae80170b0 con 0x7f1af01f28c0 2024-04-25T00:41:42.898 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:41:42.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.900+0000 7f1af5d26640 1 -- 172.21.15.29:0/4162240194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1adc068310 msgr2=0x7f1adc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.900+0000 7f1af5d26640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1adc068310 0x7f1adc06a7d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1af013d420 tx=0x7f1ae0008040 comp rx=0 tx=0).stop 2024-04-25T00:41:42.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.900+0000 7f1af5d26640 1 -- 172.21.15.29:0/4162240194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af01f28c0 msgr2=0x7f1af0139640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.900+0000 7f1af5d26640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af01f28c0 0x7f1af0139640 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f1ae801dc40 tx=0x7f1ad8005840 comp rx=0 tx=0).stop 2024-04-25T00:41:42.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.900+0000 7f1af5d26640 1 -- 172.21.15.29:0/4162240194 shutdown_connections 2024-04-25T00:41:42.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.900+0000 7f1af5d26640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1adc068310 0x7f1adc06a7d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.900+0000 7f1af5d26640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af01f28c0 0x7f1af0139640 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.900+0000 7f1af5d26640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1af0137580 0x7f1af015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.900+0000 7f1af5d26640 1 --2- 172.21.15.29:0/4162240194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad80051d0 0x7f1af0138ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.900+0000 7f1af5d26640 1 -- 172.21.15.29:0/4162240194 >> 172.21.15.29:0/4162240194 conn(0x7f1af00b1870 msgr2=0x7f1af00b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:42.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.900+0000 7f1af5d26640 1 -- 172.21.15.29:0/4162240194 shutdown_connections 2024-04-25T00:41:42.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:42.900+0000 7f1af5d26640 1 -- 172.21.15.29:0/4162240194 wait complete. 2024-04-25T00:41:42.916 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:41:43.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc4a347d640 1 Processor -- start 2024-04-25T00:41:43.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc4a347d640 1 -- start start 2024-04-25T00:41:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc4a347d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc49c0b42a0 0x7fc49c0cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc4a347d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc49c15ccd0 con 0x7fc49c0af040 2024-04-25T00:41:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc4a347d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc49c15cfb0 con 0x7fc49c0aec60 2024-04-25T00:41:43.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc4a347d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc49c15d290 con 0x7fc49c0b42a0 2024-04-25T00:41:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc4a0a18640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc49c0af040 0x7fc49c0b3b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:60004/0 (socket says 172.21.15.29:60004) 2024-04-25T00:41:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc4a1a1a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc49c0b42a0 0x7fc49c0cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc4a0a18640 1 -- 172.21.15.29:0/3666802307 learned_addr learned my addr 172.21.15.29:0/3666802307 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc48ffff640 1 -- 172.21.15.29:0/3666802307 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2795977972 0 0) 0x7fc49c15ccd0 con 0x7fc49c0af040 2024-04-25T00:41:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc48ffff640 1 -- 172.21.15.29:0/3666802307 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc488003530 con 0x7fc49c0af040 2024-04-25T00:41:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc4a1a1a640 1 -- 172.21.15.29:0/3666802307 >> v1:172.21.15.157:6789/0 conn(0x7fc49c0aec60 legacy=0x7fc49c15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc4a1a1a640 1 -- 172.21.15.29:0/3666802307 >> v1:172.21.15.29:6789/0 conn(0x7fc49c0af040 legacy=0x7fc49c0b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.040+0000 7fc4a1a1a640 1 -- 172.21.15.29:0/3666802307 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c0b3670 con 0x7fc49c0b42a0 2024-04-25T00:41:43.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.044+0000 7fc4a1a1a640 1 --2- 172.21.15.29:0/3666802307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c0b42a0 0x7fc49c0cfe40 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7fc494009dc0 tx=0x7fc49401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afb56efca2609d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:43.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.044+0000 7fc48ffff640 1 -- 172.21.15.29:0/3666802307 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940b2070 con 0x7fc49c0b42a0 2024-04-25T00:41:43.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.044+0000 7fc48ffff640 1 -- 172.21.15.29:0/3666802307 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4940a5740 con 0x7fc49c0b42a0 2024-04-25T00:41:43.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.044+0000 7fc48ffff640 1 -- 172.21.15.29:0/3666802307 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940a5b10 con 0x7fc49c0b42a0 2024-04-25T00:41:43.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.044+0000 7fc4a347d640 1 -- 172.21.15.29:0/3666802307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c0b42a0 msgr2=0x7fc49c0cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.044+0000 7fc4a347d640 1 --2- 172.21.15.29:0/3666802307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c0b42a0 0x7fc49c0cfe40 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7fc494009dc0 tx=0x7fc49401c880 comp rx=0 tx=0).stop 2024-04-25T00:41:43.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.044+0000 7fc4a347d640 1 -- 172.21.15.29:0/3666802307 shutdown_connections 2024-04-25T00:41:43.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.044+0000 7fc4a347d640 1 --2- 172.21.15.29:0/3666802307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c0b42a0 0x7fc49c0cfe40 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.044+0000 7fc4a347d640 1 -- 172.21.15.29:0/3666802307 >> 172.21.15.29:0/3666802307 conn(0x7fc49c075030 msgr2=0x7fc49c0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:43.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.044+0000 7fc4a347d640 1 -- 172.21.15.29:0/3666802307 shutdown_connections 2024-04-25T00:41:43.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.044+0000 7fc4a347d640 1 -- 172.21.15.29:0/3666802307 wait complete. 2024-04-25T00:41:43.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.044+0000 7fc4a347d640 1 Processor -- start 2024-04-25T00:41:43.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.044+0000 7fc4a347d640 1 -- start start 2024-04-25T00:41:43.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a347d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0aec60 0x7fc49c1555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a347d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c0af040 0x7fc49c0abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a347d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c0b42a0 0x7fc49c14a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a347d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c169f40 con 0x7fc49c0b42a0 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a347d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c169dc0 con 0x7fc49c0af040 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a347d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc49c16a0c0 con 0x7fc49c0aec60 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a1a1a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c0b42a0 0x7fc49c14a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a1a1a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c0b42a0 0x7fc49c14a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53698/0 (socket says 172.21.15.29:53698) 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a1a1a640 1 -- 172.21.15.29:0/2876361454 learned_addr learned my addr 172.21.15.29:0/2876361454 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a1219640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0aec60 0x7fc49c1555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a1a1a640 1 -- 172.21.15.29:0/2876361454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0aec60 msgr2=0x7fc49c1555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a0a18640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c0af040 0x7fc49c0abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a1a1a640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0aec60 0x7fc49c1555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a1a1a640 1 -- 172.21.15.29:0/2876361454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c0af040 msgr2=0x7fc49c0abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a1a1a640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c0af040 0x7fc49c0abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a1a1a640 1 -- 172.21.15.29:0/2876361454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c155b20 con 0x7fc49c0b42a0 2024-04-25T00:41:43.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a0a18640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c0af040 0x7fc49c0abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:43.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a1a1a640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c0b42a0 0x7fc49c14a880 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fc4940b0040 tx=0x7fc494019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:43.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc48dffb640 1 -- 172.21.15.29:0/2876361454 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940bd020 con 0x7fc49c0b42a0 2024-04-25T00:41:43.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc48dffb640 1 -- 172.21.15.29:0/2876361454 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4940b8470 con 0x7fc49c0b42a0 2024-04-25T00:41:43.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc48dffb640 1 -- 172.21.15.29:0/2876361454 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940b2040 con 0x7fc49c0b42a0 2024-04-25T00:41:43.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a347d640 1 -- 172.21.15.29:0/2876361454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c0ac890 con 0x7fc49c0b42a0 2024-04-25T00:41:43.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a347d640 1 -- 172.21.15.29:0/2876361454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c154de0 con 0x7fc49c0b42a0 2024-04-25T00:41:43.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a347d640 1 -- 172.21.15.29:0/2876361454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49c069270 con 0x7fc49c0b42a0 2024-04-25T00:41:43.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc48dffb640 1 -- 172.21.15.29:0/2876361454 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fc4940c0020 con 0x7fc49c0b42a0 2024-04-25T00:41:43.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc48dffb640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc480068400 0x7fc48006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc48dffb640 1 -- 172.21.15.29:0/2876361454 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7fc49411f3c0 con 0x7fc49c0b42a0 2024-04-25T00:41:43.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a1219640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc480068400 0x7fc48006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.048+0000 7fc4a1219640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc480068400 0x7fc48006a8c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc49800a750 tx=0x7fc498008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:43.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.052+0000 7fc48dffb640 1 -- 172.21.15.29:0/2876361454 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc4940c7070 con 0x7fc49c0b42a0 2024-04-25T00:41:43.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.220+0000 7fc4a347d640 1 -- 172.21.15.29:0/2876361454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc49c0acdb0 con 0x7fc49c0b42a0 2024-04-25T00:41:43.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.220+0000 7fc48dffb640 1 -- 172.21.15.29:0/2876361454 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v168) ==== 80+0+0 (secure 0 0 0) 0x7fc494016500 con 0x7fc49c0b42a0 2024-04-25T00:41:43.220 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:41:43.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.220+0000 7fc4a347d640 1 -- 172.21.15.29:0/2876361454 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc480068400 msgr2=0x7fc48006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.220+0000 7fc4a347d640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc480068400 0x7fc48006a8c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc49800a750 tx=0x7fc498008040 comp rx=0 tx=0).stop 2024-04-25T00:41:43.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.220+0000 7fc4a347d640 1 -- 172.21.15.29:0/2876361454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c0b42a0 msgr2=0x7fc49c14a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.220+0000 7fc4a347d640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c0b42a0 0x7fc49c14a880 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fc4940b0040 tx=0x7fc494019620 comp rx=0 tx=0).stop 2024-04-25T00:41:43.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.220+0000 7fc4a347d640 1 -- 172.21.15.29:0/2876361454 shutdown_connections 2024-04-25T00:41:43.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.220+0000 7fc4a347d640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc480068400 0x7fc48006a8c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.220+0000 7fc4a347d640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c0b42a0 0x7fc49c14a880 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.220+0000 7fc4a347d640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c0af040 0x7fc49c0abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.220+0000 7fc4a347d640 1 --2- 172.21.15.29:0/2876361454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0aec60 0x7fc49c1555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.220+0000 7fc4a347d640 1 -- 172.21.15.29:0/2876361454 >> 172.21.15.29:0/2876361454 conn(0x7fc49c075030 msgr2=0x7fc49c0b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:43.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.220+0000 7fc4a347d640 1 -- 172.21.15.29:0/2876361454 shutdown_connections 2024-04-25T00:41:43.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.224+0000 7fc4a347d640 1 -- 172.21.15.29:0/2876361454 wait complete. 2024-04-25T00:41:43.238 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-04-25T00:41:43.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:43.239 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:41:43.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi029.stderr:ceph-fuse[58708]: fuse finished with error 0 and tester_r 0 2024-04-25T00:41:43.251 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:41:43.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:43.251 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:41:43.426 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:41:43.426 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:41:43.426 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:41:43.433 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:43.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f48008c3640 1 Processor -- start 2024-04-25T00:41:43.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f48008c3640 1 -- start start 2024-04-25T00:41:43.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f48008c3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc15e200 0x7f47fc0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f48008c3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47fc0cf690 0x7f47fc0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f48008c3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47fc0cfc10 con 0x7f47fc0d2a90 2024-04-25T00:41:43.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f48008c3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f47fc0d2ee0 con 0x7f47fc0cf690 2024-04-25T00:41:43.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f48008c3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f47fc0b38d0 con 0x7f47fc15e200 2024-04-25T00:41:43.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47fad9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f47fc0d2a90 0x7f47fc0d05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:60010/0 (socket says 172.21.15.29:60010) 2024-04-25T00:41:43.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47fad9d640 1 -- 172.21.15.29:0/2519267487 learned_addr learned my addr 172.21.15.29:0/2519267487 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:43.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47fa59c640 1 --2- 172.21.15.29:0/2519267487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc15e200 0x7f47fc0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f9d9b640 1 --2- 172.21.15.29:0/2519267487 >> v2:172.21.15.157:3300/0 conn(0x7f47fc0cf690 0x7f47fc0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3545153650 0 0) 0x7f47fc0cfc10 con 0x7f47fc0d2a90 2024-04-25T00:41:43.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47e4003530 con 0x7f47fc0d2a90 2024-04-25T00:41:43.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 721090201 0 0) 0x7f47e4003530 con 0x7f47fc0d2a90 2024-04-25T00:41:43.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc15e200 msgr2=0x7f47fc0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:43.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 --2- 172.21.15.29:0/2519267487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc15e200 0x7f47fc0d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 >> v2:172.21.15.157:3300/0 conn(0x7f47fc0cf690 msgr2=0x7f47fc0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:43.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 --2- 172.21.15.29:0/2519267487 >> v2:172.21.15.157:3300/0 conn(0x7f47fc0cf690 0x7f47fc0d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47fc0cfef0 con 0x7f47fc0d2a90 2024-04-25T00:41:43.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f47f4001a00 con 0x7f47fc0d2a90 2024-04-25T00:41:43.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 >> v1:172.21.15.29:6789/0 conn(0x7f47fc0d2a90 legacy=0x7f47fc0d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 --2- 172.21.15.29:0/2519267487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc15e200 0x7f47e40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 --2- 172.21.15.29:0/2519267487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47e4003f40 0x7f47e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 --2- 172.21.15.29:0/2519267487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47e4004d50 0x7f47e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc0d2ee0 con 0x7f47e4004d50 2024-04-25T00:41:43.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc0b38d0 con 0x7f47e4003f40 2024-04-25T00:41:43.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f47e400b9d0 con 0x7f47fc15e200 2024-04-25T00:41:43.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47fad9d640 1 --2- 172.21.15.29:0/2519267487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47e4004d50 0x7f47e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47f9d9b640 1 --2- 172.21.15.29:0/2519267487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47e4003f40 0x7f47e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47fa59c640 1 --2- 172.21.15.29:0/2519267487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc15e200 0x7f47e40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47fad9d640 1 -- 172.21.15.29:0/2519267487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc15e200 msgr2=0x7f47e40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:43.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47fad9d640 1 --2- 172.21.15.29:0/2519267487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc15e200 0x7f47e40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47fad9d640 1 -- 172.21.15.29:0/2519267487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47e4003f40 msgr2=0x7f47e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:43.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47fad9d640 1 --2- 172.21.15.29:0/2519267487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47e4003f40 0x7f47e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47fad9d640 1 -- 172.21.15.29:0/2519267487 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e4003530 con 0x7f47e4004d50 2024-04-25T00:41:43.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.592+0000 7f47fa59c640 1 --2- 172.21.15.29:0/2519267487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc15e200 0x7f47e40039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:43.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.596+0000 7f47fad9d640 1 --2- 172.21.15.29:0/2519267487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47e4004d50 0x7f47e40051d0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f47f40013e0 tx=0x7f47f401d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549eaa4b2702f813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:43.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.596+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f400a4e0 con 0x7f47e4004d50 2024-04-25T00:41:43.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.596+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f40072f0 con 0x7f47e4004d50 2024-04-25T00:41:43.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.596+0000 7f47f959a640 1 -- 172.21.15.29:0/2519267487 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f40076e0 con 0x7f47e4004d50 2024-04-25T00:41:43.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.596+0000 7f48008c3640 1 -- 172.21.15.29:0/2519267487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47e4004d50 msgr2=0x7f47e40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.596+0000 7f48008c3640 1 --2- 172.21.15.29:0/2519267487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47e4004d50 0x7f47e40051d0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f47f40013e0 tx=0x7f47f401d340 comp rx=0 tx=0).stop 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.596+0000 7f47fad9d640 1 -- 172.21.15.29:0/2519267487 reap_dead start 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.596+0000 7f48008c3640 1 -- 172.21.15.29:0/2519267487 shutdown_connections 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.596+0000 7f48008c3640 1 -- 172.21.15.29:0/2519267487 >> 172.21.15.29:0/2519267487 conn(0x7f47fc09d280 msgr2=0x7f47fc0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.596+0000 7f48008c3640 1 -- 172.21.15.29:0/2519267487 shutdown_connections 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.596+0000 7f48008c3640 1 -- 172.21.15.29:0/2519267487 wait complete. 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.596+0000 7f48008c3640 1 Processor -- start 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.596+0000 7f48008c3640 1 -- start start 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f48008c3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47e4004d50 0x7f47fc1f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f48008c3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc0ca1e0 0x7f47fc1f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f48008c3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47fc1f4a70 0x7f47fc0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f48008c3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc15e5e0 con 0x7f47e4004d50 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f48008c3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc0b3e00 con 0x7f47fc1f4a70 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f48008c3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f47fc15e760 con 0x7f47fc0ca1e0 2024-04-25T00:41:43.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47fa59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47e4004d50 0x7f47fc1f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47fa59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47e4004d50 0x7f47fc1f6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53704/0 (socket says 172.21.15.29:53704) 2024-04-25T00:41:43.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47fa59c640 1 -- 172.21.15.29:0/2025701401 learned_addr learned my addr 172.21.15.29:0/2025701401 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:43.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47fad9d640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47fc1f4a70 0x7f47fc0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47f9d9b640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc0ca1e0 0x7f47fc1f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47fa59c640 1 -- 172.21.15.29:0/2025701401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc0ca1e0 msgr2=0x7f47fc1f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47fa59c640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc0ca1e0 0x7f47fc1f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47fa59c640 1 -- 172.21.15.29:0/2025701401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47fc1f4a70 msgr2=0x7f47fc0b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47fa59c640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47fc1f4a70 0x7f47fc0b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47fa59c640 1 -- 172.21.15.29:0/2025701401 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47fc0b6b90 con 0x7f47e4004d50 2024-04-25T00:41:43.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47fad9d640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47fc1f4a70 0x7f47fc0b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:43.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47fa59c640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47e4004d50 0x7f47fc1f6b20 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0108d0 tx=0x7f47ec010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:43.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47eaffd640 1 -- 172.21.15.29:0/2025701401 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec00f040 con 0x7f47e4004d50 2024-04-25T00:41:43.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47eaffd640 1 -- 172.21.15.29:0/2025701401 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec00eb60 con 0x7f47e4004d50 2024-04-25T00:41:43.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f48008c3640 1 -- 172.21.15.29:0/2025701401 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47fc1f7060 con 0x7f47e4004d50 2024-04-25T00:41:43.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47eaffd640 1 -- 172.21.15.29:0/2025701401 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec01c400 con 0x7f47e4004d50 2024-04-25T00:41:43.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f48008c3640 1 -- 172.21.15.29:0/2025701401 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47fc0b70e0 con 0x7f47e4004d50 2024-04-25T00:41:43.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f48008c3640 1 -- 172.21.15.29:0/2025701401 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47fc069270 con 0x7f47e4004d50 2024-04-25T00:41:43.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47eaffd640 1 -- 172.21.15.29:0/2025701401 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f47ec01c5a0 con 0x7f47e4004d50 2024-04-25T00:41:43.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47eaffd640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f47dc068400 0x7f47dc06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47eaffd640 1 -- 172.21.15.29:0/2025701401 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f47ec084fa0 con 0x7f47e4004d50 2024-04-25T00:41:43.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47f9d9b640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f47dc068400 0x7f47dc06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.600+0000 7f47f9d9b640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f47dc068400 0x7f47dc06a8c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0d24a0 tx=0x7f47f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:43.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.604+0000 7f47eaffd640 1 -- 172.21.15.29:0/2025701401 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f47fc1f7060 con 0x7f47e4004d50 2024-04-25T00:41:43.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.776+0000 7f48008c3640 1 -- 172.21.15.29:0/2025701401 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47fc06d740 con 0x7f47e4004d50 2024-04-25T00:41:43.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.776+0000 7f47eaffd640 1 -- 172.21.15.29:0/2025701401 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) ==== 74+0+16635 (secure 0 0 0) 0x7f47ec018030 con 0x7f47e4004d50 2024-04-25T00:41:43.778 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:43.778 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":168,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:41:21.712101+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:41:18.370636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:41:19.067830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:43.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.780+0000 7f48008c3640 1 -- 172.21.15.29:0/2025701401 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f47dc068400 msgr2=0x7f47dc06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.780+0000 7f48008c3640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f47dc068400 0x7f47dc06a8c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0d24a0 tx=0x7f47f0008040 comp rx=0 tx=0).stop 2024-04-25T00:41:43.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.780+0000 7f48008c3640 1 -- 172.21.15.29:0/2025701401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47e4004d50 msgr2=0x7f47fc1f6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.780+0000 7f48008c3640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47e4004d50 0x7f47fc1f6b20 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0108d0 tx=0x7f47ec010d90 comp rx=0 tx=0).stop 2024-04-25T00:41:43.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.780+0000 7f48008c3640 1 -- 172.21.15.29:0/2025701401 shutdown_connections 2024-04-25T00:41:43.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.780+0000 7f48008c3640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f47dc068400 0x7f47dc06a8c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.780+0000 7f48008c3640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47fc1f4a70 0x7f47fc0b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.780+0000 7f48008c3640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47fc0ca1e0 0x7f47fc1f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.780+0000 7f48008c3640 1 --2- 172.21.15.29:0/2025701401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47e4004d50 0x7f47fc1f6b20 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.780+0000 7f48008c3640 1 -- 172.21.15.29:0/2025701401 >> 172.21.15.29:0/2025701401 conn(0x7f47fc09d280 msgr2=0x7f47fc15d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:43.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.780+0000 7f48008c3640 1 -- 172.21.15.29:0/2025701401 shutdown_connections 2024-04-25T00:41:43.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.780+0000 7f48008c3640 1 -- 172.21.15.29:0/2025701401 wait complete. 2024-04-25T00:41:43.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:41:18.370636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:41:19.067830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:41:43.798 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:41:43.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e5a503640 1 Processor -- start 2024-04-25T00:41:43.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e5a503640 1 -- start start 2024-04-25T00:41:43.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e5a503640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2e5415e200 0x7f2e540d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e5a503640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e540d2c60 0x7f2e540b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e5a503640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e540cfc10 con 0x7f2e540cf690 2024-04-25T00:41:43.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e5a503640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2e540b38d0 con 0x7f2e5415e200 2024-04-25T00:41:43.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e5a503640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2e540cfef0 con 0x7f2e540d2c60 2024-04-25T00:41:43.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e53fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2e5415e200 0x7f2e540d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e53fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2e5415e200 0x7f2e540d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53710/0 (socket says 172.21.15.29:53710) 2024-04-25T00:41:43.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e53fff640 1 -- 172.21.15.29:0/271620925 learned_addr learned my addr 172.21.15.29:0/271620925 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:43.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e53fff640 1 -- 172.21.15.29:0/271620925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e540d2c60 msgr2=0x7f2e540b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:43.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e53fff640 1 --2- 172.21.15.29:0/271620925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e540d2c60 0x7f2e540b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e53fff640 1 -- 172.21.15.29:0/271620925 >> v1:172.21.15.29:6789/0 conn(0x7f2e540cf690 legacy=0x7f2e540d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.924+0000 7f2e53fff640 1 -- 172.21.15.29:0/271620925 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e540b3b20 con 0x7f2e5415e200 2024-04-25T00:41:43.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e53fff640 1 --2- 172.21.15.29:0/271620925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e5415e200 0x7f2e540d47f0 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009870 tx=0x7f2e4401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40f4f7b021c4427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:43.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e537fe640 1 -- 172.21.15.29:0/271620925 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e440a9610 con 0x7f2e5415e200 2024-04-25T00:41:43.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e537fe640 1 -- 172.21.15.29:0/271620925 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e440a97b0 con 0x7f2e5415e200 2024-04-25T00:41:43.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e537fe640 1 -- 172.21.15.29:0/271620925 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e440a9b80 con 0x7f2e5415e200 2024-04-25T00:41:43.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 -- 172.21.15.29:0/271620925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e5415e200 msgr2=0x7f2e540d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 --2- 172.21.15.29:0/271620925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e5415e200 0x7f2e540d47f0 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009870 tx=0x7f2e4401c530 comp rx=0 tx=0).stop 2024-04-25T00:41:43.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 -- 172.21.15.29:0/271620925 shutdown_connections 2024-04-25T00:41:43.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 --2- 172.21.15.29:0/271620925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e540d2c60 0x7f2e540b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 --2- 172.21.15.29:0/271620925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e5415e200 0x7f2e540d47f0 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 -- 172.21.15.29:0/271620925 >> 172.21.15.29:0/271620925 conn(0x7f2e5409d280 msgr2=0x7f2e540d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:43.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 -- 172.21.15.29:0/271620925 shutdown_connections 2024-04-25T00:41:43.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 -- 172.21.15.29:0/271620925 wait complete. 2024-04-25T00:41:43.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 Processor -- start 2024-04-25T00:41:43.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 -- start start 2024-04-25T00:41:43.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e540cf690 0x7f2e541faa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e540d2c60 0x7f2e541587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e5415e200 0x7f2e54146fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2e540b51f0 con 0x7f2e540cf690 2024-04-25T00:41:43.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e540b5070 con 0x7f2e540d2c60 2024-04-25T00:41:43.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e5a503640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2e540b5370 con 0x7f2e5415e200 2024-04-25T00:41:43.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e53fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e540cf690 0x7f2e541faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e53fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e540cf690 0x7f2e541faa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53718/0 (socket says 172.21.15.29:53718) 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e53fff640 1 -- 172.21.15.29:0/34157932 learned_addr learned my addr 172.21.15.29:0/34157932 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e4bfff640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e540d2c60 0x7f2e541587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e58aa0640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e5415e200 0x7f2e54146fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e53fff640 1 -- 172.21.15.29:0/34157932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e5415e200 msgr2=0x7f2e54146fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e53fff640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e5415e200 0x7f2e54146fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e53fff640 1 -- 172.21.15.29:0/34157932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e540d2c60 msgr2=0x7f2e541587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e53fff640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e540d2c60 0x7f2e541587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e53fff640 1 -- 172.21.15.29:0/34157932 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e54147a60 con 0x7f2e540cf690 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.928+0000 7f2e53fff640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e540cf690 0x7f2e541faa50 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f2e4401abe0 tx=0x7f2e440192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.932+0000 7f2e517fa640 1 -- 172.21.15.29:0/34157932 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e440bb5f0 con 0x7f2e540cf690 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.932+0000 7f2e517fa640 1 -- 172.21.15.29:0/34157932 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e440bb7b0 con 0x7f2e540cf690 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.932+0000 7f2e5a503640 1 -- 172.21.15.29:0/34157932 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e54147d40 con 0x7f2e540cf690 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.932+0000 7f2e517fa640 1 -- 172.21.15.29:0/34157932 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e440bbba0 con 0x7f2e540cf690 2024-04-25T00:41:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.932+0000 7f2e5a503640 1 -- 172.21.15.29:0/34157932 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5415a780 con 0x7f2e540cf690 2024-04-25T00:41:43.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.932+0000 7f2e5a503640 1 -- 172.21.15.29:0/34157932 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e541fa040 con 0x7f2e540cf690 2024-04-25T00:41:43.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.932+0000 7f2e517fa640 1 -- 172.21.15.29:0/34157932 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f2e440c43f0 con 0x7f2e540cf690 2024-04-25T00:41:43.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.932+0000 7f2e517fa640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2e34068400 0x7f2e3406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:43.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.932+0000 7f2e517fa640 1 -- 172.21.15.29:0/34157932 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f2e4411f010 con 0x7f2e540cf690 2024-04-25T00:41:43.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.932+0000 7f2e4bfff640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2e34068400 0x7f2e3406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:43.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.932+0000 7f2e4bfff640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2e34068400 0x7f2e3406a8c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2e54161880 tx=0x7f2e3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:43.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:43.936+0000 7f2e517fa640 1 -- 172.21.15.29:0/34157932 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2e440ad020 con 0x7f2e540cf690 2024-04-25T00:41:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.096+0000 7f2e5a503640 1 -- 172.21.15.29:0/34157932 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2e54069270 con 0x7f2e540cf690 2024-04-25T00:41:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.100+0000 7f2e517fa640 1 -- 172.21.15.29:0/34157932 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v168) ==== 93+0+10 (secure 0 0 0) 0x7f2e44016150 con 0x7f2e540cf690 2024-04-25T00:41:44.100 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:41:44.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.100+0000 7f2e5a503640 1 -- 172.21.15.29:0/34157932 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2e34068400 msgr2=0x7f2e3406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.100+0000 7f2e5a503640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2e34068400 0x7f2e3406a8c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2e54161880 tx=0x7f2e3c009040 comp rx=0 tx=0).stop 2024-04-25T00:41:44.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.100+0000 7f2e5a503640 1 -- 172.21.15.29:0/34157932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e540cf690 msgr2=0x7f2e541faa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.100+0000 7f2e5a503640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e540cf690 0x7f2e541faa50 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f2e4401abe0 tx=0x7f2e440192d0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.100+0000 7f2e5a503640 1 -- 172.21.15.29:0/34157932 shutdown_connections 2024-04-25T00:41:44.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.100+0000 7f2e5a503640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2e34068400 0x7f2e3406a8c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.100+0000 7f2e5a503640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e5415e200 0x7f2e54146fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.100+0000 7f2e5a503640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e540d2c60 0x7f2e541587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.100+0000 7f2e5a503640 1 --2- 172.21.15.29:0/34157932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e540cf690 0x7f2e541faa50 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.100+0000 7f2e5a503640 1 -- 172.21.15.29:0/34157932 >> 172.21.15.29:0/34157932 conn(0x7f2e5409d280 msgr2=0x7f2e540d2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:44.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.100+0000 7f2e5a503640 1 -- 172.21.15.29:0/34157932 shutdown_connections 2024-04-25T00:41:44.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.100+0000 7f2e5a503640 1 -- 172.21.15.29:0/34157932 wait complete. 2024-04-25T00:41:44.119 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:41:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f65254e8640 1 Processor -- start 2024-04-25T00:41:44.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f65254e8640 1 -- start start 2024-04-25T00:41:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f65254e8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65200cf690 0x7f65200d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f65254e8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200d2a90 0x7f65200b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f65254e8640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65200cfc10 con 0x7f652015e200 2024-04-25T00:41:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f65254e8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f65200d2ee0 con 0x7f65200cf690 2024-04-25T00:41:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f65254e8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65200b38d0 con 0x7f65200d2a90 2024-04-25T00:41:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651effd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f652015e200 0x7f65200d47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:60038/0 (socket says 172.21.15.29:60038) 2024-04-25T00:41:44.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651effd640 1 -- 172.21.15.29:0/3368558235 learned_addr learned my addr 172.21.15.29:0/3368558235 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:44.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651f7fe640 1 --2- 172.21.15.29:0/3368558235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200d2a90 0x7f65200b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651e7fc640 1 --2- 172.21.15.29:0/3368558235 >> v2:172.21.15.157:3300/0 conn(0x7f65200cf690 0x7f65200d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1667324614 0 0) 0x7f65200cfc10 con 0x7f652015e200 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6508003530 con 0x7f652015e200 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1575528415 0 0) 0x7f6508003530 con 0x7f652015e200 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200d2a90 msgr2=0x7f65200b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 --2- 172.21.15.29:0/3368558235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200d2a90 0x7f65200b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 >> v2:172.21.15.157:3300/0 conn(0x7f65200cf690 msgr2=0x7f65200d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 --2- 172.21.15.29:0/3368558235 >> v2:172.21.15.157:3300/0 conn(0x7f65200cf690 0x7f65200d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65200cfef0 con 0x7f652015e200 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f6510001680 con 0x7f652015e200 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651f7fe640 1 --2- 172.21.15.29:0/3368558235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200d2a90 0x7f65200b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 >> v1:172.21.15.29:6789/0 conn(0x7f652015e200 legacy=0x7f65200d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 --2- 172.21.15.29:0/3368558235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65080039c0 0x7f6508003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 --2- 172.21.15.29:0/3368558235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6508004810 0x7f6508004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 --2- 172.21.15.29:0/3368558235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200d2a90 0x7f65080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f65200d2ee0 con 0x7f65080039c0 2024-04-25T00:41:44.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f65200b38d0 con 0x7f6508004810 2024-04-25T00:41:44.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f650800b9d0 con 0x7f65200d2a90 2024-04-25T00:41:44.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651effd640 1 --2- 172.21.15.29:0/3368558235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65080039c0 0x7f6508003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651e7fc640 1 --2- 172.21.15.29:0/3368558235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6508004810 0x7f6508004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651f7fe640 1 --2- 172.21.15.29:0/3368558235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200d2a90 0x7f65080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651effd640 1 -- 172.21.15.29:0/3368558235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200d2a90 msgr2=0x7f65080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:44.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651effd640 1 --2- 172.21.15.29:0/3368558235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200d2a90 0x7f65080051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651effd640 1 -- 172.21.15.29:0/3368558235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6508004810 msgr2=0x7f6508004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:44.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651effd640 1 --2- 172.21.15.29:0/3368558235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6508004810 0x7f6508004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.252+0000 7f651effd640 1 -- 172.21.15.29:0/3368558235 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6508003530 con 0x7f65080039c0 2024-04-25T00:41:44.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.256+0000 7f651effd640 1 --2- 172.21.15.29:0/3368558235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65080039c0 0x7f6508003de0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f651000c270 tx=0x7f651001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f94f39999f1f83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:44.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.256+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510007d60 con 0x7f65080039c0 2024-04-25T00:41:44.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.256+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65100a9020 con 0x7f65080039c0 2024-04-25T00:41:44.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.256+0000 7f651dffb640 1 -- 172.21.15.29:0/3368558235 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510005ae0 con 0x7f65080039c0 2024-04-25T00:41:44.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.256+0000 7f65254e8640 1 -- 172.21.15.29:0/3368558235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65080039c0 msgr2=0x7f6508003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.256+0000 7f65254e8640 1 --2- 172.21.15.29:0/3368558235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65080039c0 0x7f6508003de0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f651000c270 tx=0x7f651001ccc0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.256+0000 7f651f7fe640 1 -- 172.21.15.29:0/3368558235 reap_dead start 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.256+0000 7f65254e8640 1 -- 172.21.15.29:0/3368558235 shutdown_connections 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.256+0000 7f65254e8640 1 -- 172.21.15.29:0/3368558235 >> 172.21.15.29:0/3368558235 conn(0x7f652009d280 msgr2=0x7f65200d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.256+0000 7f65254e8640 1 -- 172.21.15.29:0/3368558235 shutdown_connections 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.256+0000 7f65254e8640 1 -- 172.21.15.29:0/3368558235 wait complete. 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.256+0000 7f65254e8640 1 Processor -- start 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.256+0000 7f65254e8640 1 -- start start 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65254e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65080039c0 0x7f65201557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65254e8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200cbf60 0x7f65200d0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65254e8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65200cb040 0x7f65200d0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65254e8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f65200b4cb0 con 0x7f65200cb040 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65254e8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f65200b4b30 con 0x7f65080039c0 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65254e8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65200b4e30 con 0x7f65200cbf60 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f651f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65200cb040 0x7f65200d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f651f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65200cb040 0x7f65200d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53726/0 (socket says 172.21.15.29:53726) 2024-04-25T00:41:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f651f7fe640 1 -- 172.21.15.29:0/2000282570 learned_addr learned my addr 172.21.15.29:0/2000282570 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f651effd640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65080039c0 0x7f65201557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f651e7fc640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200cbf60 0x7f65200d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f651f7fe640 1 -- 172.21.15.29:0/2000282570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200cbf60 msgr2=0x7f65200d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f651f7fe640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200cbf60 0x7f65200d0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f651f7fe640 1 -- 172.21.15.29:0/2000282570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65080039c0 msgr2=0x7f65201557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f651f7fe640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65080039c0 0x7f65201557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f651f7fe640 1 -- 172.21.15.29:0/2000282570 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65200cc460 con 0x7f65200cb040 2024-04-25T00:41:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f651f7fe640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65200cb040 0x7f65200d0650 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f65200b6a70 tx=0x7f6518004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65077fe640 1 -- 172.21.15.29:0/2000282570 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518019070 con 0x7f65200cb040 2024-04-25T00:41:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65077fe640 1 -- 172.21.15.29:0/2000282570 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6518007d30 con 0x7f65200cb040 2024-04-25T00:41:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65254e8640 1 -- 172.21.15.29:0/2000282570 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65200cc740 con 0x7f65200cb040 2024-04-25T00:41:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65077fe640 1 -- 172.21.15.29:0/2000282570 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518014640 con 0x7f65200cb040 2024-04-25T00:41:44.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65254e8640 1 -- 172.21.15.29:0/2000282570 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65200ccd00 con 0x7f65200cb040 2024-04-25T00:41:44.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65254e8640 1 -- 172.21.15.29:0/2000282570 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6520069270 con 0x7f65200cb040 2024-04-25T00:41:44.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65077fe640 1 -- 172.21.15.29:0/2000282570 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f651801c070 con 0x7f65200cb040 2024-04-25T00:41:44.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65077fe640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f650c068430 0x7f650c06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f65077fe640 1 -- 172.21.15.29:0/2000282570 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f6518083f70 con 0x7f65200cb040 2024-04-25T00:41:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f651effd640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f650c068430 0x7f650c06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.260+0000 7f651effd640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f650c068430 0x7f650c06a8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f65100aa000 tx=0x7f65100b0040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:44.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.264+0000 7f65077fe640 1 -- 172.21.15.29:0/2000282570 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6518055f10 con 0x7f65200cb040 2024-04-25T00:41:44.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.424+0000 7f65254e8640 1 -- 172.21.15.29:0/2000282570 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f652015cc10 con 0x7f65200cb040 2024-04-25T00:41:44.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.428+0000 7f65077fe640 1 -- 172.21.15.29:0/2000282570 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v168) ==== 104+0+11 (secure 0 0 0) 0x7f651805a210 con 0x7f65200cb040 2024-04-25T00:41:44.427 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:41:44.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.428+0000 7f65254e8640 1 -- 172.21.15.29:0/2000282570 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f650c068430 msgr2=0x7f650c06a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.428+0000 7f65254e8640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f650c068430 0x7f650c06a8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f65100aa000 tx=0x7f65100b0040 comp rx=0 tx=0).stop 2024-04-25T00:41:44.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.428+0000 7f65254e8640 1 -- 172.21.15.29:0/2000282570 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65200cb040 msgr2=0x7f65200d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.428+0000 7f65254e8640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65200cb040 0x7f65200d0650 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f65200b6a70 tx=0x7f6518004700 comp rx=0 tx=0).stop 2024-04-25T00:41:44.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.428+0000 7f65254e8640 1 -- 172.21.15.29:0/2000282570 shutdown_connections 2024-04-25T00:41:44.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.428+0000 7f65254e8640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f650c068430 0x7f650c06a8f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.428+0000 7f65254e8640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65200cb040 0x7f65200d0650 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.428+0000 7f65254e8640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65200cbf60 0x7f65200d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.428+0000 7f65254e8640 1 --2- 172.21.15.29:0/2000282570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65080039c0 0x7f65201557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.428+0000 7f65254e8640 1 -- 172.21.15.29:0/2000282570 >> 172.21.15.29:0/2000282570 conn(0x7f652009d280 msgr2=0x7f652015bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:44.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.428+0000 7f65254e8640 1 -- 172.21.15.29:0/2000282570 shutdown_connections 2024-04-25T00:41:44.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.428+0000 7f65254e8640 1 -- 172.21.15.29:0/2000282570 wait complete. 2024-04-25T00:41:44.447 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:41:44.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f336c021640 1 Processor -- start 2024-04-25T00:41:44.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f336c021640 1 -- start start 2024-04-25T00:41:44.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f336c021640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33640b3140 0x7f336406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f336c021640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33640b41e0 con 0x7f33640b4b10 2024-04-25T00:41:44.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f336c021640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f336406d960 con 0x7f33640b3c30 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f336c021640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f336406dc40 con 0x7f33640b3140 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f33695bc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f33640b4b10 0x7f33640b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:60050/0 (socket says 172.21.15.29:60050) 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f33695bc640 1 -- 172.21.15.29:0/1862615523 learned_addr learned my addr 172.21.15.29:0/1862615523 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3369dbd640 1 --2- 172.21.15.29:0/1862615523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33640b3140 0x7f336406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1134140461 0 0) 0x7f33640b41e0 con 0x7f33640b4b10 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3350003530 con 0x7f33640b4b10 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3163988293 0 0) 0x7f3350003530 con 0x7f33640b4b10 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33640b3140 msgr2=0x7f336406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 --2- 172.21.15.29:0/1862615523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33640b3140 0x7f336406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 >> v1:172.21.15.157:6789/0 conn(0x7f33640b3c30 legacy=0x7f33640ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336406df20 con 0x7f33640b4b10 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f3354001680 con 0x7f33640b4b10 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 >> v1:172.21.15.29:6789/0 conn(0x7f33640b4b10 legacy=0x7f33640b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 --2- 172.21.15.29:0/1862615523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33500039f0 0x7f3350003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 --2- 172.21.15.29:0/1862615523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3350004810 0x7f3350004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 --2- 172.21.15.29:0/1862615523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33640b3140 0x7f33500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f336406dc40 con 0x7f33500039f0 2024-04-25T00:41:44.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f335000ba80 con 0x7f3350004810 2024-04-25T00:41:44.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f335000bc80 con 0x7f33640b3140 2024-04-25T00:41:44.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.568+0000 7f3369dbd640 1 --2- 172.21.15.29:0/1862615523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33500039f0 0x7f3350003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f336a5be640 1 --2- 172.21.15.29:0/1862615523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33640b3140 0x7f33500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f33695bc640 1 --2- 172.21.15.29:0/1862615523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3350004810 0x7f3350004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f3369dbd640 1 -- 172.21.15.29:0/1862615523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33640b3140 msgr2=0x7f33500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:44.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f3369dbd640 1 --2- 172.21.15.29:0/1862615523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33640b3140 0x7f33500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f3369dbd640 1 -- 172.21.15.29:0/1862615523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3350004810 msgr2=0x7f3350004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:44.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f3369dbd640 1 --2- 172.21.15.29:0/1862615523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3350004810 0x7f3350004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f3369dbd640 1 -- 172.21.15.29:0/1862615523 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3350003530 con 0x7f33500039f0 2024-04-25T00:41:44.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f336a5be640 1 --2- 172.21.15.29:0/1862615523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33640b3140 0x7f33500051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:44.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f3369dbd640 1 --2- 172.21.15.29:0/1862615523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33500039f0 0x7f3350003e10 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f3360009a90 tx=0x7f336001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f41a2115f033d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:44.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360009690 con 0x7f33500039f0 2024-04-25T00:41:44.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33600a8450 con 0x7f33500039f0 2024-04-25T00:41:44.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f3368dbb640 1 -- 172.21.15.29:0/1862615523 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33600a8820 con 0x7f33500039f0 2024-04-25T00:41:44.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f336c021640 1 -- 172.21.15.29:0/1862615523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33500039f0 msgr2=0x7f3350003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f336c021640 1 --2- 172.21.15.29:0/1862615523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33500039f0 0x7f3350003e10 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f3360009a90 tx=0x7f336001ca10 comp rx=0 tx=0).stop 2024-04-25T00:41:44.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f336a5be640 1 -- 172.21.15.29:0/1862615523 reap_dead start 2024-04-25T00:41:44.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f336c021640 1 -- 172.21.15.29:0/1862615523 shutdown_connections 2024-04-25T00:41:44.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f336c021640 1 -- 172.21.15.29:0/1862615523 >> 172.21.15.29:0/1862615523 conn(0x7f33640b1870 msgr2=0x7f33640b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:44.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f336c021640 1 -- 172.21.15.29:0/1862615523 shutdown_connections 2024-04-25T00:41:44.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.572+0000 7f336c021640 1 -- 172.21.15.29:0/1862615523 wait complete. 2024-04-25T00:41:44.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f336c021640 1 Processor -- start 2024-04-25T00:41:44.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f336c021640 1 -- start start 2024-04-25T00:41:44.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f336c021640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3364153910 0x7f3364072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f336c021640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336413c750 0x7f3364071610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f336c021640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3364071b50 0x7f3364156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f336c021640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3364167d80 con 0x7f3364153910 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f336c021640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3364167c00 con 0x7f336413c750 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f336c021640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3364167f00 con 0x7f3364071b50 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f3369dbd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3364153910 0x7f3364072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f3369dbd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3364153910 0x7f3364072980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53756/0 (socket says 172.21.15.29:53756) 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f3369dbd640 1 -- 172.21.15.29:0/4218158678 learned_addr learned my addr 172.21.15.29:0/4218158678 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f336a5be640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3364071b50 0x7f3364156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f3369dbd640 1 -- 172.21.15.29:0/4218158678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3364071b50 msgr2=0x7f3364156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f33695bc640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336413c750 0x7f3364071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f3369dbd640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3364071b50 0x7f3364156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f3369dbd640 1 -- 172.21.15.29:0/4218158678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336413c750 msgr2=0x7f3364071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f3369dbd640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336413c750 0x7f3364071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f3369dbd640 1 -- 172.21.15.29:0/4218158678 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336415c380 con 0x7f3364153910 2024-04-25T00:41:44.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f3369dbd640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3364153910 0x7f3364072980 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f336001cef0 tx=0x7f33600197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f335a7fc640 1 -- 172.21.15.29:0/4218158678 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33600a8c80 con 0x7f3364153910 2024-04-25T00:41:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f335a7fc640 1 -- 172.21.15.29:0/4218158678 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33600a8e40 con 0x7f3364153910 2024-04-25T00:41:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f336c021640 1 -- 172.21.15.29:0/4218158678 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336415c660 con 0x7f3364153910 2024-04-25T00:41:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f335a7fc640 1 -- 172.21.15.29:0/4218158678 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33600bc690 con 0x7f3364153910 2024-04-25T00:41:44.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f336c021640 1 -- 172.21.15.29:0/4218158678 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336415cc20 con 0x7f3364153910 2024-04-25T00:41:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f336c021640 1 -- 172.21.15.29:0/4218158678 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3364069270 con 0x7f3364153910 2024-04-25T00:41:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.576+0000 7f335a7fc640 1 -- 172.21.15.29:0/4218158678 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f33600a6040 con 0x7f3364153910 2024-04-25T00:41:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.580+0000 7f335a7fc640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3348068430 0x7f334806a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.580+0000 7f335a7fc640 1 -- 172.21.15.29:0/4218158678 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f336011ebb0 con 0x7f3364153910 2024-04-25T00:41:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.580+0000 7f33695bc640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3348068430 0x7f334806a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.580+0000 7f33695bc640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3348068430 0x7f334806a8f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3354001210 tx=0x7f3354009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:44.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.584+0000 7f335a7fc640 1 -- 172.21.15.29:0/4218158678 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f33600ac030 con 0x7f3364153910 2024-04-25T00:41:44.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.744+0000 7f336c021640 1 -- 172.21.15.29:0/4218158678 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3364071f30 con 0x7f3364153910 2024-04-25T00:41:44.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.748+0000 7f335a7fc640 1 -- 172.21.15.29:0/4218158678 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v168) ==== 100+0+11 (secure 0 0 0) 0x7f3360016720 con 0x7f3364153910 2024-04-25T00:41:44.748 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:41:44.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.748+0000 7f336c021640 1 -- 172.21.15.29:0/4218158678 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3348068430 msgr2=0x7f334806a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.748+0000 7f336c021640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3348068430 0x7f334806a8f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3354001210 tx=0x7f3354009040 comp rx=0 tx=0).stop 2024-04-25T00:41:44.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.748+0000 7f336c021640 1 -- 172.21.15.29:0/4218158678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3364153910 msgr2=0x7f3364072980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.748+0000 7f336c021640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3364153910 0x7f3364072980 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f336001cef0 tx=0x7f33600197b0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.748+0000 7f336c021640 1 -- 172.21.15.29:0/4218158678 shutdown_connections 2024-04-25T00:41:44.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.748+0000 7f336c021640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3348068430 0x7f334806a8f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.748+0000 7f336c021640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3364071b50 0x7f3364156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.748+0000 7f336c021640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336413c750 0x7f3364071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.748+0000 7f336c021640 1 --2- 172.21.15.29:0/4218158678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3364153910 0x7f3364072980 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.748+0000 7f336c021640 1 -- 172.21.15.29:0/4218158678 >> 172.21.15.29:0/4218158678 conn(0x7f33640b1870 msgr2=0x7f3364144a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:44.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.748+0000 7f336c021640 1 -- 172.21.15.29:0/4218158678 shutdown_connections 2024-04-25T00:41:44.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.748+0000 7f336c021640 1 -- 172.21.15.29:0/4218158678 wait complete. 2024-04-25T00:41:44.767 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:41:44.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.892+0000 7fdc27e2f640 1 Processor -- start 2024-04-25T00:41:44.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.892+0000 7fdc27e2f640 1 -- start start 2024-04-25T00:41:44.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.892+0000 7fdc27e2f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc2015e8d0 0x7fdc200cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.892+0000 7fdc27e2f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2015f3c0 0x7fdc2015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.892+0000 7fdc27e2f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc200b3720 con 0x7fdc200b4780 2024-04-25T00:41:44.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.892+0000 7fdc27e2f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdc200b42f0 con 0x7fdc2015e8d0 2024-04-25T00:41:44.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.892+0000 7fdc27e2f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc200b39c0 con 0x7fdc2015f3c0 2024-04-25T00:41:44.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.892+0000 7fdc263cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2015f3c0 0x7fdc2015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.892+0000 7fdc25bcb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc2015e8d0 0x7fdc200cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.892+0000 7fdc253ca640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdc200b4780 0x7fdc2015ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.29:59430/0 (socket says 172.21.15.29:59430) 2024-04-25T00:41:44.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.892+0000 7fdc253ca640 1 -- 172.21.15.29:0/3021175460 learned_addr learned my addr 172.21.15.29:0/3021175460 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:44.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc24bc9640 1 -- 172.21.15.29:0/3021175460 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2598169692 0 0) 0x7fdc200b3720 con 0x7fdc200b4780 2024-04-25T00:41:44.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc24bc9640 1 -- 172.21.15.29:0/3021175460 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc0c003530 con 0x7fdc200b4780 2024-04-25T00:41:44.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc25bcb640 1 -- 172.21.15.29:0/3021175460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2015f3c0 msgr2=0x7fdc2015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:44.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc25bcb640 1 --2- 172.21.15.29:0/3021175460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2015f3c0 0x7fdc2015c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc25bcb640 1 -- 172.21.15.29:0/3021175460 >> v1:172.21.15.157:6789/0 conn(0x7fdc200b4780 legacy=0x7fdc2015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc25bcb640 1 -- 172.21.15.29:0/3021175460 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc200b3ca0 con 0x7fdc2015e8d0 2024-04-25T00:41:44.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc25bcb640 1 --2- 172.21.15.29:0/3021175460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2015e8d0 0x7fdc200cc240 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009d90 tx=0x7fdc1c01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a897da4c70a800a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:44.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc24bc9640 1 -- 172.21.15.29:0/3021175460 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0b2070 con 0x7fdc2015e8d0 2024-04-25T00:41:44.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc24bc9640 1 -- 172.21.15.29:0/3021175460 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc1c0a5910 con 0x7fdc2015e8d0 2024-04-25T00:41:44.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc24bc9640 1 -- 172.21.15.29:0/3021175460 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0a5cc0 con 0x7fdc2015e8d0 2024-04-25T00:41:44.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc27e2f640 1 -- 172.21.15.29:0/3021175460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2015e8d0 msgr2=0x7fdc200cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc27e2f640 1 --2- 172.21.15.29:0/3021175460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2015e8d0 0x7fdc200cc240 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009d90 tx=0x7fdc1c01c940 comp rx=0 tx=0).stop 2024-04-25T00:41:44.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc27e2f640 1 -- 172.21.15.29:0/3021175460 shutdown_connections 2024-04-25T00:41:44.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc27e2f640 1 --2- 172.21.15.29:0/3021175460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2015f3c0 0x7fdc2015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc27e2f640 1 --2- 172.21.15.29:0/3021175460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2015e8d0 0x7fdc200cc240 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc27e2f640 1 -- 172.21.15.29:0/3021175460 >> 172.21.15.29:0/3021175460 conn(0x7fdc200b1870 msgr2=0x7fdc200b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:44.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc27e2f640 1 -- 172.21.15.29:0/3021175460 shutdown_connections 2024-04-25T00:41:44.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.896+0000 7fdc27e2f640 1 -- 172.21.15.29:0/3021175460 wait complete. 2024-04-25T00:41:44.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc27e2f640 1 Processor -- start 2024-04-25T00:41:44.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc27e2f640 1 -- start start 2024-04-25T00:41:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc27e2f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc200b4780 0x7fdc200cf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc27e2f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2015e8d0 0x7fdc2015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc27e2f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc2015f3c0 0x7fdc20135310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc27e2f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2015d6a0 con 0x7fdc2015f3c0 2024-04-25T00:41:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc27e2f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2015d520 con 0x7fdc200b4780 2024-04-25T00:41:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc27e2f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc20167c70 con 0x7fdc2015e8d0 2024-04-25T00:41:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc263cc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc2015f3c0 0x7fdc20135310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc263cc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc2015f3c0 0x7fdc20135310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53766/0 (socket says 172.21.15.29:53766) 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc263cc640 1 -- 172.21.15.29:0/1800893604 learned_addr learned my addr 172.21.15.29:0/1800893604 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc253ca640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2015e8d0 0x7fdc2015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc25bcb640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc200b4780 0x7fdc200cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc263cc640 1 -- 172.21.15.29:0/1800893604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2015e8d0 msgr2=0x7fdc2015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc263cc640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2015e8d0 0x7fdc2015a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc263cc640 1 -- 172.21.15.29:0/1800893604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc200b4780 msgr2=0x7fdc200cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc263cc640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc200b4780 0x7fdc200cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc263cc640 1 -- 172.21.15.29:0/1800893604 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc20135a60 con 0x7fdc2015f3c0 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc263cc640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc2015f3c0 0x7fdc20135310 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800e820 tx=0x7fdc1800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc167fc640 1 -- 172.21.15.29:0/1800893604 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18017070 con 0x7fdc2015f3c0 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc167fc640 1 -- 172.21.15.29:0/1800893604 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc1800c940 con 0x7fdc2015f3c0 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc167fc640 1 -- 172.21.15.29:0/1800893604 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800cd10 con 0x7fdc2015f3c0 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc27e2f640 1 -- 172.21.15.29:0/1800893604 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc20135d40 con 0x7fdc2015f3c0 2024-04-25T00:41:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc27e2f640 1 -- 172.21.15.29:0/1800893604 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc20136300 con 0x7fdc2015f3c0 2024-04-25T00:41:44.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.900+0000 7fdc27e2f640 1 -- 172.21.15.29:0/1800893604 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc20069270 con 0x7fdc2015f3c0 2024-04-25T00:41:44.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.904+0000 7fdc167fc640 1 -- 172.21.15.29:0/1800893604 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fdc1801a070 con 0x7fdc2015f3c0 2024-04-25T00:41:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.904+0000 7fdc167fc640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc04068400 0x7fdc0406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.904+0000 7fdc167fc640 1 -- 172.21.15.29:0/1800893604 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7fdc18084ad0 con 0x7fdc2015f3c0 2024-04-25T00:41:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.904+0000 7fdc25bcb640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc04068400 0x7fdc0406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.904+0000 7fdc25bcb640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc04068400 0x7fdc0406a8c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0af000 tx=0x7fdc1c0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:44.908+0000 7fdc167fc640 1 -- 172.21.15.29:0/1800893604 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdc18056760 con 0x7fdc2015f3c0 2024-04-25T00:41:45.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.072+0000 7fdc27e2f640 1 -- 172.21.15.29:0/1800893604 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdc2006e7b0 con 0x7fdc2015f3c0 2024-04-25T00:41:45.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.076+0000 7fdc167fc640 1 -- 172.21.15.29:0/1800893604 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v168) ==== 80+0+0 (secure 0 0 0) 0x7fdc1805aa60 con 0x7fdc2015f3c0 2024-04-25T00:41:45.075 INFO:teuthology.orchestra.run.smithi029.stderr:listed 0 entries 2024-04-25T00:41:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.076+0000 7fdc27e2f640 1 -- 172.21.15.29:0/1800893604 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc04068400 msgr2=0x7fdc0406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.076+0000 7fdc27e2f640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc04068400 0x7fdc0406a8c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0af000 tx=0x7fdc1c0a7070 comp rx=0 tx=0).stop 2024-04-25T00:41:45.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.076+0000 7fdc27e2f640 1 -- 172.21.15.29:0/1800893604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc2015f3c0 msgr2=0x7fdc20135310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.076+0000 7fdc27e2f640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc2015f3c0 0x7fdc20135310 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800e820 tx=0x7fdc1800ece0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.076+0000 7fdc27e2f640 1 -- 172.21.15.29:0/1800893604 shutdown_connections 2024-04-25T00:41:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.076+0000 7fdc27e2f640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fdc04068400 0x7fdc0406a8c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.076+0000 7fdc27e2f640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc2015f3c0 0x7fdc20135310 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.076+0000 7fdc27e2f640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2015e8d0 0x7fdc2015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.076+0000 7fdc27e2f640 1 --2- 172.21.15.29:0/1800893604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc200b4780 0x7fdc200cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.076+0000 7fdc27e2f640 1 -- 172.21.15.29:0/1800893604 >> 172.21.15.29:0/1800893604 conn(0x7fdc200b1870 msgr2=0x7fdc2006dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.076+0000 7fdc27e2f640 1 -- 172.21.15.29:0/1800893604 shutdown_connections 2024-04-25T00:41:45.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.076+0000 7fdc27e2f640 1 -- 172.21.15.29:0/1800893604 wait complete. 2024-04-25T00:41:45.093 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-25T00:41:45.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:45.093 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T00:41:45.106 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[35373]: fuse finished with error 0 and tester_r 0 2024-04-25T00:41:45.108 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:41:45.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:45.108 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:41:45.174 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:41:45.174 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:41:45.174 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:41:45.180 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:41:45.180 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:41:45.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:45.180 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:41:45.227 INFO:teuthology.orchestra.run.smithi029.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T00:41:45.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:41:45.228 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:41:45.228 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:41:45.228 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:41:45.234 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:41:45.234 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:41:45.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:45.234 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:41:45.282 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T00:41:45.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:41:45.283 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:41:45.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de86c1640 1 Processor -- start 2024-04-25T00:41:45.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de86c1640 1 -- start start 2024-04-25T00:41:45.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de86c1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1de015e8d0 0x7f1de00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de86c1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1de015f3c0 0x7f1de015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de86c1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1de00b3720 con 0x7f1de00b4780 2024-04-25T00:41:45.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de86c1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1de00b42f0 con 0x7f1de015e8d0 2024-04-25T00:41:45.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de86c1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1de00b39c0 con 0x7f1de015f3c0 2024-04-25T00:41:45.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de645d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1de015e8d0 0x7f1de00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de645d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1de015e8d0 0x7f1de00cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53770/0 (socket says 172.21.15.29:53770) 2024-04-25T00:41:45.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de645d640 1 -- 172.21.15.29:0/2488260902 learned_addr learned my addr 172.21.15.29:0/2488260902 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:45.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de6c5e640 1 --2- 172.21.15.29:0/2488260902 >> v2:172.21.15.157:3300/0 conn(0x7f1de015f3c0 0x7f1de015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de645d640 1 -- 172.21.15.29:0/2488260902 >> v2:172.21.15.157:3300/0 conn(0x7f1de015f3c0 msgr2=0x7f1de015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:45.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de645d640 1 --2- 172.21.15.29:0/2488260902 >> v2:172.21.15.157:3300/0 conn(0x7f1de015f3c0 0x7f1de015c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de645d640 1 -- 172.21.15.29:0/2488260902 >> v1:172.21.15.157:6789/0 conn(0x7f1de00b4780 legacy=0x7f1de015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:45.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.408+0000 7f1de645d640 1 -- 172.21.15.29:0/2488260902 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de00b3ca0 con 0x7f1de015e8d0 2024-04-25T00:41:45.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de645d640 1 --2- 172.21.15.29:0/2488260902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de015e8d0 0x7f1de00cc240 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f1de015f9a0 tx=0x7f1ddc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ace936eb534734 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:45.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de545b640 1 -- 172.21.15.29:0/2488260902 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc0b1070 con 0x7f1de015e8d0 2024-04-25T00:41:45.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de545b640 1 -- 172.21.15.29:0/2488260902 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ddc0ac410 con 0x7f1de015e8d0 2024-04-25T00:41:45.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de545b640 1 -- 172.21.15.29:0/2488260902 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc0ac7e0 con 0x7f1de015e8d0 2024-04-25T00:41:45.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 -- 172.21.15.29:0/2488260902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de015e8d0 msgr2=0x7f1de00cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:45.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 --2- 172.21.15.29:0/2488260902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de015e8d0 0x7f1de00cc240 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f1de015f9a0 tx=0x7f1ddc01c4e0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 -- 172.21.15.29:0/2488260902 shutdown_connections 2024-04-25T00:41:45.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 --2- 172.21.15.29:0/2488260902 >> v2:172.21.15.157:3300/0 conn(0x7f1de015f3c0 0x7f1de015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 --2- 172.21.15.29:0/2488260902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de015e8d0 0x7f1de00cc240 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 -- 172.21.15.29:0/2488260902 >> 172.21.15.29:0/2488260902 conn(0x7f1de00b1870 msgr2=0x7f1de00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:45.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 -- 172.21.15.29:0/2488260902 shutdown_connections 2024-04-25T00:41:45.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 -- 172.21.15.29:0/2488260902 wait complete. 2024-04-25T00:41:45.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 Processor -- start 2024-04-25T00:41:45.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 -- start start 2024-04-25T00:41:45.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1de00b4780 0x7f1de01faf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de015e8d0 0x7f1de01fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1de015f3c0 0x7f1de01f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1de015d6a0 con 0x7f1de015e8d0 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1de015d520 con 0x7f1de00b4780 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de86c1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1de0167c70 con 0x7f1de015f3c0 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de5c5c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de015e8d0 0x7f1de01fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de5c5c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de015e8d0 0x7f1de01fb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53774/0 (socket says 172.21.15.29:53774) 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de5c5c640 1 -- 172.21.15.29:0/2206286194 learned_addr learned my addr 172.21.15.29:0/2206286194 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de645d640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1de00b4780 0x7f1de01faf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de5c5c640 1 -- 172.21.15.29:0/2206286194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1de015f3c0 msgr2=0x7f1de01f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de5c5c640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1de015f3c0 0x7f1de01f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de5c5c640 1 -- 172.21.15.29:0/2206286194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1de00b4780 msgr2=0x7f1de01faf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de5c5c640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1de00b4780 0x7f1de01faf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de5c5c640 1 -- 172.21.15.29:0/2206286194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de013f020 con 0x7f1de015e8d0 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.412+0000 7f1de645d640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1de00b4780 0x7f1de01faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.416+0000 7f1de5c5c640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de015e8d0 0x7f1de01fb4d0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000c490 tx=0x7f1dd0001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:45.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.416+0000 7f1dd6ffd640 1 -- 172.21.15.29:0/2206286194 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0017070 con 0x7f1de015e8d0 2024-04-25T00:41:45.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.416+0000 7f1dd6ffd640 1 -- 172.21.15.29:0/2206286194 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dd0003600 con 0x7f1de015e8d0 2024-04-25T00:41:45.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.416+0000 7f1de86c1640 1 -- 172.21.15.29:0/2206286194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de013f300 con 0x7f1de015e8d0 2024-04-25T00:41:45.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.416+0000 7f1dd6ffd640 1 -- 172.21.15.29:0/2206286194 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0003a30 con 0x7f1de015e8d0 2024-04-25T00:41:45.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.416+0000 7f1de86c1640 1 -- 172.21.15.29:0/2206286194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de013fba0 con 0x7f1de015e8d0 2024-04-25T00:41:45.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.416+0000 7f1de86c1640 1 -- 172.21.15.29:0/2206286194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de013fda0 con 0x7f1de015e8d0 2024-04-25T00:41:45.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.416+0000 7f1dd6ffd640 1 -- 172.21.15.29:0/2206286194 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f1dd001a070 con 0x7f1de015e8d0 2024-04-25T00:41:45.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.416+0000 7f1dd6ffd640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1dc4068400 0x7f1dc406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.416+0000 7f1dd6ffd640 1 -- 172.21.15.29:0/2206286194 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f1dd0084180 con 0x7f1de015e8d0 2024-04-25T00:41:45.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.416+0000 7f1de645d640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1dc4068400 0x7f1dc406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.416+0000 7f1de645d640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1dc4068400 0x7f1dc406a8c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0ae000 tx=0x7f1ddc0a6040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:45.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.420+0000 7f1dd6ffd640 1 -- 172.21.15.29:0/2206286194 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1de013fda0 con 0x7f1de015e8d0 2024-04-25T00:41:45.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.608+0000 7f1de86c1640 1 -- 172.21.15.29:0/2206286194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1de0069270 con 0x7f1de015e8d0 2024-04-25T00:41:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.608+0000 7f1dd6ffd640 1 -- 172.21.15.29:0/2206286194 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5195 (secure 0 0 0) 0x7f1de0069270 con 0x7f1de015e8d0 2024-04-25T00:41:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:45.611 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":113,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7427,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/929849226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":929849226},{"type":"v1","addr":"172.21.15.157:6834","nonce":929849226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1459365951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1459365951},{"type":"v1","addr":"172.21.15.29:6833","nonce":1459365951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1011867355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1011867355},{"type":"v1","addr":"172.21.15.157:6835","nonce":1011867355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:41:20.082895+0000","modified":"2024-04-25T00:41:23.093215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10578":{"gid":10578,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/85311029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":85311029},{"type":"v1","addr":"172.21.15.29:6835","nonce":85311029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10578,"qdb_cluster":[10578]},"id":13}]} 2024-04-25T00:41:45.611 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 113 2024-04-25T00:41:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.612+0000 7f1de86c1640 1 -- 172.21.15.29:0/2206286194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1dc4068400 msgr2=0x7f1dc406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.612+0000 7f1de86c1640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1dc4068400 0x7f1dc406a8c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0ae000 tx=0x7f1ddc0a6040 comp rx=0 tx=0).stop 2024-04-25T00:41:45.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.612+0000 7f1de86c1640 1 -- 172.21.15.29:0/2206286194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de015e8d0 msgr2=0x7f1de01fb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.612+0000 7f1de86c1640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de015e8d0 0x7f1de01fb4d0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000c490 tx=0x7f1dd0001e80 comp rx=0 tx=0).stop 2024-04-25T00:41:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.612+0000 7f1de86c1640 1 -- 172.21.15.29:0/2206286194 shutdown_connections 2024-04-25T00:41:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.612+0000 7f1de86c1640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1dc4068400 0x7f1dc406a8c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.612+0000 7f1de86c1640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1de015f3c0 0x7f1de01f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.612+0000 7f1de86c1640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de015e8d0 0x7f1de01fb4d0 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.612+0000 7f1de86c1640 1 --2- 172.21.15.29:0/2206286194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1de00b4780 0x7f1de01faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.612+0000 7f1de86c1640 1 -- 172.21.15.29:0/2206286194 >> 172.21.15.29:0/2206286194 conn(0x7f1de00b1870 msgr2=0x7f1de00adc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.612+0000 7f1de86c1640 1 -- 172.21.15.29:0/2206286194 shutdown_connections 2024-04-25T00:41:45.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.612+0000 7f1de86c1640 1 -- 172.21.15.29:0/2206286194 wait complete. 2024-04-25T00:41:45.629 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:45.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31d10c640 1 Processor -- start 2024-04-25T00:41:45.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31d10c640 1 -- start start 2024-04-25T00:41:45.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31d10c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff31815e8d0 0x7ff3180cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31d10c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3180b4780 0x7ff31815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31d10c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3180b3720 con 0x7ff31815f1f0 2024-04-25T00:41:45.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31d10c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff3180b42f0 con 0x7ff3180b4780 2024-04-25T00:41:45.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31d10c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff31815d7e0 con 0x7ff31815e8d0 2024-04-25T00:41:45.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31759e640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff31815f1f0 0x7ff31815b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:60062/0 (socket says 172.21.15.29:60062) 2024-04-25T00:41:45.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31759e640 1 -- 172.21.15.29:0/4012468835 learned_addr learned my addr 172.21.15.29:0/4012468835 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:45.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff316d9d640 1 --2- 172.21.15.29:0/4012468835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff31815e8d0 0x7ff3180cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31659c640 1 --2- 172.21.15.29:0/4012468835 >> v2:172.21.15.157:3300/0 conn(0x7ff3180b4780 0x7ff31815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2497911243 0 0) 0x7ff3180b3720 con 0x7ff31815f1f0 2024-04-25T00:41:45.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff300003530 con 0x7ff31815f1f0 2024-04-25T00:41:45.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 377232269 0 0) 0x7ff300003530 con 0x7ff31815f1f0 2024-04-25T00:41:45.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff31815e8d0 msgr2=0x7ff3180cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:45.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 --2- 172.21.15.29:0/4012468835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff31815e8d0 0x7ff3180cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 >> v2:172.21.15.157:3300/0 conn(0x7ff3180b4780 msgr2=0x7ff31815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:45.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 --2- 172.21.15.29:0/4012468835 >> v2:172.21.15.157:3300/0 conn(0x7ff3180b4780 0x7ff31815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3180b39c0 con 0x7ff31815f1f0 2024-04-25T00:41:45.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ff310001a00 con 0x7ff31815f1f0 2024-04-25T00:41:45.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 >> v1:172.21.15.29:6789/0 conn(0x7ff31815f1f0 legacy=0x7ff31815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:45.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 --2- 172.21.15.29:0/4012468835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff31815e8d0 0x7ff3000039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 --2- 172.21.15.29:0/4012468835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff300003f40 0x7ff300004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 --2- 172.21.15.29:0/4012468835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff300004d50 0x7ff3000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff3180b42f0 con 0x7ff300004d50 2024-04-25T00:41:45.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff31815d7e0 con 0x7ff300003f40 2024-04-25T00:41:45.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff30000b9d0 con 0x7ff31815e8d0 2024-04-25T00:41:45.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31759e640 1 --2- 172.21.15.29:0/4012468835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff300004d50 0x7ff3000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff316d9d640 1 --2- 172.21.15.29:0/4012468835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff31815e8d0 0x7ff3000039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31659c640 1 --2- 172.21.15.29:0/4012468835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff300003f40 0x7ff300004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31759e640 1 -- 172.21.15.29:0/4012468835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff31815e8d0 msgr2=0x7ff3000039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:45.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31759e640 1 --2- 172.21.15.29:0/4012468835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff31815e8d0 0x7ff3000039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31759e640 1 -- 172.21.15.29:0/4012468835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff300003f40 msgr2=0x7ff300004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:45.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31759e640 1 --2- 172.21.15.29:0/4012468835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff300003f40 0x7ff300004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.740+0000 7ff31759e640 1 -- 172.21.15.29:0/4012468835 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff300003530 con 0x7ff300004d50 2024-04-25T00:41:45.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.744+0000 7ff31759e640 1 --2- 172.21.15.29:0/4012468835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff300004d50 0x7ff3000051d0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7ff3100013e0 tx=0x7ff31001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c9801b5ecfe115 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:45.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.744+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31000a4e0 con 0x7ff300004d50 2024-04-25T00:41:45.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.744+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3100072f0 con 0x7ff300004d50 2024-04-25T00:41:45.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.744+0000 7ff315d9b640 1 -- 172.21.15.29:0/4012468835 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3100076e0 con 0x7ff300004d50 2024-04-25T00:41:45.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.744+0000 7ff31d10c640 1 -- 172.21.15.29:0/4012468835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff300004d50 msgr2=0x7ff3000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:45.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.744+0000 7ff31d10c640 1 --2- 172.21.15.29:0/4012468835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff300004d50 0x7ff3000051d0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7ff3100013e0 tx=0x7ff31001d3a0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.744+0000 7ff31759e640 1 -- 172.21.15.29:0/4012468835 reap_dead start 2024-04-25T00:41:45.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.744+0000 7ff31d10c640 1 -- 172.21.15.29:0/4012468835 shutdown_connections 2024-04-25T00:41:45.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.744+0000 7ff31d10c640 1 -- 172.21.15.29:0/4012468835 >> 172.21.15.29:0/4012468835 conn(0x7ff3180b1870 msgr2=0x7ff3180b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:45.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.744+0000 7ff31d10c640 1 -- 172.21.15.29:0/4012468835 shutdown_connections 2024-04-25T00:41:45.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.744+0000 7ff31d10c640 1 -- 172.21.15.29:0/4012468835 wait complete. 2024-04-25T00:41:45.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.744+0000 7ff31d10c640 1 Processor -- start 2024-04-25T00:41:45.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.744+0000 7ff31d10c640 1 -- start start 2024-04-25T00:41:45.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff31d10c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff318159dd0 0x7ff3180726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff31d10c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff318072bf0 0x7ff3181f0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff31d10c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3181f0670 0x7ff318139cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff31d10c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff31806cf10 con 0x7ff318159dd0 2024-04-25T00:41:45.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff31d10c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff3180b3f80 con 0x7ff3181f0670 2024-04-25T00:41:45.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff31d10c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff31815f660 con 0x7ff318072bf0 2024-04-25T00:41:45.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff316d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff318159dd0 0x7ff3180726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff316d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff318159dd0 0x7ff3180726b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53782/0 (socket says 172.21.15.29:53782) 2024-04-25T00:41:45.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff316d9d640 1 -- 172.21.15.29:0/3437882198 learned_addr learned my addr 172.21.15.29:0/3437882198 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:45.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff316d9d640 1 -- 172.21.15.29:0/3437882198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff318072bf0 msgr2=0x7ff3181f0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:45.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff31659c640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff318072bf0 0x7ff3181f0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff31759e640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3181f0670 0x7ff318139cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff316d9d640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff318072bf0 0x7ff3181f0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff316d9d640 1 -- 172.21.15.29:0/3437882198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3181f0670 msgr2=0x7ff318139cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:45.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff316d9d640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3181f0670 0x7ff318139cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff316d9d640 1 -- 172.21.15.29:0/3437882198 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff318068a30 con 0x7ff318159dd0 2024-04-25T00:41:45.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff316d9d640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff318159dd0 0x7ff3180726b0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0115e0 tx=0x7ff30c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:45.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff2ff7fe640 1 -- 172.21.15.29:0/3437882198 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c015870 con 0x7ff318159dd0 2024-04-25T00:41:45.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff2ff7fe640 1 -- 172.21.15.29:0/3437882198 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30c015a10 con 0x7ff318159dd0 2024-04-25T00:41:45.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff2ff7fe640 1 -- 172.21.15.29:0/3437882198 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c015dc0 con 0x7ff318159dd0 2024-04-25T00:41:45.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff31d10c640 1 -- 172.21.15.29:0/3437882198 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff318067820 con 0x7ff318159dd0 2024-04-25T00:41:45.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff31d10c640 1 -- 172.21.15.29:0/3437882198 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff318071c30 con 0x7ff318159dd0 2024-04-25T00:41:45.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff31d10c640 1 -- 172.21.15.29:0/3437882198 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff318069270 con 0x7ff318159dd0 2024-04-25T00:41:45.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff2ff7fe640 1 -- 172.21.15.29:0/3437882198 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7ff30c00f480 con 0x7ff318159dd0 2024-04-25T00:41:45.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff2ff7fe640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff304068400 0x7ff30406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:45.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff2ff7fe640 1 -- 172.21.15.29:0/3437882198 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7ff30c084e10 con 0x7ff318159dd0 2024-04-25T00:41:45.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff31659c640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff304068400 0x7ff30406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:45.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.748+0000 7ff31659c640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff304068400 0x7ff30406a8c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff3181444e0 tx=0x7ff308008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:45.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.752+0000 7ff2ff7fe640 1 -- 172.21.15.29:0/3437882198 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff30c019030 con 0x7ff318159dd0 2024-04-25T00:41:45.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.916+0000 7ff31d10c640 1 -- 172.21.15.29:0/3437882198 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff31806e7b0 con 0x7ff318159dd0 2024-04-25T00:41:45.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.916+0000 7ff2ff7fe640 1 -- 172.21.15.29:0/3437882198 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) ==== 74+0+16635 (secure 0 0 0) 0x7ff30c057150 con 0x7ff318159dd0 2024-04-25T00:41:45.918 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:45.918 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":168,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:41:21.712101+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:41:18.370636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:41:19.067830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:45.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.920+0000 7ff31d10c640 1 -- 172.21.15.29:0/3437882198 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff304068400 msgr2=0x7ff30406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:45.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.920+0000 7ff31d10c640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff304068400 0x7ff30406a8c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff3181444e0 tx=0x7ff308008040 comp rx=0 tx=0).stop 2024-04-25T00:41:45.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.920+0000 7ff31d10c640 1 -- 172.21.15.29:0/3437882198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff318159dd0 msgr2=0x7ff3180726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:45.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.920+0000 7ff31d10c640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff318159dd0 0x7ff3180726b0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0115e0 tx=0x7ff30c011aa0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.920+0000 7ff31d10c640 1 -- 172.21.15.29:0/3437882198 shutdown_connections 2024-04-25T00:41:45.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.920+0000 7ff31d10c640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff304068400 0x7ff30406a8c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.920+0000 7ff31d10c640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3181f0670 0x7ff318139cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.920+0000 7ff31d10c640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff318072bf0 0x7ff3181f0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.920+0000 7ff31d10c640 1 --2- 172.21.15.29:0/3437882198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff318159dd0 0x7ff3180726b0 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:45.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.920+0000 7ff31d10c640 1 -- 172.21.15.29:0/3437882198 >> 172.21.15.29:0/3437882198 conn(0x7ff3180b1870 msgr2=0x7ff3180cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:45.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.920+0000 7ff31d10c640 1 -- 172.21.15.29:0/3437882198 shutdown_connections 2024-04-25T00:41:45.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:45.920+0000 7ff31d10c640 1 -- 172.21.15.29:0/3437882198 wait complete. 2024-04-25T00:41:45.937 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:31:23.504264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:41:18.370636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:41:19.067830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:41:45.938 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:41:46.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.068+0000 7fde56b0c640 1 Processor -- start 2024-04-25T00:41:46.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.068+0000 7fde56b0c640 1 -- start start 2024-04-25T00:41:46.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.068+0000 7fde56b0c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fde500cf690 0x7fde500d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.068+0000 7fde56b0c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde500d2ea0 con 0x7fde500d2a90 2024-04-25T00:41:46.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.068+0000 7fde56b0c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde500cfc10 con 0x7fde5015e200 2024-04-25T00:41:46.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.068+0000 7fde56b0c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fde500b38d0 con 0x7fde500cf690 2024-04-25T00:41:46.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.068+0000 7fde4ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fde500cf690 0x7fde500d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.068+0000 7fde550a9640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fde500d2a90 0x7fde500b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:60064/0 (socket says 172.21.15.29:60064) 2024-04-25T00:41:46.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.068+0000 7fde550a9640 1 -- 172.21.15.29:0/4193294572 learned_addr learned my addr 172.21.15.29:0/4193294572 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:46.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.068+0000 7fde4ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fde500cf690 0x7fde500d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53786/0 (socket says 172.21.15.29:53786) 2024-04-25T00:41:46.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.068+0000 7fde4ffff640 1 -- 172.21.15.29:0/4193294572 >> v1:172.21.15.157:6789/0 conn(0x7fde5015e200 legacy=0x7fde500d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.068+0000 7fde4ffff640 1 -- 172.21.15.29:0/4193294572 >> v1:172.21.15.29:6789/0 conn(0x7fde500d2a90 legacy=0x7fde500b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.068+0000 7fde4ffff640 1 -- 172.21.15.29:0/4193294572 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde500cfef0 con 0x7fde500cf690 2024-04-25T00:41:46.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.072+0000 7fde4ffff640 1 --2- 172.21.15.29:0/4193294572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde500cf690 0x7fde500d2550 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0097c0 tx=0x7fde3c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbac264e1760b003 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:46.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.072+0000 7fde4f7fe640 1 -- 172.21.15.29:0/4193294572 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c0b1070 con 0x7fde500cf690 2024-04-25T00:41:46.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.072+0000 7fde4f7fe640 1 -- 172.21.15.29:0/4193294572 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3c0ac5f0 con 0x7fde500cf690 2024-04-25T00:41:46.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.072+0000 7fde4f7fe640 1 -- 172.21.15.29:0/4193294572 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c0ac9c0 con 0x7fde500cf690 2024-04-25T00:41:46.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.072+0000 7fde56b0c640 1 -- 172.21.15.29:0/4193294572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde500cf690 msgr2=0x7fde500d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.072+0000 7fde56b0c640 1 --2- 172.21.15.29:0/4193294572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde500cf690 0x7fde500d2550 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0097c0 tx=0x7fde3c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.072+0000 7fde56b0c640 1 -- 172.21.15.29:0/4193294572 shutdown_connections 2024-04-25T00:41:46.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.072+0000 7fde56b0c640 1 --2- 172.21.15.29:0/4193294572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde500cf690 0x7fde500d2550 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.072+0000 7fde56b0c640 1 -- 172.21.15.29:0/4193294572 >> 172.21.15.29:0/4193294572 conn(0x7fde5009d280 msgr2=0x7fde500d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:46.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.072+0000 7fde56b0c640 1 -- 172.21.15.29:0/4193294572 shutdown_connections 2024-04-25T00:41:46.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.072+0000 7fde56b0c640 1 -- 172.21.15.29:0/4193294572 wait complete. 2024-04-25T00:41:46.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde56b0c640 1 Processor -- start 2024-04-25T00:41:46.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde56b0c640 1 -- start start 2024-04-25T00:41:46.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde56b0c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde500cf690 0x7fde50152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde56b0c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde500d2a90 0x7fde500b8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde56b0c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde5015e200 0x7fde500b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde56b0c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde500b53c0 con 0x7fde5015e200 2024-04-25T00:41:46.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde56b0c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fde500b5240 con 0x7fde500d2a90 2024-04-25T00:41:46.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde56b0c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fde500b5540 con 0x7fde500cf690 2024-04-25T00:41:46.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde550a9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde5015e200 0x7fde500b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde550a9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde5015e200 0x7fde500b8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53796/0 (socket says 172.21.15.29:53796) 2024-04-25T00:41:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde550a9640 1 -- 172.21.15.29:0/1815711829 learned_addr learned my addr 172.21.15.29:0/1815711829 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde4ffff640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde500d2a90 0x7fde500b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde548a8640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde500cf690 0x7fde50152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde550a9640 1 -- 172.21.15.29:0/1815711829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde500cf690 msgr2=0x7fde50152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde550a9640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde500cf690 0x7fde50152b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde550a9640 1 -- 172.21.15.29:0/1815711829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde500d2a90 msgr2=0x7fde500b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde550a9640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde500d2a90 0x7fde500b8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde550a9640 1 -- 172.21.15.29:0/1815711829 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde500b6fe0 con 0x7fde5015e200 2024-04-25T00:41:46.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde4ffff640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde500d2a90 0x7fde500b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:46.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde550a9640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde5015e200 0x7fde500b8c80 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7fde4800ca80 tx=0x7fde4800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:46.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde4d7fa640 1 -- 172.21.15.29:0/1815711829 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48017070 con 0x7fde5015e200 2024-04-25T00:41:46.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde4d7fa640 1 -- 172.21.15.29:0/1815711829 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4800ac30 con 0x7fde5015e200 2024-04-25T00:41:46.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde56b0c640 1 -- 172.21.15.29:0/1815711829 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde500b72c0 con 0x7fde5015e200 2024-04-25T00:41:46.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde4d7fa640 1 -- 172.21.15.29:0/1815711829 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48012600 con 0x7fde5015e200 2024-04-25T00:41:46.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde56b0c640 1 -- 172.21.15.29:0/1815711829 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde500b7880 con 0x7fde5015e200 2024-04-25T00:41:46.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde56b0c640 1 -- 172.21.15.29:0/1815711829 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde50069270 con 0x7fde5015e200 2024-04-25T00:41:46.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.076+0000 7fde4d7fa640 1 -- 172.21.15.29:0/1815711829 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fde4801a070 con 0x7fde5015e200 2024-04-25T00:41:46.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.080+0000 7fde4d7fa640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fde34068400 0x7fde3406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.080+0000 7fde4d7fa640 1 -- 172.21.15.29:0/1815711829 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7fde48084490 con 0x7fde5015e200 2024-04-25T00:41:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.080+0000 7fde548a8640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fde34068400 0x7fde3406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.080+0000 7fde548a8640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fde34068400 0x7fde3406a8c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fde44007fd0 tx=0x7fde44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.084+0000 7fde4d7fa640 1 -- 172.21.15.29:0/1815711829 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fde500d2ea0 con 0x7fde5015e200 2024-04-25T00:41:46.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.244+0000 7fde56b0c640 1 -- 172.21.15.29:0/1815711829 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fde50156270 con 0x7fde5015e200 2024-04-25T00:41:46.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.244+0000 7fde4d7fa640 1 -- 172.21.15.29:0/1815711829 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v168) ==== 93+0+10 (secure 0 0 0) 0x7fde4800ae20 con 0x7fde5015e200 2024-04-25T00:41:46.246 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-04-25T00:41:46.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.248+0000 7fde56b0c640 1 -- 172.21.15.29:0/1815711829 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fde34068400 msgr2=0x7fde3406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.248+0000 7fde56b0c640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fde34068400 0x7fde3406a8c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fde44007fd0 tx=0x7fde44009040 comp rx=0 tx=0).stop 2024-04-25T00:41:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.248+0000 7fde56b0c640 1 -- 172.21.15.29:0/1815711829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde5015e200 msgr2=0x7fde500b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.248+0000 7fde56b0c640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde5015e200 0x7fde500b8c80 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7fde4800ca80 tx=0x7fde4800cf40 comp rx=0 tx=0).stop 2024-04-25T00:41:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.248+0000 7fde56b0c640 1 -- 172.21.15.29:0/1815711829 shutdown_connections 2024-04-25T00:41:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.248+0000 7fde56b0c640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fde34068400 0x7fde3406a8c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.248+0000 7fde56b0c640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde5015e200 0x7fde500b8c80 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.248+0000 7fde56b0c640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde500d2a90 0x7fde500b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.248+0000 7fde56b0c640 1 --2- 172.21.15.29:0/1815711829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde500cf690 0x7fde50152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.248+0000 7fde56b0c640 1 -- 172.21.15.29:0/1815711829 >> 172.21.15.29:0/1815711829 conn(0x7fde5009d280 msgr2=0x7fde5006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.248+0000 7fde56b0c640 1 -- 172.21.15.29:0/1815711829 shutdown_connections 2024-04-25T00:41:46.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.248+0000 7fde56b0c640 1 -- 172.21.15.29:0/1815711829 wait complete. 2024-04-25T00:41:46.266 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:41:46.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cc18dc640 1 Processor -- start 2024-04-25T00:41:46.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cc18dc640 1 -- start start 2024-04-25T00:41:46.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cc18dc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15bd60 0x7f2cbc06b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cc18dc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2cbc06dd10 0x7f2cbc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cc18dc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cbc06e1a0 con 0x7f2cbc0cb330 2024-04-25T00:41:46.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cc18dc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2cbc06edb0 con 0x7f2cbc06dd10 2024-04-25T00:41:46.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cc18dc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2cbc06e480 con 0x7f2cbc15bd60 2024-04-25T00:41:46.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cba7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2cbc06dd10 0x7f2cbc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cba7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2cbc06dd10 0x7f2cbc06b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53810/0 (socket says 172.21.15.29:53810) 2024-04-25T00:41:46.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cba7fc640 1 -- 172.21.15.29:0/3383484315 learned_addr learned my addr 172.21.15.29:0/3383484315 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:46.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cbaffd640 1 --2- 172.21.15.29:0/3383484315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15bd60 0x7f2cbc06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cba7fc640 1 -- 172.21.15.29:0/3383484315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15bd60 msgr2=0x7f2cbc06b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:46.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cba7fc640 1 --2- 172.21.15.29:0/3383484315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15bd60 0x7f2cbc06b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cba7fc640 1 -- 172.21.15.29:0/3383484315 >> v1:172.21.15.157:6789/0 conn(0x7f2cbc0cb330 legacy=0x7f2cbc15f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.396+0000 7f2cba7fc640 1 -- 172.21.15.29:0/3383484315 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc06e760 con 0x7f2cbc06dd10 2024-04-25T00:41:46.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cba7fc640 1 --2- 172.21.15.29:0/3383484315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cbc06dd10 0x7f2cbc06b750 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009870 tx=0x7f2ca401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2900db7f4ced43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:46.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cb9ffb640 1 -- 172.21.15.29:0/3383484315 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca40a9610 con 0x7f2cbc06dd10 2024-04-25T00:41:46.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cb9ffb640 1 -- 172.21.15.29:0/3383484315 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca40a97b0 con 0x7f2cbc06dd10 2024-04-25T00:41:46.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cb9ffb640 1 -- 172.21.15.29:0/3383484315 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca40a9b80 con 0x7f2cbc06dd10 2024-04-25T00:41:46.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 -- 172.21.15.29:0/3383484315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cbc06dd10 msgr2=0x7f2cbc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 --2- 172.21.15.29:0/3383484315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cbc06dd10 0x7f2cbc06b750 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009870 tx=0x7f2ca401c530 comp rx=0 tx=0).stop 2024-04-25T00:41:46.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 -- 172.21.15.29:0/3383484315 shutdown_connections 2024-04-25T00:41:46.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 --2- 172.21.15.29:0/3383484315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cbc06dd10 0x7f2cbc06b750 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 --2- 172.21.15.29:0/3383484315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15bd60 0x7f2cbc06b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 -- 172.21.15.29:0/3383484315 >> 172.21.15.29:0/3383484315 conn(0x7f2cbc06c320 msgr2=0x7f2cbc06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:46.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 -- 172.21.15.29:0/3383484315 shutdown_connections 2024-04-25T00:41:46.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 -- 172.21.15.29:0/3383484315 wait complete. 2024-04-25T00:41:46.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 Processor -- start 2024-04-25T00:41:46.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 -- start start 2024-04-25T00:41:46.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cbc0cb330 0x7f2cbc150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc15bd60 0x7f2cbc152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15c400 0x7f2cbc153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc168f30 con 0x7f2cbc0cb330 2024-04-25T00:41:46.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc0cdea0 con 0x7f2cbc15bd60 2024-04-25T00:41:46.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cc18dc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2cbc1690b0 con 0x7f2cbc15c400 2024-04-25T00:41:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cbaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cbc0cb330 0x7f2cbc150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cbaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cbc0cb330 0x7f2cbc150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53818/0 (socket says 172.21.15.29:53818) 2024-04-25T00:41:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cbaffd640 1 -- 172.21.15.29:0/2002686822 learned_addr learned my addr 172.21.15.29:0/2002686822 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cba7fc640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc15bd60 0x7f2cbc152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cbb7fe640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15c400 0x7f2cbc153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cbaffd640 1 -- 172.21.15.29:0/2002686822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15c400 msgr2=0x7f2cbc153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cbaffd640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15c400 0x7f2cbc153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cbaffd640 1 -- 172.21.15.29:0/2002686822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc15bd60 msgr2=0x7f2cbc152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cbaffd640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc15bd60 0x7f2cbc152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cbaffd640 1 -- 172.21.15.29:0/2002686822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc15c8c0 con 0x7f2cbc0cb330 2024-04-25T00:41:46.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cbb7fe640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15c400 0x7f2cbc153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:46.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cbaffd640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cbc0cb330 0x7f2cbc150d40 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400d5f0 tx=0x7f2cb400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:46.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cb3fff640 1 -- 172.21.15.29:0/2002686822 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40125f0 con 0x7f2cbc0cb330 2024-04-25T00:41:46.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cb3fff640 1 -- 172.21.15.29:0/2002686822 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb4019070 con 0x7f2cbc0cb330 2024-04-25T00:41:46.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.400+0000 7f2cb3fff640 1 -- 172.21.15.29:0/2002686822 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40129b0 con 0x7f2cbc0cb330 2024-04-25T00:41:46.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.404+0000 7f2cc18dc640 1 -- 172.21.15.29:0/2002686822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc15cba0 con 0x7f2cbc0cb330 2024-04-25T00:41:46.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.404+0000 7f2cc18dc640 1 -- 172.21.15.29:0/2002686822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc15d160 con 0x7f2cbc0cb330 2024-04-25T00:41:46.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.404+0000 7f2cc18dc640 1 -- 172.21.15.29:0/2002686822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cbc069270 con 0x7f2cbc0cb330 2024-04-25T00:41:46.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.404+0000 7f2cb3fff640 1 -- 172.21.15.29:0/2002686822 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f2cb4012b50 con 0x7f2cbc0cb330 2024-04-25T00:41:46.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.404+0000 7f2cb3fff640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2ca0068400 0x7f2ca006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.404+0000 7f2cb3fff640 1 -- 172.21.15.29:0/2002686822 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f2cb4084a50 con 0x7f2cbc0cb330 2024-04-25T00:41:46.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.404+0000 7f2cba7fc640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2ca0068400 0x7f2ca006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.404+0000 7f2cba7fc640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2ca0068400 0x7f2ca006a8c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40ae000 tx=0x7f2ca40b3040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:46.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.408+0000 7f2cb3fff640 1 -- 172.21.15.29:0/2002686822 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2cb4056fa0 con 0x7f2cbc0cb330 2024-04-25T00:41:46.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.572+0000 7f2cc18dc640 1 -- 172.21.15.29:0/2002686822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2cbc068380 con 0x7f2cbc0cb330 2024-04-25T00:41:46.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.572+0000 7f2cb3fff640 1 -- 172.21.15.29:0/2002686822 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v168) ==== 104+0+11 (secure 0 0 0) 0x7f2cb40568a0 con 0x7f2cbc0cb330 2024-04-25T00:41:46.575 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-04-25T00:41:46.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.576+0000 7f2cc18dc640 1 -- 172.21.15.29:0/2002686822 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2ca0068400 msgr2=0x7f2ca006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.576+0000 7f2cc18dc640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2ca0068400 0x7f2ca006a8c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40ae000 tx=0x7f2ca40b3040 comp rx=0 tx=0).stop 2024-04-25T00:41:46.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.576+0000 7f2cc18dc640 1 -- 172.21.15.29:0/2002686822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cbc0cb330 msgr2=0x7f2cbc150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.576+0000 7f2cc18dc640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cbc0cb330 0x7f2cbc150d40 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400d5f0 tx=0x7f2cb400dab0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.576+0000 7f2cc18dc640 1 -- 172.21.15.29:0/2002686822 shutdown_connections 2024-04-25T00:41:46.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.576+0000 7f2cc18dc640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2ca0068400 0x7f2ca006a8c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.576+0000 7f2cc18dc640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cbc15c400 0x7f2cbc153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.576+0000 7f2cc18dc640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cbc15bd60 0x7f2cbc152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.576+0000 7f2cc18dc640 1 --2- 172.21.15.29:0/2002686822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cbc0cb330 0x7f2cbc150d40 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.576+0000 7f2cc18dc640 1 -- 172.21.15.29:0/2002686822 >> 172.21.15.29:0/2002686822 conn(0x7f2cbc06c320 msgr2=0x7f2cbc0ce050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:46.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.576+0000 7f2cc18dc640 1 -- 172.21.15.29:0/2002686822 shutdown_connections 2024-04-25T00:41:46.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.576+0000 7f2cc18dc640 1 -- 172.21.15.29:0/2002686822 wait complete. 2024-04-25T00:41:46.594 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:41:46.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec2b570640 1 Processor -- start 2024-04-25T00:41:46.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec2b570640 1 -- start start 2024-04-25T00:41:46.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec2b570640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fec240ce270 0x7fec240cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec2b570640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec24072120 0x7fec240add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec2b570640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec240cd1d0 con 0x7fec2406b210 2024-04-25T00:41:46.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec2b570640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fec240cd4b0 con 0x7fec240ce270 2024-04-25T00:41:46.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec2b570640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fec240cd790 con 0x7fec24072120 2024-04-25T00:41:46.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec2930c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fec2406b210 0x7fec2406b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:60066/0 (socket says 172.21.15.29:60066) 2024-04-25T00:41:46.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec2930c640 1 -- 172.21.15.29:0/3617490759 learned_addr learned my addr 172.21.15.29:0/3617490759 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:46.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2118844776 0 0) 0x7fec240cd1d0 con 0x7fec2406b210 2024-04-25T00:41:46.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec0c003530 con 0x7fec2406b210 2024-04-25T00:41:46.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec29b0d640 1 --2- 172.21.15.29:0/3617490759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec24072120 0x7fec240add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec28b0b640 1 --2- 172.21.15.29:0/3617490759 >> v2:172.21.15.157:3300/0 conn(0x7fec240ce270 0x7fec240cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 541788661 0 0) 0x7fec0c003530 con 0x7fec2406b210 2024-04-25T00:41:46.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec24072120 msgr2=0x7fec240add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:46.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 --2- 172.21.15.29:0/3617490759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec24072120 0x7fec240add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 >> v2:172.21.15.157:3300/0 conn(0x7fec240ce270 msgr2=0x7fec240cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:46.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 --2- 172.21.15.29:0/3617490759 >> v2:172.21.15.157:3300/0 conn(0x7fec240ce270 0x7fec240cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec240cda70 con 0x7fec2406b210 2024-04-25T00:41:46.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fec1c001680 con 0x7fec2406b210 2024-04-25T00:41:46.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 >> v1:172.21.15.29:6789/0 conn(0x7fec2406b210 legacy=0x7fec2406b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 --2- 172.21.15.29:0/3617490759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec24072120 0x7fec0c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 --2- 172.21.15.29:0/3617490759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec0c003f40 0x7fec0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 --2- 172.21.15.29:0/3617490759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0c004d50 0x7fec0c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fec240cd4b0 con 0x7fec0c003f40 2024-04-25T00:41:46.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec240cd790 con 0x7fec0c004d50 2024-04-25T00:41:46.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fec0c00b9d0 con 0x7fec24072120 2024-04-25T00:41:46.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec28b0b640 1 --2- 172.21.15.29:0/3617490759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec0c003f40 0x7fec0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec2930c640 1 --2- 172.21.15.29:0/3617490759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec24072120 0x7fec0c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec29b0d640 1 --2- 172.21.15.29:0/3617490759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0c004d50 0x7fec0c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec28b0b640 1 -- 172.21.15.29:0/3617490759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec24072120 msgr2=0x7fec0c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:46.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec28b0b640 1 --2- 172.21.15.29:0/3617490759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec24072120 0x7fec0c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec28b0b640 1 -- 172.21.15.29:0/3617490759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0c004d50 msgr2=0x7fec0c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:46.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec28b0b640 1 --2- 172.21.15.29:0/3617490759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0c004d50 0x7fec0c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec28b0b640 1 -- 172.21.15.29:0/3617490759 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0c003530 con 0x7fec0c003f40 2024-04-25T00:41:46.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.716+0000 7fec29b0d640 1 --2- 172.21.15.29:0/3617490759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0c004d50 0x7fec0c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:46.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.720+0000 7fec28b0b640 1 --2- 172.21.15.29:0/3617490759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec0c003f40 0x7fec0c004810 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7fec18009f90 tx=0x7fec1801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0f9b4502f7e277 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:46.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.720+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec180b2070 con 0x7fec0c003f40 2024-04-25T00:41:46.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.720+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec180a5c90 con 0x7fec0c003f40 2024-04-25T00:41:46.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.720+0000 7fec23fff640 1 -- 172.21.15.29:0/3617490759 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18009c10 con 0x7fec0c003f40 2024-04-25T00:41:46.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.720+0000 7fec2b570640 1 -- 172.21.15.29:0/3617490759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec0c003f40 msgr2=0x7fec0c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.720+0000 7fec2b570640 1 --2- 172.21.15.29:0/3617490759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec0c003f40 0x7fec0c004810 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7fec18009f90 tx=0x7fec1801cc20 comp rx=0 tx=0).stop 2024-04-25T00:41:46.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.720+0000 7fec29b0d640 1 -- 172.21.15.29:0/3617490759 reap_dead start 2024-04-25T00:41:46.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.720+0000 7fec2b570640 1 -- 172.21.15.29:0/3617490759 shutdown_connections 2024-04-25T00:41:46.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.720+0000 7fec2b570640 1 -- 172.21.15.29:0/3617490759 >> 172.21.15.29:0/3617490759 conn(0x7fec240b2920 msgr2=0x7fec240b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.720+0000 7fec2b570640 1 -- 172.21.15.29:0/3617490759 shutdown_connections 2024-04-25T00:41:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.720+0000 7fec2b570640 1 -- 172.21.15.29:0/3617490759 wait complete. 2024-04-25T00:41:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2b570640 1 Processor -- start 2024-04-25T00:41:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2b570640 1 -- start start 2024-04-25T00:41:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2b570640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec0c003f40 0x7fec241e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2b570640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec241e5dd0 0x7fec241e4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2b570640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec240add70 0x7fec240ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2b570640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fec240ccdc0 con 0x7fec0c003f40 2024-04-25T00:41:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2b570640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec24145b80 con 0x7fec241e5dd0 2024-04-25T00:41:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2b570640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fec240ccf40 con 0x7fec240add70 2024-04-25T00:41:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2930c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec0c003f40 0x7fec241e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2930c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec0c003f40 0x7fec241e3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53850/0 (socket says 172.21.15.29:53850) 2024-04-25T00:41:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2930c640 1 -- 172.21.15.29:0/863428322 learned_addr learned my addr 172.21.15.29:0/863428322 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec28b0b640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec241e5dd0 0x7fec241e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec29b0d640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec240add70 0x7fec240ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2930c640 1 -- 172.21.15.29:0/863428322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec240add70 msgr2=0x7fec240ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2930c640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec240add70 0x7fec240ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2930c640 1 -- 172.21.15.29:0/863428322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec241e5dd0 msgr2=0x7fec241e4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2930c640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec241e5dd0 0x7fec241e4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2930c640 1 -- 172.21.15.29:0/863428322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec24140fc0 con 0x7fec0c003f40 2024-04-25T00:41:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2930c640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec0c003f40 0x7fec241e3fc0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7fec1c011890 tx=0x7fec1c011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec21ffb640 1 -- 172.21.15.29:0/863428322 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c010040 con 0x7fec0c003f40 2024-04-25T00:41:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec21ffb640 1 -- 172.21.15.29:0/863428322 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1c01a070 con 0x7fec0c003f40 2024-04-25T00:41:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec21ffb640 1 -- 172.21.15.29:0/863428322 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c00fdc0 con 0x7fec0c003f40 2024-04-25T00:41:46.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2b570640 1 -- 172.21.15.29:0/863428322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec241e4d30 con 0x7fec0c003f40 2024-04-25T00:41:46.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2b570640 1 -- 172.21.15.29:0/863428322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec241e52f0 con 0x7fec0c003f40 2024-04-25T00:41:46.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.724+0000 7fec2b570640 1 -- 172.21.15.29:0/863428322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec24069270 con 0x7fec0c003f40 2024-04-25T00:41:46.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.728+0000 7fec21ffb640 1 -- 172.21.15.29:0/863428322 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fec1c01f460 con 0x7fec0c003f40 2024-04-25T00:41:46.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.728+0000 7fec21ffb640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fec10068400 0x7fec1006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:46.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.728+0000 7fec21ffb640 1 -- 172.21.15.29:0/863428322 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7fec1c087710 con 0x7fec0c003f40 2024-04-25T00:41:46.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.728+0000 7fec28b0b640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fec10068400 0x7fec1006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:46.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.728+0000 7fec28b0b640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fec10068400 0x7fec1006a8c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fec180af000 tx=0x7fec180a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:46.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.732+0000 7fec21ffb640 1 -- 172.21.15.29:0/863428322 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fec1c0016c0 con 0x7fec0c003f40 2024-04-25T00:41:46.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.892+0000 7fec2b570640 1 -- 172.21.15.29:0/863428322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fec240b1510 con 0x7fec0c003f40 2024-04-25T00:41:46.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.892+0000 7fec21ffb640 1 -- 172.21.15.29:0/863428322 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v168) ==== 100+0+11 (secure 0 0 0) 0x7fec1c059d00 con 0x7fec0c003f40 2024-04-25T00:41:46.898 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-04-25T00:41:46.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.896+0000 7fec2b570640 1 -- 172.21.15.29:0/863428322 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fec10068400 msgr2=0x7fec1006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.896+0000 7fec2b570640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fec10068400 0x7fec1006a8c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fec180af000 tx=0x7fec180a7040 comp rx=0 tx=0).stop 2024-04-25T00:41:46.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.896+0000 7fec2b570640 1 -- 172.21.15.29:0/863428322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec0c003f40 msgr2=0x7fec241e3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:46.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.896+0000 7fec2b570640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec0c003f40 0x7fec241e3fc0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7fec1c011890 tx=0x7fec1c011d50 comp rx=0 tx=0).stop 2024-04-25T00:41:46.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.896+0000 7fec2b570640 1 -- 172.21.15.29:0/863428322 shutdown_connections 2024-04-25T00:41:46.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.896+0000 7fec2b570640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fec10068400 0x7fec1006a8c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.896+0000 7fec2b570640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec240add70 0x7fec240ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.896+0000 7fec2b570640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec241e5dd0 0x7fec241e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.896+0000 7fec2b570640 1 --2- 172.21.15.29:0/863428322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec0c003f40 0x7fec241e3fc0 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:46.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.896+0000 7fec2b570640 1 -- 172.21.15.29:0/863428322 >> 172.21.15.29:0/863428322 conn(0x7fec240b2920 msgr2=0x7fec240b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:46.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.896+0000 7fec2b570640 1 -- 172.21.15.29:0/863428322 shutdown_connections 2024-04-25T00:41:46.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:46.896+0000 7fec2b570640 1 -- 172.21.15.29:0/863428322 wait complete. 2024-04-25T00:41:46.914 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:41:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.036+0000 7f4ffaf0e640 1 Processor -- start 2024-04-25T00:41:47.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.036+0000 7f4ffaf0e640 1 -- start start 2024-04-25T00:41:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ffaf0e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ff40b3140 0x7f4ff406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ffaf0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ff40b4b10 0x7f4ff40b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ffaf0e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ff40b41e0 con 0x7f4ff40b3a60 2024-04-25T00:41:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ffaf0e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4ff406d960 con 0x7f4ff40b3140 2024-04-25T00:41:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ffaf0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff406dc40 con 0x7f4ff40b4b10 2024-04-25T00:41:47.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff94ab640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4ff40b3a60 0x7f4ff415c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:60080/0 (socket says 172.21.15.29:60080) 2024-04-25T00:41:47.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff94ab640 1 -- 172.21.15.29:0/458215958 learned_addr learned my addr 172.21.15.29:0/458215958 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:47.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff3fff640 1 --2- 172.21.15.29:0/458215958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ff40b4b10 0x7f4ff40b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff8caa640 1 --2- 172.21.15.29:0/458215958 >> v2:172.21.15.157:3300/0 conn(0x7f4ff40b3140 0x7f4ff406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 545133682 0 0) 0x7f4ff40b41e0 con 0x7f4ff40b3a60 2024-04-25T00:41:47.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fdc003530 con 0x7f4ff40b3a60 2024-04-25T00:41:47.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1308173078 0 0) 0x7f4fdc003530 con 0x7f4ff40b3a60 2024-04-25T00:41:47.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ff40b4b10 msgr2=0x7f4ff40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:47.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 --2- 172.21.15.29:0/458215958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ff40b4b10 0x7f4ff40b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 >> v2:172.21.15.157:3300/0 conn(0x7f4ff40b3140 msgr2=0x7f4ff406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:47.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 --2- 172.21.15.29:0/458215958 >> v2:172.21.15.157:3300/0 conn(0x7f4ff40b3140 0x7f4ff406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff406df20 con 0x7f4ff40b3a60 2024-04-25T00:41:47.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4fec001680 con 0x7f4ff40b3a60 2024-04-25T00:41:47.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 >> v1:172.21.15.29:6789/0 conn(0x7f4ff40b3a60 legacy=0x7f4ff415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 --2- 172.21.15.29:0/458215958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fdc0039c0 0x7f4fdc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 --2- 172.21.15.29:0/458215958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fdc004810 0x7f4fdc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 --2- 172.21.15.29:0/458215958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ff40b4b10 0x7f4fdc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff406d960 con 0x7f4fdc004810 2024-04-25T00:41:47.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff406dc40 con 0x7f4fdc0039c0 2024-04-25T00:41:47.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4fdc00b9d0 con 0x7f4ff40b4b10 2024-04-25T00:41:47.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff3fff640 1 --2- 172.21.15.29:0/458215958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fdc004810 0x7f4fdc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff8caa640 1 --2- 172.21.15.29:0/458215958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fdc0039c0 0x7f4fdc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff94ab640 1 --2- 172.21.15.29:0/458215958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ff40b4b10 0x7f4fdc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff3fff640 1 -- 172.21.15.29:0/458215958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ff40b4b10 msgr2=0x7f4fdc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:47.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff3fff640 1 --2- 172.21.15.29:0/458215958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ff40b4b10 0x7f4fdc0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff3fff640 1 -- 172.21.15.29:0/458215958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fdc0039c0 msgr2=0x7f4fdc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:47.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff3fff640 1 --2- 172.21.15.29:0/458215958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fdc0039c0 0x7f4fdc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.040+0000 7f4ff3fff640 1 -- 172.21.15.29:0/458215958 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc003530 con 0x7f4fdc004810 2024-04-25T00:41:47.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff3fff640 1 --2- 172.21.15.29:0/458215958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fdc004810 0x7f4fdc004c90 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009fc0 tx=0x7f4fe801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7aafdc37762329 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:47.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe80b2070 con 0x7f4fdc004810 2024-04-25T00:41:47.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fe80a5bb0 con 0x7f4fdc004810 2024-04-25T00:41:47.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff37fe640 1 -- 172.21.15.29:0/458215958 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8009c10 con 0x7f4fdc004810 2024-04-25T00:41:47.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/458215958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fdc004810 msgr2=0x7f4fdc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 --2- 172.21.15.29:0/458215958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fdc004810 0x7f4fdc004c90 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009fc0 tx=0x7f4fe801cc80 comp rx=0 tx=0).stop 2024-04-25T00:41:47.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff94ab640 1 -- 172.21.15.29:0/458215958 reap_dead start 2024-04-25T00:41:47.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/458215958 shutdown_connections 2024-04-25T00:41:47.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/458215958 >> 172.21.15.29:0/458215958 conn(0x7f4ff40b1870 msgr2=0x7f4ff40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:47.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/458215958 shutdown_connections 2024-04-25T00:41:47.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/458215958 wait complete. 2024-04-25T00:41:47.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 Processor -- start 2024-04-25T00:41:47.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 -- start start 2024-04-25T00:41:47.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fdc004810 0x7f4ff40c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff413f440 0x7f4ff41f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ff4138120 0x7f4ff41385a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff4145910 con 0x7f4fdc004810 2024-04-25T00:41:47.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff40b4f70 con 0x7f4ff413f440 2024-04-25T00:41:47.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff4145a90 con 0x7f4ff4138120 2024-04-25T00:41:47.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff8caa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fdc004810 0x7f4ff40c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff8caa640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fdc004810 0x7f4ff40c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53858/0 (socket says 172.21.15.29:53858) 2024-04-25T00:41:47.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff8caa640 1 -- 172.21.15.29:0/2763651305 learned_addr learned my addr 172.21.15.29:0/2763651305 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:47.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff3fff640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff413f440 0x7f4ff41f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff94ab640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ff4138120 0x7f4ff41385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff8caa640 1 -- 172.21.15.29:0/2763651305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ff4138120 msgr2=0x7f4ff41385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff8caa640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ff4138120 0x7f4ff41385a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff8caa640 1 -- 172.21.15.29:0/2763651305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff413f440 msgr2=0x7f4ff41f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff8caa640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff413f440 0x7f4ff41f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff8caa640 1 -- 172.21.15.29:0/2763651305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff413cc80 con 0x7f4fdc004810 2024-04-25T00:41:47.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff8caa640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fdc004810 0x7f4ff40c4c30 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4011690 tx=0x7f4fe4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:47.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff17fa640 1 -- 172.21.15.29:0/2763651305 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe400f9c0 con 0x7f4fdc004810 2024-04-25T00:41:47.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff17fa640 1 -- 172.21.15.29:0/2763651305 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fe400fb60 con 0x7f4fdc004810 2024-04-25T00:41:47.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/2763651305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff413cf60 con 0x7f4fdc004810 2024-04-25T00:41:47.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.044+0000 7f4ff17fa640 1 -- 172.21.15.29:0/2763651305 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4015460 con 0x7f4fdc004810 2024-04-25T00:41:47.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.048+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/2763651305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff413d520 con 0x7f4fdc004810 2024-04-25T00:41:47.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.048+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/2763651305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff4069270 con 0x7f4fdc004810 2024-04-25T00:41:47.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.048+0000 7f4ff17fa640 1 -- 172.21.15.29:0/2763651305 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f4fe4015600 con 0x7f4fdc004810 2024-04-25T00:41:47.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.048+0000 7f4ff17fa640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4fe0068400 0x7f4fe006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.048+0000 7f4ff17fa640 1 -- 172.21.15.29:0/2763651305 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f4fe40852c0 con 0x7f4fdc004810 2024-04-25T00:41:47.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.048+0000 7f4ff3fff640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4fe0068400 0x7f4fe006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.048+0000 7f4ff3fff640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4fe0068400 0x7f4fe006a8c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80af000 tx=0x7f4fe80a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:47.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.052+0000 7f4ff17fa640 1 -- 172.21.15.29:0/2763651305 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4fe4019030 con 0x7f4fdc004810 2024-04-25T00:41:47.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.244+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/2763651305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ff40cf840 con 0x7f4fdc004810 2024-04-25T00:41:47.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.248+0000 7f4ff17fa640 1 -- 172.21.15.29:0/2763651305 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5195 (secure 0 0 0) 0x7f4fe4057660 con 0x7f4fdc004810 2024-04-25T00:41:47.248 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:47.248 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":113,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7427,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/929849226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":929849226},{"type":"v1","addr":"172.21.15.157:6834","nonce":929849226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1459365951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1459365951},{"type":"v1","addr":"172.21.15.29:6833","nonce":1459365951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1011867355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1011867355},{"type":"v1","addr":"172.21.15.157:6835","nonce":1011867355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:41:20.082895+0000","modified":"2024-04-25T00:41:23.093215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10578":{"gid":10578,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/85311029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":85311029},{"type":"v1","addr":"172.21.15.29:6835","nonce":85311029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10578,"qdb_cluster":[10578]},"id":13}]} 2024-04-25T00:41:47.248 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 113 2024-04-25T00:41:47.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.248+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/2763651305 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4fe0068400 msgr2=0x7f4fe006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.248+0000 7f4ffaf0e640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4fe0068400 0x7f4fe006a8c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80af000 tx=0x7f4fe80a7040 comp rx=0 tx=0).stop 2024-04-25T00:41:47.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.248+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/2763651305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fdc004810 msgr2=0x7f4ff40c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.248+0000 7f4ffaf0e640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fdc004810 0x7f4ff40c4c30 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4011690 tx=0x7f4fe4011b50 comp rx=0 tx=0).stop 2024-04-25T00:41:47.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.248+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/2763651305 shutdown_connections 2024-04-25T00:41:47.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.248+0000 7f4ffaf0e640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4fe0068400 0x7f4fe006a8c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.248+0000 7f4ffaf0e640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ff4138120 0x7f4ff41385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.248+0000 7f4ffaf0e640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff413f440 0x7f4ff41f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.248+0000 7f4ffaf0e640 1 --2- 172.21.15.29:0/2763651305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fdc004810 0x7f4ff40c4c30 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.248+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/2763651305 >> 172.21.15.29:0/2763651305 conn(0x7f4ff40b1870 msgr2=0x7f4ff415f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:47.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.248+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/2763651305 shutdown_connections 2024-04-25T00:41:47.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.248+0000 7f4ffaf0e640 1 -- 172.21.15.29:0/2763651305 wait complete. 2024-04-25T00:41:47.266 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5826b76640 1 Processor -- start 2024-04-25T00:41:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5826b76640 1 -- start start 2024-04-25T00:41:47.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5826b76640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f582015e200 0x7f58200d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5826b76640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200d2c60 0x7f58200b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5826b76640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58200cfc10 con 0x7f58200cf690 2024-04-25T00:41:47.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5826b76640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f58200b38d0 con 0x7f582015e200 2024-04-25T00:41:47.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5826b76640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f58200cfef0 con 0x7f58200d2c60 2024-04-25T00:41:47.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5824912640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f582015e200 0x7f58200d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5824912640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f582015e200 0x7f58200d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53860/0 (socket says 172.21.15.29:53860) 2024-04-25T00:41:47.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5824912640 1 -- 172.21.15.29:0/3756793909 learned_addr learned my addr 172.21.15.29:0/3756793909 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:47.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5825113640 1 --2- 172.21.15.29:0/3756793909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200d2c60 0x7f58200b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5824912640 1 -- 172.21.15.29:0/3756793909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200d2c60 msgr2=0x7f58200b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:47.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5824912640 1 --2- 172.21.15.29:0/3756793909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200d2c60 0x7f58200b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5824912640 1 -- 172.21.15.29:0/3756793909 >> v1:172.21.15.157:6789/0 conn(0x7f58200cf690 legacy=0x7f58200d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:47.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.388+0000 7f5824912640 1 -- 172.21.15.29:0/3756793909 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58200b3b20 con 0x7f582015e200 2024-04-25T00:41:47.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5824912640 1 --2- 172.21.15.29:0/3756793909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f582015e200 0x7f58200d47f0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f5810009870 tx=0x7f581001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537a0c7c95acadb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:47.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f581f7fe640 1 -- 172.21.15.29:0/3756793909 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58100a9610 con 0x7f582015e200 2024-04-25T00:41:47.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f581f7fe640 1 -- 172.21.15.29:0/3756793909 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58100a97b0 con 0x7f582015e200 2024-04-25T00:41:47.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f581f7fe640 1 -- 172.21.15.29:0/3756793909 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58100a9b80 con 0x7f582015e200 2024-04-25T00:41:47.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 -- 172.21.15.29:0/3756793909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f582015e200 msgr2=0x7f58200d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 --2- 172.21.15.29:0/3756793909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f582015e200 0x7f58200d47f0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f5810009870 tx=0x7f581001c530 comp rx=0 tx=0).stop 2024-04-25T00:41:47.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 -- 172.21.15.29:0/3756793909 shutdown_connections 2024-04-25T00:41:47.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 --2- 172.21.15.29:0/3756793909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200d2c60 0x7f58200b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 --2- 172.21.15.29:0/3756793909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f582015e200 0x7f58200d47f0 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 -- 172.21.15.29:0/3756793909 >> 172.21.15.29:0/3756793909 conn(0x7f582009d280 msgr2=0x7f58200d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:47.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 -- 172.21.15.29:0/3756793909 shutdown_connections 2024-04-25T00:41:47.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 -- 172.21.15.29:0/3756793909 wait complete. 2024-04-25T00:41:47.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 Processor -- start 2024-04-25T00:41:47.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 -- start start 2024-04-25T00:41:47.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58200cf690 0x7f58200b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200d2c60 0x7f5820161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f582015e200 0x7f5820156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f58200b51f0 con 0x7f58200cf690 2024-04-25T00:41:47.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58200b5070 con 0x7f58200d2c60 2024-04-25T00:41:47.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5826b76640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f58200b5370 con 0x7f582015e200 2024-04-25T00:41:47.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5824912640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58200cf690 0x7f58200b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5824912640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58200cf690 0x7f58200b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53864/0 (socket says 172.21.15.29:53864) 2024-04-25T00:41:47.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.392+0000 7f5824912640 1 -- 172.21.15.29:0/4167402871 learned_addr learned my addr 172.21.15.29:0/4167402871 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:47.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f5825113640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f582015e200 0x7f5820156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f581ffff640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200d2c60 0x7f5820161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f5824912640 1 -- 172.21.15.29:0/4167402871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f582015e200 msgr2=0x7f5820156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f5824912640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f582015e200 0x7f5820156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f5824912640 1 -- 172.21.15.29:0/4167402871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200d2c60 msgr2=0x7f5820161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f5824912640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200d2c60 0x7f5820161050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f5824912640 1 -- 172.21.15.29:0/4167402871 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58200c9ca0 con 0x7f58200cf690 2024-04-25T00:41:47.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f5824912640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58200cf690 0x7f58200b6b70 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f581001ca10 tx=0x7f58100192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:47.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f581d7fa640 1 -- 172.21.15.29:0/4167402871 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58100b9630 con 0x7f58200cf690 2024-04-25T00:41:47.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f581d7fa640 1 -- 172.21.15.29:0/4167402871 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58100b97d0 con 0x7f58200cf690 2024-04-25T00:41:47.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f5826b76640 1 -- 172.21.15.29:0/4167402871 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58200c9f80 con 0x7f58200cf690 2024-04-25T00:41:47.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f581d7fa640 1 -- 172.21.15.29:0/4167402871 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58100b9bc0 con 0x7f58200cf690 2024-04-25T00:41:47.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f5826b76640 1 -- 172.21.15.29:0/4167402871 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58200c7e20 con 0x7f58200cf690 2024-04-25T00:41:47.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f5826b76640 1 -- 172.21.15.29:0/4167402871 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5820069270 con 0x7f58200cf690 2024-04-25T00:41:47.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f581d7fa640 1 -- 172.21.15.29:0/4167402871 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f58100c23f0 con 0x7f58200cf690 2024-04-25T00:41:47.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f581d7fa640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f580c068400 0x7f580c06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f581d7fa640 1 -- 172.21.15.29:0/4167402871 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f581011eb20 con 0x7f58200cf690 2024-04-25T00:41:47.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f581ffff640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f580c068400 0x7f580c06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.396+0000 7f581ffff640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f580c068400 0x7f580c06a8c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f58200b9da0 tx=0x7f5814009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:47.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.400+0000 7f581d7fa640 1 -- 172.21.15.29:0/4167402871 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f58100ad020 con 0x7f58200cf690 2024-04-25T00:41:47.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.564+0000 7f5826b76640 1 -- 172.21.15.29:0/4167402871 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f582006cb20 con 0x7f58200cf690 2024-04-25T00:41:47.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.564+0000 7f581d7fa640 1 -- 172.21.15.29:0/4167402871 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) ==== 74+0+16635 (secure 0 0 0) 0x7f5810016150 con 0x7f58200cf690 2024-04-25T00:41:47.566 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:47.567 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":168,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:41:21.712101+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:41:18.370636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:41:19.067830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:47.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.568+0000 7f5826b76640 1 -- 172.21.15.29:0/4167402871 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f580c068400 msgr2=0x7f580c06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.568+0000 7f5826b76640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f580c068400 0x7f580c06a8c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f58200b9da0 tx=0x7f5814009040 comp rx=0 tx=0).stop 2024-04-25T00:41:47.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.568+0000 7f5826b76640 1 -- 172.21.15.29:0/4167402871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58200cf690 msgr2=0x7f58200b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.568+0000 7f5826b76640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58200cf690 0x7f58200b6b70 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f581001ca10 tx=0x7f58100192d0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.568+0000 7f5826b76640 1 -- 172.21.15.29:0/4167402871 shutdown_connections 2024-04-25T00:41:47.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.568+0000 7f5826b76640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f580c068400 0x7f580c06a8c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.568+0000 7f5826b76640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f582015e200 0x7f5820156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.568+0000 7f5826b76640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200d2c60 0x7f5820161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.568+0000 7f5826b76640 1 --2- 172.21.15.29:0/4167402871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58200cf690 0x7f58200b6b70 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.568+0000 7f5826b76640 1 -- 172.21.15.29:0/4167402871 >> 172.21.15.29:0/4167402871 conn(0x7f582009d280 msgr2=0x7f58200b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:47.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.568+0000 7f5826b76640 1 -- 172.21.15.29:0/4167402871 shutdown_connections 2024-04-25T00:41:47.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.568+0000 7f5826b76640 1 -- 172.21.15.29:0/4167402871 wait complete. 2024-04-25T00:41:47.585 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:41:47.585 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:41:47.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3234077640 1 Processor -- start 2024-04-25T00:41:47.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3234077640 1 -- start start 2024-04-25T00:41:47.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3234077640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c0b4780 0x7f322c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3234077640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f322c0b3720 con 0x7f322c0cc240 2024-04-25T00:41:47.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3234077640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f322c0b39c0 con 0x7f322c15e8d0 2024-04-25T00:41:47.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3234077640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f322c0b42f0 con 0x7f322c0b4780 2024-04-25T00:41:47.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3232614640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f322c0cc240 0x7f322c15c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55720/0 (socket says 172.21.15.29:55720) 2024-04-25T00:41:47.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3232614640 1 -- 172.21.15.29:0/985265049 learned_addr learned my addr 172.21.15.29:0/985265049 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:47.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3231612640 1 --2- 172.21.15.29:0/985265049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c0b4780 0x7f322c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3230e11640 1 -- 172.21.15.29:0/985265049 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3545934583 0 0) 0x7f322c0b3720 con 0x7f322c0cc240 2024-04-25T00:41:47.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3230e11640 1 -- 172.21.15.29:0/985265049 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3218003530 con 0x7f322c0cc240 2024-04-25T00:41:47.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3230e11640 1 -- 172.21.15.29:0/985265049 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 203623969 0 0) 0x7f322c0b39c0 con 0x7f322c15e8d0 2024-04-25T00:41:47.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3230e11640 1 -- 172.21.15.29:0/985265049 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f322c0b3720 con 0x7f322c15e8d0 2024-04-25T00:41:47.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3230e11640 1 -- 172.21.15.29:0/985265049 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4266806698 0 0) 0x7f3218003530 con 0x7f322c0cc240 2024-04-25T00:41:47.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3231612640 1 -- 172.21.15.29:0/985265049 >> v1:172.21.15.157:6789/0 conn(0x7f322c15e8d0 legacy=0x7f322c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3231612640 1 -- 172.21.15.29:0/985265049 >> v1:172.21.15.29:6789/0 conn(0x7f322c0cc240 legacy=0x7f322c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.700+0000 7f3231612640 1 -- 172.21.15.29:0/985265049 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c0b3ca0 con 0x7f322c0b4780 2024-04-25T00:41:47.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.704+0000 7f3231612640 1 --2- 172.21.15.29:0/985265049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c0b4780 0x7f322c15f3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f321c0097f0 tx=0x7f321c01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=778ffaf73d0cf4ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:47.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.704+0000 7f3230e11640 1 -- 172.21.15.29:0/985265049 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c0b0070 con 0x7f322c0b4780 2024-04-25T00:41:47.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.704+0000 7f3230e11640 1 -- 172.21.15.29:0/985265049 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321c0ab400 con 0x7f322c0b4780 2024-04-25T00:41:47.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.704+0000 7f3230e11640 1 -- 172.21.15.29:0/985265049 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228001a00 con 0x7f322c0b4780 2024-04-25T00:41:47.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.704+0000 7f3234077640 1 -- 172.21.15.29:0/985265049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c0b4780 msgr2=0x7f322c15f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.704+0000 7f3234077640 1 --2- 172.21.15.29:0/985265049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c0b4780 0x7f322c15f3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f321c0097f0 tx=0x7f321c01c400 comp rx=0 tx=0).stop 2024-04-25T00:41:47.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.704+0000 7f3234077640 1 -- 172.21.15.29:0/985265049 shutdown_connections 2024-04-25T00:41:47.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.704+0000 7f3234077640 1 --2- 172.21.15.29:0/985265049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c0b4780 0x7f322c15f3c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.704+0000 7f3234077640 1 -- 172.21.15.29:0/985265049 >> 172.21.15.29:0/985265049 conn(0x7f322c0b1870 msgr2=0x7f322c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:47.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.704+0000 7f3234077640 1 -- 172.21.15.29:0/985265049 shutdown_connections 2024-04-25T00:41:47.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.704+0000 7f3234077640 1 -- 172.21.15.29:0/985265049 wait complete. 2024-04-25T00:41:47.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.704+0000 7f3234077640 1 Processor -- start 2024-04-25T00:41:47.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.704+0000 7f3234077640 1 -- start start 2024-04-25T00:41:47.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3234077640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c0b4780 0x7f322c0aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3234077640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c0cc240 0x7f322c0cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3234077640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f322c15e8d0 0x7f322c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3234077640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f322c166df0 con 0x7f322c15e8d0 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3234077640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f322c166c70 con 0x7f322c0b4780 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3234077640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f322c166f70 con 0x7f322c0cc240 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3232614640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f322c15e8d0 0x7f322c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3232614640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f322c15e8d0 0x7f322c0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56226/0 (socket says 172.21.15.29:56226) 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3232614640 1 -- 172.21.15.29:0/3740333063 learned_addr learned my addr 172.21.15.29:0/3740333063 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3231612640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c0cc240 0x7f322c0cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3231e13640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c0b4780 0x7f322c0aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3232614640 1 -- 172.21.15.29:0/3740333063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c0cc240 msgr2=0x7f322c0cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3232614640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c0cc240 0x7f322c0cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3232614640 1 -- 172.21.15.29:0/3740333063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c0b4780 msgr2=0x7f322c0aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3232614640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c0b4780 0x7f322c0aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3232614640 1 -- 172.21.15.29:0/3740333063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c0ae1f0 con 0x7f322c15e8d0 2024-04-25T00:41:47.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3231612640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c0cc240 0x7f322c0cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:47.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3231e13640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c0b4780 0x7f322c0aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:47.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3232614640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f322c15e8d0 0x7f322c0ce8f0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f3228002240 tx=0x7f322800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:47.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f32227fc640 1 -- 172.21.15.29:0/3740333063 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228017070 con 0x7f322c15e8d0 2024-04-25T00:41:47.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f32227fc640 1 -- 172.21.15.29:0/3740333063 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3228012400 con 0x7f322c15e8d0 2024-04-25T00:41:47.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3234077640 1 -- 172.21.15.29:0/3740333063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322c0ae4d0 con 0x7f322c15e8d0 2024-04-25T00:41:47.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f32227fc640 1 -- 172.21.15.29:0/3740333063 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32280127f0 con 0x7f322c15e8d0 2024-04-25T00:41:47.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3234077640 1 -- 172.21.15.29:0/3740333063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322c0aea90 con 0x7f322c15e8d0 2024-04-25T00:41:47.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3234077640 1 -- 172.21.15.29:0/3740333063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322c069270 con 0x7f322c15e8d0 2024-04-25T00:41:47.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f32227fc640 1 -- 172.21.15.29:0/3740333063 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f322801a070 con 0x7f322c15e8d0 2024-04-25T00:41:47.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f32227fc640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3210068400 0x7f321006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:47.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3231e13640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3210068400 0x7f321006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:47.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f32227fc640 1 -- 172.21.15.29:0/3740333063 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f32280847f0 con 0x7f322c15e8d0 2024-04-25T00:41:47.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.708+0000 7f3231e13640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3210068400 0x7f321006a8c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f32240011f0 tx=0x7f3224009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:47.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.712+0000 7f32227fc640 1 -- 172.21.15.29:0/3740333063 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f32280564c0 con 0x7f322c15e8d0 2024-04-25T00:41:47.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.904+0000 7f3234077640 1 -- 172.21.15.29:0/3740333063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f322c068a20 con 0x7f322c15e8d0 2024-04-25T00:41:47.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.904+0000 7f32227fc640 1 -- 172.21.15.29:0/3740333063 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5195 (secure 0 0 0) 0x7f322805a7c0 con 0x7f322c15e8d0 2024-04-25T00:41:47.906 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:47.906 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":113,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7427,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/929849226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":929849226},{"type":"v1","addr":"172.21.15.157:6834","nonce":929849226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1459365951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1459365951},{"type":"v1","addr":"172.21.15.29:6833","nonce":1459365951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1011867355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1011867355},{"type":"v1","addr":"172.21.15.157:6835","nonce":1011867355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:41:20.082895+0000","modified":"2024-04-25T00:41:23.093215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10578":{"gid":10578,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/85311029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":85311029},{"type":"v1","addr":"172.21.15.29:6835","nonce":85311029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10578,"qdb_cluster":[10578]},"id":13}]} 2024-04-25T00:41:47.906 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 113 2024-04-25T00:41:47.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.908+0000 7f3234077640 1 -- 172.21.15.29:0/3740333063 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3210068400 msgr2=0x7f321006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.908+0000 7f3234077640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3210068400 0x7f321006a8c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f32240011f0 tx=0x7f3224009040 comp rx=0 tx=0).stop 2024-04-25T00:41:47.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.908+0000 7f3234077640 1 -- 172.21.15.29:0/3740333063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f322c15e8d0 msgr2=0x7f322c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:47.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.908+0000 7f3234077640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f322c15e8d0 0x7f322c0ce8f0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f3228002240 tx=0x7f322800a430 comp rx=0 tx=0).stop 2024-04-25T00:41:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.908+0000 7f3234077640 1 -- 172.21.15.29:0/3740333063 shutdown_connections 2024-04-25T00:41:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.908+0000 7f3234077640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3210068400 0x7f321006a8c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.908+0000 7f3234077640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f322c15e8d0 0x7f322c0ce8f0 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.908+0000 7f3234077640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c0cc240 0x7f322c0cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.908+0000 7f3234077640 1 --2- 172.21.15.29:0/3740333063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c0b4780 0x7f322c0aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.908+0000 7f3234077640 1 -- 172.21.15.29:0/3740333063 >> 172.21.15.29:0/3740333063 conn(0x7f322c0b1870 msgr2=0x7f322c06dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.908+0000 7f3234077640 1 -- 172.21.15.29:0/3740333063 shutdown_connections 2024-04-25T00:41:47.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:47.908+0000 7f3234077640 1 -- 172.21.15.29:0/3740333063 wait complete. 2024-04-25T00:41:47.924 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:41:48.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f55059e2640 1 Processor -- start 2024-04-25T00:41:48.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f55059e2640 1 -- start start 2024-04-25T00:41:48.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f55059e2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f550015c170 0x7f55000b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f55059e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55000ce350 0x7f55000d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f55059e2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55000cd440 con 0x7f55000d0920 2024-04-25T00:41:48.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f55059e2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f55000d0d70 con 0x7f550015c170 2024-04-25T00:41:48.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f55059e2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f55000cd1f0 con 0x7f55000ce350 2024-04-25T00:41:48.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f54feffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f550015c170 0x7f55000b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f54feffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f550015c170 0x7f55000b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56236/0 (socket says 172.21.15.29:56236) 2024-04-25T00:41:48.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f54feffd640 1 -- 172.21.15.29:0/4221380053 learned_addr learned my addr 172.21.15.29:0/4221380053 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:48.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f54fe7fc640 1 --2- 172.21.15.29:0/4221380053 >> v2:172.21.15.157:3300/0 conn(0x7f55000ce350 0x7f55000d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f54feffd640 1 -- 172.21.15.29:0/4221380053 >> v2:172.21.15.157:3300/0 conn(0x7f55000ce350 msgr2=0x7f55000d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:48.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f54feffd640 1 --2- 172.21.15.29:0/4221380053 >> v2:172.21.15.157:3300/0 conn(0x7f55000ce350 0x7f55000d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f54feffd640 1 -- 172.21.15.29:0/4221380053 >> v1:172.21.15.157:6789/0 conn(0x7f55000d0920 legacy=0x7f550015fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:48.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f54feffd640 1 -- 172.21.15.29:0/4221380053 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55000cd720 con 0x7f550015c170 2024-04-25T00:41:48.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.044+0000 7f54fe7fc640 1 --2- 172.21.15.29:0/4221380053 >> v2:172.21.15.157:3300/0 conn(0x7f55000ce350 0x7f55000d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f54feffd640 1 --2- 172.21.15.29:0/4221380053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f550015c170 0x7f55000b3ac0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f5500073950 tx=0x7f54f801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3792cff6345594 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f54fdffb640 1 -- 172.21.15.29:0/4221380053 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f80a57b0 con 0x7f550015c170 2024-04-25T00:41:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f54fdffb640 1 -- 172.21.15.29:0/4221380053 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f80a5950 con 0x7f550015c170 2024-04-25T00:41:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f54fdffb640 1 -- 172.21.15.29:0/4221380053 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f80a5d20 con 0x7f550015c170 2024-04-25T00:41:48.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 -- 172.21.15.29:0/4221380053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f550015c170 msgr2=0x7f55000b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 --2- 172.21.15.29:0/4221380053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f550015c170 0x7f55000b3ac0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f5500073950 tx=0x7f54f801c8f0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 -- 172.21.15.29:0/4221380053 shutdown_connections 2024-04-25T00:41:48.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 --2- 172.21.15.29:0/4221380053 >> v2:172.21.15.157:3300/0 conn(0x7f55000ce350 0x7f55000d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 --2- 172.21.15.29:0/4221380053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f550015c170 0x7f55000b3ac0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 -- 172.21.15.29:0/4221380053 >> 172.21.15.29:0/4221380053 conn(0x7f55000b5820 msgr2=0x7f55000b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:48.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 -- 172.21.15.29:0/4221380053 shutdown_connections 2024-04-25T00:41:48.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 -- 172.21.15.29:0/4221380053 wait complete. 2024-04-25T00:41:48.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 Processor -- start 2024-04-25T00:41:48.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 -- start start 2024-04-25T00:41:48.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55000ce350 0x7f55000b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55000d0920 0x7f550014f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015c170 0x7f550014e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f55000ccd80 con 0x7f55000d0920 2024-04-25T00:41:48.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f550015c550 con 0x7f55000ce350 2024-04-25T00:41:48.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f55059e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f55000b4000 con 0x7f550015c170 2024-04-25T00:41:48.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f54fe7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55000d0920 0x7f550014f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f54fe7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55000d0920 0x7f550014f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56242/0 (socket says 172.21.15.29:56242) 2024-04-25T00:41:48.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.048+0000 7f54fe7fc640 1 -- 172.21.15.29:0/21279123 learned_addr learned my addr 172.21.15.29:0/21279123 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:48.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54fe7fc640 1 -- 172.21.15.29:0/21279123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015c170 msgr2=0x7f550014e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54ff7fe640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015c170 0x7f550014e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54feffd640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55000ce350 0x7f55000b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54fe7fc640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015c170 0x7f550014e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54fe7fc640 1 -- 172.21.15.29:0/21279123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55000ce350 msgr2=0x7f55000b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54fe7fc640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55000ce350 0x7f55000b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54fe7fc640 1 -- 172.21.15.29:0/21279123 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55000b1cd0 con 0x7f55000d0920 2024-04-25T00:41:48.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54fe7fc640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55000d0920 0x7f550014f7f0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f54f400c7f0 tx=0x7f54f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:48.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54e7fff640 1 -- 172.21.15.29:0/21279123 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4015070 con 0x7f55000d0920 2024-04-25T00:41:48.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54e7fff640 1 -- 172.21.15.29:0/21279123 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f4007820 con 0x7f55000d0920 2024-04-25T00:41:48.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54e7fff640 1 -- 172.21.15.29:0/21279123 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4007c10 con 0x7f55000d0920 2024-04-25T00:41:48.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f55059e2640 1 -- 172.21.15.29:0/21279123 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55000b1040 con 0x7f55000d0920 2024-04-25T00:41:48.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f55059e2640 1 -- 172.21.15.29:0/21279123 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5500155d00 con 0x7f55000d0920 2024-04-25T00:41:48.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f55059e2640 1 -- 172.21.15.29:0/21279123 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5500155af0 con 0x7f55000d0920 2024-04-25T00:41:48.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54e7fff640 1 -- 172.21.15.29:0/21279123 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f54f4018070 con 0x7f55000d0920 2024-04-25T00:41:48.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54e7fff640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f54ec068430 0x7f54ec06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54e7fff640 1 -- 172.21.15.29:0/21279123 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f54f4084a20 con 0x7f55000d0920 2024-04-25T00:41:48.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54feffd640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f54ec068430 0x7f54ec06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.052+0000 7f54feffd640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f54ec068430 0x7f54ec06a8f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f54f80ae000 tx=0x7f54f80a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:48.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.056+0000 7f54e7fff640 1 -- 172.21.15.29:0/21279123 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f54f4056cc0 con 0x7f55000d0920 2024-04-25T00:41:48.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f55059e2640 1 -- 172.21.15.29:0/21279123 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5500069270 con 0x7f55000d0920 2024-04-25T00:41:48.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f54e7fff640 1 -- 172.21.15.29:0/21279123 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5195 (secure 0 0 0) 0x7f54f40565c0 con 0x7f55000d0920 2024-04-25T00:41:48.245 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:48.245 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":113,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7427,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/929849226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":929849226},{"type":"v1","addr":"172.21.15.157:6834","nonce":929849226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1459365951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1459365951},{"type":"v1","addr":"172.21.15.29:6833","nonce":1459365951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1011867355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1011867355},{"type":"v1","addr":"172.21.15.157:6835","nonce":1011867355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:41:20.082895+0000","modified":"2024-04-25T00:41:23.093215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10578":{"gid":10578,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/85311029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":85311029},{"type":"v1","addr":"172.21.15.29:6835","nonce":85311029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10578,"qdb_cluster":[10578]},"id":13}]} 2024-04-25T00:41:48.245 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 113 2024-04-25T00:41:48.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f55059e2640 1 -- 172.21.15.29:0/21279123 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f54ec068430 msgr2=0x7f54ec06a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f55059e2640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f54ec068430 0x7f54ec06a8f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f54f80ae000 tx=0x7f54f80a7070 comp rx=0 tx=0).stop 2024-04-25T00:41:48.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f55059e2640 1 -- 172.21.15.29:0/21279123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55000d0920 msgr2=0x7f550014f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f55059e2640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55000d0920 0x7f550014f7f0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f54f400c7f0 tx=0x7f54f400ccb0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f55059e2640 1 -- 172.21.15.29:0/21279123 shutdown_connections 2024-04-25T00:41:48.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f55059e2640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f54ec068430 0x7f54ec06a8f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f55059e2640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f550015c170 0x7f550014e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f55059e2640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55000d0920 0x7f550014f7f0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f55059e2640 1 --2- 172.21.15.29:0/21279123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55000ce350 0x7f55000b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f55059e2640 1 -- 172.21.15.29:0/21279123 >> 172.21.15.29:0/21279123 conn(0x7f55000b5820 msgr2=0x7f550015fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:48.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f55059e2640 1 -- 172.21.15.29:0/21279123 shutdown_connections 2024-04-25T00:41:48.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.244+0000 7f55059e2640 1 -- 172.21.15.29:0/21279123 wait complete. 2024-04-25T00:41:48.262 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:48.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.372+0000 7f6e894e2640 1 Processor -- start 2024-04-25T00:41:48.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.372+0000 7f6e894e2640 1 -- start start 2024-04-25T00:41:48.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.372+0000 7f6e894e2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6e8415f3c0 0x7f6e840cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.372+0000 7f6e894e2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e840b3720 con 0x7f6e840b4780 2024-04-25T00:41:48.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.372+0000 7f6e894e2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e840b39c0 con 0x7f6e8415e8d0 2024-04-25T00:41:48.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.372+0000 7f6e894e2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6e840b42f0 con 0x7f6e8415f3c0 2024-04-25T00:41:48.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.372+0000 7f6e837fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6e8415f3c0 0x7f6e840cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.372+0000 7f6e837fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6e8415f3c0 0x7f6e840cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56254/0 (socket says 172.21.15.29:56254) 2024-04-25T00:41:48.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.372+0000 7f6e837fe640 1 -- 172.21.15.29:0/2069985965 learned_addr learned my addr 172.21.15.29:0/2069985965 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:48.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.372+0000 7f6e837fe640 1 -- 172.21.15.29:0/2069985965 >> v1:172.21.15.157:6789/0 conn(0x7f6e8415e8d0 legacy=0x7f6e8415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:48.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.372+0000 7f6e837fe640 1 -- 172.21.15.29:0/2069985965 >> v1:172.21.15.29:6789/0 conn(0x7f6e840b4780 legacy=0x7f6e8415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.372+0000 7f6e837fe640 1 -- 172.21.15.29:0/2069985965 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e840b3ca0 con 0x7f6e8415f3c0 2024-04-25T00:41:48.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.372+0000 7f6e837fe640 1 --2- 172.21.15.29:0/2069985965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8415f3c0 0x7f6e840cc240 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009e50 tx=0x7f6e7c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ddfc125e2a6a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:48.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e827fc640 1 -- 172.21.15.29:0/2069985965 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0a5790 con 0x7f6e8415f3c0 2024-04-25T00:41:48.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e827fc640 1 -- 172.21.15.29:0/2069985965 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e7c0a5930 con 0x7f6e8415f3c0 2024-04-25T00:41:48.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e827fc640 1 -- 172.21.15.29:0/2069985965 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0a5d00 con 0x7f6e8415f3c0 2024-04-25T00:41:48.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 -- 172.21.15.29:0/2069985965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8415f3c0 msgr2=0x7f6e840cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 --2- 172.21.15.29:0/2069985965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8415f3c0 0x7f6e840cc240 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009e50 tx=0x7f6e7c01c9f0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 -- 172.21.15.29:0/2069985965 shutdown_connections 2024-04-25T00:41:48.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 --2- 172.21.15.29:0/2069985965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8415f3c0 0x7f6e840cc240 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 -- 172.21.15.29:0/2069985965 >> 172.21.15.29:0/2069985965 conn(0x7f6e840b1870 msgr2=0x7f6e840b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:48.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 -- 172.21.15.29:0/2069985965 shutdown_connections 2024-04-25T00:41:48.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 -- 172.21.15.29:0/2069985965 wait complete. 2024-04-25T00:41:48.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 Processor -- start 2024-04-25T00:41:48.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 -- start start 2024-04-25T00:41:48.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e840b4780 0x7f6e84072710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e8415e8d0 0x7f6e8415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8415f3c0 0x7f6e8415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6e84168e00 con 0x7f6e8415f3c0 2024-04-25T00:41:48.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6e84168c80 con 0x7f6e8415e8d0 2024-04-25T00:41:48.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6e84168f80 con 0x7f6e840b4780 2024-04-25T00:41:48.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e837fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8415f3c0 0x7f6e8415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e837fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8415f3c0 0x7f6e8415a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56262/0 (socket says 172.21.15.29:56262) 2024-04-25T00:41:48.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e837fe640 1 -- 172.21.15.29:0/3626975203 learned_addr learned my addr 172.21.15.29:0/3626975203 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e82ffd640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e840b4780 0x7f6e84072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e7bfff640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e8415e8d0 0x7f6e8415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e837fe640 1 -- 172.21.15.29:0/3626975203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e840b4780 msgr2=0x7f6e84072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e837fe640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e840b4780 0x7f6e84072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e837fe640 1 -- 172.21.15.29:0/3626975203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e8415e8d0 msgr2=0x7f6e8415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e837fe640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e8415e8d0 0x7f6e8415a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e837fe640 1 -- 172.21.15.29:0/3626975203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e84072c50 con 0x7f6e8415f3c0 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e82ffd640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e840b4780 0x7f6e84072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e837fe640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8415f3c0 0x7f6e8415a8e0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0af040 tx=0x7f6e7c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e7b7fe640 1 -- 172.21.15.29:0/3626975203 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0b8470 con 0x7f6e8415f3c0 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e7b7fe640 1 -- 172.21.15.29:0/3626975203 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e7c0b8630 con 0x7f6e8415f3c0 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 -- 172.21.15.29:0/3626975203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e840cd770 con 0x7f6e8415f3c0 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e7b7fe640 1 -- 172.21.15.29:0/3626975203 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0b8a20 con 0x7f6e8415f3c0 2024-04-25T00:41:48.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.376+0000 7f6e894e2640 1 -- 172.21.15.29:0/3626975203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e840cdd30 con 0x7f6e8415f3c0 2024-04-25T00:41:48.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.380+0000 7f6e894e2640 1 -- 172.21.15.29:0/3626975203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e84069270 con 0x7f6e8415f3c0 2024-04-25T00:41:48.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.380+0000 7f6e7b7fe640 1 -- 172.21.15.29:0/3626975203 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f6e7c0c0020 con 0x7f6e8415f3c0 2024-04-25T00:41:48.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.380+0000 7f6e7b7fe640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e64068400 0x7f6e6406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.380+0000 7f6e7b7fe640 1 -- 172.21.15.29:0/3626975203 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f6e7c11f000 con 0x7f6e8415f3c0 2024-04-25T00:41:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.380+0000 7f6e82ffd640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e64068400 0x7f6e6406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.380+0000 7f6e82ffd640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e64068400 0x7f6e6406a8c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6e74003e40 tx=0x7f6e74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:48.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.384+0000 7f6e7b7fe640 1 -- 172.21.15.29:0/3626975203 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6e7c0c7020 con 0x7f6e8415f3c0 2024-04-25T00:41:48.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.540+0000 7f6e894e2640 1 -- 172.21.15.29:0/3626975203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e840d06a0 con 0x7f6e8415f3c0 2024-04-25T00:41:48.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.540+0000 7f6e7b7fe640 1 -- 172.21.15.29:0/3626975203 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) ==== 74+0+16635 (secure 0 0 0) 0x7f6e7c0165e0 con 0x7f6e8415f3c0 2024-04-25T00:41:48.542 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:48.542 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":168,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:41:21.712101+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:41:18.370636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:41:19.067830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:48.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.544+0000 7f6e894e2640 1 -- 172.21.15.29:0/3626975203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e64068400 msgr2=0x7f6e6406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.544+0000 7f6e894e2640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e64068400 0x7f6e6406a8c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6e74003e40 tx=0x7f6e74009040 comp rx=0 tx=0).stop 2024-04-25T00:41:48.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.544+0000 7f6e894e2640 1 -- 172.21.15.29:0/3626975203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8415f3c0 msgr2=0x7f6e8415a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.544+0000 7f6e894e2640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8415f3c0 0x7f6e8415a8e0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0af040 tx=0x7f6e7c019790 comp rx=0 tx=0).stop 2024-04-25T00:41:48.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.544+0000 7f6e894e2640 1 -- 172.21.15.29:0/3626975203 shutdown_connections 2024-04-25T00:41:48.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.544+0000 7f6e894e2640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6e64068400 0x7f6e6406a8c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.544+0000 7f6e894e2640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e8415f3c0 0x7f6e8415a8e0 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.544+0000 7f6e894e2640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e8415e8d0 0x7f6e8415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.544+0000 7f6e894e2640 1 --2- 172.21.15.29:0/3626975203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e840b4780 0x7f6e84072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.544+0000 7f6e894e2640 1 -- 172.21.15.29:0/3626975203 >> 172.21.15.29:0/3626975203 conn(0x7f6e840b1870 msgr2=0x7f6e840cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:48.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.544+0000 7f6e894e2640 1 -- 172.21.15.29:0/3626975203 shutdown_connections 2024-04-25T00:41:48.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.544+0000 7f6e894e2640 1 -- 172.21.15.29:0/3626975203 wait complete. 2024-04-25T00:41:48.560 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:41:48.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d3e471640 1 Processor -- start 2024-04-25T00:41:48.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d3e471640 1 -- start start 2024-04-25T00:41:48.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d3e471640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6d38071e10 0x7f6d3815ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d3e471640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6d380721f0 0x7f6d38072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d3e471640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815e860 0x7f6d3815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d3e471640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6d3806f5e0 con 0x7f6d38071e10 2024-04-25T00:41:48.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d3e471640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f6d380b0370 con 0x7f6d380721f0 2024-04-25T00:41:48.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d3e471640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3806e580 con 0x7f6d3815e860 2024-04-25T00:41:48.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d37fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6d38071e10 0x7f6d3815ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d37fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6d38071e10 0x7f6d3815ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56278/0 (socket says 172.21.15.29:56278) 2024-04-25T00:41:48.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d37fff640 1 -- 172.21.15.29:0/3508195773 learned_addr learned my addr 172.21.15.29:0/3508195773 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:48.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d3ca0e640 1 --2- 172.21.15.29:0/3508195773 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815e860 0x7f6d3815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d377fe640 1 --2- 172.21.15.29:0/3508195773 >> v2:172.21.15.157:3300/0 conn(0x7f6d380721f0 0x7f6d38072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d37fff640 1 -- 172.21.15.29:0/3508195773 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815e860 msgr2=0x7f6d3815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:48.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d37fff640 1 --2- 172.21.15.29:0/3508195773 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815e860 0x7f6d3815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d37fff640 1 -- 172.21.15.29:0/3508195773 >> v2:172.21.15.157:3300/0 conn(0x7f6d380721f0 msgr2=0x7f6d38072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:48.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d37fff640 1 --2- 172.21.15.29:0/3508195773 >> v2:172.21.15.157:3300/0 conn(0x7f6d380721f0 0x7f6d38072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.684+0000 7f6d37fff640 1 -- 172.21.15.29:0/3508195773 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3806e820 con 0x7f6d38071e10 2024-04-25T00:41:48.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.688+0000 7f6d37fff640 1 --2- 172.21.15.29:0/3508195773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d38071e10 0x7f6d3815ba90 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f6d380b0820 tx=0x7f6d3001ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e300596d4be05b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:48.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.688+0000 7f6d36ffd640 1 -- 172.21.15.29:0/3508195773 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000b2d0 con 0x7f6d38071e10 2024-04-25T00:41:48.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.688+0000 7f6d36ffd640 1 -- 172.21.15.29:0/3508195773 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d3000b470 con 0x7f6d38071e10 2024-04-25T00:41:48.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.688+0000 7f6d36ffd640 1 -- 172.21.15.29:0/3508195773 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d300a7b70 con 0x7f6d38071e10 2024-04-25T00:41:48.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.688+0000 7f6d3e471640 1 -- 172.21.15.29:0/3508195773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d38071e10 msgr2=0x7f6d3815ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.688+0000 7f6d3e471640 1 --2- 172.21.15.29:0/3508195773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d38071e10 0x7f6d3815ba90 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f6d380b0820 tx=0x7f6d3001ebc0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.688+0000 7f6d3e471640 1 -- 172.21.15.29:0/3508195773 shutdown_connections 2024-04-25T00:41:48.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.688+0000 7f6d3e471640 1 --2- 172.21.15.29:0/3508195773 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815e860 0x7f6d3815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.688+0000 7f6d3e471640 1 --2- 172.21.15.29:0/3508195773 >> v2:172.21.15.157:3300/0 conn(0x7f6d380721f0 0x7f6d38072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.688+0000 7f6d3e471640 1 --2- 172.21.15.29:0/3508195773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d38071e10 0x7f6d3815ba90 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.688+0000 7f6d3e471640 1 -- 172.21.15.29:0/3508195773 >> 172.21.15.29:0/3508195773 conn(0x7f6d380b2920 msgr2=0x7f6d380b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:48.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.688+0000 7f6d3e471640 1 -- 172.21.15.29:0/3508195773 shutdown_connections 2024-04-25T00:41:48.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.688+0000 7f6d3e471640 1 -- 172.21.15.29:0/3508195773 wait complete. 2024-04-25T00:41:48.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d3e471640 1 Processor -- start 2024-04-25T00:41:48.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d3e471640 1 -- start start 2024-04-25T00:41:48.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d3e471640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d38071e10 0x7f6d3814dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d3e471640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d380721f0 0x7f6d38145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d3e471640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815e860 0x7f6d3814f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d3e471640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3815f600 con 0x7f6d38071e10 2024-04-25T00:41:48.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d3e471640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6d38072bc0 con 0x7f6d380721f0 2024-04-25T00:41:48.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d3e471640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3815bfd0 con 0x7f6d3815e860 2024-04-25T00:41:48.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d37fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d38071e10 0x7f6d3814dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d37fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d38071e10 0x7f6d3814dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56288/0 (socket says 172.21.15.29:56288) 2024-04-25T00:41:48.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d37fff640 1 -- 172.21.15.29:0/2296218604 learned_addr learned my addr 172.21.15.29:0/2296218604 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:48.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d377fe640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d380721f0 0x7f6d38145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d3ca0e640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815e860 0x7f6d3814f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d37fff640 1 -- 172.21.15.29:0/2296218604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815e860 msgr2=0x7f6d3814f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d37fff640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815e860 0x7f6d3814f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d37fff640 1 -- 172.21.15.29:0/2296218604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d380721f0 msgr2=0x7f6d38145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d37fff640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d380721f0 0x7f6d38145a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d37fff640 1 -- 172.21.15.29:0/2296218604 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d38145f70 con 0x7f6d38071e10 2024-04-25T00:41:48.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d37fff640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d38071e10 0x7f6d3814dcc0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f6d3001d270 tx=0x7f6d3001b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:48.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d34ff9640 1 -- 172.21.15.29:0/2296218604 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d300be020 con 0x7f6d38071e10 2024-04-25T00:41:48.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d34ff9640 1 -- 172.21.15.29:0/2296218604 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d30009800 con 0x7f6d38071e10 2024-04-25T00:41:48.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d34ff9640 1 -- 172.21.15.29:0/2296218604 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30009bf0 con 0x7f6d38071e10 2024-04-25T00:41:48.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d3e471640 1 -- 172.21.15.29:0/2296218604 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d380b5eb0 con 0x7f6d38071e10 2024-04-25T00:41:48.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d3e471640 1 -- 172.21.15.29:0/2296218604 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3814d4c0 con 0x7f6d38071e10 2024-04-25T00:41:48.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.692+0000 7f6d3e471640 1 -- 172.21.15.29:0/2296218604 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d3814e200 con 0x7f6d38071e10 2024-04-25T00:41:48.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.696+0000 7f6d34ff9640 1 -- 172.21.15.29:0/2296218604 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f6d300c1020 con 0x7f6d38071e10 2024-04-25T00:41:48.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.696+0000 7f6d34ff9640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6d18068400 0x7f6d1806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:48.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.696+0000 7f6d34ff9640 1 -- 172.21.15.29:0/2296218604 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f6d301218a0 con 0x7f6d38071e10 2024-04-25T00:41:48.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.696+0000 7f6d377fe640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6d18068400 0x7f6d1806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:48.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.696+0000 7f6d377fe640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6d18068400 0x7f6d1806a8c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6d380cd060 tx=0x7f6d20008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:48.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.700+0000 7f6d34ff9640 1 -- 172.21.15.29:0/2296218604 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6d300c8030 con 0x7f6d38071e10 2024-04-25T00:41:48.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.900+0000 7f6d3e471640 1 -- 172.21.15.29:0/2296218604 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6d380d1b30 con 0x7f6d38071e10 2024-04-25T00:41:48.901 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-04-25T00:41:48.900+0000 7f3ac00b5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:41:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.904+0000 7f6d34ff9640 1 -- 172.21.15.29:0/2296218604 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v114) ==== 161+0+0 (secure 0 0 0) 0x7f6d300187e0 con 0x7f6d38071e10 2024-04-25T00:41:48.904 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:41:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.908+0000 7f6d3e471640 1 -- 172.21.15.29:0/2296218604 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6d18068400 msgr2=0x7f6d1806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.908+0000 7f6d3e471640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6d18068400 0x7f6d1806a8c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6d380cd060 tx=0x7f6d20008040 comp rx=0 tx=0).stop 2024-04-25T00:41:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.908+0000 7f6d3e471640 1 -- 172.21.15.29:0/2296218604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d38071e10 msgr2=0x7f6d3814dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.908+0000 7f6d3e471640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d38071e10 0x7f6d3814dcc0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f6d3001d270 tx=0x7f6d3001b960 comp rx=0 tx=0).stop 2024-04-25T00:41:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.908+0000 7f6d3e471640 1 -- 172.21.15.29:0/2296218604 shutdown_connections 2024-04-25T00:41:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.908+0000 7f6d3e471640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f6d18068400 0x7f6d1806a8c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.908+0000 7f6d3e471640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815e860 0x7f6d3814f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.908+0000 7f6d3e471640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d380721f0 0x7f6d38145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.908+0000 7f6d3e471640 1 --2- 172.21.15.29:0/2296218604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d38071e10 0x7f6d3814dcc0 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.908+0000 7f6d3e471640 1 -- 172.21.15.29:0/2296218604 >> 172.21.15.29:0/2296218604 conn(0x7f6d380b2920 msgr2=0x7f6d3815cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.908+0000 7f6d3e471640 1 -- 172.21.15.29:0/2296218604 shutdown_connections 2024-04-25T00:41:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:48.908+0000 7f6d3e471640 1 -- 172.21.15.29:0/2296218604 wait complete. 2024-04-25T00:41:48.927 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:41:48.945 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-04-25T00:41:49.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f143311b640 1 Processor -- start 2024-04-25T00:41:49.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f143311b640 1 -- start start 2024-04-25T00:41:49.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f143311b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f142c15e8d0 0x7f142c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:49.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f143311b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f142c15f3c0 0x7f142c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:49.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f143311b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f142c0b3720 con 0x7f142c0b4780 2024-04-25T00:41:49.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f143311b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f142c0b42f0 con 0x7f142c15e8d0 2024-04-25T00:41:49.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f143311b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f142c0b39c0 con 0x7f142c15f3c0 2024-04-25T00:41:49.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f1430eb7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f142c15e8d0 0x7f142c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:49.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f142bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f142c0b4780 0x7f142c15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55744/0 (socket says 172.21.15.29:55744) 2024-04-25T00:41:49.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f1430eb7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f142c15e8d0 0x7f142c0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56324/0 (socket says 172.21.15.29:56324) 2024-04-25T00:41:49.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f142bfff640 1 -- 172.21.15.29:0/2784127439 learned_addr learned my addr 172.21.15.29:0/2784127439 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:49.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f14316b8640 1 --2- 172.21.15.29:0/2784127439 >> v2:172.21.15.157:3300/0 conn(0x7f142c15f3c0 0x7f142c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:49.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f1430eb7640 1 -- 172.21.15.29:0/2784127439 >> v2:172.21.15.157:3300/0 conn(0x7f142c15f3c0 msgr2=0x7f142c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:49.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f1430eb7640 1 --2- 172.21.15.29:0/2784127439 >> v2:172.21.15.157:3300/0 conn(0x7f142c15f3c0 0x7f142c15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:49.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f1430eb7640 1 -- 172.21.15.29:0/2784127439 >> v1:172.21.15.29:6789/0 conn(0x7f142c0b4780 legacy=0x7f142c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:49.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f142b7fe640 1 -- 172.21.15.29:0/2784127439 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2990952826 0 0) 0x7f142c0b3720 con 0x7f142c0b4780 2024-04-25T00:41:49.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.040+0000 7f1430eb7640 1 -- 172.21.15.29:0/2784127439 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142c0b3ca0 con 0x7f142c15e8d0 2024-04-25T00:41:49.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f1430eb7640 1 --2- 172.21.15.29:0/2784127439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f142c15e8d0 0x7f142c0cc240 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f142c15f9a0 tx=0x7f142001eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87372dbe5299894 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:49.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f142b7fe640 1 -- 172.21.15.29:0/2784127439 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14200b4070 con 0x7f142c15e8d0 2024-04-25T00:41:49.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f142b7fe640 1 -- 172.21.15.29:0/2784127439 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14200a7af0 con 0x7f142c15e8d0 2024-04-25T00:41:49.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f142b7fe640 1 -- 172.21.15.29:0/2784127439 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14200af440 con 0x7f142c15e8d0 2024-04-25T00:41:49.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 -- 172.21.15.29:0/2784127439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f142c15e8d0 msgr2=0x7f142c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:49.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 --2- 172.21.15.29:0/2784127439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f142c15e8d0 0x7f142c0cc240 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f142c15f9a0 tx=0x7f142001eb50 comp rx=0 tx=0).stop 2024-04-25T00:41:49.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 -- 172.21.15.29:0/2784127439 shutdown_connections 2024-04-25T00:41:49.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 --2- 172.21.15.29:0/2784127439 >> v2:172.21.15.157:3300/0 conn(0x7f142c15f3c0 0x7f142c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 --2- 172.21.15.29:0/2784127439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f142c15e8d0 0x7f142c0cc240 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 -- 172.21.15.29:0/2784127439 >> 172.21.15.29:0/2784127439 conn(0x7f142c0b1870 msgr2=0x7f142c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 -- 172.21.15.29:0/2784127439 shutdown_connections 2024-04-25T00:41:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 -- 172.21.15.29:0/2784127439 wait complete. 2024-04-25T00:41:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 Processor -- start 2024-04-25T00:41:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 -- start start 2024-04-25T00:41:49.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f142c0b4780 0x7f142c158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:49.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f142c15e8d0 0x7f142c0afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:49.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f142c15f3c0 0x7f142c0b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f142c15d6a0 con 0x7f142c15e8d0 2024-04-25T00:41:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f142c15d520 con 0x7f142c0b4780 2024-04-25T00:41:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f142c06d5d0 con 0x7f142c15f3c0 2024-04-25T00:41:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f142bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f142c15e8d0 0x7f142c0afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f142bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f142c15e8d0 0x7f142c0afb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56326/0 (socket says 172.21.15.29:56326) 2024-04-25T00:41:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f142bfff640 1 -- 172.21.15.29:0/2290113231 learned_addr learned my addr 172.21.15.29:0/2290113231 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f1430eb7640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f142c0b4780 0x7f142c158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f142bfff640 1 -- 172.21.15.29:0/2290113231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f142c15f3c0 msgr2=0x7f142c0b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f142bfff640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f142c15f3c0 0x7f142c0b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f142bfff640 1 -- 172.21.15.29:0/2290113231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f142c0b4780 msgr2=0x7f142c158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f142bfff640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f142c0b4780 0x7f142c158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f142bfff640 1 -- 172.21.15.29:0/2290113231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142c13e860 con 0x7f142c15e8d0 2024-04-25T00:41:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f1430eb7640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f142c0b4780 0x7f142c158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f142bfff640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f142c15e8d0 0x7f142c0afb20 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f141800c520 tx=0x7f141800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f14297fa640 1 -- 172.21.15.29:0/2290113231 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418018070 con 0x7f142c15e8d0 2024-04-25T00:41:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f14297fa640 1 -- 172.21.15.29:0/2290113231 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14180074e0 con 0x7f142c15e8d0 2024-04-25T00:41:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 -- 172.21.15.29:0/2290113231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142c13eb40 con 0x7f142c15e8d0 2024-04-25T00:41:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f14297fa640 1 -- 172.21.15.29:0/2290113231 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14180049f0 con 0x7f142c15e8d0 2024-04-25T00:41:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.044+0000 7f143311b640 1 -- 172.21.15.29:0/2290113231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f142c158c60 con 0x7f142c15e8d0 2024-04-25T00:41:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.048+0000 7f143311b640 1 -- 172.21.15.29:0/2290113231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f142c069270 con 0x7f142c15e8d0 2024-04-25T00:41:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.048+0000 7f14297fa640 1 -- 172.21.15.29:0/2290113231 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f141801b070 con 0x7f142c15e8d0 2024-04-25T00:41:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.048+0000 7f14297fa640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1410068430 0x7f141006a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.048+0000 7f1430eb7640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1410068430 0x7f141006a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.048+0000 7f14297fa640 1 -- 172.21.15.29:0/2290113231 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5795+0+0 (secure 0 0 0) 0x7f1418083c20 con 0x7f142c15e8d0 2024-04-25T00:41:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.048+0000 7f1430eb7640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1410068430 0x7f141006a8f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f14200b1000 tx=0x7f14200a9070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:49.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.052+0000 7f14297fa640 1 -- 172.21.15.29:0/2290113231 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1418055fc0 con 0x7f142c15e8d0 2024-04-25T00:41:49.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.240+0000 7f143311b640 1 -- 172.21.15.29:0/2290113231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f142c06e7b0 con 0x7f142c15e8d0 2024-04-25T00:41:49.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.904+0000 7f14297fa640 1 -- 172.21.15.29:0/2290113231 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v115) ==== 104+0+0 (secure 0 0 0) 0x7f14180558c0 con 0x7f142c15e8d0 2024-04-25T00:41:49.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.908+0000 7f143311b640 1 -- 172.21.15.29:0/2290113231 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1410068430 msgr2=0x7f141006a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:49.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.908+0000 7f143311b640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1410068430 0x7f141006a8f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f14200b1000 tx=0x7f14200a9070 comp rx=0 tx=0).stop 2024-04-25T00:41:49.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.908+0000 7f143311b640 1 -- 172.21.15.29:0/2290113231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f142c15e8d0 msgr2=0x7f142c0afb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:49.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.908+0000 7f143311b640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f142c15e8d0 0x7f142c0afb20 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f141800c520 tx=0x7f141800c9e0 comp rx=0 tx=0).stop 2024-04-25T00:41:49.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.908+0000 7f143311b640 1 -- 172.21.15.29:0/2290113231 shutdown_connections 2024-04-25T00:41:49.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.908+0000 7f143311b640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f1410068430 0x7f141006a8f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:49.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.908+0000 7f143311b640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f142c15f3c0 0x7f142c0b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:49.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.908+0000 7f143311b640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f142c15e8d0 0x7f142c0afb20 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:49.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.908+0000 7f143311b640 1 --2- 172.21.15.29:0/2290113231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f142c0b4780 0x7f142c158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:49.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.908+0000 7f143311b640 1 -- 172.21.15.29:0/2290113231 >> 172.21.15.29:0/2290113231 conn(0x7f142c0b1870 msgr2=0x7f142c0cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:49.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.908+0000 7f143311b640 1 -- 172.21.15.29:0/2290113231 shutdown_connections 2024-04-25T00:41:49.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:49.908+0000 7f143311b640 1 -- 172.21.15.29:0/2290113231 wait complete. 2024-04-25T00:41:49.965 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:41:50.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.144+0000 7f50fe310640 1 Processor -- start 2024-04-25T00:41:50.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.144+0000 7f50fe310640 1 -- start start 2024-04-25T00:41:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.144+0000 7f50fe310640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f815e560 0x7f50f80d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.144+0000 7f50fe310640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50f80d2680 0x7f50f80d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.144+0000 7f50fe310640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f80d3050 con 0x7f50f815e180 2024-04-25T00:41:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.144+0000 7f50fe310640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f50f80cfed0 con 0x7f50f80d2680 2024-04-25T00:41:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.144+0000 7f50fe310640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f50f80b7100 con 0x7f50f815e560 2024-04-25T00:41:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.144+0000 7f50f7fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f50f815e180 0x7f50f80cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55754/0 (socket says 172.21.15.29:55754) 2024-04-25T00:41:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.144+0000 7f50f7fff640 1 -- 172.21.15.29:0/2209463078 learned_addr learned my addr 172.21.15.29:0/2209463078 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.144+0000 7f50fc8ad640 1 --2- 172.21.15.29:0/2209463078 >> v2:172.21.15.157:3300/0 conn(0x7f50f80d2680 0x7f50f80d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.144+0000 7f50f77fe640 1 --2- 172.21.15.29:0/2209463078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f815e560 0x7f50f80d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:50.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.144+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1528831394 0 0) 0x7f50f80d3050 con 0x7f50f815e180 2024-04-25T00:41:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.144+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50e0003530 con 0x7f50f815e180 2024-04-25T00:41:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3512864275 0 0) 0x7f50e0003530 con 0x7f50f815e180 2024-04-25T00:41:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f815e560 msgr2=0x7f50f80d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 --2- 172.21.15.29:0/2209463078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f815e560 0x7f50f80d20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 >> v2:172.21.15.157:3300/0 conn(0x7f50f80d2680 msgr2=0x7f50f80d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 --2- 172.21.15.29:0/2209463078 >> v2:172.21.15.157:3300/0 conn(0x7f50f80d2680 0x7f50f80d2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f80b73e0 con 0x7f50f815e180 2024-04-25T00:41:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f50e8001680 con 0x7f50f815e180 2024-04-25T00:41:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50fc8ad640 1 --2- 172.21.15.29:0/2209463078 >> v2:172.21.15.157:3300/0 conn(0x7f50f80d2680 0x7f50f80d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 >> v1:172.21.15.29:6789/0 conn(0x7f50f815e180 legacy=0x7f50f80cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 --2- 172.21.15.29:0/2209463078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f815e560 0x7f50e00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 --2- 172.21.15.29:0/2209463078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50e0003f40 0x7f50e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 --2- 172.21.15.29:0/2209463078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0004d50 0x7f50e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f50f80cfed0 con 0x7f50e0003f40 2024-04-25T00:41:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50f80b7100 con 0x7f50e0004d50 2024-04-25T00:41:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f50e000ba00 con 0x7f50f815e560 2024-04-25T00:41:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f77fe640 1 --2- 172.21.15.29:0/2209463078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50e0003f40 0x7f50e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f7fff640 1 --2- 172.21.15.29:0/2209463078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f815e560 0x7f50e00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50fc8ad640 1 --2- 172.21.15.29:0/2209463078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0004d50 0x7f50e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f77fe640 1 -- 172.21.15.29:0/2209463078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f815e560 msgr2=0x7f50e00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f77fe640 1 --2- 172.21.15.29:0/2209463078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f815e560 0x7f50e00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f77fe640 1 -- 172.21.15.29:0/2209463078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0004d50 msgr2=0x7f50e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f77fe640 1 --2- 172.21.15.29:0/2209463078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0004d50 0x7f50e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f77fe640 1 -- 172.21.15.29:0/2209463078 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e0003530 con 0x7f50e0003f40 2024-04-25T00:41:50.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50fc8ad640 1 --2- 172.21.15.29:0/2209463078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0004d50 0x7f50e00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:50.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f77fe640 1 --2- 172.21.15.29:0/2209463078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50e0003f40 0x7f50e0004810 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009ae0 tx=0x7f50ec01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa7efdfe44dc367 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:50.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0b2070 con 0x7f50e0003f40 2024-04-25T00:41:50.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec0a5660 con 0x7f50e0003f40 2024-04-25T00:41:50.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.148+0000 7f50f6ffd640 1 -- 172.21.15.29:0/2209463078 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0096b0 con 0x7f50e0003f40 2024-04-25T00:41:50.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 -- 172.21.15.29:0/2209463078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50e0003f40 msgr2=0x7f50e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:50.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 --2- 172.21.15.29:0/2209463078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50e0003f40 0x7f50e0004810 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009ae0 tx=0x7f50ec01ca70 comp rx=0 tx=0).stop 2024-04-25T00:41:50.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fc8ad640 1 -- 172.21.15.29:0/2209463078 reap_dead start 2024-04-25T00:41:50.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 -- 172.21.15.29:0/2209463078 shutdown_connections 2024-04-25T00:41:50.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 -- 172.21.15.29:0/2209463078 >> 172.21.15.29:0/2209463078 conn(0x7f50f80d40a0 msgr2=0x7f50f80d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:50.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 -- 172.21.15.29:0/2209463078 shutdown_connections 2024-04-25T00:41:50.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 -- 172.21.15.29:0/2209463078 wait complete. 2024-04-25T00:41:50.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 Processor -- start 2024-04-25T00:41:50.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 -- start start 2024-04-25T00:41:50.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50e0003f40 0x7f50f80c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:50.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f814ec00 0x7f50f80b8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:50.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f814f080 0x7f50f80b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:50.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f50f80d2260 con 0x7f50f814f080 2024-04-25T00:41:50.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50f80d20e0 con 0x7f50f814ec00 2024-04-25T00:41:50.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f50f80d23e0 con 0x7f50e0003f40 2024-04-25T00:41:50.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fc8ad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f814f080 0x7f50f80b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:50.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fc8ad640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f814f080 0x7f50f80b8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56348/0 (socket says 172.21.15.29:56348) 2024-04-25T00:41:50.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fc8ad640 1 -- 172.21.15.29:0/3047299034 learned_addr learned my addr 172.21.15.29:0/3047299034 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:50.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50f7fff640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50e0003f40 0x7f50f80c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:50.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50f77fe640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f814ec00 0x7f50f80b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:50.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fc8ad640 1 -- 172.21.15.29:0/3047299034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50e0003f40 msgr2=0x7f50f80c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:50.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fc8ad640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50e0003f40 0x7f50f80c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:50.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fc8ad640 1 -- 172.21.15.29:0/3047299034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f814ec00 msgr2=0x7f50f80b8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:50.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fc8ad640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f814ec00 0x7f50f80b8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:50.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fc8ad640 1 -- 172.21.15.29:0/3047299034 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f8151d00 con 0x7f50f814f080 2024-04-25T00:41:50.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fc8ad640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f814f080 0x7f50f80b8e60 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f50f80d0640 tx=0x7f50f000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:50.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50f4ff9640 1 -- 172.21.15.29:0/3047299034 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f001c070 con 0x7f50f814f080 2024-04-25T00:41:50.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50f4ff9640 1 -- 172.21.15.29:0/3047299034 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f0017600 con 0x7f50f814f080 2024-04-25T00:41:50.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 -- 172.21.15.29:0/3047299034 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f814f500 con 0x7f50f814f080 2024-04-25T00:41:50.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50f4ff9640 1 -- 172.21.15.29:0/3047299034 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f00179d0 con 0x7f50f814f080 2024-04-25T00:41:50.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.152+0000 7f50fe310640 1 -- 172.21.15.29:0/3047299034 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f815ffd0 con 0x7f50f814f080 2024-04-25T00:41:50.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.156+0000 7f50fe310640 1 -- 172.21.15.29:0/3047299034 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f80692e0 con 0x7f50f814f080 2024-04-25T00:41:50.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.156+0000 7f50f4ff9640 1 -- 172.21.15.29:0/3047299034 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f50f001f070 con 0x7f50f814f080 2024-04-25T00:41:50.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.156+0000 7f50f4ff9640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f50e4068430 0x7f50e406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:50.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.156+0000 7f50f4ff9640 1 -- 172.21.15.29:0/3047299034 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5795+0+0 (secure 0 0 0) 0x7f50f0084980 con 0x7f50f814f080 2024-04-25T00:41:50.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.156+0000 7f50f7fff640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f50e4068430 0x7f50e406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:50.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.156+0000 7f50f7fff640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f50e4068430 0x7f50e406a8f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f50e8005a00 tx=0x7f50e8007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:50.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.160+0000 7f50f4ff9640 1 -- 172.21.15.29:0/3047299034 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f50f0012070 con 0x7f50f814f080 2024-04-25T00:41:50.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.320+0000 7f50fe310640 1 -- 172.21.15.29:0/3047299034 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f50f80c8920 con 0x7f50f814f080 2024-04-25T00:41:50.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.912+0000 7f50f4ff9640 1 -- 172.21.15.29:0/3047299034 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v170) ==== 181+0+0 (secure 0 0 0) 0x7f50f0056880 con 0x7f50f814f080 2024-04-25T00:41:50.915 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:41:50.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.924+0000 7f50fe310640 1 -- 172.21.15.29:0/3047299034 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f50e4068430 msgr2=0x7f50e406a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:50.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.924+0000 7f50fe310640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f50e4068430 0x7f50e406a8f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f50e8005a00 tx=0x7f50e8007040 comp rx=0 tx=0).stop 2024-04-25T00:41:50.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.924+0000 7f50fe310640 1 -- 172.21.15.29:0/3047299034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f814f080 msgr2=0x7f50f80b8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:50.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.924+0000 7f50fe310640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f814f080 0x7f50f80b8e60 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f50f80d0640 tx=0x7f50f000f790 comp rx=0 tx=0).stop 2024-04-25T00:41:50.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.924+0000 7f50fe310640 1 -- 172.21.15.29:0/3047299034 shutdown_connections 2024-04-25T00:41:50.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.924+0000 7f50fe310640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f50e4068430 0x7f50e406a8f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:50.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.924+0000 7f50fe310640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f814f080 0x7f50f80b8e60 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:50.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.924+0000 7f50fe310640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f814ec00 0x7f50f80b8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:50.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.924+0000 7f50fe310640 1 --2- 172.21.15.29:0/3047299034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50e0003f40 0x7f50f80c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:50.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.924+0000 7f50fe310640 1 -- 172.21.15.29:0/3047299034 >> 172.21.15.29:0/3047299034 conn(0x7f50f80d40a0 msgr2=0x7f50f80cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:50.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.924+0000 7f50fe310640 1 -- 172.21.15.29:0/3047299034 shutdown_connections 2024-04-25T00:41:50.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:50.924+0000 7f50fe310640 1 -- 172.21.15.29:0/3047299034 wait complete. 2024-04-25T00:41:50.944 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:41:51.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30d649f640 1 Processor -- start 2024-04-25T00:41:51.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30d649f640 1 -- start start 2024-04-25T00:41:51.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30d649f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f30d015c420 0x7f30d013f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:51.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30d649f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d006b210 0x7f30d006b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:51.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30d649f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30d013f130 con 0x7f30d013ed50 2024-04-25T00:41:51.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30d649f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f30d00cb5b0 con 0x7f30d015c420 2024-04-25T00:41:51.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30d649f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f30d015fa10 con 0x7f30d006b210 2024-04-25T00:41:51.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30cf7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f30d015c420 0x7f30d013f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:51.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30d4a3c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d006b210 0x7f30d006b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:51.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30cf7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f30d015c420 0x7f30d013f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:44004/0 (socket says 172.21.15.29:44004) 2024-04-25T00:41:51.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30cf7fe640 1 -- 172.21.15.29:0/3604244939 learned_addr learned my addr 172.21.15.29:0/3604244939 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:51.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30d4a3c640 1 -- 172.21.15.29:0/3604244939 >> v2:172.21.15.157:3300/0 conn(0x7f30d015c420 msgr2=0x7f30d013f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:51.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30d4a3c640 1 --2- 172.21.15.29:0/3604244939 >> v2:172.21.15.157:3300/0 conn(0x7f30d015c420 0x7f30d013f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:51.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30d4a3c640 1 -- 172.21.15.29:0/3604244939 >> v1:172.21.15.157:6789/0 conn(0x7f30d013ed50 legacy=0x7f30d0141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:51.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.080+0000 7f30d4a3c640 1 -- 172.21.15.29:0/3604244939 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d015c9c0 con 0x7f30d006b210 2024-04-25T00:41:51.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.084+0000 7f30d4a3c640 1 --2- 172.21.15.29:0/3604244939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d006b210 0x7f30d006b6a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009d90 tx=0x7f30c801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ff7255e7591f452 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:51.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.084+0000 7f30ceffd640 1 -- 172.21.15.29:0/3604244939 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c80b2070 con 0x7f30d006b210 2024-04-25T00:41:51.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.084+0000 7f30ceffd640 1 -- 172.21.15.29:0/3604244939 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c80a5940 con 0x7f30d006b210 2024-04-25T00:41:51.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.084+0000 7f30ceffd640 1 -- 172.21.15.29:0/3604244939 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c80a5d10 con 0x7f30d006b210 2024-04-25T00:41:51.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.084+0000 7f30d649f640 1 -- 172.21.15.29:0/3604244939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d006b210 msgr2=0x7f30d006b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:51.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.084+0000 7f30d649f640 1 --2- 172.21.15.29:0/3604244939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d006b210 0x7f30d006b6a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009d90 tx=0x7f30c801c970 comp rx=0 tx=0).stop 2024-04-25T00:41:51.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.084+0000 7f30d649f640 1 -- 172.21.15.29:0/3604244939 shutdown_connections 2024-04-25T00:41:51.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.084+0000 7f30d649f640 1 --2- 172.21.15.29:0/3604244939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d006b210 0x7f30d006b6a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:51.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.084+0000 7f30d649f640 1 --2- 172.21.15.29:0/3604244939 >> v2:172.21.15.157:3300/0 conn(0x7f30d015c420 0x7f30d013f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:51.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.084+0000 7f30d649f640 1 -- 172.21.15.29:0/3604244939 >> 172.21.15.29:0/3604244939 conn(0x7f30d00d0c30 msgr2=0x7f30d00d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:51.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.084+0000 7f30d649f640 1 -- 172.21.15.29:0/3604244939 shutdown_connections 2024-04-25T00:41:51.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.084+0000 7f30d649f640 1 -- 172.21.15.29:0/3604244939 wait complete. 2024-04-25T00:41:51.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30d649f640 1 Processor -- start 2024-04-25T00:41:51.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30d649f640 1 -- start start 2024-04-25T00:41:51.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30d649f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d006b210 0x7f30d006f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:51.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30d649f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d013ed50 0x7f30d014f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:51.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30d649f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d015c420 0x7f30d012f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:51.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30d649f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30d013fb30 con 0x7f30d006b210 2024-04-25T00:41:51.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30d649f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f30d013f9b0 con 0x7f30d013ed50 2024-04-25T00:41:51.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30d649f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f30d01606b0 con 0x7f30d015c420 2024-04-25T00:41:51.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30cffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d006b210 0x7f30d006f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:51.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30cffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d006b210 0x7f30d006f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56356/0 (socket says 172.21.15.29:56356) 2024-04-25T00:41:51.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30cffff640 1 -- 172.21.15.29:0/195967204 learned_addr learned my addr 172.21.15.29:0/195967204 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:51.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30d4a3c640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d015c420 0x7f30d012f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:51.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30cf7fe640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d013ed50 0x7f30d014f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:51.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30cffff640 1 -- 172.21.15.29:0/195967204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d015c420 msgr2=0x7f30d012f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:51.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30cffff640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d015c420 0x7f30d012f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:51.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30cffff640 1 -- 172.21.15.29:0/195967204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d013ed50 msgr2=0x7f30d014f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:51.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30cffff640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d013ed50 0x7f30d014f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:51.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30cffff640 1 -- 172.21.15.29:0/195967204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d014ff00 con 0x7f30d006b210 2024-04-25T00:41:51.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30cf7fe640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d013ed50 0x7f30d014f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:51.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30cffff640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d006b210 0x7f30d006f0a0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f30c000c450 tx=0x7f30c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:51.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30ccff9640 1 -- 172.21.15.29:0/195967204 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0016020 con 0x7f30d006b210 2024-04-25T00:41:51.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30ccff9640 1 -- 172.21.15.29:0/195967204 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c00052d0 con 0x7f30d006b210 2024-04-25T00:41:51.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30d649f640 1 -- 172.21.15.29:0/195967204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d012f8e0 con 0x7f30d006b210 2024-04-25T00:41:51.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30ccff9640 1 -- 172.21.15.29:0/195967204 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0007260 con 0x7f30d006b210 2024-04-25T00:41:51.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30d649f640 1 -- 172.21.15.29:0/195967204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d0151850 con 0x7f30d006b210 2024-04-25T00:41:51.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.088+0000 7f30d649f640 1 -- 172.21.15.29:0/195967204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d00692e0 con 0x7f30d006b210 2024-04-25T00:41:51.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.092+0000 7f30ccff9640 1 -- 172.21.15.29:0/195967204 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f30c0007400 con 0x7f30d006b210 2024-04-25T00:41:51.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.092+0000 7f30ccff9640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f30bc068400 0x7f30bc06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:51.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.092+0000 7f30ccff9640 1 -- 172.21.15.29:0/195967204 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5357+0+0 (secure 0 0 0) 0x7f30c00841e0 con 0x7f30d006b210 2024-04-25T00:41:51.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.092+0000 7f30cf7fe640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f30bc068400 0x7f30bc06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:51.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.092+0000 7f30cf7fe640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f30bc068400 0x7f30bc06a8c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f30d00b0aa0 tx=0x7f30c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:51.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.096+0000 7f30ccff9640 1 -- 172.21.15.29:0/195967204 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f30c0020020 con 0x7f30d006b210 2024-04-25T00:41:51.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.260+0000 7f30d649f640 1 -- 172.21.15.29:0/195967204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f30d00b25f0 con 0x7f30d006b210 2024-04-25T00:41:51.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.924+0000 7f30ccff9640 1 -- 172.21.15.29:0/195967204 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v171) ==== 169+0+0 (secure 0 0 0) 0x7f30c0005480 con 0x7f30d006b210 2024-04-25T00:41:51.924 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-04-25T00:41:51.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.924+0000 7f30d649f640 1 -- 172.21.15.29:0/195967204 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f30bc068400 msgr2=0x7f30bc06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:51.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.924+0000 7f30d649f640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f30bc068400 0x7f30bc06a8c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f30d00b0aa0 tx=0x7f30c4008040 comp rx=0 tx=0).stop 2024-04-25T00:41:51.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.924+0000 7f30d649f640 1 -- 172.21.15.29:0/195967204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d006b210 msgr2=0x7f30d006f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:51.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.924+0000 7f30d649f640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d006b210 0x7f30d006f0a0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f30c000c450 tx=0x7f30c000c910 comp rx=0 tx=0).stop 2024-04-25T00:41:51.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.924+0000 7f30d649f640 1 -- 172.21.15.29:0/195967204 shutdown_connections 2024-04-25T00:41:51.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.924+0000 7f30d649f640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f30bc068400 0x7f30bc06a8c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:51.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.924+0000 7f30d649f640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d015c420 0x7f30d012f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:51.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.924+0000 7f30d649f640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d013ed50 0x7f30d014f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:51.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.924+0000 7f30d649f640 1 --2- 172.21.15.29:0/195967204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d006b210 0x7f30d006f0a0 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:51.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.924+0000 7f30d649f640 1 -- 172.21.15.29:0/195967204 >> 172.21.15.29:0/195967204 conn(0x7f30d00d0c30 msgr2=0x7f30d00c7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:51.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.928+0000 7f30d649f640 1 -- 172.21.15.29:0/195967204 shutdown_connections 2024-04-25T00:41:51.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:51.928+0000 7f30d649f640 1 -- 172.21.15.29:0/195967204 wait complete. 2024-04-25T00:41:51.952 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:41:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.080+0000 7fd602b2a640 1 Processor -- start 2024-04-25T00:41:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.080+0000 7fd602b2a640 1 -- start start 2024-04-25T00:41:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.080+0000 7fd602b2a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc0b3140 0x7fd5fc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd602b2a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd5fc0b4b10 0x7fd5fc0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd602b2a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5fc0b41e0 con 0x7fd5fc0b3a60 2024-04-25T00:41:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd602b2a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd5fc06d960 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd602b2a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd5fc06dc40 con 0x7fd5fc0b3140 2024-04-25T00:41:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd5fbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd5fc0b4b10 0x7fd5fc0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd5fbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd5fc0b4b10 0x7fd5fc0b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56372/0 (socket says 172.21.15.29:56372) 2024-04-25T00:41:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd5fbfff640 1 -- 172.21.15.29:0/624056575 learned_addr learned my addr 172.21.15.29:0/624056575 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:52.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd5fbfff640 1 -- 172.21.15.29:0/624056575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc0b3140 msgr2=0x7fd5fc06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:52.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd6008c6640 1 --2- 172.21.15.29:0/624056575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc0b3140 0x7fd5fc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:52.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd5fbfff640 1 --2- 172.21.15.29:0/624056575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc0b3140 0x7fd5fc06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd5fbfff640 1 -- 172.21.15.29:0/624056575 >> v1:172.21.15.157:6789/0 conn(0x7fd5fc0b3a60 legacy=0x7fd5fc15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:52.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd5fbfff640 1 -- 172.21.15.29:0/624056575 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc06df20 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd5fbfff640 1 --2- 172.21.15.29:0/624056575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc0b4b10 0x7fd5fc0b3520 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009870 tx=0x7fd5ec01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2f56671ee6b174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:52.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd5fb7fe640 1 -- 172.21.15.29:0/624056575 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec0b0070 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd5fb7fe640 1 -- 172.21.15.29:0/624056575 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5ec0ab600 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd5fb7fe640 1 -- 172.21.15.29:0/624056575 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec0a4e60 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd602b2a640 1 -- 172.21.15.29:0/624056575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc0b4b10 msgr2=0x7fd5fc0b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd602b2a640 1 --2- 172.21.15.29:0/624056575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc0b4b10 0x7fd5fc0b3520 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009870 tx=0x7fd5ec01c520 comp rx=0 tx=0).stop 2024-04-25T00:41:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd602b2a640 1 -- 172.21.15.29:0/624056575 shutdown_connections 2024-04-25T00:41:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd602b2a640 1 --2- 172.21.15.29:0/624056575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc0b4b10 0x7fd5fc0b3520 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd602b2a640 1 --2- 172.21.15.29:0/624056575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc0b3140 0x7fd5fc06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd602b2a640 1 -- 172.21.15.29:0/624056575 >> 172.21.15.29:0/624056575 conn(0x7fd5fc0b1870 msgr2=0x7fd5fc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd602b2a640 1 -- 172.21.15.29:0/624056575 shutdown_connections 2024-04-25T00:41:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.084+0000 7fd602b2a640 1 -- 172.21.15.29:0/624056575 wait complete. 2024-04-25T00:41:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd602b2a640 1 Processor -- start 2024-04-25T00:41:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd602b2a640 1 -- start start 2024-04-25T00:41:52.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd602b2a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc0b3140 0x7fd5fc133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd602b2a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc0b3a60 0x7fd5fc140700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd602b2a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc0b4b10 0x7fd5fc140c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd602b2a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc168b70 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd602b2a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1689f0 con 0x7fd5fc0b3a60 2024-04-25T00:41:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd602b2a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd5fc168cf0 con 0x7fd5fc0b3140 2024-04-25T00:41:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd6010c7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc0b4b10 0x7fd5fc140c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd6010c7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc0b4b10 0x7fd5fc140c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56380/0 (socket says 172.21.15.29:56380) 2024-04-25T00:41:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd6010c7640 1 -- 172.21.15.29:0/4052670984 learned_addr learned my addr 172.21.15.29:0/4052670984 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd6008c6640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc0b3140 0x7fd5fc133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd5fbfff640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc0b3a60 0x7fd5fc140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd6010c7640 1 -- 172.21.15.29:0/4052670984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc0b3140 msgr2=0x7fd5fc133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd6010c7640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc0b3140 0x7fd5fc133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd6010c7640 1 -- 172.21.15.29:0/4052670984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc0b3a60 msgr2=0x7fd5fc140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd6010c7640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc0b3a60 0x7fd5fc140700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd6010c7640 1 -- 172.21.15.29:0/4052670984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc1412e0 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd5fbfff640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc0b3a60 0x7fd5fc140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd6010c7640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc0b4b10 0x7fd5fc140c40 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400c780 tx=0x7fd5f400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd5f97fa640 1 -- 172.21.15.29:0/4052670984 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4004b70 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd5f97fa640 1 -- 172.21.15.29:0/4052670984 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f4004d10 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd602b2a640 1 -- 172.21.15.29:0/4052670984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5fc1415c0 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd5f97fa640 1 -- 172.21.15.29:0/4052670984 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400aa90 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd602b2a640 1 -- 172.21.15.29:0/4052670984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5fc141b80 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.088+0000 7fd602b2a640 1 -- 172.21.15.29:0/4052670984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5fc069270 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.092+0000 7fd5f97fa640 1 -- 172.21.15.29:0/4052670984 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fd5f400ac30 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.092+0000 7fd5f97fa640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd5dc068400 0x7fd5dc06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:52.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.092+0000 7fd6008c6640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd5dc068400 0x7fd5dc06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:52.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.092+0000 7fd5f97fa640 1 -- 172.21.15.29:0/4052670984 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7fd5f4084b30 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.092+0000 7fd6008c6640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd5dc068400 0x7fd5dc06a8c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400a8d0 tx=0x7fd5e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:52.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.096+0000 7fd5f97fa640 1 -- 172.21.15.29:0/4052670984 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd5f4020070 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.256+0000 7fd602b2a640 1 -- 172.21.15.29:0/4052670984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd5fc0cf700 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.260+0000 7fd5f97fa640 1 -- 172.21.15.29:0/4052670984 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7fd5f4056ed0 con 0x7fd5fc0b4b10 2024-04-25T00:41:52.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.264+0000 7fd602b2a640 1 -- 172.21.15.29:0/4052670984 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd5dc068400 msgr2=0x7fd5dc06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:52.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.264+0000 7fd602b2a640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd5dc068400 0x7fd5dc06a8c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400a8d0 tx=0x7fd5e4008040 comp rx=0 tx=0).stop 2024-04-25T00:41:52.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.264+0000 7fd602b2a640 1 -- 172.21.15.29:0/4052670984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc0b4b10 msgr2=0x7fd5fc140c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:52.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.264+0000 7fd602b2a640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc0b4b10 0x7fd5fc140c40 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400c780 tx=0x7fd5f400cc40 comp rx=0 tx=0).stop 2024-04-25T00:41:52.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.264+0000 7fd602b2a640 1 -- 172.21.15.29:0/4052670984 shutdown_connections 2024-04-25T00:41:52.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.264+0000 7fd602b2a640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fd5dc068400 0x7fd5dc06a8c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.264+0000 7fd602b2a640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5fc0b4b10 0x7fd5fc140c40 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.264+0000 7fd602b2a640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5fc0b3a60 0x7fd5fc140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.264+0000 7fd602b2a640 1 --2- 172.21.15.29:0/4052670984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5fc0b3140 0x7fd5fc133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.264+0000 7fd602b2a640 1 -- 172.21.15.29:0/4052670984 >> 172.21.15.29:0/4052670984 conn(0x7fd5fc0b1870 msgr2=0x7fd5fc142aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:52.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.264+0000 7fd602b2a640 1 -- 172.21.15.29:0/4052670984 shutdown_connections 2024-04-25T00:41:52.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.264+0000 7fd602b2a640 1 -- 172.21.15.29:0/4052670984 wait complete. 2024-04-25T00:41:52.282 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-04-25T00:41:52.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.408+0000 7ff1cab2c640 1 Processor -- start 2024-04-25T00:41:52.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.408+0000 7ff1cab2c640 1 -- start start 2024-04-25T00:41:52.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1cab2c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff1c415d980 0x7ff1c406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:52.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1cab2c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1c406db10 0x7ff1c4151500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:52.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1cab2c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c40b3130 con 0x7ff1c40b2b70 2024-04-25T00:41:52.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1cab2c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff1c40cf560 con 0x7ff1c415d980 2024-04-25T00:41:52.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1cab2c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c40b3410 con 0x7ff1c406db10 2024-04-25T00:41:52.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1c88c8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff1c415d980 0x7ff1c406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:52.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1c88c8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff1c415d980 0x7ff1c406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56388/0 (socket says 172.21.15.29:56388) 2024-04-25T00:41:52.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1c88c8640 1 -- 172.21.15.29:0/620509125 learned_addr learned my addr 172.21.15.29:0/620509125 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:52.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1c90c9640 1 --2- 172.21.15.29:0/620509125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1c406db10 0x7ff1c4151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:52.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1c88c8640 1 -- 172.21.15.29:0/620509125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1c406db10 msgr2=0x7ff1c4151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:52.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1c88c8640 1 --2- 172.21.15.29:0/620509125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1c406db10 0x7ff1c4151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1c88c8640 1 -- 172.21.15.29:0/620509125 >> v1:172.21.15.157:6789/0 conn(0x7ff1c40b2b70 legacy=0x7ff1c4150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:52.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1c88c8640 1 -- 172.21.15.29:0/620509125 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c40b36f0 con 0x7ff1c415d980 2024-04-25T00:41:52.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.412+0000 7ff1c88c8640 1 --2- 172.21.15.29:0/620509125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c415d980 0x7ff1c406d5d0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009e50 tx=0x7ff1b401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce207ac434392c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:52.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1c37fe640 1 -- 172.21.15.29:0/620509125 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40a5680 con 0x7ff1c415d980 2024-04-25T00:41:52.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1c37fe640 1 -- 172.21.15.29:0/620509125 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b40a5820 con 0x7ff1c415d980 2024-04-25T00:41:52.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1c37fe640 1 -- 172.21.15.29:0/620509125 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40a5bf0 con 0x7ff1c415d980 2024-04-25T00:41:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 -- 172.21.15.29:0/620509125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c415d980 msgr2=0x7ff1c406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 --2- 172.21.15.29:0/620509125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c415d980 0x7ff1c406d5d0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009e50 tx=0x7ff1b401ca30 comp rx=0 tx=0).stop 2024-04-25T00:41:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 -- 172.21.15.29:0/620509125 shutdown_connections 2024-04-25T00:41:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 --2- 172.21.15.29:0/620509125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1c406db10 0x7ff1c4151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 --2- 172.21.15.29:0/620509125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c415d980 0x7ff1c406d5d0 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 -- 172.21.15.29:0/620509125 >> 172.21.15.29:0/620509125 conn(0x7ff1c40b1870 msgr2=0x7ff1c40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:52.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 -- 172.21.15.29:0/620509125 shutdown_connections 2024-04-25T00:41:52.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 -- 172.21.15.29:0/620509125 wait complete. 2024-04-25T00:41:52.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 Processor -- start 2024-04-25T00:41:52.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 -- start start 2024-04-25T00:41:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1c406db10 0x7ff1c4143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1c40b2b70 0x7ff1c41497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c415d980 0x7ff1c4142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c415b630 con 0x7ff1c415d980 2024-04-25T00:41:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c415b4b0 con 0x7ff1c40b2b70 2024-04-25T00:41:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1cab2c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c415b7b0 con 0x7ff1c406db10 2024-04-25T00:41:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1c90c9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c415d980 0x7ff1c4142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1c90c9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c415d980 0x7ff1c4142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56402/0 (socket says 172.21.15.29:56402) 2024-04-25T00:41:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.416+0000 7ff1c90c9640 1 -- 172.21.15.29:0/19165448 learned_addr learned my addr 172.21.15.29:0/19165448 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c3fff640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1c40b2b70 0x7ff1c41497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c88c8640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1c406db10 0x7ff1c4143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c90c9640 1 -- 172.21.15.29:0/19165448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1c406db10 msgr2=0x7ff1c4143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c90c9640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1c406db10 0x7ff1c4143d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c90c9640 1 -- 172.21.15.29:0/19165448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1c40b2b70 msgr2=0x7ff1c41497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c90c9640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1c40b2b70 0x7ff1c41497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c90c9640 1 -- 172.21.15.29:0/19165448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c41443f0 con 0x7ff1c415d980 2024-04-25T00:41:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c90c9640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c415d980 0x7ff1c4142ac0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00e930 tx=0x7ff1bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c17fa640 1 -- 172.21.15.29:0/19165448 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc017070 con 0x7ff1c415d980 2024-04-25T00:41:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c17fa640 1 -- 172.21.15.29:0/19165448 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1bc00c9a0 con 0x7ff1c415d980 2024-04-25T00:41:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1cab2c640 1 -- 172.21.15.29:0/19165448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c4150440 con 0x7ff1c415d980 2024-04-25T00:41:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c17fa640 1 -- 172.21.15.29:0/19165448 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc00cd70 con 0x7ff1c415d980 2024-04-25T00:41:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1cab2c640 1 -- 172.21.15.29:0/19165448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c4150a00 con 0x7ff1c415d980 2024-04-25T00:41:52.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1cab2c640 1 -- 172.21.15.29:0/19165448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c4150270 con 0x7ff1c415d980 2024-04-25T00:41:52.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c17fa640 1 -- 172.21.15.29:0/19165448 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7ff1bc01a070 con 0x7ff1c415d980 2024-04-25T00:41:52.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c17fa640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff1b0068400 0x7ff1b006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:52.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c17fa640 1 -- 172.21.15.29:0/19165448 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7ff1bc085060 con 0x7ff1c415d980 2024-04-25T00:41:52.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c88c8640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff1b0068400 0x7ff1b006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:52.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.420+0000 7ff1c88c8640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff1b0068400 0x7ff1b006a8c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40ae000 tx=0x7ff1b40a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:52.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.428+0000 7ff1c17fa640 1 -- 172.21.15.29:0/19165448 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff1bc0574a0 con 0x7ff1c415d980 2024-04-25T00:41:52.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.660+0000 7ff1cab2c640 1 -- 172.21.15.29:0/19165448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7ff1c4069270 con 0x7ff1c415d980 2024-04-25T00:41:52.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.924+0000 7ff1c17fa640 1 -- 172.21.15.29:0/19165448 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v556) ==== 163+0+0 (secure 0 0 0) 0x7ff1bc056da0 con 0x7ff1c415d980 2024-04-25T00:41:52.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.928+0000 7ff1cab2c640 1 -- 172.21.15.29:0/19165448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff1b0068400 msgr2=0x7ff1b006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:52.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.928+0000 7ff1cab2c640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff1b0068400 0x7ff1b006a8c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40ae000 tx=0x7ff1b40a7070 comp rx=0 tx=0).stop 2024-04-25T00:41:52.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.928+0000 7ff1cab2c640 1 -- 172.21.15.29:0/19165448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c415d980 msgr2=0x7ff1c4142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:52.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.928+0000 7ff1cab2c640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c415d980 0x7ff1c4142ac0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00e930 tx=0x7ff1bc00edf0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.928+0000 7ff1cab2c640 1 -- 172.21.15.29:0/19165448 shutdown_connections 2024-04-25T00:41:52.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.928+0000 7ff1cab2c640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff1b0068400 0x7ff1b006a8c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.928+0000 7ff1cab2c640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1c415d980 0x7ff1c4142ac0 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.928+0000 7ff1cab2c640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1c40b2b70 0x7ff1c41497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.928+0000 7ff1cab2c640 1 --2- 172.21.15.29:0/19165448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1c406db10 0x7ff1c4143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:52.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.928+0000 7ff1cab2c640 1 -- 172.21.15.29:0/19165448 >> 172.21.15.29:0/19165448 conn(0x7ff1c40b1870 msgr2=0x7ff1c415bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:52.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.928+0000 7ff1cab2c640 1 -- 172.21.15.29:0/19165448 shutdown_connections 2024-04-25T00:41:52.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:52.928+0000 7ff1cab2c640 1 -- 172.21.15.29:0/19165448 wait complete. 2024-04-25T00:41:52.949 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2024-04-25T00:41:52.950 INFO:tasks.cephfs_test_runner: 2024-04-25T00:41:52.950 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T00:41:52.950 INFO:tasks.cephfs_test_runner:Ran 11 tests in 588.818s 2024-04-25T00:41:52.950 INFO:tasks.cephfs_test_runner: 2024-04-25T00:41:52.950 INFO:tasks.cephfs_test_runner:OK 2024-04-25T00:41:52.950 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T00:41:52.960 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T00:41:52.974 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T00:41:52.975 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:41:52.975 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:41:52.981 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:41:52.981 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-25T00:41:52.981 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:41:53.032 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339600k,nr_inodes=4084900,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17961 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:41:53.032 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:41:53.032 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:41:53.038 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:41:53.039 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-25T00:41:53.039 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:41:53.086 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18867 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:41:53.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:53.086 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-25T00:41:53.135 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:41:53.135 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:41:53.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:53.135 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:41:53.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:53.191 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:41:53.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:53.263 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-04-25T00:41:53.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:53.436 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-25T00:41:53.443 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:41:53.443 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:41:53.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:53.443 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:41:53.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:53.501 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:41:53.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:41:53.581 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link delete ceph-brx 2024-04-25T00:41:53.725 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T00:41:53.734 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T00:41:53.734 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:41:53.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa7c68640 1 Processor -- start 2024-04-25T00:41:53.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa7c68640 1 -- start start 2024-04-25T00:41:53.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa7c68640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7aa00b4780 0x7f7aa015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa7c68640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7aa00cc240 0x7f7aa015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa7c68640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aa00b3720 con 0x7f7aa015e8d0 2024-04-25T00:41:53.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa7c68640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7aa00cc650 con 0x7f7aa00b4780 2024-04-25T00:41:53.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa7c68640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7aa00b42f0 con 0x7f7aa00cc240 2024-04-25T00:41:53.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa5203640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7aa00b4780 0x7f7aa015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa5203640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7aa00b4780 0x7f7aa015f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56418/0 (socket says 172.21.15.29:56418) 2024-04-25T00:41:53.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa5203640 1 -- 172.21.15.29:0/3615353602 learned_addr learned my addr 172.21.15.29:0/3615353602 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:53.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa5203640 1 -- 172.21.15.29:0/3615353602 >> v2:172.21.15.157:3300/0 conn(0x7f7aa00cc240 msgr2=0x7f7aa015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:53.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa5203640 1 --2- 172.21.15.29:0/3615353602 >> v2:172.21.15.157:3300/0 conn(0x7f7aa00cc240 0x7f7aa015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa5203640 1 -- 172.21.15.29:0/3615353602 >> v1:172.21.15.157:6789/0 conn(0x7f7aa015e8d0 legacy=0x7f7aa015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:53.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa5203640 1 -- 172.21.15.29:0/3615353602 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa00b39c0 con 0x7f7aa00b4780 2024-04-25T00:41:53.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.860+0000 7f7aa5203640 1 --2- 172.21.15.29:0/3615353602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7aa00b4780 0x7f7aa015f3c0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f7aa015f9a0 tx=0x7f7a9001eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccc996fe5a9ffc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:53.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa4a02640 1 -- 172.21.15.29:0/3615353602 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900b4070 con 0x7f7aa00b4780 2024-04-25T00:41:53.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa4a02640 1 -- 172.21.15.29:0/3615353602 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a900a7c50 con 0x7f7aa00b4780 2024-04-25T00:41:53.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa4a02640 1 -- 172.21.15.29:0/3615353602 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900af440 con 0x7f7aa00b4780 2024-04-25T00:41:53.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 -- 172.21.15.29:0/3615353602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7aa00b4780 msgr2=0x7f7aa015f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:53.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 --2- 172.21.15.29:0/3615353602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7aa00b4780 0x7f7aa015f3c0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f7aa015f9a0 tx=0x7f7a9001eb80 comp rx=0 tx=0).stop 2024-04-25T00:41:53.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 -- 172.21.15.29:0/3615353602 shutdown_connections 2024-04-25T00:41:53.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 --2- 172.21.15.29:0/3615353602 >> v2:172.21.15.157:3300/0 conn(0x7f7aa00cc240 0x7f7aa015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 --2- 172.21.15.29:0/3615353602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7aa00b4780 0x7f7aa015f3c0 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 -- 172.21.15.29:0/3615353602 >> 172.21.15.29:0/3615353602 conn(0x7f7aa00b1870 msgr2=0x7f7aa00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:53.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 -- 172.21.15.29:0/3615353602 shutdown_connections 2024-04-25T00:41:53.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 -- 172.21.15.29:0/3615353602 wait complete. 2024-04-25T00:41:53.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 Processor -- start 2024-04-25T00:41:53.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 -- start start 2024-04-25T00:41:53.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa00b4780 0x7f7aa0158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7aa00cc240 0x7f7aa0136170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015e8d0 0x7f7aa0133620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa0144a20 con 0x7f7aa00cc240 2024-04-25T00:41:53.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa015d520 con 0x7f7aa00b4780 2024-04-25T00:41:53.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa01588d0 con 0x7f7aa015e8d0 2024-04-25T00:41:53.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa5203640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7aa00cc240 0x7f7aa0136170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa5203640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7aa00cc240 0x7f7aa0136170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56432/0 (socket says 172.21.15.29:56432) 2024-04-25T00:41:53.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa5203640 1 -- 172.21.15.29:0/2048922944 learned_addr learned my addr 172.21.15.29:0/2048922944 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:53.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa6205640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015e8d0 0x7f7aa0133620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa5a04640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa00b4780 0x7f7aa0158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa5203640 1 -- 172.21.15.29:0/2048922944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015e8d0 msgr2=0x7f7aa0133620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:53.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa5203640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015e8d0 0x7f7aa0133620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa5203640 1 -- 172.21.15.29:0/2048922944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa00b4780 msgr2=0x7f7aa0158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:53.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa5203640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa00b4780 0x7f7aa0158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:53.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa5203640 1 -- 172.21.15.29:0/2048922944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa01366b0 con 0x7f7aa00cc240 2024-04-25T00:41:53.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa5203640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7aa00cc240 0x7f7aa0136170 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000b2c0 tx=0x7f7a9001b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:53.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7a967fc640 1 -- 172.21.15.29:0/2048922944 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900b4040 con 0x7f7aa00cc240 2024-04-25T00:41:53.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7a967fc640 1 -- 172.21.15.29:0/2048922944 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a900c1070 con 0x7f7aa00cc240 2024-04-25T00:41:53.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7aa7c68640 1 -- 172.21.15.29:0/2048922944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa00cd740 con 0x7f7aa00cc240 2024-04-25T00:41:53.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.864+0000 7f7a967fc640 1 -- 172.21.15.29:0/2048922944 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900afca0 con 0x7f7aa00cc240 2024-04-25T00:41:53.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.868+0000 7f7aa7c68640 1 -- 172.21.15.29:0/2048922944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa00cdd00 con 0x7f7aa00cc240 2024-04-25T00:41:53.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.868+0000 7f7aa7c68640 1 -- 172.21.15.29:0/2048922944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa0069270 con 0x7f7aa00cc240 2024-04-25T00:41:53.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.868+0000 7f7a967fc640 1 -- 172.21.15.29:0/2048922944 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f7a900b7070 con 0x7f7aa00cc240 2024-04-25T00:41:53.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.868+0000 7f7a967fc640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7a84068400 0x7f7a8406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:53.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.868+0000 7f7a967fc640 1 -- 172.21.15.29:0/2048922944 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f7a901215a0 con 0x7f7aa00cc240 2024-04-25T00:41:53.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.868+0000 7f7aa5a04640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7a84068400 0x7f7a8406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:53.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.868+0000 7f7aa5a04640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7a84068400 0x7f7a8406a8c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00a790 tx=0x7f7a9c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:53.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:53.872+0000 7f7a967fc640 1 -- 172.21.15.29:0/2048922944 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7a900187a0 con 0x7f7aa00cc240 2024-04-25T00:41:54.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.032+0000 7f7aa7c68640 1 -- 172.21.15.29:0/2048922944 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7aa006e7b0 con 0x7f7a84068400 2024-04-25T00:41:54.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.032+0000 7f7a967fc640 1 -- 172.21.15.29:0/2048922944 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7f7aa006e7b0 con 0x7f7a84068400 2024-04-25T00:41:54.034 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:54.034 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:41:54.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.036+0000 7f7aa7c68640 1 -- 172.21.15.29:0/2048922944 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7a84068400 msgr2=0x7f7a8406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.036+0000 7f7aa7c68640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7a84068400 0x7f7a8406a8c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00a790 tx=0x7f7a9c008040 comp rx=0 tx=0).stop 2024-04-25T00:41:54.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.036+0000 7f7aa7c68640 1 -- 172.21.15.29:0/2048922944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7aa00cc240 msgr2=0x7f7aa0136170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.036+0000 7f7aa7c68640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7aa00cc240 0x7f7aa0136170 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000b2c0 tx=0x7f7a9001b920 comp rx=0 tx=0).stop 2024-04-25T00:41:54.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.036+0000 7f7aa7c68640 1 -- 172.21.15.29:0/2048922944 shutdown_connections 2024-04-25T00:41:54.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.036+0000 7f7aa7c68640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7a84068400 0x7f7a8406a8c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.036+0000 7f7aa7c68640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015e8d0 0x7f7aa0133620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.036+0000 7f7aa7c68640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7aa00cc240 0x7f7aa0136170 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.036+0000 7f7aa7c68640 1 --2- 172.21.15.29:0/2048922944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa00b4780 0x7f7aa0158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.036+0000 7f7aa7c68640 1 -- 172.21.15.29:0/2048922944 >> 172.21.15.29:0/2048922944 conn(0x7f7aa00b1870 msgr2=0x7f7aa00b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:54.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.036+0000 7f7aa7c68640 1 -- 172.21.15.29:0/2048922944 shutdown_connections 2024-04-25T00:41:54.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.036+0000 7f7aa7c68640 1 -- 172.21.15.29:0/2048922944 wait complete. 2024-04-25T00:41:54.055 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-04-25T00:41:53.429819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":317256,"kb_used_data":17924,"kb_used_omap":12,"kb_used_meta":299251,"kb_avail":754657464,"statfs":{"total":773094113280,"available":772769243136,"internally_reserved":0,"allocated":18354176,"data_stored":8806579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":306433584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53200,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-78,"num_write_kb":-82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001822"},"pg_stats":[{"pgid":"1.0","version":"73'128","reported_seq":490,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725087+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:41:21.725087+0000","last_peered":"2024-04-25T00:41:21.725087+0000","last_clean":"2024-04-25T00:41:21.725087+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:41:21.725087+0000","last_undegraded":"2024-04-25T00:41:21.725087+0000","last_fullsized":"2024-04-25T00:41:21.725087+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38052,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333788,"statfs":{"total":96636764160,"available":96597798912,"internally_reserved":0,"allocated":2781184,"data_stored":1585862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37416,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334424,"statfs":{"total":96636764160,"available":96598450176,"internally_reserved":0,"allocated":2129920,"data_stored":938647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574977,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2138112,"data_stored":944425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41644,"kb_used_data":2148,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330196,"statfs":{"total":96636764160,"available":96594120704,"internally_reserved":0,"allocated":2199552,"data_stored":995901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574977,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41580,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330260,"statfs":{"total":96636764160,"available":96594186240,"internally_reserved":0,"allocated":2134016,"data_stored":937944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":2121728,"data_stored":937762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41576,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330264,"statfs":{"total":96636764160,"available":96594190336,"internally_reserved":0,"allocated":2129920,"data_stored":937818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42152,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329688,"statfs":{"total":96636764160,"available":96593600512,"internally_reserved":0,"allocated":2719744,"data_stored":1528220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:41:54.055 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:41:54.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa20623b640 1 Processor -- start 2024-04-25T00:41:54.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa20623b640 1 -- start start 2024-04-25T00:41:54.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa20623b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa2000b8790 0x7fa2001692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa20623b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2000d2cd0 con 0x7fa2000d4bf0 2024-04-25T00:41:54.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa20623b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2000cfe30 con 0x7fa20015e4e0 2024-04-25T00:41:54.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa20623b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa2000b6f80 con 0x7fa2000b8790 2024-04-25T00:41:54.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1feffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa2000d4bf0 0x7fa2000d25c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55768/0 (socket says 172.21.15.29:55768) 2024-04-25T00:41:54.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1feffd640 1 -- 172.21.15.29:0/1586882755 learned_addr learned my addr 172.21.15.29:0/1586882755 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:54.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3741685385 0 0) 0x7fa2000d2cd0 con 0x7fa2000d4bf0 2024-04-25T00:41:54.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1e8003530 con 0x7fa2000d4bf0 2024-04-25T00:41:54.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fffff640 1 --2- 172.21.15.29:0/1586882755 >> v2:172.21.15.157:3300/0 conn(0x7fa2000b8790 0x7fa2001692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1789902161 0 0) 0x7fa1e8003530 con 0x7fa2000d4bf0 2024-04-25T00:41:54.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 >> v2:172.21.15.157:3300/0 conn(0x7fa2000b8790 msgr2=0x7fa2001692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:54.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 --2- 172.21.15.29:0/1586882755 >> v2:172.21.15.157:3300/0 conn(0x7fa2000b8790 0x7fa2001692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 >> v1:172.21.15.157:6789/0 conn(0x7fa20015e4e0 legacy=0x7fa2000d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2000b7260 con 0x7fa2000d4bf0 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fa1f4001680 con 0x7fa2000d4bf0 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 >> v1:172.21.15.29:6789/0 conn(0x7fa2000d4bf0 legacy=0x7fa2000d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 --2- 172.21.15.29:0/1586882755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1e80039c0 0x7fa1e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 --2- 172.21.15.29:0/1586882755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1e8004810 0x7fa1e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 --2- 172.21.15.29:0/1586882755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1e80051d0 0x7fa1e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa2000b6f80 con 0x7fa1e8004810 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e800c100 con 0x7fa1e80051d0 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e800c300 con 0x7fa1e80039c0 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1feffd640 1 --2- 172.21.15.29:0/1586882755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1e8004810 0x7fa1e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1feffd640 1 -- 172.21.15.29:0/1586882755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1e80039c0 msgr2=0x7fa1e8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1feffd640 1 --2- 172.21.15.29:0/1586882755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1e80039c0 0x7fa1e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fffff640 1 --2- 172.21.15.29:0/1586882755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1e80051d0 0x7fa1e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1feffd640 1 -- 172.21.15.29:0/1586882755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1e80051d0 msgr2=0x7fa1e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:54.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1feffd640 1 --2- 172.21.15.29:0/1586882755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1e80051d0 0x7fa1e800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1fffff640 1 -- 172.21.15.29:0/1586882755 reap_dead start 2024-04-25T00:41:54.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.180+0000 7fa1feffd640 1 -- 172.21.15.29:0/1586882755 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e8003530 con 0x7fa1e8004810 2024-04-25T00:41:54.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.184+0000 7fa1feffd640 1 --2- 172.21.15.29:0/1586882755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1e8004810 0x7fa1e8004c90 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400c210 tx=0x7fa1f401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56acf710906e3bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.184+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4007d40 con 0x7fa1e8004810 2024-04-25T00:41:54.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.184+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f40a9070 con 0x7fa1e8004810 2024-04-25T00:41:54.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.184+0000 7fa1fe7fc640 1 -- 172.21.15.29:0/1586882755 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f40a5e40 con 0x7fa1e8004810 2024-04-25T00:41:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.184+0000 7fa20623b640 1 -- 172.21.15.29:0/1586882755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1e8004810 msgr2=0x7fa1e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.184+0000 7fa20623b640 1 --2- 172.21.15.29:0/1586882755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1e8004810 0x7fa1e8004c90 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400c210 tx=0x7fa1f401cca0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.184+0000 7fa20623b640 1 -- 172.21.15.29:0/1586882755 shutdown_connections 2024-04-25T00:41:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.184+0000 7fa20623b640 1 --2- 172.21.15.29:0/1586882755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1e8004810 0x7fa1e8004c90 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.184+0000 7fa20623b640 1 -- 172.21.15.29:0/1586882755 >> 172.21.15.29:0/1586882755 conn(0x7fa2000d3fb0 msgr2=0x7fa2000d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.184+0000 7fa20623b640 1 -- 172.21.15.29:0/1586882755 shutdown_connections 2024-04-25T00:41:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.184+0000 7fa20623b640 1 -- 172.21.15.29:0/1586882755 wait complete. 2024-04-25T00:41:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.184+0000 7fa20623b640 1 Processor -- start 2024-04-25T00:41:54.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.184+0000 7fa20623b640 1 -- start start 2024-04-25T00:41:54.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa20623b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1e8004810 0x7fa200160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa20623b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2000ca570 0x7fa2000c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa20623b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa20015efa0 0x7fa2000cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa20623b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa20015ea40 con 0x7fa1e8004810 2024-04-25T00:41:54.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa20623b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa20015e8c0 con 0x7fa20015efa0 2024-04-25T00:41:54.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa20623b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa20015ebc0 con 0x7fa2000ca570 2024-04-25T00:41:54.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1ff7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1e8004810 0x7fa200160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1ff7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1e8004810 0x7fa200160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56450/0 (socket says 172.21.15.29:56450) 2024-04-25T00:41:54.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1ff7fe640 1 -- 172.21.15.29:0/3434879407 learned_addr learned my addr 172.21.15.29:0/3434879407 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:54.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1feffd640 1 --2- 172.21.15.29:0/3434879407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2000ca570 0x7fa2000c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1ff7fe640 1 -- 172.21.15.29:0/3434879407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2000ca570 msgr2=0x7fa2000c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1ff7fe640 1 --2- 172.21.15.29:0/3434879407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2000ca570 0x7fa2000c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1ff7fe640 1 -- 172.21.15.29:0/3434879407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa20015efa0 msgr2=0x7fa2000cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1ff7fe640 1 --2- 172.21.15.29:0/3434879407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa20015efa0 0x7fa2000cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1ff7fe640 1 -- 172.21.15.29:0/3434879407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2000c94d0 con 0x7fa1e8004810 2024-04-25T00:41:54.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1ff7fe640 1 --2- 172.21.15.29:0/3434879407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1e8004810 0x7fa200160f60 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c4d0 tx=0x7fa1f000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1e7fff640 1 -- 172.21.15.29:0/3434879407 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00052d0 con 0x7fa1e8004810 2024-04-25T00:41:54.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1e7fff640 1 -- 172.21.15.29:0/3434879407 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f0005470 con 0x7fa1e8004810 2024-04-25T00:41:54.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1e7fff640 1 -- 172.21.15.29:0/3434879407 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0012870 con 0x7fa1e8004810 2024-04-25T00:41:54.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa20623b640 1 -- 172.21.15.29:0/3434879407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2000c97b0 con 0x7fa1e8004810 2024-04-25T00:41:54.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa20623b640 1 -- 172.21.15.29:0/3434879407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2000c9d70 con 0x7fa1e8004810 2024-04-25T00:41:54.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa20623b640 1 -- 172.21.15.29:0/3434879407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa200069270 con 0x7fa1e8004810 2024-04-25T00:41:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1e7fff640 1 -- 172.21.15.29:0/3434879407 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fa1f0012a10 con 0x7fa1e8004810 2024-04-25T00:41:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1e7fff640 1 --2- 172.21.15.29:0/3434879407 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1ec068400 0x7fa1ec06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1e7fff640 1 -- 172.21.15.29:0/3434879407 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7fa1f00844d0 con 0x7fa1e8004810 2024-04-25T00:41:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1feffd640 1 --2- 172.21.15.29:0/3434879407 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1ec068400 0x7fa1ec06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.188+0000 7fa1feffd640 1 --2- 172.21.15.29:0/3434879407 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1ec068400 0x7fa1ec06a8c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40ab000 tx=0x7fa1f40b0040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.192+0000 7fa1e7fff640 1 -- 172.21.15.29:0/3434879407 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa1f0056870 con 0x7fa1e8004810 2024-04-25T00:41:54.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.344+0000 7fa20623b640 1 -- 172.21.15.29:0/3434879407 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa20015fb20 con 0x7fa1ec068400 2024-04-25T00:41:54.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.344+0000 7fa1e7fff640 1 -- 172.21.15.29:0/3434879407 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7fa1f40bf4b0 con 0x7fa1ec068400 2024-04-25T00:41:54.347 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:54.347 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:41:54.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.348+0000 7fa20623b640 1 -- 172.21.15.29:0/3434879407 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1ec068400 msgr2=0x7fa1ec06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.348+0000 7fa20623b640 1 --2- 172.21.15.29:0/3434879407 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1ec068400 0x7fa1ec06a8c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40ab000 tx=0x7fa1f40b0040 comp rx=0 tx=0).stop 2024-04-25T00:41:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.348+0000 7fa20623b640 1 -- 172.21.15.29:0/3434879407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1e8004810 msgr2=0x7fa200160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.348+0000 7fa20623b640 1 --2- 172.21.15.29:0/3434879407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1e8004810 0x7fa200160f60 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c4d0 tx=0x7fa1f000c990 comp rx=0 tx=0).stop 2024-04-25T00:41:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.348+0000 7fa20623b640 1 -- 172.21.15.29:0/3434879407 shutdown_connections 2024-04-25T00:41:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.348+0000 7fa20623b640 1 --2- 172.21.15.29:0/3434879407 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fa1ec068400 0x7fa1ec06a8c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.348+0000 7fa20623b640 1 --2- 172.21.15.29:0/3434879407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa20015efa0 0x7fa2000cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.348+0000 7fa20623b640 1 --2- 172.21.15.29:0/3434879407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2000ca570 0x7fa2000c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.348+0000 7fa20623b640 1 --2- 172.21.15.29:0/3434879407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1e8004810 0x7fa200160f60 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.348+0000 7fa20623b640 1 -- 172.21.15.29:0/3434879407 >> 172.21.15.29:0/3434879407 conn(0x7fa2000d3fb0 msgr2=0x7fa2000b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.348+0000 7fa20623b640 1 -- 172.21.15.29:0/3434879407 shutdown_connections 2024-04-25T00:41:54.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.348+0000 7fa20623b640 1 -- 172.21.15.29:0/3434879407 wait complete. 2024-04-25T00:41:54.364 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-04-25T00:41:53.429819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":317256,"kb_used_data":17924,"kb_used_omap":12,"kb_used_meta":299251,"kb_avail":754657464,"statfs":{"total":773094113280,"available":772769243136,"internally_reserved":0,"allocated":18354176,"data_stored":8806579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":306433584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53200,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-78,"num_write_kb":-82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001822"},"pg_stats":[{"pgid":"1.0","version":"73'128","reported_seq":490,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725087+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:41:21.725087+0000","last_peered":"2024-04-25T00:41:21.725087+0000","last_clean":"2024-04-25T00:41:21.725087+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:41:21.725087+0000","last_undegraded":"2024-04-25T00:41:21.725087+0000","last_fullsized":"2024-04-25T00:41:21.725087+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38052,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333788,"statfs":{"total":96636764160,"available":96597798912,"internally_reserved":0,"allocated":2781184,"data_stored":1585862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37416,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334424,"statfs":{"total":96636764160,"available":96598450176,"internally_reserved":0,"allocated":2129920,"data_stored":938647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574977,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2138112,"data_stored":944425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41644,"kb_used_data":2148,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330196,"statfs":{"total":96636764160,"available":96594120704,"internally_reserved":0,"allocated":2199552,"data_stored":995901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574977,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41580,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330260,"statfs":{"total":96636764160,"available":96594186240,"internally_reserved":0,"allocated":2134016,"data_stored":937944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":2121728,"data_stored":937762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41576,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330264,"statfs":{"total":96636764160,"available":96594190336,"internally_reserved":0,"allocated":2129920,"data_stored":937818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42152,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329688,"statfs":{"total":96636764160,"available":96593600512,"internally_reserved":0,"allocated":2719744,"data_stored":1528220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:41:54.365 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T00:41:54.365 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:41:54.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48c7af640 1 Processor -- start 2024-04-25T00:41:54.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48c7af640 1 -- start start 2024-04-25T00:41:54.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48c7af640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4840d2d40 0x7fc4840b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48c7af640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4840b4080 0x7fc4840b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48c7af640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4840b5050 con 0x7fc4840d2250 2024-04-25T00:41:54.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48c7af640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc4840b0de0 con 0x7fc4840b4080 2024-04-25T00:41:54.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48c7af640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc4840b5330 con 0x7fc4840d2d40 2024-04-25T00:41:54.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48ad4c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4840b4080 0x7fc4840b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48ad4c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4840b4080 0x7fc4840b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56462/0 (socket says 172.21.15.29:56462) 2024-04-25T00:41:54.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48ad4c640 1 -- 172.21.15.29:0/3479269920 learned_addr learned my addr 172.21.15.29:0/3479269920 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:54.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc489d4a640 1 --2- 172.21.15.29:0/3479269920 >> v2:172.21.15.157:3300/0 conn(0x7fc4840d2d40 0x7fc4840b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48ad4c640 1 -- 172.21.15.29:0/3479269920 >> v2:172.21.15.157:3300/0 conn(0x7fc4840d2d40 msgr2=0x7fc4840b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:54.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48ad4c640 1 --2- 172.21.15.29:0/3479269920 >> v2:172.21.15.157:3300/0 conn(0x7fc4840d2d40 0x7fc4840b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48ad4c640 1 -- 172.21.15.29:0/3479269920 >> v1:172.21.15.157:6789/0 conn(0x7fc4840d2250 legacy=0x7fc4840d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:54.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.476+0000 7fc48ad4c640 1 -- 172.21.15.29:0/3479269920 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4840b1030 con 0x7fc4840b4080 2024-04-25T00:41:54.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48ad4c640 1 --2- 172.21.15.29:0/3479269920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4840b4080 0x7fc4840b44b0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fc48406bbc0 tx=0x7fc47c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0833605aa953056 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc489549640 1 -- 172.21.15.29:0/3479269920 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0a5670 con 0x7fc4840b4080 2024-04-25T00:41:54.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc489549640 1 -- 172.21.15.29:0/3479269920 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47c0a5810 con 0x7fc4840b4080 2024-04-25T00:41:54.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc489549640 1 -- 172.21.15.29:0/3479269920 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0a5be0 con 0x7fc4840b4080 2024-04-25T00:41:54.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 -- 172.21.15.29:0/3479269920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4840b4080 msgr2=0x7fc4840b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 --2- 172.21.15.29:0/3479269920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4840b4080 0x7fc4840b44b0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fc48406bbc0 tx=0x7fc47c01c9b0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 -- 172.21.15.29:0/3479269920 shutdown_connections 2024-04-25T00:41:54.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 --2- 172.21.15.29:0/3479269920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4840b4080 0x7fc4840b44b0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 --2- 172.21.15.29:0/3479269920 >> v2:172.21.15.157:3300/0 conn(0x7fc4840d2d40 0x7fc4840b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 -- 172.21.15.29:0/3479269920 >> 172.21.15.29:0/3479269920 conn(0x7fc48409d580 msgr2=0x7fc4840d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:54.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 -- 172.21.15.29:0/3479269920 shutdown_connections 2024-04-25T00:41:54.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 -- 172.21.15.29:0/3479269920 wait complete. 2024-04-25T00:41:54.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 Processor -- start 2024-04-25T00:41:54.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 -- start start 2024-04-25T00:41:54.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4840b4080 0x7fc484157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4840d2250 0x7fc48414d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4840d2d40 0x7fc4841fa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc4840b23d0 con 0x7fc4840d2d40 2024-04-25T00:41:54.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4840b2250 con 0x7fc4840b4080 2024-04-25T00:41:54.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4840b2550 con 0x7fc4840d2250 2024-04-25T00:41:54.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48ad4c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4840d2d40 0x7fc4841fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48ad4c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4840d2d40 0x7fc4841fa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56474/0 (socket says 172.21.15.29:56474) 2024-04-25T00:41:54.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48ad4c640 1 -- 172.21.15.29:0/3470011951 learned_addr learned my addr 172.21.15.29:0/3470011951 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48ad4c640 1 -- 172.21.15.29:0/3470011951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4840d2250 msgr2=0x7fc48414d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48ad4c640 1 --2- 172.21.15.29:0/3470011951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4840d2250 0x7fc48414d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48a54b640 1 --2- 172.21.15.29:0/3470011951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4840b4080 0x7fc484157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48ad4c640 1 -- 172.21.15.29:0/3470011951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4840b4080 msgr2=0x7fc484157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48ad4c640 1 --2- 172.21.15.29:0/3470011951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4840b4080 0x7fc484157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48ad4c640 1 -- 172.21.15.29:0/3470011951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48415a8b0 con 0x7fc4840d2d40 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48ad4c640 1 --2- 172.21.15.29:0/3470011951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4840d2d40 0x7fc4841fa500 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fc47c01a5e0 tx=0x7fc47c019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc47affd640 1 -- 172.21.15.29:0/3470011951 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c009510 con 0x7fc4840d2d40 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc47affd640 1 -- 172.21.15.29:0/3470011951 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47c0be070 con 0x7fc4840d2d40 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 -- 172.21.15.29:0/3470011951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc484157070 con 0x7fc4840d2d40 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc47affd640 1 -- 172.21.15.29:0/3470011951 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0b8ac0 con 0x7fc4840d2d40 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.480+0000 7fc48c7af640 1 -- 172.21.15.29:0/3470011951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48414d780 con 0x7fc4840d2d40 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.484+0000 7fc48c7af640 1 -- 172.21.15.29:0/3470011951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc48415ae80 con 0x7fc4840d2d40 2024-04-25T00:41:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.484+0000 7fc47affd640 1 -- 172.21.15.29:0/3470011951 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fc47c0b8c60 con 0x7fc4840d2d40 2024-04-25T00:41:54.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.484+0000 7fc47affd640 1 --2- 172.21.15.29:0/3470011951 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc468068400 0x7fc46806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.484+0000 7fc47affd640 1 -- 172.21.15.29:0/3470011951 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7fc47c11f940 con 0x7fc4840d2d40 2024-04-25T00:41:54.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.484+0000 7fc48a54b640 1 --2- 172.21.15.29:0/3470011951 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc468068400 0x7fc46806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.484+0000 7fc48a54b640 1 --2- 172.21.15.29:0/3470011951 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc468068400 0x7fc46806a8c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc480003d60 tx=0x7fc480009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.488+0000 7fc47affd640 1 -- 172.21.15.29:0/3470011951 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc47c0165a0 con 0x7fc4840d2d40 2024-04-25T00:41:54.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.640+0000 7fc48c7af640 1 -- 172.21.15.29:0/3470011951 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc484069270 con 0x7fc468068400 2024-04-25T00:41:54.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.644+0000 7fc47affd640 1 -- 172.21.15.29:0/3470011951 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7fc484069270 con 0x7fc468068400 2024-04-25T00:41:54.644 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:54.644 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:41:54.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.644+0000 7fc48c7af640 1 -- 172.21.15.29:0/3470011951 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc468068400 msgr2=0x7fc46806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.644+0000 7fc48c7af640 1 --2- 172.21.15.29:0/3470011951 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc468068400 0x7fc46806a8c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc480003d60 tx=0x7fc480009040 comp rx=0 tx=0).stop 2024-04-25T00:41:54.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.644+0000 7fc48c7af640 1 -- 172.21.15.29:0/3470011951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4840d2d40 msgr2=0x7fc4841fa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.644+0000 7fc48c7af640 1 --2- 172.21.15.29:0/3470011951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4840d2d40 0x7fc4841fa500 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fc47c01a5e0 tx=0x7fc47c019750 comp rx=0 tx=0).stop 2024-04-25T00:41:54.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.644+0000 7fc48c7af640 1 -- 172.21.15.29:0/3470011951 shutdown_connections 2024-04-25T00:41:54.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.644+0000 7fc48c7af640 1 --2- 172.21.15.29:0/3470011951 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc468068400 0x7fc46806a8c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.644+0000 7fc48c7af640 1 --2- 172.21.15.29:0/3470011951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4840d2d40 0x7fc4841fa500 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.644+0000 7fc48c7af640 1 --2- 172.21.15.29:0/3470011951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4840d2250 0x7fc48414d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.644+0000 7fc48c7af640 1 --2- 172.21.15.29:0/3470011951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4840b4080 0x7fc484157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.644+0000 7fc48c7af640 1 -- 172.21.15.29:0/3470011951 >> 172.21.15.29:0/3470011951 conn(0x7fc48409d580 msgr2=0x7fc4840d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:54.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.644+0000 7fc48c7af640 1 -- 172.21.15.29:0/3470011951 shutdown_connections 2024-04-25T00:41:54.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.644+0000 7fc48c7af640 1 -- 172.21.15.29:0/3470011951 wait complete. 2024-04-25T00:41:54.661 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-04-25T00:41:53.429819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":317256,"kb_used_data":17924,"kb_used_omap":12,"kb_used_meta":299251,"kb_avail":754657464,"statfs":{"total":773094113280,"available":772769243136,"internally_reserved":0,"allocated":18354176,"data_stored":8806579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":306433584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53200,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-78,"num_write_kb":-82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001822"},"pg_stats":[{"pgid":"1.0","version":"73'128","reported_seq":490,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T00:41:21.725087+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:41:21.725087+0000","last_peered":"2024-04-25T00:41:21.725087+0000","last_clean":"2024-04-25T00:41:21.725087+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:41:21.725087+0000","last_undegraded":"2024-04-25T00:41:21.725087+0000","last_fullsized":"2024-04-25T00:41:21.725087+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38052,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333788,"statfs":{"total":96636764160,"available":96597798912,"internally_reserved":0,"allocated":2781184,"data_stored":1585862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37416,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334424,"statfs":{"total":96636764160,"available":96598450176,"internally_reserved":0,"allocated":2129920,"data_stored":938647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574977,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2138112,"data_stored":944425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41644,"kb_used_data":2148,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330196,"statfs":{"total":96636764160,"available":96594120704,"internally_reserved":0,"allocated":2199552,"data_stored":995901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574977,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41580,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330260,"statfs":{"total":96636764160,"available":96594186240,"internally_reserved":0,"allocated":2134016,"data_stored":937944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":2121728,"data_stored":937762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41576,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330264,"statfs":{"total":96636764160,"available":96594190336,"internally_reserved":0,"allocated":2129920,"data_stored":937818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42152,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329688,"statfs":{"total":96636764160,"available":96593600512,"internally_reserved":0,"allocated":2719744,"data_stored":1528220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:41:54.662 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:41:54.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3c474640 1 Processor -- start 2024-04-25T00:41:54.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3c474640 1 -- start start 2024-04-25T00:41:54.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3c474640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b34156420 0x7f8b3415ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3c474640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b3415fa80 0x7f8b3406b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3c474640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b3415ff10 0x7f8b3406b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3c474640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8b3415d3a0 con 0x7f8b3415ff10 2024-04-25T00:41:54.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3c474640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8b3415c680 con 0x7f8b3415fa80 2024-04-25T00:41:54.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3c474640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b34160440 con 0x7f8b34156420 2024-04-25T00:41:54.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3aa11640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b3415ff10 0x7f8b3406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3aa11640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b3415ff10 0x7f8b3406b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56490/0 (socket says 172.21.15.29:56490) 2024-04-25T00:41:54.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3aa11640 1 -- 172.21.15.29:0/3897129442 learned_addr learned my addr 172.21.15.29:0/3897129442 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:54.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3a210640 1 --2- 172.21.15.29:0/3897129442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b34156420 0x7f8b3415ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b39a0f640 1 --2- 172.21.15.29:0/3897129442 >> v2:172.21.15.157:3300/0 conn(0x7f8b3415fa80 0x7f8b3406b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3aa11640 1 -- 172.21.15.29:0/3897129442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b34156420 msgr2=0x7f8b3415ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:54.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3aa11640 1 --2- 172.21.15.29:0/3897129442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b34156420 0x7f8b3415ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3aa11640 1 -- 172.21.15.29:0/3897129442 >> v2:172.21.15.157:3300/0 conn(0x7f8b3415fa80 msgr2=0x7f8b3406b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:54.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3aa11640 1 --2- 172.21.15.29:0/3897129442 >> v2:172.21.15.157:3300/0 conn(0x7f8b3415fa80 0x7f8b3406b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3aa11640 1 -- 172.21.15.29:0/3897129442 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b34160720 con 0x7f8b3415ff10 2024-04-25T00:41:54.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b3a210640 1 --2- 172.21.15.29:0/3897129442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b34156420 0x7f8b3415ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:54.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.776+0000 7f8b39a0f640 1 --2- 172.21.15.29:0/3897129442 >> v2:172.21.15.157:3300/0 conn(0x7f8b3415fa80 0x7f8b3406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:54.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3aa11640 1 --2- 172.21.15.29:0/3897129442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3415ff10 0x7f8b3406b750 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f8b3406d370 tx=0x7f8b2c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb0dc992c0571ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3920e640 1 -- 172.21.15.29:0/3897129442 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0a5780 con 0x7f8b3415ff10 2024-04-25T00:41:54.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3920e640 1 -- 172.21.15.29:0/3897129442 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b2c0a5920 con 0x7f8b3415ff10 2024-04-25T00:41:54.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3920e640 1 -- 172.21.15.29:0/3897129442 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0a5cf0 con 0x7f8b3415ff10 2024-04-25T00:41:54.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3c474640 1 -- 172.21.15.29:0/3897129442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3415ff10 msgr2=0x7f8b3406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3c474640 1 --2- 172.21.15.29:0/3897129442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3415ff10 0x7f8b3406b750 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f8b3406d370 tx=0x7f8b2c01c9f0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3c474640 1 -- 172.21.15.29:0/3897129442 shutdown_connections 2024-04-25T00:41:54.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3c474640 1 --2- 172.21.15.29:0/3897129442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3415ff10 0x7f8b3406b750 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3c474640 1 --2- 172.21.15.29:0/3897129442 >> v2:172.21.15.157:3300/0 conn(0x7f8b3415fa80 0x7f8b3406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3c474640 1 --2- 172.21.15.29:0/3897129442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b34156420 0x7f8b3415ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3c474640 1 -- 172.21.15.29:0/3897129442 >> 172.21.15.29:0/3897129442 conn(0x7f8b340d0c40 msgr2=0x7f8b340d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:54.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3c474640 1 -- 172.21.15.29:0/3897129442 shutdown_connections 2024-04-25T00:41:54.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3c474640 1 -- 172.21.15.29:0/3897129442 wait complete. 2024-04-25T00:41:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3c474640 1 Processor -- start 2024-04-25T00:41:54.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.780+0000 7f8b3c474640 1 -- start start 2024-04-25T00:41:54.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b3c474640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b34156420 0x7f8b3414c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b3c474640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3415fa80 0x7f8b34155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b3c474640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3415ff10 0x7f8b3415b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b3c474640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8b340cbc20 con 0x7f8b3415fa80 2024-04-25T00:41:54.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b3c474640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b340cbaa0 con 0x7f8b34156420 2024-04-25T00:41:54.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b3c474640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b340cbda0 con 0x7f8b3415ff10 2024-04-25T00:41:54.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b39a0f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3415fa80 0x7f8b34155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b39a0f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3415fa80 0x7f8b34155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56498/0 (socket says 172.21.15.29:56498) 2024-04-25T00:41:54.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b39a0f640 1 -- 172.21.15.29:0/3416423567 learned_addr learned my addr 172.21.15.29:0/3416423567 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:54.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b3a210640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b34156420 0x7f8b3414c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b3aa11640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3415ff10 0x7f8b3415b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b39a0f640 1 -- 172.21.15.29:0/3416423567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3415ff10 msgr2=0x7f8b3415b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b39a0f640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3415ff10 0x7f8b3415b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b39a0f640 1 -- 172.21.15.29:0/3416423567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b34156420 msgr2=0x7f8b3414c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b39a0f640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b34156420 0x7f8b3414c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b39a0f640 1 -- 172.21.15.29:0/3416423567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3414cb10 con 0x7f8b3415fa80 2024-04-25T00:41:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b39a0f640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3415fa80 0x7f8b34155a00 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400c7f0 tx=0x7f8b2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b2affd640 1 -- 172.21.15.29:0/3416423567 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24015070 con 0x7f8b3415fa80 2024-04-25T00:41:54.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b2affd640 1 -- 172.21.15.29:0/3416423567 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b24007820 con 0x7f8b3415fa80 2024-04-25T00:41:54.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b2affd640 1 -- 172.21.15.29:0/3416423567 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24007c10 con 0x7f8b3415fa80 2024-04-25T00:41:54.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b3c474640 1 -- 172.21.15.29:0/3416423567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3414cdf0 con 0x7f8b3415fa80 2024-04-25T00:41:54.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b3c474640 1 -- 172.21.15.29:0/3416423567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b3414d3b0 con 0x7f8b3415fa80 2024-04-25T00:41:54.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b3c474640 1 -- 172.21.15.29:0/3416423567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b3414d790 con 0x7f8b3415fa80 2024-04-25T00:41:54.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b2affd640 1 -- 172.21.15.29:0/3416423567 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f8b24018070 con 0x7f8b3415fa80 2024-04-25T00:41:54.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b2affd640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b18068400 0x7f8b1806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:54.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b3a210640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b18068400 0x7f8b1806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:54.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b2affd640 1 -- 172.21.15.29:0/3416423567 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f8b24084ad0 con 0x7f8b3415fa80 2024-04-25T00:41:54.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.784+0000 7f8b3a210640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b18068400 0x7f8b1806a8c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000b7e0 tx=0x7f8b30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:54.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.788+0000 7f8b2affd640 1 -- 172.21.15.29:0/3416423567 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8b3414d790 con 0x7f8b3415fa80 2024-04-25T00:41:54.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.948+0000 7f8b3c474640 1 -- 172.21.15.29:0/3416423567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b34069270 con 0x7f8b3415fa80 2024-04-25T00:41:54.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.948+0000 7f8b2affd640 1 -- 172.21.15.29:0/3416423567 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) ==== 74+0+12978 (secure 0 0 0) 0x7f8b24056d70 con 0x7f8b3415fa80 2024-04-25T00:41:54.950 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:41:54.950 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":171,"fsid":"59afbe7d-aaa8-45e2-b468-69ee150cacea","created":"2024-04-25T00:31:12.657856+0000","modified":"2024-04-25T00:41:51.917658+0000","last_up_change":"2024-04-25T00:31:23.211682+0000","last_in_change":"2024-04-25T00:31:14.972478+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:31:23.504264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e92106-8ca5-4141-9322-34a10ee2457b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6809","nonce":2412845288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6811","nonce":2412845288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6815","nonce":2412845288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":2412845288},{"type":"v1","addr":"172.21.15.29:6813","nonce":2412845288}]},"public_addr":"172.21.15.29:6809/2412845288","cluster_addr":"172.21.15.29:6811/2412845288","heartbeat_back_addr":"172.21.15.29:6815/2412845288","heartbeat_front_addr":"172.21.15.29:6813/2412845288","state":["exists","up"]},{"osd":1,"uuid":"8cee5ae1-c34c-4c4d-9402-eb383be68cac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6801","nonce":738833181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6803","nonce":738833181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6807","nonce":738833181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":738833181},{"type":"v1","addr":"172.21.15.29:6805","nonce":738833181}]},"public_addr":"172.21.15.29:6801/738833181","cluster_addr":"172.21.15.29:6803/738833181","heartbeat_back_addr":"172.21.15.29:6807/738833181","heartbeat_front_addr":"172.21.15.29:6805/738833181","state":["exists","up"]},{"osd":2,"uuid":"1ddd0b31-1ca0-4cd7-8a15-5231d58ccdf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6818","nonce":802770835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6822","nonce":802770835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6830","nonce":802770835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":802770835},{"type":"v1","addr":"172.21.15.29:6826","nonce":802770835}]},"public_addr":"172.21.15.29:6818/802770835","cluster_addr":"172.21.15.29:6822/802770835","heartbeat_back_addr":"172.21.15.29:6830/802770835","heartbeat_front_addr":"172.21.15.29:6826/802770835","state":["exists","up"]},{"osd":3,"uuid":"03f003b6-1663-4b90-b6d0-d470e68a52c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6817","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6819","nonce":1979009155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6821","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6823","nonce":1979009155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6829","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6831","nonce":1979009155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6825","nonce":1979009155},{"type":"v1","addr":"172.21.15.29:6827","nonce":1979009155}]},"public_addr":"172.21.15.29:6819/1979009155","cluster_addr":"172.21.15.29:6823/1979009155","heartbeat_back_addr":"172.21.15.29:6831/1979009155","heartbeat_front_addr":"172.21.15.29:6827/1979009155","state":["exists","up"]},{"osd":4,"uuid":"80ae2f99-2daf-47c9-b454-978268265b24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6803","nonce":674816998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6807","nonce":674816998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6815","nonce":674816998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":674816998},{"type":"v1","addr":"172.21.15.157:6811","nonce":674816998}]},"public_addr":"172.21.15.157:6803/674816998","cluster_addr":"172.21.15.157:6807/674816998","heartbeat_back_addr":"172.21.15.157:6815/674816998","heartbeat_front_addr":"172.21.15.157:6811/674816998","state":["exists","up"]},{"osd":5,"uuid":"ba24b2d8-51c4-4a54-abd1-cafb823ae33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6802","nonce":486133226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6806","nonce":486133226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6814","nonce":486133226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":486133226},{"type":"v1","addr":"172.21.15.157:6810","nonce":486133226}]},"public_addr":"172.21.15.157:6802/486133226","cluster_addr":"172.21.15.157:6806/486133226","heartbeat_back_addr":"172.21.15.157:6814/486133226","heartbeat_front_addr":"172.21.15.157:6810/486133226","state":["exists","up"]},{"osd":6,"uuid":"99523015-c5be-4adc-bc1c-6b44b0cbe928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6817","nonce":2845420196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6819","nonce":2845420196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6823","nonce":2845420196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2845420196},{"type":"v1","addr":"172.21.15.157:6821","nonce":2845420196}]},"public_addr":"172.21.15.157:6817/2845420196","cluster_addr":"172.21.15.157:6819/2845420196","heartbeat_back_addr":"172.21.15.157:6823/2845420196","heartbeat_front_addr":"172.21.15.157:6821/2845420196","state":["exists","up"]},{"osd":7,"uuid":"898aac88-8174-4b35-8e59-cd06cef7d993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6825","nonce":739472638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6827","nonce":739472638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6831","nonce":739472638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":739472638},{"type":"v1","addr":"172.21.15.157:6829","nonce":739472638}]},"public_addr":"172.21.15.157:6825/739472638","cluster_addr":"172.21.15.157:6827/739472638","heartbeat_back_addr":"172.21.15.157:6831/739472638","heartbeat_front_addr":"172.21.15.157:6829/739472638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.534837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.523982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.541581+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.607731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.437693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.545200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.485880+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:31:20.560814+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6835/85311029":"2024-04-26T00:41:48.903767+0000","172.21.15.29:6834/85311029":"2024-04-26T00:41:48.903767+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:41:54.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.952+0000 7f8b3c474640 1 -- 172.21.15.29:0/3416423567 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b18068400 msgr2=0x7f8b1806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.952+0000 7f8b3c474640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b18068400 0x7f8b1806a8c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000b7e0 tx=0x7f8b30008040 comp rx=0 tx=0).stop 2024-04-25T00:41:54.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.952+0000 7f8b3c474640 1 -- 172.21.15.29:0/3416423567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3415fa80 msgr2=0x7f8b34155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:54.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.952+0000 7f8b3c474640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3415fa80 0x7f8b34155a00 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400c7f0 tx=0x7f8b2400ccb0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.952+0000 7f8b3c474640 1 -- 172.21.15.29:0/3416423567 shutdown_connections 2024-04-25T00:41:54.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.952+0000 7f8b3c474640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f8b18068400 0x7f8b1806a8c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.952+0000 7f8b3c474640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3415ff10 0x7f8b3415b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.952+0000 7f8b3c474640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b3415fa80 0x7f8b34155a00 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.952+0000 7f8b3c474640 1 --2- 172.21.15.29:0/3416423567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b34156420 0x7f8b3414c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:54.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.952+0000 7f8b3c474640 1 -- 172.21.15.29:0/3416423567 >> 172.21.15.29:0/3416423567 conn(0x7f8b340d0c40 msgr2=0x7f8b34071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:54.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.952+0000 7f8b3c474640 1 -- 172.21.15.29:0/3416423567 shutdown_connections 2024-04-25T00:41:54.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:54.952+0000 7f8b3c474640 1 -- 172.21.15.29:0/3416423567 wait complete. 2024-04-25T00:41:55.968 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T00:41:55.969 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:41:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.108+0000 7f3abe999640 1 Processor -- start 2024-04-25T00:41:56.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.108+0000 7f3abe999640 1 -- start start 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.108+0000 7f3abe999640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3ab815e9a0 0x7f3ab8070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.108+0000 7f3abe999640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ab80712c0 con 0x7f3ab80b4590 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.108+0000 7f3abe999640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ab80715a0 con 0x7f3ab80cd1c0 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.108+0000 7f3abe999640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f3ab8071ed0 con 0x7f3ab815e9a0 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3ab7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3ab815e9a0 0x7f3ab8070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3ab77fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3ab80b4590 0x7f3ab815ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55776/0 (socket says 172.21.15.29:55776) 2024-04-25T00:41:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3ab77fe640 1 -- 172.21.15.29:0/1691783833 learned_addr learned my addr 172.21.15.29:0/1691783833 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3ab6ffd640 1 -- 172.21.15.29:0/1691783833 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2473806533 0 0) 0x7f3ab80712c0 con 0x7f3ab80b4590 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3ab6ffd640 1 -- 172.21.15.29:0/1691783833 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3aa0005740 con 0x7f3ab80b4590 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1691783833 >> v1:172.21.15.157:6789/0 conn(0x7f3ab80cd1c0 legacy=0x7f3ab8072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1691783833 >> v1:172.21.15.29:6789/0 conn(0x7f3ab80b4590 legacy=0x7f3ab815ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3ab7fff640 1 -- 172.21.15.29:0/1691783833 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab8071880 con 0x7f3ab815e9a0 2024-04-25T00:41:56.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3ab7fff640 1 --2- 172.21.15.29:0/1691783833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ab815e9a0 0x7f3ab8070ce0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80097c0 tx=0x7f3aa801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7d7779ee68ab47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3ab6ffd640 1 -- 172.21.15.29:0/1691783833 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80b1070 con 0x7f3ab815e9a0 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3ab6ffd640 1 -- 172.21.15.29:0/1691783833 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aa80ac5f0 con 0x7f3ab815e9a0 2024-04-25T00:41:56.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3ab6ffd640 1 -- 172.21.15.29:0/1691783833 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80ac9c0 con 0x7f3ab815e9a0 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3abe999640 1 -- 172.21.15.29:0/1691783833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ab815e9a0 msgr2=0x7f3ab8070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3abe999640 1 --2- 172.21.15.29:0/1691783833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ab815e9a0 0x7f3ab8070ce0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80097c0 tx=0x7f3aa801c470 comp rx=0 tx=0).stop 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3abe999640 1 -- 172.21.15.29:0/1691783833 shutdown_connections 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3abe999640 1 --2- 172.21.15.29:0/1691783833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ab815e9a0 0x7f3ab8070ce0 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3abe999640 1 -- 172.21.15.29:0/1691783833 >> 172.21.15.29:0/1691783833 conn(0x7f3ab80b1870 msgr2=0x7f3ab80b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3abe999640 1 -- 172.21.15.29:0/1691783833 shutdown_connections 2024-04-25T00:41:56.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.112+0000 7f3abe999640 1 -- 172.21.15.29:0/1691783833 wait complete. 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3abe999640 1 Processor -- start 2024-04-25T00:41:56.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3abe999640 1 -- start start 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3abe999640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab80b4590 0x7f3ab813c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3abe999640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ab80cd1c0 0x7f3ab80b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3abe999640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ab815e9a0 0x7f3ab81531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3abe999640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81449f0 con 0x7f3ab80cd1c0 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3abe999640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8144870 con 0x7f3ab80b4590 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3abe999640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3ab80cd5d0 con 0x7f3ab815e9a0 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ab80cd1c0 0x7f3ab80b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ab80cd1c0 0x7f3ab80b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56512/0 (socket says 172.21.15.29:56512) 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab77fe640 1 -- 172.21.15.29:0/3588254322 learned_addr learned my addr 172.21.15.29:0/3588254322 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab7fff640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab80b4590 0x7f3ab813c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab77fe640 1 -- 172.21.15.29:0/3588254322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ab815e9a0 msgr2=0x7f3ab81531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3abcf36640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ab815e9a0 0x7f3ab81531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab77fe640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ab815e9a0 0x7f3ab81531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab77fe640 1 -- 172.21.15.29:0/3588254322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab80b4590 msgr2=0x7f3ab813c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab77fe640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab80b4590 0x7f3ab813c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab77fe640 1 -- 172.21.15.29:0/3588254322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab813fe20 con 0x7f3ab80cd1c0 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab7fff640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab80b4590 0x7f3ab813c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3abcf36640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ab815e9a0 0x7f3ab81531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab77fe640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ab80cd1c0 0x7f3ab80b3310 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00c450 tx=0x7f3aac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab4ff9640 1 -- 172.21.15.29:0/3588254322 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac017070 con 0x7f3ab80cd1c0 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab4ff9640 1 -- 172.21.15.29:0/3588254322 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aac005cf0 con 0x7f3ab80cd1c0 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3abe999640 1 -- 172.21.15.29:0/3588254322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab81393e0 con 0x7f3ab80cd1c0 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3ab4ff9640 1 -- 172.21.15.29:0/3588254322 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac007270 con 0x7f3ab80cd1c0 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3abe999640 1 -- 172.21.15.29:0/3588254322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab8140380 con 0x7f3ab80cd1c0 2024-04-25T00:41:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.116+0000 7f3abe999640 1 -- 172.21.15.29:0/3588254322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ab80692e0 con 0x7f3ab80cd1c0 2024-04-25T00:41:56.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.120+0000 7f3ab4ff9640 1 -- 172.21.15.29:0/3588254322 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f3aac01a070 con 0x7f3ab80cd1c0 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.120+0000 7f3ab4ff9640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3aa4068400 0x7f3aa406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.120+0000 7f3ab4ff9640 1 -- 172.21.15.29:0/3588254322 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f3aac0842c0 con 0x7f3ab80cd1c0 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.120+0000 7f3ab7fff640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3aa4068400 0x7f3aa406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.120+0000 7f3ab4ff9640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] conn(0x7f3aa406e2c0 0x7f3aa40706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.120+0000 7f3ab4ff9640 1 -- 172.21.15.29:0/3588254322 --> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3aa4070d90 con 0x7f3aa406e2c0 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.120+0000 7f3ab4ff9640 1 -- 172.21.15.29:0/3588254322 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=171) ==== 24+0+0 (secure 0 0 0) 0x7f3aac086d40 con 0x7f3ab80cd1c0 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.120+0000 7f3abcf36640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] conn(0x7f3aa406e2c0 0x7f3aa40706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.120+0000 7f3ab7fff640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3aa4068400 0x7f3aa406a8c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80ae000 tx=0x7f3aa8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.120+0000 7f3abcf36640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] conn(0x7f3aa406e2c0 0x7f3aa40706e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.120+0000 7f3ab4ff9640 1 -- 172.21.15.29:0/3588254322 <== osd.0 v2:172.21.15.29:6808/2412845288 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3aa4070d90 con 0x7f3aa406e2c0 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.132+0000 7f3abe999640 1 -- 172.21.15.29:0/3588254322 --> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3ab806cff0 con 0x7f3aa406e2c0 2024-04-25T00:41:56.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.132+0000 7f3ab4ff9640 1 -- 172.21.15.29:0/3588254322 <== osd.0 v2:172.21.15.29:6808/2412845288 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3ab806cff0 con 0x7f3aa406e2c0 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:41:56.135 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.132+0000 7f3abe999640 1 -- 172.21.15.29:0/3588254322 >> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] conn(0x7f3aa406e2c0 msgr2=0x7f3aa40706e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.132+0000 7f3abe999640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.29:6808/2412845288,v1:172.21.15.29:6809/2412845288] conn(0x7f3aa406e2c0 0x7f3aa40706e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.136+0000 7f3abe999640 1 -- 172.21.15.29:0/3588254322 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3aa4068400 msgr2=0x7f3aa406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.136+0000 7f3abe999640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f3aa4068400 0x7f3aa406a8c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80ae000 tx=0x7f3aa8008040 comp rx=0 tx=0).stop 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.136+0000 7f3abe999640 1 -- 172.21.15.29:0/3588254322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ab80cd1c0 msgr2=0x7f3ab80b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.136+0000 7f3abe999640 1 --2- 172.21.15.29:0/3588254322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ab80cd1c0 0x7f3ab80b3310 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00c450 tx=0x7f3aac00c910 comp rx=0 tx=0).stop 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.136+0000 7f3abcf36640 1 -- 172.21.15.29:0/3588254322 reap_dead start 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.136+0000 7f3abe999640 1 -- 172.21.15.29:0/3588254322 shutdown_connections 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.136+0000 7f3abe999640 1 -- 172.21.15.29:0/3588254322 >> 172.21.15.29:0/3588254322 conn(0x7f3ab80b1870 msgr2=0x7f3ab8143430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.136+0000 7f3abe999640 1 -- 172.21.15.29:0/3588254322 shutdown_connections 2024-04-25T00:41:56.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.136+0000 7f3abe999640 1 -- 172.21.15.29:0/3588254322 wait complete. 2024-04-25T00:41:56.151 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d939d0640 1 Processor -- start 2024-04-25T00:41:56.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d939d0640 1 -- start start 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d939d0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d8c15a790 0x7f7d8c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d939d0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d8c0ce670 0x7f7d8c0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d939d0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d8c0c3c90 con 0x7f7d8c0cfe70 2024-04-25T00:41:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d939d0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7d8c0ae610 con 0x7f7d8c0ce670 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d939d0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c0af8c0 con 0x7f7d8c15a790 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d91f6d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d8c0ce670 0x7f7d8c0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d91f6d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d8c0ce670 0x7f7d8c0aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56522/0 (socket says 172.21.15.29:56522) 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d91f6d640 1 -- 172.21.15.29:0/2059998710 learned_addr learned my addr 172.21.15.29:0/2059998710 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d9176c640 1 --2- 172.21.15.29:0/2059998710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d8c15a790 0x7f7d8c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d91f6d640 1 -- 172.21.15.29:0/2059998710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d8c15a790 msgr2=0x7f7d8c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d91f6d640 1 --2- 172.21.15.29:0/2059998710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d8c15a790 0x7f7d8c0afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d91f6d640 1 -- 172.21.15.29:0/2059998710 >> v1:172.21.15.29:6789/0 conn(0x7f7d8c0cfe70 legacy=0x7f7d8c0cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d8bfff640 1 -- 172.21.15.29:0/2059998710 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4075412005 0 0) 0x7f7d8c0c3c90 con 0x7f7d8c0cfe70 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d91f6d640 1 -- 172.21.15.29:0/2059998710 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c0adce0 con 0x7f7d8c0ce670 2024-04-25T00:41:56.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.268+0000 7f7d9176c640 1 --2- 172.21.15.29:0/2059998710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d8c15a790 0x7f7d8c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d91f6d640 1 --2- 172.21.15.29:0/2059998710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d8c0ce670 0x7f7d8c0aeaa0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009870 tx=0x7f7d8401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb8d7c38ddacc85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d8bfff640 1 -- 172.21.15.29:0/2059998710 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840b1070 con 0x7f7d8c0ce670 2024-04-25T00:41:56.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d8bfff640 1 -- 172.21.15.29:0/2059998710 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d840a8050 con 0x7f7d8c0ce670 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d8bfff640 1 -- 172.21.15.29:0/2059998710 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840ac860 con 0x7f7d8c0ce670 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d939d0640 1 -- 172.21.15.29:0/2059998710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d8c0ce670 msgr2=0x7f7d8c0aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d939d0640 1 --2- 172.21.15.29:0/2059998710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d8c0ce670 0x7f7d8c0aeaa0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009870 tx=0x7f7d8401c410 comp rx=0 tx=0).stop 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d939d0640 1 -- 172.21.15.29:0/2059998710 shutdown_connections 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d939d0640 1 --2- 172.21.15.29:0/2059998710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d8c0ce670 0x7f7d8c0aeaa0 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d939d0640 1 --2- 172.21.15.29:0/2059998710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d8c15a790 0x7f7d8c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d939d0640 1 -- 172.21.15.29:0/2059998710 >> 172.21.15.29:0/2059998710 conn(0x7f7d8c0b1ec0 msgr2=0x7f7d8c0b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d939d0640 1 -- 172.21.15.29:0/2059998710 shutdown_connections 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d939d0640 1 -- 172.21.15.29:0/2059998710 wait complete. 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d939d0640 1 Processor -- start 2024-04-25T00:41:56.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.272+0000 7f7d939d0640 1 -- start start 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d939d0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8c0ce670 0x7f7d8c0c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d939d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d8c0cfe70 0x7f7d8c1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d939d0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d8c15a790 0x7f7d8c155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d939d0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c0b2d70 con 0x7f7d8c0cfe70 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d939d0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c0b2bf0 con 0x7f7d8c0ce670 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d939d0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c0741a0 con 0x7f7d8c15a790 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d90f6b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d8c0cfe70 0x7f7d8c1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d90f6b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d8c0cfe70 0x7f7d8c1558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56534/0 (socket says 172.21.15.29:56534) 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d90f6b640 1 -- 172.21.15.29:0/1506815320 learned_addr learned my addr 172.21.15.29:0/1506815320 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:56.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d90f6b640 1 -- 172.21.15.29:0/1506815320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d8c15a790 msgr2=0x7f7d8c155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d90f6b640 1 --2- 172.21.15.29:0/1506815320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d8c15a790 0x7f7d8c155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d90f6b640 1 -- 172.21.15.29:0/1506815320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8c0ce670 msgr2=0x7f7d8c0c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d90f6b640 1 --2- 172.21.15.29:0/1506815320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8c0ce670 0x7f7d8c0c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d90f6b640 1 -- 172.21.15.29:0/1506815320 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c0c6580 con 0x7f7d8c0cfe70 2024-04-25T00:41:56.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d90f6b640 1 --2- 172.21.15.29:0/1506815320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d8c0cfe70 0x7f7d8c1558c0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000c480 tx=0x7f7d8000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d89ffb640 1 -- 172.21.15.29:0/1506815320 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80005480 con 0x7f7d8c0cfe70 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d89ffb640 1 -- 172.21.15.29:0/1506815320 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d80003600 con 0x7f7d8c0cfe70 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d939d0640 1 -- 172.21.15.29:0/1506815320 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8c0c62a0 con 0x7f7d8c0cfe70 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d89ffb640 1 -- 172.21.15.29:0/1506815320 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d800039f0 con 0x7f7d8c0cfe70 2024-04-25T00:41:56.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d939d0640 1 -- 172.21.15.29:0/1506815320 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8c15b9c0 con 0x7f7d8c0cfe70 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d939d0640 1 -- 172.21.15.29:0/1506815320 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d8c069270 con 0x7f7d8c0cfe70 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d89ffb640 1 -- 172.21.15.29:0/1506815320 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f7d80003b90 con 0x7f7d8c0cfe70 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d89ffb640 1 --2- 172.21.15.29:0/1506815320 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d78068400 0x7f7d7806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d89ffb640 1 -- 172.21.15.29:0/1506815320 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f7d800845a0 con 0x7f7d8c0cfe70 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d9176c640 1 --2- 172.21.15.29:0/1506815320 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d78068400 0x7f7d7806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.276+0000 7f7d9176c640 1 --2- 172.21.15.29:0/1506815320 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d78068400 0x7f7d7806a8c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009eb0 tx=0x7f7d7c0094a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.280+0000 7f7d89ffb640 1 -- 172.21.15.29:0/1506815320 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7d800568e0 con 0x7f7d8c0cfe70 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d939d0640 1 -- 172.21.15.29:0/1506815320 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f7d8c0c3be0 con 0x7f7d78068400 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d89ffb640 1 -- 172.21.15.29:0/1506815320 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7d8c0c3be0 con 0x7f7d78068400 2024-04-25T00:41:56.444 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d939d0640 1 -- 172.21.15.29:0/1506815320 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d78068400 msgr2=0x7f7d7806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d939d0640 1 --2- 172.21.15.29:0/1506815320 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d78068400 0x7f7d7806a8c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009eb0 tx=0x7f7d7c0094a0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d939d0640 1 -- 172.21.15.29:0/1506815320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d8c0cfe70 msgr2=0x7f7d8c1558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d939d0640 1 --2- 172.21.15.29:0/1506815320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d8c0cfe70 0x7f7d8c1558c0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000c480 tx=0x7f7d8000c940 comp rx=0 tx=0).stop 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d939d0640 1 -- 172.21.15.29:0/1506815320 shutdown_connections 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d939d0640 1 --2- 172.21.15.29:0/1506815320 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f7d78068400 0x7f7d7806a8c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d939d0640 1 --2- 172.21.15.29:0/1506815320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d8c15a790 0x7f7d8c155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d939d0640 1 --2- 172.21.15.29:0/1506815320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d8c0cfe70 0x7f7d8c1558c0 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d939d0640 1 --2- 172.21.15.29:0/1506815320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8c0ce670 0x7f7d8c0c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d939d0640 1 -- 172.21.15.29:0/1506815320 >> 172.21.15.29:0/1506815320 conn(0x7f7d8c0b1ec0 msgr2=0x7f7d8c0cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d939d0640 1 -- 172.21.15.29:0/1506815320 shutdown_connections 2024-04-25T00:41:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.444+0000 7f7d939d0640 1 -- 172.21.15.29:0/1506815320 wait complete. 2024-04-25T00:41:56.462 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T00:41:56.462 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:41:56.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.584+0000 7f5fd9953640 1 Processor -- start 2024-04-25T00:41:56.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.584+0000 7f5fd9953640 1 -- start start 2024-04-25T00:41:56.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.584+0000 7f5fd9953640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5fd40b4590 0x7f5fd4070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.584+0000 7f5fd9953640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fd40cd1c0 0x7f5fd4072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.584+0000 7f5fd9953640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fd40712c0 con 0x7f5fd415e9a0 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.584+0000 7f5fd9953640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5fd4071ed0 con 0x7f5fd40b4590 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.584+0000 7f5fd9953640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5fd40715a0 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd27fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5fd40b4590 0x7f5fd4070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd37fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fd40cd1c0 0x7f5fd4072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd27fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5fd40b4590 0x7f5fd4070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.29:44130/0 (socket says 172.21.15.29:44130) 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd37fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fd40cd1c0 0x7f5fd4072360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.29:41146/0 (socket says 172.21.15.29:41146) 2024-04-25T00:41:56.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd27fc640 1 -- 172.21.15.29:0/1871648442 learned_addr learned my addr 172.21.15.29:0/1871648442 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd37fe640 1 -- 172.21.15.29:0/1871648442 >> v2:172.21.15.157:3300/0 conn(0x7f5fd40b4590 msgr2=0x7f5fd4070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd37fe640 1 --2- 172.21.15.29:0/1871648442 >> v2:172.21.15.157:3300/0 conn(0x7f5fd40b4590 0x7f5fd4070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd37fe640 1 -- 172.21.15.29:0/1871648442 >> v1:172.21.15.157:6789/0 conn(0x7f5fd415e9a0 legacy=0x7f5fd415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd37fe640 1 -- 172.21.15.29:0/1871648442 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd4071880 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd27fc640 1 --2- 172.21.15.29:0/1871648442 >> v2:172.21.15.157:3300/0 conn(0x7f5fd40b4590 0x7f5fd4070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd37fe640 1 --2- 172.21.15.29:0/1871648442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fd40cd1c0 0x7f5fd4072360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009d90 tx=0x7f5fcc01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7507d419d5ce5dce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd1ffb640 1 -- 172.21.15.29:0/1871648442 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0b2070 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd1ffb640 1 -- 172.21.15.29:0/1871648442 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fcc0a5990 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd1ffb640 1 -- 172.21.15.29:0/1871648442 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0a5d60 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.588+0000 7f5fd9953640 1 -- 172.21.15.29:0/1871648442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fd40cd1c0 msgr2=0x7f5fd4072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 --2- 172.21.15.29:0/1871648442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fd40cd1c0 0x7f5fd4072360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009d90 tx=0x7f5fcc01c8b0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 -- 172.21.15.29:0/1871648442 shutdown_connections 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 --2- 172.21.15.29:0/1871648442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fd40cd1c0 0x7f5fd4072360 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 --2- 172.21.15.29:0/1871648442 >> v2:172.21.15.157:3300/0 conn(0x7f5fd40b4590 0x7f5fd4070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 -- 172.21.15.29:0/1871648442 >> 172.21.15.29:0/1871648442 conn(0x7f5fd40b1870 msgr2=0x7f5fd40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 -- 172.21.15.29:0/1871648442 shutdown_connections 2024-04-25T00:41:56.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 -- 172.21.15.29:0/1871648442 wait complete. 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 Processor -- start 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 -- start start 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fd40b4590 0x7f5fd4158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd40cd1c0 0x7f5fd4158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fd415e9a0 0x7f5fd41593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41449f0 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd4144870 con 0x7f5fd40b4590 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5fd4165d30 con 0x7f5fd415e9a0 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd40cd1c0 0x7f5fd4158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd27fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd40cd1c0 0x7f5fd4158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56546/0 (socket says 172.21.15.29:56546) 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd27fc640 1 -- 172.21.15.29:0/1019555335 learned_addr learned my addr 172.21.15.29:0/1019555335 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd2ffd640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fd40b4590 0x7f5fd4158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd37fe640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fd415e9a0 0x7f5fd41593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd27fc640 1 -- 172.21.15.29:0/1019555335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fd415e9a0 msgr2=0x7f5fd41593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd27fc640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5fd415e9a0 0x7f5fd41593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd27fc640 1 -- 172.21.15.29:0/1019555335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fd40b4590 msgr2=0x7f5fd4158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd27fc640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fd40b4590 0x7f5fd4158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd27fc640 1 -- 172.21.15.29:0/1019555335 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd41f6770 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd2ffd640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fd40b4590 0x7f5fd4158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd27fc640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd40cd1c0 0x7f5fd4158e70 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800e9d0 tx=0x7f5fc800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fbbfff640 1 -- 172.21.15.29:0/1019555335 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8017070 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fbbfff640 1 -- 172.21.15.29:0/1019555335 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc800ca60 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fbbfff640 1 -- 172.21.15.29:0/1019555335 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc800ce10 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 -- 172.21.15.29:0/1019555335 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd41f6a30 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.592+0000 7f5fd9953640 1 -- 172.21.15.29:0/1019555335 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd41f6ff0 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.596+0000 7f5fd9953640 1 -- 172.21.15.29:0/1019555335 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5fd40692e0 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.596+0000 7f5fbbfff640 1 -- 172.21.15.29:0/1019555335 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f5fc801a070 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.596+0000 7f5fbbfff640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fc0068400 0x7f5fc006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.596+0000 7f5fbbfff640 1 -- 172.21.15.29:0/1019555335 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f5fc8084b70 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.596+0000 7f5fd2ffd640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fc0068400 0x7f5fc006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.596+0000 7f5fbbfff640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] conn(0x7f5fc006e2c0 0x7f5fc00706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.596+0000 7f5fbbfff640 1 -- 172.21.15.29:0/1019555335 --> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5fc8014e50 con 0x7f5fc006e2c0 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.596+0000 7f5fbbfff640 1 -- 172.21.15.29:0/1019555335 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=171) ==== 24+0+0 (secure 0 0 0) 0x7f5fc8087690 con 0x7f5fd40cd1c0 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.596+0000 7f5fd37fe640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] conn(0x7f5fc006e2c0 0x7f5fc00706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.596+0000 7f5fd2ffd640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fc0068400 0x7f5fc006a8c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4001670 tx=0x7f5fc4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.596+0000 7f5fd37fe640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] conn(0x7f5fc006e2c0 0x7f5fc00706e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.596+0000 7f5fbbfff640 1 -- 172.21.15.29:0/1019555335 <== osd.1 v2:172.21.15.29:6800/738833181 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5fc8014e50 con 0x7f5fc006e2c0 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.608+0000 7f5fd9953640 1 -- 172.21.15.29:0/1019555335 --> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5fd41f7440 con 0x7f5fc006e2c0 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.608+0000 7f5fbbfff640 1 -- 172.21.15.29:0/1019555335 <== osd.1 v2:172.21.15.29:6800/738833181 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5fd41f7440 con 0x7f5fc006e2c0 2024-04-25T00:41:56.611 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:41:56.612 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:41:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.612+0000 7f5fd9953640 1 -- 172.21.15.29:0/1019555335 >> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] conn(0x7f5fc006e2c0 msgr2=0x7f5fc00706e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.612+0000 7f5fd9953640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.29:6800/738833181,v1:172.21.15.29:6801/738833181] conn(0x7f5fc006e2c0 0x7f5fc00706e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.612+0000 7f5fd9953640 1 -- 172.21.15.29:0/1019555335 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fc0068400 msgr2=0x7f5fc006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.612+0000 7f5fd9953640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5fc0068400 0x7f5fc006a8c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4001670 tx=0x7f5fc4009040 comp rx=0 tx=0).stop 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.612+0000 7f5fd9953640 1 -- 172.21.15.29:0/1019555335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd40cd1c0 msgr2=0x7f5fd4158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.612+0000 7f5fd9953640 1 --2- 172.21.15.29:0/1019555335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd40cd1c0 0x7f5fd4158e70 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800e9d0 tx=0x7f5fc800ee90 comp rx=0 tx=0).stop 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.612+0000 7f5fd37fe640 1 -- 172.21.15.29:0/1019555335 reap_dead start 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.612+0000 7f5fd9953640 1 -- 172.21.15.29:0/1019555335 shutdown_connections 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.612+0000 7f5fd9953640 1 -- 172.21.15.29:0/1019555335 >> 172.21.15.29:0/1019555335 conn(0x7f5fd40b1870 msgr2=0x7f5fd4166170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.612+0000 7f5fd9953640 1 -- 172.21.15.29:0/1019555335 shutdown_connections 2024-04-25T00:41:56.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.612+0000 7f5fd9953640 1 -- 172.21.15.29:0/1019555335 wait complete. 2024-04-25T00:41:56.626 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb643135640 1 Processor -- start 2024-04-25T00:41:56.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb643135640 1 -- start start 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb643135640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb63c0b3380 0x7fb63c0b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb643135640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63c147af0 0x7fb63c147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb643135640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb63c0b3ca0 con 0x7fb63c0b5600 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb643135640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb63c163a80 con 0x7fb63c0b3380 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb643135640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb63c148c00 con 0x7fb63c147af0 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb640ed1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb63c0b3380 0x7fb63c0b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb63bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb63c0b5600 0x7fb63c163370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55802/0 (socket says 172.21.15.29:55802) 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb63bfff640 1 -- 172.21.15.29:0/4077231588 learned_addr learned my addr 172.21.15.29:0/4077231588 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb640ed1640 1 -- 172.21.15.29:0/4077231588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63c147af0 msgr2=0x7fb63c147fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb640ed1640 1 --2- 172.21.15.29:0/4077231588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63c147af0 0x7fb63c147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb640ed1640 1 -- 172.21.15.29:0/4077231588 >> v1:172.21.15.29:6789/0 conn(0x7fb63c0b5600 legacy=0x7fb63c163370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb63b7fe640 1 -- 172.21.15.29:0/4077231588 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3879022727 0 0) 0x7fb63c0b3ca0 con 0x7fb63c0b5600 2024-04-25T00:41:56.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.736+0000 7fb640ed1640 1 -- 172.21.15.29:0/4077231588 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63c148640 con 0x7fb63c0b3380 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb640ed1640 1 --2- 172.21.15.29:0/4077231588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63c0b3380 0x7fb63c0b3760 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fb6340097c0 tx=0x7fb63401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a5f773f79da555 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb63b7fe640 1 -- 172.21.15.29:0/4077231588 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6340a7070 con 0x7fb63c0b3380 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb63b7fe640 1 -- 172.21.15.29:0/4077231588 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6340b0070 con 0x7fb63c0b3380 2024-04-25T00:41:56.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb63b7fe640 1 -- 172.21.15.29:0/4077231588 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6340ab890 con 0x7fb63c0b3380 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 -- 172.21.15.29:0/4077231588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63c0b3380 msgr2=0x7fb63c0b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 --2- 172.21.15.29:0/4077231588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63c0b3380 0x7fb63c0b3760 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fb6340097c0 tx=0x7fb63401c470 comp rx=0 tx=0).stop 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 -- 172.21.15.29:0/4077231588 shutdown_connections 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 --2- 172.21.15.29:0/4077231588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63c147af0 0x7fb63c147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 --2- 172.21.15.29:0/4077231588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63c0b3380 0x7fb63c0b3760 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 -- 172.21.15.29:0/4077231588 >> 172.21.15.29:0/4077231588 conn(0x7fb63c074c00 msgr2=0x7fb63c06b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 -- 172.21.15.29:0/4077231588 shutdown_connections 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 -- 172.21.15.29:0/4077231588 wait complete. 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 Processor -- start 2024-04-25T00:41:56.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 -- start start 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63c0b3380 0x7fb63c1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63c0b5600 0x7fb63c0ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c147af0 0x7fb63c161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1669e0 con 0x7fb63c0b3380 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c166860 con 0x7fb63c147af0 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb643135640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb63c16f3f0 con 0x7fb63c0b5600 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb640ed1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63c0b3380 0x7fb63c1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb640ed1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63c0b3380 0x7fb63c1593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56566/0 (socket says 172.21.15.29:56566) 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb640ed1640 1 -- 172.21.15.29:0/865110423 learned_addr learned my addr 172.21.15.29:0/865110423 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb640ed1640 1 -- 172.21.15.29:0/865110423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63c0b5600 msgr2=0x7fb63c0ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb640ed1640 1 --2- 172.21.15.29:0/865110423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63c0b5600 0x7fb63c0ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb640ed1640 1 -- 172.21.15.29:0/865110423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c147af0 msgr2=0x7fb63c161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb640ed1640 1 --2- 172.21.15.29:0/865110423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c147af0 0x7fb63c161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb640ed1640 1 -- 172.21.15.29:0/865110423 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63c162000 con 0x7fb63c0b3380 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb640ed1640 1 --2- 172.21.15.29:0/865110423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63c0b3380 0x7fb63c1593c0 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fb6340097c0 tx=0x7fb63401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.744+0000 7fb6397fa640 1 -- 172.21.15.29:0/865110423 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634004220 con 0x7fb63c0b3380 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.748+0000 7fb6397fa640 1 -- 172.21.15.29:0/865110423 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6340043c0 con 0x7fb63c0b3380 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.748+0000 7fb643135640 1 -- 172.21.15.29:0/865110423 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63c0cf2c0 con 0x7fb63c0b3380 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.748+0000 7fb6397fa640 1 -- 172.21.15.29:0/865110423 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634003820 con 0x7fb63c0b3380 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.748+0000 7fb643135640 1 -- 172.21.15.29:0/865110423 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb63c0cf880 con 0x7fb63c0b3380 2024-04-25T00:41:56.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.748+0000 7fb643135640 1 -- 172.21.15.29:0/865110423 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb63c069270 con 0x7fb63c0b3380 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.748+0000 7fb6397fa640 1 -- 172.21.15.29:0/865110423 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fb6340039c0 con 0x7fb63c0b3380 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.748+0000 7fb6397fa640 1 --2- 172.21.15.29:0/865110423 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb628068430 0x7fb62806a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.748+0000 7fb6397fa640 1 -- 172.21.15.29:0/865110423 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7fb63411ef10 con 0x7fb63c0b3380 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.748+0000 7fb63bfff640 1 --2- 172.21.15.29:0/865110423 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb628068430 0x7fb62806a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.748+0000 7fb63bfff640 1 --2- 172.21.15.29:0/865110423 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb628068430 0x7fb62806a8f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb6300011f0 tx=0x7fb630007930 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.752+0000 7fb6397fa640 1 -- 172.21.15.29:0/865110423 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb63c162000 con 0x7fb63c0b3380 2024-04-25T00:41:56.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.912+0000 7fb643135640 1 -- 172.21.15.29:0/865110423 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fb63c0cf070 con 0x7fb628068430 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.916+0000 7fb6397fa640 1 -- 172.21.15.29:0/865110423 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb63c0cf070 con 0x7fb628068430 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.916+0000 7fb643135640 1 -- 172.21.15.29:0/865110423 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb628068430 msgr2=0x7fb62806a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.916+0000 7fb643135640 1 --2- 172.21.15.29:0/865110423 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb628068430 0x7fb62806a8f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb6300011f0 tx=0x7fb630007930 comp rx=0 tx=0).stop 2024-04-25T00:41:56.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.916+0000 7fb643135640 1 -- 172.21.15.29:0/865110423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63c0b3380 msgr2=0x7fb63c1593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.916+0000 7fb643135640 1 --2- 172.21.15.29:0/865110423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63c0b3380 0x7fb63c1593c0 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fb6340097c0 tx=0x7fb63401acb0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.916+0000 7fb643135640 1 -- 172.21.15.29:0/865110423 shutdown_connections 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.916+0000 7fb643135640 1 --2- 172.21.15.29:0/865110423 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb628068430 0x7fb62806a8f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.916+0000 7fb643135640 1 --2- 172.21.15.29:0/865110423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c147af0 0x7fb63c161ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.916+0000 7fb643135640 1 --2- 172.21.15.29:0/865110423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb63c0b5600 0x7fb63c0ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.916+0000 7fb643135640 1 --2- 172.21.15.29:0/865110423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63c0b3380 0x7fb63c1593c0 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.916+0000 7fb643135640 1 -- 172.21.15.29:0/865110423 >> 172.21.15.29:0/865110423 conn(0x7fb63c074c00 msgr2=0x7fb63c165370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.916+0000 7fb643135640 1 -- 172.21.15.29:0/865110423 shutdown_connections 2024-04-25T00:41:56.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:56.916+0000 7fb643135640 1 -- 172.21.15.29:0/865110423 wait complete. 2024-04-25T00:41:56.934 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T00:41:56.934 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:41:57.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.060+0000 7f2e4d927640 1 Processor -- start 2024-04-25T00:41:57.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.060+0000 7f2e4d927640 1 -- start start 2024-04-25T00:41:57.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.060+0000 7f2e4d927640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2e480b2b70 0x7f2e480b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.060+0000 7f2e4d927640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e4806cee0 con 0x7f2e4814bd70 2024-04-25T00:41:57.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.060+0000 7f2e4d927640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e4806d1c0 con 0x7f2e4806dca0 2024-04-25T00:41:57.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.060+0000 7f2e4d927640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2e480b3510 con 0x7f2e480b2b70 2024-04-25T00:41:57.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.060+0000 7f2e467fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2e480b2b70 0x7f2e480b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.060+0000 7f2e46ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2e4814bd70 0x7f2e48162a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55804/0 (socket says 172.21.15.29:55804) 2024-04-25T00:41:57.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.060+0000 7f2e46ffd640 1 -- 172.21.15.29:0/3739898143 learned_addr learned my addr 172.21.15.29:0/3739898143 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:57.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.060+0000 7f2e467fc640 1 -- 172.21.15.29:0/3739898143 >> v1:172.21.15.157:6789/0 conn(0x7f2e4806dca0 legacy=0x7f2e4806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:57.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.060+0000 7f2e467fc640 1 -- 172.21.15.29:0/3739898143 >> v1:172.21.15.29:6789/0 conn(0x7f2e4814bd70 legacy=0x7f2e48162a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.060+0000 7f2e45ffb640 1 -- 172.21.15.29:0/3739898143 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358837469 0 0) 0x7f2e4806cee0 con 0x7f2e4814bd70 2024-04-25T00:41:57.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.060+0000 7f2e467fc640 1 -- 172.21.15.29:0/3739898143 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4806d4a0 con 0x7f2e480b2b70 2024-04-25T00:41:57.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.064+0000 7f2e467fc640 1 --2- 172.21.15.29:0/3739898143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e480b2b70 0x7f2e480b2fd0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0097c0 tx=0x7f2e3c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c28fa777c0fa70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.064+0000 7f2e45ffb640 1 -- 172.21.15.29:0/3739898143 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0b0070 con 0x7f2e480b2b70 2024-04-25T00:41:57.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.064+0000 7f2e45ffb640 1 -- 172.21.15.29:0/3739898143 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e3c0ab600 con 0x7f2e480b2b70 2024-04-25T00:41:57.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.064+0000 7f2e45ffb640 1 -- 172.21.15.29:0/3739898143 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0ab9d0 con 0x7f2e480b2b70 2024-04-25T00:41:57.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.064+0000 7f2e4d927640 1 -- 172.21.15.29:0/3739898143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e480b2b70 msgr2=0x7f2e480b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.064+0000 7f2e4d927640 1 --2- 172.21.15.29:0/3739898143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e480b2b70 0x7f2e480b2fd0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0097c0 tx=0x7f2e3c01c460 comp rx=0 tx=0).stop 2024-04-25T00:41:57.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.064+0000 7f2e4d927640 1 -- 172.21.15.29:0/3739898143 shutdown_connections 2024-04-25T00:41:57.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.064+0000 7f2e4d927640 1 --2- 172.21.15.29:0/3739898143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e480b2b70 0x7f2e480b2fd0 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.064+0000 7f2e4d927640 1 -- 172.21.15.29:0/3739898143 >> 172.21.15.29:0/3739898143 conn(0x7f2e480b1870 msgr2=0x7f2e480b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:57.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.064+0000 7f2e4d927640 1 -- 172.21.15.29:0/3739898143 shutdown_connections 2024-04-25T00:41:57.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.064+0000 7f2e4d927640 1 -- 172.21.15.29:0/3739898143 wait complete. 2024-04-25T00:41:57.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e4d927640 1 Processor -- start 2024-04-25T00:41:57.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e4d927640 1 -- start start 2024-04-25T00:41:57.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e4d927640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4806dca0 0x7f2e48149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e4d927640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480b2b70 0x7f2e480aa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e4d927640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e4814bd70 0x7f2e4814a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e4d927640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2e48072260 con 0x7f2e4814bd70 2024-04-25T00:41:57.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e4d927640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e480720e0 con 0x7f2e4806dca0 2024-04-25T00:41:57.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e4d927640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2e480723e0 con 0x7f2e480b2b70 2024-04-25T00:41:57.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e477fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e4814bd70 0x7f2e4814a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e477fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e4814bd70 0x7f2e4814a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56584/0 (socket says 172.21.15.29:56584) 2024-04-25T00:41:57.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e477fe640 1 -- 172.21.15.29:0/538712560 learned_addr learned my addr 172.21.15.29:0/538712560 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:57.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e477fe640 1 -- 172.21.15.29:0/538712560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480b2b70 msgr2=0x7f2e480aa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e477fe640 1 --2- 172.21.15.29:0/538712560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480b2b70 0x7f2e480aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e477fe640 1 -- 172.21.15.29:0/538712560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4806dca0 msgr2=0x7f2e48149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e46ffd640 1 --2- 172.21.15.29:0/538712560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4806dca0 0x7f2e48149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e477fe640 1 --2- 172.21.15.29:0/538712560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4806dca0 0x7f2e48149110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e477fe640 1 -- 172.21.15.29:0/538712560 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e481459c0 con 0x7f2e4814bd70 2024-04-25T00:41:57.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e477fe640 1 --2- 172.21.15.29:0/538712560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e4814bd70 0x7f2e4814a960 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000b9a0 tx=0x7f2e4000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e2ffff640 1 -- 172.21.15.29:0/538712560 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40019070 con 0x7f2e4814bd70 2024-04-25T00:41:57.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e2ffff640 1 -- 172.21.15.29:0/538712560 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e4000cba0 con 0x7f2e4814bd70 2024-04-25T00:41:57.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e4d927640 1 -- 172.21.15.29:0/538712560 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e48145c80 con 0x7f2e4814bd70 2024-04-25T00:41:57.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e2ffff640 1 -- 172.21.15.29:0/538712560 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40014440 con 0x7f2e4814bd70 2024-04-25T00:41:57.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e4d927640 1 -- 172.21.15.29:0/538712560 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e48146240 con 0x7f2e4814bd70 2024-04-25T00:41:57.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e4d927640 1 -- 172.21.15.29:0/538712560 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2e480692e0 con 0x7f2e4814bd70 2024-04-25T00:41:57.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.068+0000 7f2e2ffff640 1 -- 172.21.15.29:0/538712560 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f2e4001c070 con 0x7f2e4814bd70 2024-04-25T00:41:57.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.072+0000 7f2e2ffff640 1 --2- 172.21.15.29:0/538712560 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2e34068400 0x7f2e3406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.072+0000 7f2e2ffff640 1 -- 172.21.15.29:0/538712560 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f2e40084b00 con 0x7f2e4814bd70 2024-04-25T00:41:57.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.072+0000 7f2e46ffd640 1 --2- 172.21.15.29:0/538712560 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2e34068400 0x7f2e3406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.072+0000 7f2e2ffff640 1 --2- 172.21.15.29:0/538712560 >> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] conn(0x7f2e3406e2c0 0x7f2e340706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.072+0000 7f2e2ffff640 1 -- 172.21.15.29:0/538712560 --> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e40020e50 con 0x7f2e3406e2c0 2024-04-25T00:41:57.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.072+0000 7f2e2ffff640 1 -- 172.21.15.29:0/538712560 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=171) ==== 24+0+0 (secure 0 0 0) 0x7f2e40087570 con 0x7f2e4814bd70 2024-04-25T00:41:57.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.072+0000 7f2e467fc640 1 --2- 172.21.15.29:0/538712560 >> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] conn(0x7f2e3406e2c0 0x7f2e340706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.072+0000 7f2e46ffd640 1 --2- 172.21.15.29:0/538712560 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2e34068400 0x7f2e3406a8c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2e38000f70 tx=0x7f2e38009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.072+0000 7f2e467fc640 1 --2- 172.21.15.29:0/538712560 >> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] conn(0x7f2e3406e2c0 0x7f2e340706e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.072+0000 7f2e2ffff640 1 -- 172.21.15.29:0/538712560 <== osd.2 v2:172.21.15.29:6816/802770835 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e40020e50 con 0x7f2e3406e2c0 2024-04-25T00:41:57.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.084+0000 7f2e4d927640 1 -- 172.21.15.29:0/538712560 --> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2e480b2540 con 0x7f2e3406e2c0 2024-04-25T00:41:57.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.084+0000 7f2e2ffff640 1 -- 172.21.15.29:0/538712560 <== osd.2 v2:172.21.15.29:6816/802770835 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2e480b2540 con 0x7f2e3406e2c0 2024-04-25T00:41:57.085 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:41:57.085 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:41:57.085 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:41:57.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.084+0000 7f2e4d927640 1 -- 172.21.15.29:0/538712560 >> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] conn(0x7f2e3406e2c0 msgr2=0x7f2e340706e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.084+0000 7f2e4d927640 1 --2- 172.21.15.29:0/538712560 >> [v2:172.21.15.29:6816/802770835,v1:172.21.15.29:6818/802770835] conn(0x7f2e3406e2c0 0x7f2e340706e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.084+0000 7f2e4d927640 1 -- 172.21.15.29:0/538712560 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2e34068400 msgr2=0x7f2e3406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.084+0000 7f2e4d927640 1 --2- 172.21.15.29:0/538712560 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f2e34068400 0x7f2e3406a8c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2e38000f70 tx=0x7f2e38009040 comp rx=0 tx=0).stop 2024-04-25T00:41:57.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.084+0000 7f2e4d927640 1 -- 172.21.15.29:0/538712560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e4814bd70 msgr2=0x7f2e4814a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.084+0000 7f2e4d927640 1 --2- 172.21.15.29:0/538712560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e4814bd70 0x7f2e4814a960 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000b9a0 tx=0x7f2e4000be60 comp rx=0 tx=0).stop 2024-04-25T00:41:57.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.084+0000 7f2e477fe640 1 -- 172.21.15.29:0/538712560 reap_dead start 2024-04-25T00:41:57.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.084+0000 7f2e4d927640 1 -- 172.21.15.29:0/538712560 shutdown_connections 2024-04-25T00:41:57.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.084+0000 7f2e4d927640 1 -- 172.21.15.29:0/538712560 >> 172.21.15.29:0/538712560 conn(0x7f2e480b1870 msgr2=0x7f2e480a9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:57.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.084+0000 7f2e4d927640 1 -- 172.21.15.29:0/538712560 shutdown_connections 2024-04-25T00:41:57.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.084+0000 7f2e4d927640 1 -- 172.21.15.29:0/538712560 wait complete. 2024-04-25T00:41:57.100 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T00:41:57.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77c1fb6640 1 Processor -- start 2024-04-25T00:41:57.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77c1fb6640 1 -- start start 2024-04-25T00:41:57.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77c1fb6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77bc15df70 0x7f77bc0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77c1fb6640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f77bc0b4d30 0x7f77bc0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77c1fb6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77bc06d980 con 0x7f77bc06d210 2024-04-25T00:41:57.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77c1fb6640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f77bc06dc60 con 0x7f77bc0b4d30 2024-04-25T00:41:57.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77c1fb6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f77bc06df40 con 0x7f77bc15df70 2024-04-25T00:41:57.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77bbfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f77bc0b4d30 0x7f77bc0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77b3fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f77bc06d210 0x7f77bc0b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55808/0 (socket says 172.21.15.29:55808) 2024-04-25T00:41:57.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77b3fff640 1 -- 172.21.15.29:0/896172355 learned_addr learned my addr 172.21.15.29:0/896172355 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:57.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77bb7fe640 1 --2- 172.21.15.29:0/896172355 >> v2:172.21.15.157:3300/0 conn(0x7f77bc15df70 0x7f77bc0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77bbfff640 1 -- 172.21.15.29:0/896172355 >> v2:172.21.15.157:3300/0 conn(0x7f77bc15df70 msgr2=0x7f77bc0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:57.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77bbfff640 1 --2- 172.21.15.29:0/896172355 >> v2:172.21.15.157:3300/0 conn(0x7f77bc15df70 0x7f77bc0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77bbfff640 1 -- 172.21.15.29:0/896172355 >> v1:172.21.15.29:6789/0 conn(0x7f77bc06d210 legacy=0x7f77bc0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77baffd640 1 -- 172.21.15.29:0/896172355 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4081023891 0 0) 0x7f77bc06d980 con 0x7f77bc06d210 2024-04-25T00:41:57.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.216+0000 7f77bbfff640 1 -- 172.21.15.29:0/896172355 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc06e220 con 0x7f77bc0b4d30 2024-04-25T00:41:57.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77bbfff640 1 --2- 172.21.15.29:0/896172355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc0b4d30 0x7f77bc0ad8a0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0d0bf0 tx=0x7f77b401d2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a744362ec1742db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77baffd640 1 -- 172.21.15.29:0/896172355 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b40b2070 con 0x7f77bc0b4d30 2024-04-25T00:41:57.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77baffd640 1 -- 172.21.15.29:0/896172355 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b40a5950 con 0x7f77bc0b4d30 2024-04-25T00:41:57.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77baffd640 1 -- 172.21.15.29:0/896172355 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b40a5d20 con 0x7f77bc0b4d30 2024-04-25T00:41:57.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 -- 172.21.15.29:0/896172355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc0b4d30 msgr2=0x7f77bc0ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 --2- 172.21.15.29:0/896172355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc0b4d30 0x7f77bc0ad8a0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0d0bf0 tx=0x7f77b401d2d0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 -- 172.21.15.29:0/896172355 shutdown_connections 2024-04-25T00:41:57.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 --2- 172.21.15.29:0/896172355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc0b4d30 0x7f77bc0ad8a0 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 --2- 172.21.15.29:0/896172355 >> v2:172.21.15.157:3300/0 conn(0x7f77bc15df70 0x7f77bc0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 -- 172.21.15.29:0/896172355 >> 172.21.15.29:0/896172355 conn(0x7f77bc0b1870 msgr2=0x7f77bc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:57.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 -- 172.21.15.29:0/896172355 shutdown_connections 2024-04-25T00:41:57.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 -- 172.21.15.29:0/896172355 wait complete. 2024-04-25T00:41:57.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 Processor -- start 2024-04-25T00:41:57.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 -- start start 2024-04-25T00:41:57.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc06d210 0x7f77bc14fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc0b4d30 0x7f77bc0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc15df70 0x7f77bc0ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc06eba0 con 0x7f77bc0b4d30 2024-04-25T00:41:57.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc06ea20 con 0x7f77bc06d210 2024-04-25T00:41:57.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f77bc0ae5f0 con 0x7f77bc15df70 2024-04-25T00:41:57.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77b3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc0b4d30 0x7f77bc0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77b3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc0b4d30 0x7f77bc0ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56600/0 (socket says 172.21.15.29:56600) 2024-04-25T00:41:57.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77b3fff640 1 -- 172.21.15.29:0/332154756 learned_addr learned my addr 172.21.15.29:0/332154756 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:57.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77bb7fe640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc06d210 0x7f77bc14fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77b3fff640 1 -- 172.21.15.29:0/332154756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc15df70 msgr2=0x7f77bc0ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77b3fff640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc15df70 0x7f77bc0ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77b3fff640 1 -- 172.21.15.29:0/332154756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc06d210 msgr2=0x7f77bc14fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77b3fff640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc06d210 0x7f77bc14fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77b3fff640 1 -- 172.21.15.29:0/332154756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc151750 con 0x7f77bc0b4d30 2024-04-25T00:41:57.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77bb7fe640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc06d210 0x7f77bc14fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:57.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77b3fff640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc0b4d30 0x7f77bc0ce210 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f77a40011f0 tx=0x7f77a400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77b8ff9640 1 -- 172.21.15.29:0/332154756 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4018070 con 0x7f77bc0b4d30 2024-04-25T00:41:57.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77b8ff9640 1 -- 172.21.15.29:0/332154756 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a40074e0 con 0x7f77bc0b4d30 2024-04-25T00:41:57.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.220+0000 7f77c1fb6640 1 -- 172.21.15.29:0/332154756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77bc0cbf90 con 0x7f77bc0b4d30 2024-04-25T00:41:57.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.224+0000 7f77b8ff9640 1 -- 172.21.15.29:0/332154756 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4004a10 con 0x7f77bc0b4d30 2024-04-25T00:41:57.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.224+0000 7f77c1fb6640 1 -- 172.21.15.29:0/332154756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77bc156780 con 0x7f77bc0b4d30 2024-04-25T00:41:57.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.224+0000 7f77c1fb6640 1 -- 172.21.15.29:0/332154756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77bc0cc160 con 0x7f77bc0b4d30 2024-04-25T00:41:57.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.224+0000 7f77b8ff9640 1 -- 172.21.15.29:0/332154756 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f77a401b070 con 0x7f77bc0b4d30 2024-04-25T00:41:57.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.224+0000 7f77b8ff9640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f779c068430 0x7f779c06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.224+0000 7f77b8ff9640 1 -- 172.21.15.29:0/332154756 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f77a4083b70 con 0x7f77bc0b4d30 2024-04-25T00:41:57.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.224+0000 7f77bb7fe640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f779c068430 0x7f779c06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.224+0000 7f77bb7fe640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f779c068430 0x7f779c06a8f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0b0d30 tx=0x7f77ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.228+0000 7f77b8ff9640 1 -- 172.21.15.29:0/332154756 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f77a4055f10 con 0x7f77bc0b4d30 2024-04-25T00:41:57.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.388+0000 7f77c1fb6640 1 -- 172.21.15.29:0/332154756 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f77bc0cec90 con 0x7f779c068430 2024-04-25T00:41:57.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.388+0000 7f77b8ff9640 1 -- 172.21.15.29:0/332154756 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f77bc0cec90 con 0x7f779c068430 2024-04-25T00:41:57.389 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T00:41:57.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.392+0000 7f77c1fb6640 1 -- 172.21.15.29:0/332154756 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f779c068430 msgr2=0x7f779c06a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.392+0000 7f77c1fb6640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f779c068430 0x7f779c06a8f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0b0d30 tx=0x7f77ac008040 comp rx=0 tx=0).stop 2024-04-25T00:41:57.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.392+0000 7f77c1fb6640 1 -- 172.21.15.29:0/332154756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc0b4d30 msgr2=0x7f77bc0ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.392+0000 7f77c1fb6640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc0b4d30 0x7f77bc0ce210 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f77a40011f0 tx=0x7f77a400c9b0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.392+0000 7f77c1fb6640 1 -- 172.21.15.29:0/332154756 shutdown_connections 2024-04-25T00:41:57.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.392+0000 7f77c1fb6640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f779c068430 0x7f779c06a8f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.392+0000 7f77c1fb6640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77bc15df70 0x7f77bc0ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.392+0000 7f77c1fb6640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f77bc0b4d30 0x7f77bc0ce210 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.392+0000 7f77c1fb6640 1 --2- 172.21.15.29:0/332154756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77bc06d210 0x7f77bc14fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.392+0000 7f77c1fb6640 1 -- 172.21.15.29:0/332154756 >> 172.21.15.29:0/332154756 conn(0x7f77bc0b1870 msgr2=0x7f77bc15d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:57.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.392+0000 7f77c1fb6640 1 -- 172.21.15.29:0/332154756 shutdown_connections 2024-04-25T00:41:57.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.392+0000 7f77c1fb6640 1 -- 172.21.15.29:0/332154756 wait complete. 2024-04-25T00:41:57.408 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T00:41:57.408 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:41:57.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.528+0000 7f4715042640 1 Processor -- start 2024-04-25T00:41:57.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.528+0000 7f4715042640 1 -- start start 2024-04-25T00:41:57.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f4715042640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100b4d00 0x7f4710168d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f4715042640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47100b3ac0 con 0x7f471015e1a0 2024-04-25T00:41:57.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f4715042640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47100b3d60 con 0x7f4710071b40 2024-04-25T00:41:57.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f4715042640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f47100714f0 con 0x7f47100b4d00 2024-04-25T00:41:57.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470ed9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f471015e1a0 0x7f4710070ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:55822/0 (socket says 172.21.15.29:55822) 2024-04-25T00:41:57.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470ed9d640 1 -- 172.21.15.29:0/815858941 learned_addr learned my addr 172.21.15.29:0/815858941 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:57.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2430505115 0 0) 0x7f47100b3ac0 con 0x7f471015e1a0 2024-04-25T00:41:57.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470f59e640 1 --2- 172.21.15.29:0/815858941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100b4d00 0x7f4710168d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46fc003530 con 0x7f471015e1a0 2024-04-25T00:41:57.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 402050526 0 0) 0x7f46fc003530 con 0x7f471015e1a0 2024-04-25T00:41:57.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100b4d00 msgr2=0x7f4710168d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:57.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 --2- 172.21.15.29:0/815858941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100b4d00 0x7f4710168d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 >> v1:172.21.15.157:6789/0 conn(0x7f4710071b40 legacy=0x7f47100b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47100b4040 con 0x7f471015e1a0 2024-04-25T00:41:57.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4700001680 con 0x7f471015e1a0 2024-04-25T00:41:57.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 >> v1:172.21.15.29:6789/0 conn(0x7f471015e1a0 legacy=0x7f4710070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 --2- 172.21.15.29:0/815858941 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46fc0039f0 0x7f46fc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 --2- 172.21.15.29:0/815858941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100b4d00 0x7f46fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 --2- 172.21.15.29:0/815858941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46fc004d50 0x7f46fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f47100714f0 con 0x7f46fc0039f0 2024-04-25T00:41:57.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc00bb90 con 0x7f46fc004d50 2024-04-25T00:41:57.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f46fc00bd90 con 0x7f47100b4d00 2024-04-25T00:41:57.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470ed9d640 1 --2- 172.21.15.29:0/815858941 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46fc0039f0 0x7f46fc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470659c640 1 --2- 172.21.15.29:0/815858941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100b4d00 0x7f46fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470ed9d640 1 -- 172.21.15.29:0/815858941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100b4d00 msgr2=0x7f46fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:57.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470f59e640 1 --2- 172.21.15.29:0/815858941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46fc004d50 0x7f46fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470ed9d640 1 --2- 172.21.15.29:0/815858941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100b4d00 0x7f46fc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470ed9d640 1 -- 172.21.15.29:0/815858941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46fc004d50 msgr2=0x7f46fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:57.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470ed9d640 1 --2- 172.21.15.29:0/815858941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46fc004d50 0x7f46fc0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470ed9d640 1 -- 172.21.15.29:0/815858941 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc003530 con 0x7f46fc0039f0 2024-04-25T00:41:57.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.532+0000 7f470659c640 1 --2- 172.21.15.29:0/815858941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100b4d00 0x7f46fc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:57.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470ed9d640 1 --2- 172.21.15.29:0/815858941 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46fc0039f0 0x7f46fc003e10 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f470000c190 tx=0x7f470001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae164cefbc0716e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700007c70 con 0x7f46fc0039f0 2024-04-25T00:41:57.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47000a9070 con 0x7f46fc0039f0 2024-04-25T00:41:57.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470e59c640 1 -- 172.21.15.29:0/815858941 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700005d80 con 0x7f46fc0039f0 2024-04-25T00:41:57.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 -- 172.21.15.29:0/815858941 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46fc0039f0 msgr2=0x7f46fc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 --2- 172.21.15.29:0/815858941 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46fc0039f0 0x7f46fc003e10 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f470000c190 tx=0x7f470001cbe0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470f59e640 1 -- 172.21.15.29:0/815858941 reap_dead start 2024-04-25T00:41:57.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 -- 172.21.15.29:0/815858941 shutdown_connections 2024-04-25T00:41:57.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 -- 172.21.15.29:0/815858941 >> 172.21.15.29:0/815858941 conn(0x7f47100b1870 msgr2=0x7f47100b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:57.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 -- 172.21.15.29:0/815858941 shutdown_connections 2024-04-25T00:41:57.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 -- 172.21.15.29:0/815858941 wait complete. 2024-04-25T00:41:57.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 Processor -- start 2024-04-25T00:41:57.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 -- start start 2024-04-25T00:41:57.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46fc0039f0 0x7f471013bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4710158090 0x7f47101584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4710162d80 0x7f4710163200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4710169760 con 0x7f4710158090 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f471006d210 con 0x7f46fc0039f0 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4715042640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f47101698e0 con 0x7f4710162d80 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4710158090 0x7f47101584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4710158090 0x7f47101584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56620/0 (socket says 172.21.15.29:56620) 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470659c640 1 -- 172.21.15.29:0/2581225952 learned_addr learned my addr 172.21.15.29:0/2581225952 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470ed9d640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46fc0039f0 0x7f471013bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470f59e640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4710162d80 0x7f4710163200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470659c640 1 -- 172.21.15.29:0/2581225952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4710162d80 msgr2=0x7f4710163200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470659c640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4710162d80 0x7f4710163200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470659c640 1 -- 172.21.15.29:0/2581225952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46fc0039f0 msgr2=0x7f471013bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470659c640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46fc0039f0 0x7f471013bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470659c640 1 -- 172.21.15.29:0/2581225952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471013d2c0 con 0x7f4710158090 2024-04-25T00:41:57.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f470659c640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4710158090 0x7f47101584f0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f46f800d5a0 tx=0x7f46f800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4707fff640 1 -- 172.21.15.29:0/2581225952 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8005480 con 0x7f4710158090 2024-04-25T00:41:57.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.536+0000 7f4707fff640 1 -- 172.21.15.29:0/2581225952 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f8017070 con 0x7f4710158090 2024-04-25T00:41:57.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f4715042640 1 -- 172.21.15.29:0/2581225952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471013c250 con 0x7f4710158090 2024-04-25T00:41:57.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f4707fff640 1 -- 172.21.15.29:0/2581225952 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8003840 con 0x7f4710158090 2024-04-25T00:41:57.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f4715042640 1 -- 172.21.15.29:0/2581225952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f471013c7e0 con 0x7f4710158090 2024-04-25T00:41:57.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f4715042640 1 -- 172.21.15.29:0/2581225952 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4710163ad0 con 0x7f4710158090 2024-04-25T00:41:57.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f4707fff640 1 -- 172.21.15.29:0/2581225952 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f46f801c460 con 0x7f4710158090 2024-04-25T00:41:57.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f4707fff640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f46f0068400 0x7f46f006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f4707fff640 1 -- 172.21.15.29:0/2581225952 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f46f8084e60 con 0x7f4710158090 2024-04-25T00:41:57.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f470ed9d640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f46f0068400 0x7f46f006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f4707fff640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] conn(0x7f46f006e2c0 0x7f46f00706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f4707fff640 1 -- 172.21.15.29:0/2581225952 --> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46f800cfa0 con 0x7f46f006e2c0 2024-04-25T00:41:57.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f4707fff640 1 -- 172.21.15.29:0/2581225952 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=171) ==== 24+0+0 (secure 0 0 0) 0x7f46f8087710 con 0x7f4710158090 2024-04-25T00:41:57.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f470f59e640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] conn(0x7f46f006e2c0 0x7f46f00706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f470ed9d640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f46f0068400 0x7f46f006a8c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f470000c160 tx=0x7f47000a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f470f59e640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] conn(0x7f46f006e2c0 0x7f46f00706e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.540+0000 7f4707fff640 1 -- 172.21.15.29:0/2581225952 <== osd.3 v2:172.21.15.29:6817/1979009155 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46f800cfa0 con 0x7f46f006e2c0 2024-04-25T00:41:57.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.556+0000 7f4715042640 1 -- 172.21.15.29:0/2581225952 --> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f47100692e0 con 0x7f46f006e2c0 2024-04-25T00:41:57.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.556+0000 7f4707fff640 1 -- 172.21.15.29:0/2581225952 <== osd.3 v2:172.21.15.29:6817/1979009155 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f47100692e0 con 0x7f46f006e2c0 2024-04-25T00:41:57.556 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:41:57.556 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:41:57.556 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:41:57.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.556+0000 7f4715042640 1 -- 172.21.15.29:0/2581225952 >> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] conn(0x7f46f006e2c0 msgr2=0x7f46f00706e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.556+0000 7f4715042640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.29:6817/1979009155,v1:172.21.15.29:6819/1979009155] conn(0x7f46f006e2c0 0x7f46f00706e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.556+0000 7f4715042640 1 -- 172.21.15.29:0/2581225952 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f46f0068400 msgr2=0x7f46f006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.556+0000 7f4715042640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f46f0068400 0x7f46f006a8c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f470000c160 tx=0x7f47000a7070 comp rx=0 tx=0).stop 2024-04-25T00:41:57.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.556+0000 7f4715042640 1 -- 172.21.15.29:0/2581225952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4710158090 msgr2=0x7f47101584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.556+0000 7f4715042640 1 --2- 172.21.15.29:0/2581225952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4710158090 0x7f47101584f0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f46f800d5a0 tx=0x7f46f800da60 comp rx=0 tx=0).stop 2024-04-25T00:41:57.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.556+0000 7f470f59e640 1 -- 172.21.15.29:0/2581225952 reap_dead start 2024-04-25T00:41:57.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.556+0000 7f4715042640 1 -- 172.21.15.29:0/2581225952 shutdown_connections 2024-04-25T00:41:57.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.556+0000 7f4715042640 1 -- 172.21.15.29:0/2581225952 >> 172.21.15.29:0/2581225952 conn(0x7f47100b1870 msgr2=0x7f4710166200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:57.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.556+0000 7f4715042640 1 -- 172.21.15.29:0/2581225952 shutdown_connections 2024-04-25T00:41:57.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.556+0000 7f4715042640 1 -- 172.21.15.29:0/2581225952 wait complete. 2024-04-25T00:41:57.572 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T00:41:57.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.676+0000 7f5239c33640 1 Processor -- start 2024-04-25T00:41:57.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.676+0000 7f5239c33640 1 -- start start 2024-04-25T00:41:57.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.676+0000 7f5239c33640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f523415e8d0 0x7f52340cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.676+0000 7f5239c33640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f52340b4780 0x7f523415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.676+0000 7f5239c33640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52340b3720 con 0x7f523415f1f0 2024-04-25T00:41:57.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.676+0000 7f5239c33640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f52340b42f0 con 0x7f52340b4780 2024-04-25T00:41:57.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.676+0000 7f5239c33640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f523415d7e0 con 0x7f523415e8d0 2024-04-25T00:41:57.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.676+0000 7f5233fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f523415f1f0 0x7f523415b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41854/0 (socket says 172.21.15.29:41854) 2024-04-25T00:41:57.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.676+0000 7f5233fff640 1 -- 172.21.15.29:0/232322952 learned_addr learned my addr 172.21.15.29:0/232322952 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:57.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.676+0000 7f5232ffd640 1 --2- 172.21.15.29:0/232322952 >> v2:172.21.15.29:3300/0 conn(0x7f52340b4780 0x7f523415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f52337fe640 1 --2- 172.21.15.29:0/232322952 >> v2:172.21.15.157:3300/0 conn(0x7f523415e8d0 0x7f52340cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f52327fc640 1 -- 172.21.15.29:0/232322952 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 263361279 0 0) 0x7f52340b3720 con 0x7f523415f1f0 2024-04-25T00:41:57.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f52327fc640 1 -- 172.21.15.29:0/232322952 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f521c003530 con 0x7f523415f1f0 2024-04-25T00:41:57.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5232ffd640 1 -- 172.21.15.29:0/232322952 >> v2:172.21.15.157:3300/0 conn(0x7f523415e8d0 msgr2=0x7f52340cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:57.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5232ffd640 1 --2- 172.21.15.29:0/232322952 >> v2:172.21.15.157:3300/0 conn(0x7f523415e8d0 0x7f52340cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5232ffd640 1 -- 172.21.15.29:0/232322952 >> v1:172.21.15.29:6789/0 conn(0x7f523415f1f0 legacy=0x7f523415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5232ffd640 1 -- 172.21.15.29:0/232322952 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52340b39c0 con 0x7f52340b4780 2024-04-25T00:41:57.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f52337fe640 1 --2- 172.21.15.29:0/232322952 >> v2:172.21.15.157:3300/0 conn(0x7f523415e8d0 0x7f52340cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:57.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5232ffd640 1 --2- 172.21.15.29:0/232322952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52340b4780 0x7f523415ecb0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f52340b5260 tx=0x7f522801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5589d7854322fb6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f52327fc640 1 -- 172.21.15.29:0/232322952 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523415c1c0 con 0x7f52340b4780 2024-04-25T00:41:57.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f52327fc640 1 -- 172.21.15.29:0/232322952 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52280b0070 con 0x7f52340b4780 2024-04-25T00:41:57.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f52327fc640 1 -- 172.21.15.29:0/232322952 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52280ab870 con 0x7f52340b4780 2024-04-25T00:41:57.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5239c33640 1 -- 172.21.15.29:0/232322952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52340b4780 msgr2=0x7f523415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5239c33640 1 --2- 172.21.15.29:0/232322952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52340b4780 0x7f523415ecb0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f52340b5260 tx=0x7f522801c450 comp rx=0 tx=0).stop 2024-04-25T00:41:57.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5239c33640 1 -- 172.21.15.29:0/232322952 shutdown_connections 2024-04-25T00:41:57.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5239c33640 1 --2- 172.21.15.29:0/232322952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52340b4780 0x7f523415ecb0 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5239c33640 1 --2- 172.21.15.29:0/232322952 >> v2:172.21.15.157:3300/0 conn(0x7f523415e8d0 0x7f52340cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5239c33640 1 -- 172.21.15.29:0/232322952 >> 172.21.15.29:0/232322952 conn(0x7f52340b1870 msgr2=0x7f52340b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:57.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5239c33640 1 -- 172.21.15.29:0/232322952 shutdown_connections 2024-04-25T00:41:57.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5239c33640 1 -- 172.21.15.29:0/232322952 wait complete. 2024-04-25T00:41:57.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.680+0000 7f5239c33640 1 Processor -- start 2024-04-25T00:41:57.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5239c33640 1 -- start start 2024-04-25T00:41:57.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5239c33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52340b4780 0x7f52340cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5239c33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f523415e8d0 0x7f5234158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5239c33640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523415f1f0 0x7f52341591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5239c33640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5234144a20 con 0x7f523415f1f0 2024-04-25T00:41:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5239c33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f523415f600 con 0x7f523415e8d0 2024-04-25T00:41:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5239c33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f52340aedc0 con 0x7f52340b4780 2024-04-25T00:41:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5233fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523415f1f0 0x7f52341591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5233fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523415f1f0 0x7f52341591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34666/0 (socket says 172.21.15.29:34666) 2024-04-25T00:41:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5233fff640 1 -- 172.21.15.29:0/1418721774 learned_addr learned my addr 172.21.15.29:0/1418721774 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5232ffd640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f523415e8d0 0x7f5234158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f52337fe640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52340b4780 0x7f52340cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5233fff640 1 -- 172.21.15.29:0/1418721774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52340b4780 msgr2=0x7f52340cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5233fff640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52340b4780 0x7f52340cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5233fff640 1 -- 172.21.15.29:0/1418721774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f523415e8d0 msgr2=0x7f5234158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5233fff640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f523415e8d0 0x7f5234158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5233fff640 1 -- 172.21.15.29:0/1418721774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52340af0a0 con 0x7f523415f1f0 2024-04-25T00:41:57.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5233fff640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523415f1f0 0x7f52341591d0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f522c00caa0 tx=0x7f522c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f521bfff640 1 -- 172.21.15.29:0/1418721774 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c017070 con 0x7f523415f1f0 2024-04-25T00:41:57.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f521bfff640 1 -- 172.21.15.29:0/1418721774 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f522c00ac00 con 0x7f523415f1f0 2024-04-25T00:41:57.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5239c33640 1 -- 172.21.15.29:0/1418721774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5234166c70 con 0x7f523415f1f0 2024-04-25T00:41:57.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f521bfff640 1 -- 172.21.15.29:0/1418721774 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c012400 con 0x7f523415f1f0 2024-04-25T00:41:57.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5239c33640 1 -- 172.21.15.29:0/1418721774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5234167230 con 0x7f523415f1f0 2024-04-25T00:41:57.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f5239c33640 1 -- 172.21.15.29:0/1418721774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5234069270 con 0x7f523415f1f0 2024-04-25T00:41:57.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f521bfff640 1 -- 172.21.15.29:0/1418721774 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f522c01a070 con 0x7f523415f1f0 2024-04-25T00:41:57.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f521bfff640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5220068400 0x7f522006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f521bfff640 1 -- 172.21.15.29:0/1418721774 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f522c084b40 con 0x7f523415f1f0 2024-04-25T00:41:57.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.684+0000 7f52337fe640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5220068400 0x7f522006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.688+0000 7f52337fe640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5220068400 0x7f522006a8c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5224006fd0 tx=0x7f5224008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.688+0000 7f521bfff640 1 -- 172.21.15.29:0/1418721774 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f522c021020 con 0x7f523415f1f0 2024-04-25T00:41:57.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.844+0000 7f5239c33640 1 -- 172.21.15.29:0/1418721774 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f5234167470 con 0x7f5220068400 2024-04-25T00:41:57.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.844+0000 7f521bfff640 1 -- 172.21.15.29:0/1418721774 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5224008560 con 0x7f5220068400 2024-04-25T00:41:57.846 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T00:41:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.848+0000 7f5239c33640 1 -- 172.21.15.29:0/1418721774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5220068400 msgr2=0x7f522006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.848+0000 7f5239c33640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5220068400 0x7f522006a8c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5224006fd0 tx=0x7f5224008040 comp rx=0 tx=0).stop 2024-04-25T00:41:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.848+0000 7f5239c33640 1 -- 172.21.15.29:0/1418721774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523415f1f0 msgr2=0x7f52341591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.848+0000 7f5239c33640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523415f1f0 0x7f52341591d0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f522c00caa0 tx=0x7f522c00cf60 comp rx=0 tx=0).stop 2024-04-25T00:41:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.848+0000 7f5239c33640 1 -- 172.21.15.29:0/1418721774 shutdown_connections 2024-04-25T00:41:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.848+0000 7f5239c33640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5220068400 0x7f522006a8c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.848+0000 7f5239c33640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f523415f1f0 0x7f52341591d0 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.848+0000 7f5239c33640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f523415e8d0 0x7f5234158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.848+0000 7f5239c33640 1 --2- 172.21.15.29:0/1418721774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52340b4780 0x7f52340cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.848+0000 7f5239c33640 1 -- 172.21.15.29:0/1418721774 >> 172.21.15.29:0/1418721774 conn(0x7f52340b1870 msgr2=0x7f52340b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:57.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.848+0000 7f5239c33640 1 -- 172.21.15.29:0/1418721774 shutdown_connections 2024-04-25T00:41:57.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.848+0000 7f5239c33640 1 -- 172.21.15.29:0/1418721774 wait complete. 2024-04-25T00:41:57.864 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T00:41:57.864 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:41:57.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5cc3ca640 1 Processor -- start 2024-04-25T00:41:57.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5cc3ca640 1 -- start start 2024-04-25T00:41:57.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5cc3ca640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c40b3b50 0x7ff5c40b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5cc3ca640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c40cf670 con 0x7ff5c41456d0 2024-04-25T00:41:57.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5cc3ca640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c40b4380 con 0x7ff5c406cd40 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5cc3ca640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c40b40a0 con 0x7ff5c40b3b50 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5ca166640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff5c41456d0 0x7ff5c40cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41856/0 (socket says 172.21.15.29:41856) 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5ca166640 1 -- 172.21.15.29:0/4011404615 learned_addr learned my addr 172.21.15.29:0/4011404615 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 581581034 0 0) 0x7ff5c40cf670 con 0x7ff5c41456d0 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5b0003530 con 0x7ff5c41456d0 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9965640 1 --2- 172.21.15.29:0/4011404615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c40b3b50 0x7ff5c40b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1689520623 0 0) 0x7ff5b0003530 con 0x7ff5c41456d0 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c40b3b50 msgr2=0x7ff5c40b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 --2- 172.21.15.29:0/4011404615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c40b3b50 0x7ff5c40b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 >> v1:172.21.15.157:6789/0 conn(0x7ff5c406cd40 legacy=0x7ff5c406d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c40ce480 con 0x7ff5c41456d0 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7ff5c0001a00 con 0x7ff5c41456d0 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 >> v1:172.21.15.29:6789/0 conn(0x7ff5c41456d0 legacy=0x7ff5c40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 --2- 172.21.15.29:0/4011404615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c40b3b50 0x7ff5b00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 --2- 172.21.15.29:0/4011404615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0004020 0x7ff5b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 --2- 172.21.15.29:0/4011404615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b0004d50 0x7ff5b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:57.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c40b40a0 con 0x7ff5b0004d50 2024-04-25T00:41:57.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b000bb90 con 0x7ff5b0004020 2024-04-25T00:41:57.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b000bd90 con 0x7ff5c40b3b50 2024-04-25T00:41:57.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5ca967640 1 --2- 172.21.15.29:0/4011404615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b0004d50 0x7ff5b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5c9965640 1 --2- 172.21.15.29:0/4011404615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0004020 0x7ff5b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5ca166640 1 --2- 172.21.15.29:0/4011404615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c40b3b50 0x7ff5b00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:57.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5ca967640 1 -- 172.21.15.29:0/4011404615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c40b3b50 msgr2=0x7ff5b00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:57.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5ca967640 1 --2- 172.21.15.29:0/4011404615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c40b3b50 0x7ff5b00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5ca967640 1 -- 172.21.15.29:0/4011404615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0004020 msgr2=0x7ff5b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:57.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5ca967640 1 --2- 172.21.15.29:0/4011404615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0004020 0x7ff5b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:57.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.992+0000 7ff5ca967640 1 -- 172.21.15.29:0/4011404615 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b0003530 con 0x7ff5b0004d50 2024-04-25T00:41:57.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5ca967640 1 --2- 172.21.15.29:0/4011404615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b0004d50 0x7ff5b00051d0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc001780 tx=0x7ff5bc01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ea3ad493d283a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:57.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc0013a0 con 0x7ff5b0004d50 2024-04-25T00:41:57.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5bc0a9070 con 0x7ff5b0004d50 2024-04-25T00:41:57.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5c9164640 1 -- 172.21.15.29:0/4011404615 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc003600 con 0x7ff5b0004d50 2024-04-25T00:41:57.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/4011404615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b0004d50 msgr2=0x7ff5b00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:57.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 --2- 172.21.15.29:0/4011404615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b0004d50 0x7ff5b00051d0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc001780 tx=0x7ff5bc01c420 comp rx=0 tx=0).stop 2024-04-25T00:41:57.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5ca967640 1 -- 172.21.15.29:0/4011404615 reap_dead start 2024-04-25T00:41:57.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/4011404615 shutdown_connections 2024-04-25T00:41:57.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/4011404615 >> 172.21.15.29:0/4011404615 conn(0x7ff5c40b1870 msgr2=0x7ff5c40b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:57.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/4011404615 shutdown_connections 2024-04-25T00:41:57.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/4011404615 wait complete. 2024-04-25T00:41:57.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 Processor -- start 2024-04-25T00:41:57.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 -- start start 2024-04-25T00:41:58.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0004d50 0x7ff5c41f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c40a9490 0x7ff5c40a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c4071d30 0x7ff5c40721b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c4145ab0 con 0x7ff5c4071d30 2024-04-25T00:41:58.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c40cea40 con 0x7ff5b0004d50 2024-04-25T00:41:58.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5cc3ca640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c40b5000 con 0x7ff5c40a9490 2024-04-25T00:41:58.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5ca967640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c4071d30 0x7ff5c40721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5ca967640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c4071d30 0x7ff5c40721b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34672/0 (socket says 172.21.15.29:34672) 2024-04-25T00:41:58.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:57.996+0000 7ff5ca967640 1 -- 172.21.15.29:0/2657808203 learned_addr learned my addr 172.21.15.29:0/2657808203 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:58.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5ca166640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0004d50 0x7ff5c41f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5c9965640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c40a9490 0x7ff5c40a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5ca967640 1 -- 172.21.15.29:0/2657808203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c40a9490 msgr2=0x7ff5c40a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5ca967640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c40a9490 0x7ff5c40a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5ca967640 1 -- 172.21.15.29:0/2657808203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0004d50 msgr2=0x7ff5c41f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5ca967640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0004d50 0x7ff5c41f3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5ca967640 1 -- 172.21.15.29:0/2657808203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c41f3760 con 0x7ff5c4071d30 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5ca166640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0004d50 0x7ff5c41f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5c9965640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c40a9490 0x7ff5c40a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5ca967640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c4071d30 0x7ff5c40721b0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc097e30 tx=0x7ff5b0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5baffd640 1 -- 172.21.15.29:0/2657808203 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc0b2040 con 0x7ff5c4071d30 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5baffd640 1 -- 172.21.15.29:0/2657808203 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5bc0bc070 con 0x7ff5c4071d30 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/2657808203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c40a9f50 con 0x7ff5c4071d30 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5baffd640 1 -- 172.21.15.29:0/2657808203 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc0013a0 con 0x7ff5c4071d30 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/2657808203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c41582d0 con 0x7ff5c4071d30 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/2657808203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff5c40726f0 con 0x7ff5c4071d30 2024-04-25T00:41:58.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5baffd640 1 -- 172.21.15.29:0/2657808203 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7ff5bc0ab070 con 0x7ff5c4071d30 2024-04-25T00:41:58.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5baffd640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5a8068400 0x7ff5a806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5baffd640 1 -- 172.21.15.29:0/2657808203 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7ff5bc11ee40 con 0x7ff5c4071d30 2024-04-25T00:41:58.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5ca166640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5a8068400 0x7ff5a806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5baffd640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] conn(0x7ff5a806e2c0 0x7ff5a80706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5baffd640 1 -- 172.21.15.29:0/2657808203 --> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5a8070d90 con 0x7ff5a806e2c0 2024-04-25T00:41:58.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5baffd640 1 -- 172.21.15.29:0/2657808203 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=171) ==== 24+0+0 (secure 0 0 0) 0x7ff5bc121600 con 0x7ff5c4071d30 2024-04-25T00:41:58.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5ca166640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5a8068400 0x7ff5a806a8c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0005d20 tx=0x7ff5c0001260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5c9965640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] conn(0x7ff5a806e2c0 0x7ff5a80706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.000+0000 7ff5c9965640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] conn(0x7ff5a806e2c0 0x7ff5a80706e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.004+0000 7ff5baffd640 1 -- 172.21.15.29:0/2657808203 <== osd.4 v2:172.21.15.157:6801/674816998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff5a8070d90 con 0x7ff5a806e2c0 2024-04-25T00:41:58.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.016+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/2657808203 --> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff5c4158b70 con 0x7ff5a806e2c0 2024-04-25T00:41:58.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.016+0000 7ff5baffd640 1 -- 172.21.15.29:0/2657808203 <== osd.4 v2:172.21.15.157:6801/674816998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff5c4158b70 con 0x7ff5a806e2c0 2024-04-25T00:41:58.018 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:41:58.018 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:41:58.018 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:41:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.016+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/2657808203 >> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] conn(0x7ff5a806e2c0 msgr2=0x7ff5a80706e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.016+0000 7ff5cc3ca640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.157:6801/674816998,v1:172.21.15.157:6803/674816998] conn(0x7ff5a806e2c0 0x7ff5a80706e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.016+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/2657808203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5a8068400 msgr2=0x7ff5a806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.016+0000 7ff5cc3ca640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7ff5a8068400 0x7ff5a806a8c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0005d20 tx=0x7ff5c0001260 comp rx=0 tx=0).stop 2024-04-25T00:41:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.016+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/2657808203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c4071d30 msgr2=0x7ff5c40721b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.016+0000 7ff5cc3ca640 1 --2- 172.21.15.29:0/2657808203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c4071d30 0x7ff5c40721b0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc097e30 tx=0x7ff5b0005840 comp rx=0 tx=0).stop 2024-04-25T00:41:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.016+0000 7ff5ca967640 1 -- 172.21.15.29:0/2657808203 reap_dead start 2024-04-25T00:41:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.016+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/2657808203 shutdown_connections 2024-04-25T00:41:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.016+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/2657808203 >> 172.21.15.29:0/2657808203 conn(0x7ff5c40b1870 msgr2=0x7ff5c413b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.016+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/2657808203 shutdown_connections 2024-04-25T00:41:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.016+0000 7ff5cc3ca640 1 -- 172.21.15.29:0/2657808203 wait complete. 2024-04-25T00:41:58.033 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T00:41:58.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face4571640 1 Processor -- start 2024-04-25T00:41:58.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face4571640 1 -- start start 2024-04-25T00:41:58.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face4571640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facdc0d2c60 0x7facdc168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face4571640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facdc0cfc10 con 0x7facdc0cf690 2024-04-25T00:41:58.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face4571640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facdc0cfef0 con 0x7facdc15e200 2024-04-25T00:41:58.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face4571640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7facdc0b38d0 con 0x7facdc0d2c60 2024-04-25T00:41:58.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face1b0c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7facdc0cf690 0x7facdc0d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41858/0 (socket says 172.21.15.29:41858) 2024-04-25T00:41:58.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face1b0c640 1 -- 172.21.15.29:0/3080241038 learned_addr learned my addr 172.21.15.29:0/3080241038 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:58.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face2b0e640 1 --2- 172.21.15.29:0/3080241038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facdc0d2c60 0x7facdc168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2671418937 0 0) 0x7facdc0cfc10 con 0x7facdc0cf690 2024-04-25T00:41:58.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facc8003530 con 0x7facdc0cf690 2024-04-25T00:41:58.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1431629672 0 0) 0x7facc8003530 con 0x7facdc0cf690 2024-04-25T00:41:58.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facdc0d2c60 msgr2=0x7facdc168e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:58.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 --2- 172.21.15.29:0/3080241038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facdc0d2c60 0x7facdc168e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 >> v1:172.21.15.157:6789/0 conn(0x7facdc15e200 legacy=0x7facdc0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc0b3b20 con 0x7facdc0cf690 2024-04-25T00:41:58.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7facd80016a0 con 0x7facdc0cf690 2024-04-25T00:41:58.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 >> v1:172.21.15.29:6789/0 conn(0x7facdc0cf690 legacy=0x7facdc0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 --2- 172.21.15.29:0/3080241038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facc80039f0 0x7facc8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 --2- 172.21.15.29:0/3080241038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facc8004810 0x7facc8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 --2- 172.21.15.29:0/3080241038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facdc0d2c60 0x7facc80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7facdc0b38d0 con 0x7facc80039f0 2024-04-25T00:41:58.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.140+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7facc800bae0 con 0x7facc8004810 2024-04-25T00:41:58.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7facc800bce0 con 0x7facdc0d2c60 2024-04-25T00:41:58.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face230d640 1 --2- 172.21.15.29:0/3080241038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facc80039f0 0x7facc8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face1b0c640 1 --2- 172.21.15.29:0/3080241038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facc8004810 0x7facc8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face2b0e640 1 --2- 172.21.15.29:0/3080241038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facdc0d2c60 0x7facc80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face230d640 1 -- 172.21.15.29:0/3080241038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facdc0d2c60 msgr2=0x7facc80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:58.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face230d640 1 --2- 172.21.15.29:0/3080241038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facdc0d2c60 0x7facc80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face230d640 1 -- 172.21.15.29:0/3080241038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facc8004810 msgr2=0x7facc8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:58.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face230d640 1 --2- 172.21.15.29:0/3080241038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facc8004810 0x7facc8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face230d640 1 -- 172.21.15.29:0/3080241038 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc8003530 con 0x7facc80039f0 2024-04-25T00:41:58.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face230d640 1 --2- 172.21.15.29:0/3080241038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facc80039f0 0x7facc8003e10 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7faccc001750 tx=0x7faccc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13448265a0c061f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc0013a0 con 0x7facc80039f0 2024-04-25T00:41:58.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faccc0052f0 con 0x7facc80039f0 2024-04-25T00:41:58.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face130b640 1 -- 172.21.15.29:0/3080241038 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc003840 con 0x7facc80039f0 2024-04-25T00:41:58.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face4571640 1 -- 172.21.15.29:0/3080241038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facc80039f0 msgr2=0x7facc8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face4571640 1 --2- 172.21.15.29:0/3080241038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facc80039f0 0x7facc8003e10 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7faccc001750 tx=0x7faccc01c3c0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face2b0e640 1 -- 172.21.15.29:0/3080241038 reap_dead start 2024-04-25T00:41:58.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face4571640 1 -- 172.21.15.29:0/3080241038 shutdown_connections 2024-04-25T00:41:58.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face4571640 1 -- 172.21.15.29:0/3080241038 >> 172.21.15.29:0/3080241038 conn(0x7facdc09d280 msgr2=0x7facdc0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:58.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face4571640 1 -- 172.21.15.29:0/3080241038 shutdown_connections 2024-04-25T00:41:58.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.144+0000 7face4571640 1 -- 172.21.15.29:0/3080241038 wait complete. 2024-04-25T00:41:58.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face4571640 1 Processor -- start 2024-04-25T00:41:58.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face4571640 1 -- start start 2024-04-25T00:41:58.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face4571640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facc80039f0 0x7facdc14e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face4571640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facdc0c7aa0 0x7facdc15f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face4571640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facdc0c7f20 0x7facdc15fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face4571640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1699c0 con 0x7facc80039f0 2024-04-25T00:41:58.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face4571640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7facdc169840 con 0x7facdc0c7aa0 2024-04-25T00:41:58.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face4571640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7facdc169b40 con 0x7facdc0c7f20 2024-04-25T00:41:58.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face230d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facc80039f0 0x7facdc14e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face230d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facc80039f0 0x7facdc14e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34680/0 (socket says 172.21.15.29:34680) 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face230d640 1 -- 172.21.15.29:0/1336322715 learned_addr learned my addr 172.21.15.29:0/1336322715 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face230d640 1 -- 172.21.15.29:0/1336322715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facdc0c7f20 msgr2=0x7facdc15fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face2b0e640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facdc0c7f20 0x7facdc15fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face1b0c640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facdc0c7aa0 0x7facdc15f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face230d640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facdc0c7f20 0x7facdc15fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face230d640 1 -- 172.21.15.29:0/1336322715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facdc0c7aa0 msgr2=0x7facdc15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face230d640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facdc0c7aa0 0x7facdc15f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face230d640 1 -- 172.21.15.29:0/1336322715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc14ecd0 con 0x7facc80039f0 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face230d640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facc80039f0 0x7facdc14e200 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7faccc005040 tx=0x7facc8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7facd2ffd640 1 -- 172.21.15.29:0/1336322715 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc007a40 con 0x7facc80039f0 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7facd2ffd640 1 -- 172.21.15.29:0/1336322715 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faccc0bb070 con 0x7facc80039f0 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face4571640 1 -- 172.21.15.29:0/1336322715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc159c20 con 0x7facc80039f0 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7facd2ffd640 1 -- 172.21.15.29:0/1336322715 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc0013a0 con 0x7facc80039f0 2024-04-25T00:41:58.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face4571640 1 -- 172.21.15.29:0/1336322715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc15a1e0 con 0x7facc80039f0 2024-04-25T00:41:58.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7face4571640 1 -- 172.21.15.29:0/1336322715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facdc069270 con 0x7facc80039f0 2024-04-25T00:41:58.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.148+0000 7facd2ffd640 1 -- 172.21.15.29:0/1336322715 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7faccc0b4040 con 0x7facc80039f0 2024-04-25T00:41:58.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.152+0000 7facd2ffd640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7facc0068430 0x7facc006a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.152+0000 7facd2ffd640 1 -- 172.21.15.29:0/1336322715 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7faccc11e520 con 0x7facc80039f0 2024-04-25T00:41:58.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.152+0000 7face1b0c640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7facc0068430 0x7facc006a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.152+0000 7face1b0c640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7facc0068430 0x7facc006a8f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7facd8001ec0 tx=0x7facd8000f20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.156+0000 7facd2ffd640 1 -- 172.21.15.29:0/1336322715 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faccc0160d0 con 0x7facc80039f0 2024-04-25T00:41:58.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.308+0000 7face4571640 1 -- 172.21.15.29:0/1336322715 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7facdc158510 con 0x7facc0068430 2024-04-25T00:41:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.312+0000 7facd2ffd640 1 -- 172.21.15.29:0/1336322715 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7facdc158510 con 0x7facc0068430 2024-04-25T00:41:58.311 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T00:41:58.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.312+0000 7face4571640 1 -- 172.21.15.29:0/1336322715 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7facc0068430 msgr2=0x7facc006a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.312+0000 7face4571640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7facc0068430 0x7facc006a8f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7facd8001ec0 tx=0x7facd8000f20 comp rx=0 tx=0).stop 2024-04-25T00:41:58.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.312+0000 7face4571640 1 -- 172.21.15.29:0/1336322715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facc80039f0 msgr2=0x7facdc14e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.312+0000 7face4571640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facc80039f0 0x7facdc14e200 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7faccc005040 tx=0x7facc8005840 comp rx=0 tx=0).stop 2024-04-25T00:41:58.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.312+0000 7face4571640 1 -- 172.21.15.29:0/1336322715 shutdown_connections 2024-04-25T00:41:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.312+0000 7face4571640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7facc0068430 0x7facc006a8f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.312+0000 7face4571640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facdc0c7f20 0x7facdc15fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.312+0000 7face4571640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facdc0c7aa0 0x7facdc15f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.312+0000 7face4571640 1 --2- 172.21.15.29:0/1336322715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facc80039f0 0x7facdc14e200 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.312+0000 7face4571640 1 -- 172.21.15.29:0/1336322715 >> 172.21.15.29:0/1336322715 conn(0x7facdc09d280 msgr2=0x7facdc0b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.312+0000 7face4571640 1 -- 172.21.15.29:0/1336322715 shutdown_connections 2024-04-25T00:41:58.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.312+0000 7face4571640 1 -- 172.21.15.29:0/1336322715 wait complete. 2024-04-25T00:41:58.329 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T00:41:58.329 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:41:58.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bc60df640 1 Processor -- start 2024-04-25T00:41:58.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bc60df640 1 -- start start 2024-04-25T00:41:58.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bc60df640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5bc015e180 0x7f5bc00d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bc60df640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5bc015e560 0x7f5bc00cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bc60df640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc00cfed0 con 0x7f5bc00d2680 2024-04-25T00:41:58.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bc60df640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5bc00b7100 con 0x7f5bc015e180 2024-04-25T00:41:58.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bc60df640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5bc00b73e0 con 0x7f5bc015e560 2024-04-25T00:41:58.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bbffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5bc00d2680 0x7f5bc00d2ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41870/0 (socket says 172.21.15.29:41870) 2024-04-25T00:41:58.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bbffff640 1 -- 172.21.15.29:0/686068309 learned_addr learned my addr 172.21.15.29:0/686068309 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:58.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bbf7fe640 1 --2- 172.21.15.29:0/686068309 >> v2:172.21.15.29:3300/0 conn(0x7f5bc015e180 0x7f5bc00d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bbeffd640 1 --2- 172.21.15.29:0/686068309 >> v2:172.21.15.157:3300/0 conn(0x7f5bc015e560 0x7f5bc00cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bbe7fc640 1 -- 172.21.15.29:0/686068309 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3831312167 0 0) 0x7f5bc00cfed0 con 0x7f5bc00d2680 2024-04-25T00:41:58.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bbe7fc640 1 -- 172.21.15.29:0/686068309 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ba8003530 con 0x7f5bc00d2680 2024-04-25T00:41:58.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bbf7fe640 1 -- 172.21.15.29:0/686068309 >> v2:172.21.15.157:3300/0 conn(0x7f5bc015e560 msgr2=0x7f5bc00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:58.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bbf7fe640 1 --2- 172.21.15.29:0/686068309 >> v2:172.21.15.157:3300/0 conn(0x7f5bc015e560 0x7f5bc00cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bbf7fe640 1 -- 172.21.15.29:0/686068309 >> v1:172.21.15.29:6789/0 conn(0x7f5bc00d2680 legacy=0x7f5bc00d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.456+0000 7f5bbf7fe640 1 -- 172.21.15.29:0/686068309 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc00b76c0 con 0x7f5bc015e180 2024-04-25T00:41:58.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bbf7fe640 1 --2- 172.21.15.29:0/686068309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bc015e180 0x7f5bc00d20e0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f5bc008e810 tx=0x7f5bb001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b7b4350852f750 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bbe7fc640 1 -- 172.21.15.29:0/686068309 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00b5070 con 0x7f5bc015e180 2024-04-25T00:41:58.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bbe7fc640 1 -- 172.21.15.29:0/686068309 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb00a8970 con 0x7f5bc015e180 2024-04-25T00:41:58.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bbe7fc640 1 -- 172.21.15.29:0/686068309 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00a8d20 con 0x7f5bc015e180 2024-04-25T00:41:58.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bc60df640 1 -- 172.21.15.29:0/686068309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bc015e180 msgr2=0x7f5bc00d20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bc60df640 1 --2- 172.21.15.29:0/686068309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bc015e180 0x7f5bc00d20e0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f5bc008e810 tx=0x7f5bb001ec70 comp rx=0 tx=0).stop 2024-04-25T00:41:58.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bc60df640 1 -- 172.21.15.29:0/686068309 shutdown_connections 2024-04-25T00:41:58.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bc60df640 1 --2- 172.21.15.29:0/686068309 >> v2:172.21.15.157:3300/0 conn(0x7f5bc015e560 0x7f5bc00cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bc60df640 1 --2- 172.21.15.29:0/686068309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bc015e180 0x7f5bc00d20e0 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bc60df640 1 -- 172.21.15.29:0/686068309 >> 172.21.15.29:0/686068309 conn(0x7f5bc00d40a0 msgr2=0x7f5bc00d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:58.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bc60df640 1 -- 172.21.15.29:0/686068309 shutdown_connections 2024-04-25T00:41:58.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bc60df640 1 -- 172.21.15.29:0/686068309 wait complete. 2024-04-25T00:41:58.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bc60df640 1 Processor -- start 2024-04-25T00:41:58.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.460+0000 7f5bc60df640 1 -- start start 2024-04-25T00:41:58.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bc60df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bc00d2680 0x7f5bc015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bc60df640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bc015e180 0x7f5bc00cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bc60df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bc015e560 0x7f5bc00cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bc60df640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc00b7d60 con 0x7f5bc015e180 2024-04-25T00:41:58.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bc60df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc00b7be0 con 0x7f5bc015e560 2024-04-25T00:41:58.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bc60df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc00b7ee0 con 0x7f5bc00d2680 2024-04-25T00:41:58.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bbeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bc015e180 0x7f5bc00cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bbeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bc015e180 0x7f5bc00cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34688/0 (socket says 172.21.15.29:34688) 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bbeffd640 1 -- 172.21.15.29:0/2916471937 learned_addr learned my addr 172.21.15.29:0/2916471937 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bbeffd640 1 -- 172.21.15.29:0/2916471937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bc00d2680 msgr2=0x7f5bc015ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bbffff640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bc015e560 0x7f5bc00cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bbf7fe640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bc00d2680 0x7f5bc015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bbeffd640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bc00d2680 0x7f5bc015ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bbeffd640 1 -- 172.21.15.29:0/2916471937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bc015e560 msgr2=0x7f5bc00cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bbeffd640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bc015e560 0x7f5bc00cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bbeffd640 1 -- 172.21.15.29:0/2916471937 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc00cbe60 con 0x7f5bc015e180 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bbeffd640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bc015e180 0x7f5bc00cd710 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400d5f0 tx=0x7f5bb400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5ba7fff640 1 -- 172.21.15.29:0/2916471937 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb40125f0 con 0x7f5bc015e180 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5ba7fff640 1 -- 172.21.15.29:0/2916471937 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb4019070 con 0x7f5bc015e180 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5ba7fff640 1 -- 172.21.15.29:0/2916471937 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb40129b0 con 0x7f5bc015e180 2024-04-25T00:41:58.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bc60df640 1 -- 172.21.15.29:0/2916471937 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc00cc140 con 0x7f5bc015e180 2024-04-25T00:41:58.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bc60df640 1 -- 172.21.15.29:0/2916471937 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc00cc700 con 0x7f5bc015e180 2024-04-25T00:41:58.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bc60df640 1 -- 172.21.15.29:0/2916471937 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5bc00692e0 con 0x7f5bc015e180 2024-04-25T00:41:58.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5ba7fff640 1 -- 172.21.15.29:0/2916471937 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f5bb4012b50 con 0x7f5bc015e180 2024-04-25T00:41:58.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5ba7fff640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5bac068430 0x7f5bac06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5ba7fff640 1 -- 172.21.15.29:0/2916471937 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f5bb40854a0 con 0x7f5bc015e180 2024-04-25T00:41:58.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5bbf7fe640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5bac068430 0x7f5bac06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5ba7fff640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] conn(0x7f5bac06e2f0 0x7f5bac070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5ba7fff640 1 -- 172.21.15.29:0/2916471937 --> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5bac070dc0 con 0x7f5bac06e2f0 2024-04-25T00:41:58.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.464+0000 7f5ba7fff640 1 -- 172.21.15.29:0/2916471937 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=171) ==== 24+0+0 (secure 0 0 0) 0x7f5bb4088030 con 0x7f5bc015e180 2024-04-25T00:41:58.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.468+0000 7f5bbffff640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] conn(0x7f5bac06e2f0 0x7f5bac070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.468+0000 7f5bbf7fe640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5bac068430 0x7f5bac06a8f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00b2000 tx=0x7f5bb00aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.468+0000 7f5bbffff640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] conn(0x7f5bac06e2f0 0x7f5bac070710 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.468+0000 7f5ba7fff640 1 -- 172.21.15.29:0/2916471937 <== osd.5 v2:172.21.15.157:6800/486133226 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bac070dc0 con 0x7f5bac06e2f0 2024-04-25T00:41:58.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.480+0000 7f5bc60df640 1 -- 172.21.15.29:0/2916471937 --> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5bc00b51d0 con 0x7f5bac06e2f0 2024-04-25T00:41:58.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.484+0000 7f5ba7fff640 1 -- 172.21.15.29:0/2916471937 <== osd.5 v2:172.21.15.157:6800/486133226 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5bc00b51d0 con 0x7f5bac06e2f0 2024-04-25T00:41:58.483 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:41:58.483 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:41:58.483 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:41:58.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.484+0000 7f5bc60df640 1 -- 172.21.15.29:0/2916471937 >> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] conn(0x7f5bac06e2f0 msgr2=0x7f5bac070710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.484+0000 7f5bc60df640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.157:6800/486133226,v1:172.21.15.157:6802/486133226] conn(0x7f5bac06e2f0 0x7f5bac070710 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.484+0000 7f5bc60df640 1 -- 172.21.15.29:0/2916471937 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5bac068430 msgr2=0x7f5bac06a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.484+0000 7f5bc60df640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5bac068430 0x7f5bac06a8f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00b2000 tx=0x7f5bb00aa070 comp rx=0 tx=0).stop 2024-04-25T00:41:58.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.484+0000 7f5bc60df640 1 -- 172.21.15.29:0/2916471937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bc015e180 msgr2=0x7f5bc00cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.484+0000 7f5bc60df640 1 --2- 172.21.15.29:0/2916471937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bc015e180 0x7f5bc00cd710 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400d5f0 tx=0x7f5bb400dab0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.484+0000 7f5bbffff640 1 -- 172.21.15.29:0/2916471937 reap_dead start 2024-04-25T00:41:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.484+0000 7f5bc60df640 1 -- 172.21.15.29:0/2916471937 shutdown_connections 2024-04-25T00:41:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.484+0000 7f5bc60df640 1 -- 172.21.15.29:0/2916471937 >> 172.21.15.29:0/2916471937 conn(0x7f5bc00d40a0 msgr2=0x7f5bc00d1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.484+0000 7f5bc60df640 1 -- 172.21.15.29:0/2916471937 shutdown_connections 2024-04-25T00:41:58.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.484+0000 7f5bc60df640 1 -- 172.21.15.29:0/2916471937 wait complete. 2024-04-25T00:41:58.499 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T00:41:58.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f1601be8640 1 Processor -- start 2024-04-25T00:41:58.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f1601be8640 1 -- start start 2024-04-25T00:41:58.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f1601be8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f15fc0b3140 0x7f15fc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f1601be8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15fc0b3c30 0x7f15fc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f1601be8640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15fc0b41e0 con 0x7f15fc0b4b10 2024-04-25T00:41:58.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f1601be8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f15fc06d960 con 0x7f15fc0b3140 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f1601be8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f15fc06dc40 con 0x7f15fc0b3c30 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15faffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f15fc0b4b10 0x7f15fc0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41880/0 (socket says 172.21.15.29:41880) 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15faffd640 1 -- 172.21.15.29:0/2812202678 learned_addr learned my addr 172.21.15.29:0/2812202678 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fb7fe640 1 --2- 172.21.15.29:0/2812202678 >> v2:172.21.15.157:3300/0 conn(0x7f15fc0b3140 0x7f15fc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fbfff640 1 --2- 172.21.15.29:0/2812202678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15fc0b3c30 0x7f15fc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1554363650 0 0) 0x7f15fc0b41e0 con 0x7f15fc0b4b10 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15e4003530 con 0x7f15fc0b4b10 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3587277410 0 0) 0x7f15e4003530 con 0x7f15fc0b4b10 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15fc0b3c30 msgr2=0x7f15fc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 --2- 172.21.15.29:0/2812202678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15fc0b3c30 0x7f15fc0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 >> v2:172.21.15.157:3300/0 conn(0x7f15fc0b3140 msgr2=0x7f15fc06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 --2- 172.21.15.29:0/2812202678 >> v2:172.21.15.157:3300/0 conn(0x7f15fc0b3140 0x7f15fc06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc06df20 con 0x7f15fc0b4b10 2024-04-25T00:41:58.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f15f0001680 con 0x7f15fc0b4b10 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 >> v1:172.21.15.29:6789/0 conn(0x7f15fc0b4b10 legacy=0x7f15fc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 --2- 172.21.15.29:0/2812202678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e40039c0 0x7f15e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 --2- 172.21.15.29:0/2812202678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15fc0b3c30 0x7f15e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 --2- 172.21.15.29:0/2812202678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e4004d50 0x7f15e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc06d960 con 0x7f15e4004d50 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc06dc40 con 0x7f15e40039c0 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f15e400b9d0 con 0x7f15fc0b3c30 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fbfff640 1 --2- 172.21.15.29:0/2812202678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e4004d50 0x7f15e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fb7fe640 1 --2- 172.21.15.29:0/2812202678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e40039c0 0x7f15e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15faffd640 1 --2- 172.21.15.29:0/2812202678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15fc0b3c30 0x7f15e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fbfff640 1 -- 172.21.15.29:0/2812202678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15fc0b3c30 msgr2=0x7f15e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fbfff640 1 --2- 172.21.15.29:0/2812202678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15fc0b3c30 0x7f15e4004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fbfff640 1 -- 172.21.15.29:0/2812202678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e40039c0 msgr2=0x7f15e4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:58.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fbfff640 1 --2- 172.21.15.29:0/2812202678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e40039c0 0x7f15e4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.608+0000 7f15fbfff640 1 -- 172.21.15.29:0/2812202678 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e4003530 con 0x7f15e4004d50 2024-04-25T00:41:58.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.612+0000 7f15fbfff640 1 --2- 172.21.15.29:0/2812202678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e4004d50 0x7f15e40051d0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009fc0 tx=0x7f15f401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60330b916dae35d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.612+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f40b2070 con 0x7f15e4004d50 2024-04-25T00:41:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.612+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f40a5c90 con 0x7f15e4004d50 2024-04-25T00:41:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.612+0000 7f15fa7fc640 1 -- 172.21.15.29:0/2812202678 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4009c10 con 0x7f15e4004d50 2024-04-25T00:41:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.612+0000 7f1601be8640 1 -- 172.21.15.29:0/2812202678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e4004d50 msgr2=0x7f15e40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.612+0000 7f1601be8640 1 --2- 172.21.15.29:0/2812202678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e4004d50 0x7f15e40051d0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009fc0 tx=0x7f15f401cc80 comp rx=0 tx=0).stop 2024-04-25T00:41:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.612+0000 7f15fbfff640 1 -- 172.21.15.29:0/2812202678 reap_dead start 2024-04-25T00:41:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.612+0000 7f1601be8640 1 -- 172.21.15.29:0/2812202678 shutdown_connections 2024-04-25T00:41:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.612+0000 7f1601be8640 1 -- 172.21.15.29:0/2812202678 >> 172.21.15.29:0/2812202678 conn(0x7f15fc0b1870 msgr2=0x7f15fc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.612+0000 7f1601be8640 1 -- 172.21.15.29:0/2812202678 shutdown_connections 2024-04-25T00:41:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.612+0000 7f1601be8640 1 -- 172.21.15.29:0/2812202678 wait complete. 2024-04-25T00:41:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.612+0000 7f1601be8640 1 Processor -- start 2024-04-25T00:41:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.612+0000 7f1601be8640 1 -- start start 2024-04-25T00:41:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f1601be8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15fc1f6930 0x7f15fc071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f1601be8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15fc070dc0 0x7f15fc0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f1601be8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15fc0cbc30 0x7f15fc0ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f1601be8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc145a90 con 0x7f15fc070dc0 2024-04-25T00:41:58.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f1601be8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc145910 con 0x7f15fc1f6930 2024-04-25T00:41:58.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f1601be8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f15fc0ce240 con 0x7f15fc0cbc30 2024-04-25T00:41:58.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15faffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15fc070dc0 0x7f15fc0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15faffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15fc070dc0 0x7f15fc0cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34702/0 (socket says 172.21.15.29:34702) 2024-04-25T00:41:58.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15faffd640 1 -- 172.21.15.29:0/2215928377 learned_addr learned my addr 172.21.15.29:0/2215928377 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:58.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15fb7fe640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15fc1f6930 0x7f15fc071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15fbfff640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15fc0cbc30 0x7f15fc0ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15faffd640 1 -- 172.21.15.29:0/2215928377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15fc0cbc30 msgr2=0x7f15fc0ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15faffd640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15fc0cbc30 0x7f15fc0ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15faffd640 1 -- 172.21.15.29:0/2215928377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15fc1f6930 msgr2=0x7f15fc071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15faffd640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15fc1f6930 0x7f15fc071bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15faffd640 1 -- 172.21.15.29:0/2215928377 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc072550 con 0x7f15fc070dc0 2024-04-25T00:41:58.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15faffd640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15fc070dc0 0x7f15fc0cb6f0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f15f000dae0 tx=0x7f15f0001e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15e3fff640 1 -- 172.21.15.29:0/2215928377 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0019070 con 0x7f15fc070dc0 2024-04-25T00:41:58.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15e3fff640 1 -- 172.21.15.29:0/2215928377 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f0004cc0 con 0x7f15fc070dc0 2024-04-25T00:41:58.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15e3fff640 1 -- 172.21.15.29:0/2215928377 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f00072d0 con 0x7f15fc070dc0 2024-04-25T00:41:58.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f1601be8640 1 -- 172.21.15.29:0/2215928377 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15fc0c95a0 con 0x7f15fc070dc0 2024-04-25T00:41:58.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f1601be8640 1 -- 172.21.15.29:0/2215928377 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15fc0c9b60 con 0x7f15fc070dc0 2024-04-25T00:41:58.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f1601be8640 1 -- 172.21.15.29:0/2215928377 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15fc0cc040 con 0x7f15fc070dc0 2024-04-25T00:41:58.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15e3fff640 1 -- 172.21.15.29:0/2215928377 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f15f001c070 con 0x7f15fc070dc0 2024-04-25T00:41:58.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15e3fff640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f15e8068400 0x7f15e806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15e3fff640 1 -- 172.21.15.29:0/2215928377 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f15f00843f0 con 0x7f15fc070dc0 2024-04-25T00:41:58.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15fb7fe640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f15e8068400 0x7f15e806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.616+0000 7f15fb7fe640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f15e8068400 0x7f15e806a8c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009a30 tx=0x7f15ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.620+0000 7f15e3fff640 1 -- 172.21.15.29:0/2215928377 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f15fc0cc040 con 0x7f15fc070dc0 2024-04-25T00:41:58.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f1601be8640 1 -- 172.21.15.29:0/2215928377 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f15fc0c9d60 con 0x7f15e8068400 2024-04-25T00:41:58.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f15e3fff640 1 -- 172.21.15.29:0/2215928377 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f15fc0c9d60 con 0x7f15e8068400 2024-04-25T00:41:58.776 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T00:41:58.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f1601be8640 1 -- 172.21.15.29:0/2215928377 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f15e8068400 msgr2=0x7f15e806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f1601be8640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f15e8068400 0x7f15e806a8c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009a30 tx=0x7f15ec008040 comp rx=0 tx=0).stop 2024-04-25T00:41:58.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f1601be8640 1 -- 172.21.15.29:0/2215928377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15fc070dc0 msgr2=0x7f15fc0cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f1601be8640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15fc070dc0 0x7f15fc0cb6f0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f15f000dae0 tx=0x7f15f0001e20 comp rx=0 tx=0).stop 2024-04-25T00:41:58.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f1601be8640 1 -- 172.21.15.29:0/2215928377 shutdown_connections 2024-04-25T00:41:58.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f1601be8640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f15e8068400 0x7f15e806a8c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f1601be8640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15fc0cbc30 0x7f15fc0ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f1601be8640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15fc070dc0 0x7f15fc0cb6f0 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f1601be8640 1 --2- 172.21.15.29:0/2215928377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15fc1f6930 0x7f15fc071bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f1601be8640 1 -- 172.21.15.29:0/2215928377 >> 172.21.15.29:0/2215928377 conn(0x7f15fc0b1870 msgr2=0x7f15fc144a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:58.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f1601be8640 1 -- 172.21.15.29:0/2215928377 shutdown_connections 2024-04-25T00:41:58.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.776+0000 7f1601be8640 1 -- 172.21.15.29:0/2215928377 wait complete. 2024-04-25T00:41:58.794 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T00:41:58.794 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:41:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e0daa4640 1 Processor -- start 2024-04-25T00:41:58.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e0daa4640 1 -- start start 2024-04-25T00:41:58.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e0daa4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e08071590 0x7f5e0806e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e0daa4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e0806d640 con 0x7f5e080ce240 2024-04-25T00:41:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e0daa4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e0806efe0 con 0x7f5e080b5200 2024-04-25T00:41:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e0daa4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5e0806d900 con 0x7f5e08071590 2024-04-25T00:41:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e07fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5e080ce240 0x7f5e080ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41894/0 (socket says 172.21.15.29:41894) 2024-04-25T00:41:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e07fff640 1 -- 172.21.15.29:0/1327754779 learned_addr learned my addr 172.21.15.29:0/1327754779 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e077fe640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e08071590 0x7f5e0806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 35107742 0 0) 0x7f5e0806d640 con 0x7f5e080ce240 2024-04-25T00:41:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5df0003530 con 0x7f5e080ce240 2024-04-25T00:41:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1707239354 0 0) 0x7f5df0003530 con 0x7f5e080ce240 2024-04-25T00:41:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e08071590 msgr2=0x7f5e0806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e08071590 0x7f5e0806e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 >> v1:172.21.15.157:6789/0 conn(0x7f5e080b5200 legacy=0x7f5e08071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0806dbe0 con 0x7f5e080ce240 2024-04-25T00:41:58.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f5e00001a00 con 0x7f5e080ce240 2024-04-25T00:41:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 >> v1:172.21.15.29:6789/0 conn(0x7f5e080ce240 legacy=0x7f5e080ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5df00039f0 0x7f5df0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e08071590 0x7f5df0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df0004d50 0x7f5df00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0806d900 con 0x7f5df00039f0 2024-04-25T00:41:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5df000ba80 con 0x7f5df0004d50 2024-04-25T00:41:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5df000bc80 con 0x7f5e08071590 2024-04-25T00:41:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e077fe640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5df00039f0 0x7f5df0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e06ffd640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e08071590 0x7f5df0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e07fff640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df0004d50 0x7f5df00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e077fe640 1 -- 172.21.15.29:0/1327754779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e08071590 msgr2=0x7f5df0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:58.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e077fe640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e08071590 0x7f5df0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e077fe640 1 -- 172.21.15.29:0/1327754779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df0004d50 msgr2=0x7f5df00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:58.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e077fe640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df0004d50 0x7f5df00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e077fe640 1 -- 172.21.15.29:0/1327754779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df0003530 con 0x7f5df00039f0 2024-04-25T00:41:58.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e06ffd640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e08071590 0x7f5df0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:58.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.924+0000 7f5e07fff640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df0004d50 0x7f5df00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:58.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.928+0000 7f5e077fe640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5df00039f0 0x7f5df0003e10 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009a90 tx=0x7f5df801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a59c5338ddcb811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.928+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80b2070 con 0x7f5df00039f0 2024-04-25T00:41:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.928+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df80a5520 con 0x7f5df00039f0 2024-04-25T00:41:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.928+0000 7f5e067fc640 1 -- 172.21.15.29:0/1327754779 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8009690 con 0x7f5df00039f0 2024-04-25T00:41:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.928+0000 7f5e0daa4640 1 -- 172.21.15.29:0/1327754779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5df00039f0 msgr2=0x7f5df0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.928+0000 7f5e0daa4640 1 --2- 172.21.15.29:0/1327754779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5df00039f0 0x7f5df0003e10 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009a90 tx=0x7f5df801ca30 comp rx=0 tx=0).stop 2024-04-25T00:41:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.928+0000 7f5e07fff640 1 -- 172.21.15.29:0/1327754779 reap_dead start 2024-04-25T00:41:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.928+0000 7f5e0daa4640 1 -- 172.21.15.29:0/1327754779 shutdown_connections 2024-04-25T00:41:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.928+0000 7f5e0daa4640 1 -- 172.21.15.29:0/1327754779 >> 172.21.15.29:0/1327754779 conn(0x7f5e080b21d0 msgr2=0x7f5e080b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.928+0000 7f5e0daa4640 1 -- 172.21.15.29:0/1327754779 shutdown_connections 2024-04-25T00:41:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.928+0000 7f5e0daa4640 1 -- 172.21.15.29:0/1327754779 wait complete. 2024-04-25T00:41:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.928+0000 7f5e0daa4640 1 Processor -- start 2024-04-25T00:41:58.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.928+0000 7f5e0daa4640 1 -- start start 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e0daa4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e08149720 0x7f5e08149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e0daa4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e08153d40 0x7f5e0815bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e0daa4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e0815c2b0 0x7f5e0815c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e0daa4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5e080b55e0 con 0x7f5e08149720 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e0daa4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0806e510 con 0x7f5e0815c2b0 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e0daa4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5e080b5760 con 0x7f5e08153d40 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e08149720 0x7f5e08149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e077fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e08149720 0x7f5e08149b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34722/0 (socket says 172.21.15.29:34722) 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e077fe640 1 -- 172.21.15.29:0/56343352 learned_addr learned my addr 172.21.15.29:0/56343352 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e06ffd640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e08153d40 0x7f5e0815bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e07fff640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e0815c2b0 0x7f5e0815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e077fe640 1 -- 172.21.15.29:0/56343352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e08153d40 msgr2=0x7f5e0815bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e077fe640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e08153d40 0x7f5e0815bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e077fe640 1 -- 172.21.15.29:0/56343352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e0815c2b0 msgr2=0x7f5e0815c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e077fe640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e0815c2b0 0x7f5e0815c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e077fe640 1 -- 172.21.15.29:0/56343352 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0814a040 con 0x7f5e08149720 2024-04-25T00:41:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e07fff640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e0815c2b0 0x7f5e0815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e077fe640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e08149720 0x7f5e08149b00 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009a90 tx=0x7f5df80197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5deffff640 1 -- 172.21.15.29:0/56343352 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80b2040 con 0x7f5e08149720 2024-04-25T00:41:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5deffff640 1 -- 172.21.15.29:0/56343352 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df80a5d50 con 0x7f5e08149720 2024-04-25T00:41:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5deffff640 1 -- 172.21.15.29:0/56343352 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80ad640 con 0x7f5e08149720 2024-04-25T00:41:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e0daa4640 1 -- 172.21.15.29:0/56343352 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e08154590 con 0x7f5e08149720 2024-04-25T00:41:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e0daa4640 1 -- 172.21.15.29:0/56343352 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0815b570 con 0x7f5e08149720 2024-04-25T00:41:58.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e0daa4640 1 -- 172.21.15.29:0/56343352 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e080692e0 con 0x7f5e08149720 2024-04-25T00:41:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5deffff640 1 -- 172.21.15.29:0/56343352 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f5df80ad7e0 con 0x7f5e08149720 2024-04-25T00:41:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5deffff640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5df4068430 0x7f5df406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5deffff640 1 -- 172.21.15.29:0/56343352 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f5df811f620 con 0x7f5e08149720 2024-04-25T00:41:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5e06ffd640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5df4068430 0x7f5df406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5deffff640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] conn(0x7f5df406e2f0 0x7f5df4070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5deffff640 1 -- 172.21.15.29:0/56343352 --> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5df80acec0 con 0x7f5df406e2f0 2024-04-25T00:41:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.932+0000 7f5deffff640 1 -- 172.21.15.29:0/56343352 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=171) ==== 24+0+0 (secure 0 0 0) 0x7f5df8121dc0 con 0x7f5e08149720 2024-04-25T00:41:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.936+0000 7f5e06ffd640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5df4068430 0x7f5df406a8f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5e081f8910 tx=0x7f5dfc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.936+0000 7f5e07fff640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] conn(0x7f5df406e2f0 0x7f5df4070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:58.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.936+0000 7f5e07fff640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] conn(0x7f5df406e2f0 0x7f5df4070710 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:58.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.936+0000 7f5deffff640 1 -- 172.21.15.29:0/56343352 <== osd.6 v2:172.21.15.157:6816/2845420196 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5df80acec0 con 0x7f5df406e2f0 2024-04-25T00:41:58.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.948+0000 7f5e0daa4640 1 -- 172.21.15.29:0/56343352 --> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5e0815dd70 con 0x7f5df406e2f0 2024-04-25T00:41:58.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.952+0000 7f5deffff640 1 -- 172.21.15.29:0/56343352 <== osd.6 v2:172.21.15.157:6816/2845420196 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5e0815dd70 con 0x7f5df406e2f0 2024-04-25T00:41:58.952 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:41:58.952 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:41:58.952 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:41:58.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.952+0000 7f5e0daa4640 1 -- 172.21.15.29:0/56343352 >> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] conn(0x7f5df406e2f0 msgr2=0x7f5df4070710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.952+0000 7f5e0daa4640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.157:6816/2845420196,v1:172.21.15.157:6817/2845420196] conn(0x7f5df406e2f0 0x7f5df4070710 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.952+0000 7f5e0daa4640 1 -- 172.21.15.29:0/56343352 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5df4068430 msgr2=0x7f5df406a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.952+0000 7f5e0daa4640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f5df4068430 0x7f5df406a8f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5e081f8910 tx=0x7f5dfc009040 comp rx=0 tx=0).stop 2024-04-25T00:41:58.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.952+0000 7f5e0daa4640 1 -- 172.21.15.29:0/56343352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e08149720 msgr2=0x7f5e08149b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:58.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.952+0000 7f5e0daa4640 1 --2- 172.21.15.29:0/56343352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e08149720 0x7f5e08149b00 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009a90 tx=0x7f5df80197d0 comp rx=0 tx=0).stop 2024-04-25T00:41:58.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.952+0000 7f5e07fff640 1 -- 172.21.15.29:0/56343352 reap_dead start 2024-04-25T00:41:58.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.952+0000 7f5e0daa4640 1 -- 172.21.15.29:0/56343352 shutdown_connections 2024-04-25T00:41:58.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.952+0000 7f5e0daa4640 1 -- 172.21.15.29:0/56343352 >> 172.21.15.29:0/56343352 conn(0x7f5e080b21d0 msgr2=0x7f5e080b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:58.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.952+0000 7f5e0daa4640 1 -- 172.21.15.29:0/56343352 shutdown_connections 2024-04-25T00:41:58.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:58.952+0000 7f5e0daa4640 1 -- 172.21.15.29:0/56343352 wait complete. 2024-04-25T00:41:58.969 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T00:41:59.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9107797640 1 Processor -- start 2024-04-25T00:41:59.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9107797640 1 -- start start 2024-04-25T00:41:59.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9107797640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f910015e8d0 0x7f91000cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9107797640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91000b3720 con 0x7f910015f3c0 2024-04-25T00:41:59.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9107797640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91000b39c0 con 0x7f91000b4780 2024-04-25T00:41:59.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9107797640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f91000b42f0 con 0x7f910015e8d0 2024-04-25T00:41:59.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9105d34640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f910015f3c0 0x7f910015c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41910/0 (socket says 172.21.15.29:41910) 2024-04-25T00:41:59.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9105d34640 1 -- 172.21.15.29:0/1586086134 learned_addr learned my addr 172.21.15.29:0/1586086134 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:59.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9105533640 1 --2- 172.21.15.29:0/1586086134 >> v2:172.21.15.157:3300/0 conn(0x7f910015e8d0 0x7f91000cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2119780340 0 0) 0x7f91000b3720 con 0x7f910015f3c0 2024-04-25T00:41:59.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90e8003530 con 0x7f910015f3c0 2024-04-25T00:41:59.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2071399417 0 0) 0x7f90e8003530 con 0x7f910015f3c0 2024-04-25T00:41:59.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 >> v2:172.21.15.157:3300/0 conn(0x7f910015e8d0 msgr2=0x7f91000cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:59.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 --2- 172.21.15.29:0/1586086134 >> v2:172.21.15.157:3300/0 conn(0x7f910015e8d0 0x7f91000cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 >> v1:172.21.15.157:6789/0 conn(0x7f91000b4780 legacy=0x7f910015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91000b3ca0 con 0x7f910015f3c0 2024-04-25T00:41:59.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f90f8001680 con 0x7f910015f3c0 2024-04-25T00:41:59.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 >> v1:172.21.15.29:6789/0 conn(0x7f910015f3c0 legacy=0x7f910015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 --2- 172.21.15.29:0/1586086134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e8003a10 0x7f90e8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 --2- 172.21.15.29:0/1586086134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e8004810 0x7f90e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 --2- 172.21.15.29:0/1586086134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e80051d0 0x7f90e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f91000b42f0 con 0x7f90e80051d0 2024-04-25T00:41:59.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90e800c0d0 con 0x7f90e8004810 2024-04-25T00:41:59.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f90e800c2d0 con 0x7f90e8003a10 2024-04-25T00:41:59.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9105d34640 1 --2- 172.21.15.29:0/1586086134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e80051d0 0x7f90e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9104d32640 1 --2- 172.21.15.29:0/1586086134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e8004810 0x7f90e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9105533640 1 --2- 172.21.15.29:0/1586086134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e8003a10 0x7f90e8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9105d34640 1 -- 172.21.15.29:0/1586086134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e8003a10 msgr2=0x7f90e8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:59.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9105d34640 1 --2- 172.21.15.29:0/1586086134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e8003a10 0x7f90e8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9105d34640 1 -- 172.21.15.29:0/1586086134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e8004810 msgr2=0x7f90e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:59.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9105d34640 1 --2- 172.21.15.29:0/1586086134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e8004810 0x7f90e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9105d34640 1 -- 172.21.15.29:0/1586086134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e8003530 con 0x7f90e80051d0 2024-04-25T00:41:59.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.084+0000 7f9105533640 1 --2- 172.21.15.29:0/1586086134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90e8003a10 0x7f90e8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:59.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f9105d34640 1 -- 172.21.15.29:0/1586086134 reap_dead start 2024-04-25T00:41:59.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f9105d34640 1 --2- 172.21.15.29:0/1586086134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e80051d0 0x7f90e800b9d0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f90f800c190 tx=0x7f90f801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9875f51c6e88d7ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:59.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8007c70 con 0x7f90e80051d0 2024-04-25T00:41:59.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f80a9070 con 0x7f90e80051d0 2024-04-25T00:41:59.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f90fffff640 1 -- 172.21.15.29:0/1586086134 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8005d40 con 0x7f90e80051d0 2024-04-25T00:41:59.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f9107797640 1 -- 172.21.15.29:0/1586086134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e80051d0 msgr2=0x7f90e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f9107797640 1 --2- 172.21.15.29:0/1586086134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e80051d0 0x7f90e800b9d0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f90f800c190 tx=0x7f90f801cbe0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f9107797640 1 -- 172.21.15.29:0/1586086134 shutdown_connections 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f9107797640 1 --2- 172.21.15.29:0/1586086134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e80051d0 0x7f90e800b9d0 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f9107797640 1 -- 172.21.15.29:0/1586086134 >> 172.21.15.29:0/1586086134 conn(0x7f91000b1870 msgr2=0x7f91000b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f9107797640 1 -- 172.21.15.29:0/1586086134 shutdown_connections 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f9107797640 1 -- 172.21.15.29:0/1586086134 wait complete. 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f9107797640 1 Processor -- start 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.088+0000 7f9107797640 1 -- start start 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9107797640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e80051d0 0x7f910006df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9107797640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91000ad8a0 0x7f910015b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9107797640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f910015bc50 0x7f910013cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9107797640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f910006cf10 con 0x7f90e80051d0 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9107797640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91000b3580 con 0x7f910015bc50 2024-04-25T00:41:59.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9107797640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9100144a20 con 0x7f91000ad8a0 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9105533640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e80051d0 0x7f910006df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9105533640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e80051d0 0x7f910006df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34744/0 (socket says 172.21.15.29:34744) 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9105533640 1 -- 172.21.15.29:0/1970142483 learned_addr learned my addr 172.21.15.29:0/1970142483 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9105d34640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f910015bc50 0x7f910013cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9104d32640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91000ad8a0 0x7f910015b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9105533640 1 -- 172.21.15.29:0/1970142483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91000ad8a0 msgr2=0x7f910015b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9105533640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91000ad8a0 0x7f910015b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9105533640 1 -- 172.21.15.29:0/1970142483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f910015bc50 msgr2=0x7f910013cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9105533640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f910015bc50 0x7f910013cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9105533640 1 -- 172.21.15.29:0/1970142483 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91000ae080 con 0x7f90e80051d0 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9104d32640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91000ad8a0 0x7f910015b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9105533640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e80051d0 0x7f910006df70 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f90f000f380 tx=0x7f90f000f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f90fdffb640 1 -- 172.21.15.29:0/1970142483 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0003ba0 con 0x7f90e80051d0 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f90fdffb640 1 -- 172.21.15.29:0/1970142483 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f0003d40 con 0x7f90e80051d0 2024-04-25T00:41:59.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9107797640 1 -- 172.21.15.29:0/1970142483 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91000ae360 con 0x7f90e80051d0 2024-04-25T00:41:59.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f90fdffb640 1 -- 172.21.15.29:0/1970142483 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0004220 con 0x7f90e80051d0 2024-04-25T00:41:59.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9107797640 1 -- 172.21.15.29:0/1970142483 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91000add00 con 0x7f90e80051d0 2024-04-25T00:41:59.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9107797640 1 -- 172.21.15.29:0/1970142483 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9100069270 con 0x7f90e80051d0 2024-04-25T00:41:59.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f90fdffb640 1 -- 172.21.15.29:0/1970142483 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f90f00043c0 con 0x7f90e80051d0 2024-04-25T00:41:59.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f90fdffb640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90ec068430 0x7f90ec06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f90fdffb640 1 -- 172.21.15.29:0/1970142483 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f90f0085720 con 0x7f90e80051d0 2024-04-25T00:41:59.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.092+0000 7f9104d32640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90ec068430 0x7f90ec06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.096+0000 7f9104d32640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90ec068430 0x7f90ec06a8f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f90f4007fd0 tx=0x7f90f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:59.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.096+0000 7f90fdffb640 1 -- 172.21.15.29:0/1970142483 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f90f0025070 con 0x7f90e80051d0 2024-04-25T00:41:59.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.260+0000 7f9107797640 1 -- 172.21.15.29:0/1970142483 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f910015c030 con 0x7f90ec068430 2024-04-25T00:41:59.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.260+0000 7f90fdffb640 1 -- 172.21.15.29:0/1970142483 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f910015c030 con 0x7f90ec068430 2024-04-25T00:41:59.261 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T00:41:59.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.260+0000 7f9107797640 1 -- 172.21.15.29:0/1970142483 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90ec068430 msgr2=0x7f90ec06a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.260+0000 7f9107797640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90ec068430 0x7f90ec06a8f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f90f4007fd0 tx=0x7f90f4009040 comp rx=0 tx=0).stop 2024-04-25T00:41:59.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.260+0000 7f9107797640 1 -- 172.21.15.29:0/1970142483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e80051d0 msgr2=0x7f910006df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.260+0000 7f9107797640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e80051d0 0x7f910006df70 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f90f000f380 tx=0x7f90f000f840 comp rx=0 tx=0).stop 2024-04-25T00:41:59.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.260+0000 7f9107797640 1 -- 172.21.15.29:0/1970142483 shutdown_connections 2024-04-25T00:41:59.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.260+0000 7f9107797640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f90ec068430 0x7f90ec06a8f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.260+0000 7f9107797640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f910015bc50 0x7f910013cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.260+0000 7f9107797640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91000ad8a0 0x7f910015b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.260+0000 7f9107797640 1 --2- 172.21.15.29:0/1970142483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90e80051d0 0x7f910006df70 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.264+0000 7f9107797640 1 -- 172.21.15.29:0/1970142483 >> 172.21.15.29:0/1970142483 conn(0x7f91000b1870 msgr2=0x7f910013f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:59.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.264+0000 7f9107797640 1 -- 172.21.15.29:0/1970142483 shutdown_connections 2024-04-25T00:41:59.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.264+0000 7f9107797640 1 -- 172.21.15.29:0/1970142483 wait complete. 2024-04-25T00:41:59.280 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T00:41:59.280 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:41:59.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f495e0d5640 1 Processor -- start 2024-04-25T00:41:59.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f495e0d5640 1 -- start start 2024-04-25T00:41:59.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f495e0d5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f49580b3b50 0x7f49580b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f495e0d5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f495806cd40 0x7f495806d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f495e0d5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49580cf670 con 0x7f49581456d0 2024-04-25T00:41:59.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f495e0d5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f49580b4380 con 0x7f49580b3b50 2024-04-25T00:41:59.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f495e0d5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f49580b40a0 con 0x7f495806cd40 2024-04-25T00:41:59.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49577fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f49581456d0 0x7f49580cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41914/0 (socket says 172.21.15.29:41914) 2024-04-25T00:41:59.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49577fe640 1 -- 172.21.15.29:0/3257227461 learned_addr learned my addr 172.21.15.29:0/3257227461 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:59.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 991370936 0 0) 0x7f49580cf670 con 0x7f49581456d0 2024-04-25T00:41:59.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4940003530 con 0x7f49581456d0 2024-04-25T00:41:59.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f4957fff640 1 --2- 172.21.15.29:0/3257227461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f495806cd40 0x7f495806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f4956ffd640 1 --2- 172.21.15.29:0/3257227461 >> v2:172.21.15.157:3300/0 conn(0x7f49580b3b50 0x7f49580b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3674629645 0 0) 0x7f4940003530 con 0x7f49581456d0 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f495806cd40 msgr2=0x7f495806d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 --2- 172.21.15.29:0/3257227461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f495806cd40 0x7f495806d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 >> v2:172.21.15.157:3300/0 conn(0x7f49580b3b50 msgr2=0x7f49580b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 --2- 172.21.15.29:0/3257227461 >> v2:172.21.15.157:3300/0 conn(0x7f49580b3b50 0x7f49580b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49580ce480 con 0x7f49581456d0 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7f4950001a00 con 0x7f49581456d0 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 >> v1:172.21.15.29:6789/0 conn(0x7f49581456d0 legacy=0x7f49580cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 --2- 172.21.15.29:0/3257227461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f495806cd40 0x7f49400039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 --2- 172.21.15.29:0/3257227461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4940003ff0 0x7f4940004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 --2- 172.21.15.29:0/3257227461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4940004d50 0x7f49400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f49580b4380 con 0x7f4940003ff0 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f49580b40a0 con 0x7f4940004d50 2024-04-25T00:41:59.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f494000bad0 con 0x7f495806cd40 2024-04-25T00:41:59.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f4956ffd640 1 --2- 172.21.15.29:0/3257227461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4940003ff0 0x7f4940004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f49577fe640 1 --2- 172.21.15.29:0/3257227461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f495806cd40 0x7f49400039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f4957fff640 1 --2- 172.21.15.29:0/3257227461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4940004d50 0x7f49400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f4956ffd640 1 -- 172.21.15.29:0/3257227461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f495806cd40 msgr2=0x7f49400039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:59.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f4956ffd640 1 --2- 172.21.15.29:0/3257227461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f495806cd40 0x7f49400039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f4956ffd640 1 -- 172.21.15.29:0/3257227461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4940004d50 msgr2=0x7f49400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:59.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f4956ffd640 1 --2- 172.21.15.29:0/3257227461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4940004d50 0x7f49400051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.408+0000 7f4956ffd640 1 -- 172.21.15.29:0/3257227461 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4940003530 con 0x7f4940003ff0 2024-04-25T00:41:59.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.412+0000 7f4956ffd640 1 --2- 172.21.15.29:0/3257227461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4940003ff0 0x7f4940004810 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f494c00ba30 tx=0x7f494c01c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156e1255853364d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:59.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.412+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c00b600 con 0x7f4940003ff0 2024-04-25T00:41:59.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.412+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494c0b1070 con 0x7f4940003ff0 2024-04-25T00:41:59.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.412+0000 7f49567fc640 1 -- 172.21.15.29:0/3257227461 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c0ac8c0 con 0x7f4940003ff0 2024-04-25T00:41:59.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.412+0000 7f495e0d5640 1 -- 172.21.15.29:0/3257227461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4940003ff0 msgr2=0x7f4940004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.412+0000 7f495e0d5640 1 --2- 172.21.15.29:0/3257227461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4940003ff0 0x7f4940004810 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f494c00ba30 tx=0x7f494c01c6e0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.412+0000 7f4957fff640 1 -- 172.21.15.29:0/3257227461 reap_dead start 2024-04-25T00:41:59.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.412+0000 7f495e0d5640 1 -- 172.21.15.29:0/3257227461 shutdown_connections 2024-04-25T00:41:59.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.412+0000 7f495e0d5640 1 -- 172.21.15.29:0/3257227461 >> 172.21.15.29:0/3257227461 conn(0x7f49580b1870 msgr2=0x7f49580b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:59.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.412+0000 7f495e0d5640 1 -- 172.21.15.29:0/3257227461 shutdown_connections 2024-04-25T00:41:59.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.412+0000 7f495e0d5640 1 -- 172.21.15.29:0/3257227461 wait complete. 2024-04-25T00:41:59.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.412+0000 7f495e0d5640 1 Processor -- start 2024-04-25T00:41:59.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.412+0000 7f495e0d5640 1 -- start start 2024-04-25T00:41:59.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f495e0d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f495813bb10 0x7f495813c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f495e0d5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f495813c710 0x7f4958140540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f495e0d5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4958140a80 0x7f4958158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f495e0d5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4958145ab0 con 0x7f495813bb10 2024-04-25T00:41:59.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f495e0d5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f49580cea40 con 0x7f495813c710 2024-04-25T00:41:59.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f495e0d5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f49580b5000 con 0x7f4958140a80 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f49577fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f495813bb10 0x7f495813c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f49577fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f495813bb10 0x7f495813c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34770/0 (socket says 172.21.15.29:34770) 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f49577fe640 1 -- 172.21.15.29:0/3431213064 learned_addr learned my addr 172.21.15.29:0/3431213064 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f4956ffd640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f495813c710 0x7f4958140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f4957fff640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4958140a80 0x7f4958158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f49577fe640 1 -- 172.21.15.29:0/3431213064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4958140a80 msgr2=0x7f4958158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f49577fe640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4958140a80 0x7f4958158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f49577fe640 1 -- 172.21.15.29:0/3431213064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f495813c710 msgr2=0x7f4958140540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f49577fe640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f495813c710 0x7f4958140540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f49577fe640 1 -- 172.21.15.29:0/3431213064 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495813cc70 con 0x7f495813bb10 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f4956ffd640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f495813c710 0x7f4958140540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f49577fe640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f495813bb10 0x7f495813c1d0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f4950011950 tx=0x7f4950011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f493ffff640 1 -- 172.21.15.29:0/3431213064 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950010040 con 0x7f495813bb10 2024-04-25T00:41:59.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f493ffff640 1 -- 172.21.15.29:0/3431213064 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f495001a070 con 0x7f495813bb10 2024-04-25T00:41:59.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f495e0d5640 1 -- 172.21.15.29:0/3431213064 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495813cf50 con 0x7f495813bb10 2024-04-25T00:41:59.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f493ffff640 1 -- 172.21.15.29:0/3431213064 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495000fde0 con 0x7f495813bb10 2024-04-25T00:41:59.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f495e0d5640 1 -- 172.21.15.29:0/3431213064 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4958140020 con 0x7f495813bb10 2024-04-25T00:41:59.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f495e0d5640 1 -- 172.21.15.29:0/3431213064 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49580692e0 con 0x7f495813bb10 2024-04-25T00:41:59.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f493ffff640 1 -- 172.21.15.29:0/3431213064 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f495001f460 con 0x7f495813bb10 2024-04-25T00:41:59.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f493ffff640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4944068430 0x7f494406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f493ffff640 1 -- 172.21.15.29:0/3431213064 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f4950088330 con 0x7f495813bb10 2024-04-25T00:41:59.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f4956ffd640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4944068430 0x7f494406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f493ffff640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] conn(0x7f494406e2f0 0x7f4944070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f493ffff640 1 -- 172.21.15.29:0/3431213064 --> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4944070dc0 con 0x7f494406e2f0 2024-04-25T00:41:59.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f493ffff640 1 -- 172.21.15.29:0/3431213064 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=171) ==== 24+0+0 (secure 0 0 0) 0x7f495008ae50 con 0x7f495813bb10 2024-04-25T00:41:59.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.416+0000 7f4956ffd640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4944068430 0x7f494406a8f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f494c0b0000 tx=0x7f494c0046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:59.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.420+0000 7f4957fff640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] conn(0x7f494406e2f0 0x7f4944070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.420+0000 7f4957fff640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] conn(0x7f494406e2f0 0x7f4944070710 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:59.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.420+0000 7f493ffff640 1 -- 172.21.15.29:0/3431213064 <== osd.7 v2:172.21.15.157:6824/739472638 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4944070dc0 con 0x7f494406e2f0 2024-04-25T00:41:59.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.432+0000 7f495e0d5640 1 -- 172.21.15.29:0/3431213064 --> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f49580b4de0 con 0x7f494406e2f0 2024-04-25T00:41:59.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.432+0000 7f493ffff640 1 -- 172.21.15.29:0/3431213064 <== osd.7 v2:172.21.15.157:6824/739472638 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f49580b4de0 con 0x7f494406e2f0 2024-04-25T00:41:59.435 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-04-25T00:41:59.435 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:41:59.435 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-04-25T00:41:59.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.436+0000 7f495e0d5640 1 -- 172.21.15.29:0/3431213064 >> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] conn(0x7f494406e2f0 msgr2=0x7f4944070710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.436+0000 7f495e0d5640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.157:6824/739472638,v1:172.21.15.157:6825/739472638] conn(0x7f494406e2f0 0x7f4944070710 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.436+0000 7f495e0d5640 1 -- 172.21.15.29:0/3431213064 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4944068430 msgr2=0x7f494406a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.436+0000 7f495e0d5640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f4944068430 0x7f494406a8f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f494c0b0000 tx=0x7f494c0046d0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.436+0000 7f495e0d5640 1 -- 172.21.15.29:0/3431213064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f495813bb10 msgr2=0x7f495813c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.436+0000 7f495e0d5640 1 --2- 172.21.15.29:0/3431213064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f495813bb10 0x7f495813c1d0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f4950011950 tx=0x7f4950011e10 comp rx=0 tx=0).stop 2024-04-25T00:41:59.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.436+0000 7f4957fff640 1 -- 172.21.15.29:0/3431213064 reap_dead start 2024-04-25T00:41:59.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.436+0000 7f495e0d5640 1 -- 172.21.15.29:0/3431213064 shutdown_connections 2024-04-25T00:41:59.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.436+0000 7f495e0d5640 1 -- 172.21.15.29:0/3431213064 >> 172.21.15.29:0/3431213064 conn(0x7f49580b1870 msgr2=0x7f49580b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:59.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.436+0000 7f495e0d5640 1 -- 172.21.15.29:0/3431213064 shutdown_connections 2024-04-25T00:41:59.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.436+0000 7f495e0d5640 1 -- 172.21.15.29:0/3431213064 wait complete. 2024-04-25T00:41:59.450 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T00:41:59.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde9972640 1 Processor -- start 2024-04-25T00:41:59.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde9972640 1 -- start start 2024-04-25T00:41:59.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde9972640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efde415e170 0x7efde40d49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde9972640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde40b1320 con 0x7efde415e550 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde9972640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde40cfc70 con 0x7efde40cf770 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde9972640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efde40d1c60 con 0x7efde415e170 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde27fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7efde415e550 0x7efde40b0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41922/0 (socket says 172.21.15.29:41922) 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde27fc640 1 -- 172.21.15.29:0/3627612502 learned_addr learned my addr 172.21.15.29:0/3627612502 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2847083514 0 0) 0x7efde40b1320 con 0x7efde415e550 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdcc003530 con 0x7efde415e550 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde2ffd640 1 --2- 172.21.15.29:0/3627612502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efde415e170 0x7efde40d49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 936917511 0 0) 0x7efdcc003530 con 0x7efde415e550 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efde415e170 msgr2=0x7efde40d49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 --2- 172.21.15.29:0/3627612502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efde415e170 0x7efde40d49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 >> v1:172.21.15.157:6789/0 conn(0x7efde40cf770 legacy=0x7efde40d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde40cff50 con 0x7efde415e550 2024-04-25T00:41:59.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7efdd8001680 con 0x7efde415e550 2024-04-25T00:41:59.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 >> v1:172.21.15.29:6789/0 conn(0x7efde415e550 legacy=0x7efde40b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 --2- 172.21.15.29:0/3627612502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdcc0039f0 0x7efdcc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 --2- 172.21.15.29:0/3627612502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efde415e170 0x7efdcc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 --2- 172.21.15.29:0/3627612502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdcc004d50 0x7efdcc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efde40d1c60 con 0x7efdcc0039f0 2024-04-25T00:41:59.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc00bb00 con 0x7efdcc004d50 2024-04-25T00:41:59.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.556+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efdcc00bd00 con 0x7efde415e170 2024-04-25T00:41:59.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde2ffd640 1 --2- 172.21.15.29:0/3627612502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdcc0039f0 0x7efdcc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde2ffd640 1 -- 172.21.15.29:0/3627612502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efde415e170 msgr2=0x7efdcc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:59.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde2ffd640 1 --2- 172.21.15.29:0/3627612502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efde415e170 0x7efdcc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde2ffd640 1 -- 172.21.15.29:0/3627612502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdcc004d50 msgr2=0x7efdcc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:59.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde2ffd640 1 --2- 172.21.15.29:0/3627612502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdcc004d50 0x7efdcc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde2ffd640 1 -- 172.21.15.29:0/3627612502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdcc003530 con 0x7efdcc0039f0 2024-04-25T00:41:59.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde2ffd640 1 --2- 172.21.15.29:0/3627612502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdcc0039f0 0x7efdcc003e10 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009ae0 tx=0x7efdd401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637118371e16d325 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:59.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd40096b0 con 0x7efdcc0039f0 2024-04-25T00:41:59.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd40a64e0 con 0x7efdcc0039f0 2024-04-25T00:41:59.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde1ffb640 1 -- 172.21.15.29:0/3627612502 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd40a68d0 con 0x7efdcc0039f0 2024-04-25T00:41:59.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde9972640 1 -- 172.21.15.29:0/3627612502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdcc0039f0 msgr2=0x7efdcc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde9972640 1 --2- 172.21.15.29:0/3627612502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdcc0039f0 0x7efdcc003e10 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009ae0 tx=0x7efdd401c9f0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde37fe640 1 -- 172.21.15.29:0/3627612502 reap_dead start 2024-04-25T00:41:59.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde9972640 1 -- 172.21.15.29:0/3627612502 shutdown_connections 2024-04-25T00:41:59.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde9972640 1 -- 172.21.15.29:0/3627612502 >> 172.21.15.29:0/3627612502 conn(0x7efde409d320 msgr2=0x7efde40d4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:59.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde9972640 1 -- 172.21.15.29:0/3627612502 shutdown_connections 2024-04-25T00:41:59.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.560+0000 7efde9972640 1 -- 172.21.15.29:0/3627612502 wait complete. 2024-04-25T00:41:59.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde9972640 1 Processor -- start 2024-04-25T00:41:59.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde9972640 1 -- start start 2024-04-25T00:41:59.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde9972640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdcc0039f0 0x7efde40cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde9972640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efde41f7b70 0x7efde415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde9972640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efde415c580 0x7efde415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde9972640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efde40d2ea0 con 0x7efdcc0039f0 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde9972640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efde40d4fd0 con 0x7efde41f7b70 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde9972640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efde40d3020 con 0x7efde415c580 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdcc0039f0 0x7efde40cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde2ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdcc0039f0 0x7efde40cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34784/0 (socket says 172.21.15.29:34784) 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde2ffd640 1 -- 172.21.15.29:0/226827224 learned_addr learned my addr 172.21.15.29:0/226827224 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde37fe640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efde415c580 0x7efde415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde27fc640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efde41f7b70 0x7efde415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde2ffd640 1 -- 172.21.15.29:0/226827224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efde415c580 msgr2=0x7efde415ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde2ffd640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efde415c580 0x7efde415ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde2ffd640 1 -- 172.21.15.29:0/226827224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efde41f7b70 msgr2=0x7efde415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde2ffd640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efde41f7b70 0x7efde415a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde2ffd640 1 -- 172.21.15.29:0/226827224 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde41f7ff0 con 0x7efdcc0039f0 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde37fe640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efde415c580 0x7efde415ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde2ffd640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdcc0039f0 0x7efde40cc1b0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7efdd40b0040 tx=0x7efdcc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efdcbfff640 1 -- 172.21.15.29:0/226827224 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd40be020 con 0x7efdcc0039f0 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efdcbfff640 1 -- 172.21.15.29:0/226827224 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd40a68b0 con 0x7efdcc0039f0 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efdcbfff640 1 -- 172.21.15.29:0/226827224 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd40a6ca0 con 0x7efdcc0039f0 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde9972640 1 -- 172.21.15.29:0/226827224 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde40cc790 con 0x7efdcc0039f0 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde9972640 1 -- 172.21.15.29:0/226827224 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde40ccd50 con 0x7efdcc0039f0 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efde9972640 1 -- 172.21.15.29:0/226827224 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde4069270 con 0x7efdcc0039f0 2024-04-25T00:41:59.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.564+0000 7efdcbfff640 1 -- 172.21.15.29:0/226827224 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7efdd40c1020 con 0x7efdcc0039f0 2024-04-25T00:41:59.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.568+0000 7efdcbfff640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efdd0068430 0x7efdd006a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.568+0000 7efdcbfff640 1 -- 172.21.15.29:0/226827224 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7efdd411f1f0 con 0x7efdcc0039f0 2024-04-25T00:41:59.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.568+0000 7efde27fc640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efdd0068430 0x7efdd006a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.568+0000 7efde27fc640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efdd0068430 0x7efdd006a8f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7efdd8001870 tx=0x7efdd8001700 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:59.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.572+0000 7efdcbfff640 1 -- 172.21.15.29:0/226827224 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efdd40c8030 con 0x7efdcc0039f0 2024-04-25T00:41:59.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.728+0000 7efde9972640 1 -- 172.21.15.29:0/226827224 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7efde40d17f0 con 0x7efdd0068430 2024-04-25T00:41:59.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.728+0000 7efdcbfff640 1 -- 172.21.15.29:0/226827224 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efde40d17f0 con 0x7efdd0068430 2024-04-25T00:41:59.730 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T00:41:59.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.732+0000 7efde9972640 1 -- 172.21.15.29:0/226827224 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efdd0068430 msgr2=0x7efdd006a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.732+0000 7efde9972640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efdd0068430 0x7efdd006a8f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7efdd8001870 tx=0x7efdd8001700 comp rx=0 tx=0).stop 2024-04-25T00:41:59.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.732+0000 7efde9972640 1 -- 172.21.15.29:0/226827224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdcc0039f0 msgr2=0x7efde40cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.732+0000 7efde9972640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdcc0039f0 0x7efde40cc1b0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7efdd40b0040 tx=0x7efdcc005840 comp rx=0 tx=0).stop 2024-04-25T00:41:59.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.732+0000 7efde9972640 1 -- 172.21.15.29:0/226827224 shutdown_connections 2024-04-25T00:41:59.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.732+0000 7efde9972640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efdd0068430 0x7efdd006a8f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.732+0000 7efde9972640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efde415c580 0x7efde415ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.732+0000 7efde9972640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efde41f7b70 0x7efde415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.732+0000 7efde9972640 1 --2- 172.21.15.29:0/226827224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdcc0039f0 0x7efde40cc1b0 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.732+0000 7efde9972640 1 -- 172.21.15.29:0/226827224 >> 172.21.15.29:0/226827224 conn(0x7efde409d320 msgr2=0x7efde40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:59.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.732+0000 7efde9972640 1 -- 172.21.15.29:0/226827224 shutdown_connections 2024-04-25T00:41:59.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.732+0000 7efde9972640 1 -- 172.21.15.29:0/226827224 wait complete. 2024-04-25T00:41:59.749 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:41:59.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f0081c64640 1 Processor -- start 2024-04-25T00:41:59.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f0081c64640 1 -- start start 2024-04-25T00:41:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f0081c64640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f007c15ad20 0x7f007c0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f0081c64640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f007c0d0390 0x7f007c15b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f0081c64640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f007c15bc30 con 0x7f007c15b640 2024-04-25T00:41:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f0081c64640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f007c0cb6a0 con 0x7f007c0d0390 2024-04-25T00:41:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f0081c64640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f007c15bf10 con 0x7f007c15ad20 2024-04-25T00:41:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f007bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f007c15b640 0x7f007c071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41928/0 (socket says 172.21.15.29:41928) 2024-04-25T00:41:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f007bfff640 1 -- 172.21.15.29:0/3401052583 learned_addr learned my addr 172.21.15.29:0/3401052583 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:41:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f007affd640 1 --2- 172.21.15.29:0/3401052583 >> v2:172.21.15.29:3300/0 conn(0x7f007c0d0390 0x7f007c15b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f007b7fe640 1 --2- 172.21.15.29:0/3401052583 >> v2:172.21.15.157:3300/0 conn(0x7f007c15ad20 0x7f007c0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f007affd640 1 -- 172.21.15.29:0/3401052583 >> v2:172.21.15.157:3300/0 conn(0x7f007c15ad20 msgr2=0x7f007c0cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f007affd640 1 --2- 172.21.15.29:0/3401052583 >> v2:172.21.15.157:3300/0 conn(0x7f007c15ad20 0x7f007c0cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f007affd640 1 -- 172.21.15.29:0/3401052583 >> v1:172.21.15.29:6789/0 conn(0x7f007c15b640 legacy=0x7f007c071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f007affd640 1 -- 172.21.15.29:0/3401052583 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007c15c1f0 con 0x7f007c0d0390 2024-04-25T00:41:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.872+0000 7f007a7fc640 1 -- 172.21.15.29:0/3401052583 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2384006387 0 0) 0x7f007c15bc30 con 0x7f007c15b640 2024-04-25T00:41:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007affd640 1 --2- 172.21.15.29:0/3401052583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f007c0d0390 0x7f007c15b100 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f007c0cd620 tx=0x7f007001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445019f8bc75ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007a7fc640 1 -- 172.21.15.29:0/3401052583 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00700b1070 con 0x7f007c0d0390 2024-04-25T00:41:59.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007a7fc640 1 -- 172.21.15.29:0/3401052583 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00700ac5f0 con 0x7f007c0d0390 2024-04-25T00:41:59.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007a7fc640 1 -- 172.21.15.29:0/3401052583 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00700ac9c0 con 0x7f007c0d0390 2024-04-25T00:41:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 -- 172.21.15.29:0/3401052583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f007c0d0390 msgr2=0x7f007c15b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 --2- 172.21.15.29:0/3401052583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f007c0d0390 0x7f007c15b100 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f007c0cd620 tx=0x7f007001c480 comp rx=0 tx=0).stop 2024-04-25T00:41:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 -- 172.21.15.29:0/3401052583 shutdown_connections 2024-04-25T00:41:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 --2- 172.21.15.29:0/3401052583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f007c0d0390 0x7f007c15b100 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 --2- 172.21.15.29:0/3401052583 >> v2:172.21.15.157:3300/0 conn(0x7f007c15ad20 0x7f007c0cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 -- 172.21.15.29:0/3401052583 >> 172.21.15.29:0/3401052583 conn(0x7f007c0d0c40 msgr2=0x7f007c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 -- 172.21.15.29:0/3401052583 shutdown_connections 2024-04-25T00:41:59.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 -- 172.21.15.29:0/3401052583 wait complete. 2024-04-25T00:41:59.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 Processor -- start 2024-04-25T00:41:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 -- start start 2024-04-25T00:41:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f007c0d0390 0x7f007c0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15ad20 0x7f007c162ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f007c15b640 0x7f007c0c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f007c06f530 con 0x7f007c0d0390 2024-04-25T00:41:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f007c0726a0 con 0x7f007c15ad20 2024-04-25T00:41:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0081c64640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f007c06f6b0 con 0x7f007c15b640 2024-04-25T00:41:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f007c0d0390 0x7f007c0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f007c0d0390 0x7f007c0b1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34802/0 (socket says 172.21.15.29:34802) 2024-04-25T00:41:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007b7fe640 1 -- 172.21.15.29:0/2724629207 learned_addr learned my addr 172.21.15.29:0/2724629207 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:41:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007b7fe640 1 -- 172.21.15.29:0/2724629207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f007c15b640 msgr2=0x7f007c0c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007b7fe640 1 --2- 172.21.15.29:0/2724629207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f007c15b640 0x7f007c0c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007affd640 1 --2- 172.21.15.29:0/2724629207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15ad20 0x7f007c162ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007b7fe640 1 -- 172.21.15.29:0/2724629207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15ad20 msgr2=0x7f007c162ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007b7fe640 1 --2- 172.21.15.29:0/2724629207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15ad20 0x7f007c162ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007b7fe640 1 -- 172.21.15.29:0/2724629207 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007c151c50 con 0x7f007c0d0390 2024-04-25T00:41:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f007b7fe640 1 --2- 172.21.15.29:0/2724629207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f007c0d0390 0x7f007c0b1060 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f006c00d800 tx=0x7f006c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.876+0000 7f0063fff640 1 -- 172.21.15.29:0/2724629207 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c00b6c0 con 0x7f007c0d0390 2024-04-25T00:41:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.880+0000 7f0063fff640 1 -- 172.21.15.29:0/2724629207 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f006c019070 con 0x7f007c0d0390 2024-04-25T00:41:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.880+0000 7f0081c64640 1 -- 172.21.15.29:0/2724629207 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007c151f30 con 0x7f007c0d0390 2024-04-25T00:41:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.880+0000 7f0063fff640 1 -- 172.21.15.29:0/2724629207 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c00bae0 con 0x7f007c0d0390 2024-04-25T00:41:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.880+0000 7f0081c64640 1 -- 172.21.15.29:0/2724629207 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f007c1524f0 con 0x7f007c0d0390 2024-04-25T00:41:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.880+0000 7f0081c64640 1 -- 172.21.15.29:0/2724629207 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f007c069270 con 0x7f007c0d0390 2024-04-25T00:41:59.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.880+0000 7f0063fff640 1 -- 172.21.15.29:0/2724629207 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f006c00bc80 con 0x7f007c0d0390 2024-04-25T00:41:59.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.880+0000 7f0063fff640 1 --2- 172.21.15.29:0/2724629207 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0068068430 0x7f006806a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:59.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.880+0000 7f0063fff640 1 -- 172.21.15.29:0/2724629207 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7f006c084ad0 con 0x7f007c0d0390 2024-04-25T00:41:59.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.880+0000 7f007affd640 1 --2- 172.21.15.29:0/2724629207 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0068068430 0x7f006806a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:59.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.880+0000 7f007affd640 1 --2- 172.21.15.29:0/2724629207 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0068068430 0x7f006806a8f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f00700ae000 tx=0x7f00700a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:59.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:41:59.884+0000 7f0063fff640 1 -- 172.21.15.29:0/2724629207 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f006c0570c0 con 0x7f007c0d0390 2024-04-25T00:42:00.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.048+0000 7f0081c64640 1 -- 172.21.15.29:0/2724629207 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f007c0008e0 con 0x7f0068068430 2024-04-25T00:42:00.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.048+0000 7f0063fff640 1 -- 172.21.15.29:0/2724629207 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13373 (secure 0 0 0) 0x7f007c0008e0 con 0x7f0068068430 2024-04-25T00:42:00.049 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:42:00.049 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:42:00.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.052+0000 7f0081c64640 1 -- 172.21.15.29:0/2724629207 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0068068430 msgr2=0x7f006806a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:00.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.052+0000 7f0081c64640 1 --2- 172.21.15.29:0/2724629207 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0068068430 0x7f006806a8f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f00700ae000 tx=0x7f00700a7000 comp rx=0 tx=0).stop 2024-04-25T00:42:00.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.052+0000 7f0081c64640 1 -- 172.21.15.29:0/2724629207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f007c0d0390 msgr2=0x7f007c0b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:00.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.052+0000 7f0081c64640 1 --2- 172.21.15.29:0/2724629207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f007c0d0390 0x7f007c0b1060 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f006c00d800 tx=0x7f006c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:42:00.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.052+0000 7f0081c64640 1 -- 172.21.15.29:0/2724629207 shutdown_connections 2024-04-25T00:42:00.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.052+0000 7f0081c64640 1 --2- 172.21.15.29:0/2724629207 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7f0068068430 0x7f006806a8f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:00.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.052+0000 7f0081c64640 1 --2- 172.21.15.29:0/2724629207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f007c15b640 0x7f007c0c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:00.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.052+0000 7f0081c64640 1 --2- 172.21.15.29:0/2724629207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15ad20 0x7f007c162ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:00.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.052+0000 7f0081c64640 1 --2- 172.21.15.29:0/2724629207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f007c0d0390 0x7f007c0b1060 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:00.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.052+0000 7f0081c64640 1 -- 172.21.15.29:0/2724629207 >> 172.21.15.29:0/2724629207 conn(0x7f007c0d0c40 msgr2=0x7f007c153d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:00.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.052+0000 7f0081c64640 1 -- 172.21.15.29:0/2724629207 shutdown_connections 2024-04-25T00:42:00.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:00.052+0000 7f0081c64640 1 -- 172.21.15.29:0/2724629207 wait complete. 2024-04-25T00:42:00.068 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-04-25T00:41:59.431163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":321632,"kb_used_data":18088,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754653088,"statfs":{"total":773094113280,"available":772764762112,"internally_reserved":0,"allocated":18522112,"data_stored":8824024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-4,"num_write_kb":-2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.513027"},"pg_stats":[{"pgid":"1.0","version":"73'128","reported_seq":496,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T00:41:52.844418+0000","last_change":"2024-04-25T00:31:25.453045+0000","last_active":"2024-04-25T00:41:52.844418+0000","last_peered":"2024-04-25T00:41:52.844418+0000","last_clean":"2024-04-25T00:41:52.844418+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:41:52.844418+0000","last_undegraded":"2024-04-25T00:41:52.844418+0000","last_fullsized":"2024-04-25T00:41:52.844418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:31:24.214108+0000","last_clean_scrub_stamp":"2024-04-25T00:31:24.214108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:20.247505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574978,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38036,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333804,"statfs":{"total":96636764160,"available":96597815296,"internally_reserved":0,"allocated":2760704,"data_stored":1545779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574979,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37456,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2166784,"data_stored":955411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574979,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37460,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334380,"statfs":{"total":96636764160,"available":96598405120,"internally_reserved":0,"allocated":2166784,"data_stored":955411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41624,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330216,"statfs":{"total":96636764160,"available":96594141184,"internally_reserved":0,"allocated":2166784,"data_stored":955411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574979,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41620,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330220,"statfs":{"total":96636764160,"available":96594145280,"internally_reserved":0,"allocated":2166784,"data_stored":955411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41620,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330220,"statfs":{"total":96636764160,"available":96594145280,"internally_reserved":0,"allocated":2166784,"data_stored":955411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574979,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41616,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330224,"statfs":{"total":96636764160,"available":96594149376,"internally_reserved":0,"allocated":2166784,"data_stored":955411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574978,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42200,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329640,"statfs":{"total":96636764160,"available":96593551360,"internally_reserved":0,"allocated":2760704,"data_stored":1545779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:42:00.071 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T00:31:24.214108+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=31, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=41, tm_sec=54, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:42:00.072 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T00:42:11.717 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:42:11.720 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:42:20.073 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:42:20.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7c052d640 1 Processor -- start 2024-04-25T00:42:20.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7c052d640 1 -- start start 2024-04-25T00:42:20.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7c052d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b815c220 0x7fc7b80cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7c052d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7b80d0670 0x7fc7b80b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7c052d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b80cdbd0 con 0x7fc7b80cfa10 2024-04-25T00:42:20.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7c052d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc7b815c600 con 0x7fc7b80d0670 2024-04-25T00:42:20.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7c052d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b80b1a30 con 0x7fc7b815c220 2024-04-25T00:42:20.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bdac8640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc7b80cfa10 0x7fc7b80cff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58704/0 (socket says 172.21.15.29:58704) 2024-04-25T00:42:20.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bdac8640 1 -- 172.21.15.29:0/2699270774 learned_addr learned my addr 172.21.15.29:0/2699270774 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:42:20.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7beaca640 1 --2- 172.21.15.29:0/2699270774 >> v2:172.21.15.157:3300/0 conn(0x7fc7b80d0670 0x7fc7b80b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1769828319 0 0) 0x7fc7b80cdbd0 con 0x7fc7b80cfa10 2024-04-25T00:42:20.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7a0003530 con 0x7fc7b80cfa10 2024-04-25T00:42:20.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7be2c9640 1 --2- 172.21.15.29:0/2699270774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b815c220 0x7fc7b80cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1030418276 0 0) 0x7fc7a0003530 con 0x7fc7b80cfa10 2024-04-25T00:42:20.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b815c220 msgr2=0x7fc7b80cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:20.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 --2- 172.21.15.29:0/2699270774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b815c220 0x7fc7b80cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 >> v2:172.21.15.157:3300/0 conn(0x7fc7b80d0670 msgr2=0x7fc7b80b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:20.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 --2- 172.21.15.29:0/2699270774 >> v2:172.21.15.157:3300/0 conn(0x7fc7b80d0670 0x7fc7b80b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b80b13e0 con 0x7fc7b80cfa10 2024-04-25T00:42:20.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fc7ac001a00 con 0x7fc7b80cfa10 2024-04-25T00:42:20.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 >> v1:172.21.15.29:6789/0 conn(0x7fc7b80cfa10 legacy=0x7fc7b80cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:20.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 --2- 172.21.15.29:0/2699270774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b815c220 0x7fc7a00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 --2- 172.21.15.29:0/2699270774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7a0003f40 0x7fc7a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 --2- 172.21.15.29:0/2699270774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7a0004d50 0x7fc7a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b815c600 con 0x7fc7a0004d50 2024-04-25T00:42:20.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b80b1a30 con 0x7fc7a0003f40 2024-04-25T00:42:20.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a000ba20 con 0x7fc7b815c220 2024-04-25T00:42:20.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7beaca640 1 --2- 172.21.15.29:0/2699270774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7a0004d50 0x7fc7a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7bdac8640 1 --2- 172.21.15.29:0/2699270774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7a0003f40 0x7fc7a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7be2c9640 1 --2- 172.21.15.29:0/2699270774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b815c220 0x7fc7a00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7beaca640 1 -- 172.21.15.29:0/2699270774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b815c220 msgr2=0x7fc7a00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:20.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7beaca640 1 --2- 172.21.15.29:0/2699270774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b815c220 0x7fc7a00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7beaca640 1 -- 172.21.15.29:0/2699270774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7a0003f40 msgr2=0x7fc7a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:20.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7beaca640 1 --2- 172.21.15.29:0/2699270774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7a0003f40 0x7fc7a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.207+0000 7fc7beaca640 1 -- 172.21.15.29:0/2699270774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a0003530 con 0x7fc7a0004d50 2024-04-25T00:42:20.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.211+0000 7fc7beaca640 1 --2- 172.21.15.29:0/2699270774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7a0004d50 0x7fc7a00051d0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009ae0 tx=0x7fc7b001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305c5cc53e2c54d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:20.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.211+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00b2070 con 0x7fc7a0004d50 2024-04-25T00:42:20.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.211+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b00a56a0 con 0x7fc7a0004d50 2024-04-25T00:42:20.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.211+0000 7fc7bd2c7640 1 -- 172.21.15.29:0/2699270774 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00096b0 con 0x7fc7a0004d50 2024-04-25T00:42:20.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.211+0000 7fc7c052d640 1 -- 172.21.15.29:0/2699270774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7a0004d50 msgr2=0x7fc7a00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:20.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.211+0000 7fc7c052d640 1 --2- 172.21.15.29:0/2699270774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7a0004d50 0x7fc7a00051d0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009ae0 tx=0x7fc7b001cab0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.211+0000 7fc7beaca640 1 -- 172.21.15.29:0/2699270774 reap_dead start 2024-04-25T00:42:20.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.211+0000 7fc7c052d640 1 -- 172.21.15.29:0/2699270774 shutdown_connections 2024-04-25T00:42:20.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.211+0000 7fc7c052d640 1 -- 172.21.15.29:0/2699270774 >> 172.21.15.29:0/2699270774 conn(0x7fc7b80d1d60 msgr2=0x7fc7b80d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:20.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.211+0000 7fc7c052d640 1 -- 172.21.15.29:0/2699270774 shutdown_connections 2024-04-25T00:42:20.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.211+0000 7fc7c052d640 1 -- 172.21.15.29:0/2699270774 wait complete. 2024-04-25T00:42:20.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.211+0000 7fc7c052d640 1 Processor -- start 2024-04-25T00:42:20.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.211+0000 7fc7c052d640 1 -- start start 2024-04-25T00:42:20.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7c052d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7b80b0070 0x7fc7b80b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7c052d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b80b67f0 0x7fc7b8150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7c052d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80b0450 0x7fc7b8151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7c052d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b80cd4d0 con 0x7fc7b80b0070 2024-04-25T00:42:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7c052d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b80cd350 con 0x7fc7b80b67f0 2024-04-25T00:42:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7c052d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b80cd650 con 0x7fc7b80b0450 2024-04-25T00:42:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7be2c9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7b80b0070 0x7fc7b80b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7be2c9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7b80b0070 0x7fc7b80b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36962/0 (socket says 172.21.15.29:36962) 2024-04-25T00:42:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7be2c9640 1 -- 172.21.15.29:0/2939252446 learned_addr learned my addr 172.21.15.29:0/2939252446 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:42:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7bdac8640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b80b67f0 0x7fc7b8150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7beaca640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80b0450 0x7fc7b8151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7be2c9640 1 -- 172.21.15.29:0/2939252446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80b0450 msgr2=0x7fc7b8151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7be2c9640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80b0450 0x7fc7b8151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7be2c9640 1 -- 172.21.15.29:0/2939252446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b80b67f0 msgr2=0x7fc7b8150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7be2c9640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b80b67f0 0x7fc7b8150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7be2c9640 1 -- 172.21.15.29:0/2939252446 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b80b45e0 con 0x7fc7b80b0070 2024-04-25T00:42:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7bdac8640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b80b67f0 0x7fc7b8150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7be2c9640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7b80b0070 0x7fc7b80b62b0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800f580 tx=0x7fc7a800fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7b6ffd640 1 -- 172.21.15.29:0/2939252446 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8019020 con 0x7fc7b80b0070 2024-04-25T00:42:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7b6ffd640 1 -- 172.21.15.29:0/2939252446 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a8014870 con 0x7fc7b80b0070 2024-04-25T00:42:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7c052d640 1 -- 172.21.15.29:0/2939252446 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b80b48c0 con 0x7fc7b80b0070 2024-04-25T00:42:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7b6ffd640 1 -- 172.21.15.29:0/2939252446 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8014c40 con 0x7fc7b80b0070 2024-04-25T00:42:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7c052d640 1 -- 172.21.15.29:0/2939252446 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b80b4e80 con 0x7fc7b80b0070 2024-04-25T00:42:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7c052d640 1 -- 172.21.15.29:0/2939252446 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b8069270 con 0x7fc7b80b0070 2024-04-25T00:42:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7b6ffd640 1 -- 172.21.15.29:0/2939252446 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fc7a801c020 con 0x7fc7b80b0070 2024-04-25T00:42:20.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7b6ffd640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc7a4068400 0x7fc7a406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7b6ffd640 1 -- 172.21.15.29:0/2939252446 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7fc7a8084f40 con 0x7fc7b80b0070 2024-04-25T00:42:20.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.215+0000 7fc7bdac8640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc7a4068400 0x7fc7a406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.219+0000 7fc7bdac8640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc7a4068400 0x7fc7a406a8c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac001b70 tx=0x7fc7ac001260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.223+0000 7fc7b6ffd640 1 -- 172.21.15.29:0/2939252446 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc7a8023020 con 0x7fc7b80b0070 2024-04-25T00:42:20.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.375+0000 7fc7c052d640 1 -- 172.21.15.29:0/2939252446 --> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7b8068ba0 con 0x7fc7a4068400 2024-04-25T00:42:20.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.379+0000 7fc7b6ffd640 1 -- 172.21.15.29:0/2939252446 <== mgr.4103 v2:172.21.15.29:6836/1230377703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7fc7b8068ba0 con 0x7fc7a4068400 2024-04-25T00:42:20.381 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:42:20.381 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-04-25T00:42:20.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.379+0000 7fc7c052d640 1 -- 172.21.15.29:0/2939252446 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc7a4068400 msgr2=0x7fc7a406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:20.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.379+0000 7fc7c052d640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc7a4068400 0x7fc7a406a8c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac001b70 tx=0x7fc7ac001260 comp rx=0 tx=0).stop 2024-04-25T00:42:20.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.379+0000 7fc7c052d640 1 -- 172.21.15.29:0/2939252446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7b80b0070 msgr2=0x7fc7b80b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:20.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.379+0000 7fc7c052d640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7b80b0070 0x7fc7b80b62b0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800f580 tx=0x7fc7a800fa40 comp rx=0 tx=0).stop 2024-04-25T00:42:20.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.379+0000 7fc7c052d640 1 -- 172.21.15.29:0/2939252446 shutdown_connections 2024-04-25T00:42:20.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.379+0000 7fc7c052d640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fc7a4068400 0x7fc7a406a8c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.379+0000 7fc7c052d640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80b0450 0x7fc7b8151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.379+0000 7fc7c052d640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b80b67f0 0x7fc7b8150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.379+0000 7fc7c052d640 1 --2- 172.21.15.29:0/2939252446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7b80b0070 0x7fc7b80b62b0 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.379+0000 7fc7c052d640 1 -- 172.21.15.29:0/2939252446 >> 172.21.15.29:0/2939252446 conn(0x7fc7b80d1d60 msgr2=0x7fc7b80af290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:20.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.379+0000 7fc7c052d640 1 -- 172.21.15.29:0/2939252446 shutdown_connections 2024-04-25T00:42:20.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.379+0000 7fc7c052d640 1 -- 172.21.15.29:0/2939252446 wait complete. 2024-04-25T00:42:20.398 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-04-25T00:42:19.437390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":321620,"kb_used_data":18088,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754653100,"statfs":{"total":773094113280,"available":772764774400,"internally_reserved":0,"allocated":18522112,"data_stored":8824024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003843"},"pg_stats":[{"pgid":"1.0","version":"171'130","reported_seq":507,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T00:42:00.356810+0000","last_change":"2024-04-25T00:42:00.356878+0000","last_active":"2024-04-25T00:42:00.356810+0000","last_peered":"2024-04-25T00:42:00.356810+0000","last_clean":"2024-04-25T00:42:00.356810+0000","last_became_active":"2024-04-25T00:31:25.450771+0000","last_became_peered":"2024-04-25T00:31:25.450771+0000","last_unstale":"2024-04-25T00:42:00.356810+0000","last_undegraded":"2024-04-25T00:42:00.356810+0000","last_fullsized":"2024-04-25T00:42:00.356810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"171'130","last_scrub_stamp":"2024-04-25T00:42:00.356633+0000","last_deep_scrub":"171'130","last_deep_scrub_stamp":"2024-04-25T00:42:00.356633+0000","last_clean_scrub_stamp":"2024-04-25T00:42:00.356633+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:03.635085+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574982,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38032,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333808,"statfs":{"total":96636764160,"available":96597819392,"internally_reserved":0,"allocated":2760704,"data_stored":1545779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37456,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2166784,"data_stored":955411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37460,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334380,"statfs":{"total":96636764160,"available":96598405120,"internally_reserved":0,"allocated":2166784,"data_stored":955411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41624,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330216,"statfs":{"total":96636764160,"available":96594141184,"internally_reserved":0,"allocated":2166784,"data_stored":955411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41620,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330220,"statfs":{"total":96636764160,"available":96594145280,"internally_reserved":0,"allocated":2166784,"data_stored":955411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41620,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330220,"statfs":{"total":96636764160,"available":96594145280,"internally_reserved":0,"allocated":2166784,"data_stored":955411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41616,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330224,"statfs":{"total":96636764160,"available":96594149376,"internally_reserved":0,"allocated":2166784,"data_stored":955411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42192,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329648,"statfs":{"total":96636764160,"available":96593559552,"internally_reserved":0,"allocated":2760704,"data_stored":1545779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:42:20.399 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T00:42:20.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efecba02640 1 Processor -- start 2024-04-25T00:42:20.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efecba02640 1 -- start start 2024-04-25T00:42:20.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efecba02640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efec40d29a0 0x7efec40cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efecba02640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efec40ce750 0x7efec40cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efecba02640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec40cdda0 con 0x7efec415c110 2024-04-25T00:42:20.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efecba02640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7efec40cfa80 con 0x7efec40d29a0 2024-04-25T00:42:20.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efecba02640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efec40cfd60 con 0x7efec40ce750 2024-04-25T00:42:20.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efec979e640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7efec415c110 0x7efec40d0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58718/0 (socket says 172.21.15.29:58718) 2024-04-25T00:42:20.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efec979e640 1 -- 172.21.15.29:0/483796865 learned_addr learned my addr 172.21.15.29:0/483796865 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:42:20.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efec8f9d640 1 --2- 172.21.15.29:0/483796865 >> v2:172.21.15.29:3300/0 conn(0x7efec40d29a0 0x7efec40cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efebbfff640 1 -- 172.21.15.29:0/483796865 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1786960524 0 0) 0x7efec40cdda0 con 0x7efec415c110 2024-04-25T00:42:20.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efec9f9f640 1 --2- 172.21.15.29:0/483796865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efec40ce750 0x7efec40cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efebbfff640 1 -- 172.21.15.29:0/483796865 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efeb0003530 con 0x7efec415c110 2024-04-25T00:42:20.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efec8f9d640 1 -- 172.21.15.29:0/483796865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efec40ce750 msgr2=0x7efec40cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:20.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efec8f9d640 1 --2- 172.21.15.29:0/483796865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efec40ce750 0x7efec40cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efec8f9d640 1 -- 172.21.15.29:0/483796865 >> v1:172.21.15.29:6789/0 conn(0x7efec415c110 legacy=0x7efec40d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:20.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.515+0000 7efec8f9d640 1 -- 172.21.15.29:0/483796865 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec40d0040 con 0x7efec40d29a0 2024-04-25T00:42:20.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efec8f9d640 1 --2- 172.21.15.29:0/483796865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efec40d29a0 0x7efec40cd4a0 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7efeb40097a0 tx=0x7efeb401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284914fb8add6fd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:20.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efebbfff640 1 -- 172.21.15.29:0/483796865 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb40b5070 con 0x7efec40d29a0 2024-04-25T00:42:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efebbfff640 1 -- 172.21.15.29:0/483796865 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb40a8970 con 0x7efec40d29a0 2024-04-25T00:42:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efebbfff640 1 -- 172.21.15.29:0/483796865 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb40a8d40 con 0x7efec40d29a0 2024-04-25T00:42:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 -- 172.21.15.29:0/483796865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efec40d29a0 msgr2=0x7efec40cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 --2- 172.21.15.29:0/483796865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efec40d29a0 0x7efec40cd4a0 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7efeb40097a0 tx=0x7efeb401ec70 comp rx=0 tx=0).stop 2024-04-25T00:42:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 -- 172.21.15.29:0/483796865 shutdown_connections 2024-04-25T00:42:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 --2- 172.21.15.29:0/483796865 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efec40ce750 0x7efec40cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 --2- 172.21.15.29:0/483796865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efec40d29a0 0x7efec40cd4a0 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 -- 172.21.15.29:0/483796865 >> 172.21.15.29:0/483796865 conn(0x7efec40d1e10 msgr2=0x7efec40d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:20.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 -- 172.21.15.29:0/483796865 shutdown_connections 2024-04-25T00:42:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 -- 172.21.15.29:0/483796865 wait complete. 2024-04-25T00:42:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 Processor -- start 2024-04-25T00:42:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 -- start start 2024-04-25T00:42:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efec40ce750 0x7efec414f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efec40d29a0 0x7efec4159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efec415c110 0x7efec406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efec415c670 con 0x7efec415c110 2024-04-25T00:42:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efec415c4f0 con 0x7efec40d29a0 2024-04-25T00:42:20.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efecba02640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efec40cd9e0 con 0x7efec40ce750 2024-04-25T00:42:20.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efec9f9f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efec415c110 0x7efec406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efec9f9f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efec415c110 0x7efec406d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36976/0 (socket says 172.21.15.29:36976) 2024-04-25T00:42:20.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efec9f9f640 1 -- 172.21.15.29:0/1120213506 learned_addr learned my addr 172.21.15.29:0/1120213506 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:42:20.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efec8f9d640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efec40d29a0 0x7efec4159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efec979e640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efec40ce750 0x7efec414f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efec9f9f640 1 -- 172.21.15.29:0/1120213506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efec40ce750 msgr2=0x7efec414f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:20.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efec9f9f640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efec40ce750 0x7efec414f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efec9f9f640 1 -- 172.21.15.29:0/1120213506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efec40d29a0 msgr2=0x7efec4159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:20.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efec9f9f640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efec40d29a0 0x7efec4159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efec9f9f640 1 -- 172.21.15.29:0/1120213506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec4153d80 con 0x7efec415c110 2024-04-25T00:42:20.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.519+0000 7efec9f9f640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efec415c110 0x7efec406d360 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7efebc007e40 tx=0x7efebc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:20.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.523+0000 7efeb9ffb640 1 -- 172.21.15.29:0/1120213506 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc014620 con 0x7efec415c110 2024-04-25T00:42:20.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.523+0000 7efeb9ffb640 1 -- 172.21.15.29:0/1120213506 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efebc019070 con 0x7efec415c110 2024-04-25T00:42:20.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.523+0000 7efecba02640 1 -- 172.21.15.29:0/1120213506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec41542d0 con 0x7efec415c110 2024-04-25T00:42:20.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.523+0000 7efeb9ffb640 1 -- 172.21.15.29:0/1120213506 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc014a20 con 0x7efec415c110 2024-04-25T00:42:20.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.523+0000 7efecba02640 1 -- 172.21.15.29:0/1120213506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec406ce40 con 0x7efec415c110 2024-04-25T00:42:20.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.523+0000 7efecba02640 1 -- 172.21.15.29:0/1120213506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec4069270 con 0x7efec415c110 2024-04-25T00:42:20.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.523+0000 7efeb9ffb640 1 -- 172.21.15.29:0/1120213506 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7efebc014bc0 con 0x7efec415c110 2024-04-25T00:42:20.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.523+0000 7efeb9ffb640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efea8068400 0x7efea806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:20.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.523+0000 7efeb9ffb640 1 -- 172.21.15.29:0/1120213506 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7efebc084eb0 con 0x7efec415c110 2024-04-25T00:42:20.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.523+0000 7efec979e640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efea8068400 0x7efea806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:20.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.523+0000 7efec979e640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efea8068400 0x7efea806a8c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7efec0005890 tx=0x7efec0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:20.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.527+0000 7efeb9ffb640 1 -- 172.21.15.29:0/1120213506 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efebc0574a0 con 0x7efec415c110 2024-04-25T00:42:20.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.691+0000 7efecba02640 1 -- 172.21.15.29:0/1120213506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7efec406d010 con 0x7efec415c110 2024-04-25T00:42:20.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.695+0000 7efeb9ffb640 1 -- 172.21.15.29:0/1120213506 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v32)=0 v32) ==== 123+0+0 (secure 0 0 0) 0x7efebc056da0 con 0x7efec415c110 2024-04-25T00:42:20.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.699+0000 7efeb9ffb640 1 -- 172.21.15.29:0/1120213506 <== mon.0 v2:172.21.15.29:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efebc05b150 con 0x7efec415c110 2024-04-25T00:42:20.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.699+0000 7efecba02640 1 -- 172.21.15.29:0/1120213506 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efea8068400 msgr2=0x7efea806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:20.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.699+0000 7efecba02640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efea8068400 0x7efea806a8c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7efec0005890 tx=0x7efec0009040 comp rx=0 tx=0).stop 2024-04-25T00:42:20.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.699+0000 7efecba02640 1 -- 172.21.15.29:0/1120213506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efec415c110 msgr2=0x7efec406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:20.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.699+0000 7efecba02640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efec415c110 0x7efec406d360 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7efebc007e40 tx=0x7efebc00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.699+0000 7efecba02640 1 -- 172.21.15.29:0/1120213506 shutdown_connections 2024-04-25T00:42:20.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.699+0000 7efecba02640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7efea8068400 0x7efea806a8c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.699+0000 7efecba02640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efec415c110 0x7efec406d360 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.699+0000 7efecba02640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efec40d29a0 0x7efec4159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.699+0000 7efecba02640 1 --2- 172.21.15.29:0/1120213506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efec40ce750 0x7efec414f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:20.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.699+0000 7efecba02640 1 -- 172.21.15.29:0/1120213506 >> 172.21.15.29:0/1120213506 conn(0x7efec40d1e10 msgr2=0x7efec40d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:20.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.699+0000 7efecba02640 1 -- 172.21.15.29:0/1120213506 shutdown_connections 2024-04-25T00:42:20.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.699+0000 7efecba02640 1 -- 172.21.15.29:0/1120213506 wait complete. 2024-04-25T00:42:20.719 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:42:20.719 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:42:20.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.839+0000 7fb096c07640 1 Processor -- start 2024-04-25T00:42:21.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.839+0000 7fb096c07640 1 -- start start 2024-04-25T00:42:21.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.839+0000 7fb096c07640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb0900b4e00 0x7fb0900b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.839+0000 7fb096c07640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0900b30b0 0x7fb09006cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.839+0000 7fb096c07640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb090071690 con 0x7fb090162420 2024-04-25T00:42:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.839+0000 7fb096c07640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb0900b3500 con 0x7fb0900b4e00 2024-04-25T00:42:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.839+0000 7fb096c07640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb090071910 con 0x7fb0900b30b0 2024-04-25T00:42:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb0949a3640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb090162420 0x7fb090070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58726/0 (socket says 172.21.15.29:58726) 2024-04-25T00:42:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb0949a3640 1 -- 172.21.15.29:0/475309130 learned_addr learned my addr 172.21.15.29:0/475309130 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-25T00:42:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 250844166 0 0) 0x7fb0880014a0 con 0x7fb090162420 2024-04-25T00:42:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb0951a4640 1 --2- 172.21.15.29:0/475309130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0900b30b0 0x7fb09006cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08ffff640 1 --2- 172.21.15.29:0/475309130 >> v2:172.21.15.157:3300/0 conn(0x7fb0900b4e00 0x7fb0900b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb078003530 con 0x7fb090162420 2024-04-25T00:42:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2935495812 0 0) 0x7fb078003530 con 0x7fb090162420 2024-04-25T00:42:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0900b30b0 msgr2=0x7fb09006cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 --2- 172.21.15.29:0/475309130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0900b30b0 0x7fb09006cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 >> v2:172.21.15.157:3300/0 conn(0x7fb0900b4e00 msgr2=0x7fb0900b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 --2- 172.21.15.29:0/475309130 >> v2:172.21.15.157:3300/0 conn(0x7fb0900b4e00 0x7fb0900b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb090071bf0 con 0x7fb090162420 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 164731384 0 0) 0x7fb088001bf0 con 0x7fb090162420 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 >> v1:172.21.15.29:6789/0 conn(0x7fb090162420 legacy=0x7fb090070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 --2- 172.21.15.29:0/475309130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0900b30b0 0x7fb0780039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 --2- 172.21.15.29:0/475309130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb078003f40 0x7fb078004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 --2- 172.21.15.29:0/475309130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb078004d50 0x7fb0780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb0900b3500 con 0x7fb078003f40 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb090071910 con 0x7fb078004d50 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb07800b9d0 con 0x7fb0900b30b0 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08ffff640 1 --2- 172.21.15.29:0/475309130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb078003f40 0x7fb078004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb0949a3640 1 --2- 172.21.15.29:0/475309130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0900b30b0 0x7fb0780039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb0951a4640 1 --2- 172.21.15.29:0/475309130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb078004d50 0x7fb0780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08ffff640 1 -- 172.21.15.29:0/475309130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0900b30b0 msgr2=0x7fb0780039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08ffff640 1 --2- 172.21.15.29:0/475309130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0900b30b0 0x7fb0780039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08ffff640 1 -- 172.21.15.29:0/475309130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb078004d50 msgr2=0x7fb0780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08ffff640 1 --2- 172.21.15.29:0/475309130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb078004d50 0x7fb0780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08ffff640 1 -- 172.21.15.29:0/475309130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb078003530 con 0x7fb078003f40 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08ffff640 1 --2- 172.21.15.29:0/475309130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb078003f40 0x7fb078004810 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7fb084009ae0 tx=0x7fb08401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc75cc6181a66b4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0840b2070 con 0x7fb078003f40 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb0840a56a0 con 0x7fb078003f40 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.843+0000 7fb08f7fe640 1 -- 172.21.15.29:0/475309130 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0840096b0 con 0x7fb078003f40 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 -- 172.21.15.29:0/475309130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb078003f40 msgr2=0x7fb078004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 --2- 172.21.15.29:0/475309130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb078003f40 0x7fb078004810 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7fb084009ae0 tx=0x7fb08401ca50 comp rx=0 tx=0).stop 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb0951a4640 1 -- 172.21.15.29:0/475309130 reap_dead start 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 -- 172.21.15.29:0/475309130 shutdown_connections 2024-04-25T00:42:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 -- 172.21.15.29:0/475309130 >> 172.21.15.29:0/475309130 conn(0x7fb0900b1870 msgr2=0x7fb0900b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 -- 172.21.15.29:0/475309130 shutdown_connections 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 -- 172.21.15.29:0/475309130 wait complete. 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 Processor -- start 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 -- start start 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb078003f40 0x7fb0900aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb090153f80 0x7fb0900c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0900c5020 0x7fb0900accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb090162800 con 0x7fb090153f80 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb0900721b0 con 0x7fb078003f40 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb09006cf80 con 0x7fb0900c5020 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb08ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb090153f80 0x7fb0900c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb08ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb090153f80 0x7fb0900c4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36998/0 (socket says 172.21.15.29:36998) 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb08ffff640 1 -- 172.21.15.29:0/3254389136 learned_addr learned my addr 172.21.15.29:0/3254389136 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-25T00:42:21.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb0949a3640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb078003f40 0x7fb0900aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb0951a4640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0900c5020 0x7fb0900accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb08ffff640 1 -- 172.21.15.29:0/3254389136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0900c5020 msgr2=0x7fb0900accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb08ffff640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0900c5020 0x7fb0900accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb08ffff640 1 -- 172.21.15.29:0/3254389136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb078003f40 msgr2=0x7fb0900aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb08ffff640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb078003f40 0x7fb0900aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb08ffff640 1 -- 172.21.15.29:0/3254389136 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0901543e0 con 0x7fb090153f80 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb08ffff640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb090153f80 0x7fb0900c4ae0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7fb08401c880 tx=0x7fb078005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb08d7fa640 1 -- 172.21.15.29:0/3254389136 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0840b2040 con 0x7fb090153f80 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb08d7fa640 1 -- 172.21.15.29:0/3254389136 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb084004020 con 0x7fb090153f80 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 -- 172.21.15.29:0/3254389136 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0900ad9c0 con 0x7fb090153f80 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb08d7fa640 1 -- 172.21.15.29:0/3254389136 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084004450 con 0x7fb090153f80 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.847+0000 7fb096c07640 1 -- 172.21.15.29:0/3254389136 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0900adf80 con 0x7fb090153f80 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.851+0000 7fb096c07640 1 -- 172.21.15.29:0/3254389136 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0900c5430 con 0x7fb090153f80 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.851+0000 7fb08d7fa640 1 -- 172.21.15.29:0/3254389136 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fb084003620 con 0x7fb090153f80 2024-04-25T00:42:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.851+0000 7fb08d7fa640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb07c068400 0x7fb07c06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:21.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.851+0000 7fb08d7fa640 1 -- 172.21.15.29:0/3254389136 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 4943+0+0 (secure 0 0 0) 0x7fb08411e930 con 0x7fb090153f80 2024-04-25T00:42:21.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.851+0000 7fb0949a3640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb07c068400 0x7fb07c06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:21.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.851+0000 7fb0949a3640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb07c068400 0x7fb07c06a8c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb088001780 tx=0x7fb0880015d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:21.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:20.855+0000 7fb08d7fa640 1 -- 172.21.15.29:0/3254389136 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb0900c5430 con 0x7fb090153f80 2024-04-25T00:42:21.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.051+0000 7fb096c07640 1 -- 172.21.15.29:0/3254389136 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb090072fa0 con 0x7fb090153f80 2024-04-25T00:42:21.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.051+0000 7fb08d7fa640 1 -- 172.21.15.29:0/3254389136 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+3880 (secure 0 0 0) 0x7fb0840167c0 con 0x7fb090153f80 2024-04-25T00:42:21.080 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-25T00:42:21.080 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":115,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7427,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/929849226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":929849226},{"type":"v1","addr":"172.21.15.157:6834","nonce":929849226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/1459365951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1459365951},{"type":"v1","addr":"172.21.15.29:6833","nonce":1459365951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1011867355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1011867355},{"type":"v1","addr":"172.21.15.157:6835","nonce":1011867355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":10989,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4012165990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":4012165990},{"type":"v1","addr":"172.21.15.29:6835","nonce":4012165990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115}],"filesystems":[]} 2024-04-25T00:42:21.082 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 115 2024-04-25T00:42:21.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.055+0000 7fb096c07640 1 -- 172.21.15.29:0/3254389136 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb07c068400 msgr2=0x7fb07c06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:21.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.055+0000 7fb096c07640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb07c068400 0x7fb07c06a8c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb088001780 tx=0x7fb0880015d0 comp rx=0 tx=0).stop 2024-04-25T00:42:21.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.055+0000 7fb096c07640 1 -- 172.21.15.29:0/3254389136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb090153f80 msgr2=0x7fb0900c4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:21.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.055+0000 7fb096c07640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb090153f80 0x7fb0900c4ae0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7fb08401c880 tx=0x7fb078005840 comp rx=0 tx=0).stop 2024-04-25T00:42:21.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.055+0000 7fb096c07640 1 -- 172.21.15.29:0/3254389136 shutdown_connections 2024-04-25T00:42:21.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.055+0000 7fb096c07640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.29:6836/1230377703,v1:172.21.15.29:6837/1230377703] conn(0x7fb07c068400 0x7fb07c06a8c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:21.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.055+0000 7fb096c07640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0900c5020 0x7fb0900accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:21.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.055+0000 7fb096c07640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb090153f80 0x7fb0900c4ae0 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:21.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.055+0000 7fb096c07640 1 --2- 172.21.15.29:0/3254389136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb078003f40 0x7fb0900aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:21.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.055+0000 7fb096c07640 1 -- 172.21.15.29:0/3254389136 >> 172.21.15.29:0/3254389136 conn(0x7fb0900b1870 msgr2=0x7fb090067160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:21.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.055+0000 7fb096c07640 1 -- 172.21.15.29:0/3254389136 shutdown_connections 2024-04-25T00:42:21.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-25T00:42:21.055+0000 7fb096c07640 1 -- 172.21.15.29:0/3254389136 wait complete. 2024-04-25T00:42:21.083 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T00:42:21.083 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T00:42:21.083 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:42:21.083 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:21.094 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:42:21.094 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:42:21.094 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:21.148 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:42:21.148 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:42:21.148 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:21.164 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:42:21.164 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:42:21.164 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:21.220 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:42:21.220 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T00:42:21.221 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T00:42:21.221 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:21.294 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T00:42:21.294 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T00:42:21.295 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:21.368 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T00:42:21.368 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T00:42:21.369 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:24.640 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-25T00:42:27.470 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T00:42:27.470 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T00:42:27.470 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:27.541 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T00:42:27.541 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T00:42:27.542 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:27.602 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T00:42:27.602 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T00:42:27.602 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:27.675 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T00:42:27.675 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T00:42:27.675 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:27.742 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T00:42:27.742 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T00:42:27.742 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:27.823 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T00:42:27.824 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T00:42:27.824 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T00:42:27.824 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:27.857 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T00:42:27.857 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T00:42:27.857 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:27.886 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T00:42:27.886 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T00:42:27.886 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T00:42:27.886 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:27.915 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T00:42:27.915 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T00:42:27.915 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:27.946 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T00:42:27.946 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T00:42:27.946 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:42:28.004 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T00:42:28.004 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T00:42:28.004 DEBUG:teuthology.orchestra.run.smithi029:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-04-25T00:42:28.023 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:42:28.023 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T00:42:28.414 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:42:28.414 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T00:42:28.714 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:42:28.715 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T00:42:28.971 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:42:28.971 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T00:42:29.231 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:42:29.231 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T00:42:29.507 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:42:29.508 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T00:42:29.642 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-25T00:42:29.715 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:42:29.715 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T00:42:29.906 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:42:29.906 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T00:42:30.074 INFO:tasks.ceph:Archiving mon data... 2024-04-25T00:42:30.076 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260/data/mon.a.tgz 2024-04-25T00:42:30.076 DEBUG:teuthology.orchestra.run.smithi029:> mktemp 2024-04-25T00:42:30.082 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/tmp.2sZjyeKS6p 2024-04-25T00:42:30.082 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.2sZjyeKS6p 2024-04-25T00:42:30.334 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0666 /tmp/tmp.2sZjyeKS6p 2024-04-25T00:42:30.401 DEBUG:teuthology.orchestra.remote:smithi029:/tmp/tmp.2sZjyeKS6p is 1MB 2024-04-25T00:42:30.469 DEBUG:teuthology.orchestra.run.smithi029:> rm -fr /tmp/tmp.2sZjyeKS6p 2024-04-25T00:42:30.475 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260/data/mon.b.tgz 2024-04-25T00:42:30.476 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-04-25T00:42:30.482 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.83r0HpUaKg 2024-04-25T00:42:30.485 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.83r0HpUaKg 2024-04-25T00:42:30.744 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.83r0HpUaKg 2024-04-25T00:42:30.805 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.83r0HpUaKg is 1MB 2024-04-25T00:42:30.876 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.83r0HpUaKg 2024-04-25T00:42:30.882 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260/data/mon.c.tgz 2024-04-25T00:42:30.882 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-04-25T00:42:30.930 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.ErxqKHwVYt 2024-04-25T00:42:30.930 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ErxqKHwVYt 2024-04-25T00:42:31.182 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.ErxqKHwVYt 2024-04-25T00:42:31.245 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.ErxqKHwVYt is 1MB 2024-04-25T00:42:31.316 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.ErxqKHwVYt 2024-04-25T00:42:31.322 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T00:42:31.322 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T00:42:31.324 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T00:42:31.384 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T00:42:31.426 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T00:42:31.427 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T00:42:31.968 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260/remote/smithi029/crash 2024-04-25T00:42:31.999 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T00:42:32.014 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260/remote/smithi157/crash 2024-04-25T00:42:32.015 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T00:42:32.029 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T00:42:32.029 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:42:32.064 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:42:32.114 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T00:42:32.127 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T00:42:32.169 INFO:tasks.ceph:Compressing logs... 2024-04-25T00:42:32.170 DEBUG:teuthology.orchestra.run.smithi029:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:42:32.172 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:42:32.192 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50722.log 2024-04-25T00:42:32.192 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44209.log 2024-04-25T00:42:32.193 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43947.log 2024-04-25T00:42:32.194 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50722.log.gz 2024-04-25T00:42:32.194 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59193.log 2024-04-25T00:42:32.194 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44209.log.gz 2024-04-25T00:42:32.194 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35018.log 2024-04-25T00:42:32.194 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37678.log 2024-04-25T00:42:32.197 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43947.log.gz 2024-04-25T00:42:32.197 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35018.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59193.log.gz 2024-04-25T00:42:32.197 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37851.log 2024-04-25T00:42:32.197 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35018.log.gz 2024-04-25T00:42:32.197 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44979.log 2024-04-25T00:42:32.197 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37678.log.gz 2024-04-25T00:42:32.197 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56604.log 2024-04-25T00:42:32.197 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37851.log: /var/log/ceph/ceph-client.admin.44979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49708.log 2024-04-25T00:42:32.198 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37851.log.gz 2024-04-25T00:42:32.198 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44979.log.gz 2024-04-25T00:42:32.198 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46568.log 2024-04-25T00:42:32.198 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2%/var/log/ceph/ceph-client.admin.49708.log: -- replaced with /var/log/ceph/ceph-client.admin.56604.log.gz 2024-04-25T00:42:32.198 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51037.log 2024-04-25T00:42:32.198 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45135.log 2024-04-25T00:42:32.198 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49708.log.gz 2024-04-25T00:42:32.198 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51037.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46568.log.gz 2024-04-25T00:42:32.198 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58116.log 2024-04-25T00:42:32.203 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51037.log.gz 2024-04-25T00:42:32.203 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44026.log 2024-04-25T00:42:32.203 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58116.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45135.log.gz 2024-04-25T00:42:32.203 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57089.log 2024-04-25T00:42:32.203 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58116.log.gz 2024-04-25T00:42:32.203 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51801.log 2024-04-25T00:42:32.203 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57089.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44026.log.gz 2024-04-25T00:42:32.203 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54180.log 2024-04-25T00:42:32.203 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57089.log.gz 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53037.log 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51801.log.gz 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52088.log 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54180.log.gz 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46862.log 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52088.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53037.log.gz 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34810.log 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52088.log.gz 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44666.log 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46862.log.gz 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40182.log 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44666.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34810.log.gz 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58497.log 2024-04-25T00:42:32.204 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45005.log 2024-04-25T00:42:32.205 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44666.log.gz 2024-04-25T00:42:32.205 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58497.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40182.log.gz 2024-04-25T00:42:32.205 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35841.log 2024-04-25T00:42:32.205 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58497.log.gz 2024-04-25T00:42:32.205 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46434.log 2024-04-25T00:42:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35631.log 2024-04-25T00:42:32.212 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45005.log.gz 2024-04-25T00:42:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46434.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35841.log.gz 2024-04-25T00:42:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55035.log 2024-04-25T00:42:32.212 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46434.log.gz 2024-04-25T00:42:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55367.log 2024-04-25T00:42:32.212 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35631.log.gz 2024-04-25T00:42:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53833.log 2024-04-25T00:42:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55367.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55035.log.gz 2024-04-25T00:42:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T00:42:32.212 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55367.log.gz 2024-04-25T00:42:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40126.log 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53833.log.gz 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44528.log 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52473.log 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40126.log: /var/log/ceph/ceph-client.admin.44528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37386.log 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52473.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44528.log.gz 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40126.log.gz 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40208.log 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39466.log 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37386.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.52473.log.gz 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40208.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37386.log.gz 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53781.log 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40208.log.gz 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46968.log 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.39466.log: /var/log/ceph/ceph-client.admin.53781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54679.log 2024-04-25T00:42:32.213 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53781.log.gz 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47916.log 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46968.log: /var/log/ceph/ceph-client.admin.54679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55805.log 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46968.log.gz 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54679.log.gz 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39204.log 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.0.39466.log.gz 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47916.log.gz 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57337.log 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55805.log: /var/log/ceph/ceph-client.admin.39204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41195.log 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55805.log.gz 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39204.log.gz 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59430.log 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57337.log.gz 2024-04-25T00:42:32.214 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53274.log 2024-04-25T00:42:32.215 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41195.log.gz 2024-04-25T00:42:32.215 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46347.log 2024-04-25T00:42:32.231 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42060.log 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59430.log.gz 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53274.log.gz 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40330.log 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46347.log.gz 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59482.log 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42060.log: /var/log/ceph/ceph-client.admin.40330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54706.log 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42060.log.gz 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59482.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40330.log.gz 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38385.log 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59482.log.gz 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33258.log 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54706.log: /var/log/ceph/ceph-client.admin.38385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45428.log 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38385.log.gz 2024-04-25T00:42:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36081.log 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50326.log 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33258.log.gz 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr: 91.3% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54706.log.gz -- replaced with /var/log/ceph/ceph-client.admin.45428.log.gz 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38664.log 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36081.log.gz 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52233.log 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42257.log 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50326.log.gz 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52233.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38664.log.gz 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49429.log 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52233.log.gz 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48996.log 2024-04-25T00:42:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:gzip/var/log/ceph/ceph-client.admin.49429.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36848.log 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42257.log.gz 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48996.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49429.log.gz 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57879.log 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48996.log.gz 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57444.log 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36848.log: /var/log/ceph/ceph-client.admin.57879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34120.log 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36848.log.gz 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57444.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57879.log.gz 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45955.log 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57444.log.gz 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55393.log 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34120.log.gz 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38957.log 2024-04-25T00:42:32.234 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45955.log: /var/log/ceph/ceph-client.admin.55393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54467.log 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45955.log.gz 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38957.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55393.log.gz 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53124.log 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38957.log.gz 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41036.log 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54467.log: /var/log/ceph/ceph-client.admin.53124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58367.log 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.53124.log.gz 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42963.log 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3%/var/log/ceph/ceph-client.admin.41036.log: -- replaced with /var/log/ceph/ceph-client.admin.54467.log.gz 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40546.log 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41036.log.gz 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58367.log.gz 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36549.log 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42963.log.gz 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48494.log 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40546.log: /var/log/ceph/ceph-client.admin.36549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54350.log 2024-04-25T00:42:32.235 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40546.log.gz 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36549.log.gz 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48494.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48494.log.gz 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47117.log 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53885.log 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54350.log.gz 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42909.log 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47117.log.gz 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35468.log 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45284.log 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53885.log.gz 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42909.log: /var/log/ceph/ceph-client.admin.35468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57652.log 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42909.log.gz 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35468.log.gz 2024-04-25T00:42:32.236 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59167.log 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45715.log 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57652.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45284.log.gz 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50074.log 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr: 89.5% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59167.log.gz 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57652.log.gz 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37230.log 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39126.log 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50074.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45715.log.gz 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37230.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50074.log.gz 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38508.log 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37230.log.gz 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57007.log 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51827.log 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38508.log: /var/log/ceph/ceph-client.admin.57007.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39126.log.gz 2024-04-25T00:42:32.237 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38508.log.gz 2024-04-25T00:42:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48860.log 2024-04-25T00:42:32.238 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57007.log.gz 2024-04-25T00:42:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40455.log 2024-04-25T00:42:32.238 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51827.log.gz 2024-04-25T00:42:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37730.log 2024-04-25T00:42:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54523.log 2024-04-25T00:42:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40455.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48860.log.gz 2024-04-25T00:42:32.238 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37730.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.40455.log.gz -5 2024-04-25T00:42:32.238 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38612.log 2024-04-25T00:42:32.238 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37730.log.gz 2024-04-25T00:42:32.239 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51600.log 2024-04-25T00:42:32.239 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50901.log 2024-04-25T00:42:32.267 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34041.log 2024-04-25T00:42:32.267 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33664.log 2024-04-25T00:42:32.267 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.34041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34437.log 2024-04-25T00:42:32.267 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34907.log 2024-04-25T00:42:32.267 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.33664.log: 90.1% -- replaced with /var/log/ceph/ceph-client.1.34041.log.gz 2024-04-25T00:42:32.268 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.34437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35219.log 2024-04-25T00:42:32.268 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.34907.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T00:42:32.268 INFO:teuthology.orchestra.run.smithi157.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.1.33664.log.gz 2024-04-25T00:42:32.268 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.35219.log: 90.0% -- replaced with /var/log/ceph/ceph-client.1.34437.log.gz 2024-04-25T00:42:32.268 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T00:42:32.268 INFO:teuthology.orchestra.run.smithi157.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.1.34907.log.gz 2024-04-25T00:42:32.268 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27878.log 2024-04-25T00:42:32.268 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27734.log 2024-04-25T00:42:32.268 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.27878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35063.log 2024-04-25T00:42:32.268 INFO:teuthology.orchestra.run.smithi157.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.1.35219.log.gz 2024-04-25T00:42:32.268 INFO:teuthology.orchestra.run.smithi157.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27878.log.gz 2024-04-25T00:42:32.268 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.27734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27830.log 2024-04-25T00:42:32.268 INFO:teuthology.orchestra.run.smithi157.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27734.log.gz 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.35063.log: /var/log/ceph/ceph-client.admin.27830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34232.log 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27830.log.gz 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27806.log 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34753.log 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.34232.log: /var/log/ceph/ceph-client.admin.27806.log: 90.2% -- replaced with /var/log/ceph/ceph-client.1.35063.log.gz 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27854.log 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.27806.log.gz 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.34753.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.1.34232.log.gz 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.27854.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35373.log 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27854.log.gz 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.1.34753.log.gz 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27710.log 2024-04-25T00:42:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.35373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27758.log 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33858.log 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.27710.log: 89.3% -- replaced with /var/log/ceph/ceph-client.1.35373.log.gz 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27710.log.gz 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.27758.log: 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27758.log.gz 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.33858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33387.log 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.33387.log: 89.6% -- replaced with /var/log/ceph/ceph-client.1.33858.log.gz 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.1.33387.log.gz 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph.audit.log: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T00:42:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mds.b.log: 93.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T00:42:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T00:42:32.271 INFO:teuthology.orchestra.run.smithi157.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz/var/log/ceph/ceph-mon.b.log: 2024-04-25T00:42:32.271 INFO:teuthology.orchestra.run.smithi157.stderr: 93.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T00:42:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33506.log 2024-04-25T00:42:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34591.log 2024-04-25T00:42:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.3.33506.log: 90.9% -- replaced with /var/log/ceph/ceph-client.3.33506.log.gz 2024-04-25T00:42:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27782.log 2024-04-25T00:42:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.34591.log: 92.0% -- replaced with /var/log/ceph/ceph-client.1.34591.log.gz 2024-04-25T00:42:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.27782.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27782.log.gz 2024-04-25T00:42:32.335 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38612.log: /var/log/ceph/ceph-client.admin.51600.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54523.log.gz 2024-04-25T00:42:32.335 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40939.log 2024-04-25T00:42:32.335 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51600.log.gz 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38612.log.gz 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60228.log 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50901.log: /var/log/ceph/ceph-client.admin.40939.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.50901.log.gz 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59560.log 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40939.log.gz 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52421.log 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46033.log 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60228.log.gz 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47785.log 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52421.log: /var/log/ceph/ceph-client.admin.46033.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59560.log.gz 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36497.log 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46033.log.gz 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52421.log.gz 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41709.log 2024-04-25T00:42:32.336 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53299.log 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36497.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47785.log.gz 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32776.log 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36497.log.gz 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41709.log.gz 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53408.log 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53299.log.gz 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60358.log 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32776.log: /var/log/ceph/ceph-client.admin.53408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51392.log 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53408.log.gz 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32776.log.gz 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49559.log 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37256.log 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51392.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60358.log.gz 2024-04-25T00:42:32.337 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43657.log 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51392.log.gz 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49559.log.gz 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42578.log 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.34578.log 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43657.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37256.log.gz 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55578.log 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43657.log.gz 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42578.log.gz 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45981.log 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.2.34578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48967.log 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55578.log: /var/log/ceph/ceph-client.admin.45981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33262.log 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55578.log.gz 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45981.log.gz 2024-04-25T00:42:32.338 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36821.log 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48967.log: /var/log/ceph/ceph-client.admin.33262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35815.log 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.2.34578.log.gz 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48967.log.gz 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33262.log.gz 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57496.log 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51202.log 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36821.log.gz 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35815.log.gz 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48834.log 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57496.log.gz 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44078.log 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51202.log: /var/log/ceph/ceph-client.admin.48834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34784.log 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51202.log.gz 2024-04-25T00:42:32.339 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44078.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48834.log.gz 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39410.log 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44078.log.gz 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53174.log 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52207.log 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34784.log.gz 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53174.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39410.log.gz 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47625.log 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53174.log.gz 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59786.log 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47625.log: gzip 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52207.log.gz -5 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55760.log 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47625.log.gz 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45258.log 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50849.log 2024-04-25T00:42:32.340 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55760.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59786.log.gz 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52291.log 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55760.log.gz 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45258.log.gz 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34250.log 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56630.log 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52291.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50849.log.gz 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53957.log 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52291.log.gz 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40022.log 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.34250.log.gz 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39730.log 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56630.log.gz 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40022.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53957.log.gz 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58038.log 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40022.log.gz 2024-04-25T00:42:32.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27697.log 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39730.log.gz 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39152.log 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58038.log: /var/log/ceph/ceph-client.admin.27697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36341.log 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58038.log.gz 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27697.log.gz 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51337.log 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39152.log.gz 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40783.log 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr:gzip/var/log/ceph/ceph-client.admin.51337.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50217.log 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36341.log.gz 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40783.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51337.log.gz 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34172.log 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40783.log.gz 2024-04-25T00:42:32.342 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50243.log 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34172.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50217.log.gz 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53983.log 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34172.log.gz 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46836.log 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35605.log 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53983.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50243.log.gz 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43015.log 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53983.log.gz 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46836.log.gz 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48314.log 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35605.log.gz 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33466.log 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57176.log 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43015.log.gz 2024-04-25T00:42:32.343 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48314.log: /var/log/ceph/ceph-client.admin.33466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44890.log 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48314.log.gz 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33466.log.gz 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45903.log 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45396.log 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57176.log.gz 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44890.log: /var/log/ceph/ceph-client.admin.45903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56788.log 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45903.log.gz 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44890.log.gz 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44052.log 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42734.log 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.56788.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45396.log.gz 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39520.log 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44052.log.gz 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39892.log 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39520.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42734.log.gz 2024-04-25T00:42:32.344 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38534.log 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39520.log.gz 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39651.log 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35442.log 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38534.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39892.log.gz 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35285.log 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38534.log.gz 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39651.log.gz 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56732.log 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35285.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35442.log.gz 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35337.log 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35285.log.gz 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42196.log 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42361.log 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35337.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56732.log.gz 2024-04-25T00:42:32.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45083.log 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35337.log.gz 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33261.log 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42196.log.gz 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42361.log: /var/log/ceph/ceph-client.admin.45083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50785.log 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42361.log.gz 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45083.log.gz 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56844.log 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33261.log: /var/log/ceph/ceph-client.admin.50785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40887.log 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33261.log.gz 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50785.log.gz 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52633.log 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40887.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.56844.log.gz -5 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51233.log 2024-04-25T00:42:32.346 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40887.log.gz 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52962.log 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52633.log.gz 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41404.log 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54912.log 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51233.log.gz 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52962.log: /var/log/ceph/ceph-client.admin.41404.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52962.log.gz 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50823.log 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38821.log 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41404.log.gz 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54912.log: /var/log/ceph/ceph-client.admin.50823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57678.log 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54912.log.gz 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50823.log.gz 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43358.log 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50126.log 2024-04-25T00:42:32.347 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57678.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38821.log.gz 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58878.log 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57678.log.gz 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43358.log.gz 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43867.log 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50126.log.gz 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47247.log 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42656.log 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.0.56788.log.gz 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.43867.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58878.log.gz 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58170.log 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41430.log 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47247.log.gz 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42656.log.gz 2024-04-25T00:42:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37360.log 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58170.log: 89.8% -- replaced with /var/log/ceph/ceph-client.0.43867.log.gz 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43605.log 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41430.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58170.log.gz 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50022.log 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41430.log.gz 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37360.log.gz 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35709.log 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45773.log 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50022.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43605.log.gz 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43198.log 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50022.log.gz 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35709.log.gz 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41947.log 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53599.log 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45773.log.gz 2024-04-25T00:42:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41947.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T00:42:32.350 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43198.log.gz 2024-04-25T00:42:32.350 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41947.log.gz 2024-04-25T00:42:32.350 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35226.log 2024-04-25T00:42:32.350 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35148.log 2024-04-25T00:42:32.350 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53599.log.gz 2024-04-25T00:42:32.350 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.35226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50482.log 2024-04-25T00:42:32.350 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35226.log.gz 2024-04-25T00:42:32.350 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36575.log 2024-04-25T00:42:32.350 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37767.log 2024-04-25T00:42:32.350 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8%/var/log/ceph/ceph-client.admin.50482.log: -- replaced with /var/log/ceph/ceph-client.admin.35148.log.gz 2024-04-25T00:42:32.350 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32865.log 2024-04-25T00:42:32.350 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2%/var/log/ceph/ceph-client.admin.37767.log: -- replaced with /var/log/ceph/ceph-client.admin.36575.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52837.log 2024-04-25T00:42:32.350 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50482.log.gz 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32865.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37767.log.gz 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48340.log 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32865.log.gz 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33470.log 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52837.log: /var/log/ceph/ceph-client.admin.48340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38638.log 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.52837.log.gz 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48340.log.gz 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38560.log 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36952.log 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33470.log.gz 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38638.log.gz 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37652.log 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38044.log 2024-04-25T00:42:32.351 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36952.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38560.log.gz 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57574.log 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36952.log.gz 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1%/var/log/ceph/ceph-client.admin.38044.log: -- replaced with /var/log/ceph/ceph-client.admin.37652.log.gz 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44838.log 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34940.log 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38044.log.gz 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57574.log.gz 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55064.log 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34940.log: gzip 89.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.44838.log.gz -- 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr: /var/log/ceph/ceph-client.admin.59812.log 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34940.log.gz 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48018.log 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59812.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55064.log.gz 2024-04-25T00:42:32.352 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39861.log 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59812.log.gz 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36367.log 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39861.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48018.log.gz 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59864.log 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39861.log.gz 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43921.log 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40152.log 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36367.log.gz 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59864.log: /var/log/ceph/ceph-client.admin.43921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48208.log 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59864.log.gz 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43921.log.gz 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57259.log 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52937.log 2024-04-25T00:42:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48208.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40152.log.gz 2024-04-25T00:42:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57548.log 2024-04-25T00:42:32.354 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48208.log.gz 2024-04-25T00:42:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52937.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57259.log.gz 2024-04-25T00:42:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42526.log 2024-04-25T00:42:32.354 INFO:teuthology.orchestra.run.smithi029.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52937.log.gz 2024-04-25T00:42:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55204.log 2024-04-25T00:42:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32722.log 2024-04-25T00:42:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42526.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57548.log.gz 2024-04-25T00:42:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32586.log 2024-04-25T00:42:32.354 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42526.log.gz 2024-04-25T00:42:32.354 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55204.log.gz 2024-04-25T00:42:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43631.log 2024-04-25T00:42:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32586.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.32722.log.gz 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39757.log 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43631.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32586.log.gz 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46594.log 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.43631.log.gz 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57522.log/var/log/ceph/ceph-client.admin.39757.log: 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39757.log.gz 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46729.log 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45206.log 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57522.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46594.log.gz 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38586.log 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57522.log.gz 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45206.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46729.log.gz 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T00:42:32.355 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45206.log.gz 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43501.log 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38586.log: /var/log/ceph/ceph-mon.a.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38586.log.gz 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35044.log 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58393.log 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43501.log: /var/log/ceph/ceph-client.admin.35044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39677.log 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43501.log.gz 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45337.log 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58393.log: -- replaced with /var/log/ceph/ceph-client.admin.35044.log.gz 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42708.log 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39677.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58393.log.gz 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39677.log.gz 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45337.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45337.log.gz 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48418.log 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37174.log 2024-04-25T00:42:32.356 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50404.log 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48418.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42708.log.gz 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50456.log 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48418.log.gz 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37174.log.gz 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51522.log 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50404.log: /var/log/ceph/ceph-client.admin.50456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46542.log 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50404.log.gz 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40705.log 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51522.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50456.log.gz 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51522.log.gz 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46542.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46542.log.gz 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57363.log 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34042.log 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33026.log 2024-04-25T00:42:32.357 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57363.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40705.log.gz 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34042.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49135.log 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57363.log.gz 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33026.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43276.log 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34042.log.gz 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33026.log.gz 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35174.log 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33250.log 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49135.log.gz 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43276.log: /var/log/ceph/ceph-client.admin.35174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38482.log 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43276.log.gz 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35174.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53199.log 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33250.log: /var/log/ceph/ceph-client.admin.38482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48128.log 2024-04-25T00:42:32.358 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33250.log.gz 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5/var/log/ceph/ceph-client.admin.53199.log: --verbose -- /var/log/ceph/ceph-client.admin.55143.log 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38482.log.gz 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.53199.log.gz 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48128.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48128.log.gz 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37793.log 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55630.log 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37793.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55143.log.gz 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36024.log 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55630.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37793.log.gz 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42630.log 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55630.log.gz 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51697.log 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36024.log: /var/log/ceph/ceph-client.admin.42630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60202.log 2024-04-25T00:42:32.359 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36024.log.gz 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51697.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42630.log.gz 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35735.log 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51697.log.gz 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54812.log 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60202.log: /var/log/ceph/ceph-client.admin.35735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50270.log 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9%/var/log/ceph/ceph-client.0.54812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39918.log 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60202.log.gz 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35735.log.gz 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50270.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50270.log.gz 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52787.log 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45180.log 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52787.log: /var/log/ceph/ceph-client.admin.43419.log 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39918.log.gz 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr: 86.1%/var/log/ceph/ceph-client.admin.45180.log: -- replaced with /var/log/ceph/ceph-client.admin.52787.log.gz 2024-04-25T00:42:32.360 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45180.log.gz 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55604.log 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54575.log 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55604.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43419.log.gz 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42760.log 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55604.log.gz 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54575.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54575.log.gz 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47404.log 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49839.log 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41573.log 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47404.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42760.log.gz 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47404.log.gz 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49839.log: 92.5% -- replaced with /var/log/ceph/ceph-client.0.54812.log.gz 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49839.log.gz 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54441.log 2024-04-25T00:42:32.361 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47681.log 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40361.log 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41573.log.gz 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51912.log 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54441.log.gz 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47681.log.gz 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40361.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40361.log.gz 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49611.log 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38875.log 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51912.log: /var/log/ceph/ceph-client.admin.49611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43579.log 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51912.log.gz 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34648.log 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49611.log.gz 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50100.log 2024-04-25T00:42:32.362 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3%/var/log/ceph/ceph-client.admin.43579.log: -- replaced with /var/log/ceph/ceph-client.admin.38875.log.gz 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34648.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43579.log.gz 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34648.log.gz 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33249.log 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47759.log 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37466.log 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50100.log.gz 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39308.log 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33249.log.gz 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50048.log 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47759.log.gz 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7%/var/log/ceph/ceph-client.admin.39308.log: -- replaced with /var/log/ceph/ceph-client.admin.37466.log.gz 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52317.log 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr:gzip 89.0% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.39308.log.gz /var/log/ceph/ceph-client.admin.41547.log 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.363 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50048.log: /var/log/ceph/ceph-client.admin.52317.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54148.log 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50048.log.gz 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41547.log: gzip -5 --verbose 88.8% -- /var/log/ceph/ceph-client.admin.45057.log -- replaced with /var/log/ceph/ceph-client.admin.52317.log.gz 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48808.log 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41547.log.gz 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54148.log.gz 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45057.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45057.log.gz 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52987.log 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55987.log 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48808.log.gz 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51879.log 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47707.log 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52987.log.gz 2024-04-25T00:42:32.364 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34094.log 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55987.log.gz 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56474.log 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51879.log.gz 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47707.log.gz 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40485.log 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44104.log 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34094.log.gz 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56474.log: /var/log/ceph/ceph-client.admin.40485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35122.log 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56474.log.gz 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44104.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40485.log.gz 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40048.log 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35122.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38018.log 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44104.log.gz 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40048.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35122.log.gz 2024-04-25T00:42:32.365 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46297.log 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr:gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40048.log.gz -5 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33449.log 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38018.log: /var/log/ceph/ceph-client.admin.46297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44398.log 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33449.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38018.log.gz 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45877.log 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46297.log.gz 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60020.log 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33449.log.gz 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44398.log: /var/log/ceph/ceph-client.admin.45877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39572.log 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45877.log.gz 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr:gzip/var/log/ceph/ceph-client.admin.60020.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44398.log.gz 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39572.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60020.log.gz 2024-04-25T00:42:32.366 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37542.log 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39572.log.gz 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48075.log 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37542.log: gzip 88.3% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr: /var/log/ceph/ceph-client.admin.49956.log 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37542.log.gz 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.48075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33156.log 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40576.log 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49956.log: /var/log/ceph/ceph-client.admin.33156.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42883.log 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49956.log.gz 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33156.log.gz 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40576.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40576.log.gz 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58841.log 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33672.log 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58841.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42883.log.gz 2024-04-25T00:42:32.367 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43250.log 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48629.log 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58841.log.gz/var/log/ceph/ceph-client.admin.33672.log: 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43250.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33672.log.gz 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43250.log.gz 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42170.log 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47020.log 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48629.log: /var/log/ceph/ceph-client.admin.42170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46516.log 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47020.log: 88.5% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48629.log.gz -- replaced with /var/log/ceph/ceph-client.admin.42170.log.gz 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49244.log 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47020.log.gz 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49787.log 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4%/var/log/ceph/ceph-client.admin.49244.log: -- replaced with /var/log/ceph/ceph-client.admin.46516.log.gz 2024-04-25T00:42:32.368 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46323.log 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49244.log.gz 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49787.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49787.log.gz 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37987.log 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53544.log 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37987.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46219.log 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46323.log.gz 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37987.log.gz 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53544.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53544.log.gz 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53651.log 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42025.log 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4%/var/log/ceph/ceph-client.admin.53651.log: -- replaced with /var/log/ceph/ceph-client.admin.46219.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37883.log 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53651.log.gz 2024-04-25T00:42:32.369 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42025.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42025.log.gz 2024-04-25T00:42:32.418 INFO:teuthology.orchestra.run.smithi157.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T00:42:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59508.log 2024-04-25T00:42:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32633.log 2024-04-25T00:42:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46625.log 2024-04-25T00:42:32.420 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37883.log.gz 2024-04-25T00:42:32.420 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59508.log.gz 2024-04-25T00:42:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32633.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32633.log.gz 2024-04-25T00:42:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34622.log 2024-04-25T00:42:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34305.log 2024-04-25T00:42:32.420 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4%/var/log/ceph/ceph-client.admin.34622.log: -- replaced with /var/log/ceph/ceph-client.admin.46625.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50875.log 2024-04-25T00:42:32.420 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.420 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34622.log.gz 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34305.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34305.log.gz 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44157.log 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34706.log 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44157.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38992.log 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50875.log.gz 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44157.log.gz 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34706.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34706.log.gz 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47811.log 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58233.log 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47811.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53491.log 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38992.log.gz 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47811.log.gz 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58233.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58233.log.gz 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56870.log 2024-04-25T00:42:32.421 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51098.log 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56870.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53491.log.gz -5 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49377.log 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56870.log.gz 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51098.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51098.log.gz 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40809.log 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48468.log 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40809.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49377.log.gz 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39625.log 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44320.log 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40809.log.gz 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39625.log: /var/log/ceph/ceph-client.admin.48468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40757.log 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3%/var/log/ceph/ceph-client.admin.44320.log: -- replaced with /var/log/ceph/ceph-client.admin.39625.log.gz 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48468.log.gz 2024-04-25T00:42:32.422 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44320.log.gz 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39970.log 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46888.log 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40757.log: /var/log/ceph/ceph-client.admin.39970.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47273.log 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40757.log.gz 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39970.log.gz 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46888.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46888.log.gz 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34862.log 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59326.log 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4%/var/log/ceph/ceph-client.admin.34862.log: -- replaced with /var/log/ceph/ceph-client.admin.47273.log.gz 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39835.log 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34862.log.gz 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59326.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59326.log.gz 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51652.log 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59136.log 2024-04-25T00:42:32.423 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0%/var/log/ceph/ceph-client.admin.51652.log: -- replaced with /var/log/ceph/ceph-client.admin.39835.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56222.log 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51652.log.gz 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59136.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59136.log.gz 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46109.log 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36237.log 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46109.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56222.log.gz 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45232.log 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46109.log.gz -5 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56313.log 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36237.log: /var/log/ceph/ceph-client.admin.45232.log: 89.1% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36237.log.gz -- replaced with /var/log/ceph/ceph-client.admin.45232.log.gz 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57801.log 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44946.log 2024-04-25T00:42:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57801.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56313.log.gz 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51009.log 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55500.log 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57801.log.gz 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44946.log: /var/log/ceph/ceph-client.admin.51009.log: 85.1% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44946.log.gz -- replaced with /var/log/ceph/ceph-client.admin.51009.log.gz 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45626.log 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43224.log 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8%/var/log/ceph/ceph-client.admin.45626.log: -- replaced with /var/log/ceph/ceph-client.admin.55500.log.gz 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56196.log 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45626.log.gz 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43224.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43224.log.gz 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50532.log 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40965.log 2024-04-25T00:42:32.425 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56196.log.gz 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42418.log 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33659.log 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50532.log.gz 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40965.log: /var/log/ceph/ceph-client.admin.42418.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40965.log.gz 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42418.log.gz 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57600.log 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56969.log 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3%/var/log/ceph/ceph-client.admin.57600.log: -- replaced with /var/log/ceph/ceph-client.admin.33659.log.gzgzip 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55656.log 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57600.log.gz 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56969.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56969.log.gz 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38795.log 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52144.log 2024-04-25T00:42:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38795.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55656.log.gz 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56020.log 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52144.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38795.log.gz 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52144.log.gz 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43683.log 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.3.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56020.log.gz 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44235.log 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33469.log 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43683.log: /var/log/ceph/ceph-client.admin.44235.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43683.log.gz 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44235.log.gz 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40861.log 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37148.log 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33469.log: /var/log/ceph/ceph-client.admin.40861.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33469.log.gz 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40861.log.gz 2024-04-25T00:42:32.427 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41761.log 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57124.log 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37148.log: /var/log/ceph/ceph-client.admin.41761.log: 88.7% 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41761.log.gz -- replaced with /var/log/ceph/ceph-client.admin.37148.log.gz 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51176.log 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47536.log 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51176.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57124.log.gz 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51176.log.gz 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58815.log 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58945.log 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58815.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47536.log.gz 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58815.log.gz 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52172.log 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33000.log 2024-04-25T00:42:32.428 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52172.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58945.log.gz 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52172.log.gz 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42682.log 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37492.log 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42682.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33000.log.gz 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42682.log.gz 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40913.log 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47430.log 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40913.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40913.log.gz 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.37492.log.gz 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52369.log 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36107.log 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47430.log: /var/log/ceph/ceph-client.admin.52369.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47430.log.gz 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52369.log.gz 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57150.log 2024-04-25T00:42:32.429 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52523.log 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9%/var/log/ceph/ceph-client.admin.57150.log: -- replaced with /var/log/ceph/ceph-client.admin.36107.log.gz 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57150.log.gz 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39018.log 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59968.log 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52523.log.gz 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39018.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39018.log.gz 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55341.log 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47221.log 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4%/var/log/ceph/ceph-client.admin.55341.log: -- replaced with /var/log/ceph/ceph-client.admin.59968.log.gz 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55341.log.gz 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59110.log 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39178.log 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59110.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47221.log.gz 2024-04-25T00:42:32.430 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59110.log.gz 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33671.log 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42335.log 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39178.log.gz/var/log/ceph/ceph-client.admin.33671.log: 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33671.log.gz 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41683.log 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48599.log 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7%/var/log/ceph/ceph-client.admin.41683.log: -- replaced with /var/log/ceph/ceph-client.admin.42335.log.gz 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41683.log.gz 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53755.log 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41973.log 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9%/var/log/ceph/ceph-client.admin.53755.log: -- replaced with /var/log/ceph/ceph-client.admin.48599.log.gz 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53755.log.gz 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36672.log 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55857.log 2024-04-25T00:42:32.431 INFO:teuthology.orchestra.run.smithi029.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.48075.log.gz 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41973.log.gz 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36672.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36672.log.gz 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54653.log 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48366.log 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7%/var/log/ceph/ceph-client.admin.54653.log: -- replaced with /var/log/ceph/ceph-client.admin.55857.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32487.log 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54653.log.gz 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48366.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48366.log.gz 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40679.log 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42445.log 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40679.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32487.log.gz 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35496.log 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58916.log 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40679.log.gz 2024-04-25T00:42:32.432 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42445.log: /var/log/ceph/ceph-client.admin.35496.log: 89.5% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35496.log.gz 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42445.log.gz 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45493.log 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45310.log 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45493.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58916.log.gz 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40100.log 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44640.log 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45493.log.gz/var/log/ceph/ceph-client.admin.45310.log: 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40100.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45310.log.gz 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40100.log.gz 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35311.log 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39944.log 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35311.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44640.log.gz 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47091.log 2024-04-25T00:42:32.433 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39944.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35311.log.gz 2024-04-25T00:42:32.434 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39944.log.gz 2024-04-25T00:42:32.434 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40300.log 2024-04-25T00:42:32.434 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60124.log 2024-04-25T00:42:32.434 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40300.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47091.log.gz 2024-04-25T00:42:32.434 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54245.log 2024-04-25T00:42:32.434 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43330.log 2024-04-25T00:42:32.434 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40300.log.gz 2024-04-25T00:42:32.434 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60124.log: /var/log/ceph/ceph-client.admin.54245.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60124.log.gz 2024-04-25T00:42:32.434 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54245.log.gz 2024-04-25T00:42:32.434 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51723.log 2024-04-25T00:42:32.434 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43761.log 2024-04-25T00:42:32.434 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2%/var/log/ceph/ceph-client.admin.51723.log: -- replaced with /var/log/ceph/ceph-client.admin.43330.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46942.log 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51723.log.gz 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43761.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43761.log.gz 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33448.log 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56682.log 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33448.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46942.log.gz 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48544.log 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42089.log 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3%/var/log/ceph/ceph-client.admin.56682.log: -- replaced with /var/log/ceph/ceph-client.admin.33448.log.gz 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48544.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48544.log.gz 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.56682.log.gz 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50154.log 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35683.log 2024-04-25T00:42:32.435 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49663.log 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42089.log.gz 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50154.log.gz 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35683.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35683.log.gz 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46784.log 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51259.log 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46784.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49663.log.gz 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60098.log 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53677.log 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46784.log.gz 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51259.log: /var/log/ceph/ceph-client.admin.60098.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51259.log.gz 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60098.log.gz 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58708.log 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37440.log 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8%/var/log/ceph/ceph-client.0.58708.log: -- replaced with /var/log/ceph/ceph-client.admin.53677.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37704.log 2024-04-25T00:42:32.436 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37440.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37440.log.gz 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59219.log 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41228.log 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59219.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37704.log.gz 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59219.log.gz 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50300.log 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52036.log 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50300.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41228.log.gz 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50300.log.gz 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58652.log 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45851.log 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52036.log.gz 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58652.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58652.log.gz 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51150.log 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47338.log 2024-04-25T00:42:32.437 INFO:teuthology.orchestra.run.smithi029.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.58708.log.gz 2024-04-25T00:42:32.438 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6%/var/log/ceph/ceph-client.admin.51150.log: -- replaced with /var/log/ceph/ceph-client.admin.45851.log.gz 2024-04-25T00:42:32.438 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51150.log.gz 2024-04-25T00:42:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56339.log 2024-04-25T00:42:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41140.log 2024-04-25T00:42:32.438 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2%/var/log/ceph/ceph-client.admin.56339.log: -- replaced with /var/log/ceph/ceph-client.admin.47338.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41486.log 2024-04-25T00:42:32.438 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.438 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56339.log.gz 2024-04-25T00:42:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41140.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41140.log.gz 2024-04-25T00:42:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33465.log 2024-04-25T00:42:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57749.log 2024-04-25T00:42:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41486.log: /var/log/ceph/ceph-client.admin.33465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53434.log 2024-04-25T00:42:32.438 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57749.log: 87.7% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41486.log.gz 2024-04-25T00:42:32.438 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33465.log.gz 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55093.log 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57749.log.gz 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49322.log 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55093.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53434.log.gz 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39074.log 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33990.log 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55093.log.gz 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49322.log: /var/log/ceph/ceph-client.admin.39074.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49322.log.gz 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39074.log.gz 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54089.log 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33658.log 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33990.log.gz 2024-04-25T00:42:32.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54089.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54089.log.gz 2024-04-25T00:42:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46165.log 2024-04-25T00:42:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38307.log 2024-04-25T00:42:32.458 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33658.log: /var/log/ceph/ceph-client.0.46165.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33658.log.gz 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38096.log 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38307.log: 90.6% -- replaced with /var/log/ceph/ceph-client.0.46165.log.gz 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38307.log.gz 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39256.log 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38096.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38096.log.gz 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42786.log 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42552.log 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39256.log: /var/log/ceph/ceph-client.admin.42786.log: 88.4% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42786.log.gz 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39256.log.gz 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46810.log 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41351.log 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42552.log: /var/log/ceph/ceph-client.admin.46810.log: 88.3% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46810.log.gz 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42552.log.gz 2024-04-25T00:42:32.459 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49072.log 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51853.log 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49072.log: /var/log/ceph/ceph-client.admin.41351.log: 87.8% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49072.log.gz 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41351.log.gz 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36393.log 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54406.log 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36393.log: /var/log/ceph/ceph-client.admin.51853.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51853.log.gz 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36393.log.gz 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57392.log 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52685.log 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57392.log: /var/log/ceph/ceph-client.admin.54406.log: 89.3% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54406.log.gz 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57392.log.gz 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35998.log 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38333.log 2024-04-25T00:42:32.460 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52685.log: /var/log/ceph/ceph-client.admin.35998.log: 89.1% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35998.log.gz 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52685.log.gz 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40425.log 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59245.log 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38333.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38333.log.gz 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41010.log 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59245.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59245.log.gz 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49046.log 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41010.log: 88.1%/var/log/ceph/ceph-client.admin.40425.log: -- replaced with /var/log/ceph/ceph-client.admin.41010.log.gz 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48392.log 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40425.log.gz 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49046.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49046.log.gz 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50378.log 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42604.log 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48392.log: /var/log/ceph/ceph-client.admin.50378.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48392.log.gz 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50378.log.gz 2024-04-25T00:42:32.461 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43172.log 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53224.log 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42604.log: /var/log/ceph/ceph-client.admin.43172.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42604.log.gz 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43172.log.gz 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60332.log 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49924.log 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53224.log: /var/log/ceph/ceph-client.admin.60332.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.53224.log.gz 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60332.log.gz 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55961.log 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32974.log 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49924.log: /var/log/ceph/ceph-client.admin.55961.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49924.log.gz 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55961.log.gz 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55310.log 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36795.log 2024-04-25T00:42:32.462 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32974.log: /var/log/ceph/ceph-client.admin.55310.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32974.log.gz 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55310.log.gz 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57202.log 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43106.log 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36795.log: /var/log/ceph/ceph-client.admin.57202.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36795.log.gz 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57202.log.gz 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55172.log 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43393.log 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43106.log: /var/log/ceph/ceph-client.admin.55172.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55172.log.gz 89.7% -- replaced with /var/log/ceph/ceph-client.admin.43106.log.gz 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49533.log 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46703.log 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43393.log: /var/log/ceph/ceph-client.admin.49533.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43393.log.gz 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49533.log.gz 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58789.log 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36289.log 2024-04-25T00:42:32.463 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58789.log: /var/log/ceph/ceph-client.admin.46703.log: 88.5% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58789.log.gz 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46703.log.gz 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55419.log 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38697.log 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36289.log.gz 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55419.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55419.log.gz 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53859.log 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36471.log 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38697.log: /var/log/ceph/ceph-client.admin.53859.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38697.log.gz 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53859.log.gz 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54009.log 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44916.log 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36471.log: /var/log/ceph/ceph-client.admin.54009.log: 89.0% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54009.log.gz 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36471.log.gz 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59942.log 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32520.log 2024-04-25T00:42:32.464 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44916.log: /var/log/ceph/ceph-client.admin.59942.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59942.log.gz 2024-04-25T00:42:32.465 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44916.log.gz 2024-04-25T00:42:32.465 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35096.log 2024-04-25T00:42:32.465 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52912.log 2024-04-25T00:42:32.465 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32520.log: /var/log/ceph/ceph-client.admin.35096.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32520.log.gz 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35096.log.gz 2024-04-25T00:42:32.465 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.465 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59657.log 2024-04-25T00:42:32.465 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49455.log 2024-04-25T00:42:32.474 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59657.log: /var/log/ceph/ceph-client.admin.52912.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52912.log.gz 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59657.log.gz 2024-04-25T00:42:32.474 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.474 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39100.log 2024-04-25T00:42:32.474 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45109.log 2024-04-25T00:42:32.474 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49455.log: /var/log/ceph/ceph-client.admin.39100.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39100.log.gz 2024-04-25T00:42:32.474 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49455.log.gz 2024-04-25T00:42:32.474 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59352.log 2024-04-25T00:42:32.474 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36211.log 2024-04-25T00:42:32.474 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45109.log: /var/log/ceph/ceph-client.admin.59352.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45109.log.gz 2024-04-25T00:42:32.474 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59352.log.gz 2024-04-25T00:42:32.474 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56500.log 2024-04-25T00:42:32.474 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54627.log 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56500.log: /var/log/ceph/ceph-client.admin.36211.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36211.log.gz 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56500.log.gz 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36133.log 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33467.log 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54627.log: /var/log/ceph/ceph-client.admin.36133.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54627.log.gz 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36133.log.gz 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54971.log 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39783.log 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33467.log: /var/log/ceph/ceph-client.admin.54971.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33467.log.gz 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54971.log.gz 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52737.log 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54271.log 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39783.log: /var/log/ceph/ceph-client.admin.52737.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39783.log.gz 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52737.log.gz 2024-04-25T00:42:32.475 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36159.log 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51285.log 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54271.log: /var/log/ceph/ceph-client.admin.36159.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54271.log.gz 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36159.log.gz 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54886.log 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44372.log 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51285.log: /var/log/ceph/ceph-client.admin.54886.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51285.log.gz 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54886.log.gz 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47733.log 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33860.log 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44372.log: /var/log/ceph/ceph-client.admin.47733.log: 88.6% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47733.log.gz -- replaced with /var/log/ceph/ceph-client.admin.44372.log.gz 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53807.log 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59586.log 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33860.log: /var/log/ceph/ceph-client.admin.53807.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33860.log.gz 2024-04-25T00:42:32.476 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53807.log.gz 2024-04-25T00:42:32.477 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46410.log 2024-04-25T00:42:32.477 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54549.log 2024-04-25T00:42:32.480 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59586.log: /var/log/ceph/ceph-client.admin.46410.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59586.log.gz 2024-04-25T00:42:32.480 INFO:teuthology.orchestra.run.smithi029.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.46410.log.gz 2024-04-25T00:42:32.480 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47564.log 2024-04-25T00:42:32.480 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59916.log 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54549.log: /var/log/ceph/ceph-client.admin.47564.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54549.log.gz 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47564.log.gz 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44424.log 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43475.log 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59916.log: /var/log/ceph/ceph-client.admin.44424.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59916.log.gz 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44424.log.gz 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49109.log 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33666.log 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43475.log: /var/log/ceph/ceph-client.admin.49109.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43475.log.gz 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49109.log.gz 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52887.log 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.33666.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33666.log.gz 2024-04-25T00:42:32.481 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52062.log 2024-04-25T00:42:32.483 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52887.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.52887.log.gz 2024-04-25T00:42:32.483 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33450.log 2024-04-25T00:42:32.483 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52062.log: 88.5% 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T00:42:32.483 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52062.log.gz 2024-04-25T00:42:32.483 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35789.log 2024-04-25T00:42:32.483 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34732.log 2024-04-25T00:42:32.484 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33450.log: /var/log/ceph/ceph-client.admin.35789.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35789.log.gz 2024-04-25T00:42:32.484 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33450.log.gz 2024-04-25T00:42:32.484 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42283.log 2024-04-25T00:42:32.484 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32839.log 2024-04-25T00:42:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34732.log: /var/log/ceph/ceph-client.admin.42283.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34732.log.gz 2024-04-25T00:42:32.486 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42283.log.gz 2024-04-25T00:42:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52343.log 2024-04-25T00:42:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36926.log 2024-04-25T00:42:32.487 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32839.log: /var/log/ceph/ceph-client.admin.52343.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32839.log.gz 2024-04-25T00:42:32.487 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52343.log.gz 2024-04-25T00:42:32.487 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58198.log 2024-04-25T00:42:32.487 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60308.log 2024-04-25T00:42:32.488 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36926.log: /var/log/ceph/ceph-client.admin.58198.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36926.log.gz 2024-04-25T00:42:32.488 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58198.log.gz 2024-04-25T00:42:32.488 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32664.log 2024-04-25T00:42:32.488 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41657.log 2024-04-25T00:42:32.489 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60308.log: /var/log/ceph/ceph-client.admin.32664.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60308.log.gz 2024-04-25T00:42:32.489 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32664.log.gz 2024-04-25T00:42:32.489 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59271.log 2024-04-25T00:42:32.490 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37004.log 2024-04-25T00:42:32.490 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41657.log: /var/log/ceph/ceph-client.admin.59271.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41657.log.gz 2024-04-25T00:42:32.490 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59271.log.gz 2024-04-25T00:42:32.491 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46758.log 2024-04-25T00:42:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52762.log 2024-04-25T00:42:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37004.log: /var/log/ceph/ceph-client.admin.46758.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37004.log.gz 2024-04-25T00:42:32.492 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46758.log.gz 2024-04-25T00:42:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33104.log 2024-04-25T00:42:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33260.log 2024-04-25T00:42:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52762.log: /var/log/ceph/ceph-client.admin.33104.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52762.log.gz 2024-04-25T00:42:32.493 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33104.log.gz 2024-04-25T00:42:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49213.log 2024-04-25T00:42:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33260.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33260.log.gz 2024-04-25T00:42:32.494 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33246.log 2024-04-25T00:42:32.494 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40731.log 2024-04-25T00:42:32.495 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49213.log: /var/log/ceph/ceph-client.admin.33246.log: 89.2% 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33246.log.gz 2024-04-25T00:42:32.495 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49213.log.gz 2024-04-25T00:42:32.495 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58576.log 2024-04-25T00:42:32.496 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53249.log 2024-04-25T00:42:32.497 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40731.log: /var/log/ceph/ceph-client.admin.58576.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58576.log.gz 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40731.log.gz 2024-04-25T00:42:32.497 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.497 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35579.log 2024-04-25T00:42:32.498 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53518.log 2024-04-25T00:42:32.498 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53249.log: /var/log/ceph/ceph-client.admin.35579.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.53249.log.gz 2024-04-25T00:42:32.498 INFO:teuthology.orchestra.run.smithi029.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35579.log.gz 2024-04-25T00:42:32.498 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53356.log 2024-04-25T00:42:32.499 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47169.log 2024-04-25T00:42:32.499 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53518.log: /var/log/ceph/ceph-client.admin.53356.log: 89.0% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53356.log.gz 2024-04-25T00:42:32.499 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53518.log.gz 2024-04-25T00:42:32.500 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54115.log 2024-04-25T00:42:32.500 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34758.log 2024-04-25T00:42:32.501 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47169.log: /var/log/ceph/ceph-client.admin.54115.log: 88.5% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54115.log.gz 2024-04-25T00:42:32.501 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47169.log.gz 2024-04-25T00:42:32.501 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46651.log 2024-04-25T00:42:32.502 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51548.log 2024-04-25T00:42:32.502 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34758.log: /var/log/ceph/ceph-client.admin.46651.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46651.log.gz 2024-04-25T00:42:32.502 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34758.log.gz 2024-04-25T00:42:32.503 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42812.log 2024-04-25T00:42:32.503 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52579.log 2024-04-25T00:42:32.504 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42812.log: /var/log/ceph/ceph-client.admin.51548.log: 88.6% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51548.log.gz 2024-04-25T00:42:32.504 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42812.log.gz 2024-04-25T00:42:32.504 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33964.log 2024-04-25T00:42:32.505 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57723.log 2024-04-25T00:42:32.505 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.52579.log: /var/log/ceph/ceph-client.admin.33964.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33964.log.gz 2024-04-25T00:42:32.505 INFO:teuthology.orchestra.run.smithi029.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.0.52579.log.gz 2024-04-25T00:42:32.506 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53930.log 2024-04-25T00:42:32.506 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46007.log 2024-04-25T00:42:32.507 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57723.log: /var/log/ceph/ceph-client.admin.53930.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57723.log.gz 2024-04-25T00:42:32.507 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53930.log.gz 2024-04-25T00:42:32.507 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59058.log 2024-04-25T00:42:32.508 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39996.log 2024-04-25T00:42:32.508 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46007.log: /var/log/ceph/ceph-client.admin.59058.log: 89.5% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46007.log.gz 2024-04-25T00:42:32.508 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59058.log.gz 2024-04-25T00:42:32.508 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58090.log 2024-04-25T00:42:32.509 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57285.log 2024-04-25T00:42:32.510 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39996.log: /var/log/ceph/ceph-client.admin.58090.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39996.log.gz 2024-04-25T00:42:32.510 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58090.log.gz 2024-04-25T00:42:32.510 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49891.log 2024-04-25T00:42:32.510 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33182.log 2024-04-25T00:42:32.511 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57285.log: /var/log/ceph/ceph-client.admin.49891.log: 89.1% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57285.log.gz 2024-04-25T00:42:32.511 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49891.log.gz 2024-04-25T00:42:32.511 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38281.log 2024-04-25T00:42:32.512 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51470.log 2024-04-25T00:42:32.512 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33182.log: /var/log/ceph/ceph-client.admin.38281.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33182.log.gz 2024-04-25T00:42:32.512 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38281.log.gz 2024-04-25T00:42:32.513 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34198.log 2024-04-25T00:42:32.513 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51311.log 2024-04-25T00:42:32.514 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51470.log: /var/log/ceph/ceph-client.admin.34198.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51470.log.gz 2024-04-25T00:42:32.514 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34198.log.gz 2024-04-25T00:42:32.514 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33660.log 2024-04-25T00:42:32.514 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55445.log 2024-04-25T00:42:32.515 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51311.log: /var/log/ceph/ceph-client.admin.33660.log: 88.8% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51311.log.gz 2024-04-25T00:42:32.515 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33660.log.gz 2024-04-25T00:42:32.515 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58550.log 2024-04-25T00:42:32.516 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57418.log 2024-04-25T00:42:32.516 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55445.log: /var/log/ceph/ceph-client.admin.58550.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55445.log.gz 2024-04-25T00:42:32.516 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58550.log.gz 2024-04-25T00:42:32.516 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56526.log 2024-04-25T00:42:32.518 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56552.log 2024-04-25T00:42:32.518 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57418.log: /var/log/ceph/ceph-client.admin.56526.log: 89.1% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56526.log.gz 2024-04-25T00:42:32.518 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57418.log.gz 2024-04-25T00:42:32.518 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54756.log 2024-04-25T00:42:32.519 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50430.log 2024-04-25T00:42:32.519 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56552.log: /var/log/ceph/ceph-client.admin.54756.log: 87.8% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54756.log.gz -- replaced with /var/log/ceph/ceph-client.admin.56552.log.gz 2024-04-25T00:42:32.519 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.519 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47055.log 2024-04-25T00:42:32.520 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54378.log 2024-04-25T00:42:32.520 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50430.log: /var/log/ceph/ceph-client.admin.47055.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50430.log.gz 2024-04-25T00:42:32.520 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47055.log.gz 2024-04-25T00:42:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42115.log 2024-04-25T00:42:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52862.log 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54378.log: /var/log/ceph/ceph-client.admin.42115.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42115.log.gz 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54378.log.gz 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50983.log 2024-04-25T00:42:32.522 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37935.log 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52862.log: /var/log/ceph/ceph-client.admin.50983.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.52862.log.gz 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50983.log.gz 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58142.log 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32696.log 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58142.log: /var/log/ceph/ceph-client.admin.37935.log: 87.7% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37935.log.gz 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58142.log.gz 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34966.log 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57470.log 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32696.log: /var/log/ceph/ceph-client.admin.34966.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32696.log.gz 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34966.log.gz 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58471.log 2024-04-25T00:42:32.528 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58445.log 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57470.log: /var/log/ceph/ceph-client.admin.58471.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57470.log.gz 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58471.log.gz 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55256.log 2024-04-25T00:42:32.529 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37308.log 2024-04-25T00:42:32.531 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58445.log: /var/log/ceph/ceph-client.admin.55256.log: 88.8% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58445.log.gz 2024-04-25T00:42:32.531 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55256.log.gz 2024-04-25T00:42:32.531 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54324.log 2024-04-25T00:42:32.531 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49403.log 2024-04-25T00:42:32.531 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37308.log: /var/log/ceph/ceph-client.admin.54324.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37308.log.gz 2024-04-25T00:42:32.532 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54324.log.gz 2024-04-25T00:42:32.532 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39282.log 2024-04-25T00:42:32.532 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58064.log 2024-04-25T00:42:32.532 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49403.log: /var/log/ceph/ceph-client.admin.39282.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49403.log.gz 2024-04-25T00:42:32.532 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39282.log.gz 2024-04-25T00:42:32.532 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40608.log 2024-04-25T00:42:32.534 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42937.log 2024-04-25T00:42:32.534 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58064.log: /var/log/ceph/ceph-client.admin.40608.log: 88.0% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58064.log.gz 2024-04-25T00:42:32.534 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40608.log.gz 2024-04-25T00:42:32.534 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54493.log 2024-04-25T00:42:32.534 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51626.log 2024-04-25T00:42:32.534 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42937.log: /var/log/ceph/ceph-client.admin.54493.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42937.log.gz 2024-04-25T00:42:32.534 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54493.log.gz 2024-04-25T00:42:32.534 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58602.log 2024-04-25T00:42:32.535 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51626.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51626.log.gz 2024-04-25T00:42:32.535 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59456.log 2024-04-25T00:42:32.535 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33078.log 2024-04-25T00:42:32.536 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58602.log: /var/log/ceph/ceph-client.admin.59456.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59456.log.gz 2024-04-25T00:42:32.536 INFO:teuthology.orchestra.run.smithi029.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58602.log.gz 2024-04-25T00:42:32.536 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49865.log 2024-04-25T00:42:32.537 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46386.log 2024-04-25T00:42:32.537 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33078.log: /var/log/ceph/ceph-client.admin.49865.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33078.log.gz 2024-04-25T00:42:32.538 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49865.log.gz 2024-04-25T00:42:32.538 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56052.log 2024-04-25T00:42:32.539 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41521.log 2024-04-25T00:42:32.539 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46386.log: /var/log/ceph/ceph-client.admin.56052.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46386.log.gz 2024-04-25T00:42:32.539 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56052.log.gz 2024-04-25T00:42:32.539 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56365.log 2024-04-25T00:42:32.540 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38151.log 2024-04-25T00:42:32.540 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41521.log: /var/log/ceph/ceph-client.admin.56365.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41521.log.gz 2024-04-25T00:42:32.540 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56365.log.gz 2024-04-25T00:42:32.540 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60150.log 2024-04-25T00:42:32.541 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42989.log 2024-04-25T00:42:32.541 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38151.log: /var/log/ceph/ceph-client.admin.60150.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38151.log.gz 2024-04-25T00:42:32.541 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60150.log.gz 2024-04-25T00:42:32.541 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51496.log 2024-04-25T00:42:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44183.log 2024-04-25T00:42:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42989.log: /var/log/ceph/ceph-client.admin.51496.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42989.log.gz 2024-04-25T00:42:32.543 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51496.log.gz 2024-04-25T00:42:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58998.log 2024-04-25T00:42:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51444.log 2024-04-25T00:42:32.544 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44183.log: /var/log/ceph/ceph-client.admin.58998.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44183.log.gz 2024-04-25T00:42:32.544 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58998.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60046.log 2024-04-25T00:42:32.544 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:32.545 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51444.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51444.log.gz 2024-04-25T00:42:32.545 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57992.log 2024-04-25T00:42:32.546 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45689.log 2024-04-25T00:42:32.546 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60046.log: /var/log/ceph/ceph-client.admin.57992.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60046.log.gz 2024-04-25T00:42:32.546 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57992.log.gz 2024-04-25T00:42:32.546 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32921.log 2024-04-25T00:42:32.547 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34068.log 2024-04-25T00:42:32.547 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45689.log: /var/log/ceph/ceph-client.admin.32921.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45689.log.gz 2024-04-25T00:42:32.548 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32921.log.gz 2024-04-25T00:42:32.548 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35200.log 2024-04-25T00:42:32.548 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34068.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34068.log.gz 2024-04-25T00:42:32.549 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46059.log 2024-04-25T00:42:32.549 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45031.log 2024-04-25T00:42:32.549 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35200.log: /var/log/ceph/ceph-client.admin.46059.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35200.log.gz 2024-04-25T00:42:32.550 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38437.log 2024-04-25T00:42:32.550 INFO:teuthology.orchestra.run.smithi029.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46059.log.gz 2024-04-25T00:42:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45031.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45031.log.gz 2024-04-25T00:42:32.551 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22736.log 2024-04-25T00:42:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60176.log 2024-04-25T00:42:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38437.log: /var/log/ceph/ceph.tmp-client.admin.22736.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22736.log.gz 2024-04-25T00:42:32.552 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38437.log.gz 2024-04-25T00:42:32.552 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33886.log 2024-04-25T00:42:32.553 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54860.log 2024-04-25T00:42:32.554 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60176.log: /var/log/ceph/ceph-client.admin.33886.log: 88.5% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60176.log.gz 2024-04-25T00:42:32.554 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33886.log.gz 2024-04-25T00:42:32.554 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39598.log 2024-04-25T00:42:32.555 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39809.log 2024-04-25T00:42:32.555 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54860.log: /var/log/ceph/ceph-client.admin.39598.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54860.log.gz 2024-04-25T00:42:32.555 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39598.log.gz 2024-04-25T00:42:32.555 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51124.log 2024-04-25T00:42:32.556 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52711.log 2024-04-25T00:42:32.556 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39809.log: /var/log/ceph/ceph-client.admin.51124.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39809.log.gz 2024-04-25T00:42:32.556 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51124.log.gz 2024-04-25T00:42:32.557 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54035.log 2024-04-25T00:42:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59709.log 2024-04-25T00:42:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52711.log: /var/log/ceph/ceph-client.admin.54035.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52711.log.gz 2024-04-25T00:42:32.558 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54035.log.gz 2024-04-25T00:42:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39360.log 2024-04-25T00:42:32.558 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47195.log 2024-04-25T00:42:32.559 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59709.log: /var/log/ceph/ceph-client.admin.39360.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59709.log.gz 2024-04-25T00:42:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40270.log 2024-04-25T00:42:32.560 INFO:teuthology.orchestra.run.smithi029.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.39360.log.gz 2024-04-25T00:42:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47195.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47195.log.gz 2024-04-25T00:42:32.560 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34836.log 2024-04-25T00:42:32.561 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59300.log 2024-04-25T00:42:32.562 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40270.log: /var/log/ceph/ceph-client.admin.34836.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40270.log.gz 2024-04-25T00:42:32.562 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34836.log.gz 2024-04-25T00:42:32.562 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51366.log 2024-04-25T00:42:32.562 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38729.log 2024-04-25T00:42:32.563 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59300.log: /var/log/ceph/ceph-client.admin.51366.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59300.log.gz 2024-04-25T00:42:32.563 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51366.log.gz 2024-04-25T00:42:32.563 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52259.log 2024-04-25T00:42:32.563 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38929.log 2024-04-25T00:42:32.564 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38729.log: /var/log/ceph/ceph-client.admin.52259.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38729.log.gz 2024-04-25T00:42:32.564 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52259.log.gz 2024-04-25T00:42:32.564 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54063.log 2024-04-25T00:42:32.565 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49813.log 2024-04-25T00:42:32.566 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6%/var/log/ceph/ceph-client.admin.54063.log: -- replaced with /var/log/ceph/ceph-client.admin.38929.log.gz 2024-04-25T00:42:32.566 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54063.log.gz 2024-04-25T00:42:32.566 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32891.log 2024-04-25T00:42:32.567 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59404.log 2024-04-25T00:42:32.567 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49813.log: /var/log/ceph/ceph-client.admin.32891.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49813.log.gz 2024-04-25T00:42:32.567 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32891.log.gz 2024-04-25T00:42:32.567 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52659.log 2024-04-25T00:42:32.568 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59404.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59404.log.gz 2024-04-25T00:42:32.568 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55734.log 2024-04-25T00:42:32.568 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43735.log 2024-04-25T00:42:32.569 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52659.log: /var/log/ceph/ceph-client.admin.55734.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52659.log.gz 2024-04-25T00:42:32.569 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55734.log.gz 2024-04-25T00:42:32.569 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34224.log 2024-04-25T00:42:32.570 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51418.log 2024-04-25T00:42:32.571 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43735.log: /var/log/ceph/ceph-client.admin.34224.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43735.log.gz 2024-04-25T00:42:32.571 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34224.log.gz 2024-04-25T00:42:32.571 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34992.log 2024-04-25T00:42:32.571 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51418.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51418.log.gz 2024-04-25T00:42:32.572 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44450.log 2024-04-25T00:42:32.573 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57853.log 2024-04-25T00:42:32.573 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34992.log: /var/log/ceph/ceph-client.admin.44450.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34992.log.gz 2024-04-25T00:42:32.573 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44450.log.gz 2024-04-25T00:42:32.573 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53465.log 2024-04-25T00:42:32.574 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36769.log 2024-04-25T00:42:32.574 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57853.log: /var/log/ceph/ceph-client.admin.53465.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57853.log.gz 2024-04-25T00:42:32.574 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53465.log.gz 2024-04-25T00:42:32.574 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41999.log 2024-04-25T00:42:32.575 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36769.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36769.log.gz 2024-04-25T00:42:32.575 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35920.log 2024-04-25T00:42:32.576 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43973.log 2024-04-25T00:42:32.577 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41999.log: /var/log/ceph/ceph-client.admin.35920.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41999.log.gz 2024-04-25T00:42:32.577 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35920.log.gz 2024-04-25T00:42:32.577 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45741.log 2024-04-25T00:42:32.577 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41787.log 2024-04-25T00:42:32.578 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43973.log: /var/log/ceph/ceph-client.admin.45741.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43973.log.gz 2024-04-25T00:42:32.578 INFO:teuthology.orchestra.run.smithi029.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45741.log.gz 2024-04-25T00:42:32.578 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40238.log 2024-04-25T00:42:32.578 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48941.log 2024-04-25T00:42:32.579 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41787.log: /var/log/ceph/ceph-client.admin.40238.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40238.log.gz 2024-04-25T00:42:32.579 INFO:teuthology.orchestra.run.smithi029.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41787.log.gz 2024-04-25T00:42:32.580 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39704.log 2024-04-25T00:42:32.580 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49270.log 2024-04-25T00:42:32.581 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48941.log: /var/log/ceph/ceph-client.admin.39704.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48941.log.gz 2024-04-25T00:42:32.581 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39704.log.gz 2024-04-25T00:42:32.581 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41735.log 2024-04-25T00:42:32.581 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27588.log 2024-04-25T00:42:32.582 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49270.log: /var/log/ceph/ceph-client.admin.41735.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49270.log.gz 2024-04-25T00:42:32.582 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41735.log.gz 2024-04-25T00:42:32.582 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35363.log 2024-04-25T00:42:32.583 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45519.log 2024-04-25T00:42:32.584 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35363.log: /var/log/ceph/ceph-client.admin.27588.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35363.log.gz 2024-04-25T00:42:32.584 INFO:teuthology.orchestra.run.smithi029.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.27588.log.gz 2024-04-25T00:42:32.584 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49761.log 2024-04-25T00:42:32.585 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38255.log 2024-04-25T00:42:32.585 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45519.log: /var/log/ceph/ceph-client.admin.49761.log: 87.8% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45519.log.gz 2024-04-25T00:42:32.585 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49761.log.gz 2024-04-25T00:42:32.585 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47482.log 2024-04-25T00:42:32.586 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T00:42:32.587 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38255.log: /var/log/ceph/ceph-client.admin.47482.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38255.log.gz 2024-04-25T00:42:32.587 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47482.log.gz 2024-04-25T00:42:32.587 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57626.log 2024-04-25T00:42:32.587 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47143.log 2024-04-25T00:42:32.587 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.57626.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57626.log.gz 2024-04-25T00:42:32.591 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55230.log 2024-04-25T00:42:32.591 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47143.log: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T00:42:32.592 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47143.log.gz 2024-04-25T00:42:32.592 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53012.log 2024-04-25T00:42:32.592 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51574.log 2024-04-25T00:42:32.594 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53012.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55230.log.gz 2024-04-25T00:42:32.594 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53012.log.gz 2024-04-25T00:42:32.594 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T00:42:32.594 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41458.log 2024-04-25T00:42:32.594 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51574.log.gz 2024-04-25T00:42:32.595 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36900.log 2024-04-25T00:42:32.595 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41458.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41458.log.gz 2024-04-25T00:42:32.595 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56395.log 2024-04-25T00:42:32.596 INFO:teuthology.orchestra.run.smithi157.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T00:42:32.597 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36900.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36900.log.gz 2024-04-25T00:42:32.597 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47837.log 2024-04-25T00:42:32.598 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56395.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56395.log.gz 2024-04-25T00:42:32.598 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53729.log 2024-04-25T00:42:32.599 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47837.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47837.log.gz 2024-04-25T00:42:32.599 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33208.log 2024-04-25T00:42:32.600 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53729.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53729.log.gz 2024-04-25T00:42:32.601 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37122.log 2024-04-25T00:42:32.601 INFO:teuthology.orchestra.run.smithi157.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T00:42:32.602 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33208.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33208.log.gz 2024-04-25T00:42:32.602 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37030.log 2024-04-25T00:42:32.603 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37122.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37122.log.gz 2024-04-25T00:42:32.603 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50642.log 2024-04-25T00:42:32.604 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37030.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37030.log.gz 2024-04-25T00:42:32.604 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35259.log 2024-04-25T00:42:32.605 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50642.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50642.log.gz 2024-04-25T00:42:32.606 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57938.log 2024-04-25T00:42:32.606 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35259.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35259.log.gz 2024-04-25T00:42:32.607 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47508.log 2024-04-25T00:42:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57938.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57938.log.gz 2024-04-25T00:42:32.608 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50696.log 2024-04-25T00:42:32.609 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47508.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47508.log.gz 2024-04-25T00:42:32.609 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48248.log 2024-04-25T00:42:32.610 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50696.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50696.log.gz 2024-04-25T00:42:32.611 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T00:42:32.611 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48248.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48248.log.gz 2024-04-25T00:42:32.612 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41893.log 2024-04-25T00:42:32.631 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41325.log 2024-04-25T00:42:32.639 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.41893.log: 92.7% -- replaced with /var/log/ceph/ceph-client.0.41893.log.gz 2024-04-25T00:42:32.663 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44707.log 2024-04-25T00:42:32.664 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41325.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41325.log.gz 2024-04-25T00:42:32.671 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34016.log 2024-04-25T00:42:32.671 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44707.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44707.log.gz 2024-04-25T00:42:32.679 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48288.log 2024-04-25T00:42:32.679 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34016.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34016.log.gz 2024-04-25T00:42:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56117.log 2024-04-25T00:42:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48288.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48288.log.gz 2024-04-25T00:42:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38901.log 2024-04-25T00:42:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56117.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56117.log.gz 2024-04-25T00:42:32.707 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60254.log 2024-04-25T00:42:32.707 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38901.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38901.log.gz 2024-04-25T00:42:32.712 INFO:teuthology.orchestra.run.smithi029.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T00:42:32.713 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48573.log 2024-04-25T00:42:32.714 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60254.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60254.log.gz 2024-04-25T00:42:32.714 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45572.log 2024-04-25T00:42:32.715 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48573.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48573.log.gz 2024-04-25T00:42:32.715 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43553.log 2024-04-25T00:42:32.716 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45572.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45572.log.gz 2024-04-25T00:42:32.716 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50352.log 2024-04-25T00:42:32.717 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43553.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43553.log.gz 2024-04-25T00:42:32.717 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43445.log 2024-04-25T00:42:32.718 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50352.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50352.log.gz 2024-04-25T00:42:32.718 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46490.log 2024-04-25T00:42:32.719 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43445.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43445.log.gz 2024-04-25T00:42:32.719 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42309.log 2024-04-25T00:42:32.720 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46490.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46490.log.gz 2024-04-25T00:42:32.720 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56143.log 2024-04-25T00:42:32.721 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42309.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42309.log.gz 2024-04-25T00:42:32.721 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45598.log 2024-04-25T00:42:32.722 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56143.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56143.log.gz 2024-04-25T00:42:32.723 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58285.log 2024-04-25T00:42:32.723 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45598.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45598.log.gz 2024-04-25T00:42:32.724 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41377.log 2024-04-25T00:42:32.724 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58285.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58285.log.gz 2024-04-25T00:42:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57039.log 2024-04-25T00:42:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41377.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41377.log.gz 2024-04-25T00:42:32.726 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34433.log 2024-04-25T00:42:32.726 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57039.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57039.log.gz 2024-04-25T00:42:32.727 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36315.log 2024-04-25T00:42:32.728 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.34433.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.34433.log.gz 2024-04-25T00:42:32.728 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34483.log 2024-04-25T00:42:32.729 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36315.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36315.log.gz 2024-04-25T00:42:32.729 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45799.log 2024-04-25T00:42:32.730 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34483.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34483.log.gz 2024-04-25T00:42:32.731 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49187.log 2024-04-25T00:42:32.734 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45799.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45799.log.gz 2024-04-25T00:42:32.735 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43811.log 2024-04-25T00:42:32.735 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49187.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49187.log.gz 2024-04-25T00:42:32.735 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44864.log 2024-04-25T00:42:32.735 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43811.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43811.log.gz 2024-04-25T00:42:32.735 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57228.log 2024-04-25T00:42:32.736 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44864.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44864.log.gz 2024-04-25T00:42:32.736 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37598.log 2024-04-25T00:42:32.736 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57228.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57228.log.gz 2024-04-25T00:42:32.737 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37909.log 2024-04-25T00:42:32.744 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.37598.log: 92.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59683.log 2024-04-25T00:42:32.744 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37909.log: -- replaced with /var/log/ceph/ceph-client.0.37598.log.gz 2024-04-25T00:42:32.745 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37909.log.gz 2024-04-25T00:42:32.745 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34888.log 2024-04-25T00:42:32.746 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59683.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59683.log.gz 2024-04-25T00:42:32.746 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54938.log 2024-04-25T00:42:32.747 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34888.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34888.log.gz 2024-04-25T00:42:32.748 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48154.log 2024-04-25T00:42:32.749 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54938.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54938.log.gz 2024-04-25T00:42:32.749 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36874.log 2024-04-25T00:42:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48154.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48154.log.gz 2024-04-25T00:42:32.750 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57827.log 2024-04-25T00:42:32.751 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36874.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36874.log.gz 2024-04-25T00:42:32.751 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56448.log 2024-04-25T00:42:32.752 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57827.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57827.log.gz 2024-04-25T00:42:32.752 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44554.log 2024-04-25T00:42:32.753 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56448.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56448.log.gz 2024-04-25T00:42:32.754 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42392.log 2024-04-25T00:42:32.755 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44554.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44554.log.gz 2024-04-25T00:42:32.755 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42497.log 2024-04-25T00:42:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42392.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42392.log.gz 2024-04-25T00:42:32.756 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44610.log 2024-04-25T00:42:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42497.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42497.log.gz 2024-04-25T00:42:32.757 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50588.log 2024-04-25T00:42:32.758 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44610.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44610.log.gz 2024-04-25T00:42:32.759 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40638.log 2024-04-25T00:42:32.775 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.50588.log: 92.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35761.log 2024-04-25T00:42:32.776 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40638.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40638.log.gz 2024-04-25T00:42:32.776 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.0.50588.log.gz 2024-04-25T00:42:32.776 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54601.log 2024-04-25T00:42:32.777 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35761.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35761.log.gz 2024-04-25T00:42:32.777 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55883.log 2024-04-25T00:42:32.778 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54601.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54601.log.gz 2024-04-25T00:42:32.779 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59378.log 2024-04-25T00:42:32.780 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55883.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55883.log.gz 2024-04-25T00:42:32.780 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37204.log 2024-04-25T00:42:32.781 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59378.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59378.log.gz 2024-04-25T00:42:32.781 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57905.log 2024-04-25T00:42:32.782 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37204.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37204.log.gz 2024-04-25T00:42:32.783 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49507.log 2024-04-25T00:42:32.783 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57905.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57905.log.gz 2024-04-25T00:42:32.784 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42222.log 2024-04-25T00:42:32.785 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49507.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49507.log.gz 2024-04-25T00:42:32.785 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34914.log 2024-04-25T00:42:32.786 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42222.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42222.log.gz 2024-04-25T00:42:32.786 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33912.log 2024-04-25T00:42:32.787 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34914.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34914.log.gz 2024-04-25T00:42:32.788 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47651.log 2024-04-25T00:42:32.788 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33912.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33912.log.gz 2024-04-25T00:42:32.789 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55909.log 2024-04-25T00:42:32.790 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47651.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47651.log.gz 2024-04-25T00:42:32.790 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38070.log 2024-04-25T00:42:32.791 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55909.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55909.log.gz 2024-04-25T00:42:32.791 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36419.log 2024-04-25T00:42:32.792 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38070.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38070.log.gz 2024-04-25T00:42:32.792 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36185.log 2024-04-25T00:42:32.793 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36419.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36419.log.gz 2024-04-25T00:42:32.794 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53570.log 2024-04-25T00:42:32.795 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36185.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36185.log.gz 2024-04-25T00:42:32.795 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43041.log 2024-04-25T00:42:32.796 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53570.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53570.log.gz 2024-04-25T00:42:32.796 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53330.log 2024-04-25T00:42:32.797 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43041.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43041.log.gz 2024-04-25T00:42:32.797 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44476.log 2024-04-25T00:42:32.798 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53330.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53330.log.gz 2024-04-25T00:42:32.799 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37334.log 2024-04-25T00:42:32.800 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44476.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44476.log.gz 2024-04-25T00:42:32.800 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46994.log 2024-04-25T00:42:32.801 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37334.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37334.log.gz 2024-04-25T00:42:32.801 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49348.log 2024-04-25T00:42:32.802 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46994.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46994.log.gz 2024-04-25T00:42:32.802 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37087.log 2024-04-25T00:42:32.803 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49348.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49348.log.gz 2024-04-25T00:42:32.803 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37059.log 2024-04-25T00:42:32.804 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37087.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37087.log.gz 2024-04-25T00:42:32.805 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35070.log 2024-04-25T00:42:32.806 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37059.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37059.log.gz 2024-04-25T00:42:32.806 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50670.log 2024-04-25T00:42:32.807 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35070.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35070.log.gz 2024-04-25T00:42:32.807 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38229.log 2024-04-25T00:42:32.808 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50670.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50670.log.gz 2024-04-25T00:42:32.808 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52395.log 2024-04-25T00:42:32.809 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38229.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38229.log.gz 2024-04-25T00:42:32.810 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53062.log 2024-04-25T00:42:32.811 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52395.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52395.log.gz 2024-04-25T00:42:32.811 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55682.log 2024-04-25T00:42:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53062.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53062.log.gz 2024-04-25T00:42:32.812 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53382.log 2024-04-25T00:42:32.813 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55682.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55682.log.gz 2024-04-25T00:42:32.813 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44502.log 2024-04-25T00:42:32.814 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53382.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53382.log.gz 2024-04-25T00:42:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57311.log 2024-04-25T00:42:32.815 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44502.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44502.log.gz 2024-04-25T00:42:32.816 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42471.log 2024-04-25T00:42:32.817 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57311.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57311.log.gz 2024-04-25T00:42:32.817 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41631.log 2024-04-25T00:42:32.818 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42471.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42471.log.gz 2024-04-25T00:42:32.818 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42857.log 2024-04-25T00:42:32.819 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41631.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41631.log.gz 2024-04-25T00:42:32.819 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35390.log 2024-04-25T00:42:32.820 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42857.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42857.log.gz 2024-04-25T00:42:32.821 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47306.log 2024-04-25T00:42:32.822 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35390.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35390.log.gz 2024-04-25T00:42:32.822 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41088.log 2024-04-25T00:42:32.823 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47306.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47306.log.gz 2024-04-25T00:42:32.823 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33257.log 2024-04-25T00:42:32.824 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41088.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41088.log.gz 2024-04-25T00:42:32.824 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34680.log 2024-04-25T00:42:32.825 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33257.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33257.log.gz 2024-04-25T00:42:32.826 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51775.log 2024-04-25T00:42:32.826 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34680.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34680.log.gz 2024-04-25T00:42:32.827 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38122.log 2024-04-25T00:42:32.828 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51775.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51775.log.gz 2024-04-25T00:42:32.828 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47599.log 2024-04-25T00:42:32.829 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38122.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38122.log.gz 2024-04-25T00:42:32.829 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43709.log 2024-04-25T00:42:32.830 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47599.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47599.log.gz 2024-04-25T00:42:32.830 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53099.log 2024-04-25T00:42:32.831 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43709.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43709.log.gz 2024-04-25T00:42:32.832 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53625.log 2024-04-25T00:42:32.833 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53099.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53099.log.gz 2024-04-25T00:42:32.833 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40835.log 2024-04-25T00:42:32.834 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53625.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53625.log.gz 2024-04-25T00:42:32.834 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45654.log 2024-04-25T00:42:32.835 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40835.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40835.log.gz 2024-04-25T00:42:32.835 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49161.log 2024-04-25T00:42:32.836 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45654.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45654.log.gz 2024-04-25T00:42:32.837 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36523.log 2024-04-25T00:42:32.837 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49161.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49161.log.gz 2024-04-25T00:42:32.838 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35416.log 2024-04-25T00:42:32.839 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36523.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36523.log.gz 2024-04-25T00:42:32.839 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40074.log 2024-04-25T00:42:32.840 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35416.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35416.log.gz 2024-04-25T00:42:32.840 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38203.log 2024-04-25T00:42:32.841 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40074.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40074.log.gz 2024-04-25T00:42:32.842 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38177.log 2024-04-25T00:42:32.843 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38203.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38203.log.gz 2024-04-25T00:42:32.843 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56278.log 2024-04-25T00:42:32.844 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38177.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38177.log.gz 2024-04-25T00:42:32.844 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47942.log 2024-04-25T00:42:32.845 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56278.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56278.log.gz 2024-04-25T00:42:32.845 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48182.log 2024-04-25T00:42:32.846 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47942.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47942.log.gz 2024-04-25T00:42:32.847 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56422.log 2024-04-25T00:42:32.848 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48182.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48182.log.gz 2024-04-25T00:42:32.848 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52010.log 2024-04-25T00:42:32.849 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56422.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56422.log.gz 2024-04-25T00:42:32.849 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41837.log 2024-04-25T00:42:32.850 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52010.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52010.log.gz 2024-04-25T00:42:32.850 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33680.log 2024-04-25T00:42:32.851 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41837.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41837.log.gz 2024-04-25T00:42:32.852 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50182.log 2024-04-25T00:42:32.852 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33680.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33680.log.gz 2024-04-25T00:42:32.853 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50933.log 2024-04-25T00:42:32.854 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50182.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50182.log.gz 2024-04-25T00:42:32.854 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49481.log 2024-04-25T00:42:32.855 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50933.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50933.log.gz 2024-04-25T00:42:32.855 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43302.log 2024-04-25T00:42:32.856 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49481.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49481.log.gz 2024-04-25T00:42:32.857 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48889.log 2024-04-25T00:42:32.857 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43302.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43302.log.gz 2024-04-25T00:42:32.858 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43527.log 2024-04-25T00:42:32.859 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48889.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48889.log.gz 2024-04-25T00:42:32.859 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36601.log 2024-04-25T00:42:32.860 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43527.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43527.log.gz 2024-04-25T00:42:32.860 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46271.log 2024-04-25T00:42:32.861 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36601.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36601.log.gz 2024-04-25T00:42:32.862 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59084.log 2024-04-25T00:42:32.862 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46271.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46271.log.gz 2024-04-25T00:42:32.863 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45545.log 2024-04-25T00:42:32.864 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59084.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59084.log.gz 2024-04-25T00:42:32.864 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44130.log 2024-04-25T00:42:32.865 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45545.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45545.log.gz 2024-04-25T00:42:32.865 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38847.log 2024-04-25T00:42:32.866 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44130.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44130.log.gz 2024-04-25T00:42:32.867 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55831.log 2024-04-25T00:42:32.867 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38847.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38847.log.gz 2024-04-25T00:42:32.868 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58259.log 2024-04-25T00:42:32.869 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55831.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55831.log.gz 2024-04-25T00:42:32.869 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58315.log 2024-04-25T00:42:32.870 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58259.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58259.log.gz 2024-04-25T00:42:32.870 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41260.log 2024-04-25T00:42:32.871 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58315.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58315.log.gz 2024-04-25T00:42:32.872 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32748.log 2024-04-25T00:42:32.873 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41260.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41260.log.gz 2024-04-25T00:42:32.873 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36445.log 2024-04-25T00:42:32.874 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32748.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32748.log.gz 2024-04-25T00:42:32.874 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48915.log 2024-04-25T00:42:32.875 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36445.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36445.log.gz 2024-04-25T00:42:32.875 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46914.log 2024-04-25T00:42:32.876 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48915.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48915.log.gz 2024-04-25T00:42:32.877 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39334.log 2024-04-25T00:42:32.877 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46914.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46914.log.gz 2024-04-25T00:42:32.878 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35946.log 2024-04-25T00:42:32.879 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39334.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39334.log.gz 2024-04-25T00:42:32.879 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36704.log 2024-04-25T00:42:32.880 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35946.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35946.log.gz 2024-04-25T00:42:32.880 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51749.log 2024-04-25T00:42:32.881 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36704.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36704.log.gz 2024-04-25T00:42:32.881 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49585.log 2024-04-25T00:42:32.882 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51749.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51749.log.gz 2024-04-25T00:42:32.883 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33052.log 2024-04-25T00:42:32.884 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49585.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49585.log.gz 2024-04-25T00:42:32.884 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55009.log 2024-04-25T00:42:32.885 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33052.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33052.log.gz 2024-04-25T00:42:32.885 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55474.log 2024-04-25T00:42:32.886 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55009.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55009.log.gz 2024-04-25T00:42:32.886 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36978.log 2024-04-25T00:42:32.887 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55474.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55474.log.gz 2024-04-25T00:42:32.888 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37282.log 2024-04-25T00:42:32.889 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36978.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36978.log.gz 2024-04-25T00:42:32.889 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32804.log 2024-04-25T00:42:32.890 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37282.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37282.log.gz 2024-04-25T00:42:32.890 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58341.log 2024-04-25T00:42:32.891 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32804.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32804.log.gz 2024-04-25T00:42:32.891 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49735.log 2024-04-25T00:42:32.892 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58341.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58341.log.gz 2024-04-25T00:42:32.893 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44261.log 2024-04-25T00:42:32.893 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49735.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49735.log.gz 2024-04-25T00:42:32.894 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47968.log 2024-04-25T00:42:32.895 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44261.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44261.log.gz 2024-04-25T00:42:32.895 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37822.log 2024-04-25T00:42:32.896 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47968.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.47968.log.gz 2024-04-25T00:42:32.897 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41114.log 2024-04-25T00:42:32.897 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37822.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37822.log.gz 2024-04-25T00:42:32.898 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47863.log 2024-04-25T00:42:32.899 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41114.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41114.log.gz 2024-04-25T00:42:32.899 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56250.log 2024-04-25T00:42:32.900 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47863.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47863.log.gz 2024-04-25T00:42:32.900 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41062.log 2024-04-25T00:42:32.901 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56250.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56250.log.gz 2024-04-25T00:42:32.901 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33679.log 2024-04-25T00:42:32.902 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41062.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41062.log.gz 2024-04-25T00:42:32.903 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35893.log 2024-04-25T00:42:32.904 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33679.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33679.log.gz 2024-04-25T00:42:32.904 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41603.log 2024-04-25T00:42:32.905 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35893.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35893.log.gz 2024-04-25T00:42:32.905 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37961.log 2024-04-25T00:42:32.906 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41603.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41603.log.gz 2024-04-25T00:42:32.906 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56896.log 2024-04-25T00:42:32.907 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37961.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37961.log.gz 2024-04-25T00:42:32.908 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46677.log 2024-04-25T00:42:32.908 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56896.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56896.log.gz 2024-04-25T00:42:32.909 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38359.log 2024-04-25T00:42:32.910 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46677.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46677.log.gz 2024-04-25T00:42:32.910 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44584.log 2024-04-25T00:42:32.911 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38359.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38359.log.gz 2024-04-25T00:42:32.911 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33130.log 2024-04-25T00:42:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44584.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44584.log.gz 2024-04-25T00:42:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43999.log 2024-04-25T00:42:32.913 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33130.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33130.log.gz 2024-04-25T00:42:32.914 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42144.log 2024-04-25T00:42:32.915 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43999.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43999.log.gz 2024-04-25T00:42:32.915 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40515.log 2024-04-25T00:42:32.916 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.42144.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42144.log.gz 2024-04-25T00:42:32.916 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51944.log 2024-04-25T00:42:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40515.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40515.log.gz 2024-04-25T00:42:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44346.log 2024-04-25T00:42:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51944.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51944.log.gz 2024-04-25T00:42:32.919 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49296.log 2024-04-25T00:42:32.920 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44346.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44346.log.gz 2024-04-25T00:42:32.920 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55552.log 2024-04-25T00:42:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49296.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49296.log.gz 2024-04-25T00:42:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59760.log 2024-04-25T00:42:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55552.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55552.log.gz 2024-04-25T00:42:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44294.log 2024-04-25T00:42:32.923 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59760.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59760.log.gz 2024-04-25T00:42:32.923 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41166.log 2024-04-25T00:42:32.924 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.44294.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44294.log.gz 2024-04-25T00:42:32.925 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45929.log 2024-04-25T00:42:32.926 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.41166.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41166.log.gz 2024-04-25T00:42:32.926 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58419.log 2024-04-25T00:42:32.927 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45929.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45929.log.gz 2024-04-25T00:42:32.927 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39230.log 2024-04-25T00:42:32.928 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58419.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58419.log.gz 2024-04-25T00:42:32.928 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52114.log 2024-04-25T00:42:32.929 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39230.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39230.log.gz 2024-04-25T00:42:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57775.log 2024-04-25T00:42:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52114.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52114.log.gz 2024-04-25T00:42:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36055.log 2024-04-25T00:42:32.932 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57775.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57775.log.gz 2024-04-25T00:42:32.932 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39546.log 2024-04-25T00:42:32.933 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36055.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36055.log.gz 2024-04-25T00:42:32.933 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47456.log 2024-04-25T00:42:32.937 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39546.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39546.log.gz 2024-04-25T00:42:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34146.log 2024-04-25T00:42:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47456.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47456.log.gz 2024-04-25T00:42:32.939 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56922.log 2024-04-25T00:42:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34146.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34146.log.gz 2024-04-25T00:42:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35553.log 2024-04-25T00:42:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56922.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56922.log.gz 2024-04-25T00:42:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59838.log 2024-04-25T00:42:32.940 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35553.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35553.log.gz 2024-04-25T00:42:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35867.log 2024-04-25T00:42:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59838.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.59838.log.gz 2024-04-25T00:42:32.941 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45363.log 2024-04-25T00:42:32.942 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35867.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35867.log.gz 2024-04-25T00:42:32.942 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38411.log 2024-04-25T00:42:32.943 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.45363.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.45363.log.gz 2024-04-25T00:42:32.943 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59534.log 2024-04-25T00:42:32.944 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.38411.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38411.log.gz 2024-04-25T00:42:32.944 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46245.log 2024-04-25T00:42:32.945 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59534.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59534.log.gz 2024-04-25T00:42:32.946 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59890.log 2024-04-25T00:42:32.947 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.46245.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46245.log.gz 2024-04-25T00:42:32.947 INFO:teuthology.orchestra.run.smithi157.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T00:42:32.947 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55935.log 2024-04-25T00:42:32.948 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59890.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59890.log.gz 2024-04-25T00:42:32.948 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55708.log 2024-04-25T00:42:32.949 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55935.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55935.log.gz 2024-04-25T00:42:32.949 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60280.log 2024-04-25T00:42:32.950 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55708.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55708.log.gz 2024-04-25T00:42:32.951 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54297.log 2024-04-25T00:42:32.952 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60280.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60280.log.gz 2024-04-25T00:42:32.952 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59994.log 2024-04-25T00:42:32.953 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54297.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54297.log.gz 2024-04-25T00:42:32.953 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35657.log 2024-04-25T00:42:32.954 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59994.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59994.log.gz 2024-04-25T00:42:32.954 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56656.log 2024-04-25T00:42:32.955 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35657.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35657.log.gz 2024-04-25T00:42:32.956 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60072.log 2024-04-25T00:42:32.957 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56656.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56656.log.gz 2024-04-25T00:42:32.957 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59032.log 2024-04-25T00:42:32.957 INFO:teuthology.orchestra.run.smithi157.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T00:42:32.958 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60072.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60072.log.gz 2024-04-25T00:42:32.958 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33938.log 2024-04-25T00:42:32.959 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59032.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59032.log.gz 2024-04-25T00:42:32.959 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53703.log 2024-04-25T00:42:32.960 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33938.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33938.log.gz 2024-04-25T00:42:32.961 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58763.log 2024-04-25T00:42:32.962 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53703.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53703.log.gz 2024-04-25T00:42:32.962 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49637.log 2024-04-25T00:42:32.963 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58763.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58763.log.gz 2024-04-25T00:42:32.963 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47890.log 2024-04-25T00:42:32.964 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49637.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49637.log.gz 2024-04-25T00:42:32.965 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27823.log 2024-04-25T00:42:32.965 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47890.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47890.log.gz 2024-04-25T00:42:32.966 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55526.log 2024-04-25T00:42:32.967 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27823.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27823.log.gz 2024-04-25T00:42:32.967 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56578.log 2024-04-25T00:42:32.968 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55526.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55526.log.gz 2024-04-25T00:42:32.968 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55284.log 2024-04-25T00:42:32.969 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56578.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56578.log.gz 2024-04-25T00:42:32.969 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51063.log 2024-04-25T00:42:32.970 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55284.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55284.log.gz 2024-04-25T00:42:32.971 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35527.log 2024-04-25T00:42:32.972 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51063.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51063.log.gz 2024-04-25T00:42:32.972 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52812.log 2024-04-25T00:42:32.973 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35527.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35527.log.gz 2024-04-25T00:42:32.973 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32948.log 2024-04-25T00:42:32.974 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52812.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.52812.log.gz 2024-04-25T00:42:32.974 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37414.log 2024-04-25T00:42:32.975 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32948.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32948.log.gz 2024-04-25T00:42:32.976 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58524.log 2024-04-25T00:42:32.977 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.37414.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37414.log.gz 2024-04-25T00:42:32.977 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35972.log 2024-04-25T00:42:32.978 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58524.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58524.log.gz 2024-04-25T00:42:32.978 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52447.log 2024-04-25T00:42:32.979 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35972.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35972.log.gz 2024-04-25T00:42:32.980 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40393.log 2024-04-25T00:42:32.980 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52447.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52447.log.gz 2024-04-25T00:42:32.981 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43074.log 2024-04-25T00:42:32.982 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.40393.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40393.log.gz 2024-04-25T00:42:32.982 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53149.log 2024-04-25T00:42:32.983 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43074.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43074.log.gz 2024-04-25T00:42:32.983 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39044.log 2024-04-25T00:42:32.984 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53149.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.53149.log.gz 2024-04-25T00:42:32.984 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56169.log 2024-04-25T00:42:32.985 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.39044.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39044.log.gz 2024-04-25T00:42:32.986 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36263.log 2024-04-25T00:42:32.987 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56169.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56169.log.gz 2024-04-25T00:42:32.987 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59631.log 2024-04-25T00:42:32.988 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36263.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36263.log.gz 2024-04-25T00:42:32.988 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36646.log 2024-04-25T00:42:32.989 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59631.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59631.log.gz 2024-04-25T00:42:32.990 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.36646.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36646.log.gz 2024-04-25T00:42:33.016 INFO:teuthology.orchestra.run.smithi157.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T00:42:33.036 INFO:teuthology.orchestra.run.smithi157.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T00:42:33.038 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-04-25T00:42:33.038 INFO:teuthology.orchestra.run.smithi157.stderr:real 0m0.819s 2024-04-25T00:42:33.038 INFO:teuthology.orchestra.run.smithi157.stderr:user 0m3.732s 2024-04-25T00:42:33.038 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m0.177s 2024-04-25T00:42:33.127 INFO:teuthology.orchestra.run.smithi029.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T00:42:33.256 INFO:teuthology.orchestra.run.smithi029.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T00:42:33.312 INFO:teuthology.orchestra.run.smithi029.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T00:42:33.347 INFO:teuthology.orchestra.run.smithi029.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T00:42:33.407 INFO:teuthology.orchestra.run.smithi029.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T00:42:33.427 INFO:teuthology.orchestra.run.smithi029.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T00:42:33.428 INFO:teuthology.orchestra.run.smithi029.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T00:42:33.429 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-25T00:42:33.429 INFO:teuthology.orchestra.run.smithi029.stderr:real 0m1.255s 2024-04-25T00:42:33.429 INFO:teuthology.orchestra.run.smithi029.stderr:user 0m7.747s 2024-04-25T00:42:33.429 INFO:teuthology.orchestra.run.smithi029.stderr:sys 0m0.526s 2024-04-25T00:42:33.430 INFO:tasks.ceph:Archiving logs... 2024-04-25T00:42:33.430 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260/remote/smithi029/log 2024-04-25T00:42:33.431 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:42:34.884 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260/remote/smithi157/log 2024-04-25T00:42:34.884 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:42:35.310 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T00:42:35.320 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T00:42:35.320 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:42:35.322 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:42:35.395 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T00:42:35.407 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T00:42:35.407 DEBUG:teuthology.orchestra.run.smithi029:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:42:35.410 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:42:35.422 INFO:teuthology.orchestra.run.smithi029.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:42:35.422 INFO:teuthology.orchestra.run.smithi029.stdout:============================================================================== 2024-04-25T00:42:35.422 INFO:teuthology.orchestra.run.smithi029.stdout:+hv01.front.sepi 44.4.53.2 2 u 51 64 377 0.069 -3.392 1.243 2024-04-25T00:42:35.422 INFO:teuthology.orchestra.run.smithi029.stdout:*hv02.front.sepi 63.231.80.2 3 u 51 64 377 0.067 -1.876 0.882 2024-04-25T00:42:35.422 INFO:teuthology.orchestra.run.smithi029.stdout:+hv03.front.sepi 74.6.168.72 3 u 47 64 377 0.091 -0.729 1.203 2024-04-25T00:42:35.422 INFO:teuthology.orchestra.run.smithi029.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-04-25T00:42:35.423 INFO:teuthology.orchestra.run.smithi157.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:42:35.424 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================== 2024-04-25T00:42:35.424 INFO:teuthology.orchestra.run.smithi157.stdout:*hv01.front.sepi 44.4.53.2 2 u 54 64 377 0.056 -3.099 1.074 2024-04-25T00:42:35.424 INFO:teuthology.orchestra.run.smithi157.stdout:+hv02.front.sepi 63.231.80.2 3 u 52 64 377 0.084 -1.217 1.137 2024-04-25T00:42:35.424 INFO:teuthology.orchestra.run.smithi157.stdout:+hv03.front.sepi 74.6.168.72 3 u 57 64 377 0.086 -2.626 1.301 2024-04-25T00:42:35.424 INFO:teuthology.orchestra.run.smithi157.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-04-25T00:42:35.424 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T00:42:35.433 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T00:42:35.433 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T00:42:35.442 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T00:42:35.450 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T00:42:35.460 INFO:teuthology.task.internal:Duration was 1358.027258 seconds 2024-04-25T00:42:35.460 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T00:42:35.471 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T00:42:35.471 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:42:35.474 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:42:35.510 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T00:42:35.511 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi029.front.sepia.ceph.com 2024-04-25T00:42:35.511 DEBUG:teuthology.orchestra.run.smithi029:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:42:35.564 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-04-25T00:42:35.564 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:42:35.574 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T00:42:35.575 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:42:35.608 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:42:35.642 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T00:42:35.643 DEBUG:teuthology.orchestra.run.smithi029:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:42:35.684 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:42:35.770 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T00:42:35.783 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T00:42:35.783 DEBUG:teuthology.orchestra.run.smithi029:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:42:35.798 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:42:35.811 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T00:42:35.830 DEBUG:teuthology.orchestra.run.smithi029:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:42:35.840 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:42:35.849 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = core 2024-04-25T00:42:35.863 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-04-25T00:42:35.877 DEBUG:teuthology.orchestra.run.smithi029:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:42:35.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:42:35.907 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:42:35.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:42:35.925 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T00:42:35.936 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T00:42:35.936 DEBUG:teuthology.misc:Transferring archived files from smithi029:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260/remote/smithi029 2024-04-25T00:42:35.937 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:42:35.973 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672260/remote/smithi157 2024-04-25T00:42:35.973 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:42:35.995 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T00:42:35.995 DEBUG:teuthology.orchestra.run.smithi029:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:42:36.016 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:42:36.038 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T00:42:36.049 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T00:42:36.049 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T00:42:36.060 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T00:42:36.060 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:42:36.064 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:42:36.067 INFO:teuthology.orchestra.run.smithi029.stdout: 658471 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:42 /home/ubuntu/cephtest 2024-04-25T00:42:36.086 INFO:teuthology.orchestra.run.smithi157.stdout: 658641 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:42 /home/ubuntu/cephtest 2024-04-25T00:42:36.087 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T00:42:36.098 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T00:42:36.121 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} duration: 1358.027257680893 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T00:42:36.121 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:42:36.201 INFO:teuthology.run:pass